2023-10-08T05:19:29.595 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-08T05:19:29.595 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --verbose -- /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325/orig.config.yaml 2023-10-08T05:19:29.641 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T05:19:29.726 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: yweinste@redhat.com first_in_suite: false job_id: '7417325' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 9725b08fd801c82987c737f7e6d354878f036ee8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9725b08fd801c82987c737f7e6d354878f036ee8 kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 9725b08fd801c82987c737f7e6d354878f036ee8 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9589 sha1: 9725b08fd801c82987c737f7e6d354878f036ee8 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_9725b08fd801c82987c737f7e6d354878f036ee8/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 9725b08fd801c82987c737f7e6d354878f036ee8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyT4shKd9O1F/X50hR7TtBU+d+CcOtra9ZcJ6f/8QsjtDrQEJiQ3oK0WUgaGiNfxC9mA0tjeusS9MGpaHtVKHQ= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwinXEppRvqq99rlroBeSrntliLaKSVHRBP/v0OZZD7HQAcQGpINfIV8Wgu2P4/XcSMUsBD46x/YqwhJvKNuFI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-07_14:05:27 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-08T05:19:29.727 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_9725b08fd801c82987c737f7e6d354878f036ee8/qa; will attempt to use it 2023-10-08T05:19:29.728 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_9725b08fd801c82987c737f7e6d354878f036ee8/qa/tasks 2023-10-08T05:19:29.728 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-08T05:19:29.730 INFO:teuthology.task.internal:Checking packages... 2023-10-08T05:19:29.751 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9725b08fd801c82987c737f7e6d354878f036ee8' 2023-10-08T05:19:29.751 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-08T05:19:29.751 INFO:teuthology.packaging:ref: None 2023-10-08T05:19:29.751 INFO:teuthology.packaging:tag: None 2023-10-08T05:19:29.751 INFO:teuthology.packaging:branch: main 2023-10-08T05:19:29.751 INFO:teuthology.packaging:sha1: 9725b08fd801c82987c737f7e6d354878f036ee8 2023-10-08T05:19:29.752 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2023-10-08T05:19:30.085 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6613.gb05d167b 2023-10-08T05:19:30.085 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-08T05:19:30.093 INFO:teuthology.task.internal:no buildpackages task found 2023-10-08T05:19:30.093 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-08T05:19:30.139 INFO:teuthology.task.internal:Saving configuration 2023-10-08T05:19:30.156 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-08T05:19:30.173 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T05:19:30.199 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 05:14:30.401820', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyT4shKd9O1F/X50hR7TtBU+d+CcOtra9ZcJ6f/8QsjtDrQEJiQ3oK0WUgaGiNfxC9mA0tjeusS9MGpaHtVKHQ='} 2023-10-08T05:19:30.221 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 05:14:30.403596', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwinXEppRvqq99rlroBeSrntliLaKSVHRBP/v0OZZD7HQAcQGpINfIV8Wgu2P4/XcSMUsBD46x/YqwhJvKNuFI='} 2023-10-08T05:19:30.221 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-08T05:19:30.228 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-08T05:19:30.228 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-08T05:19:30.228 INFO:teuthology.run_tasks:Running task console_log... 2023-10-08T05:19:30.324 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5ac4175040>, signals=[15]) 2023-10-08T05:19:30.325 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-08T05:19:30.331 INFO:teuthology.task.internal:Opening connections... 2023-10-08T05:19:30.331 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:19:30.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T05:19:30.419 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T05:19:30.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T05:19:30.505 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-08T05:19:30.511 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2023-10-08T05:19:30.528 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2023-10-08T05:19:30.529 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2023-10-08T05:19:30.588 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2023-10-08T05:19:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2023-10-08T05:19:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2023-10-08T05:19:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2023-10-08T05:19:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2023-10-08T05:19:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2023-10-08T05:19:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-08T05:19:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2023-10-08T05:19:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2023-10-08T05:19:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-08T05:19:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2023-10-08T05:19:30.590 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T05:19:30.590 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-08T05:19:30.590 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-08T05:19:30.590 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2023-10-08T05:19:30.625 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2023-10-08T05:19:30.642 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2023-10-08T05:19:30.643 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="CentOS Stream" 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="9" 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:ID="centos" 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE="rhel fedora" 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="9" 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:PLATFORM_ID="platform:el9" 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="CentOS Stream 9" 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:ANSI_COLOR="0;31" 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:LOGO="fedora-logo-icon" 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-10-08T05:19:30.699 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://centos.org/" 2023-10-08T05:19:30.700 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T05:19:30.700 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-10-08T05:19:30.700 INFO:teuthology.orchestra.run.smithi181.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-08T05:19:30.700 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2023-10-08T05:19:30.725 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-08T05:19:30.734 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-08T05:19:30.783 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-08T05:19:30.784 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2023-10-08T05:19:30.786 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2023-10-08T05:19:30.803 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-08T05:19:30.814 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-08T05:19:30.814 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2023-10-08T05:19:30.846 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2023-10-08T05:19:31.081 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-08T05:19:31.136 INFO:teuthology.run_tasks:Running task kernel... 2023-10-08T05:19:31.153 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-08T05:19:31.153 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-08T05:19:31.153 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-08T05:19:31.153 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T05:19:31.154 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T05:19:31.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T05:19:31.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-10-08T05:19:31.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T05:19:31.175 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-10-08T05:19:31.324 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T05:19:31.324 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-08T05:19:31.324 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2023-10-08T05:19:31.341 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-361.el9.x86_64 2023-10-08T05:19:31.342 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T05:19:31.342 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-08T05:19:31.342 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum upgrade -y linux-firmware 2023-10-08T05:19:31.392 DEBUG:teuthology.task.kernel:sha1 for client.1 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T05:19:31.392 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-08T05:19:31.392 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2023-10-08T05:19:31.409 INFO:teuthology.orchestra.run.smithi181.stdout:5.14.0-361.el9.x86_64 2023-10-08T05:19:31.409 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T05:19:31.409 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-08T05:19:31.409 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum upgrade -y linux-firmware 2023-10-08T05:19:33.403 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - AppStream 15 MB/s | 6.3 MB 00:00 2023-10-08T05:19:33.436 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-9 - AppStream 15 MB/s | 6.3 MB 00:00 2023-10-08T05:19:34.745 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2023-10-08T05:19:34.969 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2023-10-08T05:19:35.944 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 7.9 MB 00:00 2023-10-08T05:19:36.784 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 7.9 MB 00:01 2023-10-08T05:19:37.687 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 23 MB/s | 18 MB 00:00 2023-10-08T05:19:42.554 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 5.7 MB 00:01 2023-10-08T05:19:42.959 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 3.4 MB/s | 18 MB 00:05 2023-10-08T05:19:44.022 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 14 kB 00:00 2023-10-08T05:19:44.914 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 24 MB/s | 19 MB 00:00 2023-10-08T05:19:47.776 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 5.7 MB 00:01 2023-10-08T05:19:49.406 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2023-10-08T05:19:49.878 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 14 kB 00:01 2023-10-08T05:19:51.535 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-10-08T05:19:51.548 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-10-08T05:19:51.549 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-10-08T05:19:51.549 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-10-08T05:19:51.549 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2023-10-08T05:19:51.549 INFO:teuthology.orchestra.run.smithi181.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-08T05:19:51.549 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:19:51.549 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-10-08T05:19:51.549 INFO:teuthology.orchestra.run.smithi181.stdout:================================================================================ 2023-10-08T05:19:51.549 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 1 Package 2023-10-08T05:19:51.550 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:19:51.550 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 326 M 2023-10-08T05:19:51.550 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-10-08T05:19:53.128 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 19 MB 00:03 2023-10-08T05:19:57.527 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2023-10-08T05:19:58.882 INFO:teuthology.orchestra.run.smithi181.stdout:linux-firmware-20230926-140.el9.noarch.rpm 44 MB/s | 326 MB 00:07 2023-10-08T05:19:58.883 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-10-08T05:19:58.883 INFO:teuthology.orchestra.run.smithi181.stdout:Total 44 MB/s | 326 MB 00:07 2023-10-08T05:19:58.884 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-10-08T05:19:58.970 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-10-08T05:19:58.970 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-10-08T05:19:59.624 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-10-08T05:19:59.637 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-10-08T05:19:59.637 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2023-10-08T05:19:59.637 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-10-08T05:19:59.638 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2023-10-08T05:19:59.638 INFO:teuthology.orchestra.run.smithi070.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-10-08T05:19:59.638 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:19:59.638 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2023-10-08T05:19:59.638 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-10-08T05:19:59.638 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 1 Package 2023-10-08T05:19:59.638 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:19:59.638 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 326 M 2023-10-08T05:19:59.638 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2023-10-08T05:20:01.123 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-10-08T05:20:01.123 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-10-08T05:20:06.879 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-10-08T05:20:07.273 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-08T05:20:07.353 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-08T05:20:08.044 INFO:teuthology.orchestra.run.smithi070.stdout:linux-firmware-20230926-140.el9.noarch.rpm 39 MB/s | 326 MB 00:08 2023-10-08T05:20:08.045 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2023-10-08T05:20:08.045 INFO:teuthology.orchestra.run.smithi070.stdout:Total 39 MB/s | 326 MB 00:08 2023-10-08T05:20:08.046 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2023-10-08T05:20:08.138 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2023-10-08T05:20:08.139 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2023-10-08T05:20:08.327 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-08T05:20:08.327 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-08T05:20:08.696 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-08T05:20:08.696 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:20:08.696 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2023-10-08T05:20:08.696 INFO:teuthology.orchestra.run.smithi181.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-08T05:20:08.696 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:20:08.697 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-10-08T05:20:08.991 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-08T05:20:08.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T05:20:09.139 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ 2023-10-08T05:20:09.140 DEBUG:teuthology.orchestra.run.smithi181:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-08T05:20:09.143 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi181.front.sepia.ceph.com to complete... 2023-10-08T05:20:10.289 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2023-10-08T05:20:10.289 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2023-10-08T05:20:15.998 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2023-10-08T05:20:16.467 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-08T05:20:16.549 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-08T05:20:17.524 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-10-08T05:20:17.524 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-10-08T05:20:17.891 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-10-08T05:20:17.891 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:20:17.891 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2023-10-08T05:20:17.891 INFO:teuthology.orchestra.run.smithi070.stdout: linux-firmware-20230926-140.el9.noarch 2023-10-08T05:20:17.891 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:20:17.891 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-10-08T05:20:18.198 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-08T05:20:18.198 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T05:20:18.337 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ 2023-10-08T05:20:18.338 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ --input-file=- 2023-10-08T05:20:18.340 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi070.front.sepia.ceph.com to complete... 2023-10-08T05:20:25.027 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08 05:20:25 URL:https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [318991812/318991812] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-08T05:20:25.027 INFO:teuthology.orchestra.run.smithi181.stderr:FINISHED --2023-10-08 05:20:25-- 2023-10-08T05:20:25.027 INFO:teuthology.orchestra.run.smithi181.stderr:Total wall clock time: 16s 2023-10-08T05:20:25.028 INFO:teuthology.orchestra.run.smithi181.stderr:Downloaded: 1 files, 304M in 16s (19.5 MB/s) 2023-10-08T05:20:25.029 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-08T05:20:25.029 DEBUG:teuthology.orchestra.run.smithi181:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-08T05:20:26.868 INFO:teuthology.orchestra.run.smithi181.stdout:Verifying... ######################################## 2023-10-08T05:20:26.920 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing... ######################################## 2023-10-08T05:20:26.999 INFO:teuthology.orchestra.run.smithi181.stdout:Updating / installing... 2023-10-08T05:20:33.998 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-08T05:20:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08 05:20:45 URL:https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [318991812/318991812] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-08T05:20:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:FINISHED --2023-10-08 05:20:45-- 2023-10-08T05:20:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:Total wall clock time: 27s 2023-10-08T05:20:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:Downloaded: 1 files, 304M in 26s (11.5 MB/s) 2023-10-08T05:20:45.284 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-08T05:20:45.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-08T05:20:47.112 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying... ######################################## 2023-10-08T05:20:47.161 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing... ######################################## 2023-10-08T05:20:47.240 INFO:teuthology.orchestra.run.smithi070.stdout:Updating / installing... 2023-10-08T05:20:54.238 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-08T05:21:10.131 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi181.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-08T05:21:10.131 DEBUG:teuthology.orchestra.run.smithi181:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-08T05:21:10.191 INFO:teuthology.orchestra.run.smithi181.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:10.191 INFO:teuthology.orchestra.run.smithi181.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:10.192 INFO:teuthology.orchestra.run.smithi181.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:10.192 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:10.192 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-08T05:21:10.192 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-08T05:21:10.192 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-08T05:21:10.192 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-08T05:21:10.192 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-08T05:21:10.192 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-08T05:21:10.192 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-08T05:21:10.193 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-08T05:21:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-08T05:21:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-08T05:21:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-08T05:21:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-08T05:21:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-08T05:21:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-08T05:21:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-08T05:21:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-08T05:21:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-08T05:21:10.194 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-08T05:21:10.195 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-08T05:21:10.196 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-08T05:21:10.197 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-08T05:21:10.198 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-08T05:21:10.199 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-08T05:21:10.200 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-08T05:21:10.200 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-08T05:21:10.200 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-08T05:21:10.200 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-08T05:21:10.200 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-08T05:21:10.200 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-08T05:21:10.200 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-08T05:21:10.200 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-08T05:21:10.200 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-08T05:21:10.200 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-08T05:21:10.201 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-08T05:21:10.202 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-08T05:21:10.202 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-08T05:21:10.202 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-08T05:21:10.202 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-08T05:21:10.202 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-08T05:21:10.202 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-08T05:21:10.202 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-08T05:21:10.202 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-08T05:21:10.202 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-08T05:21:10.202 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-08T05:21:10.203 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-08T05:21:10.204 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-08T05:21:10.205 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-08T05:21:10.206 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-08T05:21:10.206 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-08T05:21:10.206 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-08T05:21:10.206 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-08T05:21:10.206 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-08T05:21:10.206 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-08T05:21:10.206 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-08T05:21:10.206 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-08T05:21:10.206 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-08T05:21:10.206 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-08T05:21:10.207 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-08T05:21:10.208 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-08T05:21:10.208 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-08T05:21:10.208 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-08T05:21:10.208 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-08T05:21:10.208 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-08T05:21:10.208 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-08T05:21:10.208 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-08T05:21:10.208 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-08T05:21:10.208 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-08T05:21:10.208 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-08T05:21:10.209 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-08T05:21:10.209 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-08T05:21:10.209 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-08T05:21:10.209 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-08T05:21:10.209 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-08T05:21:10.209 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-08T05:21:10.209 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-08T05:21:10.209 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-08T05:21:10.209 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-08T05:21:10.209 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-08T05:21:10.210 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-08T05:21:10.210 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-08T05:21:10.210 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-08T05:21:10.210 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-08T05:21:10.210 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-08T05:21:10.210 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-08T05:21:10.210 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-08T05:21:10.210 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-08T05:21:10.210 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-08T05:21:10.210 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-08T05:21:10.211 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-08T05:21:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-08T05:21:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-08T05:21:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-08T05:21:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-08T05:21:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-08T05:21:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-08T05:21:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-08T05:21:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-08T05:21:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-08T05:21:10.212 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-08T05:21:10.213 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-08T05:21:10.213 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-08T05:21:10.213 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-08T05:21:10.213 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-08T05:21:10.213 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-08T05:21:10.213 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-08T05:21:10.213 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-08T05:21:10.213 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-08T05:21:10.213 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-08T05:21:10.213 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-08T05:21:10.214 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-08T05:21:10.215 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-08T05:21:10.216 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-08T05:21:10.217 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-08T05:21:10.218 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-08T05:21:10.219 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-08T05:21:10.220 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-08T05:21:10.221 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-08T05:21:10.222 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-08T05:21:10.223 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-08T05:21:10.224 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-08T05:21:10.225 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-08T05:21:10.226 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-08T05:21:10.227 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-08T05:21:10.228 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-08T05:21:10.229 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-08T05:21:10.230 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-08T05:21:10.233 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-08T05:21:10.234 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-08T05:21:10.234 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-08T05:21:10.234 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-08T05:21:10.234 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-08T05:21:10.234 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-08T05:21:10.234 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-08T05:21:10.234 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-08T05:21:10.234 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-08T05:21:10.234 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-08T05:21:10.235 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-08T05:21:10.236 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-08T05:21:10.237 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-08T05:21:10.238 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-08T05:21:10.238 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-08T05:21:10.238 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-08T05:21:10.238 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-08T05:21:10.238 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-08T05:21:10.238 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-08T05:21:10.238 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-08T05:21:10.238 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-08T05:21:10.238 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-08T05:21:10.239 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-08T05:21:10.240 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-08T05:21:10.241 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-08T05:21:10.242 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-08T05:21:10.243 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-08T05:21:10.244 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-08T05:21:10.245 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-08T05:21:10.246 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-08T05:21:10.247 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-08T05:21:10.248 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-08T05:21:10.249 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-08T05:21:10.250 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-08T05:21:10.251 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-08T05:21:10.252 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-08T05:21:10.253 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-08T05:21:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-08T05:21:10.255 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-08T05:21:10.256 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-08T05:21:10.257 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-08T05:21:10.258 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-08T05:21:10.259 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-08T05:21:10.260 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-08T05:21:10.261 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-08T05:21:10.262 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-08T05:21:10.263 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-08T05:21:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-08T05:21:10.265 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-08T05:21:10.266 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-08T05:21:10.267 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-08T05:21:10.268 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-08T05:21:10.269 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-08T05:21:10.270 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-08T05:21:10.271 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-08T05:21:10.272 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-08T05:21:10.273 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-08T05:21:10.274 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-08T05:21:10.274 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-08T05:21:10.274 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-08T05:21:10.274 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-08T05:21:10.274 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-08T05:21:10.274 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-08T05:21:10.274 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-08T05:21:10.274 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-08T05:21:10.275 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-08T05:21:10.276 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-08T05:21:10.277 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-08T05:21:10.278 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-08T05:21:10.279 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-08T05:21:10.280 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-08T05:21:10.281 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-08T05:21:10.281 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-08T05:21:10.281 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-08T05:21:10.281 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-08T05:21:10.281 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-08T05:21:10.281 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-08T05:21:10.281 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-08T05:21:10.281 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-08T05:21:10.282 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-08T05:21:10.283 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-08T05:21:10.284 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-08T05:21:10.285 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-08T05:21:10.286 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-08T05:21:10.287 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-08T05:21:10.288 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-08T05:21:10.289 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-08T05:21:10.290 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-08T05:21:10.291 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-08T05:21:10.292 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-08T05:21:10.293 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-08T05:21:10.294 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-08T05:21:10.295 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-08T05:21:10.296 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-08T05:21:10.297 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-08T05:21:10.298 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-08T05:21:10.299 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-08T05:21:10.300 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-08T05:21:10.301 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-08T05:21:10.302 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-08T05:21:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-08T05:21:10.304 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-08T05:21:10.305 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-08T05:21:10.306 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-08T05:21:10.306 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-08T05:21:10.306 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-08T05:21:10.306 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-08T05:21:10.306 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-08T05:21:10.306 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-08T05:21:10.306 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-08T05:21:10.306 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-08T05:21:10.306 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-08T05:21:10.307 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-08T05:21:10.308 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-08T05:21:10.309 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-08T05:21:10.310 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-08T05:21:10.311 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-08T05:21:10.312 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-08T05:21:10.313 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-08T05:21:10.314 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-08T05:21:10.315 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-08T05:21:10.315 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-08T05:21:10.315 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-08T05:21:10.315 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-08T05:21:10.315 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-08T05:21:10.315 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-08T05:21:10.315 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-08T05:21:10.315 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-08T05:21:10.315 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-08T05:21:10.315 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-08T05:21:10.316 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-08T05:21:10.317 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-08T05:21:10.318 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-08T05:21:10.319 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-08T05:21:10.320 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-08T05:21:10.321 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-08T05:21:10.322 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-08T05:21:10.323 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-08T05:21:10.324 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-08T05:21:10.324 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-08T05:21:10.324 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-08T05:21:10.324 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-08T05:21:10.324 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-08T05:21:10.324 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-08T05:21:10.324 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-08T05:21:10.324 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-08T05:21:10.324 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-08T05:21:10.325 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-08T05:21:10.326 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-08T05:21:10.327 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-08T05:21:10.327 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-08T05:21:10.327 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-08T05:21:10.327 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-08T05:21:10.327 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-08T05:21:10.327 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-08T05:21:10.327 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-08T05:21:10.327 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-08T05:21:10.327 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-08T05:21:10.328 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-08T05:21:10.329 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-08T05:21:10.329 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-08T05:21:10.329 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-08T05:21:10.329 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-08T05:21:10.329 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-08T05:21:10.329 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-08T05:21:10.329 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-08T05:21:10.329 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-08T05:21:10.329 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-08T05:21:10.329 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-08T05:21:10.330 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-08T05:21:10.331 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-08T05:21:10.332 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-08T05:21:10.333 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-08T05:21:10.334 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-08T05:21:10.335 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-08T05:21:10.336 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-08T05:21:10.337 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-08T05:21:10.338 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-08T05:21:10.339 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-08T05:21:10.340 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-08T05:21:10.341 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-08T05:21:10.342 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-08T05:21:10.343 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-08T05:21:10.344 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-08T05:21:10.345 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-08T05:21:10.346 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-08T05:21:10.346 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-08T05:21:10.346 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-08T05:21:10.346 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-08T05:21:10.346 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-08T05:21:10.346 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-08T05:21:10.346 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-08T05:21:10.346 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-08T05:21:10.346 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-08T05:21:10.346 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-08T05:21:10.347 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-08T05:21:10.348 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-08T05:21:10.348 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-08T05:21:10.348 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-08T05:21:10.348 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-08T05:21:10.348 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-08T05:21:10.348 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-08T05:21:10.348 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-08T05:21:10.348 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-08T05:21:10.348 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-08T05:21:10.348 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-08T05:21:10.349 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-08T05:21:10.350 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-08T05:21:10.350 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-08T05:21:10.350 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-08T05:21:10.350 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-08T05:21:10.350 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-08T05:21:10.350 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-08T05:21:10.350 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-08T05:21:10.350 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-08T05:21:10.350 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-08T05:21:10.351 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-08T05:21:10.352 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-08T05:21:10.353 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-08T05:21:10.354 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-08T05:21:10.354 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-08T05:21:10.354 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-08T05:21:10.354 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-08T05:21:10.354 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-08T05:21:10.354 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-08T05:21:10.354 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-08T05:21:10.354 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-08T05:21:10.354 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-08T05:21:10.354 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-08T05:21:10.355 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-08T05:21:10.356 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-08T05:21:10.356 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-08T05:21:10.356 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-08T05:21:10.356 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-08T05:21:10.356 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-08T05:21:10.356 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-08T05:21:10.356 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-08T05:21:10.356 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-08T05:21:10.356 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-08T05:21:10.356 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-08T05:21:10.357 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-08T05:21:10.358 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-08T05:21:10.359 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-08T05:21:10.359 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-08T05:21:10.359 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-08T05:21:10.359 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-08T05:21:10.359 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-08T05:21:10.359 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-08T05:21:10.359 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-08T05:21:10.359 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-08T05:21:10.359 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-08T05:21:10.359 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-08T05:21:10.360 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-08T05:21:10.361 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-08T05:21:10.362 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-08T05:21:10.363 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-08T05:21:10.364 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-08T05:21:10.365 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-08T05:21:10.366 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-08T05:21:10.366 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-08T05:21:10.366 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-08T05:21:10.366 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-08T05:21:10.366 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-08T05:21:10.366 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-08T05:21:10.366 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-08T05:21:10.366 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-08T05:21:10.366 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-08T05:21:10.367 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-08T05:21:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-08T05:21:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-08T05:21:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-08T05:21:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-08T05:21:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-08T05:21:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-08T05:21:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-08T05:21:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.alias.bin 2023-10-08T05:21:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-08T05:21:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-08T05:21:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-08T05:21:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-08T05:21:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-08T05:21:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-08T05:21:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-08T05:21:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-08T05:21:10.369 INFO:teuthology.orchestra.run.smithi181.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-08T05:21:10.370 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:10.370 DEBUG:teuthology.orchestra.run.smithi181:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-08T05:21:10.393 INFO:teuthology.orchestra.run.smithi181.stdout:postinstall scriptlet (using /bin/sh): 2023-10-08T05:21:10.393 INFO:teuthology.orchestra.run.smithi181.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-08T05:21:10.393 INFO:teuthology.orchestra.run.smithi181.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T05:21:10.393 INFO:teuthology.orchestra.run.smithi181.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T05:21:10.393 INFO:teuthology.orchestra.run.smithi181.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:10.393 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:fi 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:fi 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:10.394 INFO:teuthology.orchestra.run.smithi181.stdout:fi 2023-10-08T05:21:10.395 DEBUG:teuthology.orchestra.run.smithi181:> sudo rpm -qi grub2-tools 2023-10-08T05:21:10.467 INFO:teuthology.orchestra.run.smithi181.stdout:Name : grub2-tools 2023-10-08T05:21:10.468 INFO:teuthology.orchestra.run.smithi181.stdout:Epoch : 1 2023-10-08T05:21:10.468 INFO:teuthology.orchestra.run.smithi181.stdout:Version : 2.06 2023-10-08T05:21:10.468 INFO:teuthology.orchestra.run.smithi181.stdout:Release : 68.el9 2023-10-08T05:21:10.468 INFO:teuthology.orchestra.run.smithi181.stdout:Architecture: x86_64 2023-10-08T05:21:10.468 INFO:teuthology.orchestra.run.smithi181.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-08T05:21:10.468 INFO:teuthology.orchestra.run.smithi181.stdout:Group : Unspecified 2023-10-08T05:21:10.468 INFO:teuthology.orchestra.run.smithi181.stdout:Size : 8264254 2023-10-08T05:21:10.468 INFO:teuthology.orchestra.run.smithi181.stdout:License : GPLv3+ 2023-10-08T05:21:10.468 INFO:teuthology.orchestra.run.smithi181.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-08T05:21:10.468 INFO:teuthology.orchestra.run.smithi181.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:Packager : builder@centos.org 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:Vendor : CentOS 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:Summary : Support tools for GRUB. 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:Description : 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-08T05:21:10.469 INFO:teuthology.orchestra.run.smithi181.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-08T05:21:10.470 INFO:teuthology.orchestra.run.smithi181.stdout:hardware devices. 2023-10-08T05:21:10.470 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:21:10.470 INFO:teuthology.orchestra.run.smithi181.stdout:This subpackage provides tools for support of all platforms. 2023-10-08T05:21:10.471 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-08T05:21:10.471 INFO:teuthology.task.kernel:Updating grub on smithi181 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:10.471 DEBUG:teuthology.orchestra.run.smithi181:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-08T05:21:11.149 INFO:teuthology.orchestra.run.smithi181.stderr:Generating grub configuration file ... 2023-10-08T05:21:12.634 INFO:teuthology.orchestra.run.smithi181.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-08T05:21:12.660 INFO:teuthology.orchestra.run.smithi181.stderr:done 2023-10-08T05:21:12.662 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-10-08T05:21:12.677 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.YOSKT0j2mL 2023-10-08T05:21:12.677 DEBUG:teuthology.orchestra.run.smithi181:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.YOSKT0j2mL 2023-10-08T05:21:12.753 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.YOSKT0j2mL 2023-10-08T05:21:12.894 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.YOSKT0j2mL is 6KB 2023-10-08T05:21:12.942 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.YOSKT0j2mL 2023-10-08T05:21:12.957 DEBUG:teuthology.orchestra.run.smithi181:> sudo /bin/ls /boot/loader/entries || true 2023-10-08T05:21:13.021 INFO:teuthology.orchestra.run.smithi181.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-08T05:21:13.021 INFO:teuthology.orchestra.run.smithi181.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-08T05:21:13.021 INFO:teuthology.orchestra.run.smithi181.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-08T05:21:13.021 INFO:teuthology.orchestra.run.smithi181.stdout:15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b.conf 2023-10-08T05:21:13.022 DEBUG:teuthology.orchestra.run.smithi181:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:13.160 DEBUG:teuthology.orchestra.run.smithi181:> sudo shutdown -r now 2023-10-08T05:21:29.688 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-08T05:21:29.689 DEBUG:teuthology.orchestra.run.smithi070:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-08T05:21:29.767 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:29.767 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:29.767 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:29.767 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:29.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-08T05:21:29.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-08T05:21:29.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-08T05:21:29.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-08T05:21:29.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-08T05:21:29.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-08T05:21:29.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-08T05:21:29.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-08T05:21:29.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-08T05:21:29.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-08T05:21:29.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-08T05:21:29.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-08T05:21:29.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-08T05:21:29.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-08T05:21:29.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-08T05:21:29.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-08T05:21:29.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-08T05:21:29.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-08T05:21:29.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-08T05:21:29.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-08T05:21:29.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-08T05:21:29.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-08T05:21:29.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-08T05:21:29.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-08T05:21:29.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-08T05:21:29.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-08T05:21:29.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-08T05:21:29.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-08T05:21:29.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-08T05:21:29.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-08T05:21:29.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-08T05:21:29.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-08T05:21:29.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-08T05:21:29.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-08T05:21:29.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-08T05:21:29.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-08T05:21:29.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-08T05:21:29.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-08T05:21:29.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-08T05:21:29.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-08T05:21:29.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-08T05:21:29.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-08T05:21:29.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-08T05:21:29.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-08T05:21:29.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-08T05:21:29.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-08T05:21:29.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-08T05:21:29.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-08T05:21:29.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-08T05:21:29.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-08T05:21:29.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-08T05:21:29.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-08T05:21:29.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-08T05:21:29.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-08T05:21:29.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-08T05:21:29.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-08T05:21:29.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-08T05:21:29.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-08T05:21:29.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-08T05:21:29.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-08T05:21:29.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-08T05:21:29.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-08T05:21:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-08T05:21:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-08T05:21:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-08T05:21:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-08T05:21:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-08T05:21:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-08T05:21:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-08T05:21:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-08T05:21:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-08T05:21:29.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-08T05:21:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-08T05:21:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-08T05:21:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-08T05:21:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-08T05:21:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-08T05:21:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-08T05:21:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-08T05:21:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-08T05:21:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-08T05:21:29.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-08T05:21:29.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-08T05:21:29.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-08T05:21:29.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-08T05:21:29.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-08T05:21:29.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-08T05:21:29.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-08T05:21:29.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-08T05:21:29.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-08T05:21:29.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-08T05:21:29.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-08T05:21:29.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-08T05:21:29.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-08T05:21:29.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-08T05:21:29.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-08T05:21:29.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-08T05:21:29.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-08T05:21:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-08T05:21:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-08T05:21:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-08T05:21:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-08T05:21:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-08T05:21:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-08T05:21:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-08T05:21:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-08T05:21:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-08T05:21:29.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-08T05:21:29.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-08T05:21:29.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-08T05:21:29.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-08T05:21:29.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-08T05:21:29.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-08T05:21:29.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-08T05:21:29.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-08T05:21:29.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-08T05:21:29.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-08T05:21:29.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-08T05:21:29.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-08T05:21:29.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-08T05:21:29.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-08T05:21:29.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-08T05:21:29.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-08T05:21:29.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-08T05:21:29.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-08T05:21:29.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-08T05:21:29.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-08T05:21:29.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-08T05:21:29.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-08T05:21:29.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-08T05:21:29.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-08T05:21:29.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-08T05:21:29.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-08T05:21:29.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-08T05:21:29.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-08T05:21:29.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-08T05:21:29.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-08T05:21:29.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-08T05:21:29.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-08T05:21:29.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-08T05:21:29.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-08T05:21:29.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-08T05:21:29.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-08T05:21:29.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-08T05:21:29.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-08T05:21:29.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-08T05:21:29.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-08T05:21:29.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-08T05:21:29.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-08T05:21:29.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-08T05:21:29.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-08T05:21:29.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-08T05:21:29.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-08T05:21:29.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-08T05:21:29.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-08T05:21:29.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-08T05:21:29.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-08T05:21:29.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-08T05:21:29.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-08T05:21:29.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-08T05:21:29.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-08T05:21:29.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-08T05:21:29.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-08T05:21:29.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-08T05:21:29.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-08T05:21:29.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-08T05:21:29.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-08T05:21:29.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-08T05:21:29.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-08T05:21:29.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-08T05:21:29.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-08T05:21:29.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-08T05:21:29.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-08T05:21:29.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-08T05:21:29.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-08T05:21:29.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-08T05:21:29.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-08T05:21:29.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-08T05:21:29.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-08T05:21:29.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-08T05:21:29.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-08T05:21:29.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-08T05:21:29.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-08T05:21:29.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-08T05:21:29.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-08T05:21:29.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-08T05:21:29.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-08T05:21:29.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-08T05:21:29.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-08T05:21:29.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-08T05:21:29.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-08T05:21:29.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-08T05:21:29.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-08T05:21:29.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-08T05:21:29.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-08T05:21:29.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-08T05:21:29.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-08T05:21:29.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-08T05:21:29.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-08T05:21:29.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-08T05:21:29.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-08T05:21:29.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-08T05:21:29.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-08T05:21:29.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-08T05:21:29.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-08T05:21:29.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-08T05:21:29.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-08T05:21:29.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-08T05:21:29.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-08T05:21:29.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-08T05:21:29.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-08T05:21:29.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-08T05:21:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-08T05:21:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-08T05:21:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-08T05:21:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-08T05:21:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-08T05:21:29.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-08T05:21:29.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-08T05:21:29.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-08T05:21:29.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-08T05:21:29.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-08T05:21:29.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-08T05:21:29.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-08T05:21:29.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-08T05:21:29.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-08T05:21:29.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-08T05:21:29.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-08T05:21:29.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-08T05:21:29.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-08T05:21:29.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-08T05:21:29.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-08T05:21:29.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-08T05:21:29.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-08T05:21:29.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-08T05:21:29.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-08T05:21:29.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-08T05:21:29.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-08T05:21:29.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-08T05:21:29.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-08T05:21:29.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-08T05:21:29.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-08T05:21:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-08T05:21:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-08T05:21:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-08T05:21:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-08T05:21:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-08T05:21:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-08T05:21:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-08T05:21:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-08T05:21:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-08T05:21:29.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-08T05:21:29.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-08T05:21:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-08T05:21:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-08T05:21:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-08T05:21:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-08T05:21:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-08T05:21:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-08T05:21:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-08T05:21:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-08T05:21:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-08T05:21:29.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-08T05:21:29.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-08T05:21:29.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-08T05:21:29.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-08T05:21:29.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-08T05:21:29.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-08T05:21:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-08T05:21:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-08T05:21:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-08T05:21:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-08T05:21:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-08T05:21:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-08T05:21:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-08T05:21:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-08T05:21:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-08T05:21:29.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-08T05:21:29.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-08T05:21:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-08T05:21:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-08T05:21:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-08T05:21:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-08T05:21:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-08T05:21:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-08T05:21:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-08T05:21:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-08T05:21:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-08T05:21:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-08T05:21:29.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-08T05:21:29.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-08T05:21:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-08T05:21:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-08T05:21:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-08T05:21:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-08T05:21:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-08T05:21:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-08T05:21:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-08T05:21:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-08T05:21:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-08T05:21:29.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-08T05:21:29.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-08T05:21:29.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-08T05:21:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-08T05:21:29.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-08T05:21:29.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-08T05:21:29.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-08T05:21:29.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-08T05:21:29.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-08T05:21:29.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-08T05:21:29.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-08T05:21:29.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-08T05:21:29.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-08T05:21:29.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-08T05:21:29.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-08T05:21:29.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-08T05:21:29.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-08T05:21:29.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-08T05:21:29.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-08T05:21:29.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-08T05:21:29.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-08T05:21:29.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-08T05:21:29.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-08T05:21:29.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-08T05:21:29.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-08T05:21:29.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-08T05:21:29.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-08T05:21:29.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-08T05:21:29.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-08T05:21:29.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-08T05:21:29.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-08T05:21:29.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-08T05:21:29.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-08T05:21:29.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-08T05:21:29.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-08T05:21:29.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-08T05:21:29.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-08T05:21:29.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-08T05:21:29.837 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-08T05:21:29.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-08T05:21:29.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-08T05:21:29.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-08T05:21:29.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-08T05:21:29.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-08T05:21:29.838 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-08T05:21:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-08T05:21:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-08T05:21:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-08T05:21:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-08T05:21:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-08T05:21:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-08T05:21:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-08T05:21:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-08T05:21:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-08T05:21:29.839 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-08T05:21:29.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-08T05:21:29.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-08T05:21:29.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-08T05:21:29.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-08T05:21:29.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-08T05:21:29.840 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-08T05:21:29.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-08T05:21:29.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-08T05:21:29.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-08T05:21:29.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-08T05:21:29.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-08T05:21:29.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-08T05:21:29.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-08T05:21:29.841 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-08T05:21:29.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-08T05:21:29.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-08T05:21:29.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-08T05:21:29.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-08T05:21:29.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-08T05:21:29.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-08T05:21:29.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-08T05:21:29.842 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-08T05:21:29.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-08T05:21:29.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-08T05:21:29.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-08T05:21:29.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-08T05:21:29.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-08T05:21:29.843 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-08T05:21:29.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-08T05:21:29.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-08T05:21:29.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-08T05:21:29.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-08T05:21:29.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-08T05:21:29.844 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-08T05:21:29.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-08T05:21:29.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-08T05:21:29.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-08T05:21:29.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-08T05:21:29.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-08T05:21:29.845 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-08T05:21:29.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-08T05:21:29.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-08T05:21:29.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-08T05:21:29.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-08T05:21:29.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-08T05:21:29.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-08T05:21:29.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-08T05:21:29.846 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-08T05:21:29.847 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-08T05:21:29.848 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-08T05:21:29.849 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-08T05:21:29.850 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-08T05:21:29.851 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-08T05:21:29.852 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-08T05:21:29.853 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-08T05:21:29.854 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-08T05:21:29.855 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-08T05:21:29.856 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-08T05:21:29.857 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-08T05:21:29.858 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-08T05:21:29.859 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-08T05:21:29.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-08T05:21:29.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-08T05:21:29.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-08T05:21:29.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-08T05:21:29.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-08T05:21:29.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-08T05:21:29.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-08T05:21:29.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-08T05:21:29.860 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-08T05:21:29.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-08T05:21:29.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-08T05:21:29.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-08T05:21:29.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-08T05:21:29.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-08T05:21:29.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-08T05:21:29.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-08T05:21:29.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-08T05:21:29.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-08T05:21:29.861 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-08T05:21:29.862 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-08T05:21:29.863 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-08T05:21:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-08T05:21:29.865 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-08T05:21:29.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-08T05:21:29.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-08T05:21:29.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-08T05:21:29.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-08T05:21:29.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-08T05:21:29.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-08T05:21:29.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-08T05:21:29.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-08T05:21:29.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-08T05:21:29.866 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-08T05:21:29.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-08T05:21:29.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-08T05:21:29.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-08T05:21:29.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-08T05:21:29.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-08T05:21:29.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-08T05:21:29.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-08T05:21:29.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-08T05:21:29.867 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-08T05:21:29.868 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-08T05:21:29.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-08T05:21:29.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-08T05:21:29.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-08T05:21:29.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-08T05:21:29.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-08T05:21:29.869 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-08T05:21:29.870 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-08T05:21:29.871 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-08T05:21:29.872 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-08T05:21:29.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-08T05:21:29.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-08T05:21:29.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-08T05:21:29.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-08T05:21:29.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-08T05:21:29.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-08T05:21:29.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-08T05:21:29.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-08T05:21:29.873 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-08T05:21:29.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-08T05:21:29.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-08T05:21:29.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-08T05:21:29.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-08T05:21:29.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-08T05:21:29.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-08T05:21:29.874 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-08T05:21:29.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-08T05:21:29.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-08T05:21:29.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-08T05:21:29.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-08T05:21:29.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-08T05:21:29.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-08T05:21:29.875 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-08T05:21:29.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-08T05:21:29.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-08T05:21:29.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-08T05:21:29.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-08T05:21:29.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-08T05:21:29.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-08T05:21:29.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-08T05:21:29.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-08T05:21:29.876 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-08T05:21:29.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-08T05:21:29.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-08T05:21:29.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-08T05:21:29.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-08T05:21:29.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-08T05:21:29.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-08T05:21:29.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-08T05:21:29.877 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-08T05:21:29.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-08T05:21:29.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-08T05:21:29.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-08T05:21:29.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-08T05:21:29.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-08T05:21:29.878 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-08T05:21:29.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-08T05:21:29.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-08T05:21:29.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-08T05:21:29.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-08T05:21:29.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-08T05:21:29.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-08T05:21:29.879 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-08T05:21:29.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-08T05:21:29.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-08T05:21:29.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-08T05:21:29.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-08T05:21:29.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-08T05:21:29.880 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-08T05:21:29.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-08T05:21:29.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-08T05:21:29.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-08T05:21:29.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-08T05:21:29.881 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-08T05:21:29.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-08T05:21:29.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-08T05:21:29.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-08T05:21:29.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-08T05:21:29.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-08T05:21:29.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-08T05:21:29.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-08T05:21:29.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-08T05:21:29.882 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-08T05:21:29.883 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-08T05:21:29.884 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-08T05:21:29.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-08T05:21:29.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-08T05:21:29.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-08T05:21:29.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-08T05:21:29.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-08T05:21:29.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-08T05:21:29.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-08T05:21:29.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-08T05:21:29.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-08T05:21:29.885 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-08T05:21:29.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-08T05:21:29.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-08T05:21:29.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-08T05:21:29.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-08T05:21:29.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-08T05:21:29.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-08T05:21:29.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-08T05:21:29.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-08T05:21:29.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-08T05:21:29.886 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-08T05:21:29.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-08T05:21:29.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-08T05:21:29.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-08T05:21:29.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-08T05:21:29.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-08T05:21:29.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-08T05:21:29.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-08T05:21:29.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-08T05:21:29.887 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-08T05:21:29.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-08T05:21:29.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-08T05:21:29.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-08T05:21:29.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-08T05:21:29.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-08T05:21:29.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-08T05:21:29.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-08T05:21:29.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-08T05:21:29.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-08T05:21:29.888 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-08T05:21:29.889 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-08T05:21:29.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-08T05:21:29.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-08T05:21:29.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-08T05:21:29.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-08T05:21:29.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-08T05:21:29.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-08T05:21:29.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-08T05:21:29.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-08T05:21:29.890 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-08T05:21:29.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-08T05:21:29.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-08T05:21:29.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-08T05:21:29.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-08T05:21:29.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-08T05:21:29.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-08T05:21:29.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-08T05:21:29.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-08T05:21:29.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-08T05:21:29.891 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-08T05:21:29.892 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-08T05:21:29.893 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-08T05:21:29.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-08T05:21:29.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-08T05:21:29.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-08T05:21:29.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-08T05:21:29.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-08T05:21:29.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-08T05:21:29.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-08T05:21:29.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-08T05:21:29.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-08T05:21:29.894 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-08T05:21:29.895 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-08T05:21:29.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-08T05:21:29.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-08T05:21:29.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-08T05:21:29.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-08T05:21:29.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-08T05:21:29.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-08T05:21:29.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-08T05:21:29.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-08T05:21:29.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-08T05:21:29.896 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-08T05:21:29.897 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-08T05:21:29.898 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-08T05:21:29.899 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-08T05:21:29.900 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-08T05:21:29.901 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-08T05:21:29.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-08T05:21:29.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-08T05:21:29.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-08T05:21:29.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-08T05:21:29.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-08T05:21:29.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-08T05:21:29.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-08T05:21:29.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-08T05:21:29.902 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-08T05:21:29.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-08T05:21:29.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-08T05:21:29.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-08T05:21:29.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-08T05:21:29.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-08T05:21:29.903 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-08T05:21:29.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-08T05:21:29.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-08T05:21:29.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-08T05:21:29.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-08T05:21:29.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-08T05:21:29.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-08T05:21:29.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-08T05:21:29.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-08T05:21:29.904 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-08T05:21:29.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-08T05:21:29.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-08T05:21:29.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-08T05:21:29.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-08T05:21:29.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-08T05:21:29.905 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-08T05:21:29.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-08T05:21:29.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-08T05:21:29.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-08T05:21:29.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-08T05:21:29.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-08T05:21:29.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-08T05:21:29.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-08T05:21:29.906 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-08T05:21:29.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-08T05:21:29.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-08T05:21:29.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-08T05:21:29.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-08T05:21:29.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-08T05:21:29.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-08T05:21:29.907 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-08T05:21:29.908 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-08T05:21:29.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-08T05:21:29.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-08T05:21:29.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-08T05:21:29.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-08T05:21:29.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-08T05:21:29.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-08T05:21:29.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-08T05:21:29.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-08T05:21:29.909 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-08T05:21:29.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-08T05:21:29.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-08T05:21:29.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-08T05:21:29.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-08T05:21:29.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-08T05:21:29.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-08T05:21:29.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-08T05:21:29.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-08T05:21:29.910 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-08T05:21:29.911 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-08T05:21:29.912 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-08T05:21:29.913 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-08T05:21:29.914 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-08T05:21:29.915 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-08T05:21:29.916 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-08T05:21:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-08T05:21:29.918 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-08T05:21:29.919 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-08T05:21:29.920 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-08T05:21:29.921 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-08T05:21:29.922 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-08T05:21:29.923 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-08T05:21:29.924 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-08T05:21:29.925 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-08T05:21:29.926 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-08T05:21:29.927 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-08T05:21:29.927 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-08T05:21:29.930 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-08T05:21:29.930 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-08T05:21:29.930 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-08T05:21:29.930 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-08T05:21:29.930 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-08T05:21:29.931 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-08T05:21:29.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-08T05:21:29.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-08T05:21:29.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-08T05:21:29.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-08T05:21:29.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-08T05:21:29.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-08T05:21:29.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-08T05:21:29.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-08T05:21:29.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-08T05:21:29.932 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-08T05:21:29.933 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-08T05:21:29.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-08T05:21:29.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-08T05:21:29.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-08T05:21:29.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-08T05:21:29.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-08T05:21:29.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-08T05:21:29.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-08T05:21:29.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-08T05:21:29.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-08T05:21:29.934 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-08T05:21:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-08T05:21:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-08T05:21:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-08T05:21:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-08T05:21:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-08T05:21:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-08T05:21:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-08T05:21:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-08T05:21:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-08T05:21:29.935 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-08T05:21:29.936 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-08T05:21:29.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-08T05:21:29.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-08T05:21:29.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-08T05:21:29.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-08T05:21:29.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-08T05:21:29.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-08T05:21:29.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-08T05:21:29.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-08T05:21:29.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-08T05:21:29.937 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-08T05:21:29.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-08T05:21:29.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-08T05:21:29.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-08T05:21:29.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-08T05:21:29.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-08T05:21:29.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-08T05:21:29.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-08T05:21:29.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-08T05:21:29.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-08T05:21:29.938 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-08T05:21:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-08T05:21:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-08T05:21:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-08T05:21:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-08T05:21:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-08T05:21:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-08T05:21:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-08T05:21:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-08T05:21:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-08T05:21:29.939 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-08T05:21:29.940 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-08T05:21:29.941 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-08T05:21:29.942 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-08T05:21:29.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-08T05:21:29.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-08T05:21:29.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-08T05:21:29.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-08T05:21:29.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-08T05:21:29.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-08T05:21:29.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-08T05:21:29.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-08T05:21:29.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-08T05:21:29.943 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-08T05:21:29.944 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-08T05:21:29.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-08T05:21:29.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-08T05:21:29.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-08T05:21:29.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-08T05:21:29.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-08T05:21:29.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-08T05:21:29.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-08T05:21:29.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-08T05:21:29.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-08T05:21:29.945 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-08T05:21:29.946 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-08T05:21:29.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-08T05:21:29.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-08T05:21:29.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-08T05:21:29.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-08T05:21:29.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-08T05:21:29.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-08T05:21:29.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-08T05:21:29.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-08T05:21:29.947 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-08T05:21:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-08T05:21:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-08T05:21:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-08T05:21:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-08T05:21:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-08T05:21:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-08T05:21:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-08T05:21:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-08T05:21:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-08T05:21:29.948 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-08T05:21:29.949 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-08T05:21:29.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-08T05:21:29.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-08T05:21:29.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-08T05:21:29.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-08T05:21:29.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-08T05:21:29.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-08T05:21:29.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-08T05:21:29.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-08T05:21:29.950 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-08T05:21:29.951 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-08T05:21:29.952 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-08T05:21:29.953 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-08T05:21:29.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-08T05:21:29.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-08T05:21:29.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-08T05:21:29.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-08T05:21:29.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-08T05:21:29.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-08T05:21:29.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-08T05:21:29.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-08T05:21:29.954 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-08T05:21:29.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-08T05:21:29.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-08T05:21:29.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-08T05:21:29.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-08T05:21:29.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-08T05:21:29.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-08T05:21:29.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-08T05:21:29.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-08T05:21:29.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-08T05:21:29.955 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-08T05:21:29.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-08T05:21:29.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-08T05:21:29.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-08T05:21:29.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-08T05:21:29.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-08T05:21:29.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-08T05:21:29.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-08T05:21:29.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-08T05:21:29.956 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-08T05:21:29.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-08T05:21:29.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-08T05:21:29.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-08T05:21:29.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-08T05:21:29.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-08T05:21:29.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-08T05:21:29.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-08T05:21:29.957 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-08T05:21:29.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-08T05:21:29.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-08T05:21:29.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-08T05:21:29.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-08T05:21:29.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-08T05:21:29.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-08T05:21:29.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-08T05:21:29.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-08T05:21:29.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-08T05:21:29.958 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-08T05:21:29.959 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-08T05:21:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-08T05:21:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-08T05:21:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-08T05:21:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-08T05:21:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-08T05:21:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-08T05:21:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-08T05:21:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-08T05:21:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-08T05:21:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-08T05:21:29.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-08T05:21:29.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-08T05:21:29.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-08T05:21:29.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-08T05:21:29.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-08T05:21:29.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-08T05:21:29.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-08T05:21:29.961 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-08T05:21:29.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-08T05:21:29.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-08T05:21:29.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-08T05:21:29.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-08T05:21:29.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-08T05:21:29.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-08T05:21:29.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-08T05:21:29.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-08T05:21:29.962 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-08T05:21:29.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-08T05:21:29.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-08T05:21:29.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-08T05:21:29.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-08T05:21:29.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-08T05:21:29.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-08T05:21:29.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-08T05:21:29.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-08T05:21:29.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-08T05:21:29.963 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-08T05:21:29.964 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-08T05:21:29.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-08T05:21:29.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-08T05:21:29.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-08T05:21:29.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-08T05:21:29.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-08T05:21:29.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-08T05:21:29.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-08T05:21:29.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-08T05:21:29.965 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-08T05:21:29.966 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-08T05:21:29.967 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-08T05:21:29.968 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-08T05:21:29.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.alias.bin 2023-10-08T05:21:29.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-08T05:21:29.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-08T05:21:29.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-08T05:21:29.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-08T05:21:29.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-08T05:21:29.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-08T05:21:29.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-08T05:21:29.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-08T05:21:29.969 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-08T05:21:29.970 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:29.970 DEBUG:teuthology.orchestra.run.smithi070:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-08T05:21:29.992 INFO:teuthology.orchestra.run.smithi070.stdout:postinstall scriptlet (using /bin/sh): 2023-10-08T05:21:29.992 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-08T05:21:29.992 INFO:teuthology.orchestra.run.smithi070.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T05:21:29.992 INFO:teuthology.orchestra.run.smithi070.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T05:21:29.992 INFO:teuthology.orchestra.run.smithi070.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:29.992 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T05:21:29.992 INFO:teuthology.orchestra.run.smithi070.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T05:21:29.992 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-10-08T05:21:29.992 INFO:teuthology.orchestra.run.smithi070.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-08T05:21:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-08T05:21:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-08T05:21:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-08T05:21:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-10-08T05:21:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-08T05:21:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-08T05:21:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-10-08T05:21:29.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2023-10-08T05:21:30.069 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2023-10-08T05:21:30.069 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2023-10-08T05:21:30.069 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2023-10-08T05:21:30.069 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 68.el9 2023-10-08T05:21:30.069 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2023-10-08T05:21:30.069 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-10-08T05:21:30.069 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2023-10-08T05:21:30.069 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8264254 2023-10-08T05:21:30.070 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2023-10-08T05:21:30.070 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-10-08T05:21:30.070 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-10-08T05:21:30.070 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-10-08T05:21:30.070 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-10-08T05:21:30.070 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2023-10-08T05:21:30.070 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2023-10-08T05:21:30.070 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-08T05:21:30.071 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2023-10-08T05:21:30.071 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2023-10-08T05:21:30.071 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:21:30.071 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-08T05:21:30.071 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-08T05:21:30.071 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-08T05:21:30.071 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2023-10-08T05:21:30.071 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:21:30.071 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2023-10-08T05:21:30.072 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-08T05:21:30.072 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:30.072 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-08T05:21:30.674 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2023-10-08T05:21:32.009 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-10-08T05:21:32.043 INFO:teuthology.orchestra.run.smithi070.stderr:done 2023-10-08T05:21:32.045 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2023-10-08T05:21:32.061 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.58sQgTfe14 2023-10-08T05:21:32.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.58sQgTfe14 2023-10-08T05:21:32.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.58sQgTfe14 2023-10-08T05:21:32.272 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.58sQgTfe14 is 6KB 2023-10-08T05:21:32.321 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.58sQgTfe14 2023-10-08T05:21:32.335 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2023-10-08T05:21:32.400 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-10-08T05:21:32.400 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-10-08T05:21:32.400 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-10-08T05:21:32.400 INFO:teuthology.orchestra.run.smithi070.stdout:15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b.conf 2023-10-08T05:21:32.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b 2023-10-08T05:21:32.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2023-10-08T05:21:43.169 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-08T05:21:43.169 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-08T05:21:43.170 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T05:22:02.570 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-08T05:22:02.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-08T05:22:02.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T05:22:43.207 DEBUG:teuthology.orchestra.remote:timed out 2023-10-08T05:22:52.214 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-08T05:22:52.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T05:22:59.815 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-10-08T05:23:00.402 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-08T05:23:00.402 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-08T05:23:00.403 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2023-10-08T05:23:00.443 INFO:teuthology.orchestra.run.smithi181.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-08T05:23:00.443 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-08T05:23:00.444 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-08T05:23:00.444 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-08T05:23:01.445 DEBUG:teuthology.parallel:result is None 2023-10-08T05:23:02.574 DEBUG:teuthology.orchestra.remote:timed out 2023-10-08T05:23:11.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-08T05:23:11.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T05:23:19.165 DEBUG:teuthology.orchestra.run.smithi070:> true 2023-10-08T05:23:19.780 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-08T05:23:19.780 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-08T05:23:19.780 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2023-10-08T05:23:19.819 INFO:teuthology.orchestra.run.smithi070.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-08T05:23:19.819 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-08T05:23:19.820 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-08T05:23:19.820 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-08T05:23:20.821 DEBUG:teuthology.parallel:result is None 2023-10-08T05:23:20.821 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-08T05:23:20.831 INFO:teuthology.task.internal:Creating test directory... 2023-10-08T05:23:20.831 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T05:23:20.842 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T05:23:20.907 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-08T05:23:20.916 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-08T05:23:20.952 INFO:teuthology.task.internal:Creating archive directory... 2023-10-08T05:23:20.953 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T05:23:20.958 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T05:23:21.037 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-08T05:23:21.044 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-08T05:23:21.044 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T05:23:21.050 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T05:23:21.202 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T05:23:21.215 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T05:23:21.284 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T05:23:21.291 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T05:23:21.296 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-08T05:23:21.303 INFO:teuthology.task.internal:Configuring sudo... 2023-10-08T05:23:21.303 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T05:23:21.309 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T05:23:21.411 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-08T05:23:21.421 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-08T05:23:21.421 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T05:23:21.426 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T05:23:21.481 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T05:23:21.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T05:23:21.691 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T05:23:21.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T05:23:21.855 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:23:21.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T05:23:21.942 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T05:23:22.035 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T05:23:22.148 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T05:23:22.232 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T05:23:22.336 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:23:22.336 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T05:23:22.434 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2023-10-08T05:23:22.442 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2023-10-08T05:23:22.568 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T05:23:22.613 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T05:23:23.108 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-08T05:23:23.116 INFO:teuthology.task.internal:Starting timer... 2023-10-08T05:23:23.116 INFO:teuthology.run_tasks:Running task pcp... 2023-10-08T05:23:23.154 INFO:teuthology.run_tasks:Running task selinux... 2023-10-08T05:23:23.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2023-10-08T05:23:23.334 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2023-10-08T05:23:23.338 DEBUG:teuthology.orchestra.run.smithi181:> sudo service auditd rotate 2023-10-08T05:23:23.470 INFO:teuthology.orchestra.run.smithi181.stdout:Rotating logs: 2023-10-08T05:23:23.474 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-08T05:23:23.475 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2023-10-08T05:23:23.529 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2023-10-08T05:23:23.529 DEBUG:teuthology.orchestra.run.smithi181:> /usr/sbin/getenforce 2023-10-08T05:23:23.592 INFO:teuthology.orchestra.run.smithi181.stdout:Permissive 2023-10-08T05:23:23.592 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi181.front.sepia.ceph.com': 'permissive'} 2023-10-08T05:23:23.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T05:23:23.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T05:23:23.668 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T05:23:23.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T05:23:23.766 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-08T05:23:23.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2023-10-08T05:23:23.849 DEBUG:teuthology.orchestra.run.smithi181:> sudo /usr/sbin/setenforce permissive 2023-10-08T05:23:23.964 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-08T05:23:23.977 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-08T05:23:23.977 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-08T05:23:23.995 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-08T05:23:23.996 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi070.front.sepia.ceph.com,smithi181.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-08T05:33:14.401 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com')] 2023-10-08T05:33:14.402 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-08T05:33:14.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T05:33:14.541 DEBUG:teuthology.orchestra.run.smithi070:> true 2023-10-08T05:33:14.705 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-10-08T05:33:14.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-08T05:33:14.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T05:33:14.838 DEBUG:teuthology.orchestra.run.smithi181:> true 2023-10-08T05:33:14.991 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2023-10-08T05:33:14.992 INFO:teuthology.run_tasks:Running task clock... 2023-10-08T05:33:15.003 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-08T05:33:15.003 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T05:33:15.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T05:33:15.012 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T05:33:15.012 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T05:33:15.110 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T05:33:15.111 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T05:33:15.164 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T05:33:15.165 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T05:33:15.256 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: ntpd: command not found 2023-10-08T05:33:15.256 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2023-10-08T05:33:15.295 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2023-10-08T05:33:15.296 INFO:teuthology.orchestra.run.smithi181.stdout:506 Cannot talk to daemon 2023-10-08T05:33:15.345 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T05:33:15.347 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T05:33:15.396 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T05:33:15.402 INFO:teuthology.orchestra.run.smithi181.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T05:33:15.521 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2023-10-08T05:33:15.529 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2023-10-08T05:33:15.529 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T05:33:15.529 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2023-10-08T05:33:15.529 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T05:33:15.529 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T05:33:15.530 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T05:33:15.530 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T05:33:15.536 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T05:33:15.536 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-10-08T05:33:15.536 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T05:33:15.536 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T05:33:15.536 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T05:33:15.537 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T05:33:15.537 INFO:teuthology.run_tasks:Running task install... 2023-10-08T05:33:15.545 DEBUG:teuthology.task.install:project ceph 2023-10-08T05:33:15.546 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9725b08fd801c82987c737f7e6d354878f036ee8'}} 2023-10-08T05:33:15.546 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9725b08fd801c82987c737f7e6d354878f036ee8'} 2023-10-08T05:33:15.546 INFO:teuthology.task.install:Using flavor: default 2023-10-08T05:33:15.551 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-08T05:33:15.552 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-08T05:33:15.552 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9725b08fd801c82987c737f7e6d354878f036ee8', 'tag': None, 'wait_for_package': False} 2023-10-08T05:33:15.552 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9725b08fd801c82987c737f7e6d354878f036ee8 2023-10-08T05:33:15.554 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9725b08fd801c82987c737f7e6d354878f036ee8', 'tag': None, 'wait_for_package': False} 2023-10-08T05:33:15.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9725b08fd801c82987c737f7e6d354878f036ee8 2023-10-08T05:33:15.700 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/9725b08fd801c82987c737f7e6d354878f036ee8/centos/9/flavors/default/ 2023-10-08T05:33:15.701 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6609.g9725b08f 2023-10-08T05:33:15.774 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/9725b08fd801c82987c737f7e6d354878f036ee8/centos/9/flavors/default/ 2023-10-08T05:33:15.774 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6609.g9725b08f 2023-10-08T05:33:15.811 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/9725b08fd801c82987c737f7e6d354878f036ee8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/9725b08fd801c82987c737f7e6d354878f036ee8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/9725b08fd801c82987c737f7e6d354878f036ee8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T05:33:15.811 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:33:15.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T05:33:15.914 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T05:33:15.914 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9725b08fd801c82987c737f7e6d354878f036ee8/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T05:33:15.942 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/9725b08fd801c82987c737f7e6d354878f036ee8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/9725b08fd801c82987c737f7e6d354878f036ee8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/9725b08fd801c82987c737f7e6d354878f036ee8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T05:33:15.942 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:33:15.942 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T05:33:16.047 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-08T05:33:16.048 DEBUG:teuthology.orchestra.run.smithi181:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9725b08fd801c82987c737f7e6d354878f036ee8/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T05:33:16.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T05:33:16.192 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T05:33:16.192 DEBUG:teuthology.orchestra.run.smithi181:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T05:33:16.280 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2023-10-08T05:33:16.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2023-10-08T05:33:16.329 DEBUG:teuthology.orchestra.run.smithi181:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T05:33:16.423 INFO:teuthology.orchestra.run.smithi181.stdout:check_obsoletes = 1 2023-10-08T05:33:16.427 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum clean all 2023-10-08T05:33:16.639 INFO:teuthology.orchestra.run.smithi070.stdout:70 files removed 2023-10-08T05:33:16.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T05:33:16.778 INFO:teuthology.orchestra.run.smithi181.stdout:70 files removed 2023-10-08T05:33:16.817 DEBUG:teuthology.orchestra.run.smithi181:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-08T05:33:17.611 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 178 kB/s | 78 kB 00:00 2023-10-08T05:33:17.815 INFO:teuthology.orchestra.run.smithi181.stdout:ceph packages for x86_64 160 kB/s | 78 kB 00:00 2023-10-08T05:33:18.128 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 22 kB/s | 9.4 kB 00:00 2023-10-08T05:33:18.374 INFO:teuthology.orchestra.run.smithi181.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-10-08T05:33:18.680 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2023-10-08T05:33:18.825 INFO:teuthology.orchestra.run.smithi181.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2023-10-08T05:33:19.282 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-10-08T05:33:19.412 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-10-08T05:33:20.898 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2023-10-08T05:33:20.988 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS-9 - BaseOS 5.5 MB/s | 2.0 MB 00:00 2023-10-08T05:33:21.912 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 7.9 MB 00:00 2023-10-08T05:33:22.350 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 7.9 MB 00:00 2023-10-08T05:33:26.060 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - AppStream 6.9 MB/s | 18 MB 00:02 2023-10-08T05:33:27.378 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 18 MB 00:04 2023-10-08T05:33:30.144 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - CRB 11 MB/s | 5.7 MB 00:00 2023-10-08T05:33:31.462 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 11 MB/s | 5.7 MB 00:00 2023-10-08T05:33:31.644 INFO:teuthology.orchestra.run.smithi181.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 14 kB 00:00 2023-10-08T05:33:32.896 INFO:teuthology.orchestra.run.smithi181.stdout:Extra Packages for Enterprise Linux 17 MB/s | 19 MB 00:01 2023-10-08T05:33:33.238 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 14 kB 00:00 2023-10-08T05:33:34.156 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 25 MB/s | 19 MB 00:00 2023-10-08T05:33:37.980 INFO:teuthology.orchestra.run.smithi181.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2023-10-08T05:33:39.197 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2023-10-08T05:33:40.396 INFO:teuthology.orchestra.run.smithi181.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-08T05:33:40.397 INFO:teuthology.orchestra.run.smithi181.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-08T05:33:40.417 INFO:teuthology.orchestra.run.smithi181.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-08T05:33:40.418 INFO:teuthology.orchestra.run.smithi181.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-08T05:33:40.420 INFO:teuthology.orchestra.run.smithi181.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-08T05:33:40.421 INFO:teuthology.orchestra.run.smithi181.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-08T05:33:40.423 INFO:teuthology.orchestra.run.smithi181.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-08T05:33:40.427 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-08T05:33:40.428 INFO:teuthology.orchestra.run.smithi181.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-08T05:33:40.503 INFO:teuthology.orchestra.run.smithi181.stdout:Dependencies resolved. 2023-10-08T05:33:40.521 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================ 2023-10-08T05:33:40.521 INFO:teuthology.orchestra.run.smithi181.stdout: Package Arch Version Repository Size 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================ 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout:Installing: 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout: ceph x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 6.5 k 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 5.2 M 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 855 k 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 141 k 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 1.5 M 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 132 k 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 1.8 M 2023-10-08T05:33:40.522 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 7.4 M 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 47 k 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 8.6 M 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 41 M 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 262 k 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 269 k 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 218 k 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 18 k 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 31 k 2023-10-08T05:33:40.523 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2 x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 715 k 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 126 k 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 162 k 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 321 k 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 298 k 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 100 k 2023-10-08T05:33:40.524 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 85 k 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 3.0 M 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 169 k 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrading: 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout: librados2 x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 3.3 M 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1 x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 3.0 M 2023-10-08T05:33:40.525 INFO:teuthology.orchestra.run.smithi181.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout:Installing dependencies: 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 19 M 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 23 k 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 2.1 M 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 242 k 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 4.4 M 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 18 M 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 15 k 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 24 k 2023-10-08T05:33:40.526 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 164 k 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-08T05:33:40.527 INFO:teuthology.orchestra.run.smithi181.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1 x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 472 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2 x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 4.8 M 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-08T05:33:40.528 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 45 k 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 127 k 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-08T05:33:40.529 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth noarch 1:2.23.0-1.el9 epel 196 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-08T05:33:40.530 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-08T05:33:40.531 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-08T05:33:40.532 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-08T05:33:40.533 INFO:teuthology.orchestra.run.smithi181.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout:Installing weak dependencies: 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction Summary 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================================ 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout:Install 133 Packages 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout:Upgrade 7 Packages 2023-10-08T05:33:40.534 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:33:40.535 INFO:teuthology.orchestra.run.smithi181.stdout:Total download size: 199 M 2023-10-08T05:33:40.535 INFO:teuthology.orchestra.run.smithi181.stdout:Downloading Packages: 2023-10-08T05:33:41.564 INFO:teuthology.orchestra.run.smithi181.stdout:(1/140): ceph-18.0.0-6609.g9725b08f.el9.x86_64. 26 kB/s | 6.5 kB 00:00 2023-10-08T05:33:41.628 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-08T05:33:41.630 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-10-08T05:33:41.649 INFO:teuthology.orchestra.run.smithi070.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-10-08T05:33:41.650 INFO:teuthology.orchestra.run.smithi070.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-10-08T05:33:41.652 INFO:teuthology.orchestra.run.smithi070.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-10-08T05:33:41.653 INFO:teuthology.orchestra.run.smithi070.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-10-08T05:33:41.656 INFO:teuthology.orchestra.run.smithi070.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-10-08T05:33:41.659 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-10-08T05:33:41.661 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-10-08T05:33:41.734 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-10-08T05:33:41.747 INFO:teuthology.orchestra.run.smithi181.stdout:(2/140): ceph-fuse-18.0.0-6609.g9725b08f.el9.x8 4.6 MB/s | 855 kB 00:00 2023-10-08T05:33:41.756 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2023-10-08T05:33:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2023-10-08T05:33:41.756 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2023-10-08T05:33:41.756 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2023-10-08T05:33:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-10-08T05:33:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 6.5 k 2023-10-08T05:33:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 5.2 M 2023-10-08T05:33:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 855 k 2023-10-08T05:33:41.756 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 141 k 2023-10-08T05:33:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 1.5 M 2023-10-08T05:33:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 132 k 2023-10-08T05:33:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 1.8 M 2023-10-08T05:33:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 7.4 M 2023-10-08T05:33:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 47 k 2023-10-08T05:33:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 8.6 M 2023-10-08T05:33:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 41 M 2023-10-08T05:33:41.757 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 262 k 2023-10-08T05:33:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 269 k 2023-10-08T05:33:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 218 k 2023-10-08T05:33:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 18 k 2023-10-08T05:33:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-10-08T05:33:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-10-08T05:33:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-10-08T05:33:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 31 k 2023-10-08T05:33:41.758 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 715 k 2023-10-08T05:33:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 126 k 2023-10-08T05:33:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-10-08T05:33:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-10-08T05:33:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-10-08T05:33:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-10-08T05:33:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-10-08T05:33:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-10-08T05:33:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 162 k 2023-10-08T05:33:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 321 k 2023-10-08T05:33:41.759 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 298 k 2023-10-08T05:33:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 100 k 2023-10-08T05:33:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 85 k 2023-10-08T05:33:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 3.0 M 2023-10-08T05:33:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 169 k 2023-10-08T05:33:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-10-08T05:33:41.760 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2023-10-08T05:33:41.760 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager x86_64 1:1.45.4-1.el9 CentOS-BaseOS 2.2 M 2023-10-08T05:33:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-libnm x86_64 1:1.45.4-1.el9 CentOS-BaseOS 1.8 M 2023-10-08T05:33:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-team x86_64 1:1.45.4-1.el9 CentOS-BaseOS 41 k 2023-10-08T05:33:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-tui x86_64 1:1.45.4-1.el9 CentOS-BaseOS 244 k 2023-10-08T05:33:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 3.3 M 2023-10-08T05:33:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 3.0 M 2023-10-08T05:33:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-10-08T05:33:41.761 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2023-10-08T05:33:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-10-08T05:33:41.761 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-10-08T05:33:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 19 M 2023-10-08T05:33:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 23 k 2023-10-08T05:33:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 2.1 M 2023-10-08T05:33:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 242 k 2023-10-08T05:33:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 4.4 M 2023-10-08T05:33:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 18 M 2023-10-08T05:33:41.762 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6609.g9725b08f.el9 ceph-noarch 15 k 2023-10-08T05:33:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 24 k 2023-10-08T05:33:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-10-08T05:33:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-10-08T05:33:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-10-08T05:33:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-10-08T05:33:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-10-08T05:33:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-10-08T05:33:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-10-08T05:33:41.763 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-10-08T05:33:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-10-08T05:33:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 164 k 2023-10-08T05:33:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-10-08T05:33:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-10-08T05:33:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-10-08T05:33:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-10-08T05:33:41.764 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-10-08T05:33:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 472 k 2023-10-08T05:33:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-10-08T05:33:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 4.8 M 2023-10-08T05:33:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-10-08T05:33:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-10-08T05:33:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-10-08T05:33:41.765 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-10-08T05:33:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-10-08T05:33:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-10-08T05:33:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-10-08T05:33:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-10-08T05:33:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-10-08T05:33:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-10-08T05:33:41.766 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-10-08T05:33:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-10-08T05:33:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 45 k 2023-10-08T05:33:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:18.0.0-6609.g9725b08f.el9 ceph 127 k 2023-10-08T05:33:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-10-08T05:33:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-10-08T05:33:41.767 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-10-08T05:33:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-10-08T05:33:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-10-08T05:33:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-10-08T05:33:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.23.0-1.el9 epel 196 k 2023-10-08T05:33:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-10-08T05:33:41.768 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-10-08T05:33:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-10-08T05:33:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-10-08T05:33:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-10-08T05:33:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-10-08T05:33:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-10-08T05:33:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-10-08T05:33:41.769 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-10-08T05:33:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-10-08T05:33:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-10-08T05:33:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-10-08T05:33:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-10-08T05:33:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-10-08T05:33:41.770 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-10-08T05:33:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-10-08T05:33:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-10-08T05:33:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-10-08T05:33:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-10-08T05:33:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-10-08T05:33:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-10-08T05:33:41.771 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-10-08T05:33:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-10-08T05:33:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-10-08T05:33:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-10-08T05:33:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-10-08T05:33:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-10-08T05:33:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-10-08T05:33:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-10-08T05:33:41.772 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-10-08T05:33:41.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-10-08T05:33:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-10-08T05:33:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-10-08T05:33:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-10-08T05:33:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-10-08T05:33:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-10-08T05:33:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-10-08T05:33:41.774 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2023-10-08T05:33:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-10-08T05:33:41.774 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:33:41.775 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2023-10-08T05:33:41.775 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2023-10-08T05:33:41.775 INFO:teuthology.orchestra.run.smithi070.stdout:Install 133 Packages 2023-10-08T05:33:41.775 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 7 Packages 2023-10-08T05:33:41.775 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:33:41.775 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 199 M 2023-10-08T05:33:41.776 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2023-10-08T05:33:41.814 INFO:teuthology.orchestra.run.smithi181.stdout:(3/140): ceph-immutable-object-cache-18.0.0-660 2.1 MB/s | 141 kB 00:00 2023-10-08T05:33:42.048 INFO:teuthology.orchestra.run.smithi181.stdout:(4/140): ceph-base-18.0.0-6609.g9725b08f.el9.x8 7.1 MB/s | 5.2 MB 00:00 2023-10-08T05:33:42.132 INFO:teuthology.orchestra.run.smithi181.stdout:(5/140): ceph-mds-18.0.0-6609.g9725b08f.el9.x86 6.6 MB/s | 2.1 MB 00:00 2023-10-08T05:33:42.215 INFO:teuthology.orchestra.run.smithi181.stdout:(6/140): ceph-mgr-18.0.0-6609.g9725b08f.el9.x86 8.8 MB/s | 1.5 MB 00:00 2023-10-08T05:33:42.482 INFO:teuthology.orchestra.run.smithi181.stdout:(7/140): ceph-mon-18.0.0-6609.g9725b08f.el9.x86 13 MB/s | 4.4 MB 00:00 2023-10-08T05:33:42.973 INFO:teuthology.orchestra.run.smithi070.stdout:(1/140): ceph-18.0.0-6609.g9725b08f.el9.x86_64. 15 kB/s | 6.5 kB 00:00 2023-10-08T05:33:43.157 INFO:teuthology.orchestra.run.smithi181.stdout:(8/140): ceph-radosgw-18.0.0-6609.g9725b08f.el9 13 MB/s | 8.6 MB 00:00 2023-10-08T05:33:43.224 INFO:teuthology.orchestra.run.smithi181.stdout:(9/140): ceph-selinux-18.0.0-6609.g9725b08f.el9 368 kB/s | 24 kB 00:00 2023-10-08T05:33:43.365 INFO:teuthology.orchestra.run.smithi070.stdout:(2/140): ceph-fuse-18.0.0-6609.g9725b08f.el9.x8 2.1 MB/s | 855 kB 00:00 2023-10-08T05:33:43.465 INFO:teuthology.orchestra.run.smithi070.stdout:(3/140): ceph-immutable-object-cache-18.0.0-660 1.4 MB/s | 141 kB 00:00 2023-10-08T05:33:44.140 INFO:teuthology.orchestra.run.smithi070.stdout:(4/140): ceph-mds-18.0.0-6609.g9725b08f.el9.x86 3.1 MB/s | 2.1 MB 00:00 2023-10-08T05:33:44.274 INFO:teuthology.orchestra.run.smithi181.stdout:(10/140): ceph-osd-18.0.0-6609.g9725b08f.el9.x8 8.6 MB/s | 18 MB 00:02 2023-10-08T05:33:44.375 INFO:teuthology.orchestra.run.smithi181.stdout:(11/140): cephfs-mirror-18.0.0-6609.g9725b08f.e 2.2 MB/s | 218 kB 00:00 2023-10-08T05:33:44.458 INFO:teuthology.orchestra.run.smithi181.stdout:(12/140): libcephfs-devel-18.0.0-6609.g9725b08f 378 kB/s | 31 kB 00:00 2023-10-08T05:33:44.465 INFO:teuthology.orchestra.run.smithi070.stdout:(5/140): ceph-mgr-18.0.0-6609.g9725b08f.el9.x86 4.5 MB/s | 1.5 MB 00:00 2023-10-08T05:33:44.625 INFO:teuthology.orchestra.run.smithi181.stdout:(13/140): libcephfs2-18.0.0-6609.g9725b08f.el9. 4.2 MB/s | 715 kB 00:00 2023-10-08T05:33:44.726 INFO:teuthology.orchestra.run.smithi181.stdout:(14/140): libcephsqlite-18.0.0-6609.g9725b08f.e 1.6 MB/s | 164 kB 00:00 2023-10-08T05:33:44.817 INFO:teuthology.orchestra.run.smithi181.stdout:(15/140): librados-devel-18.0.0-6609.g9725b08f. 1.4 MB/s | 126 kB 00:00 2023-10-08T05:33:44.993 INFO:teuthology.orchestra.run.smithi181.stdout:(16/140): libradosstriper1-18.0.0-6609.g9725b08 2.6 MB/s | 472 kB 00:00 2023-10-08T05:33:45.307 INFO:teuthology.orchestra.run.smithi070.stdout:(6/140): ceph-mon-18.0.0-6609.g9725b08f.el9.x86 5.2 MB/s | 4.4 MB 00:00 2023-10-08T05:33:45.541 INFO:teuthology.orchestra.run.smithi070.stdout:(7/140): ceph-base-18.0.0-6609.g9725b08f.el9.x8 1.7 MB/s | 5.2 MB 00:03 2023-10-08T05:33:45.801 INFO:teuthology.orchestra.run.smithi181.stdout:(17/140): librgw2-18.0.0-6609.g9725b08f.el9.x86 5.9 MB/s | 4.8 MB 00:00 2023-10-08T05:33:45.926 INFO:teuthology.orchestra.run.smithi181.stdout:(18/140): python3-ceph-argparse-18.0.0-6609.g97 367 kB/s | 45 kB 00:00 2023-10-08T05:33:46.027 INFO:teuthology.orchestra.run.smithi181.stdout:(19/140): python3-ceph-common-18.0.0-6609.g9725 1.2 MB/s | 127 kB 00:00 2023-10-08T05:33:46.127 INFO:teuthology.orchestra.run.smithi181.stdout:(20/140): python3-cephfs-18.0.0-6609.g9725b08f. 1.6 MB/s | 162 kB 00:00 2023-10-08T05:33:46.227 INFO:teuthology.orchestra.run.smithi181.stdout:(21/140): python3-rados-18.0.0-6609.g9725b08f.e 3.2 MB/s | 321 kB 00:00 2023-10-08T05:33:46.336 INFO:teuthology.orchestra.run.smithi181.stdout:(22/140): python3-rbd-18.0.0-6609.g9725b08f.el9 2.7 MB/s | 298 kB 00:00 2023-10-08T05:33:46.428 INFO:teuthology.orchestra.run.smithi181.stdout:(23/140): python3-rgw-18.0.0-6609.g9725b08f.el9 1.1 MB/s | 100 kB 00:00 2023-10-08T05:33:46.503 INFO:teuthology.orchestra.run.smithi181.stdout:(24/140): rbd-fuse-18.0.0-6609.g9725b08f.el9.x8 1.1 MB/s | 85 kB 00:00 2023-10-08T05:33:46.591 INFO:teuthology.orchestra.run.smithi070.stdout:(8/140): ceph-common-18.0.0-6609.g9725b08f.el9. 4.7 MB/s | 19 MB 00:04 2023-10-08T05:33:46.691 INFO:teuthology.orchestra.run.smithi070.stdout:(9/140): ceph-selinux-18.0.0-6609.g9725b08f.el9 243 kB/s | 24 kB 00:00 2023-10-08T05:33:47.104 INFO:teuthology.orchestra.run.smithi181.stdout:(25/140): rbd-mirror-18.0.0-6609.g9725b08f.el9. 5.0 MB/s | 3.0 MB 00:00 2023-10-08T05:33:47.171 INFO:teuthology.orchestra.run.smithi181.stdout:(26/140): rbd-nbd-18.0.0-6609.g9725b08f.el9.x86 2.5 MB/s | 169 kB 00:00 2023-10-08T05:33:47.175 INFO:teuthology.orchestra.run.smithi070.stdout:(10/140): ceph-radosgw-18.0.0-6609.g9725b08f.el 5.3 MB/s | 8.6 MB 00:01 2023-10-08T05:33:47.229 INFO:teuthology.orchestra.run.smithi181.stdout:(27/140): ceph-grafana-dashboards-18.0.0-6609.g 404 kB/s | 23 kB 00:00 2023-10-08T05:33:47.267 INFO:teuthology.orchestra.run.smithi070.stdout:(11/140): cephfs-mirror-18.0.0-6609.g9725b08f.e 2.4 MB/s | 218 kB 00:00 2023-10-08T05:33:47.296 INFO:teuthology.orchestra.run.smithi181.stdout:(28/140): ceph-mgr-cephadm-18.0.0-6609.g9725b08 2.0 MB/s | 132 kB 00:00 2023-10-08T05:33:47.342 INFO:teuthology.orchestra.run.smithi070.stdout:(12/140): libcephfs-devel-18.0.0-6609.g9725b08f 421 kB/s | 31 kB 00:00 2023-10-08T05:33:47.542 INFO:teuthology.orchestra.run.smithi070.stdout:(13/140): libcephfs2-18.0.0-6609.g9725b08f.el9. 3.5 MB/s | 715 kB 00:00 2023-10-08T05:33:47.651 INFO:teuthology.orchestra.run.smithi070.stdout:(14/140): libcephsqlite-18.0.0-6609.g9725b08f.e 1.5 MB/s | 164 kB 00:00 2023-10-08T05:33:47.663 INFO:teuthology.orchestra.run.smithi181.stdout:(29/140): ceph-mgr-dashboard-18.0.0-6609.g9725b 4.8 MB/s | 1.8 MB 00:00 2023-10-08T05:33:47.726 INFO:teuthology.orchestra.run.smithi070.stdout:(15/140): librados-devel-18.0.0-6609.g9725b08f. 1.7 MB/s | 126 kB 00:00 2023-10-08T05:33:47.818 INFO:teuthology.orchestra.run.smithi070.stdout:(16/140): libradosstriper1-18.0.0-6609.g9725b08 5.1 MB/s | 472 kB 00:00 2023-10-08T05:33:48.105 INFO:teuthology.orchestra.run.smithi181.stdout:(30/140): ceph-test-18.0.0-6609.g9725b08f.el9.x 8.4 MB/s | 41 MB 00:04 2023-10-08T05:33:48.205 INFO:teuthology.orchestra.run.smithi181.stdout:(31/140): ceph-mgr-modules-core-18.0.0-6609.g97 2.4 MB/s | 242 kB 00:00 2023-10-08T05:33:48.272 INFO:teuthology.orchestra.run.smithi181.stdout:(32/140): ceph-mgr-rook-18.0.0-6609.g9725b08f.e 725 kB/s | 47 kB 00:00 2023-10-08T05:33:48.339 INFO:teuthology.orchestra.run.smithi181.stdout:(33/140): ceph-prometheus-alerts-18.0.0-6609.g9 223 kB/s | 15 kB 00:00 2023-10-08T05:33:48.351 INFO:teuthology.orchestra.run.smithi070.stdout:(17/140): ceph-osd-18.0.0-6609.g9725b08f.el9.x8 5.8 MB/s | 18 MB 00:03 2023-10-08T05:33:48.406 INFO:teuthology.orchestra.run.smithi181.stdout:(34/140): ceph-volume-18.0.0-6609.g9725b08f.el9 3.9 MB/s | 262 kB 00:00 2023-10-08T05:33:48.435 INFO:teuthology.orchestra.run.smithi070.stdout:(18/140): python3-ceph-argparse-18.0.0-6609.g97 550 kB/s | 45 kB 00:00 2023-10-08T05:33:48.473 INFO:teuthology.orchestra.run.smithi181.stdout:(35/140): cephadm-18.0.0-6609.g9725b08f.el9.noa 4.0 MB/s | 269 kB 00:00 2023-10-08T05:33:48.527 INFO:teuthology.orchestra.run.smithi070.stdout:(19/140): librgw2-18.0.0-6609.g9725b08f.el9.x86 6.7 MB/s | 4.8 MB 00:00 2023-10-08T05:33:48.560 INFO:teuthology.orchestra.run.smithi070.stdout:(20/140): python3-ceph-common-18.0.0-6609.g9725 1.0 MB/s | 127 kB 00:00 2023-10-08T05:33:48.594 INFO:teuthology.orchestra.run.smithi070.stdout:(21/140): python3-cephfs-18.0.0-6609.g9725b08f. 2.4 MB/s | 162 kB 00:00 2023-10-08T05:33:48.636 INFO:teuthology.orchestra.run.smithi070.stdout:(22/140): python3-rados-18.0.0-6609.g9725b08f.e 4.2 MB/s | 321 kB 00:00 2023-10-08T05:33:48.678 INFO:teuthology.orchestra.run.smithi070.stdout:(23/140): python3-rbd-18.0.0-6609.g9725b08f.el9 3.5 MB/s | 298 kB 00:00 2023-10-08T05:33:48.740 INFO:teuthology.orchestra.run.smithi181.stdout:(36/140): ceph-common-18.0.0-6609.g9725b08f.el9 2.6 MB/s | 19 MB 00:07 2023-10-08T05:33:48.744 INFO:teuthology.orchestra.run.smithi070.stdout:(24/140): python3-rgw-18.0.0-6609.g9725b08f.el9 926 kB/s | 100 kB 00:00 2023-10-08T05:33:48.765 INFO:teuthology.orchestra.run.smithi181.stdout:(37/140): cephfs-top-18.0.0-6609.g9725b08f.el9. 63 kB/s | 18 kB 00:00 2023-10-08T05:33:48.770 INFO:teuthology.orchestra.run.smithi070.stdout:(25/140): rbd-fuse-18.0.0-6609.g9725b08f.el9.x8 933 kB/s | 85 kB 00:00 2023-10-08T05:33:48.846 INFO:teuthology.orchestra.run.smithi070.stdout:(26/140): rbd-nbd-18.0.0-6609.g9725b08f.el9.x86 2.2 MB/s | 169 kB 00:00 2023-10-08T05:33:48.921 INFO:teuthology.orchestra.run.smithi070.stdout:(27/140): ceph-grafana-dashboards-18.0.0-6609.g 313 kB/s | 23 kB 00:00 2023-10-08T05:33:48.996 INFO:teuthology.orchestra.run.smithi070.stdout:(28/140): ceph-mgr-cephadm-18.0.0-6609.g9725b08 1.7 MB/s | 132 kB 00:00 2023-10-08T05:33:49.007 INFO:teuthology.orchestra.run.smithi181.stdout:(38/140): ceph-mgr-diskprediction-local-18.0.0- 5.5 MB/s | 7.4 MB 00:01 2023-10-08T05:33:49.049 INFO:teuthology.orchestra.run.smithi181.stdout:(39/140): boost-program-options-1.75.0-8.el9.x8 374 kB/s | 106 kB 00:00 2023-10-08T05:33:49.080 INFO:teuthology.orchestra.run.smithi070.stdout:(29/140): rbd-mirror-18.0.0-6609.g9725b08f.el9. 9.0 MB/s | 3.0 MB 00:00 2023-10-08T05:33:49.199 INFO:teuthology.orchestra.run.smithi181.stdout:(40/140): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 920 kB 00:00 2023-10-08T05:33:49.238 INFO:teuthology.orchestra.run.smithi070.stdout:(30/140): ceph-mgr-dashboard-18.0.0-6609.g9725b 7.3 MB/s | 1.8 MB 00:00 2023-10-08T05:33:49.241 INFO:teuthology.orchestra.run.smithi181.stdout:(41/140): flex-2.6.4-9.el9.x86_64.rpm 1.3 MB/s | 307 kB 00:00 2023-10-08T05:33:49.267 INFO:teuthology.orchestra.run.smithi181.stdout:(42/140): flexiblas-3.0.4-8.el9.x86_64.rpm 147 kB/s | 32 kB 00:00 2023-10-08T05:33:49.313 INFO:teuthology.orchestra.run.smithi070.stdout:(31/140): ceph-mgr-modules-core-18.0.0-6609.g97 3.2 MB/s | 242 kB 00:00 2023-10-08T05:33:49.389 INFO:teuthology.orchestra.run.smithi070.stdout:(32/140): ceph-mgr-rook-18.0.0-6609.g9725b08f.e 642 kB/s | 47 kB 00:00 2023-10-08T05:33:49.392 INFO:teuthology.orchestra.run.smithi181.stdout:(43/140): flexiblas-openblas-openmp-3.0.4-8.el9 118 kB/s | 17 kB 00:00 2023-10-08T05:33:49.456 INFO:teuthology.orchestra.run.smithi070.stdout:(33/140): ceph-prometheus-alerts-18.0.0-6609.g9 223 kB/s | 15 kB 00:00 2023-10-08T05:33:49.467 INFO:teuthology.orchestra.run.smithi181.stdout:(44/140): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2023-10-08T05:33:49.484 INFO:teuthology.orchestra.run.smithi181.stdout:(45/140): libaio-devel-0.3.111-13.el9.x86_64.rp 47 kB/s | 10 kB 00:00 2023-10-08T05:33:49.518 INFO:teuthology.orchestra.run.smithi181.stdout:(46/140): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2023-10-08T05:33:49.547 INFO:teuthology.orchestra.run.smithi070.stdout:(34/140): ceph-volume-18.0.0-6609.g9725b08f.el9 2.8 MB/s | 262 kB 00:00 2023-10-08T05:33:49.568 INFO:teuthology.orchestra.run.smithi181.stdout:(47/140): librdkafka-1.6.1-102.el9.x86_64.rpm 8.0 MB/s | 661 kB 00:00 2023-10-08T05:33:49.602 INFO:teuthology.orchestra.run.smithi181.stdout:(48/140): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 578 kB 00:00 2023-10-08T05:33:49.627 INFO:teuthology.orchestra.run.smithi181.stdout:(49/140): libzstd-devel-1.5.1-2.el9.x86_64.rpm 433 kB/s | 46 kB 00:00 2023-10-08T05:33:49.657 INFO:teuthology.orchestra.run.smithi070.stdout:(35/140): cephadm-18.0.0-6609.g9725b08f.el9.noa 2.4 MB/s | 269 kB 00:00 2023-10-08T05:33:49.660 INFO:teuthology.orchestra.run.smithi181.stdout:(50/140): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 282 kB 00:00 2023-10-08T05:33:49.686 INFO:teuthology.orchestra.run.smithi181.stdout:(51/140): openblas-0.3.21-2.el9.x86_64.rpm 626 kB/s | 34 kB 00:00 2023-10-08T05:33:49.723 INFO:teuthology.orchestra.run.smithi070.stdout:(36/140): cephfs-top-18.0.0-6609.g9725b08f.el9. 280 kB/s | 18 kB 00:00 2023-10-08T05:33:49.728 INFO:teuthology.orchestra.run.smithi181.stdout:(52/140): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.2 MB/s | 217 kB 00:00 2023-10-08T05:33:49.878 INFO:teuthology.orchestra.run.smithi181.stdout:(53/140): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2023-10-08T05:33:49.907 INFO:teuthology.orchestra.run.smithi070.stdout:(37/140): ceph-mgr-diskprediction-local-18.0.0- 9.0 MB/s | 7.4 MB 00:00 2023-10-08T05:33:49.945 INFO:teuthology.orchestra.run.smithi181.stdout:(54/140): postgresql-13.11-1.el9.x86_64.rpm 6.3 MB/s | 1.5 MB 00:00 2023-10-08T05:33:49.966 INFO:teuthology.orchestra.run.smithi070.stdout:(38/140): boost-program-options-1.75.0-8.el9.x8 1.9 MB/s | 106 kB 00:00 2023-10-08T05:33:49.978 INFO:teuthology.orchestra.run.smithi181.stdout:(55/140): postgresql-contrib-13.11-1.el9.x86_64 3.4 MB/s | 811 kB 00:00 2023-10-08T05:33:50.004 INFO:teuthology.orchestra.run.smithi181.stdout:(56/140): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2023-10-08T05:33:50.024 INFO:teuthology.orchestra.run.smithi070.stdout:(39/140): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 920 kB 00:00 2023-10-08T05:33:50.058 INFO:teuthology.orchestra.run.smithi070.stdout:(40/140): flex-2.6.4-9.el9.x86_64.rpm 3.4 MB/s | 307 kB 00:00 2023-10-08T05:33:50.075 INFO:teuthology.orchestra.run.smithi070.stdout:(41/140): flexiblas-3.0.4-8.el9.x86_64.rpm 684 kB/s | 32 kB 00:00 2023-10-08T05:33:50.183 INFO:teuthology.orchestra.run.smithi070.stdout:(42/140): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2023-10-08T05:33:50.200 INFO:teuthology.orchestra.run.smithi070.stdout:(43/140): flexiblas-openblas-openmp-3.0.4-8.el9 155 kB/s | 17 kB 00:00 2023-10-08T05:33:50.234 INFO:teuthology.orchestra.run.smithi070.stdout:(44/140): libaio-devel-0.3.111-13.el9.x86_64.rp 314 kB/s | 10 kB 00:00 2023-10-08T05:33:50.246 INFO:teuthology.orchestra.run.smithi181.stdout:(57/140): postgresql-server-13.11-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2023-10-08T05:33:50.251 INFO:teuthology.orchestra.run.smithi070.stdout:(45/140): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2023-10-08T05:33:50.309 INFO:teuthology.orchestra.run.smithi070.stdout:(46/140): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2023-10-08T05:33:50.343 INFO:teuthology.orchestra.run.smithi070.stdout:(47/140): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 578 kB 00:00 2023-10-08T05:33:50.346 INFO:teuthology.orchestra.run.smithi181.stdout:(58/140): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2023-10-08T05:33:50.371 INFO:teuthology.orchestra.run.smithi181.stdout:(59/140): python3-devel-3.9.17-2.el9.x86_64.rpm 584 kB/s | 205 kB 00:00 2023-10-08T05:33:50.376 INFO:teuthology.orchestra.run.smithi070.stdout:(48/140): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2023-10-08T05:33:50.401 INFO:teuthology.orchestra.run.smithi070.stdout:(49/140): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-10-08T05:33:50.413 INFO:teuthology.orchestra.run.smithi181.stdout:(60/140): python3-jsonpatch-1.21-16.el9.noarch. 611 kB/s | 24 kB 00:00 2023-10-08T05:33:50.435 INFO:teuthology.orchestra.run.smithi070.stdout:(50/140): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2023-10-08T05:33:50.438 INFO:teuthology.orchestra.run.smithi181.stdout:(61/140): python3-jsonpointer-2.0-4.el9.noarch. 278 kB/s | 18 kB 00:00 2023-10-08T05:33:50.460 INFO:teuthology.orchestra.run.smithi070.stdout:(51/140): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.7 MB/s | 217 kB 00:00 2023-10-08T05:33:50.472 INFO:teuthology.orchestra.run.smithi181.stdout:(62/140): python3-jinja2-2.11.3-4.el9.noarch.rp 2.3 MB/s | 228 kB 00:00 2023-10-08T05:33:50.497 INFO:teuthology.orchestra.run.smithi181.stdout:(63/140): python3-markupsafe-1.1.1-12.el9.x86_6 570 kB/s | 32 kB 00:00 2023-10-08T05:33:50.539 INFO:teuthology.orchestra.run.smithi181.stdout:(64/140): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 144 kB 00:00 2023-10-08T05:33:50.602 INFO:teuthology.orchestra.run.smithi070.stdout:(52/140): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2023-10-08T05:33:50.653 INFO:teuthology.orchestra.run.smithi070.stdout:(53/140): postgresql-13.11-1.el9.x86_64.rpm 8.7 MB/s | 1.5 MB 00:00 2023-10-08T05:33:50.672 INFO:teuthology.orchestra.run.smithi181.stdout:(65/140): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2023-10-08T05:33:50.695 INFO:teuthology.orchestra.run.smithi070.stdout:(54/140): postgresql-contrib-13.11-1.el9.x86_64 9.1 MB/s | 811 kB 00:00 2023-10-08T05:33:50.697 INFO:teuthology.orchestra.run.smithi181.stdout:(66/140): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2023-10-08T05:33:50.719 INFO:teuthology.orchestra.run.smithi070.stdout:(55/140): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2023-10-08T05:33:50.739 INFO:teuthology.orchestra.run.smithi181.stdout:(67/140): python3-oauthlib-3.1.1-5.el9.noarch.r 979 kB/s | 174 kB 00:00 2023-10-08T05:33:50.790 INFO:teuthology.orchestra.run.smithi181.stdout:(68/140): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 41 kB 00:00 2023-10-08T05:33:50.815 INFO:teuthology.orchestra.run.smithi181.stdout:(69/140): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2023-10-08T05:33:50.849 INFO:teuthology.orchestra.run.smithi181.stdout:(70/140): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 211 kB 00:00 2023-10-08T05:33:50.896 INFO:teuthology.orchestra.run.smithi070.stdout:(56/140): postgresql-server-13.11-1.el9.x86_64. 29 MB/s | 5.7 MB 00:00 2023-10-08T05:33:50.899 INFO:teuthology.orchestra.run.smithi181.stdout:(71/140): python3-requests-oauthlib-1.3.0-12.el 530 kB/s | 43 kB 00:00 2023-10-08T05:33:51.062 INFO:teuthology.orchestra.run.smithi070.stdout:(57/140): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2023-10-08T05:33:51.087 INFO:teuthology.orchestra.run.smithi070.stdout:(58/140): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 205 kB 00:00 2023-10-08T05:33:51.129 INFO:teuthology.orchestra.run.smithi070.stdout:(59/140): python3-jsonpatch-1.21-16.el9.noarch. 608 kB/s | 24 kB 00:00 2023-10-08T05:33:51.163 INFO:teuthology.orchestra.run.smithi070.stdout:(60/140): python3-jinja2-2.11.3-4.el9.noarch.rp 3.0 MB/s | 228 kB 00:00 2023-10-08T05:33:51.183 INFO:teuthology.orchestra.run.smithi181.stdout:(72/140): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2023-10-08T05:33:51.188 INFO:teuthology.orchestra.run.smithi070.stdout:(61/140): python3-jsonpointer-2.0-4.el9.noarch. 325 kB/s | 18 kB 00:00 2023-10-08T05:33:51.208 INFO:teuthology.orchestra.run.smithi181.stdout:(73/140): python3-toml-0.10.2-6.el9.noarch.rpm 108 kB/s | 39 kB 00:00 2023-10-08T05:33:51.221 INFO:teuthology.orchestra.run.smithi070.stdout:(62/140): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2023-10-08T05:33:51.233 INFO:teuthology.orchestra.run.smithi181.stdout:(74/140): socat-1.7.4.1-5.el9.x86_64.rpm 951 kB/s | 300 kB 00:00 2023-10-08T05:33:51.247 INFO:teuthology.orchestra.run.smithi070.stdout:(63/140): python3-markupsafe-1.1.1-12.el9.x86_6 561 kB/s | 32 kB 00:00 2023-10-08T05:33:51.275 INFO:teuthology.orchestra.run.smithi181.stdout:(75/140): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 56 kB 00:00 2023-10-08T05:33:51.301 INFO:teuthology.orchestra.run.smithi181.stdout:(76/140): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2023-10-08T05:33:51.325 INFO:teuthology.orchestra.run.smithi181.stdout:(77/140): xmlstarlet-1.6.1-20.el9.x86_64.rpm 692 kB/s | 63 kB 00:00 2023-10-08T05:33:51.359 INFO:teuthology.orchestra.run.smithi181.stdout:(78/140): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2023-10-08T05:33:51.380 INFO:teuthology.orchestra.run.smithi070.stdout:(64/140): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2023-10-08T05:33:51.401 INFO:teuthology.orchestra.run.smithi181.stdout:(79/140): libgfortran-11.4.1-2.3.el9.x86_64.rpm 7.9 MB/s | 803 kB 00:00 2023-10-08T05:33:51.405 INFO:teuthology.orchestra.run.smithi070.stdout:(65/140): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2023-10-08T05:33:51.426 INFO:teuthology.orchestra.run.smithi181.stdout:(80/140): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.4 MB/s | 241 kB 00:00 2023-10-08T05:33:51.448 INFO:teuthology.orchestra.run.smithi070.stdout:(66/140): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2023-10-08T05:33:51.451 INFO:teuthology.orchestra.run.smithi181.stdout:(81/140): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2023-10-08T05:33:51.485 INFO:teuthology.orchestra.run.smithi181.stdout:(82/140): python3-requests-2.25.1-7.el9.noarch. 2.0 MB/s | 113 kB 00:00 2023-10-08T05:33:51.503 INFO:teuthology.orchestra.run.smithi070.stdout:(67/140): python3-oauthlib-3.1.1-5.el9.noarch.r 1.8 MB/s | 174 kB 00:00 2023-10-08T05:33:51.519 INFO:teuthology.orchestra.run.smithi181.stdout:(83/140): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2023-10-08T05:33:51.540 INFO:teuthology.orchestra.run.smithi070.stdout:(68/140): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 132 kB 00:00 2023-10-08T05:33:51.544 INFO:teuthology.orchestra.run.smithi181.stdout:(84/140): python3-urllib3-1.26.5-3.el9.noarch.r 2.1 MB/s | 188 kB 00:00 2023-10-08T05:33:51.573 INFO:teuthology.orchestra.run.smithi070.stdout:(69/140): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 211 kB 00:00 2023-10-08T05:33:51.599 INFO:teuthology.orchestra.run.smithi070.stdout:(70/140): python3-requests-oauthlib-1.3.0-12.el 770 kB/s | 43 kB 00:00 2023-10-08T05:33:51.686 INFO:teuthology.orchestra.run.smithi181.stdout:(85/140): elfutils-libelf-devel-0.189-3.el9.x86 159 kB/s | 22 kB 00:00 2023-10-08T05:33:51.766 INFO:teuthology.orchestra.run.smithi070.stdout:(71/140): python3-toml-0.10.2-6.el9.noarch.rpm 262 kB/s | 39 kB 00:00 2023-10-08T05:33:51.811 INFO:teuthology.orchestra.run.smithi181.stdout:(86/140): jq-1.6-15.el9.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2023-10-08T05:33:51.853 INFO:teuthology.orchestra.run.smithi181.stdout:(87/140): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2023-10-08T05:33:51.870 INFO:teuthology.orchestra.run.smithi181.stdout:(88/140): librabbitmq-0.11.0-7.el9.x86_64.rpm 785 kB/s | 45 kB 00:00 2023-10-08T05:33:51.904 INFO:teuthology.orchestra.run.smithi181.stdout:(89/140): libuuid-devel-2.37.4-15.el9.x86_64.rp 518 kB/s | 26 kB 00:00 2023-10-08T05:33:51.962 INFO:teuthology.orchestra.run.smithi181.stdout:(90/140): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2023-10-08T05:33:51.966 INFO:teuthology.orchestra.run.smithi070.stdout:(72/140): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2023-10-08T05:33:51.988 INFO:teuthology.orchestra.run.smithi181.stdout:(91/140): python3-pytz-2021.1-5.el9.noarch.rpm 616 kB/s | 51 kB 00:00 2023-10-08T05:33:51.999 INFO:teuthology.orchestra.run.smithi070.stdout:(73/140): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2023-10-08T05:33:52.025 INFO:teuthology.orchestra.run.smithi070.stdout:(74/140): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2023-10-08T05:33:52.046 INFO:teuthology.orchestra.run.smithi181.stdout:(92/140): zlib-devel-1.2.11-40.el9.x86_64.rpm 784 kB/s | 45 kB 00:00 2023-10-08T05:33:52.050 INFO:teuthology.orchestra.run.smithi070.stdout:(75/140): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2023-10-08T05:33:52.067 INFO:teuthology.orchestra.run.smithi070.stdout:(76/140): fuse-2.9.9-15.el9.x86_64.rpm 1.9 MB/s | 79 kB 00:00 2023-10-08T05:33:52.117 INFO:teuthology.orchestra.run.smithi070.stdout:(77/140): libgfortran-11.4.1-2.3.el9.x86_64.rpm 12 MB/s | 803 kB 00:00 2023-10-08T05:33:52.130 INFO:teuthology.orchestra.run.smithi181.stdout:(93/140): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 519 kB/s | 86 kB 00:00 2023-10-08T05:33:52.151 INFO:teuthology.orchestra.run.smithi070.stdout:(78/140): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2023-10-08T05:33:52.185 INFO:teuthology.orchestra.run.smithi070.stdout:(79/140): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 241 kB 00:00 2023-10-08T05:33:52.210 INFO:teuthology.orchestra.run.smithi070.stdout:(80/140): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2023-10-08T05:33:52.227 INFO:teuthology.orchestra.run.smithi070.stdout:(81/140): python3-pycparser-2.20-6.el9.noarch.r 3.0 MB/s | 124 kB 00:00 2023-10-08T05:33:52.252 INFO:teuthology.orchestra.run.smithi070.stdout:(82/140): python3-requests-2.25.1-7.el9.noarch. 2.8 MB/s | 113 kB 00:00 2023-10-08T05:33:52.277 INFO:teuthology.orchestra.run.smithi070.stdout:(83/140): python3-urllib3-1.26.5-3.el9.noarch.r 3.8 MB/s | 188 kB 00:00 2023-10-08T05:33:52.405 INFO:teuthology.orchestra.run.smithi181.stdout:(94/140): openssl-devel-3.0.7-24.el9.x86_64.rpm 7.7 MB/s | 4.1 MB 00:00 2023-10-08T05:33:52.472 INFO:teuthology.orchestra.run.smithi181.stdout:(95/140): fmt-8.1.1-5.el9.x86_64.rpm 261 kB/s | 111 kB 00:00 2023-10-08T05:33:52.547 INFO:teuthology.orchestra.run.smithi181.stdout:(96/140): gperftools-libs-2.9.1-2.el9.x86_64.rp 742 kB/s | 309 kB 00:00 2023-10-08T05:33:52.589 INFO:teuthology.orchestra.run.smithi181.stdout:(97/140): libarrow-doc-9.0.0-9.el9.noarch.rpm 209 kB/s | 24 kB 00:00 2023-10-08T05:33:52.594 INFO:teuthology.orchestra.run.smithi070.stdout:(84/140): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2023-10-08T05:33:52.645 INFO:teuthology.orchestra.run.smithi070.stdout:(85/140): elfutils-libelf-devel-0.189-3.el9.x86 454 kB/s | 22 kB 00:00 2023-10-08T05:33:52.665 INFO:teuthology.orchestra.run.smithi181.stdout:(98/140): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2023-10-08T05:33:52.690 INFO:teuthology.orchestra.run.smithi181.stdout:(99/140): libunwind-1.6.2-1.el9.x86_64.rpm 676 kB/s | 67 kB 00:00 2023-10-08T05:33:52.729 INFO:teuthology.orchestra.run.smithi070.stdout:(86/140): jq-1.6-15.el9.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2023-10-08T05:33:52.786 INFO:teuthology.orchestra.run.smithi070.stdout:(87/140): librabbitmq-0.11.0-7.el9.x86_64.rpm 793 kB/s | 45 kB 00:00 2023-10-08T05:33:52.836 INFO:teuthology.orchestra.run.smithi070.stdout:(88/140): libuuid-devel-2.37.4-15.el9.x86_64.rp 519 kB/s | 26 kB 00:00 2023-10-08T05:33:52.915 INFO:teuthology.orchestra.run.smithi181.stdout:(100/140): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.3 MB/s | 837 kB 00:00 2023-10-08T05:33:52.949 INFO:teuthology.orchestra.run.smithi181.stdout:(101/140): python3-asyncssh-2.13.2-3.el9.noarch 2.1 MB/s | 548 kB 00:00 2023-10-08T05:33:53.049 INFO:teuthology.orchestra.run.smithi181.stdout:(102/140): python3-bcrypt-3.2.2-1.el9.x86_64.rp 327 kB/s | 43 kB 00:00 2023-10-08T05:33:53.053 INFO:teuthology.orchestra.run.smithi070.stdout:(89/140): python3-cryptography-36.0.1-4.el9.x86 1.5 MB/s | 1.2 MB 00:00 2023-10-08T05:33:53.074 INFO:teuthology.orchestra.run.smithi181.stdout:(103/140): python3-cachetools-4.2.4-1.el9.noarc 258 kB/s | 32 kB 00:00 2023-10-08T05:33:53.183 INFO:teuthology.orchestra.run.smithi181.stdout:(104/140): libarrow-9.0.0-9.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2023-10-08T05:33:53.203 INFO:teuthology.orchestra.run.smithi070.stdout:(90/140): python3-pytz-2021.1-5.el9.noarch.rpm 342 kB/s | 51 kB 00:00 2023-10-08T05:33:53.208 INFO:teuthology.orchestra.run.smithi181.stdout:(105/140): python3-certifi-2021.10.8-2.el9.noar 95 kB/s | 15 kB 00:00 2023-10-08T05:33:53.242 INFO:teuthology.orchestra.run.smithi181.stdout:(106/140): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2023-10-08T05:33:53.295 INFO:teuthology.orchestra.run.smithi070.stdout:(91/140): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 950 kB/s | 86 kB 00:00 2023-10-08T05:33:53.309 INFO:teuthology.orchestra.run.smithi181.stdout:(107/140): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2023-10-08T05:33:53.342 INFO:teuthology.orchestra.run.smithi181.stdout:(108/140): python3-google-auth-2.23.0-1.el9.noa 1.4 MB/s | 196 kB 00:00 2023-10-08T05:33:53.354 INFO:teuthology.orchestra.run.smithi070.stdout:(92/140): zlib-devel-1.2.11-40.el9.x86_64.rpm 785 kB/s | 45 kB 00:00 2023-10-08T05:33:53.368 INFO:teuthology.orchestra.run.smithi181.stdout:(109/140): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2023-10-08T05:33:53.418 INFO:teuthology.orchestra.run.smithi181.stdout:(110/140): python3-jaraco-classes-3.2.1-5.el9.n 164 kB/s | 18 kB 00:00 2023-10-08T05:33:53.460 INFO:teuthology.orchestra.run.smithi181.stdout:(111/140): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2023-10-08T05:33:53.485 INFO:teuthology.orchestra.run.smithi181.stdout:(112/140): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2023-10-08T05:33:53.552 INFO:teuthology.orchestra.run.smithi181.stdout:(113/140): python3-jaraco-text-3.2.0-6.el9.noar 147 kB/s | 20 kB 00:00 2023-10-08T05:33:53.577 INFO:teuthology.orchestra.run.smithi181.stdout:(114/140): python3-jwt+crypto-2.4.0-1.el9.noarc 78 kB/s | 9.0 kB 00:00 2023-10-08T05:33:53.628 INFO:teuthology.orchestra.run.smithi181.stdout:(115/140): python3-jwt-2.4.0-1.el9.noarch.rpm 289 kB/s | 41 kB 00:00 2023-10-08T05:33:53.695 INFO:teuthology.orchestra.run.smithi181.stdout:(116/140): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2023-10-08T05:33:53.728 INFO:teuthology.orchestra.run.smithi181.stdout:(117/140): python3-more-itertools-8.12.0-2.el9. 791 kB/s | 79 kB 00:00 2023-10-08T05:33:53.770 INFO:teuthology.orchestra.run.smithi181.stdout:(118/140): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2023-10-08T05:33:53.812 INFO:teuthology.orchestra.run.smithi181.stdout:(119/140): python3-natsort-7.1.1-5.el9.noarch.r 496 kB/s | 58 kB 00:00 2023-10-08T05:33:53.862 INFO:teuthology.orchestra.run.smithi181.stdout:(120/140): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2023-10-08T05:33:53.890 INFO:teuthology.orchestra.run.smithi181.stdout:(121/140): python3-portend-3.1.0-2.el9.noarch.r 138 kB/s | 16 kB 00:00 2023-10-08T05:33:53.916 INFO:teuthology.orchestra.run.smithi181.stdout:(122/140): python3-pyOpenSSL-21.0.0-1.el9.noarc 878 kB/s | 90 kB 00:00 2023-10-08T05:33:53.963 INFO:teuthology.orchestra.run.smithi070.stdout:(93/140): ceph-test-18.0.0-6609.g9725b08f.el9.x 5.6 MB/s | 41 MB 00:07 2023-10-08T05:33:53.974 INFO:teuthology.orchestra.run.smithi181.stdout:(123/140): python3-repoze-lru-0.7-16.el9.noarch 277 kB/s | 31 kB 00:00 2023-10-08T05:33:54.025 INFO:teuthology.orchestra.run.smithi181.stdout:(124/140): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2023-10-08T05:33:54.066 INFO:teuthology.orchestra.run.smithi181.stdout:(125/140): python3-rsa-4.9-2.el9.noarch.rpm 393 kB/s | 59 kB 00:00 2023-10-08T05:33:54.080 INFO:teuthology.orchestra.run.smithi070.stdout:(94/140): fmt-8.1.1-5.el9.x86_64.rpm 153 kB/s | 111 kB 00:00 2023-10-08T05:33:54.092 INFO:teuthology.orchestra.run.smithi181.stdout:(126/140): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2023-10-08T05:33:54.150 INFO:teuthology.orchestra.run.smithi181.stdout:(127/140): python3-typing-extensions-4.4.0-2.el 408 kB/s | 51 kB 00:00 2023-10-08T05:33:54.192 INFO:teuthology.orchestra.run.smithi181.stdout:(128/140): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2023-10-08T05:33:54.218 INFO:teuthology.orchestra.run.smithi181.stdout:(129/140): python3-websocket-client-1.2.3-2.el9 718 kB/s | 90 kB 00:00 2023-10-08T05:33:54.259 INFO:teuthology.orchestra.run.smithi181.stdout:(130/140): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2023-10-08T05:33:54.285 INFO:teuthology.orchestra.run.smithi181.stdout:(131/140): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2023-10-08T05:33:54.296 INFO:teuthology.orchestra.run.smithi070.stdout:(95/140): openssl-devel-3.0.7-24.el9.x86_64.rpm 2.8 MB/s | 4.1 MB 00:01 2023-10-08T05:33:54.338 INFO:teuthology.orchestra.run.smithi070.stdout:(96/140): gperftools-libs-2.9.1-2.el9.x86_64.rp 825 kB/s | 309 kB 00:00 2023-10-08T05:33:54.477 INFO:teuthology.orchestra.run.smithi181.stdout:(132/140): re2-20211101-3.el9.x86_64.rpm 742 kB/s | 192 kB 00:00 2023-10-08T05:33:54.480 INFO:teuthology.orchestra.run.smithi070.stdout:(97/140): libarrow-doc-9.0.0-9.el9.noarch.rpm 133 kB/s | 24 kB 00:00 2023-10-08T05:33:54.505 INFO:teuthology.orchestra.run.smithi070.stdout:(98/140): liboath-2.6.7-2.el9.x86_64.rpm 292 kB/s | 49 kB 00:00 2023-10-08T05:33:54.606 INFO:teuthology.orchestra.run.smithi070.stdout:(99/140): libarrow-9.0.0-9.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2023-10-08T05:33:54.635 INFO:teuthology.orchestra.run.smithi181.stdout:(133/140): librados2-18.0.0-6609.g9725b08f.el9. 9.5 MB/s | 3.3 MB 00:00 2023-10-08T05:33:54.639 INFO:teuthology.orchestra.run.smithi070.stdout:(100/140): libunwind-1.6.2-1.el9.x86_64.rpm 427 kB/s | 67 kB 00:00 2023-10-08T05:33:54.681 INFO:teuthology.orchestra.run.smithi070.stdout:(101/140): parquet-libs-9.0.0-9.el9.x86_64.rpm 4.7 MB/s | 837 kB 00:00 2023-10-08T05:33:54.719 INFO:teuthology.orchestra.run.smithi181.stdout:(134/140): thrift-0.15.0-2.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2023-10-08T05:33:54.740 INFO:teuthology.orchestra.run.smithi070.stdout:(102/140): python3-bcrypt-3.2.2-1.el9.x86_64.rp 437 kB/s | 43 kB 00:00 2023-10-08T05:33:54.766 INFO:teuthology.orchestra.run.smithi070.stdout:(103/140): python3-cachetools-4.2.4-1.el9.noarc 388 kB/s | 32 kB 00:00 2023-10-08T05:33:54.787 INFO:teuthology.orchestra.run.smithi181.stdout:(135/140): NetworkManager-1.45.4-1.el9.x86_64.r 15 MB/s | 2.2 MB 00:00 2023-10-08T05:33:54.808 INFO:teuthology.orchestra.run.smithi070.stdout:(104/140): python3-asyncssh-2.13.2-3.el9.noarch 2.7 MB/s | 548 kB 00:00 2023-10-08T05:33:54.833 INFO:teuthology.orchestra.run.smithi070.stdout:(105/140): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2023-10-08T05:33:54.861 INFO:teuthology.orchestra.run.smithi181.stdout:(136/140): NetworkManager-libnm-1.45.4-1.el9.x8 13 MB/s | 1.8 MB 00:00 2023-10-08T05:33:54.875 INFO:teuthology.orchestra.run.smithi070.stdout:(106/140): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2023-10-08T05:33:54.887 INFO:teuthology.orchestra.run.smithi181.stdout:(137/140): NetworkManager-team-1.45.4-1.el9.x86 489 kB/s | 41 kB 00:00 2023-10-08T05:33:54.920 INFO:teuthology.orchestra.run.smithi181.stdout:(138/140): NetworkManager-tui-1.45.4-1.el9.x86_ 4.2 MB/s | 244 kB 00:00 2023-10-08T05:33:54.933 INFO:teuthology.orchestra.run.smithi070.stdout:(107/140): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2023-10-08T05:33:54.954 INFO:teuthology.orchestra.run.smithi181.stdout:(139/140): xfsdump-3.1.12-4.el9.x86_64.rpm 4.9 MB/s | 321 kB 00:00 2023-10-08T05:33:54.958 INFO:teuthology.orchestra.run.smithi070.stdout:(108/140): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2023-10-08T05:33:54.992 INFO:teuthology.orchestra.run.smithi070.stdout:(109/140): python3-google-auth-2.23.0-1.el9.noa 1.2 MB/s | 196 kB 00:00 2023-10-08T05:33:55.034 INFO:teuthology.orchestra.run.smithi070.stdout:(110/140): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2023-10-08T05:33:55.054 INFO:teuthology.orchestra.run.smithi181.stdout:(140/140): librbd1-18.0.0-6609.g9725b08f.el9.x8 5.3 MB/s | 3.0 MB 00:00 2023-10-08T05:33:55.059 INFO:teuthology.orchestra.run.smithi070.stdout:(111/140): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2023-10-08T05:33:55.062 INFO:teuthology.orchestra.run.smithi181.stdout:-------------------------------------------------------------------------------- 2023-10-08T05:33:55.062 INFO:teuthology.orchestra.run.smithi181.stdout:Total 14 MB/s | 199 MB 00:14 2023-10-08T05:33:55.084 INFO:teuthology.orchestra.run.smithi070.stdout:(112/140): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2023-10-08T05:33:55.143 INFO:teuthology.orchestra.run.smithi070.stdout:(113/140): python3-jwt+crypto-2.4.0-1.el9.noarc 110 kB/s | 9.0 kB 00:00 2023-10-08T05:33:55.168 INFO:teuthology.orchestra.run.smithi070.stdout:(114/140): python3-jaraco-text-3.2.0-6.el9.noar 182 kB/s | 20 kB 00:00 2023-10-08T05:33:55.210 INFO:teuthology.orchestra.run.smithi070.stdout:(115/140): python3-jwt-2.4.0-1.el9.noarch.rpm 327 kB/s | 41 kB 00:00 2023-10-08T05:33:55.269 INFO:teuthology.orchestra.run.smithi070.stdout:(116/140): python3-logutils-0.3.5-21.el9.noarch 465 kB/s | 46 kB 00:00 2023-10-08T05:33:55.302 INFO:teuthology.orchestra.run.smithi070.stdout:(117/140): python3-more-itertools-8.12.0-2.el9. 865 kB/s | 79 kB 00:00 2023-10-08T05:33:55.352 INFO:teuthology.orchestra.run.smithi070.stdout:(118/140): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2023-10-08T05:33:55.378 INFO:teuthology.orchestra.run.smithi070.stdout:(119/140): python3-natsort-7.1.1-5.el9.noarch.r 534 kB/s | 58 kB 00:00 2023-10-08T05:33:55.411 INFO:teuthology.orchestra.run.smithi070.stdout:(120/140): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2023-10-08T05:33:55.445 INFO:teuthology.orchestra.run.smithi070.stdout:(121/140): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2023-10-08T05:33:55.470 INFO:teuthology.orchestra.run.smithi070.stdout:(122/140): python3-pyOpenSSL-21.0.0-1.el9.noarc 983 kB/s | 90 kB 00:00 2023-10-08T05:33:55.491 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction check 2023-10-08T05:33:55.495 INFO:teuthology.orchestra.run.smithi070.stdout:(123/140): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2023-10-08T05:33:55.554 INFO:teuthology.orchestra.run.smithi070.stdout:(124/140): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2023-10-08T05:33:55.579 INFO:teuthology.orchestra.run.smithi070.stdout:(125/140): python3-rsa-4.9-2.el9.noarch.rpm 548 kB/s | 59 kB 00:00 2023-10-08T05:33:55.604 INFO:teuthology.orchestra.run.smithi070.stdout:(126/140): python3-tempora-5.0.0-2.el9.noarch.r 330 kB/s | 36 kB 00:00 2023-10-08T05:33:55.691 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction check succeeded. 2023-10-08T05:33:55.691 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction test 2023-10-08T05:33:55.705 INFO:teuthology.orchestra.run.smithi070.stdout:(127/140): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2023-10-08T05:33:55.730 INFO:teuthology.orchestra.run.smithi070.stdout:(128/140): python3-typing-extensions-4.4.0-2.el 291 kB/s | 51 kB 00:00 2023-10-08T05:33:55.755 INFO:teuthology.orchestra.run.smithi070.stdout:(129/140): python3-websocket-client-1.2.3-2.el9 597 kB/s | 90 kB 00:00 2023-10-08T05:33:55.822 INFO:teuthology.orchestra.run.smithi070.stdout:(130/140): python3-zc-lockfile-2.0-10.el9.noarc 219 kB/s | 20 kB 00:00 2023-10-08T05:33:55.856 INFO:teuthology.orchestra.run.smithi070.stdout:(131/140): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2023-10-08T05:33:55.898 INFO:teuthology.orchestra.run.smithi070.stdout:(132/140): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2023-10-08T05:33:56.023 INFO:teuthology.orchestra.run.smithi070.stdout:(133/140): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2023-10-08T05:33:56.140 INFO:teuthology.orchestra.run.smithi070.stdout:(134/140): NetworkManager-1.45.4-1.el9.x86_64.r 19 MB/s | 2.2 MB 00:00 2023-10-08T05:33:56.233 INFO:teuthology.orchestra.run.smithi070.stdout:(135/140): NetworkManager-libnm-1.45.4-1.el9.x8 20 MB/s | 1.8 MB 00:00 2023-10-08T05:33:56.258 INFO:teuthology.orchestra.run.smithi070.stdout:(136/140): NetworkManager-team-1.45.4-1.el9.x86 1.7 MB/s | 41 kB 00:00 2023-10-08T05:33:56.299 INFO:teuthology.orchestra.run.smithi070.stdout:(137/140): NetworkManager-tui-1.45.4-1.el9.x86_ 5.9 MB/s | 244 kB 00:00 2023-10-08T05:33:56.350 INFO:teuthology.orchestra.run.smithi070.stdout:(138/140): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2023-10-08T05:33:56.442 INFO:teuthology.orchestra.run.smithi070.stdout:(139/140): librados2-18.0.0-6609.g9725b08f.el9. 5.7 MB/s | 3.3 MB 00:00 2023-10-08T05:33:56.525 INFO:teuthology.orchestra.run.smithi070.stdout:(140/140): librbd1-18.0.0-6609.g9725b08f.el9.x8 4.8 MB/s | 3.0 MB 00:00 2023-10-08T05:33:56.535 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2023-10-08T05:33:56.535 INFO:teuthology.orchestra.run.smithi070.stdout:Total 13 MB/s | 199 MB 00:14 2023-10-08T05:33:56.968 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2023-10-08T05:33:57.164 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2023-10-08T05:33:57.164 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2023-10-08T05:33:57.994 INFO:teuthology.orchestra.run.smithi181.stdout:Transaction test succeeded. 2023-10-08T05:33:57.995 INFO:teuthology.orchestra.run.smithi181.stdout:Running transaction 2023-10-08T05:33:59.361 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2023-10-08T05:33:59.361 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2023-10-08T05:34:03.721 INFO:teuthology.orchestra.run.smithi181.stdout: Preparing : 1/1 2023-10-08T05:34:03.812 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/147 2023-10-08T05:34:03.849 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/147 2023-10-08T05:34:03.886 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/147 2023-10-08T05:34:03.997 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/147 2023-10-08T05:34:04.424 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/147 2023-10-08T05:34:04.460 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librados2-2:18.0.0-6609.g9725b08f.el9.x86_64 6/147 2023-10-08T05:34:04.601 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:18.0.0-6609.g9725b08f.el9.x86_64 6/147 2023-10-08T05:34:04.636 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs2-2:18.0.0-6609.g9725b08f.el9.x86_64 7/147 2023-10-08T05:34:04.760 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephfs2-2:18.0.0-6609.g9725b08f.el9.x86_64 7/147 2023-10-08T05:34:04.949 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rados-2:18.0.0-6609.g9725b08f.el9.x86_64 8/147 2023-10-08T05:34:05.101 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/147 2023-10-08T05:34:05.180 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 10/147 2023-10-08T05:34:05.219 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 11/147 2023-10-08T05:34:05.385 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2023-10-08T05:34:05.460 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/147 2023-10-08T05:34:05.511 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/147 2023-10-08T05:34:05.516 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/147 2023-10-08T05:34:05.542 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephsqlite-2:18.0.0-6609.g9725b08f.el9.x86_64 14/147 2023-10-08T05:34:05.546 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/147 2023-10-08T05:34:05.590 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/147 2023-10-08T05:34:05.693 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/147 2023-10-08T05:34:05.714 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6609.g9725b08f.el9.x86_64 14/147 2023-10-08T05:34:05.760 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libradosstriper1-2:18.0.0-6609.g9725b08f.el9.x86 15/147 2023-10-08T05:34:06.126 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6609.g9725b08f.el9.x86 15/147 2023-10-08T05:34:06.135 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/147 2023-10-08T05:34:06.178 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:18.0.0-6609.g9725b08f.el9.x86_64 6/147 2023-10-08T05:34:06.336 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:18.0.0-6609.g9725b08f.el9.x86_64 6/147 2023-10-08T05:34:06.356 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 16/147 2023-10-08T05:34:06.379 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:18.0.0-6609.g9725b08f.el9.x86_64 7/147 2023-10-08T05:34:06.427 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 16/147 2023-10-08T05:34:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:18.0.0-6609.g9725b08f.el9.x86_64 7/147 2023-10-08T05:34:06.572 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:18.0.0-6609.g9725b08f.el9.x86_64 8/147 2023-10-08T05:34:06.658 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 17/147 2023-10-08T05:34:06.674 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/147 2023-10-08T05:34:06.709 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 17/147 2023-10-08T05:34:06.749 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 10/147 2023-10-08T05:34:06.787 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 11/147 2023-10-08T05:34:07.053 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/147 2023-10-08T05:34:07.110 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/147 2023-10-08T05:34:07.135 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:18.0.0-6609.g9725b08f.el9.x86_64 14/147 2023-10-08T05:34:07.142 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 17/147 2023-10-08T05:34:07.189 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/147 2023-10-08T05:34:07.238 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/147 2023-10-08T05:34:07.257 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6609.g9725b08f.el9.x86_64 14/147 2023-10-08T05:34:07.285 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/147 2023-10-08T05:34:07.294 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:18.0.0-6609.g9725b08f.el9.x86 15/147 2023-10-08T05:34:07.351 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 21/147 2023-10-08T05:34:07.466 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 22/147 2023-10-08T05:34:07.512 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/147 2023-10-08T05:34:07.544 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/147 2023-10-08T05:34:07.625 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6609.g9725b08f.el9.x86 15/147 2023-10-08T05:34:07.665 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/147 2023-10-08T05:34:07.699 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-common-2:18.0.0-6609.g9725b08f.el9. 26/147 2023-10-08T05:34:07.749 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ceph-argparse-2:18.0.0-6609.g9725b08f.el 27/147 2023-10-08T05:34:07.799 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 16/147 2023-10-08T05:34:07.851 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cephfs-2:18.0.0-6609.g9725b08f.el9.x86_6 28/147 2023-10-08T05:34:07.886 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 16/147 2023-10-08T05:34:08.027 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/147 2023-10-08T05:34:08.092 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 30/147 2023-10-08T05:34:08.117 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 17/147 2023-10-08T05:34:08.177 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-1:1.45.4-1.el9.x86_64 17/147 2023-10-08T05:34:08.593 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.45.4-1.el9.x86_64 17/147 2023-10-08T05:34:08.631 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/147 2023-10-08T05:34:08.679 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/147 2023-10-08T05:34:08.744 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/147 2023-10-08T05:34:08.796 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rsa-4.9-2.el9.noarch 31/147 2023-10-08T05:34:08.811 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 21/147 2023-10-08T05:34:08.890 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 32/147 2023-10-08T05:34:08.923 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 22/147 2023-10-08T05:34:08.936 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 33/147 2023-10-08T05:34:08.972 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/147 2023-10-08T05:34:08.998 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librados-devel-2:18.0.0-6609.g9725b08f.el9.x86_6 34/147 2023-10-08T05:34:09.005 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/147 2023-10-08T05:34:09.030 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 35/147 2023-10-08T05:34:09.119 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/147 2023-10-08T05:34:09.187 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:18.0.0-6609.g9725b08f.el9. 26/147 2023-10-08T05:34:09.187 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : re2-1:20211101-3.el9.x86_64 36/147 2023-10-08T05:34:09.232 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 37/147 2023-10-08T05:34:09.245 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:18.0.0-6609.g9725b08f.el 27/147 2023-10-08T05:34:09.300 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 38/147 2023-10-08T05:34:09.374 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:18.0.0-6609.g9725b08f.el9.x86_6 28/147 2023-10-08T05:34:09.395 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 39/147 2023-10-08T05:34:09.441 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 40/147 2023-10-08T05:34:09.481 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 41/147 2023-10-08T05:34:09.548 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 42/147 2023-10-08T05:34:09.572 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/147 2023-10-08T05:34:09.604 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 43/147 2023-10-08T05:34:09.666 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 30/147 2023-10-08T05:34:09.666 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 44/147 2023-10-08T05:34:09.854 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 45/147 2023-10-08T05:34:09.898 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 46/147 2023-10-08T05:34:09.958 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 47/147 2023-10-08T05:34:10.097 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 48/147 2023-10-08T05:34:10.140 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-google-auth-1:2.23.0-1.el9.noarch 49/147 2023-10-08T05:34:10.393 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 31/147 2023-10-08T05:34:10.495 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 32/147 2023-10-08T05:34:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 50/147 2023-10-08T05:34:10.548 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 33/147 2023-10-08T05:34:10.611 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:18.0.0-6609.g9725b08f.el9.x86_6 34/147 2023-10-08T05:34:10.651 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 35/147 2023-10-08T05:34:10.812 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-3.el9.x86_64 36/147 2023-10-08T05:34:10.926 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 51/147 2023-10-08T05:34:10.967 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 52/147 2023-10-08T05:34:11.007 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fmt-8.1.1-5.el9.x86_64 53/147 2023-10-08T05:34:11.058 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/147 2023-10-08T05:34:11.079 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 37/147 2023-10-08T05:34:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 38/147 2023-10-08T05:34:11.213 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/147 2023-10-08T05:34:11.264 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : automake-1.16.2-8.el9.noarch 56/147 2023-10-08T05:34:11.283 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 39/147 2023-10-08T05:34:11.321 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 40/147 2023-10-08T05:34:11.344 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/147 2023-10-08T05:34:11.369 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 41/147 2023-10-08T05:34:11.445 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 42/147 2023-10-08T05:34:11.464 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/147 2023-10-08T05:34:11.503 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 43/147 2023-10-08T05:34:11.577 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 44/147 2023-10-08T05:34:11.738 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/147 2023-10-08T05:34:11.773 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 45/147 2023-10-08T05:34:11.799 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/147 2023-10-08T05:34:12.002 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/147 2023-10-08T05:34:12.060 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/147 2023-10-08T05:34:12.085 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/147 2023-10-08T05:34:12.123 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/147 2023-10-08T05:34:12.167 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/147 2023-10-08T05:34:12.207 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 46/147 2023-10-08T05:34:12.234 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/147 2023-10-08T05:34:12.281 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 47/147 2023-10-08T05:34:12.287 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/147 2023-10-08T05:34:12.371 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/147 2023-10-08T05:34:12.427 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/147 2023-10-08T05:34:12.441 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 48/147 2023-10-08T05:34:12.469 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/147 2023-10-08T05:34:12.480 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.23.0-1.el9.noarch 49/147 2023-10-08T05:34:12.551 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 50/147 2023-10-08T05:34:12.594 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 51/147 2023-10-08T05:34:12.595 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/147 2023-10-08T05:34:12.642 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/147 2023-10-08T05:34:12.656 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 52/147 2023-10-08T05:34:12.676 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/147 2023-10-08T05:34:12.703 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fmt-8.1.1-5.el9.x86_64 53/147 2023-10-08T05:34:12.728 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/147 2023-10-08T05:34:12.754 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/147 2023-10-08T05:34:12.770 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/147 2023-10-08T05:34:12.909 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/147 2023-10-08T05:34:12.951 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/147 2023-10-08T05:34:12.976 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : automake-1.16.2-8.el9.noarch 56/147 2023-10-08T05:34:13.010 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/147 2023-10-08T05:34:13.046 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/147 2023-10-08T05:34:13.155 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/147 2023-10-08T05:34:13.157 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/147 2023-10-08T05:34:13.225 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/147 2023-10-08T05:34:13.433 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/147 2023-10-08T05:34:13.497 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/147 2023-10-08T05:34:13.672 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/147 2023-10-08T05:34:13.732 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/147 2023-10-08T05:34:13.773 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/147 2023-10-08T05:34:13.821 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/147 2023-10-08T05:34:13.875 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/147 2023-10-08T05:34:13.928 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/147 2023-10-08T05:34:13.985 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/147 2023-10-08T05:34:14.097 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/147 2023-10-08T05:34:14.160 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/147 2023-10-08T05:34:14.201 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/147 2023-10-08T05:34:14.223 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/147 2023-10-08T05:34:14.282 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/147 2023-10-08T05:34:14.316 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/147 2023-10-08T05:34:14.329 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/147 2023-10-08T05:34:14.379 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/147 2023-10-08T05:34:14.415 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/147 2023-10-08T05:34:14.442 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/147 2023-10-08T05:34:14.482 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/147 2023-10-08T05:34:14.486 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/147 2023-10-08T05:34:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/147 2023-10-08T05:34:14.691 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/147 2023-10-08T05:34:14.729 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/147 2023-10-08T05:34:14.741 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/147 2023-10-08T05:34:14.763 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/147 2023-10-08T05:34:14.865 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/147 2023-10-08T05:34:14.886 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/147 2023-10-08T05:34:14.974 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/147 2023-10-08T05:34:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/147 2023-10-08T05:34:15.819 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/147 2023-10-08T05:34:15.972 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/147 2023-10-08T05:34:16.025 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/147 2023-10-08T05:34:16.059 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/147 2023-10-08T05:34:16.183 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/147 2023-10-08T05:34:16.217 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/147 2023-10-08T05:34:16.457 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/147 2023-10-08T05:34:16.498 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/147 2023-10-08T05:34:16.611 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/147 2023-10-08T05:34:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/147 2023-10-08T05:34:17.436 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/147 2023-10-08T05:34:17.510 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/147 2023-10-08T05:34:17.547 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/147 2023-10-08T05:34:17.579 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/147 2023-10-08T05:34:17.607 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : jq-1.6-15.el9.x86_64 93/147 2023-10-08T05:34:17.644 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/147 2023-10-08T05:34:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/147 2023-10-08T05:34:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : librbd1-2:18.0.0-6609.g9725b08f.el9.x86_64 96/147 2023-10-08T05:34:18.223 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:18.0.0-6609.g9725b08f.el9.x86_64 96/147 2023-10-08T05:34:18.256 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rbd-2:18.0.0-6609.g9725b08f.el9.x86_64 97/147 2023-10-08T05:34:18.316 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/147 2023-10-08T05:34:18.839 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/147 2023-10-08T05:34:18.878 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : librgw2-2:18.0.0-6609.g9725b08f.el9.x86_64 100/147 2023-10-08T05:34:18.953 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librgw2-2:18.0.0-6609.g9725b08f.el9.x86_64 100/147 2023-10-08T05:34:18.980 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : python3-rgw-2:18.0.0-6609.g9725b08f.el9.x86_64 101/147 2023-10-08T05:34:19.179 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/147 2023-10-08T05:34:19.286 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/147 2023-10-08T05:34:19.328 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/147 2023-10-08T05:34:19.362 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : jq-1.6-15.el9.x86_64 93/147 2023-10-08T05:34:19.404 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/147 2023-10-08T05:34:19.772 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/147 2023-10-08T05:34:19.818 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:18.0.0-6609.g9725b08f.el9.x86_64 96/147 2023-10-08T05:34:19.938 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:18.0.0-6609.g9725b08f.el9.x86_64 96/147 2023-10-08T05:34:20.003 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:18.0.0-6609.g9725b08f.el9.x86_64 97/147 2023-10-08T05:34:20.077 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/147 2023-10-08T05:34:20.572 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/147 2023-10-08T05:34:20.604 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:18.0.0-6609.g9725b08f.el9.x86_64 100/147 2023-10-08T05:34:20.694 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:18.0.0-6609.g9725b08f.el9.x86_64 100/147 2023-10-08T05:34:20.722 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:18.0.0-6609.g9725b08f.el9.x86_64 101/147 2023-10-08T05:34:21.457 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 102/147 2023-10-08T05:34:21.501 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 102/147 2023-10-08T05:34:22.148 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 102/147 2023-10-08T05:34:22.204 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-base-2:18.0.0-6609.g9725b08f.el9.x86_64 103/147 2023-10-08T05:34:22.363 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-base-2:18.0.0-6609.g9725b08f.el9.x86_64 103/147 2023-10-08T05:34:22.363 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-08T05:34:22.363 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-08T05:34:22.363 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:22.422 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-selinux-2:18.0.0-6609.g9725b08f.el9.x86_64 104/147 2023-10-08T05:34:23.254 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 102/147 2023-10-08T05:34:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 102/147 2023-10-08T05:34:23.931 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 102/147 2023-10-08T05:34:23.981 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:18.0.0-6609.g9725b08f.el9.x86_64 103/147 2023-10-08T05:34:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:18.0.0-6609.g9725b08f.el9.x86_64 103/147 2023-10-08T05:34:24.169 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-10-08T05:34:24.169 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-10-08T05:34:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:18.0.0-6609.g9725b08f.el9.x86_64 104/147 2023-10-08T05:34:35.065 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6609.g9725b08f.el9.x86_64 104/147 2023-10-08T05:34:35.065 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /sys 2023-10-08T05:34:35.065 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /proc 2023-10-08T05:34:35.065 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /mnt 2023-10-08T05:34:35.065 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /var/tmp 2023-10-08T05:34:35.065 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /home 2023-10-08T05:34:35.065 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /root 2023-10-08T05:34:35.066 INFO:teuthology.orchestra.run.smithi181.stdout:skipping the directory /tmp 2023-10-08T05:34:35.066 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:35.275 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-volume-2:18.0.0-6609.g9725b08f.el9.noarch 105/147 2023-10-08T05:34:35.318 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-volume-2:18.0.0-6609.g9725b08f.el9.noarch 105/147 2023-10-08T05:34:35.319 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:35.319 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-08T05:34:35.319 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:36.827 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6609.g9725b08f.el9.x86_64 104/147 2023-10-08T05:34:36.828 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2023-10-08T05:34:36.828 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2023-10-08T05:34:36.828 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2023-10-08T05:34:36.828 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2023-10-08T05:34:36.828 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2023-10-08T05:34:36.828 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2023-10-08T05:34:36.828 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2023-10-08T05:34:36.828 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:37.020 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-osd-2:18.0.0-6609.g9725b08f.el9.x86_64 106/147 2023-10-08T05:34:37.062 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:18.0.0-6609.g9725b08f.el9.noarch 105/147 2023-10-08T05:34:37.104 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-osd-2:18.0.0-6609.g9725b08f.el9.x86_64 106/147 2023-10-08T05:34:37.105 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:37.105 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-08T05:34:37.105 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-08T05:34:37.105 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-08T05:34:37.105 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:37.109 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:18.0.0-6609.g9725b08f.el9.noarch 105/147 2023-10-08T05:34:37.109 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:37.109 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-10-08T05:34:37.109 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:37.337 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mds-2:18.0.0-6609.g9725b08f.el9.x86_64 107/147 2023-10-08T05:34:37.442 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mds-2:18.0.0-6609.g9725b08f.el9.x86_64 107/147 2023-10-08T05:34:37.442 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:37.442 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-08T05:34:37.442 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-08T05:34:37.442 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-08T05:34:37.443 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:37.946 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mon-2:18.0.0-6609.g9725b08f.el9.x86_64 108/147 2023-10-08T05:34:38.052 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mon-2:18.0.0-6609.g9725b08f.el9.x86_64 108/147 2023-10-08T05:34:38.052 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:38.052 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-08T05:34:38.053 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-08T05:34:38.053 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-08T05:34:38.053 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:38.805 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:18.0.0-6609.g9725b08f.el9.x86_64 106/147 2023-10-08T05:34:38.918 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:18.0.0-6609.g9725b08f.el9.x86_64 106/147 2023-10-08T05:34:38.919 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:38.919 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-10-08T05:34:38.919 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-08T05:34:38.919 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-10-08T05:34:38.919 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:38.955 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephadm-2:18.0.0-6609.g9725b08f.el9.noarch 109/147 2023-10-08T05:34:38.992 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephadm-2:18.0.0-6609.g9725b08f.el9.noarch 109/147 2023-10-08T05:34:39.040 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6609.g9725b08f.e 110/147 2023-10-08T05:34:39.151 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6609.g9725b08f. 111/147 2023-10-08T05:34:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:18.0.0-6609.g9725b08f.el9.x86_64 107/147 2023-10-08T05:34:39.185 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6609.g9725b08f.el9.noa 112/147 2023-10-08T05:34:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:18.0.0-6609.g9725b08f.el9.x86_64 107/147 2023-10-08T05:34:39.247 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:39.247 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-10-08T05:34:39.247 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-08T05:34:39.247 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-10-08T05:34:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:39.511 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6609.g9725b08f.el9.noa 112/147 2023-10-08T05:34:39.553 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6609.g9725b08f.el9.n 113/147 2023-10-08T05:34:39.786 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:18.0.0-6609.g9725b08f.el9.x86_64 108/147 2023-10-08T05:34:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:18.0.0-6609.g9725b08f.el9.x86_64 108/147 2023-10-08T05:34:39.894 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:39.894 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-10-08T05:34:39.894 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-08T05:34:39.894 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-10-08T05:34:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:40.564 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:18.0.0-6609.g9725b08f.el9.noarch 109/147 2023-10-08T05:34:40.602 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:18.0.0-6609.g9725b08f.el9.noarch 109/147 2023-10-08T05:34:40.649 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6609.g9725b08f.e 110/147 2023-10-08T05:34:40.679 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6609.g9725b08f.el9.n 113/147 2023-10-08T05:34:40.711 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6609.g972 114/147 2023-10-08T05:34:40.761 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6609.g9725b08f. 111/147 2023-10-08T05:34:40.796 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6609.g9725b08f.el9.noa 112/147 2023-10-08T05:34:40.854 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6609.g972 114/147 2023-10-08T05:34:41.032 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6609.g9725b08f.el 115/147 2023-10-08T05:34:41.101 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-2:18.0.0-6609.g9725b08f.el9.x86_64 116/147 2023-10-08T05:34:41.123 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6609.g9725b08f.el9.noa 112/147 2023-10-08T05:34:41.186 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6609.g9725b08f.el9.n 113/147 2023-10-08T05:34:41.187 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6609.g9725b08f.el9.x86_64 116/147 2023-10-08T05:34:41.187 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:41.187 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-08T05:34:41.187 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-08T05:34:41.187 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-08T05:34:41.187 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:41.256 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-mgr-rook-2:18.0.0-6609.g9725b08f.el9.noarch 117/147 2023-10-08T05:34:41.298 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6609.g9725b08f.el9.noarch 117/147 2023-10-08T05:34:42.176 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-2:18.0.0-6609.g9725b08f.el9.x86_64 118/147 2023-10-08T05:34:42.215 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-radosgw-2:18.0.0-6609.g9725b08f.el9.x86_64 119/147 2023-10-08T05:34:42.307 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6609.g9725b08f.el9.x86_64 119/147 2023-10-08T05:34:42.307 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:42.307 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-08T05:34:42.307 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-08T05:34:42.308 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-08T05:34:42.308 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:42.352 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6609.g9725b08f.el9.n 113/147 2023-10-08T05:34:42.357 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6609.g9725b 120/147 2023-10-08T05:34:42.380 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6609.g972 114/147 2023-10-08T05:34:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6609.g9725b 120/147 2023-10-08T05:34:42.441 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:42.441 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-08T05:34:42.441 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:42.516 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 121/147 2023-10-08T05:34:42.520 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6609.g972 114/147 2023-10-08T05:34:42.604 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 121/147 2023-10-08T05:34:42.604 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:42.604 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-08T05:34:42.605 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-08T05:34:42.605 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-08T05:34:42.605 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:42.722 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6609.g9725b08f.el 115/147 2023-10-08T05:34:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:18.0.0-6609.g9725b08f.el9.x86_64 116/147 2023-10-08T05:34:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6609.g9725b08f.el9.x86_64 116/147 2023-10-08T05:34:42.831 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:42.831 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-10-08T05:34:42.831 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-08T05:34:42.831 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-10-08T05:34:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:42.891 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:18.0.0-6609.g9725b08f.el9.noarch 117/147 2023-10-08T05:34:42.930 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6609.g9725b08f.el9.noarch 117/147 2023-10-08T05:34:43.042 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 122/147 2023-10-08T05:34:43.126 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 122/147 2023-10-08T05:34:43.126 INFO:teuthology.orchestra.run.smithi181.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:43.126 INFO:teuthology.orchestra.run.smithi181.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-08T05:34:43.126 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-08T05:34:43.126 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-08T05:34:43.126 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:43.811 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:18.0.0-6609.g9725b08f.el9.x86_64 118/147 2023-10-08T05:34:43.842 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:18.0.0-6609.g9725b08f.el9.x86_64 119/147 2023-10-08T05:34:43.933 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6609.g9725b08f.el9.x86_64 119/147 2023-10-08T05:34:43.933 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:43.933 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-10-08T05:34:43.933 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-08T05:34:43.934 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-10-08T05:34:43.934 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:43.985 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6609.g9725b 120/147 2023-10-08T05:34:44.058 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6609.g9725b 120/147 2023-10-08T05:34:44.058 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:44.059 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-10-08T05:34:44.059 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:44.127 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 121/147 2023-10-08T05:34:44.210 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 121/147 2023-10-08T05:34:44.210 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:44.210 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-10-08T05:34:44.210 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-08T05:34:44.210 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-10-08T05:34:44.210 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:44.561 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 122/147 2023-10-08T05:34:44.640 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 122/147 2023-10-08T05:34:44.640 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-10-08T05:34:44.640 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-10-08T05:34:44.640 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-08T05:34:44.640 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-10-08T05:34:44.640 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:47.342 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-test-2:18.0.0-6609.g9725b08f.el9.x86_64 123/147 2023-10-08T05:34:47.399 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 124/147 2023-10-08T05:34:47.438 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : rbd-nbd-2:18.0.0-6609.g9725b08f.el9.x86_64 125/147 2023-10-08T05:34:47.733 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/147 2023-10-08T05:34:47.854 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/147 2023-10-08T05:34:47.933 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : ceph-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 128/147 2023-10-08T05:34:47.981 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/147 2023-10-08T05:34:48.019 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/147 2023-10-08T05:34:48.049 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libcephfs-devel-2:18.0.0-6609.g9725b08f.el9.x86_ 131/147 2023-10-08T05:34:48.094 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 132/147 2023-10-08T05:34:48.141 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 133/147 2023-10-08T05:34:48.774 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:18.0.0-6609.g9725b08f.el9.x86_64 123/147 2023-10-08T05:34:48.838 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 124/147 2023-10-08T05:34:48.872 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:18.0.0-6609.g9725b08f.el9.x86_64 125/147 2023-10-08T05:34:49.103 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 134/147 2023-10-08T05:34:49.148 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 134/147 2023-10-08T05:34:49.173 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/147 2023-10-08T05:34:49.210 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 134/147 2023-10-08T05:34:49.272 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : cephfs-top-2:18.0.0-6609.g9725b08f.el9.noarch 135/147 2023-10-08T05:34:49.289 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/147 2023-10-08T05:34:49.352 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 128/147 2023-10-08T05:34:49.428 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/147 2023-10-08T05:34:49.471 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/147 2023-10-08T05:34:49.501 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:18.0.0-6609.g9725b08f.el9.x86_ 131/147 2023-10-08T05:34:49.549 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-team-1:1.45.4-1.el9.x86_64 132/147 2023-10-08T05:34:49.593 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-tui-1:1.45.4-1.el9.x86_64 133/147 2023-10-08T05:34:50.519 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 134/147 2023-10-08T05:34:50.567 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 134/147 2023-10-08T05:34:50.625 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 134/147 2023-10-08T05:34:50.652 INFO:teuthology.orchestra.run.smithi181.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 136/147 2023-10-08T05:34:50.679 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-top-2:18.0.0-6609.g9725b08f.el9.noarch 135/147 2023-10-08T05:34:50.754 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 137/147 2023-10-08T05:34:50.815 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 138/147 2023-10-08T05:34:50.951 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : flex-2.6.4-9.el9.x86_64 139/147 2023-10-08T05:34:50.998 INFO:teuthology.orchestra.run.smithi181.stdout: Installing : bison-3.7.4-5.el9.x86_64 140/147 2023-10-08T05:34:51.035 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 141/147 2023-10-08T05:34:51.035 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 142/147 2023-10-08T05:34:51.136 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 142/147 2023-10-08T05:34:51.167 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 143/147 2023-10-08T05:34:51.205 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 144/147 2023-10-08T05:34:51.205 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 144/147 2023-10-08T05:34:51.550 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 144/147 2023-10-08T05:34:51.550 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 145/147 2023-10-08T05:34:51.722 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 145/147 2023-10-08T05:34:51.723 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 146/147 2023-10-08T05:34:51.910 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 146/147 2023-10-08T05:34:52.031 INFO:teuthology.orchestra.run.smithi181.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 147/147 2023-10-08T05:34:52.049 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 136/147 2023-10-08T05:34:52.182 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 137/147 2023-10-08T05:34:52.250 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 138/147 2023-10-08T05:34:52.283 INFO:teuthology.orchestra.run.smithi181.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 147/147 2023-10-08T05:34:52.283 INFO:teuthology.orchestra.run.smithi181.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-08T05:34:52.283 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:52.397 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flex-2.6.4-9.el9.x86_64 139/147 2023-10-08T05:34:52.434 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : bison-3.7.4-5.el9.x86_64 140/147 2023-10-08T05:34:52.471 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 141/147 2023-10-08T05:34:52.472 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 142/147 2023-10-08T05:34:52.589 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 142/147 2023-10-08T05:34:52.642 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 143/147 2023-10-08T05:34:52.681 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 144/147 2023-10-08T05:34:52.681 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 144/147 2023-10-08T05:34:53.020 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 144/147 2023-10-08T05:34:53.020 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 145/147 2023-10-08T05:34:53.184 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 145/147 2023-10-08T05:34:53.184 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 146/147 2023-10-08T05:34:53.319 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 146/147 2023-10-08T05:34:53.461 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 147/147 2023-10-08T05:34:53.724 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 147/147 2023-10-08T05:34:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-08T05:34:53.725 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:34:56.780 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-2:18.0.0-6609.g9725b08f.el9.x86_64 1/147 2023-10-08T05:34:56.780 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-base-2:18.0.0-6609.g9725b08f.el9.x86_64 2/147 2023-10-08T05:34:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 3/147 2023-10-08T05:34:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 4/147 2023-10-08T05:34:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6609.g9725b 5/147 2023-10-08T05:34:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mds-2:18.0.0-6609.g9725b08f.el9.x86_64 6/147 2023-10-08T05:34:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-2:18.0.0-6609.g9725b08f.el9.x86_64 7/147 2023-10-08T05:34:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mon-2:18.0.0-6609.g9725b08f.el9.x86_64 8/147 2023-10-08T05:34:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-osd-2:18.0.0-6609.g9725b08f.el9.x86_64 9/147 2023-10-08T05:34:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-radosgw-2:18.0.0-6609.g9725b08f.el9.x86_64 10/147 2023-10-08T05:34:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-selinux-2:18.0.0-6609.g9725b08f.el9.x86_64 11/147 2023-10-08T05:34:56.781 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-test-2:18.0.0-6609.g9725b08f.el9.x86_64 12/147 2023-10-08T05:34:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 13/147 2023-10-08T05:34:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs-devel-2:18.0.0-6609.g9725b08f.el9.x86_ 14/147 2023-10-08T05:34:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephfs2-2:18.0.0-6609.g9725b08f.el9.x86_64 15/147 2023-10-08T05:34:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libcephsqlite-2:18.0.0-6609.g9725b08f.el9.x86_64 16/147 2023-10-08T05:34:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados-devel-2:18.0.0-6609.g9725b08f.el9.x86_6 17/147 2023-10-08T05:34:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libradosstriper1-2:18.0.0-6609.g9725b08f.el9.x86 18/147 2023-10-08T05:34:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librgw2-2:18.0.0-6609.g9725b08f.el9.x86_64 19/147 2023-10-08T05:34:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6609.g9725b08f.el 20/147 2023-10-08T05:34:56.782 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ceph-common-2:18.0.0-6609.g9725b08f.el9. 21/147 2023-10-08T05:34:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cephfs-2:18.0.0-6609.g9725b08f.el9.x86_6 22/147 2023-10-08T05:34:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rados-2:18.0.0-6609.g9725b08f.el9.x86_64 23/147 2023-10-08T05:34:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rbd-2:18.0.0-6609.g9725b08f.el9.x86_64 24/147 2023-10-08T05:34:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rgw-2:18.0.0-6609.g9725b08f.el9.x86_64 25/147 2023-10-08T05:34:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 26/147 2023-10-08T05:34:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 27/147 2023-10-08T05:34:56.783 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : rbd-nbd-2:18.0.0-6609.g9725b08f.el9.x86_64 28/147 2023-10-08T05:34:56.796 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6609.g9725b08f. 29/147 2023-10-08T05:34:56.797 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6609.g9725b08f.el9.noa 30/147 2023-10-08T05:34:56.797 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6609.g9725b08f.el9.n 31/147 2023-10-08T05:34:56.797 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6609.g972 32/147 2023-10-08T05:34:56.797 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6609.g9725b08f.el 33/147 2023-10-08T05:34:56.797 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6609.g9725b08f.el9.noarch 34/147 2023-10-08T05:34:56.798 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6609.g9725b08f.e 35/147 2023-10-08T05:34:56.798 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : ceph-volume-2:18.0.0-6609.g9725b08f.el9.noarch 36/147 2023-10-08T05:34:56.798 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephadm-2:18.0.0-6609.g9725b08f.el9.noarch 37/147 2023-10-08T05:34:56.798 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : cephfs-top-2:18.0.0-6609.g9725b08f.el9.noarch 38/147 2023-10-08T05:34:56.798 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/147 2023-10-08T05:34:56.799 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/147 2023-10-08T05:34:56.799 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/147 2023-10-08T05:34:56.799 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/147 2023-10-08T05:34:56.799 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/147 2023-10-08T05:34:56.799 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/147 2023-10-08T05:34:56.799 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/147 2023-10-08T05:34:56.800 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/147 2023-10-08T05:34:56.800 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/147 2023-10-08T05:34:56.800 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/147 2023-10-08T05:34:56.800 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/147 2023-10-08T05:34:56.800 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/147 2023-10-08T05:34:56.801 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/147 2023-10-08T05:34:56.801 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/147 2023-10-08T05:34:56.801 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/147 2023-10-08T05:34:56.801 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/147 2023-10-08T05:34:56.801 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/147 2023-10-08T05:34:56.801 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/147 2023-10-08T05:34:56.802 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/147 2023-10-08T05:34:56.802 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/147 2023-10-08T05:34:56.802 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/147 2023-10-08T05:34:56.802 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/147 2023-10-08T05:34:56.802 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/147 2023-10-08T05:34:56.803 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/147 2023-10-08T05:34:56.803 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/147 2023-10-08T05:34:56.803 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/147 2023-10-08T05:34:56.803 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/147 2023-10-08T05:34:56.803 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/147 2023-10-08T05:34:56.803 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/147 2023-10-08T05:34:56.804 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/147 2023-10-08T05:34:56.804 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/147 2023-10-08T05:34:56.804 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/147 2023-10-08T05:34:56.804 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/147 2023-10-08T05:34:56.804 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/147 2023-10-08T05:34:56.804 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/147 2023-10-08T05:34:56.805 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/147 2023-10-08T05:34:56.805 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/147 2023-10-08T05:34:56.805 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/147 2023-10-08T05:34:56.805 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/147 2023-10-08T05:34:56.805 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/147 2023-10-08T05:34:56.805 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/147 2023-10-08T05:34:56.806 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/147 2023-10-08T05:34:56.806 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/147 2023-10-08T05:34:56.806 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/147 2023-10-08T05:34:56.806 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/147 2023-10-08T05:34:56.806 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/147 2023-10-08T05:34:56.807 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/147 2023-10-08T05:34:56.807 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/147 2023-10-08T05:34:56.807 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/147 2023-10-08T05:34:56.807 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : jq-1.6-15.el9.x86_64 88/147 2023-10-08T05:34:56.807 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/147 2023-10-08T05:34:56.807 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/147 2023-10-08T05:34:56.808 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/147 2023-10-08T05:34:56.808 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/147 2023-10-08T05:34:56.808 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/147 2023-10-08T05:34:56.808 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/147 2023-10-08T05:34:56.808 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/147 2023-10-08T05:34:56.808 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/147 2023-10-08T05:34:56.809 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/147 2023-10-08T05:34:56.809 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/147 2023-10-08T05:34:56.809 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/147 2023-10-08T05:34:56.809 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/147 2023-10-08T05:34:56.809 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/147 2023-10-08T05:34:56.809 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/147 2023-10-08T05:34:56.810 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/147 2023-10-08T05:34:56.810 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/147 2023-10-08T05:34:56.810 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/147 2023-10-08T05:34:56.810 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/147 2023-10-08T05:34:56.810 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/147 2023-10-08T05:34:56.811 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-google-auth-1:2.23.0-1.el9.noarch 108/147 2023-10-08T05:34:56.811 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/147 2023-10-08T05:34:56.811 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/147 2023-10-08T05:34:56.811 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/147 2023-10-08T05:34:56.811 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/147 2023-10-08T05:34:56.811 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/147 2023-10-08T05:34:56.812 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/147 2023-10-08T05:34:56.812 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/147 2023-10-08T05:34:56.812 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/147 2023-10-08T05:34:56.812 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/147 2023-10-08T05:34:56.812 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/147 2023-10-08T05:34:56.812 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/147 2023-10-08T05:34:56.813 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/147 2023-10-08T05:34:56.813 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/147 2023-10-08T05:34:56.813 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/147 2023-10-08T05:34:56.813 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/147 2023-10-08T05:34:56.813 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/147 2023-10-08T05:34:56.813 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/147 2023-10-08T05:34:56.814 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/147 2023-10-08T05:34:56.814 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/147 2023-10-08T05:34:56.814 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/147 2023-10-08T05:34:56.814 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/147 2023-10-08T05:34:56.814 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/147 2023-10-08T05:34:56.815 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/147 2023-10-08T05:34:56.815 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/147 2023-10-08T05:34:56.815 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/147 2023-10-08T05:34:56.815 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:18.0.0-6609.g9725b08f.el9.x86_64 134/147 2023-10-08T05:34:56.815 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/147 2023-10-08T05:34:56.815 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:18.0.0-6609.g9725b08f.el9.x86_64 136/147 2023-10-08T05:34:56.816 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/147 2023-10-08T05:34:56.816 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 138/147 2023-10-08T05:34:56.816 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 139/147 2023-10-08T05:34:56.816 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 140/147 2023-10-08T05:34:56.816 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 141/147 2023-10-08T05:34:56.816 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 142/147 2023-10-08T05:34:56.817 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 143/147 2023-10-08T05:34:56.817 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 144/147 2023-10-08T05:34:56.817 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 145/147 2023-10-08T05:34:56.817 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 146/147 2023-10-08T05:34:58.456 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:18.0.0-6609.g9725b08f.el9.x86_64 1/147 2023-10-08T05:34:58.456 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:18.0.0-6609.g9725b08f.el9.x86_64 2/147 2023-10-08T05:34:58.456 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 3/147 2023-10-08T05:34:58.457 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 4/147 2023-10-08T05:34:58.457 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6609.g9725b 5/147 2023-10-08T05:34:58.457 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:18.0.0-6609.g9725b08f.el9.x86_64 6/147 2023-10-08T05:34:58.457 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:18.0.0-6609.g9725b08f.el9.x86_64 7/147 2023-10-08T05:34:58.457 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:18.0.0-6609.g9725b08f.el9.x86_64 8/147 2023-10-08T05:34:58.457 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:18.0.0-6609.g9725b08f.el9.x86_64 9/147 2023-10-08T05:34:58.457 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:18.0.0-6609.g9725b08f.el9.x86_64 10/147 2023-10-08T05:34:58.457 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:18.0.0-6609.g9725b08f.el9.x86_64 11/147 2023-10-08T05:34:58.457 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:18.0.0-6609.g9725b08f.el9.x86_64 12/147 2023-10-08T05:34:58.457 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 13/147 2023-10-08T05:34:58.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:18.0.0-6609.g9725b08f.el9.x86_ 14/147 2023-10-08T05:34:58.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:18.0.0-6609.g9725b08f.el9.x86_64 15/147 2023-10-08T05:34:58.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:18.0.0-6609.g9725b08f.el9.x86_64 16/147 2023-10-08T05:34:58.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:18.0.0-6609.g9725b08f.el9.x86_6 17/147 2023-10-08T05:34:58.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:18.0.0-6609.g9725b08f.el9.x86 18/147 2023-10-08T05:34:58.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:18.0.0-6609.g9725b08f.el9.x86_64 19/147 2023-10-08T05:34:58.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6609.g9725b08f.el 20/147 2023-10-08T05:34:58.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:18.0.0-6609.g9725b08f.el9. 21/147 2023-10-08T05:34:58.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:18.0.0-6609.g9725b08f.el9.x86_6 22/147 2023-10-08T05:34:58.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:18.0.0-6609.g9725b08f.el9.x86_64 23/147 2023-10-08T05:34:58.459 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:18.0.0-6609.g9725b08f.el9.x86_64 24/147 2023-10-08T05:34:58.459 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:18.0.0-6609.g9725b08f.el9.x86_64 25/147 2023-10-08T05:34:58.459 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 26/147 2023-10-08T05:34:58.459 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 27/147 2023-10-08T05:34:58.459 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:18.0.0-6609.g9725b08f.el9.x86_64 28/147 2023-10-08T05:34:58.470 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6609.g9725b08f. 29/147 2023-10-08T05:34:58.470 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6609.g9725b08f.el9.noa 30/147 2023-10-08T05:34:58.470 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6609.g9725b08f.el9.n 31/147 2023-10-08T05:34:58.470 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6609.g972 32/147 2023-10-08T05:34:58.470 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6609.g9725b08f.el 33/147 2023-10-08T05:34:58.470 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6609.g9725b08f.el9.noarch 34/147 2023-10-08T05:34:58.470 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6609.g9725b08f.e 35/147 2023-10-08T05:34:58.471 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:18.0.0-6609.g9725b08f.el9.noarch 36/147 2023-10-08T05:34:58.471 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:18.0.0-6609.g9725b08f.el9.noarch 37/147 2023-10-08T05:34:58.471 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-top-2:18.0.0-6609.g9725b08f.el9.noarch 38/147 2023-10-08T05:34:58.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/147 2023-10-08T05:34:58.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/147 2023-10-08T05:34:58.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/147 2023-10-08T05:34:58.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/147 2023-10-08T05:34:58.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/147 2023-10-08T05:34:58.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/147 2023-10-08T05:34:58.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/147 2023-10-08T05:34:58.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/147 2023-10-08T05:34:58.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/147 2023-10-08T05:34:58.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/147 2023-10-08T05:34:58.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/147 2023-10-08T05:34:58.474 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 78/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 79/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 80/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 81/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 82/147 2023-10-08T05:34:58.475 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 83/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 84/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : jq-1.6-15.el9.x86_64 88/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/147 2023-10-08T05:34:58.476 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/147 2023-10-08T05:34:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.23.0-1.el9.noarch 108/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/147 2023-10-08T05:34:58.478 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/147 2023-10-08T05:34:58.479 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:18.0.0-6609.g9725b08f.el9.x86_64 134/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:18.0.0-6609.g9725b08f.el9.x86_64 136/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-1:1.45.4-1.el9.x86_64 138/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 139/147 2023-10-08T05:34:58.480 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-libnm-1:1.45.4-1.el9.x86_64 140/147 2023-10-08T05:34:58.481 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 141/147 2023-10-08T05:34:58.481 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-team-1:1.45.4-1.el9.x86_64 142/147 2023-10-08T05:34:58.481 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 143/147 2023-10-08T05:34:58.481 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-tui-1:1.45.4-1.el9.x86_64 144/147 2023-10-08T05:34:58.481 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 145/147 2023-10-08T05:34:58.481 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 146/147 2023-10-08T05:34:58.787 INFO:teuthology.orchestra.run.smithi181.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 147/147 2023-10-08T05:34:58.788 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:58.788 INFO:teuthology.orchestra.run.smithi181.stdout:Upgraded: 2023-10-08T05:34:58.788 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-08T05:34:58.788 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-08T05:34:58.788 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-08T05:34:58.788 INFO:teuthology.orchestra.run.smithi181.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-08T05:34:58.788 INFO:teuthology.orchestra.run.smithi181.stdout: librados2-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.788 INFO:teuthology.orchestra.run.smithi181.stdout: librbd1-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.788 INFO:teuthology.orchestra.run.smithi181.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout:Installed: 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: automake-1.16.2-8.el9.noarch 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: bison-3.7.4-5.el9.x86_64 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-grafana-dashboards-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-immutable-object-cache-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mds-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.789 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-cephadm-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:34:58.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:34:58.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:34:58.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:34:58.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-rook-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:34:58.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mon-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-osd-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.790 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-prometheus-alerts-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:34:58.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-radosgw-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-selinux-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-test-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.791 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-volume-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:34:58.791 INFO:teuthology.orchestra.run.smithi181.stdout: cephadm-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:34:58.791 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.791 INFO:teuthology.orchestra.run.smithi181.stdout: cephfs-top-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:34:58.791 INFO:teuthology.orchestra.run.smithi181.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-08T05:34:58.791 INFO:teuthology.orchestra.run.smithi181.stdout: flex-2.6.4-9.el9.x86_64 2023-10-08T05:34:58.791 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: jq-1.6-15.el9.x86_64 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-devel-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.792 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs2-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: libcephsqlite-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: librados-devel-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: libradosstriper1-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: librgw2-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-08T05:34:58.793 INFO:teuthology.orchestra.run.smithi181.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-13.11-1.el9.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-08T05:34:58.794 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cephfs-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-08T05:34:58.795 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-google-auth-1:2.23.0-1.el9.noarch 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-08T05:34:58.796 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-08T05:34:58.797 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ply-3.11-14.el9.noarch 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rados-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rbd-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-08T05:34:58.798 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rgw-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-08T05:34:58.799 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-08T05:34:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-08T05:34:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: rbd-nbd-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:34:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: re2-1:20211101-3.el9.x86_64 2023-10-08T05:34:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-08T05:34:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-08T05:34:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-08T05:34:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-08T05:34:58.800 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-08T05:34:58.801 INFO:teuthology.orchestra.run.smithi181.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-08T05:34:58.801 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:34:58.801 INFO:teuthology.orchestra.run.smithi181.stdout:Complete! 2023-10-08T05:34:59.338 DEBUG:teuthology.parallel:result is None 2023-10-08T05:35:00.767 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 147/147 2023-10-08T05:35:00.767 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:00.767 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2023-10-08T05:35:00.767 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-1:1.45.4-1.el9.x86_64 2023-10-08T05:35:00.767 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-libnm-1:1.45.4-1.el9.x86_64 2023-10-08T05:35:00.767 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-team-1:1.45.4-1.el9.x86_64 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-tui-1:1.45.4-1.el9.x86_64 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout: automake-1.16.2-8.el9.noarch 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout: bison-3.7.4-5.el9.x86_64 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.768 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.769 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top-2:18.0.0-6609.g9725b08f.el9.noarch 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: flex-2.6.4-9.el9.x86_64 2023-10-08T05:35:00.770 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: fmt-8.1.1-5.el9.x86_64 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: fuse-2.9.9-15.el9.x86_64 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: jq-1.6-15.el9.x86_64 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-9.el9.x86_64 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.771 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: libtool-2.4.6-45.el9.x86_64 2023-10-08T05:35:00.772 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.21-2.el9.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-13.11-1.el9.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-10-08T05:35:00.773 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server-13.11-1.el9.x86_64 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-10-08T05:35:00.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.23.0-1.el9.noarch 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-10-08T05:35:00.775 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-10-08T05:35:00.776 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-7.el9.noarch 2023-10-08T05:35:00.777 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.7-6.el9.noarch 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-10-08T05:35:00.778 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-10-08T05:35:00.779 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-10-08T05:35:00.779 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.779 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.779 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:18.0.0-6609.g9725b08f.el9.x86_64 2023-10-08T05:35:00.779 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-3.el9.x86_64 2023-10-08T05:35:00.779 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-5.el9.x86_64 2023-10-08T05:35:00.779 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2023-10-08T05:35:00.779 INFO:teuthology.orchestra.run.smithi070.stdout: uuid-1.6.2-55.el9.x86_64 2023-10-08T05:35:00.779 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-10-08T05:35:00.779 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-10-08T05:35:00.780 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-10-08T05:35:00.780 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:00.780 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-10-08T05:35:01.295 DEBUG:teuthology.parallel:result is None 2023-10-08T05:35:01.295 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9725b08fd801c82987c737f7e6d354878f036ee8 2023-10-08T05:35:01.512 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T05:35:01.573 INFO:teuthology.orchestra.run.smithi070.stdout:18.0.0-6609.g9725b08f.el9 2023-10-08T05:35:01.573 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6609.g9725b08f.el9 2023-10-08T05:35:01.573 INFO:teuthology.task.install:The correct ceph version 18.0.0-6609.g9725b08f is installed. 2023-10-08T05:35:01.574 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9725b08fd801c82987c737f7e6d354878f036ee8 2023-10-08T05:35:01.791 DEBUG:teuthology.orchestra.run.smithi181:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T05:35:01.848 INFO:teuthology.orchestra.run.smithi181.stdout:18.0.0-6609.g9725b08f.el9 2023-10-08T05:35:01.848 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6609.g9725b08f.el9 2023-10-08T05:35:01.848 INFO:teuthology.task.install:The correct ceph version 18.0.0-6609.g9725b08f is installed. 2023-10-08T05:35:01.849 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-08T05:35:01.849 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:01.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T05:35:01.930 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:01.930 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T05:35:02.007 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-08T05:35:02.008 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:02.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T05:35:02.084 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T05:35:02.212 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:02.212 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T05:35:02.289 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T05:35:02.409 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-08T05:35:02.409 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:02.409 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T05:35:02.485 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T05:35:02.623 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:02.623 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T05:35:02.706 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T05:35:02.827 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-08T05:35:02.827 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:02.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T05:35:02.918 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T05:35:03.038 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:03.038 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T05:35:03.144 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T05:35:03.247 INFO:teuthology.run_tasks:Running task ceph... 2023-10-08T05:35:03.313 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-08T05:35:03.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /var/log/ceph 2023-10-08T05:35:03.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2023-10-08T05:35:03.401 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-08T05:35:03.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-08T05:35:03.407 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-08T05:35:03.485 INFO:tasks.ceph:Creating extra log directories... 2023-10-08T05:35:03.485 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-08T05:35:03.490 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-08T05:35:03.575 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-08T05:35:03.575 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-08T05:35:03.575 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-08T05:35:03.576 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:03.576 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-08T05:35:03.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-08T05:35:03.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-08T05:35:03.912 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:03.912 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-08T05:35:03.972 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-08T05:35:04.241 DEBUG:teuthology.orchestra.run.smithi181:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-08T05:35:04.343 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-08T05:35:04.343 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-08T05:35:04.343 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7417325', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '9725b08fd801c82987c737f7e6d354878f036ee8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9725b08fd801c82987c737f7e6d354878f036ee8'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '9725b08fd801c82987c737f7e6d354878f036ee8'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9589, 'sha1': '9725b08fd801c82987c737f7e6d354878f036ee8', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_9725b08fd801c82987c737f7e6d354878f036ee8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '9725b08fd801c82987c737f7e6d354878f036ee8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi070.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyT4shKd9O1F/X50hR7TtBU+d+CcOtra9ZcJ6f/8QsjtDrQEJiQ3oK0WUgaGiNfxC9mA0tjeusS9MGpaHtVKHQ=', 'smithi181.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwinXEppRvqq99rlroBeSrntliLaKSVHRBP/v0OZZD7HQAcQGpINfIV8Wgu2P4/XcSMUsBD46x/YqwhJvKNuFI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9725b08fd801c82987c737f7e6d354878f036ee8'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '9725b08fd801c82987c737f7e6d354878f036ee8', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-07_14:05:27', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-08T05:35:04.344 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-08T05:35:04.350 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-08T05:35:04.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/run/ceph 2023-10-08T05:35:04.440 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2023-10-08T05:35:04.539 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:04.539 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T05:35:04.580 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T05:35:04.580 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2023-10-08T05:35:04.666 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T05:35:04.666 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T05:35:04.667 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 637 Links: 1 2023-10-08T05:35:04.667 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T05:35:04.667 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T05:35:04.667 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-10-08 05:34:54.191789333 +0000 2023-10-08T05:35:04.667 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-10-08 05:32:30.672091920 +0000 2023-10-08T05:35:04.667 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-10-08 05:32:30.672091920 +0000 2023-10-08T05:35:04.667 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-10-08 05:32:30.672091920 +0000 2023-10-08T05:35:04.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T05:35:04.796 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-10-08T05:35:04.796 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-10-08T05:35:04.796 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000345878 s, 1.5 MB/s 2023-10-08T05:35:04.800 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T05:35:04.853 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2023-10-08T05:35:04.901 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T05:35:04.902 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T05:35:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 654 Links: 1 2023-10-08T05:35:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T05:35:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T05:35:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-10-08 05:34:54.193789274 +0000 2023-10-08T05:35:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-10-08 05:32:31.667062624 +0000 2023-10-08T05:35:04.902 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-10-08 05:32:31.667062624 +0000 2023-10-08T05:35:04.902 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-10-08 05:32:31.667062624 +0000 2023-10-08T05:35:04.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T05:35:04.981 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-10-08T05:35:04.981 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-10-08T05:35:04.981 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000240396 s, 2.1 MB/s 2023-10-08T05:35:04.984 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T05:35:05.033 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2023-10-08T05:35:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T05:35:05.131 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T05:35:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 673 Links: 1 2023-10-08T05:35:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T05:35:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T05:35:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-10-08 05:34:54.193789274 +0000 2023-10-08T05:35:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-10-08 05:32:32.629034300 +0000 2023-10-08T05:35:05.132 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-10-08 05:32:32.629034300 +0000 2023-10-08T05:35:05.132 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-10-08 05:32:32.629034300 +0000 2023-10-08T05:35:05.132 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T05:35:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-10-08T05:35:05.221 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-10-08T05:35:05.221 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000234514 s, 2.2 MB/s 2023-10-08T05:35:05.223 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T05:35:05.275 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2023-10-08T05:35:05.318 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T05:35:05.318 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T05:35:05.318 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 695 Links: 1 2023-10-08T05:35:05.318 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T05:35:05.318 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T05:35:05.319 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-10-08 05:34:54.195789216 +0000 2023-10-08T05:35:05.319 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-10-08 05:32:33.662003885 +0000 2023-10-08T05:35:05.319 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-10-08 05:32:33.662003885 +0000 2023-10-08T05:35:05.319 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-10-08 05:32:33.662003885 +0000 2023-10-08T05:35:05.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T05:35:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-10-08T05:35:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-10-08T05:35:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000256502 s, 2.0 MB/s 2023-10-08T05:35:05.451 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T05:35:05.501 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T05:35:05.501 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:05.501 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T05:35:05.563 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T05:35:05.563 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2023-10-08T05:35:05.618 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T05:35:05.618 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T05:35:05.619 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 631 Links: 1 2023-10-08T05:35:05.619 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T05:35:05.619 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T05:35:05.619 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-08 05:34:52.748661517 +0000 2023-10-08T05:35:05.619 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-08 05:32:30.540907146 +0000 2023-10-08T05:35:05.619 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-08 05:32:30.540907146 +0000 2023-10-08T05:35:05.619 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2023-10-08 05:32:30.540907146 +0000 2023-10-08T05:35:05.619 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T05:35:05.714 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-08T05:35:05.714 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-08T05:35:05.714 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000518695 s, 987 kB/s 2023-10-08T05:35:05.718 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T05:35:05.774 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2023-10-08T05:35:05.826 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T05:35:05.826 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T05:35:05.826 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 652 Links: 1 2023-10-08T05:35:05.827 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T05:35:05.827 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T05:35:05.827 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-08 05:34:52.749661497 +0000 2023-10-08T05:35:05.827 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-08 05:32:31.575887251 +0000 2023-10-08T05:35:05.827 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-08 05:32:31.575887251 +0000 2023-10-08T05:35:05.827 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2023-10-08 05:32:31.575887251 +0000 2023-10-08T05:35:05.827 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T05:35:05.917 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-08T05:35:05.917 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-08T05:35:05.917 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000342564 s, 1.5 MB/s 2023-10-08T05:35:05.920 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T05:35:05.979 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2023-10-08T05:35:06.028 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T05:35:06.028 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T05:35:06.028 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-10-08T05:35:06.029 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T05:35:06.029 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T05:35:06.029 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-08 05:34:52.750661478 +0000 2023-10-08T05:35:06.029 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-08 05:32:32.609867375 +0000 2023-10-08T05:35:06.029 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-08 05:32:32.609867375 +0000 2023-10-08T05:35:06.029 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2023-10-08 05:32:32.609867375 +0000 2023-10-08T05:35:06.029 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T05:35:06.120 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-08T05:35:06.120 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-08T05:35:06.120 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000316584 s, 1.6 MB/s 2023-10-08T05:35:06.124 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T05:35:06.183 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2023-10-08T05:35:06.235 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T05:35:06.235 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T05:35:06.235 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 691 Links: 1 2023-10-08T05:35:06.235 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T05:35:06.235 INFO:teuthology.orchestra.run.smithi181.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T05:35:06.235 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2023-10-08 05:34:52.751661459 +0000 2023-10-08T05:35:06.235 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2023-10-08 05:32:33.638847595 +0000 2023-10-08T05:35:06.235 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2023-10-08 05:32:33.638847595 +0000 2023-10-08T05:35:06.236 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: 2023-10-08 05:32:33.638847595 +0000 2023-10-08T05:35:06.236 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T05:35:06.327 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2023-10-08T05:35:06.327 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2023-10-08T05:35:06.327 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000482129 s, 1.1 MB/s 2023-10-08T05:35:06.330 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T05:35:06.388 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T05:35:06.389 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-08T05:35:06.389 INFO:tasks.ceph:Generating config... 2023-10-08T05:35:06.390 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-08T05:35:06.390 INFO:tasks.ceph:[client] debug client = 20 2023-10-08T05:35:06.390 INFO:tasks.ceph:[client] debug ms = 1 2023-10-08T05:35:06.390 INFO:tasks.ceph:[client] pid file = 2023-10-08T05:35:06.390 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-08T05:35:06.391 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-08T05:35:06.391 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-08T05:35:06.391 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-08T05:35:06.391 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-08T05:35:06.391 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-08T05:35:06.391 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-08T05:35:06.391 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-08T05:35:06.391 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-08T05:35:06.391 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-08T05:35:06.391 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-08T05:35:06.391 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-08T05:35:06.391 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-08T05:35:06.392 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-08T05:35:06.392 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-08T05:35:06.392 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-08T05:35:06.392 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-08T05:35:06.392 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-08T05:35:06.392 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-08T05:35:06.392 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-08T05:35:06.392 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-08T05:35:06.392 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-08T05:35:06.392 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-08T05:35:06.392 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-10-08T05:35:06.392 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-10-08T05:35:06.392 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-08T05:35:06.393 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-08T05:35:06.393 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-10-08T05:35:06.393 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-08T05:35:06.393 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-08T05:35:06.393 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-08T05:35:06.393 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-08T05:35:06.393 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-08T05:35:06.393 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-08T05:35:06.393 INFO:tasks.ceph:Setting up mon.a... 2023-10-08T05:35:06.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-08T05:35:06.540 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.keyring 2023-10-08T05:35:06.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-08T05:35:06.684 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T05:35:06.778 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.70'), ('mon.b', '172.21.15.181'), ('mon.c', '[v2:172.21.15.181:3301,v1:172.21.15.181:6790]')] 2023-10-08T05:35:06.778 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.70,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-08T05:35:06.779 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:06.779 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-08T05:35:06.834 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.70 --add b 172.21.15.181 --addv c '[v2:172.21.15.181:3301,v1:172.21.15.181:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:06.936 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:06.937 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: generated fsid cbcacf35-14e2-4581-b83a-6dd588939bbc 2023-10-08T05:35:06.937 INFO:teuthology.orchestra.run.smithi070.stdout:setting min_mon_release = pacific 2023-10-08T05:35:06.937 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 0 2023-10-08T05:35:06.937 INFO:teuthology.orchestra.run.smithi070.stdout:fsid cbcacf35-14e2-4581-b83a-6dd588939bbc 2023-10-08T05:35:06.937 INFO:teuthology.orchestra.run.smithi070.stdout:last_changed 2023-10-08T05:35:06.936896+0000 2023-10-08T05:35:06.937 INFO:teuthology.orchestra.run.smithi070.stdout:created 2023-10-08T05:35:06.936896+0000 2023-10-08T05:35:06.937 INFO:teuthology.orchestra.run.smithi070.stdout:min_mon_release 16 (pacific) 2023-10-08T05:35:06.937 INFO:teuthology.orchestra.run.smithi070.stdout:election_strategy: 1 2023-10-08T05:35:06.937 INFO:teuthology.orchestra.run.smithi070.stdout:0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2023-10-08T05:35:06.937 INFO:teuthology.orchestra.run.smithi070.stdout:1: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.b 2023-10-08T05:35:06.938 INFO:teuthology.orchestra.run.smithi070.stdout:2: [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] mon.c 2023-10-08T05:35:06.938 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-08T05:35:06.940 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-08T05:35:06.992 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cbcacf35-14e2-4581-b83a-6dd588939bbc... 2023-10-08T05:35:06.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-08T05:35:06.998 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-08T05:35:07.163 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2023-10-08T05:35:07.163 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2023-10-08T05:35:07.163 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-08T05:35:07.163 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2023-10-08T05:35:07.163 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-08T05:35:07.164 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2023-10-08T05:35:07.165 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: mon_allow_pool_size_one = true 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2023-10-08T05:35:07.166 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "172.21.15.70,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]" 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on skipped message = False 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = cbcacf35-14e2-4581-b83a-6dd588939bbc 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2023-10-08T05:35:07.167 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: osd mclock profile = high_recovery_ops 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2023-10-08T05:35:07.168 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2023-10-08T05:35:07.169 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 20 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 20 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 10 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 0.95 2023-10-08T05:35:07.170 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = bluestore 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2023-10-08T05:35:07.171 INFO:teuthology.orchestra.run.smithi070.stdout: debug finisher = 20 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2023-10-08T05:35:07.172 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: # 1m isn't quite enough 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: mon_down_mkfs_grace = 2m 2023-10-08T05:35:07.173 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_filestore_osds = false 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: mon op complaint time = 120 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: client mount timeout = 600 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2023-10-08T05:35:07.174 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = "" 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c] 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d] 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout:[mds] 2023-10-08T05:35:07.175 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds = 20 2023-10-08T05:35:07.176 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds balancer = 20 2023-10-08T05:35:07.176 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2023-10-08T05:35:07.176 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug frag = True 2023-10-08T05:35:07.176 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug scatterstat = True 2023-10-08T05:35:07.176 INFO:teuthology.orchestra.run.smithi070.stdout: mds op complaint time = 180 2023-10-08T05:35:07.176 INFO:teuthology.orchestra.run.smithi070.stdout: mds verify scatter = True 2023-10-08T05:35:07.176 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2023-10-08T05:35:07.176 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2023-10-08T05:35:07.176 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2023-10-08T05:35:07.177 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2023-10-08T05:35:07.177 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2023-10-08T05:35:07.177 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-08T05:35:07.177 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2023-10-08T05:35:07.177 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.177 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2023-10-08T05:35:07.177 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.177 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2023-10-08T05:35:07.177 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2023-10-08T05:35:07.178 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-08T05:35:07.179 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2023-10-08T05:35:07.180 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = "172.21.15.70,172.21.15.181,[v2:172.21.15.181:3301,v1:172.21.15.181:6790]" 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on skipped message = False 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = cbcacf35-14e2-4581-b83a-6dd588939bbc 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 600 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: osd mclock profile = high_recovery_ops 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.181 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2023-10-08T05:35:07.182 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2023-10-08T05:35:07.182 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.182 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2023-10-08T05:35:07.182 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2023-10-08T05:35:07.182 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2023-10-08T05:35:07.182 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.182 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2023-10-08T05:35:07.182 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2023-10-08T05:35:07.182 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.182 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore block size = 96636764160 2023-10-08T05:35:07.183 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore fsck on mount = True 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluefs = 20 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluestore = 20 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 20 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: debug rocksdb = 10 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd backfillfull_ratio = 0.85 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd full ratio = 0.9 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd nearfull ratio = 0.8 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: osd failsafe full ratio = 0.95 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: osd objectstore = bluestore 2023-10-08T05:35:07.184 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2023-10-08T05:35:07.185 INFO:teuthology.orchestra.run.smithi181.stdout: debug finisher = 20 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2023-10-08T05:35:07.186 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: auth mon ticket ttl = 660 # 11m 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: auth service ticket ttl = 240 # 4m 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: # don't complain about insecure global_id in the test suite 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: # 1m isn't quite enough 2023-10-08T05:35:07.187 INFO:teuthology.orchestra.run.smithi181.stdout: mon_down_mkfs_grace = 2m 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_filestore_osds = false 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout: mon op complaint time = 120 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout: 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-08T05:35:07.188 INFO:teuthology.orchestra.run.smithi181.stdout: client mount timeout = 600 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout: debug client = 20 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = "" 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 900 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 900 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.b] 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.c] 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2023-10-08T05:35:07.189 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout:[mds] 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout: debug mds = 20 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout: debug mds balancer = 20 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug frag = True 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout: mds debug scatterstat = True 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout: mds op complaint time = 180 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout: mds verify scatter = True 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout: osd op complaint time = 180 2023-10-08T05:35:07.190 INFO:teuthology.orchestra.run.smithi181.stdout: rados mon op timeout = 900 2023-10-08T05:35:07.191 INFO:teuthology.orchestra.run.smithi181.stdout: rados osd op timeout = 900 2023-10-08T05:35:07.219 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-08T05:35:07.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-08T05:35:07.370 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-08T05:35:07.370 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:07.370 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-08T05:35:07.421 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:07.421 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-08T05:35:07.471 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:35:07.471 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:07.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-08T05:35:07.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T05:35:07.606 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:07.606 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:07.709 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T05:35:07.710 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:07.710 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-08T05:35:07.710 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T05:35:07.848 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:07.848 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:07.938 INFO:tasks.ceph:Setting up mon nodes... 2023-10-08T05:35:07.938 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-08T05:35:07.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-08T05:35:08.134 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-08T05:35:08.144 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-08T05:35:08.349 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-08T05:35:08.358 INFO:tasks.ceph:Setting up mds nodes... 2023-10-08T05:35:08.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-08T05:35:08.551 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-08T05:35:08.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-08T05:35:08.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-08T05:35:08.843 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-08T05:35:08.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-08T05:35:08.944 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-08T05:35:09.109 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-08T05:35:09.118 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-08T05:35:09.214 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-08T05:35:09.395 INFO:teuthology.orchestra.run.smithi181.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-08T05:35:09.404 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-08T05:35:09.500 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-08T05:35:09.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-08T05:35:09.624 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-08T05:35:09.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-08T05:35:09.780 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-08T05:35:09.820 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-08T05:35:09.954 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-08T05:35:10.002 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-08T05:35:10.129 INFO:teuthology.orchestra.run.smithi181.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-08T05:35:10.185 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-08T05:35:10.186 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-08T05:35:10.186 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-08T05:35:10.274 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T05:35:10.274 INFO:tasks.ceph:role: osd.0 2023-10-08T05:35:10.274 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:35:10.274 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-08T05:35:10.354 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T05:35:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T05:35:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T05:35:10.354 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-08T05:35:10.355 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T05:35:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2023-10-08T05:35:10.355 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T05:35:10.355 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-08T05:35:10.355 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T05:35:10.355 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T05:35:10.388 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2023-10-08T05:35:10.396 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2023-10-08T05:35:10.396 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-08T05:35:10.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-08T05:35:10.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-08T05:35:10.651 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T05:35:10.652 INFO:tasks.ceph:role: osd.1 2023-10-08T05:35:10.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:35:10.652 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-08T05:35:10.726 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T05:35:10.727 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T05:35:10.727 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T05:35:10.727 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-08T05:35:10.727 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T05:35:10.727 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2023-10-08T05:35:10.727 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T05:35:10.728 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-08T05:35:10.728 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T05:35:10.728 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T05:35:10.760 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2023-10-08T05:35:10.765 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2023-10-08T05:35:10.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-08T05:35:10.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-08T05:35:10.958 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-08T05:35:11.038 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T05:35:11.039 INFO:tasks.ceph:role: osd.2 2023-10-08T05:35:11.039 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:35:11.039 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-08T05:35:11.123 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T05:35:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T05:35:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T05:35:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-08T05:35:11.124 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T05:35:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2023-10-08T05:35:11.125 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T05:35:11.125 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-08T05:35:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T05:35:11.125 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T05:35:11.156 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2023-10-08T05:35:11.160 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2023-10-08T05:35:11.161 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-08T05:35:11.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-08T05:35:11.354 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-08T05:35:11.437 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T05:35:11.438 INFO:tasks.ceph:role: osd.3 2023-10-08T05:35:11.438 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:35:11.438 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-08T05:35:11.526 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T05:35:11.526 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T05:35:11.527 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T05:35:11.527 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-08T05:35:11.527 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T05:35:11.527 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2023-10-08T05:35:11.527 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T05:35:11.527 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-08T05:35:11.527 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T05:35:11.527 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T05:35:11.560 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2023-10-08T05:35:11.565 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2023-10-08T05:35:11.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-08T05:35:11.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-08T05:35:11.754 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:11.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:11.895+0000 7fc8550fa500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-08T05:35:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:11.896+0000 7fc8550fa500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-08T05:35:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:11.896+0000 7fc8550fa500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T05:35:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:11.896+0000 7fc8550fa500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T05:35:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:11.896+0000 7fc8550fa500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T05:35:11.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:11.899+0000 7fc8550fa500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-08T05:35:14.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-08T05:35:14.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:15.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:15.052+0000 7f91b6307500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-08T05:35:15.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:15.052+0000 7f91b6307500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-08T05:35:15.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:15.052+0000 7f91b6307500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T05:35:15.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:15.052+0000 7f91b6307500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T05:35:15.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:15.052+0000 7f91b6307500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T05:35:15.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:15.054+0000 7f91b6307500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-08T05:35:17.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-08T05:35:18.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:18.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:18.201+0000 7efe5480c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-08T05:35:18.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:18.202+0000 7efe5480c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-08T05:35:18.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:18.202+0000 7efe5480c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T05:35:18.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:18.202+0000 7efe5480c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T05:35:18.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:18.202+0000 7efe5480c500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T05:35:18.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:18.204+0000 7efe5480c500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-08T05:35:21.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-08T05:35:21.222 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:21.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:21.355+0000 7f3743574500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-08T05:35:21.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:21.355+0000 7f3743574500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-08T05:35:21.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:21.355+0000 7f3743574500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T05:35:21.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:21.355+0000 7f3743574500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T05:35:21.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:21.355+0000 7f3743574500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T05:35:21.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:21.357+0000 7f3743574500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-08T05:35:24.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-08T05:35:24.385 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-08T05:35:24.490 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T05:35:24.491 INFO:tasks.ceph:role: osd.4 2023-10-08T05:35:24.491 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T05:35:24.491 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-08T05:35:24.596 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T05:35:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T05:35:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T05:35:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-08T05:35:24.596 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T05:35:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-08T05:35:24.596 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T05:35:24.596 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-08T05:35:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T05:35:24.596 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T05:35:25.108 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-08T05:35:25.114 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-08T05:35:25.114 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-08T05:35:25.296 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-08T05:35:25.403 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-08T05:35:25.496 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T05:35:25.496 INFO:tasks.ceph:role: osd.5 2023-10-08T05:35:25.496 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T05:35:25.496 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-08T05:35:25.597 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T05:35:25.597 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T05:35:25.597 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T05:35:25.597 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-08T05:35:25.597 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T05:35:25.597 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-08T05:35:25.598 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T05:35:25.598 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-08T05:35:25.598 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T05:35:25.598 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T05:35:26.094 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-08T05:35:26.101 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-08T05:35:26.101 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-08T05:35:26.262 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-08T05:35:26.366 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-08T05:35:26.451 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T05:35:26.451 INFO:tasks.ceph:role: osd.6 2023-10-08T05:35:26.451 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T05:35:26.452 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-08T05:35:26.549 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T05:35:26.549 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T05:35:26.549 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T05:35:26.549 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-08T05:35:26.549 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T05:35:26.549 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-08T05:35:26.549 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T05:35:26.550 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-08T05:35:26.550 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T05:35:26.550 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T05:35:27.053 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-08T05:35:27.059 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-08T05:35:27.059 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-08T05:35:27.241 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-08T05:35:27.349 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-08T05:35:27.433 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T05:35:27.433 INFO:tasks.ceph:role: osd.7 2023-10-08T05:35:27.433 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T05:35:27.433 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-08T05:35:27.533 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T05:35:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T05:35:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T05:35:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-10-08T05:35:27.533 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T05:35:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2023-10-08T05:35:27.533 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T05:35:27.534 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-10-08T05:35:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T05:35:27.534 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T05:35:28.002 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2023-10-08T05:35:28.008 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2023-10-08T05:35:28.008 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-08T05:35:28.164 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-08T05:35:28.266 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:28.424 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:28.422+0000 7feec9aaf500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-08T05:35:28.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:28.427+0000 7feec9aaf500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-08T05:35:28.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:28.427+0000 7feec9aaf500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T05:35:28.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:28.427+0000 7feec9aaf500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T05:35:28.429 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:28.427+0000 7feec9aaf500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T05:35:28.438 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:28.437+0000 7feec9aaf500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-08T05:35:31.367 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-08T05:35:31.463 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:31.597 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:31.596+0000 7f7c7872b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-08T05:35:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:31.596+0000 7f7c7872b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-08T05:35:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:31.597+0000 7f7c7872b500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T05:35:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:31.597+0000 7f7c7872b500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T05:35:31.598 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:31.597+0000 7f7c7872b500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T05:35:31.599 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:31.599+0000 7f7c7872b500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-08T05:35:34.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:35:34.355 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:35:34.522 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-08T05:35:34.661 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:34.800 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:34.799+0000 7fc534312500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-08T05:35:34.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:34.800+0000 7fc534312500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-08T05:35:34.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:34.800+0000 7fc534312500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T05:35:34.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:34.800+0000 7fc534312500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T05:35:34.801 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:34.800+0000 7fc534312500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T05:35:34.803 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:34.802+0000 7fc534312500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-08T05:35:37.735 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-08T05:35:37.835 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:37.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:37.971+0000 7fe3158a5500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-08T05:35:37.972 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:37.971+0000 7fe3158a5500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-08T05:35:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:37.971+0000 7fe3158a5500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T05:35:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:37.971+0000 7fe3158a5500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T05:35:37.973 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:37.971+0000 7fe3158a5500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T05:35:37.974 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:37.974+0000 7fe3158a5500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-08T05:35:40.893 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-08T05:35:40.989 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-08T05:35:40.989 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:40.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-08T05:35:41.095 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:41.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-08T05:35:41.193 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:41.193 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-08T05:35:41.286 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:41.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-08T05:35:41.383 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:41.384 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-08T05:35:41.473 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:41.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-08T05:35:41.564 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:41.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-08T05:35:41.709 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:41.709 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-08T05:35:41.813 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:41.813 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-08T05:35:41.907 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:41.907 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-08T05:35:42.006 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:42.006 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-08T05:35:42.102 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:42.102 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-08T05:35:42.197 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:42.197 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-08T05:35:42.289 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:42.289 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-08T05:35:42.430 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:42.431 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-08T05:35:42.490 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:42.490 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-08T05:35:42.540 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:42.540 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-08T05:35:42.601 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:42.601 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-08T05:35:42.654 INFO:tasks.ceph:Adding keys to all mons... 2023-10-08T05:35:42.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-08T05:35:42.663 DEBUG:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-08T05:35:42.748 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.y] 2023-10-08T05:35:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g== 2023-10-08T05:35:42.748 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2023-10-08T05:35:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA== 2023-10-08T05:35:42.748 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c] 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw== 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.0] 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A== 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.1] 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A== 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.2] 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg== 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.3] 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg== 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.x] 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA== 2023-10-08T05:35:42.749 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ== 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d] 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ== 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.4] 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw== 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.5] 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw== 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.6] 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA== 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.7] 2023-10-08T05:35:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg== 2023-10-08T05:35:42.751 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T05:35:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T05:35:42.751 INFO:teuthology.orchestra.run.smithi070.stdout:[client.2] 2023-10-08T05:35:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q== 2023-10-08T05:35:42.751 INFO:teuthology.orchestra.run.smithi070.stdout:[client.1] 2023-10-08T05:35:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg== 2023-10-08T05:35:42.751 INFO:teuthology.orchestra.run.smithi070.stdout:[client.3] 2023-10-08T05:35:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg== 2023-10-08T05:35:42.756 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2023-10-08T05:35:42.756 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g== 2023-10-08T05:35:42.756 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.a] 2023-10-08T05:35:42.756 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA== 2023-10-08T05:35:42.756 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.c] 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw== 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A== 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A== 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg== 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg== 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2023-10-08T05:35:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA== 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.b] 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ== 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout:[mds.d] 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ== 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.4] 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw== 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.5] 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw== 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.6] 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA== 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.7] 2023-10-08T05:35:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg== 2023-10-08T05:35:42.759 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2023-10-08T05:35:42.759 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T05:35:42.759 INFO:teuthology.orchestra.run.smithi181.stdout:[client.2] 2023-10-08T05:35:42.759 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q== 2023-10-08T05:35:42.760 INFO:teuthology.orchestra.run.smithi181.stdout:[client.1] 2023-10-08T05:35:42.760 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg== 2023-10-08T05:35:42.760 INFO:teuthology.orchestra.run.smithi181.stdout:[client.3] 2023-10-08T05:35:42.760 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg== 2023-10-08T05:35:42.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T05:35:42.801 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T05:35:42.954 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T05:35:42.962 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T05:35:43.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T05:35:43.108 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T05:35:43.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:43.256 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:43.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:43.409 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:43.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:43.567 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:43.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:43.715 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:43.860 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T05:35:43.866 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T05:35:44.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T05:35:44.019 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T05:35:44.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T05:35:44.171 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T05:35:44.317 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:44.326 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:44.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:44.472 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:44.613 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:44.618 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:44.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:44.769 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T05:35:44.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T05:35:44.928 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T05:35:45.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T05:35:45.079 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T05:35:45.224 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T05:35:45.233 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T05:35:45.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T05:35:45.395 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T05:35:45.540 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-08T05:35:45.540 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-08T05:35:45.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T05:35:45.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-08T05:35:45.931 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-08T05:35:46.064 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T05:35:46.223 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-08T05:35:46.313 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-08T05:35:46.407 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T05:35:46.561 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-08T05:35:46.645 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:46.655 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-08T05:35:46.707 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-08T05:35:46.708 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-08T05:35:46.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-08T05:35:46.714 INFO:tasks.ceph.mon.a:Started 2023-10-08T05:35:46.714 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-08T05:35:46.714 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-08T05:35:46.719 INFO:tasks.ceph.mon.b:Started 2023-10-08T05:35:46.719 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-08T05:35:46.720 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-08T05:35:46.725 INFO:tasks.ceph.mon.c:Started 2023-10-08T05:35:46.725 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-08T05:35:46.726 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-08T05:35:46.726 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-08T05:35:46.731 INFO:tasks.ceph.mgr.y:Started 2023-10-08T05:35:46.732 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-08T05:35:46.732 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-08T05:35:46.737 INFO:tasks.ceph.mgr.x:Started 2023-10-08T05:35:46.737 INFO:tasks.ceph:Setting crush tunables to default 2023-10-08T05:35:46.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd crush tunables default 2023-10-08T05:35:46.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.921+0000 7ff953cda640 1 Processor -- start 2023-10-08T05:35:46.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.921+0000 7ff953cda640 1 -- start start 2023-10-08T05:35:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.922+0000 7ff953cda640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.922+0000 7ff953cda640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.922+0000 7ff953cda640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94c15bf30 con 0x7ff94c06cd00 2023-10-08T05:35:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.922+0000 7ff953cda640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94c15b320 con 0x7ff94c0c82b0 2023-10-08T05:35:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.922+0000 7ff953cda640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c15b600 con 0x7ff94c15aad0 2023-10-08T05:35:46.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.923+0000 7ff951275640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.923+0000 7ff952277640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff94c06cd00 0x7ff94c070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33860/0 (socket says 172.21.15.70:33860) 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.923+0000 7ff952277640 1 -- 172.21.15.70:0/4182243265 learned_addr learned my addr 172.21.15.70:0/4182243265 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.923+0000 7ff951275640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56970/0 (socket says 172.21.15.70:56970) 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.923+0000 7ff951275640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 msgr2=0x7ff94c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.923+0000 7ff951275640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 msgr2=0x7ff94c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.923+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.923+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.923+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.924+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 msgr2=0x7ff94c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.924+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 msgr2=0x7ff94c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.924+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:46.924+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:35:47.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.124+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:47.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.125+0000 7ff951275640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 msgr2=0x7ff94c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T05:35:47.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.125+0000 7ff951275640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 msgr2=0x7ff94c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:47.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.125+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:47.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.125+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:35:47.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.126+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:47.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.126+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 msgr2=0x7ff94c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T05:35:47.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.126+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 msgr2=0x7ff94c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:47.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.126+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:47.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.126+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:35:47.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.526+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:47.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.527+0000 7ff951275640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 msgr2=0x7ff94c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T05:35:47.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.527+0000 7ff951275640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 msgr2=0x7ff94c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:47.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.527+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:47.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.527+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:35:47.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.527+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:47.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.528+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 msgr2=0x7ff94c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T05:35:47.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.528+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 msgr2=0x7ff94c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:47.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.528+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:47.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:47.528+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:35:48.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:48.328+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:48.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:48.329+0000 7ff951275640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 msgr2=0x7ff94c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T05:35:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:48.329+0000 7ff951275640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 msgr2=0x7ff94c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:48.329+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:48.329+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:35:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:48.330+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:48.330+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 msgr2=0x7ff94c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T05:35:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:48.330+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 msgr2=0x7ff94c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:48.330+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:48.330+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:35:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.923+0000 7ff9437fe640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 msgr2=0x7ff94c0b1750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T05:35:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.923+0000 7ff9437fe640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff94c15aad0 0x7ff94c0b1750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.923+0000 7ff9437fe640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 msgr2=0x7ff94c0c8710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T05:35:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.923+0000 7ff9437fe640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.70:3300/0 conn(0x7ff94c0c82b0 0x7ff94c0c8710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.923+0000 7ff9437fe640 1 -- 172.21.15.70:0/4182243265 >> v1:172.21.15.70:6789/0 conn(0x7ff94c06cd00 legacy=0x7ff94c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.923+0000 7ff9437fe640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.923+0000 7ff9437fe640 1 -- 172.21.15.70:0/4182243265 --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94c15b8e0 con 0x7ff930006170 2023-10-08T05:35:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.923+0000 7ff9437fe640 1 -- 172.21.15.70:0/4182243265 --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94c15b320 con 0x7ff9300036e0 2023-10-08T05:35:49.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.924+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:49.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.924+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 msgr2=0x7ff930005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T05:35:49.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.924+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 msgr2=0x7ff930005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:49.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.924+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:49.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:49.924+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:35:50.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:50.125+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:50.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:50.126+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 msgr2=0x7ff930005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T05:35:50.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:50.126+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 msgr2=0x7ff930005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:50.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:50.126+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:50.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:50.126+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:35:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:50.527+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:50.528+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 msgr2=0x7ff930005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T05:35:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:50.528+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 msgr2=0x7ff930005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:50.528+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:50.528+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:35:51.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.329+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:51.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.330+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 msgr2=0x7ff930005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-08T05:35:51.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.330+0000 7ff951a76640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 msgr2=0x7ff930005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T05:35:51.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.330+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-08T05:35:51.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.330+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:35:51.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.970+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849191724 0 0) 0x7ff94c15b8e0 con 0x7ff930006170 2023-10-08T05:35:51.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.970+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff938003520 con 0x7ff930006170 2023-10-08T05:35:51.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.975+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2949322303 0 0) 0x7ff938003520 con 0x7ff930006170 2023-10-08T05:35:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.975+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 msgr2=0x7ff930005b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T05:35:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.975+0000 7ff950a74640 1 --2- 172.21.15.70:0/4182243265 >> v2:172.21.15.181:3300/0 conn(0x7ff9300036e0 0x7ff930005b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.975+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93000af10 con 0x7ff930006170 2023-10-08T05:35:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.975+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff93c00d580 con 0x7ff930006170 2023-10-08T05:35:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.976+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 >> v1:172.21.15.181:6789/0 conn(0x7ff930006170 legacy=0x7ff930008630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.976+0000 7ff952277640 1 -- 172.21.15.70:0/4182243265 reap_dead start 2023-10-08T05:35:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.976+0000 7ff950a74640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380039c0 0x7ff938003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.976+0000 7ff950a74640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9380047c0 0x7ff938004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.976+0000 7ff950a74640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff938005380 0x7ff93800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.976+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c15b320 con 0x7ff938005380 2023-10-08T05:35:51.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.976+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93800c240 con 0x7ff9380039c0 2023-10-08T05:35:51.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.976+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93800c400 con 0x7ff9380047c0 2023-10-08T05:35:51.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.977+0000 7ff952277640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff938005380 0x7ff93800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:51.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.977+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380039c0 0x7ff938003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:51.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.977+0000 7ff951275640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9380047c0 0x7ff938004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:51.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.978+0000 7ff952277640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9380047c0 msgr2=0x7ff938004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:51.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.978+0000 7ff952277640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9380047c0 0x7ff938004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:51.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.978+0000 7ff952277640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380039c0 msgr2=0x7ff938003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:51.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.978+0000 7ff952277640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380039c0 0x7ff938003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:51.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.978+0000 7ff952277640 1 -- 172.21.15.70:0/4182243265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938003520 con 0x7ff938005380 2023-10-08T05:35:51.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.978+0000 7ff951a76640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380039c0 0x7ff938003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:35:51.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.983+0000 7ff952277640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff938005380 0x7ff93800b980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0cb120 tx=0x7ff9380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de814415f7386515 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:51.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.984+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94402d620 con 0x7ff938005380 2023-10-08T05:35:51.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.984+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94402d780 con 0x7ff938005380 2023-10-08T05:35:51.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.984+0000 7ff950a74640 1 -- 172.21.15.70:0/4182243265 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94402db30 con 0x7ff938005380 2023-10-08T05:35:51.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.985+0000 7ff953cda640 1 -- 172.21.15.70:0/4182243265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff938005380 msgr2=0x7ff93800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:51.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.985+0000 7ff953cda640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff938005380 0x7ff93800b980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0cb120 tx=0x7ff9380057f0 comp rx=0 tx=0).stop 2023-10-08T05:35:51.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.985+0000 7ff953cda640 1 -- 172.21.15.70:0/4182243265 shutdown_connections 2023-10-08T05:35:51.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.985+0000 7ff953cda640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff938005380 0x7ff93800b980 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:51.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.985+0000 7ff953cda640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9380047c0 0x7ff938004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:51.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.985+0000 7ff953cda640 1 --2- 172.21.15.70:0/4182243265 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380039c0 0x7ff938003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:51.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.985+0000 7ff953cda640 1 -- 172.21.15.70:0/4182243265 >> 172.21.15.70:0/4182243265 conn(0x7ff94c069d50 msgr2=0x7ff94c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:51.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.986+0000 7ff953cda640 1 -- 172.21.15.70:0/4182243265 shutdown_connections 2023-10-08T05:35:51.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.986+0000 7ff953cda640 1 -- 172.21.15.70:0/4182243265 wait complete. 2023-10-08T05:35:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.990+0000 7ff953cda640 1 Processor -- start 2023-10-08T05:35:51.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.990+0000 7ff953cda640 1 -- start start 2023-10-08T05:35:51.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.992+0000 7ff953cda640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9380039c0 0x7ff94c14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:51.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.992+0000 7ff953cda640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380047c0 0x7ff94c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:51.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.992+0000 7ff953cda640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff938005380 0x7ff94c152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:51.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.992+0000 7ff953cda640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c0cae50 con 0x7ff9380039c0 2023-10-08T05:35:51.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.992+0000 7ff953cda640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c1e62a0 con 0x7ff9380047c0 2023-10-08T05:35:51.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.992+0000 7ff953cda640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c1e4530 con 0x7ff938005380 2023-10-08T05:35:51.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.992+0000 7ff951a76640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9380039c0 0x7ff94c14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:51.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.992+0000 7ff951a76640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9380039c0 0x7ff94c14abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57054/0 (socket says 172.21.15.70:57054) 2023-10-08T05:35:51.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.992+0000 7ff951a76640 1 -- 172.21.15.70:0/3037084985 learned_addr learned my addr 172.21.15.70:0/3037084985 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:35:51.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.993+0000 7ff951a76640 1 -- 172.21.15.70:0/3037084985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff938005380 msgr2=0x7ff94c152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:51.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.993+0000 7ff951a76640 1 --2- 172.21.15.70:0/3037084985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff938005380 0x7ff94c152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:51.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.993+0000 7ff951a76640 1 -- 172.21.15.70:0/3037084985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380047c0 msgr2=0x7ff94c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:51.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.993+0000 7ff951a76640 1 --2- 172.21.15.70:0/3037084985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380047c0 0x7ff94c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:51.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.993+0000 7ff951a76640 1 -- 172.21.15.70:0/3037084985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c1e4810 con 0x7ff9380039c0 2023-10-08T05:35:51.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.993+0000 7ff951a76640 1 --2- 172.21.15.70:0/3037084985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9380039c0 0x7ff94c14abf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff948017cb0 tx=0x7ff9480167a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:51.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.993+0000 7ff9427fc640 1 -- 172.21.15.70:0/3037084985 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480233f0 con 0x7ff9380039c0 2023-10-08T05:35:51.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.993+0000 7ff9427fc640 1 -- 172.21.15.70:0/3037084985 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94801f040 con 0x7ff9380039c0 2023-10-08T05:35:51.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.993+0000 7ff9427fc640 1 -- 172.21.15.70:0/3037084985 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff948023750 con 0x7ff9380039c0 2023-10-08T05:35:51.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.994+0000 7ff953cda640 1 -- 172.21.15.70:0/3037084985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c0cab60 con 0x7ff9380039c0 2023-10-08T05:35:51.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.994+0000 7ff953cda640 1 -- 172.21.15.70:0/3037084985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c1e4af0 con 0x7ff9380039c0 2023-10-08T05:35:51.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.994+0000 7ff9427fc640 1 -- 172.21.15.70:0/3037084985 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff948005990 con 0x7ff9380039c0 2023-10-08T05:35:51.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.994+0000 7ff9427fc640 1 -- 172.21.15.70:0/3037084985 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff948023c90 con 0x7ff9380039c0 2023-10-08T05:35:51.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.995+0000 7ff953cda640 1 -- 172.21.15.70:0/3037084985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c1e6070 con 0x7ff9380039c0 2023-10-08T05:35:51.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:51.998+0000 7ff9427fc640 1 -- 172.21.15.70:0/3037084985 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ff94c1e6070 con 0x7ff9380039c0 2023-10-08T05:35:52.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.070+0000 7ff953cda640 1 -- 172.21.15.70:0/3037084985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff94c04e160 con 0x7ff9380039c0 2023-10-08T05:35:52.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.074+0000 7ff9427fc640 1 -- 172.21.15.70:0/3037084985 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff948035430 con 0x7ff9380039c0 2023-10-08T05:35:52.075 INFO:teuthology.orchestra.run.smithi070.stderr:adjusted tunables profile to default 2023-10-08T05:35:52.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.076+0000 7ff953cda640 1 -- 172.21.15.70:0/3037084985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9380039c0 msgr2=0x7ff94c14abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:52.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.076+0000 7ff953cda640 1 --2- 172.21.15.70:0/3037084985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9380039c0 0x7ff94c14abf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff948017cb0 tx=0x7ff9480167a0 comp rx=0 tx=0).stop 2023-10-08T05:35:52.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.076+0000 7ff953cda640 1 -- 172.21.15.70:0/3037084985 shutdown_connections 2023-10-08T05:35:52.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.076+0000 7ff953cda640 1 --2- 172.21.15.70:0/3037084985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff938005380 0x7ff94c152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:52.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.076+0000 7ff953cda640 1 --2- 172.21.15.70:0/3037084985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380047c0 0x7ff94c0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:52.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.076+0000 7ff953cda640 1 --2- 172.21.15.70:0/3037084985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9380039c0 0x7ff94c14abf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:52.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.076+0000 7ff953cda640 1 -- 172.21.15.70:0/3037084985 >> 172.21.15.70:0/3037084985 conn(0x7ff94c069d50 msgr2=0x7ff94c070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:52.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.076+0000 7ff953cda640 1 -- 172.21.15.70:0/3037084985 shutdown_connections 2023-10-08T05:35:52.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:52.076+0000 7ff953cda640 1 -- 172.21.15.70:0/3037084985 wait complete. 2023-10-08T05:35:52.095 INFO:tasks.ceph:check_enable_crimson: False 2023-10-08T05:35:52.095 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-08T05:35:52.095 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:52.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-08T05:35:52.189 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:52.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-08T05:35:52.331 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:52.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-08T05:35:52.430 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:35:52.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-08T05:35:52.525 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:52.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-08T05:35:52.564 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:52.563+0000 7f3633822040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:35:52.618 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:52.619 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-08T05:35:52.629 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:52.628+0000 7f3633822040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:35:52.715 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:52.715 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-08T05:35:52.843 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:35:52.843 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-08T05:35:52.975 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 2150863d-4f36-4fdb-becd-23d5e739c8bc 0 2023-10-08T05:35:53.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.209+0000 7fe7a7cf2640 1 Processor -- start 2023-10-08T05:35:53.210 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.209+0000 7fe7a7cf2640 1 -- start start 2023-10-08T05:35:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.211+0000 7fe7a7cf2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a006efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.211+0000 7fe7a7cf2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a00ccbb0 con 0x7fe7a0069d50 2023-10-08T05:35:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.211+0000 7fe7a7cf2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a015e040 con 0x7fe7a0159940 2023-10-08T05:35:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.211+0000 7fe7a7cf2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a015e320 con 0x7fe7a00b48c0 2023-10-08T05:35:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.211+0000 7fe7a5a8e640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe7a0159940 0x7fe7a00cd600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57404/0 (socket says 172.21.15.181:57404) 2023-10-08T05:35:53.212 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.211+0000 7fe7a528d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a006efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.211+0000 7fe7a5a8e640 1 -- 172.21.15.181:0/1851456068 learned_addr learned my addr 172.21.15.181:0/1851456068 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-08T05:35:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.212+0000 7fe7a4a8c640 1 -- 172.21.15.181:0/1851456068 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718634905 0 0) 0x7fe7a015e040 con 0x7fe7a0159940 2023-10-08T05:35:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.212+0000 7fe7a528d640 1 -- 172.21.15.181:0/1851456068 >> v1:172.21.15.181:6789/0 conn(0x7fe7a0159940 legacy=0x7fe7a00cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.212+0000 7fe7a528d640 1 -- 172.21.15.181:0/1851456068 >> v1:172.21.15.70:6789/0 conn(0x7fe7a0069d50 legacy=0x7fe7a00ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:53.213 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.212+0000 7fe7a528d640 1 -- 172.21.15.181:0/1851456068 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00cce00 con 0x7fe7a00b48c0 2023-10-08T05:35:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.213+0000 7fe7a528d640 1 --2- 172.21.15.181:0/1851456068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a006efd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe7900097d0 tx=0x7fe79002b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=411caab97da78f82 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.213+0000 7fe7a4a8c640 1 -- 172.21.15.181:0/1851456068 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790031770 con 0x7fe7a00b48c0 2023-10-08T05:35:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.213+0000 7fe7a4a8c640 1 -- 172.21.15.181:0/1851456068 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7900318d0 con 0x7fe7a00b48c0 2023-10-08T05:35:53.214 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.213+0000 7fe7a4a8c640 1 -- 172.21.15.181:0/1851456068 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790031c60 con 0x7fe7a00b48c0 2023-10-08T05:35:53.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.214+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1851456068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 msgr2=0x7fe7a006efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.214+0000 7fe7a7cf2640 1 --2- 172.21.15.181:0/1851456068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a006efd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe7900097d0 tx=0x7fe79002b810 comp rx=0 tx=0).stop 2023-10-08T05:35:53.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.214+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1851456068 shutdown_connections 2023-10-08T05:35:53.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.214+0000 7fe7a7cf2640 1 --2- 172.21.15.181:0/1851456068 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a006efd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.214+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1851456068 >> 172.21.15.181:0/1851456068 conn(0x7fe7a00cba60 msgr2=0x7fe7a00cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:53.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.214+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1851456068 shutdown_connections 2023-10-08T05:35:53.216 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.214+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1851456068 wait complete. 2023-10-08T05:35:53.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.216+0000 7fe7a7cf2640 1 Processor -- start 2023-10-08T05:35:53.217 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.216+0000 7fe7a7cf2640 1 -- start start 2023-10-08T05:35:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a7cf2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0069d50 0x7fe7a014bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.219 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a7cf2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a00c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a7cf2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0159940 0x7fe7a00ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a7cf2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00ad720 con 0x7fe7a0069d50 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a7cf2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00acaa0 con 0x7fe7a0159940 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a7cf2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00acd80 con 0x7fe7a00b48c0 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a528d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a00c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a628f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0159940 0x7fe7a00ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a628f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0159940 0x7fe7a00ac1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51822/0 (socket says 172.21.15.181:51822) 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a628f640 1 -- 172.21.15.181:0/1125868518 learned_addr learned my addr 172.21.15.181:0/1125868518 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a528d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a00c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55350/0 (socket says 172.21.15.181:55350) 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a5a8e640 1 --2- 172.21.15.181:0/1125868518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0069d50 0x7fe7a014bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.220 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a528d640 1 -- 172.21.15.181:0/1125868518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0159940 msgr2=0x7fe7a00ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a528d640 1 --2- 172.21.15.181:0/1125868518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0159940 0x7fe7a00ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a528d640 1 -- 172.21.15.181:0/1125868518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0069d50 msgr2=0x7fe7a014bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.217+0000 7fe7a528d640 1 --2- 172.21.15.181:0/1125868518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0069d50 0x7fe7a014bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.218+0000 7fe7a528d640 1 -- 172.21.15.181:0/1125868518 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe790009400 con 0x7fe7a00b48c0 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.218+0000 7fe7a628f640 1 --2- 172.21.15.181:0/1125868518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0159940 0x7fe7a00ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.218+0000 7fe7a528d640 1 --2- 172.21.15.181:0/1125868518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a00c5420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe790033000 tx=0x7fe7900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.218+0000 7fe7967fc640 1 -- 172.21.15.181:0/1125868518 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790004220 con 0x7fe7a00b48c0 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.218+0000 7fe7967fc640 1 -- 172.21.15.181:0/1125868518 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7900043a0 con 0x7fe7a00b48c0 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.218+0000 7fe7967fc640 1 -- 172.21.15.181:0/1125868518 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790003850 con 0x7fe7a00b48c0 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.218+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1125868518 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a00ad060 con 0x7fe7a00b48c0 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.218+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1125868518 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a00c5960 con 0x7fe7a00b48c0 2023-10-08T05:35:53.221 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.218+0000 7fe7967fc640 1 -- 172.21.15.181:0/1125868518 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe790003cb0 con 0x7fe7a00b48c0 2023-10-08T05:35:53.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.219+0000 7fe7967fc640 1 -- 172.21.15.181:0/1125868518 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe790032de0 con 0x7fe7a00b48c0 2023-10-08T05:35:53.222 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.219+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1125868518 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a0145e30 con 0x7fe7a00b48c0 2023-10-08T05:35:53.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.222+0000 7fe7967fc640 1 -- 172.21.15.181:0/1125868518 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fe79004b020 con 0x7fe7a00b48c0 2023-10-08T05:35:53.259 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:53.258+0000 7f3633822040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:35:53.301 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.300+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1125868518 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2150863d-4f36-4fdb-becd-23d5e739c8bc", "id": 0} v 0) v1 -- 0x7fe7a00acec0 con 0x7fe7a00b48c0 2023-10-08T05:35:53.308 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.307+0000 7fe7967fc640 1 -- 172.21.15.181:0/1125868518 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2150863d-4f36-4fdb-becd-23d5e739c8bc", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe7a00acec0 con 0x7fe7a00b48c0 2023-10-08T05:35:53.309 INFO:teuthology.orchestra.run.smithi181.stdout:0 2023-10-08T05:35:53.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.309+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1125868518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 msgr2=0x7fe7a00c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.310 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.309+0000 7fe7a7cf2640 1 --2- 172.21.15.181:0/1125868518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a00c5420 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe790033000 tx=0x7fe7900046c0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.311 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.309+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1125868518 shutdown_connections 2023-10-08T05:35:53.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.309+0000 7fe7a7cf2640 1 --2- 172.21.15.181:0/1125868518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7a0159940 0x7fe7a00ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.309+0000 7fe7a7cf2640 1 --2- 172.21.15.181:0/1125868518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7a00b48c0 0x7fe7a00c5420 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.316 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.309+0000 7fe7a7cf2640 1 --2- 172.21.15.181:0/1125868518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0069d50 0x7fe7a014bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.309+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1125868518 >> 172.21.15.181:0/1125868518 conn(0x7fe7a00cba60 msgr2=0x7fe7a006b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:53.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.309+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1125868518 shutdown_connections 2023-10-08T05:35:53.317 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.309+0000 7fe7a7cf2640 1 -- 172.21.15.181:0/1125868518 wait complete. 2023-10-08T05:35:53.325 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:53.323+0000 7f3633822040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:35:53.326 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 8c8f42c7-3b3d-4fea-a994-593607500020 1 2023-10-08T05:35:53.379 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:53.377+0000 7f3633822040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:35:53.444 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:53.443+0000 7f3633822040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:35:53.496 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:53.495+0000 7f3633822040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:35:53.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.513+0000 7ff9ba256640 1 Processor -- start 2023-10-08T05:35:53.514 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.513+0000 7ff9ba256640 1 -- start start 2023-10-08T05:35:53.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9ba256640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9b415cf80 0x7ff9b40b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9ba256640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b40b4b40 0x7ff9b40d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.516 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9ba256640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b40b54a0 con 0x7ff9b415cba0 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9ba256640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b40b4ff0 con 0x7ff9b415cf80 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9ba256640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40b8bb0 con 0x7ff9b40b4b40 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9b37fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff9b415cba0 0x7ff9b40b84a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57416/0 (socket says 172.21.15.181:57416) 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9b37fe640 1 -- 172.21.15.181:0/3336888916 learned_addr learned my addr 172.21.15.181:0/3336888916 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9b3fff640 1 --2- 172.21.15.181:0/3336888916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b40b4b40 0x7ff9b40d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9b2ffd640 1 --2- 172.21.15.181:0/3336888916 >> v2:172.21.15.70:3300/0 conn(0x7ff9b415cf80 0x7ff9b40b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9b27fc640 1 -- 172.21.15.181:0/3336888916 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738511890 0 0) 0x7ff9a4001110 con 0x7ff9b415cba0 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9b3fff640 1 -- 172.21.15.181:0/3336888916 >> v2:172.21.15.70:3300/0 conn(0x7ff9b415cf80 msgr2=0x7ff9b40b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9b3fff640 1 --2- 172.21.15.181:0/3336888916 >> v2:172.21.15.70:3300/0 conn(0x7ff9b415cf80 0x7ff9b40b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9b3fff640 1 -- 172.21.15.181:0/3336888916 >> v1:172.21.15.181:6789/0 conn(0x7ff9b415cba0 legacy=0x7ff9b40b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.517 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.515+0000 7ff9b3fff640 1 -- 172.21.15.181:0/3336888916 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b40b7130 con 0x7ff9b40b4b40 2023-10-08T05:35:53.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.516+0000 7ff9b3fff640 1 --2- 172.21.15.181:0/3336888916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b40b4b40 0x7ff9b40d0ee0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009dc0 tx=0x7ff9ac02be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5616c0c9073bf401 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:53.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.517+0000 7ff9b27fc640 1 -- 172.21.15.181:0/3336888916 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac02cc40 con 0x7ff9b40b4b40 2023-10-08T05:35:53.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.517+0000 7ff9b27fc640 1 -- 172.21.15.181:0/3336888916 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac02cda0 con 0x7ff9b40b4b40 2023-10-08T05:35:53.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.517+0000 7ff9b27fc640 1 -- 172.21.15.181:0/3336888916 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac034660 con 0x7ff9b40b4b40 2023-10-08T05:35:53.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.517+0000 7ff9ba256640 1 -- 172.21.15.181:0/3336888916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b40b4b40 msgr2=0x7ff9b40d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.517+0000 7ff9ba256640 1 --2- 172.21.15.181:0/3336888916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b40b4b40 0x7ff9b40d0ee0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009dc0 tx=0x7ff9ac02be90 comp rx=0 tx=0).stop 2023-10-08T05:35:53.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.517+0000 7ff9ba256640 1 -- 172.21.15.181:0/3336888916 shutdown_connections 2023-10-08T05:35:53.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.517+0000 7ff9ba256640 1 --2- 172.21.15.181:0/3336888916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b40b4b40 0x7ff9b40d0ee0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.519 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.517+0000 7ff9ba256640 1 --2- 172.21.15.181:0/3336888916 >> v2:172.21.15.70:3300/0 conn(0x7ff9b415cf80 0x7ff9b40b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.517+0000 7ff9ba256640 1 -- 172.21.15.181:0/3336888916 >> 172.21.15.181:0/3336888916 conn(0x7ff9b40718a0 msgr2=0x7ff9b4071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:53.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.518+0000 7ff9ba256640 1 -- 172.21.15.181:0/3336888916 shutdown_connections 2023-10-08T05:35:53.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.518+0000 7ff9ba256640 1 -- 172.21.15.181:0/3336888916 wait complete. 2023-10-08T05:35:53.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.519+0000 7ff9ba256640 1 Processor -- start 2023-10-08T05:35:53.520 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.519+0000 7ff9ba256640 1 -- start start 2023-10-08T05:35:53.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.520+0000 7ff9ba256640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b40b4b40 0x7ff9b4151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.520+0000 7ff9ba256640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b415cba0 0x7ff9b41541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.520+0000 7ff9ba256640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b415cf80 0x7ff9b40cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.520+0000 7ff9ba256640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b40a95e0 con 0x7ff9b40b4b40 2023-10-08T05:35:53.522 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.520+0000 7ff9ba256640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b4154c40 con 0x7ff9b415cf80 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.520+0000 7ff9ba256640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b4154f20 con 0x7ff9b415cba0 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b2ffd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b415cba0 0x7ff9b41541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b415cf80 0x7ff9b40cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b2ffd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b415cba0 0x7ff9b41541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55374/0 (socket says 172.21.15.181:55374) 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b2ffd640 1 -- 172.21.15.181:0/4133349085 learned_addr learned my addr 172.21.15.181:0/4133349085 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b37fe640 1 --2- 172.21.15.181:0/4133349085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b40b4b40 0x7ff9b4151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b3fff640 1 -- 172.21.15.181:0/4133349085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b415cba0 msgr2=0x7ff9b41541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b3fff640 1 --2- 172.21.15.181:0/4133349085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b415cba0 0x7ff9b41541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b3fff640 1 -- 172.21.15.181:0/4133349085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b40b4b40 msgr2=0x7ff9b4151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b3fff640 1 --2- 172.21.15.181:0/4133349085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b40b4b40 0x7ff9b4151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.523 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b3fff640 1 -- 172.21.15.181:0/4133349085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac009a70 con 0x7ff9b415cf80 2023-10-08T05:35:53.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.521+0000 7ff9b3fff640 1 --2- 172.21.15.181:0/4133349085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b415cf80 0x7ff9b40cd160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac033000 tx=0x7ff9ac004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:53.524 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.522+0000 7ff99bfff640 1 -- 172.21.15.181:0/4133349085 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac040650 con 0x7ff9b415cf80 2023-10-08T05:35:53.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.522+0000 7ff99bfff640 1 -- 172.21.15.181:0/4133349085 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac0407b0 con 0x7ff9b415cf80 2023-10-08T05:35:53.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.522+0000 7ff99bfff640 1 -- 172.21.15.181:0/4133349085 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac040b40 con 0x7ff9b415cf80 2023-10-08T05:35:53.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.522+0000 7ff9ba256640 1 -- 172.21.15.181:0/4133349085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b4155200 con 0x7ff9b415cf80 2023-10-08T05:35:53.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.522+0000 7ff9ba256640 1 -- 172.21.15.181:0/4133349085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b40cbdd0 con 0x7ff9b415cf80 2023-10-08T05:35:53.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.522+0000 7ff99bfff640 1 -- 172.21.15.181:0/4133349085 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff9ac049400 con 0x7ff9b415cf80 2023-10-08T05:35:53.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.522+0000 7ff99bfff640 1 -- 172.21.15.181:0/4133349085 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff9ac03e7b0 con 0x7ff9b415cf80 2023-10-08T05:35:53.525 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.523+0000 7ff9ba256640 1 -- 172.21.15.181:0/4133349085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b4155060 con 0x7ff9b415cf80 2023-10-08T05:35:53.525 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:53.524+0000 7f27c7643040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:35:53.527 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.526+0000 7ff99bfff640 1 -- 172.21.15.181:0/4133349085 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ff9ac067b40 con 0x7ff9b415cf80 2023-10-08T05:35:53.603 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:53.602+0000 7f27c7643040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:35:53.604 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.603+0000 7ff9ba256640 1 -- 172.21.15.181:0/4133349085 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8c8f42c7-3b3d-4fea-a994-593607500020", "id": 1} v 0) v1 -- 0x7ff9b404e160 con 0x7ff9b415cf80 2023-10-08T05:35:53.611 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.610+0000 7ff99bfff640 1 -- 172.21.15.181:0/4133349085 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8c8f42c7-3b3d-4fea-a994-593607500020", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff9ac056030 con 0x7ff9b415cf80 2023-10-08T05:35:53.611 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-08T05:35:53.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.611+0000 7ff9ba256640 1 -- 172.21.15.181:0/4133349085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b415cf80 msgr2=0x7ff9b40cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.611+0000 7ff9ba256640 1 --2- 172.21.15.181:0/4133349085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b415cf80 0x7ff9b40cd160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac033000 tx=0x7ff9ac004a40 comp rx=0 tx=0).stop 2023-10-08T05:35:53.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.611+0000 7ff9ba256640 1 -- 172.21.15.181:0/4133349085 shutdown_connections 2023-10-08T05:35:53.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.611+0000 7ff9ba256640 1 --2- 172.21.15.181:0/4133349085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b415cf80 0x7ff9b40cd160 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.611+0000 7ff9ba256640 1 --2- 172.21.15.181:0/4133349085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b415cba0 0x7ff9b41541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.612 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.611+0000 7ff9ba256640 1 --2- 172.21.15.181:0/4133349085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b40b4b40 0x7ff9b4151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.611+0000 7ff9ba256640 1 -- 172.21.15.181:0/4133349085 >> 172.21.15.181:0/4133349085 conn(0x7ff9b40718a0 msgr2=0x7ff9b40d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:53.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.611+0000 7ff9ba256640 1 -- 172.21.15.181:0/4133349085 shutdown_connections 2023-10-08T05:35:53.613 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.611+0000 7ff9ba256640 1 -- 172.21.15.181:0/4133349085 wait complete. 2023-10-08T05:35:53.616 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:53.615+0000 7f3633822040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:35:53.629 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new f711dd07-9194-464e-9101-3794ef6132c3 2 2023-10-08T05:35:53.700 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:53.699+0000 7f3633822040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:35:53.756 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:53.755+0000 7f3633822040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:35:53.819 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:53.818+0000 7f3633822040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:35:53.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.824+0000 7fae7ca20640 1 Processor -- start 2023-10-08T05:35:53.825 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.824+0000 7fae7ca20640 1 -- start start 2023-10-08T05:35:53.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.826+0000 7fae7ca20640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780cf2b0 0x7fae780d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.826+0000 7fae7ca20640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae780cf690 0x7fae780ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.826+0000 7fae7ca20640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae780cd520 con 0x7fae780d1100 2023-10-08T05:35:53.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.826+0000 7fae7ca20640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae780d1910 con 0x7fae780cf690 2023-10-08T05:35:53.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.826+0000 7fae7ca20640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae780ce080 con 0x7fae780cf2b0 2023-10-08T05:35:53.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.826+0000 7fae76d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780cf2b0 0x7fae780d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.828 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.826+0000 7fae76d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780cf2b0 0x7fae780d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55376/0 (socket says 172.21.15.181:55376) 2023-10-08T05:35:53.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.826+0000 7fae76d9d640 1 -- 172.21.15.181:0/3582183936 learned_addr learned my addr 172.21.15.181:0/3582183936 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:53.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.826+0000 7fae7659c640 1 --2- 172.21.15.181:0/3582183936 >> v2:172.21.15.70:3300/0 conn(0x7fae780cf690 0x7fae780ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.827+0000 7fae76d9d640 1 -- 172.21.15.181:0/3582183936 >> v2:172.21.15.70:3300/0 conn(0x7fae780cf690 msgr2=0x7fae780ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:53.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.827+0000 7fae76d9d640 1 --2- 172.21.15.181:0/3582183936 >> v2:172.21.15.70:3300/0 conn(0x7fae780cf690 0x7fae780ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.827+0000 7fae76d9d640 1 -- 172.21.15.181:0/3582183936 >> v1:172.21.15.70:6789/0 conn(0x7fae780d1100 legacy=0x7fae7815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:53.829 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.827+0000 7fae76d9d640 1 -- 172.21.15.181:0/3582183936 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae780cd750 con 0x7fae780cf2b0 2023-10-08T05:35:53.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.828+0000 7fae76d9d640 1 --2- 172.21.15.181:0/3582183936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780cf2b0 0x7fae780d34d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae68009870 tx=0x7fae6802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de277549686ff81c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:53.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.828+0000 7fae75d9b640 1 -- 172.21.15.181:0/3582183936 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae680317c0 con 0x7fae780cf2b0 2023-10-08T05:35:53.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.828+0000 7fae75d9b640 1 -- 172.21.15.181:0/3582183936 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae68031920 con 0x7fae780cf2b0 2023-10-08T05:35:53.830 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.828+0000 7fae75d9b640 1 -- 172.21.15.181:0/3582183936 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68031cb0 con 0x7fae780cf2b0 2023-10-08T05:35:53.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.829+0000 7fae7ca20640 1 -- 172.21.15.181:0/3582183936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780cf2b0 msgr2=0x7fae780d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.829+0000 7fae7ca20640 1 --2- 172.21.15.181:0/3582183936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780cf2b0 0x7fae780d34d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae68009870 tx=0x7fae6802b860 comp rx=0 tx=0).stop 2023-10-08T05:35:53.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.829+0000 7fae7ca20640 1 -- 172.21.15.181:0/3582183936 shutdown_connections 2023-10-08T05:35:53.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.829+0000 7fae7ca20640 1 --2- 172.21.15.181:0/3582183936 >> v2:172.21.15.70:3300/0 conn(0x7fae780cf690 0x7fae780ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.829+0000 7fae7ca20640 1 --2- 172.21.15.181:0/3582183936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780cf2b0 0x7fae780d34d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.829+0000 7fae7ca20640 1 -- 172.21.15.181:0/3582183936 >> 172.21.15.181:0/3582183936 conn(0x7fae7809bc40 msgr2=0x7fae780d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:53.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.829+0000 7fae7ca20640 1 -- 172.21.15.181:0/3582183936 shutdown_connections 2023-10-08T05:35:53.831 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.829+0000 7fae7ca20640 1 -- 172.21.15.181:0/3582183936 wait complete. 2023-10-08T05:35:53.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.831+0000 7fae7ca20640 1 Processor -- start 2023-10-08T05:35:53.832 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.831+0000 7fae7ca20640 1 -- start start 2023-10-08T05:35:53.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7ca20640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cf2b0 0x7fae7815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7ca20640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae780cf690 0x7fae7815ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7ca20640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780d1100 0x7fae7809da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:53.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7ca20640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7815a6f0 con 0x7fae780cf690 2023-10-08T05:35:53.834 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7ca20640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7815ef70 con 0x7fae780cf2b0 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7ca20640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7815d990 con 0x7fae780d1100 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780d1100 0x7fae7809da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae76d9d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cf2b0 0x7fae7815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780d1100 0x7fae7809da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55380/0 (socket says 172.21.15.181:55380) 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7759e640 1 -- 172.21.15.181:0/1622638928 learned_addr learned my addr 172.21.15.181:0/1622638928 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7659c640 1 --2- 172.21.15.181:0/1622638928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae780cf690 0x7fae7815ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7759e640 1 -- 172.21.15.181:0/1622638928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cf2b0 msgr2=0x7fae7815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7759e640 1 --2- 172.21.15.181:0/1622638928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cf2b0 0x7fae7815a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7759e640 1 -- 172.21.15.181:0/1622638928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae780cf690 msgr2=0x7fae7815ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7759e640 1 --2- 172.21.15.181:0/1622638928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae780cf690 0x7fae7815ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7759e640 1 -- 172.21.15.181:0/1622638928 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae680094d0 con 0x7fae780d1100 2023-10-08T05:35:53.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.832+0000 7fae7659c640 1 --2- 172.21.15.181:0/1622638928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae780cf690 0x7fae7815ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:35:53.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.833+0000 7fae7759e640 1 --2- 172.21.15.181:0/1622638928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780d1100 0x7fae7809da80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c780 tx=0x7fae7000cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:53.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.833+0000 7fae5f7fe640 1 -- 172.21.15.181:0/1622638928 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70004b60 con 0x7fae780d1100 2023-10-08T05:35:53.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.833+0000 7fae5f7fe640 1 -- 172.21.15.181:0/1622638928 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7000ce90 con 0x7fae780d1100 2023-10-08T05:35:53.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.833+0000 7fae5f7fe640 1 -- 172.21.15.181:0/1622638928 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7000a850 con 0x7fae780d1100 2023-10-08T05:35:53.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.833+0000 7fae7ca20640 1 -- 172.21.15.181:0/1622638928 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7815dc70 con 0x7fae780d1100 2023-10-08T05:35:53.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.833+0000 7fae7ca20640 1 -- 172.21.15.181:0/1622638928 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7815e230 con 0x7fae780d1100 2023-10-08T05:35:53.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.833+0000 7fae5f7fe640 1 -- 172.21.15.181:0/1622638928 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fae7000a9b0 con 0x7fae780d1100 2023-10-08T05:35:53.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.833+0000 7fae5f7fe640 1 -- 172.21.15.181:0/1622638928 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fae70005030 con 0x7fae780d1100 2023-10-08T05:35:53.836 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.834+0000 7fae7ca20640 1 -- 172.21.15.181:0/1622638928 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae78074150 con 0x7fae780d1100 2023-10-08T05:35:53.839 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.838+0000 7fae5f7fe640 1 -- 172.21.15.181:0/1622638928 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fae70006e50 con 0x7fae780d1100 2023-10-08T05:35:53.879 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:53.878+0000 7f3633822040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:35:53.918 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.917+0000 7fae7ca20640 1 -- 172.21.15.181:0/1622638928 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f711dd07-9194-464e-9101-3794ef6132c3", "id": 2} v 0) v1 -- 0x7fae780d0070 con 0x7fae780d1100 2023-10-08T05:35:53.924 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.923+0000 7fae5f7fe640 1 -- 172.21.15.181:0/1622638928 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f711dd07-9194-464e-9101-3794ef6132c3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fae7000ad30 con 0x7fae780d1100 2023-10-08T05:35:53.924 INFO:teuthology.orchestra.run.smithi181.stdout:2 2023-10-08T05:35:53.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.924+0000 7fae7ca20640 1 -- 172.21.15.181:0/1622638928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780d1100 msgr2=0x7fae7809da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:53.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.924+0000 7fae7ca20640 1 --2- 172.21.15.181:0/1622638928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780d1100 0x7fae7809da80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c780 tx=0x7fae7000cc40 comp rx=0 tx=0).stop 2023-10-08T05:35:53.926 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.924+0000 7fae7ca20640 1 -- 172.21.15.181:0/1622638928 shutdown_connections 2023-10-08T05:35:53.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.924+0000 7fae7ca20640 1 --2- 172.21.15.181:0/1622638928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae780d1100 0x7fae7809da80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.924+0000 7fae7ca20640 1 --2- 172.21.15.181:0/1622638928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae780cf690 0x7fae7815ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.924+0000 7fae7ca20640 1 --2- 172.21.15.181:0/1622638928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae780cf2b0 0x7fae7815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:53.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.924+0000 7fae7ca20640 1 -- 172.21.15.181:0/1622638928 >> 172.21.15.181:0/1622638928 conn(0x7fae7809bc40 msgr2=0x7fae780b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:53.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.925+0000 7fae7ca20640 1 -- 172.21.15.181:0/1622638928 shutdown_connections 2023-10-08T05:35:53.927 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:53.925+0000 7fae7ca20640 1 -- 172.21.15.181:0/1622638928 wait complete. 2023-10-08T05:35:53.943 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 786e1a64-be49-44e4-8b17-f3721d79b96f 3 2023-10-08T05:35:54.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.132+0000 7fa25f59e640 1 Processor -- start 2023-10-08T05:35:54.134 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.132+0000 7fa25f59e640 1 -- start start 2023-10-08T05:35:54.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.134+0000 7fa25f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa260069fc0 0x7fa260071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.134+0000 7fa25f59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa26006ec00 0x7fa2600c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.134+0000 7fa25f59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa260160d10 con 0x7fa2600ca210 2023-10-08T05:35:54.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.134+0000 7fa25f59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa260072070 con 0x7fa26006ec00 2023-10-08T05:35:54.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.134+0000 7fa25f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa260071580 con 0x7fa260069fc0 2023-10-08T05:35:54.135 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.134+0000 7fa257fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa26006ec00 0x7fa2600c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.134+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa260069fc0 0x7fa260071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.134+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa260069fc0 0x7fa260071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55382/0 (socket says 172.21.15.181:55382) 2023-10-08T05:35:54.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.134+0000 7fa25e59c640 1 -- 172.21.15.181:0/2447475169 learned_addr learned my addr 172.21.15.181:0/2447475169 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:54.136 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.134+0000 7fa257fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa26006ec00 0x7fa2600c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51846/0 (socket says 172.21.15.181:51846) 2023-10-08T05:35:54.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.135+0000 7fa25e59c640 1 -- 172.21.15.181:0/2447475169 >> v2:172.21.15.181:3300/0 conn(0x7fa26006ec00 msgr2=0x7fa2600c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:54.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.135+0000 7fa25e59c640 1 --2- 172.21.15.181:0/2447475169 >> v2:172.21.15.181:3300/0 conn(0x7fa26006ec00 0x7fa2600c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.135+0000 7fa25e59c640 1 -- 172.21.15.181:0/2447475169 >> v1:172.21.15.70:6789/0 conn(0x7fa2600ca210 legacy=0x7fa260160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:54.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.135+0000 7fa25e59c640 1 -- 172.21.15.181:0/2447475169 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa260072330 con 0x7fa260069fc0 2023-10-08T05:35:54.137 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.136+0000 7fa25e59c640 1 --2- 172.21.15.181:0/2447475169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa260069fc0 0x7fa260071860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa250009870 tx=0x7fa25002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38446561c3a73158 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:54.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.136+0000 7fa25dd9b640 1 -- 172.21.15.181:0/2447475169 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250030840 con 0x7fa260069fc0 2023-10-08T05:35:54.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.136+0000 7fa25dd9b640 1 -- 172.21.15.181:0/2447475169 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2500309a0 con 0x7fa260069fc0 2023-10-08T05:35:54.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.136+0000 7fa25dd9b640 1 -- 172.21.15.181:0/2447475169 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250030d30 con 0x7fa260069fc0 2023-10-08T05:35:54.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.137+0000 7fa25f59e640 1 -- 172.21.15.181:0/2447475169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa260069fc0 msgr2=0x7fa260071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.137+0000 7fa25f59e640 1 --2- 172.21.15.181:0/2447475169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa260069fc0 0x7fa260071860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa250009870 tx=0x7fa25002b980 comp rx=0 tx=0).stop 2023-10-08T05:35:54.138 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.137+0000 7fa25f59e640 1 -- 172.21.15.181:0/2447475169 shutdown_connections 2023-10-08T05:35:54.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.137+0000 7fa25f59e640 1 --2- 172.21.15.181:0/2447475169 >> v2:172.21.15.181:3300/0 conn(0x7fa26006ec00 0x7fa2600c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.137+0000 7fa25f59e640 1 --2- 172.21.15.181:0/2447475169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa260069fc0 0x7fa260071860 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.137+0000 7fa25f59e640 1 -- 172.21.15.181:0/2447475169 >> 172.21.15.181:0/2447475169 conn(0x7fa2600cf5f0 msgr2=0x7fa2600cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:54.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.137+0000 7fa25f59e640 1 -- 172.21.15.181:0/2447475169 shutdown_connections 2023-10-08T05:35:54.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.137+0000 7fa25f59e640 1 -- 172.21.15.181:0/2447475169 wait complete. 2023-10-08T05:35:54.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.139+0000 7fa25f59e640 1 Processor -- start 2023-10-08T05:35:54.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.139+0000 7fa25f59e640 1 -- start start 2023-10-08T05:35:54.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa260069fc0 0x7fa260152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.142 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa26006ec00 0x7fa2601e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2600ca210 0x7fa2601e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa260153430 con 0x7fa26006ec00 2023-10-08T05:35:54.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25f59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600af7f0 con 0x7fa260069fc0 2023-10-08T05:35:54.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600afab0 con 0x7fa2600ca210 2023-10-08T05:35:54.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa260069fc0 0x7fa260152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.143 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25e59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa260069fc0 0x7fa260152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51854/0 (socket says 172.21.15.181:51854) 2023-10-08T05:35:54.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25e59c640 1 -- 172.21.15.181:0/745930910 learned_addr learned my addr 172.21.15.181:0/745930910 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:54.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25ed9d640 1 --2- 172.21.15.181:0/745930910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2600ca210 0x7fa2601e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa257fff640 1 --2- 172.21.15.181:0/745930910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa26006ec00 0x7fa2601e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25e59c640 1 -- 172.21.15.181:0/745930910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2600ca210 msgr2=0x7fa2601e63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25e59c640 1 --2- 172.21.15.181:0/745930910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2600ca210 0x7fa2601e63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25e59c640 1 -- 172.21.15.181:0/745930910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa26006ec00 msgr2=0x7fa2601e5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25e59c640 1 --2- 172.21.15.181:0/745930910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa26006ec00 0x7fa2601e5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.140+0000 7fa25e59c640 1 -- 172.21.15.181:0/745930910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2500094d0 con 0x7fa260069fc0 2023-10-08T05:35:54.144 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.141+0000 7fa25e59c640 1 --2- 172.21.15.181:0/745930910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa260069fc0 0x7fa260152ef0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa250000c00 tx=0x7fa250004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:54.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.141+0000 7fa256ffd640 1 -- 172.21.15.181:0/745930910 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250039c90 con 0x7fa260069fc0 2023-10-08T05:35:54.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.141+0000 7fa256ffd640 1 -- 172.21.15.181:0/745930910 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa250039df0 con 0x7fa260069fc0 2023-10-08T05:35:54.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.141+0000 7fa256ffd640 1 -- 172.21.15.181:0/745930910 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250043640 con 0x7fa260069fc0 2023-10-08T05:35:54.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.141+0000 7fa25f59e640 1 -- 172.21.15.181:0/745930910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2600afd90 con 0x7fa260069fc0 2023-10-08T05:35:54.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.141+0000 7fa25f59e640 1 -- 172.21.15.181:0/745930910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2600b0350 con 0x7fa260069fc0 2023-10-08T05:35:54.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.141+0000 7fa256ffd640 1 -- 172.21.15.181:0/745930910 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa250043aa0 con 0x7fa260069fc0 2023-10-08T05:35:54.145 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.142+0000 7fa256ffd640 1 -- 172.21.15.181:0/745930910 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fa25003e070 con 0x7fa260069fc0 2023-10-08T05:35:54.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.142+0000 7fa25f59e640 1 -- 172.21.15.181:0/745930910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa260074150 con 0x7fa260069fc0 2023-10-08T05:35:54.146 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.145+0000 7fa256ffd640 1 -- 172.21.15.181:0/745930910 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa250034070 con 0x7fa260069fc0 2023-10-08T05:35:54.223 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.222+0000 7fa25f59e640 1 -- 172.21.15.181:0/745930910 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "786e1a64-be49-44e4-8b17-f3721d79b96f", "id": 3} v 0) v1 -- 0x7fa2600b5570 con 0x7fa260069fc0 2023-10-08T05:35:54.227 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:54.226+0000 7f27c7643040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:35:54.230 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.229+0000 7fa256ffd640 1 -- 172.21.15.181:0/745930910 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "786e1a64-be49-44e4-8b17-f3721d79b96f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa25004d460 con 0x7fa260069fc0 2023-10-08T05:35:54.230 INFO:teuthology.orchestra.run.smithi181.stdout:3 2023-10-08T05:35:54.231 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.230+0000 7fa25f59e640 1 -- 172.21.15.181:0/745930910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa260069fc0 msgr2=0x7fa260152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.230+0000 7fa25f59e640 1 --2- 172.21.15.181:0/745930910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa260069fc0 0x7fa260152ef0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa250000c00 tx=0x7fa250004730 comp rx=0 tx=0).stop 2023-10-08T05:35:54.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.230+0000 7fa25f59e640 1 -- 172.21.15.181:0/745930910 shutdown_connections 2023-10-08T05:35:54.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.230+0000 7fa25f59e640 1 --2- 172.21.15.181:0/745930910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2600ca210 0x7fa2601e63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.230+0000 7fa25f59e640 1 --2- 172.21.15.181:0/745930910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa26006ec00 0x7fa2601e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.230+0000 7fa25f59e640 1 --2- 172.21.15.181:0/745930910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa260069fc0 0x7fa260152ef0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.232 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.230+0000 7fa25f59e640 1 -- 172.21.15.181:0/745930910 >> 172.21.15.181:0/745930910 conn(0x7fa2600cf5f0 msgr2=0x7fa2600abb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:54.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.230+0000 7fa25f59e640 1 -- 172.21.15.181:0/745930910 shutdown_connections 2023-10-08T05:35:54.233 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.230+0000 7fa25f59e640 1 -- 172.21.15.181:0/745930910 wait complete. 2023-10-08T05:35:54.247 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new cf10be13-ea48-4ddf-9c23-ec14cff3c09f 4 2023-10-08T05:35:54.288 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:54.287+0000 7f27c7643040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:35:54.338 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:54.337+0000 7f27c7643040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:35:54.353 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:54.352+0000 7f3633822040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:35:54.401 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:54.400+0000 7f27c7643040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:35:54.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.442+0000 7ffac2683640 1 Processor -- start 2023-10-08T05:35:54.444 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.443+0000 7ffac2683640 1 -- start start 2023-10-08T05:35:54.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.444+0000 7ffac2683640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffabc15cf30 0x7ffabc0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.445 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.444+0000 7ffac2683640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffabc0cd520 con 0x7ffabc0cf2b0 2023-10-08T05:35:54.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.444+0000 7ffac2683640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffabc0d1910 con 0x7ffabc0cf690 2023-10-08T05:35:54.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.444+0000 7ffac2683640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffabc0ce080 con 0x7ffabc15cf30 2023-10-08T05:35:54.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.445+0000 7ffac0e80640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ffabc0cf690 0x7ffabc15c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57430/0 (socket says 172.21.15.181:57430) 2023-10-08T05:35:54.446 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.445+0000 7ffac0e80640 1 -- 172.21.15.181:0/358719388 learned_addr learned my addr 172.21.15.181:0/358719388 (peer_addr_for_me v1:172.21.15.181:0/0) 2023-10-08T05:35:54.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.445+0000 7ffac1e82640 1 --2- 172.21.15.181:0/358719388 >> v2:172.21.15.70:3300/0 conn(0x7ffabc15cf30 0x7ffabc0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.448 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.445+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576173447 0 0) 0x7ffabc0d1910 con 0x7ffabc0cf690 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.445+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaa8003520 con 0x7ffabc0cf690 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.445+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1918980081 0 0) 0x7ffaa8003520 con 0x7ffabc0cf690 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.445+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 >> v2:172.21.15.70:3300/0 conn(0x7ffabc15cf30 msgr2=0x7ffabc0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.445+0000 7ffab3fff640 1 --2- 172.21.15.181:0/358719388 >> v2:172.21.15.70:3300/0 conn(0x7ffabc15cf30 0x7ffabc0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.445+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 >> v1:172.21.15.70:6789/0 conn(0x7ffabc0cf2b0 legacy=0x7ffabc0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 --> v1:172.21.15.181:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0cd750 con 0x7ffabc0cf690 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 <== mon.1 v1:172.21.15.181:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ffaac001670 con 0x7ffabc0cf690 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 >> v1:172.21.15.181:6789/0 conn(0x7ffabc0cf690 legacy=0x7ffabc15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffab3fff640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaa8003970 0x7ffaa8003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffab3fff640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8004370 0x7ffaa8004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffab3fff640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa80050e0 0x7ffaa800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0ce080 con 0x7ffaa80050e0 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa800c020 con 0x7ffaa8003970 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa800c1e0 con 0x7ffaa8004370 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffac1681640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaa8003970 0x7ffaa8003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffac0e80640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8004370 0x7ffaa8004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffac1e82640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa80050e0 0x7ffaa800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffac1681640 1 -- 172.21.15.181:0/358719388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8004370 msgr2=0x7ffaa8004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffac1681640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8004370 0x7ffaa8004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffac1681640 1 -- 172.21.15.181:0/358719388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa80050e0 msgr2=0x7ffaa800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffac1681640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa80050e0 0x7ffaa800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.450 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.446+0000 7ffac1681640 1 -- 172.21.15.181:0/358719388 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8003520 con 0x7ffaa8003970 2023-10-08T05:35:54.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.447+0000 7ffac1e82640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaa80050e0 0x7ffaa800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:35:54.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.447+0000 7ffac1e82640 1 -- 172.21.15.181:0/358719388 reap_dead start 2023-10-08T05:35:54.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.447+0000 7ffac1681640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaa8003970 0x7ffaa8003d50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffab8001860 tx=0x7ffaa80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c9149339aa8af6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:54.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.448+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8033410 con 0x7ffaa8003970 2023-10-08T05:35:54.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.448+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab8033590 con 0x7ffaa8003970 2023-10-08T05:35:54.451 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.448+0000 7ffab3fff640 1 -- 172.21.15.181:0/358719388 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8033940 con 0x7ffaa8003970 2023-10-08T05:35:54.451 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:54.449+0000 7f27c7643040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:35:54.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.448+0000 7ffac2683640 1 -- 172.21.15.181:0/358719388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaa8003970 msgr2=0x7ffaa8003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.448+0000 7ffac2683640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaa8003970 0x7ffaa8003d50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ffab8001860 tx=0x7ffaa80057f0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.448+0000 7ffac2683640 1 -- 172.21.15.181:0/358719388 shutdown_connections 2023-10-08T05:35:54.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.448+0000 7ffac2683640 1 --2- 172.21.15.181:0/358719388 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaa8003970 0x7ffaa8003d50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.454 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.448+0000 7ffac2683640 1 -- 172.21.15.181:0/358719388 >> 172.21.15.181:0/358719388 conn(0x7ffabc09bc40 msgr2=0x7ffabc0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.448+0000 7ffac2683640 1 -- 172.21.15.181:0/358719388 shutdown_connections 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.449+0000 7ffac2683640 1 -- 172.21.15.181:0/358719388 wait complete. 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.450+0000 7ffac2683640 1 Processor -- start 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.450+0000 7ffac2683640 1 -- start start 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.451+0000 7ffac2683640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8003970 0x7ffabc0b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.451+0000 7ffac2683640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffabc1f5810 0x7ffabc0d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.451+0000 7ffac2683640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc1f5c90 0x7ffabc0d2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.451+0000 7ffac2683640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1f6740 con 0x7ffabc1f5810 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.451+0000 7ffac2683640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0cb110 con 0x7ffabc1f5c90 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.451+0000 7ffac2683640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0cab20 con 0x7ffaa8003970 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1681640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8003970 0x7ffabc0b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.455 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1681640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8003970 0x7ffabc0b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55412/0 (socket says 172.21.15.181:55412) 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1681640 1 -- 172.21.15.181:0/3034964139 learned_addr learned my addr 172.21.15.181:0/3034964139 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1e82640 1 --2- 172.21.15.181:0/3034964139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc1f5c90 0x7ffabc0d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac0e80640 1 --2- 172.21.15.181:0/3034964139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffabc1f5810 0x7ffabc0d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1681640 1 -- 172.21.15.181:0/3034964139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc1f5c90 msgr2=0x7ffabc0d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1681640 1 --2- 172.21.15.181:0/3034964139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc1f5c90 0x7ffabc0d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1681640 1 -- 172.21.15.181:0/3034964139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffabc1f5810 msgr2=0x7ffabc0d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1681640 1 --2- 172.21.15.181:0/3034964139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffabc1f5810 0x7ffabc0d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1681640 1 -- 172.21.15.181:0/3034964139 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc1f6180 con 0x7ffaa8003970 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1e82640 1 --2- 172.21.15.181:0/3034964139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc1f5c90 0x7ffabc0d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.452+0000 7ffac1681640 1 --2- 172.21.15.181:0/3034964139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8003970 0x7ffabc0b7d20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffab802bd00 tx=0x7ffab8001230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.453+0000 7ffab1ffb640 1 -- 172.21.15.181:0/3034964139 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8007320 con 0x7ffaa8003970 2023-10-08T05:35:54.456 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.453+0000 7ffab1ffb640 1 -- 172.21.15.181:0/3034964139 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab8007480 con 0x7ffaa8003970 2023-10-08T05:35:54.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.453+0000 7ffac2683640 1 -- 172.21.15.181:0/3034964139 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc1f6460 con 0x7ffaa8003970 2023-10-08T05:35:54.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.453+0000 7ffab1ffb640 1 -- 172.21.15.181:0/3034964139 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab80067d0 con 0x7ffaa8003970 2023-10-08T05:35:54.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.453+0000 7ffac2683640 1 -- 172.21.15.181:0/3034964139 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc0b8260 con 0x7ffaa8003970 2023-10-08T05:35:54.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.453+0000 7ffab1ffb640 1 -- 172.21.15.181:0/3034964139 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffab803a070 con 0x7ffaa8003970 2023-10-08T05:35:54.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.453+0000 7ffab1ffb640 1 -- 172.21.15.181:0/3034964139 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ffab803d410 con 0x7ffaa8003970 2023-10-08T05:35:54.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.454+0000 7ffac2683640 1 -- 172.21.15.181:0/3034964139 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffabc074150 con 0x7ffaa8003970 2023-10-08T05:35:54.457 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.456+0000 7ffab1ffb640 1 -- 172.21.15.181:0/3034964139 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ffab8065b40 con 0x7ffaa8003970 2023-10-08T05:35:54.522 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:54.521+0000 7f3633822040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:35:54.535 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.534+0000 7ffac2683640 1 -- 172.21.15.181:0/3034964139 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cf10be13-ea48-4ddf-9c23-ec14cff3c09f", "id": 4} v 0) v1 -- 0x7ffabc0af4a0 con 0x7ffaa8003970 2023-10-08T05:35:54.541 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.540+0000 7ffab1ffb640 1 -- 172.21.15.181:0/3034964139 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cf10be13-ea48-4ddf-9c23-ec14cff3c09f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffab8054030 con 0x7ffaa8003970 2023-10-08T05:35:54.542 INFO:teuthology.orchestra.run.smithi181.stdout:4 2023-10-08T05:35:54.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.542+0000 7ffac2683640 1 -- 172.21.15.181:0/3034964139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8003970 msgr2=0x7ffabc0b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.542+0000 7ffac2683640 1 --2- 172.21.15.181:0/3034964139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8003970 0x7ffabc0b7d20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffab802bd00 tx=0x7ffab8001230 comp rx=0 tx=0).stop 2023-10-08T05:35:54.543 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.542+0000 7ffac2683640 1 -- 172.21.15.181:0/3034964139 shutdown_connections 2023-10-08T05:35:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.542+0000 7ffac2683640 1 --2- 172.21.15.181:0/3034964139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffabc1f5c90 0x7ffabc0d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.542+0000 7ffac2683640 1 --2- 172.21.15.181:0/3034964139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffabc1f5810 0x7ffabc0d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.542+0000 7ffac2683640 1 --2- 172.21.15.181:0/3034964139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaa8003970 0x7ffabc0b7d20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.542+0000 7ffac2683640 1 -- 172.21.15.181:0/3034964139 >> 172.21.15.181:0/3034964139 conn(0x7ffabc09bc40 msgr2=0x7ffabc0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.542+0000 7ffac2683640 1 -- 172.21.15.181:0/3034964139 shutdown_connections 2023-10-08T05:35:54.544 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.542+0000 7ffac2683640 1 -- 172.21.15.181:0/3034964139 wait complete. 2023-10-08T05:35:54.560 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new a9980cf7-cff6-4408-a3d7-09730c6943cc 5 2023-10-08T05:35:54.564 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:54.563+0000 7f27c7643040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:35:54.644 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:54.642+0000 7f27c7643040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:35:54.680 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:54.678+0000 7f3633822040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:35:54.696 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:54.695+0000 7f27c7643040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:35:54.734 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:54.733+0000 7f3633822040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:35:54.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.746+0000 7f9a0a890640 1 Processor -- start 2023-10-08T05:35:54.747 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.746+0000 7f9a0a890640 1 -- start start 2023-10-08T05:35:54.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.747+0000 7f9a0a890640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a040b4d20 0x7f9a04161310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.748 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.747+0000 7f9a0a890640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9a040cd3f0 0x7f9a040d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.747+0000 7f9a0a890640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a040cdd90 con 0x7f9a0415cc60 2023-10-08T05:35:54.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.747+0000 7f9a0a890640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040ce070 con 0x7f9a040cd3f0 2023-10-08T05:35:54.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.747+0000 7f9a0a890640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040d1030 con 0x7f9a040b4d20 2023-10-08T05:35:54.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.747+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a040b4d20 0x7f9a04161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.747+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a040b4d20 0x7f9a04161310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55416/0 (socket says 172.21.15.181:55416) 2023-10-08T05:35:54.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.747+0000 7f9a03fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f9a0415cc60 0x7f9a040b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.181:57440/0 (socket says 172.21.15.181:57440) 2023-10-08T05:35:54.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.747+0000 7f99fbfff640 1 -- 172.21.15.181:0/2621812042 learned_addr learned my addr 172.21.15.181:0/2621812042 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:54.749 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.747+0000 7f9a08e2d640 1 --2- 172.21.15.181:0/2621812042 >> v2:172.21.15.181:3300/0 conn(0x7f9a040cd3f0 0x7f9a040d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.748+0000 7f99fbfff640 1 -- 172.21.15.181:0/2621812042 >> v2:172.21.15.181:3300/0 conn(0x7f9a040cd3f0 msgr2=0x7f9a040d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:54.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.748+0000 7f99fbfff640 1 --2- 172.21.15.181:0/2621812042 >> v2:172.21.15.181:3300/0 conn(0x7f9a040cd3f0 0x7f9a040d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.748+0000 7f99fbfff640 1 -- 172.21.15.181:0/2621812042 >> v1:172.21.15.181:6789/0 conn(0x7f9a0415cc60 legacy=0x7f9a040b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.750 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.748+0000 7f99fbfff640 1 -- 172.21.15.181:0/2621812042 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a040d1310 con 0x7f9a040b4d20 2023-10-08T05:35:54.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.748+0000 7f9a037fe640 1 -- 172.21.15.181:0/2621812042 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792465502 0 0) 0x7f9a040cdd90 con 0x7f9a0415cc60 2023-10-08T05:35:54.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.749+0000 7f99fbfff640 1 --2- 172.21.15.181:0/2621812042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a040b4d20 0x7f9a04161310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009870 tx=0x7f99ec02ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3211873976974 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:54.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.749+0000 7f9a037fe640 1 -- 172.21.15.181:0/2621812042 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec02f830 con 0x7f9a040b4d20 2023-10-08T05:35:54.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.749+0000 7f9a037fe640 1 -- 172.21.15.181:0/2621812042 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec02f990 con 0x7f9a040b4d20 2023-10-08T05:35:54.751 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.749+0000 7f9a037fe640 1 -- 172.21.15.181:0/2621812042 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec02fd20 con 0x7f9a040b4d20 2023-10-08T05:35:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.750+0000 7f9a0a890640 1 -- 172.21.15.181:0/2621812042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a040b4d20 msgr2=0x7f9a04161310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.750+0000 7f9a0a890640 1 --2- 172.21.15.181:0/2621812042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a040b4d20 0x7f9a04161310 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009870 tx=0x7f99ec02ba10 comp rx=0 tx=0).stop 2023-10-08T05:35:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.750+0000 7f9a0a890640 1 -- 172.21.15.181:0/2621812042 shutdown_connections 2023-10-08T05:35:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.750+0000 7f9a0a890640 1 --2- 172.21.15.181:0/2621812042 >> v2:172.21.15.181:3300/0 conn(0x7f9a040cd3f0 0x7f9a040d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.750+0000 7f9a0a890640 1 --2- 172.21.15.181:0/2621812042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a040b4d20 0x7f9a04161310 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.750+0000 7f9a0a890640 1 -- 172.21.15.181:0/2621812042 >> 172.21.15.181:0/2621812042 conn(0x7f9a040cf610 msgr2=0x7f9a040cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.750+0000 7f9a0a890640 1 -- 172.21.15.181:0/2621812042 shutdown_connections 2023-10-08T05:35:54.752 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.750+0000 7f9a0a890640 1 -- 172.21.15.181:0/2621812042 wait complete. 2023-10-08T05:35:54.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.752+0000 7f9a0a890640 1 Processor -- start 2023-10-08T05:35:54.753 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.752+0000 7f9a0a890640 1 -- start start 2023-10-08T05:35:54.755 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f9a0a890640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a040b4d20 0x7f9a040cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f9a0a890640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a040cd3f0 0x7f9a041597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f9a0a890640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a0415cc60 0x7f9a040cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f9a0a890640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040ca640 con 0x7f9a040b4d20 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f9a0a890640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040ca920 con 0x7f9a040cd3f0 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f9a0a890640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040cac00 con 0x7f9a0415cc60 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a040cd3f0 0x7f9a041597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f9a08e2d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a0415cc60 0x7f9a040cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a040cd3f0 0x7f9a041597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51884/0 (socket says 172.21.15.181:51884) 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f99fbfff640 1 -- 172.21.15.181:0/68711904 learned_addr learned my addr 172.21.15.181:0/68711904 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f9a08e2d640 1 --2- 172.21.15.181:0/68711904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a0415cc60 0x7f9a040cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55420/0 (socket says 172.21.15.181:55420) 2023-10-08T05:35:54.756 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f9a03fff640 1 --2- 172.21.15.181:0/68711904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a040b4d20 0x7f9a040cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.753+0000 7f9a08e2d640 1 -- 172.21.15.181:0/68711904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a040cd3f0 msgr2=0x7f9a041597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a08e2d640 1 --2- 172.21.15.181:0/68711904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a040cd3f0 0x7f9a041597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a08e2d640 1 -- 172.21.15.181:0/68711904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a040b4d20 msgr2=0x7f9a040cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a08e2d640 1 --2- 172.21.15.181:0/68711904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a040b4d20 0x7f9a040cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a08e2d640 1 -- 172.21.15.181:0/68711904 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec0094d0 con 0x7f9a0415cc60 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a03fff640 1 --2- 172.21.15.181:0/68711904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a040b4d20 0x7f9a040cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a08e2d640 1 --2- 172.21.15.181:0/68711904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a0415cc60 0x7f9a040cc4b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00efc0 tx=0x7f99fc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a017fa640 1 -- 172.21.15.181:0/68711904 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc009c00 con 0x7f9a0415cc60 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a017fa640 1 -- 172.21.15.181:0/68711904 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc00f040 con 0x7f9a0415cc60 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a017fa640 1 -- 172.21.15.181:0/68711904 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0146c0 con 0x7f9a0415cc60 2023-10-08T05:35:54.757 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a0a890640 1 -- 172.21.15.181:0/68711904 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a040caee0 con 0x7f9a0415cc60 2023-10-08T05:35:54.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.754+0000 7f9a0a890640 1 -- 172.21.15.181:0/68711904 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a040cbc20 con 0x7f9a0415cc60 2023-10-08T05:35:54.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.755+0000 7f9a017fa640 1 -- 172.21.15.181:0/68711904 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f99fc014b20 con 0x7f9a0415cc60 2023-10-08T05:35:54.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.755+0000 7f9a017fa640 1 -- 172.21.15.181:0/68711904 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f99fc014dd0 con 0x7f9a0415cc60 2023-10-08T05:35:54.758 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:54.755+0000 7f27c7643040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:35:54.758 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.755+0000 7f9a0a890640 1 -- 172.21.15.181:0/68711904 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0404e160 con 0x7f9a0415cc60 2023-10-08T05:35:54.759 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.758+0000 7f9a017fa640 1 -- 172.21.15.181:0/68711904 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f99fc03bb40 con 0x7f9a0415cc60 2023-10-08T05:35:54.810 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:54.809+0000 7f27c7643040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:35:54.835 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.834+0000 7f9a0a890640 1 -- 172.21.15.181:0/68711904 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a9980cf7-cff6-4408-a3d7-09730c6943cc", "id": 5} v 0) v1 -- 0x7f9a0415c460 con 0x7f9a0415cc60 2023-10-08T05:35:54.841 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.840+0000 7f9a017fa640 1 -- 172.21.15.181:0/68711904 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9980cf7-cff6-4408-a3d7-09730c6943cc", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f99fc02a070 con 0x7f9a0415cc60 2023-10-08T05:35:54.841 INFO:teuthology.orchestra.run.smithi181.stdout:5 2023-10-08T05:35:54.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.841+0000 7f9a0a890640 1 -- 172.21.15.181:0/68711904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a0415cc60 msgr2=0x7f9a040cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:54.842 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.841+0000 7f9a0a890640 1 --2- 172.21.15.181:0/68711904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a0415cc60 0x7f9a040cc4b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00efc0 tx=0x7f99fc00c490 comp rx=0 tx=0).stop 2023-10-08T05:35:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.841+0000 7f9a0a890640 1 -- 172.21.15.181:0/68711904 shutdown_connections 2023-10-08T05:35:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.841+0000 7f9a0a890640 1 --2- 172.21.15.181:0/68711904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a0415cc60 0x7f9a040cc4b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.841+0000 7f9a0a890640 1 --2- 172.21.15.181:0/68711904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a040cd3f0 0x7f9a041597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.841+0000 7f9a0a890640 1 --2- 172.21.15.181:0/68711904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a040b4d20 0x7f9a040cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.841+0000 7f9a0a890640 1 -- 172.21.15.181:0/68711904 >> 172.21.15.181:0/68711904 conn(0x7f9a040cf610 msgr2=0x7f9a040b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.842+0000 7f9a0a890640 1 -- 172.21.15.181:0/68711904 shutdown_connections 2023-10-08T05:35:54.843 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:54.842+0000 7f9a0a890640 1 -- 172.21.15.181:0/68711904 wait complete. 2023-10-08T05:35:54.851 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:54.850+0000 7f3633822040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:35:54.859 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new fa744b48-bf88-4b68-a372-697543a5a47e 6 2023-10-08T05:35:54.969 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:54.968+0000 7f3633822040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:35:55.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.049+0000 7fec3ffff640 1 Processor -- start 2023-10-08T05:35:55.050 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.049+0000 7fec3ffff640 1 -- start start 2023-10-08T05:35:55.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fec400ce510 0x7fec4015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3ffff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400cd520 con 0x7fec400cf690 2023-10-08T05:35:55.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3ffff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec400d1910 con 0x7fec400ce510 2023-10-08T05:35:55.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3ffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec400ce080 con 0x7fec400cf2b0 2023-10-08T05:35:55.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3effd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3effd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55432/0 (socket says 172.21.15.181:55432) 2023-10-08T05:35:55.052 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3effd640 1 -- 172.21.15.181:0/1432521812 learned_addr learned my addr 172.21.15.181:0/1432521812 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:55.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3f7fe640 1 --2- 172.21.15.181:0/1432521812 >> v2:172.21.15.70:3300/0 conn(0x7fec400ce510 0x7fec4015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3effd640 1 -- 172.21.15.181:0/1432521812 >> v2:172.21.15.70:3300/0 conn(0x7fec400ce510 msgr2=0x7fec4015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:55.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3effd640 1 --2- 172.21.15.181:0/1432521812 >> v2:172.21.15.70:3300/0 conn(0x7fec400ce510 0x7fec4015c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3effd640 1 -- 172.21.15.181:0/1432521812 >> v1:172.21.15.70:6789/0 conn(0x7fec400cf690 legacy=0x7fec400d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:55.053 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.051+0000 7fec3effd640 1 -- 172.21.15.181:0/1432521812 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400cd750 con 0x7fec400cf2b0 2023-10-08T05:35:55.054 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.052+0000 7fec3effd640 1 --2- 172.21.15.181:0/1432521812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400d34d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec30009870 tx=0x7fec3002b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f2f83bef4288c81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.053+0000 7fec3e7fc640 1 -- 172.21.15.181:0/1432521812 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300317c0 con 0x7fec400cf2b0 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.053+0000 7fec3e7fc640 1 -- 172.21.15.181:0/1432521812 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec30031920 con 0x7fec400cf2b0 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.053+0000 7fec3e7fc640 1 -- 172.21.15.181:0/1432521812 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30031cb0 con 0x7fec400cf2b0 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.053+0000 7fec3ffff640 1 -- 172.21.15.181:0/1432521812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 msgr2=0x7fec400d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.053+0000 7fec3ffff640 1 --2- 172.21.15.181:0/1432521812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400d34d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec30009870 tx=0x7fec3002b860 comp rx=0 tx=0).stop 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.053+0000 7fec3ffff640 1 -- 172.21.15.181:0/1432521812 shutdown_connections 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.053+0000 7fec3ffff640 1 --2- 172.21.15.181:0/1432521812 >> v2:172.21.15.70:3300/0 conn(0x7fec400ce510 0x7fec4015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.053+0000 7fec3ffff640 1 --2- 172.21.15.181:0/1432521812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400d34d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.053+0000 7fec3ffff640 1 -- 172.21.15.181:0/1432521812 >> 172.21.15.181:0/1432521812 conn(0x7fec4009bc40 msgr2=0x7fec400d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.053+0000 7fec3ffff640 1 -- 172.21.15.181:0/1432521812 shutdown_connections 2023-10-08T05:35:55.055 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.054+0000 7fec3ffff640 1 -- 172.21.15.181:0/1432521812 wait complete. 2023-10-08T05:35:55.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.055+0000 7fec3ffff640 1 Processor -- start 2023-10-08T05:35:55.056 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.055+0000 7fec3ffff640 1 -- start start 2023-10-08T05:35:55.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.056+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec400ce510 0x7fec401f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.056+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.056+0000 7fec3ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec400cf690 0x7fec400b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.056+0000 7fec3ffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b6c70 con 0x7fec400ce510 2023-10-08T05:35:55.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.056+0000 7fec3ffff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b6060 con 0x7fec400cf690 2023-10-08T05:35:55.058 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.056+0000 7fec3ffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b6340 con 0x7fec400cf2b0 2023-10-08T05:35:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.056+0000 7fec37fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.056+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec400cf690 0x7fec400b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.056+0000 7fec3f7fe640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec400cf690 0x7fec400b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51886/0 (socket says 172.21.15.181:51886) 2023-10-08T05:35:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec3f7fe640 1 -- 172.21.15.181:0/4286408016 learned_addr learned my addr 172.21.15.181:0/4286408016 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec37fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55444/0 (socket says 172.21.15.181:55444) 2023-10-08T05:35:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec3effd640 1 --2- 172.21.15.181:0/4286408016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec400ce510 0x7fec401f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec3f7fe640 1 -- 172.21.15.181:0/4286408016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 msgr2=0x7fec400b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec3f7fe640 1 --2- 172.21.15.181:0/4286408016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec3f7fe640 1 -- 172.21.15.181:0/4286408016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec400ce510 msgr2=0x7fec401f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.059 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec3f7fe640 1 --2- 172.21.15.181:0/4286408016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec400ce510 0x7fec401f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec3f7fe640 1 -- 172.21.15.181:0/4286408016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec300094d0 con 0x7fec400cf690 2023-10-08T05:35:55.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec37fff640 1 --2- 172.21.15.181:0/4286408016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:35:55.060 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec3f7fe640 1 --2- 172.21.15.181:0/4286408016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec400cf690 0x7fec400b5a70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fec3800cd70 tx=0x7fec3800a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:55.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.057+0000 7fec377fe640 1 -- 172.21.15.181:0/4286408016 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38003b90 con 0x7fec400cf690 2023-10-08T05:35:55.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.058+0000 7fec377fe640 1 -- 172.21.15.181:0/4286408016 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec38003cf0 con 0x7fec400cf690 2023-10-08T05:35:55.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.058+0000 7fec377fe640 1 -- 172.21.15.181:0/4286408016 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38004240 con 0x7fec400cf690 2023-10-08T05:35:55.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.058+0000 7fec3ffff640 1 -- 172.21.15.181:0/4286408016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec400b6620 con 0x7fec400cf690 2023-10-08T05:35:55.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.058+0000 7fec3ffff640 1 -- 172.21.15.181:0/4286408016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec401f2340 con 0x7fec400cf690 2023-10-08T05:35:55.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.058+0000 7fec377fe640 1 -- 172.21.15.181:0/4286408016 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fec380043a0 con 0x7fec400cf690 2023-10-08T05:35:55.061 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.058+0000 7fec377fe640 1 -- 172.21.15.181:0/4286408016 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fec38012070 con 0x7fec400cf690 2023-10-08T05:35:55.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.059+0000 7fec3ffff640 1 -- 172.21.15.181:0/4286408016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec400b6e20 con 0x7fec400cf690 2023-10-08T05:35:55.062 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.061+0000 7fec377fe640 1 -- 172.21.15.181:0/4286408016 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fec400b6e20 con 0x7fec400cf690 2023-10-08T05:35:55.137 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:55.135+0000 7f3633822040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:35:55.139 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.138+0000 7fec3ffff640 1 -- 172.21.15.181:0/4286408016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fa744b48-bf88-4b68-a372-697543a5a47e", "id": 6} v 0) v1 -- 0x7fec40074150 con 0x7fec400cf690 2023-10-08T05:35:55.148 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.147+0000 7fec377fe640 1 -- 172.21.15.181:0/4286408016 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fa744b48-bf88-4b68-a372-697543a5a47e", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fec3801f440 con 0x7fec400cf690 2023-10-08T05:35:55.148 INFO:teuthology.orchestra.run.smithi181.stdout:6 2023-10-08T05:35:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.148+0000 7fec3ffff640 1 -- 172.21.15.181:0/4286408016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec400cf690 msgr2=0x7fec400b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.148+0000 7fec3ffff640 1 --2- 172.21.15.181:0/4286408016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec400cf690 0x7fec400b5a70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fec3800cd70 tx=0x7fec3800a660 comp rx=0 tx=0).stop 2023-10-08T05:35:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.148+0000 7fec3ffff640 1 -- 172.21.15.181:0/4286408016 shutdown_connections 2023-10-08T05:35:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.148+0000 7fec3ffff640 1 --2- 172.21.15.181:0/4286408016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec400cf690 0x7fec400b5a70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.149 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.148+0000 7fec3ffff640 1 --2- 172.21.15.181:0/4286408016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec400cf2b0 0x7fec400b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.148+0000 7fec3ffff640 1 --2- 172.21.15.181:0/4286408016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec400ce510 0x7fec401f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.148+0000 7fec3ffff640 1 -- 172.21.15.181:0/4286408016 >> 172.21.15.181:0/4286408016 conn(0x7fec4009bc40 msgr2=0x7fec400af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.148+0000 7fec3ffff640 1 -- 172.21.15.181:0/4286408016 shutdown_connections 2023-10-08T05:35:55.150 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.148+0000 7fec3ffff640 1 -- 172.21.15.181:0/4286408016 wait complete. 2023-10-08T05:35:55.165 DEBUG:teuthology.orchestra.run.smithi181:> sudo ceph --cluster ceph osd new 2b6cdf62-a741-46b3-8df2-3f2098d41613 7 2023-10-08T05:35:55.190 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:55.188+0000 7f3633822040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:35:55.261 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:55.259+0000 7f27c7643040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:35:55.354 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:55.353+0000 7f3633822040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:35:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.356+0000 7f900de64640 1 Processor -- start 2023-10-08T05:35:55.358 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.357+0000 7f900de64640 1 -- start start 2023-10-08T05:35:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.358+0000 7f900de64640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f900815cd70 0x7f90081612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.359 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.358+0000 7f900de64640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f90080b4e10 0x7f90080d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.358+0000 7f900de64640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90080b8c30 con 0x7f90080cd4e0 2023-10-08T05:35:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.358+0000 7f900de64640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90080cdd80 con 0x7f90080b4e10 2023-10-08T05:35:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.358+0000 7f900de64640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90080ce060 con 0x7f900815cd70 2023-10-08T05:35:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.358+0000 7f90077fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f900815cd70 0x7f90081612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.358+0000 7f9007fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f90080b4e10 0x7f90080d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.359+0000 7f9007fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f90080b4e10 0x7f90080d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51890/0 (socket says 172.21.15.181:51890) 2023-10-08T05:35:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.359+0000 7f90077fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f900815cd70 0x7f90081612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55454/0 (socket says 172.21.15.181:55454) 2023-10-08T05:35:55.360 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.359+0000 7f9007fff640 1 -- 172.21.15.181:0/406780562 learned_addr learned my addr 172.21.15.181:0/406780562 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:55.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.359+0000 7f9007fff640 1 -- 172.21.15.181:0/406780562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f900815cd70 msgr2=0x7f90081612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:55.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.359+0000 7f9007fff640 1 --2- 172.21.15.181:0/406780562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f900815cd70 0x7f90081612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.359+0000 7f9007fff640 1 -- 172.21.15.181:0/406780562 >> v1:172.21.15.70:6789/0 conn(0x7f90080cd4e0 legacy=0x7f90080b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:55.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.359+0000 7f9007fff640 1 -- 172.21.15.181:0/406780562 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90080ce340 con 0x7f90080b4e10 2023-10-08T05:35:55.361 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.359+0000 7f90077fe640 1 --2- 172.21.15.181:0/406780562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f900815cd70 0x7f90081612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:35:55.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.361+0000 7f9007fff640 1 --2- 172.21.15.181:0/406780562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90080b4e10 0x7f90080d1650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f90080c9cb0 tx=0x7f900002be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=357957940ce4c6ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:55.362 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.361+0000 7f9006ffd640 1 -- 172.21.15.181:0/406780562 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900002ca30 con 0x7f90080b4e10 2023-10-08T05:35:55.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.361+0000 7f9006ffd640 1 -- 172.21.15.181:0/406780562 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900002cb90 con 0x7f90080b4e10 2023-10-08T05:35:55.363 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.361+0000 7f9006ffd640 1 -- 172.21.15.181:0/406780562 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000343f0 con 0x7f90080b4e10 2023-10-08T05:35:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.362+0000 7f900de64640 1 -- 172.21.15.181:0/406780562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90080b4e10 msgr2=0x7f90080d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.362+0000 7f900de64640 1 --2- 172.21.15.181:0/406780562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90080b4e10 0x7f90080d1650 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f90080c9cb0 tx=0x7f900002be90 comp rx=0 tx=0).stop 2023-10-08T05:35:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.362+0000 7f900de64640 1 -- 172.21.15.181:0/406780562 shutdown_connections 2023-10-08T05:35:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.362+0000 7f900de64640 1 --2- 172.21.15.181:0/406780562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90080b4e10 0x7f90080d1650 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.362+0000 7f900de64640 1 --2- 172.21.15.181:0/406780562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f900815cd70 0x7f90081612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.362+0000 7f900de64640 1 -- 172.21.15.181:0/406780562 >> 172.21.15.181:0/406780562 conn(0x7f90080cf700 msgr2=0x7f90080cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:55.364 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.362+0000 7f900de64640 1 -- 172.21.15.181:0/406780562 shutdown_connections 2023-10-08T05:35:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.362+0000 7f900de64640 1 -- 172.21.15.181:0/406780562 wait complete. 2023-10-08T05:35:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.364+0000 7f900de64640 1 Processor -- start 2023-10-08T05:35:55.365 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.364+0000 7f900de64640 1 -- start start 2023-10-08T05:35:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f900de64640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90080b4e10 0x7f90081f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f900de64640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90080cd4e0 0x7f90081f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f900de64640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f900815cd70 0x7f90080d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f900de64640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900815f5c0 con 0x7f90080cd4e0 2023-10-08T05:35:55.367 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f900de64640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90081f6ba0 con 0x7f900815cd70 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f900de64640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900815e9b0 con 0x7f90080b4e10 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f9007fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f900815cd70 0x7f90080d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f90077fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90080b4e10 0x7f90081f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f90077fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90080b4e10 0x7f90081f6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.181:55462/0 (socket says 172.21.15.181:55462) 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f9007fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f900815cd70 0x7f90080d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.181:51902/0 (socket says 172.21.15.181:51902) 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f90077fe640 1 -- 172.21.15.181:0/2220953748 learned_addr learned my addr 172.21.15.181:0/2220953748 (peer_addr_for_me v2:172.21.15.181:0/0) 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f8ffffff640 1 --2- 172.21.15.181:0/2220953748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90080cd4e0 0x7f90081f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f9007fff640 1 -- 172.21.15.181:0/2220953748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90080b4e10 msgr2=0x7f90081f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.365+0000 7f9007fff640 1 --2- 172.21.15.181:0/2220953748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90080b4e10 0x7f90081f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.366+0000 7f9007fff640 1 -- 172.21.15.181:0/2220953748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90080cd4e0 msgr2=0x7f90081f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.366+0000 7f9007fff640 1 --2- 172.21.15.181:0/2220953748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90080cd4e0 0x7f90081f1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.368 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.366+0000 7f9007fff640 1 -- 172.21.15.181:0/2220953748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff8009400 con 0x7f900815cd70 2023-10-08T05:35:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.366+0000 7f8ffffff640 1 --2- 172.21.15.181:0/2220953748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90080cd4e0 0x7f90081f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:35:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.366+0000 7f9007fff640 1 --2- 172.21.15.181:0/2220953748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f900815cd70 0x7f90080d1c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9000032000 tx=0x7f9000004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.366+0000 7f9004ff9640 1 -- 172.21.15.181:0/2220953748 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000009510 con 0x7f900815cd70 2023-10-08T05:35:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.366+0000 7f9004ff9640 1 -- 172.21.15.181:0/2220953748 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900003e450 con 0x7f900815cd70 2023-10-08T05:35:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.366+0000 7f900de64640 1 -- 172.21.15.181:0/2220953748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9000009b30 con 0x7f900815cd70 2023-10-08T05:35:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.366+0000 7f9004ff9640 1 -- 172.21.15.181:0/2220953748 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900003e800 con 0x7f900815cd70 2023-10-08T05:35:55.369 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.366+0000 7f900de64640 1 -- 172.21.15.181:0/2220953748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900815ef70 con 0x7f900815cd70 2023-10-08T05:35:55.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.367+0000 7f9004ff9640 1 -- 172.21.15.181:0/2220953748 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f900003e960 con 0x7f900815cd70 2023-10-08T05:35:55.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.367+0000 7f9004ff9640 1 -- 172.21.15.181:0/2220953748 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f9000048c90 con 0x7f900815cd70 2023-10-08T05:35:55.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.367+0000 7f900de64640 1 -- 172.21.15.181:0/2220953748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900804e160 con 0x7f900815cd70 2023-10-08T05:35:55.370 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.370+0000 7f9004ff9640 1 -- 172.21.15.181:0/2220953748 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9000043020 con 0x7f900815cd70 2023-10-08T05:35:55.419 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:55.418+0000 7f27c7643040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:35:55.449 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.447+0000 7f900de64640 1 -- 172.21.15.181:0/2220953748 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2b6cdf62-a741-46b3-8df2-3f2098d41613", "id": 7} v 0) v1 -- 0x7f90080d0a00 con 0x7f900815cd70 2023-10-08T05:35:55.458 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.457+0000 7f9004ff9640 1 -- 172.21.15.181:0/2220953748 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2b6cdf62-a741-46b3-8df2-3f2098d41613", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f90000570b0 con 0x7f900815cd70 2023-10-08T05:35:55.458 INFO:teuthology.orchestra.run.smithi181.stdout:7 2023-10-08T05:35:55.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.459+0000 7f900de64640 1 -- 172.21.15.181:0/2220953748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f900815cd70 msgr2=0x7f90080d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.459+0000 7f900de64640 1 --2- 172.21.15.181:0/2220953748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f900815cd70 0x7f90080d1c80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9000032000 tx=0x7f9000004a40 comp rx=0 tx=0).stop 2023-10-08T05:35:55.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.459+0000 7f900de64640 1 -- 172.21.15.181:0/2220953748 shutdown_connections 2023-10-08T05:35:55.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.459+0000 7f900de64640 1 --2- 172.21.15.181:0/2220953748 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f900815cd70 0x7f90080d1c80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.460 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.459+0000 7f900de64640 1 --2- 172.21.15.181:0/2220953748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90080cd4e0 0x7f90081f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.459+0000 7f900de64640 1 --2- 172.21.15.181:0/2220953748 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90080b4e10 0x7f90081f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.459+0000 7f900de64640 1 -- 172.21.15.181:0/2220953748 >> 172.21.15.181:0/2220953748 conn(0x7f90080cf700 msgr2=0x7f900815b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:55.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.459+0000 7f900de64640 1 -- 172.21.15.181:0/2220953748 shutdown_connections 2023-10-08T05:35:55.461 INFO:teuthology.orchestra.run.smithi181.stderr:2023-10-08T05:35:55.459+0000 7f900de64640 1 -- 172.21.15.181:0/2220953748 wait complete. 2023-10-08T05:35:55.477 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-08T05:35:55.477 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-08T05:35:55.486 INFO:tasks.ceph.osd.0:Started 2023-10-08T05:35:55.486 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-08T05:35:55.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-08T05:35:55.494 INFO:tasks.ceph.osd.1:Started 2023-10-08T05:35:55.494 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-08T05:35:55.494 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-08T05:35:55.501 INFO:tasks.ceph.osd.2:Started 2023-10-08T05:35:55.501 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-08T05:35:55.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-08T05:35:55.507 INFO:tasks.ceph.osd.3:Started 2023-10-08T05:35:55.507 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-08T05:35:55.507 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-08T05:35:55.525 INFO:tasks.ceph.osd.4:Started 2023-10-08T05:35:55.526 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-08T05:35:55.526 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-08T05:35:55.533 INFO:tasks.ceph.osd.5:Started 2023-10-08T05:35:55.534 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-08T05:35:55.534 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-08T05:35:55.539 INFO:tasks.ceph.osd.6:Started 2023-10-08T05:35:55.540 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-08T05:35:55.540 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-08T05:35:55.544 INFO:tasks.ceph.osd.7:Started 2023-10-08T05:35:55.545 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:35:55.576 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:55.575+0000 7f27c7643040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:35:55.644 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:55.642+0000 7f27c7643040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:35:55.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.751+0000 7f24aa3bd640 1 Processor -- start 2023-10-08T05:35:55.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.752+0000 7f24aa3bd640 1 -- start start 2023-10-08T05:35:55.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24aa3bd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f24a415ab30 0x7f24a415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24aa3bd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24a40b07d0 0x7f24a406cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24aa3bd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a415f7d0 con 0x7f24a40adc70 2023-10-08T05:35:55.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24aa3bd640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a415f210 con 0x7f24a415ab30 2023-10-08T05:35:55.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24aa3bd640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a415f4f0 con 0x7f24a40b07d0 2023-10-08T05:35:55.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24a8bba640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f24a40adc70 0x7f24a40b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49234/0 (socket says 172.21.15.70:49234) 2023-10-08T05:35:55.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24a8bba640 1 -- 172.21.15.70:0/3310823143 learned_addr learned my addr 172.21.15.70:0/3310823143 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:35:55.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24a93bb640 1 --2- 172.21.15.70:0/3310823143 >> v2:172.21.15.181:3300/0 conn(0x7f24a415ab30 0x7f24a415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24a9bbc640 1 --2- 172.21.15.70:0/3310823143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24a40b07d0 0x7f24a406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24a93bb640 1 -- 172.21.15.70:0/3310823143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24a40b07d0 msgr2=0x7f24a406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:55.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24a93bb640 1 --2- 172.21.15.70:0/3310823143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24a40b07d0 0x7f24a406cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f249bfff640 1 -- 172.21.15.70:0/3310823143 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407216008 0 0) 0x7f24a415f7d0 con 0x7f24a40adc70 2023-10-08T05:35:55.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24a93bb640 1 -- 172.21.15.70:0/3310823143 >> v1:172.21.15.70:6789/0 conn(0x7f24a40adc70 legacy=0x7f24a40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.753+0000 7f24a93bb640 1 -- 172.21.15.70:0/3310823143 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a406c550 con 0x7f24a415ab30 2023-10-08T05:35:55.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.754+0000 7f24a93bb640 1 --2- 172.21.15.70:0/3310823143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24a415ab30 0x7f24a415e8a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f24a00097a0 tx=0x7f24a002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46115c7423556400 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:55.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.755+0000 7f249bfff640 1 -- 172.21.15.70:0/3310823143 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a002d040 con 0x7f24a415ab30 2023-10-08T05:35:55.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.755+0000 7f249bfff640 1 -- 172.21.15.70:0/3310823143 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a0031810 con 0x7f24a415ab30 2023-10-08T05:35:55.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.755+0000 7f249bfff640 1 -- 172.21.15.70:0/3310823143 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0031ba0 con 0x7f24a415ab30 2023-10-08T05:35:55.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.755+0000 7f24aa3bd640 1 -- 172.21.15.70:0/3310823143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24a415ab30 msgr2=0x7f24a415e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.755+0000 7f24aa3bd640 1 --2- 172.21.15.70:0/3310823143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24a415ab30 0x7f24a415e8a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f24a00097a0 tx=0x7f24a002b8b0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.755+0000 7f24aa3bd640 1 -- 172.21.15.70:0/3310823143 shutdown_connections 2023-10-08T05:35:55.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.755+0000 7f24aa3bd640 1 --2- 172.21.15.70:0/3310823143 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24a40b07d0 0x7f24a406cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.755+0000 7f24aa3bd640 1 --2- 172.21.15.70:0/3310823143 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24a415ab30 0x7f24a415e8a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.755+0000 7f24aa3bd640 1 -- 172.21.15.70:0/3310823143 >> 172.21.15.70:0/3310823143 conn(0x7f24a40cb0c0 msgr2=0x7f24a40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:55.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.756+0000 7f24aa3bd640 1 -- 172.21.15.70:0/3310823143 shutdown_connections 2023-10-08T05:35:55.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.756+0000 7f24aa3bd640 1 -- 172.21.15.70:0/3310823143 wait complete. 2023-10-08T05:35:55.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.757+0000 7f24aa3bd640 1 Processor -- start 2023-10-08T05:35:55.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.757+0000 7f24aa3bd640 1 -- start start 2023-10-08T05:35:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.758+0000 7f24aa3bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24a40adc70 0x7f24a415af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.758+0000 7f24aa3bd640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24a40b07d0 0x7f24a415d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.758+0000 7f24aa3bd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24a415ab30 0x7f24a4148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.758+0000 7f24aa3bd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a40c7030 con 0x7f24a40adc70 2023-10-08T05:35:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.758+0000 7f24aa3bd640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a41492a0 con 0x7f24a40b07d0 2023-10-08T05:35:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.758+0000 7f24aa3bd640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a415def0 con 0x7f24a415ab30 2023-10-08T05:35:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a93bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24a40adc70 0x7f24a415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a93bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24a40adc70 0x7f24a415af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49102/0 (socket says 172.21.15.70:49102) 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a93bb640 1 -- 172.21.15.70:0/779982686 learned_addr learned my addr 172.21.15.70:0/779982686 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a8bba640 1 --2- 172.21.15.70:0/779982686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24a40b07d0 0x7f24a415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a9bbc640 1 --2- 172.21.15.70:0/779982686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24a415ab30 0x7f24a4148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a8bba640 1 -- 172.21.15.70:0/779982686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24a415ab30 msgr2=0x7f24a4148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a8bba640 1 --2- 172.21.15.70:0/779982686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24a415ab30 0x7f24a4148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a8bba640 1 -- 172.21.15.70:0/779982686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24a40adc70 msgr2=0x7f24a415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a8bba640 1 --2- 172.21.15.70:0/779982686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24a40adc70 0x7f24a415af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a8bba640 1 -- 172.21.15.70:0/779982686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a0009400 con 0x7f24a40b07d0 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a93bb640 1 --2- 172.21.15.70:0/779982686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24a40adc70 0x7f24a415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:35:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.759+0000 7f24a8bba640 1 --2- 172.21.15.70:0/779982686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24a40b07d0 0x7f24a415d830 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f249400c450 tx=0x7f249400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.760+0000 7f2499ffb640 1 -- 172.21.15.70:0/779982686 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24940074e0 con 0x7f24a40b07d0 2023-10-08T05:35:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.760+0000 7f2499ffb640 1 -- 172.21.15.70:0/779982686 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2494007640 con 0x7f24a40b07d0 2023-10-08T05:35:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.760+0000 7f24aa3bd640 1 -- 172.21.15.70:0/779982686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a40c6690 con 0x7f24a40b07d0 2023-10-08T05:35:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.760+0000 7f2499ffb640 1 -- 172.21.15.70:0/779982686 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494004a20 con 0x7f24a40b07d0 2023-10-08T05:35:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.760+0000 7f24aa3bd640 1 -- 172.21.15.70:0/779982686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a40c6c50 con 0x7f24a40b07d0 2023-10-08T05:35:55.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.760+0000 7f2499ffb640 1 -- 172.21.15.70:0/779982686 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f249401d440 con 0x7f24a40b07d0 2023-10-08T05:35:55.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.760+0000 7f2499ffb640 1 -- 172.21.15.70:0/779982686 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f249401ddc0 con 0x7f24a40b07d0 2023-10-08T05:35:55.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.761+0000 7f24aa3bd640 1 -- 172.21.15.70:0/779982686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a404e160 con 0x7f24a40b07d0 2023-10-08T05:35:55.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.765+0000 7f2499ffb640 1 -- 172.21.15.70:0/779982686 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f2494018070 con 0x7f24a40b07d0 2023-10-08T05:35:55.784 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:55.782+0000 7f27c7643040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:35:55.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.826+0000 7f24aa3bd640 1 -- 172.21.15.70:0/779982686 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24a40ada50 con 0x7f24a40b07d0 2023-10-08T05:35:55.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.827+0000 7f2499ffb640 1 -- 172.21.15.70:0/779982686 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f249401d6f0 con 0x7f24a40b07d0 2023-10-08T05:35:55.829 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:35:55.829 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":10,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:35:55.450098+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:35:55.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.828+0000 7f24aa3bd640 1 -- 172.21.15.70:0/779982686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24a40b07d0 msgr2=0x7f24a415d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:55.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.828+0000 7f24aa3bd640 1 --2- 172.21.15.70:0/779982686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24a40b07d0 0x7f24a415d830 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f249400c450 tx=0x7f249400c910 comp rx=0 tx=0).stop 2023-10-08T05:35:55.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.828+0000 7f24aa3bd640 1 -- 172.21.15.70:0/779982686 shutdown_connections 2023-10-08T05:35:55.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.828+0000 7f24aa3bd640 1 --2- 172.21.15.70:0/779982686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f24a415ab30 0x7f24a4148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.828+0000 7f24aa3bd640 1 --2- 172.21.15.70:0/779982686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f24a40b07d0 0x7f24a415d830 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.828+0000 7f24aa3bd640 1 --2- 172.21.15.70:0/779982686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24a40adc70 0x7f24a415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:55.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.828+0000 7f24aa3bd640 1 -- 172.21.15.70:0/779982686 >> 172.21.15.70:0/779982686 conn(0x7f24a40cb0c0 msgr2=0x7f24a406cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:55.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.829+0000 7f24aa3bd640 1 -- 172.21.15.70:0/779982686 shutdown_connections 2023-10-08T05:35:55.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:35:55.829+0000 7f24aa3bd640 1 -- 172.21.15.70:0/779982686 wait complete. 2023-10-08T05:35:55.846 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-08T05:35:55.847 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-08T05:35:55.899 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:55.897+0000 7f27c7643040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:35:56.061 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:56.060+0000 7f27c7643040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:35:56.110 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:56.109+0000 7f27c7643040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:35:56.202 INFO:tasks.ceph.osd.1.smithi070.stderr:2023-10-08T05:35:56.201+0000 7fa9caa1a500 -1 Falling back to public interface 2023-10-08T05:35:56.219 INFO:tasks.ceph.osd.3.smithi070.stderr:2023-10-08T05:35:56.218+0000 7feb0210e500 -1 Falling back to public interface 2023-10-08T05:35:56.222 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-10-08T05:35:56.221+0000 7fe1bc838500 -1 Falling back to public interface 2023-10-08T05:35:56.223 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-10-08T05:35:56.222+0000 7f3f23e14500 -1 Falling back to public interface 2023-10-08T05:35:56.223 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-10-08T05:35:56.223+0000 7fe1bb7e1640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6802/0: (98) Address already in use 2023-10-08T05:35:56.224 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-10-08T05:35:56.223+0000 7fe1bb7e1640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6803/0: (98) Address already in use 2023-10-08T05:35:56.225 INFO:tasks.ceph.osd.2.smithi070.stderr:2023-10-08T05:35:56.223+0000 7f7d186b6500 -1 Falling back to public interface 2023-10-08T05:35:56.225 INFO:tasks.ceph.osd.0.smithi070.stderr:2023-10-08T05:35:56.223+0000 7fa3cc1e2500 -1 Falling back to public interface 2023-10-08T05:35:56.247 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-10-08T05:35:56.246+0000 7f384350e500 -1 Falling back to public interface 2023-10-08T05:35:56.256 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-10-08T05:35:56.255+0000 7fcc91120500 -1 Falling back to public interface 2023-10-08T05:35:56.275 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:56.274+0000 7f27c7643040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:35:56.528 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:56.527+0000 7f3633822040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:35:56.652 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T05:35:56.806 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.805+0000 7f1ad4ab4640 1 Processor -- start 2023-10-08T05:35:56.807 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.805+0000 7f1ad4ab4640 1 -- start start 2023-10-08T05:35:56.808 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.806+0000 7f1ad4ab4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1ad00ce120 0x7f1ad00b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:56.808 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.806+0000 7f1ad4ab4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ad00d2660 0x7f1ad00b6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:56.808 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.806+0000 7f1ad4ab4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad00d10b0 con 0x7f1ad015cd10 2023-10-08T05:35:56.808 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.806+0000 7f1ad4ab4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad00d1390 con 0x7f1ad00ce120 2023-10-08T05:35:56.809 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.806+0000 7f1ad4ab4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad00d1670 con 0x7f1ad00d2660 2023-10-08T05:35:56.809 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.806+0000 7f1acdd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1ad00ce120 0x7f1ad00b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:56.809 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.806+0000 7f1ace59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1ad015cd10 0x7f1ad00cda10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49240/0 (socket says 172.21.15.70:49240) 2023-10-08T05:35:56.809 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.806+0000 7f1ace59c640 1 -- 172.21.15.70:0/1614066745 learned_addr learned my addr 172.21.15.70:0/1614066745 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:35:56.809 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.807+0000 7f1aced9d640 1 --2- 172.21.15.70:0/1614066745 >> v2:172.21.15.181:3300/0 conn(0x7f1ad00d2660 0x7f1ad00b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:56.809 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.807+0000 7f1acdd9b640 1 -- 172.21.15.70:0/1614066745 >> v2:172.21.15.181:3300/0 conn(0x7f1ad00d2660 msgr2=0x7f1ad00b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:35:56.809 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.807+0000 7f1acdd9b640 1 --2- 172.21.15.70:0/1614066745 >> v2:172.21.15.181:3300/0 conn(0x7f1ad00d2660 0x7f1ad00b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:56.809 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.807+0000 7f1acdd9b640 1 -- 172.21.15.70:0/1614066745 >> v1:172.21.15.70:6789/0 conn(0x7f1ad015cd10 legacy=0x7f1ad00cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:56.809 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.807+0000 7f1acdd9b640 1 -- 172.21.15.70:0/1614066745 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00d1950 con 0x7f1ad00ce120 2023-10-08T05:35:56.809 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.807+0000 7f1acd59a640 1 -- 172.21.15.70:0/1614066745 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921217729 0 0) 0x7f1ad00d10b0 con 0x7f1ad015cd10 2023-10-08T05:35:56.811 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.808+0000 7f1acdd9b640 1 --2- 172.21.15.70:0/1614066745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 0x7f1ad00b5de0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00ce580 tx=0x7f1ac402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a5947e9adb6c10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:56.811 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.808+0000 7f1acd59a640 1 -- 172.21.15.70:0/1614066745 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4031830 con 0x7f1ad00ce120 2023-10-08T05:35:56.811 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.809+0000 7f1acd59a640 1 -- 172.21.15.70:0/1614066745 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac4031990 con 0x7f1ad00ce120 2023-10-08T05:35:56.811 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.809+0000 7f1acd59a640 1 -- 172.21.15.70:0/1614066745 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4031d20 con 0x7f1ad00ce120 2023-10-08T05:35:56.811 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.809+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/1614066745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 msgr2=0x7f1ad00b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:56.811 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.809+0000 7f1ad4ab4640 1 --2- 172.21.15.70:0/1614066745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 0x7f1ad00b5de0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00ce580 tx=0x7f1ac402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:35:56.812 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.809+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/1614066745 shutdown_connections 2023-10-08T05:35:56.812 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.809+0000 7f1ad4ab4640 1 --2- 172.21.15.70:0/1614066745 >> v2:172.21.15.181:3300/0 conn(0x7f1ad00d2660 0x7f1ad00b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:56.812 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.809+0000 7f1ad4ab4640 1 --2- 172.21.15.70:0/1614066745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 0x7f1ad00b5de0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:56.812 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.809+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/1614066745 >> 172.21.15.70:0/1614066745 conn(0x7f1ad0071480 msgr2=0x7f1ad0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:56.812 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.809+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/1614066745 shutdown_connections 2023-10-08T05:35:56.812 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.809+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/1614066745 wait complete. 2023-10-08T05:35:56.813 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.811+0000 7f1ad4ab4640 1 Processor -- start 2023-10-08T05:35:56.813 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.811+0000 7f1ad4ab4640 1 -- start start 2023-10-08T05:35:56.816 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.812+0000 7f1ad4ab4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 0x7f1ad00cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:56.816 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.812+0000 7f1ad4ab4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad00d2660 0x7f1ad00cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:56.816 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.812+0000 7f1ad4ab4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad015cd10 0x7f1ad0151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:35:56.817 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.812+0000 7f1ad4ab4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00ccaf0 con 0x7f1ad00ce120 2023-10-08T05:35:56.817 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.812+0000 7f1ad4ab4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00ccdd0 con 0x7f1ad015cd10 2023-10-08T05:35:56.817 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.812+0000 7f1ad4ab4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00cb0b0 con 0x7f1ad00d2660 2023-10-08T05:35:56.817 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.812+0000 7f1ace59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 0x7f1ad00cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:56.817 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.812+0000 7f1ace59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 0x7f1ad00cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49122/0 (socket says 172.21.15.70:49122) 2023-10-08T05:35:56.817 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.812+0000 7f1ace59c640 1 -- 172.21.15.70:0/834947867 learned_addr learned my addr 172.21.15.70:0/834947867 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:35:56.817 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1aced9d640 1 --2- 172.21.15.70:0/834947867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad015cd10 0x7f1ad0151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:56.817 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1acdd9b640 1 --2- 172.21.15.70:0/834947867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad00d2660 0x7f1ad00cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:35:56.818 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1ace59c640 1 -- 172.21.15.70:0/834947867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad00d2660 msgr2=0x7f1ad00cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:56.818 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1ace59c640 1 --2- 172.21.15.70:0/834947867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad00d2660 0x7f1ad00cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:56.818 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1ace59c640 1 -- 172.21.15.70:0/834947867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad015cd10 msgr2=0x7f1ad0151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:56.818 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1ace59c640 1 --2- 172.21.15.70:0/834947867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad015cd10 0x7f1ad0151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:56.818 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1ace59c640 1 -- 172.21.15.70:0/834947867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac4009400 con 0x7f1ad00ce120 2023-10-08T05:35:56.818 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1acdd9b640 1 --2- 172.21.15.70:0/834947867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad00d2660 0x7f1ad00cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:35:56.819 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1ace59c640 1 --2- 172.21.15.70:0/834947867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 0x7f1ad00cb890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c460 tx=0x7f1ac000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:35:56.819 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1abeffd640 1 -- 172.21.15.70:0/834947867 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00074e0 con 0x7f1ad00ce120 2023-10-08T05:35:56.819 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1abeffd640 1 -- 172.21.15.70:0/834947867 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac0007660 con 0x7f1ad00ce120 2023-10-08T05:35:56.819 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1abeffd640 1 -- 172.21.15.70:0/834947867 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0004a20 con 0x7f1ad00ce120 2023-10-08T05:35:56.819 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/834947867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad00cbe70 con 0x7f1ad00ce120 2023-10-08T05:35:56.819 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.813+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/834947867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad00cc430 con 0x7f1ad00ce120 2023-10-08T05:35:56.819 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.814+0000 7f1abeffd640 1 -- 172.21.15.70:0/834947867 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1ac0012400 con 0x7f1ad00ce120 2023-10-08T05:35:56.819 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.814+0000 7f1abeffd640 1 -- 172.21.15.70:0/834947867 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f1ac0012d80 con 0x7f1ad00ce120 2023-10-08T05:35:56.820 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.815+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/834947867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad00ccc30 con 0x7f1ad00ce120 2023-10-08T05:35:56.820 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.817+0000 7f1abeffd640 1 -- 172.21.15.70:0/834947867 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f1ad00ccc30 con 0x7f1ad00ce120 2023-10-08T05:35:56.861 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:56.860+0000 7f3633822040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:35:56.883 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.882+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/834947867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ad00b4b30 con 0x7f1ad00ce120 2023-10-08T05:35:56.884 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.883+0000 7f1abeffd640 1 -- 172.21.15.70:0/834947867 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1ac0018070 con 0x7f1ad00ce120 2023-10-08T05:35:56.884 INFO:teuthology.misc.health.smithi070.stdout: 2023-10-08T05:35:56.884 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":10,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:35:55.450098+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:35:56.885 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.884+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/834947867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 msgr2=0x7f1ad00cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:35:56.886 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.884+0000 7f1ad4ab4640 1 --2- 172.21.15.70:0/834947867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 0x7f1ad00cb890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c460 tx=0x7f1ac000c920 comp rx=0 tx=0).stop 2023-10-08T05:35:56.886 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.884+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/834947867 shutdown_connections 2023-10-08T05:35:56.886 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.884+0000 7f1ad4ab4640 1 --2- 172.21.15.70:0/834947867 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad015cd10 0x7f1ad0151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:56.886 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.884+0000 7f1ad4ab4640 1 --2- 172.21.15.70:0/834947867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad00d2660 0x7f1ad00cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:56.886 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.884+0000 7f1ad4ab4640 1 --2- 172.21.15.70:0/834947867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad00ce120 0x7f1ad00cb890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:35:56.886 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.884+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/834947867 >> 172.21.15.70:0/834947867 conn(0x7f1ad0071480 msgr2=0x7f1ad00b82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:35:56.886 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.884+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/834947867 shutdown_connections 2023-10-08T05:35:56.886 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:35:56.884+0000 7f1ad4ab4640 1 -- 172.21.15.70:0/834947867 wait complete. 2023-10-08T05:35:56.898 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-08T05:35:56.928 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:56.927+0000 7f3633822040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:35:57.044 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:57.043+0000 7f3633822040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:35:57.342 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:57.340+0000 7f27c7643040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:35:57.676 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:57.674+0000 7f27c7643040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:35:57.738 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:57.736+0000 7f27c7643040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:35:57.852 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:35:57.850+0000 7f27c7643040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:35:58.095 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:58.094+0000 7f35d479c640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-08T05:35:58.096 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:35:58.095+0000 7f35d479c640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-08T05:35:59.290 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-10-08T05:35:59.289+0000 7f384350e500 -1 osd.7 0 log_to_monitors true 2023-10-08T05:35:59.328 INFO:tasks.ceph.osd.2.smithi070.stderr:2023-10-08T05:35:59.326+0000 7f7d186b6500 -1 osd.2 0 log_to_monitors true 2023-10-08T05:35:59.762 INFO:tasks.ceph.osd.1.smithi070.stderr:2023-10-08T05:35:59.760+0000 7fa9caa1a500 -1 osd.1 0 log_to_monitors true 2023-10-08T05:35:59.807 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-10-08T05:35:59.806+0000 7fe1bc838500 -1 osd.4 0 log_to_monitors true 2023-10-08T05:35:59.814 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-10-08T05:35:59.813+0000 7f3f23e14500 -1 osd.6 0 log_to_monitors true 2023-10-08T05:35:59.822 INFO:tasks.ceph.osd.0.smithi070.stderr:2023-10-08T05:35:59.821+0000 7fa3cc1e2500 -1 osd.0 0 log_to_monitors true 2023-10-08T05:35:59.828 INFO:tasks.ceph.osd.3.smithi070.stderr:2023-10-08T05:35:59.827+0000 7feb0210e500 -1 osd.3 0 log_to_monitors true 2023-10-08T05:35:59.834 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-10-08T05:35:59.834+0000 7fcc91120500 -1 osd.5 0 log_to_monitors true 2023-10-08T05:36:02.826 INFO:tasks.ceph.osd.2.smithi070.stderr:2023-10-08T05:36:02.823+0000 7f7d14644640 -1 osd.2 0 waiting for initial osdmap 2023-10-08T05:36:02.839 INFO:tasks.ceph.osd.3.smithi070.stderr:2023-10-08T05:36:02.838+0000 7feafe09a640 -1 osd.3 0 waiting for initial osdmap 2023-10-08T05:36:02.843 INFO:tasks.ceph.osd.2.smithi070.stderr:2023-10-08T05:36:02.842+0000 7f7d0c464640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T05:36:02.859 INFO:tasks.ceph.osd.3.smithi070.stderr:2023-10-08T05:36:02.857+0000 7feaf5eba640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T05:36:02.874 INFO:tasks.ceph.osd.1.smithi070.stderr:2023-10-08T05:36:02.873+0000 7fa9c6c91640 -1 osd.1 0 waiting for initial osdmap 2023-10-08T05:36:02.879 INFO:tasks.ceph.osd.0.smithi070.stderr:2023-10-08T05:36:02.878+0000 7fa3c8986640 -1 osd.0 0 waiting for initial osdmap 2023-10-08T05:36:02.895 INFO:tasks.ceph.osd.1.smithi070.stderr:2023-10-08T05:36:02.894+0000 7fa9be29b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T05:36:02.901 INFO:tasks.ceph.osd.0.smithi070.stderr:2023-10-08T05:36:02.899+0000 7fa3bff90640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T05:36:03.331 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-10-08T05:36:03.330+0000 7f3f1fda2640 -1 osd.6 0 waiting for initial osdmap 2023-10-08T05:36:03.366 INFO:tasks.ceph.osd.6.smithi181.stderr:2023-10-08T05:36:03.365+0000 7f3f17bc2640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T05:36:03.396 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-10-08T05:36:03.395+0000 7f383f49a640 -1 osd.7 0 waiting for initial osdmap 2023-10-08T05:36:03.419 INFO:tasks.ceph.osd.7.smithi181.stderr:2023-10-08T05:36:03.418+0000 7f38372ba640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T05:36:03.424 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-10-08T05:36:03.424+0000 7fcc8d0ac640 -1 osd.5 0 waiting for initial osdmap 2023-10-08T05:36:03.437 INFO:tasks.ceph.osd.5.smithi181.stderr:2023-10-08T05:36:03.436+0000 7fcc84ecc640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T05:36:03.585 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-10-08T05:36:03.584+0000 7fe1b8fdc640 -1 osd.4 0 waiting for initial osdmap 2023-10-08T05:36:03.613 INFO:tasks.ceph.osd.4.smithi181.stderr:2023-10-08T05:36:03.612+0000 7fe1b05e6640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T05:36:03.704 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:03.900 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.898+0000 7f3bc0aea640 1 Processor -- start 2023-10-08T05:36:03.900 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.898+0000 7f3bc0aea640 1 -- start start 2023-10-08T05:36:03.902 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bc0aea640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3bbc15cc80 0x7f3bbc0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:03.902 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bc0aea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bbc15d060 0x7f3bbc0c9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:03.902 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bc0aea640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bbc0c9ba0 con 0x7f3bbc0cd480 2023-10-08T05:36:03.902 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bc0aea640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bbc0ca8d0 con 0x7f3bbc15cc80 2023-10-08T05:36:03.902 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bc0aea640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0b2850 con 0x7f3bbc15d060 2023-10-08T05:36:03.902 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bbb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3bbc15cc80 0x7f3bbc0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:03.902 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bbb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3bbc15cc80 0x7f3bbc0ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56946/0 (socket says 172.21.15.70:56946) 2023-10-08T05:36:03.902 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bbb7fe640 1 -- 172.21.15.70:0/4201315736 learned_addr learned my addr 172.21.15.70:0/4201315736 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:03.903 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bb3fff640 1 --2- 172.21.15.70:0/4201315736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bbc15d060 0x7f3bbc0c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:03.903 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bbb7fe640 1 -- 172.21.15.70:0/4201315736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bbc15d060 msgr2=0x7f3bbc0c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:03.903 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bbb7fe640 1 --2- 172.21.15.70:0/4201315736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bbc15d060 0x7f3bbc0c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:03.903 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bbb7fe640 1 -- 172.21.15.70:0/4201315736 >> v1:172.21.15.181:6789/0 conn(0x7f3bbc0cd480 legacy=0x7f3bbc0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:03.903 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.900+0000 7f3bbb7fe640 1 -- 172.21.15.70:0/4201315736 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc161920 con 0x7f3bbc15cc80 2023-10-08T05:36:03.904 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.901+0000 7f3bbb7fe640 1 --2- 172.21.15.70:0/4201315736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 0x7f3bbc0ca240 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc06ad90 tx=0x7f3bac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7124490a4948a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:03.904 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.902+0000 7f3bbaffd640 1 -- 172.21.15.70:0/4201315736 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac02e070 con 0x7f3bbc15cc80 2023-10-08T05:36:03.905 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.902+0000 7f3bbaffd640 1 -- 172.21.15.70:0/4201315736 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bac0315a0 con 0x7f3bbc15cc80 2023-10-08T05:36:03.905 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.902+0000 7f3bbaffd640 1 -- 172.21.15.70:0/4201315736 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac031930 con 0x7f3bbc15cc80 2023-10-08T05:36:03.905 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.902+0000 7f3bc0aea640 1 -- 172.21.15.70:0/4201315736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 msgr2=0x7f3bbc0ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:03.905 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.902+0000 7f3bc0aea640 1 --2- 172.21.15.70:0/4201315736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 0x7f3bbc0ca240 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc06ad90 tx=0x7f3bac02b840 comp rx=0 tx=0).stop 2023-10-08T05:36:03.905 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.902+0000 7f3bc0aea640 1 -- 172.21.15.70:0/4201315736 shutdown_connections 2023-10-08T05:36:03.905 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.902+0000 7f3bc0aea640 1 --2- 172.21.15.70:0/4201315736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bbc15d060 0x7f3bbc0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:03.905 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.902+0000 7f3bc0aea640 1 --2- 172.21.15.70:0/4201315736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 0x7f3bbc0ca240 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:03.905 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.902+0000 7f3bc0aea640 1 -- 172.21.15.70:0/4201315736 >> 172.21.15.70:0/4201315736 conn(0x7f3bbc069d50 msgr2=0x7f3bbc06a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:03.905 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.903+0000 7f3bc0aea640 1 -- 172.21.15.70:0/4201315736 shutdown_connections 2023-10-08T05:36:03.905 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.903+0000 7f3bc0aea640 1 -- 172.21.15.70:0/4201315736 wait complete. 2023-10-08T05:36:03.906 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.905+0000 7f3bc0aea640 1 Processor -- start 2023-10-08T05:36:03.907 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.905+0000 7f3bc0aea640 1 -- start start 2023-10-08T05:36:03.910 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.907+0000 7f3bc0aea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bbc0cd480 0x7f3bbc14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:03.910 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.907+0000 7f3bc0aea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 0x7f3bbc149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:03.910 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.907+0000 7f3bc0aea640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bbc15d060 0x7f3bbc0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:03.910 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.907+0000 7f3bc0aea640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0ac760 con 0x7f3bbc15cc80 2023-10-08T05:36:03.910 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.907+0000 7f3bc0aea640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0ab8d0 con 0x7f3bbc15d060 2023-10-08T05:36:03.910 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.907+0000 7f3bc0aea640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc14c500 con 0x7f3bbc0cd480 2023-10-08T05:36:03.910 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.907+0000 7f3bb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 0x7f3bbc149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:03.910 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.907+0000 7f3bb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 0x7f3bbc149520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56962/0 (socket says 172.21.15.70:56962) 2023-10-08T05:36:03.911 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.907+0000 7f3bb3fff640 1 -- 172.21.15.70:0/592875768 learned_addr learned my addr 172.21.15.70:0/592875768 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:03.911 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bbb7fe640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bbc0cd480 0x7f3bbc14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:03.911 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bb3fff640 1 -- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bbc0cd480 msgr2=0x7f3bbc14bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:03.911 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bb3fff640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bbc0cd480 0x7f3bbc14bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:03.911 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bbbfff640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bbc15d060 0x7f3bbc0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:03.911 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bb3fff640 1 -- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bbc15d060 msgr2=0x7f3bbc0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:03.911 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bb3fff640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bbc15d060 0x7f3bbc0b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:03.911 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bb3fff640 1 -- 172.21.15.70:0/592875768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac0094d0 con 0x7f3bbc15cc80 2023-10-08T05:36:03.911 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bbbfff640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bbc15d060 0x7f3bbc0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:03.911 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bb3fff640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 0x7f3bbc149520 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400c7f0 tx=0x7f3ba400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:03.912 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bb8ff9640 1 -- 172.21.15.70:0/592875768 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4004220 con 0x7f3bbc15cc80 2023-10-08T05:36:03.912 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bb8ff9640 1 -- 172.21.15.70:0/592875768 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba4004380 con 0x7f3bbc15cc80 2023-10-08T05:36:03.912 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bb8ff9640 1 -- 172.21.15.70:0/592875768 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4007860 con 0x7f3bbc15cc80 2023-10-08T05:36:03.912 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bc0aea640 1 -- 172.21.15.70:0/592875768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc0abae0 con 0x7f3bbc15cc80 2023-10-08T05:36:03.912 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.908+0000 7f3bc0aea640 1 -- 172.21.15.70:0/592875768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc0ac0a0 con 0x7f3bbc15cc80 2023-10-08T05:36:03.912 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.910+0000 7f3bc0aea640 1 -- 172.21.15.70:0/592875768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc04e160 con 0x7f3bbc15cc80 2023-10-08T05:36:03.913 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.911+0000 7f3bb8ff9640 1 -- 172.21.15.70:0/592875768 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f3ba4007bd0 con 0x7f3bbc15cc80 2023-10-08T05:36:03.913 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.912+0000 7f3bb8ff9640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3b9c06ade0 0x7f3b9c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:03.913 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.912+0000 7f3bb8ff9640 1 -- 172.21.15.70:0/592875768 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f3ba40892a0 con 0x7f3bbc15cc80 2023-10-08T05:36:03.914 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.912+0000 7f3bbb7fe640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3b9c06ade0 0x7f3b9c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:03.915 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.913+0000 7f3bbb7fe640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3b9c06ade0 0x7f3b9c06d2a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3bac036000 tx=0x7f3bac03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:03.923 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:03.922+0000 7f3bb8ff9640 1 -- 172.21.15.70:0/592875768 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ba4054990 con 0x7f3bbc15cc80 2023-10-08T05:36:04.102 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.101+0000 7f3bc0aea640 1 -- 172.21.15.70:0/592875768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bbc0c9d60 con 0x7f3bbc15cc80 2023-10-08T05:36:04.104 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.102+0000 7f3bb8ff9640 1 -- 172.21.15.70:0/592875768 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10896 (secure 0 0 0) 0x7f3ba401e090 con 0x7f3bbc15cc80 2023-10-08T05:36:04.104 INFO:teuthology.misc.health.smithi070.stdout: 2023-10-08T05:36:04.104 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":13,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:03.842373+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:04.107 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 -- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3b9c06ade0 msgr2=0x7f3b9c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.107 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3b9c06ade0 0x7f3b9c06d2a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3bac036000 tx=0x7f3bac03d040 comp rx=0 tx=0).stop 2023-10-08T05:36:04.107 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 -- 172.21.15.70:0/592875768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 msgr2=0x7f3bbc149520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.107 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 0x7f3bbc149520 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400c7f0 tx=0x7f3ba400ccb0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.108 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 -- 172.21.15.70:0/592875768 shutdown_connections 2023-10-08T05:36:04.108 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3b9c06ade0 0x7f3b9c06d2a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.108 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bbc15d060 0x7f3bbc0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.108 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bbc15cc80 0x7f3bbc149520 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.108 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 --2- 172.21.15.70:0/592875768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bbc0cd480 0x7f3bbc14bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.108 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 -- 172.21.15.70:0/592875768 >> 172.21.15.70:0/592875768 conn(0x7f3bbc069d50 msgr2=0x7f3bbc0cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:04.108 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 -- 172.21.15.70:0/592875768 shutdown_connections 2023-10-08T05:36:04.108 INFO:teuthology.misc.health.smithi070.stderr:2023-10-08T05:36:04.105+0000 7f3bc0aea640 1 -- 172.21.15.70:0/592875768 wait complete. 2023-10-08T05:36:04.121 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-08T05:36:04.121 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-08T05:36:04.121 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T05:36:04.121 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T05:36:04.129 INFO:tasks.ceph.mds.a:Started 2023-10-08T05:36:04.130 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T05:36:04.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T05:36:04.137 INFO:tasks.ceph.mds.c:Started 2023-10-08T05:36:04.137 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T05:36:04.137 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T05:36:04.147 INFO:tasks.ceph.mds.b:Started 2023-10-08T05:36:04.147 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T05:36:04.147 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T05:36:04.155 INFO:tasks.ceph.mds.d:Started 2023-10-08T05:36:04.155 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-08T05:36:04.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:04.275 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T05:36:04.289 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:36:04.307 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T05:36:04.307 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:36:04.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.338+0000 7f6b0f610640 1 Processor -- start 2023-10-08T05:36:04.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.339+0000 7f6b0f610640 1 -- start start 2023-10-08T05:36:04.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.340+0000 7f6b0f610640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 0x7f6b080d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.340+0000 7f6b0f610640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b080d1750 con 0x7f6b080b5d40 2023-10-08T05:36:04.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.340+0000 7f6b0f610640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b080d1a30 con 0x7f6b081600d0 2023-10-08T05:36:04.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.340+0000 7f6b0f610640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b080d05b0 con 0x7f6b080cfbe0 2023-10-08T05:36:04.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.340+0000 7f6b0d3ac640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6b081600d0 0x7f6b080d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:50336/0 (socket says 172.21.15.70:50336) 2023-10-08T05:36:04.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.340+0000 7f6b0dbad640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6b080b5d40 0x7f6b080b61d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49786/0 (socket says 172.21.15.70:49786) 2023-10-08T05:36:04.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.340+0000 7f6b0d3ac640 1 -- 172.21.15.70:0/3430425288 learned_addr learned my addr 172.21.15.70:0/3430425288 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:04.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.340+0000 7f6b0cbab640 1 --2- 172.21.15.70:0/3430425288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 0x7f6b080d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.340+0000 7f6afffff640 1 -- 172.21.15.70:0/3430425288 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379959569 0 0) 0x7f6b080d1750 con 0x7f6b080b5d40 2023-10-08T05:36:04.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.341+0000 7f6afffff640 1 -- 172.21.15.70:0/3430425288 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6af4003520 con 0x7f6b080b5d40 2023-10-08T05:36:04.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.341+0000 7f6b0cbab640 1 -- 172.21.15.70:0/3430425288 >> v1:172.21.15.181:6789/0 conn(0x7f6b081600d0 legacy=0x7f6b080d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.341+0000 7f6b0cbab640 1 -- 172.21.15.70:0/3430425288 >> v1:172.21.15.70:6789/0 conn(0x7f6b080b5d40 legacy=0x7f6b080b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.341+0000 7f6b0cbab640 1 -- 172.21.15.70:0/3430425288 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080d1d10 con 0x7f6b080cfbe0 2023-10-08T05:36:04.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.342+0000 7f6b0cbab640 1 --2- 172.21.15.70:0/3430425288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 0x7f6b080d0070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6af80097d0 tx=0x7f6af802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e30fddc2c124c65 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:04.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.345+0000 7f6afffff640 1 -- 172.21.15.70:0/3430425288 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8031890 con 0x7f6b080cfbe0 2023-10-08T05:36:04.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.345+0000 7f6afffff640 1 -- 172.21.15.70:0/3430425288 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af80319f0 con 0x7f6b080cfbe0 2023-10-08T05:36:04.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.345+0000 7f6afffff640 1 -- 172.21.15.70:0/3430425288 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8031d80 con 0x7f6b080cfbe0 2023-10-08T05:36:04.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.345+0000 7f6b0f610640 1 -- 172.21.15.70:0/3430425288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 msgr2=0x7f6b080d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.345+0000 7f6b0f610640 1 --2- 172.21.15.70:0/3430425288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 0x7f6b080d0070 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6af80097d0 tx=0x7f6af802b930 comp rx=0 tx=0).stop 2023-10-08T05:36:04.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.345+0000 7f6b0f610640 1 -- 172.21.15.70:0/3430425288 shutdown_connections 2023-10-08T05:36:04.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.345+0000 7f6b0f610640 1 --2- 172.21.15.70:0/3430425288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 0x7f6b080d0070 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.345+0000 7f6b0f610640 1 -- 172.21.15.70:0/3430425288 >> 172.21.15.70:0/3430425288 conn(0x7f6b080718a0 msgr2=0x7f6b08071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:04.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.345+0000 7f6b0f610640 1 -- 172.21.15.70:0/3430425288 shutdown_connections 2023-10-08T05:36:04.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.345+0000 7f6b0f610640 1 -- 172.21.15.70:0/3430425288 wait complete. 2023-10-08T05:36:04.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.347+0000 7f6b0f610640 1 Processor -- start 2023-10-08T05:36:04.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.347+0000 7f6b0f610640 1 -- start start 2023-10-08T05:36:04.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.348+0000 7f6b0f610640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b080b5d40 0x7f6b080cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.348+0000 7f6b0f610640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 0x7f6b081542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.348+0000 7f6b0f610640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b081600d0 0x7f6b081f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.348+0000 7f6b0f610640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b081f2140 con 0x7f6b081600d0 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.348+0000 7f6b0f610640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b080c7910 con 0x7f6b080b5d40 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.348+0000 7f6b0f610640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b080c7bf0 con 0x7f6b080cfbe0 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.348+0000 7f6b0dbad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b081600d0 0x7f6b081f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.348+0000 7f6b0dbad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b081600d0 0x7f6b081f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56988/0 (socket says 172.21.15.70:56988) 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.348+0000 7f6b0dbad640 1 -- 172.21.15.70:0/2110564900 learned_addr learned my addr 172.21.15.70:0/2110564900 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6b0d3ac640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b080b5d40 0x7f6b080cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6b0cbab640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 0x7f6b081542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6b0dbad640 1 -- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 msgr2=0x7f6b081542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6b0dbad640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 0x7f6b081542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6b0dbad640 1 -- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b080b5d40 msgr2=0x7f6b080cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6b0dbad640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b080b5d40 0x7f6b080cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6b0dbad640 1 -- 172.21.15.70:0/2110564900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af8009400 con 0x7f6b081600d0 2023-10-08T05:36:04.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6b0dbad640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b081600d0 0x7f6b081f1c00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000c7c0 tx=0x7f6b0000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:04.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6afdffb640 1 -- 172.21.15.70:0/2110564900 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000a640 con 0x7f6b081600d0 2023-10-08T05:36:04.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6afdffb640 1 -- 172.21.15.70:0/2110564900 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0000a7a0 con 0x7f6b081600d0 2023-10-08T05:36:04.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6afdffb640 1 -- 172.21.15.70:0/2110564900 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000ab30 con 0x7f6b081600d0 2023-10-08T05:36:04.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.349+0000 7f6b0f610640 1 -- 172.21.15.70:0/2110564900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b080c7ed0 con 0x7f6b081600d0 2023-10-08T05:36:04.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.350+0000 7f6b0f610640 1 -- 172.21.15.70:0/2110564900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b080c8490 con 0x7f6b081600d0 2023-10-08T05:36:04.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.351+0000 7f6b0f610640 1 -- 172.21.15.70:0/2110564900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b08067490 con 0x7f6b081600d0 2023-10-08T05:36:04.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.351+0000 7f6afdffb640 1 -- 172.21.15.70:0/2110564900 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f6b0000ac90 con 0x7f6b081600d0 2023-10-08T05:36:04.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.352+0000 7f6afdffb640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6aec06ade0 0x7f6aec06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.352+0000 7f6afdffb640 1 -- 172.21.15.70:0/2110564900 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f6b00088810 con 0x7f6b081600d0 2023-10-08T05:36:04.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.353+0000 7f6b0d3ac640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6aec06ade0 0x7f6aec06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.354+0000 7f6b0d3ac640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6aec06ade0 0x7f6aec06d2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b04005880 tx=0x7f6b04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:04.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.357+0000 7f6afdffb640 1 -- 172.21.15.70:0/2110564900 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b00053fb0 con 0x7f6b081600d0 2023-10-08T05:36:04.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:36:04.509 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:36:04.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.544+0000 7f6b0f610640 1 -- 172.21.15.70:0/2110564900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b080d30d0 con 0x7f6b081600d0 2023-10-08T05:36:04.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.545+0000 7f6afdffb640 1 -- 172.21.15.70:0/2110564900 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10896 (secure 0 0 0) 0x7f6b00017020 con 0x7f6b081600d0 2023-10-08T05:36:04.549 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:04.549 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":13,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:03.842373+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:04.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 -- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6aec06ade0 msgr2=0x7f6aec06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6aec06ade0 0x7f6aec06d2a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b04005880 tx=0x7f6b04009040 comp rx=0 tx=0).stop 2023-10-08T05:36:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 -- 172.21.15.70:0/2110564900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b081600d0 msgr2=0x7f6b081f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b081600d0 0x7f6b081f1c00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000c7c0 tx=0x7f6b0000cc80 comp rx=0 tx=0).stop 2023-10-08T05:36:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 -- 172.21.15.70:0/2110564900 shutdown_connections 2023-10-08T05:36:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6aec06ade0 0x7f6aec06d2a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b081600d0 0x7f6b081f1c00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b080cfbe0 0x7f6b081542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 --2- 172.21.15.70:0/2110564900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b080b5d40 0x7f6b080cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 -- 172.21.15.70:0/2110564900 >> 172.21.15.70:0/2110564900 conn(0x7f6b080718a0 msgr2=0x7f6b080b61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 -- 172.21.15.70:0/2110564900 shutdown_connections 2023-10-08T05:36:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.547+0000 7f6b0f610640 1 -- 172.21.15.70:0/2110564900 wait complete. 2023-10-08T05:36:04.564 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-10-08T05:36:04.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:36:04.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.814+0000 7fb529905640 1 Processor -- start 2023-10-08T05:36:04.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.814+0000 7fb529905640 1 -- start start 2023-10-08T05:36:04.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.815+0000 7fb529905640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5240c90b0 0x7fb5240cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.816+0000 7fb529905640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52415b2b0 con 0x7fb5240cb3b0 2023-10-08T05:36:04.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.816+0000 7fb529905640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52415b590 con 0x7fb52415ab40 2023-10-08T05:36:04.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.816+0000 7fb529905640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52415bf30 con 0x7fb5240c90b0 2023-10-08T05:36:04.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.816+0000 7fb5237fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb5240cb3b0 0x7fb52415e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49800/0 (socket says 172.21.15.70:49800) 2023-10-08T05:36:04.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.816+0000 7fb5237fe640 1 -- 172.21.15.70:0/3396282899 learned_addr learned my addr 172.21.15.70:0/3396282899 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:04.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.816+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049336241 0 0) 0x7fb52415b2b0 con 0x7fb5240cb3b0 2023-10-08T05:36:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.816+0000 7fb51bfff640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5240c90b0 0x7fb5240cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.816+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb510003520 con 0x7fb5240cb3b0 2023-10-08T05:36:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3327436439 0 0) 0x7fb510003520 con 0x7fb5240cb3b0 2023-10-08T05:36:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5240c90b0 msgr2=0x7fb5240cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5240c90b0 0x7fb5240cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 >> v1:172.21.15.181:6789/0 conn(0x7fb52415ab40 legacy=0x7fb5240ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52415b870 con 0x7fb5240cb3b0 2023-10-08T05:36:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fb51c0019f0 con 0x7fb5240cb3b0 2023-10-08T05:36:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 >> v1:172.21.15.70:6789/0 conn(0x7fb5240cb3b0 legacy=0x7fb52415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5100039a0 0x7fb510003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5100047c0 0x7fb510004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5240c90b0 0x7fb510005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52415bf30 con 0x7fb5100047c0 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51000bb10 con 0x7fb5100039a0 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51000bcd0 con 0x7fb5240c90b0 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb51bfff640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5100047c0 0x7fb510004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb5237fe640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5240c90b0 0x7fb510005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb522ffd640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5100039a0 0x7fb510003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.817+0000 7fb51bfff640 1 -- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5240c90b0 msgr2=0x7fb510005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.818+0000 7fb51bfff640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5240c90b0 0x7fb510005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.818+0000 7fb51bfff640 1 -- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5100039a0 msgr2=0x7fb510003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:04.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.818+0000 7fb51bfff640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5100039a0 0x7fb510003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.818+0000 7fb51bfff640 1 -- 172.21.15.70:0/3396282899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb510003520 con 0x7fb5100047c0 2023-10-08T05:36:04.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.818+0000 7fb5237fe640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5240c90b0 0x7fb510005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:04.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.818+0000 7fb522ffd640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5100039a0 0x7fb510003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:04.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.819+0000 7fb51bfff640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5100047c0 0x7fb510004c40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00ba30 tx=0x7fb5100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9ae6ffc8ff9b3f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:04.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.819+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c032640 con 0x7fb5100047c0 2023-10-08T05:36:04.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.819+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c0327a0 con 0x7fb5100047c0 2023-10-08T05:36:04.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.819+0000 7fb5227fc640 1 -- 172.21.15.70:0/3396282899 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c032b30 con 0x7fb5100047c0 2023-10-08T05:36:04.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.819+0000 7fb529905640 1 -- 172.21.15.70:0/3396282899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5100047c0 msgr2=0x7fb510004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.819+0000 7fb529905640 1 --2- 172.21.15.70:0/3396282899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5100047c0 0x7fb510004c40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00ba30 tx=0x7fb5100057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.819+0000 7fb5237fe640 1 -- 172.21.15.70:0/3396282899 reap_dead start 2023-10-08T05:36:04.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.820+0000 7fb529905640 1 -- 172.21.15.70:0/3396282899 shutdown_connections 2023-10-08T05:36:04.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.820+0000 7fb529905640 1 -- 172.21.15.70:0/3396282899 >> 172.21.15.70:0/3396282899 conn(0x7fb52406a380 msgr2=0x7fb5240b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:04.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.820+0000 7fb529905640 1 -- 172.21.15.70:0/3396282899 shutdown_connections 2023-10-08T05:36:04.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.820+0000 7fb529905640 1 -- 172.21.15.70:0/3396282899 wait complete. 2023-10-08T05:36:04.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.821+0000 7fb529905640 1 Processor -- start 2023-10-08T05:36:04.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.822+0000 7fb529905640 1 -- start start 2023-10-08T05:36:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb529905640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5100047c0 0x7fb5240c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb529905640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240c69c0 0x7fb5241558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb529905640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb524159b20 0x7fb52406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb529905640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240c7b00 con 0x7fb5240c69c0 2023-10-08T05:36:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb529905640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240c6e80 con 0x7fb5100047c0 2023-10-08T05:36:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb529905640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5240c7160 con 0x7fb524159b20 2023-10-08T05:36:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb51bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240c69c0 0x7fb5241558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb51bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240c69c0 0x7fb5241558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57010/0 (socket says 172.21.15.70:57010) 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb51bfff640 1 -- 172.21.15.70:0/3930532156 learned_addr learned my addr 172.21.15.70:0/3930532156 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb51bfff640 1 -- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb524159b20 msgr2=0x7fb52406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb51bfff640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb524159b20 0x7fb52406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb51bfff640 1 -- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5100047c0 msgr2=0x7fb5240c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb51bfff640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5100047c0 0x7fb5240c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.823+0000 7fb51bfff640 1 -- 172.21.15.70:0/3930532156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5240c7440 con 0x7fb5240c69c0 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.824+0000 7fb51bfff640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240c69c0 0x7fb5241558d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00ba30 tx=0x7fb50c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.824+0000 7fb520ff9640 1 -- 172.21.15.70:0/3930532156 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c004220 con 0x7fb5240c69c0 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.824+0000 7fb520ff9640 1 -- 172.21.15.70:0/3930532156 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c004380 con 0x7fb5240c69c0 2023-10-08T05:36:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.824+0000 7fb520ff9640 1 -- 172.21.15.70:0/3930532156 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c003810 con 0x7fb5240c69c0 2023-10-08T05:36:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.824+0000 7fb529905640 1 -- 172.21.15.70:0/3930532156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5240c7720 con 0x7fb5240c69c0 2023-10-08T05:36:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.824+0000 7fb529905640 1 -- 172.21.15.70:0/3930532156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb524155e10 con 0x7fb5240c69c0 2023-10-08T05:36:04.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.825+0000 7fb529905640 1 -- 172.21.15.70:0/3930532156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb524067490 con 0x7fb5240c69c0 2023-10-08T05:36:04.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.826+0000 7fb520ff9640 1 -- 172.21.15.70:0/3930532156 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fb50c044070 con 0x7fb5240c69c0 2023-10-08T05:36:04.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.827+0000 7fb520ff9640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb50406ade0 0x7fb50406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:04.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.827+0000 7fb520ff9640 1 -- 172.21.15.70:0/3930532156 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fb50c0ab740 con 0x7fb5240c69c0 2023-10-08T05:36:04.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.828+0000 7fb522ffd640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb50406ade0 0x7fb50406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:04.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.829+0000 7fb522ffd640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb50406ade0 0x7fb50406d2a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb514001670 tx=0x7fb514009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:04.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:04.831+0000 7fb520ff9640 1 -- 172.21.15.70:0/3930532156 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb50c076e30 con 0x7fb5240c69c0 2023-10-08T05:36:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.058+0000 7fb529905640 1 -- 172.21.15.70:0/3930532156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5240b0ef0 con 0x7fb5240c69c0 2023-10-08T05:36:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.059+0000 7fb520ff9640 1 -- 172.21.15.70:0/3930532156 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7fb50c036090 con 0x7fb5240c69c0 2023-10-08T05:36:05.061 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:05.061 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-08T05:36:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 2 2023-10-08T05:36:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 -- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb50406ade0 msgr2=0x7fb50406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb50406ade0 0x7fb50406d2a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb514001670 tx=0x7fb514009040 comp rx=0 tx=0).stop 2023-10-08T05:36:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 -- 172.21.15.70:0/3930532156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240c69c0 msgr2=0x7fb5241558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240c69c0 0x7fb5241558d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00ba30 tx=0x7fb50c004730 comp rx=0 tx=0).stop 2023-10-08T05:36:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 -- 172.21.15.70:0/3930532156 shutdown_connections 2023-10-08T05:36:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb50406ade0 0x7fb50406d2a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb524159b20 0x7fb52406d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240c69c0 0x7fb5241558d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 --2- 172.21.15.70:0/3930532156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5100047c0 0x7fb5240c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 -- 172.21.15.70:0/3930532156 >> 172.21.15.70:0/3930532156 conn(0x7fb52406a380 msgr2=0x7fb5240cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 -- 172.21.15.70:0/3930532156 shutdown_connections 2023-10-08T05:36:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:05.062+0000 7fb529905640 1 -- 172.21.15.70:0/3930532156 wait complete. 2023-10-08T05:36:07.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:36:07.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.301+0000 7f7f4a155640 1 Processor -- start 2023-10-08T05:36:07.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.301+0000 7f7f4a155640 1 -- start start 2023-10-08T05:36:07.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.303+0000 7f7f4a155640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f4415ab40 0x7f7f440ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.303+0000 7f7f4a155640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f440c90b0 0x7f7f440adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.303+0000 7f7f4a155640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f440cae70 0x7f7f440cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.303+0000 7f7f4a155640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4415bf30 con 0x7f7f440cae70 2023-10-08T05:36:07.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.303+0000 7f7f4a155640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f440cb840 con 0x7f7f440c90b0 2023-10-08T05:36:07.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.303+0000 7f7f4a155640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4415b2b0 con 0x7f7f4415ab40 2023-10-08T05:36:07.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.303+0000 7f7f43fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f440cae70 0x7f7f440cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.303+0000 7f7f43fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f440cae70 0x7f7f440cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57016/0 (socket says 172.21.15.70:57016) 2023-10-08T05:36:07.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.303+0000 7f7f43fff640 1 -- 172.21.15.70:0/3324754582 learned_addr learned my addr 172.21.15.70:0/3324754582 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:07.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.304+0000 7f7f42ffd640 1 --2- 172.21.15.70:0/3324754582 >> v2:172.21.15.181:3300/0 conn(0x7f7f440c90b0 0x7f7f440adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.304+0000 7f7f437fe640 1 --2- 172.21.15.70:0/3324754582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f4415ab40 0x7f7f440ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.304+0000 7f7f43fff640 1 -- 172.21.15.70:0/3324754582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f4415ab40 msgr2=0x7f7f440ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:07.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.304+0000 7f7f43fff640 1 --2- 172.21.15.70:0/3324754582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f4415ab40 0x7f7f440ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.304+0000 7f7f43fff640 1 -- 172.21.15.70:0/3324754582 >> v2:172.21.15.181:3300/0 conn(0x7f7f440c90b0 msgr2=0x7f7f440adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.304+0000 7f7f43fff640 1 --2- 172.21.15.70:0/3324754582 >> v2:172.21.15.181:3300/0 conn(0x7f7f440c90b0 0x7f7f440adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.304+0000 7f7f43fff640 1 -- 172.21.15.70:0/3324754582 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4415b590 con 0x7f7f440cae70 2023-10-08T05:36:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.304+0000 7f7f42ffd640 1 --2- 172.21.15.70:0/3324754582 >> v2:172.21.15.181:3300/0 conn(0x7f7f440c90b0 0x7f7f440adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.304+0000 7f7f437fe640 1 --2- 172.21.15.70:0/3324754582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f4415ab40 0x7f7f440ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.305+0000 7f7f43fff640 1 --2- 172.21.15.70:0/3324754582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f440cae70 0x7f7f440cb300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7f4406e470 tx=0x7f7f3c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de7345e14a1ef8e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:07.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.306+0000 7f7f427fc640 1 -- 172.21.15.70:0/3324754582 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c02ca80 con 0x7f7f440cae70 2023-10-08T05:36:07.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.306+0000 7f7f427fc640 1 -- 172.21.15.70:0/3324754582 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c02cbe0 con 0x7f7f440cae70 2023-10-08T05:36:07.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.306+0000 7f7f427fc640 1 -- 172.21.15.70:0/3324754582 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c0343f0 con 0x7f7f440cae70 2023-10-08T05:36:07.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.306+0000 7f7f4a155640 1 -- 172.21.15.70:0/3324754582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f440cae70 msgr2=0x7f7f440cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:07.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.306+0000 7f7f4a155640 1 --2- 172.21.15.70:0/3324754582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f440cae70 0x7f7f440cb300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7f4406e470 tx=0x7f7f3c02bee0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.307+0000 7f7f4a155640 1 -- 172.21.15.70:0/3324754582 shutdown_connections 2023-10-08T05:36:07.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.307+0000 7f7f4a155640 1 --2- 172.21.15.70:0/3324754582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f440cae70 0x7f7f440cb300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.307+0000 7f7f4a155640 1 --2- 172.21.15.70:0/3324754582 >> v2:172.21.15.181:3300/0 conn(0x7f7f440c90b0 0x7f7f440adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.307+0000 7f7f4a155640 1 --2- 172.21.15.70:0/3324754582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f4415ab40 0x7f7f440ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.307+0000 7f7f4a155640 1 -- 172.21.15.70:0/3324754582 >> 172.21.15.70:0/3324754582 conn(0x7f7f4406a380 msgr2=0x7f7f440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:07.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.307+0000 7f7f4a155640 1 -- 172.21.15.70:0/3324754582 shutdown_connections 2023-10-08T05:36:07.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.307+0000 7f7f4a155640 1 -- 172.21.15.70:0/3324754582 wait complete. 2023-10-08T05:36:07.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.308+0000 7f7f4a155640 1 Processor -- start 2023-10-08T05:36:07.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.309+0000 7f7f4a155640 1 -- start start 2023-10-08T05:36:07.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.309+0000 7f7f4a155640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f440c90b0 0x7f7f441f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.309+0000 7f7f4a155640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f440cae70 0x7f7f440c3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.309+0000 7f7f4a155640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f4415ab40 0x7f7f4415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.309+0000 7f7f4a155640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440c56f0 con 0x7f7f4415ab40 2023-10-08T05:36:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.309+0000 7f7f4a155640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441f7020 con 0x7f7f440cae70 2023-10-08T05:36:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.309+0000 7f7f4a155640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f441f6570 con 0x7f7f440c90b0 2023-10-08T05:36:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f4415ab40 0x7f7f4415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f43fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f4415ab40 0x7f7f4415d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57032/0 (socket says 172.21.15.70:57032) 2023-10-08T05:36:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f43fff640 1 -- 172.21.15.70:0/3999228240 learned_addr learned my addr 172.21.15.70:0/3999228240 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f437fe640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f440c90b0 0x7f7f441f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f43fff640 1 -- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f440c90b0 msgr2=0x7f7f441f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f42ffd640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f440cae70 0x7f7f440c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f43fff640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f440c90b0 0x7f7f441f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f43fff640 1 -- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f440cae70 msgr2=0x7f7f440c3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f43fff640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f440cae70 0x7f7f440c3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f43fff640 1 -- 172.21.15.70:0/3999228240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c009b30 con 0x7f7f4415ab40 2023-10-08T05:36:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.310+0000 7f7f43fff640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f4415ab40 0x7f7f4415d570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c036040 tx=0x7f7f3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:07.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.311+0000 7f7f2bfff640 1 -- 172.21.15.70:0/3999228240 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c009510 con 0x7f7f4415ab40 2023-10-08T05:36:07.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.311+0000 7f7f2bfff640 1 -- 172.21.15.70:0/3999228240 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c03e450 con 0x7f7f4415ab40 2023-10-08T05:36:07.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.311+0000 7f7f2bfff640 1 -- 172.21.15.70:0/3999228240 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c03e800 con 0x7f7f4415ab40 2023-10-08T05:36:07.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.311+0000 7f7f4a155640 1 -- 172.21.15.70:0/3999228240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f440b3000 con 0x7f7f4415ab40 2023-10-08T05:36:07.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.311+0000 7f7f4a155640 1 -- 172.21.15.70:0/3999228240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f441f6d40 con 0x7f7f4415ab40 2023-10-08T05:36:07.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.312+0000 7f7f4a155640 1 -- 172.21.15.70:0/3999228240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f44067490 con 0x7f7f4415ab40 2023-10-08T05:36:07.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.313+0000 7f7f2bfff640 1 -- 172.21.15.70:0/3999228240 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f7f3c02cd50 con 0x7f7f4415ab40 2023-10-08T05:36:07.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.314+0000 7f7f2bfff640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f3006ae10 0x7f7f3006d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.314+0000 7f7f2bfff640 1 -- 172.21.15.70:0/3999228240 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7f3c0abd10 con 0x7f7f4415ab40 2023-10-08T05:36:07.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.315+0000 7f7f437fe640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f3006ae10 0x7f7f3006d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.316+0000 7f7f437fe640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f3006ae10 0x7f7f3006d2d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400d8b0 tx=0x7f7f34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:07.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.319+0000 7f7f2bfff640 1 -- 172.21.15.70:0/3999228240 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f3c0b2080 con 0x7f7f4415ab40 2023-10-08T05:36:07.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.572+0000 7f7f4a155640 1 -- 172.21.15.70:0/3999228240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f4415bb20 con 0x7f7f4415ab40 2023-10-08T05:36:07.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.573+0000 7f7f2bfff640 1 -- 172.21.15.70:0/3999228240 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f7f3c077300 con 0x7f7f4415ab40 2023-10-08T05:36:07.575 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:07.576 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-08T05:36:07.576 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 3 2023-10-08T05:36:07.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.576+0000 7f7f4a155640 1 -- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f3006ae10 msgr2=0x7f7f3006d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:07.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.576+0000 7f7f4a155640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f3006ae10 0x7f7f3006d2d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f3400d8b0 tx=0x7f7f34008040 comp rx=0 tx=0).stop 2023-10-08T05:36:07.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.576+0000 7f7f4a155640 1 -- 172.21.15.70:0/3999228240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f4415ab40 msgr2=0x7f7f4415d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.576+0000 7f7f4a155640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f4415ab40 0x7f7f4415d570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c036040 tx=0x7f7f3c004a40 comp rx=0 tx=0).stop 2023-10-08T05:36:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.576+0000 7f7f4a155640 1 -- 172.21.15.70:0/3999228240 shutdown_connections 2023-10-08T05:36:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.576+0000 7f7f4a155640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f3006ae10 0x7f7f3006d2d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.576+0000 7f7f4a155640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f4415ab40 0x7f7f4415d570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.576+0000 7f7f4a155640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f440cae70 0x7f7f440c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.576+0000 7f7f4a155640 1 --2- 172.21.15.70:0/3999228240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f440c90b0 0x7f7f441f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.576+0000 7f7f4a155640 1 -- 172.21.15.70:0/3999228240 >> 172.21.15.70:0/3999228240 conn(0x7f7f4406a380 msgr2=0x7f7f440af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.577+0000 7f7f4a155640 1 -- 172.21.15.70:0/3999228240 shutdown_connections 2023-10-08T05:36:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.577+0000 7f7f4a155640 1 -- 172.21.15.70:0/3999228240 wait complete. 2023-10-08T05:36:07.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.799+0000 7f5d0e1f5640 1 Processor -- start 2023-10-08T05:36:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.799+0000 7f5d0e1f5640 1 -- start start 2023-10-08T05:36:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.800+0000 7f5d0e1f5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d08160940 0x7f5d080709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.800+0000 7f5d0e1f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0806c890 0x7f5d0806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.800+0000 7f5d0e1f5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d080c48f0 con 0x7f5d0815cd90 2023-10-08T05:36:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.800+0000 7f5d0e1f5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d080c4d80 con 0x7f5d08160940 2023-10-08T05:36:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.800+0000 7f5d0e1f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080c3ce0 con 0x7f5d0806c890 2023-10-08T05:36:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.801+0000 7f5d06ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d08160940 0x7f5d080709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.801+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0806c890 0x7f5d0806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.801+0000 7f5d06ffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d08160940 0x7f5d080709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:52932/0 (socket says 172.21.15.70:52932) 2023-10-08T05:36:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.801+0000 7f5d06ffd640 1 -- 172.21.15.70:0/1208277625 learned_addr learned my addr 172.21.15.70:0/1208277625 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.802+0000 7f5d06ffd640 1 -- 172.21.15.70:0/1208277625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0806c890 msgr2=0x7f5d0806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.802+0000 7f5d06ffd640 1 --2- 172.21.15.70:0/1208277625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0806c890 0x7f5d0806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.802+0000 7f5d06ffd640 1 -- 172.21.15.70:0/1208277625 >> v1:172.21.15.181:6789/0 conn(0x7f5d0815cd90 legacy=0x7f5d080702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.802+0000 7f5d06ffd640 1 -- 172.21.15.70:0/1208277625 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d080c3fc0 con 0x7f5d08160940 2023-10-08T05:36:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.803+0000 7f5d06ffd640 1 --2- 172.21.15.70:0/1208277625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d08160940 0x7f5d080709e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009870 tx=0x7f5cfc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c6329c1d309ad08 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.803+0000 7f5d067fc640 1 -- 172.21.15.70:0/1208277625 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc030840 con 0x7f5d08160940 2023-10-08T05:36:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.803+0000 7f5d067fc640 1 -- 172.21.15.70:0/1208277625 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc0309a0 con 0x7f5d08160940 2023-10-08T05:36:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.803+0000 7f5d067fc640 1 -- 172.21.15.70:0/1208277625 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc030d30 con 0x7f5d08160940 2023-10-08T05:36:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.804+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1208277625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d08160940 msgr2=0x7f5d080709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.804+0000 7f5d0e1f5640 1 --2- 172.21.15.70:0/1208277625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d08160940 0x7f5d080709e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009870 tx=0x7f5cfc02b980 comp rx=0 tx=0).stop 2023-10-08T05:36:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.804+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1208277625 shutdown_connections 2023-10-08T05:36:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.804+0000 7f5d0e1f5640 1 --2- 172.21.15.70:0/1208277625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d0806c890 0x7f5d0806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.804+0000 7f5d0e1f5640 1 --2- 172.21.15.70:0/1208277625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d08160940 0x7f5d080709e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.804+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1208277625 >> 172.21.15.70:0/1208277625 conn(0x7f5d0806a380 msgr2=0x7f5d080b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.804+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1208277625 shutdown_connections 2023-10-08T05:36:07.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.804+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1208277625 wait complete. 2023-10-08T05:36:07.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.806+0000 7f5d0e1f5640 1 Processor -- start 2023-10-08T05:36:07.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.806+0000 7f5d0e1f5640 1 -- start start 2023-10-08T05:36:07.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.807+0000 7f5d0e1f5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0806c890 0x7f5d080cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.807+0000 7f5d0e1f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815cd90 0x7f5d080c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.807+0000 7f5d0e1f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d08160940 0x7f5d080c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.807+0000 7f5d0e1f5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080ccf80 con 0x7f5d0806c890 2023-10-08T05:36:07.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.807+0000 7f5d0e1f5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0815d420 con 0x7f5d0815cd90 2023-10-08T05:36:07.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.807+0000 7f5d0e1f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0815d700 con 0x7f5d08160940 2023-10-08T05:36:07.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.807+0000 7f5d077fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0806c890 0x7f5d080cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.807+0000 7f5d077fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0806c890 0x7f5d080cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57034/0 (socket says 172.21.15.70:57034) 2023-10-08T05:36:07.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.807+0000 7f5d077fe640 1 -- 172.21.15.70:0/1802695805 learned_addr learned my addr 172.21.15.70:0/1802695805 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:07.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5d06ffd640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815cd90 0x7f5d080c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5d077fe640 1 -- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d08160940 msgr2=0x7f5d080c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:07.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5d07fff640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d08160940 0x7f5d080c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5d077fe640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d08160940 0x7f5d080c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5d077fe640 1 -- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815cd90 msgr2=0x7f5d080c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:07.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5d077fe640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815cd90 0x7f5d080c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:07.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5d077fe640 1 -- 172.21.15.70:0/1802695805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0094d0 con 0x7f5d0806c890 2023-10-08T05:36:07.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5d077fe640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0806c890 0x7f5d080cc680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800c450 tx=0x7f5cf800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:07.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5ceffff640 1 -- 172.21.15.70:0/1802695805 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80052d0 con 0x7f5d0806c890 2023-10-08T05:36:07.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5ceffff640 1 -- 172.21.15.70:0/1802695805 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf8005430 con 0x7f5d0806c890 2023-10-08T05:36:07.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5ceffff640 1 -- 172.21.15.70:0/1802695805 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8012870 con 0x7f5d0806c890 2023-10-08T05:36:07.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1802695805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0815d9e0 con 0x7f5d0806c890 2023-10-08T05:36:07.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.808+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1802695805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d080cb1a0 con 0x7f5d0806c890 2023-10-08T05:36:07.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.810+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1802695805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0815e030 con 0x7f5d0806c890 2023-10-08T05:36:07.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.811+0000 7f5ceffff640 1 -- 172.21.15.70:0/1802695805 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f5cf80129d0 con 0x7f5d0806c890 2023-10-08T05:36:07.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.811+0000 7f5ceffff640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5cf406ade0 0x7f5cf406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:07.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.812+0000 7f5ceffff640 1 -- 172.21.15.70:0/1802695805 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5cf8088cf0 con 0x7f5d0806c890 2023-10-08T05:36:07.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.812+0000 7f5d06ffd640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5cf406ade0 0x7f5cf406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:07.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.813+0000 7f5d06ffd640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5cf406ade0 0x7f5cf406d2a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc036000 tx=0x7f5cfc03b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:07.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:07.816+0000 7f5ceffff640 1 -- 172.21.15.70:0/1802695805 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5cf80541d0 con 0x7f5d0806c890 2023-10-08T05:36:08.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.021+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1802695805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d08067490 con 0x7f5d0806c890 2023-10-08T05:36:08.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.022+0000 7f5ceffff640 1 -- 172.21.15.70:0/1802695805 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12838 (secure 0 0 0) 0x7f5cf80577f0 con 0x7f5d0806c890 2023-10-08T05:36:08.023 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:08.023 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":17,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:07.902745+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:08.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5cf406ade0 msgr2=0x7f5cf406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5cf406ade0 0x7f5cf406d2a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc036000 tx=0x7f5cfc03b040 comp rx=0 tx=0).stop 2023-10-08T05:36:08.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1802695805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0806c890 msgr2=0x7f5d080cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0806c890 0x7f5d080cc680 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5cf800c450 tx=0x7f5cf800c910 comp rx=0 tx=0).stop 2023-10-08T05:36:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1802695805 shutdown_connections 2023-10-08T05:36:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5cf406ade0 0x7f5cf406d2a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d08160940 0x7f5d080c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d0815cd90 0x7f5d080c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 --2- 172.21.15.70:0/1802695805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d0806c890 0x7f5d080cc680 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1802695805 >> 172.21.15.70:0/1802695805 conn(0x7f5d0806a380 msgr2=0x7f5d08155220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.024+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1802695805 shutdown_connections 2023-10-08T05:36:08.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.025+0000 7f5d0e1f5640 1 -- 172.21.15.70:0/1802695805 wait complete. 2023-10-08T05:36:08.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:36:08.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:36:08.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.233+0000 7f2d6d840640 1 Processor -- start 2023-10-08T05:36:08.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.233+0000 7f2d6d840640 1 -- start start 2023-10-08T05:36:08.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.235+0000 7f2d6d840640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d6806d370 0x7f2d680c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.235+0000 7f2d6d840640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2d680cb140 0x7f2d680ca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.235+0000 7f2d6d840640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6806dbd0 con 0x7f2d6815cd80 2023-10-08T05:36:08.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.235+0000 7f2d6d840640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6806deb0 con 0x7f2d6806d370 2023-10-08T05:36:08.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.235+0000 7f2d6d840640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d680c9a00 con 0x7f2d680cb140 2023-10-08T05:36:08.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.235+0000 7f2d5ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d6806d370 0x7f2d680c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.235+0000 7f2d5ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d6806d370 0x7f2d680c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57046/0 (socket says 172.21.15.70:57046) 2023-10-08T05:36:08.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.235+0000 7f2d5ffff640 1 -- 172.21.15.70:0/4118615609 learned_addr learned my addr 172.21.15.70:0/4118615609 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:08.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.236+0000 7f2d677fe640 1 --2- 172.21.15.70:0/4118615609 >> v2:172.21.15.181:3300/0 conn(0x7f2d680cb140 0x7f2d680ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.236+0000 7f2d5ffff640 1 -- 172.21.15.70:0/4118615609 >> v2:172.21.15.181:3300/0 conn(0x7f2d680cb140 msgr2=0x7f2d680ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:08.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.236+0000 7f2d5ffff640 1 --2- 172.21.15.70:0/4118615609 >> v2:172.21.15.181:3300/0 conn(0x7f2d680cb140 0x7f2d680ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.236+0000 7f2d5ffff640 1 -- 172.21.15.70:0/4118615609 >> v1:172.21.15.70:6789/0 conn(0x7f2d6815cd80 legacy=0x7f2d6806cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.236+0000 7f2d5ffff640 1 -- 172.21.15.70:0/4118615609 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6815d160 con 0x7f2d6806d370 2023-10-08T05:36:08.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.237+0000 7f2d5ffff640 1 --2- 172.21.15.70:0/4118615609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6806d370 0x7f2d680c9f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2d680665c0 tx=0x7f2d5002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19def34190ed319f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.237+0000 7f2d667fc640 1 -- 172.21.15.70:0/4118615609 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50031870 con 0x7f2d6806d370 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.237+0000 7f2d667fc640 1 -- 172.21.15.70:0/4118615609 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d500319d0 con 0x7f2d6806d370 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.237+0000 7f2d667fc640 1 -- 172.21.15.70:0/4118615609 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50031d60 con 0x7f2d6806d370 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.238+0000 7f2d6d840640 1 -- 172.21.15.70:0/4118615609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6806d370 msgr2=0x7f2d680c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.238+0000 7f2d6d840640 1 --2- 172.21.15.70:0/4118615609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6806d370 0x7f2d680c9f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2d680665c0 tx=0x7f2d5002b910 comp rx=0 tx=0).stop 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.238+0000 7f2d6d840640 1 -- 172.21.15.70:0/4118615609 shutdown_connections 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.238+0000 7f2d6d840640 1 --2- 172.21.15.70:0/4118615609 >> v2:172.21.15.181:3300/0 conn(0x7f2d680cb140 0x7f2d680ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.238+0000 7f2d6d840640 1 --2- 172.21.15.70:0/4118615609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6806d370 0x7f2d680c9f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.238+0000 7f2d6d840640 1 -- 172.21.15.70:0/4118615609 >> 172.21.15.70:0/4118615609 conn(0x7f2d680b0aa0 msgr2=0x7f2d680b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.238+0000 7f2d6d840640 1 -- 172.21.15.70:0/4118615609 shutdown_connections 2023-10-08T05:36:08.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.238+0000 7f2d6d840640 1 -- 172.21.15.70:0/4118615609 wait complete. 2023-10-08T05:36:08.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.240+0000 7f2d6d840640 1 Processor -- start 2023-10-08T05:36:08.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.240+0000 7f2d6d840640 1 -- start start 2023-10-08T05:36:08.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.241+0000 7f2d6d840640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d6806d370 0x7f2d680c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.241+0000 7f2d6d840640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d680cb140 0x7f2d681f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.241+0000 7f2d6d840640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6815cd80 0x7f2d6815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.241+0000 7f2d6d840640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681f66b0 con 0x7f2d6815cd80 2023-10-08T05:36:08.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.241+0000 7f2d6d840640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681f6990 con 0x7f2d6806d370 2023-10-08T05:36:08.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.241+0000 7f2d6d840640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680c45b0 con 0x7f2d680cb140 2023-10-08T05:36:08.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.241+0000 7f2d677fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6815cd80 0x7f2d6815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.241+0000 7f2d677fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6815cd80 0x7f2d6815b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57048/0 (socket says 172.21.15.70:57048) 2023-10-08T05:36:08.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.241+0000 7f2d677fe640 1 -- 172.21.15.70:0/4132358654 learned_addr learned my addr 172.21.15.70:0/4132358654 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.241+0000 7f2d66ffd640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d6806d370 0x7f2d680c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d677fe640 1 -- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d680cb140 msgr2=0x7f2d681f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d5ffff640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d680cb140 0x7f2d681f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d677fe640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d680cb140 0x7f2d681f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d677fe640 1 -- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d6806d370 msgr2=0x7f2d680c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d677fe640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d6806d370 0x7f2d680c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d677fe640 1 -- 172.21.15.70:0/4132358654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d500094d0 con 0x7f2d6815cd80 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d66ffd640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d6806d370 0x7f2d680c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d5ffff640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d680cb140 0x7f2d681f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d677fe640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6815cd80 0x7f2d6815b3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d60007e40 tx=0x7f2d6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:08.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d64ff9640 1 -- 172.21.15.70:0/4132358654 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60014620 con 0x7f2d6815cd80 2023-10-08T05:36:08.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d64ff9640 1 -- 172.21.15.70:0/4132358654 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6000ee90 con 0x7f2d6815cd80 2023-10-08T05:36:08.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d64ff9640 1 -- 172.21.15.70:0/4132358654 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d600149e0 con 0x7f2d6815cd80 2023-10-08T05:36:08.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d6d840640 1 -- 172.21.15.70:0/4132358654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d680c4890 con 0x7f2d6815cd80 2023-10-08T05:36:08.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.242+0000 7f2d6d840640 1 -- 172.21.15.70:0/4132358654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d680c5790 con 0x7f2d6815cd80 2023-10-08T05:36:08.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.244+0000 7f2d6d840640 1 -- 172.21.15.70:0/4132358654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6804e160 con 0x7f2d6815cd80 2023-10-08T05:36:08.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.244+0000 7f2d64ff9640 1 -- 172.21.15.70:0/4132358654 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f2d60014b40 con 0x7f2d6815cd80 2023-10-08T05:36:08.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.245+0000 7f2d64ff9640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d4806ade0 0x7f2d4806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.245+0000 7f2d64ff9640 1 -- 172.21.15.70:0/4132358654 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2d60089be0 con 0x7f2d6815cd80 2023-10-08T05:36:08.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.246+0000 7f2d66ffd640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d4806ade0 0x7f2d4806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.247+0000 7f2d66ffd640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d4806ade0 0x7f2d4806d2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2d58000f60 tx=0x7f2d58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:08.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.249+0000 7f2d64ff9640 1 -- 172.21.15.70:0/4132358654 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d600550b0 con 0x7f2d6815cd80 2023-10-08T05:36:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.439+0000 7f2d6d840640 1 -- 172.21.15.70:0/4132358654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d6815a130 con 0x7f2d6815cd80 2023-10-08T05:36:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.440+0000 7f2d64ff9640 1 -- 172.21.15.70:0/4132358654 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d600586d0 con 0x7f2d6815cd80 2023-10-08T05:36:08.442 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:36:08.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.443+0000 7f2d6d840640 1 -- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d4806ade0 msgr2=0x7f2d4806d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.443+0000 7f2d6d840640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d4806ade0 0x7f2d4806d2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2d58000f60 tx=0x7f2d58009040 comp rx=0 tx=0).stop 2023-10-08T05:36:08.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.443+0000 7f2d6d840640 1 -- 172.21.15.70:0/4132358654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6815cd80 msgr2=0x7f2d6815b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.443+0000 7f2d6d840640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6815cd80 0x7f2d6815b3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d60007e40 tx=0x7f2d6000e4b0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.443+0000 7f2d6d840640 1 -- 172.21.15.70:0/4132358654 shutdown_connections 2023-10-08T05:36:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.444+0000 7f2d6d840640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d4806ade0 0x7f2d4806d2a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.444+0000 7f2d6d840640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d6815cd80 0x7f2d6815b3c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.444+0000 7f2d6d840640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d680cb140 0x7f2d681f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.444+0000 7f2d6d840640 1 --2- 172.21.15.70:0/4132358654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d6806d370 0x7f2d680c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.444+0000 7f2d6d840640 1 -- 172.21.15.70:0/4132358654 >> 172.21.15.70:0/4132358654 conn(0x7f2d680b0aa0 msgr2=0x7f2d680aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.444+0000 7f2d6d840640 1 -- 172.21.15.70:0/4132358654 shutdown_connections 2023-10-08T05:36:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.444+0000 7f2d6d840640 1 -- 172.21.15.70:0/4132358654 wait complete. 2023-10-08T05:36:08.465 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T05:36:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.652+0000 7fe60eada640 1 Processor -- start 2023-10-08T05:36:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.653+0000 7fe60eada640 1 -- start start 2023-10-08T05:36:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.654+0000 7fe60eada640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe60815caf0 0x7fe6080cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.654+0000 7fe60eada640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6080d1eb0 0x7fe6080b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.654+0000 7fe60eada640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60815ced0 con 0x7fe6080d0a00 2023-10-08T05:36:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.654+0000 7fe60eada640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6080d2470 con 0x7fe6080d1eb0 2023-10-08T05:36:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.654+0000 7fe60eada640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080d2750 con 0x7fe60815caf0 2023-10-08T05:36:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.654+0000 7fe60e2d9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6080d1eb0 0x7fe6080b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.654+0000 7fe60e2d9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6080d1eb0 0x7fe6080b6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57052/0 (socket says 172.21.15.70:57052) 2023-10-08T05:36:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.654+0000 7fe60e2d9640 1 -- 172.21.15.70:0/819174373 learned_addr learned my addr 172.21.15.70:0/819174373 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:08.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.655+0000 7fe60dad8640 1 --2- 172.21.15.70:0/819174373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe60815caf0 0x7fe6080cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.655+0000 7fe60e2d9640 1 -- 172.21.15.70:0/819174373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe60815caf0 msgr2=0x7fe6080cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.655+0000 7fe60e2d9640 1 --2- 172.21.15.70:0/819174373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe60815caf0 0x7fe6080cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.655+0000 7fe60e2d9640 1 -- 172.21.15.70:0/819174373 >> v1:172.21.15.181:6789/0 conn(0x7fe6080d0a00 legacy=0x7fe6080b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.655+0000 7fe60e2d9640 1 -- 172.21.15.70:0/819174373 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6080d2a30 con 0x7fe6080d1eb0 2023-10-08T05:36:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.656+0000 7fe60e2d9640 1 --2- 172.21.15.70:0/819174373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 0x7fe6080b6a10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe600009e50 tx=0x7fe60002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79832c5d418ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.656+0000 7fe60cad6640 1 -- 172.21.15.70:0/819174373 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60002ca70 con 0x7fe6080d1eb0 2023-10-08T05:36:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.656+0000 7fe60cad6640 1 -- 172.21.15.70:0/819174373 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60002cbd0 con 0x7fe6080d1eb0 2023-10-08T05:36:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.656+0000 7fe60cad6640 1 -- 172.21.15.70:0/819174373 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6000343f0 con 0x7fe6080d1eb0 2023-10-08T05:36:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.657+0000 7fe60eada640 1 -- 172.21.15.70:0/819174373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 msgr2=0x7fe6080b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.657+0000 7fe60eada640 1 --2- 172.21.15.70:0/819174373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 0x7fe6080b6a10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe600009e50 tx=0x7fe60002bed0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.657+0000 7fe60eada640 1 -- 172.21.15.70:0/819174373 shutdown_connections 2023-10-08T05:36:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.657+0000 7fe60eada640 1 --2- 172.21.15.70:0/819174373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 0x7fe6080b6a10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.657+0000 7fe60eada640 1 --2- 172.21.15.70:0/819174373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe60815caf0 0x7fe6080cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.657+0000 7fe60eada640 1 -- 172.21.15.70:0/819174373 >> 172.21.15.70:0/819174373 conn(0x7fe608071870 msgr2=0x7fe608071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.657+0000 7fe60eada640 1 -- 172.21.15.70:0/819174373 shutdown_connections 2023-10-08T05:36:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.657+0000 7fe60eada640 1 -- 172.21.15.70:0/819174373 wait complete. 2023-10-08T05:36:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.658+0000 7fe60eada640 1 Processor -- start 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.659+0000 7fe60eada640 1 -- start start 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.659+0000 7fe60eada640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080d0a00 0x7fe60814cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.659+0000 7fe60eada640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 0x7fe6080b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.659+0000 7fe60eada640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815caf0 0x7fe6080b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.659+0000 7fe60eada640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080aaf80 con 0x7fe6080d1eb0 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.659+0000 7fe60eada640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080aa370 con 0x7fe60815caf0 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.659+0000 7fe60eada640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080aa650 con 0x7fe6080d0a00 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60d2d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 0x7fe6080b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60d2d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 0x7fe6080b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57056/0 (socket says 172.21.15.70:57056) 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60d2d7640 1 -- 172.21.15.70:0/572417615 learned_addr learned my addr 172.21.15.70:0/572417615 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60e2d9640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815caf0 0x7fe6080b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60d2d7640 1 -- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080d0a00 msgr2=0x7fe60814cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60dad8640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080d0a00 0x7fe60814cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60d2d7640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080d0a00 0x7fe60814cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60d2d7640 1 -- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815caf0 msgr2=0x7fe6080b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60d2d7640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815caf0 0x7fe6080b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60d2d7640 1 -- 172.21.15.70:0/572417615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe600009b30 con 0x7fe6080d1eb0 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.660+0000 7fe60d2d7640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 0x7fe6080b3620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800b3f0 tx=0x7fe5f800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.661+0000 7fe5fe7fc640 1 -- 172.21.15.70:0/572417615 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8011830 con 0x7fe6080d1eb0 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.661+0000 7fe5fe7fc640 1 -- 172.21.15.70:0/572417615 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f8011990 con 0x7fe6080d1eb0 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.661+0000 7fe5fe7fc640 1 -- 172.21.15.70:0/572417615 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8011d40 con 0x7fe6080d1eb0 2023-10-08T05:36:08.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.661+0000 7fe60eada640 1 -- 172.21.15.70:0/572417615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6080aa930 con 0x7fe6080d1eb0 2023-10-08T05:36:08.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.661+0000 7fe60eada640 1 -- 172.21.15.70:0/572417615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60814d280 con 0x7fe6080d1eb0 2023-10-08T05:36:08.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.662+0000 7fe60eada640 1 -- 172.21.15.70:0/572417615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6080ab130 con 0x7fe6080d1eb0 2023-10-08T05:36:08.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.663+0000 7fe5fe7fc640 1 -- 172.21.15.70:0/572417615 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7fe5f801a400 con 0x7fe6080d1eb0 2023-10-08T05:36:08.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.664+0000 7fe5fe7fc640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe5ec06ade0 0x7fe5ec06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:08.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.664+0000 7fe5fe7fc640 1 -- 172.21.15.70:0/572417615 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe5f8088fb0 con 0x7fe6080d1eb0 2023-10-08T05:36:08.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.664+0000 7fe60dad8640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe5ec06ade0 0x7fe5ec06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:08.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.665+0000 7fe60dad8640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe5ec06ade0 0x7fe5ec06d2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe604006fd0 tx=0x7fe604008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:08.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.671+0000 7fe5fe7fc640 1 -- 172.21.15.70:0/572417615 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5f80bbca0 con 0x7fe6080d1eb0 2023-10-08T05:36:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.861+0000 7fe60eada640 1 -- 172.21.15.70:0/572417615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe60804e160 con 0x7fe6080d1eb0 2023-10-08T05:36:08.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.862+0000 7fe5fe7fc640 1 -- 172.21.15.70:0/572417615 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe5f8015060 con 0x7fe6080d1eb0 2023-10-08T05:36:08.866 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:08.866 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:36:08.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:36:08.866 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T05:36:08.866 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T05:36:08.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:36:08.866 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:36:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 -- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe5ec06ade0 msgr2=0x7fe5ec06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe5ec06ade0 0x7fe5ec06d2a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe604006fd0 tx=0x7fe604008040 comp rx=0 tx=0).stop 2023-10-08T05:36:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 -- 172.21.15.70:0/572417615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 msgr2=0x7fe6080b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 0x7fe6080b3620 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800b3f0 tx=0x7fe5f800b8b0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 -- 172.21.15.70:0/572417615 shutdown_connections 2023-10-08T05:36:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe5ec06ade0 0x7fe5ec06d2a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe60815caf0 0x7fe6080b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6080d1eb0 0x7fe6080b3620 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 --2- 172.21.15.70:0/572417615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6080d0a00 0x7fe60814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 -- 172.21.15.70:0/572417615 >> 172.21.15.70:0/572417615 conn(0x7fe608071870 msgr2=0x7fe6080cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 -- 172.21.15.70:0/572417615 shutdown_connections 2023-10-08T05:36:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:08.865+0000 7fe60eada640 1 -- 172.21.15.70:0/572417615 wait complete. 2023-10-08T05:36:08.885 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T05:36:08.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T05:36:09.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.106+0000 7f79eb59e640 1 Processor -- start 2023-10-08T05:36:09.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.106+0000 7f79eb59e640 1 -- start start 2023-10-08T05:36:09.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.108+0000 7f79eb59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f79ec070b50 0x7f79ec0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:09.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.108+0000 7f79eb59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79ec0b04a0 0x7f79ec0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:09.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.108+0000 7f79eb59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ec15ae90 con 0x7f79ec15aab0 2023-10-08T05:36:09.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.108+0000 7f79eb59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0cb600 con 0x7f79ec070b50 2023-10-08T05:36:09.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.108+0000 7f79eb59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0b0930 con 0x7f79ec0b04a0 2023-10-08T05:36:09.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.108+0000 7f79ea59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f79ec15aab0 0x7f79ec0caef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49818/0 (socket says 172.21.15.70:49818) 2023-10-08T05:36:09.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.108+0000 7f79ea59c640 1 -- 172.21.15.70:0/1935082611 learned_addr learned my addr 172.21.15.70:0/1935082611 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:09.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.108+0000 7f79e1d9b640 1 --2- 172.21.15.70:0/1935082611 >> v2:172.21.15.70:3300/0 conn(0x7f79ec070b50 0x7f79ec0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:09.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.108+0000 7f79ead9d640 1 --2- 172.21.15.70:0/1935082611 >> v2:172.21.15.181:3300/0 conn(0x7f79ec0b04a0 0x7f79ec0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:09.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.109+0000 7f79e9d9b640 1 -- 172.21.15.70:0/1935082611 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057249944 0 0) 0x7f79dc001490 con 0x7f79ec15aab0 2023-10-08T05:36:09.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.109+0000 7f79e9d9b640 1 -- 172.21.15.70:0/1935082611 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d8003520 con 0x7f79ec15aab0 2023-10-08T05:36:09.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.109+0000 7f79e1d9b640 1 -- 172.21.15.70:0/1935082611 >> v2:172.21.15.181:3300/0 conn(0x7f79ec0b04a0 msgr2=0x7f79ec0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:09.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.109+0000 7f79e1d9b640 1 --2- 172.21.15.70:0/1935082611 >> v2:172.21.15.181:3300/0 conn(0x7f79ec0b04a0 0x7f79ec0b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:09.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.109+0000 7f79e1d9b640 1 -- 172.21.15.70:0/1935082611 >> v1:172.21.15.70:6789/0 conn(0x7f79ec15aab0 legacy=0x7f79ec0caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:09.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.109+0000 7f79e1d9b640 1 -- 172.21.15.70:0/1935082611 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0cbdb0 con 0x7f79ec070b50 2023-10-08T05:36:09.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.110+0000 7f79e1d9b640 1 --2- 172.21.15.70:0/1935082611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec070b50 0x7f79ec0aff60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79d40097a0 tx=0x7f79d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5b148419d6da36 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:09.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.110+0000 7f79e9d9b640 1 -- 172.21.15.70:0/1935082611 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d402e070 con 0x7f79ec070b50 2023-10-08T05:36:09.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.110+0000 7f79e9d9b640 1 -- 172.21.15.70:0/1935082611 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d40315a0 con 0x7f79ec070b50 2023-10-08T05:36:09.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.110+0000 7f79e9d9b640 1 -- 172.21.15.70:0/1935082611 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d4031930 con 0x7f79ec070b50 2023-10-08T05:36:09.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.111+0000 7f79eb59e640 1 -- 172.21.15.70:0/1935082611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec070b50 msgr2=0x7f79ec0aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:09.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.111+0000 7f79eb59e640 1 --2- 172.21.15.70:0/1935082611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec070b50 0x7f79ec0aff60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79d40097a0 tx=0x7f79d402b840 comp rx=0 tx=0).stop 2023-10-08T05:36:09.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.111+0000 7f79eb59e640 1 -- 172.21.15.70:0/1935082611 shutdown_connections 2023-10-08T05:36:09.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.111+0000 7f79eb59e640 1 --2- 172.21.15.70:0/1935082611 >> v2:172.21.15.181:3300/0 conn(0x7f79ec0b04a0 0x7f79ec0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:09.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.111+0000 7f79eb59e640 1 --2- 172.21.15.70:0/1935082611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec070b50 0x7f79ec0aff60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:09.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.111+0000 7f79eb59e640 1 -- 172.21.15.70:0/1935082611 >> 172.21.15.70:0/1935082611 conn(0x7f79ec06c010 msgr2=0x7f79ec06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:09.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.111+0000 7f79eb59e640 1 -- 172.21.15.70:0/1935082611 shutdown_connections 2023-10-08T05:36:09.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.111+0000 7f79eb59e640 1 -- 172.21.15.70:0/1935082611 wait complete. 2023-10-08T05:36:09.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.113+0000 7f79eb59e640 1 Processor -- start 2023-10-08T05:36:09.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.113+0000 7f79eb59e640 1 -- start start 2023-10-08T05:36:09.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79eb59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79ec070b50 0x7f79ec1f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:09.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79eb59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec0b04a0 0x7f79ec1ec730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:09.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79eb59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec15aab0 0x7f79ec0ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:09.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79eb59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec157400 con 0x7f79ec15aab0 2023-10-08T05:36:09.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79eb59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec15be20 con 0x7f79ec0b04a0 2023-10-08T05:36:09.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79eb59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1f3870 con 0x7f79ec070b50 2023-10-08T05:36:09.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79ead9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec15aab0 0x7f79ec0ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:09.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79ead9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec15aab0 0x7f79ec0ac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57076/0 (socket says 172.21.15.70:57076) 2023-10-08T05:36:09.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79ead9d640 1 -- 172.21.15.70:0/3782283127 learned_addr learned my addr 172.21.15.70:0/3782283127 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:09.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79ea59c640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79ec070b50 0x7f79ec1f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:09.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.114+0000 7f79e1d9b640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec0b04a0 0x7f79ec1ec730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:09.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79ead9d640 1 -- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79ec070b50 msgr2=0x7f79ec1f3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:09.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79ead9d640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79ec070b50 0x7f79ec1f3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:09.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79ead9d640 1 -- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec0b04a0 msgr2=0x7f79ec1ec730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:09.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79ead9d640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec0b04a0 0x7f79ec1ec730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:09.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79ead9d640 1 -- 172.21.15.70:0/3782283127 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4009400 con 0x7f79ec15aab0 2023-10-08T05:36:09.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79ead9d640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec15aab0 0x7f79ec0ac920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f79e400e930 tx=0x7f79e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:09.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79e37fe640 1 -- 172.21.15.70:0/3782283127 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4009690 con 0x7f79ec15aab0 2023-10-08T05:36:09.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79e37fe640 1 -- 172.21.15.70:0/3782283127 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e400c980 con 0x7f79ec15aab0 2023-10-08T05:36:09.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79e37fe640 1 -- 172.21.15.70:0/3782283127 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e400cd30 con 0x7f79ec15aab0 2023-10-08T05:36:09.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79eb59e640 1 -- 172.21.15.70:0/3782283127 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec156880 con 0x7f79ec15aab0 2023-10-08T05:36:09.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.115+0000 7f79eb59e640 1 -- 172.21.15.70:0/3782283127 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec156e40 con 0x7f79ec15aab0 2023-10-08T05:36:09.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.116+0000 7f79eb59e640 1 -- 172.21.15.70:0/3782283127 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec074150 con 0x7f79ec15aab0 2023-10-08T05:36:09.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.117+0000 7f79e37fe640 1 -- 172.21.15.70:0/3782283127 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 4) v1 ==== 89173+0+0 (secure 0 0 0) 0x7f79e4013470 con 0x7f79ec15aab0 2023-10-08T05:36:09.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.119+0000 7f79e37fe640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f79cc06ade0 0x7f79cc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:09.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.119+0000 7f79e37fe640 1 -- 172.21.15.70:0/3782283127 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f79e4089360 con 0x7f79ec15aab0 2023-10-08T05:36:09.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.119+0000 7f79ea59c640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f79cc06ade0 0x7f79cc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:09.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.120+0000 7f79ea59c640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f79cc06ade0 0x7f79cc06d2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0015f0 tx=0x7f79dc001a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:09.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.123+0000 7f79e37fe640 1 -- 172.21.15.70:0/3782283127 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79e4054a20 con 0x7f79ec15aab0 2023-10-08T05:36:09.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:09.303+0000 7f79eb59e640 1 -- 172.21.15.70:0/3782283127 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f79ec1571e0 con 0x7f79ec15aab0 2023-10-08T05:36:10.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.124+0000 7f79e37fe640 1 -- 172.21.15.70:0/3782283127 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f79e4058040 con 0x7f79ec15aab0 2023-10-08T05:36:10.127 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T05:36:10.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.128+0000 7f79eb59e640 1 -- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f79cc06ade0 msgr2=0x7f79cc06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:10.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.128+0000 7f79eb59e640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f79cc06ade0 0x7f79cc06d2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0015f0 tx=0x7f79dc001a00 comp rx=0 tx=0).stop 2023-10-08T05:36:10.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.128+0000 7f79eb59e640 1 -- 172.21.15.70:0/3782283127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec15aab0 msgr2=0x7f79ec0ac920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:10.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.128+0000 7f79eb59e640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec15aab0 0x7f79ec0ac920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f79e400e930 tx=0x7f79e400edf0 comp rx=0 tx=0).stop 2023-10-08T05:36:10.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.128+0000 7f79eb59e640 1 -- 172.21.15.70:0/3782283127 shutdown_connections 2023-10-08T05:36:10.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.128+0000 7f79eb59e640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f79cc06ade0 0x7f79cc06d2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:10.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.128+0000 7f79eb59e640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec15aab0 0x7f79ec0ac920 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:10.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.128+0000 7f79eb59e640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec0b04a0 0x7f79ec1ec730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:10.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.128+0000 7f79eb59e640 1 --2- 172.21.15.70:0/3782283127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79ec070b50 0x7f79ec1f3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:10.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.128+0000 7f79eb59e640 1 -- 172.21.15.70:0/3782283127 >> 172.21.15.70:0/3782283127 conn(0x7f79ec06c010 msgr2=0x7f79ec0cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:10.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.129+0000 7f79eb59e640 1 -- 172.21.15.70:0/3782283127 shutdown_connections 2023-10-08T05:36:10.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.129+0000 7f79eb59e640 1 -- 172.21.15.70:0/3782283127 wait complete. 2023-10-08T05:36:10.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-08T05:36:10.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.356+0000 7fa457e7c640 1 Processor -- start 2023-10-08T05:36:10.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.357+0000 7fa457e7c640 1 -- start start 2023-10-08T05:36:10.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.358+0000 7fa457e7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa45006cfa0 0x7fa45015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:10.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.358+0000 7fa457e7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa45015b630 0x7fa450070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.358+0000 7fa457e7c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4500c8c60 con 0x7fa4500cb500 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.358+0000 7fa457e7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4500b1ac0 con 0x7fa45015b630 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.358+0000 7fa457e7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500b1da0 con 0x7fa45006cfa0 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.358+0000 7fa455c18640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa4500cb500 0x7fa4500cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49832/0 (socket says 172.21.15.70:49832) 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.358+0000 7fa455c18640 1 -- 172.21.15.70:0/3073329710 learned_addr learned my addr 172.21.15.70:0/3073329710 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.359+0000 7fa456419640 1 --2- 172.21.15.70:0/3073329710 >> v2:172.21.15.70:3300/0 conn(0x7fa45015b630 0x7fa450070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.359+0000 7fa455417640 1 --2- 172.21.15.70:0/3073329710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa45006cfa0 0x7fa45015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.359+0000 7fa454c16640 1 -- 172.21.15.70:0/3073329710 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177217669 0 0) 0x7fa4500c8c60 con 0x7fa4500cb500 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.359+0000 7fa454c16640 1 -- 172.21.15.70:0/3073329710 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa43c003520 con 0x7fa4500cb500 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.359+0000 7fa456419640 1 -- 172.21.15.70:0/3073329710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa45006cfa0 msgr2=0x7fa45015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:10.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.359+0000 7fa456419640 1 --2- 172.21.15.70:0/3073329710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa45006cfa0 0x7fa45015b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:10.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.359+0000 7fa456419640 1 -- 172.21.15.70:0/3073329710 >> v1:172.21.15.70:6789/0 conn(0x7fa4500cb500 legacy=0x7fa4500cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:10.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.359+0000 7fa456419640 1 -- 172.21.15.70:0/3073329710 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500b2080 con 0x7fa45015b630 2023-10-08T05:36:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.360+0000 7fa456419640 1 --2- 172.21.15.70:0/3073329710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45015b630 0x7fa450070690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa448009d90 tx=0x7fa44802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea81779b09a4a67c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.360+0000 7fa454c16640 1 -- 172.21.15.70:0/3073329710 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44802cb40 con 0x7fa45015b630 2023-10-08T05:36:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.360+0000 7fa454c16640 1 -- 172.21.15.70:0/3073329710 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44802cca0 con 0x7fa45015b630 2023-10-08T05:36:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.361+0000 7fa454c16640 1 -- 172.21.15.70:0/3073329710 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448034660 con 0x7fa45015b630 2023-10-08T05:36:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.361+0000 7fa457e7c640 1 -- 172.21.15.70:0/3073329710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45015b630 msgr2=0x7fa450070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.361+0000 7fa457e7c640 1 --2- 172.21.15.70:0/3073329710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45015b630 0x7fa450070690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa448009d90 tx=0x7fa44802bd90 comp rx=0 tx=0).stop 2023-10-08T05:36:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.361+0000 7fa457e7c640 1 -- 172.21.15.70:0/3073329710 shutdown_connections 2023-10-08T05:36:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.361+0000 7fa457e7c640 1 --2- 172.21.15.70:0/3073329710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45015b630 0x7fa450070690 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.361+0000 7fa457e7c640 1 --2- 172.21.15.70:0/3073329710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa45006cfa0 0x7fa45015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:10.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.361+0000 7fa457e7c640 1 -- 172.21.15.70:0/3073329710 >> 172.21.15.70:0/3073329710 conn(0x7fa45006a140 msgr2=0x7fa45006a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:10.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.361+0000 7fa457e7c640 1 -- 172.21.15.70:0/3073329710 shutdown_connections 2023-10-08T05:36:10.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.361+0000 7fa457e7c640 1 -- 172.21.15.70:0/3073329710 wait complete. 2023-10-08T05:36:10.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.363+0000 7fa457e7c640 1 Processor -- start 2023-10-08T05:36:10.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.363+0000 7fa457e7c640 1 -- start start 2023-10-08T05:36:10.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.364+0000 7fa457e7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45006cfa0 0x7fa45014b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:10.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.364+0000 7fa457e7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4500cb500 0x7fa450149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:10.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.364+0000 7fa457e7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa45015b630 0x7fa4501f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:10.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.364+0000 7fa457e7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa450149b50 con 0x7fa45006cfa0 2023-10-08T05:36:10.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.364+0000 7fa457e7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4501496d0 con 0x7fa4500cb500 2023-10-08T05:36:10.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.364+0000 7fa457e7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4501f1730 con 0x7fa45015b630 2023-10-08T05:36:10.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.364+0000 7fa455c18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45006cfa0 0x7fa45014b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:10.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.364+0000 7fa455c18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45006cfa0 0x7fa45014b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57090/0 (socket says 172.21.15.70:57090) 2023-10-08T05:36:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.364+0000 7fa455c18640 1 -- 172.21.15.70:0/3010854110 learned_addr learned my addr 172.21.15.70:0/3010854110 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa456419640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa45015b630 0x7fa4501f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa455c18640 1 -- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa45015b630 msgr2=0x7fa4501f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa455c18640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa45015b630 0x7fa4501f11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa455c18640 1 -- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4500cb500 msgr2=0x7fa450149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa455c18640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4500cb500 0x7fa450149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa455c18640 1 -- 172.21.15.70:0/3010854110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa448009a70 con 0x7fa45006cfa0 2023-10-08T05:36:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa455c18640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45006cfa0 0x7fa45014b9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c500 tx=0x7fa44c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa4467fc640 1 -- 172.21.15.70:0/3010854110 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c005ce0 con 0x7fa45006cfa0 2023-10-08T05:36:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa4467fc640 1 -- 172.21.15.70:0/3010854110 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44c005e40 con 0x7fa45006cfa0 2023-10-08T05:36:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa4467fc640 1 -- 172.21.15.70:0/3010854110 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44c007330 con 0x7fa45006cfa0 2023-10-08T05:36:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa457e7c640 1 -- 172.21.15.70:0/3010854110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45014c000 con 0x7fa45006cfa0 2023-10-08T05:36:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.365+0000 7fa457e7c640 1 -- 172.21.15.70:0/3010854110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45014c5c0 con 0x7fa45006cfa0 2023-10-08T05:36:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.366+0000 7fa457e7c640 1 -- 172.21.15.70:0/3010854110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa450149c90 con 0x7fa45006cfa0 2023-10-08T05:36:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.367+0000 7fa4467fc640 1 -- 172.21.15.70:0/3010854110 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa44c007550 con 0x7fa45006cfa0 2023-10-08T05:36:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.368+0000 7fa4467fc640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa43406ae70 0x7fa43406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.368+0000 7fa4467fc640 1 -- 172.21.15.70:0/3010854110 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fa44c088ae0 con 0x7fa45006cfa0 2023-10-08T05:36:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.369+0000 7fa455417640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa43406ae70 0x7fa43406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:10.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.370+0000 7fa455417640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa43406ae70 0x7fa43406d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa440006fd0 tx=0x7fa440008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:10.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.373+0000 7fa4467fc640 1 -- 172.21.15.70:0/3010854110 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa44c053e20 con 0x7fa45006cfa0 2023-10-08T05:36:10.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:10.562+0000 7fa457e7c640 1 -- 172.21.15.70:0/3010854110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fa450071cb0 con 0x7fa45006cfa0 2023-10-08T05:36:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.141+0000 7fa4467fc640 1 -- 172.21.15.70:0/3010854110 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7fa44c017020 con 0x7fa45006cfa0 2023-10-08T05:36:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T05:36:11.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.144+0000 7fa457e7c640 1 -- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa43406ae70 msgr2=0x7fa43406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:11.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.144+0000 7fa457e7c640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa43406ae70 0x7fa43406d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa440006fd0 tx=0x7fa440008040 comp rx=0 tx=0).stop 2023-10-08T05:36:11.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.144+0000 7fa457e7c640 1 -- 172.21.15.70:0/3010854110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45006cfa0 msgr2=0x7fa45014b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:11.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.144+0000 7fa457e7c640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45006cfa0 0x7fa45014b9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c500 tx=0x7fa44c00c9c0 comp rx=0 tx=0).stop 2023-10-08T05:36:11.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.145+0000 7fa457e7c640 1 -- 172.21.15.70:0/3010854110 shutdown_connections 2023-10-08T05:36:11.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.145+0000 7fa457e7c640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa43406ae70 0x7fa43406d330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:11.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.145+0000 7fa457e7c640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa45015b630 0x7fa4501f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:11.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.145+0000 7fa457e7c640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4500cb500 0x7fa450149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:11.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.145+0000 7fa457e7c640 1 --2- 172.21.15.70:0/3010854110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45006cfa0 0x7fa45014b9f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:11.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.145+0000 7fa457e7c640 1 -- 172.21.15.70:0/3010854110 >> 172.21.15.70:0/3010854110 conn(0x7fa45006a140 msgr2=0x7fa4500c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:11.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.145+0000 7fa457e7c640 1 -- 172.21.15.70:0/3010854110 shutdown_connections 2023-10-08T05:36:11.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.145+0000 7fa457e7c640 1 -- 172.21.15.70:0/3010854110 wait complete. 2023-10-08T05:36:11.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T05:36:11.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.366+0000 7ff747dd0640 1 Processor -- start 2023-10-08T05:36:11.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.366+0000 7ff747dd0640 1 -- start start 2023-10-08T05:36:11.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff747dd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400702b0 0x7ff740070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:11.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff747dd0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff7400b21f0 0x7ff7400c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:11.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff747dd0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74006c3a0 con 0x7ff7400ca1a0 2023-10-08T05:36:11.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff747dd0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff740070bd0 con 0x7ff7400b21f0 2023-10-08T05:36:11.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff747dd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74006c640 con 0x7ff7400702b0 2023-10-08T05:36:11.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff74536b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff7400b21f0 0x7ff7400c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:11.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff74536b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff7400b21f0 0x7ff7400c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57100/0 (socket says 172.21.15.70:57100) 2023-10-08T05:36:11.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff74536b640 1 -- 172.21.15.70:0/2021451515 learned_addr learned my addr 172.21.15.70:0/2021451515 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:11.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff74536b640 1 -- 172.21.15.70:0/2021451515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400702b0 msgr2=0x7ff740070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:11.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff745b6c640 1 --2- 172.21.15.70:0/2021451515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400702b0 0x7ff740070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:11.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff744b6a640 1 -- 172.21.15.70:0/2021451515 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940800473 0 0) 0x7ff74006c3a0 con 0x7ff7400ca1a0 2023-10-08T05:36:11.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff74536b640 1 --2- 172.21.15.70:0/2021451515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400702b0 0x7ff740070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:11.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff74536b640 1 -- 172.21.15.70:0/2021451515 >> v1:172.21.15.70:6789/0 conn(0x7ff7400ca1a0 legacy=0x7ff74006d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:11.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.368+0000 7ff74536b640 1 -- 172.21.15.70:0/2021451515 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74006c920 con 0x7ff7400b21f0 2023-10-08T05:36:11.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.369+0000 7ff745b6c640 1 --2- 172.21.15.70:0/2021451515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400702b0 0x7ff740070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:11.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.370+0000 7ff74536b640 1 --2- 172.21.15.70:0/2021451515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 0x7ff7400c9c60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff730009870 tx=0x7ff73002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a662cc7eaafe9f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:11.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.370+0000 7ff744b6a640 1 -- 172.21.15.70:0/2021451515 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730030840 con 0x7ff7400b21f0 2023-10-08T05:36:11.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.370+0000 7ff744b6a640 1 -- 172.21.15.70:0/2021451515 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7300309a0 con 0x7ff7400b21f0 2023-10-08T05:36:11.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.370+0000 7ff744b6a640 1 -- 172.21.15.70:0/2021451515 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730030d30 con 0x7ff7400b21f0 2023-10-08T05:36:11.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.370+0000 7ff747dd0640 1 -- 172.21.15.70:0/2021451515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 msgr2=0x7ff7400c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:11.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.370+0000 7ff747dd0640 1 --2- 172.21.15.70:0/2021451515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 0x7ff7400c9c60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff730009870 tx=0x7ff73002b980 comp rx=0 tx=0).stop 2023-10-08T05:36:11.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.371+0000 7ff747dd0640 1 -- 172.21.15.70:0/2021451515 shutdown_connections 2023-10-08T05:36:11.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.371+0000 7ff747dd0640 1 --2- 172.21.15.70:0/2021451515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 0x7ff7400c9c60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:11.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.371+0000 7ff747dd0640 1 --2- 172.21.15.70:0/2021451515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400702b0 0x7ff740070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:11.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.371+0000 7ff747dd0640 1 -- 172.21.15.70:0/2021451515 >> 172.21.15.70:0/2021451515 conn(0x7ff74006a5d0 msgr2=0x7ff7400b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:11.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.371+0000 7ff747dd0640 1 -- 172.21.15.70:0/2021451515 shutdown_connections 2023-10-08T05:36:11.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.371+0000 7ff747dd0640 1 -- 172.21.15.70:0/2021451515 wait complete. 2023-10-08T05:36:11.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.372+0000 7ff747dd0640 1 Processor -- start 2023-10-08T05:36:11.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.372+0000 7ff747dd0640 1 -- start start 2023-10-08T05:36:11.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.373+0000 7ff747dd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7400702b0 0x7ff7401501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:11.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.373+0000 7ff747dd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 0x7ff7400c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:11.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.373+0000 7ff747dd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400ca1a0 0x7ff7400c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:11.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.373+0000 7ff747dd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7401506f0 con 0x7ff7400b21f0 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.373+0000 7ff747dd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74015bc10 con 0x7ff7400702b0 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.373+0000 7ff747dd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74015c0a0 con 0x7ff7400ca1a0 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff74536b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 0x7ff7400c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff74536b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 0x7ff7400c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57112/0 (socket says 172.21.15.70:57112) 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff74536b640 1 -- 172.21.15.70:0/2204170950 learned_addr learned my addr 172.21.15.70:0/2204170950 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff745b6c640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7400702b0 0x7ff7401501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff74636d640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400ca1a0 0x7ff7400c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff74536b640 1 -- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400ca1a0 msgr2=0x7ff7400c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff74536b640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400ca1a0 0x7ff7400c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff74536b640 1 -- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7400702b0 msgr2=0x7ff7401501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff74536b640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7400702b0 0x7ff7401501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff74536b640 1 -- 172.21.15.70:0/2204170950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300094d0 con 0x7ff7400b21f0 2023-10-08T05:36:11.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff745b6c640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7400702b0 0x7ff7401501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:11.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.374+0000 7ff74536b640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 0x7ff7400c75e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff73002b9c0 tx=0x7ff730004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:11.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.375+0000 7ff7367fc640 1 -- 172.21.15.70:0/2204170950 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730030810 con 0x7ff7400b21f0 2023-10-08T05:36:11.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.375+0000 7ff7367fc640 1 -- 172.21.15.70:0/2204170950 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff730044480 con 0x7ff7400b21f0 2023-10-08T05:36:11.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.375+0000 7ff7367fc640 1 -- 172.21.15.70:0/2204170950 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730044850 con 0x7ff7400b21f0 2023-10-08T05:36:11.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.375+0000 7ff747dd0640 1 -- 172.21.15.70:0/2204170950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7401584f0 con 0x7ff7400b21f0 2023-10-08T05:36:11.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.375+0000 7ff747dd0640 1 -- 172.21.15.70:0/2204170950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74015b5c0 con 0x7ff7400b21f0 2023-10-08T05:36:11.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.376+0000 7ff747dd0640 1 -- 172.21.15.70:0/2204170950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74006cbe0 con 0x7ff7400b21f0 2023-10-08T05:36:11.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.377+0000 7ff7367fc640 1 -- 172.21.15.70:0/2204170950 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff73002f390 con 0x7ff7400b21f0 2023-10-08T05:36:11.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.378+0000 7ff7367fc640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72406ae70 0x7ff72406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:11.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.378+0000 7ff7367fc640 1 -- 172.21.15.70:0/2204170950 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff73003e070 con 0x7ff7400b21f0 2023-10-08T05:36:11.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.378+0000 7ff745b6c640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72406ae70 0x7ff72406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:11.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.380+0000 7ff745b6c640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72406ae70 0x7ff72406d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff73c006fd0 tx=0x7ff73c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:11.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.382+0000 7ff7367fc640 1 -- 172.21.15.70:0/2204170950 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff74006cbe0 con 0x7ff7400b21f0 2023-10-08T05:36:11.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:11.610+0000 7ff747dd0640 1 -- 172.21.15.70:0/2204170950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff74006cbe0 con 0x7ff7400b21f0 2023-10-08T05:36:12.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.143+0000 7ff7367fc640 1 -- 172.21.15.70:0/2204170950 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff74006cbe0 con 0x7ff7400b21f0 2023-10-08T05:36:12.145 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 2 and data pool 3 2023-10-08T05:36:12.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 -- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72406ae70 msgr2=0x7ff72406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:12.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72406ae70 0x7ff72406d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff73c006fd0 tx=0x7ff73c008040 comp rx=0 tx=0).stop 2023-10-08T05:36:12.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 -- 172.21.15.70:0/2204170950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 msgr2=0x7ff7400c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:12.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 0x7ff7400c75e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff73002b9c0 tx=0x7ff730004730 comp rx=0 tx=0).stop 2023-10-08T05:36:12.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 -- 172.21.15.70:0/2204170950 shutdown_connections 2023-10-08T05:36:12.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72406ae70 0x7ff72406d330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7400ca1a0 0x7ff7400c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7400b21f0 0x7ff7400c75e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 --2- 172.21.15.70:0/2204170950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7400702b0 0x7ff7401501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 -- 172.21.15.70:0/2204170950 >> 172.21.15.70:0/2204170950 conn(0x7ff74006a5d0 msgr2=0x7ff7400b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:12.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 -- 172.21.15.70:0/2204170950 shutdown_connections 2023-10-08T05:36:12.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.147+0000 7ff747dd0640 1 -- 172.21.15.70:0/2204170950 wait complete. 2023-10-08T05:36:12.173 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-08T05:36:12.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-08T05:36:12.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.366+0000 7f4acc7b6640 1 Processor -- start 2023-10-08T05:36:12.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.366+0000 7f4acc7b6640 1 -- start start 2023-10-08T05:36:12.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.368+0000 7f4acc7b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac415aad0 0x7f4ac40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:12.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.368+0000 7f4acc7b6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ac40c82b0 0x7f4ac40c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:12.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.368+0000 7f4acc7b6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac415bf30 con 0x7f4ac406cd00 2023-10-08T05:36:12.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.368+0000 7f4acc7b6640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac415b320 con 0x7f4ac40c82b0 2023-10-08T05:36:12.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.368+0000 7f4acc7b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac415b600 con 0x7f4ac415aad0 2023-10-08T05:36:12.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.369+0000 7f4aca552640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac415aad0 0x7f4ac40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:12.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.369+0000 7f4ac9d51640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ac40c82b0 0x7f4ac40c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:12.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.369+0000 7f4ac9d51640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ac40c82b0 0x7f4ac40c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:53012/0 (socket says 172.21.15.70:53012) 2023-10-08T05:36:12.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.369+0000 7f4aca552640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac415aad0 0x7f4ac40b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:56098/0 (socket says 172.21.15.70:56098) 2023-10-08T05:36:12.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.369+0000 7f4ac9d51640 1 -- 172.21.15.70:0/2713593151 learned_addr learned my addr 172.21.15.70:0/2713593151 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:12.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.370+0000 7f4ac9d51640 1 -- 172.21.15.70:0/2713593151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac415aad0 msgr2=0x7f4ac40b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:12.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.370+0000 7f4ac9d51640 1 --2- 172.21.15.70:0/2713593151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac415aad0 0x7f4ac40b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.370+0000 7f4ac9d51640 1 -- 172.21.15.70:0/2713593151 >> v1:172.21.15.181:6789/0 conn(0x7f4ac406cd00 legacy=0x7f4ac4070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:12.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.370+0000 7f4ac9d51640 1 -- 172.21.15.70:0/2713593151 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac415b8e0 con 0x7f4ac40c82b0 2023-10-08T05:36:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.371+0000 7f4ac9d51640 1 --2- 172.21.15.70:0/2713593151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac40c82b0 0x7f4ac40c8710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009870 tx=0x7f4ab402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0d3f5eb941f5b25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.371+0000 7f4ac9550640 1 -- 172.21.15.70:0/2713593151 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4030840 con 0x7f4ac40c82b0 2023-10-08T05:36:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.371+0000 7f4ac9550640 1 -- 172.21.15.70:0/2713593151 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab40309a0 con 0x7f4ac40c82b0 2023-10-08T05:36:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.371+0000 7f4ac9550640 1 -- 172.21.15.70:0/2713593151 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4030d30 con 0x7f4ac40c82b0 2023-10-08T05:36:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.372+0000 7f4acc7b6640 1 -- 172.21.15.70:0/2713593151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac40c82b0 msgr2=0x7f4ac40c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.372+0000 7f4acc7b6640 1 --2- 172.21.15.70:0/2713593151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac40c82b0 0x7f4ac40c8710 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009870 tx=0x7f4ab402b980 comp rx=0 tx=0).stop 2023-10-08T05:36:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.372+0000 7f4acc7b6640 1 -- 172.21.15.70:0/2713593151 shutdown_connections 2023-10-08T05:36:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.372+0000 7f4acc7b6640 1 --2- 172.21.15.70:0/2713593151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac40c82b0 0x7f4ac40c8710 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.372+0000 7f4acc7b6640 1 --2- 172.21.15.70:0/2713593151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac415aad0 0x7f4ac40b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.372+0000 7f4acc7b6640 1 -- 172.21.15.70:0/2713593151 >> 172.21.15.70:0/2713593151 conn(0x7f4ac4069d50 msgr2=0x7f4ac406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.372+0000 7f4acc7b6640 1 -- 172.21.15.70:0/2713593151 shutdown_connections 2023-10-08T05:36:12.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.372+0000 7f4acc7b6640 1 -- 172.21.15.70:0/2713593151 wait complete. 2023-10-08T05:36:12.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.374+0000 7f4acc7b6640 1 Processor -- start 2023-10-08T05:36:12.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.374+0000 7f4acc7b6640 1 -- start start 2023-10-08T05:36:12.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4acc7b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac406cd00 0x7f4ac4152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:12.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4acc7b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ac40c82b0 0x7f4ac41f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4acc7b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac415aad0 0x7f4ac41f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4acc7b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac41f6980 con 0x7f4ac40c82b0 2023-10-08T05:36:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4acc7b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac41f33d0 con 0x7f4ac415aad0 2023-10-08T05:36:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4acc7b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac41f60e0 con 0x7f4ac406cd00 2023-10-08T05:36:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4ac9d51640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ac40c82b0 0x7f4ac41f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4ac9d51640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ac40c82b0 0x7f4ac41f5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57116/0 (socket says 172.21.15.70:57116) 2023-10-08T05:36:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4ac9d51640 1 -- 172.21.15.70:0/311748417 learned_addr learned my addr 172.21.15.70:0/311748417 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4acad53640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac415aad0 0x7f4ac41f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4ac9d51640 1 -- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac406cd00 msgr2=0x7f4ac4152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4aca552640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac406cd00 0x7f4ac4152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.375+0000 7f4ac9d51640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac406cd00 0x7f4ac4152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.376+0000 7f4ac9d51640 1 -- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac415aad0 msgr2=0x7f4ac41f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.376+0000 7f4ac9d51640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac415aad0 0x7f4ac41f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.376+0000 7f4ac9d51640 1 -- 172.21.15.70:0/311748417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab40094d0 con 0x7f4ac40c82b0 2023-10-08T05:36:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.376+0000 7f4acad53640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac415aad0 0x7f4ac41f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.376+0000 7f4ac9d51640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ac40c82b0 0x7f4ac41f5a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4000c00 tx=0x7f4ab4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:12.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.376+0000 7f4abaffd640 1 -- 172.21.15.70:0/311748417 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4039c90 con 0x7f4ac40c82b0 2023-10-08T05:36:12.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.376+0000 7f4abaffd640 1 -- 172.21.15.70:0/311748417 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab4039df0 con 0x7f4ac40c82b0 2023-10-08T05:36:12.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.376+0000 7f4abaffd640 1 -- 172.21.15.70:0/311748417 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4043640 con 0x7f4ac40c82b0 2023-10-08T05:36:12.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.376+0000 7f4acc7b6640 1 -- 172.21.15.70:0/311748417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac41f63c0 con 0x7f4ac40c82b0 2023-10-08T05:36:12.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.376+0000 7f4acc7b6640 1 -- 172.21.15.70:0/311748417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac41572d0 con 0x7f4ac40c82b0 2023-10-08T05:36:12.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.377+0000 7f4acc7b6640 1 -- 172.21.15.70:0/311748417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac4157130 con 0x7f4ac40c82b0 2023-10-08T05:36:12.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.378+0000 7f4abaffd640 1 -- 172.21.15.70:0/311748417 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4ab40437a0 con 0x7f4ac40c82b0 2023-10-08T05:36:12.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.379+0000 7f4abaffd640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4aa806ae70 0x7f4aa806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:12.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.380+0000 7f4abaffd640 1 -- 172.21.15.70:0/311748417 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ab403e070 con 0x7f4ac40c82b0 2023-10-08T05:36:12.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.380+0000 7f4aca552640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4aa806ae70 0x7f4aa806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:12.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.381+0000 7f4aca552640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4aa806ae70 0x7f4aa806d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40cd960 tx=0x7f4ac0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:12.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.384+0000 7f4abaffd640 1 -- 172.21.15.70:0/311748417 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ac4157130 con 0x7f4ac40c82b0 2023-10-08T05:36:12.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.578+0000 7f4acc7b6640 1 -- 172.21.15.70:0/311748417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f4ac4157130 con 0x7f4ac40c82b0 2023-10-08T05:36:12.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.908+0000 7f4abaffd640 1 -- 172.21.15.70:0/311748417 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f4ac4157130 con 0x7f4ac40c82b0 2023-10-08T05:36:12.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.913+0000 7f4acc7b6640 1 -- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4aa806ae70 msgr2=0x7f4aa806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:12.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.913+0000 7f4acc7b6640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4aa806ae70 0x7f4aa806d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40cd960 tx=0x7f4ac0008040 comp rx=0 tx=0).stop 2023-10-08T05:36:12.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.913+0000 7f4acc7b6640 1 -- 172.21.15.70:0/311748417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ac40c82b0 msgr2=0x7f4ac41f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:12.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.913+0000 7f4acc7b6640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ac40c82b0 0x7f4ac41f5a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4000c00 tx=0x7f4ab4004730 comp rx=0 tx=0).stop 2023-10-08T05:36:12.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.913+0000 7f4acc7b6640 1 -- 172.21.15.70:0/311748417 shutdown_connections 2023-10-08T05:36:12.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.913+0000 7f4acc7b6640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4aa806ae70 0x7f4aa806d330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.913+0000 7f4acc7b6640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ac415aad0 0x7f4ac41f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.913+0000 7f4acc7b6640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ac40c82b0 0x7f4ac41f5a00 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4000c00 tx=0x7f4ab4004730 comp rx=0 tx=0).stop 2023-10-08T05:36:12.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.913+0000 7f4acc7b6640 1 --2- 172.21.15.70:0/311748417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ac406cd00 0x7f4ac4152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:12.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.913+0000 7f4acc7b6640 1 -- 172.21.15.70:0/311748417 >> 172.21.15.70:0/311748417 conn(0x7f4ac4069d50 msgr2=0x7f4ac415e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:12.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.914+0000 7f4acc7b6640 1 -- 172.21.15.70:0/311748417 shutdown_connections 2023-10-08T05:36:12.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:12.914+0000 7f4acc7b6640 1 -- 172.21.15.70:0/311748417 wait complete. 2023-10-08T05:36:12.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T05:36:13.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.141+0000 7f3da67ae640 1 Processor -- start 2023-10-08T05:36:13.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.141+0000 7f3da67ae640 1 -- start start 2023-10-08T05:36:13.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.143+0000 7f3da67ae640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3da00caf20 0x7f3da00ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:13.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.143+0000 7f3da67ae640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da00b1360 con 0x7f3da00aff90 2023-10-08T05:36:13.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.143+0000 7f3da67ae640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da00cb350 con 0x7f3da015ab70 2023-10-08T05:36:13.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.143+0000 7f3da67ae640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da00b16b0 con 0x7f3da00caf20 2023-10-08T05:36:13.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.143+0000 7f3da57ac640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3da00aff90 0x7f3da00b0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49858/0 (socket says 172.21.15.70:49858) 2023-10-08T05:36:13.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.143+0000 7f3da57ac640 1 -- 172.21.15.70:0/2583351436 learned_addr learned my addr 172.21.15.70:0/2583351436 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:13.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.143+0000 7f3da5fad640 1 --2- 172.21.15.70:0/2583351436 >> v2:172.21.15.181:3300/0 conn(0x7f3da00caf20 0x7f3da00ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:13.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858895004 0 0) 0x7f3da00b1360 con 0x7f3da00aff90 2023-10-08T05:36:13.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d8c003520 con 0x7f3da00aff90 2023-10-08T05:36:13.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 499043178 0 0) 0x7f3d8c003520 con 0x7f3da00aff90 2023-10-08T05:36:13.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 >> v2:172.21.15.181:3300/0 conn(0x7f3da00caf20 msgr2=0x7f3da00ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:13.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 --2- 172.21.15.70:0/2583351436 >> v2:172.21.15.181:3300/0 conn(0x7f3da00caf20 0x7f3da00ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:13.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 >> v1:172.21.15.181:6789/0 conn(0x7f3da015ab70 legacy=0x7f3da00b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:13.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da00cb620 con 0x7f3da00aff90 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3d9c001670 con 0x7f3da00aff90 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 >> v1:172.21.15.70:6789/0 conn(0x7f3da00aff90 legacy=0x7f3da00b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d8c003970 0x7f3d8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d8c0047c0 0x7f3d8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3d8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da00b16b0 con 0x7f3d8c005180 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c00c120 con 0x7f3d8c0047c0 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.144+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c00c2e0 con 0x7f3d8c003970 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.145+0000 7f3da5fad640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3d8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.145+0000 7f3da4fab640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d8c0047c0 0x7f3d8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.145+0000 7f3da57ac640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d8c003970 0x7f3d8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:13.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.145+0000 7f3da5fad640 1 -- 172.21.15.70:0/2583351436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d8c003970 msgr2=0x7f3d8c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:13.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.145+0000 7f3da5fad640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d8c003970 0x7f3d8c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:13.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.145+0000 7f3da5fad640 1 -- 172.21.15.70:0/2583351436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d8c0047c0 msgr2=0x7f3d8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:13.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.145+0000 7f3da5fad640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d8c0047c0 0x7f3d8c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:13.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.145+0000 7f3da5fad640 1 -- 172.21.15.70:0/2583351436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c003520 con 0x7f3d8c005180 2023-10-08T05:36:13.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.146+0000 7f3da5fad640 1 -- 172.21.15.70:0/2583351436 reap_dead start 2023-10-08T05:36:13.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.146+0000 7f3da5fad640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3d8c00b980 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009fc0 tx=0x7f3d8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a533d67d47c0adc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:13.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.148+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9802cc60 con 0x7f3d8c005180 2023-10-08T05:36:13.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.148+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9800f040 con 0x7f3d8c005180 2023-10-08T05:36:13.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.148+0000 7f3d97fff640 1 -- 172.21.15.70:0/2583351436 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98003990 con 0x7f3d8c005180 2023-10-08T05:36:13.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.148+0000 7f3da67ae640 1 -- 172.21.15.70:0/2583351436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 msgr2=0x7f3d8c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:13.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.148+0000 7f3da67ae640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3d8c00b980 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009fc0 tx=0x7f3d8c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:13.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.148+0000 7f3da67ae640 1 -- 172.21.15.70:0/2583351436 shutdown_connections 2023-10-08T05:36:13.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.148+0000 7f3da67ae640 1 --2- 172.21.15.70:0/2583351436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3d8c00b980 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:13.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.148+0000 7f3da67ae640 1 -- 172.21.15.70:0/2583351436 >> 172.21.15.70:0/2583351436 conn(0x7f3da006c0c0 msgr2=0x7f3da006c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:13.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.148+0000 7f3da67ae640 1 -- 172.21.15.70:0/2583351436 shutdown_connections 2023-10-08T05:36:13.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.149+0000 7f3da67ae640 1 -- 172.21.15.70:0/2583351436 wait complete. 2023-10-08T05:36:13.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.150+0000 7f3da67ae640 1 Processor -- start 2023-10-08T05:36:13.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.150+0000 7f3da67ae640 1 -- start start 2023-10-08T05:36:13.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.151+0000 7f3da67ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3da014d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:13.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.151+0000 7f3da67ae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3da01e3100 0x7f3da01e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:13.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.151+0000 7f3da67ae640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3da01588e0 0x7f3da0159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:13.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.151+0000 7f3da67ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da00cad60 con 0x7f3d8c005180 2023-10-08T05:36:13.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.151+0000 7f3da67ae640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da01584c0 con 0x7f3da01e3100 2023-10-08T05:36:13.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.151+0000 7f3da67ae640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da01581b0 con 0x7f3da01588e0 2023-10-08T05:36:13.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.152+0000 7f3da57ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3da014d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:13.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.152+0000 7f3da57ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3da014d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57138/0 (socket says 172.21.15.70:57138) 2023-10-08T05:36:13.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.152+0000 7f3da57ac640 1 -- 172.21.15.70:0/1734526690 learned_addr learned my addr 172.21.15.70:0/1734526690 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:13.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.152+0000 7f3da4fab640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3da01e3100 0x7f3da01e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:13.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.152+0000 7f3da57ac640 1 -- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3da01588e0 msgr2=0x7f3da0159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:13.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.152+0000 7f3da57ac640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3da01588e0 0x7f3da0159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:13.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.152+0000 7f3da57ac640 1 -- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3da01e3100 msgr2=0x7f3da01e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:13.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.152+0000 7f3da57ac640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3da01e3100 0x7f3da01e4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:13.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.152+0000 7f3da57ac640 1 -- 172.21.15.70:0/1734526690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da00ac880 con 0x7f3d8c005180 2023-10-08T05:36:13.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.152+0000 7f3da4fab640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3da01e3100 0x7f3da01e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:13.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.153+0000 7f3da57ac640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3da014d9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c001050 tx=0x7f3d9c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:13.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.154+0000 7f3d95ffb640 1 -- 172.21.15.70:0/1734526690 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00ee60 con 0x7f3d8c005180 2023-10-08T05:36:13.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.154+0000 7f3d95ffb640 1 -- 172.21.15.70:0/1734526690 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c00c8d0 con 0x7f3d8c005180 2023-10-08T05:36:13.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.154+0000 7f3d95ffb640 1 -- 172.21.15.70:0/1734526690 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00cc80 con 0x7f3d8c005180 2023-10-08T05:36:13.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.154+0000 7f3da67ae640 1 -- 172.21.15.70:0/1734526690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da00acb60 con 0x7f3d8c005180 2023-10-08T05:36:13.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.154+0000 7f3da67ae640 1 -- 172.21.15.70:0/1734526690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da014dee0 con 0x7f3d8c005180 2023-10-08T05:36:13.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.155+0000 7f3da67ae640 1 -- 172.21.15.70:0/1734526690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da00cbbc0 con 0x7f3d8c005180 2023-10-08T05:36:13.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.156+0000 7f3d95ffb640 1 -- 172.21.15.70:0/1734526690 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3d9c01d460 con 0x7f3d8c005180 2023-10-08T05:36:13.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.157+0000 7f3d95ffb640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3d8406aea0 0x7f3d8406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:13.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.157+0000 7f3d95ffb640 1 -- 172.21.15.70:0/1734526690 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5935+0+0 (secure 0 0 0) 0x7f3d9c0895b0 con 0x7f3d8c005180 2023-10-08T05:36:13.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.158+0000 7f3da4fab640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3d8406aea0 0x7f3d8406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:13.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.159+0000 7f3da4fab640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3d8406aea0 0x7f3d8406d360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3da0071950 tx=0x7f3d90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:13.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.162+0000 7f3d95ffb640 1 -- 172.21.15.70:0/1734526690 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3da00cbbc0 con 0x7f3d8c005180 2023-10-08T05:36:13.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:13.354+0000 7f3da67ae640 1 -- 172.21.15.70:0/1734526690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3da00cbbc0 con 0x7f3d8c005180 2023-10-08T05:36:15.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.169+0000 7f3d95ffb640 1 -- 172.21.15.70:0/1734526690 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f3da00cbbc0 con 0x7f3d8c005180 2023-10-08T05:36:15.170 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data_ec' created 2023-10-08T05:36:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.173+0000 7f3da67ae640 1 -- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3d8406aea0 msgr2=0x7f3d8406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.173+0000 7f3da67ae640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3d8406aea0 0x7f3d8406d360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3da0071950 tx=0x7f3d90009040 comp rx=0 tx=0).stop 2023-10-08T05:36:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.173+0000 7f3da67ae640 1 -- 172.21.15.70:0/1734526690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 msgr2=0x7f3da014d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.173+0000 7f3da67ae640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3da014d9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c001050 tx=0x7f3d9c00ec10 comp rx=0 tx=0).stop 2023-10-08T05:36:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.174+0000 7f3da67ae640 1 -- 172.21.15.70:0/1734526690 shutdown_connections 2023-10-08T05:36:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.174+0000 7f3da67ae640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3d8406aea0 0x7f3d8406d360 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.174+0000 7f3da67ae640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3da01588e0 0x7f3da0159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.174+0000 7f3da67ae640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3da01e3100 0x7f3da01e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.174+0000 7f3da67ae640 1 --2- 172.21.15.70:0/1734526690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d8c005180 0x7f3da014d9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.174+0000 7f3da67ae640 1 -- 172.21.15.70:0/1734526690 >> 172.21.15.70:0/1734526690 conn(0x7f3da006c0c0 msgr2=0x7f3da014fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.175+0000 7f3da67ae640 1 -- 172.21.15.70:0/1734526690 shutdown_connections 2023-10-08T05:36:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.175+0000 7f3da67ae640 1 -- 172.21.15.70:0/1734526690 wait complete. 2023-10-08T05:36:15.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-08T05:36:15.303 INFO:tasks.ceph.mds.b.smithi181.stderr:2023-10-08T05:36:15.302+0000 7f6c0aa5d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:36:15.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.395+0000 7f38196a2640 1 Processor -- start 2023-10-08T05:36:15.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.395+0000 7f38196a2640 1 -- start start 2023-10-08T05:36:15.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.397+0000 7f38196a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140b1750 0x7f381415d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:15.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.397+0000 7f38196a2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38140b40e0 0x7f38140c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.397+0000 7f38196a2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38140b30a0 con 0x7f38140702d0 2023-10-08T05:36:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.397+0000 7f38196a2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381415dcf0 con 0x7f38140b40e0 2023-10-08T05:36:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.397+0000 7f38196a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140b3320 con 0x7f38140b1750 2023-10-08T05:36:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.398+0000 7f38137fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38140b40e0 0x7f38140c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.398+0000 7f38127fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140b1750 0x7f381415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:15.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.398+0000 7f3812ffd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f38140702d0 0x7f38140706b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:59452/0 (socket says 172.21.15.70:59452) 2023-10-08T05:36:15.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.398+0000 7f3812ffd640 1 -- 172.21.15.70:0/41069559 learned_addr learned my addr 172.21.15.70:0/41069559 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:15.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.398+0000 7f38137fe640 1 -- 172.21.15.70:0/41069559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140b1750 msgr2=0x7f381415d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:15.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.398+0000 7f38137fe640 1 --2- 172.21.15.70:0/41069559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140b1750 0x7f381415d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:15.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.398+0000 7f38137fe640 1 -- 172.21.15.70:0/41069559 >> v1:172.21.15.181:6789/0 conn(0x7f38140702d0 legacy=0x7f38140706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:15.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.398+0000 7f38137fe640 1 -- 172.21.15.70:0/41069559 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140b3600 con 0x7f38140b40e0 2023-10-08T05:36:15.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.399+0000 7f38127fc640 1 --2- 172.21.15.70:0/41069559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140b1750 0x7f381415d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:36:15.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.400+0000 7f38137fe640 1 --2- 172.21.15.70:0/41069559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140b40e0 0x7f38140c8dd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f380c00aea0 tx=0x7f380c02e020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d657cf0f645a3555 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:15.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f3811ffb640 1 -- 172.21.15.70:0/41069559 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c02ebc0 con 0x7f38140b40e0 2023-10-08T05:36:15.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f3811ffb640 1 -- 172.21.15.70:0/41069559 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c02ed20 con 0x7f38140b40e0 2023-10-08T05:36:15.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f3811ffb640 1 -- 172.21.15.70:0/41069559 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c008660 con 0x7f38140b40e0 2023-10-08T05:36:15.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f38196a2640 1 -- 172.21.15.70:0/41069559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140b40e0 msgr2=0x7f38140c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:15.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f38196a2640 1 --2- 172.21.15.70:0/41069559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140b40e0 0x7f38140c8dd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f380c00aea0 tx=0x7f380c02e020 comp rx=0 tx=0).stop 2023-10-08T05:36:15.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f38196a2640 1 -- 172.21.15.70:0/41069559 shutdown_connections 2023-10-08T05:36:15.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f38196a2640 1 --2- 172.21.15.70:0/41069559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140b40e0 0x7f38140c8dd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:15.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f38196a2640 1 --2- 172.21.15.70:0/41069559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140b1750 0x7f381415d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:15.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f38196a2640 1 -- 172.21.15.70:0/41069559 >> 172.21.15.70:0/41069559 conn(0x7f3814069d50 msgr2=0x7f381406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:15.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f38196a2640 1 -- 172.21.15.70:0/41069559 shutdown_connections 2023-10-08T05:36:15.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.401+0000 7f38196a2640 1 -- 172.21.15.70:0/41069559 wait complete. 2023-10-08T05:36:15.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.403+0000 7f38196a2640 1 Processor -- start 2023-10-08T05:36:15.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.404+0000 7f38196a2640 1 -- start start 2023-10-08T05:36:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f38196a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140702d0 0x7f3814151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f38196a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140b1750 0x7f38141f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f38196a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38140b40e0 0x7f38140c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f38196a2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3814151890 con 0x7f38140b40e0 2023-10-08T05:36:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f38196a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381414cc60 con 0x7f38140b1750 2023-10-08T05:36:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f38196a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3814150ad0 con 0x7f38140702d0 2023-10-08T05:36:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f38137fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38140b40e0 0x7f38140c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:15.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f38137fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38140b40e0 0x7f38140c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45288/0 (socket says 172.21.15.70:45288) 2023-10-08T05:36:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f38137fe640 1 -- 172.21.15.70:0/3446706120 learned_addr learned my addr 172.21.15.70:0/3446706120 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f3812ffd640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140702d0 0x7f3814151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.405+0000 7f38127fc640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140b1750 0x7f38141f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f38137fe640 1 -- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140702d0 msgr2=0x7f3814151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f38137fe640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140702d0 0x7f3814151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f38137fe640 1 -- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140b1750 msgr2=0x7f38141f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f38137fe640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140b1750 0x7f38141f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:15.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f38137fe640 1 -- 172.21.15.70:0/3446706120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3808009400 con 0x7f38140b40e0 2023-10-08T05:36:15.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f38137fe640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38140b40e0 0x7f38140c9a70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f380c00aea0 tx=0x7f380c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:15.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f37fb7fe640 1 -- 172.21.15.70:0/3446706120 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c040620 con 0x7f38140b40e0 2023-10-08T05:36:15.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f37fb7fe640 1 -- 172.21.15.70:0/3446706120 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c02ee90 con 0x7f38140b40e0 2023-10-08T05:36:15.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f37fb7fe640 1 -- 172.21.15.70:0/3446706120 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c0409c0 con 0x7f38140b40e0 2023-10-08T05:36:15.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f38196a2640 1 -- 172.21.15.70:0/3446706120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c00ab80 con 0x7f38140b40e0 2023-10-08T05:36:15.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.406+0000 7f38196a2640 1 -- 172.21.15.70:0/3446706120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3814151090 con 0x7f38140b40e0 2023-10-08T05:36:15.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.407+0000 7f38196a2640 1 -- 172.21.15.70:0/3446706120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381404e160 con 0x7f38140b40e0 2023-10-08T05:36:15.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.409+0000 7f37fb7fe640 1 -- 172.21.15.70:0/3446706120 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f380c040b20 con 0x7f38140b40e0 2023-10-08T05:36:15.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.409+0000 7f37fb7fe640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f380006ae70 0x7f380006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:15.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.410+0000 7f37fb7fe640 1 -- 172.21.15.70:0/3446706120 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f380c0ac240 con 0x7f38140b40e0 2023-10-08T05:36:15.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.410+0000 7f3812ffd640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f380006ae70 0x7f380006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:15.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.411+0000 7f3812ffd640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f380006ae70 0x7f380006d330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3804005880 tx=0x7f3804009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:15.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.414+0000 7f37fb7fe640 1 -- 172.21.15.70:0/3446706120 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f380c0771d0 con 0x7f38140b40e0 2023-10-08T05:36:15.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:15.602+0000 7f38196a2640 1 -- 172.21.15.70:0/3446706120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f38140710b0 con 0x7f38140b40e0 2023-10-08T05:36:16.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.185+0000 7f37fb7fe640 1 -- 172.21.15.70:0/3446706120 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f380c00a560 con 0x7f38140b40e0 2023-10-08T05:36:16.186 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 4 allow_ec_overwrites to true 2023-10-08T05:36:16.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.189+0000 7f38196a2640 1 -- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f380006ae70 msgr2=0x7f380006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:16.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.189+0000 7f38196a2640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f380006ae70 0x7f380006d330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3804005880 tx=0x7f3804009040 comp rx=0 tx=0).stop 2023-10-08T05:36:16.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.189+0000 7f38196a2640 1 -- 172.21.15.70:0/3446706120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38140b40e0 msgr2=0x7f38140c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:16.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.189+0000 7f38196a2640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38140b40e0 0x7f38140c9a70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f380c00aea0 tx=0x7f380c004a40 comp rx=0 tx=0).stop 2023-10-08T05:36:16.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.191+0000 7f38196a2640 1 -- 172.21.15.70:0/3446706120 shutdown_connections 2023-10-08T05:36:16.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.191+0000 7f38196a2640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f380006ae70 0x7f380006d330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:16.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.191+0000 7f38196a2640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38140b40e0 0x7f38140c9a70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:16.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.191+0000 7f38196a2640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38140b1750 0x7f38141f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:16.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.191+0000 7f38196a2640 1 --2- 172.21.15.70:0/3446706120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38140702d0 0x7f3814151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:16.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.191+0000 7f38196a2640 1 -- 172.21.15.70:0/3446706120 >> 172.21.15.70:0/3446706120 conn(0x7f3814069d50 msgr2=0x7f3814153960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:16.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.192+0000 7f38196a2640 1 -- 172.21.15.70:0/3446706120 shutdown_connections 2023-10-08T05:36:16.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.192+0000 7f38196a2640 1 -- 172.21.15.70:0/3446706120 wait complete. 2023-10-08T05:36:16.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-08T05:36:16.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.418+0000 7f47b527c640 1 Processor -- start 2023-10-08T05:36:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.418+0000 7f47b527c640 1 -- start start 2023-10-08T05:36:16.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.419+0000 7f47b527c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f47b006c3a0 0x7f47b00c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:16.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.419+0000 7f47b527c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47b006db60 0x7f47b006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:16.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.419+0000 7f47b527c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b015b310 con 0x7f47b015aa00 2023-10-08T05:36:16.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.419+0000 7f47b527c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b00ca230 con 0x7f47b006c3a0 2023-10-08T05:36:16.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.419+0000 7f47b527c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b006d290 con 0x7f47b006db60 2023-10-08T05:36:16.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.419+0000 7f47aeffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f47b006c3a0 0x7f47b00c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:16.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.420+0000 7f47aeffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f47b006c3a0 0x7f47b00c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45304/0 (socket says 172.21.15.70:45304) 2023-10-08T05:36:16.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.420+0000 7f47aeffd640 1 -- 172.21.15.70:0/998085046 learned_addr learned my addr 172.21.15.70:0/998085046 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:16.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.420+0000 7f47ae7fc640 1 --2- 172.21.15.70:0/998085046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47b006db60 0x7f47b006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:16.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.420+0000 7f47aeffd640 1 -- 172.21.15.70:0/998085046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47b006db60 msgr2=0x7f47b006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:16.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.420+0000 7f47aeffd640 1 --2- 172.21.15.70:0/998085046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47b006db60 0x7f47b006c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:16.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.420+0000 7f47aeffd640 1 -- 172.21.15.70:0/998085046 >> v1:172.21.15.70:6789/0 conn(0x7f47b015aa00 legacy=0x7f47b00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.420+0000 7f47aeffd640 1 -- 172.21.15.70:0/998085046 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00c9860 con 0x7f47b006c3a0 2023-10-08T05:36:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.420+0000 7f47adffb640 1 -- 172.21.15.70:0/998085046 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052716593 0 0) 0x7f47a8001490 con 0x7f47b015aa00 2023-10-08T05:36:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.421+0000 7f47aeffd640 1 --2- 172.21.15.70:0/998085046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006c3a0 0x7f47b00c9cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009870 tx=0x7f47a002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5926c7739d476594 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.421+0000 7f47adffb640 1 -- 172.21.15.70:0/998085046 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0030840 con 0x7f47b006c3a0 2023-10-08T05:36:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.421+0000 7f47adffb640 1 -- 172.21.15.70:0/998085046 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a00309a0 con 0x7f47b006c3a0 2023-10-08T05:36:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.421+0000 7f47adffb640 1 -- 172.21.15.70:0/998085046 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0030d30 con 0x7f47b006c3a0 2023-10-08T05:36:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.422+0000 7f47b527c640 1 -- 172.21.15.70:0/998085046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006c3a0 msgr2=0x7f47b00c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:16.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.422+0000 7f47b527c640 1 --2- 172.21.15.70:0/998085046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006c3a0 0x7f47b00c9cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009870 tx=0x7f47a002b980 comp rx=0 tx=0).stop 2023-10-08T05:36:16.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.422+0000 7f47b527c640 1 -- 172.21.15.70:0/998085046 shutdown_connections 2023-10-08T05:36:16.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.422+0000 7f47b527c640 1 --2- 172.21.15.70:0/998085046 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47b006db60 0x7f47b006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:16.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.422+0000 7f47b527c640 1 --2- 172.21.15.70:0/998085046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006c3a0 0x7f47b00c9cf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:16.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.422+0000 7f47b527c640 1 -- 172.21.15.70:0/998085046 >> 172.21.15.70:0/998085046 conn(0x7f47b006a5d0 msgr2=0x7f47b00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:16.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.422+0000 7f47b527c640 1 -- 172.21.15.70:0/998085046 shutdown_connections 2023-10-08T05:36:16.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.422+0000 7f47b527c640 1 -- 172.21.15.70:0/998085046 wait complete. 2023-10-08T05:36:16.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.424+0000 7f47b527c640 1 Processor -- start 2023-10-08T05:36:16.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.424+0000 7f47b527c640 1 -- start start 2023-10-08T05:36:16.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.425+0000 7f47b527c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47b006c3a0 0x7f47b0154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:16.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.425+0000 7f47b527c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006db60 0x7f47b0153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:16.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.425+0000 7f47b527c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47b015aa00 0x7f47b00b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:16.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.425+0000 7f47b527c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00b4100 con 0x7f47b006db60 2023-10-08T05:36:16.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.425+0000 7f47b527c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0154b40 con 0x7f47b006c3a0 2023-10-08T05:36:16.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.425+0000 7f47b527c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00b3860 con 0x7f47b015aa00 2023-10-08T05:36:16.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.425+0000 7f47ae7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006db60 0x7f47b0153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:16.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.425+0000 7f47ae7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006db60 0x7f47b0153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45312/0 (socket says 172.21.15.70:45312) 2023-10-08T05:36:16.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.425+0000 7f47ae7fc640 1 -- 172.21.15.70:0/4161834849 learned_addr learned my addr 172.21.15.70:0/4161834849 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47aeffd640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47b006c3a0 0x7f47b0154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47ae7fc640 1 -- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47b015aa00 msgr2=0x7f47b00b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47ae7fc640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47b015aa00 0x7f47b00b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47ae7fc640 1 -- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47b006c3a0 msgr2=0x7f47b0154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47ae7fc640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47b006c3a0 0x7f47b0154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47ae7fc640 1 -- 172.21.15.70:0/4161834849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a00094d0 con 0x7f47b006db60 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47aeffd640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47b006c3a0 0x7f47b0154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47ae7fc640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006db60 0x7f47b0153d30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47a400d830 tx=0x7f47a400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47977fe640 1 -- 172.21.15.70:0/4161834849 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a400b6c0 con 0x7f47b006db60 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47977fe640 1 -- 172.21.15.70:0/4161834849 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a400b820 con 0x7f47b006db60 2023-10-08T05:36:16.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47977fe640 1 -- 172.21.15.70:0/4161834849 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a400bbb0 con 0x7f47b006db60 2023-10-08T05:36:16.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47b527c640 1 -- 172.21.15.70:0/4161834849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b00b3b40 con 0x7f47b006db60 2023-10-08T05:36:16.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.426+0000 7f47b527c640 1 -- 172.21.15.70:0/4161834849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b00c2410 con 0x7f47b006db60 2023-10-08T05:36:16.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.427+0000 7f47b527c640 1 -- 172.21.15.70:0/4161834849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b004e160 con 0x7f47b006db60 2023-10-08T05:36:16.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.428+0000 7f47977fe640 1 -- 172.21.15.70:0/4161834849 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f47a401e460 con 0x7f47b006db60 2023-10-08T05:36:16.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.430+0000 7f47977fe640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f479c06ae70 0x7f479c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:16.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.430+0000 7f47977fe640 1 -- 172.21.15.70:0/4161834849 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f47a408a050 con 0x7f47b006db60 2023-10-08T05:36:16.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.430+0000 7f47aeffd640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f479c06ae70 0x7f479c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:16.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.431+0000 7f47aeffd640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f479c06ae70 0x7f479c06d330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47a0036000 tx=0x7f47a003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:16.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.434+0000 7f47977fe640 1 -- 172.21.15.70:0/4161834849 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47a4054e10 con 0x7f47b006db60 2023-10-08T05:36:16.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:16.665+0000 7f47b527c640 1 -- 172.21.15.70:0/4161834849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f47b00c21f0 con 0x7f47b006db60 2023-10-08T05:36:17.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.214+0000 7f47977fe640 1 -- 172.21.15.70:0/4161834849 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f47a4058430 con 0x7f47b006db60 2023-10-08T05:36:17.215 INFO:teuthology.orchestra.run.smithi070.stderr:added data pool 4 to fsmap 2023-10-08T05:36:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 -- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f479c06ae70 msgr2=0x7f479c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f479c06ae70 0x7f479c06d330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47a0036000 tx=0x7f47a003b040 comp rx=0 tx=0).stop 2023-10-08T05:36:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 -- 172.21.15.70:0/4161834849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006db60 msgr2=0x7f47b0153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006db60 0x7f47b0153d30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47a400d830 tx=0x7f47a400dcf0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 -- 172.21.15.70:0/4161834849 shutdown_connections 2023-10-08T05:36:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f479c06ae70 0x7f479c06d330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47b015aa00 0x7f47b00b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47b006db60 0x7f47b0153d30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 --2- 172.21.15.70:0/4161834849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47b006c3a0 0x7f47b0154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 -- 172.21.15.70:0/4161834849 >> 172.21.15.70:0/4161834849 conn(0x7f47b006a5d0 msgr2=0x7f47b00ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.218+0000 7f47b527c640 1 -- 172.21.15.70:0/4161834849 shutdown_connections 2023-10-08T05:36:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.219+0000 7f47b527c640 1 -- 172.21.15.70:0/4161834849 wait complete. 2023-10-08T05:36:17.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:36:17.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.449+0000 7fc56af35640 1 Processor -- start 2023-10-08T05:36:17.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.450+0000 7fc56af35640 1 -- start start 2023-10-08T05:36:17.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.451+0000 7fc56af35640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5641599d0 0x7fc5640ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.451+0000 7fc56af35640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5640cacf0 con 0x7fc5640b07d0 2023-10-08T05:36:17.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.451+0000 7fc56af35640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5640ca3c0 con 0x7fc5640cc210 2023-10-08T05:36:17.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.451+0000 7fc56af35640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5640b10f0 con 0x7fc5641599d0 2023-10-08T05:36:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.451+0000 7fc569f33640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5641599d0 0x7fc5640ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.451+0000 7fc569f33640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5641599d0 0x7fc5640ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45320/0 (socket says 172.21.15.70:45320) 2023-10-08T05:36:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.451+0000 7fc569f33640 1 -- 172.21.15.70:0/1796292678 learned_addr learned my addr 172.21.15.70:0/1796292678 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.451+0000 7fc56a734640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc5640b07d0 0x7fc56406c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52518/0 (socket says 172.21.15.70:52518) 2023-10-08T05:36:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.452+0000 7fc569f33640 1 -- 172.21.15.70:0/1796292678 >> v1:172.21.15.181:6789/0 conn(0x7fc5640cc210 legacy=0x7fc5640b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.452+0000 7fc569f33640 1 -- 172.21.15.70:0/1796292678 >> v1:172.21.15.70:6789/0 conn(0x7fc5640b07d0 legacy=0x7fc56406c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.452+0000 7fc568f31640 1 -- 172.21.15.70:0/1796292678 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288494086 0 0) 0x7fc5640cacf0 con 0x7fc5640b07d0 2023-10-08T05:36:17.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.452+0000 7fc569f33640 1 -- 172.21.15.70:0/1796292678 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5640ca6a0 con 0x7fc5641599d0 2023-10-08T05:36:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.453+0000 7fc569f33640 1 --2- 172.21.15.70:0/1796292678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5641599d0 0x7fc5640ad730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc56000bb10 tx=0x7fc56002e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f56d3bafd734d72 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.453+0000 7fc568f31640 1 -- 172.21.15.70:0/1796292678 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc560036440 con 0x7fc5641599d0 2023-10-08T05:36:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.453+0000 7fc568f31640 1 -- 172.21.15.70:0/1796292678 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5600365a0 con 0x7fc5641599d0 2023-10-08T05:36:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.453+0000 7fc568f31640 1 -- 172.21.15.70:0/1796292678 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc560036930 con 0x7fc5641599d0 2023-10-08T05:36:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.454+0000 7fc56af35640 1 -- 172.21.15.70:0/1796292678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5641599d0 msgr2=0x7fc5640ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.454+0000 7fc56af35640 1 --2- 172.21.15.70:0/1796292678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5641599d0 0x7fc5640ad730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc56000bb10 tx=0x7fc56002e160 comp rx=0 tx=0).stop 2023-10-08T05:36:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.454+0000 7fc56af35640 1 -- 172.21.15.70:0/1796292678 shutdown_connections 2023-10-08T05:36:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.454+0000 7fc56af35640 1 --2- 172.21.15.70:0/1796292678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5641599d0 0x7fc5640ad730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.454+0000 7fc56af35640 1 -- 172.21.15.70:0/1796292678 >> 172.21.15.70:0/1796292678 conn(0x7fc5640cb0c0 msgr2=0x7fc5640cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.454+0000 7fc56af35640 1 -- 172.21.15.70:0/1796292678 shutdown_connections 2023-10-08T05:36:17.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.454+0000 7fc56af35640 1 -- 172.21.15.70:0/1796292678 wait complete. 2023-10-08T05:36:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.456+0000 7fc56af35640 1 Processor -- start 2023-10-08T05:36:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.456+0000 7fc56af35640 1 -- start start 2023-10-08T05:36:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.457+0000 7fc56af35640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5640b07d0 0x7fc564150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.457+0000 7fc56af35640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5640cc210 0x7fc56415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.457+0000 7fc56af35640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5641599d0 0x7fc5640af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.457+0000 7fc56af35640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5640c7090 con 0x7fc5640cc210 2023-10-08T05:36:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.457+0000 7fc56af35640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56415b1e0 con 0x7fc5640b07d0 2023-10-08T05:36:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.457+0000 7fc56af35640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56415b4c0 con 0x7fc5641599d0 2023-10-08T05:36:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.457+0000 7fc569732640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5640cc210 0x7fc56415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.457+0000 7fc569732640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5640cc210 0x7fc56415a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45334/0 (socket says 172.21.15.70:45334) 2023-10-08T05:36:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.457+0000 7fc569732640 1 -- 172.21.15.70:0/2540767669 learned_addr learned my addr 172.21.15.70:0/2540767669 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc569f33640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5640b07d0 0x7fc564150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc569732640 1 -- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5641599d0 msgr2=0x7fc5640af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc569732640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5641599d0 0x7fc5640af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc569732640 1 -- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5640b07d0 msgr2=0x7fc564150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc569732640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5640b07d0 0x7fc564150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc569732640 1 -- 172.21.15.70:0/2540767669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56000b770 con 0x7fc5640cc210 2023-10-08T05:36:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc569732640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5640cc210 0x7fc56415a8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc55400c430 tx=0x7fc55400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc55a7fc640 1 -- 172.21.15.70:0/2540767669 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554011790 con 0x7fc5640cc210 2023-10-08T05:36:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc55a7fc640 1 -- 172.21.15.70:0/2540767669 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5540118f0 con 0x7fc5640cc210 2023-10-08T05:36:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc55a7fc640 1 -- 172.21.15.70:0/2540767669 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554011ce0 con 0x7fc5640cc210 2023-10-08T05:36:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc56af35640 1 -- 172.21.15.70:0/2540767669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5640c6a40 con 0x7fc5640cc210 2023-10-08T05:36:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.458+0000 7fc56af35640 1 -- 172.21.15.70:0/2540767669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc564150e80 con 0x7fc5640cc210 2023-10-08T05:36:17.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.459+0000 7fc56af35640 1 -- 172.21.15.70:0/2540767669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5640c7200 con 0x7fc5640cc210 2023-10-08T05:36:17.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.460+0000 7fc55a7fc640 1 -- 172.21.15.70:0/2540767669 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc55401c460 con 0x7fc5640cc210 2023-10-08T05:36:17.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.461+0000 7fc55a7fc640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc54806ae70 0x7fc54806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.461+0000 7fc55a7fc640 1 -- 172.21.15.70:0/2540767669 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc554089980 con 0x7fc5640cc210 2023-10-08T05:36:17.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.462+0000 7fc569f33640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc54806ae70 0x7fc54806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:17.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.463+0000 7fc569f33640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc54806ae70 0x7fc54806d330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc560038000 tx=0x7fc560008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.465+0000 7fc55a7fc640 1 -- 172.21.15.70:0/2540767669 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc554054610 con 0x7fc5640cc210 2023-10-08T05:36:17.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.692+0000 7fc56af35640 1 -- 172.21.15.70:0/2540767669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc56404e160 con 0x7fc5640cc210 2023-10-08T05:36:17.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.693+0000 7fc55a7fc640 1 -- 172.21.15.70:0/2540767669 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fc554057c30 con 0x7fc5640cc210 2023-10-08T05:36:17.693 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:17.693 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:36:11.613175+0000","modified":"2023-10-08T05:36:16.667280+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-08T05:36:17.694 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 7 2023-10-08T05:36:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 -- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc54806ae70 msgr2=0x7fc54806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc54806ae70 0x7fc54806d330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc560038000 tx=0x7fc560008040 comp rx=0 tx=0).stop 2023-10-08T05:36:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 -- 172.21.15.70:0/2540767669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5640cc210 msgr2=0x7fc56415a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5640cc210 0x7fc56415a8e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc55400c430 tx=0x7fc55400c8f0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 -- 172.21.15.70:0/2540767669 shutdown_connections 2023-10-08T05:36:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc54806ae70 0x7fc54806d330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5641599d0 0x7fc5640af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5640cc210 0x7fc56415a8e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 --2- 172.21.15.70:0/2540767669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5640b07d0 0x7fc564150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 -- 172.21.15.70:0/2540767669 >> 172.21.15.70:0/2540767669 conn(0x7fc5640cb0c0 msgr2=0x7fc56406cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:17.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 -- 172.21.15.70:0/2540767669 shutdown_connections 2023-10-08T05:36:17.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.695+0000 7fc56af35640 1 -- 172.21.15.70:0/2540767669 wait complete. 2023-10-08T05:36:17.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:17.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.924+0000 7f00c9290640 1 Processor -- start 2023-10-08T05:36:17.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.924+0000 7f00c9290640 1 -- start start 2023-10-08T05:36:17.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.926+0000 7f00c9290640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f00c40c90b0 0x7f00c40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.926+0000 7f00c9290640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00c40cb3b0 0x7f00c415e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.926+0000 7f00c9290640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c415b2b0 con 0x7f00c415ab40 2023-10-08T05:36:17.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.926+0000 7f00c9290640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c415bf30 con 0x7f00c40c90b0 2023-10-08T05:36:17.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.926+0000 7f00c9290640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c40cb820 con 0x7f00c40cb3b0 2023-10-08T05:36:17.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.926+0000 7f00ba7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f00c40c90b0 0x7f00c40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:17.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.926+0000 7f00ba7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f00c40c90b0 0x7f00c40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45346/0 (socket says 172.21.15.70:45346) 2023-10-08T05:36:17.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.926+0000 7f00c2ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f00c415ab40 0x7f00c40ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52524/0 (socket says 172.21.15.70:52524) 2023-10-08T05:36:17.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.926+0000 7f00ba7fc640 1 -- 172.21.15.70:0/2265795306 learned_addr learned my addr 172.21.15.70:0/2265795306 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:17.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.927+0000 7f00ba7fc640 1 -- 172.21.15.70:0/2265795306 >> v2:172.21.15.181:3300/0 conn(0x7f00c40cb3b0 msgr2=0x7f00c415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:17.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.927+0000 7f00c37fe640 1 --2- 172.21.15.70:0/2265795306 >> v2:172.21.15.181:3300/0 conn(0x7f00c40cb3b0 0x7f00c415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:17.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.927+0000 7f00ba7fc640 1 --2- 172.21.15.70:0/2265795306 >> v2:172.21.15.181:3300/0 conn(0x7f00c40cb3b0 0x7f00c415e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.927+0000 7f00ba7fc640 1 -- 172.21.15.70:0/2265795306 >> v1:172.21.15.70:6789/0 conn(0x7f00c415ab40 legacy=0x7f00c40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.927+0000 7f00ba7fc640 1 -- 172.21.15.70:0/2265795306 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c415b590 con 0x7f00c40c90b0 2023-10-08T05:36:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.927+0000 7f00c27fc640 1 -- 172.21.15.70:0/2265795306 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176956645 0 0) 0x7f00c415b2b0 con 0x7f00c415ab40 2023-10-08T05:36:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.928+0000 7f00ba7fc640 1 --2- 172.21.15.70:0/2265795306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c40c90b0 0x7f00c40cae70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00c40b0ec0 tx=0x7f00ac02b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbb2f1e4d38986a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.928+0000 7f00c27fc640 1 -- 172.21.15.70:0/2265795306 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac030880 con 0x7f00c40c90b0 2023-10-08T05:36:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.928+0000 7f00c27fc640 1 -- 172.21.15.70:0/2265795306 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac0309e0 con 0x7f00c40c90b0 2023-10-08T05:36:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.928+0000 7f00c27fc640 1 -- 172.21.15.70:0/2265795306 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac030d70 con 0x7f00c40c90b0 2023-10-08T05:36:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.929+0000 7f00c9290640 1 -- 172.21.15.70:0/2265795306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c40c90b0 msgr2=0x7f00c40cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.929+0000 7f00c9290640 1 --2- 172.21.15.70:0/2265795306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c40c90b0 0x7f00c40cae70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00c40b0ec0 tx=0x7f00ac02b9c0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.929+0000 7f00c9290640 1 -- 172.21.15.70:0/2265795306 shutdown_connections 2023-10-08T05:36:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.929+0000 7f00c9290640 1 --2- 172.21.15.70:0/2265795306 >> v2:172.21.15.181:3300/0 conn(0x7f00c40cb3b0 0x7f00c415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.929+0000 7f00c9290640 1 --2- 172.21.15.70:0/2265795306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c40c90b0 0x7f00c40cae70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.929+0000 7f00c9290640 1 -- 172.21.15.70:0/2265795306 >> 172.21.15.70:0/2265795306 conn(0x7f00c406a380 msgr2=0x7f00c40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.929+0000 7f00c9290640 1 -- 172.21.15.70:0/2265795306 shutdown_connections 2023-10-08T05:36:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.929+0000 7f00c9290640 1 -- 172.21.15.70:0/2265795306 wait complete. 2023-10-08T05:36:17.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.930+0000 7f00c9290640 1 Processor -- start 2023-10-08T05:36:17.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.931+0000 7f00c9290640 1 -- start start 2023-10-08T05:36:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c9290640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c40c90b0 0x7f00c414d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c9290640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00c40cb3b0 0x7f00c40ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c9290640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c415ab40 0x7f00c41552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c9290640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40c5d10 con 0x7f00c415ab40 2023-10-08T05:36:17.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c9290640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40ad7c0 con 0x7f00c40c90b0 2023-10-08T05:36:17.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c9290640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40acc40 con 0x7f00c40cb3b0 2023-10-08T05:36:17.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c415ab40 0x7f00c41552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:17.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c415ab40 0x7f00c41552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45360/0 (socket says 172.21.15.70:45360) 2023-10-08T05:36:17.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c37fe640 1 -- 172.21.15.70:0/4190401899 learned_addr learned my addr 172.21.15.70:0/4190401899 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:17.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c37fe640 1 -- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00c40cb3b0 msgr2=0x7f00c40ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c37fe640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00c40cb3b0 0x7f00c40ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c2ffd640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c40c90b0 0x7f00c414d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:17.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c37fe640 1 -- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c40c90b0 msgr2=0x7f00c414d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:17.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c37fe640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c40c90b0 0x7f00c414d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:17.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c37fe640 1 -- 172.21.15.70:0/4190401899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0094d0 con 0x7f00c415ab40 2023-10-08T05:36:17.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.932+0000 7f00c2ffd640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c40c90b0 0x7f00c414d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:17.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.933+0000 7f00c37fe640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c415ab40 0x7f00c41552e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00cd70 tx=0x7f00bc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:17.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.933+0000 7f00bbfff640 1 -- 172.21.15.70:0/4190401899 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc003b90 con 0x7f00c415ab40 2023-10-08T05:36:17.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.933+0000 7f00bbfff640 1 -- 172.21.15.70:0/4190401899 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc003cf0 con 0x7f00c415ab40 2023-10-08T05:36:17.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.933+0000 7f00c9290640 1 -- 172.21.15.70:0/4190401899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c40acf20 con 0x7f00c415ab40 2023-10-08T05:36:17.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.933+0000 7f00bbfff640 1 -- 172.21.15.70:0/4190401899 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc004240 con 0x7f00c415ab40 2023-10-08T05:36:17.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.933+0000 7f00c9290640 1 -- 172.21.15.70:0/4190401899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c40ad4e0 con 0x7f00c415ab40 2023-10-08T05:36:17.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.934+0000 7f00c9290640 1 -- 172.21.15.70:0/4190401899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c40c5e80 con 0x7f00c415ab40 2023-10-08T05:36:17.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.935+0000 7f00bbfff640 1 -- 172.21.15.70:0/4190401899 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f00bc0043a0 con 0x7f00c415ab40 2023-10-08T05:36:17.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.937+0000 7f00bbfff640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f00a406ae70 0x7f00a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:17.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.937+0000 7f00bbfff640 1 -- 172.21.15.70:0/4190401899 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f00bc012070 con 0x7f00c415ab40 2023-10-08T05:36:17.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.937+0000 7f00c2ffd640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f00a406ae70 0x7f00a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:17.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.938+0000 7f00c2ffd640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f00a406ae70 0x7f00a406d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f00b4000f60 tx=0x7f00b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:17.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:17.941+0000 7f00bbfff640 1 -- 172.21.15.70:0/4190401899 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f00bc0550b0 con 0x7f00c415ab40 2023-10-08T05:36:18.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.130+0000 7f00c9290640 1 -- 172.21.15.70:0/4190401899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00c4067490 con 0x7f00c415ab40 2023-10-08T05:36:18.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.133+0000 7f00bbfff640 1 -- 172.21.15.70:0/4190401899 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f00bc0586d0 con 0x7f00c415ab40 2023-10-08T05:36:18.134 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:18.134 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:18.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 -- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f00a406ae70 msgr2=0x7f00a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:18.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f00a406ae70 0x7f00a406d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f00b4000f60 tx=0x7f00b4009040 comp rx=0 tx=0).stop 2023-10-08T05:36:18.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 -- 172.21.15.70:0/4190401899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c415ab40 msgr2=0x7f00c41552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:18.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c415ab40 0x7f00c41552e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00cd70 tx=0x7f00bc00a660 comp rx=0 tx=0).stop 2023-10-08T05:36:18.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 -- 172.21.15.70:0/4190401899 shutdown_connections 2023-10-08T05:36:18.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f00a406ae70 0x7f00a406d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00c415ab40 0x7f00c41552e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00c40cb3b0 0x7f00c40ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 --2- 172.21.15.70:0/4190401899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00c40c90b0 0x7f00c414d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 -- 172.21.15.70:0/4190401899 >> 172.21.15.70:0/4190401899 conn(0x7f00c406a380 msgr2=0x7f00c40cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:18.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 -- 172.21.15.70:0/4190401899 shutdown_connections 2023-10-08T05:36:18.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.136+0000 7f00c9290640 1 -- 172.21.15.70:0/4190401899 wait complete. 2023-10-08T05:36:18.153 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:18.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.361+0000 7fca97339640 1 Processor -- start 2023-10-08T05:36:18.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.361+0000 7fca97339640 1 -- start start 2023-10-08T05:36:18.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca97339640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fca9015ab40 0x7fca900ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca97339640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 0x7fca900adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca97339640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fca900cae70 0x7fca900cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca97339640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9015bf30 con 0x7fca900cae70 2023-10-08T05:36:18.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca97339640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca900cb840 con 0x7fca9015ab40 2023-10-08T05:36:18.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca97339640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9015b2b0 con 0x7fca900c90b0 2023-10-08T05:36:18.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca958d6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fca900cae70 0x7fca900cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca958d6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fca900cae70 0x7fca900cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45376/0 (socket says 172.21.15.70:45376) 2023-10-08T05:36:18.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca958d6640 1 -- 172.21.15.70:0/2230445242 learned_addr learned my addr 172.21.15.70:0/2230445242 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:18.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca950d5640 1 --2- 172.21.15.70:0/2230445242 >> v2:172.21.15.181:3300/0 conn(0x7fca9015ab40 0x7fca900ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca948d4640 1 --2- 172.21.15.70:0/2230445242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 0x7fca900adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca958d6640 1 -- 172.21.15.70:0/2230445242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 msgr2=0x7fca900adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:18.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca958d6640 1 --2- 172.21.15.70:0/2230445242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 0x7fca900adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca958d6640 1 -- 172.21.15.70:0/2230445242 >> v2:172.21.15.181:3300/0 conn(0x7fca9015ab40 msgr2=0x7fca900ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:18.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca958d6640 1 --2- 172.21.15.70:0/2230445242 >> v2:172.21.15.181:3300/0 conn(0x7fca9015ab40 0x7fca900ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca958d6640 1 -- 172.21.15.70:0/2230445242 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9015b590 con 0x7fca900cae70 2023-10-08T05:36:18.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.363+0000 7fca948d4640 1 --2- 172.21.15.70:0/2230445242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 0x7fca900adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:18.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca958d6640 1 --2- 172.21.15.70:0/2230445242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 0x7fca900cb300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fca9006e440 tx=0x7fca8802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1e443e47add037 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:18.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca87fff640 1 -- 172.21.15.70:0/2230445242 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8802ca80 con 0x7fca900cae70 2023-10-08T05:36:18.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca87fff640 1 -- 172.21.15.70:0/2230445242 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8802cbe0 con 0x7fca900cae70 2023-10-08T05:36:18.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca87fff640 1 -- 172.21.15.70:0/2230445242 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca880343f0 con 0x7fca900cae70 2023-10-08T05:36:18.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca97339640 1 -- 172.21.15.70:0/2230445242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 msgr2=0x7fca900cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:18.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca97339640 1 --2- 172.21.15.70:0/2230445242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 0x7fca900cb300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fca9006e440 tx=0x7fca8802bee0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca97339640 1 -- 172.21.15.70:0/2230445242 shutdown_connections 2023-10-08T05:36:18.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca97339640 1 --2- 172.21.15.70:0/2230445242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 0x7fca900cb300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca97339640 1 --2- 172.21.15.70:0/2230445242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 0x7fca900adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca97339640 1 --2- 172.21.15.70:0/2230445242 >> v2:172.21.15.181:3300/0 conn(0x7fca9015ab40 0x7fca900ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.365+0000 7fca97339640 1 -- 172.21.15.70:0/2230445242 >> 172.21.15.70:0/2230445242 conn(0x7fca9006a380 msgr2=0x7fca900b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:18.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.366+0000 7fca97339640 1 -- 172.21.15.70:0/2230445242 shutdown_connections 2023-10-08T05:36:18.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.366+0000 7fca97339640 1 -- 172.21.15.70:0/2230445242 wait complete. 2023-10-08T05:36:18.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.367+0000 7fca97339640 1 Processor -- start 2023-10-08T05:36:18.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.367+0000 7fca97339640 1 -- start start 2023-10-08T05:36:18.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.368+0000 7fca97339640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 0x7fca901f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.368+0000 7fca97339640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 0x7fca901f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.368+0000 7fca97339640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca9015ab40 0x7fca90159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.368+0000 7fca97339640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca901f6200 con 0x7fca900cae70 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.368+0000 7fca97339640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90146f70 con 0x7fca9015ab40 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.368+0000 7fca97339640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca90143b00 con 0x7fca900c90b0 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.369+0000 7fca948d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 0x7fca901f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.369+0000 7fca948d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 0x7fca901f1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45386/0 (socket says 172.21.15.70:45386) 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.369+0000 7fca948d4640 1 -- 172.21.15.70:0/828303825 learned_addr learned my addr 172.21.15.70:0/828303825 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.369+0000 7fca950d5640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 0x7fca901f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.369+0000 7fca948d4640 1 -- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 msgr2=0x7fca901f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.369+0000 7fca948d4640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 0x7fca901f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.369+0000 7fca948d4640 1 -- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca9015ab40 msgr2=0x7fca90159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.369+0000 7fca948d4640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca9015ab40 0x7fca90159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.369+0000 7fca948d4640 1 -- 172.21.15.70:0/828303825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca88009b30 con 0x7fca900cae70 2023-10-08T05:36:18.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.369+0000 7fca948d4640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 0x7fca901f1d70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fca8000c820 tx=0x7fca8000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:18.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.370+0000 7fca85ffb640 1 -- 172.21.15.70:0/828303825 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80007a30 con 0x7fca900cae70 2023-10-08T05:36:18.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.370+0000 7fca85ffb640 1 -- 172.21.15.70:0/828303825 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca80007b90 con 0x7fca900cae70 2023-10-08T05:36:18.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.370+0000 7fca85ffb640 1 -- 172.21.15.70:0/828303825 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8000ce10 con 0x7fca900cae70 2023-10-08T05:36:18.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.370+0000 7fca97339640 1 -- 172.21.15.70:0/828303825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca90143820 con 0x7fca900cae70 2023-10-08T05:36:18.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.370+0000 7fca97339640 1 -- 172.21.15.70:0/828303825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca901f5e20 con 0x7fca900cae70 2023-10-08T05:36:18.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.371+0000 7fca97339640 1 -- 172.21.15.70:0/828303825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca901f6340 con 0x7fca900cae70 2023-10-08T05:36:18.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.372+0000 7fca85ffb640 1 -- 172.21.15.70:0/828303825 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fca800113f0 con 0x7fca900cae70 2023-10-08T05:36:18.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.373+0000 7fca85ffb640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fca7406ae70 0x7fca7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.373+0000 7fca85ffb640 1 -- 172.21.15.70:0/828303825 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fca800899d0 con 0x7fca900cae70 2023-10-08T05:36:18.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.373+0000 7fca950d5640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fca7406ae70 0x7fca7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.374+0000 7fca950d5640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fca7406ae70 0x7fca7406d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca8c006fd0 tx=0x7fca8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:18.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.377+0000 7fca85ffb640 1 -- 172.21.15.70:0/828303825 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca901f6340 con 0x7fca900cae70 2023-10-08T05:36:18.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.554+0000 7fca97339640 1 -- 172.21.15.70:0/828303825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca901f6340 con 0x7fca900cae70 2023-10-08T05:36:18.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.558+0000 7fca85ffb640 1 -- 172.21.15.70:0/828303825 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fca901f6340 con 0x7fca900cae70 2023-10-08T05:36:18.559 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:18.559 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:18.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.560+0000 7fca97339640 1 -- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fca7406ae70 msgr2=0x7fca7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:18.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.560+0000 7fca97339640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fca7406ae70 0x7fca7406d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca8c006fd0 tx=0x7fca8c008040 comp rx=0 tx=0).stop 2023-10-08T05:36:18.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.560+0000 7fca97339640 1 -- 172.21.15.70:0/828303825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 msgr2=0x7fca901f1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:18.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.560+0000 7fca97339640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 0x7fca901f1d70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fca8000c820 tx=0x7fca8000cce0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.560+0000 7fca97339640 1 -- 172.21.15.70:0/828303825 shutdown_connections 2023-10-08T05:36:18.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.560+0000 7fca97339640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fca7406ae70 0x7fca7406d330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.560+0000 7fca97339640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca9015ab40 0x7fca90159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.560+0000 7fca97339640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca900cae70 0x7fca901f1d70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.560+0000 7fca97339640 1 --2- 172.21.15.70:0/828303825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca900c90b0 0x7fca901f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.560+0000 7fca97339640 1 -- 172.21.15.70:0/828303825 >> 172.21.15.70:0/828303825 conn(0x7fca9006a380 msgr2=0x7fca900ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:18.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.561+0000 7fca97339640 1 -- 172.21.15.70:0/828303825 shutdown_connections 2023-10-08T05:36:18.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.561+0000 7fca97339640 1 -- 172.21.15.70:0/828303825 wait complete. 2023-10-08T05:36:18.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:36:18.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.788+0000 7ffab207c640 1 Processor -- start 2023-10-08T05:36:18.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.788+0000 7ffab207c640 1 -- start start 2023-10-08T05:36:18.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.790+0000 7ffab207c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffaac15ab40 0x7ffaac0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.790+0000 7ffab207c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffaac0c90b0 0x7ffaac0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.790+0000 7ffab207c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac15b2b0 con 0x7ffaac0cae70 2023-10-08T05:36:18.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.790+0000 7ffab207c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac15bf30 con 0x7ffaac0c90b0 2023-10-08T05:36:18.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.790+0000 7ffab207c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac15f7a0 con 0x7ffaac15ab40 2023-10-08T05:36:18.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.790+0000 7ffaa3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffaac0c90b0 0x7ffaac0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.790+0000 7ffaa3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffaac0c90b0 0x7ffaac0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45394/0 (socket says 172.21.15.70:45394) 2023-10-08T05:36:18.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.790+0000 7ffaa3fff640 1 -- 172.21.15.70:0/2906363541 learned_addr learned my addr 172.21.15.70:0/2906363541 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:18.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.791+0000 7ffaab7fe640 1 --2- 172.21.15.70:0/2906363541 >> v2:172.21.15.181:3300/0 conn(0x7ffaac15ab40 0x7ffaac0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.791+0000 7ffaa3fff640 1 -- 172.21.15.70:0/2906363541 >> v2:172.21.15.181:3300/0 conn(0x7ffaac15ab40 msgr2=0x7ffaac0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:18.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.791+0000 7ffaa3fff640 1 --2- 172.21.15.70:0/2906363541 >> v2:172.21.15.181:3300/0 conn(0x7ffaac15ab40 0x7ffaac0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.791+0000 7ffaa3fff640 1 -- 172.21.15.70:0/2906363541 >> v1:172.21.15.181:6789/0 conn(0x7ffaac0cae70 legacy=0x7ffaac0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:18.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.791+0000 7ffaa3fff640 1 -- 172.21.15.70:0/2906363541 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac15b590 con 0x7ffaac0c90b0 2023-10-08T05:36:18.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.792+0000 7ffaa3fff640 1 --2- 172.21.15.70:0/2906363541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0c90b0 0x7ffaac0adc60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffaac06e440 tx=0x7ffa9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460a8eea26305c15 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:18.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.792+0000 7ffaaaffd640 1 -- 172.21.15.70:0/2906363541 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94031870 con 0x7ffaac0c90b0 2023-10-08T05:36:18.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.792+0000 7ffaaaffd640 1 -- 172.21.15.70:0/2906363541 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa940319d0 con 0x7ffaac0c90b0 2023-10-08T05:36:18.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.792+0000 7ffaaaffd640 1 -- 172.21.15.70:0/2906363541 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94031d60 con 0x7ffaac0c90b0 2023-10-08T05:36:18.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.793+0000 7ffab207c640 1 -- 172.21.15.70:0/2906363541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0c90b0 msgr2=0x7ffaac0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:18.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.793+0000 7ffab207c640 1 --2- 172.21.15.70:0/2906363541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0c90b0 0x7ffaac0adc60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffaac06e440 tx=0x7ffa9402b910 comp rx=0 tx=0).stop 2023-10-08T05:36:18.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.793+0000 7ffab207c640 1 -- 172.21.15.70:0/2906363541 shutdown_connections 2023-10-08T05:36:18.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.793+0000 7ffab207c640 1 --2- 172.21.15.70:0/2906363541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0c90b0 0x7ffaac0adc60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.793+0000 7ffab207c640 1 --2- 172.21.15.70:0/2906363541 >> v2:172.21.15.181:3300/0 conn(0x7ffaac15ab40 0x7ffaac0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.793+0000 7ffab207c640 1 -- 172.21.15.70:0/2906363541 >> 172.21.15.70:0/2906363541 conn(0x7ffaac06a380 msgr2=0x7ffaac0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:18.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.793+0000 7ffab207c640 1 -- 172.21.15.70:0/2906363541 shutdown_connections 2023-10-08T05:36:18.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.793+0000 7ffab207c640 1 -- 172.21.15.70:0/2906363541 wait complete. 2023-10-08T05:36:18.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.795+0000 7ffab207c640 1 Processor -- start 2023-10-08T05:36:18.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.795+0000 7ffab207c640 1 -- start start 2023-10-08T05:36:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.796+0000 7ffab207c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaac0c90b0 0x7ffaac0c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.796+0000 7ffab207c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0cae70 0x7ffaac159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.796+0000 7ffab207c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaac15ab40 0x7ffaac0c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.796+0000 7ffab207c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac0c6af0 con 0x7ffaac0cae70 2023-10-08T05:36:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.796+0000 7ffab207c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac15a030 con 0x7ffaac15ab40 2023-10-08T05:36:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.796+0000 7ffab207c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac0c74c0 con 0x7ffaac0c90b0 2023-10-08T05:36:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.796+0000 7ffaa3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0cae70 0x7ffaac159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.796+0000 7ffaa3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0cae70 0x7ffaac159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45410/0 (socket says 172.21.15.70:45410) 2023-10-08T05:36:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.796+0000 7ffaa3fff640 1 -- 172.21.15.70:0/2651558522 learned_addr learned my addr 172.21.15.70:0/2651558522 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaab7fe640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaac0c90b0 0x7ffaac0c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaa3fff640 1 -- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaac0c90b0 msgr2=0x7ffaac0c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaa3fff640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaac0c90b0 0x7ffaac0c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaa3fff640 1 -- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaac15ab40 msgr2=0x7ffaac0c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaabfff640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaac15ab40 0x7ffaac0c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaa3fff640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaac15ab40 0x7ffaac0c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaa3fff640 1 -- 172.21.15.70:0/2651558522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa940094d0 con 0x7ffaac0cae70 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaab7fe640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaac0c90b0 0x7ffaac0c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaabfff640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaac15ab40 0x7ffaac0c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaa3fff640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0cae70 0x7ffaac159af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa94000c00 tx=0x7ffa940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaa8ff9640 1 -- 172.21.15.70:0/2651558522 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9403abe0 con 0x7ffaac0cae70 2023-10-08T05:36:18.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaa8ff9640 1 -- 172.21.15.70:0/2651558522 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9403ad40 con 0x7ffaac0cae70 2023-10-08T05:36:18.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.797+0000 7ffaa8ff9640 1 -- 172.21.15.70:0/2651558522 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94044610 con 0x7ffaac0cae70 2023-10-08T05:36:18.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.798+0000 7ffab207c640 1 -- 172.21.15.70:0/2651558522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac0c5ee0 con 0x7ffaac0cae70 2023-10-08T05:36:18.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.798+0000 7ffab207c640 1 -- 172.21.15.70:0/2651558522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac0c64a0 con 0x7ffaac0cae70 2023-10-08T05:36:18.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.799+0000 7ffab207c640 1 -- 172.21.15.70:0/2651558522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac0afa00 con 0x7ffaac0cae70 2023-10-08T05:36:18.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.800+0000 7ffaa8ff9640 1 -- 172.21.15.70:0/2651558522 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffa9404d450 con 0x7ffaac0cae70 2023-10-08T05:36:18.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.800+0000 7ffaa8ff9640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffa8c06ae70 0x7ffa8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:18.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.800+0000 7ffaa8ff9640 1 -- 172.21.15.70:0/2651558522 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ffa9402d040 con 0x7ffaac0cae70 2023-10-08T05:36:18.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.801+0000 7ffaab7fe640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffa8c06ae70 0x7ffa8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:18.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.802+0000 7ffaab7fe640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffa8c06ae70 0x7ffa8c06d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c006fd0 tx=0x7ffa9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:18.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:18.805+0000 7ffaa8ff9640 1 -- 172.21.15.70:0/2651558522 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa94076c40 con 0x7ffaac0cae70 2023-10-08T05:36:19.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.030+0000 7ffab207c640 1 -- 172.21.15.70:0/2651558522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaac067490 con 0x7ffaac0cae70 2023-10-08T05:36:19.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.032+0000 7ffaa8ff9640 1 -- 172.21.15.70:0/2651558522 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4912 (secure 0 0 0) 0x7ffa94035020 con 0x7ffaac0cae70 2023-10-08T05:36:19.032 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:19.033 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:36:11.613175+0000","modified":"2023-10-08T05:36:16.667280+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-08T05:36:19.033 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 7 2023-10-08T05:36:19.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.034+0000 7ffab207c640 1 -- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffa8c06ae70 msgr2=0x7ffa8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:19.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.034+0000 7ffab207c640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffa8c06ae70 0x7ffa8c06d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c006fd0 tx=0x7ffa9c008040 comp rx=0 tx=0).stop 2023-10-08T05:36:19.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.034+0000 7ffab207c640 1 -- 172.21.15.70:0/2651558522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0cae70 msgr2=0x7ffaac159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:19.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.034+0000 7ffab207c640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0cae70 0x7ffaac159af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa94000c00 tx=0x7ffa940046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.034+0000 7ffab207c640 1 -- 172.21.15.70:0/2651558522 shutdown_connections 2023-10-08T05:36:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.034+0000 7ffab207c640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffa8c06ae70 0x7ffa8c06d330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.034+0000 7ffab207c640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaac15ab40 0x7ffaac0c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.034+0000 7ffab207c640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffaac0cae70 0x7ffaac159af0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.034+0000 7ffab207c640 1 --2- 172.21.15.70:0/2651558522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaac0c90b0 0x7ffaac0c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.034+0000 7ffab207c640 1 -- 172.21.15.70:0/2651558522 >> 172.21.15.70:0/2651558522 conn(0x7ffaac06a380 msgr2=0x7ffaac06d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.035+0000 7ffab207c640 1 -- 172.21.15.70:0/2651558522 shutdown_connections 2023-10-08T05:36:19.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.035+0000 7ffab207c640 1 -- 172.21.15.70:0/2651558522 wait complete. 2023-10-08T05:36:19.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:19.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.261+0000 7f3440cbb640 1 Processor -- start 2023-10-08T05:36:19.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.262+0000 7f3440cbb640 1 -- start start 2023-10-08T05:36:19.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.263+0000 7f3440cbb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c15cd90 0x7f343c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:19.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.263+0000 7f3440cbb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f343c160940 0x7f343c070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:19.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.263+0000 7f3440cbb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343c0c48f0 con 0x7f343c070d50 2023-10-08T05:36:19.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.263+0000 7f3440cbb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c0c4d80 con 0x7f343c160940 2023-10-08T05:36:19.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.263+0000 7f3440cbb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0c3ce0 con 0x7f343c15cd90 2023-10-08T05:36:19.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.264+0000 7f3433fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f343c160940 0x7f343c070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:19.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.264+0000 7f3433fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f343c160940 0x7f343c070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45418/0 (socket says 172.21.15.70:45418) 2023-10-08T05:36:19.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.264+0000 7f3433fff640 1 -- 172.21.15.70:0/2619535758 learned_addr learned my addr 172.21.15.70:0/2619535758 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:19.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.264+0000 7f343a59c640 1 --2- 172.21.15.70:0/2619535758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c15cd90 0x7f343c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:19.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.264+0000 7f3433fff640 1 -- 172.21.15.70:0/2619535758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c15cd90 msgr2=0x7f343c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:19.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.264+0000 7f3433fff640 1 --2- 172.21.15.70:0/2619535758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c15cd90 0x7f343c0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.264+0000 7f3433fff640 1 -- 172.21.15.70:0/2619535758 >> v1:172.21.15.70:6789/0 conn(0x7f343c070d50 legacy=0x7f343c0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:19.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.264+0000 7f3433fff640 1 -- 172.21.15.70:0/2619535758 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0c3fc0 con 0x7f343c160940 2023-10-08T05:36:19.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.264+0000 7f3439d9b640 1 -- 172.21.15.70:0/2619535758 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658646565 0 0) 0x7f343c0c48f0 con 0x7f343c070d50 2023-10-08T05:36:19.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.265+0000 7f3433fff640 1 --2- 172.21.15.70:0/2619535758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c160940 0x7f343c070810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3424009870 tx=0x7f342402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27064c5000dfaa74 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:19.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3439d9b640 1 -- 172.21.15.70:0/2619535758 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424030840 con 0x7f343c160940 2023-10-08T05:36:19.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3439d9b640 1 -- 172.21.15.70:0/2619535758 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34240309a0 con 0x7f343c160940 2023-10-08T05:36:19.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3439d9b640 1 -- 172.21.15.70:0/2619535758 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424030d30 con 0x7f343c160940 2023-10-08T05:36:19.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3440cbb640 1 -- 172.21.15.70:0/2619535758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c160940 msgr2=0x7f343c070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:19.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3440cbb640 1 --2- 172.21.15.70:0/2619535758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c160940 0x7f343c070810 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3424009870 tx=0x7f342402b980 comp rx=0 tx=0).stop 2023-10-08T05:36:19.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3440cbb640 1 -- 172.21.15.70:0/2619535758 shutdown_connections 2023-10-08T05:36:19.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3440cbb640 1 --2- 172.21.15.70:0/2619535758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c160940 0x7f343c070810 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3440cbb640 1 --2- 172.21.15.70:0/2619535758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c15cd90 0x7f343c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3440cbb640 1 -- 172.21.15.70:0/2619535758 >> 172.21.15.70:0/2619535758 conn(0x7f343c06a380 msgr2=0x7f343c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:19.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3440cbb640 1 -- 172.21.15.70:0/2619535758 shutdown_connections 2023-10-08T05:36:19.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.266+0000 7f3440cbb640 1 -- 172.21.15.70:0/2619535758 wait complete. 2023-10-08T05:36:19.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.268+0000 7f3440cbb640 1 Processor -- start 2023-10-08T05:36:19.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.268+0000 7f3440cbb640 1 -- start start 2023-10-08T05:36:19.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.269+0000 7f3440cbb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c070d50 0x7f343c0b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:19.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.269+0000 7f3440cbb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f343c15cd90 0x7f343c15ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:19.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.269+0000 7f3440cbb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c160940 0x7f343c0b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:19.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.269+0000 7f3440cbb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0cbed0 con 0x7f343c070d50 2023-10-08T05:36:19.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.269+0000 7f3440cbb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0b3eb0 con 0x7f343c15cd90 2023-10-08T05:36:19.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.269+0000 7f3440cbb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0b4190 con 0x7f343c160940 2023-10-08T05:36:19.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.269+0000 7f343a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c070d50 0x7f343c0b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:19.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.269+0000 7f343a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c070d50 0x7f343c0b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45430/0 (socket says 172.21.15.70:45430) 2023-10-08T05:36:19.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.269+0000 7f343a59c640 1 -- 172.21.15.70:0/1866050275 learned_addr learned my addr 172.21.15.70:0/1866050275 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:19.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f343a59c640 1 -- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c160940 msgr2=0x7f343c0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:19.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f3433fff640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f343c15cd90 0x7f343c15ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:19.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f343ad9d640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c160940 0x7f343c0b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:19.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f343a59c640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c160940 0x7f343c0b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f343a59c640 1 -- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f343c15cd90 msgr2=0x7f343c15ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:19.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f343a59c640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f343c15cd90 0x7f343c15ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f343a59c640 1 -- 172.21.15.70:0/1866050275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34240094d0 con 0x7f343c070d50 2023-10-08T05:36:19.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f343ad9d640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c160940 0x7f343c0b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:19.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f3433fff640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f343c15cd90 0x7f343c15ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:19.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f343a59c640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c070d50 0x7f343c0b26c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d800 tx=0x7f342c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:19.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.270+0000 7f3432ffd640 1 -- 172.21.15.70:0/1866050275 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c00b6c0 con 0x7f343c070d50 2023-10-08T05:36:19.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.271+0000 7f3432ffd640 1 -- 172.21.15.70:0/1866050275 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342c00b820 con 0x7f343c070d50 2023-10-08T05:36:19.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.271+0000 7f3432ffd640 1 -- 172.21.15.70:0/1866050275 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c00bbb0 con 0x7f343c070d50 2023-10-08T05:36:19.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.271+0000 7f3440cbb640 1 -- 172.21.15.70:0/1866050275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c15e2d0 con 0x7f343c070d50 2023-10-08T05:36:19.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.271+0000 7f3440cbb640 1 -- 172.21.15.70:0/1866050275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c0b2c00 con 0x7f343c070d50 2023-10-08T05:36:19.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.272+0000 7f3440cbb640 1 -- 172.21.15.70:0/1866050275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c0b2dc0 con 0x7f343c070d50 2023-10-08T05:36:19.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.273+0000 7f3432ffd640 1 -- 172.21.15.70:0/1866050275 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f342c00bd10 con 0x7f343c070d50 2023-10-08T05:36:19.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.273+0000 7f3432ffd640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f341c06aea0 0x7f341c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:19.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.274+0000 7f3432ffd640 1 -- 172.21.15.70:0/1866050275 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f342c089e40 con 0x7f343c070d50 2023-10-08T05:36:19.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.274+0000 7f3433fff640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f341c06aea0 0x7f341c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:19.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.275+0000 7f3433fff640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f341c06aea0 0x7f341c06d360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3424036000 tx=0x7f342403b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:19.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.278+0000 7f3432ffd640 1 -- 172.21.15.70:0/1866050275 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f342c054bf0 con 0x7f343c070d50 2023-10-08T05:36:19.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.467+0000 7f3440cbb640 1 -- 172.21.15.70:0/1866050275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f343c067490 con 0x7f343c070d50 2023-10-08T05:36:19.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.470+0000 7f3432ffd640 1 -- 172.21.15.70:0/1866050275 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f342c058210 con 0x7f343c070d50 2023-10-08T05:36:19.472 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:19.472 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:19.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 -- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f341c06aea0 msgr2=0x7f341c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:19.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f341c06aea0 0x7f341c06d360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3424036000 tx=0x7f342403b040 comp rx=0 tx=0).stop 2023-10-08T05:36:19.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 -- 172.21.15.70:0/1866050275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c070d50 msgr2=0x7f343c0b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:19.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c070d50 0x7f343c0b26c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f342c00d800 tx=0x7f342c00dcc0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 -- 172.21.15.70:0/1866050275 shutdown_connections 2023-10-08T05:36:19.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f341c06aea0 0x7f341c06d360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f343c160940 0x7f343c0b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f343c15cd90 0x7f343c15ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 --2- 172.21.15.70:0/1866050275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f343c070d50 0x7f343c0b26c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:19.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 -- 172.21.15.70:0/1866050275 >> 172.21.15.70:0/1866050275 conn(0x7f343c06a380 msgr2=0x7f343c0ad6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:19.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.473+0000 7f3440cbb640 1 -- 172.21.15.70:0/1866050275 shutdown_connections 2023-10-08T05:36:19.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:19.474+0000 7f3440cbb640 1 -- 172.21.15.70:0/1866050275 wait complete. 2023-10-08T05:36:19.895 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T05:36:11.613175+0000', 'modified': '2023-10-08T05:36:16.667280+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4255}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4255': {'gid': 4255, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/1675009602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 1675009602}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1675009602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-08T05:36:19.895 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T05:36:19.895 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T05:36:20.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.109+0000 7f9460656640 1 Processor -- start 2023-10-08T05:36:20.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.110+0000 7f9460656640 1 -- start start 2023-10-08T05:36:20.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.111+0000 7f9460656640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f945815ce00 0x7f94580cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.111+0000 7f9460656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9458161090 0x7f94580cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.111+0000 7f9460656640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94580cde10 con 0x7f94580cd470 2023-10-08T05:36:20.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.111+0000 7f9460656640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9458161920 con 0x7f945815ce00 2023-10-08T05:36:20.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.111+0000 7f9460656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94580cb8b0 con 0x7f9458161090 2023-10-08T05:36:20.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.111+0000 7f945e3f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f945815ce00 0x7f94580cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.111+0000 7f945e3f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f945815ce00 0x7f94580cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45440/0 (socket says 172.21.15.70:45440) 2023-10-08T05:36:20.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.111+0000 7f945e3f2640 1 -- 172.21.15.70:0/1877807067 learned_addr learned my addr 172.21.15.70:0/1877807067 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:20.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.112+0000 7f945e3f2640 1 -- 172.21.15.70:0/1877807067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9458161090 msgr2=0x7f94580cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.112+0000 7f945ebf3640 1 --2- 172.21.15.70:0/1877807067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9458161090 0x7f94580cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.112+0000 7f945e3f2640 1 --2- 172.21.15.70:0/1877807067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9458161090 0x7f94580cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.112+0000 7f945e3f2640 1 -- 172.21.15.70:0/1877807067 >> v1:172.21.15.70:6789/0 conn(0x7f94580cd470 legacy=0x7f9458160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.112+0000 7f945e3f2640 1 -- 172.21.15.70:0/1877807067 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458161640 con 0x7f945815ce00 2023-10-08T05:36:20.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.112+0000 7f945ebf3640 1 --2- 172.21.15.70:0/1877807067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9458161090 0x7f94580cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:20.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.112+0000 7f945d3f0640 1 -- 172.21.15.70:0/1877807067 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445239866 0 0) 0x7f94580cde10 con 0x7f94580cd470 2023-10-08T05:36:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.113+0000 7f945e3f2640 1 --2- 172.21.15.70:0/1877807067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f945815ce00 0x7f94580cb370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9454009850 tx=0x7f945402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665bd6d8957e9552 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.113+0000 7f945d3f0640 1 -- 172.21.15.70:0/1877807067 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454004230 con 0x7f945815ce00 2023-10-08T05:36:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.113+0000 7f945d3f0640 1 -- 172.21.15.70:0/1877807067 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454004390 con 0x7f945815ce00 2023-10-08T05:36:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.113+0000 7f945d3f0640 1 -- 172.21.15.70:0/1877807067 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454031830 con 0x7f945815ce00 2023-10-08T05:36:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.114+0000 7f9460656640 1 -- 172.21.15.70:0/1877807067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f945815ce00 msgr2=0x7f94580cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.114+0000 7f9460656640 1 --2- 172.21.15.70:0/1877807067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f945815ce00 0x7f94580cb370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9454009850 tx=0x7f945402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.114+0000 7f9460656640 1 -- 172.21.15.70:0/1877807067 shutdown_connections 2023-10-08T05:36:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.114+0000 7f9460656640 1 --2- 172.21.15.70:0/1877807067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9458161090 0x7f94580cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.114+0000 7f9460656640 1 --2- 172.21.15.70:0/1877807067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f945815ce00 0x7f94580cb370 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.114+0000 7f9460656640 1 -- 172.21.15.70:0/1877807067 >> 172.21.15.70:0/1877807067 conn(0x7f945806e650 msgr2=0x7f945806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:20.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.114+0000 7f9460656640 1 -- 172.21.15.70:0/1877807067 shutdown_connections 2023-10-08T05:36:20.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.114+0000 7f9460656640 1 -- 172.21.15.70:0/1877807067 wait complete. 2023-10-08T05:36:20.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.116+0000 7f9460656640 1 Processor -- start 2023-10-08T05:36:20.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.116+0000 7f9460656640 1 -- start start 2023-10-08T05:36:20.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f9460656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94580cd470 0x7f9458150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f9460656640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f945815ce00 0x7f94580c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f9460656640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458161090 0x7f94580ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f9460656640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580b05f0 con 0x7f9458161090 2023-10-08T05:36:20.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f9460656640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94581514d0 con 0x7f945815ce00 2023-10-08T05:36:20.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f9460656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945815fd00 con 0x7f94580cd470 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f945ebf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458161090 0x7f94580ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f945ebf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458161090 0x7f94580ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45446/0 (socket says 172.21.15.70:45446) 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f945ebf3640 1 -- 172.21.15.70:0/3465672218 learned_addr learned my addr 172.21.15.70:0/3465672218 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f945e3f2640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94580cd470 0x7f9458150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f945dbf1640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f945815ce00 0x7f94580c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f945ebf3640 1 -- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94580cd470 msgr2=0x7f9458150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.117+0000 7f945ebf3640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94580cd470 0x7f9458150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.118+0000 7f945ebf3640 1 -- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f945815ce00 msgr2=0x7f94580c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.118+0000 7f945ebf3640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f945815ce00 0x7f94580c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.118+0000 7f945ebf3640 1 -- 172.21.15.70:0/3465672218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9454009460 con 0x7f9458161090 2023-10-08T05:36:20.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.118+0000 7f945dbf1640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f945815ce00 0x7f94580c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.118+0000 7f945ebf3640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458161090 0x7f94580ca0b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f945000cd50 tx=0x7f945000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.118+0000 7f944effd640 1 -- 172.21.15.70:0/3465672218 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000f680 con 0x7f9458161090 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.118+0000 7f944effd640 1 -- 172.21.15.70:0/3465672218 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945000f800 con 0x7f9458161090 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.118+0000 7f944effd640 1 -- 172.21.15.70:0/3465672218 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945000fbb0 con 0x7f9458161090 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.118+0000 7f9460656640 1 -- 172.21.15.70:0/3465672218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94580b5eb0 con 0x7f9458161090 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.118+0000 7f9460656640 1 -- 172.21.15.70:0/3465672218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94580b6470 con 0x7f9458161090 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.120+0000 7f944effd640 1 -- 172.21.15.70:0/3465672218 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f945000b040 con 0x7f9458161090 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.121+0000 7f944effd640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f943c06ae70 0x7f943c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.122+0000 7f944effd640 1 -- 172.21.15.70:0/3465672218 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f945008a210 con 0x7f9458161090 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.122+0000 7f945e3f2640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f943c06ae70 0x7f943c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.122+0000 7f94631ed740 1 Processor -- start 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.123+0000 7f94631ed740 1 -- start start 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.123+0000 7f945e3f2640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f943c06ae70 0x7f943c06d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9454035000 tx=0x7f9454008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.123+0000 7f94631ed740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 0x55929d868a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.123+0000 7f94631ed740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d8988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.123+0000 7f94631ed740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d868fa0 0x55929d898e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f94631ed740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929d7dc700 con 0x55929d895f40 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f94631ed740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929d7d9df0 con 0x55929d895b60 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f94631ed740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55929d7d56e0 con 0x55929d868fa0 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945dbf1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d8988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945dbf1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d8988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45462/0 (socket says 172.21.15.70:45462) 2023-10-08T05:36:20.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945dbf1640 1 -- 172.21.15.70:0/1760550916 learned_addr learned my addr 172.21.15.70:0/1760550916 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945e3f2640 1 --2- 172.21.15.70:0/1760550916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 0x55929d868a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945ebf3640 1 --2- 172.21.15.70:0/1760550916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d868fa0 0x55929d898e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945dbf1640 1 -- 172.21.15.70:0/1760550916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d868fa0 msgr2=0x55929d898e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945dbf1640 1 --2- 172.21.15.70:0/1760550916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d868fa0 0x55929d898e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945dbf1640 1 -- 172.21.15.70:0/1760550916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 msgr2=0x55929d868a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945dbf1640 1 --2- 172.21.15.70:0/1760550916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 0x55929d868a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945dbf1640 1 -- 172.21.15.70:0/1760550916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55929d799910 con 0x55929d895f40 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945e3f2640 1 --2- 172.21.15.70:0/1760550916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 0x55929d868a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.124+0000 7f945dbf1640 1 --2- 172.21.15.70:0/1760550916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d8988f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9448001750 tx=0x7f9448001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd809f44b8b17a95 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.125+0000 7f943bfff640 1 -- 172.21.15.70:0/1760550916 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800b070 con 0x55929d895f40 2023-10-08T05:36:20.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.125+0000 7f943bfff640 1 -- 172.21.15.70:0/1760550916 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944800d730 con 0x55929d895f40 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.125+0000 7f943bfff640 1 -- 172.21.15.70:0/1760550916 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800dae0 con 0x55929d895f40 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.125+0000 7f94631ed740 1 -- 172.21.15.70:0/1760550916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 msgr2=0x55929d8988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.125+0000 7f94631ed740 1 --2- 172.21.15.70:0/1760550916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d8988f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9448001750 tx=0x7f9448001c10 comp rx=0 tx=0).stop 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.125+0000 7f94631ed740 1 -- 172.21.15.70:0/1760550916 shutdown_connections 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.125+0000 7f94631ed740 1 --2- 172.21.15.70:0/1760550916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d868fa0 0x55929d898e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.125+0000 7f94631ed740 1 --2- 172.21.15.70:0/1760550916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d8988f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.125+0000 7f94631ed740 1 --2- 172.21.15.70:0/1760550916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 0x55929d868a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.125+0000 7f94631ed740 1 -- 172.21.15.70:0/1760550916 >> 172.21.15.70:0/1760550916 conn(0x55929d86ad70 msgr2=0x55929d8b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.126+0000 7f94631ed740 1 -- 172.21.15.70:0/1760550916 shutdown_connections 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.126+0000 7f94631ed740 1 -- 172.21.15.70:0/1760550916 wait complete. 2023-10-08T05:36:20.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.126+0000 7f94631ed740 1 -- start start 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.126+0000 7f94631ed740 1 Processor -- start 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.127+0000 7f94631ed740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 0x55929d8836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.127+0000 7f94631ed740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895f40 0x55929d89b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.127+0000 7f94631ed740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.127+0000 7f94631ed740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929d773970 con 0x55929d8ab000 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.127+0000 7f94631ed740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929d773610 con 0x55929d895f40 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.127+0000 7f94631ed740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55929d539f20 con 0x55929d895b60 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f94631ed740 10 client.0 _ll_register_callbacks cb 0x55929d7b1610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945ebf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945ebf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45472/0 (socket says 172.21.15.70:45472) 2023-10-08T05:36:20.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945ebf3640 1 -- 172.21.15.70:0/1746510525 learned_addr learned my addr 172.21.15.70:0/1746510525 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945e3f2640 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 0x55929d8836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945ebf3640 1 -- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 msgr2=0x55929d8836c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945ebf3640 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 0x55929d8836c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945ebf3640 1 -- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895f40 msgr2=0x55929d89b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945dbf1640 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895f40 0x55929d89b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945ebf3640 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895f40 0x55929d89b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945ebf3640 1 -- 172.21.15.70:0/1746510525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9448001350 con 0x55929d8ab000 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945e3f2640 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 0x55929d8836c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f945ebf3640 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8adf20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f945000ab50 tx=0x7f94500594e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af5e0dcdd25dd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.128+0000 7f94397fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:36:20.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94397fa640 1 -- 172.21.15.70:0/1746510525 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450061020 con 0x55929d8ab000 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94397fa640 1 -- 172.21.15.70:0/1746510525 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f945000fd10 con 0x55929d8ab000 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94631ed740 1 -- 172.21.15.70:0/1746510525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55929d7ce6c0 con 0x55929d8ab000 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94397fa640 1 -- 172.21.15.70:0/1746510525 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9450058210 con 0x55929d8ab000 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94631ed740 10 client.4404 fetch_fsmap learned FSMap version 7 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94631ed740 1 -- 172.21.15.70:0/1746510525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55929d7d3d60 con 0x55929d8ab000 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94397fa640 1 -- 172.21.15.70:0/1746510525 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55929d7ce6c0 con 0x55929d8ab000 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94397fa640 1 -- 172.21.15.70:0/1746510525 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f9450020070 con 0x55929d8ab000 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94631ed740 10 client.4404 fetch_fsmap finished waiting for FSMap version 7 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94631ed740 10 client.4404 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94631ed740 20 client.4404 populate_metadata read hostname 'smithi070' 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94631ed740 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55929d885700 0x55929d885ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94631ed740 4 client.4404 mds_command: new command op to 4255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.129+0000 7f94631ed740 1 -- 172.21.15.70:0/1746510525 --> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55929d7ce3e0 con 0x55929d885700 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.130+0000 7f945e3f2640 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55929d885700 0x55929d885ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.131+0000 7f945e3f2640 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55929d885700 0x55929d885ea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c19064c2d75b88f server_cookie=706bd5522d565b43 in_seq=0 out_seq=0 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.131+0000 7f94397fa640 10 client.4404 ms_handle_connect on v2:172.21.15.181:6835/1675009602 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94397fa640 1 -- 172.21.15.70:0/1746510525 <== mds.0 v2:172.21.15.181:6835/1675009602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55929d7ce3e0 con 0x55929d885700 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94397fa640 10 client.4404 handle_command_reply: tid=0 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 client.4404 shutdown 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 -- 172.21.15.70:0/1746510525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 msgr2=0x55929d8adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8adf20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f945000ab50 tx=0x7f94500594e0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 -- 172.21.15.70:0/1746510525 shutdown_connections 2023-10-08T05:36:20.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55929d885700 0x55929d885ea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8adf20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895f40 0x55929d89b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 --2- 172.21.15.70:0/1746510525 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 0x55929d8836c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94397fa640 0 client.4404 ms_handle_reset on v2:172.21.15.181:6835/1675009602 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 -- 172.21.15.70:0/1746510525 >> 172.21.15.70:0/1746510525 conn(0x55929d868fa0 msgr2=0x55929d86c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 -- 172.21.15.70:0/1746510525 shutdown_connections 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 1 -- 172.21.15.70:0/1746510525 wait complete. 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.133+0000 7f94631ed740 20 client.4404 trim_cache size 0 max 16384 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.150+0000 7f94631ed740 1 Processor -- start 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.150+0000 7f94631ed740 1 -- start start 2023-10-08T05:36:20.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f94631ed740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 0x55929d8825d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f94631ed740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d882b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f94631ed740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d8ab000 0x55929d89a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f94631ed740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929d8af280 con 0x55929d895f40 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f94631ed740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929d894be0 con 0x55929d8ab000 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f94631ed740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55929d7d75c0 con 0x55929d895b60 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f945dbf1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d882b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f945dbf1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d882b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45474/0 (socket says 172.21.15.70:45474) 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f945dbf1640 1 -- 172.21.15.70:0/3539016738 learned_addr learned my addr 172.21.15.70:0/3539016738 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f945e3f2640 1 --2- 172.21.15.70:0/3539016738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 0x55929d8825d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f945dbf1640 1 -- 172.21.15.70:0/3539016738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 msgr2=0x55929d8825d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f945dbf1640 1 --2- 172.21.15.70:0/3539016738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 0x55929d8825d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f945ebf3640 1 --2- 172.21.15.70:0/3539016738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d8ab000 0x55929d89a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f945dbf1640 1 -- 172.21.15.70:0/3539016738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d8ab000 msgr2=0x55929d89a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f945dbf1640 1 --2- 172.21.15.70:0/3539016738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d8ab000 0x55929d89a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.151+0000 7f945dbf1640 1 -- 172.21.15.70:0/3539016738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55929d772b80 con 0x55929d895f40 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f945dbf1640 1 --2- 172.21.15.70:0/3539016738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d882b10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9448016040 tx=0x7f944800b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a479439618a60142 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f944cff9640 1 -- 172.21.15.70:0/3539016738 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94480079a0 con 0x55929d895f40 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f944cff9640 1 -- 172.21.15.70:0/3539016738 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9448007ba0 con 0x55929d895f40 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f944cff9640 1 -- 172.21.15.70:0/3539016738 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800dae0 con 0x55929d895f40 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f94631ed740 1 -- 172.21.15.70:0/3539016738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 msgr2=0x55929d882b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f94631ed740 1 --2- 172.21.15.70:0/3539016738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d882b10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9448016040 tx=0x7f944800b000 comp rx=0 tx=0).stop 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f94631ed740 1 -- 172.21.15.70:0/3539016738 shutdown_connections 2023-10-08T05:36:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f94631ed740 1 --2- 172.21.15.70:0/3539016738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d8ab000 0x55929d89a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f94631ed740 1 --2- 172.21.15.70:0/3539016738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d895f40 0x55929d882b10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f94631ed740 1 --2- 172.21.15.70:0/3539016738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895b60 0x55929d8825d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.152+0000 7f94631ed740 1 -- 172.21.15.70:0/3539016738 >> 172.21.15.70:0/3539016738 conn(0x55929d868fa0 msgr2=0x55929d86c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.153+0000 7f94631ed740 1 -- 172.21.15.70:0/3539016738 shutdown_connections 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.153+0000 7f94631ed740 1 -- 172.21.15.70:0/3539016738 wait complete. 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.153+0000 7f94631ed740 1 -- start start 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.153+0000 7f94631ed740 1 Processor -- start 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.154+0000 7f94631ed740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 0x55929d8825d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.154+0000 7f94631ed740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895f40 0x55929d882b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.154+0000 7f94631ed740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.154+0000 7f94631ed740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929d886db0 con 0x55929d8ab000 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.154+0000 7f94631ed740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55929d925960 con 0x55929d895b60 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.154+0000 7f94631ed740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55929d8aa740 con 0x55929d895f40 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.154+0000 7f94631ed740 10 client.0 _ll_register_callbacks cb 0x7f9448007d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945ebf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945ebf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45486/0 (socket says 172.21.15.70:45486) 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945ebf3640 1 -- 172.21.15.70:0/2273959374 learned_addr learned my addr 172.21.15.70:0/2273959374 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945dbf1640 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895f40 0x55929d882b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945e3f2640 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 0x55929d8825d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945ebf3640 1 -- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895f40 msgr2=0x55929d882b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945ebf3640 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895f40 0x55929d882b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945ebf3640 1 -- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 msgr2=0x55929d8825d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945ebf3640 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 0x55929d8825d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945ebf3640 1 -- 172.21.15.70:0/2273959374 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55929d799910 con 0x55929d8ab000 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945e3f2640 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 0x55929d8825d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f945ebf3640 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8a9280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9450058a20 tx=0x7f945005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b543b1c209919d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f94397fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f94397fa640 1 -- 172.21.15.70:0/2273959374 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9450007960 con 0x55929d8ab000 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f94631ed740 1 -- 172.21.15.70:0/2273959374 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f944800d730 con 0x55929d8ab000 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.155+0000 7f94397fa640 1 -- 172.21.15.70:0/2273959374 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f945000fd10 con 0x55929d8ab000 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94397fa640 1 -- 172.21.15.70:0/2273959374 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9450061020 con 0x55929d8ab000 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94631ed740 10 client.4410 fetch_fsmap learned FSMap version 7 2023-10-08T05:36:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94631ed740 1 -- 172.21.15.70:0/2273959374 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55929d798170 con 0x55929d8ab000 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94397fa640 1 -- 172.21.15.70:0/2273959374 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945007c020 con 0x55929d8ab000 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94397fa640 1 -- 172.21.15.70:0/2273959374 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f9450020040 con 0x55929d8ab000 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94631ed740 10 client.4410 fetch_fsmap finished waiting for FSMap version 7 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94631ed740 10 client.4410 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94631ed740 20 client.4410 populate_metadata read hostname 'smithi070' 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94631ed740 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55929d8a97c0 0x55929d8a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94631ed740 4 client.4410 mds_command: new command op to 4255 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.156+0000 7f94631ed740 1 -- 172.21.15.70:0/2273959374 --> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55929d4d76c0 con 0x55929d8a97c0 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.157+0000 7f945e3f2640 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55929d8a97c0 0x55929d8a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.157+0000 7f94397fa640 10 client.4410 ms_handle_connect on v2:172.21.15.181:6835/1675009602 2023-10-08T05:36:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.157+0000 7f945e3f2640 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55929d8a97c0 0x55929d8a9ba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8041eb564e7d57f6 server_cookie=c3164c94cda57906 in_seq=0 out_seq=0 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94397fa640 1 -- 172.21.15.70:0/2273959374 <== mds.0 v2:172.21.15.181:6835/1675009602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55929d4d76c0 con 0x55929d8a97c0 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94397fa640 10 client.4410 handle_command_reply: tid=0 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94631ed740 1 client.4410 shutdown 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94631ed740 1 -- 172.21.15.70:0/2273959374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 msgr2=0x55929d8a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94631ed740 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8a9280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9450058a20 tx=0x7f945005b000 comp rx=0 tx=0).stop 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94631ed740 1 -- 172.21.15.70:0/2273959374 shutdown_connections 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94631ed740 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55929d8a97c0 0x55929d8a9ba0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94631ed740 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55929d8ab000 0x55929d8a9280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94631ed740 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55929d895f40 0x55929d882b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94631ed740 1 --2- 172.21.15.70:0/2273959374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55929d895b60 0x55929d8825d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94397fa640 0 client.4410 ms_handle_reset on v2:172.21.15.181:6835/1675009602 2023-10-08T05:36:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.158+0000 7f94631ed740 1 -- 172.21.15.70:0/2273959374 >> 172.21.15.70:0/2273959374 conn(0x55929d868fa0 msgr2=0x55929d87db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:20.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f94631ed740 1 -- 172.21.15.70:0/2273959374 shutdown_connections 2023-10-08T05:36:20.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f94631ed740 1 -- 172.21.15.70:0/2273959374 wait complete. 2023-10-08T05:36:20.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f94631ed740 20 client.4410 trim_cache size 0 max 16384 2023-10-08T05:36:20.666 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:36:20.666 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T05:36:20.666 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T05:36:20.666 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 4255, 2023-10-08T05:36:20.667 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T05:36:20.667 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T05:36:20.667 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T05:36:20.667 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 8.0110575770000008, 2023-10-08T05:36:20.667 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 7, 2023-10-08T05:36:20.667 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 25, 2023-10-08T05:36:20.667 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 25, 2023-10-08T05:36:20.667 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 15.846117202 2023-10-08T05:36:20.667 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:36:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 -- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f943c06ae70 msgr2=0x7f943c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f943c06ae70 0x7f943c06d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9454035000 tx=0x7f9454008030 comp rx=0 tx=0).stop 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 -- 172.21.15.70:0/3465672218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458161090 msgr2=0x7f94580ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458161090 0x7f94580ca0b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f945000cd50 tx=0x7f945000a650 comp rx=0 tx=0).stop 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 -- 172.21.15.70:0/3465672218 shutdown_connections 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f943c06ae70 0x7f943c06d330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458161090 0x7f94580ca0b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f945815ce00 0x7f94580c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 --2- 172.21.15.70:0/3465672218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94580cd470 0x7f9458150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 -- 172.21.15.70:0/3465672218 >> 172.21.15.70:0/3465672218 conn(0x7f945806e650 msgr2=0x7f94580c8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 -- 172.21.15.70:0/3465672218 shutdown_connections 2023-10-08T05:36:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.159+0000 7f9460656640 1 -- 172.21.15.70:0/3465672218 wait complete. 2023-10-08T05:36:20.670 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T05:36:20.670 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-08T05:36:20.670 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:36:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.892+0000 7f60d3f33640 1 Processor -- start 2023-10-08T05:36:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.893+0000 7f60d3f33640 1 -- start start 2023-10-08T05:36:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.894+0000 7f60d3f33640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60cc15ab40 0x7f60cc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.894+0000 7f60d3f33640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f60cc0c90b0 0x7f60cc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.894+0000 7f60d3f33640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f60cc0cae70 0x7f60cc0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.894+0000 7f60d3f33640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60cc15bf30 con 0x7f60cc0c90b0 2023-10-08T05:36:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.894+0000 7f60d3f33640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60cc0cb840 con 0x7f60cc0cae70 2023-10-08T05:36:20.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.894+0000 7f60d3f33640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc15b2b0 con 0x7f60cc15ab40 2023-10-08T05:36:20.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.895+0000 7f60d14ce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f60cc0c90b0 0x7f60cc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.895+0000 7f60d14ce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f60cc0c90b0 0x7f60cc0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45498/0 (socket says 172.21.15.70:45498) 2023-10-08T05:36:20.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.895+0000 7f60d14ce640 1 -- 172.21.15.70:0/1403772042 learned_addr learned my addr 172.21.15.70:0/1403772042 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:20.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.895+0000 7f60d1ccf640 1 --2- 172.21.15.70:0/1403772042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60cc15ab40 0x7f60cc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.895+0000 7f60d24d0640 1 --2- 172.21.15.70:0/1403772042 >> v2:172.21.15.181:3300/0 conn(0x7f60cc0cae70 0x7f60cc0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.895+0000 7f60d14ce640 1 -- 172.21.15.70:0/1403772042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60cc15ab40 msgr2=0x7f60cc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.895+0000 7f60d14ce640 1 --2- 172.21.15.70:0/1403772042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60cc15ab40 0x7f60cc0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.895+0000 7f60d14ce640 1 -- 172.21.15.70:0/1403772042 >> v2:172.21.15.181:3300/0 conn(0x7f60cc0cae70 msgr2=0x7f60cc0cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.895+0000 7f60d14ce640 1 --2- 172.21.15.70:0/1403772042 >> v2:172.21.15.181:3300/0 conn(0x7f60cc0cae70 0x7f60cc0cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.895+0000 7f60d14ce640 1 -- 172.21.15.70:0/1403772042 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc15b590 con 0x7f60cc0c90b0 2023-10-08T05:36:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.896+0000 7f60d14ce640 1 --2- 172.21.15.70:0/1403772042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0c90b0 0x7f60cc0adc60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0c8d50 tx=0x7f60bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cd90d43489f883 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d0ccd640 1 -- 172.21.15.70:0/1403772042 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc031870 con 0x7f60cc0c90b0 2023-10-08T05:36:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d0ccd640 1 -- 172.21.15.70:0/1403772042 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc0319d0 con 0x7f60cc0c90b0 2023-10-08T05:36:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d0ccd640 1 -- 172.21.15.70:0/1403772042 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc031d60 con 0x7f60cc0c90b0 2023-10-08T05:36:20.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d3f33640 1 -- 172.21.15.70:0/1403772042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0c90b0 msgr2=0x7f60cc0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d3f33640 1 --2- 172.21.15.70:0/1403772042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0c90b0 0x7f60cc0adc60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0c8d50 tx=0x7f60bc02b910 comp rx=0 tx=0).stop 2023-10-08T05:36:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d3f33640 1 -- 172.21.15.70:0/1403772042 shutdown_connections 2023-10-08T05:36:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d3f33640 1 --2- 172.21.15.70:0/1403772042 >> v2:172.21.15.181:3300/0 conn(0x7f60cc0cae70 0x7f60cc0cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d3f33640 1 --2- 172.21.15.70:0/1403772042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0c90b0 0x7f60cc0adc60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d3f33640 1 --2- 172.21.15.70:0/1403772042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60cc15ab40 0x7f60cc0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d3f33640 1 -- 172.21.15.70:0/1403772042 >> 172.21.15.70:0/1403772042 conn(0x7f60cc06a380 msgr2=0x7f60cc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d3f33640 1 -- 172.21.15.70:0/1403772042 shutdown_connections 2023-10-08T05:36:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.897+0000 7f60d3f33640 1 -- 172.21.15.70:0/1403772042 wait complete. 2023-10-08T05:36:20.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.899+0000 7f60d3f33640 1 Processor -- start 2023-10-08T05:36:20.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.899+0000 7f60d3f33640 1 -- start start 2023-10-08T05:36:20.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.900+0000 7f60d3f33640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60cc0c90b0 0x7f60cc149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.900+0000 7f60d3f33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0cae70 0x7f60cc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.900+0000 7f60d3f33640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60cc15ab40 0x7f60cc06da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.900+0000 7f60d3f33640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0b39d0 con 0x7f60cc0cae70 2023-10-08T05:36:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.900+0000 7f60d3f33640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0b2dc0 con 0x7f60cc15ab40 2023-10-08T05:36:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.900+0000 7f60d3f33640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0b30a0 con 0x7f60cc0c90b0 2023-10-08T05:36:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d14ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0cae70 0x7f60cc06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d14ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0cae70 0x7f60cc06d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45504/0 (socket says 172.21.15.70:45504) 2023-10-08T05:36:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d14ce640 1 -- 172.21.15.70:0/3757473371 learned_addr learned my addr 172.21.15.70:0/3757473371 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d1ccf640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60cc0c90b0 0x7f60cc149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d24d0640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60cc15ab40 0x7f60cc06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d14ce640 1 -- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60cc0c90b0 msgr2=0x7f60cc149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d14ce640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60cc0c90b0 0x7f60cc149910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d14ce640 1 -- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60cc15ab40 msgr2=0x7f60cc06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d14ce640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60cc15ab40 0x7f60cc06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d14ce640 1 -- 172.21.15.70:0/3757473371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc0094d0 con 0x7f60cc0cae70 2023-10-08T05:36:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.901+0000 7f60d14ce640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0cae70 0x7f60cc06d530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f60bc02b950 tx=0x7f60bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.902+0000 7f60c27fc640 1 -- 172.21.15.70:0/3757473371 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc03ad30 con 0x7f60cc0cae70 2023-10-08T05:36:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.902+0000 7f60c27fc640 1 -- 172.21.15.70:0/3757473371 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc0443f0 con 0x7f60cc0cae70 2023-10-08T05:36:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.902+0000 7f60c27fc640 1 -- 172.21.15.70:0/3757473371 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc0447a0 con 0x7f60cc0cae70 2023-10-08T05:36:20.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.902+0000 7f60d3f33640 1 -- 172.21.15.70:0/3757473371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60cc0b3380 con 0x7f60cc0cae70 2023-10-08T05:36:20.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.902+0000 7f60d3f33640 1 -- 172.21.15.70:0/3757473371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60cc154d70 con 0x7f60cc0cae70 2023-10-08T05:36:20.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.903+0000 7f60d3f33640 1 -- 172.21.15.70:0/3757473371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60cc0b3b80 con 0x7f60cc0cae70 2023-10-08T05:36:20.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.904+0000 7f60c27fc640 1 -- 172.21.15.70:0/3757473371 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f60bc04d450 con 0x7f60cc0cae70 2023-10-08T05:36:20.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.905+0000 7f60c27fc640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f60b006aea0 0x7f60b006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:20.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.905+0000 7f60c27fc640 1 -- 172.21.15.70:0/3757473371 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f60bc02d040 con 0x7f60cc0cae70 2023-10-08T05:36:20.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.906+0000 7f60d1ccf640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f60b006aea0 0x7f60b006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:20.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.907+0000 7f60d1ccf640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f60b006aea0 0x7f60b006d360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f60c8006fd0 tx=0x7f60c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:20.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:20.910+0000 7f60c27fc640 1 -- 172.21.15.70:0/3757473371 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60cc0b3b80 con 0x7f60cc0cae70 2023-10-08T05:36:21.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.138+0000 7f60d3f33640 1 -- 172.21.15.70:0/3757473371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f60cc0b3b80 con 0x7f60cc0cae70 2023-10-08T05:36:21.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.139+0000 7f60c27fc640 1 -- 172.21.15.70:0/3757473371 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f60cc0b3b80 con 0x7f60cc0cae70 2023-10-08T05:36:21.139 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T05:36:21.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.141+0000 7f60d3f33640 1 -- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f60b006aea0 msgr2=0x7f60b006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:21.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.141+0000 7f60d3f33640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f60b006aea0 0x7f60b006d360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f60c8006fd0 tx=0x7f60c8008040 comp rx=0 tx=0).stop 2023-10-08T05:36:21.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.142+0000 7f60d3f33640 1 -- 172.21.15.70:0/3757473371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0cae70 msgr2=0x7f60cc06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:21.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.142+0000 7f60d3f33640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0cae70 0x7f60cc06d530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f60bc02b950 tx=0x7f60bc0046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:21.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.142+0000 7f60d3f33640 1 -- 172.21.15.70:0/3757473371 shutdown_connections 2023-10-08T05:36:21.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.142+0000 7f60d3f33640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f60b006aea0 0x7f60b006d360 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:21.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.142+0000 7f60d3f33640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60cc15ab40 0x7f60cc06da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:21.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.142+0000 7f60d3f33640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60cc0cae70 0x7f60cc06d530 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:21.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.142+0000 7f60d3f33640 1 --2- 172.21.15.70:0/3757473371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60cc0c90b0 0x7f60cc149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:21.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.142+0000 7f60d3f33640 1 -- 172.21.15.70:0/3757473371 >> 172.21.15.70:0/3757473371 conn(0x7f60cc06a380 msgr2=0x7f60cc0af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:21.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.142+0000 7f60d3f33640 1 -- 172.21.15.70:0/3757473371 shutdown_connections 2023-10-08T05:36:21.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:21.142+0000 7f60d3f33640 1 -- 172.21.15.70:0/3757473371 wait complete. 2023-10-08T05:36:21.160 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T05:36:21.160 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:36:21.160 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:36:21.160 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-08T05:36:21.160 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-08T05:36:21.160 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T05:36:21.160 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T05:36:21.160 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:36:21.160 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-08T05:36:21.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:21.161 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:36:21.214 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:36:21.214 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:36:21.215 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:36:21.215 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:21.215 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:36:21.215 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:21.215 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:21.215 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:21.216 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:36:21.216 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:21.216 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:21.216 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:36:21.216 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:21.217 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:21.217 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:36:21.217 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:21.217 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:36:21.217 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42415sec preferred_lft 42415sec 2023-10-08T05:36:21.217 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:36:21.218 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:21.218 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T05:36:21.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:21.219 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:21.219 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:21.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add name ceph-brx type bridge 2023-10-08T05:36:21.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr flush dev ceph-brx 2023-10-08T05:36:21.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set ceph-brx up 2023-10-08T05:36:21.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T05:36:21.220 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:21.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:21.709 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:36:21.786 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:36:21.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:21.790 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:36:21.852 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:36:21.853 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:36:21.853 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:36:21.853 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:36:21.853 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:36:21.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:21.853 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:21.853 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:21.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:36:21.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:36:21.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:36:21.854 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:22.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:22.632 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:36:22.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:22.689 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2023-10-08T05:36:22.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:22.741 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:22.741 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:22.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T05:36:22.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-08T05:36:22.742 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:22.882 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-08T05:36:22.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:22.883 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:22.883 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:22.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-08T05:36:22.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T05:36:22.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-08T05:36:22.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-08T05:36:22.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-08T05:36:22.883 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:23.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:23.165 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:23.165 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:23.165 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T05:36:23.165 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T05:36:23.165 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:23.318 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-08T05:36:23.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:36:23.318 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-08T05:36:23.365 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-08T05:36:23.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:36:23.365 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-08T05:36:23.411 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-10-08T05:36:23.555 DEBUG:teuthology.orchestra.run.smithi070:> cat /proc/self/mounts | awk '{print $2}' 2023-10-08T05:36:23.618 INFO:teuthology.orchestra.run.smithi070.stdout:/proc 2023-10-08T05:36:23.619 INFO:teuthology.orchestra.run.smithi070.stdout:/sys 2023-10-08T05:36:23.619 INFO:teuthology.orchestra.run.smithi070.stdout:/dev 2023-10-08T05:36:23.619 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/security 2023-10-08T05:36:23.619 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/shm 2023-10-08T05:36:23.619 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/pts 2023-10-08T05:36:23.619 INFO:teuthology.orchestra.run.smithi070.stdout:/run 2023-10-08T05:36:23.619 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup 2023-10-08T05:36:23.619 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/pstore 2023-10-08T05:36:23.619 INFO:teuthology.orchestra.run.smithi070.stdout:/ 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/selinux 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/hugepages 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/mqueue 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/debug 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/tracing 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/fuse/connections 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/config 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-sysctl.service 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-08T05:36:23.620 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/nfs/rpc_pipefs 2023-10-08T05:36:23.621 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/1000 2023-10-08T05:36:23.621 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph 2023-10-08T05:36:23.621 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-0 2023-10-08T05:36:23.621 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-1 2023-10-08T05:36:23.621 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-2 2023-10-08T05:36:23.621 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-3 2023-10-08T05:36:23.621 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns 2023-10-08T05:36:23.621 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T05:36:23.621 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T05:36:23.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:23.622 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-10-08T05:36:23.669 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-08T05:36:23.669 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-08T05:36:23.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-10-08T05:36:23.751 DEBUG:teuthology.orchestra.run.smithi070:> cat /proc/self/mounts | awk '{print $2}' 2023-10-08T05:36:23.792 INFO:teuthology.orchestra.run.smithi070.stdout:/proc 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/sys 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/dev 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/security 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/shm 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/pts 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/run 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/pstore 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/ 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/selinux 2023-10-08T05:36:23.793 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/hugepages 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/mqueue 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/debug 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/tracing 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/fuse/connections 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/config 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-sysctl.service 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/nfs/rpc_pipefs 2023-10-08T05:36:23.794 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/1000 2023-10-08T05:36:23.795 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph 2023-10-08T05:36:23.795 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-0 2023-10-08T05:36:23.795 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-1 2023-10-08T05:36:23.795 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-2 2023-10-08T05:36:23.795 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-3 2023-10-08T05:36:23.795 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns 2023-10-08T05:36:23.795 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T05:36:23.796 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T05:36:23.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:23.797 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-10-08T05:36:23.904 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi070.stderr:2023-10-08T05:36:23.903+0000 7ff8e5971480 -1 init, newargv = 0x55d29be38780 newargc=15 2023-10-08T05:36:23.904 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi070.stderr:ceph-fuse[92651]: starting ceph client 2023-10-08T05:36:23.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi070.stderr:ceph-fuse[92651]: starting fuse 2023-10-08T05:36:24.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-10-08T05:36:25.102 DEBUG:teuthology.orchestra.run.smithi070:> cat /proc/self/mounts | awk '{print $2}' 2023-10-08T05:36:25.159 INFO:teuthology.orchestra.run.smithi070.stdout:/proc 2023-10-08T05:36:25.159 INFO:teuthology.orchestra.run.smithi070.stdout:/sys 2023-10-08T05:36:25.159 INFO:teuthology.orchestra.run.smithi070.stdout:/dev 2023-10-08T05:36:25.159 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/security 2023-10-08T05:36:25.159 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/shm 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/pts 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/run 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/pstore 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/ 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/selinux 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/hugepages 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/mqueue 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/debug 2023-10-08T05:36:25.160 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/tracing 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/fuse/connections 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/config 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-sysctl.service 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/nfs/rpc_pipefs 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/1000 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-0 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-1 2023-10-08T05:36:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-2 2023-10-08T05:36:25.162 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph/osd/ceph-3 2023-10-08T05:36:25.162 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns 2023-10-08T05:36:25.162 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T05:36:25.162 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T05:36:25.162 INFO:teuthology.orchestra.run.smithi070.stdout:/home/ubuntu/cephtest/mnt.admin 2023-10-08T05:36:25.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:25.163 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-10-08T05:36:25.215 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-10-08T05:36:25.215 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-10-08T05:36:25.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-10-08T05:36:25.216 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-10-08T05:36:25.216 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-10-08T05:36:25.216 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-10-08T05:36:25.216 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-10-08T05:36:25.216 DEBUG:teuthology.orchestra.run.smithi070:> 2023-10-08T05:36:25.216 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-10-08T05:36:25.216 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-08T05:36:25.216 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-08T05:36:25.217 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-10-08T05:36:25.218 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-10-08T05:36:25.218 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-10-08T05:36:25.218 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-08T05:36:25.218 DEBUG:teuthology.orchestra.run.smithi070:> 2023-10-08T05:36:25.218 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.admin")) 2023-10-08T05:36:25.218 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-10-08T05:36:25.397 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.admin.92651.asok 2023-10-08T05:36:25.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-08T05:36:25.414 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.92651.asok 2023-10-08T05:36:25.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:25.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.92651.asok status 2023-10-08T05:36:25.658 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:36:25.658 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-10-08T05:36:25.658 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "9725b08fd801c82987c737f7e6d354878f036ee8", 2023-10-08T05:36:25.658 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-6609-g9725b08f (9725b08fd801c82987c737f7e6d354878f036ee8) reef (dev)", 2023-10-08T05:36:25.658 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "admin", 2023-10-08T05:36:25.658 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-10-08T05:36:25.658 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-08T05:36:25.658 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "92651", 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 4422, 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 4422 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-10-08T05:36:25.659 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:0", 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3397493686 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:0", 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3397493686 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.4422 172.21.15.70:0/3397493686", 2023-10-08T05:36:25.660 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "172.21.15.70:0/3397493686", 2023-10-08T05:36:25.661 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-10-08T05:36:25.661 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 7, 2023-10-08T05:36:25.661 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 25, 2023-10-08T05:36:25.661 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-10-08T05:36:25.661 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-10-08T05:36:25.661 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-10-08T05:36:25.661 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:36:25.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:25.672 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-10-08T05:36:25.672 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-08T05:36:25.726 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-10-08T05:36:25.726 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-08T05:36:25.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:25.726 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-08T05:36:25.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:25.816 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-08T05:36:25.909 INFO:teuthology.orchestra.run.smithi070.stdout:# file: . 2023-10-08T05:36:25.909 INFO:teuthology.orchestra.run.smithi070.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-08T05:36:25.909 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:25.910 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:36:25.910 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:36:25.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:26.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.170+0000 7fc550a4b640 1 Processor -- start 2023-10-08T05:36:26.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.170+0000 7fc550a4b640 1 -- start start 2023-10-08T05:36:26.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.172+0000 7fc550a4b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc54c157000 0x7fc54c0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.172+0000 7fc550a4b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc54c0cb190 0x7fc54c0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.172+0000 7fc550a4b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c0adfc0 con 0x7fc54c1597d0 2023-10-08T05:36:26.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.172+0000 7fc550a4b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c0cbb40 con 0x7fc54c0cb190 2023-10-08T05:36:26.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.172+0000 7fc550a4b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c0ccce0 con 0x7fc54c157000 2023-10-08T05:36:26.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.172+0000 7fc54ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc54c0cb190 0x7fc54c0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.173+0000 7fc54ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc54c0cb190 0x7fc54c0cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49186/0 (socket says 172.21.15.70:49186) 2023-10-08T05:36:26.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.173+0000 7fc54ad9d640 1 -- 172.21.15.70:0/3726215810 learned_addr learned my addr 172.21.15.70:0/3726215810 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:26.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.173+0000 7fc543fff640 1 --1- 172.21.15.70:0/3726215810 >> v1:172.21.15.70:6789/0 conn(0x7fc54c1597d0 0x7fc54c06ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58608/0 (socket says 172.21.15.70:58608) 2023-10-08T05:36:26.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.173+0000 7fc54a59c640 1 --2- 172.21.15.70:0/3726215810 >> v2:172.21.15.181:3300/0 conn(0x7fc54c157000 0x7fc54c0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.173+0000 7fc54ad9d640 1 -- 172.21.15.70:0/3726215810 >> v2:172.21.15.181:3300/0 conn(0x7fc54c157000 msgr2=0x7fc54c0cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:26.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.173+0000 7fc54ad9d640 1 --2- 172.21.15.70:0/3726215810 >> v2:172.21.15.181:3300/0 conn(0x7fc54c157000 0x7fc54c0cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.173+0000 7fc54ad9d640 1 -- 172.21.15.70:0/3726215810 >> v1:172.21.15.70:6789/0 conn(0x7fc54c1597d0 legacy=0x7fc54c06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.173+0000 7fc54ad9d640 1 -- 172.21.15.70:0/3726215810 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c159cb0 con 0x7fc54c0cb190 2023-10-08T05:36:26.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.173+0000 7fc549d9b640 1 -- 172.21.15.70:0/3726215810 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570182989 0 0) 0x7fc54c0adfc0 con 0x7fc54c1597d0 2023-10-08T05:36:26.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.174+0000 7fc54ad9d640 1 --2- 172.21.15.70:0/3726215810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c0cb190 0x7fc54c0cb600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc54c071700 tx=0x7fc54402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb96dc4bac7d68 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:26.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.174+0000 7fc549d9b640 1 -- 172.21.15.70:0/3726215810 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54402c920 con 0x7fc54c0cb190 2023-10-08T05:36:26.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.174+0000 7fc549d9b640 1 -- 172.21.15.70:0/3726215810 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc544030070 con 0x7fc54c0cb190 2023-10-08T05:36:26.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.174+0000 7fc549d9b640 1 -- 172.21.15.70:0/3726215810 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54402ccc0 con 0x7fc54c0cb190 2023-10-08T05:36:26.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.175+0000 7fc550a4b640 1 -- 172.21.15.70:0/3726215810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c0cb190 msgr2=0x7fc54c0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.175+0000 7fc550a4b640 1 --2- 172.21.15.70:0/3726215810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c0cb190 0x7fc54c0cb600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc54c071700 tx=0x7fc54402bf80 comp rx=0 tx=0).stop 2023-10-08T05:36:26.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.175+0000 7fc550a4b640 1 -- 172.21.15.70:0/3726215810 shutdown_connections 2023-10-08T05:36:26.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.175+0000 7fc550a4b640 1 --2- 172.21.15.70:0/3726215810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c0cb190 0x7fc54c0cb600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.175+0000 7fc550a4b640 1 --2- 172.21.15.70:0/3726215810 >> v2:172.21.15.181:3300/0 conn(0x7fc54c157000 0x7fc54c0cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.175+0000 7fc550a4b640 1 -- 172.21.15.70:0/3726215810 >> 172.21.15.70:0/3726215810 conn(0x7fc54c06a380 msgr2=0x7fc54c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:26.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.175+0000 7fc550a4b640 1 -- 172.21.15.70:0/3726215810 shutdown_connections 2023-10-08T05:36:26.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.175+0000 7fc550a4b640 1 -- 172.21.15.70:0/3726215810 wait complete. 2023-10-08T05:36:26.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.177+0000 7fc550a4b640 1 Processor -- start 2023-10-08T05:36:26.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.177+0000 7fc550a4b640 1 -- start start 2023-10-08T05:36:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.178+0000 7fc550a4b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54c0cb190 0x7fc54c0ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.178+0000 7fc550a4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c1597d0 0x7fc54c0c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.178+0000 7fc550a4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc54c0af240 0x7fc54c154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.178+0000 7fc550a4b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0c9820 con 0x7fc54c1597d0 2023-10-08T05:36:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.178+0000 7fc550a4b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0c9a30 con 0x7fc54c0cb190 2023-10-08T05:36:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.178+0000 7fc550a4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0c9d10 con 0x7fc54c0af240 2023-10-08T05:36:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.178+0000 7fc543fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c1597d0 0x7fc54c0c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.178+0000 7fc543fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c1597d0 0x7fc54c0c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49190/0 (socket says 172.21.15.70:49190) 2023-10-08T05:36:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.178+0000 7fc543fff640 1 -- 172.21.15.70:0/846193301 learned_addr learned my addr 172.21.15.70:0/846193301 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc54a59c640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54c0cb190 0x7fc54c0ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc543fff640 1 -- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc54c0af240 msgr2=0x7fc54c154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc543fff640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc54c0af240 0x7fc54c154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc543fff640 1 -- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54c0cb190 msgr2=0x7fc54c0ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc543fff640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54c0cb190 0x7fc54c0ad4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc543fff640 1 -- 172.21.15.70:0/846193301 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc544009b30 con 0x7fc54c1597d0 2023-10-08T05:36:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc543fff640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c1597d0 0x7fc54c0c92e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c4d0 tx=0x7fc53400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc542ffd640 1 -- 172.21.15.70:0/846193301 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534006480 con 0x7fc54c1597d0 2023-10-08T05:36:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc542ffd640 1 -- 172.21.15.70:0/846193301 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc534006600 con 0x7fc54c1597d0 2023-10-08T05:36:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc550a4b640 1 -- 172.21.15.70:0/846193301 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c0c9ff0 con 0x7fc54c1597d0 2023-10-08T05:36:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.179+0000 7fc542ffd640 1 -- 172.21.15.70:0/846193301 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534004a00 con 0x7fc54c1597d0 2023-10-08T05:36:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.180+0000 7fc550a4b640 1 -- 172.21.15.70:0/846193301 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c0ca5b0 con 0x7fc54c1597d0 2023-10-08T05:36:26.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.181+0000 7fc550a4b640 1 -- 172.21.15.70:0/846193301 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c0ca7d0 con 0x7fc54c1597d0 2023-10-08T05:36:26.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.182+0000 7fc542ffd640 1 -- 172.21.15.70:0/846193301 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc534004b60 con 0x7fc54c1597d0 2023-10-08T05:36:26.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.182+0000 7fc542ffd640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc52c06aea0 0x7fc52c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.183+0000 7fc542ffd640 1 -- 172.21.15.70:0/846193301 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc534088ed0 con 0x7fc54c1597d0 2023-10-08T05:36:26.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.183+0000 7fc54a59c640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc52c06aea0 0x7fc52c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.184+0000 7fc54a59c640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc52c06aea0 0x7fc52c06d360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc53c006fd0 tx=0x7fc53c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.187+0000 7fc542ffd640 1 -- 172.21.15.70:0/846193301 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc534053c80 con 0x7fc54c1597d0 2023-10-08T05:36:26.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.378+0000 7fc550a4b640 1 -- 172.21.15.70:0/846193301 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc54c067490 con 0x7fc54c1597d0 2023-10-08T05:36:26.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.379+0000 7fc542ffd640 1 -- 172.21.15.70:0/846193301 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fc534017070 con 0x7fc54c1597d0 2023-10-08T05:36:26.380 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:26.381 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:26.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 -- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc52c06aea0 msgr2=0x7fc52c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc52c06aea0 0x7fc52c06d360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc53c006fd0 tx=0x7fc53c008040 comp rx=0 tx=0).stop 2023-10-08T05:36:26.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 -- 172.21.15.70:0/846193301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c1597d0 msgr2=0x7fc54c0c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c1597d0 0x7fc54c0c92e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c4d0 tx=0x7fc53400c990 comp rx=0 tx=0).stop 2023-10-08T05:36:26.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 -- 172.21.15.70:0/846193301 shutdown_connections 2023-10-08T05:36:26.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc52c06aea0 0x7fc52c06d360 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc54c0af240 0x7fc54c154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc54c1597d0 0x7fc54c0c92e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 --2- 172.21.15.70:0/846193301 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc54c0cb190 0x7fc54c0ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 -- 172.21.15.70:0/846193301 >> 172.21.15.70:0/846193301 conn(0x7fc54c06a380 msgr2=0x7fc54c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:26.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.382+0000 7fc550a4b640 1 -- 172.21.15.70:0/846193301 shutdown_connections 2023-10-08T05:36:26.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.383+0000 7fc550a4b640 1 -- 172.21.15.70:0/846193301 wait complete. 2023-10-08T05:36:26.402 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:36:26.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:36:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.618+0000 7fadfbdd0640 1 Processor -- start 2023-10-08T05:36:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.618+0000 7fadfbdd0640 1 -- start start 2023-10-08T05:36:26.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.619+0000 7fadfbdd0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fadf41432f0 0x7fadf40cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.619+0000 7fadfbdd0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fadf40cb6b0 0x7fadf40b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.619+0000 7fadfbdd0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf40cbb30 con 0x7fadf40b4070 2023-10-08T05:36:26.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.619+0000 7fadfbdd0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf41436d0 con 0x7fadf41432f0 2023-10-08T05:36:26.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.619+0000 7fadfbdd0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf415ae60 con 0x7fadf40cb6b0 2023-10-08T05:36:26.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.620+0000 7fadf9b6c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fadf41432f0 0x7fadf40cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.620+0000 7fadf9b6c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fadf41432f0 0x7fadf40cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49198/0 (socket says 172.21.15.70:49198) 2023-10-08T05:36:26.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.620+0000 7fadf9b6c640 1 -- 172.21.15.70:0/123680454 learned_addr learned my addr 172.21.15.70:0/123680454 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:26.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.620+0000 7fadf936b640 1 --2- 172.21.15.70:0/123680454 >> v2:172.21.15.181:3300/0 conn(0x7fadf40cb6b0 0x7fadf40b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.620+0000 7fadf9b6c640 1 -- 172.21.15.70:0/123680454 >> v2:172.21.15.181:3300/0 conn(0x7fadf40cb6b0 msgr2=0x7fadf40b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:26.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.620+0000 7fadf9b6c640 1 --2- 172.21.15.70:0/123680454 >> v2:172.21.15.181:3300/0 conn(0x7fadf40cb6b0 0x7fadf40b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.620+0000 7fadf9b6c640 1 -- 172.21.15.70:0/123680454 >> v1:172.21.15.181:6789/0 conn(0x7fadf40b4070 legacy=0x7fadf406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:26.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.620+0000 7fadf9b6c640 1 -- 172.21.15.70:0/123680454 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf415d440 con 0x7fadf41432f0 2023-10-08T05:36:26.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.620+0000 7fadf936b640 1 --2- 172.21.15.70:0/123680454 >> v2:172.21.15.181:3300/0 conn(0x7fadf40cb6b0 0x7fadf40b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:26.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.621+0000 7fadf9b6c640 1 --2- 172.21.15.70:0/123680454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 0x7fadf40cb170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fadf406ab90 tx=0x7fadf002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afff952b2938e126 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:26.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.622+0000 7fadf8b6a640 1 -- 172.21.15.70:0/123680454 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00317e0 con 0x7fadf41432f0 2023-10-08T05:36:26.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.622+0000 7fadf8b6a640 1 -- 172.21.15.70:0/123680454 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0031940 con 0x7fadf41432f0 2023-10-08T05:36:26.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.622+0000 7fadf8b6a640 1 -- 172.21.15.70:0/123680454 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0031cd0 con 0x7fadf41432f0 2023-10-08T05:36:26.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.622+0000 7fadfbdd0640 1 -- 172.21.15.70:0/123680454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 msgr2=0x7fadf40cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.622+0000 7fadfbdd0640 1 --2- 172.21.15.70:0/123680454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 0x7fadf40cb170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fadf406ab90 tx=0x7fadf002b880 comp rx=0 tx=0).stop 2023-10-08T05:36:26.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.623+0000 7fadfbdd0640 1 -- 172.21.15.70:0/123680454 shutdown_connections 2023-10-08T05:36:26.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.623+0000 7fadfbdd0640 1 --2- 172.21.15.70:0/123680454 >> v2:172.21.15.181:3300/0 conn(0x7fadf40cb6b0 0x7fadf40b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.623+0000 7fadfbdd0640 1 --2- 172.21.15.70:0/123680454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 0x7fadf40cb170 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.623+0000 7fadfbdd0640 1 -- 172.21.15.70:0/123680454 >> 172.21.15.70:0/123680454 conn(0x7fadf4069d50 msgr2=0x7fadf406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:26.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.623+0000 7fadfbdd0640 1 -- 172.21.15.70:0/123680454 shutdown_connections 2023-10-08T05:36:26.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.623+0000 7fadfbdd0640 1 -- 172.21.15.70:0/123680454 wait complete. 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.624+0000 7fadfbdd0640 1 Processor -- start 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.625+0000 7fadfbdd0640 1 -- start start 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.625+0000 7fadfbdd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf40b4070 0x7fadf41f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfbdd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf40cb6b0 0x7fadf40c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfbdd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 0x7fadf40c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfbdd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4153aa0 con 0x7fadf41432f0 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfbdd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf41f57f0 con 0x7fadf40cb6b0 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfbdd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4159790 con 0x7fadf40b4070 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfa36d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 0x7fadf40c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfa36d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 0x7fadf40c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49200/0 (socket says 172.21.15.70:49200) 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfa36d640 1 -- 172.21.15.70:0/3828291930 learned_addr learned my addr 172.21.15.70:0/3828291930 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:26.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadf936b640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf40cb6b0 0x7fadf40c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadf9b6c640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf40b4070 0x7fadf41f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfa36d640 1 -- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf40b4070 msgr2=0x7fadf41f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfa36d640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf40b4070 0x7fadf41f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfa36d640 1 -- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf40cb6b0 msgr2=0x7fadf40c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfa36d640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf40cb6b0 0x7fadf40c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.626+0000 7fadfa36d640 1 -- 172.21.15.70:0/3828291930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf00094d0 con 0x7fadf41432f0 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.627+0000 7fadfa36d640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 0x7fadf40c8010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fadec00c780 tx=0x7fadec00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.627+0000 7fadea7fc640 1 -- 172.21.15.70:0/3828291930 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec003d80 con 0x7fadf41432f0 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.627+0000 7fadea7fc640 1 -- 172.21.15.70:0/3828291930 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec00ce90 con 0x7fadf41432f0 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.627+0000 7fadea7fc640 1 -- 172.21.15.70:0/3828291930 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec0052b0 con 0x7fadf41432f0 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.627+0000 7fadfbdd0640 1 -- 172.21.15.70:0/3828291930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf41f5a90 con 0x7fadf41432f0 2023-10-08T05:36:26.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.627+0000 7fadfbdd0640 1 -- 172.21.15.70:0/3828291930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf41f6050 con 0x7fadf41432f0 2023-10-08T05:36:26.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.628+0000 7fadfbdd0640 1 -- 172.21.15.70:0/3828291930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf41f6270 con 0x7fadf41432f0 2023-10-08T05:36:26.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.629+0000 7fadea7fc640 1 -- 172.21.15.70:0/3828291930 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fadec005410 con 0x7fadf41432f0 2023-10-08T05:36:26.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.630+0000 7fadea7fc640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fadd806ae70 0x7fadd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:26.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.630+0000 7fadea7fc640 1 -- 172.21.15.70:0/3828291930 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fadec088de0 con 0x7fadf41432f0 2023-10-08T05:36:26.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.631+0000 7fadf9b6c640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fadd806ae70 0x7fadd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:26.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.632+0000 7fadf9b6c640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fadd806ae70 0x7fadd806d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fadf0036000 tx=0x7fadf003d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:26.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.634+0000 7fadea7fc640 1 -- 172.21.15.70:0/3828291930 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fadec053b90 con 0x7fadf41432f0 2023-10-08T05:36:26.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.822+0000 7fadfbdd0640 1 -- 172.21.15.70:0/3828291930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fadf415d940 con 0x7fadf41432f0 2023-10-08T05:36:26.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.823+0000 7fadea7fc640 1 -- 172.21.15.70:0/3828291930 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadec017020 con 0x7fadf41432f0 2023-10-08T05:36:26.824 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:36:26.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 -- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fadd806ae70 msgr2=0x7fadd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fadd806ae70 0x7fadd806d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fadf0036000 tx=0x7fadf003d040 comp rx=0 tx=0).stop 2023-10-08T05:36:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 -- 172.21.15.70:0/3828291930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 msgr2=0x7fadf40c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 0x7fadf40c8010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fadec00c780 tx=0x7fadec00cc40 comp rx=0 tx=0).stop 2023-10-08T05:36:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 -- 172.21.15.70:0/3828291930 shutdown_connections 2023-10-08T05:36:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fadd806ae70 0x7fadd806d330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf41432f0 0x7fadf40c8010 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf40cb6b0 0x7fadf40c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 --2- 172.21.15.70:0/3828291930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf40b4070 0x7fadf41f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 -- 172.21.15.70:0/3828291930 >> 172.21.15.70:0/3828291930 conn(0x7fadf4069d50 msgr2=0x7fadf4159ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 -- 172.21.15.70:0/3828291930 shutdown_connections 2023-10-08T05:36:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:26.826+0000 7fadfbdd0640 1 -- 172.21.15.70:0/3828291930 wait complete. 2023-10-08T05:36:26.843 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:36:27.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.062+0000 7f532557c640 1 Processor -- start 2023-10-08T05:36:27.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.062+0000 7f532557c640 1 -- start start 2023-10-08T05:36:27.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.064+0000 7f532557c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f53200c9cf0 0x7f532006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.064+0000 7f532557c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f532015aa00 0x7f53200b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.064+0000 7f532557c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532015b310 con 0x7f532006c3a0 2023-10-08T05:36:27.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.064+0000 7f532557c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532006e0a0 con 0x7f532015aa00 2023-10-08T05:36:27.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.064+0000 7f532557c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532006d290 con 0x7f53200c9cf0 2023-10-08T05:36:27.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.064+0000 7f531f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f532015aa00 0x7f53200b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.064+0000 7f531f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f532015aa00 0x7f53200b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49206/0 (socket says 172.21.15.70:49206) 2023-10-08T05:36:27.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.064+0000 7f531f7fe640 1 -- 172.21.15.70:0/2306308762 learned_addr learned my addr 172.21.15.70:0/2306308762 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:27.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.065+0000 7f531e7fc640 1 --2- 172.21.15.70:0/2306308762 >> v2:172.21.15.181:3300/0 conn(0x7f53200c9cf0 0x7f532006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.065+0000 7f531f7fe640 1 -- 172.21.15.70:0/2306308762 >> v2:172.21.15.181:3300/0 conn(0x7f53200c9cf0 msgr2=0x7f532006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:27.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.065+0000 7f531f7fe640 1 --2- 172.21.15.70:0/2306308762 >> v2:172.21.15.181:3300/0 conn(0x7f53200c9cf0 0x7f532006db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.065+0000 7f531f7fe640 1 -- 172.21.15.70:0/2306308762 >> v1:172.21.15.181:6789/0 conn(0x7f532006c3a0 legacy=0x7f532006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:27.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.065+0000 7f531f7fe640 1 -- 172.21.15.70:0/2306308762 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200c9860 con 0x7f532015aa00 2023-10-08T05:36:27.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.066+0000 7f531f7fe640 1 --2- 172.21.15.70:0/2306308762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 0x7f53200b2a90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f53200715c0 tx=0x7f531802e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8c7067881e2670 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:27.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.066+0000 7f531dffb640 1 -- 172.21.15.70:0/2306308762 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531802ec90 con 0x7f532015aa00 2023-10-08T05:36:27.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.067+0000 7f531dffb640 1 -- 172.21.15.70:0/2306308762 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531802edf0 con 0x7f532015aa00 2023-10-08T05:36:27.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.067+0000 7f531dffb640 1 -- 172.21.15.70:0/2306308762 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318036610 con 0x7f532015aa00 2023-10-08T05:36:27.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.067+0000 7f532557c640 1 -- 172.21.15.70:0/2306308762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 msgr2=0x7f53200b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.067+0000 7f532557c640 1 --2- 172.21.15.70:0/2306308762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 0x7f53200b2a90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f53200715c0 tx=0x7f531802e0f0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.067+0000 7f532557c640 1 -- 172.21.15.70:0/2306308762 shutdown_connections 2023-10-08T05:36:27.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.067+0000 7f532557c640 1 --2- 172.21.15.70:0/2306308762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 0x7f53200b2a90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.067+0000 7f532557c640 1 --2- 172.21.15.70:0/2306308762 >> v2:172.21.15.181:3300/0 conn(0x7f53200c9cf0 0x7f532006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.067+0000 7f532557c640 1 -- 172.21.15.70:0/2306308762 >> 172.21.15.70:0/2306308762 conn(0x7f532006a5d0 msgr2=0x7f53200b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:27.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.067+0000 7f532557c640 1 -- 172.21.15.70:0/2306308762 shutdown_connections 2023-10-08T05:36:27.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.068+0000 7f532557c640 1 -- 172.21.15.70:0/2306308762 wait complete. 2023-10-08T05:36:27.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.069+0000 7f532557c640 1 Processor -- start 2023-10-08T05:36:27.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.069+0000 7f532557c640 1 -- start start 2023-10-08T05:36:27.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.070+0000 7f532557c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f532006c3a0 0x7f53200c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.070+0000 7f532557c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53200c9cf0 0x7f53200c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.070+0000 7f532557c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 0x7f5320156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.070+0000 7f532557c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200c31c0 con 0x7f532015aa00 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.070+0000 7f532557c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200c4e30 con 0x7f532006c3a0 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.070+0000 7f532557c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53200c33b0 con 0x7f53200c9cf0 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.071+0000 7f531f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 0x7f5320156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.071+0000 7f531f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 0x7f5320156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49208/0 (socket says 172.21.15.70:49208) 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.071+0000 7f531f7fe640 1 -- 172.21.15.70:0/2192257440 learned_addr learned my addr 172.21.15.70:0/2192257440 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.071+0000 7f531f7fe640 1 -- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53200c9cf0 msgr2=0x7f53200c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.071+0000 7f531f7fe640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53200c9cf0 0x7f53200c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.071+0000 7f531f7fe640 1 -- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f532006c3a0 msgr2=0x7f53200c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.071+0000 7f531effd640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f532006c3a0 0x7f53200c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.071+0000 7f531f7fe640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f532006c3a0 0x7f53200c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.071+0000 7f531f7fe640 1 -- 172.21.15.70:0/2192257440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5318009b30 con 0x7f532015aa00 2023-10-08T05:36:27.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.071+0000 7f531f7fe640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 0x7f5320156b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5318009f40 tx=0x7f5318004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.072+0000 7f53077fe640 1 -- 172.21.15.70:0/2192257440 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531802ec90 con 0x7f532015aa00 2023-10-08T05:36:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.072+0000 7f53077fe640 1 -- 172.21.15.70:0/2192257440 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531803f470 con 0x7f532015aa00 2023-10-08T05:36:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.072+0000 7f53077fe640 1 -- 172.21.15.70:0/2192257440 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531803f800 con 0x7f532015aa00 2023-10-08T05:36:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.072+0000 7f532557c640 1 -- 172.21.15.70:0/2192257440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532015b550 con 0x7f532015aa00 2023-10-08T05:36:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.072+0000 7f532557c640 1 -- 172.21.15.70:0/2192257440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532015bb10 con 0x7f532015aa00 2023-10-08T05:36:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.073+0000 7f532557c640 1 -- 172.21.15.70:0/2192257440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532004e160 con 0x7f532015aa00 2023-10-08T05:36:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.074+0000 7f53077fe640 1 -- 172.21.15.70:0/2192257440 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f531803f960 con 0x7f532015aa00 2023-10-08T05:36:27.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.075+0000 7f53077fe640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f530c06ae70 0x7f530c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.075+0000 7f53077fe640 1 -- 172.21.15.70:0/2192257440 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f53180abfd0 con 0x7f532015aa00 2023-10-08T05:36:27.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.075+0000 7f531effd640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f530c06ae70 0x7f530c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.076+0000 7f531effd640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f530c06ae70 0x7f530c06d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5310003d90 tx=0x7f5310009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:27.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.079+0000 7f53077fe640 1 -- 172.21.15.70:0/2192257440 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5318076d80 con 0x7f532015aa00 2023-10-08T05:36:27.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.272+0000 7f532557c640 1 -- 172.21.15.70:0/2192257440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f532015b310 con 0x7f532015aa00 2023-10-08T05:36:27.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.272+0000 7f53077fe640 1 -- 172.21.15.70:0/2192257440 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f532015b310 con 0x7f532015aa00 2023-10-08T05:36:27.273 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:36:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 -- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f530c06ae70 msgr2=0x7f530c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f530c06ae70 0x7f530c06d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5310003d90 tx=0x7f5310009040 comp rx=0 tx=0).stop 2023-10-08T05:36:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 -- 172.21.15.70:0/2192257440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 msgr2=0x7f5320156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 0x7f5320156b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5318009f40 tx=0x7f5318004a40 comp rx=0 tx=0).stop 2023-10-08T05:36:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 -- 172.21.15.70:0/2192257440 shutdown_connections 2023-10-08T05:36:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f530c06ae70 0x7f530c06d330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f532015aa00 0x7f5320156b80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53200c9cf0 0x7f53200c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 --2- 172.21.15.70:0/2192257440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f532006c3a0 0x7f53200c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 -- 172.21.15.70:0/2192257440 >> 172.21.15.70:0/2192257440 conn(0x7f532006a5d0 msgr2=0x7f532006d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:27.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 -- 172.21.15.70:0/2192257440 shutdown_connections 2023-10-08T05:36:27.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.275+0000 7f532557c640 1 -- 172.21.15.70:0/2192257440 wait complete. 2023-10-08T05:36:27.293 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:36:27.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.501+0000 7f94ea939640 1 Processor -- start 2023-10-08T05:36:27.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.502+0000 7f94ea939640 1 -- start start 2023-10-08T05:36:27.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.503+0000 7f94ea939640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94e4160660 0x7f94e4070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.503+0000 7f94ea939640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94e40c9910 0x7f94e40caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.503+0000 7f94ea939640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94e40cb450 0x7f94e40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.503+0000 7f94ea939640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e406e150 con 0x7f94e40cb450 2023-10-08T05:36:27.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.503+0000 7f94ea939640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e40cb8c0 con 0x7f94e4160660 2023-10-08T05:36:27.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.503+0000 7f94ea939640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e406de70 con 0x7f94e40c9910 2023-10-08T05:36:27.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.504+0000 7f94e8ed6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94e40cb450 0x7f94e40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.504+0000 7f94e8ed6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94e40cb450 0x7f94e40b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49216/0 (socket says 172.21.15.70:49216) 2023-10-08T05:36:27.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.504+0000 7f94e8ed6640 1 -- 172.21.15.70:0/3910078572 learned_addr learned my addr 172.21.15.70:0/3910078572 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:27.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.504+0000 7f94db7fe640 1 --2- 172.21.15.70:0/3910078572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94e40c9910 0x7f94e40caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.504+0000 7f94e3fff640 1 --2- 172.21.15.70:0/3910078572 >> v2:172.21.15.181:3300/0 conn(0x7f94e4160660 0x7f94e4070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.504+0000 7f94e8ed6640 1 -- 172.21.15.70:0/3910078572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94e40c9910 msgr2=0x7f94e40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:27.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.504+0000 7f94e8ed6640 1 --2- 172.21.15.70:0/3910078572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94e40c9910 0x7f94e40caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.504+0000 7f94e8ed6640 1 -- 172.21.15.70:0/3910078572 >> v2:172.21.15.181:3300/0 conn(0x7f94e4160660 msgr2=0x7f94e4070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:27.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.504+0000 7f94e8ed6640 1 --2- 172.21.15.70:0/3910078572 >> v2:172.21.15.181:3300/0 conn(0x7f94e4160660 0x7f94e4070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.504+0000 7f94e8ed6640 1 -- 172.21.15.70:0/3910078572 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40c8e30 con 0x7f94e40cb450 2023-10-08T05:36:27.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.505+0000 7f94e8ed6640 1 --2- 172.21.15.70:0/3910078572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40cb450 0x7f94e40b1990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f94e40b0d80 tx=0x7f94dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f92703113bef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:27.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.506+0000 7f94e37fe640 1 -- 172.21.15.70:0/3910078572 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc02ca80 con 0x7f94e40cb450 2023-10-08T05:36:27.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.506+0000 7f94e37fe640 1 -- 172.21.15.70:0/3910078572 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc02cbe0 con 0x7f94e40cb450 2023-10-08T05:36:27.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.506+0000 7f94e37fe640 1 -- 172.21.15.70:0/3910078572 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0343f0 con 0x7f94e40cb450 2023-10-08T05:36:27.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.506+0000 7f94ea939640 1 -- 172.21.15.70:0/3910078572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40cb450 msgr2=0x7f94e40b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.506+0000 7f94ea939640 1 --2- 172.21.15.70:0/3910078572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40cb450 0x7f94e40b1990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f94e40b0d80 tx=0x7f94dc02bee0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.506+0000 7f94ea939640 1 -- 172.21.15.70:0/3910078572 shutdown_connections 2023-10-08T05:36:27.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.506+0000 7f94ea939640 1 --2- 172.21.15.70:0/3910078572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40cb450 0x7f94e40b1990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.506+0000 7f94ea939640 1 --2- 172.21.15.70:0/3910078572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94e40c9910 0x7f94e40caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.506+0000 7f94ea939640 1 --2- 172.21.15.70:0/3910078572 >> v2:172.21.15.181:3300/0 conn(0x7f94e4160660 0x7f94e4070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.506+0000 7f94ea939640 1 -- 172.21.15.70:0/3910078572 >> 172.21.15.70:0/3910078572 conn(0x7f94e406a5d0 msgr2=0x7f94e40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:27.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.507+0000 7f94ea939640 1 -- 172.21.15.70:0/3910078572 shutdown_connections 2023-10-08T05:36:27.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.507+0000 7f94ea939640 1 -- 172.21.15.70:0/3910078572 wait complete. 2023-10-08T05:36:27.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.508+0000 7f94ea939640 1 Processor -- start 2023-10-08T05:36:27.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.508+0000 7f94ea939640 1 -- start start 2023-10-08T05:36:27.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.509+0000 7f94ea939640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40c9910 0x7f94e40cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.509+0000 7f94ea939640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94e40cb450 0x7f94e40c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.509+0000 7f94ea939640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94e4160660 0x7f94e40c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.509+0000 7f94ea939640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40cdb90 con 0x7f94e40c9910 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.509+0000 7f94ea939640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40cde70 con 0x7f94e40cb450 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.509+0000 7f94ea939640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40ce150 con 0x7f94e4160660 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40c9910 0x7f94e40cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40c9910 0x7f94e40cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49232/0 (socket says 172.21.15.70:49232) 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94e3fff640 1 -- 172.21.15.70:0/4111825968 learned_addr learned my addr 172.21.15.70:0/4111825968 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94e8ed6640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94e4160660 0x7f94e40c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94db7fe640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94e40cb450 0x7f94e40c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94e3fff640 1 -- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94e4160660 msgr2=0x7f94e40c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94e3fff640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94e4160660 0x7f94e40c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94e3fff640 1 -- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94e40cb450 msgr2=0x7f94e40c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94e3fff640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94e40cb450 0x7f94e40c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94e3fff640 1 -- 172.21.15.70:0/4111825968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc009b30 con 0x7f94e40c9910 2023-10-08T05:36:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.510+0000 7f94e3fff640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40c9910 0x7f94e40cef70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f94d400e9b0 tx=0x7f94d400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.511+0000 7f94e17fa640 1 -- 172.21.15.70:0/4111825968 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4009670 con 0x7f94e40c9910 2023-10-08T05:36:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.511+0000 7f94e17fa640 1 -- 172.21.15.70:0/4111825968 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d400c9a0 con 0x7f94e40c9910 2023-10-08T05:36:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.511+0000 7f94e17fa640 1 -- 172.21.15.70:0/4111825968 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d400cd50 con 0x7f94e40c9910 2023-10-08T05:36:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.511+0000 7f94ea939640 1 -- 172.21.15.70:0/4111825968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e40ce430 con 0x7f94e40c9910 2023-10-08T05:36:27.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.511+0000 7f94ea939640 1 -- 172.21.15.70:0/4111825968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e40c4e70 con 0x7f94e40c9910 2023-10-08T05:36:27.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.512+0000 7f94ea939640 1 -- 172.21.15.70:0/4111825968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e40cf690 con 0x7f94e40c9910 2023-10-08T05:36:27.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.513+0000 7f94e17fa640 1 -- 172.21.15.70:0/4111825968 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f94d4013470 con 0x7f94e40c9910 2023-10-08T05:36:27.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.514+0000 7f94e17fa640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f94c406ae70 0x7f94c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.514+0000 7f94e17fa640 1 -- 172.21.15.70:0/4111825968 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f94d4089bb0 con 0x7f94e40c9910 2023-10-08T05:36:27.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.514+0000 7f94db7fe640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f94c406ae70 0x7f94c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.515+0000 7f94db7fe640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f94c406ae70 0x7f94c406d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94e40c7ac0 tx=0x7f94cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:27.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.518+0000 7f94e17fa640 1 -- 172.21.15.70:0/4111825968 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94d4054970 con 0x7f94e40c9910 2023-10-08T05:36:27.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.709+0000 7f94ea939640 1 -- 172.21.15.70:0/4111825968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94e404e160 con 0x7f94e40c9910 2023-10-08T05:36:27.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.710+0000 7f94e17fa640 1 -- 172.21.15.70:0/4111825968 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f94d40170a0 con 0x7f94e40c9910 2023-10-08T05:36:27.710 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:36:27.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.712+0000 7f94ea939640 1 -- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f94c406ae70 msgr2=0x7f94c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.712+0000 7f94ea939640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f94c406ae70 0x7f94c406d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f94e40c7ac0 tx=0x7f94cc008040 comp rx=0 tx=0).stop 2023-10-08T05:36:27.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.712+0000 7f94ea939640 1 -- 172.21.15.70:0/4111825968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40c9910 msgr2=0x7f94e40cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.712+0000 7f94ea939640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40c9910 0x7f94e40cef70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f94d400e9b0 tx=0x7f94d400ee70 comp rx=0 tx=0).stop 2023-10-08T05:36:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.713+0000 7f94ea939640 1 -- 172.21.15.70:0/4111825968 shutdown_connections 2023-10-08T05:36:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.713+0000 7f94ea939640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f94c406ae70 0x7f94c406d330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.713+0000 7f94ea939640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94e4160660 0x7f94e40c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.713+0000 7f94ea939640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94e40cb450 0x7f94e40c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.713+0000 7f94ea939640 1 --2- 172.21.15.70:0/4111825968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e40c9910 0x7f94e40cef70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.713+0000 7f94ea939640 1 -- 172.21.15.70:0/4111825968 >> 172.21.15.70:0/4111825968 conn(0x7f94e406a5d0 msgr2=0x7f94e415ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.713+0000 7f94ea939640 1 -- 172.21.15.70:0/4111825968 shutdown_connections 2023-10-08T05:36:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.713+0000 7f94ea939640 1 -- 172.21.15.70:0/4111825968 wait complete. 2023-10-08T05:36:27.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:36:27.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.952+0000 7fce615f4640 1 Processor -- start 2023-10-08T05:36:27.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.952+0000 7fce615f4640 1 -- start start 2023-10-08T05:36:27.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.954+0000 7fce615f4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce5c0cf2b0 0x7fce5c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.954+0000 7fce615f4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fce5c0cf690 0x7fce5c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.954+0000 7fce615f4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c0cd520 con 0x7fce5c0d1100 2023-10-08T05:36:27.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.954+0000 7fce615f4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5c0d1910 con 0x7fce5c0cf690 2023-10-08T05:36:27.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.954+0000 7fce615f4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c0ce080 con 0x7fce5c0cf2b0 2023-10-08T05:36:27.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.954+0000 7fce5b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fce5c0cf690 0x7fce5c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.954+0000 7fce5b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fce5c0cf690 0x7fce5c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49248/0 (socket says 172.21.15.70:49248) 2023-10-08T05:36:27.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.954+0000 7fce5b7fe640 1 -- 172.21.15.70:0/3497461441 learned_addr learned my addr 172.21.15.70:0/3497461441 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:27.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.954+0000 7fce5bfff640 1 --2- 172.21.15.70:0/3497461441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce5c0cf2b0 0x7fce5c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.955+0000 7fce5b7fe640 1 -- 172.21.15.70:0/3497461441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce5c0cf2b0 msgr2=0x7fce5c0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:27.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.955+0000 7fce5b7fe640 1 --2- 172.21.15.70:0/3497461441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce5c0cf2b0 0x7fce5c0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.955+0000 7fce5b7fe640 1 -- 172.21.15.70:0/3497461441 >> v1:172.21.15.70:6789/0 conn(0x7fce5c0d1100 legacy=0x7fce5c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.955+0000 7fce5b7fe640 1 -- 172.21.15.70:0/3497461441 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c0cd750 con 0x7fce5c0cf690 2023-10-08T05:36:27.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.956+0000 7fce5b7fe640 1 --2- 172.21.15.70:0/3497461441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 0x7fce5c0ce510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce44009870 tx=0x7fce4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646ec541b6f945f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:27.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.956+0000 7fce5affd640 1 -- 172.21.15.70:0/3497461441 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44031640 con 0x7fce5c0cf690 2023-10-08T05:36:27.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.957+0000 7fce5affd640 1 -- 172.21.15.70:0/3497461441 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce440317a0 con 0x7fce5c0cf690 2023-10-08T05:36:27.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.957+0000 7fce5affd640 1 -- 172.21.15.70:0/3497461441 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44031b30 con 0x7fce5c0cf690 2023-10-08T05:36:27.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.957+0000 7fce615f4640 1 -- 172.21.15.70:0/3497461441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 msgr2=0x7fce5c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.957+0000 7fce615f4640 1 --2- 172.21.15.70:0/3497461441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 0x7fce5c0ce510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fce44009870 tx=0x7fce4402b8f0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.957+0000 7fce615f4640 1 -- 172.21.15.70:0/3497461441 shutdown_connections 2023-10-08T05:36:27.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.957+0000 7fce615f4640 1 --2- 172.21.15.70:0/3497461441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 0x7fce5c0ce510 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.957+0000 7fce615f4640 1 --2- 172.21.15.70:0/3497461441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce5c0cf2b0 0x7fce5c0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.957+0000 7fce615f4640 1 -- 172.21.15.70:0/3497461441 >> 172.21.15.70:0/3497461441 conn(0x7fce5c09bc40 msgr2=0x7fce5c0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:27.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.957+0000 7fce615f4640 1 -- 172.21.15.70:0/3497461441 shutdown_connections 2023-10-08T05:36:27.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.957+0000 7fce615f4640 1 -- 172.21.15.70:0/3497461441 wait complete. 2023-10-08T05:36:27.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.959+0000 7fce615f4640 1 Processor -- start 2023-10-08T05:36:27.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.959+0000 7fce615f4640 1 -- start start 2023-10-08T05:36:27.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.960+0000 7fce615f4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c0cf2b0 0x7fce5c15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.960+0000 7fce615f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 0x7fce5c09e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.960+0000 7fce615f4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce5c0d1100 0x7fce5c09eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.960+0000 7fce615f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c15f5d0 con 0x7fce5c0cf690 2023-10-08T05:36:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.960+0000 7fce615f4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c15ccf0 con 0x7fce5c0cf2b0 2023-10-08T05:36:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.960+0000 7fce615f4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c15cfd0 con 0x7fce5c0d1100 2023-10-08T05:36:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.960+0000 7fce5b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 0x7fce5c09e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce5b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 0x7fce5c09e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49250/0 (socket says 172.21.15.70:49250) 2023-10-08T05:36:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce5b7fe640 1 -- 172.21.15.70:0/1411820958 learned_addr learned my addr 172.21.15.70:0/1411820958 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce5bfff640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c0cf2b0 0x7fce5c15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce5b7fe640 1 -- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce5c0d1100 msgr2=0x7fce5c09eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce5b7fe640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce5c0d1100 0x7fce5c09eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce5b7fe640 1 -- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c0cf2b0 msgr2=0x7fce5c15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce5b7fe640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c0cf2b0 0x7fce5c15c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce5b7fe640 1 -- 172.21.15.70:0/1411820958 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce440094d0 con 0x7fce5c0cf690 2023-10-08T05:36:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce5bfff640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c0cf2b0 0x7fce5c15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce5b7fe640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 0x7fce5c09e580 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce44000c00 tx=0x7fce440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.961+0000 7fce58ff9640 1 -- 172.21.15.70:0/1411820958 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4403ac60 con 0x7fce5c0cf690 2023-10-08T05:36:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.962+0000 7fce58ff9640 1 -- 172.21.15.70:0/1411820958 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4403adc0 con 0x7fce5c0cf690 2023-10-08T05:36:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.962+0000 7fce58ff9640 1 -- 172.21.15.70:0/1411820958 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce44044630 con 0x7fce5c0cf690 2023-10-08T05:36:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.962+0000 7fce615f4640 1 -- 172.21.15.70:0/1411820958 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c15d2b0 con 0x7fce5c0cf690 2023-10-08T05:36:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.962+0000 7fce615f4640 1 -- 172.21.15.70:0/1411820958 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c15d870 con 0x7fce5c0cf690 2023-10-08T05:36:27.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.963+0000 7fce615f4640 1 -- 172.21.15.70:0/1411820958 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c15da60 con 0x7fce5c0cf690 2023-10-08T05:36:27.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.964+0000 7fce58ff9640 1 -- 172.21.15.70:0/1411820958 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fce4402f050 con 0x7fce5c0cf690 2023-10-08T05:36:27.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.964+0000 7fce58ff9640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce3c06ae70 0x7fce3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:27.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.965+0000 7fce58ff9640 1 -- 172.21.15.70:0/1411820958 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fce440ab710 con 0x7fce5c0cf690 2023-10-08T05:36:27.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.965+0000 7fce5bfff640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce3c06ae70 0x7fce3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:27.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.966+0000 7fce5bfff640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce3c06ae70 0x7fce3c06d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce5c15f260 tx=0x7fce4c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:27.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:27.969+0000 7fce58ff9640 1 -- 172.21.15.70:0/1411820958 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce44076560 con 0x7fce5c0cf690 2023-10-08T05:36:28.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.155+0000 7fce615f4640 1 -- 172.21.15.70:0/1411820958 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fce5c074150 con 0x7fce5c0cf690 2023-10-08T05:36:28.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.156+0000 7fce58ff9640 1 -- 172.21.15.70:0/1411820958 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fce44031e60 con 0x7fce5c0cf690 2023-10-08T05:36:28.156 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:36:28.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.158+0000 7fce615f4640 1 -- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce3c06ae70 msgr2=0x7fce3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.158+0000 7fce615f4640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce3c06ae70 0x7fce3c06d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fce5c15f260 tx=0x7fce4c00b040 comp rx=0 tx=0).stop 2023-10-08T05:36:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.158+0000 7fce615f4640 1 -- 172.21.15.70:0/1411820958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 msgr2=0x7fce5c09e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.159+0000 7fce615f4640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 0x7fce5c09e580 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce44000c00 tx=0x7fce440046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.159+0000 7fce615f4640 1 -- 172.21.15.70:0/1411820958 shutdown_connections 2023-10-08T05:36:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.159+0000 7fce615f4640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce3c06ae70 0x7fce3c06d330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.159+0000 7fce615f4640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce5c0d1100 0x7fce5c09eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.159+0000 7fce615f4640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce5c0cf690 0x7fce5c09e580 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.159+0000 7fce615f4640 1 --2- 172.21.15.70:0/1411820958 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce5c0cf2b0 0x7fce5c15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.159+0000 7fce615f4640 1 -- 172.21.15.70:0/1411820958 >> 172.21.15.70:0/1411820958 conn(0x7fce5c09bc40 msgr2=0x7fce5c0cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.159+0000 7fce615f4640 1 -- 172.21.15.70:0/1411820958 shutdown_connections 2023-10-08T05:36:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.159+0000 7fce615f4640 1 -- 172.21.15.70:0/1411820958 wait complete. 2023-10-08T05:36:28.177 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:36:28.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.383+0000 7f52b1b2c640 1 Processor -- start 2023-10-08T05:36:28.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.384+0000 7f52b1b2c640 1 -- start start 2023-10-08T05:36:28.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.385+0000 7f52b1b2c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52ac0ae490 0x7f52ac0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.385+0000 7f52b1b2c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52ac0c9d50 0x7f52ac0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.385+0000 7f52b1b2c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ac0cb560 con 0x7f52ac0cd190 2023-10-08T05:36:28.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.385+0000 7f52b1b2c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac0ae300 con 0x7f52ac0c9d50 2023-10-08T05:36:28.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.385+0000 7f52b1b2c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0ae8f0 con 0x7f52ac0ae490 2023-10-08T05:36:28.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.386+0000 7f52abfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52ac0c9d50 0x7f52ac0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.386+0000 7f52abfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52ac0c9d50 0x7f52ac0ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:55538/0 (socket says 172.21.15.70:55538) 2023-10-08T05:36:28.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.386+0000 7f52aaffd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52ac0ae490 0x7f52ac0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.386+0000 7f52abfff640 1 -- 172.21.15.70:0/3112516635 learned_addr learned my addr 172.21.15.70:0/3112516635 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:28.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.386+0000 7f52abfff640 1 -- 172.21.15.70:0/3112516635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52ac0ae490 msgr2=0x7f52ac0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:28.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.386+0000 7f52abfff640 1 --2- 172.21.15.70:0/3112516635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52ac0ae490 0x7f52ac0ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.386+0000 7f52abfff640 1 -- 172.21.15.70:0/3112516635 >> v1:172.21.15.181:6789/0 conn(0x7f52ac0cd190 legacy=0x7f52ac0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.387+0000 7f52abfff640 1 -- 172.21.15.70:0/3112516635 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0cb840 con 0x7f52ac0c9d50 2023-10-08T05:36:28.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.388+0000 7f52abfff640 1 --2- 172.21.15.70:0/3112516635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ac0c9d50 0x7f52ac0ca1e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009e50 tx=0x7f52a402be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef8fdddfde238a9e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:28.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.388+0000 7f52aa7fc640 1 -- 172.21.15.70:0/3112516635 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a402c9e0 con 0x7f52ac0c9d50 2023-10-08T05:36:28.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.388+0000 7f52aa7fc640 1 -- 172.21.15.70:0/3112516635 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a402cb40 con 0x7f52ac0c9d50 2023-10-08T05:36:28.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.388+0000 7f52aa7fc640 1 -- 172.21.15.70:0/3112516635 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40343f0 con 0x7f52ac0c9d50 2023-10-08T05:36:28.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.389+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3112516635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ac0c9d50 msgr2=0x7f52ac0ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.389+0000 7f52b1b2c640 1 --2- 172.21.15.70:0/3112516635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ac0c9d50 0x7f52ac0ca1e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009e50 tx=0x7f52a402be40 comp rx=0 tx=0).stop 2023-10-08T05:36:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.389+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3112516635 shutdown_connections 2023-10-08T05:36:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.389+0000 7f52b1b2c640 1 --2- 172.21.15.70:0/3112516635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ac0c9d50 0x7f52ac0ca1e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.389+0000 7f52b1b2c640 1 --2- 172.21.15.70:0/3112516635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52ac0ae490 0x7f52ac0ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.389+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3112516635 >> 172.21.15.70:0/3112516635 conn(0x7f52ac0b0af0 msgr2=0x7f52ac0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.389+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3112516635 shutdown_connections 2023-10-08T05:36:28.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.389+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3112516635 wait complete. 2023-10-08T05:36:28.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.391+0000 7f52b1b2c640 1 Processor -- start 2023-10-08T05:36:28.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.391+0000 7f52b1b2c640 1 -- start start 2023-10-08T05:36:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52b1b2c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52ac0ae490 0x7f52ac14d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52b1b2c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ac0c9d50 0x7f52ac151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52b1b2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac0cd190 0x7f52ac0aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52b1b2c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac1516f0 con 0x7f52ac0cd190 2023-10-08T05:36:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52b1b2c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0c0ab0 con 0x7f52ac0c9d50 2023-10-08T05:36:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52b1b2c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac150b70 con 0x7f52ac0ae490 2023-10-08T05:36:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac0cd190 0x7f52ac0aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac0cd190 0x7f52ac0aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49266/0 (socket says 172.21.15.70:49266) 2023-10-08T05:36:28.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52abfff640 1 -- 172.21.15.70:0/3370754184 learned_addr learned my addr 172.21.15.70:0/3370754184 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52ab7fe640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52ac0ae490 0x7f52ac14d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52aaffd640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ac0c9d50 0x7f52ac151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.392+0000 7f52abfff640 1 -- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52ac0ae490 msgr2=0x7f52ac14d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.393+0000 7f52abfff640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52ac0ae490 0x7f52ac14d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.393+0000 7f52abfff640 1 -- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ac0c9d50 msgr2=0x7f52ac151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.393+0000 7f52abfff640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ac0c9d50 0x7f52ac151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.393+0000 7f52abfff640 1 -- 172.21.15.70:0/3370754184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a4009b30 con 0x7f52ac0cd190 2023-10-08T05:36:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.393+0000 7f52abfff640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac0cd190 0x7f52ac0aea30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52a4036040 tx=0x7f52a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.393+0000 7f52a3fff640 1 -- 172.21.15.70:0/3370754184 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a403e450 con 0x7f52ac0cd190 2023-10-08T05:36:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.393+0000 7f52a3fff640 1 -- 172.21.15.70:0/3370754184 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a403e5d0 con 0x7f52ac0cd190 2023-10-08T05:36:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.393+0000 7f52a3fff640 1 -- 172.21.15.70:0/3370754184 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a403e980 con 0x7f52ac0cd190 2023-10-08T05:36:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.393+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3370754184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac150e50 con 0x7f52ac0cd190 2023-10-08T05:36:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.393+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3370754184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac151410 con 0x7f52ac0cd190 2023-10-08T05:36:28.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.395+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3370754184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac04e160 con 0x7f52ac0cd190 2023-10-08T05:36:28.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.395+0000 7f52a3fff640 1 -- 172.21.15.70:0/3370754184 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f52a403eae0 con 0x7f52ac0cd190 2023-10-08T05:36:28.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.397+0000 7f52a3fff640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f528c06ae70 0x7f528c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.397+0000 7f52a3fff640 1 -- 172.21.15.70:0/3370754184 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f52a40ac9e0 con 0x7f52ac0cd190 2023-10-08T05:36:28.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.397+0000 7f52ab7fe640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f528c06ae70 0x7f528c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.398+0000 7f52ab7fe640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f528c06ae70 0x7f528c06d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f529c000f60 tx=0x7f529c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:28.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.401+0000 7f52a3fff640 1 -- 172.21.15.70:0/3370754184 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52a40778c0 con 0x7f52ac0cd190 2023-10-08T05:36:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.607+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3370754184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f52ac0c2880 con 0x7f52ac0cd190 2023-10-08T05:36:28.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.607+0000 7f52a3fff640 1 -- 172.21.15.70:0/3370754184 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f52a4033e80 con 0x7f52ac0cd190 2023-10-08T05:36:28.608 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-10-08T05:36:28.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f528c06ae70 msgr2=0x7f528c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f528c06ae70 0x7f528c06d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f529c000f60 tx=0x7f529c009040 comp rx=0 tx=0).stop 2023-10-08T05:36:28.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3370754184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac0cd190 msgr2=0x7f52ac0aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac0cd190 0x7f52ac0aea30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52a4036040 tx=0x7f52a4004a40 comp rx=0 tx=0).stop 2023-10-08T05:36:28.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3370754184 shutdown_connections 2023-10-08T05:36:28.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f528c06ae70 0x7f528c06d330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52ac0cd190 0x7f52ac0aea30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52ac0c9d50 0x7f52ac151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 --2- 172.21.15.70:0/3370754184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52ac0ae490 0x7f52ac14d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3370754184 >> 172.21.15.70:0/3370754184 conn(0x7f52ac0b0af0 msgr2=0x7f52ac14f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:28.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3370754184 shutdown_connections 2023-10-08T05:36:28.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.610+0000 7f52b1b2c640 1 -- 172.21.15.70:0/3370754184 wait complete. 2023-10-08T05:36:28.629 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:36:28.630 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:36:28.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.897+0000 7efddc656640 1 Processor -- start 2023-10-08T05:36:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.898+0000 7efddc656640 1 -- start start 2023-10-08T05:36:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.899+0000 7efddc656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd406d4c0 0x7efdd4162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.899+0000 7efddc656640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdd40cb150 0x7efdd40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.899+0000 7efddc656640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd40ae140 con 0x7efdd40add60 2023-10-08T05:36:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.899+0000 7efddc656640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd4161f10 con 0x7efdd40cb150 2023-10-08T05:36:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.899+0000 7efddc656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd40cbb20 con 0x7efdd406d4c0 2023-10-08T05:36:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.900+0000 7efddabf3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdd40cb150 0x7efdd40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.900+0000 7efdd9bf1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd406d4c0 0x7efdd4162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.900+0000 7efddabf3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdd40cb150 0x7efdd40cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:55562/0 (socket says 172.21.15.70:55562) 2023-10-08T05:36:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.900+0000 7efddabf3640 1 -- 172.21.15.70:0/2407410045 learned_addr learned my addr 172.21.15.70:0/2407410045 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.900+0000 7efddabf3640 1 -- 172.21.15.70:0/2407410045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd406d4c0 msgr2=0x7efdd4162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.900+0000 7efddabf3640 1 --2- 172.21.15.70:0/2407410045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd406d4c0 0x7efdd4162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.900+0000 7efddabf3640 1 -- 172.21.15.70:0/2407410045 >> v1:172.21.15.181:6789/0 conn(0x7efdd40add60 legacy=0x7efdd40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.900+0000 7efddabf3640 1 -- 172.21.15.70:0/2407410045 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd40ccf20 con 0x7efdd40cb150 2023-10-08T05:36:28.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.902+0000 7efddabf3640 1 --2- 172.21.15.70:0/2407410045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40cb150 0x7efdd40cb5e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009870 tx=0x7efdcc02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4395718002489aec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:28.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.902+0000 7efdd93f0640 1 -- 172.21.15.70:0/2407410045 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc031640 con 0x7efdd40cb150 2023-10-08T05:36:28.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.902+0000 7efdd93f0640 1 -- 172.21.15.70:0/2407410045 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc0317a0 con 0x7efdd40cb150 2023-10-08T05:36:28.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.902+0000 7efdd93f0640 1 -- 172.21.15.70:0/2407410045 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc031b30 con 0x7efdd40cb150 2023-10-08T05:36:28.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.903+0000 7efddc656640 1 -- 172.21.15.70:0/2407410045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40cb150 msgr2=0x7efdd40cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.903+0000 7efddc656640 1 --2- 172.21.15.70:0/2407410045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40cb150 0x7efdd40cb5e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009870 tx=0x7efdcc02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.903+0000 7efddc656640 1 -- 172.21.15.70:0/2407410045 shutdown_connections 2023-10-08T05:36:28.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.903+0000 7efddc656640 1 --2- 172.21.15.70:0/2407410045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40cb150 0x7efdd40cb5e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.903+0000 7efddc656640 1 --2- 172.21.15.70:0/2407410045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd406d4c0 0x7efdd4162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.903+0000 7efddc656640 1 -- 172.21.15.70:0/2407410045 >> 172.21.15.70:0/2407410045 conn(0x7efdd406a380 msgr2=0x7efdd40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:28.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.903+0000 7efddc656640 1 -- 172.21.15.70:0/2407410045 shutdown_connections 2023-10-08T05:36:28.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.903+0000 7efddc656640 1 -- 172.21.15.70:0/2407410045 wait complete. 2023-10-08T05:36:28.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.905+0000 7efddc656640 1 Processor -- start 2023-10-08T05:36:28.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.905+0000 7efddc656640 1 -- start start 2023-10-08T05:36:28.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.906+0000 7efddc656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd406d4c0 0x7efdd41535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.906+0000 7efddc656640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40add60 0x7efdd40b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.906+0000 7efddc656640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdd40cb150 0x7efdd415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.906+0000 7efddc656640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd40b4040 con 0x7efdd40cb150 2023-10-08T05:36:28.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.906+0000 7efddc656640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd40b3a80 con 0x7efdd40add60 2023-10-08T05:36:28.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.906+0000 7efddc656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd40b3d60 con 0x7efdd406d4c0 2023-10-08T05:36:28.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.906+0000 7efddabf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdd40cb150 0x7efdd415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.907+0000 7efddabf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdd40cb150 0x7efdd415b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49274/0 (socket says 172.21.15.70:49274) 2023-10-08T05:36:28.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.907+0000 7efddabf3640 1 -- 172.21.15.70:0/2521473319 learned_addr learned my addr 172.21.15.70:0/2521473319 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:28.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.907+0000 7efdda3f2640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd406d4c0 0x7efdd41535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.907+0000 7efdd9bf1640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40add60 0x7efdd40b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.907+0000 7efddabf3640 1 -- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd406d4c0 msgr2=0x7efdd41535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.907+0000 7efddabf3640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd406d4c0 0x7efdd41535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.907+0000 7efddabf3640 1 -- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40add60 msgr2=0x7efdd40b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:28.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.907+0000 7efddabf3640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40add60 0x7efdd40b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:28.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.907+0000 7efddabf3640 1 -- 172.21.15.70:0/2521473319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc0094d0 con 0x7efdd40cb150 2023-10-08T05:36:28.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.907+0000 7efddabf3640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdd40cb150 0x7efdd415b260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009870 tx=0x7efdcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:28.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.908+0000 7efdd2ffd640 1 -- 172.21.15.70:0/2521473319 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc03a930 con 0x7efdd40cb150 2023-10-08T05:36:28.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.908+0000 7efdd2ffd640 1 -- 172.21.15.70:0/2521473319 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc03aab0 con 0x7efdd40cb150 2023-10-08T05:36:28.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.908+0000 7efdd2ffd640 1 -- 172.21.15.70:0/2521473319 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc03ae60 con 0x7efdd40cb150 2023-10-08T05:36:28.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.908+0000 7efddc656640 1 -- 172.21.15.70:0/2521473319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd415e5c0 con 0x7efdd40cb150 2023-10-08T05:36:28.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.908+0000 7efddc656640 1 -- 172.21.15.70:0/2521473319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd41f6d30 con 0x7efdd40cb150 2023-10-08T05:36:28.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.909+0000 7efddc656640 1 -- 172.21.15.70:0/2521473319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd415eae0 con 0x7efdd40cb150 2023-10-08T05:36:28.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.910+0000 7efdd2ffd640 1 -- 172.21.15.70:0/2521473319 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7efdcc02f050 con 0x7efdd40cb150 2023-10-08T05:36:28.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.911+0000 7efdd2ffd640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7efdb806ae70 0x7efdb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:28.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.911+0000 7efdd2ffd640 1 -- 172.21.15.70:0/2521473319 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efdcc0ab880 con 0x7efdd40cb150 2023-10-08T05:36:28.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.912+0000 7efdda3f2640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7efdb806ae70 0x7efdb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:28.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.913+0000 7efdda3f2640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7efdb806ae70 0x7efdb806d330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efdc8001f10 tx=0x7efdc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:28.916+0000 7efdd2ffd640 1 -- 172.21.15.70:0/2521473319 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efdd415eae0 con 0x7efdd40cb150 2023-10-08T05:36:29.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.105+0000 7efddc656640 1 -- 172.21.15.70:0/2521473319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdd415eae0 con 0x7efdd40cb150 2023-10-08T05:36:29.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.109+0000 7efdd2ffd640 1 -- 172.21.15.70:0/2521473319 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7efdd415eae0 con 0x7efdd40cb150 2023-10-08T05:36:29.110 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:29.110 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.111+0000 7efddc656640 1 -- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7efdb806ae70 msgr2=0x7efdb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.111+0000 7efddc656640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7efdb806ae70 0x7efdb806d330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efdc8001f10 tx=0x7efdc8009040 comp rx=0 tx=0).stop 2023-10-08T05:36:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.111+0000 7efddc656640 1 -- 172.21.15.70:0/2521473319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdd40cb150 msgr2=0x7efdd415b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.111+0000 7efddc656640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdd40cb150 0x7efdd415b260 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009870 tx=0x7efdcc0046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.111+0000 7efddc656640 1 -- 172.21.15.70:0/2521473319 shutdown_connections 2023-10-08T05:36:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.111+0000 7efddc656640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7efdb806ae70 0x7efdb806d330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.111+0000 7efddc656640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdd40cb150 0x7efdd415b260 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.111+0000 7efddc656640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdd40add60 0x7efdd40b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.111+0000 7efddc656640 1 --2- 172.21.15.70:0/2521473319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdd406d4c0 0x7efdd41535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.111+0000 7efddc656640 1 -- 172.21.15.70:0/2521473319 >> 172.21.15.70:0/2521473319 conn(0x7efdd406a380 msgr2=0x7efdd41640d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.112+0000 7efddc656640 1 -- 172.21.15.70:0/2521473319 shutdown_connections 2023-10-08T05:36:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.112+0000 7efddc656640 1 -- 172.21.15.70:0/2521473319 wait complete. 2023-10-08T05:36:29.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:36:29.132 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:36:29.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.343+0000 7fb177821640 1 Processor -- start 2023-10-08T05:36:29.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.343+0000 7fb177821640 1 -- start start 2023-10-08T05:36:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.345+0000 7fb177821640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 0x7fb1700b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.345+0000 7fb177821640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb1700c82b0 0x7fb1700c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.345+0000 7fb177821640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb17015bf30 con 0x7fb17006cd00 2023-10-08T05:36:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.345+0000 7fb177821640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb17015b320 con 0x7fb1700c82b0 2023-10-08T05:36:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.345+0000 7fb177821640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb17015b600 con 0x7fb17015aad0 2023-10-08T05:36:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.345+0000 7fb174dbc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb1700c82b0 0x7fb1700c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.345+0000 7fb175dbe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb17006cd00 0x7fb170070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58620/0 (socket says 172.21.15.70:58620) 2023-10-08T05:36:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.345+0000 7fb174dbc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb1700c82b0 0x7fb1700c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49288/0 (socket says 172.21.15.70:49288) 2023-10-08T05:36:29.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.345+0000 7fb175dbe640 1 -- 172.21.15.70:0/642537463 learned_addr learned my addr 172.21.15.70:0/642537463 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:29.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.346+0000 7fb1755bd640 1 --2- 172.21.15.70:0/642537463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 0x7fb1700b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.346+0000 7fb174dbc640 1 -- 172.21.15.70:0/642537463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 msgr2=0x7fb1700b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.346+0000 7fb174dbc640 1 --2- 172.21.15.70:0/642537463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 0x7fb1700b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.346+0000 7fb174dbc640 1 -- 172.21.15.70:0/642537463 >> v1:172.21.15.70:6789/0 conn(0x7fb17006cd00 legacy=0x7fb170070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.346+0000 7fb174dbc640 1 -- 172.21.15.70:0/642537463 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17015b8e0 con 0x7fb1700c82b0 2023-10-08T05:36:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.346+0000 7fb167fff640 1 -- 172.21.15.70:0/642537463 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131265676 0 0) 0x7fb17015bf30 con 0x7fb17006cd00 2023-10-08T05:36:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.347+0000 7fb174dbc640 1 --2- 172.21.15.70:0/642537463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 0x7fb1700c8710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1600097a0 tx=0x7fb16002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa47ee960819cc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.347+0000 7fb167fff640 1 -- 172.21.15.70:0/642537463 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160031830 con 0x7fb1700c82b0 2023-10-08T05:36:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.347+0000 7fb167fff640 1 -- 172.21.15.70:0/642537463 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb160031990 con 0x7fb1700c82b0 2023-10-08T05:36:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.347+0000 7fb167fff640 1 -- 172.21.15.70:0/642537463 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160031d20 con 0x7fb1700c82b0 2023-10-08T05:36:29.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.348+0000 7fb177821640 1 -- 172.21.15.70:0/642537463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 msgr2=0x7fb1700c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.348+0000 7fb177821640 1 --2- 172.21.15.70:0/642537463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 0x7fb1700c8710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb1600097a0 tx=0x7fb16002b8d0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.348+0000 7fb177821640 1 -- 172.21.15.70:0/642537463 shutdown_connections 2023-10-08T05:36:29.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.348+0000 7fb177821640 1 --2- 172.21.15.70:0/642537463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 0x7fb1700c8710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.348+0000 7fb177821640 1 --2- 172.21.15.70:0/642537463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 0x7fb1700b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.348+0000 7fb177821640 1 -- 172.21.15.70:0/642537463 >> 172.21.15.70:0/642537463 conn(0x7fb170069d50 msgr2=0x7fb17006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:29.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.348+0000 7fb177821640 1 -- 172.21.15.70:0/642537463 shutdown_connections 2023-10-08T05:36:29.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.348+0000 7fb177821640 1 -- 172.21.15.70:0/642537463 wait complete. 2023-10-08T05:36:29.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.350+0000 7fb177821640 1 Processor -- start 2023-10-08T05:36:29.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.350+0000 7fb177821640 1 -- start start 2023-10-08T05:36:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.351+0000 7fb177821640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17006cd00 0x7fb170153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.351+0000 7fb177821640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 0x7fb1701f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.351+0000 7fb177821640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 0x7fb170070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.351+0000 7fb177821640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170153e60 con 0x7fb1700c82b0 2023-10-08T05:36:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.351+0000 7fb177821640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170144170 con 0x7fb17006cd00 2023-10-08T05:36:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.351+0000 7fb177821640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb170153bc0 con 0x7fb17015aad0 2023-10-08T05:36:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.351+0000 7fb174dbc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 0x7fb1701f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.351+0000 7fb174dbc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 0x7fb1701f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49298/0 (socket says 172.21.15.70:49298) 2023-10-08T05:36:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.351+0000 7fb174dbc640 1 -- 172.21.15.70:0/1996354612 learned_addr learned my addr 172.21.15.70:0/1996354612 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:29.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.351+0000 7fb1755bd640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17006cd00 0x7fb170153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb175dbe640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 0x7fb170070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb174dbc640 1 -- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 msgr2=0x7fb170070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb174dbc640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 0x7fb170070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb174dbc640 1 -- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17006cd00 msgr2=0x7fb170153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb174dbc640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17006cd00 0x7fb170153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb174dbc640 1 -- 172.21.15.70:0/1996354612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb160009400 con 0x7fb1700c82b0 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb175dbe640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 0x7fb170070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb174dbc640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 0x7fb1701f5bd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb1600097a0 tx=0x7fb1600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb165ffb640 1 -- 172.21.15.70:0/1996354612 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160004230 con 0x7fb1700c82b0 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb165ffb640 1 -- 172.21.15.70:0/1996354612 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb160004390 con 0x7fb1700c82b0 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb165ffb640 1 -- 172.21.15.70:0/1996354612 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16003a880 con 0x7fb1700c82b0 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb177821640 1 -- 172.21.15.70:0/1996354612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb170144600 con 0x7fb1700c82b0 2023-10-08T05:36:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.352+0000 7fb177821640 1 -- 172.21.15.70:0/1996354612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb170143b20 con 0x7fb1700c82b0 2023-10-08T05:36:29.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.353+0000 7fb177821640 1 -- 172.21.15.70:0/1996354612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb170144300 con 0x7fb1700c82b0 2023-10-08T05:36:29.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.354+0000 7fb165ffb640 1 -- 172.21.15.70:0/1996354612 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb16003a9e0 con 0x7fb1700c82b0 2023-10-08T05:36:29.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.356+0000 7fb165ffb640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb15406aea0 0x7fb15406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.356+0000 7fb165ffb640 1 -- 172.21.15.70:0/1996354612 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb1600ac340 con 0x7fb1700c82b0 2023-10-08T05:36:29.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.356+0000 7fb1755bd640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb15406aea0 0x7fb15406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.358+0000 7fb1755bd640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb15406aea0 0x7fb15406d360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0099c0 tx=0x7fb16c003990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:29.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.360+0000 7fb165ffb640 1 -- 172.21.15.70:0/1996354612 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb160077220 con 0x7fb1700c82b0 2023-10-08T05:36:29.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.545+0000 7fb177821640 1 -- 172.21.15.70:0/1996354612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb17004e160 con 0x7fb1700c82b0 2023-10-08T05:36:29.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.546+0000 7fb165ffb640 1 -- 172.21.15.70:0/1996354612 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb16007a840 con 0x7fb1700c82b0 2023-10-08T05:36:29.548 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:36:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.548+0000 7fb177821640 1 -- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb15406aea0 msgr2=0x7fb15406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.548+0000 7fb177821640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb15406aea0 0x7fb15406d360 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0099c0 tx=0x7fb16c003990 comp rx=0 tx=0).stop 2023-10-08T05:36:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.548+0000 7fb177821640 1 -- 172.21.15.70:0/1996354612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 msgr2=0x7fb1701f5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.548+0000 7fb177821640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 0x7fb1701f5bd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb1600097a0 tx=0x7fb1600046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.549+0000 7fb177821640 1 -- 172.21.15.70:0/1996354612 shutdown_connections 2023-10-08T05:36:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.549+0000 7fb177821640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb15406aea0 0x7fb15406d360 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.549+0000 7fb177821640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb17015aad0 0x7fb170070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.549+0000 7fb177821640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1700c82b0 0x7fb1701f5bd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.549+0000 7fb177821640 1 --2- 172.21.15.70:0/1996354612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb17006cd00 0x7fb170153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.549+0000 7fb177821640 1 -- 172.21.15.70:0/1996354612 >> 172.21.15.70:0/1996354612 conn(0x7fb170069d50 msgr2=0x7fb17015ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.549+0000 7fb177821640 1 -- 172.21.15.70:0/1996354612 shutdown_connections 2023-10-08T05:36:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.549+0000 7fb177821640 1 -- 172.21.15.70:0/1996354612 wait complete. 2023-10-08T05:36:29.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:36:29.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.788+0000 7f07f7ac2640 1 Processor -- start 2023-10-08T05:36:29.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.788+0000 7f07f7ac2640 1 -- start start 2023-10-08T05:36:29.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.790+0000 7f07f7ac2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07f00afbe0 0x7f07f00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.790+0000 7f07f7ac2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f07f00cb130 0x7f07f00cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.790+0000 7f07f7ac2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f00cbad0 con 0x7f07f006dfc0 2023-10-08T05:36:29.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.790+0000 7f07f7ac2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f00b2620 con 0x7f07f00cb130 2023-10-08T05:36:29.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.790+0000 7f07f7ac2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f00b2060 con 0x7f07f00afbe0 2023-10-08T05:36:29.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.790+0000 7f07f505d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f07f006dfc0 0x7f07f0070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58636/0 (socket says 172.21.15.70:58636) 2023-10-08T05:36:29.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.790+0000 7f07f505d640 1 -- 172.21.15.70:0/328800822 learned_addr learned my addr 172.21.15.70:0/328800822 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:29.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.790+0000 7f07f605f640 1 --2- 172.21.15.70:0/328800822 >> v2:172.21.15.70:3300/0 conn(0x7f07f00cb130 0x7f07f00cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.791+0000 7f07f585e640 1 --2- 172.21.15.70:0/328800822 >> v2:172.21.15.181:3300/0 conn(0x7f07f00afbe0 0x7f07f00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.791+0000 7f07f485c640 1 -- 172.21.15.70:0/328800822 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113421344 0 0) 0x7f07f00cbad0 con 0x7f07f006dfc0 2023-10-08T05:36:29.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.791+0000 7f07f485c640 1 -- 172.21.15.70:0/328800822 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07dc003520 con 0x7f07f006dfc0 2023-10-08T05:36:29.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.791+0000 7f07f605f640 1 -- 172.21.15.70:0/328800822 >> v2:172.21.15.181:3300/0 conn(0x7f07f00afbe0 msgr2=0x7f07f00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:29.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.791+0000 7f07f605f640 1 --2- 172.21.15.70:0/328800822 >> v2:172.21.15.181:3300/0 conn(0x7f07f00afbe0 0x7f07f00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.791+0000 7f07f605f640 1 -- 172.21.15.70:0/328800822 >> v1:172.21.15.70:6789/0 conn(0x7f07f006dfc0 legacy=0x7f07f0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.791+0000 7f07f605f640 1 -- 172.21.15.70:0/328800822 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f00b2340 con 0x7f07f00cb130 2023-10-08T05:36:29.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.792+0000 7f07f605f640 1 --2- 172.21.15.70:0/328800822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00cb130 0x7f07f00cb590 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f07f006e640 tx=0x7f07e802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868f0546950cef72 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:29.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.793+0000 7f07f485c640 1 -- 172.21.15.70:0/328800822 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e802cc70 con 0x7f07f00cb130 2023-10-08T05:36:29.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.793+0000 7f07f485c640 1 -- 172.21.15.70:0/328800822 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e802cdd0 con 0x7f07f00cb130 2023-10-08T05:36:29.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.793+0000 7f07f485c640 1 -- 172.21.15.70:0/328800822 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8034640 con 0x7f07f00cb130 2023-10-08T05:36:29.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.793+0000 7f07f7ac2640 1 -- 172.21.15.70:0/328800822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00cb130 msgr2=0x7f07f00cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.793+0000 7f07f7ac2640 1 --2- 172.21.15.70:0/328800822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00cb130 0x7f07f00cb590 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f07f006e640 tx=0x7f07e802bec0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.793+0000 7f07f7ac2640 1 -- 172.21.15.70:0/328800822 shutdown_connections 2023-10-08T05:36:29.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.793+0000 7f07f7ac2640 1 --2- 172.21.15.70:0/328800822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00cb130 0x7f07f00cb590 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.793+0000 7f07f7ac2640 1 --2- 172.21.15.70:0/328800822 >> v2:172.21.15.181:3300/0 conn(0x7f07f00afbe0 0x7f07f00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.793+0000 7f07f7ac2640 1 -- 172.21.15.70:0/328800822 >> 172.21.15.70:0/328800822 conn(0x7f07f006a5d0 msgr2=0x7f07f00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:29.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.794+0000 7f07f7ac2640 1 -- 172.21.15.70:0/328800822 shutdown_connections 2023-10-08T05:36:29.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.794+0000 7f07f7ac2640 1 -- 172.21.15.70:0/328800822 wait complete. 2023-10-08T05:36:29.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.795+0000 7f07f7ac2640 1 Processor -- start 2023-10-08T05:36:29.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.795+0000 7f07f7ac2640 1 -- start start 2023-10-08T05:36:29.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.796+0000 7f07f7ac2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07f006dfc0 0x7f07f00c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.796+0000 7f07f7ac2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00afbe0 0x7f07f014fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.796+0000 7f07f7ac2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f00cb130 0x7f07f015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.796+0000 7f07f7ac2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00c3fe0 con 0x7f07f00afbe0 2023-10-08T05:36:29.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.796+0000 7f07f7ac2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00c42c0 con 0x7f07f00cb130 2023-10-08T05:36:29.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.796+0000 7f07f7ac2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00c45a0 con 0x7f07f006dfc0 2023-10-08T05:36:29.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f505d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00afbe0 0x7f07f014fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f505d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00afbe0 0x7f07f014fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49316/0 (socket says 172.21.15.70:49316) 2023-10-08T05:36:29.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f505d640 1 -- 172.21.15.70:0/3286199562 learned_addr learned my addr 172.21.15.70:0/3286199562 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:29.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f605f640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f00cb130 0x7f07f015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f585e640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07f006dfc0 0x7f07f00c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f505d640 1 -- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07f006dfc0 msgr2=0x7f07f00c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f505d640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07f006dfc0 0x7f07f00c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f505d640 1 -- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f00cb130 msgr2=0x7f07f015cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f505d640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f00cb130 0x7f07f015cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f505d640 1 -- 172.21.15.70:0/3286199562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8009a70 con 0x7f07f00afbe0 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.797+0000 7f07f505d640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00afbe0 0x7f07f014fc40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c400 tx=0x7f07e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.798+0000 7f07e67fc640 1 -- 172.21.15.70:0/3286199562 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0005ce0 con 0x7f07f00afbe0 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.798+0000 7f07e67fc640 1 -- 172.21.15.70:0/3286199562 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e0005e60 con 0x7f07f00afbe0 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.798+0000 7f07e67fc640 1 -- 172.21.15.70:0/3286199562 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e00072f0 con 0x7f07f00afbe0 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.798+0000 7f07f7ac2640 1 -- 172.21.15.70:0/3286199562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f00c4880 con 0x7f07f00afbe0 2023-10-08T05:36:29.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.798+0000 7f07f7ac2640 1 -- 172.21.15.70:0/3286199562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f00c3280 con 0x7f07f00afbe0 2023-10-08T05:36:29.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.799+0000 7f07f7ac2640 1 -- 172.21.15.70:0/3286199562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f004e160 con 0x7f07f00afbe0 2023-10-08T05:36:29.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.800+0000 7f07e67fc640 1 -- 172.21.15.70:0/3286199562 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f07e0007450 con 0x7f07f00afbe0 2023-10-08T05:36:29.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.801+0000 7f07e67fc640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f07d406aea0 0x7f07d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:29.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.801+0000 7f07e67fc640 1 -- 172.21.15.70:0/3286199562 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f07e0088f60 con 0x7f07f00afbe0 2023-10-08T05:36:29.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.802+0000 7f07f585e640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f07d406aea0 0x7f07d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:29.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.803+0000 7f07f585e640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f07d406aea0 0x7f07d406d360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07ec006fd0 tx=0x7f07ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:29.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.805+0000 7f07e67fc640 1 -- 172.21.15.70:0/3286199562 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07e0053d10 con 0x7f07f00afbe0 2023-10-08T05:36:29.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.996+0000 7f07f7ac2640 1 -- 172.21.15.70:0/3286199562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07f00c4120 con 0x7f07f00afbe0 2023-10-08T05:36:29.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.996+0000 7f07e67fc640 1 -- 172.21.15.70:0/3286199562 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07f00c4120 con 0x7f07f00afbe0 2023-10-08T05:36:29.998 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:36:30.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.999+0000 7f07f7ac2640 1 -- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f07d406aea0 msgr2=0x7f07d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.999+0000 7f07f7ac2640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f07d406aea0 0x7f07d406d360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07ec006fd0 tx=0x7f07ec008040 comp rx=0 tx=0).stop 2023-10-08T05:36:30.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.999+0000 7f07f7ac2640 1 -- 172.21.15.70:0/3286199562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00afbe0 msgr2=0x7f07f014fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.999+0000 7f07f7ac2640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00afbe0 0x7f07f014fc40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f07e000c400 tx=0x7f07e000c8c0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.999+0000 7f07f7ac2640 1 -- 172.21.15.70:0/3286199562 shutdown_connections 2023-10-08T05:36:30.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.999+0000 7f07f7ac2640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f07d406aea0 0x7f07d406d360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.999+0000 7f07f7ac2640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07f00cb130 0x7f07f015cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.999+0000 7f07f7ac2640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07f00afbe0 0x7f07f014fc40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.999+0000 7f07f7ac2640 1 --2- 172.21.15.70:0/3286199562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f07f006dfc0 0x7f07f00c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:29.999+0000 7f07f7ac2640 1 -- 172.21.15.70:0/3286199562 >> 172.21.15.70:0/3286199562 conn(0x7f07f006a5d0 msgr2=0x7f07f00cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:30.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.000+0000 7f07f7ac2640 1 -- 172.21.15.70:0/3286199562 shutdown_connections 2023-10-08T05:36:30.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.000+0000 7f07f7ac2640 1 -- 172.21.15.70:0/3286199562 wait complete. 2023-10-08T05:36:30.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:36:30.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.229+0000 7fc02cbd0640 1 Processor -- start 2023-10-08T05:36:30.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.229+0000 7fc02cbd0640 1 -- start start 2023-10-08T05:36:30.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.231+0000 7fc02cbd0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc02815ccd0 0x7fc0280ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.231+0000 7fc02cbd0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc02806bcb0 0x7fc028160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.231+0000 7fc02cbd0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0280ce9b0 con 0x7fc0281612e0 2023-10-08T05:36:30.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.231+0000 7fc02cbd0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0280cec90 con 0x7fc02806bcb0 2023-10-08T05:36:30.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.231+0000 7fc02cbd0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0280cef70 con 0x7fc02815ccd0 2023-10-08T05:36:30.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.231+0000 7fc01ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc02806bcb0 0x7fc028160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:30.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.231+0000 7fc01ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc02806bcb0 0x7fc028160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49322/0 (socket says 172.21.15.70:49322) 2023-10-08T05:36:30.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.231+0000 7fc01ffff640 1 -- 172.21.15.70:0/1819735177 learned_addr learned my addr 172.21.15.70:0/1819735177 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:30.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.232+0000 7fc0277fe640 1 --2- 172.21.15.70:0/1819735177 >> v2:172.21.15.181:3300/0 conn(0x7fc02815ccd0 0x7fc0280ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:30.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.232+0000 7fc01ffff640 1 -- 172.21.15.70:0/1819735177 >> v2:172.21.15.181:3300/0 conn(0x7fc02815ccd0 msgr2=0x7fc0280ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:30.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.232+0000 7fc01ffff640 1 --2- 172.21.15.70:0/1819735177 >> v2:172.21.15.181:3300/0 conn(0x7fc02815ccd0 0x7fc0280ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.232+0000 7fc01ffff640 1 -- 172.21.15.70:0/1819735177 >> v1:172.21.15.181:6789/0 conn(0x7fc0281612e0 legacy=0x7fc0280b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:30.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.232+0000 7fc01ffff640 1 -- 172.21.15.70:0/1819735177 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc028160880 con 0x7fc02806bcb0 2023-10-08T05:36:30.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.233+0000 7fc01ffff640 1 --2- 172.21.15.70:0/1819735177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc02806bcb0 0x7fc028160da0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc0280ce280 tx=0x7fc01002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ca5ec55d018c3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:30.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.233+0000 7fc026ffd640 1 -- 172.21.15.70:0/1819735177 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010031870 con 0x7fc02806bcb0 2023-10-08T05:36:30.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.234+0000 7fc026ffd640 1 -- 172.21.15.70:0/1819735177 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0100319d0 con 0x7fc02806bcb0 2023-10-08T05:36:30.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.234+0000 7fc026ffd640 1 -- 172.21.15.70:0/1819735177 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc010031d60 con 0x7fc02806bcb0 2023-10-08T05:36:30.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.234+0000 7fc02cbd0640 1 -- 172.21.15.70:0/1819735177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc02806bcb0 msgr2=0x7fc028160da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.234+0000 7fc02cbd0640 1 --2- 172.21.15.70:0/1819735177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc02806bcb0 0x7fc028160da0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc0280ce280 tx=0x7fc01002b910 comp rx=0 tx=0).stop 2023-10-08T05:36:30.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.234+0000 7fc02cbd0640 1 -- 172.21.15.70:0/1819735177 shutdown_connections 2023-10-08T05:36:30.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.234+0000 7fc02cbd0640 1 --2- 172.21.15.70:0/1819735177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc02806bcb0 0x7fc028160da0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.234+0000 7fc02cbd0640 1 --2- 172.21.15.70:0/1819735177 >> v2:172.21.15.181:3300/0 conn(0x7fc02815ccd0 0x7fc0280ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.234+0000 7fc02cbd0640 1 -- 172.21.15.70:0/1819735177 >> 172.21.15.70:0/1819735177 conn(0x7fc0280cd180 msgr2=0x7fc0280cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:30.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.234+0000 7fc02cbd0640 1 -- 172.21.15.70:0/1819735177 shutdown_connections 2023-10-08T05:36:30.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.234+0000 7fc02cbd0640 1 -- 172.21.15.70:0/1819735177 wait complete. 2023-10-08T05:36:30.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.237+0000 7fc02cbd0640 1 Processor -- start 2023-10-08T05:36:30.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.237+0000 7fc02cbd0640 1 -- start start 2023-10-08T05:36:30.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.238+0000 7fc02cbd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02806bcb0 0x7fc02815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.238+0000 7fc02cbd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc02815ccd0 0x7fc02815ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.238+0000 7fc02cbd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0281612e0 0x7fc0281f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.238+0000 7fc02cbd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02815ecb0 con 0x7fc0281612e0 2023-10-08T05:36:30.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.238+0000 7fc02cbd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0281f3240 con 0x7fc02815ccd0 2023-10-08T05:36:30.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.238+0000 7fc02cbd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0281f26c0 con 0x7fc02806bcb0 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.238+0000 7fc027fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0281612e0 0x7fc0281f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.238+0000 7fc027fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0281612e0 0x7fc0281f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49324/0 (socket says 172.21.15.70:49324) 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.238+0000 7fc027fff640 1 -- 172.21.15.70:0/3135121807 learned_addr learned my addr 172.21.15.70:0/3135121807 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc01ffff640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc02815ccd0 0x7fc02815ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc0277fe640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02806bcb0 0x7fc02815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc027fff640 1 -- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02806bcb0 msgr2=0x7fc02815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc027fff640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02806bcb0 0x7fc02815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc027fff640 1 -- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc02815ccd0 msgr2=0x7fc02815ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc027fff640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc02815ccd0 0x7fc02815ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc027fff640 1 -- 172.21.15.70:0/3135121807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100094d0 con 0x7fc0281612e0 2023-10-08T05:36:30.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc0277fe640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02806bcb0 0x7fc02815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:30.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc027fff640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0281612e0 0x7fc0281f5ad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc02000b970 tx=0x7fc02000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc024ff9640 1 -- 172.21.15.70:0/3135121807 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02000cb40 con 0x7fc0281612e0 2023-10-08T05:36:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc024ff9640 1 -- 172.21.15.70:0/3135121807 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02000cca0 con 0x7fc0281612e0 2023-10-08T05:36:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc024ff9640 1 -- 172.21.15.70:0/3135121807 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020014680 con 0x7fc0281612e0 2023-10-08T05:36:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.239+0000 7fc02cbd0640 1 -- 172.21.15.70:0/3135121807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0281f29a0 con 0x7fc0281612e0 2023-10-08T05:36:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.240+0000 7fc02cbd0640 1 -- 172.21.15.70:0/3135121807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0281f2f60 con 0x7fc0281612e0 2023-10-08T05:36:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.241+0000 7fc02cbd0640 1 -- 172.21.15.70:0/3135121807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02815ee20 con 0x7fc0281612e0 2023-10-08T05:36:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.242+0000 7fc024ff9640 1 -- 172.21.15.70:0/3135121807 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc02001d3b0 con 0x7fc0281612e0 2023-10-08T05:36:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.243+0000 7fc024ff9640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc00806ae70 0x7fc00806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.243+0000 7fc024ff9640 1 -- 172.21.15.70:0/3135121807 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc0200897d0 con 0x7fc0281612e0 2023-10-08T05:36:30.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.244+0000 7fc0277fe640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc00806ae70 0x7fc00806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:30.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.245+0000 7fc0277fe640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc00806ae70 0x7fc00806d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc018006fd0 tx=0x7fc018008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:30.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.248+0000 7fc024ff9640 1 -- 172.21.15.70:0/3135121807 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc020054580 con 0x7fc0281612e0 2023-10-08T05:36:30.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.435+0000 7fc02cbd0640 1 -- 172.21.15.70:0/3135121807 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc028074150 con 0x7fc0281612e0 2023-10-08T05:36:30.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.436+0000 7fc024ff9640 1 -- 172.21.15.70:0/3135121807 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc020057ba0 con 0x7fc0281612e0 2023-10-08T05:36:30.438 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:36:30.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 -- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc00806ae70 msgr2=0x7fc00806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc00806ae70 0x7fc00806d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc018006fd0 tx=0x7fc018008040 comp rx=0 tx=0).stop 2023-10-08T05:36:30.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 -- 172.21.15.70:0/3135121807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0281612e0 msgr2=0x7fc0281f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0281612e0 0x7fc0281f5ad0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc02000b970 tx=0x7fc02000be30 comp rx=0 tx=0).stop 2023-10-08T05:36:30.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 -- 172.21.15.70:0/3135121807 shutdown_connections 2023-10-08T05:36:30.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc00806ae70 0x7fc00806d330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0281612e0 0x7fc0281f5ad0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc02815ccd0 0x7fc02815ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 --2- 172.21.15.70:0/3135121807 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02806bcb0 0x7fc02815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 -- 172.21.15.70:0/3135121807 >> 172.21.15.70:0/3135121807 conn(0x7fc0280cd180 msgr2=0x7fc02815bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:30.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 -- 172.21.15.70:0/3135121807 shutdown_connections 2023-10-08T05:36:30.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.439+0000 7fc02cbd0640 1 -- 172.21.15.70:0/3135121807 wait complete. 2023-10-08T05:36:30.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:36:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.671+0000 7f58f5013640 1 Processor -- start 2023-10-08T05:36:30.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.671+0000 7f58f5013640 1 -- start start 2023-10-08T05:36:30.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.672+0000 7f58f5013640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f58f015a820 0x7f58f00aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58f5013640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f00ca680 0x7f58f00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58f5013640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f00cab60 con 0x7f58f015ac00 2023-10-08T05:36:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58f5013640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f00cb960 con 0x7f58f015a820 2023-10-08T05:36:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58f5013640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00cbdf0 con 0x7f58f00ca680 2023-10-08T05:36:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58effff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f58f015a820 0x7f58f00aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58effff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f58f015a820 0x7f58f00aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49340/0 (socket says 172.21.15.70:49340) 2023-10-08T05:36:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58effff640 1 -- 172.21.15.70:0/567069385 learned_addr learned my addr 172.21.15.70:0/567069385 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58effff640 1 -- 172.21.15.70:0/567069385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f00ca680 msgr2=0x7f58f00b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58effff640 1 --2- 172.21.15.70:0/567069385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f00ca680 0x7f58f00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58effff640 1 -- 172.21.15.70:0/567069385 >> v1:172.21.15.181:6789/0 conn(0x7f58f015ac00 legacy=0x7f58f00c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.673+0000 7f58effff640 1 -- 172.21.15.70:0/567069385 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f00cb030 con 0x7f58f015a820 2023-10-08T05:36:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.675+0000 7f58effff640 1 --2- 172.21.15.70:0/567069385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 0x7f58f00aff70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009870 tx=0x7f58e002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22bd37b6de71ebe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.675+0000 7f58eeffd640 1 -- 172.21.15.70:0/567069385 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00317c0 con 0x7f58f015a820 2023-10-08T05:36:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.675+0000 7f58eeffd640 1 -- 172.21.15.70:0/567069385 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e0031920 con 0x7f58f015a820 2023-10-08T05:36:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.675+0000 7f58eeffd640 1 -- 172.21.15.70:0/567069385 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0031cb0 con 0x7f58f015a820 2023-10-08T05:36:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.675+0000 7f58f5013640 1 -- 172.21.15.70:0/567069385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 msgr2=0x7f58f00aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.675+0000 7f58f5013640 1 --2- 172.21.15.70:0/567069385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 0x7f58f00aff70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009870 tx=0x7f58e002b860 comp rx=0 tx=0).stop 2023-10-08T05:36:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.676+0000 7f58f5013640 1 -- 172.21.15.70:0/567069385 shutdown_connections 2023-10-08T05:36:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.676+0000 7f58f5013640 1 --2- 172.21.15.70:0/567069385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f00ca680 0x7f58f00b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.676+0000 7f58f5013640 1 --2- 172.21.15.70:0/567069385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 0x7f58f00aff70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.676+0000 7f58f5013640 1 -- 172.21.15.70:0/567069385 >> 172.21.15.70:0/567069385 conn(0x7f58f006c050 msgr2=0x7f58f006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.676+0000 7f58f5013640 1 -- 172.21.15.70:0/567069385 shutdown_connections 2023-10-08T05:36:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.676+0000 7f58f5013640 1 -- 172.21.15.70:0/567069385 wait complete. 2023-10-08T05:36:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.677+0000 7f58f5013640 1 Processor -- start 2023-10-08T05:36:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.678+0000 7f58f5013640 1 -- start start 2023-10-08T05:36:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.678+0000 7f58f5013640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00ca680 0x7f58f01ee880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.678+0000 7f58f5013640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 0x7f58f014db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58f5013640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f015ac00 0x7f58f00af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58f5013640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f01eedc0 con 0x7f58f015a820 2023-10-08T05:36:30.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58f5013640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f014e050 con 0x7f58f00ca680 2023-10-08T05:36:30.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58f5013640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00aeb50 con 0x7f58f015ac00 2023-10-08T05:36:30.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 0x7f58f014db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:30.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58ef7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 0x7f58f014db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49348/0 (socket says 172.21.15.70:49348) 2023-10-08T05:36:30.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58ef7fe640 1 -- 172.21.15.70:0/2299154163 learned_addr learned my addr 172.21.15.70:0/2299154163 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:30.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58ef7fe640 1 -- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f015ac00 msgr2=0x7f58f00af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58ef7fe640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f015ac00 0x7f58f00af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58ef7fe640 1 -- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00ca680 msgr2=0x7f58f01ee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58ef7fe640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00ca680 0x7f58f01ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58ef7fe640 1 -- 172.21.15.70:0/2299154163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e00094d0 con 0x7f58f015a820 2023-10-08T05:36:30.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.679+0000 7f58ef7fe640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 0x7f58f014db10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f58e400b410 tx=0x7f58e400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:30.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.680+0000 7f58ecff9640 1 -- 172.21.15.70:0/2299154163 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4011830 con 0x7f58f015a820 2023-10-08T05:36:30.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.680+0000 7f58ecff9640 1 -- 172.21.15.70:0/2299154163 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e40119b0 con 0x7f58f015a820 2023-10-08T05:36:30.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.680+0000 7f58ecff9640 1 -- 172.21.15.70:0/2299154163 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4011d60 con 0x7f58f015a820 2023-10-08T05:36:30.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.680+0000 7f58f5013640 1 -- 172.21.15.70:0/2299154163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f00c8000 con 0x7f58f015a820 2023-10-08T05:36:30.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.680+0000 7f58f5013640 1 -- 172.21.15.70:0/2299154163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f00af0a0 con 0x7f58f015a820 2023-10-08T05:36:30.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.681+0000 7f58f5013640 1 -- 172.21.15.70:0/2299154163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f00af2c0 con 0x7f58f015a820 2023-10-08T05:36:30.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.682+0000 7f58ecff9640 1 -- 172.21.15.70:0/2299154163 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f58e401a400 con 0x7f58f015a820 2023-10-08T05:36:30.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.683+0000 7f58ecff9640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f58dc06ae70 0x7f58dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:30.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.683+0000 7f58ecff9640 1 -- 172.21.15.70:0/2299154163 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f58e4089a90 con 0x7f58f015a820 2023-10-08T05:36:30.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.683+0000 7f58effff640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f58dc06ae70 0x7f58dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:30.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.684+0000 7f58effff640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f58dc06ae70 0x7f58dc06d330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58e0036000 tx=0x7f58e0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:30.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.687+0000 7f58ecff9640 1 -- 172.21.15.70:0/2299154163 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58e4054840 con 0x7f58f015a820 2023-10-08T05:36:30.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.875+0000 7f58f5013640 1 -- 172.21.15.70:0/2299154163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f58f0074150 con 0x7f58f015a820 2023-10-08T05:36:30.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.876+0000 7f58ecff9640 1 -- 172.21.15.70:0/2299154163 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f58e4015020 con 0x7f58f015a820 2023-10-08T05:36:30.878 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:36:30.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 -- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f58dc06ae70 msgr2=0x7f58dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f58dc06ae70 0x7f58dc06d330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f58e0036000 tx=0x7f58e0030000 comp rx=0 tx=0).stop 2023-10-08T05:36:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 -- 172.21.15.70:0/2299154163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 msgr2=0x7f58f014db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 0x7f58f014db10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f58e400b410 tx=0x7f58e400b8d0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 -- 172.21.15.70:0/2299154163 shutdown_connections 2023-10-08T05:36:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f58dc06ae70 0x7f58dc06d330 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f58f015ac00 0x7f58f00af5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58f015a820 0x7f58f014db10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 --2- 172.21.15.70:0/2299154163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58f00ca680 0x7f58f01ee880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 -- 172.21.15.70:0/2299154163 >> 172.21.15.70:0/2299154163 conn(0x7f58f006c050 msgr2=0x7f58f0158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:30.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 -- 172.21.15.70:0/2299154163 shutdown_connections 2023-10-08T05:36:30.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:30.879+0000 7f58f5013640 1 -- 172.21.15.70:0/2299154163 wait complete. 2023-10-08T05:36:30.900 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:36:31.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.111+0000 7fc011c75640 1 Processor -- start 2023-10-08T05:36:31.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.111+0000 7fc011c75640 1 -- start start 2023-10-08T05:36:31.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.113+0000 7fc011c75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 0x7fc00c0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.113+0000 7fc011c75640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc00c0b1f20 0x7fc00c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.113+0000 7fc011c75640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00c06e1e0 con 0x7fc00c161690 2023-10-08T05:36:31.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.113+0000 7fc011c75640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00c0cbb10 con 0x7fc00c0b1f20 2023-10-08T05:36:31.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.113+0000 7fc011c75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c06cc50 con 0x7fc00c0cb150 2023-10-08T05:36:31.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.114+0000 7fc00bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc00c0b1f20 0x7fc00c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.114+0000 7fc003fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 0x7fc00c0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.114+0000 7fc00bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc00c0b1f20 0x7fc00c0b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:55644/0 (socket says 172.21.15.70:55644) 2023-10-08T05:36:31.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.114+0000 7fc00bfff640 1 -- 172.21.15.70:0/1213812875 learned_addr learned my addr 172.21.15.70:0/1213812875 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:31.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.115+0000 7fc003fff640 1 -- 172.21.15.70:0/1213812875 >> v2:172.21.15.181:3300/0 conn(0x7fc00c0b1f20 msgr2=0x7fc00c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:31.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.115+0000 7fc003fff640 1 --2- 172.21.15.70:0/1213812875 >> v2:172.21.15.181:3300/0 conn(0x7fc00c0b1f20 0x7fc00c0b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.115+0000 7fc003fff640 1 -- 172.21.15.70:0/1213812875 >> v1:172.21.15.181:6789/0 conn(0x7fc00c161690 legacy=0x7fc00c06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.115+0000 7fc003fff640 1 -- 172.21.15.70:0/1213812875 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c06cf30 con 0x7fc00c0cb150 2023-10-08T05:36:31.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.116+0000 7fc003fff640 1 --2- 172.21.15.70:0/1213812875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 0x7fc00c0cb5d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbff40097a0 tx=0x7fbff402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26947e67a39c2b4a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:31.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc00affd640 1 -- 172.21.15.70:0/1213812875 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff402d040 con 0x7fc00c0cb150 2023-10-08T05:36:31.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc00affd640 1 -- 172.21.15.70:0/1213812875 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4031810 con 0x7fc00c0cb150 2023-10-08T05:36:31.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc00affd640 1 -- 172.21.15.70:0/1213812875 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4031ba0 con 0x7fc00c0cb150 2023-10-08T05:36:31.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc011c75640 1 -- 172.21.15.70:0/1213812875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 msgr2=0x7fc00c0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc011c75640 1 --2- 172.21.15.70:0/1213812875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 0x7fc00c0cb5d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbff40097a0 tx=0x7fbff402b8b0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc011c75640 1 -- 172.21.15.70:0/1213812875 shutdown_connections 2023-10-08T05:36:31.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc011c75640 1 --2- 172.21.15.70:0/1213812875 >> v2:172.21.15.181:3300/0 conn(0x7fc00c0b1f20 0x7fc00c0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc011c75640 1 --2- 172.21.15.70:0/1213812875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 0x7fc00c0cb5d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc011c75640 1 -- 172.21.15.70:0/1213812875 >> 172.21.15.70:0/1213812875 conn(0x7fc00c06a5d0 msgr2=0x7fc00c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:31.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc011c75640 1 -- 172.21.15.70:0/1213812875 shutdown_connections 2023-10-08T05:36:31.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.117+0000 7fc011c75640 1 -- 172.21.15.70:0/1213812875 wait complete. 2023-10-08T05:36:31.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.119+0000 7fc011c75640 1 Processor -- start 2023-10-08T05:36:31.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.119+0000 7fc011c75640 1 -- start start 2023-10-08T05:36:31.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.120+0000 7fc011c75640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc00c0b1f20 0x7fc00c162690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.120+0000 7fc011c75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 0x7fc00c162bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.120+0000 7fc011c75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00c161690 0x7fc00c0c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.120+0000 7fc011c75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1631e0 con 0x7fc00c161690 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.120+0000 7fc011c75640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c0c3530 con 0x7fc00c0b1f20 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.120+0000 7fc011c75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c0c3810 con 0x7fc00c0cb150 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00c161690 0x7fc00c0c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00c161690 0x7fc00c0c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49352/0 (socket says 172.21.15.70:49352) 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc00bfff640 1 -- 172.21.15.70:0/2767318453 learned_addr learned my addr 172.21.15.70:0/2767318453 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc00b7fe640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc00c0b1f20 0x7fc00c162690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc003fff640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 0x7fc00c162bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc00bfff640 1 -- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 msgr2=0x7fc00c162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc00bfff640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 0x7fc00c162bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc00bfff640 1 -- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc00c0b1f20 msgr2=0x7fc00c162690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc00bfff640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc00c0b1f20 0x7fc00c162690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc00bfff640 1 -- 172.21.15.70:0/2767318453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4009400 con 0x7fc00c161690 2023-10-08T05:36:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.121+0000 7fc00bfff640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00c161690 0x7fc00c0c7820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc00400efc0 tx=0x7fc00400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.122+0000 7fc008ff9640 1 -- 172.21.15.70:0/2767318453 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00400cd20 con 0x7fc00c161690 2023-10-08T05:36:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.122+0000 7fc008ff9640 1 -- 172.21.15.70:0/2767318453 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00400f040 con 0x7fc00c161690 2023-10-08T05:36:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.122+0000 7fc008ff9640 1 -- 172.21.15.70:0/2767318453 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004003bd0 con 0x7fc00c161690 2023-10-08T05:36:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.122+0000 7fc011c75640 1 -- 172.21.15.70:0/2767318453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00c0c3af0 con 0x7fc00c161690 2023-10-08T05:36:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.122+0000 7fc011c75640 1 -- 172.21.15.70:0/2767318453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c0c3040 con 0x7fc00c161690 2023-10-08T05:36:31.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.123+0000 7fc011c75640 1 -- 172.21.15.70:0/2767318453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00c0c4010 con 0x7fc00c161690 2023-10-08T05:36:31.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.124+0000 7fc008ff9640 1 -- 172.21.15.70:0/2767318453 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc004003d30 con 0x7fc00c161690 2023-10-08T05:36:31.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.125+0000 7fc008ff9640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbfec06aea0 0x7fbfec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.125+0000 7fc008ff9640 1 -- 172.21.15.70:0/2767318453 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc004089f40 con 0x7fc00c161690 2023-10-08T05:36:31.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.126+0000 7fc00b7fe640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbfec06aea0 0x7fbfec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.127+0000 7fc00b7fe640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbfec06aea0 0x7fbfec06d360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbffc007fd0 tx=0x7fbffc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:31.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.130+0000 7fc008ff9640 1 -- 172.21.15.70:0/2767318453 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc00c0c4010 con 0x7fc00c161690 2023-10-08T05:36:31.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.338+0000 7fc011c75640 1 -- 172.21.15.70:0/2767318453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc00c0c4010 con 0x7fc00c161690 2023-10-08T05:36:31.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.339+0000 7fc008ff9640 1 -- 172.21.15.70:0/2767318453 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc00c0c4010 con 0x7fc00c161690 2023-10-08T05:36:31.340 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-10-08T05:36:31.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 -- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbfec06aea0 msgr2=0x7fbfec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbfec06aea0 0x7fbfec06d360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbffc007fd0 tx=0x7fbffc009040 comp rx=0 tx=0).stop 2023-10-08T05:36:31.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 -- 172.21.15.70:0/2767318453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00c161690 msgr2=0x7fc00c0c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00c161690 0x7fc00c0c7820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc00400efc0 tx=0x7fc00400c490 comp rx=0 tx=0).stop 2023-10-08T05:36:31.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 -- 172.21.15.70:0/2767318453 shutdown_connections 2023-10-08T05:36:31.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbfec06aea0 0x7fbfec06d360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00c161690 0x7fc00c0c7820 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00c0cb150 0x7fc00c162bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 --2- 172.21.15.70:0/2767318453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc00c0b1f20 0x7fc00c162690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 -- 172.21.15.70:0/2767318453 >> 172.21.15.70:0/2767318453 conn(0x7fc00c06a5d0 msgr2=0x7fc00c0c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:31.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 -- 172.21.15.70:0/2767318453 shutdown_connections 2023-10-08T05:36:31.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.342+0000 7fc011c75640 1 -- 172.21.15.70:0/2767318453 wait complete. 2023-10-08T05:36:31.362 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-10-08T05:36:31.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:31.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-08T05:36:31.446 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi070.stderr:ceph-fuse[92651]: fuse finished with error 0 and tester_r 0 2023-10-08T05:36:31.452 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:36:31.496 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:36:31.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:31.496 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-08T05:36:31.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:31.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.774+0000 7f5076a85640 1 Processor -- start 2023-10-08T05:36:31.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.774+0000 7f5076a85640 1 -- start start 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.776+0000 7f5076a85640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50701599d0 0x7f50700ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.776+0000 7f5076a85640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f50700b07d0 0x7f507006c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.776+0000 7f5076a85640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50700cacf0 con 0x7f50700cc210 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.776+0000 7f5076a85640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50700b10f0 con 0x7f50700b07d0 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.776+0000 7f5076a85640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50700ca3c0 con 0x7f50701599d0 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.776+0000 7f5076284640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f50700b07d0 0x7f507006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.776+0000 7f5076284640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f50700b07d0 0x7f507006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49368/0 (socket says 172.21.15.70:49368) 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.776+0000 7f5076284640 1 -- 172.21.15.70:0/3346688693 learned_addr learned my addr 172.21.15.70:0/3346688693 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.776+0000 7f5075a83640 1 --2- 172.21.15.70:0/3346688693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50701599d0 0x7f50700ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.777+0000 7f5076284640 1 -- 172.21.15.70:0/3346688693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50701599d0 msgr2=0x7f50700ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.777+0000 7f5076284640 1 --2- 172.21.15.70:0/3346688693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50701599d0 0x7f50700ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.777+0000 7f5076284640 1 -- 172.21.15.70:0/3346688693 >> v1:172.21.15.181:6789/0 conn(0x7f50700cc210 legacy=0x7f50700b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:31.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.777+0000 7f5076284640 1 -- 172.21.15.70:0/3346688693 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50700ca6a0 con 0x7f50700b07d0 2023-10-08T05:36:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.778+0000 7f5076284640 1 --2- 172.21.15.70:0/3346688693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50700b07d0 0x7f507006c310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5068009e50 tx=0x7f506802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d556fd212eb0bb82 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.778+0000 7f5074a81640 1 -- 172.21.15.70:0/3346688693 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506802ca70 con 0x7f50700b07d0 2023-10-08T05:36:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.778+0000 7f5074a81640 1 -- 172.21.15.70:0/3346688693 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506802cbd0 con 0x7f50700b07d0 2023-10-08T05:36:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.778+0000 7f5074a81640 1 -- 172.21.15.70:0/3346688693 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50680343f0 con 0x7f50700b07d0 2023-10-08T05:36:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.779+0000 7f5076a85640 1 -- 172.21.15.70:0/3346688693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50700b07d0 msgr2=0x7f507006c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.779+0000 7f5076a85640 1 --2- 172.21.15.70:0/3346688693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50700b07d0 0x7f507006c310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5068009e50 tx=0x7f506802bed0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.779+0000 7f5076a85640 1 -- 172.21.15.70:0/3346688693 shutdown_connections 2023-10-08T05:36:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.779+0000 7f5076a85640 1 --2- 172.21.15.70:0/3346688693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50700b07d0 0x7f507006c310 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.779+0000 7f5076a85640 1 --2- 172.21.15.70:0/3346688693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50701599d0 0x7f50700ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.779+0000 7f5076a85640 1 -- 172.21.15.70:0/3346688693 >> 172.21.15.70:0/3346688693 conn(0x7f50700cb0c0 msgr2=0x7f50700cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.779+0000 7f5076a85640 1 -- 172.21.15.70:0/3346688693 shutdown_connections 2023-10-08T05:36:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.779+0000 7f5076a85640 1 -- 172.21.15.70:0/3346688693 wait complete. 2023-10-08T05:36:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.781+0000 7f5076a85640 1 Processor -- start 2023-10-08T05:36:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.781+0000 7f5076a85640 1 -- start start 2023-10-08T05:36:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.782+0000 7f5076a85640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50700b07d0 0x7f50700b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.782+0000 7f5076a85640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50700cc210 0x7f50700ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.782+0000 7f5076a85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50701599d0 0x7f50700ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.782+0000 7f5076a85640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50700afd30 con 0x7f50701599d0 2023-10-08T05:36:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.782+0000 7f5076a85640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5070145fc0 con 0x7f50700b07d0 2023-10-08T05:36:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.782+0000 7f5076a85640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50701462a0 con 0x7f50700cc210 2023-10-08T05:36:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.782+0000 7f5076284640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50701599d0 0x7f50700ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.782+0000 7f5076284640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50701599d0 0x7f50700ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49374/0 (socket says 172.21.15.70:49374) 2023-10-08T05:36:31.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.782+0000 7f5076284640 1 -- 172.21.15.70:0/3104994943 learned_addr learned my addr 172.21.15.70:0/3104994943 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:31.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.783+0000 7f5075282640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50700cc210 0x7f50700ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.783+0000 7f5075a83640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50700b07d0 0x7f50700b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.783+0000 7f5076284640 1 -- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50700cc210 msgr2=0x7f50700ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.783+0000 7f5076284640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50700cc210 0x7f50700ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.783+0000 7f5076284640 1 -- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50700b07d0 msgr2=0x7f50700b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.783+0000 7f5076284640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50700b07d0 0x7f50700b2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.783+0000 7f5076284640 1 -- 172.21.15.70:0/3104994943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5068009b30 con 0x7f50701599d0 2023-10-08T05:36:31.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.783+0000 7f5075282640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50700cc210 0x7f50700ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:31.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.783+0000 7f5076284640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50701599d0 0x7f50700ad580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5068036040 tx=0x7f5068004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:31.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.784+0000 7f50667fc640 1 -- 172.21.15.70:0/3104994943 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068009510 con 0x7f50701599d0 2023-10-08T05:36:31.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.784+0000 7f50667fc640 1 -- 172.21.15.70:0/3104994943 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506803e450 con 0x7f50701599d0 2023-10-08T05:36:31.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.784+0000 7f50667fc640 1 -- 172.21.15.70:0/3104994943 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506803e7e0 con 0x7f50701599d0 2023-10-08T05:36:31.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.784+0000 7f5076a85640 1 -- 172.21.15.70:0/3104994943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5070146580 con 0x7f50701599d0 2023-10-08T05:36:31.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.784+0000 7f5076a85640 1 -- 172.21.15.70:0/3104994943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50700b26a0 con 0x7f50701599d0 2023-10-08T05:36:31.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.785+0000 7f5076a85640 1 -- 172.21.15.70:0/3104994943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50700b2830 con 0x7f50701599d0 2023-10-08T05:36:31.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.786+0000 7f50667fc640 1 -- 172.21.15.70:0/3104994943 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f506803e940 con 0x7f50701599d0 2023-10-08T05:36:31.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.787+0000 7f50667fc640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f505406ae70 0x7f505406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:31.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.787+0000 7f50667fc640 1 -- 172.21.15.70:0/3104994943 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f50680ac770 con 0x7f50701599d0 2023-10-08T05:36:31.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.787+0000 7f5075a83640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f505406ae70 0x7f505406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:31.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.788+0000 7f5075a83640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f505406ae70 0x7f505406d330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f506c006fd0 tx=0x7f506c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:31.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.791+0000 7f50667fc640 1 -- 172.21.15.70:0/3104994943 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50680774a0 con 0x7f50701599d0 2023-10-08T05:36:31.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.984+0000 7f5076a85640 1 -- 172.21.15.70:0/3104994943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f507004e160 con 0x7f50701599d0 2023-10-08T05:36:31.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.987+0000 7f50667fc640 1 -- 172.21.15.70:0/3104994943 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f506807aac0 con 0x7f50701599d0 2023-10-08T05:36:31.989 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:31.989 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:31.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.989+0000 7f5076a85640 1 -- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f505406ae70 msgr2=0x7f505406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.989+0000 7f5076a85640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f505406ae70 0x7f505406d330 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f506c006fd0 tx=0x7f506c008040 comp rx=0 tx=0).stop 2023-10-08T05:36:31.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.989+0000 7f5076a85640 1 -- 172.21.15.70:0/3104994943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50701599d0 msgr2=0x7f50700ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:31.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.989+0000 7f5076a85640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50701599d0 0x7f50700ad580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5068036040 tx=0x7f5068004a40 comp rx=0 tx=0).stop 2023-10-08T05:36:31.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.990+0000 7f5076a85640 1 -- 172.21.15.70:0/3104994943 shutdown_connections 2023-10-08T05:36:31.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.990+0000 7f5076a85640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f505406ae70 0x7f505406d330 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.990+0000 7f5076a85640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50701599d0 0x7f50700ad580 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.990+0000 7f5076a85640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50700cc210 0x7f50700ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.990+0000 7f5076a85640 1 --2- 172.21.15.70:0/3104994943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50700b07d0 0x7f50700b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:31.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.990+0000 7f5076a85640 1 -- 172.21.15.70:0/3104994943 >> 172.21.15.70:0/3104994943 conn(0x7f50700cb0c0 msgr2=0x7f5070157a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:31.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.990+0000 7f5076a85640 1 -- 172.21.15.70:0/3104994943 shutdown_connections 2023-10-08T05:36:31.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:31.990+0000 7f5076a85640 1 -- 172.21.15.70:0/3104994943 wait complete. 2023-10-08T05:36:32.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:32.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.218+0000 7fb26a0b4640 1 Processor -- start 2023-10-08T05:36:32.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.218+0000 7fb26a0b4640 1 -- start start 2023-10-08T05:36:32.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.220+0000 7fb26a0b4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb26415ab40 0x7fb2640ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.220+0000 7fb26a0b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2640c90b0 0x7fb2640adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.220+0000 7fb26a0b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2640cae70 0x7fb2640cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.220+0000 7fb26a0b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26415bf30 con 0x7fb2640c90b0 2023-10-08T05:36:32.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.220+0000 7fb26a0b4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2640cb840 con 0x7fb26415ab40 2023-10-08T05:36:32.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.220+0000 7fb26a0b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26415b2b0 con 0x7fb2640cae70 2023-10-08T05:36:32.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.220+0000 7fb262ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2640c90b0 0x7fb2640adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.220+0000 7fb262ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2640c90b0 0x7fb2640adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49378/0 (socket says 172.21.15.70:49378) 2023-10-08T05:36:32.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.220+0000 7fb262ffd640 1 -- 172.21.15.70:0/2688880924 learned_addr learned my addr 172.21.15.70:0/2688880924 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:32.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.221+0000 7fb2637fe640 1 --2- 172.21.15.70:0/2688880924 >> v2:172.21.15.181:3300/0 conn(0x7fb26415ab40 0x7fb2640ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.221+0000 7fb263fff640 1 --2- 172.21.15.70:0/2688880924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2640cae70 0x7fb2640cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.221+0000 7fb262ffd640 1 -- 172.21.15.70:0/2688880924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2640cae70 msgr2=0x7fb2640cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:32.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.221+0000 7fb262ffd640 1 --2- 172.21.15.70:0/2688880924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2640cae70 0x7fb2640cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.221+0000 7fb262ffd640 1 -- 172.21.15.70:0/2688880924 >> v2:172.21.15.181:3300/0 conn(0x7fb26415ab40 msgr2=0x7fb2640ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:32.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.221+0000 7fb262ffd640 1 --2- 172.21.15.70:0/2688880924 >> v2:172.21.15.181:3300/0 conn(0x7fb26415ab40 0x7fb2640ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.221+0000 7fb262ffd640 1 -- 172.21.15.70:0/2688880924 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26415b590 con 0x7fb2640c90b0 2023-10-08T05:36:32.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.221+0000 7fb2637fe640 1 --2- 172.21.15.70:0/2688880924 >> v2:172.21.15.181:3300/0 conn(0x7fb26415ab40 0x7fb2640ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:32.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.221+0000 7fb263fff640 1 --2- 172.21.15.70:0/2688880924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2640cae70 0x7fb2640cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:32.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.222+0000 7fb262ffd640 1 --2- 172.21.15.70:0/2688880924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 0x7fb2640adc60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb26406e440 tx=0x7fb25802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a3ced4e031d2e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb2627fc640 1 -- 172.21.15.70:0/2688880924 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258031870 con 0x7fb2640c90b0 2023-10-08T05:36:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb2627fc640 1 -- 172.21.15.70:0/2688880924 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2580319d0 con 0x7fb2640c90b0 2023-10-08T05:36:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb2627fc640 1 -- 172.21.15.70:0/2688880924 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb258031d60 con 0x7fb2640c90b0 2023-10-08T05:36:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb26a0b4640 1 -- 172.21.15.70:0/2688880924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 msgr2=0x7fb2640adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb26a0b4640 1 --2- 172.21.15.70:0/2688880924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 0x7fb2640adc60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb26406e440 tx=0x7fb25802b910 comp rx=0 tx=0).stop 2023-10-08T05:36:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb26a0b4640 1 -- 172.21.15.70:0/2688880924 shutdown_connections 2023-10-08T05:36:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb26a0b4640 1 --2- 172.21.15.70:0/2688880924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2640cae70 0x7fb2640cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb26a0b4640 1 --2- 172.21.15.70:0/2688880924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 0x7fb2640adc60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb26a0b4640 1 --2- 172.21.15.70:0/2688880924 >> v2:172.21.15.181:3300/0 conn(0x7fb26415ab40 0x7fb2640ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb26a0b4640 1 -- 172.21.15.70:0/2688880924 >> 172.21.15.70:0/2688880924 conn(0x7fb26406a380 msgr2=0x7fb2640b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.223+0000 7fb26a0b4640 1 -- 172.21.15.70:0/2688880924 shutdown_connections 2023-10-08T05:36:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.224+0000 7fb26a0b4640 1 -- 172.21.15.70:0/2688880924 wait complete. 2023-10-08T05:36:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.225+0000 7fb26a0b4640 1 Processor -- start 2023-10-08T05:36:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.226+0000 7fb26a0b4640 1 -- start start 2023-10-08T05:36:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb26a0b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 0x7fb264155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb26a0b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2640cae70 0x7fb264145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb26a0b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb26415ab40 0x7fb2640ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb26a0b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2641460d0 con 0x7fb2640c90b0 2023-10-08T05:36:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb26a0b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb264145b10 con 0x7fb2640cae70 2023-10-08T05:36:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb26a0b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb264145df0 con 0x7fb26415ab40 2023-10-08T05:36:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb2637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 0x7fb264155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb2637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 0x7fb264155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49388/0 (socket says 172.21.15.70:49388) 2023-10-08T05:36:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb2637fe640 1 -- 172.21.15.70:0/4033524216 learned_addr learned my addr 172.21.15.70:0/4033524216 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:32.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb2637fe640 1 -- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb26415ab40 msgr2=0x7fb2640ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb2637fe640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb26415ab40 0x7fb2640ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb2637fe640 1 -- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2640cae70 msgr2=0x7fb264145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb2637fe640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2640cae70 0x7fb264145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.227+0000 7fb2637fe640 1 -- 172.21.15.70:0/4033524216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2580094d0 con 0x7fb2640c90b0 2023-10-08T05:36:32.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.228+0000 7fb2637fe640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 0x7fb264155480 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb25400c7f0 tx=0x7fb25400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:32.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.228+0000 7fb24bfff640 1 -- 172.21.15.70:0/4033524216 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254007820 con 0x7fb2640c90b0 2023-10-08T05:36:32.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.228+0000 7fb24bfff640 1 -- 172.21.15.70:0/4033524216 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb254007980 con 0x7fb2640c90b0 2023-10-08T05:36:32.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.228+0000 7fb24bfff640 1 -- 172.21.15.70:0/4033524216 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb254007d10 con 0x7fb2640c90b0 2023-10-08T05:36:32.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.228+0000 7fb26a0b4640 1 -- 172.21.15.70:0/4033524216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb264143240 con 0x7fb2640c90b0 2023-10-08T05:36:32.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.228+0000 7fb26a0b4640 1 -- 172.21.15.70:0/4033524216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb264145690 con 0x7fb2640c90b0 2023-10-08T05:36:32.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.229+0000 7fb26a0b4640 1 -- 172.21.15.70:0/4033524216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb264145880 con 0x7fb2640c90b0 2023-10-08T05:36:32.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.230+0000 7fb24bfff640 1 -- 172.21.15.70:0/4033524216 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb2540113f0 con 0x7fb2640c90b0 2023-10-08T05:36:32.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.231+0000 7fb24bfff640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb25006ae70 0x7fb25006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.231+0000 7fb24bfff640 1 -- 172.21.15.70:0/4033524216 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb254089c60 con 0x7fb2640c90b0 2023-10-08T05:36:32.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.232+0000 7fb262ffd640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb25006ae70 0x7fb25006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.233+0000 7fb262ffd640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb25006ae70 0x7fb25006d330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb258036000 tx=0x7fb258030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:32.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.236+0000 7fb24bfff640 1 -- 172.21.15.70:0/4033524216 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb254054a10 con 0x7fb2640c90b0 2023-10-08T05:36:32.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.429+0000 7fb26a0b4640 1 -- 172.21.15.70:0/4033524216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb264067490 con 0x7fb2640c90b0 2023-10-08T05:36:32.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.432+0000 7fb24bfff640 1 -- 172.21.15.70:0/4033524216 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7fb25401f090 con 0x7fb2640c90b0 2023-10-08T05:36:32.434 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:32.435 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:32.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.435+0000 7fb26a0b4640 1 -- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb25006ae70 msgr2=0x7fb25006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.435+0000 7fb26a0b4640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb25006ae70 0x7fb25006d330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb258036000 tx=0x7fb258030000 comp rx=0 tx=0).stop 2023-10-08T05:36:32.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.435+0000 7fb26a0b4640 1 -- 172.21.15.70:0/4033524216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 msgr2=0x7fb264155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.435+0000 7fb26a0b4640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 0x7fb264155480 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb25400c7f0 tx=0x7fb25400ccb0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.435+0000 7fb26a0b4640 1 -- 172.21.15.70:0/4033524216 shutdown_connections 2023-10-08T05:36:32.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.435+0000 7fb26a0b4640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb25006ae70 0x7fb25006d330 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.435+0000 7fb26a0b4640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb26415ab40 0x7fb2640ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.435+0000 7fb26a0b4640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2640cae70 0x7fb264145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.436+0000 7fb26a0b4640 1 --2- 172.21.15.70:0/4033524216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2640c90b0 0x7fb264155480 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.436+0000 7fb26a0b4640 1 -- 172.21.15.70:0/4033524216 >> 172.21.15.70:0/4033524216 conn(0x7fb26406a380 msgr2=0x7fb26406da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:32.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.436+0000 7fb26a0b4640 1 -- 172.21.15.70:0/4033524216 shutdown_connections 2023-10-08T05:36:32.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.436+0000 7fb26a0b4640 1 -- 172.21.15.70:0/4033524216 wait complete. 2023-10-08T05:36:32.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T05:36:32.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.675+0000 7f274cd1d640 1 Processor -- start 2023-10-08T05:36:32.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.675+0000 7f274cd1d640 1 -- start start 2023-10-08T05:36:32.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.677+0000 7f274cd1d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480cb150 0x7f27480cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.677+0000 7f274cd1d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274806e1e0 con 0x7f27480b1f20 2023-10-08T05:36:32.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.677+0000 7f274cd1d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274806cc50 con 0x7f2748161690 2023-10-08T05:36:32.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.677+0000 7f274cd1d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27480cbb10 con 0x7f27480cb150 2023-10-08T05:36:32.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.677+0000 7f2746d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f27480b1f20 0x7f27480b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58650/0 (socket says 172.21.15.70:58650) 2023-10-08T05:36:32.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.677+0000 7f2746d9d640 1 -- 172.21.15.70:0/2053273490 learned_addr learned my addr 172.21.15.70:0/2053273490 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:32.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f273dd9b640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480cb150 0x7f27480cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535312005 0 0) 0x7f274806e1e0 con 0x7f27480b1f20 2023-10-08T05:36:32.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2734003520 con 0x7f27480b1f20 2023-10-08T05:36:32.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 116311801 0 0) 0x7f2734003520 con 0x7f27480b1f20 2023-10-08T05:36:32.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480cb150 msgr2=0x7f27480cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:32.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480cb150 0x7f27480cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 >> v1:172.21.15.181:6789/0 conn(0x7f2748161690 legacy=0x7f274806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274806cf30 con 0x7f27480b1f20 2023-10-08T05:36:32.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2740001690 con 0x7f27480b1f20 2023-10-08T05:36:32.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 >> v1:172.21.15.70:6789/0 conn(0x7f27480b1f20 legacy=0x7f27480b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480cb150 0x7f27340039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2734003f20 0x7f27340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2734004d00 0x7f2734005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27480cbb10 con 0x7f2734004d00 2023-10-08T05:36:32.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273400ba60 con 0x7f2734003f20 2023-10-08T05:36:32.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.678+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273400bc20 con 0x7f27480cb150 2023-10-08T05:36:32.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.679+0000 7f2746d9d640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2734004d00 0x7f2734005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.679+0000 7f273dd9b640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2734003f20 0x7f27340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.679+0000 7f274659c640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480cb150 0x7f27340039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.679+0000 7f2746d9d640 1 -- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480cb150 msgr2=0x7f27340039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:32.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.679+0000 7f2746d9d640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480cb150 0x7f27340039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.679+0000 7f2746d9d640 1 -- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2734003f20 msgr2=0x7f27340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:32.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.679+0000 7f2746d9d640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2734003f20 0x7f27340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.679+0000 7f2746d9d640 1 -- 172.21.15.70:0/2053273490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2734003520 con 0x7f2734004d00 2023-10-08T05:36:32.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.680+0000 7f2746d9d640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2734004d00 0x7f2734005180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f274806e370 tx=0x7f27340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f93f605f7abea6e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:32.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.681+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740005ad0 con 0x7f2734004d00 2023-10-08T05:36:32.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.681+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2740005c30 con 0x7f2734004d00 2023-10-08T05:36:32.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.681+0000 7f2745d9b640 1 -- 172.21.15.70:0/2053273490 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274002c890 con 0x7f2734004d00 2023-10-08T05:36:32.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.681+0000 7f274cd1d640 1 -- 172.21.15.70:0/2053273490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2734004d00 msgr2=0x7f2734005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.681+0000 7f274cd1d640 1 --2- 172.21.15.70:0/2053273490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2734004d00 0x7f2734005180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f274806e370 tx=0x7f27340057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.681+0000 7f2746d9d640 1 -- 172.21.15.70:0/2053273490 reap_dead start 2023-10-08T05:36:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.681+0000 7f274cd1d640 1 -- 172.21.15.70:0/2053273490 shutdown_connections 2023-10-08T05:36:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.681+0000 7f274cd1d640 1 -- 172.21.15.70:0/2053273490 >> 172.21.15.70:0/2053273490 conn(0x7f274806a5d0 msgr2=0x7f27480b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.681+0000 7f274cd1d640 1 -- 172.21.15.70:0/2053273490 shutdown_connections 2023-10-08T05:36:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.681+0000 7f274cd1d640 1 -- 172.21.15.70:0/2053273490 wait complete. 2023-10-08T05:36:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.683+0000 7f274cd1d640 1 Processor -- start 2023-10-08T05:36:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.683+0000 7f274cd1d640 1 -- start start 2023-10-08T05:36:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.684+0000 7f274cd1d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2734004d00 0x7f2748152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.684+0000 7f274cd1d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480c4640 0x7f27480cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.684+0000 7f274cd1d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480c4ac0 0x7f27480cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.684+0000 7f274cd1d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2748153160 con 0x7f27480c4640 2023-10-08T05:36:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.684+0000 7f274cd1d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27480c20e0 con 0x7f2734004d00 2023-10-08T05:36:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.684+0000 7f274cd1d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27480c5070 con 0x7f27480c4ac0 2023-10-08T05:36:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f273dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480c4640 0x7f27480cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f273dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480c4640 0x7f27480cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49404/0 (socket says 172.21.15.70:49404) 2023-10-08T05:36:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f273dd9b640 1 -- 172.21.15.70:0/1453273229 learned_addr learned my addr 172.21.15.70:0/1453273229 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f274659c640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2734004d00 0x7f2748152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f2746d9d640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480c4ac0 0x7f27480cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f273dd9b640 1 -- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480c4ac0 msgr2=0x7f27480cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f273dd9b640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480c4ac0 0x7f27480cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f273dd9b640 1 -- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2734004d00 msgr2=0x7f2748152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f273dd9b640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2734004d00 0x7f2748152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f273dd9b640 1 -- 172.21.15.70:0/1453273229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27480c2380 con 0x7f27480c4640 2023-10-08T05:36:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.685+0000 7f273dd9b640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480c4640 0x7f27480cda00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f27300119a0 tx=0x7f2730011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.686+0000 7f273f7fe640 1 -- 172.21.15.70:0/1453273229 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273000fa90 con 0x7f27480c4640 2023-10-08T05:36:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.686+0000 7f273f7fe640 1 -- 172.21.15.70:0/1453273229 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273000fbf0 con 0x7f27480c4640 2023-10-08T05:36:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.686+0000 7f273f7fe640 1 -- 172.21.15.70:0/1453273229 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2730014440 con 0x7f27480c4640 2023-10-08T05:36:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.686+0000 7f274cd1d640 1 -- 172.21.15.70:0/1453273229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27480c2660 con 0x7f27480c4640 2023-10-08T05:36:32.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.686+0000 7f274cd1d640 1 -- 172.21.15.70:0/1453273229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27481f75a0 con 0x7f27480c4640 2023-10-08T05:36:32.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.687+0000 7f274cd1d640 1 -- 172.21.15.70:0/1453273229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274804e160 con 0x7f27480c4640 2023-10-08T05:36:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.688+0000 7f273f7fe640 1 -- 172.21.15.70:0/1453273229 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f27300145a0 con 0x7f27480c4640 2023-10-08T05:36:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.689+0000 7f273f7fe640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f272806aea0 0x7f272806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.689+0000 7f273f7fe640 1 -- 172.21.15.70:0/1453273229 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f273008a180 con 0x7f27480c4640 2023-10-08T05:36:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.689+0000 7f274659c640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f272806aea0 0x7f272806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.690+0000 7f274659c640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f272806aea0 0x7f272806d360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27480be090 tx=0x7f2738009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:32.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.693+0000 7f273f7fe640 1 -- 172.21.15.70:0/1453273229 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2730054ea0 con 0x7f27480c4640 2023-10-08T05:36:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.921+0000 7f274cd1d640 1 -- 172.21.15.70:0/1453273229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2748073900 con 0x7f27480c4640 2023-10-08T05:36:32.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.963+0000 7f273f7fe640 1 -- 172.21.15.70:0/1453273229 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f27300584c0 con 0x7f27480c4640 2023-10-08T05:36:32.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 -- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f272806aea0 msgr2=0x7f272806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f272806aea0 0x7f272806d360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27480be090 tx=0x7f2738009040 comp rx=0 tx=0).stop 2023-10-08T05:36:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 -- 172.21.15.70:0/1453273229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480c4640 msgr2=0x7f27480cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480c4640 0x7f27480cda00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f27300119a0 tx=0x7f2730011e60 comp rx=0 tx=0).stop 2023-10-08T05:36:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 -- 172.21.15.70:0/1453273229 shutdown_connections 2023-10-08T05:36:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f272806aea0 0x7f272806d360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27480c4ac0 0x7f27480cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27480c4640 0x7f27480cda00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 --2- 172.21.15.70:0/1453273229 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2734004d00 0x7f2748152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 -- 172.21.15.70:0/1453273229 >> 172.21.15.70:0/1453273229 conn(0x7f274806a5d0 msgr2=0x7f2748160d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 -- 172.21.15.70:0/1453273229 shutdown_connections 2023-10-08T05:36:32.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:32.966+0000 7f274cd1d640 1 -- 172.21.15.70:0/1453273229 wait complete. 2023-10-08T05:36:32.987 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-10-08T05:36:32.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T05:36:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.204+0000 7ff423cbb640 1 Processor -- start 2023-10-08T05:36:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.205+0000 7ff423cbb640 1 -- start start 2023-10-08T05:36:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.206+0000 7ff423cbb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff41c15abe0 0x7ff41c0d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.206+0000 7ff423cbb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41c0b69d0 con 0x7ff41c0b62a0 2023-10-08T05:36:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.206+0000 7ff423cbb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41c0cf770 con 0x7ff41c0cd430 2023-10-08T05:36:33.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.206+0000 7ff423cbb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41c0b6700 con 0x7ff41c15abe0 2023-10-08T05:36:33.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.206+0000 7ff421a57640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff41c15abe0 0x7ff41c0d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:33.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.207+0000 7ff421256640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff41c0b62a0 0x7ff41c0cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58660/0 (socket says 172.21.15.70:58660) 2023-10-08T05:36:33.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.207+0000 7ff421256640 1 -- 172.21.15.70:0/2811364763 learned_addr learned my addr 172.21.15.70:0/2811364763 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:33.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.207+0000 7ff421a57640 1 -- 172.21.15.70:0/2811364763 >> v1:172.21.15.181:6789/0 conn(0x7ff41c0cd430 legacy=0x7ff41c0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:33.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.207+0000 7ff421a57640 1 -- 172.21.15.70:0/2811364763 >> v1:172.21.15.70:6789/0 conn(0x7ff41c0b62a0 legacy=0x7ff41c0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:33.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.207+0000 7ff421a57640 1 -- 172.21.15.70:0/2811364763 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c0cee40 con 0x7ff41c15abe0 2023-10-08T05:36:33.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.207+0000 7ff420a55640 1 -- 172.21.15.70:0/2811364763 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202131478 0 0) 0x7ff41c0b69d0 con 0x7ff41c0b62a0 2023-10-08T05:36:33.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.208+0000 7ff421a57640 1 --2- 172.21.15.70:0/2811364763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c15abe0 0x7ff41c0d0420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009790 tx=0x7ff40c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e92dc58fd71e12 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:33.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.208+0000 7ff420a55640 1 -- 172.21.15.70:0/2811364763 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c02d040 con 0x7ff41c15abe0 2023-10-08T05:36:33.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.209+0000 7ff420a55640 1 -- 172.21.15.70:0/2811364763 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40c031810 con 0x7ff41c15abe0 2023-10-08T05:36:33.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.209+0000 7ff420a55640 1 -- 172.21.15.70:0/2811364763 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c031ba0 con 0x7ff41c15abe0 2023-10-08T05:36:33.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.209+0000 7ff423cbb640 1 -- 172.21.15.70:0/2811364763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c15abe0 msgr2=0x7ff41c0d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:33.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.209+0000 7ff423cbb640 1 --2- 172.21.15.70:0/2811364763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c15abe0 0x7ff41c0d0420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009790 tx=0x7ff40c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:36:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.209+0000 7ff423cbb640 1 -- 172.21.15.70:0/2811364763 shutdown_connections 2023-10-08T05:36:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.209+0000 7ff423cbb640 1 --2- 172.21.15.70:0/2811364763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c15abe0 0x7ff41c0d0420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.209+0000 7ff423cbb640 1 -- 172.21.15.70:0/2811364763 >> 172.21.15.70:0/2811364763 conn(0x7ff41c06e650 msgr2=0x7ff41c06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.209+0000 7ff423cbb640 1 -- 172.21.15.70:0/2811364763 shutdown_connections 2023-10-08T05:36:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.209+0000 7ff423cbb640 1 -- 172.21.15.70:0/2811364763 wait complete. 2023-10-08T05:36:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.211+0000 7ff423cbb640 1 Processor -- start 2023-10-08T05:36:33.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.211+0000 7ff423cbb640 1 -- start start 2023-10-08T05:36:33.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.212+0000 7ff423cbb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c0b62a0 0x7ff41c0ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:33.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.212+0000 7ff423cbb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c0cd430 0x7ff41c153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:33.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.212+0000 7ff423cbb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff41c15abe0 0x7ff41c0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:33.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.212+0000 7ff423cbb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c1505d0 con 0x7ff41c0b62a0 2023-10-08T05:36:33.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.212+0000 7ff423cbb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c1564e0 con 0x7ff41c0cd430 2023-10-08T05:36:33.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.212+0000 7ff423cbb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c156d80 con 0x7ff41c15abe0 2023-10-08T05:36:33.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.212+0000 7ff421a57640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c0b62a0 0x7ff41c0ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:33.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.212+0000 7ff421a57640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c0b62a0 0x7ff41c0ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49416/0 (socket says 172.21.15.70:49416) 2023-10-08T05:36:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.212+0000 7ff421a57640 1 -- 172.21.15.70:0/884886091 learned_addr learned my addr 172.21.15.70:0/884886091 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.212+0000 7ff421256640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c0cd430 0x7ff41c153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff422258640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff41c15abe0 0x7ff41c0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff421a57640 1 -- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff41c15abe0 msgr2=0x7ff41c0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff421a57640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff41c15abe0 0x7ff41c0c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:33.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff421a57640 1 -- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c0cd430 msgr2=0x7ff41c153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:33.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff421a57640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c0cd430 0x7ff41c153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:33.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff421a57640 1 -- 172.21.15.70:0/884886091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0093a0 con 0x7ff41c0b62a0 2023-10-08T05:36:33.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff422258640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff41c15abe0 0x7ff41c0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:33.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff421a57640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c0b62a0 0x7ff41c0ca250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0098b0 tx=0x7ff40c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff41a7fc640 1 -- 172.21.15.70:0/884886091 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c03bae0 con 0x7ff41c0b62a0 2023-10-08T05:36:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff41a7fc640 1 -- 172.21.15.70:0/884886091 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40c03bc40 con 0x7ff41c0b62a0 2023-10-08T05:36:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff423cbb640 1 -- 172.21.15.70:0/884886091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c0ca830 con 0x7ff41c0b62a0 2023-10-08T05:36:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff41a7fc640 1 -- 172.21.15.70:0/884886091 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c005990 con 0x7ff41c0b62a0 2023-10-08T05:36:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.213+0000 7ff423cbb640 1 -- 172.21.15.70:0/884886091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c156a10 con 0x7ff41c0b62a0 2023-10-08T05:36:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.215+0000 7ff423cbb640 1 -- 172.21.15.70:0/884886091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c04e160 con 0x7ff41c0b62a0 2023-10-08T05:36:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.215+0000 7ff41a7fc640 1 -- 172.21.15.70:0/884886091 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff40c043030 con 0x7ff41c0b62a0 2023-10-08T05:36:33.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.216+0000 7ff41a7fc640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff40806ae70 0x7ff40806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:33.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.216+0000 7ff41a7fc640 1 -- 172.21.15.70:0/884886091 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff40c0abe00 con 0x7ff41c0b62a0 2023-10-08T05:36:33.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.217+0000 7ff421256640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff40806ae70 0x7ff40806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:33.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.218+0000 7ff421256640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff40806ae70 0x7ff40806d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff41c153690 tx=0x7ff410007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:33.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.220+0000 7ff41a7fc640 1 -- 172.21.15.70:0/884886091 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff40c076cd0 con 0x7ff41c0b62a0 2023-10-08T05:36:33.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.448+0000 7ff423cbb640 1 -- 172.21.15.70:0/884886091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff41c04e160 con 0x7ff41c0b62a0 2023-10-08T05:36:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.973+0000 7ff41a7fc640 1 -- 172.21.15.70:0/884886091 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff40c07a2f0 con 0x7ff41c0b62a0 2023-10-08T05:36:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.976+0000 7ff423cbb640 1 -- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff40806ae70 msgr2=0x7ff40806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.976+0000 7ff423cbb640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff40806ae70 0x7ff40806d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff41c153690 tx=0x7ff410007c00 comp rx=0 tx=0).stop 2023-10-08T05:36:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.976+0000 7ff423cbb640 1 -- 172.21.15.70:0/884886091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c0b62a0 msgr2=0x7ff41c0ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.976+0000 7ff423cbb640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c0b62a0 0x7ff41c0ca250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0098b0 tx=0x7ff40c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.976+0000 7ff423cbb640 1 -- 172.21.15.70:0/884886091 shutdown_connections 2023-10-08T05:36:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.976+0000 7ff423cbb640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff40806ae70 0x7ff40806d330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.977+0000 7ff423cbb640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff41c15abe0 0x7ff41c0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.977+0000 7ff423cbb640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff41c0cd430 0x7ff41c153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.977+0000 7ff423cbb640 1 --2- 172.21.15.70:0/884886091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff41c0b62a0 0x7ff41c0ca250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.977+0000 7ff423cbb640 1 -- 172.21.15.70:0/884886091 >> 172.21.15.70:0/884886091 conn(0x7ff41c06e650 msgr2=0x7ff41c15b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.977+0000 7ff423cbb640 1 -- 172.21.15.70:0/884886091 shutdown_connections 2023-10-08T05:36:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:33.977+0000 7ff423cbb640 1 -- 172.21.15.70:0/884886091 wait complete. 2023-10-08T05:36:33.998 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T05:36:33.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:36:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.213+0000 7f760b002640 1 Processor -- start 2023-10-08T05:36:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.213+0000 7f760b002640 1 -- start start 2023-10-08T05:36:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.214+0000 7f760b002640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7604160940 0x7f76040709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f760b002640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f760406c890 0x7f760406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f760b002640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76040c48f0 con 0x7f760415cd90 2023-10-08T05:36:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f760b002640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76040c4d80 con 0x7f7604160940 2023-10-08T05:36:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f760b002640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76040c3ce0 con 0x7f760406c890 2023-10-08T05:36:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f7608d9e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f760415cd90 0x7f76040702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43578/0 (socket says 172.21.15.70:43578) 2023-10-08T05:36:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f7608d9e640 1 -- 172.21.15.70:0/257921863 learned_addr learned my addr 172.21.15.70:0/257921863 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f7603fff640 1 --2- 172.21.15.70:0/257921863 >> v2:172.21.15.70:3300/0 conn(0x7f7604160940 0x7f76040709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f760959f640 1 --2- 172.21.15.70:0/257921863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f760406c890 0x7f760406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f76037fe640 1 -- 172.21.15.70:0/257921863 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155801079 0 0) 0x7f76040c48f0 con 0x7f760415cd90 2023-10-08T05:36:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f76037fe640 1 -- 172.21.15.70:0/257921863 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75f4003520 con 0x7f760415cd90 2023-10-08T05:36:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f7603fff640 1 -- 172.21.15.70:0/257921863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f760406c890 msgr2=0x7f760406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f7603fff640 1 --2- 172.21.15.70:0/257921863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f760406c890 0x7f760406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.215+0000 7f7603fff640 1 -- 172.21.15.70:0/257921863 >> v1:172.21.15.70:6789/0 conn(0x7f760415cd90 legacy=0x7f76040702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.216+0000 7f7603fff640 1 -- 172.21.15.70:0/257921863 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76040c3fc0 con 0x7f7604160940 2023-10-08T05:36:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.217+0000 7f7603fff640 1 --2- 172.21.15.70:0/257921863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7604160940 0x7f76040709e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f75f00097a0 tx=0x7f75f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95447216ac2d244e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:34.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.217+0000 7f76037fe640 1 -- 172.21.15.70:0/257921863 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0031830 con 0x7f7604160940 2023-10-08T05:36:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.217+0000 7f76037fe640 1 -- 172.21.15.70:0/257921863 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f0031990 con 0x7f7604160940 2023-10-08T05:36:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.217+0000 7f76037fe640 1 -- 172.21.15.70:0/257921863 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0031d00 con 0x7f7604160940 2023-10-08T05:36:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.217+0000 7f760b002640 1 -- 172.21.15.70:0/257921863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7604160940 msgr2=0x7f76040709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.217+0000 7f760b002640 1 --2- 172.21.15.70:0/257921863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7604160940 0x7f76040709e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f75f00097a0 tx=0x7f75f002b8d0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.218+0000 7f760b002640 1 -- 172.21.15.70:0/257921863 shutdown_connections 2023-10-08T05:36:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.218+0000 7f760b002640 1 --2- 172.21.15.70:0/257921863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f760406c890 0x7f760406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.218+0000 7f760b002640 1 --2- 172.21.15.70:0/257921863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7604160940 0x7f76040709e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.218+0000 7f760b002640 1 -- 172.21.15.70:0/257921863 >> 172.21.15.70:0/257921863 conn(0x7f760406a380 msgr2=0x7f76040b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.218+0000 7f760b002640 1 -- 172.21.15.70:0/257921863 shutdown_connections 2023-10-08T05:36:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.218+0000 7f760b002640 1 -- 172.21.15.70:0/257921863 wait complete. 2023-10-08T05:36:34.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.219+0000 7f760b002640 1 Processor -- start 2023-10-08T05:36:34.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.220+0000 7f760b002640 1 -- start start 2023-10-08T05:36:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.220+0000 7f760b002640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f760406c890 0x7f760415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.220+0000 7f760b002640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f760415cd90 0x7f7604152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.220+0000 7f760b002640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7604160940 0x7f7604153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f760b002640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7604154a00 con 0x7f760415cd90 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f760b002640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7604153470 con 0x7f760406c890 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f760b002640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7604153e80 con 0x7f7604160940 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f7603fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f760415cd90 0x7f7604152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f7603fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f760415cd90 0x7f7604152f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59708/0 (socket says 172.21.15.70:59708) 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f7603fff640 1 -- 172.21.15.70:0/3516832819 learned_addr learned my addr 172.21.15.70:0/3516832819 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f7608d9e640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f760406c890 0x7f760415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f7603fff640 1 -- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7604160940 msgr2=0x7f7604153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f7603fff640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7604160940 0x7f7604153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f7603fff640 1 -- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f760406c890 msgr2=0x7f760415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f7603fff640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f760406c890 0x7f760415fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f7603fff640 1 -- 172.21.15.70:0/3516832819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f0009400 con 0x7f760415cd90 2023-10-08T05:36:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.221+0000 7f7603fff640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f760415cd90 0x7f7604152f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f75f002b910 tx=0x7f75f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.222+0000 7f76017fa640 1 -- 172.21.15.70:0/3516832819 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f003a8f0 con 0x7f760415cd90 2023-10-08T05:36:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.222+0000 7f76017fa640 1 -- 172.21.15.70:0/3516832819 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f003aa70 con 0x7f760415cd90 2023-10-08T05:36:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.222+0000 7f76017fa640 1 -- 172.21.15.70:0/3516832819 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f003ae20 con 0x7f760415cd90 2023-10-08T05:36:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.222+0000 7f760b002640 1 -- 172.21.15.70:0/3516832819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7604154160 con 0x7f760415cd90 2023-10-08T05:36:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.222+0000 7f760b002640 1 -- 172.21.15.70:0/3516832819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7604154720 con 0x7f760415cd90 2023-10-08T05:36:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.223+0000 7f760b002640 1 -- 172.21.15.70:0/3516832819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7604067490 con 0x7f760415cd90 2023-10-08T05:36:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.224+0000 7f76017fa640 1 -- 172.21.15.70:0/3516832819 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f75f004d450 con 0x7f760415cd90 2023-10-08T05:36:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.225+0000 7f76017fa640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f75e806aea0 0x7f75e806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.225+0000 7f76017fa640 1 -- 172.21.15.70:0/3516832819 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f75f00ab910 con 0x7f760415cd90 2023-10-08T05:36:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.226+0000 7f7608d9e640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f75e806aea0 0x7f75e806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.227+0000 7f7608d9e640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f75e806aea0 0x7f75e806d360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f75f8000f60 tx=0x7f75f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.229+0000 7f76017fa640 1 -- 172.21.15.70:0/3516832819 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75f00766d0 con 0x7f760415cd90 2023-10-08T05:36:34.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.454+0000 7f760b002640 1 -- 172.21.15.70:0/3516832819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76040b0ef0 con 0x7f760415cd90 2023-10-08T05:36:34.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.455+0000 7f76017fa640 1 -- 172.21.15.70:0/3516832819 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f75f0079cf0 con 0x7f760415cd90 2023-10-08T05:36:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:34.457 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:36:11.613175+0000","modified":"2023-10-08T05:36:33.450447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T05:36:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 9 2023-10-08T05:36:34.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 -- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f75e806aea0 msgr2=0x7f75e806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f75e806aea0 0x7f75e806d360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f75f8000f60 tx=0x7f75f8009040 comp rx=0 tx=0).stop 2023-10-08T05:36:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 -- 172.21.15.70:0/3516832819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f760415cd90 msgr2=0x7f7604152f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f760415cd90 0x7f7604152f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f75f002b910 tx=0x7f75f00046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 -- 172.21.15.70:0/3516832819 shutdown_connections 2023-10-08T05:36:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f75e806aea0 0x7f75e806d360 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7604160940 0x7f7604153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f760415cd90 0x7f7604152f30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 --2- 172.21.15.70:0/3516832819 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f760406c890 0x7f760415fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 -- 172.21.15.70:0/3516832819 >> 172.21.15.70:0/3516832819 conn(0x7f760406a380 msgr2=0x7f76040abca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 -- 172.21.15.70:0/3516832819 shutdown_connections 2023-10-08T05:36:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.458+0000 7f760b002640 1 -- 172.21.15.70:0/3516832819 wait complete. 2023-10-08T05:36:34.477 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:34.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:36:34.651 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:36:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.687+0000 7f5d553e2640 1 Processor -- start 2023-10-08T05:36:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.688+0000 7f5d553e2640 1 -- start start 2023-10-08T05:36:34.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.689+0000 7f5d553e2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d5015ab40 0x7f5d500ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.689+0000 7f5d553e2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d500c90b0 0x7f5d500adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.689+0000 7f5d553e2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5015b2b0 con 0x7f5d500cae70 2023-10-08T05:36:34.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.689+0000 7f5d553e2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5015bf30 con 0x7f5d500c90b0 2023-10-08T05:36:34.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.689+0000 7f5d553e2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5015f7a0 con 0x7f5d5015ab40 2023-10-08T05:36:34.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.689+0000 7f5d4e7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d500c90b0 0x7f5d500adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.689+0000 7f5d4e7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d500c90b0 0x7f5d500adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59720/0 (socket says 172.21.15.70:59720) 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.689+0000 7f5d4e7fc640 1 -- 172.21.15.70:0/4281545669 learned_addr learned my addr 172.21.15.70:0/4281545669 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.690+0000 7f5d4effd640 1 --2- 172.21.15.70:0/4281545669 >> v2:172.21.15.181:3300/0 conn(0x7f5d5015ab40 0x7f5d500ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.690+0000 7f5d4e7fc640 1 -- 172.21.15.70:0/4281545669 >> v2:172.21.15.181:3300/0 conn(0x7f5d5015ab40 msgr2=0x7f5d500ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.690+0000 7f5d4e7fc640 1 --2- 172.21.15.70:0/4281545669 >> v2:172.21.15.181:3300/0 conn(0x7f5d5015ab40 0x7f5d500ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.690+0000 7f5d4e7fc640 1 -- 172.21.15.70:0/4281545669 >> v1:172.21.15.181:6789/0 conn(0x7f5d500cae70 legacy=0x7f5d500702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.690+0000 7f5d4e7fc640 1 -- 172.21.15.70:0/4281545669 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5015b590 con 0x7f5d500c90b0 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.691+0000 7f5d4e7fc640 1 --2- 172.21.15.70:0/4281545669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d500c90b0 0x7f5d500adc60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d5006e440 tx=0x7f5d4402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3df1a70c9088f6c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.691+0000 7f5d4dffb640 1 -- 172.21.15.70:0/4281545669 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44030860 con 0x7f5d500c90b0 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.691+0000 7f5d4dffb640 1 -- 172.21.15.70:0/4281545669 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d440309c0 con 0x7f5d500c90b0 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.691+0000 7f5d4dffb640 1 -- 172.21.15.70:0/4281545669 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44030d50 con 0x7f5d500c90b0 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.692+0000 7f5d553e2640 1 -- 172.21.15.70:0/4281545669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d500c90b0 msgr2=0x7f5d500adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.692+0000 7f5d553e2640 1 --2- 172.21.15.70:0/4281545669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d500c90b0 0x7f5d500adc60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d5006e440 tx=0x7f5d4402b9a0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.692+0000 7f5d553e2640 1 -- 172.21.15.70:0/4281545669 shutdown_connections 2023-10-08T05:36:34.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.692+0000 7f5d553e2640 1 --2- 172.21.15.70:0/4281545669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d500c90b0 0x7f5d500adc60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.692+0000 7f5d553e2640 1 --2- 172.21.15.70:0/4281545669 >> v2:172.21.15.181:3300/0 conn(0x7f5d5015ab40 0x7f5d500ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.692+0000 7f5d553e2640 1 -- 172.21.15.70:0/4281545669 >> 172.21.15.70:0/4281545669 conn(0x7f5d5006a380 msgr2=0x7f5d500b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:34.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.692+0000 7f5d553e2640 1 -- 172.21.15.70:0/4281545669 shutdown_connections 2023-10-08T05:36:34.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.692+0000 7f5d553e2640 1 -- 172.21.15.70:0/4281545669 wait complete. 2023-10-08T05:36:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.694+0000 7f5d553e2640 1 Processor -- start 2023-10-08T05:36:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.694+0000 7f5d553e2640 1 -- start start 2023-10-08T05:36:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d553e2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d500c90b0 0x7f5d501589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d553e2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500cae70 0x7f5d501f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d553e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5015ab40 0x7f5d500aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d553e2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500c4500 con 0x7f5d5015ab40 2023-10-08T05:36:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d553e2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500c4710 con 0x7f5d500cae70 2023-10-08T05:36:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d553e2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500c49f0 con 0x7f5d500c90b0 2023-10-08T05:36:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d4f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5015ab40 0x7f5d500aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d4f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5015ab40 0x7f5d500aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59732/0 (socket says 172.21.15.70:59732) 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d4f7fe640 1 -- 172.21.15.70:0/871525202 learned_addr learned my addr 172.21.15.70:0/871525202 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d4e7fc640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500cae70 0x7f5d501f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.695+0000 7f5d4effd640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d500c90b0 0x7f5d501589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.696+0000 7f5d4f7fe640 1 -- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d500c90b0 msgr2=0x7f5d501589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.696+0000 7f5d4f7fe640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d500c90b0 0x7f5d501589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.696+0000 7f5d4f7fe640 1 -- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500cae70 msgr2=0x7f5d501f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.696+0000 7f5d4f7fe640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500cae70 0x7f5d501f1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.696+0000 7f5d4f7fe640 1 -- 172.21.15.70:0/871525202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440094d0 con 0x7f5d5015ab40 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.696+0000 7f5d4f7fe640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5015ab40 0x7f5d500aca40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800c7a0 tx=0x7f5d4800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.696+0000 7f5d377fe640 1 -- 172.21.15.70:0/871525202 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48003b70 con 0x7f5d5015ab40 2023-10-08T05:36:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.696+0000 7f5d377fe640 1 -- 172.21.15.70:0/871525202 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d48003cf0 con 0x7f5d5015ab40 2023-10-08T05:36:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.696+0000 7f5d553e2640 1 -- 172.21.15.70:0/871525202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d501581d0 con 0x7f5d5015ab40 2023-10-08T05:36:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.696+0000 7f5d377fe640 1 -- 172.21.15.70:0/871525202 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48003ea0 con 0x7f5d5015ab40 2023-10-08T05:36:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.697+0000 7f5d553e2640 1 -- 172.21.15.70:0/871525202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d501f2460 con 0x7f5d5015ab40 2023-10-08T05:36:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.698+0000 7f5d553e2640 1 -- 172.21.15.70:0/871525202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d501586f0 con 0x7f5d5015ab40 2023-10-08T05:36:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.700+0000 7f5d377fe640 1 -- 172.21.15.70:0/871525202 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5d4800aa90 con 0x7f5d5015ab40 2023-10-08T05:36:34.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.700+0000 7f5d377fe640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5d3c06ae70 0x7f5d3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:34.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.701+0000 7f5d377fe640 1 -- 172.21.15.70:0/871525202 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5d48089540 con 0x7f5d5015ab40 2023-10-08T05:36:34.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.701+0000 7f5d4effd640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5d3c06ae70 0x7f5d3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:34.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.702+0000 7f5d4effd640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5d3c06ae70 0x7f5d3c06d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d40006fd0 tx=0x7f5d40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.705+0000 7f5d377fe640 1 -- 172.21.15.70:0/871525202 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d501586f0 con 0x7f5d5015ab40 2023-10-08T05:36:34.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.898+0000 7f5d553e2640 1 -- 172.21.15.70:0/871525202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d501586f0 con 0x7f5d5015ab40 2023-10-08T05:36:34.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.901+0000 7f5d377fe640 1 -- 172.21.15.70:0/871525202 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f5d501586f0 con 0x7f5d5015ab40 2023-10-08T05:36:34.903 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:34.903 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:34.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 -- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5d3c06ae70 msgr2=0x7f5d3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5d3c06ae70 0x7f5d3c06d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d40006fd0 tx=0x7f5d40008040 comp rx=0 tx=0).stop 2023-10-08T05:36:34.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 -- 172.21.15.70:0/871525202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5015ab40 msgr2=0x7f5d500aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:34.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5015ab40 0x7f5d500aca40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800c7a0 tx=0x7f5d4800cc60 comp rx=0 tx=0).stop 2023-10-08T05:36:34.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 -- 172.21.15.70:0/871525202 shutdown_connections 2023-10-08T05:36:34.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5d3c06ae70 0x7f5d3c06d330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5015ab40 0x7f5d500aca40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d500cae70 0x7f5d501f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 --2- 172.21.15.70:0/871525202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d500c90b0 0x7f5d501589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:34.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 -- 172.21.15.70:0/871525202 >> 172.21.15.70:0/871525202 conn(0x7f5d5006a380 msgr2=0x7f5d500cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:34.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.904+0000 7f5d553e2640 1 -- 172.21.15.70:0/871525202 shutdown_connections 2023-10-08T05:36:34.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:34.905+0000 7f5d553e2640 1 -- 172.21.15.70:0/871525202 wait complete. 2023-10-08T05:36:34.924 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T05:36:34.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:36:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.139+0000 7faf4c938640 1 Processor -- start 2023-10-08T05:36:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.140+0000 7faf4c938640 1 -- start start 2023-10-08T05:36:35.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.141+0000 7faf4c938640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf480cc210 0x7faf480b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.141+0000 7faf4c938640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf480cacf0 con 0x7faf48159db0 2023-10-08T05:36:35.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.141+0000 7faf4c938640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf480ca3c0 con 0x7faf481599d0 2023-10-08T05:36:35.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.141+0000 7faf4c938640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf480b10f0 con 0x7faf480cc210 2023-10-08T05:36:35.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.141+0000 7faf4759e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7faf48159db0 0x7faf4806c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43584/0 (socket says 172.21.15.70:43584) 2023-10-08T05:36:35.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.141+0000 7faf4759e640 1 -- 172.21.15.70:0/2177674700 learned_addr learned my addr 172.21.15.70:0/2177674700 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:35.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf3ffff640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf480cc210 0x7faf480b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722861088 0 0) 0x7faf480cacf0 con 0x7faf48159db0 2023-10-08T05:36:35.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf34003520 con 0x7faf48159db0 2023-10-08T05:36:35.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2275150633 0 0) 0x7faf34003520 con 0x7faf48159db0 2023-10-08T05:36:35.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf480cc210 msgr2=0x7faf480b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf480cc210 0x7faf480b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 >> v1:172.21.15.181:6789/0 conn(0x7faf481599d0 legacy=0x7faf480ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf480ca6a0 con 0x7faf48159db0 2023-10-08T05:36:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7faf400019f0 con 0x7faf48159db0 2023-10-08T05:36:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 >> v1:172.21.15.70:6789/0 conn(0x7faf48159db0 legacy=0x7faf4806c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf340039a0 0x7faf34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf480cc210 0x7faf340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34004d00 0x7faf34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf480b10f0 con 0x7faf340039a0 2023-10-08T05:36:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3400ba60 con 0x7faf34004d00 2023-10-08T05:36:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.142+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3400bc20 con 0x7faf480cc210 2023-10-08T05:36:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.143+0000 7faf46d9d640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf340039a0 0x7faf34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.143+0000 7faf4759e640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34004d00 0x7faf34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.143+0000 7faf46d9d640 1 -- 172.21.15.70:0/2177674700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf480cc210 msgr2=0x7faf340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.143+0000 7faf46d9d640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf480cc210 0x7faf340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.143+0000 7faf46d9d640 1 -- 172.21.15.70:0/2177674700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34004d00 msgr2=0x7faf34005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.143+0000 7faf46d9d640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf34004d00 0x7faf34005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.143+0000 7faf46d9d640 1 -- 172.21.15.70:0/2177674700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf34003520 con 0x7faf340039a0 2023-10-08T05:36:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.144+0000 7faf46d9d640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf340039a0 0x7faf34003dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf38001720 tx=0x7faf340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a619bc112eafc77b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.144+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf380052d0 con 0x7faf340039a0 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.144+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3802e050 con 0x7faf340039a0 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.144+0000 7faf4659c640 1 -- 172.21.15.70:0/2177674700 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf380035f0 con 0x7faf340039a0 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.145+0000 7faf4c938640 1 -- 172.21.15.70:0/2177674700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf340039a0 msgr2=0x7faf34003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.145+0000 7faf4c938640 1 --2- 172.21.15.70:0/2177674700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf340039a0 0x7faf34003dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf38001720 tx=0x7faf340057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.145+0000 7faf4759e640 1 -- 172.21.15.70:0/2177674700 reap_dead start 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.145+0000 7faf4c938640 1 -- 172.21.15.70:0/2177674700 shutdown_connections 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.145+0000 7faf4c938640 1 -- 172.21.15.70:0/2177674700 >> 172.21.15.70:0/2177674700 conn(0x7faf480cb0c0 msgr2=0x7faf480cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.145+0000 7faf4c938640 1 -- 172.21.15.70:0/2177674700 shutdown_connections 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.145+0000 7faf4c938640 1 -- 172.21.15.70:0/2177674700 wait complete. 2023-10-08T05:36:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.147+0000 7faf4c938640 1 Processor -- start 2023-10-08T05:36:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.147+0000 7faf4c938640 1 -- start start 2023-10-08T05:36:35.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.148+0000 7faf4c938640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf340039a0 0x7faf480b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.148+0000 7faf4c938640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf481f6700 0x7faf480b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.148+0000 7faf4c938640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf481f3160 0x7faf48156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.148+0000 7faf4c938640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf481f2470 con 0x7faf481f3160 2023-10-08T05:36:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.148+0000 7faf4c938640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf480b1ba0 con 0x7faf481f6700 2023-10-08T05:36:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.148+0000 7faf4c938640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf481f2680 con 0x7faf340039a0 2023-10-08T05:36:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.148+0000 7faf4759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf481f3160 0x7faf48156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.148+0000 7faf4759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf481f3160 0x7faf48156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59756/0 (socket says 172.21.15.70:59756) 2023-10-08T05:36:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.148+0000 7faf4759e640 1 -- 172.21.15.70:0/1660904877 learned_addr learned my addr 172.21.15.70:0/1660904877 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf3ffff640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf481f6700 0x7faf480b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf46d9d640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf340039a0 0x7faf480b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf4759e640 1 -- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf340039a0 msgr2=0x7faf480b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf4759e640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf340039a0 0x7faf480b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf4759e640 1 -- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf481f6700 msgr2=0x7faf480b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf4759e640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf481f6700 0x7faf480b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf4759e640 1 -- 172.21.15.70:0/1660904877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf481f2960 con 0x7faf481f3160 2023-10-08T05:36:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf3ffff640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf481f6700 0x7faf480b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf4759e640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf481f3160 0x7faf48156a80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf40001330 tx=0x7faf4000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf3f7fe640 1 -- 172.21.15.70:0/1660904877 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4000deb0 con 0x7faf481f3160 2023-10-08T05:36:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf3f7fe640 1 -- 172.21.15.70:0/1660904877 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4000bb80 con 0x7faf481f3160 2023-10-08T05:36:35.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf3f7fe640 1 -- 172.21.15.70:0/1660904877 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf40007070 con 0x7faf481f3160 2023-10-08T05:36:35.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf4c938640 1 -- 172.21.15.70:0/1660904877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf481f2c40 con 0x7faf481f3160 2023-10-08T05:36:35.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.149+0000 7faf4c938640 1 -- 172.21.15.70:0/1660904877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf480b2e40 con 0x7faf481f3160 2023-10-08T05:36:35.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.150+0000 7faf4c938640 1 -- 172.21.15.70:0/1660904877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf481f2e30 con 0x7faf481f3160 2023-10-08T05:36:35.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.151+0000 7faf3f7fe640 1 -- 172.21.15.70:0/1660904877 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7faf400071d0 con 0x7faf481f3160 2023-10-08T05:36:35.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.152+0000 7faf3f7fe640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7faf2806ae70 0x7faf2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.152+0000 7faf3f7fe640 1 -- 172.21.15.70:0/1660904877 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7faf40089a40 con 0x7faf481f3160 2023-10-08T05:36:35.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.153+0000 7faf46d9d640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7faf2806ae70 0x7faf2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.154+0000 7faf46d9d640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7faf2806ae70 0x7faf2806d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faf38033000 tx=0x7faf38009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:35.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.157+0000 7faf3f7fe640 1 -- 172.21.15.70:0/1660904877 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf40054760 con 0x7faf481f3160 2023-10-08T05:36:35.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.343+0000 7faf4c938640 1 -- 172.21.15.70:0/1660904877 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf480ca070 con 0x7faf2806ae70 2023-10-08T05:36:35.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.377+0000 7faf3f7fe640 1 -- 172.21.15.70:0/1660904877 <== mgr.4105 v2:172.21.15.181:6832/2737793492 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345086 (secure 0 0 0) 0x7faf380053b0 con 0x7faf2806ae70 2023-10-08T05:36:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:35.386 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:36:35.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 -- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7faf2806ae70 msgr2=0x7faf2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7faf2806ae70 0x7faf2806d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faf38033000 tx=0x7faf38009030 comp rx=0 tx=0).stop 2023-10-08T05:36:35.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 -- 172.21.15.70:0/1660904877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf481f3160 msgr2=0x7faf48156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf481f3160 0x7faf48156a80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faf40001330 tx=0x7faf4000b4a0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 -- 172.21.15.70:0/1660904877 shutdown_connections 2023-10-08T05:36:35.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7faf2806ae70 0x7faf2806d330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf481f3160 0x7faf48156a80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faf481f6700 0x7faf480b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 --2- 172.21.15.70:0/1660904877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faf340039a0 0x7faf480b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 -- 172.21.15.70:0/1660904877 >> 172.21.15.70:0/1660904877 conn(0x7faf480cb0c0 msgr2=0x7faf4806cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:35.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 -- 172.21.15.70:0/1660904877 shutdown_connections 2023-10-08T05:36:35.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.383+0000 7faf4c938640 1 -- 172.21.15.70:0/1660904877 wait complete. 2023-10-08T05:36:35.405 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-10-08T05:36:35.077806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236096,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738624,"statfs":{"total":773094113280,"available":772852350976,"internally_reserved":0,"allocated":3383296,"data_stored":2025622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004840"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205680+0000","last_change":"2023-10-08T05:36:11.183116+0000","last_active":"2023-10-08T05:36:16.205680+0000","last_peered":"2023-10-08T05:36:16.205680+0000","last_clean":"2023-10-08T05:36:16.205680+0000","last_became_active":"2023-10-08T05:36:11.182654+0000","last_became_peered":"2023-10-08T05:36:11.182654+0000","last_unstale":"2023-10-08T05:36:16.205680+0000","last_undegraded":"2023-10-08T05:36:16.205680+0000","last_fullsized":"2023-10-08T05:36:16.205680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:49.167371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198007+0000","last_change":"2023-10-08T05:36:11.938929+0000","last_active":"2023-10-08T05:36:16.198007+0000","last_peered":"2023-10-08T05:36:16.198007+0000","last_clean":"2023-10-08T05:36:16.198007+0000","last_became_active":"2023-10-08T05:36:11.938616+0000","last_became_peered":"2023-10-08T05:36:11.938616+0000","last_unstale":"2023-10-08T05:36:16.198007+0000","last_undegraded":"2023-10-08T05:36:16.198007+0000","last_fullsized":"2023-10-08T05:36:16.198007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.911637+0000","last_change":"2023-10-08T05:36:16.241991+0000","last_active":"2023-10-08T05:36:16.911637+0000","last_peered":"2023-10-08T05:36:16.911637+0000","last_clean":"2023-10-08T05:36:16.911637+0000","last_became_active":"2023-10-08T05:36:16.241629+0000","last_became_peered":"2023-10-08T05:36:16.241629+0000","last_unstale":"2023-10-08T05:36:16.911637+0000","last_undegraded":"2023-10-08T05:36:16.911637+0000","last_fullsized":"2023-10-08T05:36:16.911637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:41.216984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198078+0000","last_change":"2023-10-08T05:36:11.171819+0000","last_active":"2023-10-08T05:36:16.198078+0000","last_peered":"2023-10-08T05:36:16.198078+0000","last_clean":"2023-10-08T05:36:16.198078+0000","last_became_active":"2023-10-08T05:36:11.171384+0000","last_became_peered":"2023-10-08T05:36:11.171384+0000","last_unstale":"2023-10-08T05:36:16.198078+0000","last_undegraded":"2023-10-08T05:36:16.198078+0000","last_fullsized":"2023-10-08T05:36:16.198078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:08.838858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205587+0000","last_change":"2023-10-08T05:36:11.939107+0000","last_active":"2023-10-08T05:36:16.205587+0000","last_peered":"2023-10-08T05:36:16.205587+0000","last_clean":"2023-10-08T05:36:16.205587+0000","last_became_active":"2023-10-08T05:36:11.938788+0000","last_became_peered":"2023-10-08T05:36:11.938788+0000","last_unstale":"2023-10-08T05:36:16.205587+0000","last_undegraded":"2023-10-08T05:36:16.205587+0000","last_fullsized":"2023-10-08T05:36:16.205587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450376+0000","last_change":"2023-10-08T05:36:16.220918+0000","last_active":"2023-10-08T05:36:18.450376+0000","last_peered":"2023-10-08T05:36:18.450376+0000","last_clean":"2023-10-08T05:36:18.450376+0000","last_became_active":"2023-10-08T05:36:16.220450+0000","last_became_peered":"2023-10-08T05:36:16.220450+0000","last_unstale":"2023-10-08T05:36:18.450376+0000","last_undegraded":"2023-10-08T05:36:18.450376+0000","last_fullsized":"2023-10-08T05:36:18.450376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:45.730636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239438+0000","last_change":"2023-10-08T05:36:11.187132+0000","last_active":"2023-10-08T05:36:16.239438+0000","last_peered":"2023-10-08T05:36:16.239438+0000","last_clean":"2023-10-08T05:36:16.239438+0000","last_became_active":"2023-10-08T05:36:11.186856+0000","last_became_peered":"2023-10-08T05:36:11.186856+0000","last_unstale":"2023-10-08T05:36:16.239438+0000","last_undegraded":"2023-10-08T05:36:16.239438+0000","last_fullsized":"2023-10-08T05:36:16.239438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:42.738249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215030+0000","last_change":"2023-10-08T05:36:11.946617+0000","last_active":"2023-10-08T05:36:16.215030+0000","last_peered":"2023-10-08T05:36:16.215030+0000","last_clean":"2023-10-08T05:36:16.215030+0000","last_became_active":"2023-10-08T05:36:11.946360+0000","last_became_peered":"2023-10-08T05:36:11.946360+0000","last_unstale":"2023-10-08T05:36:16.215030+0000","last_undegraded":"2023-10-08T05:36:16.215030+0000","last_fullsized":"2023-10-08T05:36:16.215030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:21.717397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450713+0000","last_change":"2023-10-08T05:36:16.229002+0000","last_active":"2023-10-08T05:36:18.450713+0000","last_peered":"2023-10-08T05:36:18.450713+0000","last_clean":"2023-10-08T05:36:18.450713+0000","last_became_active":"2023-10-08T05:36:16.228230+0000","last_became_peered":"2023-10-08T05:36:16.228230+0000","last_unstale":"2023-10-08T05:36:18.450713+0000","last_undegraded":"2023-10-08T05:36:18.450713+0000","last_fullsized":"2023-10-08T05:36:18.450713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:33.520862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239150+0000","last_change":"2023-10-08T05:36:11.191776+0000","last_active":"2023-10-08T05:36:16.239150+0000","last_peered":"2023-10-08T05:36:16.239150+0000","last_clean":"2023-10-08T05:36:16.239150+0000","last_became_active":"2023-10-08T05:36:11.191532+0000","last_became_peered":"2023-10-08T05:36:11.191532+0000","last_unstale":"2023-10-08T05:36:16.239150+0000","last_undegraded":"2023-10-08T05:36:16.239150+0000","last_fullsized":"2023-10-08T05:36:16.239150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:16.799817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.217226+0000","last_change":"2023-10-08T05:36:11.950652+0000","last_active":"2023-10-08T05:36:16.217226+0000","last_peered":"2023-10-08T05:36:16.217226+0000","last_clean":"2023-10-08T05:36:16.217226+0000","last_became_active":"2023-10-08T05:36:11.950359+0000","last_became_peered":"2023-10-08T05:36:11.950359+0000","last_unstale":"2023-10-08T05:36:16.217226+0000","last_undegraded":"2023-10-08T05:36:16.217226+0000","last_fullsized":"2023-10-08T05:36:16.217226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451049+0000","last_change":"2023-10-08T05:36:16.222866+0000","last_active":"2023-10-08T05:36:18.451049+0000","last_peered":"2023-10-08T05:36:18.451049+0000","last_clean":"2023-10-08T05:36:18.451049+0000","last_became_active":"2023-10-08T05:36:16.222444+0000","last_became_peered":"2023-10-08T05:36:16.222444+0000","last_unstale":"2023-10-08T05:36:18.451049+0000","last_undegraded":"2023-10-08T05:36:18.451049+0000","last_fullsized":"2023-10-08T05:36:18.451049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:04.460179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195161+0000","last_change":"2023-10-08T05:36:11.936259+0000","last_active":"2023-10-08T05:36:16.195161+0000","last_peered":"2023-10-08T05:36:16.195161+0000","last_clean":"2023-10-08T05:36:16.195161+0000","last_became_active":"2023-10-08T05:36:11.935923+0000","last_became_peered":"2023-10-08T05:36:11.935923+0000","last_unstale":"2023-10-08T05:36:16.195161+0000","last_undegraded":"2023-10-08T05:36:16.195161+0000","last_fullsized":"2023-10-08T05:36:16.195161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239194+0000","last_change":"2023-10-08T05:36:11.187292+0000","last_active":"2023-10-08T05:36:16.239194+0000","last_peered":"2023-10-08T05:36:16.239194+0000","last_clean":"2023-10-08T05:36:16.239194+0000","last_became_active":"2023-10-08T05:36:11.186706+0000","last_became_peered":"2023-10-08T05:36:11.186706+0000","last_unstale":"2023-10-08T05:36:16.239194+0000","last_undegraded":"2023-10-08T05:36:16.239194+0000","last_fullsized":"2023-10-08T05:36:16.239194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:22.361671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451570+0000","last_change":"2023-10-08T05:36:16.228736+0000","last_active":"2023-10-08T05:36:18.451570+0000","last_peered":"2023-10-08T05:36:18.451570+0000","last_clean":"2023-10-08T05:36:18.451570+0000","last_became_active":"2023-10-08T05:36:16.227909+0000","last_became_peered":"2023-10-08T05:36:16.227909+0000","last_unstale":"2023-10-08T05:36:18.451570+0000","last_undegraded":"2023-10-08T05:36:18.451570+0000","last_fullsized":"2023-10-08T05:36:18.451570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:29.122804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194544+0000","last_change":"2023-10-08T05:36:11.945361+0000","last_active":"2023-10-08T05:36:16.194544+0000","last_peered":"2023-10-08T05:36:16.194544+0000","last_clean":"2023-10-08T05:36:16.194544+0000","last_became_active":"2023-10-08T05:36:11.944819+0000","last_became_peered":"2023-10-08T05:36:11.944819+0000","last_unstale":"2023-10-08T05:36:16.194544+0000","last_undegraded":"2023-10-08T05:36:16.194544+0000","last_fullsized":"2023-10-08T05:36:16.194544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:56.745041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.208276+0000","last_change":"2023-10-08T05:36:11.164749+0000","last_active":"2023-10-08T05:36:16.208276+0000","last_peered":"2023-10-08T05:36:16.208276+0000","last_clean":"2023-10-08T05:36:16.208276+0000","last_became_active":"2023-10-08T05:36:11.164382+0000","last_became_peered":"2023-10-08T05:36:11.164382+0000","last_unstale":"2023-10-08T05:36:16.208276+0000","last_undegraded":"2023-10-08T05:36:16.208276+0000","last_fullsized":"2023-10-08T05:36:16.208276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:13.509993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450966+0000","last_change":"2023-10-08T05:36:16.229430+0000","last_active":"2023-10-08T05:36:18.450966+0000","last_peered":"2023-10-08T05:36:18.450966+0000","last_clean":"2023-10-08T05:36:18.450966+0000","last_became_active":"2023-10-08T05:36:16.228957+0000","last_became_peered":"2023-10-08T05:36:16.228957+0000","last_unstale":"2023-10-08T05:36:18.450966+0000","last_undegraded":"2023-10-08T05:36:18.450966+0000","last_fullsized":"2023-10-08T05:36:18.450966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:00.782220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195462+0000","last_change":"2023-10-08T05:36:11.934612+0000","last_active":"2023-10-08T05:36:16.195462+0000","last_peered":"2023-10-08T05:36:16.195462+0000","last_clean":"2023-10-08T05:36:16.195462+0000","last_became_active":"2023-10-08T05:36:11.934284+0000","last_became_peered":"2023-10-08T05:36:11.934284+0000","last_unstale":"2023-10-08T05:36:16.195462+0000","last_undegraded":"2023-10-08T05:36:16.195462+0000","last_fullsized":"2023-10-08T05:36:16.195462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203211+0000","last_change":"2023-10-08T05:36:11.182216+0000","last_active":"2023-10-08T05:36:16.203211+0000","last_peered":"2023-10-08T05:36:16.203211+0000","last_clean":"2023-10-08T05:36:16.203211+0000","last_became_active":"2023-10-08T05:36:11.181927+0000","last_became_peered":"2023-10-08T05:36:11.181927+0000","last_unstale":"2023-10-08T05:36:16.203211+0000","last_undegraded":"2023-10-08T05:36:16.203211+0000","last_fullsized":"2023-10-08T05:36:16.203211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:42.738249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.917366+0000","last_change":"2023-10-08T05:36:16.243897+0000","last_active":"2023-10-08T05:36:16.917366+0000","last_peered":"2023-10-08T05:36:16.917366+0000","last_clean":"2023-10-08T05:36:16.917366+0000","last_became_active":"2023-10-08T05:36:16.243421+0000","last_became_peered":"2023-10-08T05:36:16.243421+0000","last_unstale":"2023-10-08T05:36:16.917366+0000","last_undegraded":"2023-10-08T05:36:16.917366+0000","last_fullsized":"2023-10-08T05:36:16.917366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:16.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.209892+0000","last_change":"2023-10-08T05:36:11.934209+0000","last_active":"2023-10-08T05:36:16.209892+0000","last_peered":"2023-10-08T05:36:16.209892+0000","last_clean":"2023-10-08T05:36:16.209892+0000","last_became_active":"2023-10-08T05:36:11.933067+0000","last_became_peered":"2023-10-08T05:36:11.933067+0000","last_unstale":"2023-10-08T05:36:16.209892+0000","last_undegraded":"2023-10-08T05:36:16.209892+0000","last_fullsized":"2023-10-08T05:36:16.209892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:17.827053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238717+0000","last_change":"2023-10-08T05:36:11.181498+0000","last_active":"2023-10-08T05:36:16.238717+0000","last_peered":"2023-10-08T05:36:16.238717+0000","last_clean":"2023-10-08T05:36:16.238717+0000","last_became_active":"2023-10-08T05:36:11.181264+0000","last_became_peered":"2023-10-08T05:36:11.181264+0000","last_unstale":"2023-10-08T05:36:16.238717+0000","last_undegraded":"2023-10-08T05:36:16.238717+0000","last_fullsized":"2023-10-08T05:36:16.238717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:35.005158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.452011+0000","last_change":"2023-10-08T05:36:16.228065+0000","last_active":"2023-10-08T05:36:18.452011+0000","last_peered":"2023-10-08T05:36:18.452011+0000","last_clean":"2023-10-08T05:36:18.452011+0000","last_became_active":"2023-10-08T05:36:16.227665+0000","last_became_peered":"2023-10-08T05:36:16.227665+0000","last_unstale":"2023-10-08T05:36:18.452011+0000","last_undegraded":"2023-10-08T05:36:18.452011+0000","last_fullsized":"2023-10-08T05:36:18.452011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:32.619157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205947+0000","last_change":"2023-10-08T05:36:11.934150+0000","last_active":"2023-10-08T05:36:16.205947+0000","last_peered":"2023-10-08T05:36:16.205947+0000","last_clean":"2023-10-08T05:36:16.205947+0000","last_became_active":"2023-10-08T05:36:11.933875+0000","last_became_peered":"2023-10-08T05:36:11.933875+0000","last_unstale":"2023-10-08T05:36:16.205947+0000","last_undegraded":"2023-10-08T05:36:16.205947+0000","last_fullsized":"2023-10-08T05:36:16.205947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196218+0000","last_change":"2023-10-08T05:36:11.187989+0000","last_active":"2023-10-08T05:36:16.196218+0000","last_peered":"2023-10-08T05:36:16.196218+0000","last_clean":"2023-10-08T05:36:16.196218+0000","last_became_active":"2023-10-08T05:36:11.187695+0000","last_became_peered":"2023-10-08T05:36:11.187695+0000","last_unstale":"2023-10-08T05:36:16.196218+0000","last_undegraded":"2023-10-08T05:36:16.196218+0000","last_fullsized":"2023-10-08T05:36:16.196218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:35.005158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451149+0000","last_change":"2023-10-08T05:36:16.244139+0000","last_active":"2023-10-08T05:36:18.451149+0000","last_peered":"2023-10-08T05:36:18.451149+0000","last_clean":"2023-10-08T05:36:18.451149+0000","last_became_active":"2023-10-08T05:36:16.243569+0000","last_became_peered":"2023-10-08T05:36:16.243569+0000","last_unstale":"2023-10-08T05:36:18.451149+0000","last_undegraded":"2023-10-08T05:36:18.451149+0000","last_fullsized":"2023-10-08T05:36:18.451149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:30.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198170+0000","last_change":"2023-10-08T05:36:11.941731+0000","last_active":"2023-10-08T05:36:16.198170+0000","last_peered":"2023-10-08T05:36:16.198170+0000","last_clean":"2023-10-08T05:36:16.198170+0000","last_became_active":"2023-10-08T05:36:11.941393+0000","last_became_peered":"2023-10-08T05:36:11.941393+0000","last_unstale":"2023-10-08T05:36:16.198170+0000","last_undegraded":"2023-10-08T05:36:16.198170+0000","last_fullsized":"2023-10-08T05:36:16.198170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207549+0000","last_change":"2023-10-08T05:36:11.180784+0000","last_active":"2023-10-08T05:36:16.207549+0000","last_peered":"2023-10-08T05:36:16.207549+0000","last_clean":"2023-10-08T05:36:16.207549+0000","last_became_active":"2023-10-08T05:36:11.180398+0000","last_became_peered":"2023-10-08T05:36:11.180398+0000","last_unstale":"2023-10-08T05:36:16.207549+0000","last_undegraded":"2023-10-08T05:36:16.207549+0000","last_fullsized":"2023-10-08T05:36:16.207549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:49.167371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913389+0000","last_change":"2023-10-08T05:36:16.240627+0000","last_active":"2023-10-08T05:36:16.913389+0000","last_peered":"2023-10-08T05:36:16.913389+0000","last_clean":"2023-10-08T05:36:16.913389+0000","last_became_active":"2023-10-08T05:36:16.240153+0000","last_became_peered":"2023-10-08T05:36:16.240153+0000","last_unstale":"2023-10-08T05:36:16.913389+0000","last_undegraded":"2023-10-08T05:36:16.913389+0000","last_fullsized":"2023-10-08T05:36:16.913389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:06.106435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207447+0000","last_change":"2023-10-08T05:36:11.941834+0000","last_active":"2023-10-08T05:36:16.207447+0000","last_peered":"2023-10-08T05:36:16.207447+0000","last_clean":"2023-10-08T05:36:16.207447+0000","last_became_active":"2023-10-08T05:36:11.941546+0000","last_became_peered":"2023-10-08T05:36:11.941546+0000","last_unstale":"2023-10-08T05:36:16.207447+0000","last_undegraded":"2023-10-08T05:36:16.207447+0000","last_fullsized":"2023-10-08T05:36:16.207447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239749+0000","last_change":"2023-10-08T05:36:11.182530+0000","last_active":"2023-10-08T05:36:16.239749+0000","last_peered":"2023-10-08T05:36:16.239749+0000","last_clean":"2023-10-08T05:36:16.239749+0000","last_became_active":"2023-10-08T05:36:11.182156+0000","last_became_peered":"2023-10-08T05:36:11.182156+0000","last_unstale":"2023-10-08T05:36:16.239749+0000","last_undegraded":"2023-10-08T05:36:16.239749+0000","last_fullsized":"2023-10-08T05:36:16.239749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:10.731974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449389+0000","last_change":"2023-10-08T05:36:16.240659+0000","last_active":"2023-10-08T05:36:18.449389+0000","last_peered":"2023-10-08T05:36:18.449389+0000","last_clean":"2023-10-08T05:36:18.449389+0000","last_became_active":"2023-10-08T05:36:16.240244+0000","last_became_peered":"2023-10-08T05:36:16.240244+0000","last_unstale":"2023-10-08T05:36:18.449389+0000","last_undegraded":"2023-10-08T05:36:18.449389+0000","last_fullsized":"2023-10-08T05:36:18.449389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:15.664459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.216378+0000","last_change":"2023-10-08T05:36:11.943995+0000","last_active":"2023-10-08T05:36:16.216378+0000","last_peered":"2023-10-08T05:36:16.216378+0000","last_clean":"2023-10-08T05:36:16.216378+0000","last_became_active":"2023-10-08T05:36:11.943601+0000","last_became_peered":"2023-10-08T05:36:11.943601+0000","last_unstale":"2023-10-08T05:36:16.216378+0000","last_undegraded":"2023-10-08T05:36:16.216378+0000","last_fullsized":"2023-10-08T05:36:16.216378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:28.581977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207661+0000","last_change":"2023-10-08T05:36:11.166965+0000","last_active":"2023-10-08T05:36:16.207661+0000","last_peered":"2023-10-08T05:36:16.207661+0000","last_clean":"2023-10-08T05:36:16.207661+0000","last_became_active":"2023-10-08T05:36:11.166704+0000","last_became_peered":"2023-10-08T05:36:11.166704+0000","last_unstale":"2023-10-08T05:36:16.207661+0000","last_undegraded":"2023-10-08T05:36:16.207661+0000","last_fullsized":"2023-10-08T05:36:16.207661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.915670+0000","last_change":"2023-10-08T05:36:16.226001+0000","last_active":"2023-10-08T05:36:16.915670+0000","last_peered":"2023-10-08T05:36:16.915670+0000","last_clean":"2023-10-08T05:36:16.915670+0000","last_became_active":"2023-10-08T05:36:16.225623+0000","last_became_peered":"2023-10-08T05:36:16.225623+0000","last_unstale":"2023-10-08T05:36:16.915670+0000","last_undegraded":"2023-10-08T05:36:16.915670+0000","last_fullsized":"2023-10-08T05:36:16.915670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213447+0000","last_change":"2023-10-08T05:36:11.935396+0000","last_active":"2023-10-08T05:36:16.213447+0000","last_peered":"2023-10-08T05:36:16.213447+0000","last_clean":"2023-10-08T05:36:16.213447+0000","last_became_active":"2023-10-08T05:36:11.934642+0000","last_became_peered":"2023-10-08T05:36:11.934642+0000","last_unstale":"2023-10-08T05:36:16.213447+0000","last_undegraded":"2023-10-08T05:36:16.213447+0000","last_fullsized":"2023-10-08T05:36:16.213447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:09.850609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215985+0000","last_change":"2023-10-08T05:36:11.189030+0000","last_active":"2023-10-08T05:36:16.215985+0000","last_peered":"2023-10-08T05:36:16.215985+0000","last_clean":"2023-10-08T05:36:16.215985+0000","last_became_active":"2023-10-08T05:36:11.184720+0000","last_became_peered":"2023-10-08T05:36:11.184720+0000","last_unstale":"2023-10-08T05:36:16.215985+0000","last_undegraded":"2023-10-08T05:36:16.215985+0000","last_fullsized":"2023-10-08T05:36:16.215985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:41.625077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.916396+0000","last_change":"2023-10-08T05:36:16.223249+0000","last_active":"2023-10-08T05:36:16.916396+0000","last_peered":"2023-10-08T05:36:16.916396+0000","last_clean":"2023-10-08T05:36:16.916396+0000","last_became_active":"2023-10-08T05:36:16.222656+0000","last_became_peered":"2023-10-08T05:36:16.222656+0000","last_unstale":"2023-10-08T05:36:16.916396+0000","last_undegraded":"2023-10-08T05:36:16.916396+0000","last_fullsized":"2023-10-08T05:36:16.916396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:25.754577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203206+0000","last_change":"2023-10-08T05:36:11.942804+0000","last_active":"2023-10-08T05:36:16.203206+0000","last_peered":"2023-10-08T05:36:16.203206+0000","last_clean":"2023-10-08T05:36:16.203206+0000","last_became_active":"2023-10-08T05:36:11.942486+0000","last_became_peered":"2023-10-08T05:36:11.942486+0000","last_unstale":"2023-10-08T05:36:16.203206+0000","last_undegraded":"2023-10-08T05:36:16.203206+0000","last_fullsized":"2023-10-08T05:36:16.203206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190110+0000","last_change":"2023-10-08T05:36:11.160795+0000","last_active":"2023-10-08T05:36:16.190110+0000","last_peered":"2023-10-08T05:36:16.190110+0000","last_clean":"2023-10-08T05:36:16.190110+0000","last_became_active":"2023-10-08T05:36:11.160498+0000","last_became_peered":"2023-10-08T05:36:11.160498+0000","last_unstale":"2023-10-08T05:36:16.190110+0000","last_undegraded":"2023-10-08T05:36:16.190110+0000","last_fullsized":"2023-10-08T05:36:16.190110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:08.847794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.917236+0000","last_change":"2023-10-08T05:36:16.224778+0000","last_active":"2023-10-08T05:36:16.917236+0000","last_peered":"2023-10-08T05:36:16.917236+0000","last_clean":"2023-10-08T05:36:16.917236+0000","last_became_active":"2023-10-08T05:36:16.224249+0000","last_became_peered":"2023-10-08T05:36:16.224249+0000","last_unstale":"2023-10-08T05:36:16.917236+0000","last_undegraded":"2023-10-08T05:36:16.917236+0000","last_fullsized":"2023-10-08T05:36:16.917236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:07.031678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.204855+0000","last_change":"2023-10-08T05:36:11.944248+0000","last_active":"2023-10-08T05:36:16.204855+0000","last_peered":"2023-10-08T05:36:16.204855+0000","last_clean":"2023-10-08T05:36:16.204855+0000","last_became_active":"2023-10-08T05:36:11.943941+0000","last_became_peered":"2023-10-08T05:36:11.943941+0000","last_unstale":"2023-10-08T05:36:16.204855+0000","last_undegraded":"2023-10-08T05:36:16.204855+0000","last_fullsized":"2023-10-08T05:36:16.204855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238762+0000","last_change":"2023-10-08T05:36:11.186476+0000","last_active":"2023-10-08T05:36:16.238762+0000","last_peered":"2023-10-08T05:36:16.238762+0000","last_clean":"2023-10-08T05:36:16.238762+0000","last_became_active":"2023-10-08T05:36:11.186165+0000","last_became_peered":"2023-10-08T05:36:11.186165+0000","last_unstale":"2023-10-08T05:36:16.238762+0000","last_undegraded":"2023-10-08T05:36:16.238762+0000","last_fullsized":"2023-10-08T05:36:16.238762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:08.838858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450006+0000","last_change":"2023-10-08T05:36:16.241072+0000","last_active":"2023-10-08T05:36:18.450006+0000","last_peered":"2023-10-08T05:36:18.450006+0000","last_clean":"2023-10-08T05:36:18.450006+0000","last_became_active":"2023-10-08T05:36:16.240154+0000","last_became_peered":"2023-10-08T05:36:16.240154+0000","last_unstale":"2023-10-08T05:36:18.450006+0000","last_undegraded":"2023-10-08T05:36:18.450006+0000","last_fullsized":"2023-10-08T05:36:18.450006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:16.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194060+0000","last_change":"2023-10-08T05:36:11.936616+0000","last_active":"2023-10-08T05:36:16.194060+0000","last_peered":"2023-10-08T05:36:16.194060+0000","last_clean":"2023-10-08T05:36:16.194060+0000","last_became_active":"2023-10-08T05:36:11.936320+0000","last_became_peered":"2023-10-08T05:36:11.936320+0000","last_unstale":"2023-10-08T05:36:16.194060+0000","last_undegraded":"2023-10-08T05:36:16.194060+0000","last_fullsized":"2023-10-08T05:36:16.194060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215473+0000","last_change":"2023-10-08T05:36:11.182263+0000","last_active":"2023-10-08T05:36:16.215473+0000","last_peered":"2023-10-08T05:36:16.215473+0000","last_clean":"2023-10-08T05:36:16.215473+0000","last_became_active":"2023-10-08T05:36:11.181876+0000","last_became_peered":"2023-10-08T05:36:11.181876+0000","last_unstale":"2023-10-08T05:36:16.215473+0000","last_undegraded":"2023-10-08T05:36:16.215473+0000","last_fullsized":"2023-10-08T05:36:16.215473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:16.815302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238841+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:36:16.238841+0000","last_peered":"2023-10-08T05:36:16.238841+0000","last_clean":"2023-10-08T05:36:16.238841+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:36:16.238841+0000","last_undegraded":"2023-10-08T05:36:16.238841+0000","last_fullsized":"2023-10-08T05:36:16.238841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914962+0000","last_change":"2023-10-08T05:36:16.241252+0000","last_active":"2023-10-08T05:36:16.914962+0000","last_peered":"2023-10-08T05:36:16.914962+0000","last_clean":"2023-10-08T05:36:16.914962+0000","last_became_active":"2023-10-08T05:36:16.240543+0000","last_became_peered":"2023-10-08T05:36:16.240543+0000","last_unstale":"2023-10-08T05:36:16.914962+0000","last_undegraded":"2023-10-08T05:36:16.914962+0000","last_fullsized":"2023-10-08T05:36:16.914962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:20.640196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239326+0000","last_change":"2023-10-08T05:36:11.954327+0000","last_active":"2023-10-08T05:36:16.239326+0000","last_peered":"2023-10-08T05:36:16.239326+0000","last_clean":"2023-10-08T05:36:16.239326+0000","last_became_active":"2023-10-08T05:36:11.953852+0000","last_became_peered":"2023-10-08T05:36:11.953852+0000","last_unstale":"2023-10-08T05:36:16.239326+0000","last_undegraded":"2023-10-08T05:36:16.239326+0000","last_fullsized":"2023-10-08T05:36:16.239326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:12.498480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190495+0000","last_change":"2023-10-08T05:36:11.179188+0000","last_active":"2023-10-08T05:36:16.190495+0000","last_peered":"2023-10-08T05:36:16.190495+0000","last_clean":"2023-10-08T05:36:16.190495+0000","last_became_active":"2023-10-08T05:36:11.178872+0000","last_became_peered":"2023-10-08T05:36:11.178872+0000","last_unstale":"2023-10-08T05:36:16.190495+0000","last_undegraded":"2023-10-08T05:36:16.190495+0000","last_fullsized":"2023-10-08T05:36:16.190495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:44.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.915127+0000","last_change":"2023-10-08T05:36:16.241452+0000","last_active":"2023-10-08T05:36:16.915127+0000","last_peered":"2023-10-08T05:36:16.915127+0000","last_clean":"2023-10-08T05:36:16.915127+0000","last_became_active":"2023-10-08T05:36:16.240541+0000","last_became_peered":"2023-10-08T05:36:16.240541+0000","last_unstale":"2023-10-08T05:36:16.915127+0000","last_undegraded":"2023-10-08T05:36:16.915127+0000","last_fullsized":"2023-10-08T05:36:16.915127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215138+0000","last_change":"2023-10-08T05:36:11.948239+0000","last_active":"2023-10-08T05:36:16.215138+0000","last_peered":"2023-10-08T05:36:16.215138+0000","last_clean":"2023-10-08T05:36:16.215138+0000","last_became_active":"2023-10-08T05:36:11.947960+0000","last_became_peered":"2023-10-08T05:36:11.947960+0000","last_unstale":"2023-10-08T05:36:16.215138+0000","last_undegraded":"2023-10-08T05:36:16.215138+0000","last_fullsized":"2023-10-08T05:36:16.215138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:02.994498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207516+0000","last_change":"2023-10-08T05:36:11.173210+0000","last_active":"2023-10-08T05:36:16.207516+0000","last_peered":"2023-10-08T05:36:16.207516+0000","last_clean":"2023-10-08T05:36:16.207516+0000","last_became_active":"2023-10-08T05:36:11.172899+0000","last_became_peered":"2023-10-08T05:36:11.172899+0000","last_unstale":"2023-10-08T05:36:16.207516+0000","last_undegraded":"2023-10-08T05:36:16.207516+0000","last_fullsized":"2023-10-08T05:36:16.207516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:59.559506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449356+0000","last_change":"2023-10-08T05:36:16.226571+0000","last_active":"2023-10-08T05:36:18.449356+0000","last_peered":"2023-10-08T05:36:18.449356+0000","last_clean":"2023-10-08T05:36:18.449356+0000","last_became_active":"2023-10-08T05:36:16.226098+0000","last_became_peered":"2023-10-08T05:36:16.226098+0000","last_unstale":"2023-10-08T05:36:18.449356+0000","last_undegraded":"2023-10-08T05:36:18.449356+0000","last_fullsized":"2023-10-08T05:36:18.449356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.246459+0000","last_change":"2023-10-08T05:36:11.948021+0000","last_active":"2023-10-08T05:36:16.246459+0000","last_peered":"2023-10-08T05:36:16.246459+0000","last_clean":"2023-10-08T05:36:16.246459+0000","last_became_active":"2023-10-08T05:36:11.947456+0000","last_became_peered":"2023-10-08T05:36:11.947456+0000","last_unstale":"2023-10-08T05:36:16.246459+0000","last_undegraded":"2023-10-08T05:36:16.246459+0000","last_fullsized":"2023-10-08T05:36:16.246459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:21.717397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.191082+0000","last_change":"2023-10-08T05:36:11.164907+0000","last_active":"2023-10-08T05:36:16.191082+0000","last_peered":"2023-10-08T05:36:16.191082+0000","last_clean":"2023-10-08T05:36:16.191082+0000","last_became_active":"2023-10-08T05:36:11.164630+0000","last_became_peered":"2023-10-08T05:36:11.164630+0000","last_unstale":"2023-10-08T05:36:16.191082+0000","last_undegraded":"2023-10-08T05:36:16.191082+0000","last_fullsized":"2023-10-08T05:36:16.191082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:58.268025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.347745+0000","last_change":"2023-10-08T05:36:16.241024+0000","last_active":"2023-10-08T05:36:17.347745+0000","last_peered":"2023-10-08T05:36:17.347745+0000","last_clean":"2023-10-08T05:36:17.347745+0000","last_became_active":"2023-10-08T05:36:16.240379+0000","last_became_peered":"2023-10-08T05:36:16.240379+0000","last_unstale":"2023-10-08T05:36:17.347745+0000","last_undegraded":"2023-10-08T05:36:17.347745+0000","last_fullsized":"2023-10-08T05:36:17.347745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:09:12.638961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240155+0000","last_change":"2023-10-08T05:36:11.946599+0000","last_active":"2023-10-08T05:36:16.240155+0000","last_peered":"2023-10-08T05:36:16.240155+0000","last_clean":"2023-10-08T05:36:16.240155+0000","last_became_active":"2023-10-08T05:36:11.946295+0000","last_became_peered":"2023-10-08T05:36:11.946295+0000","last_unstale":"2023-10-08T05:36:16.240155+0000","last_undegraded":"2023-10-08T05:36:16.240155+0000","last_fullsized":"2023-10-08T05:36:16.240155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:28.581977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214835+0000","last_change":"2023-10-08T05:36:11.181737+0000","last_active":"2023-10-08T05:36:16.214835+0000","last_peered":"2023-10-08T05:36:16.214835+0000","last_clean":"2023-10-08T05:36:16.214835+0000","last_became_active":"2023-10-08T05:36:11.181386+0000","last_became_peered":"2023-10-08T05:36:11.181386+0000","last_unstale":"2023-10-08T05:36:16.214835+0000","last_undegraded":"2023-10-08T05:36:16.214835+0000","last_fullsized":"2023-10-08T05:36:16.214835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:05.139990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450290+0000","last_change":"2023-10-08T05:36:16.223098+0000","last_active":"2023-10-08T05:36:18.450290+0000","last_peered":"2023-10-08T05:36:18.450290+0000","last_clean":"2023-10-08T05:36:18.450290+0000","last_became_active":"2023-10-08T05:36:16.222649+0000","last_became_peered":"2023-10-08T05:36:16.222649+0000","last_unstale":"2023-10-08T05:36:18.450290+0000","last_undegraded":"2023-10-08T05:36:18.450290+0000","last_fullsized":"2023-10-08T05:36:18.450290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:27.410602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238995+0000","last_change":"2023-10-08T05:36:11.946258+0000","last_active":"2023-10-08T05:36:16.238995+0000","last_peered":"2023-10-08T05:36:16.238995+0000","last_clean":"2023-10-08T05:36:16.238995+0000","last_became_active":"2023-10-08T05:36:11.945932+0000","last_became_peered":"2023-10-08T05:36:11.945932+0000","last_unstale":"2023-10-08T05:36:16.238995+0000","last_undegraded":"2023-10-08T05:36:16.238995+0000","last_fullsized":"2023-10-08T05:36:16.238995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:16.603017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.189924+0000","last_change":"2023-10-08T05:36:11.162339+0000","last_active":"2023-10-08T05:36:16.189924+0000","last_peered":"2023-10-08T05:36:16.189924+0000","last_clean":"2023-10-08T05:36:16.189924+0000","last_became_active":"2023-10-08T05:36:11.162023+0000","last_became_peered":"2023-10-08T05:36:11.162023+0000","last_unstale":"2023-10-08T05:36:16.189924+0000","last_undegraded":"2023-10-08T05:36:16.189924+0000","last_fullsized":"2023-10-08T05:36:16.189924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:10.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449880+0000","last_change":"2023-10-08T05:36:16.217935+0000","last_active":"2023-10-08T05:36:18.449880+0000","last_peered":"2023-10-08T05:36:18.449880+0000","last_clean":"2023-10-08T05:36:18.449880+0000","last_became_active":"2023-10-08T05:36:16.217530+0000","last_became_peered":"2023-10-08T05:36:16.217530+0000","last_unstale":"2023-10-08T05:36:18.449880+0000","last_undegraded":"2023-10-08T05:36:18.449880+0000","last_fullsized":"2023-10-08T05:36:18.449880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:07.031678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194389+0000","last_change":"2023-10-08T05:36:11.945050+0000","last_active":"2023-10-08T05:36:16.194389+0000","last_peered":"2023-10-08T05:36:16.194389+0000","last_clean":"2023-10-08T05:36:16.194389+0000","last_became_active":"2023-10-08T05:36:11.944419+0000","last_became_peered":"2023-10-08T05:36:11.944419+0000","last_unstale":"2023-10-08T05:36:16.194389+0000","last_undegraded":"2023-10-08T05:36:16.194389+0000","last_fullsized":"2023-10-08T05:36:16.194389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194769+0000","last_change":"2023-10-08T05:36:11.176221+0000","last_active":"2023-10-08T05:36:16.194769+0000","last_peered":"2023-10-08T05:36:16.194769+0000","last_clean":"2023-10-08T05:36:16.194769+0000","last_became_active":"2023-10-08T05:36:11.175839+0000","last_became_peered":"2023-10-08T05:36:11.175839+0000","last_unstale":"2023-10-08T05:36:16.194769+0000","last_undegraded":"2023-10-08T05:36:16.194769+0000","last_fullsized":"2023-10-08T05:36:16.194769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:44.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913652+0000","last_change":"2023-10-08T05:36:16.240543+0000","last_active":"2023-10-08T05:36:16.913652+0000","last_peered":"2023-10-08T05:36:16.913652+0000","last_clean":"2023-10-08T05:36:16.913652+0000","last_became_active":"2023-10-08T05:36:16.240104+0000","last_became_peered":"2023-10-08T05:36:16.240104+0000","last_unstale":"2023-10-08T05:36:16.913652+0000","last_undegraded":"2023-10-08T05:36:16.913652+0000","last_fullsized":"2023-10-08T05:36:16.913652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:21.958405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239177+0000","last_change":"2023-10-08T05:36:11.952574+0000","last_active":"2023-10-08T05:36:16.239177+0000","last_peered":"2023-10-08T05:36:16.239177+0000","last_clean":"2023-10-08T05:36:16.239177+0000","last_became_active":"2023-10-08T05:36:11.952287+0000","last_became_peered":"2023-10-08T05:36:11.952287+0000","last_unstale":"2023-10-08T05:36:16.239177+0000","last_undegraded":"2023-10-08T05:36:16.239177+0000","last_fullsized":"2023-10-08T05:36:16.239177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215860+0000","last_change":"2023-10-08T05:36:11.183076+0000","last_active":"2023-10-08T05:36:16.215860+0000","last_peered":"2023-10-08T05:36:16.215860+0000","last_clean":"2023-10-08T05:36:16.215860+0000","last_became_active":"2023-10-08T05:36:11.182702+0000","last_became_peered":"2023-10-08T05:36:11.182702+0000","last_unstale":"2023-10-08T05:36:16.215860+0000","last_undegraded":"2023-10-08T05:36:16.215860+0000","last_fullsized":"2023-10-08T05:36:16.215860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:45.848562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.912758+0000","last_change":"2023-10-08T05:36:16.240654+0000","last_active":"2023-10-08T05:36:16.912758+0000","last_peered":"2023-10-08T05:36:16.912758+0000","last_clean":"2023-10-08T05:36:16.912758+0000","last_became_active":"2023-10-08T05:36:16.240179+0000","last_became_peered":"2023-10-08T05:36:16.240179+0000","last_unstale":"2023-10-08T05:36:16.912758+0000","last_undegraded":"2023-10-08T05:36:16.912758+0000","last_fullsized":"2023-10-08T05:36:16.912758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:09:12.638961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205225+0000","last_change":"2023-10-08T05:36:11.948816+0000","last_active":"2023-10-08T05:36:16.205225+0000","last_peered":"2023-10-08T05:36:16.205225+0000","last_clean":"2023-10-08T05:36:16.205225+0000","last_became_active":"2023-10-08T05:36:11.948501+0000","last_became_peered":"2023-10-08T05:36:11.948501+0000","last_unstale":"2023-10-08T05:36:16.205225+0000","last_undegraded":"2023-10-08T05:36:16.205225+0000","last_fullsized":"2023-10-08T05:36:16.205225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:43.750000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206453+0000","last_change":"2023-10-08T05:36:11.172445+0000","last_active":"2023-10-08T05:36:16.206453+0000","last_peered":"2023-10-08T05:36:16.206453+0000","last_clean":"2023-10-08T05:36:16.206453+0000","last_became_active":"2023-10-08T05:36:11.172090+0000","last_became_peered":"2023-10-08T05:36:11.172090+0000","last_unstale":"2023-10-08T05:36:16.206453+0000","last_undegraded":"2023-10-08T05:36:16.206453+0000","last_fullsized":"2023-10-08T05:36:16.206453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:58.268025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.348870+0000","last_change":"2023-10-08T05:36:16.240874+0000","last_active":"2023-10-08T05:36:17.348870+0000","last_peered":"2023-10-08T05:36:17.348870+0000","last_clean":"2023-10-08T05:36:17.348870+0000","last_became_active":"2023-10-08T05:36:16.240420+0000","last_became_peered":"2023-10-08T05:36:16.240420+0000","last_unstale":"2023-10-08T05:36:17.348870+0000","last_undegraded":"2023-10-08T05:36:17.348870+0000","last_fullsized":"2023-10-08T05:36:17.348870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:06.106435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239249+0000","last_change":"2023-10-08T05:36:11.949913+0000","last_active":"2023-10-08T05:36:16.239249+0000","last_peered":"2023-10-08T05:36:16.239249+0000","last_clean":"2023-10-08T05:36:16.239249+0000","last_became_active":"2023-10-08T05:36:11.946993+0000","last_became_peered":"2023-10-08T05:36:11.946993+0000","last_unstale":"2023-10-08T05:36:16.239249+0000","last_undegraded":"2023-10-08T05:36:16.239249+0000","last_fullsized":"2023-10-08T05:36:16.239249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:02.994498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:31.450726+0000","last_change":"2023-10-08T05:36:11.177181+0000","last_active":"2023-10-08T05:36:31.450726+0000","last_peered":"2023-10-08T05:36:31.450726+0000","last_clean":"2023-10-08T05:36:31.450726+0000","last_became_active":"2023-10-08T05:36:11.176830+0000","last_became_peered":"2023-10-08T05:36:11.176830+0000","last_unstale":"2023-10-08T05:36:31.450726+0000","last_undegraded":"2023-10-08T05:36:31.450726+0000","last_fullsized":"2023-10-08T05:36:31.450726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:13.509993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.916338+0000","last_change":"2023-10-08T05:36:16.223391+0000","last_active":"2023-10-08T05:36:16.916338+0000","last_peered":"2023-10-08T05:36:16.916338+0000","last_clean":"2023-10-08T05:36:16.916338+0000","last_became_active":"2023-10-08T05:36:16.222997+0000","last_became_peered":"2023-10-08T05:36:16.222997+0000","last_unstale":"2023-10-08T05:36:16.916338+0000","last_undegraded":"2023-10-08T05:36:16.916338+0000","last_fullsized":"2023-10-08T05:36:16.916338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:15.664459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213269+0000","last_change":"2023-10-08T05:36:11.936436+0000","last_active":"2023-10-08T05:36:16.213269+0000","last_peered":"2023-10-08T05:36:16.213269+0000","last_clean":"2023-10-08T05:36:16.213269+0000","last_became_active":"2023-10-08T05:36:11.936161+0000","last_became_peered":"2023-10-08T05:36:11.936161+0000","last_unstale":"2023-10-08T05:36:16.213269+0000","last_undegraded":"2023-10-08T05:36:16.213269+0000","last_fullsized":"2023-10-08T05:36:16.213269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:36.016909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207221+0000","last_change":"2023-10-08T05:36:11.183582+0000","last_active":"2023-10-08T05:36:16.207221+0000","last_peered":"2023-10-08T05:36:16.207221+0000","last_clean":"2023-10-08T05:36:16.207221+0000","last_became_active":"2023-10-08T05:36:11.183264+0000","last_became_peered":"2023-10-08T05:36:11.183264+0000","last_unstale":"2023-10-08T05:36:16.207221+0000","last_undegraded":"2023-10-08T05:36:16.207221+0000","last_fullsized":"2023-10-08T05:36:16.207221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:16.799817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451297+0000","last_change":"2023-10-08T05:36:16.228720+0000","last_active":"2023-10-08T05:36:18.451297+0000","last_peered":"2023-10-08T05:36:18.451297+0000","last_clean":"2023-10-08T05:36:18.451297+0000","last_became_active":"2023-10-08T05:36:16.227638+0000","last_became_peered":"2023-10-08T05:36:16.227638+0000","last_unstale":"2023-10-08T05:36:18.451297+0000","last_undegraded":"2023-10-08T05:36:18.451297+0000","last_fullsized":"2023-10-08T05:36:18.451297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:20.640196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205640+0000","last_change":"2023-10-08T05:36:11.933926+0000","last_active":"2023-10-08T05:36:16.205640+0000","last_peered":"2023-10-08T05:36:16.205640+0000","last_clean":"2023-10-08T05:36:16.205640+0000","last_became_active":"2023-10-08T05:36:11.933648+0000","last_became_peered":"2023-10-08T05:36:11.933648+0000","last_unstale":"2023-10-08T05:36:16.205640+0000","last_undegraded":"2023-10-08T05:36:16.205640+0000","last_fullsized":"2023-10-08T05:36:16.205640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215215+0000","last_change":"2023-10-08T05:36:11.183379+0000","last_active":"2023-10-08T05:36:16.215215+0000","last_peered":"2023-10-08T05:36:16.215215+0000","last_clean":"2023-10-08T05:36:16.215215+0000","last_became_active":"2023-10-08T05:36:11.182862+0000","last_became_peered":"2023-10-08T05:36:11.182862+0000","last_unstale":"2023-10-08T05:36:16.215215+0000","last_undegraded":"2023-10-08T05:36:16.215215+0000","last_fullsized":"2023-10-08T05:36:16.215215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:10.147911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914439+0000","last_change":"2023-10-08T05:36:16.216716+0000","last_active":"2023-10-08T05:36:16.914439+0000","last_peered":"2023-10-08T05:36:16.914439+0000","last_clean":"2023-10-08T05:36:16.914439+0000","last_became_active":"2023-10-08T05:36:16.216266+0000","last_became_peered":"2023-10-08T05:36:16.216266+0000","last_unstale":"2023-10-08T05:36:16.914439+0000","last_undegraded":"2023-10-08T05:36:16.914439+0000","last_fullsized":"2023-10-08T05:36:16.914439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:45.158099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214937+0000","last_change":"2023-10-08T05:36:11.949702+0000","last_active":"2023-10-08T05:36:16.214937+0000","last_peered":"2023-10-08T05:36:16.214937+0000","last_clean":"2023-10-08T05:36:16.214937+0000","last_became_active":"2023-10-08T05:36:11.949421+0000","last_became_peered":"2023-10-08T05:36:11.949421+0000","last_unstale":"2023-10-08T05:36:16.214937+0000","last_undegraded":"2023-10-08T05:36:16.214937+0000","last_fullsized":"2023-10-08T05:36:16.214937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195931+0000","last_change":"2023-10-08T05:36:11.189802+0000","last_active":"2023-10-08T05:36:16.195931+0000","last_peered":"2023-10-08T05:36:16.195931+0000","last_clean":"2023-10-08T05:36:16.195931+0000","last_became_active":"2023-10-08T05:36:11.189447+0000","last_became_peered":"2023-10-08T05:36:11.189447+0000","last_unstale":"2023-10-08T05:36:16.195931+0000","last_undegraded":"2023-10-08T05:36:16.195931+0000","last_fullsized":"2023-10-08T05:36:16.195931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:08.847794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.911933+0000","last_change":"2023-10-08T05:36:16.241184+0000","last_active":"2023-10-08T05:36:16.911933+0000","last_peered":"2023-10-08T05:36:16.911933+0000","last_clean":"2023-10-08T05:36:16.911933+0000","last_became_active":"2023-10-08T05:36:16.240531+0000","last_became_peered":"2023-10-08T05:36:16.240531+0000","last_unstale":"2023-10-08T05:36:16.911933+0000","last_undegraded":"2023-10-08T05:36:16.911933+0000","last_fullsized":"2023-10-08T05:36:16.911933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:17:23.235254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449495+0000","last_change":"2023-10-08T05:36:16.241230+0000","last_active":"2023-10-08T05:36:18.449495+0000","last_peered":"2023-10-08T05:36:18.449495+0000","last_clean":"2023-10-08T05:36:18.449495+0000","last_became_active":"2023-10-08T05:36:16.240814+0000","last_became_peered":"2023-10-08T05:36:16.240814+0000","last_unstale":"2023-10-08T05:36:18.449495+0000","last_undegraded":"2023-10-08T05:36:18.449495+0000","last_fullsized":"2023-10-08T05:36:18.449495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:25.754577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.202997+0000","last_change":"2023-10-08T05:36:11.941419+0000","last_active":"2023-10-08T05:36:16.202997+0000","last_peered":"2023-10-08T05:36:16.202997+0000","last_clean":"2023-10-08T05:36:16.202997+0000","last_became_active":"2023-10-08T05:36:11.941131+0000","last_became_peered":"2023-10-08T05:36:11.941131+0000","last_unstale":"2023-10-08T05:36:16.202997+0000","last_undegraded":"2023-10-08T05:36:16.202997+0000","last_fullsized":"2023-10-08T05:36:16.202997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214657+0000","last_change":"2023-10-08T05:36:11.182594+0000","last_active":"2023-10-08T05:36:16.214657+0000","last_peered":"2023-10-08T05:36:16.214657+0000","last_clean":"2023-10-08T05:36:16.214657+0000","last_became_active":"2023-10-08T05:36:11.182297+0000","last_became_peered":"2023-10-08T05:36:11.182297+0000","last_unstale":"2023-10-08T05:36:16.214657+0000","last_undegraded":"2023-10-08T05:36:16.214657+0000","last_fullsized":"2023-10-08T05:36:16.214657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:59.799745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914977+0000","last_change":"2023-10-08T05:36:16.220333+0000","last_active":"2023-10-08T05:36:16.914977+0000","last_peered":"2023-10-08T05:36:16.914977+0000","last_clean":"2023-10-08T05:36:16.914977+0000","last_became_active":"2023-10-08T05:36:16.219836+0000","last_became_peered":"2023-10-08T05:36:16.219836+0000","last_unstale":"2023-10-08T05:36:16.914977+0000","last_undegraded":"2023-10-08T05:36:16.914977+0000","last_fullsized":"2023-10-08T05:36:16.914977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:32.619157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195436+0000","last_change":"2023-10-08T05:36:11.945676+0000","last_active":"2023-10-08T05:36:16.195436+0000","last_peered":"2023-10-08T05:36:16.195436+0000","last_clean":"2023-10-08T05:36:16.195436+0000","last_became_active":"2023-10-08T05:36:11.945314+0000","last_became_peered":"2023-10-08T05:36:11.945314+0000","last_unstale":"2023-10-08T05:36:16.195436+0000","last_undegraded":"2023-10-08T05:36:16.195436+0000","last_fullsized":"2023-10-08T05:36:16.195436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.197371+0000","last_change":"2023-10-08T05:36:11.176854+0000","last_active":"2023-10-08T05:36:16.197371+0000","last_peered":"2023-10-08T05:36:16.197371+0000","last_clean":"2023-10-08T05:36:16.197371+0000","last_became_active":"2023-10-08T05:36:11.176546+0000","last_became_peered":"2023-10-08T05:36:11.176546+0000","last_unstale":"2023-10-08T05:36:16.197371+0000","last_undegraded":"2023-10-08T05:36:16.197371+0000","last_fullsized":"2023-10-08T05:36:16.197371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:16.815302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195412+0000","last_change":"2023-10-08T05:36:11.184441+0000","last_active":"2023-10-08T05:36:16.195412+0000","last_peered":"2023-10-08T05:36:16.195412+0000","last_clean":"2023-10-08T05:36:16.195412+0000","last_became_active":"2023-10-08T05:36:11.184159+0000","last_became_peered":"2023-10-08T05:36:11.184159+0000","last_unstale":"2023-10-08T05:36:16.195412+0000","last_undegraded":"2023-10-08T05:36:16.195412+0000","last_fullsized":"2023-10-08T05:36:16.195412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206365+0000","last_change":"2023-10-08T05:36:11.935252+0000","last_active":"2023-10-08T05:36:16.206365+0000","last_peered":"2023-10-08T05:36:16.206365+0000","last_clean":"2023-10-08T05:36:16.206365+0000","last_became_active":"2023-10-08T05:36:11.934390+0000","last_became_peered":"2023-10-08T05:36:11.934390+0000","last_unstale":"2023-10-08T05:36:16.206365+0000","last_undegraded":"2023-10-08T05:36:16.206365+0000","last_fullsized":"2023-10-08T05:36:16.206365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.242390+0000","last_change":"2023-10-08T05:36:16.217984+0000","last_active":"2023-10-08T05:36:17.242390+0000","last_peered":"2023-10-08T05:36:17.242390+0000","last_clean":"2023-10-08T05:36:17.242390+0000","last_became_active":"2023-10-08T05:36:16.217614+0000","last_became_peered":"2023-10-08T05:36:16.217614+0000","last_unstale":"2023-10-08T05:36:17.242390+0000","last_undegraded":"2023-10-08T05:36:17.242390+0000","last_fullsized":"2023-10-08T05:36:17.242390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:47.787180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194733+0000","last_change":"2023-10-08T05:36:11.182558+0000","last_active":"2023-10-08T05:36:16.194733+0000","last_peered":"2023-10-08T05:36:16.194733+0000","last_clean":"2023-10-08T05:36:16.194733+0000","last_became_active":"2023-10-08T05:36:11.181822+0000","last_became_peered":"2023-10-08T05:36:11.181822+0000","last_unstale":"2023-10-08T05:36:16.194733+0000","last_undegraded":"2023-10-08T05:36:16.194733+0000","last_fullsized":"2023-10-08T05:36:16.194733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214758+0000","last_change":"2023-10-08T05:36:11.941704+0000","last_active":"2023-10-08T05:36:16.214758+0000","last_peered":"2023-10-08T05:36:16.214758+0000","last_clean":"2023-10-08T05:36:16.214758+0000","last_became_active":"2023-10-08T05:36:11.941431+0000","last_became_peered":"2023-10-08T05:36:11.941431+0000","last_unstale":"2023-10-08T05:36:16.214758+0000","last_undegraded":"2023-10-08T05:36:16.214758+0000","last_fullsized":"2023-10-08T05:36:16.214758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:00.811496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913836+0000","last_change":"2023-10-08T05:36:16.220904+0000","last_active":"2023-10-08T05:36:16.913836+0000","last_peered":"2023-10-08T05:36:16.913836+0000","last_clean":"2023-10-08T05:36:16.913836+0000","last_became_active":"2023-10-08T05:36:16.220521+0000","last_became_peered":"2023-10-08T05:36:16.220521+0000","last_unstale":"2023-10-08T05:36:16.913836+0000","last_undegraded":"2023-10-08T05:36:16.913836+0000","last_fullsized":"2023-10-08T05:36:16.913836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:52:02.322999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194151+0000","last_change":"2023-10-08T05:36:11.945315+0000","last_active":"2023-10-08T05:36:16.194151+0000","last_peered":"2023-10-08T05:36:16.194151+0000","last_clean":"2023-10-08T05:36:16.194151+0000","last_became_active":"2023-10-08T05:36:11.944775+0000","last_became_peered":"2023-10-08T05:36:11.944775+0000","last_unstale":"2023-10-08T05:36:16.194151+0000","last_undegraded":"2023-10-08T05:36:16.194151+0000","last_fullsized":"2023-10-08T05:36:16.194151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.189968+0000","last_change":"2023-10-08T05:36:11.938156+0000","last_active":"2023-10-08T05:36:16.189968+0000","last_peered":"2023-10-08T05:36:16.189968+0000","last_clean":"2023-10-08T05:36:16.189968+0000","last_became_active":"2023-10-08T05:36:11.937878+0000","last_became_peered":"2023-10-08T05:36:11.937878+0000","last_unstale":"2023-10-08T05:36:16.189968+0000","last_undegraded":"2023-10-08T05:36:16.189968+0000","last_fullsized":"2023-10-08T05:36:16.189968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:00.811496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215846+0000","last_change":"2023-10-08T05:36:11.944484+0000","last_active":"2023-10-08T05:36:16.215846+0000","last_peered":"2023-10-08T05:36:16.215846+0000","last_clean":"2023-10-08T05:36:16.215846+0000","last_became_active":"2023-10-08T05:36:11.943696+0000","last_became_peered":"2023-10-08T05:36:11.943696+0000","last_unstale":"2023-10-08T05:36:16.215846+0000","last_undegraded":"2023-10-08T05:36:16.215846+0000","last_fullsized":"2023-10-08T05:36:16.215846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:12.498480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206284+0000","last_change":"2023-10-08T05:36:11.944505+0000","last_active":"2023-10-08T05:36:16.206284+0000","last_peered":"2023-10-08T05:36:16.206284+0000","last_clean":"2023-10-08T05:36:16.206284+0000","last_became_active":"2023-10-08T05:36:11.944084+0000","last_became_peered":"2023-10-08T05:36:11.944084+0000","last_unstale":"2023-10-08T05:36:16.206284+0000","last_undegraded":"2023-10-08T05:36:16.206284+0000","last_fullsized":"2023-10-08T05:36:16.206284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.197543+0000","last_change":"2023-10-08T05:36:11.936235+0000","last_active":"2023-10-08T05:36:16.197543+0000","last_peered":"2023-10-08T05:36:16.197543+0000","last_clean":"2023-10-08T05:36:16.197543+0000","last_became_active":"2023-10-08T05:36:11.935816+0000","last_became_peered":"2023-10-08T05:36:11.935816+0000","last_unstale":"2023-10-08T05:36:16.197543+0000","last_undegraded":"2023-10-08T05:36:16.197543+0000","last_fullsized":"2023-10-08T05:36:16.197543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206800+0000","last_change":"2023-10-08T05:36:11.940291+0000","last_active":"2023-10-08T05:36:16.206800+0000","last_peered":"2023-10-08T05:36:16.206800+0000","last_clean":"2023-10-08T05:36:16.206800+0000","last_became_active":"2023-10-08T05:36:11.940019+0000","last_became_peered":"2023-10-08T05:36:11.940019+0000","last_unstale":"2023-10-08T05:36:16.206800+0000","last_undegraded":"2023-10-08T05:36:16.206800+0000","last_fullsized":"2023-10-08T05:36:16.206800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195886+0000","last_change":"2023-10-08T05:36:11.949017+0000","last_active":"2023-10-08T05:36:16.195886+0000","last_peered":"2023-10-08T05:36:16.195886+0000","last_clean":"2023-10-08T05:36:16.195886+0000","last_became_active":"2023-10-08T05:36:11.948741+0000","last_became_peered":"2023-10-08T05:36:11.948741+0000","last_unstale":"2023-10-08T05:36:16.195886+0000","last_undegraded":"2023-10-08T05:36:16.195886+0000","last_fullsized":"2023-10-08T05:36:16.195886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198027+0000","last_change":"2023-10-08T05:36:11.937078+0000","last_active":"2023-10-08T05:36:16.198027+0000","last_peered":"2023-10-08T05:36:16.198027+0000","last_clean":"2023-10-08T05:36:16.198027+0000","last_became_active":"2023-10-08T05:36:11.936737+0000","last_became_peered":"2023-10-08T05:36:11.936737+0000","last_unstale":"2023-10-08T05:36:16.198027+0000","last_undegraded":"2023-10-08T05:36:16.198027+0000","last_fullsized":"2023-10-08T05:36:16.198027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214525+0000","last_change":"2023-10-08T05:36:11.945582+0000","last_active":"2023-10-08T05:36:16.214525+0000","last_peered":"2023-10-08T05:36:16.214525+0000","last_clean":"2023-10-08T05:36:16.214525+0000","last_became_active":"2023-10-08T05:36:11.945066+0000","last_became_peered":"2023-10-08T05:36:11.945066+0000","last_unstale":"2023-10-08T05:36:16.214525+0000","last_undegraded":"2023-10-08T05:36:16.214525+0000","last_fullsized":"2023-10-08T05:36:16.214525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196604+0000","last_change":"2023-10-08T05:36:11.945936+0000","last_active":"2023-10-08T05:36:16.196604+0000","last_peered":"2023-10-08T05:36:16.196604+0000","last_clean":"2023-10-08T05:36:16.196604+0000","last_became_active":"2023-10-08T05:36:11.945384+0000","last_became_peered":"2023-10-08T05:36:11.945384+0000","last_unstale":"2023-10-08T05:36:16.196604+0000","last_undegraded":"2023-10-08T05:36:16.196604+0000","last_fullsized":"2023-10-08T05:36:16.196604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215763+0000","last_change":"2023-10-08T05:36:11.949155+0000","last_active":"2023-10-08T05:36:16.215763+0000","last_peered":"2023-10-08T05:36:16.215763+0000","last_clean":"2023-10-08T05:36:16.215763+0000","last_became_active":"2023-10-08T05:36:11.948867+0000","last_became_peered":"2023-10-08T05:36:11.948867+0000","last_unstale":"2023-10-08T05:36:16.215763+0000","last_undegraded":"2023-10-08T05:36:16.215763+0000","last_fullsized":"2023-10-08T05:36:16.215763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:11.627279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206186+0000","last_change":"2023-10-08T05:36:11.949363+0000","last_active":"2023-10-08T05:36:16.206186+0000","last_peered":"2023-10-08T05:36:16.206186+0000","last_clean":"2023-10-08T05:36:16.206186+0000","last_became_active":"2023-10-08T05:36:11.949059+0000","last_became_peered":"2023-10-08T05:36:11.949059+0000","last_unstale":"2023-10-08T05:36:16.206186+0000","last_undegraded":"2023-10-08T05:36:16.206186+0000","last_fullsized":"2023-10-08T05:36:16.206186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:00.571257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195787+0000","last_change":"2023-10-08T05:36:11.936518+0000","last_active":"2023-10-08T05:36:16.195787+0000","last_peered":"2023-10-08T05:36:16.195787+0000","last_clean":"2023-10-08T05:36:16.195787+0000","last_became_active":"2023-10-08T05:36:11.936138+0000","last_became_peered":"2023-10-08T05:36:11.936138+0000","last_unstale":"2023-10-08T05:36:16.195787+0000","last_undegraded":"2023-10-08T05:36:16.195787+0000","last_fullsized":"2023-10-08T05:36:16.195787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239530+0000","last_change":"2023-10-08T05:36:11.940992+0000","last_active":"2023-10-08T05:36:16.239530+0000","last_peered":"2023-10-08T05:36:16.239530+0000","last_clean":"2023-10-08T05:36:16.239530+0000","last_became_active":"2023-10-08T05:36:11.940700+0000","last_became_peered":"2023-10-08T05:36:11.940700+0000","last_unstale":"2023-10-08T05:36:16.239530+0000","last_undegraded":"2023-10-08T05:36:16.239530+0000","last_fullsized":"2023-10-08T05:36:16.239530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:20.449484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190822+0000","last_change":"2023-10-08T05:36:11.937136+0000","last_active":"2023-10-08T05:36:16.190822+0000","last_peered":"2023-10-08T05:36:16.190822+0000","last_clean":"2023-10-08T05:36:16.190822+0000","last_became_active":"2023-10-08T05:36:11.936761+0000","last_became_peered":"2023-10-08T05:36:11.936761+0000","last_unstale":"2023-10-08T05:36:16.190822+0000","last_undegraded":"2023-10-08T05:36:16.190822+0000","last_fullsized":"2023-10-08T05:36:16.190822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:50.179123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215403+0000","last_change":"2023-10-08T05:36:11.943099+0000","last_active":"2023-10-08T05:36:16.215403+0000","last_peered":"2023-10-08T05:36:16.215403+0000","last_clean":"2023-10-08T05:36:16.215403+0000","last_became_active":"2023-10-08T05:36:11.942792+0000","last_became_peered":"2023-10-08T05:36:11.942792+0000","last_unstale":"2023-10-08T05:36:16.215403+0000","last_undegraded":"2023-10-08T05:36:16.215403+0000","last_fullsized":"2023-10-08T05:36:16.215403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214354+0000","last_change":"2023-10-08T05:36:11.938711+0000","last_active":"2023-10-08T05:36:16.214354+0000","last_peered":"2023-10-08T05:36:16.214354+0000","last_clean":"2023-10-08T05:36:16.214354+0000","last_became_active":"2023-10-08T05:36:11.938449+0000","last_became_peered":"2023-10-08T05:36:11.938449+0000","last_unstale":"2023-10-08T05:36:16.214354+0000","last_undegraded":"2023-10-08T05:36:16.214354+0000","last_fullsized":"2023-10-08T05:36:16.214354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:56.745041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203081+0000","last_change":"2023-10-08T05:36:11.950960+0000","last_active":"2023-10-08T05:36:16.203081+0000","last_peered":"2023-10-08T05:36:16.203081+0000","last_clean":"2023-10-08T05:36:16.203081+0000","last_became_active":"2023-10-08T05:36:11.950582+0000","last_became_peered":"2023-10-08T05:36:11.950582+0000","last_unstale":"2023-10-08T05:36:16.203081+0000","last_undegraded":"2023-10-08T05:36:16.203081+0000","last_fullsized":"2023-10-08T05:36:16.203081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205140+0000","last_change":"2023-10-08T05:36:11.938997+0000","last_active":"2023-10-08T05:36:16.205140+0000","last_peered":"2023-10-08T05:36:16.205140+0000","last_clean":"2023-10-08T05:36:16.205140+0000","last_became_active":"2023-10-08T05:36:11.938641+0000","last_became_peered":"2023-10-08T05:36:11.938641+0000","last_unstale":"2023-10-08T05:36:16.205140+0000","last_undegraded":"2023-10-08T05:36:16.205140+0000","last_fullsized":"2023-10-08T05:36:16.205140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:54.023070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194362+0000","last_change":"2023-10-08T05:36:11.938973+0000","last_active":"2023-10-08T05:36:16.194362+0000","last_peered":"2023-10-08T05:36:16.194362+0000","last_clean":"2023-10-08T05:36:16.194362+0000","last_became_active":"2023-10-08T05:36:11.938633+0000","last_became_peered":"2023-10-08T05:36:11.938633+0000","last_unstale":"2023-10-08T05:36:16.194362+0000","last_undegraded":"2023-10-08T05:36:16.194362+0000","last_fullsized":"2023-10-08T05:36:16.194362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213118+0000","last_change":"2023-10-08T05:36:11.938748+0000","last_active":"2023-10-08T05:36:16.213118+0000","last_peered":"2023-10-08T05:36:16.213118+0000","last_clean":"2023-10-08T05:36:16.213118+0000","last_became_active":"2023-10-08T05:36:11.937801+0000","last_became_peered":"2023-10-08T05:36:11.937801+0000","last_unstale":"2023-10-08T05:36:16.213118+0000","last_undegraded":"2023-10-08T05:36:16.213118+0000","last_fullsized":"2023-10-08T05:36:16.213118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:59.279776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196976+0000","last_change":"2023-10-08T05:36:11.937561+0000","last_active":"2023-10-08T05:36:16.196976+0000","last_peered":"2023-10-08T05:36:16.196976+0000","last_clean":"2023-10-08T05:36:16.196976+0000","last_became_active":"2023-10-08T05:36:11.937299+0000","last_became_peered":"2023-10-08T05:36:11.937299+0000","last_unstale":"2023-10-08T05:36:16.196976+0000","last_undegraded":"2023-10-08T05:36:16.196976+0000","last_fullsized":"2023-10-08T05:36:16.196976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195655+0000","last_change":"2023-10-08T05:36:11.938330+0000","last_active":"2023-10-08T05:36:16.195655+0000","last_peered":"2023-10-08T05:36:16.195655+0000","last_clean":"2023-10-08T05:36:16.195655+0000","last_became_active":"2023-10-08T05:36:11.937993+0000","last_became_peered":"2023-10-08T05:36:11.937993+0000","last_unstale":"2023-10-08T05:36:16.195655+0000","last_undegraded":"2023-10-08T05:36:16.195655+0000","last_fullsized":"2023-10-08T05:36:16.195655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240495+0000","last_change":"2023-10-08T05:36:11.954185+0000","last_active":"2023-10-08T05:36:16.240495+0000","last_peered":"2023-10-08T05:36:16.240495+0000","last_clean":"2023-10-08T05:36:16.240495+0000","last_became_active":"2023-10-08T05:36:11.953900+0000","last_became_peered":"2023-10-08T05:36:11.953900+0000","last_unstale":"2023-10-08T05:36:16.240495+0000","last_undegraded":"2023-10-08T05:36:16.240495+0000","last_fullsized":"2023-10-08T05:36:16.240495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:11.627279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195359+0000","last_change":"2023-10-08T05:36:11.938140+0000","last_active":"2023-10-08T05:36:16.195359+0000","last_peered":"2023-10-08T05:36:16.195359+0000","last_clean":"2023-10-08T05:36:16.195359+0000","last_became_active":"2023-10-08T05:36:11.937817+0000","last_became_peered":"2023-10-08T05:36:11.937817+0000","last_unstale":"2023-10-08T05:36:16.195359+0000","last_undegraded":"2023-10-08T05:36:16.195359+0000","last_fullsized":"2023-10-08T05:36:16.195359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238846+0000","last_change":"2023-10-08T05:36:11.948351+0000","last_active":"2023-10-08T05:36:16.238846+0000","last_peered":"2023-10-08T05:36:16.238846+0000","last_clean":"2023-10-08T05:36:16.238846+0000","last_became_active":"2023-10-08T05:36:11.948045+0000","last_became_peered":"2023-10-08T05:36:11.948045+0000","last_unstale":"2023-10-08T05:36:16.238846+0000","last_undegraded":"2023-10-08T05:36:16.238846+0000","last_fullsized":"2023-10-08T05:36:16.238846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215673+0000","last_change":"2023-10-08T05:36:11.944663+0000","last_active":"2023-10-08T05:36:16.215673+0000","last_peered":"2023-10-08T05:36:16.215673+0000","last_clean":"2023-10-08T05:36:16.215673+0000","last_became_active":"2023-10-08T05:36:11.944158+0000","last_became_peered":"2023-10-08T05:36:11.944158+0000","last_unstale":"2023-10-08T05:36:16.215673+0000","last_undegraded":"2023-10-08T05:36:16.215673+0000","last_fullsized":"2023-10-08T05:36:16.215673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:16.603017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239112+0000","last_change":"2023-10-08T05:36:11.944406+0000","last_active":"2023-10-08T05:36:16.239112+0000","last_peered":"2023-10-08T05:36:16.239112+0000","last_clean":"2023-10-08T05:36:16.239112+0000","last_became_active":"2023-10-08T05:36:11.944031+0000","last_became_peered":"2023-10-08T05:36:11.944031+0000","last_unstale":"2023-10-08T05:36:16.239112+0000","last_undegraded":"2023-10-08T05:36:16.239112+0000","last_fullsized":"2023-10-08T05:36:16.239112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214934+0000","last_change":"2023-10-08T05:36:11.949094+0000","last_active":"2023-10-08T05:36:16.214934+0000","last_peered":"2023-10-08T05:36:16.214934+0000","last_clean":"2023-10-08T05:36:16.214934+0000","last_became_active":"2023-10-08T05:36:11.948663+0000","last_became_peered":"2023-10-08T05:36:11.948663+0000","last_unstale":"2023-10-08T05:36:16.214934+0000","last_undegraded":"2023-10-08T05:36:16.214934+0000","last_fullsized":"2023-10-08T05:36:16.214934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:46:57.883695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205484+0000","last_change":"2023-10-08T05:36:11.927796+0000","last_active":"2023-10-08T05:36:16.205484+0000","last_peered":"2023-10-08T05:36:16.205484+0000","last_clean":"2023-10-08T05:36:16.205484+0000","last_became_active":"2023-10-08T05:36:11.927200+0000","last_became_peered":"2023-10-08T05:36:11.927200+0000","last_unstale":"2023-10-08T05:36:16.205484+0000","last_undegraded":"2023-10-08T05:36:16.205484+0000","last_fullsized":"2023-10-08T05:36:16.205484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.208328+0000","last_change":"2023-10-08T05:36:11.937916+0000","last_active":"2023-10-08T05:36:16.208328+0000","last_peered":"2023-10-08T05:36:16.208328+0000","last_clean":"2023-10-08T05:36:16.208328+0000","last_became_active":"2023-10-08T05:36:11.937646+0000","last_became_peered":"2023-10-08T05:36:11.937646+0000","last_unstale":"2023-10-08T05:36:16.208328+0000","last_undegraded":"2023-10-08T05:36:16.208328+0000","last_fullsized":"2023-10-08T05:36:16.208328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240404+0000","last_change":"2023-10-08T05:36:11.952124+0000","last_active":"2023-10-08T05:36:16.240404+0000","last_peered":"2023-10-08T05:36:16.240404+0000","last_clean":"2023-10-08T05:36:16.240404+0000","last_became_active":"2023-10-08T05:36:11.949353+0000","last_became_peered":"2023-10-08T05:36:11.949353+0000","last_unstale":"2023-10-08T05:36:16.240404+0000","last_undegraded":"2023-10-08T05:36:16.240404+0000","last_fullsized":"2023-10-08T05:36:16.240404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:46:57.883695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8149999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5209999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.931}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5129999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:36:35.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:36:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.667+0000 7f3691db0640 1 Processor -- start 2023-10-08T05:36:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.668+0000 7f3691db0640 1 -- start start 2023-10-08T05:36:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.669+0000 7f3691db0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f368c15ab40 0x7f368c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.669+0000 7f3691db0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0c90b0 0x7f368c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.669+0000 7f3691db0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c15b2b0 con 0x7f368c0cae70 2023-10-08T05:36:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.669+0000 7f3691db0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368c15bf30 con 0x7f368c15ab40 2023-10-08T05:36:35.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.669+0000 7f3691db0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c15f7a0 con 0x7f368c0c90b0 2023-10-08T05:36:35.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.669+0000 7f368bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f368c0cae70 0x7f368c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43600/0 (socket says 172.21.15.70:43600) 2023-10-08T05:36:35.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.669+0000 7f368bfff640 1 -- 172.21.15.70:0/1352967552 learned_addr learned my addr 172.21.15.70:0/1352967552 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:35.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368b7fe640 1 --2- 172.21.15.70:0/1352967552 >> v2:172.21.15.181:3300/0 conn(0x7f368c15ab40 0x7f368c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368affd640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0c90b0 0x7f368c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525477696 0 0) 0x7f368c15b2b0 con 0x7f368c0cae70 2023-10-08T05:36:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3678003520 con 0x7f368c0cae70 2023-10-08T05:36:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1004100158 0 0) 0x7f3678003520 con 0x7f368c0cae70 2023-10-08T05:36:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0c90b0 msgr2=0x7f368c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0c90b0 0x7f368c0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 >> v2:172.21.15.181:3300/0 conn(0x7f368c15ab40 msgr2=0x7f368c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 --2- 172.21.15.70:0/1352967552 >> v2:172.21.15.181:3300/0 conn(0x7f368c15ab40 0x7f368c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c15b590 con 0x7f368c0cae70 2023-10-08T05:36:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3684001670 con 0x7f368c0cae70 2023-10-08T05:36:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 >> v1:172.21.15.70:6789/0 conn(0x7f368c0cae70 legacy=0x7f368c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0c90b0 0x7f3678003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3678003f40 0x7f36780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3678004d00 0x7f3678005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c15bf30 con 0x7f3678003f40 2023-10-08T05:36:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c15f7a0 con 0x7f3678004d00 2023-10-08T05:36:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.670+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367800b980 con 0x7f368c0c90b0 2023-10-08T05:36:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.671+0000 7f368affd640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3678003f40 0x7f36780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.671+0000 7f368b7fe640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0c90b0 0x7f3678003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.671+0000 7f368bfff640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3678004d00 0x7f3678005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.671+0000 7f368affd640 1 -- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0c90b0 msgr2=0x7f3678003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:35.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.671+0000 7f368affd640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0c90b0 0x7f3678003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.671+0000 7f368affd640 1 -- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3678004d00 msgr2=0x7f3678005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:35.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.671+0000 7f368affd640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3678004d00 0x7f3678005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.671+0000 7f368affd640 1 -- 172.21.15.70:0/1352967552 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3678003520 con 0x7f3678003f40 2023-10-08T05:36:35.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.672+0000 7f368affd640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3678003f40 0x7f36780047c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3674009af0 tx=0x7f36780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe4d95b53620fe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:35.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.672+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367402c6b0 con 0x7f3678003f40 2023-10-08T05:36:35.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.673+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367402c810 con 0x7f3678003f40 2023-10-08T05:36:35.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.673+0000 7f368a7fc640 1 -- 172.21.15.70:0/1352967552 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367402cba0 con 0x7f3678003f40 2023-10-08T05:36:35.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.673+0000 7f3691db0640 1 -- 172.21.15.70:0/1352967552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3678003f40 msgr2=0x7f36780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.673+0000 7f3691db0640 1 --2- 172.21.15.70:0/1352967552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3678003f40 0x7f36780047c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3674009af0 tx=0x7f36780057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.673+0000 7f368bfff640 1 -- 172.21.15.70:0/1352967552 reap_dead start 2023-10-08T05:36:35.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.673+0000 7f3691db0640 1 -- 172.21.15.70:0/1352967552 shutdown_connections 2023-10-08T05:36:35.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.673+0000 7f3691db0640 1 -- 172.21.15.70:0/1352967552 >> 172.21.15.70:0/1352967552 conn(0x7f368c06a380 msgr2=0x7f368c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:35.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.673+0000 7f3691db0640 1 -- 172.21.15.70:0/1352967552 shutdown_connections 2023-10-08T05:36:35.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.673+0000 7f3691db0640 1 -- 172.21.15.70:0/1352967552 wait complete. 2023-10-08T05:36:35.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.675+0000 7f3691db0640 1 Processor -- start 2023-10-08T05:36:35.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.675+0000 7f3691db0640 1 -- start start 2023-10-08T05:36:35.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.676+0000 7f3691db0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3678003f40 0x7f368c0c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.676+0000 7f3691db0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f368c0c8410 0x7f368c0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.676+0000 7f3691db0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0abe00 0x7f368c0ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.676+0000 7f3691db0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0c7f80 con 0x7f368c0c8410 2023-10-08T05:36:35.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.676+0000 7f3691db0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c146cd0 con 0x7f3678003f40 2023-10-08T05:36:35.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.676+0000 7f3691db0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0af510 con 0x7f368c0abe00 2023-10-08T05:36:35.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.676+0000 7f368affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f368c0c8410 0x7f368c0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.676+0000 7f368affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f368c0c8410 0x7f368c0ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59788/0 (socket says 172.21.15.70:59788) 2023-10-08T05:36:35.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.676+0000 7f368affd640 1 -- 172.21.15.70:0/1009125900 learned_addr learned my addr 172.21.15.70:0/1009125900 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:35.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f368b7fe640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3678003f40 0x7f368c0c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f368bfff640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0abe00 0x7f368c0ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f368affd640 1 -- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0abe00 msgr2=0x7f368c0ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f368affd640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0abe00 0x7f368c0ac280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f368affd640 1 -- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3678003f40 msgr2=0x7f368c0c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f368affd640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3678003f40 0x7f368c0c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f368affd640 1 -- 172.21.15.70:0/1009125900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0c5ea0 con 0x7f368c0c8410 2023-10-08T05:36:35.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f368affd640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f368c0c8410 0x7f368c0ab8c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3674037040 tx=0x7f3674004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:35.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f3683fff640 1 -- 172.21.15.70:0/1009125900 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674003810 con 0x7f368c0c8410 2023-10-08T05:36:35.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f3683fff640 1 -- 172.21.15.70:0/1009125900 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3674003970 con 0x7f368c0c8410 2023-10-08T05:36:35.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.677+0000 7f3683fff640 1 -- 172.21.15.70:0/1009125900 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674003d00 con 0x7f368c0c8410 2023-10-08T05:36:35.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.678+0000 7f3691db0640 1 -- 172.21.15.70:0/1009125900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c0af7b0 con 0x7f368c0c8410 2023-10-08T05:36:35.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.678+0000 7f3691db0640 1 -- 172.21.15.70:0/1009125900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c142970 con 0x7f368c0c8410 2023-10-08T05:36:35.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.679+0000 7f3691db0640 1 -- 172.21.15.70:0/1009125900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c15af20 con 0x7f368c0c8410 2023-10-08T05:36:35.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.680+0000 7f3683fff640 1 -- 172.21.15.70:0/1009125900 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3674040450 con 0x7f368c0c8410 2023-10-08T05:36:35.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.681+0000 7f3683fff640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f366c06ae70 0x7f366c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:35.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.681+0000 7f3683fff640 1 -- 172.21.15.70:0/1009125900 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f36740ab8a0 con 0x7f368c0c8410 2023-10-08T05:36:35.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.682+0000 7f368b7fe640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f366c06ae70 0x7f366c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:35.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.683+0000 7f368b7fe640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f366c06ae70 0x7f366c06d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f367c007520 tx=0x7f367c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:35.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.686+0000 7f3683fff640 1 -- 172.21.15.70:0/1009125900 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36740765c0 con 0x7f368c0c8410 2023-10-08T05:36:35.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.872+0000 7f3691db0640 1 -- 172.21.15.70:0/1009125900 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f368c15bb50 con 0x7f366c06ae70 2023-10-08T05:36:35.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.899+0000 7f3683fff640 1 -- 172.21.15.70:0/1009125900 <== mgr.4105 v2:172.21.15.181:6832/2737793492 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345086 (secure 0 0 0) 0x7f367c003990 con 0x7f366c06ae70 2023-10-08T05:36:35.905 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:36:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 -- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f366c06ae70 msgr2=0x7f366c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f366c06ae70 0x7f366c06d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f367c007520 tx=0x7f367c00b040 comp rx=0 tx=0).stop 2023-10-08T05:36:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 -- 172.21.15.70:0/1009125900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f368c0c8410 msgr2=0x7f368c0ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f368c0c8410 0x7f368c0ab8c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3674037040 tx=0x7f3674004730 comp rx=0 tx=0).stop 2023-10-08T05:36:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 -- 172.21.15.70:0/1009125900 shutdown_connections 2023-10-08T05:36:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f366c06ae70 0x7f366c06d330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f368c0abe00 0x7f368c0ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f368c0c8410 0x7f368c0ab8c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 --2- 172.21.15.70:0/1009125900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3678003f40 0x7f368c0c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 -- 172.21.15.70:0/1009125900 >> 172.21.15.70:0/1009125900 conn(0x7f368c06a380 msgr2=0x7f368c0ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 -- 172.21.15.70:0/1009125900 shutdown_connections 2023-10-08T05:36:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:35.905+0000 7f3691db0640 1 -- 172.21.15.70:0/1009125900 wait complete. 2023-10-08T05:36:35.927 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-10-08T05:36:35.077806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236096,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738624,"statfs":{"total":773094113280,"available":772852350976,"internally_reserved":0,"allocated":3383296,"data_stored":2025622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2229,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004840"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205680+0000","last_change":"2023-10-08T05:36:11.183116+0000","last_active":"2023-10-08T05:36:16.205680+0000","last_peered":"2023-10-08T05:36:16.205680+0000","last_clean":"2023-10-08T05:36:16.205680+0000","last_became_active":"2023-10-08T05:36:11.182654+0000","last_became_peered":"2023-10-08T05:36:11.182654+0000","last_unstale":"2023-10-08T05:36:16.205680+0000","last_undegraded":"2023-10-08T05:36:16.205680+0000","last_fullsized":"2023-10-08T05:36:16.205680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:49.167371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198007+0000","last_change":"2023-10-08T05:36:11.938929+0000","last_active":"2023-10-08T05:36:16.198007+0000","last_peered":"2023-10-08T05:36:16.198007+0000","last_clean":"2023-10-08T05:36:16.198007+0000","last_became_active":"2023-10-08T05:36:11.938616+0000","last_became_peered":"2023-10-08T05:36:11.938616+0000","last_unstale":"2023-10-08T05:36:16.198007+0000","last_undegraded":"2023-10-08T05:36:16.198007+0000","last_fullsized":"2023-10-08T05:36:16.198007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.911637+0000","last_change":"2023-10-08T05:36:16.241991+0000","last_active":"2023-10-08T05:36:16.911637+0000","last_peered":"2023-10-08T05:36:16.911637+0000","last_clean":"2023-10-08T05:36:16.911637+0000","last_became_active":"2023-10-08T05:36:16.241629+0000","last_became_peered":"2023-10-08T05:36:16.241629+0000","last_unstale":"2023-10-08T05:36:16.911637+0000","last_undegraded":"2023-10-08T05:36:16.911637+0000","last_fullsized":"2023-10-08T05:36:16.911637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:41.216984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198078+0000","last_change":"2023-10-08T05:36:11.171819+0000","last_active":"2023-10-08T05:36:16.198078+0000","last_peered":"2023-10-08T05:36:16.198078+0000","last_clean":"2023-10-08T05:36:16.198078+0000","last_became_active":"2023-10-08T05:36:11.171384+0000","last_became_peered":"2023-10-08T05:36:11.171384+0000","last_unstale":"2023-10-08T05:36:16.198078+0000","last_undegraded":"2023-10-08T05:36:16.198078+0000","last_fullsized":"2023-10-08T05:36:16.198078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:08.838858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205587+0000","last_change":"2023-10-08T05:36:11.939107+0000","last_active":"2023-10-08T05:36:16.205587+0000","last_peered":"2023-10-08T05:36:16.205587+0000","last_clean":"2023-10-08T05:36:16.205587+0000","last_became_active":"2023-10-08T05:36:11.938788+0000","last_became_peered":"2023-10-08T05:36:11.938788+0000","last_unstale":"2023-10-08T05:36:16.205587+0000","last_undegraded":"2023-10-08T05:36:16.205587+0000","last_fullsized":"2023-10-08T05:36:16.205587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450376+0000","last_change":"2023-10-08T05:36:16.220918+0000","last_active":"2023-10-08T05:36:18.450376+0000","last_peered":"2023-10-08T05:36:18.450376+0000","last_clean":"2023-10-08T05:36:18.450376+0000","last_became_active":"2023-10-08T05:36:16.220450+0000","last_became_peered":"2023-10-08T05:36:16.220450+0000","last_unstale":"2023-10-08T05:36:18.450376+0000","last_undegraded":"2023-10-08T05:36:18.450376+0000","last_fullsized":"2023-10-08T05:36:18.450376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:45.730636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239438+0000","last_change":"2023-10-08T05:36:11.187132+0000","last_active":"2023-10-08T05:36:16.239438+0000","last_peered":"2023-10-08T05:36:16.239438+0000","last_clean":"2023-10-08T05:36:16.239438+0000","last_became_active":"2023-10-08T05:36:11.186856+0000","last_became_peered":"2023-10-08T05:36:11.186856+0000","last_unstale":"2023-10-08T05:36:16.239438+0000","last_undegraded":"2023-10-08T05:36:16.239438+0000","last_fullsized":"2023-10-08T05:36:16.239438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:42.738249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215030+0000","last_change":"2023-10-08T05:36:11.946617+0000","last_active":"2023-10-08T05:36:16.215030+0000","last_peered":"2023-10-08T05:36:16.215030+0000","last_clean":"2023-10-08T05:36:16.215030+0000","last_became_active":"2023-10-08T05:36:11.946360+0000","last_became_peered":"2023-10-08T05:36:11.946360+0000","last_unstale":"2023-10-08T05:36:16.215030+0000","last_undegraded":"2023-10-08T05:36:16.215030+0000","last_fullsized":"2023-10-08T05:36:16.215030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:21.717397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450713+0000","last_change":"2023-10-08T05:36:16.229002+0000","last_active":"2023-10-08T05:36:18.450713+0000","last_peered":"2023-10-08T05:36:18.450713+0000","last_clean":"2023-10-08T05:36:18.450713+0000","last_became_active":"2023-10-08T05:36:16.228230+0000","last_became_peered":"2023-10-08T05:36:16.228230+0000","last_unstale":"2023-10-08T05:36:18.450713+0000","last_undegraded":"2023-10-08T05:36:18.450713+0000","last_fullsized":"2023-10-08T05:36:18.450713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:33.520862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239150+0000","last_change":"2023-10-08T05:36:11.191776+0000","last_active":"2023-10-08T05:36:16.239150+0000","last_peered":"2023-10-08T05:36:16.239150+0000","last_clean":"2023-10-08T05:36:16.239150+0000","last_became_active":"2023-10-08T05:36:11.191532+0000","last_became_peered":"2023-10-08T05:36:11.191532+0000","last_unstale":"2023-10-08T05:36:16.239150+0000","last_undegraded":"2023-10-08T05:36:16.239150+0000","last_fullsized":"2023-10-08T05:36:16.239150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:16.799817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.217226+0000","last_change":"2023-10-08T05:36:11.950652+0000","last_active":"2023-10-08T05:36:16.217226+0000","last_peered":"2023-10-08T05:36:16.217226+0000","last_clean":"2023-10-08T05:36:16.217226+0000","last_became_active":"2023-10-08T05:36:11.950359+0000","last_became_peered":"2023-10-08T05:36:11.950359+0000","last_unstale":"2023-10-08T05:36:16.217226+0000","last_undegraded":"2023-10-08T05:36:16.217226+0000","last_fullsized":"2023-10-08T05:36:16.217226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451049+0000","last_change":"2023-10-08T05:36:16.222866+0000","last_active":"2023-10-08T05:36:18.451049+0000","last_peered":"2023-10-08T05:36:18.451049+0000","last_clean":"2023-10-08T05:36:18.451049+0000","last_became_active":"2023-10-08T05:36:16.222444+0000","last_became_peered":"2023-10-08T05:36:16.222444+0000","last_unstale":"2023-10-08T05:36:18.451049+0000","last_undegraded":"2023-10-08T05:36:18.451049+0000","last_fullsized":"2023-10-08T05:36:18.451049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:04.460179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195161+0000","last_change":"2023-10-08T05:36:11.936259+0000","last_active":"2023-10-08T05:36:16.195161+0000","last_peered":"2023-10-08T05:36:16.195161+0000","last_clean":"2023-10-08T05:36:16.195161+0000","last_became_active":"2023-10-08T05:36:11.935923+0000","last_became_peered":"2023-10-08T05:36:11.935923+0000","last_unstale":"2023-10-08T05:36:16.195161+0000","last_undegraded":"2023-10-08T05:36:16.195161+0000","last_fullsized":"2023-10-08T05:36:16.195161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239194+0000","last_change":"2023-10-08T05:36:11.187292+0000","last_active":"2023-10-08T05:36:16.239194+0000","last_peered":"2023-10-08T05:36:16.239194+0000","last_clean":"2023-10-08T05:36:16.239194+0000","last_became_active":"2023-10-08T05:36:11.186706+0000","last_became_peered":"2023-10-08T05:36:11.186706+0000","last_unstale":"2023-10-08T05:36:16.239194+0000","last_undegraded":"2023-10-08T05:36:16.239194+0000","last_fullsized":"2023-10-08T05:36:16.239194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:22.361671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451570+0000","last_change":"2023-10-08T05:36:16.228736+0000","last_active":"2023-10-08T05:36:18.451570+0000","last_peered":"2023-10-08T05:36:18.451570+0000","last_clean":"2023-10-08T05:36:18.451570+0000","last_became_active":"2023-10-08T05:36:16.227909+0000","last_became_peered":"2023-10-08T05:36:16.227909+0000","last_unstale":"2023-10-08T05:36:18.451570+0000","last_undegraded":"2023-10-08T05:36:18.451570+0000","last_fullsized":"2023-10-08T05:36:18.451570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:29.122804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194544+0000","last_change":"2023-10-08T05:36:11.945361+0000","last_active":"2023-10-08T05:36:16.194544+0000","last_peered":"2023-10-08T05:36:16.194544+0000","last_clean":"2023-10-08T05:36:16.194544+0000","last_became_active":"2023-10-08T05:36:11.944819+0000","last_became_peered":"2023-10-08T05:36:11.944819+0000","last_unstale":"2023-10-08T05:36:16.194544+0000","last_undegraded":"2023-10-08T05:36:16.194544+0000","last_fullsized":"2023-10-08T05:36:16.194544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:56.745041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.208276+0000","last_change":"2023-10-08T05:36:11.164749+0000","last_active":"2023-10-08T05:36:16.208276+0000","last_peered":"2023-10-08T05:36:16.208276+0000","last_clean":"2023-10-08T05:36:16.208276+0000","last_became_active":"2023-10-08T05:36:11.164382+0000","last_became_peered":"2023-10-08T05:36:11.164382+0000","last_unstale":"2023-10-08T05:36:16.208276+0000","last_undegraded":"2023-10-08T05:36:16.208276+0000","last_fullsized":"2023-10-08T05:36:16.208276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:13.509993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450966+0000","last_change":"2023-10-08T05:36:16.229430+0000","last_active":"2023-10-08T05:36:18.450966+0000","last_peered":"2023-10-08T05:36:18.450966+0000","last_clean":"2023-10-08T05:36:18.450966+0000","last_became_active":"2023-10-08T05:36:16.228957+0000","last_became_peered":"2023-10-08T05:36:16.228957+0000","last_unstale":"2023-10-08T05:36:18.450966+0000","last_undegraded":"2023-10-08T05:36:18.450966+0000","last_fullsized":"2023-10-08T05:36:18.450966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:00.782220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195462+0000","last_change":"2023-10-08T05:36:11.934612+0000","last_active":"2023-10-08T05:36:16.195462+0000","last_peered":"2023-10-08T05:36:16.195462+0000","last_clean":"2023-10-08T05:36:16.195462+0000","last_became_active":"2023-10-08T05:36:11.934284+0000","last_became_peered":"2023-10-08T05:36:11.934284+0000","last_unstale":"2023-10-08T05:36:16.195462+0000","last_undegraded":"2023-10-08T05:36:16.195462+0000","last_fullsized":"2023-10-08T05:36:16.195462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203211+0000","last_change":"2023-10-08T05:36:11.182216+0000","last_active":"2023-10-08T05:36:16.203211+0000","last_peered":"2023-10-08T05:36:16.203211+0000","last_clean":"2023-10-08T05:36:16.203211+0000","last_became_active":"2023-10-08T05:36:11.181927+0000","last_became_peered":"2023-10-08T05:36:11.181927+0000","last_unstale":"2023-10-08T05:36:16.203211+0000","last_undegraded":"2023-10-08T05:36:16.203211+0000","last_fullsized":"2023-10-08T05:36:16.203211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:42.738249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.917366+0000","last_change":"2023-10-08T05:36:16.243897+0000","last_active":"2023-10-08T05:36:16.917366+0000","last_peered":"2023-10-08T05:36:16.917366+0000","last_clean":"2023-10-08T05:36:16.917366+0000","last_became_active":"2023-10-08T05:36:16.243421+0000","last_became_peered":"2023-10-08T05:36:16.243421+0000","last_unstale":"2023-10-08T05:36:16.917366+0000","last_undegraded":"2023-10-08T05:36:16.917366+0000","last_fullsized":"2023-10-08T05:36:16.917366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:16.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.209892+0000","last_change":"2023-10-08T05:36:11.934209+0000","last_active":"2023-10-08T05:36:16.209892+0000","last_peered":"2023-10-08T05:36:16.209892+0000","last_clean":"2023-10-08T05:36:16.209892+0000","last_became_active":"2023-10-08T05:36:11.933067+0000","last_became_peered":"2023-10-08T05:36:11.933067+0000","last_unstale":"2023-10-08T05:36:16.209892+0000","last_undegraded":"2023-10-08T05:36:16.209892+0000","last_fullsized":"2023-10-08T05:36:16.209892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:17.827053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238717+0000","last_change":"2023-10-08T05:36:11.181498+0000","last_active":"2023-10-08T05:36:16.238717+0000","last_peered":"2023-10-08T05:36:16.238717+0000","last_clean":"2023-10-08T05:36:16.238717+0000","last_became_active":"2023-10-08T05:36:11.181264+0000","last_became_peered":"2023-10-08T05:36:11.181264+0000","last_unstale":"2023-10-08T05:36:16.238717+0000","last_undegraded":"2023-10-08T05:36:16.238717+0000","last_fullsized":"2023-10-08T05:36:16.238717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:35.005158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.452011+0000","last_change":"2023-10-08T05:36:16.228065+0000","last_active":"2023-10-08T05:36:18.452011+0000","last_peered":"2023-10-08T05:36:18.452011+0000","last_clean":"2023-10-08T05:36:18.452011+0000","last_became_active":"2023-10-08T05:36:16.227665+0000","last_became_peered":"2023-10-08T05:36:16.227665+0000","last_unstale":"2023-10-08T05:36:18.452011+0000","last_undegraded":"2023-10-08T05:36:18.452011+0000","last_fullsized":"2023-10-08T05:36:18.452011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:32.619157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205947+0000","last_change":"2023-10-08T05:36:11.934150+0000","last_active":"2023-10-08T05:36:16.205947+0000","last_peered":"2023-10-08T05:36:16.205947+0000","last_clean":"2023-10-08T05:36:16.205947+0000","last_became_active":"2023-10-08T05:36:11.933875+0000","last_became_peered":"2023-10-08T05:36:11.933875+0000","last_unstale":"2023-10-08T05:36:16.205947+0000","last_undegraded":"2023-10-08T05:36:16.205947+0000","last_fullsized":"2023-10-08T05:36:16.205947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196218+0000","last_change":"2023-10-08T05:36:11.187989+0000","last_active":"2023-10-08T05:36:16.196218+0000","last_peered":"2023-10-08T05:36:16.196218+0000","last_clean":"2023-10-08T05:36:16.196218+0000","last_became_active":"2023-10-08T05:36:11.187695+0000","last_became_peered":"2023-10-08T05:36:11.187695+0000","last_unstale":"2023-10-08T05:36:16.196218+0000","last_undegraded":"2023-10-08T05:36:16.196218+0000","last_fullsized":"2023-10-08T05:36:16.196218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:35.005158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451149+0000","last_change":"2023-10-08T05:36:16.244139+0000","last_active":"2023-10-08T05:36:18.451149+0000","last_peered":"2023-10-08T05:36:18.451149+0000","last_clean":"2023-10-08T05:36:18.451149+0000","last_became_active":"2023-10-08T05:36:16.243569+0000","last_became_peered":"2023-10-08T05:36:16.243569+0000","last_unstale":"2023-10-08T05:36:18.451149+0000","last_undegraded":"2023-10-08T05:36:18.451149+0000","last_fullsized":"2023-10-08T05:36:18.451149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:30.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198170+0000","last_change":"2023-10-08T05:36:11.941731+0000","last_active":"2023-10-08T05:36:16.198170+0000","last_peered":"2023-10-08T05:36:16.198170+0000","last_clean":"2023-10-08T05:36:16.198170+0000","last_became_active":"2023-10-08T05:36:11.941393+0000","last_became_peered":"2023-10-08T05:36:11.941393+0000","last_unstale":"2023-10-08T05:36:16.198170+0000","last_undegraded":"2023-10-08T05:36:16.198170+0000","last_fullsized":"2023-10-08T05:36:16.198170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207549+0000","last_change":"2023-10-08T05:36:11.180784+0000","last_active":"2023-10-08T05:36:16.207549+0000","last_peered":"2023-10-08T05:36:16.207549+0000","last_clean":"2023-10-08T05:36:16.207549+0000","last_became_active":"2023-10-08T05:36:11.180398+0000","last_became_peered":"2023-10-08T05:36:11.180398+0000","last_unstale":"2023-10-08T05:36:16.207549+0000","last_undegraded":"2023-10-08T05:36:16.207549+0000","last_fullsized":"2023-10-08T05:36:16.207549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:49.167371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913389+0000","last_change":"2023-10-08T05:36:16.240627+0000","last_active":"2023-10-08T05:36:16.913389+0000","last_peered":"2023-10-08T05:36:16.913389+0000","last_clean":"2023-10-08T05:36:16.913389+0000","last_became_active":"2023-10-08T05:36:16.240153+0000","last_became_peered":"2023-10-08T05:36:16.240153+0000","last_unstale":"2023-10-08T05:36:16.913389+0000","last_undegraded":"2023-10-08T05:36:16.913389+0000","last_fullsized":"2023-10-08T05:36:16.913389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:06.106435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207447+0000","last_change":"2023-10-08T05:36:11.941834+0000","last_active":"2023-10-08T05:36:16.207447+0000","last_peered":"2023-10-08T05:36:16.207447+0000","last_clean":"2023-10-08T05:36:16.207447+0000","last_became_active":"2023-10-08T05:36:11.941546+0000","last_became_peered":"2023-10-08T05:36:11.941546+0000","last_unstale":"2023-10-08T05:36:16.207447+0000","last_undegraded":"2023-10-08T05:36:16.207447+0000","last_fullsized":"2023-10-08T05:36:16.207447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239749+0000","last_change":"2023-10-08T05:36:11.182530+0000","last_active":"2023-10-08T05:36:16.239749+0000","last_peered":"2023-10-08T05:36:16.239749+0000","last_clean":"2023-10-08T05:36:16.239749+0000","last_became_active":"2023-10-08T05:36:11.182156+0000","last_became_peered":"2023-10-08T05:36:11.182156+0000","last_unstale":"2023-10-08T05:36:16.239749+0000","last_undegraded":"2023-10-08T05:36:16.239749+0000","last_fullsized":"2023-10-08T05:36:16.239749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:10.731974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449389+0000","last_change":"2023-10-08T05:36:16.240659+0000","last_active":"2023-10-08T05:36:18.449389+0000","last_peered":"2023-10-08T05:36:18.449389+0000","last_clean":"2023-10-08T05:36:18.449389+0000","last_became_active":"2023-10-08T05:36:16.240244+0000","last_became_peered":"2023-10-08T05:36:16.240244+0000","last_unstale":"2023-10-08T05:36:18.449389+0000","last_undegraded":"2023-10-08T05:36:18.449389+0000","last_fullsized":"2023-10-08T05:36:18.449389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:15.664459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.216378+0000","last_change":"2023-10-08T05:36:11.943995+0000","last_active":"2023-10-08T05:36:16.216378+0000","last_peered":"2023-10-08T05:36:16.216378+0000","last_clean":"2023-10-08T05:36:16.216378+0000","last_became_active":"2023-10-08T05:36:11.943601+0000","last_became_peered":"2023-10-08T05:36:11.943601+0000","last_unstale":"2023-10-08T05:36:16.216378+0000","last_undegraded":"2023-10-08T05:36:16.216378+0000","last_fullsized":"2023-10-08T05:36:16.216378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:28.581977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207661+0000","last_change":"2023-10-08T05:36:11.166965+0000","last_active":"2023-10-08T05:36:16.207661+0000","last_peered":"2023-10-08T05:36:16.207661+0000","last_clean":"2023-10-08T05:36:16.207661+0000","last_became_active":"2023-10-08T05:36:11.166704+0000","last_became_peered":"2023-10-08T05:36:11.166704+0000","last_unstale":"2023-10-08T05:36:16.207661+0000","last_undegraded":"2023-10-08T05:36:16.207661+0000","last_fullsized":"2023-10-08T05:36:16.207661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.915670+0000","last_change":"2023-10-08T05:36:16.226001+0000","last_active":"2023-10-08T05:36:16.915670+0000","last_peered":"2023-10-08T05:36:16.915670+0000","last_clean":"2023-10-08T05:36:16.915670+0000","last_became_active":"2023-10-08T05:36:16.225623+0000","last_became_peered":"2023-10-08T05:36:16.225623+0000","last_unstale":"2023-10-08T05:36:16.915670+0000","last_undegraded":"2023-10-08T05:36:16.915670+0000","last_fullsized":"2023-10-08T05:36:16.915670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213447+0000","last_change":"2023-10-08T05:36:11.935396+0000","last_active":"2023-10-08T05:36:16.213447+0000","last_peered":"2023-10-08T05:36:16.213447+0000","last_clean":"2023-10-08T05:36:16.213447+0000","last_became_active":"2023-10-08T05:36:11.934642+0000","last_became_peered":"2023-10-08T05:36:11.934642+0000","last_unstale":"2023-10-08T05:36:16.213447+0000","last_undegraded":"2023-10-08T05:36:16.213447+0000","last_fullsized":"2023-10-08T05:36:16.213447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:09.850609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215985+0000","last_change":"2023-10-08T05:36:11.189030+0000","last_active":"2023-10-08T05:36:16.215985+0000","last_peered":"2023-10-08T05:36:16.215985+0000","last_clean":"2023-10-08T05:36:16.215985+0000","last_became_active":"2023-10-08T05:36:11.184720+0000","last_became_peered":"2023-10-08T05:36:11.184720+0000","last_unstale":"2023-10-08T05:36:16.215985+0000","last_undegraded":"2023-10-08T05:36:16.215985+0000","last_fullsized":"2023-10-08T05:36:16.215985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:41.625077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.916396+0000","last_change":"2023-10-08T05:36:16.223249+0000","last_active":"2023-10-08T05:36:16.916396+0000","last_peered":"2023-10-08T05:36:16.916396+0000","last_clean":"2023-10-08T05:36:16.916396+0000","last_became_active":"2023-10-08T05:36:16.222656+0000","last_became_peered":"2023-10-08T05:36:16.222656+0000","last_unstale":"2023-10-08T05:36:16.916396+0000","last_undegraded":"2023-10-08T05:36:16.916396+0000","last_fullsized":"2023-10-08T05:36:16.916396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:25.754577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203206+0000","last_change":"2023-10-08T05:36:11.942804+0000","last_active":"2023-10-08T05:36:16.203206+0000","last_peered":"2023-10-08T05:36:16.203206+0000","last_clean":"2023-10-08T05:36:16.203206+0000","last_became_active":"2023-10-08T05:36:11.942486+0000","last_became_peered":"2023-10-08T05:36:11.942486+0000","last_unstale":"2023-10-08T05:36:16.203206+0000","last_undegraded":"2023-10-08T05:36:16.203206+0000","last_fullsized":"2023-10-08T05:36:16.203206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190110+0000","last_change":"2023-10-08T05:36:11.160795+0000","last_active":"2023-10-08T05:36:16.190110+0000","last_peered":"2023-10-08T05:36:16.190110+0000","last_clean":"2023-10-08T05:36:16.190110+0000","last_became_active":"2023-10-08T05:36:11.160498+0000","last_became_peered":"2023-10-08T05:36:11.160498+0000","last_unstale":"2023-10-08T05:36:16.190110+0000","last_undegraded":"2023-10-08T05:36:16.190110+0000","last_fullsized":"2023-10-08T05:36:16.190110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:08.847794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.917236+0000","last_change":"2023-10-08T05:36:16.224778+0000","last_active":"2023-10-08T05:36:16.917236+0000","last_peered":"2023-10-08T05:36:16.917236+0000","last_clean":"2023-10-08T05:36:16.917236+0000","last_became_active":"2023-10-08T05:36:16.224249+0000","last_became_peered":"2023-10-08T05:36:16.224249+0000","last_unstale":"2023-10-08T05:36:16.917236+0000","last_undegraded":"2023-10-08T05:36:16.917236+0000","last_fullsized":"2023-10-08T05:36:16.917236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:07.031678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.204855+0000","last_change":"2023-10-08T05:36:11.944248+0000","last_active":"2023-10-08T05:36:16.204855+0000","last_peered":"2023-10-08T05:36:16.204855+0000","last_clean":"2023-10-08T05:36:16.204855+0000","last_became_active":"2023-10-08T05:36:11.943941+0000","last_became_peered":"2023-10-08T05:36:11.943941+0000","last_unstale":"2023-10-08T05:36:16.204855+0000","last_undegraded":"2023-10-08T05:36:16.204855+0000","last_fullsized":"2023-10-08T05:36:16.204855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238762+0000","last_change":"2023-10-08T05:36:11.186476+0000","last_active":"2023-10-08T05:36:16.238762+0000","last_peered":"2023-10-08T05:36:16.238762+0000","last_clean":"2023-10-08T05:36:16.238762+0000","last_became_active":"2023-10-08T05:36:11.186165+0000","last_became_peered":"2023-10-08T05:36:11.186165+0000","last_unstale":"2023-10-08T05:36:16.238762+0000","last_undegraded":"2023-10-08T05:36:16.238762+0000","last_fullsized":"2023-10-08T05:36:16.238762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:08.838858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450006+0000","last_change":"2023-10-08T05:36:16.241072+0000","last_active":"2023-10-08T05:36:18.450006+0000","last_peered":"2023-10-08T05:36:18.450006+0000","last_clean":"2023-10-08T05:36:18.450006+0000","last_became_active":"2023-10-08T05:36:16.240154+0000","last_became_peered":"2023-10-08T05:36:16.240154+0000","last_unstale":"2023-10-08T05:36:18.450006+0000","last_undegraded":"2023-10-08T05:36:18.450006+0000","last_fullsized":"2023-10-08T05:36:18.450006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:16.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194060+0000","last_change":"2023-10-08T05:36:11.936616+0000","last_active":"2023-10-08T05:36:16.194060+0000","last_peered":"2023-10-08T05:36:16.194060+0000","last_clean":"2023-10-08T05:36:16.194060+0000","last_became_active":"2023-10-08T05:36:11.936320+0000","last_became_peered":"2023-10-08T05:36:11.936320+0000","last_unstale":"2023-10-08T05:36:16.194060+0000","last_undegraded":"2023-10-08T05:36:16.194060+0000","last_fullsized":"2023-10-08T05:36:16.194060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215473+0000","last_change":"2023-10-08T05:36:11.182263+0000","last_active":"2023-10-08T05:36:16.215473+0000","last_peered":"2023-10-08T05:36:16.215473+0000","last_clean":"2023-10-08T05:36:16.215473+0000","last_became_active":"2023-10-08T05:36:11.181876+0000","last_became_peered":"2023-10-08T05:36:11.181876+0000","last_unstale":"2023-10-08T05:36:16.215473+0000","last_undegraded":"2023-10-08T05:36:16.215473+0000","last_fullsized":"2023-10-08T05:36:16.215473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:16.815302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238841+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:36:16.238841+0000","last_peered":"2023-10-08T05:36:16.238841+0000","last_clean":"2023-10-08T05:36:16.238841+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:36:16.238841+0000","last_undegraded":"2023-10-08T05:36:16.238841+0000","last_fullsized":"2023-10-08T05:36:16.238841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914962+0000","last_change":"2023-10-08T05:36:16.241252+0000","last_active":"2023-10-08T05:36:16.914962+0000","last_peered":"2023-10-08T05:36:16.914962+0000","last_clean":"2023-10-08T05:36:16.914962+0000","last_became_active":"2023-10-08T05:36:16.240543+0000","last_became_peered":"2023-10-08T05:36:16.240543+0000","last_unstale":"2023-10-08T05:36:16.914962+0000","last_undegraded":"2023-10-08T05:36:16.914962+0000","last_fullsized":"2023-10-08T05:36:16.914962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:20.640196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239326+0000","last_change":"2023-10-08T05:36:11.954327+0000","last_active":"2023-10-08T05:36:16.239326+0000","last_peered":"2023-10-08T05:36:16.239326+0000","last_clean":"2023-10-08T05:36:16.239326+0000","last_became_active":"2023-10-08T05:36:11.953852+0000","last_became_peered":"2023-10-08T05:36:11.953852+0000","last_unstale":"2023-10-08T05:36:16.239326+0000","last_undegraded":"2023-10-08T05:36:16.239326+0000","last_fullsized":"2023-10-08T05:36:16.239326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:12.498480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190495+0000","last_change":"2023-10-08T05:36:11.179188+0000","last_active":"2023-10-08T05:36:16.190495+0000","last_peered":"2023-10-08T05:36:16.190495+0000","last_clean":"2023-10-08T05:36:16.190495+0000","last_became_active":"2023-10-08T05:36:11.178872+0000","last_became_peered":"2023-10-08T05:36:11.178872+0000","last_unstale":"2023-10-08T05:36:16.190495+0000","last_undegraded":"2023-10-08T05:36:16.190495+0000","last_fullsized":"2023-10-08T05:36:16.190495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:44.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.915127+0000","last_change":"2023-10-08T05:36:16.241452+0000","last_active":"2023-10-08T05:36:16.915127+0000","last_peered":"2023-10-08T05:36:16.915127+0000","last_clean":"2023-10-08T05:36:16.915127+0000","last_became_active":"2023-10-08T05:36:16.240541+0000","last_became_peered":"2023-10-08T05:36:16.240541+0000","last_unstale":"2023-10-08T05:36:16.915127+0000","last_undegraded":"2023-10-08T05:36:16.915127+0000","last_fullsized":"2023-10-08T05:36:16.915127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215138+0000","last_change":"2023-10-08T05:36:11.948239+0000","last_active":"2023-10-08T05:36:16.215138+0000","last_peered":"2023-10-08T05:36:16.215138+0000","last_clean":"2023-10-08T05:36:16.215138+0000","last_became_active":"2023-10-08T05:36:11.947960+0000","last_became_peered":"2023-10-08T05:36:11.947960+0000","last_unstale":"2023-10-08T05:36:16.215138+0000","last_undegraded":"2023-10-08T05:36:16.215138+0000","last_fullsized":"2023-10-08T05:36:16.215138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:02.994498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207516+0000","last_change":"2023-10-08T05:36:11.173210+0000","last_active":"2023-10-08T05:36:16.207516+0000","last_peered":"2023-10-08T05:36:16.207516+0000","last_clean":"2023-10-08T05:36:16.207516+0000","last_became_active":"2023-10-08T05:36:11.172899+0000","last_became_peered":"2023-10-08T05:36:11.172899+0000","last_unstale":"2023-10-08T05:36:16.207516+0000","last_undegraded":"2023-10-08T05:36:16.207516+0000","last_fullsized":"2023-10-08T05:36:16.207516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:59.559506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449356+0000","last_change":"2023-10-08T05:36:16.226571+0000","last_active":"2023-10-08T05:36:18.449356+0000","last_peered":"2023-10-08T05:36:18.449356+0000","last_clean":"2023-10-08T05:36:18.449356+0000","last_became_active":"2023-10-08T05:36:16.226098+0000","last_became_peered":"2023-10-08T05:36:16.226098+0000","last_unstale":"2023-10-08T05:36:18.449356+0000","last_undegraded":"2023-10-08T05:36:18.449356+0000","last_fullsized":"2023-10-08T05:36:18.449356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.246459+0000","last_change":"2023-10-08T05:36:11.948021+0000","last_active":"2023-10-08T05:36:16.246459+0000","last_peered":"2023-10-08T05:36:16.246459+0000","last_clean":"2023-10-08T05:36:16.246459+0000","last_became_active":"2023-10-08T05:36:11.947456+0000","last_became_peered":"2023-10-08T05:36:11.947456+0000","last_unstale":"2023-10-08T05:36:16.246459+0000","last_undegraded":"2023-10-08T05:36:16.246459+0000","last_fullsized":"2023-10-08T05:36:16.246459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:21.717397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.191082+0000","last_change":"2023-10-08T05:36:11.164907+0000","last_active":"2023-10-08T05:36:16.191082+0000","last_peered":"2023-10-08T05:36:16.191082+0000","last_clean":"2023-10-08T05:36:16.191082+0000","last_became_active":"2023-10-08T05:36:11.164630+0000","last_became_peered":"2023-10-08T05:36:11.164630+0000","last_unstale":"2023-10-08T05:36:16.191082+0000","last_undegraded":"2023-10-08T05:36:16.191082+0000","last_fullsized":"2023-10-08T05:36:16.191082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:58.268025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.347745+0000","last_change":"2023-10-08T05:36:16.241024+0000","last_active":"2023-10-08T05:36:17.347745+0000","last_peered":"2023-10-08T05:36:17.347745+0000","last_clean":"2023-10-08T05:36:17.347745+0000","last_became_active":"2023-10-08T05:36:16.240379+0000","last_became_peered":"2023-10-08T05:36:16.240379+0000","last_unstale":"2023-10-08T05:36:17.347745+0000","last_undegraded":"2023-10-08T05:36:17.347745+0000","last_fullsized":"2023-10-08T05:36:17.347745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:09:12.638961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240155+0000","last_change":"2023-10-08T05:36:11.946599+0000","last_active":"2023-10-08T05:36:16.240155+0000","last_peered":"2023-10-08T05:36:16.240155+0000","last_clean":"2023-10-08T05:36:16.240155+0000","last_became_active":"2023-10-08T05:36:11.946295+0000","last_became_peered":"2023-10-08T05:36:11.946295+0000","last_unstale":"2023-10-08T05:36:16.240155+0000","last_undegraded":"2023-10-08T05:36:16.240155+0000","last_fullsized":"2023-10-08T05:36:16.240155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:28.581977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214835+0000","last_change":"2023-10-08T05:36:11.181737+0000","last_active":"2023-10-08T05:36:16.214835+0000","last_peered":"2023-10-08T05:36:16.214835+0000","last_clean":"2023-10-08T05:36:16.214835+0000","last_became_active":"2023-10-08T05:36:11.181386+0000","last_became_peered":"2023-10-08T05:36:11.181386+0000","last_unstale":"2023-10-08T05:36:16.214835+0000","last_undegraded":"2023-10-08T05:36:16.214835+0000","last_fullsized":"2023-10-08T05:36:16.214835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:05.139990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450290+0000","last_change":"2023-10-08T05:36:16.223098+0000","last_active":"2023-10-08T05:36:18.450290+0000","last_peered":"2023-10-08T05:36:18.450290+0000","last_clean":"2023-10-08T05:36:18.450290+0000","last_became_active":"2023-10-08T05:36:16.222649+0000","last_became_peered":"2023-10-08T05:36:16.222649+0000","last_unstale":"2023-10-08T05:36:18.450290+0000","last_undegraded":"2023-10-08T05:36:18.450290+0000","last_fullsized":"2023-10-08T05:36:18.450290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:27.410602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238995+0000","last_change":"2023-10-08T05:36:11.946258+0000","last_active":"2023-10-08T05:36:16.238995+0000","last_peered":"2023-10-08T05:36:16.238995+0000","last_clean":"2023-10-08T05:36:16.238995+0000","last_became_active":"2023-10-08T05:36:11.945932+0000","last_became_peered":"2023-10-08T05:36:11.945932+0000","last_unstale":"2023-10-08T05:36:16.238995+0000","last_undegraded":"2023-10-08T05:36:16.238995+0000","last_fullsized":"2023-10-08T05:36:16.238995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:16.603017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.189924+0000","last_change":"2023-10-08T05:36:11.162339+0000","last_active":"2023-10-08T05:36:16.189924+0000","last_peered":"2023-10-08T05:36:16.189924+0000","last_clean":"2023-10-08T05:36:16.189924+0000","last_became_active":"2023-10-08T05:36:11.162023+0000","last_became_peered":"2023-10-08T05:36:11.162023+0000","last_unstale":"2023-10-08T05:36:16.189924+0000","last_undegraded":"2023-10-08T05:36:16.189924+0000","last_fullsized":"2023-10-08T05:36:16.189924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:10.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449880+0000","last_change":"2023-10-08T05:36:16.217935+0000","last_active":"2023-10-08T05:36:18.449880+0000","last_peered":"2023-10-08T05:36:18.449880+0000","last_clean":"2023-10-08T05:36:18.449880+0000","last_became_active":"2023-10-08T05:36:16.217530+0000","last_became_peered":"2023-10-08T05:36:16.217530+0000","last_unstale":"2023-10-08T05:36:18.449880+0000","last_undegraded":"2023-10-08T05:36:18.449880+0000","last_fullsized":"2023-10-08T05:36:18.449880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:07.031678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194389+0000","last_change":"2023-10-08T05:36:11.945050+0000","last_active":"2023-10-08T05:36:16.194389+0000","last_peered":"2023-10-08T05:36:16.194389+0000","last_clean":"2023-10-08T05:36:16.194389+0000","last_became_active":"2023-10-08T05:36:11.944419+0000","last_became_peered":"2023-10-08T05:36:11.944419+0000","last_unstale":"2023-10-08T05:36:16.194389+0000","last_undegraded":"2023-10-08T05:36:16.194389+0000","last_fullsized":"2023-10-08T05:36:16.194389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194769+0000","last_change":"2023-10-08T05:36:11.176221+0000","last_active":"2023-10-08T05:36:16.194769+0000","last_peered":"2023-10-08T05:36:16.194769+0000","last_clean":"2023-10-08T05:36:16.194769+0000","last_became_active":"2023-10-08T05:36:11.175839+0000","last_became_peered":"2023-10-08T05:36:11.175839+0000","last_unstale":"2023-10-08T05:36:16.194769+0000","last_undegraded":"2023-10-08T05:36:16.194769+0000","last_fullsized":"2023-10-08T05:36:16.194769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:44.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913652+0000","last_change":"2023-10-08T05:36:16.240543+0000","last_active":"2023-10-08T05:36:16.913652+0000","last_peered":"2023-10-08T05:36:16.913652+0000","last_clean":"2023-10-08T05:36:16.913652+0000","last_became_active":"2023-10-08T05:36:16.240104+0000","last_became_peered":"2023-10-08T05:36:16.240104+0000","last_unstale":"2023-10-08T05:36:16.913652+0000","last_undegraded":"2023-10-08T05:36:16.913652+0000","last_fullsized":"2023-10-08T05:36:16.913652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:21.958405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239177+0000","last_change":"2023-10-08T05:36:11.952574+0000","last_active":"2023-10-08T05:36:16.239177+0000","last_peered":"2023-10-08T05:36:16.239177+0000","last_clean":"2023-10-08T05:36:16.239177+0000","last_became_active":"2023-10-08T05:36:11.952287+0000","last_became_peered":"2023-10-08T05:36:11.952287+0000","last_unstale":"2023-10-08T05:36:16.239177+0000","last_undegraded":"2023-10-08T05:36:16.239177+0000","last_fullsized":"2023-10-08T05:36:16.239177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215860+0000","last_change":"2023-10-08T05:36:11.183076+0000","last_active":"2023-10-08T05:36:16.215860+0000","last_peered":"2023-10-08T05:36:16.215860+0000","last_clean":"2023-10-08T05:36:16.215860+0000","last_became_active":"2023-10-08T05:36:11.182702+0000","last_became_peered":"2023-10-08T05:36:11.182702+0000","last_unstale":"2023-10-08T05:36:16.215860+0000","last_undegraded":"2023-10-08T05:36:16.215860+0000","last_fullsized":"2023-10-08T05:36:16.215860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:45.848562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.912758+0000","last_change":"2023-10-08T05:36:16.240654+0000","last_active":"2023-10-08T05:36:16.912758+0000","last_peered":"2023-10-08T05:36:16.912758+0000","last_clean":"2023-10-08T05:36:16.912758+0000","last_became_active":"2023-10-08T05:36:16.240179+0000","last_became_peered":"2023-10-08T05:36:16.240179+0000","last_unstale":"2023-10-08T05:36:16.912758+0000","last_undegraded":"2023-10-08T05:36:16.912758+0000","last_fullsized":"2023-10-08T05:36:16.912758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:09:12.638961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205225+0000","last_change":"2023-10-08T05:36:11.948816+0000","last_active":"2023-10-08T05:36:16.205225+0000","last_peered":"2023-10-08T05:36:16.205225+0000","last_clean":"2023-10-08T05:36:16.205225+0000","last_became_active":"2023-10-08T05:36:11.948501+0000","last_became_peered":"2023-10-08T05:36:11.948501+0000","last_unstale":"2023-10-08T05:36:16.205225+0000","last_undegraded":"2023-10-08T05:36:16.205225+0000","last_fullsized":"2023-10-08T05:36:16.205225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:43.750000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206453+0000","last_change":"2023-10-08T05:36:11.172445+0000","last_active":"2023-10-08T05:36:16.206453+0000","last_peered":"2023-10-08T05:36:16.206453+0000","last_clean":"2023-10-08T05:36:16.206453+0000","last_became_active":"2023-10-08T05:36:11.172090+0000","last_became_peered":"2023-10-08T05:36:11.172090+0000","last_unstale":"2023-10-08T05:36:16.206453+0000","last_undegraded":"2023-10-08T05:36:16.206453+0000","last_fullsized":"2023-10-08T05:36:16.206453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:58.268025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.348870+0000","last_change":"2023-10-08T05:36:16.240874+0000","last_active":"2023-10-08T05:36:17.348870+0000","last_peered":"2023-10-08T05:36:17.348870+0000","last_clean":"2023-10-08T05:36:17.348870+0000","last_became_active":"2023-10-08T05:36:16.240420+0000","last_became_peered":"2023-10-08T05:36:16.240420+0000","last_unstale":"2023-10-08T05:36:17.348870+0000","last_undegraded":"2023-10-08T05:36:17.348870+0000","last_fullsized":"2023-10-08T05:36:17.348870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:06.106435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239249+0000","last_change":"2023-10-08T05:36:11.949913+0000","last_active":"2023-10-08T05:36:16.239249+0000","last_peered":"2023-10-08T05:36:16.239249+0000","last_clean":"2023-10-08T05:36:16.239249+0000","last_became_active":"2023-10-08T05:36:11.946993+0000","last_became_peered":"2023-10-08T05:36:11.946993+0000","last_unstale":"2023-10-08T05:36:16.239249+0000","last_undegraded":"2023-10-08T05:36:16.239249+0000","last_fullsized":"2023-10-08T05:36:16.239249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:02.994498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:31.450726+0000","last_change":"2023-10-08T05:36:11.177181+0000","last_active":"2023-10-08T05:36:31.450726+0000","last_peered":"2023-10-08T05:36:31.450726+0000","last_clean":"2023-10-08T05:36:31.450726+0000","last_became_active":"2023-10-08T05:36:11.176830+0000","last_became_peered":"2023-10-08T05:36:11.176830+0000","last_unstale":"2023-10-08T05:36:31.450726+0000","last_undegraded":"2023-10-08T05:36:31.450726+0000","last_fullsized":"2023-10-08T05:36:31.450726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:13.509993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.916338+0000","last_change":"2023-10-08T05:36:16.223391+0000","last_active":"2023-10-08T05:36:16.916338+0000","last_peered":"2023-10-08T05:36:16.916338+0000","last_clean":"2023-10-08T05:36:16.916338+0000","last_became_active":"2023-10-08T05:36:16.222997+0000","last_became_peered":"2023-10-08T05:36:16.222997+0000","last_unstale":"2023-10-08T05:36:16.916338+0000","last_undegraded":"2023-10-08T05:36:16.916338+0000","last_fullsized":"2023-10-08T05:36:16.916338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:15.664459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213269+0000","last_change":"2023-10-08T05:36:11.936436+0000","last_active":"2023-10-08T05:36:16.213269+0000","last_peered":"2023-10-08T05:36:16.213269+0000","last_clean":"2023-10-08T05:36:16.213269+0000","last_became_active":"2023-10-08T05:36:11.936161+0000","last_became_peered":"2023-10-08T05:36:11.936161+0000","last_unstale":"2023-10-08T05:36:16.213269+0000","last_undegraded":"2023-10-08T05:36:16.213269+0000","last_fullsized":"2023-10-08T05:36:16.213269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:36.016909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207221+0000","last_change":"2023-10-08T05:36:11.183582+0000","last_active":"2023-10-08T05:36:16.207221+0000","last_peered":"2023-10-08T05:36:16.207221+0000","last_clean":"2023-10-08T05:36:16.207221+0000","last_became_active":"2023-10-08T05:36:11.183264+0000","last_became_peered":"2023-10-08T05:36:11.183264+0000","last_unstale":"2023-10-08T05:36:16.207221+0000","last_undegraded":"2023-10-08T05:36:16.207221+0000","last_fullsized":"2023-10-08T05:36:16.207221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:16.799817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451297+0000","last_change":"2023-10-08T05:36:16.228720+0000","last_active":"2023-10-08T05:36:18.451297+0000","last_peered":"2023-10-08T05:36:18.451297+0000","last_clean":"2023-10-08T05:36:18.451297+0000","last_became_active":"2023-10-08T05:36:16.227638+0000","last_became_peered":"2023-10-08T05:36:16.227638+0000","last_unstale":"2023-10-08T05:36:18.451297+0000","last_undegraded":"2023-10-08T05:36:18.451297+0000","last_fullsized":"2023-10-08T05:36:18.451297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:20.640196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205640+0000","last_change":"2023-10-08T05:36:11.933926+0000","last_active":"2023-10-08T05:36:16.205640+0000","last_peered":"2023-10-08T05:36:16.205640+0000","last_clean":"2023-10-08T05:36:16.205640+0000","last_became_active":"2023-10-08T05:36:11.933648+0000","last_became_peered":"2023-10-08T05:36:11.933648+0000","last_unstale":"2023-10-08T05:36:16.205640+0000","last_undegraded":"2023-10-08T05:36:16.205640+0000","last_fullsized":"2023-10-08T05:36:16.205640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215215+0000","last_change":"2023-10-08T05:36:11.183379+0000","last_active":"2023-10-08T05:36:16.215215+0000","last_peered":"2023-10-08T05:36:16.215215+0000","last_clean":"2023-10-08T05:36:16.215215+0000","last_became_active":"2023-10-08T05:36:11.182862+0000","last_became_peered":"2023-10-08T05:36:11.182862+0000","last_unstale":"2023-10-08T05:36:16.215215+0000","last_undegraded":"2023-10-08T05:36:16.215215+0000","last_fullsized":"2023-10-08T05:36:16.215215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:10.147911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914439+0000","last_change":"2023-10-08T05:36:16.216716+0000","last_active":"2023-10-08T05:36:16.914439+0000","last_peered":"2023-10-08T05:36:16.914439+0000","last_clean":"2023-10-08T05:36:16.914439+0000","last_became_active":"2023-10-08T05:36:16.216266+0000","last_became_peered":"2023-10-08T05:36:16.216266+0000","last_unstale":"2023-10-08T05:36:16.914439+0000","last_undegraded":"2023-10-08T05:36:16.914439+0000","last_fullsized":"2023-10-08T05:36:16.914439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:45.158099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214937+0000","last_change":"2023-10-08T05:36:11.949702+0000","last_active":"2023-10-08T05:36:16.214937+0000","last_peered":"2023-10-08T05:36:16.214937+0000","last_clean":"2023-10-08T05:36:16.214937+0000","last_became_active":"2023-10-08T05:36:11.949421+0000","last_became_peered":"2023-10-08T05:36:11.949421+0000","last_unstale":"2023-10-08T05:36:16.214937+0000","last_undegraded":"2023-10-08T05:36:16.214937+0000","last_fullsized":"2023-10-08T05:36:16.214937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195931+0000","last_change":"2023-10-08T05:36:11.189802+0000","last_active":"2023-10-08T05:36:16.195931+0000","last_peered":"2023-10-08T05:36:16.195931+0000","last_clean":"2023-10-08T05:36:16.195931+0000","last_became_active":"2023-10-08T05:36:11.189447+0000","last_became_peered":"2023-10-08T05:36:11.189447+0000","last_unstale":"2023-10-08T05:36:16.195931+0000","last_undegraded":"2023-10-08T05:36:16.195931+0000","last_fullsized":"2023-10-08T05:36:16.195931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:08.847794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.911933+0000","last_change":"2023-10-08T05:36:16.241184+0000","last_active":"2023-10-08T05:36:16.911933+0000","last_peered":"2023-10-08T05:36:16.911933+0000","last_clean":"2023-10-08T05:36:16.911933+0000","last_became_active":"2023-10-08T05:36:16.240531+0000","last_became_peered":"2023-10-08T05:36:16.240531+0000","last_unstale":"2023-10-08T05:36:16.911933+0000","last_undegraded":"2023-10-08T05:36:16.911933+0000","last_fullsized":"2023-10-08T05:36:16.911933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:17:23.235254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449495+0000","last_change":"2023-10-08T05:36:16.241230+0000","last_active":"2023-10-08T05:36:18.449495+0000","last_peered":"2023-10-08T05:36:18.449495+0000","last_clean":"2023-10-08T05:36:18.449495+0000","last_became_active":"2023-10-08T05:36:16.240814+0000","last_became_peered":"2023-10-08T05:36:16.240814+0000","last_unstale":"2023-10-08T05:36:18.449495+0000","last_undegraded":"2023-10-08T05:36:18.449495+0000","last_fullsized":"2023-10-08T05:36:18.449495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:25.754577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.202997+0000","last_change":"2023-10-08T05:36:11.941419+0000","last_active":"2023-10-08T05:36:16.202997+0000","last_peered":"2023-10-08T05:36:16.202997+0000","last_clean":"2023-10-08T05:36:16.202997+0000","last_became_active":"2023-10-08T05:36:11.941131+0000","last_became_peered":"2023-10-08T05:36:11.941131+0000","last_unstale":"2023-10-08T05:36:16.202997+0000","last_undegraded":"2023-10-08T05:36:16.202997+0000","last_fullsized":"2023-10-08T05:36:16.202997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214657+0000","last_change":"2023-10-08T05:36:11.182594+0000","last_active":"2023-10-08T05:36:16.214657+0000","last_peered":"2023-10-08T05:36:16.214657+0000","last_clean":"2023-10-08T05:36:16.214657+0000","last_became_active":"2023-10-08T05:36:11.182297+0000","last_became_peered":"2023-10-08T05:36:11.182297+0000","last_unstale":"2023-10-08T05:36:16.214657+0000","last_undegraded":"2023-10-08T05:36:16.214657+0000","last_fullsized":"2023-10-08T05:36:16.214657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:59.799745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914977+0000","last_change":"2023-10-08T05:36:16.220333+0000","last_active":"2023-10-08T05:36:16.914977+0000","last_peered":"2023-10-08T05:36:16.914977+0000","last_clean":"2023-10-08T05:36:16.914977+0000","last_became_active":"2023-10-08T05:36:16.219836+0000","last_became_peered":"2023-10-08T05:36:16.219836+0000","last_unstale":"2023-10-08T05:36:16.914977+0000","last_undegraded":"2023-10-08T05:36:16.914977+0000","last_fullsized":"2023-10-08T05:36:16.914977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:32.619157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195436+0000","last_change":"2023-10-08T05:36:11.945676+0000","last_active":"2023-10-08T05:36:16.195436+0000","last_peered":"2023-10-08T05:36:16.195436+0000","last_clean":"2023-10-08T05:36:16.195436+0000","last_became_active":"2023-10-08T05:36:11.945314+0000","last_became_peered":"2023-10-08T05:36:11.945314+0000","last_unstale":"2023-10-08T05:36:16.195436+0000","last_undegraded":"2023-10-08T05:36:16.195436+0000","last_fullsized":"2023-10-08T05:36:16.195436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.197371+0000","last_change":"2023-10-08T05:36:11.176854+0000","last_active":"2023-10-08T05:36:16.197371+0000","last_peered":"2023-10-08T05:36:16.197371+0000","last_clean":"2023-10-08T05:36:16.197371+0000","last_became_active":"2023-10-08T05:36:11.176546+0000","last_became_peered":"2023-10-08T05:36:11.176546+0000","last_unstale":"2023-10-08T05:36:16.197371+0000","last_undegraded":"2023-10-08T05:36:16.197371+0000","last_fullsized":"2023-10-08T05:36:16.197371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:16.815302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195412+0000","last_change":"2023-10-08T05:36:11.184441+0000","last_active":"2023-10-08T05:36:16.195412+0000","last_peered":"2023-10-08T05:36:16.195412+0000","last_clean":"2023-10-08T05:36:16.195412+0000","last_became_active":"2023-10-08T05:36:11.184159+0000","last_became_peered":"2023-10-08T05:36:11.184159+0000","last_unstale":"2023-10-08T05:36:16.195412+0000","last_undegraded":"2023-10-08T05:36:16.195412+0000","last_fullsized":"2023-10-08T05:36:16.195412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206365+0000","last_change":"2023-10-08T05:36:11.935252+0000","last_active":"2023-10-08T05:36:16.206365+0000","last_peered":"2023-10-08T05:36:16.206365+0000","last_clean":"2023-10-08T05:36:16.206365+0000","last_became_active":"2023-10-08T05:36:11.934390+0000","last_became_peered":"2023-10-08T05:36:11.934390+0000","last_unstale":"2023-10-08T05:36:16.206365+0000","last_undegraded":"2023-10-08T05:36:16.206365+0000","last_fullsized":"2023-10-08T05:36:16.206365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.242390+0000","last_change":"2023-10-08T05:36:16.217984+0000","last_active":"2023-10-08T05:36:17.242390+0000","last_peered":"2023-10-08T05:36:17.242390+0000","last_clean":"2023-10-08T05:36:17.242390+0000","last_became_active":"2023-10-08T05:36:16.217614+0000","last_became_peered":"2023-10-08T05:36:16.217614+0000","last_unstale":"2023-10-08T05:36:17.242390+0000","last_undegraded":"2023-10-08T05:36:17.242390+0000","last_fullsized":"2023-10-08T05:36:17.242390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:47.787180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194733+0000","last_change":"2023-10-08T05:36:11.182558+0000","last_active":"2023-10-08T05:36:16.194733+0000","last_peered":"2023-10-08T05:36:16.194733+0000","last_clean":"2023-10-08T05:36:16.194733+0000","last_became_active":"2023-10-08T05:36:11.181822+0000","last_became_peered":"2023-10-08T05:36:11.181822+0000","last_unstale":"2023-10-08T05:36:16.194733+0000","last_undegraded":"2023-10-08T05:36:16.194733+0000","last_fullsized":"2023-10-08T05:36:16.194733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214758+0000","last_change":"2023-10-08T05:36:11.941704+0000","last_active":"2023-10-08T05:36:16.214758+0000","last_peered":"2023-10-08T05:36:16.214758+0000","last_clean":"2023-10-08T05:36:16.214758+0000","last_became_active":"2023-10-08T05:36:11.941431+0000","last_became_peered":"2023-10-08T05:36:11.941431+0000","last_unstale":"2023-10-08T05:36:16.214758+0000","last_undegraded":"2023-10-08T05:36:16.214758+0000","last_fullsized":"2023-10-08T05:36:16.214758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:00.811496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913836+0000","last_change":"2023-10-08T05:36:16.220904+0000","last_active":"2023-10-08T05:36:16.913836+0000","last_peered":"2023-10-08T05:36:16.913836+0000","last_clean":"2023-10-08T05:36:16.913836+0000","last_became_active":"2023-10-08T05:36:16.220521+0000","last_became_peered":"2023-10-08T05:36:16.220521+0000","last_unstale":"2023-10-08T05:36:16.913836+0000","last_undegraded":"2023-10-08T05:36:16.913836+0000","last_fullsized":"2023-10-08T05:36:16.913836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:52:02.322999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194151+0000","last_change":"2023-10-08T05:36:11.945315+0000","last_active":"2023-10-08T05:36:16.194151+0000","last_peered":"2023-10-08T05:36:16.194151+0000","last_clean":"2023-10-08T05:36:16.194151+0000","last_became_active":"2023-10-08T05:36:11.944775+0000","last_became_peered":"2023-10-08T05:36:11.944775+0000","last_unstale":"2023-10-08T05:36:16.194151+0000","last_undegraded":"2023-10-08T05:36:16.194151+0000","last_fullsized":"2023-10-08T05:36:16.194151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.189968+0000","last_change":"2023-10-08T05:36:11.938156+0000","last_active":"2023-10-08T05:36:16.189968+0000","last_peered":"2023-10-08T05:36:16.189968+0000","last_clean":"2023-10-08T05:36:16.189968+0000","last_became_active":"2023-10-08T05:36:11.937878+0000","last_became_peered":"2023-10-08T05:36:11.937878+0000","last_unstale":"2023-10-08T05:36:16.189968+0000","last_undegraded":"2023-10-08T05:36:16.189968+0000","last_fullsized":"2023-10-08T05:36:16.189968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:00.811496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215846+0000","last_change":"2023-10-08T05:36:11.944484+0000","last_active":"2023-10-08T05:36:16.215846+0000","last_peered":"2023-10-08T05:36:16.215846+0000","last_clean":"2023-10-08T05:36:16.215846+0000","last_became_active":"2023-10-08T05:36:11.943696+0000","last_became_peered":"2023-10-08T05:36:11.943696+0000","last_unstale":"2023-10-08T05:36:16.215846+0000","last_undegraded":"2023-10-08T05:36:16.215846+0000","last_fullsized":"2023-10-08T05:36:16.215846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:12.498480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206284+0000","last_change":"2023-10-08T05:36:11.944505+0000","last_active":"2023-10-08T05:36:16.206284+0000","last_peered":"2023-10-08T05:36:16.206284+0000","last_clean":"2023-10-08T05:36:16.206284+0000","last_became_active":"2023-10-08T05:36:11.944084+0000","last_became_peered":"2023-10-08T05:36:11.944084+0000","last_unstale":"2023-10-08T05:36:16.206284+0000","last_undegraded":"2023-10-08T05:36:16.206284+0000","last_fullsized":"2023-10-08T05:36:16.206284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.197543+0000","last_change":"2023-10-08T05:36:11.936235+0000","last_active":"2023-10-08T05:36:16.197543+0000","last_peered":"2023-10-08T05:36:16.197543+0000","last_clean":"2023-10-08T05:36:16.197543+0000","last_became_active":"2023-10-08T05:36:11.935816+0000","last_became_peered":"2023-10-08T05:36:11.935816+0000","last_unstale":"2023-10-08T05:36:16.197543+0000","last_undegraded":"2023-10-08T05:36:16.197543+0000","last_fullsized":"2023-10-08T05:36:16.197543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206800+0000","last_change":"2023-10-08T05:36:11.940291+0000","last_active":"2023-10-08T05:36:16.206800+0000","last_peered":"2023-10-08T05:36:16.206800+0000","last_clean":"2023-10-08T05:36:16.206800+0000","last_became_active":"2023-10-08T05:36:11.940019+0000","last_became_peered":"2023-10-08T05:36:11.940019+0000","last_unstale":"2023-10-08T05:36:16.206800+0000","last_undegraded":"2023-10-08T05:36:16.206800+0000","last_fullsized":"2023-10-08T05:36:16.206800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195886+0000","last_change":"2023-10-08T05:36:11.949017+0000","last_active":"2023-10-08T05:36:16.195886+0000","last_peered":"2023-10-08T05:36:16.195886+0000","last_clean":"2023-10-08T05:36:16.195886+0000","last_became_active":"2023-10-08T05:36:11.948741+0000","last_became_peered":"2023-10-08T05:36:11.948741+0000","last_unstale":"2023-10-08T05:36:16.195886+0000","last_undegraded":"2023-10-08T05:36:16.195886+0000","last_fullsized":"2023-10-08T05:36:16.195886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198027+0000","last_change":"2023-10-08T05:36:11.937078+0000","last_active":"2023-10-08T05:36:16.198027+0000","last_peered":"2023-10-08T05:36:16.198027+0000","last_clean":"2023-10-08T05:36:16.198027+0000","last_became_active":"2023-10-08T05:36:11.936737+0000","last_became_peered":"2023-10-08T05:36:11.936737+0000","last_unstale":"2023-10-08T05:36:16.198027+0000","last_undegraded":"2023-10-08T05:36:16.198027+0000","last_fullsized":"2023-10-08T05:36:16.198027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214525+0000","last_change":"2023-10-08T05:36:11.945582+0000","last_active":"2023-10-08T05:36:16.214525+0000","last_peered":"2023-10-08T05:36:16.214525+0000","last_clean":"2023-10-08T05:36:16.214525+0000","last_became_active":"2023-10-08T05:36:11.945066+0000","last_became_peered":"2023-10-08T05:36:11.945066+0000","last_unstale":"2023-10-08T05:36:16.214525+0000","last_undegraded":"2023-10-08T05:36:16.214525+0000","last_fullsized":"2023-10-08T05:36:16.214525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196604+0000","last_change":"2023-10-08T05:36:11.945936+0000","last_active":"2023-10-08T05:36:16.196604+0000","last_peered":"2023-10-08T05:36:16.196604+0000","last_clean":"2023-10-08T05:36:16.196604+0000","last_became_active":"2023-10-08T05:36:11.945384+0000","last_became_peered":"2023-10-08T05:36:11.945384+0000","last_unstale":"2023-10-08T05:36:16.196604+0000","last_undegraded":"2023-10-08T05:36:16.196604+0000","last_fullsized":"2023-10-08T05:36:16.196604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215763+0000","last_change":"2023-10-08T05:36:11.949155+0000","last_active":"2023-10-08T05:36:16.215763+0000","last_peered":"2023-10-08T05:36:16.215763+0000","last_clean":"2023-10-08T05:36:16.215763+0000","last_became_active":"2023-10-08T05:36:11.948867+0000","last_became_peered":"2023-10-08T05:36:11.948867+0000","last_unstale":"2023-10-08T05:36:16.215763+0000","last_undegraded":"2023-10-08T05:36:16.215763+0000","last_fullsized":"2023-10-08T05:36:16.215763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:11.627279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206186+0000","last_change":"2023-10-08T05:36:11.949363+0000","last_active":"2023-10-08T05:36:16.206186+0000","last_peered":"2023-10-08T05:36:16.206186+0000","last_clean":"2023-10-08T05:36:16.206186+0000","last_became_active":"2023-10-08T05:36:11.949059+0000","last_became_peered":"2023-10-08T05:36:11.949059+0000","last_unstale":"2023-10-08T05:36:16.206186+0000","last_undegraded":"2023-10-08T05:36:16.206186+0000","last_fullsized":"2023-10-08T05:36:16.206186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:00.571257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195787+0000","last_change":"2023-10-08T05:36:11.936518+0000","last_active":"2023-10-08T05:36:16.195787+0000","last_peered":"2023-10-08T05:36:16.195787+0000","last_clean":"2023-10-08T05:36:16.195787+0000","last_became_active":"2023-10-08T05:36:11.936138+0000","last_became_peered":"2023-10-08T05:36:11.936138+0000","last_unstale":"2023-10-08T05:36:16.195787+0000","last_undegraded":"2023-10-08T05:36:16.195787+0000","last_fullsized":"2023-10-08T05:36:16.195787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239530+0000","last_change":"2023-10-08T05:36:11.940992+0000","last_active":"2023-10-08T05:36:16.239530+0000","last_peered":"2023-10-08T05:36:16.239530+0000","last_clean":"2023-10-08T05:36:16.239530+0000","last_became_active":"2023-10-08T05:36:11.940700+0000","last_became_peered":"2023-10-08T05:36:11.940700+0000","last_unstale":"2023-10-08T05:36:16.239530+0000","last_undegraded":"2023-10-08T05:36:16.239530+0000","last_fullsized":"2023-10-08T05:36:16.239530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:20.449484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190822+0000","last_change":"2023-10-08T05:36:11.937136+0000","last_active":"2023-10-08T05:36:16.190822+0000","last_peered":"2023-10-08T05:36:16.190822+0000","last_clean":"2023-10-08T05:36:16.190822+0000","last_became_active":"2023-10-08T05:36:11.936761+0000","last_became_peered":"2023-10-08T05:36:11.936761+0000","last_unstale":"2023-10-08T05:36:16.190822+0000","last_undegraded":"2023-10-08T05:36:16.190822+0000","last_fullsized":"2023-10-08T05:36:16.190822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:50.179123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215403+0000","last_change":"2023-10-08T05:36:11.943099+0000","last_active":"2023-10-08T05:36:16.215403+0000","last_peered":"2023-10-08T05:36:16.215403+0000","last_clean":"2023-10-08T05:36:16.215403+0000","last_became_active":"2023-10-08T05:36:11.942792+0000","last_became_peered":"2023-10-08T05:36:11.942792+0000","last_unstale":"2023-10-08T05:36:16.215403+0000","last_undegraded":"2023-10-08T05:36:16.215403+0000","last_fullsized":"2023-10-08T05:36:16.215403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214354+0000","last_change":"2023-10-08T05:36:11.938711+0000","last_active":"2023-10-08T05:36:16.214354+0000","last_peered":"2023-10-08T05:36:16.214354+0000","last_clean":"2023-10-08T05:36:16.214354+0000","last_became_active":"2023-10-08T05:36:11.938449+0000","last_became_peered":"2023-10-08T05:36:11.938449+0000","last_unstale":"2023-10-08T05:36:16.214354+0000","last_undegraded":"2023-10-08T05:36:16.214354+0000","last_fullsized":"2023-10-08T05:36:16.214354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:56.745041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203081+0000","last_change":"2023-10-08T05:36:11.950960+0000","last_active":"2023-10-08T05:36:16.203081+0000","last_peered":"2023-10-08T05:36:16.203081+0000","last_clean":"2023-10-08T05:36:16.203081+0000","last_became_active":"2023-10-08T05:36:11.950582+0000","last_became_peered":"2023-10-08T05:36:11.950582+0000","last_unstale":"2023-10-08T05:36:16.203081+0000","last_undegraded":"2023-10-08T05:36:16.203081+0000","last_fullsized":"2023-10-08T05:36:16.203081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205140+0000","last_change":"2023-10-08T05:36:11.938997+0000","last_active":"2023-10-08T05:36:16.205140+0000","last_peered":"2023-10-08T05:36:16.205140+0000","last_clean":"2023-10-08T05:36:16.205140+0000","last_became_active":"2023-10-08T05:36:11.938641+0000","last_became_peered":"2023-10-08T05:36:11.938641+0000","last_unstale":"2023-10-08T05:36:16.205140+0000","last_undegraded":"2023-10-08T05:36:16.205140+0000","last_fullsized":"2023-10-08T05:36:16.205140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:54.023070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194362+0000","last_change":"2023-10-08T05:36:11.938973+0000","last_active":"2023-10-08T05:36:16.194362+0000","last_peered":"2023-10-08T05:36:16.194362+0000","last_clean":"2023-10-08T05:36:16.194362+0000","last_became_active":"2023-10-08T05:36:11.938633+0000","last_became_peered":"2023-10-08T05:36:11.938633+0000","last_unstale":"2023-10-08T05:36:16.194362+0000","last_undegraded":"2023-10-08T05:36:16.194362+0000","last_fullsized":"2023-10-08T05:36:16.194362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213118+0000","last_change":"2023-10-08T05:36:11.938748+0000","last_active":"2023-10-08T05:36:16.213118+0000","last_peered":"2023-10-08T05:36:16.213118+0000","last_clean":"2023-10-08T05:36:16.213118+0000","last_became_active":"2023-10-08T05:36:11.937801+0000","last_became_peered":"2023-10-08T05:36:11.937801+0000","last_unstale":"2023-10-08T05:36:16.213118+0000","last_undegraded":"2023-10-08T05:36:16.213118+0000","last_fullsized":"2023-10-08T05:36:16.213118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:59.279776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196976+0000","last_change":"2023-10-08T05:36:11.937561+0000","last_active":"2023-10-08T05:36:16.196976+0000","last_peered":"2023-10-08T05:36:16.196976+0000","last_clean":"2023-10-08T05:36:16.196976+0000","last_became_active":"2023-10-08T05:36:11.937299+0000","last_became_peered":"2023-10-08T05:36:11.937299+0000","last_unstale":"2023-10-08T05:36:16.196976+0000","last_undegraded":"2023-10-08T05:36:16.196976+0000","last_fullsized":"2023-10-08T05:36:16.196976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195655+0000","last_change":"2023-10-08T05:36:11.938330+0000","last_active":"2023-10-08T05:36:16.195655+0000","last_peered":"2023-10-08T05:36:16.195655+0000","last_clean":"2023-10-08T05:36:16.195655+0000","last_became_active":"2023-10-08T05:36:11.937993+0000","last_became_peered":"2023-10-08T05:36:11.937993+0000","last_unstale":"2023-10-08T05:36:16.195655+0000","last_undegraded":"2023-10-08T05:36:16.195655+0000","last_fullsized":"2023-10-08T05:36:16.195655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240495+0000","last_change":"2023-10-08T05:36:11.954185+0000","last_active":"2023-10-08T05:36:16.240495+0000","last_peered":"2023-10-08T05:36:16.240495+0000","last_clean":"2023-10-08T05:36:16.240495+0000","last_became_active":"2023-10-08T05:36:11.953900+0000","last_became_peered":"2023-10-08T05:36:11.953900+0000","last_unstale":"2023-10-08T05:36:16.240495+0000","last_undegraded":"2023-10-08T05:36:16.240495+0000","last_fullsized":"2023-10-08T05:36:16.240495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:11.627279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195359+0000","last_change":"2023-10-08T05:36:11.938140+0000","last_active":"2023-10-08T05:36:16.195359+0000","last_peered":"2023-10-08T05:36:16.195359+0000","last_clean":"2023-10-08T05:36:16.195359+0000","last_became_active":"2023-10-08T05:36:11.937817+0000","last_became_peered":"2023-10-08T05:36:11.937817+0000","last_unstale":"2023-10-08T05:36:16.195359+0000","last_undegraded":"2023-10-08T05:36:16.195359+0000","last_fullsized":"2023-10-08T05:36:16.195359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238846+0000","last_change":"2023-10-08T05:36:11.948351+0000","last_active":"2023-10-08T05:36:16.238846+0000","last_peered":"2023-10-08T05:36:16.238846+0000","last_clean":"2023-10-08T05:36:16.238846+0000","last_became_active":"2023-10-08T05:36:11.948045+0000","last_became_peered":"2023-10-08T05:36:11.948045+0000","last_unstale":"2023-10-08T05:36:16.238846+0000","last_undegraded":"2023-10-08T05:36:16.238846+0000","last_fullsized":"2023-10-08T05:36:16.238846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215673+0000","last_change":"2023-10-08T05:36:11.944663+0000","last_active":"2023-10-08T05:36:16.215673+0000","last_peered":"2023-10-08T05:36:16.215673+0000","last_clean":"2023-10-08T05:36:16.215673+0000","last_became_active":"2023-10-08T05:36:11.944158+0000","last_became_peered":"2023-10-08T05:36:11.944158+0000","last_unstale":"2023-10-08T05:36:16.215673+0000","last_undegraded":"2023-10-08T05:36:16.215673+0000","last_fullsized":"2023-10-08T05:36:16.215673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:16.603017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239112+0000","last_change":"2023-10-08T05:36:11.944406+0000","last_active":"2023-10-08T05:36:16.239112+0000","last_peered":"2023-10-08T05:36:16.239112+0000","last_clean":"2023-10-08T05:36:16.239112+0000","last_became_active":"2023-10-08T05:36:11.944031+0000","last_became_peered":"2023-10-08T05:36:11.944031+0000","last_unstale":"2023-10-08T05:36:16.239112+0000","last_undegraded":"2023-10-08T05:36:16.239112+0000","last_fullsized":"2023-10-08T05:36:16.239112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214934+0000","last_change":"2023-10-08T05:36:11.949094+0000","last_active":"2023-10-08T05:36:16.214934+0000","last_peered":"2023-10-08T05:36:16.214934+0000","last_clean":"2023-10-08T05:36:16.214934+0000","last_became_active":"2023-10-08T05:36:11.948663+0000","last_became_peered":"2023-10-08T05:36:11.948663+0000","last_unstale":"2023-10-08T05:36:16.214934+0000","last_undegraded":"2023-10-08T05:36:16.214934+0000","last_fullsized":"2023-10-08T05:36:16.214934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:46:57.883695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205484+0000","last_change":"2023-10-08T05:36:11.927796+0000","last_active":"2023-10-08T05:36:16.205484+0000","last_peered":"2023-10-08T05:36:16.205484+0000","last_clean":"2023-10-08T05:36:16.205484+0000","last_became_active":"2023-10-08T05:36:11.927200+0000","last_became_peered":"2023-10-08T05:36:11.927200+0000","last_unstale":"2023-10-08T05:36:16.205484+0000","last_undegraded":"2023-10-08T05:36:16.205484+0000","last_fullsized":"2023-10-08T05:36:16.205484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.208328+0000","last_change":"2023-10-08T05:36:11.937916+0000","last_active":"2023-10-08T05:36:16.208328+0000","last_peered":"2023-10-08T05:36:16.208328+0000","last_clean":"2023-10-08T05:36:16.208328+0000","last_became_active":"2023-10-08T05:36:11.937646+0000","last_became_peered":"2023-10-08T05:36:11.937646+0000","last_unstale":"2023-10-08T05:36:16.208328+0000","last_undegraded":"2023-10-08T05:36:16.208328+0000","last_fullsized":"2023-10-08T05:36:16.208328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240404+0000","last_change":"2023-10-08T05:36:11.952124+0000","last_active":"2023-10-08T05:36:16.240404+0000","last_peered":"2023-10-08T05:36:16.240404+0000","last_clean":"2023-10-08T05:36:16.240404+0000","last_became_active":"2023-10-08T05:36:11.949353+0000","last_became_peered":"2023-10-08T05:36:11.949353+0000","last_unstale":"2023-10-08T05:36:16.240404+0000","last_undegraded":"2023-10-08T05:36:16.240404+0000","last_fullsized":"2023-10-08T05:36:16.240404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:46:57.883695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.319}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8149999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5209999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4430000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.931}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.248}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2909999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.508}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.363}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5129999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:36:35.935 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T05:36:35.936 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-08T05:36:35.936 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-08T05:36:35.936 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-08T05:36:35.937 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-08T05:36:36.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.186+0000 7f9c0c51d640 1 Processor -- start 2023-10-08T05:36:36.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.186+0000 7f9c0c51d640 1 -- start start 2023-10-08T05:36:36.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.187+0000 7f9c0c51d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c04159ac0 0x7f9c0406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.187+0000 7f9c0c51d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0406dee0 con 0x7f9c040709e0 2023-10-08T05:36:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.187+0000 7f9c0c51d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0415d850 con 0x7f9c040ad790 2023-10-08T05:36:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.187+0000 7f9c0c51d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0415db30 con 0x7f9c04159ac0 2023-10-08T05:36:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c0aaba640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9c040709e0 0x7f9c040c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43612/0 (socket says 172.21.15.70:43612) 2023-10-08T05:36:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c0aaba640 1 -- 172.21.15.70:0/1321065760 learned_addr learned my addr 172.21.15.70:0/1321065760 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c0a2b9640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c04159ac0 0x7f9c0406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553801864 0 0) 0x7f9c0406dee0 con 0x7f9c040709e0 2023-10-08T05:36:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf0003520 con 0x7f9c040709e0 2023-10-08T05:36:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1082071574 0 0) 0x7f9bf0003520 con 0x7f9c040709e0 2023-10-08T05:36:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c04159ac0 msgr2=0x7f9c0406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c092b7640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c04159ac0 0x7f9c0406d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 >> v1:172.21.15.181:6789/0 conn(0x7f9c040ad790 legacy=0x7f9c040702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0415de10 con 0x7f9c040709e0 2023-10-08T05:36:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.188+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9bfc0019f0 con 0x7f9c040709e0 2023-10-08T05:36:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 >> v1:172.21.15.70:6789/0 conn(0x7f9c040709e0 legacy=0x7f9c040c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c092b7640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c04159ac0 0x7f9bf00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c092b7640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 0x7f9bf00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c092b7640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0004d00 0x7f9bf0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0415db30 con 0x7f9bf0003f20 2023-10-08T05:36:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf000ba60 con 0x7f9bf0004d00 2023-10-08T05:36:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf000bc20 con 0x7f9c04159ac0 2023-10-08T05:36:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c09ab8640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 0x7f9bf00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c0a2b9640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c04159ac0 0x7f9bf00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c0aaba640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0004d00 0x7f9bf0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c09ab8640 1 -- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c04159ac0 msgr2=0x7f9bf00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c09ab8640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c04159ac0 0x7f9bf00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c09ab8640 1 -- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0004d00 msgr2=0x7f9bf0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c09ab8640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0004d00 0x7f9bf0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.189+0000 7f9c09ab8640 1 -- 172.21.15.70:0/1321065760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0003520 con 0x7f9bf0003f20 2023-10-08T05:36:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.190+0000 7f9c09ab8640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 0x7f9bf00047c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4001720 tx=0x7f9bf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb6842e86680ccf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.191+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40052d0 con 0x7f9bf0003f20 2023-10-08T05:36:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.191+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf402e050 con 0x7f9bf0003f20 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.191+0000 7f9c092b7640 1 -- 172.21.15.70:0/1321065760 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4003830 con 0x7f9bf0003f20 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.191+0000 7f9c0c51d640 1 -- 172.21.15.70:0/1321065760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 msgr2=0x7f9bf00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.191+0000 7f9c0c51d640 1 --2- 172.21.15.70:0/1321065760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 0x7f9bf00047c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4001720 tx=0x7f9bf00057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.191+0000 7f9c0aaba640 1 -- 172.21.15.70:0/1321065760 reap_dead start 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.191+0000 7f9c0c51d640 1 -- 172.21.15.70:0/1321065760 shutdown_connections 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.191+0000 7f9c0c51d640 1 -- 172.21.15.70:0/1321065760 >> 172.21.15.70:0/1321065760 conn(0x7f9c0406a380 msgr2=0x7f9c040b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.191+0000 7f9c0c51d640 1 -- 172.21.15.70:0/1321065760 shutdown_connections 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.192+0000 7f9c0c51d640 1 -- 172.21.15.70:0/1321065760 wait complete. 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.193+0000 7f9c0c51d640 1 Processor -- start 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.193+0000 7f9c0c51d640 1 -- start start 2023-10-08T05:36:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.194+0000 7f9c0c51d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 0x7f9c0414e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.194+0000 7f9c0c51d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0414e100 0x7f9c040b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.194+0000 7f9c0c51d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414e580 0x7f9c040b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.194+0000 7f9c0c51d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0415a9f0 con 0x7f9bf0003f20 2023-10-08T05:36:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.194+0000 7f9c0c51d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04151200 con 0x7f9c0414e100 2023-10-08T05:36:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.194+0000 7f9c0c51d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040b2a90 con 0x7f9c0414e580 2023-10-08T05:36:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0a2b9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 0x7f9c0414e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0a2b9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 0x7f9c0414e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59796/0 (socket says 172.21.15.70:59796) 2023-10-08T05:36:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0a2b9640 1 -- 172.21.15.70:0/3681679894 learned_addr learned my addr 172.21.15.70:0/3681679894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c09ab8640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0414e100 0x7f9c040b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0aaba640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414e580 0x7f9c040b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0a2b9640 1 -- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414e580 msgr2=0x7f9c040b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0a2b9640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414e580 0x7f9c040b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0a2b9640 1 -- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0414e100 msgr2=0x7f9c040b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0a2b9640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0414e100 0x7f9c040b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0a2b9640 1 -- 172.21.15.70:0/3681679894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0415a0c0 con 0x7f9bf0003f20 2023-10-08T05:36:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0aaba640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414e580 0x7f9c040b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.195+0000 7f9c0a2b9640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 0x7f9c0414e980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c00011600 tx=0x7f9c00011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:36.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.196+0000 7f9bfaffd640 1 -- 172.21.15.70:0/3681679894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000f920 con 0x7f9bf0003f20 2023-10-08T05:36:36.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.196+0000 7f9bfaffd640 1 -- 172.21.15.70:0/3681679894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0000fa80 con 0x7f9bf0003f20 2023-10-08T05:36:36.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.196+0000 7f9bfaffd640 1 -- 172.21.15.70:0/3681679894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000fdf0 con 0x7f9bf0003f20 2023-10-08T05:36:36.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.196+0000 7f9c0c51d640 1 -- 172.21.15.70:0/3681679894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0415a3a0 con 0x7f9bf0003f20 2023-10-08T05:36:36.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.196+0000 7f9c0c51d640 1 -- 172.21.15.70:0/3681679894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0414eec0 con 0x7f9bf0003f20 2023-10-08T05:36:36.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.197+0000 7f9c0c51d640 1 -- 172.21.15.70:0/3681679894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c04067490 con 0x7f9bf0003f20 2023-10-08T05:36:36.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.198+0000 7f9bfaffd640 1 -- 172.21.15.70:0/3681679894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9c00015460 con 0x7f9bf0003f20 2023-10-08T05:36:36.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.199+0000 7f9bfaffd640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9be806ae70 0x7f9be806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.199+0000 7f9bfaffd640 1 -- 172.21.15.70:0/3681679894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9c00089e80 con 0x7f9bf0003f20 2023-10-08T05:36:36.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.200+0000 7f9c09ab8640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9be806ae70 0x7f9be806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.201+0000 7f9c09ab8640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9be806ae70 0x7f9be806d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4033000 tx=0x7f9bf4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:36.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.203+0000 7f9bfaffd640 1 -- 172.21.15.70:0/3681679894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c00054ba0 con 0x7f9bf0003f20 2023-10-08T05:36:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.440+0000 7f9c0c51d640 1 -- 172.21.15.70:0/3681679894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9c040b0ff0 con 0x7f9bf0003f20 2023-10-08T05:36:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.445+0000 7f9bfaffd640 1 -- 172.21.15.70:0/3681679894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171037 (secure 0 0 0) 0x7f9c00022090 con 0x7f9bf0003f20 2023-10-08T05:36:36.449 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.449+0000 7f9c0c51d640 1 -- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9be806ae70 msgr2=0x7f9be806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.449+0000 7f9c0c51d640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9be806ae70 0x7f9be806d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4033000 tx=0x7f9bf4009030 comp rx=0 tx=0).stop 2023-10-08T05:36:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.449+0000 7f9c0c51d640 1 -- 172.21.15.70:0/3681679894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 msgr2=0x7f9c0414e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.449+0000 7f9c0c51d640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 0x7f9c0414e980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c00011600 tx=0x7f9c00011ac0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.450+0000 7f9c0c51d640 1 -- 172.21.15.70:0/3681679894 shutdown_connections 2023-10-08T05:36:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.450+0000 7f9c0c51d640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9be806ae70 0x7f9be806d330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.450+0000 7f9c0c51d640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414e580 0x7f9c040b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.450+0000 7f9c0c51d640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c0414e100 0x7f9c040b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.450+0000 7f9c0c51d640 1 --2- 172.21.15.70:0/3681679894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0003f20 0x7f9c0414e980 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.450+0000 7f9c0c51d640 1 -- 172.21.15.70:0/3681679894 >> 172.21.15.70:0/3681679894 conn(0x7f9c0406a380 msgr2=0x7f9c040c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.450+0000 7f9c0c51d640 1 -- 172.21.15.70:0/3681679894 shutdown_connections 2023-10-08T05:36:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.450+0000 7f9c0c51d640 1 -- 172.21.15.70:0/3681679894 wait complete. 2023-10-08T05:36:36.469 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":5,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":2737793492},{"type":"v1","addr":"172.21.15.181:6833","nonce":2737793492}]},"active_addr":"172.21.15.181:6833/2737793492","active_change":"2023-10-08T05:35:57.048517+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4114,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3573082646}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3620192672}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3280597969}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":3583094356}]}]} 2023-10-08T05:36:36.473 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-08T05:36:36.474 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-08T05:36:36.474 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:36.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.689+0000 7f1f108d4640 1 Processor -- start 2023-10-08T05:36:36.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.689+0000 7f1f108d4640 1 -- start start 2023-10-08T05:36:36.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.691+0000 7f1f108d4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 0x7f1f0c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.691+0000 7f1f108d4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f0c0709e0 0x7f1f0c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.691+0000 7f1f108d4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c06dee0 con 0x7f1f0c0ad790 2023-10-08T05:36:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.691+0000 7f1f108d4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c15d850 con 0x7f1f0c0709e0 2023-10-08T05:36:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.691+0000 7f1f108d4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c15db30 con 0x7f1f0c159ac0 2023-10-08T05:36:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.691+0000 7f1f0ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f0c0709e0 0x7f1f0c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.691+0000 7f1f01d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1f0c0ad790 0x7f1f0c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43626/0 (socket says 172.21.15.70:43626) 2023-10-08T05:36:36.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.691+0000 7f1f01d9b640 1 -- 172.21.15.70:0/1605120628 learned_addr learned my addr 172.21.15.70:0/1605120628 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:36.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.692+0000 7f1f0a59c640 1 --2- 172.21.15.70:0/1605120628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 0x7f1f0c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.692+0000 7f1f0ad9d640 1 -- 172.21.15.70:0/1605120628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 msgr2=0x7f1f0c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:36.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.692+0000 7f1f0ad9d640 1 --2- 172.21.15.70:0/1605120628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 0x7f1f0c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.692+0000 7f1f0ad9d640 1 -- 172.21.15.70:0/1605120628 >> v1:172.21.15.70:6789/0 conn(0x7f1f0c0ad790 legacy=0x7f1f0c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.692+0000 7f1f0ad9d640 1 -- 172.21.15.70:0/1605120628 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c15de10 con 0x7f1f0c0709e0 2023-10-08T05:36:36.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.692+0000 7f1f0a59c640 1 --2- 172.21.15.70:0/1605120628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 0x7f1f0c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:36.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.693+0000 7f1f0ad9d640 1 --2- 172.21.15.70:0/1605120628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 0x7f1f0c0c8dd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1f040097a0 tx=0x7f1f0402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b429df68016f920e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:36.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.693+0000 7f1f09d9b640 1 -- 172.21.15.70:0/1605120628 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0402d040 con 0x7f1f0c0709e0 2023-10-08T05:36:36.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.693+0000 7f1f09d9b640 1 -- 172.21.15.70:0/1605120628 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f04031810 con 0x7f1f0c0709e0 2023-10-08T05:36:36.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.693+0000 7f1f09d9b640 1 -- 172.21.15.70:0/1605120628 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04031ba0 con 0x7f1f0c0709e0 2023-10-08T05:36:36.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.694+0000 7f1f108d4640 1 -- 172.21.15.70:0/1605120628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 msgr2=0x7f1f0c0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.694+0000 7f1f108d4640 1 --2- 172.21.15.70:0/1605120628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 0x7f1f0c0c8dd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1f040097a0 tx=0x7f1f0402b8b0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.694+0000 7f1f108d4640 1 -- 172.21.15.70:0/1605120628 shutdown_connections 2023-10-08T05:36:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.694+0000 7f1f108d4640 1 --2- 172.21.15.70:0/1605120628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 0x7f1f0c0c8dd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.694+0000 7f1f108d4640 1 --2- 172.21.15.70:0/1605120628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 0x7f1f0c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.694+0000 7f1f108d4640 1 -- 172.21.15.70:0/1605120628 >> 172.21.15.70:0/1605120628 conn(0x7f1f0c06a380 msgr2=0x7f1f0c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.694+0000 7f1f108d4640 1 -- 172.21.15.70:0/1605120628 shutdown_connections 2023-10-08T05:36:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.694+0000 7f1f108d4640 1 -- 172.21.15.70:0/1605120628 wait complete. 2023-10-08T05:36:36.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.696+0000 7f1f108d4640 1 Processor -- start 2023-10-08T05:36:36.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.696+0000 7f1f108d4640 1 -- start start 2023-10-08T05:36:36.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f108d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 0x7f1f0c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f108d4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0c0ad790 0x7f1f0c1f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f108d4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 0x7f1f0c152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f108d4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c1f28f0 con 0x7f1f0c0709e0 2023-10-08T05:36:36.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f108d4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c1f2b90 con 0x7f1f0c0ad790 2023-10-08T05:36:36.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f108d4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c1f2e70 con 0x7f1f0c159ac0 2023-10-08T05:36:36.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f0a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 0x7f1f0c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f0a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 0x7f1f0c15ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59808/0 (socket says 172.21.15.70:59808) 2023-10-08T05:36:36.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f0a59c640 1 -- 172.21.15.70:0/4182770385 learned_addr learned my addr 172.21.15.70:0/4182770385 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f0ad9d640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 0x7f1f0c152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f0a59c640 1 -- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 msgr2=0x7f1f0c152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f0a59c640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 0x7f1f0c152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f01d9b640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0c0ad790 0x7f1f0c1f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f0a59c640 1 -- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0c0ad790 msgr2=0x7f1f0c1f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f0a59c640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0c0ad790 0x7f1f0c1f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.697+0000 7f1f0a59c640 1 -- 172.21.15.70:0/4182770385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f04009400 con 0x7f1f0c0709e0 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.698+0000 7f1f0a59c640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 0x7f1f0c15ccb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00ce00 tx=0x7f1efc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.698+0000 7f1f037fe640 1 -- 172.21.15.70:0/4182770385 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00f890 con 0x7f1f0c0709e0 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.698+0000 7f1f037fe640 1 -- 172.21.15.70:0/4182770385 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc00f9f0 con 0x7f1f0c0709e0 2023-10-08T05:36:36.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.698+0000 7f1f108d4640 1 -- 172.21.15.70:0/4182770385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c1f3150 con 0x7f1f0c0709e0 2023-10-08T05:36:36.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.698+0000 7f1f037fe640 1 -- 172.21.15.70:0/4182770385 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00fdc0 con 0x7f1f0c0709e0 2023-10-08T05:36:36.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.698+0000 7f1f108d4640 1 -- 172.21.15.70:0/4182770385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c1f3430 con 0x7f1f0c0709e0 2023-10-08T05:36:36.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.699+0000 7f1f108d4640 1 -- 172.21.15.70:0/4182770385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c067490 con 0x7f1f0c0709e0 2023-10-08T05:36:36.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.700+0000 7f1f037fe640 1 -- 172.21.15.70:0/4182770385 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1efc00b040 con 0x7f1f0c0709e0 2023-10-08T05:36:36.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.701+0000 7f1f037fe640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1eec06aea0 0x7f1eec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:36.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.701+0000 7f1f037fe640 1 -- 172.21.15.70:0/4182770385 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1efc08a2f0 con 0x7f1f0c0709e0 2023-10-08T05:36:36.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.702+0000 7f1f01d9b640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1eec06aea0 0x7f1eec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:36.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.703+0000 7f1f01d9b640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1eec06aea0 0x7f1eec06d360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c1567f0 tx=0x7f1ef4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:36.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.706+0000 7f1f037fe640 1 -- 172.21.15.70:0/4182770385 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1efc0550a0 con 0x7f1f0c0709e0 2023-10-08T05:36:36.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.899+0000 7f1f108d4640 1 -- 172.21.15.70:0/4182770385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f0c0b0ff0 con 0x7f1f0c0709e0 2023-10-08T05:36:36.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.902+0000 7f1f037fe640 1 -- 172.21.15.70:0/4182770385 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f1efc0586c0 con 0x7f1f0c0709e0 2023-10-08T05:36:36.904 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:36.904 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:36.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.904+0000 7f1f108d4640 1 -- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1eec06aea0 msgr2=0x7f1eec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.904+0000 7f1f108d4640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1eec06aea0 0x7f1eec06d360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c1567f0 tx=0x7f1ef4009040 comp rx=0 tx=0).stop 2023-10-08T05:36:36.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.905+0000 7f1f108d4640 1 -- 172.21.15.70:0/4182770385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 msgr2=0x7f1f0c15ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.905+0000 7f1f108d4640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 0x7f1f0c15ccb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00ce00 tx=0x7f1efc00a650 comp rx=0 tx=0).stop 2023-10-08T05:36:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.905+0000 7f1f108d4640 1 -- 172.21.15.70:0/4182770385 shutdown_connections 2023-10-08T05:36:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.905+0000 7f1f108d4640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1eec06aea0 0x7f1eec06d360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.905+0000 7f1f108d4640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f0c159ac0 0x7f1f0c152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.905+0000 7f1f108d4640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f0c0ad790 0x7f1f0c1f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.905+0000 7f1f108d4640 1 --2- 172.21.15.70:0/4182770385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f0c0709e0 0x7f1f0c15ccb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.905+0000 7f1f108d4640 1 -- 172.21.15.70:0/4182770385 >> 172.21.15.70:0/4182770385 conn(0x7f1f0c06a380 msgr2=0x7f1f0c0c4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.905+0000 7f1f108d4640 1 -- 172.21.15.70:0/4182770385 shutdown_connections 2023-10-08T05:36:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:36.905+0000 7f1f108d4640 1 -- 172.21.15.70:0/4182770385 wait complete. 2023-10-08T05:36:36.926 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-08T05:36:36.926 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:37.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.135+0000 7f0f606e5640 1 Processor -- start 2023-10-08T05:36:37.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.135+0000 7f0f606e5640 1 -- start start 2023-10-08T05:36:37.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.136+0000 7f0f606e5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f58159ac0 0x7f0f5806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.136+0000 7f0f606e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f580ad790 0x7f0f580702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.136+0000 7f0f606e5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5806dee0 con 0x7f0f5815e330 2023-10-08T05:36:37.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.136+0000 7f0f606e5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5815d850 con 0x7f0f58159ac0 2023-10-08T05:36:37.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.136+0000 7f0f606e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5815db30 con 0x7f0f580ad790 2023-10-08T05:36:37.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.137+0000 7f0f5e481640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f58159ac0 0x7f0f5806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.137+0000 7f0f5e481640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f58159ac0 0x7f0f5806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59820/0 (socket says 172.21.15.70:59820) 2023-10-08T05:36:37.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.137+0000 7f0f5e481640 1 -- 172.21.15.70:0/1215396617 learned_addr learned my addr 172.21.15.70:0/1215396617 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.137+0000 7f0f5dc80640 1 --2- 172.21.15.70:0/1215396617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f580ad790 0x7f0f580702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.137+0000 7f0f5e481640 1 -- 172.21.15.70:0/1215396617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f580ad790 msgr2=0x7f0f580702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.137+0000 7f0f5e481640 1 --2- 172.21.15.70:0/1215396617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f580ad790 0x7f0f580702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.137+0000 7f0f5e481640 1 -- 172.21.15.70:0/1215396617 >> v1:172.21.15.181:6789/0 conn(0x7f0f5815e330 legacy=0x7f0f5806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.137+0000 7f0f5e481640 1 -- 172.21.15.70:0/1215396617 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5815de10 con 0x7f0f58159ac0 2023-10-08T05:36:37.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.137+0000 7f0f5dc80640 1 --2- 172.21.15.70:0/1215396617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f580ad790 0x7f0f580702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:37.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.138+0000 7f0f5e481640 1 --2- 172.21.15.70:0/1215396617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f58159ac0 0x7f0f5806d5e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009870 tx=0x7f0f5402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75445dfbe1dcdf60 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.138+0000 7f0f5d47f640 1 -- 172.21.15.70:0/1215396617 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54031640 con 0x7f0f58159ac0 2023-10-08T05:36:37.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.139+0000 7f0f5d47f640 1 -- 172.21.15.70:0/1215396617 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f540317a0 con 0x7f0f58159ac0 2023-10-08T05:36:37.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.139+0000 7f0f5d47f640 1 -- 172.21.15.70:0/1215396617 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54031b30 con 0x7f0f58159ac0 2023-10-08T05:36:37.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.139+0000 7f0f606e5640 1 -- 172.21.15.70:0/1215396617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f58159ac0 msgr2=0x7f0f5806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.139+0000 7f0f606e5640 1 --2- 172.21.15.70:0/1215396617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f58159ac0 0x7f0f5806d5e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009870 tx=0x7f0f5402b8f0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.139+0000 7f0f606e5640 1 -- 172.21.15.70:0/1215396617 shutdown_connections 2023-10-08T05:36:37.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.139+0000 7f0f606e5640 1 --2- 172.21.15.70:0/1215396617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f580ad790 0x7f0f580702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.139+0000 7f0f606e5640 1 --2- 172.21.15.70:0/1215396617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f58159ac0 0x7f0f5806d5e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.139+0000 7f0f606e5640 1 -- 172.21.15.70:0/1215396617 >> 172.21.15.70:0/1215396617 conn(0x7f0f5806a380 msgr2=0x7f0f580b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.139+0000 7f0f606e5640 1 -- 172.21.15.70:0/1215396617 shutdown_connections 2023-10-08T05:36:37.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.139+0000 7f0f606e5640 1 -- 172.21.15.70:0/1215396617 wait complete. 2023-10-08T05:36:37.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.141+0000 7f0f606e5640 1 Processor -- start 2023-10-08T05:36:37.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.141+0000 7f0f606e5640 1 -- start start 2023-10-08T05:36:37.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.142+0000 7f0f606e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f580ad790 0x7f0f580b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.142+0000 7f0f606e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f58159ac0 0x7f0f580b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.142+0000 7f0f606e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5815e330 0x7f0f581419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.142+0000 7f0f606e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581e34d0 con 0x7f0f580ad790 2023-10-08T05:36:37.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.142+0000 7f0f606e5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581e37b0 con 0x7f0f5815e330 2023-10-08T05:36:37.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.142+0000 7f0f606e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581e3a90 con 0x7f0f58159ac0 2023-10-08T05:36:37.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5e481640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f580ad790 0x7f0f580b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5e481640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f580ad790 0x7f0f580b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59824/0 (socket says 172.21.15.70:59824) 2023-10-08T05:36:37.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5e481640 1 -- 172.21.15.70:0/1202954232 learned_addr learned my addr 172.21.15.70:0/1202954232 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5dc80640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f58159ac0 0x7f0f580b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5ec82640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5815e330 0x7f0f581419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5e481640 1 -- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f58159ac0 msgr2=0x7f0f580b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5e481640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f58159ac0 0x7f0f580b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5e481640 1 -- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5815e330 msgr2=0x7f0f581419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5e481640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5815e330 0x7f0f581419f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5e481640 1 -- 172.21.15.70:0/1202954232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f540094d0 con 0x7f0f580ad790 2023-10-08T05:36:37.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.143+0000 7f0f5e481640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f580ad790 0x7f0f580b2f90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009870 tx=0x7f0f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.144+0000 7f0f4effd640 1 -- 172.21.15.70:0/1202954232 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5403ac60 con 0x7f0f580ad790 2023-10-08T05:36:37.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.144+0000 7f0f4effd640 1 -- 172.21.15.70:0/1202954232 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5403ade0 con 0x7f0f580ad790 2023-10-08T05:36:37.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.144+0000 7f0f4effd640 1 -- 172.21.15.70:0/1202954232 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54044630 con 0x7f0f580ad790 2023-10-08T05:36:37.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.144+0000 7f0f606e5640 1 -- 172.21.15.70:0/1202954232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f581e3d70 con 0x7f0f580ad790 2023-10-08T05:36:37.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.144+0000 7f0f606e5640 1 -- 172.21.15.70:0/1202954232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f581e4330 con 0x7f0f580ad790 2023-10-08T05:36:37.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.145+0000 7f0f606e5640 1 -- 172.21.15.70:0/1202954232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f58067490 con 0x7f0f580ad790 2023-10-08T05:36:37.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.146+0000 7f0f4effd640 1 -- 172.21.15.70:0/1202954232 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0f5402f050 con 0x7f0f580ad790 2023-10-08T05:36:37.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.147+0000 7f0f4effd640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0f3c06ae70 0x7f0f3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.147+0000 7f0f4effd640 1 -- 172.21.15.70:0/1202954232 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0f540ab9f0 con 0x7f0f580ad790 2023-10-08T05:36:37.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.147+0000 7f0f5dc80640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0f3c06ae70 0x7f0f3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.148+0000 7f0f5dc80640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0f3c06ae70 0x7f0f3c06d330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0f581f3b50 tx=0x7f0f48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.152+0000 7f0f4effd640 1 -- 172.21.15.70:0/1202954232 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f54076840 con 0x7f0f580ad790 2023-10-08T05:36:37.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.345+0000 7f0f606e5640 1 -- 172.21.15.70:0/1202954232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f580b0ff0 con 0x7f0f580ad790 2023-10-08T05:36:37.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.347+0000 7f0f4effd640 1 -- 172.21.15.70:0/1202954232 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f0f54035090 con 0x7f0f580ad790 2023-10-08T05:36:37.349 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:37.349 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:37.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 -- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0f3c06ae70 msgr2=0x7f0f3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0f3c06ae70 0x7f0f3c06d330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0f581f3b50 tx=0x7f0f48008040 comp rx=0 tx=0).stop 2023-10-08T05:36:37.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 -- 172.21.15.70:0/1202954232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f580ad790 msgr2=0x7f0f580b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f580ad790 0x7f0f580b2f90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009870 tx=0x7f0f540046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 -- 172.21.15.70:0/1202954232 shutdown_connections 2023-10-08T05:36:37.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0f3c06ae70 0x7f0f3c06d330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f5815e330 0x7f0f581419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f58159ac0 0x7f0f580b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 --2- 172.21.15.70:0/1202954232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f580ad790 0x7f0f580b2f90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 -- 172.21.15.70:0/1202954232 >> 172.21.15.70:0/1202954232 conn(0x7f0f5806a380 msgr2=0x7f0f58155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 -- 172.21.15.70:0/1202954232 shutdown_connections 2023-10-08T05:36:37.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.350+0000 7f0f606e5640 1 -- 172.21.15.70:0/1202954232 wait complete. 2023-10-08T05:36:37.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-08T05:36:37.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-08T05:36:37.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-08T05:36:37.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-08T05:36:37.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-08T05:36:37.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-08T05:36:37.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-08T05:36:37.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-08T05:36:37.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.677+0000 7f42caeb4640 1 Processor -- start 2023-10-08T05:36:37.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.678+0000 7f42caeb4640 1 -- start start 2023-10-08T05:36:37.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.679+0000 7f42caeb4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f42bc050380 0x7f42bc007580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.679+0000 7f42caeb4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f42bc005fe0 0x7f42bc054260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.679+0000 7f42caeb4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42bc0064e0 con 0x7f42bc0df500 2023-10-08T05:36:37.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.679+0000 7f42caeb4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc0df8e0 con 0x7f42bc050380 2023-10-08T05:36:37.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.679+0000 7f42caeb4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc0067c0 con 0x7f42bc005fe0 2023-10-08T05:36:37.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.680+0000 7f42c96b1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f42bc050380 0x7f42bc007580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.680+0000 7f42c96b1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f42bc050380 0x7f42bc007580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59826/0 (socket says 172.21.15.70:59826) 2023-10-08T05:36:37.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.680+0000 7f42c96b1640 1 -- 172.21.15.70:0/2585973826 learned_addr learned my addr 172.21.15.70:0/2585973826 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.681+0000 7f42c96b1640 1 -- 172.21.15.70:0/2585973826 >> v2:172.21.15.181:3300/0 conn(0x7f42bc005fe0 msgr2=0x7f42bc054260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T05:36:37.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.681+0000 7f42c96b1640 1 --2- 172.21.15.70:0/2585973826 >> v2:172.21.15.181:3300/0 conn(0x7f42bc005fe0 0x7f42bc054260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.681+0000 7f42c96b1640 1 -- 172.21.15.70:0/2585973826 >> v1:172.21.15.181:6789/0 conn(0x7f42bc0df500 legacy=0x7f42bc04fc70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T05:36:37.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.681+0000 7f42c96b1640 1 -- 172.21.15.70:0/2585973826 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc006aa0 con 0x7f42bc050380 2023-10-08T05:36:37.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.683+0000 7f42c96b1640 1 --2- 172.21.15.70:0/2585973826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42bc050380 0x7f42bc007580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00dae0 tx=0x7f42c002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b8eec211a3acb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.683+0000 7f42c8eb0640 1 -- 172.21.15.70:0/2585973826 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c002c7d0 con 0x7f42bc050380 2023-10-08T05:36:37.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.684+0000 7f42c8eb0640 1 -- 172.21.15.70:0/2585973826 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c002c930 con 0x7f42bc050380 2023-10-08T05:36:37.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.684+0000 7f42c8eb0640 1 -- 172.21.15.70:0/2585973826 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c002ccc0 con 0x7f42bc050380 2023-10-08T05:36:37.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.685+0000 7f42caeb4640 1 -- 172.21.15.70:0/2585973826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42bc050380 msgr2=0x7f42bc007580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.685+0000 7f42caeb4640 1 --2- 172.21.15.70:0/2585973826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42bc050380 0x7f42bc007580 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00dae0 tx=0x7f42c002bc30 comp rx=0 tx=0).stop 2023-10-08T05:36:37.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.683+0000 7fa539656640 1 Processor -- start 2023-10-08T05:36:37.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.685+0000 7f1f1759e640 1 Processor -- start 2023-10-08T05:36:37.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.684+0000 7fa539656640 1 -- start start 2023-10-08T05:36:37.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.685+0000 7f1f1759e640 1 -- start start 2023-10-08T05:36:37.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.686+0000 7fa539656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb730 0x7fa52c007050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.686+0000 7fa539656640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c050370 con 0x7fa52c0eb2f0 2023-10-08T05:36:37.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.686+0000 7fa539656640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c0064b0 con 0x7fa52c0e78e0 2023-10-08T05:36:37.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.686+0000 7fa539656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c04fdb0 con 0x7fa52c0eb730 2023-10-08T05:36:37.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.687+0000 7fa5327fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa52c0eb2f0 0x7fa52c006940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43640/0 (socket says 172.21.15.70:43640) 2023-10-08T05:36:37.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.687+0000 7fa5337fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb730 0x7fa52c007050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.687+0000 7fa532ffd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fa52c0e78e0 0x7fa52c04f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:52024/0 (socket says 172.21.15.70:52024) 2023-10-08T05:36:37.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.687+0000 7fa532ffd640 1 -- 172.21.15.70:0/794241849 learned_addr learned my addr 172.21.15.70:0/794241849 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:37.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.687+0000 7f1f1759e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f100e34a0 0x7f1f100543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.687+0000 7f1f1759e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f10057400 0x7f1f1003c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.687+0000 7f1f1759e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f10058dd0 con 0x7f1f100582c0 2023-10-08T05:36:37.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.687+0000 7f1f1759e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f10057b90 con 0x7f1f10057400 2023-10-08T05:36:37.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.687+0000 7f1f1759e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f10057e70 con 0x7f1f100e34a0 2023-10-08T05:36:37.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7fa5337fe640 1 -- 172.21.15.70:0/794241849 >> v1:172.21.15.181:6789/0 conn(0x7fa52c0e78e0 legacy=0x7fa52c04f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7fa5337fe640 1 -- 172.21.15.70:0/794241849 >> v1:172.21.15.70:6789/0 conn(0x7fa52c0eb2f0 legacy=0x7fa52c006940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7fa531ffb640 1 -- 172.21.15.70:0/794241849 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117165255 0 0) 0x7fa52c0064b0 con 0x7fa52c0e78e0 2023-10-08T05:36:37.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7fa5337fe640 1 -- 172.21.15.70:0/794241849 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c050090 con 0x7fa52c0eb730 2023-10-08T05:36:37.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7f1f15d9b640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f1f100582c0 0x7f1f10056cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:52036/0 (socket says 172.21.15.70:52036) 2023-10-08T05:36:37.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7f1f15d9b640 1 -- 172.21.15.70:0/1310169161 learned_addr learned my addr 172.21.15.70:0/1310169161 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:37.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7f42caeb4640 1 -- 172.21.15.70:0/2585973826 shutdown_connections 2023-10-08T05:36:37.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7f42caeb4640 1 --2- 172.21.15.70:0/2585973826 >> v2:172.21.15.181:3300/0 conn(0x7f42bc005fe0 0x7f42bc054260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7f42caeb4640 1 --2- 172.21.15.70:0/2585973826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42bc050380 0x7f42bc007580 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7f42caeb4640 1 -- 172.21.15.70:0/2585973826 >> 172.21.15.70:0/2585973826 conn(0x7f42bc054c10 msgr2=0x7f42bc055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7f1f16d9d640 1 --2- 172.21.15.70:0/1310169161 >> v2:172.21.15.70:3300/0 conn(0x7f1f10057400 0x7f1f1003c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.688+0000 7f1f1659c640 1 --2- 172.21.15.70:0/1310169161 >> v2:172.21.15.181:3300/0 conn(0x7f1f100e34a0 0x7f1f100543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.689+0000 7f1f1559a640 1 -- 172.21.15.70:0/1310169161 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459544312 0 0) 0x7f1f10058dd0 con 0x7f1f100582c0 2023-10-08T05:36:37.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.689+0000 7f1f1559a640 1 -- 172.21.15.70:0/1310169161 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f00003520 con 0x7f1f100582c0 2023-10-08T05:36:37.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.689+0000 7f42caeb4640 1 -- 172.21.15.70:0/2585973826 shutdown_connections 2023-10-08T05:36:37.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.689+0000 7f42caeb4640 1 -- 172.21.15.70:0/2585973826 wait complete. 2023-10-08T05:36:37.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.690+0000 7f1f16d9d640 1 -- 172.21.15.70:0/1310169161 >> v2:172.21.15.181:3300/0 conn(0x7f1f100e34a0 msgr2=0x7f1f100543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.690+0000 7f1f1559a640 1 -- 172.21.15.70:0/1310169161 <== mon.1 v1:172.21.15.181:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2322940057 0 0) 0x7f1f00003520 con 0x7f1f100582c0 2023-10-08T05:36:37.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.690+0000 7fa5337fe640 1 --2- 172.21.15.70:0/794241849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb730 0x7fa52c007050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa53402cb40 tx=0x7fa5340a56f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9d82a2f561652a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.690+0000 7fa531ffb640 1 -- 172.21.15.70:0/794241849 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5340aa7d0 con 0x7fa52c0eb730 2023-10-08T05:36:37.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.691+0000 7fa531ffb640 1 -- 172.21.15.70:0/794241849 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5340a8050 con 0x7fa52c0eb730 2023-10-08T05:36:37.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.691+0000 7fa531ffb640 1 -- 172.21.15.70:0/794241849 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5340aab30 con 0x7fa52c0eb730 2023-10-08T05:36:37.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.690+0000 7f1f16d9d640 1 --2- 172.21.15.70:0/1310169161 >> v2:172.21.15.181:3300/0 conn(0x7f1f100e34a0 0x7f1f100543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.690+0000 7f1f16d9d640 1 -- 172.21.15.70:0/1310169161 >> v1:172.21.15.181:6789/0 conn(0x7f1f100582c0 legacy=0x7f1f10056cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.690+0000 7f1f16d9d640 1 -- 172.21.15.70:0/1310169161 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f10053810 con 0x7f1f10057400 2023-10-08T05:36:37.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.691+0000 7f1f1659c640 1 --2- 172.21.15.70:0/1310169161 >> v2:172.21.15.181:3300/0 conn(0x7f1f100e34a0 0x7f1f100543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:36:37.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.692+0000 7f1f16d9d640 1 --2- 172.21.15.70:0/1310169161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f10057400 0x7f1f1003c2c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f080097a0 tx=0x7f1f0802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1392d4604b1f1b20 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.692+0000 7f42caeb4640 1 Processor -- start 2023-10-08T05:36:37.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.692+0000 7f42caeb4640 1 -- start start 2023-10-08T05:36:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.692+0000 7fa539656640 1 -- 172.21.15.70:0/794241849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb730 msgr2=0x7fa52c007050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.692+0000 7fa539656640 1 --2- 172.21.15.70:0/794241849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb730 0x7fa52c007050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa53402cb40 tx=0x7fa5340a56f0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.692+0000 7f1f1559a640 1 -- 172.21.15.70:0/1310169161 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1804e9b0 con 0x7f1f10057400 2023-10-08T05:36:37.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.693+0000 7f1f1559a640 1 -- 172.21.15.70:0/1310169161 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0802e050 con 0x7f1f10057400 2023-10-08T05:36:37.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.693+0000 7fa539656640 1 -- 172.21.15.70:0/794241849 shutdown_connections 2023-10-08T05:36:37.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.693+0000 7fa539656640 1 --2- 172.21.15.70:0/794241849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb730 0x7fa52c007050 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.693+0000 7fa539656640 1 -- 172.21.15.70:0/794241849 >> 172.21.15.70:0/794241849 conn(0x7fa52c054c40 msgr2=0x7fa52c055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.693+0000 7fa539656640 1 -- 172.21.15.70:0/794241849 shutdown_connections 2023-10-08T05:36:37.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.693+0000 7f5c2e0d4640 1 Processor -- start 2023-10-08T05:36:37.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.693+0000 7fa539656640 1 -- 172.21.15.70:0/794241849 wait complete. 2023-10-08T05:36:37.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f1f1759e640 1 -- 172.21.15.70:0/1310169161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f10057400 msgr2=0x7f1f1003c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7fe66d6f2640 1 Processor -- start 2023-10-08T05:36:37.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.693+0000 7f5c2e0d4640 1 -- start start 2023-10-08T05:36:37.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f1f1759e640 1 --2- 172.21.15.70:0/1310169161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f10057400 0x7f1f1003c2c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f080097a0 tx=0x7f1f0802b7b0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7f1f1759e640 1 -- 172.21.15.70:0/1310169161 shutdown_connections 2023-10-08T05:36:37.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f42caeb4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42bc005fe0 0x7f42bc1889d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f42caeb4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42bc050380 0x7f42bc04e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f42caeb4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42bc0df500 0x7f42bc049030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f42caeb4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc048c10 con 0x7f42bc050380 2023-10-08T05:36:37.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f42caeb4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc04f320 con 0x7f42bc0df500 2023-10-08T05:36:37.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f42caeb4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc04ed60 con 0x7f42bc005fe0 2023-10-08T05:36:37.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f42c9eb2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42bc005fe0 0x7f42bc1889d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f42c9eb2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42bc005fe0 0x7f42bc1889d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:60556/0 (socket says 172.21.15.70:60556) 2023-10-08T05:36:37.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7f42c9eb2640 1 -- 172.21.15.70:0/3540283483 learned_addr learned my addr 172.21.15.70:0/3540283483 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7f42ca6b3640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42bc0df500 0x7f42bc049030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7f42c96b1640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42bc050380 0x7f42bc04e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7fa539656640 1 Processor -- start 2023-10-08T05:36:37.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7f1f1759e640 1 --2- 172.21.15.70:0/1310169161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f10057400 0x7f1f1003c2c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7f1f1759e640 1 --2- 172.21.15.70:0/1310169161 >> v2:172.21.15.181:3300/0 conn(0x7f1f100e34a0 0x7f1f100543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7f1f1759e640 1 -- 172.21.15.70:0/1310169161 >> 172.21.15.70:0/1310169161 conn(0x7f1f10059120 msgr2=0x7f1f10059530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7f42c9eb2640 1 -- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42bc0df500 msgr2=0x7f42bc049030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7f42c9eb2640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42bc0df500 0x7f42bc049030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7f42c9eb2640 1 -- 172.21.15.70:0/3540283483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42bc050380 msgr2=0x7f42bc04e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.695+0000 7f42c9eb2640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42bc050380 0x7f42bc04e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7f42c9eb2640 1 -- 172.21.15.70:0/3540283483 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c00094d0 con 0x7f42bc005fe0 2023-10-08T05:36:37.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7f1f1759e640 1 -- 172.21.15.70:0/1310169161 shutdown_connections 2023-10-08T05:36:37.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7fa539656640 1 -- start start 2023-10-08T05:36:37.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.694+0000 7fe66d6f2640 1 -- start start 2023-10-08T05:36:37.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7f1f1759e640 1 -- 172.21.15.70:0/1310169161 wait complete. 2023-10-08T05:36:37.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7fe66d6f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6681591d0 0x7fe6680ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7fe66d6f2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66815d9f0 con 0x7fe6680cae90 2023-10-08T05:36:37.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7fe66d6f2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6680cb3c0 con 0x7fe6680ca220 2023-10-08T05:36:37.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7fe66d6f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680cb6a0 con 0x7fe6681591d0 2023-10-08T05:36:37.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7f5c2e0d4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5c28072060 0x7f5c280700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7f5c2e0d4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c280ce8e0 0x7f5c280ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7f5c2e0d4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c280ccf90 con 0x7f5c280cad90 2023-10-08T05:36:37.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7f5c2e0d4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c280cbe10 con 0x7f5c28072060 2023-10-08T05:36:37.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.696+0000 7f5c2e0d4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280cb930 con 0x7f5c280ce8e0 2023-10-08T05:36:37.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.697+0000 7f42c9eb2640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42bc005fe0 0x7f42bc1889d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f42b400b480 tx=0x7f42b400b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.697+0000 7fe6667fc640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fe6680ca220 0x7fe66815cbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:52038/0 (socket says 172.21.15.70:52038) 2023-10-08T05:36:37.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.697+0000 7fe6667fc640 1 -- 172.21.15.70:0/413439563 learned_addr learned my addr 172.21.15.70:0/413439563 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:37.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.697+0000 7f5c27fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5c280cad90 0x7f5c280cb220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43650/0 (socket says 172.21.15.70:43650) 2023-10-08T05:36:37.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.697+0000 7f5c27fff640 1 -- 172.21.15.70:0/4277855927 learned_addr learned my addr 172.21.15.70:0/4277855927 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:37.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.697+0000 7fdca2cbb640 1 Processor -- start 2023-10-08T05:36:37.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.697+0000 7fdca2cbb640 1 -- start start 2023-10-08T05:36:37.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.698+0000 7f42ba7fc640 1 -- 172.21.15.70:0/3540283483 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4010870 con 0x7f42bc005fe0 2023-10-08T05:36:37.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.698+0000 7f42ba7fc640 1 -- 172.21.15.70:0/3540283483 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b40109d0 con 0x7f42bc005fe0 2023-10-08T05:36:37.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.698+0000 7fa539656640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa52c0e78e0 0x7fa52c039030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.698+0000 7fa539656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb2f0 0x7fa52c0e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.698+0000 7fa539656640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0eb730 0x7fa52c0d6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7fa5327fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb2f0 0x7fa52c0e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7fa5327fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb2f0 0x7fa52c0e69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:60578/0 (socket says 172.21.15.70:60578) 2023-10-08T05:36:37.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7fa5327fc640 1 -- 172.21.15.70:0/3132789104 learned_addr learned my addr 172.21.15.70:0/3132789104 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7fa5337fe640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0eb730 0x7fa52c0d6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0e14b0 con 0x7fa52c0eb730 2023-10-08T05:36:37.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.698+0000 7f1f1759e640 1 Processor -- start 2023-10-08T05:36:37.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f1f1759e640 1 -- start start 2023-10-08T05:36:37.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.698+0000 7fe666ffd640 1 --2- 172.21.15.70:0/413439563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6681591d0 0x7fe6680ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.698+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081543820 0 0) 0x7f5c280ccf90 con 0x7f5c280cad90 2023-10-08T05:36:37.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.698+0000 7f5c277fe640 1 --2- 172.21.15.70:0/4277855927 >> v2:172.21.15.181:3300/0 conn(0x7f5c28072060 0x7f5c280700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.698+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c18003520 con 0x7f5c280cad90 2023-10-08T05:36:37.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c26ffd640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c280ce8e0 0x7f5c280ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3879449186 0 0) 0x7f5c18003520 con 0x7f5c280cad90 2023-10-08T05:36:37.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c280ce8e0 msgr2=0x7f5c280ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c280ce8e0 0x7f5c280ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 >> v2:172.21.15.181:3300/0 conn(0x7f5c28072060 msgr2=0x7f5c280700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 --2- 172.21.15.70:0/4277855927 >> v2:172.21.15.181:3300/0 conn(0x7f5c28072060 0x7f5c280700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c280cf280 con 0x7f5c280cad90 2023-10-08T05:36:37.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5c20001890 con 0x7f5c280cad90 2023-10-08T05:36:37.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 >> v1:172.21.15.70:6789/0 conn(0x7f5c280cad90 legacy=0x7f5c280cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c180039e0 0x7f5c18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c280ce8e0 0x7f5c180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c267fc640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c18004e20 0x7f5c1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.699+0000 7f5c27fff640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c18004e20 0x7f5c1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c17e580 con 0x7fa52c0e78e0 2023-10-08T05:36:37.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c17da00 con 0x7fa52c0eb2f0 2023-10-08T05:36:37.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280cbe10 con 0x7f5c18004e20 2023-10-08T05:36:37.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280cb930 con 0x7f5c180039e0 2023-10-08T05:36:37.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7fa532ffd640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa52c0e78e0 0x7fa52c039030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c18005650 con 0x7f5c280ce8e0 2023-10-08T05:36:37.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f5c26ffd640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c280ce8e0 0x7f5c180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f1f1759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f10057400 0x7f1f10052550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f1f1759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f100582c0 0x7f1f100d39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f1f1759e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f100e34a0 0x7f1f100507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f1f1759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100e38b0 con 0x7f1f100582c0 2023-10-08T05:36:37.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f1f1759e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f10052b30 con 0x7f1f100e34a0 2023-10-08T05:36:37.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f1f1759e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1003a670 con 0x7f1f10057400 2023-10-08T05:36:37.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f5c277fe640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c180039e0 0x7f5c18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.700+0000 7f1f15d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f100582c0 0x7f1f100d39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f1f15d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f100582c0 0x7f1f100d39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59856/0 (socket says 172.21.15.70:59856) 2023-10-08T05:36:37.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f1f15d9b640 1 -- 172.21.15.70:0/3255994779 learned_addr learned my addr 172.21.15.70:0/3255994779 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f42ba7fc640 1 -- 172.21.15.70:0/3540283483 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4010d80 con 0x7f42bc005fe0 2023-10-08T05:36:37.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f1f16d9d640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f100e34a0 0x7f1f100507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7fa5337fe640 1 -- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb2f0 msgr2=0x7fa52c0e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7fa5337fe640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb2f0 0x7fa52c0e69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7fa5337fe640 1 -- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa52c0e78e0 msgr2=0x7fa52c039030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7fa5337fe640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa52c0e78e0 0x7fa52c039030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7fa5337fe640 1 -- 172.21.15.70:0/3132789104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53404f050 con 0x7fa52c0eb730 2023-10-08T05:36:37.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7fa5327fc640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa52c0eb2f0 0x7fa52c0e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:36:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f1f15d9b640 1 -- 172.21.15.70:0/3255994779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f10057400 msgr2=0x7f1f10052550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f1f15d9b640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f10057400 0x7f1f10052550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f1f15d9b640 1 -- 172.21.15.70:0/3255994779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f100e34a0 msgr2=0x7f1f100507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f1f15d9b640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f100e34a0 0x7f1f100507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f1f15d9b640 1 -- 172.21.15.70:0/3255994779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c009400 con 0x7f1f100582c0 2023-10-08T05:36:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7fe666ffd640 1 -- 172.21.15.70:0/413439563 >> v1:172.21.15.181:6789/0 conn(0x7fe6680ca220 legacy=0x7fe66815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7fe666ffd640 1 -- 172.21.15.70:0/413439563 >> v1:172.21.15.70:6789/0 conn(0x7fe6680cae90 legacy=0x7fe66815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7fe666ffd640 1 -- 172.21.15.70:0/413439563 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6680cb980 con 0x7fe6681591d0 2023-10-08T05:36:37.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7fdca2cbb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9c15cd70 0x7fdc9c1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc04f040 con 0x7f42bc005fe0 2023-10-08T05:36:37.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.701+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc188f10 con 0x7f42bc005fe0 2023-10-08T05:36:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7fdca2cbb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc9c0cd4e0 0x7fdc9c0b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7fdca2cbb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc9c0b4c40 0x7fdc9c06b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7fa5337fe640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0eb730 0x7fa52c0d6bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa5340639b0 tx=0x7fa534066bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7fdca2cbb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b5090 con 0x7fdc9c0b4c40 2023-10-08T05:36:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7fdca2cbb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9c0b8c30 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7fdca2cbb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0cdd80 con 0x7fdc9c15cd70 2023-10-08T05:36:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42bc007060 con 0x7f42bc005fe0 2023-10-08T05:36:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7fdca1258640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc9c0b4c40 0x7fdc9c06b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7fdca1258640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc9c0b4c40 0x7fdc9c06b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59870/0 (socket says 172.21.15.70:59870) 2023-10-08T05:36:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7fdca1258640 1 -- 172.21.15.70:0/1133235180 learned_addr learned my addr 172.21.15.70:0/1133235180 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7fdca0a57640 1 --2- 172.21.15.70:0/1133235180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9c15cd70 0x7fdc9c1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.702+0000 7f1f15d9b640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f100582c0 0x7f1f100d39f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f1804f940 tx=0x7f1f180734e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fa51b7fe640 1 -- 172.21.15.70:0/3132789104 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5340ac9d0 con 0x7fa52c0eb730 2023-10-08T05:36:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fdc9bfff640 1 --2- 172.21.15.70:0/1133235180 >> v2:172.21.15.181:3300/0 conn(0x7fdc9c0cd4e0 0x7fdc9c0b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f5c27fff640 1 -- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c280ce8e0 msgr2=0x7f5c180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c17dce0 con 0x7fa52c0eb730 2023-10-08T05:36:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fa51b7fe640 1 -- 172.21.15.70:0/3132789104 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5340acb30 con 0x7fa52c0eb730 2023-10-08T05:36:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f5c27fff640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c280ce8e0 0x7f5c180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fa51b7fe640 1 -- 172.21.15.70:0/3132789104 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5340acea0 con 0x7fa52c0eb730 2023-10-08T05:36:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f5c27fff640 1 -- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c180039e0 msgr2=0x7f5c18003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f5c27fff640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c180039e0 0x7f5c18003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c17e2a0 con 0x7fa52c0eb730 2023-10-08T05:36:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f5c27fff640 1 -- 172.21.15.70:0/4277855927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c18003520 con 0x7f5c18004e20 2023-10-08T05:36:37.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fdca1258640 1 -- 172.21.15.70:0/1133235180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9c15cd70 msgr2=0x7fdc9c1612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fdca1258640 1 --2- 172.21.15.70:0/1133235180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9c15cd70 0x7fdc9c1612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f5c277fe640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c180039e0 0x7f5c18003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f1f06ffd640 1 -- 172.21.15.70:0/3255994779 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f180723d0 con 0x7f1f100582c0 2023-10-08T05:36:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f1f06ffd640 1 -- 172.21.15.70:0/3255994779 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1806b110 con 0x7f1f100582c0 2023-10-08T05:36:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f1f06ffd640 1 -- 172.21.15.70:0/3255994779 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1806b2c0 con 0x7f1f100582c0 2023-10-08T05:36:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f1f1759e640 1 -- 172.21.15.70:0/3255994779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f08009400 con 0x7f1f100582c0 2023-10-08T05:36:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7f1f1759e640 1 -- 172.21.15.70:0/3255994779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1003abf0 con 0x7f1f100582c0 2023-10-08T05:36:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.704+0000 7f65aedb7640 1 Processor -- start 2023-10-08T05:36:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.704+0000 7f5c26ffd640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c280ce8e0 0x7f5c180047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:36:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fdca1258640 1 -- 172.21.15.70:0/1133235180 >> v2:172.21.15.181:3300/0 conn(0x7fdc9c0cd4e0 msgr2=0x7fdc9c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.704+0000 7f65aedb7640 1 -- start start 2023-10-08T05:36:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fdca1258640 1 --2- 172.21.15.70:0/1133235180 >> v2:172.21.15.181:3300/0 conn(0x7fdc9c0cd4e0 0x7fdc9c0b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.703+0000 7fdca1258640 1 -- 172.21.15.70:0/1133235180 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c0ce060 con 0x7fdc9c0b4c40 2023-10-08T05:36:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.705+0000 7fe666ffd640 1 --2- 172.21.15.70:0/413439563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6681591d0 0x7fe6680ce6d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0097d0 tx=0x7fe65c02dac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbf9b546bdf4db98 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.705+0000 7f5c27fff640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c18004e20 0x7f5c1800b980 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000c540 tx=0x7f5c180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162e3f2213acfe3b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.706+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa52c0ebbc0 con 0x7fa52c0eb730 2023-10-08T05:36:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.706+0000 7f1f1759e640 1 -- 172.21.15.70:0/3255994779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1efc000fc0 con 0x7f1f100582c0 2023-10-08T05:36:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.705+0000 7fe665ffb640 1 -- 172.21.15.70:0/413439563 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c02f880 con 0x7fe6681591d0 2023-10-08T05:36:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.706+0000 7fe665ffb640 1 -- 172.21.15.70:0/413439563 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65c02f9e0 con 0x7fe6681591d0 2023-10-08T05:36:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.706+0000 7fe665ffb640 1 -- 172.21.15.70:0/413439563 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65c02fd70 con 0x7fe6681591d0 2023-10-08T05:36:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.706+0000 7fdca1258640 1 --2- 172.21.15.70:0/1133235180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc9c0b4c40 0x7fdc9c06b290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c08aa60 tx=0x7fdc9402bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743611fa42d6215c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.706+0000 7fdc9b7fe640 1 -- 172.21.15.70:0/1133235180 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9402caa0 con 0x7fdc9c0b4c40 2023-10-08T05:36:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.706+0000 7fdc9b7fe640 1 -- 172.21.15.70:0/1133235180 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9402cc00 con 0x7fdc9c0b4c40 2023-10-08T05:36:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.706+0000 7fdc9b7fe640 1 -- 172.21.15.70:0/1133235180 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc940343f0 con 0x7fdc9c0b4c40 2023-10-08T05:36:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.706+0000 7f65aedb7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65a80b3ec0 0x7f65a80ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7feda31f0640 1 Processor -- start 2023-10-08T05:36:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.706+0000 7f65aedb7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a80c8e70 0x7f65a80cae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7f65aedb7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a806d490 con 0x7f65a80cb3d0 2023-10-08T05:36:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7f65aedb7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a80b42a0 con 0x7f65a80b3ec0 2023-10-08T05:36:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7f65aedb7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a806c880 con 0x7f65a80c8e70 2023-10-08T05:36:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7f42ba7fc640 1 -- 172.21.15.70:0/3540283483 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f42b4019450 con 0x7f42bc005fe0 2023-10-08T05:36:37.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7feda31f0640 1 -- start start 2023-10-08T05:36:37.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7fe66d6f2640 1 -- 172.21.15.70:0/413439563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6681591d0 msgr2=0x7fe6680ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7fe66d6f2640 1 --2- 172.21.15.70:0/413439563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6681591d0 0x7fe6680ce6d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0097d0 tx=0x7fe65c02dac0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7f65acb53640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65a80b3ec0 0x7f65a80ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7fe66d6f2640 1 -- 172.21.15.70:0/413439563 shutdown_connections 2023-10-08T05:36:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7fe66d6f2640 1 --2- 172.21.15.70:0/413439563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6681591d0 0x7fe6680ce6d0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0097d0 tx=0x7fe65c02dac0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7fe66d6f2640 1 -- 172.21.15.70:0/413439563 >> 172.21.15.70:0/413439563 conn(0x7fe6680660e0 msgr2=0x7fe668069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7fdca2cbb640 1 -- 172.21.15.70:0/1133235180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc9c0b4c40 msgr2=0x7fdc9c06b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7fdca2cbb640 1 --2- 172.21.15.70:0/1133235180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc9c0b4c40 0x7fdc9c06b290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c08aa60 tx=0x7fdc9402bf00 comp rx=0 tx=0).stop 2023-10-08T05:36:37.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7f65ad354640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f65a80cb3d0 0x7f65a80702b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43674/0 (socket says 172.21.15.70:43674) 2023-10-08T05:36:37.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.707+0000 7f65ad354640 1 -- 172.21.15.70:0/2881454355 learned_addr learned my addr 172.21.15.70:0/2881454355 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:37.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20035470 con 0x7f5c18004e20 2023-10-08T05:36:37.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c200355f0 con 0x7f5c18004e20 2023-10-08T05:36:37.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7f5c267fc640 1 -- 172.21.15.70:0/4277855927 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200359a0 con 0x7f5c18004e20 2023-10-08T05:36:37.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7fe66d6f2640 1 -- 172.21.15.70:0/413439563 shutdown_connections 2023-10-08T05:36:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7fdca2cbb640 1 -- 172.21.15.70:0/1133235180 shutdown_connections 2023-10-08T05:36:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7f65acb53640 1 -- 172.21.15.70:0/2881454355 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a80c8e70 msgr2=0x7f65a80cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7f65acb53640 1 --2- 172.21.15.70:0/2881454355 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a80c8e70 0x7f65a80cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7f65a77fe640 1 -- 172.21.15.70:0/2881454355 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76905265 0 0) 0x7f65a806d490 con 0x7f65a80cb3d0 2023-10-08T05:36:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7f65acb53640 1 -- 172.21.15.70:0/2881454355 >> v1:172.21.15.70:6789/0 conn(0x7f65a80cb3d0 legacy=0x7f65a80702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7f42ba7fc640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f42a806af70 0x7f42a806d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7fdca2cbb640 1 --2- 172.21.15.70:0/1133235180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc9c0b4c40 0x7fdc9c06b290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.709+0000 7f1f06ffd640 1 -- 172.21.15.70:0/3255994779 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1f18069d50 con 0x7f1f100582c0 2023-10-08T05:36:37.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7fdca2cbb640 1 --2- 172.21.15.70:0/1133235180 >> v2:172.21.15.181:3300/0 conn(0x7fdc9c0cd4e0 0x7fdc9c0b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.709+0000 7f65acb53640 1 -- 172.21.15.70:0/2881454355 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a806cb60 con 0x7f65a80b3ec0 2023-10-08T05:36:37.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.709+0000 7f42ba7fc640 1 -- 172.21.15.70:0/3540283483 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f42b4089df0 con 0x7f42bc005fe0 2023-10-08T05:36:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7fdca2cbb640 1 --2- 172.21.15.70:0/1133235180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9c15cd70 0x7fdc9c1612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7fdca2cbb640 1 -- 172.21.15.70:0/1133235180 >> 172.21.15.70:0/1133235180 conn(0x7fdc9c0cf700 msgr2=0x7fdc9c0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.709+0000 7f42c96b1640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f42a806af70 0x7f42a806d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.709+0000 7feda31f0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed9c0b3ec0 0x7fed9c0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.709+0000 7fdca2cbb640 1 -- 172.21.15.70:0/1133235180 shutdown_connections 2023-10-08T05:36:37.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7feda31f0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed9c0cb5a0 0x7fed9c0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.709+0000 7fdca2cbb640 1 -- 172.21.15.70:0/1133235180 wait complete. 2023-10-08T05:36:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7f42ba7fc640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] conn(0x7f42a8074440 0x7f42a8076880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7feda0f8c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed9c0b3ec0 0x7fed9c0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7f42ba7fc640 1 -- 172.21.15.70:0/3540283483 --> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42a8076f50 con 0x7f42a8074440 2023-10-08T05:36:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7feda0f8c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed9c0b3ec0 0x7fed9c0ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39662/0 (socket says 172.21.15.70:39662) 2023-10-08T05:36:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7f42c96b1640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f42a806af70 0x7f42a806d430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42c0036000 tx=0x7f42c0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7f42ba7fc640 1 -- 172.21.15.70:0/3540283483 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42b408a1a0 con 0x7f42bc005fe0 2023-10-08T05:36:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7feda178d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed9c0cb5a0 0x7fed9c0b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7f65acb53640 1 --2- 172.21.15.70:0/2881454355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65a80b3ec0 0x7f65a80ca0b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f65980071d0 tx=0x7f659802c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a1724dd62958b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7feda31f0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9c06d490 con 0x7fed9c0c8e70 2023-10-08T05:36:37.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7feda31f0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9c0b42a0 con 0x7fed9c0cb5a0 2023-10-08T05:36:37.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7feda31f0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9c06c880 con 0x7fed9c0b3ec0 2023-10-08T05:36:37.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7feda178d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed9c0cb5a0 0x7fed9c0b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59892/0 (socket says 172.21.15.70:59892) 2023-10-08T05:36:37.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7feda178d640 1 -- 172.21.15.70:0/1869495245 learned_addr learned my addr 172.21.15.70:0/1869495245 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.708+0000 7fe66d6f2640 1 -- 172.21.15.70:0/413439563 wait complete. 2023-10-08T05:36:37.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7fe66d6f2640 1 Processor -- start 2023-10-08T05:36:37.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7fe66d6f2640 1 -- start start 2023-10-08T05:36:37.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7feda178d640 1 -- 172.21.15.70:0/1869495245 >> v2:172.21.15.181:3300/0 conn(0x7fed9c0b3ec0 msgr2=0x7fed9c0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7feda178d640 1 --2- 172.21.15.70:0/1869495245 >> v2:172.21.15.181:3300/0 conn(0x7fed9c0b3ec0 0x7fed9c0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7feda178d640 1 -- 172.21.15.70:0/1869495245 >> v1:172.21.15.70:6789/0 conn(0x7fed9c0c8e70 legacy=0x7fed9c0cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:37.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7feda178d640 1 -- 172.21.15.70:0/1869495245 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c06cb60 con 0x7fed9c0cb5a0 2023-10-08T05:36:37.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/4277855927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c18004e20 msgr2=0x7f5c1800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7f5c2e0d4640 1 --2- 172.21.15.70:0/4277855927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c18004e20 0x7f5c1800b980 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000c540 tx=0x7f5c180057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7f5c27fff640 1 -- 172.21.15.70:0/4277855927 reap_dead start 2023-10-08T05:36:37.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.712+0000 7f65a77fe640 1 -- 172.21.15.70:0/2881454355 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598008df0 con 0x7f65a80b3ec0 2023-10-08T05:36:37.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/4277855927 shutdown_connections 2023-10-08T05:36:37.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/4277855927 >> 172.21.15.70:0/4277855927 conn(0x7f5c2806a170 msgr2=0x7f5c2806a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.710+0000 7fa51b7fe640 1 -- 172.21.15.70:0/3132789104 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa5340b8450 con 0x7fa52c0eb730 2023-10-08T05:36:37.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7fa51b7fe640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa52006af40 0x7fa52006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7fa51b7fe640 1 -- 172.21.15.70:0/3132789104 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa5341259d0 con 0x7fa52c0eb730 2023-10-08T05:36:37.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7fa51b7fe640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] conn(0x7fa520074580 0x7fa5200769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7fa51b7fe640 1 -- 172.21.15.70:0/3132789104 --> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa520077090 con 0x7fa520074580 2023-10-08T05:36:37.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7fa51b7fe640 1 -- 172.21.15.70:0/3132789104 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa534125d80 con 0x7fa52c0eb730 2023-10-08T05:36:37.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.712+0000 7f65a77fe640 1 -- 172.21.15.70:0/2881454355 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659800a0d0 con 0x7f65a80b3ec0 2023-10-08T05:36:37.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.712+0000 7f65a77fe640 1 -- 172.21.15.70:0/2881454355 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659800a460 con 0x7f65a80b3ec0 2023-10-08T05:36:37.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.711+0000 7f42ca6b3640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] conn(0x7f42a8074440 0x7f42a8076880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.712+0000 7fa532ffd640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa52006af40 0x7fa52006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.712+0000 7fa5327fc640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] conn(0x7fa520074580 0x7fa5200769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7fa5327fc640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] conn(0x7fa520074580 0x7fa5200769c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f65aedb7640 1 -- 172.21.15.70:0/2881454355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65a80b3ec0 msgr2=0x7f65a80ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f65aedb7640 1 --2- 172.21.15.70:0/2881454355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65a80b3ec0 0x7f65a80ca0b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f65980071d0 tx=0x7f659802c9d0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f1f06ffd640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ef806af40 0x7f1ef806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7fdca2cbb640 1 Processor -- start 2023-10-08T05:36:37.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f65aedb7640 1 -- 172.21.15.70:0/2881454355 shutdown_connections 2023-10-08T05:36:37.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f65aedb7640 1 --2- 172.21.15.70:0/2881454355 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a80c8e70 0x7f65a80cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f65aedb7640 1 --2- 172.21.15.70:0/2881454355 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65a80b3ec0 0x7f65a80ca0b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f65aedb7640 1 -- 172.21.15.70:0/2881454355 >> 172.21.15.70:0/2881454355 conn(0x7f65a806a5d0 msgr2=0x7f65a80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7fe66d6f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6680ca220 0x7fe6680c1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f1f06ffd640 1 -- 172.21.15.70:0/3255994779 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1f18102f50 con 0x7f1f100582c0 2023-10-08T05:36:37.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f1f06ffd640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] conn(0x7f1ef8074580 0x7f1ef80769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f1f06ffd640 1 -- 172.21.15.70:0/3255994779 --> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ef8077090 con 0x7f1ef8074580 2023-10-08T05:36:37.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f1f06ffd640 1 -- 172.21.15.70:0/3255994779 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f18108520 con 0x7f1f100582c0 2023-10-08T05:36:37.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f42ca6b3640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] conn(0x7f42a8074440 0x7f42a8076880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.713+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/4277855927 shutdown_connections 2023-10-08T05:36:37.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7fe66d6f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6680cae90 0x7fe668159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/4277855927 wait complete. 2023-10-08T05:36:37.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7fdca2cbb640 1 -- start start 2023-10-08T05:36:37.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7f1f16d9d640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] conn(0x7f1ef8074580 0x7f1ef80769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7f1f1659c640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ef806af40 0x7f1ef806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7feda178d640 1 --2- 172.21.15.70:0/1869495245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed9c0cb5a0 0x7fed9c0b2190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0cc120 tx=0x7fed9400ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4e4a66deb6f4b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7fa51b7fe640 1 -- 172.21.15.70:0/3132789104 <== osd.1 v2:172.21.15.70:6800/435902020 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa520077090 con 0x7fa520074580 2023-10-08T05:36:37.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7f65aedb7640 1 -- 172.21.15.70:0/2881454355 shutdown_connections 2023-10-08T05:36:37.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7fe6667fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6680cae90 0x7fe668159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7fe6667fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6680cae90 0x7fe668159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59904/0 (socket says 172.21.15.70:59904) 2023-10-08T05:36:37.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7fe6667fc640 1 -- 172.21.15.70:0/2976239571 learned_addr learned my addr 172.21.15.70:0/2976239571 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.714+0000 7f65aedb7640 1 -- 172.21.15.70:0/2881454355 wait complete. 2023-10-08T05:36:37.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fe66d6f2640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6681f3930 0x7fe66815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fa532ffd640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa52006af40 0x7fa52006d400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa524000f60 tx=0x7fa524007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7f42ba7fc640 1 -- 172.21.15.70:0/3540283483 <== osd.0 v2:172.21.15.70:6816/1284805295 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f42a8076f50 con 0x7f42a8074440 2023-10-08T05:36:37.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680c2960 con 0x7fe6680cae90 2023-10-08T05:36:37.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66815aa10 con 0x7fe6680ca220 2023-10-08T05:36:37.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680c2310 con 0x7fe6681f3930 2023-10-08T05:36:37.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fdca2cbb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc9c0b4c40 0x7fdc9c0ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fed9b7fe640 1 -- 172.21.15.70:0/1869495245 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9402ce50 con 0x7fed9c0cb5a0 2023-10-08T05:36:37.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fdca2cbb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc9c0cd4e0 0x7fdc9c1f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fed9b7fe640 1 -- 172.21.15.70:0/1869495245 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed94030070 con 0x7fed9c0cb5a0 2023-10-08T05:36:37.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fe6677fe640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6681f3930 0x7fe66815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fe6667fc640 1 -- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6681f3930 msgr2=0x7fe66815b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fe6667fc640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6681f3930 0x7fe66815b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fe6667fc640 1 -- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6680ca220 msgr2=0x7fe6680c1300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T05:36:37.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fe6667fc640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6680ca220 0x7fe6680c1300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fe6667fc640 1 -- 172.21.15.70:0/2976239571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c009400 con 0x7fe6680cae90 2023-10-08T05:36:37.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.715+0000 7fdca2cbb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9c15cd70 0x7fdc9c1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fdca2cbb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c15a7a0 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fdca2cbb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c15afb0 con 0x7fdc9c0b4c40 2023-10-08T05:36:37.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fdca2cbb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c0cad40 con 0x7fdc9c15cd70 2023-10-08T05:36:37.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fe6667fc640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6680cae90 0x7fe668159f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe65800c500 tx=0x7fe65800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fed9b7fe640 1 -- 172.21.15.70:0/1869495245 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94036620 con 0x7fed9c0cb5a0 2023-10-08T05:36:37.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fdc9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc9c0cd4e0 0x7fdc9c1f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fe64f7fe640 1 -- 172.21.15.70:0/2976239571 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658005ce0 con 0x7fe6680cae90 2023-10-08T05:36:37.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.716+0000 7fdc9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc9c0cd4e0 0x7fdc9c1f1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59918/0 (socket says 172.21.15.70:59918) 2023-10-08T05:36:37.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.717+0000 7fdc9bfff640 1 -- 172.21.15.70:0/3711221041 learned_addr learned my addr 172.21.15.70:0/3711221041 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.717+0000 7fe64f7fe640 1 -- 172.21.15.70:0/2976239571 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe658005e40 con 0x7fe6680cae90 2023-10-08T05:36:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.717+0000 7fe64f7fe640 1 -- 172.21.15.70:0/2976239571 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6580072f0 con 0x7fe6680cae90 2023-10-08T05:36:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.717+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6680c25f0 con 0x7fe6680cae90 2023-10-08T05:36:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.717+0000 7f5c2e0d4640 1 Processor -- start 2023-10-08T05:36:37.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.718+0000 7f5c2e0d4640 1 -- start start 2023-10-08T05:36:37.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.718+0000 7f65aedb7640 1 Processor -- start 2023-10-08T05:36:37.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.718+0000 7feda31f0640 1 -- 172.21.15.70:0/1869495245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed9c0cb5a0 msgr2=0x7fed9c0b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.718+0000 7f65aedb7640 1 -- start start 2023-10-08T05:36:37.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.718+0000 7feda31f0640 1 --2- 172.21.15.70:0/1869495245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed9c0cb5a0 0x7fed9c0b2190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0cc120 tx=0x7fed9400ace0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.718+0000 7f1f16d9d640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] conn(0x7f1ef8074580 0x7f1ef80769c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.719+0000 7feda31f0640 1 -- 172.21.15.70:0/1869495245 shutdown_connections 2023-10-08T05:36:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.719+0000 7feda31f0640 1 --2- 172.21.15.70:0/1869495245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed9c0cb5a0 0x7fed9c0b2190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.719+0000 7feda31f0640 1 --2- 172.21.15.70:0/1869495245 >> v2:172.21.15.181:3300/0 conn(0x7fed9c0b3ec0 0x7fed9c0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.719+0000 7feda31f0640 1 -- 172.21.15.70:0/1869495245 >> 172.21.15.70:0/1869495245 conn(0x7fed9c06a5d0 msgr2=0x7fed9c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.720+0000 7f5c2e0d4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c18004e20 0x7f5c2806c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.719+0000 7fdc9bfff640 1 -- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9c15cd70 msgr2=0x7fdc9c1f5b40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T05:36:37.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.719+0000 7fdc9bfff640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc9c15cd70 0x7fdc9c1f5b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.719+0000 7fdc9bfff640 1 -- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc9c0b4c40 msgr2=0x7fdc9c0ca760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:36:37.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.719+0000 7fdc9bfff640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc9c0b4c40 0x7fdc9c0ca760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.719+0000 7fdc9bfff640 1 -- 172.21.15.70:0/3711221041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc94009b30 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.720+0000 7fdc9bfff640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc9c0cd4e0 0x7fdc9c1f1e90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00e870 tx=0x7fdc8c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.720+0000 7f1f1659c640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ef806af40 0x7f1ef806d400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c006e80 tx=0x7f1f0c006dd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.720+0000 7f5c2e0d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c2815ecf0 0x7f5c280c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.720+0000 7f65aedb7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a80b3ec0 0x7f65a8155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.720+0000 7f5c2e0d4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c280b3290 0x7f5c280b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.720+0000 7f5c2e0d4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280b3c50 con 0x7f5c2815ecf0 2023-10-08T05:36:37.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.720+0000 7f65aedb7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a80c8e70 0x7f65a8153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.718+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66815acf0 con 0x7fe6680cae90 2023-10-08T05:36:37.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.720+0000 7f5c277fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c18004e20 0x7f5c2806c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7f5c277fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c18004e20 0x7f5c2806c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:60634/0 (socket says 172.21.15.70:60634) 2023-10-08T05:36:37.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.720+0000 7fe64f7fe640 1 -- 172.21.15.70:0/2976239571 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe658007510 con 0x7fe6680cae90 2023-10-08T05:36:37.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7f65aedb7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65a80cb3d0 0x7f65a81577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7fdc997fa640 1 -- 172.21.15.70:0/3711221041 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00ee60 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7fdc997fa640 1 -- 172.21.15.70:0/3711221041 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c00c940 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7fdc997fa640 1 -- 172.21.15.70:0/3711221041 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00ccf0 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7f65acb53640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a80b3ec0 0x7f65a8155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7f65acb53640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a80b3ec0 0x7f65a8155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:60646/0 (socket says 172.21.15.70:60646) 2023-10-08T05:36:37.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7feda31f0640 1 -- 172.21.15.70:0/1869495245 shutdown_connections 2023-10-08T05:36:37.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7fdca2cbb640 1 -- 172.21.15.70:0/3711221041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c15a9f0 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7f5c26ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c2815ecf0 0x7f5c280c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7f5c277fe640 1 -- 172.21.15.70:0/2104372849 learned_addr learned my addr 172.21.15.70:0/2104372849 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7f1f06ffd640 1 -- 172.21.15.70:0/3255994779 <== osd.2 v2:172.21.15.70:6817/3799730343 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1ef8077090 con 0x7f1ef8074580 2023-10-08T05:36:37.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7f5c27fff640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c280b3290 0x7f5c280b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c280c8410 con 0x7f5c280b3290 2023-10-08T05:36:37.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.721+0000 7feda31f0640 1 -- 172.21.15.70:0/1869495245 wait complete. 2023-10-08T05:36:37.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2806c780 con 0x7f5c18004e20 2023-10-08T05:36:37.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7f65ad354640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65a80cb3d0 0x7f65a81577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe6680cbf40 con 0x7fe6680cae90 2023-10-08T05:36:37.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7f65ad354640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65a80cb3d0 0x7f65a81577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59938/0 (socket says 172.21.15.70:59938) 2023-10-08T05:36:37.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7f65a7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a80c8e70 0x7f65a8153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7f65a7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a80c8e70 0x7f65a8153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39702/0 (socket says 172.21.15.70:39702) 2023-10-08T05:36:37.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7f65acb53640 1 -- 172.21.15.70:0/3793368201 learned_addr learned my addr 172.21.15.70:0/3793368201 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a81563d0 con 0x7f65a80cb3d0 2023-10-08T05:36:37.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a8153880 con 0x7f65a80c8e70 2023-10-08T05:36:37.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a80c6bc0 con 0x7f65a80b3ec0 2023-10-08T05:36:37.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7fe64f7fe640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe65406af40 0x7fe65406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.722+0000 7fe64f7fe640 1 -- 172.21.15.70:0/2976239571 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe658088d30 con 0x7fe6680cae90 2023-10-08T05:36:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7fe64f7fe640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] conn(0x7fe654074500 0x7fe654076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7fe64f7fe640 1 -- 172.21.15.70:0/2976239571 --> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe654077010 con 0x7fe654074500 2023-10-08T05:36:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7fe64f7fe640 1 -- 172.21.15.70:0/2976239571 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe658053c10 con 0x7fe6680cae90 2023-10-08T05:36:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7fe6677fe640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] conn(0x7fe654074500 0x7fe654076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7f65ad354640 1 -- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a80b3ec0 msgr2=0x7f65a8155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7f65ad354640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65a80b3ec0 0x7f65a8155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7f65ad354640 1 -- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a80c8e70 msgr2=0x7f65a8153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7f65ad354640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a80c8e70 0x7f65a8153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7f65ad354640 1 -- 172.21.15.70:0/3793368201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659800ae30 con 0x7f65a80cb3d0 2023-10-08T05:36:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.724+0000 7fe666ffd640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe65406af40 0x7fe65406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.724+0000 7f65ad354640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65a80cb3d0 0x7f65a81577d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f65a000ced0 tx=0x7f65a0001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.724+0000 7f65a7fff640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65a80c8e70 0x7f65a8153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:37.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.725+0000 7fe6677fe640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] conn(0x7fe654074500 0x7fe654076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.725+0000 7fe666ffd640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe65406af40 0x7fe65406d400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe65c038000 tx=0x7fe65c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.724+0000 7f65a57fa640 1 -- 172.21.15.70:0/3793368201 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0001fa0 con 0x7f65a80cb3d0 2023-10-08T05:36:37.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.725+0000 7f65a57fa640 1 -- 172.21.15.70:0/3793368201 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a000ecd0 con 0x7f65a80cb3d0 2023-10-08T05:36:37.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.725+0000 7f65a57fa640 1 -- 172.21.15.70:0/3793368201 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0002100 con 0x7f65a80cb3d0 2023-10-08T05:36:37.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.726+0000 7feda31f0640 1 Processor -- start 2023-10-08T05:36:37.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7f5c27fff640 1 -- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c18004e20 msgr2=0x7f5c2806c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7f5c27fff640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c18004e20 0x7f5c2806c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7f5c27fff640 1 -- 172.21.15.70:0/2104372849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c2815ecf0 msgr2=0x7f5c280c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7f5c27fff640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c2815ecf0 0x7f5c280c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7f5c27fff640 1 -- 172.21.15.70:0/2104372849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2806ca60 con 0x7f5c280b3290 2023-10-08T05:36:37.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.724+0000 7f5c27fff640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c280b3290 0x7f5c280b3710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5c20031000 tx=0x7f5c280ccf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.726+0000 7f5c26ffd640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c2815ecf0 0x7f5c280c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:37.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.723+0000 7fdca2cbb640 1 -- 172.21.15.70:0/3711221041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c1599a0 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.726+0000 7feda31f0640 1 -- start start 2023-10-08T05:36:37.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.726+0000 7fe64f7fe640 1 -- 172.21.15.70:0/2976239571 <== osd.3 v2:172.21.15.70:6808/471550804 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe654077010 con 0x7fe654074500 2023-10-08T05:36:37.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.725+0000 7fdc92ffd640 1 -- 172.21.15.70:0/3711221041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdc9c0d0e50 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.727+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a8154000 con 0x7f65a80cb3d0 2023-10-08T05:36:37.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.727+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a81545c0 con 0x7f65a80cb3d0 2023-10-08T05:36:37.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.728+0000 7feda31f0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed9c0b3ec0 0x7fed9c0c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.728+0000 7f5c13fff640 1 -- 172.21.15.70:0/2104372849 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20035470 con 0x7f5c280b3290 2023-10-08T05:36:37.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.728+0000 7f5c13fff640 1 -- 172.21.15.70:0/2104372849 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c20040420 con 0x7f5c280b3290 2023-10-08T05:36:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.728+0000 7f5c13fff640 1 -- 172.21.15.70:0/2104372849 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20040790 con 0x7f5c280b3290 2023-10-08T05:36:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2806cd40 con 0x7f5c280b3290 2023-10-08T05:36:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.729+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65a806d120 con 0x7f65a80cb3d0 2023-10-08T05:36:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7f65a57fa640 1 -- 172.21.15.70:0/3793368201 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f65a0013770 con 0x7f65a80cb3d0 2023-10-08T05:36:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7feda31f0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed9c0c8e70 0x7fed9c147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7feda31f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed9c0c2250 0x7fed9c0c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7feda31f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c157530 con 0x7fed9c0c2250 2023-10-08T05:36:37.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7feda31f0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c1485c0 con 0x7fed9c0b3ec0 2023-10-08T05:36:37.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7feda31f0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c0c5130 con 0x7fed9c0c8e70 2023-10-08T05:36:37.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.728+0000 7fdc997fa640 1 -- 172.21.15.70:0/3711221041 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdc8c013470 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7fdc997fa640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fdc7c06af40 0x7fdc7c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7fdc997fa640 1 -- 172.21.15.70:0/3711221041 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdc8c089c10 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7fdc997fa640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] conn(0x7fdc7c074580 0x7fdc7c0769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7fdc997fa640 1 -- 172.21.15.70:0/3711221041 --> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc7c077090 con 0x7fdc7c074580 2023-10-08T05:36:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.730+0000 7fdc997fa640 1 -- 172.21.15.70:0/3711221041 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc8c089fc0 con 0x7fdc9c0cd4e0 2023-10-08T05:36:37.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.731+0000 7feda178d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed9c0c2250 0x7fed9c0c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.731+0000 7feda178d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed9c0c2250 0x7fed9c0c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59950/0 (socket says 172.21.15.70:59950) 2023-10-08T05:36:37.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.731+0000 7feda178d640 1 -- 172.21.15.70:0/3449347825 learned_addr learned my addr 172.21.15.70:0/3449347825 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:37.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.731+0000 7fdca0a57640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fdc7c06af40 0x7fdc7c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.731+0000 7f65a57fa640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f659406af40 0x7f659406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7f65a57fa640 1 -- 172.21.15.70:0/3793368201 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f65a008db40 con 0x7f65a80cb3d0 2023-10-08T05:36:37.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7feda178d640 1 -- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed9c0c8e70 msgr2=0x7fed9c147b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T05:36:37.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7feda178d640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed9c0c8e70 0x7fed9c147b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7feda178d640 1 -- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed9c0b3ec0 msgr2=0x7fed9c0c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7feda178d640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed9c0b3ec0 0x7fed9c0c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7feda178d640 1 -- 172.21.15.70:0/3449347825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed94007790 con 0x7fed9c0c2250 2023-10-08T05:36:37.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7f65a57fa640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] conn(0x7f6594074580 0x7f65940769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7f65a57fa640 1 -- 172.21.15.70:0/3793368201 --> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6594077090 con 0x7f6594074580 2023-10-08T05:36:37.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7f65a57fa640 1 -- 172.21.15.70:0/3793368201 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65a00922a0 con 0x7f65a80cb3d0 2023-10-08T05:36:37.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7feda178d640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed9c0c2250 0x7fed9c0c9270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fed94007210 tx=0x7fed94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.733+0000 7fdca0a57640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fdc7c06af40 0x7fdc7c06d400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009870 tx=0x7fdc84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.733+0000 7fed997fa640 1 -- 172.21.15.70:0/3449347825 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94008780 con 0x7fed9c0c2250 2023-10-08T05:36:37.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.733+0000 7fed997fa640 1 -- 172.21.15.70:0/3449347825 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed940088e0 con 0x7fed9c0c2250 2023-10-08T05:36:37.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.733+0000 7fed997fa640 1 -- 172.21.15.70:0/3449347825 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed940416b0 con 0x7fed9c0c2250 2023-10-08T05:36:37.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.733+0000 7fdca1258640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] conn(0x7fdc7c074580 0x7fdc7c0769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.733+0000 7f65acb53640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f659406af40 0x7f659406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.732+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c280c2180 con 0x7f5c280b3290 2023-10-08T05:36:37.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.734+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9c0c26e0 con 0x7fed9c0c2250 2023-10-08T05:36:37.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.734+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c0c2ca0 con 0x7fed9c0c2250 2023-10-08T05:36:37.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.736+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed9c06d120 con 0x7fed9c0c2250 2023-10-08T05:36:37.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.735+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5c280cbb30 con 0x7f5c280b3290 2023-10-08T05:36:37.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.736+0000 7f65a7fff640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] conn(0x7f6594074580 0x7f65940769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.736+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 --> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa52c006140 con 0x7fa520074580 2023-10-08T05:36:37.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.736+0000 7f65acb53640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f659406af40 0x7f659406d400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f659802ca10 tx=0x7f659803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.736+0000 7f5c13fff640 1 -- 172.21.15.70:0/2104372849 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5c20032070 con 0x7f5c280b3290 2023-10-08T05:36:37.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.737+0000 7fed997fa640 1 -- 172.21.15.70:0/3449347825 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fed94041810 con 0x7fed9c0c2250 2023-10-08T05:36:37.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.737+0000 7fdca1258640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] conn(0x7fdc7c074580 0x7fdc7c0769c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.737+0000 7fa51b7fe640 1 -- 172.21.15.70:0/3132789104 <== osd.1 v2:172.21.15.70:6800/435902020 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa528001a20 con 0x7fa520074580 2023-10-08T05:36:37.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7f5c13fff640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c0c06aec0 0x7f5c0c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7f5c13fff640 1 -- 172.21.15.70:0/2104372849 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5c200ab8c0 con 0x7f5c280b3290 2023-10-08T05:36:37.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 >> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] conn(0x7fa520074580 msgr2=0x7fa5200769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7fa539656640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] conn(0x7fa520074580 0x7fa5200769c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7fed997fa640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed8806aec0 0x7fed8806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7fed997fa640 1 -- 172.21.15.70:0/3449347825 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fed940ac780 con 0x7fed9c0c2250 2023-10-08T05:36:37.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7f5c13fff640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] conn(0x7f5c0c074500 0x7f5c0c076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7f5c13fff640 1 -- 172.21.15.70:0/2104372849 --> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c0c077010 con 0x7f5c0c074500 2023-10-08T05:36:37.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7f5c13fff640 1 -- 172.21.15.70:0/2104372849 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c200b02a0 con 0x7f5c280b3290 2023-10-08T05:36:37.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7fed997fa640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] conn(0x7fed88074500 0x7fed88076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:37.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7fed997fa640 1 -- 172.21.15.70:0/3449347825 --> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed88077010 con 0x7fed88074500 2023-10-08T05:36:37.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.738+0000 7fed997fa640 1 -- 172.21.15.70:0/3449347825 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed940aed60 con 0x7fed9c0c2250 2023-10-08T05:36:37.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.739+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa52006af40 msgr2=0x7fa52006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.739+0000 7fa539656640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa52006af40 0x7fa52006d400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa524000f60 tx=0x7fa524007d50 comp rx=0 tx=0).stop 2023-10-08T05:36:37.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.739+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0eb730 msgr2=0x7fa52c0d6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.739+0000 7f5c277fe640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c0c06aec0 0x7f5c0c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.739+0000 7fa539656640 1 --2- 172.21.15.70:0/3132789104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa52c0eb730 0x7fa52c0d6bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa5340639b0 tx=0x7fa534066bf0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.739+0000 7feda0f8c640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed8806aec0 0x7fed8806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.739+0000 7fa5337fe640 1 -- 172.21.15.70:0/3132789104 reap_dead start 2023-10-08T05:36:37.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.739+0000 7f5c26ffd640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] conn(0x7f5c0c074500 0x7f5c0c076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.739+0000 7fed9bfff640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] conn(0x7fed88074500 0x7fed88076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:37.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.739+0000 7f65a7fff640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] conn(0x7f6594074580 0x7f65940769c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.740+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 shutdown_connections 2023-10-08T05:36:37.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.740+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 >> 172.21.15.70:0/3132789104 conn(0x7fa52c054c40 msgr2=0x7fa52c04d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.740+0000 7fdc997fa640 1 -- 172.21.15.70:0/3711221041 <== osd.5 v2:172.21.15.181:6824/1806853192 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fdc7c077090 con 0x7fdc7c074580 2023-10-08T05:36:37.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.740+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 shutdown_connections 2023-10-08T05:36:37.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.740+0000 7f5c26ffd640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] conn(0x7f5c0c074500 0x7f5c0c076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.741+0000 7fa539656640 1 -- 172.21.15.70:0/3132789104 wait complete. 2023-10-08T05:36:37.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.741+0000 7fed9bfff640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] conn(0x7fed88074500 0x7fed88076940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.741+0000 7feda0f8c640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed8806aec0 0x7fed8806d380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00de70 tx=0x7fed8c00f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.741+0000 7f5c277fe640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c0c06aec0 0x7f5c0c06d380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c14006fd0 tx=0x7f5c14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:37.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.743+0000 7f5c13fff640 1 -- 172.21.15.70:0/2104372849 <== osd.4 v2:172.21.15.181:6800/2785838626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f5c0c077010 con 0x7f5c0c074500 2023-10-08T05:36:37.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.744+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 --> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f42bc006d80 con 0x7f42a8074440 2023-10-08T05:36:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.745+0000 7f65a57fa640 1 -- 172.21.15.70:0/3793368201 <== osd.6 v2:172.21.15.181:6802/4245641278 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f6594077090 con 0x7f6594074580 2023-10-08T05:36:37.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.746+0000 7f42ba7fc640 1 -- 172.21.15.70:0/3540283483 <== osd.0 v2:172.21.15.70:6816/1284805295 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f42c404f050 con 0x7f42a8074440 2023-10-08T05:36:37.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.743+0000 7fed997fa640 1 -- 172.21.15.70:0/3449347825 <== osd.7 v2:172.21.15.181:6816/1658798817 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fed88077010 con 0x7fed88074500 2023-10-08T05:36:37.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.748+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 >> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] conn(0x7f42a8074440 msgr2=0x7f42a8076880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.748+0000 7f42caeb4640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] conn(0x7f42a8074440 0x7f42a8076880 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.749+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f42a806af70 msgr2=0x7f42a806d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.749+0000 7f42caeb4640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f42a806af70 0x7f42a806d430 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42c0036000 tx=0x7f42c0030000 comp rx=0 tx=0).stop 2023-10-08T05:36:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.749+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42bc005fe0 msgr2=0x7f42bc1889d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.749+0000 7f42caeb4640 1 --2- 172.21.15.70:0/3540283483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42bc005fe0 0x7f42bc1889d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f42b400b480 tx=0x7f42b400b940 comp rx=0 tx=0).stop 2023-10-08T05:36:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.749+0000 7f42ca6b3640 1 -- 172.21.15.70:0/3540283483 reap_dead start 2023-10-08T05:36:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.750+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 shutdown_connections 2023-10-08T05:36:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.750+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 >> 172.21.15.70:0/3540283483 conn(0x7f42bc054c10 msgr2=0x7f42bc004c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.750+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 shutdown_connections 2023-10-08T05:36:37.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.750+0000 7f42caeb4640 1 -- 172.21.15.70:0/3540283483 wait complete. 2023-10-08T05:36:37.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.751+0000 7f1f1759e640 1 -- 172.21.15.70:0/3255994779 --> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1efc002cb0 con 0x7f1ef8074580 2023-10-08T05:36:37.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.752+0000 7f1f06ffd640 1 -- 172.21.15.70:0/3255994779 <== osd.2 v2:172.21.15.70:6817/3799730343 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1efc002cb0 con 0x7f1ef8074580 2023-10-08T05:36:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.753+0000 7f1f04ff9640 1 -- 172.21.15.70:0/3255994779 >> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] conn(0x7f1ef8074580 msgr2=0x7f1ef80769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.753+0000 7f1f04ff9640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] conn(0x7f1ef8074580 0x7f1ef80769c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.753+0000 7f1f04ff9640 1 -- 172.21.15.70:0/3255994779 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ef806af40 msgr2=0x7f1ef806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.754+0000 7f1f04ff9640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ef806af40 0x7f1ef806d400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c006e80 tx=0x7f1f0c006dd0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.754+0000 7f1f04ff9640 1 -- 172.21.15.70:0/3255994779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f100582c0 msgr2=0x7f1f100d39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.754+0000 7f1f04ff9640 1 --2- 172.21.15.70:0/3255994779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f100582c0 0x7f1f100d39f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f1804f940 tx=0x7f1f180734e0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.754+0000 7f1f16d9d640 1 -- 172.21.15.70:0/3255994779 reap_dead start 2023-10-08T05:36:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.754+0000 7f1f04ff9640 1 -- 172.21.15.70:0/3255994779 shutdown_connections 2023-10-08T05:36:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.754+0000 7f1f04ff9640 1 -- 172.21.15.70:0/3255994779 >> 172.21.15.70:0/3255994779 conn(0x7f1f10059120 msgr2=0x7f1f100d86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.756+0000 7f1f04ff9640 1 -- 172.21.15.70:0/3255994779 shutdown_connections 2023-10-08T05:36:37.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.756+0000 7f1f04ff9640 1 -- 172.21.15.70:0/3255994779 wait complete. 2023-10-08T05:36:37.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.758+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 --> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe6680cbc60 con 0x7fe654074500 2023-10-08T05:36:37.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.760+0000 7fe64f7fe640 1 -- 172.21.15.70:0/2976239571 <== osd.3 v2:172.21.15.70:6808/471550804 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe660009a70 con 0x7fe654074500 2023-10-08T05:36:37.851 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:37.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.763+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 >> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] conn(0x7fe654074500 msgr2=0x7fe654076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.763+0000 7fe66d6f2640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] conn(0x7fe654074500 0x7fe654076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-08T05:36:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.763+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe65406af40 msgr2=0x7fe65406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.763+0000 7fe66d6f2640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe65406af40 0x7fe65406d400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe65c038000 tx=0x7fe65c008030 comp rx=0 tx=0).stop 2023-10-08T05:36:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.763+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6680cae90 msgr2=0x7fe668159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.763+0000 7fe66d6f2640 1 --2- 172.21.15.70:0/2976239571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6680cae90 0x7fe668159f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe65800c500 tx=0x7fe65800c9c0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.763+0000 7fe6677fe640 1 -- 172.21.15.70:0/2976239571 reap_dead start 2023-10-08T05:36:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.764+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 shutdown_connections 2023-10-08T05:36:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.764+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 >> 172.21.15.70:0/2976239571 conn(0x7fe6680660e0 msgr2=0x7fe6680b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.765+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 shutdown_connections 2023-10-08T05:36:37.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.765+0000 7fe66d6f2640 1 -- 172.21.15.70:0/2976239571 wait complete. 2023-10-08T05:36:37.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.769+0000 7fdc92ffd640 1 -- 172.21.15.70:0/3711221041 --> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdc9c0ce340 con 0x7fdc7c074580 2023-10-08T05:36:37.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.771+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 --> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65a806ce40 con 0x7f6594074580 2023-10-08T05:36:37.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.771+0000 7fdc997fa640 1 -- 172.21.15.70:0/3711221041 <== osd.5 v2:172.21.15.181:6824/1806853192 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdc9402c9c0 con 0x7fdc7c074580 2023-10-08T05:36:37.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.772+0000 7fdc92ffd640 1 -- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] conn(0x7fdc7c074580 msgr2=0x7fdc7c0769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.772+0000 7fdc92ffd640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] conn(0x7fdc7c074580 0x7fdc7c0769c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.772+0000 7f65a57fa640 1 -- 172.21.15.70:0/3793368201 <== osd.6 v2:172.21.15.181:6802/4245641278 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6594077090 con 0x7f6594074580 2023-10-08T05:36:37.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.772+0000 7fdc92ffd640 1 -- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fdc7c06af40 msgr2=0x7fdc7c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7fdc92ffd640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fdc7c06af40 0x7fdc7c06d400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdc84009870 tx=0x7fdc84008040 comp rx=0 tx=0).stop 2023-10-08T05:36:37.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] conn(0x7f6594074580 msgr2=0x7f65940769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7fdc92ffd640 1 -- 172.21.15.70:0/3711221041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc9c0cd4e0 msgr2=0x7fdc9c1f1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7fdc92ffd640 1 --2- 172.21.15.70:0/3711221041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc9c0cd4e0 0x7fdc9c1f1e90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c00e870 tx=0x7fdc8c00ed30 comp rx=0 tx=0).stop 2023-10-08T05:36:37.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7f65aedb7640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] conn(0x7f6594074580 0x7f65940769c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f659406af40 msgr2=0x7f659406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7f65aedb7640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f659406af40 0x7f659406d400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f659802ca10 tx=0x7f659803d040 comp rx=0 tx=0).stop 2023-10-08T05:36:37.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65a80cb3d0 msgr2=0x7f65a81577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7f65aedb7640 1 --2- 172.21.15.70:0/3793368201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65a80cb3d0 0x7f65a81577d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f65a000ced0 tx=0x7f65a0001830 comp rx=0 tx=0).stop 2023-10-08T05:36:37.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7f65ad354640 1 -- 172.21.15.70:0/3793368201 reap_dead start 2023-10-08T05:36:37.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 shutdown_connections 2023-10-08T05:36:37.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.773+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 >> 172.21.15.70:0/3793368201 conn(0x7f65a806a5d0 msgr2=0x7f65a80cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.774+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 shutdown_connections 2023-10-08T05:36:37.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.774+0000 7fdca1258640 1 -- 172.21.15.70:0/3711221041 reap_dead start 2023-10-08T05:36:37.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.774+0000 7fdc92ffd640 1 -- 172.21.15.70:0/3711221041 shutdown_connections 2023-10-08T05:36:37.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.774+0000 7fdc92ffd640 1 -- 172.21.15.70:0/3711221041 >> 172.21.15.70:0/3711221041 conn(0x7fdc9c0cf700 msgr2=0x7fdc9c0b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.859 INFO:teuthology.orchestra.run.smithi070.stdout:55834574857 2023-10-08T05:36:37.860 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-08T05:36:37.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.776+0000 7f65aedb7640 1 -- 172.21.15.70:0/3793368201 wait complete. 2023-10-08T05:36:37.861 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:37.861 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-08T05:36:37.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.777+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 --> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5c28161560 con 0x7f5c0c074500 2023-10-08T05:36:37.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.777+0000 7fdc92ffd640 1 -- 172.21.15.70:0/3711221041 shutdown_connections 2023-10-08T05:36:37.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.777+0000 7fdc92ffd640 1 -- 172.21.15.70:0/3711221041 wait complete. 2023-10-08T05:36:37.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.777+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 --> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed9c06ce40 con 0x7fed88074500 2023-10-08T05:36:37.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.778+0000 7f5c13fff640 1 -- 172.21.15.70:0/2104372849 <== osd.4 v2:172.21.15.181:6800/2785838626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5c1c00c040 con 0x7f5c0c074500 2023-10-08T05:36:37.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.778+0000 7fed997fa640 1 -- 172.21.15.70:0/3449347825 <== osd.7 v2:172.21.15.181:6816/1658798817 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fed90009200 con 0x7fed88074500 2023-10-08T05:36:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.778+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] conn(0x7f5c0c074500 msgr2=0x7f5c0c076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.778+0000 7f5c2e0d4640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] conn(0x7f5c0c074500 0x7f5c0c076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c0c06aec0 msgr2=0x7f5c0c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7f5c2e0d4640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c0c06aec0 0x7f5c0c06d380 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c14006fd0 tx=0x7f5c14008040 comp rx=0 tx=0).stop 2023-10-08T05:36:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c280b3290 msgr2=0x7f5c280b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7f5c2e0d4640 1 --2- 172.21.15.70:0/2104372849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c280b3290 0x7f5c280b3710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5c20031000 tx=0x7f5c280ccf00 comp rx=0 tx=0).stop 2023-10-08T05:36:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7f5c27fff640 1 -- 172.21.15.70:0/2104372849 reap_dead start 2023-10-08T05:36:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 shutdown_connections 2023-10-08T05:36:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 >> 172.21.15.70:0/2104372849 conn(0x7f5c2806a170 msgr2=0x7f5c280ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] conn(0x7fed88074500 msgr2=0x7fed88076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7feda31f0640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] conn(0x7fed88074500 0x7fed88076940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed8806aec0 msgr2=0x7fed8806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 shutdown_connections 2023-10-08T05:36:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7feda31f0640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed8806aec0 0x7fed8806d380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00de70 tx=0x7fed8c00f040 comp rx=0 tx=0).stop 2023-10-08T05:36:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed9c0c2250 msgr2=0x7fed9c0c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7feda31f0640 1 --2- 172.21.15.70:0/3449347825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed9c0c2250 0x7fed9c0c9270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fed94007210 tx=0x7fed94004a40 comp rx=0 tx=0).stop 2023-10-08T05:36:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7feda178d640 1 -- 172.21.15.70:0/3449347825 reap_dead start 2023-10-08T05:36:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7f5c2e0d4640 1 -- 172.21.15.70:0/2104372849 wait complete. 2023-10-08T05:36:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 shutdown_connections 2023-10-08T05:36:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.779+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 >> 172.21.15.70:0/3449347825 conn(0x7fed9c06a5d0 msgr2=0x7fed9c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.780+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 shutdown_connections 2023-10-08T05:36:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:37.780+0000 7feda31f0640 1 -- 172.21.15.70:0/3449347825 wait complete. 2023-10-08T05:36:37.868 INFO:teuthology.orchestra.run.smithi070.stdout:55834574858 2023-10-08T05:36:37.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-08T05:36:37.869 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:37.869 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:37.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-08T05:36:37.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-08T05:36:37.871 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:37.871 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-08T05:36:37.872 INFO:teuthology.orchestra.run.smithi070.stdout:55834574857 2023-10-08T05:36:37.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-08T05:36:38.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.203+0000 7f14b3bb6640 1 Processor -- start 2023-10-08T05:36:38.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.203+0000 7f14b3bb6640 1 -- start start 2023-10-08T05:36:38.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.205+0000 7f14b3bb6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f14a4033ff0 0x7f14a4053180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.205+0000 7f14b3bb6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14a4050260 0x7f14a40e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.205+0000 7f14b3bb6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14a40e3380 0x7f14a40e3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.205+0000 7f14b3bb6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a4054420 con 0x7f14a4050260 2023-10-08T05:36:38.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.205+0000 7f14b3bb6640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a40343d0 con 0x7f14a4033ff0 2023-10-08T05:36:38.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.205+0000 7f14b3bb6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a4053810 con 0x7f14a40e3380 2023-10-08T05:36:38.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.206+0000 7f14b1151640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14a4050260 0x7f14a40e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.206+0000 7f14b1952640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f14a4033ff0 0x7f14a4053180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.206+0000 7f14b1151640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14a4050260 0x7f14a40e2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59960/0 (socket says 172.21.15.70:59960) 2023-10-08T05:36:38.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.206+0000 7f14b1151640 1 -- 172.21.15.70:0/1971281319 learned_addr learned my addr 172.21.15.70:0/1971281319 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.207+0000 7f14b1952640 1 -- 172.21.15.70:0/1971281319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14a40e3380 msgr2=0x7f14a40e3810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T05:36:38.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.207+0000 7f14b1952640 1 --2- 172.21.15.70:0/1971281319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14a40e3380 0x7f14a40e3810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.207+0000 7f14b1952640 1 -- 172.21.15.70:0/1971281319 >> v2:172.21.15.70:3300/0 conn(0x7f14a4050260 msgr2=0x7f14a40e2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:38.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.207+0000 7f14b1952640 1 --2- 172.21.15.70:0/1971281319 >> v2:172.21.15.70:3300/0 conn(0x7f14a4050260 0x7f14a40e2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.207+0000 7f14b1952640 1 -- 172.21.15.70:0/1971281319 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a4053af0 con 0x7f14a4033ff0 2023-10-08T05:36:38.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.208+0000 7f9c6b59e640 1 Processor -- start 2023-10-08T05:36:38.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.209+0000 7f14b1952640 1 --2- 172.21.15.70:0/1971281319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4033ff0 0x7f14a4053180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14ac02cb40 tx=0x7f14ac0a56f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbcf7a41758cf19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.208+0000 7f9c6b59e640 1 -- start start 2023-10-08T05:36:38.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.209+0000 7fa5f359e640 1 Processor -- start 2023-10-08T05:36:38.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.210+0000 7fa5f359e640 1 -- start start 2023-10-08T05:36:38.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.211+0000 7f9c6b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c640e18f0 0x7f9c640062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.211+0000 7f9c69d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9c640e1cd0 0x7f9c640eb170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43690/0 (socket says 172.21.15.70:43690) 2023-10-08T05:36:38.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7f9c6a59c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c640e18f0 0x7f9c640062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.211+0000 7f14b0950640 1 -- 172.21.15.70:0/1971281319 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0aa7d0 con 0x7f14a4033ff0 2023-10-08T05:36:38.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.211+0000 7f14b0950640 1 -- 172.21.15.70:0/1971281319 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac0a8050 con 0x7f14a4033ff0 2023-10-08T05:36:38.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7f14b3bb6640 1 -- 172.21.15.70:0/1971281319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4033ff0 msgr2=0x7f14a4053180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7f14b3bb6640 1 --2- 172.21.15.70:0/1971281319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4033ff0 0x7f14a4053180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14ac02cb40 tx=0x7f14ac0a56f0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7f9c6a59c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c640e18f0 0x7f9c640062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:60668/0 (socket says 172.21.15.70:60668) 2023-10-08T05:36:38.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7fa5f359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5ec0e78b0 0x7fa5ec04f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7fa5f359e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec050340 con 0x7fa5ec0eb2c0 2023-10-08T05:36:38.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7fa5f359e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ec006480 con 0x7fa5ec0eb700 2023-10-08T05:36:38.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7fa5f359e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ec053e00 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.213+0000 7f9c69d9b640 1 -- 172.21.15.70:0/2078193699 learned_addr learned my addr 172.21.15.70:0/2078193699 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:38.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.213+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2078193699 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c64006b20 con 0x7f9c640e1cd0 2023-10-08T05:36:38.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7f14b3bb6640 1 -- 172.21.15.70:0/1971281319 shutdown_connections 2023-10-08T05:36:38.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7f14b3bb6640 1 --2- 172.21.15.70:0/1971281319 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14a40e3380 0x7f14a40e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.212+0000 7f14b3bb6640 1 --2- 172.21.15.70:0/1971281319 >> v2:172.21.15.70:3300/0 conn(0x7f14a4050260 0x7f14a40e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.213+0000 7f14b3bb6640 1 --2- 172.21.15.70:0/1971281319 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4033ff0 0x7f14a4053180 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14ac02cb40 tx=0x7f14ac0a56f0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.213+0000 7f14b3bb6640 1 -- 172.21.15.70:0/1971281319 >> 172.21.15.70:0/1971281319 conn(0x7f14a4054c40 msgr2=0x7f14a4055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.213+0000 7fa5f259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5ec0e78b0 0x7fa5ec04f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.213+0000 7fa5f259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5ec0e78b0 0x7fa5ec04f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59970/0 (socket says 172.21.15.70:59970) 2023-10-08T05:36:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.213+0000 7fa5f259c640 1 -- 172.21.15.70:0/3668720906 learned_addr learned my addr 172.21.15.70:0/3668720906 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.214+0000 7f14b3bb6640 1 -- 172.21.15.70:0/1971281319 shutdown_connections 2023-10-08T05:36:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.214+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2078193699 --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c64006e00 con 0x7f9c640eb880 2023-10-08T05:36:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.214+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2078193699 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640070e0 con 0x7f9c640e18f0 2023-10-08T05:36:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.214+0000 7fa5f259c640 1 -- 172.21.15.70:0/3668720906 >> v1:172.21.15.181:6789/0 conn(0x7fa5ec0eb700 legacy=0x7fa5ec0536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.214+0000 7fa5f259c640 1 -- 172.21.15.70:0/3668720906 >> v1:172.21.15.70:6789/0 conn(0x7fa5ec0eb2c0 legacy=0x7fa5ec052fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.214+0000 7fa5f259c640 1 -- 172.21.15.70:0/3668720906 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ec007920 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.214+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518462815 0 0) 0x7f9c64006b20 con 0x7f9c640e1cd0 2023-10-08T05:36:38.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.214+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c54003520 con 0x7f9c640e1cd0 2023-10-08T05:36:38.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 201847115 0 0) 0x7f9c54003520 con 0x7f9c640e1cd0 2023-10-08T05:36:38.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c640e18f0 msgr2=0x7f9c640062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:38.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c640e18f0 0x7f9c640062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 >> v1:172.21.15.181:6789/0 conn(0x7f9c640eb880 legacy=0x7f9c64057cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c640073c0 con 0x7f9c640e1cd0 2023-10-08T05:36:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9c6c04e9b0 con 0x7f9c640e1cd0 2023-10-08T05:36:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 >> v1:172.21.15.70:6789/0 conn(0x7f9c640e1cd0 legacy=0x7f9c640eb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540039a0 0x7f9c54003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c540047c0 0x7f9c54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c54005180 0x7f9c5400b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640070e0 con 0x7f9c540039a0 2023-10-08T05:36:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5400bf40 con 0x7f9c540047c0 2023-10-08T05:36:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.215+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5400c100 con 0x7f9c54005180 2023-10-08T05:36:38.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.216+0000 7fa5f259c640 1 --2- 172.21.15.70:0/3668720906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 0x7fa5ec04f260 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009870 tx=0x7fa5e402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8057463a422c49f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.216+0000 7f9c69d9b640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c540047c0 0x7f9c54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.216+0000 7f9c6a59c640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540039a0 0x7f9c54003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.217+0000 7f9c6ad9d640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c54005180 0x7f9c5400b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.217+0000 7fa5f159a640 1 -- 172.21.15.70:0/3668720906 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4030860 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.217+0000 7f9c6a59c640 1 -- 172.21.15.70:0/2078193699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c54005180 msgr2=0x7f9c5400b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:38.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.217+0000 7f9c6a59c640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c54005180 0x7f9c5400b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.214+0000 7f14b3bb6640 1 -- 172.21.15.70:0/1971281319 wait complete. 2023-10-08T05:36:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.217+0000 7fa5f159a640 1 -- 172.21.15.70:0/3668720906 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e40309c0 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.218+0000 7fa5f359e640 1 -- 172.21.15.70:0/3668720906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 msgr2=0x7fa5ec04f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.218+0000 7fa5f359e640 1 --2- 172.21.15.70:0/3668720906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 0x7fa5ec04f260 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009870 tx=0x7fa5e402b9a0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.218+0000 7fa5f359e640 1 -- 172.21.15.70:0/3668720906 shutdown_connections 2023-10-08T05:36:38.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.218+0000 7fa5f359e640 1 --2- 172.21.15.70:0/3668720906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 0x7fa5ec04f260 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.218+0000 7fa5f359e640 1 -- 172.21.15.70:0/3668720906 >> 172.21.15.70:0/3668720906 conn(0x7fa5ec054c10 msgr2=0x7fa5ec054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.217+0000 7f14b3bb6640 1 Processor -- start 2023-10-08T05:36:38.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.217+0000 7f14b3bb6640 1 -- start start 2023-10-08T05:36:38.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.219+0000 7fa5f359e640 1 -- 172.21.15.70:0/3668720906 shutdown_connections 2023-10-08T05:36:38.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.219+0000 7f14b3bb6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4050260 0x7f14a404d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.220+0000 7f14b3bb6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14a40e3380 0x7f14a404ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.220+0000 7f14b3bb6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14a404da10 0x7f14a404f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.220+0000 7f14b3bb6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a404d040 con 0x7f14a404da10 2023-10-08T05:36:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.220+0000 7f14b3bb6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a404c430 con 0x7f14a4050260 2023-10-08T05:36:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.220+0000 7f14b3bb6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a404c710 con 0x7f14a40e3380 2023-10-08T05:36:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.217+0000 7f9c6a59c640 1 -- 172.21.15.70:0/2078193699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c540047c0 msgr2=0x7f9c54004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.217+0000 7f9c6a59c640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c540047c0 0x7f9c54004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.217+0000 7f9c6a59c640 1 -- 172.21.15.70:0/2078193699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c640073c0 con 0x7f9c540039a0 2023-10-08T05:36:38.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.220+0000 7f14b1952640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4050260 0x7f14a404d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.220+0000 7f14b1952640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4050260 0x7f14a404d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39734/0 (socket says 172.21.15.70:39734) 2023-10-08T05:36:38.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.220+0000 7f14b1952640 1 -- 172.21.15.70:0/1436311010 learned_addr learned my addr 172.21.15.70:0/1436311010 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.219+0000 7fa5f359e640 1 -- 172.21.15.70:0/3668720906 wait complete. 2023-10-08T05:36:38.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.221+0000 7fc07921a640 1 Processor -- start 2023-10-08T05:36:38.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.221+0000 7f14b2153640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14a404da10 0x7f14a404f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.221+0000 7f14b1952640 1 -- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14a40e3380 msgr2=0x7f14a404ecb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T05:36:38.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.221+0000 7f14b1952640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14a40e3380 0x7f14a404ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.221+0000 7f14b1952640 1 -- 172.21.15.70:0/1436311010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14a404da10 msgr2=0x7f14a404f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.221+0000 7f56bd887640 1 Processor -- start 2023-10-08T05:36:38.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.221+0000 7f14b1952640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14a404da10 0x7f14a404f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.221+0000 7f14b1952640 1 -- 172.21.15.70:0/1436311010 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac04f050 con 0x7f14a4050260 2023-10-08T05:36:38.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.221+0000 7f14b2153640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14a404da10 0x7f14a404f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.222+0000 7f56bd887640 1 -- start start 2023-10-08T05:36:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.222+0000 7fa5f359e640 1 Processor -- start 2023-10-08T05:36:38.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.222+0000 7fa5f359e640 1 -- start start 2023-10-08T05:36:38.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.222+0000 7fe6fe555640 1 Processor -- start 2023-10-08T05:36:38.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.222+0000 7f9c6a59c640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540039a0 0x7f9c54003dc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c6403bc50 tx=0x7f9c540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512f4b6f2b46f4f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.222+0000 7fe6fe555640 1 -- start start 2023-10-08T05:36:38.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.223+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c034610 con 0x7f9c540039a0 2023-10-08T05:36:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.223+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c02e070 con 0x7f9c540039a0 2023-10-08T05:36:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7f9c6959a640 1 -- 172.21.15.70:0/2078193699 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c034610 con 0x7f9c540039a0 2023-10-08T05:36:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fa5f359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 0x7fa5ec0d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.221+0000 7fc07921a640 1 -- start start 2023-10-08T05:36:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fc07921a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc0740b4e10 0x7fc0741612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fc07921a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0740cd4e0 0x7fc0740d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fc07921a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0740b8c30 con 0x7fc07415cd70 2023-10-08T05:36:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fc07921a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0740cdd80 con 0x7fc0740cd4e0 2023-10-08T05:36:38.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fc07921a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0740ce060 con 0x7fc0740b4e10 2023-10-08T05:36:38.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7f56bd887640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b80cede0 0x7f56b806c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.222+0000 7f14b1952640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4050260 0x7f14a404d4d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f14ac02cb40 tx=0x7f14ac066bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.223+0000 7f14a27fc640 1 -- 172.21.15.70:0/1436311010 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0acd10 con 0x7f14a4050260 2023-10-08T05:36:38.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.223+0000 7f14b3bb6640 1 -- 172.21.15.70:0/1436311010 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a404c9f0 con 0x7f14a4050260 2023-10-08T05:36:38.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.223+0000 7f14b3bb6640 1 -- 172.21.15.70:0/1436311010 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a417ad80 con 0x7f14a4050260 2023-10-08T05:36:38.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.223+0000 7f14a27fc640 1 -- 172.21.15.70:0/1436311010 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac0ace70 con 0x7f14a4050260 2023-10-08T05:36:38.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.223+0000 7f14a27fc640 1 -- 172.21.15.70:0/1436311010 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0b6640 con 0x7f14a4050260 2023-10-08T05:36:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2078193699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540039a0 msgr2=0x7f9c54003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7f9c6b59e640 1 --2- 172.21.15.70:0/2078193699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c540039a0 0x7f9c54003dc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c6403bc50 tx=0x7f9c540057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7f9c6ad9d640 1 -- 172.21.15.70:0/2078193699 reap_dead start 2023-10-08T05:36:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fe6fe555640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6f806a460 0x7fe6f8158f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fe6fe555640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6f80cede0 0x7fe6f806c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fe6fe555640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f81594a0 con 0x7fe6f80ca220 2023-10-08T05:36:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fe6fe555640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f80cb6a0 con 0x7fe6f806a460 2023-10-08T05:36:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.224+0000 7fe6fe555640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f80cb980 con 0x7fe6f80cede0 2023-10-08T05:36:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7f56bd887640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f56b80ca220 0x7f56b806cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7fa5f359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5ec0eb2c0 0x7fa5ec04c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7fa5f359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5ec0eb700 0x7fa5ec04c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7fa5f359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0d2d80 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7fa5f359e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0e28f0 con 0x7fa5ec0eb2c0 2023-10-08T05:36:38.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7fa5f359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ec0d2470 con 0x7fa5ec0eb700 2023-10-08T05:36:38.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7f56b6ffd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f56b81591d0 0x7f56b80ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:52070/0 (socket says 172.21.15.70:52070) 2023-10-08T05:36:38.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7fc07259c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc0740b4e10 0x7fc0741612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7f56b6ffd640 1 -- 172.21.15.70:0/3045535779 learned_addr learned my addr 172.21.15.70:0/3045535779 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:38.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7f56bd887640 1 -- 172.21.15.70:0/3045535779 --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b80cb3c0 con 0x7f56b81591d0 2023-10-08T05:36:38.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7fe6f7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6f806a460 0x7fe6f8158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7fe6f7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6f806a460 0x7fe6f8158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59990/0 (socket says 172.21.15.70:59990) 2023-10-08T05:36:38.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7f56bd887640 1 -- 172.21.15.70:0/3045535779 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b80cb6a0 con 0x7f56b80ca220 2023-10-08T05:36:38.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7fe6f7fff640 1 -- 172.21.15.70:0/2989709495 learned_addr learned my addr 172.21.15.70:0/2989709495 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.225+0000 7f56b67fc640 1 --2- 172.21.15.70:0/3045535779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b80cede0 0x7f56b806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7f56bd887640 1 -- 172.21.15.70:0/3045535779 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b80cb980 con 0x7f56b80cede0 2023-10-08T05:36:38.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7f56b77fe640 1 --2- 172.21.15.70:0/3045535779 >> v2:172.21.15.70:3300/0 conn(0x7f56b80ca220 0x7f56b806cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7fc07359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0740cd4e0 0x7fc0740d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7fa5f259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 0x7fa5ec0d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7fa5f259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 0x7fa5ec0d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60012/0 (socket says 172.21.15.70:60012) 2023-10-08T05:36:38.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7fa5f259c640 1 -- 172.21.15.70:0/283680561 learned_addr learned my addr 172.21.15.70:0/283680561 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7f56b5ffb640 1 -- 172.21.15.70:0/3045535779 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3468432217 0 0) 0x7f56b80cb3c0 con 0x7f56b81591d0 2023-10-08T05:36:38.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7f14b3bb6640 1 -- 172.21.15.70:0/1436311010 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a40054c0 con 0x7f14a4050260 2023-10-08T05:36:38.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7fe6fcaf2640 1 --2- 172.21.15.70:0/2989709495 >> v2:172.21.15.181:3300/0 conn(0x7fe6f80cede0 0x7fe6f806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7fc072d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc07415cd70 0x7fc0740b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43702/0 (socket says 172.21.15.70:43702) 2023-10-08T05:36:38.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7fc072d9d640 1 -- 172.21.15.70:0/3435488673 learned_addr learned my addr 172.21.15.70:0/3435488673 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:38.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7f56b5ffb640 1 -- 172.21.15.70:0/3045535779 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a0003520 con 0x7f56b81591d0 2023-10-08T05:36:38.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7fa5f1d9b640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5ec0eb2c0 0x7fa5ec04c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2078193699 shutdown_connections 2023-10-08T05:36:38.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.226+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2078193699 >> 172.21.15.70:0/2078193699 conn(0x7f9c640590c0 msgr2=0x7f9c640594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2078193699 shutdown_connections 2023-10-08T05:36:38.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2078193699 wait complete. 2023-10-08T05:36:38.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7f56b67fc640 1 -- 172.21.15.70:0/3045535779 >> v2:172.21.15.70:3300/0 conn(0x7f56b80ca220 msgr2=0x7f56b806cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:38.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7f56b67fc640 1 --2- 172.21.15.70:0/3045535779 >> v2:172.21.15.70:3300/0 conn(0x7f56b80ca220 0x7f56b806cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7f56b67fc640 1 -- 172.21.15.70:0/3045535779 >> v1:172.21.15.181:6789/0 conn(0x7f56b81591d0 legacy=0x7f56b80ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7f56b67fc640 1 -- 172.21.15.70:0/3045535779 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80cbc60 con 0x7f56b80cede0 2023-10-08T05:36:38.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7fa5f2d9d640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5ec0eb700 0x7fa5ec04c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.228+0000 7fc07359e640 1 -- 172.21.15.70:0/3435488673 >> v2:172.21.15.181:3300/0 conn(0x7fc0740b4e10 msgr2=0x7fc0741612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:38.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.228+0000 7fc07359e640 1 --2- 172.21.15.70:0/3435488673 >> v2:172.21.15.181:3300/0 conn(0x7fc0740b4e10 0x7fc0741612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.228+0000 7fc07359e640 1 -- 172.21.15.70:0/3435488673 >> v1:172.21.15.70:6789/0 conn(0x7fc07415cd70 legacy=0x7fc0740b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.228+0000 7fa5f259c640 1 -- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5ec0eb700 msgr2=0x7fa5ec04c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.228+0000 7fa5f259c640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5ec0eb700 0x7fa5ec04c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.228+0000 7fa5f259c640 1 -- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5ec0eb2c0 msgr2=0x7fa5ec04c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.228+0000 7fa5f259c640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5ec0eb2c0 0x7fa5ec04c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.228+0000 7fa5f259c640 1 -- 172.21.15.70:0/283680561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e40094d0 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7fe6f7fff640 1 -- 172.21.15.70:0/2989709495 >> v2:172.21.15.181:3300/0 conn(0x7fe6f80cede0 msgr2=0x7fe6f806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:38.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7fe6f7fff640 1 --2- 172.21.15.70:0/2989709495 >> v2:172.21.15.181:3300/0 conn(0x7fe6f80cede0 0x7fe6f806c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7fe6f7fff640 1 -- 172.21.15.70:0/2989709495 >> v1:172.21.15.181:6789/0 conn(0x7fe6f80ca220 legacy=0x7fe6f80ce6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T05:36:38.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.227+0000 7fe6f7fff640 1 -- 172.21.15.70:0/2989709495 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f80cbc60 con 0x7fe6f806a460 2023-10-08T05:36:38.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.228+0000 7fc07359e640 1 -- 172.21.15.70:0/3435488673 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0740ce340 con 0x7fc0740cd4e0 2023-10-08T05:36:38.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.229+0000 7fe6f7fff640 1 --2- 172.21.15.70:0/2989709495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f806a460 0x7fe6f8158f60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80cb5a0 tx=0x7fe6ec02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3eb36be9f04b86e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.229+0000 7fa5f259c640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 0x7fa5ec0d1c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009990 tx=0x7fa5e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.228+0000 7f14a27fc640 1 -- 172.21.15.70:0/1436311010 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f14ac0aa980 con 0x7f14a4050260 2023-10-08T05:36:38.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.229+0000 7f14a27fc640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f149006aec0 0x7f149006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.229+0000 7fa5e2ffd640 1 -- 172.21.15.70:0/283680561 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4030840 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.229+0000 7f14a27fc640 1 -- 172.21.15.70:0/1436311010 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f14ac124f60 con 0x7f14a4050260 2023-10-08T05:36:38.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.230+0000 7fa5e2ffd640 1 -- 172.21.15.70:0/283680561 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e402f5c0 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.230+0000 7fe6f6ffd640 1 -- 172.21.15.70:0/2989709495 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec02cb20 con 0x7fe6f806a460 2023-10-08T05:36:38.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.230+0000 7fe6f6ffd640 1 -- 172.21.15.70:0/2989709495 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec02cc80 con 0x7fe6f806a460 2023-10-08T05:36:38.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.230+0000 7fe6f6ffd640 1 -- 172.21.15.70:0/2989709495 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0343f0 con 0x7fe6f806a460 2023-10-08T05:36:38.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.230+0000 7f14b1151640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f149006aec0 0x7f149006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.230+0000 7fa5f359e640 1 -- 172.21.15.70:0/283680561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ec0e3ce0 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.230+0000 7fa5f359e640 1 -- 172.21.15.70:0/283680561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ec0e9ac0 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.230+0000 7f56b67fc640 1 --2- 172.21.15.70:0/3045535779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b80cede0 0x7f56b806c680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f56ac008260 tx=0x7f56ac00ab30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b1314d8972f6f27 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.231+0000 7fc07359e640 1 --2- 172.21.15.70:0/3435488673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0740cd4e0 0x7fc0740d1650 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc0740d07d0 tx=0x7fc06802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71524555db44c958 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.231+0000 7fe6fe555640 1 -- 172.21.15.70:0/2989709495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f806a460 msgr2=0x7fe6f8158f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.231+0000 7fe6fe555640 1 --2- 172.21.15.70:0/2989709495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f806a460 0x7fe6f8158f60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80cb5a0 tx=0x7fe6ec02bf80 comp rx=0 tx=0).stop 2023-10-08T05:36:38.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.231+0000 7f56b5ffb640 1 -- 172.21.15.70:0/3045535779 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac008df0 con 0x7f56b80cede0 2023-10-08T05:36:38.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.231+0000 7f56b5ffb640 1 -- 172.21.15.70:0/3045535779 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac00a0d0 con 0x7f56b80cede0 2023-10-08T05:36:38.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.231+0000 7f56b5ffb640 1 -- 172.21.15.70:0/3045535779 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00a4a0 con 0x7f56b80cede0 2023-10-08T05:36:38.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.231+0000 7fa5e2ffd640 1 -- 172.21.15.70:0/283680561 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e402f930 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7fe6fe555640 1 -- 172.21.15.70:0/2989709495 shutdown_connections 2023-10-08T05:36:38.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7fc071d9b640 1 -- 172.21.15.70:0/3435488673 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068031830 con 0x7fc0740cd4e0 2023-10-08T05:36:38.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7f56bd887640 1 -- 172.21.15.70:0/3045535779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b80cede0 msgr2=0x7f56b806c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7fe6fe555640 1 --2- 172.21.15.70:0/2989709495 >> v2:172.21.15.181:3300/0 conn(0x7fe6f80cede0 0x7fe6f806c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7f56bd887640 1 --2- 172.21.15.70:0/3045535779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b80cede0 0x7f56b806c680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f56ac008260 tx=0x7f56ac00ab30 comp rx=0 tx=0).stop 2023-10-08T05:36:38.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7f9c6b59e640 1 Processor -- start 2023-10-08T05:36:38.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7fc071d9b640 1 -- 172.21.15.70:0/3435488673 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068031990 con 0x7fc0740cd4e0 2023-10-08T05:36:38.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7fe6fe555640 1 --2- 172.21.15.70:0/2989709495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f806a460 0x7fe6f8158f60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7fe6fe555640 1 -- 172.21.15.70:0/2989709495 >> 172.21.15.70:0/2989709495 conn(0x7fe6f80660e0 msgr2=0x7fe6f8069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7f56bd887640 1 -- 172.21.15.70:0/3045535779 shutdown_connections 2023-10-08T05:36:38.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7f56bd887640 1 --2- 172.21.15.70:0/3045535779 >> v2:172.21.15.70:3300/0 conn(0x7f56b80ca220 0x7f56b806cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7f56bd887640 1 --2- 172.21.15.70:0/3045535779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b80cede0 0x7f56b806c680 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7fe6fe555640 1 -- 172.21.15.70:0/2989709495 shutdown_connections 2023-10-08T05:36:38.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7fe6fe555640 1 -- 172.21.15.70:0/2989709495 wait complete. 2023-10-08T05:36:38.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7f14b1151640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f149006aec0 0x7f149006d380 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f14a4006c60 tx=0x7f149c009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7f56bd887640 1 -- 172.21.15.70:0/3045535779 >> 172.21.15.70:0/3045535779 conn(0x7f56b80660e0 msgr2=0x7f56b8069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.232+0000 7f9c6b59e640 1 -- start start 2023-10-08T05:36:38.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.233+0000 7fc07921a640 1 -- 172.21.15.70:0/3435488673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0740cd4e0 msgr2=0x7fc0740d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.233+0000 7fc07921a640 1 --2- 172.21.15.70:0/3435488673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0740cd4e0 0x7fc0740d1650 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc0740d07d0 tx=0x7fc06802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.233+0000 7f64d931a640 1 Processor -- start 2023-10-08T05:36:38.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.233+0000 7fa5e2ffd640 1 -- 172.21.15.70:0/283680561 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa5e402fa90 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.233+0000 7f64d931a640 1 -- start start 2023-10-08T05:36:38.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.233+0000 7fc07921a640 1 -- 172.21.15.70:0/3435488673 shutdown_connections 2023-10-08T05:36:38.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.233+0000 7fc07921a640 1 --2- 172.21.15.70:0/3435488673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0740cd4e0 0x7fc0740d1650 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.233+0000 7fc07921a640 1 --2- 172.21.15.70:0/3435488673 >> v2:172.21.15.181:3300/0 conn(0x7fc0740b4e10 0x7fc0741612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.233+0000 7fc07921a640 1 -- 172.21.15.70:0/3435488673 >> 172.21.15.70:0/3435488673 conn(0x7fc0740cf700 msgr2=0x7fc0740cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.234+0000 7f56bd887640 1 -- 172.21.15.70:0/3045535779 shutdown_connections 2023-10-08T05:36:38.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.234+0000 7fc07921a640 1 -- 172.21.15.70:0/3435488673 shutdown_connections 2023-10-08T05:36:38.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.234+0000 7f9c6b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c540039a0 0x7f9c64051300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.234+0000 7f9c6b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c640cd5c0 0x7f9c6403e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.234+0000 7f56bd887640 1 -- 172.21.15.70:0/3045535779 wait complete. 2023-10-08T05:36:38.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.234+0000 7fe6fe555640 1 Processor -- start 2023-10-08T05:36:38.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.234+0000 7fa5f359e640 1 -- 172.21.15.70:0/283680561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ec005490 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.235+0000 7f9c69d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c640cd5c0 0x7f9c6403e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.235+0000 7fe6fe555640 1 -- start start 2023-10-08T05:36:38.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.235+0000 7f9c69d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c640cd5c0 0x7f9c6403e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60016/0 (socket says 172.21.15.70:60016) 2023-10-08T05:36:38.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.235+0000 7f9c69d9b640 1 -- 172.21.15.70:0/2635704721 learned_addr learned my addr 172.21.15.70:0/2635704721 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.235+0000 7f9c6b59e640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c640529f0 0x7f9c6403c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.235+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2635704721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6404f1c0 con 0x7f9c640cd5c0 2023-10-08T05:36:38.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2635704721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64052560 con 0x7f9c540039a0 2023-10-08T05:36:38.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2635704721 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6403ebb0 con 0x7f9c640529f0 2023-10-08T05:36:38.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7f9c6ad9d640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c640529f0 0x7f9c6403c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7fe6fe555640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6f806a460 0x7fe6f8153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7fe6fe555640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f80ca220 0x7fe6f8159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7fe6fe555640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6f80cede0 0x7fe6f80b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7fe6fe555640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f8154e60 con 0x7fe6f80ca220 2023-10-08T05:36:38.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7fe6fe555640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f815a710 con 0x7fe6f80cede0 2023-10-08T05:36:38.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7fe6fe555640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f815a150 con 0x7fe6f806a460 2023-10-08T05:36:38.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7f64d3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f64d41530f0 0x7f64d4159960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43718/0 (socket says 172.21.15.70:43718) 2023-10-08T05:36:38.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.234+0000 7fc07921a640 1 -- 172.21.15.70:0/3435488673 wait complete. 2023-10-08T05:36:38.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7f64d3fff640 1 -- 172.21.15.70:0/3350835419 learned_addr learned my addr 172.21.15.70:0/3350835419 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:38.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7f9c6a59c640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c540039a0 0x7f9c64051300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7fc07921a640 1 Processor -- start 2023-10-08T05:36:38.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7fe6f77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f80ca220 0x7fe6f8159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7fe6f77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f80ca220 0x7fe6f8159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60024/0 (socket says 172.21.15.70:60024) 2023-10-08T05:36:38.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7fe6f77fe640 1 -- 172.21.15.70:0/4138988124 learned_addr learned my addr 172.21.15.70:0/4138988124 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7fe6f7fff640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6f806a460 0x7fe6f8153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7f56bd887640 1 Processor -- start 2023-10-08T05:36:38.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7f64d931a640 1 --2- 172.21.15.70:0/3350835419 >> v2:172.21.15.181:3300/0 conn(0x7f64d415d490 0x7f64d40caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7fe6fcaf2640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6f80cede0 0x7fe6f80b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7f9c69d9b640 1 -- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c640529f0 msgr2=0x7f9c6403c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7f9c69d9b640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c640529f0 0x7f9c6403c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7f9c69d9b640 1 -- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c540039a0 msgr2=0x7f9c64051300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7f9c69d9b640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c540039a0 0x7f9c64051300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7f9c69d9b640 1 -- 172.21.15.70:0/2635704721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c640cda40 con 0x7f9c640cd5c0 2023-10-08T05:36:38.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7f56bd887640 1 -- start start 2023-10-08T05:36:38.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7f64d931a640 1 --2- 172.21.15.70:0/3350835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40c9c10 0x7f64d40cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7f64d931a640 1 -- 172.21.15.70:0/3350835419 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d40c8be0 con 0x7f64d41530f0 2023-10-08T05:36:38.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7f64d931a640 1 -- 172.21.15.70:0/3350835419 --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d40c8e50 con 0x7f64d415d490 2023-10-08T05:36:38.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7f64d931a640 1 -- 172.21.15.70:0/3350835419 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40c9130 con 0x7f64d40c9c10 2023-10-08T05:36:38.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7f64cbfff640 1 --2- 172.21.15.70:0/3350835419 >> v2:172.21.15.181:3300/0 conn(0x7f64d415d490 0x7f64d40caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7fe6f77fe640 1 -- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6f806a460 msgr2=0x7fe6f8153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7fe6f77fe640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6f806a460 0x7fe6f8153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7fe6f77fe640 1 -- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6f80cede0 msgr2=0x7fe6f80b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7fe6f77fe640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6f80cede0 0x7fe6f80b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7fe6f77fe640 1 -- 172.21.15.70:0/4138988124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec009b30 con 0x7fe6f80ca220 2023-10-08T05:36:38.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7f9c69d9b640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c640cd5c0 0x7f9c6403e670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c02cb70 tx=0x7f9c6c0734e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.238+0000 7f64d8b19640 1 --2- 172.21.15.70:0/3350835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40c9c10 0x7f64d40cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.235+0000 7fa5e2ffd640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa5d406af40 0x7fa5d406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7fa5e2ffd640 1 -- 172.21.15.70:0/283680561 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa5e40abfb0 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.236+0000 7fa5f1d9b640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa5d406af40 0x7fa5d406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.237+0000 7fc07921a640 1 -- start start 2023-10-08T05:36:38.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7fc07921a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0740b4e10 0x7fc0740d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7fc07921a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0740cd4e0 0x7fc0740cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7fe6f77fe640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f80ca220 0x7fe6f8159690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000b470 tx=0x7fe6f000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7f64cbfff640 1 -- 172.21.15.70:0/3350835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40c9c10 msgr2=0x7f64d40cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:38.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7f64cbfff640 1 --2- 172.21.15.70:0/3350835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40c9c10 0x7f64d40cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7f64cbfff640 1 -- 172.21.15.70:0/3350835419 >> v1:172.21.15.70:6789/0 conn(0x7f64d41530f0 legacy=0x7f64d4159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7fc07921a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07415cd70 0x7fc0741f8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7f64cbfff640 1 -- 172.21.15.70:0/3350835419 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d40c9410 con 0x7f64d415d490 2023-10-08T05:36:38.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7f64d8b19640 1 --2- 172.21.15.70:0/3350835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40c9c10 0x7f64d40cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:38.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7fc07921a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740cada0 con 0x7fc07415cd70 2023-10-08T05:36:38.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7fc07921a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740cc900 con 0x7fc0740cd4e0 2023-10-08T05:36:38.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7f56bd887640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b80ca220 0x7f56b81595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7fc07921a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740caff0 con 0x7fc0740b4e10 2023-10-08T05:36:38.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7f56bd887640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b80cede0 0x7f56b80c1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.239+0000 7fc07259c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0740cd4e0 0x7fc0740cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7fc07259c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0740cd4e0 0x7fc0740cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39806/0 (socket says 172.21.15.70:39806) 2023-10-08T05:36:38.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7fc07259c640 1 -- 172.21.15.70:0/2321596564 learned_addr learned my addr 172.21.15.70:0/2321596564 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7f56bd887640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b81591d0 0x7f56b8157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7fc07359e640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07415cd70 0x7fc0741f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7f56bd887640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b8159af0 con 0x7f56b80ca220 2023-10-08T05:36:38.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7f56bd887640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b80c17d0 con 0x7f56b80cede0 2023-10-08T05:36:38.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7f56bd887640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b81404b0 con 0x7f56b81591d0 2023-10-08T05:36:38.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7f56b6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b80ca220 0x7f56b81595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7f56b6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b80ca220 0x7f56b81595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60038/0 (socket says 172.21.15.70:60038) 2023-10-08T05:36:38.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7f56b6ffd640 1 -- 172.21.15.70:0/2750884771 learned_addr learned my addr 172.21.15.70:0/2750884771 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7f56b67fc640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b80cede0 0x7f56b80c1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7fc072d9d640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0740b4e10 0x7fc0740d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7fc07359e640 1 -- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0740b4e10 msgr2=0x7fc0740d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7fa5f1d9b640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa5d406af40 0x7fa5d406d400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0d8b80 tx=0x7fa5e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7f56b77fe640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b81591d0 0x7f56b8157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.240+0000 7fc07359e640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0740b4e10 0x7fc0740d1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7fc07359e640 1 -- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0740cd4e0 msgr2=0x7fc0740cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7fc07359e640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0740cd4e0 0x7fc0740cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7fc07359e640 1 -- 172.21.15.70:0/2321596564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068009400 con 0x7fc07415cd70 2023-10-08T05:36:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7f56b6ffd640 1 -- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b81591d0 msgr2=0x7f56b8157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7f56b6ffd640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b81591d0 0x7f56b8157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7f56b6ffd640 1 -- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b80cede0 msgr2=0x7f56b80c1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7f56b6ffd640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b80cede0 0x7f56b80c1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7f56b6ffd640 1 -- 172.21.15.70:0/2750884771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac00ae30 con 0x7f56b80ca220 2023-10-08T05:36:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7fc07259c640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0740cd4e0 0x7fc0740cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:36:38.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7f64cbfff640 1 --2- 172.21.15.70:0/3350835419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d415d490 0x7f64d40caf00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009d90 tx=0x7f64c402e0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38428aeed0553318 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7f80287ce640 1 Processor -- start 2023-10-08T05:36:38.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7fc072d9d640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0740b4e10 0x7fc0740d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:38.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.241+0000 7f80287ce640 1 -- start start 2023-10-08T05:36:38.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.242+0000 7f56b6ffd640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b80ca220 0x7f56b81595b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56a800c410 tx=0x7f56a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.242+0000 7fc07359e640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07415cd70 0x7fc0741f8780 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc068033000 tx=0x7fc0680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.243+0000 7f80287ce640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f80200cfb70 0x7f802006afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.243+0000 7f80287ce640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80200d3750 con 0x7f80200d3250 2023-10-08T05:36:38.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.243+0000 7f8025d69640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f80200d3250 0x7f80200ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43726/0 (socket says 172.21.15.70:43726) 2023-10-08T05:36:38.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.243+0000 7f8025d69640 1 -- 172.21.15.70:0/3569010470 learned_addr learned my addr 172.21.15.70:0/3569010470 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.243+0000 7f80287ce640 1 -- 172.21.15.70:0/3569010470 --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80200cdb00 con 0x7f80200d0f10 2023-10-08T05:36:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.243+0000 7f80287ce640 1 -- 172.21.15.70:0/3569010470 --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200cdde0 con 0x7f80200cfb70 2023-10-08T05:36:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.243+0000 7f8026d6b640 1 --2- 172.21.15.70:0/3569010470 >> v2:172.21.15.181:3300/0 conn(0x7f80200cfb70 0x7f802006afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7f9c5affd640 1 -- 172.21.15.70:0/2635704721 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c06b2d0 con 0x7f9c640cd5c0 2023-10-08T05:36:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2635704721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c64051950 con 0x7f9c640cd5c0 2023-10-08T05:36:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7fa5e2ffd640 1 -- 172.21.15.70:0/283680561 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5e4077280 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2635704721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c64051f10 con 0x7f9c640cd5c0 2023-10-08T05:36:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7fc05b7fe640 1 -- 172.21.15.70:0/2321596564 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06803bc40 con 0x7fc07415cd70 2023-10-08T05:36:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7fc07921a640 1 -- 172.21.15.70:0/2321596564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0740cb2d0 con 0x7fc07415cd70 2023-10-08T05:36:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7fc07921a640 1 -- 172.21.15.70:0/2321596564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0740c7ae0 con 0x7fc07415cd70 2023-10-08T05:36:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7f8026d6b640 1 -- 172.21.15.70:0/3569010470 >> v1:172.21.15.181:6789/0 conn(0x7f80200d0f10 legacy=0x7f80200d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7f8026d6b640 1 -- 172.21.15.70:0/3569010470 >> v1:172.21.15.70:6789/0 conn(0x7f80200d3250 legacy=0x7f80200ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7f8026d6b640 1 -- 172.21.15.70:0/3569010470 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80200ce0c0 con 0x7f80200cfb70 2023-10-08T05:36:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7f9c5affd640 1 -- 172.21.15.70:0/2635704721 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c06b430 con 0x7f9c640cd5c0 2023-10-08T05:36:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7f9c5affd640 1 -- 172.21.15.70:0/2635704721 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c06b5a0 con 0x7f9c640cd5c0 2023-10-08T05:36:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.245+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2635704721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c50005250 con 0x7f9c640cd5c0 2023-10-08T05:36:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7fe6f4ff9640 1 -- 172.21.15.70:0/4138988124 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0011890 con 0x7fe6f80ca220 2023-10-08T05:36:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7fe6fe555640 1 -- 172.21.15.70:0/4138988124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f815a430 con 0x7fe6f80ca220 2023-10-08T05:36:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7fe6f4ff9640 1 -- 172.21.15.70:0/4138988124 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f00119f0 con 0x7fe6f80ca220 2023-10-08T05:36:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.244+0000 7fe6f4ff9640 1 -- 172.21.15.70:0/4138988124 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0011da0 con 0x7fe6f80ca220 2023-10-08T05:36:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.245+0000 7fe6fe555640 1 -- 172.21.15.70:0/4138988124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f80c80a0 con 0x7fe6f80ca220 2023-10-08T05:36:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.246+0000 7f8026d6b640 1 --2- 172.21.15.70:0/3569010470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80200cfb70 0x7f802006afb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8018009d90 tx=0x7f801802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a2cc0f377aeca93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.246+0000 7f569ffff640 1 -- 172.21.15.70:0/2750884771 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8005ce0 con 0x7f56b80ca220 2023-10-08T05:36:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.246+0000 7f569ffff640 1 -- 172.21.15.70:0/2750884771 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a8005e40 con 0x7f56b80ca220 2023-10-08T05:36:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.246+0000 7f569ffff640 1 -- 172.21.15.70:0/2750884771 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a80072f0 con 0x7f56b80ca220 2023-10-08T05:36:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.246+0000 7f56bd887640 1 -- 172.21.15.70:0/2750884771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b8140790 con 0x7f56b80ca220 2023-10-08T05:36:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.246+0000 7fc0597fa640 1 -- 172.21.15.70:0/2321596564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc054005250 con 0x7fc07415cd70 2023-10-08T05:36:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.246+0000 7f56bd887640 1 -- 172.21.15.70:0/2750884771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b8140d50 con 0x7f56b80ca220 2023-10-08T05:36:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.247+0000 7fc05b7fe640 1 -- 172.21.15.70:0/2321596564 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068031b00 con 0x7fc07415cd70 2023-10-08T05:36:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7fe6fe555640 1 -- 172.21.15.70:0/4138988124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f80cae10 con 0x7fe6f80ca220 2023-10-08T05:36:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.247+0000 7f9c5affd640 1 -- 172.21.15.70:0/2635704721 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9c6c069d50 con 0x7f9c640cd5c0 2023-10-08T05:36:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7fc05b7fe640 1 -- 172.21.15.70:0/2321596564 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068031d20 con 0x7fc07415cd70 2023-10-08T05:36:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7f8025568640 1 -- 172.21.15.70:0/3569010470 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801802cc90 con 0x7f80200cfb70 2023-10-08T05:36:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7f14a27fc640 1 -- 172.21.15.70:0/1436311010 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14ac0efd90 con 0x7f14a4050260 2023-10-08T05:36:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7f64d37fe640 1 -- 172.21.15.70:0/3350835419 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c402ee60 con 0x7f64d415d490 2023-10-08T05:36:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7f64d37fe640 1 -- 172.21.15.70:0/3350835419 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4036440 con 0x7f64d415d490 2023-10-08T05:36:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7f64d37fe640 1 -- 172.21.15.70:0/3350835419 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40367d0 con 0x7f64d415d490 2023-10-08T05:36:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7f8025568640 1 -- 172.21.15.70:0/3569010470 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801802cdf0 con 0x7f80200cfb70 2023-10-08T05:36:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7f56bd887640 1 -- 172.21.15.70:0/2750884771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b804e160 con 0x7f56b80ca220 2023-10-08T05:36:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7f8025568640 1 -- 172.21.15.70:0/3569010470 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018034660 con 0x7f80200cfb70 2023-10-08T05:36:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.248+0000 7f9c5affd640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9c4c06ae70 0x7f9c4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.249+0000 7f9c5affd640 1 -- 172.21.15.70:0/2635704721 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9c6c103210 con 0x7f9c640cd5c0 2023-10-08T05:36:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.249+0000 7f80287ce640 1 -- 172.21.15.70:0/3569010470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80200cfb70 msgr2=0x7f802006afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.249+0000 7f80287ce640 1 --2- 172.21.15.70:0/3569010470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80200cfb70 0x7f802006afb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8018009d90 tx=0x7f801802bee0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.249+0000 7f64d931a640 1 -- 172.21.15.70:0/3350835419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d415d490 msgr2=0x7f64d40caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.249+0000 7f64d931a640 1 --2- 172.21.15.70:0/3350835419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d415d490 0x7f64d40caf00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009d90 tx=0x7f64c402e0b0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.250+0000 7f9c6a59c640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9c4c06ae70 0x7f9c4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.250+0000 7f64d931a640 1 -- 172.21.15.70:0/3350835419 shutdown_connections 2023-10-08T05:36:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.250+0000 7f64d931a640 1 --2- 172.21.15.70:0/3350835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40c9c10 0x7f64d40cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.250+0000 7f64d931a640 1 --2- 172.21.15.70:0/3350835419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d415d490 0x7f64d40caf00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.250+0000 7f64d931a640 1 -- 172.21.15.70:0/3350835419 >> 172.21.15.70:0/3350835419 conn(0x7f64d406c050 msgr2=0x7f64d406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.251+0000 7f64d931a640 1 -- 172.21.15.70:0/3350835419 shutdown_connections 2023-10-08T05:36:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.251+0000 7f569ffff640 1 -- 172.21.15.70:0/2750884771 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f56a8007450 con 0x7f56b80ca220 2023-10-08T05:36:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.251+0000 7f64d931a640 1 -- 172.21.15.70:0/3350835419 wait complete. 2023-10-08T05:36:38.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.251+0000 7f80287ce640 1 -- 172.21.15.70:0/3569010470 shutdown_connections 2023-10-08T05:36:38.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.251+0000 7f80287ce640 1 --2- 172.21.15.70:0/3569010470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80200cfb70 0x7f802006afb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.251+0000 7f80287ce640 1 -- 172.21.15.70:0/3569010470 >> 172.21.15.70:0/3569010470 conn(0x7f802009bc60 msgr2=0x7f80200d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.252+0000 7f569ffff640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f56a406aef0 0x7f56a406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.250+0000 7fe6f4ff9640 1 -- 172.21.15.70:0/4138988124 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe6f001a400 con 0x7fe6f80ca220 2023-10-08T05:36:38.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.252+0000 7fe6f4ff9640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe6e406ae70 0x7fe6e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.252+0000 7f569ffff640 1 -- 172.21.15.70:0/2750884771 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f56a8088fa0 con 0x7f56b80ca220 2023-10-08T05:36:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.252+0000 7fe6f4ff9640 1 -- 172.21.15.70:0/4138988124 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe6f0089b90 con 0x7fe6f80ca220 2023-10-08T05:36:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.253+0000 7f56b67fc640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f56a406aef0 0x7f56a406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.253+0000 7f9c6a59c640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9c4c06ae70 0x7f9c4c06d330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c037000 tx=0x7f9c5c030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.253+0000 7f80287ce640 1 -- 172.21.15.70:0/3569010470 shutdown_connections 2023-10-08T05:36:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.253+0000 7fc05b7fe640 1 -- 172.21.15.70:0/2321596564 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc06804e450 con 0x7fc07415cd70 2023-10-08T05:36:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.253+0000 7f80287ce640 1 -- 172.21.15.70:0/3569010470 wait complete. 2023-10-08T05:36:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.254+0000 7f56b67fc640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f56a406aef0 0x7f56a406d3b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56ac039000 tx=0x7f56ac034000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.254+0000 7f64d931a640 1 Processor -- start 2023-10-08T05:36:38.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.255+0000 7f64d931a640 1 -- start start 2023-10-08T05:36:38.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.255+0000 7fc05b7fe640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc06006ae70 0x7fc06006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.255+0000 7fc05b7fe640 1 -- 172.21.15.70:0/2321596564 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc0680abe50 con 0x7fc07415cd70 2023-10-08T05:36:38.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.255+0000 7fc072d9d640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc06006ae70 0x7fc06006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.256+0000 7f80287ce640 1 Processor -- start 2023-10-08T05:36:38.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.256+0000 7fe6f7fff640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe6e406ae70 0x7fe6e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.256+0000 7f80287ce640 1 -- start start 2023-10-08T05:36:38.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.257+0000 7f64d931a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d40c9c10 0x7f64d413f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.257+0000 7f64d931a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d41530f0 0x7f64d40adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.257+0000 7fc072d9d640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc06006ae70 0x7fc06006d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc06c005c00 tx=0x7fc06c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.257+0000 7f9c5affd640 1 -- 172.21.15.70:0/2635704721 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c6c0ce060 con 0x7f9c640cd5c0 2023-10-08T05:36:38.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.257+0000 7fe6f7fff640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe6e406ae70 0x7fe6e406d330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec035000 tx=0x7fe6ec030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f80287ce640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80200cfb70 0x7f80201558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d40c9c10 0x7f64d413f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f80287ce640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80200d0f10 0x7f8020155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f64d3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d40c9c10 0x7f64d413f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39826/0 (socket says 172.21.15.70:39826) 2023-10-08T05:36:38.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f80287ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80200d3250 0x7f8020157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f80287ce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80201e94f0 con 0x7f80200d3250 2023-10-08T05:36:38.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f80287ce640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020156970 con 0x7f80200cfb70 2023-10-08T05:36:38.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f80287ce640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8020156c50 con 0x7f80200d0f10 2023-10-08T05:36:38.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f64cbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d41530f0 0x7f64d40adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f8026d6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80200d3250 0x7f8020157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f8026d6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80200d3250 0x7f8020157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60052/0 (socket says 172.21.15.70:60052) 2023-10-08T05:36:38.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f8026d6b640 1 -- 172.21.15.70:0/4181263073 learned_addr learned my addr 172.21.15.70:0/4181263073 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.258+0000 7f64cbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d41530f0 0x7f64d40adf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60046/0 (socket says 172.21.15.70:60046) 2023-10-08T05:36:38.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f64d3fff640 1 -- 172.21.15.70:0/1755674521 learned_addr learned my addr 172.21.15.70:0/1755674521 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:38.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f8026d6b640 1 -- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80200d0f10 msgr2=0x7f8020155340 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T05:36:38.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f64d931a640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40ae490 0x7f64d40c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f8026d6b640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80200d0f10 0x7f8020155340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f8026d6b640 1 -- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80200cfb70 msgr2=0x7f80201558f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T05:36:38.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f8026d6b640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80200cfb70 0x7f80201558f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f8026d6b640 1 -- 172.21.15.70:0/4181263073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8018009a70 con 0x7f80200d3250 2023-10-08T05:36:38.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40c4e90 con 0x7f64d41530f0 2023-10-08T05:36:38.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40c4630 con 0x7f64d40c9c10 2023-10-08T05:36:38.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f8026d6b640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80200d3250 0x7f8020157730 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8018009d90 tx=0x7f8018004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d413fce0 con 0x7f64d40ae490 2023-10-08T05:36:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.259+0000 7f64d8b19640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40ae490 0x7f64d40c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.260+0000 7f64d8b19640 1 -- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d40c9c10 msgr2=0x7f64d413f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.260+0000 7f64d8b19640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d40c9c10 0x7f64d413f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.260+0000 7f64d8b19640 1 -- 172.21.15.70:0/1755674521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d41530f0 msgr2=0x7f64d40adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.260+0000 7f64d8b19640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d41530f0 0x7f64d40adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.261+0000 7f64d8b19640 1 -- 172.21.15.70:0/1755674521 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c4009a70 con 0x7f64d40ae490 2023-10-08T05:36:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.261+0000 7f64d8b19640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40ae490 0x7f64d40c2a10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00d5a0 tx=0x7f64cc00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.262+0000 7f64d17fa640 1 -- 172.21.15.70:0/1755674521 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc004030 con 0x7f64d40ae490 2023-10-08T05:36:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.262+0000 7f64d17fa640 1 -- 172.21.15.70:0/1755674521 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc004190 con 0x7f64d40ae490 2023-10-08T05:36:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.262+0000 7f64d17fa640 1 -- 172.21.15.70:0/1755674521 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc004030 con 0x7f64d40ae490 2023-10-08T05:36:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.262+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d40c48d0 con 0x7f64d40ae490 2023-10-08T05:36:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.263+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d4157f10 con 0x7f64d40ae490 2023-10-08T05:36:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.264+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d4074150 con 0x7f64d40ae490 2023-10-08T05:36:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.264+0000 7fc05b7fe640 1 -- 172.21.15.70:0/2321596564 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc068076da0 con 0x7fc07415cd70 2023-10-08T05:36:38.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.267+0000 7f64d17fa640 1 -- 172.21.15.70:0/1755674521 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f64cc004300 con 0x7f64d40ae490 2023-10-08T05:36:38.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.268+0000 7f64d17fa640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f64b406af60 0x7f64b406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.268+0000 7f64d17fa640 1 -- 172.21.15.70:0/1755674521 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f64cc0890a0 con 0x7f64d40ae490 2023-10-08T05:36:38.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.269+0000 7f64d3fff640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f64b406af60 0x7f64b406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.270+0000 7f64d3fff640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f64b406af60 0x7f64b406d420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f64bc004e40 tx=0x7f64bc007f10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.272+0000 7f569ffff640 1 -- 172.21.15.70:0/2750884771 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56a8053e20 con 0x7f56b80ca220 2023-10-08T05:36:38.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.272+0000 7f8016ffd640 1 -- 172.21.15.70:0/4181263073 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801803e610 con 0x7f80200d3250 2023-10-08T05:36:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.273+0000 7f80287ce640 1 -- 172.21.15.70:0/4181263073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8020156f30 con 0x7f80200d3250 2023-10-08T05:36:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.273+0000 7f80287ce640 1 -- 172.21.15.70:0/4181263073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8020155f90 con 0x7f80200d3250 2023-10-08T05:36:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.274+0000 7f80287ce640 1 -- 172.21.15.70:0/4181263073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80201561e0 con 0x7f80200d3250 2023-10-08T05:36:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.281+0000 7f8016ffd640 1 -- 172.21.15.70:0/4181263073 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801803e770 con 0x7f80200d3250 2023-10-08T05:36:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.281+0000 7f8016ffd640 1 -- 172.21.15.70:0/4181263073 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801803eae0 con 0x7f80200d3250 2023-10-08T05:36:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.281+0000 7fe6f4ff9640 1 -- 172.21.15.70:0/4138988124 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6f0054940 con 0x7fe6f80ca220 2023-10-08T05:36:38.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.283+0000 7f64d17fa640 1 -- 172.21.15.70:0/1755674521 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64cc0579b0 con 0x7f64d40ae490 2023-10-08T05:36:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.283+0000 7f8016ffd640 1 -- 172.21.15.70:0/4181263073 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f801803b050 con 0x7f80200d3250 2023-10-08T05:36:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.285+0000 7f8016ffd640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f800406ae70 0x7f800406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.285+0000 7f8016ffd640 1 -- 172.21.15.70:0/4181263073 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f80180ac370 con 0x7f80200d3250 2023-10-08T05:36:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.288+0000 7f802656a640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f800406ae70 0x7f800406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.290+0000 7f8016ffd640 1 -- 172.21.15.70:0/4181263073 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80180771a0 con 0x7f80200d3250 2023-10-08T05:36:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.301+0000 7f802656a640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f800406ae70 0x7f800406d330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f801c000f60 tx=0x7f801c0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:38.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.662+0000 7f14b3bb6640 1 -- 172.21.15.70:0/1436311010 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f14a4052d30 con 0x7f14a4050260 2023-10-08T05:36:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.664+0000 7f14a27fc640 1 -- 172.21.15.70:0/1436311010 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14ac0ba090 con 0x7f14a4050260 2023-10-08T05:36:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.664+0000 7fa5f359e640 1 -- 172.21.15.70:0/283680561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa5ec03a6e0 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.666 INFO:teuthology.orchestra.run.smithi070.stdout:55834574855 2023-10-08T05:36:38.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.665+0000 7fa5e2ffd640 1 -- 172.21.15.70:0/283680561 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa5e407a8a0 con 0x7fa5ec0e78b0 2023-10-08T05:36:38.666 INFO:teuthology.orchestra.run.smithi070.stdout:55834574855 2023-10-08T05:36:38.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.667+0000 7f148ffff640 1 -- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f149006aec0 msgr2=0x7f149006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.668+0000 7f148ffff640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f149006aec0 0x7f149006d380 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f14a4006c60 tx=0x7f149c009110 comp rx=0 tx=0).stop 2023-10-08T05:36:38.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.668+0000 7f148ffff640 1 -- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4050260 msgr2=0x7f14a404d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.668+0000 7f148ffff640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4050260 0x7f14a404d4d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f14ac02cb40 tx=0x7f14ac066bf0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.668+0000 7f148ffff640 1 -- 172.21.15.70:0/1436311010 shutdown_connections 2023-10-08T05:36:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.668+0000 7f148ffff640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f149006aec0 0x7f149006d380 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.668+0000 7f148ffff640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14a404da10 0x7f14a404f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.668+0000 7f148ffff640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14a40e3380 0x7f14a404ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.668+0000 7f148ffff640 1 --2- 172.21.15.70:0/1436311010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14a4050260 0x7f14a404d4d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.668+0000 7f148ffff640 1 -- 172.21.15.70:0/1436311010 >> 172.21.15.70:0/1436311010 conn(0x7f14a4054c40 msgr2=0x7f14a40dfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.668+0000 7f148ffff640 1 -- 172.21.15.70:0/1436311010 shutdown_connections 2023-10-08T05:36:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.669+0000 7f148ffff640 1 -- 172.21.15.70:0/1436311010 wait complete. 2023-10-08T05:36:38.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.669+0000 7fa5e0ff9640 1 -- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa5d406af40 msgr2=0x7fa5d406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.669+0000 7fa5e0ff9640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa5d406af40 0x7fa5d406d400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec0d8b80 tx=0x7fa5e8009040 comp rx=0 tx=0).stop 2023-10-08T05:36:38.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.669+0000 7fa5e0ff9640 1 -- 172.21.15.70:0/283680561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 msgr2=0x7fa5ec0d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.669+0000 7fa5e0ff9640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 0x7fa5ec0d1c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009990 tx=0x7fa5e4004730 comp rx=0 tx=0).stop 2023-10-08T05:36:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.669+0000 7fa5e0ff9640 1 -- 172.21.15.70:0/283680561 shutdown_connections 2023-10-08T05:36:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.669+0000 7fa5e0ff9640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa5d406af40 0x7fa5d406d400 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.669+0000 7fa5e0ff9640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5ec0eb700 0x7fa5ec04c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.669+0000 7fa5e0ff9640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5ec0eb2c0 0x7fa5ec04c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.669+0000 7fa5e0ff9640 1 --2- 172.21.15.70:0/283680561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5ec0e78b0 0x7fa5ec0d1c30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.670+0000 7fa5e0ff9640 1 -- 172.21.15.70:0/283680561 >> 172.21.15.70:0/283680561 conn(0x7fa5ec054c10 msgr2=0x7fa5ec0e3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.670+0000 7fa5e0ff9640 1 -- 172.21.15.70:0/283680561 shutdown_connections 2023-10-08T05:36:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.670+0000 7fa5e0ff9640 1 -- 172.21.15.70:0/283680561 wait complete. 2023-10-08T05:36:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.672+0000 7f9c6b59e640 1 -- 172.21.15.70:0/2635704721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9c500057d0 con 0x7f9c640cd5c0 2023-10-08T05:36:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.673+0000 7f9c5affd640 1 -- 172.21.15.70:0/2635704721 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9c6c0d1680 con 0x7f9c640cd5c0 2023-10-08T05:36:38.675 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:38.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.676+0000 7fc0597fa640 1 -- 172.21.15.70:0/2321596564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc0540057d0 con 0x7fc07415cd70 2023-10-08T05:36:38.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.677+0000 7fc05b7fe640 1 -- 172.21.15.70:0/2321596564 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc068035020 con 0x7fc07415cd70 2023-10-08T05:36:38.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.677+0000 7f9c58ff9640 1 -- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9c4c06ae70 msgr2=0x7f9c4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.679 INFO:teuthology.orchestra.run.smithi070.stdout:55834574855 2023-10-08T05:36:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.677+0000 7f9c58ff9640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9c4c06ae70 0x7f9c4c06d330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c037000 tx=0x7f9c5c030000 comp rx=0 tx=0).stop 2023-10-08T05:36:38.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.677+0000 7f9c58ff9640 1 -- 172.21.15.70:0/2635704721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c640cd5c0 msgr2=0x7f9c6403e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.678+0000 7f9c58ff9640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c640cd5c0 0x7f9c6403e670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c02cb70 tx=0x7f9c6c0734e0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.678+0000 7f9c58ff9640 1 -- 172.21.15.70:0/2635704721 shutdown_connections 2023-10-08T05:36:38.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.678+0000 7f9c58ff9640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9c4c06ae70 0x7f9c4c06d330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.678+0000 7f9c58ff9640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c640529f0 0x7f9c6403c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.679+0000 7f9c58ff9640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c640cd5c0 0x7f9c6403e670 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.679+0000 7f9c58ff9640 1 --2- 172.21.15.70:0/2635704721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c540039a0 0x7f9c64051300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.679+0000 7f9c58ff9640 1 -- 172.21.15.70:0/2635704721 >> 172.21.15.70:0/2635704721 conn(0x7f9c640590c0 msgr2=0x7f9c640e0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.679+0000 7f9c58ff9640 1 -- 172.21.15.70:0/2635704721 shutdown_connections 2023-10-08T05:36:38.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.680+0000 7f9c58ff9640 1 -- 172.21.15.70:0/2635704721 wait complete. 2023-10-08T05:36:38.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.680+0000 7fc0597fa640 1 -- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc06006ae70 msgr2=0x7fc06006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.680+0000 7fc0597fa640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc06006ae70 0x7fc06006d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc06c005c00 tx=0x7fc06c009040 comp rx=0 tx=0).stop 2023-10-08T05:36:38.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.681+0000 7fc0597fa640 1 -- 172.21.15.70:0/2321596564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07415cd70 msgr2=0x7fc0741f8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.681+0000 7fc0597fa640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07415cd70 0x7fc0741f8780 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc068033000 tx=0x7fc0680046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.681+0000 7fc0597fa640 1 -- 172.21.15.70:0/2321596564 shutdown_connections 2023-10-08T05:36:38.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.681+0000 7fc0597fa640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc06006ae70 0x7fc06006d330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.682+0000 7fc0597fa640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc07415cd70 0x7fc0741f8780 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.682+0000 7fc0597fa640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0740cd4e0 0x7fc0740cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.682+0000 7fc0597fa640 1 --2- 172.21.15.70:0/2321596564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0740b4e10 0x7fc0740d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.682+0000 7fc0597fa640 1 -- 172.21.15.70:0/2321596564 >> 172.21.15.70:0/2321596564 conn(0x7fc0740cf700 msgr2=0x7fc0740b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.683+0000 7fc0597fa640 1 -- 172.21.15.70:0/2321596564 shutdown_connections 2023-10-08T05:36:38.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.684+0000 7fc0597fa640 1 -- 172.21.15.70:0/2321596564 wait complete. 2023-10-08T05:36:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.693+0000 7fe6fe555640 1 -- 172.21.15.70:0/4138988124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe6f804e160 con 0x7fe6f80ca220 2023-10-08T05:36:38.697 INFO:teuthology.orchestra.run.smithi070.stdout:55834574855 2023-10-08T05:36:38.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.694+0000 7fe6f4ff9640 1 -- 172.21.15.70:0/4138988124 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe6f0015020 con 0x7fe6f80ca220 2023-10-08T05:36:38.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.696+0000 7f56bd887640 1 -- 172.21.15.70:0/2750884771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f56b806acf0 con 0x7f56b80ca220 2023-10-08T05:36:38.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.699+0000 7f569ffff640 1 -- 172.21.15.70:0/2750884771 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f56a8017020 con 0x7f56b80ca220 2023-10-08T05:36:38.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.699+0000 7f80287ce640 1 -- 172.21.15.70:0/4181263073 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f80200cecd0 con 0x7f80200d3250 2023-10-08T05:36:38.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2023-10-08T05:36:38.702 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:38.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.699+0000 7f8016ffd640 1 -- 172.21.15.70:0/4181263073 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8018039020 con 0x7f80200d3250 2023-10-08T05:36:38.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2023-10-08T05:36:38.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.700+0000 7fe6fe555640 1 -- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe6e406ae70 msgr2=0x7fe6e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.700+0000 7fe6fe555640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe6e406ae70 0x7fe6e406d330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec035000 tx=0x7fe6ec030000 comp rx=0 tx=0).stop 2023-10-08T05:36:38.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.700+0000 7fe6fe555640 1 -- 172.21.15.70:0/4138988124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f80ca220 msgr2=0x7fe6f8159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.704 INFO:teuthology.orchestra.run.smithi070.stdout:55834574857 2023-10-08T05:36:38.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.700+0000 7fe6fe555640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f80ca220 0x7fe6f8159690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000b470 tx=0x7fe6f000b930 comp rx=0 tx=0).stop 2023-10-08T05:36:38.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.700+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f64d40b4470 con 0x7f64d40ae490 2023-10-08T05:36:38.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.700+0000 7fe6fe555640 1 -- 172.21.15.70:0/4138988124 shutdown_connections 2023-10-08T05:36:38.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.700+0000 7fe6fe555640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe6e406ae70 0x7fe6e406d330 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.700+0000 7fe6fe555640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6f80cede0 0x7fe6f80b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.700+0000 7fe6fe555640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6f80ca220 0x7fe6f8159690 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.700+0000 7fe6fe555640 1 --2- 172.21.15.70:0/4138988124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6f806a460 0x7fe6f8153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.701+0000 7fe6fe555640 1 -- 172.21.15.70:0/4138988124 >> 172.21.15.70:0/4138988124 conn(0x7fe6f80660e0 msgr2=0x7fe6f80c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.701+0000 7fe6fe555640 1 -- 172.21.15.70:0/4138988124 shutdown_connections 2023-10-08T05:36:38.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.701+0000 7fe6fe555640 1 -- 172.21.15.70:0/4138988124 wait complete. 2023-10-08T05:36:38.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.701+0000 7f64d17fa640 1 -- 172.21.15.70:0/1755674521 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64cc0579b0 con 0x7f64d40ae490 2023-10-08T05:36:38.707 INFO:teuthology.orchestra.run.smithi070.stdout:55834574855 2023-10-08T05:36:38.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.702+0000 7f80287ce640 1 -- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f800406ae70 msgr2=0x7f800406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.702+0000 7f80287ce640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f800406ae70 0x7f800406d330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f801c000f60 tx=0x7f801c0079b0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.702+0000 7f80287ce640 1 -- 172.21.15.70:0/4181263073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80200d3250 msgr2=0x7f8020157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.702+0000 7f80287ce640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80200d3250 0x7f8020157730 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8018009d90 tx=0x7f8018004a40 comp rx=0 tx=0).stop 2023-10-08T05:36:38.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f80287ce640 1 -- 172.21.15.70:0/4181263073 shutdown_connections 2023-10-08T05:36:38.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f80287ce640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f800406ae70 0x7f800406d330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f80287ce640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80200d3250 0x7f8020157730 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f80287ce640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80200d0f10 0x7f8020155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f80287ce640 1 --2- 172.21.15.70:0/4181263073 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80200cfb70 0x7f80201558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f80287ce640 1 -- 172.21.15.70:0/4181263073 >> 172.21.15.70:0/4181263073 conn(0x7f802009bc60 msgr2=0x7f80200c6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f80287ce640 1 -- 172.21.15.70:0/4181263073 shutdown_connections 2023-10-08T05:36:38.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f56bd887640 1 -- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f56a406aef0 msgr2=0x7f56a406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f56bd887640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f56a406aef0 0x7f56a406d3b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f56ac039000 tx=0x7f56ac034000 comp rx=0 tx=0).stop 2023-10-08T05:36:38.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f56bd887640 1 -- 172.21.15.70:0/2750884771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b80ca220 msgr2=0x7f56b81595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f56bd887640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b80ca220 0x7f56b81595b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56a800c410 tx=0x7f56a800c8d0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f56bd887640 1 -- 172.21.15.70:0/2750884771 shutdown_connections 2023-10-08T05:36:38.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f56bd887640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f56a406aef0 0x7f56a406d3b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f56bd887640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56b81591d0 0x7f56b8157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f56bd887640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b80cede0 0x7f56b80c1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f56bd887640 1 --2- 172.21.15.70:0/2750884771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b80ca220 0x7f56b81595b0 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56a800c410 tx=0x7f56a800c8d0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f56bd887640 1 -- 172.21.15.70:0/2750884771 >> 172.21.15.70:0/2750884771 conn(0x7f56b80660e0 msgr2=0x7f56b80cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.703+0000 7f80287ce640 1 -- 172.21.15.70:0/4181263073 wait complete. 2023-10-08T05:36:38.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f56bd887640 1 -- 172.21.15.70:0/2750884771 shutdown_connections 2023-10-08T05:36:38.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f56bd887640 1 -- 172.21.15.70:0/2750884771 wait complete. 2023-10-08T05:36:38.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f64b406af60 msgr2=0x7f64b406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f64d931a640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f64b406af60 0x7f64b406d420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f64bc004e40 tx=0x7f64bc007f10 comp rx=0 tx=0).stop 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40ae490 msgr2=0x7f64d40c2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f64d931a640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40ae490 0x7f64d40c2a10 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00d5a0 tx=0x7f64cc00da60 comp rx=0 tx=0).stop 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.704+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 shutdown_connections 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.705+0000 7f64d931a640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f64b406af60 0x7f64b406d420 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.705+0000 7f64d931a640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64d40ae490 0x7f64d40c2a10 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.705+0000 7f64d931a640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d41530f0 0x7f64d40adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.705+0000 7f64d931a640 1 --2- 172.21.15.70:0/1755674521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64d40c9c10 0x7f64d413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.705+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 >> 172.21.15.70:0/1755674521 conn(0x7f64d406c050 msgr2=0x7f64d4152720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.705+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 shutdown_connections 2023-10-08T05:36:38.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:38.705+0000 7f64d931a640 1 -- 172.21.15.70:0/1755674521 wait complete. 2023-10-08T05:36:38.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-10-08T05:36:38.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-10-08T05:36:38.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2023-10-08T05:36:38.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2023-10-08T05:36:38.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.4 2023-10-08T05:36:38.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2023-10-08T05:36:39.703 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-08T05:36:39.704 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-08T05:36:39.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-08T05:36:39.715 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-08T05:36:39.726 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-08T05:36:39.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-08T05:36:39.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-08T05:36:39.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-08T05:36:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.969+0000 7ff98eba7640 1 Processor -- start 2023-10-08T05:36:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.969+0000 7ff98eba7640 1 -- start start 2023-10-08T05:36:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.971+0000 7ff98eba7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9880b1b90 0x7ff98806afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.971+0000 7ff98eba7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff9880c5910 0x7ff9880d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.971+0000 7ff98eba7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1f70 0x7ff9880cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.971+0000 7ff98eba7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9880d33d0 con 0x7ff9880b1b90 2023-10-08T05:36:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.971+0000 7ff98eba7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9880d3670 con 0x7ff9880c5910 2023-10-08T05:36:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.971+0000 7ff98eba7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880cd870 con 0x7ff9880b1f70 2023-10-08T05:36:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.972+0000 7ff98e3a6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1f70 0x7ff9880cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.972+0000 7ff98e3a6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1f70 0x7ff9880cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:60790/0 (socket says 172.21.15.70:60790) 2023-10-08T05:36:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.972+0000 7ff98e3a6640 1 -- 172.21.15.70:0/3253271642 learned_addr learned my addr 172.21.15.70:0/3253271642 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.972+0000 7ff98d3a4640 1 --2- 172.21.15.70:0/3253271642 >> v2:172.21.15.181:3300/0 conn(0x7ff9880c5910 0x7ff9880d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.973+0000 7ff98e3a6640 1 -- 172.21.15.70:0/3253271642 >> v2:172.21.15.181:3300/0 conn(0x7ff9880c5910 msgr2=0x7ff9880d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.973+0000 7ff98e3a6640 1 --2- 172.21.15.70:0/3253271642 >> v2:172.21.15.181:3300/0 conn(0x7ff9880c5910 0x7ff9880d12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.973+0000 7ff98e3a6640 1 -- 172.21.15.70:0/3253271642 >> v2:172.21.15.70:3300/0 conn(0x7ff9880b1b90 msgr2=0x7ff98806afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.973+0000 7ff98dba5640 1 --2- 172.21.15.70:0/3253271642 >> v2:172.21.15.70:3300/0 conn(0x7ff9880b1b90 0x7ff98806afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.973+0000 7ff98e3a6640 1 --2- 172.21.15.70:0/3253271642 >> v2:172.21.15.70:3300/0 conn(0x7ff9880b1b90 0x7ff98806afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.973+0000 7ff98e3a6640 1 -- 172.21.15.70:0/3253271642 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9880cdb50 con 0x7ff9880b1f70 2023-10-08T05:36:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.975+0000 7ff98dba5640 1 --2- 172.21.15.70:0/3253271642 >> v2:172.21.15.70:3300/0 conn(0x7ff9880b1b90 0x7ff98806afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.976+0000 7ff98e3a6640 1 --2- 172.21.15.70:0/3253271642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1f70 0x7ff9880cf300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff9880d28c0 tx=0x7ff98004df10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fef7cba880e83f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.977+0000 7ff98cba3640 1 -- 172.21.15.70:0/3253271642 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98004eab0 con 0x7ff9880b1f70 2023-10-08T05:36:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.977+0000 7ff98cba3640 1 -- 172.21.15.70:0/3253271642 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98004ec10 con 0x7ff9880b1f70 2023-10-08T05:36:39.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.978+0000 7ff98eba7640 1 -- 172.21.15.70:0/3253271642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1f70 msgr2=0x7ff9880cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:39.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.978+0000 7ff98eba7640 1 --2- 172.21.15.70:0/3253271642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1f70 0x7ff9880cf300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff9880d28c0 tx=0x7ff98004df10 comp rx=0 tx=0).stop 2023-10-08T05:36:39.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.978+0000 7ff98eba7640 1 -- 172.21.15.70:0/3253271642 shutdown_connections 2023-10-08T05:36:39.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.978+0000 7ff98eba7640 1 --2- 172.21.15.70:0/3253271642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1f70 0x7ff9880cf300 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:39.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.978+0000 7ff98eba7640 1 --2- 172.21.15.70:0/3253271642 >> v2:172.21.15.181:3300/0 conn(0x7ff9880c5910 0x7ff9880d12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:39.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.978+0000 7ff98eba7640 1 --2- 172.21.15.70:0/3253271642 >> v2:172.21.15.70:3300/0 conn(0x7ff9880b1b90 0x7ff98806afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:39.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.978+0000 7ff98eba7640 1 -- 172.21.15.70:0/3253271642 >> 172.21.15.70:0/3253271642 conn(0x7ff98809bc40 msgr2=0x7ff9880d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:39.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.978+0000 7ff98eba7640 1 -- 172.21.15.70:0/3253271642 shutdown_connections 2023-10-08T05:36:39.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.978+0000 7ff98eba7640 1 -- 172.21.15.70:0/3253271642 wait complete. 2023-10-08T05:36:39.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.981+0000 7ff98eba7640 1 Processor -- start 2023-10-08T05:36:39.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.982+0000 7ff98eba7640 1 -- start start 2023-10-08T05:36:39.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.983+0000 7ff98eba7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1b90 0x7ff98815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:39.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.983+0000 7ff98eba7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9880b1f70 0x7ff9880cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:39.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.984+0000 7ff98d3a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9880b1f70 0x7ff9880cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:39.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.984+0000 7ff98d3a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9880b1f70 0x7ff9880cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60066/0 (socket says 172.21.15.70:60066) 2023-10-08T05:36:39.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.984+0000 7ff98dba5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1b90 0x7ff98815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:39.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.984+0000 7ff98d3a4640 1 -- 172.21.15.70:0/3343832637 learned_addr learned my addr 172.21.15.70:0/3343832637 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:39.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.984+0000 7ff98eba7640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9880c5910 0x7ff9880cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:39.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.984+0000 7ff98eba7640 1 -- 172.21.15.70:0/3343832637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880c4d80 con 0x7ff9880b1f70 2023-10-08T05:36:39.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.984+0000 7ff98eba7640 1 -- 172.21.15.70:0/3343832637 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880c9e60 con 0x7ff9880c5910 2023-10-08T05:36:39.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.984+0000 7ff98eba7640 1 -- 172.21.15.70:0/3343832637 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880c52b0 con 0x7ff9880b1b90 2023-10-08T05:36:39.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.985+0000 7ff98e3a6640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9880c5910 0x7ff9880cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:39.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.985+0000 7ff98d3a4640 1 -- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1b90 msgr2=0x7ff98815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:39.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.985+0000 7ff98d3a4640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1b90 0x7ff98815ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:39.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.985+0000 7ff98d3a4640 1 -- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9880c5910 msgr2=0x7ff9880cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:39.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.985+0000 7ff98d3a4640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9880c5910 0x7ff9880cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:39.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.986+0000 7ff98d3a4640 1 -- 172.21.15.70:0/3343832637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff980009b30 con 0x7ff9880b1f70 2023-10-08T05:36:39.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.986+0000 7ff98d3a4640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9880b1f70 0x7ff9880cb7c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff98400ea10 tx=0x7ff98400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.987+0000 7ff97e7fc640 1 -- 172.21.15.70:0/3343832637 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400c850 con 0x7ff9880b1f70 2023-10-08T05:36:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.987+0000 7ff97e7fc640 1 -- 172.21.15.70:0/3343832637 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98400c9b0 con 0x7ff9880b1f70 2023-10-08T05:36:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.987+0000 7ff97e7fc640 1 -- 172.21.15.70:0/3343832637 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400cd60 con 0x7ff9880b1f70 2023-10-08T05:36:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.987+0000 7ff98eba7640 1 -- 172.21.15.70:0/3343832637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9880c4fd0 con 0x7ff9880b1f70 2023-10-08T05:36:39.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.990+0000 7f102f59e640 1 Processor -- start 2023-10-08T05:36:39.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.990+0000 7f102f59e640 1 -- start start 2023-10-08T05:36:39.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.991+0000 7ff97e7fc640 1 -- 172.21.15.70:0/3343832637 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff984013470 con 0x7ff9880b1f70 2023-10-08T05:36:39.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.992+0000 7ff97e7fc640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff96c06aef0 0x7ff96c06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:39.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.992+0000 7ff98eba7640 1 -- 172.21.15.70:0/3343832637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98815b1b0 con 0x7ff9880b1f70 2023-10-08T05:36:39.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.993+0000 7ff97e7fc640 1 -- 172.21.15.70:0/3343832637 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff984057b20 con 0x7ff9880b1f70 2023-10-08T05:36:39.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.993+0000 7ff98dba5640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff96c06aef0 0x7ff96c06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:39.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.992+0000 7f102f59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1028005fe0 0x7f1028007580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:39.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.992+0000 7f102f59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280064e0 con 0x7f10280df500 2023-10-08T05:36:39.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.992+0000 7f102f59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280067c0 con 0x7f1028050380 2023-10-08T05:36:39.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.992+0000 7f102f59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10280df8e0 con 0x7f1028005fe0 2023-10-08T05:36:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.994+0000 7f102e59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f10280df500 0x7f102804fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43734/0 (socket says 172.21.15.70:43734) 2023-10-08T05:36:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.994+0000 7f102e59c640 1 -- 172.21.15.70:0/3305086240 learned_addr learned my addr 172.21.15.70:0/3305086240 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.995+0000 7f102ed9d640 1 --2- 172.21.15.70:0/3305086240 >> v2:172.21.15.181:3300/0 conn(0x7f1028005fe0 0x7f1028007580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.995+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539095686 0 0) 0x7f10280064e0 con 0x7f10280df500 2023-10-08T05:36:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.995+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1018003440 con 0x7f10280df500 2023-10-08T05:36:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.995+0000 7ff98eba7640 1 -- 172.21.15.70:0/3343832637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98815b340 con 0x7ff9880b1f70 2023-10-08T05:36:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.996+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1694292811 0 0) 0x7f1018003440 con 0x7f10280df500 2023-10-08T05:36:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.996+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 >> v2:172.21.15.181:3300/0 conn(0x7f1028005fe0 msgr2=0x7f1028007580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:39.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.996+0000 7f102d59a640 1 --2- 172.21.15.70:0/3305086240 >> v2:172.21.15.181:3300/0 conn(0x7f1028005fe0 0x7f1028007580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:39.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.996+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 >> v1:172.21.15.181:6789/0 conn(0x7f1028050380 legacy=0x7f10280381d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.996+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028006aa0 con 0x7f10280df500 2023-10-08T05:36:40.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.996+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1020004cd0 con 0x7f10280df500 2023-10-08T05:36:40.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.996+0000 7f102ed9d640 1 --2- 172.21.15.70:0/3305086240 >> v2:172.21.15.181:3300/0 conn(0x7f1028005fe0 0x7f1028007580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:40.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.997+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 >> v1:172.21.15.70:6789/0 conn(0x7f10280df500 legacy=0x7f102804fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.997+0000 7f102d59a640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1018003a60 0x7f1018003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.998+0000 7f102e59c640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1018003a60 0x7f1018003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.998+0000 7f102d59a640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1018008850 0x7f101800ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.998+0000 7f102d59a640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101800b460 0x7f101800d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.998+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280df8e0 con 0x7f1018003a60 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.998+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101800e160 con 0x7f101800b460 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.998+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101800e320 con 0x7f1018008850 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.999+0000 7f102dd9b640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1018008850 0x7f101800ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.999+0000 7f102ed9d640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101800b460 0x7f101800d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.999+0000 7f102dd9b640 1 -- 172.21.15.70:0/3305086240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101800b460 msgr2=0x7f101800d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.999+0000 7f102dd9b640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f101800b460 0x7f101800d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:39.999+0000 7f102dd9b640 1 -- 172.21.15.70:0/3305086240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1018003a60 msgr2=0x7f1018003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.000+0000 7f102dd9b640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1018003a60 0x7f1018003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.000+0000 7f102dd9b640 1 -- 172.21.15.70:0/3305086240 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1018003440 con 0x7f1018008850 2023-10-08T05:36:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.001+0000 7f102ed9d640 1 -- 172.21.15.70:0/3305086240 reap_dead start 2023-10-08T05:36:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.001+0000 7f102e59c640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1018003a60 0x7f1018003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:36:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.001+0000 7f102dd9b640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1018008850 0x7f101800ac60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f103004fa60 tx=0x7f10180036f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f2b30393c2ad5c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.002+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10300ac600 con 0x7f1018008850 2023-10-08T05:36:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.002+0000 7f102d59a640 1 -- 172.21.15.70:0/3305086240 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10300a8050 con 0x7f1018008850 2023-10-08T05:36:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.003+0000 7f102f59e640 1 -- 172.21.15.70:0/3305086240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1018008850 msgr2=0x7f101800ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.003+0000 7f102f59e640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1018008850 0x7f101800ac60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f103004fa60 tx=0x7f10180036f0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.004+0000 7f102f59e640 1 -- 172.21.15.70:0/3305086240 shutdown_connections 2023-10-08T05:36:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.004+0000 7f102f59e640 1 --2- 172.21.15.70:0/3305086240 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1018008850 0x7f101800ac60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.004+0000 7f102f59e640 1 -- 172.21.15.70:0/3305086240 >> 172.21.15.70:0/3305086240 conn(0x7f1028054c10 msgr2=0x7f1028055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.004+0000 7f102f59e640 1 -- 172.21.15.70:0/3305086240 shutdown_connections 2023-10-08T05:36:40.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.004+0000 7f102f59e640 1 -- 172.21.15.70:0/3305086240 wait complete. 2023-10-08T05:36:40.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.005+0000 7ff98dba5640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff96c06aef0 0x7ff96c06d3b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff978006b10 tx=0x7ff978008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.006+0000 7ff97e7fc640 1 -- 172.21.15.70:0/3343832637 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff984017020 con 0x7ff9880b1f70 2023-10-08T05:36:40.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.006+0000 7fed79d5a640 1 Processor -- start 2023-10-08T05:36:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.007+0000 7fed79d5a640 1 -- start start 2023-10-08T05:36:40.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.009+0000 7fed79d5a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed74161690 0x7fed7406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.009+0000 7fed79d5a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed740cb150 0x7fed740cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.009+0000 7fed79d5a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed740702b0 0x7fed74070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.009+0000 7fed79d5a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed740cbb40 con 0x7fed740cb150 2023-10-08T05:36:40.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.009+0000 7fed79d5a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7406cc50 con 0x7fed74161690 2023-10-08T05:36:40.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.009+0000 7fed79d5a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7406cf30 con 0x7fed740702b0 2023-10-08T05:36:40.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.010+0000 7fed737fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed74161690 0x7fed7406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.010+0000 7fed737fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed74161690 0x7fed7406dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39858/0 (socket says 172.21.15.70:39858) 2023-10-08T05:36:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.010+0000 7fed737fe640 1 -- 172.21.15.70:0/4092908126 learned_addr learned my addr 172.21.15.70:0/4092908126 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.010+0000 7fed72ffd640 1 --2- 172.21.15.70:0/4092908126 >> v2:172.21.15.70:3300/0 conn(0x7fed740cb150 0x7fed740cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.010+0000 7fed737fe640 1 -- 172.21.15.70:0/4092908126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed740702b0 msgr2=0x7fed74070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.010+0000 7fed73fff640 1 --2- 172.21.15.70:0/4092908126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed740702b0 0x7fed74070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.010+0000 7fed737fe640 1 --2- 172.21.15.70:0/4092908126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed740702b0 0x7fed74070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.009+0000 7f102f59e640 1 Processor -- start 2023-10-08T05:36:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.010+0000 7fed737fe640 1 -- 172.21.15.70:0/4092908126 >> v2:172.21.15.70:3300/0 conn(0x7fed740cb150 msgr2=0x7fed740cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.010+0000 7fed737fe640 1 --2- 172.21.15.70:0/4092908126 >> v2:172.21.15.70:3300/0 conn(0x7fed740cb150 0x7fed740cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.010+0000 7fed737fe640 1 -- 172.21.15.70:0/4092908126 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7406d210 con 0x7fed74161690 2023-10-08T05:36:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.011+0000 7fed72ffd640 1 --2- 172.21.15.70:0/4092908126 >> v2:172.21.15.70:3300/0 conn(0x7fed740cb150 0x7fed740cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:40.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.011+0000 7fed73fff640 1 --2- 172.21.15.70:0/4092908126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed740702b0 0x7fed74070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:40.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.013+0000 7fed737fe640 1 --2- 172.21.15.70:0/4092908126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed74161690 0x7fed7406dad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fed740b0d50 tx=0x7fed6c02bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31cbd9c4355cf508 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.012+0000 7f102f59e640 1 -- start start 2023-10-08T05:36:40.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.013+0000 7f102f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1018008850 0x7f102804bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.013+0000 7f102f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1028048720 0x7f1028053a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.013+0000 7f102f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102804a680 0x7f102804ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.013+0000 7f102f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102804a210 con 0x7f102804a680 2023-10-08T05:36:40.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.013+0000 7f102f59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028048f30 con 0x7f1018008850 2023-10-08T05:36:40.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.013+0000 7f102f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1028049210 con 0x7f1028048720 2023-10-08T05:36:40.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.014+0000 7f102ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102804a680 0x7f102804ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.012+0000 7f797359e640 1 Processor -- start 2023-10-08T05:36:40.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.012+0000 7f797359e640 1 -- start start 2023-10-08T05:36:40.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.014+0000 7f102ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102804a680 0x7f102804ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60110/0 (socket says 172.21.15.70:60110) 2023-10-08T05:36:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.014+0000 7f102ed9d640 1 -- 172.21.15.70:0/2031331363 learned_addr learned my addr 172.21.15.70:0/2031331363 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.014+0000 7f102e59c640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1018008850 0x7f102804bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.014+0000 7f102dd9b640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1028048720 0x7f1028053a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7fed727fc640 1 -- 172.21.15.70:0/4092908126 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c02ca80 con 0x7fed74161690 2023-10-08T05:36:40.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7fed727fc640 1 -- 172.21.15.70:0/4092908126 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c02cbe0 con 0x7fed74161690 2023-10-08T05:36:40.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7fed727fc640 1 -- 172.21.15.70:0/4092908126 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0343f0 con 0x7fed74161690 2023-10-08T05:36:40.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7f102ed9d640 1 -- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1028048720 msgr2=0x7f1028053a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7f102ed9d640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1028048720 0x7f1028053a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7f102ed9d640 1 -- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1018008850 msgr2=0x7f102804bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7f102ed9d640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1018008850 0x7f102804bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7f102ed9d640 1 -- 172.21.15.70:0/2031331363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280494f0 con 0x7f102804a680 2023-10-08T05:36:40.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7f102ed9d640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102804a680 0x7f102804ab00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1028007af0 tx=0x7f1024011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.016+0000 7f101effd640 1 -- 172.21.15.70:0/2031331363 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102401b070 con 0x7f102804a680 2023-10-08T05:36:40.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.014+0000 7f797359e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f796c00c0d0 0x7f796c054110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.014+0000 7f797359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f796c0e2960 0x7f796c038000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.014+0000 7f797359e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c006060 con 0x7f796c038540 2023-10-08T05:36:40.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.014+0000 7f797359e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c0549a0 con 0x7f796c0e2960 2023-10-08T05:36:40.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.014+0000 7f797359e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c050710 con 0x7f796c00c0d0 2023-10-08T05:36:40.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7f797259c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f796c00c0d0 0x7f796c054110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7f797259c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f796c00c0d0 0x7f796c054110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39872/0 (socket says 172.21.15.70:39872) 2023-10-08T05:36:40.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.015+0000 7f797259c640 1 -- 172.21.15.70:0/3796134743 learned_addr learned my addr 172.21.15.70:0/3796134743 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.016+0000 7f102f59e640 1 -- 172.21.15.70:0/2031331363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10280497d0 con 0x7f102804a680 2023-10-08T05:36:40.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.016+0000 7f102f59e640 1 -- 172.21.15.70:0/2031331363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1028053f60 con 0x7f102804a680 2023-10-08T05:36:40.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.016+0000 7f797259c640 1 -- 172.21.15.70:0/3796134743 >> v2:172.21.15.70:3300/0 conn(0x7f796c0e2960 msgr2=0x7f796c038000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.016+0000 7f101effd640 1 -- 172.21.15.70:0/2031331363 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1024004020 con 0x7f102804a680 2023-10-08T05:36:40.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.016+0000 7f7971d9b640 1 --2- 172.21.15.70:0/3796134743 >> v2:172.21.15.70:3300/0 conn(0x7f796c0e2960 0x7f796c038000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.016+0000 7f797259c640 1 --2- 172.21.15.70:0/3796134743 >> v2:172.21.15.70:3300/0 conn(0x7f796c0e2960 0x7f796c038000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.016+0000 7f797259c640 1 -- 172.21.15.70:0/3796134743 >> v1:172.21.15.181:6789/0 conn(0x7f796c038540 legacy=0x7f796c04fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.016+0000 7f797259c640 1 -- 172.21.15.70:0/3796134743 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c006340 con 0x7f796c00c0d0 2023-10-08T05:36:40.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.016+0000 7f7971d9b640 1 --2- 172.21.15.70:0/3796134743 >> v2:172.21.15.70:3300/0 conn(0x7f796c0e2960 0x7f796c038000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:40.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.017+0000 7f101effd640 1 -- 172.21.15.70:0/2031331363 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102401b070 con 0x7f102804a680 2023-10-08T05:36:40.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.019+0000 7f102f59e640 1 -- 172.21.15.70:0/2031331363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1014005250 con 0x7f102804a680 2023-10-08T05:36:40.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.019+0000 7f797259c640 1 --2- 172.21.15.70:0/3796134743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f796c00c0d0 0x7f796c054110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7974067fb0 tx=0x7f79740ae150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=407b93ed606c523f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7fed79d5a640 1 -- 172.21.15.70:0/4092908126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed74161690 msgr2=0x7fed7406dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7fed79d5a640 1 --2- 172.21.15.70:0/4092908126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed74161690 0x7fed7406dad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fed740b0d50 tx=0x7fed6c02bee0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7f797159a640 1 -- 172.21.15.70:0/3796134743 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740b6440 con 0x7f796c00c0d0 2023-10-08T05:36:40.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7f797159a640 1 -- 172.21.15.70:0/3796134743 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79740b65a0 con 0x7f796c00c0d0 2023-10-08T05:36:40.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7f797159a640 1 -- 172.21.15.70:0/3796134743 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740b6990 con 0x7f796c00c0d0 2023-10-08T05:36:40.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7fed79d5a640 1 -- 172.21.15.70:0/4092908126 shutdown_connections 2023-10-08T05:36:40.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7fed79d5a640 1 --2- 172.21.15.70:0/4092908126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed740702b0 0x7fed74070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7fed79d5a640 1 --2- 172.21.15.70:0/4092908126 >> v2:172.21.15.70:3300/0 conn(0x7fed740cb150 0x7fed740cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7fed79d5a640 1 --2- 172.21.15.70:0/4092908126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed74161690 0x7fed7406dad0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7fed79d5a640 1 -- 172.21.15.70:0/4092908126 >> 172.21.15.70:0/4092908126 conn(0x7fed7406a5d0 msgr2=0x7fed740b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.020+0000 7fed79d5a640 1 -- 172.21.15.70:0/4092908126 shutdown_connections 2023-10-08T05:36:40.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.021+0000 7f797359e640 1 -- 172.21.15.70:0/3796134743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f796c00c0d0 msgr2=0x7f796c054110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.021+0000 7f797359e640 1 --2- 172.21.15.70:0/3796134743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f796c00c0d0 0x7f796c054110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7974067fb0 tx=0x7f79740ae150 comp rx=0 tx=0).stop 2023-10-08T05:36:40.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.021+0000 7f797359e640 1 -- 172.21.15.70:0/3796134743 shutdown_connections 2023-10-08T05:36:40.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.021+0000 7f797359e640 1 --2- 172.21.15.70:0/3796134743 >> v2:172.21.15.70:3300/0 conn(0x7f796c0e2960 0x7f796c038000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.021+0000 7f797359e640 1 --2- 172.21.15.70:0/3796134743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f796c00c0d0 0x7f796c054110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.021+0000 7f797359e640 1 -- 172.21.15.70:0/3796134743 >> 172.21.15.70:0/3796134743 conn(0x7f796c009fe0 msgr2=0x7f796c00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.021+0000 7f101effd640 1 -- 172.21.15.70:0/2031331363 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1024004190 con 0x7f102804a680 2023-10-08T05:36:40.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.021+0000 7f797359e640 1 -- 172.21.15.70:0/3796134743 shutdown_connections 2023-10-08T05:36:40.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.021+0000 7f797359e640 1 -- 172.21.15.70:0/3796134743 wait complete. 2023-10-08T05:36:40.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.024+0000 7f797359e640 1 Processor -- start 2023-10-08T05:36:40.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.024+0000 7f797359e640 1 -- start start 2023-10-08T05:36:40.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.022+0000 7f101effd640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006af60 0x7f101006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.024+0000 7f102e59c640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006af60 0x7f101006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.025+0000 7f101effd640 1 -- 172.21.15.70:0/2031331363 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1024057f80 con 0x7f102804a680 2023-10-08T05:36:40.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.021+0000 7fed79d5a640 1 -- 172.21.15.70:0/4092908126 wait complete. 2023-10-08T05:36:40.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.024+0000 7fed79d5a640 1 Processor -- start 2023-10-08T05:36:40.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.024+0000 7fed79d5a640 1 -- start start 2023-10-08T05:36:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.027+0000 7fed79d5a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed740702b0 0x7fed74161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.027+0000 7fed79d5a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed740cb150 0x7fed740c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.027+0000 7fed79d5a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed74161690 0x7fed740c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.027+0000 7fed737fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed740702b0 0x7fed74161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.027+0000 7fed737fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed740702b0 0x7fed74161ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60132/0 (socket says 172.21.15.70:60132) 2023-10-08T05:36:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.027+0000 7fed737fe640 1 -- 172.21.15.70:0/3462741565 learned_addr learned my addr 172.21.15.70:0/3462741565 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.027+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed74164300 con 0x7fed740702b0 2023-10-08T05:36:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.027+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed74163280 con 0x7fed74161690 2023-10-08T05:36:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.027+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed74163560 con 0x7fed740cb150 2023-10-08T05:36:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.028+0000 7fed737fe640 1 -- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed740cb150 msgr2=0x7fed740c4670 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T05:36:40.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.028+0000 7fed737fe640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed740cb150 0x7fed740c4670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.028+0000 7fed737fe640 1 -- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed74161690 msgr2=0x7fed740c4bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T05:36:40.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.028+0000 7fed73fff640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed74161690 0x7fed740c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.028+0000 7fed737fe640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed74161690 0x7fed740c4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.028+0000 7fed737fe640 1 -- 172.21.15.70:0/3462741565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c009b30 con 0x7fed740702b0 2023-10-08T05:36:40.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.029+0000 7fed737fe640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed740702b0 0x7fed74161ce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed6c000f80 tx=0x7fed6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.031+0000 7f797359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f796c00c0d0 0x7f796c04e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.032+0000 7f797359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796c038540 0x7f796c04a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.032+0000 7f797359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f796c0e2960 0x7f796c168ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.032+0000 7f797359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c04b380 con 0x7f796c038540 2023-10-08T05:36:40.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.032+0000 7f797359e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c04b0a0 con 0x7f796c0e2960 2023-10-08T05:36:40.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.032+0000 7f797359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0d4070 con 0x7f796c00c0d0 2023-10-08T05:36:40.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.033+0000 7f7971d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796c038540 0x7f796c04a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.033+0000 7f7971d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796c038540 0x7f796c04a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60136/0 (socket says 172.21.15.70:60136) 2023-10-08T05:36:40.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.033+0000 7f7971d9b640 1 -- 172.21.15.70:0/1101682288 learned_addr learned my addr 172.21.15.70:0/1101682288 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.033+0000 7f797259c640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f796c00c0d0 0x7f796c04e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.033+0000 7f102e59c640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006af60 0x7f101006d420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1028003f50 tx=0x7f1020001c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.033+0000 7f7972d9d640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f796c0e2960 0x7f796c168ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.035+0000 7f7971d9b640 1 -- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f796c00c0d0 msgr2=0x7f796c04e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.035+0000 7f7971d9b640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f796c00c0d0 0x7f796c04e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.035+0000 7f7971d9b640 1 -- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f796c0e2960 msgr2=0x7f796c168ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.035+0000 7f7971d9b640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f796c0e2960 0x7f796c168ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.035+0000 7f7971d9b640 1 -- 172.21.15.70:0/1101682288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c0d4350 con 0x7f796c038540 2023-10-08T05:36:40.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.035+0000 7fed5bfff640 1 -- 172.21.15.70:0/3462741565 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c03e450 con 0x7fed740702b0 2023-10-08T05:36:40.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.036+0000 7f7971d9b640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796c038540 0x7f796c04a850 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f796800c7f0 tx=0x7f796800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.035+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed74163840 con 0x7fed740702b0 2023-10-08T05:36:40.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.036+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed74163e00 con 0x7fed740702b0 2023-10-08T05:36:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.036+0000 7f7962ffd640 1 -- 172.21.15.70:0/1101682288 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968007a30 con 0x7f796c038540 2023-10-08T05:36:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.036+0000 7f7962ffd640 1 -- 172.21.15.70:0/1101682288 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7968007b90 con 0x7f796c038540 2023-10-08T05:36:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.036+0000 7f797359e640 1 -- 172.21.15.70:0/1101682288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c0d4630 con 0x7f796c038540 2023-10-08T05:36:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.036+0000 7fed5bfff640 1 -- 172.21.15.70:0/3462741565 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c03e5b0 con 0x7fed740702b0 2023-10-08T05:36:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.037+0000 7fed5bfff640 1 -- 172.21.15.70:0/3462741565 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c03e960 con 0x7fed740702b0 2023-10-08T05:36:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.037+0000 7f7962ffd640 1 -- 172.21.15.70:0/1101682288 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796800ce40 con 0x7f796c038540 2023-10-08T05:36:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.038+0000 7f797359e640 1 -- 172.21.15.70:0/1101682288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c169400 con 0x7f796c038540 2023-10-08T05:36:40.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.038+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed54005250 con 0x7fed740702b0 2023-10-08T05:36:40.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.035+0000 7f101effd640 1 -- 172.21.15.70:0/2031331363 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1024054350 con 0x7f102804a680 2023-10-08T05:36:40.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.044+0000 7fed5bfff640 1 -- 172.21.15.70:0/3462741565 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fed6c02cd50 con 0x7fed740702b0 2023-10-08T05:36:40.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.045+0000 7fed5bfff640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed6006af40 0x7fed6006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.043+0000 7f7962ffd640 1 -- 172.21.15.70:0/1101682288 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f79680113f0 con 0x7f796c038540 2023-10-08T05:36:40.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.044+0000 7f7962ffd640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f795406af40 0x7f795406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.044+0000 7f7962ffd640 1 -- 172.21.15.70:0/1101682288 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f79680899d0 con 0x7f796c038540 2023-10-08T05:36:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.046+0000 7fed5bfff640 1 -- 172.21.15.70:0/3462741565 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fed6c0acb00 con 0x7fed740702b0 2023-10-08T05:36:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.046+0000 7f797259c640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f795406af40 0x7f795406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.046+0000 7fed72ffd640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed6006af40 0x7fed6006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.047+0000 7f797259c640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f795406af40 0x7f795406d400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f796c050850 tx=0x7f79740b2000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.048+0000 7f797359e640 1 -- 172.21.15.70:0/1101682288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c0d3e80 con 0x7f796c038540 2023-10-08T05:36:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.048+0000 7fed72ffd640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed6006af40 0x7fed6006d400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fed740afba0 tx=0x7fed68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.053+0000 7fed5bfff640 1 -- 172.21.15.70:0/3462741565 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed6c0779c0 con 0x7fed740702b0 2023-10-08T05:36:40.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.059+0000 7f7962ffd640 1 -- 172.21.15.70:0/1101682288 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7968054820 con 0x7f796c038540 2023-10-08T05:36:40.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.069+0000 7f757359e640 1 Processor -- start 2023-10-08T05:36:40.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.069+0000 7f757359e640 1 -- start start 2023-10-08T05:36:40.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.071+0000 7f757359e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f756c0e4740 0x7f756c0543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.071+0000 7f757359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f756c038fb0 0x7f756c054e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.072+0000 7f757359e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c056b90 con 0x7f756c035cc0 2023-10-08T05:36:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.072+0000 7f757359e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756c0549f0 con 0x7f756c038fb0 2023-10-08T05:36:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.072+0000 7f757359e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756c036340 con 0x7f756c0e4740 2023-10-08T05:36:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.072+0000 7f7572d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f756c038fb0 0x7f756c054e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.072+0000 7f7572d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f756c038fb0 0x7f756c054e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60148/0 (socket says 172.21.15.70:60148) 2023-10-08T05:36:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.072+0000 7f7572d9d640 1 -- 172.21.15.70:0/4156354839 learned_addr learned my addr 172.21.15.70:0/4156354839 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.073+0000 7f7572d9d640 1 -- 172.21.15.70:0/4156354839 >> v2:172.21.15.181:3300/0 conn(0x7f756c0e4740 msgr2=0x7f756c0543c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T05:36:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.073+0000 7f7572d9d640 1 --2- 172.21.15.70:0/4156354839 >> v2:172.21.15.181:3300/0 conn(0x7f756c0e4740 0x7f756c0543c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.073+0000 7f7572d9d640 1 -- 172.21.15.70:0/4156354839 >> v1:172.21.15.181:6789/0 conn(0x7f756c035cc0 legacy=0x7f756c0388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.073+0000 7f7572d9d640 1 -- 172.21.15.70:0/4156354839 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c051720 con 0x7f756c038fb0 2023-10-08T05:36:40.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.075+0000 7fe54bd6c640 1 Processor -- start 2023-10-08T05:36:40.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.075+0000 7fe54bd6c640 1 -- start start 2023-10-08T05:36:40.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.077+0000 7fe54bd6c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5440c8e70 0x7fe5440ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.077+0000 7fe54bd6c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe54406e3a0 0x7fe5440b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.077+0000 7fe54bd6c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54406d490 con 0x7fe5440cae90 2023-10-08T05:36:40.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.077+0000 7fe54bd6c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54406c880 con 0x7fe54406e3a0 2023-10-08T05:36:40.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.077+0000 7fe54bd6c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54406cb60 con 0x7fe5440c8e70 2023-10-08T05:36:40.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.078+0000 7fe549b08640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5440c8e70 0x7fe5440ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.078+0000 7fe549b08640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5440c8e70 0x7fe5440ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:60846/0 (socket says 172.21.15.70:60846) 2023-10-08T05:36:40.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.078+0000 7fe549b08640 1 -- 172.21.15.70:0/3244532582 learned_addr learned my addr 172.21.15.70:0/3244532582 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.075+0000 7f7572d9d640 1 --2- 172.21.15.70:0/4156354839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c038fb0 0x7f756c054e80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f756c00b2a0 tx=0x7f756402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478d617c07c1f1d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.076+0000 7f757159a640 1 -- 172.21.15.70:0/4156354839 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756402d040 con 0x7f756c038fb0 2023-10-08T05:36:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.076+0000 7f757159a640 1 -- 172.21.15.70:0/4156354839 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564031810 con 0x7f756c038fb0 2023-10-08T05:36:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.076+0000 7f757159a640 1 -- 172.21.15.70:0/4156354839 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564031ba0 con 0x7f756c038fb0 2023-10-08T05:36:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.077+0000 7f757359e640 1 -- 172.21.15.70:0/4156354839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c038fb0 msgr2=0x7f756c054e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.077+0000 7f757359e640 1 --2- 172.21.15.70:0/4156354839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c038fb0 0x7f756c054e80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f756c00b2a0 tx=0x7f756402b8b0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.078+0000 7f757359e640 1 -- 172.21.15.70:0/4156354839 shutdown_connections 2023-10-08T05:36:40.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.078+0000 7f757359e640 1 --2- 172.21.15.70:0/4156354839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c038fb0 0x7f756c054e80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.078+0000 7f757359e640 1 --2- 172.21.15.70:0/4156354839 >> v2:172.21.15.181:3300/0 conn(0x7f756c0e4740 0x7f756c0543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.078+0000 7f757359e640 1 -- 172.21.15.70:0/4156354839 >> 172.21.15.70:0/4156354839 conn(0x7f756c0538a0 msgr2=0x7f756c053cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.079+0000 7f757359e640 1 -- 172.21.15.70:0/4156354839 shutdown_connections 2023-10-08T05:36:40.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.079+0000 7f757359e640 1 -- 172.21.15.70:0/4156354839 wait complete. 2023-10-08T05:36:40.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.079+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700623519 0 0) 0x7fe54406d490 con 0x7fe5440cae90 2023-10-08T05:36:40.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.079+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe530003520 con 0x7fe5440cae90 2023-10-08T05:36:40.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3783186715 0 0) 0x7fe530003520 con 0x7fe5440cae90 2023-10-08T05:36:40.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5440c8e70 msgr2=0x7fe5440ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 --2- 172.21.15.70:0/3244532582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5440c8e70 0x7fe5440ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 >> v2:172.21.15.181:3300/0 conn(0x7fe54406e3a0 msgr2=0x7fe5440b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 --2- 172.21.15.70:0/3244532582 >> v2:172.21.15.181:3300/0 conn(0x7fe54406e3a0 0x7fe5440b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54406ce40 con 0x7fe5440cae90 2023-10-08T05:36:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe53c0019f0 con 0x7fe5440cae90 2023-10-08T05:36:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 >> v1:172.21.15.70:6789/0 conn(0x7fe5440cae90 legacy=0x7fe5440cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 --2- 172.21.15.70:0/3244532582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe530003970 0x7fe530003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 --2- 172.21.15.70:0/3244532582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5300047c0 0x7fe530004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 --2- 172.21.15.70:0/3244532582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe530005180 0x7fe53000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54406c880 con 0x7fe530003970 2023-10-08T05:36:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54406cb60 con 0x7fe530005180 2023-10-08T05:36:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.080+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53000bff0 con 0x7fe5300047c0 2023-10-08T05:36:40.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.081+0000 7fe549b08640 1 --2- 172.21.15.70:0/3244532582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe530003970 0x7fe530003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.081+0000 7fe54a309640 1 --2- 172.21.15.70:0/3244532582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe530005180 0x7fe53000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.082+0000 7fe549b08640 1 -- 172.21.15.70:0/3244532582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5300047c0 msgr2=0x7fe530004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.082+0000 7fe549b08640 1 --2- 172.21.15.70:0/3244532582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5300047c0 0x7fe530004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.082+0000 7fe549b08640 1 -- 172.21.15.70:0/3244532582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe530005180 msgr2=0x7fe53000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.082+0000 7fe549b08640 1 --2- 172.21.15.70:0/3244532582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe530005180 0x7fe53000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.082+0000 7fe549b08640 1 -- 172.21.15.70:0/3244532582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54406ce40 con 0x7fe530003970 2023-10-08T05:36:40.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.081+0000 7f757359e640 1 Processor -- start 2023-10-08T05:36:40.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.082+0000 7f757359e640 1 -- start start 2023-10-08T05:36:40.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.083+0000 7f757359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c035cc0 0x7f756c0e50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.083+0000 7f757359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756c038fb0 0x7f756c1a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7f757359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0e4740 0x7f756c1a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7f757259c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c035cc0 0x7f756c0e50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7f757259c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c035cc0 0x7f756c0e50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39926/0 (socket says 172.21.15.70:39926) 2023-10-08T05:36:40.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.083+0000 7fe549b08640 1 --2- 172.21.15.70:0/3244532582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe530003970 0x7fe530003d90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe5440b0cc0 tx=0x7fe5300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfff90e347071e65 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54002c610 con 0x7fe530003970 2023-10-08T05:36:40.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54002c770 con 0x7fe530003970 2023-10-08T05:36:40.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7fe548b06640 1 -- 172.21.15.70:0/3244532582 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54002cb20 con 0x7fe530003970 2023-10-08T05:36:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.083+0000 7f603ee46640 1 Processor -- start 2023-10-08T05:36:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.083+0000 7f603ee46640 1 -- start start 2023-10-08T05:36:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7fe54bd6c640 1 -- 172.21.15.70:0/3244532582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe530003970 msgr2=0x7fe530003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7fe54bd6c640 1 --2- 172.21.15.70:0/3244532582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe530003970 0x7fe530003d90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe5440b0cc0 tx=0x7fe5300057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7fe54a309640 1 -- 172.21.15.70:0/3244532582 reap_dead start 2023-10-08T05:36:40.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7fe54bd6c640 1 -- 172.21.15.70:0/3244532582 shutdown_connections 2023-10-08T05:36:40.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7fe54bd6c640 1 -- 172.21.15.70:0/3244532582 >> 172.21.15.70:0/3244532582 conn(0x7fe54406a5d0 msgr2=0x7fe5440b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7f603ee46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6038161170 0x7f60381658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7f7571d9b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756c038fb0 0x7f756c1a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7f7571d9b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756c038fb0 0x7f756c1a28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:60856/0 (socket says 172.21.15.70:60856) 2023-10-08T05:36:40.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7f757259c640 1 -- 172.21.15.70:0/2174680432 learned_addr learned my addr 172.21.15.70:0/2174680432 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7f757359e640 1 -- 172.21.15.70:0/2174680432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c04a100 con 0x7f756c0e4740 2023-10-08T05:36:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7f757359e640 1 -- 172.21.15.70:0/2174680432 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c049470 con 0x7f756c035cc0 2023-10-08T05:36:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7f757359e640 1 -- 172.21.15.70:0/2174680432 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c048e40 con 0x7f756c038fb0 2023-10-08T05:36:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.084+0000 7f7572d9d640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0e4740 0x7f756c1a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7f7572d9d640 1 -- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756c038fb0 msgr2=0x7f756c1a28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7f7572d9d640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756c038fb0 0x7f756c1a28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7f7572d9d640 1 -- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c035cc0 msgr2=0x7f756c0e50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7f7572d9d640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c035cc0 0x7f756c0e50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7f7572d9d640 1 -- 172.21.15.70:0/2174680432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75640094d0 con 0x7f756c0e4740 2023-10-08T05:36:40.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.086+0000 7f7572d9d640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0e4740 0x7f756c1a2e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7564033000 tx=0x7f75640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.086+0000 7f757259c640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c035cc0 0x7f756c0e50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.086+0000 7f7571d9b640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756c038fb0 0x7f756c1a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7f603ee46640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f60380cd2d0 0x7f60380cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7f603ee46640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60380cdc60 con 0x7f60381622f0 2023-10-08T05:36:40.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7f603ee46640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6038162780 con 0x7f6038161170 2023-10-08T05:36:40.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.085+0000 7f603ee46640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60380b4ec0 con 0x7f60380cd2d0 2023-10-08T05:36:40.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.087+0000 7fe54bd6c640 1 -- 172.21.15.70:0/3244532582 shutdown_connections 2023-10-08T05:36:40.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.087+0000 7f603d3e3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f60380cd2d0 0x7f60380cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.087+0000 7f6037fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f60381622f0 0x7f60380b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:52116/0 (socket says 172.21.15.70:52116) 2023-10-08T05:36:40.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.087+0000 7f603d3e3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f60380cd2d0 0x7f60380cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39940/0 (socket says 172.21.15.70:39940) 2023-10-08T05:36:40.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.087+0000 7f6037fff640 1 -- 172.21.15.70:0/2115289902 learned_addr learned my addr 172.21.15.70:0/2115289902 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:40.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.089+0000 7f603cbe2640 1 --2- 172.21.15.70:0/2115289902 >> v2:172.21.15.70:3300/0 conn(0x7f6038161170 0x7f60381658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.087+0000 7f7562ffd640 1 -- 172.21.15.70:0/2174680432 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756403ace0 con 0x7f756c0e4740 2023-10-08T05:36:40.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.087+0000 7f7562ffd640 1 -- 172.21.15.70:0/2174680432 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756403ae40 con 0x7f756c0e4740 2023-10-08T05:36:40.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.087+0000 7f757359e640 1 -- 172.21.15.70:0/2174680432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c0cdf00 con 0x7f756c0e4740 2023-10-08T05:36:40.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.087+0000 7f7562ffd640 1 -- 172.21.15.70:0/2174680432 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75640445f0 con 0x7f756c0e4740 2023-10-08T05:36:40.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.088+0000 7f757359e640 1 -- 172.21.15.70:0/2174680432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c049100 con 0x7f756c0e4740 2023-10-08T05:36:40.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.090+0000 7f603cbe2640 1 -- 172.21.15.70:0/2115289902 >> v2:172.21.15.181:3300/0 conn(0x7f60380cd2d0 msgr2=0x7f60380cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.090+0000 7f603cbe2640 1 --2- 172.21.15.70:0/2115289902 >> v2:172.21.15.181:3300/0 conn(0x7f60380cd2d0 0x7f60380cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.090+0000 7f603cbe2640 1 -- 172.21.15.70:0/2115289902 >> v1:172.21.15.181:6789/0 conn(0x7f60381622f0 legacy=0x7f60380b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.090+0000 7f603cbe2640 1 -- 172.21.15.70:0/2115289902 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60380b51a0 con 0x7f6038161170 2023-10-08T05:36:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.087+0000 7fe54bd6c640 1 -- 172.21.15.70:0/3244532582 wait complete. 2023-10-08T05:36:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.090+0000 7fe54bd6c640 1 Processor -- start 2023-10-08T05:36:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.090+0000 7fe54bd6c640 1 -- start start 2023-10-08T05:36:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.092+0000 7fe54bd6c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5440cc940 0x7fe5440c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.092+0000 7fe54bd6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5440ccd20 0x7fe5440c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.092+0000 7fe549b08640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5440cc940 0x7fe5440c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.092+0000 7fe549b08640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5440cc940 0x7fe5440c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39954/0 (socket says 172.21.15.70:39954) 2023-10-08T05:36:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.093+0000 7fe549307640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5440ccd20 0x7fe5440c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.093+0000 7fe549307640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5440ccd20 0x7fe5440c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60194/0 (socket says 172.21.15.70:60194) 2023-10-08T05:36:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.093+0000 7fe549b08640 1 -- 172.21.15.70:0/308753389 learned_addr learned my addr 172.21.15.70:0/308753389 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.092+0000 7f603cbe2640 1 --2- 172.21.15.70:0/2115289902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6038161170 0x7f60381658a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f60380cf630 tx=0x7f602802b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26c34d372674c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.093+0000 7f7562ffd640 1 -- 172.21.15.70:0/2174680432 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f756404d450 con 0x7f756c0e4740 2023-10-08T05:36:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.091+0000 7f310fecf640 1 Processor -- start 2023-10-08T05:36:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.093+0000 7fe54bd6c640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5441540e0 0x7fe544157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.092+0000 7f310fecf640 1 -- start start 2023-10-08T05:36:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.091+0000 7f7560ff9640 1 -- 172.21.15.70:0/2174680432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c005250 con 0x7f756c0e4740 2023-10-08T05:36:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.093+0000 7f310fecf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f310806a460 0x7f3108159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f310fecf640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31080ca220 0x7f31080ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f310fecf640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 0x7f31080cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f310fecf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31080cb3c0 con 0x7f310806a460 2023-10-08T05:36:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f310fecf640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31080cb6a0 con 0x7f31080ca220 2023-10-08T05:36:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f310fecf640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31080cb980 con 0x7f31080cae90 2023-10-08T05:36:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f60377fe640 1 -- 172.21.15.70:0/2115289902 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028030710 con 0x7f6038161170 2023-10-08T05:36:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f60377fe640 1 -- 172.21.15.70:0/2115289902 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602802e050 con 0x7f6038161170 2023-10-08T05:36:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f60377fe640 1 -- 172.21.15.70:0/2115289902 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028030a90 con 0x7f6038161170 2023-10-08T05:36:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f310dc6b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f310806a460 0x7f3108159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f310dc6b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f310806a460 0x7f3108159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60208/0 (socket says 172.21.15.70:60208) 2023-10-08T05:36:40.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f310dc6b640 1 -- 172.21.15.70:0/3006504160 learned_addr learned my addr 172.21.15.70:0/3006504160 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.094+0000 7f7562ffd640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f755406af40 0x7f755406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.093+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440c82a0 con 0x7fe5440ccd20 2023-10-08T05:36:40.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.093+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe544153600 con 0x7fe5440cc940 2023-10-08T05:36:40.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.093+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5441538e0 con 0x7fe5441540e0 2023-10-08T05:36:40.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7f7562ffd640 1 -- 172.21.15.70:0/2174680432 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f75640abcc0 con 0x7f756c0e4740 2023-10-08T05:36:40.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7fe549307640 1 -- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5441540e0 msgr2=0x7fe544157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7fe549307640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5441540e0 0x7fe544157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7fe549307640 1 -- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5440cc940 msgr2=0x7fe5440c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7fe549307640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5440cc940 0x7fe5440c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7fe549307640 1 -- 172.21.15.70:0/308753389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe544153bc0 con 0x7fe5440ccd20 2023-10-08T05:36:40.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7f310d46a640 1 --2- 172.21.15.70:0/3006504160 >> v2:172.21.15.181:3300/0 conn(0x7f31080ca220 0x7f31080ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7f310e46c640 1 --2- 172.21.15.70:0/3006504160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 0x7f31080cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7f310dc6b640 1 -- 172.21.15.70:0/3006504160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 msgr2=0x7f31080cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7f757259c640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f755406af40 0x7f755406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.096+0000 7fe549307640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5440ccd20 0x7fe5440c7a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe54406e7c0 tx=0x7fe53400bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7f310dc6b640 1 --2- 172.21.15.70:0/3006504160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 0x7f31080cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7f310dc6b640 1 -- 172.21.15.70:0/3006504160 >> v2:172.21.15.181:3300/0 conn(0x7f31080ca220 msgr2=0x7f31080ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7f310dc6b640 1 --2- 172.21.15.70:0/3006504160 >> v2:172.21.15.181:3300/0 conn(0x7f31080ca220 0x7f31080ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7f310dc6b640 1 -- 172.21.15.70:0/3006504160 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31080cbc60 con 0x7f310806a460 2023-10-08T05:36:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.095+0000 7f310d46a640 1 --2- 172.21.15.70:0/3006504160 >> v2:172.21.15.181:3300/0 conn(0x7f31080ca220 0x7f31080ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.097+0000 7f757259c640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f755406af40 0x7f755406d400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7574074bc0 tx=0x7f757406d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.097+0000 7f603ee46640 1 -- 172.21.15.70:0/2115289902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6038161170 msgr2=0x7f60381658a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.097+0000 7f603ee46640 1 --2- 172.21.15.70:0/2115289902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6038161170 0x7f60381658a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f60380cf630 tx=0x7f602802b680 comp rx=0 tx=0).stop 2023-10-08T05:36:40.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.097+0000 7f310dc6b640 1 --2- 172.21.15.70:0/3006504160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310806a460 0x7f3108159140 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3108070ca0 tx=0x7f30f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef24dfebff4245a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.097+0000 7f603ee46640 1 -- 172.21.15.70:0/2115289902 shutdown_connections 2023-10-08T05:36:40.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.097+0000 7f603ee46640 1 --2- 172.21.15.70:0/2115289902 >> v2:172.21.15.181:3300/0 conn(0x7f60380cd2d0 0x7f60380cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.097+0000 7f603ee46640 1 --2- 172.21.15.70:0/2115289902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6038161170 0x7f60381658a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.097+0000 7f603ee46640 1 -- 172.21.15.70:0/2115289902 >> 172.21.15.70:0/2115289902 conn(0x7f60380cf700 msgr2=0x7f60380cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.098+0000 7f603ee46640 1 -- 172.21.15.70:0/2115289902 shutdown_connections 2023-10-08T05:36:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.098+0000 7f603ee46640 1 -- 172.21.15.70:0/2115289902 wait complete. 2023-10-08T05:36:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.101+0000 7f603ee46640 1 Processor -- start 2023-10-08T05:36:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.101+0000 7f603ee46640 1 -- start start 2023-10-08T05:36:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.103+0000 7f603ee46640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60380cd2d0 0x7f603814f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.103+0000 7f603ee46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6038161170 0x7f603814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.104+0000 7f603ee46640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60381622f0 0x7f603814e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.104+0000 7f603cbe2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60380cd2d0 0x7f603814f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.104+0000 7f6037fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6038161170 0x7f603814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.104+0000 7f603cbe2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60380cd2d0 0x7f603814f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39982/0 (socket says 172.21.15.70:39982) 2023-10-08T05:36:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.104+0000 7f603cbe2640 1 -- 172.21.15.70:0/2052257439 learned_addr learned my addr 172.21.15.70:0/2052257439 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.104+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603814c5f0 con 0x7f6038161170 2023-10-08T05:36:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.104+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603814e6a0 con 0x7f60380cd2d0 2023-10-08T05:36:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.104+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603814d0c0 con 0x7f60381622f0 2023-10-08T05:36:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.104+0000 7f603d3e3640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60381622f0 0x7f603814e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.105+0000 7f6037fff640 1 -- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60381622f0 msgr2=0x7f603814e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.105+0000 7f6037fff640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60381622f0 0x7f603814e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.105+0000 7f6037fff640 1 -- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60380cd2d0 msgr2=0x7f603814f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.105+0000 7f6037fff640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60380cd2d0 0x7f603814f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.105+0000 7f6037fff640 1 -- 172.21.15.70:0/2052257439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028009400 con 0x7f6038161170 2023-10-08T05:36:40.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.106+0000 7fe53a7fc640 1 -- 172.21.15.70:0/308753389 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400c720 con 0x7fe5440ccd20 2023-10-08T05:36:40.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.106+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5440c7f90 con 0x7fe5440ccd20 2023-10-08T05:36:40.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.106+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe544154590 con 0x7fe5440ccd20 2023-10-08T05:36:40.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.106+0000 7f310cc69640 1 -- 172.21.15.70:0/3006504160 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f8030860 con 0x7f310806a460 2023-10-08T05:36:40.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.107+0000 7f7562ffd640 1 -- 172.21.15.70:0/2174680432 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7564076a60 con 0x7f756c0e4740 2023-10-08T05:36:40.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.105+0000 7f603cbe2640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60380cd2d0 0x7f603814f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:36:40.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.106+0000 7f603d3e3640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60381622f0 0x7f603814e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:40.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.108+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54416a050 con 0x7fe5440ccd20 2023-10-08T05:36:40.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.108+0000 7f6037fff640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6038161170 0x7f603814c0b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f603000cb30 tx=0x7f603000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.110+0000 7f310cc69640 1 -- 172.21.15.70:0/3006504160 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f80309c0 con 0x7f310806a460 2023-10-08T05:36:40.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.110+0000 7fe53a7fc640 1 -- 172.21.15.70:0/308753389 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe534009200 con 0x7fe5440ccd20 2023-10-08T05:36:40.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.110+0000 7fe53a7fc640 1 -- 172.21.15.70:0/308753389 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53400ca80 con 0x7fe5440ccd20 2023-10-08T05:36:40.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.110+0000 7f60357fa640 1 -- 172.21.15.70:0/2052257439 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000a830 con 0x7f6038161170 2023-10-08T05:36:40.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.111+0000 7f310fecf640 1 -- 172.21.15.70:0/3006504160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310806a460 msgr2=0x7f3108159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.111+0000 7f310fecf640 1 --2- 172.21.15.70:0/3006504160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310806a460 0x7f3108159140 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3108070ca0 tx=0x7f30f802b9a0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.111+0000 7f60357fa640 1 -- 172.21.15.70:0/2052257439 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603000a990 con 0x7f6038161170 2023-10-08T05:36:40.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.111+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603814d3a0 con 0x7f6038161170 2023-10-08T05:36:40.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.112+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603814d960 con 0x7f6038161170 2023-10-08T05:36:40.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.113+0000 7fe53a7fc640 1 -- 172.21.15.70:0/308753389 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe53400cca0 con 0x7fe5440ccd20 2023-10-08T05:36:40.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.113+0000 7f60357fa640 1 -- 172.21.15.70:0/2052257439 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000ad60 con 0x7f6038161170 2023-10-08T05:36:40.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.114+0000 7f310fecf640 1 -- 172.21.15.70:0/3006504160 shutdown_connections 2023-10-08T05:36:40.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.114+0000 7f310fecf640 1 --2- 172.21.15.70:0/3006504160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 0x7f31080cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.114+0000 7f310fecf640 1 --2- 172.21.15.70:0/3006504160 >> v2:172.21.15.181:3300/0 conn(0x7f31080ca220 0x7f31080ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.114+0000 7f310fecf640 1 --2- 172.21.15.70:0/3006504160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310806a460 0x7f3108159140 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.114+0000 7f310fecf640 1 -- 172.21.15.70:0/3006504160 >> 172.21.15.70:0/3006504160 conn(0x7f31080660e0 msgr2=0x7f3108069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.114+0000 7fe53a7fc640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe52806aea0 0x7fe52806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.115+0000 7fe53a7fc640 1 -- 172.21.15.70:0/308753389 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe534088fb0 con 0x7fe5440ccd20 2023-10-08T05:36:40.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.115+0000 7f310fecf640 1 -- 172.21.15.70:0/3006504160 shutdown_connections 2023-10-08T05:36:40.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.115+0000 7fe549b08640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe52806aea0 0x7fe52806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.116+0000 7f310fecf640 1 -- 172.21.15.70:0/3006504160 wait complete. 2023-10-08T05:36:40.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.115+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603804e160 con 0x7f6038161170 2023-10-08T05:36:40.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.118+0000 7f60357fa640 1 -- 172.21.15.70:0/2052257439 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f60300070d0 con 0x7f6038161170 2023-10-08T05:36:40.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.119+0000 7f310fecf640 1 Processor -- start 2023-10-08T05:36:40.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.119+0000 7f310fecf640 1 -- start start 2023-10-08T05:36:40.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.120+0000 7fe549b08640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe52806aea0 0x7fe52806d360 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe5440cd290 tx=0x7fe540030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.120+0000 7f60357fa640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f602406af40 0x7f602406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.120+0000 7f60357fa640 1 -- 172.21.15.70:0/2052257439 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6030089380 con 0x7f6038161170 2023-10-08T05:36:40.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.121+0000 7f603cbe2640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f602406af40 0x7f602406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.122+0000 7f310fecf640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310806a460 0x7f310815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.122+0000 7f310fecf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31080ca220 0x7f310816a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.122+0000 7f310fecf640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 0x7f3108156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.122+0000 7f310dc6b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310806a460 0x7f310815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.122+0000 7f310dc6b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310806a460 0x7f310815bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39990/0 (socket says 172.21.15.70:39990) 2023-10-08T05:36:40.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.123+0000 7f310d46a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31080ca220 0x7f310816a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.123+0000 7f603cbe2640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f602406af40 0x7f602406d400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f602802b6c0 tx=0x7f6028008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.123+0000 7f310d46a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31080ca220 0x7f310816a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60228/0 (socket says 172.21.15.70:60228) 2023-10-08T05:36:40.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.123+0000 7f310dc6b640 1 -- 172.21.15.70:0/1973608817 learned_addr learned my addr 172.21.15.70:0/1973608817 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.123+0000 7f310e46c640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 0x7f3108156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.124+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310816a970 con 0x7f31080ca220 2023-10-08T05:36:40.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.124+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310816ae60 con 0x7f310806a460 2023-10-08T05:36:40.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.124+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3108172420 con 0x7f31080cae90 2023-10-08T05:36:40.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.124+0000 7fe53a7fc640 1 -- 172.21.15.70:0/308753389 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe534057810 con 0x7fe5440ccd20 2023-10-08T05:36:40.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.125+0000 7f310d46a640 1 -- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 msgr2=0x7f3108156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.125+0000 7f310d46a640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 0x7f3108156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.125+0000 7f310d46a640 1 -- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310806a460 msgr2=0x7f310815bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.125+0000 7f310d46a640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310806a460 0x7f310815bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.125+0000 7f310d46a640 1 -- 172.21.15.70:0/1973608817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f80094d0 con 0x7f31080ca220 2023-10-08T05:36:40.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.126+0000 7f310d46a640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31080ca220 0x7f310816a430 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3100007dc0 tx=0x7f310000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.127+0000 7f310e46c640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 0x7f3108156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:40.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.133+0000 7f60357fa640 1 -- 172.21.15.70:0/2052257439 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60300541d0 con 0x7f6038161170 2023-10-08T05:36:40.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.134+0000 7f30fe7fc640 1 -- 172.21.15.70:0/1973608817 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100003990 con 0x7f31080ca220 2023-10-08T05:36:40.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.134+0000 7f30fe7fc640 1 -- 172.21.15.70:0/1973608817 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310000f040 con 0x7f31080ca220 2023-10-08T05:36:40.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.135+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3108172700 con 0x7f31080ca220 2023-10-08T05:36:40.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.135+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31080cdde0 con 0x7f31080ca220 2023-10-08T05:36:40.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.135+0000 7f30fe7fc640 1 -- 172.21.15.70:0/1973608817 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100003cf0 con 0x7f31080ca220 2023-10-08T05:36:40.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.140+0000 7f30fe7fc640 1 -- 172.21.15.70:0/1973608817 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3100004030 con 0x7f31080ca220 2023-10-08T05:36:40.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.141+0000 7f30fe7fc640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f30ec06aec0 0x7f30ec06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.141+0000 7f30fe7fc640 1 -- 172.21.15.70:0/1973608817 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3100089a30 con 0x7f31080ca220 2023-10-08T05:36:40.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.142+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3108172940 con 0x7f31080ca220 2023-10-08T05:36:40.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.148+0000 7f310dc6b640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f30ec06aec0 0x7f30ec06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.152+0000 7f30fe7fc640 1 -- 172.21.15.70:0/1973608817 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3108172940 con 0x7f31080ca220 2023-10-08T05:36:40.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.162+0000 7f310dc6b640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f30ec06aec0 0x7f30ec06d380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f30f802b9e0 tx=0x7f30f8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.397+0000 7ff98eba7640 1 -- 172.21.15.70:0/3343832637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff988074150 con 0x7ff9880b1f70 2023-10-08T05:36:40.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.398+0000 7ff97e7fc640 1 -- 172.21.15.70:0/3343832637 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff984053e60 con 0x7ff9880b1f70 2023-10-08T05:36:40.399 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:40.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.401+0000 7ff96bfff640 1 -- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff96c06aef0 msgr2=0x7ff96c06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.402+0000 7ff96bfff640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff96c06aef0 0x7ff96c06d3b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff978006b10 tx=0x7ff978008040 comp rx=0 tx=0).stop 2023-10-08T05:36:40.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.402+0000 7ff96bfff640 1 -- 172.21.15.70:0/3343832637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9880b1f70 msgr2=0x7ff9880cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.402+0000 7ff96bfff640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9880b1f70 0x7ff9880cb7c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff98400ea10 tx=0x7ff98400eed0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.403+0000 7ff96bfff640 1 -- 172.21.15.70:0/3343832637 shutdown_connections 2023-10-08T05:36:40.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.403+0000 7ff96bfff640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff96c06aef0 0x7ff96c06d3b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.403+0000 7ff96bfff640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9880c5910 0x7ff9880cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.403+0000 7ff96bfff640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9880b1f70 0x7ff9880cb7c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.403+0000 7ff96bfff640 1 --2- 172.21.15.70:0/3343832637 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9880b1b90 0x7ff98815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.403+0000 7ff96bfff640 1 -- 172.21.15.70:0/3343832637 >> 172.21.15.70:0/3343832637 conn(0x7ff98809bc40 msgr2=0x7ff9880af510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.404+0000 7ff96bfff640 1 -- 172.21.15.70:0/3343832637 shutdown_connections 2023-10-08T05:36:40.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.404+0000 7ff96bfff640 1 -- 172.21.15.70:0/3343832637 wait complete. 2023-10-08T05:36:40.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-10-08T05:36:40.432 DEBUG:teuthology.parallel:result is None 2023-10-08T05:36:40.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.437+0000 7f102f59e640 1 -- 172.21.15.70:0/2031331363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f10140057d0 con 0x7f102804a680 2023-10-08T05:36:40.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.439+0000 7f101effd640 1 -- 172.21.15.70:0/2031331363 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1024054350 con 0x7f102804a680 2023-10-08T05:36:40.440 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:40.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.442+0000 7f102f59e640 1 -- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006af60 msgr2=0x7f101006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.442+0000 7f102f59e640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006af60 0x7f101006d420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1028003f50 tx=0x7f1020001c50 comp rx=0 tx=0).stop 2023-10-08T05:36:40.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.443+0000 7f102f59e640 1 -- 172.21.15.70:0/2031331363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102804a680 msgr2=0x7f102804ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.443+0000 7f102f59e640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102804a680 0x7f102804ab00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1028007af0 tx=0x7f1024011920 comp rx=0 tx=0).stop 2023-10-08T05:36:40.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.443+0000 7f102f59e640 1 -- 172.21.15.70:0/2031331363 shutdown_connections 2023-10-08T05:36:40.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.443+0000 7f102f59e640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006af60 0x7f101006d420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.443+0000 7f102f59e640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102804a680 0x7f102804ab00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.443+0000 7f102f59e640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1028048720 0x7f1028053a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.444+0000 7f102f59e640 1 --2- 172.21.15.70:0/2031331363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1018008850 0x7f102804bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.444+0000 7f102f59e640 1 -- 172.21.15.70:0/2031331363 >> 172.21.15.70:0/2031331363 conn(0x7f1028054c10 msgr2=0x7f10280de640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.444+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fed540057d0 con 0x7fed740702b0 2023-10-08T05:36:40.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.445+0000 7fed5bfff640 1 -- 172.21.15.70:0/3462741565 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed6c07afe0 con 0x7fed740702b0 2023-10-08T05:36:40.448 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:40.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.445+0000 7f102f59e640 1 -- 172.21.15.70:0/2031331363 shutdown_connections 2023-10-08T05:36:40.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.445+0000 7f102f59e640 1 -- 172.21.15.70:0/2031331363 wait complete. 2023-10-08T05:36:40.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed6006af40 msgr2=0x7fed6006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed6006af40 0x7fed6006d400 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fed740afba0 tx=0x7fed68008040 comp rx=0 tx=0).stop 2023-10-08T05:36:40.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed740702b0 msgr2=0x7fed74161ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed740702b0 0x7fed74161ce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fed6c000f80 tx=0x7fed6c004a40 comp rx=0 tx=0).stop 2023-10-08T05:36:40.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 shutdown_connections 2023-10-08T05:36:40.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fed6006af40 0x7fed6006d400 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fed740afba0 tx=0x7fed68008040 comp rx=0 tx=0).stop 2023-10-08T05:36:40.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed74161690 0x7fed740c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed740cb150 0x7fed740c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 --2- 172.21.15.70:0/3462741565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed740702b0 0x7fed74161ce0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 >> 172.21.15.70:0/3462741565 conn(0x7fed7406a5d0 msgr2=0x7fed740706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.449+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 shutdown_connections 2023-10-08T05:36:40.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.450+0000 7fed79d5a640 1 -- 172.21.15.70:0/3462741565 wait complete. 2023-10-08T05:36:40.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.459+0000 7f797359e640 1 -- 172.21.15.70:0/1101682288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f796c005330 con 0x7f796c038540 2023-10-08T05:36:40.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.460+0000 7f7962ffd640 1 -- 172.21.15.70:0/1101682288 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f796801f020 con 0x7f796c038540 2023-10-08T05:36:40.462 INFO:teuthology.orchestra.run.smithi070.stdout:55834574857 2023-10-08T05:36:40.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.464+0000 7f797359e640 1 -- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f795406af40 msgr2=0x7f795406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.464+0000 7f797359e640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f795406af40 0x7f795406d400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f796c050850 tx=0x7f79740b2000 comp rx=0 tx=0).stop 2023-10-08T05:36:40.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.464+0000 7f797359e640 1 -- 172.21.15.70:0/1101682288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796c038540 msgr2=0x7f796c04a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.464+0000 7f797359e640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796c038540 0x7f796c04a850 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f796800c7f0 tx=0x7f796800cbf0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.465+0000 7f797359e640 1 -- 172.21.15.70:0/1101682288 shutdown_connections 2023-10-08T05:36:40.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.465+0000 7f797359e640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f795406af40 0x7f795406d400 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.465+0000 7f797359e640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f796c0e2960 0x7f796c168ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.465+0000 7f797359e640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796c038540 0x7f796c04a850 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.465+0000 7f797359e640 1 --2- 172.21.15.70:0/1101682288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f796c00c0d0 0x7f796c04e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.465+0000 7f797359e640 1 -- 172.21.15.70:0/1101682288 >> 172.21.15.70:0/1101682288 conn(0x7f796c009fe0 msgr2=0x7f796c036600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.466+0000 7f797359e640 1 -- 172.21.15.70:0/1101682288 shutdown_connections 2023-10-08T05:36:40.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.466+0000 7f797359e640 1 -- 172.21.15.70:0/1101682288 wait complete. 2023-10-08T05:36:40.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-10-08T05:36:40.475 DEBUG:teuthology.parallel:result is None 2023-10-08T05:36:40.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-10-08T05:36:40.478 DEBUG:teuthology.parallel:result is None 2023-10-08T05:36:40.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-10-08T05:36:40.495 DEBUG:teuthology.parallel:result is None 2023-10-08T05:36:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.495+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe5441547a0 con 0x7fe5440ccd20 2023-10-08T05:36:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.495+0000 7f7560ff9640 1 -- 172.21.15.70:0/2174680432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f754c005500 con 0x7f756c0e4740 2023-10-08T05:36:40.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.495+0000 7f7562ffd640 1 -- 172.21.15.70:0/2174680432 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f756407a080 con 0x7f756c0e4740 2023-10-08T05:36:40.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.495+0000 7fe53a7fc640 1 -- 172.21.15.70:0/308753389 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe534019090 con 0x7fe5440ccd20 2023-10-08T05:36:40.497 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:40.497 INFO:teuthology.orchestra.run.smithi070.stdout:55834574856 2023-10-08T05:36:40.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.497+0000 7f7560ff9640 1 -- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f755406af40 msgr2=0x7f755406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.497+0000 7f7560ff9640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f755406af40 0x7f755406d400 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7574074bc0 tx=0x7f757406d040 comp rx=0 tx=0).stop 2023-10-08T05:36:40.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.497+0000 7f7560ff9640 1 -- 172.21.15.70:0/2174680432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0e4740 msgr2=0x7f756c1a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.497+0000 7f7560ff9640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0e4740 0x7f756c1a2e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7564033000 tx=0x7f75640046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7f7560ff9640 1 -- 172.21.15.70:0/2174680432 shutdown_connections 2023-10-08T05:36:40.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7f7560ff9640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f755406af40 0x7f755406d400 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7f7560ff9640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756c0e4740 0x7f756c1a2e00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7f7560ff9640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756c038fb0 0x7f756c1a28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7f7560ff9640 1 --2- 172.21.15.70:0/2174680432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f756c035cc0 0x7f756c0e50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7f7560ff9640 1 -- 172.21.15.70:0/2174680432 >> 172.21.15.70:0/2174680432 conn(0x7f756c0538a0 msgr2=0x7f756c0d9950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7f7560ff9640 1 -- 172.21.15.70:0/2174680432 shutdown_connections 2023-10-08T05:36:40.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7f7560ff9640 1 -- 172.21.15.70:0/2174680432 wait complete. 2023-10-08T05:36:40.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe52806aea0 msgr2=0x7fe52806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7fe54bd6c640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe52806aea0 0x7fe52806d360 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe5440cd290 tx=0x7fe540030000 comp rx=0 tx=0).stop 2023-10-08T05:36:40.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5440ccd20 msgr2=0x7fe5440c7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.498+0000 7fe54bd6c640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5440ccd20 0x7fe5440c7a50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe54406e7c0 tx=0x7fe53400bb30 comp rx=0 tx=0).stop 2023-10-08T05:36:40.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.499+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 shutdown_connections 2023-10-08T05:36:40.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.499+0000 7fe54bd6c640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe52806aea0 0x7fe52806d360 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.499+0000 7fe54bd6c640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5441540e0 0x7fe544157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.499+0000 7fe54bd6c640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5440ccd20 0x7fe5440c7a50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.499+0000 7fe54bd6c640 1 --2- 172.21.15.70:0/308753389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5440cc940 0x7fe5440c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.499+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 >> 172.21.15.70:0/308753389 conn(0x7fe54406a5d0 msgr2=0x7fe544158970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.499+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 shutdown_connections 2023-10-08T05:36:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.499+0000 7fe54bd6c640 1 -- 172.21.15.70:0/308753389 wait complete. 2023-10-08T05:36:40.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.506+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f60380cdc60 con 0x7f6038161170 2023-10-08T05:36:40.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.507+0000 7f60357fa640 1 -- 172.21.15.70:0/2052257439 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6030018020 con 0x7f6038161170 2023-10-08T05:36:40.508 INFO:teuthology.orchestra.run.smithi070.stdout:55834574857 2023-10-08T05:36:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f602406af40 msgr2=0x7f602406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f602406af40 0x7f602406d400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f602802b6c0 tx=0x7f6028008030 comp rx=0 tx=0).stop 2023-10-08T05:36:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6038161170 msgr2=0x7f603814c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6038161170 0x7f603814c0b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f603000cb30 tx=0x7f603000a060 comp rx=0 tx=0).stop 2023-10-08T05:36:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 shutdown_connections 2023-10-08T05:36:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f602406af40 0x7f602406d400 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60381622f0 0x7f603814e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6038161170 0x7f603814c0b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 --2- 172.21.15.70:0/2052257439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60380cd2d0 0x7f603814f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 >> 172.21.15.70:0/2052257439 conn(0x7f60380cf700 msgr2=0x7f6038164000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 shutdown_connections 2023-10-08T05:36:40.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.509+0000 7f603ee46640 1 -- 172.21.15.70:0/2052257439 wait complete. 2023-10-08T05:36:40.516 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.4 2023-10-08T05:36:40.516 DEBUG:teuthology.parallel:result is None 2023-10-08T05:36:40.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.518+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3108172940 con 0x7f31080ca220 2023-10-08T05:36:40.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.519+0000 7f30fe7fc640 1 -- 172.21.15.70:0/1973608817 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3108172940 con 0x7f31080ca220 2023-10-08T05:36:40.520 INFO:teuthology.orchestra.run.smithi070.stdout:55834574858 2023-10-08T05:36:40.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.521+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f30ec06aec0 msgr2=0x7f30ec06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.521+0000 7f310fecf640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f30ec06aec0 0x7f30ec06d380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f30f802b9e0 tx=0x7f30f8008030 comp rx=0 tx=0).stop 2023-10-08T05:36:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.521+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31080ca220 msgr2=0x7f310816a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.521+0000 7f310fecf640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31080ca220 0x7f310816a430 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3100007dc0 tx=0x7f310000d4a0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.522+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 shutdown_connections 2023-10-08T05:36:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.522+0000 7f310fecf640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f30ec06aec0 0x7f30ec06d380 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.522+0000 7f310fecf640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31080cae90 0x7f3108156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.522+0000 7f310fecf640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31080ca220 0x7f310816a430 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.522+0000 7f310fecf640 1 --2- 172.21.15.70:0/1973608817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310806a460 0x7f310815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.522+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 >> 172.21.15.70:0/1973608817 conn(0x7f31080660e0 msgr2=0x7f31080c2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.522+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 shutdown_connections 2023-10-08T05:36:40.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.522+0000 7f310fecf640 1 -- 172.21.15.70:0/1973608817 wait complete. 2023-10-08T05:36:40.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2023-10-08T05:36:40.526 DEBUG:teuthology.parallel:result is None 2023-10-08T05:36:40.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-10-08T05:36:40.527 DEBUG:teuthology.parallel:result is None 2023-10-08T05:36:40.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2023-10-08T05:36:40.539 DEBUG:teuthology.parallel:result is None 2023-10-08T05:36:40.539 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-08T05:36:40.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:36:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.790+0000 7fc2b9590640 1 Processor -- start 2023-10-08T05:36:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.790+0000 7fc2b9590640 1 -- start start 2023-10-08T05:36:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.793+0000 7fc2b9590640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2b40af8b0 0x7fc2b406ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.793+0000 7fc2b9590640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2b40b0ab0 0x7fc2b40b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.793+0000 7fc2b9590640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b40b0340 con 0x7fc2b406b520 2023-10-08T05:36:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.793+0000 7fc2b9590640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b406b3a0 con 0x7fc2b40b0ab0 2023-10-08T05:36:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.793+0000 7fc2b9590640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40b0150 con 0x7fc2b40af8b0 2023-10-08T05:36:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.793+0000 7fc2b2ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc2b406b520 0x7fc2b406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43752/0 (socket says 172.21.15.70:43752) 2023-10-08T05:36:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.793+0000 7fc2b2ffd640 1 -- 172.21.15.70:0/3323805109 learned_addr learned my addr 172.21.15.70:0/3323805109 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2abfff640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2b40af8b0 0x7fc2b406ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978738434 0 0) 0x7fc2b40b0340 con 0x7fc2b406b520 2023-10-08T05:36:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a0003520 con 0x7fc2b406b520 2023-10-08T05:36:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b37fe640 1 --2- 172.21.15.70:0/3323805109 >> v2:172.21.15.70:3300/0 conn(0x7fc2b40b0ab0 0x7fc2b40b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2587251248 0 0) 0x7fc2a0003520 con 0x7fc2b406b520 2023-10-08T05:36:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2b40af8b0 msgr2=0x7fc2b406ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2b40af8b0 0x7fc2b406ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 >> v2:172.21.15.70:3300/0 conn(0x7fc2b40b0ab0 msgr2=0x7fc2b40b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 --2- 172.21.15.70:0/3323805109 >> v2:172.21.15.70:3300/0 conn(0x7fc2b40b0ab0 0x7fc2b40b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40cf8f0 con 0x7fc2b406b520 2023-10-08T05:36:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc2a40017f0 con 0x7fc2b406b520 2023-10-08T05:36:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 >> v1:172.21.15.70:6789/0 conn(0x7fc2b406b520 legacy=0x7fc2b406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a0003970 0x7fc2a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a00047c0 0x7fc2a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2b40af8b0 0x7fc2a00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b406b3a0 con 0x7fc2a0003970 2023-10-08T05:36:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40b0150 con 0x7fc2a00047c0 2023-10-08T05:36:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.794+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a000bbc0 con 0x7fc2b40af8b0 2023-10-08T05:36:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.795+0000 7fc2b37fe640 1 --2- 172.21.15.70:0/3323805109 >> v2:172.21.15.70:3300/0 conn(0x7fc2b40b0ab0 0x7fc2b40b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.795+0000 7fc2b2ffd640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a0003970 0x7fc2a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.795+0000 7fc2abfff640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a00047c0 0x7fc2a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.795+0000 7fc2b37fe640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2b40af8b0 0x7fc2a00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.796+0000 7fc2b2ffd640 1 -- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2b40af8b0 msgr2=0x7fc2a00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.796+0000 7fc2b2ffd640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2b40af8b0 0x7fc2a00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.796+0000 7fc2b2ffd640 1 -- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a00047c0 msgr2=0x7fc2a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:40.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.796+0000 7fc2b2ffd640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2a00047c0 0x7fc2a0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.796+0000 7fc2b2ffd640 1 -- 172.21.15.70:0/3323805109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0003520 con 0x7fc2a0003970 2023-10-08T05:36:40.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.797+0000 7fc2b2ffd640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a0003970 0x7fc2a0003d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400c110 tx=0x7fc2a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82fb02ee4f468d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.797+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4007270 con 0x7fc2a0003970 2023-10-08T05:36:40.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.797+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a40073f0 con 0x7fc2a0003970 2023-10-08T05:36:40.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.797+0000 7fc2b27fc640 1 -- 172.21.15.70:0/3323805109 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a402c830 con 0x7fc2a0003970 2023-10-08T05:36:40.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.798+0000 7fc2b9590640 1 -- 172.21.15.70:0/3323805109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a0003970 msgr2=0x7fc2a0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.798+0000 7fc2b9590640 1 --2- 172.21.15.70:0/3323805109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a0003970 0x7fc2a0003d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400c110 tx=0x7fc2a00057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.798+0000 7fc2b37fe640 1 -- 172.21.15.70:0/3323805109 reap_dead start 2023-10-08T05:36:40.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.798+0000 7fc2b9590640 1 -- 172.21.15.70:0/3323805109 shutdown_connections 2023-10-08T05:36:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.798+0000 7fc2b9590640 1 -- 172.21.15.70:0/3323805109 >> 172.21.15.70:0/3323805109 conn(0x7fc2b40d09e0 msgr2=0x7fc2b40d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.798+0000 7fc2b9590640 1 -- 172.21.15.70:0/3323805109 shutdown_connections 2023-10-08T05:36:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.798+0000 7fc2b9590640 1 -- 172.21.15.70:0/3323805109 wait complete. 2023-10-08T05:36:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.800+0000 7fc2b9590640 1 Processor -- start 2023-10-08T05:36:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.800+0000 7fc2b9590640 1 -- start start 2023-10-08T05:36:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.801+0000 7fc2b9590640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a0003970 0x7fc2b4160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.801+0000 7fc2b9590640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b409cd30 0x7fc2b409d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.801+0000 7fc2b9590640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b40c61e0 0x7fc2b40c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.801+0000 7fc2b9590640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b409d6d0 con 0x7fc2b40c61e0 2023-10-08T05:36:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.801+0000 7fc2b9590640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b409a6a0 con 0x7fc2b409cd30 2023-10-08T05:36:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.801+0000 7fc2b9590640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b41f24d0 con 0x7fc2a0003970 2023-10-08T05:36:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.801+0000 7fc2b37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b40c61e0 0x7fc2b40c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.801+0000 7fc2b37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b40c61e0 0x7fc2b40c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60250/0 (socket says 172.21.15.70:60250) 2023-10-08T05:36:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.801+0000 7fc2b37fe640 1 -- 172.21.15.70:0/594898601 learned_addr learned my addr 172.21.15.70:0/594898601 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2abfff640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b409cd30 0x7fc2b409d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2b37fe640 1 -- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a0003970 msgr2=0x7fc2b4160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2b37fe640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a0003970 0x7fc2b4160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2b37fe640 1 -- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b409cd30 msgr2=0x7fc2b409d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2b37fe640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b409cd30 0x7fc2b409d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2b37fe640 1 -- 172.21.15.70:0/594898601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40c6f60 con 0x7fc2b40c61e0 2023-10-08T05:36:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2b37fe640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b40c61e0 0x7fc2b40c6660 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4161980 tx=0x7fc2ac00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2ab7fe640 1 -- 172.21.15.70:0/594898601 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac012070 con 0x7fc2b40c61e0 2023-10-08T05:36:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2ab7fe640 1 -- 172.21.15.70:0/594898601 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac00a670 con 0x7fc2b40c61e0 2023-10-08T05:36:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2ab7fe640 1 -- 172.21.15.70:0/594898601 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac00aa20 con 0x7fc2b40c61e0 2023-10-08T05:36:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2b9590640 1 -- 172.21.15.70:0/594898601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b409a8f0 con 0x7fc2b40c61e0 2023-10-08T05:36:40.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.802+0000 7fc2b9590640 1 -- 172.21.15.70:0/594898601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b409aeb0 con 0x7fc2b40c61e0 2023-10-08T05:36:40.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.804+0000 7fc2b9590640 1 -- 172.21.15.70:0/594898601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b409b0d0 con 0x7fc2b40c61e0 2023-10-08T05:36:40.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.804+0000 7fc2ab7fe640 1 -- 172.21.15.70:0/594898601 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc2ac00ab80 con 0x7fc2b40c61e0 2023-10-08T05:36:40.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.805+0000 7fc2ab7fe640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc29406aea0 0x7fc29406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:40.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.805+0000 7fc2ab7fe640 1 -- 172.21.15.70:0/594898601 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc2ac08aa30 con 0x7fc2b40c61e0 2023-10-08T05:36:40.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.806+0000 7fc2b2ffd640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc29406aea0 0x7fc29406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:40.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.807+0000 7fc2b2ffd640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc29406aea0 0x7fc29406d360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4034000 tx=0x7fc2a4007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:40.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.810+0000 7fc2ab7fe640 1 -- 172.21.15.70:0/594898601 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2ac055880 con 0x7fc2b40c61e0 2023-10-08T05:36:40.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:40.982+0000 7fc2b9590640 1 -- 172.21.15.70:0/594898601 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2b40cfeb0 con 0x7fc29406aea0 2023-10-08T05:36:41.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.011+0000 7fc2ab7fe640 1 -- 172.21.15.70:0/594898601 <== mgr.4105 v2:172.21.15.181:6832/2737793492 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345114 (secure 0 0 0) 0x7fc2a4007070 con 0x7fc29406aea0 2023-10-08T05:36:41.016 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:41.018 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:36:41.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 -- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc29406aea0 msgr2=0x7fc29406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc29406aea0 0x7fc29406d360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4034000 tx=0x7fc2a4007000 comp rx=0 tx=0).stop 2023-10-08T05:36:41.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 -- 172.21.15.70:0/594898601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b40c61e0 msgr2=0x7fc2b40c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b40c61e0 0x7fc2b40c6660 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4161980 tx=0x7fc2ac00c810 comp rx=0 tx=0).stop 2023-10-08T05:36:41.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 -- 172.21.15.70:0/594898601 shutdown_connections 2023-10-08T05:36:41.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc29406aea0 0x7fc29406d360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2b40c61e0 0x7fc2b40c6660 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc2b409cd30 0x7fc2b409d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 --2- 172.21.15.70:0/594898601 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2a0003970 0x7fc2b4160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 -- 172.21.15.70:0/594898601 >> 172.21.15.70:0/594898601 conn(0x7fc2b40d09e0 msgr2=0x7fc2b406f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:41.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 -- 172.21.15.70:0/594898601 shutdown_connections 2023-10-08T05:36:41.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.016+0000 7fc2b9590640 1 -- 172.21.15.70:0/594898601 wait complete. 2023-10-08T05:36:41.038 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-10-08T05:36:39.079639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236096,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738624,"statfs":{"total":773094113280,"available":772852350976,"internally_reserved":0,"allocated":3383296,"data_stored":2025622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004816"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205680+0000","last_change":"2023-10-08T05:36:11.183116+0000","last_active":"2023-10-08T05:36:16.205680+0000","last_peered":"2023-10-08T05:36:16.205680+0000","last_clean":"2023-10-08T05:36:16.205680+0000","last_became_active":"2023-10-08T05:36:11.182654+0000","last_became_peered":"2023-10-08T05:36:11.182654+0000","last_unstale":"2023-10-08T05:36:16.205680+0000","last_undegraded":"2023-10-08T05:36:16.205680+0000","last_fullsized":"2023-10-08T05:36:16.205680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:49.167371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198007+0000","last_change":"2023-10-08T05:36:11.938929+0000","last_active":"2023-10-08T05:36:16.198007+0000","last_peered":"2023-10-08T05:36:16.198007+0000","last_clean":"2023-10-08T05:36:16.198007+0000","last_became_active":"2023-10-08T05:36:11.938616+0000","last_became_peered":"2023-10-08T05:36:11.938616+0000","last_unstale":"2023-10-08T05:36:16.198007+0000","last_undegraded":"2023-10-08T05:36:16.198007+0000","last_fullsized":"2023-10-08T05:36:16.198007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.911637+0000","last_change":"2023-10-08T05:36:16.241991+0000","last_active":"2023-10-08T05:36:16.911637+0000","last_peered":"2023-10-08T05:36:16.911637+0000","last_clean":"2023-10-08T05:36:16.911637+0000","last_became_active":"2023-10-08T05:36:16.241629+0000","last_became_peered":"2023-10-08T05:36:16.241629+0000","last_unstale":"2023-10-08T05:36:16.911637+0000","last_undegraded":"2023-10-08T05:36:16.911637+0000","last_fullsized":"2023-10-08T05:36:16.911637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:41.216984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198078+0000","last_change":"2023-10-08T05:36:11.171819+0000","last_active":"2023-10-08T05:36:16.198078+0000","last_peered":"2023-10-08T05:36:16.198078+0000","last_clean":"2023-10-08T05:36:16.198078+0000","last_became_active":"2023-10-08T05:36:11.171384+0000","last_became_peered":"2023-10-08T05:36:11.171384+0000","last_unstale":"2023-10-08T05:36:16.198078+0000","last_undegraded":"2023-10-08T05:36:16.198078+0000","last_fullsized":"2023-10-08T05:36:16.198078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:08.838858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205587+0000","last_change":"2023-10-08T05:36:11.939107+0000","last_active":"2023-10-08T05:36:16.205587+0000","last_peered":"2023-10-08T05:36:16.205587+0000","last_clean":"2023-10-08T05:36:16.205587+0000","last_became_active":"2023-10-08T05:36:11.938788+0000","last_became_peered":"2023-10-08T05:36:11.938788+0000","last_unstale":"2023-10-08T05:36:16.205587+0000","last_undegraded":"2023-10-08T05:36:16.205587+0000","last_fullsized":"2023-10-08T05:36:16.205587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450376+0000","last_change":"2023-10-08T05:36:16.220918+0000","last_active":"2023-10-08T05:36:18.450376+0000","last_peered":"2023-10-08T05:36:18.450376+0000","last_clean":"2023-10-08T05:36:18.450376+0000","last_became_active":"2023-10-08T05:36:16.220450+0000","last_became_peered":"2023-10-08T05:36:16.220450+0000","last_unstale":"2023-10-08T05:36:18.450376+0000","last_undegraded":"2023-10-08T05:36:18.450376+0000","last_fullsized":"2023-10-08T05:36:18.450376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:45.730636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239438+0000","last_change":"2023-10-08T05:36:11.187132+0000","last_active":"2023-10-08T05:36:16.239438+0000","last_peered":"2023-10-08T05:36:16.239438+0000","last_clean":"2023-10-08T05:36:16.239438+0000","last_became_active":"2023-10-08T05:36:11.186856+0000","last_became_peered":"2023-10-08T05:36:11.186856+0000","last_unstale":"2023-10-08T05:36:16.239438+0000","last_undegraded":"2023-10-08T05:36:16.239438+0000","last_fullsized":"2023-10-08T05:36:16.239438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:42.738249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215030+0000","last_change":"2023-10-08T05:36:11.946617+0000","last_active":"2023-10-08T05:36:16.215030+0000","last_peered":"2023-10-08T05:36:16.215030+0000","last_clean":"2023-10-08T05:36:16.215030+0000","last_became_active":"2023-10-08T05:36:11.946360+0000","last_became_peered":"2023-10-08T05:36:11.946360+0000","last_unstale":"2023-10-08T05:36:16.215030+0000","last_undegraded":"2023-10-08T05:36:16.215030+0000","last_fullsized":"2023-10-08T05:36:16.215030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:21.717397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450713+0000","last_change":"2023-10-08T05:36:16.229002+0000","last_active":"2023-10-08T05:36:18.450713+0000","last_peered":"2023-10-08T05:36:18.450713+0000","last_clean":"2023-10-08T05:36:18.450713+0000","last_became_active":"2023-10-08T05:36:16.228230+0000","last_became_peered":"2023-10-08T05:36:16.228230+0000","last_unstale":"2023-10-08T05:36:18.450713+0000","last_undegraded":"2023-10-08T05:36:18.450713+0000","last_fullsized":"2023-10-08T05:36:18.450713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:33.520862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239150+0000","last_change":"2023-10-08T05:36:11.191776+0000","last_active":"2023-10-08T05:36:16.239150+0000","last_peered":"2023-10-08T05:36:16.239150+0000","last_clean":"2023-10-08T05:36:16.239150+0000","last_became_active":"2023-10-08T05:36:11.191532+0000","last_became_peered":"2023-10-08T05:36:11.191532+0000","last_unstale":"2023-10-08T05:36:16.239150+0000","last_undegraded":"2023-10-08T05:36:16.239150+0000","last_fullsized":"2023-10-08T05:36:16.239150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:16.799817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.217226+0000","last_change":"2023-10-08T05:36:11.950652+0000","last_active":"2023-10-08T05:36:16.217226+0000","last_peered":"2023-10-08T05:36:16.217226+0000","last_clean":"2023-10-08T05:36:16.217226+0000","last_became_active":"2023-10-08T05:36:11.950359+0000","last_became_peered":"2023-10-08T05:36:11.950359+0000","last_unstale":"2023-10-08T05:36:16.217226+0000","last_undegraded":"2023-10-08T05:36:16.217226+0000","last_fullsized":"2023-10-08T05:36:16.217226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451049+0000","last_change":"2023-10-08T05:36:16.222866+0000","last_active":"2023-10-08T05:36:18.451049+0000","last_peered":"2023-10-08T05:36:18.451049+0000","last_clean":"2023-10-08T05:36:18.451049+0000","last_became_active":"2023-10-08T05:36:16.222444+0000","last_became_peered":"2023-10-08T05:36:16.222444+0000","last_unstale":"2023-10-08T05:36:18.451049+0000","last_undegraded":"2023-10-08T05:36:18.451049+0000","last_fullsized":"2023-10-08T05:36:18.451049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:04.460179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195161+0000","last_change":"2023-10-08T05:36:11.936259+0000","last_active":"2023-10-08T05:36:16.195161+0000","last_peered":"2023-10-08T05:36:16.195161+0000","last_clean":"2023-10-08T05:36:16.195161+0000","last_became_active":"2023-10-08T05:36:11.935923+0000","last_became_peered":"2023-10-08T05:36:11.935923+0000","last_unstale":"2023-10-08T05:36:16.195161+0000","last_undegraded":"2023-10-08T05:36:16.195161+0000","last_fullsized":"2023-10-08T05:36:16.195161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239194+0000","last_change":"2023-10-08T05:36:11.187292+0000","last_active":"2023-10-08T05:36:16.239194+0000","last_peered":"2023-10-08T05:36:16.239194+0000","last_clean":"2023-10-08T05:36:16.239194+0000","last_became_active":"2023-10-08T05:36:11.186706+0000","last_became_peered":"2023-10-08T05:36:11.186706+0000","last_unstale":"2023-10-08T05:36:16.239194+0000","last_undegraded":"2023-10-08T05:36:16.239194+0000","last_fullsized":"2023-10-08T05:36:16.239194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:22.361671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451570+0000","last_change":"2023-10-08T05:36:16.228736+0000","last_active":"2023-10-08T05:36:18.451570+0000","last_peered":"2023-10-08T05:36:18.451570+0000","last_clean":"2023-10-08T05:36:18.451570+0000","last_became_active":"2023-10-08T05:36:16.227909+0000","last_became_peered":"2023-10-08T05:36:16.227909+0000","last_unstale":"2023-10-08T05:36:18.451570+0000","last_undegraded":"2023-10-08T05:36:18.451570+0000","last_fullsized":"2023-10-08T05:36:18.451570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:29.122804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194544+0000","last_change":"2023-10-08T05:36:11.945361+0000","last_active":"2023-10-08T05:36:16.194544+0000","last_peered":"2023-10-08T05:36:16.194544+0000","last_clean":"2023-10-08T05:36:16.194544+0000","last_became_active":"2023-10-08T05:36:11.944819+0000","last_became_peered":"2023-10-08T05:36:11.944819+0000","last_unstale":"2023-10-08T05:36:16.194544+0000","last_undegraded":"2023-10-08T05:36:16.194544+0000","last_fullsized":"2023-10-08T05:36:16.194544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:56.745041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.208276+0000","last_change":"2023-10-08T05:36:11.164749+0000","last_active":"2023-10-08T05:36:16.208276+0000","last_peered":"2023-10-08T05:36:16.208276+0000","last_clean":"2023-10-08T05:36:16.208276+0000","last_became_active":"2023-10-08T05:36:11.164382+0000","last_became_peered":"2023-10-08T05:36:11.164382+0000","last_unstale":"2023-10-08T05:36:16.208276+0000","last_undegraded":"2023-10-08T05:36:16.208276+0000","last_fullsized":"2023-10-08T05:36:16.208276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:13.509993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450966+0000","last_change":"2023-10-08T05:36:16.229430+0000","last_active":"2023-10-08T05:36:18.450966+0000","last_peered":"2023-10-08T05:36:18.450966+0000","last_clean":"2023-10-08T05:36:18.450966+0000","last_became_active":"2023-10-08T05:36:16.228957+0000","last_became_peered":"2023-10-08T05:36:16.228957+0000","last_unstale":"2023-10-08T05:36:18.450966+0000","last_undegraded":"2023-10-08T05:36:18.450966+0000","last_fullsized":"2023-10-08T05:36:18.450966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:00.782220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195462+0000","last_change":"2023-10-08T05:36:11.934612+0000","last_active":"2023-10-08T05:36:16.195462+0000","last_peered":"2023-10-08T05:36:16.195462+0000","last_clean":"2023-10-08T05:36:16.195462+0000","last_became_active":"2023-10-08T05:36:11.934284+0000","last_became_peered":"2023-10-08T05:36:11.934284+0000","last_unstale":"2023-10-08T05:36:16.195462+0000","last_undegraded":"2023-10-08T05:36:16.195462+0000","last_fullsized":"2023-10-08T05:36:16.195462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203211+0000","last_change":"2023-10-08T05:36:11.182216+0000","last_active":"2023-10-08T05:36:16.203211+0000","last_peered":"2023-10-08T05:36:16.203211+0000","last_clean":"2023-10-08T05:36:16.203211+0000","last_became_active":"2023-10-08T05:36:11.181927+0000","last_became_peered":"2023-10-08T05:36:11.181927+0000","last_unstale":"2023-10-08T05:36:16.203211+0000","last_undegraded":"2023-10-08T05:36:16.203211+0000","last_fullsized":"2023-10-08T05:36:16.203211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:42.738249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.917366+0000","last_change":"2023-10-08T05:36:16.243897+0000","last_active":"2023-10-08T05:36:16.917366+0000","last_peered":"2023-10-08T05:36:16.917366+0000","last_clean":"2023-10-08T05:36:16.917366+0000","last_became_active":"2023-10-08T05:36:16.243421+0000","last_became_peered":"2023-10-08T05:36:16.243421+0000","last_unstale":"2023-10-08T05:36:16.917366+0000","last_undegraded":"2023-10-08T05:36:16.917366+0000","last_fullsized":"2023-10-08T05:36:16.917366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:16.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.209892+0000","last_change":"2023-10-08T05:36:11.934209+0000","last_active":"2023-10-08T05:36:16.209892+0000","last_peered":"2023-10-08T05:36:16.209892+0000","last_clean":"2023-10-08T05:36:16.209892+0000","last_became_active":"2023-10-08T05:36:11.933067+0000","last_became_peered":"2023-10-08T05:36:11.933067+0000","last_unstale":"2023-10-08T05:36:16.209892+0000","last_undegraded":"2023-10-08T05:36:16.209892+0000","last_fullsized":"2023-10-08T05:36:16.209892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:17.827053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238717+0000","last_change":"2023-10-08T05:36:11.181498+0000","last_active":"2023-10-08T05:36:16.238717+0000","last_peered":"2023-10-08T05:36:16.238717+0000","last_clean":"2023-10-08T05:36:16.238717+0000","last_became_active":"2023-10-08T05:36:11.181264+0000","last_became_peered":"2023-10-08T05:36:11.181264+0000","last_unstale":"2023-10-08T05:36:16.238717+0000","last_undegraded":"2023-10-08T05:36:16.238717+0000","last_fullsized":"2023-10-08T05:36:16.238717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:35.005158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.452011+0000","last_change":"2023-10-08T05:36:16.228065+0000","last_active":"2023-10-08T05:36:18.452011+0000","last_peered":"2023-10-08T05:36:18.452011+0000","last_clean":"2023-10-08T05:36:18.452011+0000","last_became_active":"2023-10-08T05:36:16.227665+0000","last_became_peered":"2023-10-08T05:36:16.227665+0000","last_unstale":"2023-10-08T05:36:18.452011+0000","last_undegraded":"2023-10-08T05:36:18.452011+0000","last_fullsized":"2023-10-08T05:36:18.452011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:32.619157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205947+0000","last_change":"2023-10-08T05:36:11.934150+0000","last_active":"2023-10-08T05:36:16.205947+0000","last_peered":"2023-10-08T05:36:16.205947+0000","last_clean":"2023-10-08T05:36:16.205947+0000","last_became_active":"2023-10-08T05:36:11.933875+0000","last_became_peered":"2023-10-08T05:36:11.933875+0000","last_unstale":"2023-10-08T05:36:16.205947+0000","last_undegraded":"2023-10-08T05:36:16.205947+0000","last_fullsized":"2023-10-08T05:36:16.205947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196218+0000","last_change":"2023-10-08T05:36:11.187989+0000","last_active":"2023-10-08T05:36:16.196218+0000","last_peered":"2023-10-08T05:36:16.196218+0000","last_clean":"2023-10-08T05:36:16.196218+0000","last_became_active":"2023-10-08T05:36:11.187695+0000","last_became_peered":"2023-10-08T05:36:11.187695+0000","last_unstale":"2023-10-08T05:36:16.196218+0000","last_undegraded":"2023-10-08T05:36:16.196218+0000","last_fullsized":"2023-10-08T05:36:16.196218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:35.005158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451149+0000","last_change":"2023-10-08T05:36:16.244139+0000","last_active":"2023-10-08T05:36:18.451149+0000","last_peered":"2023-10-08T05:36:18.451149+0000","last_clean":"2023-10-08T05:36:18.451149+0000","last_became_active":"2023-10-08T05:36:16.243569+0000","last_became_peered":"2023-10-08T05:36:16.243569+0000","last_unstale":"2023-10-08T05:36:18.451149+0000","last_undegraded":"2023-10-08T05:36:18.451149+0000","last_fullsized":"2023-10-08T05:36:18.451149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:30.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198170+0000","last_change":"2023-10-08T05:36:11.941731+0000","last_active":"2023-10-08T05:36:16.198170+0000","last_peered":"2023-10-08T05:36:16.198170+0000","last_clean":"2023-10-08T05:36:16.198170+0000","last_became_active":"2023-10-08T05:36:11.941393+0000","last_became_peered":"2023-10-08T05:36:11.941393+0000","last_unstale":"2023-10-08T05:36:16.198170+0000","last_undegraded":"2023-10-08T05:36:16.198170+0000","last_fullsized":"2023-10-08T05:36:16.198170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207549+0000","last_change":"2023-10-08T05:36:11.180784+0000","last_active":"2023-10-08T05:36:16.207549+0000","last_peered":"2023-10-08T05:36:16.207549+0000","last_clean":"2023-10-08T05:36:16.207549+0000","last_became_active":"2023-10-08T05:36:11.180398+0000","last_became_peered":"2023-10-08T05:36:11.180398+0000","last_unstale":"2023-10-08T05:36:16.207549+0000","last_undegraded":"2023-10-08T05:36:16.207549+0000","last_fullsized":"2023-10-08T05:36:16.207549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:49.167371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913389+0000","last_change":"2023-10-08T05:36:16.240627+0000","last_active":"2023-10-08T05:36:16.913389+0000","last_peered":"2023-10-08T05:36:16.913389+0000","last_clean":"2023-10-08T05:36:16.913389+0000","last_became_active":"2023-10-08T05:36:16.240153+0000","last_became_peered":"2023-10-08T05:36:16.240153+0000","last_unstale":"2023-10-08T05:36:16.913389+0000","last_undegraded":"2023-10-08T05:36:16.913389+0000","last_fullsized":"2023-10-08T05:36:16.913389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:06.106435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207447+0000","last_change":"2023-10-08T05:36:11.941834+0000","last_active":"2023-10-08T05:36:16.207447+0000","last_peered":"2023-10-08T05:36:16.207447+0000","last_clean":"2023-10-08T05:36:16.207447+0000","last_became_active":"2023-10-08T05:36:11.941546+0000","last_became_peered":"2023-10-08T05:36:11.941546+0000","last_unstale":"2023-10-08T05:36:16.207447+0000","last_undegraded":"2023-10-08T05:36:16.207447+0000","last_fullsized":"2023-10-08T05:36:16.207447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239749+0000","last_change":"2023-10-08T05:36:11.182530+0000","last_active":"2023-10-08T05:36:16.239749+0000","last_peered":"2023-10-08T05:36:16.239749+0000","last_clean":"2023-10-08T05:36:16.239749+0000","last_became_active":"2023-10-08T05:36:11.182156+0000","last_became_peered":"2023-10-08T05:36:11.182156+0000","last_unstale":"2023-10-08T05:36:16.239749+0000","last_undegraded":"2023-10-08T05:36:16.239749+0000","last_fullsized":"2023-10-08T05:36:16.239749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:10.731974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449389+0000","last_change":"2023-10-08T05:36:16.240659+0000","last_active":"2023-10-08T05:36:18.449389+0000","last_peered":"2023-10-08T05:36:18.449389+0000","last_clean":"2023-10-08T05:36:18.449389+0000","last_became_active":"2023-10-08T05:36:16.240244+0000","last_became_peered":"2023-10-08T05:36:16.240244+0000","last_unstale":"2023-10-08T05:36:18.449389+0000","last_undegraded":"2023-10-08T05:36:18.449389+0000","last_fullsized":"2023-10-08T05:36:18.449389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:15.664459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.216378+0000","last_change":"2023-10-08T05:36:11.943995+0000","last_active":"2023-10-08T05:36:16.216378+0000","last_peered":"2023-10-08T05:36:16.216378+0000","last_clean":"2023-10-08T05:36:16.216378+0000","last_became_active":"2023-10-08T05:36:11.943601+0000","last_became_peered":"2023-10-08T05:36:11.943601+0000","last_unstale":"2023-10-08T05:36:16.216378+0000","last_undegraded":"2023-10-08T05:36:16.216378+0000","last_fullsized":"2023-10-08T05:36:16.216378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:28.581977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207661+0000","last_change":"2023-10-08T05:36:11.166965+0000","last_active":"2023-10-08T05:36:16.207661+0000","last_peered":"2023-10-08T05:36:16.207661+0000","last_clean":"2023-10-08T05:36:16.207661+0000","last_became_active":"2023-10-08T05:36:11.166704+0000","last_became_peered":"2023-10-08T05:36:11.166704+0000","last_unstale":"2023-10-08T05:36:16.207661+0000","last_undegraded":"2023-10-08T05:36:16.207661+0000","last_fullsized":"2023-10-08T05:36:16.207661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.915670+0000","last_change":"2023-10-08T05:36:16.226001+0000","last_active":"2023-10-08T05:36:16.915670+0000","last_peered":"2023-10-08T05:36:16.915670+0000","last_clean":"2023-10-08T05:36:16.915670+0000","last_became_active":"2023-10-08T05:36:16.225623+0000","last_became_peered":"2023-10-08T05:36:16.225623+0000","last_unstale":"2023-10-08T05:36:16.915670+0000","last_undegraded":"2023-10-08T05:36:16.915670+0000","last_fullsized":"2023-10-08T05:36:16.915670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213447+0000","last_change":"2023-10-08T05:36:11.935396+0000","last_active":"2023-10-08T05:36:16.213447+0000","last_peered":"2023-10-08T05:36:16.213447+0000","last_clean":"2023-10-08T05:36:16.213447+0000","last_became_active":"2023-10-08T05:36:11.934642+0000","last_became_peered":"2023-10-08T05:36:11.934642+0000","last_unstale":"2023-10-08T05:36:16.213447+0000","last_undegraded":"2023-10-08T05:36:16.213447+0000","last_fullsized":"2023-10-08T05:36:16.213447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:09.850609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215985+0000","last_change":"2023-10-08T05:36:11.189030+0000","last_active":"2023-10-08T05:36:16.215985+0000","last_peered":"2023-10-08T05:36:16.215985+0000","last_clean":"2023-10-08T05:36:16.215985+0000","last_became_active":"2023-10-08T05:36:11.184720+0000","last_became_peered":"2023-10-08T05:36:11.184720+0000","last_unstale":"2023-10-08T05:36:16.215985+0000","last_undegraded":"2023-10-08T05:36:16.215985+0000","last_fullsized":"2023-10-08T05:36:16.215985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:41.625077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.916396+0000","last_change":"2023-10-08T05:36:16.223249+0000","last_active":"2023-10-08T05:36:16.916396+0000","last_peered":"2023-10-08T05:36:16.916396+0000","last_clean":"2023-10-08T05:36:16.916396+0000","last_became_active":"2023-10-08T05:36:16.222656+0000","last_became_peered":"2023-10-08T05:36:16.222656+0000","last_unstale":"2023-10-08T05:36:16.916396+0000","last_undegraded":"2023-10-08T05:36:16.916396+0000","last_fullsized":"2023-10-08T05:36:16.916396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:25.754577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203206+0000","last_change":"2023-10-08T05:36:11.942804+0000","last_active":"2023-10-08T05:36:16.203206+0000","last_peered":"2023-10-08T05:36:16.203206+0000","last_clean":"2023-10-08T05:36:16.203206+0000","last_became_active":"2023-10-08T05:36:11.942486+0000","last_became_peered":"2023-10-08T05:36:11.942486+0000","last_unstale":"2023-10-08T05:36:16.203206+0000","last_undegraded":"2023-10-08T05:36:16.203206+0000","last_fullsized":"2023-10-08T05:36:16.203206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190110+0000","last_change":"2023-10-08T05:36:11.160795+0000","last_active":"2023-10-08T05:36:16.190110+0000","last_peered":"2023-10-08T05:36:16.190110+0000","last_clean":"2023-10-08T05:36:16.190110+0000","last_became_active":"2023-10-08T05:36:11.160498+0000","last_became_peered":"2023-10-08T05:36:11.160498+0000","last_unstale":"2023-10-08T05:36:16.190110+0000","last_undegraded":"2023-10-08T05:36:16.190110+0000","last_fullsized":"2023-10-08T05:36:16.190110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:08.847794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.917236+0000","last_change":"2023-10-08T05:36:16.224778+0000","last_active":"2023-10-08T05:36:16.917236+0000","last_peered":"2023-10-08T05:36:16.917236+0000","last_clean":"2023-10-08T05:36:16.917236+0000","last_became_active":"2023-10-08T05:36:16.224249+0000","last_became_peered":"2023-10-08T05:36:16.224249+0000","last_unstale":"2023-10-08T05:36:16.917236+0000","last_undegraded":"2023-10-08T05:36:16.917236+0000","last_fullsized":"2023-10-08T05:36:16.917236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:07.031678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.204855+0000","last_change":"2023-10-08T05:36:11.944248+0000","last_active":"2023-10-08T05:36:16.204855+0000","last_peered":"2023-10-08T05:36:16.204855+0000","last_clean":"2023-10-08T05:36:16.204855+0000","last_became_active":"2023-10-08T05:36:11.943941+0000","last_became_peered":"2023-10-08T05:36:11.943941+0000","last_unstale":"2023-10-08T05:36:16.204855+0000","last_undegraded":"2023-10-08T05:36:16.204855+0000","last_fullsized":"2023-10-08T05:36:16.204855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238762+0000","last_change":"2023-10-08T05:36:11.186476+0000","last_active":"2023-10-08T05:36:16.238762+0000","last_peered":"2023-10-08T05:36:16.238762+0000","last_clean":"2023-10-08T05:36:16.238762+0000","last_became_active":"2023-10-08T05:36:11.186165+0000","last_became_peered":"2023-10-08T05:36:11.186165+0000","last_unstale":"2023-10-08T05:36:16.238762+0000","last_undegraded":"2023-10-08T05:36:16.238762+0000","last_fullsized":"2023-10-08T05:36:16.238762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:08.838858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450006+0000","last_change":"2023-10-08T05:36:16.241072+0000","last_active":"2023-10-08T05:36:18.450006+0000","last_peered":"2023-10-08T05:36:18.450006+0000","last_clean":"2023-10-08T05:36:18.450006+0000","last_became_active":"2023-10-08T05:36:16.240154+0000","last_became_peered":"2023-10-08T05:36:16.240154+0000","last_unstale":"2023-10-08T05:36:18.450006+0000","last_undegraded":"2023-10-08T05:36:18.450006+0000","last_fullsized":"2023-10-08T05:36:18.450006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:16.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194060+0000","last_change":"2023-10-08T05:36:11.936616+0000","last_active":"2023-10-08T05:36:16.194060+0000","last_peered":"2023-10-08T05:36:16.194060+0000","last_clean":"2023-10-08T05:36:16.194060+0000","last_became_active":"2023-10-08T05:36:11.936320+0000","last_became_peered":"2023-10-08T05:36:11.936320+0000","last_unstale":"2023-10-08T05:36:16.194060+0000","last_undegraded":"2023-10-08T05:36:16.194060+0000","last_fullsized":"2023-10-08T05:36:16.194060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215473+0000","last_change":"2023-10-08T05:36:11.182263+0000","last_active":"2023-10-08T05:36:16.215473+0000","last_peered":"2023-10-08T05:36:16.215473+0000","last_clean":"2023-10-08T05:36:16.215473+0000","last_became_active":"2023-10-08T05:36:11.181876+0000","last_became_peered":"2023-10-08T05:36:11.181876+0000","last_unstale":"2023-10-08T05:36:16.215473+0000","last_undegraded":"2023-10-08T05:36:16.215473+0000","last_fullsized":"2023-10-08T05:36:16.215473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:16.815302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238841+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:36:16.238841+0000","last_peered":"2023-10-08T05:36:16.238841+0000","last_clean":"2023-10-08T05:36:16.238841+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:36:16.238841+0000","last_undegraded":"2023-10-08T05:36:16.238841+0000","last_fullsized":"2023-10-08T05:36:16.238841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914962+0000","last_change":"2023-10-08T05:36:16.241252+0000","last_active":"2023-10-08T05:36:16.914962+0000","last_peered":"2023-10-08T05:36:16.914962+0000","last_clean":"2023-10-08T05:36:16.914962+0000","last_became_active":"2023-10-08T05:36:16.240543+0000","last_became_peered":"2023-10-08T05:36:16.240543+0000","last_unstale":"2023-10-08T05:36:16.914962+0000","last_undegraded":"2023-10-08T05:36:16.914962+0000","last_fullsized":"2023-10-08T05:36:16.914962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:20.640196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239326+0000","last_change":"2023-10-08T05:36:11.954327+0000","last_active":"2023-10-08T05:36:16.239326+0000","last_peered":"2023-10-08T05:36:16.239326+0000","last_clean":"2023-10-08T05:36:16.239326+0000","last_became_active":"2023-10-08T05:36:11.953852+0000","last_became_peered":"2023-10-08T05:36:11.953852+0000","last_unstale":"2023-10-08T05:36:16.239326+0000","last_undegraded":"2023-10-08T05:36:16.239326+0000","last_fullsized":"2023-10-08T05:36:16.239326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:12.498480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190495+0000","last_change":"2023-10-08T05:36:11.179188+0000","last_active":"2023-10-08T05:36:16.190495+0000","last_peered":"2023-10-08T05:36:16.190495+0000","last_clean":"2023-10-08T05:36:16.190495+0000","last_became_active":"2023-10-08T05:36:11.178872+0000","last_became_peered":"2023-10-08T05:36:11.178872+0000","last_unstale":"2023-10-08T05:36:16.190495+0000","last_undegraded":"2023-10-08T05:36:16.190495+0000","last_fullsized":"2023-10-08T05:36:16.190495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:44.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.915127+0000","last_change":"2023-10-08T05:36:16.241452+0000","last_active":"2023-10-08T05:36:16.915127+0000","last_peered":"2023-10-08T05:36:16.915127+0000","last_clean":"2023-10-08T05:36:16.915127+0000","last_became_active":"2023-10-08T05:36:16.240541+0000","last_became_peered":"2023-10-08T05:36:16.240541+0000","last_unstale":"2023-10-08T05:36:16.915127+0000","last_undegraded":"2023-10-08T05:36:16.915127+0000","last_fullsized":"2023-10-08T05:36:16.915127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215138+0000","last_change":"2023-10-08T05:36:11.948239+0000","last_active":"2023-10-08T05:36:16.215138+0000","last_peered":"2023-10-08T05:36:16.215138+0000","last_clean":"2023-10-08T05:36:16.215138+0000","last_became_active":"2023-10-08T05:36:11.947960+0000","last_became_peered":"2023-10-08T05:36:11.947960+0000","last_unstale":"2023-10-08T05:36:16.215138+0000","last_undegraded":"2023-10-08T05:36:16.215138+0000","last_fullsized":"2023-10-08T05:36:16.215138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:02.994498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207516+0000","last_change":"2023-10-08T05:36:11.173210+0000","last_active":"2023-10-08T05:36:16.207516+0000","last_peered":"2023-10-08T05:36:16.207516+0000","last_clean":"2023-10-08T05:36:16.207516+0000","last_became_active":"2023-10-08T05:36:11.172899+0000","last_became_peered":"2023-10-08T05:36:11.172899+0000","last_unstale":"2023-10-08T05:36:16.207516+0000","last_undegraded":"2023-10-08T05:36:16.207516+0000","last_fullsized":"2023-10-08T05:36:16.207516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:59.559506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449356+0000","last_change":"2023-10-08T05:36:16.226571+0000","last_active":"2023-10-08T05:36:18.449356+0000","last_peered":"2023-10-08T05:36:18.449356+0000","last_clean":"2023-10-08T05:36:18.449356+0000","last_became_active":"2023-10-08T05:36:16.226098+0000","last_became_peered":"2023-10-08T05:36:16.226098+0000","last_unstale":"2023-10-08T05:36:18.449356+0000","last_undegraded":"2023-10-08T05:36:18.449356+0000","last_fullsized":"2023-10-08T05:36:18.449356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.246459+0000","last_change":"2023-10-08T05:36:11.948021+0000","last_active":"2023-10-08T05:36:16.246459+0000","last_peered":"2023-10-08T05:36:16.246459+0000","last_clean":"2023-10-08T05:36:16.246459+0000","last_became_active":"2023-10-08T05:36:11.947456+0000","last_became_peered":"2023-10-08T05:36:11.947456+0000","last_unstale":"2023-10-08T05:36:16.246459+0000","last_undegraded":"2023-10-08T05:36:16.246459+0000","last_fullsized":"2023-10-08T05:36:16.246459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:21.717397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.191082+0000","last_change":"2023-10-08T05:36:11.164907+0000","last_active":"2023-10-08T05:36:16.191082+0000","last_peered":"2023-10-08T05:36:16.191082+0000","last_clean":"2023-10-08T05:36:16.191082+0000","last_became_active":"2023-10-08T05:36:11.164630+0000","last_became_peered":"2023-10-08T05:36:11.164630+0000","last_unstale":"2023-10-08T05:36:16.191082+0000","last_undegraded":"2023-10-08T05:36:16.191082+0000","last_fullsized":"2023-10-08T05:36:16.191082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:58.268025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.347745+0000","last_change":"2023-10-08T05:36:16.241024+0000","last_active":"2023-10-08T05:36:17.347745+0000","last_peered":"2023-10-08T05:36:17.347745+0000","last_clean":"2023-10-08T05:36:17.347745+0000","last_became_active":"2023-10-08T05:36:16.240379+0000","last_became_peered":"2023-10-08T05:36:16.240379+0000","last_unstale":"2023-10-08T05:36:17.347745+0000","last_undegraded":"2023-10-08T05:36:17.347745+0000","last_fullsized":"2023-10-08T05:36:17.347745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:09:12.638961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240155+0000","last_change":"2023-10-08T05:36:11.946599+0000","last_active":"2023-10-08T05:36:16.240155+0000","last_peered":"2023-10-08T05:36:16.240155+0000","last_clean":"2023-10-08T05:36:16.240155+0000","last_became_active":"2023-10-08T05:36:11.946295+0000","last_became_peered":"2023-10-08T05:36:11.946295+0000","last_unstale":"2023-10-08T05:36:16.240155+0000","last_undegraded":"2023-10-08T05:36:16.240155+0000","last_fullsized":"2023-10-08T05:36:16.240155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:28.581977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214835+0000","last_change":"2023-10-08T05:36:11.181737+0000","last_active":"2023-10-08T05:36:16.214835+0000","last_peered":"2023-10-08T05:36:16.214835+0000","last_clean":"2023-10-08T05:36:16.214835+0000","last_became_active":"2023-10-08T05:36:11.181386+0000","last_became_peered":"2023-10-08T05:36:11.181386+0000","last_unstale":"2023-10-08T05:36:16.214835+0000","last_undegraded":"2023-10-08T05:36:16.214835+0000","last_fullsized":"2023-10-08T05:36:16.214835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:05.139990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450290+0000","last_change":"2023-10-08T05:36:16.223098+0000","last_active":"2023-10-08T05:36:18.450290+0000","last_peered":"2023-10-08T05:36:18.450290+0000","last_clean":"2023-10-08T05:36:18.450290+0000","last_became_active":"2023-10-08T05:36:16.222649+0000","last_became_peered":"2023-10-08T05:36:16.222649+0000","last_unstale":"2023-10-08T05:36:18.450290+0000","last_undegraded":"2023-10-08T05:36:18.450290+0000","last_fullsized":"2023-10-08T05:36:18.450290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:27.410602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238995+0000","last_change":"2023-10-08T05:36:11.946258+0000","last_active":"2023-10-08T05:36:16.238995+0000","last_peered":"2023-10-08T05:36:16.238995+0000","last_clean":"2023-10-08T05:36:16.238995+0000","last_became_active":"2023-10-08T05:36:11.945932+0000","last_became_peered":"2023-10-08T05:36:11.945932+0000","last_unstale":"2023-10-08T05:36:16.238995+0000","last_undegraded":"2023-10-08T05:36:16.238995+0000","last_fullsized":"2023-10-08T05:36:16.238995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:16.603017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.189924+0000","last_change":"2023-10-08T05:36:11.162339+0000","last_active":"2023-10-08T05:36:16.189924+0000","last_peered":"2023-10-08T05:36:16.189924+0000","last_clean":"2023-10-08T05:36:16.189924+0000","last_became_active":"2023-10-08T05:36:11.162023+0000","last_became_peered":"2023-10-08T05:36:11.162023+0000","last_unstale":"2023-10-08T05:36:16.189924+0000","last_undegraded":"2023-10-08T05:36:16.189924+0000","last_fullsized":"2023-10-08T05:36:16.189924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:10.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449880+0000","last_change":"2023-10-08T05:36:16.217935+0000","last_active":"2023-10-08T05:36:18.449880+0000","last_peered":"2023-10-08T05:36:18.449880+0000","last_clean":"2023-10-08T05:36:18.449880+0000","last_became_active":"2023-10-08T05:36:16.217530+0000","last_became_peered":"2023-10-08T05:36:16.217530+0000","last_unstale":"2023-10-08T05:36:18.449880+0000","last_undegraded":"2023-10-08T05:36:18.449880+0000","last_fullsized":"2023-10-08T05:36:18.449880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:07.031678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194389+0000","last_change":"2023-10-08T05:36:11.945050+0000","last_active":"2023-10-08T05:36:16.194389+0000","last_peered":"2023-10-08T05:36:16.194389+0000","last_clean":"2023-10-08T05:36:16.194389+0000","last_became_active":"2023-10-08T05:36:11.944419+0000","last_became_peered":"2023-10-08T05:36:11.944419+0000","last_unstale":"2023-10-08T05:36:16.194389+0000","last_undegraded":"2023-10-08T05:36:16.194389+0000","last_fullsized":"2023-10-08T05:36:16.194389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194769+0000","last_change":"2023-10-08T05:36:11.176221+0000","last_active":"2023-10-08T05:36:16.194769+0000","last_peered":"2023-10-08T05:36:16.194769+0000","last_clean":"2023-10-08T05:36:16.194769+0000","last_became_active":"2023-10-08T05:36:11.175839+0000","last_became_peered":"2023-10-08T05:36:11.175839+0000","last_unstale":"2023-10-08T05:36:16.194769+0000","last_undegraded":"2023-10-08T05:36:16.194769+0000","last_fullsized":"2023-10-08T05:36:16.194769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:44.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913652+0000","last_change":"2023-10-08T05:36:16.240543+0000","last_active":"2023-10-08T05:36:16.913652+0000","last_peered":"2023-10-08T05:36:16.913652+0000","last_clean":"2023-10-08T05:36:16.913652+0000","last_became_active":"2023-10-08T05:36:16.240104+0000","last_became_peered":"2023-10-08T05:36:16.240104+0000","last_unstale":"2023-10-08T05:36:16.913652+0000","last_undegraded":"2023-10-08T05:36:16.913652+0000","last_fullsized":"2023-10-08T05:36:16.913652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:21.958405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239177+0000","last_change":"2023-10-08T05:36:11.952574+0000","last_active":"2023-10-08T05:36:16.239177+0000","last_peered":"2023-10-08T05:36:16.239177+0000","last_clean":"2023-10-08T05:36:16.239177+0000","last_became_active":"2023-10-08T05:36:11.952287+0000","last_became_peered":"2023-10-08T05:36:11.952287+0000","last_unstale":"2023-10-08T05:36:16.239177+0000","last_undegraded":"2023-10-08T05:36:16.239177+0000","last_fullsized":"2023-10-08T05:36:16.239177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215860+0000","last_change":"2023-10-08T05:36:11.183076+0000","last_active":"2023-10-08T05:36:16.215860+0000","last_peered":"2023-10-08T05:36:16.215860+0000","last_clean":"2023-10-08T05:36:16.215860+0000","last_became_active":"2023-10-08T05:36:11.182702+0000","last_became_peered":"2023-10-08T05:36:11.182702+0000","last_unstale":"2023-10-08T05:36:16.215860+0000","last_undegraded":"2023-10-08T05:36:16.215860+0000","last_fullsized":"2023-10-08T05:36:16.215860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:45.848562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.912758+0000","last_change":"2023-10-08T05:36:16.240654+0000","last_active":"2023-10-08T05:36:16.912758+0000","last_peered":"2023-10-08T05:36:16.912758+0000","last_clean":"2023-10-08T05:36:16.912758+0000","last_became_active":"2023-10-08T05:36:16.240179+0000","last_became_peered":"2023-10-08T05:36:16.240179+0000","last_unstale":"2023-10-08T05:36:16.912758+0000","last_undegraded":"2023-10-08T05:36:16.912758+0000","last_fullsized":"2023-10-08T05:36:16.912758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:09:12.638961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205225+0000","last_change":"2023-10-08T05:36:11.948816+0000","last_active":"2023-10-08T05:36:16.205225+0000","last_peered":"2023-10-08T05:36:16.205225+0000","last_clean":"2023-10-08T05:36:16.205225+0000","last_became_active":"2023-10-08T05:36:11.948501+0000","last_became_peered":"2023-10-08T05:36:11.948501+0000","last_unstale":"2023-10-08T05:36:16.205225+0000","last_undegraded":"2023-10-08T05:36:16.205225+0000","last_fullsized":"2023-10-08T05:36:16.205225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:43.750000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206453+0000","last_change":"2023-10-08T05:36:11.172445+0000","last_active":"2023-10-08T05:36:16.206453+0000","last_peered":"2023-10-08T05:36:16.206453+0000","last_clean":"2023-10-08T05:36:16.206453+0000","last_became_active":"2023-10-08T05:36:11.172090+0000","last_became_peered":"2023-10-08T05:36:11.172090+0000","last_unstale":"2023-10-08T05:36:16.206453+0000","last_undegraded":"2023-10-08T05:36:16.206453+0000","last_fullsized":"2023-10-08T05:36:16.206453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:58.268025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.348870+0000","last_change":"2023-10-08T05:36:16.240874+0000","last_active":"2023-10-08T05:36:17.348870+0000","last_peered":"2023-10-08T05:36:17.348870+0000","last_clean":"2023-10-08T05:36:17.348870+0000","last_became_active":"2023-10-08T05:36:16.240420+0000","last_became_peered":"2023-10-08T05:36:16.240420+0000","last_unstale":"2023-10-08T05:36:17.348870+0000","last_undegraded":"2023-10-08T05:36:17.348870+0000","last_fullsized":"2023-10-08T05:36:17.348870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:06.106435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239249+0000","last_change":"2023-10-08T05:36:11.949913+0000","last_active":"2023-10-08T05:36:16.239249+0000","last_peered":"2023-10-08T05:36:16.239249+0000","last_clean":"2023-10-08T05:36:16.239249+0000","last_became_active":"2023-10-08T05:36:11.946993+0000","last_became_peered":"2023-10-08T05:36:11.946993+0000","last_unstale":"2023-10-08T05:36:16.239249+0000","last_undegraded":"2023-10-08T05:36:16.239249+0000","last_fullsized":"2023-10-08T05:36:16.239249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:02.994498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:31.450726+0000","last_change":"2023-10-08T05:36:11.177181+0000","last_active":"2023-10-08T05:36:31.450726+0000","last_peered":"2023-10-08T05:36:31.450726+0000","last_clean":"2023-10-08T05:36:31.450726+0000","last_became_active":"2023-10-08T05:36:11.176830+0000","last_became_peered":"2023-10-08T05:36:11.176830+0000","last_unstale":"2023-10-08T05:36:31.450726+0000","last_undegraded":"2023-10-08T05:36:31.450726+0000","last_fullsized":"2023-10-08T05:36:31.450726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:13.509993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.916338+0000","last_change":"2023-10-08T05:36:16.223391+0000","last_active":"2023-10-08T05:36:16.916338+0000","last_peered":"2023-10-08T05:36:16.916338+0000","last_clean":"2023-10-08T05:36:16.916338+0000","last_became_active":"2023-10-08T05:36:16.222997+0000","last_became_peered":"2023-10-08T05:36:16.222997+0000","last_unstale":"2023-10-08T05:36:16.916338+0000","last_undegraded":"2023-10-08T05:36:16.916338+0000","last_fullsized":"2023-10-08T05:36:16.916338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:15.664459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213269+0000","last_change":"2023-10-08T05:36:11.936436+0000","last_active":"2023-10-08T05:36:16.213269+0000","last_peered":"2023-10-08T05:36:16.213269+0000","last_clean":"2023-10-08T05:36:16.213269+0000","last_became_active":"2023-10-08T05:36:11.936161+0000","last_became_peered":"2023-10-08T05:36:11.936161+0000","last_unstale":"2023-10-08T05:36:16.213269+0000","last_undegraded":"2023-10-08T05:36:16.213269+0000","last_fullsized":"2023-10-08T05:36:16.213269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:36.016909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207221+0000","last_change":"2023-10-08T05:36:11.183582+0000","last_active":"2023-10-08T05:36:16.207221+0000","last_peered":"2023-10-08T05:36:16.207221+0000","last_clean":"2023-10-08T05:36:16.207221+0000","last_became_active":"2023-10-08T05:36:11.183264+0000","last_became_peered":"2023-10-08T05:36:11.183264+0000","last_unstale":"2023-10-08T05:36:16.207221+0000","last_undegraded":"2023-10-08T05:36:16.207221+0000","last_fullsized":"2023-10-08T05:36:16.207221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:16.799817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451297+0000","last_change":"2023-10-08T05:36:16.228720+0000","last_active":"2023-10-08T05:36:18.451297+0000","last_peered":"2023-10-08T05:36:18.451297+0000","last_clean":"2023-10-08T05:36:18.451297+0000","last_became_active":"2023-10-08T05:36:16.227638+0000","last_became_peered":"2023-10-08T05:36:16.227638+0000","last_unstale":"2023-10-08T05:36:18.451297+0000","last_undegraded":"2023-10-08T05:36:18.451297+0000","last_fullsized":"2023-10-08T05:36:18.451297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:20.640196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205640+0000","last_change":"2023-10-08T05:36:11.933926+0000","last_active":"2023-10-08T05:36:16.205640+0000","last_peered":"2023-10-08T05:36:16.205640+0000","last_clean":"2023-10-08T05:36:16.205640+0000","last_became_active":"2023-10-08T05:36:11.933648+0000","last_became_peered":"2023-10-08T05:36:11.933648+0000","last_unstale":"2023-10-08T05:36:16.205640+0000","last_undegraded":"2023-10-08T05:36:16.205640+0000","last_fullsized":"2023-10-08T05:36:16.205640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215215+0000","last_change":"2023-10-08T05:36:11.183379+0000","last_active":"2023-10-08T05:36:16.215215+0000","last_peered":"2023-10-08T05:36:16.215215+0000","last_clean":"2023-10-08T05:36:16.215215+0000","last_became_active":"2023-10-08T05:36:11.182862+0000","last_became_peered":"2023-10-08T05:36:11.182862+0000","last_unstale":"2023-10-08T05:36:16.215215+0000","last_undegraded":"2023-10-08T05:36:16.215215+0000","last_fullsized":"2023-10-08T05:36:16.215215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:10.147911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914439+0000","last_change":"2023-10-08T05:36:16.216716+0000","last_active":"2023-10-08T05:36:16.914439+0000","last_peered":"2023-10-08T05:36:16.914439+0000","last_clean":"2023-10-08T05:36:16.914439+0000","last_became_active":"2023-10-08T05:36:16.216266+0000","last_became_peered":"2023-10-08T05:36:16.216266+0000","last_unstale":"2023-10-08T05:36:16.914439+0000","last_undegraded":"2023-10-08T05:36:16.914439+0000","last_fullsized":"2023-10-08T05:36:16.914439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:45.158099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214937+0000","last_change":"2023-10-08T05:36:11.949702+0000","last_active":"2023-10-08T05:36:16.214937+0000","last_peered":"2023-10-08T05:36:16.214937+0000","last_clean":"2023-10-08T05:36:16.214937+0000","last_became_active":"2023-10-08T05:36:11.949421+0000","last_became_peered":"2023-10-08T05:36:11.949421+0000","last_unstale":"2023-10-08T05:36:16.214937+0000","last_undegraded":"2023-10-08T05:36:16.214937+0000","last_fullsized":"2023-10-08T05:36:16.214937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195931+0000","last_change":"2023-10-08T05:36:11.189802+0000","last_active":"2023-10-08T05:36:16.195931+0000","last_peered":"2023-10-08T05:36:16.195931+0000","last_clean":"2023-10-08T05:36:16.195931+0000","last_became_active":"2023-10-08T05:36:11.189447+0000","last_became_peered":"2023-10-08T05:36:11.189447+0000","last_unstale":"2023-10-08T05:36:16.195931+0000","last_undegraded":"2023-10-08T05:36:16.195931+0000","last_fullsized":"2023-10-08T05:36:16.195931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:08.847794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.911933+0000","last_change":"2023-10-08T05:36:16.241184+0000","last_active":"2023-10-08T05:36:16.911933+0000","last_peered":"2023-10-08T05:36:16.911933+0000","last_clean":"2023-10-08T05:36:16.911933+0000","last_became_active":"2023-10-08T05:36:16.240531+0000","last_became_peered":"2023-10-08T05:36:16.240531+0000","last_unstale":"2023-10-08T05:36:16.911933+0000","last_undegraded":"2023-10-08T05:36:16.911933+0000","last_fullsized":"2023-10-08T05:36:16.911933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:17:23.235254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449495+0000","last_change":"2023-10-08T05:36:16.241230+0000","last_active":"2023-10-08T05:36:18.449495+0000","last_peered":"2023-10-08T05:36:18.449495+0000","last_clean":"2023-10-08T05:36:18.449495+0000","last_became_active":"2023-10-08T05:36:16.240814+0000","last_became_peered":"2023-10-08T05:36:16.240814+0000","last_unstale":"2023-10-08T05:36:18.449495+0000","last_undegraded":"2023-10-08T05:36:18.449495+0000","last_fullsized":"2023-10-08T05:36:18.449495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:25.754577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.202997+0000","last_change":"2023-10-08T05:36:11.941419+0000","last_active":"2023-10-08T05:36:16.202997+0000","last_peered":"2023-10-08T05:36:16.202997+0000","last_clean":"2023-10-08T05:36:16.202997+0000","last_became_active":"2023-10-08T05:36:11.941131+0000","last_became_peered":"2023-10-08T05:36:11.941131+0000","last_unstale":"2023-10-08T05:36:16.202997+0000","last_undegraded":"2023-10-08T05:36:16.202997+0000","last_fullsized":"2023-10-08T05:36:16.202997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214657+0000","last_change":"2023-10-08T05:36:11.182594+0000","last_active":"2023-10-08T05:36:16.214657+0000","last_peered":"2023-10-08T05:36:16.214657+0000","last_clean":"2023-10-08T05:36:16.214657+0000","last_became_active":"2023-10-08T05:36:11.182297+0000","last_became_peered":"2023-10-08T05:36:11.182297+0000","last_unstale":"2023-10-08T05:36:16.214657+0000","last_undegraded":"2023-10-08T05:36:16.214657+0000","last_fullsized":"2023-10-08T05:36:16.214657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:59.799745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914977+0000","last_change":"2023-10-08T05:36:16.220333+0000","last_active":"2023-10-08T05:36:16.914977+0000","last_peered":"2023-10-08T05:36:16.914977+0000","last_clean":"2023-10-08T05:36:16.914977+0000","last_became_active":"2023-10-08T05:36:16.219836+0000","last_became_peered":"2023-10-08T05:36:16.219836+0000","last_unstale":"2023-10-08T05:36:16.914977+0000","last_undegraded":"2023-10-08T05:36:16.914977+0000","last_fullsized":"2023-10-08T05:36:16.914977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:32.619157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195436+0000","last_change":"2023-10-08T05:36:11.945676+0000","last_active":"2023-10-08T05:36:16.195436+0000","last_peered":"2023-10-08T05:36:16.195436+0000","last_clean":"2023-10-08T05:36:16.195436+0000","last_became_active":"2023-10-08T05:36:11.945314+0000","last_became_peered":"2023-10-08T05:36:11.945314+0000","last_unstale":"2023-10-08T05:36:16.195436+0000","last_undegraded":"2023-10-08T05:36:16.195436+0000","last_fullsized":"2023-10-08T05:36:16.195436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.197371+0000","last_change":"2023-10-08T05:36:11.176854+0000","last_active":"2023-10-08T05:36:16.197371+0000","last_peered":"2023-10-08T05:36:16.197371+0000","last_clean":"2023-10-08T05:36:16.197371+0000","last_became_active":"2023-10-08T05:36:11.176546+0000","last_became_peered":"2023-10-08T05:36:11.176546+0000","last_unstale":"2023-10-08T05:36:16.197371+0000","last_undegraded":"2023-10-08T05:36:16.197371+0000","last_fullsized":"2023-10-08T05:36:16.197371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:16.815302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195412+0000","last_change":"2023-10-08T05:36:11.184441+0000","last_active":"2023-10-08T05:36:16.195412+0000","last_peered":"2023-10-08T05:36:16.195412+0000","last_clean":"2023-10-08T05:36:16.195412+0000","last_became_active":"2023-10-08T05:36:11.184159+0000","last_became_peered":"2023-10-08T05:36:11.184159+0000","last_unstale":"2023-10-08T05:36:16.195412+0000","last_undegraded":"2023-10-08T05:36:16.195412+0000","last_fullsized":"2023-10-08T05:36:16.195412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206365+0000","last_change":"2023-10-08T05:36:11.935252+0000","last_active":"2023-10-08T05:36:16.206365+0000","last_peered":"2023-10-08T05:36:16.206365+0000","last_clean":"2023-10-08T05:36:16.206365+0000","last_became_active":"2023-10-08T05:36:11.934390+0000","last_became_peered":"2023-10-08T05:36:11.934390+0000","last_unstale":"2023-10-08T05:36:16.206365+0000","last_undegraded":"2023-10-08T05:36:16.206365+0000","last_fullsized":"2023-10-08T05:36:16.206365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.242390+0000","last_change":"2023-10-08T05:36:16.217984+0000","last_active":"2023-10-08T05:36:17.242390+0000","last_peered":"2023-10-08T05:36:17.242390+0000","last_clean":"2023-10-08T05:36:17.242390+0000","last_became_active":"2023-10-08T05:36:16.217614+0000","last_became_peered":"2023-10-08T05:36:16.217614+0000","last_unstale":"2023-10-08T05:36:17.242390+0000","last_undegraded":"2023-10-08T05:36:17.242390+0000","last_fullsized":"2023-10-08T05:36:17.242390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:47.787180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194733+0000","last_change":"2023-10-08T05:36:11.182558+0000","last_active":"2023-10-08T05:36:16.194733+0000","last_peered":"2023-10-08T05:36:16.194733+0000","last_clean":"2023-10-08T05:36:16.194733+0000","last_became_active":"2023-10-08T05:36:11.181822+0000","last_became_peered":"2023-10-08T05:36:11.181822+0000","last_unstale":"2023-10-08T05:36:16.194733+0000","last_undegraded":"2023-10-08T05:36:16.194733+0000","last_fullsized":"2023-10-08T05:36:16.194733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214758+0000","last_change":"2023-10-08T05:36:11.941704+0000","last_active":"2023-10-08T05:36:16.214758+0000","last_peered":"2023-10-08T05:36:16.214758+0000","last_clean":"2023-10-08T05:36:16.214758+0000","last_became_active":"2023-10-08T05:36:11.941431+0000","last_became_peered":"2023-10-08T05:36:11.941431+0000","last_unstale":"2023-10-08T05:36:16.214758+0000","last_undegraded":"2023-10-08T05:36:16.214758+0000","last_fullsized":"2023-10-08T05:36:16.214758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:00.811496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913836+0000","last_change":"2023-10-08T05:36:16.220904+0000","last_active":"2023-10-08T05:36:16.913836+0000","last_peered":"2023-10-08T05:36:16.913836+0000","last_clean":"2023-10-08T05:36:16.913836+0000","last_became_active":"2023-10-08T05:36:16.220521+0000","last_became_peered":"2023-10-08T05:36:16.220521+0000","last_unstale":"2023-10-08T05:36:16.913836+0000","last_undegraded":"2023-10-08T05:36:16.913836+0000","last_fullsized":"2023-10-08T05:36:16.913836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:52:02.322999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194151+0000","last_change":"2023-10-08T05:36:11.945315+0000","last_active":"2023-10-08T05:36:16.194151+0000","last_peered":"2023-10-08T05:36:16.194151+0000","last_clean":"2023-10-08T05:36:16.194151+0000","last_became_active":"2023-10-08T05:36:11.944775+0000","last_became_peered":"2023-10-08T05:36:11.944775+0000","last_unstale":"2023-10-08T05:36:16.194151+0000","last_undegraded":"2023-10-08T05:36:16.194151+0000","last_fullsized":"2023-10-08T05:36:16.194151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.189968+0000","last_change":"2023-10-08T05:36:11.938156+0000","last_active":"2023-10-08T05:36:16.189968+0000","last_peered":"2023-10-08T05:36:16.189968+0000","last_clean":"2023-10-08T05:36:16.189968+0000","last_became_active":"2023-10-08T05:36:11.937878+0000","last_became_peered":"2023-10-08T05:36:11.937878+0000","last_unstale":"2023-10-08T05:36:16.189968+0000","last_undegraded":"2023-10-08T05:36:16.189968+0000","last_fullsized":"2023-10-08T05:36:16.189968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:00.811496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215846+0000","last_change":"2023-10-08T05:36:11.944484+0000","last_active":"2023-10-08T05:36:16.215846+0000","last_peered":"2023-10-08T05:36:16.215846+0000","last_clean":"2023-10-08T05:36:16.215846+0000","last_became_active":"2023-10-08T05:36:11.943696+0000","last_became_peered":"2023-10-08T05:36:11.943696+0000","last_unstale":"2023-10-08T05:36:16.215846+0000","last_undegraded":"2023-10-08T05:36:16.215846+0000","last_fullsized":"2023-10-08T05:36:16.215846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:12.498480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206284+0000","last_change":"2023-10-08T05:36:11.944505+0000","last_active":"2023-10-08T05:36:16.206284+0000","last_peered":"2023-10-08T05:36:16.206284+0000","last_clean":"2023-10-08T05:36:16.206284+0000","last_became_active":"2023-10-08T05:36:11.944084+0000","last_became_peered":"2023-10-08T05:36:11.944084+0000","last_unstale":"2023-10-08T05:36:16.206284+0000","last_undegraded":"2023-10-08T05:36:16.206284+0000","last_fullsized":"2023-10-08T05:36:16.206284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.197543+0000","last_change":"2023-10-08T05:36:11.936235+0000","last_active":"2023-10-08T05:36:16.197543+0000","last_peered":"2023-10-08T05:36:16.197543+0000","last_clean":"2023-10-08T05:36:16.197543+0000","last_became_active":"2023-10-08T05:36:11.935816+0000","last_became_peered":"2023-10-08T05:36:11.935816+0000","last_unstale":"2023-10-08T05:36:16.197543+0000","last_undegraded":"2023-10-08T05:36:16.197543+0000","last_fullsized":"2023-10-08T05:36:16.197543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206800+0000","last_change":"2023-10-08T05:36:11.940291+0000","last_active":"2023-10-08T05:36:16.206800+0000","last_peered":"2023-10-08T05:36:16.206800+0000","last_clean":"2023-10-08T05:36:16.206800+0000","last_became_active":"2023-10-08T05:36:11.940019+0000","last_became_peered":"2023-10-08T05:36:11.940019+0000","last_unstale":"2023-10-08T05:36:16.206800+0000","last_undegraded":"2023-10-08T05:36:16.206800+0000","last_fullsized":"2023-10-08T05:36:16.206800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195886+0000","last_change":"2023-10-08T05:36:11.949017+0000","last_active":"2023-10-08T05:36:16.195886+0000","last_peered":"2023-10-08T05:36:16.195886+0000","last_clean":"2023-10-08T05:36:16.195886+0000","last_became_active":"2023-10-08T05:36:11.948741+0000","last_became_peered":"2023-10-08T05:36:11.948741+0000","last_unstale":"2023-10-08T05:36:16.195886+0000","last_undegraded":"2023-10-08T05:36:16.195886+0000","last_fullsized":"2023-10-08T05:36:16.195886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198027+0000","last_change":"2023-10-08T05:36:11.937078+0000","last_active":"2023-10-08T05:36:16.198027+0000","last_peered":"2023-10-08T05:36:16.198027+0000","last_clean":"2023-10-08T05:36:16.198027+0000","last_became_active":"2023-10-08T05:36:11.936737+0000","last_became_peered":"2023-10-08T05:36:11.936737+0000","last_unstale":"2023-10-08T05:36:16.198027+0000","last_undegraded":"2023-10-08T05:36:16.198027+0000","last_fullsized":"2023-10-08T05:36:16.198027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214525+0000","last_change":"2023-10-08T05:36:11.945582+0000","last_active":"2023-10-08T05:36:16.214525+0000","last_peered":"2023-10-08T05:36:16.214525+0000","last_clean":"2023-10-08T05:36:16.214525+0000","last_became_active":"2023-10-08T05:36:11.945066+0000","last_became_peered":"2023-10-08T05:36:11.945066+0000","last_unstale":"2023-10-08T05:36:16.214525+0000","last_undegraded":"2023-10-08T05:36:16.214525+0000","last_fullsized":"2023-10-08T05:36:16.214525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196604+0000","last_change":"2023-10-08T05:36:11.945936+0000","last_active":"2023-10-08T05:36:16.196604+0000","last_peered":"2023-10-08T05:36:16.196604+0000","last_clean":"2023-10-08T05:36:16.196604+0000","last_became_active":"2023-10-08T05:36:11.945384+0000","last_became_peered":"2023-10-08T05:36:11.945384+0000","last_unstale":"2023-10-08T05:36:16.196604+0000","last_undegraded":"2023-10-08T05:36:16.196604+0000","last_fullsized":"2023-10-08T05:36:16.196604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215763+0000","last_change":"2023-10-08T05:36:11.949155+0000","last_active":"2023-10-08T05:36:16.215763+0000","last_peered":"2023-10-08T05:36:16.215763+0000","last_clean":"2023-10-08T05:36:16.215763+0000","last_became_active":"2023-10-08T05:36:11.948867+0000","last_became_peered":"2023-10-08T05:36:11.948867+0000","last_unstale":"2023-10-08T05:36:16.215763+0000","last_undegraded":"2023-10-08T05:36:16.215763+0000","last_fullsized":"2023-10-08T05:36:16.215763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:11.627279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206186+0000","last_change":"2023-10-08T05:36:11.949363+0000","last_active":"2023-10-08T05:36:16.206186+0000","last_peered":"2023-10-08T05:36:16.206186+0000","last_clean":"2023-10-08T05:36:16.206186+0000","last_became_active":"2023-10-08T05:36:11.949059+0000","last_became_peered":"2023-10-08T05:36:11.949059+0000","last_unstale":"2023-10-08T05:36:16.206186+0000","last_undegraded":"2023-10-08T05:36:16.206186+0000","last_fullsized":"2023-10-08T05:36:16.206186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:00.571257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195787+0000","last_change":"2023-10-08T05:36:11.936518+0000","last_active":"2023-10-08T05:36:16.195787+0000","last_peered":"2023-10-08T05:36:16.195787+0000","last_clean":"2023-10-08T05:36:16.195787+0000","last_became_active":"2023-10-08T05:36:11.936138+0000","last_became_peered":"2023-10-08T05:36:11.936138+0000","last_unstale":"2023-10-08T05:36:16.195787+0000","last_undegraded":"2023-10-08T05:36:16.195787+0000","last_fullsized":"2023-10-08T05:36:16.195787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239530+0000","last_change":"2023-10-08T05:36:11.940992+0000","last_active":"2023-10-08T05:36:16.239530+0000","last_peered":"2023-10-08T05:36:16.239530+0000","last_clean":"2023-10-08T05:36:16.239530+0000","last_became_active":"2023-10-08T05:36:11.940700+0000","last_became_peered":"2023-10-08T05:36:11.940700+0000","last_unstale":"2023-10-08T05:36:16.239530+0000","last_undegraded":"2023-10-08T05:36:16.239530+0000","last_fullsized":"2023-10-08T05:36:16.239530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:20.449484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190822+0000","last_change":"2023-10-08T05:36:11.937136+0000","last_active":"2023-10-08T05:36:16.190822+0000","last_peered":"2023-10-08T05:36:16.190822+0000","last_clean":"2023-10-08T05:36:16.190822+0000","last_became_active":"2023-10-08T05:36:11.936761+0000","last_became_peered":"2023-10-08T05:36:11.936761+0000","last_unstale":"2023-10-08T05:36:16.190822+0000","last_undegraded":"2023-10-08T05:36:16.190822+0000","last_fullsized":"2023-10-08T05:36:16.190822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:50.179123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215403+0000","last_change":"2023-10-08T05:36:11.943099+0000","last_active":"2023-10-08T05:36:16.215403+0000","last_peered":"2023-10-08T05:36:16.215403+0000","last_clean":"2023-10-08T05:36:16.215403+0000","last_became_active":"2023-10-08T05:36:11.942792+0000","last_became_peered":"2023-10-08T05:36:11.942792+0000","last_unstale":"2023-10-08T05:36:16.215403+0000","last_undegraded":"2023-10-08T05:36:16.215403+0000","last_fullsized":"2023-10-08T05:36:16.215403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214354+0000","last_change":"2023-10-08T05:36:11.938711+0000","last_active":"2023-10-08T05:36:16.214354+0000","last_peered":"2023-10-08T05:36:16.214354+0000","last_clean":"2023-10-08T05:36:16.214354+0000","last_became_active":"2023-10-08T05:36:11.938449+0000","last_became_peered":"2023-10-08T05:36:11.938449+0000","last_unstale":"2023-10-08T05:36:16.214354+0000","last_undegraded":"2023-10-08T05:36:16.214354+0000","last_fullsized":"2023-10-08T05:36:16.214354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:56.745041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203081+0000","last_change":"2023-10-08T05:36:11.950960+0000","last_active":"2023-10-08T05:36:16.203081+0000","last_peered":"2023-10-08T05:36:16.203081+0000","last_clean":"2023-10-08T05:36:16.203081+0000","last_became_active":"2023-10-08T05:36:11.950582+0000","last_became_peered":"2023-10-08T05:36:11.950582+0000","last_unstale":"2023-10-08T05:36:16.203081+0000","last_undegraded":"2023-10-08T05:36:16.203081+0000","last_fullsized":"2023-10-08T05:36:16.203081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205140+0000","last_change":"2023-10-08T05:36:11.938997+0000","last_active":"2023-10-08T05:36:16.205140+0000","last_peered":"2023-10-08T05:36:16.205140+0000","last_clean":"2023-10-08T05:36:16.205140+0000","last_became_active":"2023-10-08T05:36:11.938641+0000","last_became_peered":"2023-10-08T05:36:11.938641+0000","last_unstale":"2023-10-08T05:36:16.205140+0000","last_undegraded":"2023-10-08T05:36:16.205140+0000","last_fullsized":"2023-10-08T05:36:16.205140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:54.023070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194362+0000","last_change":"2023-10-08T05:36:11.938973+0000","last_active":"2023-10-08T05:36:16.194362+0000","last_peered":"2023-10-08T05:36:16.194362+0000","last_clean":"2023-10-08T05:36:16.194362+0000","last_became_active":"2023-10-08T05:36:11.938633+0000","last_became_peered":"2023-10-08T05:36:11.938633+0000","last_unstale":"2023-10-08T05:36:16.194362+0000","last_undegraded":"2023-10-08T05:36:16.194362+0000","last_fullsized":"2023-10-08T05:36:16.194362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213118+0000","last_change":"2023-10-08T05:36:11.938748+0000","last_active":"2023-10-08T05:36:16.213118+0000","last_peered":"2023-10-08T05:36:16.213118+0000","last_clean":"2023-10-08T05:36:16.213118+0000","last_became_active":"2023-10-08T05:36:11.937801+0000","last_became_peered":"2023-10-08T05:36:11.937801+0000","last_unstale":"2023-10-08T05:36:16.213118+0000","last_undegraded":"2023-10-08T05:36:16.213118+0000","last_fullsized":"2023-10-08T05:36:16.213118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:59.279776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196976+0000","last_change":"2023-10-08T05:36:11.937561+0000","last_active":"2023-10-08T05:36:16.196976+0000","last_peered":"2023-10-08T05:36:16.196976+0000","last_clean":"2023-10-08T05:36:16.196976+0000","last_became_active":"2023-10-08T05:36:11.937299+0000","last_became_peered":"2023-10-08T05:36:11.937299+0000","last_unstale":"2023-10-08T05:36:16.196976+0000","last_undegraded":"2023-10-08T05:36:16.196976+0000","last_fullsized":"2023-10-08T05:36:16.196976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195655+0000","last_change":"2023-10-08T05:36:11.938330+0000","last_active":"2023-10-08T05:36:16.195655+0000","last_peered":"2023-10-08T05:36:16.195655+0000","last_clean":"2023-10-08T05:36:16.195655+0000","last_became_active":"2023-10-08T05:36:11.937993+0000","last_became_peered":"2023-10-08T05:36:11.937993+0000","last_unstale":"2023-10-08T05:36:16.195655+0000","last_undegraded":"2023-10-08T05:36:16.195655+0000","last_fullsized":"2023-10-08T05:36:16.195655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240495+0000","last_change":"2023-10-08T05:36:11.954185+0000","last_active":"2023-10-08T05:36:16.240495+0000","last_peered":"2023-10-08T05:36:16.240495+0000","last_clean":"2023-10-08T05:36:16.240495+0000","last_became_active":"2023-10-08T05:36:11.953900+0000","last_became_peered":"2023-10-08T05:36:11.953900+0000","last_unstale":"2023-10-08T05:36:16.240495+0000","last_undegraded":"2023-10-08T05:36:16.240495+0000","last_fullsized":"2023-10-08T05:36:16.240495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:11.627279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195359+0000","last_change":"2023-10-08T05:36:11.938140+0000","last_active":"2023-10-08T05:36:16.195359+0000","last_peered":"2023-10-08T05:36:16.195359+0000","last_clean":"2023-10-08T05:36:16.195359+0000","last_became_active":"2023-10-08T05:36:11.937817+0000","last_became_peered":"2023-10-08T05:36:11.937817+0000","last_unstale":"2023-10-08T05:36:16.195359+0000","last_undegraded":"2023-10-08T05:36:16.195359+0000","last_fullsized":"2023-10-08T05:36:16.195359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238846+0000","last_change":"2023-10-08T05:36:11.948351+0000","last_active":"2023-10-08T05:36:16.238846+0000","last_peered":"2023-10-08T05:36:16.238846+0000","last_clean":"2023-10-08T05:36:16.238846+0000","last_became_active":"2023-10-08T05:36:11.948045+0000","last_became_peered":"2023-10-08T05:36:11.948045+0000","last_unstale":"2023-10-08T05:36:16.238846+0000","last_undegraded":"2023-10-08T05:36:16.238846+0000","last_fullsized":"2023-10-08T05:36:16.238846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215673+0000","last_change":"2023-10-08T05:36:11.944663+0000","last_active":"2023-10-08T05:36:16.215673+0000","last_peered":"2023-10-08T05:36:16.215673+0000","last_clean":"2023-10-08T05:36:16.215673+0000","last_became_active":"2023-10-08T05:36:11.944158+0000","last_became_peered":"2023-10-08T05:36:11.944158+0000","last_unstale":"2023-10-08T05:36:16.215673+0000","last_undegraded":"2023-10-08T05:36:16.215673+0000","last_fullsized":"2023-10-08T05:36:16.215673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:16.603017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239112+0000","last_change":"2023-10-08T05:36:11.944406+0000","last_active":"2023-10-08T05:36:16.239112+0000","last_peered":"2023-10-08T05:36:16.239112+0000","last_clean":"2023-10-08T05:36:16.239112+0000","last_became_active":"2023-10-08T05:36:11.944031+0000","last_became_peered":"2023-10-08T05:36:11.944031+0000","last_unstale":"2023-10-08T05:36:16.239112+0000","last_undegraded":"2023-10-08T05:36:16.239112+0000","last_fullsized":"2023-10-08T05:36:16.239112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214934+0000","last_change":"2023-10-08T05:36:11.949094+0000","last_active":"2023-10-08T05:36:16.214934+0000","last_peered":"2023-10-08T05:36:16.214934+0000","last_clean":"2023-10-08T05:36:16.214934+0000","last_became_active":"2023-10-08T05:36:11.948663+0000","last_became_peered":"2023-10-08T05:36:11.948663+0000","last_unstale":"2023-10-08T05:36:16.214934+0000","last_undegraded":"2023-10-08T05:36:16.214934+0000","last_fullsized":"2023-10-08T05:36:16.214934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:46:57.883695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205484+0000","last_change":"2023-10-08T05:36:11.927796+0000","last_active":"2023-10-08T05:36:16.205484+0000","last_peered":"2023-10-08T05:36:16.205484+0000","last_clean":"2023-10-08T05:36:16.205484+0000","last_became_active":"2023-10-08T05:36:11.927200+0000","last_became_peered":"2023-10-08T05:36:11.927200+0000","last_unstale":"2023-10-08T05:36:16.205484+0000","last_undegraded":"2023-10-08T05:36:16.205484+0000","last_fullsized":"2023-10-08T05:36:16.205484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.208328+0000","last_change":"2023-10-08T05:36:11.937916+0000","last_active":"2023-10-08T05:36:16.208328+0000","last_peered":"2023-10-08T05:36:16.208328+0000","last_clean":"2023-10-08T05:36:16.208328+0000","last_became_active":"2023-10-08T05:36:11.937646+0000","last_became_peered":"2023-10-08T05:36:11.937646+0000","last_unstale":"2023-10-08T05:36:16.208328+0000","last_undegraded":"2023-10-08T05:36:16.208328+0000","last_fullsized":"2023-10-08T05:36:16.208328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240404+0000","last_change":"2023-10-08T05:36:11.952124+0000","last_active":"2023-10-08T05:36:16.240404+0000","last_peered":"2023-10-08T05:36:16.240404+0000","last_clean":"2023-10-08T05:36:16.240404+0000","last_became_active":"2023-10-08T05:36:11.949353+0000","last_became_peered":"2023-10-08T05:36:11.949353+0000","last_unstale":"2023-10-08T05:36:16.240404+0000","last_undegraded":"2023-10-08T05:36:16.240404+0000","last_fullsized":"2023-10-08T05:36:16.240404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:46:57.883695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.452}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8149999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6990000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8859999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1830000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.115}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6419999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:36:41.048 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:36:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.291+0000 7f355f59e640 1 Processor -- start 2023-10-08T05:36:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.291+0000 7f355f59e640 1 -- start start 2023-10-08T05:36:41.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.292+0000 7f355f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f356015ab30 0x7f356015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.292+0000 7f355f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35600b07d0 0x7f356006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.292+0000 7f355f59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356015f7d0 con 0x7f35600adc70 2023-10-08T05:36:41.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.292+0000 7f355f59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356015f210 con 0x7f35600b07d0 2023-10-08T05:36:41.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.292+0000 7f355f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356015f4f0 con 0x7f356015ab30 2023-10-08T05:36:41.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.293+0000 7f355ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35600b07d0 0x7f356006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.293+0000 7f355ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35600b07d0 0x7f356006cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60264/0 (socket says 172.21.15.70:60264) 2023-10-08T05:36:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.293+0000 7f355ed9d640 1 -- 172.21.15.70:0/3550425874 learned_addr learned my addr 172.21.15.70:0/3550425874 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.293+0000 7f3557fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f35600adc70 0x7f35600b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43764/0 (socket says 172.21.15.70:43764) 2023-10-08T05:36:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.293+0000 7f355e59c640 1 --2- 172.21.15.70:0/3550425874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f356015ab30 0x7f356015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.293+0000 7f355ed9d640 1 -- 172.21.15.70:0/3550425874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f356015ab30 msgr2=0x7f356015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.293+0000 7f355ed9d640 1 --2- 172.21.15.70:0/3550425874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f356015ab30 0x7f356015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.293+0000 7f355ed9d640 1 -- 172.21.15.70:0/3550425874 >> v1:172.21.15.70:6789/0 conn(0x7f35600adc70 legacy=0x7f35600b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.293+0000 7f355ed9d640 1 -- 172.21.15.70:0/3550425874 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356006c550 con 0x7f35600b07d0 2023-10-08T05:36:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.293+0000 7f355dd9b640 1 -- 172.21.15.70:0/3550425874 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573081253 0 0) 0x7f356015f7d0 con 0x7f35600adc70 2023-10-08T05:36:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.294+0000 7f355ed9d640 1 --2- 172.21.15.70:0/3550425874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 0x7f356006cd50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3558009e50 tx=0x7f355802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4803cd6cc376815 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355dd9b640 1 -- 172.21.15.70:0/3550425874 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355802cb00 con 0x7f35600b07d0 2023-10-08T05:36:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355dd9b640 1 -- 172.21.15.70:0/3550425874 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355802cc60 con 0x7f35600b07d0 2023-10-08T05:36:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355dd9b640 1 -- 172.21.15.70:0/3550425874 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35580343f0 con 0x7f35600b07d0 2023-10-08T05:36:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355f59e640 1 -- 172.21.15.70:0/3550425874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 msgr2=0x7f356006cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355f59e640 1 --2- 172.21.15.70:0/3550425874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 0x7f356006cd50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3558009e50 tx=0x7f355802bf60 comp rx=0 tx=0).stop 2023-10-08T05:36:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355f59e640 1 -- 172.21.15.70:0/3550425874 shutdown_connections 2023-10-08T05:36:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355f59e640 1 --2- 172.21.15.70:0/3550425874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 0x7f356006cd50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355f59e640 1 --2- 172.21.15.70:0/3550425874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f356015ab30 0x7f356015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355f59e640 1 -- 172.21.15.70:0/3550425874 >> 172.21.15.70:0/3550425874 conn(0x7f35600cb0c0 msgr2=0x7f35600cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355f59e640 1 -- 172.21.15.70:0/3550425874 shutdown_connections 2023-10-08T05:36:41.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.295+0000 7f355f59e640 1 -- 172.21.15.70:0/3550425874 wait complete. 2023-10-08T05:36:41.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.297+0000 7f355f59e640 1 Processor -- start 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.297+0000 7f355f59e640 1 -- start start 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.298+0000 7f355f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35600adc70 0x7f35600ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.298+0000 7f355f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 0x7f35600ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.298+0000 7f355f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f356015ab30 0x7f35601f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.298+0000 7f355f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35600ad210 con 0x7f35600b07d0 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.298+0000 7f355f59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35600acf30 con 0x7f356015ab30 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.298+0000 7f355f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35600c5840 con 0x7f35600adc70 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.298+0000 7f3557fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 0x7f35600ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.298+0000 7f3557fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 0x7f35600ac820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60270/0 (socket says 172.21.15.70:60270) 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.298+0000 7f3557fff640 1 -- 172.21.15.70:0/780969561 learned_addr learned my addr 172.21.15.70:0/780969561 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f355e59c640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35600adc70 0x7f35600ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f355ed9d640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f356015ab30 0x7f35601f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f3557fff640 1 -- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35600adc70 msgr2=0x7f35600ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f3557fff640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35600adc70 0x7f35600ac2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f3557fff640 1 -- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f356015ab30 msgr2=0x7f35601f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f3557fff640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f356015ab30 0x7f35601f2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f3557fff640 1 -- 172.21.15.70:0/780969561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3558009b30 con 0x7f35600b07d0 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f3557fff640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 0x7f35600ac820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f354800c4d0 tx=0x7f354800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f3556ffd640 1 -- 172.21.15.70:0/780969561 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35480074e0 con 0x7f35600b07d0 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f3556ffd640 1 -- 172.21.15.70:0/780969561 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3548007660 con 0x7f35600b07d0 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f3556ffd640 1 -- 172.21.15.70:0/780969561 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548004a20 con 0x7f35600b07d0 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f355f59e640 1 -- 172.21.15.70:0/780969561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35600c5b20 con 0x7f35600b07d0 2023-10-08T05:36:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.299+0000 7f355f59e640 1 -- 172.21.15.70:0/780969561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35601f2d80 con 0x7f35600b07d0 2023-10-08T05:36:41.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.300+0000 7f355f59e640 1 -- 172.21.15.70:0/780969561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35600c6040 con 0x7f35600b07d0 2023-10-08T05:36:41.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.301+0000 7f3556ffd640 1 -- 172.21.15.70:0/780969561 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3548004b80 con 0x7f35600b07d0 2023-10-08T05:36:41.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.302+0000 7f3556ffd640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 0x7f354006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.302+0000 7f3556ffd640 1 -- 172.21.15.70:0/780969561 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3548088aa0 con 0x7f35600b07d0 2023-10-08T05:36:41.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.303+0000 7f355e59c640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 0x7f354006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.304+0000 7f355e59c640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 0x7f354006d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3550006fd0 tx=0x7f3550008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:41.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.306+0000 7f3556ffd640 1 -- 172.21.15.70:0/780969561 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35600c6040 con 0x7f35600b07d0 2023-10-08T05:36:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.478+0000 7f355f59e640 1 -- 172.21.15.70:0/780969561 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f356006c830 con 0x7f354006ae70 2023-10-08T05:36:41.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.505+0000 7f3556ffd640 1 -- 172.21.15.70:0/780969561 <== mgr.4105 v2:172.21.15.181:6832/2737793492 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345139 (secure 0 0 0) 0x7f355000e4c0 con 0x7f354006ae70 2023-10-08T05:36:41.511 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:41.513 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:36:41.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.510+0000 7f355f59e640 1 -- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 msgr2=0x7f354006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.510+0000 7f355f59e640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 0x7f354006d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3550006fd0 tx=0x7f3550008040 comp rx=0 tx=0).stop 2023-10-08T05:36:41.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.510+0000 7f355f59e640 1 -- 172.21.15.70:0/780969561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 msgr2=0x7f35600ac820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.510+0000 7f355f59e640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 0x7f35600ac820 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f354800c4d0 tx=0x7f354800c990 comp rx=0 tx=0).stop 2023-10-08T05:36:41.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.510+0000 7f355f59e640 1 -- 172.21.15.70:0/780969561 shutdown_connections 2023-10-08T05:36:41.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.510+0000 7f355f59e640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 0x7f354006d330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.510+0000 7f355f59e640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f356015ab30 0x7f35601f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.511+0000 7f355f59e640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35600b07d0 0x7f35600ac820 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.511+0000 7f355f59e640 1 --2- 172.21.15.70:0/780969561 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f35600adc70 0x7f35600ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.511+0000 7f355f59e640 1 -- 172.21.15.70:0/780969561 >> 172.21.15.70:0/780969561 conn(0x7f35600cb0c0 msgr2=0x7f356006cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:41.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.511+0000 7f355f59e640 1 -- 172.21.15.70:0/780969561 shutdown_connections 2023-10-08T05:36:41.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.511+0000 7f355f59e640 1 -- 172.21.15.70:0/780969561 wait complete. 2023-10-08T05:36:41.530 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-10-08T05:36:41.080825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236096,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738624,"statfs":{"total":773094113280,"available":772852350976,"internally_reserved":0,"allocated":3383296,"data_stored":2025622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005363"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205680+0000","last_change":"2023-10-08T05:36:11.183116+0000","last_active":"2023-10-08T05:36:16.205680+0000","last_peered":"2023-10-08T05:36:16.205680+0000","last_clean":"2023-10-08T05:36:16.205680+0000","last_became_active":"2023-10-08T05:36:11.182654+0000","last_became_peered":"2023-10-08T05:36:11.182654+0000","last_unstale":"2023-10-08T05:36:16.205680+0000","last_undegraded":"2023-10-08T05:36:16.205680+0000","last_fullsized":"2023-10-08T05:36:16.205680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:49.167371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198007+0000","last_change":"2023-10-08T05:36:11.938929+0000","last_active":"2023-10-08T05:36:16.198007+0000","last_peered":"2023-10-08T05:36:16.198007+0000","last_clean":"2023-10-08T05:36:16.198007+0000","last_became_active":"2023-10-08T05:36:11.938616+0000","last_became_peered":"2023-10-08T05:36:11.938616+0000","last_unstale":"2023-10-08T05:36:16.198007+0000","last_undegraded":"2023-10-08T05:36:16.198007+0000","last_fullsized":"2023-10-08T05:36:16.198007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.911637+0000","last_change":"2023-10-08T05:36:16.241991+0000","last_active":"2023-10-08T05:36:16.911637+0000","last_peered":"2023-10-08T05:36:16.911637+0000","last_clean":"2023-10-08T05:36:16.911637+0000","last_became_active":"2023-10-08T05:36:16.241629+0000","last_became_peered":"2023-10-08T05:36:16.241629+0000","last_unstale":"2023-10-08T05:36:16.911637+0000","last_undegraded":"2023-10-08T05:36:16.911637+0000","last_fullsized":"2023-10-08T05:36:16.911637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:41.216984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198078+0000","last_change":"2023-10-08T05:36:11.171819+0000","last_active":"2023-10-08T05:36:16.198078+0000","last_peered":"2023-10-08T05:36:16.198078+0000","last_clean":"2023-10-08T05:36:16.198078+0000","last_became_active":"2023-10-08T05:36:11.171384+0000","last_became_peered":"2023-10-08T05:36:11.171384+0000","last_unstale":"2023-10-08T05:36:16.198078+0000","last_undegraded":"2023-10-08T05:36:16.198078+0000","last_fullsized":"2023-10-08T05:36:16.198078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:08.838858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205587+0000","last_change":"2023-10-08T05:36:11.939107+0000","last_active":"2023-10-08T05:36:16.205587+0000","last_peered":"2023-10-08T05:36:16.205587+0000","last_clean":"2023-10-08T05:36:16.205587+0000","last_became_active":"2023-10-08T05:36:11.938788+0000","last_became_peered":"2023-10-08T05:36:11.938788+0000","last_unstale":"2023-10-08T05:36:16.205587+0000","last_undegraded":"2023-10-08T05:36:16.205587+0000","last_fullsized":"2023-10-08T05:36:16.205587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450376+0000","last_change":"2023-10-08T05:36:16.220918+0000","last_active":"2023-10-08T05:36:18.450376+0000","last_peered":"2023-10-08T05:36:18.450376+0000","last_clean":"2023-10-08T05:36:18.450376+0000","last_became_active":"2023-10-08T05:36:16.220450+0000","last_became_peered":"2023-10-08T05:36:16.220450+0000","last_unstale":"2023-10-08T05:36:18.450376+0000","last_undegraded":"2023-10-08T05:36:18.450376+0000","last_fullsized":"2023-10-08T05:36:18.450376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:45.730636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239438+0000","last_change":"2023-10-08T05:36:11.187132+0000","last_active":"2023-10-08T05:36:16.239438+0000","last_peered":"2023-10-08T05:36:16.239438+0000","last_clean":"2023-10-08T05:36:16.239438+0000","last_became_active":"2023-10-08T05:36:11.186856+0000","last_became_peered":"2023-10-08T05:36:11.186856+0000","last_unstale":"2023-10-08T05:36:16.239438+0000","last_undegraded":"2023-10-08T05:36:16.239438+0000","last_fullsized":"2023-10-08T05:36:16.239438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:42.738249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215030+0000","last_change":"2023-10-08T05:36:11.946617+0000","last_active":"2023-10-08T05:36:16.215030+0000","last_peered":"2023-10-08T05:36:16.215030+0000","last_clean":"2023-10-08T05:36:16.215030+0000","last_became_active":"2023-10-08T05:36:11.946360+0000","last_became_peered":"2023-10-08T05:36:11.946360+0000","last_unstale":"2023-10-08T05:36:16.215030+0000","last_undegraded":"2023-10-08T05:36:16.215030+0000","last_fullsized":"2023-10-08T05:36:16.215030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:21.717397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450713+0000","last_change":"2023-10-08T05:36:16.229002+0000","last_active":"2023-10-08T05:36:18.450713+0000","last_peered":"2023-10-08T05:36:18.450713+0000","last_clean":"2023-10-08T05:36:18.450713+0000","last_became_active":"2023-10-08T05:36:16.228230+0000","last_became_peered":"2023-10-08T05:36:16.228230+0000","last_unstale":"2023-10-08T05:36:18.450713+0000","last_undegraded":"2023-10-08T05:36:18.450713+0000","last_fullsized":"2023-10-08T05:36:18.450713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:33.520862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239150+0000","last_change":"2023-10-08T05:36:11.191776+0000","last_active":"2023-10-08T05:36:16.239150+0000","last_peered":"2023-10-08T05:36:16.239150+0000","last_clean":"2023-10-08T05:36:16.239150+0000","last_became_active":"2023-10-08T05:36:11.191532+0000","last_became_peered":"2023-10-08T05:36:11.191532+0000","last_unstale":"2023-10-08T05:36:16.239150+0000","last_undegraded":"2023-10-08T05:36:16.239150+0000","last_fullsized":"2023-10-08T05:36:16.239150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:16.799817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.217226+0000","last_change":"2023-10-08T05:36:11.950652+0000","last_active":"2023-10-08T05:36:16.217226+0000","last_peered":"2023-10-08T05:36:16.217226+0000","last_clean":"2023-10-08T05:36:16.217226+0000","last_became_active":"2023-10-08T05:36:11.950359+0000","last_became_peered":"2023-10-08T05:36:11.950359+0000","last_unstale":"2023-10-08T05:36:16.217226+0000","last_undegraded":"2023-10-08T05:36:16.217226+0000","last_fullsized":"2023-10-08T05:36:16.217226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451049+0000","last_change":"2023-10-08T05:36:16.222866+0000","last_active":"2023-10-08T05:36:18.451049+0000","last_peered":"2023-10-08T05:36:18.451049+0000","last_clean":"2023-10-08T05:36:18.451049+0000","last_became_active":"2023-10-08T05:36:16.222444+0000","last_became_peered":"2023-10-08T05:36:16.222444+0000","last_unstale":"2023-10-08T05:36:18.451049+0000","last_undegraded":"2023-10-08T05:36:18.451049+0000","last_fullsized":"2023-10-08T05:36:18.451049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:04.460179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195161+0000","last_change":"2023-10-08T05:36:11.936259+0000","last_active":"2023-10-08T05:36:16.195161+0000","last_peered":"2023-10-08T05:36:16.195161+0000","last_clean":"2023-10-08T05:36:16.195161+0000","last_became_active":"2023-10-08T05:36:11.935923+0000","last_became_peered":"2023-10-08T05:36:11.935923+0000","last_unstale":"2023-10-08T05:36:16.195161+0000","last_undegraded":"2023-10-08T05:36:16.195161+0000","last_fullsized":"2023-10-08T05:36:16.195161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239194+0000","last_change":"2023-10-08T05:36:11.187292+0000","last_active":"2023-10-08T05:36:16.239194+0000","last_peered":"2023-10-08T05:36:16.239194+0000","last_clean":"2023-10-08T05:36:16.239194+0000","last_became_active":"2023-10-08T05:36:11.186706+0000","last_became_peered":"2023-10-08T05:36:11.186706+0000","last_unstale":"2023-10-08T05:36:16.239194+0000","last_undegraded":"2023-10-08T05:36:16.239194+0000","last_fullsized":"2023-10-08T05:36:16.239194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:22.361671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451570+0000","last_change":"2023-10-08T05:36:16.228736+0000","last_active":"2023-10-08T05:36:18.451570+0000","last_peered":"2023-10-08T05:36:18.451570+0000","last_clean":"2023-10-08T05:36:18.451570+0000","last_became_active":"2023-10-08T05:36:16.227909+0000","last_became_peered":"2023-10-08T05:36:16.227909+0000","last_unstale":"2023-10-08T05:36:18.451570+0000","last_undegraded":"2023-10-08T05:36:18.451570+0000","last_fullsized":"2023-10-08T05:36:18.451570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:29.122804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194544+0000","last_change":"2023-10-08T05:36:11.945361+0000","last_active":"2023-10-08T05:36:16.194544+0000","last_peered":"2023-10-08T05:36:16.194544+0000","last_clean":"2023-10-08T05:36:16.194544+0000","last_became_active":"2023-10-08T05:36:11.944819+0000","last_became_peered":"2023-10-08T05:36:11.944819+0000","last_unstale":"2023-10-08T05:36:16.194544+0000","last_undegraded":"2023-10-08T05:36:16.194544+0000","last_fullsized":"2023-10-08T05:36:16.194544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:56.745041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.208276+0000","last_change":"2023-10-08T05:36:11.164749+0000","last_active":"2023-10-08T05:36:16.208276+0000","last_peered":"2023-10-08T05:36:16.208276+0000","last_clean":"2023-10-08T05:36:16.208276+0000","last_became_active":"2023-10-08T05:36:11.164382+0000","last_became_peered":"2023-10-08T05:36:11.164382+0000","last_unstale":"2023-10-08T05:36:16.208276+0000","last_undegraded":"2023-10-08T05:36:16.208276+0000","last_fullsized":"2023-10-08T05:36:16.208276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:13.509993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450966+0000","last_change":"2023-10-08T05:36:16.229430+0000","last_active":"2023-10-08T05:36:18.450966+0000","last_peered":"2023-10-08T05:36:18.450966+0000","last_clean":"2023-10-08T05:36:18.450966+0000","last_became_active":"2023-10-08T05:36:16.228957+0000","last_became_peered":"2023-10-08T05:36:16.228957+0000","last_unstale":"2023-10-08T05:36:18.450966+0000","last_undegraded":"2023-10-08T05:36:18.450966+0000","last_fullsized":"2023-10-08T05:36:18.450966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:00.782220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195462+0000","last_change":"2023-10-08T05:36:11.934612+0000","last_active":"2023-10-08T05:36:16.195462+0000","last_peered":"2023-10-08T05:36:16.195462+0000","last_clean":"2023-10-08T05:36:16.195462+0000","last_became_active":"2023-10-08T05:36:11.934284+0000","last_became_peered":"2023-10-08T05:36:11.934284+0000","last_unstale":"2023-10-08T05:36:16.195462+0000","last_undegraded":"2023-10-08T05:36:16.195462+0000","last_fullsized":"2023-10-08T05:36:16.195462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203211+0000","last_change":"2023-10-08T05:36:11.182216+0000","last_active":"2023-10-08T05:36:16.203211+0000","last_peered":"2023-10-08T05:36:16.203211+0000","last_clean":"2023-10-08T05:36:16.203211+0000","last_became_active":"2023-10-08T05:36:11.181927+0000","last_became_peered":"2023-10-08T05:36:11.181927+0000","last_unstale":"2023-10-08T05:36:16.203211+0000","last_undegraded":"2023-10-08T05:36:16.203211+0000","last_fullsized":"2023-10-08T05:36:16.203211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:42.738249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.917366+0000","last_change":"2023-10-08T05:36:16.243897+0000","last_active":"2023-10-08T05:36:16.917366+0000","last_peered":"2023-10-08T05:36:16.917366+0000","last_clean":"2023-10-08T05:36:16.917366+0000","last_became_active":"2023-10-08T05:36:16.243421+0000","last_became_peered":"2023-10-08T05:36:16.243421+0000","last_unstale":"2023-10-08T05:36:16.917366+0000","last_undegraded":"2023-10-08T05:36:16.917366+0000","last_fullsized":"2023-10-08T05:36:16.917366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:16.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.209892+0000","last_change":"2023-10-08T05:36:11.934209+0000","last_active":"2023-10-08T05:36:16.209892+0000","last_peered":"2023-10-08T05:36:16.209892+0000","last_clean":"2023-10-08T05:36:16.209892+0000","last_became_active":"2023-10-08T05:36:11.933067+0000","last_became_peered":"2023-10-08T05:36:11.933067+0000","last_unstale":"2023-10-08T05:36:16.209892+0000","last_undegraded":"2023-10-08T05:36:16.209892+0000","last_fullsized":"2023-10-08T05:36:16.209892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:17.827053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238717+0000","last_change":"2023-10-08T05:36:11.181498+0000","last_active":"2023-10-08T05:36:16.238717+0000","last_peered":"2023-10-08T05:36:16.238717+0000","last_clean":"2023-10-08T05:36:16.238717+0000","last_became_active":"2023-10-08T05:36:11.181264+0000","last_became_peered":"2023-10-08T05:36:11.181264+0000","last_unstale":"2023-10-08T05:36:16.238717+0000","last_undegraded":"2023-10-08T05:36:16.238717+0000","last_fullsized":"2023-10-08T05:36:16.238717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:35.005158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.452011+0000","last_change":"2023-10-08T05:36:16.228065+0000","last_active":"2023-10-08T05:36:18.452011+0000","last_peered":"2023-10-08T05:36:18.452011+0000","last_clean":"2023-10-08T05:36:18.452011+0000","last_became_active":"2023-10-08T05:36:16.227665+0000","last_became_peered":"2023-10-08T05:36:16.227665+0000","last_unstale":"2023-10-08T05:36:18.452011+0000","last_undegraded":"2023-10-08T05:36:18.452011+0000","last_fullsized":"2023-10-08T05:36:18.452011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:32.619157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205947+0000","last_change":"2023-10-08T05:36:11.934150+0000","last_active":"2023-10-08T05:36:16.205947+0000","last_peered":"2023-10-08T05:36:16.205947+0000","last_clean":"2023-10-08T05:36:16.205947+0000","last_became_active":"2023-10-08T05:36:11.933875+0000","last_became_peered":"2023-10-08T05:36:11.933875+0000","last_unstale":"2023-10-08T05:36:16.205947+0000","last_undegraded":"2023-10-08T05:36:16.205947+0000","last_fullsized":"2023-10-08T05:36:16.205947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196218+0000","last_change":"2023-10-08T05:36:11.187989+0000","last_active":"2023-10-08T05:36:16.196218+0000","last_peered":"2023-10-08T05:36:16.196218+0000","last_clean":"2023-10-08T05:36:16.196218+0000","last_became_active":"2023-10-08T05:36:11.187695+0000","last_became_peered":"2023-10-08T05:36:11.187695+0000","last_unstale":"2023-10-08T05:36:16.196218+0000","last_undegraded":"2023-10-08T05:36:16.196218+0000","last_fullsized":"2023-10-08T05:36:16.196218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:35.005158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451149+0000","last_change":"2023-10-08T05:36:16.244139+0000","last_active":"2023-10-08T05:36:18.451149+0000","last_peered":"2023-10-08T05:36:18.451149+0000","last_clean":"2023-10-08T05:36:18.451149+0000","last_became_active":"2023-10-08T05:36:16.243569+0000","last_became_peered":"2023-10-08T05:36:16.243569+0000","last_unstale":"2023-10-08T05:36:18.451149+0000","last_undegraded":"2023-10-08T05:36:18.451149+0000","last_fullsized":"2023-10-08T05:36:18.451149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:30.440133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198170+0000","last_change":"2023-10-08T05:36:11.941731+0000","last_active":"2023-10-08T05:36:16.198170+0000","last_peered":"2023-10-08T05:36:16.198170+0000","last_clean":"2023-10-08T05:36:16.198170+0000","last_became_active":"2023-10-08T05:36:11.941393+0000","last_became_peered":"2023-10-08T05:36:11.941393+0000","last_unstale":"2023-10-08T05:36:16.198170+0000","last_undegraded":"2023-10-08T05:36:16.198170+0000","last_fullsized":"2023-10-08T05:36:16.198170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207549+0000","last_change":"2023-10-08T05:36:11.180784+0000","last_active":"2023-10-08T05:36:16.207549+0000","last_peered":"2023-10-08T05:36:16.207549+0000","last_clean":"2023-10-08T05:36:16.207549+0000","last_became_active":"2023-10-08T05:36:11.180398+0000","last_became_peered":"2023-10-08T05:36:11.180398+0000","last_unstale":"2023-10-08T05:36:16.207549+0000","last_undegraded":"2023-10-08T05:36:16.207549+0000","last_fullsized":"2023-10-08T05:36:16.207549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:49.167371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913389+0000","last_change":"2023-10-08T05:36:16.240627+0000","last_active":"2023-10-08T05:36:16.913389+0000","last_peered":"2023-10-08T05:36:16.913389+0000","last_clean":"2023-10-08T05:36:16.913389+0000","last_became_active":"2023-10-08T05:36:16.240153+0000","last_became_peered":"2023-10-08T05:36:16.240153+0000","last_unstale":"2023-10-08T05:36:16.913389+0000","last_undegraded":"2023-10-08T05:36:16.913389+0000","last_fullsized":"2023-10-08T05:36:16.913389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:06.106435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207447+0000","last_change":"2023-10-08T05:36:11.941834+0000","last_active":"2023-10-08T05:36:16.207447+0000","last_peered":"2023-10-08T05:36:16.207447+0000","last_clean":"2023-10-08T05:36:16.207447+0000","last_became_active":"2023-10-08T05:36:11.941546+0000","last_became_peered":"2023-10-08T05:36:11.941546+0000","last_unstale":"2023-10-08T05:36:16.207447+0000","last_undegraded":"2023-10-08T05:36:16.207447+0000","last_fullsized":"2023-10-08T05:36:16.207447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239749+0000","last_change":"2023-10-08T05:36:11.182530+0000","last_active":"2023-10-08T05:36:16.239749+0000","last_peered":"2023-10-08T05:36:16.239749+0000","last_clean":"2023-10-08T05:36:16.239749+0000","last_became_active":"2023-10-08T05:36:11.182156+0000","last_became_peered":"2023-10-08T05:36:11.182156+0000","last_unstale":"2023-10-08T05:36:16.239749+0000","last_undegraded":"2023-10-08T05:36:16.239749+0000","last_fullsized":"2023-10-08T05:36:16.239749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:10.731974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449389+0000","last_change":"2023-10-08T05:36:16.240659+0000","last_active":"2023-10-08T05:36:18.449389+0000","last_peered":"2023-10-08T05:36:18.449389+0000","last_clean":"2023-10-08T05:36:18.449389+0000","last_became_active":"2023-10-08T05:36:16.240244+0000","last_became_peered":"2023-10-08T05:36:16.240244+0000","last_unstale":"2023-10-08T05:36:18.449389+0000","last_undegraded":"2023-10-08T05:36:18.449389+0000","last_fullsized":"2023-10-08T05:36:18.449389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:15.664459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.216378+0000","last_change":"2023-10-08T05:36:11.943995+0000","last_active":"2023-10-08T05:36:16.216378+0000","last_peered":"2023-10-08T05:36:16.216378+0000","last_clean":"2023-10-08T05:36:16.216378+0000","last_became_active":"2023-10-08T05:36:11.943601+0000","last_became_peered":"2023-10-08T05:36:11.943601+0000","last_unstale":"2023-10-08T05:36:16.216378+0000","last_undegraded":"2023-10-08T05:36:16.216378+0000","last_fullsized":"2023-10-08T05:36:16.216378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:28.581977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207661+0000","last_change":"2023-10-08T05:36:11.166965+0000","last_active":"2023-10-08T05:36:16.207661+0000","last_peered":"2023-10-08T05:36:16.207661+0000","last_clean":"2023-10-08T05:36:16.207661+0000","last_became_active":"2023-10-08T05:36:11.166704+0000","last_became_peered":"2023-10-08T05:36:11.166704+0000","last_unstale":"2023-10-08T05:36:16.207661+0000","last_undegraded":"2023-10-08T05:36:16.207661+0000","last_fullsized":"2023-10-08T05:36:16.207661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.915670+0000","last_change":"2023-10-08T05:36:16.226001+0000","last_active":"2023-10-08T05:36:16.915670+0000","last_peered":"2023-10-08T05:36:16.915670+0000","last_clean":"2023-10-08T05:36:16.915670+0000","last_became_active":"2023-10-08T05:36:16.225623+0000","last_became_peered":"2023-10-08T05:36:16.225623+0000","last_unstale":"2023-10-08T05:36:16.915670+0000","last_undegraded":"2023-10-08T05:36:16.915670+0000","last_fullsized":"2023-10-08T05:36:16.915670+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213447+0000","last_change":"2023-10-08T05:36:11.935396+0000","last_active":"2023-10-08T05:36:16.213447+0000","last_peered":"2023-10-08T05:36:16.213447+0000","last_clean":"2023-10-08T05:36:16.213447+0000","last_became_active":"2023-10-08T05:36:11.934642+0000","last_became_peered":"2023-10-08T05:36:11.934642+0000","last_unstale":"2023-10-08T05:36:16.213447+0000","last_undegraded":"2023-10-08T05:36:16.213447+0000","last_fullsized":"2023-10-08T05:36:16.213447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:09.850609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215985+0000","last_change":"2023-10-08T05:36:11.189030+0000","last_active":"2023-10-08T05:36:16.215985+0000","last_peered":"2023-10-08T05:36:16.215985+0000","last_clean":"2023-10-08T05:36:16.215985+0000","last_became_active":"2023-10-08T05:36:11.184720+0000","last_became_peered":"2023-10-08T05:36:11.184720+0000","last_unstale":"2023-10-08T05:36:16.215985+0000","last_undegraded":"2023-10-08T05:36:16.215985+0000","last_fullsized":"2023-10-08T05:36:16.215985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:41.625077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.916396+0000","last_change":"2023-10-08T05:36:16.223249+0000","last_active":"2023-10-08T05:36:16.916396+0000","last_peered":"2023-10-08T05:36:16.916396+0000","last_clean":"2023-10-08T05:36:16.916396+0000","last_became_active":"2023-10-08T05:36:16.222656+0000","last_became_peered":"2023-10-08T05:36:16.222656+0000","last_unstale":"2023-10-08T05:36:16.916396+0000","last_undegraded":"2023-10-08T05:36:16.916396+0000","last_fullsized":"2023-10-08T05:36:16.916396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:25.754577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203206+0000","last_change":"2023-10-08T05:36:11.942804+0000","last_active":"2023-10-08T05:36:16.203206+0000","last_peered":"2023-10-08T05:36:16.203206+0000","last_clean":"2023-10-08T05:36:16.203206+0000","last_became_active":"2023-10-08T05:36:11.942486+0000","last_became_peered":"2023-10-08T05:36:11.942486+0000","last_unstale":"2023-10-08T05:36:16.203206+0000","last_undegraded":"2023-10-08T05:36:16.203206+0000","last_fullsized":"2023-10-08T05:36:16.203206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190110+0000","last_change":"2023-10-08T05:36:11.160795+0000","last_active":"2023-10-08T05:36:16.190110+0000","last_peered":"2023-10-08T05:36:16.190110+0000","last_clean":"2023-10-08T05:36:16.190110+0000","last_became_active":"2023-10-08T05:36:11.160498+0000","last_became_peered":"2023-10-08T05:36:11.160498+0000","last_unstale":"2023-10-08T05:36:16.190110+0000","last_undegraded":"2023-10-08T05:36:16.190110+0000","last_fullsized":"2023-10-08T05:36:16.190110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:08.847794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.917236+0000","last_change":"2023-10-08T05:36:16.224778+0000","last_active":"2023-10-08T05:36:16.917236+0000","last_peered":"2023-10-08T05:36:16.917236+0000","last_clean":"2023-10-08T05:36:16.917236+0000","last_became_active":"2023-10-08T05:36:16.224249+0000","last_became_peered":"2023-10-08T05:36:16.224249+0000","last_unstale":"2023-10-08T05:36:16.917236+0000","last_undegraded":"2023-10-08T05:36:16.917236+0000","last_fullsized":"2023-10-08T05:36:16.917236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:07.031678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.204855+0000","last_change":"2023-10-08T05:36:11.944248+0000","last_active":"2023-10-08T05:36:16.204855+0000","last_peered":"2023-10-08T05:36:16.204855+0000","last_clean":"2023-10-08T05:36:16.204855+0000","last_became_active":"2023-10-08T05:36:11.943941+0000","last_became_peered":"2023-10-08T05:36:11.943941+0000","last_unstale":"2023-10-08T05:36:16.204855+0000","last_undegraded":"2023-10-08T05:36:16.204855+0000","last_fullsized":"2023-10-08T05:36:16.204855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238762+0000","last_change":"2023-10-08T05:36:11.186476+0000","last_active":"2023-10-08T05:36:16.238762+0000","last_peered":"2023-10-08T05:36:16.238762+0000","last_clean":"2023-10-08T05:36:16.238762+0000","last_became_active":"2023-10-08T05:36:11.186165+0000","last_became_peered":"2023-10-08T05:36:11.186165+0000","last_unstale":"2023-10-08T05:36:16.238762+0000","last_undegraded":"2023-10-08T05:36:16.238762+0000","last_fullsized":"2023-10-08T05:36:16.238762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:08.838858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450006+0000","last_change":"2023-10-08T05:36:16.241072+0000","last_active":"2023-10-08T05:36:18.450006+0000","last_peered":"2023-10-08T05:36:18.450006+0000","last_clean":"2023-10-08T05:36:18.450006+0000","last_became_active":"2023-10-08T05:36:16.240154+0000","last_became_peered":"2023-10-08T05:36:16.240154+0000","last_unstale":"2023-10-08T05:36:18.450006+0000","last_undegraded":"2023-10-08T05:36:18.450006+0000","last_fullsized":"2023-10-08T05:36:18.450006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:16.535660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194060+0000","last_change":"2023-10-08T05:36:11.936616+0000","last_active":"2023-10-08T05:36:16.194060+0000","last_peered":"2023-10-08T05:36:16.194060+0000","last_clean":"2023-10-08T05:36:16.194060+0000","last_became_active":"2023-10-08T05:36:11.936320+0000","last_became_peered":"2023-10-08T05:36:11.936320+0000","last_unstale":"2023-10-08T05:36:16.194060+0000","last_undegraded":"2023-10-08T05:36:16.194060+0000","last_fullsized":"2023-10-08T05:36:16.194060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215473+0000","last_change":"2023-10-08T05:36:11.182263+0000","last_active":"2023-10-08T05:36:16.215473+0000","last_peered":"2023-10-08T05:36:16.215473+0000","last_clean":"2023-10-08T05:36:16.215473+0000","last_became_active":"2023-10-08T05:36:11.181876+0000","last_became_peered":"2023-10-08T05:36:11.181876+0000","last_unstale":"2023-10-08T05:36:16.215473+0000","last_undegraded":"2023-10-08T05:36:16.215473+0000","last_fullsized":"2023-10-08T05:36:16.215473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:16.815302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":169,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238841+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:36:16.238841+0000","last_peered":"2023-10-08T05:36:16.238841+0000","last_clean":"2023-10-08T05:36:16.238841+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:36:16.238841+0000","last_undegraded":"2023-10-08T05:36:16.238841+0000","last_fullsized":"2023-10-08T05:36:16.238841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914962+0000","last_change":"2023-10-08T05:36:16.241252+0000","last_active":"2023-10-08T05:36:16.914962+0000","last_peered":"2023-10-08T05:36:16.914962+0000","last_clean":"2023-10-08T05:36:16.914962+0000","last_became_active":"2023-10-08T05:36:16.240543+0000","last_became_peered":"2023-10-08T05:36:16.240543+0000","last_unstale":"2023-10-08T05:36:16.914962+0000","last_undegraded":"2023-10-08T05:36:16.914962+0000","last_fullsized":"2023-10-08T05:36:16.914962+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:20.640196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239326+0000","last_change":"2023-10-08T05:36:11.954327+0000","last_active":"2023-10-08T05:36:16.239326+0000","last_peered":"2023-10-08T05:36:16.239326+0000","last_clean":"2023-10-08T05:36:16.239326+0000","last_became_active":"2023-10-08T05:36:11.953852+0000","last_became_peered":"2023-10-08T05:36:11.953852+0000","last_unstale":"2023-10-08T05:36:16.239326+0000","last_undegraded":"2023-10-08T05:36:16.239326+0000","last_fullsized":"2023-10-08T05:36:16.239326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:12.498480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190495+0000","last_change":"2023-10-08T05:36:11.179188+0000","last_active":"2023-10-08T05:36:16.190495+0000","last_peered":"2023-10-08T05:36:16.190495+0000","last_clean":"2023-10-08T05:36:16.190495+0000","last_became_active":"2023-10-08T05:36:11.178872+0000","last_became_peered":"2023-10-08T05:36:11.178872+0000","last_unstale":"2023-10-08T05:36:16.190495+0000","last_undegraded":"2023-10-08T05:36:16.190495+0000","last_fullsized":"2023-10-08T05:36:16.190495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:44.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.915127+0000","last_change":"2023-10-08T05:36:16.241452+0000","last_active":"2023-10-08T05:36:16.915127+0000","last_peered":"2023-10-08T05:36:16.915127+0000","last_clean":"2023-10-08T05:36:16.915127+0000","last_became_active":"2023-10-08T05:36:16.240541+0000","last_became_peered":"2023-10-08T05:36:16.240541+0000","last_unstale":"2023-10-08T05:36:16.915127+0000","last_undegraded":"2023-10-08T05:36:16.915127+0000","last_fullsized":"2023-10-08T05:36:16.915127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215138+0000","last_change":"2023-10-08T05:36:11.948239+0000","last_active":"2023-10-08T05:36:16.215138+0000","last_peered":"2023-10-08T05:36:16.215138+0000","last_clean":"2023-10-08T05:36:16.215138+0000","last_became_active":"2023-10-08T05:36:11.947960+0000","last_became_peered":"2023-10-08T05:36:11.947960+0000","last_unstale":"2023-10-08T05:36:16.215138+0000","last_undegraded":"2023-10-08T05:36:16.215138+0000","last_fullsized":"2023-10-08T05:36:16.215138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:02.994498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207516+0000","last_change":"2023-10-08T05:36:11.173210+0000","last_active":"2023-10-08T05:36:16.207516+0000","last_peered":"2023-10-08T05:36:16.207516+0000","last_clean":"2023-10-08T05:36:16.207516+0000","last_became_active":"2023-10-08T05:36:11.172899+0000","last_became_peered":"2023-10-08T05:36:11.172899+0000","last_unstale":"2023-10-08T05:36:16.207516+0000","last_undegraded":"2023-10-08T05:36:16.207516+0000","last_fullsized":"2023-10-08T05:36:16.207516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:59.559506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449356+0000","last_change":"2023-10-08T05:36:16.226571+0000","last_active":"2023-10-08T05:36:18.449356+0000","last_peered":"2023-10-08T05:36:18.449356+0000","last_clean":"2023-10-08T05:36:18.449356+0000","last_became_active":"2023-10-08T05:36:16.226098+0000","last_became_peered":"2023-10-08T05:36:16.226098+0000","last_unstale":"2023-10-08T05:36:18.449356+0000","last_undegraded":"2023-10-08T05:36:18.449356+0000","last_fullsized":"2023-10-08T05:36:18.449356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:01.920875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.246459+0000","last_change":"2023-10-08T05:36:11.948021+0000","last_active":"2023-10-08T05:36:16.246459+0000","last_peered":"2023-10-08T05:36:16.246459+0000","last_clean":"2023-10-08T05:36:16.246459+0000","last_became_active":"2023-10-08T05:36:11.947456+0000","last_became_peered":"2023-10-08T05:36:11.947456+0000","last_unstale":"2023-10-08T05:36:16.246459+0000","last_undegraded":"2023-10-08T05:36:16.246459+0000","last_fullsized":"2023-10-08T05:36:16.246459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:21.717397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.191082+0000","last_change":"2023-10-08T05:36:11.164907+0000","last_active":"2023-10-08T05:36:16.191082+0000","last_peered":"2023-10-08T05:36:16.191082+0000","last_clean":"2023-10-08T05:36:16.191082+0000","last_became_active":"2023-10-08T05:36:11.164630+0000","last_became_peered":"2023-10-08T05:36:11.164630+0000","last_unstale":"2023-10-08T05:36:16.191082+0000","last_undegraded":"2023-10-08T05:36:16.191082+0000","last_fullsized":"2023-10-08T05:36:16.191082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:58.268025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.347745+0000","last_change":"2023-10-08T05:36:16.241024+0000","last_active":"2023-10-08T05:36:17.347745+0000","last_peered":"2023-10-08T05:36:17.347745+0000","last_clean":"2023-10-08T05:36:17.347745+0000","last_became_active":"2023-10-08T05:36:16.240379+0000","last_became_peered":"2023-10-08T05:36:16.240379+0000","last_unstale":"2023-10-08T05:36:17.347745+0000","last_undegraded":"2023-10-08T05:36:17.347745+0000","last_fullsized":"2023-10-08T05:36:17.347745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:09:12.638961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240155+0000","last_change":"2023-10-08T05:36:11.946599+0000","last_active":"2023-10-08T05:36:16.240155+0000","last_peered":"2023-10-08T05:36:16.240155+0000","last_clean":"2023-10-08T05:36:16.240155+0000","last_became_active":"2023-10-08T05:36:11.946295+0000","last_became_peered":"2023-10-08T05:36:11.946295+0000","last_unstale":"2023-10-08T05:36:16.240155+0000","last_undegraded":"2023-10-08T05:36:16.240155+0000","last_fullsized":"2023-10-08T05:36:16.240155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:28.581977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214835+0000","last_change":"2023-10-08T05:36:11.181737+0000","last_active":"2023-10-08T05:36:16.214835+0000","last_peered":"2023-10-08T05:36:16.214835+0000","last_clean":"2023-10-08T05:36:16.214835+0000","last_became_active":"2023-10-08T05:36:11.181386+0000","last_became_peered":"2023-10-08T05:36:11.181386+0000","last_unstale":"2023-10-08T05:36:16.214835+0000","last_undegraded":"2023-10-08T05:36:16.214835+0000","last_fullsized":"2023-10-08T05:36:16.214835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:05.139990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.450290+0000","last_change":"2023-10-08T05:36:16.223098+0000","last_active":"2023-10-08T05:36:18.450290+0000","last_peered":"2023-10-08T05:36:18.450290+0000","last_clean":"2023-10-08T05:36:18.450290+0000","last_became_active":"2023-10-08T05:36:16.222649+0000","last_became_peered":"2023-10-08T05:36:16.222649+0000","last_unstale":"2023-10-08T05:36:18.450290+0000","last_undegraded":"2023-10-08T05:36:18.450290+0000","last_fullsized":"2023-10-08T05:36:18.450290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:27.410602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238995+0000","last_change":"2023-10-08T05:36:11.946258+0000","last_active":"2023-10-08T05:36:16.238995+0000","last_peered":"2023-10-08T05:36:16.238995+0000","last_clean":"2023-10-08T05:36:16.238995+0000","last_became_active":"2023-10-08T05:36:11.945932+0000","last_became_peered":"2023-10-08T05:36:11.945932+0000","last_unstale":"2023-10-08T05:36:16.238995+0000","last_undegraded":"2023-10-08T05:36:16.238995+0000","last_fullsized":"2023-10-08T05:36:16.238995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:16.603017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.189924+0000","last_change":"2023-10-08T05:36:11.162339+0000","last_active":"2023-10-08T05:36:16.189924+0000","last_peered":"2023-10-08T05:36:16.189924+0000","last_clean":"2023-10-08T05:36:16.189924+0000","last_became_active":"2023-10-08T05:36:11.162023+0000","last_became_peered":"2023-10-08T05:36:11.162023+0000","last_unstale":"2023-10-08T05:36:16.189924+0000","last_undegraded":"2023-10-08T05:36:16.189924+0000","last_fullsized":"2023-10-08T05:36:16.189924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:10.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449880+0000","last_change":"2023-10-08T05:36:16.217935+0000","last_active":"2023-10-08T05:36:18.449880+0000","last_peered":"2023-10-08T05:36:18.449880+0000","last_clean":"2023-10-08T05:36:18.449880+0000","last_became_active":"2023-10-08T05:36:16.217530+0000","last_became_peered":"2023-10-08T05:36:16.217530+0000","last_unstale":"2023-10-08T05:36:18.449880+0000","last_undegraded":"2023-10-08T05:36:18.449880+0000","last_fullsized":"2023-10-08T05:36:18.449880+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:07.031678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194389+0000","last_change":"2023-10-08T05:36:11.945050+0000","last_active":"2023-10-08T05:36:16.194389+0000","last_peered":"2023-10-08T05:36:16.194389+0000","last_clean":"2023-10-08T05:36:16.194389+0000","last_became_active":"2023-10-08T05:36:11.944419+0000","last_became_peered":"2023-10-08T05:36:11.944419+0000","last_unstale":"2023-10-08T05:36:16.194389+0000","last_undegraded":"2023-10-08T05:36:16.194389+0000","last_fullsized":"2023-10-08T05:36:16.194389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194769+0000","last_change":"2023-10-08T05:36:11.176221+0000","last_active":"2023-10-08T05:36:16.194769+0000","last_peered":"2023-10-08T05:36:16.194769+0000","last_clean":"2023-10-08T05:36:16.194769+0000","last_became_active":"2023-10-08T05:36:11.175839+0000","last_became_peered":"2023-10-08T05:36:11.175839+0000","last_unstale":"2023-10-08T05:36:16.194769+0000","last_undegraded":"2023-10-08T05:36:16.194769+0000","last_fullsized":"2023-10-08T05:36:16.194769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:44.301013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913652+0000","last_change":"2023-10-08T05:36:16.240543+0000","last_active":"2023-10-08T05:36:16.913652+0000","last_peered":"2023-10-08T05:36:16.913652+0000","last_clean":"2023-10-08T05:36:16.913652+0000","last_became_active":"2023-10-08T05:36:16.240104+0000","last_became_peered":"2023-10-08T05:36:16.240104+0000","last_unstale":"2023-10-08T05:36:16.913652+0000","last_undegraded":"2023-10-08T05:36:16.913652+0000","last_fullsized":"2023-10-08T05:36:16.913652+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:21.958405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239177+0000","last_change":"2023-10-08T05:36:11.952574+0000","last_active":"2023-10-08T05:36:16.239177+0000","last_peered":"2023-10-08T05:36:16.239177+0000","last_clean":"2023-10-08T05:36:16.239177+0000","last_became_active":"2023-10-08T05:36:11.952287+0000","last_became_peered":"2023-10-08T05:36:11.952287+0000","last_unstale":"2023-10-08T05:36:16.239177+0000","last_undegraded":"2023-10-08T05:36:16.239177+0000","last_fullsized":"2023-10-08T05:36:16.239177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215860+0000","last_change":"2023-10-08T05:36:11.183076+0000","last_active":"2023-10-08T05:36:16.215860+0000","last_peered":"2023-10-08T05:36:16.215860+0000","last_clean":"2023-10-08T05:36:16.215860+0000","last_became_active":"2023-10-08T05:36:11.182702+0000","last_became_peered":"2023-10-08T05:36:11.182702+0000","last_unstale":"2023-10-08T05:36:16.215860+0000","last_undegraded":"2023-10-08T05:36:16.215860+0000","last_fullsized":"2023-10-08T05:36:16.215860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:45.848562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.912758+0000","last_change":"2023-10-08T05:36:16.240654+0000","last_active":"2023-10-08T05:36:16.912758+0000","last_peered":"2023-10-08T05:36:16.912758+0000","last_clean":"2023-10-08T05:36:16.912758+0000","last_became_active":"2023-10-08T05:36:16.240179+0000","last_became_peered":"2023-10-08T05:36:16.240179+0000","last_unstale":"2023-10-08T05:36:16.912758+0000","last_undegraded":"2023-10-08T05:36:16.912758+0000","last_fullsized":"2023-10-08T05:36:16.912758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:09:12.638961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205225+0000","last_change":"2023-10-08T05:36:11.948816+0000","last_active":"2023-10-08T05:36:16.205225+0000","last_peered":"2023-10-08T05:36:16.205225+0000","last_clean":"2023-10-08T05:36:16.205225+0000","last_became_active":"2023-10-08T05:36:11.948501+0000","last_became_peered":"2023-10-08T05:36:11.948501+0000","last_unstale":"2023-10-08T05:36:16.205225+0000","last_undegraded":"2023-10-08T05:36:16.205225+0000","last_fullsized":"2023-10-08T05:36:16.205225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:43.750000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206453+0000","last_change":"2023-10-08T05:36:11.172445+0000","last_active":"2023-10-08T05:36:16.206453+0000","last_peered":"2023-10-08T05:36:16.206453+0000","last_clean":"2023-10-08T05:36:16.206453+0000","last_became_active":"2023-10-08T05:36:11.172090+0000","last_became_peered":"2023-10-08T05:36:11.172090+0000","last_unstale":"2023-10-08T05:36:16.206453+0000","last_undegraded":"2023-10-08T05:36:16.206453+0000","last_fullsized":"2023-10-08T05:36:16.206453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:58.268025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.348870+0000","last_change":"2023-10-08T05:36:16.240874+0000","last_active":"2023-10-08T05:36:17.348870+0000","last_peered":"2023-10-08T05:36:17.348870+0000","last_clean":"2023-10-08T05:36:17.348870+0000","last_became_active":"2023-10-08T05:36:16.240420+0000","last_became_peered":"2023-10-08T05:36:16.240420+0000","last_unstale":"2023-10-08T05:36:17.348870+0000","last_undegraded":"2023-10-08T05:36:17.348870+0000","last_fullsized":"2023-10-08T05:36:17.348870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:06.106435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239249+0000","last_change":"2023-10-08T05:36:11.949913+0000","last_active":"2023-10-08T05:36:16.239249+0000","last_peered":"2023-10-08T05:36:16.239249+0000","last_clean":"2023-10-08T05:36:16.239249+0000","last_became_active":"2023-10-08T05:36:11.946993+0000","last_became_peered":"2023-10-08T05:36:11.946993+0000","last_unstale":"2023-10-08T05:36:16.239249+0000","last_undegraded":"2023-10-08T05:36:16.239249+0000","last_fullsized":"2023-10-08T05:36:16.239249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:02.994498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:31.450726+0000","last_change":"2023-10-08T05:36:11.177181+0000","last_active":"2023-10-08T05:36:31.450726+0000","last_peered":"2023-10-08T05:36:31.450726+0000","last_clean":"2023-10-08T05:36:31.450726+0000","last_became_active":"2023-10-08T05:36:11.176830+0000","last_became_peered":"2023-10-08T05:36:11.176830+0000","last_unstale":"2023-10-08T05:36:31.450726+0000","last_undegraded":"2023-10-08T05:36:31.450726+0000","last_fullsized":"2023-10-08T05:36:31.450726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:13.509993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.916338+0000","last_change":"2023-10-08T05:36:16.223391+0000","last_active":"2023-10-08T05:36:16.916338+0000","last_peered":"2023-10-08T05:36:16.916338+0000","last_clean":"2023-10-08T05:36:16.916338+0000","last_became_active":"2023-10-08T05:36:16.222997+0000","last_became_peered":"2023-10-08T05:36:16.222997+0000","last_unstale":"2023-10-08T05:36:16.916338+0000","last_undegraded":"2023-10-08T05:36:16.916338+0000","last_fullsized":"2023-10-08T05:36:16.916338+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:15.664459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213269+0000","last_change":"2023-10-08T05:36:11.936436+0000","last_active":"2023-10-08T05:36:16.213269+0000","last_peered":"2023-10-08T05:36:16.213269+0000","last_clean":"2023-10-08T05:36:16.213269+0000","last_became_active":"2023-10-08T05:36:11.936161+0000","last_became_peered":"2023-10-08T05:36:11.936161+0000","last_unstale":"2023-10-08T05:36:16.213269+0000","last_undegraded":"2023-10-08T05:36:16.213269+0000","last_fullsized":"2023-10-08T05:36:16.213269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:36.016909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.207221+0000","last_change":"2023-10-08T05:36:11.183582+0000","last_active":"2023-10-08T05:36:16.207221+0000","last_peered":"2023-10-08T05:36:16.207221+0000","last_clean":"2023-10-08T05:36:16.207221+0000","last_became_active":"2023-10-08T05:36:11.183264+0000","last_became_peered":"2023-10-08T05:36:11.183264+0000","last_unstale":"2023-10-08T05:36:16.207221+0000","last_undegraded":"2023-10-08T05:36:16.207221+0000","last_fullsized":"2023-10-08T05:36:16.207221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:30:16.799817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.451297+0000","last_change":"2023-10-08T05:36:16.228720+0000","last_active":"2023-10-08T05:36:18.451297+0000","last_peered":"2023-10-08T05:36:18.451297+0000","last_clean":"2023-10-08T05:36:18.451297+0000","last_became_active":"2023-10-08T05:36:16.227638+0000","last_became_peered":"2023-10-08T05:36:16.227638+0000","last_unstale":"2023-10-08T05:36:18.451297+0000","last_undegraded":"2023-10-08T05:36:18.451297+0000","last_fullsized":"2023-10-08T05:36:18.451297+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:20.640196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205640+0000","last_change":"2023-10-08T05:36:11.933926+0000","last_active":"2023-10-08T05:36:16.205640+0000","last_peered":"2023-10-08T05:36:16.205640+0000","last_clean":"2023-10-08T05:36:16.205640+0000","last_became_active":"2023-10-08T05:36:11.933648+0000","last_became_peered":"2023-10-08T05:36:11.933648+0000","last_unstale":"2023-10-08T05:36:16.205640+0000","last_undegraded":"2023-10-08T05:36:16.205640+0000","last_fullsized":"2023-10-08T05:36:16.205640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215215+0000","last_change":"2023-10-08T05:36:11.183379+0000","last_active":"2023-10-08T05:36:16.215215+0000","last_peered":"2023-10-08T05:36:16.215215+0000","last_clean":"2023-10-08T05:36:16.215215+0000","last_became_active":"2023-10-08T05:36:11.182862+0000","last_became_peered":"2023-10-08T05:36:11.182862+0000","last_unstale":"2023-10-08T05:36:16.215215+0000","last_undegraded":"2023-10-08T05:36:16.215215+0000","last_fullsized":"2023-10-08T05:36:16.215215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:10.147911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914439+0000","last_change":"2023-10-08T05:36:16.216716+0000","last_active":"2023-10-08T05:36:16.914439+0000","last_peered":"2023-10-08T05:36:16.914439+0000","last_clean":"2023-10-08T05:36:16.914439+0000","last_became_active":"2023-10-08T05:36:16.216266+0000","last_became_peered":"2023-10-08T05:36:16.216266+0000","last_unstale":"2023-10-08T05:36:16.914439+0000","last_undegraded":"2023-10-08T05:36:16.914439+0000","last_fullsized":"2023-10-08T05:36:16.914439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:45.158099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214937+0000","last_change":"2023-10-08T05:36:11.949702+0000","last_active":"2023-10-08T05:36:16.214937+0000","last_peered":"2023-10-08T05:36:16.214937+0000","last_clean":"2023-10-08T05:36:16.214937+0000","last_became_active":"2023-10-08T05:36:11.949421+0000","last_became_peered":"2023-10-08T05:36:11.949421+0000","last_unstale":"2023-10-08T05:36:16.214937+0000","last_undegraded":"2023-10-08T05:36:16.214937+0000","last_fullsized":"2023-10-08T05:36:16.214937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195931+0000","last_change":"2023-10-08T05:36:11.189802+0000","last_active":"2023-10-08T05:36:16.195931+0000","last_peered":"2023-10-08T05:36:16.195931+0000","last_clean":"2023-10-08T05:36:16.195931+0000","last_became_active":"2023-10-08T05:36:11.189447+0000","last_became_peered":"2023-10-08T05:36:11.189447+0000","last_unstale":"2023-10-08T05:36:16.195931+0000","last_undegraded":"2023-10-08T05:36:16.195931+0000","last_fullsized":"2023-10-08T05:36:16.195931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:08.847794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.911933+0000","last_change":"2023-10-08T05:36:16.241184+0000","last_active":"2023-10-08T05:36:16.911933+0000","last_peered":"2023-10-08T05:36:16.911933+0000","last_clean":"2023-10-08T05:36:16.911933+0000","last_became_active":"2023-10-08T05:36:16.240531+0000","last_became_peered":"2023-10-08T05:36:16.240531+0000","last_unstale":"2023-10-08T05:36:16.911933+0000","last_undegraded":"2023-10-08T05:36:16.911933+0000","last_fullsized":"2023-10-08T05:36:16.911933+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:17:23.235254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:18.449495+0000","last_change":"2023-10-08T05:36:16.241230+0000","last_active":"2023-10-08T05:36:18.449495+0000","last_peered":"2023-10-08T05:36:18.449495+0000","last_clean":"2023-10-08T05:36:18.449495+0000","last_became_active":"2023-10-08T05:36:16.240814+0000","last_became_peered":"2023-10-08T05:36:16.240814+0000","last_unstale":"2023-10-08T05:36:18.449495+0000","last_undegraded":"2023-10-08T05:36:18.449495+0000","last_fullsized":"2023-10-08T05:36:18.449495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:25.754577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.202997+0000","last_change":"2023-10-08T05:36:11.941419+0000","last_active":"2023-10-08T05:36:16.202997+0000","last_peered":"2023-10-08T05:36:16.202997+0000","last_clean":"2023-10-08T05:36:16.202997+0000","last_became_active":"2023-10-08T05:36:11.941131+0000","last_became_peered":"2023-10-08T05:36:11.941131+0000","last_unstale":"2023-10-08T05:36:16.202997+0000","last_undegraded":"2023-10-08T05:36:16.202997+0000","last_fullsized":"2023-10-08T05:36:16.202997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214657+0000","last_change":"2023-10-08T05:36:11.182594+0000","last_active":"2023-10-08T05:36:16.214657+0000","last_peered":"2023-10-08T05:36:16.214657+0000","last_clean":"2023-10-08T05:36:16.214657+0000","last_became_active":"2023-10-08T05:36:11.182297+0000","last_became_peered":"2023-10-08T05:36:11.182297+0000","last_unstale":"2023-10-08T05:36:16.214657+0000","last_undegraded":"2023-10-08T05:36:16.214657+0000","last_fullsized":"2023-10-08T05:36:16.214657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:59.799745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.914977+0000","last_change":"2023-10-08T05:36:16.220333+0000","last_active":"2023-10-08T05:36:16.914977+0000","last_peered":"2023-10-08T05:36:16.914977+0000","last_clean":"2023-10-08T05:36:16.914977+0000","last_became_active":"2023-10-08T05:36:16.219836+0000","last_became_peered":"2023-10-08T05:36:16.219836+0000","last_unstale":"2023-10-08T05:36:16.914977+0000","last_undegraded":"2023-10-08T05:36:16.914977+0000","last_fullsized":"2023-10-08T05:36:16.914977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:32.619157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195436+0000","last_change":"2023-10-08T05:36:11.945676+0000","last_active":"2023-10-08T05:36:16.195436+0000","last_peered":"2023-10-08T05:36:16.195436+0000","last_clean":"2023-10-08T05:36:16.195436+0000","last_became_active":"2023-10-08T05:36:11.945314+0000","last_became_peered":"2023-10-08T05:36:11.945314+0000","last_unstale":"2023-10-08T05:36:16.195436+0000","last_undegraded":"2023-10-08T05:36:16.195436+0000","last_fullsized":"2023-10-08T05:36:16.195436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.197371+0000","last_change":"2023-10-08T05:36:11.176854+0000","last_active":"2023-10-08T05:36:16.197371+0000","last_peered":"2023-10-08T05:36:16.197371+0000","last_clean":"2023-10-08T05:36:16.197371+0000","last_became_active":"2023-10-08T05:36:11.176546+0000","last_became_peered":"2023-10-08T05:36:11.176546+0000","last_unstale":"2023-10-08T05:36:16.197371+0000","last_undegraded":"2023-10-08T05:36:16.197371+0000","last_fullsized":"2023-10-08T05:36:16.197371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:16.815302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195412+0000","last_change":"2023-10-08T05:36:11.184441+0000","last_active":"2023-10-08T05:36:16.195412+0000","last_peered":"2023-10-08T05:36:16.195412+0000","last_clean":"2023-10-08T05:36:16.195412+0000","last_became_active":"2023-10-08T05:36:11.184159+0000","last_became_peered":"2023-10-08T05:36:11.184159+0000","last_unstale":"2023-10-08T05:36:16.195412+0000","last_undegraded":"2023-10-08T05:36:16.195412+0000","last_fullsized":"2023-10-08T05:36:16.195412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206365+0000","last_change":"2023-10-08T05:36:11.935252+0000","last_active":"2023-10-08T05:36:16.206365+0000","last_peered":"2023-10-08T05:36:16.206365+0000","last_clean":"2023-10-08T05:36:16.206365+0000","last_became_active":"2023-10-08T05:36:11.934390+0000","last_became_peered":"2023-10-08T05:36:11.934390+0000","last_unstale":"2023-10-08T05:36:16.206365+0000","last_undegraded":"2023-10-08T05:36:16.206365+0000","last_fullsized":"2023-10-08T05:36:16.206365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:17.242390+0000","last_change":"2023-10-08T05:36:16.217984+0000","last_active":"2023-10-08T05:36:17.242390+0000","last_peered":"2023-10-08T05:36:17.242390+0000","last_clean":"2023-10-08T05:36:17.242390+0000","last_became_active":"2023-10-08T05:36:16.217614+0000","last_became_peered":"2023-10-08T05:36:16.217614+0000","last_unstale":"2023-10-08T05:36:17.242390+0000","last_undegraded":"2023-10-08T05:36:17.242390+0000","last_fullsized":"2023-10-08T05:36:17.242390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:47.787180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194733+0000","last_change":"2023-10-08T05:36:11.182558+0000","last_active":"2023-10-08T05:36:16.194733+0000","last_peered":"2023-10-08T05:36:16.194733+0000","last_clean":"2023-10-08T05:36:16.194733+0000","last_became_active":"2023-10-08T05:36:11.181822+0000","last_became_peered":"2023-10-08T05:36:11.181822+0000","last_unstale":"2023-10-08T05:36:16.194733+0000","last_undegraded":"2023-10-08T05:36:16.194733+0000","last_fullsized":"2023-10-08T05:36:16.194733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:10.115707+0000","last_clean_scrub_stamp":"2023-10-08T05:36:10.115707+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:53.011319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214758+0000","last_change":"2023-10-08T05:36:11.941704+0000","last_active":"2023-10-08T05:36:16.214758+0000","last_peered":"2023-10-08T05:36:16.214758+0000","last_clean":"2023-10-08T05:36:16.214758+0000","last_became_active":"2023-10-08T05:36:11.941431+0000","last_became_peered":"2023-10-08T05:36:11.941431+0000","last_unstale":"2023-10-08T05:36:16.214758+0000","last_undegraded":"2023-10-08T05:36:16.214758+0000","last_fullsized":"2023-10-08T05:36:16.214758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:00.811496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.913836+0000","last_change":"2023-10-08T05:36:16.220904+0000","last_active":"2023-10-08T05:36:16.913836+0000","last_peered":"2023-10-08T05:36:16.913836+0000","last_clean":"2023-10-08T05:36:16.913836+0000","last_became_active":"2023-10-08T05:36:16.220521+0000","last_became_peered":"2023-10-08T05:36:16.220521+0000","last_unstale":"2023-10-08T05:36:16.913836+0000","last_undegraded":"2023-10-08T05:36:16.913836+0000","last_fullsized":"2023-10-08T05:36:16.913836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:15.164638+0000","last_clean_scrub_stamp":"2023-10-08T05:36:15.164638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:52:02.322999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194151+0000","last_change":"2023-10-08T05:36:11.945315+0000","last_active":"2023-10-08T05:36:16.194151+0000","last_peered":"2023-10-08T05:36:16.194151+0000","last_clean":"2023-10-08T05:36:16.194151+0000","last_became_active":"2023-10-08T05:36:11.944775+0000","last_became_peered":"2023-10-08T05:36:11.944775+0000","last_unstale":"2023-10-08T05:36:16.194151+0000","last_undegraded":"2023-10-08T05:36:16.194151+0000","last_fullsized":"2023-10-08T05:36:16.194151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.189968+0000","last_change":"2023-10-08T05:36:11.938156+0000","last_active":"2023-10-08T05:36:16.189968+0000","last_peered":"2023-10-08T05:36:16.189968+0000","last_clean":"2023-10-08T05:36:16.189968+0000","last_became_active":"2023-10-08T05:36:11.937878+0000","last_became_peered":"2023-10-08T05:36:11.937878+0000","last_unstale":"2023-10-08T05:36:16.189968+0000","last_undegraded":"2023-10-08T05:36:16.189968+0000","last_fullsized":"2023-10-08T05:36:16.189968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:00.811496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215846+0000","last_change":"2023-10-08T05:36:11.944484+0000","last_active":"2023-10-08T05:36:16.215846+0000","last_peered":"2023-10-08T05:36:16.215846+0000","last_clean":"2023-10-08T05:36:16.215846+0000","last_became_active":"2023-10-08T05:36:11.943696+0000","last_became_peered":"2023-10-08T05:36:11.943696+0000","last_unstale":"2023-10-08T05:36:16.215846+0000","last_undegraded":"2023-10-08T05:36:16.215846+0000","last_fullsized":"2023-10-08T05:36:16.215846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:12.498480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206284+0000","last_change":"2023-10-08T05:36:11.944505+0000","last_active":"2023-10-08T05:36:16.206284+0000","last_peered":"2023-10-08T05:36:16.206284+0000","last_clean":"2023-10-08T05:36:16.206284+0000","last_became_active":"2023-10-08T05:36:11.944084+0000","last_became_peered":"2023-10-08T05:36:11.944084+0000","last_unstale":"2023-10-08T05:36:16.206284+0000","last_undegraded":"2023-10-08T05:36:16.206284+0000","last_fullsized":"2023-10-08T05:36:16.206284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:46.860313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.197543+0000","last_change":"2023-10-08T05:36:11.936235+0000","last_active":"2023-10-08T05:36:16.197543+0000","last_peered":"2023-10-08T05:36:16.197543+0000","last_clean":"2023-10-08T05:36:16.197543+0000","last_became_active":"2023-10-08T05:36:11.935816+0000","last_became_peered":"2023-10-08T05:36:11.935816+0000","last_unstale":"2023-10-08T05:36:16.197543+0000","last_undegraded":"2023-10-08T05:36:16.197543+0000","last_fullsized":"2023-10-08T05:36:16.197543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206800+0000","last_change":"2023-10-08T05:36:11.940291+0000","last_active":"2023-10-08T05:36:16.206800+0000","last_peered":"2023-10-08T05:36:16.206800+0000","last_clean":"2023-10-08T05:36:16.206800+0000","last_became_active":"2023-10-08T05:36:11.940019+0000","last_became_peered":"2023-10-08T05:36:11.940019+0000","last_unstale":"2023-10-08T05:36:16.206800+0000","last_undegraded":"2023-10-08T05:36:16.206800+0000","last_fullsized":"2023-10-08T05:36:16.206800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195886+0000","last_change":"2023-10-08T05:36:11.949017+0000","last_active":"2023-10-08T05:36:16.195886+0000","last_peered":"2023-10-08T05:36:16.195886+0000","last_clean":"2023-10-08T05:36:16.195886+0000","last_became_active":"2023-10-08T05:36:11.948741+0000","last_became_peered":"2023-10-08T05:36:11.948741+0000","last_unstale":"2023-10-08T05:36:16.195886+0000","last_undegraded":"2023-10-08T05:36:16.195886+0000","last_fullsized":"2023-10-08T05:36:16.195886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:29.483682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.198027+0000","last_change":"2023-10-08T05:36:11.937078+0000","last_active":"2023-10-08T05:36:16.198027+0000","last_peered":"2023-10-08T05:36:16.198027+0000","last_clean":"2023-10-08T05:36:16.198027+0000","last_became_active":"2023-10-08T05:36:11.936737+0000","last_became_peered":"2023-10-08T05:36:11.936737+0000","last_unstale":"2023-10-08T05:36:16.198027+0000","last_undegraded":"2023-10-08T05:36:16.198027+0000","last_fullsized":"2023-10-08T05:36:16.198027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:11.159662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214525+0000","last_change":"2023-10-08T05:36:11.945582+0000","last_active":"2023-10-08T05:36:16.214525+0000","last_peered":"2023-10-08T05:36:16.214525+0000","last_clean":"2023-10-08T05:36:16.214525+0000","last_became_active":"2023-10-08T05:36:11.945066+0000","last_became_peered":"2023-10-08T05:36:11.945066+0000","last_unstale":"2023-10-08T05:36:16.214525+0000","last_undegraded":"2023-10-08T05:36:16.214525+0000","last_fullsized":"2023-10-08T05:36:16.214525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:25.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196604+0000","last_change":"2023-10-08T05:36:11.945936+0000","last_active":"2023-10-08T05:36:16.196604+0000","last_peered":"2023-10-08T05:36:16.196604+0000","last_clean":"2023-10-08T05:36:16.196604+0000","last_became_active":"2023-10-08T05:36:11.945384+0000","last_became_peered":"2023-10-08T05:36:11.945384+0000","last_unstale":"2023-10-08T05:36:16.196604+0000","last_undegraded":"2023-10-08T05:36:16.196604+0000","last_fullsized":"2023-10-08T05:36:16.196604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215763+0000","last_change":"2023-10-08T05:36:11.949155+0000","last_active":"2023-10-08T05:36:16.215763+0000","last_peered":"2023-10-08T05:36:16.215763+0000","last_clean":"2023-10-08T05:36:16.215763+0000","last_became_active":"2023-10-08T05:36:11.948867+0000","last_became_peered":"2023-10-08T05:36:11.948867+0000","last_unstale":"2023-10-08T05:36:16.215763+0000","last_undegraded":"2023-10-08T05:36:16.215763+0000","last_fullsized":"2023-10-08T05:36:16.215763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:11.627279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.206186+0000","last_change":"2023-10-08T05:36:11.949363+0000","last_active":"2023-10-08T05:36:16.206186+0000","last_peered":"2023-10-08T05:36:16.206186+0000","last_clean":"2023-10-08T05:36:16.206186+0000","last_became_active":"2023-10-08T05:36:11.949059+0000","last_became_peered":"2023-10-08T05:36:11.949059+0000","last_unstale":"2023-10-08T05:36:16.206186+0000","last_undegraded":"2023-10-08T05:36:16.206186+0000","last_fullsized":"2023-10-08T05:36:16.206186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:00.571257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195787+0000","last_change":"2023-10-08T05:36:11.936518+0000","last_active":"2023-10-08T05:36:16.195787+0000","last_peered":"2023-10-08T05:36:16.195787+0000","last_clean":"2023-10-08T05:36:16.195787+0000","last_became_active":"2023-10-08T05:36:11.936138+0000","last_became_peered":"2023-10-08T05:36:11.936138+0000","last_unstale":"2023-10-08T05:36:16.195787+0000","last_undegraded":"2023-10-08T05:36:16.195787+0000","last_fullsized":"2023-10-08T05:36:16.195787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239530+0000","last_change":"2023-10-08T05:36:11.940992+0000","last_active":"2023-10-08T05:36:16.239530+0000","last_peered":"2023-10-08T05:36:16.239530+0000","last_clean":"2023-10-08T05:36:16.239530+0000","last_became_active":"2023-10-08T05:36:11.940700+0000","last_became_peered":"2023-10-08T05:36:11.940700+0000","last_unstale":"2023-10-08T05:36:16.239530+0000","last_undegraded":"2023-10-08T05:36:16.239530+0000","last_fullsized":"2023-10-08T05:36:16.239530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:20.449484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.190822+0000","last_change":"2023-10-08T05:36:11.937136+0000","last_active":"2023-10-08T05:36:16.190822+0000","last_peered":"2023-10-08T05:36:16.190822+0000","last_clean":"2023-10-08T05:36:16.190822+0000","last_became_active":"2023-10-08T05:36:11.936761+0000","last_became_peered":"2023-10-08T05:36:11.936761+0000","last_unstale":"2023-10-08T05:36:16.190822+0000","last_undegraded":"2023-10-08T05:36:16.190822+0000","last_fullsized":"2023-10-08T05:36:16.190822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:34:50.179123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215403+0000","last_change":"2023-10-08T05:36:11.943099+0000","last_active":"2023-10-08T05:36:16.215403+0000","last_peered":"2023-10-08T05:36:16.215403+0000","last_clean":"2023-10-08T05:36:16.215403+0000","last_became_active":"2023-10-08T05:36:11.942792+0000","last_became_peered":"2023-10-08T05:36:11.942792+0000","last_unstale":"2023-10-08T05:36:16.215403+0000","last_undegraded":"2023-10-08T05:36:16.215403+0000","last_fullsized":"2023-10-08T05:36:16.215403+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214354+0000","last_change":"2023-10-08T05:36:11.938711+0000","last_active":"2023-10-08T05:36:16.214354+0000","last_peered":"2023-10-08T05:36:16.214354+0000","last_clean":"2023-10-08T05:36:16.214354+0000","last_became_active":"2023-10-08T05:36:11.938449+0000","last_became_peered":"2023-10-08T05:36:11.938449+0000","last_unstale":"2023-10-08T05:36:16.214354+0000","last_undegraded":"2023-10-08T05:36:16.214354+0000","last_fullsized":"2023-10-08T05:36:16.214354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:56.745041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.203081+0000","last_change":"2023-10-08T05:36:11.950960+0000","last_active":"2023-10-08T05:36:16.203081+0000","last_peered":"2023-10-08T05:36:16.203081+0000","last_clean":"2023-10-08T05:36:16.203081+0000","last_became_active":"2023-10-08T05:36:11.950582+0000","last_became_peered":"2023-10-08T05:36:11.950582+0000","last_unstale":"2023-10-08T05:36:16.203081+0000","last_undegraded":"2023-10-08T05:36:16.203081+0000","last_fullsized":"2023-10-08T05:36:16.203081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:42.636828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205140+0000","last_change":"2023-10-08T05:36:11.938997+0000","last_active":"2023-10-08T05:36:16.205140+0000","last_peered":"2023-10-08T05:36:16.205140+0000","last_clean":"2023-10-08T05:36:16.205140+0000","last_became_active":"2023-10-08T05:36:11.938641+0000","last_became_peered":"2023-10-08T05:36:11.938641+0000","last_unstale":"2023-10-08T05:36:16.205140+0000","last_undegraded":"2023-10-08T05:36:16.205140+0000","last_fullsized":"2023-10-08T05:36:16.205140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:54.023070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.194362+0000","last_change":"2023-10-08T05:36:11.938973+0000","last_active":"2023-10-08T05:36:16.194362+0000","last_peered":"2023-10-08T05:36:16.194362+0000","last_clean":"2023-10-08T05:36:16.194362+0000","last_became_active":"2023-10-08T05:36:11.938633+0000","last_became_peered":"2023-10-08T05:36:11.938633+0000","last_unstale":"2023-10-08T05:36:16.194362+0000","last_undegraded":"2023-10-08T05:36:16.194362+0000","last_fullsized":"2023-10-08T05:36:16.194362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.213118+0000","last_change":"2023-10-08T05:36:11.938748+0000","last_active":"2023-10-08T05:36:16.213118+0000","last_peered":"2023-10-08T05:36:16.213118+0000","last_clean":"2023-10-08T05:36:16.213118+0000","last_became_active":"2023-10-08T05:36:11.937801+0000","last_became_peered":"2023-10-08T05:36:11.937801+0000","last_unstale":"2023-10-08T05:36:16.213118+0000","last_undegraded":"2023-10-08T05:36:16.213118+0000","last_fullsized":"2023-10-08T05:36:16.213118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:59.279776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.196976+0000","last_change":"2023-10-08T05:36:11.937561+0000","last_active":"2023-10-08T05:36:16.196976+0000","last_peered":"2023-10-08T05:36:16.196976+0000","last_clean":"2023-10-08T05:36:16.196976+0000","last_became_active":"2023-10-08T05:36:11.937299+0000","last_became_peered":"2023-10-08T05:36:11.937299+0000","last_unstale":"2023-10-08T05:36:16.196976+0000","last_undegraded":"2023-10-08T05:36:16.196976+0000","last_fullsized":"2023-10-08T05:36:16.196976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195655+0000","last_change":"2023-10-08T05:36:11.938330+0000","last_active":"2023-10-08T05:36:16.195655+0000","last_peered":"2023-10-08T05:36:16.195655+0000","last_clean":"2023-10-08T05:36:16.195655+0000","last_became_active":"2023-10-08T05:36:11.937993+0000","last_became_peered":"2023-10-08T05:36:11.937993+0000","last_unstale":"2023-10-08T05:36:16.195655+0000","last_undegraded":"2023-10-08T05:36:16.195655+0000","last_fullsized":"2023-10-08T05:36:16.195655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:41.693456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240495+0000","last_change":"2023-10-08T05:36:11.954185+0000","last_active":"2023-10-08T05:36:16.240495+0000","last_peered":"2023-10-08T05:36:16.240495+0000","last_clean":"2023-10-08T05:36:16.240495+0000","last_became_active":"2023-10-08T05:36:11.953900+0000","last_became_peered":"2023-10-08T05:36:11.953900+0000","last_unstale":"2023-10-08T05:36:16.240495+0000","last_undegraded":"2023-10-08T05:36:16.240495+0000","last_fullsized":"2023-10-08T05:36:16.240495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:11.627279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.195359+0000","last_change":"2023-10-08T05:36:11.938140+0000","last_active":"2023-10-08T05:36:16.195359+0000","last_peered":"2023-10-08T05:36:16.195359+0000","last_clean":"2023-10-08T05:36:16.195359+0000","last_became_active":"2023-10-08T05:36:11.937817+0000","last_became_peered":"2023-10-08T05:36:11.937817+0000","last_unstale":"2023-10-08T05:36:16.195359+0000","last_undegraded":"2023-10-08T05:36:16.195359+0000","last_fullsized":"2023-10-08T05:36:16.195359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.238846+0000","last_change":"2023-10-08T05:36:11.948351+0000","last_active":"2023-10-08T05:36:16.238846+0000","last_peered":"2023-10-08T05:36:16.238846+0000","last_clean":"2023-10-08T05:36:16.238846+0000","last_became_active":"2023-10-08T05:36:11.948045+0000","last_became_peered":"2023-10-08T05:36:11.948045+0000","last_unstale":"2023-10-08T05:36:16.238846+0000","last_undegraded":"2023-10-08T05:36:16.238846+0000","last_fullsized":"2023-10-08T05:36:16.238846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:26.402954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.215673+0000","last_change":"2023-10-08T05:36:11.944663+0000","last_active":"2023-10-08T05:36:16.215673+0000","last_peered":"2023-10-08T05:36:16.215673+0000","last_clean":"2023-10-08T05:36:16.215673+0000","last_became_active":"2023-10-08T05:36:11.944158+0000","last_became_peered":"2023-10-08T05:36:11.944158+0000","last_unstale":"2023-10-08T05:36:16.215673+0000","last_undegraded":"2023-10-08T05:36:16.215673+0000","last_fullsized":"2023-10-08T05:36:16.215673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:16.603017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.239112+0000","last_change":"2023-10-08T05:36:11.944406+0000","last_active":"2023-10-08T05:36:16.239112+0000","last_peered":"2023-10-08T05:36:16.239112+0000","last_clean":"2023-10-08T05:36:16.239112+0000","last_became_active":"2023-10-08T05:36:11.944031+0000","last_became_peered":"2023-10-08T05:36:11.944031+0000","last_unstale":"2023-10-08T05:36:16.239112+0000","last_undegraded":"2023-10-08T05:36:16.239112+0000","last_fullsized":"2023-10-08T05:36:16.239112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:00.422999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.214934+0000","last_change":"2023-10-08T05:36:11.949094+0000","last_active":"2023-10-08T05:36:16.214934+0000","last_peered":"2023-10-08T05:36:16.214934+0000","last_clean":"2023-10-08T05:36:16.214934+0000","last_became_active":"2023-10-08T05:36:11.948663+0000","last_became_peered":"2023-10-08T05:36:11.948663+0000","last_unstale":"2023-10-08T05:36:16.214934+0000","last_undegraded":"2023-10-08T05:36:16.214934+0000","last_fullsized":"2023-10-08T05:36:16.214934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:46:57.883695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.205484+0000","last_change":"2023-10-08T05:36:11.927796+0000","last_active":"2023-10-08T05:36:16.205484+0000","last_peered":"2023-10-08T05:36:16.205484+0000","last_clean":"2023-10-08T05:36:16.205484+0000","last_became_active":"2023-10-08T05:36:11.927200+0000","last_became_peered":"2023-10-08T05:36:11.927200+0000","last_unstale":"2023-10-08T05:36:16.205484+0000","last_undegraded":"2023-10-08T05:36:16.205484+0000","last_fullsized":"2023-10-08T05:36:16.205484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:11.743725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.208328+0000","last_change":"2023-10-08T05:36:11.937916+0000","last_active":"2023-10-08T05:36:16.208328+0000","last_peered":"2023-10-08T05:36:16.208328+0000","last_clean":"2023-10-08T05:36:16.208328+0000","last_became_active":"2023-10-08T05:36:11.937646+0000","last_became_peered":"2023-10-08T05:36:11.937646+0000","last_unstale":"2023-10-08T05:36:16.208328+0000","last_undegraded":"2023-10-08T05:36:16.208328+0000","last_fullsized":"2023-10-08T05:36:16.208328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:23.373422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-10-08T05:36:16.240404+0000","last_change":"2023-10-08T05:36:11.952124+0000","last_active":"2023-10-08T05:36:16.240404+0000","last_peered":"2023-10-08T05:36:16.240404+0000","last_clean":"2023-10-08T05:36:16.240404+0000","last_became_active":"2023-10-08T05:36:11.949353+0000","last_became_peered":"2023-10-08T05:36:11.949353+0000","last_unstale":"2023-10-08T05:36:16.240404+0000","last_undegraded":"2023-10-08T05:36:16.240404+0000","last_fullsized":"2023-10-08T05:36:16.240404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:11.127458+0000","last_clean_scrub_stamp":"2023-10-08T05:36:11.127458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:46:57.883695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":695401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":104541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.331}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.444}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.236}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":104577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6990000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8859999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":278528,"data_stored":108292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":266240,"data_stored":104451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0369999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4950000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":274432,"data_stored":104611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":872448,"data_stored":695457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:36:41.537 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-08T05:36:41.537 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-08T05:36:41.537 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-08T05:36:41.538 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-08T05:36:41.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.776+0000 7f5b8c0f2640 1 Processor -- start 2023-10-08T05:36:41.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.776+0000 7f5b8c0f2640 1 -- start start 2023-10-08T05:36:41.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.778+0000 7f5b8c0f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b84151aa0 0x7f5b840c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.778+0000 7f5b8c0f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5b8415b730 0x7f5b8415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.778+0000 7f5b8c0f2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b840b1320 con 0x7f5b840c93c0 2023-10-08T05:36:41.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.778+0000 7f5b8c0f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b840b0a80 con 0x7f5b8415b730 2023-10-08T05:36:41.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.778+0000 7f5b8c0f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840b0d60 con 0x7f5b84151aa0 2023-10-08T05:36:41.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.779+0000 7f5b89e8e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b84151aa0 0x7f5b840c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.779+0000 7f5b8a68f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5b8415b730 0x7f5b8415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.779+0000 7f5b8a68f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5b8415b730 0x7f5b8415bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:40030/0 (socket says 172.21.15.70:40030) 2023-10-08T05:36:41.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.779+0000 7f5b89e8e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b84151aa0 0x7f5b840c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:60942/0 (socket says 172.21.15.70:60942) 2023-10-08T05:36:41.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.779+0000 7f5b8a68f640 1 -- 172.21.15.70:0/3995601161 learned_addr learned my addr 172.21.15.70:0/3995601161 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:41.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.780+0000 7f5b8a68f640 1 -- 172.21.15.70:0/3995601161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b84151aa0 msgr2=0x7f5b840c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:41.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.780+0000 7f5b8a68f640 1 --2- 172.21.15.70:0/3995601161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b84151aa0 0x7f5b840c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.780+0000 7f5b8a68f640 1 -- 172.21.15.70:0/3995601161 >> v1:172.21.15.181:6789/0 conn(0x7f5b840c93c0 legacy=0x7f5b8415e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.780+0000 7f5b8a68f640 1 -- 172.21.15.70:0/3995601161 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b840b1040 con 0x7f5b8415b730 2023-10-08T05:36:41.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.780+0000 7f5b89e8e640 1 --2- 172.21.15.70:0/3995601161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b84151aa0 0x7f5b840c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:36:41.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.781+0000 7f5b8a68f640 1 --2- 172.21.15.70:0/3995601161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b8415b730 0x7f5b8415bb80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009e50 tx=0x7f5b7c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c40663a76eda9cd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:41.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.781+0000 7f5b88e8c640 1 -- 172.21.15.70:0/3995601161 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c02cb00 con 0x7f5b8415b730 2023-10-08T05:36:41.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.781+0000 7f5b88e8c640 1 -- 172.21.15.70:0/3995601161 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7c02cc60 con 0x7f5b8415b730 2023-10-08T05:36:41.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.781+0000 7f5b88e8c640 1 -- 172.21.15.70:0/3995601161 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c0343f0 con 0x7f5b8415b730 2023-10-08T05:36:41.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.782+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/3995601161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b8415b730 msgr2=0x7f5b8415bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.782+0000 7f5b8c0f2640 1 --2- 172.21.15.70:0/3995601161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b8415b730 0x7f5b8415bb80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009e50 tx=0x7f5b7c02bf60 comp rx=0 tx=0).stop 2023-10-08T05:36:41.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.782+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/3995601161 shutdown_connections 2023-10-08T05:36:41.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.782+0000 7f5b8c0f2640 1 --2- 172.21.15.70:0/3995601161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b8415b730 0x7f5b8415bb80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.782+0000 7f5b8c0f2640 1 --2- 172.21.15.70:0/3995601161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b84151aa0 0x7f5b840c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.782+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/3995601161 >> 172.21.15.70:0/3995601161 conn(0x7f5b840b20b0 msgr2=0x7f5b84069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:41.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.782+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/3995601161 shutdown_connections 2023-10-08T05:36:41.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.782+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/3995601161 wait complete. 2023-10-08T05:36:41.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.784+0000 7f5b8c0f2640 1 Processor -- start 2023-10-08T05:36:41.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.784+0000 7f5b8c0f2640 1 -- start start 2023-10-08T05:36:41.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.785+0000 7f5b8c0f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b840c93c0 0x7f5b8413d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.785+0000 7f5b8c0f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b84151aa0 0x7f5b84143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.785+0000 7f5b8c0f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8415b730 0x7f5b8414fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.785+0000 7f5b8c0f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840c5a60 con 0x7f5b8415b730 2023-10-08T05:36:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.785+0000 7f5b8c0f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b840cbc50 con 0x7f5b84151aa0 2023-10-08T05:36:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.785+0000 7f5b8c0f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b84140f30 con 0x7f5b840c93c0 2023-10-08T05:36:41.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.785+0000 7f5b8a68f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8415b730 0x7f5b8414fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.786+0000 7f5b8a68f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8415b730 0x7f5b8414fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60274/0 (socket says 172.21.15.70:60274) 2023-10-08T05:36:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.786+0000 7f5b8a68f640 1 -- 172.21.15.70:0/2906485695 learned_addr learned my addr 172.21.15.70:0/2906485695 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.786+0000 7f5b89e8e640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b840c93c0 0x7f5b8413d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.786+0000 7f5b8968d640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b84151aa0 0x7f5b84143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.786+0000 7f5b8a68f640 1 -- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b840c93c0 msgr2=0x7f5b8413d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.786+0000 7f5b8a68f640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b840c93c0 0x7f5b8413d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.786+0000 7f5b8a68f640 1 -- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b84151aa0 msgr2=0x7f5b84143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:41.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.786+0000 7f5b8a68f640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b84151aa0 0x7f5b84143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:41.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.786+0000 7f5b8a68f640 1 -- 172.21.15.70:0/2906485695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b80009400 con 0x7f5b8415b730 2023-10-08T05:36:41.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.786+0000 7f5b8a68f640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8415b730 0x7f5b8414fc80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c036040 tx=0x7f5b7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:41.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.787+0000 7f5b7a7fc640 1 -- 172.21.15.70:0/2906485695 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c009510 con 0x7f5b8415b730 2023-10-08T05:36:41.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.787+0000 7f5b7a7fc640 1 -- 172.21.15.70:0/2906485695 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7c03e450 con 0x7f5b8415b730 2023-10-08T05:36:41.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.787+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/2906485695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c009b30 con 0x7f5b8415b730 2023-10-08T05:36:41.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.787+0000 7f5b7a7fc640 1 -- 172.21.15.70:0/2906485695 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c03e800 con 0x7f5b8415b730 2023-10-08T05:36:41.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.787+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/2906485695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b84140690 con 0x7f5b8415b730 2023-10-08T05:36:41.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.788+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/2906485695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b840c5bd0 con 0x7f5b8415b730 2023-10-08T05:36:41.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.790+0000 7f5b7a7fc640 1 -- 172.21.15.70:0/2906485695 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5b7c03e960 con 0x7f5b8415b730 2023-10-08T05:36:41.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.791+0000 7f5b7a7fc640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5b6806ae70 0x7f5b6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:41.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.791+0000 7f5b7a7fc640 1 -- 172.21.15.70:0/2906485695 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5b7c0ac610 con 0x7f5b8415b730 2023-10-08T05:36:41.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.792+0000 7f5b89e8e640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5b6806ae70 0x7f5b6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:41.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.793+0000 7f5b89e8e640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5b6806ae70 0x7f5b6806d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009c60 tx=0x7f5b80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:41.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:41.795+0000 7f5b7a7fc640 1 -- 172.21.15.70:0/2906485695 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5b7c0773d0 con 0x7f5b8415b730 2023-10-08T05:36:42.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.023+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/2906485695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5b84069d50 con 0x7f5b8415b730 2023-10-08T05:36:42.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.023+0000 7f5b7a7fc640 1 -- 172.21.15.70:0/2906485695 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5b7c07a9f0 con 0x7f5b8415b730 2023-10-08T05:36:42.025 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:42.025 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-08T05:36:42.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5b6806ae70 msgr2=0x7f5b6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5b6806ae70 0x7f5b6806d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5b80009c60 tx=0x7f5b80008040 comp rx=0 tx=0).stop 2023-10-08T05:36:42.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/2906485695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8415b730 msgr2=0x7f5b8414fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8415b730 0x7f5b8414fc80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c036040 tx=0x7f5b7c004a40 comp rx=0 tx=0).stop 2023-10-08T05:36:42.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/2906485695 shutdown_connections 2023-10-08T05:36:42.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5b6806ae70 0x7f5b6806d330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8415b730 0x7f5b8414fc80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b84151aa0 0x7f5b84143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 --2- 172.21.15.70:0/2906485695 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b840c93c0 0x7f5b8413d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/2906485695 >> 172.21.15.70:0/2906485695 conn(0x7f5b840b20b0 msgr2=0x7f5b840b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:42.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/2906485695 shutdown_connections 2023-10-08T05:36:42.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.026+0000 7f5b8c0f2640 1 -- 172.21.15.70:0/2906485695 wait complete. 2023-10-08T05:36:42.045 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-08T05:36:42.046 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:42.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.236+0000 7f9d656af640 1 Processor -- start 2023-10-08T05:36:42.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.236+0000 7f9d656af640 1 -- start start 2023-10-08T05:36:42.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.238+0000 7f9d656af640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d6006a460 0x7f9d60071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.238+0000 7f9d656af640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d600724a0 con 0x7f9d6015a990 2023-10-08T05:36:42.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.238+0000 7f9d656af640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d60071ad0 con 0x7f9d6015ad70 2023-10-08T05:36:42.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.238+0000 7f9d656af640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d600b5500 con 0x7f9d6006a460 2023-10-08T05:36:42.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.238+0000 7f9d5effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9d6015a990 0x7f9d60069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43766/0 (socket says 172.21.15.70:43766) 2023-10-08T05:36:42.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.238+0000 7f9d5effd640 1 -- 172.21.15.70:0/1226506952 learned_addr learned my addr 172.21.15.70:0/1226506952 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5f7fe640 1 --2- 172.21.15.70:0/1226506952 >> v2:172.21.15.181:3300/0 conn(0x7f9d6006a460 0x7f9d60071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104097618 0 0) 0x7f9d50001110 con 0x7f9d6015a990 2023-10-08T05:36:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d4c003520 con 0x7f9d6015a990 2023-10-08T05:36:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3186360494 0 0) 0x7f9d4c003520 con 0x7f9d6015a990 2023-10-08T05:36:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 >> v2:172.21.15.181:3300/0 conn(0x7f9d6006a460 msgr2=0x7f9d60071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 --2- 172.21.15.70:0/1226506952 >> v2:172.21.15.181:3300/0 conn(0x7f9d6006a460 0x7f9d60071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 >> v1:172.21.15.181:6789/0 conn(0x7f9d6015ad70 legacy=0x7f9d600b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d60071480 con 0x7f9d6015a990 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9d50001800 con 0x7f9d6015a990 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 >> v1:172.21.15.70:6789/0 conn(0x7f9d6015a990 legacy=0x7f9d60069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c0039c0 0x7f9d4c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d4c0047c0 0x7f9d4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d4c005180 0x7f9d4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600b5500 con 0x7f9d4c0039c0 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c00c080 con 0x7f9d4c005180 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.239+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4c00c240 con 0x7f9d4c0047c0 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.240+0000 7f9d5effd640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c0039c0 0x7f9d4c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.240+0000 7f9d5effd640 1 -- 172.21.15.70:0/1226506952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d4c0047c0 msgr2=0x7f9d4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:42.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.240+0000 7f9d5effd640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d4c0047c0 0x7f9d4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.240+0000 7f9d5f7fe640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d4c005180 0x7f9d4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:42.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.240+0000 7f9d5effd640 1 -- 172.21.15.70:0/1226506952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d4c005180 msgr2=0x7f9d4c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:42.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.240+0000 7f9d5effd640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d4c005180 0x7f9d4c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.240+0000 7f9d5effd640 1 -- 172.21.15.70:0/1226506952 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c003520 con 0x7f9d4c0039c0 2023-10-08T05:36:42.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.240+0000 7f9d5f7fe640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d4c005180 0x7f9d4c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:42.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.240+0000 7f9d5f7fe640 1 -- 172.21.15.70:0/1226506952 reap_dead start 2023-10-08T05:36:42.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.241+0000 7f9d5effd640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c0039c0 0x7f9d4c003de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9d50007fc0 tx=0x7f9d4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3bf123e9bb8e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:42.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.241+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50005ad0 con 0x7f9d4c0039c0 2023-10-08T05:36:42.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.241+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d50005c50 con 0x7f9d4c0039c0 2023-10-08T05:36:42.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.241+0000 7f9d5e7fc640 1 -- 172.21.15.70:0/1226506952 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5002c7e0 con 0x7f9d4c0039c0 2023-10-08T05:36:42.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.242+0000 7f9d656af640 1 -- 172.21.15.70:0/1226506952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c0039c0 msgr2=0x7f9d4c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.242+0000 7f9d656af640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c0039c0 0x7f9d4c003de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9d50007fc0 tx=0x7f9d4c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.242+0000 7f9d656af640 1 -- 172.21.15.70:0/1226506952 shutdown_connections 2023-10-08T05:36:42.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.242+0000 7f9d656af640 1 --2- 172.21.15.70:0/1226506952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d4c0039c0 0x7f9d4c003de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.242+0000 7f9d656af640 1 -- 172.21.15.70:0/1226506952 >> 172.21.15.70:0/1226506952 conn(0x7f9d6006d450 msgr2=0x7f9d6006d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:42.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.242+0000 7f9d656af640 1 -- 172.21.15.70:0/1226506952 shutdown_connections 2023-10-08T05:36:42.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.242+0000 7f9d656af640 1 -- 172.21.15.70:0/1226506952 wait complete. 2023-10-08T05:36:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.244+0000 7f9d656af640 1 Processor -- start 2023-10-08T05:36:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.244+0000 7f9d656af640 1 -- start start 2023-10-08T05:36:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.245+0000 7f9d656af640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d4c0039c0 0x7f9d60150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.245+0000 7f9d656af640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d601f1ef0 0x7f9d600b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.245+0000 7f9d656af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d600b0d20 0x7f9d601f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.245+0000 7f9d656af640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600b11a0 con 0x7f9d600b0d20 2023-10-08T05:36:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.245+0000 7f9d656af640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600aca50 con 0x7f9d4c0039c0 2023-10-08T05:36:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.245+0000 7f9d656af640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600acd30 con 0x7f9d601f1ef0 2023-10-08T05:36:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.245+0000 7f9d5f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d600b0d20 0x7f9d601f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:42.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.245+0000 7f9d5f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d600b0d20 0x7f9d601f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60296/0 (socket says 172.21.15.70:60296) 2023-10-08T05:36:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.245+0000 7f9d5f7fe640 1 -- 172.21.15.70:0/1755651568 learned_addr learned my addr 172.21.15.70:0/1755651568 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d5effd640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d4c0039c0 0x7f9d60150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d57fff640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d601f1ef0 0x7f9d600b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d5f7fe640 1 -- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d601f1ef0 msgr2=0x7f9d600b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d5f7fe640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d601f1ef0 0x7f9d600b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d5f7fe640 1 -- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d4c0039c0 msgr2=0x7f9d60150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d5f7fe640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d4c0039c0 0x7f9d60150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d5f7fe640 1 -- 172.21.15.70:0/1755651568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d600ad010 con 0x7f9d600b0d20 2023-10-08T05:36:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d57fff640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d601f1ef0 0x7f9d600b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:42.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d5f7fe640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d600b0d20 0x7f9d601f5950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9d600cb520 tx=0x7f9d5800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d577fe640 1 -- 172.21.15.70:0/1755651568 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58016420 con 0x7f9d600b0d20 2023-10-08T05:36:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d577fe640 1 -- 172.21.15.70:0/1755651568 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d58016580 con 0x7f9d600b0d20 2023-10-08T05:36:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d577fe640 1 -- 172.21.15.70:0/1755651568 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d580168f0 con 0x7f9d600b0d20 2023-10-08T05:36:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.246+0000 7f9d656af640 1 -- 172.21.15.70:0/1755651568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d600ad2f0 con 0x7f9d600b0d20 2023-10-08T05:36:42.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.247+0000 7f9d656af640 1 -- 172.21.15.70:0/1755651568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d600ad8b0 con 0x7f9d600b0d20 2023-10-08T05:36:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.248+0000 7f9d656af640 1 -- 172.21.15.70:0/1755651568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d600ada70 con 0x7f9d600b0d20 2023-10-08T05:36:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.249+0000 7f9d577fe640 1 -- 172.21.15.70:0/1755651568 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9d58016a50 con 0x7f9d600b0d20 2023-10-08T05:36:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.250+0000 7f9d577fe640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9d4006aea0 0x7f9d4006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.250+0000 7f9d577fe640 1 -- 172.21.15.70:0/1755651568 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9d5808a700 con 0x7f9d600b0d20 2023-10-08T05:36:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.250+0000 7f9d5effd640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9d4006aea0 0x7f9d4006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:42.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.251+0000 7f9d5effd640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9d4006aea0 0x7f9d4006d360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d50034000 tx=0x7f9d50009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:42.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.254+0000 7f9d577fe640 1 -- 172.21.15.70:0/1755651568 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d580554c0 con 0x7f9d600b0d20 2023-10-08T05:36:42.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.437+0000 7f9d656af640 1 -- 172.21.15.70:0/1755651568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d60067490 con 0x7f9d600b0d20 2023-10-08T05:36:42.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.440+0000 7f9d577fe640 1 -- 172.21.15.70:0/1755651568 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f9d58058ae0 con 0x7f9d600b0d20 2023-10-08T05:36:42.442 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:42.442 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:42.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.442+0000 7f9d656af640 1 -- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9d4006aea0 msgr2=0x7f9d4006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.442+0000 7f9d656af640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9d4006aea0 0x7f9d4006d360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d50034000 tx=0x7f9d50009030 comp rx=0 tx=0).stop 2023-10-08T05:36:42.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.442+0000 7f9d656af640 1 -- 172.21.15.70:0/1755651568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d600b0d20 msgr2=0x7f9d601f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.442+0000 7f9d656af640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d600b0d20 0x7f9d601f5950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9d600cb520 tx=0x7f9d5800f520 comp rx=0 tx=0).stop 2023-10-08T05:36:42.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.443+0000 7f9d656af640 1 -- 172.21.15.70:0/1755651568 shutdown_connections 2023-10-08T05:36:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.443+0000 7f9d656af640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9d4006aea0 0x7f9d4006d360 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.443+0000 7f9d656af640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d600b0d20 0x7f9d601f5950 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.443+0000 7f9d656af640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d601f1ef0 0x7f9d600b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.443+0000 7f9d656af640 1 --2- 172.21.15.70:0/1755651568 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d4c0039c0 0x7f9d60150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.443+0000 7f9d656af640 1 -- 172.21.15.70:0/1755651568 >> 172.21.15.70:0/1755651568 conn(0x7f9d6006d450 msgr2=0x7f9d600c2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.443+0000 7f9d656af640 1 -- 172.21.15.70:0/1755651568 shutdown_connections 2023-10-08T05:36:42.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.443+0000 7f9d656af640 1 -- 172.21.15.70:0/1755651568 wait complete. 2023-10-08T05:36:42.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:36:42.461 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:36:42.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.662+0000 7f1809221640 1 Processor -- start 2023-10-08T05:36:42.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.662+0000 7f1809221640 1 -- start start 2023-10-08T05:36:42.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f1809221640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18041432f0 0x7f18040cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f1809221640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18040cbb30 con 0x7f18040cb6b0 2023-10-08T05:36:42.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f1809221640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180415d440 con 0x7f18040b4240 2023-10-08T05:36:42.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f1809221640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18041436d0 con 0x7f18041432f0 2023-10-08T05:36:42.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f1802d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18041432f0 0x7f18040cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:42.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f17fbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f18040cb6b0 0x7f18040b3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43770/0 (socket says 172.21.15.70:43770) 2023-10-08T05:36:42.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f1802d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18041432f0 0x7f18040cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60310/0 (socket says 172.21.15.70:60310) 2023-10-08T05:36:42.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f17fbfff640 1 -- 172.21.15.70:0/2500785434 learned_addr learned my addr 172.21.15.70:0/2500785434 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:42.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f1802d9d640 1 -- 172.21.15.70:0/2500785434 >> v1:172.21.15.181:6789/0 conn(0x7f18040b4240 legacy=0x7f1804159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:42.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f1802d9d640 1 -- 172.21.15.70:0/2500785434 >> v1:172.21.15.70:6789/0 conn(0x7f18040cb6b0 legacy=0x7f18040b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.665+0000 7f1802d9d640 1 -- 172.21.15.70:0/2500785434 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180415d690 con 0x7f18041432f0 2023-10-08T05:36:42.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.664+0000 7f180259c640 1 -- 172.21.15.70:0/2500785434 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702951868 0 0) 0x7f18040cbb30 con 0x7f18040cb6b0 2023-10-08T05:36:42.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.665+0000 7f1802d9d640 1 --2- 172.21.15.70:0/2500785434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18041432f0 0x7f18040cb170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f17f40097a0 tx=0x7f17f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7640ab1c4642cd48 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:42.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.666+0000 7f180259c640 1 -- 172.21.15.70:0/2500785434 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f402e070 con 0x7f18041432f0 2023-10-08T05:36:42.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.666+0000 7f180259c640 1 -- 172.21.15.70:0/2500785434 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f40315a0 con 0x7f18041432f0 2023-10-08T05:36:42.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.666+0000 7f180259c640 1 -- 172.21.15.70:0/2500785434 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4031930 con 0x7f18041432f0 2023-10-08T05:36:42.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.666+0000 7f1809221640 1 -- 172.21.15.70:0/2500785434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18041432f0 msgr2=0x7f18040cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.666+0000 7f1809221640 1 --2- 172.21.15.70:0/2500785434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18041432f0 0x7f18040cb170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f17f40097a0 tx=0x7f17f402b840 comp rx=0 tx=0).stop 2023-10-08T05:36:42.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.666+0000 7f1809221640 1 -- 172.21.15.70:0/2500785434 shutdown_connections 2023-10-08T05:36:42.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.666+0000 7f1809221640 1 --2- 172.21.15.70:0/2500785434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18041432f0 0x7f18040cb170 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.666+0000 7f1809221640 1 -- 172.21.15.70:0/2500785434 >> 172.21.15.70:0/2500785434 conn(0x7f1804069d50 msgr2=0x7f180406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:42.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.666+0000 7f1809221640 1 -- 172.21.15.70:0/2500785434 shutdown_connections 2023-10-08T05:36:42.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.667+0000 7f1809221640 1 -- 172.21.15.70:0/2500785434 wait complete. 2023-10-08T05:36:42.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.668+0000 7f1809221640 1 Processor -- start 2023-10-08T05:36:42.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.668+0000 7f1809221640 1 -- start start 2023-10-08T05:36:42.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.669+0000 7f1809221640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18040b4240 0x7f18041f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.669+0000 7f1809221640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18040cb6b0 0x7f180415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.669+0000 7f1809221640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18041432f0 0x7f1804140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.669+0000 7f1809221640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180413fce0 con 0x7f18040b4240 2023-10-08T05:36:42.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.669+0000 7f1809221640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18041f2220 con 0x7f18041432f0 2023-10-08T05:36:42.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.669+0000 7f1809221640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180413e3a0 con 0x7f18040cb6b0 2023-10-08T05:36:42.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.669+0000 7f1802d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18040b4240 0x7f18041f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:42.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.669+0000 7f1802d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18040b4240 0x7f18041f1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60326/0 (socket says 172.21.15.70:60326) 2023-10-08T05:36:42.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.669+0000 7f1802d9d640 1 -- 172.21.15.70:0/2859159813 learned_addr learned my addr 172.21.15.70:0/2859159813 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.670+0000 7f1802d9d640 1 -- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18040cb6b0 msgr2=0x7f180415aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.670+0000 7f1802d9d640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18040cb6b0 0x7f180415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.670+0000 7f1802d9d640 1 -- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18041432f0 msgr2=0x7f1804140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.670+0000 7f1802d9d640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18041432f0 0x7f1804140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.670+0000 7f1802d9d640 1 -- 172.21.15.70:0/2859159813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f4009400 con 0x7f18040b4240 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.670+0000 7f1802d9d640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18040b4240 0x7f18041f1ce0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f17f4034040 tx=0x7f17f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.670+0000 7f17fb7fe640 1 -- 172.21.15.70:0/2859159813 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4032900 con 0x7f18040b4240 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.670+0000 7f17fb7fe640 1 -- 172.21.15.70:0/2859159813 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f4032a80 con 0x7f18040b4240 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.670+0000 7f17fb7fe640 1 -- 172.21.15.70:0/2859159813 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4032e30 con 0x7f18040b4240 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.670+0000 7f1809221640 1 -- 172.21.15.70:0/2859159813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180413b350 con 0x7f18040b4240 2023-10-08T05:36:42.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.671+0000 7f1809221640 1 -- 172.21.15.70:0/2859159813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180413b910 con 0x7f18040b4240 2023-10-08T05:36:42.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.672+0000 7f1809221640 1 -- 172.21.15.70:0/2859159813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180404e160 con 0x7f18040b4240 2023-10-08T05:36:42.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.672+0000 7f17fb7fe640 1 -- 172.21.15.70:0/2859159813 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f17f403b3f0 con 0x7f18040b4240 2023-10-08T05:36:42.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.673+0000 7f17fb7fe640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f17e406ae70 0x7f17e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:42.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.673+0000 7f17fb7fe640 1 -- 172.21.15.70:0/2859159813 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f17f40ab930 con 0x7f18040b4240 2023-10-08T05:36:42.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.674+0000 7f17fbfff640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f17e406ae70 0x7f17e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:42.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.675+0000 7f17fbfff640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f17e406ae70 0x7f17e406d330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f17ec007fd0 tx=0x7f17ec007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:42.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.679+0000 7f17fb7fe640 1 -- 172.21.15.70:0/2859159813 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17f4076780 con 0x7f18040b4240 2023-10-08T05:36:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.859+0000 7f1809221640 1 -- 172.21.15.70:0/2859159813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f180406d660 con 0x7f18040b4240 2023-10-08T05:36:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.860+0000 7f17fb7fe640 1 -- 172.21.15.70:0/2859159813 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17f4079da0 con 0x7f18040b4240 2023-10-08T05:36:42.862 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:36:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.862+0000 7f1809221640 1 -- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f17e406ae70 msgr2=0x7f17e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.862+0000 7f1809221640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f17e406ae70 0x7f17e406d330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f17ec007fd0 tx=0x7f17ec007920 comp rx=0 tx=0).stop 2023-10-08T05:36:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.862+0000 7f1809221640 1 -- 172.21.15.70:0/2859159813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18040b4240 msgr2=0x7f18041f1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.862+0000 7f1809221640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18040b4240 0x7f18041f1ce0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f17f4034040 tx=0x7f17f40046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.863+0000 7f1809221640 1 -- 172.21.15.70:0/2859159813 shutdown_connections 2023-10-08T05:36:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.863+0000 7f1809221640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f17e406ae70 0x7f17e406d330 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.863+0000 7f1809221640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f18041432f0 0x7f1804140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.863+0000 7f1809221640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f18040cb6b0 0x7f180415aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.863+0000 7f1809221640 1 --2- 172.21.15.70:0/2859159813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18040b4240 0x7f18041f1ce0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.863+0000 7f1809221640 1 -- 172.21.15.70:0/2859159813 >> 172.21.15.70:0/2859159813 conn(0x7f1804069d50 msgr2=0x7f1804158130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.863+0000 7f1809221640 1 -- 172.21.15.70:0/2859159813 shutdown_connections 2023-10-08T05:36:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:42.863+0000 7f1809221640 1 -- 172.21.15.70:0/2859159813 wait complete. 2023-10-08T05:36:42.881 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:36:43.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.075+0000 7f7448497640 1 Processor -- start 2023-10-08T05:36:43.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.075+0000 7f7448497640 1 -- start start 2023-10-08T05:36:43.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.076+0000 7f7448497640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f744015cba0 0x7f74400b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.076+0000 7f7448497640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f74400b4d10 0x7f74400d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.076+0000 7f7448497640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74400b54a0 con 0x7f744015cf80 2023-10-08T05:36:43.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.076+0000 7f7448497640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74400b7130 con 0x7f74400b4d10 2023-10-08T05:36:43.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.076+0000 7f7448497640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74400b7410 con 0x7f744015cba0 2023-10-08T05:36:43.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.076+0000 7f7446a34640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f74400b4d10 0x7f74400d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.076+0000 7f7446a34640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f74400b4d10 0x7f74400d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60332/0 (socket says 172.21.15.70:60332) 2023-10-08T05:36:43.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.077+0000 7f7446a34640 1 -- 172.21.15.70:0/966523861 learned_addr learned my addr 172.21.15.70:0/966523861 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:43.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.077+0000 7f7446233640 1 --2- 172.21.15.70:0/966523861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f744015cba0 0x7f74400b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.077+0000 7f7446a34640 1 -- 172.21.15.70:0/966523861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f744015cba0 msgr2=0x7f74400b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:43.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.077+0000 7f7446a34640 1 --2- 172.21.15.70:0/966523861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f744015cba0 0x7f74400b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.077+0000 7f7446a34640 1 -- 172.21.15.70:0/966523861 >> v1:172.21.15.181:6789/0 conn(0x7f744015cf80 legacy=0x7f74400b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.077+0000 7f7446a34640 1 -- 172.21.15.70:0/966523861 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74400b76f0 con 0x7f74400b4d10 2023-10-08T05:36:43.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.077+0000 7f7446233640 1 --2- 172.21.15.70:0/966523861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f744015cba0 0x7f74400b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:36:43.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.078+0000 7f7446a34640 1 --2- 172.21.15.70:0/966523861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74400b4d10 0x7f74400d0ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7438009e50 tx=0x7f743802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8ec3e9398694f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:43.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.078+0000 7f7445231640 1 -- 172.21.15.70:0/966523861 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743802cb00 con 0x7f74400b4d10 2023-10-08T05:36:43.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.078+0000 7f7445231640 1 -- 172.21.15.70:0/966523861 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743802cc60 con 0x7f74400b4d10 2023-10-08T05:36:43.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.078+0000 7f7445231640 1 -- 172.21.15.70:0/966523861 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74380343f0 con 0x7f74400b4d10 2023-10-08T05:36:43.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.079+0000 7f7448497640 1 -- 172.21.15.70:0/966523861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74400b4d10 msgr2=0x7f74400d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.079+0000 7f7448497640 1 --2- 172.21.15.70:0/966523861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74400b4d10 0x7f74400d0ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7438009e50 tx=0x7f743802bf60 comp rx=0 tx=0).stop 2023-10-08T05:36:43.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.079+0000 7f7448497640 1 -- 172.21.15.70:0/966523861 shutdown_connections 2023-10-08T05:36:43.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.079+0000 7f7448497640 1 --2- 172.21.15.70:0/966523861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74400b4d10 0x7f74400d0ee0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.079+0000 7f7448497640 1 --2- 172.21.15.70:0/966523861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f744015cba0 0x7f74400b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.079+0000 7f7448497640 1 -- 172.21.15.70:0/966523861 >> 172.21.15.70:0/966523861 conn(0x7f74400718a0 msgr2=0x7f7440071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:43.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.079+0000 7f7448497640 1 -- 172.21.15.70:0/966523861 shutdown_connections 2023-10-08T05:36:43.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.079+0000 7f7448497640 1 -- 172.21.15.70:0/966523861 wait complete. 2023-10-08T05:36:43.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.081+0000 7f7448497640 1 Processor -- start 2023-10-08T05:36:43.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.081+0000 7f7448497640 1 -- start start 2023-10-08T05:36:43.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7448497640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74400b4d10 0x7f7440157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7448497640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f744015cba0 0x7f74401e2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7448497640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f744015cf80 0x7f74401e2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7448497640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74400c70c0 con 0x7f744015cf80 2023-10-08T05:36:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7448497640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74401579c0 con 0x7f744015cba0 2023-10-08T05:36:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7448497640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74400c7310 con 0x7f74400b4d10 2023-10-08T05:36:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7446a34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f744015cf80 0x7f74401e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7446a34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f744015cf80 0x7f74401e2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60334/0 (socket says 172.21.15.70:60334) 2023-10-08T05:36:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7446a34640 1 -- 172.21.15.70:0/189910509 learned_addr learned my addr 172.21.15.70:0/189910509 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7446a34640 1 -- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74400b4d10 msgr2=0x7f7440157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7445a32640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f744015cba0 0x7f74401e2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7446a34640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74400b4d10 0x7f7440157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7446a34640 1 -- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f744015cba0 msgr2=0x7f74401e2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7446a34640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f744015cba0 0x7f74401e2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.082+0000 7f7446a34640 1 -- 172.21.15.70:0/189910509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438009b30 con 0x7f744015cf80 2023-10-08T05:36:43.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.083+0000 7f7446a34640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f744015cf80 0x7f74401e2690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7438009e50 tx=0x7f7438004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:43.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.083+0000 7f7436ffd640 1 -- 172.21.15.70:0/189910509 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438009510 con 0x7f744015cf80 2023-10-08T05:36:43.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.083+0000 7f7436ffd640 1 -- 172.21.15.70:0/189910509 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743803e450 con 0x7f744015cf80 2023-10-08T05:36:43.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.083+0000 7f7436ffd640 1 -- 172.21.15.70:0/189910509 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743803e800 con 0x7f744015cf80 2023-10-08T05:36:43.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.083+0000 7f7448497640 1 -- 172.21.15.70:0/189910509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74400c75f0 con 0x7f744015cf80 2023-10-08T05:36:43.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.083+0000 7f7448497640 1 -- 172.21.15.70:0/189910509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74400c7bb0 con 0x7f744015cf80 2023-10-08T05:36:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.084+0000 7f7448497640 1 -- 172.21.15.70:0/189910509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744004e160 con 0x7f744015cf80 2023-10-08T05:36:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.085+0000 7f7436ffd640 1 -- 172.21.15.70:0/189910509 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f743803e960 con 0x7f744015cf80 2023-10-08T05:36:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.086+0000 7f7436ffd640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f742406ae70 0x7f742406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.086+0000 7f7436ffd640 1 -- 172.21.15.70:0/189910509 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f74380ac740 con 0x7f744015cf80 2023-10-08T05:36:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.087+0000 7f7446233640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f742406ae70 0x7f742406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.088+0000 7f7446233640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f742406ae70 0x7f742406d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f743c006fd0 tx=0x7f743c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.090+0000 7f7436ffd640 1 -- 172.21.15.70:0/189910509 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7438077460 con 0x7f744015cf80 2023-10-08T05:36:43.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.268+0000 7f7448497640 1 -- 172.21.15.70:0/189910509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74400d3530 con 0x7f744015cf80 2023-10-08T05:36:43.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.269+0000 7f7436ffd640 1 -- 172.21.15.70:0/189910509 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f743807aa80 con 0x7f744015cf80 2023-10-08T05:36:43.270 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:36:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 -- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f742406ae70 msgr2=0x7f742406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f742406ae70 0x7f742406d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f743c006fd0 tx=0x7f743c008040 comp rx=0 tx=0).stop 2023-10-08T05:36:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 -- 172.21.15.70:0/189910509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f744015cf80 msgr2=0x7f74401e2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f744015cf80 0x7f74401e2690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7438009e50 tx=0x7f7438004a40 comp rx=0 tx=0).stop 2023-10-08T05:36:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 -- 172.21.15.70:0/189910509 shutdown_connections 2023-10-08T05:36:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f742406ae70 0x7f742406d330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f744015cf80 0x7f74401e2690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f744015cba0 0x7f74401e2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 --2- 172.21.15.70:0/189910509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74400b4d10 0x7f7440157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 -- 172.21.15.70:0/189910509 >> 172.21.15.70:0/189910509 conn(0x7f74400718a0 msgr2=0x7f74400b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:43.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 -- 172.21.15.70:0/189910509 shutdown_connections 2023-10-08T05:36:43.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.271+0000 7f7448497640 1 -- 172.21.15.70:0/189910509 wait complete. 2023-10-08T05:36:43.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:36:43.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.492+0000 7f2db35b0640 1 Processor -- start 2023-10-08T05:36:43.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.492+0000 7f2db35b0640 1 -- start start 2023-10-08T05:36:43.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.493+0000 7f2db35b0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dac0afbe0 0x7f2dac0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.493+0000 7f2db35b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 0x7f2dac0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.493+0000 7f2db35b0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dac0cbad0 con 0x7f2dac06dfc0 2023-10-08T05:36:43.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.493+0000 7f2db35b0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dac0b2620 con 0x7f2dac0afbe0 2023-10-08T05:36:43.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.493+0000 7f2db35b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac0b2060 con 0x7f2dac0cb130 2023-10-08T05:36:43.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.494+0000 7f2db134c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dac0afbe0 0x7f2dac0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.494+0000 7f2db134c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dac0afbe0 0x7f2dac0b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60346/0 (socket says 172.21.15.70:60346) 2023-10-08T05:36:43.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.494+0000 7f2db134c640 1 -- 172.21.15.70:0/3041960153 learned_addr learned my addr 172.21.15.70:0/3041960153 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:43.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.494+0000 7f2db1b4d640 1 --2- 172.21.15.70:0/3041960153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 0x7f2dac0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.494+0000 7f2db134c640 1 -- 172.21.15.70:0/3041960153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 msgr2=0x7f2dac0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.494+0000 7f2db134c640 1 --2- 172.21.15.70:0/3041960153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 0x7f2dac0cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.494+0000 7f2db134c640 1 -- 172.21.15.70:0/3041960153 >> v1:172.21.15.181:6789/0 conn(0x7f2dac06dfc0 legacy=0x7f2dac070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.494+0000 7f2db134c640 1 -- 172.21.15.70:0/3041960153 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac0b2340 con 0x7f2dac0afbe0 2023-10-08T05:36:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.495+0000 7f2db134c640 1 --2- 172.21.15.70:0/3041960153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac0afbe0 0x7f2dac0b15b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009870 tx=0x7f2da802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06a1c034b05ac41 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.496+0000 7f2da3fff640 1 -- 172.21.15.70:0/3041960153 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8031640 con 0x7f2dac0afbe0 2023-10-08T05:36:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.496+0000 7f2da3fff640 1 -- 172.21.15.70:0/3041960153 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da80317a0 con 0x7f2dac0afbe0 2023-10-08T05:36:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.496+0000 7f2da3fff640 1 -- 172.21.15.70:0/3041960153 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8031b30 con 0x7f2dac0afbe0 2023-10-08T05:36:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.496+0000 7f2db35b0640 1 -- 172.21.15.70:0/3041960153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac0afbe0 msgr2=0x7f2dac0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.496+0000 7f2db35b0640 1 --2- 172.21.15.70:0/3041960153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac0afbe0 0x7f2dac0b15b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009870 tx=0x7f2da802b8f0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.496+0000 7f2db35b0640 1 -- 172.21.15.70:0/3041960153 shutdown_connections 2023-10-08T05:36:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.496+0000 7f2db35b0640 1 --2- 172.21.15.70:0/3041960153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 0x7f2dac0cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.496+0000 7f2db35b0640 1 --2- 172.21.15.70:0/3041960153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac0afbe0 0x7f2dac0b15b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.496+0000 7f2db35b0640 1 -- 172.21.15.70:0/3041960153 >> 172.21.15.70:0/3041960153 conn(0x7f2dac06a5d0 msgr2=0x7f2dac0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.496+0000 7f2db35b0640 1 -- 172.21.15.70:0/3041960153 shutdown_connections 2023-10-08T05:36:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.497+0000 7f2db35b0640 1 -- 172.21.15.70:0/3041960153 wait complete. 2023-10-08T05:36:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.498+0000 7f2db35b0640 1 Processor -- start 2023-10-08T05:36:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.498+0000 7f2db35b0640 1 -- start start 2023-10-08T05:36:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.499+0000 7f2db35b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac06dfc0 0x7f2dac0c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.499+0000 7f2db35b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac0afbe0 0x7f2dac14a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.499+0000 7f2db35b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 0x7f2dac15c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.499+0000 7f2db35b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac1f50e0 con 0x7f2dac06dfc0 2023-10-08T05:36:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.499+0000 7f2db35b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac0c5110 con 0x7f2dac0afbe0 2023-10-08T05:36:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.499+0000 7f2db35b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac1f4e00 con 0x7f2dac0cb130 2023-10-08T05:36:43.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db134c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac06dfc0 0x7f2dac0c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db134c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac06dfc0 0x7f2dac0c4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60350/0 (socket says 172.21.15.70:60350) 2023-10-08T05:36:43.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db134c640 1 -- 172.21.15.70:0/430267740 learned_addr learned my addr 172.21.15.70:0/430267740 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:43.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db1b4d640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 0x7f2dac15c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db0b4b640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac0afbe0 0x7f2dac14a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db134c640 1 -- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 msgr2=0x7f2dac15c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db134c640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 0x7f2dac15c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db134c640 1 -- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac0afbe0 msgr2=0x7f2dac14a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db134c640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac0afbe0 0x7f2dac14a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db134c640 1 -- 172.21.15.70:0/430267740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da80094d0 con 0x7f2dac06dfc0 2023-10-08T05:36:43.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db1b4d640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 0x7f2dac15c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:43.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.500+0000 7f2db134c640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac06dfc0 0x7f2dac0c4bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009870 tx=0x7f2da80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:43.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.501+0000 7f2da1ffb640 1 -- 172.21.15.70:0/430267740 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da803ac60 con 0x7f2dac06dfc0 2023-10-08T05:36:43.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.501+0000 7f2da1ffb640 1 -- 172.21.15.70:0/430267740 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da803ade0 con 0x7f2dac06dfc0 2023-10-08T05:36:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.501+0000 7f2db35b0640 1 -- 172.21.15.70:0/430267740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dac1495e0 con 0x7f2dac06dfc0 2023-10-08T05:36:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.501+0000 7f2da1ffb640 1 -- 172.21.15.70:0/430267740 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8044630 con 0x7f2dac06dfc0 2023-10-08T05:36:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.501+0000 7f2db35b0640 1 -- 172.21.15.70:0/430267740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dac149ba0 con 0x7f2dac06dfc0 2023-10-08T05:36:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.502+0000 7f2db35b0640 1 -- 172.21.15.70:0/430267740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dac149d90 con 0x7f2dac06dfc0 2023-10-08T05:36:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.503+0000 7f2da1ffb640 1 -- 172.21.15.70:0/430267740 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2da802f050 con 0x7f2dac06dfc0 2023-10-08T05:36:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.504+0000 7f2da1ffb640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d9006ae70 0x7f2d9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.504+0000 7f2da1ffb640 1 -- 172.21.15.70:0/430267740 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2da80ab990 con 0x7f2dac06dfc0 2023-10-08T05:36:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.505+0000 7f2db0b4b640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d9006ae70 0x7f2d9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.506+0000 7f2db0b4b640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d9006ae70 0x7f2d9006d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2dac15a580 tx=0x7f2d9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.508+0000 7f2da1ffb640 1 -- 172.21.15.70:0/430267740 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2da80768a0 con 0x7f2dac06dfc0 2023-10-08T05:36:43.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.684+0000 7f2db35b0640 1 -- 172.21.15.70:0/430267740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2dac04e160 con 0x7f2dac06dfc0 2023-10-08T05:36:43.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.685+0000 7f2da1ffb640 1 -- 172.21.15.70:0/430267740 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2da80350a0 con 0x7f2dac06dfc0 2023-10-08T05:36:43.686 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:36:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 -- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d9006ae70 msgr2=0x7f2d9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d9006ae70 0x7f2d9006d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2dac15a580 tx=0x7f2d9c008040 comp rx=0 tx=0).stop 2023-10-08T05:36:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 -- 172.21.15.70:0/430267740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac06dfc0 msgr2=0x7f2dac0c4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac06dfc0 0x7f2dac0c4bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009870 tx=0x7f2da80046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 -- 172.21.15.70:0/430267740 shutdown_connections 2023-10-08T05:36:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2d9006ae70 0x7f2d9006d330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dac0cb130 0x7f2dac15c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dac0afbe0 0x7f2dac14a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 --2- 172.21.15.70:0/430267740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dac06dfc0 0x7f2dac0c4bd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 -- 172.21.15.70:0/430267740 >> 172.21.15.70:0/430267740 conn(0x7f2dac06a5d0 msgr2=0x7f2dac0c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 -- 172.21.15.70:0/430267740 shutdown_connections 2023-10-08T05:36:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.689+0000 7f2db35b0640 1 -- 172.21.15.70:0/430267740 wait complete. 2023-10-08T05:36:43.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:36:43.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.919+0000 7f88831a4640 1 Processor -- start 2023-10-08T05:36:43.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.920+0000 7f88831a4640 1 -- start start 2023-10-08T05:36:43.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.924+0000 7f88831a4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f887c06dfc0 0x7f887c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.924+0000 7f88831a4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f887c071070 0x7f887c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.925+0000 7f88831a4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0b1cc0 con 0x7f887c0afbe0 2023-10-08T05:36:43.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.925+0000 7f88831a4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c0b2620 con 0x7f887c06dfc0 2023-10-08T05:36:43.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.925+0000 7f88831a4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c0b2060 con 0x7f887c071070 2023-10-08T05:36:43.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.925+0000 7f887bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f887c06dfc0 0x7f887c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.925+0000 7f8880f40640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f887c0afbe0 0x7f887c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42016/0 (socket says 172.21.15.70:42016) 2023-10-08T05:36:43.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.925+0000 7f8880f40640 1 -- 172.21.15.70:0/1625872191 learned_addr learned my addr 172.21.15.70:0/1625872191 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:43.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.925+0000 7f8881741640 1 --2- 172.21.15.70:0/1625872191 >> v2:172.21.15.181:3300/0 conn(0x7f887c071070 0x7f887c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.926+0000 7f887bfff640 1 -- 172.21.15.70:0/1625872191 >> v2:172.21.15.181:3300/0 conn(0x7f887c071070 msgr2=0x7f887c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:43.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.926+0000 7f887bfff640 1 --2- 172.21.15.70:0/1625872191 >> v2:172.21.15.181:3300/0 conn(0x7f887c071070 0x7f887c0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.926+0000 7f887bfff640 1 -- 172.21.15.70:0/1625872191 >> v1:172.21.15.70:6789/0 conn(0x7f887c0afbe0 legacy=0x7f887c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.926+0000 7f887bfff640 1 -- 172.21.15.70:0/1625872191 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0b2340 con 0x7f887c06dfc0 2023-10-08T05:36:43.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.927+0000 7f887bfff640 1 --2- 172.21.15.70:0/1625872191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 0x7f887c070b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f887c0c8d10 tx=0x7f886802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851d043c5390832 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:43.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.928+0000 7f887b7fe640 1 -- 172.21.15.70:0/1625872191 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868031830 con 0x7f887c06dfc0 2023-10-08T05:36:43.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.928+0000 7f887b7fe640 1 -- 172.21.15.70:0/1625872191 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8868031990 con 0x7f887c06dfc0 2023-10-08T05:36:43.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.928+0000 7f887b7fe640 1 -- 172.21.15.70:0/1625872191 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868031d20 con 0x7f887c06dfc0 2023-10-08T05:36:43.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.928+0000 7f88831a4640 1 -- 172.21.15.70:0/1625872191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 msgr2=0x7f887c070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.928+0000 7f88831a4640 1 --2- 172.21.15.70:0/1625872191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 0x7f887c070b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f887c0c8d10 tx=0x7f886802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.929+0000 7f88831a4640 1 -- 172.21.15.70:0/1625872191 shutdown_connections 2023-10-08T05:36:43.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.929+0000 7f88831a4640 1 --2- 172.21.15.70:0/1625872191 >> v2:172.21.15.181:3300/0 conn(0x7f887c071070 0x7f887c0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.929+0000 7f88831a4640 1 --2- 172.21.15.70:0/1625872191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 0x7f887c070b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.929+0000 7f88831a4640 1 -- 172.21.15.70:0/1625872191 >> 172.21.15.70:0/1625872191 conn(0x7f887c06a5d0 msgr2=0x7f887c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:43.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.929+0000 7f88831a4640 1 -- 172.21.15.70:0/1625872191 shutdown_connections 2023-10-08T05:36:43.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.929+0000 7f88831a4640 1 -- 172.21.15.70:0/1625872191 wait complete. 2023-10-08T05:36:43.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.930+0000 7f88831a4640 1 Processor -- start 2023-10-08T05:36:43.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.931+0000 7f88831a4640 1 -- start start 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.931+0000 7f88831a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 0x7f887c158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f88831a4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887c071070 0x7f887c0aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f88831a4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887c0afbe0 0x7f887c0c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f88831a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c15e3e0 con 0x7f887c06dfc0 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f88831a4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c159000 con 0x7f887c0afbe0 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f88831a4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0af4a0 con 0x7f887c071070 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f8880f40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 0x7f887c158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f8880f40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 0x7f887c158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34854/0 (socket says 172.21.15.70:34854) 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f8880f40640 1 -- 172.21.15.70:0/895159371 learned_addr learned my addr 172.21.15.70:0/895159371 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f8880f40640 1 -- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887c071070 msgr2=0x7f887c0aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f887bfff640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887c071070 0x7f887c0aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f8881741640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887c0afbe0 0x7f887c0c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f8880f40640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887c071070 0x7f887c0aef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f8880f40640 1 -- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887c0afbe0 msgr2=0x7f887c0c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:43.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f8880f40640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887c0afbe0 0x7f887c0c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:43.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.932+0000 7f8880f40640 1 -- 172.21.15.70:0/895159371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8868009400 con 0x7f887c06dfc0 2023-10-08T05:36:43.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.933+0000 7f8880f40640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 0x7f887c158ac0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f887000c4b0 tx=0x7f887000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.933+0000 7f88797fa640 1 -- 172.21.15.70:0/895159371 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8870005ce0 con 0x7f887c06dfc0 2023-10-08T05:36:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.933+0000 7f88797fa640 1 -- 172.21.15.70:0/895159371 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8870005e40 con 0x7f887c06dfc0 2023-10-08T05:36:43.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.933+0000 7f88797fa640 1 -- 172.21.15.70:0/895159371 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8870007270 con 0x7f887c06dfc0 2023-10-08T05:36:43.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.933+0000 7f88831a4640 1 -- 172.21.15.70:0/895159371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c0c4db0 con 0x7f887c06dfc0 2023-10-08T05:36:43.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.933+0000 7f88831a4640 1 -- 172.21.15.70:0/895159371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c15dd20 con 0x7f887c06dfc0 2023-10-08T05:36:43.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.934+0000 7f88831a4640 1 -- 172.21.15.70:0/895159371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c04e160 con 0x7f887c06dfc0 2023-10-08T05:36:43.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.935+0000 7f88797fa640 1 -- 172.21.15.70:0/895159371 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f88700073d0 con 0x7f887c06dfc0 2023-10-08T05:36:43.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.936+0000 7f88797fa640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f886006aea0 0x7f886006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:43.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.936+0000 7f88797fa640 1 -- 172.21.15.70:0/895159371 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8870088d90 con 0x7f887c06dfc0 2023-10-08T05:36:43.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.937+0000 7f887bfff640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f886006aea0 0x7f886006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:43.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.939+0000 7f887bfff640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f886006aea0 0x7f886006d360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8868036000 tx=0x7f8868008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:43.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:43.942+0000 7f88797fa640 1 -- 172.21.15.70:0/895159371 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8870053bf0 con 0x7f887c06dfc0 2023-10-08T05:36:44.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.130+0000 7f88831a4640 1 -- 172.21.15.70:0/895159371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f887c0c6000 con 0x7f887c06dfc0 2023-10-08T05:36:44.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.131+0000 7f88797fa640 1 -- 172.21.15.70:0/895159371 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8870017020 con 0x7f887c06dfc0 2023-10-08T05:36:44.133 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 -- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f886006aea0 msgr2=0x7f886006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f886006aea0 0x7f886006d360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8868036000 tx=0x7f8868008030 comp rx=0 tx=0).stop 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 -- 172.21.15.70:0/895159371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 msgr2=0x7f887c158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 0x7f887c158ac0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f887000c4b0 tx=0x7f887000c970 comp rx=0 tx=0).stop 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 -- 172.21.15.70:0/895159371 shutdown_connections 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f886006aea0 0x7f886006d360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f887c0afbe0 0x7f887c0c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f887c071070 0x7f887c0aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 --2- 172.21.15.70:0/895159371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887c06dfc0 0x7f887c158ac0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 -- 172.21.15.70:0/895159371 >> 172.21.15.70:0/895159371 conn(0x7f887c06a5d0 msgr2=0x7f887c0b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 -- 172.21.15.70:0/895159371 shutdown_connections 2023-10-08T05:36:44.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.134+0000 7f88831a4640 1 -- 172.21.15.70:0/895159371 wait complete. 2023-10-08T05:36:44.153 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:36:44.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.363+0000 7fd000819640 1 Processor -- start 2023-10-08T05:36:44.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.363+0000 7fd000819640 1 -- start start 2023-10-08T05:36:44.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.364+0000 7fd000819640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcffc1599d0 0x7fcffc0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.364+0000 7fd000819640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcffc0cacf0 con 0x7fcffc0cc210 2023-10-08T05:36:44.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.364+0000 7fd000819640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcffc0ca3c0 con 0x7fcffc0b07d0 2023-10-08T05:36:44.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.364+0000 7fd000819640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcffc0b10f0 con 0x7fcffc1599d0 2023-10-08T05:36:44.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.364+0000 7fcffad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcffc1599d0 0x7fcffc0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.364+0000 7fcff3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcffc0cc210 0x7fcffc0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42020/0 (socket says 172.21.15.70:42020) 2023-10-08T05:36:44.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.364+0000 7fcffad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcffc1599d0 0x7fcffc0ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34864/0 (socket says 172.21.15.70:34864) 2023-10-08T05:36:44.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.364+0000 7fcff3fff640 1 -- 172.21.15.70:0/1494449982 learned_addr learned my addr 172.21.15.70:0/1494449982 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:44.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.365+0000 7fcffad9d640 1 -- 172.21.15.70:0/1494449982 >> v1:172.21.15.181:6789/0 conn(0x7fcffc0b07d0 legacy=0x7fcffc06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:44.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.365+0000 7fcffad9d640 1 -- 172.21.15.70:0/1494449982 >> v1:172.21.15.70:6789/0 conn(0x7fcffc0cc210 legacy=0x7fcffc0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.365+0000 7fcffad9d640 1 -- 172.21.15.70:0/1494449982 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc0ca6a0 con 0x7fcffc1599d0 2023-10-08T05:36:44.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.365+0000 7fcffa59c640 1 -- 172.21.15.70:0/1494449982 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030349012 0 0) 0x7fcffc0cacf0 con 0x7fcffc0cc210 2023-10-08T05:36:44.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.366+0000 7fcffad9d640 1 --2- 172.21.15.70:0/1494449982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc1599d0 0x7fcffc0ad730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00ba40 tx=0x7fcfec02e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc79f2df828ec78 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.366+0000 7fcffa59c640 1 -- 172.21.15.70:0/1494449982 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec02ee50 con 0x7fcffc1599d0 2023-10-08T05:36:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.366+0000 7fcffa59c640 1 -- 172.21.15.70:0/1494449982 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec036440 con 0x7fcffc1599d0 2023-10-08T05:36:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.366+0000 7fcffa59c640 1 -- 172.21.15.70:0/1494449982 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0367d0 con 0x7fcffc1599d0 2023-10-08T05:36:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.367+0000 7fd000819640 1 -- 172.21.15.70:0/1494449982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc1599d0 msgr2=0x7fcffc0ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.367+0000 7fd000819640 1 --2- 172.21.15.70:0/1494449982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc1599d0 0x7fcffc0ad730 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcfec00ba40 tx=0x7fcfec02e0a0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.367+0000 7fd000819640 1 -- 172.21.15.70:0/1494449982 shutdown_connections 2023-10-08T05:36:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.367+0000 7fd000819640 1 --2- 172.21.15.70:0/1494449982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc1599d0 0x7fcffc0ad730 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.367+0000 7fd000819640 1 -- 172.21.15.70:0/1494449982 >> 172.21.15.70:0/1494449982 conn(0x7fcffc0cb0c0 msgr2=0x7fcffc0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.367+0000 7fd000819640 1 -- 172.21.15.70:0/1494449982 shutdown_connections 2023-10-08T05:36:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.367+0000 7fd000819640 1 -- 172.21.15.70:0/1494449982 wait complete. 2023-10-08T05:36:44.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.369+0000 7fd000819640 1 Processor -- start 2023-10-08T05:36:44.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.369+0000 7fd000819640 1 -- start start 2023-10-08T05:36:44.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fd000819640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc0b07d0 0x7fcffc158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fd000819640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cc210 0x7fcffc15a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fd000819640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcffc1599d0 0x7fcffc1e4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fd000819640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc1e6490 con 0x7fcffc0b07d0 2023-10-08T05:36:44.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fd000819640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc159150 con 0x7fcffc1599d0 2023-10-08T05:36:44.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fd000819640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc15c4f0 con 0x7fcffc0cc210 2023-10-08T05:36:44.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fcffad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc0b07d0 0x7fcffc158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fcffad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc0b07d0 0x7fcffc158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34870/0 (socket says 172.21.15.70:34870) 2023-10-08T05:36:44.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fcffad9d640 1 -- 172.21.15.70:0/3644214969 learned_addr learned my addr 172.21.15.70:0/3644214969 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:44.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fcffad9d640 1 -- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cc210 msgr2=0x7fcffc15a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fcffad9d640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cc210 0x7fcffc15a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fcffad9d640 1 -- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcffc1599d0 msgr2=0x7fcffc1e4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fcffad9d640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcffc1599d0 0x7fcffc1e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.370+0000 7fcffad9d640 1 -- 172.21.15.70:0/3644214969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec00b6a0 con 0x7fcffc0b07d0 2023-10-08T05:36:44.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.371+0000 7fcffad9d640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc0b07d0 0x7fcffc158c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcfec000c00 tx=0x7fcfec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:44.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.371+0000 7fcff37fe640 1 -- 172.21.15.70:0/3644214969 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec004230 con 0x7fcffc0b07d0 2023-10-08T05:36:44.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.371+0000 7fcff37fe640 1 -- 172.21.15.70:0/3644214969 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec004390 con 0x7fcffc0b07d0 2023-10-08T05:36:44.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.371+0000 7fd000819640 1 -- 172.21.15.70:0/3644214969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc0aff80 con 0x7fcffc0b07d0 2023-10-08T05:36:44.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.371+0000 7fcff37fe640 1 -- 172.21.15.70:0/3644214969 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec003850 con 0x7fcffc0b07d0 2023-10-08T05:36:44.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.371+0000 7fd000819640 1 -- 172.21.15.70:0/3644214969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc15bf30 con 0x7fcffc0b07d0 2023-10-08T05:36:44.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.372+0000 7fd000819640 1 -- 172.21.15.70:0/3644214969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcffc04e160 con 0x7fcffc0b07d0 2023-10-08T05:36:44.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.373+0000 7fcff37fe640 1 -- 172.21.15.70:0/3644214969 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcfec0039b0 con 0x7fcffc0b07d0 2023-10-08T05:36:44.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.374+0000 7fcff37fe640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcfdc06ae70 0x7fcfdc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.374+0000 7fcff37fe640 1 -- 172.21.15.70:0/3644214969 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcfec0ac280 con 0x7fcffc0b07d0 2023-10-08T05:36:44.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.375+0000 7fcff3fff640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcfdc06ae70 0x7fcfdc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.376+0000 7fcff3fff640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcfdc06ae70 0x7fcfdc06d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4007fd0 tx=0x7fcfe4007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:44.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.379+0000 7fcff37fe640 1 -- 172.21.15.70:0/3644214969 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfec077040 con 0x7fcffc0b07d0 2023-10-08T05:36:44.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.606+0000 7fd000819640 1 -- 172.21.15.70:0/3644214969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcffc1e6260 con 0x7fcffc0b07d0 2023-10-08T05:36:44.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.607+0000 7fcff37fe640 1 -- 172.21.15.70:0/3644214969 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fcfec07a660 con 0x7fcffc0b07d0 2023-10-08T05:36:44.609 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:44.609 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:36:11.613175+0000","modified":"2023-10-08T05:36:33.450447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T05:36:44.609 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 9 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 -- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcfdc06ae70 msgr2=0x7fcfdc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcfdc06ae70 0x7fcfdc06d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4007fd0 tx=0x7fcfe4007920 comp rx=0 tx=0).stop 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 -- 172.21.15.70:0/3644214969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc0b07d0 msgr2=0x7fcffc158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc0b07d0 0x7fcffc158c10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcfec000c00 tx=0x7fcfec0046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 -- 172.21.15.70:0/3644214969 shutdown_connections 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcfdc06ae70 0x7fcfdc06d330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcffc1599d0 0x7fcffc1e4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cc210 0x7fcffc15a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 --2- 172.21.15.70:0/3644214969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc0b07d0 0x7fcffc158c10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 -- 172.21.15.70:0/3644214969 >> 172.21.15.70:0/3644214969 conn(0x7fcffc0cb0c0 msgr2=0x7fcffc0703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 -- 172.21.15.70:0/3644214969 shutdown_connections 2023-10-08T05:36:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.610+0000 7fd000819640 1 -- 172.21.15.70:0/3644214969 wait complete. 2023-10-08T05:36:44.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:44.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.841+0000 7f1d1fcc2640 1 Processor -- start 2023-10-08T05:36:44.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.841+0000 7f1d1fcc2640 1 -- start start 2023-10-08T05:36:44.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.843+0000 7f1d1fcc2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180709e0 0x7f1d18164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.843+0000 7f1d1fcc2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1806dee0 con 0x7f1d18159ac0 2023-10-08T05:36:44.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.843+0000 7f1d1fcc2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1815d850 con 0x7f1d180ad790 2023-10-08T05:36:44.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.843+0000 7f1d1fcc2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1806dcf0 con 0x7f1d180709e0 2023-10-08T05:36:44.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.843+0000 7f1d1da5e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1d18159ac0 0x7f1d1806d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42028/0 (socket says 172.21.15.70:42028) 2023-10-08T05:36:44.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.843+0000 7f1d1da5e640 1 -- 172.21.15.70:0/1332815628 learned_addr learned my addr 172.21.15.70:0/1332815628 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:44.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.843+0000 7f1d1e25f640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180709e0 0x7f1d18164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889808165 0 0) 0x7f1d1806dee0 con 0x7f1d18159ac0 2023-10-08T05:36:44.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d04003520 con 0x7f1d18159ac0 2023-10-08T05:36:44.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 821631799 0 0) 0x7f1d04003520 con 0x7f1d18159ac0 2023-10-08T05:36:44.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180709e0 msgr2=0x7f1d18164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:44.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180709e0 0x7f1d18164a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 >> v1:172.21.15.181:6789/0 conn(0x7f1d180ad790 legacy=0x7f1d180702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1815db30 con 0x7f1d18159ac0 2023-10-08T05:36:44.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1d14001670 con 0x7f1d18159ac0 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 >> v1:172.21.15.70:6789/0 conn(0x7f1d18159ac0 legacy=0x7f1d1806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180709e0 0x7f1d040039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04003f20 0x7f1d040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04004d00 0x7f1d04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0400ba30 con 0x7f1d04004d00 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0400bba0 con 0x7f1d04003f20 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0400df70 con 0x7f1d180709e0 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.844+0000 7f1d1e25f640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04004d00 0x7f1d04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.845+0000 7f1d1da5e640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180709e0 0x7f1d040039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.845+0000 7f1d1d25d640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04003f20 0x7f1d040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.845+0000 7f1d1e25f640 1 -- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180709e0 msgr2=0x7f1d040039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:44.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.845+0000 7f1d1e25f640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180709e0 0x7f1d040039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.845+0000 7f1d1e25f640 1 -- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04003f20 msgr2=0x7f1d040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:44.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.845+0000 7f1d1e25f640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04003f20 0x7f1d040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.845+0000 7f1d1e25f640 1 -- 172.21.15.70:0/1332815628 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d04003520 con 0x7f1d04004d00 2023-10-08T05:36:44.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.845+0000 7f1d1d25d640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04003f20 0x7f1d040047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:44.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.845+0000 7f1d1da5e640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d180709e0 0x7f1d040039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.846+0000 7f1d1e25f640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04004d00 0x7f1d04005180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d180ce750 tx=0x7f1d040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8c89204accbad6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.846+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1002cb50 con 0x7f1d04004d00 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.846+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1000f040 con 0x7f1d04004d00 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.846+0000 7f1d1ca5c640 1 -- 172.21.15.70:0/1332815628 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d10034440 con 0x7f1d04004d00 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.847+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/1332815628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04004d00 msgr2=0x7f1d04005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.847+0000 7f1d1fcc2640 1 --2- 172.21.15.70:0/1332815628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d04004d00 0x7f1d04005180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d180ce750 tx=0x7f1d040057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.847+0000 7f1d1e25f640 1 -- 172.21.15.70:0/1332815628 reap_dead start 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.847+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/1332815628 shutdown_connections 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.847+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/1332815628 >> 172.21.15.70:0/1332815628 conn(0x7f1d1806a380 msgr2=0x7f1d180b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.847+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/1332815628 shutdown_connections 2023-10-08T05:36:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.847+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/1332815628 wait complete. 2023-10-08T05:36:44.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.849+0000 7f1d1fcc2640 1 Processor -- start 2023-10-08T05:36:44.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.849+0000 7f1d1fcc2640 1 -- start start 2023-10-08T05:36:44.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.850+0000 7f1d1fcc2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04004d00 0x7f1d180b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.850+0000 7f1d1fcc2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d1815bdf0 0x7f1d180ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.850+0000 7f1d1fcc2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d1815c270 0x7f1d180cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.850+0000 7f1d1fcc2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180cd0c0 con 0x7f1d1815c270 2023-10-08T05:36:44.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.850+0000 7f1d1fcc2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1815b130 con 0x7f1d04004d00 2023-10-08T05:36:44.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.850+0000 7f1d1fcc2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180cf520 con 0x7f1d1815bdf0 2023-10-08T05:36:44.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.850+0000 7f1d1e25f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d1815c270 0x7f1d180cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.850+0000 7f1d1e25f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d1815c270 0x7f1d180cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34892/0 (socket says 172.21.15.70:34892) 2023-10-08T05:36:44.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.850+0000 7f1d1e25f640 1 -- 172.21.15.70:0/951423391 learned_addr learned my addr 172.21.15.70:0/951423391 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:44.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d1da5e640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04004d00 0x7f1d180b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d1d25d640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d1815bdf0 0x7f1d180ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d1e25f640 1 -- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d1815bdf0 msgr2=0x7f1d180ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d1e25f640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d1815bdf0 0x7f1d180ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d1e25f640 1 -- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04004d00 msgr2=0x7f1d180b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d1e25f640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04004d00 0x7f1d180b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d1e25f640 1 -- 172.21.15.70:0/951423391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d180cc720 con 0x7f1d1815c270 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d1da5e640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04004d00 0x7f1d180b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d1e25f640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d1815c270 0x7f1d180cee80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000e200 tx=0x7f1d10009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d0e7fc640 1 -- 172.21.15.70:0/951423391 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d10034a80 con 0x7f1d1815c270 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d0e7fc640 1 -- 172.21.15.70:0/951423391 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d10034be0 con 0x7f1d1815c270 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d0e7fc640 1 -- 172.21.15.70:0/951423391 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1003e440 con 0x7f1d1815c270 2023-10-08T05:36:44.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.851+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/951423391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d180cca00 con 0x7f1d1815c270 2023-10-08T05:36:44.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.852+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/951423391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d180b22c0 con 0x7f1d1815c270 2023-10-08T05:36:44.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.853+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/951423391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d180cd200 con 0x7f1d1815c270 2023-10-08T05:36:44.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.854+0000 7f1d0e7fc640 1 -- 172.21.15.70:0/951423391 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1d1003e5a0 con 0x7f1d1815c270 2023-10-08T05:36:44.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.854+0000 7f1d0e7fc640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1cfc06ae70 0x7f1cfc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:44.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.855+0000 7f1d0e7fc640 1 -- 172.21.15.70:0/951423391 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1d100ab840 con 0x7f1d1815c270 2023-10-08T05:36:44.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.855+0000 7f1d1da5e640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1cfc06ae70 0x7f1cfc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:44.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.856+0000 7f1d1da5e640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1cfc06ae70 0x7f1cfc06d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1d14001eb0 tx=0x7f1d140014f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:44.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:44.859+0000 7f1d0e7fc640 1 -- 172.21.15.70:0/951423391 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d180cd200 con 0x7f1d1815c270 2023-10-08T05:36:45.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.052+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/951423391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d180cd200 con 0x7f1d1815c270 2023-10-08T05:36:45.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.053+0000 7f1d0e7fc640 1 -- 172.21.15.70:0/951423391 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f1d180cd200 con 0x7f1d1815c270 2023-10-08T05:36:45.055 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:45.056 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:45.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.056+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1cfc06ae70 msgr2=0x7f1cfc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.056+0000 7f1d1fcc2640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1cfc06ae70 0x7f1cfc06d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1d14001eb0 tx=0x7f1d140014f0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.056+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/951423391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d1815c270 msgr2=0x7f1d180cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.056+0000 7f1d1fcc2640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d1815c270 0x7f1d180cee80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000e200 tx=0x7f1d10009750 comp rx=0 tx=0).stop 2023-10-08T05:36:45.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.056+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/951423391 shutdown_connections 2023-10-08T05:36:45.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.056+0000 7f1d1fcc2640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1cfc06ae70 0x7f1cfc06d330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.056+0000 7f1d1fcc2640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d1815c270 0x7f1d180cee80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.056+0000 7f1d1fcc2640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d1815bdf0 0x7f1d180ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.056+0000 7f1d1fcc2640 1 --2- 172.21.15.70:0/951423391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d04004d00 0x7f1d180b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.056+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/951423391 >> 172.21.15.70:0/951423391 conn(0x7f1d1806a380 msgr2=0x7f1d1806d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:45.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.057+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/951423391 shutdown_connections 2023-10-08T05:36:45.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.057+0000 7f1d1fcc2640 1 -- 172.21.15.70:0/951423391 wait complete. 2023-10-08T05:36:45.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:36:45.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:36:45.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.290+0000 7f3907b8a640 1 Processor -- start 2023-10-08T05:36:45.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.291+0000 7f3907b8a640 1 -- start start 2023-10-08T05:36:45.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.292+0000 7f3907b8a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390015aad0 0x7f39000b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.292+0000 7f3907b8a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f39000c82b0 0x7f39000c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.292+0000 7f3907b8a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f390006cd00 0x7f390006d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.292+0000 7f3907b8a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390006d6d0 con 0x7f390006cd00 2023-10-08T05:36:45.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.292+0000 7f3907b8a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390015bf30 con 0x7f39000c82b0 2023-10-08T05:36:45.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.292+0000 7f3907b8a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390015b320 con 0x7f390015aad0 2023-10-08T05:36:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.293+0000 7f3906127640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f390006cd00 0x7f390006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.293+0000 7f3906127640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f390006cd00 0x7f390006d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34898/0 (socket says 172.21.15.70:34898) 2023-10-08T05:36:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.293+0000 7f3906127640 1 -- 172.21.15.70:0/2994835696 learned_addr learned my addr 172.21.15.70:0/2994835696 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:45.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.293+0000 7f3905125640 1 --2- 172.21.15.70:0/2994835696 >> v2:172.21.15.181:3300/0 conn(0x7f39000c82b0 0x7f39000c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.293+0000 7f3905926640 1 --2- 172.21.15.70:0/2994835696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390015aad0 0x7f39000b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.293+0000 7f3906127640 1 -- 172.21.15.70:0/2994835696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390015aad0 msgr2=0x7f39000b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:45.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.293+0000 7f3906127640 1 --2- 172.21.15.70:0/2994835696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390015aad0 0x7f39000b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.293+0000 7f3906127640 1 -- 172.21.15.70:0/2994835696 >> v2:172.21.15.181:3300/0 conn(0x7f39000c82b0 msgr2=0x7f39000c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:45.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.293+0000 7f3906127640 1 --2- 172.21.15.70:0/2994835696 >> v2:172.21.15.181:3300/0 conn(0x7f39000c82b0 0x7f39000c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.293+0000 7f3906127640 1 -- 172.21.15.70:0/2994835696 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390015b600 con 0x7f390006cd00 2023-10-08T05:36:45.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.294+0000 7f3906127640 1 --2- 172.21.15.70:0/2994835696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390006cd00 0x7f390006d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f39000cc300 tx=0x7f38f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b38598b942f4a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:45.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.295+0000 7f3904924640 1 -- 172.21.15.70:0/2994835696 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8031870 con 0x7f390006cd00 2023-10-08T05:36:45.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.295+0000 7f3904924640 1 -- 172.21.15.70:0/2994835696 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f80319d0 con 0x7f390006cd00 2023-10-08T05:36:45.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.295+0000 7f3904924640 1 -- 172.21.15.70:0/2994835696 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8031d60 con 0x7f390006cd00 2023-10-08T05:36:45.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.295+0000 7f3907b8a640 1 -- 172.21.15.70:0/2994835696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390006cd00 msgr2=0x7f390006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.295+0000 7f3907b8a640 1 --2- 172.21.15.70:0/2994835696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390006cd00 0x7f390006d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f39000cc300 tx=0x7f38f802b910 comp rx=0 tx=0).stop 2023-10-08T05:36:45.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.295+0000 7f3907b8a640 1 -- 172.21.15.70:0/2994835696 shutdown_connections 2023-10-08T05:36:45.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.295+0000 7f3907b8a640 1 --2- 172.21.15.70:0/2994835696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390006cd00 0x7f390006d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.295+0000 7f3907b8a640 1 --2- 172.21.15.70:0/2994835696 >> v2:172.21.15.181:3300/0 conn(0x7f39000c82b0 0x7f39000c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.295+0000 7f3907b8a640 1 --2- 172.21.15.70:0/2994835696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390015aad0 0x7f39000b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.295+0000 7f3907b8a640 1 -- 172.21.15.70:0/2994835696 >> 172.21.15.70:0/2994835696 conn(0x7f3900069d50 msgr2=0x7f390006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:45.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.296+0000 7f3907b8a640 1 -- 172.21.15.70:0/2994835696 shutdown_connections 2023-10-08T05:36:45.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.296+0000 7f3907b8a640 1 -- 172.21.15.70:0/2994835696 wait complete. 2023-10-08T05:36:45.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.297+0000 7f3907b8a640 1 Processor -- start 2023-10-08T05:36:45.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.297+0000 7f3907b8a640 1 -- start start 2023-10-08T05:36:45.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.298+0000 7f3907b8a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390006cd00 0x7f390014b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.298+0000 7f3907b8a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39000c82b0 0x7f39000cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.298+0000 7f3907b8a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390015aad0 0x7f390014e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.298+0000 7f3907b8a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39000cc540 con 0x7f390015aad0 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.298+0000 7f3907b8a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390014b800 con 0x7f390006cd00 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.298+0000 7f3907b8a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39000cc7e0 con 0x7f39000c82b0 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3906127640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390015aad0 0x7f390014e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3906127640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390015aad0 0x7f390014e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34914/0 (socket says 172.21.15.70:34914) 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3906127640 1 -- 172.21.15.70:0/1424182702 learned_addr learned my addr 172.21.15.70:0/1424182702 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3905926640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390006cd00 0x7f390014b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3905125640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39000c82b0 0x7f39000cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3906127640 1 -- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39000c82b0 msgr2=0x7f39000cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3906127640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39000c82b0 0x7f39000cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3906127640 1 -- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390006cd00 msgr2=0x7f390014b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3906127640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390006cd00 0x7f390014b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3906127640 1 -- 172.21.15.70:0/1424182702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80094d0 con 0x7f390015aad0 2023-10-08T05:36:45.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3905125640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39000c82b0 0x7f39000cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:45.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.299+0000 7f3906127640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390015aad0 0x7f390014e990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f38f80090b0 tx=0x7f38f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:45.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.300+0000 7f38fe7fc640 1 -- 172.21.15.70:0/1424182702 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f803ad20 con 0x7f390015aad0 2023-10-08T05:36:45.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.300+0000 7f38fe7fc640 1 -- 172.21.15.70:0/1424182702 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f80443f0 con 0x7f390015aad0 2023-10-08T05:36:45.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.300+0000 7f3907b8a640 1 -- 172.21.15.70:0/1424182702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39000ccac0 con 0x7f390015aad0 2023-10-08T05:36:45.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.300+0000 7f38fe7fc640 1 -- 172.21.15.70:0/1424182702 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f803aea0 con 0x7f390015aad0 2023-10-08T05:36:45.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.300+0000 7f3907b8a640 1 -- 172.21.15.70:0/1424182702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39000cd080 con 0x7f390015aad0 2023-10-08T05:36:45.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.301+0000 7f3907b8a640 1 -- 172.21.15.70:0/1424182702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39000cc920 con 0x7f390015aad0 2023-10-08T05:36:45.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.302+0000 7f38fe7fc640 1 -- 172.21.15.70:0/1424182702 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f38f804d450 con 0x7f390015aad0 2023-10-08T05:36:45.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.303+0000 7f38fe7fc640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38e406ae70 0x7f38e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.303+0000 7f38fe7fc640 1 -- 172.21.15.70:0/1424182702 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f38f802d040 con 0x7f390015aad0 2023-10-08T05:36:45.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.303+0000 7f3905926640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38e406ae70 0x7f38e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.304+0000 7f3905926640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38e406ae70 0x7f38e406d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009a60 tx=0x7f38f4003990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:45.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.307+0000 7f38fe7fc640 1 -- 172.21.15.70:0/1424182702 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39000cc920 con 0x7f390015aad0 2023-10-08T05:36:45.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.492+0000 7f3907b8a640 1 -- 172.21.15.70:0/1424182702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39000cc920 con 0x7f390015aad0 2023-10-08T05:36:45.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.493+0000 7f38fe7fc640 1 -- 172.21.15.70:0/1424182702 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39000cc920 con 0x7f390015aad0 2023-10-08T05:36:45.495 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:36:45.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.495+0000 7f3907b8a640 1 -- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38e406ae70 msgr2=0x7f38e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.495+0000 7f3907b8a640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38e406ae70 0x7f38e406d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009a60 tx=0x7f38f4003990 comp rx=0 tx=0).stop 2023-10-08T05:36:45.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.495+0000 7f3907b8a640 1 -- 172.21.15.70:0/1424182702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390015aad0 msgr2=0x7f390014e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.495+0000 7f3907b8a640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390015aad0 0x7f390014e990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f38f80090b0 tx=0x7f38f80046c0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.495+0000 7f3907b8a640 1 -- 172.21.15.70:0/1424182702 shutdown_connections 2023-10-08T05:36:45.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.495+0000 7f3907b8a640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38e406ae70 0x7f38e406d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.495+0000 7f3907b8a640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390015aad0 0x7f390014e990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.495+0000 7f3907b8a640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39000c82b0 0x7f39000cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.495+0000 7f3907b8a640 1 --2- 172.21.15.70:0/1424182702 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390006cd00 0x7f390014b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.495+0000 7f3907b8a640 1 -- 172.21.15.70:0/1424182702 >> 172.21.15.70:0/1424182702 conn(0x7f3900069d50 msgr2=0x7f390015d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:45.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.496+0000 7f3907b8a640 1 -- 172.21.15.70:0/1424182702 shutdown_connections 2023-10-08T05:36:45.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.496+0000 7f3907b8a640 1 -- 172.21.15.70:0/1424182702 wait complete. 2023-10-08T05:36:45.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:36:45.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.725+0000 7f5cf1656640 1 Processor -- start 2023-10-08T05:36:45.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.726+0000 7f5cf1656640 1 -- start start 2023-10-08T05:36:45.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.727+0000 7f5cf1656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0afbe0 0x7f5cec0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.727+0000 7f5cf1656640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec0b2620 con 0x7f5cec06dfc0 2023-10-08T05:36:45.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.727+0000 7f5cf1656640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec0b2060 con 0x7f5cec0cb130 2023-10-08T05:36:45.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.727+0000 7f5cf1656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0b2340 con 0x7f5cec0afbe0 2023-10-08T05:36:45.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.728+0000 7f5cea7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5cec06dfc0 0x7f5cec070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42036/0 (socket says 172.21.15.70:42036) 2023-10-08T05:36:45.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.728+0000 7f5cea7fc640 1 -- 172.21.15.70:0/605256335 learned_addr learned my addr 172.21.15.70:0/605256335 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:45.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.728+0000 7f5ceaffd640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0afbe0 0x7f5cec0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.728+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334975478 0 0) 0x7f5cec0b2620 con 0x7f5cec06dfc0 2023-10-08T05:36:45.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.728+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cd8003520 con 0x7f5cec06dfc0 2023-10-08T05:36:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.728+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 845368798 0 0) 0x7f5cd8003520 con 0x7f5cec06dfc0 2023-10-08T05:36:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0afbe0 msgr2=0x7f5cec0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0afbe0 0x7f5cec0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 >> v1:172.21.15.181:6789/0 conn(0x7f5cec0cb130 legacy=0x7f5cec06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec070350 con 0x7f5cec06dfc0 2023-10-08T05:36:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5cd4001670 con 0x7f5cec06dfc0 2023-10-08T05:36:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 >> v1:172.21.15.70:6789/0 conn(0x7f5cec06dfc0 legacy=0x7f5cec070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd80039a0 0x7f5cd8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd80047c0 0x7f5cd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0afbe0 0x7f5cd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0b2340 con 0x7f5cd80039a0 2023-10-08T05:36:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd800ba30 con 0x7f5cd80047c0 2023-10-08T05:36:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd800bbf0 con 0x7f5cec0afbe0 2023-10-08T05:36:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ceaffd640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd80039a0 0x7f5cd8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5cea7fc640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd80047c0 0x7f5cd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ceaffd640 1 -- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0afbe0 msgr2=0x7f5cd8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ceaffd640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0afbe0 0x7f5cd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ceaffd640 1 -- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd80047c0 msgr2=0x7f5cd8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ceaffd640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd80047c0 0x7f5cd8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5ceaffd640 1 -- 172.21.15.70:0/605256335 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd8003520 con 0x7f5cd80039a0 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.729+0000 7f5cea7fc640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd80047c0 0x7f5cd8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.730+0000 7f5ceaffd640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd80039a0 0x7f5cd8003dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009f90 tx=0x7f5cd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359c30836bfa0acd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.731+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc02cbf0 con 0x7f5cd80039a0 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.731+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc00f040 con 0x7f5cd80039a0 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.731+0000 7f5ce9ffb640 1 -- 172.21.15.70:0/605256335 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc034440 con 0x7f5cd80039a0 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.731+0000 7f5cf1656640 1 -- 172.21.15.70:0/605256335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd80039a0 msgr2=0x7f5cd8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.731+0000 7f5cf1656640 1 --2- 172.21.15.70:0/605256335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd80039a0 0x7f5cd8003dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009f90 tx=0x7f5cd80057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.731+0000 7f5ceb7fe640 1 -- 172.21.15.70:0/605256335 reap_dead start 2023-10-08T05:36:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.731+0000 7f5cf1656640 1 -- 172.21.15.70:0/605256335 shutdown_connections 2023-10-08T05:36:45.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.731+0000 7f5cf1656640 1 -- 172.21.15.70:0/605256335 >> 172.21.15.70:0/605256335 conn(0x7f5cec06a5d0 msgr2=0x7f5cec0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:45.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.731+0000 7f5cf1656640 1 -- 172.21.15.70:0/605256335 shutdown_connections 2023-10-08T05:36:45.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.731+0000 7f5cf1656640 1 -- 172.21.15.70:0/605256335 wait complete. 2023-10-08T05:36:45.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.733+0000 7f5cf1656640 1 Processor -- start 2023-10-08T05:36:45.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.733+0000 7f5cf1656640 1 -- start start 2023-10-08T05:36:45.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.734+0000 7f5cf1656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cd80039a0 0x7f5cec0c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.734+0000 7f5cf1656640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec0c3d40 0x7f5cec14fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.734+0000 7f5cf1656640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cec158370 0x7f5cec1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.734+0000 7f5cf1656640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec158d30 con 0x7f5cec158370 2023-10-08T05:36:45.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.734+0000 7f5cf1656640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec1f0980 con 0x7f5cec0c3d40 2023-10-08T05:36:45.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.734+0000 7f5cf1656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0c2830 con 0x7f5cd80039a0 2023-10-08T05:36:45.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.734+0000 7f5ceb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cec158370 0x7f5cec1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.734+0000 7f5ceb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cec158370 0x7f5cec1587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34932/0 (socket says 172.21.15.70:34932) 2023-10-08T05:36:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.734+0000 7f5ceb7fe640 1 -- 172.21.15.70:0/2663380856 learned_addr learned my addr 172.21.15.70:0/2663380856 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5ceaffd640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cd80039a0 0x7f5cec0c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5cea7fc640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec0c3d40 0x7f5cec14fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5ceb7fe640 1 -- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cd80039a0 msgr2=0x7f5cec0c2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5ceb7fe640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cd80039a0 0x7f5cec0c2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5ceb7fe640 1 -- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec0c3d40 msgr2=0x7f5cec14fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5ceb7fe640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec0c3d40 0x7f5cec14fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5ceb7fe640 1 -- 172.21.15.70:0/2663380856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec0c2b10 con 0x7f5cec158370 2023-10-08T05:36:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5cea7fc640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec0c3d40 0x7f5cec14fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:45.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5ceb7fe640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cec158370 0x7f5cec1587f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0c8d40 tx=0x7f5ce400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:45.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5ce37fe640 1 -- 172.21.15.70:0/2663380856 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4004220 con 0x7f5cec158370 2023-10-08T05:36:45.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5ce37fe640 1 -- 172.21.15.70:0/2663380856 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce40043a0 con 0x7f5cec158370 2023-10-08T05:36:45.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5ce37fe640 1 -- 172.21.15.70:0/2663380856 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4007310 con 0x7f5cec158370 2023-10-08T05:36:45.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.735+0000 7f5cf1656640 1 -- 172.21.15.70:0/2663380856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec0c2df0 con 0x7f5cec158370 2023-10-08T05:36:45.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.736+0000 7f5cf1656640 1 -- 172.21.15.70:0/2663380856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec159230 con 0x7f5cec158370 2023-10-08T05:36:45.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.737+0000 7f5cf1656640 1 -- 172.21.15.70:0/2663380856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec04e160 con 0x7f5cec158370 2023-10-08T05:36:45.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.737+0000 7f5ce37fe640 1 -- 172.21.15.70:0/2663380856 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5ce4007470 con 0x7f5cec158370 2023-10-08T05:36:45.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.738+0000 7f5ce37fe640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ccc06aea0 0x7f5ccc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:45.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.739+0000 7f5ce37fe640 1 -- 172.21.15.70:0/2663380856 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5ce4088a20 con 0x7f5cec158370 2023-10-08T05:36:45.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.739+0000 7f5ceaffd640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ccc06aea0 0x7f5ccc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:45.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.740+0000 7f5ceaffd640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ccc06aea0 0x7f5ccc06d360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc036000 tx=0x7f5cdc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:45.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.742+0000 7f5ce37fe640 1 -- 172.21.15.70:0/2663380856 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ce4053740 con 0x7f5cec158370 2023-10-08T05:36:45.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.933+0000 7f5cf1656640 1 -- 172.21.15.70:0/2663380856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5cec0b0ef0 con 0x7f5cec158370 2023-10-08T05:36:45.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.934+0000 7f5ce37fe640 1 -- 172.21.15.70:0/2663380856 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ce4056d60 con 0x7f5cec158370 2023-10-08T05:36:45.936 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:36:45.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.938+0000 7f5cf1656640 1 -- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ccc06aea0 msgr2=0x7f5ccc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.938+0000 7f5cf1656640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ccc06aea0 0x7f5ccc06d360 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc036000 tx=0x7f5cdc02e000 comp rx=0 tx=0).stop 2023-10-08T05:36:45.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.938+0000 7f5cf1656640 1 -- 172.21.15.70:0/2663380856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cec158370 msgr2=0x7f5cec1587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:45.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.938+0000 7f5cf1656640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cec158370 0x7f5cec1587f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0c8d40 tx=0x7f5ce400c820 comp rx=0 tx=0).stop 2023-10-08T05:36:45.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.938+0000 7f5cf1656640 1 -- 172.21.15.70:0/2663380856 shutdown_connections 2023-10-08T05:36:45.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.938+0000 7f5cf1656640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ccc06aea0 0x7f5ccc06d360 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.938+0000 7f5cf1656640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cec158370 0x7f5cec1587f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.938+0000 7f5cf1656640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec0c3d40 0x7f5cec14fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.938+0000 7f5cf1656640 1 --2- 172.21.15.70:0/2663380856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cd80039a0 0x7f5cec0c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:45.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.938+0000 7f5cf1656640 1 -- 172.21.15.70:0/2663380856 >> 172.21.15.70:0/2663380856 conn(0x7f5cec06a5d0 msgr2=0x7f5cec159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:45.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.939+0000 7f5cf1656640 1 -- 172.21.15.70:0/2663380856 shutdown_connections 2023-10-08T05:36:45.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:45.939+0000 7f5cf1656640 1 -- 172.21.15.70:0/2663380856 wait complete. 2023-10-08T05:36:45.959 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:36:46.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.168+0000 7f4e06590640 1 Processor -- start 2023-10-08T05:36:46.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.168+0000 7f4e06590640 1 -- start start 2023-10-08T05:36:46.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.169+0000 7f4e06590640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4e000afbe0 0x7f4e000b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.169+0000 7f4e06590640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e0006dfc0 0x7f4e00070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.169+0000 7f4e06590640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e000b2620 con 0x7f4e00071070 2023-10-08T05:36:46.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.170+0000 7f4e06590640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e000b2060 con 0x7f4e000afbe0 2023-10-08T05:36:46.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.170+0000 7f4e06590640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e000b2340 con 0x7f4e0006dfc0 2023-10-08T05:36:46.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.170+0000 7f4dfffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4e000afbe0 0x7f4e000b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:46.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.170+0000 7f4dfffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4e000afbe0 0x7f4e000b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34942/0 (socket says 172.21.15.70:34942) 2023-10-08T05:36:46.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.170+0000 7f4dfffff640 1 -- 172.21.15.70:0/232095127 learned_addr learned my addr 172.21.15.70:0/232095127 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:46.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.170+0000 7f4dfffff640 1 -- 172.21.15.70:0/232095127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e0006dfc0 msgr2=0x7f4e00070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:46.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.170+0000 7f4dfffff640 1 --2- 172.21.15.70:0/232095127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e0006dfc0 0x7f4e00070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.170+0000 7f4dfffff640 1 -- 172.21.15.70:0/232095127 >> v1:172.21.15.181:6789/0 conn(0x7f4e00071070 legacy=0x7f4e000cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:46.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.170+0000 7f4dfffff640 1 -- 172.21.15.70:0/232095127 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e00070350 con 0x7f4e000afbe0 2023-10-08T05:36:46.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.171+0000 7f4dfffff640 1 --2- 172.21.15.70:0/232095127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e000afbe0 0x7f4e000b15b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009870 tx=0x7f4df002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6534be5b3ea9475c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:46.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4dff7fe640 1 -- 172.21.15.70:0/232095127 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0031640 con 0x7f4e000afbe0 2023-10-08T05:36:46.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4dff7fe640 1 -- 172.21.15.70:0/232095127 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df00317a0 con 0x7f4e000afbe0 2023-10-08T05:36:46.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4dff7fe640 1 -- 172.21.15.70:0/232095127 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0031b30 con 0x7f4e000afbe0 2023-10-08T05:36:46.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4e06590640 1 -- 172.21.15.70:0/232095127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e000afbe0 msgr2=0x7f4e000b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4e06590640 1 --2- 172.21.15.70:0/232095127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e000afbe0 0x7f4e000b15b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009870 tx=0x7f4df002b8f0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4e06590640 1 -- 172.21.15.70:0/232095127 shutdown_connections 2023-10-08T05:36:46.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4e06590640 1 --2- 172.21.15.70:0/232095127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e0006dfc0 0x7f4e00070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4e06590640 1 --2- 172.21.15.70:0/232095127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e000afbe0 0x7f4e000b15b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4e06590640 1 -- 172.21.15.70:0/232095127 >> 172.21.15.70:0/232095127 conn(0x7f4e0006a5d0 msgr2=0x7f4e000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:46.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4e06590640 1 -- 172.21.15.70:0/232095127 shutdown_connections 2023-10-08T05:36:46.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.172+0000 7f4e06590640 1 -- 172.21.15.70:0/232095127 wait complete. 2023-10-08T05:36:46.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.174+0000 7f4e06590640 1 Processor -- start 2023-10-08T05:36:46.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.174+0000 7f4e06590640 1 -- start start 2023-10-08T05:36:46.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.175+0000 7f4e06590640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e0006dfc0 0x7f4e0015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.175+0000 7f4e06590640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e00071070 0x7f4e00144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.175+0000 7f4e06590640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e000afbe0 0x7f4e000c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.175+0000 7f4e06590640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e000c44d0 con 0x7f4e00071070 2023-10-08T05:36:46.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.175+0000 7f4e06590640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e000c35e0 con 0x7f4e0006dfc0 2023-10-08T05:36:46.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.175+0000 7f4e06590640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e000c38c0 con 0x7f4e000afbe0 2023-10-08T05:36:46.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.175+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e00071070 0x7f4e00144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:46.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.175+0000 7f4df7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e00071070 0x7f4e00144910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34948/0 (socket says 172.21.15.70:34948) 2023-10-08T05:36:46.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.175+0000 7f4df7fff640 1 -- 172.21.15.70:0/1418797216 learned_addr learned my addr 172.21.15.70:0/1418797216 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:46.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4dfffff640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e0006dfc0 0x7f4e0015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:46.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4df7fff640 1 -- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e000afbe0 msgr2=0x7f4e000c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4e04b2d640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e000afbe0 0x7f4e000c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:46.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4df7fff640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e000afbe0 0x7f4e000c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4df7fff640 1 -- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e0006dfc0 msgr2=0x7f4e0015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4df7fff640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e0006dfc0 0x7f4e0015b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4df7fff640 1 -- 172.21.15.70:0/1418797216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00094d0 con 0x7f4e00071070 2023-10-08T05:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4df7fff640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e00071070 0x7f4e00144910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4de800b410 tx=0x7f4de800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4dfd7fa640 1 -- 172.21.15.70:0/1418797216 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8011830 con 0x7f4e00071070 2023-10-08T05:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4dfd7fa640 1 -- 172.21.15.70:0/1418797216 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de8011990 con 0x7f4e00071070 2023-10-08T05:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4dfd7fa640 1 -- 172.21.15.70:0/1418797216 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8011d40 con 0x7f4e00071070 2023-10-08T05:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.176+0000 7f4e06590640 1 -- 172.21.15.70:0/1418797216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e000c3ba0 con 0x7f4e00071070 2023-10-08T05:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.177+0000 7f4e06590640 1 -- 172.21.15.70:0/1418797216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e000c4160 con 0x7f4e00071070 2023-10-08T05:36:46.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.178+0000 7f4e06590640 1 -- 172.21.15.70:0/1418797216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0004e160 con 0x7f4e00071070 2023-10-08T05:36:46.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.178+0000 7f4dfd7fa640 1 -- 172.21.15.70:0/1418797216 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4de801a400 con 0x7f4e00071070 2023-10-08T05:36:46.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.179+0000 7f4dfd7fa640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4de006ae70 0x7f4de006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.179+0000 7f4dfd7fa640 1 -- 172.21.15.70:0/1418797216 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4de8089b90 con 0x7f4e00071070 2023-10-08T05:36:46.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.180+0000 7f4dfffff640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4de006ae70 0x7f4de006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:46.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.181+0000 7f4dfffff640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4de006ae70 0x7f4de006d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4df0036000 tx=0x7f4df003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:46.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.183+0000 7f4dfd7fa640 1 -- 172.21.15.70:0/1418797216 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4de80548b0 con 0x7f4e00071070 2023-10-08T05:36:46.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.368+0000 7f4e06590640 1 -- 172.21.15.70:0/1418797216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e00070600 con 0x7f4e00071070 2023-10-08T05:36:46.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.368+0000 7f4dfd7fa640 1 -- 172.21.15.70:0/1418797216 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4de8015020 con 0x7f4e00071070 2023-10-08T05:36:46.370 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:36:46.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 -- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4de006ae70 msgr2=0x7f4de006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4de006ae70 0x7f4de006d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4df0036000 tx=0x7f4df003c040 comp rx=0 tx=0).stop 2023-10-08T05:36:46.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 -- 172.21.15.70:0/1418797216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e00071070 msgr2=0x7f4e00144910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e00071070 0x7f4e00144910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4de800b410 tx=0x7f4de800b8d0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 -- 172.21.15.70:0/1418797216 shutdown_connections 2023-10-08T05:36:46.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4de006ae70 0x7f4de006d330 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4e000afbe0 0x7f4e000c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e00071070 0x7f4e00144910 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4de800b410 tx=0x7f4de800b8d0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 --2- 172.21.15.70:0/1418797216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4e0006dfc0 0x7f4e0015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 -- 172.21.15.70:0/1418797216 >> 172.21.15.70:0/1418797216 conn(0x7f4e0006a5d0 msgr2=0x7f4e000c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:46.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 -- 172.21.15.70:0/1418797216 shutdown_connections 2023-10-08T05:36:46.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.371+0000 7f4e06590640 1 -- 172.21.15.70:0/1418797216 wait complete. 2023-10-08T05:36:46.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:36:46.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.602+0000 7f138621a640 1 Processor -- start 2023-10-08T05:36:46.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.603+0000 7f138621a640 1 -- start start 2023-10-08T05:36:46.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.604+0000 7f138621a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f138015cae0 0x7f13800cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.604+0000 7f138621a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13800ccbc0 con 0x7f13800c8b40 2023-10-08T05:36:46.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.604+0000 7f138621a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13800ce1f0 con 0x7f13800b0680 2023-10-08T05:36:46.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.604+0000 7f138621a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13800cd590 con 0x7f138015cae0 2023-10-08T05:36:46.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.604+0000 7f1385218640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f138015cae0 0x7f13800cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:46.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.604+0000 7f1385218640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f138015cae0 0x7f13800cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34954/0 (socket says 172.21.15.70:34954) 2023-10-08T05:36:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.604+0000 7f1385218640 1 -- 172.21.15.70:0/220267590 learned_addr learned my addr 172.21.15.70:0/220267590 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.604+0000 7f1384a17640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f13800c8b40 0x7f13800aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42052/0 (socket says 172.21.15.70:42052) 2023-10-08T05:36:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.605+0000 7f1385218640 1 -- 172.21.15.70:0/220267590 >> v1:172.21.15.181:6789/0 conn(0x7f13800b0680 legacy=0x7f1380070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.605+0000 7f1385218640 1 -- 172.21.15.70:0/220267590 >> v1:172.21.15.70:6789/0 conn(0x7f13800c8b40 legacy=0x7f13800aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.605+0000 7f1385218640 1 -- 172.21.15.70:0/220267590 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13800c9720 con 0x7f138015cae0 2023-10-08T05:36:46.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.606+0000 7f1385218640 1 --2- 172.21.15.70:0/220267590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 0x7f13800cd050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f137c009870 tx=0x7f137c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ecfbfe337487ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:46.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.606+0000 7f1377fff640 1 -- 172.21.15.70:0/220267590 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c030860 con 0x7f138015cae0 2023-10-08T05:36:46.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.606+0000 7f1377fff640 1 -- 172.21.15.70:0/220267590 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c0309c0 con 0x7f138015cae0 2023-10-08T05:36:46.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.606+0000 7f1377fff640 1 -- 172.21.15.70:0/220267590 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c030d50 con 0x7f138015cae0 2023-10-08T05:36:46.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.607+0000 7f138621a640 1 -- 172.21.15.70:0/220267590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 msgr2=0x7f13800cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.607+0000 7f138621a640 1 --2- 172.21.15.70:0/220267590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 0x7f13800cd050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f137c009870 tx=0x7f137c02b9a0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.607+0000 7f138621a640 1 -- 172.21.15.70:0/220267590 shutdown_connections 2023-10-08T05:36:46.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.607+0000 7f138621a640 1 --2- 172.21.15.70:0/220267590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 0x7f13800cd050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.607+0000 7f138621a640 1 -- 172.21.15.70:0/220267590 >> 172.21.15.70:0/220267590 conn(0x7f138006c050 msgr2=0x7f138006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:46.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.607+0000 7f138621a640 1 -- 172.21.15.70:0/220267590 shutdown_connections 2023-10-08T05:36:46.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.607+0000 7f138621a640 1 -- 172.21.15.70:0/220267590 wait complete. 2023-10-08T05:36:46.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.608+0000 7f138621a640 1 Processor -- start 2023-10-08T05:36:46.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.609+0000 7f138621a640 1 -- start start 2023-10-08T05:36:46.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f138621a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13800b0680 0x7f13800af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f138621a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13800c8b40 0x7f1380156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f138621a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 0x7f138015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f138621a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800c44c0 con 0x7f138015cae0 2023-10-08T05:36:46.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f138621a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13800b17b0 con 0x7f13800c8b40 2023-10-08T05:36:46.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f138621a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13800b0e80 con 0x7f13800b0680 2023-10-08T05:36:46.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1385a19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 0x7f138015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1385a19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 0x7f138015d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34960/0 (socket says 172.21.15.70:34960) 2023-10-08T05:36:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1385a19640 1 -- 172.21.15.70:0/1621393657 learned_addr learned my addr 172.21.15.70:0/1621393657 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1384a17640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13800c8b40 0x7f1380156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1385218640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13800b0680 0x7f13800af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1385a19640 1 -- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13800b0680 msgr2=0x7f13800af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1385a19640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13800b0680 0x7f13800af510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1385a19640 1 -- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13800c8b40 msgr2=0x7f1380156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1385a19640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13800c8b40 0x7f1380156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1385a19640 1 -- 172.21.15.70:0/1621393657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c0094d0 con 0x7f138015cae0 2023-10-08T05:36:46.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.610+0000 7f1384a17640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13800c8b40 0x7f1380156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:46.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.611+0000 7f1385a19640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 0x7f138015d6f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f137800c7b0 tx=0x7f137800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:46.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.611+0000 7f1375ffb640 1 -- 172.21.15.70:0/1621393657 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378003d80 con 0x7f138015cae0 2023-10-08T05:36:46.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.611+0000 7f1375ffb640 1 -- 172.21.15.70:0/1621393657 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378005070 con 0x7f138015cae0 2023-10-08T05:36:46.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.611+0000 7f1375ffb640 1 -- 172.21.15.70:0/1621393657 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378005420 con 0x7f138015cae0 2023-10-08T05:36:46.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.611+0000 7f138621a640 1 -- 172.21.15.70:0/1621393657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13800b1160 con 0x7f138015cae0 2023-10-08T05:36:46.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.611+0000 7f138621a640 1 -- 172.21.15.70:0/1621393657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1380157ae0 con 0x7f138015cae0 2023-10-08T05:36:46.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.612+0000 7f138621a640 1 -- 172.21.15.70:0/1621393657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1380074150 con 0x7f138015cae0 2023-10-08T05:36:46.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.613+0000 7f1375ffb640 1 -- 172.21.15.70:0/1621393657 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1378005580 con 0x7f138015cae0 2023-10-08T05:36:46.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.614+0000 7f1375ffb640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f136406aea0 0x7f136406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:46.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.614+0000 7f1375ffb640 1 -- 172.21.15.70:0/1621393657 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1378088c40 con 0x7f138015cae0 2023-10-08T05:36:46.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.615+0000 7f1385218640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f136406aea0 0x7f136406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:46.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.616+0000 7f1385218640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f136406aea0 0x7f136406d360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f137c036000 tx=0x7f137c03b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:46.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.618+0000 7f1375ffb640 1 -- 172.21.15.70:0/1621393657 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f13780539f0 con 0x7f138015cae0 2023-10-08T05:36:46.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.790+0000 7f138621a640 1 -- 172.21.15.70:0/1621393657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f13800b18f0 con 0x7f138015cae0 2023-10-08T05:36:46.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.790+0000 7f1375ffb640 1 -- 172.21.15.70:0/1621393657 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1378057010 con 0x7f138015cae0 2023-10-08T05:36:46.793 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:36:46.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.793+0000 7f138621a640 1 -- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f136406aea0 msgr2=0x7f136406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.793+0000 7f138621a640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f136406aea0 0x7f136406d360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f137c036000 tx=0x7f137c03b040 comp rx=0 tx=0).stop 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.793+0000 7f138621a640 1 -- 172.21.15.70:0/1621393657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 msgr2=0x7f138015d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.793+0000 7f138621a640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 0x7f138015d6f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f137800c7b0 tx=0x7f137800cc70 comp rx=0 tx=0).stop 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.794+0000 7f138621a640 1 -- 172.21.15.70:0/1621393657 shutdown_connections 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.794+0000 7f138621a640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f136406aea0 0x7f136406d360 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.794+0000 7f138621a640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f138015cae0 0x7f138015d6f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.794+0000 7f138621a640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13800c8b40 0x7f1380156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.794+0000 7f138621a640 1 --2- 172.21.15.70:0/1621393657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13800b0680 0x7f13800af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.794+0000 7f138621a640 1 -- 172.21.15.70:0/1621393657 >> 172.21.15.70:0/1621393657 conn(0x7f138006c050 msgr2=0x7f13800ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.794+0000 7f138621a640 1 -- 172.21.15.70:0/1621393657 shutdown_connections 2023-10-08T05:36:46.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:46.794+0000 7f138621a640 1 -- 172.21.15.70:0/1621393657 wait complete. 2023-10-08T05:36:46.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:36:47.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.026+0000 7fb9e7646640 1 Processor -- start 2023-10-08T05:36:47.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.026+0000 7fb9e7646640 1 -- start start 2023-10-08T05:36:47.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.028+0000 7fb9e7646640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb9e00ad790 0x7fb9e00702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.028+0000 7fb9e7646640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e006dee0 con 0x7fb9e015e330 2023-10-08T05:36:47.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.028+0000 7fb9e7646640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e015d850 con 0x7fb9e0159ac0 2023-10-08T05:36:47.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.028+0000 7fb9e7646640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e006dcf0 con 0x7fb9e00ad790 2023-10-08T05:36:47.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.028+0000 7fb9e5be3640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb9e015e330 0x7fb9e00c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42064/0 (socket says 172.21.15.70:42064) 2023-10-08T05:36:47.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.028+0000 7fb9e5be3640 1 -- 172.21.15.70:0/4252924017 learned_addr learned my addr 172.21.15.70:0/4252924017 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:47.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.028+0000 7fb9e4be1640 1 --2- 172.21.15.70:0/4252924017 >> v2:172.21.15.181:3300/0 conn(0x7fb9e00ad790 0x7fb9e00702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.028+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232254988 0 0) 0x7fb9e006dee0 con 0x7fb9e015e330 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.028+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9cc005730 con 0x7fb9e015e330 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 989534303 0 0) 0x7fb9cc005730 con 0x7fb9e015e330 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 >> v2:172.21.15.181:3300/0 conn(0x7fb9e00ad790 msgr2=0x7fb9e00702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 --2- 172.21.15.70:0/4252924017 >> v2:172.21.15.181:3300/0 conn(0x7fb9e00ad790 0x7fb9e00702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 >> v1:172.21.15.181:6789/0 conn(0x7fb9e0159ac0 legacy=0x7fb9e006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e015db30 con 0x7fb9e015e330 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fb9d80019f0 con 0x7fb9e015e330 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 >> v1:172.21.15.70:6789/0 conn(0x7fb9e015e330 legacy=0x7fb9e00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9cc005bd0 0x7fb9cc005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9cc006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9cc007390 0x7fb9cc00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc00e260 con 0x7fb9cc0069d0 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc00e3d0 con 0x7fb9cc007390 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc00e590 con 0x7fb9cc005bd0 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.029+0000 7fb9e4be1640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9cc006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.030+0000 7fb9e5be3640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9cc007390 0x7fb9cc00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.030+0000 7fb9e53e2640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9cc005bd0 0x7fb9cc005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.030+0000 7fb9e4be1640 1 -- 172.21.15.70:0/4252924017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9cc005bd0 msgr2=0x7fb9cc005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.030+0000 7fb9e4be1640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9cc005bd0 0x7fb9cc005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.030+0000 7fb9e4be1640 1 -- 172.21.15.70:0/4252924017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9cc007390 msgr2=0x7fb9cc00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.030+0000 7fb9e4be1640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9cc007390 0x7fb9cc00db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.030+0000 7fb9e4be1640 1 -- 172.21.15.70:0/4252924017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc005730 con 0x7fb9cc0069d0 2023-10-08T05:36:47.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.030+0000 7fb9e5be3640 1 -- 172.21.15.70:0/4252924017 reap_dead start 2023-10-08T05:36:47.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.031+0000 7fb9e4be1640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9cc006e50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009ac0 tx=0x7fb9cc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a968cb6198721b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:47.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.031+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d002c6b0 con 0x7fb9cc0069d0 2023-10-08T05:36:47.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.031+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d002c810 con 0x7fb9cc0069d0 2023-10-08T05:36:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.031+0000 7fb9d7fff640 1 -- 172.21.15.70:0/4252924017 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d002cba0 con 0x7fb9cc0069d0 2023-10-08T05:36:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.032+0000 7fb9e7646640 1 -- 172.21.15.70:0/4252924017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 msgr2=0x7fb9cc006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.032+0000 7fb9e7646640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9cc006e50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009ac0 tx=0x7fb9cc007a00 comp rx=0 tx=0).stop 2023-10-08T05:36:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.032+0000 7fb9e7646640 1 -- 172.21.15.70:0/4252924017 shutdown_connections 2023-10-08T05:36:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.032+0000 7fb9e7646640 1 --2- 172.21.15.70:0/4252924017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9cc006e50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.032+0000 7fb9e7646640 1 -- 172.21.15.70:0/4252924017 >> 172.21.15.70:0/4252924017 conn(0x7fb9e006a380 msgr2=0x7fb9e00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.032+0000 7fb9e7646640 1 -- 172.21.15.70:0/4252924017 shutdown_connections 2023-10-08T05:36:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.032+0000 7fb9e7646640 1 -- 172.21.15.70:0/4252924017 wait complete. 2023-10-08T05:36:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.034+0000 7fb9e7646640 1 Processor -- start 2023-10-08T05:36:47.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.034+0000 7fb9e7646640 1 -- start start 2023-10-08T05:36:47.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e7646640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9e0158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e7646640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9e01573e0 0x7fb9e015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e7646640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9e00af640 0x7fb9e00afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e7646640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e01f6b00 con 0x7fb9cc0069d0 2023-10-08T05:36:47.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e7646640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00b0030 con 0x7fb9e00af640 2023-10-08T05:36:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e7646640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e01f5ec0 con 0x7fb9e01573e0 2023-10-08T05:36:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e53e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9e0158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e53e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9e0158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34982/0 (socket says 172.21.15.70:34982) 2023-10-08T05:36:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e53e2640 1 -- 172.21.15.70:0/3966494028 learned_addr learned my addr 172.21.15.70:0/3966494028 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e5be3640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9e00af640 0x7fb9e00afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e53e2640 1 -- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9e01573e0 msgr2=0x7fb9e015a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.035+0000 7fb9e53e2640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9e01573e0 0x7fb9e015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.036+0000 7fb9e53e2640 1 -- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9e00af640 msgr2=0x7fb9e00afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.036+0000 7fb9e53e2640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9e00af640 0x7fb9e00afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.036+0000 7fb9e53e2640 1 -- 172.21.15.70:0/3966494028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e01f6160 con 0x7fb9cc0069d0 2023-10-08T05:36:47.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.036+0000 7fb9e53e2640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9e0158ca0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00e3c0 tx=0x7fb9dc00e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:47.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.036+0000 7fb9d5ffb640 1 -- 172.21.15.70:0/3966494028 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0052e0 con 0x7fb9cc0069d0 2023-10-08T05:36:47.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.036+0000 7fb9d5ffb640 1 -- 172.21.15.70:0/3966494028 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc005440 con 0x7fb9cc0069d0 2023-10-08T05:36:47.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.036+0000 7fb9d5ffb640 1 -- 172.21.15.70:0/3966494028 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc003810 con 0x7fb9cc0069d0 2023-10-08T05:36:47.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.036+0000 7fb9e7646640 1 -- 172.21.15.70:0/3966494028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e01f6440 con 0x7fb9cc0069d0 2023-10-08T05:36:47.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.036+0000 7fb9e7646640 1 -- 172.21.15.70:0/3966494028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e01591e0 con 0x7fb9cc0069d0 2023-10-08T05:36:47.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.037+0000 7fb9e7646640 1 -- 172.21.15.70:0/3966494028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e00716a0 con 0x7fb9cc0069d0 2023-10-08T05:36:47.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.038+0000 7fb9d5ffb640 1 -- 172.21.15.70:0/3966494028 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb9dc003970 con 0x7fb9cc0069d0 2023-10-08T05:36:47.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.039+0000 7fb9d5ffb640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb9c406aea0 0x7fb9c406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.039+0000 7fb9d5ffb640 1 -- 172.21.15.70:0/3966494028 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb9dc088fd0 con 0x7fb9cc0069d0 2023-10-08T05:36:47.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.040+0000 7fb9e4be1640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb9c406aea0 0x7fb9c406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.041+0000 7fb9e4be1640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb9c406aea0 0x7fb9c406d360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0036000 tx=0x7fb9d002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:47.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.043+0000 7fb9d5ffb640 1 -- 172.21.15.70:0/3966494028 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9dc053e30 con 0x7fb9cc0069d0 2023-10-08T05:36:47.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.265+0000 7fb9e7646640 1 -- 172.21.15.70:0/3966494028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9e0159370 con 0x7fb9cc0069d0 2023-10-08T05:36:47.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.266+0000 7fb9d5ffb640 1 -- 172.21.15.70:0/3966494028 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fb9dc019020 con 0x7fb9cc0069d0 2023-10-08T05:36:47.268 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:47.268 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:36:11.613175+0000","modified":"2023-10-08T05:36:33.450447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T05:36:47.268 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 9 2023-10-08T05:36:47.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 -- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb9c406aea0 msgr2=0x7fb9c406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb9c406aea0 0x7fb9c406d360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0036000 tx=0x7fb9d002e000 comp rx=0 tx=0).stop 2023-10-08T05:36:47.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 -- 172.21.15.70:0/3966494028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 msgr2=0x7fb9e0158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9e0158ca0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00e3c0 tx=0x7fb9dc00e880 comp rx=0 tx=0).stop 2023-10-08T05:36:47.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 -- 172.21.15.70:0/3966494028 shutdown_connections 2023-10-08T05:36:47.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb9c406aea0 0x7fb9c406d360 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9e00af640 0x7fb9e00afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9e01573e0 0x7fb9e015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 --2- 172.21.15.70:0/3966494028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0069d0 0x7fb9e0158ca0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 -- 172.21.15.70:0/3966494028 >> 172.21.15.70:0/3966494028 conn(0x7fb9e006a380 msgr2=0x7fb9e0154180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:47.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 -- 172.21.15.70:0/3966494028 shutdown_connections 2023-10-08T05:36:47.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.269+0000 7fb9e7646640 1 -- 172.21.15.70:0/3966494028 wait complete. 2023-10-08T05:36:47.288 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:36:47.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.495+0000 7f1c4c5d0640 1 Processor -- start 2023-10-08T05:36:47.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.495+0000 7f1c4c5d0640 1 -- start start 2023-10-08T05:36:47.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4c5d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c440ad790 0x7f1c440702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4c5d0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1c4415e330 0x7f1c44070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4c5d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4406dee0 con 0x7f1c44159ac0 2023-10-08T05:36:47.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4c5d0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4406dcf0 con 0x7f1c4415e330 2023-10-08T05:36:47.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4c5d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4415d850 con 0x7f1c440ad790 2023-10-08T05:36:47.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4a36c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1c44159ac0 0x7f1c4406d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42072/0 (socket says 172.21.15.70:42072) 2023-10-08T05:36:47.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4a36c640 1 -- 172.21.15.70:0/3368567436 learned_addr learned my addr 172.21.15.70:0/3368567436 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:47.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406616884 0 0) 0x7f1c4406dee0 con 0x7f1c44159ac0 2023-10-08T05:36:47.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c49b6b640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c440ad790 0x7f1c440702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4ab6d640 1 --2- 172.21.15.70:0/3368567436 >> v2:172.21.15.181:3300/0 conn(0x7f1c4415e330 0x7f1c44070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c30003520 con 0x7f1c44159ac0 2023-10-08T05:36:47.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2673886059 0 0) 0x7f1c30003520 con 0x7f1c44159ac0 2023-10-08T05:36:47.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c440ad790 msgr2=0x7f1c440702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:47.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c440ad790 0x7f1c440702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 >> v2:172.21.15.181:3300/0 conn(0x7f1c4415e330 msgr2=0x7f1c44070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:47.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 --2- 172.21.15.70:0/3368567436 >> v2:172.21.15.181:3300/0 conn(0x7f1c4415e330 0x7f1c44070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4415db30 con 0x7f1c44159ac0 2023-10-08T05:36:47.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1c40001670 con 0x7f1c44159ac0 2023-10-08T05:36:47.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 >> v1:172.21.15.70:6789/0 conn(0x7f1c44159ac0 legacy=0x7f1c4406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c30003970 0x7f1c30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c440ad790 0x7f1c300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c30004d00 0x7f1c30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4415d850 con 0x7f1c30003970 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3000b980 con 0x7f1c30004d00 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3000bac0 con 0x7f1c440ad790 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.497+0000 7f1c4a36c640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c30003970 0x7f1c30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.498+0000 7f1c4ab6d640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c30004d00 0x7f1c30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.498+0000 7f1c4a36c640 1 -- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c440ad790 msgr2=0x7f1c300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.498+0000 7f1c49b6b640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c440ad790 0x7f1c300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.498+0000 7f1c4a36c640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c440ad790 0x7f1c300047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.498+0000 7f1c4a36c640 1 -- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c30004d00 msgr2=0x7f1c30005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:47.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.498+0000 7f1c4a36c640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c30004d00 0x7f1c30005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.498+0000 7f1c4a36c640 1 -- 172.21.15.70:0/3368567436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c30003520 con 0x7f1c30003970 2023-10-08T05:36:47.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.498+0000 7f1c4ab6d640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c30004d00 0x7f1c30005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:47.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.499+0000 7f1c4a36c640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c30003970 0x7f1c30003d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c290 tx=0x7f1c300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ce0bd2b19e7931 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:47.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.499+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c40005ad0 con 0x7f1c30003970 2023-10-08T05:36:47.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.499+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c40005c30 con 0x7f1c30003970 2023-10-08T05:36:47.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.499+0000 7f1c4936a640 1 -- 172.21.15.70:0/3368567436 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4002c8d0 con 0x7f1c30003970 2023-10-08T05:36:47.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.499+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3368567436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c30003970 msgr2=0x7f1c30003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.500+0000 7f1c4c5d0640 1 --2- 172.21.15.70:0/3368567436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c30003970 0x7f1c30003d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000c290 tx=0x7f1c300057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.500+0000 7f1c4ab6d640 1 -- 172.21.15.70:0/3368567436 reap_dead start 2023-10-08T05:36:47.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.500+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3368567436 shutdown_connections 2023-10-08T05:36:47.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.500+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3368567436 >> 172.21.15.70:0/3368567436 conn(0x7f1c4406a380 msgr2=0x7f1c440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:47.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.500+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3368567436 shutdown_connections 2023-10-08T05:36:47.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.500+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3368567436 wait complete. 2023-10-08T05:36:47.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.501+0000 7f1c4c5d0640 1 Processor -- start 2023-10-08T05:36:47.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.502+0000 7f1c4c5d0640 1 -- start start 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.502+0000 7f1c4c5d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c30003970 0x7f1c4414e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c4c5d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c4415bf70 0x7f1c4414d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c4c5d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c4414d9b0 0x7f1c4414bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c4c5d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4415bae0 con 0x7f1c4415bf70 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c4c5d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4415b1b0 con 0x7f1c4414d9b0 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c4c5d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4415b490 con 0x7f1c30003970 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c49b6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c4415bf70 0x7f1c4414d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c49b6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c4415bf70 0x7f1c4414d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35002/0 (socket says 172.21.15.70:35002) 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c49b6b640 1 -- 172.21.15.70:0/3275427360 learned_addr learned my addr 172.21.15.70:0/3275427360 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c4ab6d640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c4414d9b0 0x7f1c4414bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c49b6b640 1 -- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c30003970 msgr2=0x7f1c4414e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c49b6b640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c30003970 0x7f1c4414e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c49b6b640 1 -- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c4414d9b0 msgr2=0x7f1c4414bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c49b6b640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c4414d9b0 0x7f1c4414bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c49b6b640 1 -- 172.21.15.70:0/3275427360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4415b770 con 0x7f1c4415bf70 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.503+0000 7f1c49b6b640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c4415bf70 0x7f1c4414d470 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1c34010890 tx=0x7f1c34010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.504+0000 7f1c3affd640 1 -- 172.21.15.70:0/3275427360 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340095b0 con 0x7f1c4415bf70 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.504+0000 7f1c3affd640 1 -- 172.21.15.70:0/3275427360 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3400eb80 con 0x7f1c4415bf70 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.504+0000 7f1c3affd640 1 -- 172.21.15.70:0/3275427360 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3401c400 con 0x7f1c4415bf70 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.504+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3275427360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4414a410 con 0x7f1c4415bf70 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.504+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3275427360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4414ed30 con 0x7f1c4415bf70 2023-10-08T05:36:47.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.505+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3275427360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4415b900 con 0x7f1c4415bf70 2023-10-08T05:36:47.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.506+0000 7f1c3affd640 1 -- 172.21.15.70:0/3275427360 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1c340143f0 con 0x7f1c4415bf70 2023-10-08T05:36:47.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.507+0000 7f1c3affd640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c2806ae70 0x7f1c2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.507+0000 7f1c3affd640 1 -- 172.21.15.70:0/3275427360 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1c34089b90 con 0x7f1c4415bf70 2023-10-08T05:36:47.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.507+0000 7f1c4a36c640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c2806ae70 0x7f1c2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.509+0000 7f1c4a36c640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c2806ae70 0x7f1c2806d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c40034000 tx=0x7f1c4002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:47.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.512+0000 7f1c3affd640 1 -- 172.21.15.70:0/3275427360 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c340548b0 con 0x7f1c4415bf70 2023-10-08T05:36:47.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.690+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3275427360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c44067490 con 0x7f1c4415bf70 2023-10-08T05:36:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.693+0000 7f1c3affd640 1 -- 172.21.15.70:0/3275427360 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18334 (secure 0 0 0) 0x7f1c34057ed0 con 0x7f1c4415bf70 2023-10-08T05:36:47.694 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:47.694 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:16.666907+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:36:47.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c2806ae70 msgr2=0x7f1c2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c2806ae70 0x7f1c2806d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1c40034000 tx=0x7f1c4002e000 comp rx=0 tx=0).stop 2023-10-08T05:36:47.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3275427360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c4415bf70 msgr2=0x7f1c4414d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c4415bf70 0x7f1c4414d470 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1c34010890 tx=0x7f1c34010d50 comp rx=0 tx=0).stop 2023-10-08T05:36:47.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3275427360 shutdown_connections 2023-10-08T05:36:47.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c2806ae70 0x7f1c2806d330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c4414d9b0 0x7f1c4414bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c4415bf70 0x7f1c4414d470 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 --2- 172.21.15.70:0/3275427360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c30003970 0x7f1c4414e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3275427360 >> 172.21.15.70:0/3275427360 conn(0x7f1c4406a380 msgr2=0x7f1c440cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:47.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3275427360 shutdown_connections 2023-10-08T05:36:47.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.695+0000 7f1c4c5d0640 1 -- 172.21.15.70:0/3275427360 wait complete. 2023-10-08T05:36:47.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:36:47.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.922+0000 7f04ac01a640 1 Processor -- start 2023-10-08T05:36:47.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.923+0000 7f04ac01a640 1 -- start start 2023-10-08T05:36:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.924+0000 7f04ac01a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04a40c90b0 0x7f04a40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.924+0000 7f04ac01a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a415b2b0 con 0x7f04a40cb3b0 2023-10-08T05:36:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.924+0000 7f04ac01a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04a415b590 con 0x7f04a415ab40 2023-10-08T05:36:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.924+0000 7f04ac01a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04a415bf30 con 0x7f04a40c90b0 2023-10-08T05:36:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.924+0000 7f04aa5b7640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f04a40cb3b0 0x7f04a415e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42078/0 (socket says 172.21.15.70:42078) 2023-10-08T05:36:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.924+0000 7f04aa5b7640 1 -- 172.21.15.70:0/3066843679 learned_addr learned my addr 172.21.15.70:0/3066843679 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:47.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a95b5640 1 --2- 172.21.15.70:0/3066843679 >> v2:172.21.15.181:3300/0 conn(0x7f04a40c90b0 0x7f04a40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296339356 0 0) 0x7f04a415b2b0 con 0x7f04a40cb3b0 2023-10-08T05:36:47.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0490003520 con 0x7f04a40cb3b0 2023-10-08T05:36:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 938967023 0 0) 0x7f0490003520 con 0x7f04a40cb3b0 2023-10-08T05:36:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 >> v2:172.21.15.181:3300/0 conn(0x7f04a40c90b0 msgr2=0x7f04a40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 --2- 172.21.15.70:0/3066843679 >> v2:172.21.15.181:3300/0 conn(0x7f04a40c90b0 0x7f04a40cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 >> v1:172.21.15.181:6789/0 conn(0x7f04a415ab40 legacy=0x7f04a40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a415b870 con 0x7f04a40cb3b0 2023-10-08T05:36:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f049c0019f0 con 0x7f04a40cb3b0 2023-10-08T05:36:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 >> v1:172.21.15.70:6789/0 conn(0x7f04a40cb3b0 legacy=0x7f04a415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0490003970 0x7f0490003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04900047c0 0x7f0490004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490005180 0x7f049000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a415bf30 con 0x7f04900047c0 2023-10-08T05:36:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.925+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049000c020 con 0x7f0490005180 2023-10-08T05:36:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.926+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049000c1e0 con 0x7f0490003970 2023-10-08T05:36:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.926+0000 7f04a95b5640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04900047c0 0x7f0490004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.926+0000 7f04aa5b7640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490005180 0x7f049000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.926+0000 7f04a9db6640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0490003970 0x7f0490003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.926+0000 7f04a95b5640 1 -- 172.21.15.70:0/3066843679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0490003970 msgr2=0x7f0490003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.926+0000 7f04a95b5640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0490003970 0x7f0490003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.926+0000 7f04a95b5640 1 -- 172.21.15.70:0/3066843679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490005180 msgr2=0x7f049000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:47.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.926+0000 7f04a95b5640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0490005180 0x7f049000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.926+0000 7f04a95b5640 1 -- 172.21.15.70:0/3066843679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490003520 con 0x7f04900047c0 2023-10-08T05:36:47.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.926+0000 7f04aa5b7640 1 -- 172.21.15.70:0/3066843679 reap_dead start 2023-10-08T05:36:47.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.927+0000 7f04a95b5640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04900047c0 0x7f0490004c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0494009a90 tx=0x7f04900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f8d6a23bc39906 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.927+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049402c680 con 0x7f04900047c0 2023-10-08T05:36:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.927+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049402c7e0 con 0x7f04900047c0 2023-10-08T05:36:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.927+0000 7f04a8db4640 1 -- 172.21.15.70:0/3066843679 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049402cb70 con 0x7f04900047c0 2023-10-08T05:36:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.928+0000 7f04ac01a640 1 -- 172.21.15.70:0/3066843679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04900047c0 msgr2=0x7f0490004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.928+0000 7f04ac01a640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04900047c0 0x7f0490004c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0494009a90 tx=0x7f04900057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.928+0000 7f04ac01a640 1 -- 172.21.15.70:0/3066843679 shutdown_connections 2023-10-08T05:36:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.928+0000 7f04ac01a640 1 --2- 172.21.15.70:0/3066843679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04900047c0 0x7f0490004c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.928+0000 7f04ac01a640 1 -- 172.21.15.70:0/3066843679 >> 172.21.15.70:0/3066843679 conn(0x7f04a406a380 msgr2=0x7f04a40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.928+0000 7f04ac01a640 1 -- 172.21.15.70:0/3066843679 shutdown_connections 2023-10-08T05:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.928+0000 7f04ac01a640 1 -- 172.21.15.70:0/3066843679 wait complete. 2023-10-08T05:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.930+0000 7f04ac01a640 1 Processor -- start 2023-10-08T05:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.930+0000 7f04ac01a640 1 -- start start 2023-10-08T05:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.931+0000 7f04ac01a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04900047c0 0x7f04a40c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.931+0000 7f04ac01a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04a40adda0 0x7f04a4156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.931+0000 7f04ac01a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a41491c0 0x7f04a40b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.931+0000 7f04ac01a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a41583b0 con 0x7f04a40adda0 2023-10-08T05:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.931+0000 7f04ac01a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04a4157730 con 0x7f04a41491c0 2023-10-08T05:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.931+0000 7f04ac01a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04a4157a10 con 0x7f04900047c0 2023-10-08T05:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.931+0000 7f04a95b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04a40adda0 0x7f04a4156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.931+0000 7f04a95b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04a40adda0 0x7f04a4156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35022/0 (socket says 172.21.15.70:35022) 2023-10-08T05:36:47.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.931+0000 7f04a95b5640 1 -- 172.21.15.70:0/4039282556 learned_addr learned my addr 172.21.15.70:0/4039282556 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:47.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.931+0000 7f04a9db6640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04900047c0 0x7f04a40c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f04a95b5640 1 -- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04900047c0 msgr2=0x7f04a40c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f04a95b5640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04900047c0 0x7f04a40c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f04aa5b7640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a41491c0 0x7f04a40b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f04a95b5640 1 -- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a41491c0 msgr2=0x7f04a40b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:47.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f04a95b5640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a41491c0 0x7f04a40b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:47.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f04a95b5640 1 -- 172.21.15.70:0/4039282556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a4157cf0 con 0x7f04a40adda0 2023-10-08T05:36:47.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f04a9db6640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04900047c0 0x7f04a40c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:47.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f04a95b5640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04a40adda0 0x7f04a4156f00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0494009a90 tx=0x7f0494004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:47.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f049a7fc640 1 -- 172.21.15.70:0/4039282556 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494003810 con 0x7f04a40adda0 2023-10-08T05:36:47.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f049a7fc640 1 -- 172.21.15.70:0/4039282556 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0494003970 con 0x7f04a40adda0 2023-10-08T05:36:47.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f04ac01a640 1 -- 172.21.15.70:0/4039282556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04a4157fd0 con 0x7f04a40adda0 2023-10-08T05:36:47.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f049a7fc640 1 -- 172.21.15.70:0/4039282556 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494003d00 con 0x7f04a40adda0 2023-10-08T05:36:47.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.932+0000 7f04ac01a640 1 -- 172.21.15.70:0/4039282556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04a4149640 con 0x7f04a40adda0 2023-10-08T05:36:47.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.933+0000 7f04ac01a640 1 -- 172.21.15.70:0/4039282556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04a41584f0 con 0x7f04a40adda0 2023-10-08T05:36:47.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.935+0000 7f049a7fc640 1 -- 172.21.15.70:0/4039282556 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0494040450 con 0x7f04a40adda0 2023-10-08T05:36:47.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.935+0000 7f049a7fc640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f048806aea0 0x7f048806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:47.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.935+0000 7f049a7fc640 1 -- 172.21.15.70:0/4039282556 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f04940ab7e0 con 0x7f04a40adda0 2023-10-08T05:36:47.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.936+0000 7f04a9db6640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f048806aea0 0x7f048806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:47.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.937+0000 7f04a9db6640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f048806aea0 0x7f048806d360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f04a0000c00 tx=0x7f04a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:47.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:47.940+0000 7f049a7fc640 1 -- 172.21.15.70:0/4039282556 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04a41584f0 con 0x7f04a40adda0 2023-10-08T05:36:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.169+0000 7f04ac01a640 1 -- 172.21.15.70:0/4039282556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04a41584f0 con 0x7f04a40adda0 2023-10-08T05:36:48.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.171+0000 7f049a7fc640 1 -- 172.21.15.70:0/4039282556 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f04a41584f0 con 0x7f04a40adda0 2023-10-08T05:36:48.171 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:36:48.172 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:36:11.613175+0000","modified":"2023-10-08T05:36:33.450447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T05:36:48.172 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 9 2023-10-08T05:36:48.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.173+0000 7f04ac01a640 1 -- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f048806aea0 msgr2=0x7f048806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.173+0000 7f04ac01a640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f048806aea0 0x7f048806d360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f04a0000c00 tx=0x7f04a0009040 comp rx=0 tx=0).stop 2023-10-08T05:36:48.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.173+0000 7f04ac01a640 1 -- 172.21.15.70:0/4039282556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04a40adda0 msgr2=0x7f04a4156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.173+0000 7f04ac01a640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04a40adda0 0x7f04a4156f00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0494009a90 tx=0x7f0494004730 comp rx=0 tx=0).stop 2023-10-08T05:36:48.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.173+0000 7f04ac01a640 1 -- 172.21.15.70:0/4039282556 shutdown_connections 2023-10-08T05:36:48.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.173+0000 7f04ac01a640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f048806aea0 0x7f048806d360 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.173+0000 7f04ac01a640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04a41491c0 0x7f04a40b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.173+0000 7f04ac01a640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04a40adda0 0x7f04a4156f00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.173+0000 7f04ac01a640 1 --2- 172.21.15.70:0/4039282556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04900047c0 0x7f04a40c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.173+0000 7f04ac01a640 1 -- 172.21.15.70:0/4039282556 >> 172.21.15.70:0/4039282556 conn(0x7f04a406a380 msgr2=0x7f04a40af480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:48.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.174+0000 7f04ac01a640 1 -- 172.21.15.70:0/4039282556 shutdown_connections 2023-10-08T05:36:48.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.174+0000 7f04ac01a640 1 -- 172.21.15.70:0/4039282556 wait complete. 2023-10-08T05:36:48.593 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T05:36:11.613175+0000', 'modified': '2023-10-08T05:36:33.450447+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4255}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4255': {'gid': 4255, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/1675009602', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6835', 'nonce': 1675009602}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 1675009602}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T05:36:48.594 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T05:36:48.594 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T05:36:48.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.820+0000 7f8fd6e39640 1 Processor -- start 2023-10-08T05:36:48.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.820+0000 7f8fd6e39640 1 -- start start 2023-10-08T05:36:48.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.821+0000 7f8fd6e39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8fd0070cc0 0x7f8fd00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.821+0000 7f8fd6e39640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8fd00cb450 0x7f8fd00b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.821+0000 7f8fd6e39640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd0071170 con 0x7f8fd0160660 2023-10-08T05:36:48.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.821+0000 7f8fd6e39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd006e150 con 0x7f8fd0070cc0 2023-10-08T05:36:48.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.821+0000 7f8fd6e39640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd006de70 con 0x7f8fd00cb450 2023-10-08T05:36:48.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.822+0000 7f8fcffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8fd0070cc0 0x7f8fd00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.822+0000 7f8fcffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8fd0070cc0 0x7f8fd00caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35028/0 (socket says 172.21.15.70:35028) 2023-10-08T05:36:48.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.822+0000 7f8fcffff640 1 -- 172.21.15.70:0/4125231157 learned_addr learned my addr 172.21.15.70:0/4125231157 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:48.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.822+0000 7f8fd53d6640 1 --2- 172.21.15.70:0/4125231157 >> v2:172.21.15.181:3300/0 conn(0x7f8fd00cb450 0x7f8fd00b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.822+0000 7f8fcffff640 1 -- 172.21.15.70:0/4125231157 >> v2:172.21.15.181:3300/0 conn(0x7f8fd00cb450 msgr2=0x7f8fd00b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:48.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.822+0000 7f8fcffff640 1 --2- 172.21.15.70:0/4125231157 >> v2:172.21.15.181:3300/0 conn(0x7f8fd00cb450 0x7f8fd00b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.822+0000 7f8fcffff640 1 -- 172.21.15.70:0/4125231157 >> v1:172.21.15.70:6789/0 conn(0x7f8fd0160660 legacy=0x7f8fd00c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.822+0000 7f8fcffff640 1 -- 172.21.15.70:0/4125231157 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd00c8e30 con 0x7f8fd0070cc0 2023-10-08T05:36:48.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.823+0000 7f8fcffff640 1 --2- 172.21.15.70:0/4125231157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0070cc0 0x7f8fd00caf10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00b1760 tx=0x7f8fbc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61e09d783bae6ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:48.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.824+0000 7f8fcf7fe640 1 -- 172.21.15.70:0/4125231157 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc030860 con 0x7f8fd0070cc0 2023-10-08T05:36:48.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.824+0000 7f8fcf7fe640 1 -- 172.21.15.70:0/4125231157 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0309c0 con 0x7f8fd0070cc0 2023-10-08T05:36:48.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.824+0000 7f8fcf7fe640 1 -- 172.21.15.70:0/4125231157 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc030d50 con 0x7f8fd0070cc0 2023-10-08T05:36:48.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.824+0000 7f8fd6e39640 1 -- 172.21.15.70:0/4125231157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0070cc0 msgr2=0x7f8fd00caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.824+0000 7f8fd6e39640 1 --2- 172.21.15.70:0/4125231157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0070cc0 0x7f8fd00caf10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00b1760 tx=0x7f8fbc02b9a0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.825+0000 7f8fd6e39640 1 -- 172.21.15.70:0/4125231157 shutdown_connections 2023-10-08T05:36:48.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.825+0000 7f8fd6e39640 1 --2- 172.21.15.70:0/4125231157 >> v2:172.21.15.181:3300/0 conn(0x7f8fd00cb450 0x7f8fd00b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.825+0000 7f8fd6e39640 1 --2- 172.21.15.70:0/4125231157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0070cc0 0x7f8fd00caf10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.825+0000 7f8fd6e39640 1 -- 172.21.15.70:0/4125231157 >> 172.21.15.70:0/4125231157 conn(0x7f8fd006a5d0 msgr2=0x7f8fd00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:48.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.825+0000 7f8fd6e39640 1 -- 172.21.15.70:0/4125231157 shutdown_connections 2023-10-08T05:36:48.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.825+0000 7f8fd6e39640 1 -- 172.21.15.70:0/4125231157 wait complete. 2023-10-08T05:36:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.827+0000 7f8fd6e39640 1 Processor -- start 2023-10-08T05:36:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.827+0000 7f8fd6e39640 1 -- start start 2023-10-08T05:36:48.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd6e39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fd0070cc0 0x7f8fd0155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd6e39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd00cb450 0x7f8fd0145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd6e39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0160660 0x7f8fd015f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd6e39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0157ab0 con 0x7f8fd0160660 2023-10-08T05:36:48.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd6e39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd0155c80 con 0x7f8fd00cb450 2023-10-08T05:36:48.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd6e39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd01489b0 con 0x7f8fd0070cc0 2023-10-08T05:36:48.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd53d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0160660 0x7f8fd015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd53d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0160660 0x7f8fd015f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35042/0 (socket says 172.21.15.70:35042) 2023-10-08T05:36:48.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd53d6640 1 -- 172.21.15.70:0/2533804933 learned_addr learned my addr 172.21.15.70:0/2533804933 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fcffff640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd00cb450 0x7f8fd0145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd53d6640 1 -- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fd0070cc0 msgr2=0x7f8fd0155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd53d6640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fd0070cc0 0x7f8fd0155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd53d6640 1 -- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd00cb450 msgr2=0x7f8fd0145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.828+0000 7f8fd53d6640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd00cb450 0x7f8fd0145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.829+0000 7f8fd53d6640 1 -- 172.21.15.70:0/2533804933 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc0094d0 con 0x7f8fd0160660 2023-10-08T05:36:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.829+0000 7f8fd53d6640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0160660 0x7f8fd015f450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8007e40 tx=0x7f8fc800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.829+0000 7f8fcd7fa640 1 -- 172.21.15.70:0/2533804933 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8014620 con 0x7f8fd0160660 2023-10-08T05:36:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.829+0000 7f8fcd7fa640 1 -- 172.21.15.70:0/2533804933 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc800ee90 con 0x7f8fd0160660 2023-10-08T05:36:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.829+0000 7f8fd6e39640 1 -- 172.21.15.70:0/2533804933 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd0157460 con 0x7f8fd0160660 2023-10-08T05:36:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.829+0000 7f8fcd7fa640 1 -- 172.21.15.70:0/2533804933 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc80149e0 con 0x7f8fd0160660 2023-10-08T05:36:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.829+0000 7f8fd6e39640 1 -- 172.21.15.70:0/2533804933 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd01460b0 con 0x7f8fd0160660 2023-10-08T05:36:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.832+0000 7f8fcd7fa640 1 -- 172.21.15.70:0/2533804933 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8fc8014b40 con 0x7f8fd0160660 2023-10-08T05:36:48.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.833+0000 7f8fcd7fa640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8fb406ae70 0x7f8fb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.833+0000 7f8fcd7fa640 1 -- 172.21.15.70:0/2533804933 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8fc808a3c0 con 0x7f8fd0160660 2023-10-08T05:36:48.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.833+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8fb406ae70 0x7f8fb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.833+0000 7f8fd99c8740 1 Processor -- start 2023-10-08T05:36:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.834+0000 7f8fd99c8740 1 -- start start 2023-10-08T05:36:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.834+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8fb406ae70 0x7f8fb406d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4000f60 tx=0x7f8fc40079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4fb60 0x55b443d22a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4ff40 0x55b443d528f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d22fa0 0x55b443d52e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b443c96700 con 0x55b443d4fb60 2023-10-08T05:36:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b443c93df0 con 0x55b443d4ff40 2023-10-08T05:36:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b443c8f6e0 con 0x55b443d22fa0 2023-10-08T05:36:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd4bd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4fb60 0x55b443d22a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd4bd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4fb60 0x55b443d22a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35046/0 (socket says 172.21.15.70:35046) 2023-10-08T05:36:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd4bd5640 1 -- 172.21.15.70:0/2460167595 learned_addr learned my addr 172.21.15.70:0/2460167595 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fcffff640 1 --2- 172.21.15.70:0/2460167595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4ff40 0x55b443d528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd4bd5640 1 -- 172.21.15.70:0/2460167595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d22fa0 msgr2=0x55b443d52e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/2460167595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d22fa0 0x55b443d52e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd4bd5640 1 -- 172.21.15.70:0/2460167595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4ff40 msgr2=0x55b443d528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/2460167595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4ff40 0x55b443d528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.835+0000 7f8fd4bd5640 1 -- 172.21.15.70:0/2460167595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b443c53910 con 0x55b443d4fb60 2023-10-08T05:36:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.836+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/2460167595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4fb60 0x55b443d22a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4016880 tx=0x7f8fc400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e332114bcb565d81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.836+0000 7f8fba7fc640 1 -- 172.21.15.70:0/2460167595 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc401b070 con 0x55b443d4fb60 2023-10-08T05:36:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.836+0000 7f8fba7fc640 1 -- 172.21.15.70:0/2460167595 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc40162e0 con 0x55b443d4fb60 2023-10-08T05:36:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.836+0000 7f8fba7fc640 1 -- 172.21.15.70:0/2460167595 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4005330 con 0x55b443d4fb60 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.836+0000 7f8fd99c8740 1 -- 172.21.15.70:0/2460167595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4fb60 msgr2=0x55b443d22a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.836+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/2460167595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4fb60 0x55b443d22a60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4016880 tx=0x7f8fc400f000 comp rx=0 tx=0).stop 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.837+0000 7f8fd99c8740 1 -- 172.21.15.70:0/2460167595 shutdown_connections 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.837+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/2460167595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d22fa0 0x55b443d52e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.837+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/2460167595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4ff40 0x55b443d528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.837+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/2460167595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4fb60 0x55b443d22a60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.837+0000 7f8fd99c8740 1 -- 172.21.15.70:0/2460167595 >> 172.21.15.70:0/2460167595 conn(0x55b443d24d70 msgr2=0x55b443d6d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.837+0000 7f8fd99c8740 1 -- 172.21.15.70:0/2460167595 shutdown_connections 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.837+0000 7f8fd99c8740 1 -- 172.21.15.70:0/2460167595 wait complete. 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.837+0000 7f8fd99c8740 1 -- start start 2023-10-08T05:36:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.838+0000 7f8fd99c8740 1 Processor -- start 2023-10-08T05:36:48.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d4fb60 0x55b443d40f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4ff40 0x55b443d67f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d66600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b443c2d970 con 0x55b443d415b0 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b443c2d610 con 0x55b443d4ff40 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4439f3f20 con 0x55b443d4fb60 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd99c8740 10 client.0 _ll_register_callbacks cb 0x55b443c6b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd53d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d66600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd53d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d66600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35060/0 (socket says 172.21.15.70:35060) 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd53d6640 1 -- 172.21.15.70:0/1541252871 learned_addr learned my addr 172.21.15.70:0/1541252871 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd53d6640 1 -- 172.21.15.70:0/1541252871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d4fb60 msgr2=0x55b443d40f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd53d6640 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d4fb60 0x55b443d40f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd53d6640 1 -- 172.21.15.70:0/1541252871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4ff40 msgr2=0x55b443d67f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd53d6640 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4ff40 0x55b443d67f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.839+0000 7f8fd53d6640 1 -- 172.21.15.70:0/1541252871 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc4016500 con 0x55b443d415b0 2023-10-08T05:36:48.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8fd53d6640 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d66600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80552a0 tx=0x7f8fc8058da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae13b75568867ef7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8faf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8faf7fe640 1 -- 172.21.15.70:0/1541252871 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc805c070 con 0x55b443d415b0 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8faf7fe640 1 -- 172.21.15.70:0/1541252871 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8fc808a0b0 con 0x55b443d415b0 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8fd99c8740 1 -- 172.21.15.70:0/1541252871 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b443c886c0 con 0x55b443d415b0 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8faf7fe640 1 -- 172.21.15.70:0/1541252871 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fc8088170 con 0x55b443d415b0 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8fd99c8740 10 client.4833 fetch_fsmap learned FSMap version 9 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8fd99c8740 1 -- 172.21.15.70:0/1541252871 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55b443c8dd60 con 0x55b443d415b0 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8faf7fe640 1 -- 172.21.15.70:0/1541252871 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b443c886c0 con 0x55b443d415b0 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8faf7fe640 1 -- 172.21.15.70:0/1541252871 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f8fc80554b0 con 0x55b443d415b0 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.840+0000 7f8fd99c8740 10 client.4833 fetch_fsmap finished waiting for FSMap version 9 2023-10-08T05:36:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.841+0000 7f8fd99c8740 10 client.4833 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T05:36:48.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.841+0000 7f8fd99c8740 20 client.4833 populate_metadata read hostname 'smithi070' 2023-10-08T05:36:48.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.841+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55b443d56080 0x55b443d8b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.841+0000 7f8fd99c8740 4 client.4833 mds_command: new command op to 4255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T05:36:48.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.841+0000 7f8fd99c8740 1 -- 172.21.15.70:0/1541252871 --> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b443c883e0 con 0x55b443d56080 2023-10-08T05:36:48.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.841+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55b443d56080 0x55b443d8b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.842+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55b443d56080 0x55b443d8b000 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a5ab032868aef8 server_cookie=66f6e229191a0cb6 in_seq=0 out_seq=0 2023-10-08T05:36:48.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.842+0000 7f8faf7fe640 10 client.4833 ms_handle_connect on v2:172.21.15.181:6835/1675009602 2023-10-08T05:36:48.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8faf7fe640 1 -- 172.21.15.70:0/1541252871 <== mds.0 v2:172.21.15.181:6835/1675009602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b443c883e0 con 0x55b443d56080 2023-10-08T05:36:48.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8faf7fe640 10 client.4833 handle_command_reply: tid=0 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 client.4833 shutdown 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 -- 172.21.15.70:0/1541252871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 msgr2=0x55b443d66600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d66600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80552a0 tx=0x7f8fc8058da0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 -- 172.21.15.70:0/1541252871 shutdown_connections 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55b443d56080 0x55b443d8b000 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d66600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4ff40 0x55b443d67f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/1541252871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d4fb60 0x55b443d40f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8faf7fe640 0 client.4833 ms_handle_reset on v2:172.21.15.181:6835/1675009602 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 -- 172.21.15.70:0/1541252871 >> 172.21.15.70:0/1541252871 conn(0x55b443d22fa0 msgr2=0x55b443d269e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:48.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 -- 172.21.15.70:0/1541252871 shutdown_connections 2023-10-08T05:36:48.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.844+0000 7f8fd99c8740 1 -- 172.21.15.70:0/1541252871 wait complete. 2023-10-08T05:36:48.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.845+0000 7f8fd99c8740 20 client.4833 trim_cache size 0 max 16384 2023-10-08T05:36:48.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.861+0000 7f8fd99c8740 1 Processor -- start 2023-10-08T05:36:48.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.861+0000 7f8fd99c8740 1 -- start start 2023-10-08T05:36:48.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.862+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d21c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.862+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 0x55b443d27800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.862+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d4ff40 0x55b443d27d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.862+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b443d8f600 con 0x55b443d415b0 2023-10-08T05:36:48.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.862+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b443c84ea0 con 0x55b443d4fb60 2023-10-08T05:36:48.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.862+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b443d24870 con 0x55b443d4ff40 2023-10-08T05:36:48.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fd4bd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d21c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fd4bd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d21c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35070/0 (socket says 172.21.15.70:35070) 2023-10-08T05:36:48.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fd4bd5640 1 -- 172.21.15.70:0/3526904845 learned_addr learned my addr 172.21.15.70:0/3526904845 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:48.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fcffff640 1 --2- 172.21.15.70:0/3526904845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 0x55b443d27800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fd4bd5640 1 -- 172.21.15.70:0/3526904845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d4ff40 msgr2=0x55b443d27d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:48.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/3526904845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d4ff40 0x55b443d27d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fd4bd5640 1 -- 172.21.15.70:0/3526904845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 msgr2=0x55b443d27800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:48.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/3526904845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 0x55b443d27800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fd4bd5640 1 -- 172.21.15.70:0/3526904845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b443c2cb80 con 0x55b443d415b0 2023-10-08T05:36:48.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/3526904845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d21c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4013780 tx=0x7f8fc4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d0d3e9aad16b74 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:48.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.863+0000 7f8fbaffd640 1 -- 172.21.15.70:0/3526904845 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc401b070 con 0x55b443d415b0 2023-10-08T05:36:48.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fbaffd640 1 -- 172.21.15.70:0/3526904845 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc401c040 con 0x55b443d415b0 2023-10-08T05:36:48.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fbaffd640 1 -- 172.21.15.70:0/3526904845 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4025040 con 0x55b443d415b0 2023-10-08T05:36:48.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fd99c8740 1 -- 172.21.15.70:0/3526904845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 msgr2=0x55b443d21c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/3526904845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d21c30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4013780 tx=0x7f8fc4013000 comp rx=0 tx=0).stop 2023-10-08T05:36:48.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fd99c8740 1 -- 172.21.15.70:0/3526904845 shutdown_connections 2023-10-08T05:36:48.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/3526904845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d4ff40 0x55b443d27d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/3526904845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 0x55b443d27800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/3526904845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d415b0 0x55b443d21c30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fd99c8740 1 -- 172.21.15.70:0/3526904845 >> 172.21.15.70:0/3526904845 conn(0x55b443d22fa0 msgr2=0x55b443d80580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:48.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fd99c8740 1 -- 172.21.15.70:0/3526904845 shutdown_connections 2023-10-08T05:36:48.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.864+0000 7f8fd99c8740 1 -- 172.21.15.70:0/3526904845 wait complete. 2023-10-08T05:36:48.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.865+0000 7f8fd99c8740 1 -- start start 2023-10-08T05:36:48.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.865+0000 7f8fd99c8740 1 Processor -- start 2023-10-08T05:36:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.866+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d415b0 0x55b443db9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.866+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 0x55b443d28250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.866+0000 7f8fd99c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4ff40 0x55b443dcccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.866+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b443d28790 con 0x55b443d4ff40 2023-10-08T05:36:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.866+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b443dcd6c0 con 0x55b443d4fb60 2023-10-08T05:36:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.866+0000 7f8fd99c8740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b443d64260 con 0x55b443d415b0 2023-10-08T05:36:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.866+0000 7f8fd99c8740 10 client.0 _ll_register_callbacks cb 0x7f8fc4001230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:36:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.866+0000 7f8fd53d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4ff40 0x55b443dcccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fd53d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4ff40 0x55b443dcccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35072/0 (socket says 172.21.15.70:35072) 2023-10-08T05:36:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fd53d6640 1 -- 172.21.15.70:0/528792263 learned_addr learned my addr 172.21.15.70:0/528792263 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fcffff640 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 0x55b443d28250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fd53d6640 1 -- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d415b0 msgr2=0x55b443db9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fd53d6640 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d415b0 0x55b443db9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fd53d6640 1 -- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 msgr2=0x55b443d28250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fd53d6640 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 0x55b443d28250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fd53d6640 1 -- 172.21.15.70:0/528792263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b443c53910 con 0x55b443d4ff40 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fcffff640 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 0x55b443d28250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fd53d6640 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4ff40 0x55b443dcccb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800c040 tx=0x7f8fc8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333e4c59b114963b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8faf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8faf7fe640 1 -- 172.21.15.70:0/528792263 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc805c040 con 0x55b443d4ff40 2023-10-08T05:36:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8faf7fe640 1 -- 172.21.15.70:0/528792263 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8fc808a0b0 con 0x55b443d4ff40 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.867+0000 7f8fd99c8740 1 -- 172.21.15.70:0/528792263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fc40162e0 con 0x55b443d4ff40 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8faf7fe640 1 -- 172.21.15.70:0/528792263 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fc80550e0 con 0x55b443d4ff40 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8fd99c8740 10 client.4839 fetch_fsmap learned FSMap version 9 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8fd99c8740 1 -- 172.21.15.70:0/528792263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55b443c52170 con 0x55b443d4ff40 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8faf7fe640 1 -- 172.21.15.70:0/528792263 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc40162e0 con 0x55b443d4ff40 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8faf7fe640 1 -- 172.21.15.70:0/528792263 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f8fc8058290 con 0x55b443d4ff40 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8fd99c8740 10 client.4839 fetch_fsmap finished waiting for FSMap version 9 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8fd99c8740 10 client.4839 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8fd99c8740 20 client.4839 populate_metadata read hostname 'smithi070' 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55b443d4f210 0x55b443d3fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8fd99c8740 4 client.4839 mds_command: new command op to 4255 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T05:36:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.868+0000 7f8fd99c8740 1 -- 172.21.15.70:0/528792263 --> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b4439916c0 con 0x55b443d4f210 2023-10-08T05:36:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.869+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55b443d4f210 0x55b443d3fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.869+0000 7f8fd4bd5640 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55b443d4f210 0x55b443d3fbd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd0a9d345396668 server_cookie=d43a98d7d52a209f in_seq=0 out_seq=0 2023-10-08T05:36:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.869+0000 7f8faf7fe640 10 client.4839 ms_handle_connect on v2:172.21.15.181:6835/1675009602 2023-10-08T05:36:48.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8faf7fe640 1 -- 172.21.15.70:0/528792263 <== mds.0 v2:172.21.15.181:6835/1675009602 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55b4439916c0 con 0x55b443d4f210 2023-10-08T05:36:48.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8faf7fe640 10 client.4839 handle_command_reply: tid=0 2023-10-08T05:36:48.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 client.4839 shutdown 2023-10-08T05:36:48.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 -- 172.21.15.70:0/528792263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4ff40 msgr2=0x55b443dcccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4ff40 0x55b443dcccb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800c040 tx=0x7f8fc8007f30 comp rx=0 tx=0).stop 2023-10-08T05:36:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 -- 172.21.15.70:0/528792263 shutdown_connections 2023-10-08T05:36:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:6835/1675009602,v1:172.21.15.181:6837/1675009602] conn(0x55b443d4f210 0x55b443d3fbd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55b443d4ff40 0x55b443dcccb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55b443d4fb60 0x55b443d28250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 --2- 172.21.15.70:0/528792263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55b443d415b0 0x55b443db9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8faf7fe640 0 client.4839 ms_handle_reset on v2:172.21.15.181:6835/1675009602 2023-10-08T05:36:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 -- 172.21.15.70:0/528792263 >> 172.21.15.70:0/528792263 conn(0x55b443d22fa0 msgr2=0x55b443d685c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 -- 172.21.15.70:0/528792263 shutdown_connections 2023-10-08T05:36:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.870+0000 7f8fd99c8740 1 -- 172.21.15.70:0/528792263 wait complete. 2023-10-08T05:36:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd99c8740 20 client.4839 trim_cache size 0 max 16384 2023-10-08T05:36:48.876 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:36:48.876 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T05:36:48.876 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T05:36:48.876 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 4255, 2023-10-08T05:36:48.876 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T05:36:48.876 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T05:36:48.876 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T05:36:48.876 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 36.722959009, 2023-10-08T05:36:48.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 9, 2023-10-08T05:36:48.877 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 25, 2023-10-08T05:36:48.877 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 25, 2023-10-08T05:36:48.877 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 44.558020837999997 2023-10-08T05:36:48.877 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:36:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 -- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8fb406ae70 msgr2=0x7f8fb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8fb406ae70 0x7f8fb406d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4000f60 tx=0x7f8fc40079b0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 -- 172.21.15.70:0/2533804933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0160660 msgr2=0x7f8fd015f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0160660 0x7f8fd015f450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8007e40 tx=0x7f8fc800e4b0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 -- 172.21.15.70:0/2533804933 shutdown_connections 2023-10-08T05:36:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8fb406ae70 0x7f8fb406d330 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fd0160660 0x7f8fd015f450 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fd00cb450 0x7f8fd0145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 --2- 172.21.15.70:0/2533804933 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fd0070cc0 0x7f8fd0155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:48.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 -- 172.21.15.70:0/2533804933 >> 172.21.15.70:0/2533804933 conn(0x7f8fd006a5d0 msgr2=0x7f8fd00cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:48.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 -- 172.21.15.70:0/2533804933 shutdown_connections 2023-10-08T05:36:48.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:48.871+0000 7f8fd6e39640 1 -- 172.21.15.70:0/2533804933 wait complete. 2023-10-08T05:36:48.888 INFO:teuthology.run_tasks:Running task kclient... 2023-10-08T05:36:48.900 INFO:tasks.kclient:Mounting kernel clients... 2023-10-08T05:36:48.901 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-08T05:36:48.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:48.901 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:36:48.954 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-08T05:36:48.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:48.954 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-08T05:36:49.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:49.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2023-10-08T05:36:49.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:49.180 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:36:49.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:49.224 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2023-10-08T05:36:49.358 INFO:teuthology.orchestra.run.smithi070.stderr:Cannot find device "ceph-brx" 2023-10-08T05:36:49.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T05:36:49.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:49.361 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-08T05:36:49.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:49.422 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-08T05:36:49.525 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2023-10-08T05:36:49.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T05:36:49.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:49.529 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-08T05:36:49.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:49.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-08T05:36:49.671 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2023-10-08T05:36:49.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T05:36:49.675 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-08T05:36:49.675 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T05:36:49.675 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T05:36:49.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:36:49.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.898+0000 7f027c33c640 1 Processor -- start 2023-10-08T05:36:49.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.898+0000 7f027c33c640 1 -- start start 2023-10-08T05:36:49.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.900+0000 7f027c33c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f027406d4c0 0x7f0274162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:49.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.900+0000 7f027c33c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02740ae140 con 0x7f02740add60 2023-10-08T05:36:49.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.900+0000 7f027c33c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02740ccf20 con 0x7f02740cb150 2023-10-08T05:36:49.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.900+0000 7f027c33c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0274161f10 con 0x7f027406d4c0 2023-10-08T05:36:49.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.901+0000 7f02798d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f027406d4c0 0x7f0274162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:49.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.901+0000 7f02798d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f027406d4c0 0x7f0274162330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35078/0 (socket says 172.21.15.70:35078) 2023-10-08T05:36:49.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.901+0000 7f027a0d8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f02740add60 0x7f02740cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42092/0 (socket says 172.21.15.70:42092) 2023-10-08T05:36:49.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.901+0000 7f02798d7640 1 -- 172.21.15.70:0/4047972915 learned_addr learned my addr 172.21.15.70:0/4047972915 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:49.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.901+0000 7f02798d7640 1 -- 172.21.15.70:0/4047972915 >> v1:172.21.15.181:6789/0 conn(0x7f02740cb150 legacy=0x7f0274164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:49.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.901+0000 7f02798d7640 1 -- 172.21.15.70:0/4047972915 >> v1:172.21.15.70:6789/0 conn(0x7f02740add60 legacy=0x7f02740cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:49.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.901+0000 7f02798d7640 1 -- 172.21.15.70:0/4047972915 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02740cd1c0 con 0x7f027406d4c0 2023-10-08T05:36:49.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.902+0000 7f02798d7640 1 --2- 172.21.15.70:0/4047972915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 0x7f0274162330 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0264009870 tx=0x7f026402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5eaffd396da998d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:49.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.903+0000 7f02790d6640 1 -- 172.21.15.70:0/4047972915 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264031870 con 0x7f027406d4c0 2023-10-08T05:36:49.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.903+0000 7f02790d6640 1 -- 172.21.15.70:0/4047972915 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02640319d0 con 0x7f027406d4c0 2023-10-08T05:36:49.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.903+0000 7f02790d6640 1 -- 172.21.15.70:0/4047972915 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264031d60 con 0x7f027406d4c0 2023-10-08T05:36:49.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.903+0000 7f027c33c640 1 -- 172.21.15.70:0/4047972915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 msgr2=0x7f0274162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:49.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.903+0000 7f027c33c640 1 --2- 172.21.15.70:0/4047972915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 0x7f0274162330 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0264009870 tx=0x7f026402b910 comp rx=0 tx=0).stop 2023-10-08T05:36:49.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.903+0000 7f027c33c640 1 -- 172.21.15.70:0/4047972915 shutdown_connections 2023-10-08T05:36:49.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.903+0000 7f027c33c640 1 --2- 172.21.15.70:0/4047972915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 0x7f0274162330 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:49.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.903+0000 7f027c33c640 1 -- 172.21.15.70:0/4047972915 >> 172.21.15.70:0/4047972915 conn(0x7f027406a380 msgr2=0x7f02740b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:49.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.903+0000 7f027c33c640 1 -- 172.21.15.70:0/4047972915 shutdown_connections 2023-10-08T05:36:49.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.903+0000 7f027c33c640 1 -- 172.21.15.70:0/4047972915 wait complete. 2023-10-08T05:36:49.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.905+0000 7f027c33c640 1 Processor -- start 2023-10-08T05:36:49.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.905+0000 7f027c33c640 1 -- start start 2023-10-08T05:36:49.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.906+0000 7f027c33c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 0x7f02741f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:49.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.906+0000 7f027c33c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02740add60 0x7f027415b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:49.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.906+0000 7f027c33c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02740cb150 0x7f02741fa220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:49.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.906+0000 7f027c33c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027415c230 con 0x7f027406d4c0 2023-10-08T05:36:49.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.906+0000 7f027c33c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027415c510 con 0x7f02740cb150 2023-10-08T05:36:49.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.906+0000 7f027c33c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027415c7f0 con 0x7f02740add60 2023-10-08T05:36:49.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.906+0000 7f027a0d8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 0x7f02741f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f027a0d8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 0x7f02741f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35094/0 (socket says 172.21.15.70:35094) 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f027a0d8640 1 -- 172.21.15.70:0/1305839690 learned_addr learned my addr 172.21.15.70:0/1305839690 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f027a0d8640 1 -- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02740add60 msgr2=0x7f027415b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f027a0d8640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02740add60 0x7f027415b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f027a0d8640 1 -- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02740cb150 msgr2=0x7f02741fa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f027a0d8640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02740cb150 0x7f02741fa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f027a0d8640 1 -- 172.21.15.70:0/1305839690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02640094d0 con 0x7f027406d4c0 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f027a0d8640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 0x7f02741f75c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f027000c800 tx=0x7f027000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f026affd640 1 -- 172.21.15.70:0/1305839690 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000a640 con 0x7f027406d4c0 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f026affd640 1 -- 172.21.15.70:0/1305839690 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027000a7a0 con 0x7f027406d4c0 2023-10-08T05:36:49.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f026affd640 1 -- 172.21.15.70:0/1305839690 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000ab50 con 0x7f027406d4c0 2023-10-08T05:36:49.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.907+0000 7f027c33c640 1 -- 172.21.15.70:0/1305839690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027415cad0 con 0x7f027406d4c0 2023-10-08T05:36:49.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.908+0000 7f027c33c640 1 -- 172.21.15.70:0/1305839690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02741f7b00 con 0x7f027406d4c0 2023-10-08T05:36:49.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.909+0000 7f027c33c640 1 -- 172.21.15.70:0/1305839690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0274067490 con 0x7f027406d4c0 2023-10-08T05:36:49.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.910+0000 7f026affd640 1 -- 172.21.15.70:0/1305839690 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f027000acb0 con 0x7f027406d4c0 2023-10-08T05:36:49.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.911+0000 7f026affd640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f025806aea0 0x7f025806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:49.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.911+0000 7f026affd640 1 -- 172.21.15.70:0/1305839690 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0270089020 con 0x7f027406d4c0 2023-10-08T05:36:49.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.911+0000 7f02798d7640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f025806aea0 0x7f025806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:49.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.913+0000 7f02798d7640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f025806aea0 0x7f025806d360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0264036000 tx=0x7f0264030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:49.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:49.915+0000 7f026affd640 1 -- 172.21.15.70:0/1305839690 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0270057920 con 0x7f027406d4c0 2023-10-08T05:36:50.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.148+0000 7f027c33c640 1 -- 172.21.15.70:0/1305839690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f02741561d0 con 0x7f027406d4c0 2023-10-08T05:36:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.148+0000 7f026affd640 1 -- 172.21.15.70:0/1305839690 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0270017070 con 0x7f027406d4c0 2023-10-08T05:36:50.149 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T05:36:50.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.151+0000 7f027c33c640 1 -- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f025806aea0 msgr2=0x7f025806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:50.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.151+0000 7f027c33c640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f025806aea0 0x7f025806d360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0264036000 tx=0x7f0264030000 comp rx=0 tx=0).stop 2023-10-08T05:36:50.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.151+0000 7f027c33c640 1 -- 172.21.15.70:0/1305839690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 msgr2=0x7f02741f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:50.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.151+0000 7f027c33c640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 0x7f02741f75c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f027000c800 tx=0x7f027000ccc0 comp rx=0 tx=0).stop 2023-10-08T05:36:50.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.151+0000 7f027c33c640 1 -- 172.21.15.70:0/1305839690 shutdown_connections 2023-10-08T05:36:50.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.151+0000 7f027c33c640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f025806aea0 0x7f025806d360 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:50.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.151+0000 7f027c33c640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02740cb150 0x7f02741fa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:50.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.151+0000 7f027c33c640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02740add60 0x7f027415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:50.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.151+0000 7f027c33c640 1 --2- 172.21.15.70:0/1305839690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027406d4c0 0x7f02741f75c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:50.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.151+0000 7f027c33c640 1 -- 172.21.15.70:0/1305839690 >> 172.21.15.70:0/1305839690 conn(0x7f027406a380 msgr2=0x7f027406c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:50.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.152+0000 7f027c33c640 1 -- 172.21.15.70:0/1305839690 shutdown_connections 2023-10-08T05:36:50.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:50.152+0000 7f027c33c640 1 -- 172.21.15.70:0/1305839690 wait complete. 2023-10-08T05:36:50.171 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T05:36:50.171 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:36:50.171 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:36:50.172 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T05:36:50.172 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T05:36:50.172 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T05:36:50.172 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T05:36:50.172 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:36:50.172 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:36:50.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:50.173 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:36:50.223 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:36:50.224 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:36:50.224 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:36:50.224 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:50.224 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:36:50.224 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:50.224 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:50.224 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:50.225 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:36:50.225 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:50.225 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:50.226 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:36:50.226 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:50.227 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:36:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:36:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42386sec preferred_lft 42386sec 2023-10-08T05:36:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:36:50.228 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:50.228 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T05:36:50.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:50.228 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:50.228 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:50.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add name ceph-brx type bridge 2023-10-08T05:36:50.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr flush dev ceph-brx 2023-10-08T05:36:50.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set ceph-brx up 2023-10-08T05:36:50.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T05:36:50.229 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:50.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:50.501 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:36:50.578 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:36:50.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:50.582 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:36:50.634 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:36:50.634 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:36:50.635 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:36:50.635 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:36:50.635 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:36:50.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:50.635 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:50.635 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:50.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:36:50.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:36:50.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:36:50.636 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:50.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:50.814 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:36:50.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:50.864 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2023-10-08T05:36:50.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:50.916 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:50.916 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:50.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T05:36:50.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-08T05:36:50.916 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:51.057 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-08T05:36:51.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:51.057 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:51.057 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:51.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-08T05:36:51.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T05:36:51.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-08T05:36:51.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-08T05:36:51.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-08T05:36:51.058 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:51.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:51.314 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:51.314 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:51.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T05:36:51.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T05:36:51.314 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:51.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:36:51.475 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T05:36:51.528 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:36:51.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:36:51.529 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:36:51.577 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T05:36:51.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:51.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:36:51.771 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:36:51.772 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:36:52.268 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:36:52.268 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:36:52.268 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:36:52.269 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:36:52.269 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:36:52.276 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T05:36:52.276 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:36:52.277 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:36:52.277 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:36:52.277 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:36:52.281 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:36:52.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:52.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:36:52.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:52.374 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:36:52.430 INFO:teuthology.orchestra.run.smithi070.stdout:client4460 2023-10-08T05:36:52.431 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:36:52.431 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:36:52.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:52.485 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:36:52.539 INFO:teuthology.orchestra.run.smithi070.stdout:client4460 2023-10-08T05:36:52.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:52.539 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:36:52.593 INFO:teuthology.orchestra.run.smithi070.stdout:client4460 2023-10-08T05:36:52.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:52.594 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cbcacf35-14e2-4581-b83a-6dd588939bbc.client4460/status') 2023-10-08T05:36:52.687 INFO:teuthology.orchestra.run.smithi070.stdout:instance: client.4460 (3)172.21.15.70:0/3063115230 2023-10-08T05:36:52.688 INFO:teuthology.orchestra.run.smithi070.stdout:blocklisted: false 2023-10-08T05:36:52.688 INFO:teuthology.orchestra.run.smithi070.stderr:0+1 records in 2023-10-08T05:36:52.688 INFO:teuthology.orchestra.run.smithi070.stderr:0+1 records out 2023-10-08T05:36:52.688 INFO:teuthology.orchestra.run.smithi070.stderr:70 bytes copied, 7.1646e-05 s, 977 kB/s 2023-10-08T05:36:52.692 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-08T05:36:52.692 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T05:36:52.692 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-08T05:36:52.692 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:36:52.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.906+0000 7f6dfdec2640 1 Processor -- start 2023-10-08T05:36:52.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.906+0000 7f6dfdec2640 1 -- start start 2023-10-08T05:36:52.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.907+0000 7f6dfdec2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6df8159ac0 0x7f6df806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:52.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.908+0000 7f6dfdec2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df806dee0 con 0x7f6df80ad790 2023-10-08T05:36:52.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.908+0000 7f6dfdec2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df815d850 con 0x7f6df80709e0 2023-10-08T05:36:52.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.908+0000 7f6dfdec2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df815db30 con 0x7f6df8159ac0 2023-10-08T05:36:52.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.908+0000 7f6deffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6df80ad790 0x7f6df80702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42098/0 (socket says 172.21.15.70:42098) 2023-10-08T05:36:52.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.908+0000 7f6deffff640 1 -- 172.21.15.70:0/1773444956 learned_addr learned my addr 172.21.15.70:0/1773444956 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:52.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.908+0000 7f6df77fe640 1 --2- 172.21.15.70:0/1773444956 >> v2:172.21.15.181:3300/0 conn(0x7f6df8159ac0 0x7f6df806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:52.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.908+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11253829 0 0) 0x7f6df806dee0 con 0x7f6df80ad790 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.908+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6de4003520 con 0x7f6df80ad790 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 322874590 0 0) 0x7f6de4003520 con 0x7f6df80ad790 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 >> v2:172.21.15.181:3300/0 conn(0x7f6df8159ac0 msgr2=0x7f6df806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 --2- 172.21.15.70:0/1773444956 >> v2:172.21.15.181:3300/0 conn(0x7f6df8159ac0 0x7f6df806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 >> v1:172.21.15.181:6789/0 conn(0x7f6df80709e0 legacy=0x7f6df80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df815de10 con 0x7f6df80ad790 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6de0001670 con 0x7f6df80ad790 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 >> v1:172.21.15.70:6789/0 conn(0x7f6df80ad790 legacy=0x7f6df80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 --2- 172.21.15.70:0/1773444956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6de4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 --2- 172.21.15.70:0/1773444956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6de40047c0 0x7f6de4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 --2- 172.21.15.70:0/1773444956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6de4005180 0x7f6de400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:52.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df815db30 con 0x7f6de40039c0 2023-10-08T05:36:52.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de400c080 con 0x7f6de40047c0 2023-10-08T05:36:52.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de400c240 con 0x7f6de4005180 2023-10-08T05:36:52.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.909+0000 7f6df77fe640 1 --2- 172.21.15.70:0/1773444956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6de4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:52.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.910+0000 7f6df77fe640 1 -- 172.21.15.70:0/1773444956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6de4005180 msgr2=0x7f6de400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:52.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.910+0000 7f6df77fe640 1 --2- 172.21.15.70:0/1773444956 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6de4005180 0x7f6de400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:52.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.910+0000 7f6df77fe640 1 -- 172.21.15.70:0/1773444956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6de40047c0 msgr2=0x7f6de4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:52.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.910+0000 7f6df77fe640 1 --2- 172.21.15.70:0/1773444956 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6de40047c0 0x7f6de4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:52.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.910+0000 7f6df77fe640 1 -- 172.21.15.70:0/1773444956 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4003520 con 0x7f6de40039c0 2023-10-08T05:36:52.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.910+0000 7f6df7fff640 1 -- 172.21.15.70:0/1773444956 reap_dead start 2023-10-08T05:36:52.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.911+0000 7f6df77fe640 1 --2- 172.21.15.70:0/1773444956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6de4003de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009ac0 tx=0x7f6de40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecc1df4ba572431 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:52.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.911+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de802c6d0 con 0x7f6de40039c0 2023-10-08T05:36:52.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.911+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de802c830 con 0x7f6de40039c0 2023-10-08T05:36:52.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.911+0000 7f6df6ffd640 1 -- 172.21.15.70:0/1773444956 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de802cbc0 con 0x7f6de40039c0 2023-10-08T05:36:52.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.912+0000 7f6dfdec2640 1 -- 172.21.15.70:0/1773444956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 msgr2=0x7f6de4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:52.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.912+0000 7f6dfdec2640 1 --2- 172.21.15.70:0/1773444956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6de4003de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009ac0 tx=0x7f6de40057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:52.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.912+0000 7f6dfdec2640 1 -- 172.21.15.70:0/1773444956 shutdown_connections 2023-10-08T05:36:52.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.912+0000 7f6dfdec2640 1 --2- 172.21.15.70:0/1773444956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6de4003de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:52.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.912+0000 7f6dfdec2640 1 -- 172.21.15.70:0/1773444956 >> 172.21.15.70:0/1773444956 conn(0x7f6df806a380 msgr2=0x7f6df80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:52.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.912+0000 7f6dfdec2640 1 -- 172.21.15.70:0/1773444956 shutdown_connections 2023-10-08T05:36:52.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.912+0000 7f6dfdec2640 1 -- 172.21.15.70:0/1773444956 wait complete. 2023-10-08T05:36:52.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.914+0000 7f6dfdec2640 1 Processor -- start 2023-10-08T05:36:52.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.914+0000 7f6dfdec2640 1 -- start start 2023-10-08T05:36:52.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.915+0000 7f6dfdec2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6df8157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:52.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.915+0000 7f6dfdec2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6df814fc40 0x7f6df8150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:52.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.915+0000 7f6dfdec2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6df8158300 0x7f6df81512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:52.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.915+0000 7f6dfdec2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df80b4340 con 0x7f6de40039c0 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.915+0000 7f6dfdec2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df8159150 con 0x7f6df8158300 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.915+0000 7f6dfdec2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df81588b0 con 0x7f6df814fc40 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.915+0000 7f6df77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6df8157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.915+0000 7f6df77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6df8157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35116/0 (socket says 172.21.15.70:35116) 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.915+0000 7f6df77fe640 1 -- 172.21.15.70:0/3903015684 learned_addr learned my addr 172.21.15.70:0/3903015684 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.916+0000 7f6deffff640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6df814fc40 0x7f6df8150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.916+0000 7f6df7fff640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6df8158300 0x7f6df81512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.916+0000 7f6df77fe640 1 -- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6df814fc40 msgr2=0x7f6df8150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.916+0000 7f6df77fe640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6df814fc40 0x7f6df8150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.916+0000 7f6df77fe640 1 -- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6df8158300 msgr2=0x7f6df81512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:52.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.916+0000 7f6df77fe640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6df8158300 0x7f6df81512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.916+0000 7f6df77fe640 1 -- 172.21.15.70:0/3903015684 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df8158b90 con 0x7f6de40039c0 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.916+0000 7f6deffff640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6df814fc40 0x7f6df8150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.916+0000 7f6df7fff640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6df8158300 0x7f6df81512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.916+0000 7f6df77fe640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6df8157dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009ac0 tx=0x7f6de8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.917+0000 7f6df4ff9640 1 -- 172.21.15.70:0/3903015684 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8003810 con 0x7f6de40039c0 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.917+0000 7f6df4ff9640 1 -- 172.21.15.70:0/3903015684 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de8003970 con 0x7f6de40039c0 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.917+0000 7f6df4ff9640 1 -- 172.21.15.70:0/3903015684 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8003d00 con 0x7f6de40039c0 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.917+0000 7f6dfdec2640 1 -- 172.21.15.70:0/3903015684 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df8158e70 con 0x7f6de40039c0 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.917+0000 7f6dfdec2640 1 -- 172.21.15.70:0/3903015684 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df8150820 con 0x7f6de40039c0 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.918+0000 7f6dfdec2640 1 -- 172.21.15.70:0/3903015684 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df80b1c50 con 0x7f6de40039c0 2023-10-08T05:36:52.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.919+0000 7f6df4ff9640 1 -- 172.21.15.70:0/3903015684 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6de8040450 con 0x7f6de40039c0 2023-10-08T05:36:52.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.920+0000 7f6df4ff9640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6dd806aea0 0x7f6dd806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:52.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.920+0000 7f6df4ff9640 1 -- 172.21.15.70:0/3903015684 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6de80ab790 con 0x7f6de40039c0 2023-10-08T05:36:52.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.921+0000 7f6deffff640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6dd806aea0 0x7f6dd806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:52.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.922+0000 7f6deffff640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6dd806aea0 0x7f6dd806d360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6de00017a0 tx=0x7f6de0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:52.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:52.925+0000 7f6df4ff9640 1 -- 172.21.15.70:0/3903015684 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6df80b1c50 con 0x7f6de40039c0 2023-10-08T05:36:53.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.153+0000 7f6dfdec2640 1 -- 172.21.15.70:0/3903015684 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6df80b1c50 con 0x7f6de40039c0 2023-10-08T05:36:53.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.153+0000 7f6df4ff9640 1 -- 172.21.15.70:0/3903015684 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6df80b1c50 con 0x7f6de40039c0 2023-10-08T05:36:53.154 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T05:36:53.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 -- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6dd806aea0 msgr2=0x7f6dd806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:53.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6dd806aea0 0x7f6dd806d360 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6de00017a0 tx=0x7f6de0009040 comp rx=0 tx=0).stop 2023-10-08T05:36:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 -- 172.21.15.70:0/3903015684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 msgr2=0x7f6df8157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6df8157dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009ac0 tx=0x7f6de8004730 comp rx=0 tx=0).stop 2023-10-08T05:36:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 -- 172.21.15.70:0/3903015684 shutdown_connections 2023-10-08T05:36:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6dd806aea0 0x7f6dd806d360 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6df8158300 0x7f6df81512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6df814fc40 0x7f6df8150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 --2- 172.21.15.70:0/3903015684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6de40039c0 0x7f6df8157dc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 -- 172.21.15.70:0/3903015684 >> 172.21.15.70:0/3903015684 conn(0x7f6df806a380 msgr2=0x7f6df80cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.156+0000 7f6dfdec2640 1 -- 172.21.15.70:0/3903015684 shutdown_connections 2023-10-08T05:36:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:53.157+0000 7f6dfdec2640 1 -- 172.21.15.70:0/3903015684 wait complete. 2023-10-08T05:36:53.175 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T05:36:53.175 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:36:53.175 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:36:53.175 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-08T05:36:53.175 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-08T05:36:53.175 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T05:36:53.175 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T05:36:53.175 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:36:53.175 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-08T05:36:53.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:53.176 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:36:53.230 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:36:53.230 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:36:53.230 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:36:53.231 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:53.231 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:36:53.231 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:53.231 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:53.231 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:53.231 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:36:53.231 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:53.231 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:53.231 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:36:53.231 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42383sec preferred_lft 42383sec 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T05:36:53.232 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:53.233 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T05:36:53.233 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:53.233 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:36:53.233 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T05:36:53.233 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T05:36:53.233 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:53.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:53.234 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:36:53.326 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:36:53.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:53.330 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:36:53.382 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:36:53.382 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:36:53.382 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:36:53.383 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:36:53.383 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:36:53.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:53.383 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:53.383 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:53.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:36:53.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:36:53.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:36:53.383 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:53.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:53.570 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:36:53.619 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T05:36:53.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:53.621 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2023-10-08T05:36:53.671 INFO:teuthology.orchestra.run.smithi070.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-08T05:36:53.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:53.672 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:53.673 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:53.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T05:36:53.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-08T05:36:53.673 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:53.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:53.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-08T05:36:53.912 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:36:53.912 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:36:53.913 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:36:53.913 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:53.913 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:36:53.913 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:53.913 INFO:teuthology.orchestra.run.smithi070.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:36:53.913 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 22:db:de:f3:c9:14 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T05:36:53.913 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T05:36:53.913 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:53.913 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::20db:deff:fef3:c914/64 scope link 2023-10-08T05:36:53.914 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:53.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:53.917 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-08T05:36:54.009 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:36:54.009 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:36:54.009 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:36:54.010 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:54.010 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:36:54.010 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:54.010 INFO:teuthology.orchestra.run.smithi070.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:36:54.010 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 22:db:de:f3:c9:14 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T05:36:54.010 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T05:36:54.010 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:54.010 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::20db:deff:fef3:c914/64 scope link 2023-10-08T05:36:54.010 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:54.013 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-08T05:36:54.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:54.013 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:54.013 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:54.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-08T05:36:54.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-08T05:36:54.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-08T05:36:54.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-08T05:36:54.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-08T05:36:54.014 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:54.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:54.291 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:36:54.292 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:36:54.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.1 up 2023-10-08T05:36:54.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.1 master ceph-brx 2023-10-08T05:36:54.292 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:36:54.466 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:36:54.466 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-08T05:36:54.514 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-08T05:36:54.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:36:54.515 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-08T05:36:54.567 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-08T05:36:54.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:54.567 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:36:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:36:54.708 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:36:54.743 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:36:54.743 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:36:54.743 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:36:54.744 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 2@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:36:54.744 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:36:54.745 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2023-10-08T05:36:54.745 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:36:54.745 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:36:54.745 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:36:54.746 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:36:54.748 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:36:54.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:54.749 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-08T05:36:54.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:54.833 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:36:54.890 INFO:teuthology.orchestra.run.smithi070.stdout:client4857 2023-10-08T05:36:54.890 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:36:54.890 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:36:54.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:54.943 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:36:55.047 INFO:teuthology.orchestra.run.smithi070.stdout:client4857 2023-10-08T05:36:55.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:55.047 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:36:55.102 INFO:teuthology.orchestra.run.smithi070.stdout:client4857 2023-10-08T05:36:55.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:55.103 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cbcacf35-14e2-4581-b83a-6dd588939bbc.client4857/status') 2023-10-08T05:36:55.192 INFO:teuthology.orchestra.run.smithi070.stdout:instance: client.4857 (3)192.168.0.2:0/561479934 2023-10-08T05:36:55.192 INFO:teuthology.orchestra.run.smithi070.stdout:blocklisted: false 2023-10-08T05:36:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:0+1 records in 2023-10-08T05:36:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:0+1 records out 2023-10-08T05:36:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:68 bytes copied, 6.7062e-05 s, 1.0 MB/s 2023-10-08T05:36:55.196 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-08T05:36:55.197 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T05:36:55.197 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T05:36:55.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:36:55.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.413+0000 7f7e324ca640 1 Processor -- start 2023-10-08T05:36:55.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.413+0000 7f7e324ca640 1 -- start start 2023-10-08T05:36:55.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.414+0000 7f7e324ca640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c15a970 0x7f7e2c0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:55.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.414+0000 7f7e324ca640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e2c06dd00 0x7f7e2c0cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:55.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.414+0000 7f7e324ca640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e2c0c9e60 0x7f7e2c0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:55.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.414+0000 7f7e324ca640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c06cfd0 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.414+0000 7f7e324ca640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c06d870 con 0x7f7e2c06dd00 2023-10-08T05:36:55.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.414+0000 7f7e324ca640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c15ad50 con 0x7f7e2c15a970 2023-10-08T05:36:55.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.415+0000 7f7e31cc9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e2c0c9e60 0x7f7e2c0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:55.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.415+0000 7f7e31cc9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e2c0c9e60 0x7f7e2c0ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50024/0 (socket says 172.21.15.70:50024) 2023-10-08T05:36:55.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.415+0000 7f7e31cc9640 1 -- 172.21.15.70:0/1689462575 learned_addr learned my addr 172.21.15.70:0/1689462575 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:55.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.415+0000 7f7e314c8640 1 --2- 172.21.15.70:0/1689462575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c15a970 0x7f7e2c0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:55.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.415+0000 7f7e30cc7640 1 --2- 172.21.15.70:0/1689462575 >> v2:172.21.15.181:3300/0 conn(0x7f7e2c06dd00 0x7f7e2c0cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:55.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.415+0000 7f7e31cc9640 1 -- 172.21.15.70:0/1689462575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c15a970 msgr2=0x7f7e2c0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:55.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.415+0000 7f7e31cc9640 1 --2- 172.21.15.70:0/1689462575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c15a970 0x7f7e2c0cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.415+0000 7f7e31cc9640 1 -- 172.21.15.70:0/1689462575 >> v2:172.21.15.181:3300/0 conn(0x7f7e2c06dd00 msgr2=0x7f7e2c0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:55.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.415+0000 7f7e31cc9640 1 --2- 172.21.15.70:0/1689462575 >> v2:172.21.15.181:3300/0 conn(0x7f7e2c06dd00 0x7f7e2c0cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.415+0000 7f7e31cc9640 1 -- 172.21.15.70:0/1689462575 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0cac20 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.416+0000 7f7e31cc9640 1 --2- 172.21.15.70:0/1689462575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 0x7f7e2c0ca2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c06ae10 tx=0x7f7e2402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799350bb86e70da6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:55.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e23fff640 1 -- 172.21.15.70:0/1689462575 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2402c950 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e23fff640 1 -- 172.21.15.70:0/1689462575 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2402cab0 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e23fff640 1 -- 172.21.15.70:0/1689462575 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2402ce40 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e324ca640 1 -- 172.21.15.70:0/1689462575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 msgr2=0x7f7e2c0ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e324ca640 1 --2- 172.21.15.70:0/1689462575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 0x7f7e2c0ca2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c06ae10 tx=0x7f7e2402bdb0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e324ca640 1 -- 172.21.15.70:0/1689462575 shutdown_connections 2023-10-08T05:36:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e324ca640 1 --2- 172.21.15.70:0/1689462575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 0x7f7e2c0ca2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e324ca640 1 --2- 172.21.15.70:0/1689462575 >> v2:172.21.15.181:3300/0 conn(0x7f7e2c06dd00 0x7f7e2c0cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e324ca640 1 --2- 172.21.15.70:0/1689462575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c15a970 0x7f7e2c0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e324ca640 1 -- 172.21.15.70:0/1689462575 >> 172.21.15.70:0/1689462575 conn(0x7f7e2c0ce9d0 msgr2=0x7f7e2c0cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e324ca640 1 -- 172.21.15.70:0/1689462575 shutdown_connections 2023-10-08T05:36:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.417+0000 7f7e324ca640 1 -- 172.21.15.70:0/1689462575 wait complete. 2023-10-08T05:36:55.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.419+0000 7f7e324ca640 1 Processor -- start 2023-10-08T05:36:55.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.419+0000 7f7e324ca640 1 -- start start 2023-10-08T05:36:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.420+0000 7f7e324ca640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e2c06dd00 0x7f7e2c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.420+0000 7f7e324ca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 0x7f7e2c1f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.420+0000 7f7e324ca640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c15a970 0x7f7e2c154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.420+0000 7f7e324ca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c15d4d0 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.420+0000 7f7e324ca640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c15c900 con 0x7f7e2c06dd00 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.420+0000 7f7e324ca640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c15cba0 con 0x7f7e2c15a970 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.420+0000 7f7e30cc7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 0x7f7e2c1f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e30cc7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 0x7f7e2c1f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50030/0 (socket says 172.21.15.70:50030) 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e30cc7640 1 -- 172.21.15.70:0/3834747712 learned_addr learned my addr 172.21.15.70:0/3834747712 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e314c8640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e2c06dd00 0x7f7e2c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e31cc9640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c15a970 0x7f7e2c154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e30cc7640 1 -- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c15a970 msgr2=0x7f7e2c154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e30cc7640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c15a970 0x7f7e2c154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e30cc7640 1 -- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e2c06dd00 msgr2=0x7f7e2c15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e30cc7640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e2c06dd00 0x7f7e2c15d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e30cc7640 1 -- 172.21.15.70:0/3834747712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e24009b30 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e30cc7640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 0x7f7e2c1f2790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00c7c0 tx=0x7f7e1c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e21ffb640 1 -- 172.21.15.70:0/3834747712 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c007820 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e21ffb640 1 -- 172.21.15.70:0/3834747712 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1c007980 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.421+0000 7f7e324ca640 1 -- 172.21.15.70:0/3834747712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c15ce80 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.422+0000 7f7e21ffb640 1 -- 172.21.15.70:0/3834747712 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c007cf0 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.422+0000 7f7e324ca640 1 -- 172.21.15.70:0/3834747712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c1f2cd0 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.423+0000 7f7e324ca640 1 -- 172.21.15.70:0/3834747712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c15d640 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.424+0000 7f7e21ffb640 1 -- 172.21.15.70:0/3834747712 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7e1c0113f0 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.425+0000 7f7e21ffb640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7e1006aea0 0x7f7e1006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.425+0000 7f7e21ffb640 1 -- 172.21.15.70:0/3834747712 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7e1c089660 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.425+0000 7f7e314c8640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7e1006aea0 0x7f7e1006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:55.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.426+0000 7f7e314c8640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7e1006aea0 0x7f7e1006d360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800a8f0 tx=0x7f7e28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:55.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.429+0000 7f7e21ffb640 1 -- 172.21.15.70:0/3834747712 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e1c054380 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.657+0000 7f7e324ca640 1 -- 172.21.15.70:0/3834747712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7e2c04e160 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.658+0000 7f7e21ffb640 1 -- 172.21.15.70:0/3834747712 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7e1c0579a0 con 0x7f7e2c0c9e60 2023-10-08T05:36:55.659 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T05:36:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.660+0000 7f7e324ca640 1 -- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7e1006aea0 msgr2=0x7f7e1006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.660+0000 7f7e324ca640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7e1006aea0 0x7f7e1006d360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800a8f0 tx=0x7f7e28008040 comp rx=0 tx=0).stop 2023-10-08T05:36:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.660+0000 7f7e324ca640 1 -- 172.21.15.70:0/3834747712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 msgr2=0x7f7e2c1f2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.660+0000 7f7e324ca640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 0x7f7e2c1f2790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00c7c0 tx=0x7f7e1c00cc80 comp rx=0 tx=0).stop 2023-10-08T05:36:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.661+0000 7f7e324ca640 1 -- 172.21.15.70:0/3834747712 shutdown_connections 2023-10-08T05:36:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.661+0000 7f7e324ca640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7e1006aea0 0x7f7e1006d360 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.661+0000 7f7e324ca640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e2c15a970 0x7f7e2c154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.661+0000 7f7e324ca640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e2c0c9e60 0x7f7e2c1f2790 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.661+0000 7f7e324ca640 1 --2- 172.21.15.70:0/3834747712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e2c06dd00 0x7f7e2c15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.661+0000 7f7e324ca640 1 -- 172.21.15.70:0/3834747712 >> 172.21.15.70:0/3834747712 conn(0x7f7e2c0ce9d0 msgr2=0x7f7e2c0b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.661+0000 7f7e324ca640 1 -- 172.21.15.70:0/3834747712 shutdown_connections 2023-10-08T05:36:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:55.661+0000 7f7e324ca640 1 -- 172.21.15.70:0/3834747712 wait complete. 2023-10-08T05:36:55.678 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T05:36:55.678 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:36:55.678 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-08T05:36:55.678 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T05:36:55.678 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T05:36:55.678 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T05:36:55.678 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T05:36:55.678 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:36:55.679 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T05:36:55.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:55.679 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-08T05:36:55.739 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:36:55.739 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:36:55.739 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:36:55.739 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:55.739 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-08T05:36:55.739 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:55.739 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:36:55.740 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42361sec preferred_lft 42361sec 2023-10-08T05:36:55.741 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-08T05:36:55.741 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:55.742 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T05:36:55.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:55.742 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-08T05:36:55.742 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-08T05:36:55.742 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add name ceph-brx type bridge 2023-10-08T05:36:55.742 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr flush dev ceph-brx 2023-10-08T05:36:55.742 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set ceph-brx up 2023-10-08T05:36:55.742 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T05:36:55.742 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-08T05:36:56.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:56.202 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:36:56.284 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-08T05:36:56.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:56.288 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-08T05:36:56.345 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-08T05:36:56.345 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:36:56.346 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:36:56.346 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:36:56.346 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:36:56.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:56.346 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-08T05:36:56.347 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-08T05:36:56.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:36:56.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:36:56.347 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:36:56.347 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-08T05:36:57.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:57.087 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-08T05:36:57.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:57.142 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2023-10-08T05:36:57.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:57.198 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-08T05:36:57.198 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-08T05:36:57.198 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T05:36:57.198 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-08T05:36:57.198 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-08T05:36:57.340 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-08T05:36:57.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:57.340 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-08T05:36:57.340 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-08T05:36:57.340 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-08T05:36:57.340 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T05:36:57.341 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-08T05:36:57.341 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-08T05:36:57.341 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-08T05:36:57.341 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-08T05:36:57.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:57.643 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-08T05:36:57.643 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-08T05:36:57.643 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.0 up 2023-10-08T05:36:57.644 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T05:36:57.644 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-08T05:36:57.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:36:57.815 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T05:36:57.861 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T05:36:57.862 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:36:57.862 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T05:36:57.911 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T05:36:57.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:57.911 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:36:58.051 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:36:58.051 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:36:58.476 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:36:58.477 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:36:58.477 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:36:58.477 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 1@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:36:58.477 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:36:58.480 INFO:teuthology.orchestra.run.smithi181.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-10-08T05:36:58.480 INFO:teuthology.orchestra.run.smithi181.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:36:58.481 INFO:teuthology.orchestra.run.smithi181.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:36:58.481 INFO:teuthology.orchestra.run.smithi181.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:36:58.481 INFO:teuthology.orchestra.run.smithi181.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:36:58.484 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:36:58.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:58.485 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T05:36:58.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:58.575 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:36:58.627 INFO:teuthology.orchestra.run.smithi181.stdout:client4603 2023-10-08T05:36:58.628 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:36:58.628 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:36:58.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:58.676 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:36:58.778 INFO:teuthology.orchestra.run.smithi181.stdout:client4603 2023-10-08T05:36:58.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:58.778 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:36:58.836 INFO:teuthology.orchestra.run.smithi181.stdout:client4603 2023-10-08T05:36:58.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:58.837 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cbcacf35-14e2-4581-b83a-6dd588939bbc.client4603/status') 2023-10-08T05:36:58.928 INFO:teuthology.orchestra.run.smithi181.stdout:instance: client.4603 (3)192.168.0.1:0/2994065581 2023-10-08T05:36:58.929 INFO:teuthology.orchestra.run.smithi181.stdout:blocklisted: false 2023-10-08T05:36:58.929 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2023-10-08T05:36:58.929 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2023-10-08T05:36:58.929 INFO:teuthology.orchestra.run.smithi181.stderr:69 bytes copied, 6.4616e-05 s, 1.1 MB/s 2023-10-08T05:36:58.932 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-08T05:36:58.933 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-08T05:36:58.933 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-08T05:36:58.933 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:36:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.162+0000 7f7ee8e4f640 1 Processor -- start 2023-10-08T05:36:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.163+0000 7f7ee8e4f640 1 -- start start 2023-10-08T05:36:59.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.164+0000 7f7ee8e4f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ee4157000 0x7f7ee40cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:59.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.164+0000 7f7ee8e4f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee40adfc0 con 0x7f7ee40cb190 2023-10-08T05:36:59.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.164+0000 7f7ee8e4f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee40ccce0 con 0x7f7ee41597d0 2023-10-08T05:36:59.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.164+0000 7f7ee8e4f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee4159cb0 con 0x7f7ee4157000 2023-10-08T05:36:59.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.165+0000 7f7ee2d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7ee40cb190 0x7f7ee415d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59316/0 (socket says 172.21.15.70:59316) 2023-10-08T05:36:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.165+0000 7f7ee2d9d640 1 -- 172.21.15.70:0/2748855037 learned_addr learned my addr 172.21.15.70:0/2748855037 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:36:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.165+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695843131 0 0) 0x7f7ee40adfc0 con 0x7f7ee40cb190 2023-10-08T05:36:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.165+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ecc003520 con 0x7f7ee40cb190 2023-10-08T05:36:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.165+0000 7f7ee259c640 1 --2- 172.21.15.70:0/2748855037 >> v2:172.21.15.181:3300/0 conn(0x7f7ee4157000 0x7f7ee40cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1262404173 0 0) 0x7f7ecc003520 con 0x7f7ee40cb190 2023-10-08T05:36:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 >> v2:172.21.15.181:3300/0 conn(0x7f7ee4157000 msgr2=0x7f7ee40cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 --2- 172.21.15.70:0/2748855037 >> v2:172.21.15.181:3300/0 conn(0x7f7ee4157000 0x7f7ee40cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 >> v1:172.21.15.181:6789/0 conn(0x7f7ee41597d0 legacy=0x7f7ee406ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee4159f90 con 0x7f7ee40cb190 2023-10-08T05:36:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7edc001670 con 0x7f7ee40cb190 2023-10-08T05:36:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 >> v1:172.21.15.70:6789/0 conn(0x7f7ee40cb190 legacy=0x7f7ee415d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 --2- 172.21.15.70:0/2748855037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ecc0039c0 0x7f7ecc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 --2- 172.21.15.70:0/2748855037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc0047c0 0x7f7ecc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 --2- 172.21.15.70:0/2748855037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ecc0051d0 0x7f7ecc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee4159cb0 con 0x7f7ecc0047c0 2023-10-08T05:36:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc00c120 con 0x7f7ecc0051d0 2023-10-08T05:36:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc00c2e0 con 0x7f7ecc0039c0 2023-10-08T05:36:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee1d9b640 1 --2- 172.21.15.70:0/2748855037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc0047c0 0x7f7ecc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee259c640 1 --2- 172.21.15.70:0/2748855037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ecc0039c0 0x7f7ecc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee1d9b640 1 -- 172.21.15.70:0/2748855037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ecc0039c0 msgr2=0x7f7ecc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee1d9b640 1 --2- 172.21.15.70:0/2748855037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ecc0039c0 0x7f7ecc003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee1d9b640 1 -- 172.21.15.70:0/2748855037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ecc0051d0 msgr2=0x7f7ecc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:36:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.166+0000 7f7ee1d9b640 1 --2- 172.21.15.70:0/2748855037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ecc0051d0 0x7f7ecc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.167+0000 7f7ee1d9b640 1 -- 172.21.15.70:0/2748855037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc003520 con 0x7f7ecc0047c0 2023-10-08T05:36:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.167+0000 7f7ee2d9d640 1 -- 172.21.15.70:0/2748855037 reap_dead start 2023-10-08T05:36:59.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.168+0000 7f7ee1d9b640 1 --2- 172.21.15.70:0/2748855037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc0047c0 0x7f7ecc004c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8001750 tx=0x7f7ecc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9079e10221617051 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.168+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80052d0 con 0x7f7ecc0047c0 2023-10-08T05:36:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.168+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed802e050 con 0x7f7ecc0047c0 2023-10-08T05:36:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.168+0000 7f7ee159a640 1 -- 172.21.15.70:0/2748855037 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8003830 con 0x7f7ecc0047c0 2023-10-08T05:36:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.169+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/2748855037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc0047c0 msgr2=0x7f7ecc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.169+0000 7f7ee8e4f640 1 --2- 172.21.15.70:0/2748855037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc0047c0 0x7f7ecc004c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8001750 tx=0x7f7ecc0057f0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.169+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/2748855037 shutdown_connections 2023-10-08T05:36:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.169+0000 7f7ee8e4f640 1 --2- 172.21.15.70:0/2748855037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc0047c0 0x7f7ecc004c40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.169+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/2748855037 >> 172.21.15.70:0/2748855037 conn(0x7f7ee406a380 msgr2=0x7f7ee40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:59.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.169+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/2748855037 shutdown_connections 2023-10-08T05:36:59.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.169+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/2748855037 wait complete. 2023-10-08T05:36:59.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.171+0000 7f7ee8e4f640 1 Processor -- start 2023-10-08T05:36:59.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.171+0000 7f7ee8e4f640 1 -- start start 2023-10-08T05:36:59.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.172+0000 7f7ee8e4f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ecc0047c0 0x7f7ee41f6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.172+0000 7f7ee8e4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ee4153d30 0x7f7ee414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.172+0000 7f7ee8e4f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ee41527a0 0x7f7ee4152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.172+0000 7f7ee8e4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee4153190 con 0x7f7ee4153d30 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.172+0000 7f7ee8e4f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee41f6d90 con 0x7f7ecc0047c0 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.172+0000 7f7ee8e4f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee4154ee0 con 0x7f7ee41527a0 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.172+0000 7f7ee1d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ee4153d30 0x7f7ee414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.172+0000 7f7ee1d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ee4153d30 0x7f7ee414f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50052/0 (socket says 172.21.15.70:50052) 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.172+0000 7f7ee1d9b640 1 -- 172.21.15.70:0/1311227165 learned_addr learned my addr 172.21.15.70:0/1311227165 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.172+0000 7f7ee2d9d640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ee41527a0 0x7f7ee4152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ee259c640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ecc0047c0 0x7f7ee41f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ee1d9b640 1 -- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ee41527a0 msgr2=0x7f7ee4152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ee1d9b640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ee41527a0 0x7f7ee4152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ee1d9b640 1 -- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ecc0047c0 msgr2=0x7f7ee41f6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ee1d9b640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ecc0047c0 0x7f7ee41f6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ee1d9b640 1 -- 172.21.15.70:0/1311227165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee41541f0 con 0x7f7ee4153d30 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ee259c640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ecc0047c0 0x7f7ee41f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ee1d9b640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ee4153d30 0x7f7ee414f080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8005040 tx=0x7f7ed8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ed2ffd640 1 -- 172.21.15.70:0/1311227165 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80076c0 con 0x7f7ee4153d30 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ed2ffd640 1 -- 172.21.15.70:0/1311227165 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed8007a30 con 0x7f7ee4153d30 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/1311227165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee4154640 con 0x7f7ee4153d30 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.173+0000 7f7ed2ffd640 1 -- 172.21.15.70:0/1311227165 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8006be0 con 0x7f7ee4153d30 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.174+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/1311227165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee4154c00 con 0x7f7ee4153d30 2023-10-08T05:36:59.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.175+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/1311227165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee4067490 con 0x7f7ee4153d30 2023-10-08T05:36:59.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.176+0000 7f7ed2ffd640 1 -- 172.21.15.70:0/1311227165 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ed8006d40 con 0x7f7ee4153d30 2023-10-08T05:36:59.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.177+0000 7f7ed2ffd640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7ec406ae70 0x7f7ec406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:36:59.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.177+0000 7f7ed2ffd640 1 -- 172.21.15.70:0/1311227165 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7ed80ab5a0 con 0x7f7ee4153d30 2023-10-08T05:36:59.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.177+0000 7f7ee259c640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7ec406ae70 0x7f7ec406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:36:59.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.179+0000 7f7ee259c640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7ec406ae70 0x7f7ec406d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400a470 tx=0x7f7ed400a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:36:59.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.181+0000 7f7ed2ffd640 1 -- 172.21.15.70:0/1311227165 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ed8076200 con 0x7f7ee4153d30 2023-10-08T05:36:59.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.412+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/1311227165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7ee415a240 con 0x7f7ee4153d30 2023-10-08T05:36:59.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.412+0000 7f7ed2ffd640 1 -- 172.21.15.70:0/1311227165 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7ed8041070 con 0x7f7ee4153d30 2023-10-08T05:36:59.413 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-08T05:36:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7ec406ae70 msgr2=0x7f7ec406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7ec406ae70 0x7f7ec406d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400a470 tx=0x7f7ed400a040 comp rx=0 tx=0).stop 2023-10-08T05:36:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/1311227165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ee4153d30 msgr2=0x7f7ee414f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:36:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ee4153d30 0x7f7ee414f080 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8005040 tx=0x7f7ed8001120 comp rx=0 tx=0).stop 2023-10-08T05:36:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/1311227165 shutdown_connections 2023-10-08T05:36:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7ec406ae70 0x7f7ec406d330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ee41527a0 0x7f7ee4152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ee4153d30 0x7f7ee414f080 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 --2- 172.21.15.70:0/1311227165 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ecc0047c0 0x7f7ee41f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:36:59.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/1311227165 >> 172.21.15.70:0/1311227165 conn(0x7f7ee406a380 msgr2=0x7f7ee406d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:36:59.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/1311227165 shutdown_connections 2023-10-08T05:36:59.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:36:59.415+0000 7f7ee8e4f640 1 -- 172.21.15.70:0/1311227165 wait complete. 2023-10-08T05:36:59.433 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T05:36:59.433 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:36:59.433 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi181.front.sepia.ceph.com 2023-10-08T05:36:59.433 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-08T05:36:59.433 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-08T05:36:59.434 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T05:36:59.434 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T05:36:59.434 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:36:59.434 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-08T05:36:59.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:59.434 DEBUG:teuthology.orchestra.run.smithi181:> ip addr 2023-10-08T05:36:59.495 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:36:59.496 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:36:59.496 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:36:59.496 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:59.497 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-08T05:36:59.497 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:59.497 INFO:teuthology.orchestra.run.smithi181.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:59.497 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:59.498 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f0 2023-10-08T05:36:59.498 INFO:teuthology.orchestra.run.smithi181.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:59.498 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:d9:39:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:59.498 INFO:teuthology.orchestra.run.smithi181.stdout: altname enp7s0f1 2023-10-08T05:36:59.498 INFO:teuthology.orchestra.run.smithi181.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:36:59.498 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9c brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:59.498 INFO:teuthology.orchestra.run.smithi181.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:36:59.498 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether 0c:c4:7a:88:7b:9d brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:59.498 INFO:teuthology.orchestra.run.smithi181.stdout: inet 172.21.15.181/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft 42357sec preferred_lft 42357sec 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::ec4:7aff:fe88:7b9d/64 scope link 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether ea:99:5e:8b:7f:09 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::984b:38ff:fe3b:4648/64 scope link 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:36:59.499 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether ea:99:5e:8b:7f:09 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T05:36:59.500 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::e899:5eff:fe8b:7f09/64 scope link 2023-10-08T05:36:59.500 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:36:59.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:59.500 DEBUG:teuthology.orchestra.run.smithi181:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:36:59.593 INFO:teuthology.orchestra.run.smithi181.stdout:1 2023-10-08T05:36:59.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:59.597 DEBUG:teuthology.orchestra.run.smithi181:> route 2023-10-08T05:36:59.646 INFO:teuthology.orchestra.run.smithi181.stdout:Kernel IP routing table 2023-10-08T05:36:59.647 INFO:teuthology.orchestra.run.smithi181.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:36:59.647 INFO:teuthology.orchestra.run.smithi181.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:36:59.647 INFO:teuthology.orchestra.run.smithi181.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:36:59.647 INFO:teuthology.orchestra.run.smithi181.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:36:59.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:59.648 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-08T05:36:59.648 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-08T05:36:59.648 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:36:59.648 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:36:59.648 DEBUG:teuthology.orchestra.run.smithi181:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:36:59.648 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-08T05:36:59.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:59.831 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-08T05:36:59.883 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T05:36:59.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:59.884 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list-id 2023-10-08T05:36:59.932 INFO:teuthology.orchestra.run.smithi181.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-08T05:36:59.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:36:59.934 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-08T05:36:59.934 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-08T05:36:59.934 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T05:36:59.934 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-08T05:36:59.934 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-08T05:37:00.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:00.068 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-08T05:37:00.156 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether be:4f:fa:55:1b:d9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::bc4f:faff:fe55:1bd9/64 scope link 2023-10-08T05:37:00.157 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:00.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:00.160 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-08T05:37:00.248 INFO:teuthology.orchestra.run.smithi181.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:37:00.248 INFO:teuthology.orchestra.run.smithi181.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:37:00.249 INFO:teuthology.orchestra.run.smithi181.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:37:00.249 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:00.249 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 ::1/128 scope host 2023-10-08T05:37:00.249 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:00.249 INFO:teuthology.orchestra.run.smithi181.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:37:00.249 INFO:teuthology.orchestra.run.smithi181.stdout: link/ether be:4f:fa:55:1b:d9 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T05:37:00.249 INFO:teuthology.orchestra.run.smithi181.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T05:37:00.249 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:00.249 INFO:teuthology.orchestra.run.smithi181.stdout: inet6 fe80::bc4f:faff:fe55:1bd9/64 scope link 2023-10-08T05:37:00.249 INFO:teuthology.orchestra.run.smithi181.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:00.252 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-08T05:37:00.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:00.252 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-08T05:37:00.252 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-08T05:37:00.253 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-08T05:37:00.253 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-08T05:37:00.253 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-08T05:37:00.253 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-08T05:37:00.253 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-08T05:37:00.253 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-08T05:37:00.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:00.525 DEBUG:teuthology.orchestra.run.smithi181:> (cd / && exec bash -c ' 2023-10-08T05:37:00.525 DEBUG:teuthology.orchestra.run.smithi181:> set -e 2023-10-08T05:37:00.525 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set brx.1 up 2023-10-08T05:37:00.525 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link set dev brx.1 master ceph-brx 2023-10-08T05:37:00.525 DEBUG:teuthology.orchestra.run.smithi181:> ') 2023-10-08T05:37:00.697 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:37:00.697 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-08T05:37:00.744 INFO:teuthology.orchestra.run.smithi181.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-08T05:37:00.744 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:37:00.744 DEBUG:teuthology.orchestra.run.smithi181:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-08T05:37:00.843 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-08T05:37:00.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:00.843 DEBUG:teuthology.orchestra.run.smithi181:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:37:00.985 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:37:00.985 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:37:01.020 INFO:teuthology.orchestra.run.smithi181.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:37:01.020 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:37:01.020 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:37:01.020 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: trying mount with new device syntax: 3@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:37:01.020 INFO:teuthology.orchestra.run.smithi181.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:37:01.021 INFO:teuthology.orchestra.run.smithi181.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2023-10-08T05:37:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:37:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:37:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:37:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:37:01.027 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:37:01.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:01.027 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-08T05:37:01.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:01.108 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:37:01.162 INFO:teuthology.orchestra.run.smithi181.stdout:client4612 2023-10-08T05:37:01.162 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:37:01.162 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:01.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:01.212 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:37:01.316 INFO:teuthology.orchestra.run.smithi181.stdout:client4612 2023-10-08T05:37:01.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:01.317 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:37:01.371 INFO:teuthology.orchestra.run.smithi181.stdout:client4612 2023-10-08T05:37:01.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:01.372 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/cbcacf35-14e2-4581-b83a-6dd588939bbc.client4612/status') 2023-10-08T05:37:01.454 INFO:teuthology.orchestra.run.smithi181.stdout:instance: client.4612 (3)192.168.0.2:0/281098194 2023-10-08T05:37:01.454 INFO:teuthology.orchestra.run.smithi181.stdout:blocklisted: false 2023-10-08T05:37:01.454 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records in 2023-10-08T05:37:01.454 INFO:teuthology.orchestra.run.smithi181.stderr:0+1 records out 2023-10-08T05:37:01.454 INFO:teuthology.orchestra.run.smithi181.stderr:68 bytes copied, 8.4655e-05 s, 803 kB/s 2023-10-08T05:37:01.457 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-08T05:37:01.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:01.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.696+0000 7fa10b24f640 1 Processor -- start 2023-10-08T05:37:01.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.697+0000 7fa10b24f640 1 -- start start 2023-10-08T05:37:01.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.698+0000 7fa10b24f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1040709e0 0x7fa104164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:01.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.698+0000 7fa10b24f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10406dee0 con 0x7fa104159ac0 2023-10-08T05:37:01.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.698+0000 7fa10b24f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10415d850 con 0x7fa1040ad790 2023-10-08T05:37:01.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.698+0000 7fa10b24f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10406dcf0 con 0x7fa1040709e0 2023-10-08T05:37:01.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.698+0000 7fa1097ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1040709e0 0x7fa104164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:01.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.699+0000 7fa1097ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1040709e0 0x7fa104164a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50064/0 (socket says 172.21.15.70:50064) 2023-10-08T05:37:01.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.699+0000 7fa108feb640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa104159ac0 0x7fa10406d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59318/0 (socket says 172.21.15.70:59318) 2023-10-08T05:37:01.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.699+0000 7fa1097ec640 1 -- 172.21.15.70:0/82257867 learned_addr learned my addr 172.21.15.70:0/82257867 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:01.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.699+0000 7fa1097ec640 1 -- 172.21.15.70:0/82257867 >> v1:172.21.15.181:6789/0 conn(0x7fa1040ad790 legacy=0x7fa1040702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:01.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.699+0000 7fa1097ec640 1 -- 172.21.15.70:0/82257867 >> v1:172.21.15.70:6789/0 conn(0x7fa104159ac0 legacy=0x7fa10406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:01.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.699+0000 7fa1037fe640 1 -- 172.21.15.70:0/82257867 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198133780 0 0) 0x7fa10406dee0 con 0x7fa104159ac0 2023-10-08T05:37:01.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.699+0000 7fa1097ec640 1 -- 172.21.15.70:0/82257867 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10415db30 con 0x7fa1040709e0 2023-10-08T05:37:01.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.700+0000 7fa1097ec640 1 --2- 172.21.15.70:0/82257867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 0x7fa104164a60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009e50 tx=0x7fa0fc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1040b18bd09aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:01.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.700+0000 7fa1037fe640 1 -- 172.21.15.70:0/82257867 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc02cb20 con 0x7fa1040709e0 2023-10-08T05:37:01.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.700+0000 7fa1037fe640 1 -- 172.21.15.70:0/82257867 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc02cc80 con 0x7fa1040709e0 2023-10-08T05:37:01.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.700+0000 7fa1037fe640 1 -- 172.21.15.70:0/82257867 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0343f0 con 0x7fa1040709e0 2023-10-08T05:37:01.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.701+0000 7fa10b24f640 1 -- 172.21.15.70:0/82257867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 msgr2=0x7fa104164a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:01.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.701+0000 7fa10b24f640 1 --2- 172.21.15.70:0/82257867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 0x7fa104164a60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009e50 tx=0x7fa0fc02bf80 comp rx=0 tx=0).stop 2023-10-08T05:37:01.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.701+0000 7fa10b24f640 1 -- 172.21.15.70:0/82257867 shutdown_connections 2023-10-08T05:37:01.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.701+0000 7fa10b24f640 1 --2- 172.21.15.70:0/82257867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 0x7fa104164a60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:01.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.701+0000 7fa10b24f640 1 -- 172.21.15.70:0/82257867 >> 172.21.15.70:0/82257867 conn(0x7fa10406a380 msgr2=0x7fa1040b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:01.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.701+0000 7fa10b24f640 1 -- 172.21.15.70:0/82257867 shutdown_connections 2023-10-08T05:37:01.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.701+0000 7fa10b24f640 1 -- 172.21.15.70:0/82257867 wait complete. 2023-10-08T05:37:01.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.703+0000 7fa10b24f640 1 Processor -- start 2023-10-08T05:37:01.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.704+0000 7fa10b24f640 1 -- start start 2023-10-08T05:37:01.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.705+0000 7fa10b24f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 0x7fa104150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:01.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.705+0000 7fa10b24f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1040ad790 0x7fa10415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:01.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.705+0000 7fa10b24f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa104159ac0 0x7fa1041519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:01.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.705+0000 7fa10b24f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040b42d0 con 0x7fa1040709e0 2023-10-08T05:37:01.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.705+0000 7fa10b24f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104150890 con 0x7fa104159ac0 2023-10-08T05:37:01.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.705+0000 7fa10b24f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040b3ff0 con 0x7fa1040ad790 2023-10-08T05:37:01.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.705+0000 7fa108feb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 0x7fa104150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:01.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.705+0000 7fa108feb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 0x7fa104150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50072/0 (socket says 172.21.15.70:50072) 2023-10-08T05:37:01.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.705+0000 7fa108feb640 1 -- 172.21.15.70:0/3406769239 learned_addr learned my addr 172.21.15.70:0/3406769239 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:01.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa108feb640 1 -- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1040ad790 msgr2=0x7fa10415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:01.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa108feb640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1040ad790 0x7fa10415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:01.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa108feb640 1 -- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa104159ac0 msgr2=0x7fa1041519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:01.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa108feb640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa104159ac0 0x7fa1041519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:01.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa108feb640 1 -- 172.21.15.70:0/3406769239 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc009b30 con 0x7fa1040709e0 2023-10-08T05:37:01.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa108feb640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 0x7fa104150350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800c430 tx=0x7fa0f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:01.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa1017fa640 1 -- 172.21.15.70:0/3406769239 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80074e0 con 0x7fa1040709e0 2023-10-08T05:37:01.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa1017fa640 1 -- 172.21.15.70:0/3406769239 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f8007660 con 0x7fa1040709e0 2023-10-08T05:37:01.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa1017fa640 1 -- 172.21.15.70:0/3406769239 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8004a20 con 0x7fa1040709e0 2023-10-08T05:37:01.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa10b24f640 1 -- 172.21.15.70:0/3406769239 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa104150ed0 con 0x7fa1040709e0 2023-10-08T05:37:01.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.706+0000 7fa10b24f640 1 -- 172.21.15.70:0/3406769239 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa104151490 con 0x7fa1040709e0 2023-10-08T05:37:01.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.707+0000 7fa10b24f640 1 -- 172.21.15.70:0/3406769239 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa104067490 con 0x7fa1040709e0 2023-10-08T05:37:01.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.708+0000 7fa1017fa640 1 -- 172.21.15.70:0/3406769239 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa0f8004b80 con 0x7fa1040709e0 2023-10-08T05:37:01.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.709+0000 7fa1017fa640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa0e806ae70 0x7fa0e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:01.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.709+0000 7fa1017fa640 1 -- 172.21.15.70:0/3406769239 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa0f8089870 con 0x7fa1040709e0 2023-10-08T05:37:01.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.710+0000 7fa103fff640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa0e806ae70 0x7fa0e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:01.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.711+0000 7fa103fff640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa0e806ae70 0x7fa0e806d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa1040b44a0 tx=0x7fa0f00079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:01.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.714+0000 7fa1017fa640 1 -- 172.21.15.70:0/3406769239 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0f8054550 con 0x7fa1040709e0 2023-10-08T05:37:01.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.905+0000 7fa10b24f640 1 -- 172.21.15.70:0/3406769239 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1040b0ff0 con 0x7fa1040709e0 2023-10-08T05:37:01.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.906+0000 7fa1017fa640 1 -- 172.21.15.70:0/3406769239 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7fa0f8057b70 con 0x7fa1040709e0 2023-10-08T05:37:01.907 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:01.908 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:01.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 -- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa0e806ae70 msgr2=0x7fa0e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:01.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa0e806ae70 0x7fa0e806d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa1040b44a0 tx=0x7fa0f00079b0 comp rx=0 tx=0).stop 2023-10-08T05:37:01.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 -- 172.21.15.70:0/3406769239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 msgr2=0x7fa104150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:01.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 0x7fa104150350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800c430 tx=0x7fa0f800c8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:01.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 -- 172.21.15.70:0/3406769239 shutdown_connections 2023-10-08T05:37:01.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa0e806ae70 0x7fa0e806d330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:01.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa104159ac0 0x7fa1041519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:01.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1040ad790 0x7fa10415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:01.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 --2- 172.21.15.70:0/3406769239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1040709e0 0x7fa104150350 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:01.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 -- 172.21.15.70:0/3406769239 >> 172.21.15.70:0/3406769239 conn(0x7fa10406a380 msgr2=0x7fa104155880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:01.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 -- 172.21.15.70:0/3406769239 shutdown_connections 2023-10-08T05:37:01.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:01.909+0000 7fa10b24f640 1 -- 172.21.15.70:0/3406769239 wait complete. 2023-10-08T05:37:01.928 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:37:01.928 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:02.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.141+0000 7fb68beb4640 1 Processor -- start 2023-10-08T05:37:02.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.141+0000 7fb68beb4640 1 -- start start 2023-10-08T05:37:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.143+0000 7fb68beb4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb68415aad0 0x7fb6840b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.143+0000 7fb68beb4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68415bf30 con 0x7fb6840c82b0 2023-10-08T05:37:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.143+0000 7fb68beb4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68415b320 con 0x7fb68406d410 2023-10-08T05:37:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.143+0000 7fb68beb4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68415b600 con 0x7fb68415aad0 2023-10-08T05:37:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.144+0000 7fb689c50640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb68415aad0 0x7fb6840b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.144+0000 7fb689c50640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb68415aad0 0x7fb6840b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50080/0 (socket says 172.21.15.70:50080) 2023-10-08T05:37:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.144+0000 7fb68944f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb6840c82b0 0x7fb68406cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59320/0 (socket says 172.21.15.70:59320) 2023-10-08T05:37:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.144+0000 7fb689c50640 1 -- 172.21.15.70:0/1796309966 learned_addr learned my addr 172.21.15.70:0/1796309966 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:02.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.144+0000 7fb689c50640 1 -- 172.21.15.70:0/1796309966 >> v1:172.21.15.181:6789/0 conn(0x7fb68406d410 legacy=0x7fb684070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:02.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.144+0000 7fb689c50640 1 -- 172.21.15.70:0/1796309966 >> v1:172.21.15.70:6789/0 conn(0x7fb6840c82b0 legacy=0x7fb68406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.144+0000 7fb689c50640 1 -- 172.21.15.70:0/1796309966 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68415b8e0 con 0x7fb68415aad0 2023-10-08T05:37:02.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.145+0000 7fb689c50640 1 --2- 172.21.15.70:0/1796309966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb68415aad0 0x7fb6840b1750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb680009870 tx=0x7fb68002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22b51caab6307fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:02.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.148+0000 7fb688c4e640 1 -- 172.21.15.70:0/1796309966 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680030850 con 0x7fb68415aad0 2023-10-08T05:37:02.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.148+0000 7fb688c4e640 1 -- 172.21.15.70:0/1796309966 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6800309b0 con 0x7fb68415aad0 2023-10-08T05:37:02.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.148+0000 7fb688c4e640 1 -- 172.21.15.70:0/1796309966 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb680030d40 con 0x7fb68415aad0 2023-10-08T05:37:02.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.148+0000 7fb68beb4640 1 -- 172.21.15.70:0/1796309966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb68415aad0 msgr2=0x7fb6840b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.148+0000 7fb68beb4640 1 --2- 172.21.15.70:0/1796309966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb68415aad0 0x7fb6840b1750 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb680009870 tx=0x7fb68002b990 comp rx=0 tx=0).stop 2023-10-08T05:37:02.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.149+0000 7fb68beb4640 1 -- 172.21.15.70:0/1796309966 shutdown_connections 2023-10-08T05:37:02.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.149+0000 7fb68beb4640 1 --2- 172.21.15.70:0/1796309966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb68415aad0 0x7fb6840b1750 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.149+0000 7fb68beb4640 1 -- 172.21.15.70:0/1796309966 >> 172.21.15.70:0/1796309966 conn(0x7fb684069d50 msgr2=0x7fb68406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:02.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.149+0000 7fb68beb4640 1 -- 172.21.15.70:0/1796309966 shutdown_connections 2023-10-08T05:37:02.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.149+0000 7fb68beb4640 1 -- 172.21.15.70:0/1796309966 wait complete. 2023-10-08T05:37:02.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.151+0000 7fb68beb4640 1 Processor -- start 2023-10-08T05:37:02.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.151+0000 7fb68beb4640 1 -- start start 2023-10-08T05:37:02.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68beb4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb68406d410 0x7fb684151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68beb4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6840c82b0 0x7fb68414e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68beb4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb68415aad0 0x7fb68414e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68beb4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb684150950 con 0x7fb6840c82b0 2023-10-08T05:37:02.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68beb4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840ccdc0 con 0x7fb68415aad0 2023-10-08T05:37:02.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68beb4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6840cd0a0 con 0x7fb68406d410 2023-10-08T05:37:02.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68944f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6840c82b0 0x7fb68414e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68944f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6840c82b0 0x7fb68414e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50090/0 (socket says 172.21.15.70:50090) 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68944f640 1 -- 172.21.15.70:0/713698855 learned_addr learned my addr 172.21.15.70:0/713698855 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb689c50640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb68406d410 0x7fb684151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68944f640 1 -- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb68406d410 msgr2=0x7fb684151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68944f640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb68406d410 0x7fb684151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68a451640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb68415aad0 0x7fb68414e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68944f640 1 -- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb68415aad0 msgr2=0x7fb68414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.152+0000 7fb68944f640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb68415aad0 0x7fb68414e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.153+0000 7fb68944f640 1 -- 172.21.15.70:0/713698855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6800094d0 con 0x7fb6840c82b0 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.153+0000 7fb68a451640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb68415aad0 0x7fb68414e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:02.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.153+0000 7fb68944f640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6840c82b0 0x7fb68414e3d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb67400c450 tx=0x7fb67400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:02.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.153+0000 7fb67a7fc640 1 -- 172.21.15.70:0/713698855 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674007480 con 0x7fb6840c82b0 2023-10-08T05:37:02.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.153+0000 7fb67a7fc640 1 -- 172.21.15.70:0/713698855 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb674007600 con 0x7fb6840c82b0 2023-10-08T05:37:02.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.153+0000 7fb68beb4640 1 -- 172.21.15.70:0/713698855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6840cd380 con 0x7fb6840c82b0 2023-10-08T05:37:02.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.153+0000 7fb67a7fc640 1 -- 172.21.15.70:0/713698855 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb674004a20 con 0x7fb6840c82b0 2023-10-08T05:37:02.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.153+0000 7fb68beb4640 1 -- 172.21.15.70:0/713698855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6840cd940 con 0x7fb6840c82b0 2023-10-08T05:37:02.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.154+0000 7fb68beb4640 1 -- 172.21.15.70:0/713698855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6840cdb60 con 0x7fb6840c82b0 2023-10-08T05:37:02.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.156+0000 7fb67a7fc640 1 -- 172.21.15.70:0/713698855 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb674004b80 con 0x7fb6840c82b0 2023-10-08T05:37:02.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.157+0000 7fb67a7fc640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb66806ae70 0x7fb66806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.157+0000 7fb67a7fc640 1 -- 172.21.15.70:0/713698855 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb674088e60 con 0x7fb6840c82b0 2023-10-08T05:37:02.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.158+0000 7fb689c50640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb66806ae70 0x7fb66806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.159+0000 7fb689c50640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb66806ae70 0x7fb66806d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb680036000 tx=0x7fb68003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:02.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.162+0000 7fb67a7fc640 1 -- 172.21.15.70:0/713698855 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb674053ab0 con 0x7fb6840c82b0 2023-10-08T05:37:02.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.335+0000 7fb68beb4640 1 -- 172.21.15.70:0/713698855 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb68404e160 con 0x7fb6840c82b0 2023-10-08T05:37:02.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.336+0000 7fb67a7fc640 1 -- 172.21.15.70:0/713698855 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6740180a0 con 0x7fb6840c82b0 2023-10-08T05:37:02.337 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 -- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb66806ae70 msgr2=0x7fb66806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb66806ae70 0x7fb66806d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb680036000 tx=0x7fb68003b040 comp rx=0 tx=0).stop 2023-10-08T05:37:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 -- 172.21.15.70:0/713698855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6840c82b0 msgr2=0x7fb68414e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6840c82b0 0x7fb68414e3d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb67400c450 tx=0x7fb67400c910 comp rx=0 tx=0).stop 2023-10-08T05:37:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 -- 172.21.15.70:0/713698855 shutdown_connections 2023-10-08T05:37:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb66806ae70 0x7fb66806d330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb68415aad0 0x7fb68414e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6840c82b0 0x7fb68414e3d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 --2- 172.21.15.70:0/713698855 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb68406d410 0x7fb684151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 -- 172.21.15.70:0/713698855 >> 172.21.15.70:0/713698855 conn(0x7fb684069d50 msgr2=0x7fb68406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 -- 172.21.15.70:0/713698855 shutdown_connections 2023-10-08T05:37:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.339+0000 7fb68beb4640 1 -- 172.21.15.70:0/713698855 wait complete. 2023-10-08T05:37:02.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:37:02.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.572+0000 7f5cb9197640 1 Processor -- start 2023-10-08T05:37:02.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.573+0000 7f5cb9197640 1 -- start start 2023-10-08T05:37:02.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.574+0000 7f5cb9197640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5cb415abe0 0x7f5cb40d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.574+0000 7f5cb9197640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40cd430 0x7f5cb40cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.574+0000 7f5cb9197640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb40b69d0 con 0x7f5cb40b62a0 2023-10-08T05:37:02.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.574+0000 7f5cb9197640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb40cdde0 con 0x7f5cb415abe0 2023-10-08T05:37:02.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.574+0000 7f5cb9197640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb40cf770 con 0x7f5cb40cd430 2023-10-08T05:37:02.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5cb40b62a0 0x7f5cb40cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59334/0 (socket says 172.21.15.70:59334) 2023-10-08T05:37:02.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb259c640 1 -- 172.21.15.70:0/2380293204 learned_addr learned my addr 172.21.15.70:0/2380293204 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb2d9d640 1 --2- 172.21.15.70:0/2380293204 >> v2:172.21.15.181:3300/0 conn(0x7f5cb415abe0 0x7f5cb40d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb359e640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40cd430 0x7f5cb40cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157524907 0 0) 0x7f5cb40b69d0 con 0x7f5cb40b62a0 2023-10-08T05:37:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ca0003520 con 0x7f5cb40b62a0 2023-10-08T05:37:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4216805198 0 0) 0x7f5ca0003520 con 0x7f5cb40b62a0 2023-10-08T05:37:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40cd430 msgr2=0x7f5cb40cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb1d9b640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40cd430 0x7f5cb40cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 >> v2:172.21.15.181:3300/0 conn(0x7f5cb415abe0 msgr2=0x7f5cb40d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb1d9b640 1 --2- 172.21.15.70:0/2380293204 >> v2:172.21.15.181:3300/0 conn(0x7f5cb415abe0 0x7f5cb40d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb40b6700 con 0x7f5cb40b62a0 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5c9c001670 con 0x7f5cb40b62a0 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.575+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 >> v1:172.21.15.70:6789/0 conn(0x7f5cb40b62a0 legacy=0x7f5cb40cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb1d9b640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0003970 0x7f5ca0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb1d9b640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40cd430 0x7f5ca00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb1d9b640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca0004d00 0x7f5ca0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb40cdde0 con 0x7f5ca0004d00 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb40cf770 con 0x7f5ca0003970 2023-10-08T05:37:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca000b980 con 0x7f5cb40cd430 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb359e640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca0004d00 0x7f5ca0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb2d9d640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0003970 0x7f5ca0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb259c640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40cd430 0x7f5ca00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb359e640 1 -- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40cd430 msgr2=0x7f5ca00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb359e640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40cd430 0x7f5ca00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb359e640 1 -- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0003970 msgr2=0x7f5ca0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb359e640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0003970 0x7f5ca0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb359e640 1 -- 172.21.15.70:0/2380293204 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca0003520 con 0x7f5ca0004d00 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.576+0000 7f5cb2d9d640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0003970 0x7f5ca0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.577+0000 7f5cb359e640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca0004d00 0x7f5ca0005180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009ac0 tx=0x7f5ca00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7774b2a0bf25c851 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.578+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac02c6b0 con 0x7f5ca0004d00 2023-10-08T05:37:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.578+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac02c810 con 0x7f5ca0004d00 2023-10-08T05:37:02.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.578+0000 7f5cb1d9b640 1 -- 172.21.15.70:0/2380293204 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac02cba0 con 0x7f5ca0004d00 2023-10-08T05:37:02.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.578+0000 7f5cb9197640 1 -- 172.21.15.70:0/2380293204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca0004d00 msgr2=0x7f5ca0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.578+0000 7f5cb9197640 1 --2- 172.21.15.70:0/2380293204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ca0004d00 0x7f5ca0005180 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009ac0 tx=0x7f5ca00057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.578+0000 7f5cb359e640 1 -- 172.21.15.70:0/2380293204 reap_dead start 2023-10-08T05:37:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.578+0000 7f5cb9197640 1 -- 172.21.15.70:0/2380293204 shutdown_connections 2023-10-08T05:37:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.578+0000 7f5cb9197640 1 -- 172.21.15.70:0/2380293204 >> 172.21.15.70:0/2380293204 conn(0x7f5cb406e650 msgr2=0x7f5cb406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.578+0000 7f5cb9197640 1 -- 172.21.15.70:0/2380293204 shutdown_connections 2023-10-08T05:37:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.578+0000 7f5cb9197640 1 -- 172.21.15.70:0/2380293204 wait complete. 2023-10-08T05:37:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.580+0000 7f5cb9197640 1 Processor -- start 2023-10-08T05:37:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.580+0000 7f5cb9197640 1 -- start start 2023-10-08T05:37:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.581+0000 7f5cb9197640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0004d00 0x7f5cb40ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.581+0000 7f5cb9197640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40b1b40 0x7f5cb40b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.581+0000 7f5cb9197640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb4148ef0 0x7f5cb40b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.581+0000 7f5cb9197640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb40b1fc0 con 0x7f5cb4148ef0 2023-10-08T05:37:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.581+0000 7f5cb9197640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb40b0890 con 0x7f5ca0004d00 2023-10-08T05:37:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.581+0000 7f5cb9197640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4149370 con 0x7f5cb40b1b40 2023-10-08T05:37:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb4148ef0 0x7f5cb40b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb4148ef0 0x7f5cb40b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50120/0 (socket says 172.21.15.70:50120) 2023-10-08T05:37:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb359e640 1 -- 172.21.15.70:0/1147899562 learned_addr learned my addr 172.21.15.70:0/1147899562 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb2d9d640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0004d00 0x7f5cb40ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb259c640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40b1b40 0x7f5cb40b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb359e640 1 -- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40b1b40 msgr2=0x7f5cb40b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb359e640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40b1b40 0x7f5cb40b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb359e640 1 -- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0004d00 msgr2=0x7f5cb40ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb359e640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0004d00 0x7f5cb40ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb359e640 1 -- 172.21.15.70:0/1147899562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb40b0aa0 con 0x7f5cb4148ef0 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb259c640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40b1b40 0x7f5cb40b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb2d9d640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0004d00 0x7f5cb40ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.582+0000 7f5cb359e640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb4148ef0 0x7f5cb40b4580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5cac037040 tx=0x7f5cac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.583+0000 7f5cab7fe640 1 -- 172.21.15.70:0/1147899562 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac02c6b0 con 0x7f5cb4148ef0 2023-10-08T05:37:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.583+0000 7f5cab7fe640 1 -- 172.21.15.70:0/1147899562 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac003830 con 0x7f5cb4148ef0 2023-10-08T05:37:02.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.583+0000 7f5cab7fe640 1 -- 172.21.15.70:0/1147899562 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac003be0 con 0x7f5cb4148ef0 2023-10-08T05:37:02.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.583+0000 7f5cb9197640 1 -- 172.21.15.70:0/1147899562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb40b0d80 con 0x7f5cb4148ef0 2023-10-08T05:37:02.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.583+0000 7f5cb9197640 1 -- 172.21.15.70:0/1147899562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb40b1340 con 0x7f5cb4148ef0 2023-10-08T05:37:02.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.584+0000 7f5cb9197640 1 -- 172.21.15.70:0/1147899562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb404e160 con 0x7f5cb4148ef0 2023-10-08T05:37:02.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.585+0000 7f5cab7fe640 1 -- 172.21.15.70:0/1147899562 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5cac003d40 con 0x7f5cb4148ef0 2023-10-08T05:37:02.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.586+0000 7f5cab7fe640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c9406aea0 0x7f5c9406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:02.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.586+0000 7f5cab7fe640 1 -- 172.21.15.70:0/1147899562 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5cac000e40 con 0x7f5cb4148ef0 2023-10-08T05:37:02.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.586+0000 7f5cb2d9d640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c9406aea0 0x7f5c9406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:02.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.588+0000 7f5cb2d9d640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c9406aea0 0x7f5c9406d360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4007580 tx=0x7f5ca400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:02.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.590+0000 7f5cab7fe640 1 -- 172.21.15.70:0/1147899562 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5cac075f20 con 0x7f5cb4148ef0 2023-10-08T05:37:02.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.777+0000 7f5cb9197640 1 -- 172.21.15.70:0/1147899562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5cb4159c70 con 0x7f5cb4148ef0 2023-10-08T05:37:02.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.778+0000 7f5cab7fe640 1 -- 172.21.15.70:0/1147899562 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5cac079540 con 0x7f5cb4148ef0 2023-10-08T05:37:02.779 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:02.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.780+0000 7f5cb9197640 1 -- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c9406aea0 msgr2=0x7f5c9406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.780+0000 7f5cb9197640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c9406aea0 0x7f5c9406d360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4007580 tx=0x7f5ca400b040 comp rx=0 tx=0).stop 2023-10-08T05:37:02.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.780+0000 7f5cb9197640 1 -- 172.21.15.70:0/1147899562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb4148ef0 msgr2=0x7f5cb40b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:02.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.780+0000 7f5cb9197640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb4148ef0 0x7f5cb40b4580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5cac037040 tx=0x7f5cac004730 comp rx=0 tx=0).stop 2023-10-08T05:37:02.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.780+0000 7f5cb9197640 1 -- 172.21.15.70:0/1147899562 shutdown_connections 2023-10-08T05:37:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.780+0000 7f5cb9197640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5c9406aea0 0x7f5c9406d360 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.780+0000 7f5cb9197640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cb4148ef0 0x7f5cb40b4580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.780+0000 7f5cb9197640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cb40b1b40 0x7f5cb40b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.780+0000 7f5cb9197640 1 --2- 172.21.15.70:0/1147899562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ca0004d00 0x7f5cb40ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.780+0000 7f5cb9197640 1 -- 172.21.15.70:0/1147899562 >> 172.21.15.70:0/1147899562 conn(0x7f5cb406e650 msgr2=0x7f5cb4158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.781+0000 7f5cb9197640 1 -- 172.21.15.70:0/1147899562 shutdown_connections 2023-10-08T05:37:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:02.781+0000 7f5cb9197640 1 -- 172.21.15.70:0/1147899562 wait complete. 2023-10-08T05:37:02.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:37:03.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.014+0000 7fd032860640 1 Processor -- start 2023-10-08T05:37:03.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.014+0000 7fd032860640 1 -- start start 2023-10-08T05:37:03.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.015+0000 7fd032860640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd02c1599d0 0x7fd02c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.015+0000 7fd032860640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd02c0cb5e0 0x7fd02c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.015+0000 7fd032860640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c159db0 con 0x7fd02c0b3c50 2023-10-08T05:37:03.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.015+0000 7fd032860640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02c071fe0 con 0x7fd02c0cb5e0 2023-10-08T05:37:03.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.015+0000 7fd032860640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02c0b44c0 con 0x7fd02c1599d0 2023-10-08T05:37:03.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.016+0000 7fd03205f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd02c0cb5e0 0x7fd02c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.016+0000 7fd03205f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd02c0cb5e0 0x7fd02c0cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50122/0 (socket says 172.21.15.70:50122) 2023-10-08T05:37:03.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.016+0000 7fd03205f640 1 -- 172.21.15.70:0/572357108 learned_addr learned my addr 172.21.15.70:0/572357108 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:03.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.016+0000 7fd03185e640 1 --2- 172.21.15.70:0/572357108 >> v2:172.21.15.181:3300/0 conn(0x7fd02c1599d0 0x7fd02c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.016+0000 7fd03205f640 1 -- 172.21.15.70:0/572357108 >> v2:172.21.15.181:3300/0 conn(0x7fd02c1599d0 msgr2=0x7fd02c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:03.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.016+0000 7fd03205f640 1 --2- 172.21.15.70:0/572357108 >> v2:172.21.15.181:3300/0 conn(0x7fd02c1599d0 0x7fd02c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.016+0000 7fd03205f640 1 -- 172.21.15.70:0/572357108 >> v1:172.21.15.181:6789/0 conn(0x7fd02c0b3c50 legacy=0x7fd02c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:03.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.016+0000 7fd03205f640 1 -- 172.21.15.70:0/572357108 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0713d0 con 0x7fd02c0cb5e0 2023-10-08T05:37:03.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.016+0000 7fd03185e640 1 --2- 172.21.15.70:0/572357108 >> v2:172.21.15.181:3300/0 conn(0x7fd02c1599d0 0x7fd02c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.017+0000 7fd03205f640 1 --2- 172.21.15.70:0/572357108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0cb5e0 0x7fd02c0cc0c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd02c06e150 tx=0x7fd02402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b3cc37af7d8275 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.017+0000 7fd03085c640 1 -- 172.21.15.70:0/572357108 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02402ca80 con 0x7fd02c0cb5e0 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.017+0000 7fd03085c640 1 -- 172.21.15.70:0/572357108 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02402cbe0 con 0x7fd02c0cb5e0 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.017+0000 7fd03085c640 1 -- 172.21.15.70:0/572357108 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0240343f0 con 0x7fd02c0cb5e0 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.018+0000 7fd032860640 1 -- 172.21.15.70:0/572357108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0cb5e0 msgr2=0x7fd02c0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.018+0000 7fd032860640 1 --2- 172.21.15.70:0/572357108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0cb5e0 0x7fd02c0cc0c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd02c06e150 tx=0x7fd02402bee0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.018+0000 7fd032860640 1 -- 172.21.15.70:0/572357108 shutdown_connections 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.018+0000 7fd032860640 1 --2- 172.21.15.70:0/572357108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0cb5e0 0x7fd02c0cc0c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.018+0000 7fd032860640 1 --2- 172.21.15.70:0/572357108 >> v2:172.21.15.181:3300/0 conn(0x7fd02c1599d0 0x7fd02c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.018+0000 7fd032860640 1 -- 172.21.15.70:0/572357108 >> 172.21.15.70:0/572357108 conn(0x7fd02c06c050 msgr2=0x7fd02c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:03.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.018+0000 7fd032860640 1 -- 172.21.15.70:0/572357108 shutdown_connections 2023-10-08T05:37:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.018+0000 7fd032860640 1 -- 172.21.15.70:0/572357108 wait complete. 2023-10-08T05:37:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.020+0000 7fd032860640 1 Processor -- start 2023-10-08T05:37:03.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.020+0000 7fd032860640 1 -- start start 2023-10-08T05:37:03.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.021+0000 7fd032860640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0b3c50 0x7fd02c0c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.021+0000 7fd032860640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd02c0cb5e0 0x7fd02c0c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.021+0000 7fd032860640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd02c1599d0 0x7fd02c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.021+0000 7fd032860640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0af150 con 0x7fd02c0b3c50 2023-10-08T05:37:03.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.021+0000 7fd032860640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0c7290 con 0x7fd02c0cb5e0 2023-10-08T05:37:03.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.021+0000 7fd032860640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0c7570 con 0x7fd02c1599d0 2023-10-08T05:37:03.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.021+0000 7fd03185e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0b3c50 0x7fd02c0c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.021+0000 7fd03185e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0b3c50 0x7fd02c0c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50128/0 (socket says 172.21.15.70:50128) 2023-10-08T05:37:03.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.021+0000 7fd03185e640 1 -- 172.21.15.70:0/2578074587 learned_addr learned my addr 172.21.15.70:0/2578074587 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:03.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd03185e640 1 -- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd02c1599d0 msgr2=0x7fd02c0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd03185e640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd02c1599d0 0x7fd02c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd03185e640 1 -- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd02c0cb5e0 msgr2=0x7fd02c0c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd03185e640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd02c0cb5e0 0x7fd02c0c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd03185e640 1 -- 172.21.15.70:0/2578074587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd024009b30 con 0x7fd02c0b3c50 2023-10-08T05:37:03.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd03185e640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0b3c50 0x7fd02c0c8330 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c7f0 tx=0x7fd02800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:03.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd0227fc640 1 -- 172.21.15.70:0/2578074587 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028007a30 con 0x7fd02c0b3c50 2023-10-08T05:37:03.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd0227fc640 1 -- 172.21.15.70:0/2578074587 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd028007b90 con 0x7fd02c0b3c50 2023-10-08T05:37:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd0227fc640 1 -- 172.21.15.70:0/2578074587 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02800ce40 con 0x7fd02c0b3c50 2023-10-08T05:37:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd032860640 1 -- 172.21.15.70:0/2578074587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02c0c7850 con 0x7fd02c0b3c50 2023-10-08T05:37:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.022+0000 7fd032860640 1 -- 172.21.15.70:0/2578074587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02c0c7e10 con 0x7fd02c0b3c50 2023-10-08T05:37:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.024+0000 7fd032860640 1 -- 172.21.15.70:0/2578074587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd02c0c8000 con 0x7fd02c0b3c50 2023-10-08T05:37:03.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.024+0000 7fd0227fc640 1 -- 172.21.15.70:0/2578074587 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd0280113f0 con 0x7fd02c0b3c50 2023-10-08T05:37:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.025+0000 7fd0227fc640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd01006ae70 0x7fd01006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.025+0000 7fd0227fc640 1 -- 172.21.15.70:0/2578074587 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd028089c10 con 0x7fd02c0b3c50 2023-10-08T05:37:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.026+0000 7fd03105d640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd01006ae70 0x7fd01006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.027+0000 7fd03105d640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd01006ae70 0x7fd01006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0cd330 tx=0x7fd01c006ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:03.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.030+0000 7fd0227fc640 1 -- 172.21.15.70:0/2578074587 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd028054860 con 0x7fd02c0b3c50 2023-10-08T05:37:03.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.208+0000 7fd032860640 1 -- 172.21.15.70:0/2578074587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd02c074150 con 0x7fd02c0b3c50 2023-10-08T05:37:03.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.209+0000 7fd0227fc640 1 -- 172.21.15.70:0/2578074587 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd02801f0a0 con 0x7fd02c0b3c50 2023-10-08T05:37:03.210 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:37:03.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.211+0000 7fd032860640 1 -- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd01006ae70 msgr2=0x7fd01006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.211+0000 7fd032860640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd01006ae70 0x7fd01006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0cd330 tx=0x7fd01c006ba0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.211+0000 7fd032860640 1 -- 172.21.15.70:0/2578074587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0b3c50 msgr2=0x7fd02c0c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.211+0000 7fd032860640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0b3c50 0x7fd02c0c8330 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd02800c7f0 tx=0x7fd02800cbf0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.212+0000 7fd032860640 1 -- 172.21.15.70:0/2578074587 shutdown_connections 2023-10-08T05:37:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.212+0000 7fd032860640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd01006ae70 0x7fd01006d330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.212+0000 7fd032860640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd02c1599d0 0x7fd02c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.212+0000 7fd032860640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd02c0cb5e0 0x7fd02c0c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.212+0000 7fd032860640 1 --2- 172.21.15.70:0/2578074587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd02c0b3c50 0x7fd02c0c8330 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.212+0000 7fd032860640 1 -- 172.21.15.70:0/2578074587 >> 172.21.15.70:0/2578074587 conn(0x7fd02c06c050 msgr2=0x7fd02c0b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.212+0000 7fd032860640 1 -- 172.21.15.70:0/2578074587 shutdown_connections 2023-10-08T05:37:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.212+0000 7fd032860640 1 -- 172.21.15.70:0/2578074587 wait complete. 2023-10-08T05:37:03.230 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:37:03.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.430+0000 7fe796dff640 1 Processor -- start 2023-10-08T05:37:03.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.431+0000 7fe796dff640 1 -- start start 2023-10-08T05:37:03.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.432+0000 7fe796dff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe79015cae0 0x7fe7900cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.432+0000 7fe796dff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7900c8b40 0x7fe7900c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.432+0000 7fe796dff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7900aff70 0x7fe7900b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.432+0000 7fe796dff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7900c9510 con 0x7fe7900c8b40 2023-10-08T05:37:03.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.432+0000 7fe796dff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7900ccbc0 con 0x7fe79015cae0 2023-10-08T05:37:03.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.433+0000 7fe796dff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900b0940 con 0x7fe7900aff70 2023-10-08T05:37:03.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.433+0000 7fe7955fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7900c8b40 0x7fe7900c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.433+0000 7fe7955fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7900c8b40 0x7fe7900c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50144/0 (socket says 172.21.15.70:50144) 2023-10-08T05:37:03.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.433+0000 7fe7955fc640 1 -- 172.21.15.70:0/616896814 learned_addr learned my addr 172.21.15.70:0/616896814 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:03.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.433+0000 7fe7955fc640 1 -- 172.21.15.70:0/616896814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7900aff70 msgr2=0x7fe7900b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:03.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.433+0000 7fe7955fc640 1 --2- 172.21.15.70:0/616896814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7900aff70 0x7fe7900b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.433+0000 7fe7955fc640 1 -- 172.21.15.70:0/616896814 >> v2:172.21.15.181:3300/0 conn(0x7fe79015cae0 msgr2=0x7fe7900cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:03.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.433+0000 7fe795dfd640 1 --2- 172.21.15.70:0/616896814 >> v2:172.21.15.181:3300/0 conn(0x7fe79015cae0 0x7fe7900cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.433+0000 7fe7955fc640 1 --2- 172.21.15.70:0/616896814 >> v2:172.21.15.181:3300/0 conn(0x7fe79015cae0 0x7fe7900cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.433+0000 7fe7955fc640 1 -- 172.21.15.70:0/616896814 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7900ce1f0 con 0x7fe7900c8b40 2023-10-08T05:37:03.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.434+0000 7fe7955fc640 1 --2- 172.21.15.70:0/616896814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 0x7fe7900c8fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe7900caed0 tx=0x7fe78002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6802c93d7df672c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:03.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.435+0000 7fe794dfb640 1 -- 172.21.15.70:0/616896814 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780031860 con 0x7fe7900c8b40 2023-10-08T05:37:03.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.435+0000 7fe794dfb640 1 -- 172.21.15.70:0/616896814 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7800319c0 con 0x7fe7900c8b40 2023-10-08T05:37:03.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.435+0000 7fe794dfb640 1 -- 172.21.15.70:0/616896814 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780031d50 con 0x7fe7900c8b40 2023-10-08T05:37:03.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.435+0000 7fe796dff640 1 -- 172.21.15.70:0/616896814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 msgr2=0x7fe7900c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.435+0000 7fe796dff640 1 --2- 172.21.15.70:0/616896814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 0x7fe7900c8fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe7900caed0 tx=0x7fe78002b900 comp rx=0 tx=0).stop 2023-10-08T05:37:03.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.436+0000 7fe796dff640 1 -- 172.21.15.70:0/616896814 shutdown_connections 2023-10-08T05:37:03.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.436+0000 7fe796dff640 1 --2- 172.21.15.70:0/616896814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7900aff70 0x7fe7900b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.436+0000 7fe796dff640 1 --2- 172.21.15.70:0/616896814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 0x7fe7900c8fd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.436+0000 7fe796dff640 1 --2- 172.21.15.70:0/616896814 >> v2:172.21.15.181:3300/0 conn(0x7fe79015cae0 0x7fe7900cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.436+0000 7fe796dff640 1 -- 172.21.15.70:0/616896814 >> 172.21.15.70:0/616896814 conn(0x7fe79006c050 msgr2=0x7fe79006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:03.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.436+0000 7fe796dff640 1 -- 172.21.15.70:0/616896814 shutdown_connections 2023-10-08T05:37:03.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.436+0000 7fe796dff640 1 -- 172.21.15.70:0/616896814 wait complete. 2023-10-08T05:37:03.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.437+0000 7fe796dff640 1 Processor -- start 2023-10-08T05:37:03.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.437+0000 7fe796dff640 1 -- start start 2023-10-08T05:37:03.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.438+0000 7fe796dff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7900aff70 0x7fe79014bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.438+0000 7fe796dff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 0x7fe7900b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.438+0000 7fe796dff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe79015cae0 0x7fe7900b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.438+0000 7fe796dff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7900b2720 con 0x7fe7900c8b40 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.438+0000 7fe796dff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79014c140 con 0x7fe7900aff70 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.438+0000 7fe796dff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7901460b0 con 0x7fe79015cae0 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.439+0000 7fe7955fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 0x7fe7900b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.439+0000 7fe7955fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 0x7fe7900b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50156/0 (socket says 172.21.15.70:50156) 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.439+0000 7fe7955fc640 1 -- 172.21.15.70:0/1783230831 learned_addr learned my addr 172.21.15.70:0/1783230831 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.439+0000 7fe7955fc640 1 -- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe79015cae0 msgr2=0x7fe7900b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.439+0000 7fe7955fc640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe79015cae0 0x7fe7900b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.439+0000 7fe7955fc640 1 -- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7900aff70 msgr2=0x7fe79014bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.439+0000 7fe7955fc640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7900aff70 0x7fe79014bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.439+0000 7fe7955fc640 1 -- 172.21.15.70:0/1783230831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7800094d0 con 0x7fe7900c8b40 2023-10-08T05:37:03.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.439+0000 7fe7955fc640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 0x7fe7900b0d50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe780000c00 tx=0x7fe7800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:03.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.439+0000 7fe7867fc640 1 -- 172.21.15.70:0/1783230831 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780031830 con 0x7fe7900c8b40 2023-10-08T05:37:03.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.440+0000 7fe7867fc640 1 -- 172.21.15.70:0/1783230831 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7800443f0 con 0x7fe7900c8b40 2023-10-08T05:37:03.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.440+0000 7fe7867fc640 1 -- 172.21.15.70:0/1783230831 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7800447a0 con 0x7fe7900c8b40 2023-10-08T05:37:03.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.440+0000 7fe796dff640 1 -- 172.21.15.70:0/1783230831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7900b2f80 con 0x7fe7900c8b40 2023-10-08T05:37:03.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.440+0000 7fe796dff640 1 -- 172.21.15.70:0/1783230831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7900af9c0 con 0x7fe7900c8b40 2023-10-08T05:37:03.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.441+0000 7fe796dff640 1 -- 172.21.15.70:0/1783230831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe790074150 con 0x7fe7900c8b40 2023-10-08T05:37:03.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.442+0000 7fe7867fc640 1 -- 172.21.15.70:0/1783230831 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fe78004d450 con 0x7fe7900c8b40 2023-10-08T05:37:03.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.443+0000 7fe7867fc640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe77406ae70 0x7fe77406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.443+0000 7fe7867fc640 1 -- 172.21.15.70:0/1783230831 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fe7800ac170 con 0x7fe7900c8b40 2023-10-08T05:37:03.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.443+0000 7fe795dfd640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe77406ae70 0x7fe77406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.444+0000 7fe795dfd640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe77406ae70 0x7fe77406d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00a810 tx=0x7fe78c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:03.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.447+0000 7fe7867fc640 1 -- 172.21.15.70:0/1783230831 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe780076e60 con 0x7fe7900c8b40 2023-10-08T05:37:03.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.635+0000 7fe796dff640 1 -- 172.21.15.70:0/1783230831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe7900b4470 con 0x7fe7900c8b40 2023-10-08T05:37:03.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.636+0000 7fe7867fc640 1 -- 172.21.15.70:0/1783230831 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe78007a480 con 0x7fe7900c8b40 2023-10-08T05:37:03.638 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:03.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 -- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe77406ae70 msgr2=0x7fe77406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe77406ae70 0x7fe77406d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00a810 tx=0x7fe78c008040 comp rx=0 tx=0).stop 2023-10-08T05:37:03.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 -- 172.21.15.70:0/1783230831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 msgr2=0x7fe7900b0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 0x7fe7900b0d50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe780000c00 tx=0x7fe7800046c0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 -- 172.21.15.70:0/1783230831 shutdown_connections 2023-10-08T05:37:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe77406ae70 0x7fe77406d330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe79015cae0 0x7fe7900b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7900c8b40 0x7fe7900b0d50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 --2- 172.21.15.70:0/1783230831 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe7900aff70 0x7fe79014bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 -- 172.21.15.70:0/1783230831 >> 172.21.15.70:0/1783230831 conn(0x7fe79006c050 msgr2=0x7fe7900cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 -- 172.21.15.70:0/1783230831 shutdown_connections 2023-10-08T05:37:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.639+0000 7fe796dff640 1 -- 172.21.15.70:0/1783230831 wait complete. 2023-10-08T05:37:03.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:03.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.868+0000 7f168514c640 1 Processor -- start 2023-10-08T05:37:03.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.869+0000 7f168514c640 1 -- start start 2023-10-08T05:37:03.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.870+0000 7f168514c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f168015ab40 0x7f16800ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.870+0000 7f168514c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800c90b0 0x7f16800adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.870+0000 7f168514c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168015b2b0 con 0x7f16800cae70 2023-10-08T05:37:03.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.870+0000 7f168514c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168015bf30 con 0x7f168015ab40 2023-10-08T05:37:03.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.870+0000 7f168514c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168015f7a0 con 0x7f16800c90b0 2023-10-08T05:37:03.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.871+0000 7f167ed9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f168015ab40 0x7f16800ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.871+0000 7f167659c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800c90b0 0x7f16800adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.871+0000 7f167ed9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f168015ab40 0x7f16800ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:50552/0 (socket says 172.21.15.70:50552) 2023-10-08T05:37:03.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.871+0000 7f167ed9d640 1 -- 172.21.15.70:0/2966466157 learned_addr learned my addr 172.21.15.70:0/2966466157 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:03.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.872+0000 7f167ed9d640 1 -- 172.21.15.70:0/2966466157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800c90b0 msgr2=0x7f16800adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:03.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.872+0000 7f167ed9d640 1 --2- 172.21.15.70:0/2966466157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800c90b0 0x7f16800adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.872+0000 7f167ed9d640 1 -- 172.21.15.70:0/2966466157 >> v1:172.21.15.181:6789/0 conn(0x7f16800cae70 legacy=0x7f16800702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.872+0000 7f167ed9d640 1 -- 172.21.15.70:0/2966466157 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168015b590 con 0x7f168015ab40 2023-10-08T05:37:03.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.872+0000 7f167659c640 1 --2- 172.21.15.70:0/2966466157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800c90b0 0x7f16800adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:37:03.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.873+0000 7f167ed9d640 1 --2- 172.21.15.70:0/2966466157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f168015ab40 0x7f16800ad720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1670009e50 tx=0x7f167002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cc98e59d61c8961 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:03.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.874+0000 7f167e59c640 1 -- 172.21.15.70:0/2966466157 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167002cb00 con 0x7f168015ab40 2023-10-08T05:37:03.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.874+0000 7f167e59c640 1 -- 172.21.15.70:0/2966466157 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167002cc60 con 0x7f168015ab40 2023-10-08T05:37:03.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.874+0000 7f167e59c640 1 -- 172.21.15.70:0/2966466157 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16700343f0 con 0x7f168015ab40 2023-10-08T05:37:03.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.874+0000 7f168514c640 1 -- 172.21.15.70:0/2966466157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f168015ab40 msgr2=0x7f16800ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.874+0000 7f168514c640 1 --2- 172.21.15.70:0/2966466157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f168015ab40 0x7f16800ad720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1670009e50 tx=0x7f167002bf60 comp rx=0 tx=0).stop 2023-10-08T05:37:03.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.875+0000 7f168514c640 1 -- 172.21.15.70:0/2966466157 shutdown_connections 2023-10-08T05:37:03.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.875+0000 7f168514c640 1 --2- 172.21.15.70:0/2966466157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800c90b0 0x7f16800adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.875+0000 7f168514c640 1 --2- 172.21.15.70:0/2966466157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f168015ab40 0x7f16800ad720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.875+0000 7f168514c640 1 -- 172.21.15.70:0/2966466157 >> 172.21.15.70:0/2966466157 conn(0x7f168006a380 msgr2=0x7f16800b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:03.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.875+0000 7f168514c640 1 -- 172.21.15.70:0/2966466157 shutdown_connections 2023-10-08T05:37:03.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.875+0000 7f168514c640 1 -- 172.21.15.70:0/2966466157 wait complete. 2023-10-08T05:37:03.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.876+0000 7f168514c640 1 Processor -- start 2023-10-08T05:37:03.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.876+0000 7f168514c640 1 -- start start 2023-10-08T05:37:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.877+0000 7f168514c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16800c90b0 0x7f1680154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.877+0000 7f168514c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800cae70 0x7f16800b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.877+0000 7f168514c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f168015ab40 0x7f16800b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.877+0000 7f168514c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1680153e90 con 0x7f16800c90b0 2023-10-08T05:37:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.877+0000 7f168514c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1680154860 con 0x7f168015ab40 2023-10-08T05:37:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.877+0000 7f168514c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1680153280 con 0x7f16800cae70 2023-10-08T05:37:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.878+0000 7f167ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16800c90b0 0x7f1680154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.878+0000 7f167ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16800c90b0 0x7f1680154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43556/0 (socket says 172.21.15.70:43556) 2023-10-08T05:37:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.878+0000 7f167ed9d640 1 -- 172.21.15.70:0/62252295 learned_addr learned my addr 172.21.15.70:0/62252295 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.878+0000 7f167659c640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800cae70 0x7f16800b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.878+0000 7f167f59e640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f168015ab40 0x7f16800b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.878+0000 7f167ed9d640 1 -- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800cae70 msgr2=0x7f16800b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.878+0000 7f167ed9d640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800cae70 0x7f16800b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.878+0000 7f167ed9d640 1 -- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f168015ab40 msgr2=0x7f16800b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.878+0000 7f167ed9d640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f168015ab40 0x7f16800b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.878+0000 7f167ed9d640 1 -- 172.21.15.70:0/62252295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1668009400 con 0x7f16800c90b0 2023-10-08T05:37:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.879+0000 7f167ed9d640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16800c90b0 0x7f1680154320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1670000f80 tx=0x7f1670004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:03.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.879+0000 7f1677fff640 1 -- 172.21.15.70:0/62252295 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670009510 con 0x7f16800c90b0 2023-10-08T05:37:03.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.879+0000 7f1677fff640 1 -- 172.21.15.70:0/62252295 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167003e450 con 0x7f16800c90b0 2023-10-08T05:37:03.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.879+0000 7f1677fff640 1 -- 172.21.15.70:0/62252295 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167003e800 con 0x7f16800c90b0 2023-10-08T05:37:03.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.879+0000 7f168514c640 1 -- 172.21.15.70:0/62252295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1670009b30 con 0x7f16800c90b0 2023-10-08T05:37:03.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.879+0000 7f168514c640 1 -- 172.21.15.70:0/62252295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1680153840 con 0x7f16800c90b0 2023-10-08T05:37:03.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.880+0000 7f168514c640 1 -- 172.21.15.70:0/62252295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1680067490 con 0x7f16800c90b0 2023-10-08T05:37:03.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.881+0000 7f1677fff640 1 -- 172.21.15.70:0/62252295 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f167003e960 con 0x7f16800c90b0 2023-10-08T05:37:03.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.882+0000 7f1677fff640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f166006ae70 0x7f166006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:03.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.882+0000 7f1677fff640 1 -- 172.21.15.70:0/62252295 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f16700ac790 con 0x7f16800c90b0 2023-10-08T05:37:03.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.882+0000 7f167659c640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f166006ae70 0x7f166006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:03.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.884+0000 7f167659c640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f166006ae70 0x7f166006d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1680152ad0 tx=0x7f1668008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:03.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:03.886+0000 7f1677fff640 1 -- 172.21.15.70:0/62252295 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1670077510 con 0x7f16800c90b0 2023-10-08T05:37:04.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.076+0000 7f168514c640 1 -- 172.21.15.70:0/62252295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16800b0120 con 0x7f16800c90b0 2023-10-08T05:37:04.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.079+0000 7f1677fff640 1 -- 172.21.15.70:0/62252295 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7f167007ab30 con 0x7f16800c90b0 2023-10-08T05:37:04.081 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:04.081 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:04.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 -- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f166006ae70 msgr2=0x7f166006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f166006ae70 0x7f166006d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1680152ad0 tx=0x7f1668008040 comp rx=0 tx=0).stop 2023-10-08T05:37:04.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 -- 172.21.15.70:0/62252295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16800c90b0 msgr2=0x7f1680154320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16800c90b0 0x7f1680154320 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1670000f80 tx=0x7f1670004a40 comp rx=0 tx=0).stop 2023-10-08T05:37:04.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 -- 172.21.15.70:0/62252295 shutdown_connections 2023-10-08T05:37:04.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f166006ae70 0x7f166006d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f168015ab40 0x7f16800b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16800cae70 0x7f16800b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 --2- 172.21.15.70:0/62252295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16800c90b0 0x7f1680154320 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 -- 172.21.15.70:0/62252295 >> 172.21.15.70:0/62252295 conn(0x7f168006a380 msgr2=0x7f16800701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:04.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 -- 172.21.15.70:0/62252295 shutdown_connections 2023-10-08T05:37:04.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.081+0000 7f168514c640 1 -- 172.21.15.70:0/62252295 wait complete. 2023-10-08T05:37:04.101 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:37:04.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:04.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.315+0000 7f236fde2640 1 Processor -- start 2023-10-08T05:37:04.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.315+0000 7f236fde2640 1 -- start start 2023-10-08T05:37:04.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.316+0000 7f236fde2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f236815aad0 0x7f23680b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.316+0000 7f236fde2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f23680c82b0 0x7f23680c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.316+0000 7f236fde2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236815bf30 con 0x7f236806cd00 2023-10-08T05:37:04.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.317+0000 7f236fde2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236815b320 con 0x7f23680c82b0 2023-10-08T05:37:04.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.317+0000 7f236fde2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236815b600 con 0x7f236815aad0 2023-10-08T05:37:04.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.317+0000 7f236d37d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f23680c82b0 0x7f23680c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.317+0000 7f236db7e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f236815aad0 0x7f23680b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.317+0000 7f236e37f640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f236806cd00 0x7f2368070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:34048/0 (socket says 172.21.15.70:34048) 2023-10-08T05:37:04.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.317+0000 7f236e37f640 1 -- 172.21.15.70:0/712819363 learned_addr learned my addr 172.21.15.70:0/712819363 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:04.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.318+0000 7f236d37d640 1 -- 172.21.15.70:0/712819363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f236815aad0 msgr2=0x7f23680b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:04.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.318+0000 7f236d37d640 1 --2- 172.21.15.70:0/712819363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f236815aad0 0x7f23680b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.318+0000 7f236d37d640 1 -- 172.21.15.70:0/712819363 >> v1:172.21.15.181:6789/0 conn(0x7f236806cd00 legacy=0x7f2368070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.318+0000 7f236d37d640 1 -- 172.21.15.70:0/712819363 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236815b8e0 con 0x7f23680c82b0 2023-10-08T05:37:04.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.320+0000 7f236d37d640 1 --2- 172.21.15.70:0/712819363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23680c82b0 0x7f23680c8710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f23580097a0 tx=0x7f235802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10bd22701f4b9732 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:04.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.320+0000 7f236cb7c640 1 -- 172.21.15.70:0/712819363 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358031830 con 0x7f23680c82b0 2023-10-08T05:37:04.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.320+0000 7f236cb7c640 1 -- 172.21.15.70:0/712819363 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2358031990 con 0x7f23680c82b0 2023-10-08T05:37:04.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.320+0000 7f236cb7c640 1 -- 172.21.15.70:0/712819363 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358031d20 con 0x7f23680c82b0 2023-10-08T05:37:04.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.321+0000 7f236fde2640 1 -- 172.21.15.70:0/712819363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23680c82b0 msgr2=0x7f23680c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.321+0000 7f236fde2640 1 --2- 172.21.15.70:0/712819363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23680c82b0 0x7f23680c8710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f23580097a0 tx=0x7f235802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.321+0000 7f236fde2640 1 -- 172.21.15.70:0/712819363 shutdown_connections 2023-10-08T05:37:04.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.321+0000 7f236fde2640 1 --2- 172.21.15.70:0/712819363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23680c82b0 0x7f23680c8710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.321+0000 7f236fde2640 1 --2- 172.21.15.70:0/712819363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f236815aad0 0x7f23680b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.321+0000 7f236fde2640 1 -- 172.21.15.70:0/712819363 >> 172.21.15.70:0/712819363 conn(0x7f2368069d50 msgr2=0x7f236806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:04.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.321+0000 7f236fde2640 1 -- 172.21.15.70:0/712819363 shutdown_connections 2023-10-08T05:37:04.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.321+0000 7f236fde2640 1 -- 172.21.15.70:0/712819363 wait complete. 2023-10-08T05:37:04.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.323+0000 7f236fde2640 1 Processor -- start 2023-10-08T05:37:04.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.323+0000 7f236fde2640 1 -- start start 2023-10-08T05:37:04.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.324+0000 7f236fde2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f236806cd00 0x7f23680c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.324+0000 7f236fde2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23680c82b0 0x7f2368157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.324+0000 7f236fde2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f236815aad0 0x7f23681f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.324+0000 7f236fde2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368158910 con 0x7f236815aad0 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.324+0000 7f236fde2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368158070 con 0x7f23680c82b0 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.324+0000 7f236fde2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2368158350 con 0x7f236806cd00 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.324+0000 7f236e37f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f236815aad0 0x7f23681f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.324+0000 7f236e37f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f236815aad0 0x7f23681f1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43570/0 (socket says 172.21.15.70:43570) 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.324+0000 7f236e37f640 1 -- 172.21.15.70:0/3486663390 learned_addr learned my addr 172.21.15.70:0/3486663390 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f236db7e640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f236806cd00 0x7f23680c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f236e37f640 1 -- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f236806cd00 msgr2=0x7f23680c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f236d37d640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23680c82b0 0x7f2368157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f236e37f640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f236806cd00 0x7f23680c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f236e37f640 1 -- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23680c82b0 msgr2=0x7f2368157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f236e37f640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23680c82b0 0x7f2368157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f236e37f640 1 -- 172.21.15.70:0/3486663390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2358009400 con 0x7f236815aad0 2023-10-08T05:37:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f236e37f640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f236815aad0 0x7f23681f1b20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f236000cbc0 tx=0x7f236000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f235e7fc640 1 -- 172.21.15.70:0/3486663390 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000ad40 con 0x7f236815aad0 2023-10-08T05:37:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f235e7fc640 1 -- 172.21.15.70:0/3486663390 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360012400 con 0x7f236815aad0 2023-10-08T05:37:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f235e7fc640 1 -- 172.21.15.70:0/3486663390 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236000aea0 con 0x7f236815aad0 2023-10-08T05:37:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f236fde2640 1 -- 172.21.15.70:0/3486663390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2368158630 con 0x7f236815aad0 2023-10-08T05:37:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.325+0000 7f236fde2640 1 -- 172.21.15.70:0/3486663390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23681f2060 con 0x7f236815aad0 2023-10-08T05:37:04.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.326+0000 7f236fde2640 1 -- 172.21.15.70:0/3486663390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2368148fb0 con 0x7f236815aad0 2023-10-08T05:37:04.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.327+0000 7f235e7fc640 1 -- 172.21.15.70:0/3486663390 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2360007070 con 0x7f236815aad0 2023-10-08T05:37:04.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.329+0000 7f235e7fc640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f234c06ae70 0x7f234c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.329+0000 7f235e7fc640 1 -- 172.21.15.70:0/3486663390 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f23600898e0 con 0x7f236815aad0 2023-10-08T05:37:04.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.329+0000 7f236db7e640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f234c06ae70 0x7f234c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.330+0000 7f236db7e640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f234c06ae70 0x7f234c06d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2364009970 tx=0x7f2364008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:04.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.332+0000 7f235e7fc640 1 -- 172.21.15.70:0/3486663390 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2368148fb0 con 0x7f236815aad0 2023-10-08T05:37:04.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.518+0000 7f236fde2640 1 -- 172.21.15.70:0/3486663390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2368148fb0 con 0x7f236815aad0 2023-10-08T05:37:04.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.518+0000 7f235e7fc640 1 -- 172.21.15.70:0/3486663390 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2368148fb0 con 0x7f236815aad0 2023-10-08T05:37:04.521 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:04.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 -- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f234c06ae70 msgr2=0x7f234c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f234c06ae70 0x7f234c06d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2364009970 tx=0x7f2364008040 comp rx=0 tx=0).stop 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 -- 172.21.15.70:0/3486663390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f236815aad0 msgr2=0x7f23681f1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f236815aad0 0x7f23681f1b20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f236000cbc0 tx=0x7f236000a430 comp rx=0 tx=0).stop 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 -- 172.21.15.70:0/3486663390 shutdown_connections 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f234c06ae70 0x7f234c06d330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f236815aad0 0x7f23681f1b20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23680c82b0 0x7f2368157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 --2- 172.21.15.70:0/3486663390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f236806cd00 0x7f23680c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 -- 172.21.15.70:0/3486663390 >> 172.21.15.70:0/3486663390 conn(0x7f2368069d50 msgr2=0x7f23680b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 -- 172.21.15.70:0/3486663390 shutdown_connections 2023-10-08T05:37:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.521+0000 7f236fde2640 1 -- 172.21.15.70:0/3486663390 wait complete. 2023-10-08T05:37:04.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:37:04.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.751+0000 7f7558207640 1 Processor -- start 2023-10-08T05:37:04.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.751+0000 7f7558207640 1 -- start start 2023-10-08T05:37:04.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.753+0000 7f7558207640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f755006c3a0 0x7f75500c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.753+0000 7f7558207640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f755015aa00 0x7f75500b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.753+0000 7f7558207640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f755015b310 con 0x7f755006db60 2023-10-08T05:37:04.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.753+0000 7f7558207640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75500ca230 con 0x7f755006c3a0 2023-10-08T05:37:04.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.753+0000 7f7558207640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755006d290 con 0x7f755015aa00 2023-10-08T05:37:04.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.753+0000 7f75557a2640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f755006db60 0x7f755006c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33048/0 (socket says 172.21.15.70:33048) 2023-10-08T05:37:04.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.753+0000 7f75557a2640 1 -- 172.21.15.70:0/2542543173 learned_addr learned my addr 172.21.15.70:0/2542543173 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:04.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.753+0000 7f7555fa3640 1 --2- 172.21.15.70:0/2542543173 >> v2:172.21.15.70:3300/0 conn(0x7f755006c3a0 0x7f75500c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.753+0000 7f75567a4640 1 --2- 172.21.15.70:0/2542543173 >> v2:172.21.15.181:3300/0 conn(0x7f755015aa00 0x7f75500b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.754+0000 7f7555fa3640 1 -- 172.21.15.70:0/2542543173 >> v2:172.21.15.181:3300/0 conn(0x7f755015aa00 msgr2=0x7f75500b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:04.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.754+0000 7f7555fa3640 1 --2- 172.21.15.70:0/2542543173 >> v2:172.21.15.181:3300/0 conn(0x7f755015aa00 0x7f75500b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.754+0000 7f7555fa3640 1 -- 172.21.15.70:0/2542543173 >> v1:172.21.15.70:6789/0 conn(0x7f755006db60 legacy=0x7f755006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.754+0000 7f7555fa3640 1 -- 172.21.15.70:0/2542543173 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75500c9860 con 0x7f755006c3a0 2023-10-08T05:37:04.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.755+0000 7f7555fa3640 1 --2- 172.21.15.70:0/2542543173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755006c3a0 0x7f75500c9cf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f75500c57a0 tx=0x7f754c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e79d3b1a68db58e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:04.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.755+0000 7f7554fa1640 1 -- 172.21.15.70:0/2542543173 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c031830 con 0x7f755006c3a0 2023-10-08T05:37:04.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.755+0000 7f7554fa1640 1 -- 172.21.15.70:0/2542543173 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c031990 con 0x7f755006c3a0 2023-10-08T05:37:04.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.755+0000 7f7554fa1640 1 -- 172.21.15.70:0/2542543173 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c031d20 con 0x7f755006c3a0 2023-10-08T05:37:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.756+0000 7f7558207640 1 -- 172.21.15.70:0/2542543173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755006c3a0 msgr2=0x7f75500c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.756+0000 7f7558207640 1 --2- 172.21.15.70:0/2542543173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755006c3a0 0x7f75500c9cf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f75500c57a0 tx=0x7f754c02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.756+0000 7f7558207640 1 -- 172.21.15.70:0/2542543173 shutdown_connections 2023-10-08T05:37:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.756+0000 7f7558207640 1 --2- 172.21.15.70:0/2542543173 >> v2:172.21.15.181:3300/0 conn(0x7f755015aa00 0x7f75500b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.756+0000 7f7558207640 1 --2- 172.21.15.70:0/2542543173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755006c3a0 0x7f75500c9cf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.756+0000 7f7558207640 1 -- 172.21.15.70:0/2542543173 >> 172.21.15.70:0/2542543173 conn(0x7f755006a5d0 msgr2=0x7f75500b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.756+0000 7f7558207640 1 -- 172.21.15.70:0/2542543173 shutdown_connections 2023-10-08T05:37:04.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.756+0000 7f7558207640 1 -- 172.21.15.70:0/2542543173 wait complete. 2023-10-08T05:37:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.758+0000 7f7558207640 1 Processor -- start 2023-10-08T05:37:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.758+0000 7f7558207640 1 -- start start 2023-10-08T05:37:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.759+0000 7f7558207640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f755006c3a0 0x7f75500c3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.759+0000 7f7558207640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f755006db60 0x7f75501505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.759+0000 7f7558207640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755015aa00 0x7f7550150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.759+0000 7f7558207640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755015d0f0 con 0x7f755015aa00 2023-10-08T05:37:04.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.759+0000 7f7558207640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755015d3d0 con 0x7f755006c3a0 2023-10-08T05:37:04.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.759+0000 7f7558207640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75500c6990 con 0x7f755006db60 2023-10-08T05:37:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.760+0000 7f75567a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755015aa00 0x7f7550150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.760+0000 7f75567a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755015aa00 0x7f7550150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43584/0 (socket says 172.21.15.70:43584) 2023-10-08T05:37:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.760+0000 7f75567a4640 1 -- 172.21.15.70:0/4071232098 learned_addr learned my addr 172.21.15.70:0/4071232098 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.760+0000 7f7555fa3640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f755006c3a0 0x7f75500c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.760+0000 7f75557a2640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f755006db60 0x7f75501505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.760+0000 7f75567a4640 1 -- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f755006db60 msgr2=0x7f75501505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.760+0000 7f75567a4640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f755006db60 0x7f75501505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.760+0000 7f75567a4640 1 -- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f755006c3a0 msgr2=0x7f75500c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.760+0000 7f75567a4640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f755006c3a0 0x7f75500c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.760+0000 7f75567a4640 1 -- 172.21.15.70:0/4071232098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c009400 con 0x7f755015aa00 2023-10-08T05:37:04.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.761+0000 7f75567a4640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755015aa00 0x7f7550150b10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7548007e60 tx=0x7f754800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:04.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.761+0000 7f75467fc640 1 -- 172.21.15.70:0/4071232098 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548014420 con 0x7f755015aa00 2023-10-08T05:37:04.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.761+0000 7f75467fc640 1 -- 172.21.15.70:0/4071232098 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75480145a0 con 0x7f755015aa00 2023-10-08T05:37:04.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.761+0000 7f75467fc640 1 -- 172.21.15.70:0/4071232098 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548014950 con 0x7f755015aa00 2023-10-08T05:37:04.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.761+0000 7f7558207640 1 -- 172.21.15.70:0/4071232098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75500c6c70 con 0x7f755015aa00 2023-10-08T05:37:04.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.761+0000 7f7558207640 1 -- 172.21.15.70:0/4071232098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7550151050 con 0x7f755015aa00 2023-10-08T05:37:04.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.762+0000 7f7558207640 1 -- 172.21.15.70:0/4071232098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f755004e160 con 0x7f755015aa00 2023-10-08T05:37:04.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.763+0000 7f75467fc640 1 -- 172.21.15.70:0/4071232098 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7548014ab0 con 0x7f755015aa00 2023-10-08T05:37:04.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.764+0000 7f75467fc640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f753406ae70 0x7f753406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:04.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.764+0000 7f75467fc640 1 -- 172.21.15.70:0/4071232098 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f754808a3c0 con 0x7f755015aa00 2023-10-08T05:37:04.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.764+0000 7f7555fa3640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f753406ae70 0x7f753406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:04.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.766+0000 7f7555fa3640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f753406ae70 0x7f753406d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f754c036000 tx=0x7f754c03c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:04.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.768+0000 7f75467fc640 1 -- 172.21.15.70:0/4071232098 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7548055010 con 0x7f755015aa00 2023-10-08T05:37:04.790 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:37:04.797 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:37:04.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.958+0000 7f7558207640 1 -- 172.21.15.70:0/4071232098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f755015d230 con 0x7f755015aa00 2023-10-08T05:37:04.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.959+0000 7f75467fc640 1 -- 172.21.15.70:0/4071232098 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f755015d230 con 0x7f755015aa00 2023-10-08T05:37:04.961 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:04.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 -- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f753406ae70 msgr2=0x7f753406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f753406ae70 0x7f753406d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f754c036000 tx=0x7f754c03c040 comp rx=0 tx=0).stop 2023-10-08T05:37:04.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 -- 172.21.15.70:0/4071232098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755015aa00 msgr2=0x7f7550150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:04.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755015aa00 0x7f7550150b10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7548007e60 tx=0x7f754800e4b0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 -- 172.21.15.70:0/4071232098 shutdown_connections 2023-10-08T05:37:04.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f753406ae70 0x7f753406d330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f755015aa00 0x7f7550150b10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f755006db60 0x7f75501505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 --2- 172.21.15.70:0/4071232098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f755006c3a0 0x7f75500c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 -- 172.21.15.70:0/4071232098 >> 172.21.15.70:0/4071232098 conn(0x7f755006a5d0 msgr2=0x7f7550159a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 -- 172.21.15.70:0/4071232098 shutdown_connections 2023-10-08T05:37:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:04.962+0000 7f7558207640 1 -- 172.21.15.70:0/4071232098 wait complete. 2023-10-08T05:37:04.981 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:37:05.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.202+0000 7f397012c640 1 Processor -- start 2023-10-08T05:37:05.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.202+0000 7f397012c640 1 -- start start 2023-10-08T05:37:05.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.203+0000 7f397012c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39680afbe0 0x7f39680b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.203+0000 7f397012c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39680cb130 0x7f39680cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.203+0000 7f397012c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39680b2620 con 0x7f396806dfc0 2023-10-08T05:37:05.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.203+0000 7f397012c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39680cbae0 con 0x7f39680cb130 2023-10-08T05:37:05.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.203+0000 7f397012c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39680b2060 con 0x7f39680afbe0 2023-10-08T05:37:05.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.204+0000 7f396e6c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39680cb130 0x7f39680cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:05.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.204+0000 7f396d6c7640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f396806dfc0 0x7f3968070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33058/0 (socket says 172.21.15.70:33058) 2023-10-08T05:37:05.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.204+0000 7f396d6c7640 1 -- 172.21.15.70:0/86899173 learned_addr learned my addr 172.21.15.70:0/86899173 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:05.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.204+0000 7f396e6c9640 1 -- 172.21.15.70:0/86899173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39680afbe0 msgr2=0x7f39680b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:05.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.204+0000 7f396e6c9640 1 --2- 172.21.15.70:0/86899173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39680afbe0 0x7f39680b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.204+0000 7f396e6c9640 1 -- 172.21.15.70:0/86899173 >> v1:172.21.15.70:6789/0 conn(0x7f396806dfc0 legacy=0x7f3968070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.204+0000 7f396e6c9640 1 -- 172.21.15.70:0/86899173 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39680b2340 con 0x7f39680cb130 2023-10-08T05:37:05.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.204+0000 7f396cec6640 1 -- 172.21.15.70:0/86899173 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230918739 0 0) 0x7f39680b2620 con 0x7f396806dfc0 2023-10-08T05:37:05.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.205+0000 7f396e6c9640 1 --2- 172.21.15.70:0/86899173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39680cb130 0x7f39680cb5a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3960009d90 tx=0x7f396002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faedd9c9bf3725eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:05.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f396cec6640 1 -- 172.21.15.70:0/86899173 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396002cc60 con 0x7f39680cb130 2023-10-08T05:37:05.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f396cec6640 1 -- 172.21.15.70:0/86899173 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396002cdc0 con 0x7f39680cb130 2023-10-08T05:37:05.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f396cec6640 1 -- 172.21.15.70:0/86899173 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960034660 con 0x7f39680cb130 2023-10-08T05:37:05.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f397012c640 1 -- 172.21.15.70:0/86899173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39680cb130 msgr2=0x7f39680cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f397012c640 1 --2- 172.21.15.70:0/86899173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39680cb130 0x7f39680cb5a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3960009d90 tx=0x7f396002beb0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f397012c640 1 -- 172.21.15.70:0/86899173 shutdown_connections 2023-10-08T05:37:05.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f397012c640 1 --2- 172.21.15.70:0/86899173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39680cb130 0x7f39680cb5a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f397012c640 1 --2- 172.21.15.70:0/86899173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39680afbe0 0x7f39680b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f397012c640 1 -- 172.21.15.70:0/86899173 >> 172.21.15.70:0/86899173 conn(0x7f396806a5d0 msgr2=0x7f39680b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:05.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f397012c640 1 -- 172.21.15.70:0/86899173 shutdown_connections 2023-10-08T05:37:05.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.206+0000 7f397012c640 1 -- 172.21.15.70:0/86899173 wait complete. 2023-10-08T05:37:05.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.208+0000 7f397012c640 1 Processor -- start 2023-10-08T05:37:05.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.208+0000 7f397012c640 1 -- start start 2023-10-08T05:37:05.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.209+0000 7f397012c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f396806dfc0 0x7f39680ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.209+0000 7f397012c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39680afbe0 0x7f39680c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.209+0000 7f397012c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39680cb130 0x7f39680c3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.209+0000 7f397012c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39680c9750 con 0x7f396806dfc0 2023-10-08T05:37:05.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.209+0000 7f397012c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39680c9980 con 0x7f39680cb130 2023-10-08T05:37:05.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.209+0000 7f397012c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39680c9c60 con 0x7f39680afbe0 2023-10-08T05:37:05.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.210+0000 7f396dec8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f396806dfc0 0x7f39680ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:05.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.210+0000 7f396dec8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f396806dfc0 0x7f39680ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43602/0 (socket says 172.21.15.70:43602) 2023-10-08T05:37:05.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.210+0000 7f396dec8640 1 -- 172.21.15.70:0/3989040238 learned_addr learned my addr 172.21.15.70:0/3989040238 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.210+0000 7f396dec8640 1 -- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39680afbe0 msgr2=0x7f39680c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.210+0000 7f396dec8640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39680afbe0 0x7f39680c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.210+0000 7f396dec8640 1 -- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39680cb130 msgr2=0x7f39680c3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.210+0000 7f396e6c9640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39680cb130 0x7f39680c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.210+0000 7f396dec8640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39680cb130 0x7f39680c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.210+0000 7f396dec8640 1 -- 172.21.15.70:0/3989040238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3960009a70 con 0x7f396806dfc0 2023-10-08T05:37:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.210+0000 7f396dec8640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f396806dfc0 0x7f39680ca740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f396400b410 tx=0x7f396400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:05.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.211+0000 7f395e7fc640 1 -- 172.21.15.70:0/3989040238 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964011830 con 0x7f396806dfc0 2023-10-08T05:37:05.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.211+0000 7f395e7fc640 1 -- 172.21.15.70:0/3989040238 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39640119b0 con 0x7f396806dfc0 2023-10-08T05:37:05.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.211+0000 7f395e7fc640 1 -- 172.21.15.70:0/3989040238 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964011d60 con 0x7f396806dfc0 2023-10-08T05:37:05.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.211+0000 7f397012c640 1 -- 172.21.15.70:0/3989040238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39680c9f40 con 0x7f396806dfc0 2023-10-08T05:37:05.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.211+0000 7f397012c640 1 -- 172.21.15.70:0/3989040238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39680cac80 con 0x7f396806dfc0 2023-10-08T05:37:05.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.212+0000 7f397012c640 1 -- 172.21.15.70:0/3989040238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39680ca460 con 0x7f396806dfc0 2023-10-08T05:37:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.213+0000 7f395e7fc640 1 -- 172.21.15.70:0/3989040238 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f396401a400 con 0x7f396806dfc0 2023-10-08T05:37:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.214+0000 7f395e7fc640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f394c06aea0 0x7f394c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.214+0000 7f395e7fc640 1 -- 172.21.15.70:0/3989040238 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3964089b20 con 0x7f396806dfc0 2023-10-08T05:37:05.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.215+0000 7f396d6c7640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f394c06aea0 0x7f394c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:05.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.216+0000 7f396d6c7640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f394c06aea0 0x7f394c06d360 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3958007fd0 tx=0x7f3958007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.218+0000 7f395e7fc640 1 -- 172.21.15.70:0/3989040238 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39680ca460 con 0x7f396806dfc0 2023-10-08T05:37:05.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.405+0000 7f397012c640 1 -- 172.21.15.70:0/3989040238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39680ca460 con 0x7f396806dfc0 2023-10-08T05:37:05.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.406+0000 7f395e7fc640 1 -- 172.21.15.70:0/3989040238 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f39680ca460 con 0x7f396806dfc0 2023-10-08T05:37:05.407 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:37:05.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 -- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f394c06aea0 msgr2=0x7f394c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f394c06aea0 0x7f394c06d360 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3958007fd0 tx=0x7f3958007960 comp rx=0 tx=0).stop 2023-10-08T05:37:05.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 -- 172.21.15.70:0/3989040238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f396806dfc0 msgr2=0x7f39680ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f396806dfc0 0x7f39680ca740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f396400b410 tx=0x7f396400b8d0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 -- 172.21.15.70:0/3989040238 shutdown_connections 2023-10-08T05:37:05.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f394c06aea0 0x7f394c06d360 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39680cb130 0x7f39680c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39680afbe0 0x7f39680c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 --2- 172.21.15.70:0/3989040238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f396806dfc0 0x7f39680ca740 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 -- 172.21.15.70:0/3989040238 >> 172.21.15.70:0/3989040238 conn(0x7f396806a5d0 msgr2=0x7f39680c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:05.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 -- 172.21.15.70:0/3989040238 shutdown_connections 2023-10-08T05:37:05.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.408+0000 7f397012c640 1 -- 172.21.15.70:0/3989040238 wait complete. 2023-10-08T05:37:05.428 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:37:05.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.629+0000 7f68b309a640 1 Processor -- start 2023-10-08T05:37:05.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.629+0000 7f68b309a640 1 -- start start 2023-10-08T05:37:05.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.631+0000 7f68b309a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15a820 0x7f68ac0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.631+0000 7f68b309a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac0cab60 con 0x7f68ac15ac00 2023-10-08T05:37:05.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.631+0000 7f68b309a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac0cb960 con 0x7f68ac0ca680 2023-10-08T05:37:05.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.631+0000 7f68b309a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0cbdf0 con 0x7f68ac15a820 2023-10-08T05:37:05.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.631+0000 7f68b2899640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f68ac15ac00 0x7f68ac0c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33066/0 (socket says 172.21.15.70:33066) 2023-10-08T05:37:05.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.631+0000 7f68b2899640 1 -- 172.21.15.70:0/1299181451 learned_addr learned my addr 172.21.15.70:0/1299181451 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:05.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.631+0000 7f68b2098640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15a820 0x7f68ac0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:05.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540850723 0 0) 0x7f68ac0cab60 con 0x7f68ac15ac00 2023-10-08T05:37:05.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6898003520 con 0x7f68ac15ac00 2023-10-08T05:37:05.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2285331734 0 0) 0x7f6898003520 con 0x7f68ac15ac00 2023-10-08T05:37:05.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15a820 msgr2=0x7f68ac0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15a820 0x7f68ac0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 >> v1:172.21.15.181:6789/0 conn(0x7f68ac0ca680 legacy=0x7f68ac0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac0c9f20 con 0x7f68ac15ac00 2023-10-08T05:37:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f68a40019f0 con 0x7f68ac15ac00 2023-10-08T05:37:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 >> v1:172.21.15.70:6789/0 conn(0x7f68ac15ac00 legacy=0x7f68ac0c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 0x7f6898003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15a820 0x7f68980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6898004d00 0x7f6898005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0cbdf0 con 0x7f68980039a0 2023-10-08T05:37:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689800bab0 con 0x7f6898004d00 2023-10-08T05:37:05.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689800bc70 con 0x7f68ac15a820 2023-10-08T05:37:05.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.632+0000 7f68b2098640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 0x7f6898003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:05.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.633+0000 7f68b2899640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6898004d00 0x7f6898005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:05.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.633+0000 7f68b1897640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15a820 0x7f68980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:05.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.633+0000 7f68b2098640 1 -- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15a820 msgr2=0x7f68980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:05.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.633+0000 7f68b2098640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15a820 0x7f68980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.633+0000 7f68b2098640 1 -- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6898004d00 msgr2=0x7f6898005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:05.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.633+0000 7f68b2098640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6898004d00 0x7f6898005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.633+0000 7f68b2098640 1 -- 172.21.15.70:0/1299181451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6898003520 con 0x7f68980039a0 2023-10-08T05:37:05.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.633+0000 7f68b2899640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6898004d00 0x7f6898005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:05.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.633+0000 7f68b1897640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac15a820 0x7f68980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:05.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.634+0000 7f68b2098640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 0x7f6898003dc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009ac0 tx=0x7f68980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c21508aee11d68 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:05.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.634+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a802c600 con 0x7f68980039a0 2023-10-08T05:37:05.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.634+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a802c760 con 0x7f68980039a0 2023-10-08T05:37:05.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.634+0000 7f68b1096640 1 -- 172.21.15.70:0/1299181451 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a802caf0 con 0x7f68980039a0 2023-10-08T05:37:05.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.635+0000 7f68b309a640 1 -- 172.21.15.70:0/1299181451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 msgr2=0x7f6898003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.635+0000 7f68b309a640 1 --2- 172.21.15.70:0/1299181451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 0x7f6898003dc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009ac0 tx=0x7f68980057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.635+0000 7f68b2899640 1 -- 172.21.15.70:0/1299181451 reap_dead start 2023-10-08T05:37:05.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.635+0000 7f68b309a640 1 -- 172.21.15.70:0/1299181451 shutdown_connections 2023-10-08T05:37:05.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.635+0000 7f68b309a640 1 -- 172.21.15.70:0/1299181451 >> 172.21.15.70:0/1299181451 conn(0x7f68ac06c050 msgr2=0x7f68ac06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:05.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.635+0000 7f68b309a640 1 -- 172.21.15.70:0/1299181451 shutdown_connections 2023-10-08T05:37:05.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.635+0000 7f68b309a640 1 -- 172.21.15.70:0/1299181451 wait complete. 2023-10-08T05:37:05.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.638+0000 7f68b309a640 1 Processor -- start 2023-10-08T05:37:05.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.638+0000 7f68b309a640 1 -- start start 2023-10-08T05:37:05.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.639+0000 7f68b309a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 0x7f68ac0ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b309a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac0aec80 0x7f68ac15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b309a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac14ea20 0x7f68ac15d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b309a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0c6c30 con 0x7f68980039a0 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b309a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0c6f10 con 0x7f68ac14ea20 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b309a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac0c71f0 con 0x7f68ac0aec80 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b2098640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 0x7f68ac0ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b2098640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 0x7f68ac0ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43616/0 (socket says 172.21.15.70:43616) 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b2098640 1 -- 172.21.15.70:0/2875553281 learned_addr learned my addr 172.21.15.70:0/2875553281 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b2098640 1 -- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac0aec80 msgr2=0x7f68ac15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b2098640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac0aec80 0x7f68ac15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b2098640 1 -- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac14ea20 msgr2=0x7f68ac15d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b2098640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac14ea20 0x7f68ac15d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b2098640 1 -- 172.21.15.70:0/2875553281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac0c74d0 con 0x7f68980039a0 2023-10-08T05:37:05.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.640+0000 7f68b2098640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 0x7f68ac0ae740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009ac0 tx=0x7f68a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:05.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.641+0000 7f68a2ffd640 1 -- 172.21.15.70:0/2875553281 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8003810 con 0x7f68980039a0 2023-10-08T05:37:05.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.641+0000 7f68a2ffd640 1 -- 172.21.15.70:0/2875553281 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a8003970 con 0x7f68980039a0 2023-10-08T05:37:05.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.641+0000 7f68a2ffd640 1 -- 172.21.15.70:0/2875553281 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8003d00 con 0x7f68980039a0 2023-10-08T05:37:05.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.641+0000 7f68b309a640 1 -- 172.21.15.70:0/2875553281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac0c77b0 con 0x7f68980039a0 2023-10-08T05:37:05.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.641+0000 7f68b309a640 1 -- 172.21.15.70:0/2875553281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac0c6850 con 0x7f68980039a0 2023-10-08T05:37:05.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.642+0000 7f68b309a640 1 -- 172.21.15.70:0/2875553281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac074150 con 0x7f68980039a0 2023-10-08T05:37:05.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.643+0000 7f68a2ffd640 1 -- 172.21.15.70:0/2875553281 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f68a8040450 con 0x7f68980039a0 2023-10-08T05:37:05.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.644+0000 7f68a2ffd640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f689006ae70 0x7f689006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:05.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.644+0000 7f68a2ffd640 1 -- 172.21.15.70:0/2875553281 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f68a80ab940 con 0x7f68980039a0 2023-10-08T05:37:05.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.644+0000 7f68b1897640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f689006ae70 0x7f689006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:05.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.646+0000 7f68b1897640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f689006ae70 0x7f689006d330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68ac156370 tx=0x7f689c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:05.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.648+0000 7f68a2ffd640 1 -- 172.21.15.70:0/2875553281 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68a8076590 con 0x7f68980039a0 2023-10-08T05:37:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.835+0000 7f68b309a640 1 -- 172.21.15.70:0/2875553281 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f68ac14ef00 con 0x7f68980039a0 2023-10-08T05:37:05.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.836+0000 7f68a2ffd640 1 -- 172.21.15.70:0/2875553281 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f68a80044e0 con 0x7f68980039a0 2023-10-08T05:37:05.837 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:05.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 -- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f689006ae70 msgr2=0x7f689006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f689006ae70 0x7f689006d330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68ac156370 tx=0x7f689c009040 comp rx=0 tx=0).stop 2023-10-08T05:37:05.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 -- 172.21.15.70:0/2875553281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 msgr2=0x7f68ac0ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:05.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 0x7f68ac0ae740 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009ac0 tx=0x7f68a8004730 comp rx=0 tx=0).stop 2023-10-08T05:37:05.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 -- 172.21.15.70:0/2875553281 shutdown_connections 2023-10-08T05:37:05.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f689006ae70 0x7f689006d330 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68ac14ea20 0x7f68ac15d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68ac0aec80 0x7f68ac15cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 --2- 172.21.15.70:0/2875553281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68980039a0 0x7f68ac0ae740 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:05.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 -- 172.21.15.70:0/2875553281 >> 172.21.15.70:0/2875553281 conn(0x7f68ac06c050 msgr2=0x7f68ac0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:05.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 -- 172.21.15.70:0/2875553281 shutdown_connections 2023-10-08T05:37:05.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:05.838+0000 7f68b309a640 1 -- 172.21.15.70:0/2875553281 wait complete. 2023-10-08T05:37:05.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:06.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.070+0000 7fc5e7426640 1 Processor -- start 2023-10-08T05:37:06.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.070+0000 7fc5e7426640 1 -- start start 2023-10-08T05:37:06.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.072+0000 7fc5e7426640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5e015ab40 0x7fc5e00ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.072+0000 7fc5e7426640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 0x7fc5e00adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.072+0000 7fc5e7426640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e015b2b0 con 0x7fc5e00c90b0 2023-10-08T05:37:06.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.072+0000 7fc5e7426640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e015bf30 con 0x7fc5e015ab40 2023-10-08T05:37:06.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.072+0000 7fc5e7426640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e015f7a0 con 0x7fc5e00cb580 2023-10-08T05:37:06.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.073+0000 7fc5e51c2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5e015ab40 0x7fc5e00ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.073+0000 7fc5e59c3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 0x7fc5e00adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.073+0000 7fc5e49c1640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fc5e00c90b0 0x7fc5e00cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:34072/0 (socket says 172.21.15.70:34072) 2023-10-08T05:37:06.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.073+0000 7fc5e49c1640 1 -- 172.21.15.70:0/658893546 learned_addr learned my addr 172.21.15.70:0/658893546 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:06.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.073+0000 7fc5d7fff640 1 -- 172.21.15.70:0/658893546 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875231073 0 0) 0x7fc5e015b2b0 con 0x7fc5e00c90b0 2023-10-08T05:37:06.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.074+0000 7fc5d7fff640 1 -- 172.21.15.70:0/658893546 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5cc003520 con 0x7fc5e00c90b0 2023-10-08T05:37:06.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.074+0000 7fc5e51c2640 1 -- 172.21.15.70:0/658893546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 msgr2=0x7fc5e00adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:06.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.074+0000 7fc5e51c2640 1 --2- 172.21.15.70:0/658893546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 0x7fc5e00adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.074+0000 7fc5e51c2640 1 -- 172.21.15.70:0/658893546 >> v1:172.21.15.181:6789/0 conn(0x7fc5e00c90b0 legacy=0x7fc5e00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.074+0000 7fc5e51c2640 1 -- 172.21.15.70:0/658893546 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e015b590 con 0x7fc5e015ab40 2023-10-08T05:37:06.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.074+0000 7fc5e59c3640 1 --2- 172.21.15.70:0/658893546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 0x7fc5e00adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:37:06.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.075+0000 7fc5e51c2640 1 --2- 172.21.15.70:0/658893546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e015ab40 0x7fc5e00ad720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0097d0 tx=0x7fc5dc02b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8986aa3b2a0bbeaa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:06.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.076+0000 7fc5d7fff640 1 -- 172.21.15.70:0/658893546 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc031800 con 0x7fc5e015ab40 2023-10-08T05:37:06.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.076+0000 7fc5d7fff640 1 -- 172.21.15.70:0/658893546 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5dc031960 con 0x7fc5e015ab40 2023-10-08T05:37:06.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.076+0000 7fc5d7fff640 1 -- 172.21.15.70:0/658893546 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc031cf0 con 0x7fc5e015ab40 2023-10-08T05:37:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.076+0000 7fc5e7426640 1 -- 172.21.15.70:0/658893546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e015ab40 msgr2=0x7fc5e00ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.076+0000 7fc5e7426640 1 --2- 172.21.15.70:0/658893546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e015ab40 0x7fc5e00ad720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0097d0 tx=0x7fc5dc02b8a0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.076+0000 7fc5e7426640 1 -- 172.21.15.70:0/658893546 shutdown_connections 2023-10-08T05:37:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.076+0000 7fc5e7426640 1 --2- 172.21.15.70:0/658893546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 0x7fc5e00adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.076+0000 7fc5e7426640 1 --2- 172.21.15.70:0/658893546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e015ab40 0x7fc5e00ad720 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.076+0000 7fc5e7426640 1 -- 172.21.15.70:0/658893546 >> 172.21.15.70:0/658893546 conn(0x7fc5e006a380 msgr2=0x7fc5e00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.077+0000 7fc5e7426640 1 -- 172.21.15.70:0/658893546 shutdown_connections 2023-10-08T05:37:06.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.077+0000 7fc5e7426640 1 -- 172.21.15.70:0/658893546 wait complete. 2023-10-08T05:37:06.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.078+0000 7fc5e7426640 1 Processor -- start 2023-10-08T05:37:06.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.079+0000 7fc5e7426640 1 -- start start 2023-10-08T05:37:06.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e7426640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e00c90b0 0x7fc5e0155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e7426640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 0x7fc5e00b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e7426640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e015ab40 0x7fc5e015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e7426640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e00c6eb0 con 0x7fc5e015ab40 2023-10-08T05:37:06.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e7426640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e00c7750 con 0x7fc5e00c90b0 2023-10-08T05:37:06.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e7426640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e00ac820 con 0x7fc5e00cb580 2023-10-08T05:37:06.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e59c3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e015ab40 0x7fc5e015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e59c3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e015ab40 0x7fc5e015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43628/0 (socket says 172.21.15.70:43628) 2023-10-08T05:37:06.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e59c3640 1 -- 172.21.15.70:0/1205822219 learned_addr learned my addr 172.21.15.70:0/1205822219 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e49c1640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 0x7fc5e00b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e51c2640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e00c90b0 0x7fc5e0155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e59c3640 1 -- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 msgr2=0x7fc5e00b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.080+0000 7fc5e59c3640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 0x7fc5e00b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5e59c3640 1 -- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e00c90b0 msgr2=0x7fc5e0155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5e59c3640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e00c90b0 0x7fc5e0155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5e59c3640 1 -- 172.21.15.70:0/1205822219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8009a70 con 0x7fc5e015ab40 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5e51c2640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e00c90b0 0x7fc5e0155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5e49c1640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 0x7fc5e00b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5e59c3640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e015ab40 0x7fc5e015d4c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800efc0 tx=0x7fc5d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:06.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5d5ffb640 1 -- 172.21.15.70:0/1205822219 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8009cd0 con 0x7fc5e015ab40 2023-10-08T05:37:06.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5d5ffb640 1 -- 172.21.15.70:0/1205822219 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d800f040 con 0x7fc5e015ab40 2023-10-08T05:37:06.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5d5ffb640 1 -- 172.21.15.70:0/1205822219 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8003990 con 0x7fc5e015ab40 2023-10-08T05:37:06.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5e7426640 1 -- 172.21.15.70:0/1205822219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5dc009400 con 0x7fc5e015ab40 2023-10-08T05:37:06.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.081+0000 7fc5e7426640 1 -- 172.21.15.70:0/1205822219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e00c73e0 con 0x7fc5e015ab40 2023-10-08T05:37:06.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.082+0000 7fc5e7426640 1 -- 172.21.15.70:0/1205822219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e00c78c0 con 0x7fc5e015ab40 2023-10-08T05:37:06.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.083+0000 7fc5d5ffb640 1 -- 172.21.15.70:0/1205822219 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc5d8014480 con 0x7fc5e015ab40 2023-10-08T05:37:06.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.084+0000 7fc5d5ffb640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5c406ae70 0x7fc5c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.084+0000 7fc5d5ffb640 1 -- 172.21.15.70:0/1205822219 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc5d808a650 con 0x7fc5e015ab40 2023-10-08T05:37:06.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.085+0000 7fc5e51c2640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5c406ae70 0x7fc5c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.086+0000 7fc5e51c2640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5c406ae70 0x7fc5c406d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc036000 tx=0x7fc5dc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:06.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.089+0000 7fc5d5ffb640 1 -- 172.21.15.70:0/1205822219 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5d80552a0 con 0x7fc5e015ab40 2023-10-08T05:37:06.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.280+0000 7fc5e7426640 1 -- 172.21.15.70:0/1205822219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5e0067490 con 0x7fc5e015ab40 2023-10-08T05:37:06.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.282+0000 7fc5d5ffb640 1 -- 172.21.15.70:0/1205822219 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7fc5d80588c0 con 0x7fc5e015ab40 2023-10-08T05:37:06.284 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:06.284 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 -- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5c406ae70 msgr2=0x7fc5c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5c406ae70 0x7fc5c406d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc036000 tx=0x7fc5dc008030 comp rx=0 tx=0).stop 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 -- 172.21.15.70:0/1205822219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e015ab40 msgr2=0x7fc5e015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e015ab40 0x7fc5e015d4c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800efc0 tx=0x7fc5d800c490 comp rx=0 tx=0).stop 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 -- 172.21.15.70:0/1205822219 shutdown_connections 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5c406ae70 0x7fc5c406d330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e015ab40 0x7fc5e015d4c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e00cb580 0x7fc5e00b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 --2- 172.21.15.70:0/1205822219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e00c90b0 0x7fc5e0155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 -- 172.21.15.70:0/1205822219 >> 172.21.15.70:0/1205822219 conn(0x7fc5e006a380 msgr2=0x7fc5e00cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:06.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 -- 172.21.15.70:0/1205822219 shutdown_connections 2023-10-08T05:37:06.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.284+0000 7fc5e7426640 1 -- 172.21.15.70:0/1205822219 wait complete. 2023-10-08T05:37:06.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:37:06.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.523+0000 7ffb911a5640 1 Processor -- start 2023-10-08T05:37:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.523+0000 7ffb911a5640 1 -- start start 2023-10-08T05:37:06.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.524+0000 7ffb911a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 0x7ffb8c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.524+0000 7ffb911a5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb8c0cb350 0x7ffb8c0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.524+0000 7ffb911a5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c15ae90 con 0x7ffb8c0caef0 2023-10-08T05:37:06.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.524+0000 7ffb911a5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c0cbdb0 con 0x7ffb8c0cb350 2023-10-08T05:37:06.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.524+0000 7ffb911a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0cb7f0 con 0x7ffb8c15aab0 2023-10-08T05:37:06.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.525+0000 7ffb8bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 0x7ffb8c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.525+0000 7ffb909a4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb8c0cb350 0x7ffb8c0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.525+0000 7ffb909a4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb8c0cb350 0x7ffb8c0b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:50610/0 (socket says 172.21.15.70:50610) 2023-10-08T05:37:06.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.525+0000 7ffb8bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 0x7ffb8c070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:56462/0 (socket says 172.21.15.70:56462) 2023-10-08T05:37:06.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.525+0000 7ffb909a4640 1 -- 172.21.15.70:0/2892202735 learned_addr learned my addr 172.21.15.70:0/2892202735 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:06.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.525+0000 7ffb83fff640 1 --1- 172.21.15.70:0/2892202735 >> v1:172.21.15.181:6789/0 conn(0x7ffb8c0caef0 0x7ffb8c0aff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:34086/0 (socket says 172.21.15.70:34086) 2023-10-08T05:37:06.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.526+0000 7ffb909a4640 1 -- 172.21.15.70:0/2892202735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 msgr2=0x7ffb8c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:06.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.526+0000 7ffb909a4640 1 --2- 172.21.15.70:0/2892202735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 0x7ffb8c070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.526+0000 7ffb909a4640 1 -- 172.21.15.70:0/2892202735 >> v1:172.21.15.181:6789/0 conn(0x7ffb8c0caef0 legacy=0x7ffb8c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.526+0000 7ffb909a4640 1 -- 172.21.15.70:0/2892202735 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0cbad0 con 0x7ffb8c0cb350 2023-10-08T05:37:06.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.527+0000 7ffb909a4640 1 --2- 172.21.15.70:0/2892202735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0cb350 0x7ffb8c0b1c60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009e50 tx=0x7ffb8402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9aedbfea73622a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.528+0000 7ffb8b7fe640 1 -- 172.21.15.70:0/2892202735 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8402ca70 con 0x7ffb8c0cb350 2023-10-08T05:37:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.528+0000 7ffb8b7fe640 1 -- 172.21.15.70:0/2892202735 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8402cbd0 con 0x7ffb8c0cb350 2023-10-08T05:37:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.528+0000 7ffb8b7fe640 1 -- 172.21.15.70:0/2892202735 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840343f0 con 0x7ffb8c0cb350 2023-10-08T05:37:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.528+0000 7ffb911a5640 1 -- 172.21.15.70:0/2892202735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0cb350 msgr2=0x7ffb8c0b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.528+0000 7ffb911a5640 1 --2- 172.21.15.70:0/2892202735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0cb350 0x7ffb8c0b1c60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009e50 tx=0x7ffb8402bed0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.529+0000 7ffb911a5640 1 -- 172.21.15.70:0/2892202735 shutdown_connections 2023-10-08T05:37:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.529+0000 7ffb911a5640 1 --2- 172.21.15.70:0/2892202735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0cb350 0x7ffb8c0b1c60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.529+0000 7ffb911a5640 1 --2- 172.21.15.70:0/2892202735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 0x7ffb8c070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.529+0000 7ffb911a5640 1 -- 172.21.15.70:0/2892202735 >> 172.21.15.70:0/2892202735 conn(0x7ffb8c06c010 msgr2=0x7ffb8c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:06.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.529+0000 7ffb911a5640 1 -- 172.21.15.70:0/2892202735 shutdown_connections 2023-10-08T05:37:06.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.529+0000 7ffb911a5640 1 -- 172.21.15.70:0/2892202735 wait complete. 2023-10-08T05:37:06.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.530+0000 7ffb911a5640 1 Processor -- start 2023-10-08T05:37:06.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.531+0000 7ffb911a5640 1 -- start start 2023-10-08T05:37:06.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.531+0000 7ffb911a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0caef0 0x7ffb8c0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.531+0000 7ffb911a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0cb350 0x7ffb8c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.531+0000 7ffb911a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 0x7ffb8c1ee660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb911a5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0cc8c0 con 0x7ffb8c0cb350 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb911a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0ccba0 con 0x7ffb8c0caef0 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb911a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c1ede60 con 0x7ffb8c15aab0 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb83fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0cb350 0x7ffb8c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb83fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0cb350 0x7ffb8c0cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43630/0 (socket says 172.21.15.70:43630) 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb83fff640 1 -- 172.21.15.70:0/354488014 learned_addr learned my addr 172.21.15.70:0/354488014 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb8bfff640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0caef0 0x7ffb8c0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb909a4640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 0x7ffb8c1ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb83fff640 1 -- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 msgr2=0x7ffb8c1ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb83fff640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 0x7ffb8c1ee660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb83fff640 1 -- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0caef0 msgr2=0x7ffb8c0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb83fff640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0caef0 0x7ffb8c0cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb83fff640 1 -- 172.21.15.70:0/354488014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb84009b30 con 0x7ffb8c0cb350 2023-10-08T05:37:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.532+0000 7ffb8bfff640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0caef0 0x7ffb8c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.533+0000 7ffb83fff640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0cb350 0x7ffb8c0cd5c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400c3e0 tx=0x7ffb7400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.533+0000 7ffb897fa640 1 -- 172.21.15.70:0/354488014 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb740074e0 con 0x7ffb8c0cb350 2023-10-08T05:37:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.533+0000 7ffb897fa640 1 -- 172.21.15.70:0/354488014 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb74007640 con 0x7ffb8c0cb350 2023-10-08T05:37:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.533+0000 7ffb897fa640 1 -- 172.21.15.70:0/354488014 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb740049e0 con 0x7ffb8c0cb350 2023-10-08T05:37:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.533+0000 7ffb911a5640 1 -- 172.21.15.70:0/354488014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c1ee140 con 0x7ffb8c0cb350 2023-10-08T05:37:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.533+0000 7ffb911a5640 1 -- 172.21.15.70:0/354488014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c0cc5d0 con 0x7ffb8c0cb350 2023-10-08T05:37:06.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.534+0000 7ffb911a5640 1 -- 172.21.15.70:0/354488014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c074150 con 0x7ffb8c0cb350 2023-10-08T05:37:06.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.535+0000 7ffb897fa640 1 -- 172.21.15.70:0/354488014 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ffb74004b40 con 0x7ffb8c0cb350 2023-10-08T05:37:06.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.536+0000 7ffb897fa640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffb6c06ae70 0x7ffb6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.536+0000 7ffb897fa640 1 -- 172.21.15.70:0/354488014 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7ffb740891a0 con 0x7ffb8c0cb350 2023-10-08T05:37:06.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.537+0000 7ffb8bfff640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffb6c06ae70 0x7ffb6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.538+0000 7ffb8bfff640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffb6c06ae70 0x7ffb6c06d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00d440 tx=0x7ffb7c00f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:06.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.541+0000 7ffb897fa640 1 -- 172.21.15.70:0/354488014 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb74053f20 con 0x7ffb8c0cb350 2023-10-08T05:37:06.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.730+0000 7ffb911a5640 1 -- 172.21.15.70:0/354488014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb8c15ae90 con 0x7ffb8c0cb350 2023-10-08T05:37:06.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.731+0000 7ffb897fa640 1 -- 172.21.15.70:0/354488014 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb8c15ae90 con 0x7ffb8c0cb350 2023-10-08T05:37:06.733 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:06.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 -- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffb6c06ae70 msgr2=0x7ffb6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffb6c06ae70 0x7ffb6c06d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00d440 tx=0x7ffb7c00f040 comp rx=0 tx=0).stop 2023-10-08T05:37:06.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 -- 172.21.15.70:0/354488014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0cb350 msgr2=0x7ffb8c0cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0cb350 0x7ffb8c0cd5c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400c3e0 tx=0x7ffb7400c8a0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 -- 172.21.15.70:0/354488014 shutdown_connections 2023-10-08T05:37:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ffb6c06ae70 0x7ffb6c06d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c15aab0 0x7ffb8c1ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0cb350 0x7ffb8c0cd5c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 --2- 172.21.15.70:0/354488014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0caef0 0x7ffb8c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 -- 172.21.15.70:0/354488014 >> 172.21.15.70:0/354488014 conn(0x7ffb8c06c010 msgr2=0x7ffb8c0aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:06.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 -- 172.21.15.70:0/354488014 shutdown_connections 2023-10-08T05:37:06.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.734+0000 7ffb911a5640 1 -- 172.21.15.70:0/354488014 wait complete. 2023-10-08T05:37:06.755 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:37:06.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.973+0000 7fce7c97b640 1 Processor -- start 2023-10-08T05:37:06.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.973+0000 7fce7c97b640 1 -- start start 2023-10-08T05:37:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.974+0000 7fce7c97b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce780b0680 0x7fce7815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.974+0000 7fce7c97b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce780cab60 con 0x7fce780ca680 2023-10-08T05:37:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.974+0000 7fce7c97b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce780cb960 con 0x7fce7815a820 2023-10-08T05:37:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.974+0000 7fce7c97b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce780cbdf0 con 0x7fce780b0680 2023-10-08T05:37:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.975+0000 7fce7759e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fce780ca680 0x7fce7815ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33076/0 (socket says 172.21.15.70:33076) 2023-10-08T05:37:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.975+0000 7fce7759e640 1 -- 172.21.15.70:0/4194356320 learned_addr learned my addr 172.21.15.70:0/4194356320 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.975+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155962113 0 0) 0x7fce780cab60 con 0x7fce780ca680 2023-10-08T05:37:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.975+0000 7fce6ffff640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce780b0680 0x7fce7815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.975+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce64003520 con 0x7fce780ca680 2023-10-08T05:37:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2918113612 0 0) 0x7fce64003520 con 0x7fce780ca680 2023-10-08T05:37:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce780b0680 msgr2=0x7fce7815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce780b0680 0x7fce7815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 >> v1:172.21.15.181:6789/0 conn(0x7fce7815a820 legacy=0x7fce780aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780cb030 con 0x7fce780ca680 2023-10-08T05:37:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fce700019f0 con 0x7fce780ca680 2023-10-08T05:37:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce6ffff640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce780b0680 0x7fce7815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:37:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 >> v1:172.21.15.70:6789/0 conn(0x7fce780ca680 legacy=0x7fce7815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce780b0680 0x7fce640039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce64003f90 0x7fce640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce64004d00 0x7fce64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780cbdf0 con 0x7fce64003f90 2023-10-08T05:37:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6400bb70 con 0x7fce64004d00 2023-10-08T05:37:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6400bd30 con 0x7fce780b0680 2023-10-08T05:37:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce6ffff640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce64003f90 0x7fce640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce76d9d640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce780b0680 0x7fce640039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce7759e640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce64004d00 0x7fce64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce6ffff640 1 -- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce780b0680 msgr2=0x7fce640039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.976+0000 7fce6ffff640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce780b0680 0x7fce640039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.977+0000 7fce6ffff640 1 -- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce64004d00 msgr2=0x7fce64005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.977+0000 7fce6ffff640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce64004d00 0x7fce64005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.977+0000 7fce6ffff640 1 -- 172.21.15.70:0/4194356320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64003520 con 0x7fce64003f90 2023-10-08T05:37:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.977+0000 7fce7759e640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce64004d00 0x7fce64005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:06.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.978+0000 7fce6ffff640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce64003f90 0x7fce640047c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fce600098a0 tx=0x7fce640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52384d9e94269f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.978+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60004230 con 0x7fce64003f90 2023-10-08T05:37:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.978+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce60004390 con 0x7fce64003f90 2023-10-08T05:37:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.978+0000 7fce7659c640 1 -- 172.21.15.70:0/4194356320 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60003830 con 0x7fce64003f90 2023-10-08T05:37:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.979+0000 7fce7c97b640 1 -- 172.21.15.70:0/4194356320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce64003f90 msgr2=0x7fce640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.979+0000 7fce7c97b640 1 --2- 172.21.15.70:0/4194356320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce64003f90 0x7fce640047c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fce600098a0 tx=0x7fce640057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.979+0000 7fce7759e640 1 -- 172.21.15.70:0/4194356320 reap_dead start 2023-10-08T05:37:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.979+0000 7fce7c97b640 1 -- 172.21.15.70:0/4194356320 shutdown_connections 2023-10-08T05:37:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.979+0000 7fce7c97b640 1 -- 172.21.15.70:0/4194356320 >> 172.21.15.70:0/4194356320 conn(0x7fce7806c050 msgr2=0x7fce7806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.979+0000 7fce7c97b640 1 -- 172.21.15.70:0/4194356320 shutdown_connections 2023-10-08T05:37:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.979+0000 7fce7c97b640 1 -- 172.21.15.70:0/4194356320 wait complete. 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.981+0000 7fce7c97b640 1 Processor -- start 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.981+0000 7fce7c97b640 1 -- start start 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce7c97b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce64003f90 0x7fce780c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce7c97b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce7815ae70 0x7fce7815d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce7c97b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce7815b2d0 0x7fce780c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce7c97b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780c7fb0 con 0x7fce7815ae70 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce7c97b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780c75e0 con 0x7fce64003f90 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce7c97b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce780c69d0 con 0x7fce7815b2d0 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce6ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce7815ae70 0x7fce7815d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce6ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce7815ae70 0x7fce7815d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43648/0 (socket says 172.21.15.70:43648) 2023-10-08T05:37:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce6ffff640 1 -- 172.21.15.70:0/3188344784 learned_addr learned my addr 172.21.15.70:0/3188344784 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce76d9d640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce64003f90 0x7fce780c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce6ffff640 1 -- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce7815b2d0 msgr2=0x7fce780c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce6ffff640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce7815b2d0 0x7fce780c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce6ffff640 1 -- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce64003f90 msgr2=0x7fce780c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce6ffff640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce64003f90 0x7fce780c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.982+0000 7fce6ffff640 1 -- 172.21.15.70:0/3188344784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780c6cb0 con 0x7fce7815ae70 2023-10-08T05:37:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.983+0000 7fce6ffff640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce7815ae70 0x7fce7815d610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fce600098a0 tx=0x7fce60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.983+0000 7fce6f7fe640 1 -- 172.21.15.70:0/3188344784 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60030630 con 0x7fce7815ae70 2023-10-08T05:37:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.983+0000 7fce6f7fe640 1 -- 172.21.15.70:0/3188344784 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce60030790 con 0x7fce7815ae70 2023-10-08T05:37:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.983+0000 7fce7c97b640 1 -- 172.21.15.70:0/3188344784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce780c6f90 con 0x7fce7815ae70 2023-10-08T05:37:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.983+0000 7fce6f7fe640 1 -- 172.21.15.70:0/3188344784 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60030b00 con 0x7fce7815ae70 2023-10-08T05:37:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.983+0000 7fce7c97b640 1 -- 172.21.15.70:0/3188344784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce780c6610 con 0x7fce7815ae70 2023-10-08T05:37:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.984+0000 7fce7c97b640 1 -- 172.21.15.70:0/3188344784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce780cac60 con 0x7fce7815ae70 2023-10-08T05:37:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.985+0000 7fce6f7fe640 1 -- 172.21.15.70:0/3188344784 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fce60030c60 con 0x7fce7815ae70 2023-10-08T05:37:06.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.986+0000 7fce6f7fe640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce5806aea0 0x7fce5806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:06.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.986+0000 7fce6f7fe640 1 -- 172.21.15.70:0/3188344784 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fce600ab750 con 0x7fce7815ae70 2023-10-08T05:37:06.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.987+0000 7fce76d9d640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce5806aea0 0x7fce5806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:06.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.988+0000 7fce76d9d640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce5806aea0 0x7fce5806d360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce68001590 tx=0x7fce68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:06.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:06.990+0000 7fce6f7fe640 1 -- 172.21.15.70:0/3188344784 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce60076580 con 0x7fce7815ae70 2023-10-08T05:37:07.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.181+0000 7fce7c97b640 1 -- 172.21.15.70:0/3188344784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce78074150 con 0x7fce7815ae70 2023-10-08T05:37:07.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.182+0000 7fce6f7fe640 1 -- 172.21.15.70:0/3188344784 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce6000f040 con 0x7fce7815ae70 2023-10-08T05:37:07.183 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:07.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.184+0000 7fce7c97b640 1 -- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce5806aea0 msgr2=0x7fce5806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:07.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.184+0000 7fce7c97b640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce5806aea0 0x7fce5806d360 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fce68001590 tx=0x7fce68009040 comp rx=0 tx=0).stop 2023-10-08T05:37:07.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.184+0000 7fce7c97b640 1 -- 172.21.15.70:0/3188344784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce7815ae70 msgr2=0x7fce7815d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.184+0000 7fce7c97b640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce7815ae70 0x7fce7815d610 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fce600098a0 tx=0x7fce60004730 comp rx=0 tx=0).stop 2023-10-08T05:37:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.184+0000 7fce7c97b640 1 -- 172.21.15.70:0/3188344784 shutdown_connections 2023-10-08T05:37:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.184+0000 7fce7c97b640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fce5806aea0 0x7fce5806d360 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.184+0000 7fce7c97b640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce7815b2d0 0x7fce780c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.184+0000 7fce7c97b640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce7815ae70 0x7fce7815d610 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.184+0000 7fce7c97b640 1 --2- 172.21.15.70:0/3188344784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce64003f90 0x7fce780c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.185+0000 7fce7c97b640 1 -- 172.21.15.70:0/3188344784 >> 172.21.15.70:0/3188344784 conn(0x7fce7806c050 msgr2=0x7fce780ce390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.185+0000 7fce7c97b640 1 -- 172.21.15.70:0/3188344784 shutdown_connections 2023-10-08T05:37:07.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.185+0000 7fce7c97b640 1 -- 172.21.15.70:0/3188344784 wait complete. 2023-10-08T05:37:07.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:37:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.419+0000 7f6b6ebc5640 1 Processor -- start 2023-10-08T05:37:07.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.419+0000 7f6b6ebc5640 1 -- start start 2023-10-08T05:37:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.420+0000 7f6b6ebc5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b6806c3a0 0x7f6b680c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.420+0000 7f6b6ebc5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6b6806db60 0x7f6b6806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.420+0000 7f6b6ebc5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b6815aa00 0x7f6b680b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.420+0000 7f6b6ebc5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b680ca230 con 0x7f6b6815aa00 2023-10-08T05:37:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.420+0000 7f6b6ebc5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6815b310 con 0x7f6b6806db60 2023-10-08T05:37:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.420+0000 7f6b6ebc5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6806d290 con 0x7f6b6806c3a0 2023-10-08T05:37:07.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.421+0000 7f6b6d162640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b6815aa00 0x7f6b680b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.421+0000 7f6b6d162640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b6815aa00 0x7f6b680b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43660/0 (socket says 172.21.15.70:43660) 2023-10-08T05:37:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.421+0000 7f6b6d162640 1 -- 172.21.15.70:0/252809436 learned_addr learned my addr 172.21.15.70:0/252809436 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:07.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.421+0000 7f6b67fff640 1 --2- 172.21.15.70:0/252809436 >> v2:172.21.15.181:3300/0 conn(0x7f6b6806db60 0x7f6b6806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.421+0000 7f6b6c961640 1 --2- 172.21.15.70:0/252809436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b6806c3a0 0x7f6b680c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.421+0000 7f6b6d162640 1 -- 172.21.15.70:0/252809436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b6806c3a0 msgr2=0x7f6b680c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.421+0000 7f6b6d162640 1 --2- 172.21.15.70:0/252809436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b6806c3a0 0x7f6b680c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.421+0000 7f6b6d162640 1 -- 172.21.15.70:0/252809436 >> v2:172.21.15.181:3300/0 conn(0x7f6b6806db60 msgr2=0x7f6b6806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.421+0000 7f6b6d162640 1 --2- 172.21.15.70:0/252809436 >> v2:172.21.15.181:3300/0 conn(0x7f6b6806db60 0x7f6b6806c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.421+0000 7f6b6d162640 1 -- 172.21.15.70:0/252809436 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680c9860 con 0x7f6b6815aa00 2023-10-08T05:37:07.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.422+0000 7f6b6d162640 1 --2- 172.21.15.70:0/252809436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6815aa00 0x7f6b680b2a90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6b680714d0 tx=0x7f6b6002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4360a69def344f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b677fe640 1 -- 172.21.15.70:0/252809436 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60031860 con 0x7f6b6815aa00 2023-10-08T05:37:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b677fe640 1 -- 172.21.15.70:0/252809436 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b600319c0 con 0x7f6b6815aa00 2023-10-08T05:37:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b677fe640 1 -- 172.21.15.70:0/252809436 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60031d50 con 0x7f6b6815aa00 2023-10-08T05:37:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/252809436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6815aa00 msgr2=0x7f6b680b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:07.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b6ebc5640 1 --2- 172.21.15.70:0/252809436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6815aa00 0x7f6b680b2a90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6b680714d0 tx=0x7f6b6002b900 comp rx=0 tx=0).stop 2023-10-08T05:37:07.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/252809436 shutdown_connections 2023-10-08T05:37:07.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b6ebc5640 1 --2- 172.21.15.70:0/252809436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6815aa00 0x7f6b680b2a90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b6ebc5640 1 --2- 172.21.15.70:0/252809436 >> v2:172.21.15.181:3300/0 conn(0x7f6b6806db60 0x7f6b6806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b6ebc5640 1 --2- 172.21.15.70:0/252809436 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b6806c3a0 0x7f6b680c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/252809436 >> 172.21.15.70:0/252809436 conn(0x7f6b6806a5d0 msgr2=0x7f6b680b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:07.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/252809436 shutdown_connections 2023-10-08T05:37:07.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.423+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/252809436 wait complete. 2023-10-08T05:37:07.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.425+0000 7f6b6ebc5640 1 Processor -- start 2023-10-08T05:37:07.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.425+0000 7f6b6ebc5640 1 -- start start 2023-10-08T05:37:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.426+0000 7f6b6ebc5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6806c3a0 0x7f6b681573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.426+0000 7f6b6ebc5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b6806db60 0x7f6b68164db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.426+0000 7f6b6ebc5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b6815aa00 0x7f6b681469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.426+0000 7f6b6ebc5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68165ce0 con 0x7f6b6806c3a0 2023-10-08T05:37:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.426+0000 7f6b6ebc5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68157900 con 0x7f6b6806db60 2023-10-08T05:37:07.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.426+0000 7f6b6ebc5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68165720 con 0x7f6b6815aa00 2023-10-08T05:37:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.426+0000 7f6b6c961640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6806c3a0 0x7f6b681573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.426+0000 7f6b6c961640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6806c3a0 0x7f6b681573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43672/0 (socket says 172.21.15.70:43672) 2023-10-08T05:37:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.426+0000 7f6b6c961640 1 -- 172.21.15.70:0/1034606572 learned_addr learned my addr 172.21.15.70:0/1034606572 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b67fff640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b6806db60 0x7f6b68164db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b6d162640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b6815aa00 0x7f6b681469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b6c961640 1 -- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b6815aa00 msgr2=0x7f6b681469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b6c961640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b6815aa00 0x7f6b681469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b6c961640 1 -- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b6806db60 msgr2=0x7f6b68164db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b6c961640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b6806db60 0x7f6b68164db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b6c961640 1 -- 172.21.15.70:0/1034606572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600094d0 con 0x7f6b6806c3a0 2023-10-08T05:37:07.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b6c961640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6806c3a0 0x7f6b681573c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800ce40 tx=0x7f6b5800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b657fa640 1 -- 172.21.15.70:0/1034606572 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58003b90 con 0x7f6b6806c3a0 2023-10-08T05:37:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b657fa640 1 -- 172.21.15.70:0/1034606572 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b58003cf0 con 0x7f6b6806c3a0 2023-10-08T05:37:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/1034606572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b68165a00 con 0x7f6b6806c3a0 2023-10-08T05:37:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.427+0000 7f6b657fa640 1 -- 172.21.15.70:0/1034606572 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58004260 con 0x7f6b6806c3a0 2023-10-08T05:37:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.428+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/1034606572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6815c920 con 0x7f6b6806c3a0 2023-10-08T05:37:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.429+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/1034606572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6815c4a0 con 0x7f6b6806c3a0 2023-10-08T05:37:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.429+0000 7f6b657fa640 1 -- 172.21.15.70:0/1034606572 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6b580043c0 con 0x7f6b6806c3a0 2023-10-08T05:37:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.430+0000 7f6b657fa640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6b4806ae70 0x7f6b4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.430+0000 7f6b657fa640 1 -- 172.21.15.70:0/1034606572 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6b58012070 con 0x7f6b6806c3a0 2023-10-08T05:37:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.431+0000 7f6b67fff640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6b4806ae70 0x7f6b4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.433+0000 7f6b67fff640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6b4806ae70 0x7f6b4806d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b50006fd0 tx=0x7f6b50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.434+0000 7f6b657fa640 1 -- 172.21.15.70:0/1034606572 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b6815c4a0 con 0x7f6b6806c3a0 2023-10-08T05:37:07.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.623+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/1034606572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b6815c4a0 con 0x7f6b6806c3a0 2023-10-08T05:37:07.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.624+0000 7f6b657fa640 1 -- 172.21.15.70:0/1034606572 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b6815c4a0 con 0x7f6b6806c3a0 2023-10-08T05:37:07.625 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:37:07.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6b4806ae70 msgr2=0x7f6b4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:07.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6b4806ae70 0x7f6b4806d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b50006fd0 tx=0x7f6b50008040 comp rx=0 tx=0).stop 2023-10-08T05:37:07.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/1034606572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6806c3a0 msgr2=0x7f6b681573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:07.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6806c3a0 0x7f6b681573c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b5800ce40 tx=0x7f6b5800a660 comp rx=0 tx=0).stop 2023-10-08T05:37:07.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/1034606572 shutdown_connections 2023-10-08T05:37:07.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6b4806ae70 0x7f6b4806d330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b6815aa00 0x7f6b681469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b6806db60 0x7f6b68164db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 --2- 172.21.15.70:0/1034606572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b6806c3a0 0x7f6b681573c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/1034606572 >> 172.21.15.70:0/1034606572 conn(0x7f6b6806a5d0 msgr2=0x7f6b680b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:07.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/1034606572 shutdown_connections 2023-10-08T05:37:07.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.627+0000 7f6b6ebc5640 1 -- 172.21.15.70:0/1034606572 wait complete. 2023-10-08T05:37:07.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:37:07.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.861+0000 7f5f1ca90640 1 Processor -- start 2023-10-08T05:37:07.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.861+0000 7f5f1ca90640 1 -- start start 2023-10-08T05:37:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.863+0000 7f5f1ca90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f1806c3a0 0x7f5f180c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.863+0000 7f5f1ca90640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5f1806db60 0x7f5f1806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.863+0000 7f5f1ca90640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1815b310 con 0x7f5f1815aa00 2023-10-08T05:37:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.863+0000 7f5f1ca90640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f180ca230 con 0x7f5f1806db60 2023-10-08T05:37:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.863+0000 7f5f1ca90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1806d290 con 0x7f5f1806c3a0 2023-10-08T05:37:07.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.863+0000 7f5f0dd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5f1806db60 0x7f5f1806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.863+0000 7f5f0dd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5f1806db60 0x7f5f1806c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43676/0 (socket says 172.21.15.70:43676) 2023-10-08T05:37:07.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.863+0000 7f5f0dd9b640 1 -- 172.21.15.70:0/1205281713 learned_addr learned my addr 172.21.15.70:0/1205281713 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:07.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.864+0000 7f5f1659c640 1 --2- 172.21.15.70:0/1205281713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f1806c3a0 0x7f5f180c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.864+0000 7f5f0dd9b640 1 -- 172.21.15.70:0/1205281713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f1806c3a0 msgr2=0x7f5f180c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:07.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.864+0000 7f5f0dd9b640 1 --2- 172.21.15.70:0/1205281713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f1806c3a0 0x7f5f180c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.864+0000 7f5f0dd9b640 1 -- 172.21.15.70:0/1205281713 >> v1:172.21.15.181:6789/0 conn(0x7f5f1815aa00 legacy=0x7f5f180b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:07.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.864+0000 7f5f0dd9b640 1 -- 172.21.15.70:0/1205281713 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f180c9860 con 0x7f5f1806db60 2023-10-08T05:37:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.865+0000 7f5f0dd9b640 1 --2- 172.21.15.70:0/1205281713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 0x7f5f1806c780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000bb10 tx=0x7f5f0002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8e2d048e22ffc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.865+0000 7f5f15d9b640 1 -- 172.21.15.70:0/1205281713 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0002ee70 con 0x7f5f1806db60 2023-10-08T05:37:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.865+0000 7f5f15d9b640 1 -- 172.21.15.70:0/1205281713 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f00036440 con 0x7f5f1806db60 2023-10-08T05:37:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.865+0000 7f5f15d9b640 1 -- 172.21.15.70:0/1205281713 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f000367d0 con 0x7f5f1806db60 2023-10-08T05:37:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.866+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1205281713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 msgr2=0x7f5f1806c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.866+0000 7f5f1ca90640 1 --2- 172.21.15.70:0/1205281713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 0x7f5f1806c780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000bb10 tx=0x7f5f0002e0c0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.866+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1205281713 shutdown_connections 2023-10-08T05:37:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.866+0000 7f5f1ca90640 1 --2- 172.21.15.70:0/1205281713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 0x7f5f1806c780 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.866+0000 7f5f1ca90640 1 --2- 172.21.15.70:0/1205281713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f1806c3a0 0x7f5f180c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.866+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1205281713 >> 172.21.15.70:0/1205281713 conn(0x7f5f1806a5d0 msgr2=0x7f5f180b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.866+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1205281713 shutdown_connections 2023-10-08T05:37:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.866+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1205281713 wait complete. 2023-10-08T05:37:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.868+0000 7f5f1ca90640 1 Processor -- start 2023-10-08T05:37:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.868+0000 7f5f1ca90640 1 -- start start 2023-10-08T05:37:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.869+0000 7f5f1ca90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f1806c3a0 0x7f5f1815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.869+0000 7f5f1ca90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 0x7f5f181456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.869+0000 7f5f1ca90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f1815aa00 0x7f5f18145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.869+0000 7f5f1ca90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180cd840 con 0x7f5f1806db60 2023-10-08T05:37:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.869+0000 7f5f1ca90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f181453f0 con 0x7f5f1806c3a0 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.869+0000 7f5f1ca90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f180c2ac0 con 0x7f5f1815aa00 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.869+0000 7f5f0dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 0x7f5f181456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.869+0000 7f5f0dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 0x7f5f181456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43688/0 (socket says 172.21.15.70:43688) 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.869+0000 7f5f0dd9b640 1 -- 172.21.15.70:0/1236812985 learned_addr learned my addr 172.21.15.70:0/1236812985 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f0dd9b640 1 -- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f1815aa00 msgr2=0x7f5f18145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f0dd9b640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f1815aa00 0x7f5f18145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f1659c640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f1806c3a0 0x7f5f1815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f0dd9b640 1 -- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f1806c3a0 msgr2=0x7f5f1815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f0dd9b640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f1806c3a0 0x7f5f1815dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f0dd9b640 1 -- 172.21.15.70:0/1236812985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0000b770 con 0x7f5f1806db60 2023-10-08T05:37:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f0dd9b640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 0x7f5f181456b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f0002e100 tx=0x7f5f000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f0f7fe640 1 -- 172.21.15.70:0/1236812985 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f000348d0 con 0x7f5f1806db60 2023-10-08T05:37:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f0f7fe640 1 -- 172.21.15.70:0/1236812985 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f00034a50 con 0x7f5f1806db60 2023-10-08T05:37:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f0f7fe640 1 -- 172.21.15.70:0/1236812985 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00034e00 con 0x7f5f1806db60 2023-10-08T05:37:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1236812985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1814ba60 con 0x7f5f1806db60 2023-10-08T05:37:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.870+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1236812985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f180c27e0 con 0x7f5f1806db60 2023-10-08T05:37:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.871+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1236812985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f180cd050 con 0x7f5f1806db60 2023-10-08T05:37:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.872+0000 7f5f0f7fe640 1 -- 172.21.15.70:0/1236812985 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5f0003d070 con 0x7f5f1806db60 2023-10-08T05:37:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.873+0000 7f5f0f7fe640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ef806aea0 0x7f5ef806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.873+0000 7f5f0f7fe640 1 -- 172.21.15.70:0/1236812985 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f5f000ac0e0 con 0x7f5f1806db60 2023-10-08T05:37:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.874+0000 7f5f1659c640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ef806aea0 0x7f5ef806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.875+0000 7f5f1659c640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ef806aea0 0x7f5ef806d360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:07.877+0000 7f5f0f7fe640 1 -- 172.21.15.70:0/1236812985 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f180cd050 con 0x7f5f1806db60 2023-10-08T05:37:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.070+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1236812985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5f1815b310 con 0x7f5f1806db60 2023-10-08T05:37:08.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.071+0000 7f5f0f7fe640 1 -- 172.21.15.70:0/1236812985 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5f1815b310 con 0x7f5f1806db60 2023-10-08T05:37:08.072 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.073+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ef806aea0 msgr2=0x7f5ef806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.073+0000 7f5f1ca90640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ef806aea0 0x7f5ef806d360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08008040 comp rx=0 tx=0).stop 2023-10-08T05:37:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.073+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1236812985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 msgr2=0x7f5f181456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.073+0000 7f5f1ca90640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 0x7f5f181456b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5f0002e100 tx=0x7f5f000046c0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.073+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1236812985 shutdown_connections 2023-10-08T05:37:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.073+0000 7f5f1ca90640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ef806aea0 0x7f5ef806d360 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.073+0000 7f5f1ca90640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f1815aa00 0x7f5f18145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.073+0000 7f5f1ca90640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f1806db60 0x7f5f181456b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.073+0000 7f5f1ca90640 1 --2- 172.21.15.70:0/1236812985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f1806c3a0 0x7f5f1815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.073+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1236812985 >> 172.21.15.70:0/1236812985 conn(0x7f5f1806a5d0 msgr2=0x7f5f18159880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:08.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.074+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1236812985 shutdown_connections 2023-10-08T05:37:08.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.074+0000 7f5f1ca90640 1 -- 172.21.15.70:0/1236812985 wait complete. 2023-10-08T05:37:08.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:08.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.298+0000 7f5600e00640 1 Processor -- start 2023-10-08T05:37:08.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.298+0000 7f5600e00640 1 -- start start 2023-10-08T05:37:08.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.300+0000 7f5600e00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55fc15ab40 0x7f55fc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.300+0000 7f5600e00640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55fc0cb580 0x7f55fc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.300+0000 7f5600e00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55fc15b2b0 con 0x7f55fc0c90b0 2023-10-08T05:37:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.300+0000 7f5600e00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc15bf30 con 0x7f55fc15ab40 2023-10-08T05:37:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.300+0000 7f5600e00640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc15f7a0 con 0x7f55fc0cb580 2023-10-08T05:37:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.300+0000 7f55fa59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55fc15ab40 0x7f55fc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.300+0000 7f55fa59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55fc15ab40 0x7f55fc0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43704/0 (socket says 172.21.15.70:43704) 2023-10-08T05:37:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.300+0000 7f55fa59c640 1 -- 172.21.15.70:0/695838761 learned_addr learned my addr 172.21.15.70:0/695838761 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.301+0000 7f55fa59c640 1 -- 172.21.15.70:0/695838761 >> v2:172.21.15.181:3300/0 conn(0x7f55fc0cb580 msgr2=0x7f55fc0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.301+0000 7f55fa59c640 1 --2- 172.21.15.70:0/695838761 >> v2:172.21.15.181:3300/0 conn(0x7f55fc0cb580 0x7f55fc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.301+0000 7f55fa59c640 1 -- 172.21.15.70:0/695838761 >> v1:172.21.15.70:6789/0 conn(0x7f55fc0c90b0 legacy=0x7f55fc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.301+0000 7f55fa59c640 1 -- 172.21.15.70:0/695838761 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc15b590 con 0x7f55fc15ab40 2023-10-08T05:37:08.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.302+0000 7f55fa59c640 1 --2- 172.21.15.70:0/695838761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc15ab40 0x7f55fc0ad720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0b0ec0 tx=0x7f55ec02b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c677a5f818873dc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:08.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.302+0000 7f55f9d9b640 1 -- 172.21.15.70:0/695838761 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec030880 con 0x7f55fc15ab40 2023-10-08T05:37:08.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.302+0000 7f55f9d9b640 1 -- 172.21.15.70:0/695838761 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec0309e0 con 0x7f55fc15ab40 2023-10-08T05:37:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.302+0000 7f55f9d9b640 1 -- 172.21.15.70:0/695838761 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec030d70 con 0x7f55fc15ab40 2023-10-08T05:37:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.303+0000 7f5600e00640 1 -- 172.21.15.70:0/695838761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc15ab40 msgr2=0x7f55fc0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.303+0000 7f5600e00640 1 --2- 172.21.15.70:0/695838761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc15ab40 0x7f55fc0ad720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0b0ec0 tx=0x7f55ec02b9c0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.303+0000 7f5600e00640 1 -- 172.21.15.70:0/695838761 shutdown_connections 2023-10-08T05:37:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.303+0000 7f5600e00640 1 --2- 172.21.15.70:0/695838761 >> v2:172.21.15.181:3300/0 conn(0x7f55fc0cb580 0x7f55fc0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.303+0000 7f5600e00640 1 --2- 172.21.15.70:0/695838761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc15ab40 0x7f55fc0ad720 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.303+0000 7f5600e00640 1 -- 172.21.15.70:0/695838761 >> 172.21.15.70:0/695838761 conn(0x7f55fc06a380 msgr2=0x7f55fc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.303+0000 7f5600e00640 1 -- 172.21.15.70:0/695838761 shutdown_connections 2023-10-08T05:37:08.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.303+0000 7f5600e00640 1 -- 172.21.15.70:0/695838761 wait complete. 2023-10-08T05:37:08.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.305+0000 7f5600e00640 1 Processor -- start 2023-10-08T05:37:08.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.305+0000 7f5600e00640 1 -- start start 2023-10-08T05:37:08.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.306+0000 7f5600e00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0c90b0 0x7f55fc154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.306+0000 7f5600e00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc0cb580 0x7f55fc155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.306+0000 7f5600e00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc15ab40 0x7f55fc0c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.306+0000 7f5600e00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc154de0 con 0x7f55fc0c90b0 2023-10-08T05:37:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.306+0000 7f5600e00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc156310 con 0x7f55fc0cb580 2023-10-08T05:37:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.306+0000 7f5600e00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc0c7c70 con 0x7f55fc15ab40 2023-10-08T05:37:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.306+0000 7f55fa59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0c90b0 0x7f55fc154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.306+0000 7f55fa59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0c90b0 0x7f55fc154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43710/0 (socket says 172.21.15.70:43710) 2023-10-08T05:37:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.306+0000 7f55fa59c640 1 -- 172.21.15.70:0/3629219626 learned_addr learned my addr 172.21.15.70:0/3629219626 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55f3fff640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc0cb580 0x7f55fc155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55fad9d640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc15ab40 0x7f55fc0c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:08.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55fa59c640 1 -- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc15ab40 msgr2=0x7f55fc0c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55fa59c640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc15ab40 0x7f55fc0c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55fa59c640 1 -- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc0cb580 msgr2=0x7f55fc155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55fa59c640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc0cb580 0x7f55fc155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55fa59c640 1 -- 172.21.15.70:0/3629219626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec0094d0 con 0x7f55fc0c90b0 2023-10-08T05:37:08.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55f3fff640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc0cb580 0x7f55fc155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:08.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55fa59c640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0c90b0 0x7f55fc154f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55ec000c00 tx=0x7f55ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:08.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55f2ffd640 1 -- 172.21.15.70:0/3629219626 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec030850 con 0x7f55fc0c90b0 2023-10-08T05:37:08.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55f2ffd640 1 -- 172.21.15.70:0/3629219626 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec039d90 con 0x7f55fc0c90b0 2023-10-08T05:37:08.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f55f2ffd640 1 -- 172.21.15.70:0/3629219626 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec043630 con 0x7f55fc0c90b0 2023-10-08T05:37:08.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.307+0000 7f5600e00640 1 -- 172.21.15.70:0/3629219626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc0c7f50 con 0x7f55fc0c90b0 2023-10-08T05:37:08.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.308+0000 7f5600e00640 1 -- 172.21.15.70:0/3629219626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc0c58f0 con 0x7f55fc0c90b0 2023-10-08T05:37:08.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.309+0000 7f5600e00640 1 -- 172.21.15.70:0/3629219626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc067490 con 0x7f55fc0c90b0 2023-10-08T05:37:08.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.309+0000 7f55f2ffd640 1 -- 172.21.15.70:0/3629219626 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f55ec02f3d0 con 0x7f55fc0c90b0 2023-10-08T05:37:08.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.310+0000 7f55f2ffd640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f55dc06ae70 0x7f55dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.310+0000 7f55f2ffd640 1 -- 172.21.15.70:0/3629219626 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f55ec0abf70 con 0x7f55fc0c90b0 2023-10-08T05:37:08.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.311+0000 7f55f3fff640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f55dc06ae70 0x7f55dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:08.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.312+0000 7f55f3fff640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f55dc06ae70 0x7f55dc06d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0ad6f0 tx=0x7f55e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:08.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.315+0000 7f55f2ffd640 1 -- 172.21.15.70:0/3629219626 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55ec076c50 con 0x7f55fc0c90b0 2023-10-08T05:37:08.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.504+0000 7f5600e00640 1 -- 172.21.15.70:0/3629219626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55fc15b840 con 0x7f55fc0c90b0 2023-10-08T05:37:08.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.507+0000 7f55f2ffd640 1 -- 172.21.15.70:0/3629219626 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7f55ec048090 con 0x7f55fc0c90b0 2023-10-08T05:37:08.509 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:08.509 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 -- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f55dc06ae70 msgr2=0x7f55dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f55dc06ae70 0x7f55dc06d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0ad6f0 tx=0x7f55e4009040 comp rx=0 tx=0).stop 2023-10-08T05:37:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 -- 172.21.15.70:0/3629219626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0c90b0 msgr2=0x7f55fc154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0c90b0 0x7f55fc154f70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f55ec000c00 tx=0x7f55ec004730 comp rx=0 tx=0).stop 2023-10-08T05:37:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 -- 172.21.15.70:0/3629219626 shutdown_connections 2023-10-08T05:37:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f55dc06ae70 0x7f55dc06d330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55fc15ab40 0x7f55fc0c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55fc0cb580 0x7f55fc155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 --2- 172.21.15.70:0/3629219626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55fc0c90b0 0x7f55fc154f70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 -- 172.21.15.70:0/3629219626 >> 172.21.15.70:0/3629219626 conn(0x7f55fc06a380 msgr2=0x7f55fc06cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:08.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 -- 172.21.15.70:0/3629219626 shutdown_connections 2023-10-08T05:37:08.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.510+0000 7f5600e00640 1 -- 172.21.15.70:0/3629219626 wait complete. 2023-10-08T05:37:08.531 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:37:08.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:08.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.746+0000 7f89db1d7640 1 Processor -- start 2023-10-08T05:37:08.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.746+0000 7f89db1d7640 1 -- start start 2023-10-08T05:37:08.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.747+0000 7f89db1d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f89d415aad0 0x7f89d40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.748+0000 7f89db1d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d415bf30 con 0x7f89d40c82b0 2023-10-08T05:37:08.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.748+0000 7f89db1d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d415b320 con 0x7f89d406d410 2023-10-08T05:37:08.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.748+0000 7f89db1d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d415b600 con 0x7f89d415aad0 2023-10-08T05:37:08.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.748+0000 7f89d8f73640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f89d415aad0 0x7f89d40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:08.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.748+0000 7f89d3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f89d40c82b0 0x7f89d406cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33098/0 (socket says 172.21.15.70:33098) 2023-10-08T05:37:08.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.748+0000 7f89d8f73640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f89d415aad0 0x7f89d40b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43726/0 (socket says 172.21.15.70:43726) 2023-10-08T05:37:08.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.748+0000 7f89d3fff640 1 -- 172.21.15.70:0/3303702037 learned_addr learned my addr 172.21.15.70:0/3303702037 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:08.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.748+0000 7f89d8f73640 1 -- 172.21.15.70:0/3303702037 >> v1:172.21.15.181:6789/0 conn(0x7f89d406d410 legacy=0x7f89d4070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:08.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.748+0000 7f89d8f73640 1 -- 172.21.15.70:0/3303702037 >> v1:172.21.15.70:6789/0 conn(0x7f89d40c82b0 legacy=0x7f89d406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.748+0000 7f89d8f73640 1 -- 172.21.15.70:0/3303702037 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d415b8e0 con 0x7f89d415aad0 2023-10-08T05:37:08.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.750+0000 7f89d8f73640 1 --2- 172.21.15.70:0/3303702037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d415aad0 0x7f89d40b1750 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009d90 tx=0x7f89c802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e98a47dd44e643c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.750+0000 7f89d37fe640 1 -- 172.21.15.70:0/3303702037 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c802cc60 con 0x7f89d415aad0 2023-10-08T05:37:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.750+0000 7f89d37fe640 1 -- 172.21.15.70:0/3303702037 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c802cdc0 con 0x7f89d415aad0 2023-10-08T05:37:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.750+0000 7f89d37fe640 1 -- 172.21.15.70:0/3303702037 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8034660 con 0x7f89d415aad0 2023-10-08T05:37:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.750+0000 7f89db1d7640 1 -- 172.21.15.70:0/3303702037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d415aad0 msgr2=0x7f89d40b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.750+0000 7f89db1d7640 1 --2- 172.21.15.70:0/3303702037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d415aad0 0x7f89d40b1750 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f89c8009d90 tx=0x7f89c802beb0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.751+0000 7f89db1d7640 1 -- 172.21.15.70:0/3303702037 shutdown_connections 2023-10-08T05:37:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.751+0000 7f89db1d7640 1 --2- 172.21.15.70:0/3303702037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d415aad0 0x7f89d40b1750 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.751+0000 7f89db1d7640 1 -- 172.21.15.70:0/3303702037 >> 172.21.15.70:0/3303702037 conn(0x7f89d4069d50 msgr2=0x7f89d406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.751+0000 7f89db1d7640 1 -- 172.21.15.70:0/3303702037 shutdown_connections 2023-10-08T05:37:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.751+0000 7f89db1d7640 1 -- 172.21.15.70:0/3303702037 wait complete. 2023-10-08T05:37:08.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.753+0000 7f89db1d7640 1 Processor -- start 2023-10-08T05:37:08.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.753+0000 7f89db1d7640 1 -- start start 2023-10-08T05:37:08.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.754+0000 7f89db1d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d406d410 0x7f89d4158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.754+0000 7f89db1d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d40c82b0 0x7f89d41524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.754+0000 7f89db1d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d415aad0 0x7f89d40cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.754+0000 7f89db1d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4159380 con 0x7f89d40c82b0 2023-10-08T05:37:08.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.754+0000 7f89db1d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d414fd10 con 0x7f89d406d410 2023-10-08T05:37:08.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.754+0000 7f89db1d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d4150480 con 0x7f89d415aad0 2023-10-08T05:37:08.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.754+0000 7f89d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d40c82b0 0x7f89d41524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.754+0000 7f89d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d40c82b0 0x7f89d41524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43742/0 (socket says 172.21.15.70:43742) 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.754+0000 7f89d3fff640 1 -- 172.21.15.70:0/1701051957 learned_addr learned my addr 172.21.15.70:0/1701051957 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d9774640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d415aad0 0x7f89d40cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d8f73640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d406d410 0x7f89d4158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d3fff640 1 -- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d415aad0 msgr2=0x7f89d40cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d3fff640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d415aad0 0x7f89d40cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d3fff640 1 -- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d406d410 msgr2=0x7f89d4158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d3fff640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d406d410 0x7f89d4158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d3fff640 1 -- 172.21.15.70:0/1701051957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c8009a70 con 0x7f89d40c82b0 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d9774640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d415aad0 0x7f89d40cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:08.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d8f73640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d406d410 0x7f89d4158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:08.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d3fff640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d40c82b0 0x7f89d41524b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c450 tx=0x7f89c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:08.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d17fa640 1 -- 172.21.15.70:0/1701051957 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c0005ce0 con 0x7f89d40c82b0 2023-10-08T05:37:08.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d17fa640 1 -- 172.21.15.70:0/1701051957 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c0005e40 con 0x7f89d40c82b0 2023-10-08T05:37:08.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89d17fa640 1 -- 172.21.15.70:0/1701051957 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c00072b0 con 0x7f89d40c82b0 2023-10-08T05:37:08.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89db1d7640 1 -- 172.21.15.70:0/1701051957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d4150760 con 0x7f89d40c82b0 2023-10-08T05:37:08.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.755+0000 7f89db1d7640 1 -- 172.21.15.70:0/1701051957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d4150d20 con 0x7f89d40c82b0 2023-10-08T05:37:08.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.757+0000 7f89db1d7640 1 -- 172.21.15.70:0/1701051957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d4150f40 con 0x7f89d40c82b0 2023-10-08T05:37:08.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.758+0000 7f89d17fa640 1 -- 172.21.15.70:0/1701051957 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f89c0007410 con 0x7f89d40c82b0 2023-10-08T05:37:08.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.758+0000 7f89d17fa640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f89b806ae70 0x7f89b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:08.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.759+0000 7f89d17fa640 1 -- 172.21.15.70:0/1701051957 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f89c0088e80 con 0x7f89d40c82b0 2023-10-08T05:37:08.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.759+0000 7f89d8f73640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f89b806ae70 0x7f89b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:08.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.760+0000 7f89d8f73640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f89b806ae70 0x7f89b806d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f89c8036000 tx=0x7f89c8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:08.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.763+0000 7f89d17fa640 1 -- 172.21.15.70:0/1701051957 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f89c0053c00 con 0x7f89d40c82b0 2023-10-08T05:37:08.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.951+0000 7f89db1d7640 1 -- 172.21.15.70:0/1701051957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89d415db00 con 0x7f89d40c82b0 2023-10-08T05:37:08.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.952+0000 7f89d17fa640 1 -- 172.21.15.70:0/1701051957 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f89c0017020 con 0x7f89d40c82b0 2023-10-08T05:37:08.953 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:08.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.954+0000 7f89db1d7640 1 -- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f89b806ae70 msgr2=0x7f89b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.954+0000 7f89db1d7640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f89b806ae70 0x7f89b806d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f89c8036000 tx=0x7f89c8030000 comp rx=0 tx=0).stop 2023-10-08T05:37:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.954+0000 7f89db1d7640 1 -- 172.21.15.70:0/1701051957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d40c82b0 msgr2=0x7f89d41524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.954+0000 7f89db1d7640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d40c82b0 0x7f89d41524b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f89c000c450 tx=0x7f89c000c910 comp rx=0 tx=0).stop 2023-10-08T05:37:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.955+0000 7f89db1d7640 1 -- 172.21.15.70:0/1701051957 shutdown_connections 2023-10-08T05:37:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.955+0000 7f89db1d7640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f89b806ae70 0x7f89b806d330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.955+0000 7f89db1d7640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89d415aad0 0x7f89d40cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.955+0000 7f89db1d7640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89d40c82b0 0x7f89d41524b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.955+0000 7f89db1d7640 1 --2- 172.21.15.70:0/1701051957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89d406d410 0x7f89d4158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.955+0000 7f89db1d7640 1 -- 172.21.15.70:0/1701051957 >> 172.21.15.70:0/1701051957 conn(0x7f89d4069d50 msgr2=0x7f89d406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.955+0000 7f89db1d7640 1 -- 172.21.15.70:0/1701051957 shutdown_connections 2023-10-08T05:37:08.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:08.955+0000 7f89db1d7640 1 -- 172.21.15.70:0/1701051957 wait complete. 2023-10-08T05:37:08.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:37:09.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.194+0000 7f1033c96640 1 Processor -- start 2023-10-08T05:37:09.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.194+0000 7f1033c96640 1 -- start start 2023-10-08T05:37:09.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.195+0000 7f1033c96640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f102c06dfc0 0x7f102c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.196+0000 7f1033c96640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f102c071070 0x7f102c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.196+0000 7f1033c96640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102c0b1cc0 con 0x7f102c0afbe0 2023-10-08T05:37:09.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.196+0000 7f1033c96640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102c0b2620 con 0x7f102c06dfc0 2023-10-08T05:37:09.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.196+0000 7f1033c96640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102c0b2060 con 0x7f102c071070 2023-10-08T05:37:09.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.196+0000 7f1031a32640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f102c0afbe0 0x7f102c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33108/0 (socket says 172.21.15.70:33108) 2023-10-08T05:37:09.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.196+0000 7f1031a32640 1 -- 172.21.15.70:0/1426529303 learned_addr learned my addr 172.21.15.70:0/1426529303 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.196+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260388953 0 0) 0x7f102c0b1cc0 con 0x7f102c0afbe0 2023-10-08T05:37:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.196+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1018003520 con 0x7f102c0afbe0 2023-10-08T05:37:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.196+0000 7f1031231640 1 --2- 172.21.15.70:0/1426529303 >> v2:172.21.15.181:3300/0 conn(0x7f102c06dfc0 0x7f102c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.196+0000 7f1032233640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f102c071070 0x7f102c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2469412971 0 0) 0x7f1018003520 con 0x7f102c0afbe0 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f102c071070 msgr2=0x7f102c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f102c071070 0x7f102c0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 >> v2:172.21.15.181:3300/0 conn(0x7f102c06dfc0 msgr2=0x7f102c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 --2- 172.21.15.70:0/1426529303 >> v2:172.21.15.181:3300/0 conn(0x7f102c06dfc0 0x7f102c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c0b2340 con 0x7f102c0afbe0 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1028001670 con 0x7f102c0afbe0 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 >> v1:172.21.15.70:6789/0 conn(0x7f102c0afbe0 legacy=0x7f102c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1018003970 0x7f1018003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f102c071070 0x7f10180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1018004d00 0x7f1018005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c0b2620 con 0x7f1018004d00 2023-10-08T05:37:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c0b2060 con 0x7f1018003970 2023-10-08T05:37:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101800b980 con 0x7f102c071070 2023-10-08T05:37:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1032233640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1018004d00 0x7f1018005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.197+0000 7f1031a32640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1018003970 0x7f1018003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.198+0000 7f1031231640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f102c071070 0x7f10180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.198+0000 7f1032233640 1 -- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f102c071070 msgr2=0x7f10180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.198+0000 7f1032233640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f102c071070 0x7f10180047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.198+0000 7f1032233640 1 -- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1018003970 msgr2=0x7f1018003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.198+0000 7f1032233640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1018003970 0x7f1018003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.198+0000 7f1032233640 1 -- 172.21.15.70:0/1426529303 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1018003520 con 0x7f1018004d00 2023-10-08T05:37:09.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.199+0000 7f1032233640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1018004d00 0x7f1018005180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1024009f90 tx=0x7f10180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8669c98efcbac5c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:09.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.199+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102402cbd0 con 0x7f1018004d00 2023-10-08T05:37:09.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.199+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102400f040 con 0x7f1018004d00 2023-10-08T05:37:09.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.199+0000 7f1030a30640 1 -- 172.21.15.70:0/1426529303 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024034440 con 0x7f1018004d00 2023-10-08T05:37:09.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.199+0000 7f1033c96640 1 -- 172.21.15.70:0/1426529303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1018004d00 msgr2=0x7f1018005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.200+0000 7f1033c96640 1 --2- 172.21.15.70:0/1426529303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1018004d00 0x7f1018005180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1024009f90 tx=0x7f10180057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.200+0000 7f1032233640 1 -- 172.21.15.70:0/1426529303 reap_dead start 2023-10-08T05:37:09.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.200+0000 7f1033c96640 1 -- 172.21.15.70:0/1426529303 shutdown_connections 2023-10-08T05:37:09.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.200+0000 7f1033c96640 1 -- 172.21.15.70:0/1426529303 >> 172.21.15.70:0/1426529303 conn(0x7f102c06a5d0 msgr2=0x7f102c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:09.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.200+0000 7f1033c96640 1 -- 172.21.15.70:0/1426529303 shutdown_connections 2023-10-08T05:37:09.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.200+0000 7f1033c96640 1 -- 172.21.15.70:0/1426529303 wait complete. 2023-10-08T05:37:09.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.201+0000 7f1033c96640 1 Processor -- start 2023-10-08T05:37:09.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.202+0000 7f1033c96640 1 -- start start 2023-10-08T05:37:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.202+0000 7f1033c96640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1018004d00 0x7f102c0c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1033c96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102c15d1f0 0x7f102c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1033c96640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f102c155590 0x7f102c155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1033c96640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c1f26c0 con 0x7f102c15d1f0 2023-10-08T05:37:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1033c96640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c0c22d0 con 0x7f102c155590 2023-10-08T05:37:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1033c96640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102c15dcb0 con 0x7f1018004d00 2023-10-08T05:37:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1031231640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102c15d1f0 0x7f102c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1031231640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102c15d1f0 0x7f102c15d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43766/0 (socket says 172.21.15.70:43766) 2023-10-08T05:37:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1031231640 1 -- 172.21.15.70:0/2271861521 learned_addr learned my addr 172.21.15.70:0/2271861521 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1031231640 1 -- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1018004d00 msgr2=0x7f102c0c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1031231640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1018004d00 0x7f102c0c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1031231640 1 -- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f102c155590 msgr2=0x7f102c155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1031231640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f102c155590 0x7f102c155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1031231640 1 -- 172.21.15.70:0/2271861521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c15df90 con 0x7f102c15d1f0 2023-10-08T05:37:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.203+0000 7f1031231640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102c15d1f0 0x7f102c15d670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f101c0108d0 tx=0x7f101c010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.204+0000 7f10227fc640 1 -- 172.21.15.70:0/2271861521 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c009650 con 0x7f102c15d1f0 2023-10-08T05:37:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.204+0000 7f10227fc640 1 -- 172.21.15.70:0/2271861521 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c00eb00 con 0x7f102c15d1f0 2023-10-08T05:37:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.204+0000 7f10227fc640 1 -- 172.21.15.70:0/2271861521 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c00ee90 con 0x7f102c15d1f0 2023-10-08T05:37:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.204+0000 7f1033c96640 1 -- 172.21.15.70:0/2271861521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102c0c2570 con 0x7f102c15d1f0 2023-10-08T05:37:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.204+0000 7f1033c96640 1 -- 172.21.15.70:0/2271861521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102c15e270 con 0x7f102c15d1f0 2023-10-08T05:37:09.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.205+0000 7f1033c96640 1 -- 172.21.15.70:0/2271861521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f102c04e160 con 0x7f102c15d1f0 2023-10-08T05:37:09.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.206+0000 7f10227fc640 1 -- 172.21.15.70:0/2271861521 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f101c01c400 con 0x7f102c15d1f0 2023-10-08T05:37:09.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.207+0000 7f10227fc640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006aea0 0x7f101006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.207+0000 7f10227fc640 1 -- 172.21.15.70:0/2271861521 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f101c089f60 con 0x7f102c15d1f0 2023-10-08T05:37:09.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.208+0000 7f1031a32640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006aea0 0x7f101006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:09.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.209+0000 7f1031a32640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006aea0 0x7f101006d360 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1028001050 tx=0x7f1028000fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:09.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.211+0000 7f10227fc640 1 -- 172.21.15.70:0/2271861521 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f101c054bb0 con 0x7f102c15d1f0 2023-10-08T05:37:09.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.399+0000 7f1033c96640 1 -- 172.21.15.70:0/2271861521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f102c0cad80 con 0x7f102c15d1f0 2023-10-08T05:37:09.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.401+0000 7f10227fc640 1 -- 172.21.15.70:0/2271861521 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f101c0581d0 con 0x7f102c15d1f0 2023-10-08T05:37:09.402 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:09.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.403+0000 7f1033c96640 1 -- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006aea0 msgr2=0x7f101006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.403+0000 7f1033c96640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006aea0 0x7f101006d360 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1028001050 tx=0x7f1028000fc0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.403+0000 7f1033c96640 1 -- 172.21.15.70:0/2271861521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102c15d1f0 msgr2=0x7f102c15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.403+0000 7f1033c96640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102c15d1f0 0x7f102c15d670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f101c0108d0 tx=0x7f101c010d90 comp rx=0 tx=0).stop 2023-10-08T05:37:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.404+0000 7f1033c96640 1 -- 172.21.15.70:0/2271861521 shutdown_connections 2023-10-08T05:37:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.404+0000 7f1033c96640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f101006aea0 0x7f101006d360 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.404+0000 7f1033c96640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f102c155590 0x7f102c155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.404+0000 7f1033c96640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f102c15d1f0 0x7f102c15d670 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.404+0000 7f1033c96640 1 --2- 172.21.15.70:0/2271861521 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1018004d00 0x7f102c0c9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.404+0000 7f1033c96640 1 -- 172.21.15.70:0/2271861521 >> 172.21.15.70:0/2271861521 conn(0x7f102c06a5d0 msgr2=0x7f102c15eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.404+0000 7f1033c96640 1 -- 172.21.15.70:0/2271861521 shutdown_connections 2023-10-08T05:37:09.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.404+0000 7f1033c96640 1 -- 172.21.15.70:0/2271861521 wait complete. 2023-10-08T05:37:09.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:37:09.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.643+0000 7fa925f2e640 1 Processor -- start 2023-10-08T05:37:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.643+0000 7fa925f2e640 1 -- start start 2023-10-08T05:37:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.644+0000 7fa925f2e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9200b0680 0x7fa9200c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.644+0000 7fa925f2e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9200c94f0 con 0x7fa9200c8b40 2023-10-08T05:37:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.644+0000 7fa925f2e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9200ccbc0 con 0x7fa92015cae0 2023-10-08T05:37:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.644+0000 7fa925f2e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9200ce1f0 con 0x7fa9200b0680 2023-10-08T05:37:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.645+0000 7fa91ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa9200c8b40 0x7fa9200aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33118/0 (socket says 172.21.15.70:33118) 2023-10-08T05:37:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.645+0000 7fa91ffff640 1 -- 172.21.15.70:0/2188406957 learned_addr learned my addr 172.21.15.70:0/2188406957 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.645+0000 7fa92572d640 1 --2- 172.21.15.70:0/2188406957 >> v2:172.21.15.181:3300/0 conn(0x7fa9200b0680 0x7fa9200c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.645+0000 7fa91f7fe640 1 -- 172.21.15.70:0/2188406957 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925769031 0 0) 0x7fa9200c94f0 con 0x7fa9200c8b40 2023-10-08T05:37:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.645+0000 7fa91f7fe640 1 -- 172.21.15.70:0/2188406957 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa908003520 con 0x7fa9200c8b40 2023-10-08T05:37:09.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.645+0000 7fa92572d640 1 -- 172.21.15.70:0/2188406957 >> v1:172.21.15.181:6789/0 conn(0x7fa92015cae0 legacy=0x7fa9200cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.645+0000 7fa92572d640 1 -- 172.21.15.70:0/2188406957 >> v1:172.21.15.70:6789/0 conn(0x7fa9200c8b40 legacy=0x7fa9200aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.646+0000 7fa91f7fe640 1 -- 172.21.15.70:0/2188406957 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2412664971 0 0) 0x7fa908003520 con 0x7fa9200c8b40 2023-10-08T05:37:09.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.646+0000 7fa92572d640 1 -- 172.21.15.70:0/2188406957 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9200c9720 con 0x7fa9200b0680 2023-10-08T05:37:09.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.647+0000 7fa92572d640 1 --2- 172.21.15.70:0/2188406957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9200b0680 0x7fa9200c8fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa9180097a0 tx=0x7fa91802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6110cc87a789355 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:09.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.647+0000 7fa91f7fe640 1 -- 172.21.15.70:0/2188406957 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91802f050 con 0x7fa9200b0680 2023-10-08T05:37:09.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.647+0000 7fa91f7fe640 1 -- 172.21.15.70:0/2188406957 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa918031500 con 0x7fa9200b0680 2023-10-08T05:37:09.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.647+0000 7fa91f7fe640 1 -- 172.21.15.70:0/2188406957 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914001670 con 0x7fa9200b0680 2023-10-08T05:37:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.648+0000 7fa925f2e640 1 -- 172.21.15.70:0/2188406957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9200b0680 msgr2=0x7fa9200c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.648+0000 7fa925f2e640 1 --2- 172.21.15.70:0/2188406957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9200b0680 0x7fa9200c8fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa9180097a0 tx=0x7fa91802b820 comp rx=0 tx=0).stop 2023-10-08T05:37:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.648+0000 7fa925f2e640 1 -- 172.21.15.70:0/2188406957 shutdown_connections 2023-10-08T05:37:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.648+0000 7fa925f2e640 1 --2- 172.21.15.70:0/2188406957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9200b0680 0x7fa9200c8fb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.648+0000 7fa925f2e640 1 -- 172.21.15.70:0/2188406957 >> 172.21.15.70:0/2188406957 conn(0x7fa92006c050 msgr2=0x7fa92006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.648+0000 7fa925f2e640 1 -- 172.21.15.70:0/2188406957 shutdown_connections 2023-10-08T05:37:09.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.648+0000 7fa925f2e640 1 -- 172.21.15.70:0/2188406957 wait complete. 2023-10-08T05:37:09.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.650+0000 7fa925f2e640 1 Processor -- start 2023-10-08T05:37:09.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.650+0000 7fa925f2e640 1 -- start start 2023-10-08T05:37:09.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.651+0000 7fa925f2e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9200b0680 0x7fa9201f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.651+0000 7fa925f2e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9200c8b40 0x7fa920154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.651+0000 7fa925f2e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015cae0 0x7fa920154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.651+0000 7fa925f2e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92015e4a0 con 0x7fa9200c8b40 2023-10-08T05:37:09.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.651+0000 7fa925f2e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9200c3a30 con 0x7fa92015cae0 2023-10-08T05:37:09.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.651+0000 7fa925f2e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9200c3750 con 0x7fa9200b0680 2023-10-08T05:37:09.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.651+0000 7fa91ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9200c8b40 0x7fa920154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:09.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.651+0000 7fa91ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9200c8b40 0x7fa920154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43778/0 (socket says 172.21.15.70:43778) 2023-10-08T05:37:09.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.651+0000 7fa91ffff640 1 -- 172.21.15.70:0/605126544 learned_addr learned my addr 172.21.15.70:0/605126544 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:09.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa91ffff640 1 -- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9200b0680 msgr2=0x7fa9201f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa91ffff640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9200b0680 0x7fa9201f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa91ffff640 1 -- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015cae0 msgr2=0x7fa920154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa91ffff640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015cae0 0x7fa920154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa91ffff640 1 -- 172.21.15.70:0/605126544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa918009400 con 0x7fa9200c8b40 2023-10-08T05:37:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa91ffff640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9200c8b40 0x7fa920154650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa9140011e0 tx=0x7fa91400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:09.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa91d7fa640 1 -- 172.21.15.70:0/605126544 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914005ad0 con 0x7fa9200c8b40 2023-10-08T05:37:09.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa91d7fa640 1 -- 172.21.15.70:0/605126544 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa914005c50 con 0x7fa9200c8b40 2023-10-08T05:37:09.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa91d7fa640 1 -- 172.21.15.70:0/605126544 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9140072b0 con 0x7fa9200c8b40 2023-10-08T05:37:09.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa925f2e640 1 -- 172.21.15.70:0/605126544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92015e6b0 con 0x7fa9200c8b40 2023-10-08T05:37:09.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.652+0000 7fa925f2e640 1 -- 172.21.15.70:0/605126544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92015ec70 con 0x7fa9200c8b40 2023-10-08T05:37:09.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.653+0000 7fa925f2e640 1 -- 172.21.15.70:0/605126544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa920074150 con 0x7fa9200c8b40 2023-10-08T05:37:09.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.654+0000 7fa91d7fa640 1 -- 172.21.15.70:0/605126544 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa914007410 con 0x7fa9200c8b40 2023-10-08T05:37:09.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.655+0000 7fa91d7fa640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa90c06ae70 0x7fa90c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:09.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.655+0000 7fa91d7fa640 1 -- 172.21.15.70:0/605126544 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa9140891b0 con 0x7fa9200c8b40 2023-10-08T05:37:09.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.656+0000 7fa924f2c640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa90c06ae70 0x7fa90c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:09.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.657+0000 7fa924f2c640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa90c06ae70 0x7fa90c06d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9100012e0 tx=0x7fa910007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:09.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.659+0000 7fa91d7fa640 1 -- 172.21.15.70:0/605126544 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa914053e00 con 0x7fa9200c8b40 2023-10-08T05:37:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.849+0000 7fa925f2e640 1 -- 172.21.15.70:0/605126544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9200b4470 con 0x7fa9200c8b40 2023-10-08T05:37:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.850+0000 7fa91d7fa640 1 -- 172.21.15.70:0/605126544 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa914057420 con 0x7fa9200c8b40 2023-10-08T05:37:09.852 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:37:09.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.852+0000 7fa925f2e640 1 -- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa90c06ae70 msgr2=0x7fa90c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.852+0000 7fa925f2e640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa90c06ae70 0x7fa90c06d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa9100012e0 tx=0x7fa910007d30 comp rx=0 tx=0).stop 2023-10-08T05:37:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.852+0000 7fa925f2e640 1 -- 172.21.15.70:0/605126544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9200c8b40 msgr2=0x7fa920154650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.852+0000 7fa925f2e640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9200c8b40 0x7fa920154650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa9140011e0 tx=0x7fa91400cbe0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.853+0000 7fa925f2e640 1 -- 172.21.15.70:0/605126544 shutdown_connections 2023-10-08T05:37:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.853+0000 7fa925f2e640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa90c06ae70 0x7fa90c06d330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.853+0000 7fa925f2e640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa92015cae0 0x7fa920154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.853+0000 7fa925f2e640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9200c8b40 0x7fa920154650 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.853+0000 7fa925f2e640 1 --2- 172.21.15.70:0/605126544 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9200b0680 0x7fa9201f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.853+0000 7fa925f2e640 1 -- 172.21.15.70:0/605126544 >> 172.21.15.70:0/605126544 conn(0x7fa92006c050 msgr2=0x7fa920155310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.853+0000 7fa925f2e640 1 -- 172.21.15.70:0/605126544 shutdown_connections 2023-10-08T05:37:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:09.853+0000 7fa925f2e640 1 -- 172.21.15.70:0/605126544 wait complete. 2023-10-08T05:37:09.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:37:10.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.083+0000 7fb74aa97640 1 Processor -- start 2023-10-08T05:37:10.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.083+0000 7fb74aa97640 1 -- start start 2023-10-08T05:37:10.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.085+0000 7fb74aa97640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7440afbe0 0x7fb7440b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.085+0000 7fb74aa97640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb74406dfc0 0x7fb744070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.085+0000 7fb74aa97640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7440b2620 con 0x7fb744071070 2023-10-08T05:37:10.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.085+0000 7fb74aa97640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7440b2060 con 0x7fb74406dfc0 2023-10-08T05:37:10.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.085+0000 7fb74aa97640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440b2340 con 0x7fb7440afbe0 2023-10-08T05:37:10.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.085+0000 7fb749034640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb744071070 0x7fb7440cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33128/0 (socket says 172.21.15.70:33128) 2023-10-08T05:37:10.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.085+0000 7fb749034640 1 -- 172.21.15.70:0/2052100167 learned_addr learned my addr 172.21.15.70:0/2052100167 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:10.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb743fff640 1 --2- 172.21.15.70:0/2052100167 >> v2:172.21.15.181:3300/0 conn(0x7fb74406dfc0 0x7fb744070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb748833640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7440afbe0 0x7fb7440b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304103607 0 0) 0x7fb7440b2620 con 0x7fb744071070 2023-10-08T05:37:10.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb734003520 con 0x7fb744071070 2023-10-08T05:37:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2467663864 0 0) 0x7fb734003520 con 0x7fb744071070 2023-10-08T05:37:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7440afbe0 msgr2=0x7fb7440b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7440afbe0 0x7fb7440b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 >> v2:172.21.15.181:3300/0 conn(0x7fb74406dfc0 msgr2=0x7fb744070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 --2- 172.21.15.70:0/2052100167 >> v2:172.21.15.181:3300/0 conn(0x7fb74406dfc0 0x7fb744070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744070350 con 0x7fb744071070 2023-10-08T05:37:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fb73c0019f0 con 0x7fb744071070 2023-10-08T05:37:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 >> v1:172.21.15.70:6789/0 conn(0x7fb744071070 legacy=0x7fb7440cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb734003970 0x7fb734003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.086+0000 7fb7437fe640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7340047c0 0x7fb734004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb7437fe640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7440afbe0 0x7fb734005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440b2060 con 0x7fb7340047c0 2023-10-08T05:37:10.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440b2340 con 0x7fb734003970 2023-10-08T05:37:10.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73400b980 con 0x7fb7440afbe0 2023-10-08T05:37:10.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb743fff640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7340047c0 0x7fb734004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb748833640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb734003970 0x7fb734003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb749034640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7440afbe0 0x7fb734005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb743fff640 1 -- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7440afbe0 msgr2=0x7fb734005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:10.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb743fff640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7440afbe0 0x7fb734005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb743fff640 1 -- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb734003970 msgr2=0x7fb734003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:10.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb743fff640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb734003970 0x7fb734003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.087+0000 7fb743fff640 1 -- 172.21.15.70:0/2052100167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb734003520 con 0x7fb7340047c0 2023-10-08T05:37:10.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.088+0000 7fb743fff640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7340047c0 0x7fb734004c40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb730009ac0 tx=0x7fb7340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8feebc0ec56b3bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:10.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.089+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73002c710 con 0x7fb7340047c0 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.089+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73002c870 con 0x7fb7340047c0 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.089+0000 7fb7437fe640 1 -- 172.21.15.70:0/2052100167 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73002cc00 con 0x7fb7340047c0 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.089+0000 7fb74aa97640 1 -- 172.21.15.70:0/2052100167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7340047c0 msgr2=0x7fb734004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.089+0000 7fb74aa97640 1 --2- 172.21.15.70:0/2052100167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7340047c0 0x7fb734004c40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb730009ac0 tx=0x7fb7340057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.089+0000 7fb749034640 1 -- 172.21.15.70:0/2052100167 reap_dead start 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.089+0000 7fb74aa97640 1 -- 172.21.15.70:0/2052100167 shutdown_connections 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.089+0000 7fb74aa97640 1 -- 172.21.15.70:0/2052100167 >> 172.21.15.70:0/2052100167 conn(0x7fb74406a5d0 msgr2=0x7fb7440b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.089+0000 7fb74aa97640 1 -- 172.21.15.70:0/2052100167 shutdown_connections 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.089+0000 7fb74aa97640 1 -- 172.21.15.70:0/2052100167 wait complete. 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.091+0000 7fb74aa97640 1 Processor -- start 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.091+0000 7fb74aa97640 1 -- start start 2023-10-08T05:37:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.092+0000 7fb74aa97640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7340047c0 0x7fb7441f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.092+0000 7fb74aa97640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb744152dd0 0x7fb7441554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.092+0000 7fb74aa97640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7441f5d00 0x7fb7441f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.092+0000 7fb74aa97640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440c3c90 con 0x7fb744152dd0 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.092+0000 7fb74aa97640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441f66c0 con 0x7fb7340047c0 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.092+0000 7fb74aa97640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440c4c60 con 0x7fb7441f5d00 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.093+0000 7fb743fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb744152dd0 0x7fb7441554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.093+0000 7fb743fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb744152dd0 0x7fb7441554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43804/0 (socket says 172.21.15.70:43804) 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.093+0000 7fb743fff640 1 -- 172.21.15.70:0/1279257142 learned_addr learned my addr 172.21.15.70:0/1279257142 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.093+0000 7fb743fff640 1 -- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7441f5d00 msgr2=0x7fb7441f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.093+0000 7fb743fff640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7441f5d00 0x7fb7441f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.093+0000 7fb743fff640 1 -- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7340047c0 msgr2=0x7fb7441f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.093+0000 7fb748833640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7340047c0 0x7fb7441f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.093+0000 7fb743fff640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7340047c0 0x7fb7441f22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.093+0000 7fb743fff640 1 -- 172.21.15.70:0/1279257142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440c3ea0 con 0x7fb744152dd0 2023-10-08T05:37:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.093+0000 7fb743fff640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb744152dd0 0x7fb7441554e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb73002c2b0 tx=0x7fb730004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.094+0000 7fb7417fa640 1 -- 172.21.15.70:0/1279257142 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730003810 con 0x7fb744152dd0 2023-10-08T05:37:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.094+0000 7fb7417fa640 1 -- 172.21.15.70:0/1279257142 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73002de90 con 0x7fb744152dd0 2023-10-08T05:37:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.094+0000 7fb7417fa640 1 -- 172.21.15.70:0/1279257142 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730003bb0 con 0x7fb744152dd0 2023-10-08T05:37:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.094+0000 7fb74aa97640 1 -- 172.21.15.70:0/1279257142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7440c4180 con 0x7fb744152dd0 2023-10-08T05:37:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.094+0000 7fb74aa97640 1 -- 172.21.15.70:0/1279257142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7440c4740 con 0x7fb744152dd0 2023-10-08T05:37:10.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.095+0000 7fb74aa97640 1 -- 172.21.15.70:0/1279257142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7440c4900 con 0x7fb744152dd0 2023-10-08T05:37:10.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.096+0000 7fb7417fa640 1 -- 172.21.15.70:0/1279257142 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb730003d10 con 0x7fb744152dd0 2023-10-08T05:37:10.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.097+0000 7fb7417fa640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb72806aea0 0x7fb72806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.097+0000 7fb7417fa640 1 -- 172.21.15.70:0/1279257142 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fb7300aba10 con 0x7fb744152dd0 2023-10-08T05:37:10.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.097+0000 7fb748833640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb72806aea0 0x7fb72806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.098+0000 7fb748833640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb72806aea0 0x7fb72806d360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb738009830 tx=0x7fb738008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:10.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.101+0000 7fb7417fa640 1 -- 172.21.15.70:0/1279257142 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb730076660 con 0x7fb744152dd0 2023-10-08T05:37:10.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.275+0000 7fb74aa97640 1 -- 172.21.15.70:0/1279257142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb74404e160 con 0x7fb744152dd0 2023-10-08T05:37:10.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.276+0000 7fb7417fa640 1 -- 172.21.15.70:0/1279257142 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb730079c80 con 0x7fb744152dd0 2023-10-08T05:37:10.278 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:10.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.278+0000 7fb74aa97640 1 -- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb72806aea0 msgr2=0x7fb72806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.278+0000 7fb74aa97640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb72806aea0 0x7fb72806d360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb738009830 tx=0x7fb738008040 comp rx=0 tx=0).stop 2023-10-08T05:37:10.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.279+0000 7fb74aa97640 1 -- 172.21.15.70:0/1279257142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb744152dd0 msgr2=0x7fb7441554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.279+0000 7fb74aa97640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb744152dd0 0x7fb7441554e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb73002c2b0 tx=0x7fb730004730 comp rx=0 tx=0).stop 2023-10-08T05:37:10.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.279+0000 7fb74aa97640 1 -- 172.21.15.70:0/1279257142 shutdown_connections 2023-10-08T05:37:10.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.279+0000 7fb74aa97640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb72806aea0 0x7fb72806d360 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.279+0000 7fb74aa97640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7441f5d00 0x7fb7441f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.279+0000 7fb74aa97640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb744152dd0 0x7fb7441554e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.279+0000 7fb74aa97640 1 --2- 172.21.15.70:0/1279257142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7340047c0 0x7fb7441f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.279+0000 7fb74aa97640 1 -- 172.21.15.70:0/1279257142 >> 172.21.15.70:0/1279257142 conn(0x7fb74406a5d0 msgr2=0x7fb74415a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:10.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.279+0000 7fb74aa97640 1 -- 172.21.15.70:0/1279257142 shutdown_connections 2023-10-08T05:37:10.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.279+0000 7fb74aa97640 1 -- 172.21.15.70:0/1279257142 wait complete. 2023-10-08T05:37:10.318 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-10-08T05:37:10.319 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T05:37:10.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.531+0000 7f7b62105640 1 Processor -- start 2023-10-08T05:37:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.531+0000 7f7b62105640 1 -- start start 2023-10-08T05:37:10.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.532+0000 7f7b62105640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7b5c15ab40 0x7f7b5c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.532+0000 7f7b62105640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7b5c0c90b0 0x7f7b5c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.533+0000 7f7b62105640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5c15b2b0 con 0x7f7b5c0cae70 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.533+0000 7f7b62105640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5c15bf30 con 0x7f7b5c0c90b0 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.533+0000 7f7b62105640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5c15f7a0 con 0x7f7b5c15ab40 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.533+0000 7f7b53fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7b5c0c90b0 0x7f7b5c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.533+0000 7f7b53fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7b5c0c90b0 0x7f7b5c0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43806/0 (socket says 172.21.15.70:43806) 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.533+0000 7f7b53fff640 1 -- 172.21.15.70:0/979218636 learned_addr learned my addr 172.21.15.70:0/979218636 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.533+0000 7f7b53fff640 1 -- 172.21.15.70:0/979218636 >> v2:172.21.15.181:3300/0 conn(0x7f7b5c15ab40 msgr2=0x7f7b5c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.533+0000 7f7b53fff640 1 --2- 172.21.15.70:0/979218636 >> v2:172.21.15.181:3300/0 conn(0x7f7b5c15ab40 0x7f7b5c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.533+0000 7f7b53fff640 1 -- 172.21.15.70:0/979218636 >> v1:172.21.15.181:6789/0 conn(0x7f7b5c0cae70 legacy=0x7f7b5c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.533+0000 7f7b53fff640 1 -- 172.21.15.70:0/979218636 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c15b590 con 0x7f7b5c0c90b0 2023-10-08T05:37:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.534+0000 7f7b53fff640 1 --2- 172.21.15.70:0/979218636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c0c90b0 0x7f7b5c0adc60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c06e440 tx=0x7f7b4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f223d39ad3a74651 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.535+0000 7f7b5affd640 1 -- 172.21.15.70:0/979218636 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44031870 con 0x7f7b5c0c90b0 2023-10-08T05:37:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.535+0000 7f7b5affd640 1 -- 172.21.15.70:0/979218636 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b440319d0 con 0x7f7b5c0c90b0 2023-10-08T05:37:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.535+0000 7f7b5affd640 1 -- 172.21.15.70:0/979218636 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44031d60 con 0x7f7b5c0c90b0 2023-10-08T05:37:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.535+0000 7f7b62105640 1 -- 172.21.15.70:0/979218636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c0c90b0 msgr2=0x7f7b5c0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.535+0000 7f7b62105640 1 --2- 172.21.15.70:0/979218636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c0c90b0 0x7f7b5c0adc60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c06e440 tx=0x7f7b4402b910 comp rx=0 tx=0).stop 2023-10-08T05:37:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.535+0000 7f7b62105640 1 -- 172.21.15.70:0/979218636 shutdown_connections 2023-10-08T05:37:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.535+0000 7f7b62105640 1 --2- 172.21.15.70:0/979218636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c0c90b0 0x7f7b5c0adc60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.535+0000 7f7b62105640 1 --2- 172.21.15.70:0/979218636 >> v2:172.21.15.181:3300/0 conn(0x7f7b5c15ab40 0x7f7b5c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.535+0000 7f7b62105640 1 -- 172.21.15.70:0/979218636 >> 172.21.15.70:0/979218636 conn(0x7f7b5c06a380 msgr2=0x7f7b5c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:10.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.536+0000 7f7b62105640 1 -- 172.21.15.70:0/979218636 shutdown_connections 2023-10-08T05:37:10.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.536+0000 7f7b62105640 1 -- 172.21.15.70:0/979218636 wait complete. 2023-10-08T05:37:10.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.537+0000 7f7b62105640 1 Processor -- start 2023-10-08T05:37:10.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.537+0000 7f7b62105640 1 -- start start 2023-10-08T05:37:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.538+0000 7f7b62105640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b5c0c90b0 0x7f7b5c1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.538+0000 7f7b62105640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b5c0cae70 0x7f7b5c15dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.538+0000 7f7b62105640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c15ab40 0x7f7b5c0ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.538+0000 7f7b62105640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c0c85e0 con 0x7f7b5c15ab40 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.538+0000 7f7b62105640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c1f2c40 con 0x7f7b5c0c90b0 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.538+0000 7f7b62105640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c1f2f20 con 0x7f7b5c0cae70 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b5bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c15ab40 0x7f7b5c0ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b5bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c15ab40 0x7f7b5c0ac910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43812/0 (socket says 172.21.15.70:43812) 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b5bfff640 1 -- 172.21.15.70:0/3098712215 learned_addr learned my addr 172.21.15.70:0/3098712215 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b53fff640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b5c0cae70 0x7f7b5c15dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b5bfff640 1 -- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b5c0cae70 msgr2=0x7f7b5c15dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b5b7fe640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b5c0c90b0 0x7f7b5c1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b5bfff640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b5c0cae70 0x7f7b5c15dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b5bfff640 1 -- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b5c0c90b0 msgr2=0x7f7b5c1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b5bfff640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b5c0c90b0 0x7f7b5c1586f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b5bfff640 1 -- 172.21.15.70:0/3098712215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b440094d0 con 0x7f7b5c15ab40 2023-10-08T05:37:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.539+0000 7f7b5bfff640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c15ab40 0x7f7b5c0ac910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400c780 tx=0x7f7b5400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.540+0000 7f7b58ff9640 1 -- 172.21.15.70:0/3098712215 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54003d80 con 0x7f7b5c15ab40 2023-10-08T05:37:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.540+0000 7f7b58ff9640 1 -- 172.21.15.70:0/3098712215 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5400ce90 con 0x7f7b5c15ab40 2023-10-08T05:37:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.540+0000 7f7b58ff9640 1 -- 172.21.15.70:0/3098712215 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b540052b0 con 0x7f7b5c15ab40 2023-10-08T05:37:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.540+0000 7f7b62105640 1 -- 172.21.15.70:0/3098712215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5c1f3200 con 0x7f7b5c15ab40 2023-10-08T05:37:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.540+0000 7f7b62105640 1 -- 172.21.15.70:0/3098712215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c158c30 con 0x7f7b5c15ab40 2023-10-08T05:37:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.541+0000 7f7b62105640 1 -- 172.21.15.70:0/3098712215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5c1f3720 con 0x7f7b5c15ab40 2023-10-08T05:37:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.542+0000 7f7b58ff9640 1 -- 172.21.15.70:0/3098712215 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7b54005410 con 0x7f7b5c15ab40 2023-10-08T05:37:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.543+0000 7f7b58ff9640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7b3c06ae70 0x7f7b3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.543+0000 7f7b58ff9640 1 -- 172.21.15.70:0/3098712215 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7b54088ea0 con 0x7f7b5c15ab40 2023-10-08T05:37:10.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.543+0000 7f7b5b7fe640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7b3c06ae70 0x7f7b3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.545+0000 7f7b5b7fe640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7b3c06ae70 0x7f7b3c06d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00a810 tx=0x7f7b4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.547+0000 7f7b58ff9640 1 -- 172.21.15.70:0/3098712215 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b54053b90 con 0x7f7b5c15ab40 2023-10-08T05:37:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.737+0000 7f7b62105640 1 -- 172.21.15.70:0/3098712215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b5c067490 con 0x7f7b5c15ab40 2023-10-08T05:37:10.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.739+0000 7f7b58ff9640 1 -- 172.21.15.70:0/3098712215 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7f7b54017020 con 0x7f7b5c15ab40 2023-10-08T05:37:10.741 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:10.741 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:10.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 -- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7b3c06ae70 msgr2=0x7f7b3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7b3c06ae70 0x7f7b3c06d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c00a810 tx=0x7f7b4c008040 comp rx=0 tx=0).stop 2023-10-08T05:37:10.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 -- 172.21.15.70:0/3098712215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c15ab40 msgr2=0x7f7b5c0ac910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c15ab40 0x7f7b5c0ac910 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400c780 tx=0x7f7b5400cc40 comp rx=0 tx=0).stop 2023-10-08T05:37:10.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 -- 172.21.15.70:0/3098712215 shutdown_connections 2023-10-08T05:37:10.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7b3c06ae70 0x7f7b3c06d330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b5c15ab40 0x7f7b5c0ac910 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b5c0cae70 0x7f7b5c15dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 --2- 172.21.15.70:0/3098712215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b5c0c90b0 0x7f7b5c1586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 -- 172.21.15.70:0/3098712215 >> 172.21.15.70:0/3098712215 conn(0x7f7b5c06a380 msgr2=0x7f7b5c0701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:10.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 -- 172.21.15.70:0/3098712215 shutdown_connections 2023-10-08T05:37:10.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.742+0000 7f7b62105640 1 -- 172.21.15.70:0/3098712215 wait complete. 2023-10-08T05:37:10.764 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:37:10.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:10.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.979+0000 7f1c3d2b0640 1 Processor -- start 2023-10-08T05:37:10.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.979+0000 7f1c3d2b0640 1 -- start start 2023-10-08T05:37:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.981+0000 7f1c3d2b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c3815aad0 0x7f1c380b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.981+0000 7f1c3d2b0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1c380c82b0 0x7f1c380c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.981+0000 7f1c3d2b0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c3815bf30 con 0x7f1c3806cd00 2023-10-08T05:37:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.981+0000 7f1c3d2b0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3815b320 con 0x7f1c380c82b0 2023-10-08T05:37:10.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.981+0000 7f1c3d2b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3815b600 con 0x7f1c3815aad0 2023-10-08T05:37:10.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.982+0000 7f1c2e7fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1c380c82b0 0x7f1c380c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.982+0000 7f1c36ffd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c3815aad0 0x7f1c380b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.982+0000 7f1c2e7fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1c380c82b0 0x7f1c380c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:50788/0 (socket says 172.21.15.70:50788) 2023-10-08T05:37:10.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.982+0000 7f1c2e7fc640 1 -- 172.21.15.70:0/3921911481 learned_addr learned my addr 172.21.15.70:0/3921911481 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.982+0000 7f1c2e7fc640 1 -- 172.21.15.70:0/3921911481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c3815aad0 msgr2=0x7f1c380b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.983+0000 7f1c2e7fc640 1 --2- 172.21.15.70:0/3921911481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c3815aad0 0x7f1c380b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.983+0000 7f1c2e7fc640 1 -- 172.21.15.70:0/3921911481 >> v1:172.21.15.181:6789/0 conn(0x7f1c3806cd00 legacy=0x7f1c38070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.983+0000 7f1c2e7fc640 1 -- 172.21.15.70:0/3921911481 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3815b8e0 con 0x7f1c380c82b0 2023-10-08T05:37:10.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.984+0000 7f1c2e7fc640 1 --2- 172.21.15.70:0/3921911481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c380c82b0 0x7f1c380c8710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009870 tx=0x7f1c2002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81ff229fa279f17e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:10.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.984+0000 7f1c367fc640 1 -- 172.21.15.70:0/3921911481 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20030840 con 0x7f1c380c82b0 2023-10-08T05:37:10.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.984+0000 7f1c367fc640 1 -- 172.21.15.70:0/3921911481 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c200309a0 con 0x7f1c380c82b0 2023-10-08T05:37:10.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.984+0000 7f1c367fc640 1 -- 172.21.15.70:0/3921911481 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c20030d30 con 0x7f1c380c82b0 2023-10-08T05:37:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.985+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/3921911481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c380c82b0 msgr2=0x7f1c380c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.985+0000 7f1c3d2b0640 1 --2- 172.21.15.70:0/3921911481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c380c82b0 0x7f1c380c8710 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009870 tx=0x7f1c2002b980 comp rx=0 tx=0).stop 2023-10-08T05:37:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.985+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/3921911481 shutdown_connections 2023-10-08T05:37:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.985+0000 7f1c3d2b0640 1 --2- 172.21.15.70:0/3921911481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c380c82b0 0x7f1c380c8710 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.985+0000 7f1c3d2b0640 1 --2- 172.21.15.70:0/3921911481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c3815aad0 0x7f1c380b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.985+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/3921911481 >> 172.21.15.70:0/3921911481 conn(0x7f1c38069d50 msgr2=0x7f1c3806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.985+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/3921911481 shutdown_connections 2023-10-08T05:37:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.985+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/3921911481 wait complete. 2023-10-08T05:37:10.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.987+0000 7f1c3d2b0640 1 Processor -- start 2023-10-08T05:37:10.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.987+0000 7f1c3d2b0640 1 -- start start 2023-10-08T05:37:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.988+0000 7f1c3d2b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c3806cd00 0x7f1c381f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.988+0000 7f1c3d2b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c380c82b0 0x7f1c380cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.988+0000 7f1c3d2b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3815aad0 0x7f1c380cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.988+0000 7f1c3d2b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c381f5920 con 0x7f1c3815aad0 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.988+0000 7f1c3d2b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c381f5c00 con 0x7f1c380c82b0 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.988+0000 7f1c3d2b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c381f5ee0 con 0x7f1c3806cd00 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.988+0000 7f1c377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3815aad0 0x7f1c380cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3815aad0 0x7f1c380cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43814/0 (socket says 172.21.15.70:43814) 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c377fe640 1 -- 172.21.15.70:0/4101149159 learned_addr learned my addr 172.21.15.70:0/4101149159 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c36ffd640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c3806cd00 0x7f1c381f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c2e7fc640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c380c82b0 0x7f1c380cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c377fe640 1 -- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c3806cd00 msgr2=0x7f1c381f1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c377fe640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c3806cd00 0x7f1c381f1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c377fe640 1 -- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c380c82b0 msgr2=0x7f1c380cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c377fe640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c380c82b0 0x7f1c380cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c377fe640 1 -- 172.21.15.70:0/4101149159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200094d0 con 0x7f1c3815aad0 2023-10-08T05:37:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c377fe640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3815aad0 0x7f1c380cc930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1c30001eb0 tx=0x7f1c3000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.989+0000 7f1c2ffff640 1 -- 172.21.15.70:0/4101149159 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c300052d0 con 0x7f1c3815aad0 2023-10-08T05:37:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.990+0000 7f1c2ffff640 1 -- 172.21.15.70:0/4101149159 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c30005430 con 0x7f1c3815aad0 2023-10-08T05:37:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.990+0000 7f1c2ffff640 1 -- 172.21.15.70:0/4101149159 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30003810 con 0x7f1c3815aad0 2023-10-08T05:37:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.990+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/4101149159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c381f61c0 con 0x7f1c3815aad0 2023-10-08T05:37:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.990+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/4101149159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c381f6780 con 0x7f1c3815aad0 2023-10-08T05:37:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.991+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/4101149159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3814c2d0 con 0x7f1c3815aad0 2023-10-08T05:37:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.992+0000 7f1c2ffff640 1 -- 172.21.15.70:0/4101149159 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1c30003970 con 0x7f1c3815aad0 2023-10-08T05:37:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.993+0000 7f1c2ffff640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c1806ae70 0x7f1c1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.993+0000 7f1c2ffff640 1 -- 172.21.15.70:0/4101149159 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1c30088fe0 con 0x7f1c3815aad0 2023-10-08T05:37:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.993+0000 7f1c36ffd640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c1806ae70 0x7f1c1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.994+0000 7f1c36ffd640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c1806ae70 0x7f1c1806d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1c280073f0 tx=0x7f1c2800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:10.997+0000 7f1c2ffff640 1 -- 172.21.15.70:0/4101149159 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c30053c90 con 0x7f1c3815aad0 2023-10-08T05:37:11.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.187+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/4101149159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c38145f10 con 0x7f1c3815aad0 2023-10-08T05:37:11.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.188+0000 7f1c2ffff640 1 -- 172.21.15.70:0/4101149159 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c30017020 con 0x7f1c3815aad0 2023-10-08T05:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.190+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c1806ae70 msgr2=0x7f1c1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.190+0000 7f1c3d2b0640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c1806ae70 0x7f1c1806d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1c280073f0 tx=0x7f1c2800b040 comp rx=0 tx=0).stop 2023-10-08T05:37:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.190+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/4101149159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3815aad0 msgr2=0x7f1c380cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.190+0000 7f1c3d2b0640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3815aad0 0x7f1c380cc930 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1c30001eb0 tx=0x7f1c3000c890 comp rx=0 tx=0).stop 2023-10-08T05:37:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.191+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/4101149159 shutdown_connections 2023-10-08T05:37:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.191+0000 7f1c3d2b0640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1c1806ae70 0x7f1c1806d330 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.191+0000 7f1c3d2b0640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c3815aad0 0x7f1c380cc930 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.191+0000 7f1c3d2b0640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1c380c82b0 0x7f1c380cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.191+0000 7f1c3d2b0640 1 --2- 172.21.15.70:0/4101149159 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1c3806cd00 0x7f1c381f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.191+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/4101149159 >> 172.21.15.70:0/4101149159 conn(0x7f1c38069d50 msgr2=0x7f1c38070570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.191+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/4101149159 shutdown_connections 2023-10-08T05:37:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.191+0000 7f1c3d2b0640 1 -- 172.21.15.70:0/4101149159 wait complete. 2023-10-08T05:37:11.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:37:11.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.425+0000 7fcc77300640 1 Processor -- start 2023-10-08T05:37:11.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.425+0000 7fcc77300640 1 -- start start 2023-10-08T05:37:11.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.427+0000 7fcc77300640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcc700c9cf0 0x7fcc7006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.427+0000 7fcc77300640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc7015aa00 0x7fcc700b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.427+0000 7fcc77300640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc7015b310 con 0x7fcc7006c3a0 2023-10-08T05:37:11.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.427+0000 7fcc77300640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7006e0a0 con 0x7fcc7015aa00 2023-10-08T05:37:11.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.427+0000 7fcc77300640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7006d290 con 0x7fcc700c9cf0 2023-10-08T05:37:11.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.427+0000 7fcc7509c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcc7006c3a0 0x7fcc7006c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33140/0 (socket says 172.21.15.70:33140) 2023-10-08T05:37:11.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.427+0000 7fcc7509c640 1 -- 172.21.15.70:0/1272819449 learned_addr learned my addr 172.21.15.70:0/1272819449 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:11.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.427+0000 7fcc7589d640 1 --2- 172.21.15.70:0/1272819449 >> v2:172.21.15.70:3300/0 conn(0x7fcc7015aa00 0x7fcc700b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.428+0000 7fcc7489b640 1 --2- 172.21.15.70:0/1272819449 >> v2:172.21.15.181:3300/0 conn(0x7fcc700c9cf0 0x7fcc7006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.428+0000 7fcc67fff640 1 -- 172.21.15.70:0/1272819449 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181761385 0 0) 0x7fcc6c001110 con 0x7fcc7006c3a0 2023-10-08T05:37:11.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.428+0000 7fcc67fff640 1 -- 172.21.15.70:0/1272819449 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc5c005730 con 0x7fcc7006c3a0 2023-10-08T05:37:11.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.428+0000 7fcc7589d640 1 -- 172.21.15.70:0/1272819449 >> v2:172.21.15.181:3300/0 conn(0x7fcc700c9cf0 msgr2=0x7fcc7006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:11.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.428+0000 7fcc7589d640 1 --2- 172.21.15.70:0/1272819449 >> v2:172.21.15.181:3300/0 conn(0x7fcc700c9cf0 0x7fcc7006db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.428+0000 7fcc7589d640 1 -- 172.21.15.70:0/1272819449 >> v1:172.21.15.70:6789/0 conn(0x7fcc7006c3a0 legacy=0x7fcc7006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.428+0000 7fcc7589d640 1 -- 172.21.15.70:0/1272819449 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc700c9860 con 0x7fcc7015aa00 2023-10-08T05:37:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.428+0000 7fcc7489b640 1 --2- 172.21.15.70:0/1272819449 >> v2:172.21.15.181:3300/0 conn(0x7fcc700c9cf0 0x7fcc7006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:37:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.429+0000 7fcc7589d640 1 --2- 172.21.15.70:0/1272819449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 0x7fcc700b2a90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcc700715c0 tx=0x7fcc6802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4370f3e5cb763c9d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.429+0000 7fcc67fff640 1 -- 172.21.15.70:0/1272819449 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6802cc80 con 0x7fcc7015aa00 2023-10-08T05:37:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.430+0000 7fcc67fff640 1 -- 172.21.15.70:0/1272819449 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6802cde0 con 0x7fcc7015aa00 2023-10-08T05:37:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.430+0000 7fcc67fff640 1 -- 172.21.15.70:0/1272819449 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68034660 con 0x7fcc7015aa00 2023-10-08T05:37:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.430+0000 7fcc77300640 1 -- 172.21.15.70:0/1272819449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 msgr2=0x7fcc700b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.430+0000 7fcc77300640 1 --2- 172.21.15.70:0/1272819449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 0x7fcc700b2a90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcc700715c0 tx=0x7fcc6802bed0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.430+0000 7fcc77300640 1 -- 172.21.15.70:0/1272819449 shutdown_connections 2023-10-08T05:37:11.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.430+0000 7fcc77300640 1 --2- 172.21.15.70:0/1272819449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 0x7fcc700b2a90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.430+0000 7fcc77300640 1 --2- 172.21.15.70:0/1272819449 >> v2:172.21.15.181:3300/0 conn(0x7fcc700c9cf0 0x7fcc7006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.430+0000 7fcc77300640 1 -- 172.21.15.70:0/1272819449 >> 172.21.15.70:0/1272819449 conn(0x7fcc7006a5d0 msgr2=0x7fcc700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:11.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.430+0000 7fcc77300640 1 -- 172.21.15.70:0/1272819449 shutdown_connections 2023-10-08T05:37:11.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.430+0000 7fcc77300640 1 -- 172.21.15.70:0/1272819449 wait complete. 2023-10-08T05:37:11.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.432+0000 7fcc77300640 1 Processor -- start 2023-10-08T05:37:11.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.432+0000 7fcc77300640 1 -- start start 2023-10-08T05:37:11.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.433+0000 7fcc77300640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc7006c3a0 0x7fcc701f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.433+0000 7fcc77300640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc700c9cf0 0x7fcc70155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.433+0000 7fcc77300640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 0x7fcc700c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.433+0000 7fcc77300640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc700c1550 con 0x7fcc7015aa00 2023-10-08T05:37:11.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.433+0000 7fcc77300640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc700c1d60 con 0x7fcc700c9cf0 2023-10-08T05:37:11.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.433+0000 7fcc77300640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc701f15a0 con 0x7fcc7006c3a0 2023-10-08T05:37:11.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.433+0000 7fcc7589d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 0x7fcc700c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.433+0000 7fcc7589d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 0x7fcc700c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43844/0 (socket says 172.21.15.70:43844) 2023-10-08T05:37:11.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.433+0000 7fcc7589d640 1 -- 172.21.15.70:0/2795371834 learned_addr learned my addr 172.21.15.70:0/2795371834 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:11.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.434+0000 7fcc7489b640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc700c9cf0 0x7fcc70155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.434+0000 7fcc7589d640 1 -- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc7006c3a0 msgr2=0x7fcc701f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.434+0000 7fcc7589d640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc7006c3a0 0x7fcc701f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.434+0000 7fcc7589d640 1 -- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc700c9cf0 msgr2=0x7fcc70155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.434+0000 7fcc7589d640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc700c9cf0 0x7fcc70155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.434+0000 7fcc7589d640 1 -- 172.21.15.70:0/2795371834 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68009a70 con 0x7fcc7015aa00 2023-10-08T05:37:11.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.434+0000 7fcc7589d640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 0x7fcc700c72a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcc68037040 tx=0x7fcc68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:11.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.434+0000 7fcc65ffb640 1 -- 172.21.15.70:0/2795371834 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6803e610 con 0x7fcc7015aa00 2023-10-08T05:37:11.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.435+0000 7fcc65ffb640 1 -- 172.21.15.70:0/2795371834 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6803e770 con 0x7fcc7015aa00 2023-10-08T05:37:11.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.435+0000 7fcc65ffb640 1 -- 172.21.15.70:0/2795371834 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6803eb20 con 0x7fcc7015aa00 2023-10-08T05:37:11.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.435+0000 7fcc77300640 1 -- 172.21.15.70:0/2795371834 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc700c17a0 con 0x7fcc7015aa00 2023-10-08T05:37:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.435+0000 7fcc77300640 1 -- 172.21.15.70:0/2795371834 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc701f1840 con 0x7fcc7015aa00 2023-10-08T05:37:11.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.436+0000 7fcc77300640 1 -- 172.21.15.70:0/2795371834 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc701f1a60 con 0x7fcc7015aa00 2023-10-08T05:37:11.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.437+0000 7fcc65ffb640 1 -- 172.21.15.70:0/2795371834 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcc6803b050 con 0x7fcc7015aa00 2023-10-08T05:37:11.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.438+0000 7fcc65ffb640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcc5406ae70 0x7fcc5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.438+0000 7fcc65ffb640 1 -- 172.21.15.70:0/2795371834 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcc680ac1b0 con 0x7fcc7015aa00 2023-10-08T05:37:11.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.439+0000 7fcc7509c640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcc5406ae70 0x7fcc5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.440+0000 7fcc7509c640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcc5406ae70 0x7fcc5406d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c005880 tx=0x7fcc6c001150 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.443+0000 7fcc65ffb640 1 -- 172.21.15.70:0/2795371834 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc68076f10 con 0x7fcc7015aa00 2023-10-08T05:37:11.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.629+0000 7fcc77300640 1 -- 172.21.15.70:0/2795371834 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc7004e160 con 0x7fcc7015aa00 2023-10-08T05:37:11.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.630+0000 7fcc65ffb640 1 -- 172.21.15.70:0/2795371834 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc68009450 con 0x7fcc7015aa00 2023-10-08T05:37:11.632 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:11.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 -- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcc5406ae70 msgr2=0x7fcc5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcc5406ae70 0x7fcc5406d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c005880 tx=0x7fcc6c001150 comp rx=0 tx=0).stop 2023-10-08T05:37:11.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 -- 172.21.15.70:0/2795371834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 msgr2=0x7fcc700c72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 0x7fcc700c72a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcc68037040 tx=0x7fcc68004a40 comp rx=0 tx=0).stop 2023-10-08T05:37:11.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 -- 172.21.15.70:0/2795371834 shutdown_connections 2023-10-08T05:37:11.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcc5406ae70 0x7fcc5406d330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc7015aa00 0x7fcc700c72a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc700c9cf0 0x7fcc70155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 --2- 172.21.15.70:0/2795371834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc7006c3a0 0x7fcc701f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 -- 172.21.15.70:0/2795371834 >> 172.21.15.70:0/2795371834 conn(0x7fcc7006a5d0 msgr2=0x7fcc700caca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:11.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 -- 172.21.15.70:0/2795371834 shutdown_connections 2023-10-08T05:37:11.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.634+0000 7fcc77300640 1 -- 172.21.15.70:0/2795371834 wait complete. 2023-10-08T05:37:11.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:37:11.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.868+0000 7f3abff83640 1 Processor -- start 2023-10-08T05:37:11.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.869+0000 7f3abff83640 1 -- start start 2023-10-08T05:37:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.870+0000 7f3abff83640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ab815ce00 0x7f3ab80cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.870+0000 7f3abff83640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab80cd470 0x7f3ab80cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.870+0000 7f3abff83640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab80cde10 con 0x7f3ab8160980 2023-10-08T05:37:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.870+0000 7f3abff83640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab8161920 con 0x7f3ab815ce00 2023-10-08T05:37:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.870+0000 7f3abff83640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80cb8b0 con 0x7f3ab80cd470 2023-10-08T05:37:11.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.871+0000 7f3abdd1f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ab815ce00 0x7f3ab80cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.871+0000 7f3abdd1f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ab815ce00 0x7f3ab80cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:50830/0 (socket says 172.21.15.70:50830) 2023-10-08T05:37:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.871+0000 7f3abdd1f640 1 -- 172.21.15.70:0/2864761747 learned_addr learned my addr 172.21.15.70:0/2864761747 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.871+0000 7f3abd51e640 1 --2- 172.21.15.70:0/2864761747 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab80cd470 0x7f3ab80cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.872+0000 7f3abdd1f640 1 -- 172.21.15.70:0/2864761747 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab80cd470 msgr2=0x7f3ab80cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.872+0000 7f3abdd1f640 1 --2- 172.21.15.70:0/2864761747 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab80cd470 0x7f3ab80cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.872+0000 7f3abdd1f640 1 -- 172.21.15.70:0/2864761747 >> v1:172.21.15.181:6789/0 conn(0x7f3ab8160980 legacy=0x7f3ab8160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.872+0000 7f3abdd1f640 1 -- 172.21.15.70:0/2864761747 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab8161640 con 0x7f3ab815ce00 2023-10-08T05:37:11.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.873+0000 7f3abdd1f640 1 --2- 172.21.15.70:0/2864761747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ab815ce00 0x7f3ab80cb370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009ea0 tx=0x7f3aa802bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce4d76e846407b2a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:11.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.873+0000 7f3abcd1d640 1 -- 172.21.15.70:0/2864761747 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa802cab0 con 0x7f3ab815ce00 2023-10-08T05:37:11.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.874+0000 7f3abcd1d640 1 -- 172.21.15.70:0/2864761747 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa802cc10 con 0x7f3ab815ce00 2023-10-08T05:37:11.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.874+0000 7f3abcd1d640 1 -- 172.21.15.70:0/2864761747 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80343f0 con 0x7f3ab815ce00 2023-10-08T05:37:11.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.874+0000 7f3abff83640 1 -- 172.21.15.70:0/2864761747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ab815ce00 msgr2=0x7f3ab80cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.874+0000 7f3abff83640 1 --2- 172.21.15.70:0/2864761747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ab815ce00 0x7f3ab80cb370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009ea0 tx=0x7f3aa802bf10 comp rx=0 tx=0).stop 2023-10-08T05:37:11.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.874+0000 7f3abff83640 1 -- 172.21.15.70:0/2864761747 shutdown_connections 2023-10-08T05:37:11.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.874+0000 7f3abff83640 1 --2- 172.21.15.70:0/2864761747 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab80cd470 0x7f3ab80cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.874+0000 7f3abff83640 1 --2- 172.21.15.70:0/2864761747 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ab815ce00 0x7f3ab80cb370 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.874+0000 7f3abff83640 1 -- 172.21.15.70:0/2864761747 >> 172.21.15.70:0/2864761747 conn(0x7f3ab806e650 msgr2=0x7f3ab806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:11.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.874+0000 7f3abff83640 1 -- 172.21.15.70:0/2864761747 shutdown_connections 2023-10-08T05:37:11.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.874+0000 7f3abff83640 1 -- 172.21.15.70:0/2864761747 wait complete. 2023-10-08T05:37:11.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.876+0000 7f3abff83640 1 Processor -- start 2023-10-08T05:37:11.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.876+0000 7f3abff83640 1 -- start start 2023-10-08T05:37:11.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.877+0000 7f3abff83640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ab80cd470 0x7f3ab81f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.877+0000 7f3abff83640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab815ce00 0x7f3ab80b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.877+0000 7f3abff83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab8160980 0x7f3ab80c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.877+0000 7f3abff83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80b1660 con 0x7f3ab8160980 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.877+0000 7f3abff83640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80b0dc0 con 0x7f3ab80cd470 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.877+0000 7f3abff83640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80b10a0 con 0x7f3ab815ce00 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.877+0000 7f3abe520640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab8160980 0x7f3ab80c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.877+0000 7f3abe520640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab8160980 0x7f3ab80c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43860/0 (socket says 172.21.15.70:43860) 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.877+0000 7f3abe520640 1 -- 172.21.15.70:0/3710240108 learned_addr learned my addr 172.21.15.70:0/3710240108 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.878+0000 7f3abd51e640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab815ce00 0x7f3ab80b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.878+0000 7f3abdd1f640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ab80cd470 0x7f3ab81f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.878+0000 7f3abe520640 1 -- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab815ce00 msgr2=0x7f3ab80b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.878+0000 7f3abe520640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab815ce00 0x7f3ab80b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.878+0000 7f3abe520640 1 -- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ab80cd470 msgr2=0x7f3ab81f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:11.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.878+0000 7f3abe520640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ab80cd470 0x7f3ab81f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:11.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.878+0000 7f3abe520640 1 -- 172.21.15.70:0/3710240108 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa8009b30 con 0x7f3ab8160980 2023-10-08T05:37:11.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.878+0000 7f3abd51e640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab815ce00 0x7f3ab80b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:37:11.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.878+0000 7f3abe520640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab8160980 0x7f3ab80c8260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000c390 tx=0x7f3ab000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:11.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.879+0000 7f3ab67fc640 1 -- 172.21.15.70:0/3710240108 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00052d0 con 0x7f3ab8160980 2023-10-08T05:37:11.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.879+0000 7f3ab67fc640 1 -- 172.21.15.70:0/3710240108 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab0005430 con 0x7f3ab8160980 2023-10-08T05:37:11.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.879+0000 7f3ab67fc640 1 -- 172.21.15.70:0/3710240108 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0003810 con 0x7f3ab8160980 2023-10-08T05:37:11.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.879+0000 7f3abff83640 1 -- 172.21.15.70:0/3710240108 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab80b1380 con 0x7f3ab8160980 2023-10-08T05:37:11.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.879+0000 7f3abff83640 1 -- 172.21.15.70:0/3710240108 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab81f54c0 con 0x7f3ab8160980 2023-10-08T05:37:11.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.880+0000 7f3abff83640 1 -- 172.21.15.70:0/3710240108 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab80c52c0 con 0x7f3ab8160980 2023-10-08T05:37:11.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.881+0000 7f3ab67fc640 1 -- 172.21.15.70:0/3710240108 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3ab0003970 con 0x7f3ab8160980 2023-10-08T05:37:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.882+0000 7f3ab67fc640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3aa406ae70 0x7f3aa406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.882+0000 7f3ab67fc640 1 -- 172.21.15.70:0/3710240108 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3ab0088f90 con 0x7f3ab8160980 2023-10-08T05:37:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.882+0000 7f3abdd1f640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3aa406ae70 0x7f3aa406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:11.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.884+0000 7f3abdd1f640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3aa406ae70 0x7f3aa406d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8035000 tx=0x7f3aa8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:11.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:11.886+0000 7f3ab67fc640 1 -- 172.21.15.70:0/3710240108 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ab80c52c0 con 0x7f3ab8160980 2023-10-08T05:37:12.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.076+0000 7f3abff83640 1 -- 172.21.15.70:0/3710240108 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ab80c52c0 con 0x7f3ab8160980 2023-10-08T05:37:12.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.077+0000 7f3ab67fc640 1 -- 172.21.15.70:0/3710240108 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ab80c52c0 con 0x7f3ab8160980 2023-10-08T05:37:12.078 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:37:12.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.079+0000 7f3abff83640 1 -- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3aa406ae70 msgr2=0x7f3aa406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.079+0000 7f3abff83640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3aa406ae70 0x7f3aa406d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8035000 tx=0x7f3aa8030000 comp rx=0 tx=0).stop 2023-10-08T05:37:12.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.079+0000 7f3abff83640 1 -- 172.21.15.70:0/3710240108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab8160980 msgr2=0x7f3ab80c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.079+0000 7f3abff83640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab8160980 0x7f3ab80c8260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000c390 tx=0x7f3ab000c850 comp rx=0 tx=0).stop 2023-10-08T05:37:12.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.080+0000 7f3abff83640 1 -- 172.21.15.70:0/3710240108 shutdown_connections 2023-10-08T05:37:12.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.080+0000 7f3abff83640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3aa406ae70 0x7f3aa406d330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.080+0000 7f3abff83640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab8160980 0x7f3ab80c8260 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.080+0000 7f3abff83640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ab815ce00 0x7f3ab80b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.080+0000 7f3abff83640 1 --2- 172.21.15.70:0/3710240108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ab80cd470 0x7f3ab81f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.080+0000 7f3abff83640 1 -- 172.21.15.70:0/3710240108 >> 172.21.15.70:0/3710240108 conn(0x7f3ab806e650 msgr2=0x7f3ab80ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:12.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.080+0000 7f3abff83640 1 -- 172.21.15.70:0/3710240108 shutdown_connections 2023-10-08T05:37:12.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.080+0000 7f3abff83640 1 -- 172.21.15.70:0/3710240108 wait complete. 2023-10-08T05:37:12.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:37:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.315+0000 7fc81fcd0640 1 Processor -- start 2023-10-08T05:37:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.316+0000 7fc81fcd0640 1 -- start start 2023-10-08T05:37:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.317+0000 7fc81fcd0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc81806db60 0x7fc8180b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.317+0000 7fc81fcd0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81815b310 con 0x7fc81806c3a0 2023-10-08T05:37:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.317+0000 7fc81fcd0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81806d290 con 0x7fc8180c9cf0 2023-10-08T05:37:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.317+0000 7fc81fcd0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81815b110 con 0x7fc81806db60 2023-10-08T05:37:12.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.317+0000 7fc81da6c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc81806c3a0 0x7fc81806c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33144/0 (socket says 172.21.15.70:33144) 2023-10-08T05:37:12.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.317+0000 7fc81da6c640 1 -- 172.21.15.70:0/927968371 learned_addr learned my addr 172.21.15.70:0/927968371 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:12.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.317+0000 7fc81e26d640 1 --2- 172.21.15.70:0/927968371 >> v2:172.21.15.181:3300/0 conn(0x7fc81806db60 0x7fc8180b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899557957 0 0) 0x7fc808001490 con 0x7fc81806c3a0 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc800003520 con 0x7fc81806c3a0 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3217632961 0 0) 0x7fc800003520 con 0x7fc81806c3a0 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 >> v2:172.21.15.181:3300/0 conn(0x7fc81806db60 msgr2=0x7fc8180b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 --2- 172.21.15.70:0/927968371 >> v2:172.21.15.181:3300/0 conn(0x7fc81806db60 0x7fc8180b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 >> v1:172.21.15.181:6789/0 conn(0x7fc8180c9cf0 legacy=0x7fc81815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180c9860 con 0x7fc81806c3a0 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc808001b80 con 0x7fc81806c3a0 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 >> v1:172.21.15.70:6789/0 conn(0x7fc81806c3a0 legacy=0x7fc81806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc800003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8000047c0 0x7fc800004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc800005180 0x7fc80000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81815b110 con 0x7fc800003970 2023-10-08T05:37:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80000c020 con 0x7fc800005180 2023-10-08T05:37:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80000c1e0 con 0x7fc8000047c0 2023-10-08T05:37:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.318+0000 7fc81da6c640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc800003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.319+0000 7fc81e26d640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc800005180 0x7fc80000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.319+0000 7fc81da6c640 1 -- 172.21.15.70:0/927968371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8000047c0 msgr2=0x7fc800004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.319+0000 7fc81da6c640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8000047c0 0x7fc800004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.319+0000 7fc81da6c640 1 -- 172.21.15.70:0/927968371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc800005180 msgr2=0x7fc80000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.319+0000 7fc81da6c640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc800005180 0x7fc80000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.319+0000 7fc81da6c640 1 -- 172.21.15.70:0/927968371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc800003520 con 0x7fc800003970 2023-10-08T05:37:12.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.319+0000 7fc81e26d640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc800005180 0x7fc80000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:12.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.319+0000 7fc81e26d640 1 -- 172.21.15.70:0/927968371 reap_dead start 2023-10-08T05:37:12.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.320+0000 7fc81da6c640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc800003d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc80800a800 tx=0x7fc8000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9b8a4aa6bdce7c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:12.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.320+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808034630 con 0x7fc800003970 2023-10-08T05:37:12.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.320+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80802f070 con 0x7fc800003970 2023-10-08T05:37:12.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.320+0000 7fc81ca6a640 1 -- 172.21.15.70:0/927968371 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808034a30 con 0x7fc800003970 2023-10-08T05:37:12.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.321+0000 7fc81fcd0640 1 -- 172.21.15.70:0/927968371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 msgr2=0x7fc800003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.321+0000 7fc81fcd0640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc800003d90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc80800a800 tx=0x7fc8000057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.321+0000 7fc81fcd0640 1 -- 172.21.15.70:0/927968371 shutdown_connections 2023-10-08T05:37:12.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.321+0000 7fc81fcd0640 1 --2- 172.21.15.70:0/927968371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc800003d90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.321+0000 7fc81fcd0640 1 -- 172.21.15.70:0/927968371 >> 172.21.15.70:0/927968371 conn(0x7fc81806a5d0 msgr2=0x7fc8180b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:12.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.321+0000 7fc81fcd0640 1 -- 172.21.15.70:0/927968371 shutdown_connections 2023-10-08T05:37:12.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.321+0000 7fc81fcd0640 1 -- 172.21.15.70:0/927968371 wait complete. 2023-10-08T05:37:12.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.323+0000 7fc81fcd0640 1 Processor -- start 2023-10-08T05:37:12.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.323+0000 7fc81fcd0640 1 -- start start 2023-10-08T05:37:12.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81fcd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc8180c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81fcd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8181521a0 0x7fc8180c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81fcd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc818157e30 0x7fc8181582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81fcd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8181579c0 con 0x7fc800003970 2023-10-08T05:37:12.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81fcd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818168a00 con 0x7fc8181521a0 2023-10-08T05:37:12.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81fcd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc818168ce0 con 0x7fc818157e30 2023-10-08T05:37:12.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81da6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc8180c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81da6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc8180c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43872/0 (socket says 172.21.15.70:43872) 2023-10-08T05:37:12.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81da6c640 1 -- 172.21.15.70:0/3497468739 learned_addr learned my addr 172.21.15.70:0/3497468739 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:12.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81d26b640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8181521a0 0x7fc8180c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81da6c640 1 -- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc818157e30 msgr2=0x7fc8181582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81e26d640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc818157e30 0x7fc8181582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81da6c640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc818157e30 0x7fc8181582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.324+0000 7fc81da6c640 1 -- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8181521a0 msgr2=0x7fc8180c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.325+0000 7fc81da6c640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8181521a0 0x7fc8180c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.325+0000 7fc81da6c640 1 -- 172.21.15.70:0/3497468739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818168fc0 con 0x7fc800003970 2023-10-08T05:37:12.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.325+0000 7fc81d26b640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8181521a0 0x7fc8180c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:12.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.325+0000 7fc81da6c640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc8180c5810 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc80802cb50 tx=0x7fc8180c5560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:12.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.325+0000 7fc8167fc640 1 -- 172.21.15.70:0/3497468739 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808034430 con 0x7fc800003970 2023-10-08T05:37:12.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.325+0000 7fc8167fc640 1 -- 172.21.15.70:0/3497468739 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80802ce90 con 0x7fc800003970 2023-10-08T05:37:12.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.325+0000 7fc81fcd0640 1 -- 172.21.15.70:0/3497468739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8181692a0 con 0x7fc800003970 2023-10-08T05:37:12.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.325+0000 7fc8167fc640 1 -- 172.21.15.70:0/3497468739 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8080347f0 con 0x7fc800003970 2023-10-08T05:37:12.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.325+0000 7fc81fcd0640 1 -- 172.21.15.70:0/3497468739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8181587f0 con 0x7fc800003970 2023-10-08T05:37:12.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.327+0000 7fc81fcd0640 1 -- 172.21.15.70:0/3497468739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81804e160 con 0x7fc800003970 2023-10-08T05:37:12.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.327+0000 7fc8167fc640 1 -- 172.21.15.70:0/3497468739 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc808031030 con 0x7fc800003970 2023-10-08T05:37:12.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.328+0000 7fc8167fc640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc80406ae70 0x7fc80406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.328+0000 7fc8167fc640 1 -- 172.21.15.70:0/3497468739 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fc8080ab6a0 con 0x7fc800003970 2023-10-08T05:37:12.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.329+0000 7fc81d26b640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc80406ae70 0x7fc80406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.330+0000 7fc81d26b640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc80406ae70 0x7fc80406d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc81815b0e0 tx=0x7fc80c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:12.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.333+0000 7fc8167fc640 1 -- 172.21.15.70:0/3497468739 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8080762f0 con 0x7fc800003970 2023-10-08T05:37:12.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.519+0000 7fc81fcd0640 1 -- 172.21.15.70:0/3497468739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc81815b310 con 0x7fc800003970 2023-10-08T05:37:12.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.519+0000 7fc8167fc640 1 -- 172.21.15.70:0/3497468739 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc81815b310 con 0x7fc800003970 2023-10-08T05:37:12.521 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 -- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc80406ae70 msgr2=0x7fc80406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc80406ae70 0x7fc80406d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc81815b0e0 tx=0x7fc80c009040 comp rx=0 tx=0).stop 2023-10-08T05:37:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 -- 172.21.15.70:0/3497468739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 msgr2=0x7fc8180c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc8180c5810 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc80802cb50 tx=0x7fc8180c5560 comp rx=0 tx=0).stop 2023-10-08T05:37:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 -- 172.21.15.70:0/3497468739 shutdown_connections 2023-10-08T05:37:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc80406ae70 0x7fc80406d330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc818157e30 0x7fc8181582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8181521a0 0x7fc8180c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 --2- 172.21.15.70:0/3497468739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc800003970 0x7fc8180c5810 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 -- 172.21.15.70:0/3497468739 >> 172.21.15.70:0/3497468739 conn(0x7fc81806a5d0 msgr2=0x7fc81815bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.522+0000 7fc81fcd0640 1 -- 172.21.15.70:0/3497468739 shutdown_connections 2023-10-08T05:37:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.523+0000 7fc81fcd0640 1 -- 172.21.15.70:0/3497468739 wait complete. 2023-10-08T05:37:12.544 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-08T05:37:12.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.759+0000 7fbe8bfff640 1 Processor -- start 2023-10-08T05:37:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.759+0000 7fbe8bfff640 1 -- start start 2023-10-08T05:37:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.760+0000 7fbe8bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c1599d0 0x7fbe8c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.761+0000 7fbe8bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe8c0b07d0 0x7fbe8c06c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.761+0000 7fbe8bfff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c0cacf0 con 0x7fbe8c0cc210 2023-10-08T05:37:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.761+0000 7fbe8bfff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8c0b10f0 con 0x7fbe8c0b07d0 2023-10-08T05:37:12.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.761+0000 7fbe8bfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0ca3c0 con 0x7fbe8c1599d0 2023-10-08T05:37:12.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.761+0000 7fbe8b7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe8c0b07d0 0x7fbe8c06c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.761+0000 7fbe8affd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c1599d0 0x7fbe8c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.762+0000 7fbe8affd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c1599d0 0x7fbe8c0ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:56734/0 (socket says 172.21.15.70:56734) 2023-10-08T05:37:12.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.762+0000 7fbe8affd640 1 -- 172.21.15.70:0/1875657401 learned_addr learned my addr 172.21.15.70:0/1875657401 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:12.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.762+0000 7fbe8b7fe640 1 -- 172.21.15.70:0/1875657401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c1599d0 msgr2=0x7fbe8c0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:12.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.762+0000 7fbe8b7fe640 1 --2- 172.21.15.70:0/1875657401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c1599d0 0x7fbe8c0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.762+0000 7fbe8b7fe640 1 -- 172.21.15.70:0/1875657401 >> v1:172.21.15.181:6789/0 conn(0x7fbe8c0cc210 legacy=0x7fbe8c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.762+0000 7fbe8b7fe640 1 -- 172.21.15.70:0/1875657401 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c0ca6a0 con 0x7fbe8c0b07d0 2023-10-08T05:37:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.762+0000 7fbe8affd640 1 --2- 172.21.15.70:0/1875657401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c1599d0 0x7fbe8c0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:37:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.764+0000 7fbe8b7fe640 1 --2- 172.21.15.70:0/1875657401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe8c0b07d0 0x7fbe8c06c310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009d90 tx=0x7fbe8402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e4174f55ded1cd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.764+0000 7fbe89ffb640 1 -- 172.21.15.70:0/1875657401 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8402cc50 con 0x7fbe8c0b07d0 2023-10-08T05:37:12.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.764+0000 7fbe89ffb640 1 -- 172.21.15.70:0/1875657401 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8402cdb0 con 0x7fbe8c0b07d0 2023-10-08T05:37:12.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.764+0000 7fbe89ffb640 1 -- 172.21.15.70:0/1875657401 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84034660 con 0x7fbe8c0b07d0 2023-10-08T05:37:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.765+0000 7fbe8bfff640 1 -- 172.21.15.70:0/1875657401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe8c0b07d0 msgr2=0x7fbe8c06c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.765+0000 7fbe8bfff640 1 --2- 172.21.15.70:0/1875657401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe8c0b07d0 0x7fbe8c06c310 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009d90 tx=0x7fbe8402bea0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.765+0000 7fbe8bfff640 1 -- 172.21.15.70:0/1875657401 shutdown_connections 2023-10-08T05:37:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.765+0000 7fbe8bfff640 1 --2- 172.21.15.70:0/1875657401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe8c0b07d0 0x7fbe8c06c310 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.765+0000 7fbe8bfff640 1 --2- 172.21.15.70:0/1875657401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c1599d0 0x7fbe8c0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.765+0000 7fbe8bfff640 1 -- 172.21.15.70:0/1875657401 >> 172.21.15.70:0/1875657401 conn(0x7fbe8c0cb0c0 msgr2=0x7fbe8c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.765+0000 7fbe8bfff640 1 -- 172.21.15.70:0/1875657401 shutdown_connections 2023-10-08T05:37:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.765+0000 7fbe8bfff640 1 -- 172.21.15.70:0/1875657401 wait complete. 2023-10-08T05:37:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.766+0000 7fbe8bfff640 1 Processor -- start 2023-10-08T05:37:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.767+0000 7fbe8bfff640 1 -- start start 2023-10-08T05:37:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c0b07d0 0x7fbe8c0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe8c0cc210 0x7fbe8c14ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c1599d0 0x7fbe8c155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8bfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0b20c0 con 0x7fbe8c1599d0 2023-10-08T05:37:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8bfff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0b23a0 con 0x7fbe8c0cc210 2023-10-08T05:37:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8bfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0b2680 con 0x7fbe8c0b07d0 2023-10-08T05:37:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c1599d0 0x7fbe8c155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c1599d0 0x7fbe8c155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43876/0 (socket says 172.21.15.70:43876) 2023-10-08T05:37:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8b7fe640 1 -- 172.21.15.70:0/20735545 learned_addr learned my addr 172.21.15.70:0/20735545 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8affd640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c0b07d0 0x7fbe8c0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8a7fc640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe8c0cc210 0x7fbe8c14ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8b7fe640 1 -- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c0b07d0 msgr2=0x7fbe8c0b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8b7fe640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c0b07d0 0x7fbe8c0b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8b7fe640 1 -- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe8c0cc210 msgr2=0x7fbe8c14ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8b7fe640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe8c0cc210 0x7fbe8c14ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8b7fe640 1 -- 172.21.15.70:0/20735545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c0094d0 con 0x7fbe8c1599d0 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.768+0000 7fbe8a7fc640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe8c0cc210 0x7fbe8c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.769+0000 7fbe8b7fe640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c1599d0 0x7fbe8c155150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009d90 tx=0x7fbe84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.769+0000 7fbe737fe640 1 -- 172.21.15.70:0/20735545 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84040650 con 0x7fbe8c1599d0 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.769+0000 7fbe737fe640 1 -- 172.21.15.70:0/20735545 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe840407b0 con 0x7fbe8c1599d0 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.769+0000 7fbe8bfff640 1 -- 172.21.15.70:0/20735545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe84009a70 con 0x7fbe8c1599d0 2023-10-08T05:37:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.769+0000 7fbe737fe640 1 -- 172.21.15.70:0/20735545 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84040b60 con 0x7fbe8c1599d0 2023-10-08T05:37:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.769+0000 7fbe8bfff640 1 -- 172.21.15.70:0/20735545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8c150530 con 0x7fbe8c1599d0 2023-10-08T05:37:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.771+0000 7fbe8bfff640 1 -- 172.21.15.70:0/20735545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8c0b2ba0 con 0x7fbe8c1599d0 2023-10-08T05:37:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.771+0000 7fbe737fe640 1 -- 172.21.15.70:0/20735545 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbe84040cc0 con 0x7fbe8c1599d0 2023-10-08T05:37:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.772+0000 7fbe737fe640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbe7806ae70 0x7fbe7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.772+0000 7fbe737fe640 1 -- 172.21.15.70:0/20735545 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbe840ac6c0 con 0x7fbe8c1599d0 2023-10-08T05:37:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.773+0000 7fbe8affd640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbe7806ae70 0x7fbe7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.774+0000 7fbe8affd640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbe7806ae70 0x7fbe7806d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c06d0d0 tx=0x7fbe7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:12.777+0000 7fbe737fe640 1 -- 172.21.15.70:0/20735545 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe8c0b2ba0 con 0x7fbe8c1599d0 2023-10-08T05:37:13.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.035+0000 7fbe8bfff640 1 -- 172.21.15.70:0/20735545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fbe8c0b2ba0 con 0x7fbe8c1599d0 2023-10-08T05:37:13.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.213+0000 7fbe737fe640 1 -- 172.21.15.70:0/20735545 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbe8c0b2ba0 con 0x7fbe8c1599d0 2023-10-08T05:37:13.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 -- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbe7806ae70 msgr2=0x7fbe7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:13.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbe7806ae70 0x7fbe7806d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c06d0d0 tx=0x7fbe7c008040 comp rx=0 tx=0).stop 2023-10-08T05:37:13.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 -- 172.21.15.70:0/20735545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c1599d0 msgr2=0x7fbe8c155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:13.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c1599d0 0x7fbe8c155150 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009d90 tx=0x7fbe84004a40 comp rx=0 tx=0).stop 2023-10-08T05:37:13.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 -- 172.21.15.70:0/20735545 shutdown_connections 2023-10-08T05:37:13.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbe7806ae70 0x7fbe7806d330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8c1599d0 0x7fbe8c155150 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe8c0cc210 0x7fbe8c14ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 --2- 172.21.15.70:0/20735545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe8c0b07d0 0x7fbe8c0b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 -- 172.21.15.70:0/20735545 >> 172.21.15.70:0/20735545 conn(0x7fbe8c0cb0c0 msgr2=0x7fbe8c0cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:13.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 -- 172.21.15.70:0/20735545 shutdown_connections 2023-10-08T05:37:13.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.216+0000 7fbe8bfff640 1 -- 172.21.15.70:0/20735545 wait complete. 2023-10-08T05:37:13.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T05:37:13.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.461+0000 7f0647fff640 1 Processor -- start 2023-10-08T05:37:13.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.461+0000 7f0647fff640 1 -- start start 2023-10-08T05:37:13.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.463+0000 7f0647fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f064815a820 0x7f06480aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:13.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.463+0000 7f0647fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06480cab60 con 0x7f064815ac00 2023-10-08T05:37:13.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.463+0000 7f0647fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06480cb960 con 0x7f06480ca680 2023-10-08T05:37:13.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.463+0000 7f0647fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06480cbdf0 con 0x7f064815a820 2023-10-08T05:37:13.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.463+0000 7f0646ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f064815a820 0x7f06480aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:13.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.463+0000 7f06477fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f064815ac00 0x7f06480c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33150/0 (socket says 172.21.15.70:33150) 2023-10-08T05:37:13.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.463+0000 7f06477fe640 1 -- 172.21.15.70:0/3453932479 learned_addr learned my addr 172.21.15.70:0/3453932479 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:13.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.463+0000 7f0646ffd640 1 -- 172.21.15.70:0/3453932479 >> v1:172.21.15.181:6789/0 conn(0x7f06480ca680 legacy=0x7f06480c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:13.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.463+0000 7f0646ffd640 1 -- 172.21.15.70:0/3453932479 >> v1:172.21.15.70:6789/0 conn(0x7f064815ac00 legacy=0x7f06480c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:13.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.463+0000 7f0646ffd640 1 -- 172.21.15.70:0/3453932479 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06480c9f20 con 0x7f064815a820 2023-10-08T05:37:13.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.465+0000 7f0646ffd640 1 --2- 172.21.15.70:0/3453932479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815a820 0x7f06480aff70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f06380097a0 tx=0x7f063802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a1e594c5485455 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.465+0000 7f06467fc640 1 -- 172.21.15.70:0/3453932479 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063802e070 con 0x7f064815a820 2023-10-08T05:37:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.465+0000 7f06467fc640 1 -- 172.21.15.70:0/3453932479 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06380315a0 con 0x7f064815a820 2023-10-08T05:37:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.465+0000 7f06467fc640 1 -- 172.21.15.70:0/3453932479 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638031930 con 0x7f064815a820 2023-10-08T05:37:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.465+0000 7f0647fff640 1 -- 172.21.15.70:0/3453932479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815a820 msgr2=0x7f06480aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.465+0000 7f0647fff640 1 --2- 172.21.15.70:0/3453932479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815a820 0x7f06480aff70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f06380097a0 tx=0x7f063802b840 comp rx=0 tx=0).stop 2023-10-08T05:37:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.466+0000 7f0647fff640 1 -- 172.21.15.70:0/3453932479 shutdown_connections 2023-10-08T05:37:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.466+0000 7f0647fff640 1 --2- 172.21.15.70:0/3453932479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815a820 0x7f06480aff70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.466+0000 7f0647fff640 1 -- 172.21.15.70:0/3453932479 >> 172.21.15.70:0/3453932479 conn(0x7f064806c050 msgr2=0x7f064806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:13.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.466+0000 7f0647fff640 1 -- 172.21.15.70:0/3453932479 shutdown_connections 2023-10-08T05:37:13.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.466+0000 7f0647fff640 1 -- 172.21.15.70:0/3453932479 wait complete. 2023-10-08T05:37:13.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.467+0000 7f0647fff640 1 Processor -- start 2023-10-08T05:37:13.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.467+0000 7f0647fff640 1 -- start start 2023-10-08T05:37:13.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.468+0000 7f0647fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06480ca680 0x7f064814f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.468+0000 7f0647fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064815a820 0x7f0648152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.468+0000 7f0647fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815ac00 0x7f0648153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.468+0000 7f0647fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06480c6fe0 con 0x7f064815ac00 2023-10-08T05:37:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.468+0000 7f0647fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0648153840 con 0x7f06480ca680 2023-10-08T05:37:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.468+0000 7f0647fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06480ae930 con 0x7f064815a820 2023-10-08T05:37:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f06477fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815ac00 0x7f0648153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f06477fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815ac00 0x7f0648153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43892/0 (socket says 172.21.15.70:43892) 2023-10-08T05:37:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f06477fe640 1 -- 172.21.15.70:0/3462309605 learned_addr learned my addr 172.21.15.70:0/3462309605 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f0646ffd640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06480ca680 0x7f064814f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:13.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f063e7fc640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064815a820 0x7f0648152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:13.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f06477fe640 1 -- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064815a820 msgr2=0x7f0648152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:13.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f06477fe640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064815a820 0x7f0648152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f06477fe640 1 -- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06480ca680 msgr2=0x7f064814f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:13.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f06477fe640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06480ca680 0x7f064814f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f06477fe640 1 -- 172.21.15.70:0/3462309605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638009400 con 0x7f064815ac00 2023-10-08T05:37:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f0646ffd640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06480ca680 0x7f064814f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.469+0000 7f06477fe640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815ac00 0x7f0648153300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f064000c340 tx=0x7f064000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.470+0000 7f0644ff9640 1 -- 172.21.15.70:0/3462309605 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640005ce0 con 0x7f064815ac00 2023-10-08T05:37:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.470+0000 7f0644ff9640 1 -- 172.21.15.70:0/3462309605 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0640005e40 con 0x7f064815ac00 2023-10-08T05:37:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.470+0000 7f0644ff9640 1 -- 172.21.15.70:0/3462309605 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06400072d0 con 0x7f064815ac00 2023-10-08T05:37:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.470+0000 7f0647fff640 1 -- 172.21.15.70:0/3462309605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06481512d0 con 0x7f064815ac00 2023-10-08T05:37:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.470+0000 7f0647fff640 1 -- 172.21.15.70:0/3462309605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06480c6c00 con 0x7f064815ac00 2023-10-08T05:37:13.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.471+0000 7f0647fff640 1 -- 172.21.15.70:0/3462309605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06480c7120 con 0x7f064815ac00 2023-10-08T05:37:13.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.472+0000 7f0644ff9640 1 -- 172.21.15.70:0/3462309605 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0640007430 con 0x7f064815ac00 2023-10-08T05:37:13.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.473+0000 7f0644ff9640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f062806ae70 0x7f062806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.473+0000 7f0644ff9640 1 -- 172.21.15.70:0/3462309605 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0640088dc0 con 0x7f064815ac00 2023-10-08T05:37:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.474+0000 7f0646ffd640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f062806ae70 0x7f062806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.475+0000 7f0646ffd640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f062806ae70 0x7f062806d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0638036000 tx=0x7f063803d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:13.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.478+0000 7f0644ff9640 1 -- 172.21.15.70:0/3462309605 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06480c7120 con 0x7f064815ac00 2023-10-08T05:37:13.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.672+0000 7f0647fff640 1 -- 172.21.15.70:0/3462309605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f06480c7120 con 0x7f064815ac00 2023-10-08T05:37:13.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.679+0000 7f0644ff9640 1 -- 172.21.15.70:0/3462309605 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f06480c7120 con 0x7f064815ac00 2023-10-08T05:37:13.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.682+0000 7f0647fff640 1 -- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f062806ae70 msgr2=0x7f062806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:13.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.682+0000 7f0647fff640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f062806ae70 0x7f062806d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0638036000 tx=0x7f063803d040 comp rx=0 tx=0).stop 2023-10-08T05:37:13.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.682+0000 7f0647fff640 1 -- 172.21.15.70:0/3462309605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815ac00 msgr2=0x7f0648153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:13.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.682+0000 7f0647fff640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815ac00 0x7f0648153300 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f064000c340 tx=0x7f064000c800 comp rx=0 tx=0).stop 2023-10-08T05:37:13.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.682+0000 7f0647fff640 1 -- 172.21.15.70:0/3462309605 shutdown_connections 2023-10-08T05:37:13.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.682+0000 7f0647fff640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f062806ae70 0x7f062806d330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.682+0000 7f0647fff640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f064815ac00 0x7f0648153300 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.682+0000 7f0647fff640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f064815a820 0x7f0648152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.682+0000 7f0647fff640 1 --2- 172.21.15.70:0/3462309605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06480ca680 0x7f064814f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:13.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.683+0000 7f0647fff640 1 -- 172.21.15.70:0/3462309605 >> 172.21.15.70:0/3462309605 conn(0x7f064806c050 msgr2=0x7f06481573c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:13.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.683+0000 7f0647fff640 1 -- 172.21.15.70:0/3462309605 shutdown_connections 2023-10-08T05:37:13.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:13.683+0000 7f0647fff640 1 -- 172.21.15.70:0/3462309605 wait complete. 2023-10-08T05:37:13.704 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:37:13.705 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:37:13.825 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:37:13.950 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:37:13.950 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:37:13.951 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:14.011 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:37:14.011 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:14.063 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:37:14.063 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:14.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:14.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.371+0000 7f846f052640 1 Processor -- start 2023-10-08T05:37:14.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.371+0000 7f846f052640 1 -- start start 2023-10-08T05:37:14.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.373+0000 7f846f052640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84680cb580 0x7f8468165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.373+0000 7f846f052640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846815b2b0 con 0x7f84680c90b0 2023-10-08T05:37:14.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.373+0000 7f846f052640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846815b590 con 0x7f846815ab40 2023-10-08T05:37:14.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.373+0000 7f846f052640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846815bf30 con 0x7f84680cb580 2023-10-08T05:37:14.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.373+0000 7f846d5ef640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84680cb580 0x7f8468165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.373+0000 7f8467fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f84680c90b0 0x7f84680cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50018/0 (socket says 172.21.15.70:50018) 2023-10-08T05:37:14.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.373+0000 7f8467fff640 1 -- 172.21.15.70:0/2612258014 learned_addr learned my addr 172.21.15.70:0/2612258014 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:14.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.374+0000 7f846d5ef640 1 -- 172.21.15.70:0/2612258014 >> v1:172.21.15.181:6789/0 conn(0x7f846815ab40 legacy=0x7f84680ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:14.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.374+0000 7f846d5ef640 1 -- 172.21.15.70:0/2612258014 >> v1:172.21.15.70:6789/0 conn(0x7f84680c90b0 legacy=0x7f84680cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.374+0000 7f846d5ef640 1 -- 172.21.15.70:0/2612258014 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846815b870 con 0x7f84680cb580 2023-10-08T05:37:14.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.374+0000 7f84677fe640 1 -- 172.21.15.70:0/2612258014 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528955981 0 0) 0x7f846815b2b0 con 0x7f84680c90b0 2023-10-08T05:37:14.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.375+0000 7f846d5ef640 1 --2- 172.21.15.70:0/2612258014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680cb580 0x7f8468165b20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8460009d90 tx=0x7f846002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a136b148968a0b03 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:14.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.375+0000 7f84677fe640 1 -- 172.21.15.70:0/2612258014 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846002cc60 con 0x7f84680cb580 2023-10-08T05:37:14.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.375+0000 7f84677fe640 1 -- 172.21.15.70:0/2612258014 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846002cdc0 con 0x7f84680cb580 2023-10-08T05:37:14.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.375+0000 7f84677fe640 1 -- 172.21.15.70:0/2612258014 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460034660 con 0x7f84680cb580 2023-10-08T05:37:14.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.376+0000 7f846f052640 1 -- 172.21.15.70:0/2612258014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680cb580 msgr2=0x7f8468165b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.376+0000 7f846f052640 1 --2- 172.21.15.70:0/2612258014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680cb580 0x7f8468165b20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8460009d90 tx=0x7f846002beb0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.376+0000 7f846f052640 1 -- 172.21.15.70:0/2612258014 shutdown_connections 2023-10-08T05:37:14.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.376+0000 7f846f052640 1 --2- 172.21.15.70:0/2612258014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680cb580 0x7f8468165b20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.376+0000 7f846f052640 1 -- 172.21.15.70:0/2612258014 >> 172.21.15.70:0/2612258014 conn(0x7f846806a380 msgr2=0x7f84680b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:14.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.376+0000 7f846f052640 1 -- 172.21.15.70:0/2612258014 shutdown_connections 2023-10-08T05:37:14.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.376+0000 7f846f052640 1 -- 172.21.15.70:0/2612258014 wait complete. 2023-10-08T05:37:14.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.378+0000 7f846f052640 1 Processor -- start 2023-10-08T05:37:14.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.378+0000 7f846f052640 1 -- start start 2023-10-08T05:37:14.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846f052640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680c90b0 0x7f8468158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846f052640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84680cb580 0x7f84680ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846f052640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f846815ab40 0x7f84680ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846f052640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846815f7a0 con 0x7f84680c90b0 2023-10-08T05:37:14.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846f052640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84680c3970 con 0x7f84680cb580 2023-10-08T05:37:14.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846f052640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8468159030 con 0x7f846815ab40 2023-10-08T05:37:14.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846cdee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680c90b0 0x7f8468158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846cdee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680c90b0 0x7f8468158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37038/0 (socket says 172.21.15.70:37038) 2023-10-08T05:37:14.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846cdee640 1 -- 172.21.15.70:0/3895142298 learned_addr learned my addr 172.21.15.70:0/3895142298 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:14.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846cdee640 1 -- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f846815ab40 msgr2=0x7f84680ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846cdee640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f846815ab40 0x7f84680ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846cdee640 1 -- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84680cb580 msgr2=0x7f84680ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846cdee640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84680cb580 0x7f84680ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.379+0000 7f846cdee640 1 -- 172.21.15.70:0/3895142298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8460009a70 con 0x7f84680c90b0 2023-10-08T05:37:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.380+0000 7f846cdee640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680c90b0 0x7f8468158af0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f845c00b3f0 tx=0x7f845c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.380+0000 7f84657fa640 1 -- 172.21.15.70:0/3895142298 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c011830 con 0x7f84680c90b0 2023-10-08T05:37:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.380+0000 7f84657fa640 1 -- 172.21.15.70:0/3895142298 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c011990 con 0x7f84680c90b0 2023-10-08T05:37:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.380+0000 7f846f052640 1 -- 172.21.15.70:0/3895142298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8468159ae0 con 0x7f84680c90b0 2023-10-08T05:37:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.380+0000 7f84657fa640 1 -- 172.21.15.70:0/3895142298 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c011d40 con 0x7f84680c90b0 2023-10-08T05:37:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.380+0000 7f846f052640 1 -- 172.21.15.70:0/3895142298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8468159240 con 0x7f84680c90b0 2023-10-08T05:37:14.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.381+0000 7f846f052640 1 -- 172.21.15.70:0/3895142298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84680c36d0 con 0x7f84680c90b0 2023-10-08T05:37:14.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.382+0000 7f84657fa640 1 -- 172.21.15.70:0/3895142298 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f845c01a400 con 0x7f84680c90b0 2023-10-08T05:37:14.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.383+0000 7f84657fa640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f844c06ae70 0x7f844c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.383+0000 7f84657fa640 1 -- 172.21.15.70:0/3895142298 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f845c089820 con 0x7f84680c90b0 2023-10-08T05:37:14.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.384+0000 7f8467fff640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f844c06ae70 0x7f844c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.385+0000 7f8467fff640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f844c06ae70 0x7f844c06d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8454007fd0 tx=0x7f8454007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:14.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.388+0000 7f84657fa640 1 -- 172.21.15.70:0/3895142298 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84680c36d0 con 0x7f84680c90b0 2023-10-08T05:37:14.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.578+0000 7f846f052640 1 -- 172.21.15.70:0/3895142298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84680c36d0 con 0x7f84680c90b0 2023-10-08T05:37:14.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.582+0000 7f84657fa640 1 -- 172.21.15.70:0/3895142298 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7f84680c36d0 con 0x7f84680c90b0 2023-10-08T05:37:14.584 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:14.584 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:14.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 -- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f844c06ae70 msgr2=0x7f844c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f844c06ae70 0x7f844c06d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8454007fd0 tx=0x7f8454007920 comp rx=0 tx=0).stop 2023-10-08T05:37:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 -- 172.21.15.70:0/3895142298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680c90b0 msgr2=0x7f8468158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680c90b0 0x7f8468158af0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f845c00b3f0 tx=0x7f845c00b8b0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 -- 172.21.15.70:0/3895142298 shutdown_connections 2023-10-08T05:37:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f844c06ae70 0x7f844c06d330 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f846815ab40 0x7f84680ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84680cb580 0x7f84680ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 --2- 172.21.15.70:0/3895142298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84680c90b0 0x7f8468158af0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 -- 172.21.15.70:0/3895142298 >> 172.21.15.70:0/3895142298 conn(0x7f846806a380 msgr2=0x7f84680ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 -- 172.21.15.70:0/3895142298 shutdown_connections 2023-10-08T05:37:14.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.585+0000 7f846f052640 1 -- 172.21.15.70:0/3895142298 wait complete. 2023-10-08T05:37:14.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:37:14.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:14.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.806+0000 7f022630c640 1 Processor -- start 2023-10-08T05:37:14.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.806+0000 7f022630c640 1 -- start start 2023-10-08T05:37:14.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.808+0000 7f022630c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200b1750 0x7f022006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.808+0000 7f022630c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022015bf30 con 0x7f022006d240 2023-10-08T05:37:14.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.808+0000 7f022630c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02200b1bb0 con 0x7f022015aad0 2023-10-08T05:37:14.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.808+0000 7f022630c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022006d6d0 con 0x7f02200b1750 2023-10-08T05:37:14.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.808+0000 7f02248a9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f022006d240 0x7f0220070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50032/0 (socket says 172.21.15.70:50032) 2023-10-08T05:37:14.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.808+0000 7f02248a9640 1 -- 172.21.15.70:0/3009469657 learned_addr learned my addr 172.21.15.70:0/3009469657 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:14.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021f7fe640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200b1750 0x7f022006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671594445 0 0) 0x7f022015bf30 con 0x7f022006d240 2023-10-08T05:37:14.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0208003520 con 0x7f022006d240 2023-10-08T05:37:14.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2595345494 0 0) 0x7f0208003520 con 0x7f022006d240 2023-10-08T05:37:14.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200b1750 msgr2=0x7f022006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:14.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021effd640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200b1750 0x7f022006cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 >> v1:172.21.15.181:6789/0 conn(0x7f022015aad0 legacy=0x7f02200c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022015b320 con 0x7f022006d240 2023-10-08T05:37:14.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f02180019f0 con 0x7f022006d240 2023-10-08T05:37:14.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 >> v1:172.21.15.70:6789/0 conn(0x7f022006d240 legacy=0x7f0220070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.809+0000 7f021effd640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200b1750 0x7f02080039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f021effd640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0208003f70 0x7f02080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f021effd640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208004d00 0x7f0208005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022006d6d0 con 0x7f0208004d00 2023-10-08T05:37:14.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020800bac0 con 0x7f0208003f70 2023-10-08T05:37:14.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020800bc80 con 0x7f02200b1750 2023-10-08T05:37:14.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f02248a9640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208004d00 0x7f0208005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f021f7fe640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0208003f70 0x7f02080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f021ffff640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200b1750 0x7f02080039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f02248a9640 1 -- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200b1750 msgr2=0x7f02080039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:14.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f02248a9640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200b1750 0x7f02080039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f02248a9640 1 -- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0208003f70 msgr2=0x7f02080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:14.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f02248a9640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0208003f70 0x7f02080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f02248a9640 1 -- 172.21.15.70:0/3009469657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0208003520 con 0x7f0208004d00 2023-10-08T05:37:14.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f021f7fe640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0208003f70 0x7f02080047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:14.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.810+0000 7f021ffff640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200b1750 0x7f02080039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:14.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.811+0000 7f02248a9640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208004d00 0x7f0208005180 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f021800a760 tx=0x7f02080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e95396313bfa5c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:14.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.812+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218034630 con 0x7f0208004d00 2023-10-08T05:37:14.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.812+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021802f070 con 0x7f0208004d00 2023-10-08T05:37:14.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.812+0000 7f021effd640 1 -- 172.21.15.70:0/3009469657 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02180349f0 con 0x7f0208004d00 2023-10-08T05:37:14.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.812+0000 7f022630c640 1 -- 172.21.15.70:0/3009469657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208004d00 msgr2=0x7f0208005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.812+0000 7f022630c640 1 --2- 172.21.15.70:0/3009469657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208004d00 0x7f0208005180 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f021800a760 tx=0x7f02080057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.812+0000 7f02248a9640 1 -- 172.21.15.70:0/3009469657 reap_dead start 2023-10-08T05:37:14.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.812+0000 7f022630c640 1 -- 172.21.15.70:0/3009469657 shutdown_connections 2023-10-08T05:37:14.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.812+0000 7f022630c640 1 -- 172.21.15.70:0/3009469657 >> 172.21.15.70:0/3009469657 conn(0x7f0220069d50 msgr2=0x7f022006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:14.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.812+0000 7f022630c640 1 -- 172.21.15.70:0/3009469657 shutdown_connections 2023-10-08T05:37:14.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.812+0000 7f022630c640 1 -- 172.21.15.70:0/3009469657 wait complete. 2023-10-08T05:37:14.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.814+0000 7f022630c640 1 Processor -- start 2023-10-08T05:37:14.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.814+0000 7f022630c640 1 -- start start 2023-10-08T05:37:14.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.815+0000 7f022630c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0220152350 0x7f02200cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.815+0000 7f022630c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200cb690 0x7f02200cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.815+0000 7f022630c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02200cd5f0 0x7f02201f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.815+0000 7f022630c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200cdda0 con 0x7f0220152350 2023-10-08T05:37:14.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.815+0000 7f022630c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200cbb10 con 0x7f02200cd5f0 2023-10-08T05:37:14.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.815+0000 7f022630c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02201f0cc0 con 0x7f02200cb690 2023-10-08T05:37:14.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.815+0000 7f021ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0220152350 0x7f02200cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.815+0000 7f021ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0220152350 0x7f02200cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37046/0 (socket says 172.21.15.70:37046) 2023-10-08T05:37:14.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.815+0000 7f021ffff640 1 -- 172.21.15.70:0/1174341066 learned_addr learned my addr 172.21.15.70:0/1174341066 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:14.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.816+0000 7f02248a9640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02200cd5f0 0x7f02201f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.816+0000 7f021f7fe640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200cb690 0x7f02200cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.816+0000 7f021ffff640 1 -- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200cb690 msgr2=0x7f02200cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.816+0000 7f021ffff640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200cb690 0x7f02200cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.816+0000 7f021ffff640 1 -- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02200cd5f0 msgr2=0x7f02201f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.816+0000 7f021ffff640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02200cd5f0 0x7f02201f3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.816+0000 7f021ffff640 1 -- 172.21.15.70:0/1174341066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02201f1150 con 0x7f0220152350 2023-10-08T05:37:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.816+0000 7f021ffff640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0220152350 0x7f02200cb150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f021000d7a0 tx=0x7f021000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.816+0000 7f021cff9640 1 -- 172.21.15.70:0/1174341066 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210005270 con 0x7f0220152350 2023-10-08T05:37:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.816+0000 7f021cff9640 1 -- 172.21.15.70:0/1174341066 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02100053d0 con 0x7f0220152350 2023-10-08T05:37:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.817+0000 7f021cff9640 1 -- 172.21.15.70:0/1174341066 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021000b870 con 0x7f0220152350 2023-10-08T05:37:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.817+0000 7f022630c640 1 -- 172.21.15.70:0/1174341066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02201f0120 con 0x7f0220152350 2023-10-08T05:37:14.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.817+0000 7f022630c640 1 -- 172.21.15.70:0/1174341066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02201f0670 con 0x7f0220152350 2023-10-08T05:37:14.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.818+0000 7f022630c640 1 -- 172.21.15.70:0/1174341066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022004e160 con 0x7f0220152350 2023-10-08T05:37:14.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.819+0000 7f021cff9640 1 -- 172.21.15.70:0/1174341066 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f021000b9d0 con 0x7f0220152350 2023-10-08T05:37:14.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.820+0000 7f021cff9640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f020c06aea0 0x7f020c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:14.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.820+0000 7f021cff9640 1 -- 172.21.15.70:0/1174341066 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f02100895a0 con 0x7f0220152350 2023-10-08T05:37:14.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.820+0000 7f021f7fe640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f020c06aea0 0x7f020c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:14.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.822+0000 7f021f7fe640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f020c06aea0 0x7f020c06d360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0220156220 tx=0x7f0214008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:14.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:14.824+0000 7f021cff9640 1 -- 172.21.15.70:0/1174341066 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0210054370 con 0x7f0220152350 2023-10-08T05:37:15.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.012+0000 7f022630c640 1 -- 172.21.15.70:0/1174341066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02200b2970 con 0x7f0220152350 2023-10-08T05:37:15.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.013+0000 7f021cff9640 1 -- 172.21.15.70:0/1174341066 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0210057990 con 0x7f0220152350 2023-10-08T05:37:15.015 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 -- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f020c06aea0 msgr2=0x7f020c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f020c06aea0 0x7f020c06d360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0220156220 tx=0x7f0214008040 comp rx=0 tx=0).stop 2023-10-08T05:37:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 -- 172.21.15.70:0/1174341066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0220152350 msgr2=0x7f02200cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0220152350 0x7f02200cb150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f021000d7a0 tx=0x7f021000dc60 comp rx=0 tx=0).stop 2023-10-08T05:37:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 -- 172.21.15.70:0/1174341066 shutdown_connections 2023-10-08T05:37:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f020c06aea0 0x7f020c06d360 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02200cd5f0 0x7f02201f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02200cb690 0x7f02200cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 --2- 172.21.15.70:0/1174341066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0220152350 0x7f02200cb150 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 -- 172.21.15.70:0/1174341066 >> 172.21.15.70:0/1174341066 conn(0x7f0220069d50 msgr2=0x7f022015be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:15.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 -- 172.21.15.70:0/1174341066 shutdown_connections 2023-10-08T05:37:15.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.016+0000 7f022630c640 1 -- 172.21.15.70:0/1174341066 wait complete. 2023-10-08T05:37:15.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:37:15.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.255+0000 7fab3cd75640 1 Processor -- start 2023-10-08T05:37:15.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.256+0000 7fab3cd75640 1 -- start start 2023-10-08T05:37:15.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.257+0000 7fab3cd75640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab381599d0 0x7fab380cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.257+0000 7fab3cd75640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fab380b3c50 0x7fab380caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.257+0000 7fab3cd75640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab38159db0 con 0x7fab380cb410 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.257+0000 7fab3cd75640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab38071fe0 con 0x7fab381599d0 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.257+0000 7fab3cd75640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab380b44c0 con 0x7fab380b3c50 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.257+0000 7fab377fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab381599d0 0x7fab380cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.257+0000 7fab37fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fab380cb410 0x7fab380c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50034/0 (socket says 172.21.15.70:50034) 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.257+0000 7fab377fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab381599d0 0x7fab380cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37062/0 (socket says 172.21.15.70:37062) 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.257+0000 7fab37fff640 1 -- 172.21.15.70:0/3323998665 learned_addr learned my addr 172.21.15.70:0/3323998665 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.258+0000 7fab2ffff640 1 --2- 172.21.15.70:0/3323998665 >> v2:172.21.15.181:3300/0 conn(0x7fab380b3c50 0x7fab380caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.258+0000 7fab377fe640 1 -- 172.21.15.70:0/3323998665 >> v2:172.21.15.181:3300/0 conn(0x7fab380b3c50 msgr2=0x7fab380caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:15.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.258+0000 7fab377fe640 1 --2- 172.21.15.70:0/3323998665 >> v2:172.21.15.181:3300/0 conn(0x7fab380b3c50 0x7fab380caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.258+0000 7fab377fe640 1 -- 172.21.15.70:0/3323998665 >> v1:172.21.15.70:6789/0 conn(0x7fab380cb410 legacy=0x7fab380c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.258+0000 7fab377fe640 1 -- 172.21.15.70:0/3323998665 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380cb860 con 0x7fab381599d0 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.259+0000 7fab377fe640 1 --2- 172.21.15.70:0/3323998665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab381599d0 0x7fab380cdcd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab380709e0 tx=0x7fab2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5db0c52e8aebd6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.259+0000 7fab36ffd640 1 -- 172.21.15.70:0/3323998665 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28031830 con 0x7fab381599d0 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.259+0000 7fab36ffd640 1 -- 172.21.15.70:0/3323998665 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab28031990 con 0x7fab381599d0 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.259+0000 7fab36ffd640 1 -- 172.21.15.70:0/3323998665 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28031d20 con 0x7fab381599d0 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.260+0000 7fab3cd75640 1 -- 172.21.15.70:0/3323998665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab381599d0 msgr2=0x7fab380cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.260+0000 7fab3cd75640 1 --2- 172.21.15.70:0/3323998665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab381599d0 0x7fab380cdcd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab380709e0 tx=0x7fab2802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.260+0000 7fab3cd75640 1 -- 172.21.15.70:0/3323998665 shutdown_connections 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.260+0000 7fab3cd75640 1 --2- 172.21.15.70:0/3323998665 >> v2:172.21.15.181:3300/0 conn(0x7fab380b3c50 0x7fab380caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.260+0000 7fab3cd75640 1 --2- 172.21.15.70:0/3323998665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab381599d0 0x7fab380cdcd0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.260+0000 7fab3cd75640 1 -- 172.21.15.70:0/3323998665 >> 172.21.15.70:0/3323998665 conn(0x7fab3806c050 msgr2=0x7fab3806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:15.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.260+0000 7fab3cd75640 1 -- 172.21.15.70:0/3323998665 shutdown_connections 2023-10-08T05:37:15.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.260+0000 7fab3cd75640 1 -- 172.21.15.70:0/3323998665 wait complete. 2023-10-08T05:37:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.262+0000 7fab3cd75640 1 Processor -- start 2023-10-08T05:37:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.262+0000 7fab3cd75640 1 -- start start 2023-10-08T05:37:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.263+0000 7fab3cd75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab380b3c50 0x7fab380c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.263+0000 7fab3cd75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab380cb410 0x7fab38146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.263+0000 7fab3cd75640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab381599d0 0x7fab381470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.263+0000 7fab3cd75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38145700 con 0x7fab380cb410 2023-10-08T05:37:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.263+0000 7fab3cd75640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38144b50 con 0x7fab381599d0 2023-10-08T05:37:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.263+0000 7fab3cd75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab38144d60 con 0x7fab380b3c50 2023-10-08T05:37:15.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.263+0000 7fab2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab380cb410 0x7fab38146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.263+0000 7fab2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab380cb410 0x7fab38146b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37066/0 (socket says 172.21.15.70:37066) 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.263+0000 7fab2ffff640 1 -- 172.21.15.70:0/827043720 learned_addr learned my addr 172.21.15.70:0/827043720 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab37fff640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab381599d0 0x7fab381470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab377fe640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab380b3c50 0x7fab380c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab2ffff640 1 -- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab380b3c50 msgr2=0x7fab380c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab2ffff640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab380b3c50 0x7fab380c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab2ffff640 1 -- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab381599d0 msgr2=0x7fab381470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab2ffff640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab381599d0 0x7fab381470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab2ffff640 1 -- 172.21.15.70:0/827043720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28009400 con 0x7fab380cb410 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab37fff640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab381599d0 0x7fab381470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab2ffff640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab380cb410 0x7fab38146b80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fab2000c7f0 tx=0x7fab2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:15.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab34ff9640 1 -- 172.21.15.70:0/827043720 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20004220 con 0x7fab380cb410 2023-10-08T05:37:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab34ff9640 1 -- 172.21.15.70:0/827043720 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab200043a0 con 0x7fab380cb410 2023-10-08T05:37:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab34ff9640 1 -- 172.21.15.70:0/827043720 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20007860 con 0x7fab380cb410 2023-10-08T05:37:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.264+0000 7fab3cd75640 1 -- 172.21.15.70:0/827043720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab38145040 con 0x7fab380cb410 2023-10-08T05:37:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.265+0000 7fab3cd75640 1 -- 172.21.15.70:0/827043720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab38157e70 con 0x7fab380cb410 2023-10-08T05:37:15.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.266+0000 7fab3cd75640 1 -- 172.21.15.70:0/827043720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab38074150 con 0x7fab380cb410 2023-10-08T05:37:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.267+0000 7fab34ff9640 1 -- 172.21.15.70:0/827043720 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fab20007bd0 con 0x7fab380cb410 2023-10-08T05:37:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.267+0000 7fab34ff9640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fab1806ae70 0x7fab1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.268+0000 7fab34ff9640 1 -- 172.21.15.70:0/827043720 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fab200899f0 con 0x7fab380cb410 2023-10-08T05:37:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.268+0000 7fab377fe640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fab1806ae70 0x7fab1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.269+0000 7fab377fe640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fab1806ae70 0x7fab1806d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab28036000 tx=0x7fab2803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:15.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.272+0000 7fab34ff9640 1 -- 172.21.15.70:0/827043720 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab20054640 con 0x7fab380cb410 2023-10-08T05:37:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.459+0000 7fab3cd75640 1 -- 172.21.15.70:0/827043720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab38071c40 con 0x7fab380cb410 2023-10-08T05:37:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.460+0000 7fab34ff9640 1 -- 172.21.15.70:0/827043720 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab2001e020 con 0x7fab380cb410 2023-10-08T05:37:15.462 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.462+0000 7fab3cd75640 1 -- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fab1806ae70 msgr2=0x7fab1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.462+0000 7fab3cd75640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fab1806ae70 0x7fab1806d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fab28036000 tx=0x7fab2803c040 comp rx=0 tx=0).stop 2023-10-08T05:37:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.462+0000 7fab3cd75640 1 -- 172.21.15.70:0/827043720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab380cb410 msgr2=0x7fab38146b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.462+0000 7fab3cd75640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab380cb410 0x7fab38146b80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fab2000c7f0 tx=0x7fab2000ccb0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.463+0000 7fab3cd75640 1 -- 172.21.15.70:0/827043720 shutdown_connections 2023-10-08T05:37:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.463+0000 7fab3cd75640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fab1806ae70 0x7fab1806d330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.463+0000 7fab3cd75640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab381599d0 0x7fab381470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.463+0000 7fab3cd75640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab380cb410 0x7fab38146b80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.463+0000 7fab3cd75640 1 --2- 172.21.15.70:0/827043720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab380b3c50 0x7fab380c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.463+0000 7fab3cd75640 1 -- 172.21.15.70:0/827043720 >> 172.21.15.70:0/827043720 conn(0x7fab3806c050 msgr2=0x7fab380cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.463+0000 7fab3cd75640 1 -- 172.21.15.70:0/827043720 shutdown_connections 2023-10-08T05:37:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.463+0000 7fab3cd75640 1 -- 172.21.15.70:0/827043720 wait complete. 2023-10-08T05:37:15.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:37:15.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.701+0000 7f3f220da640 1 Processor -- start 2023-10-08T05:37:15.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.701+0000 7f3f220da640 1 -- start start 2023-10-08T05:37:15.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.702+0000 7f3f220da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c06c3a0 0x7f3f1c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.702+0000 7f3f220da640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f1c06db60 0x7f3f1c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.702+0000 7f3f220da640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c15b310 con 0x7f3f1c15aa00 2023-10-08T05:37:15.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.702+0000 7f3f220da640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c0ca230 con 0x7f3f1c06db60 2023-10-08T05:37:15.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.702+0000 7f3f220da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c06d290 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.703+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c06c3a0 0x7f3f1c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.703+0000 7f3f13fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f1c06db60 0x7f3f1c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.703+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c06c3a0 0x7f3f1c0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:38516/0 (socket says 172.21.15.70:38516) 2023-10-08T05:37:15.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.703+0000 7f3f13fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f1c06db60 0x7f3f1c06c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:43920/0 (socket says 172.21.15.70:43920) 2023-10-08T05:37:15.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.703+0000 7f3f1b7fe640 1 -- 172.21.15.70:0/23955124 learned_addr learned my addr 172.21.15.70:0/23955124 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:15.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.703+0000 7f3f1bfff640 1 --1- 172.21.15.70:0/23955124 >> v1:172.21.15.181:6789/0 conn(0x7f3f1c15aa00 0x7f3f1c0b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:53510/0 (socket says 172.21.15.70:53510) 2023-10-08T05:37:15.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.704+0000 7f3f13fff640 1 -- 172.21.15.70:0/23955124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c06c3a0 msgr2=0x7f3f1c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:15.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.704+0000 7f3f13fff640 1 --2- 172.21.15.70:0/23955124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c06c3a0 0x7f3f1c0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.704+0000 7f3f13fff640 1 -- 172.21.15.70:0/23955124 >> v1:172.21.15.181:6789/0 conn(0x7f3f1c15aa00 legacy=0x7f3f1c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.704+0000 7f3f13fff640 1 -- 172.21.15.70:0/23955124 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0c9860 con 0x7f3f1c06db60 2023-10-08T05:37:15.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.705+0000 7f3f13fff640 1 --2- 172.21.15.70:0/23955124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f1c06db60 0x7f3f1c06c780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3f040097a0 tx=0x7f3f0402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565a9777d1b38fdd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:15.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.706+0000 7f3f1affd640 1 -- 172.21.15.70:0/23955124 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0402d040 con 0x7f3f1c06db60 2023-10-08T05:37:15.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.706+0000 7f3f1affd640 1 -- 172.21.15.70:0/23955124 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04031810 con 0x7f3f1c06db60 2023-10-08T05:37:15.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.706+0000 7f3f1affd640 1 -- 172.21.15.70:0/23955124 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04031ba0 con 0x7f3f1c06db60 2023-10-08T05:37:15.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.707+0000 7f3f220da640 1 -- 172.21.15.70:0/23955124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f1c06db60 msgr2=0x7f3f1c06c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.707+0000 7f3f220da640 1 --2- 172.21.15.70:0/23955124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f1c06db60 0x7f3f1c06c780 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3f040097a0 tx=0x7f3f0402b8b0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.707+0000 7f3f220da640 1 -- 172.21.15.70:0/23955124 shutdown_connections 2023-10-08T05:37:15.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.707+0000 7f3f220da640 1 --2- 172.21.15.70:0/23955124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f1c06db60 0x7f3f1c06c780 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.707+0000 7f3f220da640 1 --2- 172.21.15.70:0/23955124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c06c3a0 0x7f3f1c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.707+0000 7f3f220da640 1 -- 172.21.15.70:0/23955124 >> 172.21.15.70:0/23955124 conn(0x7f3f1c06a5d0 msgr2=0x7f3f1c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:15.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.707+0000 7f3f220da640 1 -- 172.21.15.70:0/23955124 shutdown_connections 2023-10-08T05:37:15.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.707+0000 7f3f220da640 1 -- 172.21.15.70:0/23955124 wait complete. 2023-10-08T05:37:15.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.709+0000 7f3f220da640 1 Processor -- start 2023-10-08T05:37:15.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.709+0000 7f3f220da640 1 -- start start 2023-10-08T05:37:15.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f220da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f1c06c3a0 0x7f3f1c150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f220da640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f1c06db60 0x7f3f1c0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f220da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c15aa00 0x7f3f1c0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f220da640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c15dbf0 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f220da640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c15c330 con 0x7f3f1c06db60 2023-10-08T05:37:15.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f220da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0c2410 con 0x7f3f1c15aa00 2023-10-08T05:37:15.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f1c06c3a0 0x7f3f1c150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f1c06c3a0 0x7f3f1c150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37068/0 (socket says 172.21.15.70:37068) 2023-10-08T05:37:15.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f1b7fe640 1 -- 172.21.15.70:0/3505098666 learned_addr learned my addr 172.21.15.70:0/3505098666 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:15.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f1bfff640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c15aa00 0x7f3f1c0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f13fff640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f1c06db60 0x7f3f1c0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.710+0000 7f3f1b7fe640 1 -- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c15aa00 msgr2=0x7f3f1c0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.711+0000 7f3f1b7fe640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c15aa00 0x7f3f1c0cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.711+0000 7f3f1b7fe640 1 -- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f1c06db60 msgr2=0x7f3f1c0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.711+0000 7f3f1b7fe640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f1c06db60 0x7f3f1c0cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.711+0000 7f3f1b7fe640 1 -- 172.21.15.70:0/3505098666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04009400 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.711+0000 7f3f1b7fe640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f1c06c3a0 0x7f3f1c150720 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009fd0 tx=0x7f3f0c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:15.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.711+0000 7f3f18ff9640 1 -- 172.21.15.70:0/3505098666 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00c940 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.711+0000 7f3f18ff9640 1 -- 172.21.15.70:0/3505098666 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c00caa0 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.711+0000 7f3f18ff9640 1 -- 172.21.15.70:0/3505098666 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00ce30 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.711+0000 7f3f220da640 1 -- 172.21.15.70:0/3505098666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c0c26f0 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.711+0000 7f3f220da640 1 -- 172.21.15.70:0/3505098666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c15b5d0 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.712+0000 7f3f220da640 1 -- 172.21.15.70:0/3505098666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c04e160 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.713+0000 7f3f18ff9640 1 -- 172.21.15.70:0/3505098666 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3f0c013470 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.714+0000 7f3f18ff9640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3efc06ae70 0x7f3efc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.714+0000 7f3f18ff9640 1 -- 172.21.15.70:0/3505098666 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3f0c089c00 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.715+0000 7f3f13fff640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3efc06ae70 0x7f3efc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.716+0000 7f3f13fff640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3efc06ae70 0x7f3efc06d330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f04036000 tx=0x7f3f0403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:15.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.719+0000 7f3f18ff9640 1 -- 172.21.15.70:0/3505098666 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f0c054850 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.906+0000 7f3f220da640 1 -- 172.21.15.70:0/3505098666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f1c0c21f0 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.907+0000 7f3f18ff9640 1 -- 172.21.15.70:0/3505098666 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f0c0170a0 con 0x7f3f1c06c3a0 2023-10-08T05:37:15.909 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:37:15.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 -- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3efc06ae70 msgr2=0x7f3efc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3efc06ae70 0x7f3efc06d330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f04036000 tx=0x7f3f0403c040 comp rx=0 tx=0).stop 2023-10-08T05:37:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 -- 172.21.15.70:0/3505098666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f1c06c3a0 msgr2=0x7f3f1c150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f1c06c3a0 0x7f3f1c150720 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009fd0 tx=0x7f3f0c00ed40 comp rx=0 tx=0).stop 2023-10-08T05:37:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 -- 172.21.15.70:0/3505098666 shutdown_connections 2023-10-08T05:37:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3efc06ae70 0x7f3efc06d330 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f1c15aa00 0x7f3f1c0cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f1c06db60 0x7f3f1c0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 --2- 172.21.15.70:0/3505098666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f1c06c3a0 0x7f3f1c150720 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 -- 172.21.15.70:0/3505098666 >> 172.21.15.70:0/3505098666 conn(0x7f3f1c06a5d0 msgr2=0x7f3f1c06d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 -- 172.21.15.70:0/3505098666 shutdown_connections 2023-10-08T05:37:15.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:15.910+0000 7f3f220da640 1 -- 172.21.15.70:0/3505098666 wait complete. 2023-10-08T05:37:15.931 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:37:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.138+0000 7feb0830f640 1 Processor -- start 2023-10-08T05:37:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.139+0000 7feb0830f640 1 -- start start 2023-10-08T05:37:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.140+0000 7feb0830f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb000afbe0 0x7feb000b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.140+0000 7feb0830f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb0006dfc0 0x7feb00070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.140+0000 7feb0830f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb000b2620 con 0x7feb00071070 2023-10-08T05:37:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.140+0000 7feb0830f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb000b2060 con 0x7feb000afbe0 2023-10-08T05:37:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.140+0000 7feb0830f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb000b2340 con 0x7feb0006dfc0 2023-10-08T05:37:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.141+0000 7feb060ab640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb000afbe0 0x7feb000b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.141+0000 7feb058aa640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb0006dfc0 0x7feb00070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.141+0000 7feb060ab640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb000afbe0 0x7feb000b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:43934/0 (socket says 172.21.15.70:43934) 2023-10-08T05:37:16.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.141+0000 7feb060ab640 1 -- 172.21.15.70:0/822107160 learned_addr learned my addr 172.21.15.70:0/822107160 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:16.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.141+0000 7feb068ac640 1 --1- 172.21.15.70:0/822107160 >> v1:172.21.15.181:6789/0 conn(0x7feb00071070 0x7feb000cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:53516/0 (socket says 172.21.15.70:53516) 2023-10-08T05:37:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.142+0000 7feb060ab640 1 -- 172.21.15.70:0/822107160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb0006dfc0 msgr2=0x7feb00070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.142+0000 7feb060ab640 1 --2- 172.21.15.70:0/822107160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb0006dfc0 0x7feb00070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.142+0000 7feb060ab640 1 -- 172.21.15.70:0/822107160 >> v1:172.21.15.181:6789/0 conn(0x7feb00071070 legacy=0x7feb000cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.142+0000 7feb050a9640 1 -- 172.21.15.70:0/822107160 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390338093 0 0) 0x7feb000b2620 con 0x7feb00071070 2023-10-08T05:37:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.142+0000 7feb060ab640 1 -- 172.21.15.70:0/822107160 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb00070350 con 0x7feb000afbe0 2023-10-08T05:37:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.144+0000 7feb060ab640 1 --2- 172.21.15.70:0/822107160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb000afbe0 0x7feb000b15b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009e50 tx=0x7feaf002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd024a44cead9603 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.144+0000 7feb050a9640 1 -- 172.21.15.70:0/822107160 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf002cb00 con 0x7feb000afbe0 2023-10-08T05:37:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.144+0000 7feb050a9640 1 -- 172.21.15.70:0/822107160 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf002cc60 con 0x7feb000afbe0 2023-10-08T05:37:16.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.144+0000 7feb050a9640 1 -- 172.21.15.70:0/822107160 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00343f0 con 0x7feb000afbe0 2023-10-08T05:37:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.145+0000 7feb0830f640 1 -- 172.21.15.70:0/822107160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb000afbe0 msgr2=0x7feb000b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.145+0000 7feb0830f640 1 --2- 172.21.15.70:0/822107160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb000afbe0 0x7feb000b15b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009e50 tx=0x7feaf002bf60 comp rx=0 tx=0).stop 2023-10-08T05:37:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.145+0000 7feb0830f640 1 -- 172.21.15.70:0/822107160 shutdown_connections 2023-10-08T05:37:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.145+0000 7feb0830f640 1 --2- 172.21.15.70:0/822107160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb0006dfc0 0x7feb00070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.145+0000 7feb0830f640 1 --2- 172.21.15.70:0/822107160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb000afbe0 0x7feb000b15b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.145+0000 7feb0830f640 1 -- 172.21.15.70:0/822107160 >> 172.21.15.70:0/822107160 conn(0x7feb0006a5d0 msgr2=0x7feb000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.145+0000 7feb0830f640 1 -- 172.21.15.70:0/822107160 shutdown_connections 2023-10-08T05:37:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.145+0000 7feb0830f640 1 -- 172.21.15.70:0/822107160 wait complete. 2023-10-08T05:37:16.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.147+0000 7feb0830f640 1 Processor -- start 2023-10-08T05:37:16.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.147+0000 7feb0830f640 1 -- start start 2023-10-08T05:37:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.148+0000 7feb0830f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb0006dfc0 0x7feb001edf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.148+0000 7feb0830f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb00071070 0x7feb00156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.148+0000 7feb0830f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb000afbe0 0x7feb0014fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.148+0000 7feb0830f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb000ca240 con 0x7feb0006dfc0 2023-10-08T05:37:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.148+0000 7feb0830f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb000c95c0 con 0x7feb00071070 2023-10-08T05:37:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.148+0000 7feb0830f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb000c98a0 con 0x7feb000afbe0 2023-10-08T05:37:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.148+0000 7feb060ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb0006dfc0 0x7feb001edf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.148+0000 7feb060ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb0006dfc0 0x7feb001edf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37078/0 (socket says 172.21.15.70:37078) 2023-10-08T05:37:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.148+0000 7feb060ab640 1 -- 172.21.15.70:0/4245600513 learned_addr learned my addr 172.21.15.70:0/4245600513 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feb058aa640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb00071070 0x7feb00156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feb060ab640 1 -- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb000afbe0 msgr2=0x7feb0014fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feb060ab640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb000afbe0 0x7feb0014fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feb060ab640 1 -- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb00071070 msgr2=0x7feb00156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feb060ab640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb00071070 0x7feb00156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feb060ab640 1 -- 172.21.15.70:0/4245600513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf0009b30 con 0x7feb0006dfc0 2023-10-08T05:37:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feb060ab640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb0006dfc0 0x7feb001edf10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feaf0000f80 tx=0x7feaf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feafeffd640 1 -- 172.21.15.70:0/4245600513 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf0009510 con 0x7feb0006dfc0 2023-10-08T05:37:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feafeffd640 1 -- 172.21.15.70:0/4245600513 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf003e450 con 0x7feb0006dfc0 2023-10-08T05:37:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feafeffd640 1 -- 172.21.15.70:0/4245600513 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf003e800 con 0x7feb0006dfc0 2023-10-08T05:37:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.149+0000 7feb0830f640 1 -- 172.21.15.70:0/4245600513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb000c9b80 con 0x7feb0006dfc0 2023-10-08T05:37:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.150+0000 7feb0830f640 1 -- 172.21.15.70:0/4245600513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb001ee450 con 0x7feb0006dfc0 2023-10-08T05:37:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.151+0000 7feb0830f640 1 -- 172.21.15.70:0/4245600513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb001ee5e0 con 0x7feb0006dfc0 2023-10-08T05:37:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.152+0000 7feafeffd640 1 -- 172.21.15.70:0/4245600513 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feaf003e960 con 0x7feb0006dfc0 2023-10-08T05:37:16.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.152+0000 7feafeffd640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feaec06ae70 0x7feaec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.153+0000 7feafeffd640 1 -- 172.21.15.70:0/4245600513 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7feaf00ac790 con 0x7feb0006dfc0 2023-10-08T05:37:16.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.153+0000 7feb058aa640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feaec06ae70 0x7feaec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.154+0000 7feb058aa640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feaec06ae70 0x7feaec06d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb0014c740 tx=0x7feaf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:16.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.157+0000 7feafeffd640 1 -- 172.21.15.70:0/4245600513 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feaf0077510 con 0x7feb0006dfc0 2023-10-08T05:37:16.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.346+0000 7feb0830f640 1 -- 172.21.15.70:0/4245600513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feb0004e160 con 0x7feb0006dfc0 2023-10-08T05:37:16.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.347+0000 7feafeffd640 1 -- 172.21.15.70:0/4245600513 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7feaf007ab30 con 0x7feb0006dfc0 2023-10-08T05:37:16.349 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:16.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 -- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feaec06ae70 msgr2=0x7feaec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feaec06ae70 0x7feaec06d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb0014c740 tx=0x7feaf4008040 comp rx=0 tx=0).stop 2023-10-08T05:37:16.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 -- 172.21.15.70:0/4245600513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb0006dfc0 msgr2=0x7feb001edf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb0006dfc0 0x7feb001edf10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feaf0000f80 tx=0x7feaf0004a40 comp rx=0 tx=0).stop 2023-10-08T05:37:16.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 -- 172.21.15.70:0/4245600513 shutdown_connections 2023-10-08T05:37:16.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feaec06ae70 0x7feaec06d330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb000afbe0 0x7feb0014fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb00071070 0x7feb00156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 --2- 172.21.15.70:0/4245600513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb0006dfc0 0x7feb001edf10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 -- 172.21.15.70:0/4245600513 >> 172.21.15.70:0/4245600513 conn(0x7feb0006a5d0 msgr2=0x7feb0015a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:16.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 -- 172.21.15.70:0/4245600513 shutdown_connections 2023-10-08T05:37:16.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.350+0000 7feb0830f640 1 -- 172.21.15.70:0/4245600513 wait complete. 2023-10-08T05:37:16.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:37:16.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.587+0000 7f7f7afd7640 1 Processor -- start 2023-10-08T05:37:16.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.588+0000 7f7f7afd7640 1 -- start start 2023-10-08T05:37:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.589+0000 7f7f7afd7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f7406dfc0 0x7f7f74070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.589+0000 7f7f7afd7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 0x7f7f740cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.589+0000 7f7f7afd7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f740b1cc0 con 0x7f7f740afbe0 2023-10-08T05:37:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.589+0000 7f7f7afd7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f740b2620 con 0x7f7f7406dfc0 2023-10-08T05:37:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.589+0000 7f7f7afd7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f740b2060 con 0x7f7f74071070 2023-10-08T05:37:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.589+0000 7f7f73fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f7406dfc0 0x7f7f74070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.589+0000 7f7f73fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f7406dfc0 0x7f7f74070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37092/0 (socket says 172.21.15.70:37092) 2023-10-08T05:37:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.589+0000 7f7f73fff640 1 -- 172.21.15.70:0/439583892 learned_addr learned my addr 172.21.15.70:0/439583892 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.590+0000 7f7f79574640 1 --2- 172.21.15.70:0/439583892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 0x7f7f740cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.590+0000 7f7f73fff640 1 -- 172.21.15.70:0/439583892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 msgr2=0x7f7f740cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.590+0000 7f7f73fff640 1 --2- 172.21.15.70:0/439583892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 0x7f7f740cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.590+0000 7f7f73fff640 1 -- 172.21.15.70:0/439583892 >> v1:172.21.15.181:6789/0 conn(0x7f7f740afbe0 legacy=0x7f7f740b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:16.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.590+0000 7f7f73fff640 1 -- 172.21.15.70:0/439583892 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f740b2340 con 0x7f7f7406dfc0 2023-10-08T05:37:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.591+0000 7f7f73fff640 1 --2- 172.21.15.70:0/439583892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 0x7f7f74070b30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009870 tx=0x7f7f6802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4912d0aac038e256 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.591+0000 7f7f737fe640 1 -- 172.21.15.70:0/439583892 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68031640 con 0x7f7f7406dfc0 2023-10-08T05:37:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.591+0000 7f7f737fe640 1 -- 172.21.15.70:0/439583892 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f680317a0 con 0x7f7f7406dfc0 2023-10-08T05:37:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.591+0000 7f7f737fe640 1 -- 172.21.15.70:0/439583892 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68031b30 con 0x7f7f7406dfc0 2023-10-08T05:37:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.592+0000 7f7f7afd7640 1 -- 172.21.15.70:0/439583892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 msgr2=0x7f7f74070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.592+0000 7f7f7afd7640 1 --2- 172.21.15.70:0/439583892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 0x7f7f74070b30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009870 tx=0x7f7f6802b8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.592+0000 7f7f7afd7640 1 -- 172.21.15.70:0/439583892 shutdown_connections 2023-10-08T05:37:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.592+0000 7f7f7afd7640 1 --2- 172.21.15.70:0/439583892 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 0x7f7f740cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.592+0000 7f7f7afd7640 1 --2- 172.21.15.70:0/439583892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 0x7f7f74070b30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.592+0000 7f7f7afd7640 1 -- 172.21.15.70:0/439583892 >> 172.21.15.70:0/439583892 conn(0x7f7f7406a5d0 msgr2=0x7f7f740b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.592+0000 7f7f7afd7640 1 -- 172.21.15.70:0/439583892 shutdown_connections 2023-10-08T05:37:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.592+0000 7f7f7afd7640 1 -- 172.21.15.70:0/439583892 wait complete. 2023-10-08T05:37:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.594+0000 7f7f7afd7640 1 Processor -- start 2023-10-08T05:37:16.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.594+0000 7f7f7afd7640 1 -- start start 2023-10-08T05:37:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.595+0000 7f7f7afd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 0x7f7f74151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.595+0000 7f7f7afd7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 0x7f7f7406c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.595+0000 7f7f7afd7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f740afbe0 0x7f7f7406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.595+0000 7f7f7afd7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7415d700 con 0x7f7f7406dfc0 2023-10-08T05:37:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.595+0000 7f7f7afd7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7415d9e0 con 0x7f7f740afbe0 2023-10-08T05:37:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.595+0000 7f7f7afd7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7415dcc0 con 0x7f7f74071070 2023-10-08T05:37:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f78d73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 0x7f7f74151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f78d73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 0x7f7f74151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37106/0 (socket says 172.21.15.70:37106) 2023-10-08T05:37:16.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f78d73640 1 -- 172.21.15.70:0/3941053509 learned_addr learned my addr 172.21.15.70:0/3941053509 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:16.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f73fff640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 0x7f7f7406c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f79574640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f740afbe0 0x7f7f7406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f78d73640 1 -- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 msgr2=0x7f7f7406c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f78d73640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 0x7f7f7406c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f78d73640 1 -- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f740afbe0 msgr2=0x7f7f7406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f78d73640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f740afbe0 0x7f7f7406caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f78d73640 1 -- 172.21.15.70:0/3941053509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680094d0 con 0x7f7f7406dfc0 2023-10-08T05:37:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f73fff640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 0x7f7f7406c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.596+0000 7f7f78d73640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 0x7f7f74151c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400ba50 tx=0x7f7f6400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.597+0000 7f7f717fa640 1 -- 172.21.15.70:0/3941053509 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400cbe0 con 0x7f7f7406dfc0 2023-10-08T05:37:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.597+0000 7f7f717fa640 1 -- 172.21.15.70:0/3941053509 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6400cd80 con 0x7f7f7406dfc0 2023-10-08T05:37:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.597+0000 7f7f717fa640 1 -- 172.21.15.70:0/3941053509 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64014680 con 0x7f7f7406dfc0 2023-10-08T05:37:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.597+0000 7f7f7afd7640 1 -- 172.21.15.70:0/3941053509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7415dfa0 con 0x7f7f7406dfc0 2023-10-08T05:37:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.597+0000 7f7f7afd7640 1 -- 172.21.15.70:0/3941053509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7415d3c0 con 0x7f7f7406dfc0 2023-10-08T05:37:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.598+0000 7f7f7afd7640 1 -- 172.21.15.70:0/3941053509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f74070610 con 0x7f7f7406dfc0 2023-10-08T05:37:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.599+0000 7f7f717fa640 1 -- 172.21.15.70:0/3941053509 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7f640147e0 con 0x7f7f7406dfc0 2023-10-08T05:37:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.600+0000 7f7f717fa640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f6006ae70 0x7f7f6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.600+0000 7f7f717fa640 1 -- 172.21.15.70:0/3941053509 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7f64089800 con 0x7f7f7406dfc0 2023-10-08T05:37:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.600+0000 7f7f73fff640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f6006ae70 0x7f7f6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.602+0000 7f7f73fff640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f6006ae70 0x7f7f6006d330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7f68036000 tx=0x7f7f6803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.604+0000 7f7f717fa640 1 -- 172.21.15.70:0/3941053509 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f74070610 con 0x7f7f7406dfc0 2023-10-08T05:37:16.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.807+0000 7f7f7afd7640 1 -- 172.21.15.70:0/3941053509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7f74070610 con 0x7f7f7406dfc0 2023-10-08T05:37:16.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.807+0000 7f7f717fa640 1 -- 172.21.15.70:0/3941053509 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7f74070610 con 0x7f7f7406dfc0 2023-10-08T05:37:16.809 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-10-08T05:37:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 -- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f6006ae70 msgr2=0x7f7f6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f6006ae70 0x7f7f6006d330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7f68036000 tx=0x7f7f6803c040 comp rx=0 tx=0).stop 2023-10-08T05:37:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 -- 172.21.15.70:0/3941053509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 msgr2=0x7f7f74151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 0x7f7f74151c10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400ba50 tx=0x7f7f6400bf10 comp rx=0 tx=0).stop 2023-10-08T05:37:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 -- 172.21.15.70:0/3941053509 shutdown_connections 2023-10-08T05:37:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7f6006ae70 0x7f7f6006d330 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f740afbe0 0x7f7f7406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f74071070 0x7f7f7406c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 --2- 172.21.15.70:0/3941053509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f7406dfc0 0x7f7f74151c10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 -- 172.21.15.70:0/3941053509 >> 172.21.15.70:0/3941053509 conn(0x7f7f7406a5d0 msgr2=0x7f7f740ca030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 -- 172.21.15.70:0/3941053509 shutdown_connections 2023-10-08T05:37:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:16.810+0000 7f7f7afd7640 1 -- 172.21.15.70:0/3941053509 wait complete. 2023-10-08T05:37:16.831 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T05:37:16.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:16.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-08T05:37:16.955 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:37:16.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:16.955 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T05:37:17.010 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:37:17.010 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:17.069 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:37:17.069 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:17.124 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:37:17.124 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:17.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:17.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.396+0000 7f3e6a84f640 1 Processor -- start 2023-10-08T05:37:17.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.396+0000 7f3e6a84f640 1 -- start start 2023-10-08T05:37:17.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.398+0000 7f3e6a84f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3e64159ac0 0x7f3e6406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.398+0000 7f3e6a84f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6406dee0 con 0x7f3e640ad790 2023-10-08T05:37:17.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.398+0000 7f3e6a84f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6415d850 con 0x7f3e640709e0 2023-10-08T05:37:17.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.398+0000 7f3e6a84f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6415db30 con 0x7f3e64159ac0 2023-10-08T05:37:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.398+0000 7f3e63fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3e64159ac0 0x7f3e6406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.398+0000 7f3e5bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3e640ad790 0x7f3e640702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50038/0 (socket says 172.21.15.70:50038) 2023-10-08T05:37:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.398+0000 7f3e5bfff640 1 -- 172.21.15.70:0/3783442656 learned_addr learned my addr 172.21.15.70:0/3783442656 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.399+0000 7f3e63fff640 1 -- 172.21.15.70:0/3783442656 >> v1:172.21.15.181:6789/0 conn(0x7f3e640709e0 legacy=0x7f3e640c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:17.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.399+0000 7f3e63fff640 1 -- 172.21.15.70:0/3783442656 >> v1:172.21.15.70:6789/0 conn(0x7f3e640ad790 legacy=0x7f3e640702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:17.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.399+0000 7f3e63fff640 1 -- 172.21.15.70:0/3783442656 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6415de10 con 0x7f3e64159ac0 2023-10-08T05:37:17.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.399+0000 7f3e637fe640 1 -- 172.21.15.70:0/3783442656 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530935667 0 0) 0x7f3e6406dee0 con 0x7f3e640ad790 2023-10-08T05:37:17.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.400+0000 7f3e63fff640 1 --2- 172.21.15.70:0/3783442656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 0x7f3e6406d5e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e540097a0 tx=0x7f3e5402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18daa08b0f59e453 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:17.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.400+0000 7f3e637fe640 1 -- 172.21.15.70:0/3783442656 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5402f8a0 con 0x7f3e64159ac0 2023-10-08T05:37:17.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.400+0000 7f3e637fe640 1 -- 172.21.15.70:0/3783442656 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5402fa00 con 0x7f3e64159ac0 2023-10-08T05:37:17.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.400+0000 7f3e637fe640 1 -- 172.21.15.70:0/3783442656 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5402fd90 con 0x7f3e64159ac0 2023-10-08T05:37:17.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.400+0000 7f3e6a84f640 1 -- 172.21.15.70:0/3783442656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 msgr2=0x7f3e6406d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:17.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.400+0000 7f3e6a84f640 1 --2- 172.21.15.70:0/3783442656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 0x7f3e6406d5e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3e540097a0 tx=0x7f3e5402dae0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.401+0000 7f3e6a84f640 1 -- 172.21.15.70:0/3783442656 shutdown_connections 2023-10-08T05:37:17.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.401+0000 7f3e6a84f640 1 --2- 172.21.15.70:0/3783442656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 0x7f3e6406d5e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.401+0000 7f3e6a84f640 1 -- 172.21.15.70:0/3783442656 >> 172.21.15.70:0/3783442656 conn(0x7f3e6406a380 msgr2=0x7f3e640b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:17.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.401+0000 7f3e6a84f640 1 -- 172.21.15.70:0/3783442656 shutdown_connections 2023-10-08T05:37:17.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.401+0000 7f3e6a84f640 1 -- 172.21.15.70:0/3783442656 wait complete. 2023-10-08T05:37:17.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.402+0000 7f3e6a84f640 1 Processor -- start 2023-10-08T05:37:17.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.403+0000 7f3e6a84f640 1 -- start start 2023-10-08T05:37:17.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e6a84f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e640709e0 0x7f3e641587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e6a84f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e640ad790 0x7f3e640b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e6a84f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 0x7f3e640ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e6a84f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6415c7b0 con 0x7f3e64159ac0 2023-10-08T05:37:17.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e6a84f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6415ca90 con 0x7f3e640ad790 2023-10-08T05:37:17.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e6a84f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6415cd70 con 0x7f3e640709e0 2023-10-08T05:37:17.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e68dec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 0x7f3e640ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:17.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e68dec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 0x7f3e640ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37114/0 (socket says 172.21.15.70:37114) 2023-10-08T05:37:17.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e68dec640 1 -- 172.21.15.70:0/1678588594 learned_addr learned my addr 172.21.15.70:0/1678588594 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:17.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e68dec640 1 -- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e640709e0 msgr2=0x7f3e641587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:17.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e68dec640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e640709e0 0x7f3e641587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e68dec640 1 -- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e640ad790 msgr2=0x7f3e640b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:17.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e68dec640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e640ad790 0x7f3e640b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.404+0000 7f3e68dec640 1 -- 172.21.15.70:0/1678588594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e54009400 con 0x7f3e64159ac0 2023-10-08T05:37:17.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.405+0000 7f3e68dec640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 0x7f3e640ab8c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00b9e0 tx=0x7f3e5c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:17.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.405+0000 7f3e617fa640 1 -- 172.21.15.70:0/1678588594 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00cb20 con 0x7f3e64159ac0 2023-10-08T05:37:17.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.405+0000 7f3e617fa640 1 -- 172.21.15.70:0/1678588594 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c00cc80 con 0x7f3e64159ac0 2023-10-08T05:37:17.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.405+0000 7f3e617fa640 1 -- 172.21.15.70:0/1678588594 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c014680 con 0x7f3e64159ac0 2023-10-08T05:37:17.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.405+0000 7f3e6a84f640 1 -- 172.21.15.70:0/1678588594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e641f2b40 con 0x7f3e64159ac0 2023-10-08T05:37:17.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.405+0000 7f3e6a84f640 1 -- 172.21.15.70:0/1678588594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e64158d10 con 0x7f3e64159ac0 2023-10-08T05:37:17.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.406+0000 7f3e6a84f640 1 -- 172.21.15.70:0/1678588594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e640c4ad0 con 0x7f3e64159ac0 2023-10-08T05:37:17.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.407+0000 7f3e617fa640 1 -- 172.21.15.70:0/1678588594 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3e5c01d3b0 con 0x7f3e64159ac0 2023-10-08T05:37:17.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.408+0000 7f3e617fa640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3e4406ae70 0x7f3e4406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.408+0000 7f3e617fa640 1 -- 172.21.15.70:0/1678588594 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f3e5c0899c0 con 0x7f3e64159ac0 2023-10-08T05:37:17.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.409+0000 7f3e63fff640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3e4406ae70 0x7f3e4406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:17.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.410+0000 7f3e63fff640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3e4406ae70 0x7f3e4406d330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3e54038000 tx=0x7f3e54032000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:17.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.412+0000 7f3e617fa640 1 -- 172.21.15.70:0/1678588594 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e5c054610 con 0x7f3e64159ac0 2023-10-08T05:37:17.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.605+0000 7f3e6a84f640 1 -- 172.21.15.70:0/1678588594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e6415e330 con 0x7f3e64159ac0 2023-10-08T05:37:17.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.607+0000 7f3e617fa640 1 -- 172.21.15.70:0/1678588594 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7f3e5c019090 con 0x7f3e64159ac0 2023-10-08T05:37:17.609 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:17.609 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:17.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 -- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3e4406ae70 msgr2=0x7f3e4406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:17.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3e4406ae70 0x7f3e4406d330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3e54038000 tx=0x7f3e54032000 comp rx=0 tx=0).stop 2023-10-08T05:37:17.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 -- 172.21.15.70:0/1678588594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 msgr2=0x7f3e640ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:17.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 0x7f3e640ab8c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00b9e0 tx=0x7f3e5c00bea0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 -- 172.21.15.70:0/1678588594 shutdown_connections 2023-10-08T05:37:17.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3e4406ae70 0x7f3e4406d330 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e64159ac0 0x7f3e640ab8c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e640ad790 0x7f3e640b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 --2- 172.21.15.70:0/1678588594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e640709e0 0x7f3e641587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 -- 172.21.15.70:0/1678588594 >> 172.21.15.70:0/1678588594 conn(0x7f3e6406a380 msgr2=0x7f3e640cedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:17.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 -- 172.21.15.70:0/1678588594 shutdown_connections 2023-10-08T05:37:17.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.610+0000 7f3e6a84f640 1 -- 172.21.15.70:0/1678588594 wait complete. 2023-10-08T05:37:17.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:37:17.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:17.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.837+0000 7f866912c640 1 Processor -- start 2023-10-08T05:37:17.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.837+0000 7f866912c640 1 -- start start 2023-10-08T05:37:17.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.838+0000 7f866912c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86641432f0 0x7f86640cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.838+0000 7f866912c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86640cb6b0 0x7f86640b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.838+0000 7f866912c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86640cbb30 con 0x7f86640b4070 2023-10-08T05:37:17.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.838+0000 7f866912c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86641436d0 con 0x7f86640cb6b0 2023-10-08T05:37:17.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.838+0000 7f866912c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866415ae60 con 0x7f86641432f0 2023-10-08T05:37:17.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.839+0000 7f865bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86640cb6b0 0x7f86640b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:17.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.839+0000 7f865bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86640cb6b0 0x7f86640b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37130/0 (socket says 172.21.15.70:37130) 2023-10-08T05:37:17.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.839+0000 7f865bfff640 1 -- 172.21.15.70:0/3819256559 learned_addr learned my addr 172.21.15.70:0/3819256559 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:17.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.839+0000 7f8662d9d640 1 --2- 172.21.15.70:0/3819256559 >> v2:172.21.15.181:3300/0 conn(0x7f86641432f0 0x7f86640cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:17.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.839+0000 7f865bfff640 1 -- 172.21.15.70:0/3819256559 >> v2:172.21.15.181:3300/0 conn(0x7f86641432f0 msgr2=0x7f86640cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:17.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.839+0000 7f865bfff640 1 --2- 172.21.15.70:0/3819256559 >> v2:172.21.15.181:3300/0 conn(0x7f86641432f0 0x7f86640cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.839+0000 7f865bfff640 1 -- 172.21.15.70:0/3819256559 >> v1:172.21.15.181:6789/0 conn(0x7f86640b4070 legacy=0x7f866406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:17.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.839+0000 7f865bfff640 1 -- 172.21.15.70:0/3819256559 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866415d440 con 0x7f86640cb6b0 2023-10-08T05:37:17.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.840+0000 7f865bfff640 1 --2- 172.21.15.70:0/3819256559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86640cb6b0 0x7f86640b3b30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f86640cc1a0 tx=0x7f864c02da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528705ba72f772a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:17.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.841+0000 7f866259c640 1 -- 172.21.15.70:0/3819256559 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c02f850 con 0x7f86640cb6b0 2023-10-08T05:37:17.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.841+0000 7f866259c640 1 -- 172.21.15.70:0/3819256559 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c02f9b0 con 0x7f86640cb6b0 2023-10-08T05:37:17.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.841+0000 7f866259c640 1 -- 172.21.15.70:0/3819256559 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c02fd40 con 0x7f86640cb6b0 2023-10-08T05:37:17.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.841+0000 7f866912c640 1 -- 172.21.15.70:0/3819256559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86640cb6b0 msgr2=0x7f86640b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:17.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.841+0000 7f866912c640 1 --2- 172.21.15.70:0/3819256559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86640cb6b0 0x7f86640b3b30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f86640cc1a0 tx=0x7f864c02da90 comp rx=0 tx=0).stop 2023-10-08T05:37:17.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.842+0000 7f866912c640 1 -- 172.21.15.70:0/3819256559 shutdown_connections 2023-10-08T05:37:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.842+0000 7f866912c640 1 --2- 172.21.15.70:0/3819256559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86640cb6b0 0x7f86640b3b30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.842+0000 7f866912c640 1 --2- 172.21.15.70:0/3819256559 >> v2:172.21.15.181:3300/0 conn(0x7f86641432f0 0x7f86640cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.842+0000 7f866912c640 1 -- 172.21.15.70:0/3819256559 >> 172.21.15.70:0/3819256559 conn(0x7f8664069d50 msgr2=0x7f866406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.842+0000 7f866912c640 1 -- 172.21.15.70:0/3819256559 shutdown_connections 2023-10-08T05:37:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.842+0000 7f866912c640 1 -- 172.21.15.70:0/3819256559 wait complete. 2023-10-08T05:37:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.843+0000 7f866912c640 1 Processor -- start 2023-10-08T05:37:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.843+0000 7f866912c640 1 -- start start 2023-10-08T05:37:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.844+0000 7f866912c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86640b4070 0x7f866413a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.844+0000 7f866912c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86640cb6b0 0x7f8664159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.844+0000 7f866912c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86641432f0 0x7f866413c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.844+0000 7f866912c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866413b7b0 con 0x7f86641432f0 2023-10-08T05:37:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f866912c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866413bbd0 con 0x7f86640cb6b0 2023-10-08T05:37:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f866912c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866415d0d0 con 0x7f86640b4070 2023-10-08T05:37:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f866359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86641432f0 0x7f866413c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f866359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86641432f0 0x7f866413c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37138/0 (socket says 172.21.15.70:37138) 2023-10-08T05:37:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f866359e640 1 -- 172.21.15.70:0/1019215277 learned_addr learned my addr 172.21.15.70:0/1019215277 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f865bfff640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86640cb6b0 0x7f8664159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f866359e640 1 -- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86640b4070 msgr2=0x7f866413a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f8662d9d640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86640b4070 0x7f866413a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f866359e640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86640b4070 0x7f866413a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f866359e640 1 -- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86640cb6b0 msgr2=0x7f8664159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f866359e640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86640cb6b0 0x7f8664159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f866359e640 1 -- 172.21.15.70:0/1019215277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c0094d0 con 0x7f86641432f0 2023-10-08T05:37:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.845+0000 7f865bfff640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86640cb6b0 0x7f8664159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.846+0000 7f866359e640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86641432f0 0x7f866413c0a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f865c00c7b0 tx=0x7f865c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.846+0000 7f865b7fe640 1 -- 172.21.15.70:0/1019215277 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c004b60 con 0x7f86641432f0 2023-10-08T05:37:17.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.846+0000 7f865b7fe640 1 -- 172.21.15.70:0/1019215277 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c004cc0 con 0x7f86641432f0 2023-10-08T05:37:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.846+0000 7f866912c640 1 -- 172.21.15.70:0/1019215277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866415afa0 con 0x7f86641432f0 2023-10-08T05:37:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.846+0000 7f865b7fe640 1 -- 172.21.15.70:0/1019215277 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c00aa90 con 0x7f86641432f0 2023-10-08T05:37:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.846+0000 7f866912c640 1 -- 172.21.15.70:0/1019215277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866415b240 con 0x7f86641432f0 2023-10-08T05:37:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.847+0000 7f866912c640 1 -- 172.21.15.70:0/1019215277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866413b8f0 con 0x7f86641432f0 2023-10-08T05:37:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.848+0000 7f865b7fe640 1 -- 172.21.15.70:0/1019215277 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f865c00abf0 con 0x7f86641432f0 2023-10-08T05:37:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.849+0000 7f865b7fe640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f864406ae70 0x7f864406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.849+0000 7f865b7fe640 1 -- 172.21.15.70:0/1019215277 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f865c089c40 con 0x7f86641432f0 2023-10-08T05:37:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.850+0000 7f8662d9d640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f864406ae70 0x7f864406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:17.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.851+0000 7f8662d9d640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f864406ae70 0x7f864406d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8654006fd0 tx=0x7f8654008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:17.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:17.853+0000 7f865b7fe640 1 -- 172.21.15.70:0/1019215277 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f866413b8f0 con 0x7f86641432f0 2023-10-08T05:37:18.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.042+0000 7f866912c640 1 -- 172.21.15.70:0/1019215277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f866413b8f0 con 0x7f86641432f0 2023-10-08T05:37:18.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.043+0000 7f865b7fe640 1 -- 172.21.15.70:0/1019215277 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f866413b8f0 con 0x7f86641432f0 2023-10-08T05:37:18.044 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:18.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.045+0000 7f866912c640 1 -- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f864406ae70 msgr2=0x7f864406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.045+0000 7f866912c640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f864406ae70 0x7f864406d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8654006fd0 tx=0x7f8654008040 comp rx=0 tx=0).stop 2023-10-08T05:37:18.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.045+0000 7f866912c640 1 -- 172.21.15.70:0/1019215277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86641432f0 msgr2=0x7f866413c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.045+0000 7f866912c640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86641432f0 0x7f866413c0a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f865c00c7b0 tx=0x7f865c00cc70 comp rx=0 tx=0).stop 2023-10-08T05:37:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.046+0000 7f866912c640 1 -- 172.21.15.70:0/1019215277 shutdown_connections 2023-10-08T05:37:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.046+0000 7f866912c640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f864406ae70 0x7f864406d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.046+0000 7f866912c640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86641432f0 0x7f866413c0a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.046+0000 7f866912c640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86640cb6b0 0x7f8664159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.046+0000 7f866912c640 1 --2- 172.21.15.70:0/1019215277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86640b4070 0x7f866413a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.046+0000 7f866912c640 1 -- 172.21.15.70:0/1019215277 >> 172.21.15.70:0/1019215277 conn(0x7f8664069d50 msgr2=0x7f86641425b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.046+0000 7f866912c640 1 -- 172.21.15.70:0/1019215277 shutdown_connections 2023-10-08T05:37:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.046+0000 7f866912c640 1 -- 172.21.15.70:0/1019215277 wait complete. 2023-10-08T05:37:18.066 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:37:18.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.280+0000 7f5471ab4640 1 Processor -- start 2023-10-08T05:37:18.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.281+0000 7f5471ab4640 1 -- start start 2023-10-08T05:37:18.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.282+0000 7f5471ab4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f546c0cb130 0x7f546c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.282+0000 7f5471ab4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f546c0b1cc0 con 0x7f546c0afbe0 2023-10-08T05:37:18.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.282+0000 7f5471ab4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f546c0cbad0 con 0x7f546c06dfc0 2023-10-08T05:37:18.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.282+0000 7f5471ab4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0b2620 con 0x7f546c0cb130 2023-10-08T05:37:18.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.282+0000 7f546b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f546c0afbe0 0x7f546c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50046/0 (socket says 172.21.15.70:50046) 2023-10-08T05:37:18.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.282+0000 7f546b7fe640 1 -- 172.21.15.70:0/2478338361 learned_addr learned my addr 172.21.15.70:0/2478338361 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:18.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.283+0000 7f546bfff640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f546c0cb130 0x7f546c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.283+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190643543 0 0) 0x7f546c0b1cc0 con 0x7f546c0afbe0 2023-10-08T05:37:18.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.283+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5458003520 con 0x7f546c0afbe0 2023-10-08T05:37:18.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.283+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 419598347 0 0) 0x7f5458003520 con 0x7f546c0afbe0 2023-10-08T05:37:18.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.283+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f546c0cb130 msgr2=0x7f546c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:18.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.283+0000 7f546a7fc640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f546c0cb130 0x7f546c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.283+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 >> v1:172.21.15.181:6789/0 conn(0x7f546c06dfc0 legacy=0x7f546c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.283+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c0b2060 con 0x7f546c0afbe0 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.283+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f545c001670 con 0x7f546c0afbe0 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 >> v1:172.21.15.70:6789/0 conn(0x7f546c0afbe0 legacy=0x7f546c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546a7fc640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54580039a0 0x7f5458003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546a7fc640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54580047c0 0x7f5458004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546a7fc640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f546c0cb130 0x7f5458005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0b2620 con 0x7f54580047c0 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545800ba60 con 0x7f54580039a0 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545800bc20 con 0x7f546c0cb130 2023-10-08T05:37:18.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546affd640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54580047c0 0x7f5458004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546b7fe640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54580039a0 0x7f5458003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546bfff640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f546c0cb130 0x7f5458005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546affd640 1 -- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f546c0cb130 msgr2=0x7f5458005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:18.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546affd640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f546c0cb130 0x7f5458005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546affd640 1 -- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54580039a0 msgr2=0x7f5458003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:18.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546affd640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54580039a0 0x7f5458003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.284+0000 7f546affd640 1 -- 172.21.15.70:0/2478338361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5458003520 con 0x7f54580047c0 2023-10-08T05:37:18.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.286+0000 7f546affd640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54580047c0 0x7f5458004c40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f546c06e410 tx=0x7f54580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a01072deb0f7a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:18.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.286+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454003b70 con 0x7f54580047c0 2023-10-08T05:37:18.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.286+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5454003cd0 con 0x7f54580047c0 2023-10-08T05:37:18.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.286+0000 7f546a7fc640 1 -- 172.21.15.70:0/2478338361 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454005310 con 0x7f54580047c0 2023-10-08T05:37:18.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.286+0000 7f5471ab4640 1 -- 172.21.15.70:0/2478338361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54580047c0 msgr2=0x7f5458004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.286+0000 7f5471ab4640 1 --2- 172.21.15.70:0/2478338361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54580047c0 0x7f5458004c40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f546c06e410 tx=0x7f54580057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.286+0000 7f546bfff640 1 -- 172.21.15.70:0/2478338361 reap_dead start 2023-10-08T05:37:18.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.287+0000 7f5471ab4640 1 -- 172.21.15.70:0/2478338361 shutdown_connections 2023-10-08T05:37:18.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.287+0000 7f5471ab4640 1 -- 172.21.15.70:0/2478338361 >> 172.21.15.70:0/2478338361 conn(0x7f546c06a5d0 msgr2=0x7f546c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:18.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.287+0000 7f5471ab4640 1 -- 172.21.15.70:0/2478338361 shutdown_connections 2023-10-08T05:37:18.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.287+0000 7f5471ab4640 1 -- 172.21.15.70:0/2478338361 wait complete. 2023-10-08T05:37:18.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.288+0000 7f5471ab4640 1 Processor -- start 2023-10-08T05:37:18.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.289+0000 7f5471ab4640 1 -- start start 2023-10-08T05:37:18.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.289+0000 7f5471ab4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54580047c0 0x7f546c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f5471ab4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c0aeb70 0x7f546c15da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f5471ab4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c0aeff0 0x7f546c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f5471ab4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0c2610 con 0x7f546c0aeb70 2023-10-08T05:37:18.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f5471ab4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0c28f0 con 0x7f546c0aeff0 2023-10-08T05:37:18.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f5471ab4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0c2bd0 con 0x7f54580047c0 2023-10-08T05:37:18.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c0aeb70 0x7f546c15da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c0aeb70 0x7f546c15da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37164/0 (socket says 172.21.15.70:37164) 2023-10-08T05:37:18.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546affd640 1 -- 172.21.15.70:0/1036114302 learned_addr learned my addr 172.21.15.70:0/1036114302 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:18.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546bfff640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c0aeff0 0x7f546c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546b7fe640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54580047c0 0x7f546c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546affd640 1 -- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54580047c0 msgr2=0x7f546c159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546affd640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54580047c0 0x7f546c159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546affd640 1 -- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c0aeff0 msgr2=0x7f546c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546affd640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c0aeff0 0x7f546c15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546affd640 1 -- 172.21.15.70:0/1036114302 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c0c2eb0 con 0x7f546c0aeb70 2023-10-08T05:37:18.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.290+0000 7f546bfff640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c0aeff0 0x7f546c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:18.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.291+0000 7f546affd640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c0aeb70 0x7f546c15da40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f545402bf30 tx=0x7f54540014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:18.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.291+0000 7f5463fff640 1 -- 172.21.15.70:0/1036114302 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545403e070 con 0x7f546c0aeb70 2023-10-08T05:37:18.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.291+0000 7f5463fff640 1 -- 172.21.15.70:0/1036114302 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5454006d60 con 0x7f546c0aeb70 2023-10-08T05:37:18.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.291+0000 7f5463fff640 1 -- 172.21.15.70:0/1036114302 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454005680 con 0x7f546c0aeb70 2023-10-08T05:37:18.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.291+0000 7f5471ab4640 1 -- 172.21.15.70:0/1036114302 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546c0c3190 con 0x7f546c0aeb70 2023-10-08T05:37:18.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.291+0000 7f5471ab4640 1 -- 172.21.15.70:0/1036114302 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546c0c2610 con 0x7f546c0aeb70 2023-10-08T05:37:18.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.292+0000 7f5471ab4640 1 -- 172.21.15.70:0/1036114302 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546c0c3350 con 0x7f546c0aeb70 2023-10-08T05:37:18.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.293+0000 7f5463fff640 1 -- 172.21.15.70:0/1036114302 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5454007030 con 0x7f546c0aeb70 2023-10-08T05:37:18.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.294+0000 7f5463fff640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f544c06ae70 0x7f544c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.294+0000 7f5463fff640 1 -- 172.21.15.70:0/1036114302 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f54540ab1e0 con 0x7f546c0aeb70 2023-10-08T05:37:18.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.295+0000 7f546b7fe640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f544c06ae70 0x7f544c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.296+0000 7f546b7fe640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f544c06ae70 0x7f544c06d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f545c001c90 tx=0x7f545c000ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:18.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.298+0000 7f5463fff640 1 -- 172.21.15.70:0/1036114302 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5454075ff0 con 0x7f546c0aeb70 2023-10-08T05:37:18.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.482+0000 7f5471ab4640 1 -- 172.21.15.70:0/1036114302 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f546c04e160 con 0x7f546c0aeb70 2023-10-08T05:37:18.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.483+0000 7f5463fff640 1 -- 172.21.15.70:0/1036114302 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5454079610 con 0x7f546c0aeb70 2023-10-08T05:37:18.485 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.485+0000 7f5471ab4640 1 -- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f544c06ae70 msgr2=0x7f544c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.485+0000 7f5471ab4640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f544c06ae70 0x7f544c06d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f545c001c90 tx=0x7f545c000ff0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.485+0000 7f5471ab4640 1 -- 172.21.15.70:0/1036114302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c0aeb70 msgr2=0x7f546c15da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.485+0000 7f5471ab4640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c0aeb70 0x7f546c15da40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f545402bf30 tx=0x7f54540014a0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.486+0000 7f5471ab4640 1 -- 172.21.15.70:0/1036114302 shutdown_connections 2023-10-08T05:37:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.486+0000 7f5471ab4640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f544c06ae70 0x7f544c06d330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.486+0000 7f5471ab4640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f546c0aeff0 0x7f546c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.486+0000 7f5471ab4640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f546c0aeb70 0x7f546c15da40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.486+0000 7f5471ab4640 1 --2- 172.21.15.70:0/1036114302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54580047c0 0x7f546c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.486+0000 7f5471ab4640 1 -- 172.21.15.70:0/1036114302 >> 172.21.15.70:0/1036114302 conn(0x7f546c06a5d0 msgr2=0x7f546c0c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.486+0000 7f5471ab4640 1 -- 172.21.15.70:0/1036114302 shutdown_connections 2023-10-08T05:37:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.486+0000 7f5471ab4640 1 -- 172.21.15.70:0/1036114302 wait complete. 2023-10-08T05:37:18.507 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:37:18.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.717+0000 7fd5fa5a4640 1 Processor -- start 2023-10-08T05:37:18.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.717+0000 7fd5fa5a4640 1 -- start start 2023-10-08T05:37:18.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.718+0000 7fd5fa5a4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f40afbe0 0x7fd5f40b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.718+0000 7fd5fa5a4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f40b2620 con 0x7fd5f406dfc0 2023-10-08T05:37:18.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.718+0000 7fd5fa5a4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f40b2060 con 0x7fd5f40cb130 2023-10-08T05:37:18.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.718+0000 7fd5fa5a4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f40b2340 con 0x7fd5f40afbe0 2023-10-08T05:37:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.719+0000 7fd5ebfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd5f406dfc0 0x7fd5f4070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50056/0 (socket says 172.21.15.70:50056) 2023-10-08T05:37:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.719+0000 7fd5ebfff640 1 -- 172.21.15.70:0/2775935445 learned_addr learned my addr 172.21.15.70:0/2775935445 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.719+0000 7fd5f3fff640 1 --2- 172.21.15.70:0/2775935445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f40afbe0 0x7fd5f40b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.719+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384311237 0 0) 0x7fd5f40b2620 con 0x7fd5f406dfc0 2023-10-08T05:37:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.719+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e0003520 con 0x7fd5f406dfc0 2023-10-08T05:37:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.719+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2184131815 0 0) 0x7fd5e0003520 con 0x7fd5f406dfc0 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f40afbe0 msgr2=0x7fd5f40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 --2- 172.21.15.70:0/2775935445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f40afbe0 0x7fd5f40b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 >> v1:172.21.15.181:6789/0 conn(0x7fd5f40cb130 legacy=0x7fd5f406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f4070350 con 0x7fd5f406dfc0 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd5dc001670 con 0x7fd5f406dfc0 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 >> v1:172.21.15.70:6789/0 conn(0x7fd5f406dfc0 legacy=0x7fd5f4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 --2- 172.21.15.70:0/2775935445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e00039a0 0x7fd5e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 --2- 172.21.15.70:0/2775935445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 0x7fd5e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 --2- 172.21.15.70:0/2775935445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f40afbe0 0x7fd5e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f40b2340 con 0x7fd5e00047c0 2023-10-08T05:37:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e000ba30 con 0x7fd5e00039a0 2023-10-08T05:37:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e000bbf0 con 0x7fd5f40afbe0 2023-10-08T05:37:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5ebfff640 1 --2- 172.21.15.70:0/2775935445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 0x7fd5e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5ebfff640 1 -- 172.21.15.70:0/2775935445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f40afbe0 msgr2=0x7fd5e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5ebfff640 1 --2- 172.21.15.70:0/2775935445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f40afbe0 0x7fd5e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5ebfff640 1 -- 172.21.15.70:0/2775935445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e00039a0 msgr2=0x7fd5e0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5ebfff640 1 --2- 172.21.15.70:0/2775935445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5e00039a0 0x7fd5e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.720+0000 7fd5ebfff640 1 -- 172.21.15.70:0/2775935445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e0003520 con 0x7fd5e00047c0 2023-10-08T05:37:18.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.721+0000 7fd5ebfff640 1 --2- 172.21.15.70:0/2775935445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 0x7fd5e0004c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc007fd0 tx=0x7fd5e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31c528dc472528e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.722+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc005ad0 con 0x7fd5e00047c0 2023-10-08T05:37:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.722+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc005c50 con 0x7fd5e00047c0 2023-10-08T05:37:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.722+0000 7fd5f37fe640 1 -- 172.21.15.70:0/2775935445 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc02c850 con 0x7fd5e00047c0 2023-10-08T05:37:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.722+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/2775935445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 msgr2=0x7fd5e0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.722+0000 7fd5fa5a4640 1 --2- 172.21.15.70:0/2775935445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 0x7fd5e0004c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc007fd0 tx=0x7fd5e00057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.722+0000 7fd5f8b41640 1 -- 172.21.15.70:0/2775935445 reap_dead start 2023-10-08T05:37:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.722+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/2775935445 shutdown_connections 2023-10-08T05:37:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.722+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/2775935445 >> 172.21.15.70:0/2775935445 conn(0x7fd5f406a5d0 msgr2=0x7fd5f40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.722+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/2775935445 shutdown_connections 2023-10-08T05:37:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.723+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/2775935445 wait complete. 2023-10-08T05:37:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.724+0000 7fd5fa5a4640 1 Processor -- start 2023-10-08T05:37:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.724+0000 7fd5fa5a4640 1 -- start start 2023-10-08T05:37:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.725+0000 7fd5fa5a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 0x7fd5f4149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.725+0000 7fd5fa5a4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5f415b510 0x7fd5f415b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.725+0000 7fd5fa5a4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f406c5b0 0x7fd5f406ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.725+0000 7fd5fa5a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f415bed0 con 0x7fd5e00047c0 2023-10-08T05:37:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.725+0000 7fd5fa5a4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f406cf70 con 0x7fd5f415b510 2023-10-08T05:37:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.725+0000 7fd5fa5a4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f415cef0 con 0x7fd5f406c5b0 2023-10-08T05:37:18.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.725+0000 7fd5f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 0x7fd5f4149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 0x7fd5f4149b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37182/0 (socket says 172.21.15.70:37182) 2023-10-08T05:37:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f3fff640 1 -- 172.21.15.70:0/3818668324 learned_addr learned my addr 172.21.15.70:0/3818668324 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5ebfff640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5f415b510 0x7fd5f415b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f8b41640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f406c5b0 0x7fd5f406ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f3fff640 1 -- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f406c5b0 msgr2=0x7fd5f406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f3fff640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f406c5b0 0x7fd5f406ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f3fff640 1 -- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5f415b510 msgr2=0x7fd5f415b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f3fff640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5f415b510 0x7fd5f415b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f3fff640 1 -- 172.21.15.70:0/3818668324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f40c8890 con 0x7fd5e00047c0 2023-10-08T05:37:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5ebfff640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5f415b510 0x7fd5f415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f3fff640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 0x7fd5f4149b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400e930 tx=0x7fd5e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f17fa640 1 -- 172.21.15.70:0/3818668324 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4009680 con 0x7fd5e00047c0 2023-10-08T05:37:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f17fa640 1 -- 172.21.15.70:0/3818668324 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e400c8e0 con 0x7fd5e00047c0 2023-10-08T05:37:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5f17fa640 1 -- 172.21.15.70:0/3818668324 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e400ccb0 con 0x7fd5e00047c0 2023-10-08T05:37:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.726+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/3818668324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f414a0b0 con 0x7fd5e00047c0 2023-10-08T05:37:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.727+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/3818668324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f415c390 con 0x7fd5e00047c0 2023-10-08T05:37:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.728+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/3818668324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f404e160 con 0x7fd5e00047c0 2023-10-08T05:37:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.729+0000 7fd5f17fa640 1 -- 172.21.15.70:0/3818668324 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd5e4013470 con 0x7fd5e00047c0 2023-10-08T05:37:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.730+0000 7fd5f17fa640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd5d406aea0 0x7fd5d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:18.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.730+0000 7fd5f17fa640 1 -- 172.21.15.70:0/3818668324 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd5e4089e90 con 0x7fd5e00047c0 2023-10-08T05:37:18.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.730+0000 7fd5ebfff640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd5d406aea0 0x7fd5d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:18.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.731+0000 7fd5ebfff640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd5d406aea0 0x7fd5d406d360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc034000 tx=0x7fd5dc009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:18.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.734+0000 7fd5f17fa640 1 -- 172.21.15.70:0/3818668324 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd5e4058600 con 0x7fd5e00047c0 2023-10-08T05:37:18.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.911+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/3818668324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5f40b0ef0 con 0x7fd5e00047c0 2023-10-08T05:37:18.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.912+0000 7fd5f17fa640 1 -- 172.21.15.70:0/3818668324 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5e40170a0 con 0x7fd5e00047c0 2023-10-08T05:37:18.914 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:37:18.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.914+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd5d406aea0 msgr2=0x7fd5d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.914+0000 7fd5fa5a4640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd5d406aea0 0x7fd5d406d360 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc034000 tx=0x7fd5dc009030 comp rx=0 tx=0).stop 2023-10-08T05:37:18.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.915+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/3818668324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 msgr2=0x7fd5f4149b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:18.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.915+0000 7fd5fa5a4640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 0x7fd5f4149b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400e930 tx=0x7fd5e400edf0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.915+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/3818668324 shutdown_connections 2023-10-08T05:37:18.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.915+0000 7fd5fa5a4640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd5d406aea0 0x7fd5d406d360 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.915+0000 7fd5fa5a4640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5f406c5b0 0x7fd5f406ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.915+0000 7fd5fa5a4640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5f415b510 0x7fd5f415b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.915+0000 7fd5fa5a4640 1 --2- 172.21.15.70:0/3818668324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5e00047c0 0x7fd5f4149b70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:18.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.915+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/3818668324 >> 172.21.15.70:0/3818668324 conn(0x7fd5f406a5d0 msgr2=0x7fd5f40c9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:18.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.915+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/3818668324 shutdown_connections 2023-10-08T05:37:18.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:18.915+0000 7fd5fa5a4640 1 -- 172.21.15.70:0/3818668324 wait complete. 2023-10-08T05:37:18.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:37:19.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.153+0000 7f566f496640 1 Processor -- start 2023-10-08T05:37:19.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.153+0000 7f566f496640 1 -- start start 2023-10-08T05:37:19.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.155+0000 7f566f496640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5668160660 0x7f5668070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.155+0000 7f566f496640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56680cb620 0x7f56680b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.155+0000 7f566f496640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566806e150 con 0x7f56680c9910 2023-10-08T05:37:19.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.155+0000 7f566f496640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566806de70 con 0x7f5668160660 2023-10-08T05:37:19.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.155+0000 7f566f496640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56680c8e30 con 0x7f56680cb620 2023-10-08T05:37:19.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.155+0000 7f566d232640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5668160660 0x7f5668070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.155+0000 7f566d232640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5668160660 0x7f5668070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37198/0 (socket says 172.21.15.70:37198) 2023-10-08T05:37:19.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.155+0000 7f566d232640 1 -- 172.21.15.70:0/3179954923 learned_addr learned my addr 172.21.15.70:0/3179954923 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:19.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.156+0000 7f566d232640 1 -- 172.21.15.70:0/3179954923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56680cb620 msgr2=0x7f56680b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:19.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.156+0000 7f566da33640 1 --2- 172.21.15.70:0/3179954923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56680cb620 0x7f56680b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.156+0000 7f566d232640 1 --2- 172.21.15.70:0/3179954923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56680cb620 0x7f56680b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.156+0000 7f566d232640 1 -- 172.21.15.70:0/3179954923 >> v1:172.21.15.70:6789/0 conn(0x7f56680c9910 legacy=0x7f56680caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.156+0000 7f566d232640 1 -- 172.21.15.70:0/3179954923 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680c9110 con 0x7f5668160660 2023-10-08T05:37:19.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.157+0000 7f566d232640 1 --2- 172.21.15.70:0/3179954923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5668160660 0x7f5668070cc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5664009870 tx=0x7f566402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ffcbf329505446 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.157+0000 7f565ffff640 1 -- 172.21.15.70:0/3179954923 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5664030840 con 0x7f5668160660 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.157+0000 7f565ffff640 1 -- 172.21.15.70:0/3179954923 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56640309a0 con 0x7f5668160660 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.157+0000 7f565ffff640 1 -- 172.21.15.70:0/3179954923 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5664030d30 con 0x7f5668160660 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.158+0000 7f566f496640 1 -- 172.21.15.70:0/3179954923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5668160660 msgr2=0x7f5668070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.158+0000 7f566f496640 1 --2- 172.21.15.70:0/3179954923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5668160660 0x7f5668070cc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5664009870 tx=0x7f566402b980 comp rx=0 tx=0).stop 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.158+0000 7f566f496640 1 -- 172.21.15.70:0/3179954923 shutdown_connections 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.158+0000 7f566f496640 1 --2- 172.21.15.70:0/3179954923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56680cb620 0x7f56680b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.158+0000 7f566f496640 1 --2- 172.21.15.70:0/3179954923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5668160660 0x7f5668070cc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.158+0000 7f566f496640 1 -- 172.21.15.70:0/3179954923 >> 172.21.15.70:0/3179954923 conn(0x7f566806a5d0 msgr2=0x7f56680b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.158+0000 7f566f496640 1 -- 172.21.15.70:0/3179954923 shutdown_connections 2023-10-08T05:37:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.158+0000 7f566f496640 1 -- 172.21.15.70:0/3179954923 wait complete. 2023-10-08T05:37:19.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.160+0000 7f566f496640 1 Processor -- start 2023-10-08T05:37:19.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.161+0000 7f566f496640 1 -- start start 2023-10-08T05:37:19.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566f496640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56680c9910 0x7f5668154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566f496640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56680cb620 0x7f5668159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566f496640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5668160660 0x7f5668159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566f496640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56681f2be0 con 0x7f56680c9910 2023-10-08T05:37:19.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566f496640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56680cc640 con 0x7f56680cb620 2023-10-08T05:37:19.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566f496640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5668158870 con 0x7f5668160660 2023-10-08T05:37:19.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566d232640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56680c9910 0x7f5668154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566d232640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56680c9910 0x7f5668154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37204/0 (socket says 172.21.15.70:37204) 2023-10-08T05:37:19.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566d232640 1 -- 172.21.15.70:0/197210320 learned_addr learned my addr 172.21.15.70:0/197210320 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:19.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566d232640 1 -- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5668160660 msgr2=0x7f5668159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566d232640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5668160660 0x7f5668159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566d232640 1 -- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56680cb620 msgr2=0x7f5668159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566d232640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56680cb620 0x7f5668159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.162+0000 7f566d232640 1 -- 172.21.15.70:0/197210320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640094d0 con 0x7f56680c9910 2023-10-08T05:37:19.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.163+0000 7f566d232640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56680c9910 0x7f5668154110 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f566402b9c0 tx=0x7f5664004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:19.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.163+0000 7f565dffb640 1 -- 172.21.15.70:0/197210320 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5664030810 con 0x7f56680c9910 2023-10-08T05:37:19.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.163+0000 7f565dffb640 1 -- 172.21.15.70:0/197210320 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5664039d90 con 0x7f56680c9910 2023-10-08T05:37:19.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.163+0000 7f565dffb640 1 -- 172.21.15.70:0/197210320 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5664043670 con 0x7f56680c9910 2023-10-08T05:37:19.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.163+0000 7f566f496640 1 -- 172.21.15.70:0/197210320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5668158b50 con 0x7f56680c9910 2023-10-08T05:37:19.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.163+0000 7f566f496640 1 -- 172.21.15.70:0/197210320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5668159110 con 0x7f56680c9910 2023-10-08T05:37:19.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.164+0000 7f566f496640 1 -- 172.21.15.70:0/197210320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566804e160 con 0x7f56680c9910 2023-10-08T05:37:19.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.165+0000 7f565dffb640 1 -- 172.21.15.70:0/197210320 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f566402f390 con 0x7f56680c9910 2023-10-08T05:37:19.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.166+0000 7f565dffb640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f564c06ae70 0x7f564c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.166+0000 7f565dffb640 1 -- 172.21.15.70:0/197210320 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f56640abd80 con 0x7f56680c9910 2023-10-08T05:37:19.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.167+0000 7f566ca31640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f564c06ae70 0x7f564c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.168+0000 7f566ca31640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f564c06ae70 0x7f564c06d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5658000f60 tx=0x7f56580079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:19.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.170+0000 7f565dffb640 1 -- 172.21.15.70:0/197210320 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5664076a60 con 0x7f56680c9910 2023-10-08T05:37:19.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.361+0000 7f566f496640 1 -- 172.21.15.70:0/197210320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f56680ca9f0 con 0x7f56680c9910 2023-10-08T05:37:19.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.362+0000 7f565dffb640 1 -- 172.21.15.70:0/197210320 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5664034020 con 0x7f56680c9910 2023-10-08T05:37:19.363 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:19.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.364+0000 7f566f496640 1 -- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f564c06ae70 msgr2=0x7f564c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.364+0000 7f566f496640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f564c06ae70 0x7f564c06d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5658000f60 tx=0x7f56580079b0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.364+0000 7f566f496640 1 -- 172.21.15.70:0/197210320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56680c9910 msgr2=0x7f5668154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.364+0000 7f566f496640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56680c9910 0x7f5668154110 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f566402b9c0 tx=0x7f5664004730 comp rx=0 tx=0).stop 2023-10-08T05:37:19.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.364+0000 7f566f496640 1 -- 172.21.15.70:0/197210320 shutdown_connections 2023-10-08T05:37:19.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.364+0000 7f566f496640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f564c06ae70 0x7f564c06d330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.364+0000 7f566f496640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5668160660 0x7f5668159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.364+0000 7f566f496640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56680cb620 0x7f5668159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.364+0000 7f566f496640 1 --2- 172.21.15.70:0/197210320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56680c9910 0x7f5668154110 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.364+0000 7f566f496640 1 -- 172.21.15.70:0/197210320 >> 172.21.15.70:0/197210320 conn(0x7f566806a5d0 msgr2=0x7f566815b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:19.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.365+0000 7f566f496640 1 -- 172.21.15.70:0/197210320 shutdown_connections 2023-10-08T05:37:19.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.365+0000 7f566f496640 1 -- 172.21.15.70:0/197210320 wait complete. 2023-10-08T05:37:19.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:37:19.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.601+0000 7fcb5a8b4640 1 Processor -- start 2023-10-08T05:37:19.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.601+0000 7fcb5a8b4640 1 -- start start 2023-10-08T05:37:19.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.603+0000 7fcb5a8b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540afbe0 0x7fcb540b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.603+0000 7fcb5a8b4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb540cb130 0x7fcb540cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.603+0000 7fcb5a8b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb540b2620 con 0x7fcb5406dfc0 2023-10-08T05:37:19.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.603+0000 7fcb5a8b4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb540cbae0 con 0x7fcb540cb130 2023-10-08T05:37:19.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.603+0000 7fcb5a8b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540b2060 con 0x7fcb540afbe0 2023-10-08T05:37:19.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.603+0000 7fcb4bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcb5406dfc0 0x7fcb54070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50070/0 (socket says 172.21.15.70:50070) 2023-10-08T05:37:19.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.603+0000 7fcb4bfff640 1 -- 172.21.15.70:0/1962481918 learned_addr learned my addr 172.21.15.70:0/1962481918 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:19.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb53fff640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540afbe0 0x7fcb540b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb58e51640 1 --2- 172.21.15.70:0/1962481918 >> v2:172.21.15.181:3300/0 conn(0x7fcb540cb130 0x7fcb540cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238454425 0 0) 0x7fcb540b2620 con 0x7fcb5406dfc0 2023-10-08T05:37:19.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb40003520 con 0x7fcb5406dfc0 2023-10-08T05:37:19.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2401997439 0 0) 0x7fcb40003520 con 0x7fcb5406dfc0 2023-10-08T05:37:19.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540afbe0 msgr2=0x7fcb540b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:19.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb537fe640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540afbe0 0x7fcb540b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 >> v2:172.21.15.181:3300/0 conn(0x7fcb540cb130 msgr2=0x7fcb540cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:19.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb537fe640 1 --2- 172.21.15.70:0/1962481918 >> v2:172.21.15.181:3300/0 conn(0x7fcb540cb130 0x7fcb540cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540b2340 con 0x7fcb5406dfc0 2023-10-08T05:37:19.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.604+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fcb3c001670 con 0x7fcb5406dfc0 2023-10-08T05:37:19.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 >> v1:172.21.15.70:6789/0 conn(0x7fcb5406dfc0 legacy=0x7fcb54070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb537fe640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540afbe0 0x7fcb40003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb537fe640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb40003f40 0x7fcb400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb537fe640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb40004d00 0x7fcb40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540cbae0 con 0x7fcb40004d00 2023-10-08T05:37:19.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540b2060 con 0x7fcb40003f40 2023-10-08T05:37:19.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4000b980 con 0x7fcb540afbe0 2023-10-08T05:37:19.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb58e51640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb40004d00 0x7fcb40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb58e51640 1 -- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540afbe0 msgr2=0x7fcb40003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:19.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb58e51640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb540afbe0 0x7fcb40003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb58e51640 1 -- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb40003f40 msgr2=0x7fcb400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:19.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb4bfff640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb40003f40 0x7fcb400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb58e51640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb40003f40 0x7fcb400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb58e51640 1 -- 172.21.15.70:0/1962481918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40003520 con 0x7fcb40004d00 2023-10-08T05:37:19.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.605+0000 7fcb4bfff640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb40003f40 0x7fcb400047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:19.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.607+0000 7fcb58e51640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb40004d00 0x7fcb40005180 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e170 tx=0x7fcb400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfd162e4d83ee02 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:19.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.607+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c02cc20 con 0x7fcb40004d00 2023-10-08T05:37:19.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.607+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c00f040 con 0x7fcb40004d00 2023-10-08T05:37:19.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.607+0000 7fcb537fe640 1 -- 172.21.15.70:0/1962481918 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c034440 con 0x7fcb40004d00 2023-10-08T05:37:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.607+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/1962481918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb40004d00 msgr2=0x7fcb40005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.607+0000 7fcb5a8b4640 1 --2- 172.21.15.70:0/1962481918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb40004d00 0x7fcb40005180 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00e170 tx=0x7fcb400057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.607+0000 7fcb58e51640 1 -- 172.21.15.70:0/1962481918 reap_dead start 2023-10-08T05:37:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.607+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/1962481918 shutdown_connections 2023-10-08T05:37:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.607+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/1962481918 >> 172.21.15.70:0/1962481918 conn(0x7fcb5406a5d0 msgr2=0x7fcb540b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.608+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/1962481918 shutdown_connections 2023-10-08T05:37:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.608+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/1962481918 wait complete. 2023-10-08T05:37:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.609+0000 7fcb5a8b4640 1 Processor -- start 2023-10-08T05:37:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.609+0000 7fcb5a8b4640 1 -- start start 2023-10-08T05:37:19.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.610+0000 7fcb5a8b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40004d00 0x7fcb54156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.610+0000 7fcb5a8b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb541550e0 0x7fcb5415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.610+0000 7fcb5a8b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb5415df00 0x7fcb541f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.610+0000 7fcb5a8b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb540c2bf0 con 0x7fcb5415df00 2023-10-08T05:37:19.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.610+0000 7fcb5a8b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb54157270 con 0x7fcb541550e0 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.610+0000 7fcb5a8b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5415e380 con 0x7fcb40004d00 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.611+0000 7fcb58e51640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb5415df00 0x7fcb541f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.611+0000 7fcb58e51640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb5415df00 0x7fcb541f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37232/0 (socket says 172.21.15.70:37232) 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.611+0000 7fcb58e51640 1 -- 172.21.15.70:0/802820169 learned_addr learned my addr 172.21.15.70:0/802820169 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.611+0000 7fcb53fff640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40004d00 0x7fcb54156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.611+0000 7fcb58e51640 1 -- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40004d00 msgr2=0x7fcb54156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.611+0000 7fcb58e51640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40004d00 0x7fcb54156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.611+0000 7fcb58e51640 1 -- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb541550e0 msgr2=0x7fcb5415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.611+0000 7fcb58e51640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb541550e0 0x7fcb5415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.611+0000 7fcb58e51640 1 -- 172.21.15.70:0/802820169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540aeeb0 con 0x7fcb5415df00 2023-10-08T05:37:19.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.611+0000 7fcb58e51640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb5415df00 0x7fcb541f5b40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009f90 tx=0x7fcb4c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:19.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.612+0000 7fcb517fa640 1 -- 172.21.15.70:0/802820169 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c043070 con 0x7fcb5415df00 2023-10-08T05:37:19.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.612+0000 7fcb517fa640 1 -- 172.21.15.70:0/802820169 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c034b50 con 0x7fcb5415df00 2023-10-08T05:37:19.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.612+0000 7fcb517fa640 1 -- 172.21.15.70:0/802820169 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c003990 con 0x7fcb5415df00 2023-10-08T05:37:19.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.612+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/802820169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb540c2250 con 0x7fcb5415df00 2023-10-08T05:37:19.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.612+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/802820169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb540c2810 con 0x7fcb5415df00 2023-10-08T05:37:19.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.613+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/802820169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb540c29a0 con 0x7fcb5415df00 2023-10-08T05:37:19.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.614+0000 7fcb517fa640 1 -- 172.21.15.70:0/802820169 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fcb4c03e440 con 0x7fcb5415df00 2023-10-08T05:37:19.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.615+0000 7fcb517fa640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcb3406ae70 0x7fcb3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:19.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.615+0000 7fcb517fa640 1 -- 172.21.15.70:0/802820169 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fcb4c0ab7e0 con 0x7fcb5415df00 2023-10-08T05:37:19.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.616+0000 7fcb53fff640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcb3406ae70 0x7fcb3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:19.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.617+0000 7fcb53fff640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcb3406ae70 0x7fcb3406d330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb44004770 tx=0x7fcb44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:19.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.619+0000 7fcb517fa640 1 -- 172.21.15.70:0/802820169 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb4c0765f0 con 0x7fcb5415df00 2023-10-08T05:37:19.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.821+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/802820169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcb5404e160 con 0x7fcb5415df00 2023-10-08T05:37:19.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.822+0000 7fcb517fa640 1 -- 172.21.15.70:0/802820169 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcb4c039080 con 0x7fcb5415df00 2023-10-08T05:37:19.823 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-10-08T05:37:19.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.824+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcb3406ae70 msgr2=0x7fcb3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.824+0000 7fcb5a8b4640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcb3406ae70 0x7fcb3406d330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb44004770 tx=0x7fcb44008040 comp rx=0 tx=0).stop 2023-10-08T05:37:19.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.824+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/802820169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb5415df00 msgr2=0x7fcb541f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:19.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.824+0000 7fcb5a8b4640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb5415df00 0x7fcb541f5b40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009f90 tx=0x7fcb4c009750 comp rx=0 tx=0).stop 2023-10-08T05:37:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.824+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/802820169 shutdown_connections 2023-10-08T05:37:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.824+0000 7fcb5a8b4640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fcb3406ae70 0x7fcb3406d330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.824+0000 7fcb5a8b4640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb5415df00 0x7fcb541f5b40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.824+0000 7fcb5a8b4640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb541550e0 0x7fcb5415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.824+0000 7fcb5a8b4640 1 --2- 172.21.15.70:0/802820169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40004d00 0x7fcb54156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.824+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/802820169 >> 172.21.15.70:0/802820169 conn(0x7fcb5406a5d0 msgr2=0x7fcb540ccd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.825+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/802820169 shutdown_connections 2023-10-08T05:37:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:19.825+0000 7fcb5a8b4640 1 -- 172.21.15.70:0/802820169 wait complete. 2023-10-08T05:37:19.846 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T05:37:19.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:19.846 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-08T05:37:19.962 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T05:37:19.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:19.962 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T05:37:20.043 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:37:20.043 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:20.103 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:37:20.104 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:20.158 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:37:20.158 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:20.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:20.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.425+0000 7f45ae84b640 1 Processor -- start 2023-10-08T05:37:20.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.425+0000 7f45ae84b640 1 -- start start 2023-10-08T05:37:20.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.426+0000 7f45ae84b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f45a8159ac0 0x7f45a806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.426+0000 7f45ae84b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45a80ad790 0x7f45a80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.426+0000 7f45ae84b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f45a815e330 0x7f45a8070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.426+0000 7f45ae84b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a806dee0 con 0x7f45a815e330 2023-10-08T05:37:20.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.426+0000 7f45ae84b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a815d850 con 0x7f45a8159ac0 2023-10-08T05:37:20.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.426+0000 7f45ae84b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a815db30 con 0x7f45a80ad790 2023-10-08T05:37:20.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f45acde8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f45a815e330 0x7f45a8070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:20.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f45acde8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f45a815e330 0x7f45a8070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37240/0 (socket says 172.21.15.70:37240) 2023-10-08T05:37:20.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f45acde8640 1 -- 172.21.15.70:0/2085157598 learned_addr learned my addr 172.21.15.70:0/2085157598 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:20.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f45a7fff640 1 --2- 172.21.15.70:0/2085157598 >> v2:172.21.15.181:3300/0 conn(0x7f45a8159ac0 0x7f45a806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:20.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f459ffff640 1 --2- 172.21.15.70:0/2085157598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45a80ad790 0x7f45a80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f45acde8640 1 -- 172.21.15.70:0/2085157598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45a80ad790 msgr2=0x7f45a80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f45acde8640 1 --2- 172.21.15.70:0/2085157598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45a80ad790 0x7f45a80702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f45acde8640 1 -- 172.21.15.70:0/2085157598 >> v2:172.21.15.181:3300/0 conn(0x7f45a8159ac0 msgr2=0x7f45a806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f45acde8640 1 --2- 172.21.15.70:0/2085157598 >> v2:172.21.15.181:3300/0 conn(0x7f45a8159ac0 0x7f45a806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f45acde8640 1 -- 172.21.15.70:0/2085157598 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a815de10 con 0x7f45a815e330 2023-10-08T05:37:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.427+0000 7f45a7fff640 1 --2- 172.21.15.70:0/2085157598 >> v2:172.21.15.181:3300/0 conn(0x7f45a8159ac0 0x7f45a806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.428+0000 7f45acde8640 1 --2- 172.21.15.70:0/2085157598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a815e330 0x7f45a8070810 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f45a806e790 tx=0x7f45a002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed98edd3eff05d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:20.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.428+0000 7f45a77fe640 1 -- 172.21.15.70:0/2085157598 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a002ca80 con 0x7f45a815e330 2023-10-08T05:37:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45a77fe640 1 -- 172.21.15.70:0/2085157598 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a002cbe0 con 0x7f45a815e330 2023-10-08T05:37:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45a77fe640 1 -- 172.21.15.70:0/2085157598 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a00343f0 con 0x7f45a815e330 2023-10-08T05:37:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45ae84b640 1 -- 172.21.15.70:0/2085157598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a815e330 msgr2=0x7f45a8070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45ae84b640 1 --2- 172.21.15.70:0/2085157598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a815e330 0x7f45a8070810 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f45a806e790 tx=0x7f45a002bee0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45ae84b640 1 -- 172.21.15.70:0/2085157598 shutdown_connections 2023-10-08T05:37:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45ae84b640 1 --2- 172.21.15.70:0/2085157598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a815e330 0x7f45a8070810 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45ae84b640 1 --2- 172.21.15.70:0/2085157598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45a80ad790 0x7f45a80702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45ae84b640 1 --2- 172.21.15.70:0/2085157598 >> v2:172.21.15.181:3300/0 conn(0x7f45a8159ac0 0x7f45a806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45ae84b640 1 -- 172.21.15.70:0/2085157598 >> 172.21.15.70:0/2085157598 conn(0x7f45a806a380 msgr2=0x7f45a80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:20.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45ae84b640 1 -- 172.21.15.70:0/2085157598 shutdown_connections 2023-10-08T05:37:20.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.429+0000 7f45ae84b640 1 -- 172.21.15.70:0/2085157598 wait complete. 2023-10-08T05:37:20.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.431+0000 7f45ae84b640 1 Processor -- start 2023-10-08T05:37:20.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.431+0000 7f45ae84b640 1 -- start start 2023-10-08T05:37:20.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.432+0000 7f45ae84b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a80ad790 0x7f45a81f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.432+0000 7f45ae84b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45a8159ac0 0x7f45a81f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.432+0000 7f45ae84b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45a815e330 0x7f45a81edd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.432+0000 7f45ae84b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81f1830 con 0x7f45a80ad790 2023-10-08T05:37:20.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.432+0000 7f45ae84b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81ee270 con 0x7f45a8159ac0 2023-10-08T05:37:20.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.432+0000 7f45ae84b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a815a3c0 con 0x7f45a815e330 2023-10-08T05:37:20.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.432+0000 7f45a7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a80ad790 0x7f45a81f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:20.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.432+0000 7f45a7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a80ad790 0x7f45a81f1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37246/0 (socket says 172.21.15.70:37246) 2023-10-08T05:37:20.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.432+0000 7f45a7fff640 1 -- 172.21.15.70:0/184118938 learned_addr learned my addr 172.21.15.70:0/184118938 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:20.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.433+0000 7f45a7fff640 1 -- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45a815e330 msgr2=0x7f45a81edd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:20.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.433+0000 7f45a7fff640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45a815e330 0x7f45a81edd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.433+0000 7f45a7fff640 1 -- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45a8159ac0 msgr2=0x7f45a81f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:20.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.433+0000 7f45a7fff640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45a8159ac0 0x7f45a81f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.433+0000 7f45a7fff640 1 -- 172.21.15.70:0/184118938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a0009b30 con 0x7f45a80ad790 2023-10-08T05:37:20.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.433+0000 7f45a7fff640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a80ad790 0x7f45a81f1a30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f459800c7f0 tx=0x7f459800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:20.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.433+0000 7f45a57fa640 1 -- 172.21.15.70:0/184118938 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598007a30 con 0x7f45a80ad790 2023-10-08T05:37:20.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.433+0000 7f45a57fa640 1 -- 172.21.15.70:0/184118938 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4598007b90 con 0x7f45a80ad790 2023-10-08T05:37:20.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.433+0000 7f45a57fa640 1 -- 172.21.15.70:0/184118938 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459800ce40 con 0x7f45a80ad790 2023-10-08T05:37:20.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.433+0000 7f45ae84b640 1 -- 172.21.15.70:0/184118938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a815a6a0 con 0x7f45a80ad790 2023-10-08T05:37:20.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.434+0000 7f45ae84b640 1 -- 172.21.15.70:0/184118938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a815ac60 con 0x7f45a80ad790 2023-10-08T05:37:20.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.435+0000 7f45ae84b640 1 -- 172.21.15.70:0/184118938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a8067490 con 0x7f45a80ad790 2023-10-08T05:37:20.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.436+0000 7f45a57fa640 1 -- 172.21.15.70:0/184118938 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f45980113f0 con 0x7f45a80ad790 2023-10-08T05:37:20.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.436+0000 7f45a57fa640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f458806ae70 0x7f458806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.437+0000 7f45a57fa640 1 -- 172.21.15.70:0/184118938 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4598089c60 con 0x7f45a80ad790 2023-10-08T05:37:20.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.437+0000 7f459ffff640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f458806ae70 0x7f458806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:20.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.438+0000 7f459ffff640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f458806ae70 0x7f458806d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4590006b10 tx=0x7f4590008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:20.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.441+0000 7f45a57fa640 1 -- 172.21.15.70:0/184118938 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45980548b0 con 0x7f45a80ad790 2023-10-08T05:37:20.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.629+0000 7f45ae84b640 1 -- 172.21.15.70:0/184118938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45a80ad510 con 0x7f45a80ad790 2023-10-08T05:37:20.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.633+0000 7f45a57fa640 1 -- 172.21.15.70:0/184118938 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7f459801f090 con 0x7f45a80ad790 2023-10-08T05:37:20.634 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:20.635 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:20.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.635+0000 7f45ae84b640 1 -- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f458806ae70 msgr2=0x7f458806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:20.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.635+0000 7f45ae84b640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f458806ae70 0x7f458806d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4590006b10 tx=0x7f4590008040 comp rx=0 tx=0).stop 2023-10-08T05:37:20.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.635+0000 7f45ae84b640 1 -- 172.21.15.70:0/184118938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a80ad790 msgr2=0x7f45a81f1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:20.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.635+0000 7f45ae84b640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a80ad790 0x7f45a81f1a30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f459800c7f0 tx=0x7f459800cbf0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.635+0000 7f45ae84b640 1 -- 172.21.15.70:0/184118938 shutdown_connections 2023-10-08T05:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.635+0000 7f45ae84b640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f458806ae70 0x7f458806d330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.635+0000 7f45ae84b640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f45a815e330 0x7f45a81edd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.635+0000 7f45ae84b640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45a8159ac0 0x7f45a81f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.635+0000 7f45ae84b640 1 --2- 172.21.15.70:0/184118938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45a80ad790 0x7f45a81f1a30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.635+0000 7f45ae84b640 1 -- 172.21.15.70:0/184118938 >> 172.21.15.70:0/184118938 conn(0x7f45a806a380 msgr2=0x7f45a80cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:20.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.636+0000 7f45ae84b640 1 -- 172.21.15.70:0/184118938 shutdown_connections 2023-10-08T05:37:20.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.636+0000 7f45ae84b640 1 -- 172.21.15.70:0/184118938 wait complete. 2023-10-08T05:37:20.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:37:20.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.871+0000 7f1007010640 1 Processor -- start 2023-10-08T05:37:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.871+0000 7f1007010640 1 -- start start 2023-10-08T05:37:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.873+0000 7f1007010640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10000cb880 0x7f10000b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.873+0000 7f1007010640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100015d440 con 0x7f10000b4070 2023-10-08T05:37:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.873+0000 7f1007010640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100015d690 con 0x7f10001432f0 2023-10-08T05:37:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.873+0000 7f1007010640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10001436d0 con 0x7f10000cb880 2023-10-08T05:37:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.873+0000 7f0ffffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10000cb880 0x7f10000b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:20.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.873+0000 7f0ffffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10000cb880 0x7f10000b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37250/0 (socket says 172.21.15.70:37250) 2023-10-08T05:37:20.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.873+0000 7f0ffffff640 1 -- 172.21.15.70:0/1572856033 learned_addr learned my addr 172.21.15.70:0/1572856033 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:20.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.873+0000 7f0ffffff640 1 -- 172.21.15.70:0/1572856033 >> v1:172.21.15.181:6789/0 conn(0x7f10001432f0 legacy=0x7f10000cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:20.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.874+0000 7f0ffffff640 1 -- 172.21.15.70:0/1572856033 >> v1:172.21.15.70:6789/0 conn(0x7f10000b4070 legacy=0x7f1000159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:20.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.874+0000 7f0ffffff640 1 -- 172.21.15.70:0/1572856033 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100015d970 con 0x7f10000cb880 2023-10-08T05:37:20.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.875+0000 7f0ffffff640 1 --2- 172.21.15.70:0/1572856033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000cb880 0x7f10000b3b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400bb10 tx=0x7f0ff402e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d082a1191e5cbd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:20.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.875+0000 7f0fff7fe640 1 -- 172.21.15.70:0/1572856033 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff402ee80 con 0x7f10000cb880 2023-10-08T05:37:20.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.875+0000 7f0fff7fe640 1 -- 172.21.15.70:0/1572856033 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff4036440 con 0x7f10000cb880 2023-10-08T05:37:20.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.875+0000 7f0fff7fe640 1 -- 172.21.15.70:0/1572856033 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40367b0 con 0x7f10000cb880 2023-10-08T05:37:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.876+0000 7f1007010640 1 -- 172.21.15.70:0/1572856033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000cb880 msgr2=0x7f10000b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.876+0000 7f1007010640 1 --2- 172.21.15.70:0/1572856033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000cb880 0x7f10000b3b30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400bb10 tx=0x7f0ff402e0d0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.876+0000 7f1007010640 1 -- 172.21.15.70:0/1572856033 shutdown_connections 2023-10-08T05:37:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.876+0000 7f1007010640 1 --2- 172.21.15.70:0/1572856033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000cb880 0x7f10000b3b30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.876+0000 7f1007010640 1 -- 172.21.15.70:0/1572856033 >> 172.21.15.70:0/1572856033 conn(0x7f1000069d50 msgr2=0x7f100006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:20.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.876+0000 7f1007010640 1 -- 172.21.15.70:0/1572856033 shutdown_connections 2023-10-08T05:37:20.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.876+0000 7f1007010640 1 -- 172.21.15.70:0/1572856033 wait complete. 2023-10-08T05:37:20.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.878+0000 7f1007010640 1 Processor -- start 2023-10-08T05:37:20.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.878+0000 7f1007010640 1 -- start start 2023-10-08T05:37:20.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.879+0000 7f1007010640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000b4070 0x7f10001572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.879+0000 7f1007010640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10000cb880 0x7f100013f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.879+0000 7f1007010640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10001432f0 0x7f10000c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.879+0000 7f1007010640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100015a0f0 con 0x7f10000b4070 2023-10-08T05:37:20.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.879+0000 7f1007010640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10001577e0 con 0x7f10000cb880 2023-10-08T05:37:20.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.879+0000 7f1007010640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100013ea10 con 0x7f10001432f0 2023-10-08T05:37:20.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.879+0000 7f1004dac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000b4070 0x7f10001572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:20.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.879+0000 7f1004dac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000b4070 0x7f10001572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37266/0 (socket says 172.21.15.70:37266) 2023-10-08T05:37:20.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.879+0000 7f1004dac640 1 -- 172.21.15.70:0/2370417821 learned_addr learned my addr 172.21.15.70:0/2370417821 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f0ffffff640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10000cb880 0x7f100013f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f10055ad640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10001432f0 0x7f10000c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f1004dac640 1 -- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10001432f0 msgr2=0x7f10000c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f1004dac640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10001432f0 0x7f10000c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f1004dac640 1 -- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10000cb880 msgr2=0x7f100013f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f1004dac640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10000cb880 0x7f100013f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f1004dac640 1 -- 172.21.15.70:0/2370417821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff400b770 con 0x7f10000b4070 2023-10-08T05:37:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f0ffffff640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10000cb880 0x7f100013f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f1004dac640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000b4070 0x7f10001572a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000ba80 tx=0x7f0ff000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f0ffd7fa640 1 -- 172.21.15.70:0/2370417821 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000cc00 con 0x7f10000b4070 2023-10-08T05:37:20.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.880+0000 7f0ffd7fa640 1 -- 172.21.15.70:0/2370417821 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff000cd60 con 0x7f10000b4070 2023-10-08T05:37:20.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.881+0000 7f0ffd7fa640 1 -- 172.21.15.70:0/2370417821 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0003bb0 con 0x7f10000b4070 2023-10-08T05:37:20.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.881+0000 7f1007010640 1 -- 172.21.15.70:0/2370417821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100013ecf0 con 0x7f10000b4070 2023-10-08T05:37:20.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.881+0000 7f1007010640 1 -- 172.21.15.70:0/2370417821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1000159d80 con 0x7f10000b4070 2023-10-08T05:37:20.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.882+0000 7f1007010640 1 -- 172.21.15.70:0/2370417821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1000159f10 con 0x7f10000b4070 2023-10-08T05:37:20.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.883+0000 7f0ffd7fa640 1 -- 172.21.15.70:0/2370417821 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0ff0014440 con 0x7f10000b4070 2023-10-08T05:37:20.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.884+0000 7f0ffd7fa640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0fec06aea0 0x7f0fec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:20.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.884+0000 7f0ffd7fa640 1 -- 172.21.15.70:0/2370417821 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0ff0089790 con 0x7f10000b4070 2023-10-08T05:37:20.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.885+0000 7f0ffffff640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0fec06aea0 0x7f0fec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:20.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.886+0000 7f0ffffff640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0fec06aea0 0x7f0fec06d360 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4038000 tx=0x7f0ff4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:20.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:20.889+0000 7f0ffd7fa640 1 -- 172.21.15.70:0/2370417821 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ff0054480 con 0x7f10000b4070 2023-10-08T05:37:21.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.079+0000 7f1007010640 1 -- 172.21.15.70:0/2370417821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f100004e160 con 0x7f10000b4070 2023-10-08T05:37:21.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.080+0000 7f0ffd7fa640 1 -- 172.21.15.70:0/2370417821 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ff0057aa0 con 0x7f10000b4070 2023-10-08T05:37:21.081 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:21.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.082+0000 7f1007010640 1 -- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0fec06aea0 msgr2=0x7f0fec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.082+0000 7f1007010640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0fec06aea0 0x7f0fec06d360 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4038000 tx=0x7f0ff4008040 comp rx=0 tx=0).stop 2023-10-08T05:37:21.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.082+0000 7f1007010640 1 -- 172.21.15.70:0/2370417821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000b4070 msgr2=0x7f10001572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.082+0000 7f1007010640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000b4070 0x7f10001572a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000ba80 tx=0x7f0ff000bf40 comp rx=0 tx=0).stop 2023-10-08T05:37:21.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.083+0000 7f1007010640 1 -- 172.21.15.70:0/2370417821 shutdown_connections 2023-10-08T05:37:21.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.083+0000 7f1007010640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0fec06aea0 0x7f0fec06d360 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.083+0000 7f1007010640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10001432f0 0x7f10000c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.083+0000 7f1007010640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10000cb880 0x7f100013f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.083+0000 7f1007010640 1 --2- 172.21.15.70:0/2370417821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000b4070 0x7f10001572a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.083+0000 7f1007010640 1 -- 172.21.15.70:0/2370417821 >> 172.21.15.70:0/2370417821 conn(0x7f1000069d50 msgr2=0x7f10000c9580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:21.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.083+0000 7f1007010640 1 -- 172.21.15.70:0/2370417821 shutdown_connections 2023-10-08T05:37:21.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.083+0000 7f1007010640 1 -- 172.21.15.70:0/2370417821 wait complete. 2023-10-08T05:37:21.104 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:37:21.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.323+0000 7f67b71d7640 1 Processor -- start 2023-10-08T05:37:21.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.323+0000 7f67b71d7640 1 -- start start 2023-10-08T05:37:21.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.324+0000 7f67b71d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67b01599d0 0x7f67b00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.324+0000 7f67b71d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67b00b3c50 0x7f67b00caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.324+0000 7f67b71d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b0159db0 con 0x7f67b00cb410 2023-10-08T05:37:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.324+0000 7f67b71d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b0071fe0 con 0x7f67b01599d0 2023-10-08T05:37:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.324+0000 7f67b71d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00b44c0 con 0x7f67b00b3c50 2023-10-08T05:37:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.324+0000 7f67b61d5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67b01599d0 0x7f67b00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.325+0000 7f67b61d5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67b01599d0 0x7f67b00cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37272/0 (socket says 172.21.15.70:37272) 2023-10-08T05:37:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.325+0000 7f67b61d5640 1 -- 172.21.15.70:0/4146032394 learned_addr learned my addr 172.21.15.70:0/4146032394 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.325+0000 7f67b61d5640 1 -- 172.21.15.70:0/4146032394 >> v2:172.21.15.181:3300/0 conn(0x7f67b00b3c50 msgr2=0x7f67b00caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.325+0000 7f67b61d5640 1 --2- 172.21.15.70:0/4146032394 >> v2:172.21.15.181:3300/0 conn(0x7f67b00b3c50 0x7f67b00caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.325+0000 7f67b61d5640 1 -- 172.21.15.70:0/4146032394 >> v1:172.21.15.181:6789/0 conn(0x7f67b00cb410 legacy=0x7f67b00c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.325+0000 7f67b61d5640 1 -- 172.21.15.70:0/4146032394 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b00cb860 con 0x7f67b01599d0 2023-10-08T05:37:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.326+0000 7f67b61d5640 1 --2- 172.21.15.70:0/4146032394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b01599d0 0x7f67b00cdcd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f67b00709e0 tx=0x7f67ac02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4660d973e7468652 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.327+0000 7f67b51d3640 1 -- 172.21.15.70:0/4146032394 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac02ee90 con 0x7f67b01599d0 2023-10-08T05:37:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.327+0000 7f67b51d3640 1 -- 172.21.15.70:0/4146032394 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac036440 con 0x7f67b01599d0 2023-10-08T05:37:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.327+0000 7f67b51d3640 1 -- 172.21.15.70:0/4146032394 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0367d0 con 0x7f67b01599d0 2023-10-08T05:37:21.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.327+0000 7f67b71d7640 1 -- 172.21.15.70:0/4146032394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b01599d0 msgr2=0x7f67b00cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.327+0000 7f67b71d7640 1 --2- 172.21.15.70:0/4146032394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b01599d0 0x7f67b00cdcd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f67b00709e0 tx=0x7f67ac02e0e0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.327+0000 7f67b71d7640 1 -- 172.21.15.70:0/4146032394 shutdown_connections 2023-10-08T05:37:21.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.328+0000 7f67b71d7640 1 --2- 172.21.15.70:0/4146032394 >> v2:172.21.15.181:3300/0 conn(0x7f67b00b3c50 0x7f67b00caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.328+0000 7f67b71d7640 1 --2- 172.21.15.70:0/4146032394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b01599d0 0x7f67b00cdcd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.328+0000 7f67b71d7640 1 -- 172.21.15.70:0/4146032394 >> 172.21.15.70:0/4146032394 conn(0x7f67b006c050 msgr2=0x7f67b006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:21.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.328+0000 7f67b71d7640 1 -- 172.21.15.70:0/4146032394 shutdown_connections 2023-10-08T05:37:21.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.328+0000 7f67b71d7640 1 -- 172.21.15.70:0/4146032394 wait complete. 2023-10-08T05:37:21.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.329+0000 7f67b71d7640 1 Processor -- start 2023-10-08T05:37:21.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.330+0000 7f67b71d7640 1 -- start start 2023-10-08T05:37:21.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b71d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67b00b3c50 0x7f67b00aa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b71d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b00cb410 0x7f67b00a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b71d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67b01599d0 0x7f67b00aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b71d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00adeb0 con 0x7f67b00cb410 2023-10-08T05:37:21.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b71d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00ae190 con 0x7f67b00b3c50 2023-10-08T05:37:21.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b71d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b00ae470 con 0x7f67b01599d0 2023-10-08T05:37:21.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b59d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b00cb410 0x7f67b00a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b59d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b00cb410 0x7f67b00a7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37274/0 (socket says 172.21.15.70:37274) 2023-10-08T05:37:21.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b59d4640 1 -- 172.21.15.70:0/1883329539 learned_addr learned my addr 172.21.15.70:0/1883329539 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:21.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b61d5640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67b00b3c50 0x7f67b00aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b59d4640 1 -- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67b01599d0 msgr2=0x7f67b00aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b69d6640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67b01599d0 0x7f67b00aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b59d4640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67b01599d0 0x7f67b00aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b59d4640 1 -- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67b00b3c50 msgr2=0x7f67b00aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b59d4640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67b00b3c50 0x7f67b00aa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.331+0000 7f67b59d4640 1 -- 172.21.15.70:0/1883329539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac00b770 con 0x7f67b00cb410 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.332+0000 7f67b59d4640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b00cb410 0x7f67b00a7d00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67a000b4a0 tx=0x7f67a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.332+0000 7f67a6ffd640 1 -- 172.21.15.70:0/1883329539 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0010610 con 0x7f67b00cb410 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.332+0000 7f67a6ffd640 1 -- 172.21.15.70:0/1883329539 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a0010770 con 0x7f67b00cb410 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.332+0000 7f67b71d7640 1 -- 172.21.15.70:0/1883329539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b00ae750 con 0x7f67b00cb410 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.332+0000 7f67a6ffd640 1 -- 172.21.15.70:0/1883329539 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0010b20 con 0x7f67b00cb410 2023-10-08T05:37:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.332+0000 7f67b71d7640 1 -- 172.21.15.70:0/1883329539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b00aaed0 con 0x7f67b00cb410 2023-10-08T05:37:21.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.333+0000 7f67b71d7640 1 -- 172.21.15.70:0/1883329539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b00aec70 con 0x7f67b00cb410 2023-10-08T05:37:21.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.335+0000 7f67a6ffd640 1 -- 172.21.15.70:0/1883329539 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f67a0010c80 con 0x7f67b00cb410 2023-10-08T05:37:21.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.335+0000 7f67a6ffd640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f679406ae70 0x7f679406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.336+0000 7f67a6ffd640 1 -- 172.21.15.70:0/1883329539 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f67a0089670 con 0x7f67b00cb410 2023-10-08T05:37:21.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.336+0000 7f67b61d5640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f679406ae70 0x7f679406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.337+0000 7f67b61d5640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f679406ae70 0x7f679406d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67ac038000 tx=0x7f67ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:21.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.341+0000 7f67a6ffd640 1 -- 172.21.15.70:0/1883329539 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67b00aec70 con 0x7f67b00cb410 2023-10-08T05:37:21.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.529+0000 7f67b71d7640 1 -- 172.21.15.70:0/1883329539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67b00aec70 con 0x7f67b00cb410 2023-10-08T05:37:21.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.529+0000 7f67a6ffd640 1 -- 172.21.15.70:0/1883329539 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67b00aec70 con 0x7f67b00cb410 2023-10-08T05:37:21.531 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:21.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.532+0000 7f67b71d7640 1 -- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f679406ae70 msgr2=0x7f679406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.532+0000 7f67b71d7640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f679406ae70 0x7f679406d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67ac038000 tx=0x7f67ac008040 comp rx=0 tx=0).stop 2023-10-08T05:37:21.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.532+0000 7f67b71d7640 1 -- 172.21.15.70:0/1883329539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b00cb410 msgr2=0x7f67b00a7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.532+0000 7f67b71d7640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b00cb410 0x7f67b00a7d00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f67a000b4a0 tx=0x7f67a000b960 comp rx=0 tx=0).stop 2023-10-08T05:37:21.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.532+0000 7f67b71d7640 1 -- 172.21.15.70:0/1883329539 shutdown_connections 2023-10-08T05:37:21.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.532+0000 7f67b71d7640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f679406ae70 0x7f679406d330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.532+0000 7f67b71d7640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67b01599d0 0x7f67b00aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.532+0000 7f67b71d7640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67b00cb410 0x7f67b00a7d00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.532+0000 7f67b71d7640 1 --2- 172.21.15.70:0/1883329539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67b00b3c50 0x7f67b00aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.532+0000 7f67b71d7640 1 -- 172.21.15.70:0/1883329539 >> 172.21.15.70:0/1883329539 conn(0x7f67b006c050 msgr2=0x7f67b015a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:21.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.533+0000 7f67b71d7640 1 -- 172.21.15.70:0/1883329539 shutdown_connections 2023-10-08T05:37:21.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.533+0000 7f67b71d7640 1 -- 172.21.15.70:0/1883329539 wait complete. 2023-10-08T05:37:21.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:37:21.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.772+0000 7fa9fa860640 1 Processor -- start 2023-10-08T05:37:21.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.772+0000 7fa9fa860640 1 -- start start 2023-10-08T05:37:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.773+0000 7fa9fa860640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9f415a820 0x7fa9f40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.773+0000 7fa9fa860640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f40ca680 0x7fa9f40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.773+0000 7fa9fa860640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f40cab60 con 0x7fa9f415ac00 2023-10-08T05:37:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.773+0000 7fa9fa860640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f40cb960 con 0x7fa9f415a820 2023-10-08T05:37:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.773+0000 7fa9fa860640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40cbdf0 con 0x7fa9f40ca680 2023-10-08T05:37:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.774+0000 7fa9f985e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9f415a820 0x7fa9f40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.774+0000 7fa9f905d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f40ca680 0x7fa9f40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.774+0000 7fa9f985e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9f415a820 0x7fa9f40aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:44078/0 (socket says 172.21.15.70:44078) 2023-10-08T05:37:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.774+0000 7fa9f985e640 1 -- 172.21.15.70:0/119374545 learned_addr learned my addr 172.21.15.70:0/119374545 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.775+0000 7fa9f985e640 1 -- 172.21.15.70:0/119374545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f40ca680 msgr2=0x7fa9f40b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.775+0000 7fa9f985e640 1 --2- 172.21.15.70:0/119374545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f40ca680 0x7fa9f40b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.775+0000 7fa9f985e640 1 -- 172.21.15.70:0/119374545 >> v1:172.21.15.181:6789/0 conn(0x7fa9f415ac00 legacy=0x7fa9f40c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.775+0000 7fa9f985e640 1 -- 172.21.15.70:0/119374545 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f40cb030 con 0x7fa9f415a820 2023-10-08T05:37:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.775+0000 7fa9f905d640 1 --2- 172.21.15.70:0/119374545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f40ca680 0x7fa9f40b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:37:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.776+0000 7fa9f985e640 1 --2- 172.21.15.70:0/119374545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f415a820 0x7fa9f40aff70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009870 tx=0x7fa9f002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=263a7674f784c755 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.776+0000 7fa9f885c640 1 -- 172.21.15.70:0/119374545 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0031640 con 0x7fa9f415a820 2023-10-08T05:37:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.776+0000 7fa9f885c640 1 -- 172.21.15.70:0/119374545 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f00317a0 con 0x7fa9f415a820 2023-10-08T05:37:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.776+0000 7fa9f885c640 1 -- 172.21.15.70:0/119374545 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0031b30 con 0x7fa9f415a820 2023-10-08T05:37:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.778+0000 7fa9fa860640 1 -- 172.21.15.70:0/119374545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f415a820 msgr2=0x7fa9f40aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.778+0000 7fa9fa860640 1 --2- 172.21.15.70:0/119374545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f415a820 0x7fa9f40aff70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009870 tx=0x7fa9f002b8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.778+0000 7fa9fa860640 1 -- 172.21.15.70:0/119374545 shutdown_connections 2023-10-08T05:37:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.778+0000 7fa9fa860640 1 --2- 172.21.15.70:0/119374545 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f40ca680 0x7fa9f40b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.778+0000 7fa9fa860640 1 --2- 172.21.15.70:0/119374545 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f415a820 0x7fa9f40aff70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.778+0000 7fa9fa860640 1 -- 172.21.15.70:0/119374545 >> 172.21.15.70:0/119374545 conn(0x7fa9f406c050 msgr2=0x7fa9f406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.778+0000 7fa9fa860640 1 -- 172.21.15.70:0/119374545 shutdown_connections 2023-10-08T05:37:21.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.778+0000 7fa9fa860640 1 -- 172.21.15.70:0/119374545 wait complete. 2023-10-08T05:37:21.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.780+0000 7fa9fa860640 1 Processor -- start 2023-10-08T05:37:21.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.780+0000 7fa9fa860640 1 -- start start 2023-10-08T05:37:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.781+0000 7fa9fa860640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f40ca680 0x7fa9f41569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.781+0000 7fa9fa860640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9f415a820 0x7fa9f4157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.781+0000 7fa9fa860640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f415ac00 0x7fa9f4157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.781+0000 7fa9fa860640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f41f2cf0 con 0x7fa9f415a820 2023-10-08T05:37:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.781+0000 7fa9fa860640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40ab8d0 con 0x7fa9f40ca680 2023-10-08T05:37:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.781+0000 7fa9fa860640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f41f2070 con 0x7fa9f415ac00 2023-10-08T05:37:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9f905d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9f415a820 0x7fa9f4157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9f905d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9f415a820 0x7fa9f4157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37290/0 (socket says 172.21.15.70:37290) 2023-10-08T05:37:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9f905d640 1 -- 172.21.15.70:0/1580897145 learned_addr learned my addr 172.21.15.70:0/1580897145 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9fa05f640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f415ac00 0x7fa9f4157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9f985e640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f40ca680 0x7fa9f41569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9f905d640 1 -- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f415ac00 msgr2=0x7fa9f4157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9f905d640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f415ac00 0x7fa9f4157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9f905d640 1 -- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f40ca680 msgr2=0x7fa9f41569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9f905d640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f40ca680 0x7fa9f41569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9f905d640 1 -- 172.21.15.70:0/1580897145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e4009400 con 0x7fa9f415a820 2023-10-08T05:37:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.782+0000 7fa9f905d640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9f415a820 0x7fa9f4157980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400e960 tx=0x7fa9e400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.783+0000 7fa9ea7fc640 1 -- 172.21.15.70:0/1580897145 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e400ca00 con 0x7fa9f415a820 2023-10-08T05:37:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.783+0000 7fa9ea7fc640 1 -- 172.21.15.70:0/1580897145 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e400cb60 con 0x7fa9f415a820 2023-10-08T05:37:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.783+0000 7fa9fa860640 1 -- 172.21.15.70:0/1580897145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f00094d0 con 0x7fa9f415a820 2023-10-08T05:37:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.783+0000 7fa9ea7fc640 1 -- 172.21.15.70:0/1580897145 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4013470 con 0x7fa9f415a820 2023-10-08T05:37:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.783+0000 7fa9fa860640 1 -- 172.21.15.70:0/1580897145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f41f2630 con 0x7fa9f415a820 2023-10-08T05:37:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.784+0000 7fa9fa860640 1 -- 172.21.15.70:0/1580897145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f4074150 con 0x7fa9f415a820 2023-10-08T05:37:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.785+0000 7fa9ea7fc640 1 -- 172.21.15.70:0/1580897145 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa9e40135d0 con 0x7fa9f415a820 2023-10-08T05:37:21.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.786+0000 7fa9ea7fc640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa9d806ae70 0x7fa9d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:21.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.787+0000 7fa9ea7fc640 1 -- 172.21.15.70:0/1580897145 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa9e4089bf0 con 0x7fa9f415a820 2023-10-08T05:37:21.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.787+0000 7fa9f985e640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa9d806ae70 0x7fa9d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:21.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.788+0000 7fa9f985e640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa9d806ae70 0x7fa9d806d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0036000 tx=0x7fa9f003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:21.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.792+0000 7fa9ea7fc640 1 -- 172.21.15.70:0/1580897145 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa9e40548f0 con 0x7fa9f415a820 2023-10-08T05:37:21.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.980+0000 7fa9fa860640 1 -- 172.21.15.70:0/1580897145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9f4159e00 con 0x7fa9f415a820 2023-10-08T05:37:21.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.981+0000 7fa9ea7fc640 1 -- 172.21.15.70:0/1580897145 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa9e40170a0 con 0x7fa9f415a820 2023-10-08T05:37:21.982 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:37:21.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 -- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa9d806ae70 msgr2=0x7fa9d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa9d806ae70 0x7fa9d806d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0036000 tx=0x7fa9f003c040 comp rx=0 tx=0).stop 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 -- 172.21.15.70:0/1580897145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9f415a820 msgr2=0x7fa9f4157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9f415a820 0x7fa9f4157980 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400e960 tx=0x7fa9e400ee20 comp rx=0 tx=0).stop 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 -- 172.21.15.70:0/1580897145 shutdown_connections 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa9d806ae70 0x7fa9d806d330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9f415ac00 0x7fa9f4157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9f415a820 0x7fa9f4157980 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 --2- 172.21.15.70:0/1580897145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9f40ca680 0x7fa9f41569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 -- 172.21.15.70:0/1580897145 >> 172.21.15.70:0/1580897145 conn(0x7fa9f406c050 msgr2=0x7fa9f415d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 -- 172.21.15.70:0/1580897145 shutdown_connections 2023-10-08T05:37:21.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:21.984+0000 7fa9fa860640 1 -- 172.21.15.70:0/1580897145 wait complete. 2023-10-08T05:37:22.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:37:22.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.221+0000 7f2378f73640 1 Processor -- start 2023-10-08T05:37:22.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.222+0000 7f2378f73640 1 -- start start 2023-10-08T05:37:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.223+0000 7f2378f73640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f23740afbe0 0x7f23740b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.223+0000 7f2378f73640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f237406dfc0 0x7f2374070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.223+0000 7f2378f73640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740b2620 con 0x7f2374071070 2023-10-08T05:37:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.223+0000 7f2378f73640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23740b2060 con 0x7f23740afbe0 2023-10-08T05:37:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.223+0000 7f2378f73640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740b2340 con 0x7f237406dfc0 2023-10-08T05:37:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.223+0000 7f237259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f23740afbe0 0x7f23740b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.224+0000 7f237259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f23740afbe0 0x7f23740b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37298/0 (socket says 172.21.15.70:37298) 2023-10-08T05:37:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.224+0000 7f237259c640 1 -- 172.21.15.70:0/3488213557 learned_addr learned my addr 172.21.15.70:0/3488213557 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.224+0000 7f237259c640 1 -- 172.21.15.70:0/3488213557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f237406dfc0 msgr2=0x7f2374070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.224+0000 7f237259c640 1 --2- 172.21.15.70:0/3488213557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f237406dfc0 0x7f2374070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.224+0000 7f237259c640 1 -- 172.21.15.70:0/3488213557 >> v1:172.21.15.181:6789/0 conn(0x7f2374071070 legacy=0x7f23740cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.224+0000 7f237259c640 1 -- 172.21.15.70:0/3488213557 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2374070350 con 0x7f23740afbe0 2023-10-08T05:37:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.225+0000 7f237259c640 1 --2- 172.21.15.70:0/3488213557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23740afbe0 0x7f23740b15b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2364009870 tx=0x7f236402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61f654795ea9b5a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.225+0000 7f2371d9b640 1 -- 172.21.15.70:0/3488213557 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364031640 con 0x7f23740afbe0 2023-10-08T05:37:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.225+0000 7f2371d9b640 1 -- 172.21.15.70:0/3488213557 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23640317a0 con 0x7f23740afbe0 2023-10-08T05:37:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.225+0000 7f2371d9b640 1 -- 172.21.15.70:0/3488213557 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364031b30 con 0x7f23740afbe0 2023-10-08T05:37:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.226+0000 7f2378f73640 1 -- 172.21.15.70:0/3488213557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23740afbe0 msgr2=0x7f23740b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.226+0000 7f2378f73640 1 --2- 172.21.15.70:0/3488213557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23740afbe0 0x7f23740b15b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2364009870 tx=0x7f236402b8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.226+0000 7f2378f73640 1 -- 172.21.15.70:0/3488213557 shutdown_connections 2023-10-08T05:37:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.226+0000 7f2378f73640 1 --2- 172.21.15.70:0/3488213557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f237406dfc0 0x7f2374070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.226+0000 7f2378f73640 1 --2- 172.21.15.70:0/3488213557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23740afbe0 0x7f23740b15b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.226+0000 7f2378f73640 1 -- 172.21.15.70:0/3488213557 >> 172.21.15.70:0/3488213557 conn(0x7f237406a5d0 msgr2=0x7f23740b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.226+0000 7f2378f73640 1 -- 172.21.15.70:0/3488213557 shutdown_connections 2023-10-08T05:37:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.226+0000 7f2378f73640 1 -- 172.21.15.70:0/3488213557 wait complete. 2023-10-08T05:37:22.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.228+0000 7f2378f73640 1 Processor -- start 2023-10-08T05:37:22.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.228+0000 7f2378f73640 1 -- start start 2023-10-08T05:37:22.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.229+0000 7f2378f73640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f237406dfc0 0x7f237415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.229+0000 7f2378f73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2374071070 0x7f2374144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.229+0000 7f2378f73640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23740afbe0 0x7f23740c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.229+0000 7f2378f73640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c44d0 con 0x7f2374071070 2023-10-08T05:37:22.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.229+0000 7f2378f73640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c35e0 con 0x7f237406dfc0 2023-10-08T05:37:22.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.229+0000 7f2378f73640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c38c0 con 0x7f23740afbe0 2023-10-08T05:37:22.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.229+0000 7f236bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2374071070 0x7f2374144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:22.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.229+0000 7f236bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2374071070 0x7f2374144910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37306/0 (socket says 172.21.15.70:37306) 2023-10-08T05:37:22.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.229+0000 7f236bfff640 1 -- 172.21.15.70:0/2289249101 learned_addr learned my addr 172.21.15.70:0/2289249101 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:22.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f237259c640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f237406dfc0 0x7f237415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f236bfff640 1 -- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23740afbe0 msgr2=0x7f23740c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f2372d9d640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23740afbe0 0x7f23740c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f236bfff640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23740afbe0 0x7f23740c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f236bfff640 1 -- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f237406dfc0 msgr2=0x7f237415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f236bfff640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f237406dfc0 0x7f237415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f236bfff640 1 -- 172.21.15.70:0/2289249101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23640094d0 con 0x7f2374071070 2023-10-08T05:37:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f2372d9d640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23740afbe0 0x7f23740c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f237259c640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f237406dfc0 0x7f237415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f236bfff640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2374071070 0x7f2374144910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f235c00b410 tx=0x7f235c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:22.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f236affd640 1 -- 172.21.15.70:0/2289249101 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c011830 con 0x7f2374071070 2023-10-08T05:37:22.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f236affd640 1 -- 172.21.15.70:0/2289249101 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c011990 con 0x7f2374071070 2023-10-08T05:37:22.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f236affd640 1 -- 172.21.15.70:0/2289249101 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c011d40 con 0x7f2374071070 2023-10-08T05:37:22.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.230+0000 7f2378f73640 1 -- 172.21.15.70:0/2289249101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23740c3ba0 con 0x7f2374071070 2023-10-08T05:37:22.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.231+0000 7f2378f73640 1 -- 172.21.15.70:0/2289249101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23740c4160 con 0x7f2374071070 2023-10-08T05:37:22.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.232+0000 7f2378f73640 1 -- 172.21.15.70:0/2289249101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237404e160 con 0x7f2374071070 2023-10-08T05:37:22.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.233+0000 7f236affd640 1 -- 172.21.15.70:0/2289249101 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f235c01a400 con 0x7f2374071070 2023-10-08T05:37:22.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.234+0000 7f236affd640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f235406ae70 0x7f235406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.234+0000 7f236affd640 1 -- 172.21.15.70:0/2289249101 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f235c089c30 con 0x7f2374071070 2023-10-08T05:37:22.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.234+0000 7f237259c640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f235406ae70 0x7f235406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:22.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.235+0000 7f237259c640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f235406ae70 0x7f235406d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2364036000 tx=0x7f236403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:22.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.238+0000 7f236affd640 1 -- 172.21.15.70:0/2289249101 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f235c054880 con 0x7f2374071070 2023-10-08T05:37:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.427+0000 7f2378f73640 1 -- 172.21.15.70:0/2289249101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2374070600 con 0x7f2374071070 2023-10-08T05:37:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.428+0000 7f236affd640 1 -- 172.21.15.70:0/2289249101 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f235c015020 con 0x7f2374071070 2023-10-08T05:37:22.430 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:22.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 -- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f235406ae70 msgr2=0x7f235406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f235406ae70 0x7f235406d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2364036000 tx=0x7f236403c040 comp rx=0 tx=0).stop 2023-10-08T05:37:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 -- 172.21.15.70:0/2289249101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2374071070 msgr2=0x7f2374144910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2374071070 0x7f2374144910 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f235c00b410 tx=0x7f235c00b8d0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 -- 172.21.15.70:0/2289249101 shutdown_connections 2023-10-08T05:37:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f235406ae70 0x7f235406d330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23740afbe0 0x7f23740c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2374071070 0x7f2374144910 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 --2- 172.21.15.70:0/2289249101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f237406dfc0 0x7f237415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 -- 172.21.15.70:0/2289249101 >> 172.21.15.70:0/2289249101 conn(0x7f237406a5d0 msgr2=0x7f23740c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 -- 172.21.15.70:0/2289249101 shutdown_connections 2023-10-08T05:37:22.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.431+0000 7f2378f73640 1 -- 172.21.15.70:0/2289249101 wait complete. 2023-10-08T05:37:22.451 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:37:22.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.660+0000 7f17217b0640 1 Processor -- start 2023-10-08T05:37:22.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.660+0000 7f17217b0640 1 -- start start 2023-10-08T05:37:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.662+0000 7f17217b0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f171c0c9cf0 0x7f171c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.662+0000 7f17217b0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c15b310 con 0x7f171c06c3a0 2023-10-08T05:37:22.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.662+0000 7f17217b0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c06d290 con 0x7f171c15aa00 2023-10-08T05:37:22.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.662+0000 7f17217b0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c06e0a0 con 0x7f171c0c9cf0 2023-10-08T05:37:22.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.662+0000 7f171affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f171c06c3a0 0x7f171c06c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50086/0 (socket says 172.21.15.70:50086) 2023-10-08T05:37:22.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.662+0000 7f171affd640 1 -- 172.21.15.70:0/1522300443 learned_addr learned my addr 172.21.15.70:0/1522300443 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:22.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.662+0000 7f17127fc640 1 --2- 172.21.15.70:0/1522300443 >> v2:172.21.15.70:3300/0 conn(0x7f171c0c9cf0 0x7f171c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:22.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.663+0000 7f171a7fc640 1 -- 172.21.15.70:0/1522300443 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352126341 0 0) 0x7f170c001110 con 0x7f171c06c3a0 2023-10-08T05:37:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.663+0000 7f171a7fc640 1 -- 172.21.15.70:0/1522300443 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1708003520 con 0x7f171c06c3a0 2023-10-08T05:37:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.663+0000 7f17127fc640 1 -- 172.21.15.70:0/1522300443 >> v1:172.21.15.181:6789/0 conn(0x7f171c15aa00 legacy=0x7f171c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.663+0000 7f17127fc640 1 -- 172.21.15.70:0/1522300443 >> v1:172.21.15.70:6789/0 conn(0x7f171c06c3a0 legacy=0x7f171c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.663+0000 7f17127fc640 1 -- 172.21.15.70:0/1522300443 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c0c9860 con 0x7f171c0c9cf0 2023-10-08T05:37:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.664+0000 7f17127fc640 1 --2- 172.21.15.70:0/1522300443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c0c9cf0 0x7f171c06db60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1704009d90 tx=0x7f170402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a70ecfdadec4143 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.664+0000 7f171a7fc640 1 -- 172.21.15.70:0/1522300443 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170402cd00 con 0x7f171c0c9cf0 2023-10-08T05:37:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.664+0000 7f171a7fc640 1 -- 172.21.15.70:0/1522300443 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170402ce60 con 0x7f171c0c9cf0 2023-10-08T05:37:22.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.664+0000 7f171a7fc640 1 -- 172.21.15.70:0/1522300443 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704034640 con 0x7f171c0c9cf0 2023-10-08T05:37:22.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.665+0000 7f17217b0640 1 -- 172.21.15.70:0/1522300443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c0c9cf0 msgr2=0x7f171c06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.665+0000 7f17217b0640 1 --2- 172.21.15.70:0/1522300443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c0c9cf0 0x7f171c06db60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1704009d90 tx=0x7f170402bf50 comp rx=0 tx=0).stop 2023-10-08T05:37:22.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.665+0000 7f17217b0640 1 -- 172.21.15.70:0/1522300443 shutdown_connections 2023-10-08T05:37:22.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.665+0000 7f17217b0640 1 --2- 172.21.15.70:0/1522300443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c0c9cf0 0x7f171c06db60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.665+0000 7f17217b0640 1 -- 172.21.15.70:0/1522300443 >> 172.21.15.70:0/1522300443 conn(0x7f171c06a5d0 msgr2=0x7f171c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:22.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.665+0000 7f17217b0640 1 -- 172.21.15.70:0/1522300443 shutdown_connections 2023-10-08T05:37:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.665+0000 7f17217b0640 1 -- 172.21.15.70:0/1522300443 wait complete. 2023-10-08T05:37:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.667+0000 7f17217b0640 1 Processor -- start 2023-10-08T05:37:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.667+0000 7f17217b0640 1 -- start start 2023-10-08T05:37:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f17217b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171c06c3a0 0x7f171c150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f17217b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f171c0c9cf0 0x7f171c157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f17217b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c15aa00 0x7f171c0c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f17217b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c157ff0 con 0x7f171c15aa00 2023-10-08T05:37:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f17217b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c1582d0 con 0x7f171c06c3a0 2023-10-08T05:37:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f17217b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c1585b0 con 0x7f171c0c9cf0 2023-10-08T05:37:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f171b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c15aa00 0x7f171c0c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f171b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c15aa00 0x7f171c0c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37334/0 (socket says 172.21.15.70:37334) 2023-10-08T05:37:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f171b7fe640 1 -- 172.21.15.70:0/3052475266 learned_addr learned my addr 172.21.15.70:0/3052475266 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f171affd640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171c06c3a0 0x7f171c150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f171b7fe640 1 -- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f171c0c9cf0 msgr2=0x7f171c157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f171b7fe640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f171c0c9cf0 0x7f171c157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f171b7fe640 1 -- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171c06c3a0 msgr2=0x7f171c150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f171b7fe640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171c06c3a0 0x7f171c150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f171b7fe640 1 -- 172.21.15.70:0/3052475266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1704009a70 con 0x7f171c15aa00 2023-10-08T05:37:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.668+0000 7f171affd640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171c06c3a0 0x7f171c150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.669+0000 7f171b7fe640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c15aa00 0x7f171c0c2da0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f171400c390 tx=0x7f171400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.669+0000 7f1713fff640 1 -- 172.21.15.70:0/3052475266 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140052d0 con 0x7f171c15aa00 2023-10-08T05:37:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.669+0000 7f1713fff640 1 -- 172.21.15.70:0/3052475266 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714005430 con 0x7f171c15aa00 2023-10-08T05:37:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.669+0000 7f17217b0640 1 -- 172.21.15.70:0/3052475266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c158890 con 0x7f171c15aa00 2023-10-08T05:37:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.669+0000 7f1713fff640 1 -- 172.21.15.70:0/3052475266 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714003810 con 0x7f171c15aa00 2023-10-08T05:37:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.669+0000 7f17217b0640 1 -- 172.21.15.70:0/3052475266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c151520 con 0x7f171c15aa00 2023-10-08T05:37:22.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.670+0000 7f17217b0640 1 -- 172.21.15.70:0/3052475266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c0caf40 con 0x7f171c15aa00 2023-10-08T05:37:22.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.671+0000 7f1713fff640 1 -- 172.21.15.70:0/3052475266 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1714003970 con 0x7f171c15aa00 2023-10-08T05:37:22.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.672+0000 7f1713fff640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f16fc06ae70 0x7f16fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:22.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.672+0000 7f1713fff640 1 -- 172.21.15.70:0/3052475266 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f1714088e60 con 0x7f171c15aa00 2023-10-08T05:37:22.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.673+0000 7f171affd640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f16fc06ae70 0x7f16fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:22.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.674+0000 7f171affd640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f16fc06ae70 0x7f16fc06d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f170c001370 tx=0x7f170c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:22.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.676+0000 7f1713fff640 1 -- 172.21.15.70:0/3052475266 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1714053be0 con 0x7f171c15aa00 2023-10-08T05:37:22.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.876+0000 7f17217b0640 1 -- 172.21.15.70:0/3052475266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f171c0c2bc0 con 0x7f171c15aa00 2023-10-08T05:37:22.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.877+0000 7f1713fff640 1 -- 172.21.15.70:0/3052475266 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1714017080 con 0x7f171c15aa00 2023-10-08T05:37:22.877 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-10-08T05:37:22.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 -- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f16fc06ae70 msgr2=0x7f16fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f16fc06ae70 0x7f16fc06d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f170c001370 tx=0x7f170c009040 comp rx=0 tx=0).stop 2023-10-08T05:37:22.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 -- 172.21.15.70:0/3052475266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c15aa00 msgr2=0x7f171c0c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:22.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c15aa00 0x7f171c0c2da0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f171400c390 tx=0x7f171400c850 comp rx=0 tx=0).stop 2023-10-08T05:37:22.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 -- 172.21.15.70:0/3052475266 shutdown_connections 2023-10-08T05:37:22.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f16fc06ae70 0x7f16fc06d330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f171c15aa00 0x7f171c0c2da0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f171c0c9cf0 0x7f171c157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 --2- 172.21.15.70:0/3052475266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f171c06c3a0 0x7f171c150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:22.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 -- 172.21.15.70:0/3052475266 >> 172.21.15.70:0/3052475266 conn(0x7f171c06a5d0 msgr2=0x7f171c0b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:22.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 -- 172.21.15.70:0/3052475266 shutdown_connections 2023-10-08T05:37:22.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:22.879+0000 7f17217b0640 1 -- 172.21.15.70:0/3052475266 wait complete. 2023-10-08T05:37:22.898 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-08T05:37:22.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:22.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-08T05:37:23.005 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:37:23.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:23.005 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T05:37:23.055 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:37:23.055 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:23.116 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:37:23.116 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:23.166 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:37:23.166 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:23.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.439+0000 7f837bae5640 1 Processor -- start 2023-10-08T05:37:23.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.439+0000 7f837bae5640 1 -- start start 2023-10-08T05:37:23.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.441+0000 7f837bae5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8374159ac0 0x7f837406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.441+0000 7f837bae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83740709e0 0x7f83740c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.441+0000 7f837bae5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837406dee0 con 0x7f83740ad790 2023-10-08T05:37:23.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.441+0000 7f837bae5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837415d850 con 0x7f8374159ac0 2023-10-08T05:37:23.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.441+0000 7f837bae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837415db30 con 0x7f83740709e0 2023-10-08T05:37:23.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.441+0000 7f8379881640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8374159ac0 0x7f837406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.441+0000 7f8379881640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8374159ac0 0x7f837406d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37336/0 (socket says 172.21.15.70:37336) 2023-10-08T05:37:23.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.441+0000 7f8379881640 1 -- 172.21.15.70:0/3199176434 learned_addr learned my addr 172.21.15.70:0/3199176434 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:23.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.441+0000 7f8379080640 1 --1- 172.21.15.70:0/3199176434 >> v1:172.21.15.70:6789/0 conn(0x7f83740ad790 0x7f83740702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50092/0 (socket says 172.21.15.70:50092) 2023-10-08T05:37:23.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.442+0000 7f8379881640 1 -- 172.21.15.70:0/3199176434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83740709e0 msgr2=0x7f83740c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:23.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.442+0000 7f837a082640 1 --2- 172.21.15.70:0/3199176434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83740709e0 0x7f83740c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.442+0000 7f8379881640 1 --2- 172.21.15.70:0/3199176434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83740709e0 0x7f83740c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.442+0000 7f8379881640 1 -- 172.21.15.70:0/3199176434 >> v1:172.21.15.70:6789/0 conn(0x7f83740ad790 legacy=0x7f83740702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:23.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.442+0000 7f8379881640 1 -- 172.21.15.70:0/3199176434 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837415de10 con 0x7f8374159ac0 2023-10-08T05:37:23.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.442+0000 7f837887f640 1 -- 172.21.15.70:0/3199176434 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440328325 0 0) 0x7f837406dee0 con 0x7f83740ad790 2023-10-08T05:37:23.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.442+0000 7f837a082640 1 --2- 172.21.15.70:0/3199176434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83740709e0 0x7f83740c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:23.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.443+0000 7f8379881640 1 --2- 172.21.15.70:0/3199176434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8374159ac0 0x7f837406d5e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8370009870 tx=0x7f837002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f1a6591ffb518a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:23.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.443+0000 7f837887f640 1 -- 172.21.15.70:0/3199176434 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370030840 con 0x7f8374159ac0 2023-10-08T05:37:23.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.443+0000 7f837887f640 1 -- 172.21.15.70:0/3199176434 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83700309a0 con 0x7f8374159ac0 2023-10-08T05:37:23.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.443+0000 7f837887f640 1 -- 172.21.15.70:0/3199176434 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370030d30 con 0x7f8374159ac0 2023-10-08T05:37:23.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.444+0000 7f837bae5640 1 -- 172.21.15.70:0/3199176434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8374159ac0 msgr2=0x7f837406d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:23.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.444+0000 7f837bae5640 1 --2- 172.21.15.70:0/3199176434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8374159ac0 0x7f837406d5e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8370009870 tx=0x7f837002b980 comp rx=0 tx=0).stop 2023-10-08T05:37:23.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.444+0000 7f837bae5640 1 -- 172.21.15.70:0/3199176434 shutdown_connections 2023-10-08T05:37:23.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.444+0000 7f837bae5640 1 --2- 172.21.15.70:0/3199176434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83740709e0 0x7f83740c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.444+0000 7f837bae5640 1 --2- 172.21.15.70:0/3199176434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8374159ac0 0x7f837406d5e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.444+0000 7f837bae5640 1 -- 172.21.15.70:0/3199176434 >> 172.21.15.70:0/3199176434 conn(0x7f837406a380 msgr2=0x7f83740b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:23.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.444+0000 7f837bae5640 1 -- 172.21.15.70:0/3199176434 shutdown_connections 2023-10-08T05:37:23.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.444+0000 7f837bae5640 1 -- 172.21.15.70:0/3199176434 wait complete. 2023-10-08T05:37:23.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.446+0000 7f837bae5640 1 Processor -- start 2023-10-08T05:37:23.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.446+0000 7f837bae5640 1 -- start start 2023-10-08T05:37:23.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.447+0000 7f837bae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83740709e0 0x7f837414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.447+0000 7f837bae5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83740ad790 0x7f837415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.447+0000 7f837bae5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8374159ac0 0x7f837415cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.447+0000 7f837bae5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b3a60 con 0x7f83740ad790 2023-10-08T05:37:23.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.447+0000 7f837bae5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83740b3780 con 0x7f8374159ac0 2023-10-08T05:37:23.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.447+0000 7f837bae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837415a150 con 0x7f83740709e0 2023-10-08T05:37:23.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.447+0000 7f8379080640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83740ad790 0x7f837415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.447+0000 7f8379080640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83740ad790 0x7f837415c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37342/0 (socket says 172.21.15.70:37342) 2023-10-08T05:37:23.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.447+0000 7f8379080640 1 -- 172.21.15.70:0/4089907282 learned_addr learned my addr 172.21.15.70:0/4089907282 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:23.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f8379080640 1 -- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83740709e0 msgr2=0x7f837414e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:23.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f8379080640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83740709e0 0x7f837414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f8379080640 1 -- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8374159ac0 msgr2=0x7f837415cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:23.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f8379080640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8374159ac0 0x7f837415cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f8379080640 1 -- 172.21.15.70:0/4089907282 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83700094d0 con 0x7f83740ad790 2023-10-08T05:37:23.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f8379080640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83740ad790 0x7f837415c5e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f836400c4b0 tx=0x7f836400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:23.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f836a7fc640 1 -- 172.21.15.70:0/4089907282 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364007480 con 0x7f83740ad790 2023-10-08T05:37:23.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f836a7fc640 1 -- 172.21.15.70:0/4089907282 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83640075e0 con 0x7f83740ad790 2023-10-08T05:37:23.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f836a7fc640 1 -- 172.21.15.70:0/4089907282 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364004a20 con 0x7f83740ad790 2023-10-08T05:37:23.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f837bae5640 1 -- 172.21.15.70:0/4089907282 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837415a430 con 0x7f83740ad790 2023-10-08T05:37:23.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.448+0000 7f837bae5640 1 -- 172.21.15.70:0/4089907282 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837414e690 con 0x7f83740ad790 2023-10-08T05:37:23.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.449+0000 7f837bae5640 1 -- 172.21.15.70:0/4089907282 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837415a950 con 0x7f83740ad790 2023-10-08T05:37:23.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.450+0000 7f836a7fc640 1 -- 172.21.15.70:0/4089907282 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8364004b80 con 0x7f83740ad790 2023-10-08T05:37:23.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.451+0000 7f836a7fc640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f835806ae70 0x7f835806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.451+0000 7f836a7fc640 1 -- 172.21.15.70:0/4089907282 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f8364088c00 con 0x7f83740ad790 2023-10-08T05:37:23.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.452+0000 7f8379881640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f835806ae70 0x7f835806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.453+0000 7f8379881640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f835806ae70 0x7f835806d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8370036000 tx=0x7f837003b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:23.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.455+0000 7f836a7fc640 1 -- 172.21.15.70:0/4089907282 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f837415a950 con 0x7f83740ad790 2023-10-08T05:37:23.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.646+0000 7f837bae5640 1 -- 172.21.15.70:0/4089907282 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f837415a950 con 0x7f83740ad790 2023-10-08T05:37:23.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.648+0000 7f836a7fc640 1 -- 172.21.15.70:0/4089907282 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7f837415a950 con 0x7f83740ad790 2023-10-08T05:37:23.649 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:23.649 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.650+0000 7f837bae5640 1 -- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f835806ae70 msgr2=0x7f835806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.650+0000 7f837bae5640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f835806ae70 0x7f835806d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8370036000 tx=0x7f837003b040 comp rx=0 tx=0).stop 2023-10-08T05:37:23.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.650+0000 7f837bae5640 1 -- 172.21.15.70:0/4089907282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83740ad790 msgr2=0x7f837415c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:23.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.650+0000 7f837bae5640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83740ad790 0x7f837415c5e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f836400c4b0 tx=0x7f836400c970 comp rx=0 tx=0).stop 2023-10-08T05:37:23.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.651+0000 7f837bae5640 1 -- 172.21.15.70:0/4089907282 shutdown_connections 2023-10-08T05:37:23.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.651+0000 7f837bae5640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f835806ae70 0x7f835806d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.651+0000 7f837bae5640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8374159ac0 0x7f837415cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.651+0000 7f837bae5640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83740ad790 0x7f837415c5e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.651+0000 7f837bae5640 1 --2- 172.21.15.70:0/4089907282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f83740709e0 0x7f837414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.651+0000 7f837bae5640 1 -- 172.21.15.70:0/4089907282 >> 172.21.15.70:0/4089907282 conn(0x7f837406a380 msgr2=0x7f8374073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:23.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.651+0000 7f837bae5640 1 -- 172.21.15.70:0/4089907282 shutdown_connections 2023-10-08T05:37:23.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.651+0000 7f837bae5640 1 -- 172.21.15.70:0/4089907282 wait complete. 2023-10-08T05:37:23.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:37:23.672 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:23.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.892+0000 7f6af53a9640 1 Processor -- start 2023-10-08T05:37:23.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.892+0000 7f6af53a9640 1 -- start start 2023-10-08T05:37:23.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.894+0000 7f6af53a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6af015aad0 0x7f6af00b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.894+0000 7f6af53a9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6af00c82b0 0x7f6af00c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.894+0000 7f6af53a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6af006cd00 0x7f6af006d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.894+0000 7f6af53a9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af006d6d0 con 0x7f6af006cd00 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.894+0000 7f6af53a9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af015bf30 con 0x7f6af00c82b0 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.894+0000 7f6af53a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af015b320 con 0x7f6af015aad0 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.895+0000 7f6aef7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6af006cd00 0x7f6af006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.895+0000 7f6aef7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6af006cd00 0x7f6af006d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55402/0 (socket says 172.21.15.70:55402) 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.895+0000 7f6aef7fe640 1 -- 172.21.15.70:0/1114196100 learned_addr learned my addr 172.21.15.70:0/1114196100 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.895+0000 7f6aeeffd640 1 --2- 172.21.15.70:0/1114196100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6af015aad0 0x7f6af00b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.895+0000 7f6ae67fc640 1 --2- 172.21.15.70:0/1114196100 >> v2:172.21.15.181:3300/0 conn(0x7f6af00c82b0 0x7f6af00c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.895+0000 7f6aef7fe640 1 -- 172.21.15.70:0/1114196100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6af015aad0 msgr2=0x7f6af00b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.895+0000 7f6aef7fe640 1 --2- 172.21.15.70:0/1114196100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6af015aad0 0x7f6af00b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.895+0000 7f6aef7fe640 1 -- 172.21.15.70:0/1114196100 >> v2:172.21.15.181:3300/0 conn(0x7f6af00c82b0 msgr2=0x7f6af00c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:23.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.895+0000 7f6aef7fe640 1 --2- 172.21.15.70:0/1114196100 >> v2:172.21.15.181:3300/0 conn(0x7f6af00c82b0 0x7f6af00c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.895+0000 7f6aef7fe640 1 -- 172.21.15.70:0/1114196100 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af015b600 con 0x7f6af006cd00 2023-10-08T05:37:23.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.896+0000 7f6aef7fe640 1 --2- 172.21.15.70:0/1114196100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af006cd00 0x7f6af006d190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6af00cc300 tx=0x7f6ae802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422a06afa19307b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:23.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.896+0000 7f6aee7fc640 1 -- 172.21.15.70:0/1114196100 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae802ca70 con 0x7f6af006cd00 2023-10-08T05:37:23.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.896+0000 7f6aee7fc640 1 -- 172.21.15.70:0/1114196100 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae802cbd0 con 0x7f6af006cd00 2023-10-08T05:37:23.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.896+0000 7f6aee7fc640 1 -- 172.21.15.70:0/1114196100 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80343f0 con 0x7f6af006cd00 2023-10-08T05:37:23.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.897+0000 7f6af53a9640 1 -- 172.21.15.70:0/1114196100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af006cd00 msgr2=0x7f6af006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:23.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.897+0000 7f6af53a9640 1 --2- 172.21.15.70:0/1114196100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af006cd00 0x7f6af006d190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6af00cc300 tx=0x7f6ae802bed0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.897+0000 7f6af53a9640 1 -- 172.21.15.70:0/1114196100 shutdown_connections 2023-10-08T05:37:23.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.897+0000 7f6af53a9640 1 --2- 172.21.15.70:0/1114196100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af006cd00 0x7f6af006d190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.897+0000 7f6af53a9640 1 --2- 172.21.15.70:0/1114196100 >> v2:172.21.15.181:3300/0 conn(0x7f6af00c82b0 0x7f6af00c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.897+0000 7f6af53a9640 1 --2- 172.21.15.70:0/1114196100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6af015aad0 0x7f6af00b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.897+0000 7f6af53a9640 1 -- 172.21.15.70:0/1114196100 >> 172.21.15.70:0/1114196100 conn(0x7f6af0069d50 msgr2=0x7f6af006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:23.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.897+0000 7f6af53a9640 1 -- 172.21.15.70:0/1114196100 shutdown_connections 2023-10-08T05:37:23.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.897+0000 7f6af53a9640 1 -- 172.21.15.70:0/1114196100 wait complete. 2023-10-08T05:37:23.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.899+0000 7f6af53a9640 1 Processor -- start 2023-10-08T05:37:23.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.899+0000 7f6af53a9640 1 -- start start 2023-10-08T05:37:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.900+0000 7f6af53a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6af006cd00 0x7f6af01e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.900+0000 7f6af53a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af00c82b0 0x7f6af00c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.900+0000 7f6af53a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6af015aad0 0x7f6af00c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.900+0000 7f6af53a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01e7310 con 0x7f6af00c82b0 2023-10-08T05:37:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.900+0000 7f6af53a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af01e65e0 con 0x7f6af015aad0 2023-10-08T05:37:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.900+0000 7f6af53a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00c6550 con 0x7f6af006cd00 2023-10-08T05:37:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.900+0000 7f6ae67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af00c82b0 0x7f6af00c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.900+0000 7f6ae67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af00c82b0 0x7f6af00c38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55418/0 (socket says 172.21.15.70:55418) 2023-10-08T05:37:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6ae67fc640 1 -- 172.21.15.70:0/3950751884 learned_addr learned my addr 172.21.15.70:0/3950751884 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:23.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6aef7fe640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6af015aad0 0x7f6af00c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6aeeffd640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6af006cd00 0x7f6af01e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6ae67fc640 1 -- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6af006cd00 msgr2=0x7f6af01e60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:23.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6ae67fc640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6af006cd00 0x7f6af01e60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6ae67fc640 1 -- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6af015aad0 msgr2=0x7f6af00c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6ae67fc640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6af015aad0 0x7f6af00c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6ae67fc640 1 -- 172.21.15.70:0/3950751884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae8009b30 con 0x7f6af00c82b0 2023-10-08T05:37:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6aef7fe640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6af015aad0 0x7f6af00c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6ae67fc640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af00c82b0 0x7f6af00c38e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800d800 tx=0x7f6ad800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.901+0000 7f6ae7fff640 1 -- 172.21.15.70:0/3950751884 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80096a0 con 0x7f6af00c82b0 2023-10-08T05:37:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.902+0000 7f6ae7fff640 1 -- 172.21.15.70:0/3950751884 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad8009800 con 0x7f6af00c82b0 2023-10-08T05:37:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.902+0000 7f6ae7fff640 1 -- 172.21.15.70:0/3950751884 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8009b90 con 0x7f6af00c82b0 2023-10-08T05:37:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.902+0000 7f6af53a9640 1 -- 172.21.15.70:0/3950751884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af00c6830 con 0x7f6af00c82b0 2023-10-08T05:37:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.902+0000 7f6af53a9640 1 -- 172.21.15.70:0/3950751884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af00c6df0 con 0x7f6af00c82b0 2023-10-08T05:37:23.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.903+0000 7f6af53a9640 1 -- 172.21.15.70:0/3950751884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af004e160 con 0x7f6af00c82b0 2023-10-08T05:37:23.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.904+0000 7f6ae7fff640 1 -- 172.21.15.70:0/3950751884 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6ad8009cf0 con 0x7f6af00c82b0 2023-10-08T05:37:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.905+0000 7f6ae7fff640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ad006ae70 0x7f6ad006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.905+0000 7f6ae7fff640 1 -- 172.21.15.70:0/3950751884 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6ad8089ef0 con 0x7f6af00c82b0 2023-10-08T05:37:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.905+0000 7f6aeeffd640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ad006ae70 0x7f6ad006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:23.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.907+0000 7f6aeeffd640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ad006ae70 0x7f6ad006d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0006fd0 tx=0x7f6ae0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:23.909+0000 7f6ae7fff640 1 -- 172.21.15.70:0/3950751884 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ad8054b40 con 0x7f6af00c82b0 2023-10-08T05:37:24.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.101+0000 7f6af53a9640 1 -- 172.21.15.70:0/3950751884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6af00b1f90 con 0x7f6af00c82b0 2023-10-08T05:37:24.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.102+0000 7f6ae7fff640 1 -- 172.21.15.70:0/3950751884 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ad800b440 con 0x7f6af00c82b0 2023-10-08T05:37:24.103 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:24.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 -- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ad006ae70 msgr2=0x7f6ad006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ad006ae70 0x7f6ad006d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0006fd0 tx=0x7f6ae0008040 comp rx=0 tx=0).stop 2023-10-08T05:37:24.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 -- 172.21.15.70:0/3950751884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af00c82b0 msgr2=0x7f6af00c38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af00c82b0 0x7f6af00c38e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800d800 tx=0x7f6ad800dcc0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 -- 172.21.15.70:0/3950751884 shutdown_connections 2023-10-08T05:37:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ad006ae70 0x7f6ad006d330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6af015aad0 0x7f6af00c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6af00c82b0 0x7f6af00c38e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 --2- 172.21.15.70:0/3950751884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6af006cd00 0x7f6af01e60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 -- 172.21.15.70:0/3950751884 >> 172.21.15.70:0/3950751884 conn(0x7f6af0069d50 msgr2=0x7f6af0154280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:24.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 -- 172.21.15.70:0/3950751884 shutdown_connections 2023-10-08T05:37:24.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.105+0000 7f6af53a9640 1 -- 172.21.15.70:0/3950751884 wait complete. 2023-10-08T05:37:24.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:37:24.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.337+0000 7f4891075640 1 Processor -- start 2023-10-08T05:37:24.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.337+0000 7f4891075640 1 -- start start 2023-10-08T05:37:24.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.338+0000 7f4891075640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f488c0702b0 0x7f488c070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.338+0000 7f4891075640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f488c0b21f0 0x7f488c0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.338+0000 7f4891075640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f488c0ca1a0 0x7f488c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.338+0000 7f4891075640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c06c3a0 con 0x7f488c0b21f0 2023-10-08T05:37:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.338+0000 7f4891075640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488c070bd0 con 0x7f488c0702b0 2023-10-08T05:37:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.338+0000 7f4891075640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c06c640 con 0x7f488c0ca1a0 2023-10-08T05:37:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.339+0000 7f488a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f488c0b21f0 0x7f488c0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.339+0000 7f488a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f488c0b21f0 0x7f488c0c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55422/0 (socket says 172.21.15.70:55422) 2023-10-08T05:37:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.339+0000 7f488a59c640 1 -- 172.21.15.70:0/1596054834 learned_addr learned my addr 172.21.15.70:0/1596054834 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.339+0000 7f488ad9d640 1 --2- 172.21.15.70:0/1596054834 >> v2:172.21.15.181:3300/0 conn(0x7f488c0702b0 0x7f488c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.339+0000 7f488b59e640 1 --2- 172.21.15.70:0/1596054834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f488c0ca1a0 0x7f488c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.339+0000 7f488a59c640 1 -- 172.21.15.70:0/1596054834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f488c0ca1a0 msgr2=0x7f488c06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.339+0000 7f488a59c640 1 --2- 172.21.15.70:0/1596054834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f488c0ca1a0 0x7f488c06d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.339+0000 7f488a59c640 1 -- 172.21.15.70:0/1596054834 >> v2:172.21.15.181:3300/0 conn(0x7f488c0702b0 msgr2=0x7f488c070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.339+0000 7f488a59c640 1 --2- 172.21.15.70:0/1596054834 >> v2:172.21.15.181:3300/0 conn(0x7f488c0702b0 0x7f488c070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.340+0000 7f488a59c640 1 -- 172.21.15.70:0/1596054834 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c06c920 con 0x7f488c0b21f0 2023-10-08T05:37:24.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.342+0000 7f488a59c640 1 --2- 172.21.15.70:0/1596054834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0b21f0 0x7f488c0c9c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f488c0b0c20 tx=0x7f488002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79cac679f868c21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:24.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.342+0000 7f4889d9b640 1 -- 172.21.15.70:0/1596054834 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880031870 con 0x7f488c0b21f0 2023-10-08T05:37:24.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.342+0000 7f4889d9b640 1 -- 172.21.15.70:0/1596054834 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48800319d0 con 0x7f488c0b21f0 2023-10-08T05:37:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.342+0000 7f4889d9b640 1 -- 172.21.15.70:0/1596054834 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880031d60 con 0x7f488c0b21f0 2023-10-08T05:37:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.343+0000 7f4891075640 1 -- 172.21.15.70:0/1596054834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0b21f0 msgr2=0x7f488c0c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.343+0000 7f4891075640 1 --2- 172.21.15.70:0/1596054834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0b21f0 0x7f488c0c9c60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f488c0b0c20 tx=0x7f488002b910 comp rx=0 tx=0).stop 2023-10-08T05:37:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.343+0000 7f4891075640 1 -- 172.21.15.70:0/1596054834 shutdown_connections 2023-10-08T05:37:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.343+0000 7f4891075640 1 --2- 172.21.15.70:0/1596054834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f488c0ca1a0 0x7f488c06d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.343+0000 7f4891075640 1 --2- 172.21.15.70:0/1596054834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0b21f0 0x7f488c0c9c60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.343+0000 7f4891075640 1 --2- 172.21.15.70:0/1596054834 >> v2:172.21.15.181:3300/0 conn(0x7f488c0702b0 0x7f488c070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.343+0000 7f4891075640 1 -- 172.21.15.70:0/1596054834 >> 172.21.15.70:0/1596054834 conn(0x7f488c06a5d0 msgr2=0x7f488c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.343+0000 7f4891075640 1 -- 172.21.15.70:0/1596054834 shutdown_connections 2023-10-08T05:37:24.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.343+0000 7f4891075640 1 -- 172.21.15.70:0/1596054834 wait complete. 2023-10-08T05:37:24.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.345+0000 7f4891075640 1 Processor -- start 2023-10-08T05:37:24.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.345+0000 7f4891075640 1 -- start start 2023-10-08T05:37:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f4891075640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0702b0 0x7f488c15c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f4891075640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f488c0b21f0 0x7f488c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f4891075640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c0ca1a0 0x7f488c1e6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f4891075640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c145890 con 0x7f488c0702b0 2023-10-08T05:37:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f4891075640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c145b90 con 0x7f488c0ca1a0 2023-10-08T05:37:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f4891075640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c15c7c0 con 0x7f488c0b21f0 2023-10-08T05:37:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f488ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0702b0 0x7f488c15c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f488ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0702b0 0x7f488c15c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55426/0 (socket says 172.21.15.70:55426) 2023-10-08T05:37:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f488ad9d640 1 -- 172.21.15.70:0/3236334662 learned_addr learned my addr 172.21.15.70:0/3236334662 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f488ad9d640 1 -- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f488c0b21f0 msgr2=0x7f488c15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f488ad9d640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f488c0b21f0 0x7f488c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f488ad9d640 1 -- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c0ca1a0 msgr2=0x7f488c1e6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f488ad9d640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c0ca1a0 0x7f488c1e6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.346+0000 7f488ad9d640 1 -- 172.21.15.70:0/3236334662 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800094d0 con 0x7f488c0702b0 2023-10-08T05:37:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.347+0000 7f488ad9d640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0702b0 0x7f488c15c280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f487c00d800 tx=0x7f487c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.347+0000 7f48737fe640 1 -- 172.21.15.70:0/3236334662 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c0096a0 con 0x7f488c0702b0 2023-10-08T05:37:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.347+0000 7f48737fe640 1 -- 172.21.15.70:0/3236334662 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487c009800 con 0x7f488c0702b0 2023-10-08T05:37:24.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.347+0000 7f4891075640 1 -- 172.21.15.70:0/3236334662 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c148100 con 0x7f488c0702b0 2023-10-08T05:37:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.347+0000 7f48737fe640 1 -- 172.21.15.70:0/3236334662 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c009b90 con 0x7f488c0702b0 2023-10-08T05:37:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.347+0000 7f4891075640 1 -- 172.21.15.70:0/3236334662 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c1486c0 con 0x7f488c0702b0 2023-10-08T05:37:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.348+0000 7f4891075640 1 -- 172.21.15.70:0/3236334662 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c04e160 con 0x7f488c0702b0 2023-10-08T05:37:24.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.349+0000 7f48737fe640 1 -- 172.21.15.70:0/3236334662 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f487c009cf0 con 0x7f488c0702b0 2023-10-08T05:37:24.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.350+0000 7f48737fe640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f487806ae70 0x7f487806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.350+0000 7f48737fe640 1 -- 172.21.15.70:0/3236334662 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f487c089ef0 con 0x7f488c0702b0 2023-10-08T05:37:24.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.351+0000 7f488a59c640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f487806ae70 0x7f487806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.352+0000 7f488a59c640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f487806ae70 0x7f487806d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4880036000 tx=0x7f4880030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:24.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.355+0000 7f48737fe640 1 -- 172.21.15.70:0/3236334662 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f487c054b40 con 0x7f488c0702b0 2023-10-08T05:37:24.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.542+0000 7f4891075640 1 -- 172.21.15.70:0/3236334662 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f488c0b41f0 con 0x7f488c0702b0 2023-10-08T05:37:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.543+0000 7f48737fe640 1 -- 172.21.15.70:0/3236334662 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f487c00b440 con 0x7f488c0702b0 2023-10-08T05:37:24.543 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:24.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 -- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f487806ae70 msgr2=0x7f487806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f487806ae70 0x7f487806d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4880036000 tx=0x7f4880030000 comp rx=0 tx=0).stop 2023-10-08T05:37:24.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 -- 172.21.15.70:0/3236334662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0702b0 msgr2=0x7f488c15c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0702b0 0x7f488c15c280 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f487c00d800 tx=0x7f487c00dcc0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 -- 172.21.15.70:0/3236334662 shutdown_connections 2023-10-08T05:37:24.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f487806ae70 0x7f487806d330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f488c0ca1a0 0x7f488c1e6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f488c0b21f0 0x7f488c15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 --2- 172.21.15.70:0/3236334662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f488c0702b0 0x7f488c15c280 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 -- 172.21.15.70:0/3236334662 >> 172.21.15.70:0/3236334662 conn(0x7f488c06a5d0 msgr2=0x7f488c0c6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 -- 172.21.15.70:0/3236334662 shutdown_connections 2023-10-08T05:37:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.545+0000 7f4891075640 1 -- 172.21.15.70:0/3236334662 wait complete. 2023-10-08T05:37:24.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:37:24.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.778+0000 7fa0910b4640 1 Processor -- start 2023-10-08T05:37:24.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.779+0000 7fa0910b4640 1 -- start start 2023-10-08T05:37:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.780+0000 7fa0910b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa08c0702b0 0x7fa08c070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.780+0000 7fa0910b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa08c0b21f0 0x7fa08c0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.780+0000 7fa0910b4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa08c0ca1a0 0x7fa08c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.780+0000 7fa0910b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08c06c3a0 con 0x7fa08c0702b0 2023-10-08T05:37:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.780+0000 7fa0910b4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa08c070bd0 con 0x7fa08c0ca1a0 2023-10-08T05:37:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.780+0000 7fa0910b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c06c640 con 0x7fa08c0b21f0 2023-10-08T05:37:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.780+0000 7fa08ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa08c0702b0 0x7fa08c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.781+0000 7fa08ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa08c0702b0 0x7fa08c070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55432/0 (socket says 172.21.15.70:55432) 2023-10-08T05:37:24.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.781+0000 7fa08ad9d640 1 -- 172.21.15.70:0/2325271073 learned_addr learned my addr 172.21.15.70:0/2325271073 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:24.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.781+0000 7fa083fff640 1 --2- 172.21.15.70:0/2325271073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa08c0b21f0 0x7fa08c0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.781+0000 7fa08b59e640 1 --2- 172.21.15.70:0/2325271073 >> v2:172.21.15.181:3300/0 conn(0x7fa08c0ca1a0 0x7fa08c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.781+0000 7fa08ad9d640 1 -- 172.21.15.70:0/2325271073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa08c0b21f0 msgr2=0x7fa08c0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:24.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.781+0000 7fa08ad9d640 1 --2- 172.21.15.70:0/2325271073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa08c0b21f0 0x7fa08c0c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.781+0000 7fa08ad9d640 1 -- 172.21.15.70:0/2325271073 >> v2:172.21.15.181:3300/0 conn(0x7fa08c0ca1a0 msgr2=0x7fa08c06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:24.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.781+0000 7fa08ad9d640 1 --2- 172.21.15.70:0/2325271073 >> v2:172.21.15.181:3300/0 conn(0x7fa08c0ca1a0 0x7fa08c06d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.781+0000 7fa08ad9d640 1 -- 172.21.15.70:0/2325271073 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c06c920 con 0x7fa08c0702b0 2023-10-08T05:37:24.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.781+0000 7fa08b59e640 1 --2- 172.21.15.70:0/2325271073 >> v2:172.21.15.181:3300/0 conn(0x7fa08c0ca1a0 0x7fa08c06d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:24.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.782+0000 7fa08ad9d640 1 --2- 172.21.15.70:0/2325271073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 0x7fa08c070690 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa08c06e310 tx=0x7fa07c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b38f61904675265 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:24.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.782+0000 7fa08a59c640 1 -- 172.21.15.70:0/2325271073 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c031870 con 0x7fa08c0702b0 2023-10-08T05:37:24.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa08a59c640 1 -- 172.21.15.70:0/2325271073 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07c0319d0 con 0x7fa08c0702b0 2023-10-08T05:37:24.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa08a59c640 1 -- 172.21.15.70:0/2325271073 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c031d60 con 0x7fa08c0702b0 2023-10-08T05:37:24.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa0910b4640 1 -- 172.21.15.70:0/2325271073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 msgr2=0x7fa08c070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa0910b4640 1 --2- 172.21.15.70:0/2325271073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 0x7fa08c070690 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa08c06e310 tx=0x7fa07c02b910 comp rx=0 tx=0).stop 2023-10-08T05:37:24.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa0910b4640 1 -- 172.21.15.70:0/2325271073 shutdown_connections 2023-10-08T05:37:24.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa0910b4640 1 --2- 172.21.15.70:0/2325271073 >> v2:172.21.15.181:3300/0 conn(0x7fa08c0ca1a0 0x7fa08c06d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa0910b4640 1 --2- 172.21.15.70:0/2325271073 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa08c0b21f0 0x7fa08c0c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa0910b4640 1 --2- 172.21.15.70:0/2325271073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 0x7fa08c070690 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa0910b4640 1 -- 172.21.15.70:0/2325271073 >> 172.21.15.70:0/2325271073 conn(0x7fa08c06a5d0 msgr2=0x7fa08c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:24.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa0910b4640 1 -- 172.21.15.70:0/2325271073 shutdown_connections 2023-10-08T05:37:24.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.783+0000 7fa0910b4640 1 -- 172.21.15.70:0/2325271073 wait complete. 2023-10-08T05:37:24.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.785+0000 7fa0910b4640 1 Processor -- start 2023-10-08T05:37:24.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.785+0000 7fa0910b4640 1 -- start start 2023-10-08T05:37:24.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.786+0000 7fa0910b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 0x7fa08c142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.786+0000 7fa0910b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa08c0b21f0 0x7fa08c159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.786+0000 7fa0910b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa08c0ca1a0 0x7fa08c159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.786+0000 7fa0910b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c161f40 con 0x7fa08c0702b0 2023-10-08T05:37:24.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.786+0000 7fa0910b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c148420 con 0x7fa08c0b21f0 2023-10-08T05:37:24.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.786+0000 7fa0910b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c161610 con 0x7fa08c0ca1a0 2023-10-08T05:37:24.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa08ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 0x7fa08c142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa08ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 0x7fa08c142030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55444/0 (socket says 172.21.15.70:55444) 2023-10-08T05:37:24.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa08ad9d640 1 -- 172.21.15.70:0/81904308 learned_addr learned my addr 172.21.15.70:0/81904308 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:24.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa08ad9d640 1 -- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa08c0ca1a0 msgr2=0x7fa08c159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa083fff640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa08c0b21f0 0x7fa08c159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa08b59e640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa08c0ca1a0 0x7fa08c159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa08ad9d640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa08c0ca1a0 0x7fa08c159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa08ad9d640 1 -- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa08c0b21f0 msgr2=0x7fa08c159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa08ad9d640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa08c0b21f0 0x7fa08c159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa08ad9d640 1 -- 172.21.15.70:0/81904308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c0094d0 con 0x7fa08c0702b0 2023-10-08T05:37:24.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa083fff640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa08c0b21f0 0x7fa08c159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:24.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.787+0000 7fa08ad9d640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 0x7fa08c142030 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa07c02b950 tx=0x7fa07c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:24.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.788+0000 7fa0837fe640 1 -- 172.21.15.70:0/81904308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c03ad30 con 0x7fa08c0702b0 2023-10-08T05:37:24.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.788+0000 7fa0837fe640 1 -- 172.21.15.70:0/81904308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07c0443f0 con 0x7fa08c0702b0 2023-10-08T05:37:24.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.788+0000 7fa0837fe640 1 -- 172.21.15.70:0/81904308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0447a0 con 0x7fa08c0702b0 2023-10-08T05:37:24.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.788+0000 7fa0910b4640 1 -- 172.21.15.70:0/81904308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c1618f0 con 0x7fa08c0702b0 2023-10-08T05:37:24.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.788+0000 7fa0910b4640 1 -- 172.21.15.70:0/81904308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08c142570 con 0x7fa08c0702b0 2023-10-08T05:37:24.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.789+0000 7fa0910b4640 1 -- 172.21.15.70:0/81904308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa08c161750 con 0x7fa08c0702b0 2023-10-08T05:37:24.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.790+0000 7fa0837fe640 1 -- 172.21.15.70:0/81904308 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa07c04d450 con 0x7fa08c0702b0 2023-10-08T05:37:24.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.791+0000 7fa0837fe640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa06c06aea0 0x7fa06c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:24.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.791+0000 7fa0837fe640 1 -- 172.21.15.70:0/81904308 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa07c02d040 con 0x7fa08c0702b0 2023-10-08T05:37:24.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.792+0000 7fa083fff640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa06c06aea0 0x7fa06c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:24.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.793+0000 7fa083fff640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa06c06aea0 0x7fa06c06d360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa074006fd0 tx=0x7fa074008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:24.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.795+0000 7fa0837fe640 1 -- 172.21.15.70:0/81904308 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa08c161750 con 0x7fa08c0702b0 2023-10-08T05:37:24.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.987+0000 7fa0910b4640 1 -- 172.21.15.70:0/81904308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa08c161750 con 0x7fa08c0702b0 2023-10-08T05:37:24.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.988+0000 7fa0837fe640 1 -- 172.21.15.70:0/81904308 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa08c161750 con 0x7fa08c0702b0 2023-10-08T05:37:24.989 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:37:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 -- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa06c06aea0 msgr2=0x7fa06c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa06c06aea0 0x7fa06c06d360 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa074006fd0 tx=0x7fa074008040 comp rx=0 tx=0).stop 2023-10-08T05:37:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 -- 172.21.15.70:0/81904308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 msgr2=0x7fa08c142030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 0x7fa08c142030 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa07c02b950 tx=0x7fa07c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 -- 172.21.15.70:0/81904308 shutdown_connections 2023-10-08T05:37:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa06c06aea0 0x7fa06c06d360 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa08c0ca1a0 0x7fa08c159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa08c0b21f0 0x7fa08c159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 --2- 172.21.15.70:0/81904308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa08c0702b0 0x7fa08c142030 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:24.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 -- 172.21.15.70:0/81904308 >> 172.21.15.70:0/81904308 conn(0x7fa08c06a5d0 msgr2=0x7fa08c157d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:24.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 -- 172.21.15.70:0/81904308 shutdown_connections 2023-10-08T05:37:24.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:24.991+0000 7fa0910b4640 1 -- 172.21.15.70:0/81904308 wait complete. 2023-10-08T05:37:25.009 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:37:25.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.228+0000 7f86f8e07640 1 Processor -- start 2023-10-08T05:37:25.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.228+0000 7f86f8e07640 1 -- start start 2023-10-08T05:37:25.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.230+0000 7f86f8e07640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86f40cb620 0x7f86f40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.230+0000 7f86f8e07640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f4071170 con 0x7f86f4160660 2023-10-08T05:37:25.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.230+0000 7f86f8e07640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f406e150 con 0x7f86f4070cc0 2023-10-08T05:37:25.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.230+0000 7f86f8e07640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f406de70 con 0x7f86f40cb620 2023-10-08T05:37:25.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.230+0000 7f86f259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f86f4160660 0x7f86f40c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57142/0 (socket says 172.21.15.70:57142) 2023-10-08T05:37:25.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.230+0000 7f86f259c640 1 -- 172.21.15.70:0/191978701 learned_addr learned my addr 172.21.15.70:0/191978701 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:25.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.230+0000 7f86f2d9d640 1 --2- 172.21.15.70:0/191978701 >> v2:172.21.15.181:3300/0 conn(0x7f86f40cb620 0x7f86f40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:25.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798813502 0 0) 0x7f86e4001490 con 0x7f86f4160660 2023-10-08T05:37:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e0003520 con 0x7f86f4160660 2023-10-08T05:37:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505640352 0 0) 0x7f86e0003520 con 0x7f86f4160660 2023-10-08T05:37:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 >> v2:172.21.15.181:3300/0 conn(0x7f86f40cb620 msgr2=0x7f86f40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 --2- 172.21.15.70:0/191978701 >> v2:172.21.15.181:3300/0 conn(0x7f86f40cb620 0x7f86f40b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 >> v1:172.21.15.181:6789/0 conn(0x7f86f4070cc0 legacy=0x7f86f40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f40c8e30 con 0x7f86f4160660 2023-10-08T05:37:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f86e4001b80 con 0x7f86f4160660 2023-10-08T05:37:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 >> v1:172.21.15.70:6789/0 conn(0x7f86f4160660 legacy=0x7f86f40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86e00039c0 0x7f86e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86e00047c0 0x7f86e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f406de70 con 0x7f86e00051f0 2023-10-08T05:37:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e000c150 con 0x7f86e00047c0 2023-10-08T05:37:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e000c310 con 0x7f86e00039c0 2023-10-08T05:37:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.231+0000 7f86f2d9d640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.232+0000 7f86e9d9b640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86e00047c0 0x7f86e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.232+0000 7f86f259c640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86e00039c0 0x7f86e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.232+0000 7f86f2d9d640 1 -- 172.21.15.70:0/191978701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86e00039c0 msgr2=0x7f86e0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.232+0000 7f86f2d9d640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86e00039c0 0x7f86e0003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.232+0000 7f86f2d9d640 1 -- 172.21.15.70:0/191978701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86e00047c0 msgr2=0x7f86e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:25.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.232+0000 7f86f2d9d640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86e00047c0 0x7f86e0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.232+0000 7f86f2d9d640 1 -- 172.21.15.70:0/191978701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0003520 con 0x7f86e00051f0 2023-10-08T05:37:25.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.233+0000 7f86f2d9d640 1 -- 172.21.15.70:0/191978701 reap_dead start 2023-10-08T05:37:25.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.233+0000 7f86f2d9d640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86e000b980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009ac0 tx=0x7f86e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a694a16a3959e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:25.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.233+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec004020 con 0x7f86e00051f0 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.233+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ec004180 con 0x7f86e00051f0 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.233+0000 7f86f1d9b640 1 -- 172.21.15.70:0/191978701 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec02c630 con 0x7f86e00051f0 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.234+0000 7f86f8e07640 1 -- 172.21.15.70:0/191978701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 msgr2=0x7f86e000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.234+0000 7f86f8e07640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86e000b980 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009ac0 tx=0x7f86e00057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.234+0000 7f86f8e07640 1 -- 172.21.15.70:0/191978701 shutdown_connections 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.234+0000 7f86f8e07640 1 --2- 172.21.15.70:0/191978701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86e000b980 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.234+0000 7f86f8e07640 1 -- 172.21.15.70:0/191978701 >> 172.21.15.70:0/191978701 conn(0x7f86f406a5d0 msgr2=0x7f86f40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.234+0000 7f86f8e07640 1 -- 172.21.15.70:0/191978701 shutdown_connections 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.234+0000 7f86f8e07640 1 -- 172.21.15.70:0/191978701 wait complete. 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.236+0000 7f86f8e07640 1 Processor -- start 2023-10-08T05:37:25.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.236+0000 7f86f8e07640 1 -- start start 2023-10-08T05:37:25.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f8e07640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86f414bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f8e07640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86f40ceed0 0x7f86f4158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f8e07640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86f40cf350 0x7f86f40cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f8e07640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f414c0d0 con 0x7f86e00051f0 2023-10-08T05:37:25.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f8e07640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f41572c0 con 0x7f86f40ceed0 2023-10-08T05:37:25.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f8e07640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f41575a0 con 0x7f86f40cf350 2023-10-08T05:37:25.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86f414bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:25.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86f414bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55474/0 (socket says 172.21.15.70:55474) 2023-10-08T05:37:25.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f259c640 1 -- 172.21.15.70:0/3152685594 learned_addr learned my addr 172.21.15.70:0/3152685594 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:25.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f259c640 1 -- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86f40cf350 msgr2=0x7f86f40cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f259c640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86f40cf350 0x7f86f40cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f259c640 1 -- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86f40ceed0 msgr2=0x7f86f4158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f259c640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86f40ceed0 0x7f86f4158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.237+0000 7f86f259c640 1 -- 172.21.15.70:0/3152685594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f4157880 con 0x7f86e00051f0 2023-10-08T05:37:25.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.238+0000 7f86f259c640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86f414bb90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f86e4001a90 tx=0x7f86e400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:25.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.238+0000 7f86eb7fe640 1 -- 172.21.15.70:0/3152685594 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4004b60 con 0x7f86e00051f0 2023-10-08T05:37:25.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.238+0000 7f86eb7fe640 1 -- 172.21.15.70:0/3152685594 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4004cc0 con 0x7f86e00051f0 2023-10-08T05:37:25.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.238+0000 7f86eb7fe640 1 -- 172.21.15.70:0/3152685594 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4007310 con 0x7f86e00051f0 2023-10-08T05:37:25.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.238+0000 7f86f8e07640 1 -- 172.21.15.70:0/3152685594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f4157b60 con 0x7f86e00051f0 2023-10-08T05:37:25.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.238+0000 7f86f8e07640 1 -- 172.21.15.70:0/3152685594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f40c78a0 con 0x7f86e00051f0 2023-10-08T05:37:25.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.239+0000 7f86f8e07640 1 -- 172.21.15.70:0/3152685594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f404e160 con 0x7f86e00051f0 2023-10-08T05:37:25.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.240+0000 7f86eb7fe640 1 -- 172.21.15.70:0/3152685594 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f86e4007470 con 0x7f86e00051f0 2023-10-08T05:37:25.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.241+0000 7f86eb7fe640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f86d406aea0 0x7f86d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.241+0000 7f86eb7fe640 1 -- 172.21.15.70:0/3152685594 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f86e4089800 con 0x7f86e00051f0 2023-10-08T05:37:25.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.242+0000 7f86e9d9b640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f86d406aea0 0x7f86d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:25.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.243+0000 7f86e9d9b640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f86d406aea0 0x7f86d406d360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86f40b0320 tx=0x7f86dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:25.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.245+0000 7f86eb7fe640 1 -- 172.21.15.70:0/3152685594 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86e40544e0 con 0x7f86e00051f0 2023-10-08T05:37:25.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.433+0000 7f86f8e07640 1 -- 172.21.15.70:0/3152685594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f86f4071170 con 0x7f86e00051f0 2023-10-08T05:37:25.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.434+0000 7f86eb7fe640 1 -- 172.21.15.70:0/3152685594 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f86e4019020 con 0x7f86e00051f0 2023-10-08T05:37:25.435 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:25.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 -- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f86d406aea0 msgr2=0x7f86d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f86d406aea0 0x7f86d406d360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f86f40b0320 tx=0x7f86dc009040 comp rx=0 tx=0).stop 2023-10-08T05:37:25.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 -- 172.21.15.70:0/3152685594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 msgr2=0x7f86f414bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86f414bb90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f86e4001a90 tx=0x7f86e400b4a0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 -- 172.21.15.70:0/3152685594 shutdown_connections 2023-10-08T05:37:25.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f86d406aea0 0x7f86d406d360 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86f40cf350 0x7f86f40cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86f40ceed0 0x7f86f4158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 --2- 172.21.15.70:0/3152685594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e00051f0 0x7f86f414bb90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 -- 172.21.15.70:0/3152685594 >> 172.21.15.70:0/3152685594 conn(0x7f86f406a5d0 msgr2=0x7f86f40caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:25.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 -- 172.21.15.70:0/3152685594 shutdown_connections 2023-10-08T05:37:25.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.437+0000 7f86f8e07640 1 -- 172.21.15.70:0/3152685594 wait complete. 2023-10-08T05:37:25.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:37:25.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.680+0000 7f7566d39640 1 Processor -- start 2023-10-08T05:37:25.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.680+0000 7f7566d39640 1 -- start start 2023-10-08T05:37:25.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f7566d39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f756006dfc0 0x7f7560070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f7566d39640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7560071070 0x7f75600cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f7566d39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75600b1cc0 con 0x7f75600afbe0 2023-10-08T05:37:25.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f7566d39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75600b2620 con 0x7f756006dfc0 2023-10-08T05:37:25.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f7566d39640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75600b2060 con 0x7f7560071070 2023-10-08T05:37:25.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f755ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f756006dfc0 0x7f7560070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:25.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f755ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f756006dfc0 0x7f7560070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55488/0 (socket says 172.21.15.70:55488) 2023-10-08T05:37:25.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f755ffff640 1 -- 172.21.15.70:0/3954672057 learned_addr learned my addr 172.21.15.70:0/3954672057 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f75652d6640 1 --2- 172.21.15.70:0/3954672057 >> v2:172.21.15.181:3300/0 conn(0x7f7560071070 0x7f75600cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f755ffff640 1 -- 172.21.15.70:0/3954672057 >> v2:172.21.15.181:3300/0 conn(0x7f7560071070 msgr2=0x7f75600cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.682+0000 7f755ffff640 1 --2- 172.21.15.70:0/3954672057 >> v2:172.21.15.181:3300/0 conn(0x7f7560071070 0x7f75600cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.683+0000 7f755ffff640 1 -- 172.21.15.70:0/3954672057 >> v1:172.21.15.181:6789/0 conn(0x7f75600afbe0 legacy=0x7f75600b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.683+0000 7f755ffff640 1 -- 172.21.15.70:0/3954672057 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75600b2340 con 0x7f756006dfc0 2023-10-08T05:37:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.684+0000 7f755ffff640 1 --2- 172.21.15.70:0/3954672057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756006dfc0 0x7f7560070b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f75600c8d10 tx=0x7f754c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6225fcf94f7037e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.684+0000 7f755f7fe640 1 -- 172.21.15.70:0/3954672057 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c031870 con 0x7f756006dfc0 2023-10-08T05:37:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.684+0000 7f755f7fe640 1 -- 172.21.15.70:0/3954672057 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c0319d0 con 0x7f756006dfc0 2023-10-08T05:37:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.684+0000 7f755f7fe640 1 -- 172.21.15.70:0/3954672057 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c031d60 con 0x7f756006dfc0 2023-10-08T05:37:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.685+0000 7f7566d39640 1 -- 172.21.15.70:0/3954672057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756006dfc0 msgr2=0x7f7560070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.685+0000 7f7566d39640 1 --2- 172.21.15.70:0/3954672057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756006dfc0 0x7f7560070b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f75600c8d10 tx=0x7f754c02b910 comp rx=0 tx=0).stop 2023-10-08T05:37:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.685+0000 7f7566d39640 1 -- 172.21.15.70:0/3954672057 shutdown_connections 2023-10-08T05:37:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.685+0000 7f7566d39640 1 --2- 172.21.15.70:0/3954672057 >> v2:172.21.15.181:3300/0 conn(0x7f7560071070 0x7f75600cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.685+0000 7f7566d39640 1 --2- 172.21.15.70:0/3954672057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f756006dfc0 0x7f7560070b30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.685+0000 7f7566d39640 1 -- 172.21.15.70:0/3954672057 >> 172.21.15.70:0/3954672057 conn(0x7f756006a5d0 msgr2=0x7f75600b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.685+0000 7f7566d39640 1 -- 172.21.15.70:0/3954672057 shutdown_connections 2023-10-08T05:37:25.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.685+0000 7f7566d39640 1 -- 172.21.15.70:0/3954672057 wait complete. 2023-10-08T05:37:25.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.686+0000 7f7566d39640 1 Processor -- start 2023-10-08T05:37:25.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.687+0000 7f7566d39640 1 -- start start 2023-10-08T05:37:25.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f7566d39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756006dfc0 0x7f75600c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f7566d39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7560071070 0x7f756015f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f7566d39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75600afbe0 0x7f756015d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f7566d39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7560142330 con 0x7f7560071070 2023-10-08T05:37:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f7566d39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75601448b0 con 0x7f75600afbe0 2023-10-08T05:37:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f7566d39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7560144b90 con 0x7f756006dfc0 2023-10-08T05:37:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f755ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7560071070 0x7f756015f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f755ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7560071070 0x7f756015f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55490/0 (socket says 172.21.15.70:55490) 2023-10-08T05:37:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f755ffff640 1 -- 172.21.15.70:0/3828444781 learned_addr learned my addr 172.21.15.70:0/3828444781 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f755ffff640 1 -- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756006dfc0 msgr2=0x7f75600c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f755ffff640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756006dfc0 0x7f75600c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f755ffff640 1 -- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75600afbe0 msgr2=0x7f756015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f755ffff640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75600afbe0 0x7f756015d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.688+0000 7f755ffff640 1 -- 172.21.15.70:0/3828444781 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0094d0 con 0x7f7560071070 2023-10-08T05:37:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.689+0000 7f755ffff640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7560071070 0x7f756015f150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f754c000c00 tx=0x7f754c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.689+0000 7f755d7fa640 1 -- 172.21.15.70:0/3828444781 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c03ad30 con 0x7f7560071070 2023-10-08T05:37:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.689+0000 7f755d7fa640 1 -- 172.21.15.70:0/3828444781 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c03ae90 con 0x7f7560071070 2023-10-08T05:37:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.689+0000 7f7566d39640 1 -- 172.21.15.70:0/3828444781 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7560144e70 con 0x7f7560071070 2023-10-08T05:37:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.689+0000 7f755d7fa640 1 -- 172.21.15.70:0/3828444781 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c044630 con 0x7f7560071070 2023-10-08T05:37:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.689+0000 7f7566d39640 1 -- 172.21.15.70:0/3828444781 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75600c2e50 con 0x7f7560071070 2023-10-08T05:37:25.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.690+0000 7f7566d39640 1 -- 172.21.15.70:0/3828444781 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75601424a0 con 0x7f7560071070 2023-10-08T05:37:25.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.691+0000 7f755d7fa640 1 -- 172.21.15.70:0/3828444781 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f754c04d450 con 0x7f7560071070 2023-10-08T05:37:25.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.693+0000 7f755d7fa640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f754406aea0 0x7f754406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:25.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.693+0000 7f755d7fa640 1 -- 172.21.15.70:0/3828444781 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f754c02d040 con 0x7f7560071070 2023-10-08T05:37:25.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.693+0000 7f7564ad5640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f754406aea0 0x7f754406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:25.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.694+0000 7f7564ad5640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f754406aea0 0x7f754406d360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7554009650 tx=0x7f7554006ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:25.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.697+0000 7f755d7fa640 1 -- 172.21.15.70:0/3828444781 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f754c076d00 con 0x7f7560071070 2023-10-08T05:37:25.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.900+0000 7f7566d39640 1 -- 172.21.15.70:0/3828444781 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f756004e160 con 0x7f7560071070 2023-10-08T05:37:25.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.901+0000 7f755d7fa640 1 -- 172.21.15.70:0/3828444781 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f754c07a320 con 0x7f7560071070 2023-10-08T05:37:25.901 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-10-08T05:37:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 -- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f754406aea0 msgr2=0x7f754406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f754406aea0 0x7f754406d360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7554009650 tx=0x7f7554006ba0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 -- 172.21.15.70:0/3828444781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7560071070 msgr2=0x7f756015f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7560071070 0x7f756015f150 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f754c000c00 tx=0x7f754c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 -- 172.21.15.70:0/3828444781 shutdown_connections 2023-10-08T05:37:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f754406aea0 0x7f754406d360 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75600afbe0 0x7f756015d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7560071070 0x7f756015f150 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 --2- 172.21.15.70:0/3828444781 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f756006dfc0 0x7f75600c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:25.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 -- 172.21.15.70:0/3828444781 >> 172.21.15.70:0/3828444781 conn(0x7f756006a5d0 msgr2=0x7f75600c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:25.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 -- 172.21.15.70:0/3828444781 shutdown_connections 2023-10-08T05:37:25.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:25.903+0000 7f7566d39640 1 -- 172.21.15.70:0/3828444781 wait complete. 2023-10-08T05:37:25.922 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-08T05:37:25.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:25.922 DEBUG:teuthology.orchestra.run.smithi181:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-08T05:37:26.047 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T05:37:26.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:26.047 DEBUG:teuthology.orchestra.run.smithi181:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T05:37:26.098 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T05:37:26.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:37:26.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.317+0000 7f6ad4300640 1 Processor -- start 2023-10-08T05:37:26.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.318+0000 7f6ad4300640 1 -- start start 2023-10-08T05:37:26.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.319+0000 7f6ad4300640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6acc15cd90 0x7f6acc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.319+0000 7f6ad4300640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6acc06c890 0x7f6acc06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.319+0000 7f6ad4300640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6acc0c48f0 con 0x7f6acc160940 2023-10-08T05:37:26.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.319+0000 7f6ad4300640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6acc0c4d80 con 0x7f6acc06c890 2023-10-08T05:37:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.319+0000 7f6ad4300640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0c3ce0 con 0x7f6acc15cd90 2023-10-08T05:37:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.319+0000 7f6ad289d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6acc06c890 0x7f6acc06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.320+0000 7f6ad289d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6acc06c890 0x7f6acc06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55504/0 (socket says 172.21.15.70:55504) 2023-10-08T05:37:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.320+0000 7f6ad189b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6acc160940 0x7f6acc070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57156/0 (socket says 172.21.15.70:57156) 2023-10-08T05:37:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.320+0000 7f6ad289d640 1 -- 172.21.15.70:0/1060786016 learned_addr learned my addr 172.21.15.70:0/1060786016 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.320+0000 7f6ad209c640 1 --2- 172.21.15.70:0/1060786016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6acc15cd90 0x7f6acc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.320+0000 7f6ad289d640 1 -- 172.21.15.70:0/1060786016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6acc15cd90 msgr2=0x7f6acc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.320+0000 7f6ad289d640 1 --2- 172.21.15.70:0/1060786016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6acc15cd90 0x7f6acc0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.320+0000 7f6ad289d640 1 -- 172.21.15.70:0/1060786016 >> v1:172.21.15.70:6789/0 conn(0x7f6acc160940 legacy=0x7f6acc070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.320+0000 7f6ad289d640 1 -- 172.21.15.70:0/1060786016 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc0c3fc0 con 0x7f6acc06c890 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.321+0000 7f6ad289d640 1 --2- 172.21.15.70:0/1060786016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 0x7f6acc06cd20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009e50 tx=0x7f6ac402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b5dffbba316a29 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.321+0000 7f6ad109a640 1 -- 172.21.15.70:0/1060786016 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac402cb00 con 0x7f6acc06c890 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.321+0000 7f6ad109a640 1 -- 172.21.15.70:0/1060786016 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac402cc60 con 0x7f6acc06c890 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.321+0000 7f6ad109a640 1 -- 172.21.15.70:0/1060786016 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac40343f0 con 0x7f6acc06c890 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.322+0000 7f6ad4300640 1 -- 172.21.15.70:0/1060786016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 msgr2=0x7f6acc06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.322+0000 7f6ad4300640 1 --2- 172.21.15.70:0/1060786016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 0x7f6acc06cd20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009e50 tx=0x7f6ac402bf60 comp rx=0 tx=0).stop 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.322+0000 7f6ad4300640 1 -- 172.21.15.70:0/1060786016 shutdown_connections 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.322+0000 7f6ad4300640 1 --2- 172.21.15.70:0/1060786016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 0x7f6acc06cd20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.322+0000 7f6ad4300640 1 --2- 172.21.15.70:0/1060786016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6acc15cd90 0x7f6acc0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.322+0000 7f6ad4300640 1 -- 172.21.15.70:0/1060786016 >> 172.21.15.70:0/1060786016 conn(0x7f6acc06a380 msgr2=0x7f6acc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:26.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.322+0000 7f6ad4300640 1 -- 172.21.15.70:0/1060786016 shutdown_connections 2023-10-08T05:37:26.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.322+0000 7f6ad4300640 1 -- 172.21.15.70:0/1060786016 wait complete. 2023-10-08T05:37:26.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.324+0000 7f6ad4300640 1 Processor -- start 2023-10-08T05:37:26.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.324+0000 7f6ad4300640 1 -- start start 2023-10-08T05:37:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.325+0000 7f6ad4300640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 0x7f6acc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.325+0000 7f6ad4300640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6acc15cd90 0x7f6acc0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.325+0000 7f6ad4300640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc160940 0x7f6acc15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.325+0000 7f6ad4300640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc14cc10 con 0x7f6acc06c890 2023-10-08T05:37:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.325+0000 7f6ad4300640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc154890 con 0x7f6acc160940 2023-10-08T05:37:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.325+0000 7f6ad4300640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc154b70 con 0x7f6acc15cd90 2023-10-08T05:37:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.325+0000 7f6ad209c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 0x7f6acc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.325+0000 7f6ad209c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 0x7f6acc0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55506/0 (socket says 172.21.15.70:55506) 2023-10-08T05:37:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.325+0000 7f6ad209c640 1 -- 172.21.15.70:0/1907942649 learned_addr learned my addr 172.21.15.70:0/1907942649 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ad209c640 1 -- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6acc15cd90 msgr2=0x7f6acc0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ad209c640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6acc15cd90 0x7f6acc0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ad209c640 1 -- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc160940 msgr2=0x7f6acc15d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ad209c640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc160940 0x7f6acc15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ad209c640 1 -- 172.21.15.70:0/1907942649 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac4009b30 con 0x7f6acc06c890 2023-10-08T05:37:26.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ad209c640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 0x7f6acc0caa90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800d800 tx=0x7f6ac800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:26.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ac2ffd640 1 -- 172.21.15.70:0/1907942649 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800b6c0 con 0x7f6acc06c890 2023-10-08T05:37:26.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ac2ffd640 1 -- 172.21.15.70:0/1907942649 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac800b820 con 0x7f6acc06c890 2023-10-08T05:37:26.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ac2ffd640 1 -- 172.21.15.70:0/1907942649 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800bbb0 con 0x7f6acc06c890 2023-10-08T05:37:26.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ad4300640 1 -- 172.21.15.70:0/1907942649 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc154e50 con 0x7f6acc06c890 2023-10-08T05:37:26.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.326+0000 7f6ad4300640 1 -- 172.21.15.70:0/1907942649 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc0cafd0 con 0x7f6acc06c890 2023-10-08T05:37:26.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.327+0000 7f6ad4300640 1 -- 172.21.15.70:0/1907942649 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc067490 con 0x7f6acc06c890 2023-10-08T05:37:26.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.328+0000 7f6ac2ffd640 1 -- 172.21.15.70:0/1907942649 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6ac800bd10 con 0x7f6acc06c890 2023-10-08T05:37:26.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.329+0000 7f6ac2ffd640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ab006ae70 0x7f6ab006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.329+0000 7f6ac2ffd640 1 -- 172.21.15.70:0/1907942649 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f6ac8089f60 con 0x7f6acc06c890 2023-10-08T05:37:26.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.330+0000 7f6ad189b640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ab006ae70 0x7f6ab006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:26.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.331+0000 7f6ad189b640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ab006ae70 0x7f6ab006d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6acc153a30 tx=0x7f6abc0079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:26.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.333+0000 7f6ac2ffd640 1 -- 172.21.15.70:0/1907942649 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ac8054c40 con 0x7f6acc06c890 2023-10-08T05:37:26.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.542+0000 7f6ad4300640 1 -- 172.21.15.70:0/1907942649 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6acc070d50 con 0x7f6acc06c890 2023-10-08T05:37:26.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.543+0000 7f6ac2ffd640 1 -- 172.21.15.70:0/1907942649 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f6ac8058260 con 0x7f6acc06c890 2023-10-08T05:37:26.544 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:26.544 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:36:11.613175+0000","modified":"2023-10-08T05:36:33.450447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T05:37:26.544 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 9 2023-10-08T05:37:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 -- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ab006ae70 msgr2=0x7f6ab006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ab006ae70 0x7f6ab006d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6acc153a30 tx=0x7f6abc0079c0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 -- 172.21.15.70:0/1907942649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 msgr2=0x7f6acc0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 0x7f6acc0caa90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800d800 tx=0x7f6ac800dcc0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 -- 172.21.15.70:0/1907942649 shutdown_connections 2023-10-08T05:37:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f6ab006ae70 0x7f6ab006d330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6acc160940 0x7f6acc15d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6acc15cd90 0x7f6acc0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 --2- 172.21.15.70:0/1907942649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6acc06c890 0x7f6acc0caa90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 -- 172.21.15.70:0/1907942649 >> 172.21.15.70:0/1907942649 conn(0x7f6acc06a380 msgr2=0x7f6acc157550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 -- 172.21.15.70:0/1907942649 shutdown_connections 2023-10-08T05:37:26.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.546+0000 7f6ad4300640 1 -- 172.21.15.70:0/1907942649 wait complete. 2023-10-08T05:37:26.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:26.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.773+0000 7fac3cb2c640 1 Processor -- start 2023-10-08T05:37:26.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.773+0000 7fac3cb2c640 1 -- start start 2023-10-08T05:37:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.775+0000 7fac3cb2c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380c90b0 0x7fac380cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.775+0000 7fac3cb2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac380cb3b0 0x7fac3815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.775+0000 7fac3cb2c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac3815b2b0 con 0x7fac3815ab40 2023-10-08T05:37:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.775+0000 7fac3cb2c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3815bf30 con 0x7fac380cb3b0 2023-10-08T05:37:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.775+0000 7fac3cb2c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac380cb840 con 0x7fac380c90b0 2023-10-08T05:37:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.775+0000 7fac36d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac380cb3b0 0x7fac3815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.775+0000 7fac36d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac380cb3b0 0x7fac3815e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55522/0 (socket says 172.21.15.70:55522) 2023-10-08T05:37:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.775+0000 7fac36d9d640 1 -- 172.21.15.70:0/189115513 learned_addr learned my addr 172.21.15.70:0/189115513 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.776+0000 7fac35d9b640 1 --2- 172.21.15.70:0/189115513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380c90b0 0x7fac380cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:26.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.776+0000 7fac36d9d640 1 -- 172.21.15.70:0/189115513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380c90b0 msgr2=0x7fac380cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:26.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.776+0000 7fac36d9d640 1 --2- 172.21.15.70:0/189115513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380c90b0 0x7fac380cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.776+0000 7fac36d9d640 1 -- 172.21.15.70:0/189115513 >> v1:172.21.15.181:6789/0 conn(0x7fac3815ab40 legacy=0x7fac380ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.776+0000 7fac36d9d640 1 -- 172.21.15.70:0/189115513 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3815b590 con 0x7fac380cb3b0 2023-10-08T05:37:26.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.777+0000 7fac36d9d640 1 --2- 172.21.15.70:0/189115513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380cb3b0 0x7fac3815e790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac30009e50 tx=0x7fac3002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09c597ecf01422f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:26.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.777+0000 7fac3559a640 1 -- 172.21.15.70:0/189115513 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3002cb00 con 0x7fac380cb3b0 2023-10-08T05:37:26.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.778+0000 7fac3559a640 1 -- 172.21.15.70:0/189115513 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3002cc60 con 0x7fac380cb3b0 2023-10-08T05:37:26.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.778+0000 7fac3559a640 1 -- 172.21.15.70:0/189115513 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac300343f0 con 0x7fac380cb3b0 2023-10-08T05:37:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.778+0000 7fac3cb2c640 1 -- 172.21.15.70:0/189115513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380cb3b0 msgr2=0x7fac3815e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.778+0000 7fac3cb2c640 1 --2- 172.21.15.70:0/189115513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380cb3b0 0x7fac3815e790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fac30009e50 tx=0x7fac3002bf60 comp rx=0 tx=0).stop 2023-10-08T05:37:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.778+0000 7fac3cb2c640 1 -- 172.21.15.70:0/189115513 shutdown_connections 2023-10-08T05:37:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.778+0000 7fac3cb2c640 1 --2- 172.21.15.70:0/189115513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380cb3b0 0x7fac3815e790 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.778+0000 7fac3cb2c640 1 --2- 172.21.15.70:0/189115513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380c90b0 0x7fac380cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.778+0000 7fac3cb2c640 1 -- 172.21.15.70:0/189115513 >> 172.21.15.70:0/189115513 conn(0x7fac3806a380 msgr2=0x7fac380b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:26.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.778+0000 7fac3cb2c640 1 -- 172.21.15.70:0/189115513 shutdown_connections 2023-10-08T05:37:26.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.779+0000 7fac3cb2c640 1 -- 172.21.15.70:0/189115513 wait complete. 2023-10-08T05:37:26.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.781+0000 7fac3cb2c640 1 Processor -- start 2023-10-08T05:37:26.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.781+0000 7fac3cb2c640 1 -- start start 2023-10-08T05:37:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.782+0000 7fac3cb2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380c90b0 0x7fac380ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.782+0000 7fac3cb2c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380cb3b0 0x7fac380ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.782+0000 7fac3cb2c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac3815ab40 0x7fac38155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.782+0000 7fac3cb2c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac380acb20 con 0x7fac380c90b0 2023-10-08T05:37:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.782+0000 7fac3cb2c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac380ad960 con 0x7fac3815ab40 2023-10-08T05:37:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.782+0000 7fac3cb2c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac380abea0 con 0x7fac380cb3b0 2023-10-08T05:37:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.782+0000 7fac3659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380c90b0 0x7fac380ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac3659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380c90b0 0x7fac380ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55524/0 (socket says 172.21.15.70:55524) 2023-10-08T05:37:26.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac3659c640 1 -- 172.21.15.70:0/1189640418 learned_addr learned my addr 172.21.15.70:0/1189640418 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:26.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac35d9b640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380cb3b0 0x7fac380ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:26.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac3659c640 1 -- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380cb3b0 msgr2=0x7fac380ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac36d9d640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac3815ab40 0x7fac38155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:26.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac3659c640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380cb3b0 0x7fac380ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac3659c640 1 -- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac3815ab40 msgr2=0x7fac38155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac3659c640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac3815ab40 0x7fac38155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac3659c640 1 -- 172.21.15.70:0/1189640418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac30009b30 con 0x7fac380c90b0 2023-10-08T05:37:26.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac35d9b640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380cb3b0 0x7fac380ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:26.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.783+0000 7fac3659c640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380c90b0 0x7fac380ab8c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fac2800c480 tx=0x7fac2800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:26.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.784+0000 7fac26ffd640 1 -- 172.21.15.70:0/1189640418 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac280052d0 con 0x7fac380c90b0 2023-10-08T05:37:26.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.784+0000 7fac26ffd640 1 -- 172.21.15.70:0/1189640418 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac28005430 con 0x7fac380c90b0 2023-10-08T05:37:26.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.784+0000 7fac26ffd640 1 -- 172.21.15.70:0/1189640418 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac28012870 con 0x7fac380c90b0 2023-10-08T05:37:26.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.784+0000 7fac3cb2c640 1 -- 172.21.15.70:0/1189640418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac380ac180 con 0x7fac380c90b0 2023-10-08T05:37:26.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.784+0000 7fac3cb2c640 1 -- 172.21.15.70:0/1189640418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac380ac740 con 0x7fac380c90b0 2023-10-08T05:37:26.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.785+0000 7fac3cb2c640 1 -- 172.21.15.70:0/1189640418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac38067490 con 0x7fac380c90b0 2023-10-08T05:37:26.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.786+0000 7fac26ffd640 1 -- 172.21.15.70:0/1189640418 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac280129d0 con 0x7fac380c90b0 2023-10-08T05:37:26.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.787+0000 7fac26ffd640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac1806ae70 0x7fac1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:26.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.787+0000 7fac26ffd640 1 -- 172.21.15.70:0/1189640418 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fac28089500 con 0x7fac380c90b0 2023-10-08T05:37:26.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.788+0000 7fac35d9b640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac1806ae70 0x7fac1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:26.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.789+0000 7fac35d9b640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac1806ae70 0x7fac1806d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fac2c006fd0 tx=0x7fac2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:26.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.792+0000 7fac26ffd640 1 -- 172.21.15.70:0/1189640418 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac28054280 con 0x7fac380c90b0 2023-10-08T05:37:26.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.986+0000 7fac3cb2c640 1 -- 172.21.15.70:0/1189640418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac3806d6e0 con 0x7fac380c90b0 2023-10-08T05:37:26.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.988+0000 7fac26ffd640 1 -- 172.21.15.70:0/1189640418 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7fac280578a0 con 0x7fac380c90b0 2023-10-08T05:37:26.989 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:26.989 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:26.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 -- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac1806ae70 msgr2=0x7fac1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac1806ae70 0x7fac1806d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fac2c006fd0 tx=0x7fac2c008040 comp rx=0 tx=0).stop 2023-10-08T05:37:26.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 -- 172.21.15.70:0/1189640418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380c90b0 msgr2=0x7fac380ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:26.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380c90b0 0x7fac380ab8c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fac2800c480 tx=0x7fac2800c940 comp rx=0 tx=0).stop 2023-10-08T05:37:26.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 -- 172.21.15.70:0/1189640418 shutdown_connections 2023-10-08T05:37:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac1806ae70 0x7fac1806d330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac3815ab40 0x7fac38155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac380cb3b0 0x7fac380ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 --2- 172.21.15.70:0/1189640418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac380c90b0 0x7fac380ab8c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 -- 172.21.15.70:0/1189640418 >> 172.21.15.70:0/1189640418 conn(0x7fac3806a380 msgr2=0x7fac380af560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:26.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 -- 172.21.15.70:0/1189640418 shutdown_connections 2023-10-08T05:37:26.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:26.991+0000 7fac3cb2c640 1 -- 172.21.15.70:0/1189640418 wait complete. 2023-10-08T05:37:27.010 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:36:09.357824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:36:10.614183+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-08T05:36:14.220463+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T05:37:27.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:27.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.219+0000 7fd8c5090640 1 Processor -- start 2023-10-08T05:37:27.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.219+0000 7fd8c5090640 1 -- start start 2023-10-08T05:37:27.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.221+0000 7fd8c5090640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8c015aad0 0x7fd8c00b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.221+0000 7fd8c5090640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c015bf30 con 0x7fd8c00c82b0 2023-10-08T05:37:27.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.221+0000 7fd8c5090640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c015b320 con 0x7fd8c006d410 2023-10-08T05:37:27.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.221+0000 7fd8c5090640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c015b600 con 0x7fd8c015aad0 2023-10-08T05:37:27.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.221+0000 7fd8b659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd8c00c82b0 0x7fd8c006cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57158/0 (socket says 172.21.15.70:57158) 2023-10-08T05:37:27.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.221+0000 7fd8b659c640 1 -- 172.21.15.70:0/3410605233 learned_addr learned my addr 172.21.15.70:0/3410605233 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:27.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8bed9d640 1 --2- 172.21.15.70:0/3410605233 >> v2:172.21.15.181:3300/0 conn(0x7fd8c015aad0 0x7fd8c00b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3562474656 0 0) 0x7fd8c015bf30 con 0x7fd8c00c82b0 2023-10-08T05:37:27.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a0003520 con 0x7fd8c00c82b0 2023-10-08T05:37:27.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4117084007 0 0) 0x7fd8a0003520 con 0x7fd8c00c82b0 2023-10-08T05:37:27.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 >> v2:172.21.15.181:3300/0 conn(0x7fd8c015aad0 msgr2=0x7fd8c00b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 --2- 172.21.15.70:0/3410605233 >> v2:172.21.15.181:3300/0 conn(0x7fd8c015aad0 0x7fd8c00b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 >> v1:172.21.15.181:6789/0 conn(0x7fd8c006d410 legacy=0x7fd8c0070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c015b8e0 con 0x7fd8c00c82b0 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd8a8001690 con 0x7fd8c00c82b0 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 >> v1:172.21.15.70:6789/0 conn(0x7fd8c00c82b0 legacy=0x7fd8c006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 --2- 172.21.15.70:0/3410605233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8a0003970 0x7fd8a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.222+0000 7fd8be59c640 1 --2- 172.21.15.70:0/3410605233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a00047c0 0x7fd8a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8be59c640 1 --2- 172.21.15.70:0/3410605233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a0005180 0x7fd8a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c015b600 con 0x7fd8a00047c0 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a000c0d0 con 0x7fd8a0005180 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a000c290 con 0x7fd8a0003970 2023-10-08T05:37:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8bed9d640 1 --2- 172.21.15.70:0/3410605233 >> v2:172.21.15.181:3300/0 conn(0x7fd8c015aad0 0x7fd8c00b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:37:27.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8b659c640 1 --2- 172.21.15.70:0/3410605233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a00047c0 0x7fd8a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8b659c640 1 -- 172.21.15.70:0/3410605233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8a0003970 msgr2=0x7fd8a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:27.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8b659c640 1 --2- 172.21.15.70:0/3410605233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8a0003970 0x7fd8a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8b659c640 1 -- 172.21.15.70:0/3410605233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a0005180 msgr2=0x7fd8a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:27.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8b659c640 1 --2- 172.21.15.70:0/3410605233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a0005180 0x7fd8a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8b659c640 1 -- 172.21.15.70:0/3410605233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0003520 con 0x7fd8a00047c0 2023-10-08T05:37:27.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.223+0000 7fd8bf59e640 1 -- 172.21.15.70:0/3410605233 reap_dead start 2023-10-08T05:37:27.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.224+0000 7fd8b659c640 1 --2- 172.21.15.70:0/3410605233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a00047c0 0x7fd8a0004c40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c210 tx=0x7fd8a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ccfa189d3c3123 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:27.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.225+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8005ad0 con 0x7fd8a00047c0 2023-10-08T05:37:27.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.225+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8005c30 con 0x7fd8a00047c0 2023-10-08T05:37:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.225+0000 7fd8be59c640 1 -- 172.21.15.70:0/3410605233 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a802c870 con 0x7fd8a00047c0 2023-10-08T05:37:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.225+0000 7fd8c5090640 1 -- 172.21.15.70:0/3410605233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a00047c0 msgr2=0x7fd8a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.225+0000 7fd8c5090640 1 --2- 172.21.15.70:0/3410605233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a00047c0 0x7fd8a0004c40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c210 tx=0x7fd8a00057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.225+0000 7fd8c5090640 1 -- 172.21.15.70:0/3410605233 shutdown_connections 2023-10-08T05:37:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.225+0000 7fd8c5090640 1 --2- 172.21.15.70:0/3410605233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8a00047c0 0x7fd8a0004c40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.225+0000 7fd8c5090640 1 -- 172.21.15.70:0/3410605233 >> 172.21.15.70:0/3410605233 conn(0x7fd8c0069d50 msgr2=0x7fd8c006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.225+0000 7fd8c5090640 1 -- 172.21.15.70:0/3410605233 shutdown_connections 2023-10-08T05:37:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.225+0000 7fd8c5090640 1 -- 172.21.15.70:0/3410605233 wait complete. 2023-10-08T05:37:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.227+0000 7fd8c5090640 1 Processor -- start 2023-10-08T05:37:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.227+0000 7fd8c5090640 1 -- start start 2023-10-08T05:37:27.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.228+0000 7fd8c5090640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a00047c0 0x7fd8c00b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.228+0000 7fd8c5090640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c0151f40 0x7fd8c01523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.228+0000 7fd8c5090640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c00c8dd0 0x7fd8c00c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.228+0000 7fd8c5090640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0152900 con 0x7fd8c0151f40 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.228+0000 7fd8c5090640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00c9b00 con 0x7fd8a00047c0 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.228+0000 7fd8c5090640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00c9de0 con 0x7fd8c00c8dd0 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.228+0000 7fd8b659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c0151f40 0x7fd8c01523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.228+0000 7fd8b659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c0151f40 0x7fd8c01523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55550/0 (socket says 172.21.15.70:55550) 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.228+0000 7fd8b659c640 1 -- 172.21.15.70:0/923753836 learned_addr learned my addr 172.21.15.70:0/923753836 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.229+0000 7fd8bed9d640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a00047c0 0x7fd8c00b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.229+0000 7fd8bf59e640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c00c8dd0 0x7fd8c00c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.229+0000 7fd8b659c640 1 -- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c00c8dd0 msgr2=0x7fd8c00c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.229+0000 7fd8b659c640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c00c8dd0 0x7fd8c00c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.229+0000 7fd8b659c640 1 -- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a00047c0 msgr2=0x7fd8c00b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.229+0000 7fd8b659c640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a00047c0 0x7fd8c00b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.229+0000 7fd8b659c640 1 -- 172.21.15.70:0/923753836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00ca0c0 con 0x7fd8c0151f40 2023-10-08T05:37:27.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.229+0000 7fd8b659c640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c0151f40 0x7fd8c01523c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8001950 tx=0x7fd8c015bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:27.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.229+0000 7fd8b7fff640 1 -- 172.21.15.70:0/923753836 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a803c070 con 0x7fd8c0151f40 2023-10-08T05:37:27.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.230+0000 7fd8b7fff640 1 -- 172.21.15.70:0/923753836 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a802ca80 con 0x7fd8c0151f40 2023-10-08T05:37:27.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.230+0000 7fd8c5090640 1 -- 172.21.15.70:0/923753836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c00ca3a0 con 0x7fd8c0151f40 2023-10-08T05:37:27.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.230+0000 7fd8b7fff640 1 -- 172.21.15.70:0/923753836 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a802ce30 con 0x7fd8c0151f40 2023-10-08T05:37:27.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.230+0000 7fd8c5090640 1 -- 172.21.15.70:0/923753836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c00c9790 con 0x7fd8c0151f40 2023-10-08T05:37:27.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.231+0000 7fd8c5090640 1 -- 172.21.15.70:0/923753836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c006a660 con 0x7fd8c0151f40 2023-10-08T05:37:27.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.232+0000 7fd8b7fff640 1 -- 172.21.15.70:0/923753836 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fd8a8032070 con 0x7fd8c0151f40 2023-10-08T05:37:27.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.233+0000 7fd8b7fff640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd8a406aea0 0x7fd8a406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.233+0000 7fd8b7fff640 1 -- 172.21.15.70:0/923753836 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fd8a80aa920 con 0x7fd8c0151f40 2023-10-08T05:37:27.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.234+0000 7fd8bed9d640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd8a406aea0 0x7fd8a406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.235+0000 7fd8bed9d640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd8a406aea0 0x7fd8a406d360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0006fd0 tx=0x7fd8b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:27.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.237+0000 7fd8b7fff640 1 -- 172.21.15.70:0/923753836 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8a8075570 con 0x7fd8c0151f40 2023-10-08T05:37:27.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.428+0000 7fd8c5090640 1 -- 172.21.15.70:0/923753836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8c00c9f20 con 0x7fd8c0151f40 2023-10-08T05:37:27.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.429+0000 7fd8b7fff640 1 -- 172.21.15.70:0/923753836 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8a8045020 con 0x7fd8c0151f40 2023-10-08T05:37:27.430 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:27.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 -- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd8a406aea0 msgr2=0x7fd8a406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd8a406aea0 0x7fd8a406d360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0006fd0 tx=0x7fd8b0008040 comp rx=0 tx=0).stop 2023-10-08T05:37:27.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 -- 172.21.15.70:0/923753836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c0151f40 msgr2=0x7fd8c01523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c0151f40 0x7fd8c01523c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8001950 tx=0x7fd8c015bea0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 -- 172.21.15.70:0/923753836 shutdown_connections 2023-10-08T05:37:27.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fd8a406aea0 0x7fd8a406d360 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8c00c8dd0 0x7fd8c00c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8c0151f40 0x7fd8c01523c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 --2- 172.21.15.70:0/923753836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8a00047c0 0x7fd8c00b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 -- 172.21.15.70:0/923753836 >> 172.21.15.70:0/923753836 conn(0x7fd8c0069d50 msgr2=0x7fd8c0159b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:27.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 -- 172.21.15.70:0/923753836 shutdown_connections 2023-10-08T05:37:27.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.432+0000 7fd8c5090640 1 -- 172.21.15.70:0/923753836 wait complete. 2023-10-08T05:37:27.450 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:37:27.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.663+0000 7f4bffb5e640 1 Processor -- start 2023-10-08T05:37:27.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.663+0000 7f4bffb5e640 1 -- start start 2023-10-08T05:37:27.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.664+0000 7f4bffb5e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf806c3a0 0x7f4bf80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.664+0000 7f4bffb5e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4bf806db60 0x7f4bf806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.664+0000 7f4bffb5e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf815b310 con 0x7f4bf815aa00 2023-10-08T05:37:27.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.664+0000 7f4bffb5e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf80ca230 con 0x7f4bf806db60 2023-10-08T05:37:27.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.664+0000 7f4bffb5e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf806d290 con 0x7f4bf806c3a0 2023-10-08T05:37:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfe0fb640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4bf815aa00 0x7f4bf80b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57170/0 (socket says 172.21.15.70:57170) 2023-10-08T05:37:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfe0fb640 1 -- 172.21.15.70:0/3530860731 learned_addr learned my addr 172.21.15.70:0/3530860731 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113206291 0 0) 0x7f4bf0001490 con 0x7f4bf815aa00 2023-10-08T05:37:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be4003520 con 0x7f4bf815aa00 2023-10-08T05:37:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfd8fa640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf806c3a0 0x7f4bf80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfd0f9640 1 --2- 172.21.15.70:0/3530860731 >> v2:172.21.15.181:3300/0 conn(0x7f4bf806db60 0x7f4bf806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1362831110 0 0) 0x7f4be4003520 con 0x7f4bf815aa00 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf806c3a0 msgr2=0x7f4bf80c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf806c3a0 0x7f4bf80c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 >> v2:172.21.15.181:3300/0 conn(0x7f4bf806db60 msgr2=0x7f4bf806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 --2- 172.21.15.70:0/3530860731 >> v2:172.21.15.181:3300/0 conn(0x7f4bf806db60 0x7f4bf806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf80c9860 con 0x7f4bf815aa00 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4bf0001b80 con 0x7f4bf815aa00 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 >> v1:172.21.15.70:6789/0 conn(0x7f4bf815aa00 legacy=0x7f4bf80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be4003970 0x7f4be4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.665+0000 7f4bfc8f8640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf806c3a0 0x7f4be40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfc8f8640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4be4004d00 0x7f4be4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf806d290 con 0x7f4be4004d00 2023-10-08T05:37:27.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be400ba30 con 0x7f4be4003970 2023-10-08T05:37:27.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be400bbf0 con 0x7f4bf806c3a0 2023-10-08T05:37:27.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfe0fb640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4be4004d00 0x7f4be4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfd8fa640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be4003970 0x7f4be4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfd0f9640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf806c3a0 0x7f4be40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfe0fb640 1 -- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf806c3a0 msgr2=0x7f4be40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfe0fb640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf806c3a0 0x7f4be40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfe0fb640 1 -- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be4003970 msgr2=0x7f4be4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfe0fb640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be4003970 0x7f4be4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.666+0000 7f4bfe0fb640 1 -- 172.21.15.70:0/3530860731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be4003520 con 0x7f4be4004d00 2023-10-08T05:37:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.667+0000 7f4bfe0fb640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4be4004d00 0x7f4be4005180 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000a760 tx=0x7f4be40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52a8cdae4643373 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.668+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0034630 con 0x7f4be4004d00 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.668+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf002f070 con 0x7f4be4004d00 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.668+0000 7f4bfc8f8640 1 -- 172.21.15.70:0/3530860731 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf00349f0 con 0x7f4be4004d00 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.668+0000 7f4bffb5e640 1 -- 172.21.15.70:0/3530860731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4be4004d00 msgr2=0x7f4be4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.668+0000 7f4bffb5e640 1 --2- 172.21.15.70:0/3530860731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4be4004d00 0x7f4be4005180 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000a760 tx=0x7f4be40057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.668+0000 7f4bfe0fb640 1 -- 172.21.15.70:0/3530860731 reap_dead start 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.668+0000 7f4bffb5e640 1 -- 172.21.15.70:0/3530860731 shutdown_connections 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.668+0000 7f4bffb5e640 1 -- 172.21.15.70:0/3530860731 >> 172.21.15.70:0/3530860731 conn(0x7f4bf806a5d0 msgr2=0x7f4bf80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.668+0000 7f4bffb5e640 1 -- 172.21.15.70:0/3530860731 shutdown_connections 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.668+0000 7f4bffb5e640 1 -- 172.21.15.70:0/3530860731 wait complete. 2023-10-08T05:37:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.670+0000 7f4bffb5e640 1 Processor -- start 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.670+0000 7f4bffb5e640 1 -- start start 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.671+0000 7f4bffb5e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be4004d00 0x7f4bf81528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.671+0000 7f4bffb5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bf8153790 0x7f4bf8153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.671+0000 7f4bffb5e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf80c7440 0x7f4bf80c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.671+0000 7f4bffb5e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8167350 con 0x7f4bf8153790 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.671+0000 7f4bffb5e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf8154150 con 0x7f4be4004d00 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.671+0000 7f4bffb5e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf80c7e00 con 0x7f4bf80c7440 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.671+0000 7f4bfd0f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bf8153790 0x7f4bf8153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.671+0000 7f4bfd0f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bf8153790 0x7f4bf8153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55576/0 (socket says 172.21.15.70:55576) 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.671+0000 7f4bfd0f9640 1 -- 172.21.15.70:0/1597042049 learned_addr learned my addr 172.21.15.70:0/1597042049 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bfd8fa640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be4004d00 0x7f4bf81528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bfd0f9640 1 -- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf80c7440 msgr2=0x7f4bf80c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bfd0f9640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf80c7440 0x7f4bf80c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bfd0f9640 1 -- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be4004d00 msgr2=0x7f4bf81528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bfd0f9640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be4004d00 0x7f4bf81528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bfd0f9640 1 -- 172.21.15.70:0/1597042049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf8153590 con 0x7f4bf8153790 2023-10-08T05:37:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bfd0f9640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bf8153790 0x7f4bf8153c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4be800f630 tx=0x7f4be800faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bee7fc640 1 -- 172.21.15.70:0/1597042049 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8014870 con 0x7f4bf8153790 2023-10-08T05:37:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bee7fc640 1 -- 172.21.15.70:0/1597042049 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be80149d0 con 0x7f4bf8153790 2023-10-08T05:37:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bffb5e640 1 -- 172.21.15.70:0/1597042049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf80c8020 con 0x7f4bf8153790 2023-10-08T05:37:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.672+0000 7f4bee7fc640 1 -- 172.21.15.70:0/1597042049 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8014d40 con 0x7f4bf8153790 2023-10-08T05:37:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.673+0000 7f4bffb5e640 1 -- 172.21.15.70:0/1597042049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf8166c90 con 0x7f4bf8153790 2023-10-08T05:37:27.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.674+0000 7f4bffb5e640 1 -- 172.21.15.70:0/1597042049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf8167490 con 0x7f4bf8153790 2023-10-08T05:37:27.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.674+0000 7f4bee7fc640 1 -- 172.21.15.70:0/1597042049 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4be8015410 con 0x7f4bf8153790 2023-10-08T05:37:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.675+0000 7f4bee7fc640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4bdc06aea0 0x7f4bdc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.675+0000 7f4bee7fc640 1 -- 172.21.15.70:0/1597042049 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f4be808a3d0 con 0x7f4bf8153790 2023-10-08T05:37:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.676+0000 7f4bfd8fa640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4bdc06aea0 0x7f4bdc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.677+0000 7f4bfd8fa640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4bdc06aea0 0x7f4bdc06d360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4006fd0 tx=0x7f4bf4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:27.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.680+0000 7f4bee7fc640 1 -- 172.21.15.70:0/1597042049 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4bf8167490 con 0x7f4bf8153790 2023-10-08T05:37:27.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.866+0000 7f4bffb5e640 1 -- 172.21.15.70:0/1597042049 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4bf8167490 con 0x7f4bf8153790 2023-10-08T05:37:27.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.867+0000 7f4bee7fc640 1 -- 172.21.15.70:0/1597042049 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4bf8167490 con 0x7f4bf8153790 2023-10-08T05:37:27.868 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:27.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 -- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4bdc06aea0 msgr2=0x7f4bdc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4bdc06aea0 0x7f4bdc06d360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4006fd0 tx=0x7f4bf4008040 comp rx=0 tx=0).stop 2023-10-08T05:37:27.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 -- 172.21.15.70:0/1597042049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bf8153790 msgr2=0x7f4bf8153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:27.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bf8153790 0x7f4bf8153c10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4be800f630 tx=0x7f4be800faf0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 -- 172.21.15.70:0/1597042049 shutdown_connections 2023-10-08T05:37:27.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4bdc06aea0 0x7f4bdc06d360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4bf80c7440 0x7f4bf80c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bf8153790 0x7f4bf8153c10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 --2- 172.21.15.70:0/1597042049 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4be4004d00 0x7f4bf81528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:27.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 -- 172.21.15.70:0/1597042049 >> 172.21.15.70:0/1597042049 conn(0x7f4bf806a5d0 msgr2=0x7f4bf80c9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:27.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 -- 172.21.15.70:0/1597042049 shutdown_connections 2023-10-08T05:37:27.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:27.870+0000 7f4bffb5e640 1 -- 172.21.15.70:0/1597042049 wait complete. 2023-10-08T05:37:27.890 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:37:28.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.108+0000 7fbaf61a0640 1 Processor -- start 2023-10-08T05:37:28.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.108+0000 7fbaf61a0640 1 -- start start 2023-10-08T05:37:28.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.110+0000 7fbaf61a0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbaf00ca680 0x7fbaf015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.110+0000 7fbaf61a0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf00cab60 con 0x7fbaf015a820 2023-10-08T05:37:28.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.110+0000 7fbaf61a0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf00cb960 con 0x7fbaf00b0680 2023-10-08T05:37:28.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.110+0000 7fbaf61a0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf00cbdf0 con 0x7fbaf00ca680 2023-10-08T05:37:28.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.110+0000 7fbaf519e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbaf015a820 0x7fbaf00aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57176/0 (socket says 172.21.15.70:57176) 2023-10-08T05:37:28.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.110+0000 7fbaf519e640 1 -- 172.21.15.70:0/1430754076 learned_addr learned my addr 172.21.15.70:0/1430754076 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.110+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248714018 0 0) 0x7fbaf00cab60 con 0x7fbaf015a820 2023-10-08T05:37:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbadc003520 con 0x7fbaf015a820 2023-10-08T05:37:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbaf599f640 1 --2- 172.21.15.70:0/1430754076 >> v2:172.21.15.181:3300/0 conn(0x7fbaf00ca680 0x7fbaf015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1998710488 0 0) 0x7fbadc003520 con 0x7fbaf015a820 2023-10-08T05:37:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 >> v2:172.21.15.181:3300/0 conn(0x7fbaf00ca680 msgr2=0x7fbaf015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 --2- 172.21.15.70:0/1430754076 >> v2:172.21.15.181:3300/0 conn(0x7fbaf00ca680 0x7fbaf015ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 >> v1:172.21.15.181:6789/0 conn(0x7fbaf00b0680 legacy=0x7fbaf015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf00cb030 con 0x7fbaf015a820 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fbaec001670 con 0x7fbaf015a820 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 >> v1:172.21.15.70:6789/0 conn(0x7fbaf015a820 legacy=0x7fbaf00aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbadc003970 0x7fbadc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0047c0 0x7fbadc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbadc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00cbdf0 con 0x7fbadc0052a0 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc00c170 con 0x7fbadc0047c0 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbadc00c330 con 0x7fbadc003970 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.111+0000 7fbaf599f640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbadc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.112+0000 7fbaf499d640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0047c0 0x7fbadc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.112+0000 7fbaf519e640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbadc003970 0x7fbadc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.112+0000 7fbaf599f640 1 -- 172.21.15.70:0/1430754076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbadc003970 msgr2=0x7fbadc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.112+0000 7fbaf599f640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbadc003970 0x7fbadc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.112+0000 7fbaf599f640 1 -- 172.21.15.70:0/1430754076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0047c0 msgr2=0x7fbadc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.112+0000 7fbaf599f640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbadc0047c0 0x7fbadc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.112+0000 7fbaf599f640 1 -- 172.21.15.70:0/1430754076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbadc003520 con 0x7fbadc0052a0 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.112+0000 7fbaf519e640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbadc003970 0x7fbadc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.113+0000 7fbaf599f640 1 -- 172.21.15.70:0/1430754076 reap_dead start 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.113+0000 7fbaf599f640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbadc00b980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00caa90 tx=0x7fbadc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519eeffd6cc17219 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.113+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae802cc70 con 0x7fbadc0052a0 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.113+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae802cdd0 con 0x7fbadc0052a0 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.113+0000 7fbae7fff640 1 -- 172.21.15.70:0/1430754076 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8034680 con 0x7fbadc0052a0 2023-10-08T05:37:28.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.114+0000 7fbaf61a0640 1 -- 172.21.15.70:0/1430754076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 msgr2=0x7fbadc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.114+0000 7fbaf61a0640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbadc00b980 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00caa90 tx=0x7fbadc0057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.114+0000 7fbaf61a0640 1 -- 172.21.15.70:0/1430754076 shutdown_connections 2023-10-08T05:37:28.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.114+0000 7fbaf61a0640 1 --2- 172.21.15.70:0/1430754076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbadc00b980 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.114+0000 7fbaf61a0640 1 -- 172.21.15.70:0/1430754076 >> 172.21.15.70:0/1430754076 conn(0x7fbaf006c050 msgr2=0x7fbaf006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:28.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.114+0000 7fbaf61a0640 1 -- 172.21.15.70:0/1430754076 shutdown_connections 2023-10-08T05:37:28.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.114+0000 7fbaf61a0640 1 -- 172.21.15.70:0/1430754076 wait complete. 2023-10-08T05:37:28.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.116+0000 7fbaf61a0640 1 Processor -- start 2023-10-08T05:37:28.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.116+0000 7fbaf61a0640 1 -- start start 2023-10-08T05:37:28.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.117+0000 7fbaf61a0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbaf014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.117+0000 7fbaf61a0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaf00c6580 0x7fbaf00c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.117+0000 7fbaf61a0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf00c2020 0x7fbaf00c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.117+0000 7fbaf61a0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00c8040 con 0x7fbadc0052a0 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.117+0000 7fbaf61a0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00c6a60 con 0x7fbaf00c2020 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.117+0000 7fbaf61a0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf00c6d40 con 0x7fbaf00c6580 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.117+0000 7fbaf519e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbaf014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.117+0000 7fbaf519e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbaf014fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55596/0 (socket says 172.21.15.70:55596) 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.117+0000 7fbaf519e640 1 -- 172.21.15.70:0/2290562793 learned_addr learned my addr 172.21.15.70:0/2290562793 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.117+0000 7fbaf519e640 1 -- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaf00c6580 msgr2=0x7fbaf00c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbaf519e640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaf00c6580 0x7fbaf00c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbaf519e640 1 -- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf00c2020 msgr2=0x7fbaf00c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbaf599f640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf00c2020 0x7fbaf00c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbaf519e640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf00c2020 0x7fbaf00c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbaf519e640 1 -- 172.21.15.70:0/2290562793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf00c7020 con 0x7fbadc0052a0 2023-10-08T05:37:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbaf519e640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbaf014fb30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00d810 tx=0x7fbaec00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbae5ffb640 1 -- 172.21.15.70:0/2290562793 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0074e0 con 0x7fbadc0052a0 2023-10-08T05:37:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbae5ffb640 1 -- 172.21.15.70:0/2290562793 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec007640 con 0x7fbadc0052a0 2023-10-08T05:37:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbaf61a0640 1 -- 172.21.15.70:0/2290562793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf00c7300 con 0x7fbadc0052a0 2023-10-08T05:37:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbae5ffb640 1 -- 172.21.15.70:0/2290562793 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0049e0 con 0x7fbadc0052a0 2023-10-08T05:37:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.118+0000 7fbaf61a0640 1 -- 172.21.15.70:0/2290562793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf015d930 con 0x7fbadc0052a0 2023-10-08T05:37:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.119+0000 7fbaf61a0640 1 -- 172.21.15.70:0/2290562793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf00b12c0 con 0x7fbadc0052a0 2023-10-08T05:37:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.120+0000 7fbae5ffb640 1 -- 172.21.15.70:0/2290562793 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fbaec004b40 con 0x7fbadc0052a0 2023-10-08T05:37:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.121+0000 7fbae5ffb640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbad406ae70 0x7fbad406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.121+0000 7fbae5ffb640 1 -- 172.21.15.70:0/2290562793 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fbaec0893d0 con 0x7fbadc0052a0 2023-10-08T05:37:28.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.122+0000 7fbaf499d640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbad406ae70 0x7fbad406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.123+0000 7fbaf499d640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbad406ae70 0x7fbad406d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0153680 tx=0x7fbae0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:28.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.125+0000 7fbae5ffb640 1 -- 172.21.15.70:0/2290562793 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbaec054020 con 0x7fbadc0052a0 2023-10-08T05:37:28.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.322+0000 7fbaf61a0640 1 -- 172.21.15.70:0/2290562793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbaf0074150 con 0x7fbadc0052a0 2023-10-08T05:37:28.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.323+0000 7fbae5ffb640 1 -- 172.21.15.70:0/2290562793 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbaec057640 con 0x7fbadc0052a0 2023-10-08T05:37:28.324 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:37:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.326+0000 7fbaf61a0640 1 -- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbad406ae70 msgr2=0x7fbad406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.326+0000 7fbaf61a0640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbad406ae70 0x7fbad406d330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0153680 tx=0x7fbae0009040 comp rx=0 tx=0).stop 2023-10-08T05:37:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.326+0000 7fbaf61a0640 1 -- 172.21.15.70:0/2290562793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 msgr2=0x7fbaf014fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.326+0000 7fbaf61a0640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbaf014fb30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00d810 tx=0x7fbaec00dcd0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.326+0000 7fbaf61a0640 1 -- 172.21.15.70:0/2290562793 shutdown_connections 2023-10-08T05:37:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.326+0000 7fbaf61a0640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fbad406ae70 0x7fbad406d330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.326+0000 7fbaf61a0640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaf00c2020 0x7fbaf00c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.326+0000 7fbaf61a0640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaf00c6580 0x7fbaf00c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.326+0000 7fbaf61a0640 1 --2- 172.21.15.70:0/2290562793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbadc0052a0 0x7fbaf014fb30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.326+0000 7fbaf61a0640 1 -- 172.21.15.70:0/2290562793 >> 172.21.15.70:0/2290562793 conn(0x7fbaf006c050 msgr2=0x7fbaf00af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.327+0000 7fbaf61a0640 1 -- 172.21.15.70:0/2290562793 shutdown_connections 2023-10-08T05:37:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.327+0000 7fbaf61a0640 1 -- 172.21.15.70:0/2290562793 wait complete. 2023-10-08T05:37:28.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-08T05:37:28.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.559+0000 7f40c8475640 1 Processor -- start 2023-10-08T05:37:28.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.559+0000 7f40c8475640 1 -- start start 2023-10-08T05:37:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.561+0000 7f40c8475640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f40c00c9cf0 0x7f40c006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.561+0000 7f40c8475640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40c015aa00 0x7f40c00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.561+0000 7f40c8475640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c015b310 con 0x7f40c006c3a0 2023-10-08T05:37:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.561+0000 7f40c8475640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c006e0a0 con 0x7f40c00c9cf0 2023-10-08T05:37:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.561+0000 7f40c8475640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c006d290 con 0x7f40c015aa00 2023-10-08T05:37:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.561+0000 7f40c5a10640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f40c00c9cf0 0x7f40c006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.561+0000 7f40c5a10640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f40c00c9cf0 0x7f40c006db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55606/0 (socket says 172.21.15.70:55606) 2023-10-08T05:37:28.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.561+0000 7f40c5a10640 1 -- 172.21.15.70:0/2295815461 learned_addr learned my addr 172.21.15.70:0/2295815461 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:28.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.561+0000 7f40c6a12640 1 --2- 172.21.15.70:0/2295815461 >> v2:172.21.15.181:3300/0 conn(0x7f40c015aa00 0x7f40c00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.562+0000 7f40c5a10640 1 -- 172.21.15.70:0/2295815461 >> v2:172.21.15.181:3300/0 conn(0x7f40c015aa00 msgr2=0x7f40c00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:28.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.562+0000 7f40c5a10640 1 --2- 172.21.15.70:0/2295815461 >> v2:172.21.15.181:3300/0 conn(0x7f40c015aa00 0x7f40c00b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.562+0000 7f40c5a10640 1 -- 172.21.15.70:0/2295815461 >> v1:172.21.15.181:6789/0 conn(0x7f40c006c3a0 legacy=0x7f40c006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:28.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.562+0000 7f40c5a10640 1 -- 172.21.15.70:0/2295815461 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00c9860 con 0x7f40c00c9cf0 2023-10-08T05:37:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.563+0000 7f40c5a10640 1 --2- 172.21.15.70:0/2295815461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 0x7f40c006db60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f40c00c57a0 tx=0x7f40b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356ddd532a042d18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.563+0000 7f40c520f640 1 -- 172.21.15.70:0/2295815461 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0031870 con 0x7f40c00c9cf0 2023-10-08T05:37:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.563+0000 7f40c520f640 1 -- 172.21.15.70:0/2295815461 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b00319d0 con 0x7f40c00c9cf0 2023-10-08T05:37:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.563+0000 7f40c520f640 1 -- 172.21.15.70:0/2295815461 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0031d60 con 0x7f40c00c9cf0 2023-10-08T05:37:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.564+0000 7f40c8475640 1 -- 172.21.15.70:0/2295815461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 msgr2=0x7f40c006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.564+0000 7f40c8475640 1 --2- 172.21.15.70:0/2295815461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 0x7f40c006db60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f40c00c57a0 tx=0x7f40b002b910 comp rx=0 tx=0).stop 2023-10-08T05:37:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.564+0000 7f40c8475640 1 -- 172.21.15.70:0/2295815461 shutdown_connections 2023-10-08T05:37:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.564+0000 7f40c8475640 1 --2- 172.21.15.70:0/2295815461 >> v2:172.21.15.181:3300/0 conn(0x7f40c015aa00 0x7f40c00b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.564+0000 7f40c8475640 1 --2- 172.21.15.70:0/2295815461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 0x7f40c006db60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.564+0000 7f40c8475640 1 -- 172.21.15.70:0/2295815461 >> 172.21.15.70:0/2295815461 conn(0x7f40c006a5d0 msgr2=0x7f40c00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.564+0000 7f40c8475640 1 -- 172.21.15.70:0/2295815461 shutdown_connections 2023-10-08T05:37:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.564+0000 7f40c8475640 1 -- 172.21.15.70:0/2295815461 wait complete. 2023-10-08T05:37:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.566+0000 7f40c8475640 1 Processor -- start 2023-10-08T05:37:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.566+0000 7f40c8475640 1 -- start start 2023-10-08T05:37:28.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.567+0000 7f40c8475640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40c006c3a0 0x7f40c00c2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.567+0000 7f40c8475640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 0x7f40c00c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.567+0000 7f40c8475640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c015aa00 0x7f40c014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.567+0000 7f40c8475640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c015c120 con 0x7f40c00c9cf0 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.567+0000 7f40c8475640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c015c3b0 con 0x7f40c015aa00 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.567+0000 7f40c8475640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c015c690 con 0x7f40c006c3a0 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.567+0000 7f40c5a10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 0x7f40c00c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.567+0000 7f40c5a10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 0x7f40c00c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55610/0 (socket says 172.21.15.70:55610) 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.567+0000 7f40c5a10640 1 -- 172.21.15.70:0/3383600605 learned_addr learned my addr 172.21.15.70:0/3383600605 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40c6a12640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c015aa00 0x7f40c014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40c5a10640 1 -- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40c006c3a0 msgr2=0x7f40c00c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40c6211640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40c006c3a0 0x7f40c00c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40c5a10640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40c006c3a0 0x7f40c00c2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40c5a10640 1 -- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c015aa00 msgr2=0x7f40c014b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40c5a10640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c015aa00 0x7f40c014b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40c5a10640 1 -- 172.21.15.70:0/3383600605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00094d0 con 0x7f40c00c9cf0 2023-10-08T05:37:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40c6a12640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c015aa00 0x7f40c014b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40c5a10640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 0x7f40c00c4a80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f40b0000c00 tx=0x7f40b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40b6ffd640 1 -- 172.21.15.70:0/3383600605 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b003abe0 con 0x7f40c00c9cf0 2023-10-08T05:37:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40b6ffd640 1 -- 172.21.15.70:0/3383600605 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b003ad40 con 0x7f40c00c9cf0 2023-10-08T05:37:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40b6ffd640 1 -- 172.21.15.70:0/3383600605 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0044610 con 0x7f40c00c9cf0 2023-10-08T05:37:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.568+0000 7f40c8475640 1 -- 172.21.15.70:0/3383600605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c015c970 con 0x7f40c00c9cf0 2023-10-08T05:37:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.569+0000 7f40c8475640 1 -- 172.21.15.70:0/3383600605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c015cf30 con 0x7f40c00c9cf0 2023-10-08T05:37:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.570+0000 7f40c8475640 1 -- 172.21.15.70:0/3383600605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c004e160 con 0x7f40c00c9cf0 2023-10-08T05:37:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.571+0000 7f40b6ffd640 1 -- 172.21.15.70:0/3383600605 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f40b004d450 con 0x7f40c00c9cf0 2023-10-08T05:37:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.572+0000 7f40b6ffd640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f40a406ae70 0x7f40a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.572+0000 7f40b6ffd640 1 -- 172.21.15.70:0/3383600605 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f40b002d040 con 0x7f40c00c9cf0 2023-10-08T05:37:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.572+0000 7f40c6211640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f40a406ae70 0x7f40a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.574+0000 7f40c6211640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f40a406ae70 0x7f40a406d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f40bc003d90 tx=0x7f40bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:28.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.576+0000 7f40b6ffd640 1 -- 172.21.15.70:0/3383600605 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40b0076b90 con 0x7f40c00c9cf0 2023-10-08T05:37:28.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.762+0000 7f40c8475640 1 -- 172.21.15.70:0/3383600605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f40c015b310 con 0x7f40c00c9cf0 2023-10-08T05:37:28.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.763+0000 7f40b6ffd640 1 -- 172.21.15.70:0/3383600605 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f40c015b310 con 0x7f40c00c9cf0 2023-10-08T05:37:28.764 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:37:28.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 -- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f40a406ae70 msgr2=0x7f40a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f40a406ae70 0x7f40a406d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f40bc003d90 tx=0x7f40bc009040 comp rx=0 tx=0).stop 2023-10-08T05:37:28.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 -- 172.21.15.70:0/3383600605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 msgr2=0x7f40c00c4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:28.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 0x7f40c00c4a80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f40b0000c00 tx=0x7f40b00046c0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 -- 172.21.15.70:0/3383600605 shutdown_connections 2023-10-08T05:37:28.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f40a406ae70 0x7f40a406d330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40c015aa00 0x7f40c014b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c00c9cf0 0x7f40c00c4a80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 --2- 172.21.15.70:0/3383600605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40c006c3a0 0x7f40c00c2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:28.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 -- 172.21.15.70:0/3383600605 >> 172.21.15.70:0/3383600605 conn(0x7f40c006a5d0 msgr2=0x7f40c00cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:28.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 -- 172.21.15.70:0/3383600605 shutdown_connections 2023-10-08T05:37:28.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.766+0000 7f40c8475640 1 -- 172.21.15.70:0/3383600605 wait complete. 2023-10-08T05:37:28.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:37:28.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.996+0000 7f88b02b4640 1 Processor -- start 2023-10-08T05:37:28.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.996+0000 7f88b02b4640 1 -- start start 2023-10-08T05:37:28.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.998+0000 7f88b02b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88a815cd90 0x7f88a80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.998+0000 7f88b02b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88a806c890 0x7f88a806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:28.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.998+0000 7f88b02b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a80c48f0 con 0x7f88a8160940 2023-10-08T05:37:28.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.998+0000 7f88b02b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a80c4d80 con 0x7f88a815cd90 2023-10-08T05:37:28.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.998+0000 7f88b02b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a80c3ce0 con 0x7f88a806c890 2023-10-08T05:37:28.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.998+0000 7f88ae050640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88a815cd90 0x7f88a80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:28.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.998+0000 7f88ad84f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f88a8160940 0x7f88a8070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57186/0 (socket says 172.21.15.70:57186) 2023-10-08T05:37:28.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.998+0000 7f88ad84f640 1 -- 172.21.15.70:0/2652324218 learned_addr learned my addr 172.21.15.70:0/2652324218 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:29.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.999+0000 7f88ae851640 1 --2- 172.21.15.70:0/2652324218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88a806c890 0x7f88a806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.999+0000 7f88ae050640 1 -- 172.21.15.70:0/2652324218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88a806c890 msgr2=0x7f88a806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:29.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.999+0000 7f88ae050640 1 --2- 172.21.15.70:0/2652324218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88a806c890 0x7f88a806cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.999+0000 7f88ae050640 1 -- 172.21.15.70:0/2652324218 >> v1:172.21.15.70:6789/0 conn(0x7f88a8160940 legacy=0x7f88a8070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:28.999+0000 7f88ae050640 1 -- 172.21.15.70:0/2652324218 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a80c3fc0 con 0x7f88a815cd90 2023-10-08T05:37:29.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.000+0000 7f88ae050640 1 --2- 172.21.15.70:0/2652324218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a815cd90 0x7f88a80702d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f88a40097a0 tx=0x7f88a402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de1737341414341 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:29.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.000+0000 7f88ad04e640 1 -- 172.21.15.70:0/2652324218 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a402d040 con 0x7f88a815cd90 2023-10-08T05:37:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.000+0000 7f88ad04e640 1 -- 172.21.15.70:0/2652324218 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a4031810 con 0x7f88a815cd90 2023-10-08T05:37:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.000+0000 7f88ad04e640 1 -- 172.21.15.70:0/2652324218 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4031ba0 con 0x7f88a815cd90 2023-10-08T05:37:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.001+0000 7f88b02b4640 1 -- 172.21.15.70:0/2652324218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a815cd90 msgr2=0x7f88a80702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.001+0000 7f88b02b4640 1 --2- 172.21.15.70:0/2652324218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a815cd90 0x7f88a80702d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f88a40097a0 tx=0x7f88a402b8b0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.001+0000 7f88b02b4640 1 -- 172.21.15.70:0/2652324218 shutdown_connections 2023-10-08T05:37:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.001+0000 7f88b02b4640 1 --2- 172.21.15.70:0/2652324218 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88a806c890 0x7f88a806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.001+0000 7f88b02b4640 1 --2- 172.21.15.70:0/2652324218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a815cd90 0x7f88a80702d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.001+0000 7f88b02b4640 1 -- 172.21.15.70:0/2652324218 >> 172.21.15.70:0/2652324218 conn(0x7f88a806a380 msgr2=0x7f88a80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.001+0000 7f88b02b4640 1 -- 172.21.15.70:0/2652324218 shutdown_connections 2023-10-08T05:37:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.001+0000 7f88b02b4640 1 -- 172.21.15.70:0/2652324218 wait complete. 2023-10-08T05:37:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.003+0000 7f88b02b4640 1 Processor -- start 2023-10-08T05:37:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.003+0000 7f88b02b4640 1 -- start start 2023-10-08T05:37:29.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.004+0000 7f88b02b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88a806c890 0x7f88a814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.004+0000 7f88b02b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88a815cd90 0x7f88a81591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.004+0000 7f88b02b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a8160940 0x7f88a815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.004+0000 7f88b02b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a815f240 con 0x7f88a8160940 2023-10-08T05:37:29.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.004+0000 7f88b02b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a815e630 con 0x7f88a815cd90 2023-10-08T05:37:29.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.004+0000 7f88b02b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a815e910 con 0x7f88a806c890 2023-10-08T05:37:29.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.004+0000 7f88ae851640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a8160940 0x7f88a815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.004+0000 7f88ae851640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a8160940 0x7f88a815f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55634/0 (socket says 172.21.15.70:55634) 2023-10-08T05:37:29.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.004+0000 7f88ae851640 1 -- 172.21.15.70:0/87539137 learned_addr learned my addr 172.21.15.70:0/87539137 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:29.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f88ad84f640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88a815cd90 0x7f88a81591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f88ae050640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88a806c890 0x7f88a814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f88ae851640 1 -- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88a806c890 msgr2=0x7f88a814e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f88ae851640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88a806c890 0x7f88a814e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f88ae851640 1 -- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88a815cd90 msgr2=0x7f88a81591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f88ae851640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88a815cd90 0x7f88a81591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f88ae851640 1 -- 172.21.15.70:0/87539137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a4009400 con 0x7f88a8160940 2023-10-08T05:37:29.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f88ae851640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a8160940 0x7f88a815f6d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f88a000cd70 tx=0x7f88a000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:29.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f889effd640 1 -- 172.21.15.70:0/87539137 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0003b90 con 0x7f88a8160940 2023-10-08T05:37:29.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f889effd640 1 -- 172.21.15.70:0/87539137 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0003cf0 con 0x7f88a8160940 2023-10-08T05:37:29.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f889effd640 1 -- 172.21.15.70:0/87539137 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004240 con 0x7f88a8160940 2023-10-08T05:37:29.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.005+0000 7f88b02b4640 1 -- 172.21.15.70:0/87539137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a815ebf0 con 0x7f88a8160940 2023-10-08T05:37:29.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.006+0000 7f88b02b4640 1 -- 172.21.15.70:0/87539137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a814e5b0 con 0x7f88a8160940 2023-10-08T05:37:29.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.007+0000 7f88b02b4640 1 -- 172.21.15.70:0/87539137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a80cf6c0 con 0x7f88a8160940 2023-10-08T05:37:29.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.008+0000 7f889effd640 1 -- 172.21.15.70:0/87539137 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f88a00043a0 con 0x7f88a8160940 2023-10-08T05:37:29.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.009+0000 7f889effd640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f888c06ae70 0x7f888c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.009+0000 7f889effd640 1 -- 172.21.15.70:0/87539137 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f88a0012070 con 0x7f88a8160940 2023-10-08T05:37:29.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.010+0000 7f88ae050640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f888c06ae70 0x7f888c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.011+0000 7f88ae050640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f888c06ae70 0x7f888c06d330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f88a4036000 tx=0x7f88a403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:29.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.013+0000 7f889effd640 1 -- 172.21.15.70:0/87539137 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88a0055060 con 0x7f88a8160940 2023-10-08T05:37:29.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.240+0000 7f88b02b4640 1 -- 172.21.15.70:0/87539137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88a8067490 con 0x7f88a8160940 2023-10-08T05:37:29.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.241+0000 7f889effd640 1 -- 172.21.15.70:0/87539137 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f88a0058680 con 0x7f88a8160940 2023-10-08T05:37:29.242 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:29.242 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:36:11.613175+0000","modified":"2023-10-08T05:36:33.450447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T05:37:29.243 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 9 2023-10-08T05:37:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 -- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f888c06ae70 msgr2=0x7f888c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f888c06ae70 0x7f888c06d330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f88a4036000 tx=0x7f88a403c040 comp rx=0 tx=0).stop 2023-10-08T05:37:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 -- 172.21.15.70:0/87539137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a8160940 msgr2=0x7f88a815f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a8160940 0x7f88a815f6d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f88a000cd70 tx=0x7f88a000a660 comp rx=0 tx=0).stop 2023-10-08T05:37:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 -- 172.21.15.70:0/87539137 shutdown_connections 2023-10-08T05:37:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f888c06ae70 0x7f888c06d330 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a8160940 0x7f88a815f6d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88a815cd90 0x7f88a81591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 --2- 172.21.15.70:0/87539137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88a806c890 0x7f88a814e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 -- 172.21.15.70:0/87539137 >> 172.21.15.70:0/87539137 conn(0x7f88a806a380 msgr2=0x7f88a80ad940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:29.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 -- 172.21.15.70:0/87539137 shutdown_connections 2023-10-08T05:37:29.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.244+0000 7f88b02b4640 1 -- 172.21.15.70:0/87539137 wait complete. 2023-10-08T05:37:29.262 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:29.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.464+0000 7f7accfcc640 1 Processor -- start 2023-10-08T05:37:29.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.464+0000 7f7accfcc640 1 -- start start 2023-10-08T05:37:29.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.466+0000 7f7accfcc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7ac8160940 0x7f7ac80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.466+0000 7f7accfcc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac806c890 0x7f7ac806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.466+0000 7f7accfcc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac80c48f0 con 0x7f7ac815cd90 2023-10-08T05:37:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.466+0000 7f7accfcc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac80c4d80 con 0x7f7ac8160940 2023-10-08T05:37:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.466+0000 7f7accfcc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80c3ce0 con 0x7f7ac806c890 2023-10-08T05:37:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.466+0000 7f7ac659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7ac815cd90 0x7f7ac80702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57192/0 (socket says 172.21.15.70:57192) 2023-10-08T05:37:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.466+0000 7f7ac659c640 1 -- 172.21.15.70:0/3535039321 learned_addr learned my addr 172.21.15.70:0/3535039321 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.466+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456565028 0 0) 0x7f7ac80c48f0 con 0x7f7ac815cd90 2023-10-08T05:37:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.466+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab4003520 con 0x7f7ac815cd90 2023-10-08T05:37:29.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.466+0000 7f7abffff640 1 --2- 172.21.15.70:0/3535039321 >> v2:172.21.15.181:3300/0 conn(0x7f7ac8160940 0x7f7ac80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac6d9d640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac806c890 0x7f7ac806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2754966450 0 0) 0x7f7ab4003520 con 0x7f7ac815cd90 2023-10-08T05:37:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac806c890 msgr2=0x7f7ac806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac806c890 0x7f7ac806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 >> v2:172.21.15.181:3300/0 conn(0x7f7ac8160940 msgr2=0x7f7ac80709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 --2- 172.21.15.70:0/3535039321 >> v2:172.21.15.181:3300/0 conn(0x7f7ac8160940 0x7f7ac80709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80c3fc0 con 0x7f7ac815cd90 2023-10-08T05:37:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7ab8001670 con 0x7f7ac815cd90 2023-10-08T05:37:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7abffff640 1 --2- 172.21.15.70:0/3535039321 >> v2:172.21.15.181:3300/0 conn(0x7f7ac8160940 0x7f7ac80709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:37:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 >> v1:172.21.15.70:6789/0 conn(0x7f7ac815cd90 legacy=0x7f7ac80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 0x7f7ab4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac806c890 0x7f7ab40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ab4004d00 0x7f7ab4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80c3ce0 con 0x7f7ab4003970 2023-10-08T05:37:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab400ba60 con 0x7f7ab4004d00 2023-10-08T05:37:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab400bc20 con 0x7f7ac806c890 2023-10-08T05:37:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.467+0000 7f7ac659c640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 0x7f7ab4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.468+0000 7f7abffff640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac806c890 0x7f7ab40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.468+0000 7f7ac6d9d640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ab4004d00 0x7f7ab4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.468+0000 7f7ac659c640 1 -- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac806c890 msgr2=0x7f7ab40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.468+0000 7f7ac659c640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac806c890 0x7f7ab40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.468+0000 7f7ac659c640 1 -- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ab4004d00 msgr2=0x7f7ab4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.468+0000 7f7ac659c640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ab4004d00 0x7f7ab4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.468+0000 7f7ac659c640 1 -- 172.21.15.70:0/3535039321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4003520 con 0x7f7ab4003970 2023-10-08T05:37:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.469+0000 7f7ac659c640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 0x7f7ab4003d90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800c0e0 tx=0x7f7ab40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdbfbbd07698215 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.469+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8005ad0 con 0x7f7ab4003970 2023-10-08T05:37:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.469+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab8005c30 con 0x7f7ab4003970 2023-10-08T05:37:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.469+0000 7f7ac5d9b640 1 -- 172.21.15.70:0/3535039321 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab802c810 con 0x7f7ab4003970 2023-10-08T05:37:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.469+0000 7f7accfcc640 1 -- 172.21.15.70:0/3535039321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 msgr2=0x7f7ab4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.469+0000 7f7accfcc640 1 --2- 172.21.15.70:0/3535039321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 0x7f7ab4003d90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800c0e0 tx=0x7f7ab40057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.470+0000 7f7ac6d9d640 1 -- 172.21.15.70:0/3535039321 reap_dead start 2023-10-08T05:37:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.470+0000 7f7accfcc640 1 -- 172.21.15.70:0/3535039321 shutdown_connections 2023-10-08T05:37:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.470+0000 7f7accfcc640 1 -- 172.21.15.70:0/3535039321 >> 172.21.15.70:0/3535039321 conn(0x7f7ac806a380 msgr2=0x7f7ac80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.470+0000 7f7accfcc640 1 -- 172.21.15.70:0/3535039321 shutdown_connections 2023-10-08T05:37:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.470+0000 7f7accfcc640 1 -- 172.21.15.70:0/3535039321 wait complete. 2023-10-08T05:37:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.471+0000 7f7accfcc640 1 Processor -- start 2023-10-08T05:37:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.472+0000 7f7accfcc640 1 -- start start 2023-10-08T05:37:29.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7accfcc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 0x7f7ac815e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7accfcc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac815d380 0x7f7ac8140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7accfcc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac80caef0 0x7f7ac8156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7accfcc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80caad0 con 0x7f7ab4003970 2023-10-08T05:37:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7accfcc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80c9e50 con 0x7f7ac80caef0 2023-10-08T05:37:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7accfcc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80ca130 con 0x7f7ac815d380 2023-10-08T05:37:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7ac659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 0x7f7ac815e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7ac659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 0x7f7ac815e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55658/0 (socket says 172.21.15.70:55658) 2023-10-08T05:37:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7ac659c640 1 -- 172.21.15.70:0/153489070 learned_addr learned my addr 172.21.15.70:0/153489070 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7ac659c640 1 -- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac815d380 msgr2=0x7f7ac8140ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7ac659c640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac815d380 0x7f7ac8140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7ac659c640 1 -- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac80caef0 msgr2=0x7f7ac8156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7ac659c640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac80caef0 0x7f7ac8156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.473+0000 7f7ac659c640 1 -- 172.21.15.70:0/153489070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80ca410 con 0x7f7ab4003970 2023-10-08T05:37:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.474+0000 7f7ac659c640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 0x7f7ac815e270 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7ab802c3b0 tx=0x7f7ac80c4860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.474+0000 7f7abeffd640 1 -- 172.21.15.70:0/153489070 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8039070 con 0x7f7ab4003970 2023-10-08T05:37:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.474+0000 7f7abeffd640 1 -- 172.21.15.70:0/153489070 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab802dd50 con 0x7f7ab4003970 2023-10-08T05:37:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.474+0000 7f7abeffd640 1 -- 172.21.15.70:0/153489070 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80072b0 con 0x7f7ab4003970 2023-10-08T05:37:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.474+0000 7f7accfcc640 1 -- 172.21.15.70:0/153489070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac80ca6f0 con 0x7f7ab4003970 2023-10-08T05:37:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.474+0000 7f7accfcc640 1 -- 172.21.15.70:0/153489070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac81413e0 con 0x7f7ab4003970 2023-10-08T05:37:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.475+0000 7f7accfcc640 1 -- 172.21.15.70:0/153489070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac8067490 con 0x7f7ab4003970 2023-10-08T05:37:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.476+0000 7f7abeffd640 1 -- 172.21.15.70:0/153489070 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f7ab8032070 con 0x7f7ab4003970 2023-10-08T05:37:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.477+0000 7f7abeffd640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7aa806aea0 0x7f7aa806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.477+0000 7f7abeffd640 1 -- 172.21.15.70:0/153489070 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f7ab803c040 con 0x7f7ab4003970 2023-10-08T05:37:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.478+0000 7f7abffff640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7aa806aea0 0x7f7aa806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.479+0000 7f7abffff640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7aa806aea0 0x7f7aa806d360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0006e80 tx=0x7f7ab0006e10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:29.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.481+0000 7f7abeffd640 1 -- 172.21.15.70:0/153489070 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ab8075b10 con 0x7f7ab4003970 2023-10-08T05:37:29.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.672+0000 7f7accfcc640 1 -- 172.21.15.70:0/153489070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ac80b0ef0 con 0x7f7ab4003970 2023-10-08T05:37:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.676+0000 7f7abeffd640 1 -- 172.21.15.70:0/153489070 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7f7ab8079130 con 0x7f7ab4003970 2023-10-08T05:37:29.677 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:29.678 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:29.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.678+0000 7f7accfcc640 1 -- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7aa806aea0 msgr2=0x7f7aa806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.678+0000 7f7accfcc640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7aa806aea0 0x7f7aa806d360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0006e80 tx=0x7f7ab0006e10 comp rx=0 tx=0).stop 2023-10-08T05:37:29.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.679+0000 7f7accfcc640 1 -- 172.21.15.70:0/153489070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 msgr2=0x7f7ac815e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.679+0000 7f7accfcc640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 0x7f7ac815e270 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7ab802c3b0 tx=0x7f7ac80c4860 comp rx=0 tx=0).stop 2023-10-08T05:37:29.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.679+0000 7f7accfcc640 1 -- 172.21.15.70:0/153489070 shutdown_connections 2023-10-08T05:37:29.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.679+0000 7f7accfcc640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f7aa806aea0 0x7f7aa806d360 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.679+0000 7f7accfcc640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ac80caef0 0x7f7ac8156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.679+0000 7f7accfcc640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ac815d380 0x7f7ac8140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.679+0000 7f7accfcc640 1 --2- 172.21.15.70:0/153489070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ab4003970 0x7f7ac815e270 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.679+0000 7f7accfcc640 1 -- 172.21.15.70:0/153489070 >> 172.21.15.70:0/153489070 conn(0x7f7ac806a380 msgr2=0x7f7ac80af540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.679+0000 7f7accfcc640 1 -- 172.21.15.70:0/153489070 shutdown_connections 2023-10-08T05:37:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.679+0000 7f7accfcc640 1 -- 172.21.15.70:0/153489070 wait complete. 2023-10-08T05:37:29.699 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T05:37:29.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:37:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.910+0000 7f2ba99b6640 1 Processor -- start 2023-10-08T05:37:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.910+0000 7f2ba99b6640 1 -- start start 2023-10-08T05:37:29.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.912+0000 7f2ba99b6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ba4160940 0x7f2ba40709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.912+0000 7f2ba99b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba406c890 0x7f2ba406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.912+0000 7f2ba99b6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba40c48f0 con 0x7f2ba415cd90 2023-10-08T05:37:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.912+0000 7f2ba99b6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba40c4d80 con 0x7f2ba4160940 2023-10-08T05:37:29.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.912+0000 7f2ba99b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba40c3ce0 con 0x7f2ba406c890 2023-10-08T05:37:29.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.913+0000 7f2ba27fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ba4160940 0x7f2ba40709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.913+0000 7f2ba27fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ba4160940 0x7f2ba40709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55664/0 (socket says 172.21.15.70:55664) 2023-10-08T05:37:29.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.913+0000 7f2ba27fc640 1 -- 172.21.15.70:0/2957225132 learned_addr learned my addr 172.21.15.70:0/2957225132 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:29.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.913+0000 7f2ba37fe640 1 --2- 172.21.15.70:0/2957225132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba406c890 0x7f2ba406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.913+0000 7f2ba27fc640 1 -- 172.21.15.70:0/2957225132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba406c890 msgr2=0x7f2ba406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:29.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.913+0000 7f2ba27fc640 1 --2- 172.21.15.70:0/2957225132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba406c890 0x7f2ba406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.913+0000 7f2ba27fc640 1 -- 172.21.15.70:0/2957225132 >> v1:172.21.15.181:6789/0 conn(0x7f2ba415cd90 legacy=0x7f2ba40702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:29.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.913+0000 7f2ba27fc640 1 -- 172.21.15.70:0/2957225132 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba40c3fc0 con 0x7f2ba4160940 2023-10-08T05:37:29.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.914+0000 7f2ba27fc640 1 --2- 172.21.15.70:0/2957225132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba4160940 0x7f2ba40709e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009870 tx=0x7f2b9802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afb5d8fcd8acd7d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:29.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.914+0000 7f2ba1ffb640 1 -- 172.21.15.70:0/2957225132 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98031640 con 0x7f2ba4160940 2023-10-08T05:37:29.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.914+0000 7f2ba1ffb640 1 -- 172.21.15.70:0/2957225132 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b980317a0 con 0x7f2ba4160940 2023-10-08T05:37:29.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.914+0000 7f2ba1ffb640 1 -- 172.21.15.70:0/2957225132 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98031b30 con 0x7f2ba4160940 2023-10-08T05:37:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.915+0000 7f2ba99b6640 1 -- 172.21.15.70:0/2957225132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba4160940 msgr2=0x7f2ba40709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.915+0000 7f2ba99b6640 1 --2- 172.21.15.70:0/2957225132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba4160940 0x7f2ba40709e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009870 tx=0x7f2b9802b8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.915+0000 7f2ba99b6640 1 -- 172.21.15.70:0/2957225132 shutdown_connections 2023-10-08T05:37:29.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.915+0000 7f2ba99b6640 1 --2- 172.21.15.70:0/2957225132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba406c890 0x7f2ba406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.915+0000 7f2ba99b6640 1 --2- 172.21.15.70:0/2957225132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba4160940 0x7f2ba40709e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.915+0000 7f2ba99b6640 1 -- 172.21.15.70:0/2957225132 >> 172.21.15.70:0/2957225132 conn(0x7f2ba406a380 msgr2=0x7f2ba40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:29.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.915+0000 7f2ba99b6640 1 -- 172.21.15.70:0/2957225132 shutdown_connections 2023-10-08T05:37:29.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.915+0000 7f2ba99b6640 1 -- 172.21.15.70:0/2957225132 wait complete. 2023-10-08T05:37:29.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.917+0000 7f2ba99b6640 1 Processor -- start 2023-10-08T05:37:29.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.917+0000 7f2ba99b6640 1 -- start start 2023-10-08T05:37:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.918+0000 7f2ba99b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba406c890 0x7f2ba41554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.918+0000 7f2ba99b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba415cd90 0x7f2ba4159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.918+0000 7f2ba99b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba4160940 0x7f2ba415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.918+0000 7f2ba99b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba415b0c0 con 0x7f2ba406c890 2023-10-08T05:37:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.918+0000 7f2ba99b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba415a440 con 0x7f2ba4160940 2023-10-08T05:37:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.918+0000 7f2ba99b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba415a720 con 0x7f2ba415cd90 2023-10-08T05:37:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.918+0000 7f2ba2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba406c890 0x7f2ba41554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.918+0000 7f2ba2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba406c890 0x7f2ba41554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55672/0 (socket says 172.21.15.70:55672) 2023-10-08T05:37:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.918+0000 7f2ba2ffd640 1 -- 172.21.15.70:0/3009255305 learned_addr learned my addr 172.21.15.70:0/3009255305 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2ba27fc640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba415cd90 0x7f2ba4159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2ba37fe640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba4160940 0x7f2ba415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2ba2ffd640 1 -- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba415cd90 msgr2=0x7f2ba4159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2ba2ffd640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba415cd90 0x7f2ba4159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2ba2ffd640 1 -- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba4160940 msgr2=0x7f2ba415e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2ba2ffd640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba4160940 0x7f2ba415e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2ba2ffd640 1 -- 172.21.15.70:0/3009255305 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b980094d0 con 0x7f2ba406c890 2023-10-08T05:37:29.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2ba27fc640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba415cd90 0x7f2ba4159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:29.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2ba2ffd640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba406c890 0x7f2ba41554a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400c430 tx=0x7f2b9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:29.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2b8bfff640 1 -- 172.21.15.70:0/3009255305 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94011790 con 0x7f2ba406c890 2023-10-08T05:37:29.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2b8bfff640 1 -- 172.21.15.70:0/3009255305 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b940118f0 con 0x7f2ba406c890 2023-10-08T05:37:29.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2ba99b6640 1 -- 172.21.15.70:0/3009255305 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba415aa00 con 0x7f2ba406c890 2023-10-08T05:37:29.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.919+0000 7f2b8bfff640 1 -- 172.21.15.70:0/3009255305 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94011ce0 con 0x7f2ba406c890 2023-10-08T05:37:29.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.920+0000 7f2ba99b6640 1 -- 172.21.15.70:0/3009255305 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba415a000 con 0x7f2ba406c890 2023-10-08T05:37:29.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.921+0000 7f2ba99b6640 1 -- 172.21.15.70:0/3009255305 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba415a190 con 0x7f2ba406c890 2023-10-08T05:37:29.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.922+0000 7f2b8bfff640 1 -- 172.21.15.70:0/3009255305 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2b9401c460 con 0x7f2ba406c890 2023-10-08T05:37:29.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.922+0000 7f2b8bfff640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2b9006ae70 0x7f2b9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:29.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.923+0000 7f2b8bfff640 1 -- 172.21.15.70:0/3009255305 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f2b940899b0 con 0x7f2ba406c890 2023-10-08T05:37:29.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.923+0000 7f2ba27fc640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2b9006ae70 0x7f2b9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:29.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.924+0000 7f2ba27fc640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2b9006ae70 0x7f2b9006d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b98036000 tx=0x7f2b9803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:29.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:29.927+0000 7f2b8bfff640 1 -- 172.21.15.70:0/3009255305 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b94054680 con 0x7f2ba406c890 2023-10-08T05:37:30.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.153+0000 7f2ba99b6640 1 -- 172.21.15.70:0/3009255305 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ba4067490 con 0x7f2ba406c890 2023-10-08T05:37:30.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.154+0000 7f2b8bfff640 1 -- 172.21.15.70:0/3009255305 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f2b94057ca0 con 0x7f2ba406c890 2023-10-08T05:37:30.155 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:30.155 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:36:11.613175+0000","modified":"2023-10-08T05:36:33.450447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T05:37:30.155 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 9 2023-10-08T05:37:30.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.156+0000 7f2ba99b6640 1 -- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2b9006ae70 msgr2=0x7f2b9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.156+0000 7f2ba99b6640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2b9006ae70 0x7f2b9006d330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b98036000 tx=0x7f2b9803c040 comp rx=0 tx=0).stop 2023-10-08T05:37:30.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.156+0000 7f2ba99b6640 1 -- 172.21.15.70:0/3009255305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba406c890 msgr2=0x7f2ba41554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.156+0000 7f2ba99b6640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba406c890 0x7f2ba41554a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400c430 tx=0x7f2b9400c8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.157+0000 7f2ba99b6640 1 -- 172.21.15.70:0/3009255305 shutdown_connections 2023-10-08T05:37:30.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.157+0000 7f2ba99b6640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2b9006ae70 0x7f2b9006d330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.157+0000 7f2ba99b6640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ba4160940 0x7f2ba415e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.157+0000 7f2ba99b6640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ba415cd90 0x7f2ba4159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.157+0000 7f2ba99b6640 1 --2- 172.21.15.70:0/3009255305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba406c890 0x7f2ba41554a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.157+0000 7f2ba99b6640 1 -- 172.21.15.70:0/3009255305 >> 172.21.15.70:0/3009255305 conn(0x7f2ba406a380 msgr2=0x7f2ba40ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:30.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.157+0000 7f2ba99b6640 1 -- 172.21.15.70:0/3009255305 shutdown_connections 2023-10-08T05:37:30.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.157+0000 7f2ba99b6640 1 -- 172.21.15.70:0/3009255305 wait complete. 2023-10-08T05:37:30.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:37:30.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.384+0000 7fa873c56640 1 Processor -- start 2023-10-08T05:37:30.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.384+0000 7fa873c56640 1 -- start start 2023-10-08T05:37:30.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.385+0000 7fa873c56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c159ac0 0x7fa86c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.385+0000 7fa873c56640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c06dee0 con 0x7fa86c0ad790 2023-10-08T05:37:30.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.385+0000 7fa873c56640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c15d850 con 0x7fa86c0709e0 2023-10-08T05:37:30.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.385+0000 7fa873c56640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c15db30 con 0x7fa86c159ac0 2023-10-08T05:37:30.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.385+0000 7fa8711f1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa86c0ad790 0x7fa86c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57198/0 (socket says 172.21.15.70:57198) 2023-10-08T05:37:30.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.385+0000 7fa8711f1640 1 -- 172.21.15.70:0/57955384 learned_addr learned my addr 172.21.15.70:0/57955384 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:30.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8719f2640 1 --2- 172.21.15.70:0/57955384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c159ac0 0x7fa86c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3486402966 0 0) 0x7fa86c06dee0 con 0x7fa86c0ad790 2023-10-08T05:37:30.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa858003520 con 0x7fa86c0ad790 2023-10-08T05:37:30.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1017668174 0 0) 0x7fa858003520 con 0x7fa86c0ad790 2023-10-08T05:37:30.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c159ac0 msgr2=0x7fa86c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:30.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8709f0640 1 --2- 172.21.15.70:0/57955384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c159ac0 0x7fa86c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 >> v1:172.21.15.181:6789/0 conn(0x7fa86c0709e0 legacy=0x7fa86c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c15de10 con 0x7fa86c0ad790 2023-10-08T05:37:30.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa85c001670 con 0x7fa86c0ad790 2023-10-08T05:37:30.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 >> v1:172.21.15.70:6789/0 conn(0x7fa86c0ad790 legacy=0x7fa86c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.386+0000 7fa8709f0640 1 --2- 172.21.15.70:0/57955384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c159ac0 0x7fa8580039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8709f0640 1 --2- 172.21.15.70:0/57955384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa858003f70 0x7fa8580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8709f0640 1 --2- 172.21.15.70:0/57955384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa858004d00 0x7fa858005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c15db30 con 0x7fa858003f70 2023-10-08T05:37:30.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85800bac0 con 0x7fa858004d00 2023-10-08T05:37:30.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85800bc80 con 0x7fa86c159ac0 2023-10-08T05:37:30.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8711f1640 1 --2- 172.21.15.70:0/57955384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa858003f70 0x7fa8580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8711f1640 1 -- 172.21.15.70:0/57955384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c159ac0 msgr2=0x7fa8580039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:30.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8711f1640 1 --2- 172.21.15.70:0/57955384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c159ac0 0x7fa8580039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8711f1640 1 -- 172.21.15.70:0/57955384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa858004d00 msgr2=0x7fa858005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:30.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8711f1640 1 --2- 172.21.15.70:0/57955384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa858004d00 0x7fa858005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.387+0000 7fa8711f1640 1 -- 172.21.15.70:0/57955384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa858003520 con 0x7fa858003f70 2023-10-08T05:37:30.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.388+0000 7fa8711f1640 1 --2- 172.21.15.70:0/57955384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa858003f70 0x7fa8580047c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0adbd0 tx=0x7fa8580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f25506518ef9ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:30.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.388+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c02eb20 con 0x7fa858003f70 2023-10-08T05:37:30.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.389+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c02ec80 con 0x7fa858003f70 2023-10-08T05:37:30.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.389+0000 7fa8709f0640 1 -- 172.21.15.70:0/57955384 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c005b10 con 0x7fa858003f70 2023-10-08T05:37:30.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.389+0000 7fa873c56640 1 -- 172.21.15.70:0/57955384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa858003f70 msgr2=0x7fa8580047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.389+0000 7fa873c56640 1 --2- 172.21.15.70:0/57955384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa858003f70 0x7fa8580047c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0adbd0 tx=0x7fa8580057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.389+0000 7fa8721f3640 1 -- 172.21.15.70:0/57955384 reap_dead start 2023-10-08T05:37:30.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.389+0000 7fa873c56640 1 -- 172.21.15.70:0/57955384 shutdown_connections 2023-10-08T05:37:30.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.389+0000 7fa873c56640 1 -- 172.21.15.70:0/57955384 >> 172.21.15.70:0/57955384 conn(0x7fa86c06a380 msgr2=0x7fa86c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:30.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.389+0000 7fa873c56640 1 -- 172.21.15.70:0/57955384 shutdown_connections 2023-10-08T05:37:30.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.389+0000 7fa873c56640 1 -- 172.21.15.70:0/57955384 wait complete. 2023-10-08T05:37:30.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.391+0000 7fa873c56640 1 Processor -- start 2023-10-08T05:37:30.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.391+0000 7fa873c56640 1 -- start start 2023-10-08T05:37:30.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.392+0000 7fa873c56640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa858003f70 0x7fa86c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.392+0000 7fa873c56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c15bae0 0x7fa86c1f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.392+0000 7fa873c56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c158430 0x7fa86c1588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.392+0000 7fa873c56640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c158df0 con 0x7fa86c158430 2023-10-08T05:37:30.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.392+0000 7fa873c56640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c15bf60 con 0x7fa858003f70 2023-10-08T05:37:30.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.392+0000 7fa873c56640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c15c210 con 0x7fa86c15bae0 2023-10-08T05:37:30.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.392+0000 7fa8721f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c158430 0x7fa86c1588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.392+0000 7fa8721f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c158430 0x7fa86c1588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55688/0 (socket says 172.21.15.70:55688) 2023-10-08T05:37:30.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.392+0000 7fa8721f3640 1 -- 172.21.15.70:0/1765726050 learned_addr learned my addr 172.21.15.70:0/1765726050 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:30.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8719f2640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa858003f70 0x7fa86c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8711f1640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c15bae0 0x7fa86c1f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8721f3640 1 -- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c15bae0 msgr2=0x7fa86c1f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8721f3640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c15bae0 0x7fa86c1f1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8721f3640 1 -- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa858003f70 msgr2=0x7fa86c15ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8721f3640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa858003f70 0x7fa86c15ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8721f3640 1 -- 172.21.15.70:0/1765726050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c15c4f0 con 0x7fa86c158430 2023-10-08T05:37:30.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8721f3640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c158430 0x7fa86c1588b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0ae2f0 tx=0x7fa86400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:30.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8627fc640 1 -- 172.21.15.70:0/1765726050 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864005d40 con 0x7fa86c158430 2023-10-08T05:37:30.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8627fc640 1 -- 172.21.15.70:0/1765726050 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86400ced0 con 0x7fa86c158430 2023-10-08T05:37:30.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa8627fc640 1 -- 172.21.15.70:0/1765726050 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86400a850 con 0x7fa86c158430 2023-10-08T05:37:30.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.393+0000 7fa873c56640 1 -- 172.21.15.70:0/1765726050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c15c7d0 con 0x7fa86c158430 2023-10-08T05:37:30.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.394+0000 7fa873c56640 1 -- 172.21.15.70:0/1765726050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c15cd90 con 0x7fa86c158430 2023-10-08T05:37:30.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.395+0000 7fa873c56640 1 -- 172.21.15.70:0/1765726050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c067490 con 0x7fa86c158430 2023-10-08T05:37:30.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.396+0000 7fa8627fc640 1 -- 172.21.15.70:0/1765726050 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa86400a9b0 con 0x7fa86c158430 2023-10-08T05:37:30.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.396+0000 7fa8627fc640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa85006ae70 0x7fa85006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.397+0000 7fa8627fc640 1 -- 172.21.15.70:0/1765726050 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa8640889a0 con 0x7fa86c158430 2023-10-08T05:37:30.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.397+0000 7fa8719f2640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa85006ae70 0x7fa85006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.398+0000 7fa8719f2640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa85006ae70 0x7fa85006d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa8680047b0 tx=0x7fa868008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:30.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.401+0000 7fa8627fc640 1 -- 172.21.15.70:0/1765726050 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa864053720 con 0x7fa86c158430 2023-10-08T05:37:30.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.626+0000 7fa873c56640 1 -- 172.21.15.70:0/1765726050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa86c0b0ff0 con 0x7fa86c158430 2023-10-08T05:37:30.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.627+0000 7fa8627fc640 1 -- 172.21.15.70:0/1765726050 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fa864056d40 con 0x7fa86c158430 2023-10-08T05:37:30.628 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:30.628 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/602796023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":602796023},{"type":"v1","addr":"172.21.15.181:6836","nonce":602796023}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3526844956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3526844956},{"type":"v1","addr":"172.21.15.70:6833","nonce":3526844956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1320470582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1320470582},{"type":"v1","addr":"172.21.15.70:6835","nonce":1320470582}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:36:11.613175+0000","modified":"2023-10-08T05:36:33.450447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/1675009602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1675009602},{"type":"v1","addr":"172.21.15.181:6837","nonce":1675009602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-08T05:37:30.628 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 9 2023-10-08T05:37:30.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.629+0000 7fa873c56640 1 -- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa85006ae70 msgr2=0x7fa85006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.629+0000 7fa873c56640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa85006ae70 0x7fa85006d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa8680047b0 tx=0x7fa868008040 comp rx=0 tx=0).stop 2023-10-08T05:37:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.629+0000 7fa873c56640 1 -- 172.21.15.70:0/1765726050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c158430 msgr2=0x7fa86c1588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.629+0000 7fa873c56640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c158430 0x7fa86c1588b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0ae2f0 tx=0x7fa86400cb90 comp rx=0 tx=0).stop 2023-10-08T05:37:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.630+0000 7fa873c56640 1 -- 172.21.15.70:0/1765726050 shutdown_connections 2023-10-08T05:37:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.630+0000 7fa873c56640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa85006ae70 0x7fa85006d330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.630+0000 7fa873c56640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c158430 0x7fa86c1588b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.630+0000 7fa873c56640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa86c15bae0 0x7fa86c1f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.630+0000 7fa873c56640 1 --2- 172.21.15.70:0/1765726050 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa858003f70 0x7fa86c15ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.630+0000 7fa873c56640 1 -- 172.21.15.70:0/1765726050 >> 172.21.15.70:0/1765726050 conn(0x7fa86c06a380 msgr2=0x7fa86c0cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:30.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.630+0000 7fa873c56640 1 -- 172.21.15.70:0/1765726050 shutdown_connections 2023-10-08T05:37:30.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.630+0000 7fa873c56640 1 -- 172.21.15.70:0/1765726050 wait complete. 2023-10-08T05:37:30.648 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:30.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.856+0000 7fa72f59e640 1 Processor -- start 2023-10-08T05:37:30.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.857+0000 7fa72f59e640 1 -- start start 2023-10-08T05:37:30.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.858+0000 7fa72f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa73015ab30 0x7fa73015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.858+0000 7fa72f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7300adc70 0x7fa7300b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.858+0000 7fa72f59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73015f7d0 con 0x7fa73015af10 2023-10-08T05:37:30.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.858+0000 7fa72f59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73015f210 con 0x7fa73015ab30 2023-10-08T05:37:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.858+0000 7fa72f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73015f4f0 con 0x7fa7300adc70 2023-10-08T05:37:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.858+0000 7fa72e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa73015ab30 0x7fa73015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.858+0000 7fa72e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa73015ab30 0x7fa73015e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55704/0 (socket says 172.21.15.70:55704) 2023-10-08T05:37:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.858+0000 7fa72e59c640 1 -- 172.21.15.70:0/3096962862 learned_addr learned my addr 172.21.15.70:0/3096962862 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.859+0000 7fa72e59c640 1 -- 172.21.15.70:0/3096962862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7300adc70 msgr2=0x7fa7300b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.859+0000 7fa72e59c640 1 --2- 172.21.15.70:0/3096962862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7300adc70 0x7fa7300b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.859+0000 7fa72e59c640 1 -- 172.21.15.70:0/3096962862 >> v1:172.21.15.181:6789/0 conn(0x7fa73015af10 legacy=0x7fa7300701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:30.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.859+0000 7fa72e59c640 1 -- 172.21.15.70:0/3096962862 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73006c550 con 0x7fa73015ab30 2023-10-08T05:37:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.860+0000 7fa72e59c640 1 --2- 172.21.15.70:0/3096962862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 0x7fa73015e8a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa73006ada0 tx=0x7fa72002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef852f0663b6937 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.860+0000 7fa72dd9b640 1 -- 172.21.15.70:0/3096962862 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720031640 con 0x7fa73015ab30 2023-10-08T05:37:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.860+0000 7fa72dd9b640 1 -- 172.21.15.70:0/3096962862 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7200317a0 con 0x7fa73015ab30 2023-10-08T05:37:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.860+0000 7fa72dd9b640 1 -- 172.21.15.70:0/3096962862 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720031b30 con 0x7fa73015ab30 2023-10-08T05:37:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.861+0000 7fa72f59e640 1 -- 172.21.15.70:0/3096962862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 msgr2=0x7fa73015e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.861+0000 7fa72f59e640 1 --2- 172.21.15.70:0/3096962862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 0x7fa73015e8a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa73006ada0 tx=0x7fa72002b8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.861+0000 7fa72f59e640 1 -- 172.21.15.70:0/3096962862 shutdown_connections 2023-10-08T05:37:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.861+0000 7fa72f59e640 1 --2- 172.21.15.70:0/3096962862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7300adc70 0x7fa7300b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.861+0000 7fa72f59e640 1 --2- 172.21.15.70:0/3096962862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 0x7fa73015e8a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.861+0000 7fa72f59e640 1 -- 172.21.15.70:0/3096962862 >> 172.21.15.70:0/3096962862 conn(0x7fa7300cb0c0 msgr2=0x7fa7300cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.861+0000 7fa72f59e640 1 -- 172.21.15.70:0/3096962862 shutdown_connections 2023-10-08T05:37:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.861+0000 7fa72f59e640 1 -- 172.21.15.70:0/3096962862 wait complete. 2023-10-08T05:37:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.863+0000 7fa72f59e640 1 Processor -- start 2023-10-08T05:37:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.863+0000 7fa72f59e640 1 -- start start 2023-10-08T05:37:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa72f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7300adc70 0x7fa7301f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa72f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 0x7fa7300c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa72f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa73015af10 0x7fa730147920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa72f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73015b560 con 0x7fa73015ab30 2023-10-08T05:37:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa72f59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73015b800 con 0x7fa73015af10 2023-10-08T05:37:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa72f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73015bae0 con 0x7fa7300adc70 2023-10-08T05:37:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa727fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 0x7fa7300c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa727fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 0x7fa7300c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55706/0 (socket says 172.21.15.70:55706) 2023-10-08T05:37:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa727fff640 1 -- 172.21.15.70:0/3556987941 learned_addr learned my addr 172.21.15.70:0/3556987941 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa72e59c640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7300adc70 0x7fa7301f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa72ed9d640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa73015af10 0x7fa730147920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa727fff640 1 -- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7300adc70 msgr2=0x7fa7301f6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa727fff640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7300adc70 0x7fa7301f6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa727fff640 1 -- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa73015af10 msgr2=0x7fa730147920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.864+0000 7fa727fff640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa73015af10 0x7fa730147920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.865+0000 7fa727fff640 1 -- 172.21.15.70:0/3556987941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7200094d0 con 0x7fa73015ab30 2023-10-08T05:37:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.865+0000 7fa72e59c640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7300adc70 0x7fa7301f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.865+0000 7fa727fff640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 0x7fa7300c5710 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa71800b4a0 tx=0x7fa71800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.865+0000 7fa726ffd640 1 -- 172.21.15.70:0/3556987941 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718010610 con 0x7fa73015ab30 2023-10-08T05:37:30.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.865+0000 7fa726ffd640 1 -- 172.21.15.70:0/3556987941 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa718010770 con 0x7fa73015ab30 2023-10-08T05:37:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.865+0000 7fa726ffd640 1 -- 172.21.15.70:0/3556987941 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718010b20 con 0x7fa73015ab30 2023-10-08T05:37:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.865+0000 7fa72f59e640 1 -- 172.21.15.70:0/3556987941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73015daf0 con 0x7fa73015ab30 2023-10-08T05:37:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.865+0000 7fa72f59e640 1 -- 172.21.15.70:0/3556987941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7300c5c50 con 0x7fa73015ab30 2023-10-08T05:37:30.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.866+0000 7fa72f59e640 1 -- 172.21.15.70:0/3556987941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73015e010 con 0x7fa73015ab30 2023-10-08T05:37:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.867+0000 7fa726ffd640 1 -- 172.21.15.70:0/3556987941 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa718010c80 con 0x7fa73015ab30 2023-10-08T05:37:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.868+0000 7fa726ffd640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa71006ae70 0x7fa71006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.868+0000 7fa726ffd640 1 -- 172.21.15.70:0/3556987941 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7fa718089560 con 0x7fa73015ab30 2023-10-08T05:37:30.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.869+0000 7fa72e59c640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa71006ae70 0x7fa71006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:30.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.870+0000 7fa72e59c640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa71006ae70 0x7fa71006d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa720036000 tx=0x7fa72003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:30.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:30.872+0000 7fa726ffd640 1 -- 172.21.15.70:0/3556987941 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa73015e010 con 0x7fa73015ab30 2023-10-08T05:37:31.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.062+0000 7fa72f59e640 1 -- 172.21.15.70:0/3556987941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa73015bca0 con 0x7fa73015ab30 2023-10-08T05:37:31.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.065+0000 7fa726ffd640 1 -- 172.21.15.70:0/3556987941 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18722 (secure 0 0 0) 0x7fa7180542e0 con 0x7fa73015ab30 2023-10-08T05:37:31.065 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:31.066 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:36:58.999723+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:36:09.357824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T05:36:10.614183+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-08T05:36:14.220463+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:31.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.067+0000 7fa72f59e640 1 -- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa71006ae70 msgr2=0x7fa71006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.067+0000 7fa72f59e640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa71006ae70 0x7fa71006d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa720036000 tx=0x7fa72003c040 comp rx=0 tx=0).stop 2023-10-08T05:37:31.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.067+0000 7fa72f59e640 1 -- 172.21.15.70:0/3556987941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 msgr2=0x7fa7300c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.067+0000 7fa72f59e640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 0x7fa7300c5710 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa71800b4a0 tx=0x7fa71800b960 comp rx=0 tx=0).stop 2023-10-08T05:37:31.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.067+0000 7fa72f59e640 1 -- 172.21.15.70:0/3556987941 shutdown_connections 2023-10-08T05:37:31.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.067+0000 7fa72f59e640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa71006ae70 0x7fa71006d330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.067+0000 7fa72f59e640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa73015af10 0x7fa730147920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.067+0000 7fa72f59e640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa73015ab30 0x7fa7300c5710 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.067+0000 7fa72f59e640 1 --2- 172.21.15.70:0/3556987941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7300adc70 0x7fa7301f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.067+0000 7fa72f59e640 1 -- 172.21.15.70:0/3556987941 >> 172.21.15.70:0/3556987941 conn(0x7fa7300cb0c0 msgr2=0x7fa730158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:31.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.068+0000 7fa72f59e640 1 -- 172.21.15.70:0/3556987941 shutdown_connections 2023-10-08T05:37:31.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.068+0000 7fa72f59e640 1 -- 172.21.15.70:0/3556987941 wait complete. 2023-10-08T05:37:31.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T05:37:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.306+0000 7f0a63997640 1 Processor -- start 2023-10-08T05:37:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.307+0000 7f0a63997640 1 -- start start 2023-10-08T05:37:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.308+0000 7f0a63997640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0a5c163da0 0x7f0a5c166190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.308+0000 7f0a63997640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5c0b12b0 con 0x7f0a5c15e3c0 2023-10-08T05:37:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.308+0000 7f0a63997640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5c0b1500 con 0x7f0a5c0b1910 2023-10-08T05:37:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.308+0000 7f0a63997640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5c0cf1d0 con 0x7f0a5c163da0 2023-10-08T05:37:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.309+0000 7f0a61733640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0a5c15e3c0 0x7f0a5c15ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57210/0 (socket says 172.21.15.70:57210) 2023-10-08T05:37:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.309+0000 7f0a61733640 1 -- 172.21.15.70:0/3810590719 learned_addr learned my addr 172.21.15.70:0/3810590719 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.309+0000 7f0a61f34640 1 --2- 172.21.15.70:0/3810590719 >> v2:172.21.15.181:3300/0 conn(0x7f0a5c163da0 0x7f0a5c166190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.309+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527587863 0 0) 0x7f0a5c0b12b0 con 0x7f0a5c15e3c0 2023-10-08T05:37:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.309+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a44003520 con 0x7f0a5c15e3c0 2023-10-08T05:37:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.309+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1583446750 0 0) 0x7f0a44003520 con 0x7f0a5c15e3c0 2023-10-08T05:37:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.309+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 >> v2:172.21.15.181:3300/0 conn(0x7f0a5c163da0 msgr2=0x7f0a5c166190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.309+0000 7f0a5bfff640 1 --2- 172.21.15.70:0/3810590719 >> v2:172.21.15.181:3300/0 conn(0x7f0a5c163da0 0x7f0a5c166190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 >> v1:172.21.15.181:6789/0 conn(0x7f0a5c0b1910 legacy=0x7f0a5c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c0cf4b0 con 0x7f0a5c15e3c0 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0a4c0019f0 con 0x7f0a5c15e3c0 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 >> v1:172.21.15.70:6789/0 conn(0x7f0a5c15e3c0 legacy=0x7f0a5c15ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a5bfff640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a5bfff640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a440047c0 0x7f0a44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a5bfff640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a44005180 0x7f0a4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0cf1d0 con 0x7f0a44003970 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4400c020 con 0x7f0a440047c0 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4400c1e0 con 0x7f0a44005180 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a61733640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a60f32640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a440047c0 0x7f0a44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a61733640 1 -- 172.21.15.70:0/3810590719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a44005180 msgr2=0x7f0a4400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a61733640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a44005180 0x7f0a4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a61733640 1 -- 172.21.15.70:0/3810590719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a440047c0 msgr2=0x7f0a44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a61733640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a440047c0 0x7f0a44004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a61733640 1 -- 172.21.15.70:0/3810590719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a44003520 con 0x7f0a44003970 2023-10-08T05:37:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a60f32640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a440047c0 0x7f0a44004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.310+0000 7f0a61f34640 1 -- 172.21.15.70:0/3810590719 reap_dead start 2023-10-08T05:37:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.311+0000 7f0a61733640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a44003d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00a830 tx=0x7f0a440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c652c843b0873a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:31.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.312+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0072d0 con 0x7f0a44003970 2023-10-08T05:37:31.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.312+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c02f070 con 0x7f0a44003970 2023-10-08T05:37:31.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.312+0000 7f0a5bfff640 1 -- 172.21.15.70:0/3810590719 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0076f0 con 0x7f0a44003970 2023-10-08T05:37:31.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.313+0000 7f0a63997640 1 -- 172.21.15.70:0/3810590719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 msgr2=0x7f0a44003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.313+0000 7f0a63997640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a44003d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00a830 tx=0x7f0a440057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.313+0000 7f0a63997640 1 -- 172.21.15.70:0/3810590719 shutdown_connections 2023-10-08T05:37:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.313+0000 7f0a63997640 1 --2- 172.21.15.70:0/3810590719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a44003d90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.313+0000 7f0a63997640 1 -- 172.21.15.70:0/3810590719 >> 172.21.15.70:0/3810590719 conn(0x7f0a5c099d40 msgr2=0x7f0a5c0d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.313+0000 7f0a63997640 1 -- 172.21.15.70:0/3810590719 shutdown_connections 2023-10-08T05:37:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.313+0000 7f0a63997640 1 -- 172.21.15.70:0/3810590719 wait complete. 2023-10-08T05:37:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.315+0000 7f0a63997640 1 Processor -- start 2023-10-08T05:37:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.315+0000 7f0a63997640 1 -- start start 2023-10-08T05:37:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.316+0000 7f0a63997640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a5c0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.316+0000 7f0a63997640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a5c0cc190 0x7f0a5c14f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.316+0000 7f0a63997640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a5c14fb10 0x7f0a5c1511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.316+0000 7f0a63997640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0c8ba0 con 0x7f0a44003970 2023-10-08T05:37:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.316+0000 7f0a63997640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0b6950 con 0x7f0a5c0cc190 2023-10-08T05:37:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.316+0000 7f0a63997640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5c0cc610 con 0x7f0a5c14fb10 2023-10-08T05:37:31.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.316+0000 7f0a61733640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a5c0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.316+0000 7f0a61733640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a5c0b6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55724/0 (socket says 172.21.15.70:55724) 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.316+0000 7f0a61733640 1 -- 172.21.15.70:0/1394400845 learned_addr learned my addr 172.21.15.70:0/1394400845 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.316+0000 7f0a60f32640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a5c0cc190 0x7f0a5c14f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a61733640 1 -- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a5c14fb10 msgr2=0x7f0a5c1511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a61733640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a5c14fb10 0x7f0a5c1511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a61733640 1 -- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a5c0cc190 msgr2=0x7f0a5c14f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a61733640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a5c0cc190 0x7f0a5c14f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a61733640 1 -- 172.21.15.70:0/1394400845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c0cc8f0 con 0x7f0a44003970 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a61733640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a5c0b6410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00a830 tx=0x7f0a4c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a59ffb640 1 -- 172.21.15.70:0/1394400845 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0074e0 con 0x7f0a44003970 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a59ffb640 1 -- 172.21.15.70:0/1394400845 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c02ce90 con 0x7f0a44003970 2023-10-08T05:37:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a59ffb640 1 -- 172.21.15.70:0/1394400845 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c007860 con 0x7f0a44003970 2023-10-08T05:37:31.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a63997640 1 -- 172.21.15.70:0/1394400845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a5c0ccbd0 con 0x7f0a44003970 2023-10-08T05:37:31.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.317+0000 7f0a63997640 1 -- 172.21.15.70:0/1394400845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5c0cd190 con 0x7f0a44003970 2023-10-08T05:37:31.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.318+0000 7f0a63997640 1 -- 172.21.15.70:0/1394400845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a5c04e160 con 0x7f0a44003970 2023-10-08T05:37:31.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.319+0000 7f0a59ffb640 1 -- 172.21.15.70:0/1394400845 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0a4c031030 con 0x7f0a44003970 2023-10-08T05:37:31.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.320+0000 7f0a59ffb640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0a4806ae70 0x7f0a4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.320+0000 7f0a59ffb640 1 -- 172.21.15.70:0/1394400845 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6663+0+0 (secure 0 0 0) 0x7f0a4c0aaf00 con 0x7f0a44003970 2023-10-08T05:37:31.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.321+0000 7f0a60f32640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0a4806ae70 0x7f0a4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.322+0000 7f0a60f32640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0a4806ae70 0x7f0a4806d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0a50000c00 tx=0x7f0a50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:31.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.324+0000 7f0a59ffb640 1 -- 172.21.15.70:0/1394400845 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0a4c075b50 con 0x7f0a44003970 2023-10-08T05:37:31.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.551+0000 7f0a63997640 1 -- 172.21.15.70:0/1394400845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0a5c0ce1d0 con 0x7f0a44003970 2023-10-08T05:37:31.553 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T05:37:31.553+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T05:37:31.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.556+0000 7f0a59ffb640 1 -- 172.21.15.70:0/1394400845 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0a4c046090 con 0x7f0a44003970 2023-10-08T05:37:31.557 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T05:37:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 -- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0a4806ae70 msgr2=0x7f0a4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0a4806ae70 0x7f0a4806d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0a50000c00 tx=0x7f0a50009040 comp rx=0 tx=0).stop 2023-10-08T05:37:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 -- 172.21.15.70:0/1394400845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 msgr2=0x7f0a5c0b6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a5c0b6410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00a830 tx=0x7f0a4c001d80 comp rx=0 tx=0).stop 2023-10-08T05:37:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 -- 172.21.15.70:0/1394400845 shutdown_connections 2023-10-08T05:37:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0a4806ae70 0x7f0a4806d330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a5c14fb10 0x7f0a5c1511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a5c0cc190 0x7f0a5c14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 --2- 172.21.15.70:0/1394400845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a44003970 0x7f0a5c0b6410 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 -- 172.21.15.70:0/1394400845 >> 172.21.15.70:0/1394400845 conn(0x7f0a5c099d40 msgr2=0x7f0a5c15b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:31.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 -- 172.21.15.70:0/1394400845 shutdown_connections 2023-10-08T05:37:31.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.559+0000 7f0a63997640 1 -- 172.21.15.70:0/1394400845 wait complete. 2023-10-08T05:37:31.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T05:37:31.592 INFO:tasks.ceph.mds.b.smithi181.stderr: -5600> 2023-10-08T05:36:15.302+0000 7f6c0aa5d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:37:31.688 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:37:31.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.792+0000 7fa2067c5640 1 Processor -- start 2023-10-08T05:37:31.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.793+0000 7fa2067c5640 1 -- start start 2023-10-08T05:37:31.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.794+0000 7fa2067c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006dfc0 0x7fa200070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.794+0000 7fa2067c5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000b1cc0 con 0x7fa2000afbe0 2023-10-08T05:37:31.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.794+0000 7fa2067c5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2000b2620 con 0x7fa200071070 2023-10-08T05:37:31.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.794+0000 7fa2067c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000b2060 con 0x7fa20006dfc0 2023-10-08T05:37:31.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.794+0000 7fa1fffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa2000afbe0 0x7fa2000b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57222/0 (socket says 172.21.15.70:57222) 2023-10-08T05:37:31.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.794+0000 7fa1fffff640 1 -- 172.21.15.70:0/107885801 learned_addr learned my addr 172.21.15.70:0/107885801 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:31.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.795+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980234108 0 0) 0x7fa2000b1cc0 con 0x7fa2000afbe0 2023-10-08T05:37:31.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.795+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1ec003520 con 0x7fa2000afbe0 2023-10-08T05:37:31.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.795+0000 7fa1ff7fe640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006dfc0 0x7fa200070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.795+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2678285349 0 0) 0x7fa1ec003520 con 0x7fa2000afbe0 2023-10-08T05:37:31.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.795+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006dfc0 msgr2=0x7fa200070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:31.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.795+0000 7fa1feffd640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006dfc0 0x7fa200070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.795+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 >> v1:172.21.15.181:6789/0 conn(0x7fa200071070 legacy=0x7fa2000cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.795+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2000b2340 con 0x7fa2000afbe0 2023-10-08T05:37:31.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.795+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa1f0001670 con 0x7fa2000afbe0 2023-10-08T05:37:31.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.795+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 >> v1:172.21.15.70:6789/0 conn(0x7fa2000afbe0 legacy=0x7fa2000b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa1feffd640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006dfc0 0x7fa1ec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa1feffd640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1ec004020 0x7fa1ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa1feffd640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec004d00 0x7fa1ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000b2060 con 0x7fa1ec004d00 2023-10-08T05:37:31.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec00bbc0 con 0x7fa1ec004020 2023-10-08T05:37:31.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec00bd80 con 0x7fa20006dfc0 2023-10-08T05:37:31.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa204d62640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec004d00 0x7fa1ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa1fffff640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006dfc0 0x7fa1ec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa1ff7fe640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1ec004020 0x7fa1ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa204d62640 1 -- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006dfc0 msgr2=0x7fa1ec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:31.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa204d62640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006dfc0 0x7fa1ec0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa204d62640 1 -- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1ec004020 msgr2=0x7fa1ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:31.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa204d62640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1ec004020 0x7fa1ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.796+0000 7fa204d62640 1 -- 172.21.15.70:0/107885801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec003520 con 0x7fa1ec004d00 2023-10-08T05:37:31.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.797+0000 7fa204d62640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec004d00 0x7fa1ec005180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8001e50 tx=0x7fa1ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1642e5b8ae79b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:31.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.798+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8003b70 con 0x7fa1ec004d00 2023-10-08T05:37:31.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.798+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f8003cd0 con 0x7fa1ec004d00 2023-10-08T05:37:31.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.798+0000 7fa1feffd640 1 -- 172.21.15.70:0/107885801 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8005310 con 0x7fa1ec004d00 2023-10-08T05:37:31.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.798+0000 7fa2067c5640 1 -- 172.21.15.70:0/107885801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec004d00 msgr2=0x7fa1ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.798+0000 7fa2067c5640 1 --2- 172.21.15.70:0/107885801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec004d00 0x7fa1ec005180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8001e50 tx=0x7fa1ec0057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.798+0000 7fa204d62640 1 -- 172.21.15.70:0/107885801 reap_dead start 2023-10-08T05:37:31.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.798+0000 7fa2067c5640 1 -- 172.21.15.70:0/107885801 shutdown_connections 2023-10-08T05:37:31.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.798+0000 7fa2067c5640 1 -- 172.21.15.70:0/107885801 >> 172.21.15.70:0/107885801 conn(0x7fa20006a5d0 msgr2=0x7fa2000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:31.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.798+0000 7fa2067c5640 1 -- 172.21.15.70:0/107885801 shutdown_connections 2023-10-08T05:37:31.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.799+0000 7fa2067c5640 1 -- 172.21.15.70:0/107885801 wait complete. 2023-10-08T05:37:31.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.800+0000 7fa2067c5640 1 Processor -- start 2023-10-08T05:37:31.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.800+0000 7fa2067c5640 1 -- start start 2023-10-08T05:37:31.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.801+0000 7fa2067c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1ec004d00 0x7fa20014fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.801+0000 7fa2067c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa20015d3b0 0x7fa20015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.801+0000 7fa2067c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006c810 0x7fa20006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa2067c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa200150230 con 0x7fa20015d3b0 2023-10-08T05:37:31.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa2067c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20006d1d0 con 0x7fa1ec004d00 2023-10-08T05:37:31.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa2067c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20015d0f0 con 0x7fa20006c810 2023-10-08T05:37:31.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa20015d3b0 0x7fa20015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa1ff7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa20015d3b0 0x7fa20015cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55740/0 (socket says 172.21.15.70:55740) 2023-10-08T05:37:31.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa1ff7fe640 1 -- 172.21.15.70:0/1865778930 learned_addr learned my addr 172.21.15.70:0/1865778930 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:31.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa1ff7fe640 1 -- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006c810 msgr2=0x7fa20006cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa1ff7fe640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006c810 0x7fa20006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa1ff7fe640 1 -- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1ec004d00 msgr2=0x7fa20014fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa1fffff640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1ec004d00 0x7fa20014fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa1ff7fe640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1ec004d00 0x7fa20014fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.802+0000 7fa1ff7fe640 1 -- 172.21.15.70:0/1865778930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20015b820 con 0x7fa20015d3b0 2023-10-08T05:37:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.803+0000 7fa1ff7fe640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa20015d3b0 0x7fa20015cbb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800f500 tx=0x7fa1e800f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.803+0000 7fa1fcff9640 1 -- 172.21.15.70:0/1865778930 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8004020 con 0x7fa20015d3b0 2023-10-08T05:37:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.803+0000 7fa1fcff9640 1 -- 172.21.15.70:0/1865778930 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e8004180 con 0x7fa20015d3b0 2023-10-08T05:37:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.803+0000 7fa1fcff9640 1 -- 172.21.15.70:0/1865778930 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8014670 con 0x7fa20015d3b0 2023-10-08T05:37:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.803+0000 7fa2067c5640 1 -- 172.21.15.70:0/1865778930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20015bb00 con 0x7fa20015d3b0 2023-10-08T05:37:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.803+0000 7fa2067c5640 1 -- 172.21.15.70:0/1865778930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20015c0c0 con 0x7fa20015d3b0 2023-10-08T05:37:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.804+0000 7fa2067c5640 1 -- 172.21.15.70:0/1865778930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2001578b0 con 0x7fa20015d3b0 2023-10-08T05:37:31.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.805+0000 7fa1fcff9640 1 -- 172.21.15.70:0/1865778930 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fa1e80147d0 con 0x7fa20015d3b0 2023-10-08T05:37:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.806+0000 7fa1fcff9640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa1e006ae70 0x7fa1e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.806+0000 7fa1fcff9640 1 -- 172.21.15.70:0/1865778930 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fa1e8089f90 con 0x7fa20015d3b0 2023-10-08T05:37:31.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.807+0000 7fa1fffff640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa1e006ae70 0x7fa1e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:31.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.808+0000 7fa1fffff640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa1e006ae70 0x7fa1e006d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0001eb0 tx=0x7fa1f0000ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:31.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:31.810+0000 7fa1fcff9640 1 -- 172.21.15.70:0/1865778930 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2001578b0 con 0x7fa20015d3b0 2023-10-08T05:37:32.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.042+0000 7fa2067c5640 1 -- 172.21.15.70:0/1865778930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa2001578b0 con 0x7fa20015d3b0 2023-10-08T05:37:32.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.560+0000 7fa1fcff9640 1 -- 172.21.15.70:0/1865778930 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa1e8054ba0 con 0x7fa20015d3b0 2023-10-08T05:37:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 -- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa1e006ae70 msgr2=0x7fa1e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa1e006ae70 0x7fa1e006d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0001eb0 tx=0x7fa1f0000ed0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 -- 172.21.15.70:0/1865778930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa20015d3b0 msgr2=0x7fa20015cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa20015d3b0 0x7fa20015cbb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800f500 tx=0x7fa1e800f9c0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 -- 172.21.15.70:0/1865778930 shutdown_connections 2023-10-08T05:37:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fa1e006ae70 0x7fa1e006d330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa20006c810 0x7fa20006cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa20015d3b0 0x7fa20015cbb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 --2- 172.21.15.70:0/1865778930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1ec004d00 0x7fa20014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 -- 172.21.15.70:0/1865778930 >> 172.21.15.70:0/1865778930 conn(0x7fa20006a5d0 msgr2=0x7fa2000c77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 -- 172.21.15.70:0/1865778930 shutdown_connections 2023-10-08T05:37:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.564+0000 7fa2067c5640 1 -- 172.21.15.70:0/1865778930 wait complete. 2023-10-08T05:37:32.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T05:37:32.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.799+0000 7fdeaf275640 1 Processor -- start 2023-10-08T05:37:32.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.799+0000 7fdeaf275640 1 -- start start 2023-10-08T05:37:32.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.801+0000 7fdeaf275640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdea80afbe0 0x7fdea80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:32.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.801+0000 7fdeaf275640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea806dfc0 0x7fdea8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:32.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.801+0000 7fdeaf275640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea80b2620 con 0x7fdea8071070 2023-10-08T05:37:32.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.801+0000 7fdeaf275640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea80b2060 con 0x7fdea80afbe0 2023-10-08T05:37:32.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.801+0000 7fdeaf275640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80b2340 con 0x7fdea806dfc0 2023-10-08T05:37:32.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.801+0000 7fdead812640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdea8071070 0x7fdea80cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57226/0 (socket says 172.21.15.70:57226) 2023-10-08T05:37:32.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.801+0000 7fdead812640 1 -- 172.21.15.70:0/2560304424 learned_addr learned my addr 172.21.15.70:0/2560304424 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:32.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.801+0000 7fdead011640 1 --2- 172.21.15.70:0/2560304424 >> v2:172.21.15.181:3300/0 conn(0x7fdea80afbe0 0x7fdea80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.801+0000 7fdeac810640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea806dfc0 0x7fdea8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.801+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468634894 0 0) 0x7fdea80b2620 con 0x7fdea8071070 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde94003520 con 0x7fdea8071070 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2494712561 0 0) 0x7fde94003520 con 0x7fdea8071070 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea806dfc0 msgr2=0x7fdea8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea806dfc0 0x7fdea8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 >> v2:172.21.15.181:3300/0 conn(0x7fdea80afbe0 msgr2=0x7fdea80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 --2- 172.21.15.70:0/2560304424 >> v2:172.21.15.181:3300/0 conn(0x7fdea80afbe0 0x7fdea80b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea8070350 con 0x7fdea8071070 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fdea00019f0 con 0x7fdea8071070 2023-10-08T05:37:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 >> v1:172.21.15.70:6789/0 conn(0x7fdea8071070 legacy=0x7fdea80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde94003970 0x7fde94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea806dfc0 0x7fde940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 0x7fde94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80b2060 con 0x7fde94004d00 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80b2340 con 0x7fde94003970 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9400ba30 con 0x7fdea806dfc0 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.802+0000 7fdead812640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 0x7fde94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.803+0000 7fdead011640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde94003970 0x7fde94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.803+0000 7fdeac810640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea806dfc0 0x7fde940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.803+0000 7fdead812640 1 -- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea806dfc0 msgr2=0x7fde940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:32.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.803+0000 7fdead812640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea806dfc0 0x7fde940047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.803+0000 7fdead812640 1 -- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde94003970 msgr2=0x7fde94003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:32.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.803+0000 7fdead812640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde94003970 0x7fde94003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.803+0000 7fdead812640 1 -- 172.21.15.70:0/2560304424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde94003520 con 0x7fde94004d00 2023-10-08T05:37:32.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.804+0000 7fdead812640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 0x7fde94005180 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdea000a790 tx=0x7fde940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23824efdf217fa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:32.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.804+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea0034630 con 0x7fde94004d00 2023-10-08T05:37:32.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.804+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea002f070 con 0x7fde94004d00 2023-10-08T05:37:32.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.804+0000 7fde9ffff640 1 -- 172.21.15.70:0/2560304424 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea00349f0 con 0x7fde94004d00 2023-10-08T05:37:32.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.805+0000 7fdeaf275640 1 -- 172.21.15.70:0/2560304424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 msgr2=0x7fde94005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:32.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.805+0000 7fdeaf275640 1 --2- 172.21.15.70:0/2560304424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 0x7fde94005180 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdea000a790 tx=0x7fde940057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.805+0000 7fdead812640 1 -- 172.21.15.70:0/2560304424 reap_dead start 2023-10-08T05:37:32.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.805+0000 7fdeaf275640 1 -- 172.21.15.70:0/2560304424 shutdown_connections 2023-10-08T05:37:32.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.805+0000 7fdeaf275640 1 -- 172.21.15.70:0/2560304424 >> 172.21.15.70:0/2560304424 conn(0x7fdea806a5d0 msgr2=0x7fdea80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:32.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.805+0000 7fdeaf275640 1 -- 172.21.15.70:0/2560304424 shutdown_connections 2023-10-08T05:37:32.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.805+0000 7fdeaf275640 1 -- 172.21.15.70:0/2560304424 wait complete. 2023-10-08T05:37:32.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.807+0000 7fdeaf275640 1 Processor -- start 2023-10-08T05:37:32.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.807+0000 7fdeaf275640 1 -- start start 2023-10-08T05:37:32.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.808+0000 7fdeaf275640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 0x7fdea81f1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:32.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.808+0000 7fdeaf275640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea80c3d90 0x7fdea81f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:32.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.808+0000 7fdeaf275640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea81eff50 0x7fdea815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:32.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.808+0000 7fdeaf275640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8150430 con 0x7fde94004d00 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.808+0000 7fdeaf275640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea81f1f00 con 0x7fdea80c3d90 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.808+0000 7fdeaf275640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8150680 con 0x7fdea81eff50 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.808+0000 7fdead011640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 0x7fdea81f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.808+0000 7fdead011640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 0x7fdea81f1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55768/0 (socket says 172.21.15.70:55768) 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.808+0000 7fdead011640 1 -- 172.21.15.70:0/3512025798 learned_addr learned my addr 172.21.15.70:0/3512025798 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.808+0000 7fdeac810640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea80c3d90 0x7fdea81f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fdead011640 1 -- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea81eff50 msgr2=0x7fdea815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fdead011640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea81eff50 0x7fdea815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fdead011640 1 -- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea80c3d90 msgr2=0x7fdea81f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fdead011640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea80c3d90 0x7fdea81f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:32.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fdead011640 1 -- 172.21.15.70:0/3512025798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea8150960 con 0x7fde94004d00 2023-10-08T05:37:32.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fdeac810640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea80c3d90 0x7fdea81f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:32.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fdead011640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 0x7fdea81f1280 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdea4011630 tx=0x7fdea4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:32.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fde9dffb640 1 -- 172.21.15.70:0/3512025798 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4015870 con 0x7fde94004d00 2023-10-08T05:37:32.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fde9dffb640 1 -- 172.21.15.70:0/3512025798 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea40159d0 con 0x7fde94004d00 2023-10-08T05:37:32.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fdeaf275640 1 -- 172.21.15.70:0/3512025798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea8150c40 con 0x7fde94004d00 2023-10-08T05:37:32.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fde9dffb640 1 -- 172.21.15.70:0/3512025798 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4015d40 con 0x7fde94004d00 2023-10-08T05:37:32.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.809+0000 7fdeaf275640 1 -- 172.21.15.70:0/3512025798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea80c5210 con 0x7fde94004d00 2023-10-08T05:37:32.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.810+0000 7fdeaf275640 1 -- 172.21.15.70:0/3512025798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea804e160 con 0x7fde94004d00 2023-10-08T05:37:32.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.811+0000 7fde9dffb640 1 -- 172.21.15.70:0/3512025798 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fdea400f490 con 0x7fde94004d00 2023-10-08T05:37:32.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.812+0000 7fde9dffb640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fde8c06ae70 0x7fde8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:32.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.812+0000 7fde9dffb640 1 -- 172.21.15.70:0/3512025798 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6749+0+0 (secure 0 0 0) 0x7fdea408a280 con 0x7fde94004d00 2023-10-08T05:37:32.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.813+0000 7fdeac810640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fde8c06ae70 0x7fde8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:32.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.814+0000 7fdeac810640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fde8c06ae70 0x7fde8c06d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdea80c4790 tx=0x7fde98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:32.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:32.817+0000 7fde9dffb640 1 -- 172.21.15.70:0/3512025798 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdea4054f10 con 0x7fde94004d00 2023-10-08T05:37:33.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.007+0000 7fdeaf275640 1 -- 172.21.15.70:0/3512025798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdea80b0ef0 con 0x7fde94004d00 2023-10-08T05:37:33.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.569+0000 7fde9dffb640 1 -- 172.21.15.70:0/3512025798 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdea4058530 con 0x7fde94004d00 2023-10-08T05:37:33.570 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T05:37:33.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.573+0000 7fdeaf275640 1 -- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fde8c06ae70 msgr2=0x7fde8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:33.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.573+0000 7fdeaf275640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fde8c06ae70 0x7fde8c06d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdea80c4790 tx=0x7fde98008040 comp rx=0 tx=0).stop 2023-10-08T05:37:33.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.573+0000 7fdeaf275640 1 -- 172.21.15.70:0/3512025798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 msgr2=0x7fdea81f1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:33.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.573+0000 7fdeaf275640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 0x7fdea81f1280 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdea4011630 tx=0x7fdea4011af0 comp rx=0 tx=0).stop 2023-10-08T05:37:33.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.575+0000 7fdeaf275640 1 -- 172.21.15.70:0/3512025798 shutdown_connections 2023-10-08T05:37:33.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.575+0000 7fdeaf275640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fde8c06ae70 0x7fde8c06d330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:33.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.575+0000 7fdeaf275640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdea81eff50 0x7fdea815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:33.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.575+0000 7fdeaf275640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdea80c3d90 0x7fdea81f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:33.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.575+0000 7fdeaf275640 1 --2- 172.21.15.70:0/3512025798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde94004d00 0x7fdea81f1280 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:33.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.575+0000 7fdeaf275640 1 -- 172.21.15.70:0/3512025798 >> 172.21.15.70:0/3512025798 conn(0x7fdea806a5d0 msgr2=0x7fdea815aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:33.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.576+0000 7fdeaf275640 1 -- 172.21.15.70:0/3512025798 shutdown_connections 2023-10-08T05:37:33.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.576+0000 7fdeaf275640 1 -- 172.21.15.70:0/3512025798 wait complete. 2023-10-08T05:37:33.603 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T05:37:33.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.812+0000 7fba28524640 1 Processor -- start 2023-10-08T05:37:33.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.813+0000 7fba28524640 1 -- start start 2023-10-08T05:37:33.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.814+0000 7fba28524640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba20160660 0x7fba20070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:33.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.814+0000 7fba28524640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba200cb620 0x7fba200b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:33.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.814+0000 7fba28524640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2006e150 con 0x7fba200c9910 2023-10-08T05:37:33.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.814+0000 7fba28524640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2006de70 con 0x7fba200cb620 2023-10-08T05:37:33.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.814+0000 7fba28524640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c8e30 con 0x7fba20160660 2023-10-08T05:37:33.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.815+0000 7fba26ac1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba200cb620 0x7fba200b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:33.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.815+0000 7fba262c0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba20160660 0x7fba20070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:33.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.815+0000 7fba26ac1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba200cb620 0x7fba200b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:44410/0 (socket says 172.21.15.70:44410) 2023-10-08T05:37:33.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.815+0000 7fba26ac1640 1 -- 172.21.15.70:0/2864097010 learned_addr learned my addr 172.21.15.70:0/2864097010 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:33.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.816+0000 7fba26ac1640 1 -- 172.21.15.70:0/2864097010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba20160660 msgr2=0x7fba20070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:33.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.816+0000 7fba26ac1640 1 --2- 172.21.15.70:0/2864097010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba20160660 0x7fba20070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:33.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.816+0000 7fba26ac1640 1 -- 172.21.15.70:0/2864097010 >> v1:172.21.15.181:6789/0 conn(0x7fba200c9910 legacy=0x7fba200caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:33.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.816+0000 7fba26ac1640 1 -- 172.21.15.70:0/2864097010 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba200c9110 con 0x7fba200cb620 2023-10-08T05:37:33.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.816+0000 7fba252be640 1 -- 172.21.15.70:0/2864097010 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384108206 0 0) 0x7fba2006e150 con 0x7fba200c9910 2023-10-08T05:37:33.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.816+0000 7fba262c0640 1 --2- 172.21.15.70:0/2864097010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba20160660 0x7fba20070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:37:33.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.817+0000 7fba26ac1640 1 --2- 172.21.15.70:0/2864097010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba200cb620 0x7fba200b1990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fba18009e50 tx=0x7fba1802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ba864e06a50206 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:33.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.818+0000 7fba252be640 1 -- 172.21.15.70:0/2864097010 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1802cb00 con 0x7fba200cb620 2023-10-08T05:37:33.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.818+0000 7fba252be640 1 -- 172.21.15.70:0/2864097010 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1802cc60 con 0x7fba200cb620 2023-10-08T05:37:33.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.818+0000 7fba252be640 1 -- 172.21.15.70:0/2864097010 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba180343f0 con 0x7fba200cb620 2023-10-08T05:37:33.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.818+0000 7fba28524640 1 -- 172.21.15.70:0/2864097010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba200cb620 msgr2=0x7fba200b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:33.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.818+0000 7fba28524640 1 --2- 172.21.15.70:0/2864097010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba200cb620 0x7fba200b1990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fba18009e50 tx=0x7fba1802bf60 comp rx=0 tx=0).stop 2023-10-08T05:37:33.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.818+0000 7fba28524640 1 -- 172.21.15.70:0/2864097010 shutdown_connections 2023-10-08T05:37:33.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.818+0000 7fba28524640 1 --2- 172.21.15.70:0/2864097010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba200cb620 0x7fba200b1990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:33.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.818+0000 7fba28524640 1 --2- 172.21.15.70:0/2864097010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba20160660 0x7fba20070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:33.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.818+0000 7fba28524640 1 -- 172.21.15.70:0/2864097010 >> 172.21.15.70:0/2864097010 conn(0x7fba2006a5d0 msgr2=0x7fba200b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:33.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.819+0000 7fba28524640 1 -- 172.21.15.70:0/2864097010 shutdown_connections 2023-10-08T05:37:33.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.819+0000 7fba28524640 1 -- 172.21.15.70:0/2864097010 wait complete. 2023-10-08T05:37:33.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.820+0000 7fba28524640 1 Processor -- start 2023-10-08T05:37:33.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.820+0000 7fba28524640 1 -- start start 2023-10-08T05:37:33.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.821+0000 7fba28524640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba200c9910 0x7fba20149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.821+0000 7fba28524640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba200cb620 0x7fba200ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.821+0000 7fba28524640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba20160660 0x7fba200abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.821+0000 7fba28524640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200ae500 con 0x7fba20160660 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.821+0000 7fba28524640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c4c80 con 0x7fba200cb620 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.821+0000 7fba28524640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba20149ec0 con 0x7fba200c9910 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.822+0000 7fba26ac1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba20160660 0x7fba200abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.822+0000 7fba26ac1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba20160660 0x7fba200abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59936/0 (socket says 172.21.15.70:59936) 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.822+0000 7fba26ac1640 1 -- 172.21.15.70:0/3184349066 learned_addr learned my addr 172.21.15.70:0/3184349066 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.822+0000 7fba25abf640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba200cb620 0x7fba200ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.822+0000 7fba262c0640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba200c9910 0x7fba20149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:33.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.822+0000 7fba26ac1640 1 -- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba200c9910 msgr2=0x7fba20149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:33.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.822+0000 7fba26ac1640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba200c9910 0x7fba20149980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:33.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.822+0000 7fba26ac1640 1 -- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba200cb620 msgr2=0x7fba200ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:33.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.822+0000 7fba26ac1640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba200cb620 0x7fba200ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:33.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.822+0000 7fba26ac1640 1 -- 172.21.15.70:0/3184349066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c009400 con 0x7fba20160660 2023-10-08T05:37:33.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.823+0000 7fba26ac1640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba20160660 0x7fba200abe10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fba18009e50 tx=0x7fba18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:33.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.823+0000 7fba16ffd640 1 -- 172.21.15.70:0/3184349066 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18009510 con 0x7fba20160660 2023-10-08T05:37:33.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.823+0000 7fba16ffd640 1 -- 172.21.15.70:0/3184349066 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1803e450 con 0x7fba20160660 2023-10-08T05:37:33.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.823+0000 7fba28524640 1 -- 172.21.15.70:0/3184349066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba18009b30 con 0x7fba20160660 2023-10-08T05:37:33.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.823+0000 7fba16ffd640 1 -- 172.21.15.70:0/3184349066 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1803e800 con 0x7fba20160660 2023-10-08T05:37:33.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.823+0000 7fba28524640 1 -- 172.21.15.70:0/3184349066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba201496f0 con 0x7fba20160660 2023-10-08T05:37:33.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.824+0000 7fba28524640 1 -- 172.21.15.70:0/3184349066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba200ae640 con 0x7fba20160660 2023-10-08T05:37:33.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.825+0000 7fba16ffd640 1 -- 172.21.15.70:0/3184349066 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fba1803e960 con 0x7fba20160660 2023-10-08T05:37:33.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.826+0000 7fba16ffd640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fba0406aea0 0x7fba0406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:33.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.826+0000 7fba16ffd640 1 -- 172.21.15.70:0/3184349066 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fba180ac690 con 0x7fba20160660 2023-10-08T05:37:33.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.827+0000 7fba262c0640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fba0406aea0 0x7fba0406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:33.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.828+0000 7fba262c0640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fba0406aea0 0x7fba0406d360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009bc0 tx=0x7fba1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:33.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:33.831+0000 7fba16ffd640 1 -- 172.21.15.70:0/3184349066 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fba200ae640 con 0x7fba20160660 2023-10-08T05:37:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.019+0000 7fba28524640 1 -- 172.21.15.70:0/3184349066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fba200ae640 con 0x7fba20160660 2023-10-08T05:37:34.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.573+0000 7fba16ffd640 1 -- 172.21.15.70:0/3184349066 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7fba200ae640 con 0x7fba20160660 2023-10-08T05:37:34.574 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T05:37:34.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.576+0000 7fba28524640 1 -- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fba0406aea0 msgr2=0x7fba0406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:34.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.576+0000 7fba28524640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fba0406aea0 0x7fba0406d360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009bc0 tx=0x7fba1c008040 comp rx=0 tx=0).stop 2023-10-08T05:37:34.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.577+0000 7fba28524640 1 -- 172.21.15.70:0/3184349066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba20160660 msgr2=0x7fba200abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:34.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.577+0000 7fba28524640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba20160660 0x7fba200abe10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fba18009e50 tx=0x7fba18004a40 comp rx=0 tx=0).stop 2023-10-08T05:37:34.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.577+0000 7fba28524640 1 -- 172.21.15.70:0/3184349066 shutdown_connections 2023-10-08T05:37:34.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.577+0000 7fba28524640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fba0406aea0 0x7fba0406d360 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.577+0000 7fba28524640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba20160660 0x7fba200abe10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.577+0000 7fba28524640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba200cb620 0x7fba200ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.577+0000 7fba28524640 1 --2- 172.21.15.70:0/3184349066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba200c9910 0x7fba20149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.577+0000 7fba28524640 1 -- 172.21.15.70:0/3184349066 >> 172.21.15.70:0/3184349066 conn(0x7fba2006a5d0 msgr2=0x7fba200c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:34.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.579+0000 7fba28524640 1 -- 172.21.15.70:0/3184349066 shutdown_connections 2023-10-08T05:37:34.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.579+0000 7fba28524640 1 -- 172.21.15.70:0/3184349066 wait complete. 2023-10-08T05:37:34.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-08T05:37:34.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.807+0000 7ff743452640 1 Processor -- start 2023-10-08T05:37:34.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.807+0000 7ff743452640 1 -- start start 2023-10-08T05:37:34.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.808+0000 7ff743452640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff73c0b10c0 0x7ff73c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:34.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.808+0000 7ff743452640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c0b1960 0x7ff73c06b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:34.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.808+0000 7ff743452640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff73c0cdd00 0x7ff73c06bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:34.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.808+0000 7ff743452640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73c0d14d0 con 0x7ff73c0b10c0 2023-10-08T05:37:34.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.808+0000 7ff743452640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73c0b1df0 con 0x7ff73c0cdd00 2023-10-08T05:37:34.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.808+0000 7ff743452640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0b1680 con 0x7ff73c0b1960 2023-10-08T05:37:34.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7411ee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff73c0b10c0 0x7ff73c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:34.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7411ee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff73c0b10c0 0x7ff73c15e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59942/0 (socket says 172.21.15.70:59942) 2023-10-08T05:37:34.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7411ee640 1 -- 172.21.15.70:0/1302187901 learned_addr learned my addr 172.21.15.70:0/1302187901 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:34.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7409ed640 1 --2- 172.21.15.70:0/1302187901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c0b1960 0x7ff73c06b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:34.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7411ee640 1 -- 172.21.15.70:0/1302187901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c0b1960 msgr2=0x7ff73c06b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:34.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7411ee640 1 --2- 172.21.15.70:0/1302187901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c0b1960 0x7ff73c06b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7419ef640 1 --2- 172.21.15.70:0/1302187901 >> v2:172.21.15.181:3300/0 conn(0x7ff73c0cdd00 0x7ff73c06bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:34.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7411ee640 1 -- 172.21.15.70:0/1302187901 >> v2:172.21.15.181:3300/0 conn(0x7ff73c0cdd00 msgr2=0x7ff73c06bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:34.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7411ee640 1 --2- 172.21.15.70:0/1302187901 >> v2:172.21.15.181:3300/0 conn(0x7ff73c0cdd00 0x7ff73c06bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7411ee640 1 -- 172.21.15.70:0/1302187901 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c06b1a0 con 0x7ff73c0b10c0 2023-10-08T05:37:34.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.809+0000 7ff7409ed640 1 --2- 172.21.15.70:0/1302187901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c0b1960 0x7ff73c06b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:34.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.810+0000 7ff7411ee640 1 --2- 172.21.15.70:0/1302187901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0b10c0 0x7ff73c15e9f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0cb050 tx=0x7ff72c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d98ac85ae6bd757 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:34.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff733fff640 1 -- 172.21.15.70:0/1302187901 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0317e0 con 0x7ff73c0b10c0 2023-10-08T05:37:34.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff733fff640 1 -- 172.21.15.70:0/1302187901 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c031940 con 0x7ff73c0b10c0 2023-10-08T05:37:34.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff733fff640 1 -- 172.21.15.70:0/1302187901 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c031cd0 con 0x7ff73c0b10c0 2023-10-08T05:37:34.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff743452640 1 -- 172.21.15.70:0/1302187901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0b10c0 msgr2=0x7ff73c15e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:34.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff743452640 1 --2- 172.21.15.70:0/1302187901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0b10c0 0x7ff73c15e9f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0cb050 tx=0x7ff72c02b880 comp rx=0 tx=0).stop 2023-10-08T05:37:34.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff743452640 1 -- 172.21.15.70:0/1302187901 shutdown_connections 2023-10-08T05:37:34.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff743452640 1 --2- 172.21.15.70:0/1302187901 >> v2:172.21.15.181:3300/0 conn(0x7ff73c0cdd00 0x7ff73c06bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff743452640 1 --2- 172.21.15.70:0/1302187901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c0b1960 0x7ff73c06b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff743452640 1 --2- 172.21.15.70:0/1302187901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0b10c0 0x7ff73c15e9f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff743452640 1 -- 172.21.15.70:0/1302187901 >> 172.21.15.70:0/1302187901 conn(0x7ff73c099d40 msgr2=0x7ff73c0d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:34.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff743452640 1 -- 172.21.15.70:0/1302187901 shutdown_connections 2023-10-08T05:37:34.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.811+0000 7ff743452640 1 -- 172.21.15.70:0/1302187901 wait complete. 2023-10-08T05:37:34.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.813+0000 7ff743452640 1 Processor -- start 2023-10-08T05:37:34.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.813+0000 7ff743452640 1 -- start start 2023-10-08T05:37:34.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.814+0000 7ff743452640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c0b10c0 0x7ff73c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:34.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.814+0000 7ff743452640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0cdd00 0x7ff73c0b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:34.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.814+0000 7ff743452640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c14e7d0 0x7ff73c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:34.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.814+0000 7ff743452640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0b0410 con 0x7ff73c0cdd00 2023-10-08T05:37:34.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.814+0000 7ff743452640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0b6540 con 0x7ff73c0b10c0 2023-10-08T05:37:34.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.814+0000 7ff743452640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0b5ca0 con 0x7ff73c14e7d0 2023-10-08T05:37:34.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7409ed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0cdd00 0x7ff73c0b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7409ed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0cdd00 0x7ff73c0b52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59952/0 (socket says 172.21.15.70:59952) 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7409ed640 1 -- 172.21.15.70:0/342521928 learned_addr learned my addr 172.21.15.70:0/342521928 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7419ef640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c14e7d0 0x7ff73c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7409ed640 1 -- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c14e7d0 msgr2=0x7ff73c0cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7409ed640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c14e7d0 0x7ff73c0cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7411ee640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c0b10c0 0x7ff73c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7409ed640 1 -- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c0b10c0 msgr2=0x7ff73c0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7409ed640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c0b10c0 0x7ff73c0afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7409ed640 1 -- 172.21.15.70:0/342521928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c0094d0 con 0x7ff73c0cdd00 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.815+0000 7ff7409ed640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0cdd00 0x7ff73c0b52e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff73800cee0 tx=0x7ff73800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:34.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.816+0000 7ff731ffb640 1 -- 172.21.15.70:0/342521928 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73800d840 con 0x7ff73c0cdd00 2023-10-08T05:37:34.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.816+0000 7ff731ffb640 1 -- 172.21.15.70:0/342521928 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73800d9a0 con 0x7ff73c0cdd00 2023-10-08T05:37:34.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.816+0000 7ff731ffb640 1 -- 172.21.15.70:0/342521928 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73800dd30 con 0x7ff73c0cdd00 2023-10-08T05:37:34.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.816+0000 7ff743452640 1 -- 172.21.15.70:0/342521928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c0b5f80 con 0x7ff73c0cdd00 2023-10-08T05:37:34.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.816+0000 7ff743452640 1 -- 172.21.15.70:0/342521928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c14bc50 con 0x7ff73c0cdd00 2023-10-08T05:37:34.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.817+0000 7ff743452640 1 -- 172.21.15.70:0/342521928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73c14be40 con 0x7ff73c0cdd00 2023-10-08T05:37:34.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.818+0000 7ff731ffb640 1 -- 172.21.15.70:0/342521928 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7ff738015440 con 0x7ff73c0cdd00 2023-10-08T05:37:34.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.819+0000 7ff731ffb640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72806ae70 0x7ff72806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:34.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.819+0000 7ff731ffb640 1 -- 172.21.15.70:0/342521928 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5781+0+0 (secure 0 0 0) 0x7ff738089f60 con 0x7ff73c0cdd00 2023-10-08T05:37:34.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.819+0000 7ff7411ee640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72806ae70 0x7ff72806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:34.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.821+0000 7ff7411ee640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72806ae70 0x7ff72806d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff72c036000 tx=0x7ff72c03d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:34.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:34.823+0000 7ff731ffb640 1 -- 172.21.15.70:0/342521928 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff738055060 con 0x7ff73c0cdd00 2023-10-08T05:37:34.932 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:37:34.941 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:37:35.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.019+0000 7ff743452640 1 -- 172.21.15.70:0/342521928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff73c04e160 con 0x7ff73c0cdd00 2023-10-08T05:37:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.594+0000 7ff731ffb640 1 -- 172.21.15.70:0/342521928 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff73801a020 con 0x7ff73c0cdd00 2023-10-08T05:37:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data_ec' removed 2023-10-08T05:37:35.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 -- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72806ae70 msgr2=0x7ff72806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:35.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72806ae70 0x7ff72806d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff72c036000 tx=0x7ff72c03d040 comp rx=0 tx=0).stop 2023-10-08T05:37:35.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 -- 172.21.15.70:0/342521928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0cdd00 msgr2=0x7ff73c0b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:35.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0cdd00 0x7ff73c0b52e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff73800cee0 tx=0x7ff73800a710 comp rx=0 tx=0).stop 2023-10-08T05:37:35.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 -- 172.21.15.70:0/342521928 shutdown_connections 2023-10-08T05:37:35.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff72806ae70 0x7ff72806d330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:35.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff73c14e7d0 0x7ff73c0cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:35.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff73c0cdd00 0x7ff73c0b52e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:35.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 --2- 172.21.15.70:0/342521928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff73c0b10c0 0x7ff73c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:35.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 -- 172.21.15.70:0/342521928 >> 172.21.15.70:0/342521928 conn(0x7ff73c099d40 msgr2=0x7ff73c1605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:35.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.598+0000 7ff743452640 1 -- 172.21.15.70:0/342521928 shutdown_connections 2023-10-08T05:37:35.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.599+0000 7ff743452640 1 -- 172.21.15.70:0/342521928 wait complete. 2023-10-08T05:37:35.623 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T05:37:35.623 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T05:37:35.624 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T05:37:35.624 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:37:35.624 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T05:37:35.624 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T05:37:35.624 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T05:37:35.624 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:37:35.624 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T05:37:35.625 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T05:37:35.625 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T05:37:35.625 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:37:35.625 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T05:37:35.625 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T05:37:35.625 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T05:37:35.625 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:37:35.644 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T05:37:35.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T05:37:35.645 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T05:37:35.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T05:37:35.649 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T05:37:35.649 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T05:37:35.653 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T05:37:35.653 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T05:37:35.694 INFO:tasks.ceph.mds.c:Started 2023-10-08T05:37:35.694 DEBUG:teuthology.parallel:result is None 2023-10-08T05:37:35.701 INFO:tasks.ceph.mds.a:Started 2023-10-08T05:37:35.701 DEBUG:teuthology.parallel:result is None 2023-10-08T05:37:35.701 INFO:tasks.ceph.mds.b:Started 2023-10-08T05:37:35.702 DEBUG:teuthology.parallel:result is None 2023-10-08T05:37:35.709 INFO:tasks.ceph.mds.d:Started 2023-10-08T05:37:35.709 DEBUG:teuthology.parallel:result is None 2023-10-08T05:37:35.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T05:37:35.854 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:37:35.855 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T05:37:35.863 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:37:35.865 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T05:37:35.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.897+0000 7f4cb53ee640 1 Processor -- start 2023-10-08T05:37:35.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.897+0000 7f4cb53ee640 1 -- start start 2023-10-08T05:37:35.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.899+0000 7f4cb53ee640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4cb0071c30 0x7f4cb0069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:35.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.899+0000 7f4cb53ee640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb00ce210 con 0x7f4cb00720c0 2023-10-08T05:37:35.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.899+0000 7f4cb53ee640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb00cdb40 con 0x7f4cb01601b0 2023-10-08T05:37:35.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.899+0000 7f4cb53ee640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb00cdee0 con 0x7f4cb0071c30 2023-10-08T05:37:35.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.899+0000 7f4caf7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4cb00720c0 0x7f4cb00ca090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39966/0 (socket says 172.21.15.70:39966) 2023-10-08T05:37:35.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.899+0000 7f4caf7fe640 1 -- 172.21.15.70:0/882932497 learned_addr learned my addr 172.21.15.70:0/882932497 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:35.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.899+0000 7f4ca67fc640 1 --2- 172.21.15.70:0/882932497 >> v2:172.21.15.181:3300/0 conn(0x7f4cb0071c30 0x7f4cb0069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:35.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.900+0000 7f4cae7fc640 1 -- 172.21.15.70:0/882932497 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610781855 0 0) 0x7f4cb00ce210 con 0x7f4cb00720c0 2023-10-08T05:37:35.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.900+0000 7f4cae7fc640 1 -- 172.21.15.70:0/882932497 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c9c003520 con 0x7f4cb00720c0 2023-10-08T05:37:35.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.900+0000 7f4ca67fc640 1 -- 172.21.15.70:0/882932497 >> v1:172.21.15.181:6789/0 conn(0x7f4cb01601b0 legacy=0x7f4cb00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:35.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.900+0000 7f4ca67fc640 1 -- 172.21.15.70:0/882932497 >> v1:172.21.15.70:6789/0 conn(0x7f4cb00720c0 legacy=0x7f4cb00ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:35.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.900+0000 7f4ca67fc640 1 -- 172.21.15.70:0/882932497 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb0161530 con 0x7f4cb0071c30 2023-10-08T05:37:35.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.901+0000 7f4ca67fc640 1 --2- 172.21.15.70:0/882932497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cb0071c30 0x7f4cb0069d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c980097a0 tx=0x7f4c9802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a44aeaad53c112b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:35.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.901+0000 7f4cae7fc640 1 -- 172.21.15.70:0/882932497 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98031830 con 0x7f4cb0071c30 2023-10-08T05:37:35.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.901+0000 7f4cae7fc640 1 -- 172.21.15.70:0/882932497 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c98031990 con 0x7f4cb0071c30 2023-10-08T05:37:35.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.901+0000 7f4cae7fc640 1 -- 172.21.15.70:0/882932497 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98031d20 con 0x7f4cb0071c30 2023-10-08T05:37:35.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.902+0000 7f4cb53ee640 1 -- 172.21.15.70:0/882932497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cb0071c30 msgr2=0x7f4cb0069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:35.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.902+0000 7f4cb53ee640 1 --2- 172.21.15.70:0/882932497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cb0071c30 0x7f4cb0069d50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c980097a0 tx=0x7f4c9802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:37:35.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.902+0000 7f4cb53ee640 1 -- 172.21.15.70:0/882932497 shutdown_connections 2023-10-08T05:37:35.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.902+0000 7f4cb53ee640 1 --2- 172.21.15.70:0/882932497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cb0071c30 0x7f4cb0069d50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:35.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.902+0000 7f4cb53ee640 1 -- 172.21.15.70:0/882932497 >> 172.21.15.70:0/882932497 conn(0x7f4cb006d430 msgr2=0x7f4cb006d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:35.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.902+0000 7f4cb53ee640 1 -- 172.21.15.70:0/882932497 shutdown_connections 2023-10-08T05:37:35.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.902+0000 7f4cb53ee640 1 -- 172.21.15.70:0/882932497 wait complete. 2023-10-08T05:37:35.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.904+0000 7f4cb53ee640 1 Processor -- start 2023-10-08T05:37:35.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.904+0000 7f4cb53ee640 1 -- start start 2023-10-08T05:37:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4cb53ee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb0071c30 0x7f4cb01f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4cb53ee640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cb00720c0 0x7f4cb01f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4cb53ee640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cb01601b0 0x7f4cb00c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4cb53ee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00c9110 con 0x7f4cb0071c30 2023-10-08T05:37:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4cb53ee640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb00ab9e0 con 0x7f4cb01601b0 2023-10-08T05:37:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4cb53ee640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb01f3e00 con 0x7f4cb00720c0 2023-10-08T05:37:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4caeffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb0071c30 0x7f4cb01f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:35.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4caeffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb0071c30 0x7f4cb01f5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59998/0 (socket says 172.21.15.70:59998) 2023-10-08T05:37:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4caeffd640 1 -- 172.21.15.70:0/2301463193 learned_addr learned my addr 172.21.15.70:0/2301463193 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4caf7fe640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cb01601b0 0x7f4cb00c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4ca67fc640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cb00720c0 0x7f4cb01f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4caeffd640 1 -- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cb00720c0 msgr2=0x7f4cb01f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4caeffd640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cb00720c0 0x7f4cb01f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4caeffd640 1 -- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cb01601b0 msgr2=0x7f4cb00c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4caeffd640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cb01601b0 0x7f4cb00c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.905+0000 7f4caeffd640 1 -- 172.21.15.70:0/2301463193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c98009400 con 0x7f4cb0071c30 2023-10-08T05:37:35.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.906+0000 7f4caeffd640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb0071c30 0x7f4cb01f5b90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000c400 tx=0x7f4ca000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.906+0000 7f4ca7fff640 1 -- 172.21.15.70:0/2301463193 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca00052d0 con 0x7f4cb0071c30 2023-10-08T05:37:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.906+0000 7f4ca7fff640 1 -- 172.21.15.70:0/2301463193 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca0005430 con 0x7f4cb0071c30 2023-10-08T05:37:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.906+0000 7f4cb53ee640 1 -- 172.21.15.70:0/2301463193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb00c8050 con 0x7f4cb0071c30 2023-10-08T05:37:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.906+0000 7f4ca7fff640 1 -- 172.21.15.70:0/2301463193 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0003830 con 0x7f4cb0071c30 2023-10-08T05:37:35.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.906+0000 7f4cb53ee640 1 -- 172.21.15.70:0/2301463193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb00c8610 con 0x7f4cb0071c30 2023-10-08T05:37:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.907+0000 7f4cb53ee640 1 -- 172.21.15.70:0/2301463193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb004e160 con 0x7f4cb0071c30 2023-10-08T05:37:35.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.908+0000 7f4ca7fff640 1 -- 172.21.15.70:0/2301463193 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4ca0003990 con 0x7f4cb0071c30 2023-10-08T05:37:35.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.910+0000 7f4ca7fff640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4c9006ae70 0x7f4c9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:35.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.910+0000 7f4ca7fff640 1 -- 172.21.15.70:0/2301463193 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f4ca0088920 con 0x7f4cb0071c30 2023-10-08T05:37:35.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.910+0000 7f4ca67fc640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4c9006ae70 0x7f4c9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:35.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.911+0000 7f4ca67fc640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4c9006ae70 0x7f4c9006d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4c98036000 tx=0x7f4c9803c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:35.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:35.914+0000 7f4ca7fff640 1 -- 172.21.15.70:0/2301463193 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ca0053c40 con 0x7f4cb0071c30 2023-10-08T05:37:36.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.127+0000 7f4cb53ee640 1 -- 172.21.15.70:0/2301463193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4cb00b13b0 con 0x7f4cb0071c30 2023-10-08T05:37:36.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.616+0000 7f4ca7fff640 1 -- 172.21.15.70:0/2301463193 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4ca0017080 con 0x7f4cb0071c30 2023-10-08T05:37:36.617 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T05:37:36.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 -- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4c9006ae70 msgr2=0x7f4c9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:36.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4c9006ae70 0x7f4c9006d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4c98036000 tx=0x7f4c9803c040 comp rx=0 tx=0).stop 2023-10-08T05:37:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 -- 172.21.15.70:0/2301463193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb0071c30 msgr2=0x7f4cb01f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb0071c30 0x7f4cb01f5b90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4ca000c400 tx=0x7f4ca000c8c0 comp rx=0 tx=0).stop 2023-10-08T05:37:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 -- 172.21.15.70:0/2301463193 shutdown_connections 2023-10-08T05:37:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4c9006ae70 0x7f4c9006d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4cb01601b0 0x7f4cb00c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4cb00720c0 0x7f4cb01f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 --2- 172.21.15.70:0/2301463193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cb0071c30 0x7f4cb01f5b90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 -- 172.21.15.70:0/2301463193 >> 172.21.15.70:0/2301463193 conn(0x7f4cb006d430 msgr2=0x7f4cb0069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 -- 172.21.15.70:0/2301463193 shutdown_connections 2023-10-08T05:37:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.620+0000 7f4cb53ee640 1 -- 172.21.15.70:0/2301463193 wait complete. 2023-10-08T05:37:36.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T05:37:36.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.847+0000 7f4fc7fff640 1 Processor -- start 2023-10-08T05:37:36.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.847+0000 7f4fc7fff640 1 -- start start 2023-10-08T05:37:36.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.849+0000 7f4fc7fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc80b07d0 0x7f4fc80cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:36.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.849+0000 7f4fc7fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fc8164f70 0x7f4fc80c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:36.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.849+0000 7f4fc7fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc8165470 con 0x7f4fc8161320 2023-10-08T05:37:36.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.849+0000 7f4fc7fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc8165750 con 0x7f4fc8164f70 2023-10-08T05:37:36.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.849+0000 7f4fc7fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8165a30 con 0x7f4fc80b07d0 2023-10-08T05:37:36.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.849+0000 7f4fbffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc80b07d0 0x7f4fc80cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:36.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.849+0000 7f4fc77fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fc8164f70 0x7f4fc80c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:36.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.849+0000 7f4fbffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc80b07d0 0x7f4fc80cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53922/0 (socket says 172.21.15.70:53922) 2023-10-08T05:37:36.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.849+0000 7f4fbffff640 1 -- 172.21.15.70:0/2739604154 learned_addr learned my addr 172.21.15.70:0/2739604154 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:36.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.849+0000 7f4fc77fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fc8164f70 0x7f4fc80c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:44520/0 (socket says 172.21.15.70:44520) 2023-10-08T05:37:36.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.850+0000 7f4fbffff640 1 -- 172.21.15.70:0/2739604154 >> v2:172.21.15.181:3300/0 conn(0x7f4fc8164f70 msgr2=0x7f4fc80c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:36.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.850+0000 7f4fbffff640 1 --2- 172.21.15.70:0/2739604154 >> v2:172.21.15.181:3300/0 conn(0x7f4fc8164f70 0x7f4fc80c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:36.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.850+0000 7f4fbffff640 1 -- 172.21.15.70:0/2739604154 >> v1:172.21.15.181:6789/0 conn(0x7f4fc8161320 legacy=0x7f4fc80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:36.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.850+0000 7f4fbffff640 1 -- 172.21.15.70:0/2739604154 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc8165d10 con 0x7f4fc80b07d0 2023-10-08T05:37:36.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.850+0000 7f4fc77fe640 1 --2- 172.21.15.70:0/2739604154 >> v2:172.21.15.181:3300/0 conn(0x7f4fc8164f70 0x7f4fc80c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:37:36.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.852+0000 7f4fbffff640 1 --2- 172.21.15.70:0/2739604154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc80b07d0 0x7f4fc80cc210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009870 tx=0x7f4fb002ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=144d3190deaddd64 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:36.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.852+0000 7f4fc67fc640 1 -- 172.21.15.70:0/2739604154 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb002f830 con 0x7f4fc80b07d0 2023-10-08T05:37:36.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.852+0000 7f4fc67fc640 1 -- 172.21.15.70:0/2739604154 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb002f990 con 0x7f4fc80b07d0 2023-10-08T05:37:36.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.852+0000 7f4fc67fc640 1 -- 172.21.15.70:0/2739604154 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb002fd20 con 0x7f4fc80b07d0 2023-10-08T05:37:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.853+0000 7f4fc7fff640 1 -- 172.21.15.70:0/2739604154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc80b07d0 msgr2=0x7f4fc80cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.853+0000 7f4fc7fff640 1 --2- 172.21.15.70:0/2739604154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc80b07d0 0x7f4fc80cc210 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009870 tx=0x7f4fb002ba10 comp rx=0 tx=0).stop 2023-10-08T05:37:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.853+0000 7f4fc7fff640 1 -- 172.21.15.70:0/2739604154 shutdown_connections 2023-10-08T05:37:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.853+0000 7f4fc7fff640 1 --2- 172.21.15.70:0/2739604154 >> v2:172.21.15.181:3300/0 conn(0x7f4fc8164f70 0x7f4fc80c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.853+0000 7f4fc7fff640 1 --2- 172.21.15.70:0/2739604154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc80b07d0 0x7f4fc80cc210 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.853+0000 7f4fc7fff640 1 -- 172.21.15.70:0/2739604154 >> 172.21.15.70:0/2739604154 conn(0x7f4fc80cb0c0 msgr2=0x7f4fc80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.853+0000 7f4fc7fff640 1 -- 172.21.15.70:0/2739604154 shutdown_connections 2023-10-08T05:37:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.853+0000 7f4fc7fff640 1 -- 172.21.15.70:0/2739604154 wait complete. 2023-10-08T05:37:36.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.855+0000 7f4fc7fff640 1 Processor -- start 2023-10-08T05:37:36.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.855+0000 7f4fc7fff640 1 -- start start 2023-10-08T05:37:36.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.856+0000 7f4fc7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc80b07d0 0x7f4fc80c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:36.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.856+0000 7f4fc7fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc8161320 0x7f4fc80c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:36.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.856+0000 7f4fc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc8164f70 0x7f4fc80c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.856+0000 7f4fc7fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc80c8950 con 0x7f4fc8164f70 2023-10-08T05:37:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.856+0000 7f4fc7fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81640c0 con 0x7f4fc80b07d0 2023-10-08T05:37:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.856+0000 7f4fc7fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc81fafa0 con 0x7f4fc8161320 2023-10-08T05:37:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fc77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc8164f70 0x7f4fc80c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fc77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc8164f70 0x7f4fc80c4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60002/0 (socket says 172.21.15.70:60002) 2023-10-08T05:37:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fc77fe640 1 -- 172.21.15.70:0/913064633 learned_addr learned my addr 172.21.15.70:0/913064633 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fc6ffd640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc80b07d0 0x7f4fc80c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fbffff640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc8161320 0x7f4fc80c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fc77fe640 1 -- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc8161320 msgr2=0x7f4fc80c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fc77fe640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc8161320 0x7f4fc80c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fc77fe640 1 -- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc80b07d0 msgr2=0x7f4fc80c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fc77fe640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc80b07d0 0x7f4fc80c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fc77fe640 1 -- 172.21.15.70:0/913064633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc0009a70 con 0x7f4fc8164f70 2023-10-08T05:37:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.857+0000 7f4fbffff640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc8161320 0x7f4fc80c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.858+0000 7f4fc77fe640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc8164f70 0x7f4fc80c4650 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000efc0 tx=0x7f4fc000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.858+0000 7f4fbf7fe640 1 -- 172.21.15.70:0/913064633 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000cd30 con 0x7f4fc8164f70 2023-10-08T05:37:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.858+0000 7f4fbf7fe640 1 -- 172.21.15.70:0/913064633 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc000f040 con 0x7f4fc8164f70 2023-10-08T05:37:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.858+0000 7f4fc7fff640 1 -- 172.21.15.70:0/913064633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb00094d0 con 0x7f4fc8164f70 2023-10-08T05:37:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.858+0000 7f4fbf7fe640 1 -- 172.21.15.70:0/913064633 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0003bd0 con 0x7f4fc8164f70 2023-10-08T05:37:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.858+0000 7f4fc7fff640 1 -- 172.21.15.70:0/913064633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc80c2510 con 0x7f4fc8164f70 2023-10-08T05:37:36.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.859+0000 7f4fc7fff640 1 -- 172.21.15.70:0/913064633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc804e160 con 0x7f4fc8164f70 2023-10-08T05:37:36.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.860+0000 7f4fbf7fe640 1 -- 172.21.15.70:0/913064633 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4fc0014480 con 0x7f4fc8164f70 2023-10-08T05:37:36.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.861+0000 7f4fbf7fe640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4fa806aea0 0x7f4fa806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:36.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.861+0000 7f4fbf7fe640 1 -- 172.21.15.70:0/913064633 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4fc0089fe0 con 0x7f4fc8164f70 2023-10-08T05:37:36.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.862+0000 7f4fc6ffd640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4fa806aea0 0x7f4fa806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:36.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.863+0000 7f4fc6ffd640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4fa806aea0 0x7f4fa806d360 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80c45e0 tx=0x7f4fb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:36.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:36.865+0000 7f4fbf7fe640 1 -- 172.21.15.70:0/913064633 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fc0055220 con 0x7f4fc8164f70 2023-10-08T05:37:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.124+0000 7f4fc7fff640 1 -- 172.21.15.70:0/913064633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4fc80cd640 con 0x7f4fc8164f70 2023-10-08T05:37:37.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.125+0000 7f4fbf7fe640 1 -- 172.21.15.70:0/913064633 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f4fc0058840 con 0x7f4fc8164f70 2023-10-08T05:37:37.128 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:37.128 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:37:37.129 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T05:37:37.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.129 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T05:37:37.129 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T05:37:37.130 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.130 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:37:37.131 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:37:37.131 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:37:37.131 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.131 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.131 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T05:37:37.131 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T05:37:37.131 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.131 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:37:37.132 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.134 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T05:37:37.134 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T05:37:37.134 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.134 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:37:37.134 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:37:37.134 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.134 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.135 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T05:37:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T05:37:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:37:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:37:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.136 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T05:37:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T05:37:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:37:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:37:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T05:37:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T05:37:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:37:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:37:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.138 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T05:37:37.139 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:37:37.140 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:37:37.141 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.143 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T05:37:37.144 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-mds", 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQC3PyJlrKjPNxAAe4woUxo8y9sQx75YilRWsw==", 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.145 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-mds" 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-mgr", 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQC3PyJlzcXPNxAAoEnrTjID0D2BOvWuzToYxg==", 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-osd", 2023-10-08T05:37:37.146 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQC3PyJloOHPNxAAUKwvB4oIA2t4fxABQ7ZqLQ==", 2023-10-08T05:37:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-osd" 2023-10-08T05:37:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-rbd", 2023-10-08T05:37:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQC3PyJlSfnPNxAAvCILx8+Dz87+QpX/cO66Dw==", 2023-10-08T05:37:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.147 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-08T05:37:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-08T05:37:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQC3PyJlUhHQNxAAwDPbx5hFKbkjasNMGHylDw==", 2023-10-08T05:37:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-08T05:37:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-rgw", 2023-10-08T05:37:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQC3PyJlfCnQNxAAQ7uahVZj5BG2TyLmLTuuHw==", 2023-10-08T05:37:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-08T05:37:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T05:37:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T05:37:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:37:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:37:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:37:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T05:37:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T05:37:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:37:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:37:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:37:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:37:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T05:37:37.151 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:37:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 -- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4fa806aea0 msgr2=0x7f4fa806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4fa806aea0 0x7f4fa806d360 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80c45e0 tx=0x7f4fb8009040 comp rx=0 tx=0).stop 2023-10-08T05:37:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 -- 172.21.15.70:0/913064633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc8164f70 msgr2=0x7f4fc80c4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc8164f70 0x7f4fc80c4650 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000efc0 tx=0x7f4fc000c490 comp rx=0 tx=0).stop 2023-10-08T05:37:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 -- 172.21.15.70:0/913064633 shutdown_connections 2023-10-08T05:37:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f4fa806aea0 0x7f4fa806d360 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc8164f70 0x7f4fc80c4650 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fc8161320 0x7f4fc80c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 --2- 172.21.15.70:0/913064633 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fc80b07d0 0x7f4fc80c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 -- 172.21.15.70:0/913064633 >> 172.21.15.70:0/913064633 conn(0x7f4fc80cb0c0 msgr2=0x7f4fc806cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:37.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 -- 172.21.15.70:0/913064633 shutdown_connections 2023-10-08T05:37:37.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.128+0000 7f4fc7fff640 1 -- 172.21.15.70:0/913064633 wait complete. 2023-10-08T05:37:37.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-08T05:37:37.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.371+0000 7fc5ddf3c640 1 Processor -- start 2023-10-08T05:37:37.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.371+0000 7fc5ddf3c640 1 -- start start 2023-10-08T05:37:37.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.372+0000 7fc5ddf3c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5d815cae0 0x7fc5d80cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.372+0000 7fc5ddf3c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5d80c8b40 0x7fc5d80c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.372+0000 7fc5ddf3c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d80ccbc0 con 0x7fc5d80aff70 2023-10-08T05:37:37.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.372+0000 7fc5ddf3c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d80c9510 con 0x7fc5d80c8b40 2023-10-08T05:37:37.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.372+0000 7fc5ddf3c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d80ce1f0 con 0x7fc5d815cae0 2023-10-08T05:37:37.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.373+0000 7fc5d7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5d80c8b40 0x7fc5d80c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.373+0000 7fc5dd73b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc5d80aff70 0x7fc5d80b03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39970/0 (socket says 172.21.15.70:39970) 2023-10-08T05:37:37.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.373+0000 7fc5dd73b640 1 -- 172.21.15.70:0/222823975 learned_addr learned my addr 172.21.15.70:0/222823975 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:37.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.373+0000 7fc5dcf3a640 1 --2- 172.21.15.70:0/222823975 >> v2:172.21.15.181:3300/0 conn(0x7fc5d815cae0 0x7fc5d80cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.373+0000 7fc5d7fff640 1 -- 172.21.15.70:0/222823975 >> v2:172.21.15.181:3300/0 conn(0x7fc5d815cae0 msgr2=0x7fc5d80cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:37.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.373+0000 7fc5d7fff640 1 --2- 172.21.15.70:0/222823975 >> v2:172.21.15.181:3300/0 conn(0x7fc5d815cae0 0x7fc5d80cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.373+0000 7fc5d7fff640 1 -- 172.21.15.70:0/222823975 >> v1:172.21.15.70:6789/0 conn(0x7fc5d80aff70 legacy=0x7fc5d80b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.373+0000 7fc5d7fff640 1 -- 172.21.15.70:0/222823975 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d80cd590 con 0x7fc5d80c8b40 2023-10-08T05:37:37.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.374+0000 7fc5d7fff640 1 --2- 172.21.15.70:0/222823975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 0x7fc5d80c8fd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80caed0 tx=0x7fc5c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05f77d5d5ede7f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:37.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.375+0000 7fc5d77fe640 1 -- 172.21.15.70:0/222823975 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4031830 con 0x7fc5d80c8b40 2023-10-08T05:37:37.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.375+0000 7fc5d77fe640 1 -- 172.21.15.70:0/222823975 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c4031990 con 0x7fc5d80c8b40 2023-10-08T05:37:37.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.375+0000 7fc5d77fe640 1 -- 172.21.15.70:0/222823975 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4031d20 con 0x7fc5d80c8b40 2023-10-08T05:37:37.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.375+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/222823975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 msgr2=0x7fc5d80c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.375+0000 7fc5ddf3c640 1 --2- 172.21.15.70:0/222823975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 0x7fc5d80c8fd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80caed0 tx=0x7fc5c402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.375+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/222823975 shutdown_connections 2023-10-08T05:37:37.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.375+0000 7fc5ddf3c640 1 --2- 172.21.15.70:0/222823975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 0x7fc5d80c8fd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.375+0000 7fc5ddf3c640 1 --2- 172.21.15.70:0/222823975 >> v2:172.21.15.181:3300/0 conn(0x7fc5d815cae0 0x7fc5d80cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.375+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/222823975 >> 172.21.15.70:0/222823975 conn(0x7fc5d806c050 msgr2=0x7fc5d806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:37.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.376+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/222823975 shutdown_connections 2023-10-08T05:37:37.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.376+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/222823975 wait complete. 2023-10-08T05:37:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.377+0000 7fc5ddf3c640 1 Processor -- start 2023-10-08T05:37:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.378+0000 7fc5ddf3c640 1 -- start start 2023-10-08T05:37:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.378+0000 7fc5ddf3c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d80aff70 0x7fc5d81f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.378+0000 7fc5ddf3c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 0x7fc5d815f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5ddf3c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d815cae0 0x7fc5d80715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5ddf3c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d815fc80 con 0x7fc5d80c8b40 2023-10-08T05:37:37.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5ddf3c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d81f33a0 con 0x7fc5d80aff70 2023-10-08T05:37:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5ddf3c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d81f2820 con 0x7fc5d815cae0 2023-10-08T05:37:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5d7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 0x7fc5d815f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5d7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 0x7fc5d815f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60032/0 (socket says 172.21.15.70:60032) 2023-10-08T05:37:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5d7fff640 1 -- 172.21.15.70:0/2992185254 learned_addr learned my addr 172.21.15.70:0/2992185254 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5d7fff640 1 -- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d815cae0 msgr2=0x7fc5d80715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5dcf3a640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d80aff70 0x7fc5d81f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5d7fff640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d815cae0 0x7fc5d80715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5d7fff640 1 -- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d80aff70 msgr2=0x7fc5d81f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5d7fff640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d80aff70 0x7fc5d81f3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.379+0000 7fc5d7fff640 1 -- 172.21.15.70:0/2992185254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c4009400 con 0x7fc5d80c8b40 2023-10-08T05:37:37.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.380+0000 7fc5d7fff640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 0x7fc5d815f740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009890 tx=0x7fc5c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:37.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.380+0000 7fc5d57fa640 1 -- 172.21.15.70:0/2992185254 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c403ba70 con 0x7fc5d80c8b40 2023-10-08T05:37:37.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.380+0000 7fc5d57fa640 1 -- 172.21.15.70:0/2992185254 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c403bbf0 con 0x7fc5d80c8b40 2023-10-08T05:37:37.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.380+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/2992185254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d81f2b00 con 0x7fc5d80c8b40 2023-10-08T05:37:37.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.380+0000 7fc5d57fa640 1 -- 172.21.15.70:0/2992185254 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4045400 con 0x7fc5d80c8b40 2023-10-08T05:37:37.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.380+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/2992185254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d81f30c0 con 0x7fc5d80c8b40 2023-10-08T05:37:37.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.381+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/2992185254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d80c3320 con 0x7fc5d80c8b40 2023-10-08T05:37:37.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.383+0000 7fc5d57fa640 1 -- 172.21.15.70:0/2992185254 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc5c404e440 con 0x7fc5d80c8b40 2023-10-08T05:37:37.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.384+0000 7fc5d57fa640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5bc06ae70 0x7fc5bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.384+0000 7fc5d57fa640 1 -- 172.21.15.70:0/2992185254 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc5c40ab6c0 con 0x7fc5d80c8b40 2023-10-08T05:37:37.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.385+0000 7fc5dcf3a640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5bc06ae70 0x7fc5bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.386+0000 7fc5dcf3a640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5bc06ae70 0x7fc5bc06d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc006fd0 tx=0x7fc5cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:37.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.389+0000 7fc5d57fa640 1 -- 172.21.15.70:0/2992185254 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5d80c3320 con 0x7fc5d80c8b40 2023-10-08T05:37:37.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.650+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/2992185254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fc5d80c3320 con 0x7fc5d80c8b40 2023-10-08T05:37:37.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.653+0000 7fc5d57fa640 1 -- 172.21.15.70:0/2992185254 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc5d80c3320 con 0x7fc5d80c8b40 2023-10-08T05:37:37.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.655+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5bc06ae70 msgr2=0x7fc5bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.655+0000 7fc5ddf3c640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5bc06ae70 0x7fc5bc06d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc006fd0 tx=0x7fc5cc008040 comp rx=0 tx=0).stop 2023-10-08T05:37:37.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.655+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/2992185254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 msgr2=0x7fc5d815f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.655+0000 7fc5ddf3c640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 0x7fc5d815f740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009890 tx=0x7fc5c40046c0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.655+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/2992185254 shutdown_connections 2023-10-08T05:37:37.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.655+0000 7fc5ddf3c640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc5bc06ae70 0x7fc5bc06d330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.655+0000 7fc5ddf3c640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5d815cae0 0x7fc5d80715e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.655+0000 7fc5ddf3c640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5d80c8b40 0x7fc5d815f740 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.655+0000 7fc5ddf3c640 1 --2- 172.21.15.70:0/2992185254 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5d80aff70 0x7fc5d81f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.655+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/2992185254 >> 172.21.15.70:0/2992185254 conn(0x7fc5d806c050 msgr2=0x7fc5d8070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:37.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.656+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/2992185254 shutdown_connections 2023-10-08T05:37:37.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.656+0000 7fc5ddf3c640 1 -- 172.21.15.70:0/2992185254 wait complete. 2023-10-08T05:37:37.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-08T05:37:37.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.899+0000 7f8c5b78a640 1 Processor -- start 2023-10-08T05:37:37.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.900+0000 7f8c5b78a640 1 -- start start 2023-10-08T05:37:37.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.901+0000 7f8c5b78a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 0x7f8c540c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.901+0000 7f8c5b78a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c5415aa00 0x7f8c540b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.901+0000 7f8c5b78a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5415b310 con 0x7f8c5406db60 2023-10-08T05:37:37.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.901+0000 7f8c5b78a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c540ca230 con 0x7f8c5415aa00 2023-10-08T05:37:37.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.901+0000 7f8c5b78a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5406d290 con 0x7f8c5406c3a0 2023-10-08T05:37:37.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.902+0000 7f8c59d27640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c5415aa00 0x7f8c540b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.902+0000 7f8c59d27640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c5415aa00 0x7f8c540b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60046/0 (socket says 172.21.15.70:60046) 2023-10-08T05:37:37.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.902+0000 7f8c59d27640 1 -- 172.21.15.70:0/3283667147 learned_addr learned my addr 172.21.15.70:0/3283667147 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:37.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.902+0000 7f8c59526640 1 --2- 172.21.15.70:0/3283667147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 0x7f8c540c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.902+0000 7f8c59d27640 1 -- 172.21.15.70:0/3283667147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 msgr2=0x7f8c540c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:37.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.902+0000 7f8c59d27640 1 --2- 172.21.15.70:0/3283667147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 0x7f8c540c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.902+0000 7f8c59d27640 1 -- 172.21.15.70:0/3283667147 >> v1:172.21.15.181:6789/0 conn(0x7f8c5406db60 legacy=0x7f8c5406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:37.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.902+0000 7f8c59d27640 1 -- 172.21.15.70:0/3283667147 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540c9860 con 0x7f8c5415aa00 2023-10-08T05:37:37.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.902+0000 7f8c59526640 1 --2- 172.21.15.70:0/3283667147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 0x7f8c540c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:37.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.903+0000 7f8c59d27640 1 --2- 172.21.15.70:0/3283667147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 0x7f8c540b2a90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00d150 tx=0x7f8c4c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c999f0460c76a584 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:37.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.904+0000 7f8c53fff640 1 -- 172.21.15.70:0/3283667147 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c02ea80 con 0x7f8c5415aa00 2023-10-08T05:37:37.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.904+0000 7f8c53fff640 1 -- 172.21.15.70:0/3283667147 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c02ebe0 con 0x7f8c5415aa00 2023-10-08T05:37:37.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.904+0000 7f8c53fff640 1 -- 172.21.15.70:0/3283667147 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c009600 con 0x7f8c5415aa00 2023-10-08T05:37:37.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.904+0000 7f8c5b78a640 1 -- 172.21.15.70:0/3283667147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 msgr2=0x7f8c540b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.904+0000 7f8c5b78a640 1 --2- 172.21.15.70:0/3283667147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 0x7f8c540b2a90 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00d150 tx=0x7f8c4c02e0e0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.905+0000 7f8c5b78a640 1 -- 172.21.15.70:0/3283667147 shutdown_connections 2023-10-08T05:37:37.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.905+0000 7f8c5b78a640 1 --2- 172.21.15.70:0/3283667147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 0x7f8c540b2a90 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.905+0000 7f8c5b78a640 1 --2- 172.21.15.70:0/3283667147 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 0x7f8c540c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.905+0000 7f8c5b78a640 1 -- 172.21.15.70:0/3283667147 >> 172.21.15.70:0/3283667147 conn(0x7f8c5406a5d0 msgr2=0x7f8c540b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:37.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.905+0000 7f8c5b78a640 1 -- 172.21.15.70:0/3283667147 shutdown_connections 2023-10-08T05:37:37.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.905+0000 7f8c5b78a640 1 -- 172.21.15.70:0/3283667147 wait complete. 2023-10-08T05:37:37.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.907+0000 7f8c5b78a640 1 Processor -- start 2023-10-08T05:37:37.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.907+0000 7f8c5b78a640 1 -- start start 2023-10-08T05:37:37.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.908+0000 7f8c5b78a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 0x7f8c5415df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.908+0000 7f8c5b78a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5406db60 0x7f8c540c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.908+0000 7f8c5b78a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 0x7f8c54156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.908+0000 7f8c5b78a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540c3b90 con 0x7f8c5415aa00 2023-10-08T05:37:37.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.908+0000 7f8c5b78a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540ab8d0 con 0x7f8c5406db60 2023-10-08T05:37:37.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.908+0000 7f8c5b78a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540c8300 con 0x7f8c5406c3a0 2023-10-08T05:37:37.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59d27640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 0x7f8c54156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59d27640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 0x7f8c54156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60056/0 (socket says 172.21.15.70:60056) 2023-10-08T05:37:37.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59d27640 1 -- 172.21.15.70:0/2773874446 learned_addr learned my addr 172.21.15.70:0/2773874446 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:37.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59526640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 0x7f8c5415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c58d25640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5406db60 0x7f8c540c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59d27640 1 -- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 msgr2=0x7f8c5415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59d27640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 0x7f8c5415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59d27640 1 -- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5406db60 msgr2=0x7f8c540c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:37.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59d27640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5406db60 0x7f8c540c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:37.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59d27640 1 -- 172.21.15.70:0/2773874446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c00bd40 con 0x7f8c5415aa00 2023-10-08T05:37:37.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c58d25640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5406db60 0x7f8c540c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:37.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59526640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 0x7f8c5415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:37.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.909+0000 7f8c59d27640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 0x7f8c54156eb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00d270 tx=0x7f8c4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:37.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.910+0000 7f8c51ffb640 1 -- 172.21.15.70:0/2773874446 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c03f850 con 0x7f8c5415aa00 2023-10-08T05:37:37.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.910+0000 7f8c51ffb640 1 -- 172.21.15.70:0/2773874446 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c03f9b0 con 0x7f8c5415aa00 2023-10-08T05:37:37.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.910+0000 7f8c51ffb640 1 -- 172.21.15.70:0/2773874446 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c03fd60 con 0x7f8c5415aa00 2023-10-08T05:37:37.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.910+0000 7f8c5b78a640 1 -- 172.21.15.70:0/2773874446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c540c34d0 con 0x7f8c5415aa00 2023-10-08T05:37:37.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.910+0000 7f8c5b78a640 1 -- 172.21.15.70:0/2773874446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5415b750 con 0x7f8c5415aa00 2023-10-08T05:37:37.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.911+0000 7f8c5b78a640 1 -- 172.21.15.70:0/2773874446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c540c3cd0 con 0x7f8c5415aa00 2023-10-08T05:37:37.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.912+0000 7f8c51ffb640 1 -- 172.21.15.70:0/2773874446 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f8c4c02ed50 con 0x7f8c5415aa00 2023-10-08T05:37:37.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.913+0000 7f8c51ffb640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8c4006aea0 0x7f8c4006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:37.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.913+0000 7f8c51ffb640 1 -- 172.21.15.70:0/2773874446 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f8c4c0ac350 con 0x7f8c5415aa00 2023-10-08T05:37:37.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.913+0000 7f8c59526640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8c4006aea0 0x7f8c4006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:37.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.915+0000 7f8c59526640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8c4006aea0 0x7f8c4006d360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006fd0 tx=0x7f8c44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:37.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:37.917+0000 7f8c51ffb640 1 -- 172.21.15.70:0/2773874446 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c540c3cd0 con 0x7f8c5415aa00 2023-10-08T05:37:38.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.175+0000 7f8c5b78a640 1 -- 172.21.15.70:0/2773874446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f8c5415b310 con 0x7f8c5415aa00 2023-10-08T05:37:38.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.179+0000 7f8c51ffb640 1 -- 172.21.15.70:0/2773874446 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8c5415b310 con 0x7f8c5415aa00 2023-10-08T05:37:38.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.182+0000 7f8c5b78a640 1 -- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8c4006aea0 msgr2=0x7f8c4006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.182+0000 7f8c5b78a640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8c4006aea0 0x7f8c4006d360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006fd0 tx=0x7f8c44008040 comp rx=0 tx=0).stop 2023-10-08T05:37:38.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.182+0000 7f8c5b78a640 1 -- 172.21.15.70:0/2773874446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 msgr2=0x7f8c54156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.182+0000 7f8c5b78a640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 0x7f8c54156eb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00d270 tx=0x7f8c4c004a40 comp rx=0 tx=0).stop 2023-10-08T05:37:38.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.182+0000 7f8c5b78a640 1 -- 172.21.15.70:0/2773874446 shutdown_connections 2023-10-08T05:37:38.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.182+0000 7f8c5b78a640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f8c4006aea0 0x7f8c4006d360 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.182+0000 7f8c5b78a640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5415aa00 0x7f8c54156eb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.182+0000 7f8c5b78a640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c5406db60 0x7f8c540c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.182+0000 7f8c5b78a640 1 --2- 172.21.15.70:0/2773874446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c5406c3a0 0x7f8c5415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.182+0000 7f8c5b78a640 1 -- 172.21.15.70:0/2773874446 >> 172.21.15.70:0/2773874446 conn(0x7f8c5406a5d0 msgr2=0x7f8c541590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:38.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.183+0000 7f8c5b78a640 1 -- 172.21.15.70:0/2773874446 shutdown_connections 2023-10-08T05:37:38.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.183+0000 7f8c5b78a640 1 -- 172.21.15.70:0/2773874446 wait complete. 2023-10-08T05:37:38.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-08T05:37:38.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.408+0000 7f0f168e5640 1 Processor -- start 2023-10-08T05:37:38.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.409+0000 7f0f168e5640 1 -- start start 2023-10-08T05:37:38.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.410+0000 7f0f168e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f1006dfc0 0x7f0f10070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.410+0000 7f0f168e5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f100b1cc0 con 0x7f0f10071070 2023-10-08T05:37:38.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.410+0000 7f0f168e5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f100b2620 con 0x7f0f100afbe0 2023-10-08T05:37:38.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.410+0000 7f0f168e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100b2060 con 0x7f0f1006dfc0 2023-10-08T05:37:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.411+0000 7f0f14e82640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0f10071070 0x7f0f100cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39974/0 (socket says 172.21.15.70:39974) 2023-10-08T05:37:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.411+0000 7f0f14e82640 1 -- 172.21.15.70:0/1612719141 learned_addr learned my addr 172.21.15.70:0/1612719141 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.411+0000 7f0f0f7fe640 1 --2- 172.21.15.70:0/1612719141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f1006dfc0 0x7f0f10070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.411+0000 7f0f0effd640 1 -- 172.21.15.70:0/1612719141 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456292346 0 0) 0x7f0f100b1cc0 con 0x7f0f10071070 2023-10-08T05:37:38.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.411+0000 7f0f0effd640 1 -- 172.21.15.70:0/1612719141 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef8003520 con 0x7f0f10071070 2023-10-08T05:37:38.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.412+0000 7f0f0effd640 1 -- 172.21.15.70:0/1612719141 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3770726916 0 0) 0x7f0ef8003520 con 0x7f0f10071070 2023-10-08T05:37:38.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.412+0000 7f0f0f7fe640 1 -- 172.21.15.70:0/1612719141 >> v1:172.21.15.181:6789/0 conn(0x7f0f100afbe0 legacy=0x7f0f100b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.412+0000 7f0f0f7fe640 1 -- 172.21.15.70:0/1612719141 >> v1:172.21.15.70:6789/0 conn(0x7f0f10071070 legacy=0x7f0f100cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.412+0000 7f0f0f7fe640 1 -- 172.21.15.70:0/1612719141 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100b2340 con 0x7f0f1006dfc0 2023-10-08T05:37:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.413+0000 7f0f0f7fe640 1 --2- 172.21.15.70:0/1612719141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f1006dfc0 0x7f0f10070b30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f040097d0 tx=0x7f0f0402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1bc58b701dab186 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.413+0000 7f0f0effd640 1 -- 172.21.15.70:0/1612719141 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08001670 con 0x7f0f1006dfc0 2023-10-08T05:37:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.413+0000 7f0f0effd640 1 -- 172.21.15.70:0/1612719141 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0402e050 con 0x7f0f1006dfc0 2023-10-08T05:37:38.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.413+0000 7f0f0effd640 1 -- 172.21.15.70:0/1612719141 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04030710 con 0x7f0f1006dfc0 2023-10-08T05:37:38.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.414+0000 7f0f168e5640 1 -- 172.21.15.70:0/1612719141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f1006dfc0 msgr2=0x7f0f10070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.414+0000 7f0f168e5640 1 --2- 172.21.15.70:0/1612719141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f1006dfc0 0x7f0f10070b30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f040097d0 tx=0x7f0f0402b800 comp rx=0 tx=0).stop 2023-10-08T05:37:38.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.414+0000 7f0f168e5640 1 -- 172.21.15.70:0/1612719141 shutdown_connections 2023-10-08T05:37:38.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.414+0000 7f0f168e5640 1 --2- 172.21.15.70:0/1612719141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f1006dfc0 0x7f0f10070b30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.414+0000 7f0f168e5640 1 -- 172.21.15.70:0/1612719141 >> 172.21.15.70:0/1612719141 conn(0x7f0f1006a5d0 msgr2=0x7f0f100b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:38.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.414+0000 7f0f168e5640 1 -- 172.21.15.70:0/1612719141 shutdown_connections 2023-10-08T05:37:38.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.414+0000 7f0f168e5640 1 -- 172.21.15.70:0/1612719141 wait complete. 2023-10-08T05:37:38.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.416+0000 7f0f168e5640 1 Processor -- start 2023-10-08T05:37:38.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.417+0000 7f0f168e5640 1 -- start start 2023-10-08T05:37:38.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.418+0000 7f0f168e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f1006dfc0 0x7f0f1015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.418+0000 7f0f168e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f10071070 0x7f0f1015c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.418+0000 7f0f168e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f100afbe0 0x7f0f1015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.418+0000 7f0f168e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100cc640 con 0x7f0f1006dfc0 2023-10-08T05:37:38.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.418+0000 7f0f168e5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100c4d50 con 0x7f0f100afbe0 2023-10-08T05:37:38.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.418+0000 7f0f168e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100ccde0 con 0x7f0f10071070 2023-10-08T05:37:38.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.418+0000 7f0f0ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f1006dfc0 0x7f0f1015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:38.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.418+0000 7f0f0ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f1006dfc0 0x7f0f1015da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60058/0 (socket says 172.21.15.70:60058) 2023-10-08T05:37:38.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.418+0000 7f0f0ffff640 1 -- 172.21.15.70:0/29109758 learned_addr learned my addr 172.21.15.70:0/29109758 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:38.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f0ffff640 1 -- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f10071070 msgr2=0x7f0f1015c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f0ffff640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f10071070 0x7f0f1015c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f0ffff640 1 -- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f100afbe0 msgr2=0x7f0f1015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f0ffff640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f100afbe0 0x7f0f1015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f0ffff640 1 -- 172.21.15.70:0/29109758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f04009400 con 0x7f0f1006dfc0 2023-10-08T05:37:38.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f0ffff640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f1006dfc0 0x7f0f1015da10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c790 tx=0x7f0f0000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:38.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f0cff9640 1 -- 172.21.15.70:0/29109758 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000cea0 con 0x7f0f1006dfc0 2023-10-08T05:37:38.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f0cff9640 1 -- 172.21.15.70:0/29109758 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f00006d50 con 0x7f0f1006dfc0 2023-10-08T05:37:38.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f0cff9640 1 -- 172.21.15.70:0/29109758 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00003bb0 con 0x7f0f1006dfc0 2023-10-08T05:37:38.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f168e5640 1 -- 172.21.15.70:0/29109758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f100cc820 con 0x7f0f1006dfc0 2023-10-08T05:37:38.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.419+0000 7f0f168e5640 1 -- 172.21.15.70:0/29109758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1015d210 con 0x7f0f1006dfc0 2023-10-08T05:37:38.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.420+0000 7f0f168e5640 1 -- 172.21.15.70:0/29109758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1004e160 con 0x7f0f1006dfc0 2023-10-08T05:37:38.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.421+0000 7f0f0cff9640 1 -- 172.21.15.70:0/29109758 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f0f00003d10 con 0x7f0f1006dfc0 2023-10-08T05:37:38.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.422+0000 7f0f0cff9640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0efc06ae70 0x7f0efc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.422+0000 7f0f0cff9640 1 -- 172.21.15.70:0/29109758 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0f00089060 con 0x7f0f1006dfc0 2023-10-08T05:37:38.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.423+0000 7f0f0f7fe640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0efc06ae70 0x7f0efc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:38.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.424+0000 7f0f0f7fe640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0efc06ae70 0x7f0efc06d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0f04034000 tx=0x7f0f0402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:38.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.427+0000 7f0f0cff9640 1 -- 172.21.15.70:0/29109758 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f000bbca0 con 0x7f0f1006dfc0 2023-10-08T05:37:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.665+0000 7f0f168e5640 1 -- 172.21.15.70:0/29109758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f0f10070320 con 0x7f0f1006dfc0 2023-10-08T05:37:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.669+0000 7f0f0cff9640 1 -- 172.21.15.70:0/29109758 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0f0005ce60 con 0x7f0f1006dfc0 2023-10-08T05:37:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 -- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0efc06ae70 msgr2=0x7f0efc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0efc06ae70 0x7f0efc06d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0f04034000 tx=0x7f0f0402f000 comp rx=0 tx=0).stop 2023-10-08T05:37:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 -- 172.21.15.70:0/29109758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f1006dfc0 msgr2=0x7f0f1015da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f1006dfc0 0x7f0f1015da10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c790 tx=0x7f0f0000cc50 comp rx=0 tx=0).stop 2023-10-08T05:37:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 -- 172.21.15.70:0/29109758 shutdown_connections 2023-10-08T05:37:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f0efc06ae70 0x7f0efc06d330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0f100afbe0 0x7f0f1015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0f10071070 0x7f0f1015c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 --2- 172.21.15.70:0/29109758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f1006dfc0 0x7f0f1015da10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 -- 172.21.15.70:0/29109758 >> 172.21.15.70:0/29109758 conn(0x7f0f1006a5d0 msgr2=0x7f0f100c6340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 -- 172.21.15.70:0/29109758 shutdown_connections 2023-10-08T05:37:38.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.672+0000 7f0f168e5640 1 -- 172.21.15.70:0/29109758 wait complete. 2023-10-08T05:37:38.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-08T05:37:38.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.903+0000 7f9f0fa40640 1 Processor -- start 2023-10-08T05:37:38.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.903+0000 7f9f0fa40640 1 -- start start 2023-10-08T05:37:38.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.905+0000 7f9f0fa40640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9f080cb150 0x7f9f080cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.905+0000 7f9f0fa40640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f080b1f20 0x7f9f080b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.905+0000 7f9f0fa40640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0806e1e0 con 0x7f9f08161690 2023-10-08T05:37:38.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.905+0000 7f9f0fa40640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f080cbb40 con 0x7f9f080cb150 2023-10-08T05:37:38.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.905+0000 7f9f0fa40640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0806cc50 con 0x7f9f080b1f20 2023-10-08T05:37:38.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.906+0000 7f9f0dfdd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f080b1f20 0x7f9f080b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:38.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.906+0000 7f9f0dfdd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f080b1f20 0x7f9f080b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53972/0 (socket says 172.21.15.70:53972) 2023-10-08T05:37:38.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.906+0000 7f9f0dfdd640 1 -- 172.21.15.70:0/827169446 learned_addr learned my addr 172.21.15.70:0/827169446 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:38.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.906+0000 7f9f0cfdb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9f080cb150 0x7f9f080cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:38.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.907+0000 7f9f0dfdd640 1 -- 172.21.15.70:0/827169446 >> v2:172.21.15.181:3300/0 conn(0x7f9f080cb150 msgr2=0x7f9f080cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:38.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.907+0000 7f9f0dfdd640 1 --2- 172.21.15.70:0/827169446 >> v2:172.21.15.181:3300/0 conn(0x7f9f080cb150 0x7f9f080cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.907+0000 7f9f0dfdd640 1 -- 172.21.15.70:0/827169446 >> v1:172.21.15.181:6789/0 conn(0x7f9f08161690 legacy=0x7f9f0806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.907+0000 7f9f0dfdd640 1 -- 172.21.15.70:0/827169446 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0806cf30 con 0x7f9f080b1f20 2023-10-08T05:37:38.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.907+0000 7f9f0cfdb640 1 --2- 172.21.15.70:0/827169446 >> v2:172.21.15.181:3300/0 conn(0x7f9f080cb150 0x7f9f080cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:37:38.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.908+0000 7f9f0dfdd640 1 --2- 172.21.15.70:0/827169446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f080b1f20 0x7f9f080b2370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009870 tx=0x7f9f0002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da19e8facdb939c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:38.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.909+0000 7f9f07fff640 1 -- 172.21.15.70:0/827169446 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f00030840 con 0x7f9f080b1f20 2023-10-08T05:37:38.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.909+0000 7f9f07fff640 1 -- 172.21.15.70:0/827169446 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f000309a0 con 0x7f9f080b1f20 2023-10-08T05:37:38.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.909+0000 7f9f07fff640 1 -- 172.21.15.70:0/827169446 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f00030d30 con 0x7f9f080b1f20 2023-10-08T05:37:38.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.909+0000 7f9f0fa40640 1 -- 172.21.15.70:0/827169446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f080b1f20 msgr2=0x7f9f080b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.909+0000 7f9f0fa40640 1 --2- 172.21.15.70:0/827169446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f080b1f20 0x7f9f080b2370 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009870 tx=0x7f9f0002b980 comp rx=0 tx=0).stop 2023-10-08T05:37:38.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.909+0000 7f9f0fa40640 1 -- 172.21.15.70:0/827169446 shutdown_connections 2023-10-08T05:37:38.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.909+0000 7f9f0fa40640 1 --2- 172.21.15.70:0/827169446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f080b1f20 0x7f9f080b2370 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.909+0000 7f9f0fa40640 1 --2- 172.21.15.70:0/827169446 >> v2:172.21.15.181:3300/0 conn(0x7f9f080cb150 0x7f9f080cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.910+0000 7f9f0fa40640 1 -- 172.21.15.70:0/827169446 >> 172.21.15.70:0/827169446 conn(0x7f9f0806a5d0 msgr2=0x7f9f080b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:38.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.910+0000 7f9f0fa40640 1 -- 172.21.15.70:0/827169446 shutdown_connections 2023-10-08T05:37:38.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.910+0000 7f9f0fa40640 1 -- 172.21.15.70:0/827169446 wait complete. 2023-10-08T05:37:38.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.911+0000 7f9f0fa40640 1 Processor -- start 2023-10-08T05:37:38.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.912+0000 7f9f0fa40640 1 -- start start 2023-10-08T05:37:38.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.912+0000 7f9f0fa40640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f080b1f20 0x7f9f081f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.912+0000 7f9f0fa40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080cb150 0x7f9f080c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.912+0000 7f9f0fa40640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f08161690 0x7f9f080c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0fa40640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0814fc10 con 0x7f9f080cb150 2023-10-08T05:37:38.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0fa40640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f081f7940 con 0x7f9f080b1f20 2023-10-08T05:37:38.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0fa40640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f081f7740 con 0x7f9f08161690 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0cfdb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080cb150 0x7f9f080c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0cfdb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080cb150 0x7f9f080c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60064/0 (socket says 172.21.15.70:60064) 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0cfdb640 1 -- 172.21.15.70:0/4196761248 learned_addr learned my addr 172.21.15.70:0/4196761248 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0d7dc640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f080b1f20 0x7f9f081f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0cfdb640 1 -- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f08161690 msgr2=0x7f9f080c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0dfdd640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f08161690 0x7f9f080c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0cfdb640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f08161690 0x7f9f080c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0cfdb640 1 -- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f080b1f20 msgr2=0x7f9f081f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0cfdb640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f080b1f20 0x7f9f081f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0cfdb640 1 -- 172.21.15.70:0/4196761248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc009400 con 0x7f9f080cb150 2023-10-08T05:37:38.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.913+0000 7f9f0d7dc640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f080b1f20 0x7f9f081f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:38.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.914+0000 7f9f0cfdb640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080cb150 0x7f9f080c7820 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00e990 tx=0x7f9efc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:38.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.914+0000 7f9f05ffb640 1 -- 172.21.15.70:0/4196761248 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc00c9e0 con 0x7f9f080cb150 2023-10-08T05:37:38.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.914+0000 7f9f05ffb640 1 -- 172.21.15.70:0/4196761248 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc00cb40 con 0x7f9f080cb150 2023-10-08T05:37:38.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.914+0000 7f9f05ffb640 1 -- 172.21.15.70:0/4196761248 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc00ced0 con 0x7f9f080cb150 2023-10-08T05:37:38.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.914+0000 7f9f0fa40640 1 -- 172.21.15.70:0/4196761248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f000094d0 con 0x7f9f080cb150 2023-10-08T05:37:38.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.914+0000 7f9f0fa40640 1 -- 172.21.15.70:0/4196761248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0814fe60 con 0x7f9f080cb150 2023-10-08T05:37:38.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.915+0000 7f9f0fa40640 1 -- 172.21.15.70:0/4196761248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0804e160 con 0x7f9f080cb150 2023-10-08T05:37:38.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.916+0000 7f9f05ffb640 1 -- 172.21.15.70:0/4196761248 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9efc013470 con 0x7f9f080cb150 2023-10-08T05:37:38.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.917+0000 7f9f05ffb640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9ef406ae70 0x7f9ef406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:38.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.917+0000 7f9f05ffb640 1 -- 172.21.15.70:0/4196761248 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f9efc0896c0 con 0x7f9f080cb150 2023-10-08T05:37:38.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.918+0000 7f9f0d7dc640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9ef406ae70 0x7f9ef406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:38.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.919+0000 7f9f0d7dc640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9ef406ae70 0x7f9ef406d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80012e0 tx=0x7f9ef8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:38.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:38.921+0000 7f9f05ffb640 1 -- 172.21.15.70:0/4196761248 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9efc054970 con 0x7f9f080cb150 2023-10-08T05:37:39.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.180+0000 7f9f0fa40640 1 -- 172.21.15.70:0/4196761248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f9f080ccb30 con 0x7f9f080cb150 2023-10-08T05:37:39.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.185+0000 7f9f05ffb640 1 -- 172.21.15.70:0/4196761248 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9efc057f90 con 0x7f9f080cb150 2023-10-08T05:37:39.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.187+0000 7f9f0fa40640 1 -- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9ef406ae70 msgr2=0x7f9ef406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:39.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.187+0000 7f9f0fa40640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9ef406ae70 0x7f9ef406d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80012e0 tx=0x7f9ef8009040 comp rx=0 tx=0).stop 2023-10-08T05:37:39.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.187+0000 7f9f0fa40640 1 -- 172.21.15.70:0/4196761248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080cb150 msgr2=0x7f9f080c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:39.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.187+0000 7f9f0fa40640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080cb150 0x7f9f080c7820 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00e990 tx=0x7f9efc00ee50 comp rx=0 tx=0).stop 2023-10-08T05:37:39.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.187+0000 7f9f0fa40640 1 -- 172.21.15.70:0/4196761248 shutdown_connections 2023-10-08T05:37:39.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.187+0000 7f9f0fa40640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9ef406ae70 0x7f9ef406d330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.187+0000 7f9f0fa40640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f08161690 0x7f9f080c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.187+0000 7f9f0fa40640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080cb150 0x7f9f080c7820 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.187+0000 7f9f0fa40640 1 --2- 172.21.15.70:0/4196761248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f080b1f20 0x7f9f081f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.187+0000 7f9f0fa40640 1 -- 172.21.15.70:0/4196761248 >> 172.21.15.70:0/4196761248 conn(0x7f9f0806a5d0 msgr2=0x7f9f080b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:39.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.188+0000 7f9f0fa40640 1 -- 172.21.15.70:0/4196761248 shutdown_connections 2023-10-08T05:37:39.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.188+0000 7f9f0fa40640 1 -- 172.21.15.70:0/4196761248 wait complete. 2023-10-08T05:37:39.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-08T05:37:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.412+0000 7f14a467c640 1 Processor -- start 2023-10-08T05:37:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.413+0000 7f14a467c640 1 -- start start 2023-10-08T05:37:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.414+0000 7f14a467c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f149c06c3a0 0x7f149c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.414+0000 7f14a467c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149c06db60 0x7f149c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.414+0000 7f14a467c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f149c15aa00 0x7f149c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.414+0000 7f14a467c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149c0ca230 con 0x7f149c15aa00 2023-10-08T05:37:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.414+0000 7f14a467c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149c15b310 con 0x7f149c06c3a0 2023-10-08T05:37:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.414+0000 7f14a467c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c06d290 con 0x7f149c06db60 2023-10-08T05:37:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.414+0000 7f14a2c19640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f149c15aa00 0x7f149c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.415+0000 7f14a2c19640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f149c15aa00 0x7f149c0b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60072/0 (socket says 172.21.15.70:60072) 2023-10-08T05:37:39.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.415+0000 7f14a2c19640 1 -- 172.21.15.70:0/3411331746 learned_addr learned my addr 172.21.15.70:0/3411331746 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.415+0000 7f14a2c19640 1 -- 172.21.15.70:0/3411331746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149c06db60 msgr2=0x7f149c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.415+0000 7f14a2c19640 1 --2- 172.21.15.70:0/3411331746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149c06db60 0x7f149c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.415+0000 7f14a2c19640 1 -- 172.21.15.70:0/3411331746 >> v2:172.21.15.181:3300/0 conn(0x7f149c06c3a0 msgr2=0x7f149c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.415+0000 7f14a2c19640 1 --2- 172.21.15.70:0/3411331746 >> v2:172.21.15.181:3300/0 conn(0x7f149c06c3a0 0x7f149c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.415+0000 7f14a2c19640 1 -- 172.21.15.70:0/3411331746 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c0c9860 con 0x7f149c15aa00 2023-10-08T05:37:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.416+0000 7f14a2c19640 1 --2- 172.21.15.70:0/3411331746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c15aa00 0x7f149c0b2a90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f149c0716b0 tx=0x7f149402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde687f2fc3808ef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.416+0000 7f14a1416640 1 -- 172.21.15.70:0/3411331746 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149402ec80 con 0x7f149c15aa00 2023-10-08T05:37:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.416+0000 7f14a1416640 1 -- 172.21.15.70:0/3411331746 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149402ede0 con 0x7f149c15aa00 2023-10-08T05:37:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.416+0000 7f14a1416640 1 -- 172.21.15.70:0/3411331746 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494036610 con 0x7f149c15aa00 2023-10-08T05:37:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.417+0000 7f14a467c640 1 -- 172.21.15.70:0/3411331746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c15aa00 msgr2=0x7f149c0b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.417+0000 7f14a467c640 1 --2- 172.21.15.70:0/3411331746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c15aa00 0x7f149c0b2a90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f149c0716b0 tx=0x7f149402e0e0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.417+0000 7f14a467c640 1 -- 172.21.15.70:0/3411331746 shutdown_connections 2023-10-08T05:37:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.417+0000 7f14a467c640 1 --2- 172.21.15.70:0/3411331746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c15aa00 0x7f149c0b2a90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.417+0000 7f14a467c640 1 --2- 172.21.15.70:0/3411331746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149c06db60 0x7f149c06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.417+0000 7f14a467c640 1 --2- 172.21.15.70:0/3411331746 >> v2:172.21.15.181:3300/0 conn(0x7f149c06c3a0 0x7f149c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.417+0000 7f14a467c640 1 -- 172.21.15.70:0/3411331746 >> 172.21.15.70:0/3411331746 conn(0x7f149c06a5d0 msgr2=0x7f149c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.417+0000 7f14a467c640 1 -- 172.21.15.70:0/3411331746 shutdown_connections 2023-10-08T05:37:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.417+0000 7f14a467c640 1 -- 172.21.15.70:0/3411331746 wait complete. 2023-10-08T05:37:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.419+0000 7f14a467c640 1 Processor -- start 2023-10-08T05:37:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.419+0000 7f14a467c640 1 -- start start 2023-10-08T05:37:39.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.420+0000 7f14a467c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f149c06c3a0 0x7f149c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.420+0000 7f14a467c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c06db60 0x7f149c0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.420+0000 7f14a467c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149c15aa00 0x7f149c0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.420+0000 7f14a467c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0c5f00 con 0x7f149c06db60 2023-10-08T05:37:39.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.420+0000 7f14a467c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0c4c60 con 0x7f149c06c3a0 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.420+0000 7f14a467c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149c0c58b0 con 0x7f149c15aa00 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.420+0000 7f14a1c17640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c06db60 0x7f149c0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.420+0000 7f14a1c17640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c06db60 0x7f149c0ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60082/0 (socket says 172.21.15.70:60082) 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.420+0000 7f14a1c17640 1 -- 172.21.15.70:0/756799308 learned_addr learned my addr 172.21.15.70:0/756799308 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f14a2418640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f149c06c3a0 0x7f149c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f14a1c17640 1 -- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149c15aa00 msgr2=0x7f149c0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f14a1c17640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149c15aa00 0x7f149c0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f14a1c17640 1 -- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f149c06c3a0 msgr2=0x7f149c0c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f14a1c17640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f149c06c3a0 0x7f149c0c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f14a1c17640 1 -- 172.21.15.70:0/756799308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1494009b30 con 0x7f149c06db60 2023-10-08T05:37:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f14a1c17640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c06db60 0x7f149c0ca7e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f148c00b4a0 tx=0x7f148c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f1492ffd640 1 -- 172.21.15.70:0/756799308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c010610 con 0x7f149c06db60 2023-10-08T05:37:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f1492ffd640 1 -- 172.21.15.70:0/756799308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c010770 con 0x7f149c06db60 2023-10-08T05:37:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f1492ffd640 1 -- 172.21.15.70:0/756799308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c010b20 con 0x7f149c06db60 2023-10-08T05:37:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f14a467c640 1 -- 172.21.15.70:0/756799308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149c0c5b90 con 0x7f149c06db60 2023-10-08T05:37:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.421+0000 7f14a467c640 1 -- 172.21.15.70:0/756799308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c0c4f40 con 0x7f149c06db60 2023-10-08T05:37:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.423+0000 7f14a467c640 1 -- 172.21.15.70:0/756799308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f149c04e160 con 0x7f149c06db60 2023-10-08T05:37:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.423+0000 7f1492ffd640 1 -- 172.21.15.70:0/756799308 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f148c010c80 con 0x7f149c06db60 2023-10-08T05:37:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.425+0000 7f1492ffd640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f148006ae70 0x7f148006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.425+0000 7f1492ffd640 1 -- 172.21.15.70:0/756799308 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f148c089070 con 0x7f149c06db60 2023-10-08T05:37:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.425+0000 7f14a2418640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f148006ae70 0x7f148006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:39.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.426+0000 7f14a2418640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f148006ae70 0x7f148006d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f149800a810 tx=0x7f1498008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:39.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.430+0000 7f1492ffd640 1 -- 172.21.15.70:0/756799308 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f148c054330 con 0x7f149c06db60 2023-10-08T05:37:39.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.681+0000 7f14a467c640 1 -- 172.21.15.70:0/756799308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f149c0c5d20 con 0x7f149c06db60 2023-10-08T05:37:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.685+0000 7f1492ffd640 1 -- 172.21.15.70:0/756799308 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f148c057950 con 0x7f149c06db60 2023-10-08T05:37:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.687+0000 7f14a467c640 1 -- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f148006ae70 msgr2=0x7f148006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.687+0000 7f14a467c640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f148006ae70 0x7f148006d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f149800a810 tx=0x7f1498008040 comp rx=0 tx=0).stop 2023-10-08T05:37:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.687+0000 7f14a467c640 1 -- 172.21.15.70:0/756799308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c06db60 msgr2=0x7f149c0ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.687+0000 7f14a467c640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c06db60 0x7f149c0ca7e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f148c00b4a0 tx=0x7f148c00b960 comp rx=0 tx=0).stop 2023-10-08T05:37:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.688+0000 7f14a467c640 1 -- 172.21.15.70:0/756799308 shutdown_connections 2023-10-08T05:37:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.688+0000 7f14a467c640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f148006ae70 0x7f148006d330 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.688+0000 7f14a467c640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f149c15aa00 0x7f149c0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.688+0000 7f14a467c640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f149c06db60 0x7f149c0ca7e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.688+0000 7f14a467c640 1 --2- 172.21.15.70:0/756799308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f149c06c3a0 0x7f149c0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.688+0000 7f14a467c640 1 -- 172.21.15.70:0/756799308 >> 172.21.15.70:0/756799308 conn(0x7f149c06a5d0 msgr2=0x7f149c157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.688+0000 7f14a467c640 1 -- 172.21.15.70:0/756799308 shutdown_connections 2023-10-08T05:37:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.688+0000 7f14a467c640 1 -- 172.21.15.70:0/756799308 wait complete. 2023-10-08T05:37:39.707 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-08T05:37:39.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.905+0000 7f263da4f640 1 Processor -- start 2023-10-08T05:37:39.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.906+0000 7f263da4f640 1 -- start start 2023-10-08T05:37:39.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.907+0000 7f263da4f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f26380afbe0 0x7f26380b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.907+0000 7f263da4f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f263806dfc0 0x7f2638070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.907+0000 7f263da4f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380b2620 con 0x7f2638071070 2023-10-08T05:37:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.907+0000 7f263da4f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26380b2060 con 0x7f263806dfc0 2023-10-08T05:37:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.907+0000 7f263da4f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26380b2340 con 0x7f26380afbe0 2023-10-08T05:37:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.907+0000 7f26367fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f263806dfc0 0x7f2638070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.908+0000 7f26367fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f263806dfc0 0x7f2638070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60090/0 (socket says 172.21.15.70:60090) 2023-10-08T05:37:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.908+0000 7f26367fc640 1 -- 172.21.15.70:0/1941802498 learned_addr learned my addr 172.21.15.70:0/1941802498 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.908+0000 7f2636ffd640 1 --2- 172.21.15.70:0/1941802498 >> v2:172.21.15.181:3300/0 conn(0x7f26380afbe0 0x7f26380b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.908+0000 7f26367fc640 1 -- 172.21.15.70:0/1941802498 >> v2:172.21.15.181:3300/0 conn(0x7f26380afbe0 msgr2=0x7f26380b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.908+0000 7f26367fc640 1 --2- 172.21.15.70:0/1941802498 >> v2:172.21.15.181:3300/0 conn(0x7f26380afbe0 0x7f26380b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.908+0000 7f26367fc640 1 -- 172.21.15.70:0/1941802498 >> v1:172.21.15.181:6789/0 conn(0x7f2638071070 legacy=0x7f26380cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:39.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.908+0000 7f26367fc640 1 -- 172.21.15.70:0/1941802498 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2638070350 con 0x7f263806dfc0 2023-10-08T05:37:39.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.909+0000 7f26367fc640 1 --2- 172.21.15.70:0/1941802498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f263806dfc0 0x7f2638070b30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f263806e640 tx=0x7f262c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c13f083d1d97dcc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.909+0000 7f2635ffb640 1 -- 172.21.15.70:0/1941802498 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c031870 con 0x7f263806dfc0 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.909+0000 7f2635ffb640 1 -- 172.21.15.70:0/1941802498 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c0319d0 con 0x7f263806dfc0 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.909+0000 7f2635ffb640 1 -- 172.21.15.70:0/1941802498 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c031d60 con 0x7f263806dfc0 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.910+0000 7f263da4f640 1 -- 172.21.15.70:0/1941802498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f263806dfc0 msgr2=0x7f2638070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.910+0000 7f263da4f640 1 --2- 172.21.15.70:0/1941802498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f263806dfc0 0x7f2638070b30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f263806e640 tx=0x7f262c02b910 comp rx=0 tx=0).stop 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.910+0000 7f263da4f640 1 -- 172.21.15.70:0/1941802498 shutdown_connections 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.910+0000 7f263da4f640 1 --2- 172.21.15.70:0/1941802498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f263806dfc0 0x7f2638070b30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.910+0000 7f263da4f640 1 --2- 172.21.15.70:0/1941802498 >> v2:172.21.15.181:3300/0 conn(0x7f26380afbe0 0x7f26380b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.910+0000 7f263da4f640 1 -- 172.21.15.70:0/1941802498 >> 172.21.15.70:0/1941802498 conn(0x7f263806a5d0 msgr2=0x7f26380b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.910+0000 7f263da4f640 1 -- 172.21.15.70:0/1941802498 shutdown_connections 2023-10-08T05:37:39.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.910+0000 7f263da4f640 1 -- 172.21.15.70:0/1941802498 wait complete. 2023-10-08T05:37:39.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.912+0000 7f263da4f640 1 Processor -- start 2023-10-08T05:37:39.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.912+0000 7f263da4f640 1 -- start start 2023-10-08T05:37:39.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.913+0000 7f263da4f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f263806dfc0 0x7f26380cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.913+0000 7f263da4f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2638071070 0x7f26380ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.913+0000 7f263da4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380afbe0 0x7f26380cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.913+0000 7f263da4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380ce490 con 0x7f26380afbe0 2023-10-08T05:37:39.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.913+0000 7f263da4f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380cd880 con 0x7f2638071070 2023-10-08T05:37:39.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.913+0000 7f263da4f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380cdb60 con 0x7f263806dfc0 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.913+0000 7f26377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380afbe0 0x7f26380cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.913+0000 7f26377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380afbe0 0x7f26380cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60096/0 (socket says 172.21.15.70:60096) 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.913+0000 7f26377fe640 1 -- 172.21.15.70:0/1206748061 learned_addr learned my addr 172.21.15.70:0/1206748061 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f26367fc640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2638071070 0x7f26380ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f2636ffd640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f263806dfc0 0x7f26380cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f26377fe640 1 -- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f263806dfc0 msgr2=0x7f26380cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f26377fe640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f263806dfc0 0x7f26380cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f26377fe640 1 -- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2638071070 msgr2=0x7f26380ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f26377fe640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2638071070 0x7f26380ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f26377fe640 1 -- 172.21.15.70:0/1206748061 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0094d0 con 0x7f26380afbe0 2023-10-08T05:37:39.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f26367fc640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2638071070 0x7f26380ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:39.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f26377fe640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380afbe0 0x7f26380cd050 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f263000c6d0 tx=0x7f263000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:39.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f261ffff640 1 -- 172.21.15.70:0/1206748061 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263000a840 con 0x7f26380afbe0 2023-10-08T05:37:39.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f261ffff640 1 -- 172.21.15.70:0/1206748061 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263000a9a0 con 0x7f26380afbe0 2023-10-08T05:37:39.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f261ffff640 1 -- 172.21.15.70:0/1206748061 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263000ad90 con 0x7f26380afbe0 2023-10-08T05:37:39.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.914+0000 7f263da4f640 1 -- 172.21.15.70:0/1206748061 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26380cde40 con 0x7f26380afbe0 2023-10-08T05:37:39.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.915+0000 7f263da4f640 1 -- 172.21.15.70:0/1206748061 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26380cd590 con 0x7f26380afbe0 2023-10-08T05:37:39.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.916+0000 7f263da4f640 1 -- 172.21.15.70:0/1206748061 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26380ce600 con 0x7f26380afbe0 2023-10-08T05:37:39.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.917+0000 7f261ffff640 1 -- 172.21.15.70:0/1206748061 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2630012630 con 0x7f26380afbe0 2023-10-08T05:37:39.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.917+0000 7f261ffff640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f262406ae70 0x7f262406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:39.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.918+0000 7f261ffff640 1 -- 172.21.15.70:0/1206748061 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2630089c40 con 0x7f26380afbe0 2023-10-08T05:37:39.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.918+0000 7f2636ffd640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f262406ae70 0x7f262406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:39.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.919+0000 7f2636ffd640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f262406ae70 0x7f262406d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2628006fd0 tx=0x7f2628008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:39.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:39.922+0000 7f261ffff640 1 -- 172.21.15.70:0/1206748061 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2630054e60 con 0x7f26380afbe0 2023-10-08T05:37:40.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.182+0000 7f263da4f640 1 -- 172.21.15.70:0/1206748061 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f263804e160 con 0x7f26380afbe0 2023-10-08T05:37:40.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.187+0000 7f261ffff640 1 -- 172.21.15.70:0/1206748061 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2630016020 con 0x7f26380afbe0 2023-10-08T05:37:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.189+0000 7f263da4f640 1 -- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f262406ae70 msgr2=0x7f262406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.189+0000 7f263da4f640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f262406ae70 0x7f262406d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2628006fd0 tx=0x7f2628008040 comp rx=0 tx=0).stop 2023-10-08T05:37:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.189+0000 7f263da4f640 1 -- 172.21.15.70:0/1206748061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380afbe0 msgr2=0x7f26380cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.189+0000 7f263da4f640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380afbe0 0x7f26380cd050 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f263000c6d0 tx=0x7f263000cb90 comp rx=0 tx=0).stop 2023-10-08T05:37:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.189+0000 7f263da4f640 1 -- 172.21.15.70:0/1206748061 shutdown_connections 2023-10-08T05:37:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.189+0000 7f263da4f640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f262406ae70 0x7f262406d330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.189+0000 7f263da4f640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380afbe0 0x7f26380cd050 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.189+0000 7f263da4f640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2638071070 0x7f26380ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.189+0000 7f263da4f640 1 --2- 172.21.15.70:0/1206748061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f263806dfc0 0x7f26380cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.190+0000 7f263da4f640 1 -- 172.21.15.70:0/1206748061 >> 172.21.15.70:0/1206748061 conn(0x7f263806a5d0 msgr2=0x7f2638073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.190+0000 7f263da4f640 1 -- 172.21.15.70:0/1206748061 shutdown_connections 2023-10-08T05:37:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.190+0000 7f263da4f640 1 -- 172.21.15.70:0/1206748061 wait complete. 2023-10-08T05:37:40.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:40.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.410+0000 7f1f8eeaf640 1 Processor -- start 2023-10-08T05:37:40.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.411+0000 7f1f8eeaf640 1 -- start start 2023-10-08T05:37:40.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.412+0000 7f1f8eeaf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f88165cc0 0x7f1f881680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:40.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.412+0000 7f1f8eeaf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f880c48f0 con 0x7f1f88160940 2023-10-08T05:37:40.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.412+0000 7f1f8eeaf640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f880710f0 con 0x7f1f8815cd90 2023-10-08T05:37:40.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.412+0000 7f1f8eeaf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f880c4d80 con 0x7f1f88165cc0 2023-10-08T05:37:40.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.413+0000 7f1f87fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1f88160940 0x7f1f880709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39976/0 (socket says 172.21.15.70:39976) 2023-10-08T05:37:40.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.413+0000 7f1f87fff640 1 -- 172.21.15.70:0/2521064547 learned_addr learned my addr 172.21.15.70:0/2521064547 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:40.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.413+0000 7f1f8d44c640 1 --2- 172.21.15.70:0/2521064547 >> v2:172.21.15.70:3300/0 conn(0x7f1f88165cc0 0x7f1f881680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:40.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.413+0000 7f1f877fe640 1 -- 172.21.15.70:0/2521064547 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681615656 0 0) 0x7f1f880c48f0 con 0x7f1f88160940 2023-10-08T05:37:40.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.413+0000 7f1f8d44c640 1 -- 172.21.15.70:0/2521064547 >> v1:172.21.15.181:6789/0 conn(0x7f1f8815cd90 legacy=0x7f1f880702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:40.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.413+0000 7f1f8d44c640 1 -- 172.21.15.70:0/2521064547 >> v1:172.21.15.70:6789/0 conn(0x7f1f88160940 legacy=0x7f1f880709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.413+0000 7f1f8d44c640 1 -- 172.21.15.70:0/2521064547 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f880c3ce0 con 0x7f1f88165cc0 2023-10-08T05:37:40.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.414+0000 7f1f8d44c640 1 --2- 172.21.15.70:0/2521064547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f88165cc0 0x7f1f881680b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009dc0 tx=0x7f1f8002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c108bc4b44ea4cc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:40.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.414+0000 7f1f877fe640 1 -- 172.21.15.70:0/2521064547 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8002cc40 con 0x7f1f88165cc0 2023-10-08T05:37:40.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.415+0000 7f1f877fe640 1 -- 172.21.15.70:0/2521064547 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8002cda0 con 0x7f1f88165cc0 2023-10-08T05:37:40.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.415+0000 7f1f877fe640 1 -- 172.21.15.70:0/2521064547 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80034660 con 0x7f1f88165cc0 2023-10-08T05:37:40.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.415+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/2521064547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f88165cc0 msgr2=0x7f1f881680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.415+0000 7f1f8eeaf640 1 --2- 172.21.15.70:0/2521064547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f88165cc0 0x7f1f881680b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009dc0 tx=0x7f1f8002be90 comp rx=0 tx=0).stop 2023-10-08T05:37:40.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.415+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/2521064547 shutdown_connections 2023-10-08T05:37:40.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.415+0000 7f1f8eeaf640 1 --2- 172.21.15.70:0/2521064547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f88165cc0 0x7f1f881680b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.415+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/2521064547 >> 172.21.15.70:0/2521064547 conn(0x7f1f8806a380 msgr2=0x7f1f880b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:40.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.415+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/2521064547 shutdown_connections 2023-10-08T05:37:40.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.415+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/2521064547 wait complete. 2023-10-08T05:37:40.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.417+0000 7f1f8eeaf640 1 Processor -- start 2023-10-08T05:37:40.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.417+0000 7f1f8eeaf640 1 -- start start 2023-10-08T05:37:40.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8eeaf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8815cd90 0x7f1f880c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:40.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8eeaf640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f88160940 0x7f1f88152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:40.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8eeaf640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f88165cc0 0x7f1f88148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:40.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8eeaf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88148ac0 con 0x7f1f8815cd90 2023-10-08T05:37:40.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8eeaf640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88148d60 con 0x7f1f88165cc0 2023-10-08T05:37:40.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8eeaf640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88149040 con 0x7f1f88160940 2023-10-08T05:37:40.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8cc4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8815cd90 0x7f1f880c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:40.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8cc4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8815cd90 0x7f1f880c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60118/0 (socket says 172.21.15.70:60118) 2023-10-08T05:37:40.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8cc4b640 1 -- 172.21.15.70:0/45469182 learned_addr learned my addr 172.21.15.70:0/45469182 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:40.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8d44c640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f88165cc0 0x7f1f88148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:40.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8cc4b640 1 -- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f88160940 msgr2=0x7f1f88152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8cc4b640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f88160940 0x7f1f88152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8cc4b640 1 -- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f88165cc0 msgr2=0x7f1f88148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.418+0000 7f1f8cc4b640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f88165cc0 0x7f1f88148580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.419+0000 7f1f8cc4b640 1 -- 172.21.15.70:0/45469182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009a70 con 0x7f1f8815cd90 2023-10-08T05:37:40.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.419+0000 7f1f8cc4b640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8815cd90 0x7f1f880c9370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00c430 tx=0x7f1f7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:40.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.419+0000 7f1f857fa640 1 -- 172.21.15.70:0/45469182 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c011790 con 0x7f1f8815cd90 2023-10-08T05:37:40.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.419+0000 7f1f857fa640 1 -- 172.21.15.70:0/45469182 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c0118f0 con 0x7f1f8815cd90 2023-10-08T05:37:40.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.419+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/45469182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f88149320 con 0x7f1f8815cd90 2023-10-08T05:37:40.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.419+0000 7f1f857fa640 1 -- 172.21.15.70:0/45469182 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c011ce0 con 0x7f1f8815cd90 2023-10-08T05:37:40.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.419+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/45469182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f88155090 con 0x7f1f8815cd90 2023-10-08T05:37:40.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.420+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/45469182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f88150850 con 0x7f1f8815cd90 2023-10-08T05:37:40.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.421+0000 7f1f857fa640 1 -- 172.21.15.70:0/45469182 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1f7c01c460 con 0x7f1f8815cd90 2023-10-08T05:37:40.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.422+0000 7f1f857fa640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f6c06ae70 0x7f1f6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:40.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.422+0000 7f1f857fa640 1 -- 172.21.15.70:0/45469182 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1f7c0893e0 con 0x7f1f8815cd90 2023-10-08T05:37:40.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.423+0000 7f1f87fff640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f6c06ae70 0x7f1f6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:40.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.424+0000 7f1f87fff640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f6c06ae70 0x7f1f6c06d330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f74007fd0 tx=0x7f1f74007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:40.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.426+0000 7f1f857fa640 1 -- 172.21.15.70:0/45469182 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f7c054600 con 0x7f1f8815cd90 2023-10-08T05:37:40.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.602+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/45469182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f88067490 con 0x7f1f8815cd90 2023-10-08T05:37:40.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.604+0000 7f1f857fa640 1 -- 172.21.15.70:0/45469182 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13050 (secure 0 0 0) 0x7f1f7c057c20 con 0x7f1f8815cd90 2023-10-08T05:37:40.605 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:40.605 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":35,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:37:36.627518+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f6c06ae70 msgr2=0x7f1f6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f6c06ae70 0x7f1f6c06d330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1f74007fd0 tx=0x7f1f74007920 comp rx=0 tx=0).stop 2023-10-08T05:37:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/45469182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8815cd90 msgr2=0x7f1f880c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8815cd90 0x7f1f880c9370 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00c430 tx=0x7f1f7c00c8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/45469182 shutdown_connections 2023-10-08T05:37:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f6c06ae70 0x7f1f6c06d330 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f88165cc0 0x7f1f88148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f88160940 0x7f1f88152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 --2- 172.21.15.70:0/45469182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f8815cd90 0x7f1f880c9370 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/45469182 >> 172.21.15.70:0/45469182 conn(0x7f1f8806a380 msgr2=0x7f1f880cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/45469182 shutdown_connections 2023-10-08T05:37:40.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.606+0000 7f1f8eeaf640 1 -- 172.21.15.70:0/45469182 wait complete. 2023-10-08T05:37:40.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:37:40.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:37:40.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.835+0000 7f4469ae7640 1 Processor -- start 2023-10-08T05:37:40.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.835+0000 7f4469ae7640 1 -- start start 2023-10-08T05:37:40.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.837+0000 7f4469ae7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4464146d10 0x7f4464162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:40.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.837+0000 7f4469ae7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f446406c9d0 con 0x7f4464146930 2023-10-08T05:37:40.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.837+0000 7f4469ae7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44641652e0 con 0x7f4464144020 2023-10-08T05:37:40.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.837+0000 7f4469ae7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4464165b80 con 0x7f4464146d10 2023-10-08T05:37:40.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.837+0000 7f4462ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4464146d10 0x7f4464162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:40.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.837+0000 7f44637fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4464146930 0x7f4464143910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39990/0 (socket says 172.21.15.70:39990) 2023-10-08T05:37:40.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.837+0000 7f4462ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4464146d10 0x7f4464162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60124/0 (socket says 172.21.15.70:60124) 2023-10-08T05:37:40.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.837+0000 7f44637fe640 1 -- 172.21.15.70:0/1347838160 learned_addr learned my addr 172.21.15.70:0/1347838160 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.837+0000 7f4462ffd640 1 -- 172.21.15.70:0/1347838160 >> v1:172.21.15.181:6789/0 conn(0x7f4464144020 legacy=0x7f446406ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.838+0000 7f4462ffd640 1 -- 172.21.15.70:0/1347838160 >> v1:172.21.15.70:6789/0 conn(0x7f4464146930 legacy=0x7f4464143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.838+0000 7f4462ffd640 1 -- 172.21.15.70:0/1347838160 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44641655c0 con 0x7f4464146d10 2023-10-08T05:37:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.838+0000 7f44627fc640 1 -- 172.21.15.70:0/1347838160 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720072676 0 0) 0x7f446406c9d0 con 0x7f4464146930 2023-10-08T05:37:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.839+0000 7f4462ffd640 1 --2- 172.21.15.70:0/1347838160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146d10 0x7f4464162000 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f44580097a0 tx=0x7f445802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdceb332c37607b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:40.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.839+0000 7f44627fc640 1 -- 172.21.15.70:0/1347838160 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458031830 con 0x7f4464146d10 2023-10-08T05:37:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.839+0000 7f44627fc640 1 -- 172.21.15.70:0/1347838160 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4458031990 con 0x7f4464146d10 2023-10-08T05:37:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.839+0000 7f44627fc640 1 -- 172.21.15.70:0/1347838160 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458031d20 con 0x7f4464146d10 2023-10-08T05:37:40.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.839+0000 7f4469ae7640 1 -- 172.21.15.70:0/1347838160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146d10 msgr2=0x7f4464162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.839+0000 7f4469ae7640 1 --2- 172.21.15.70:0/1347838160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146d10 0x7f4464162000 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f44580097a0 tx=0x7f445802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.840+0000 7f4469ae7640 1 -- 172.21.15.70:0/1347838160 shutdown_connections 2023-10-08T05:37:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.840+0000 7f4469ae7640 1 --2- 172.21.15.70:0/1347838160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146d10 0x7f4464162000 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.840+0000 7f4469ae7640 1 -- 172.21.15.70:0/1347838160 >> 172.21.15.70:0/1347838160 conn(0x7f4464069d50 msgr2=0x7f446406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.840+0000 7f4469ae7640 1 -- 172.21.15.70:0/1347838160 shutdown_connections 2023-10-08T05:37:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.840+0000 7f4469ae7640 1 -- 172.21.15.70:0/1347838160 wait complete. 2023-10-08T05:37:40.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.842+0000 7f4469ae7640 1 Processor -- start 2023-10-08T05:37:40.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.842+0000 7f4469ae7640 1 -- start start 2023-10-08T05:37:40.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4469ae7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4464144020 0x7f44641430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:40.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4469ae7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146930 0x7f44640b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:40.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4469ae7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4464146d10 0x7f44640b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:40.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4469ae7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44640b3680 con 0x7f4464146930 2023-10-08T05:37:40.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4469ae7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44640b3960 con 0x7f4464146d10 2023-10-08T05:37:40.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4469ae7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44640b3c40 con 0x7f4464144020 2023-10-08T05:37:40.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4462ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146930 0x7f44640b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:40.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4462ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146930 0x7f44640b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60138/0 (socket says 172.21.15.70:60138) 2023-10-08T05:37:40.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4462ffd640 1 -- 172.21.15.70:0/1283137648 learned_addr learned my addr 172.21.15.70:0/1283137648 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:40.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4462ffd640 1 -- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4464144020 msgr2=0x7f44641430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.843+0000 7f4462ffd640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4464144020 0x7f44641430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.844+0000 7f4462ffd640 1 -- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4464146d10 msgr2=0x7f44640b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.844+0000 7f4462ffd640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4464146d10 0x7f44640b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.844+0000 7f4462ffd640 1 -- 172.21.15.70:0/1283137648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4458009400 con 0x7f4464146930 2023-10-08T05:37:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.844+0000 7f4462ffd640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146930 0x7f44640b2220 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4458034040 tx=0x7f44580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.844+0000 7f444bfff640 1 -- 172.21.15.70:0/1283137648 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445803ba70 con 0x7f4464146930 2023-10-08T05:37:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.844+0000 7f444bfff640 1 -- 172.21.15.70:0/1283137648 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445803bc10 con 0x7f4464146930 2023-10-08T05:37:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.844+0000 7f4469ae7640 1 -- 172.21.15.70:0/1283137648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44640b3f20 con 0x7f4464146930 2023-10-08T05:37:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.844+0000 7f444bfff640 1 -- 172.21.15.70:0/1283137648 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44580453f0 con 0x7f4464146930 2023-10-08T05:37:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.844+0000 7f4469ae7640 1 -- 172.21.15.70:0/1283137648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4464159b00 con 0x7f4464146930 2023-10-08T05:37:40.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.845+0000 7f4469ae7640 1 -- 172.21.15.70:0/1283137648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446404e160 con 0x7f4464146930 2023-10-08T05:37:40.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.846+0000 7f444bfff640 1 -- 172.21.15.70:0/1283137648 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f445804e450 con 0x7f4464146930 2023-10-08T05:37:40.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.847+0000 7f444bfff640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f445006ae70 0x7f445006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:40.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.847+0000 7f444bfff640 1 -- 172.21.15.70:0/1283137648 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f44580ab620 con 0x7f4464146930 2023-10-08T05:37:40.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.848+0000 7f44637fe640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f445006ae70 0x7f445006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:40.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.849+0000 7f44637fe640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f445006ae70 0x7f445006d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4454007fd0 tx=0x7f4454007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:40.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:40.851+0000 7f444bfff640 1 -- 172.21.15.70:0/1283137648 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4458076a00 con 0x7f4464146930 2023-10-08T05:37:41.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.042+0000 7f4469ae7640 1 -- 172.21.15.70:0/1283137648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f446406c580 con 0x7f4464146930 2023-10-08T05:37:41.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.043+0000 7f444bfff640 1 -- 172.21.15.70:0/1283137648 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f445807a020 con 0x7f4464146930 2023-10-08T05:37:41.045 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:37:41.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.045+0000 7f4469ae7640 1 -- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f445006ae70 msgr2=0x7f445006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.045+0000 7f4469ae7640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f445006ae70 0x7f445006d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4454007fd0 tx=0x7f4454007920 comp rx=0 tx=0).stop 2023-10-08T05:37:41.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.045+0000 7f4469ae7640 1 -- 172.21.15.70:0/1283137648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146930 msgr2=0x7f44640b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.045+0000 7f4469ae7640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146930 0x7f44640b2220 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4458034040 tx=0x7f44580046c0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.046+0000 7f4469ae7640 1 -- 172.21.15.70:0/1283137648 shutdown_connections 2023-10-08T05:37:41.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.046+0000 7f4469ae7640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f445006ae70 0x7f445006d330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.046+0000 7f4469ae7640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4464146d10 0x7f44640b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.046+0000 7f4469ae7640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4464146930 0x7f44640b2220 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.046+0000 7f4469ae7640 1 --2- 172.21.15.70:0/1283137648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4464144020 0x7f44641430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.046+0000 7f4469ae7640 1 -- 172.21.15.70:0/1283137648 >> 172.21.15.70:0/1283137648 conn(0x7f4464069d50 msgr2=0x7f44640cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:41.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.046+0000 7f4469ae7640 1 -- 172.21.15.70:0/1283137648 shutdown_connections 2023-10-08T05:37:41.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.046+0000 7f4469ae7640 1 -- 172.21.15.70:0/1283137648 wait complete. 2023-10-08T05:37:41.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T05:37:41.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.273+0000 7f3559d93640 1 Processor -- start 2023-10-08T05:37:41.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.273+0000 7f3559d93640 1 -- start start 2023-10-08T05:37:41.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3559d93640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35540b07d0 0x7f35540cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3559d93640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3554164f70 0x7f35540c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3559d93640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3554165470 con 0x7f3554161320 2023-10-08T05:37:41.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3559d93640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3554165750 con 0x7f35540b07d0 2023-10-08T05:37:41.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3559d93640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3554165a30 con 0x7f3554164f70 2023-10-08T05:37:41.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3558d91640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3554161320 0x7f35540b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39992/0 (socket says 172.21.15.70:39992) 2023-10-08T05:37:41.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3558d91640 1 -- 172.21.15.70:0/1119800292 learned_addr learned my addr 172.21.15.70:0/1119800292 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:41.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3553fff640 1 --2- 172.21.15.70:0/1119800292 >> v2:172.21.15.70:3300/0 conn(0x7f35540b07d0 0x7f35540cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:41.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f35537fe640 1 -- 172.21.15.70:0/1119800292 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202286413 0 0) 0x7f3554165470 con 0x7f3554161320 2023-10-08T05:37:41.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3553fff640 1 -- 172.21.15.70:0/1119800292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3554164f70 msgr2=0x7f35540c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:41.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3553fff640 1 --2- 172.21.15.70:0/1119800292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3554164f70 0x7f35540c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3553fff640 1 -- 172.21.15.70:0/1119800292 >> v1:172.21.15.70:6789/0 conn(0x7f3554161320 legacy=0x7f35540b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.275+0000 7f3553fff640 1 -- 172.21.15.70:0/1119800292 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554165d10 con 0x7f35540b07d0 2023-10-08T05:37:41.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.276+0000 7f3553fff640 1 --2- 172.21.15.70:0/1119800292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 0x7f35540cc210 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3548009d90 tx=0x7f354802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852a5c96aba21b24 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:41.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f35537fe640 1 -- 172.21.15.70:0/1119800292 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354802cd00 con 0x7f35540b07d0 2023-10-08T05:37:41.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f35537fe640 1 -- 172.21.15.70:0/1119800292 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354802ce60 con 0x7f35540b07d0 2023-10-08T05:37:41.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f35537fe640 1 -- 172.21.15.70:0/1119800292 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548034660 con 0x7f35540b07d0 2023-10-08T05:37:41.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f3559d93640 1 -- 172.21.15.70:0/1119800292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 msgr2=0x7f35540cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f3559d93640 1 --2- 172.21.15.70:0/1119800292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 0x7f35540cc210 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3548009d90 tx=0x7f354802bf50 comp rx=0 tx=0).stop 2023-10-08T05:37:41.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f3559d93640 1 -- 172.21.15.70:0/1119800292 shutdown_connections 2023-10-08T05:37:41.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f3559d93640 1 --2- 172.21.15.70:0/1119800292 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3554164f70 0x7f35540c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f3559d93640 1 --2- 172.21.15.70:0/1119800292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 0x7f35540cc210 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f3559d93640 1 -- 172.21.15.70:0/1119800292 >> 172.21.15.70:0/1119800292 conn(0x7f35540cb0c0 msgr2=0x7f35540cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:41.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f3559d93640 1 -- 172.21.15.70:0/1119800292 shutdown_connections 2023-10-08T05:37:41.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.277+0000 7f3559d93640 1 -- 172.21.15.70:0/1119800292 wait complete. 2023-10-08T05:37:41.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.279+0000 7f3559d93640 1 Processor -- start 2023-10-08T05:37:41.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.279+0000 7f3559d93640 1 -- start start 2023-10-08T05:37:41.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.280+0000 7f3559d93640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 0x7f35540c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.280+0000 7f3559d93640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3554161320 0x7f3554155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.280+0000 7f3559d93640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3554164f70 0x7f35540b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.280+0000 7f3559d93640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3554161fa0 con 0x7f35540b07d0 2023-10-08T05:37:41.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.280+0000 7f3559d93640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35540c4c70 con 0x7f3554161320 2023-10-08T05:37:41.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.280+0000 7f3559d93640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35540c40f0 con 0x7f3554164f70 2023-10-08T05:37:41.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.280+0000 7f3558d91640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 0x7f35540c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:41.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f3558d91640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 0x7f35540c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60168/0 (socket says 172.21.15.70:60168) 2023-10-08T05:37:41.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f3558d91640 1 -- 172.21.15.70:0/3597350728 learned_addr learned my addr 172.21.15.70:0/3597350728 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:41.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f3558d91640 1 -- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3554164f70 msgr2=0x7f35540b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f3558d91640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3554164f70 0x7f35540b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f3558d91640 1 -- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3554161320 msgr2=0x7f3554155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f3558d91640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3554161320 0x7f3554155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f3558d91640 1 -- 172.21.15.70:0/3597350728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548009a70 con 0x7f35540b07d0 2023-10-08T05:37:41.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f3558d91640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 0x7f35540c3b10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f354400c4d0 tx=0x7f354400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:41.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f35517fa640 1 -- 172.21.15.70:0/3597350728 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35440074e0 con 0x7f35540b07d0 2023-10-08T05:37:41.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f35517fa640 1 -- 172.21.15.70:0/3597350728 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3544007660 con 0x7f35540b07d0 2023-10-08T05:37:41.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.281+0000 7f3559d93640 1 -- 172.21.15.70:0/3597350728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35540c43d0 con 0x7f35540b07d0 2023-10-08T05:37:41.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.282+0000 7f35517fa640 1 -- 172.21.15.70:0/3597350728 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3544004a20 con 0x7f35540b07d0 2023-10-08T05:37:41.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.282+0000 7f3559d93640 1 -- 172.21.15.70:0/3597350728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35540c4990 con 0x7f35540b07d0 2023-10-08T05:37:41.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.283+0000 7f3559d93640 1 -- 172.21.15.70:0/3597350728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3554162110 con 0x7f35540b07d0 2023-10-08T05:37:41.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.284+0000 7f35517fa640 1 -- 172.21.15.70:0/3597350728 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3544004b80 con 0x7f35540b07d0 2023-10-08T05:37:41.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.285+0000 7f35517fa640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 0x7f354006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.285+0000 7f35517fa640 1 -- 172.21.15.70:0/3597350728 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3544088950 con 0x7f35540b07d0 2023-10-08T05:37:41.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.285+0000 7f3553fff640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 0x7f354006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:41.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.286+0000 7f3553fff640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 0x7f354006d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3548036000 tx=0x7f354802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:41.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.289+0000 7f35517fa640 1 -- 172.21.15.70:0/3597350728 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3544053bf0 con 0x7f35540b07d0 2023-10-08T05:37:41.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.465+0000 7f3559d93640 1 -- 172.21.15.70:0/3597350728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f355404e160 con 0x7f35540b07d0 2023-10-08T05:37:41.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.466+0000 7f35517fa640 1 -- 172.21.15.70:0/3597350728 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3544018020 con 0x7f35540b07d0 2023-10-08T05:37:41.467 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:41.467 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:37:41.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:41.468 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T05:37:41.468 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T05:37:41.468 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:41.468 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:37:41.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.468+0000 7f3559d93640 1 -- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 msgr2=0x7f354006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.468+0000 7f3559d93640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 0x7f354006d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3548036000 tx=0x7f354802e000 comp rx=0 tx=0).stop 2023-10-08T05:37:41.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.468+0000 7f3559d93640 1 -- 172.21.15.70:0/3597350728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 msgr2=0x7f35540c3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.468+0000 7f3559d93640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 0x7f35540c3b10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f354400c4d0 tx=0x7f354400c990 comp rx=0 tx=0).stop 2023-10-08T05:37:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.469+0000 7f3559d93640 1 -- 172.21.15.70:0/3597350728 shutdown_connections 2023-10-08T05:37:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.469+0000 7f3559d93640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f354006ae70 0x7f354006d330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.469+0000 7f3559d93640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3554164f70 0x7f35540b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.469+0000 7f3559d93640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3554161320 0x7f3554155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.469+0000 7f3559d93640 1 --2- 172.21.15.70:0/3597350728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35540b07d0 0x7f35540c3b10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.469+0000 7f3559d93640 1 -- 172.21.15.70:0/3597350728 >> 172.21.15.70:0/3597350728 conn(0x7f35540cb0c0 msgr2=0x7f355406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.469+0000 7f3559d93640 1 -- 172.21.15.70:0/3597350728 shutdown_connections 2023-10-08T05:37:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.469+0000 7f3559d93640 1 -- 172.21.15.70:0/3597350728 wait complete. 2023-10-08T05:37:41.487 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T05:37:41.488 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T05:37:41.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.700+0000 7fb418fa5640 1 Processor -- start 2023-10-08T05:37:41.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.700+0000 7fb418fa5640 1 -- start start 2023-10-08T05:37:41.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.702+0000 7fb418fa5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb41415d020 0x7fb414069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.702+0000 7fb418fa5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb414071770 0x7fb41406a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.702+0000 7fb418fa5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4140ca380 con 0x7fb4140cc130 2023-10-08T05:37:41.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.702+0000 7fb418fa5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb414072070 con 0x7fb41415d020 2023-10-08T05:37:41.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.702+0000 7fb418fa5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb414072350 con 0x7fb414071770 2023-10-08T05:37:41.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.702+0000 7fb41259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb41415d020 0x7fb414069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:41.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.702+0000 7fb412d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb4140cc130 0x7fb4140cc5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39998/0 (socket says 172.21.15.70:39998) 2023-10-08T05:37:41.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.702+0000 7fb412d9d640 1 -- 172.21.15.70:0/1371301915 learned_addr learned my addr 172.21.15.70:0/1371301915 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:41.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.703+0000 7fb411d9b640 1 --2- 172.21.15.70:0/1371301915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb414071770 0x7fb41406a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:41.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.703+0000 7fb41259c640 1 -- 172.21.15.70:0/1371301915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb414071770 msgr2=0x7fb41406a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:41.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.703+0000 7fb41259c640 1 --2- 172.21.15.70:0/1371301915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb414071770 0x7fb41406a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.703+0000 7fb41259c640 1 -- 172.21.15.70:0/1371301915 >> v1:172.21.15.70:6789/0 conn(0x7fb4140cc130 legacy=0x7fb4140cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.703+0000 7fb41159a640 1 -- 172.21.15.70:0/1371301915 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970158009 0 0) 0x7fb4140ca380 con 0x7fb4140cc130 2023-10-08T05:37:41.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.703+0000 7fb41259c640 1 -- 172.21.15.70:0/1371301915 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4140cdf90 con 0x7fb41415d020 2023-10-08T05:37:41.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.704+0000 7fb41259c640 1 --2- 172.21.15.70:0/1371301915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41415d020 0x7fb414069d50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb4040097a0 tx=0x7fb40402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48cae156dc2bd3a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.704+0000 7fb41159a640 1 -- 172.21.15.70:0/1371301915 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404030800 con 0x7fb41415d020 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.704+0000 7fb41159a640 1 -- 172.21.15.70:0/1371301915 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb404030960 con 0x7fb41415d020 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.704+0000 7fb41159a640 1 -- 172.21.15.70:0/1371301915 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404030cf0 con 0x7fb41415d020 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.705+0000 7fb418fa5640 1 -- 172.21.15.70:0/1371301915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41415d020 msgr2=0x7fb414069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.705+0000 7fb418fa5640 1 --2- 172.21.15.70:0/1371301915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41415d020 0x7fb414069d50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb4040097a0 tx=0x7fb40402b940 comp rx=0 tx=0).stop 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.705+0000 7fb418fa5640 1 -- 172.21.15.70:0/1371301915 shutdown_connections 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.705+0000 7fb418fa5640 1 --2- 172.21.15.70:0/1371301915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb414071770 0x7fb41406a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.705+0000 7fb418fa5640 1 --2- 172.21.15.70:0/1371301915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41415d020 0x7fb414069d50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.705+0000 7fb418fa5640 1 -- 172.21.15.70:0/1371301915 >> 172.21.15.70:0/1371301915 conn(0x7fb41406d520 msgr2=0x7fb41406d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.705+0000 7fb418fa5640 1 -- 172.21.15.70:0/1371301915 shutdown_connections 2023-10-08T05:37:41.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.705+0000 7fb418fa5640 1 -- 172.21.15.70:0/1371301915 wait complete. 2023-10-08T05:37:41.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.707+0000 7fb418fa5640 1 Processor -- start 2023-10-08T05:37:41.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.707+0000 7fb418fa5640 1 -- start start 2023-10-08T05:37:41.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.708+0000 7fb418fa5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb414071770 0x7fb41415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.708+0000 7fb418fa5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4140cc130 0x7fb41415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.708+0000 7fb418fa5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41415d020 0x7fb41415c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.708+0000 7fb418fa5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140ac680 con 0x7fb414071770 2023-10-08T05:37:41.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.708+0000 7fb418fa5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140aba70 con 0x7fb41415d020 2023-10-08T05:37:41.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.708+0000 7fb418fa5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140abd50 con 0x7fb4140cc130 2023-10-08T05:37:41.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.708+0000 7fb41259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb414071770 0x7fb41415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:41.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.708+0000 7fb41259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb414071770 0x7fb41415a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60184/0 (socket says 172.21.15.70:60184) 2023-10-08T05:37:41.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.708+0000 7fb41259c640 1 -- 172.21.15.70:0/3767089724 learned_addr learned my addr 172.21.15.70:0/3767089724 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:41.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb41259c640 1 -- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4140cc130 msgr2=0x7fb41415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb41259c640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4140cc130 0x7fb41415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb412d9d640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41415d020 0x7fb41415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:41.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb41259c640 1 -- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41415d020 msgr2=0x7fb41415c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:41.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb41259c640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41415d020 0x7fb41415c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:41.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb41259c640 1 -- 172.21.15.70:0/3767089724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb404009400 con 0x7fb414071770 2023-10-08T05:37:41.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb41259c640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb414071770 0x7fb41415a310 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb4040098c0 tx=0x7fb4040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:41.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb402ffd640 1 -- 172.21.15.70:0/3767089724 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40403e070 con 0x7fb414071770 2023-10-08T05:37:41.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb402ffd640 1 -- 172.21.15.70:0/3767089724 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4040398c0 con 0x7fb414071770 2023-10-08T05:37:41.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb418fa5640 1 -- 172.21.15.70:0/3767089724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4140ac030 con 0x7fb414071770 2023-10-08T05:37:41.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb402ffd640 1 -- 172.21.15.70:0/3767089724 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404039c70 con 0x7fb414071770 2023-10-08T05:37:41.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.709+0000 7fb418fa5640 1 -- 172.21.15.70:0/3767089724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41415a850 con 0x7fb414071770 2023-10-08T05:37:41.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.710+0000 7fb418fa5640 1 -- 172.21.15.70:0/3767089724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41404e160 con 0x7fb414071770 2023-10-08T05:37:41.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.711+0000 7fb402ffd640 1 -- 172.21.15.70:0/3767089724 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fb404042070 con 0x7fb414071770 2023-10-08T05:37:41.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.713+0000 7fb402ffd640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb3f406ae70 0x7fb3f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:41.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.713+0000 7fb402ffd640 1 -- 172.21.15.70:0/3767089724 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb4040ab280 con 0x7fb414071770 2023-10-08T05:37:41.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.714+0000 7fb411d9b640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb3f406ae70 0x7fb3f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:41.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.715+0000 7fb411d9b640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb3f406ae70 0x7fb3f406d330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb414160a10 tx=0x7fb408008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:41.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.717+0000 7fb402ffd640 1 -- 172.21.15.70:0/3767089724 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb404076530 con 0x7fb414071770 2023-10-08T05:37:41.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:41.906+0000 7fb418fa5640 1 -- 172.21.15.70:0/3767089724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb4140cee30 con 0x7fb414071770 2023-10-08T05:37:42.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.695+0000 7fb402ffd640 1 -- 172.21.15.70:0/3767089724 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb404079b50 con 0x7fb414071770 2023-10-08T05:37:42.697 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T05:37:42.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.698+0000 7fb418fa5640 1 -- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb3f406ae70 msgr2=0x7fb3f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:42.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.698+0000 7fb418fa5640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb3f406ae70 0x7fb3f406d330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb414160a10 tx=0x7fb408008040 comp rx=0 tx=0).stop 2023-10-08T05:37:42.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.698+0000 7fb418fa5640 1 -- 172.21.15.70:0/3767089724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb414071770 msgr2=0x7fb41415a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:42.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.698+0000 7fb418fa5640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb414071770 0x7fb41415a310 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb4040098c0 tx=0x7fb4040046c0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.699+0000 7fb418fa5640 1 -- 172.21.15.70:0/3767089724 shutdown_connections 2023-10-08T05:37:42.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.699+0000 7fb418fa5640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb3f406ae70 0x7fb3f406d330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.699+0000 7fb418fa5640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb41415d020 0x7fb41415c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.699+0000 7fb418fa5640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb4140cc130 0x7fb41415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.699+0000 7fb418fa5640 1 --2- 172.21.15.70:0/3767089724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb414071770 0x7fb41415a310 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.699+0000 7fb418fa5640 1 -- 172.21.15.70:0/3767089724 >> 172.21.15.70:0/3767089724 conn(0x7fb41406d520 msgr2=0x7fb4140ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:42.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.699+0000 7fb418fa5640 1 -- 172.21.15.70:0/3767089724 shutdown_connections 2023-10-08T05:37:42.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.699+0000 7fb418fa5640 1 -- 172.21.15.70:0/3767089724 wait complete. 2023-10-08T05:37:42.728 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T05:37:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.928+0000 7f406d797640 1 Processor -- start 2023-10-08T05:37:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.928+0000 7f406d797640 1 -- start start 2023-10-08T05:37:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.929+0000 7f406d797640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815e300 0x7f40680cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.929+0000 7f406d797640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40680b2890 con 0x7f4068159d50 2023-10-08T05:37:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.929+0000 7f406d797640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406815d8b0 con 0x7f40680cba40 2023-10-08T05:37:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.929+0000 7f406d797640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40680cbeb0 con 0x7f406815e300 2023-10-08T05:37:42.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f4066ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4068159d50 0x7f40680b16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40012/0 (socket says 172.21.15.70:40012) 2023-10-08T05:37:42.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f4066ffd640 1 -- 172.21.15.70:0/2356719626 learned_addr learned my addr 172.21.15.70:0/2356719626 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f405e7fc640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815e300 0x7f40680cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908221687 0 0) 0x7f40680b2890 con 0x7f4068159d50 2023-10-08T05:37:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4054003520 con 0x7f4068159d50 2023-10-08T05:37:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1601778165 0 0) 0x7f4054003520 con 0x7f4068159d50 2023-10-08T05:37:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815e300 msgr2=0x7f40680cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f40667fc640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815e300 0x7f40680cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 >> v1:172.21.15.181:6789/0 conn(0x7f40680cba40 legacy=0x7f40680c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406815e760 con 0x7f4068159d50 2023-10-08T05:37:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4058001670 con 0x7f4068159d50 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.930+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 >> v1:172.21.15.70:6789/0 conn(0x7f4068159d50 legacy=0x7f40680b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f40667fc640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815e300 0x7f40540039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f40667fc640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4054003fb0 0x7f40540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f40667fc640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d00 0x7f4054005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40680cbeb0 con 0x7f4054003fb0 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405400bb10 con 0x7f4054004d00 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405400bcd0 con 0x7f406815e300 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f405e7fc640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4054003fb0 0x7f40540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f4066ffd640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815e300 0x7f40540039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f405e7fc640 1 -- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815e300 msgr2=0x7f40540039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f405e7fc640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815e300 0x7f40540039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f405e7fc640 1 -- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d00 msgr2=0x7f4054005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f40677fe640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d00 0x7f4054005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f405e7fc640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d00 0x7f4054005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f405e7fc640 1 -- 172.21.15.70:0/2356719626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4054003520 con 0x7f4054003fb0 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.931+0000 7f40677fe640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4054004d00 0x7f4054005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.932+0000 7f405e7fc640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4054003fb0 0x7f40540047c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f405000bba0 tx=0x7f40540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e61e8ae2290cf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.933+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405002cbc0 con 0x7f4054003fb0 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.933+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4050030070 con 0x7f4054003fb0 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.933+0000 7f40667fc640 1 -- 172.21.15.70:0/2356719626 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050036400 con 0x7f4054003fb0 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.933+0000 7f406d797640 1 -- 172.21.15.70:0/2356719626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4054003fb0 msgr2=0x7f40540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.933+0000 7f406d797640 1 --2- 172.21.15.70:0/2356719626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4054003fb0 0x7f40540047c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f405000bba0 tx=0x7f40540057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.933+0000 7f40677fe640 1 -- 172.21.15.70:0/2356719626 reap_dead start 2023-10-08T05:37:42.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.933+0000 7f406d797640 1 -- 172.21.15.70:0/2356719626 shutdown_connections 2023-10-08T05:37:42.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.933+0000 7f406d797640 1 -- 172.21.15.70:0/2356719626 >> 172.21.15.70:0/2356719626 conn(0x7f406806a140 msgr2=0x7f406806a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:42.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.933+0000 7f406d797640 1 -- 172.21.15.70:0/2356719626 shutdown_connections 2023-10-08T05:37:42.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.934+0000 7f406d797640 1 -- 172.21.15.70:0/2356719626 wait complete. 2023-10-08T05:37:42.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.935+0000 7f406d797640 1 Processor -- start 2023-10-08T05:37:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.935+0000 7f406d797640 1 -- start start 2023-10-08T05:37:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.936+0000 7f406d797640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4054003fb0 0x7f406814eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.936+0000 7f406d797640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40680c4fd0 0x7f40680c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.936+0000 7f406d797640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40680c9f20 0x7f40680ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.936+0000 7f406d797640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40680ca8f0 con 0x7f40680c9f20 2023-10-08T05:37:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.936+0000 7f406d797640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406814e270 con 0x7f40680c4fd0 2023-10-08T05:37:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.936+0000 7f406d797640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406814e510 con 0x7f4054003fb0 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.937+0000 7f40677fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40680c9f20 0x7f40680ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.937+0000 7f40677fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40680c9f20 0x7f40680ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60202/0 (socket says 172.21.15.70:60202) 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.937+0000 7f40677fe640 1 -- 172.21.15.70:0/1703934373 learned_addr learned my addr 172.21.15.70:0/1703934373 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.937+0000 7f405e7fc640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40680c4fd0 0x7f40680c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.937+0000 7f4066ffd640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4054003fb0 0x7f406814eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.937+0000 7f40677fe640 1 -- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4054003fb0 msgr2=0x7f406814eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.937+0000 7f40677fe640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4054003fb0 0x7f406814eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.937+0000 7f40677fe640 1 -- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40680c4fd0 msgr2=0x7f40680c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.937+0000 7f40677fe640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40680c4fd0 0x7f40680c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.937+0000 7f40677fe640 1 -- 172.21.15.70:0/1703934373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406814e7f0 con 0x7f40680c9f20 2023-10-08T05:37:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.938+0000 7f40677fe640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40680c9f20 0x7f40680ca3b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f40680c81c0 tx=0x7f406000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.938+0000 7f405ffff640 1 -- 172.21.15.70:0/1703934373 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060017070 con 0x7f40680c9f20 2023-10-08T05:37:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.938+0000 7f405ffff640 1 -- 172.21.15.70:0/1703934373 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4060007020 con 0x7f40680c9f20 2023-10-08T05:37:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.938+0000 7f405ffff640 1 -- 172.21.15.70:0/1703934373 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40600073d0 con 0x7f40680c9f20 2023-10-08T05:37:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.938+0000 7f406d797640 1 -- 172.21.15.70:0/1703934373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406814ead0 con 0x7f40680c9f20 2023-10-08T05:37:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.938+0000 7f406d797640 1 -- 172.21.15.70:0/1703934373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40680c28c0 con 0x7f40680c9f20 2023-10-08T05:37:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.939+0000 7f406d797640 1 -- 172.21.15.70:0/1703934373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406814ecc0 con 0x7f40680c9f20 2023-10-08T05:37:42.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.940+0000 7f405ffff640 1 -- 172.21.15.70:0/1703934373 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4060007530 con 0x7f40680c9f20 2023-10-08T05:37:42.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.941+0000 7f405ffff640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f404806ae70 0x7f404806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:42.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.941+0000 7f405ffff640 1 -- 172.21.15.70:0/1703934373 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f4060088c30 con 0x7f40680c9f20 2023-10-08T05:37:42.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.942+0000 7f4066ffd640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f404806ae70 0x7f404806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:42.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.943+0000 7f4066ffd640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f404806ae70 0x7f404806d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4058001eb0 tx=0x7f4058001d60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:42.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:42.946+0000 7f405ffff640 1 -- 172.21.15.70:0/1703934373 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4060053da0 con 0x7f40680c9f20 2023-10-08T05:37:43.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.135+0000 7f406d797640 1 -- 172.21.15.70:0/1703934373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4068071cb0 con 0x7f40680c9f20 2023-10-08T05:37:43.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.709+0000 7f405ffff640 1 -- 172.21.15.70:0/1703934373 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f40600573c0 con 0x7f40680c9f20 2023-10-08T05:37:43.711 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T05:37:43.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.714+0000 7f406d797640 1 -- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f404806ae70 msgr2=0x7f404806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:43.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.714+0000 7f406d797640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f404806ae70 0x7f404806d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4058001eb0 tx=0x7f4058001d60 comp rx=0 tx=0).stop 2023-10-08T05:37:43.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.714+0000 7f406d797640 1 -- 172.21.15.70:0/1703934373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40680c9f20 msgr2=0x7f40680ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:43.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.714+0000 7f406d797640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40680c9f20 0x7f40680ca3b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f40680c81c0 tx=0x7f406000d890 comp rx=0 tx=0).stop 2023-10-08T05:37:43.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.715+0000 7f406d797640 1 -- 172.21.15.70:0/1703934373 shutdown_connections 2023-10-08T05:37:43.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.715+0000 7f406d797640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f404806ae70 0x7f404806d330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:43.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.715+0000 7f406d797640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40680c9f20 0x7f40680ca3b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:43.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.715+0000 7f406d797640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40680c4fd0 0x7f40680c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:43.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.715+0000 7f406d797640 1 --2- 172.21.15.70:0/1703934373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4054003fb0 0x7f406814eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:43.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.715+0000 7f406d797640 1 -- 172.21.15.70:0/1703934373 >> 172.21.15.70:0/1703934373 conn(0x7f406806a140 msgr2=0x7f406806a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:43.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.718+0000 7f406d797640 1 -- 172.21.15.70:0/1703934373 shutdown_connections 2023-10-08T05:37:43.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.723+0000 7f406d797640 1 -- 172.21.15.70:0/1703934373 wait complete. 2023-10-08T05:37:43.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T05:37:43.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.943+0000 7fca215c5640 1 Processor -- start 2023-10-08T05:37:43.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.943+0000 7fca215c5640 1 -- start start 2023-10-08T05:37:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.945+0000 7fca215c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c0afbe0 0x7fca1c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.945+0000 7fca215c5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1c0b2620 con 0x7fca1c06dfc0 2023-10-08T05:37:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.945+0000 7fca215c5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca1c0b2060 con 0x7fca1c0cb130 2023-10-08T05:37:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.945+0000 7fca215c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0b2340 con 0x7fca1c0afbe0 2023-10-08T05:37:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.946+0000 7fca13fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fca1c06dfc0 0x7fca1c070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59934/0 (socket says 172.21.15.70:59934) 2023-10-08T05:37:43.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.946+0000 7fca13fff640 1 -- 172.21.15.70:0/2828838188 learned_addr learned my addr 172.21.15.70:0/2828838188 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.946+0000 7fca1affd640 1 --2- 172.21.15.70:0/2828838188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c0afbe0 0x7fca1c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:43.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.946+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575269809 0 0) 0x7fca1c0b2620 con 0x7fca1c06dfc0 2023-10-08T05:37:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.946+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca08003520 con 0x7fca1c06dfc0 2023-10-08T05:37:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1708361209 0 0) 0x7fca08003520 con 0x7fca1c06dfc0 2023-10-08T05:37:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c0afbe0 msgr2=0x7fca1c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:43.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 --2- 172.21.15.70:0/2828838188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c0afbe0 0x7fca1c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 >> v1:172.21.15.181:6789/0 conn(0x7fca1c0cb130 legacy=0x7fca1c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c070350 con 0x7fca1c06dfc0 2023-10-08T05:37:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fca04001670 con 0x7fca1c06dfc0 2023-10-08T05:37:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 >> v1:172.21.15.70:6789/0 conn(0x7fca1c06dfc0 legacy=0x7fca1c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 --2- 172.21.15.70:0/2828838188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 0x7fca08003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:43.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 --2- 172.21.15.70:0/2828838188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca08004350 0x7fca08004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 --2- 172.21.15.70:0/2828838188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c0afbe0 0x7fca08005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0b2340 con 0x7fca080039a0 2023-10-08T05:37:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0800b980 con 0x7fca08004350 2023-10-08T05:37:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0800bb40 con 0x7fca1c0afbe0 2023-10-08T05:37:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1affd640 1 --2- 172.21.15.70:0/2828838188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 0x7fca08003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:43.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1affd640 1 -- 172.21.15.70:0/2828838188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c0afbe0 msgr2=0x7fca08005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:43.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1affd640 1 --2- 172.21.15.70:0/2828838188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c0afbe0 0x7fca08005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:43.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1affd640 1 -- 172.21.15.70:0/2828838188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca08004350 msgr2=0x7fca08004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:43.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1affd640 1 --2- 172.21.15.70:0/2828838188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca08004350 0x7fca08004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:43.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.947+0000 7fca1affd640 1 -- 172.21.15.70:0/2828838188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca08003520 con 0x7fca080039a0 2023-10-08T05:37:43.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.949+0000 7fca1affd640 1 --2- 172.21.15.70:0/2828838188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 0x7fca08003d80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009ac0 tx=0x7fca080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73953bcdc6bf725a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:43.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.949+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c02c710 con 0x7fca080039a0 2023-10-08T05:37:43.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.949+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c02c870 con 0x7fca080039a0 2023-10-08T05:37:43.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.949+0000 7fca1a7fc640 1 -- 172.21.15.70:0/2828838188 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c02cc00 con 0x7fca080039a0 2023-10-08T05:37:43.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.949+0000 7fca215c5640 1 -- 172.21.15.70:0/2828838188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 msgr2=0x7fca08003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:43.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.949+0000 7fca215c5640 1 --2- 172.21.15.70:0/2828838188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 0x7fca08003d80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009ac0 tx=0x7fca080057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:43.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.949+0000 7fca1b7fe640 1 -- 172.21.15.70:0/2828838188 reap_dead start 2023-10-08T05:37:43.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.950+0000 7fca215c5640 1 -- 172.21.15.70:0/2828838188 shutdown_connections 2023-10-08T05:37:43.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.950+0000 7fca215c5640 1 -- 172.21.15.70:0/2828838188 >> 172.21.15.70:0/2828838188 conn(0x7fca1c06a5d0 msgr2=0x7fca1c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:43.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.950+0000 7fca215c5640 1 -- 172.21.15.70:0/2828838188 shutdown_connections 2023-10-08T05:37:43.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.950+0000 7fca215c5640 1 -- 172.21.15.70:0/2828838188 wait complete. 2023-10-08T05:37:43.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.951+0000 7fca215c5640 1 Processor -- start 2023-10-08T05:37:43.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.952+0000 7fca215c5640 1 -- start start 2023-10-08T05:37:43.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.952+0000 7fca215c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 0x7fca1c148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:43.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.952+0000 7fca215c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca1c1495d0 0x7fca1c0c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:43.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.952+0000 7fca215c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c149a50 0x7fca1c0c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:43.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.952+0000 7fca215c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0c4430 con 0x7fca080039a0 2023-10-08T05:37:43.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.952+0000 7fca215c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0c8b60 con 0x7fca1c1495d0 2023-10-08T05:37:43.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.952+0000 7fca215c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c1490c0 con 0x7fca1c149a50 2023-10-08T05:37:43.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.953+0000 7fca1affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 0x7fca1c148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:43.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.953+0000 7fca1affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 0x7fca1c148b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37474/0 (socket says 172.21.15.70:37474) 2023-10-08T05:37:43.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.953+0000 7fca1affd640 1 -- 172.21.15.70:0/2511901816 learned_addr learned my addr 172.21.15.70:0/2511901816 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:43.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.953+0000 7fca13fff640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca1c1495d0 0x7fca1c0c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:43.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.953+0000 7fca1affd640 1 -- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c149a50 msgr2=0x7fca1c0c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:43.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.953+0000 7fca1affd640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c149a50 0x7fca1c0c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:43.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.953+0000 7fca1affd640 1 -- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca1c1495d0 msgr2=0x7fca1c0c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:43.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.953+0000 7fca1affd640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca1c1495d0 0x7fca1c0c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:43.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.953+0000 7fca1affd640 1 -- 172.21.15.70:0/2511901816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c0c3a90 con 0x7fca080039a0 2023-10-08T05:37:43.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.953+0000 7fca1affd640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 0x7fca1c148b80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fca0c02c2b0 tx=0x7fca0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:43.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.954+0000 7fca137fe640 1 -- 172.21.15.70:0/2511901816 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c003810 con 0x7fca080039a0 2023-10-08T05:37:43.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.954+0000 7fca137fe640 1 -- 172.21.15.70:0/2511901816 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0c003970 con 0x7fca080039a0 2023-10-08T05:37:43.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.954+0000 7fca137fe640 1 -- 172.21.15.70:0/2511901816 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0c003d00 con 0x7fca080039a0 2023-10-08T05:37:43.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.954+0000 7fca215c5640 1 -- 172.21.15.70:0/2511901816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c0c3d70 con 0x7fca080039a0 2023-10-08T05:37:43.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.954+0000 7fca215c5640 1 -- 172.21.15.70:0/2511901816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c15c820 con 0x7fca080039a0 2023-10-08T05:37:43.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.955+0000 7fca215c5640 1 -- 172.21.15.70:0/2511901816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca1c04e160 con 0x7fca080039a0 2023-10-08T05:37:43.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.956+0000 7fca137fe640 1 -- 172.21.15.70:0/2511901816 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fca0c040450 con 0x7fca080039a0 2023-10-08T05:37:43.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.957+0000 7fca137fe640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc9fc06aea0 0x7fc9fc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:43.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.957+0000 7fca137fe640 1 -- 172.21.15.70:0/2511901816 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fca0c0ab880 con 0x7fca080039a0 2023-10-08T05:37:43.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.958+0000 7fca13fff640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc9fc06aea0 0x7fc9fc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:43.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.959+0000 7fca13fff640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc9fc06aea0 0x7fc9fc06d360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca040011e0 tx=0x7fca04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:43.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:43.962+0000 7fca137fe640 1 -- 172.21.15.70:0/2511901816 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca0c076960 con 0x7fca080039a0 2023-10-08T05:37:44.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.189+0000 7fca215c5640 1 -- 172.21.15.70:0/2511901816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fca1c0b0ef0 con 0x7fca080039a0 2023-10-08T05:37:44.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.738+0000 7fca137fe640 1 -- 172.21.15.70:0/2511901816 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fca0c003e60 con 0x7fca080039a0 2023-10-08T05:37:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 5 and data pool 6 2023-10-08T05:37:44.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.749+0000 7fca215c5640 1 -- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc9fc06aea0 msgr2=0x7fc9fc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:44.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.750+0000 7fca215c5640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc9fc06aea0 0x7fc9fc06d360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca040011e0 tx=0x7fca04009040 comp rx=0 tx=0).stop 2023-10-08T05:37:44.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.750+0000 7fca215c5640 1 -- 172.21.15.70:0/2511901816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 msgr2=0x7fca1c148b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:44.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.750+0000 7fca215c5640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 0x7fca1c148b80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fca0c02c2b0 tx=0x7fca0c004730 comp rx=0 tx=0).stop 2023-10-08T05:37:44.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.754+0000 7fca215c5640 1 -- 172.21.15.70:0/2511901816 shutdown_connections 2023-10-08T05:37:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.754+0000 7fca215c5640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc9fc06aea0 0x7fc9fc06d360 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.754+0000 7fca215c5640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca1c149a50 0x7fca1c0c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.754+0000 7fca215c5640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca1c1495d0 0x7fca1c0c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.754+0000 7fca215c5640 1 --2- 172.21.15.70:0/2511901816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca080039a0 0x7fca1c148b80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.754+0000 7fca215c5640 1 -- 172.21.15.70:0/2511901816 >> 172.21.15.70:0/2511901816 conn(0x7fca1c06a5d0 msgr2=0x7fca1c0cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.755+0000 7fca215c5640 1 -- 172.21.15.70:0/2511901816 shutdown_connections 2023-10-08T05:37:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.755+0000 7fca215c5640 1 -- 172.21.15.70:0/2511901816 wait complete. 2023-10-08T05:37:44.783 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:44.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.959+0000 7f1f804c2640 1 Processor -- start 2023-10-08T05:37:44.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.959+0000 7f1f804c2640 1 -- start start 2023-10-08T05:37:44.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.961+0000 7f1f804c2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f7815ab40 0x7f1f780ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:44.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.961+0000 7f1f804c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f780cb580 0x7f1f780adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:44.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.961+0000 7f1f804c2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7815b2b0 con 0x7f1f780c90b0 2023-10-08T05:37:44.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.961+0000 7f1f804c2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7815bf30 con 0x7f1f7815ab40 2023-10-08T05:37:44.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.961+0000 7f1f804c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7815f7a0 con 0x7f1f780cb580 2023-10-08T05:37:44.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.962+0000 7f1f7ea5f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f780cb580 0x7f1f780adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:44.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.962+0000 7f1f7e25e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f7815ab40 0x7f1f780ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:44.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.962+0000 7f1f7ea5f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f780cb580 0x7f1f780adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53268/0 (socket says 172.21.15.70:53268) 2023-10-08T05:37:44.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.962+0000 7f1f7ea5f640 1 -- 172.21.15.70:0/2038139492 learned_addr learned my addr 172.21.15.70:0/2038139492 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:44.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.962+0000 7f1f7e25e640 1 -- 172.21.15.70:0/2038139492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f780cb580 msgr2=0x7f1f780adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:44.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.962+0000 7f1f7e25e640 1 --2- 172.21.15.70:0/2038139492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f780cb580 0x7f1f780adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:44.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.962+0000 7f1f7e25e640 1 -- 172.21.15.70:0/2038139492 >> v1:172.21.15.181:6789/0 conn(0x7f1f780c90b0 legacy=0x7f1f780cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:44.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.963+0000 7f1f7e25e640 1 -- 172.21.15.70:0/2038139492 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7815b590 con 0x7f1f7815ab40 2023-10-08T05:37:44.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.964+0000 7f1f7e25e640 1 --2- 172.21.15.70:0/2038139492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f7815ab40 0x7f1f780ad720 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f740097d0 tx=0x7f1f7402b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2063b7b063fd5f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:44.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.964+0000 7f1f7d25c640 1 -- 172.21.15.70:0/2038139492 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74031800 con 0x7f1f7815ab40 2023-10-08T05:37:44.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.964+0000 7f1f7d25c640 1 -- 172.21.15.70:0/2038139492 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f74031960 con 0x7f1f7815ab40 2023-10-08T05:37:44.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.964+0000 7f1f7d25c640 1 -- 172.21.15.70:0/2038139492 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74031cf0 con 0x7f1f7815ab40 2023-10-08T05:37:44.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.965+0000 7f1f804c2640 1 -- 172.21.15.70:0/2038139492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f7815ab40 msgr2=0x7f1f780ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:44.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.965+0000 7f1f804c2640 1 --2- 172.21.15.70:0/2038139492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f7815ab40 0x7f1f780ad720 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f740097d0 tx=0x7f1f7402b8a0 comp rx=0 tx=0).stop 2023-10-08T05:37:44.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.965+0000 7f1f804c2640 1 -- 172.21.15.70:0/2038139492 shutdown_connections 2023-10-08T05:37:44.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.965+0000 7f1f804c2640 1 --2- 172.21.15.70:0/2038139492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f780cb580 0x7f1f780adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:44.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.965+0000 7f1f804c2640 1 --2- 172.21.15.70:0/2038139492 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f7815ab40 0x7f1f780ad720 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:44.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.965+0000 7f1f804c2640 1 -- 172.21.15.70:0/2038139492 >> 172.21.15.70:0/2038139492 conn(0x7f1f7806a380 msgr2=0x7f1f780b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:44.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.965+0000 7f1f804c2640 1 -- 172.21.15.70:0/2038139492 shutdown_connections 2023-10-08T05:37:44.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.965+0000 7f1f804c2640 1 -- 172.21.15.70:0/2038139492 wait complete. 2023-10-08T05:37:44.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.966+0000 7f1f804c2640 1 Processor -- start 2023-10-08T05:37:44.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.967+0000 7f1f804c2640 1 -- start start 2023-10-08T05:37:44.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.967+0000 7f1f804c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f780c90b0 0x7f1f780c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:44.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.967+0000 7f1f804c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f780cb580 0x7f1f780c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:44.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f804c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f7815ab40 0x7f1f78141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:44.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f804c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780c83f0 con 0x7f1f780c90b0 2023-10-08T05:37:44.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f804c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780b3ed0 con 0x7f1f780cb580 2023-10-08T05:37:44.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f804c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f78156460 con 0x7f1f7815ab40 2023-10-08T05:37:44.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f7e25e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f780c90b0 0x7f1f780c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:44.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f7e25e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f780c90b0 0x7f1f780c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37478/0 (socket says 172.21.15.70:37478) 2023-10-08T05:37:44.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f7e25e640 1 -- 172.21.15.70:0/1665322084 learned_addr learned my addr 172.21.15.70:0/1665322084 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:44.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f7ea5f640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f7815ab40 0x7f1f78141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:44.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f7e25e640 1 -- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f7815ab40 msgr2=0x7f1f78141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:44.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f7e25e640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f7815ab40 0x7f1f78141f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:44.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f7da5d640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f780cb580 0x7f1f780c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:44.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f7e25e640 1 -- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f780cb580 msgr2=0x7f1f780c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:44.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f7e25e640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f780cb580 0x7f1f780c4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:44.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.968+0000 7f1f7e25e640 1 -- 172.21.15.70:0/1665322084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74009400 con 0x7f1f780c90b0 2023-10-08T05:37:44.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.969+0000 7f1f7e25e640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f780c90b0 0x7f1f780c7eb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1f74000c00 tx=0x7f1f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:44.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.969+0000 7f1f6effd640 1 -- 172.21.15.70:0/1665322084 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7403bc70 con 0x7f1f780c90b0 2023-10-08T05:37:44.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.969+0000 7f1f6effd640 1 -- 172.21.15.70:0/1665322084 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7403be10 con 0x7f1f780c90b0 2023-10-08T05:37:44.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.969+0000 7f1f804c2640 1 -- 172.21.15.70:0/1665322084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f780b35e0 con 0x7f1f780c90b0 2023-10-08T05:37:44.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.969+0000 7f1f6effd640 1 -- 172.21.15.70:0/1665322084 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74045630 con 0x7f1f780c90b0 2023-10-08T05:37:44.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.969+0000 7f1f804c2640 1 -- 172.21.15.70:0/1665322084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f780b3880 con 0x7f1f780c90b0 2023-10-08T05:37:44.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.970+0000 7f1f804c2640 1 -- 172.21.15.70:0/1665322084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7815b850 con 0x7f1f780c90b0 2023-10-08T05:37:44.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.971+0000 7f1f6effd640 1 -- 172.21.15.70:0/1665322084 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1f7404e450 con 0x7f1f780c90b0 2023-10-08T05:37:44.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.972+0000 7f1f6effd640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f5c06ae70 0x7f1f5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:44.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.972+0000 7f1f6effd640 1 -- 172.21.15.70:0/1665322084 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1f7402e070 con 0x7f1f780c90b0 2023-10-08T05:37:44.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.973+0000 7f1f7da5d640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f5c06ae70 0x7f1f5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:44.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.974+0000 7f1f7da5d640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f5c06ae70 0x7f1f5c06d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f7815e250 tx=0x7f1f68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:44.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:44.977+0000 7f1f6effd640 1 -- 172.21.15.70:0/1665322084 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f7815b850 con 0x7f1f780c90b0 2023-10-08T05:37:45.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.152+0000 7f1f804c2640 1 -- 172.21.15.70:0/1665322084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f7815b850 con 0x7f1f780c90b0 2023-10-08T05:37:45.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.155+0000 7f1f6effd640 1 -- 172.21.15.70:0/1665322084 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16775 (secure 0 0 0) 0x7f1f7815b850 con 0x7f1f780c90b0 2023-10-08T05:37:45.157 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:45.157 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":38,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:37:44.191676+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:37:41.960853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-08T05:37:43.190325+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:45.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 -- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f5c06ae70 msgr2=0x7f1f5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:45.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f5c06ae70 0x7f1f5c06d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f7815e250 tx=0x7f1f68009040 comp rx=0 tx=0).stop 2023-10-08T05:37:45.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 -- 172.21.15.70:0/1665322084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f780c90b0 msgr2=0x7f1f780c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:45.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f780c90b0 0x7f1f780c7eb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1f74000c00 tx=0x7f1f740046c0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 -- 172.21.15.70:0/1665322084 shutdown_connections 2023-10-08T05:37:45.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1f5c06ae70 0x7f1f5c06d330 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f7815ab40 0x7f1f78141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f780cb580 0x7f1f780c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 --2- 172.21.15.70:0/1665322084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f780c90b0 0x7f1f780c7eb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 -- 172.21.15.70:0/1665322084 >> 172.21.15.70:0/1665322084 conn(0x7f1f7806a380 msgr2=0x7f1f780c9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:45.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 -- 172.21.15.70:0/1665322084 shutdown_connections 2023-10-08T05:37:45.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.158+0000 7f1f804c2640 1 -- 172.21.15.70:0/1665322084 wait complete. 2023-10-08T05:37:45.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:45.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.383+0000 7fac51f4e640 1 Processor -- start 2023-10-08T05:37:45.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.383+0000 7fac51f4e640 1 -- start start 2023-10-08T05:37:45.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.384+0000 7fac51f4e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac4c1599d0 0x7fac4c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.384+0000 7fac51f4e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 0x7fac4c06c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.384+0000 7fac51f4e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c0cacf0 con 0x7fac4c0cc210 2023-10-08T05:37:45.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac51f4e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c0b10f0 con 0x7fac4c1599d0 2023-10-08T05:37:45.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac51f4e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0ca3c0 con 0x7fac4c0b07d0 2023-10-08T05:37:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac50f4c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac4c1599d0 0x7fac4c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac50f4c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac4c1599d0 0x7fac4c0ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37494/0 (socket says 172.21.15.70:37494) 2023-10-08T05:37:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac50f4c640 1 -- 172.21.15.70:0/3269321496 learned_addr learned my addr 172.21.15.70:0/3269321496 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac5174d640 1 --2- 172.21.15.70:0/3269321496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 0x7fac4c06c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac50f4c640 1 -- 172.21.15.70:0/3269321496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 msgr2=0x7fac4c06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac50f4c640 1 --2- 172.21.15.70:0/3269321496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 0x7fac4c06c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac50f4c640 1 -- 172.21.15.70:0/3269321496 >> v1:172.21.15.181:6789/0 conn(0x7fac4c0cc210 legacy=0x7fac4c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac50f4c640 1 -- 172.21.15.70:0/3269321496 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0ca6a0 con 0x7fac4c1599d0 2023-10-08T05:37:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.385+0000 7fac5174d640 1 --2- 172.21.15.70:0/3269321496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 0x7fac4c06c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.386+0000 7fac50f4c640 1 --2- 172.21.15.70:0/3269321496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c1599d0 0x7fac4c0ad730 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fac40009870 tx=0x7fac4002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5335fdcbae7bdb46 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:45.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.387+0000 7fac4b7fe640 1 -- 172.21.15.70:0/3269321496 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40031640 con 0x7fac4c1599d0 2023-10-08T05:37:45.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.387+0000 7fac4b7fe640 1 -- 172.21.15.70:0/3269321496 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac400317a0 con 0x7fac4c1599d0 2023-10-08T05:37:45.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.387+0000 7fac4b7fe640 1 -- 172.21.15.70:0/3269321496 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac40031b30 con 0x7fac4c1599d0 2023-10-08T05:37:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.387+0000 7fac51f4e640 1 -- 172.21.15.70:0/3269321496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c1599d0 msgr2=0x7fac4c0ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.387+0000 7fac51f4e640 1 --2- 172.21.15.70:0/3269321496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c1599d0 0x7fac4c0ad730 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fac40009870 tx=0x7fac4002b8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.388+0000 7fac51f4e640 1 -- 172.21.15.70:0/3269321496 shutdown_connections 2023-10-08T05:37:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.388+0000 7fac51f4e640 1 --2- 172.21.15.70:0/3269321496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 0x7fac4c06c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.388+0000 7fac51f4e640 1 --2- 172.21.15.70:0/3269321496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c1599d0 0x7fac4c0ad730 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.388+0000 7fac51f4e640 1 -- 172.21.15.70:0/3269321496 >> 172.21.15.70:0/3269321496 conn(0x7fac4c0cb0c0 msgr2=0x7fac4c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.388+0000 7fac51f4e640 1 -- 172.21.15.70:0/3269321496 shutdown_connections 2023-10-08T05:37:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.388+0000 7fac51f4e640 1 -- 172.21.15.70:0/3269321496 wait complete. 2023-10-08T05:37:45.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.390+0000 7fac51f4e640 1 Processor -- start 2023-10-08T05:37:45.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.390+0000 7fac51f4e640 1 -- start start 2023-10-08T05:37:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.391+0000 7fac51f4e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 0x7fac4c1f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.391+0000 7fac51f4e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c0cc210 0x7fac4c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.391+0000 7fac51f4e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac4c1599d0 0x7fac4c15c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.391+0000 7fac51f4e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c1f6620 con 0x7fac4c0cc210 2023-10-08T05:37:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.391+0000 7fac51f4e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c1f5690 con 0x7fac4c1599d0 2023-10-08T05:37:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.391+0000 7fac51f4e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c15d060 con 0x7fac4c0b07d0 2023-10-08T05:37:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c0cc210 0x7fac4c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c0cc210 0x7fac4c158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37504/0 (socket says 172.21.15.70:37504) 2023-10-08T05:37:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac4bfff640 1 -- 172.21.15.70:0/1692592534 learned_addr learned my addr 172.21.15.70:0/1692592534 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac50f4c640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 0x7fac4c1f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac5174d640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac4c1599d0 0x7fac4c15c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac4bfff640 1 -- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 msgr2=0x7fac4c1f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac4bfff640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 0x7fac4c1f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac4bfff640 1 -- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac4c1599d0 msgr2=0x7fac4c15c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:45.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac4bfff640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac4c1599d0 0x7fac4c15c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac4bfff640 1 -- 172.21.15.70:0/1692592534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac400094d0 con 0x7fac4c0cc210 2023-10-08T05:37:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.392+0000 7fac4bfff640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c0cc210 0x7fac4c158150 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fac3800c430 tx=0x7fac3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.393+0000 7fac497fa640 1 -- 172.21.15.70:0/1692592534 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38011790 con 0x7fac4c0cc210 2023-10-08T05:37:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.393+0000 7fac497fa640 1 -- 172.21.15.70:0/1692592534 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac380118f0 con 0x7fac4c0cc210 2023-10-08T05:37:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.393+0000 7fac51f4e640 1 -- 172.21.15.70:0/1692592534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c15d340 con 0x7fac4c0cc210 2023-10-08T05:37:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.393+0000 7fac497fa640 1 -- 172.21.15.70:0/1692592534 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38011ce0 con 0x7fac4c0cc210 2023-10-08T05:37:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.393+0000 7fac51f4e640 1 -- 172.21.15.70:0/1692592534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c1f5bc0 con 0x7fac4c0cc210 2023-10-08T05:37:45.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.394+0000 7fac51f4e640 1 -- 172.21.15.70:0/1692592534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c04e160 con 0x7fac4c0cc210 2023-10-08T05:37:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.395+0000 7fac497fa640 1 -- 172.21.15.70:0/1692592534 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fac3801c460 con 0x7fac4c0cc210 2023-10-08T05:37:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.396+0000 7fac497fa640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac3006ae70 0x7fac3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.396+0000 7fac497fa640 1 -- 172.21.15.70:0/1692592534 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fac38089680 con 0x7fac4c0cc210 2023-10-08T05:37:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.396+0000 7fac50f4c640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac3006ae70 0x7fac3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:45.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.398+0000 7fac50f4c640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac3006ae70 0x7fac3006d330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fac40036000 tx=0x7fac4003c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:45.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.400+0000 7fac497fa640 1 -- 172.21.15.70:0/1692592534 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac38054680 con 0x7fac4c0cc210 2023-10-08T05:37:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.589+0000 7fac51f4e640 1 -- 172.21.15.70:0/1692592534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac4c1f5d80 con 0x7fac4c0cc210 2023-10-08T05:37:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.592+0000 7fac497fa640 1 -- 172.21.15.70:0/1692592534 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16775 (secure 0 0 0) 0x7fac38057ca0 con 0x7fac4c0cc210 2023-10-08T05:37:45.594 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:45.594 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":38,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:37:44.191676+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:37:41.960853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-08T05:37:43.190325+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.594+0000 7fac51f4e640 1 -- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac3006ae70 msgr2=0x7fac3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.594+0000 7fac51f4e640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac3006ae70 0x7fac3006d330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fac40036000 tx=0x7fac4003c040 comp rx=0 tx=0).stop 2023-10-08T05:37:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.594+0000 7fac51f4e640 1 -- 172.21.15.70:0/1692592534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c0cc210 msgr2=0x7fac4c158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.594+0000 7fac51f4e640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c0cc210 0x7fac4c158150 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fac3800c430 tx=0x7fac3800c8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.595+0000 7fac51f4e640 1 -- 172.21.15.70:0/1692592534 shutdown_connections 2023-10-08T05:37:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.595+0000 7fac51f4e640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fac3006ae70 0x7fac3006d330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.595+0000 7fac51f4e640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac4c1599d0 0x7fac4c15c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.595+0000 7fac51f4e640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c0cc210 0x7fac4c158150 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.595+0000 7fac51f4e640 1 --2- 172.21.15.70:0/1692592534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac4c0b07d0 0x7fac4c1f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.595+0000 7fac51f4e640 1 -- 172.21.15.70:0/1692592534 >> 172.21.15.70:0/1692592534 conn(0x7fac4c0cb0c0 msgr2=0x7fac4c06c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.595+0000 7fac51f4e640 1 -- 172.21.15.70:0/1692592534 shutdown_connections 2023-10-08T05:37:45.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.595+0000 7fac51f4e640 1 -- 172.21.15.70:0/1692592534 wait complete. 2023-10-08T05:37:45.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T05:37:45.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.831+0000 7f78796c2640 1 Processor -- start 2023-10-08T05:37:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.831+0000 7f78796c2640 1 -- start start 2023-10-08T05:37:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78796c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f787406dfc0 0x7f7874070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78796c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7874071070 0x7f78740cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78796c2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78740b1cc0 con 0x7f78740afbe0 2023-10-08T05:37:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78796c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78740b2620 con 0x7f7874071070 2023-10-08T05:37:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78796c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740b2060 con 0x7f787406dfc0 2023-10-08T05:37:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78737fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7874071070 0x7f78740cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78737fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7874071070 0x7f78740cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37508/0 (socket says 172.21.15.70:37508) 2023-10-08T05:37:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78737fe640 1 -- 172.21.15.70:0/3486648983 learned_addr learned my addr 172.21.15.70:0/3486648983 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78737fe640 1 -- 172.21.15.70:0/3486648983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f787406dfc0 msgr2=0x7f7874070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78737fe640 1 --2- 172.21.15.70:0/3486648983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f787406dfc0 0x7f7874070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.833+0000 7f78737fe640 1 -- 172.21.15.70:0/3486648983 >> v1:172.21.15.181:6789/0 conn(0x7f78740afbe0 legacy=0x7f78740b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.834+0000 7f78737fe640 1 -- 172.21.15.70:0/3486648983 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78740b2340 con 0x7f7874071070 2023-10-08T05:37:45.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.835+0000 7f78737fe640 1 --2- 172.21.15.70:0/3486648983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7874071070 0x7f78740cb130 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f786c009870 tx=0x7f786c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee077c22a2d422d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.835+0000 7f7871ffb640 1 -- 172.21.15.70:0/3486648983 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c031640 con 0x7f7874071070 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.835+0000 7f7871ffb640 1 -- 172.21.15.70:0/3486648983 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c0317a0 con 0x7f7874071070 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.835+0000 7f7871ffb640 1 -- 172.21.15.70:0/3486648983 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c031b30 con 0x7f7874071070 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.835+0000 7f78796c2640 1 -- 172.21.15.70:0/3486648983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7874071070 msgr2=0x7f78740cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.835+0000 7f78796c2640 1 --2- 172.21.15.70:0/3486648983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7874071070 0x7f78740cb130 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f786c009870 tx=0x7f786c02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.836+0000 7f78796c2640 1 -- 172.21.15.70:0/3486648983 shutdown_connections 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.836+0000 7f78796c2640 1 --2- 172.21.15.70:0/3486648983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7874071070 0x7f78740cb130 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.836+0000 7f78796c2640 1 --2- 172.21.15.70:0/3486648983 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f787406dfc0 0x7f7874070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.836+0000 7f78796c2640 1 -- 172.21.15.70:0/3486648983 >> 172.21.15.70:0/3486648983 conn(0x7f787406a5d0 msgr2=0x7f78740b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.836+0000 7f78796c2640 1 -- 172.21.15.70:0/3486648983 shutdown_connections 2023-10-08T05:37:45.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.836+0000 7f78796c2640 1 -- 172.21.15.70:0/3486648983 wait complete. 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.837+0000 7f78796c2640 1 Processor -- start 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.838+0000 7f78796c2640 1 -- start start 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.838+0000 7f78796c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f787406dfc0 0x7f78740c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.838+0000 7f78796c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7874071070 0x7f78741522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f78796c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78740afbe0 0x7f78741527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f78796c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787415d820 con 0x7f787406dfc0 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f78796c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787415c710 con 0x7f7874071070 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f78796c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787415b950 con 0x7f78740afbe0 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f7872ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f787406dfc0 0x7f78740c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f7872ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f787406dfc0 0x7f78740c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37524/0 (socket says 172.21.15.70:37524) 2023-10-08T05:37:45.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f7872ffd640 1 -- 172.21.15.70:0/4240385921 learned_addr learned my addr 172.21.15.70:0/4240385921 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f7872ffd640 1 -- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78740afbe0 msgr2=0x7f78741527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f7872ffd640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78740afbe0 0x7f78741527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f7872ffd640 1 -- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7874071070 msgr2=0x7f78741522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f7872ffd640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7874071070 0x7f78741522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.839+0000 7f7872ffd640 1 -- 172.21.15.70:0/4240385921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0094d0 con 0x7f787406dfc0 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.840+0000 7f7872ffd640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f787406dfc0 0x7f78740c9d30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f786400c7a0 tx=0x7f786400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.840+0000 7f785b7fe640 1 -- 172.21.15.70:0/4240385921 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864003d80 con 0x7f787406dfc0 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.840+0000 7f785b7fe640 1 -- 172.21.15.70:0/4240385921 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7864005070 con 0x7f787406dfc0 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.840+0000 7f785b7fe640 1 -- 172.21.15.70:0/4240385921 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864005460 con 0x7f787406dfc0 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.840+0000 7f78796c2640 1 -- 172.21.15.70:0/4240385921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787415bc30 con 0x7f787406dfc0 2023-10-08T05:37:45.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.840+0000 7f78796c2640 1 -- 172.21.15.70:0/4240385921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787415c1f0 con 0x7f787406dfc0 2023-10-08T05:37:45.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.841+0000 7f78796c2640 1 -- 172.21.15.70:0/4240385921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787415c410 con 0x7f787406dfc0 2023-10-08T05:37:45.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.842+0000 7f785b7fe640 1 -- 172.21.15.70:0/4240385921 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f78640055c0 con 0x7f787406dfc0 2023-10-08T05:37:45.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.843+0000 7f785b7fe640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f786006aea0 0x7f786006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:45.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.843+0000 7f785b7fe640 1 -- 172.21.15.70:0/4240385921 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7864088c80 con 0x7f787406dfc0 2023-10-08T05:37:45.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.844+0000 7f78727fc640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f786006aea0 0x7f786006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:45.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.845+0000 7f78727fc640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f786006aea0 0x7f786006d360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f78740c5470 tx=0x7f7868006bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:45.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:45.847+0000 7f785b7fe640 1 -- 172.21.15.70:0/4240385921 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7864053b60 con 0x7f787406dfc0 2023-10-08T05:37:46.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.076+0000 7f78796c2640 1 -- 172.21.15.70:0/4240385921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f787404e160 con 0x7f787406dfc0 2023-10-08T05:37:46.615 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-10-08T05:37:46.614+0000 7f6519b03640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:37:46.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.744+0000 7f785b7fe640 1 -- 172.21.15.70:0/4240385921 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7864057180 con 0x7f787406dfc0 2023-10-08T05:37:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.747+0000 7f78796c2640 1 -- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f786006aea0 msgr2=0x7f786006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.747+0000 7f78796c2640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f786006aea0 0x7f786006d360 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f78740c5470 tx=0x7f7868006bf0 comp rx=0 tx=0).stop 2023-10-08T05:37:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.748+0000 7f78796c2640 1 -- 172.21.15.70:0/4240385921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f787406dfc0 msgr2=0x7f78740c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.748+0000 7f78796c2640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f787406dfc0 0x7f78740c9d30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f786400c7a0 tx=0x7f786400cc60 comp rx=0 tx=0).stop 2023-10-08T05:37:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.748+0000 7f78796c2640 1 -- 172.21.15.70:0/4240385921 shutdown_connections 2023-10-08T05:37:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.748+0000 7f78796c2640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f786006aea0 0x7f786006d360 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.748+0000 7f78796c2640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78740afbe0 0x7f78741527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.748+0000 7f78796c2640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7874071070 0x7f78741522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.748+0000 7f78796c2640 1 --2- 172.21.15.70:0/4240385921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f787406dfc0 0x7f78740c9d30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.748+0000 7f78796c2640 1 -- 172.21.15.70:0/4240385921 >> 172.21.15.70:0/4240385921 conn(0x7f787406a5d0 msgr2=0x7f787415df30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.748+0000 7f78796c2640 1 -- 172.21.15.70:0/4240385921 shutdown_connections 2023-10-08T05:37:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.748+0000 7f78796c2640 1 -- 172.21.15.70:0/4240385921 wait complete. 2023-10-08T05:37:46.768 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T05:37:46.769 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T05:37:46.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.989+0000 7f2f5a1d0640 1 Processor -- start 2023-10-08T05:37:46.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.989+0000 7f2f5a1d0640 1 -- start start 2023-10-08T05:37:46.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.991+0000 7f2f5a1d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f540cd250 0x7f2f5415d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:46.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.991+0000 7f2f5a1d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5406b6a0 con 0x7f2f540cea30 2023-10-08T05:37:46.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.991+0000 7f2f5a1d0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f540b19e0 con 0x7f2f5415cdb0 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.991+0000 7f2f5a1d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f54069ef0 con 0x7f2f540cd250 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.991+0000 7f2f52ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2f540cea30 0x7f2f540ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59946/0 (socket says 172.21.15.70:59946) 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.991+0000 7f2f52ffd640 1 -- 172.21.15.70:0/2149722842 learned_addr learned my addr 172.21.15.70:0/2149722842 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.991+0000 7f2f53fff640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f540cd250 0x7f2f5415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.991+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940634716 0 0) 0x7f2f5406b6a0 con 0x7f2f540cea30 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.991+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f40003520 con 0x7f2f540cea30 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1359663545 0 0) 0x7f2f40003520 con 0x7f2f540cea30 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f540cd250 msgr2=0x7f2f5415d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f540cd250 0x7f2f5415d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 >> v1:172.21.15.181:6789/0 conn(0x7f2f5415cdb0 legacy=0x7f2f5406af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:46.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5406a1d0 con 0x7f2f540cea30 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2f3c0019f0 con 0x7f2f540cea30 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 >> v1:172.21.15.70:6789/0 conn(0x7f2f540cea30 legacy=0x7f2f540ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f540cd250 0x7f2f400039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f40003f70 0x7f2f400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f40004d00 0x7f2f40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f54069ef0 con 0x7f2f40003f70 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4000bac0 con 0x7f2f40004d00 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4000bc80 con 0x7f2f540cd250 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.992+0000 7f2f52ffd640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f40003f70 0x7f2f400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.993+0000 7f2f52ffd640 1 -- 172.21.15.70:0/2149722842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f540cd250 msgr2=0x7f2f400039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:46.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.993+0000 7f2f53fff640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f40004d00 0x7f2f40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.993+0000 7f2f52ffd640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f540cd250 0x7f2f400039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.993+0000 7f2f52ffd640 1 -- 172.21.15.70:0/2149722842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f40004d00 msgr2=0x7f2f40005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.993+0000 7f2f52ffd640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f40004d00 0x7f2f40005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.993+0000 7f2f52ffd640 1 -- 172.21.15.70:0/2149722842 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40003520 con 0x7f2f40003f70 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.994+0000 7f2f52ffd640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f40003f70 0x7f2f400047c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00a760 tx=0x7f2f400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8898125d4d5cfbf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.994+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c034630 con 0x7f2f40003f70 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.994+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c02f070 con 0x7f2f40003f70 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.994+0000 7f2f527fc640 1 -- 172.21.15.70:0/2149722842 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c034a50 con 0x7f2f40003f70 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.995+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/2149722842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f40003f70 msgr2=0x7f2f400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.995+0000 7f2f5a1d0640 1 --2- 172.21.15.70:0/2149722842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f40003f70 0x7f2f400047c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00a760 tx=0x7f2f400057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:46.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.995+0000 7f2f53fff640 1 -- 172.21.15.70:0/2149722842 reap_dead start 2023-10-08T05:37:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.995+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/2149722842 shutdown_connections 2023-10-08T05:37:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.995+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/2149722842 >> 172.21.15.70:0/2149722842 conn(0x7f2f540b0af0 msgr2=0x7f2f540b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.995+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/2149722842 shutdown_connections 2023-10-08T05:37:47.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.995+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/2149722842 wait complete. 2023-10-08T05:37:47.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.997+0000 7f2f5a1d0640 1 Processor -- start 2023-10-08T05:37:47.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.997+0000 7f2f5a1d0640 1 -- start start 2023-10-08T05:37:47.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f5a1d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f40003f70 0x7f2f541517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:47.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f5a1d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f5415ae60 0x7f2f540acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:47.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f5a1d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f5415b2e0 0x7f2f540ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:47.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f5a1d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5415a150 con 0x7f2f5415b2e0 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f5a1d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5415a9d0 con 0x7f2f5415ae60 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f5a1d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5415ff90 con 0x7f2f40003f70 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f5415b2e0 0x7f2f540ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f5415b2e0 0x7f2f540ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37544/0 (socket says 172.21.15.70:37544) 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f53fff640 1 -- 172.21.15.70:0/270154677 learned_addr learned my addr 172.21.15.70:0/270154677 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f53fff640 1 -- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f40003f70 msgr2=0x7f2f541517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f537fe640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f40003f70 0x7f2f541517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f52ffd640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f5415ae60 0x7f2f540acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.998+0000 7f2f53fff640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f40003f70 0x7f2f541517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.999+0000 7f2f53fff640 1 -- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f5415ae60 msgr2=0x7f2f540acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:47.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.999+0000 7f2f53fff640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f5415ae60 0x7f2f540acf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:47.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.999+0000 7f2f53fff640 1 -- 172.21.15.70:0/270154677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5415fcb0 con 0x7f2f5415b2e0 2023-10-08T05:37:47.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.999+0000 7f2f53fff640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f5415b2e0 0x7f2f540ad450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2f54066590 tx=0x7f2f4c0109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:47.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.999+0000 7f2f4bfff640 1 -- 172.21.15.70:0/270154677 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0148c0 con 0x7f2f5415b2e0 2023-10-08T05:37:47.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.999+0000 7f2f4bfff640 1 -- 172.21.15.70:0/270154677 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4c014a60 con 0x7f2f5415b2e0 2023-10-08T05:37:47.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.999+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/270154677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5415a380 con 0x7f2f5415b2e0 2023-10-08T05:37:47.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.999+0000 7f2f4bfff640 1 -- 172.21.15.70:0/270154677 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c014e10 con 0x7f2f5415b2e0 2023-10-08T05:37:47.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:46.999+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/270154677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f540b1c00 con 0x7f2f5415b2e0 2023-10-08T05:37:47.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.000+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/270154677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5415ab40 con 0x7f2f5415b2e0 2023-10-08T05:37:47.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.001+0000 7f2f4bfff640 1 -- 172.21.15.70:0/270154677 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f2f4c012390 con 0x7f2f5415b2e0 2023-10-08T05:37:47.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.002+0000 7f2f4bfff640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2f3406ae70 0x7f2f3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:47.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.002+0000 7f2f4bfff640 1 -- 172.21.15.70:0/270154677 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2f4c088f40 con 0x7f2f5415b2e0 2023-10-08T05:37:47.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.003+0000 7f2f537fe640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2f3406ae70 0x7f2f3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:47.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.004+0000 7f2f537fe640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2f3406ae70 0x7f2f3406d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f440015d0 tx=0x7f2f44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:47.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.007+0000 7f2f4bfff640 1 -- 172.21.15.70:0/270154677 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f4c053ea0 con 0x7f2f5415b2e0 2023-10-08T05:37:47.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.236+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/270154677 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2f5404e160 con 0x7f2f5415b2e0 2023-10-08T05:37:47.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.975+0000 7f2f4bfff640 1 -- 172.21.15.70:0/270154677 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2f4c000e60 con 0x7f2f5415b2e0 2023-10-08T05:37:47.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.978+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2f3406ae70 msgr2=0x7f2f3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:47.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.978+0000 7f2f5a1d0640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2f3406ae70 0x7f2f3406d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f440015d0 tx=0x7f2f44009040 comp rx=0 tx=0).stop 2023-10-08T05:37:47.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.978+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/270154677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f5415b2e0 msgr2=0x7f2f540ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:47.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.978+0000 7f2f5a1d0640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f5415b2e0 0x7f2f540ad450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2f54066590 tx=0x7f2f4c0109f0 comp rx=0 tx=0).stop 2023-10-08T05:37:47.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.979+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/270154677 shutdown_connections 2023-10-08T05:37:47.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.979+0000 7f2f5a1d0640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f2f3406ae70 0x7f2f3406d330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:47.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.979+0000 7f2f5a1d0640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f5415b2e0 0x7f2f540ad450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:47.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.979+0000 7f2f5a1d0640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f5415ae60 0x7f2f540acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:47.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.979+0000 7f2f5a1d0640 1 --2- 172.21.15.70:0/270154677 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f40003f70 0x7f2f541517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:47.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.979+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/270154677 >> 172.21.15.70:0/270154677 conn(0x7f2f540b0af0 msgr2=0x7f2f54157850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:47.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.979+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/270154677 shutdown_connections 2023-10-08T05:37:47.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:47.979+0000 7f2f5a1d0640 1 -- 172.21.15.70:0/270154677 wait complete. 2023-10-08T05:37:47.999 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T05:37:48.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:37:48.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.214+0000 7f3cb41b6640 1 Processor -- start 2023-10-08T05:37:48.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.215+0000 7f3cb41b6640 1 -- start start 2023-10-08T05:37:48.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.216+0000 7f3cb41b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 0x7f3cac0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.216+0000 7f3cb41b6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3cac0cb3b0 0x7f3cac15e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.216+0000 7f3cb41b6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cac15b2b0 con 0x7f3cac15ab40 2023-10-08T05:37:48.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.216+0000 7f3cb41b6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac15bf30 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.216+0000 7f3cb41b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0cb840 con 0x7f3cac0c90b0 2023-10-08T05:37:48.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb1f52640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3cac15ab40 0x7f3cac0ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59948/0 (socket says 172.21.15.70:59948) 2023-10-08T05:37:48.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb1f52640 1 -- 172.21.15.70:0/3287129527 learned_addr learned my addr 172.21.15.70:0/3287129527 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:48.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb2753640 1 --2- 172.21.15.70:0/3287129527 >> v2:172.21.15.70:3300/0 conn(0x7f3cac0cb3b0 0x7f3cac15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:48.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb0f50640 1 -- 172.21.15.70:0/3287129527 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921854170 0 0) 0x7f3cac15b2b0 con 0x7f3cac15ab40 2023-10-08T05:37:48.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb0f50640 1 -- 172.21.15.70:0/3287129527 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c98003520 con 0x7f3cac15ab40 2023-10-08T05:37:48.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb1751640 1 --2- 172.21.15.70:0/3287129527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 0x7f3cac0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:48.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb2753640 1 -- 172.21.15.70:0/3287129527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 msgr2=0x7f3cac0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:48.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb2753640 1 --2- 172.21.15.70:0/3287129527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 0x7f3cac0cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb2753640 1 -- 172.21.15.70:0/3287129527 >> v1:172.21.15.70:6789/0 conn(0x7f3cac15ab40 legacy=0x7f3cac0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb2753640 1 -- 172.21.15.70:0/3287129527 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac15b590 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.217+0000 7f3cb1751640 1 --2- 172.21.15.70:0/3287129527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 0x7f3cac0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:48.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.218+0000 7f3cb2753640 1 --2- 172.21.15.70:0/3287129527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 0x7f3cac15e790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009d90 tx=0x7f3ca402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10334d646f6f1ed2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:48.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.219+0000 7f3cb0f50640 1 -- 172.21.15.70:0/3287129527 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca402cc50 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.219+0000 7f3cb0f50640 1 -- 172.21.15.70:0/3287129527 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca402cdb0 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.219+0000 7f3cb0f50640 1 -- 172.21.15.70:0/3287129527 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4034660 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.219+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3287129527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 msgr2=0x7f3cac15e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.219+0000 7f3cb41b6640 1 --2- 172.21.15.70:0/3287129527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 0x7f3cac15e790 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009d90 tx=0x7f3ca402bea0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.219+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3287129527 shutdown_connections 2023-10-08T05:37:48.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.219+0000 7f3cb41b6640 1 --2- 172.21.15.70:0/3287129527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 0x7f3cac15e790 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.219+0000 7f3cb41b6640 1 --2- 172.21.15.70:0/3287129527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 0x7f3cac0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.219+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3287129527 >> 172.21.15.70:0/3287129527 conn(0x7f3cac06a380 msgr2=0x7f3cac0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:48.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.220+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3287129527 shutdown_connections 2023-10-08T05:37:48.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.220+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3287129527 wait complete. 2023-10-08T05:37:48.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.221+0000 7f3cb41b6640 1 Processor -- start 2023-10-08T05:37:48.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.221+0000 7f3cb41b6640 1 -- start start 2023-10-08T05:37:48.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.222+0000 7f3cb41b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 0x7f3cac1492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.222+0000 7f3cb41b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 0x7f3cac0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.222+0000 7f3cb41b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cac15ab40 0x7f3cac155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.222+0000 7f3cb41b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0b13d0 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.222+0000 7f3cb41b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0b1c70 con 0x7f3cac15ab40 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.222+0000 7f3cb41b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0c7d80 con 0x7f3cac0c90b0 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb1751640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 0x7f3cac0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb1751640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 0x7f3cac0c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37564/0 (socket says 172.21.15.70:37564) 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb1751640 1 -- 172.21.15.70:0/3794963666 learned_addr learned my addr 172.21.15.70:0/3794963666 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb2753640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cac15ab40 0x7f3cac155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb1f52640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 0x7f3cac1492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb1751640 1 -- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 msgr2=0x7f3cac1492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb1751640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 0x7f3cac1492e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb1751640 1 -- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cac15ab40 msgr2=0x7f3cac155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb1751640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cac15ab40 0x7f3cac155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb1751640 1 -- 172.21.15.70:0/3794963666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca4009a70 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb2753640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cac15ab40 0x7f3cac155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.223+0000 7f3cb1751640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 0x7f3cac0c72c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00c820 tx=0x7f3c9c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.224+0000 7f3ca27fc640 1 -- 172.21.15.70:0/3794963666 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c007a30 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.224+0000 7f3ca27fc640 1 -- 172.21.15.70:0/3794963666 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9c007b90 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.224+0000 7f3ca27fc640 1 -- 172.21.15.70:0/3794963666 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c00ce70 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.224+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3794963666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac0b1620 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.224+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3794963666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac1524c0 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.225+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3794963666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac0b1e00 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.226+0000 7f3ca27fc640 1 -- 172.21.15.70:0/3794963666 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f3c9c0113f0 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.227+0000 7f3ca27fc640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3c9006ae70 0x7f3c9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.228+0000 7f3ca27fc640 1 -- 172.21.15.70:0/3794963666 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3c9c089900 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.228+0000 7f3cb1f52640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3c9006ae70 0x7f3c9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:48.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.229+0000 7f3cb1f52640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3c9006ae70 0x7f3c9006d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80011e0 tx=0x7f3ca8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:48.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.232+0000 7f3ca27fc640 1 -- 172.21.15.70:0/3794963666 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c9c054860 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.457+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3794963666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cac0ce340 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.458+0000 7f3ca27fc640 1 -- 172.21.15.70:0/3794963666 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f3c9c01f090 con 0x7f3cac0cb3b0 2023-10-08T05:37:48.460 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:48.460 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/809656839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":809656839},{"type":"v1","addr":"172.21.15.70:6835","nonce":809656839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2750508216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2750508216},{"type":"v1","addr":"172.21.15.181:6836","nonce":2750508216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/607266140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":607266140},{"type":"v1","addr":"172.21.15.181:6837","nonce":607266140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:37:44.192492+0000","modified":"2023-10-08T05:37:47.238340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/480502229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":480502229},{"type":"v1","addr":"172.21.15.70:6834","nonce":480502229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-08T05:37:48.460 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 18 2023-10-08T05:37:48.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3c9006ae70 msgr2=0x7f3c9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3c9006ae70 0x7f3c9006d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80011e0 tx=0x7f3ca8009040 comp rx=0 tx=0).stop 2023-10-08T05:37:48.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3794963666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 msgr2=0x7f3cac0c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 0x7f3cac0c72c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00c820 tx=0x7f3c9c00cc20 comp rx=0 tx=0).stop 2023-10-08T05:37:48.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3794963666 shutdown_connections 2023-10-08T05:37:48.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f3c9006ae70 0x7f3c9006d330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3cac15ab40 0x7f3cac155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cac0cb3b0 0x7f3cac0c72c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 --2- 172.21.15.70:0/3794963666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3cac0c90b0 0x7f3cac1492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3794963666 >> 172.21.15.70:0/3794963666 conn(0x7f3cac06a380 msgr2=0x7f3cac0af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:48.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3794963666 shutdown_connections 2023-10-08T05:37:48.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.461+0000 7f3cb41b6640 1 -- 172.21.15.70:0/3794963666 wait complete. 2023-10-08T05:37:48.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:37:48.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.696+0000 7f36dd79e640 1 Processor -- start 2023-10-08T05:37:48.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.696+0000 7f36dd79e640 1 -- start start 2023-10-08T05:37:48.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.698+0000 7f36dd79e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f36d80ad790 0x7f36d80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.698+0000 7f36dd79e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d815e330 0x7f36d8070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.698+0000 7f36dd79e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d806dee0 con 0x7f36d8159ac0 2023-10-08T05:37:48.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.698+0000 7f36dd79e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d806dcf0 con 0x7f36d80ad790 2023-10-08T05:37:48.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.698+0000 7f36dd79e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d815d850 con 0x7f36d815e330 2023-10-08T05:37:48.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.698+0000 7f36d6ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f36d8159ac0 0x7f36d806d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59962/0 (socket says 172.21.15.70:59962) 2023-10-08T05:37:48.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.698+0000 7f36d6ffd640 1 -- 172.21.15.70:0/2198906308 learned_addr learned my addr 172.21.15.70:0/2198906308 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:48.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.698+0000 7f36ce7fc640 1 --2- 172.21.15.70:0/2198906308 >> v2:172.21.15.70:3300/0 conn(0x7f36d80ad790 0x7f36d80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:48.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.699+0000 7f36d77fe640 1 --2- 172.21.15.70:0/2198906308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d815e330 0x7f36d8070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:48.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.699+0000 7f36ce7fc640 1 -- 172.21.15.70:0/2198906308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d815e330 msgr2=0x7f36d8070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:48.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.699+0000 7f36ce7fc640 1 --2- 172.21.15.70:0/2198906308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d815e330 0x7f36d8070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.699+0000 7f36ce7fc640 1 -- 172.21.15.70:0/2198906308 >> v1:172.21.15.70:6789/0 conn(0x7f36d8159ac0 legacy=0x7f36d806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.699+0000 7f36ce7fc640 1 -- 172.21.15.70:0/2198906308 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d815db30 con 0x7f36d80ad790 2023-10-08T05:37:48.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.699+0000 7f36d77fe640 1 --2- 172.21.15.70:0/2198906308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d815e330 0x7f36d8070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:48.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.699+0000 7f36d67fc640 1 -- 172.21.15.70:0/2198906308 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 733126055 0 0) 0x7f36d806dee0 con 0x7f36d8159ac0 2023-10-08T05:37:48.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.700+0000 7f36ce7fc640 1 --2- 172.21.15.70:0/2198906308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 0x7f36d80702d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f36c00097a0 tx=0x7f36c002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1790bf74e1d090a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:48.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.700+0000 7f36d67fc640 1 -- 172.21.15.70:0/2198906308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c002d040 con 0x7f36d80ad790 2023-10-08T05:37:48.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.700+0000 7f36d67fc640 1 -- 172.21.15.70:0/2198906308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c0031810 con 0x7f36d80ad790 2023-10-08T05:37:48.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.700+0000 7f36d67fc640 1 -- 172.21.15.70:0/2198906308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0031ba0 con 0x7f36d80ad790 2023-10-08T05:37:48.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.700+0000 7f36dd79e640 1 -- 172.21.15.70:0/2198906308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 msgr2=0x7f36d80702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.700+0000 7f36dd79e640 1 --2- 172.21.15.70:0/2198906308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 0x7f36d80702d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f36c00097a0 tx=0x7f36c002b8b0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.701+0000 7f36dd79e640 1 -- 172.21.15.70:0/2198906308 shutdown_connections 2023-10-08T05:37:48.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.701+0000 7f36dd79e640 1 --2- 172.21.15.70:0/2198906308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d815e330 0x7f36d8070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.701+0000 7f36dd79e640 1 --2- 172.21.15.70:0/2198906308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 0x7f36d80702d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.701+0000 7f36dd79e640 1 -- 172.21.15.70:0/2198906308 >> 172.21.15.70:0/2198906308 conn(0x7f36d806a380 msgr2=0x7f36d80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:48.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.701+0000 7f36dd79e640 1 -- 172.21.15.70:0/2198906308 shutdown_connections 2023-10-08T05:37:48.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.701+0000 7f36dd79e640 1 -- 172.21.15.70:0/2198906308 wait complete. 2023-10-08T05:37:48.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.702+0000 7f36dd79e640 1 Processor -- start 2023-10-08T05:37:48.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.703+0000 7f36dd79e640 1 -- start start 2023-10-08T05:37:48.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.703+0000 7f36dd79e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 0x7f36d8150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.703+0000 7f36dd79e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d8159ac0 0x7f36d815b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.703+0000 7f36dd79e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d815e330 0x7f36d80acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.703+0000 7f36dd79e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d80ac670 con 0x7f36d80ad790 2023-10-08T05:37:48.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.703+0000 7f36dd79e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d8150a90 con 0x7f36d8159ac0 2023-10-08T05:37:48.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.703+0000 7f36dd79e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d80aba60 con 0x7f36d815e330 2023-10-08T05:37:48.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.704+0000 7f36d6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 0x7f36d8150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:48.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.704+0000 7f36d6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 0x7f36d8150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37576/0 (socket says 172.21.15.70:37576) 2023-10-08T05:37:48.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.704+0000 7f36d6ffd640 1 -- 172.21.15.70:0/3098927965 learned_addr learned my addr 172.21.15.70:0/3098927965 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:48.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.704+0000 7f36d6ffd640 1 -- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d815e330 msgr2=0x7f36d80acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.704+0000 7f36d6ffd640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d815e330 0x7f36d80acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.704+0000 7f36d6ffd640 1 -- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d8159ac0 msgr2=0x7f36d815b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.704+0000 7f36d6ffd640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d8159ac0 0x7f36d815b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.704+0000 7f36d6ffd640 1 -- 172.21.15.70:0/3098927965 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c0009400 con 0x7f36d80ad790 2023-10-08T05:37:48.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.704+0000 7f36d6ffd640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 0x7f36d8150550 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f36c800c4d0 tx=0x7f36c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:48.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.705+0000 7f36cffff640 1 -- 172.21.15.70:0/3098927965 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c80074e0 con 0x7f36d80ad790 2023-10-08T05:37:48.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.705+0000 7f36cffff640 1 -- 172.21.15.70:0/3098927965 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c8007660 con 0x7f36d80ad790 2023-10-08T05:37:48.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.705+0000 7f36cffff640 1 -- 172.21.15.70:0/3098927965 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c8004a20 con 0x7f36d80ad790 2023-10-08T05:37:48.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.705+0000 7f36dd79e640 1 -- 172.21.15.70:0/3098927965 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d80abd40 con 0x7f36d80ad790 2023-10-08T05:37:48.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.705+0000 7f36dd79e640 1 -- 172.21.15.70:0/3098927965 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d80ac300 con 0x7f36d80ad790 2023-10-08T05:37:48.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.706+0000 7f36dd79e640 1 -- 172.21.15.70:0/3098927965 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d8067490 con 0x7f36d80ad790 2023-10-08T05:37:48.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.707+0000 7f36cffff640 1 -- 172.21.15.70:0/3098927965 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f36c8004b80 con 0x7f36d80ad790 2023-10-08T05:37:48.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.708+0000 7f36cffff640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f36b806ae70 0x7f36b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:48.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.709+0000 7f36cffff640 1 -- 172.21.15.70:0/3098927965 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f36c80888b0 con 0x7f36d80ad790 2023-10-08T05:37:48.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.709+0000 7f36ce7fc640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f36b806ae70 0x7f36b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:48.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.710+0000 7f36ce7fc640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f36b806ae70 0x7f36b806d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f36c0036000 tx=0x7f36c0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:48.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.713+0000 7f36cffff640 1 -- 172.21.15.70:0/3098927965 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36c80538b0 con 0x7f36d80ad790 2023-10-08T05:37:48.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.903+0000 7f36dd79e640 1 -- 172.21.15.70:0/3098927965 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36d80b0ff0 con 0x7f36d80ad790 2023-10-08T05:37:48.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.906+0000 7f36cffff640 1 -- 172.21.15.70:0/3098927965 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16762 (secure 0 0 0) 0x7f36c8018020 con 0x7f36d80ad790 2023-10-08T05:37:48.908 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:48.908 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":39,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:37:45.733667+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:37:41.960853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-08T05:37:43.190325+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:37:48.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.908+0000 7f36dd79e640 1 -- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f36b806ae70 msgr2=0x7f36b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.908+0000 7f36dd79e640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f36b806ae70 0x7f36b806d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f36c0036000 tx=0x7f36c0030000 comp rx=0 tx=0).stop 2023-10-08T05:37:48.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.908+0000 7f36dd79e640 1 -- 172.21.15.70:0/3098927965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 msgr2=0x7f36d8150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.908+0000 7f36dd79e640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 0x7f36d8150550 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f36c800c4d0 tx=0x7f36c800c990 comp rx=0 tx=0).stop 2023-10-08T05:37:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.908+0000 7f36dd79e640 1 -- 172.21.15.70:0/3098927965 shutdown_connections 2023-10-08T05:37:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.908+0000 7f36dd79e640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f36b806ae70 0x7f36b806d330 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.908+0000 7f36dd79e640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f36d815e330 0x7f36d80acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.908+0000 7f36dd79e640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f36d8159ac0 0x7f36d815b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.908+0000 7f36dd79e640 1 --2- 172.21.15.70:0/3098927965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36d80ad790 0x7f36d8150550 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.908+0000 7f36dd79e640 1 -- 172.21.15.70:0/3098927965 >> 172.21.15.70:0/3098927965 conn(0x7f36d806a380 msgr2=0x7f36d80ca0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:48.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.909+0000 7f36dd79e640 1 -- 172.21.15.70:0/3098927965 shutdown_connections 2023-10-08T05:37:48.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:48.909+0000 7f36dd79e640 1 -- 172.21.15.70:0/3098927965 wait complete. 2023-10-08T05:37:48.929 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T05:37:48.930 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:37:49.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.142+0000 7f72f3fff640 1 Processor -- start 2023-10-08T05:37:49.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.142+0000 7f72f3fff640 1 -- start start 2023-10-08T05:37:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f41599d0 0x7f72f40ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72f40b07d0 0x7f72f406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f3fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f40cacf0 con 0x7f72f40cc210 2023-10-08T05:37:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f3fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f40b10f0 con 0x7f72f40b07d0 2023-10-08T05:37:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f3fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40ca3c0 con 0x7f72f41599d0 2023-10-08T05:37:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72f40b07d0 0x7f72f406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72f40b07d0 0x7f72f406c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37586/0 (socket says 172.21.15.70:37586) 2023-10-08T05:37:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f37fe640 1 -- 172.21.15.70:0/868719010 learned_addr learned my addr 172.21.15.70:0/868719010 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:49.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f2ffd640 1 --2- 172.21.15.70:0/868719010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f41599d0 0x7f72f40ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f37fe640 1 -- 172.21.15.70:0/868719010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f41599d0 msgr2=0x7f72f40ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:49.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f37fe640 1 --2- 172.21.15.70:0/868719010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f41599d0 0x7f72f40ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.144+0000 7f72f37fe640 1 -- 172.21.15.70:0/868719010 >> v1:172.21.15.181:6789/0 conn(0x7f72f40cc210 legacy=0x7f72f40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:49.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.145+0000 7f72f37fe640 1 -- 172.21.15.70:0/868719010 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f40ca6a0 con 0x7f72f40b07d0 2023-10-08T05:37:49.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.146+0000 7f72f37fe640 1 --2- 172.21.15.70:0/868719010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40b07d0 0x7f72f406c310 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009e50 tx=0x7f72ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f325338dd2265c97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:49.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.146+0000 7f72f27fc640 1 -- 172.21.15.70:0/868719010 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec02ca70 con 0x7f72f40b07d0 2023-10-08T05:37:49.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.146+0000 7f72f27fc640 1 -- 172.21.15.70:0/868719010 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec02cbd0 con 0x7f72f40b07d0 2023-10-08T05:37:49.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.146+0000 7f72f27fc640 1 -- 172.21.15.70:0/868719010 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0343f0 con 0x7f72f40b07d0 2023-10-08T05:37:49.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.146+0000 7f72f3fff640 1 -- 172.21.15.70:0/868719010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40b07d0 msgr2=0x7f72f406c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.146+0000 7f72f3fff640 1 --2- 172.21.15.70:0/868719010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40b07d0 0x7f72f406c310 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009e50 tx=0x7f72ec02bed0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.147+0000 7f72f3fff640 1 -- 172.21.15.70:0/868719010 shutdown_connections 2023-10-08T05:37:49.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.147+0000 7f72f3fff640 1 --2- 172.21.15.70:0/868719010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40b07d0 0x7f72f406c310 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.147+0000 7f72f3fff640 1 --2- 172.21.15.70:0/868719010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f41599d0 0x7f72f40ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.147+0000 7f72f3fff640 1 -- 172.21.15.70:0/868719010 >> 172.21.15.70:0/868719010 conn(0x7f72f40cb0c0 msgr2=0x7f72f40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:49.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.147+0000 7f72f3fff640 1 -- 172.21.15.70:0/868719010 shutdown_connections 2023-10-08T05:37:49.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.147+0000 7f72f3fff640 1 -- 172.21.15.70:0/868719010 wait complete. 2023-10-08T05:37:49.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.148+0000 7f72f3fff640 1 Processor -- start 2023-10-08T05:37:49.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.149+0000 7f72f3fff640 1 -- start start 2023-10-08T05:37:49.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.149+0000 7f72f3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72f40b07d0 0x7f72f4151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40cc210 0x7f72f414e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72f3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f41599d0 0x7f72f40c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72f3fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40b0640 con 0x7f72f40cc210 2023-10-08T05:37:49.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72f3fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f414eef0 con 0x7f72f40b07d0 2023-10-08T05:37:49.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72f3fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f40c8770 con 0x7f72f41599d0 2023-10-08T05:37:49.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72ebfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40cc210 0x7f72f414e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72ebfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40cc210 0x7f72f414e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37590/0 (socket says 172.21.15.70:37590) 2023-10-08T05:37:49.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72ebfff640 1 -- 172.21.15.70:0/1963591100 learned_addr learned my addr 172.21.15.70:0/1963591100 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72f2ffd640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72f40b07d0 0x7f72f4151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72ebfff640 1 -- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f41599d0 msgr2=0x7f72f40c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72ebfff640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f41599d0 0x7f72f40c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72ebfff640 1 -- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72f40b07d0 msgr2=0x7f72f4151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72ebfff640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72f40b07d0 0x7f72f4151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.150+0000 7f72ebfff640 1 -- 172.21.15.70:0/1963591100 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec009b30 con 0x7f72f40cc210 2023-10-08T05:37:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.151+0000 7f72ebfff640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40cc210 0x7f72f414e9b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c430 tx=0x7f72dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.151+0000 7f72f0ff9640 1 -- 172.21.15.70:0/1963591100 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc011790 con 0x7f72f40cc210 2023-10-08T05:37:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.151+0000 7f72f0ff9640 1 -- 172.21.15.70:0/1963591100 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc0118f0 con 0x7f72f40cc210 2023-10-08T05:37:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.151+0000 7f72f3fff640 1 -- 172.21.15.70:0/1963591100 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f40b0360 con 0x7f72f40cc210 2023-10-08T05:37:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.151+0000 7f72f0ff9640 1 -- 172.21.15.70:0/1963591100 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc011ca0 con 0x7f72f40cc210 2023-10-08T05:37:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.151+0000 7f72f3fff640 1 -- 172.21.15.70:0/1963591100 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f40af570 con 0x7f72f40cc210 2023-10-08T05:37:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.152+0000 7f72f3fff640 1 -- 172.21.15.70:0/1963591100 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f40b1230 con 0x7f72f40cc210 2023-10-08T05:37:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.153+0000 7f72f0ff9640 1 -- 172.21.15.70:0/1963591100 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f72dc01c460 con 0x7f72f40cc210 2023-10-08T05:37:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.154+0000 7f72f0ff9640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f72d406aea0 0x7f72d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.154+0000 7f72f0ff9640 1 -- 172.21.15.70:0/1963591100 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f72dc0896d0 con 0x7f72f40cc210 2023-10-08T05:37:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.155+0000 7f72f2ffd640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f72d406aea0 0x7f72d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.156+0000 7f72f2ffd640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f72d406aea0 0x7f72d406d360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72f40cd660 tx=0x7f72e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.158+0000 7f72f0ff9640 1 -- 172.21.15.70:0/1963591100 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72f40b1230 con 0x7f72f40cc210 2023-10-08T05:37:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.345+0000 7f72f3fff640 1 -- 172.21.15.70:0/1963591100 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72f40ca070 con 0x7f72d406aea0 2023-10-08T05:37:49.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.369+0000 7f72f0ff9640 1 -- 172.21.15.70:0/1963591100 <== mgr.4105 v2:172.21.15.181:6832/2737793492 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281813 (secure 0 0 0) 0x7f72e400c040 con 0x7f72d406aea0 2023-10-08T05:37:49.372 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:49.378 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:37:49.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.376+0000 7f72f3fff640 1 -- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f72d406aea0 msgr2=0x7f72d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.376+0000 7f72f3fff640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f72d406aea0 0x7f72d406d360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72f40cd660 tx=0x7f72e4008040 comp rx=0 tx=0).stop 2023-10-08T05:37:49.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.376+0000 7f72f3fff640 1 -- 172.21.15.70:0/1963591100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40cc210 msgr2=0x7f72f414e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.376+0000 7f72f3fff640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40cc210 0x7f72f414e9b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00c430 tx=0x7f72dc00c8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.376+0000 7f72f3fff640 1 -- 172.21.15.70:0/1963591100 shutdown_connections 2023-10-08T05:37:49.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.376+0000 7f72f3fff640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f72d406aea0 0x7f72d406d360 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.376+0000 7f72f3fff640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f41599d0 0x7f72f40c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.376+0000 7f72f3fff640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f40cc210 0x7f72f414e9b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.376+0000 7f72f3fff640 1 --2- 172.21.15.70:0/1963591100 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72f40b07d0 0x7f72f4151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.376+0000 7f72f3fff640 1 -- 172.21.15.70:0/1963591100 >> 172.21.15.70:0/1963591100 conn(0x7f72f40cb0c0 msgr2=0x7f72f40abcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:49.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.377+0000 7f72f3fff640 1 -- 172.21.15.70:0/1963591100 shutdown_connections 2023-10-08T05:37:49.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.377+0000 7f72f3fff640 1 -- 172.21.15.70:0/1963591100 wait complete. 2023-10-08T05:37:49.398 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2023-10-08T05:37:49.106292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237252,"kb_used_data":4440,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737468,"statfs":{"total":773094113280,"available":772851167232,"internally_reserved":0,"allocated":4546560,"data_stored":2595348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001704"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.726910+0000","last_change":"2023-10-08T05:37:43.726910+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.726910+0000","last_undegraded":"2023-10-08T05:37:43.726910+0000","last_fullsized":"2023-10-08T05:37:43.726910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.725861+0000","last_change":"2023-10-08T05:37:43.725861+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.725861+0000","last_undegraded":"2023-10-08T05:37:43.725861+0000","last_fullsized":"2023-10-08T05:37:43.725861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.722327+0000","last_change":"2023-10-08T05:37:43.722327+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.722327+0000","last_undegraded":"2023-10-08T05:37:43.722327+0000","last_fullsized":"2023-10-08T05:37:43.722327+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721430+0000","last_change":"2023-10-08T05:37:43.721430+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721430+0000","last_undegraded":"2023-10-08T05:37:43.721430+0000","last_fullsized":"2023-10-08T05:37:43.721430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723901+0000","last_change":"2023-10-08T05:37:43.723901+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723901+0000","last_undegraded":"2023-10-08T05:37:43.723901+0000","last_fullsized":"2023-10-08T05:37:43.723901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.727715+0000","last_change":"2023-10-08T05:37:43.727715+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.727715+0000","last_undegraded":"2023-10-08T05:37:43.727715+0000","last_fullsized":"2023-10-08T05:37:43.727715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.740518+0000","last_change":"2023-10-08T05:37:43.740518+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.740518+0000","last_undegraded":"2023-10-08T05:37:43.740518+0000","last_fullsized":"2023-10-08T05:37:43.740518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721832+0000","last_change":"2023-10-08T05:37:43.721832+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721832+0000","last_undegraded":"2023-10-08T05:37:43.721832+0000","last_fullsized":"2023-10-08T05:37:43.721832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724170+0000","last_change":"2023-10-08T05:37:43.724170+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724170+0000","last_undegraded":"2023-10-08T05:37:43.724170+0000","last_fullsized":"2023-10-08T05:37:43.724170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737987+0000","last_change":"2023-10-08T05:37:43.737987+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737987+0000","last_undegraded":"2023-10-08T05:37:43.737987+0000","last_fullsized":"2023-10-08T05:37:43.737987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.735624+0000","last_change":"2023-10-08T05:37:43.735624+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.735624+0000","last_undegraded":"2023-10-08T05:37:43.735624+0000","last_fullsized":"2023-10-08T05:37:43.735624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723176+0000","last_change":"2023-10-08T05:37:43.723176+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723176+0000","last_undegraded":"2023-10-08T05:37:43.723176+0000","last_fullsized":"2023-10-08T05:37:43.723176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.738432+0000","last_change":"2023-10-08T05:37:43.738432+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.738432+0000","last_undegraded":"2023-10-08T05:37:43.738432+0000","last_fullsized":"2023-10-08T05:37:43.738432+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.730462+0000","last_change":"2023-10-08T05:37:43.730462+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.730462+0000","last_undegraded":"2023-10-08T05:37:43.730462+0000","last_fullsized":"2023-10-08T05:37:43.730462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.740059+0000","last_change":"2023-10-08T05:37:43.740059+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.740059+0000","last_undegraded":"2023-10-08T05:37:43.740059+0000","last_fullsized":"2023-10-08T05:37:43.740059+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.730787+0000","last_change":"2023-10-08T05:37:43.730787+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.730787+0000","last_undegraded":"2023-10-08T05:37:43.730787+0000","last_fullsized":"2023-10-08T05:37:43.730787+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.738796+0000","last_change":"2023-10-08T05:37:43.738796+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.738796+0000","last_undegraded":"2023-10-08T05:37:43.738796+0000","last_fullsized":"2023-10-08T05:37:43.738796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723467+0000","last_change":"2023-10-08T05:37:43.723467+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723467+0000","last_undegraded":"2023-10-08T05:37:43.723467+0000","last_fullsized":"2023-10-08T05:37:43.723467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720545+0000","last_change":"2023-10-08T05:37:43.720545+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720545+0000","last_undegraded":"2023-10-08T05:37:43.720545+0000","last_fullsized":"2023-10-08T05:37:43.720545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.727938+0000","last_change":"2023-10-08T05:37:43.727938+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.727938+0000","last_undegraded":"2023-10-08T05:37:43.727938+0000","last_fullsized":"2023-10-08T05:37:43.727938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737934+0000","last_change":"2023-10-08T05:37:43.737934+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737934+0000","last_undegraded":"2023-10-08T05:37:43.737934+0000","last_fullsized":"2023-10-08T05:37:43.737934+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728439+0000","last_change":"2023-10-08T05:37:43.728439+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728439+0000","last_undegraded":"2023-10-08T05:37:43.728439+0000","last_fullsized":"2023-10-08T05:37:43.728439+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.736650+0000","last_change":"2023-10-08T05:37:43.736650+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.736650+0000","last_undegraded":"2023-10-08T05:37:43.736650+0000","last_fullsized":"2023-10-08T05:37:43.736650+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720791+0000","last_change":"2023-10-08T05:37:43.720791+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720791+0000","last_undegraded":"2023-10-08T05:37:43.720791+0000","last_fullsized":"2023-10-08T05:37:43.720791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.725014+0000","last_change":"2023-10-08T05:37:43.725014+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.725014+0000","last_undegraded":"2023-10-08T05:37:43.725014+0000","last_fullsized":"2023-10-08T05:37:43.725014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728973+0000","last_change":"2023-10-08T05:37:43.728973+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728973+0000","last_undegraded":"2023-10-08T05:37:43.728973+0000","last_fullsized":"2023-10-08T05:37:43.728973+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723172+0000","last_change":"2023-10-08T05:37:43.723172+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723172+0000","last_undegraded":"2023-10-08T05:37:43.723172+0000","last_fullsized":"2023-10-08T05:37:43.723172+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.740470+0000","last_change":"2023-10-08T05:37:43.740470+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.740470+0000","last_undegraded":"2023-10-08T05:37:43.740470+0000","last_fullsized":"2023-10-08T05:37:43.740470+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721405+0000","last_change":"2023-10-08T05:37:43.721405+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721405+0000","last_undegraded":"2023-10-08T05:37:43.721405+0000","last_fullsized":"2023-10-08T05:37:43.721405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.722018+0000","last_change":"2023-10-08T05:37:43.722018+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.722018+0000","last_undegraded":"2023-10-08T05:37:43.722018+0000","last_fullsized":"2023-10-08T05:37:43.722018+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.733264+0000","last_change":"2023-10-08T05:37:43.733264+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.733264+0000","last_undegraded":"2023-10-08T05:37:43.733264+0000","last_fullsized":"2023-10-08T05:37:43.733264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728215+0000","last_change":"2023-10-08T05:37:43.728215+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728215+0000","last_undegraded":"2023-10-08T05:37:43.728215+0000","last_fullsized":"2023-10-08T05:37:43.728215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.739149+0000","last_change":"2023-10-08T05:37:43.739149+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.739149+0000","last_undegraded":"2023-10-08T05:37:43.739149+0000","last_fullsized":"2023-10-08T05:37:43.739149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.749119+0000","last_change":"2023-10-08T05:37:43.746333+0000","last_active":"2023-10-08T05:37:43.749119+0000","last_peered":"2023-10-08T05:37:43.749119+0000","last_clean":"2023-10-08T05:37:43.749119+0000","last_became_active":"2023-10-08T05:37:43.746017+0000","last_became_peered":"2023-10-08T05:37:43.746017+0000","last_unstale":"2023-10-08T05:37:43.749119+0000","last_undegraded":"2023-10-08T05:37:43.749119+0000","last_fullsized":"2023-10-08T05:37:43.749119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:04:48.940111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.729258+0000","last_change":"2023-10-08T05:37:43.729258+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.729258+0000","last_undegraded":"2023-10-08T05:37:43.729258+0000","last_fullsized":"2023-10-08T05:37:43.729258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751007+0000","last_change":"2023-10-08T05:37:43.749778+0000","last_active":"2023-10-08T05:37:43.751007+0000","last_peered":"2023-10-08T05:37:43.751007+0000","last_clean":"2023-10-08T05:37:43.751007+0000","last_became_active":"2023-10-08T05:37:43.749435+0000","last_became_peered":"2023-10-08T05:37:43.749435+0000","last_unstale":"2023-10-08T05:37:43.751007+0000","last_undegraded":"2023-10-08T05:37:43.751007+0000","last_fullsized":"2023-10-08T05:37:43.751007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:35.696328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.726040+0000","last_change":"2023-10-08T05:37:43.726040+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.726040+0000","last_undegraded":"2023-10-08T05:37:43.726040+0000","last_fullsized":"2023-10-08T05:37:43.726040+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.750923+0000","last_change":"2023-10-08T05:37:43.749986+0000","last_active":"2023-10-08T05:37:43.750923+0000","last_peered":"2023-10-08T05:37:43.750923+0000","last_clean":"2023-10-08T05:37:43.750923+0000","last_became_active":"2023-10-08T05:37:43.749718+0000","last_became_peered":"2023-10-08T05:37:43.749718+0000","last_unstale":"2023-10-08T05:37:43.750923+0000","last_undegraded":"2023-10-08T05:37:43.750923+0000","last_fullsized":"2023-10-08T05:37:43.750923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:12.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721433+0000","last_change":"2023-10-08T05:37:43.721433+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721433+0000","last_undegraded":"2023-10-08T05:37:43.721433+0000","last_fullsized":"2023-10-08T05:37:43.721433+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.748417+0000","last_change":"2023-10-08T05:37:43.742613+0000","last_active":"2023-10-08T05:37:43.748417+0000","last_peered":"2023-10-08T05:37:43.748417+0000","last_clean":"2023-10-08T05:37:43.748417+0000","last_became_active":"2023-10-08T05:37:43.742232+0000","last_became_peered":"2023-10-08T05:37:43.742232+0000","last_unstale":"2023-10-08T05:37:43.748417+0000","last_undegraded":"2023-10-08T05:37:43.748417+0000","last_fullsized":"2023-10-08T05:37:43.748417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:35.696328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724604+0000","last_change":"2023-10-08T05:37:43.724604+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724604+0000","last_undegraded":"2023-10-08T05:37:43.724604+0000","last_fullsized":"2023-10-08T05:37:43.724604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752140+0000","last_change":"2023-10-08T05:37:43.743385+0000","last_active":"2023-10-08T05:37:43.752140+0000","last_peered":"2023-10-08T05:37:43.752140+0000","last_clean":"2023-10-08T05:37:43.752140+0000","last_became_active":"2023-10-08T05:37:43.743002+0000","last_became_peered":"2023-10-08T05:37:43.743002+0000","last_unstale":"2023-10-08T05:37:43.752140+0000","last_undegraded":"2023-10-08T05:37:43.752140+0000","last_fullsized":"2023-10-08T05:37:43.752140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:56:20.161604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.739571+0000","last_change":"2023-10-08T05:37:43.739571+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.739571+0000","last_undegraded":"2023-10-08T05:37:43.739571+0000","last_fullsized":"2023-10-08T05:37:43.739571+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.747959+0000","last_change":"2023-10-08T05:37:43.746646+0000","last_active":"2023-10-08T05:37:43.747959+0000","last_peered":"2023-10-08T05:37:43.747959+0000","last_clean":"2023-10-08T05:37:43.747959+0000","last_became_active":"2023-10-08T05:37:43.746369+0000","last_became_peered":"2023-10-08T05:37:43.746369+0000","last_unstale":"2023-10-08T05:37:43.747959+0000","last_undegraded":"2023-10-08T05:37:43.747959+0000","last_fullsized":"2023-10-08T05:37:43.747959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:12.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.725832+0000","last_change":"2023-10-08T05:37:43.725832+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.725832+0000","last_undegraded":"2023-10-08T05:37:43.725832+0000","last_fullsized":"2023-10-08T05:37:43.725832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.747537+0000","last_change":"2023-10-08T05:37:43.746978+0000","last_active":"2023-10-08T05:37:43.747537+0000","last_peered":"2023-10-08T05:37:43.747537+0000","last_clean":"2023-10-08T05:37:43.747537+0000","last_became_active":"2023-10-08T05:37:43.746728+0000","last_became_peered":"2023-10-08T05:37:43.746728+0000","last_unstale":"2023-10-08T05:37:43.747537+0000","last_undegraded":"2023-10-08T05:37:43.747537+0000","last_fullsized":"2023-10-08T05:37:43.747537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724140+0000","last_change":"2023-10-08T05:37:43.724140+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724140+0000","last_undegraded":"2023-10-08T05:37:43.724140+0000","last_fullsized":"2023-10-08T05:37:43.724140+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.745405+0000","last_change":"2023-10-08T05:37:43.736460+0000","last_active":"2023-10-08T05:37:43.745405+0000","last_peered":"2023-10-08T05:37:43.745405+0000","last_clean":"2023-10-08T05:37:43.745405+0000","last_became_active":"2023-10-08T05:37:43.736033+0000","last_became_peered":"2023-10-08T05:37:43.736033+0000","last_unstale":"2023-10-08T05:37:43.745405+0000","last_undegraded":"2023-10-08T05:37:43.745405+0000","last_fullsized":"2023-10-08T05:37:43.745405+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.017005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720387+0000","last_change":"2023-10-08T05:37:43.720387+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720387+0000","last_undegraded":"2023-10-08T05:37:43.720387+0000","last_fullsized":"2023-10-08T05:37:43.720387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.737892+0000","last_change":"2023-10-08T05:37:43.735761+0000","last_active":"2023-10-08T05:37:43.737892+0000","last_peered":"2023-10-08T05:37:43.737892+0000","last_clean":"2023-10-08T05:37:43.737892+0000","last_became_active":"2023-10-08T05:37:43.735391+0000","last_became_peered":"2023-10-08T05:37:43.735391+0000","last_unstale":"2023-10-08T05:37:43.737892+0000","last_undegraded":"2023-10-08T05:37:43.737892+0000","last_fullsized":"2023-10-08T05:37:43.737892+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:26.955372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.734924+0000","last_change":"2023-10-08T05:37:43.734924+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.734924+0000","last_undegraded":"2023-10-08T05:37:43.734924+0000","last_fullsized":"2023-10-08T05:37:43.734924+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751246+0000","last_change":"2023-10-08T05:37:43.742980+0000","last_active":"2023-10-08T05:37:43.751246+0000","last_peered":"2023-10-08T05:37:43.751246+0000","last_clean":"2023-10-08T05:37:43.751246+0000","last_became_active":"2023-10-08T05:37:43.742358+0000","last_became_peered":"2023-10-08T05:37:43.742358+0000","last_unstale":"2023-10-08T05:37:43.751246+0000","last_undegraded":"2023-10-08T05:37:43.751246+0000","last_fullsized":"2023-10-08T05:37:43.751246+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:53.615714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721234+0000","last_change":"2023-10-08T05:37:43.721234+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721234+0000","last_undegraded":"2023-10-08T05:37:43.721234+0000","last_fullsized":"2023-10-08T05:37:43.721234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752211+0000","last_change":"2023-10-08T05:37:43.743414+0000","last_active":"2023-10-08T05:37:43.752211+0000","last_peered":"2023-10-08T05:37:43.752211+0000","last_clean":"2023-10-08T05:37:43.752211+0000","last_became_active":"2023-10-08T05:37:43.742873+0000","last_became_peered":"2023-10-08T05:37:43.742873+0000","last_unstale":"2023-10-08T05:37:43.752211+0000","last_undegraded":"2023-10-08T05:37:43.752211+0000","last_fullsized":"2023-10-08T05:37:43.752211+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724763+0000","last_change":"2023-10-08T05:37:43.724763+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724763+0000","last_undegraded":"2023-10-08T05:37:43.724763+0000","last_fullsized":"2023-10-08T05:37:43.724763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.750862+0000","last_change":"2023-10-08T05:37:43.749228+0000","last_active":"2023-10-08T05:37:43.750862+0000","last_peered":"2023-10-08T05:37:43.750862+0000","last_clean":"2023-10-08T05:37:43.750862+0000","last_became_active":"2023-10-08T05:37:43.748703+0000","last_became_peered":"2023-10-08T05:37:43.748703+0000","last_unstale":"2023-10-08T05:37:43.750862+0000","last_undegraded":"2023-10-08T05:37:43.750862+0000","last_fullsized":"2023-10-08T05:37:43.750862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:11:28.561634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724118+0000","last_change":"2023-10-08T05:37:43.724118+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724118+0000","last_undegraded":"2023-10-08T05:37:43.724118+0000","last_fullsized":"2023-10-08T05:37:43.724118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.730582+0000","last_change":"2023-10-08T05:37:43.729198+0000","last_active":"2023-10-08T05:37:43.730582+0000","last_peered":"2023-10-08T05:37:43.730582+0000","last_clean":"2023-10-08T05:37:43.730582+0000","last_became_active":"2023-10-08T05:37:43.728918+0000","last_became_peered":"2023-10-08T05:37:43.728918+0000","last_unstale":"2023-10-08T05:37:43.730582+0000","last_undegraded":"2023-10-08T05:37:43.730582+0000","last_fullsized":"2023-10-08T05:37:43.730582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720385+0000","last_change":"2023-10-08T05:37:43.720385+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720385+0000","last_undegraded":"2023-10-08T05:37:43.720385+0000","last_fullsized":"2023-10-08T05:37:43.720385+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746846+0000","last_change":"2023-10-08T05:37:43.745271+0000","last_active":"2023-10-08T05:37:43.746846+0000","last_peered":"2023-10-08T05:37:43.746846+0000","last_clean":"2023-10-08T05:37:43.746846+0000","last_became_active":"2023-10-08T05:37:43.745043+0000","last_became_peered":"2023-10-08T05:37:43.745043+0000","last_unstale":"2023-10-08T05:37:43.746846+0000","last_undegraded":"2023-10-08T05:37:43.746846+0000","last_fullsized":"2023-10-08T05:37:43.746846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:34.606944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737702+0000","last_change":"2023-10-08T05:37:43.737702+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737702+0000","last_undegraded":"2023-10-08T05:37:43.737702+0000","last_fullsized":"2023-10-08T05:37:43.737702+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.749151+0000","last_change":"2023-10-08T05:37:43.746332+0000","last_active":"2023-10-08T05:37:43.749151+0000","last_peered":"2023-10-08T05:37:43.749151+0000","last_clean":"2023-10-08T05:37:43.749151+0000","last_became_active":"2023-10-08T05:37:43.746017+0000","last_became_peered":"2023-10-08T05:37:43.746017+0000","last_unstale":"2023-10-08T05:37:43.749151+0000","last_undegraded":"2023-10-08T05:37:43.749151+0000","last_fullsized":"2023-10-08T05:37:43.749151+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:10.220125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.722997+0000","last_change":"2023-10-08T05:37:43.722997+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.722997+0000","last_undegraded":"2023-10-08T05:37:43.722997+0000","last_fullsized":"2023-10-08T05:37:43.722997+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752512+0000","last_change":"2023-10-08T05:37:43.750157+0000","last_active":"2023-10-08T05:37:43.752512+0000","last_peered":"2023-10-08T05:37:43.752512+0000","last_clean":"2023-10-08T05:37:43.752512+0000","last_became_active":"2023-10-08T05:37:43.749846+0000","last_became_peered":"2023-10-08T05:37:43.749846+0000","last_unstale":"2023-10-08T05:37:43.752512+0000","last_undegraded":"2023-10-08T05:37:43.752512+0000","last_fullsized":"2023-10-08T05:37:43.752512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:53.615714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.719971+0000","last_change":"2023-10-08T05:37:43.719971+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.719971+0000","last_undegraded":"2023-10-08T05:37:43.719971+0000","last_fullsized":"2023-10-08T05:37:43.719971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.753906+0000","last_change":"2023-10-08T05:37:43.752005+0000","last_active":"2023-10-08T05:37:43.753906+0000","last_peered":"2023-10-08T05:37:43.753906+0000","last_clean":"2023-10-08T05:37:43.753906+0000","last_became_active":"2023-10-08T05:37:43.751525+0000","last_became_peered":"2023-10-08T05:37:43.751525+0000","last_unstale":"2023-10-08T05:37:43.753906+0000","last_undegraded":"2023-10-08T05:37:43.753906+0000","last_fullsized":"2023-10-08T05:37:43.753906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.734950+0000","last_change":"2023-10-08T05:37:43.734950+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.734950+0000","last_undegraded":"2023-10-08T05:37:43.734950+0000","last_fullsized":"2023-10-08T05:37:43.734950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.749409+0000","last_change":"2023-10-08T05:37:43.745487+0000","last_active":"2023-10-08T05:37:43.749409+0000","last_peered":"2023-10-08T05:37:43.749409+0000","last_clean":"2023-10-08T05:37:43.749409+0000","last_became_active":"2023-10-08T05:37:43.744512+0000","last_became_peered":"2023-10-08T05:37:43.744512+0000","last_unstale":"2023-10-08T05:37:43.749409+0000","last_undegraded":"2023-10-08T05:37:43.749409+0000","last_fullsized":"2023-10-08T05:37:43.749409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:10.719966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.727258+0000","last_change":"2023-10-08T05:37:43.727258+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.727258+0000","last_undegraded":"2023-10-08T05:37:43.727258+0000","last_fullsized":"2023-10-08T05:37:43.727258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751126+0000","last_change":"2023-10-08T05:37:43.748895+0000","last_active":"2023-10-08T05:37:43.751126+0000","last_peered":"2023-10-08T05:37:43.751126+0000","last_clean":"2023-10-08T05:37:43.751126+0000","last_became_active":"2023-10-08T05:37:43.748618+0000","last_became_peered":"2023-10-08T05:37:43.748618+0000","last_unstale":"2023-10-08T05:37:43.751126+0000","last_undegraded":"2023-10-08T05:37:43.751126+0000","last_fullsized":"2023-10-08T05:37:43.751126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:35.801089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.719846+0000","last_change":"2023-10-08T05:37:43.719846+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.719846+0000","last_undegraded":"2023-10-08T05:37:43.719846+0000","last_fullsized":"2023-10-08T05:37:43.719846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746320+0000","last_change":"2023-10-08T05:37:43.743970+0000","last_active":"2023-10-08T05:37:43.746320+0000","last_peered":"2023-10-08T05:37:43.746320+0000","last_clean":"2023-10-08T05:37:43.746320+0000","last_became_active":"2023-10-08T05:37:43.743106+0000","last_became_peered":"2023-10-08T05:37:43.743106+0000","last_unstale":"2023-10-08T05:37:43.746320+0000","last_undegraded":"2023-10-08T05:37:43.746320+0000","last_fullsized":"2023-10-08T05:37:43.746320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:27.563368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.729757+0000","last_change":"2023-10-08T05:37:43.729757+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.729757+0000","last_undegraded":"2023-10-08T05:37:43.729757+0000","last_fullsized":"2023-10-08T05:37:43.729757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.731932+0000","last_change":"2023-10-08T05:37:43.728582+0000","last_active":"2023-10-08T05:37:43.731932+0000","last_peered":"2023-10-08T05:37:43.731932+0000","last_clean":"2023-10-08T05:37:43.731932+0000","last_became_active":"2023-10-08T05:37:43.728172+0000","last_became_peered":"2023-10-08T05:37:43.728172+0000","last_unstale":"2023-10-08T05:37:43.731932+0000","last_undegraded":"2023-10-08T05:37:43.731932+0000","last_fullsized":"2023-10-08T05:37:43.731932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:34.248097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721280+0000","last_change":"2023-10-08T05:37:43.721280+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721280+0000","last_undegraded":"2023-10-08T05:37:43.721280+0000","last_fullsized":"2023-10-08T05:37:43.721280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746296+0000","last_change":"2023-10-08T05:37:43.744243+0000","last_active":"2023-10-08T05:37:43.746296+0000","last_peered":"2023-10-08T05:37:43.746296+0000","last_clean":"2023-10-08T05:37:43.746296+0000","last_became_active":"2023-10-08T05:37:43.743356+0000","last_became_peered":"2023-10-08T05:37:43.743356+0000","last_unstale":"2023-10-08T05:37:43.746296+0000","last_undegraded":"2023-10-08T05:37:43.746296+0000","last_fullsized":"2023-10-08T05:37:43.746296+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:26:44.883068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.735316+0000","last_change":"2023-10-08T05:37:43.735316+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.735316+0000","last_undegraded":"2023-10-08T05:37:43.735316+0000","last_fullsized":"2023-10-08T05:37:43.735316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752420+0000","last_change":"2023-10-08T05:37:43.749749+0000","last_active":"2023-10-08T05:37:43.752420+0000","last_peered":"2023-10-08T05:37:43.752420+0000","last_clean":"2023-10-08T05:37:43.752420+0000","last_became_active":"2023-10-08T05:37:43.749413+0000","last_became_peered":"2023-10-08T05:37:43.749413+0000","last_unstale":"2023-10-08T05:37:43.752420+0000","last_undegraded":"2023-10-08T05:37:43.752420+0000","last_fullsized":"2023-10-08T05:37:43.752420+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:27.563368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.719803+0000","last_change":"2023-10-08T05:37:43.719803+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.719803+0000","last_undegraded":"2023-10-08T05:37:43.719803+0000","last_fullsized":"2023-10-08T05:37:43.719803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.734537+0000","last_change":"2023-10-08T05:37:43.733311+0000","last_active":"2023-10-08T05:37:43.734537+0000","last_peered":"2023-10-08T05:37:43.734537+0000","last_clean":"2023-10-08T05:37:43.734537+0000","last_became_active":"2023-10-08T05:37:43.733051+0000","last_became_peered":"2023-10-08T05:37:43.733051+0000","last_unstale":"2023-10-08T05:37:43.734537+0000","last_undegraded":"2023-10-08T05:37:43.734537+0000","last_fullsized":"2023-10-08T05:37:43.734537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:02.738727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728203+0000","last_change":"2023-10-08T05:37:43.728203+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728203+0000","last_undegraded":"2023-10-08T05:37:43.728203+0000","last_fullsized":"2023-10-08T05:37:43.728203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752422+0000","last_change":"2023-10-08T05:37:43.749129+0000","last_active":"2023-10-08T05:37:43.752422+0000","last_peered":"2023-10-08T05:37:43.752422+0000","last_clean":"2023-10-08T05:37:43.752422+0000","last_became_active":"2023-10-08T05:37:43.748832+0000","last_became_peered":"2023-10-08T05:37:43.748832+0000","last_unstale":"2023-10-08T05:37:43.752422+0000","last_undegraded":"2023-10-08T05:37:43.752422+0000","last_fullsized":"2023-10-08T05:37:43.752422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:41.403273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.733504+0000","last_change":"2023-10-08T05:37:43.733504+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.733504+0000","last_undegraded":"2023-10-08T05:37:43.733504+0000","last_fullsized":"2023-10-08T05:37:43.733504+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746378+0000","last_change":"2023-10-08T05:37:43.744435+0000","last_active":"2023-10-08T05:37:43.746378+0000","last_peered":"2023-10-08T05:37:43.746378+0000","last_clean":"2023-10-08T05:37:43.746378+0000","last_became_active":"2023-10-08T05:37:43.744157+0000","last_became_peered":"2023-10-08T05:37:43.744157+0000","last_unstale":"2023-10-08T05:37:43.746378+0000","last_undegraded":"2023-10-08T05:37:43.746378+0000","last_fullsized":"2023-10-08T05:37:43.746378+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:21:09.150233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720525+0000","last_change":"2023-10-08T05:37:43.720525+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720525+0000","last_undegraded":"2023-10-08T05:37:43.720525+0000","last_fullsized":"2023-10-08T05:37:43.720525+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.736500+0000","last_change":"2023-10-08T05:37:43.733742+0000","last_active":"2023-10-08T05:37:43.736500+0000","last_peered":"2023-10-08T05:37:43.736500+0000","last_clean":"2023-10-08T05:37:43.736500+0000","last_became_active":"2023-10-08T05:37:43.733486+0000","last_became_peered":"2023-10-08T05:37:43.733486+0000","last_unstale":"2023-10-08T05:37:43.736500+0000","last_undegraded":"2023-10-08T05:37:43.736500+0000","last_fullsized":"2023-10-08T05:37:43.736500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:10.719966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723702+0000","last_change":"2023-10-08T05:37:43.723702+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723702+0000","last_undegraded":"2023-10-08T05:37:43.723702+0000","last_fullsized":"2023-10-08T05:37:43.723702+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.736440+0000","last_change":"2023-10-08T05:37:43.732879+0000","last_active":"2023-10-08T05:37:43.736440+0000","last_peered":"2023-10-08T05:37:43.736440+0000","last_clean":"2023-10-08T05:37:43.736440+0000","last_became_active":"2023-10-08T05:37:43.732616+0000","last_became_peered":"2023-10-08T05:37:43.732616+0000","last_unstale":"2023-10-08T05:37:43.736440+0000","last_undegraded":"2023-10-08T05:37:43.736440+0000","last_fullsized":"2023-10-08T05:37:43.736440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:34.606944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.731047+0000","last_change":"2023-10-08T05:37:43.731047+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.731047+0000","last_undegraded":"2023-10-08T05:37:43.731047+0000","last_fullsized":"2023-10-08T05:37:43.731047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.743947+0000","last_change":"2023-10-08T05:37:43.733421+0000","last_active":"2023-10-08T05:37:43.743947+0000","last_peered":"2023-10-08T05:37:43.743947+0000","last_clean":"2023-10-08T05:37:43.743947+0000","last_became_active":"2023-10-08T05:37:43.732803+0000","last_became_peered":"2023-10-08T05:37:43.732803+0000","last_unstale":"2023-10-08T05:37:43.743947+0000","last_undegraded":"2023-10-08T05:37:43.743947+0000","last_fullsized":"2023-10-08T05:37:43.743947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:06.562486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.729623+0000","last_change":"2023-10-08T05:37:43.729623+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.729623+0000","last_undegraded":"2023-10-08T05:37:43.729623+0000","last_fullsized":"2023-10-08T05:37:43.729623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.747480+0000","last_change":"2023-10-08T05:37:43.736772+0000","last_active":"2023-10-08T05:37:43.747480+0000","last_peered":"2023-10-08T05:37:43.747480+0000","last_clean":"2023-10-08T05:37:43.747480+0000","last_became_active":"2023-10-08T05:37:43.736441+0000","last_became_peered":"2023-10-08T05:37:43.736441+0000","last_unstale":"2023-10-08T05:37:43.747480+0000","last_undegraded":"2023-10-08T05:37:43.747480+0000","last_fullsized":"2023-10-08T05:37:43.747480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:10.220125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737617+0000","last_change":"2023-10-08T05:37:43.737617+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737617+0000","last_undegraded":"2023-10-08T05:37:43.737617+0000","last_fullsized":"2023-10-08T05:37:43.737617+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.734888+0000","last_change":"2023-10-08T05:37:43.733011+0000","last_active":"2023-10-08T05:37:43.734888+0000","last_peered":"2023-10-08T05:37:43.734888+0000","last_clean":"2023-10-08T05:37:43.734888+0000","last_became_active":"2023-10-08T05:37:43.732744+0000","last_became_peered":"2023-10-08T05:37:43.732744+0000","last_unstale":"2023-10-08T05:37:43.734888+0000","last_undegraded":"2023-10-08T05:37:43.734888+0000","last_fullsized":"2023-10-08T05:37:43.734888+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:41.403273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.741498+0000","last_change":"2023-10-08T05:37:43.741498+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.741498+0000","last_undegraded":"2023-10-08T05:37:43.741498+0000","last_fullsized":"2023-10-08T05:37:43.741498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751517+0000","last_change":"2023-10-08T05:37:43.748040+0000","last_active":"2023-10-08T05:37:43.751517+0000","last_peered":"2023-10-08T05:37:43.751517+0000","last_clean":"2023-10-08T05:37:43.751517+0000","last_became_active":"2023-10-08T05:37:43.747745+0000","last_became_peered":"2023-10-08T05:37:43.747745+0000","last_unstale":"2023-10-08T05:37:43.751517+0000","last_undegraded":"2023-10-08T05:37:43.751517+0000","last_fullsized":"2023-10-08T05:37:43.751517+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":179,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-08T05:37:42.714859+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:37:42.714859+0000","last_peered":"2023-10-08T05:37:42.714859+0000","last_clean":"2023-10-08T05:37:42.714859+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:37:42.714859+0000","last_undegraded":"2023-10-08T05:37:42.714859+0000","last_fullsized":"2023-10-08T05:37:42.714859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":770492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.91000000000000003}]},{"osd":1,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.032}]},{"osd":2,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.145}]},{"osd":3,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.095},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":1.3600000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.319},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":1.637}]},{"osd":5,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.762}]},{"osd":6,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":3.5249999999999999,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.8340000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.7050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.4390000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.93100000000000005}]},{"osd":2,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.98499999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.341},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.512}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.379}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":4.3099999999999996,"5min":4.3099999999999996,"15min":4.3099999999999996},"last":1.415},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.175,"5min":1.175,"15min":1.175},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.569}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.23,"5min":1.23,"15min":1.23},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"max":{"1min":4.1550000000000002,"5min":4.1550000000000002,"15min":4.1550000000000002},"last":1.5449999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":3.5529999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.73299999999999998}]},{"osd":1,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":1.504},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.274}]},{"osd":2,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.208},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.89200000000000002}]},{"osd":3,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":6.782,"5min":6.782,"15min":6.782},"last":1.159},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"last":1.099}]},{"osd":4,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.1349999999999998,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.762,"5min":1.762,"15min":1.762},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":7.0069999999999997,"5min":7.0069999999999997,"15min":7.0069999999999997},"last":1.3740000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":7.2569999999999997,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":6.2199999999999998,"5min":6.2199999999999998,"15min":6.2199999999999998},"last":1.621}]},{"osd":7,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":6.9279999999999999,"5min":6.9279999999999999,"15min":6.9279999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":7.335,"5min":7.335,"15min":7.335},"last":1.5649999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.224}]},{"osd":1,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.88700000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.53},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.329}]},{"osd":3,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.3560000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.135},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.7290000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.478}]},{"osd":7,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.76,"5min":1.76,"15min":1.76},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.7809999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":180124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.75}]},{"osd":1,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.143}]},{"osd":2,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.2849999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.492}]},{"osd":5,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.536}]},{"osd":6,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.337},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.597}]},{"osd":7,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.468},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.5629999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":180124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.54400000000000004}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.23}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.278},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.002}]},{"osd":6,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":1.04}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.0529999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.101}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":173529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.0369999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.98799999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.94999999999999996}]},{"osd":4,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.28},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.2470000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.345}]},{"osd":6,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.498}]},{"osd":7,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.5249999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":763897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.032}]},{"osd":2,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.169},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.94199999999999995}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.82599999999999996}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":1.131},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.363}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.278},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.413}]},{"osd":6,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.0669999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.248}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:37:49.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:37:49.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.655+0000 7f11ba0cf640 1 Processor -- start 2023-10-08T05:37:49.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.656+0000 7f11ba0cf640 1 -- start start 2023-10-08T05:37:49.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.657+0000 7f11ba0cf640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 0x7f11b40ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.657+0000 7f11ba0cf640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11b40cb580 0x7f11b40adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.657+0000 7f11ba0cf640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b415b2b0 con 0x7f11b40c90b0 2023-10-08T05:37:49.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.657+0000 7f11ba0cf640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b415bf30 con 0x7f11b40cb580 2023-10-08T05:37:49.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.657+0000 7f11ba0cf640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b415f7a0 con 0x7f11b415ab40 2023-10-08T05:37:49.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.658+0000 7f11b3fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11b40cb580 0x7f11b40adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.658+0000 7f11b37fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 0x7f11b40ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.658+0000 7f11b3fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11b40cb580 0x7f11b40adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:34788/0 (socket says 172.21.15.70:34788) 2023-10-08T05:37:49.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.658+0000 7f11b3fff640 1 -- 172.21.15.70:0/679792313 learned_addr learned my addr 172.21.15.70:0/679792313 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:49.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.658+0000 7f11abfff640 1 --1- 172.21.15.70:0/679792313 >> v1:172.21.15.181:6789/0 conn(0x7f11b40c90b0 0x7f11b40cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:33070/0 (socket says 172.21.15.70:33070) 2023-10-08T05:37:49.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.659+0000 7f11b37fe640 1 -- 172.21.15.70:0/679792313 >> v2:172.21.15.181:3300/0 conn(0x7f11b40cb580 msgr2=0x7f11b40adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.659+0000 7f11b37fe640 1 --2- 172.21.15.70:0/679792313 >> v2:172.21.15.181:3300/0 conn(0x7f11b40cb580 0x7f11b40adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.659+0000 7f11b37fe640 1 -- 172.21.15.70:0/679792313 >> v1:172.21.15.181:6789/0 conn(0x7f11b40c90b0 legacy=0x7f11b40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.659+0000 7f11b37fe640 1 -- 172.21.15.70:0/679792313 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b415b590 con 0x7f11b415ab40 2023-10-08T05:37:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.660+0000 7f11b37fe640 1 --2- 172.21.15.70:0/679792313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 0x7f11b40ad720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f11a40097a0 tx=0x7f11a402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2b06b1acdd13fd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:49.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.660+0000 7f11b2ffd640 1 -- 172.21.15.70:0/679792313 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a402d040 con 0x7f11b415ab40 2023-10-08T05:37:49.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.661+0000 7f11b2ffd640 1 -- 172.21.15.70:0/679792313 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a4031810 con 0x7f11b415ab40 2023-10-08T05:37:49.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.661+0000 7f11b2ffd640 1 -- 172.21.15.70:0/679792313 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4031ba0 con 0x7f11b415ab40 2023-10-08T05:37:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.661+0000 7f11ba0cf640 1 -- 172.21.15.70:0/679792313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 msgr2=0x7f11b40ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.661+0000 7f11ba0cf640 1 --2- 172.21.15.70:0/679792313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 0x7f11b40ad720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f11a40097a0 tx=0x7f11a402b8b0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.661+0000 7f11ba0cf640 1 -- 172.21.15.70:0/679792313 shutdown_connections 2023-10-08T05:37:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.661+0000 7f11ba0cf640 1 --2- 172.21.15.70:0/679792313 >> v2:172.21.15.181:3300/0 conn(0x7f11b40cb580 0x7f11b40adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.661+0000 7f11ba0cf640 1 --2- 172.21.15.70:0/679792313 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 0x7f11b40ad720 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.661+0000 7f11ba0cf640 1 -- 172.21.15.70:0/679792313 >> 172.21.15.70:0/679792313 conn(0x7f11b406a380 msgr2=0x7f11b40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.661+0000 7f11ba0cf640 1 -- 172.21.15.70:0/679792313 shutdown_connections 2023-10-08T05:37:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.661+0000 7f11ba0cf640 1 -- 172.21.15.70:0/679792313 wait complete. 2023-10-08T05:37:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.663+0000 7f11ba0cf640 1 Processor -- start 2023-10-08T05:37:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.663+0000 7f11ba0cf640 1 -- start start 2023-10-08T05:37:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.664+0000 7f11ba0cf640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11b40c90b0 0x7f11b40b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.664+0000 7f11ba0cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b40cb580 0x7f11b40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.664+0000 7f11ba0cf640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 0x7f11b4156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.664+0000 7f11ba0cf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40b4410 con 0x7f11b40cb580 2023-10-08T05:37:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.664+0000 7f11ba0cf640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40c7f30 con 0x7f11b40c90b0 2023-10-08T05:37:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.664+0000 7f11ba0cf640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40c8210 con 0x7f11b415ab40 2023-10-08T05:37:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.665+0000 7f11abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b40cb580 0x7f11b40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.665+0000 7f11abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b40cb580 0x7f11b40b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37592/0 (socket says 172.21.15.70:37592) 2023-10-08T05:37:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.665+0000 7f11abfff640 1 -- 172.21.15.70:0/764487095 learned_addr learned my addr 172.21.15.70:0/764487095 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.665+0000 7f11b3fff640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 0x7f11b4156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.665+0000 7f11b37fe640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11b40c90b0 0x7f11b40b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.665+0000 7f11abfff640 1 -- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 msgr2=0x7f11b4156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.665+0000 7f11abfff640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 0x7f11b4156f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.665+0000 7f11abfff640 1 -- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11b40c90b0 msgr2=0x7f11b40b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.665+0000 7f11abfff640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11b40c90b0 0x7f11b40b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.665+0000 7f11abfff640 1 -- 172.21.15.70:0/764487095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4009400 con 0x7f11b40cb580 2023-10-08T05:37:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.666+0000 7f11abfff640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b40cb580 0x7f11b40b3ac0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f119c00c3e0 tx=0x7f119c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:49.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.666+0000 7f11b0ff9640 1 -- 172.21.15.70:0/764487095 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c0074e0 con 0x7f11b40cb580 2023-10-08T05:37:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.666+0000 7f11b0ff9640 1 -- 172.21.15.70:0/764487095 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c007640 con 0x7f11b40cb580 2023-10-08T05:37:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.666+0000 7f11ba0cf640 1 -- 172.21.15.70:0/764487095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b40c84f0 con 0x7f11b40cb580 2023-10-08T05:37:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.666+0000 7f11b0ff9640 1 -- 172.21.15.70:0/764487095 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c0049e0 con 0x7f11b40cb580 2023-10-08T05:37:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.666+0000 7f11ba0cf640 1 -- 172.21.15.70:0/764487095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b40b41e0 con 0x7f11b40cb580 2023-10-08T05:37:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.667+0000 7f11ba0cf640 1 -- 172.21.15.70:0/764487095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b40c86e0 con 0x7f11b40cb580 2023-10-08T05:37:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.668+0000 7f11b0ff9640 1 -- 172.21.15.70:0/764487095 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f119c004b40 con 0x7f11b40cb580 2023-10-08T05:37:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.669+0000 7f11b0ff9640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f119406ae70 0x7f119406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.669+0000 7f11b0ff9640 1 -- 172.21.15.70:0/764487095 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f119c089120 con 0x7f11b40cb580 2023-10-08T05:37:49.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.670+0000 7f11b37fe640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f119406ae70 0x7f119406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:49.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.671+0000 7f11b37fe640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f119406ae70 0x7f119406d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f11a4036000 tx=0x7f11a403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:49.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.674+0000 7f11b0ff9640 1 -- 172.21.15.70:0/764487095 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f119c054080 con 0x7f11b40cb580 2023-10-08T05:37:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.858+0000 7f11ba0cf640 1 -- 172.21.15.70:0/764487095 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11b415bb50 con 0x7f119406ae70 2023-10-08T05:37:49.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.881+0000 7f11b0ff9640 1 -- 172.21.15.70:0/764487095 <== mgr.4105 v2:172.21.15.181:6832/2737793492 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281813 (secure 0 0 0) 0x7f11a4044400 con 0x7f119406ae70 2023-10-08T05:37:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:49.887 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:37:49.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 -- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f119406ae70 msgr2=0x7f119406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f119406ae70 0x7f119406d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f11a4036000 tx=0x7f11a403c040 comp rx=0 tx=0).stop 2023-10-08T05:37:49.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 -- 172.21.15.70:0/764487095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b40cb580 msgr2=0x7f11b40b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:49.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b40cb580 0x7f11b40b3ac0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f119c00c3e0 tx=0x7f119c00c8a0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 -- 172.21.15.70:0/764487095 shutdown_connections 2023-10-08T05:37:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f119406ae70 0x7f119406d330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11b415ab40 0x7f11b4156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11b40cb580 0x7f11b40b3ac0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 --2- 172.21.15.70:0/764487095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11b40c90b0 0x7f11b40b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 -- 172.21.15.70:0/764487095 >> 172.21.15.70:0/764487095 conn(0x7f11b406a380 msgr2=0x7f11b40701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 -- 172.21.15.70:0/764487095 shutdown_connections 2023-10-08T05:37:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:49.886+0000 7f11ba0cf640 1 -- 172.21.15.70:0/764487095 wait complete. 2023-10-08T05:37:49.907 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2023-10-08T05:37:49.106292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237252,"kb_used_data":4440,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737468,"statfs":{"total":773094113280,"available":772851167232,"internally_reserved":0,"allocated":4546560,"data_stored":2595348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001704"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.726910+0000","last_change":"2023-10-08T05:37:43.726910+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.726910+0000","last_undegraded":"2023-10-08T05:37:43.726910+0000","last_fullsized":"2023-10-08T05:37:43.726910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.725861+0000","last_change":"2023-10-08T05:37:43.725861+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.725861+0000","last_undegraded":"2023-10-08T05:37:43.725861+0000","last_fullsized":"2023-10-08T05:37:43.725861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.722327+0000","last_change":"2023-10-08T05:37:43.722327+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.722327+0000","last_undegraded":"2023-10-08T05:37:43.722327+0000","last_fullsized":"2023-10-08T05:37:43.722327+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721430+0000","last_change":"2023-10-08T05:37:43.721430+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721430+0000","last_undegraded":"2023-10-08T05:37:43.721430+0000","last_fullsized":"2023-10-08T05:37:43.721430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723901+0000","last_change":"2023-10-08T05:37:43.723901+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723901+0000","last_undegraded":"2023-10-08T05:37:43.723901+0000","last_fullsized":"2023-10-08T05:37:43.723901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.727715+0000","last_change":"2023-10-08T05:37:43.727715+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.727715+0000","last_undegraded":"2023-10-08T05:37:43.727715+0000","last_fullsized":"2023-10-08T05:37:43.727715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.740518+0000","last_change":"2023-10-08T05:37:43.740518+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.740518+0000","last_undegraded":"2023-10-08T05:37:43.740518+0000","last_fullsized":"2023-10-08T05:37:43.740518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721832+0000","last_change":"2023-10-08T05:37:43.721832+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721832+0000","last_undegraded":"2023-10-08T05:37:43.721832+0000","last_fullsized":"2023-10-08T05:37:43.721832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724170+0000","last_change":"2023-10-08T05:37:43.724170+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724170+0000","last_undegraded":"2023-10-08T05:37:43.724170+0000","last_fullsized":"2023-10-08T05:37:43.724170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737987+0000","last_change":"2023-10-08T05:37:43.737987+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737987+0000","last_undegraded":"2023-10-08T05:37:43.737987+0000","last_fullsized":"2023-10-08T05:37:43.737987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.735624+0000","last_change":"2023-10-08T05:37:43.735624+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.735624+0000","last_undegraded":"2023-10-08T05:37:43.735624+0000","last_fullsized":"2023-10-08T05:37:43.735624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723176+0000","last_change":"2023-10-08T05:37:43.723176+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723176+0000","last_undegraded":"2023-10-08T05:37:43.723176+0000","last_fullsized":"2023-10-08T05:37:43.723176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.738432+0000","last_change":"2023-10-08T05:37:43.738432+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.738432+0000","last_undegraded":"2023-10-08T05:37:43.738432+0000","last_fullsized":"2023-10-08T05:37:43.738432+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.730462+0000","last_change":"2023-10-08T05:37:43.730462+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.730462+0000","last_undegraded":"2023-10-08T05:37:43.730462+0000","last_fullsized":"2023-10-08T05:37:43.730462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.740059+0000","last_change":"2023-10-08T05:37:43.740059+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.740059+0000","last_undegraded":"2023-10-08T05:37:43.740059+0000","last_fullsized":"2023-10-08T05:37:43.740059+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.730787+0000","last_change":"2023-10-08T05:37:43.730787+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.730787+0000","last_undegraded":"2023-10-08T05:37:43.730787+0000","last_fullsized":"2023-10-08T05:37:43.730787+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.738796+0000","last_change":"2023-10-08T05:37:43.738796+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.738796+0000","last_undegraded":"2023-10-08T05:37:43.738796+0000","last_fullsized":"2023-10-08T05:37:43.738796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723467+0000","last_change":"2023-10-08T05:37:43.723467+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723467+0000","last_undegraded":"2023-10-08T05:37:43.723467+0000","last_fullsized":"2023-10-08T05:37:43.723467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720545+0000","last_change":"2023-10-08T05:37:43.720545+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720545+0000","last_undegraded":"2023-10-08T05:37:43.720545+0000","last_fullsized":"2023-10-08T05:37:43.720545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.727938+0000","last_change":"2023-10-08T05:37:43.727938+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.727938+0000","last_undegraded":"2023-10-08T05:37:43.727938+0000","last_fullsized":"2023-10-08T05:37:43.727938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737934+0000","last_change":"2023-10-08T05:37:43.737934+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737934+0000","last_undegraded":"2023-10-08T05:37:43.737934+0000","last_fullsized":"2023-10-08T05:37:43.737934+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728439+0000","last_change":"2023-10-08T05:37:43.728439+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728439+0000","last_undegraded":"2023-10-08T05:37:43.728439+0000","last_fullsized":"2023-10-08T05:37:43.728439+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.736650+0000","last_change":"2023-10-08T05:37:43.736650+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.736650+0000","last_undegraded":"2023-10-08T05:37:43.736650+0000","last_fullsized":"2023-10-08T05:37:43.736650+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720791+0000","last_change":"2023-10-08T05:37:43.720791+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720791+0000","last_undegraded":"2023-10-08T05:37:43.720791+0000","last_fullsized":"2023-10-08T05:37:43.720791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.725014+0000","last_change":"2023-10-08T05:37:43.725014+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.725014+0000","last_undegraded":"2023-10-08T05:37:43.725014+0000","last_fullsized":"2023-10-08T05:37:43.725014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728973+0000","last_change":"2023-10-08T05:37:43.728973+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728973+0000","last_undegraded":"2023-10-08T05:37:43.728973+0000","last_fullsized":"2023-10-08T05:37:43.728973+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723172+0000","last_change":"2023-10-08T05:37:43.723172+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723172+0000","last_undegraded":"2023-10-08T05:37:43.723172+0000","last_fullsized":"2023-10-08T05:37:43.723172+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.740470+0000","last_change":"2023-10-08T05:37:43.740470+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.740470+0000","last_undegraded":"2023-10-08T05:37:43.740470+0000","last_fullsized":"2023-10-08T05:37:43.740470+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721405+0000","last_change":"2023-10-08T05:37:43.721405+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721405+0000","last_undegraded":"2023-10-08T05:37:43.721405+0000","last_fullsized":"2023-10-08T05:37:43.721405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.722018+0000","last_change":"2023-10-08T05:37:43.722018+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.722018+0000","last_undegraded":"2023-10-08T05:37:43.722018+0000","last_fullsized":"2023-10-08T05:37:43.722018+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.733264+0000","last_change":"2023-10-08T05:37:43.733264+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.733264+0000","last_undegraded":"2023-10-08T05:37:43.733264+0000","last_fullsized":"2023-10-08T05:37:43.733264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728215+0000","last_change":"2023-10-08T05:37:43.728215+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728215+0000","last_undegraded":"2023-10-08T05:37:43.728215+0000","last_fullsized":"2023-10-08T05:37:43.728215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.739149+0000","last_change":"2023-10-08T05:37:43.739149+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.739149+0000","last_undegraded":"2023-10-08T05:37:43.739149+0000","last_fullsized":"2023-10-08T05:37:43.739149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.749119+0000","last_change":"2023-10-08T05:37:43.746333+0000","last_active":"2023-10-08T05:37:43.749119+0000","last_peered":"2023-10-08T05:37:43.749119+0000","last_clean":"2023-10-08T05:37:43.749119+0000","last_became_active":"2023-10-08T05:37:43.746017+0000","last_became_peered":"2023-10-08T05:37:43.746017+0000","last_unstale":"2023-10-08T05:37:43.749119+0000","last_undegraded":"2023-10-08T05:37:43.749119+0000","last_fullsized":"2023-10-08T05:37:43.749119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:04:48.940111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.729258+0000","last_change":"2023-10-08T05:37:43.729258+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.729258+0000","last_undegraded":"2023-10-08T05:37:43.729258+0000","last_fullsized":"2023-10-08T05:37:43.729258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751007+0000","last_change":"2023-10-08T05:37:43.749778+0000","last_active":"2023-10-08T05:37:43.751007+0000","last_peered":"2023-10-08T05:37:43.751007+0000","last_clean":"2023-10-08T05:37:43.751007+0000","last_became_active":"2023-10-08T05:37:43.749435+0000","last_became_peered":"2023-10-08T05:37:43.749435+0000","last_unstale":"2023-10-08T05:37:43.751007+0000","last_undegraded":"2023-10-08T05:37:43.751007+0000","last_fullsized":"2023-10-08T05:37:43.751007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:35.696328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.726040+0000","last_change":"2023-10-08T05:37:43.726040+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.726040+0000","last_undegraded":"2023-10-08T05:37:43.726040+0000","last_fullsized":"2023-10-08T05:37:43.726040+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.750923+0000","last_change":"2023-10-08T05:37:43.749986+0000","last_active":"2023-10-08T05:37:43.750923+0000","last_peered":"2023-10-08T05:37:43.750923+0000","last_clean":"2023-10-08T05:37:43.750923+0000","last_became_active":"2023-10-08T05:37:43.749718+0000","last_became_peered":"2023-10-08T05:37:43.749718+0000","last_unstale":"2023-10-08T05:37:43.750923+0000","last_undegraded":"2023-10-08T05:37:43.750923+0000","last_fullsized":"2023-10-08T05:37:43.750923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:12.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721433+0000","last_change":"2023-10-08T05:37:43.721433+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721433+0000","last_undegraded":"2023-10-08T05:37:43.721433+0000","last_fullsized":"2023-10-08T05:37:43.721433+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.748417+0000","last_change":"2023-10-08T05:37:43.742613+0000","last_active":"2023-10-08T05:37:43.748417+0000","last_peered":"2023-10-08T05:37:43.748417+0000","last_clean":"2023-10-08T05:37:43.748417+0000","last_became_active":"2023-10-08T05:37:43.742232+0000","last_became_peered":"2023-10-08T05:37:43.742232+0000","last_unstale":"2023-10-08T05:37:43.748417+0000","last_undegraded":"2023-10-08T05:37:43.748417+0000","last_fullsized":"2023-10-08T05:37:43.748417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:35.696328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724604+0000","last_change":"2023-10-08T05:37:43.724604+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724604+0000","last_undegraded":"2023-10-08T05:37:43.724604+0000","last_fullsized":"2023-10-08T05:37:43.724604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752140+0000","last_change":"2023-10-08T05:37:43.743385+0000","last_active":"2023-10-08T05:37:43.752140+0000","last_peered":"2023-10-08T05:37:43.752140+0000","last_clean":"2023-10-08T05:37:43.752140+0000","last_became_active":"2023-10-08T05:37:43.743002+0000","last_became_peered":"2023-10-08T05:37:43.743002+0000","last_unstale":"2023-10-08T05:37:43.752140+0000","last_undegraded":"2023-10-08T05:37:43.752140+0000","last_fullsized":"2023-10-08T05:37:43.752140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:56:20.161604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.739571+0000","last_change":"2023-10-08T05:37:43.739571+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.739571+0000","last_undegraded":"2023-10-08T05:37:43.739571+0000","last_fullsized":"2023-10-08T05:37:43.739571+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.747959+0000","last_change":"2023-10-08T05:37:43.746646+0000","last_active":"2023-10-08T05:37:43.747959+0000","last_peered":"2023-10-08T05:37:43.747959+0000","last_clean":"2023-10-08T05:37:43.747959+0000","last_became_active":"2023-10-08T05:37:43.746369+0000","last_became_peered":"2023-10-08T05:37:43.746369+0000","last_unstale":"2023-10-08T05:37:43.747959+0000","last_undegraded":"2023-10-08T05:37:43.747959+0000","last_fullsized":"2023-10-08T05:37:43.747959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:12.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.725832+0000","last_change":"2023-10-08T05:37:43.725832+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.725832+0000","last_undegraded":"2023-10-08T05:37:43.725832+0000","last_fullsized":"2023-10-08T05:37:43.725832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.747537+0000","last_change":"2023-10-08T05:37:43.746978+0000","last_active":"2023-10-08T05:37:43.747537+0000","last_peered":"2023-10-08T05:37:43.747537+0000","last_clean":"2023-10-08T05:37:43.747537+0000","last_became_active":"2023-10-08T05:37:43.746728+0000","last_became_peered":"2023-10-08T05:37:43.746728+0000","last_unstale":"2023-10-08T05:37:43.747537+0000","last_undegraded":"2023-10-08T05:37:43.747537+0000","last_fullsized":"2023-10-08T05:37:43.747537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724140+0000","last_change":"2023-10-08T05:37:43.724140+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724140+0000","last_undegraded":"2023-10-08T05:37:43.724140+0000","last_fullsized":"2023-10-08T05:37:43.724140+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.745405+0000","last_change":"2023-10-08T05:37:43.736460+0000","last_active":"2023-10-08T05:37:43.745405+0000","last_peered":"2023-10-08T05:37:43.745405+0000","last_clean":"2023-10-08T05:37:43.745405+0000","last_became_active":"2023-10-08T05:37:43.736033+0000","last_became_peered":"2023-10-08T05:37:43.736033+0000","last_unstale":"2023-10-08T05:37:43.745405+0000","last_undegraded":"2023-10-08T05:37:43.745405+0000","last_fullsized":"2023-10-08T05:37:43.745405+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.017005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720387+0000","last_change":"2023-10-08T05:37:43.720387+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720387+0000","last_undegraded":"2023-10-08T05:37:43.720387+0000","last_fullsized":"2023-10-08T05:37:43.720387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.737892+0000","last_change":"2023-10-08T05:37:43.735761+0000","last_active":"2023-10-08T05:37:43.737892+0000","last_peered":"2023-10-08T05:37:43.737892+0000","last_clean":"2023-10-08T05:37:43.737892+0000","last_became_active":"2023-10-08T05:37:43.735391+0000","last_became_peered":"2023-10-08T05:37:43.735391+0000","last_unstale":"2023-10-08T05:37:43.737892+0000","last_undegraded":"2023-10-08T05:37:43.737892+0000","last_fullsized":"2023-10-08T05:37:43.737892+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:26.955372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.734924+0000","last_change":"2023-10-08T05:37:43.734924+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.734924+0000","last_undegraded":"2023-10-08T05:37:43.734924+0000","last_fullsized":"2023-10-08T05:37:43.734924+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751246+0000","last_change":"2023-10-08T05:37:43.742980+0000","last_active":"2023-10-08T05:37:43.751246+0000","last_peered":"2023-10-08T05:37:43.751246+0000","last_clean":"2023-10-08T05:37:43.751246+0000","last_became_active":"2023-10-08T05:37:43.742358+0000","last_became_peered":"2023-10-08T05:37:43.742358+0000","last_unstale":"2023-10-08T05:37:43.751246+0000","last_undegraded":"2023-10-08T05:37:43.751246+0000","last_fullsized":"2023-10-08T05:37:43.751246+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:53.615714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721234+0000","last_change":"2023-10-08T05:37:43.721234+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721234+0000","last_undegraded":"2023-10-08T05:37:43.721234+0000","last_fullsized":"2023-10-08T05:37:43.721234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752211+0000","last_change":"2023-10-08T05:37:43.743414+0000","last_active":"2023-10-08T05:37:43.752211+0000","last_peered":"2023-10-08T05:37:43.752211+0000","last_clean":"2023-10-08T05:37:43.752211+0000","last_became_active":"2023-10-08T05:37:43.742873+0000","last_became_peered":"2023-10-08T05:37:43.742873+0000","last_unstale":"2023-10-08T05:37:43.752211+0000","last_undegraded":"2023-10-08T05:37:43.752211+0000","last_fullsized":"2023-10-08T05:37:43.752211+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724763+0000","last_change":"2023-10-08T05:37:43.724763+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724763+0000","last_undegraded":"2023-10-08T05:37:43.724763+0000","last_fullsized":"2023-10-08T05:37:43.724763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.750862+0000","last_change":"2023-10-08T05:37:43.749228+0000","last_active":"2023-10-08T05:37:43.750862+0000","last_peered":"2023-10-08T05:37:43.750862+0000","last_clean":"2023-10-08T05:37:43.750862+0000","last_became_active":"2023-10-08T05:37:43.748703+0000","last_became_peered":"2023-10-08T05:37:43.748703+0000","last_unstale":"2023-10-08T05:37:43.750862+0000","last_undegraded":"2023-10-08T05:37:43.750862+0000","last_fullsized":"2023-10-08T05:37:43.750862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:11:28.561634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724118+0000","last_change":"2023-10-08T05:37:43.724118+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724118+0000","last_undegraded":"2023-10-08T05:37:43.724118+0000","last_fullsized":"2023-10-08T05:37:43.724118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.730582+0000","last_change":"2023-10-08T05:37:43.729198+0000","last_active":"2023-10-08T05:37:43.730582+0000","last_peered":"2023-10-08T05:37:43.730582+0000","last_clean":"2023-10-08T05:37:43.730582+0000","last_became_active":"2023-10-08T05:37:43.728918+0000","last_became_peered":"2023-10-08T05:37:43.728918+0000","last_unstale":"2023-10-08T05:37:43.730582+0000","last_undegraded":"2023-10-08T05:37:43.730582+0000","last_fullsized":"2023-10-08T05:37:43.730582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720385+0000","last_change":"2023-10-08T05:37:43.720385+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720385+0000","last_undegraded":"2023-10-08T05:37:43.720385+0000","last_fullsized":"2023-10-08T05:37:43.720385+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746846+0000","last_change":"2023-10-08T05:37:43.745271+0000","last_active":"2023-10-08T05:37:43.746846+0000","last_peered":"2023-10-08T05:37:43.746846+0000","last_clean":"2023-10-08T05:37:43.746846+0000","last_became_active":"2023-10-08T05:37:43.745043+0000","last_became_peered":"2023-10-08T05:37:43.745043+0000","last_unstale":"2023-10-08T05:37:43.746846+0000","last_undegraded":"2023-10-08T05:37:43.746846+0000","last_fullsized":"2023-10-08T05:37:43.746846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:34.606944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737702+0000","last_change":"2023-10-08T05:37:43.737702+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737702+0000","last_undegraded":"2023-10-08T05:37:43.737702+0000","last_fullsized":"2023-10-08T05:37:43.737702+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.749151+0000","last_change":"2023-10-08T05:37:43.746332+0000","last_active":"2023-10-08T05:37:43.749151+0000","last_peered":"2023-10-08T05:37:43.749151+0000","last_clean":"2023-10-08T05:37:43.749151+0000","last_became_active":"2023-10-08T05:37:43.746017+0000","last_became_peered":"2023-10-08T05:37:43.746017+0000","last_unstale":"2023-10-08T05:37:43.749151+0000","last_undegraded":"2023-10-08T05:37:43.749151+0000","last_fullsized":"2023-10-08T05:37:43.749151+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:10.220125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.722997+0000","last_change":"2023-10-08T05:37:43.722997+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.722997+0000","last_undegraded":"2023-10-08T05:37:43.722997+0000","last_fullsized":"2023-10-08T05:37:43.722997+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752512+0000","last_change":"2023-10-08T05:37:43.750157+0000","last_active":"2023-10-08T05:37:43.752512+0000","last_peered":"2023-10-08T05:37:43.752512+0000","last_clean":"2023-10-08T05:37:43.752512+0000","last_became_active":"2023-10-08T05:37:43.749846+0000","last_became_peered":"2023-10-08T05:37:43.749846+0000","last_unstale":"2023-10-08T05:37:43.752512+0000","last_undegraded":"2023-10-08T05:37:43.752512+0000","last_fullsized":"2023-10-08T05:37:43.752512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:53.615714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.719971+0000","last_change":"2023-10-08T05:37:43.719971+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.719971+0000","last_undegraded":"2023-10-08T05:37:43.719971+0000","last_fullsized":"2023-10-08T05:37:43.719971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.753906+0000","last_change":"2023-10-08T05:37:43.752005+0000","last_active":"2023-10-08T05:37:43.753906+0000","last_peered":"2023-10-08T05:37:43.753906+0000","last_clean":"2023-10-08T05:37:43.753906+0000","last_became_active":"2023-10-08T05:37:43.751525+0000","last_became_peered":"2023-10-08T05:37:43.751525+0000","last_unstale":"2023-10-08T05:37:43.753906+0000","last_undegraded":"2023-10-08T05:37:43.753906+0000","last_fullsized":"2023-10-08T05:37:43.753906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.734950+0000","last_change":"2023-10-08T05:37:43.734950+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.734950+0000","last_undegraded":"2023-10-08T05:37:43.734950+0000","last_fullsized":"2023-10-08T05:37:43.734950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.749409+0000","last_change":"2023-10-08T05:37:43.745487+0000","last_active":"2023-10-08T05:37:43.749409+0000","last_peered":"2023-10-08T05:37:43.749409+0000","last_clean":"2023-10-08T05:37:43.749409+0000","last_became_active":"2023-10-08T05:37:43.744512+0000","last_became_peered":"2023-10-08T05:37:43.744512+0000","last_unstale":"2023-10-08T05:37:43.749409+0000","last_undegraded":"2023-10-08T05:37:43.749409+0000","last_fullsized":"2023-10-08T05:37:43.749409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:10.719966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.727258+0000","last_change":"2023-10-08T05:37:43.727258+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.727258+0000","last_undegraded":"2023-10-08T05:37:43.727258+0000","last_fullsized":"2023-10-08T05:37:43.727258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751126+0000","last_change":"2023-10-08T05:37:43.748895+0000","last_active":"2023-10-08T05:37:43.751126+0000","last_peered":"2023-10-08T05:37:43.751126+0000","last_clean":"2023-10-08T05:37:43.751126+0000","last_became_active":"2023-10-08T05:37:43.748618+0000","last_became_peered":"2023-10-08T05:37:43.748618+0000","last_unstale":"2023-10-08T05:37:43.751126+0000","last_undegraded":"2023-10-08T05:37:43.751126+0000","last_fullsized":"2023-10-08T05:37:43.751126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:35.801089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.719846+0000","last_change":"2023-10-08T05:37:43.719846+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.719846+0000","last_undegraded":"2023-10-08T05:37:43.719846+0000","last_fullsized":"2023-10-08T05:37:43.719846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746320+0000","last_change":"2023-10-08T05:37:43.743970+0000","last_active":"2023-10-08T05:37:43.746320+0000","last_peered":"2023-10-08T05:37:43.746320+0000","last_clean":"2023-10-08T05:37:43.746320+0000","last_became_active":"2023-10-08T05:37:43.743106+0000","last_became_peered":"2023-10-08T05:37:43.743106+0000","last_unstale":"2023-10-08T05:37:43.746320+0000","last_undegraded":"2023-10-08T05:37:43.746320+0000","last_fullsized":"2023-10-08T05:37:43.746320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:27.563368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.729757+0000","last_change":"2023-10-08T05:37:43.729757+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.729757+0000","last_undegraded":"2023-10-08T05:37:43.729757+0000","last_fullsized":"2023-10-08T05:37:43.729757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.731932+0000","last_change":"2023-10-08T05:37:43.728582+0000","last_active":"2023-10-08T05:37:43.731932+0000","last_peered":"2023-10-08T05:37:43.731932+0000","last_clean":"2023-10-08T05:37:43.731932+0000","last_became_active":"2023-10-08T05:37:43.728172+0000","last_became_peered":"2023-10-08T05:37:43.728172+0000","last_unstale":"2023-10-08T05:37:43.731932+0000","last_undegraded":"2023-10-08T05:37:43.731932+0000","last_fullsized":"2023-10-08T05:37:43.731932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:34.248097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721280+0000","last_change":"2023-10-08T05:37:43.721280+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721280+0000","last_undegraded":"2023-10-08T05:37:43.721280+0000","last_fullsized":"2023-10-08T05:37:43.721280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746296+0000","last_change":"2023-10-08T05:37:43.744243+0000","last_active":"2023-10-08T05:37:43.746296+0000","last_peered":"2023-10-08T05:37:43.746296+0000","last_clean":"2023-10-08T05:37:43.746296+0000","last_became_active":"2023-10-08T05:37:43.743356+0000","last_became_peered":"2023-10-08T05:37:43.743356+0000","last_unstale":"2023-10-08T05:37:43.746296+0000","last_undegraded":"2023-10-08T05:37:43.746296+0000","last_fullsized":"2023-10-08T05:37:43.746296+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:26:44.883068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.735316+0000","last_change":"2023-10-08T05:37:43.735316+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.735316+0000","last_undegraded":"2023-10-08T05:37:43.735316+0000","last_fullsized":"2023-10-08T05:37:43.735316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752420+0000","last_change":"2023-10-08T05:37:43.749749+0000","last_active":"2023-10-08T05:37:43.752420+0000","last_peered":"2023-10-08T05:37:43.752420+0000","last_clean":"2023-10-08T05:37:43.752420+0000","last_became_active":"2023-10-08T05:37:43.749413+0000","last_became_peered":"2023-10-08T05:37:43.749413+0000","last_unstale":"2023-10-08T05:37:43.752420+0000","last_undegraded":"2023-10-08T05:37:43.752420+0000","last_fullsized":"2023-10-08T05:37:43.752420+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:27.563368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.719803+0000","last_change":"2023-10-08T05:37:43.719803+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.719803+0000","last_undegraded":"2023-10-08T05:37:43.719803+0000","last_fullsized":"2023-10-08T05:37:43.719803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.734537+0000","last_change":"2023-10-08T05:37:43.733311+0000","last_active":"2023-10-08T05:37:43.734537+0000","last_peered":"2023-10-08T05:37:43.734537+0000","last_clean":"2023-10-08T05:37:43.734537+0000","last_became_active":"2023-10-08T05:37:43.733051+0000","last_became_peered":"2023-10-08T05:37:43.733051+0000","last_unstale":"2023-10-08T05:37:43.734537+0000","last_undegraded":"2023-10-08T05:37:43.734537+0000","last_fullsized":"2023-10-08T05:37:43.734537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:02.738727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728203+0000","last_change":"2023-10-08T05:37:43.728203+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728203+0000","last_undegraded":"2023-10-08T05:37:43.728203+0000","last_fullsized":"2023-10-08T05:37:43.728203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752422+0000","last_change":"2023-10-08T05:37:43.749129+0000","last_active":"2023-10-08T05:37:43.752422+0000","last_peered":"2023-10-08T05:37:43.752422+0000","last_clean":"2023-10-08T05:37:43.752422+0000","last_became_active":"2023-10-08T05:37:43.748832+0000","last_became_peered":"2023-10-08T05:37:43.748832+0000","last_unstale":"2023-10-08T05:37:43.752422+0000","last_undegraded":"2023-10-08T05:37:43.752422+0000","last_fullsized":"2023-10-08T05:37:43.752422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:41.403273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.733504+0000","last_change":"2023-10-08T05:37:43.733504+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.733504+0000","last_undegraded":"2023-10-08T05:37:43.733504+0000","last_fullsized":"2023-10-08T05:37:43.733504+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746378+0000","last_change":"2023-10-08T05:37:43.744435+0000","last_active":"2023-10-08T05:37:43.746378+0000","last_peered":"2023-10-08T05:37:43.746378+0000","last_clean":"2023-10-08T05:37:43.746378+0000","last_became_active":"2023-10-08T05:37:43.744157+0000","last_became_peered":"2023-10-08T05:37:43.744157+0000","last_unstale":"2023-10-08T05:37:43.746378+0000","last_undegraded":"2023-10-08T05:37:43.746378+0000","last_fullsized":"2023-10-08T05:37:43.746378+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:21:09.150233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720525+0000","last_change":"2023-10-08T05:37:43.720525+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720525+0000","last_undegraded":"2023-10-08T05:37:43.720525+0000","last_fullsized":"2023-10-08T05:37:43.720525+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.736500+0000","last_change":"2023-10-08T05:37:43.733742+0000","last_active":"2023-10-08T05:37:43.736500+0000","last_peered":"2023-10-08T05:37:43.736500+0000","last_clean":"2023-10-08T05:37:43.736500+0000","last_became_active":"2023-10-08T05:37:43.733486+0000","last_became_peered":"2023-10-08T05:37:43.733486+0000","last_unstale":"2023-10-08T05:37:43.736500+0000","last_undegraded":"2023-10-08T05:37:43.736500+0000","last_fullsized":"2023-10-08T05:37:43.736500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:10.719966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723702+0000","last_change":"2023-10-08T05:37:43.723702+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723702+0000","last_undegraded":"2023-10-08T05:37:43.723702+0000","last_fullsized":"2023-10-08T05:37:43.723702+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.736440+0000","last_change":"2023-10-08T05:37:43.732879+0000","last_active":"2023-10-08T05:37:43.736440+0000","last_peered":"2023-10-08T05:37:43.736440+0000","last_clean":"2023-10-08T05:37:43.736440+0000","last_became_active":"2023-10-08T05:37:43.732616+0000","last_became_peered":"2023-10-08T05:37:43.732616+0000","last_unstale":"2023-10-08T05:37:43.736440+0000","last_undegraded":"2023-10-08T05:37:43.736440+0000","last_fullsized":"2023-10-08T05:37:43.736440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:34.606944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.731047+0000","last_change":"2023-10-08T05:37:43.731047+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.731047+0000","last_undegraded":"2023-10-08T05:37:43.731047+0000","last_fullsized":"2023-10-08T05:37:43.731047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.743947+0000","last_change":"2023-10-08T05:37:43.733421+0000","last_active":"2023-10-08T05:37:43.743947+0000","last_peered":"2023-10-08T05:37:43.743947+0000","last_clean":"2023-10-08T05:37:43.743947+0000","last_became_active":"2023-10-08T05:37:43.732803+0000","last_became_peered":"2023-10-08T05:37:43.732803+0000","last_unstale":"2023-10-08T05:37:43.743947+0000","last_undegraded":"2023-10-08T05:37:43.743947+0000","last_fullsized":"2023-10-08T05:37:43.743947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:06.562486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.729623+0000","last_change":"2023-10-08T05:37:43.729623+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.729623+0000","last_undegraded":"2023-10-08T05:37:43.729623+0000","last_fullsized":"2023-10-08T05:37:43.729623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.747480+0000","last_change":"2023-10-08T05:37:43.736772+0000","last_active":"2023-10-08T05:37:43.747480+0000","last_peered":"2023-10-08T05:37:43.747480+0000","last_clean":"2023-10-08T05:37:43.747480+0000","last_became_active":"2023-10-08T05:37:43.736441+0000","last_became_peered":"2023-10-08T05:37:43.736441+0000","last_unstale":"2023-10-08T05:37:43.747480+0000","last_undegraded":"2023-10-08T05:37:43.747480+0000","last_fullsized":"2023-10-08T05:37:43.747480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:10.220125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737617+0000","last_change":"2023-10-08T05:37:43.737617+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737617+0000","last_undegraded":"2023-10-08T05:37:43.737617+0000","last_fullsized":"2023-10-08T05:37:43.737617+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.734888+0000","last_change":"2023-10-08T05:37:43.733011+0000","last_active":"2023-10-08T05:37:43.734888+0000","last_peered":"2023-10-08T05:37:43.734888+0000","last_clean":"2023-10-08T05:37:43.734888+0000","last_became_active":"2023-10-08T05:37:43.732744+0000","last_became_peered":"2023-10-08T05:37:43.732744+0000","last_unstale":"2023-10-08T05:37:43.734888+0000","last_undegraded":"2023-10-08T05:37:43.734888+0000","last_fullsized":"2023-10-08T05:37:43.734888+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:41.403273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.741498+0000","last_change":"2023-10-08T05:37:43.741498+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.741498+0000","last_undegraded":"2023-10-08T05:37:43.741498+0000","last_fullsized":"2023-10-08T05:37:43.741498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751517+0000","last_change":"2023-10-08T05:37:43.748040+0000","last_active":"2023-10-08T05:37:43.751517+0000","last_peered":"2023-10-08T05:37:43.751517+0000","last_clean":"2023-10-08T05:37:43.751517+0000","last_became_active":"2023-10-08T05:37:43.747745+0000","last_became_peered":"2023-10-08T05:37:43.747745+0000","last_unstale":"2023-10-08T05:37:43.751517+0000","last_undegraded":"2023-10-08T05:37:43.751517+0000","last_fullsized":"2023-10-08T05:37:43.751517+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":179,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-08T05:37:42.714859+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:37:42.714859+0000","last_peered":"2023-10-08T05:37:42.714859+0000","last_clean":"2023-10-08T05:37:42.714859+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:37:42.714859+0000","last_undegraded":"2023-10-08T05:37:42.714859+0000","last_fullsized":"2023-10-08T05:37:42.714859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":770492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.91000000000000003}]},{"osd":1,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.032}]},{"osd":2,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.145}]},{"osd":3,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.095},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":1.3600000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.319},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":1.637}]},{"osd":5,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.762}]},{"osd":6,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":3.5249999999999999,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.8340000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.7050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.4390000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.93100000000000005}]},{"osd":2,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.98499999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.341},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.512}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.379}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":4.3099999999999996,"5min":4.3099999999999996,"15min":4.3099999999999996},"last":1.415},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.175,"5min":1.175,"15min":1.175},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.569}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.23,"5min":1.23,"15min":1.23},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"max":{"1min":4.1550000000000002,"5min":4.1550000000000002,"15min":4.1550000000000002},"last":1.5449999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":3.5529999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.73299999999999998}]},{"osd":1,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":1.504},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.274}]},{"osd":2,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.208},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.89200000000000002}]},{"osd":3,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":6.782,"5min":6.782,"15min":6.782},"last":1.159},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"last":1.099}]},{"osd":4,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.1349999999999998,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.762,"5min":1.762,"15min":1.762},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":7.0069999999999997,"5min":7.0069999999999997,"15min":7.0069999999999997},"last":1.3740000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":7.2569999999999997,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":6.2199999999999998,"5min":6.2199999999999998,"15min":6.2199999999999998},"last":1.621}]},{"osd":7,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":6.9279999999999999,"5min":6.9279999999999999,"15min":6.9279999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":7.335,"5min":7.335,"15min":7.335},"last":1.5649999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.224}]},{"osd":1,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.88700000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.53},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.329}]},{"osd":3,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.3560000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.135},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.7290000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.478}]},{"osd":7,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.76,"5min":1.76,"15min":1.76},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.7809999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":180124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.75}]},{"osd":1,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.143}]},{"osd":2,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.2849999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.492}]},{"osd":5,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.536}]},{"osd":6,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.337},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.597}]},{"osd":7,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.468},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.5629999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":180124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.54400000000000004}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.23}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.278},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.002}]},{"osd":6,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":1.04}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.0529999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.101}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":173529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.0369999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.98799999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.94999999999999996}]},{"osd":4,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.28},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.2470000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.345}]},{"osd":6,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.498}]},{"osd":7,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.5249999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":763897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.032}]},{"osd":2,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.169},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.94199999999999995}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.82599999999999996}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":1.131},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.363}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.278},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.413}]},{"osd":6,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.0669999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.248}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:37:49.914 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:37:50.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.169+0000 7f60131f5640 1 Processor -- start 2023-10-08T05:37:50.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.169+0000 7f60131f5640 1 -- start start 2023-10-08T05:37:50.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.171+0000 7f60131f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15ab40 0x7f600c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.171+0000 7f60131f5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f600c0c90b0 0x7f600c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.171+0000 7f60131f5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600c15b2b0 con 0x7f600c0cae70 2023-10-08T05:37:50.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.171+0000 7f60131f5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c15bf30 con 0x7f600c0c90b0 2023-10-08T05:37:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.171+0000 7f60131f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c15f7a0 con 0x7f600c15ab40 2023-10-08T05:37:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.171+0000 7f6011792640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f600c0cae70 0x7f600c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59964/0 (socket says 172.21.15.70:59964) 2023-10-08T05:37:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.171+0000 7f6011792640 1 -- 172.21.15.70:0/35457534 learned_addr learned my addr 172.21.15.70:0/35457534 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.171+0000 7f600bfff640 1 --2- 172.21.15.70:0/35457534 >> v2:172.21.15.181:3300/0 conn(0x7f600c0c90b0 0x7f600c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f6010f91640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15ab40 0x7f600c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566932260 0 0) 0x7f600c15b2b0 con 0x7f600c0cae70 2023-10-08T05:37:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ffc003520 con 0x7f600c0cae70 2023-10-08T05:37:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1704902747 0 0) 0x7f5ffc003520 con 0x7f600c0cae70 2023-10-08T05:37:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15ab40 msgr2=0x7f600c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15ab40 0x7f600c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 >> v2:172.21.15.181:3300/0 conn(0x7f600c0c90b0 msgr2=0x7f600c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 --2- 172.21.15.70:0/35457534 >> v2:172.21.15.181:3300/0 conn(0x7f600c0c90b0 0x7f600c0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c15b590 con 0x7f600c0cae70 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f60040019f0 con 0x7f600c0cae70 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 >> v1:172.21.15.70:6789/0 conn(0x7f600c0cae70 legacy=0x7f600c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ffc003970 0x7f5ffc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 0x7f5ffc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15ab40 0x7f5ffc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c15bf30 con 0x7f5ffc0047c0 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c15f7a0 con 0x7f5ffc003970 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.172+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc00b980 con 0x7f600c15ab40 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.173+0000 7f600bfff640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 0x7f5ffc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.173+0000 7f6010f91640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ffc003970 0x7f5ffc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.173+0000 7f6011792640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15ab40 0x7f5ffc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.173+0000 7f600bfff640 1 -- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15ab40 msgr2=0x7f5ffc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.173+0000 7f600bfff640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15ab40 0x7f5ffc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.173+0000 7f600bfff640 1 -- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ffc003970 msgr2=0x7f5ffc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.173+0000 7f600bfff640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ffc003970 0x7f5ffc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.173+0000 7f600bfff640 1 -- 172.21.15.70:0/35457534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc003520 con 0x7f5ffc0047c0 2023-10-08T05:37:50.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.174+0000 7f600bfff640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 0x7f5ffc004c40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009ac0 tx=0x7f5ffc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c64aeee5ab5197 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:50.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.174+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff802c710 con 0x7f5ffc0047c0 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.174+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff802c870 con 0x7f5ffc0047c0 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.174+0000 7f600b7fe640 1 -- 172.21.15.70:0/35457534 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff802cc00 con 0x7f5ffc0047c0 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.175+0000 7f60131f5640 1 -- 172.21.15.70:0/35457534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 msgr2=0x7f5ffc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.175+0000 7f60131f5640 1 --2- 172.21.15.70:0/35457534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 0x7f5ffc004c40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009ac0 tx=0x7f5ffc0057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.175+0000 7f6011792640 1 -- 172.21.15.70:0/35457534 reap_dead start 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.175+0000 7f60131f5640 1 -- 172.21.15.70:0/35457534 shutdown_connections 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.175+0000 7f60131f5640 1 -- 172.21.15.70:0/35457534 >> 172.21.15.70:0/35457534 conn(0x7f600c06a380 msgr2=0x7f600c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.175+0000 7f60131f5640 1 -- 172.21.15.70:0/35457534 shutdown_connections 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.175+0000 7f60131f5640 1 -- 172.21.15.70:0/35457534 wait complete. 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.177+0000 7f60131f5640 1 Processor -- start 2023-10-08T05:37:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.177+0000 7f60131f5640 1 -- start start 2023-10-08T05:37:50.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.178+0000 7f60131f5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 0x7f600c152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.178+0000 7f60131f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15dda0 0x7f600c0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.178+0000 7f60131f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f600c0b13d0 0x7f600c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.178+0000 7f60131f5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0b4460 con 0x7f5ffc0047c0 2023-10-08T05:37:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.178+0000 7f60131f5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c1527b0 con 0x7f600c0b13d0 2023-10-08T05:37:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.178+0000 7f60131f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0b1990 con 0x7f600c15dda0 2023-10-08T05:37:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6010f91640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 0x7f600c152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6010f91640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 0x7f600c152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37616/0 (socket says 172.21.15.70:37616) 2023-10-08T05:37:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6010f91640 1 -- 172.21.15.70:0/1340641267 learned_addr learned my addr 172.21.15.70:0/1340641267 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f600bfff640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15dda0 0x7f600c0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6011792640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f600c0b13d0 0x7f600c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6010f91640 1 -- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15dda0 msgr2=0x7f600c0b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6010f91640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15dda0 0x7f600c0b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6010f91640 1 -- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f600c0b13d0 msgr2=0x7f600c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6010f91640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f600c0b13d0 0x7f600c0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6010f91640 1 -- 172.21.15.70:0/1340641267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0b1c70 con 0x7f5ffc0047c0 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f600bfff640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15dda0 0x7f600c0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6011792640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f600c0b13d0 0x7f600c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.179+0000 7f6010f91640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 0x7f600c152270 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6000011600 tx=0x7f6000011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.180+0000 7f60097fa640 1 -- 172.21.15.70:0/1340641267 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600000f920 con 0x7f5ffc0047c0 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.180+0000 7f60097fa640 1 -- 172.21.15.70:0/1340641267 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600000fa80 con 0x7f5ffc0047c0 2023-10-08T05:37:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.180+0000 7f60097fa640 1 -- 172.21.15.70:0/1340641267 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600000fdf0 con 0x7f5ffc0047c0 2023-10-08T05:37:50.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.180+0000 7f60131f5640 1 -- 172.21.15.70:0/1340641267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c0b1f50 con 0x7f5ffc0047c0 2023-10-08T05:37:50.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.180+0000 7f60131f5640 1 -- 172.21.15.70:0/1340641267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c0b2510 con 0x7f5ffc0047c0 2023-10-08T05:37:50.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.181+0000 7f60131f5640 1 -- 172.21.15.70:0/1340641267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c067490 con 0x7f5ffc0047c0 2023-10-08T05:37:50.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.182+0000 7f60097fa640 1 -- 172.21.15.70:0/1340641267 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f6000015460 con 0x7f5ffc0047c0 2023-10-08T05:37:50.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.183+0000 7f60097fa640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ff006aea0 0x7f5ff006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.183+0000 7f60097fa640 1 -- 172.21.15.70:0/1340641267 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6000089b20 con 0x7f5ffc0047c0 2023-10-08T05:37:50.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.183+0000 7f600bfff640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ff006aea0 0x7f5ff006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.185+0000 7f600bfff640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ff006aea0 0x7f5ff006d360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8036000 tx=0x7f5ff8030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:50.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.187+0000 7f60097fa640 1 -- 172.21.15.70:0/1340641267 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6000054a80 con 0x7f5ffc0047c0 2023-10-08T05:37:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.371+0000 7f60131f5640 1 -- 172.21.15.70:0/1340641267 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f600c15b870 con 0x7f5ff006aea0 2023-10-08T05:37:50.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.393+0000 7f60097fa640 1 -- 172.21.15.70:0/1340641267 <== mgr.4105 v2:172.21.15.181:6832/2737793492 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281813 (secure 0 0 0) 0x7f5ff802cc00 con 0x7f5ff006aea0 2023-10-08T05:37:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:50.399 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:37:50.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.397+0000 7f60131f5640 1 -- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ff006aea0 msgr2=0x7f5ff006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.397+0000 7f60131f5640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ff006aea0 0x7f5ff006d360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8036000 tx=0x7f5ff8030000 comp rx=0 tx=0).stop 2023-10-08T05:37:50.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.397+0000 7f60131f5640 1 -- 172.21.15.70:0/1340641267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 msgr2=0x7f600c152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.397+0000 7f60131f5640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 0x7f600c152270 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6000011600 tx=0x7f6000011ac0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.398+0000 7f60131f5640 1 -- 172.21.15.70:0/1340641267 shutdown_connections 2023-10-08T05:37:50.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.398+0000 7f60131f5640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f5ff006aea0 0x7f5ff006d360 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.398+0000 7f60131f5640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f600c0b13d0 0x7f600c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.398+0000 7f60131f5640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f600c15dda0 0x7f600c0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.398+0000 7f60131f5640 1 --2- 172.21.15.70:0/1340641267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ffc0047c0 0x7f600c152270 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.398+0000 7f60131f5640 1 -- 172.21.15.70:0/1340641267 >> 172.21.15.70:0/1340641267 conn(0x7f600c06a380 msgr2=0x7f600c0ca100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:50.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.398+0000 7f60131f5640 1 -- 172.21.15.70:0/1340641267 shutdown_connections 2023-10-08T05:37:50.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.398+0000 7f60131f5640 1 -- 172.21.15.70:0/1340641267 wait complete. 2023-10-08T05:37:50.419 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2023-10-08T05:37:49.106292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237252,"kb_used_data":4440,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737468,"statfs":{"total":773094113280,"available":772851167232,"internally_reserved":0,"allocated":4546560,"data_stored":2595348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001704"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.726910+0000","last_change":"2023-10-08T05:37:43.726910+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.726910+0000","last_undegraded":"2023-10-08T05:37:43.726910+0000","last_fullsized":"2023-10-08T05:37:43.726910+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.725861+0000","last_change":"2023-10-08T05:37:43.725861+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.725861+0000","last_undegraded":"2023-10-08T05:37:43.725861+0000","last_fullsized":"2023-10-08T05:37:43.725861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.722327+0000","last_change":"2023-10-08T05:37:43.722327+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.722327+0000","last_undegraded":"2023-10-08T05:37:43.722327+0000","last_fullsized":"2023-10-08T05:37:43.722327+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721430+0000","last_change":"2023-10-08T05:37:43.721430+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721430+0000","last_undegraded":"2023-10-08T05:37:43.721430+0000","last_fullsized":"2023-10-08T05:37:43.721430+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723901+0000","last_change":"2023-10-08T05:37:43.723901+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723901+0000","last_undegraded":"2023-10-08T05:37:43.723901+0000","last_fullsized":"2023-10-08T05:37:43.723901+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.727715+0000","last_change":"2023-10-08T05:37:43.727715+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.727715+0000","last_undegraded":"2023-10-08T05:37:43.727715+0000","last_fullsized":"2023-10-08T05:37:43.727715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.740518+0000","last_change":"2023-10-08T05:37:43.740518+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.740518+0000","last_undegraded":"2023-10-08T05:37:43.740518+0000","last_fullsized":"2023-10-08T05:37:43.740518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721832+0000","last_change":"2023-10-08T05:37:43.721832+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721832+0000","last_undegraded":"2023-10-08T05:37:43.721832+0000","last_fullsized":"2023-10-08T05:37:43.721832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724170+0000","last_change":"2023-10-08T05:37:43.724170+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724170+0000","last_undegraded":"2023-10-08T05:37:43.724170+0000","last_fullsized":"2023-10-08T05:37:43.724170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737987+0000","last_change":"2023-10-08T05:37:43.737987+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737987+0000","last_undegraded":"2023-10-08T05:37:43.737987+0000","last_fullsized":"2023-10-08T05:37:43.737987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.735624+0000","last_change":"2023-10-08T05:37:43.735624+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.735624+0000","last_undegraded":"2023-10-08T05:37:43.735624+0000","last_fullsized":"2023-10-08T05:37:43.735624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723176+0000","last_change":"2023-10-08T05:37:43.723176+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723176+0000","last_undegraded":"2023-10-08T05:37:43.723176+0000","last_fullsized":"2023-10-08T05:37:43.723176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.738432+0000","last_change":"2023-10-08T05:37:43.738432+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.738432+0000","last_undegraded":"2023-10-08T05:37:43.738432+0000","last_fullsized":"2023-10-08T05:37:43.738432+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.730462+0000","last_change":"2023-10-08T05:37:43.730462+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.730462+0000","last_undegraded":"2023-10-08T05:37:43.730462+0000","last_fullsized":"2023-10-08T05:37:43.730462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.740059+0000","last_change":"2023-10-08T05:37:43.740059+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.740059+0000","last_undegraded":"2023-10-08T05:37:43.740059+0000","last_fullsized":"2023-10-08T05:37:43.740059+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.730787+0000","last_change":"2023-10-08T05:37:43.730787+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.730787+0000","last_undegraded":"2023-10-08T05:37:43.730787+0000","last_fullsized":"2023-10-08T05:37:43.730787+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.738796+0000","last_change":"2023-10-08T05:37:43.738796+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.738796+0000","last_undegraded":"2023-10-08T05:37:43.738796+0000","last_fullsized":"2023-10-08T05:37:43.738796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723467+0000","last_change":"2023-10-08T05:37:43.723467+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723467+0000","last_undegraded":"2023-10-08T05:37:43.723467+0000","last_fullsized":"2023-10-08T05:37:43.723467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720545+0000","last_change":"2023-10-08T05:37:43.720545+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720545+0000","last_undegraded":"2023-10-08T05:37:43.720545+0000","last_fullsized":"2023-10-08T05:37:43.720545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.727938+0000","last_change":"2023-10-08T05:37:43.727938+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.727938+0000","last_undegraded":"2023-10-08T05:37:43.727938+0000","last_fullsized":"2023-10-08T05:37:43.727938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737934+0000","last_change":"2023-10-08T05:37:43.737934+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737934+0000","last_undegraded":"2023-10-08T05:37:43.737934+0000","last_fullsized":"2023-10-08T05:37:43.737934+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728439+0000","last_change":"2023-10-08T05:37:43.728439+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728439+0000","last_undegraded":"2023-10-08T05:37:43.728439+0000","last_fullsized":"2023-10-08T05:37:43.728439+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.736650+0000","last_change":"2023-10-08T05:37:43.736650+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.736650+0000","last_undegraded":"2023-10-08T05:37:43.736650+0000","last_fullsized":"2023-10-08T05:37:43.736650+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720791+0000","last_change":"2023-10-08T05:37:43.720791+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720791+0000","last_undegraded":"2023-10-08T05:37:43.720791+0000","last_fullsized":"2023-10-08T05:37:43.720791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.725014+0000","last_change":"2023-10-08T05:37:43.725014+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.725014+0000","last_undegraded":"2023-10-08T05:37:43.725014+0000","last_fullsized":"2023-10-08T05:37:43.725014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728973+0000","last_change":"2023-10-08T05:37:43.728973+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728973+0000","last_undegraded":"2023-10-08T05:37:43.728973+0000","last_fullsized":"2023-10-08T05:37:43.728973+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723172+0000","last_change":"2023-10-08T05:37:43.723172+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723172+0000","last_undegraded":"2023-10-08T05:37:43.723172+0000","last_fullsized":"2023-10-08T05:37:43.723172+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.740470+0000","last_change":"2023-10-08T05:37:43.740470+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.740470+0000","last_undegraded":"2023-10-08T05:37:43.740470+0000","last_fullsized":"2023-10-08T05:37:43.740470+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721405+0000","last_change":"2023-10-08T05:37:43.721405+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721405+0000","last_undegraded":"2023-10-08T05:37:43.721405+0000","last_fullsized":"2023-10-08T05:37:43.721405+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.722018+0000","last_change":"2023-10-08T05:37:43.722018+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.722018+0000","last_undegraded":"2023-10-08T05:37:43.722018+0000","last_fullsized":"2023-10-08T05:37:43.722018+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.733264+0000","last_change":"2023-10-08T05:37:43.733264+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.733264+0000","last_undegraded":"2023-10-08T05:37:43.733264+0000","last_fullsized":"2023-10-08T05:37:43.733264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728215+0000","last_change":"2023-10-08T05:37:43.728215+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728215+0000","last_undegraded":"2023-10-08T05:37:43.728215+0000","last_fullsized":"2023-10-08T05:37:43.728215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.739149+0000","last_change":"2023-10-08T05:37:43.739149+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.739149+0000","last_undegraded":"2023-10-08T05:37:43.739149+0000","last_fullsized":"2023-10-08T05:37:43.739149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.749119+0000","last_change":"2023-10-08T05:37:43.746333+0000","last_active":"2023-10-08T05:37:43.749119+0000","last_peered":"2023-10-08T05:37:43.749119+0000","last_clean":"2023-10-08T05:37:43.749119+0000","last_became_active":"2023-10-08T05:37:43.746017+0000","last_became_peered":"2023-10-08T05:37:43.746017+0000","last_unstale":"2023-10-08T05:37:43.749119+0000","last_undegraded":"2023-10-08T05:37:43.749119+0000","last_fullsized":"2023-10-08T05:37:43.749119+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:04:48.940111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.729258+0000","last_change":"2023-10-08T05:37:43.729258+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.729258+0000","last_undegraded":"2023-10-08T05:37:43.729258+0000","last_fullsized":"2023-10-08T05:37:43.729258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751007+0000","last_change":"2023-10-08T05:37:43.749778+0000","last_active":"2023-10-08T05:37:43.751007+0000","last_peered":"2023-10-08T05:37:43.751007+0000","last_clean":"2023-10-08T05:37:43.751007+0000","last_became_active":"2023-10-08T05:37:43.749435+0000","last_became_peered":"2023-10-08T05:37:43.749435+0000","last_unstale":"2023-10-08T05:37:43.751007+0000","last_undegraded":"2023-10-08T05:37:43.751007+0000","last_fullsized":"2023-10-08T05:37:43.751007+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:35.696328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.726040+0000","last_change":"2023-10-08T05:37:43.726040+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.726040+0000","last_undegraded":"2023-10-08T05:37:43.726040+0000","last_fullsized":"2023-10-08T05:37:43.726040+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.750923+0000","last_change":"2023-10-08T05:37:43.749986+0000","last_active":"2023-10-08T05:37:43.750923+0000","last_peered":"2023-10-08T05:37:43.750923+0000","last_clean":"2023-10-08T05:37:43.750923+0000","last_became_active":"2023-10-08T05:37:43.749718+0000","last_became_peered":"2023-10-08T05:37:43.749718+0000","last_unstale":"2023-10-08T05:37:43.750923+0000","last_undegraded":"2023-10-08T05:37:43.750923+0000","last_fullsized":"2023-10-08T05:37:43.750923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:12.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721433+0000","last_change":"2023-10-08T05:37:43.721433+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721433+0000","last_undegraded":"2023-10-08T05:37:43.721433+0000","last_fullsized":"2023-10-08T05:37:43.721433+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.748417+0000","last_change":"2023-10-08T05:37:43.742613+0000","last_active":"2023-10-08T05:37:43.748417+0000","last_peered":"2023-10-08T05:37:43.748417+0000","last_clean":"2023-10-08T05:37:43.748417+0000","last_became_active":"2023-10-08T05:37:43.742232+0000","last_became_peered":"2023-10-08T05:37:43.742232+0000","last_unstale":"2023-10-08T05:37:43.748417+0000","last_undegraded":"2023-10-08T05:37:43.748417+0000","last_fullsized":"2023-10-08T05:37:43.748417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:35.696328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724604+0000","last_change":"2023-10-08T05:37:43.724604+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724604+0000","last_undegraded":"2023-10-08T05:37:43.724604+0000","last_fullsized":"2023-10-08T05:37:43.724604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752140+0000","last_change":"2023-10-08T05:37:43.743385+0000","last_active":"2023-10-08T05:37:43.752140+0000","last_peered":"2023-10-08T05:37:43.752140+0000","last_clean":"2023-10-08T05:37:43.752140+0000","last_became_active":"2023-10-08T05:37:43.743002+0000","last_became_peered":"2023-10-08T05:37:43.743002+0000","last_unstale":"2023-10-08T05:37:43.752140+0000","last_undegraded":"2023-10-08T05:37:43.752140+0000","last_fullsized":"2023-10-08T05:37:43.752140+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:56:20.161604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.739571+0000","last_change":"2023-10-08T05:37:43.739571+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.739571+0000","last_undegraded":"2023-10-08T05:37:43.739571+0000","last_fullsized":"2023-10-08T05:37:43.739571+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.747959+0000","last_change":"2023-10-08T05:37:43.746646+0000","last_active":"2023-10-08T05:37:43.747959+0000","last_peered":"2023-10-08T05:37:43.747959+0000","last_clean":"2023-10-08T05:37:43.747959+0000","last_became_active":"2023-10-08T05:37:43.746369+0000","last_became_peered":"2023-10-08T05:37:43.746369+0000","last_unstale":"2023-10-08T05:37:43.747959+0000","last_undegraded":"2023-10-08T05:37:43.747959+0000","last_fullsized":"2023-10-08T05:37:43.747959+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:12.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.725832+0000","last_change":"2023-10-08T05:37:43.725832+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.725832+0000","last_undegraded":"2023-10-08T05:37:43.725832+0000","last_fullsized":"2023-10-08T05:37:43.725832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.747537+0000","last_change":"2023-10-08T05:37:43.746978+0000","last_active":"2023-10-08T05:37:43.747537+0000","last_peered":"2023-10-08T05:37:43.747537+0000","last_clean":"2023-10-08T05:37:43.747537+0000","last_became_active":"2023-10-08T05:37:43.746728+0000","last_became_peered":"2023-10-08T05:37:43.746728+0000","last_unstale":"2023-10-08T05:37:43.747537+0000","last_undegraded":"2023-10-08T05:37:43.747537+0000","last_fullsized":"2023-10-08T05:37:43.747537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724140+0000","last_change":"2023-10-08T05:37:43.724140+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724140+0000","last_undegraded":"2023-10-08T05:37:43.724140+0000","last_fullsized":"2023-10-08T05:37:43.724140+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.745405+0000","last_change":"2023-10-08T05:37:43.736460+0000","last_active":"2023-10-08T05:37:43.745405+0000","last_peered":"2023-10-08T05:37:43.745405+0000","last_clean":"2023-10-08T05:37:43.745405+0000","last_became_active":"2023-10-08T05:37:43.736033+0000","last_became_peered":"2023-10-08T05:37:43.736033+0000","last_unstale":"2023-10-08T05:37:43.745405+0000","last_undegraded":"2023-10-08T05:37:43.745405+0000","last_fullsized":"2023-10-08T05:37:43.745405+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:24.017005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720387+0000","last_change":"2023-10-08T05:37:43.720387+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720387+0000","last_undegraded":"2023-10-08T05:37:43.720387+0000","last_fullsized":"2023-10-08T05:37:43.720387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.737892+0000","last_change":"2023-10-08T05:37:43.735761+0000","last_active":"2023-10-08T05:37:43.737892+0000","last_peered":"2023-10-08T05:37:43.737892+0000","last_clean":"2023-10-08T05:37:43.737892+0000","last_became_active":"2023-10-08T05:37:43.735391+0000","last_became_peered":"2023-10-08T05:37:43.735391+0000","last_unstale":"2023-10-08T05:37:43.737892+0000","last_undegraded":"2023-10-08T05:37:43.737892+0000","last_fullsized":"2023-10-08T05:37:43.737892+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:26.955372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.734924+0000","last_change":"2023-10-08T05:37:43.734924+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.734924+0000","last_undegraded":"2023-10-08T05:37:43.734924+0000","last_fullsized":"2023-10-08T05:37:43.734924+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751246+0000","last_change":"2023-10-08T05:37:43.742980+0000","last_active":"2023-10-08T05:37:43.751246+0000","last_peered":"2023-10-08T05:37:43.751246+0000","last_clean":"2023-10-08T05:37:43.751246+0000","last_became_active":"2023-10-08T05:37:43.742358+0000","last_became_peered":"2023-10-08T05:37:43.742358+0000","last_unstale":"2023-10-08T05:37:43.751246+0000","last_undegraded":"2023-10-08T05:37:43.751246+0000","last_fullsized":"2023-10-08T05:37:43.751246+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:53.615714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721234+0000","last_change":"2023-10-08T05:37:43.721234+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721234+0000","last_undegraded":"2023-10-08T05:37:43.721234+0000","last_fullsized":"2023-10-08T05:37:43.721234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752211+0000","last_change":"2023-10-08T05:37:43.743414+0000","last_active":"2023-10-08T05:37:43.752211+0000","last_peered":"2023-10-08T05:37:43.752211+0000","last_clean":"2023-10-08T05:37:43.752211+0000","last_became_active":"2023-10-08T05:37:43.742873+0000","last_became_peered":"2023-10-08T05:37:43.742873+0000","last_unstale":"2023-10-08T05:37:43.752211+0000","last_undegraded":"2023-10-08T05:37:43.752211+0000","last_fullsized":"2023-10-08T05:37:43.752211+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724763+0000","last_change":"2023-10-08T05:37:43.724763+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724763+0000","last_undegraded":"2023-10-08T05:37:43.724763+0000","last_fullsized":"2023-10-08T05:37:43.724763+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.750862+0000","last_change":"2023-10-08T05:37:43.749228+0000","last_active":"2023-10-08T05:37:43.750862+0000","last_peered":"2023-10-08T05:37:43.750862+0000","last_clean":"2023-10-08T05:37:43.750862+0000","last_became_active":"2023-10-08T05:37:43.748703+0000","last_became_peered":"2023-10-08T05:37:43.748703+0000","last_unstale":"2023-10-08T05:37:43.750862+0000","last_undegraded":"2023-10-08T05:37:43.750862+0000","last_fullsized":"2023-10-08T05:37:43.750862+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:11:28.561634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.724118+0000","last_change":"2023-10-08T05:37:43.724118+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.724118+0000","last_undegraded":"2023-10-08T05:37:43.724118+0000","last_fullsized":"2023-10-08T05:37:43.724118+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.730582+0000","last_change":"2023-10-08T05:37:43.729198+0000","last_active":"2023-10-08T05:37:43.730582+0000","last_peered":"2023-10-08T05:37:43.730582+0000","last_clean":"2023-10-08T05:37:43.730582+0000","last_became_active":"2023-10-08T05:37:43.728918+0000","last_became_peered":"2023-10-08T05:37:43.728918+0000","last_unstale":"2023-10-08T05:37:43.730582+0000","last_undegraded":"2023-10-08T05:37:43.730582+0000","last_fullsized":"2023-10-08T05:37:43.730582+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720385+0000","last_change":"2023-10-08T05:37:43.720385+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720385+0000","last_undegraded":"2023-10-08T05:37:43.720385+0000","last_fullsized":"2023-10-08T05:37:43.720385+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746846+0000","last_change":"2023-10-08T05:37:43.745271+0000","last_active":"2023-10-08T05:37:43.746846+0000","last_peered":"2023-10-08T05:37:43.746846+0000","last_clean":"2023-10-08T05:37:43.746846+0000","last_became_active":"2023-10-08T05:37:43.745043+0000","last_became_peered":"2023-10-08T05:37:43.745043+0000","last_unstale":"2023-10-08T05:37:43.746846+0000","last_undegraded":"2023-10-08T05:37:43.746846+0000","last_fullsized":"2023-10-08T05:37:43.746846+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:34.606944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737702+0000","last_change":"2023-10-08T05:37:43.737702+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737702+0000","last_undegraded":"2023-10-08T05:37:43.737702+0000","last_fullsized":"2023-10-08T05:37:43.737702+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.749151+0000","last_change":"2023-10-08T05:37:43.746332+0000","last_active":"2023-10-08T05:37:43.749151+0000","last_peered":"2023-10-08T05:37:43.749151+0000","last_clean":"2023-10-08T05:37:43.749151+0000","last_became_active":"2023-10-08T05:37:43.746017+0000","last_became_peered":"2023-10-08T05:37:43.746017+0000","last_unstale":"2023-10-08T05:37:43.749151+0000","last_undegraded":"2023-10-08T05:37:43.749151+0000","last_fullsized":"2023-10-08T05:37:43.749151+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:10.220125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.722997+0000","last_change":"2023-10-08T05:37:43.722997+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.722997+0000","last_undegraded":"2023-10-08T05:37:43.722997+0000","last_fullsized":"2023-10-08T05:37:43.722997+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752512+0000","last_change":"2023-10-08T05:37:43.750157+0000","last_active":"2023-10-08T05:37:43.752512+0000","last_peered":"2023-10-08T05:37:43.752512+0000","last_clean":"2023-10-08T05:37:43.752512+0000","last_became_active":"2023-10-08T05:37:43.749846+0000","last_became_peered":"2023-10-08T05:37:43.749846+0000","last_unstale":"2023-10-08T05:37:43.752512+0000","last_undegraded":"2023-10-08T05:37:43.752512+0000","last_fullsized":"2023-10-08T05:37:43.752512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:53.615714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.719971+0000","last_change":"2023-10-08T05:37:43.719971+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.719971+0000","last_undegraded":"2023-10-08T05:37:43.719971+0000","last_fullsized":"2023-10-08T05:37:43.719971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.753906+0000","last_change":"2023-10-08T05:37:43.752005+0000","last_active":"2023-10-08T05:37:43.753906+0000","last_peered":"2023-10-08T05:37:43.753906+0000","last_clean":"2023-10-08T05:37:43.753906+0000","last_became_active":"2023-10-08T05:37:43.751525+0000","last_became_peered":"2023-10-08T05:37:43.751525+0000","last_unstale":"2023-10-08T05:37:43.753906+0000","last_undegraded":"2023-10-08T05:37:43.753906+0000","last_fullsized":"2023-10-08T05:37:43.753906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.734950+0000","last_change":"2023-10-08T05:37:43.734950+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.734950+0000","last_undegraded":"2023-10-08T05:37:43.734950+0000","last_fullsized":"2023-10-08T05:37:43.734950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.749409+0000","last_change":"2023-10-08T05:37:43.745487+0000","last_active":"2023-10-08T05:37:43.749409+0000","last_peered":"2023-10-08T05:37:43.749409+0000","last_clean":"2023-10-08T05:37:43.749409+0000","last_became_active":"2023-10-08T05:37:43.744512+0000","last_became_peered":"2023-10-08T05:37:43.744512+0000","last_unstale":"2023-10-08T05:37:43.749409+0000","last_undegraded":"2023-10-08T05:37:43.749409+0000","last_fullsized":"2023-10-08T05:37:43.749409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:10.719966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.727258+0000","last_change":"2023-10-08T05:37:43.727258+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.727258+0000","last_undegraded":"2023-10-08T05:37:43.727258+0000","last_fullsized":"2023-10-08T05:37:43.727258+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751126+0000","last_change":"2023-10-08T05:37:43.748895+0000","last_active":"2023-10-08T05:37:43.751126+0000","last_peered":"2023-10-08T05:37:43.751126+0000","last_clean":"2023-10-08T05:37:43.751126+0000","last_became_active":"2023-10-08T05:37:43.748618+0000","last_became_peered":"2023-10-08T05:37:43.748618+0000","last_unstale":"2023-10-08T05:37:43.751126+0000","last_undegraded":"2023-10-08T05:37:43.751126+0000","last_fullsized":"2023-10-08T05:37:43.751126+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:35.801089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.719846+0000","last_change":"2023-10-08T05:37:43.719846+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.719846+0000","last_undegraded":"2023-10-08T05:37:43.719846+0000","last_fullsized":"2023-10-08T05:37:43.719846+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746320+0000","last_change":"2023-10-08T05:37:43.743970+0000","last_active":"2023-10-08T05:37:43.746320+0000","last_peered":"2023-10-08T05:37:43.746320+0000","last_clean":"2023-10-08T05:37:43.746320+0000","last_became_active":"2023-10-08T05:37:43.743106+0000","last_became_peered":"2023-10-08T05:37:43.743106+0000","last_unstale":"2023-10-08T05:37:43.746320+0000","last_undegraded":"2023-10-08T05:37:43.746320+0000","last_fullsized":"2023-10-08T05:37:43.746320+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:27.563368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.729757+0000","last_change":"2023-10-08T05:37:43.729757+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.729757+0000","last_undegraded":"2023-10-08T05:37:43.729757+0000","last_fullsized":"2023-10-08T05:37:43.729757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.731932+0000","last_change":"2023-10-08T05:37:43.728582+0000","last_active":"2023-10-08T05:37:43.731932+0000","last_peered":"2023-10-08T05:37:43.731932+0000","last_clean":"2023-10-08T05:37:43.731932+0000","last_became_active":"2023-10-08T05:37:43.728172+0000","last_became_peered":"2023-10-08T05:37:43.728172+0000","last_unstale":"2023-10-08T05:37:43.731932+0000","last_undegraded":"2023-10-08T05:37:43.731932+0000","last_fullsized":"2023-10-08T05:37:43.731932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:34.248097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.721280+0000","last_change":"2023-10-08T05:37:43.721280+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.721280+0000","last_undegraded":"2023-10-08T05:37:43.721280+0000","last_fullsized":"2023-10-08T05:37:43.721280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746296+0000","last_change":"2023-10-08T05:37:43.744243+0000","last_active":"2023-10-08T05:37:43.746296+0000","last_peered":"2023-10-08T05:37:43.746296+0000","last_clean":"2023-10-08T05:37:43.746296+0000","last_became_active":"2023-10-08T05:37:43.743356+0000","last_became_peered":"2023-10-08T05:37:43.743356+0000","last_unstale":"2023-10-08T05:37:43.746296+0000","last_undegraded":"2023-10-08T05:37:43.746296+0000","last_fullsized":"2023-10-08T05:37:43.746296+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:26:44.883068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.735316+0000","last_change":"2023-10-08T05:37:43.735316+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.735316+0000","last_undegraded":"2023-10-08T05:37:43.735316+0000","last_fullsized":"2023-10-08T05:37:43.735316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752420+0000","last_change":"2023-10-08T05:37:43.749749+0000","last_active":"2023-10-08T05:37:43.752420+0000","last_peered":"2023-10-08T05:37:43.752420+0000","last_clean":"2023-10-08T05:37:43.752420+0000","last_became_active":"2023-10-08T05:37:43.749413+0000","last_became_peered":"2023-10-08T05:37:43.749413+0000","last_unstale":"2023-10-08T05:37:43.752420+0000","last_undegraded":"2023-10-08T05:37:43.752420+0000","last_fullsized":"2023-10-08T05:37:43.752420+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:27.563368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.719803+0000","last_change":"2023-10-08T05:37:43.719803+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.719803+0000","last_undegraded":"2023-10-08T05:37:43.719803+0000","last_fullsized":"2023-10-08T05:37:43.719803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.734537+0000","last_change":"2023-10-08T05:37:43.733311+0000","last_active":"2023-10-08T05:37:43.734537+0000","last_peered":"2023-10-08T05:37:43.734537+0000","last_clean":"2023-10-08T05:37:43.734537+0000","last_became_active":"2023-10-08T05:37:43.733051+0000","last_became_peered":"2023-10-08T05:37:43.733051+0000","last_unstale":"2023-10-08T05:37:43.734537+0000","last_undegraded":"2023-10-08T05:37:43.734537+0000","last_fullsized":"2023-10-08T05:37:43.734537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:02.738727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.728203+0000","last_change":"2023-10-08T05:37:43.728203+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.728203+0000","last_undegraded":"2023-10-08T05:37:43.728203+0000","last_fullsized":"2023-10-08T05:37:43.728203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.752422+0000","last_change":"2023-10-08T05:37:43.749129+0000","last_active":"2023-10-08T05:37:43.752422+0000","last_peered":"2023-10-08T05:37:43.752422+0000","last_clean":"2023-10-08T05:37:43.752422+0000","last_became_active":"2023-10-08T05:37:43.748832+0000","last_became_peered":"2023-10-08T05:37:43.748832+0000","last_unstale":"2023-10-08T05:37:43.752422+0000","last_undegraded":"2023-10-08T05:37:43.752422+0000","last_fullsized":"2023-10-08T05:37:43.752422+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:41.403273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.733504+0000","last_change":"2023-10-08T05:37:43.733504+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.733504+0000","last_undegraded":"2023-10-08T05:37:43.733504+0000","last_fullsized":"2023-10-08T05:37:43.733504+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.746378+0000","last_change":"2023-10-08T05:37:43.744435+0000","last_active":"2023-10-08T05:37:43.746378+0000","last_peered":"2023-10-08T05:37:43.746378+0000","last_clean":"2023-10-08T05:37:43.746378+0000","last_became_active":"2023-10-08T05:37:43.744157+0000","last_became_peered":"2023-10-08T05:37:43.744157+0000","last_unstale":"2023-10-08T05:37:43.746378+0000","last_undegraded":"2023-10-08T05:37:43.746378+0000","last_fullsized":"2023-10-08T05:37:43.746378+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:21:09.150233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.720525+0000","last_change":"2023-10-08T05:37:43.720525+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.720525+0000","last_undegraded":"2023-10-08T05:37:43.720525+0000","last_fullsized":"2023-10-08T05:37:43.720525+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.736500+0000","last_change":"2023-10-08T05:37:43.733742+0000","last_active":"2023-10-08T05:37:43.736500+0000","last_peered":"2023-10-08T05:37:43.736500+0000","last_clean":"2023-10-08T05:37:43.736500+0000","last_became_active":"2023-10-08T05:37:43.733486+0000","last_became_peered":"2023-10-08T05:37:43.733486+0000","last_unstale":"2023-10-08T05:37:43.736500+0000","last_undegraded":"2023-10-08T05:37:43.736500+0000","last_fullsized":"2023-10-08T05:37:43.736500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:10.719966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.723702+0000","last_change":"2023-10-08T05:37:43.723702+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.723702+0000","last_undegraded":"2023-10-08T05:37:43.723702+0000","last_fullsized":"2023-10-08T05:37:43.723702+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.736440+0000","last_change":"2023-10-08T05:37:43.732879+0000","last_active":"2023-10-08T05:37:43.736440+0000","last_peered":"2023-10-08T05:37:43.736440+0000","last_clean":"2023-10-08T05:37:43.736440+0000","last_became_active":"2023-10-08T05:37:43.732616+0000","last_became_peered":"2023-10-08T05:37:43.732616+0000","last_unstale":"2023-10-08T05:37:43.736440+0000","last_undegraded":"2023-10-08T05:37:43.736440+0000","last_fullsized":"2023-10-08T05:37:43.736440+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:34.606944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.731047+0000","last_change":"2023-10-08T05:37:43.731047+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.731047+0000","last_undegraded":"2023-10-08T05:37:43.731047+0000","last_fullsized":"2023-10-08T05:37:43.731047+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.743947+0000","last_change":"2023-10-08T05:37:43.733421+0000","last_active":"2023-10-08T05:37:43.743947+0000","last_peered":"2023-10-08T05:37:43.743947+0000","last_clean":"2023-10-08T05:37:43.743947+0000","last_became_active":"2023-10-08T05:37:43.732803+0000","last_became_peered":"2023-10-08T05:37:43.732803+0000","last_unstale":"2023-10-08T05:37:43.743947+0000","last_undegraded":"2023-10-08T05:37:43.743947+0000","last_fullsized":"2023-10-08T05:37:43.743947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:28:06.562486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.729623+0000","last_change":"2023-10-08T05:37:43.729623+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.729623+0000","last_undegraded":"2023-10-08T05:37:43.729623+0000","last_fullsized":"2023-10-08T05:37:43.729623+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.747480+0000","last_change":"2023-10-08T05:37:43.736772+0000","last_active":"2023-10-08T05:37:43.747480+0000","last_peered":"2023-10-08T05:37:43.747480+0000","last_clean":"2023-10-08T05:37:43.747480+0000","last_became_active":"2023-10-08T05:37:43.736441+0000","last_became_peered":"2023-10-08T05:37:43.736441+0000","last_unstale":"2023-10-08T05:37:43.747480+0000","last_undegraded":"2023-10-08T05:37:43.747480+0000","last_fullsized":"2023-10-08T05:37:43.747480+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:10.220125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.737617+0000","last_change":"2023-10-08T05:37:43.737617+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.737617+0000","last_undegraded":"2023-10-08T05:37:43.737617+0000","last_fullsized":"2023-10-08T05:37:43.737617+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.734888+0000","last_change":"2023-10-08T05:37:43.733011+0000","last_active":"2023-10-08T05:37:43.734888+0000","last_peered":"2023-10-08T05:37:43.734888+0000","last_clean":"2023-10-08T05:37:43.734888+0000","last_became_active":"2023-10-08T05:37:43.732744+0000","last_became_peered":"2023-10-08T05:37:43.732744+0000","last_unstale":"2023-10-08T05:37:43.734888+0000","last_undegraded":"2023-10-08T05:37:43.734888+0000","last_fullsized":"2023-10-08T05:37:43.734888+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:41.403273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2023-10-08T05:37:43.741498+0000","last_change":"2023-10-08T05:37:43.741498+0000","last_active":"2023-10-08T05:37:43.697452+0000","last_peered":"2023-10-08T05:37:43.697452+0000","last_clean":"2023-10-08T05:37:43.697452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:37:43.741498+0000","last_undegraded":"2023-10-08T05:37:43.741498+0000","last_fullsized":"2023-10-08T05:37:43.741498+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-08T05:37:43.751517+0000","last_change":"2023-10-08T05:37:43.748040+0000","last_active":"2023-10-08T05:37:43.751517+0000","last_peered":"2023-10-08T05:37:43.751517+0000","last_clean":"2023-10-08T05:37:43.751517+0000","last_became_active":"2023-10-08T05:37:43.747745+0000","last_became_peered":"2023-10-08T05:37:43.747745+0000","last_unstale":"2023-10-08T05:37:43.751517+0000","last_undegraded":"2023-10-08T05:37:43.751517+0000","last_fullsized":"2023-10-08T05:37:43.751517+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":179,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-10-08T05:37:42.714859+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:37:42.714859+0000","last_peered":"2023-10-08T05:37:42.714859+0000","last_clean":"2023-10-08T05:37:42.714859+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:37:42.714859+0000","last_undegraded":"2023-10-08T05:37:42.714859+0000","last_fullsized":"2023-10-08T05:37:42.714859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":770492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.91000000000000003}]},{"osd":1,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.032}]},{"osd":2,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.145}]},{"osd":3,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.095},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":1.3600000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.319},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":1.637}]},{"osd":5,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":1.762}]},{"osd":6,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":3.5249999999999999,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.8340000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.7050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":180124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.4390000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.93100000000000005}]},{"osd":2,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.98499999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.341},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.512}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.379}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":4.3099999999999996,"5min":4.3099999999999996,"15min":4.3099999999999996},"last":1.415},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.175,"5min":1.175,"15min":1.175},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.569}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.23,"5min":1.23,"15min":1.23},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"max":{"1min":4.1550000000000002,"5min":4.1550000000000002,"15min":4.1550000000000002},"last":1.5449999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":3.5529999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.73299999999999998}]},{"osd":1,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":1.504},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.274}]},{"osd":2,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.208},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.89200000000000002}]},{"osd":3,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":6.782,"5min":6.782,"15min":6.782},"last":1.159},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"last":1.099}]},{"osd":4,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.1349999999999998,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.762,"5min":1.762,"15min":1.762},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":7.0069999999999997,"5min":7.0069999999999997,"15min":7.0069999999999997},"last":1.3740000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":7.2569999999999997,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":6.2199999999999998,"5min":6.2199999999999998,"15min":6.2199999999999998},"last":1.621}]},{"osd":7,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":6.9279999999999999,"5min":6.9279999999999999,"15min":6.9279999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":7.335,"5min":7.335,"15min":7.335},"last":1.5649999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.224}]},{"osd":1,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.88700000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.53},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.329}]},{"osd":3,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.3560000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.135},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.7290000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.478}]},{"osd":7,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.76,"5min":1.76,"15min":1.76},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.7809999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":425984,"data_stored":180124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.75}]},{"osd":1,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.143}]},{"osd":2,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.2849999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.492}]},{"osd":5,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.536}]},{"osd":6,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.337},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.597}]},{"osd":7,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.468},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.5629999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31596,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340244,"statfs":{"total":96636764160,"available":96604409856,"internally_reserved":0,"allocated":425984,"data_stored":180124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.54400000000000004}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.23}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.278},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.002}]},{"osd":6,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":1.04}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.0529999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.101}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":173529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.0369999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.98799999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.94999999999999996}]},{"osd":4,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.28},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.2470000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":1.345}]},{"osd":6,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.498}]},{"osd":7,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.5249999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":763897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.032}]},{"osd":2,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.169},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.94199999999999995}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.82599999999999996}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":1.131},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.363}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.278},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":1.413}]},{"osd":6,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.0669999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.248}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:37:50.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2023-10-08T05:37:50.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.726910+0000', 'last_change': '2023-10-08T05:37:43.726910+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.726910+0000', 'last_undegraded': '2023-10-08T05:37:43.726910+0000', 'last_fullsized': '2023-10-08T05:37:43.726910+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T05:37:50.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2023-10-08T05:37:50.426 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.725861+0000', 'last_change': '2023-10-08T05:37:43.725861+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.725861+0000', 'last_undegraded': '2023-10-08T05:37:43.725861+0000', 'last_fullsized': '2023-10-08T05:37:43.725861+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T05:37:50.426 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2023-10-08T05:37:50.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.722327+0000', 'last_change': '2023-10-08T05:37:43.722327+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.722327+0000', 'last_undegraded': '2023-10-08T05:37:43.722327+0000', 'last_fullsized': '2023-10-08T05:37:43.722327+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T05:37:50.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2023-10-08T05:37:50.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.721430+0000', 'last_change': '2023-10-08T05:37:43.721430+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.721430+0000', 'last_undegraded': '2023-10-08T05:37:43.721430+0000', 'last_fullsized': '2023-10-08T05:37:43.721430+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T05:37:50.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2023-10-08T05:37:50.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.723901+0000', 'last_change': '2023-10-08T05:37:43.723901+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.723901+0000', 'last_undegraded': '2023-10-08T05:37:43.723901+0000', 'last_fullsized': '2023-10-08T05:37:43.723901+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T05:37:50.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2023-10-08T05:37:50.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.727715+0000', 'last_change': '2023-10-08T05:37:43.727715+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.727715+0000', 'last_undegraded': '2023-10-08T05:37:43.727715+0000', 'last_fullsized': '2023-10-08T05:37:43.727715+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T05:37:50.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2023-10-08T05:37:50.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.740518+0000', 'last_change': '2023-10-08T05:37:43.740518+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.740518+0000', 'last_undegraded': '2023-10-08T05:37:43.740518+0000', 'last_fullsized': '2023-10-08T05:37:43.740518+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2023-10-08T05:37:50.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.721832+0000', 'last_change': '2023-10-08T05:37:43.721832+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.721832+0000', 'last_undegraded': '2023-10-08T05:37:43.721832+0000', 'last_fullsized': '2023-10-08T05:37:43.721832+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T05:37:50.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2023-10-08T05:37:50.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.724170+0000', 'last_change': '2023-10-08T05:37:43.724170+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.724170+0000', 'last_undegraded': '2023-10-08T05:37:43.724170+0000', 'last_fullsized': '2023-10-08T05:37:43.724170+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T05:37:50.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2023-10-08T05:37:50.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.737987+0000', 'last_change': '2023-10-08T05:37:43.737987+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.737987+0000', 'last_undegraded': '2023-10-08T05:37:43.737987+0000', 'last_fullsized': '2023-10-08T05:37:43.737987+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2023-10-08T05:37:50.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.735624+0000', 'last_change': '2023-10-08T05:37:43.735624+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.735624+0000', 'last_undegraded': '2023-10-08T05:37:43.735624+0000', 'last_fullsized': '2023-10-08T05:37:43.735624+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2023-10-08T05:37:50.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.723176+0000', 'last_change': '2023-10-08T05:37:43.723176+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.723176+0000', 'last_undegraded': '2023-10-08T05:37:43.723176+0000', 'last_fullsized': '2023-10-08T05:37:43.723176+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T05:37:50.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2023-10-08T05:37:50.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.738432+0000', 'last_change': '2023-10-08T05:37:43.738432+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.738432+0000', 'last_undegraded': '2023-10-08T05:37:43.738432+0000', 'last_fullsized': '2023-10-08T05:37:43.738432+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2023-10-08T05:37:50.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.730462+0000', 'last_change': '2023-10-08T05:37:43.730462+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.730462+0000', 'last_undegraded': '2023-10-08T05:37:43.730462+0000', 'last_fullsized': '2023-10-08T05:37:43.730462+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T05:37:50.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2023-10-08T05:37:50.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.740059+0000', 'last_change': '2023-10-08T05:37:43.740059+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.740059+0000', 'last_undegraded': '2023-10-08T05:37:43.740059+0000', 'last_fullsized': '2023-10-08T05:37:43.740059+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2023-10-08T05:37:50.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.730787+0000', 'last_change': '2023-10-08T05:37:43.730787+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.730787+0000', 'last_undegraded': '2023-10-08T05:37:43.730787+0000', 'last_fullsized': '2023-10-08T05:37:43.730787+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T05:37:50.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2023-10-08T05:37:50.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.738796+0000', 'last_change': '2023-10-08T05:37:43.738796+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.738796+0000', 'last_undegraded': '2023-10-08T05:37:43.738796+0000', 'last_fullsized': '2023-10-08T05:37:43.738796+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T05:37:50.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2023-10-08T05:37:50.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.723467+0000', 'last_change': '2023-10-08T05:37:43.723467+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.723467+0000', 'last_undegraded': '2023-10-08T05:37:43.723467+0000', 'last_fullsized': '2023-10-08T05:37:43.723467+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T05:37:50.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2023-10-08T05:37:50.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.720545+0000', 'last_change': '2023-10-08T05:37:43.720545+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.720545+0000', 'last_undegraded': '2023-10-08T05:37:43.720545+0000', 'last_fullsized': '2023-10-08T05:37:43.720545+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T05:37:50.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2023-10-08T05:37:50.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.727938+0000', 'last_change': '2023-10-08T05:37:43.727938+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.727938+0000', 'last_undegraded': '2023-10-08T05:37:43.727938+0000', 'last_fullsized': '2023-10-08T05:37:43.727938+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T05:37:50.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2023-10-08T05:37:50.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.737934+0000', 'last_change': '2023-10-08T05:37:43.737934+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.737934+0000', 'last_undegraded': '2023-10-08T05:37:43.737934+0000', 'last_fullsized': '2023-10-08T05:37:43.737934+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2023-10-08T05:37:50.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.728439+0000', 'last_change': '2023-10-08T05:37:43.728439+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.728439+0000', 'last_undegraded': '2023-10-08T05:37:43.728439+0000', 'last_fullsized': '2023-10-08T05:37:43.728439+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T05:37:50.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2023-10-08T05:37:50.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.736650+0000', 'last_change': '2023-10-08T05:37:43.736650+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.736650+0000', 'last_undegraded': '2023-10-08T05:37:43.736650+0000', 'last_fullsized': '2023-10-08T05:37:43.736650+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T05:37:50.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2023-10-08T05:37:50.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.720791+0000', 'last_change': '2023-10-08T05:37:43.720791+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.720791+0000', 'last_undegraded': '2023-10-08T05:37:43.720791+0000', 'last_fullsized': '2023-10-08T05:37:43.720791+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T05:37:50.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2023-10-08T05:37:50.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.725014+0000', 'last_change': '2023-10-08T05:37:43.725014+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.725014+0000', 'last_undegraded': '2023-10-08T05:37:43.725014+0000', 'last_fullsized': '2023-10-08T05:37:43.725014+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T05:37:50.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2023-10-08T05:37:50.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.728973+0000', 'last_change': '2023-10-08T05:37:43.728973+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.728973+0000', 'last_undegraded': '2023-10-08T05:37:43.728973+0000', 'last_fullsized': '2023-10-08T05:37:43.728973+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T05:37:50.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2023-10-08T05:37:50.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.723172+0000', 'last_change': '2023-10-08T05:37:43.723172+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.723172+0000', 'last_undegraded': '2023-10-08T05:37:43.723172+0000', 'last_fullsized': '2023-10-08T05:37:43.723172+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T05:37:50.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2023-10-08T05:37:50.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.740470+0000', 'last_change': '2023-10-08T05:37:43.740470+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.740470+0000', 'last_undegraded': '2023-10-08T05:37:43.740470+0000', 'last_fullsized': '2023-10-08T05:37:43.740470+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2023-10-08T05:37:50.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.721405+0000', 'last_change': '2023-10-08T05:37:43.721405+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.721405+0000', 'last_undegraded': '2023-10-08T05:37:43.721405+0000', 'last_fullsized': '2023-10-08T05:37:43.721405+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T05:37:50.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2023-10-08T05:37:50.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.722018+0000', 'last_change': '2023-10-08T05:37:43.722018+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.722018+0000', 'last_undegraded': '2023-10-08T05:37:43.722018+0000', 'last_fullsized': '2023-10-08T05:37:43.722018+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T05:37:50.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2023-10-08T05:37:50.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.733264+0000', 'last_change': '2023-10-08T05:37:43.733264+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.733264+0000', 'last_undegraded': '2023-10-08T05:37:43.733264+0000', 'last_fullsized': '2023-10-08T05:37:43.733264+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T05:37:50.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2023-10-08T05:37:50.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.728215+0000', 'last_change': '2023-10-08T05:37:43.728215+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.728215+0000', 'last_undegraded': '2023-10-08T05:37:43.728215+0000', 'last_fullsized': '2023-10-08T05:37:43.728215+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T05:37:50.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-10-08T05:37:50.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.739149+0000', 'last_change': '2023-10-08T05:37:43.739149+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.739149+0000', 'last_undegraded': '2023-10-08T05:37:43.739149+0000', 'last_fullsized': '2023-10-08T05:37:43.739149+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T05:37:50.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-10-08T05:37:50.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.729258+0000', 'last_change': '2023-10-08T05:37:43.729258+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.729258+0000', 'last_undegraded': '2023-10-08T05:37:43.729258+0000', 'last_fullsized': '2023-10-08T05:37:43.729258+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T05:37:50.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-10-08T05:37:50.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.726040+0000', 'last_change': '2023-10-08T05:37:43.726040+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.726040+0000', 'last_undegraded': '2023-10-08T05:37:43.726040+0000', 'last_fullsized': '2023-10-08T05:37:43.726040+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T05:37:50.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-10-08T05:37:50.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.721433+0000', 'last_change': '2023-10-08T05:37:43.721433+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.721433+0000', 'last_undegraded': '2023-10-08T05:37:43.721433+0000', 'last_fullsized': '2023-10-08T05:37:43.721433+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T05:37:50.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-10-08T05:37:50.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.724604+0000', 'last_change': '2023-10-08T05:37:43.724604+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.724604+0000', 'last_undegraded': '2023-10-08T05:37:43.724604+0000', 'last_fullsized': '2023-10-08T05:37:43.724604+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T05:37:50.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-10-08T05:37:50.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.739571+0000', 'last_change': '2023-10-08T05:37:43.739571+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.739571+0000', 'last_undegraded': '2023-10-08T05:37:43.739571+0000', 'last_fullsized': '2023-10-08T05:37:43.739571+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-10-08T05:37:50.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.725832+0000', 'last_change': '2023-10-08T05:37:43.725832+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.725832+0000', 'last_undegraded': '2023-10-08T05:37:43.725832+0000', 'last_fullsized': '2023-10-08T05:37:43.725832+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T05:37:50.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-10-08T05:37:50.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.724140+0000', 'last_change': '2023-10-08T05:37:43.724140+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.724140+0000', 'last_undegraded': '2023-10-08T05:37:43.724140+0000', 'last_fullsized': '2023-10-08T05:37:43.724140+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T05:37:50.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-10-08T05:37:50.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.720387+0000', 'last_change': '2023-10-08T05:37:43.720387+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.720387+0000', 'last_undegraded': '2023-10-08T05:37:43.720387+0000', 'last_fullsized': '2023-10-08T05:37:43.720387+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T05:37:50.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-10-08T05:37:50.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.734924+0000', 'last_change': '2023-10-08T05:37:43.734924+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.734924+0000', 'last_undegraded': '2023-10-08T05:37:43.734924+0000', 'last_fullsized': '2023-10-08T05:37:43.734924+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T05:37:50.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-10-08T05:37:50.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.721234+0000', 'last_change': '2023-10-08T05:37:43.721234+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.721234+0000', 'last_undegraded': '2023-10-08T05:37:43.721234+0000', 'last_fullsized': '2023-10-08T05:37:43.721234+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T05:37:50.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-10-08T05:37:50.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.724763+0000', 'last_change': '2023-10-08T05:37:43.724763+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.724763+0000', 'last_undegraded': '2023-10-08T05:37:43.724763+0000', 'last_fullsized': '2023-10-08T05:37:43.724763+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T05:37:50.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-10-08T05:37:50.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.724118+0000', 'last_change': '2023-10-08T05:37:43.724118+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.724118+0000', 'last_undegraded': '2023-10-08T05:37:43.724118+0000', 'last_fullsized': '2023-10-08T05:37:43.724118+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T05:37:50.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-10-08T05:37:50.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.720385+0000', 'last_change': '2023-10-08T05:37:43.720385+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.720385+0000', 'last_undegraded': '2023-10-08T05:37:43.720385+0000', 'last_fullsized': '2023-10-08T05:37:43.720385+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T05:37:50.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-10-08T05:37:50.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.737702+0000', 'last_change': '2023-10-08T05:37:43.737702+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.737702+0000', 'last_undegraded': '2023-10-08T05:37:43.737702+0000', 'last_fullsized': '2023-10-08T05:37:43.737702+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T05:37:50.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-10-08T05:37:50.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.722997+0000', 'last_change': '2023-10-08T05:37:43.722997+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.722997+0000', 'last_undegraded': '2023-10-08T05:37:43.722997+0000', 'last_fullsized': '2023-10-08T05:37:43.722997+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T05:37:50.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-10-08T05:37:50.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.719971+0000', 'last_change': '2023-10-08T05:37:43.719971+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.719971+0000', 'last_undegraded': '2023-10-08T05:37:43.719971+0000', 'last_fullsized': '2023-10-08T05:37:43.719971+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T05:37:50.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-10-08T05:37:50.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.734950+0000', 'last_change': '2023-10-08T05:37:43.734950+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.734950+0000', 'last_undegraded': '2023-10-08T05:37:43.734950+0000', 'last_fullsized': '2023-10-08T05:37:43.734950+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-10-08T05:37:50.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.727258+0000', 'last_change': '2023-10-08T05:37:43.727258+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.727258+0000', 'last_undegraded': '2023-10-08T05:37:43.727258+0000', 'last_fullsized': '2023-10-08T05:37:43.727258+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T05:37:50.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-10-08T05:37:50.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.719846+0000', 'last_change': '2023-10-08T05:37:43.719846+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.719846+0000', 'last_undegraded': '2023-10-08T05:37:43.719846+0000', 'last_fullsized': '2023-10-08T05:37:43.719846+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T05:37:50.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-10-08T05:37:50.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.729757+0000', 'last_change': '2023-10-08T05:37:43.729757+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.729757+0000', 'last_undegraded': '2023-10-08T05:37:43.729757+0000', 'last_fullsized': '2023-10-08T05:37:43.729757+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T05:37:50.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-10-08T05:37:50.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.721280+0000', 'last_change': '2023-10-08T05:37:43.721280+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.721280+0000', 'last_undegraded': '2023-10-08T05:37:43.721280+0000', 'last_fullsized': '2023-10-08T05:37:43.721280+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T05:37:50.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-10-08T05:37:50.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.735316+0000', 'last_change': '2023-10-08T05:37:43.735316+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.735316+0000', 'last_undegraded': '2023-10-08T05:37:43.735316+0000', 'last_fullsized': '2023-10-08T05:37:43.735316+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-10-08T05:37:50.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.719803+0000', 'last_change': '2023-10-08T05:37:43.719803+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.719803+0000', 'last_undegraded': '2023-10-08T05:37:43.719803+0000', 'last_fullsized': '2023-10-08T05:37:43.719803+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T05:37:50.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-10-08T05:37:50.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.728203+0000', 'last_change': '2023-10-08T05:37:43.728203+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.728203+0000', 'last_undegraded': '2023-10-08T05:37:43.728203+0000', 'last_fullsized': '2023-10-08T05:37:43.728203+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T05:37:50.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-10-08T05:37:50.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.733504+0000', 'last_change': '2023-10-08T05:37:43.733504+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.733504+0000', 'last_undegraded': '2023-10-08T05:37:43.733504+0000', 'last_fullsized': '2023-10-08T05:37:43.733504+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T05:37:50.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-10-08T05:37:50.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.720525+0000', 'last_change': '2023-10-08T05:37:43.720525+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.720525+0000', 'last_undegraded': '2023-10-08T05:37:43.720525+0000', 'last_fullsized': '2023-10-08T05:37:43.720525+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T05:37:50.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-10-08T05:37:50.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.723702+0000', 'last_change': '2023-10-08T05:37:43.723702+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.723702+0000', 'last_undegraded': '2023-10-08T05:37:43.723702+0000', 'last_fullsized': '2023-10-08T05:37:43.723702+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T05:37:50.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-10-08T05:37:50.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.731047+0000', 'last_change': '2023-10-08T05:37:43.731047+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.731047+0000', 'last_undegraded': '2023-10-08T05:37:43.731047+0000', 'last_fullsized': '2023-10-08T05:37:43.731047+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T05:37:50.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-10-08T05:37:50.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.729623+0000', 'last_change': '2023-10-08T05:37:43.729623+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.729623+0000', 'last_undegraded': '2023-10-08T05:37:43.729623+0000', 'last_fullsized': '2023-10-08T05:37:43.729623+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T05:37:50.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-10-08T05:37:50.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.737617+0000', 'last_change': '2023-10-08T05:37:43.737617+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.737617+0000', 'last_undegraded': '2023-10-08T05:37:43.737617+0000', 'last_fullsized': '2023-10-08T05:37:43.737617+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T05:37:50.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-10-08T05:37:50.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2023-10-08T05:37:43.741498+0000', 'last_change': '2023-10-08T05:37:43.741498+0000', 'last_active': '2023-10-08T05:37:43.697452+0000', 'last_peered': '2023-10-08T05:37:43.697452+0000', 'last_clean': '2023-10-08T05:37:43.697452+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T05:37:43.741498+0000', 'last_undegraded': '2023-10-08T05:37:43.741498+0000', 'last_fullsized': '2023-10-08T05:37:43.741498+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'last_clean_scrub_stamp': '2023-10-08T05:37:43.697452+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T05:37:50.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T05:37:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.678+0000 7feee17b0640 1 Processor -- start 2023-10-08T05:37:50.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.678+0000 7feee17b0640 1 -- start start 2023-10-08T05:37:50.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.680+0000 7feee17b0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feedc151aa0 0x7feedc0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.680+0000 7feee17b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c93c0 0x7feedc15e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.680+0000 7feee17b0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc0b1320 con 0x7feedc15ec30 2023-10-08T05:37:50.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.680+0000 7feee17b0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc0b0a80 con 0x7feedc151aa0 2023-10-08T05:37:50.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.680+0000 7feee17b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b0d60 con 0x7feedc0c93c0 2023-10-08T05:37:50.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.680+0000 7feedaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feedc151aa0 0x7feedc0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.680+0000 7feedaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feedc151aa0 0x7feedc0c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37618/0 (socket says 172.21.15.70:37618) 2023-10-08T05:37:50.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.680+0000 7feedaffd640 1 -- 172.21.15.70:0/244768977 learned_addr learned my addr 172.21.15.70:0/244768977 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:50.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.681+0000 7feeda7fc640 1 --2- 172.21.15.70:0/244768977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c93c0 0x7feedc15e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.681+0000 7feedaffd640 1 -- 172.21.15.70:0/244768977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c93c0 msgr2=0x7feedc15e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:50.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.681+0000 7feedaffd640 1 --2- 172.21.15.70:0/244768977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c93c0 0x7feedc15e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.681+0000 7feedaffd640 1 -- 172.21.15.70:0/244768977 >> v1:172.21.15.181:6789/0 conn(0x7feedc15ec30 legacy=0x7feedc15b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.681+0000 7feedaffd640 1 -- 172.21.15.70:0/244768977 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc0b1040 con 0x7feedc151aa0 2023-10-08T05:37:50.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.683+0000 7feedaffd640 1 --2- 172.21.15.70:0/244768977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc151aa0 0x7feedc0c8e80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feecc009870 tx=0x7feecc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8ed57bac88ed50 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:50.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.683+0000 7feed9ffb640 1 -- 172.21.15.70:0/244768977 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc031640 con 0x7feedc151aa0 2023-10-08T05:37:50.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.683+0000 7feed9ffb640 1 -- 172.21.15.70:0/244768977 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc0317a0 con 0x7feedc151aa0 2023-10-08T05:37:50.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.683+0000 7feed9ffb640 1 -- 172.21.15.70:0/244768977 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc031b30 con 0x7feedc151aa0 2023-10-08T05:37:50.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.683+0000 7feee17b0640 1 -- 172.21.15.70:0/244768977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc151aa0 msgr2=0x7feedc0c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.683+0000 7feee17b0640 1 --2- 172.21.15.70:0/244768977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc151aa0 0x7feedc0c8e80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feecc009870 tx=0x7feecc02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.684+0000 7feee17b0640 1 -- 172.21.15.70:0/244768977 shutdown_connections 2023-10-08T05:37:50.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.684+0000 7feee17b0640 1 --2- 172.21.15.70:0/244768977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc0c93c0 0x7feedc15e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.684+0000 7feee17b0640 1 --2- 172.21.15.70:0/244768977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc151aa0 0x7feedc0c8e80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.684+0000 7feee17b0640 1 -- 172.21.15.70:0/244768977 >> 172.21.15.70:0/244768977 conn(0x7feedc0b20b0 msgr2=0x7feedc069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:50.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.684+0000 7feee17b0640 1 -- 172.21.15.70:0/244768977 shutdown_connections 2023-10-08T05:37:50.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.684+0000 7feee17b0640 1 -- 172.21.15.70:0/244768977 wait complete. 2023-10-08T05:37:50.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.685+0000 7feee17b0640 1 Processor -- start 2023-10-08T05:37:50.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.686+0000 7feee17b0640 1 -- start start 2023-10-08T05:37:50.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.686+0000 7feee17b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc0c93c0 0x7feedc14d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.686+0000 7feee17b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc151aa0 0x7feedc0704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feee17b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc15ec30 0x7feedc0709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feee17b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc14d810 con 0x7feedc0c93c0 2023-10-08T05:37:50.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feee17b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0c7610 con 0x7feedc151aa0 2023-10-08T05:37:50.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feee17b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0c6d70 con 0x7feedc15ec30 2023-10-08T05:37:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feedaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc0c93c0 0x7feedc14d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feedaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc0c93c0 0x7feedc14d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37634/0 (socket says 172.21.15.70:37634) 2023-10-08T05:37:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feedaffd640 1 -- 172.21.15.70:0/559689985 learned_addr learned my addr 172.21.15.70:0/559689985 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feedaffd640 1 -- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc15ec30 msgr2=0x7feedc0709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feedaffd640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc15ec30 0x7feedc0709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feedaffd640 1 -- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc151aa0 msgr2=0x7feedc0704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feedaffd640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc151aa0 0x7feedc0704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.687+0000 7feedaffd640 1 -- 172.21.15.70:0/559689985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc0094d0 con 0x7feedc0c93c0 2023-10-08T05:37:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.688+0000 7feedaffd640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc0c93c0 0x7feedc14d2d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feecc000c00 tx=0x7feecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.688+0000 7feec37fe640 1 -- 172.21.15.70:0/559689985 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc03ac60 con 0x7feedc0c93c0 2023-10-08T05:37:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.688+0000 7feec37fe640 1 -- 172.21.15.70:0/559689985 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc03adc0 con 0x7feedc0c93c0 2023-10-08T05:37:50.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.688+0000 7feec37fe640 1 -- 172.21.15.70:0/559689985 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc044630 con 0x7feedc0c93c0 2023-10-08T05:37:50.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.688+0000 7feee17b0640 1 -- 172.21.15.70:0/559689985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc0c7050 con 0x7feedc0c93c0 2023-10-08T05:37:50.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.688+0000 7feee17b0640 1 -- 172.21.15.70:0/559689985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc1522a0 con 0x7feedc0c93c0 2023-10-08T05:37:50.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.689+0000 7feee17b0640 1 -- 172.21.15.70:0/559689985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc152490 con 0x7feedc0c93c0 2023-10-08T05:37:50.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.690+0000 7feec37fe640 1 -- 172.21.15.70:0/559689985 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7feecc02f050 con 0x7feedc0c93c0 2023-10-08T05:37:50.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.691+0000 7feec37fe640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feec806ae70 0x7feec806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:50.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.691+0000 7feec37fe640 1 -- 172.21.15.70:0/559689985 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7feecc0ab650 con 0x7feedc0c93c0 2023-10-08T05:37:50.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.692+0000 7feeda7fc640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feec806ae70 0x7feec806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:50.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.693+0000 7feeda7fc640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feec806ae70 0x7feec806d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feedc155250 tx=0x7feed0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:50.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.695+0000 7feec37fe640 1 -- 172.21.15.70:0/559689985 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feecc0765b0 con 0x7feedc0c93c0 2023-10-08T05:37:50.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.946+0000 7feee17b0640 1 -- 172.21.15.70:0/559689985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7feedc067490 con 0x7feedc0c93c0 2023-10-08T05:37:50.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.947+0000 7feec37fe640 1 -- 172.21.15.70:0/559689985 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1033 (secure 0 0 0) 0x7feecc079bd0 con 0x7feedc0c93c0 2023-10-08T05:37:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:50.950 INFO:teuthology.orchestra.run.smithi070.stdout:{"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":119,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1696743363,"num_in_osds":8,"osd_in_since":1696743355,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":242946048,"bytes_avail":772851167232,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5304}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-10-08T05:37:17.095979+0000","services":{}},"progress_events":{}} 2023-10-08T05:37:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 -- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feec806ae70 msgr2=0x7feec806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feec806ae70 0x7feec806d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feedc155250 tx=0x7feed0008040 comp rx=0 tx=0).stop 2023-10-08T05:37:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 -- 172.21.15.70:0/559689985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc0c93c0 msgr2=0x7feedc14d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc0c93c0 0x7feedc14d2d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feecc000c00 tx=0x7feecc0046c0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 -- 172.21.15.70:0/559689985 shutdown_connections 2023-10-08T05:37:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7feec806ae70 0x7feec806d330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feedc15ec30 0x7feedc0709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feedc151aa0 0x7feedc0704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 --2- 172.21.15.70:0/559689985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feedc0c93c0 0x7feedc14d2d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 -- 172.21.15.70:0/559689985 >> 172.21.15.70:0/559689985 conn(0x7feedc0b20b0 msgr2=0x7feedc069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 -- 172.21.15.70:0/559689985 shutdown_connections 2023-10-08T05:37:50.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:50.950+0000 7feee17b0640 1 -- 172.21.15.70:0/559689985 wait complete. 2023-10-08T05:37:50.970 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T05:37:50.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:37:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.170+0000 7f2906d4c640 1 Processor -- start 2023-10-08T05:37:51.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.171+0000 7f2906d4c640 1 -- start start 2023-10-08T05:37:51.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.172+0000 7f2906d4c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29001600d0 0x7f29000d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:51.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.172+0000 7f2906d4c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29000d02f0 0x7f29000b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:51.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.172+0000 7f2906d4c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29000d1750 con 0x7f29000d11d0 2023-10-08T05:37:51.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.172+0000 7f2906d4c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29000d1a30 con 0x7f29001600d0 2023-10-08T05:37:51.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.172+0000 7f2906d4c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29000d1d10 con 0x7f29000d02f0 2023-10-08T05:37:51.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.173+0000 7f2904ae8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29001600d0 0x7f29000d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:51.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.173+0000 7f2904ae8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29001600d0 0x7f29000d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37648/0 (socket says 172.21.15.70:37648) 2023-10-08T05:37:51.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.173+0000 7f2904ae8640 1 -- 172.21.15.70:0/1487075003 learned_addr learned my addr 172.21.15.70:0/1487075003 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.173+0000 7f2904ae8640 1 -- 172.21.15.70:0/1487075003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29000d02f0 msgr2=0x7f29000b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.173+0000 7f2904ae8640 1 --2- 172.21.15.70:0/1487075003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29000d02f0 0x7f29000b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.173+0000 7f2904ae8640 1 -- 172.21.15.70:0/1487075003 >> v1:172.21.15.181:6789/0 conn(0x7f29000d11d0 legacy=0x7f29000cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.173+0000 7f2904ae8640 1 -- 172.21.15.70:0/1487075003 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29000d1ff0 con 0x7f29001600d0 2023-10-08T05:37:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.174+0000 7f2904ae8640 1 --2- 172.21.15.70:0/1487075003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 0x7f29000d0c90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009870 tx=0x7f28f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c1c07f0583eed6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.174+0000 7f28ff7fe640 1 -- 172.21.15.70:0/1487075003 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4031640 con 0x7f29001600d0 2023-10-08T05:37:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.174+0000 7f28ff7fe640 1 -- 172.21.15.70:0/1487075003 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f40317a0 con 0x7f29001600d0 2023-10-08T05:37:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.174+0000 7f28ff7fe640 1 -- 172.21.15.70:0/1487075003 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4031b30 con 0x7f29001600d0 2023-10-08T05:37:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.175+0000 7f2906d4c640 1 -- 172.21.15.70:0/1487075003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 msgr2=0x7f29000d0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.175+0000 7f2906d4c640 1 --2- 172.21.15.70:0/1487075003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 0x7f29000d0c90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009870 tx=0x7f28f402b8f0 comp rx=0 tx=0).stop 2023-10-08T05:37:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.175+0000 7f2906d4c640 1 -- 172.21.15.70:0/1487075003 shutdown_connections 2023-10-08T05:37:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.175+0000 7f2906d4c640 1 --2- 172.21.15.70:0/1487075003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29000d02f0 0x7f29000b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.175+0000 7f2906d4c640 1 --2- 172.21.15.70:0/1487075003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 0x7f29000d0c90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:51.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.175+0000 7f2906d4c640 1 -- 172.21.15.70:0/1487075003 >> 172.21.15.70:0/1487075003 conn(0x7f29000718a0 msgr2=0x7f2900071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:51.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.175+0000 7f2906d4c640 1 -- 172.21.15.70:0/1487075003 shutdown_connections 2023-10-08T05:37:51.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.175+0000 7f2906d4c640 1 -- 172.21.15.70:0/1487075003 wait complete. 2023-10-08T05:37:51.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.177+0000 7f2906d4c640 1 Processor -- start 2023-10-08T05:37:51.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.177+0000 7f2906d4c640 1 -- start start 2023-10-08T05:37:51.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.178+0000 7f2906d4c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29000d02f0 0x7f290015c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.178+0000 7f2906d4c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29000d11d0 0x7f29001474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.178+0000 7f2906d4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 0x7f29000cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.178+0000 7f2906d4c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290015cbc0 con 0x7f29001600d0 2023-10-08T05:37:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.178+0000 7f2906d4c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29001479f0 con 0x7f29000d11d0 2023-10-08T05:37:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.178+0000 7f2906d4c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2900157e80 con 0x7f29000d02f0 2023-10-08T05:37:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.178+0000 7f29052e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 0x7f29000cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f29052e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 0x7f29000cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37656/0 (socket says 172.21.15.70:37656) 2023-10-08T05:37:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f29052e9640 1 -- 172.21.15.70:0/2558770778 learned_addr learned my addr 172.21.15.70:0/2558770778 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f2904ae8640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29000d02f0 0x7f290015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f28fffff640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29000d11d0 0x7f29001474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f29052e9640 1 -- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29000d02f0 msgr2=0x7f290015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f29052e9640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29000d02f0 0x7f290015c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f29052e9640 1 -- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29000d11d0 msgr2=0x7f29001474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f29052e9640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29000d11d0 0x7f29001474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f29052e9640 1 -- 172.21.15.70:0/2558770778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f40094d0 con 0x7f29001600d0 2023-10-08T05:37:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f28fffff640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29000d11d0 0x7f29001474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.179+0000 7f29052e9640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 0x7f29000cab10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f28f800b970 tx=0x7f28f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.180+0000 7f28fd7fa640 1 -- 172.21.15.70:0/2558770778 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f800cb40 con 0x7f29001600d0 2023-10-08T05:37:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.180+0000 7f28fd7fa640 1 -- 172.21.15.70:0/2558770778 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f800ccc0 con 0x7f29001600d0 2023-10-08T05:37:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.180+0000 7f2906d4c640 1 -- 172.21.15.70:0/2558770778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2900144e60 con 0x7f29001600d0 2023-10-08T05:37:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.180+0000 7f28fd7fa640 1 -- 172.21.15.70:0/2558770778 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8014680 con 0x7f29001600d0 2023-10-08T05:37:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.180+0000 7f2906d4c640 1 -- 172.21.15.70:0/2558770778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2900145420 con 0x7f29001600d0 2023-10-08T05:37:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.181+0000 7f2906d4c640 1 -- 172.21.15.70:0/2558770778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2900067490 con 0x7f29001600d0 2023-10-08T05:37:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.182+0000 7f28fd7fa640 1 -- 172.21.15.70:0/2558770778 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f28f801d3b0 con 0x7f29001600d0 2023-10-08T05:37:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.183+0000 7f28fd7fa640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f28e406aea0 0x7f28e406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.183+0000 7f28fd7fa640 1 -- 172.21.15.70:0/2558770778 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f28f8089670 con 0x7f29001600d0 2023-10-08T05:37:51.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.184+0000 7f2904ae8640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f28e406aea0 0x7f28e406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.185+0000 7f2904ae8640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f28e406aea0 0x7f28e406d360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f28f4036000 tx=0x7f28f403c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:51.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.193+0000 7f28fd7fa640 1 -- 172.21.15.70:0/2558770778 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28f80545f0 con 0x7f29001600d0 2023-10-08T05:37:51.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.377+0000 7f2906d4c640 1 -- 172.21.15.70:0/2558770778 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29000b8ad0 con 0x7f28e406aea0 2023-10-08T05:37:51.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.399+0000 7f28fd7fa640 1 -- 172.21.15.70:0/2558770778 <== mgr.4105 v2:172.21.15.181:6832/2737793492 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287141 (secure 0 0 0) 0x7f28f4031a90 con 0x7f28e406aea0 2023-10-08T05:37:51.404 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:51.405 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:37:51.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.403+0000 7f2906d4c640 1 -- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f28e406aea0 msgr2=0x7f28e406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:51.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.403+0000 7f2906d4c640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f28e406aea0 0x7f28e406d360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f28f4036000 tx=0x7f28f403c040 comp rx=0 tx=0).stop 2023-10-08T05:37:51.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.403+0000 7f2906d4c640 1 -- 172.21.15.70:0/2558770778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 msgr2=0x7f29000cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:51.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.403+0000 7f2906d4c640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 0x7f29000cab10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f28f800b970 tx=0x7f28f800be30 comp rx=0 tx=0).stop 2023-10-08T05:37:51.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.404+0000 7f2906d4c640 1 -- 172.21.15.70:0/2558770778 shutdown_connections 2023-10-08T05:37:51.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.404+0000 7f2906d4c640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f28e406aea0 0x7f28e406d360 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.404+0000 7f2906d4c640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29001600d0 0x7f29000cab10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.404+0000 7f2906d4c640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29000d11d0 0x7f29001474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.404+0000 7f2906d4c640 1 --2- 172.21.15.70:0/2558770778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29000d02f0 0x7f290015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.404+0000 7f2906d4c640 1 -- 172.21.15.70:0/2558770778 >> 172.21.15.70:0/2558770778 conn(0x7f29000718a0 msgr2=0x7f290015f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.404+0000 7f2906d4c640 1 -- 172.21.15.70:0/2558770778 shutdown_connections 2023-10-08T05:37:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:51.404+0000 7f2906d4c640 1 -- 172.21.15.70:0/2558770778 wait complete. 2023-10-08T05:37:51.422 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-10-08T05:37:51.106853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237580,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737140,"statfs":{"total":773094113280,"available":772850831360,"internally_reserved":0,"allocated":4857856,"data_stored":2736268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.392861"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.746144+0000","last_change":"2023-10-08T05:37:44.750329+0000","last_active":"2023-10-08T05:37:45.746144+0000","last_peered":"2023-10-08T05:37:45.746144+0000","last_clean":"2023-10-08T05:37:45.746144+0000","last_became_active":"2023-10-08T05:37:44.750043+0000","last_became_peered":"2023-10-08T05:37:44.750043+0000","last_unstale":"2023-10-08T05:37:45.746144+0000","last_undegraded":"2023-10-08T05:37:45.746144+0000","last_fullsized":"2023-10-08T05:37:45.746144+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:11.877767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745958+0000","last_change":"2023-10-08T05:37:44.753676+0000","last_active":"2023-10-08T05:37:45.745958+0000","last_peered":"2023-10-08T05:37:45.745958+0000","last_clean":"2023-10-08T05:37:45.745958+0000","last_became_active":"2023-10-08T05:37:44.753212+0000","last_became_peered":"2023-10-08T05:37:44.753212+0000","last_unstale":"2023-10-08T05:37:45.745958+0000","last_undegraded":"2023-10-08T05:37:45.745958+0000","last_fullsized":"2023-10-08T05:37:45.745958+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:42.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260601+0000","last_change":"2023-10-08T05:37:44.758547+0000","last_active":"2023-10-08T05:37:46.260601+0000","last_peered":"2023-10-08T05:37:46.260601+0000","last_clean":"2023-10-08T05:37:46.260601+0000","last_became_active":"2023-10-08T05:37:44.754894+0000","last_became_peered":"2023-10-08T05:37:44.754894+0000","last_unstale":"2023-10-08T05:37:46.260601+0000","last_undegraded":"2023-10-08T05:37:46.260601+0000","last_fullsized":"2023-10-08T05:37:46.260601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:55.053932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865289+0000","last_change":"2023-10-08T05:37:44.740778+0000","last_active":"2023-10-08T05:37:45.865289+0000","last_peered":"2023-10-08T05:37:45.865289+0000","last_clean":"2023-10-08T05:37:45.865289+0000","last_became_active":"2023-10-08T05:37:44.740478+0000","last_became_peered":"2023-10-08T05:37:44.740478+0000","last_unstale":"2023-10-08T05:37:45.865289+0000","last_undegraded":"2023-10-08T05:37:45.865289+0000","last_fullsized":"2023-10-08T05:37:45.865289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:13.103220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865209+0000","last_change":"2023-10-08T05:37:44.740649+0000","last_active":"2023-10-08T05:37:45.865209+0000","last_peered":"2023-10-08T05:37:45.865209+0000","last_clean":"2023-10-08T05:37:45.865209+0000","last_became_active":"2023-10-08T05:37:44.740308+0000","last_became_peered":"2023-10-08T05:37:44.740308+0000","last_unstale":"2023-10-08T05:37:45.865209+0000","last_undegraded":"2023-10-08T05:37:45.865209+0000","last_fullsized":"2023-10-08T05:37:45.865209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:28.575600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749536+0000","last_change":"2023-10-08T05:37:44.756366+0000","last_active":"2023-10-08T05:37:45.749536+0000","last_peered":"2023-10-08T05:37:45.749536+0000","last_clean":"2023-10-08T05:37:45.749536+0000","last_became_active":"2023-10-08T05:37:44.756109+0000","last_became_peered":"2023-10-08T05:37:44.756109+0000","last_unstale":"2023-10-08T05:37:45.749536+0000","last_undegraded":"2023-10-08T05:37:45.749536+0000","last_fullsized":"2023-10-08T05:37:45.749536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:52.079157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770337+0000","last_change":"2023-10-08T05:37:44.758894+0000","last_active":"2023-10-08T05:37:45.770337+0000","last_peered":"2023-10-08T05:37:45.770337+0000","last_clean":"2023-10-08T05:37:45.770337+0000","last_became_active":"2023-10-08T05:37:44.758570+0000","last_became_peered":"2023-10-08T05:37:44.758570+0000","last_unstale":"2023-10-08T05:37:45.770337+0000","last_undegraded":"2023-10-08T05:37:45.770337+0000","last_fullsized":"2023-10-08T05:37:45.770337+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:19.962298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745267+0000","last_change":"2023-10-08T05:37:44.753088+0000","last_active":"2023-10-08T05:37:45.745267+0000","last_peered":"2023-10-08T05:37:45.745267+0000","last_clean":"2023-10-08T05:37:45.745267+0000","last_became_active":"2023-10-08T05:37:44.752787+0000","last_became_peered":"2023-10-08T05:37:44.752787+0000","last_unstale":"2023-10-08T05:37:45.745267+0000","last_undegraded":"2023-10-08T05:37:45.745267+0000","last_fullsized":"2023-10-08T05:37:45.745267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:46.902349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.261126+0000","last_change":"2023-10-08T05:37:44.762001+0000","last_active":"2023-10-08T05:37:46.261126+0000","last_peered":"2023-10-08T05:37:46.261126+0000","last_clean":"2023-10-08T05:37:46.261126+0000","last_became_active":"2023-10-08T05:37:44.761719+0000","last_became_peered":"2023-10-08T05:37:44.761719+0000","last_unstale":"2023-10-08T05:37:46.261126+0000","last_undegraded":"2023-10-08T05:37:46.261126+0000","last_fullsized":"2023-10-08T05:37:46.261126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:29.716943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770757+0000","last_change":"2023-10-08T05:37:44.748634+0000","last_active":"2023-10-08T05:37:45.770757+0000","last_peered":"2023-10-08T05:37:45.770757+0000","last_clean":"2023-10-08T05:37:45.770757+0000","last_became_active":"2023-10-08T05:37:44.748267+0000","last_became_peered":"2023-10-08T05:37:44.748267+0000","last_unstale":"2023-10-08T05:37:45.770757+0000","last_undegraded":"2023-10-08T05:37:45.770757+0000","last_fullsized":"2023-10-08T05:37:45.770757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:11.877767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770790+0000","last_change":"2023-10-08T05:37:44.752195+0000","last_active":"2023-10-08T05:37:45.770790+0000","last_peered":"2023-10-08T05:37:45.770790+0000","last_clean":"2023-10-08T05:37:45.770790+0000","last_became_active":"2023-10-08T05:37:44.750351+0000","last_became_peered":"2023-10-08T05:37:44.750351+0000","last_unstale":"2023-10-08T05:37:45.770790+0000","last_undegraded":"2023-10-08T05:37:45.770790+0000","last_fullsized":"2023-10-08T05:37:45.770790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:11:54.910375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260755+0000","last_change":"2023-10-08T05:37:44.754914+0000","last_active":"2023-10-08T05:37:46.260755+0000","last_peered":"2023-10-08T05:37:46.260755+0000","last_clean":"2023-10-08T05:37:46.260755+0000","last_became_active":"2023-10-08T05:37:44.754614+0000","last_became_peered":"2023-10-08T05:37:44.754614+0000","last_unstale":"2023-10-08T05:37:46.260755+0000","last_undegraded":"2023-10-08T05:37:46.260755+0000","last_fullsized":"2023-10-08T05:37:46.260755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:52.079157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770133+0000","last_change":"2023-10-08T05:37:44.751890+0000","last_active":"2023-10-08T05:37:45.770133+0000","last_peered":"2023-10-08T05:37:45.770133+0000","last_clean":"2023-10-08T05:37:45.770133+0000","last_became_active":"2023-10-08T05:37:44.751555+0000","last_became_peered":"2023-10-08T05:37:44.751555+0000","last_unstale":"2023-10-08T05:37:45.770133+0000","last_undegraded":"2023-10-08T05:37:45.770133+0000","last_fullsized":"2023-10-08T05:37:45.770133+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:01.744941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.261243+0000","last_change":"2023-10-08T05:37:44.742430+0000","last_active":"2023-10-08T05:37:46.261243+0000","last_peered":"2023-10-08T05:37:46.261243+0000","last_clean":"2023-10-08T05:37:46.261243+0000","last_became_active":"2023-10-08T05:37:44.742070+0000","last_became_peered":"2023-10-08T05:37:44.742070+0000","last_unstale":"2023-10-08T05:37:46.261243+0000","last_undegraded":"2023-10-08T05:37:46.261243+0000","last_fullsized":"2023-10-08T05:37:46.261243+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:35.000424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770011+0000","last_change":"2023-10-08T05:37:44.756459+0000","last_active":"2023-10-08T05:37:45.770011+0000","last_peered":"2023-10-08T05:37:45.770011+0000","last_clean":"2023-10-08T05:37:45.770011+0000","last_became_active":"2023-10-08T05:37:44.756163+0000","last_became_peered":"2023-10-08T05:37:44.756163+0000","last_unstale":"2023-10-08T05:37:45.770011+0000","last_undegraded":"2023-10-08T05:37:45.770011+0000","last_fullsized":"2023-10-08T05:37:45.770011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:26.951190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926333+0000","last_change":"2023-10-08T05:37:44.762058+0000","last_active":"2023-10-08T05:37:45.926333+0000","last_peered":"2023-10-08T05:37:45.926333+0000","last_clean":"2023-10-08T05:37:45.926333+0000","last_became_active":"2023-10-08T05:37:44.761794+0000","last_became_peered":"2023-10-08T05:37:44.761794+0000","last_unstale":"2023-10-08T05:37:45.926333+0000","last_undegraded":"2023-10-08T05:37:45.926333+0000","last_fullsized":"2023-10-08T05:37:45.926333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:29.716943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.927643+0000","last_change":"2023-10-08T05:37:44.760175+0000","last_active":"2023-10-08T05:37:45.927643+0000","last_peered":"2023-10-08T05:37:45.927643+0000","last_clean":"2023-10-08T05:37:45.927643+0000","last_became_active":"2023-10-08T05:37:44.759862+0000","last_became_peered":"2023-10-08T05:37:44.759862+0000","last_unstale":"2023-10-08T05:37:45.927643+0000","last_undegraded":"2023-10-08T05:37:45.927643+0000","last_fullsized":"2023-10-08T05:37:45.927643+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:19.962298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.753066+0000","last_change":"2023-10-08T05:37:44.746870+0000","last_active":"2023-10-08T05:37:45.753066+0000","last_peered":"2023-10-08T05:37:45.753066+0000","last_clean":"2023-10-08T05:37:45.753066+0000","last_became_active":"2023-10-08T05:37:44.746488+0000","last_became_peered":"2023-10-08T05:37:44.746488+0000","last_unstale":"2023-10-08T05:37:45.753066+0000","last_undegraded":"2023-10-08T05:37:45.753066+0000","last_fullsized":"2023-10-08T05:37:45.753066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:16:38.155637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865439+0000","last_change":"2023-10-08T05:37:44.770316+0000","last_active":"2023-10-08T05:37:45.865439+0000","last_peered":"2023-10-08T05:37:45.865439+0000","last_clean":"2023-10-08T05:37:45.865439+0000","last_became_active":"2023-10-08T05:37:44.770017+0000","last_became_peered":"2023-10-08T05:37:44.770017+0000","last_unstale":"2023-10-08T05:37:45.865439+0000","last_undegraded":"2023-10-08T05:37:45.865439+0000","last_fullsized":"2023-10-08T05:37:45.865439+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:54.627946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.752578+0000","last_change":"2023-10-08T05:37:44.746048+0000","last_active":"2023-10-08T05:37:45.752578+0000","last_peered":"2023-10-08T05:37:45.752578+0000","last_clean":"2023-10-08T05:37:45.752578+0000","last_became_active":"2023-10-08T05:37:44.745781+0000","last_became_peered":"2023-10-08T05:37:44.745781+0000","last_unstale":"2023-10-08T05:37:45.752578+0000","last_undegraded":"2023-10-08T05:37:45.752578+0000","last_fullsized":"2023-10-08T05:37:45.752578+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:45.569743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770697+0000","last_change":"2023-10-08T05:37:44.770242+0000","last_active":"2023-10-08T05:37:45.770697+0000","last_peered":"2023-10-08T05:37:45.770697+0000","last_clean":"2023-10-08T05:37:45.770697+0000","last_became_active":"2023-10-08T05:37:44.769915+0000","last_became_peered":"2023-10-08T05:37:44.769915+0000","last_unstale":"2023-10-08T05:37:45.770697+0000","last_undegraded":"2023-10-08T05:37:45.770697+0000","last_fullsized":"2023-10-08T05:37:45.770697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:04.840446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.261690+0000","last_change":"2023-10-08T05:37:44.749297+0000","last_active":"2023-10-08T05:37:46.261690+0000","last_peered":"2023-10-08T05:37:46.261690+0000","last_clean":"2023-10-08T05:37:46.261690+0000","last_became_active":"2023-10-08T05:37:44.748939+0000","last_became_peered":"2023-10-08T05:37:44.748939+0000","last_unstale":"2023-10-08T05:37:46.261690+0000","last_undegraded":"2023-10-08T05:37:46.261690+0000","last_fullsized":"2023-10-08T05:37:46.261690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:49.881310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926490+0000","last_change":"2023-10-08T05:37:44.752426+0000","last_active":"2023-10-08T05:37:45.926490+0000","last_peered":"2023-10-08T05:37:45.926490+0000","last_clean":"2023-10-08T05:37:45.926490+0000","last_became_active":"2023-10-08T05:37:44.752138+0000","last_became_peered":"2023-10-08T05:37:44.752138+0000","last_unstale":"2023-10-08T05:37:45.926490+0000","last_undegraded":"2023-10-08T05:37:45.926490+0000","last_fullsized":"2023-10-08T05:37:45.926490+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:46.902349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865055+0000","last_change":"2023-10-08T05:37:44.745348+0000","last_active":"2023-10-08T05:37:45.865055+0000","last_peered":"2023-10-08T05:37:45.865055+0000","last_clean":"2023-10-08T05:37:45.865055+0000","last_became_active":"2023-10-08T05:37:44.745086+0000","last_became_peered":"2023-10-08T05:37:44.745086+0000","last_unstale":"2023-10-08T05:37:45.865055+0000","last_undegraded":"2023-10-08T05:37:45.865055+0000","last_fullsized":"2023-10-08T05:37:45.865055+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:35.619176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.753164+0000","last_change":"2023-10-08T05:37:44.745696+0000","last_active":"2023-10-08T05:37:45.753164+0000","last_peered":"2023-10-08T05:37:45.753164+0000","last_clean":"2023-10-08T05:37:45.753164+0000","last_became_active":"2023-10-08T05:37:44.745366+0000","last_became_peered":"2023-10-08T05:37:44.745366+0000","last_unstale":"2023-10-08T05:37:45.753164+0000","last_undegraded":"2023-10-08T05:37:45.753164+0000","last_fullsized":"2023-10-08T05:37:45.753164+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:23.416317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749534+0000","last_change":"2023-10-08T05:37:44.765748+0000","last_active":"2023-10-08T05:37:45.749534+0000","last_peered":"2023-10-08T05:37:45.749534+0000","last_clean":"2023-10-08T05:37:45.749534+0000","last_became_active":"2023-10-08T05:37:44.765432+0000","last_became_peered":"2023-10-08T05:37:44.765432+0000","last_unstale":"2023-10-08T05:37:45.749534+0000","last_undegraded":"2023-10-08T05:37:45.749534+0000","last_fullsized":"2023-10-08T05:37:45.749534+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:46.616809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.752741+0000","last_change":"2023-10-08T05:37:44.748548+0000","last_active":"2023-10-08T05:37:45.752741+0000","last_peered":"2023-10-08T05:37:45.752741+0000","last_clean":"2023-10-08T05:37:45.752741+0000","last_became_active":"2023-10-08T05:37:44.748263+0000","last_became_peered":"2023-10-08T05:37:44.748263+0000","last_unstale":"2023-10-08T05:37:45.752741+0000","last_undegraded":"2023-10-08T05:37:45.752741+0000","last_fullsized":"2023-10-08T05:37:45.752741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:15.563184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770742+0000","last_change":"2023-10-08T05:37:44.757411+0000","last_active":"2023-10-08T05:37:45.770742+0000","last_peered":"2023-10-08T05:37:45.770742+0000","last_clean":"2023-10-08T05:37:45.770742+0000","last_became_active":"2023-10-08T05:37:44.757142+0000","last_became_peered":"2023-10-08T05:37:44.757142+0000","last_unstale":"2023-10-08T05:37:45.770742+0000","last_undegraded":"2023-10-08T05:37:45.770742+0000","last_fullsized":"2023-10-08T05:37:45.770742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:15.770940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.752743+0000","last_change":"2023-10-08T05:37:44.759411+0000","last_active":"2023-10-08T05:37:45.752743+0000","last_peered":"2023-10-08T05:37:45.752743+0000","last_clean":"2023-10-08T05:37:45.752743+0000","last_became_active":"2023-10-08T05:37:44.759055+0000","last_became_peered":"2023-10-08T05:37:44.759055+0000","last_unstale":"2023-10-08T05:37:45.752743+0000","last_undegraded":"2023-10-08T05:37:45.752743+0000","last_fullsized":"2023-10-08T05:37:45.752743+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:27.967603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.752761+0000","last_change":"2023-10-08T05:37:44.745364+0000","last_active":"2023-10-08T05:37:45.752761+0000","last_peered":"2023-10-08T05:37:45.752761+0000","last_clean":"2023-10-08T05:37:45.752761+0000","last_became_active":"2023-10-08T05:37:44.744992+0000","last_became_peered":"2023-10-08T05:37:44.744992+0000","last_unstale":"2023-10-08T05:37:45.752761+0000","last_undegraded":"2023-10-08T05:37:45.752761+0000","last_fullsized":"2023-10-08T05:37:45.752761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:11:29.573866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926718+0000","last_change":"2023-10-08T05:37:44.756655+0000","last_active":"2023-10-08T05:37:45.926718+0000","last_peered":"2023-10-08T05:37:45.926718+0000","last_clean":"2023-10-08T05:37:45.926718+0000","last_became_active":"2023-10-08T05:37:44.756382+0000","last_became_peered":"2023-10-08T05:37:44.756382+0000","last_unstale":"2023-10-08T05:37:45.926718+0000","last_undegraded":"2023-10-08T05:37:45.926718+0000","last_fullsized":"2023-10-08T05:37:45.926718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:20.696319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749690+0000","last_change":"2023-10-08T05:37:44.755632+0000","last_active":"2023-10-08T05:37:45.749690+0000","last_peered":"2023-10-08T05:37:45.749690+0000","last_clean":"2023-10-08T05:37:45.749690+0000","last_became_active":"2023-10-08T05:37:44.755346+0000","last_became_peered":"2023-10-08T05:37:44.755346+0000","last_unstale":"2023-10-08T05:37:45.749690+0000","last_undegraded":"2023-10-08T05:37:45.749690+0000","last_fullsized":"2023-10-08T05:37:45.749690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:29.716943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926632+0000","last_change":"2023-10-08T05:37:44.760945+0000","last_active":"2023-10-08T05:37:45.926632+0000","last_peered":"2023-10-08T05:37:45.926632+0000","last_clean":"2023-10-08T05:37:45.926632+0000","last_became_active":"2023-10-08T05:37:44.760689+0000","last_became_peered":"2023-10-08T05:37:44.760689+0000","last_unstale":"2023-10-08T05:37:45.926632+0000","last_undegraded":"2023-10-08T05:37:45.926632+0000","last_fullsized":"2023-10-08T05:37:45.926632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:56.535909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.768987+0000","last_change":"2023-10-08T05:37:43.746333+0000","last_active":"2023-10-08T05:37:44.768987+0000","last_peered":"2023-10-08T05:37:44.768987+0000","last_clean":"2023-10-08T05:37:44.768987+0000","last_became_active":"2023-10-08T05:37:43.746017+0000","last_became_peered":"2023-10-08T05:37:43.746017+0000","last_unstale":"2023-10-08T05:37:44.768987+0000","last_undegraded":"2023-10-08T05:37:44.768987+0000","last_fullsized":"2023-10-08T05:37:44.768987+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:34.606944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749926+0000","last_change":"2023-10-08T05:37:44.769935+0000","last_active":"2023-10-08T05:37:45.749926+0000","last_peered":"2023-10-08T05:37:45.749926+0000","last_clean":"2023-10-08T05:37:45.749926+0000","last_became_active":"2023-10-08T05:37:44.769592+0000","last_became_peered":"2023-10-08T05:37:44.769592+0000","last_unstale":"2023-10-08T05:37:45.749926+0000","last_undegraded":"2023-10-08T05:37:45.749926+0000","last_fullsized":"2023-10-08T05:37:45.749926+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:24.842983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.770431+0000","last_change":"2023-10-08T05:37:43.749778+0000","last_active":"2023-10-08T05:37:44.770431+0000","last_peered":"2023-10-08T05:37:44.770431+0000","last_clean":"2023-10-08T05:37:44.770431+0000","last_became_active":"2023-10-08T05:37:43.749435+0000","last_became_peered":"2023-10-08T05:37:43.749435+0000","last_unstale":"2023-10-08T05:37:44.770431+0000","last_undegraded":"2023-10-08T05:37:44.770431+0000","last_fullsized":"2023-10-08T05:37:44.770431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.262299+0000","last_change":"2023-10-08T05:37:44.740956+0000","last_active":"2023-10-08T05:37:46.262299+0000","last_peered":"2023-10-08T05:37:46.262299+0000","last_clean":"2023-10-08T05:37:46.262299+0000","last_became_active":"2023-10-08T05:37:44.740518+0000","last_became_peered":"2023-10-08T05:37:44.740518+0000","last_unstale":"2023-10-08T05:37:46.262299+0000","last_undegraded":"2023-10-08T05:37:46.262299+0000","last_fullsized":"2023-10-08T05:37:46.262299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:52.079157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.772569+0000","last_change":"2023-10-08T05:37:43.749986+0000","last_active":"2023-10-08T05:37:44.772569+0000","last_peered":"2023-10-08T05:37:44.772569+0000","last_clean":"2023-10-08T05:37:44.772569+0000","last_became_active":"2023-10-08T05:37:43.749718+0000","last_became_peered":"2023-10-08T05:37:43.749718+0000","last_unstale":"2023-10-08T05:37:44.772569+0000","last_undegraded":"2023-10-08T05:37:44.772569+0000","last_fullsized":"2023-10-08T05:37:44.772569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:30.227070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.746117+0000","last_change":"2023-10-08T05:37:44.759365+0000","last_active":"2023-10-08T05:37:45.746117+0000","last_peered":"2023-10-08T05:37:45.746117+0000","last_clean":"2023-10-08T05:37:45.746117+0000","last_became_active":"2023-10-08T05:37:44.759068+0000","last_became_peered":"2023-10-08T05:37:44.759068+0000","last_unstale":"2023-10-08T05:37:45.746117+0000","last_undegraded":"2023-10-08T05:37:45.746117+0000","last_fullsized":"2023-10-08T05:37:45.746117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:04.840446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.769723+0000","last_change":"2023-10-08T05:37:43.742613+0000","last_active":"2023-10-08T05:37:44.769723+0000","last_peered":"2023-10-08T05:37:44.769723+0000","last_clean":"2023-10-08T05:37:44.769723+0000","last_became_active":"2023-10-08T05:37:43.742232+0000","last_became_peered":"2023-10-08T05:37:43.742232+0000","last_unstale":"2023-10-08T05:37:44.769723+0000","last_undegraded":"2023-10-08T05:37:44.769723+0000","last_fullsized":"2023-10-08T05:37:44.769723+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:11:28.561634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260288+0000","last_change":"2023-10-08T05:37:44.750080+0000","last_active":"2023-10-08T05:37:46.260288+0000","last_peered":"2023-10-08T05:37:46.260288+0000","last_clean":"2023-10-08T05:37:46.260288+0000","last_became_active":"2023-10-08T05:37:44.749776+0000","last_became_peered":"2023-10-08T05:37:44.749776+0000","last_unstale":"2023-10-08T05:37:46.260288+0000","last_undegraded":"2023-10-08T05:37:46.260288+0000","last_fullsized":"2023-10-08T05:37:46.260288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:31.239302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.770777+0000","last_change":"2023-10-08T05:37:43.743385+0000","last_active":"2023-10-08T05:37:44.770777+0000","last_peered":"2023-10-08T05:37:44.770777+0000","last_clean":"2023-10-08T05:37:44.770777+0000","last_became_active":"2023-10-08T05:37:43.743002+0000","last_became_peered":"2023-10-08T05:37:43.743002+0000","last_unstale":"2023-10-08T05:37:44.770777+0000","last_undegraded":"2023-10-08T05:37:44.770777+0000","last_fullsized":"2023-10-08T05:37:44.770777+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:19.684087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770524+0000","last_change":"2023-10-08T05:37:44.770626+0000","last_active":"2023-10-08T05:37:45.770524+0000","last_peered":"2023-10-08T05:37:45.770524+0000","last_clean":"2023-10-08T05:37:45.770524+0000","last_became_active":"2023-10-08T05:37:44.770324+0000","last_became_peered":"2023-10-08T05:37:44.770324+0000","last_unstale":"2023-10-08T05:37:45.770524+0000","last_undegraded":"2023-10-08T05:37:45.770524+0000","last_fullsized":"2023-10-08T05:37:45.770524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:42.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.769338+0000","last_change":"2023-10-08T05:37:43.746646+0000","last_active":"2023-10-08T05:37:44.769338+0000","last_peered":"2023-10-08T05:37:44.769338+0000","last_clean":"2023-10-08T05:37:44.769338+0000","last_became_active":"2023-10-08T05:37:43.746369+0000","last_became_peered":"2023-10-08T05:37:43.746369+0000","last_unstale":"2023-10-08T05:37:44.769338+0000","last_undegraded":"2023-10-08T05:37:44.769338+0000","last_fullsized":"2023-10-08T05:37:44.769338+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:22.404085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.261709+0000","last_change":"2023-10-08T05:37:44.754168+0000","last_active":"2023-10-08T05:37:46.261709+0000","last_peered":"2023-10-08T05:37:46.261709+0000","last_clean":"2023-10-08T05:37:46.261709+0000","last_became_active":"2023-10-08T05:37:44.753711+0000","last_became_peered":"2023-10-08T05:37:44.753711+0000","last_unstale":"2023-10-08T05:37:46.261709+0000","last_undegraded":"2023-10-08T05:37:46.261709+0000","last_fullsized":"2023-10-08T05:37:46.261709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:31.239302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.745714+0000","last_change":"2023-10-08T05:37:43.746978+0000","last_active":"2023-10-08T05:37:44.745714+0000","last_peered":"2023-10-08T05:37:44.745714+0000","last_clean":"2023-10-08T05:37:44.745714+0000","last_became_active":"2023-10-08T05:37:43.746728+0000","last_became_peered":"2023-10-08T05:37:43.746728+0000","last_unstale":"2023-10-08T05:37:44.745714+0000","last_undegraded":"2023-10-08T05:37:44.745714+0000","last_fullsized":"2023-10-08T05:37:44.745714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:11:28.561634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745551+0000","last_change":"2023-10-08T05:37:44.753069+0000","last_active":"2023-10-08T05:37:45.745551+0000","last_peered":"2023-10-08T05:37:45.745551+0000","last_clean":"2023-10-08T05:37:45.745551+0000","last_became_active":"2023-10-08T05:37:44.752211+0000","last_became_peered":"2023-10-08T05:37:44.752211+0000","last_unstale":"2023-10-08T05:37:45.745551+0000","last_undegraded":"2023-10-08T05:37:45.745551+0000","last_fullsized":"2023-10-08T05:37:45.745551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:26.951190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.724135+0000","last_change":"2023-10-08T05:37:43.736460+0000","last_active":"2023-10-08T05:37:44.724135+0000","last_peered":"2023-10-08T05:37:44.724135+0000","last_clean":"2023-10-08T05:37:44.724135+0000","last_became_active":"2023-10-08T05:37:43.736033+0000","last_became_peered":"2023-10-08T05:37:43.736033+0000","last_unstale":"2023-10-08T05:37:44.724135+0000","last_undegraded":"2023-10-08T05:37:44.724135+0000","last_fullsized":"2023-10-08T05:37:44.724135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865334+0000","last_change":"2023-10-08T05:37:44.737974+0000","last_active":"2023-10-08T05:37:45.865334+0000","last_peered":"2023-10-08T05:37:45.865334+0000","last_clean":"2023-10-08T05:37:45.865334+0000","last_became_active":"2023-10-08T05:37:44.737434+0000","last_became_peered":"2023-10-08T05:37:44.737434+0000","last_unstale":"2023-10-08T05:37:45.865334+0000","last_undegraded":"2023-10-08T05:37:45.865334+0000","last_fullsized":"2023-10-08T05:37:45.865334+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:44.941182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.736598+0000","last_change":"2023-10-08T05:37:43.735761+0000","last_active":"2023-10-08T05:37:44.736598+0000","last_peered":"2023-10-08T05:37:44.736598+0000","last_clean":"2023-10-08T05:37:44.736598+0000","last_became_active":"2023-10-08T05:37:43.735391+0000","last_became_peered":"2023-10-08T05:37:43.735391+0000","last_unstale":"2023-10-08T05:37:44.736598+0000","last_undegraded":"2023-10-08T05:37:44.736598+0000","last_fullsized":"2023-10-08T05:37:44.736598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:19.684087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926838+0000","last_change":"2023-10-08T05:37:44.753306+0000","last_active":"2023-10-08T05:37:45.926838+0000","last_peered":"2023-10-08T05:37:45.926838+0000","last_clean":"2023-10-08T05:37:45.926838+0000","last_became_active":"2023-10-08T05:37:44.753011+0000","last_became_peered":"2023-10-08T05:37:44.753011+0000","last_unstale":"2023-10-08T05:37:45.926838+0000","last_undegraded":"2023-10-08T05:37:45.926838+0000","last_fullsized":"2023-10-08T05:37:45.926838+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:10:20.413176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.737397+0000","last_change":"2023-10-08T05:37:43.742980+0000","last_active":"2023-10-08T05:37:44.737397+0000","last_peered":"2023-10-08T05:37:44.737397+0000","last_clean":"2023-10-08T05:37:44.737397+0000","last_became_active":"2023-10-08T05:37:43.742358+0000","last_became_peered":"2023-10-08T05:37:43.742358+0000","last_unstale":"2023-10-08T05:37:44.737397+0000","last_undegraded":"2023-10-08T05:37:44.737397+0000","last_fullsized":"2023-10-08T05:37:44.737397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:33.988193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260643+0000","last_change":"2023-10-08T05:37:44.741932+0000","last_active":"2023-10-08T05:37:46.260643+0000","last_peered":"2023-10-08T05:37:46.260643+0000","last_clean":"2023-10-08T05:37:46.260643+0000","last_became_active":"2023-10-08T05:37:44.741655+0000","last_became_peered":"2023-10-08T05:37:44.741655+0000","last_unstale":"2023-10-08T05:37:46.260643+0000","last_undegraded":"2023-10-08T05:37:46.260643+0000","last_fullsized":"2023-10-08T05:37:46.260643+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:24.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.772558+0000","last_change":"2023-10-08T05:37:43.743414+0000","last_active":"2023-10-08T05:37:44.772558+0000","last_peered":"2023-10-08T05:37:44.772558+0000","last_clean":"2023-10-08T05:37:44.772558+0000","last_became_active":"2023-10-08T05:37:43.742873+0000","last_became_peered":"2023-10-08T05:37:43.742873+0000","last_unstale":"2023-10-08T05:37:44.772558+0000","last_undegraded":"2023-10-08T05:37:44.772558+0000","last_fullsized":"2023-10-08T05:37:44.772558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:48.869078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.262275+0000","last_change":"2023-10-08T05:37:44.749282+0000","last_active":"2023-10-08T05:37:46.262275+0000","last_peered":"2023-10-08T05:37:46.262275+0000","last_clean":"2023-10-08T05:37:46.262275+0000","last_became_active":"2023-10-08T05:37:44.748968+0000","last_became_peered":"2023-10-08T05:37:44.748968+0000","last_unstale":"2023-10-08T05:37:46.262275+0000","last_undegraded":"2023-10-08T05:37:46.262275+0000","last_fullsized":"2023-10-08T05:37:46.262275+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:55.053932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.775012+0000","last_change":"2023-10-08T05:37:43.749228+0000","last_active":"2023-10-08T05:37:44.775012+0000","last_peered":"2023-10-08T05:37:44.775012+0000","last_clean":"2023-10-08T05:37:44.775012+0000","last_became_active":"2023-10-08T05:37:43.748703+0000","last_became_peered":"2023-10-08T05:37:43.748703+0000","last_unstale":"2023-10-08T05:37:44.775012+0000","last_undegraded":"2023-10-08T05:37:44.775012+0000","last_fullsized":"2023-10-08T05:37:44.775012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:23.830751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260291+0000","last_change":"2023-10-08T05:37:44.754747+0000","last_active":"2023-10-08T05:37:46.260291+0000","last_peered":"2023-10-08T05:37:46.260291+0000","last_clean":"2023-10-08T05:37:46.260291+0000","last_became_active":"2023-10-08T05:37:44.754447+0000","last_became_peered":"2023-10-08T05:37:44.754447+0000","last_unstale":"2023-10-08T05:37:46.260291+0000","last_undegraded":"2023-10-08T05:37:46.260291+0000","last_fullsized":"2023-10-08T05:37:46.260291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:35.000424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.774159+0000","last_change":"2023-10-08T05:37:43.729198+0000","last_active":"2023-10-08T05:37:44.774159+0000","last_peered":"2023-10-08T05:37:44.774159+0000","last_clean":"2023-10-08T05:37:44.774159+0000","last_became_active":"2023-10-08T05:37:43.728918+0000","last_became_peered":"2023-10-08T05:37:43.728918+0000","last_unstale":"2023-10-08T05:37:44.774159+0000","last_undegraded":"2023-10-08T05:37:44.774159+0000","last_fullsized":"2023-10-08T05:37:44.774159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:48.869078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745415+0000","last_change":"2023-10-08T05:37:44.749698+0000","last_active":"2023-10-08T05:37:45.745415+0000","last_peered":"2023-10-08T05:37:45.745415+0000","last_clean":"2023-10-08T05:37:45.745415+0000","last_became_active":"2023-10-08T05:37:44.749418+0000","last_became_peered":"2023-10-08T05:37:44.749418+0000","last_unstale":"2023-10-08T05:37:45.745415+0000","last_undegraded":"2023-10-08T05:37:45.745415+0000","last_fullsized":"2023-10-08T05:37:45.745415+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:10:20.413176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.744916+0000","last_change":"2023-10-08T05:37:43.745271+0000","last_active":"2023-10-08T05:37:44.744916+0000","last_peered":"2023-10-08T05:37:44.744916+0000","last_clean":"2023-10-08T05:37:44.744916+0000","last_became_active":"2023-10-08T05:37:43.745043+0000","last_became_peered":"2023-10-08T05:37:43.745043+0000","last_unstale":"2023-10-08T05:37:44.744916+0000","last_undegraded":"2023-10-08T05:37:44.744916+0000","last_fullsized":"2023-10-08T05:37:44.744916+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926575+0000","last_change":"2023-10-08T05:37:44.752273+0000","last_active":"2023-10-08T05:37:45.926575+0000","last_peered":"2023-10-08T05:37:45.926575+0000","last_clean":"2023-10-08T05:37:45.926575+0000","last_became_active":"2023-10-08T05:37:44.751949+0000","last_became_peered":"2023-10-08T05:37:44.751949+0000","last_unstale":"2023-10-08T05:37:45.926575+0000","last_undegraded":"2023-10-08T05:37:45.926575+0000","last_fullsized":"2023-10-08T05:37:45.926575+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:24.842983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.727173+0000","last_change":"2023-10-08T05:37:43.746332+0000","last_active":"2023-10-08T05:37:44.727173+0000","last_peered":"2023-10-08T05:37:44.727173+0000","last_clean":"2023-10-08T05:37:44.727173+0000","last_became_active":"2023-10-08T05:37:43.746017+0000","last_became_peered":"2023-10-08T05:37:43.746017+0000","last_unstale":"2023-10-08T05:37:44.727173+0000","last_undegraded":"2023-10-08T05:37:44.727173+0000","last_fullsized":"2023-10-08T05:37:44.727173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745741+0000","last_change":"2023-10-08T05:37:44.753244+0000","last_active":"2023-10-08T05:37:45.745741+0000","last_peered":"2023-10-08T05:37:45.745741+0000","last_clean":"2023-10-08T05:37:45.745741+0000","last_became_active":"2023-10-08T05:37:44.752613+0000","last_became_peered":"2023-10-08T05:37:44.752613+0000","last_unstale":"2023-10-08T05:37:45.745741+0000","last_undegraded":"2023-10-08T05:37:45.745741+0000","last_fullsized":"2023-10-08T05:37:45.745741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:14.437254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.771404+0000","last_change":"2023-10-08T05:37:43.750157+0000","last_active":"2023-10-08T05:37:44.771404+0000","last_peered":"2023-10-08T05:37:44.771404+0000","last_clean":"2023-10-08T05:37:44.771404+0000","last_became_active":"2023-10-08T05:37:43.749846+0000","last_became_peered":"2023-10-08T05:37:43.749846+0000","last_unstale":"2023-10-08T05:37:44.771404+0000","last_undegraded":"2023-10-08T05:37:44.771404+0000","last_fullsized":"2023-10-08T05:37:44.771404+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745813+0000","last_change":"2023-10-08T05:37:44.759427+0000","last_active":"2023-10-08T05:37:45.745813+0000","last_peered":"2023-10-08T05:37:45.745813+0000","last_clean":"2023-10-08T05:37:45.745813+0000","last_became_active":"2023-10-08T05:37:44.759110+0000","last_became_peered":"2023-10-08T05:37:44.759110+0000","last_unstale":"2023-10-08T05:37:45.745813+0000","last_undegraded":"2023-10-08T05:37:45.745813+0000","last_fullsized":"2023-10-08T05:37:45.745813+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:01.744941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.771983+0000","last_change":"2023-10-08T05:37:43.752005+0000","last_active":"2023-10-08T05:37:44.771983+0000","last_peered":"2023-10-08T05:37:44.771983+0000","last_clean":"2023-10-08T05:37:44.771983+0000","last_became_active":"2023-10-08T05:37:43.751525+0000","last_became_peered":"2023-10-08T05:37:43.751525+0000","last_unstale":"2023-10-08T05:37:44.771983+0000","last_undegraded":"2023-10-08T05:37:44.771983+0000","last_fullsized":"2023-10-08T05:37:44.771983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:45.890117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.771054+0000","last_change":"2023-10-08T05:37:44.762466+0000","last_active":"2023-10-08T05:37:45.771054+0000","last_peered":"2023-10-08T05:37:45.771054+0000","last_clean":"2023-10-08T05:37:45.771054+0000","last_became_active":"2023-10-08T05:37:44.761918+0000","last_became_peered":"2023-10-08T05:37:44.761918+0000","last_unstale":"2023-10-08T05:37:45.771054+0000","last_undegraded":"2023-10-08T05:37:45.771054+0000","last_fullsized":"2023-10-08T05:37:45.771054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:40:49.354151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.762008+0000","last_change":"2023-10-08T05:37:43.745487+0000","last_active":"2023-10-08T05:37:44.762008+0000","last_peered":"2023-10-08T05:37:44.762008+0000","last_clean":"2023-10-08T05:37:44.762008+0000","last_became_active":"2023-10-08T05:37:43.744512+0000","last_became_peered":"2023-10-08T05:37:43.744512+0000","last_unstale":"2023-10-08T05:37:44.762008+0000","last_undegraded":"2023-10-08T05:37:44.762008+0000","last_fullsized":"2023-10-08T05:37:44.762008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:16:37.143405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.262300+0000","last_change":"2023-10-08T05:37:44.762051+0000","last_active":"2023-10-08T05:37:46.262300+0000","last_peered":"2023-10-08T05:37:46.262300+0000","last_clean":"2023-10-08T05:37:46.262300+0000","last_became_active":"2023-10-08T05:37:44.761448+0000","last_became_peered":"2023-10-08T05:37:44.761448+0000","last_unstale":"2023-10-08T05:37:46.262300+0000","last_undegraded":"2023-10-08T05:37:46.262300+0000","last_fullsized":"2023-10-08T05:37:46.262300+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:24.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.767442+0000","last_change":"2023-10-08T05:37:43.748895+0000","last_active":"2023-10-08T05:37:44.767442+0000","last_peered":"2023-10-08T05:37:44.767442+0000","last_clean":"2023-10-08T05:37:44.767442+0000","last_became_active":"2023-10-08T05:37:43.748618+0000","last_became_peered":"2023-10-08T05:37:43.748618+0000","last_unstale":"2023-10-08T05:37:44.767442+0000","last_undegraded":"2023-10-08T05:37:44.767442+0000","last_fullsized":"2023-10-08T05:37:44.767442+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:35.696328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865544+0000","last_change":"2023-10-08T05:37:44.747500+0000","last_active":"2023-10-08T05:37:45.865544+0000","last_peered":"2023-10-08T05:37:45.865544+0000","last_clean":"2023-10-08T05:37:45.865544+0000","last_became_active":"2023-10-08T05:37:44.747123+0000","last_became_peered":"2023-10-08T05:37:44.747123+0000","last_unstale":"2023-10-08T05:37:45.865544+0000","last_undegraded":"2023-10-08T05:37:45.865544+0000","last_fullsized":"2023-10-08T05:37:45.865544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:56:21.173836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.747744+0000","last_change":"2023-10-08T05:37:43.743970+0000","last_active":"2023-10-08T05:37:44.747744+0000","last_peered":"2023-10-08T05:37:44.747744+0000","last_clean":"2023-10-08T05:37:44.747744+0000","last_became_active":"2023-10-08T05:37:43.743106+0000","last_became_peered":"2023-10-08T05:37:43.743106+0000","last_unstale":"2023-10-08T05:37:44.747744+0000","last_undegraded":"2023-10-08T05:37:44.747744+0000","last_fullsized":"2023-10-08T05:37:44.747744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:22.404085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.927428+0000","last_change":"2023-10-08T05:37:44.750306+0000","last_active":"2023-10-08T05:37:45.927428+0000","last_peered":"2023-10-08T05:37:45.927428+0000","last_clean":"2023-10-08T05:37:45.927428+0000","last_became_active":"2023-10-08T05:37:44.750034+0000","last_became_peered":"2023-10-08T05:37:44.750034+0000","last_unstale":"2023-10-08T05:37:45.927428+0000","last_undegraded":"2023-10-08T05:37:45.927428+0000","last_fullsized":"2023-10-08T05:37:45.927428+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:01.744941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.763551+0000","last_change":"2023-10-08T05:37:43.728582+0000","last_active":"2023-10-08T05:37:44.763551+0000","last_peered":"2023-10-08T05:37:44.763551+0000","last_clean":"2023-10-08T05:37:44.763551+0000","last_became_active":"2023-10-08T05:37:43.728172+0000","last_became_peered":"2023-10-08T05:37:43.728172+0000","last_unstale":"2023-10-08T05:37:44.763551+0000","last_undegraded":"2023-10-08T05:37:44.763551+0000","last_fullsized":"2023-10-08T05:37:44.763551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:20:52.604321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260308+0000","last_change":"2023-10-08T05:37:44.742716+0000","last_active":"2023-10-08T05:37:46.260308+0000","last_peered":"2023-10-08T05:37:46.260308+0000","last_clean":"2023-10-08T05:37:46.260308+0000","last_became_active":"2023-10-08T05:37:44.742440+0000","last_became_peered":"2023-10-08T05:37:44.742440+0000","last_unstale":"2023-10-08T05:37:46.260308+0000","last_undegraded":"2023-10-08T05:37:46.260308+0000","last_fullsized":"2023-10-08T05:37:46.260308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:49.881310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.720348+0000","last_change":"2023-10-08T05:37:43.744243+0000","last_active":"2023-10-08T05:37:44.720348+0000","last_peered":"2023-10-08T05:37:44.720348+0000","last_clean":"2023-10-08T05:37:44.720348+0000","last_became_active":"2023-10-08T05:37:43.743356+0000","last_became_peered":"2023-10-08T05:37:43.743356+0000","last_unstale":"2023-10-08T05:37:44.720348+0000","last_undegraded":"2023-10-08T05:37:44.720348+0000","last_fullsized":"2023-10-08T05:37:44.720348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:15.449045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770034+0000","last_change":"2023-10-08T05:37:44.750298+0000","last_active":"2023-10-08T05:37:45.770034+0000","last_peered":"2023-10-08T05:37:45.770034+0000","last_clean":"2023-10-08T05:37:45.770034+0000","last_became_active":"2023-10-08T05:37:44.749905+0000","last_became_peered":"2023-10-08T05:37:44.749905+0000","last_unstale":"2023-10-08T05:37:45.770034+0000","last_undegraded":"2023-10-08T05:37:45.770034+0000","last_fullsized":"2023-10-08T05:37:45.770034+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:14.437254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.741455+0000","last_change":"2023-10-08T05:37:43.749749+0000","last_active":"2023-10-08T05:37:44.741455+0000","last_peered":"2023-10-08T05:37:44.741455+0000","last_clean":"2023-10-08T05:37:44.741455+0000","last_became_active":"2023-10-08T05:37:43.749413+0000","last_became_peered":"2023-10-08T05:37:43.749413+0000","last_unstale":"2023-10-08T05:37:44.741455+0000","last_undegraded":"2023-10-08T05:37:44.741455+0000","last_fullsized":"2023-10-08T05:37:44.741455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:23.342919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745896+0000","last_change":"2023-10-08T05:37:44.752878+0000","last_active":"2023-10-08T05:37:45.745896+0000","last_peered":"2023-10-08T05:37:45.745896+0000","last_clean":"2023-10-08T05:37:45.745896+0000","last_became_active":"2023-10-08T05:37:44.751854+0000","last_became_peered":"2023-10-08T05:37:44.751854+0000","last_unstale":"2023-10-08T05:37:45.745896+0000","last_undegraded":"2023-10-08T05:37:45.745896+0000","last_fullsized":"2023-10-08T05:37:45.745896+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:19.962298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.720290+0000","last_change":"2023-10-08T05:37:43.733311+0000","last_active":"2023-10-08T05:37:44.720290+0000","last_peered":"2023-10-08T05:37:44.720290+0000","last_clean":"2023-10-08T05:37:44.720290+0000","last_became_active":"2023-10-08T05:37:43.733051+0000","last_became_peered":"2023-10-08T05:37:43.733051+0000","last_unstale":"2023-10-08T05:37:44.720290+0000","last_undegraded":"2023-10-08T05:37:44.720290+0000","last_fullsized":"2023-10-08T05:37:44.720290+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:46:48.047033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749187+0000","last_change":"2023-10-08T05:37:44.769844+0000","last_active":"2023-10-08T05:37:45.749187+0000","last_peered":"2023-10-08T05:37:45.749187+0000","last_clean":"2023-10-08T05:37:45.749187+0000","last_became_active":"2023-10-08T05:37:44.769551+0000","last_became_peered":"2023-10-08T05:37:44.769551+0000","last_unstale":"2023-10-08T05:37:45.749187+0000","last_undegraded":"2023-10-08T05:37:45.749187+0000","last_fullsized":"2023-10-08T05:37:45.749187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:55.053932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.772565+0000","last_change":"2023-10-08T05:37:43.749129+0000","last_active":"2023-10-08T05:37:44.772565+0000","last_peered":"2023-10-08T05:37:44.772565+0000","last_clean":"2023-10-08T05:37:44.772565+0000","last_became_active":"2023-10-08T05:37:43.748832+0000","last_became_peered":"2023-10-08T05:37:43.748832+0000","last_unstale":"2023-10-08T05:37:44.772565+0000","last_undegraded":"2023-10-08T05:37:44.772565+0000","last_fullsized":"2023-10-08T05:37:44.772565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:55.523677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.927337+0000","last_change":"2023-10-08T05:37:44.753369+0000","last_active":"2023-10-08T05:37:45.927337+0000","last_peered":"2023-10-08T05:37:45.927337+0000","last_clean":"2023-10-08T05:37:45.927337+0000","last_became_active":"2023-10-08T05:37:44.753075+0000","last_became_peered":"2023-10-08T05:37:44.753075+0000","last_unstale":"2023-10-08T05:37:45.927337+0000","last_undegraded":"2023-10-08T05:37:45.927337+0000","last_fullsized":"2023-10-08T05:37:45.927337+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:14.437254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865411+0000","last_change":"2023-10-08T05:37:43.744435+0000","last_active":"2023-10-08T05:37:45.865411+0000","last_peered":"2023-10-08T05:37:45.865411+0000","last_clean":"2023-10-08T05:37:45.865411+0000","last_became_active":"2023-10-08T05:37:43.744157+0000","last_became_peered":"2023-10-08T05:37:43.744157+0000","last_unstale":"2023-10-08T05:37:45.865411+0000","last_undegraded":"2023-10-08T05:37:45.865411+0000","last_fullsized":"2023-10-08T05:37:45.865411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:04.744586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.746034+0000","last_change":"2023-10-08T05:37:44.759675+0000","last_active":"2023-10-08T05:37:45.746034+0000","last_peered":"2023-10-08T05:37:45.746034+0000","last_clean":"2023-10-08T05:37:45.746034+0000","last_became_active":"2023-10-08T05:37:44.759323+0000","last_became_peered":"2023-10-08T05:37:44.759323+0000","last_unstale":"2023-10-08T05:37:45.746034+0000","last_undegraded":"2023-10-08T05:37:45.746034+0000","last_fullsized":"2023-10-08T05:37:45.746034+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:26:53.666665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.730845+0000","last_change":"2023-10-08T05:37:43.733742+0000","last_active":"2023-10-08T05:37:44.730845+0000","last_peered":"2023-10-08T05:37:44.730845+0000","last_clean":"2023-10-08T05:37:44.730845+0000","last_became_active":"2023-10-08T05:37:43.733486+0000","last_became_peered":"2023-10-08T05:37:43.733486+0000","last_unstale":"2023-10-08T05:37:44.730845+0000","last_undegraded":"2023-10-08T05:37:44.730845+0000","last_fullsized":"2023-10-08T05:37:44.730845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:41.403273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260860+0000","last_change":"2023-10-08T05:37:44.744388+0000","last_active":"2023-10-08T05:37:46.260860+0000","last_peered":"2023-10-08T05:37:46.260860+0000","last_clean":"2023-10-08T05:37:46.260860+0000","last_became_active":"2023-10-08T05:37:44.744129+0000","last_became_peered":"2023-10-08T05:37:44.744129+0000","last_unstale":"2023-10-08T05:37:46.260860+0000","last_undegraded":"2023-10-08T05:37:46.260860+0000","last_fullsized":"2023-10-08T05:37:46.260860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:46.616809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.731200+0000","last_change":"2023-10-08T05:37:43.732879+0000","last_active":"2023-10-08T05:37:44.731200+0000","last_peered":"2023-10-08T05:37:44.731200+0000","last_clean":"2023-10-08T05:37:44.731200+0000","last_became_active":"2023-10-08T05:37:43.732616+0000","last_became_peered":"2023-10-08T05:37:43.732616+0000","last_unstale":"2023-10-08T05:37:44.731200+0000","last_undegraded":"2023-10-08T05:37:44.731200+0000","last_fullsized":"2023-10-08T05:37:44.731200+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:26.955372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749160+0000","last_change":"2023-10-08T05:37:44.757884+0000","last_active":"2023-10-08T05:37:45.749160+0000","last_peered":"2023-10-08T05:37:45.749160+0000","last_clean":"2023-10-08T05:37:45.749160+0000","last_became_active":"2023-10-08T05:37:44.757581+0000","last_became_peered":"2023-10-08T05:37:44.757581+0000","last_unstale":"2023-10-08T05:37:45.749160+0000","last_undegraded":"2023-10-08T05:37:45.749160+0000","last_fullsized":"2023-10-08T05:37:45.749160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:24.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.724702+0000","last_change":"2023-10-08T05:37:43.733421+0000","last_active":"2023-10-08T05:37:44.724702+0000","last_peered":"2023-10-08T05:37:44.724702+0000","last_clean":"2023-10-08T05:37:44.724702+0000","last_became_active":"2023-10-08T05:37:43.732803+0000","last_became_peered":"2023-10-08T05:37:43.732803+0000","last_unstale":"2023-10-08T05:37:44.724702+0000","last_undegraded":"2023-10-08T05:37:44.724702+0000","last_fullsized":"2023-10-08T05:37:44.724702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:27.563368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.261206+0000","last_change":"2023-10-08T05:37:44.744497+0000","last_active":"2023-10-08T05:37:46.261206+0000","last_peered":"2023-10-08T05:37:46.261206+0000","last_clean":"2023-10-08T05:37:46.261206+0000","last_became_active":"2023-10-08T05:37:44.744171+0000","last_became_peered":"2023-10-08T05:37:44.744171+0000","last_unstale":"2023-10-08T05:37:46.261206+0000","last_undegraded":"2023-10-08T05:37:46.261206+0000","last_fullsized":"2023-10-08T05:37:46.261206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:23.416317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.723976+0000","last_change":"2023-10-08T05:37:43.736772+0000","last_active":"2023-10-08T05:37:44.723976+0000","last_peered":"2023-10-08T05:37:44.723976+0000","last_clean":"2023-10-08T05:37:44.723976+0000","last_became_active":"2023-10-08T05:37:43.736441+0000","last_became_peered":"2023-10-08T05:37:43.736441+0000","last_unstale":"2023-10-08T05:37:44.723976+0000","last_undegraded":"2023-10-08T05:37:44.723976+0000","last_fullsized":"2023-10-08T05:37:44.723976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:12.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.927064+0000","last_change":"2023-10-08T05:37:44.752577+0000","last_active":"2023-10-08T05:37:45.927064+0000","last_peered":"2023-10-08T05:37:45.927064+0000","last_clean":"2023-10-08T05:37:45.927064+0000","last_became_active":"2023-10-08T05:37:44.752228+0000","last_became_peered":"2023-10-08T05:37:44.752228+0000","last_unstale":"2023-10-08T05:37:45.927064+0000","last_undegraded":"2023-10-08T05:37:45.927064+0000","last_fullsized":"2023-10-08T05:37:45.927064+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:26.951190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.733547+0000","last_change":"2023-10-08T05:37:43.733011+0000","last_active":"2023-10-08T05:37:44.733547+0000","last_peered":"2023-10-08T05:37:44.733547+0000","last_clean":"2023-10-08T05:37:44.733547+0000","last_became_active":"2023-10-08T05:37:43.732744+0000","last_became_peered":"2023-10-08T05:37:43.732744+0000","last_unstale":"2023-10-08T05:37:44.733547+0000","last_undegraded":"2023-10-08T05:37:44.733547+0000","last_fullsized":"2023-10-08T05:37:44.733547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:51.066926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770624+0000","last_change":"2023-10-08T05:37:44.760802+0000","last_active":"2023-10-08T05:37:45.770624+0000","last_peered":"2023-10-08T05:37:45.770624+0000","last_clean":"2023-10-08T05:37:45.770624+0000","last_became_active":"2023-10-08T05:37:44.760324+0000","last_became_peered":"2023-10-08T05:37:44.760324+0000","last_unstale":"2023-10-08T05:37:45.770624+0000","last_undegraded":"2023-10-08T05:37:45.770624+0000","last_fullsized":"2023-10-08T05:37:45.770624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:26:53.666665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.739614+0000","last_change":"2023-10-08T05:37:43.748040+0000","last_active":"2023-10-08T05:37:44.739614+0000","last_peered":"2023-10-08T05:37:44.739614+0000","last_clean":"2023-10-08T05:37:44.739614+0000","last_became_active":"2023-10-08T05:37:43.747745+0000","last_became_peered":"2023-10-08T05:37:43.747745+0000","last_unstale":"2023-10-08T05:37:44.739614+0000","last_undegraded":"2023-10-08T05:37:44.739614+0000","last_fullsized":"2023-10-08T05:37:44.739614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:30.227070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":181,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.738648+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:37:44.738648+0000","last_peered":"2023-10-08T05:37:44.738648+0000","last_clean":"2023-10-08T05:37:44.738648+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:37:44.738648+0000","last_undegraded":"2023-10-08T05:37:44.738648+0000","last_fullsized":"2023-10-08T05:37:44.738648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":785889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.75},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.877}]},{"osd":1,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.3520000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.5349999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":1.2809999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.254},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":1.605}]},{"osd":5,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":2.169},{"interface":"front","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":2.206}]},{"osd":6,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":3.5249999999999999,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.754},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":2.101}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":194421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.337},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.47}]},{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.175}]},{"osd":2,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.367},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.111}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.4219999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.2190000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":4.3099999999999996,"5min":4.3099999999999996,"15min":4.3099999999999996},"last":1.395},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.175,"5min":1.175,"15min":1.175},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.518}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.23,"5min":1.23,"15min":1.23},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"max":{"1min":4.1550000000000002,"5min":4.1550000000000002,"15min":4.1550000000000002},"last":1.496}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":193951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":3.5529999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1.4810000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.5669999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.3759999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":6.782,"5min":6.782,"15min":6.782},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"last":1.5820000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.1349999999999998,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":1.786},{"interface":"front","average":{"1min":1.762,"5min":1.762,"15min":1.762},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":7.0069999999999997,"5min":7.0069999999999997,"15min":7.0069999999999997},"last":1.2350000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":7.2569999999999997,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":1.044},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":6.2199999999999998,"5min":6.2199999999999998,"15min":6.2199999999999998},"last":1.5229999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":6.9279999999999999,"5min":6.9279999999999999,"15min":6.9279999999999999},"last":1.431},{"interface":"front","average":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":7.335,"5min":7.335,"15min":7.335},"last":1.3340000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":194443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.3029999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":1.345}]},{"osd":2,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.1499999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.97899999999999998}]},{"osd":5,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.643},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.5569999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.105},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.5149999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.448},{"interface":"front","average":{"1min":1.76,"5min":1.76,"15min":1.76},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.379}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":195485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.369},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.05}]},{"osd":1,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.204},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.272}]},{"osd":2,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":1.4399999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.107}]},{"osd":4,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.663}]},{"osd":5,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.7090000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.756}]},{"osd":7,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.609},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.48}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":454656,"data_stored":193909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.077}]},{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.71199999999999997}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.4330000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":1.137},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.254}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.3220000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":1.282}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.0529999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.379}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":454656,"data_stored":193929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.0389999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.78400000000000003}]},{"osd":3,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.75700000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.90100000000000002}]},{"osd":5,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":0.86899999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.129}]},{"osd":7,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.105},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.2769999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":784241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.439}]},{"osd":2,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.88400000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.51300000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.165}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.054},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.85399999999999998}]},{"osd":6,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.115}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.006},{"interface":"front","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.0720000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:37:54.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:37:54.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.643+0000 7fc79a890640 1 Processor -- start 2023-10-08T05:37:54.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.643+0000 7fc79a890640 1 -- start start 2023-10-08T05:37:54.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.644+0000 7fc79a890640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc79415ab30 0x7fc79415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:54.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.644+0000 7fc79a890640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79415f7d0 con 0x7fc7940adc70 2023-10-08T05:37:54.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.644+0000 7fc79a890640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79415f210 con 0x7fc7940b07d0 2023-10-08T05:37:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.644+0000 7fc79a890640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79415f4f0 con 0x7fc79415ab30 2023-10-08T05:37:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79908d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc7940adc70 0x7fc7940b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59896/0 (socket says 172.21.15.70:59896) 2023-10-08T05:37:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79908d640 1 -- 172.21.15.70:0/2689984497 learned_addr learned my addr 172.21.15.70:0/2689984497 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79988e640 1 --2- 172.21.15.70:0/2689984497 >> v2:172.21.15.181:3300/0 conn(0x7fc79415ab30 0x7fc79415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244978840 0 0) 0x7fc79415f7d0 con 0x7fc7940adc70 2023-10-08T05:37:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc780003520 con 0x7fc7940adc70 2023-10-08T05:37:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2137335741 0 0) 0x7fc780003520 con 0x7fc7940adc70 2023-10-08T05:37:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 >> v2:172.21.15.181:3300/0 conn(0x7fc79415ab30 msgr2=0x7fc79415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79888c640 1 --2- 172.21.15.70:0/2689984497 >> v2:172.21.15.181:3300/0 conn(0x7fc79415ab30 0x7fc79415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 >> v1:172.21.15.181:6789/0 conn(0x7fc7940b07d0 legacy=0x7fc79406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79406c550 con 0x7fc7940adc70 2023-10-08T05:37:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.645+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc784001670 con 0x7fc7940adc70 2023-10-08T05:37:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 >> v1:172.21.15.70:6789/0 conn(0x7fc7940adc70 legacy=0x7fc7940b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79888c640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc780003970 0x7fc780003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79888c640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7800047c0 0x7fc780004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79888c640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc780005180 0x7fc78000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79415f4f0 con 0x7fc7800047c0 2023-10-08T05:37:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78000c0d0 con 0x7fc780003970 2023-10-08T05:37:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78000c290 con 0x7fc780005180 2023-10-08T05:37:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79908d640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7800047c0 0x7fc780004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79988e640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc780003970 0x7fc780003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79a08f640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc780005180 0x7fc78000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79908d640 1 -- 172.21.15.70:0/2689984497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc780005180 msgr2=0x7fc78000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:54.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79908d640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc780005180 0x7fc78000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79908d640 1 -- 172.21.15.70:0/2689984497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc780003970 msgr2=0x7fc780003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:54.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.646+0000 7fc79908d640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc780003970 0x7fc780003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.647+0000 7fc79a08f640 1 -- 172.21.15.70:0/2689984497 reap_dead start 2023-10-08T05:37:54.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.647+0000 7fc79908d640 1 -- 172.21.15.70:0/2689984497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc780003520 con 0x7fc7800047c0 2023-10-08T05:37:54.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.648+0000 7fc79908d640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7800047c0 0x7fc780004c40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c240 tx=0x7fc7800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6efe93bc5b63a3f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:54.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.648+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784005ad0 con 0x7fc7800047c0 2023-10-08T05:37:54.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.648+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc784005c30 con 0x7fc7800047c0 2023-10-08T05:37:54.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.648+0000 7fc79888c640 1 -- 172.21.15.70:0/2689984497 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78402caf0 con 0x7fc7800047c0 2023-10-08T05:37:54.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.648+0000 7fc79a890640 1 -- 172.21.15.70:0/2689984497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7800047c0 msgr2=0x7fc780004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:54.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.648+0000 7fc79a890640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7800047c0 0x7fc780004c40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c240 tx=0x7fc7800057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.648+0000 7fc79a890640 1 -- 172.21.15.70:0/2689984497 shutdown_connections 2023-10-08T05:37:54.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.648+0000 7fc79a890640 1 --2- 172.21.15.70:0/2689984497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7800047c0 0x7fc780004c40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.648+0000 7fc79a890640 1 -- 172.21.15.70:0/2689984497 >> 172.21.15.70:0/2689984497 conn(0x7fc7940cb0c0 msgr2=0x7fc7940cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:54.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.649+0000 7fc79a890640 1 -- 172.21.15.70:0/2689984497 shutdown_connections 2023-10-08T05:37:54.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.649+0000 7fc79a890640 1 -- 172.21.15.70:0/2689984497 wait complete. 2023-10-08T05:37:54.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.650+0000 7fc79a890640 1 Processor -- start 2023-10-08T05:37:54.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.650+0000 7fc79a890640 1 -- start start 2023-10-08T05:37:54.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.651+0000 7fc79a890640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7800047c0 0x7fc7940c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:54.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.651+0000 7fc79a890640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7940c2130 0x7fc7940c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:54.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.651+0000 7fc79a890640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7941effb0 0x7fc7940b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:54.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.651+0000 7fc79a890640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940c7ff0 con 0x7fc7940c2130 2023-10-08T05:37:54.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.651+0000 7fc79a890640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940c3420 con 0x7fc7800047c0 2023-10-08T05:37:54.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.651+0000 7fc79a890640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940c2590 con 0x7fc7941effb0 2023-10-08T05:37:54.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79908d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7940c2130 0x7fc7940c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:54.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79908d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7940c2130 0x7fc7940c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45308/0 (socket says 172.21.15.70:45308) 2023-10-08T05:37:54.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79908d640 1 -- 172.21.15.70:0/2622613463 learned_addr learned my addr 172.21.15.70:0/2622613463 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:54.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79a08f640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7941effb0 0x7fc7940b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:54.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79988e640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7800047c0 0x7fc7940c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:54.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79908d640 1 -- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7941effb0 msgr2=0x7fc7940b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:54.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79908d640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7941effb0 0x7fc7940b2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79908d640 1 -- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7800047c0 msgr2=0x7fc7940c38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:54.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79908d640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7800047c0 0x7fc7940c38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79908d640 1 -- 172.21.15.70:0/2622613463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7940c2810 con 0x7fc7940c2130 2023-10-08T05:37:54.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79988e640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7800047c0 0x7fc7940c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:54.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.652+0000 7fc79908d640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7940c2130 0x7fc7940c7ab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc784001930 tx=0x7fc78402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:54.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.653+0000 7fc78a7fc640 1 -- 172.21.15.70:0/2622613463 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78403c040 con 0x7fc7940c2130 2023-10-08T05:37:54.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.653+0000 7fc78a7fc640 1 -- 172.21.15.70:0/2622613463 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78402ccc0 con 0x7fc7940c2130 2023-10-08T05:37:54.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.653+0000 7fc78a7fc640 1 -- 172.21.15.70:0/2622613463 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784007290 con 0x7fc7940c2130 2023-10-08T05:37:54.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.653+0000 7fc79a890640 1 -- 172.21.15.70:0/2622613463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7940c2af0 con 0x7fc7940c2130 2023-10-08T05:37:54.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.653+0000 7fc79a890640 1 -- 172.21.15.70:0/2622613463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7940c30b0 con 0x7fc7940c2130 2023-10-08T05:37:54.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.654+0000 7fc79a890640 1 -- 172.21.15.70:0/2622613463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7940c3240 con 0x7fc7940c2130 2023-10-08T05:37:54.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.655+0000 7fc78a7fc640 1 -- 172.21.15.70:0/2622613463 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7fc784032070 con 0x7fc7940c2130 2023-10-08T05:37:54.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.656+0000 7fc78a7fc640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc77806ae70 0x7fc77806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:54.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.656+0000 7fc78a7fc640 1 -- 172.21.15.70:0/2622613463 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc7840aad90 con 0x7fc7940c2130 2023-10-08T05:37:54.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.656+0000 7fc79988e640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc77806ae70 0x7fc77806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:54.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.658+0000 7fc79988e640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc77806ae70 0x7fc77806d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc79404f940 tx=0x7fc790008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:54.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.660+0000 7fc78a7fc640 1 -- 172.21.15.70:0/2622613463 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc784075bd0 con 0x7fc7940c2130 2023-10-08T05:37:54.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.848+0000 7fc79a890640 1 -- 172.21.15.70:0/2622613463 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc79406c830 con 0x7fc77806ae70 2023-10-08T05:37:54.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.872+0000 7fc78a7fc640 1 -- 172.21.15.70:0/2622613463 <== mgr.4105 v2:172.21.15.181:6832/2737793492 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287141 (secure 0 0 0) 0x7fc79000b5d0 con 0x7fc77806ae70 2023-10-08T05:37:54.877 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:54.879 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:37:54.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.877+0000 7fc79a890640 1 -- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc77806ae70 msgr2=0x7fc77806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:54.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.877+0000 7fc79a890640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc77806ae70 0x7fc77806d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc79404f940 tx=0x7fc790008040 comp rx=0 tx=0).stop 2023-10-08T05:37:54.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.877+0000 7fc79a890640 1 -- 172.21.15.70:0/2622613463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7940c2130 msgr2=0x7fc7940c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:54.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.877+0000 7fc79a890640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7940c2130 0x7fc7940c7ab0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc784001930 tx=0x7fc78402e070 comp rx=0 tx=0).stop 2023-10-08T05:37:54.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.878+0000 7fc79a890640 1 -- 172.21.15.70:0/2622613463 shutdown_connections 2023-10-08T05:37:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.878+0000 7fc79a890640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fc77806ae70 0x7fc77806d330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.878+0000 7fc79a890640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7941effb0 0x7fc7940b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.878+0000 7fc79a890640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7940c2130 0x7fc7940c7ab0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.878+0000 7fc79a890640 1 --2- 172.21.15.70:0/2622613463 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7800047c0 0x7fc7940c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.878+0000 7fc79a890640 1 -- 172.21.15.70:0/2622613463 >> 172.21.15.70:0/2622613463 conn(0x7fc7940cb0c0 msgr2=0x7fc7940701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.878+0000 7fc79a890640 1 -- 172.21.15.70:0/2622613463 shutdown_connections 2023-10-08T05:37:54.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:54.878+0000 7fc79a890640 1 -- 172.21.15.70:0/2622613463 wait complete. 2023-10-08T05:37:54.897 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2023-10-08T05:37:53.108145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237580,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737140,"statfs":{"total":773094113280,"available":772850831360,"internally_reserved":0,"allocated":4857856,"data_stored":2736268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":238276155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.390289"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.746144+0000","last_change":"2023-10-08T05:37:44.750329+0000","last_active":"2023-10-08T05:37:45.746144+0000","last_peered":"2023-10-08T05:37:45.746144+0000","last_clean":"2023-10-08T05:37:45.746144+0000","last_became_active":"2023-10-08T05:37:44.750043+0000","last_became_peered":"2023-10-08T05:37:44.750043+0000","last_unstale":"2023-10-08T05:37:45.746144+0000","last_undegraded":"2023-10-08T05:37:45.746144+0000","last_fullsized":"2023-10-08T05:37:45.746144+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:11.877767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745958+0000","last_change":"2023-10-08T05:37:44.753676+0000","last_active":"2023-10-08T05:37:45.745958+0000","last_peered":"2023-10-08T05:37:45.745958+0000","last_clean":"2023-10-08T05:37:45.745958+0000","last_became_active":"2023-10-08T05:37:44.753212+0000","last_became_peered":"2023-10-08T05:37:44.753212+0000","last_unstale":"2023-10-08T05:37:45.745958+0000","last_undegraded":"2023-10-08T05:37:45.745958+0000","last_fullsized":"2023-10-08T05:37:45.745958+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:42.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260601+0000","last_change":"2023-10-08T05:37:44.758547+0000","last_active":"2023-10-08T05:37:46.260601+0000","last_peered":"2023-10-08T05:37:46.260601+0000","last_clean":"2023-10-08T05:37:46.260601+0000","last_became_active":"2023-10-08T05:37:44.754894+0000","last_became_peered":"2023-10-08T05:37:44.754894+0000","last_unstale":"2023-10-08T05:37:46.260601+0000","last_undegraded":"2023-10-08T05:37:46.260601+0000","last_fullsized":"2023-10-08T05:37:46.260601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:55.053932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865289+0000","last_change":"2023-10-08T05:37:44.740778+0000","last_active":"2023-10-08T05:37:45.865289+0000","last_peered":"2023-10-08T05:37:45.865289+0000","last_clean":"2023-10-08T05:37:45.865289+0000","last_became_active":"2023-10-08T05:37:44.740478+0000","last_became_peered":"2023-10-08T05:37:44.740478+0000","last_unstale":"2023-10-08T05:37:45.865289+0000","last_undegraded":"2023-10-08T05:37:45.865289+0000","last_fullsized":"2023-10-08T05:37:45.865289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:13.103220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865209+0000","last_change":"2023-10-08T05:37:44.740649+0000","last_active":"2023-10-08T05:37:45.865209+0000","last_peered":"2023-10-08T05:37:45.865209+0000","last_clean":"2023-10-08T05:37:45.865209+0000","last_became_active":"2023-10-08T05:37:44.740308+0000","last_became_peered":"2023-10-08T05:37:44.740308+0000","last_unstale":"2023-10-08T05:37:45.865209+0000","last_undegraded":"2023-10-08T05:37:45.865209+0000","last_fullsized":"2023-10-08T05:37:45.865209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:28.575600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749536+0000","last_change":"2023-10-08T05:37:44.756366+0000","last_active":"2023-10-08T05:37:45.749536+0000","last_peered":"2023-10-08T05:37:45.749536+0000","last_clean":"2023-10-08T05:37:45.749536+0000","last_became_active":"2023-10-08T05:37:44.756109+0000","last_became_peered":"2023-10-08T05:37:44.756109+0000","last_unstale":"2023-10-08T05:37:45.749536+0000","last_undegraded":"2023-10-08T05:37:45.749536+0000","last_fullsized":"2023-10-08T05:37:45.749536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:52.079157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770337+0000","last_change":"2023-10-08T05:37:44.758894+0000","last_active":"2023-10-08T05:37:45.770337+0000","last_peered":"2023-10-08T05:37:45.770337+0000","last_clean":"2023-10-08T05:37:45.770337+0000","last_became_active":"2023-10-08T05:37:44.758570+0000","last_became_peered":"2023-10-08T05:37:44.758570+0000","last_unstale":"2023-10-08T05:37:45.770337+0000","last_undegraded":"2023-10-08T05:37:45.770337+0000","last_fullsized":"2023-10-08T05:37:45.770337+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:19.962298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745267+0000","last_change":"2023-10-08T05:37:44.753088+0000","last_active":"2023-10-08T05:37:45.745267+0000","last_peered":"2023-10-08T05:37:45.745267+0000","last_clean":"2023-10-08T05:37:45.745267+0000","last_became_active":"2023-10-08T05:37:44.752787+0000","last_became_peered":"2023-10-08T05:37:44.752787+0000","last_unstale":"2023-10-08T05:37:45.745267+0000","last_undegraded":"2023-10-08T05:37:45.745267+0000","last_fullsized":"2023-10-08T05:37:45.745267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:46.902349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.261126+0000","last_change":"2023-10-08T05:37:44.762001+0000","last_active":"2023-10-08T05:37:46.261126+0000","last_peered":"2023-10-08T05:37:46.261126+0000","last_clean":"2023-10-08T05:37:46.261126+0000","last_became_active":"2023-10-08T05:37:44.761719+0000","last_became_peered":"2023-10-08T05:37:44.761719+0000","last_unstale":"2023-10-08T05:37:46.261126+0000","last_undegraded":"2023-10-08T05:37:46.261126+0000","last_fullsized":"2023-10-08T05:37:46.261126+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:29.716943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770757+0000","last_change":"2023-10-08T05:37:44.748634+0000","last_active":"2023-10-08T05:37:45.770757+0000","last_peered":"2023-10-08T05:37:45.770757+0000","last_clean":"2023-10-08T05:37:45.770757+0000","last_became_active":"2023-10-08T05:37:44.748267+0000","last_became_peered":"2023-10-08T05:37:44.748267+0000","last_unstale":"2023-10-08T05:37:45.770757+0000","last_undegraded":"2023-10-08T05:37:45.770757+0000","last_fullsized":"2023-10-08T05:37:45.770757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:11.877767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770790+0000","last_change":"2023-10-08T05:37:44.752195+0000","last_active":"2023-10-08T05:37:45.770790+0000","last_peered":"2023-10-08T05:37:45.770790+0000","last_clean":"2023-10-08T05:37:45.770790+0000","last_became_active":"2023-10-08T05:37:44.750351+0000","last_became_peered":"2023-10-08T05:37:44.750351+0000","last_unstale":"2023-10-08T05:37:45.770790+0000","last_undegraded":"2023-10-08T05:37:45.770790+0000","last_fullsized":"2023-10-08T05:37:45.770790+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:11:54.910375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260755+0000","last_change":"2023-10-08T05:37:44.754914+0000","last_active":"2023-10-08T05:37:46.260755+0000","last_peered":"2023-10-08T05:37:46.260755+0000","last_clean":"2023-10-08T05:37:46.260755+0000","last_became_active":"2023-10-08T05:37:44.754614+0000","last_became_peered":"2023-10-08T05:37:44.754614+0000","last_unstale":"2023-10-08T05:37:46.260755+0000","last_undegraded":"2023-10-08T05:37:46.260755+0000","last_fullsized":"2023-10-08T05:37:46.260755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:52.079157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770133+0000","last_change":"2023-10-08T05:37:44.751890+0000","last_active":"2023-10-08T05:37:45.770133+0000","last_peered":"2023-10-08T05:37:45.770133+0000","last_clean":"2023-10-08T05:37:45.770133+0000","last_became_active":"2023-10-08T05:37:44.751555+0000","last_became_peered":"2023-10-08T05:37:44.751555+0000","last_unstale":"2023-10-08T05:37:45.770133+0000","last_undegraded":"2023-10-08T05:37:45.770133+0000","last_fullsized":"2023-10-08T05:37:45.770133+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:01.744941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.261243+0000","last_change":"2023-10-08T05:37:44.742430+0000","last_active":"2023-10-08T05:37:46.261243+0000","last_peered":"2023-10-08T05:37:46.261243+0000","last_clean":"2023-10-08T05:37:46.261243+0000","last_became_active":"2023-10-08T05:37:44.742070+0000","last_became_peered":"2023-10-08T05:37:44.742070+0000","last_unstale":"2023-10-08T05:37:46.261243+0000","last_undegraded":"2023-10-08T05:37:46.261243+0000","last_fullsized":"2023-10-08T05:37:46.261243+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:35.000424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770011+0000","last_change":"2023-10-08T05:37:44.756459+0000","last_active":"2023-10-08T05:37:45.770011+0000","last_peered":"2023-10-08T05:37:45.770011+0000","last_clean":"2023-10-08T05:37:45.770011+0000","last_became_active":"2023-10-08T05:37:44.756163+0000","last_became_peered":"2023-10-08T05:37:44.756163+0000","last_unstale":"2023-10-08T05:37:45.770011+0000","last_undegraded":"2023-10-08T05:37:45.770011+0000","last_fullsized":"2023-10-08T05:37:45.770011+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:26.951190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926333+0000","last_change":"2023-10-08T05:37:44.762058+0000","last_active":"2023-10-08T05:37:45.926333+0000","last_peered":"2023-10-08T05:37:45.926333+0000","last_clean":"2023-10-08T05:37:45.926333+0000","last_became_active":"2023-10-08T05:37:44.761794+0000","last_became_peered":"2023-10-08T05:37:44.761794+0000","last_unstale":"2023-10-08T05:37:45.926333+0000","last_undegraded":"2023-10-08T05:37:45.926333+0000","last_fullsized":"2023-10-08T05:37:45.926333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:29.716943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.927643+0000","last_change":"2023-10-08T05:37:44.760175+0000","last_active":"2023-10-08T05:37:45.927643+0000","last_peered":"2023-10-08T05:37:45.927643+0000","last_clean":"2023-10-08T05:37:45.927643+0000","last_became_active":"2023-10-08T05:37:44.759862+0000","last_became_peered":"2023-10-08T05:37:44.759862+0000","last_unstale":"2023-10-08T05:37:45.927643+0000","last_undegraded":"2023-10-08T05:37:45.927643+0000","last_fullsized":"2023-10-08T05:37:45.927643+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:19.962298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.753066+0000","last_change":"2023-10-08T05:37:44.746870+0000","last_active":"2023-10-08T05:37:45.753066+0000","last_peered":"2023-10-08T05:37:45.753066+0000","last_clean":"2023-10-08T05:37:45.753066+0000","last_became_active":"2023-10-08T05:37:44.746488+0000","last_became_peered":"2023-10-08T05:37:44.746488+0000","last_unstale":"2023-10-08T05:37:45.753066+0000","last_undegraded":"2023-10-08T05:37:45.753066+0000","last_fullsized":"2023-10-08T05:37:45.753066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:16:38.155637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865439+0000","last_change":"2023-10-08T05:37:44.770316+0000","last_active":"2023-10-08T05:37:45.865439+0000","last_peered":"2023-10-08T05:37:45.865439+0000","last_clean":"2023-10-08T05:37:45.865439+0000","last_became_active":"2023-10-08T05:37:44.770017+0000","last_became_peered":"2023-10-08T05:37:44.770017+0000","last_unstale":"2023-10-08T05:37:45.865439+0000","last_undegraded":"2023-10-08T05:37:45.865439+0000","last_fullsized":"2023-10-08T05:37:45.865439+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:54.627946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.752578+0000","last_change":"2023-10-08T05:37:44.746048+0000","last_active":"2023-10-08T05:37:45.752578+0000","last_peered":"2023-10-08T05:37:45.752578+0000","last_clean":"2023-10-08T05:37:45.752578+0000","last_became_active":"2023-10-08T05:37:44.745781+0000","last_became_peered":"2023-10-08T05:37:44.745781+0000","last_unstale":"2023-10-08T05:37:45.752578+0000","last_undegraded":"2023-10-08T05:37:45.752578+0000","last_fullsized":"2023-10-08T05:37:45.752578+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:45.569743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770697+0000","last_change":"2023-10-08T05:37:44.770242+0000","last_active":"2023-10-08T05:37:45.770697+0000","last_peered":"2023-10-08T05:37:45.770697+0000","last_clean":"2023-10-08T05:37:45.770697+0000","last_became_active":"2023-10-08T05:37:44.769915+0000","last_became_peered":"2023-10-08T05:37:44.769915+0000","last_unstale":"2023-10-08T05:37:45.770697+0000","last_undegraded":"2023-10-08T05:37:45.770697+0000","last_fullsized":"2023-10-08T05:37:45.770697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:04.840446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.261690+0000","last_change":"2023-10-08T05:37:44.749297+0000","last_active":"2023-10-08T05:37:46.261690+0000","last_peered":"2023-10-08T05:37:46.261690+0000","last_clean":"2023-10-08T05:37:46.261690+0000","last_became_active":"2023-10-08T05:37:44.748939+0000","last_became_peered":"2023-10-08T05:37:44.748939+0000","last_unstale":"2023-10-08T05:37:46.261690+0000","last_undegraded":"2023-10-08T05:37:46.261690+0000","last_fullsized":"2023-10-08T05:37:46.261690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:49.881310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926490+0000","last_change":"2023-10-08T05:37:44.752426+0000","last_active":"2023-10-08T05:37:45.926490+0000","last_peered":"2023-10-08T05:37:45.926490+0000","last_clean":"2023-10-08T05:37:45.926490+0000","last_became_active":"2023-10-08T05:37:44.752138+0000","last_became_peered":"2023-10-08T05:37:44.752138+0000","last_unstale":"2023-10-08T05:37:45.926490+0000","last_undegraded":"2023-10-08T05:37:45.926490+0000","last_fullsized":"2023-10-08T05:37:45.926490+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:46.902349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865055+0000","last_change":"2023-10-08T05:37:44.745348+0000","last_active":"2023-10-08T05:37:45.865055+0000","last_peered":"2023-10-08T05:37:45.865055+0000","last_clean":"2023-10-08T05:37:45.865055+0000","last_became_active":"2023-10-08T05:37:44.745086+0000","last_became_peered":"2023-10-08T05:37:44.745086+0000","last_unstale":"2023-10-08T05:37:45.865055+0000","last_undegraded":"2023-10-08T05:37:45.865055+0000","last_fullsized":"2023-10-08T05:37:45.865055+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:35.619176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.753164+0000","last_change":"2023-10-08T05:37:44.745696+0000","last_active":"2023-10-08T05:37:45.753164+0000","last_peered":"2023-10-08T05:37:45.753164+0000","last_clean":"2023-10-08T05:37:45.753164+0000","last_became_active":"2023-10-08T05:37:44.745366+0000","last_became_peered":"2023-10-08T05:37:44.745366+0000","last_unstale":"2023-10-08T05:37:45.753164+0000","last_undegraded":"2023-10-08T05:37:45.753164+0000","last_fullsized":"2023-10-08T05:37:45.753164+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:23.416317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749534+0000","last_change":"2023-10-08T05:37:44.765748+0000","last_active":"2023-10-08T05:37:45.749534+0000","last_peered":"2023-10-08T05:37:45.749534+0000","last_clean":"2023-10-08T05:37:45.749534+0000","last_became_active":"2023-10-08T05:37:44.765432+0000","last_became_peered":"2023-10-08T05:37:44.765432+0000","last_unstale":"2023-10-08T05:37:45.749534+0000","last_undegraded":"2023-10-08T05:37:45.749534+0000","last_fullsized":"2023-10-08T05:37:45.749534+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:46.616809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.752741+0000","last_change":"2023-10-08T05:37:44.748548+0000","last_active":"2023-10-08T05:37:45.752741+0000","last_peered":"2023-10-08T05:37:45.752741+0000","last_clean":"2023-10-08T05:37:45.752741+0000","last_became_active":"2023-10-08T05:37:44.748263+0000","last_became_peered":"2023-10-08T05:37:44.748263+0000","last_unstale":"2023-10-08T05:37:45.752741+0000","last_undegraded":"2023-10-08T05:37:45.752741+0000","last_fullsized":"2023-10-08T05:37:45.752741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:15.563184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770742+0000","last_change":"2023-10-08T05:37:44.757411+0000","last_active":"2023-10-08T05:37:45.770742+0000","last_peered":"2023-10-08T05:37:45.770742+0000","last_clean":"2023-10-08T05:37:45.770742+0000","last_became_active":"2023-10-08T05:37:44.757142+0000","last_became_peered":"2023-10-08T05:37:44.757142+0000","last_unstale":"2023-10-08T05:37:45.770742+0000","last_undegraded":"2023-10-08T05:37:45.770742+0000","last_fullsized":"2023-10-08T05:37:45.770742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:31:15.770940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.752743+0000","last_change":"2023-10-08T05:37:44.759411+0000","last_active":"2023-10-08T05:37:45.752743+0000","last_peered":"2023-10-08T05:37:45.752743+0000","last_clean":"2023-10-08T05:37:45.752743+0000","last_became_active":"2023-10-08T05:37:44.759055+0000","last_became_peered":"2023-10-08T05:37:44.759055+0000","last_unstale":"2023-10-08T05:37:45.752743+0000","last_undegraded":"2023-10-08T05:37:45.752743+0000","last_fullsized":"2023-10-08T05:37:45.752743+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:27.967603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.752761+0000","last_change":"2023-10-08T05:37:44.745364+0000","last_active":"2023-10-08T05:37:45.752761+0000","last_peered":"2023-10-08T05:37:45.752761+0000","last_clean":"2023-10-08T05:37:45.752761+0000","last_became_active":"2023-10-08T05:37:44.744992+0000","last_became_peered":"2023-10-08T05:37:44.744992+0000","last_unstale":"2023-10-08T05:37:45.752761+0000","last_undegraded":"2023-10-08T05:37:45.752761+0000","last_fullsized":"2023-10-08T05:37:45.752761+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:11:29.573866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926718+0000","last_change":"2023-10-08T05:37:44.756655+0000","last_active":"2023-10-08T05:37:45.926718+0000","last_peered":"2023-10-08T05:37:45.926718+0000","last_clean":"2023-10-08T05:37:45.926718+0000","last_became_active":"2023-10-08T05:37:44.756382+0000","last_became_peered":"2023-10-08T05:37:44.756382+0000","last_unstale":"2023-10-08T05:37:45.926718+0000","last_undegraded":"2023-10-08T05:37:45.926718+0000","last_fullsized":"2023-10-08T05:37:45.926718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:20.696319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749690+0000","last_change":"2023-10-08T05:37:44.755632+0000","last_active":"2023-10-08T05:37:45.749690+0000","last_peered":"2023-10-08T05:37:45.749690+0000","last_clean":"2023-10-08T05:37:45.749690+0000","last_became_active":"2023-10-08T05:37:44.755346+0000","last_became_peered":"2023-10-08T05:37:44.755346+0000","last_unstale":"2023-10-08T05:37:45.749690+0000","last_undegraded":"2023-10-08T05:37:45.749690+0000","last_fullsized":"2023-10-08T05:37:45.749690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:29.716943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926632+0000","last_change":"2023-10-08T05:37:44.760945+0000","last_active":"2023-10-08T05:37:45.926632+0000","last_peered":"2023-10-08T05:37:45.926632+0000","last_clean":"2023-10-08T05:37:45.926632+0000","last_became_active":"2023-10-08T05:37:44.760689+0000","last_became_peered":"2023-10-08T05:37:44.760689+0000","last_unstale":"2023-10-08T05:37:45.926632+0000","last_undegraded":"2023-10-08T05:37:45.926632+0000","last_fullsized":"2023-10-08T05:37:45.926632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:56.535909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.768987+0000","last_change":"2023-10-08T05:37:43.746333+0000","last_active":"2023-10-08T05:37:44.768987+0000","last_peered":"2023-10-08T05:37:44.768987+0000","last_clean":"2023-10-08T05:37:44.768987+0000","last_became_active":"2023-10-08T05:37:43.746017+0000","last_became_peered":"2023-10-08T05:37:43.746017+0000","last_unstale":"2023-10-08T05:37:44.768987+0000","last_undegraded":"2023-10-08T05:37:44.768987+0000","last_fullsized":"2023-10-08T05:37:44.768987+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:34.606944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749926+0000","last_change":"2023-10-08T05:37:44.769935+0000","last_active":"2023-10-08T05:37:45.749926+0000","last_peered":"2023-10-08T05:37:45.749926+0000","last_clean":"2023-10-08T05:37:45.749926+0000","last_became_active":"2023-10-08T05:37:44.769592+0000","last_became_peered":"2023-10-08T05:37:44.769592+0000","last_unstale":"2023-10-08T05:37:45.749926+0000","last_undegraded":"2023-10-08T05:37:45.749926+0000","last_fullsized":"2023-10-08T05:37:45.749926+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:24.842983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.770431+0000","last_change":"2023-10-08T05:37:43.749778+0000","last_active":"2023-10-08T05:37:44.770431+0000","last_peered":"2023-10-08T05:37:44.770431+0000","last_clean":"2023-10-08T05:37:44.770431+0000","last_became_active":"2023-10-08T05:37:43.749435+0000","last_became_peered":"2023-10-08T05:37:43.749435+0000","last_unstale":"2023-10-08T05:37:44.770431+0000","last_undegraded":"2023-10-08T05:37:44.770431+0000","last_fullsized":"2023-10-08T05:37:44.770431+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.262299+0000","last_change":"2023-10-08T05:37:44.740956+0000","last_active":"2023-10-08T05:37:46.262299+0000","last_peered":"2023-10-08T05:37:46.262299+0000","last_clean":"2023-10-08T05:37:46.262299+0000","last_became_active":"2023-10-08T05:37:44.740518+0000","last_became_peered":"2023-10-08T05:37:44.740518+0000","last_unstale":"2023-10-08T05:37:46.262299+0000","last_undegraded":"2023-10-08T05:37:46.262299+0000","last_fullsized":"2023-10-08T05:37:46.262299+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:52.079157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.772569+0000","last_change":"2023-10-08T05:37:43.749986+0000","last_active":"2023-10-08T05:37:44.772569+0000","last_peered":"2023-10-08T05:37:44.772569+0000","last_clean":"2023-10-08T05:37:44.772569+0000","last_became_active":"2023-10-08T05:37:43.749718+0000","last_became_peered":"2023-10-08T05:37:43.749718+0000","last_unstale":"2023-10-08T05:37:44.772569+0000","last_undegraded":"2023-10-08T05:37:44.772569+0000","last_fullsized":"2023-10-08T05:37:44.772569+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:30.227070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.746117+0000","last_change":"2023-10-08T05:37:44.759365+0000","last_active":"2023-10-08T05:37:45.746117+0000","last_peered":"2023-10-08T05:37:45.746117+0000","last_clean":"2023-10-08T05:37:45.746117+0000","last_became_active":"2023-10-08T05:37:44.759068+0000","last_became_peered":"2023-10-08T05:37:44.759068+0000","last_unstale":"2023-10-08T05:37:45.746117+0000","last_undegraded":"2023-10-08T05:37:45.746117+0000","last_fullsized":"2023-10-08T05:37:45.746117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:04.840446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.769723+0000","last_change":"2023-10-08T05:37:43.742613+0000","last_active":"2023-10-08T05:37:44.769723+0000","last_peered":"2023-10-08T05:37:44.769723+0000","last_clean":"2023-10-08T05:37:44.769723+0000","last_became_active":"2023-10-08T05:37:43.742232+0000","last_became_peered":"2023-10-08T05:37:43.742232+0000","last_unstale":"2023-10-08T05:37:44.769723+0000","last_undegraded":"2023-10-08T05:37:44.769723+0000","last_fullsized":"2023-10-08T05:37:44.769723+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:11:28.561634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260288+0000","last_change":"2023-10-08T05:37:44.750080+0000","last_active":"2023-10-08T05:37:46.260288+0000","last_peered":"2023-10-08T05:37:46.260288+0000","last_clean":"2023-10-08T05:37:46.260288+0000","last_became_active":"2023-10-08T05:37:44.749776+0000","last_became_peered":"2023-10-08T05:37:44.749776+0000","last_unstale":"2023-10-08T05:37:46.260288+0000","last_undegraded":"2023-10-08T05:37:46.260288+0000","last_fullsized":"2023-10-08T05:37:46.260288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:31.239302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.770777+0000","last_change":"2023-10-08T05:37:43.743385+0000","last_active":"2023-10-08T05:37:44.770777+0000","last_peered":"2023-10-08T05:37:44.770777+0000","last_clean":"2023-10-08T05:37:44.770777+0000","last_became_active":"2023-10-08T05:37:43.743002+0000","last_became_peered":"2023-10-08T05:37:43.743002+0000","last_unstale":"2023-10-08T05:37:44.770777+0000","last_undegraded":"2023-10-08T05:37:44.770777+0000","last_fullsized":"2023-10-08T05:37:44.770777+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:19.684087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770524+0000","last_change":"2023-10-08T05:37:44.770626+0000","last_active":"2023-10-08T05:37:45.770524+0000","last_peered":"2023-10-08T05:37:45.770524+0000","last_clean":"2023-10-08T05:37:45.770524+0000","last_became_active":"2023-10-08T05:37:44.770324+0000","last_became_peered":"2023-10-08T05:37:44.770324+0000","last_unstale":"2023-10-08T05:37:45.770524+0000","last_undegraded":"2023-10-08T05:37:45.770524+0000","last_fullsized":"2023-10-08T05:37:45.770524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:42.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.769338+0000","last_change":"2023-10-08T05:37:43.746646+0000","last_active":"2023-10-08T05:37:44.769338+0000","last_peered":"2023-10-08T05:37:44.769338+0000","last_clean":"2023-10-08T05:37:44.769338+0000","last_became_active":"2023-10-08T05:37:43.746369+0000","last_became_peered":"2023-10-08T05:37:43.746369+0000","last_unstale":"2023-10-08T05:37:44.769338+0000","last_undegraded":"2023-10-08T05:37:44.769338+0000","last_fullsized":"2023-10-08T05:37:44.769338+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:22.404085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.261709+0000","last_change":"2023-10-08T05:37:44.754168+0000","last_active":"2023-10-08T05:37:46.261709+0000","last_peered":"2023-10-08T05:37:46.261709+0000","last_clean":"2023-10-08T05:37:46.261709+0000","last_became_active":"2023-10-08T05:37:44.753711+0000","last_became_peered":"2023-10-08T05:37:44.753711+0000","last_unstale":"2023-10-08T05:37:46.261709+0000","last_undegraded":"2023-10-08T05:37:46.261709+0000","last_fullsized":"2023-10-08T05:37:46.261709+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:31.239302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.745714+0000","last_change":"2023-10-08T05:37:43.746978+0000","last_active":"2023-10-08T05:37:44.745714+0000","last_peered":"2023-10-08T05:37:44.745714+0000","last_clean":"2023-10-08T05:37:44.745714+0000","last_became_active":"2023-10-08T05:37:43.746728+0000","last_became_peered":"2023-10-08T05:37:43.746728+0000","last_unstale":"2023-10-08T05:37:44.745714+0000","last_undegraded":"2023-10-08T05:37:44.745714+0000","last_fullsized":"2023-10-08T05:37:44.745714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:11:28.561634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745551+0000","last_change":"2023-10-08T05:37:44.753069+0000","last_active":"2023-10-08T05:37:45.745551+0000","last_peered":"2023-10-08T05:37:45.745551+0000","last_clean":"2023-10-08T05:37:45.745551+0000","last_became_active":"2023-10-08T05:37:44.752211+0000","last_became_peered":"2023-10-08T05:37:44.752211+0000","last_unstale":"2023-10-08T05:37:45.745551+0000","last_undegraded":"2023-10-08T05:37:45.745551+0000","last_fullsized":"2023-10-08T05:37:45.745551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:26.951190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.724135+0000","last_change":"2023-10-08T05:37:43.736460+0000","last_active":"2023-10-08T05:37:44.724135+0000","last_peered":"2023-10-08T05:37:44.724135+0000","last_clean":"2023-10-08T05:37:44.724135+0000","last_became_active":"2023-10-08T05:37:43.736033+0000","last_became_peered":"2023-10-08T05:37:43.736033+0000","last_unstale":"2023-10-08T05:37:44.724135+0000","last_undegraded":"2023-10-08T05:37:44.724135+0000","last_fullsized":"2023-10-08T05:37:44.724135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865334+0000","last_change":"2023-10-08T05:37:44.737974+0000","last_active":"2023-10-08T05:37:45.865334+0000","last_peered":"2023-10-08T05:37:45.865334+0000","last_clean":"2023-10-08T05:37:45.865334+0000","last_became_active":"2023-10-08T05:37:44.737434+0000","last_became_peered":"2023-10-08T05:37:44.737434+0000","last_unstale":"2023-10-08T05:37:45.865334+0000","last_undegraded":"2023-10-08T05:37:45.865334+0000","last_fullsized":"2023-10-08T05:37:45.865334+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:44.941182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.736598+0000","last_change":"2023-10-08T05:37:43.735761+0000","last_active":"2023-10-08T05:37:44.736598+0000","last_peered":"2023-10-08T05:37:44.736598+0000","last_clean":"2023-10-08T05:37:44.736598+0000","last_became_active":"2023-10-08T05:37:43.735391+0000","last_became_peered":"2023-10-08T05:37:43.735391+0000","last_unstale":"2023-10-08T05:37:44.736598+0000","last_undegraded":"2023-10-08T05:37:44.736598+0000","last_fullsized":"2023-10-08T05:37:44.736598+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:19.684087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926838+0000","last_change":"2023-10-08T05:37:44.753306+0000","last_active":"2023-10-08T05:37:45.926838+0000","last_peered":"2023-10-08T05:37:45.926838+0000","last_clean":"2023-10-08T05:37:45.926838+0000","last_became_active":"2023-10-08T05:37:44.753011+0000","last_became_peered":"2023-10-08T05:37:44.753011+0000","last_unstale":"2023-10-08T05:37:45.926838+0000","last_undegraded":"2023-10-08T05:37:45.926838+0000","last_fullsized":"2023-10-08T05:37:45.926838+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:10:20.413176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.737397+0000","last_change":"2023-10-08T05:37:43.742980+0000","last_active":"2023-10-08T05:37:44.737397+0000","last_peered":"2023-10-08T05:37:44.737397+0000","last_clean":"2023-10-08T05:37:44.737397+0000","last_became_active":"2023-10-08T05:37:43.742358+0000","last_became_peered":"2023-10-08T05:37:43.742358+0000","last_unstale":"2023-10-08T05:37:44.737397+0000","last_undegraded":"2023-10-08T05:37:44.737397+0000","last_fullsized":"2023-10-08T05:37:44.737397+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:33.988193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260643+0000","last_change":"2023-10-08T05:37:44.741932+0000","last_active":"2023-10-08T05:37:46.260643+0000","last_peered":"2023-10-08T05:37:46.260643+0000","last_clean":"2023-10-08T05:37:46.260643+0000","last_became_active":"2023-10-08T05:37:44.741655+0000","last_became_peered":"2023-10-08T05:37:44.741655+0000","last_unstale":"2023-10-08T05:37:46.260643+0000","last_undegraded":"2023-10-08T05:37:46.260643+0000","last_fullsized":"2023-10-08T05:37:46.260643+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:24.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.772558+0000","last_change":"2023-10-08T05:37:43.743414+0000","last_active":"2023-10-08T05:37:44.772558+0000","last_peered":"2023-10-08T05:37:44.772558+0000","last_clean":"2023-10-08T05:37:44.772558+0000","last_became_active":"2023-10-08T05:37:43.742873+0000","last_became_peered":"2023-10-08T05:37:43.742873+0000","last_unstale":"2023-10-08T05:37:44.772558+0000","last_undegraded":"2023-10-08T05:37:44.772558+0000","last_fullsized":"2023-10-08T05:37:44.772558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:48.869078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.262275+0000","last_change":"2023-10-08T05:37:44.749282+0000","last_active":"2023-10-08T05:37:46.262275+0000","last_peered":"2023-10-08T05:37:46.262275+0000","last_clean":"2023-10-08T05:37:46.262275+0000","last_became_active":"2023-10-08T05:37:44.748968+0000","last_became_peered":"2023-10-08T05:37:44.748968+0000","last_unstale":"2023-10-08T05:37:46.262275+0000","last_undegraded":"2023-10-08T05:37:46.262275+0000","last_fullsized":"2023-10-08T05:37:46.262275+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:55.053932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.775012+0000","last_change":"2023-10-08T05:37:43.749228+0000","last_active":"2023-10-08T05:37:44.775012+0000","last_peered":"2023-10-08T05:37:44.775012+0000","last_clean":"2023-10-08T05:37:44.775012+0000","last_became_active":"2023-10-08T05:37:43.748703+0000","last_became_peered":"2023-10-08T05:37:43.748703+0000","last_unstale":"2023-10-08T05:37:44.775012+0000","last_undegraded":"2023-10-08T05:37:44.775012+0000","last_fullsized":"2023-10-08T05:37:44.775012+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:23.830751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260291+0000","last_change":"2023-10-08T05:37:44.754747+0000","last_active":"2023-10-08T05:37:46.260291+0000","last_peered":"2023-10-08T05:37:46.260291+0000","last_clean":"2023-10-08T05:37:46.260291+0000","last_became_active":"2023-10-08T05:37:44.754447+0000","last_became_peered":"2023-10-08T05:37:44.754447+0000","last_unstale":"2023-10-08T05:37:46.260291+0000","last_undegraded":"2023-10-08T05:37:46.260291+0000","last_fullsized":"2023-10-08T05:37:46.260291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:35.000424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.774159+0000","last_change":"2023-10-08T05:37:43.729198+0000","last_active":"2023-10-08T05:37:44.774159+0000","last_peered":"2023-10-08T05:37:44.774159+0000","last_clean":"2023-10-08T05:37:44.774159+0000","last_became_active":"2023-10-08T05:37:43.728918+0000","last_became_peered":"2023-10-08T05:37:43.728918+0000","last_unstale":"2023-10-08T05:37:44.774159+0000","last_undegraded":"2023-10-08T05:37:44.774159+0000","last_fullsized":"2023-10-08T05:37:44.774159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:48.869078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745415+0000","last_change":"2023-10-08T05:37:44.749698+0000","last_active":"2023-10-08T05:37:45.745415+0000","last_peered":"2023-10-08T05:37:45.745415+0000","last_clean":"2023-10-08T05:37:45.745415+0000","last_became_active":"2023-10-08T05:37:44.749418+0000","last_became_peered":"2023-10-08T05:37:44.749418+0000","last_unstale":"2023-10-08T05:37:45.745415+0000","last_undegraded":"2023-10-08T05:37:45.745415+0000","last_fullsized":"2023-10-08T05:37:45.745415+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:10:20.413176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.744916+0000","last_change":"2023-10-08T05:37:43.745271+0000","last_active":"2023-10-08T05:37:44.744916+0000","last_peered":"2023-10-08T05:37:44.744916+0000","last_clean":"2023-10-08T05:37:44.744916+0000","last_became_active":"2023-10-08T05:37:43.745043+0000","last_became_peered":"2023-10-08T05:37:43.745043+0000","last_unstale":"2023-10-08T05:37:44.744916+0000","last_undegraded":"2023-10-08T05:37:44.744916+0000","last_fullsized":"2023-10-08T05:37:44.744916+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.926575+0000","last_change":"2023-10-08T05:37:44.752273+0000","last_active":"2023-10-08T05:37:45.926575+0000","last_peered":"2023-10-08T05:37:45.926575+0000","last_clean":"2023-10-08T05:37:45.926575+0000","last_became_active":"2023-10-08T05:37:44.751949+0000","last_became_peered":"2023-10-08T05:37:44.751949+0000","last_unstale":"2023-10-08T05:37:45.926575+0000","last_undegraded":"2023-10-08T05:37:45.926575+0000","last_fullsized":"2023-10-08T05:37:45.926575+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:24.842983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.727173+0000","last_change":"2023-10-08T05:37:43.746332+0000","last_active":"2023-10-08T05:37:44.727173+0000","last_peered":"2023-10-08T05:37:44.727173+0000","last_clean":"2023-10-08T05:37:44.727173+0000","last_became_active":"2023-10-08T05:37:43.746017+0000","last_became_peered":"2023-10-08T05:37:43.746017+0000","last_unstale":"2023-10-08T05:37:44.727173+0000","last_undegraded":"2023-10-08T05:37:44.727173+0000","last_fullsized":"2023-10-08T05:37:44.727173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:43.928950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745741+0000","last_change":"2023-10-08T05:37:44.753244+0000","last_active":"2023-10-08T05:37:45.745741+0000","last_peered":"2023-10-08T05:37:45.745741+0000","last_clean":"2023-10-08T05:37:45.745741+0000","last_became_active":"2023-10-08T05:37:44.752613+0000","last_became_peered":"2023-10-08T05:37:44.752613+0000","last_unstale":"2023-10-08T05:37:45.745741+0000","last_undegraded":"2023-10-08T05:37:45.745741+0000","last_fullsized":"2023-10-08T05:37:45.745741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:14.437254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.771404+0000","last_change":"2023-10-08T05:37:43.750157+0000","last_active":"2023-10-08T05:37:44.771404+0000","last_peered":"2023-10-08T05:37:44.771404+0000","last_clean":"2023-10-08T05:37:44.771404+0000","last_became_active":"2023-10-08T05:37:43.749846+0000","last_became_peered":"2023-10-08T05:37:43.749846+0000","last_unstale":"2023-10-08T05:37:44.771404+0000","last_undegraded":"2023-10-08T05:37:44.771404+0000","last_fullsized":"2023-10-08T05:37:44.771404+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:44.557511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745813+0000","last_change":"2023-10-08T05:37:44.759427+0000","last_active":"2023-10-08T05:37:45.745813+0000","last_peered":"2023-10-08T05:37:45.745813+0000","last_clean":"2023-10-08T05:37:45.745813+0000","last_became_active":"2023-10-08T05:37:44.759110+0000","last_became_peered":"2023-10-08T05:37:44.759110+0000","last_unstale":"2023-10-08T05:37:45.745813+0000","last_undegraded":"2023-10-08T05:37:45.745813+0000","last_fullsized":"2023-10-08T05:37:45.745813+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:01.744941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.771983+0000","last_change":"2023-10-08T05:37:43.752005+0000","last_active":"2023-10-08T05:37:44.771983+0000","last_peered":"2023-10-08T05:37:44.771983+0000","last_clean":"2023-10-08T05:37:44.771983+0000","last_became_active":"2023-10-08T05:37:43.751525+0000","last_became_peered":"2023-10-08T05:37:43.751525+0000","last_unstale":"2023-10-08T05:37:44.771983+0000","last_undegraded":"2023-10-08T05:37:44.771983+0000","last_fullsized":"2023-10-08T05:37:44.771983+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:49:45.890117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.771054+0000","last_change":"2023-10-08T05:37:44.762466+0000","last_active":"2023-10-08T05:37:45.771054+0000","last_peered":"2023-10-08T05:37:45.771054+0000","last_clean":"2023-10-08T05:37:45.771054+0000","last_became_active":"2023-10-08T05:37:44.761918+0000","last_became_peered":"2023-10-08T05:37:44.761918+0000","last_unstale":"2023-10-08T05:37:45.771054+0000","last_undegraded":"2023-10-08T05:37:45.771054+0000","last_fullsized":"2023-10-08T05:37:45.771054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:40:49.354151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.762008+0000","last_change":"2023-10-08T05:37:43.745487+0000","last_active":"2023-10-08T05:37:44.762008+0000","last_peered":"2023-10-08T05:37:44.762008+0000","last_clean":"2023-10-08T05:37:44.762008+0000","last_became_active":"2023-10-08T05:37:43.744512+0000","last_became_peered":"2023-10-08T05:37:43.744512+0000","last_unstale":"2023-10-08T05:37:44.762008+0000","last_undegraded":"2023-10-08T05:37:44.762008+0000","last_fullsized":"2023-10-08T05:37:44.762008+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:16:37.143405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.262300+0000","last_change":"2023-10-08T05:37:44.762051+0000","last_active":"2023-10-08T05:37:46.262300+0000","last_peered":"2023-10-08T05:37:46.262300+0000","last_clean":"2023-10-08T05:37:46.262300+0000","last_became_active":"2023-10-08T05:37:44.761448+0000","last_became_peered":"2023-10-08T05:37:44.761448+0000","last_unstale":"2023-10-08T05:37:46.262300+0000","last_undegraded":"2023-10-08T05:37:46.262300+0000","last_fullsized":"2023-10-08T05:37:46.262300+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:24.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.767442+0000","last_change":"2023-10-08T05:37:43.748895+0000","last_active":"2023-10-08T05:37:44.767442+0000","last_peered":"2023-10-08T05:37:44.767442+0000","last_clean":"2023-10-08T05:37:44.767442+0000","last_became_active":"2023-10-08T05:37:43.748618+0000","last_became_peered":"2023-10-08T05:37:43.748618+0000","last_unstale":"2023-10-08T05:37:44.767442+0000","last_undegraded":"2023-10-08T05:37:44.767442+0000","last_fullsized":"2023-10-08T05:37:44.767442+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:35.696328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865544+0000","last_change":"2023-10-08T05:37:44.747500+0000","last_active":"2023-10-08T05:37:45.865544+0000","last_peered":"2023-10-08T05:37:45.865544+0000","last_clean":"2023-10-08T05:37:45.865544+0000","last_became_active":"2023-10-08T05:37:44.747123+0000","last_became_peered":"2023-10-08T05:37:44.747123+0000","last_unstale":"2023-10-08T05:37:45.865544+0000","last_undegraded":"2023-10-08T05:37:45.865544+0000","last_fullsized":"2023-10-08T05:37:45.865544+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:56:21.173836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.747744+0000","last_change":"2023-10-08T05:37:43.743970+0000","last_active":"2023-10-08T05:37:44.747744+0000","last_peered":"2023-10-08T05:37:44.747744+0000","last_clean":"2023-10-08T05:37:44.747744+0000","last_became_active":"2023-10-08T05:37:43.743106+0000","last_became_peered":"2023-10-08T05:37:43.743106+0000","last_unstale":"2023-10-08T05:37:44.747744+0000","last_undegraded":"2023-10-08T05:37:44.747744+0000","last_fullsized":"2023-10-08T05:37:44.747744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:22.404085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.927428+0000","last_change":"2023-10-08T05:37:44.750306+0000","last_active":"2023-10-08T05:37:45.927428+0000","last_peered":"2023-10-08T05:37:45.927428+0000","last_clean":"2023-10-08T05:37:45.927428+0000","last_became_active":"2023-10-08T05:37:44.750034+0000","last_became_peered":"2023-10-08T05:37:44.750034+0000","last_unstale":"2023-10-08T05:37:45.927428+0000","last_undegraded":"2023-10-08T05:37:45.927428+0000","last_fullsized":"2023-10-08T05:37:45.927428+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:01.744941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.763551+0000","last_change":"2023-10-08T05:37:43.728582+0000","last_active":"2023-10-08T05:37:44.763551+0000","last_peered":"2023-10-08T05:37:44.763551+0000","last_clean":"2023-10-08T05:37:44.763551+0000","last_became_active":"2023-10-08T05:37:43.728172+0000","last_became_peered":"2023-10-08T05:37:43.728172+0000","last_unstale":"2023-10-08T05:37:44.763551+0000","last_undegraded":"2023-10-08T05:37:44.763551+0000","last_fullsized":"2023-10-08T05:37:44.763551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:20:52.604321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260308+0000","last_change":"2023-10-08T05:37:44.742716+0000","last_active":"2023-10-08T05:37:46.260308+0000","last_peered":"2023-10-08T05:37:46.260308+0000","last_clean":"2023-10-08T05:37:46.260308+0000","last_became_active":"2023-10-08T05:37:44.742440+0000","last_became_peered":"2023-10-08T05:37:44.742440+0000","last_unstale":"2023-10-08T05:37:46.260308+0000","last_undegraded":"2023-10-08T05:37:46.260308+0000","last_fullsized":"2023-10-08T05:37:46.260308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:49.881310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.720348+0000","last_change":"2023-10-08T05:37:43.744243+0000","last_active":"2023-10-08T05:37:44.720348+0000","last_peered":"2023-10-08T05:37:44.720348+0000","last_clean":"2023-10-08T05:37:44.720348+0000","last_became_active":"2023-10-08T05:37:43.743356+0000","last_became_peered":"2023-10-08T05:37:43.743356+0000","last_unstale":"2023-10-08T05:37:44.720348+0000","last_undegraded":"2023-10-08T05:37:44.720348+0000","last_fullsized":"2023-10-08T05:37:44.720348+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:15.449045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770034+0000","last_change":"2023-10-08T05:37:44.750298+0000","last_active":"2023-10-08T05:37:45.770034+0000","last_peered":"2023-10-08T05:37:45.770034+0000","last_clean":"2023-10-08T05:37:45.770034+0000","last_became_active":"2023-10-08T05:37:44.749905+0000","last_became_peered":"2023-10-08T05:37:44.749905+0000","last_unstale":"2023-10-08T05:37:45.770034+0000","last_undegraded":"2023-10-08T05:37:45.770034+0000","last_fullsized":"2023-10-08T05:37:45.770034+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:14.437254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.741455+0000","last_change":"2023-10-08T05:37:43.749749+0000","last_active":"2023-10-08T05:37:44.741455+0000","last_peered":"2023-10-08T05:37:44.741455+0000","last_clean":"2023-10-08T05:37:44.741455+0000","last_became_active":"2023-10-08T05:37:43.749413+0000","last_became_peered":"2023-10-08T05:37:43.749413+0000","last_unstale":"2023-10-08T05:37:44.741455+0000","last_undegraded":"2023-10-08T05:37:44.741455+0000","last_fullsized":"2023-10-08T05:37:44.741455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:23.342919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.745896+0000","last_change":"2023-10-08T05:37:44.752878+0000","last_active":"2023-10-08T05:37:45.745896+0000","last_peered":"2023-10-08T05:37:45.745896+0000","last_clean":"2023-10-08T05:37:45.745896+0000","last_became_active":"2023-10-08T05:37:44.751854+0000","last_became_peered":"2023-10-08T05:37:44.751854+0000","last_unstale":"2023-10-08T05:37:45.745896+0000","last_undegraded":"2023-10-08T05:37:45.745896+0000","last_fullsized":"2023-10-08T05:37:45.745896+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:19.962298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.720290+0000","last_change":"2023-10-08T05:37:43.733311+0000","last_active":"2023-10-08T05:37:44.720290+0000","last_peered":"2023-10-08T05:37:44.720290+0000","last_clean":"2023-10-08T05:37:44.720290+0000","last_became_active":"2023-10-08T05:37:43.733051+0000","last_became_peered":"2023-10-08T05:37:43.733051+0000","last_unstale":"2023-10-08T05:37:44.720290+0000","last_undegraded":"2023-10-08T05:37:44.720290+0000","last_fullsized":"2023-10-08T05:37:44.720290+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:46:48.047033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749187+0000","last_change":"2023-10-08T05:37:44.769844+0000","last_active":"2023-10-08T05:37:45.749187+0000","last_peered":"2023-10-08T05:37:45.749187+0000","last_clean":"2023-10-08T05:37:45.749187+0000","last_became_active":"2023-10-08T05:37:44.769551+0000","last_became_peered":"2023-10-08T05:37:44.769551+0000","last_unstale":"2023-10-08T05:37:45.749187+0000","last_undegraded":"2023-10-08T05:37:45.749187+0000","last_fullsized":"2023-10-08T05:37:45.749187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:19:55.053932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.772565+0000","last_change":"2023-10-08T05:37:43.749129+0000","last_active":"2023-10-08T05:37:44.772565+0000","last_peered":"2023-10-08T05:37:44.772565+0000","last_clean":"2023-10-08T05:37:44.772565+0000","last_became_active":"2023-10-08T05:37:43.748832+0000","last_became_peered":"2023-10-08T05:37:43.748832+0000","last_unstale":"2023-10-08T05:37:44.772565+0000","last_undegraded":"2023-10-08T05:37:44.772565+0000","last_fullsized":"2023-10-08T05:37:44.772565+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:55.523677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.927337+0000","last_change":"2023-10-08T05:37:44.753369+0000","last_active":"2023-10-08T05:37:45.927337+0000","last_peered":"2023-10-08T05:37:45.927337+0000","last_clean":"2023-10-08T05:37:45.927337+0000","last_became_active":"2023-10-08T05:37:44.753075+0000","last_became_peered":"2023-10-08T05:37:44.753075+0000","last_unstale":"2023-10-08T05:37:45.927337+0000","last_undegraded":"2023-10-08T05:37:45.927337+0000","last_fullsized":"2023-10-08T05:37:45.927337+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:14.437254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.865411+0000","last_change":"2023-10-08T05:37:43.744435+0000","last_active":"2023-10-08T05:37:45.865411+0000","last_peered":"2023-10-08T05:37:45.865411+0000","last_clean":"2023-10-08T05:37:45.865411+0000","last_became_active":"2023-10-08T05:37:43.744157+0000","last_became_peered":"2023-10-08T05:37:43.744157+0000","last_unstale":"2023-10-08T05:37:45.865411+0000","last_undegraded":"2023-10-08T05:37:45.865411+0000","last_fullsized":"2023-10-08T05:37:45.865411+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:04.744586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.746034+0000","last_change":"2023-10-08T05:37:44.759675+0000","last_active":"2023-10-08T05:37:45.746034+0000","last_peered":"2023-10-08T05:37:45.746034+0000","last_clean":"2023-10-08T05:37:45.746034+0000","last_became_active":"2023-10-08T05:37:44.759323+0000","last_became_peered":"2023-10-08T05:37:44.759323+0000","last_unstale":"2023-10-08T05:37:45.746034+0000","last_undegraded":"2023-10-08T05:37:45.746034+0000","last_fullsized":"2023-10-08T05:37:45.746034+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:26:53.666665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.730845+0000","last_change":"2023-10-08T05:37:43.733742+0000","last_active":"2023-10-08T05:37:44.730845+0000","last_peered":"2023-10-08T05:37:44.730845+0000","last_clean":"2023-10-08T05:37:44.730845+0000","last_became_active":"2023-10-08T05:37:43.733486+0000","last_became_peered":"2023-10-08T05:37:43.733486+0000","last_unstale":"2023-10-08T05:37:44.730845+0000","last_undegraded":"2023-10-08T05:37:44.730845+0000","last_fullsized":"2023-10-08T05:37:44.730845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:41.403273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.260860+0000","last_change":"2023-10-08T05:37:44.744388+0000","last_active":"2023-10-08T05:37:46.260860+0000","last_peered":"2023-10-08T05:37:46.260860+0000","last_clean":"2023-10-08T05:37:46.260860+0000","last_became_active":"2023-10-08T05:37:44.744129+0000","last_became_peered":"2023-10-08T05:37:44.744129+0000","last_unstale":"2023-10-08T05:37:46.260860+0000","last_undegraded":"2023-10-08T05:37:46.260860+0000","last_fullsized":"2023-10-08T05:37:46.260860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:46.616809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.731200+0000","last_change":"2023-10-08T05:37:43.732879+0000","last_active":"2023-10-08T05:37:44.731200+0000","last_peered":"2023-10-08T05:37:44.731200+0000","last_clean":"2023-10-08T05:37:44.731200+0000","last_became_active":"2023-10-08T05:37:43.732616+0000","last_became_peered":"2023-10-08T05:37:43.732616+0000","last_unstale":"2023-10-08T05:37:44.731200+0000","last_undegraded":"2023-10-08T05:37:44.731200+0000","last_fullsized":"2023-10-08T05:37:44.731200+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:26.955372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.749160+0000","last_change":"2023-10-08T05:37:44.757884+0000","last_active":"2023-10-08T05:37:45.749160+0000","last_peered":"2023-10-08T05:37:45.749160+0000","last_clean":"2023-10-08T05:37:45.749160+0000","last_became_active":"2023-10-08T05:37:44.757581+0000","last_became_peered":"2023-10-08T05:37:44.757581+0000","last_unstale":"2023-10-08T05:37:45.749160+0000","last_undegraded":"2023-10-08T05:37:45.749160+0000","last_fullsized":"2023-10-08T05:37:45.749160+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:55:24.355151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.724702+0000","last_change":"2023-10-08T05:37:43.733421+0000","last_active":"2023-10-08T05:37:44.724702+0000","last_peered":"2023-10-08T05:37:44.724702+0000","last_clean":"2023-10-08T05:37:44.724702+0000","last_became_active":"2023-10-08T05:37:43.732803+0000","last_became_peered":"2023-10-08T05:37:43.732803+0000","last_unstale":"2023-10-08T05:37:44.724702+0000","last_undegraded":"2023-10-08T05:37:44.724702+0000","last_fullsized":"2023-10-08T05:37:44.724702+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:27.563368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:46.261206+0000","last_change":"2023-10-08T05:37:44.744497+0000","last_active":"2023-10-08T05:37:46.261206+0000","last_peered":"2023-10-08T05:37:46.261206+0000","last_clean":"2023-10-08T05:37:46.261206+0000","last_became_active":"2023-10-08T05:37:44.744171+0000","last_became_peered":"2023-10-08T05:37:44.744171+0000","last_unstale":"2023-10-08T05:37:46.261206+0000","last_undegraded":"2023-10-08T05:37:46.261206+0000","last_fullsized":"2023-10-08T05:37:46.261206+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:23.416317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.723976+0000","last_change":"2023-10-08T05:37:43.736772+0000","last_active":"2023-10-08T05:37:44.723976+0000","last_peered":"2023-10-08T05:37:44.723976+0000","last_clean":"2023-10-08T05:37:44.723976+0000","last_became_active":"2023-10-08T05:37:43.736441+0000","last_became_peered":"2023-10-08T05:37:43.736441+0000","last_unstale":"2023-10-08T05:37:44.723976+0000","last_undegraded":"2023-10-08T05:37:44.723976+0000","last_fullsized":"2023-10-08T05:37:44.723976+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:12.090988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.927064+0000","last_change":"2023-10-08T05:37:44.752577+0000","last_active":"2023-10-08T05:37:45.927064+0000","last_peered":"2023-10-08T05:37:45.927064+0000","last_clean":"2023-10-08T05:37:45.927064+0000","last_became_active":"2023-10-08T05:37:44.752228+0000","last_became_peered":"2023-10-08T05:37:44.752228+0000","last_unstale":"2023-10-08T05:37:45.927064+0000","last_undegraded":"2023-10-08T05:37:45.927064+0000","last_fullsized":"2023-10-08T05:37:45.927064+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:26.951190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.733547+0000","last_change":"2023-10-08T05:37:43.733011+0000","last_active":"2023-10-08T05:37:44.733547+0000","last_peered":"2023-10-08T05:37:44.733547+0000","last_clean":"2023-10-08T05:37:44.733547+0000","last_became_active":"2023-10-08T05:37:43.732744+0000","last_became_peered":"2023-10-08T05:37:43.732744+0000","last_unstale":"2023-10-08T05:37:44.733547+0000","last_undegraded":"2023-10-08T05:37:44.733547+0000","last_fullsized":"2023-10-08T05:37:44.733547+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:51.066926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-10-08T05:37:45.770624+0000","last_change":"2023-10-08T05:37:44.760802+0000","last_active":"2023-10-08T05:37:45.770624+0000","last_peered":"2023-10-08T05:37:45.770624+0000","last_clean":"2023-10-08T05:37:45.770624+0000","last_became_active":"2023-10-08T05:37:44.760324+0000","last_became_peered":"2023-10-08T05:37:44.760324+0000","last_unstale":"2023-10-08T05:37:45.770624+0000","last_undegraded":"2023-10-08T05:37:45.770624+0000","last_fullsized":"2023-10-08T05:37:45.770624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:43.697452+0000","last_clean_scrub_stamp":"2023-10-08T05:37:43.697452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:26:53.666665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.739614+0000","last_change":"2023-10-08T05:37:43.748040+0000","last_active":"2023-10-08T05:37:44.739614+0000","last_peered":"2023-10-08T05:37:44.739614+0000","last_clean":"2023-10-08T05:37:44.739614+0000","last_became_active":"2023-10-08T05:37:43.747745+0000","last_became_peered":"2023-10-08T05:37:43.747745+0000","last_unstale":"2023-10-08T05:37:44.739614+0000","last_undegraded":"2023-10-08T05:37:44.739614+0000","last_fullsized":"2023-10-08T05:37:44.739614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:37:42.685220+0000","last_clean_scrub_stamp":"2023-10-08T05:37:42.685220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:30.227070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":181,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-08T05:37:44.738648+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:37:44.738648+0000","last_peered":"2023-10-08T05:37:44.738648+0000","last_clean":"2023-10-08T05:37:44.738648+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:37:44.738648+0000","last_undegraded":"2023-10-08T05:37:44.738648+0000","last_fullsized":"2023-10-08T05:37:44.738648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343780,"statfs":{"total":96636764160,"available":96608030720,"internally_reserved":0,"allocated":1060864,"data_stored":785889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.2360000000000002,"5min":2.2360000000000002,"15min":2.2360000000000002},"last":0.75},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.877}]},{"osd":1,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.3520000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.5349999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":2.5579999999999998},"last":1.2809999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.4529999999999998,"5min":2.4529999999999998,"15min":2.4529999999999998},"last":1.254},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.677,"5min":2.677,"15min":2.677},"last":1.605}]},{"osd":5,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":2.169},{"interface":"front","average":{"1min":1.524,"5min":1.524,"15min":1.524},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":2.206}]},{"osd":6,"last update":"Sun Oct 8 05:37:06 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":3.5249999999999999,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.754},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":2.101}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":194421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.337},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.47}]},{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.175}]},{"osd":2,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.367},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.111}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":1.4219999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.484,"5min":1.484,"15min":1.484},"min":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.2190000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":4.3099999999999996,"5min":4.3099999999999996,"15min":4.3099999999999996},"last":1.395},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"min":{"1min":1.175,"5min":1.175,"15min":1.175},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.518}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.23,"5min":1.23,"15min":1.23},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"max":{"1min":4.1550000000000002,"5min":4.1550000000000002,"15min":4.1550000000000002},"last":1.496}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":193951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.302,"5min":2.302,"15min":2.302},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":3.5529999999999999,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1.4810000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.294,"15min":1.294},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.5669999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":3.6739999999999999,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.21,"5min":1.21,"15min":1.21},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.3759999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":6.782,"5min":6.782,"15min":6.782},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"last":1.5820000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.591,"15min":1.591},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.1349999999999998,"5min":6.1349999999999998,"15min":6.1349999999999998},"last":1.786},{"interface":"front","average":{"1min":1.762,"5min":1.762,"15min":1.762},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":7.0069999999999997,"5min":7.0069999999999997,"15min":7.0069999999999997},"last":1.2350000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":7.2569999999999997,"5min":7.2569999999999997,"15min":7.2569999999999997},"last":1.044},{"interface":"front","average":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":6.2199999999999998,"5min":6.2199999999999998,"15min":6.2199999999999998},"last":1.5229999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":6.9279999999999999,"5min":6.9279999999999999,"15min":6.9279999999999999},"last":1.431},{"interface":"front","average":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":7.335,"5min":7.335,"15min":7.335},"last":1.3340000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":194443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0369999999999999},"last":1.3029999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":1.345}]},{"osd":2,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.1499999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.97899999999999998}]},{"osd":5,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.2370000000000001,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.643},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.5569999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":1.105},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.5149999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:37:07 2023","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.667,"15min":1.667},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.448},{"interface":"front","average":{"1min":1.76,"5min":1.76,"15min":1.76},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.5489999999999999,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.379}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":466944,"data_stored":195485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.369},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.05}]},{"osd":1,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.204},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.272}]},{"osd":2,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":1.4399999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.107}]},{"osd":4,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.663}]},{"osd":5,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.7090000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.756}]},{"osd":7,"last update":"Sun Oct 8 05:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":1.609},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.48}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":454656,"data_stored":193909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":1.077}]},{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.71199999999999997}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":2.3889999999999998,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":1.4330000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":1.137},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.254}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":2.8170000000000002,"5min":2.8170000000000002,"15min":2.8170000000000002},"last":1.3220000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.851,"5min":2.851,"15min":2.851},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":1.282}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.498,"15min":1.498},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":3.0529999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.4729999999999999,"5min":2.4729999999999999,"15min":2.4729999999999999},"last":1.379}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31628,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340212,"statfs":{"total":96636764160,"available":96604377088,"internally_reserved":0,"allocated":454656,"data_stored":193929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.0389999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":2.0750000000000002,"5min":2.0750000000000002,"15min":2.0750000000000002},"last":0.78400000000000003}]},{"osd":3,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.75700000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.90100000000000002}]},{"osd":5,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.3069999999999999,"5min":2.3069999999999999,"15min":2.3069999999999999},"last":0.86899999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.129}]},{"osd":7,"last update":"Sun Oct 8 05:37:03 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.105},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.2769999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1048576,"data_stored":784241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.439}]},{"osd":2,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.88400000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.51300000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.26,"15min":1.26},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.6560000000000001,"5min":2.6560000000000001,"15min":2.6560000000000001},"last":1.165}]},{"osd":5,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.054},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.7069999999999999,"5min":2.7069999999999999,"15min":2.7069999999999999},"last":0.85399999999999998}]},{"osd":6,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.115}]},{"osd":7,"last update":"Sun Oct 8 05:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.006},{"interface":"front","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.0720000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:37:54.904 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T05:37:54.905 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T05:37:55.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.166+0000 7f9f2ced0640 1 Processor -- start 2023-10-08T05:37:55.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.166+0000 7f9f2ced0640 1 -- start start 2023-10-08T05:37:55.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.167+0000 7f9f2ced0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9f28070690 0x7f9f28070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.167+0000 7f9f2ced0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f280c8c60 con 0x7f9f2806cfa0 2023-10-08T05:37:55.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.167+0000 7f9f2ced0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f280b1ac0 con 0x7f9f280cb500 2023-10-08T05:37:55.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.167+0000 7f9f2ced0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280b1da0 con 0x7f9f28070690 2023-10-08T05:37:55.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.168+0000 7f9f25d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9f2806cfa0 0x7f9f2815b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59904/0 (socket says 172.21.15.70:59904) 2023-10-08T05:37:55.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.168+0000 7f9f25d9b640 1 -- 172.21.15.70:0/1765698862 learned_addr learned my addr 172.21.15.70:0/1765698862 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:55.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.168+0000 7f9f26d9d640 1 --2- 172.21.15.70:0/1765698862 >> v2:172.21.15.70:3300/0 conn(0x7f9f28070690 0x7f9f28070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.168+0000 7f9f2559a640 1 -- 172.21.15.70:0/1765698862 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493565011 0 0) 0x7f9f280c8c60 con 0x7f9f2806cfa0 2023-10-08T05:37:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.168+0000 7f9f2559a640 1 -- 172.21.15.70:0/1765698862 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f10003520 con 0x7f9f2806cfa0 2023-10-08T05:37:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.168+0000 7f9f26d9d640 1 -- 172.21.15.70:0/1765698862 >> v1:172.21.15.181:6789/0 conn(0x7f9f280cb500 legacy=0x7f9f280cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.168+0000 7f9f26d9d640 1 -- 172.21.15.70:0/1765698862 >> v1:172.21.15.70:6789/0 conn(0x7f9f2806cfa0 legacy=0x7f9f2815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.168+0000 7f9f26d9d640 1 -- 172.21.15.70:0/1765698862 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f280b2080 con 0x7f9f28070690 2023-10-08T05:37:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.170+0000 7f9f26d9d640 1 --2- 172.21.15.70:0/1765698862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f28070690 0x7f9f28070b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009d90 tx=0x7f9f2002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6a636873a18f3b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.170+0000 7f9f2559a640 1 -- 172.21.15.70:0/1765698862 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2002cbd0 con 0x7f9f28070690 2023-10-08T05:37:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.170+0000 7f9f2559a640 1 -- 172.21.15.70:0/1765698862 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2002cd30 con 0x7f9f28070690 2023-10-08T05:37:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.170+0000 7f9f2559a640 1 -- 172.21.15.70:0/1765698862 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2002cea0 con 0x7f9f28070690 2023-10-08T05:37:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.170+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1765698862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f28070690 msgr2=0x7f9f28070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.170+0000 7f9f2ced0640 1 --2- 172.21.15.70:0/1765698862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f28070690 0x7f9f28070b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009d90 tx=0x7f9f2002be20 comp rx=0 tx=0).stop 2023-10-08T05:37:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.170+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1765698862 shutdown_connections 2023-10-08T05:37:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.170+0000 7f9f2ced0640 1 --2- 172.21.15.70:0/1765698862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f28070690 0x7f9f28070b40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.170+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1765698862 >> 172.21.15.70:0/1765698862 conn(0x7f9f2806a140 msgr2=0x7f9f2806a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.171+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1765698862 shutdown_connections 2023-10-08T05:37:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.171+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1765698862 wait complete. 2023-10-08T05:37:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.172+0000 7f9f2ced0640 1 Processor -- start 2023-10-08T05:37:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.172+0000 7f9f2ced0640 1 -- start start 2023-10-08T05:37:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.173+0000 7f9f2ced0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f2806cfa0 0x7f9f280af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.173+0000 7f9f2ced0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f28070690 0x7f9f280ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.173+0000 7f9f2ced0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f280cb500 0x7f9f281e17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.173+0000 7f9f2ced0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280aed40 con 0x7f9f280cb500 2023-10-08T05:37:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.173+0000 7f9f2ced0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280af680 con 0x7f9f2806cfa0 2023-10-08T05:37:55.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.174+0000 7f9f2ced0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280ade40 con 0x7f9f28070690 2023-10-08T05:37:55.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.174+0000 7f9f26d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f280cb500 0x7f9f281e17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.174+0000 7f9f26d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f280cb500 0x7f9f281e17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45330/0 (socket says 172.21.15.70:45330) 2023-10-08T05:37:55.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.174+0000 7f9f26d9d640 1 -- 172.21.15.70:0/1499038783 learned_addr learned my addr 172.21.15.70:0/1499038783 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:55.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.174+0000 7f9f25d9b640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f28070690 0x7f9f280ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.174+0000 7f9f26d9d640 1 -- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f28070690 msgr2=0x7f9f280ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.174+0000 7f9f26d9d640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f28070690 0x7f9f280ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.174+0000 7f9f26d9d640 1 -- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f2806cfa0 msgr2=0x7f9f280af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.174+0000 7f9f26d9d640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f2806cfa0 0x7f9f280af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.174+0000 7f9f26d9d640 1 -- 172.21.15.70:0/1499038783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f20009a70 con 0x7f9f280cb500 2023-10-08T05:37:55.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.175+0000 7f9f26d9d640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f280cb500 0x7f9f281e17b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9f2002c430 tx=0x7f9f20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:55.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.175+0000 7f9f16ffd640 1 -- 172.21.15.70:0/1499038783 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2002cea0 con 0x7f9f280cb500 2023-10-08T05:37:55.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.175+0000 7f9f16ffd640 1 -- 172.21.15.70:0/1499038783 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2003e630 con 0x7f9f280cb500 2023-10-08T05:37:55.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.175+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1499038783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f280ae0c0 con 0x7f9f280cb500 2023-10-08T05:37:55.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.175+0000 7f9f16ffd640 1 -- 172.21.15.70:0/1499038783 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2003e9e0 con 0x7f9f280cb500 2023-10-08T05:37:55.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.175+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1499038783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f280ae680 con 0x7f9f280cb500 2023-10-08T05:37:55.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.176+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1499038783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2806d990 con 0x7f9f280cb500 2023-10-08T05:37:55.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.177+0000 7f9f16ffd640 1 -- 172.21.15.70:0/1499038783 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f9f2003b050 con 0x7f9f280cb500 2023-10-08T05:37:55.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.178+0000 7f9f16ffd640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9f0806ae70 0x7f9f0806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.178+0000 7f9f16ffd640 1 -- 172.21.15.70:0/1499038783 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9f200ab840 con 0x7f9f280cb500 2023-10-08T05:37:55.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.179+0000 7f9f2659c640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9f0806ae70 0x7f9f0806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.180+0000 7f9f2659c640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9f0806ae70 0x7f9f0806d330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f18003cc0 tx=0x7f9f180079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.182+0000 7f9f16ffd640 1 -- 172.21.15.70:0/1499038783 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f200767a0 con 0x7f9f280cb500 2023-10-08T05:37:55.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.442+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1499038783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9f28071cb0 con 0x7f9f280cb500 2023-10-08T05:37:55.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.447+0000 7f9f16ffd640 1 -- 172.21.15.70:0/1499038783 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f9f20009450 con 0x7f9f280cb500 2023-10-08T05:37:55.449 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T05:37:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T05:37:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T05:37:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T05:37:55.449 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T05:37:55.449 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T05:37:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.450+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9f0806ae70 msgr2=0x7f9f0806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.450+0000 7f9f2ced0640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9f0806ae70 0x7f9f0806d330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f18003cc0 tx=0x7f9f180079b0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.450+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1499038783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f280cb500 msgr2=0x7f9f281e17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.450+0000 7f9f2ced0640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f280cb500 0x7f9f281e17b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9f2002c430 tx=0x7f9f20004a40 comp rx=0 tx=0).stop 2023-10-08T05:37:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.450+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1499038783 shutdown_connections 2023-10-08T05:37:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.450+0000 7f9f2ced0640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f9f0806ae70 0x7f9f0806d330 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.450+0000 7f9f2ced0640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f280cb500 0x7f9f281e17b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.450+0000 7f9f2ced0640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9f28070690 0x7f9f280ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.451+0000 7f9f2ced0640 1 --2- 172.21.15.70:0/1499038783 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9f2806cfa0 0x7f9f280af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.451+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1499038783 >> 172.21.15.70:0/1499038783 conn(0x7f9f2806a140 msgr2=0x7f9f280cdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.451+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1499038783 shutdown_connections 2023-10-08T05:37:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.451+0000 7f9f2ced0640 1 -- 172.21.15.70:0/1499038783 wait complete. 2023-10-08T05:37:55.470 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:37:55.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.680+0000 7f4574bf5640 1 Processor -- start 2023-10-08T05:37:55.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.680+0000 7f4574bf5640 1 -- start start 2023-10-08T05:37:55.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f4574bf5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f457015cd90 0x7f45700702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f4574bf5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45700c48f0 con 0x7f457006c890 2023-10-08T05:37:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f4574bf5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45700c3ce0 con 0x7f4570160940 2023-10-08T05:37:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f4574bf5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45700c4d80 con 0x7f457015cd90 2023-10-08T05:37:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456ed9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f457006c890 0x7f457006cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59918/0 (socket says 172.21.15.70:59918) 2023-10-08T05:37:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456ed9d640 1 -- 172.21.15.70:0/1297894310 learned_addr learned my addr 172.21.15.70:0/1297894310 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:55.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456e59c640 1 --2- 172.21.15.70:0/1297894310 >> v2:172.21.15.181:3300/0 conn(0x7f457015cd90 0x7f45700702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025399241 0 0) 0x7f45700c48f0 con 0x7f457006c890 2023-10-08T05:37:55.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f455c003520 con 0x7f457006c890 2023-10-08T05:37:55.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4199804943 0 0) 0x7f455c003520 con 0x7f457006c890 2023-10-08T05:37:55.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 >> v2:172.21.15.181:3300/0 conn(0x7f457015cd90 msgr2=0x7f45700702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:55.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 --2- 172.21.15.70:0/1297894310 >> v2:172.21.15.181:3300/0 conn(0x7f457015cd90 0x7f45700702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 >> v1:172.21.15.181:6789/0 conn(0x7f4570160940 legacy=0x7f4570070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45700c3fc0 con 0x7f457006c890 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f45680019f0 con 0x7f457006c890 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 >> v1:172.21.15.70:6789/0 conn(0x7f457006c890 legacy=0x7f457006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455c003970 0x7f455c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f455c0047c0 0x7f455c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f455c005180 0x7f455c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c00bff0 con 0x7f455c0047c0 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c00c160 con 0x7f455c003970 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.682+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c00c320 con 0x7f455c005180 2023-10-08T05:37:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.683+0000 7f4567fff640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f455c0047c0 0x7f455c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.683+0000 7f456e59c640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455c003970 0x7f455c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.683+0000 7f456ed9d640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f455c005180 0x7f455c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.683+0000 7f4567fff640 1 -- 172.21.15.70:0/1297894310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f455c005180 msgr2=0x7f455c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.683+0000 7f4567fff640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f455c005180 0x7f455c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.683+0000 7f4567fff640 1 -- 172.21.15.70:0/1297894310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455c003970 msgr2=0x7f455c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.683+0000 7f4567fff640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455c003970 0x7f455c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.683+0000 7f4567fff640 1 -- 172.21.15.70:0/1297894310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c003520 con 0x7f455c0047c0 2023-10-08T05:37:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.683+0000 7f456ed9d640 1 -- 172.21.15.70:0/1297894310 reap_dead start 2023-10-08T05:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.684+0000 7f4567fff640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f455c0047c0 0x7f455c004c40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4558001750 tx=0x7f455c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2688c67621186aef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.684+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45580052d0 con 0x7f455c0047c0 2023-10-08T05:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.685+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455802e050 con 0x7f455c0047c0 2023-10-08T05:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.685+0000 7f456dd9b640 1 -- 172.21.15.70:0/1297894310 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558003830 con 0x7f455c0047c0 2023-10-08T05:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.685+0000 7f4574bf5640 1 -- 172.21.15.70:0/1297894310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f455c0047c0 msgr2=0x7f455c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.685+0000 7f4574bf5640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f455c0047c0 0x7f455c004c40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4558001750 tx=0x7f455c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.685+0000 7f4574bf5640 1 -- 172.21.15.70:0/1297894310 shutdown_connections 2023-10-08T05:37:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.685+0000 7f4574bf5640 1 --2- 172.21.15.70:0/1297894310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f455c0047c0 0x7f455c004c40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.685+0000 7f4574bf5640 1 -- 172.21.15.70:0/1297894310 >> 172.21.15.70:0/1297894310 conn(0x7f457006a380 msgr2=0x7f45700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.685+0000 7f4574bf5640 1 -- 172.21.15.70:0/1297894310 shutdown_connections 2023-10-08T05:37:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.685+0000 7f4574bf5640 1 -- 172.21.15.70:0/1297894310 wait complete. 2023-10-08T05:37:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.687+0000 7f4574bf5640 1 Processor -- start 2023-10-08T05:37:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.687+0000 7f4574bf5640 1 -- start start 2023-10-08T05:37:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.688+0000 7f4574bf5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455c0047c0 0x7f45700b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.688+0000 7f4574bf5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457015fbc0 0x7f4570154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.688+0000 7f4574bf5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f457015d350 0x7f457015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.688+0000 7f4574bf5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4570160040 con 0x7f457015d350 2023-10-08T05:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.688+0000 7f4574bf5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457015f790 con 0x7f455c0047c0 2023-10-08T05:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.688+0000 7f4574bf5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45700b22c0 con 0x7f457015fbc0 2023-10-08T05:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.688+0000 7f456ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f457015d350 0x7f457015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f456ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f457015d350 0x7f457015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45352/0 (socket says 172.21.15.70:45352) 2023-10-08T05:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f456ed9d640 1 -- 172.21.15.70:0/1560355427 learned_addr learned my addr 172.21.15.70:0/1560355427 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f456e59c640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455c0047c0 0x7f45700b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f4567fff640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457015fbc0 0x7f4570154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f456ed9d640 1 -- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457015fbc0 msgr2=0x7f4570154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f456ed9d640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457015fbc0 0x7f4570154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f456ed9d640 1 -- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455c0047c0 msgr2=0x7f45700b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f456ed9d640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455c0047c0 0x7f45700b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f456ed9d640 1 -- 172.21.15.70:0/1560355427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45700b1a20 con 0x7f457015d350 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f4567fff640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457015fbc0 0x7f4570154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f456ed9d640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f457015d350 0x7f457015d7d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f45700b0c20 tx=0x7f456800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.689+0000 7f4566ffd640 1 -- 172.21.15.70:0/1560355427 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568004d60 con 0x7f457015d350 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.690+0000 7f4566ffd640 1 -- 172.21.15.70:0/1560355427 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456800bb80 con 0x7f457015d350 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.690+0000 7f4574bf5640 1 -- 172.21.15.70:0/1560355427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45700b1d00 con 0x7f457015d350 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.690+0000 7f4566ffd640 1 -- 172.21.15.70:0/1560355427 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568007110 con 0x7f457015d350 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.690+0000 7f4574bf5640 1 -- 172.21.15.70:0/1560355427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45701f6040 con 0x7f457015d350 2023-10-08T05:37:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.691+0000 7f4574bf5640 1 -- 172.21.15.70:0/1560355427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4570067490 con 0x7f457015d350 2023-10-08T05:37:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.692+0000 7f4566ffd640 1 -- 172.21.15.70:0/1560355427 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f4568007270 con 0x7f457015d350 2023-10-08T05:37:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.693+0000 7f4566ffd640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f455006aea0 0x7f455006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.693+0000 7f4566ffd640 1 -- 172.21.15.70:0/1560355427 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4568089a80 con 0x7f457015d350 2023-10-08T05:37:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.693+0000 7f456e59c640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f455006aea0 0x7f455006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.695+0000 7f456e59c640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f455006aea0 0x7f455006d360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f457004fa60 tx=0x7f4560008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.697+0000 7f4566ffd640 1 -- 172.21.15.70:0/1560355427 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4568054a80 con 0x7f457015d350 2023-10-08T05:37:55.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.908+0000 7f4574bf5640 1 -- 172.21.15.70:0/1560355427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45700b0ef0 con 0x7f457015d350 2023-10-08T05:37:55.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.909+0000 7f4566ffd640 1 -- 172.21.15.70:0/1560355427 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f45680580a0 con 0x7f457015d350 2023-10-08T05:37:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:37:55.911 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/809656839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":809656839},{"type":"v1","addr":"172.21.15.70:6835","nonce":809656839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2750508216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2750508216},{"type":"v1","addr":"172.21.15.181:6836","nonce":2750508216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/607266140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":607266140},{"type":"v1","addr":"172.21.15.181:6837","nonce":607266140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:37:44.192492+0000","modified":"2023-10-08T05:37:47.238340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/480502229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":480502229},{"type":"v1","addr":"172.21.15.70:6834","nonce":480502229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-08T05:37:55.912 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 18 2023-10-08T05:37:55.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.911+0000 7f4574bf5640 1 -- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f455006aea0 msgr2=0x7f455006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.911+0000 7f4574bf5640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f455006aea0 0x7f455006d360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f457004fa60 tx=0x7f4560008040 comp rx=0 tx=0).stop 2023-10-08T05:37:55.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.911+0000 7f4574bf5640 1 -- 172.21.15.70:0/1560355427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f457015d350 msgr2=0x7f457015d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.911+0000 7f4574bf5640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f457015d350 0x7f457015d7d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f45700b0c20 tx=0x7f456800b4a0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.911+0000 7f4574bf5640 1 -- 172.21.15.70:0/1560355427 shutdown_connections 2023-10-08T05:37:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.911+0000 7f4574bf5640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f455006aea0 0x7f455006d360 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.911+0000 7f4574bf5640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f457015d350 0x7f457015d7d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.911+0000 7f4574bf5640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457015fbc0 0x7f4570154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.911+0000 7f4574bf5640 1 --2- 172.21.15.70:0/1560355427 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f455c0047c0 0x7f45700b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.911+0000 7f4574bf5640 1 -- 172.21.15.70:0/1560355427 >> 172.21.15.70:0/1560355427 conn(0x7f457006a380 msgr2=0x7f45700af220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.912+0000 7f4574bf5640 1 -- 172.21.15.70:0/1560355427 shutdown_connections 2023-10-08T05:37:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:55.912+0000 7f4574bf5640 1 -- 172.21.15.70:0/1560355427 wait complete. 2023-10-08T05:37:56.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T05:37:44.192492+0000', 'modified': '2023-10-08T05:37:47.238340+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5304}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5304': {'gid': 5304, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.70:6834/480502229', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 480502229}, {'type': 'v1', 'addr': '172.21.15.70:6834', 'nonce': 480502229}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T05:37:56.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T05:37:56.333 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T05:37:56.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.556+0000 7f38c0bae640 1 Processor -- start 2023-10-08T05:37:56.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.557+0000 7f38c0bae640 1 -- start start 2023-10-08T05:37:56.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.558+0000 7f38c0bae640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38bc15ccb0 0x7f38bc0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.558+0000 7f38c0bae640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38bc0d00e0 0x7f38bc0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.558+0000 7f38c0bae640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38bc0d2870 con 0x7f38bc0d0b90 2023-10-08T05:37:56.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.558+0000 7f38c0bae640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38bc0d1150 con 0x7f38bc0d00e0 2023-10-08T05:37:56.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.558+0000 7f38c0bae640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0d1430 con 0x7f38bc15ccb0 2023-10-08T05:37:56.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.558+0000 7f38baffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f38bc0d0b90 0x7f38bc0cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59922/0 (socket says 172.21.15.70:59922) 2023-10-08T05:37:56.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.559+0000 7f38baffd640 1 -- 172.21.15.70:0/3475273522 learned_addr learned my addr 172.21.15.70:0/3475273522 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:56.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.559+0000 7f38bbfff640 1 --2- 172.21.15.70:0/3475273522 >> v2:172.21.15.70:3300/0 conn(0x7f38bc0d00e0 0x7f38bc0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.559+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/3475273522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38bc15ccb0 0x7f38bc0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.559+0000 7f38bbfff640 1 -- 172.21.15.70:0/3475273522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38bc15ccb0 msgr2=0x7f38bc0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:56.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.559+0000 7f38bbfff640 1 --2- 172.21.15.70:0/3475273522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38bc15ccb0 0x7f38bc0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.559+0000 7f38bbfff640 1 -- 172.21.15.70:0/3475273522 >> v1:172.21.15.70:6789/0 conn(0x7f38bc0d0b90 legacy=0x7f38bc0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.559+0000 7f38bbfff640 1 -- 172.21.15.70:0/3475273522 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc0d1710 con 0x7f38bc0d00e0 2023-10-08T05:37:56.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.559+0000 7f38ba7fc640 1 -- 172.21.15.70:0/3475273522 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685832034 0 0) 0x7f38bc0d2870 con 0x7f38bc0d0b90 2023-10-08T05:37:56.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.560+0000 7f38bbfff640 1 --2- 172.21.15.70:0/3475273522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 0x7f38bc0b6060 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f38b40097a0 tx=0x7f38b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd7f9fd715c91e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:56.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.560+0000 7f38ba7fc640 1 -- 172.21.15.70:0/3475273522 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b4031830 con 0x7f38bc0d00e0 2023-10-08T05:37:56.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.561+0000 7f38ba7fc640 1 -- 172.21.15.70:0/3475273522 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b4031990 con 0x7f38bc0d00e0 2023-10-08T05:37:56.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.561+0000 7f38ba7fc640 1 -- 172.21.15.70:0/3475273522 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b4031d20 con 0x7f38bc0d00e0 2023-10-08T05:37:56.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.561+0000 7f38c0bae640 1 -- 172.21.15.70:0/3475273522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 msgr2=0x7f38bc0b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.561+0000 7f38c0bae640 1 --2- 172.21.15.70:0/3475273522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 0x7f38bc0b6060 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f38b40097a0 tx=0x7f38b402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.561+0000 7f38c0bae640 1 -- 172.21.15.70:0/3475273522 shutdown_connections 2023-10-08T05:37:56.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.561+0000 7f38c0bae640 1 --2- 172.21.15.70:0/3475273522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 0x7f38bc0b6060 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.561+0000 7f38c0bae640 1 --2- 172.21.15.70:0/3475273522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38bc15ccb0 0x7f38bc0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.561+0000 7f38c0bae640 1 -- 172.21.15.70:0/3475273522 >> 172.21.15.70:0/3475273522 conn(0x7f38bc071660 msgr2=0x7f38bc071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:56.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.561+0000 7f38c0bae640 1 -- 172.21.15.70:0/3475273522 shutdown_connections 2023-10-08T05:37:56.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.561+0000 7f38c0bae640 1 -- 172.21.15.70:0/3475273522 wait complete. 2023-10-08T05:37:56.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.563+0000 7f38c0bae640 1 Processor -- start 2023-10-08T05:37:56.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.563+0000 7f38c0bae640 1 -- start start 2023-10-08T05:37:56.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.564+0000 7f38c0bae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 0x7f38bc06b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.564+0000 7f38c0bae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38bc0d0b90 0x7f38bc0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.564+0000 7f38c0bae640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38bc15ccb0 0x7f38bc0c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.564+0000 7f38c0bae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0cc880 con 0x7f38bc0d00e0 2023-10-08T05:37:56.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.564+0000 7f38c0bae640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0c7e50 con 0x7f38bc0d0b90 2023-10-08T05:37:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.564+0000 7f38c0bae640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0c72d0 con 0x7f38bc15ccb0 2023-10-08T05:37:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.564+0000 7f38bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 0x7f38bc06b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.564+0000 7f38bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 0x7f38bc06b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45360/0 (socket says 172.21.15.70:45360) 2023-10-08T05:37:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.564+0000 7f38bb7fe640 1 -- 172.21.15.70:0/1563804860 learned_addr learned my addr 172.21.15.70:0/1563804860 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38bbfff640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38bc15ccb0 0x7f38bc0c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38bb7fe640 1 -- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38bc15ccb0 msgr2=0x7f38bc0c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38bc15ccb0 0x7f38bc0c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38bb7fe640 1 -- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38bc0d0b90 msgr2=0x7f38bc0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38baffd640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38bc0d0b90 0x7f38bc0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38bc0d0b90 0x7f38bc0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38bb7fe640 1 -- 172.21.15.70:0/1563804860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b4009400 con 0x7f38bc0d00e0 2023-10-08T05:37:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 0x7f38bc06b3b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f38ac007e40 tx=0x7f38ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38b3fff640 1 -- 172.21.15.70:0/1563804860 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac014620 con 0x7f38bc0d00e0 2023-10-08T05:37:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38b3fff640 1 -- 172.21.15.70:0/1563804860 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac00ee90 con 0x7f38bc0d00e0 2023-10-08T05:37:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38b3fff640 1 -- 172.21.15.70:0/1563804860 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0149c0 con 0x7f38bc0d00e0 2023-10-08T05:37:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.565+0000 7f38c0bae640 1 -- 172.21.15.70:0/1563804860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38bc0c75b0 con 0x7f38bc0d00e0 2023-10-08T05:37:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.566+0000 7f38c0bae640 1 -- 172.21.15.70:0/1563804860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc0c7b70 con 0x7f38bc0d00e0 2023-10-08T05:37:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.569+0000 7f38b3fff640 1 -- 172.21.15.70:0/1563804860 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f38ac014b20 con 0x7f38bc0d00e0 2023-10-08T05:37:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.569+0000 7f38b3fff640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f389c06ae70 0x7f389c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.570+0000 7f38b3fff640 1 -- 172.21.15.70:0/1563804860 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f38ac08a0d0 con 0x7f38bc0d00e0 2023-10-08T05:37:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.570+0000 7f38c499e740 1 Processor -- start 2023-10-08T05:37:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.570+0000 7f38c499e740 1 -- start start 2023-10-08T05:37:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.570+0000 7f38baffd640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f389c06ae70 0x7f389c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.571+0000 7f38c499e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ad00 0x55fb96c6dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.571+0000 7f38c499e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96c9da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.571+0000 7f38c499e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c6e140 0x55fb96c9dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.571+0000 7f38c499e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb96be1700 con 0x55fb96c6e140 2023-10-08T05:37:56.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.571+0000 7f38c499e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb96bdedf0 con 0x55fb96c9ad00 2023-10-08T05:37:56.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.571+0000 7f38c499e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb96bda6e0 con 0x55fb96c9b0e0 2023-10-08T05:37:56.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.571+0000 7f38bbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c6e140 0x55fb96c9dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.571+0000 7f38bbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c6e140 0x55fb96c9dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45368/0 (socket says 172.21.15.70:45368) 2023-10-08T05:37:56.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.571+0000 7f38bbfff640 1 -- 172.21.15.70:0/1211904495 learned_addr learned my addr 172.21.15.70:0/1211904495 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:56.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.571+0000 7f38baffd640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f389c06ae70 0x7f389c06d330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f38a4007fd0 tx=0x7f38a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:56.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38baffd640 1 --2- 172.21.15.70:0/1211904495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96c9da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/1211904495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ad00 0x55fb96c6dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38bbfff640 1 -- 172.21.15.70:0/1211904495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 msgr2=0x55fb96c9da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:56.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38bbfff640 1 --2- 172.21.15.70:0/1211904495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96c9da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38bbfff640 1 -- 172.21.15.70:0/1211904495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ad00 msgr2=0x55fb96c6dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:56.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38bbfff640 1 --2- 172.21.15.70:0/1211904495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ad00 0x55fb96c6dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38bbfff640 1 -- 172.21.15.70:0/1211904495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb96b9e910 con 0x55fb96c6e140 2023-10-08T05:37:56.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38bbfff640 1 --2- 172.21.15.70:0/1211904495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c6e140 0x55fb96c9dfd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009890 tx=0x7f38b4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5183a9e4700063c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:56.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38b17fa640 1 -- 172.21.15.70:0/1211904495 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b403f070 con 0x55fb96c6e140 2023-10-08T05:37:56.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38b17fa640 1 -- 172.21.15.70:0/1211904495 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b40316b0 con 0x55fb96c6e140 2023-10-08T05:37:56.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.572+0000 7f38b17fa640 1 -- 172.21.15.70:0/1211904495 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b4031d00 con 0x55fb96c6e140 2023-10-08T05:37:56.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.573+0000 7f38c499e740 1 -- 172.21.15.70:0/1211904495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c6e140 msgr2=0x55fb96c9dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.573+0000 7f38c499e740 1 --2- 172.21.15.70:0/1211904495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c6e140 0x55fb96c9dfd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009890 tx=0x7f38b4030000 comp rx=0 tx=0).stop 2023-10-08T05:37:56.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.573+0000 7f38c499e740 1 -- 172.21.15.70:0/1211904495 shutdown_connections 2023-10-08T05:37:56.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.573+0000 7f38c499e740 1 --2- 172.21.15.70:0/1211904495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c6e140 0x55fb96c9dfd0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.573+0000 7f38c499e740 1 --2- 172.21.15.70:0/1211904495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96c9da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.573+0000 7f38c499e740 1 --2- 172.21.15.70:0/1211904495 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ad00 0x55fb96c6dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.573+0000 7f38c499e740 1 -- 172.21.15.70:0/1211904495 >> 172.21.15.70:0/1211904495 conn(0x55fb96c6ff10 msgr2=0x55fb96cb8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:56.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.573+0000 7f38c499e740 1 -- 172.21.15.70:0/1211904495 shutdown_connections 2023-10-08T05:37:56.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.573+0000 7f38c499e740 1 -- 172.21.15.70:0/1211904495 wait complete. 2023-10-08T05:37:56.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.574+0000 7f38c499e740 1 -- start start 2023-10-08T05:37:56.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.574+0000 7f38c499e740 1 Processor -- start 2023-10-08T05:37:56.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.575+0000 7f38c499e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96c85400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.575+0000 7f38c499e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96c88510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.575+0000 7f38c499e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 0x55fb96cafb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.575+0000 7f38c499e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb96b78970 con 0x55fb96c9ad00 2023-10-08T05:37:56.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.575+0000 7f38c499e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb96b78610 con 0x55fb96c9ef30 2023-10-08T05:37:56.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.575+0000 7f38c499e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb9693ef20 con 0x55fb96c9b0e0 2023-10-08T05:37:56.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.575+0000 7f38c499e740 10 client.0 _ll_register_callbacks cb 0x55fb96bb6610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:37:56.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.575+0000 7f38bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96c85400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.575+0000 7f38bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96c85400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45374/0 (socket says 172.21.15.70:45374) 2023-10-08T05:37:56.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.575+0000 7f38bb7fe640 1 -- 172.21.15.70:0/1938905848 learned_addr learned my addr 172.21.15.70:0/1938905848 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:56.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38baffd640 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96c88510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38bb7fe640 1 -- 172.21.15.70:0/1938905848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 msgr2=0x55fb96c88510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:56.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96c88510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38bb7fe640 1 -- 172.21.15.70:0/1938905848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 msgr2=0x55fb96cafb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:56.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38bbfff640 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 0x55fb96cafb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 0x55fb96cafb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38bb7fe640 1 -- 172.21.15.70:0/1938905848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38bc0d1710 con 0x55fb96c9ad00 2023-10-08T05:37:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96c85400 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f38ac058a00 tx=0x7f38ac059540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89b88a8d3e67458 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f389a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:37:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f389a7fc640 1 -- 172.21.15.70:0/1938905848 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0581f0 con 0x55fb96c9ad00 2023-10-08T05:37:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38c499e740 1 -- 172.21.15.70:0/1938905848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb96bd36c0 con 0x55fb96c9ad00 2023-10-08T05:37:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f389a7fc640 1 -- 172.21.15.70:0/1938905848 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f38ac058350 con 0x55fb96c9ad00 2023-10-08T05:37:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f389a7fc640 1 -- 172.21.15.70:0/1938905848 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38ac088170 con 0x55fb96c9ad00 2023-10-08T05:37:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38c499e740 10 client.5514 fetch_fsmap learned FSMap version 18 2023-10-08T05:37:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.576+0000 7f38c499e740 1 -- 172.21.15.70:0/1938905848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55fb96bd8d60 con 0x55fb96c9ad00 2023-10-08T05:37:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.577+0000 7f389a7fc640 1 -- 172.21.15.70:0/1938905848 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb96bd36c0 con 0x55fb96c9ad00 2023-10-08T05:37:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.577+0000 7f389a7fc640 1 -- 172.21.15.70:0/1938905848 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f38ac060b70 con 0x55fb96c9ad00 2023-10-08T05:37:56.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.577+0000 7f38c499e740 10 client.5514 fetch_fsmap finished waiting for FSMap version 18 2023-10-08T05:37:56.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.577+0000 7f38c499e740 10 client.5514 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T05:37:56.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.577+0000 7f38c499e740 20 client.5514 populate_metadata read hostname 'smithi070' 2023-10-08T05:37:56.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.577+0000 7f38c499e740 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.70:6832/480502229,v1:172.21.15.70:6834/480502229] conn(0x55fb96c874d0 0x55fb96c9cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.577+0000 7f38c499e740 4 client.5514 mds_command: new command op to 5304 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T05:37:56.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.577+0000 7f38c499e740 1 -- 172.21.15.70:0/1938905848 --> [v2:172.21.15.70:6832/480502229,v1:172.21.15.70:6834/480502229] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb96bd33e0 con 0x55fb96c874d0 2023-10-08T05:37:56.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.577+0000 7f38baffd640 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.70:6832/480502229,v1:172.21.15.70:6834/480502229] conn(0x55fb96c874d0 0x55fb96c9cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.578+0000 7f38baffd640 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.70:6832/480502229,v1:172.21.15.70:6834/480502229] conn(0x55fb96c874d0 0x55fb96c9cac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f417adbbacc2f736 server_cookie=89861d6961d3d59b in_seq=0 out_seq=0 2023-10-08T05:37:56.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.578+0000 7f389a7fc640 10 client.5514 ms_handle_connect on v2:172.21.15.70:6832/480502229 2023-10-08T05:37:56.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.578+0000 7f389a7fc640 1 -- 172.21.15.70:0/1938905848 <== mds.0 v2:172.21.15.70:6832/480502229 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fb96bd33e0 con 0x55fb96c874d0 2023-10-08T05:37:56.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.578+0000 7f389a7fc640 10 client.5514 handle_command_reply: tid=0 2023-10-08T05:37:56.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.578+0000 7f38c499e740 1 client.5514 shutdown 2023-10-08T05:37:56.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 1 -- 172.21.15.70:0/1938905848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 msgr2=0x55fb96c85400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96c85400 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f38ac058a00 tx=0x7f38ac059540 comp rx=0 tx=0).stop 2023-10-08T05:37:56.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 1 -- 172.21.15.70:0/1938905848 shutdown_connections 2023-10-08T05:37:56.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.70:6832/480502229,v1:172.21.15.70:6834/480502229] conn(0x55fb96c874d0 0x55fb96c9cac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 0x55fb96cafb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96c88510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 1 --2- 172.21.15.70:0/1938905848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96c85400 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f389a7fc640 0 client.5514 ms_handle_reset on v2:172.21.15.70:6832/480502229 2023-10-08T05:37:56.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 1 -- 172.21.15.70:0/1938905848 >> 172.21.15.70:0/1938905848 conn(0x55fb96c6e140 msgr2=0x55fb96cbbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:56.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 1 -- 172.21.15.70:0/1938905848 shutdown_connections 2023-10-08T05:37:56.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 1 -- 172.21.15.70:0/1938905848 wait complete. 2023-10-08T05:37:56.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.579+0000 7f38c499e740 20 client.5514 trim_cache size 0 max 16384 2023-10-08T05:37:56.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.595+0000 7f38c499e740 1 Processor -- start 2023-10-08T05:37:56.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.595+0000 7f38c499e740 1 -- start start 2023-10-08T05:37:56.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.596+0000 7f38c499e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96cbc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.596+0000 7f38c499e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96cba280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.596+0000 7f38c499e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 0x55fb96c82dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.596+0000 7f38c499e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb96c9edf0 con 0x55fb96c9ad00 2023-10-08T05:37:56.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.596+0000 7f38c499e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb96cc7d80 con 0x55fb96c9ef30 2023-10-08T05:37:56.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.596+0000 7f38c499e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb96c9a550 con 0x55fb96c9b0e0 2023-10-08T05:37:56.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96cbc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96cbc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45380/0 (socket says 172.21.15.70:45380) 2023-10-08T05:37:56.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38bb7fe640 1 -- 172.21.15.70:0/356204111 learned_addr learned my addr 172.21.15.70:0/356204111 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:56.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38bb7fe640 1 -- 172.21.15.70:0/356204111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 msgr2=0x55fb96cba280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:56.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/356204111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96cba280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38bb7fe640 1 -- 172.21.15.70:0/356204111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 msgr2=0x55fb96c82dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/356204111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 0x55fb96c82dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38bb7fe640 1 -- 172.21.15.70:0/356204111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb96b77b80 con 0x55fb96c9ad00 2023-10-08T05:37:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/356204111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96cbc380 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f38ac000f80 tx=0x7f38ac020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2dbb91f834155d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38b1ffb640 1 -- 172.21.15.70:0/356204111 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac058ad0 con 0x55fb96c9ad00 2023-10-08T05:37:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38b1ffb640 1 -- 172.21.15.70:0/356204111 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac060b70 con 0x55fb96c9ad00 2023-10-08T05:37:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.597+0000 7f38b1ffb640 1 -- 172.21.15.70:0/356204111 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac060ce0 con 0x55fb96c9ad00 2023-10-08T05:37:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.598+0000 7f38c499e740 1 -- 172.21.15.70:0/356204111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 msgr2=0x55fb96cbc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.598+0000 7f38c499e740 1 --2- 172.21.15.70:0/356204111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96cbc380 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f38ac000f80 tx=0x7f38ac020000 comp rx=0 tx=0).stop 2023-10-08T05:37:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.598+0000 7f38c499e740 1 -- 172.21.15.70:0/356204111 shutdown_connections 2023-10-08T05:37:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.598+0000 7f38c499e740 1 --2- 172.21.15.70:0/356204111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 0x55fb96c82dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.598+0000 7f38c499e740 1 --2- 172.21.15.70:0/356204111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9b0e0 0x55fb96cba280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.598+0000 7f38c499e740 1 --2- 172.21.15.70:0/356204111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9ad00 0x55fb96cbc380 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.598+0000 7f38c499e740 1 -- 172.21.15.70:0/356204111 >> 172.21.15.70:0/356204111 conn(0x55fb96c6e140 msgr2=0x55fb96c6cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.598+0000 7f38c499e740 1 -- 172.21.15.70:0/356204111 shutdown_connections 2023-10-08T05:37:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.598+0000 7f38c499e740 1 -- 172.21.15.70:0/356204111 wait complete. 2023-10-08T05:37:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.599+0000 7f38c499e740 1 -- start start 2023-10-08T05:37:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.599+0000 7f38c499e740 1 Processor -- start 2023-10-08T05:37:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.600+0000 7f38c499e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9ad00 0x55fb96cba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.600+0000 7f38c499e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9b0e0 0x55fb96d215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.600+0000 7f38c499e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 0x55fb96caf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.600+0000 7f38c499e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb96c84700 con 0x55fb96c9b0e0 2023-10-08T05:37:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.600+0000 7f38c499e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb96ccaa80 con 0x55fb96c9ef30 2023-10-08T05:37:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.600+0000 7f38c499e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb96cdb900 con 0x55fb96c9ad00 2023-10-08T05:37:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.600+0000 7f38c499e740 10 client.0 _ll_register_callbacks cb 0x7f38bc08b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:37:56.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.600+0000 7f38baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9b0e0 0x55fb96d215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.600+0000 7f38baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9b0e0 0x55fb96d215d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45382/0 (socket says 172.21.15.70:45382) 2023-10-08T05:37:56.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.600+0000 7f38baffd640 1 -- 172.21.15.70:0/3180156726 learned_addr learned my addr 172.21.15.70:0/3180156726 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:56.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f38baffd640 1 -- 172.21.15.70:0/3180156726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9ad00 msgr2=0x55fb96cba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:56.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f38baffd640 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9ad00 0x55fb96cba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f38baffd640 1 -- 172.21.15.70:0/3180156726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 msgr2=0x55fb96caf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:56.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f38baffd640 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 0x55fb96caf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f38baffd640 1 -- 172.21.15.70:0/3180156726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb96bd8d60 con 0x55fb96c9b0e0 2023-10-08T05:37:56.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f38baffd640 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9b0e0 0x55fb96d215d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f38bc15fe80 tx=0x7f38a4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06e5916177f583c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:56.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f389a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:37:56.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f389a7fc640 1 -- 172.21.15.70:0/3180156726 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4005c40 con 0x55fb96c9b0e0 2023-10-08T05:37:56.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f389a7fc640 1 -- 172.21.15.70:0/3180156726 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f38a4005da0 con 0x55fb96c9b0e0 2023-10-08T05:37:56.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f38c499e740 1 -- 172.21.15.70:0/3180156726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38b40316b0 con 0x55fb96c9b0e0 2023-10-08T05:37:56.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f389a7fc640 1 -- 172.21.15.70:0/3180156726 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38a40239b0 con 0x55fb96c9b0e0 2023-10-08T05:37:56.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f38c499e740 10 client.5520 fetch_fsmap learned FSMap version 18 2023-10-08T05:37:56.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.601+0000 7f38c499e740 1 -- 172.21.15.70:0/3180156726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55fb96b9d170 con 0x55fb96c9b0e0 2023-10-08T05:37:56.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f389a7fc640 1 -- 172.21.15.70:0/3180156726 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b40316b0 con 0x55fb96c9b0e0 2023-10-08T05:37:56.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f389a7fc640 1 -- 172.21.15.70:0/3180156726 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f38a4023b10 con 0x55fb96c9b0e0 2023-10-08T05:37:56.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f38c499e740 10 client.5520 fetch_fsmap finished waiting for FSMap version 18 2023-10-08T05:37:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f38c499e740 10 client.5520 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T05:37:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f38c499e740 20 client.5520 populate_metadata read hostname 'smithi070' 2023-10-08T05:37:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f38c499e740 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.70:6832/480502229,v1:172.21.15.70:6834/480502229] conn(0x55fb96c94e00 0x55fb96c951e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f38c499e740 4 client.5520 mds_command: new command op to 5304 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T05:37:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f38c499e740 1 -- 172.21.15.70:0/3180156726 --> [v2:172.21.15.70:6832/480502229,v1:172.21.15.70:6834/480502229] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb968dc6c0 con 0x55fb96c94e00 2023-10-08T05:37:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.70:6832/480502229,v1:172.21.15.70:6834/480502229] conn(0x55fb96c94e00 0x55fb96c951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f38bb7fe640 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.70:6832/480502229,v1:172.21.15.70:6834/480502229] conn(0x55fb96c94e00 0x55fb96c951e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d012c5fa40bfcfd5 server_cookie=7f491ae733e8083a in_seq=0 out_seq=0 2023-10-08T05:37:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.602+0000 7f389a7fc640 10 client.5520 ms_handle_connect on v2:172.21.15.70:6832/480502229 2023-10-08T05:37:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f389a7fc640 1 -- 172.21.15.70:0/3180156726 <== mds.0 v2:172.21.15.70:6832/480502229 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55fb968dc6c0 con 0x55fb96c94e00 2023-10-08T05:37:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f389a7fc640 10 client.5520 handle_command_reply: tid=0 2023-10-08T05:37:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 client.5520 shutdown 2023-10-08T05:37:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 -- 172.21.15.70:0/3180156726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9b0e0 msgr2=0x55fb96d215d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9b0e0 0x55fb96d215d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f38bc15fe80 tx=0x7f38a4000ed0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 -- 172.21.15.70:0/3180156726 shutdown_connections 2023-10-08T05:37:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.70:6832/480502229,v1:172.21.15.70:6834/480502229] conn(0x55fb96c94e00 0x55fb96c951e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55fb96c9ef30 0x55fb96caf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55fb96c9b0e0 0x55fb96d215d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 --2- 172.21.15.70:0/3180156726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55fb96c9ad00 0x55fb96cba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f389a7fc640 0 client.5520 ms_handle_reset on v2:172.21.15.70:6832/480502229 2023-10-08T05:37:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 -- 172.21.15.70:0/3180156726 >> 172.21.15.70:0/3180156726 conn(0x55fb96c6e140 msgr2=0x55fb96cbc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 -- 172.21.15.70:0/3180156726 shutdown_connections 2023-10-08T05:37:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 1 -- 172.21.15.70:0/3180156726 wait complete. 2023-10-08T05:37:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.603+0000 7f38c499e740 20 client.5520 trim_cache size 0 max 16384 2023-10-08T05:37:56.610 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:37:56.610 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T05:37:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T05:37:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 5304, 2023-10-08T05:37:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T05:37:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T05:37:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T05:37:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 11.866005417, 2023-10-08T05:37:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 18, 2023-10-08T05:37:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 39, 2023-10-08T05:37:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 39, 2023-10-08T05:37:56.612 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 20.745389881000001 2023-10-08T05:37:56.612 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:37:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 -- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f389c06ae70 msgr2=0x7f389c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f389c06ae70 0x7f389c06d330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f38a4007fd0 tx=0x7f38a4009040 comp rx=0 tx=0).stop 2023-10-08T05:37:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 -- 172.21.15.70:0/1563804860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 msgr2=0x7f38bc06b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 0x7f38bc06b3b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f38ac007e40 tx=0x7f38ac00e4b0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 -- 172.21.15.70:0/1563804860 shutdown_connections 2023-10-08T05:37:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f389c06ae70 0x7f389c06d330 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38bc15ccb0 0x7f38bc0c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38bc0d0b90 0x7f38bc0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 --2- 172.21.15.70:0/1563804860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38bc0d00e0 0x7f38bc06b3b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 -- 172.21.15.70:0/1563804860 >> 172.21.15.70:0/1563804860 conn(0x7f38bc071660 msgr2=0x7f38bc0b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:56.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 -- 172.21.15.70:0/1563804860 shutdown_connections 2023-10-08T05:37:56.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.604+0000 7f38c0bae640 1 -- 172.21.15.70:0/1563804860 wait complete. 2023-10-08T05:37:56.621 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:37:56.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.823+0000 7f1e019a5640 1 Processor -- start 2023-10-08T05:37:56.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.824+0000 7f1e019a5640 1 -- start start 2023-10-08T05:37:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.825+0000 7f1e019a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1dfc0cfbe0 0x7f1dfc0d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.825+0000 7f1e019a5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1dfc0b5d40 0x7f1dfc0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.825+0000 7f1e019a5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc0d1750 con 0x7f1dfc1600d0 2023-10-08T05:37:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.825+0000 7f1e019a5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dfc0d05b0 con 0x7f1dfc0b5d40 2023-10-08T05:37:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.825+0000 7f1e019a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0d1a30 con 0x7f1dfc0cfbe0 2023-10-08T05:37:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.826+0000 7f1dfb7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1dfc0b5d40 0x7f1dfc0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.826+0000 7f1dfa7fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1dfc0cfbe0 0x7f1dfc0d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.826+0000 7f1dfb7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1dfc0b5d40 0x7f1dfc0b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:58544/0 (socket says 172.21.15.70:58544) 2023-10-08T05:37:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.826+0000 7f1dfb7fe640 1 -- 172.21.15.70:0/2058279420 learned_addr learned my addr 172.21.15.70:0/2058279420 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.826+0000 7f1dfb7fe640 1 -- 172.21.15.70:0/2058279420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1dfc0cfbe0 msgr2=0x7f1dfc0d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.826+0000 7f1dfb7fe640 1 --2- 172.21.15.70:0/2058279420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1dfc0cfbe0 0x7f1dfc0d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.826+0000 7f1dfb7fe640 1 -- 172.21.15.70:0/2058279420 >> v1:172.21.15.181:6789/0 conn(0x7f1dfc1600d0 legacy=0x7f1dfc0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.826+0000 7f1dfb7fe640 1 -- 172.21.15.70:0/2058279420 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc0d1d10 con 0x7f1dfc0b5d40 2023-10-08T05:37:56.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.828+0000 7f1dfb7fe640 1 --2- 172.21.15.70:0/2058279420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dfc0b5d40 0x7f1dfc0b61d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009870 tx=0x7f1df402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a783f0aef3f4c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:56.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.828+0000 7f1df9ffb640 1 -- 172.21.15.70:0/2058279420 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4030840 con 0x7f1dfc0b5d40 2023-10-08T05:37:56.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.828+0000 7f1df9ffb640 1 -- 172.21.15.70:0/2058279420 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df40309a0 con 0x7f1dfc0b5d40 2023-10-08T05:37:56.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.828+0000 7f1df9ffb640 1 -- 172.21.15.70:0/2058279420 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4030d30 con 0x7f1dfc0b5d40 2023-10-08T05:37:56.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.829+0000 7f1e019a5640 1 -- 172.21.15.70:0/2058279420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dfc0b5d40 msgr2=0x7f1dfc0b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.829+0000 7f1e019a5640 1 --2- 172.21.15.70:0/2058279420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dfc0b5d40 0x7f1dfc0b61d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009870 tx=0x7f1df402b980 comp rx=0 tx=0).stop 2023-10-08T05:37:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.829+0000 7f1e019a5640 1 -- 172.21.15.70:0/2058279420 shutdown_connections 2023-10-08T05:37:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.829+0000 7f1e019a5640 1 --2- 172.21.15.70:0/2058279420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dfc0b5d40 0x7f1dfc0b61d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.829+0000 7f1e019a5640 1 --2- 172.21.15.70:0/2058279420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1dfc0cfbe0 0x7f1dfc0d0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.829+0000 7f1e019a5640 1 -- 172.21.15.70:0/2058279420 >> 172.21.15.70:0/2058279420 conn(0x7f1dfc0718a0 msgr2=0x7f1dfc071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.829+0000 7f1e019a5640 1 -- 172.21.15.70:0/2058279420 shutdown_connections 2023-10-08T05:37:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.829+0000 7f1e019a5640 1 -- 172.21.15.70:0/2058279420 wait complete. 2023-10-08T05:37:56.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.831+0000 7f1e019a5640 1 Processor -- start 2023-10-08T05:37:56.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.831+0000 7f1e019a5640 1 -- start start 2023-10-08T05:37:56.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.832+0000 7f1e019a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1dfc0b5d40 0x7f1dfc1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.832+0000 7f1e019a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dfc0cfbe0 0x7f1dfc0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.832+0000 7f1e019a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc1600d0 0x7f1dfc1f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.832+0000 7f1e019a5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1f6990 con 0x7f1dfc1600d0 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.832+0000 7f1e019a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0cbe20 con 0x7f1dfc0cfbe0 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.832+0000 7f1e019a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc1f2420 con 0x7f1dfc0b5d40 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.832+0000 7f1dfb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc1600d0 0x7f1dfc1f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.832+0000 7f1dfb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc1600d0 0x7f1dfc1f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45388/0 (socket says 172.21.15.70:45388) 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.832+0000 7f1dfb7fe640 1 -- 172.21.15.70:0/10410095 learned_addr learned my addr 172.21.15.70:0/10410095 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1dfaffd640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1dfc0b5d40 0x7f1dfc1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1dfa7fc640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dfc0cfbe0 0x7f1dfc0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1dfb7fe640 1 -- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1dfc0b5d40 msgr2=0x7f1dfc1f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1dfb7fe640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1dfc0b5d40 0x7f1dfc1f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1dfb7fe640 1 -- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dfc0cfbe0 msgr2=0x7f1dfc0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:56.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1dfb7fe640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dfc0cfbe0 0x7f1dfc0cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:56.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1dfb7fe640 1 -- 172.21.15.70:0/10410095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df40094d0 con 0x7f1dfc1600d0 2023-10-08T05:37:56.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1dfb7fe640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc1600d0 0x7f1dfc1f2ba0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009870 tx=0x7f1df4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1df3fff640 1 -- 172.21.15.70:0/10410095 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4039c90 con 0x7f1dfc1600d0 2023-10-08T05:37:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1df3fff640 1 -- 172.21.15.70:0/10410095 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4039e10 con 0x7f1dfc1600d0 2023-10-08T05:37:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.833+0000 7f1df3fff640 1 -- 172.21.15.70:0/10410095 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4043640 con 0x7f1dfc1600d0 2023-10-08T05:37:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.834+0000 7f1e019a5640 1 -- 172.21.15.70:0/10410095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc1f60f0 con 0x7f1dfc1600d0 2023-10-08T05:37:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.834+0000 7f1e019a5640 1 -- 172.21.15.70:0/10410095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc1f66b0 con 0x7f1dfc1600d0 2023-10-08T05:37:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.835+0000 7f1e019a5640 1 -- 172.21.15.70:0/10410095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc067490 con 0x7f1dfc1600d0 2023-10-08T05:37:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.836+0000 7f1df3fff640 1 -- 172.21.15.70:0/10410095 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f1df40437a0 con 0x7f1dfc1600d0 2023-10-08T05:37:56.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.836+0000 7f1df3fff640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ddc06ae70 0x7f1ddc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:56.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.837+0000 7f1df3fff640 1 -- 172.21.15.70:0/10410095 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1df403e070 con 0x7f1dfc1600d0 2023-10-08T05:37:56.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.837+0000 7f1dfaffd640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ddc06ae70 0x7f1ddc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:56.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.838+0000 7f1dfaffd640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ddc06ae70 0x7f1ddc06d330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1dec001ca0 tx=0x7f1dec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:56.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:56.841+0000 7f1df3fff640 1 -- 172.21.15.70:0/10410095 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1df4077730 con 0x7f1dfc1600d0 2023-10-08T05:37:57.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.071+0000 7f1e019a5640 1 -- 172.21.15.70:0/10410095 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1dfc071e10 con 0x7f1dfc1600d0 2023-10-08T05:37:57.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.072+0000 7f1df3fff640 1 -- 172.21.15.70:0/10410095 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1df407ad50 con 0x7f1dfc1600d0 2023-10-08T05:37:57.072 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T05:37:57.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.074+0000 7f1e019a5640 1 -- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ddc06ae70 msgr2=0x7f1ddc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:57.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.074+0000 7f1e019a5640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ddc06ae70 0x7f1ddc06d330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1dec001ca0 tx=0x7f1dec009040 comp rx=0 tx=0).stop 2023-10-08T05:37:57.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.074+0000 7f1e019a5640 1 -- 172.21.15.70:0/10410095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc1600d0 msgr2=0x7f1dfc1f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:57.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.074+0000 7f1e019a5640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc1600d0 0x7f1dfc1f2ba0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009870 tx=0x7f1df4004730 comp rx=0 tx=0).stop 2023-10-08T05:37:57.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.075+0000 7f1e019a5640 1 -- 172.21.15.70:0/10410095 shutdown_connections 2023-10-08T05:37:57.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.075+0000 7f1e019a5640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f1ddc06ae70 0x7f1ddc06d330 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:57.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.075+0000 7f1e019a5640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1dfc1600d0 0x7f1dfc1f2ba0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:57.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.075+0000 7f1e019a5640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1dfc0cfbe0 0x7f1dfc0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:57.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.075+0000 7f1e019a5640 1 --2- 172.21.15.70:0/10410095 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1dfc0b5d40 0x7f1dfc1f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:57.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.075+0000 7f1e019a5640 1 -- 172.21.15.70:0/10410095 >> 172.21.15.70:0/10410095 conn(0x7f1dfc0718a0 msgr2=0x7f1dfc0b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:57.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.075+0000 7f1e019a5640 1 -- 172.21.15.70:0/10410095 shutdown_connections 2023-10-08T05:37:57.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:57.075+0000 7f1e019a5640 1 -- 172.21.15.70:0/10410095 wait complete. 2023-10-08T05:37:57.092 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T05:37:57.093 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:37:57.093 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:37:57.093 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T05:37:57.093 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T05:37:57.093 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T05:37:57.093 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T05:37:57.094 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:37:57.094 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:37:57.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:57.094 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:37:57.146 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:37:57.146 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:37:57.146 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:37:57.146 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:57.146 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:37:57.146 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:57.146 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:37:57.147 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:37:57.147 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:37:57.147 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:37:57.147 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:37:57.147 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:37:57.147 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42319sec preferred_lft 42319sec 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:57.148 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T05:37:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:57.149 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:37:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T05:37:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T05:37:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:57.149 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:37:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T05:37:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T05:37:57.149 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:37:57.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:57.150 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:37:57.231 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:37:57.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:57.235 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:37:57.288 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:37:57.289 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:37:57.289 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:37:57.289 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:37:57.289 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:37:57.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:57.290 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:37:57.290 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:37:57.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:37:57.291 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:37:57.291 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:37:57.291 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:37:57.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:57.465 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:37:57.513 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T05:37:57.513 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T05:37:57.513 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:37:57.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:57.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T05:37:57.658 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:37:57.658 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T05:37:57.716 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:37:57.716 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:37:57.716 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:37:57.767 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T05:37:57.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:57.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:37:57.908 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:37:57.909 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:37:57.942 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:37:57.942 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:37:57.942 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:37:57.942 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:37:57.942 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:37:57.944 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T05:37:57.944 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:37:57.944 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:37:57.945 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:37:57.945 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:37:57.948 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:37:57.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:57.948 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:37:58.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:37:58.027 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:37:58.086 INFO:teuthology.orchestra.run.smithi070.stdout:client5526 2023-10-08T05:37:58.087 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:37:58.087 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:37:58.139 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T05:37:58.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.400+0000 7f5532610640 1 Processor -- start 2023-10-08T05:37:58.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.401+0000 7f5532610640 1 -- start start 2023-10-08T05:37:58.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.402+0000 7f5532610640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c160660 0x7f552c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.402+0000 7f5532610640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f552c0c9910 0x7f552c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.402+0000 7f5532610640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552c06e150 con 0x7f552c0cb450 2023-10-08T05:37:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.402+0000 7f5532610640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552c0cb8c0 con 0x7f552c0c9910 2023-10-08T05:37:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.402+0000 7f5532610640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c06de70 con 0x7f552c160660 2023-10-08T05:37:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.402+0000 7f552b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f552c0c9910 0x7f552c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.403+0000 7f5530bad640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f552c0cb450 0x7f552c0b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59934/0 (socket says 172.21.15.70:59934) 2023-10-08T05:37:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.403+0000 7f5530bad640 1 -- 172.21.15.70:0/3976848101 learned_addr learned my addr 172.21.15.70:0/3976848101 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:37:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.403+0000 7f552bfff640 1 --2- 172.21.15.70:0/3976848101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c160660 0x7f552c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.403+0000 7f552b7fe640 1 -- 172.21.15.70:0/3976848101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c160660 msgr2=0x7f552c070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.403+0000 7f552b7fe640 1 --2- 172.21.15.70:0/3976848101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c160660 0x7f552c070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.403+0000 7f552b7fe640 1 -- 172.21.15.70:0/3976848101 >> v1:172.21.15.70:6789/0 conn(0x7f552c0cb450 legacy=0x7f552c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.403+0000 7f552b7fe640 1 -- 172.21.15.70:0/3976848101 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552c0c8e30 con 0x7f552c0c9910 2023-10-08T05:37:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.403+0000 7f552affd640 1 -- 172.21.15.70:0/3976848101 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650064442 0 0) 0x7f552c06e150 con 0x7f552c0cb450 2023-10-08T05:37:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.404+0000 7f552b7fe640 1 --2- 172.21.15.70:0/3976848101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c0c9910 0x7f552c0caf10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f55140097a0 tx=0x7f551402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6754f13456e6a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f552affd640 1 -- 172.21.15.70:0/3976848101 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514031830 con 0x7f552c0c9910 2023-10-08T05:37:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f552affd640 1 -- 172.21.15.70:0/3976848101 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5514031990 con 0x7f552c0c9910 2023-10-08T05:37:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f552affd640 1 -- 172.21.15.70:0/3976848101 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514031d20 con 0x7f552c0c9910 2023-10-08T05:37:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f5532610640 1 -- 172.21.15.70:0/3976848101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c0c9910 msgr2=0x7f552c0caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f5532610640 1 --2- 172.21.15.70:0/3976848101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c0c9910 0x7f552c0caf10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f55140097a0 tx=0x7f551402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f5532610640 1 -- 172.21.15.70:0/3976848101 shutdown_connections 2023-10-08T05:37:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f5532610640 1 --2- 172.21.15.70:0/3976848101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c0c9910 0x7f552c0caf10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f5532610640 1 --2- 172.21.15.70:0/3976848101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c160660 0x7f552c070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f5532610640 1 -- 172.21.15.70:0/3976848101 >> 172.21.15.70:0/3976848101 conn(0x7f552c06a5d0 msgr2=0x7f552c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f5532610640 1 -- 172.21.15.70:0/3976848101 shutdown_connections 2023-10-08T05:37:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.405+0000 7f5532610640 1 -- 172.21.15.70:0/3976848101 wait complete. 2023-10-08T05:37:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.407+0000 7f5532610640 1 Processor -- start 2023-10-08T05:37:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.407+0000 7f5532610640 1 -- start start 2023-10-08T05:37:58.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.408+0000 7f5532610640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c0c9910 0x7f552c0c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.408+0000 7f5532610640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f552c0cb450 0x7f552c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.408+0000 7f5532610640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c160660 0x7f552c0ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.408+0000 7f5532610640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c0c3ce0 con 0x7f552c160660 2023-10-08T05:37:58.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.408+0000 7f5532610640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c0c4ff0 con 0x7f552c0cb450 2023-10-08T05:37:58.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.408+0000 7f5532610640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c0c4850 con 0x7f552c0c9910 2023-10-08T05:37:58.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f5530bad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c160660 0x7f552c0ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f5530bad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c160660 0x7f552c0ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45412/0 (socket says 172.21.15.70:45412) 2023-10-08T05:37:58.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f5530bad640 1 -- 172.21.15.70:0/307474794 learned_addr learned my addr 172.21.15.70:0/307474794 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:58.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f5530bad640 1 -- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c0c9910 msgr2=0x7f552c0c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:58.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f5530bad640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c0c9910 0x7f552c0c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f5530bad640 1 -- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f552c0cb450 msgr2=0x7f552c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:58.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f552b7fe640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f552c0cb450 0x7f552c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f5530bad640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f552c0cb450 0x7f552c0cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f5530bad640 1 -- 172.21.15.70:0/307474794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5514009400 con 0x7f552c160660 2023-10-08T05:37:58.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f5530bad640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c160660 0x7f552c0ccb80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f552400c450 tx=0x7f552400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:58.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.409+0000 7f5528ff9640 1 -- 172.21.15.70:0/307474794 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55240074e0 con 0x7f552c160660 2023-10-08T05:37:58.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.410+0000 7f5528ff9640 1 -- 172.21.15.70:0/307474794 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5524007660 con 0x7f552c160660 2023-10-08T05:37:58.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.410+0000 7f5528ff9640 1 -- 172.21.15.70:0/307474794 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524004a20 con 0x7f552c160660 2023-10-08T05:37:58.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.410+0000 7f5532610640 1 -- 172.21.15.70:0/307474794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552c1522d0 con 0x7f552c160660 2023-10-08T05:37:58.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.410+0000 7f5532610640 1 -- 172.21.15.70:0/307474794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c0c3720 con 0x7f552c160660 2023-10-08T05:37:58.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.411+0000 7f5532610640 1 -- 172.21.15.70:0/307474794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552c04e160 con 0x7f552c160660 2023-10-08T05:37:58.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.412+0000 7f5528ff9640 1 -- 172.21.15.70:0/307474794 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f5524004b80 con 0x7f552c160660 2023-10-08T05:37:58.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.413+0000 7f5528ff9640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f550c06ae70 0x7f550c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.413+0000 7f5528ff9640 1 -- 172.21.15.70:0/307474794 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f55240887b0 con 0x7f552c160660 2023-10-08T05:37:58.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.413+0000 7f552bfff640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f550c06ae70 0x7f550c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.414+0000 7f552bfff640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f550c06ae70 0x7f550c06d330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f551c0070b0 tx=0x7f551c007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:58.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.417+0000 7f5528ff9640 1 -- 172.21.15.70:0/307474794 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55240535f0 con 0x7f552c160660 2023-10-08T05:37:58.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.630+0000 7f5532610640 1 -- 172.21.15.70:0/307474794 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f552c0c93f0 con 0x7f550c06ae70 2023-10-08T05:37:58.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.635+0000 7f5528ff9640 1 -- 172.21.15.70:0/307474794 <== mgr.4105 v2:172.21.15.181:6832/2737793492 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f551c009b10 con 0x7f550c06ae70 2023-10-08T05:37:58.635 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:37:58.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:37:58.635 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T05:37:58.636 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:37:58.636 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:37:58.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.637+0000 7f5532610640 1 -- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f550c06ae70 msgr2=0x7f550c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:58.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.637+0000 7f5532610640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f550c06ae70 0x7f550c06d330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f551c0070b0 tx=0x7f551c007040 comp rx=0 tx=0).stop 2023-10-08T05:37:58.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.637+0000 7f5532610640 1 -- 172.21.15.70:0/307474794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c160660 msgr2=0x7f552c0ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:58.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.637+0000 7f5532610640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c160660 0x7f552c0ccb80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f552400c450 tx=0x7f552400c910 comp rx=0 tx=0).stop 2023-10-08T05:37:58.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.637+0000 7f5532610640 1 -- 172.21.15.70:0/307474794 shutdown_connections 2023-10-08T05:37:58.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.637+0000 7f5532610640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f550c06ae70 0x7f550c06d330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.637+0000 7f5532610640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552c160660 0x7f552c0ccb80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.637+0000 7f5532610640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f552c0cb450 0x7f552c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.637+0000 7f5532610640 1 --2- 172.21.15.70:0/307474794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f552c0c9910 0x7f552c0c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.637+0000 7f5532610640 1 -- 172.21.15.70:0/307474794 >> 172.21.15.70:0/307474794 conn(0x7f552c06a5d0 msgr2=0x7f552c15dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:58.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.638+0000 7f5532610640 1 -- 172.21.15.70:0/307474794 shutdown_connections 2023-10-08T05:37:58.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.638+0000 7f5532610640 1 -- 172.21.15.70:0/307474794 wait complete. 2023-10-08T05:37:58.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T05:37:58.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.875+0000 7f390ee39640 1 Processor -- start 2023-10-08T05:37:58.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.876+0000 7f390ee39640 1 -- start start 2023-10-08T05:37:58.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.877+0000 7f390ee39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390815aad0 0x7f39080b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.877+0000 7f390ee39640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f390806d410 0x7f39080c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.877+0000 7f390ee39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390815bf30 con 0x7f39080c82b0 2023-10-08T05:37:58.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.877+0000 7f390ee39640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390815b320 con 0x7f390806d410 2023-10-08T05:37:58.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.877+0000 7f390ee39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390815b600 con 0x7f390815aad0 2023-10-08T05:37:58.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.878+0000 7f390d3d6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f390806d410 0x7f39080c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.878+0000 7f390cbd5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390815aad0 0x7f39080b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.878+0000 7f390d3d6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f390806d410 0x7f39080c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:58574/0 (socket says 172.21.15.70:58574) 2023-10-08T05:37:58.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.878+0000 7f390cbd5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390815aad0 0x7f39080b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53820/0 (socket says 172.21.15.70:53820) 2023-10-08T05:37:58.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.878+0000 7f390d3d6640 1 -- 172.21.15.70:0/1428955283 learned_addr learned my addr 172.21.15.70:0/1428955283 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:58.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.878+0000 7f38fffff640 1 --1- 172.21.15.70:0/1428955283 >> v1:172.21.15.181:6789/0 conn(0x7f39080c82b0 0x7f390806cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:43290/0 (socket says 172.21.15.70:43290) 2023-10-08T05:37:58.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.879+0000 7f390d3d6640 1 -- 172.21.15.70:0/1428955283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390815aad0 msgr2=0x7f39080b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:37:58.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.879+0000 7f390d3d6640 1 --2- 172.21.15.70:0/1428955283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390815aad0 0x7f39080b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.879+0000 7f390d3d6640 1 -- 172.21.15.70:0/1428955283 >> v1:172.21.15.181:6789/0 conn(0x7f39080c82b0 legacy=0x7f390806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:58.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.879+0000 7f390d3d6640 1 -- 172.21.15.70:0/1428955283 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390815b8e0 con 0x7f390806d410 2023-10-08T05:37:58.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.879+0000 7f390cbd5640 1 --2- 172.21.15.70:0/1428955283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390815aad0 0x7f39080b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:37:58.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.880+0000 7f390d3d6640 1 --2- 172.21.15.70:0/1428955283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390806d410 0x7f39080c8710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3900009870 tx=0x7f390002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d4d29d414c3592 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:58.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.881+0000 7f38ff7fe640 1 -- 172.21.15.70:0/1428955283 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900030840 con 0x7f390806d410 2023-10-08T05:37:58.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.881+0000 7f38ff7fe640 1 -- 172.21.15.70:0/1428955283 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39000309a0 con 0x7f390806d410 2023-10-08T05:37:58.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.881+0000 7f38ff7fe640 1 -- 172.21.15.70:0/1428955283 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900030d30 con 0x7f390806d410 2023-10-08T05:37:58.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.881+0000 7f390ee39640 1 -- 172.21.15.70:0/1428955283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390806d410 msgr2=0x7f39080c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:58.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.881+0000 7f390ee39640 1 --2- 172.21.15.70:0/1428955283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390806d410 0x7f39080c8710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3900009870 tx=0x7f390002b980 comp rx=0 tx=0).stop 2023-10-08T05:37:58.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.881+0000 7f390ee39640 1 -- 172.21.15.70:0/1428955283 shutdown_connections 2023-10-08T05:37:58.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.881+0000 7f390ee39640 1 --2- 172.21.15.70:0/1428955283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390806d410 0x7f39080c8710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.881+0000 7f390ee39640 1 --2- 172.21.15.70:0/1428955283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f390815aad0 0x7f39080b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.881+0000 7f390ee39640 1 -- 172.21.15.70:0/1428955283 >> 172.21.15.70:0/1428955283 conn(0x7f3908069d50 msgr2=0x7f390806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:37:58.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.882+0000 7f390ee39640 1 -- 172.21.15.70:0/1428955283 shutdown_connections 2023-10-08T05:37:58.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.882+0000 7f390ee39640 1 -- 172.21.15.70:0/1428955283 wait complete. 2023-10-08T05:37:58.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.884+0000 7f390ee39640 1 Processor -- start 2023-10-08T05:37:58.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.884+0000 7f390ee39640 1 -- start start 2023-10-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.885+0000 7f390ee39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390806d410 0x7f39081f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.885+0000 7f390ee39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39080c82b0 0x7f390814a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.885+0000 7f390ee39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390815aad0 0x7f390815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.885+0000 7f390ee39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39081f6810 con 0x7f390815aad0 2023-10-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.885+0000 7f390ee39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39081f6250 con 0x7f390806d410 2023-10-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.885+0000 7f390ee39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39081f6530 con 0x7f39080c82b0 2023-10-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f390d3d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390815aad0 0x7f390815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f390d3d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390815aad0 0x7f390815d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45416/0 (socket says 172.21.15.70:45416) 2023-10-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f390d3d6640 1 -- 172.21.15.70:0/3226163057 learned_addr learned my addr 172.21.15.70:0/3226163057 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:37:58.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f390cbd5640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390806d410 0x7f39081f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f38fffff640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39080c82b0 0x7f390814a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f390d3d6640 1 -- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39080c82b0 msgr2=0x7f390814a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:58.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f390d3d6640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39080c82b0 0x7f390814a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f390d3d6640 1 -- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390806d410 msgr2=0x7f39081f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:37:58.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f390d3d6640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390806d410 0x7f39081f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:37:58.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f390d3d6640 1 -- 172.21.15.70:0/3226163057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3904009a70 con 0x7f390815aad0 2023-10-08T05:37:58.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.886+0000 7f390cbd5640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390806d410 0x7f39081f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:37:58.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.887+0000 7f390d3d6640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390815aad0 0x7f390815d3d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3900009870 tx=0x7f3900004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:58.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.887+0000 7f38fd7fa640 1 -- 172.21.15.70:0/3226163057 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900039c90 con 0x7f390815aad0 2023-10-08T05:37:58.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.887+0000 7f38fd7fa640 1 -- 172.21.15.70:0/3226163057 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3900039e10 con 0x7f390815aad0 2023-10-08T05:37:58.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.887+0000 7f38fd7fa640 1 -- 172.21.15.70:0/3226163057 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900043640 con 0x7f390815aad0 2023-10-08T05:37:58.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.887+0000 7f390ee39640 1 -- 172.21.15.70:0/3226163057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39000094d0 con 0x7f390815aad0 2023-10-08T05:37:58.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.887+0000 7f390ee39640 1 -- 172.21.15.70:0/3226163057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3908156be0 con 0x7f390815aad0 2023-10-08T05:37:58.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.888+0000 7f390ee39640 1 -- 172.21.15.70:0/3226163057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3908156760 con 0x7f390815aad0 2023-10-08T05:37:58.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.889+0000 7f38fd7fa640 1 -- 172.21.15.70:0/3226163057 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 89229+0+0 (secure 0 0 0) 0x7f39000437a0 con 0x7f390815aad0 2023-10-08T05:37:58.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.890+0000 7f38fd7fa640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38f406ae70 0x7f38f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:37:58.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.890+0000 7f38fd7fa640 1 -- 172.21.15.70:0/3226163057 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f390003e070 con 0x7f390815aad0 2023-10-08T05:37:58.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.891+0000 7f390cbd5640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38f406ae70 0x7f38f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:37:58.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.892+0000 7f390cbd5640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38f406ae70 0x7f38f406d330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f39040075a0 tx=0x7f390400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:37:58.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:58.895+0000 7f38fd7fa640 1 -- 172.21.15.70:0/3226163057 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3908156760 con 0x7f390815aad0 2023-10-08T05:37:59.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:37:59.137+0000 7f390ee39640 1 -- 172.21.15.70:0/3226163057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f3908156760 con 0x7f390815aad0 2023-10-08T05:38:00.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.047+0000 7f38fd7fa640 1 -- 172.21.15.70:0/3226163057 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f3908156760 con 0x7f390815aad0 2023-10-08T05:38:00.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 -- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38f406ae70 msgr2=0x7f38f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38f406ae70 0x7f38f406d330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f39040075a0 tx=0x7f390400b040 comp rx=0 tx=0).stop 2023-10-08T05:38:00.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 -- 172.21.15.70:0/3226163057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390815aad0 msgr2=0x7f390815d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390815aad0 0x7f390815d3d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3900009870 tx=0x7f3900004730 comp rx=0 tx=0).stop 2023-10-08T05:38:00.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 -- 172.21.15.70:0/3226163057 shutdown_connections 2023-10-08T05:38:00.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7f38f406ae70 0x7f38f406d330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f390815aad0 0x7f390815d3d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39080c82b0 0x7f390814a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 --2- 172.21.15.70:0/3226163057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f390806d410 0x7f39081f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 -- 172.21.15.70:0/3226163057 >> 172.21.15.70:0/3226163057 conn(0x7f3908069d50 msgr2=0x7f39080cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:00.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 -- 172.21.15.70:0/3226163057 shutdown_connections 2023-10-08T05:38:00.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.051+0000 7f390ee39640 1 -- 172.21.15.70:0/3226163057 wait complete. 2023-10-08T05:38:00.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T05:38:00.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.284+0000 7fb387fff640 1 Processor -- start 2023-10-08T05:38:00.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.284+0000 7fb387fff640 1 -- start start 2023-10-08T05:38:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.286+0000 7fb387fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb38815a820 0x7fb3880aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.286+0000 7fb387fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb3880ca680 0x7fb3880b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.286+0000 7fb387fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb38815ac00 0x7fb3880c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.286+0000 7fb387fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3880cab60 con 0x7fb3880ca680 2023-10-08T05:38:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.286+0000 7fb387fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3880cb960 con 0x7fb38815a820 2023-10-08T05:38:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.286+0000 7fb387fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880cbdf0 con 0x7fb38815ac00 2023-10-08T05:38:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.286+0000 7fb37ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb3880ca680 0x7fb3880b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.286+0000 7fb37ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb3880ca680 0x7fb3880b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45442/0 (socket says 172.21.15.70:45442) 2023-10-08T05:38:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.286+0000 7fb37ffff640 1 -- 172.21.15.70:0/2304420587 learned_addr learned my addr 172.21.15.70:0/2304420587 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:00.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.287+0000 7fb386ffd640 1 --2- 172.21.15.70:0/2304420587 >> v2:172.21.15.181:3300/0 conn(0x7fb38815a820 0x7fb3880aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.287+0000 7fb3877fe640 1 --2- 172.21.15.70:0/2304420587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb38815ac00 0x7fb3880c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.287+0000 7fb37ffff640 1 -- 172.21.15.70:0/2304420587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb38815ac00 msgr2=0x7fb3880c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:00.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.287+0000 7fb37ffff640 1 --2- 172.21.15.70:0/2304420587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb38815ac00 0x7fb3880c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.287+0000 7fb37ffff640 1 -- 172.21.15.70:0/2304420587 >> v2:172.21.15.181:3300/0 conn(0x7fb38815a820 msgr2=0x7fb3880aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:00.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.287+0000 7fb37ffff640 1 --2- 172.21.15.70:0/2304420587 >> v2:172.21.15.181:3300/0 conn(0x7fb38815a820 0x7fb3880aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.287+0000 7fb37ffff640 1 -- 172.21.15.70:0/2304420587 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880cb030 con 0x7fb3880ca680 2023-10-08T05:38:00.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.287+0000 7fb3877fe640 1 --2- 172.21.15.70:0/2304420587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb38815ac00 0x7fb3880c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:00.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.288+0000 7fb37ffff640 1 --2- 172.21.15.70:0/2304420587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3880ca680 0x7fb3880b04b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb38806da70 tx=0x7fb37002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4e8ab54bb53b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:00.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.288+0000 7fb3867fc640 1 -- 172.21.15.70:0/2304420587 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3700317d0 con 0x7fb3880ca680 2023-10-08T05:38:00.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.288+0000 7fb3867fc640 1 -- 172.21.15.70:0/2304420587 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb370031930 con 0x7fb3880ca680 2023-10-08T05:38:00.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.288+0000 7fb3867fc640 1 -- 172.21.15.70:0/2304420587 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370031cc0 con 0x7fb3880ca680 2023-10-08T05:38:00.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.289+0000 7fb387fff640 1 -- 172.21.15.70:0/2304420587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3880ca680 msgr2=0x7fb3880b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.289+0000 7fb387fff640 1 --2- 172.21.15.70:0/2304420587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3880ca680 0x7fb3880b04b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb38806da70 tx=0x7fb37002b870 comp rx=0 tx=0).stop 2023-10-08T05:38:00.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.289+0000 7fb387fff640 1 -- 172.21.15.70:0/2304420587 shutdown_connections 2023-10-08T05:38:00.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.289+0000 7fb387fff640 1 --2- 172.21.15.70:0/2304420587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb38815ac00 0x7fb3880c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.289+0000 7fb387fff640 1 --2- 172.21.15.70:0/2304420587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3880ca680 0x7fb3880b04b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.289+0000 7fb387fff640 1 --2- 172.21.15.70:0/2304420587 >> v2:172.21.15.181:3300/0 conn(0x7fb38815a820 0x7fb3880aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.289+0000 7fb387fff640 1 -- 172.21.15.70:0/2304420587 >> 172.21.15.70:0/2304420587 conn(0x7fb38806c050 msgr2=0x7fb38806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.289+0000 7fb387fff640 1 -- 172.21.15.70:0/2304420587 shutdown_connections 2023-10-08T05:38:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.289+0000 7fb387fff640 1 -- 172.21.15.70:0/2304420587 wait complete. 2023-10-08T05:38:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.291+0000 7fb387fff640 1 Processor -- start 2023-10-08T05:38:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.291+0000 7fb387fff640 1 -- start start 2023-10-08T05:38:00.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.292+0000 7fb387fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3880ca680 0x7fb3880c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.292+0000 7fb387fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb38815a820 0x7fb38815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.292+0000 7fb387fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb38815ac00 0x7fb38815d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.292+0000 7fb387fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880b01a0 con 0x7fb38815ac00 2023-10-08T05:38:00.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.292+0000 7fb387fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880aed70 con 0x7fb38815a820 2023-10-08T05:38:00.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.292+0000 7fb387fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3880af2b0 con 0x7fb3880ca680 2023-10-08T05:38:00.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.292+0000 7fb3877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb38815ac00 0x7fb38815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb3877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb38815ac00 0x7fb38815d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45448/0 (socket says 172.21.15.70:45448) 2023-10-08T05:38:00.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb3877fe640 1 -- 172.21.15.70:0/3916506257 learned_addr learned my addr 172.21.15.70:0/3916506257 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:00.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb386ffd640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3880ca680 0x7fb3880c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb37ffff640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb38815a820 0x7fb38815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb3877fe640 1 -- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3880ca680 msgr2=0x7fb3880c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb3877fe640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3880ca680 0x7fb3880c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb3877fe640 1 -- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb38815a820 msgr2=0x7fb38815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb3877fe640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb38815a820 0x7fb38815cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb3877fe640 1 -- 172.21.15.70:0/3916506257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700094d0 con 0x7fb38815ac00 2023-10-08T05:38:00.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb386ffd640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3880ca680 0x7fb3880c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:00.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb37ffff640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb38815a820 0x7fb38815cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:00.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb3877fe640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb38815ac00 0x7fb38815d080 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb38000ce80 tx=0x7fb38000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:00.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.293+0000 7fb37f7fe640 1 -- 172.21.15.70:0/3916506257 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38000d840 con 0x7fb38815ac00 2023-10-08T05:38:00.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.294+0000 7fb37f7fe640 1 -- 172.21.15.70:0/3916506257 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38000d9e0 con 0x7fb38815ac00 2023-10-08T05:38:00.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.294+0000 7fb387fff640 1 -- 172.21.15.70:0/3916506257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3880af590 con 0x7fb38815ac00 2023-10-08T05:38:00.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.294+0000 7fb37f7fe640 1 -- 172.21.15.70:0/3916506257 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38000dd90 con 0x7fb38815ac00 2023-10-08T05:38:00.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.294+0000 7fb387fff640 1 -- 172.21.15.70:0/3916506257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3880afb50 con 0x7fb38815ac00 2023-10-08T05:38:00.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.295+0000 7fb387fff640 1 -- 172.21.15.70:0/3916506257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3880b0310 con 0x7fb38815ac00 2023-10-08T05:38:00.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.297+0000 7fb37f7fe640 1 -- 172.21.15.70:0/3916506257 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7fb380015440 con 0x7fb38815ac00 2023-10-08T05:38:00.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.299+0000 7fb37f7fe640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb36806aec0 0x7fb36806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.299+0000 7fb37f7fe640 1 -- 172.21.15.70:0/3916506257 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb38008a0b0 con 0x7fb38815ac00 2023-10-08T05:38:00.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.299+0000 7fb386ffd640 1 -- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb36806aec0 msgr2=0x7fb36806d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2737793492 2023-10-08T05:38:00.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.300+0000 7fb386ffd640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb36806aec0 0x7fb36806d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:38:00.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.303+0000 7fb37f7fe640 1 -- 172.21.15.70:0/3916506257 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3800550b0 con 0x7fb38815ac00 2023-10-08T05:38:00.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.495+0000 7fb387fff640 1 -- 172.21.15.70:0/3916506257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fb3880b1310 con 0x7fb38815ac00 2023-10-08T05:38:00.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.500+0000 7fb37f7fe640 1 -- 172.21.15.70:0/3916506257 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb3800586d0 con 0x7fb38815ac00 2023-10-08T05:38:00.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.501+0000 7fb386ffd640 1 -- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb36806aec0 msgr2=0x7fb36806d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2737793492 2023-10-08T05:38:00.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.501+0000 7fb386ffd640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb36806aec0 0x7fb36806d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:38:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 -- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb36806aec0 msgr2=0x7fb36806d380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:38:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb36806aec0 0x7fb36806d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 -- 172.21.15.70:0/3916506257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb38815ac00 msgr2=0x7fb38815d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb38815ac00 0x7fb38815d080 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb38000ce80 tx=0x7fb38000a710 comp rx=0 tx=0).stop 2023-10-08T05:38:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 -- 172.21.15.70:0/3916506257 shutdown_connections 2023-10-08T05:38:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fb36806aec0 0x7fb36806d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb38815ac00 0x7fb38815d080 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb38815a820 0x7fb38815cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 --2- 172.21.15.70:0/3916506257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3880ca680 0x7fb3880c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 -- 172.21.15.70:0/3916506257 >> 172.21.15.70:0/3916506257 conn(0x7fb38806c050 msgr2=0x7fb3880c9990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:00.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 -- 172.21.15.70:0/3916506257 shutdown_connections 2023-10-08T05:38:00.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.503+0000 7fb387fff640 1 -- 172.21.15.70:0/3916506257 wait complete. 2023-10-08T05:38:00.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T05:38:00.582 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:00.582+0000 7f3b76e12040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:38:00.593 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:00.593+0000 7ff53e030040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:38:00.637 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:00.637+0000 7f3b76e12040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:38:00.648 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:00.648+0000 7ff53e030040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:38:00.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.724+0000 7ff6f07b6640 1 Processor -- start 2023-10-08T05:38:00.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.724+0000 7ff6f07b6640 1 -- start start 2023-10-08T05:38:00.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.726+0000 7ff6f07b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e806dfc0 0x7ff6e8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.726+0000 7ff6f07b6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e80b1cc0 con 0x7ff6e8071070 2023-10-08T05:38:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.726+0000 7ff6f07b6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e80b2620 con 0x7ff6e80afbe0 2023-10-08T05:38:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.726+0000 7ff6f07b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b2060 con 0x7ff6e806dfc0 2023-10-08T05:38:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.726+0000 7ff6eed53640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff6e8071070 0x7ff6e80cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59966/0 (socket says 172.21.15.70:59966) 2023-10-08T05:38:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.726+0000 7ff6eed53640 1 -- 172.21.15.70:0/2726406734 learned_addr learned my addr 172.21.15.70:0/2726406734 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6edd51640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e806dfc0 0x7ff6e8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941247635 0 0) 0x7ff6e80b1cc0 con 0x7ff6e8071070 2023-10-08T05:38:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d4003520 con 0x7ff6e8071070 2023-10-08T05:38:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 601091330 0 0) 0x7ff6d4003520 con 0x7ff6e8071070 2023-10-08T05:38:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e806dfc0 msgr2=0x7ff6e8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e806dfc0 0x7ff6e8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 >> v1:172.21.15.181:6789/0 conn(0x7ff6e80afbe0 legacy=0x7ff6e80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80b2340 con 0x7ff6e8071070 2023-10-08T05:38:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff6e00019f0 con 0x7ff6e8071070 2023-10-08T05:38:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 >> v1:172.21.15.70:6789/0 conn(0x7ff6e8071070 legacy=0x7ff6e80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d40039a0 0x7ff6d4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e806dfc0 0x7ff6d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4004d00 0x7ff6d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b2060 con 0x7ff6d4004d00 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d400b9e0 con 0x7ff6d40039a0 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.727+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d400bba0 con 0x7ff6e806dfc0 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.728+0000 7ff6eed53640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4004d00 0x7ff6d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.728+0000 7ff6edd51640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e806dfc0 0x7ff6d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.728+0000 7ff6eed53640 1 -- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e806dfc0 msgr2=0x7ff6d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.728+0000 7ff6eed53640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6e806dfc0 0x7ff6d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.728+0000 7ff6eed53640 1 -- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d40039a0 msgr2=0x7ff6d4003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.728+0000 7ff6ee552640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d40039a0 0x7ff6d4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.728+0000 7ff6eed53640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d40039a0 0x7ff6d4003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.728+0000 7ff6eed53640 1 -- 172.21.15.70:0/2726406734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4003520 con 0x7ff6d4004d00 2023-10-08T05:38:00.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.728+0000 7ff6ee552640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6d40039a0 0x7ff6d4003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.729+0000 7ff6eed53640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4004d00 0x7ff6d4005180 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000a7d0 tx=0x7ff6d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbcd9106fabeb21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.729+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0034630 con 0x7ff6d4004d00 2023-10-08T05:38:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.729+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e002f070 con 0x7ff6d4004d00 2023-10-08T05:38:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.729+0000 7ff6ed550640 1 -- 172.21.15.70:0/2726406734 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e00349f0 con 0x7ff6d4004d00 2023-10-08T05:38:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.730+0000 7ff6f07b6640 1 -- 172.21.15.70:0/2726406734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4004d00 msgr2=0x7ff6d4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.730+0000 7ff6f07b6640 1 --2- 172.21.15.70:0/2726406734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6d4004d00 0x7ff6d4005180 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000a7d0 tx=0x7ff6d40057f0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.730+0000 7ff6eed53640 1 -- 172.21.15.70:0/2726406734 reap_dead start 2023-10-08T05:38:00.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.730+0000 7ff6f07b6640 1 -- 172.21.15.70:0/2726406734 shutdown_connections 2023-10-08T05:38:00.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.730+0000 7ff6f07b6640 1 -- 172.21.15.70:0/2726406734 >> 172.21.15.70:0/2726406734 conn(0x7ff6e806a5d0 msgr2=0x7ff6e80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:00.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.730+0000 7ff6f07b6640 1 -- 172.21.15.70:0/2726406734 shutdown_connections 2023-10-08T05:38:00.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.730+0000 7ff6f07b6640 1 -- 172.21.15.70:0/2726406734 wait complete. 2023-10-08T05:38:00.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.732+0000 7ff6f07b6640 1 Processor -- start 2023-10-08T05:38:00.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.732+0000 7ff6f07b6640 1 -- start start 2023-10-08T05:38:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.733+0000 7ff6f07b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6d4004d00 0x7ff6e815f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.733+0000 7ff6f07b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6e80c33c0 0x7ff6e8157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.733+0000 7ff6f07b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e815daf0 0x7ff6e815df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.733+0000 7ff6f07b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e815e4c0 con 0x7ff6e815daf0 2023-10-08T05:38:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.733+0000 7ff6f07b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80c5040 con 0x7ff6e80c33c0 2023-10-08T05:38:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.733+0000 7ff6f07b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80c7540 con 0x7ff6d4004d00 2023-10-08T05:38:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.733+0000 7ff6eed53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e815daf0 0x7ff6e815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6eed53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e815daf0 0x7ff6e815df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45460/0 (socket says 172.21.15.70:45460) 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6eed53640 1 -- 172.21.15.70:0/215400654 learned_addr learned my addr 172.21.15.70:0/215400654 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6ee552640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6d4004d00 0x7ff6e815f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6edd51640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6e80c33c0 0x7ff6e8157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6eed53640 1 -- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6d4004d00 msgr2=0x7ff6e815f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6eed53640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6d4004d00 0x7ff6e815f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6eed53640 1 -- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6e80c33c0 msgr2=0x7ff6e8157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6eed53640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6e80c33c0 0x7ff6e8157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6eed53640 1 -- 172.21.15.70:0/215400654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80c6930 con 0x7ff6e815daf0 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6edd51640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6e80c33c0 0x7ff6e8157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.734+0000 7ff6eed53640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e815daf0 0x7ff6e815df80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000a7d0 tx=0x7ff6e0001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:00.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.735+0000 7ff6deffd640 1 -- 172.21.15.70:0/215400654 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0044070 con 0x7ff6e815daf0 2023-10-08T05:38:00.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.735+0000 7ff6deffd640 1 -- 172.21.15.70:0/215400654 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e0004d70 con 0x7ff6e815daf0 2023-10-08T05:38:00.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.735+0000 7ff6f07b6640 1 -- 172.21.15.70:0/215400654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e80c6c10 con 0x7ff6e815daf0 2023-10-08T05:38:00.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.735+0000 7ff6deffd640 1 -- 172.21.15.70:0/215400654 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0034430 con 0x7ff6e815daf0 2023-10-08T05:38:00.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.735+0000 7ff6f07b6640 1 -- 172.21.15.70:0/215400654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e80c71d0 con 0x7ff6e815daf0 2023-10-08T05:38:00.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.736+0000 7ff6f07b6640 1 -- 172.21.15.70:0/215400654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e804e160 con 0x7ff6e815daf0 2023-10-08T05:38:00.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.737+0000 7ff6deffd640 1 -- 172.21.15.70:0/215400654 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7ff6e0031030 con 0x7ff6e815daf0 2023-10-08T05:38:00.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.738+0000 7ff6deffd640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff6cc06aef0 0x7ff6cc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:00.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.738+0000 7ff6deffd640 1 -- 172.21.15.70:0/215400654 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff6e002ce40 con 0x7ff6e815daf0 2023-10-08T05:38:00.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.738+0000 7ff6ee552640 1 -- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff6cc06aef0 msgr2=0x7ff6cc06d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2737793492 2023-10-08T05:38:00.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.738+0000 7ff6ee552640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff6cc06aef0 0x7ff6cc06d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:38:00.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.746+0000 7ff6deffd640 1 -- 172.21.15.70:0/215400654 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff6e0076380 con 0x7ff6e815daf0 2023-10-08T05:38:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.939+0000 7ff6ee552640 1 -- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff6cc06aef0 msgr2=0x7ff6cc06d3b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2737793492 2023-10-08T05:38:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.939+0000 7ff6f07b6640 1 -- 172.21.15.70:0/215400654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ff6e80c7680 con 0x7ff6e815daf0 2023-10-08T05:38:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.939+0000 7ff6ee552640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff6cc06aef0 0x7ff6cc06d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:38:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.943+0000 7ff6deffd640 1 -- 172.21.15.70:0/215400654 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff6e00011c0 con 0x7ff6e815daf0 2023-10-08T05:38:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.946+0000 7ff6f07b6640 1 -- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff6cc06aef0 msgr2=0x7ff6cc06d3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:38:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.946+0000 7ff6f07b6640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff6cc06aef0 0x7ff6cc06d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.946+0000 7ff6f07b6640 1 -- 172.21.15.70:0/215400654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e815daf0 msgr2=0x7ff6e815df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.946+0000 7ff6f07b6640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e815daf0 0x7ff6e815df80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000a7d0 tx=0x7ff6e0001d00 comp rx=0 tx=0).stop 2023-10-08T05:38:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.946+0000 7ff6f07b6640 1 -- 172.21.15.70:0/215400654 shutdown_connections 2023-10-08T05:38:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.946+0000 7ff6f07b6640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7ff6cc06aef0 0x7ff6cc06d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.946+0000 7ff6f07b6640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6e815daf0 0x7ff6e815df80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.946+0000 7ff6f07b6640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6e80c33c0 0x7ff6e8157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.946+0000 7ff6f07b6640 1 --2- 172.21.15.70:0/215400654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6d4004d00 0x7ff6e815f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.946+0000 7ff6f07b6640 1 -- 172.21.15.70:0/215400654 >> 172.21.15.70:0/215400654 conn(0x7ff6e806a5d0 msgr2=0x7ff6e80b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.947+0000 7ff6f07b6640 1 -- 172.21.15.70:0/215400654 shutdown_connections 2023-10-08T05:38:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:00.947+0000 7ff6f07b6640 1 -- 172.21.15.70:0/215400654 wait complete. 2023-10-08T05:38:00.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:00.964 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T05:38:01.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:01.020 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-10-08T05:38:01.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir0 1m 2023-10-08T05:38:01.093 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:01.092+0000 7f3b76e12040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:38:01.126 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:01.125+0000 7ff53e030040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:38:01.145 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:01.145+0000 7f3b76e12040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:38:01.180 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:01.180+0000 7ff53e030040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:38:01.195 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:01.195+0000 7f3b76e12040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:38:01.233 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:01.232+0000 7ff53e030040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:38:01.255 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:01.255+0000 7f3b76e12040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:38:01.296 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:01.295+0000 7ff53e030040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:38:01.304 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:01.304+0000 7f3b76e12040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:38:01.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.314+0000 7fe5b7a10640 1 Processor -- start 2023-10-08T05:38:01.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.314+0000 7fe5b7a10640 1 -- start start 2023-10-08T05:38:01.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.315+0000 7fe5b7a10640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe5b00cd930 0x7fe5b00cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b7a10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5b00d0d30 0x7fe5b00d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b7a10640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b00b7e70 con 0x7fe5b015cca0 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b7a10640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b015d080 con 0x7fe5b00cd930 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b7a10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00b7260 con 0x7fe5b00d0d30 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b4fab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe5b00cd930 0x7fe5b00cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b4fab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe5b00cd930 0x7fe5b00cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45464/0 (socket says 172.21.15.70:45464) 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b4fab640 1 -- 172.21.15.70:0/1018035080 learned_addr learned my addr 172.21.15.70:0/1018035080 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b5fad640 1 --2- 172.21.15.70:0/1018035080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5b00d0d30 0x7fe5b00d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b4fab640 1 -- 172.21.15.70:0/1018035080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5b00d0d30 msgr2=0x7fe5b00d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b4fab640 1 --2- 172.21.15.70:0/1018035080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5b00d0d30 0x7fe5b00d11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b4fab640 1 -- 172.21.15.70:0/1018035080 >> v1:172.21.15.181:6789/0 conn(0x7fe5b015cca0 legacy=0x7fe5b00b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:01.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.316+0000 7fe5b4fab640 1 -- 172.21.15.70:0/1018035080 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00b7540 con 0x7fe5b00cd930 2023-10-08T05:38:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.317+0000 7fe5b4fab640 1 --2- 172.21.15.70:0/1018035080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b00cd930 0x7fe5b00cdd50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009870 tx=0x7fe5a002ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808ea2a9a67a10e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.318+0000 7fe5a7fff640 1 -- 172.21.15.70:0/1018035080 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a002f830 con 0x7fe5b00cd930 2023-10-08T05:38:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.318+0000 7fe5a7fff640 1 -- 172.21.15.70:0/1018035080 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a002f990 con 0x7fe5b00cd930 2023-10-08T05:38:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.318+0000 7fe5a7fff640 1 -- 172.21.15.70:0/1018035080 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a002fd20 con 0x7fe5b00cd930 2023-10-08T05:38:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.318+0000 7fe5b7a10640 1 -- 172.21.15.70:0/1018035080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b00cd930 msgr2=0x7fe5b00cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.318+0000 7fe5b7a10640 1 --2- 172.21.15.70:0/1018035080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b00cd930 0x7fe5b00cdd50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009870 tx=0x7fe5a002ba10 comp rx=0 tx=0).stop 2023-10-08T05:38:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.318+0000 7fe5b7a10640 1 -- 172.21.15.70:0/1018035080 shutdown_connections 2023-10-08T05:38:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.318+0000 7fe5b7a10640 1 --2- 172.21.15.70:0/1018035080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5b00d0d30 0x7fe5b00d11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.318+0000 7fe5b7a10640 1 --2- 172.21.15.70:0/1018035080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b00cd930 0x7fe5b00cdd50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.318+0000 7fe5b7a10640 1 -- 172.21.15.70:0/1018035080 >> 172.21.15.70:0/1018035080 conn(0x7fe5b00718a0 msgr2=0x7fe5b0071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.319+0000 7fe5b7a10640 1 -- 172.21.15.70:0/1018035080 shutdown_connections 2023-10-08T05:38:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.319+0000 7fe5b7a10640 1 -- 172.21.15.70:0/1018035080 wait complete. 2023-10-08T05:38:01.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.320+0000 7fe5b7a10640 1 Processor -- start 2023-10-08T05:38:01.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.321+0000 7fe5b7a10640 1 -- start start 2023-10-08T05:38:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b7a10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b00cd930 0x7fe5b0153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b7a10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b00d0d30 0x7fe5b01539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b7a10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5b015cca0 0x7fe5b015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b7a10640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00cbd30 con 0x7fe5b00d0d30 2023-10-08T05:38:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b7a10640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b01537d0 con 0x7fe5b00cd930 2023-10-08T05:38:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b7a10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b00cb700 con 0x7fe5b015cca0 2023-10-08T05:38:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b4fab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b00d0d30 0x7fe5b01539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b4fab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b00d0d30 0x7fe5b01539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45470/0 (socket says 172.21.15.70:45470) 2023-10-08T05:38:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b4fab640 1 -- 172.21.15.70:0/2400963575 learned_addr learned my addr 172.21.15.70:0/2400963575 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b5fad640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5b015cca0 0x7fe5b015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b57ac640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b00cd930 0x7fe5b0153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b4fab640 1 -- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5b015cca0 msgr2=0x7fe5b015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b4fab640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5b015cca0 0x7fe5b015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b4fab640 1 -- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b00cd930 msgr2=0x7fe5b0153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b4fab640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5b00cd930 0x7fe5b0153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.322+0000 7fe5b4fab640 1 -- 172.21.15.70:0/2400963575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00094d0 con 0x7fe5b00d0d30 2023-10-08T05:38:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.323+0000 7fe5b4fab640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b00d0d30 0x7fe5b01539d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0000c00 tx=0x7fe5a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.323+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0037c40 con 0x7fe5b00d0d30 2023-10-08T05:38:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.323+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a0037da0 con 0x7fe5b00d0d30 2023-10-08T05:38:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.323+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0041640 con 0x7fe5b00d0d30 2023-10-08T05:38:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.323+0000 7fe5b7a10640 1 -- 172.21.15.70:0/2400963575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b00caa60 con 0x7fe5b00d0d30 2023-10-08T05:38:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.323+0000 7fe5b7a10640 1 -- 172.21.15.70:0/2400963575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b00cb950 con 0x7fe5b00d0d30 2023-10-08T05:38:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.324+0000 7fe5b7a10640 1 -- 172.21.15.70:0/2400963575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b00cbe70 con 0x7fe5b00d0d30 2023-10-08T05:38:01.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.325+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 6) v1 ==== 89246+0+0 (secure 0 0 0) 0x7fe5a00417a0 con 0x7fe5b00d0d30 2023-10-08T05:38:01.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.326+0000 7fe5a5ffb640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 0x7fe59406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:01.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.326+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe5a003c070 con 0x7fe5b00d0d30 2023-10-08T05:38:01.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.326+0000 7fe5b57ac640 1 -- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 msgr2=0x7fe59406d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2737793492 2023-10-08T05:38:01.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.326+0000 7fe5b57ac640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 0x7fe59406d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:38:01.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.330+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5b00cbe70 con 0x7fe5b00d0d30 2023-10-08T05:38:01.349 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:01.348+0000 7ff53e030040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:38:01.415 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:01.415+0000 7f3b76e12040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:38:01.464 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:01.463+0000 7ff53e030040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:38:01.477 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:01.477+0000 7f3b76e12040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:38:01.526 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:01.526+0000 7f3b76e12040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:38:01.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.527+0000 7fe5b57ac640 1 -- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 msgr2=0x7fe59406d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2737793492 2023-10-08T05:38:01.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.527+0000 7fe5b57ac640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 0x7fe59406d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:38:01.528 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:01.528+0000 7ff53e030040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:38:01.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.542+0000 7fe5b7a10640 1 -- 172.21.15.70:0/2400963575 --> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5b00b7b00 con 0x7fe59406aec0 2023-10-08T05:38:01.579 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:01.578+0000 7ff53e030040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:38:01.583 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:01.583+0000 7f3b76e12040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:38:01.632 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:01.632+0000 7f3b76e12040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:38:01.635 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:01.634+0000 7ff53e030040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:38:01.686 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:01.686+0000 7ff53e030040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:38:01.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.928+0000 7fe5b57ac640 1 -- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 msgr2=0x7fe59406d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2737793492 2023-10-08T05:38:01.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:01.928+0000 7fe5b57ac640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 0x7fe59406d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:38:02.015 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:02.015+0000 7f3b76e12040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:38:02.070 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:02.070+0000 7ff53e030040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:38:02.172 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:02.172+0000 7f3b76e12040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:38:02.228 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:02.227+0000 7ff53e030040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:38:02.280 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:02.279+0000 7f3b76e12040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:38:02.329 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:02.329+0000 7f3b76e12040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:38:02.333 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:02.333+0000 7ff53e030040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:38:02.382 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:02.382+0000 7ff53e030040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:38:02.435 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:02.435+0000 7f3b76e12040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:38:02.488 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:02.487+0000 7ff53e030040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:38:02.540 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:02.540+0000 7f3b76e12040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:38:02.594 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:02.594+0000 7ff53e030040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:38:02.695 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:02.695+0000 7f3b76e12040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:38:02.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:02.729+0000 7fe5b57ac640 1 -- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 msgr2=0x7fe59406d380 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2737793492 2023-10-08T05:38:02.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:02.729+0000 7fe5b57ac640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 0x7fe59406d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:38:02.744 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:02.744+0000 7f3b76e12040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:38:02.748 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:02.748+0000 7ff53e030040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:38:02.797 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:02.797+0000 7ff53e030040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:38:02.863 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:02.862+0000 7f3b76e12040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:38:02.916 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:02.915+0000 7ff53e030040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:38:03.687 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:03.686+0000 7f3b76e12040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:38:03.726 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:03.725+0000 7ff53e030040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:38:03.998 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:03.998+0000 7f3b76e12040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:38:04.021 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:04.021+0000 7ff53e030040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:38:04.055 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.055+0000 7f3b76e12040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:38:04.077 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:04.077+0000 7ff53e030040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:38:04.167 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.166+0000 7f3b76e12040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:38:04.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:04.189+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 7) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fe5a00778c0 con 0x7fe5b00d0d30 2023-10-08T05:38:04.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:04.190+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 msgr2=0x7fe59406d380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:38:04.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:04.190+0000 7fe5a5ffb640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2737793492,v1:172.21.15.181:6833/2737793492] conn(0x7fe59406aec0 0x7fe59406d380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:04.192 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:38:04.191+0000 7ff53e030040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:38:04.386 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.384+0000 7f3af3483640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:38:04.386 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.384+0000 7f3af3483640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:38:04.386 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.384+0000 7f3af3483640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:38:04.386 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.384+0000 7f3af3483640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:38:04.386 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.384+0000 7f3af3483640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:38:04.386 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.385+0000 7f3af64c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:38:04.386 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.385+0000 7f3af64c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:38:04.386 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.385+0000 7f3af64c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:38:04.386 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.385+0000 7f3af64c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:38:04.387 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:38:04.385+0000 7f3af64c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:38:05.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:38:05.098 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:38:05.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.197+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 8) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fe5a0049360 con 0x7fe5b00d0d30 2023-10-08T05:38:05.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.198+0000 7fe5a5ffb640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe594035aa0 0x7fe5940752f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:05.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.198+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe594035e80 con 0x7fe594035aa0 2023-10-08T05:38:05.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.199+0000 7fe5b57ac640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe594035aa0 0x7fe5940752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:05.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.201+0000 7fe5b57ac640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe594035aa0 0x7fe5940752f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00c6080 tx=0x7fe5ac001120 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:05.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.358+0000 7fe5a5ffb640 1 -- 172.21.15.70:0/2400963575 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe594035e80 con 0x7fe594035aa0 2023-10-08T05:38:05.359 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/dir0 2023-10-08T05:38:05.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.361+0000 7fe5b7a10640 1 -- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe594035aa0 msgr2=0x7fe5940752f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:05.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.361+0000 7fe5b7a10640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe594035aa0 0x7fe5940752f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00c6080 tx=0x7fe5ac001120 comp rx=0 tx=0).stop 2023-10-08T05:38:05.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.361+0000 7fe5b7a10640 1 -- 172.21.15.70:0/2400963575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b00d0d30 msgr2=0x7fe5b01539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:05.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.361+0000 7fe5b7a10640 1 --2- 172.21.15.70:0/2400963575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5b00d0d30 0x7fe5b01539d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0000c00 tx=0x7fe5a0004730 comp rx=0 tx=0).stop 2023-10-08T05:38:05.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.361+0000 7fe5b5fad640 1 -- 172.21.15.70:0/2400963575 reap_dead start 2023-10-08T05:38:05.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.361+0000 7fe5b7a10640 1 -- 172.21.15.70:0/2400963575 shutdown_connections 2023-10-08T05:38:05.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.361+0000 7fe5b7a10640 1 -- 172.21.15.70:0/2400963575 >> 172.21.15.70:0/2400963575 conn(0x7fe5b00718a0 msgr2=0x7fe5b00ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:05.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.361+0000 7fe5b7a10640 1 -- 172.21.15.70:0/2400963575 shutdown_connections 2023-10-08T05:38:05.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.361+0000 7fe5b7a10640 1 -- 172.21.15.70:0/2400963575 wait complete. 2023-10-08T05:38:05.382 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-10-08T05:38:05.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:05.382 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-10-08T05:38:05.440 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir1 1m 2023-10-08T05:38:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.653+0000 7f54a0cc2640 1 Processor -- start 2023-10-08T05:38:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.653+0000 7f54a0cc2640 1 -- start start 2023-10-08T05:38:05.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.655+0000 7f54a0cc2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f549c0cb880 0x7f549c0b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.655+0000 7f54a0cc2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c15d440 con 0x7f549c0b4070 2023-10-08T05:38:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.655+0000 7f54a0cc2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c15d690 con 0x7f549c1432f0 2023-10-08T05:38:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.655+0000 7f54a0cc2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c1436d0 con 0x7f549c0cb880 2023-10-08T05:38:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.655+0000 7f5493fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f549c0cb880 0x7f549c0b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.655+0000 7f549ad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f549c0b4070 0x7f549c159e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36230/0 (socket says 172.21.15.70:36230) 2023-10-08T05:38:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.655+0000 7f5493fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f549c0cb880 0x7f549c0b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43134/0 (socket says 172.21.15.70:43134) 2023-10-08T05:38:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.655+0000 7f549ad9d640 1 -- 172.21.15.70:0/1952818371 learned_addr learned my addr 172.21.15.70:0/1952818371 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:05.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.656+0000 7f5493fff640 1 -- 172.21.15.70:0/1952818371 >> v1:172.21.15.181:6789/0 conn(0x7f549c1432f0 legacy=0x7f549c0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:05.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.656+0000 7f5493fff640 1 -- 172.21.15.70:0/1952818371 >> v1:172.21.15.70:6789/0 conn(0x7f549c0b4070 legacy=0x7f549c159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:05.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.656+0000 7f5493fff640 1 -- 172.21.15.70:0/1952818371 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c15d970 con 0x7f549c0cb880 2023-10-08T05:38:05.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.656+0000 7f5499d9b640 1 -- 172.21.15.70:0/1952818371 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078117647 0 0) 0x7f549c15d440 con 0x7f549c0b4070 2023-10-08T05:38:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.657+0000 7f5493fff640 1 --2- 172.21.15.70:0/1952818371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 0x7f549c0b3b30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f54840097a0 tx=0x7f548402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e4c3922635ebe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.657+0000 7f5499d9b640 1 -- 172.21.15.70:0/1952818371 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484031830 con 0x7f549c0cb880 2023-10-08T05:38:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.657+0000 7f5499d9b640 1 -- 172.21.15.70:0/1952818371 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5484031990 con 0x7f549c0cb880 2023-10-08T05:38:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.657+0000 7f5499d9b640 1 -- 172.21.15.70:0/1952818371 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484031d20 con 0x7f549c0cb880 2023-10-08T05:38:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.658+0000 7f54a0cc2640 1 -- 172.21.15.70:0/1952818371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 msgr2=0x7f549c0b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.658+0000 7f54a0cc2640 1 --2- 172.21.15.70:0/1952818371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 0x7f549c0b3b30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f54840097a0 tx=0x7f548402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:38:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.658+0000 7f54a0cc2640 1 -- 172.21.15.70:0/1952818371 shutdown_connections 2023-10-08T05:38:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.658+0000 7f54a0cc2640 1 --2- 172.21.15.70:0/1952818371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 0x7f549c0b3b30 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.658+0000 7f54a0cc2640 1 -- 172.21.15.70:0/1952818371 >> 172.21.15.70:0/1952818371 conn(0x7f549c069d50 msgr2=0x7f549c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.658+0000 7f54a0cc2640 1 -- 172.21.15.70:0/1952818371 shutdown_connections 2023-10-08T05:38:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.658+0000 7f54a0cc2640 1 -- 172.21.15.70:0/1952818371 wait complete. 2023-10-08T05:38:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.660+0000 7f54a0cc2640 1 Processor -- start 2023-10-08T05:38:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.661+0000 7f54a0cc2640 1 -- start start 2023-10-08T05:38:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.662+0000 7f54a0cc2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549c0b4070 0x7f549c157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.662+0000 7f54a0cc2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 0x7f549c136d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.662+0000 7f54a0cc2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c1432f0 0x7f549c0cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.662+0000 7f54a0cc2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0a90e0 con 0x7f549c0cb880 2023-10-08T05:38:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.662+0000 7f54a0cc2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0a8460 con 0x7f549c1432f0 2023-10-08T05:38:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.662+0000 7f54a0cc2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0a8740 con 0x7f549c0b4070 2023-10-08T05:38:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.662+0000 7f5493fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 0x7f549c136d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.662+0000 7f5493fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 0x7f549c136d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43140/0 (socket says 172.21.15.70:43140) 2023-10-08T05:38:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.662+0000 7f5493fff640 1 -- 172.21.15.70:0/3928521595 learned_addr learned my addr 172.21.15.70:0/3928521595 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.662+0000 7f549a59c640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549c0b4070 0x7f549c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:05.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.663+0000 7f549ad9d640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c1432f0 0x7f549c0cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:05.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.663+0000 7f5493fff640 1 -- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549c0b4070 msgr2=0x7f549c157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:05.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.663+0000 7f5493fff640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549c0b4070 0x7f549c157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:05.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.663+0000 7f5493fff640 1 -- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c1432f0 msgr2=0x7f549c0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:05.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.663+0000 7f5493fff640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c1432f0 0x7f549c0cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:05.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.663+0000 7f5493fff640 1 -- 172.21.15.70:0/3928521595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484009400 con 0x7f549c0cb880 2023-10-08T05:38:05.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.663+0000 7f5493fff640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 0x7f549c136d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5484000c00 tx=0x7f54840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:05.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.663+0000 7f5492ffd640 1 -- 172.21.15.70:0/3928521595 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484004030 con 0x7f549c0cb880 2023-10-08T05:38:05.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.663+0000 7f5492ffd640 1 -- 172.21.15.70:0/3928521595 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5484004190 con 0x7f549c0cb880 2023-10-08T05:38:05.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.663+0000 7f5492ffd640 1 -- 172.21.15.70:0/3928521595 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484004520 con 0x7f549c0cb880 2023-10-08T05:38:05.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.664+0000 7f54a0cc2640 1 -- 172.21.15.70:0/3928521595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c0a8a20 con 0x7f549c0cb880 2023-10-08T05:38:05.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.664+0000 7f54a0cc2640 1 -- 172.21.15.70:0/3928521595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c157770 con 0x7f549c0cb880 2023-10-08T05:38:05.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.665+0000 7f54a0cc2640 1 -- 172.21.15.70:0/3928521595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c04e160 con 0x7f549c0cb880 2023-10-08T05:38:05.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.666+0000 7f5492ffd640 1 -- 172.21.15.70:0/3928521595 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 8) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f548403c980 con 0x7f549c0cb880 2023-10-08T05:38:05.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.667+0000 7f5492ffd640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f547c06ad10 0x7f547c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:05.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.667+0000 7f5492ffd640 1 -- 172.21.15.70:0/3928521595 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f54840abcd0 con 0x7f549c0cb880 2023-10-08T05:38:05.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.667+0000 7f549a59c640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f547c06ad10 0x7f547c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:05.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.668+0000 7f549a59c640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f547c06ad10 0x7f547c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f548c000f60 tx=0x7f548c009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:05.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.671+0000 7f5492ffd640 1 -- 172.21.15.70:0/3928521595 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5484076d90 con 0x7f549c0cb880 2023-10-08T05:38:05.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:05.886+0000 7f54a0cc2640 1 -- 172.21.15.70:0/3928521595 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f549c15df30 con 0x7f547c06ad10 2023-10-08T05:38:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.015+0000 7f5492ffd640 1 -- 172.21.15.70:0/3928521595 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f548c0012c0 con 0x7f547c06ad10 2023-10-08T05:38:06.016 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/dir1 2023-10-08T05:38:06.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.018+0000 7f54a0cc2640 1 -- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f547c06ad10 msgr2=0x7f547c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.018+0000 7f54a0cc2640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f547c06ad10 0x7f547c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f548c000f60 tx=0x7f548c009040 comp rx=0 tx=0).stop 2023-10-08T05:38:06.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.018+0000 7f54a0cc2640 1 -- 172.21.15.70:0/3928521595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 msgr2=0x7f549c136d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.018+0000 7f54a0cc2640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 0x7f549c136d90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5484000c00 tx=0x7f54840046c0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.018+0000 7f54a0cc2640 1 -- 172.21.15.70:0/3928521595 shutdown_connections 2023-10-08T05:38:06.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.018+0000 7f54a0cc2640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f549c1432f0 0x7f549c0cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.018+0000 7f54a0cc2640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f549c0cb880 0x7f549c136d90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.018+0000 7f54a0cc2640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f547c06ad10 0x7f547c06d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.018+0000 7f54a0cc2640 1 --2- 172.21.15.70:0/3928521595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f549c0b4070 0x7f549c157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.018+0000 7f54a0cc2640 1 -- 172.21.15.70:0/3928521595 >> 172.21.15.70:0/3928521595 conn(0x7f549c069d50 msgr2=0x7f549c06c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:06.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.019+0000 7f54a0cc2640 1 -- 172.21.15.70:0/3928521595 shutdown_connections 2023-10-08T05:38:06.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.019+0000 7f54a0cc2640 1 -- 172.21.15.70:0/3928521595 wait complete. 2023-10-08T05:38:06.039 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-10-08T05:38:06.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:06.039 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-10-08T05:38:06.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir2 1m 2023-10-08T05:38:06.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.321+0000 7f5322b93640 1 Processor -- start 2023-10-08T05:38:06.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.322+0000 7f5322b93640 1 -- start start 2023-10-08T05:38:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.323+0000 7f5322b93640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f531c15aab0 0x7f531c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.323+0000 7f5322b93640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f531c0caef0 0x7f531c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.323+0000 7f5322b93640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531c15ae90 con 0x7f531c0b04a0 2023-10-08T05:38:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.323+0000 7f5322b93640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c0b0930 con 0x7f531c15aab0 2023-10-08T05:38:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.323+0000 7f5322b93640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531c0cbdb0 con 0x7f531c0caef0 2023-10-08T05:38:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.323+0000 7f5321b91640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f531c15aab0 0x7f531c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.323+0000 7f5321b91640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f531c15aab0 0x7f531c070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43156/0 (socket says 172.21.15.70:43156) 2023-10-08T05:38:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.323+0000 7f5321b91640 1 -- 172.21.15.70:0/553151634 learned_addr learned my addr 172.21.15.70:0/553151634 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.323+0000 7f5321390640 1 --2- 172.21.15.70:0/553151634 >> v2:172.21.15.181:3300/0 conn(0x7f531c0caef0 0x7f531c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.324+0000 7f5321b91640 1 -- 172.21.15.70:0/553151634 >> v2:172.21.15.181:3300/0 conn(0x7f531c0caef0 msgr2=0x7f531c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:06.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.324+0000 7f5321b91640 1 --2- 172.21.15.70:0/553151634 >> v2:172.21.15.181:3300/0 conn(0x7f531c0caef0 0x7f531c0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.324+0000 7f5321b91640 1 -- 172.21.15.70:0/553151634 >> v1:172.21.15.70:6789/0 conn(0x7f531c0b04a0 legacy=0x7f531c0c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.324+0000 7f5321b91640 1 -- 172.21.15.70:0/553151634 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c0cb7f0 con 0x7f531c15aab0 2023-10-08T05:38:06.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.325+0000 7f5321b91640 1 --2- 172.21.15.70:0/553151634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 0x7f531c070b50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5318009870 tx=0x7f531802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fccdd05a668b7e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:06.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.325+0000 7f5320b8f640 1 -- 172.21.15.70:0/553151634 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318031870 con 0x7f531c15aab0 2023-10-08T05:38:06.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.326+0000 7f5320b8f640 1 -- 172.21.15.70:0/553151634 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53180319d0 con 0x7f531c15aab0 2023-10-08T05:38:06.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.326+0000 7f5320b8f640 1 -- 172.21.15.70:0/553151634 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318031d60 con 0x7f531c15aab0 2023-10-08T05:38:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.326+0000 7f5322b93640 1 -- 172.21.15.70:0/553151634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 msgr2=0x7f531c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.326+0000 7f5322b93640 1 --2- 172.21.15.70:0/553151634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 0x7f531c070b50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5318009870 tx=0x7f531802b910 comp rx=0 tx=0).stop 2023-10-08T05:38:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.326+0000 7f5322b93640 1 -- 172.21.15.70:0/553151634 shutdown_connections 2023-10-08T05:38:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.326+0000 7f5322b93640 1 --2- 172.21.15.70:0/553151634 >> v2:172.21.15.181:3300/0 conn(0x7f531c0caef0 0x7f531c0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.326+0000 7f5322b93640 1 --2- 172.21.15.70:0/553151634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 0x7f531c070b50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.326+0000 7f5322b93640 1 -- 172.21.15.70:0/553151634 >> 172.21.15.70:0/553151634 conn(0x7f531c06c010 msgr2=0x7f531c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.326+0000 7f5322b93640 1 -- 172.21.15.70:0/553151634 shutdown_connections 2023-10-08T05:38:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.326+0000 7f5322b93640 1 -- 172.21.15.70:0/553151634 wait complete. 2023-10-08T05:38:06.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.328+0000 7f5322b93640 1 Processor -- start 2023-10-08T05:38:06.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.329+0000 7f5322b93640 1 -- start start 2023-10-08T05:38:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5322b93640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531c0b04a0 0x7f531c148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5322b93640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0caef0 0x7f531c15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5322b93640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 0x7f531c0acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5322b93640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c15dee0 con 0x7f531c15aab0 2023-10-08T05:38:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5322b93640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c0c7880 con 0x7f531c0caef0 2023-10-08T05:38:06.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5322b93640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531c0c7b60 con 0x7f531c0b04a0 2023-10-08T05:38:06.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5322392640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 0x7f531c0acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5322392640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 0x7f531c0acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43160/0 (socket says 172.21.15.70:43160) 2023-10-08T05:38:06.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5322392640 1 -- 172.21.15.70:0/3883530719 learned_addr learned my addr 172.21.15.70:0/3883530719 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:06.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5321b91640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531c0b04a0 0x7f531c148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.330+0000 7f5321390640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0caef0 0x7f531c15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f5322392640 1 -- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531c0b04a0 msgr2=0x7f531c148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f5322392640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531c0b04a0 0x7f531c148770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f5322392640 1 -- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0caef0 msgr2=0x7f531c15d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f5322392640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0caef0 0x7f531c15d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f5322392640 1 -- 172.21.15.70:0/3883530719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53180094d0 con 0x7f531c15aab0 2023-10-08T05:38:06.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f5321390640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0caef0 0x7f531c15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:38:06.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f5322392640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 0x7f531c0acda0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f531400c7a0 tx=0x7f531400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:06.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f53127fc640 1 -- 172.21.15.70:0/3883530719 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314003b70 con 0x7f531c15aab0 2023-10-08T05:38:06.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f53127fc640 1 -- 172.21.15.70:0/3883530719 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314003cf0 con 0x7f531c15aab0 2023-10-08T05:38:06.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f53127fc640 1 -- 172.21.15.70:0/3883530719 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314003ea0 con 0x7f531c15aab0 2023-10-08T05:38:06.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f5322b93640 1 -- 172.21.15.70:0/3883530719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c0c7e40 con 0x7f531c15aab0 2023-10-08T05:38:06.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.331+0000 7f5322b93640 1 -- 172.21.15.70:0/3883530719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c0c8400 con 0x7f531c15aab0 2023-10-08T05:38:06.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.333+0000 7f5322b93640 1 -- 172.21.15.70:0/3883530719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c0c85c0 con 0x7f531c15aab0 2023-10-08T05:38:06.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.334+0000 7f53127fc640 1 -- 172.21.15.70:0/3883530719 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 9) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f531400aa90 con 0x7f531c15aab0 2023-10-08T05:38:06.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.335+0000 7f53127fc640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f530006afe0 0x7f530006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.335+0000 7f53127fc640 1 -- 172.21.15.70:0/3883530719 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f5314089680 con 0x7f531c15aab0 2023-10-08T05:38:06.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.335+0000 7f5321b91640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f530006afe0 0x7f530006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.336+0000 7f5321b91640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f530006afe0 0x7f530006d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5318036000 tx=0x7f531803c040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:06.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.340+0000 7f53127fc640 1 -- 172.21.15.70:0/3883530719 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53140543a0 con 0x7f531c15aab0 2023-10-08T05:38:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.551+0000 7f5322b93640 1 -- 172.21.15.70:0/3883530719 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir2", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f531c0b1460 con 0x7f530006afe0 2023-10-08T05:38:06.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.678+0000 7f53127fc640 1 -- 172.21.15.70:0/3883530719 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5318031b40 con 0x7f530006afe0 2023-10-08T05:38:06.679 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/dir2 2023-10-08T05:38:06.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.681+0000 7f5322b93640 1 -- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f530006afe0 msgr2=0x7f530006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f530006afe0 0x7f530006d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5318036000 tx=0x7f531803c040 comp rx=0 tx=0).stop 2023-10-08T05:38:06.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 -- 172.21.15.70:0/3883530719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 msgr2=0x7f531c0acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 0x7f531c0acda0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f531400c7a0 tx=0x7f531400cc60 comp rx=0 tx=0).stop 2023-10-08T05:38:06.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 -- 172.21.15.70:0/3883530719 shutdown_connections 2023-10-08T05:38:06.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f531c15aab0 0x7f531c0acda0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f531c0caef0 0x7f531c15d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f530006afe0 0x7f530006d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 --2- 172.21.15.70:0/3883530719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f531c0b04a0 0x7f531c148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 -- 172.21.15.70:0/3883530719 >> 172.21.15.70:0/3883530719 conn(0x7f531c06c010 msgr2=0x7f531c070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:06.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 -- 172.21.15.70:0/3883530719 shutdown_connections 2023-10-08T05:38:06.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.682+0000 7f5322b93640 1 -- 172.21.15.70:0/3883530719 wait complete. 2023-10-08T05:38:06.703 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-10-08T05:38:06.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:06.703 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-10-08T05:38:06.759 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir3 1m 2023-10-08T05:38:06.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.975+0000 7fa7ecb9e640 1 Processor -- start 2023-10-08T05:38:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.975+0000 7fa7ecb9e640 1 -- start start 2023-10-08T05:38:06.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.976+0000 7fa7ecb9e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa7e815aad0 0x7fa7e80b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.976+0000 7fa7ecb9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 0x7fa7e80c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.976+0000 7fa7ecb9e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e815bf30 con 0x7fa7e806cd00 2023-10-08T05:38:06.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.976+0000 7fa7ecb9e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e815b320 con 0x7fa7e815aad0 2023-10-08T05:38:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.976+0000 7fa7ecb9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e815b600 con 0x7fa7e80c82b0 2023-10-08T05:38:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7e6d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa7e806cd00 0x7fa7e8070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36256/0 (socket says 172.21.15.70:36256) 2023-10-08T05:38:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7e6d9d640 1 -- 172.21.15.70:0/793913005 learned_addr learned my addr 172.21.15.70:0/793913005 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7e659c640 1 --2- 172.21.15.70:0/793913005 >> v2:172.21.15.70:3300/0 conn(0x7fa7e815aad0 0x7fa7e80b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7dffff640 1 --2- 172.21.15.70:0/793913005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 0x7fa7e80c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7e5d9b640 1 -- 172.21.15.70:0/793913005 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135215832 0 0) 0x7fa7e815bf30 con 0x7fa7e806cd00 2023-10-08T05:38:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7e5d9b640 1 -- 172.21.15.70:0/793913005 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7d4003520 con 0x7fa7e806cd00 2023-10-08T05:38:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7e659c640 1 -- 172.21.15.70:0/793913005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 msgr2=0x7fa7e80c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7e659c640 1 --2- 172.21.15.70:0/793913005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 0x7fa7e80c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7e659c640 1 -- 172.21.15.70:0/793913005 >> v1:172.21.15.70:6789/0 conn(0x7fa7e806cd00 legacy=0x7fa7e8070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7e659c640 1 -- 172.21.15.70:0/793913005 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e815b8e0 con 0x7fa7e815aad0 2023-10-08T05:38:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.977+0000 7fa7dffff640 1 --2- 172.21.15.70:0/793913005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 0x7fa7e80c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.978+0000 7fa7e659c640 1 --2- 172.21.15.70:0/793913005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e815aad0 0x7fa7e80b1750 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80097a0 tx=0x7fa7d802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95106a3e7d1f1603 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.979+0000 7fa7e5d9b640 1 -- 172.21.15.70:0/793913005 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d802d040 con 0x7fa7e815aad0 2023-10-08T05:38:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.979+0000 7fa7e5d9b640 1 -- 172.21.15.70:0/793913005 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8031810 con 0x7fa7e815aad0 2023-10-08T05:38:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.979+0000 7fa7e5d9b640 1 -- 172.21.15.70:0/793913005 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8031ba0 con 0x7fa7e815aad0 2023-10-08T05:38:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.979+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/793913005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e815aad0 msgr2=0x7fa7e80b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.979+0000 7fa7ecb9e640 1 --2- 172.21.15.70:0/793913005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e815aad0 0x7fa7e80b1750 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80097a0 tx=0x7fa7d802b8b0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.979+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/793913005 shutdown_connections 2023-10-08T05:38:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.979+0000 7fa7ecb9e640 1 --2- 172.21.15.70:0/793913005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 0x7fa7e80c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.979+0000 7fa7ecb9e640 1 --2- 172.21.15.70:0/793913005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e815aad0 0x7fa7e80b1750 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.979+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/793913005 >> 172.21.15.70:0/793913005 conn(0x7fa7e8069d50 msgr2=0x7fa7e806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:06.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.980+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/793913005 shutdown_connections 2023-10-08T05:38:06.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.980+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/793913005 wait complete. 2023-10-08T05:38:06.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.981+0000 7fa7ecb9e640 1 Processor -- start 2023-10-08T05:38:06.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.981+0000 7fa7ecb9e640 1 -- start start 2023-10-08T05:38:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.982+0000 7fa7ecb9e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e806cd00 0x7fa7e8158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.982+0000 7fa7ecb9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 0x7fa7e815e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.982+0000 7fa7ecb9e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7e815aad0 0x7fa7e815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.982+0000 7fa7ecb9e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80c4520 con 0x7fa7e806cd00 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.982+0000 7fa7ecb9e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e81594c0 con 0x7fa7e815aad0 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.982+0000 7fa7ecb9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e81597a0 con 0x7fa7e80c82b0 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.983+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e806cd00 0x7fa7e8158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.983+0000 7fa7e659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e806cd00 0x7fa7e8158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43188/0 (socket says 172.21.15.70:43188) 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.983+0000 7fa7e659c640 1 -- 172.21.15.70:0/2967390349 learned_addr learned my addr 172.21.15.70:0/2967390349 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.983+0000 7fa7dffff640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 0x7fa7e815e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.983+0000 7fa7e659c640 1 -- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 msgr2=0x7fa7e815e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.983+0000 7fa7e6d9d640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7e815aad0 0x7fa7e815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.983+0000 7fa7e659c640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 0x7fa7e815e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.983+0000 7fa7e659c640 1 -- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7e815aad0 msgr2=0x7fa7e815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.983+0000 7fa7e659c640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7e815aad0 0x7fa7e815e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.983+0000 7fa7e659c640 1 -- 172.21.15.70:0/2967390349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d8009400 con 0x7fa7e806cd00 2023-10-08T05:38:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.984+0000 7fa7e659c640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e806cd00 0x7fa7e8158bc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8000c00 tx=0x7fa7d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.984+0000 7fa7deffd640 1 -- 172.21.15.70:0/2967390349 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8006e10 con 0x7fa7e806cd00 2023-10-08T05:38:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.984+0000 7fa7deffd640 1 -- 172.21.15.70:0/2967390349 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d803ad60 con 0x7fa7e806cd00 2023-10-08T05:38:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.984+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/2967390349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e8159a80 con 0x7fa7e806cd00 2023-10-08T05:38:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.984+0000 7fa7deffd640 1 -- 172.21.15.70:0/2967390349 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8044630 con 0x7fa7e806cd00 2023-10-08T05:38:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.984+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/2967390349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e8159d60 con 0x7fa7e806cd00 2023-10-08T05:38:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.985+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/2967390349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e804e160 con 0x7fa7e806cd00 2023-10-08T05:38:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.986+0000 7fa7deffd640 1 -- 172.21.15.70:0/2967390349 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 9) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fa7d804d450 con 0x7fa7e806cd00 2023-10-08T05:38:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.987+0000 7fa7deffd640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa7c806afe0 0x7fa7c806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.987+0000 7fa7deffd640 1 -- 172.21.15.70:0/2967390349 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fa7d80abdb0 con 0x7fa7e806cd00 2023-10-08T05:38:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.987+0000 7fa7dffff640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa7c806afe0 0x7fa7c806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:06.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.989+0000 7fa7dffff640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa7c806afe0 0x7fa7c806d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80caf10 tx=0x7fa7d0008040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:06.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:06.991+0000 7fa7deffd640 1 -- 172.21.15.70:0/2967390349 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7d80b0070 con 0x7fa7e806cd00 2023-10-08T05:38:07.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.205+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/2967390349 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir3", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7e815bbc0 con 0x7fa7c806afe0 2023-10-08T05:38:07.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.364+0000 7fa7deffd640 1 -- 172.21.15.70:0/2967390349 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa7d000c040 con 0x7fa7c806afe0 2023-10-08T05:38:07.365 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/dir3 2023-10-08T05:38:07.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa7c806afe0 msgr2=0x7fa7c806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:07.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa7c806afe0 0x7fa7c806d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80caf10 tx=0x7fa7d0008040 comp rx=0 tx=0).stop 2023-10-08T05:38:07.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/2967390349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e806cd00 msgr2=0x7fa7e8158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:07.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e806cd00 0x7fa7e8158bc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8000c00 tx=0x7fa7d80046c0 comp rx=0 tx=0).stop 2023-10-08T05:38:07.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/2967390349 shutdown_connections 2023-10-08T05:38:07.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa7e815aad0 0x7fa7e815e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:07.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa7c806afe0 0x7fa7c806d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:07.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7e80c82b0 0x7fa7e815e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:07.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 --2- 172.21.15.70:0/2967390349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7e806cd00 0x7fa7e8158bc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:07.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/2967390349 >> 172.21.15.70:0/2967390349 conn(0x7fa7e8069d50 msgr2=0x7fa7e80b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:07.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/2967390349 shutdown_connections 2023-10-08T05:38:07.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.368+0000 7fa7ecb9e640 1 -- 172.21.15.70:0/2967390349 wait complete. 2023-10-08T05:38:07.388 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-10-08T05:38:07.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:07.388 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-10-08T05:38:07.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir4 1m 2023-10-08T05:38:07.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.653+0000 7f8ba4924640 1 Processor -- start 2023-10-08T05:38:07.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.653+0000 7f8ba4924640 1 -- start start 2023-10-08T05:38:07.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.654+0000 7f8ba4924640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba00cfd40 0x7f8ba0071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:07.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.654+0000 7f8ba4924640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8ba01609a0 0x7f8ba015c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:07.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.654+0000 7f8ba4924640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba00ab8d0 con 0x7f8ba00d0120 2023-10-08T05:38:07.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.654+0000 7f8ba4924640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba00719a0 con 0x7f8ba01609a0 2023-10-08T05:38:07.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.654+0000 7f8ba4924640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba0071c80 con 0x7f8ba00cfd40 2023-10-08T05:38:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.655+0000 7f8b9e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8ba01609a0 0x7f8ba015c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.655+0000 7f8b9e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8ba01609a0 0x7f8ba015c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43202/0 (socket says 172.21.15.70:43202) 2023-10-08T05:38:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.655+0000 7f8b9e59c640 1 -- 172.21.15.70:0/1016625044 learned_addr learned my addr 172.21.15.70:0/1016625044 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.655+0000 7f8b9ed9d640 1 --2- 172.21.15.70:0/1016625044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba00cfd40 0x7f8ba0071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.655+0000 7f8b9e59c640 1 -- 172.21.15.70:0/1016625044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba00cfd40 msgr2=0x7f8ba0071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.655+0000 7f8b9e59c640 1 --2- 172.21.15.70:0/1016625044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba00cfd40 0x7f8ba0071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.655+0000 7f8b9e59c640 1 -- 172.21.15.70:0/1016625044 >> v1:172.21.15.181:6789/0 conn(0x7f8ba00d0120 legacy=0x7f8ba015cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:07.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.655+0000 7f8b9e59c640 1 -- 172.21.15.70:0/1016625044 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0071f60 con 0x7f8ba01609a0 2023-10-08T05:38:07.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.656+0000 7f8b9e59c640 1 --2- 172.21.15.70:0/1016625044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba01609a0 0x7f8ba015c720 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009870 tx=0x7f8b8802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbdaa77f31316d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:07.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.657+0000 7f8b9dd9b640 1 -- 172.21.15.70:0/1016625044 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b880317c0 con 0x7f8ba01609a0 2023-10-08T05:38:07.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.657+0000 7f8b9dd9b640 1 -- 172.21.15.70:0/1016625044 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b88031920 con 0x7f8ba01609a0 2023-10-08T05:38:07.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.657+0000 7f8b9dd9b640 1 -- 172.21.15.70:0/1016625044 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88031cb0 con 0x7f8ba01609a0 2023-10-08T05:38:07.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.658+0000 7f8ba4924640 1 -- 172.21.15.70:0/1016625044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba01609a0 msgr2=0x7f8ba015c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:07.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.658+0000 7f8ba4924640 1 --2- 172.21.15.70:0/1016625044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba01609a0 0x7f8ba015c720 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009870 tx=0x7f8b8802b860 comp rx=0 tx=0).stop 2023-10-08T05:38:07.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.658+0000 7f8ba4924640 1 -- 172.21.15.70:0/1016625044 shutdown_connections 2023-10-08T05:38:07.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.658+0000 7f8ba4924640 1 --2- 172.21.15.70:0/1016625044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba01609a0 0x7f8ba015c720 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:07.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.658+0000 7f8ba4924640 1 --2- 172.21.15.70:0/1016625044 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba00cfd40 0x7f8ba0071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:07.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.658+0000 7f8ba4924640 1 -- 172.21.15.70:0/1016625044 >> 172.21.15.70:0/1016625044 conn(0x7f8ba006b250 msgr2=0x7f8ba006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:07.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.658+0000 7f8ba4924640 1 -- 172.21.15.70:0/1016625044 shutdown_connections 2023-10-08T05:38:07.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.658+0000 7f8ba4924640 1 -- 172.21.15.70:0/1016625044 wait complete. 2023-10-08T05:38:07.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.660+0000 7f8ba4924640 1 Processor -- start 2023-10-08T05:38:07.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.660+0000 7f8ba4924640 1 -- start start 2023-10-08T05:38:07.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.661+0000 7f8ba4924640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba00cfd40 0x7f8ba015b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:07.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.661+0000 7f8ba4924640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba00d0120 0x7f8ba00c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:07.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.661+0000 7f8ba4924640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba01609a0 0x7f8ba00c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:07.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.661+0000 7f8ba4924640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba00b4100 con 0x7f8ba00cfd40 2023-10-08T05:38:07.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.661+0000 7f8ba4924640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba015bec0 con 0x7f8ba00d0120 2023-10-08T05:38:07.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.661+0000 7f8ba4924640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba00c7810 con 0x7f8ba01609a0 2023-10-08T05:38:07.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.662+0000 7f8b9ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba00cfd40 0x7f8ba015b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:07.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.662+0000 7f8b9ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba00cfd40 0x7f8ba015b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43204/0 (socket says 172.21.15.70:43204) 2023-10-08T05:38:07.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.662+0000 7f8b9ed9d640 1 -- 172.21.15.70:0/3969049308 learned_addr learned my addr 172.21.15.70:0/3969049308 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:07.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.662+0000 7f8b9e59c640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba00d0120 0x7f8ba00c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:07.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.662+0000 7f8b9ed9d640 1 -- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba01609a0 msgr2=0x7f8ba00c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:07.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.662+0000 7f8b9ed9d640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba01609a0 0x7f8ba00c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:07.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.662+0000 7f8b9ed9d640 1 -- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba00d0120 msgr2=0x7f8ba00c30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:07.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.662+0000 7f8b9ed9d640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba00d0120 0x7f8ba00c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:07.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.662+0000 7f8b9ed9d640 1 -- 172.21.15.70:0/3969049308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b880094d0 con 0x7f8ba00cfd40 2023-10-08T05:38:07.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.663+0000 7f8b9ed9d640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba00cfd40 0x7f8ba015b980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000ccd0 tx=0x7f8b9000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:07.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.663+0000 7f8b977fe640 1 -- 172.21.15.70:0/3969049308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90003990 con 0x7f8ba00cfd40 2023-10-08T05:38:07.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.663+0000 7f8b977fe640 1 -- 172.21.15.70:0/3969049308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b90003af0 con 0x7f8ba00cfd40 2023-10-08T05:38:07.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.663+0000 7f8b977fe640 1 -- 172.21.15.70:0/3969049308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b90004020 con 0x7f8ba00cfd40 2023-10-08T05:38:07.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.663+0000 7f8ba4924640 1 -- 172.21.15.70:0/3969049308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba00c7ab0 con 0x7f8ba00cfd40 2023-10-08T05:38:07.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.663+0000 7f8ba4924640 1 -- 172.21.15.70:0/3969049308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba00c8070 con 0x7f8ba00cfd40 2023-10-08T05:38:07.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.664+0000 7f8ba4924640 1 -- 172.21.15.70:0/3969049308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba00c8260 con 0x7f8ba00cfd40 2023-10-08T05:38:07.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.665+0000 7f8b977fe640 1 -- 172.21.15.70:0/3969049308 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 9) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f8b90004180 con 0x7f8ba00cfd40 2023-10-08T05:38:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.667+0000 7f8b977fe640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f8b8006afe0 0x7f8b8006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.667+0000 7f8b977fe640 1 -- 172.21.15.70:0/3969049308 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f8b90089f40 con 0x7f8ba00cfd40 2023-10-08T05:38:07.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.667+0000 7f8b9e59c640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f8b8006afe0 0x7f8b8006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:07.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.669+0000 7f8b9e59c640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f8b8006afe0 0x7f8b8006d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8b88036000 tx=0x7f8b8803d040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.671+0000 7f8b977fe640 1 -- 172.21.15.70:0/3969049308 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b90054b60 con 0x7f8ba00cfd40 2023-10-08T05:38:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:07.886+0000 7f8ba4924640 1 -- 172.21.15.70:0/3969049308 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir4", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ba0072240 con 0x7f8b8006afe0 2023-10-08T05:38:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.043+0000 7f8b977fe640 1 -- 172.21.15.70:0/3969049308 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f8b88031c10 con 0x7f8b8006afe0 2023-10-08T05:38:08.044 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/dir4 2023-10-08T05:38:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.047+0000 7f8ba4924640 1 -- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f8b8006afe0 msgr2=0x7f8b8006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.047+0000 7f8ba4924640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f8b8006afe0 0x7f8b8006d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8b88036000 tx=0x7f8b8803d040 comp rx=0 tx=0).stop 2023-10-08T05:38:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.047+0000 7f8ba4924640 1 -- 172.21.15.70:0/3969049308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba00cfd40 msgr2=0x7f8ba015b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.047+0000 7f8ba4924640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba00cfd40 0x7f8ba015b980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000ccd0 tx=0x7f8b9000a5c0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.047+0000 7f8ba4924640 1 -- 172.21.15.70:0/3969049308 shutdown_connections 2023-10-08T05:38:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.047+0000 7f8ba4924640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f8b8006afe0 0x7f8b8006d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.047+0000 7f8ba4924640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8ba01609a0 0x7f8ba00c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.047+0000 7f8ba4924640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8ba00d0120 0x7f8ba00c30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.047+0000 7f8ba4924640 1 --2- 172.21.15.70:0/3969049308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ba00cfd40 0x7f8ba015b980 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.047+0000 7f8ba4924640 1 -- 172.21.15.70:0/3969049308 >> 172.21.15.70:0/3969049308 conn(0x7f8ba006b250 msgr2=0x7f8ba015cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.048+0000 7f8ba4924640 1 -- 172.21.15.70:0/3969049308 shutdown_connections 2023-10-08T05:38:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.048+0000 7f8ba4924640 1 -- 172.21.15.70:0/3969049308 wait complete. 2023-10-08T05:38:08.070 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-10-08T05:38:08.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:08.070 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-10-08T05:38:08.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir5 1m 2023-10-08T05:38:08.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.350+0000 7f6c1f0c2640 1 Processor -- start 2023-10-08T05:38:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.350+0000 7f6c1f0c2640 1 -- start start 2023-10-08T05:38:08.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.352+0000 7f6c1f0c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6c180cb880 0x7f6c180b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.352+0000 7f6c1f0c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c180b4070 0x7f6c18159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.352+0000 7f6c1f0c2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1815d440 con 0x7f6c181432f0 2023-10-08T05:38:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.352+0000 7f6c1f0c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c181436d0 con 0x7f6c180cb880 2023-10-08T05:38:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.352+0000 7f6c1f0c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1815d690 con 0x7f6c180b4070 2023-10-08T05:38:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.352+0000 7f6c17fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6c180cb880 0x7f6c180b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.352+0000 7f6c1ce5e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6c181432f0 0x7f6c180cb170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36268/0 (socket says 172.21.15.70:36268) 2023-10-08T05:38:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.352+0000 7f6c1ce5e640 1 -- 172.21.15.70:0/2669684905 learned_addr learned my addr 172.21.15.70:0/2669684905 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:08.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.353+0000 7f6c1d65f640 1 --2- 172.21.15.70:0/2669684905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c180b4070 0x7f6c18159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:08.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.353+0000 7f6c177fe640 1 -- 172.21.15.70:0/2669684905 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826510232 0 0) 0x7f6c1815d440 con 0x7f6c181432f0 2023-10-08T05:38:08.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.353+0000 7f6c177fe640 1 -- 172.21.15.70:0/2669684905 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c08003520 con 0x7f6c181432f0 2023-10-08T05:38:08.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.353+0000 7f6c17fff640 1 -- 172.21.15.70:0/2669684905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c180b4070 msgr2=0x7f6c18159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:08.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.353+0000 7f6c17fff640 1 --2- 172.21.15.70:0/2669684905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c180b4070 0x7f6c18159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.353+0000 7f6c17fff640 1 -- 172.21.15.70:0/2669684905 >> v1:172.21.15.70:6789/0 conn(0x7f6c181432f0 legacy=0x7f6c180cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:08.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.353+0000 7f6c17fff640 1 -- 172.21.15.70:0/2669684905 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1815d970 con 0x7f6c180cb880 2023-10-08T05:38:08.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.354+0000 7f6c17fff640 1 --2- 172.21.15.70:0/2669684905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c180cb880 0x7f6c180b3b30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6c040097a0 tx=0x7f6c0402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8d7dca3c455d07 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:08.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.354+0000 7f6c177fe640 1 -- 172.21.15.70:0/2669684905 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04031780 con 0x7f6c180cb880 2023-10-08T05:38:08.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.354+0000 7f6c177fe640 1 -- 172.21.15.70:0/2669684905 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c040318e0 con 0x7f6c180cb880 2023-10-08T05:38:08.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.354+0000 7f6c177fe640 1 -- 172.21.15.70:0/2669684905 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04031c50 con 0x7f6c180cb880 2023-10-08T05:38:08.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.355+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2669684905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c180cb880 msgr2=0x7f6c180b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:08.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.355+0000 7f6c1f0c2640 1 --2- 172.21.15.70:0/2669684905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c180cb880 0x7f6c180b3b30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6c040097a0 tx=0x7f6c0402b820 comp rx=0 tx=0).stop 2023-10-08T05:38:08.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.355+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2669684905 shutdown_connections 2023-10-08T05:38:08.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.355+0000 7f6c1f0c2640 1 --2- 172.21.15.70:0/2669684905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c180b4070 0x7f6c18159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.355+0000 7f6c1f0c2640 1 --2- 172.21.15.70:0/2669684905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c180cb880 0x7f6c180b3b30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.355+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2669684905 >> 172.21.15.70:0/2669684905 conn(0x7f6c18069d50 msgr2=0x7f6c1806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:08.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.355+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2669684905 shutdown_connections 2023-10-08T05:38:08.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.355+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2669684905 wait complete. 2023-10-08T05:38:08.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.357+0000 7f6c1f0c2640 1 Processor -- start 2023-10-08T05:38:08.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.357+0000 7f6c1f0c2640 1 -- start start 2023-10-08T05:38:08.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.358+0000 7f6c1f0c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c180b4070 0x7f6c1813d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.358+0000 7f6c1f0c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c180cb880 0x7f6c18159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.358+0000 7f6c1f0c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c181432f0 0x7f6c1815b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.358+0000 7f6c1f0c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1815ce50 con 0x7f6c181432f0 2023-10-08T05:38:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.358+0000 7f6c1f0c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1815a650 con 0x7f6c180b4070 2023-10-08T05:38:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.358+0000 7f6c1f0c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1815a930 con 0x7f6c180cb880 2023-10-08T05:38:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.358+0000 7f6c1d65f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c181432f0 0x7f6c1815b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.358+0000 7f6c1d65f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c181432f0 0x7f6c1815b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43224/0 (socket says 172.21.15.70:43224) 2023-10-08T05:38:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.358+0000 7f6c1d65f640 1 -- 172.21.15.70:0/2626946237 learned_addr learned my addr 172.21.15.70:0/2626946237 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c1ce5e640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c180b4070 0x7f6c1813d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c17fff640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c180cb880 0x7f6c18159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c1d65f640 1 -- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c180cb880 msgr2=0x7f6c18159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c1d65f640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c180cb880 0x7f6c18159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c1d65f640 1 -- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c180b4070 msgr2=0x7f6c1813d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c1d65f640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c180b4070 0x7f6c1813d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c1d65f640 1 -- 172.21.15.70:0/2626946237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04009400 con 0x7f6c181432f0 2023-10-08T05:38:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c1ce5e640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c180b4070 0x7f6c1813d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c1d65f640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c181432f0 0x7f6c1815b6f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6c10007e40 tx=0x7f6c1000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c157fa640 1 -- 172.21.15.70:0/2626946237 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10014620 con 0x7f6c181432f0 2023-10-08T05:38:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c157fa640 1 -- 172.21.15.70:0/2626946237 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1000ee90 con 0x7f6c181432f0 2023-10-08T05:38:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c157fa640 1 -- 172.21.15.70:0/2626946237 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c100149e0 con 0x7f6c181432f0 2023-10-08T05:38:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2626946237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1815ac10 con 0x7f6c181432f0 2023-10-08T05:38:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.359+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2626946237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1815b1d0 con 0x7f6c181432f0 2023-10-08T05:38:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.360+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2626946237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1815b360 con 0x7f6c181432f0 2023-10-08T05:38:08.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.361+0000 7f6c157fa640 1 -- 172.21.15.70:0/2626946237 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f6c10014b40 con 0x7f6c181432f0 2023-10-08T05:38:08.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.362+0000 7f6c157fa640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f6bfc06b070 0x7f6bfc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:08.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.362+0000 7f6c157fa640 1 -- 172.21.15.70:0/2626946237 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f6c1008a5b0 con 0x7f6c181432f0 2023-10-08T05:38:08.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.363+0000 7f6c1ce5e640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f6bfc06b070 0x7f6bfc06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:08.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.364+0000 7f6c1ce5e640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f6bfc06b070 0x7f6bfc06d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c005880 tx=0x7f6c0c009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:08.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.367+0000 7f6c157fa640 1 -- 172.21.15.70:0/2626946237 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c10055170 con 0x7f6c181432f0 2023-10-08T05:38:08.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.577+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2626946237 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir5", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c1815df30 con 0x7f6bfc06b070 2023-10-08T05:38:08.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.733+0000 7f6c157fa640 1 -- 172.21.15.70:0/2626946237 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6c0c001350 con 0x7f6bfc06b070 2023-10-08T05:38:08.734 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/dir5 2023-10-08T05:38:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.737+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f6bfc06b070 msgr2=0x7f6bfc06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.737+0000 7f6c1f0c2640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f6bfc06b070 0x7f6bfc06d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c005880 tx=0x7f6c0c009040 comp rx=0 tx=0).stop 2023-10-08T05:38:08.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.737+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2626946237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c181432f0 msgr2=0x7f6c1815b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:08.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.737+0000 7f6c1f0c2640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c181432f0 0x7f6c1815b6f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6c10007e40 tx=0x7f6c1000e4b0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.738+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2626946237 shutdown_connections 2023-10-08T05:38:08.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.738+0000 7f6c1f0c2640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c181432f0 0x7f6c1815b6f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.738+0000 7f6c1f0c2640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f6bfc06b070 0x7f6bfc06d530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.738+0000 7f6c1f0c2640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c180cb880 0x7f6c18159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.738+0000 7f6c1f0c2640 1 --2- 172.21.15.70:0/2626946237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c180b4070 0x7f6c1813d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:08.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.738+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2626946237 >> 172.21.15.70:0/2626946237 conn(0x7f6c18069d50 msgr2=0x7f6c180cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:08.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.738+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2626946237 shutdown_connections 2023-10-08T05:38:08.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:08.738+0000 7f6c1f0c2640 1 -- 172.21.15.70:0/2626946237 wait complete. 2023-10-08T05:38:08.760 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-10-08T05:38:08.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:08.761 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-10-08T05:38:08.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir6 1m 2023-10-08T05:38:09.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.027+0000 7fbdc7c00640 1 Processor -- start 2023-10-08T05:38:09.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.028+0000 7fbdc7c00640 1 -- start start 2023-10-08T05:38:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.029+0000 7fbdc7c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbdc006d410 0x7fbdc0070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.029+0000 7fbdc7c00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc015bf30 con 0x7fbdc015aad0 2023-10-08T05:38:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.029+0000 7fbdc7c00640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc00b1bb0 con 0x7fbdc00b1750 2023-10-08T05:38:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.029+0000 7fbdc7c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc015b320 con 0x7fbdc006d410 2023-10-08T05:38:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.030+0000 7fbdc599c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbdc015aad0 0x7fbdc00c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36274/0 (socket says 172.21.15.70:36274) 2023-10-08T05:38:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.030+0000 7fbdc599c640 1 -- 172.21.15.70:0/1173736160 learned_addr learned my addr 172.21.15.70:0/1173736160 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.030+0000 7fbdc619d640 1 --2- 172.21.15.70:0/1173736160 >> v2:172.21.15.70:3300/0 conn(0x7fbdc006d410 0x7fbdc0070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.030+0000 7fbdc499a640 1 -- 172.21.15.70:0/1173736160 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014124939 0 0) 0x7fbdc015bf30 con 0x7fbdc015aad0 2023-10-08T05:38:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.030+0000 7fbdc499a640 1 -- 172.21.15.70:0/1173736160 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdac003520 con 0x7fbdc015aad0 2023-10-08T05:38:09.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.030+0000 7fbdc619d640 1 -- 172.21.15.70:0/1173736160 >> v1:172.21.15.181:6789/0 conn(0x7fbdc00b1750 legacy=0x7fbdc006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:09.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.030+0000 7fbdc619d640 1 -- 172.21.15.70:0/1173736160 >> v1:172.21.15.70:6789/0 conn(0x7fbdc015aad0 legacy=0x7fbdc00c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:09.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.030+0000 7fbdc619d640 1 -- 172.21.15.70:0/1173736160 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc015b600 con 0x7fbdc006d410 2023-10-08T05:38:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.031+0000 7fbdc619d640 1 --2- 172.21.15.70:0/1173736160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 0x7fbdc0070570 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009d90 tx=0x7fbdb802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cc42e7fb486b2d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.032+0000 7fbdc499a640 1 -- 172.21.15.70:0/1173736160 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb802cc60 con 0x7fbdc006d410 2023-10-08T05:38:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.032+0000 7fbdc499a640 1 -- 172.21.15.70:0/1173736160 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb802cdc0 con 0x7fbdc006d410 2023-10-08T05:38:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.032+0000 7fbdc499a640 1 -- 172.21.15.70:0/1173736160 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8034660 con 0x7fbdc006d410 2023-10-08T05:38:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.032+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1173736160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 msgr2=0x7fbdc0070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.032+0000 7fbdc7c00640 1 --2- 172.21.15.70:0/1173736160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 0x7fbdc0070570 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009d90 tx=0x7fbdb802beb0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.032+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1173736160 shutdown_connections 2023-10-08T05:38:09.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.032+0000 7fbdc7c00640 1 --2- 172.21.15.70:0/1173736160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 0x7fbdc0070570 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.032+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1173736160 >> 172.21.15.70:0/1173736160 conn(0x7fbdc0069d50 msgr2=0x7fbdc006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:09.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.032+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1173736160 shutdown_connections 2023-10-08T05:38:09.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.033+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1173736160 wait complete. 2023-10-08T05:38:09.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.034+0000 7fbdc7c00640 1 Processor -- start 2023-10-08T05:38:09.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.034+0000 7fbdc7c00640 1 -- start start 2023-10-08T05:38:09.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.035+0000 7fbdc7c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 0x7fbdc01f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.035+0000 7fbdc7c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdc00b1750 0x7fbdc00c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.035+0000 7fbdc7c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdc015aad0 0x7fbdc00c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.035+0000 7fbdc7c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc0158e20 con 0x7fbdc006d410 2023-10-08T05:38:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.035+0000 7fbdc7c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc01f0280 con 0x7fbdc00b1750 2023-10-08T05:38:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.035+0000 7fbdc7c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc01581a0 con 0x7fbdc015aad0 2023-10-08T05:38:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.036+0000 7fbdc599c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 0x7fbdc01f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.036+0000 7fbdc599c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 0x7fbdc01f2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43240/0 (socket says 172.21.15.70:43240) 2023-10-08T05:38:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.036+0000 7fbdc599c640 1 -- 172.21.15.70:0/1750476497 learned_addr learned my addr 172.21.15.70:0/1750476497 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.036+0000 7fbdc599c640 1 -- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdc015aad0 msgr2=0x7fbdc00c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:09.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.036+0000 7fbdc599c640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdc015aad0 0x7fbdc00c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.036+0000 7fbdc599c640 1 -- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdc00b1750 msgr2=0x7fbdc00c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:09.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.036+0000 7fbdc519b640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdc00b1750 0x7fbdc00c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.036+0000 7fbdc599c640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdc00b1750 0x7fbdc00c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.036+0000 7fbdc599c640 1 -- 172.21.15.70:0/1750476497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8009a70 con 0x7fbdc006d410 2023-10-08T05:38:09.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.036+0000 7fbdc599c640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 0x7fbdc01f2c20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00c4d0 tx=0x7fbdbc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.037+0000 7fbdb27fc640 1 -- 172.21.15.70:0/1750476497 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc005ce0 con 0x7fbdc006d410 2023-10-08T05:38:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.037+0000 7fbdb27fc640 1 -- 172.21.15.70:0/1750476497 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc005e40 con 0x7fbdc006d410 2023-10-08T05:38:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.037+0000 7fbdb27fc640 1 -- 172.21.15.70:0/1750476497 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc007270 con 0x7fbdc006d410 2023-10-08T05:38:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.037+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1750476497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc0158480 con 0x7fbdc006d410 2023-10-08T05:38:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.037+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1750476497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc0158a40 con 0x7fbdc006d410 2023-10-08T05:38:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.038+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1750476497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc004e160 con 0x7fbdc006d410 2023-10-08T05:38:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.039+0000 7fbdb27fc640 1 -- 172.21.15.70:0/1750476497 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fbdbc0073d0 con 0x7fbdc006d410 2023-10-08T05:38:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.040+0000 7fbdb27fc640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fbda406b070 0x7fbda406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.040+0000 7fbdb27fc640 1 -- 172.21.15.70:0/1750476497 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fbdbc0890b0 con 0x7fbdc006d410 2023-10-08T05:38:09.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.040+0000 7fbdc519b640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fbda406b070 0x7fbda406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.042+0000 7fbdc519b640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fbda406b070 0x7fbda406d530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00b1090 tx=0x7fbdb4009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:09.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.044+0000 7fbdb27fc640 1 -- 172.21.15.70:0/1750476497 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdbc053d20 con 0x7fbdc006d410 2023-10-08T05:38:09.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.253+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1750476497 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir6", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdc015bbc0 con 0x7fbda406b070 2023-10-08T05:38:09.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.408+0000 7fbdb27fc640 1 -- 172.21.15.70:0/1750476497 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fbdb4003980 con 0x7fbda406b070 2023-10-08T05:38:09.409 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/dir6 2023-10-08T05:38:09.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fbda406b070 msgr2=0x7fbda406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:09.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fbda406b070 0x7fbda406d530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00b1090 tx=0x7fbdb4009040 comp rx=0 tx=0).stop 2023-10-08T05:38:09.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1750476497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 msgr2=0x7fbdc01f2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:09.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 0x7fbdc01f2c20 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00c4d0 tx=0x7fbdbc00c990 comp rx=0 tx=0).stop 2023-10-08T05:38:09.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1750476497 shutdown_connections 2023-10-08T05:38:09.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fbda406b070 0x7fbda406d530 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdc015aad0 0x7fbdc00c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdc00b1750 0x7fbdc00c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 --2- 172.21.15.70:0/1750476497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdc006d410 0x7fbdc01f2c20 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1750476497 >> 172.21.15.70:0/1750476497 conn(0x7fbdc0069d50 msgr2=0x7fbdc00cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:09.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.411+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1750476497 shutdown_connections 2023-10-08T05:38:09.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.412+0000 7fbdc7c00640 1 -- 172.21.15.70:0/1750476497 wait complete. 2023-10-08T05:38:09.433 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-10-08T05:38:09.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:09.434 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-10-08T05:38:09.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir7 1m 2023-10-08T05:38:09.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.701+0000 7fcbce390640 1 Processor -- start 2023-10-08T05:38:09.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.701+0000 7fcbce390640 1 -- start start 2023-10-08T05:38:09.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.703+0000 7fcbce390640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcbc815a960 0x7fcbc806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.703+0000 7fcbce390640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc80709c0 0x7fcbc80ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.703+0000 7fcbce390640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc806dac0 con 0x7fcbc80cec00 2023-10-08T05:38:09.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.703+0000 7fcbce390640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc815e540 con 0x7fcbc815a960 2023-10-08T05:38:09.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.703+0000 7fcbce390640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80cf090 con 0x7fcbc80709c0 2023-10-08T05:38:09.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.703+0000 7fcbcc92d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcbc80cec00 0x7fcbc80cd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36290/0 (socket says 172.21.15.70:36290) 2023-10-08T05:38:09.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.703+0000 7fcbcc92d640 1 -- 172.21.15.70:0/1873039457 learned_addr learned my addr 172.21.15.70:0/1873039457 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc7fff640 1 --2- 172.21.15.70:0/1873039457 >> v2:172.21.15.181:3300/0 conn(0x7fcbc815a960 0x7fcbc806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbbffff640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc80709c0 0x7fcbc80ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382197319 0 0) 0x7fcbc806dac0 con 0x7fcbc80cec00 2023-10-08T05:38:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbb4003520 con 0x7fcbc80cec00 2023-10-08T05:38:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4197326992 0 0) 0x7fcbb4003520 con 0x7fcbc80cec00 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc80709c0 msgr2=0x7fcbc80ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc80709c0 0x7fcbc80ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 >> v2:172.21.15.181:3300/0 conn(0x7fcbc815a960 msgr2=0x7fcbc806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 --2- 172.21.15.70:0/1873039457 >> v2:172.21.15.181:3300/0 conn(0x7fcbc815a960 0x7fcbc806c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc806ce30 con 0x7fcbc80cec00 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fcbc00019f0 con 0x7fcbc80cec00 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 >> v1:172.21.15.70:6789/0 conn(0x7fcbc80cec00 legacy=0x7fcbc80cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb4003970 0x7fcbb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb40047c0 0x7fcbb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc80709c0 0x7fcbb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc80cf090 con 0x7fcbb4003970 2023-10-08T05:38:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb400fd90 con 0x7fcbb40047c0 2023-10-08T05:38:09.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.704+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb400fed0 con 0x7fcbc80709c0 2023-10-08T05:38:09.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.705+0000 7fcbc7fff640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb4003970 0x7fcbb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.705+0000 7fcbbffff640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb40047c0 0x7fcbb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.705+0000 7fcbc7fff640 1 -- 172.21.15.70:0/1873039457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc80709c0 msgr2=0x7fcbb4005180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T05:38:09.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.705+0000 7fcbc7fff640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc80709c0 0x7fcbb4005180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.705+0000 7fcbc7fff640 1 -- 172.21.15.70:0/1873039457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb40047c0 msgr2=0x7fcbb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:09.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.705+0000 7fcbc7fff640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb40047c0 0x7fcbb4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.705+0000 7fcbc7fff640 1 -- 172.21.15.70:0/1873039457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb4003520 con 0x7fcbb4003970 2023-10-08T05:38:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.707+0000 7fcbc7fff640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb4003970 0x7fcbb4003d90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009ac0 tx=0x7fcbb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdcfa3721eb18c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.707+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb802c6d0 con 0x7fcbb4003970 2023-10-08T05:38:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.707+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb802c830 con 0x7fcbb4003970 2023-10-08T05:38:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.707+0000 7fcbc77fe640 1 -- 172.21.15.70:0/1873039457 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb802cbc0 con 0x7fcbb4003970 2023-10-08T05:38:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.707+0000 7fcbce390640 1 -- 172.21.15.70:0/1873039457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb4003970 msgr2=0x7fcbb4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.707+0000 7fcbce390640 1 --2- 172.21.15.70:0/1873039457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbb4003970 0x7fcbb4003d90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009ac0 tx=0x7fcbb40057f0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.707+0000 7fcbcc92d640 1 -- 172.21.15.70:0/1873039457 reap_dead start 2023-10-08T05:38:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.708+0000 7fcbce390640 1 -- 172.21.15.70:0/1873039457 shutdown_connections 2023-10-08T05:38:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.708+0000 7fcbce390640 1 -- 172.21.15.70:0/1873039457 >> 172.21.15.70:0/1873039457 conn(0x7fcbc80662a0 msgr2=0x7fcbc8069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.708+0000 7fcbce390640 1 -- 172.21.15.70:0/1873039457 shutdown_connections 2023-10-08T05:38:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.708+0000 7fcbce390640 1 -- 172.21.15.70:0/1873039457 wait complete. 2023-10-08T05:38:09.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.709+0000 7fcbce390640 1 Processor -- start 2023-10-08T05:38:09.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.710+0000 7fcbce390640 1 -- start start 2023-10-08T05:38:09.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.710+0000 7fcbce390640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb4003970 0x7fcbc814f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbce390640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc8146720 0x7fcbc814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbce390640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc815da80 0x7fcbc814a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbce390640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc814e4d0 con 0x7fcbc8146720 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbce390640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc814e7b0 con 0x7fcbb4003970 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbce390640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc814ea90 con 0x7fcbc815da80 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc8146720 0x7fcbc814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc8146720 0x7fcbc814dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43268/0 (socket says 172.21.15.70:43268) 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbbffff640 1 -- 172.21.15.70:0/2364933180 learned_addr learned my addr 172.21.15.70:0/2364933180 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbc7fff640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb4003970 0x7fcbc814f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbbffff640 1 -- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc815da80 msgr2=0x7fcbc814a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbbffff640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc815da80 0x7fcbc814a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbbffff640 1 -- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb4003970 msgr2=0x7fcbc814f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbbffff640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb4003970 0x7fcbc814f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.711+0000 7fcbbffff640 1 -- 172.21.15.70:0/2364933180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc814ed70 con 0x7fcbc8146720 2023-10-08T05:38:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.712+0000 7fcbbffff640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc8146720 0x7fcbc814dc80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0011600 tx=0x7fcbb0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.712+0000 7fcbc57fa640 1 -- 172.21.15.70:0/2364933180 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000f920 con 0x7fcbc8146720 2023-10-08T05:38:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.712+0000 7fcbc57fa640 1 -- 172.21.15.70:0/2364933180 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb000fa80 con 0x7fcbc8146720 2023-10-08T05:38:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.712+0000 7fcbc57fa640 1 -- 172.21.15.70:0/2364933180 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000fdf0 con 0x7fcbc8146720 2023-10-08T05:38:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.712+0000 7fcbce390640 1 -- 172.21.15.70:0/2364933180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc814f050 con 0x7fcbc8146720 2023-10-08T05:38:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.712+0000 7fcbce390640 1 -- 172.21.15.70:0/2364933180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc814e1c0 con 0x7fcbc8146720 2023-10-08T05:38:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.713+0000 7fcbce390640 1 -- 172.21.15.70:0/2364933180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc804e160 con 0x7fcbc8146720 2023-10-08T05:38:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.714+0000 7fcbc57fa640 1 -- 172.21.15.70:0/2364933180 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fcbb0015460 con 0x7fcbc8146720 2023-10-08T05:38:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.715+0000 7fcbc57fa640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fcba806b0a0 0x7fcba806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.715+0000 7fcbc57fa640 1 -- 172.21.15.70:0/2364933180 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fcbb008a140 con 0x7fcbc8146720 2023-10-08T05:38:09.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.716+0000 7fcbc7fff640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fcba806b0a0 0x7fcba806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:09.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.717+0000 7fcbc7fff640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fcba806b0a0 0x7fcba806d560 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8036000 tx=0x7fcbb8030000 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:09.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.719+0000 7fcbc57fa640 1 -- 172.21.15.70:0/2364933180 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcbb0054c70 con 0x7fcbc8146720 2023-10-08T05:38:09.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:09.932+0000 7fcbce390640 1 -- 172.21.15.70:0/2364933180 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir7", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbc814e920 con 0x7fcba806b0a0 2023-10-08T05:38:10.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.091+0000 7fcbc57fa640 1 -- 172.21.15.70:0/2364933180 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fcbc814e920 con 0x7fcba806b0a0 2023-10-08T05:38:10.092 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/dir7 2023-10-08T05:38:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 -- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fcba806b0a0 msgr2=0x7fcba806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fcba806b0a0 0x7fcba806d560 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8036000 tx=0x7fcbb8030000 comp rx=0 tx=0).stop 2023-10-08T05:38:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 -- 172.21.15.70:0/2364933180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc8146720 msgr2=0x7fcbc814dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc8146720 0x7fcbc814dc80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0011600 tx=0x7fcbb0011ac0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 -- 172.21.15.70:0/2364933180 shutdown_connections 2023-10-08T05:38:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fcba806b0a0 0x7fcba806d560 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbc815da80 0x7fcbc814a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc8146720 0x7fcbc814dc80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 --2- 172.21.15.70:0/2364933180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbb4003970 0x7fcbc814f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 -- 172.21.15.70:0/2364933180 >> 172.21.15.70:0/2364933180 conn(0x7fcbc80662a0 msgr2=0x7fcbc8152390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 -- 172.21.15.70:0/2364933180 shutdown_connections 2023-10-08T05:38:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.095+0000 7fcbce390640 1 -- 172.21.15.70:0/2364933180 wait complete. 2023-10-08T05:38:10.116 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-10-08T05:38:10.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:10.117 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-10-08T05:38:10.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir8 1m 2023-10-08T05:38:10.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.386+0000 7f0667410640 1 Processor -- start 2023-10-08T05:38:10.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.387+0000 7f0667410640 1 -- start start 2023-10-08T05:38:10.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.388+0000 7f0667410640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06600cd930 0x7f06600cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:10.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.388+0000 7f0667410640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06600d0d30 0x7f06600d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.388+0000 7f0667410640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06600b7e70 con 0x7f066015cca0 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.388+0000 7f0667410640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066015d080 con 0x7f06600d0d30 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.388+0000 7f0667410640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06600b7260 con 0x7f06600cd930 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.389+0000 7f06659ad640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06600d0d30 0x7f06600d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.389+0000 7f06659ad640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06600d0d30 0x7f06600d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43278/0 (socket says 172.21.15.70:43278) 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.389+0000 7f06659ad640 1 -- 172.21.15.70:0/154538884 learned_addr learned my addr 172.21.15.70:0/154538884 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.389+0000 7f06649ab640 1 --2- 172.21.15.70:0/154538884 >> v2:172.21.15.181:3300/0 conn(0x7f06600cd930 0x7f06600cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.389+0000 7f06659ad640 1 -- 172.21.15.70:0/154538884 >> v2:172.21.15.181:3300/0 conn(0x7f06600cd930 msgr2=0x7f06600cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.389+0000 7f06659ad640 1 --2- 172.21.15.70:0/154538884 >> v2:172.21.15.181:3300/0 conn(0x7f06600cd930 0x7f06600cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.389+0000 7f06659ad640 1 -- 172.21.15.70:0/154538884 >> v1:172.21.15.181:6789/0 conn(0x7f066015cca0 legacy=0x7f06600b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:10.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.389+0000 7f06659ad640 1 -- 172.21.15.70:0/154538884 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06600b7540 con 0x7f06600d0d30 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.390+0000 7f06659ad640 1 --2- 172.21.15.70:0/154538884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600d0d30 0x7f06600d11a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f06600c9ce0 tx=0x7f065802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c560dd631f4d0291 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f065ffff640 1 -- 172.21.15.70:0/154538884 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658030860 con 0x7f06600d0d30 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f065ffff640 1 -- 172.21.15.70:0/154538884 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06580309c0 con 0x7f06600d0d30 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f065ffff640 1 -- 172.21.15.70:0/154538884 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658030d50 con 0x7f06600d0d30 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f0667410640 1 -- 172.21.15.70:0/154538884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600d0d30 msgr2=0x7f06600d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f0667410640 1 --2- 172.21.15.70:0/154538884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600d0d30 0x7f06600d11a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f06600c9ce0 tx=0x7f065802b9a0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f0667410640 1 -- 172.21.15.70:0/154538884 shutdown_connections 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f0667410640 1 --2- 172.21.15.70:0/154538884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600d0d30 0x7f06600d11a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f0667410640 1 --2- 172.21.15.70:0/154538884 >> v2:172.21.15.181:3300/0 conn(0x7f06600cd930 0x7f06600cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f0667410640 1 -- 172.21.15.70:0/154538884 >> 172.21.15.70:0/154538884 conn(0x7f06600718a0 msgr2=0x7f0660071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:10.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f0667410640 1 -- 172.21.15.70:0/154538884 shutdown_connections 2023-10-08T05:38:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.391+0000 7f0667410640 1 -- 172.21.15.70:0/154538884 wait complete. 2023-10-08T05:38:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.393+0000 7f0667410640 1 Processor -- start 2023-10-08T05:38:10.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.393+0000 7f0667410640 1 -- start start 2023-10-08T05:38:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.394+0000 7f0667410640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06600cd930 0x7f066014e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.394+0000 7f0667410640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06600d0d30 0x7f066014a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:10.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.394+0000 7f0667410640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015cca0 0x7f066014ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.394+0000 7f0667410640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066014e660 con 0x7f066015cca0 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.394+0000 7f0667410640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066015eb10 con 0x7f06600cd930 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.394+0000 7f0667410640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066015edf0 con 0x7f06600d0d30 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06659ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015cca0 0x7f066014ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06659ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015cca0 0x7f066014ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43294/0 (socket says 172.21.15.70:43294) 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06659ad640 1 -- 172.21.15.70:0/1413057896 learned_addr learned my addr 172.21.15.70:0/1413057896 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06651ac640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06600cd930 0x7f066014e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06649ab640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06600d0d30 0x7f066014a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06659ad640 1 -- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06600d0d30 msgr2=0x7f066014a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06659ad640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06600d0d30 0x7f066014a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06659ad640 1 -- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06600cd930 msgr2=0x7f066014e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:10.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06659ad640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06600cd930 0x7f066014e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06659ad640 1 -- 172.21.15.70:0/1413057896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06580094d0 con 0x7f066015cca0 2023-10-08T05:38:10.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.395+0000 7f06659ad640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015cca0 0x7f066014ad40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f06580090b0 tx=0x7f0658004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:10.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.396+0000 7f065dffb640 1 -- 172.21.15.70:0/1413057896 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658039c90 con 0x7f066015cca0 2023-10-08T05:38:10.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.396+0000 7f065dffb640 1 -- 172.21.15.70:0/1413057896 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0658039df0 con 0x7f066015cca0 2023-10-08T05:38:10.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.396+0000 7f065dffb640 1 -- 172.21.15.70:0/1413057896 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658043640 con 0x7f066015cca0 2023-10-08T05:38:10.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.396+0000 7f0667410640 1 -- 172.21.15.70:0/1413057896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066015f0d0 con 0x7f066015cca0 2023-10-08T05:38:10.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.396+0000 7f0667410640 1 -- 172.21.15.70:0/1413057896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0660148af0 con 0x7f066015cca0 2023-10-08T05:38:10.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.397+0000 7f0667410640 1 -- 172.21.15.70:0/1413057896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0660148c80 con 0x7f066015cca0 2023-10-08T05:38:10.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.398+0000 7f065dffb640 1 -- 172.21.15.70:0/1413057896 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f06580437a0 con 0x7f066015cca0 2023-10-08T05:38:10.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.399+0000 7f065dffb640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f064c06b0a0 0x7f064c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:10.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.399+0000 7f065dffb640 1 -- 172.21.15.70:0/1413057896 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f065803e070 con 0x7f066015cca0 2023-10-08T05:38:10.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.399+0000 7f06651ac640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f064c06b0a0 0x7f064c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:10.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.401+0000 7f06651ac640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f064c06b0a0 0x7f064c06d560 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0650009f20 tx=0x7f06500094d0 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:10.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.403+0000 7f065dffb640 1 -- 172.21.15.70:0/1413057896 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0658077920 con 0x7f066015cca0 2023-10-08T05:38:10.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.614+0000 7f0667410640 1 -- 172.21.15.70:0/1413057896 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir8", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f06600b7b00 con 0x7f064c06b0a0 2023-10-08T05:38:10.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.777+0000 7f065dffb640 1 -- 172.21.15.70:0/1413057896 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f0650009680 con 0x7f064c06b0a0 2023-10-08T05:38:10.778 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/dir8 2023-10-08T05:38:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.780+0000 7f0667410640 1 -- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f064c06b0a0 msgr2=0x7f064c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.780+0000 7f0667410640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f064c06b0a0 0x7f064c06d560 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0650009f20 tx=0x7f06500094d0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.780+0000 7f0667410640 1 -- 172.21.15.70:0/1413057896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015cca0 msgr2=0x7f066014ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.780+0000 7f0667410640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015cca0 0x7f066014ad40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f06580090b0 tx=0x7f0658004730 comp rx=0 tx=0).stop 2023-10-08T05:38:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.781+0000 7f0667410640 1 -- 172.21.15.70:0/1413057896 shutdown_connections 2023-10-08T05:38:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.781+0000 7f0667410640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015cca0 0x7f066014ad40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.781+0000 7f0667410640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f064c06b0a0 0x7f064c06d560 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.781+0000 7f0667410640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06600d0d30 0x7f066014a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.781+0000 7f0667410640 1 --2- 172.21.15.70:0/1413057896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06600cd930 0x7f066014e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.781+0000 7f0667410640 1 -- 172.21.15.70:0/1413057896 >> 172.21.15.70:0/1413057896 conn(0x7f06600718a0 msgr2=0x7f066015bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.781+0000 7f0667410640 1 -- 172.21.15.70:0/1413057896 shutdown_connections 2023-10-08T05:38:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:10.781+0000 7f0667410640 1 -- 172.21.15.70:0/1413057896 wait complete. 2023-10-08T05:38:10.801 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-10-08T05:38:10.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:38:10.801 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-10-08T05:38:10.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir9 1m 2023-10-08T05:38:11.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.061+0000 7f42b08e0640 1 Processor -- start 2023-10-08T05:38:11.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.062+0000 7f42b08e0640 1 -- start start 2023-10-08T05:38:11.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.063+0000 7f42b08e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f42ac070b50 0x7f42ac0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.063+0000 7f42b08e0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f42ac0b04a0 0x7f42ac0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.063+0000 7f42b08e0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ac15ae90 con 0x7f42ac15aab0 2023-10-08T05:38:11.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.063+0000 7f42b08e0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ac0cb600 con 0x7f42ac070b50 2023-10-08T05:38:11.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.063+0000 7f42b08e0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ac0b0930 con 0x7f42ac0b04a0 2023-10-08T05:38:11.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.064+0000 7f42a3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f42ac070b50 0x7f42ac0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.064+0000 7f42a3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f42ac070b50 0x7f42ac0aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43304/0 (socket says 172.21.15.70:43304) 2023-10-08T05:38:11.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.064+0000 7f42a3fff640 1 -- 172.21.15.70:0/3827845123 learned_addr learned my addr 172.21.15.70:0/3827845123 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:11.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.064+0000 7f42ab59e640 1 --2- 172.21.15.70:0/3827845123 >> v2:172.21.15.181:3300/0 conn(0x7f42ac0b04a0 0x7f42ac0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.064+0000 7f42a3fff640 1 -- 172.21.15.70:0/3827845123 >> v2:172.21.15.181:3300/0 conn(0x7f42ac0b04a0 msgr2=0x7f42ac0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:11.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.064+0000 7f42a3fff640 1 --2- 172.21.15.70:0/3827845123 >> v2:172.21.15.181:3300/0 conn(0x7f42ac0b04a0 0x7f42ac0b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.064+0000 7f42a3fff640 1 -- 172.21.15.70:0/3827845123 >> v1:172.21.15.181:6789/0 conn(0x7f42ac15aab0 legacy=0x7f42ac0caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:11.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.064+0000 7f42a3fff640 1 -- 172.21.15.70:0/3827845123 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ac0cbdb0 con 0x7f42ac070b50 2023-10-08T05:38:11.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.065+0000 7f42a3fff640 1 --2- 172.21.15.70:0/3827845123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac070b50 0x7f42ac0aff60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4294009870 tx=0x7f429402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c83e7e8bdcfb071 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.065+0000 7f42aa59c640 1 -- 172.21.15.70:0/3827845123 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42940317e0 con 0x7f42ac070b50 2023-10-08T05:38:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.066+0000 7f42aa59c640 1 -- 172.21.15.70:0/3827845123 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4294031940 con 0x7f42ac070b50 2023-10-08T05:38:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.066+0000 7f42aa59c640 1 -- 172.21.15.70:0/3827845123 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294031cd0 con 0x7f42ac070b50 2023-10-08T05:38:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.066+0000 7f42b08e0640 1 -- 172.21.15.70:0/3827845123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac070b50 msgr2=0x7f42ac0aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.066+0000 7f42b08e0640 1 --2- 172.21.15.70:0/3827845123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac070b50 0x7f42ac0aff60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4294009870 tx=0x7f429402b880 comp rx=0 tx=0).stop 2023-10-08T05:38:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.066+0000 7f42b08e0640 1 -- 172.21.15.70:0/3827845123 shutdown_connections 2023-10-08T05:38:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.066+0000 7f42b08e0640 1 --2- 172.21.15.70:0/3827845123 >> v2:172.21.15.181:3300/0 conn(0x7f42ac0b04a0 0x7f42ac0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.066+0000 7f42b08e0640 1 --2- 172.21.15.70:0/3827845123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac070b50 0x7f42ac0aff60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.066+0000 7f42b08e0640 1 -- 172.21.15.70:0/3827845123 >> 172.21.15.70:0/3827845123 conn(0x7f42ac06c010 msgr2=0x7f42ac06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.066+0000 7f42b08e0640 1 -- 172.21.15.70:0/3827845123 shutdown_connections 2023-10-08T05:38:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.066+0000 7f42b08e0640 1 -- 172.21.15.70:0/3827845123 wait complete. 2023-10-08T05:38:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.068+0000 7f42b08e0640 1 Processor -- start 2023-10-08T05:38:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.068+0000 7f42b08e0640 1 -- start start 2023-10-08T05:38:11.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.069+0000 7f42b08e0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42ac070b50 0x7f42ac1f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.069+0000 7f42b08e0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42ac0b04a0 0x7f42ac15ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.069+0000 7f42b08e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac15aab0 0x7f42ac0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.069+0000 7f42b08e0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1f1f00 con 0x7f42ac15aab0 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.069+0000 7f42b08e0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0abdd0 con 0x7f42ac070b50 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.069+0000 7f42b08e0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac0ac0b0 con 0x7f42ac0b04a0 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42ab59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac15aab0 0x7f42ac0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42ab59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac15aab0 0x7f42ac0ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43316/0 (socket says 172.21.15.70:43316) 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42ab59e640 1 -- 172.21.15.70:0/451584947 learned_addr learned my addr 172.21.15.70:0/451584947 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42a3fff640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42ac0b04a0 0x7f42ac15ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42aad9d640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42ac070b50 0x7f42ac1f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42ab59e640 1 -- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42ac0b04a0 msgr2=0x7f42ac15ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:11.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42ab59e640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42ac0b04a0 0x7f42ac15ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42ab59e640 1 -- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42ac070b50 msgr2=0x7f42ac1f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:11.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42ab59e640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42ac070b50 0x7f42ac1f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42ab59e640 1 -- 172.21.15.70:0/451584947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42940094d0 con 0x7f42ac15aab0 2023-10-08T05:38:11.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.070+0000 7f42ab59e640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac15aab0 0x7f42ac0ccae0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f42a4007e40 tx=0x7f42a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:11.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.071+0000 7f42a37fe640 1 -- 172.21.15.70:0/451584947 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4014620 con 0x7f42ac15aab0 2023-10-08T05:38:11.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.071+0000 7f42a37fe640 1 -- 172.21.15.70:0/451584947 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a400ee90 con 0x7f42ac15aab0 2023-10-08T05:38:11.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.071+0000 7f42b08e0640 1 -- 172.21.15.70:0/451584947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac0ac390 con 0x7f42ac15aab0 2023-10-08T05:38:11.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.071+0000 7f42a37fe640 1 -- 172.21.15.70:0/451584947 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40149e0 con 0x7f42ac15aab0 2023-10-08T05:38:11.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.071+0000 7f42b08e0640 1 -- 172.21.15.70:0/451584947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac0ac950 con 0x7f42ac15aab0 2023-10-08T05:38:11.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.072+0000 7f42b08e0640 1 -- 172.21.15.70:0/451584947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ac0acb40 con 0x7f42ac15aab0 2023-10-08T05:38:11.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.073+0000 7f42a37fe640 1 -- 172.21.15.70:0/451584947 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f42a4014b40 con 0x7f42ac15aab0 2023-10-08T05:38:11.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.074+0000 7f42a37fe640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f428c06b0a0 0x7f428c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.075+0000 7f42a37fe640 1 -- 172.21.15.70:0/451584947 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f42a408a5b0 con 0x7f42ac15aab0 2023-10-08T05:38:11.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.075+0000 7f42aad9d640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f428c06b0a0 0x7f428c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.076+0000 7f42aad9d640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f428c06b0a0 0x7f428c06d560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f429c006fd0 tx=0x7f429c008040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.079+0000 7f42a37fe640 1 -- 172.21.15.70:0/451584947 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42a4055170 con 0x7f42ac15aab0 2023-10-08T05:38:11.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.288+0000 7f42b08e0640 1 -- 172.21.15.70:0/451584947 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir9", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f42ac0cbad0 con 0x7f428c06b0a0 2023-10-08T05:38:11.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.436+0000 7f42a37fe640 1 -- 172.21.15.70:0/451584947 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f429c00bd80 con 0x7f428c06b0a0 2023-10-08T05:38:11.437 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/dir9 2023-10-08T05:38:11.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.439+0000 7f42b08e0640 1 -- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f428c06b0a0 msgr2=0x7f428c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:11.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f428c06b0a0 0x7f428c06d560 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f429c006fd0 tx=0x7f429c008040 comp rx=0 tx=0).stop 2023-10-08T05:38:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 -- 172.21.15.70:0/451584947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac15aab0 msgr2=0x7f42ac0ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac15aab0 0x7f42ac0ccae0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f42a4007e40 tx=0x7f42a400e4b0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 -- 172.21.15.70:0/451584947 shutdown_connections 2023-10-08T05:38:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac15aab0 0x7f42ac0ccae0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f428c06b0a0 0x7f428c06d560 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42ac0b04a0 0x7f42ac15ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 --2- 172.21.15.70:0/451584947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42ac070b50 0x7f42ac1f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 -- 172.21.15.70:0/451584947 >> 172.21.15.70:0/451584947 conn(0x7f42ac06c010 msgr2=0x7f42ac0add00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 -- 172.21.15.70:0/451584947 shutdown_connections 2023-10-08T05:38:11.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.440+0000 7f42b08e0640 1 -- 172.21.15.70:0/451584947 wait complete. 2023-10-08T05:38:11.461 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-10-08T05:38:11.461 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir0 1m 2023-10-08T05:38:11.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.668+0000 7fe379128640 1 Processor -- start 2023-10-08T05:38:11.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.668+0000 7fe379128640 1 -- start start 2023-10-08T05:38:11.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.670+0000 7fe379128640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3741599d0 0x7fe3740cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.670+0000 7fe379128640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe374159db0 con 0x7fe3740b3c50 2023-10-08T05:38:11.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.670+0000 7fe379128640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe374071fe0 con 0x7fe3740cb5e0 2023-10-08T05:38:11.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.670+0000 7fe379128640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740b44c0 con 0x7fe3741599d0 2023-10-08T05:38:11.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.670+0000 7fe36bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe3740b3c50 0x7fe3740caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36306/0 (socket says 172.21.15.70:36306) 2023-10-08T05:38:11.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.670+0000 7fe36bfff640 1 -- 172.21.15.70:0/1245924200 learned_addr learned my addr 172.21.15.70:0/1245924200 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:11.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.670+0000 7fe373fff640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3741599d0 0x7fe3740cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.670+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510276077 0 0) 0x7fe35c001110 con 0x7fe3740b3c50 2023-10-08T05:38:11.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.670+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe360003520 con 0x7fe3740b3c50 2023-10-08T05:38:11.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2795926964 0 0) 0x7fe360003520 con 0x7fe3740b3c50 2023-10-08T05:38:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3741599d0 msgr2=0x7fe3740cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3741599d0 0x7fe3740cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 >> v1:172.21.15.181:6789/0 conn(0x7fe3740cb5e0 legacy=0x7fe3740cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3740713d0 con 0x7fe3740b3c50 2023-10-08T05:38:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe35c001800 con 0x7fe3740b3c50 2023-10-08T05:38:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 >> v1:172.21.15.70:6789/0 conn(0x7fe3740b3c50 legacy=0x7fe3740caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 0x7fe360003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3741599d0 0x7fe3600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe360004d00 0x7fe360005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740b44c0 con 0x7fe3600039a0 2023-10-08T05:38:11.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36000dc70 con 0x7fe360004d00 2023-10-08T05:38:11.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36000de30 con 0x7fe3741599d0 2023-10-08T05:38:11.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.671+0000 7fe373fff640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 0x7fe360003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.672+0000 7fe36bfff640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3741599d0 0x7fe3600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.672+0000 7fe373fff640 1 -- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3741599d0 msgr2=0x7fe3600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:11.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.672+0000 7fe378927640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe360004d00 0x7fe360005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.672+0000 7fe373fff640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3741599d0 0x7fe3600047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.672+0000 7fe373fff640 1 -- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe360004d00 msgr2=0x7fe360005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:11.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.672+0000 7fe373fff640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe360004d00 0x7fe360005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.672+0000 7fe373fff640 1 -- 172.21.15.70:0/1245924200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360003520 con 0x7fe3600039a0 2023-10-08T05:38:11.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.674+0000 7fe373fff640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 0x7fe360003dc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe364009a90 tx=0x7fe3600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5952fdbf4a0c4c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:11.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.674+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36402c680 con 0x7fe3600039a0 2023-10-08T05:38:11.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.674+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36402c7e0 con 0x7fe3600039a0 2023-10-08T05:38:11.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.674+0000 7fe3737fe640 1 -- 172.21.15.70:0/1245924200 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36402cb70 con 0x7fe3600039a0 2023-10-08T05:38:11.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.674+0000 7fe379128640 1 -- 172.21.15.70:0/1245924200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 msgr2=0x7fe360003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:11.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.674+0000 7fe379128640 1 --2- 172.21.15.70:0/1245924200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 0x7fe360003dc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe364009a90 tx=0x7fe3600057f0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.674+0000 7fe378927640 1 -- 172.21.15.70:0/1245924200 reap_dead start 2023-10-08T05:38:11.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.675+0000 7fe379128640 1 -- 172.21.15.70:0/1245924200 shutdown_connections 2023-10-08T05:38:11.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.675+0000 7fe379128640 1 -- 172.21.15.70:0/1245924200 >> 172.21.15.70:0/1245924200 conn(0x7fe37406c050 msgr2=0x7fe37406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:11.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.675+0000 7fe379128640 1 -- 172.21.15.70:0/1245924200 shutdown_connections 2023-10-08T05:38:11.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.675+0000 7fe379128640 1 -- 172.21.15.70:0/1245924200 wait complete. 2023-10-08T05:38:11.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.677+0000 7fe379128640 1 Processor -- start 2023-10-08T05:38:11.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.678+0000 7fe379128640 1 -- start start 2023-10-08T05:38:11.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.679+0000 7fe379128640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 0x7fe3741ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.679+0000 7fe379128640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3740c8c80 0x7fe37415b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.679+0000 7fe379128640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3740c9100 0x7fe3741495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.679+0000 7fe379128640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3741eee10 con 0x7fe3600039a0 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.679+0000 7fe379128640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3741ef940 con 0x7fe3740c9100 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.679+0000 7fe379128640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3741ef010 con 0x7fe3740c8c80 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe373fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 0x7fe3741ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe373fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 0x7fe3741ee8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43336/0 (socket says 172.21.15.70:43336) 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe373fff640 1 -- 172.21.15.70:0/2277145226 learned_addr learned my addr 172.21.15.70:0/2277145226 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe36bfff640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3740c8c80 0x7fe37415b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe378927640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3740c9100 0x7fe3741495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe373fff640 1 -- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3740c8c80 msgr2=0x7fe37415b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:11.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe373fff640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3740c8c80 0x7fe37415b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe373fff640 1 -- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3740c9100 msgr2=0x7fe3741495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:11.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe373fff640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3740c9100 0x7fe3741495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:11.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe373fff640 1 -- 172.21.15.70:0/2277145226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3741ef2f0 con 0x7fe3600039a0 2023-10-08T05:38:11.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe36bfff640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3740c8c80 0x7fe37415b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:11.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.680+0000 7fe373fff640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 0x7fe3741ee8d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe364009a90 tx=0x7fe364004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:11.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.681+0000 7fe3717fa640 1 -- 172.21.15.70:0/2277145226 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36402dea0 con 0x7fe3600039a0 2023-10-08T05:38:11.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.681+0000 7fe3717fa640 1 -- 172.21.15.70:0/2277145226 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe364003810 con 0x7fe3600039a0 2023-10-08T05:38:11.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.681+0000 7fe379128640 1 -- 172.21.15.70:0/2277145226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3741ef5d0 con 0x7fe3600039a0 2023-10-08T05:38:11.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.681+0000 7fe3717fa640 1 -- 172.21.15.70:0/2277145226 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364003ba0 con 0x7fe3600039a0 2023-10-08T05:38:11.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.681+0000 7fe379128640 1 -- 172.21.15.70:0/2277145226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37415bed0 con 0x7fe3600039a0 2023-10-08T05:38:11.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.682+0000 7fe379128640 1 -- 172.21.15.70:0/2277145226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe374074150 con 0x7fe3600039a0 2023-10-08T05:38:11.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.684+0000 7fe3717fa640 1 -- 172.21.15.70:0/2277145226 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fe364003d00 con 0x7fe3600039a0 2023-10-08T05:38:11.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.685+0000 7fe3717fa640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe35406b070 0x7fe35406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:11.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.685+0000 7fe3717fa640 1 -- 172.21.15.70:0/2277145226 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fe3640ab940 con 0x7fe3600039a0 2023-10-08T05:38:11.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.686+0000 7fe36bfff640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe35406b070 0x7fe35406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:11.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.687+0000 7fe36bfff640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe35406b070 0x7fe35406d530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe37415cfb0 tx=0x7fe35c009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:11.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.690+0000 7fe3717fa640 1 -- 172.21.15.70:0/2277145226 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe364076510 con 0x7fe3600039a0 2023-10-08T05:38:11.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:11.906+0000 7fe379128640 1 -- 172.21.15.70:0/2277145226 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe374071990 con 0x7fe35406b070 2023-10-08T05:38:12.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.016+0000 7fe3717fa640 1 -- 172.21.15.70:0/2277145226 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe35c00d040 con 0x7fe35406b070 2023-10-08T05:38:12.017 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-10-08T05:38:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.019+0000 7fe379128640 1 -- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe35406b070 msgr2=0x7fe35406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.019+0000 7fe379128640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe35406b070 0x7fe35406d530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe37415cfb0 tx=0x7fe35c009040 comp rx=0 tx=0).stop 2023-10-08T05:38:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.019+0000 7fe379128640 1 -- 172.21.15.70:0/2277145226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 msgr2=0x7fe3741ee8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.019+0000 7fe379128640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 0x7fe3741ee8d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe364009a90 tx=0x7fe364004730 comp rx=0 tx=0).stop 2023-10-08T05:38:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.019+0000 7fe379128640 1 -- 172.21.15.70:0/2277145226 shutdown_connections 2023-10-08T05:38:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.019+0000 7fe379128640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3740c9100 0x7fe3741495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.019+0000 7fe379128640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe35406b070 0x7fe35406d530 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.019+0000 7fe379128640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3740c8c80 0x7fe37415b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.019+0000 7fe379128640 1 --2- 172.21.15.70:0/2277145226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3600039a0 0x7fe3741ee8d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.019+0000 7fe379128640 1 -- 172.21.15.70:0/2277145226 >> 172.21.15.70:0/2277145226 conn(0x7fe37406c050 msgr2=0x7fe3740c22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:12.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.020+0000 7fe379128640 1 -- 172.21.15.70:0/2277145226 shutdown_connections 2023-10-08T05:38:12.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.020+0000 7fe379128640 1 -- 172.21.15.70:0/2277145226 wait complete. 2023-10-08T05:38:12.036 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-10-08T05:38:12.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir1 1m 2023-10-08T05:38:12.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.225+0000 7f43771a9640 1 Processor -- start 2023-10-08T05:38:12.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.225+0000 7f43771a9640 1 -- start start 2023-10-08T05:38:12.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.227+0000 7f43771a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f437006db60 0x7f43700b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.227+0000 7f43771a9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437015b310 con 0x7f43700c9cf0 2023-10-08T05:38:12.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.227+0000 7f43771a9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f437006d290 con 0x7f437006c3a0 2023-10-08T05:38:12.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.227+0000 7f43771a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437015b110 con 0x7f437006db60 2023-10-08T05:38:12.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.227+0000 7f436ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f43700c9cf0 0x7f437015aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36310/0 (socket says 172.21.15.70:36310) 2023-10-08T05:38:12.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.227+0000 7f436ffff640 1 -- 172.21.15.70:0/3354105920 learned_addr learned my addr 172.21.15.70:0/3354105920 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:12.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.227+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696453623 0 0) 0x7f435c001110 con 0x7f43700c9cf0 2023-10-08T05:38:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.227+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4360003520 con 0x7f43700c9cf0 2023-10-08T05:38:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.227+0000 7f4375746640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f437006db60 0x7f43700b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3394213719 0 0) 0x7f4360003520 con 0x7f43700c9cf0 2023-10-08T05:38:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f437006db60 msgr2=0x7f43700b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f437006db60 0x7f43700b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 >> v1:172.21.15.181:6789/0 conn(0x7f437006c3a0 legacy=0x7f437006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700c9860 con 0x7f43700c9cf0 2023-10-08T05:38:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f435c001800 con 0x7f43700c9cf0 2023-10-08T05:38:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 >> v1:172.21.15.70:6789/0 conn(0x7f43700c9cf0 legacy=0x7f437015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43600039a0 0x7f4360003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 0x7f4360004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f437006db60 0x7f43600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437015b110 con 0x7f43600047c0 2023-10-08T05:38:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436000dcf0 con 0x7f43600039a0 2023-10-08T05:38:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436000deb0 con 0x7f437006db60 2023-10-08T05:38:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.228+0000 7f436ffff640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 0x7f4360004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.229+0000 7f4374f45640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43600039a0 0x7f4360003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.229+0000 7f436ffff640 1 -- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f437006db60 msgr2=0x7f43600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.229+0000 7f436ffff640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f437006db60 0x7f43600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.229+0000 7f436ffff640 1 -- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43600039a0 msgr2=0x7f4360003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.229+0000 7f436ffff640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43600039a0 0x7f4360003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.229+0000 7f436ffff640 1 -- 172.21.15.70:0/3354105920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360003520 con 0x7f43600047c0 2023-10-08T05:38:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.229+0000 7f4374f45640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43600039a0 0x7f4360003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.230+0000 7f436ffff640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 0x7f4360004c40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f435c00c190 tx=0x7f43600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2647cc52ce4ff1e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.230+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c005ad0 con 0x7f43600047c0 2023-10-08T05:38:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.230+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c005c50 con 0x7f43600047c0 2023-10-08T05:38:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.230+0000 7f436f7fe640 1 -- 172.21.15.70:0/3354105920 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c02c890 con 0x7f43600047c0 2023-10-08T05:38:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.231+0000 7f43771a9640 1 -- 172.21.15.70:0/3354105920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 msgr2=0x7f4360004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.231+0000 7f43771a9640 1 --2- 172.21.15.70:0/3354105920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 0x7f4360004c40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f435c00c190 tx=0x7f43600057f0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.231+0000 7f4375746640 1 -- 172.21.15.70:0/3354105920 reap_dead start 2023-10-08T05:38:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.231+0000 7f43771a9640 1 -- 172.21.15.70:0/3354105920 shutdown_connections 2023-10-08T05:38:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.231+0000 7f43771a9640 1 -- 172.21.15.70:0/3354105920 >> 172.21.15.70:0/3354105920 conn(0x7f437006a5d0 msgr2=0x7f43700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.231+0000 7f43771a9640 1 -- 172.21.15.70:0/3354105920 shutdown_connections 2023-10-08T05:38:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.231+0000 7f43771a9640 1 -- 172.21.15.70:0/3354105920 wait complete. 2023-10-08T05:38:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.234+0000 7f43771a9640 1 Processor -- start 2023-10-08T05:38:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.234+0000 7f43771a9640 1 -- start start 2023-10-08T05:38:12.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.235+0000 7f43771a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 0x7f43700af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.235+0000 7f43771a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4370070270 0x7f43700706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f43771a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437014b700 0x7f437014bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f43771a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4370070c30 con 0x7f43600047c0 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f43771a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43701f30a0 con 0x7f437014b700 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f43771a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437014c470 con 0x7f4370070270 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f4374f45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 0x7f43700af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f4374f45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 0x7f43700af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43362/0 (socket says 172.21.15.70:43362) 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f4374f45640 1 -- 172.21.15.70:0/2294867785 learned_addr learned my addr 172.21.15.70:0/2294867785 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f4374f45640 1 -- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4370070270 msgr2=0x7f43700706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f4374f45640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4370070270 0x7f43700706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f4374f45640 1 -- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437014b700 msgr2=0x7f437014bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f4374f45640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437014b700 0x7f437014bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f4374f45640 1 -- 172.21.15.70:0/2294867785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43701f2a50 con 0x7f43600047c0 2023-10-08T05:38:12.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.236+0000 7f4374f45640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 0x7f43700af160 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f436400d7a0 tx=0x7f436400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:12.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.237+0000 7f436d7fa640 1 -- 172.21.15.70:0/2294867785 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364005270 con 0x7f43600047c0 2023-10-08T05:38:12.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.237+0000 7f436d7fa640 1 -- 172.21.15.70:0/2294867785 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43640053f0 con 0x7f43600047c0 2023-10-08T05:38:12.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.237+0000 7f436d7fa640 1 -- 172.21.15.70:0/2294867785 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436400b8f0 con 0x7f43600047c0 2023-10-08T05:38:12.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.237+0000 7f43771a9640 1 -- 172.21.15.70:0/2294867785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43701f2d30 con 0x7f43600047c0 2023-10-08T05:38:12.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.237+0000 7f43771a9640 1 -- 172.21.15.70:0/2294867785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437014c0c0 con 0x7f43600047c0 2023-10-08T05:38:12.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.238+0000 7f43771a9640 1 -- 172.21.15.70:0/2294867785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437004e160 con 0x7f43600047c0 2023-10-08T05:38:12.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.239+0000 7f436d7fa640 1 -- 172.21.15.70:0/2294867785 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f436400ba50 con 0x7f43600047c0 2023-10-08T05:38:12.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.240+0000 7f436d7fa640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f435406b070 0x7f435406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.240+0000 7f436d7fa640 1 -- 172.21.15.70:0/2294867785 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f4364089d90 con 0x7f43600047c0 2023-10-08T05:38:12.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.240+0000 7f436ffff640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f435406b070 0x7f435406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:12.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.242+0000 7f436ffff640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f435406b070 0x7f435406d530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f435c034000 tx=0x7f435c02e000 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:12.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.245+0000 7f436d7fa640 1 -- 172.21.15.70:0/2294867785 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43640547b0 con 0x7f43600047c0 2023-10-08T05:38:12.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.448+0000 7f43771a9640 1 -- 172.21.15.70:0/2294867785 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f437006dff0 con 0x7f435406b070 2023-10-08T05:38:12.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.554+0000 7f436d7fa640 1 -- 172.21.15.70:0/2294867785 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f435c0015c0 con 0x7f435406b070 2023-10-08T05:38:12.556 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-10-08T05:38:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 -- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f435406b070 msgr2=0x7f435406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f435406b070 0x7f435406d530 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f435c034000 tx=0x7f435c02e000 comp rx=0 tx=0).stop 2023-10-08T05:38:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 -- 172.21.15.70:0/2294867785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 msgr2=0x7f43700af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 0x7f43700af160 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f436400d7a0 tx=0x7f436400dc60 comp rx=0 tx=0).stop 2023-10-08T05:38:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 -- 172.21.15.70:0/2294867785 shutdown_connections 2023-10-08T05:38:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f437014b700 0x7f437014bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f435406b070 0x7f435406d530 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4370070270 0x7f43700706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 --2- 172.21.15.70:0/2294867785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43600047c0 0x7f43700af160 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 -- 172.21.15.70:0/2294867785 >> 172.21.15.70:0/2294867785 conn(0x7f437006a5d0 msgr2=0x7f43700cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:12.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 -- 172.21.15.70:0/2294867785 shutdown_connections 2023-10-08T05:38:12.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.557+0000 7f43771a9640 1 -- 172.21.15.70:0/2294867785 wait complete. 2023-10-08T05:38:12.576 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-10-08T05:38:12.577 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir2 1m 2023-10-08T05:38:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.774+0000 7f0fd14f3640 1 Processor -- start 2023-10-08T05:38:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.774+0000 7f0fd14f3640 1 -- start start 2023-10-08T05:38:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fd14f3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fcc0cf2b0 0x7f0fcc0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fd14f3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0fcc0cf690 0x7f0fcc0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fd14f3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fcc0d1100 0x7f0fcc15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fd14f3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fcc0cd520 con 0x7f0fcc0cf2b0 2023-10-08T05:38:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fd14f3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fcc0d1910 con 0x7f0fcc0cf690 2023-10-08T05:38:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fd14f3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc0ce080 con 0x7f0fcc0d1100 2023-10-08T05:38:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fcbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fcc0cf2b0 0x7f0fcc0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fcbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fcc0cf2b0 0x7f0fcc0d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43370/0 (socket says 172.21.15.70:43370) 2023-10-08T05:38:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fcbfff640 1 -- 172.21.15.70:0/3828572497 learned_addr learned my addr 172.21.15.70:0/3828572497 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fcbfff640 1 -- 172.21.15.70:0/3828572497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fcc0d1100 msgr2=0x7f0fcc15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fcbfff640 1 --2- 172.21.15.70:0/3828572497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fcc0d1100 0x7f0fcc15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fcbfff640 1 -- 172.21.15.70:0/3828572497 >> v2:172.21.15.181:3300/0 conn(0x7f0fcc0cf690 msgr2=0x7f0fcc0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fcbfff640 1 --2- 172.21.15.70:0/3828572497 >> v2:172.21.15.181:3300/0 conn(0x7f0fcc0cf690 0x7f0fcc0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.776+0000 7f0fcbfff640 1 -- 172.21.15.70:0/3828572497 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc0cd750 con 0x7f0fcc0cf2b0 2023-10-08T05:38:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.777+0000 7f0fcbfff640 1 --2- 172.21.15.70:0/3828572497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0cf2b0 0x7f0fcc0d34d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc08baa0 tx=0x7f0fbc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba4b197447f4e15 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fcaffd640 1 -- 172.21.15.70:0/3828572497 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0317e0 con 0x7f0fcc0cf2b0 2023-10-08T05:38:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fcaffd640 1 -- 172.21.15.70:0/3828572497 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc031940 con 0x7f0fcc0cf2b0 2023-10-08T05:38:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fcaffd640 1 -- 172.21.15.70:0/3828572497 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc031cd0 con 0x7f0fcc0cf2b0 2023-10-08T05:38:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3828572497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0cf2b0 msgr2=0x7f0fcc0d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fd14f3640 1 --2- 172.21.15.70:0/3828572497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0cf2b0 0x7f0fcc0d34d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc08baa0 tx=0x7f0fbc02b880 comp rx=0 tx=0).stop 2023-10-08T05:38:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3828572497 shutdown_connections 2023-10-08T05:38:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fd14f3640 1 --2- 172.21.15.70:0/3828572497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fcc0d1100 0x7f0fcc15c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fd14f3640 1 --2- 172.21.15.70:0/3828572497 >> v2:172.21.15.181:3300/0 conn(0x7f0fcc0cf690 0x7f0fcc0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fd14f3640 1 --2- 172.21.15.70:0/3828572497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0cf2b0 0x7f0fcc0d34d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3828572497 >> 172.21.15.70:0/3828572497 conn(0x7f0fcc09bc40 msgr2=0x7f0fcc0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3828572497 shutdown_connections 2023-10-08T05:38:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.778+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3828572497 wait complete. 2023-10-08T05:38:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.780+0000 7f0fd14f3640 1 Processor -- start 2023-10-08T05:38:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.780+0000 7f0fd14f3640 1 -- start start 2023-10-08T05:38:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.781+0000 7f0fd14f3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fcc0cf2b0 0x7f0fcc09ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.781+0000 7f0fd14f3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fcc0cf690 0x7f0fcc09f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.781+0000 7f0fd14f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0d1100 0x7f0fcc0b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.781+0000 7f0fd14f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1f2130 con 0x7f0fcc0d1100 2023-10-08T05:38:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.781+0000 7f0fd14f3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1f5c20 con 0x7f0fcc0cf2b0 2023-10-08T05:38:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.781+0000 7f0fd14f3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fcc1f5f00 con 0x7f0fcc0cf690 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.781+0000 7f0fd0cf2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0d1100 0x7f0fcc0b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.781+0000 7f0fd0cf2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0d1100 0x7f0fcc0b7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43386/0 (socket says 172.21.15.70:43386) 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.781+0000 7f0fd0cf2640 1 -- 172.21.15.70:0/3407357901 learned_addr learned my addr 172.21.15.70:0/3407357901 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.782+0000 7f0fcbfff640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fcc0cf2b0 0x7f0fcc09ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.782+0000 7f0fcb7fe640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fcc0cf690 0x7f0fcc09f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.782+0000 7f0fd0cf2640 1 -- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fcc0cf690 msgr2=0x7f0fcc09f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.782+0000 7f0fd0cf2640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fcc0cf690 0x7f0fcc09f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.782+0000 7f0fd0cf2640 1 -- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fcc0cf2b0 msgr2=0x7f0fcc09ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.782+0000 7f0fd0cf2640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fcc0cf2b0 0x7f0fcc09ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.782+0000 7f0fd0cf2640 1 -- 172.21.15.70:0/3407357901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0094d0 con 0x7f0fcc0d1100 2023-10-08T05:38:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.782+0000 7f0fcbfff640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fcc0cf2b0 0x7f0fcc09ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.782+0000 7f0fd0cf2640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0d1100 0x7f0fcc0b7dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b970 tx=0x7f0fc400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.782+0000 7f0fc8ff9640 1 -- 172.21.15.70:0/3407357901 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc400cb40 con 0x7f0fcc0d1100 2023-10-08T05:38:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.783+0000 7f0fc8ff9640 1 -- 172.21.15.70:0/3407357901 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc400ccc0 con 0x7f0fcc0d1100 2023-10-08T05:38:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.783+0000 7f0fc8ff9640 1 -- 172.21.15.70:0/3407357901 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4014680 con 0x7f0fcc0d1100 2023-10-08T05:38:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.783+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3407357901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fcc1f61e0 con 0x7f0fcc0d1100 2023-10-08T05:38:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.783+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3407357901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc09d360 con 0x7f0fcc0d1100 2023-10-08T05:38:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.784+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3407357901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fcc074150 con 0x7f0fcc0d1100 2023-10-08T05:38:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.785+0000 7f0fc8ff9640 1 -- 172.21.15.70:0/3407357901 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f0fc401d3b0 con 0x7f0fcc0d1100 2023-10-08T05:38:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.786+0000 7f0fc8ff9640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f0fb806b070 0x7f0fb806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.786+0000 7f0fc8ff9640 1 -- 172.21.15.70:0/3407357901 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f0fc4089c30 con 0x7f0fcc0d1100 2023-10-08T05:38:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.787+0000 7f0fcbfff640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f0fb806b070 0x7f0fb806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.788+0000 7f0fcbfff640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f0fb806b070 0x7f0fb806d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc036000 tx=0x7f0fbc03d040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:12.790+0000 7f0fc8ff9640 1 -- 172.21.15.70:0/3407357901 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0fc4054760 con 0x7f0fcc0d1100 2023-10-08T05:38:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.028+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3407357901 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fcc0cdd10 con 0x7f0fb806b070 2023-10-08T05:38:13.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.157+0000 7f0fc8ff9640 1 -- 172.21.15.70:0/3407357901 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0fbc031c30 con 0x7f0fb806b070 2023-10-08T05:38:13.159 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-10-08T05:38:13.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.160+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f0fb806b070 msgr2=0x7f0fb806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:13.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.160+0000 7f0fd14f3640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f0fb806b070 0x7f0fb806d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc036000 tx=0x7f0fbc03d040 comp rx=0 tx=0).stop 2023-10-08T05:38:13.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.160+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3407357901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0d1100 msgr2=0x7f0fcc0b7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:13.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.160+0000 7f0fd14f3640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0d1100 0x7f0fcc0b7dc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b970 tx=0x7f0fc400be30 comp rx=0 tx=0).stop 2023-10-08T05:38:13.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.161+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3407357901 shutdown_connections 2023-10-08T05:38:13.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.161+0000 7f0fd14f3640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fcc0d1100 0x7f0fcc0b7dc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.161+0000 7f0fd14f3640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f0fb806b070 0x7f0fb806d530 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.161+0000 7f0fd14f3640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fcc0cf690 0x7f0fcc09f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.161+0000 7f0fd14f3640 1 --2- 172.21.15.70:0/3407357901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fcc0cf2b0 0x7f0fcc09ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.161+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3407357901 >> 172.21.15.70:0/3407357901 conn(0x7f0fcc09bc40 msgr2=0x7f0fcc0b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:13.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.161+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3407357901 shutdown_connections 2023-10-08T05:38:13.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.161+0000 7f0fd14f3640 1 -- 172.21.15.70:0/3407357901 wait complete. 2023-10-08T05:38:13.180 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-10-08T05:38:13.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir3 1m 2023-10-08T05:38:13.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.390+0000 7feb27fff640 1 Processor -- start 2023-10-08T05:38:13.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.391+0000 7feb27fff640 1 -- start start 2023-10-08T05:38:13.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.392+0000 7feb27fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280b0680 0x7feb2815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:13.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.392+0000 7feb27fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb280cab60 con 0x7feb2815a820 2023-10-08T05:38:13.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.392+0000 7feb27fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb280cb960 con 0x7feb280ca680 2023-10-08T05:38:13.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.392+0000 7feb27fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb280cbdf0 con 0x7feb280b0680 2023-10-08T05:38:13.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.392+0000 7feb26ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feb2815a820 0x7feb280aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36318/0 (socket says 172.21.15.70:36318) 2023-10-08T05:38:13.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.392+0000 7feb26ffd640 1 -- 172.21.15.70:0/2685065852 learned_addr learned my addr 172.21.15.70:0/2685065852 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:13.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb267fc640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280b0680 0x7feb2815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:13.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080025399 0 0) 0x7feb280cab60 con 0x7feb2815a820 2023-10-08T05:38:13.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb10003520 con 0x7feb2815a820 2023-10-08T05:38:13.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1037766350 0 0) 0x7feb10003520 con 0x7feb2815a820 2023-10-08T05:38:13.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280b0680 msgr2=0x7feb2815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:13.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb25ffb640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280b0680 0x7feb2815e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 >> v1:172.21.15.181:6789/0 conn(0x7feb280ca680 legacy=0x7feb2815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:13.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb280cb030 con 0x7feb2815a820 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7feb180019f0 con 0x7feb2815a820 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 >> v1:172.21.15.70:6789/0 conn(0x7feb2815a820 legacy=0x7feb280aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.393+0000 7feb25ffb640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb100039a0 0x7feb10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb25ffb640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280b0680 0x7feb100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb25ffb640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb10004d00 0x7feb10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb280cbdf0 con 0x7feb10004d00 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1000bb10 con 0x7feb100039a0 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb1000bcd0 con 0x7feb280b0680 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb277fe640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb10004d00 0x7feb10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb26ffd640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb100039a0 0x7feb10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:13.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb267fc640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280b0680 0x7feb100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:13.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb277fe640 1 -- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280b0680 msgr2=0x7feb100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:13.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb277fe640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280b0680 0x7feb100047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb277fe640 1 -- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb100039a0 msgr2=0x7feb10003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:13.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb277fe640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb100039a0 0x7feb10003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb277fe640 1 -- 172.21.15.70:0/2685065852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb10003520 con 0x7feb10004d00 2023-10-08T05:38:13.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.394+0000 7feb26ffd640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb100039a0 0x7feb10003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:13.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.395+0000 7feb277fe640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb10004d00 0x7feb10005180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7feb280caa90 tx=0x7feb100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b052e12de132635a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:13.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.396+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb200052d0 con 0x7feb10004d00 2023-10-08T05:38:13.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.396+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2002e050 con 0x7feb10004d00 2023-10-08T05:38:13.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.396+0000 7feb25ffb640 1 -- 172.21.15.70:0/2685065852 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb200035f0 con 0x7feb10004d00 2023-10-08T05:38:13.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.396+0000 7feb27fff640 1 -- 172.21.15.70:0/2685065852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb10004d00 msgr2=0x7feb10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:13.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.396+0000 7feb27fff640 1 --2- 172.21.15.70:0/2685065852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb10004d00 0x7feb10005180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7feb280caa90 tx=0x7feb100057f0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.396+0000 7feb277fe640 1 -- 172.21.15.70:0/2685065852 reap_dead start 2023-10-08T05:38:13.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.396+0000 7feb27fff640 1 -- 172.21.15.70:0/2685065852 shutdown_connections 2023-10-08T05:38:13.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.396+0000 7feb27fff640 1 -- 172.21.15.70:0/2685065852 >> 172.21.15.70:0/2685065852 conn(0x7feb2806c050 msgr2=0x7feb2806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:13.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.396+0000 7feb27fff640 1 -- 172.21.15.70:0/2685065852 shutdown_connections 2023-10-08T05:38:13.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.396+0000 7feb27fff640 1 -- 172.21.15.70:0/2685065852 wait complete. 2023-10-08T05:38:13.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.398+0000 7feb27fff640 1 Processor -- start 2023-10-08T05:38:13.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.398+0000 7feb27fff640 1 -- start start 2023-10-08T05:38:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.399+0000 7feb27fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10004d00 0x7feb280c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.399+0000 7feb27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb281ef840 0x7feb280c3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.399+0000 7feb27fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280c35b0 0x7feb280c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.399+0000 7feb27fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb280c60f0 con 0x7feb281ef840 2023-10-08T05:38:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.399+0000 7feb27fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb280c86f0 con 0x7feb10004d00 2023-10-08T05:38:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.399+0000 7feb27fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2815dde0 con 0x7feb280c35b0 2023-10-08T05:38:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.399+0000 7feb267fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb281ef840 0x7feb280c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.399+0000 7feb267fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb281ef840 0x7feb280c3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43408/0 (socket says 172.21.15.70:43408) 2023-10-08T05:38:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.399+0000 7feb267fc640 1 -- 172.21.15.70:0/92747674 learned_addr learned my addr 172.21.15.70:0/92747674 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:13.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb26ffd640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10004d00 0x7feb280c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb277fe640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280c35b0 0x7feb280c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb267fc640 1 -- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280c35b0 msgr2=0x7feb280c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb267fc640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280c35b0 0x7feb280c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb267fc640 1 -- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10004d00 msgr2=0x7feb280c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb267fc640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10004d00 0x7feb280c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb267fc640 1 -- 172.21.15.70:0/92747674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2815d820 con 0x7feb281ef840 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb267fc640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb281ef840 0x7feb280c3070 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7feb1c010890 tx=0x7feb1c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb0ffff640 1 -- 172.21.15.70:0/92747674 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c009610 con 0x7feb281ef840 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb0ffff640 1 -- 172.21.15.70:0/92747674 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c00eb60 con 0x7feb281ef840 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb0ffff640 1 -- 172.21.15.70:0/92747674 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c01c400 con 0x7feb281ef840 2023-10-08T05:38:13.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.400+0000 7feb27fff640 1 -- 172.21.15.70:0/92747674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2815db00 con 0x7feb281ef840 2023-10-08T05:38:13.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.401+0000 7feb27fff640 1 -- 172.21.15.70:0/92747674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb281efcc0 con 0x7feb281ef840 2023-10-08T05:38:13.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.402+0000 7feb27fff640 1 -- 172.21.15.70:0/92747674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb280c6260 con 0x7feb281ef840 2023-10-08T05:38:13.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.403+0000 7feb0ffff640 1 -- 172.21.15.70:0/92747674 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7feb1c01c560 con 0x7feb281ef840 2023-10-08T05:38:13.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.404+0000 7feb0ffff640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7feb1406b070 0x7feb1406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:13.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.404+0000 7feb0ffff640 1 -- 172.21.15.70:0/92747674 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7feb1c08a210 con 0x7feb281ef840 2023-10-08T05:38:13.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.404+0000 7feb26ffd640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7feb1406b070 0x7feb1406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:13.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.405+0000 7feb26ffd640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7feb1406b070 0x7feb1406d530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feb18007c40 tx=0x7feb18009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:13.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.408+0000 7feb0ffff640 1 -- 172.21.15.70:0/92747674 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb1c054df0 con 0x7feb281ef840 2023-10-08T05:38:13.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.618+0000 7feb27fff640 1 -- 172.21.15.70:0/92747674 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7feb280cb5f0 con 0x7feb1406b070 2023-10-08T05:38:13.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.756+0000 7feb0ffff640 1 -- 172.21.15.70:0/92747674 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7feb1800d040 con 0x7feb1406b070 2023-10-08T05:38:13.758 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-10-08T05:38:13.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 -- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7feb1406b070 msgr2=0x7feb1406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:13.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7feb1406b070 0x7feb1406d530 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feb18007c40 tx=0x7feb18009040 comp rx=0 tx=0).stop 2023-10-08T05:38:13.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 -- 172.21.15.70:0/92747674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb281ef840 msgr2=0x7feb280c3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:13.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb281ef840 0x7feb280c3070 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7feb1c010890 tx=0x7feb1c010d50 comp rx=0 tx=0).stop 2023-10-08T05:38:13.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 -- 172.21.15.70:0/92747674 shutdown_connections 2023-10-08T05:38:13.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7feb1406b070 0x7feb1406d530 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb280c35b0 0x7feb280c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb281ef840 0x7feb280c3070 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 --2- 172.21.15.70:0/92747674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10004d00 0x7feb280c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 -- 172.21.15.70:0/92747674 >> 172.21.15.70:0/92747674 conn(0x7feb2806c050 msgr2=0x7feb280af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:13.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 -- 172.21.15.70:0/92747674 shutdown_connections 2023-10-08T05:38:13.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.759+0000 7feb27fff640 1 -- 172.21.15.70:0/92747674 wait complete. 2023-10-08T05:38:13.780 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-10-08T05:38:13.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir4 1m 2023-10-08T05:38:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.993+0000 7f8282111640 1 Processor -- start 2023-10-08T05:38:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.993+0000 7f8282111640 1 -- start start 2023-10-08T05:38:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.994+0000 7f8282111640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f827c0b0680 0x7f827c15e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.994+0000 7f8282111640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f827c0ca680 0x7f827c15ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.994+0000 7f8282111640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827c0cab60 con 0x7f827c15a820 2023-10-08T05:38:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.994+0000 7f8282111640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c0cb960 con 0x7f827c0ca680 2023-10-08T05:38:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.994+0000 7f8282111640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c0cbdf0 con 0x7f827c0b0680 2023-10-08T05:38:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.995+0000 7f8281910640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f827c0ca680 0x7f827c15ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.995+0000 7f8281910640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f827c0ca680 0x7f827c15ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49122/0 (socket says 172.21.15.70:49122) 2023-10-08T05:38:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.995+0000 7f8281910640 1 -- 172.21.15.70:0/1181080415 learned_addr learned my addr 172.21.15.70:0/1181080415 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.995+0000 7f828090e640 1 --2- 172.21.15.70:0/1181080415 >> v2:172.21.15.181:3300/0 conn(0x7f827c0b0680 0x7f827c15e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.995+0000 7f8281910640 1 -- 172.21.15.70:0/1181080415 >> v2:172.21.15.181:3300/0 conn(0x7f827c0b0680 msgr2=0x7f827c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:13.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.995+0000 7f8281910640 1 --2- 172.21.15.70:0/1181080415 >> v2:172.21.15.181:3300/0 conn(0x7f827c0b0680 0x7f827c15e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:13.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.995+0000 7f8281910640 1 -- 172.21.15.70:0/1181080415 >> v1:172.21.15.181:6789/0 conn(0x7f827c15a820 legacy=0x7f827c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:13.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.995+0000 7f8281910640 1 -- 172.21.15.70:0/1181080415 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0cb030 con 0x7f827c0ca680 2023-10-08T05:38:13.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.997+0000 7f8281910640 1 --2- 172.21.15.70:0/1181080415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0ca680 0x7f827c15ea80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f827c0718e0 tx=0x7f827402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdec85a56ec628f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:13.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.997+0000 7f8273fff640 1 -- 172.21.15.70:0/1181080415 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827402c9f0 con 0x7f827c0ca680 2023-10-08T05:38:13.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.997+0000 7f8273fff640 1 -- 172.21.15.70:0/1181080415 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827402cb50 con 0x7f827c0ca680 2023-10-08T05:38:13.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.997+0000 7f8273fff640 1 -- 172.21.15.70:0/1181080415 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82740343f0 con 0x7f827c0ca680 2023-10-08T05:38:13.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.998+0000 7f8282111640 1 -- 172.21.15.70:0/1181080415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0ca680 msgr2=0x7f827c15ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.998+0000 7f8282111640 1 --2- 172.21.15.70:0/1181080415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0ca680 0x7f827c15ea80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f827c0718e0 tx=0x7f827402be50 comp rx=0 tx=0).stop 2023-10-08T05:38:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.998+0000 7f8282111640 1 -- 172.21.15.70:0/1181080415 shutdown_connections 2023-10-08T05:38:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.998+0000 7f8282111640 1 --2- 172.21.15.70:0/1181080415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0ca680 0x7f827c15ea80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.998+0000 7f8282111640 1 --2- 172.21.15.70:0/1181080415 >> v2:172.21.15.181:3300/0 conn(0x7f827c0b0680 0x7f827c15e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.998+0000 7f8282111640 1 -- 172.21.15.70:0/1181080415 >> 172.21.15.70:0/1181080415 conn(0x7f827c06c050 msgr2=0x7f827c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.998+0000 7f8282111640 1 -- 172.21.15.70:0/1181080415 shutdown_connections 2023-10-08T05:38:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:13.998+0000 7f8282111640 1 -- 172.21.15.70:0/1181080415 wait complete. 2023-10-08T05:38:14.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.000+0000 7f8282111640 1 Processor -- start 2023-10-08T05:38:14.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.000+0000 7f8282111640 1 -- start start 2023-10-08T05:38:14.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.001+0000 7f8282111640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0b0680 0x7f827c14f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:14.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.001+0000 7f8282111640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f827c0ca680 0x7f827c0cd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:14.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.001+0000 7f8282111640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f827c15a820 0x7f827c0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:14.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.001+0000 7f8282111640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0c5fa0 con 0x7f827c0b0680 2023-10-08T05:38:14.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.001+0000 7f8282111640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0acc60 con 0x7f827c15a820 2023-10-08T05:38:14.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.001+0000 7f8282111640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0acf40 con 0x7f827c0ca680 2023-10-08T05:38:14.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.001+0000 7f828110f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0b0680 0x7f827c14f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:14.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.001+0000 7f828110f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0b0680 0x7f827c14f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49124/0 (socket says 172.21.15.70:49124) 2023-10-08T05:38:14.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.001+0000 7f828110f640 1 -- 172.21.15.70:0/3632359866 learned_addr learned my addr 172.21.15.70:0/3632359866 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:14.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f828090e640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f827c0ca680 0x7f827c0cd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:14.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f8281910640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f827c15a820 0x7f827c0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:14.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f828110f640 1 -- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f827c0ca680 msgr2=0x7f827c0cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f828110f640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f827c0ca680 0x7f827c0cd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f828110f640 1 -- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f827c15a820 msgr2=0x7f827c0707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f828110f640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f827c15a820 0x7f827c0707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f828110f640 1 -- 172.21.15.70:0/3632359866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274009b30 con 0x7f827c0b0680 2023-10-08T05:38:14.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f828090e640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f827c0ca680 0x7f827c0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:14.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f828110f640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0b0680 0x7f827c14f9a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f827800b3f0 tx=0x7f827800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:14.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f8271ffb640 1 -- 172.21.15.70:0/3632359866 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278011830 con 0x7f827c0b0680 2023-10-08T05:38:14.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f8271ffb640 1 -- 172.21.15.70:0/3632359866 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8278011990 con 0x7f827c0b0680 2023-10-08T05:38:14.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f8271ffb640 1 -- 172.21.15.70:0/3632359866 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278011d40 con 0x7f827c0b0680 2023-10-08T05:38:14.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f8282111640 1 -- 172.21.15.70:0/3632359866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c0ad220 con 0x7f827c0b0680 2023-10-08T05:38:14.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.002+0000 7f8282111640 1 -- 172.21.15.70:0/3632359866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c14fee0 con 0x7f827c0b0680 2023-10-08T05:38:14.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.004+0000 7f8282111640 1 -- 172.21.15.70:0/3632359866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c074150 con 0x7f827c0b0680 2023-10-08T05:38:14.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.005+0000 7f8271ffb640 1 -- 172.21.15.70:0/3632359866 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f827801a400 con 0x7f827c0b0680 2023-10-08T05:38:14.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.005+0000 7f8271ffb640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f826006b070 0x7f826006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:14.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.006+0000 7f8271ffb640 1 -- 172.21.15.70:0/3632359866 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f8278089b70 con 0x7f827c0b0680 2023-10-08T05:38:14.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.006+0000 7f828090e640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f826006b070 0x7f826006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:14.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.007+0000 7f828090e640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f826006b070 0x7f826006d530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f827c0713a0 tx=0x7f826c008040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:14.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.010+0000 7f8271ffb640 1 -- 172.21.15.70:0/3632359866 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82780546e0 con 0x7f827c0b0680 2023-10-08T05:38:14.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.223+0000 7f8282111640 1 -- 172.21.15.70:0/3632359866 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f827c0cb5f0 con 0x7f826006b070 2023-10-08T05:38:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.349+0000 7f8271ffb640 1 -- 172.21.15.70:0/3632359866 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f826c0095d0 con 0x7f826006b070 2023-10-08T05:38:14.350 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-10-08T05:38:14.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.352+0000 7f8282111640 1 -- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f826006b070 msgr2=0x7f826006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.352+0000 7f8282111640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f826006b070 0x7f826006d530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f827c0713a0 tx=0x7f826c008040 comp rx=0 tx=0).stop 2023-10-08T05:38:14.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.352+0000 7f8282111640 1 -- 172.21.15.70:0/3632359866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0b0680 msgr2=0x7f827c14f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.352+0000 7f8282111640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0b0680 0x7f827c14f9a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f827800b3f0 tx=0x7f827800b8b0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.353+0000 7f8282111640 1 -- 172.21.15.70:0/3632359866 shutdown_connections 2023-10-08T05:38:14.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.353+0000 7f8282111640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f827c15a820 0x7f827c0707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.353+0000 7f8282111640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f826006b070 0x7f826006d530 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.353+0000 7f8282111640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f827c0ca680 0x7f827c0cd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.353+0000 7f8282111640 1 --2- 172.21.15.70:0/3632359866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c0b0680 0x7f827c14f9a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.353+0000 7f8282111640 1 -- 172.21.15.70:0/3632359866 >> 172.21.15.70:0/3632359866 conn(0x7f827c06c050 msgr2=0x7f827c0ad970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:14.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.353+0000 7f8282111640 1 -- 172.21.15.70:0/3632359866 shutdown_connections 2023-10-08T05:38:14.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.353+0000 7f8282111640 1 -- 172.21.15.70:0/3632359866 wait complete. 2023-10-08T05:38:14.377 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-10-08T05:38:14.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir5 1m 2023-10-08T05:38:14.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.592+0000 7f513ad05640 1 Processor -- start 2023-10-08T05:38:14.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.593+0000 7f513ad05640 1 -- start start 2023-10-08T05:38:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.594+0000 7f513ad05640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5134070cc0 0x7f51340caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.594+0000 7f513ad05640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51340cb450 0x7f51340b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.594+0000 7f513ad05640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5134071170 con 0x7f5134160660 2023-10-08T05:38:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.594+0000 7f513ad05640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513406e150 con 0x7f51340cb450 2023-10-08T05:38:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.594+0000 7f513ad05640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513406de70 con 0x7f5134070cc0 2023-10-08T05:38:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.594+0000 7f5138aa1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5134160660 0x7f51340c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33232/0 (socket says 172.21.15.70:33232) 2023-10-08T05:38:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.594+0000 7f5138aa1640 1 -- 172.21.15.70:0/3076021362 learned_addr learned my addr 172.21.15.70:0/3076021362 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:14.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.594+0000 7f51392a2640 1 --2- 172.21.15.70:0/3076021362 >> v2:172.21.15.70:3300/0 conn(0x7f51340cb450 0x7f51340b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:14.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.595+0000 7f5133fff640 1 --2- 172.21.15.70:0/3076021362 >> v2:172.21.15.181:3300/0 conn(0x7f5134070cc0 0x7f51340caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:14.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.595+0000 7f51337fe640 1 -- 172.21.15.70:0/3076021362 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034483640 0 0) 0x7f5124001490 con 0x7f5134160660 2023-10-08T05:38:14.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.595+0000 7f51337fe640 1 -- 172.21.15.70:0/3076021362 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5120003520 con 0x7f5134160660 2023-10-08T05:38:14.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.595+0000 7f51392a2640 1 -- 172.21.15.70:0/3076021362 >> v2:172.21.15.181:3300/0 conn(0x7f5134070cc0 msgr2=0x7f51340caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.595+0000 7f51392a2640 1 --2- 172.21.15.70:0/3076021362 >> v2:172.21.15.181:3300/0 conn(0x7f5134070cc0 0x7f51340caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.595+0000 7f51392a2640 1 -- 172.21.15.70:0/3076021362 >> v1:172.21.15.70:6789/0 conn(0x7f5134160660 legacy=0x7f51340c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.595+0000 7f51392a2640 1 -- 172.21.15.70:0/3076021362 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51340c8e30 con 0x7f51340cb450 2023-10-08T05:38:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.595+0000 7f5133fff640 1 --2- 172.21.15.70:0/3076021362 >> v2:172.21.15.181:3300/0 conn(0x7f5134070cc0 0x7f51340caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.596+0000 7f51392a2640 1 --2- 172.21.15.70:0/3076021362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51340cb450 0x7f51340b1990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f51340b0d50 tx=0x7f512c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c88fe0a19c31b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:14.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.596+0000 7f51337fe640 1 -- 172.21.15.70:0/3076021362 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c031830 con 0x7f51340cb450 2023-10-08T05:38:14.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.597+0000 7f51337fe640 1 -- 172.21.15.70:0/3076021362 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c031990 con 0x7f51340cb450 2023-10-08T05:38:14.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.597+0000 7f51337fe640 1 -- 172.21.15.70:0/3076021362 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c031d00 con 0x7f51340cb450 2023-10-08T05:38:14.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.597+0000 7f513ad05640 1 -- 172.21.15.70:0/3076021362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51340cb450 msgr2=0x7f51340b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.597+0000 7f513ad05640 1 --2- 172.21.15.70:0/3076021362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51340cb450 0x7f51340b1990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f51340b0d50 tx=0x7f512c02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.597+0000 7f513ad05640 1 -- 172.21.15.70:0/3076021362 shutdown_connections 2023-10-08T05:38:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.597+0000 7f513ad05640 1 --2- 172.21.15.70:0/3076021362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51340cb450 0x7f51340b1990 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.597+0000 7f513ad05640 1 --2- 172.21.15.70:0/3076021362 >> v2:172.21.15.181:3300/0 conn(0x7f5134070cc0 0x7f51340caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.597+0000 7f513ad05640 1 -- 172.21.15.70:0/3076021362 >> 172.21.15.70:0/3076021362 conn(0x7f513406a5d0 msgr2=0x7f51340b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.597+0000 7f513ad05640 1 -- 172.21.15.70:0/3076021362 shutdown_connections 2023-10-08T05:38:14.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.597+0000 7f513ad05640 1 -- 172.21.15.70:0/3076021362 wait complete. 2023-10-08T05:38:14.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.599+0000 7f513ad05640 1 Processor -- start 2023-10-08T05:38:14.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.599+0000 7f513ad05640 1 -- start start 2023-10-08T05:38:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.600+0000 7f513ad05640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5134070cc0 0x7f5134156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.600+0000 7f513ad05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51340cb450 0x7f51340c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.600+0000 7f513ad05640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5134160660 0x7f5134158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.600+0000 7f513ad05640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513415b960 con 0x7f5134070cc0 2023-10-08T05:38:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.600+0000 7f513ad05640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51340c4e00 con 0x7f5134160660 2023-10-08T05:38:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.600+0000 7f513ad05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51340c50e0 con 0x7f51340cb450 2023-10-08T05:38:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.600+0000 7f5138aa1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5134070cc0 0x7f5134156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.600+0000 7f5138aa1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5134070cc0 0x7f5134156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49146/0 (socket says 172.21.15.70:49146) 2023-10-08T05:38:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.600+0000 7f5138aa1640 1 -- 172.21.15.70:0/3772617058 learned_addr learned my addr 172.21.15.70:0/3772617058 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.601+0000 7f51392a2640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5134160660 0x7f5134158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.601+0000 7f5138aa1640 1 -- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51340cb450 msgr2=0x7f51340c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.601+0000 7f5133fff640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51340cb450 0x7f51340c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.601+0000 7f5138aa1640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51340cb450 0x7f51340c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.601+0000 7f5138aa1640 1 -- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5134160660 msgr2=0x7f5134158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.601+0000 7f5138aa1640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5134160660 0x7f5134158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.601+0000 7f5138aa1640 1 -- 172.21.15.70:0/3772617058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c009400 con 0x7f5134070cc0 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.601+0000 7f51392a2640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5134160660 0x7f5134158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.601+0000 7f5138aa1640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5134070cc0 0x7f5134156050 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f512400cae0 tx=0x7f512400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.601+0000 7f51317fa640 1 -- 172.21.15.70:0/3772617058 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400ac40 con 0x7f5134070cc0 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.602+0000 7f51317fa640 1 -- 172.21.15.70:0/3772617058 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512400adc0 con 0x7f5134070cc0 2023-10-08T05:38:14.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.602+0000 7f513ad05640 1 -- 172.21.15.70:0/3772617058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51340c53c0 con 0x7f5134070cc0 2023-10-08T05:38:14.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.602+0000 7f51317fa640 1 -- 172.21.15.70:0/3772617058 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124012640 con 0x7f5134070cc0 2023-10-08T05:38:14.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.602+0000 7f513ad05640 1 -- 172.21.15.70:0/3772617058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51340c49b0 con 0x7f5134070cc0 2023-10-08T05:38:14.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.603+0000 7f513ad05640 1 -- 172.21.15.70:0/3772617058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51340c5600 con 0x7f5134070cc0 2023-10-08T05:38:14.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.604+0000 7f51317fa640 1 -- 172.21.15.70:0/3772617058 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f5124007070 con 0x7f5134070cc0 2023-10-08T05:38:14.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.605+0000 7f51317fa640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f511406b0a0 0x7f511406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:14.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.605+0000 7f51317fa640 1 -- 172.21.15.70:0/3772617058 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f51240899e0 con 0x7f5134070cc0 2023-10-08T05:38:14.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.605+0000 7f5133fff640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f511406b0a0 0x7f511406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:14.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.606+0000 7f5133fff640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f511406b0a0 0x7f511406d560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5134158860 tx=0x7f511c008040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:14.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.609+0000 7f51317fa640 1 -- 172.21.15.70:0/3772617058 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51240545a0 con 0x7f5134070cc0 2023-10-08T05:38:14.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.821+0000 7f513ad05640 1 -- 172.21.15.70:0/3772617058 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f513415b6e0 con 0x7f511406b0a0 2023-10-08T05:38:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.945+0000 7f51317fa640 1 -- 172.21.15.70:0/3772617058 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f511c00c040 con 0x7f511406b0a0 2023-10-08T05:38:14.947 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-10-08T05:38:14.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 -- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f511406b0a0 msgr2=0x7f511406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f511406b0a0 0x7f511406d560 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5134158860 tx=0x7f511c008040 comp rx=0 tx=0).stop 2023-10-08T05:38:14.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 -- 172.21.15.70:0/3772617058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5134070cc0 msgr2=0x7f5134156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:14.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5134070cc0 0x7f5134156050 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f512400cae0 tx=0x7f512400a060 comp rx=0 tx=0).stop 2023-10-08T05:38:14.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 -- 172.21.15.70:0/3772617058 shutdown_connections 2023-10-08T05:38:14.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5134160660 0x7f5134158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f511406b0a0 0x7f511406d560 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51340cb450 0x7f51340c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 --2- 172.21.15.70:0/3772617058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5134070cc0 0x7f5134156050 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 -- 172.21.15.70:0/3772617058 >> 172.21.15.70:0/3772617058 conn(0x7f513406a5d0 msgr2=0x7f513415f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 -- 172.21.15.70:0/3772617058 shutdown_connections 2023-10-08T05:38:14.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:14.949+0000 7f513ad05640 1 -- 172.21.15.70:0/3772617058 wait complete. 2023-10-08T05:38:14.972 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-10-08T05:38:14.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir6 1m 2023-10-08T05:38:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.176+0000 7f433d92c640 1 Processor -- start 2023-10-08T05:38:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.177+0000 7f433d92c640 1 -- start start 2023-10-08T05:38:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.178+0000 7f433d92c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43380afbe0 0x7f43380b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.178+0000 7f433d92c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f43380cb130 0x7f43380cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.178+0000 7f433d92c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43380cbad0 con 0x7f433806dfc0 2023-10-08T05:38:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.178+0000 7f433d92c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43380b2620 con 0x7f43380cb130 2023-10-08T05:38:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.178+0000 7f433d92c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43380b2060 con 0x7f43380afbe0 2023-10-08T05:38:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.179+0000 7f43377fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f43380cb130 0x7f43380cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.179+0000 7f43377fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f43380cb130 0x7f43380cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49150/0 (socket says 172.21.15.70:49150) 2023-10-08T05:38:15.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.179+0000 7f43377fe640 1 -- 172.21.15.70:0/1299622317 learned_addr learned my addr 172.21.15.70:0/1299622317 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:15.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.179+0000 7f4336ffd640 1 --2- 172.21.15.70:0/1299622317 >> v2:172.21.15.181:3300/0 conn(0x7f43380afbe0 0x7f43380b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.179+0000 7f43377fe640 1 -- 172.21.15.70:0/1299622317 >> v2:172.21.15.181:3300/0 conn(0x7f43380afbe0 msgr2=0x7f43380b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.179+0000 7f43377fe640 1 --2- 172.21.15.70:0/1299622317 >> v2:172.21.15.181:3300/0 conn(0x7f43380afbe0 0x7f43380b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.179+0000 7f43377fe640 1 -- 172.21.15.70:0/1299622317 >> v1:172.21.15.70:6789/0 conn(0x7f433806dfc0 legacy=0x7f4338070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.179+0000 7f43377fe640 1 -- 172.21.15.70:0/1299622317 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380b2340 con 0x7f43380cb130 2023-10-08T05:38:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.179+0000 7f4335ffb640 1 -- 172.21.15.70:0/1299622317 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857340042 0 0) 0x7f43380cbad0 con 0x7f433806dfc0 2023-10-08T05:38:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.180+0000 7f43377fe640 1 --2- 172.21.15.70:0/1299622317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 0x7f43380cb590 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f433806e640 tx=0x7f433002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7824ec2b1f884a8f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.180+0000 7f4335ffb640 1 -- 172.21.15.70:0/1299622317 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330030860 con 0x7f43380cb130 2023-10-08T05:38:15.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.181+0000 7f4335ffb640 1 -- 172.21.15.70:0/1299622317 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43300309c0 con 0x7f43380cb130 2023-10-08T05:38:15.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.181+0000 7f4335ffb640 1 -- 172.21.15.70:0/1299622317 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330030d50 con 0x7f43380cb130 2023-10-08T05:38:15.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.181+0000 7f433d92c640 1 -- 172.21.15.70:0/1299622317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 msgr2=0x7f43380cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:15.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.181+0000 7f433d92c640 1 --2- 172.21.15.70:0/1299622317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 0x7f43380cb590 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f433806e640 tx=0x7f433002b9a0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.181+0000 7f433d92c640 1 -- 172.21.15.70:0/1299622317 shutdown_connections 2023-10-08T05:38:15.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.181+0000 7f433d92c640 1 --2- 172.21.15.70:0/1299622317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 0x7f43380cb590 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.181+0000 7f433d92c640 1 --2- 172.21.15.70:0/1299622317 >> v2:172.21.15.181:3300/0 conn(0x7f43380afbe0 0x7f43380b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.181+0000 7f433d92c640 1 -- 172.21.15.70:0/1299622317 >> 172.21.15.70:0/1299622317 conn(0x7f433806a5d0 msgr2=0x7f43380b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:15.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.181+0000 7f433d92c640 1 -- 172.21.15.70:0/1299622317 shutdown_connections 2023-10-08T05:38:15.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.181+0000 7f433d92c640 1 -- 172.21.15.70:0/1299622317 wait complete. 2023-10-08T05:38:15.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.183+0000 7f433d92c640 1 Processor -- start 2023-10-08T05:38:15.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.183+0000 7f433d92c640 1 -- start start 2023-10-08T05:38:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.184+0000 7f433d92c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433806dfc0 0x7f43380ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.184+0000 7f433d92c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43380afbe0 0x7f43380ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.184+0000 7f433d92c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 0x7f43380c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.184+0000 7f433d92c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43380ca4f0 con 0x7f43380cb130 2023-10-08T05:38:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.184+0000 7f433d92c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43380c98e0 con 0x7f43380afbe0 2023-10-08T05:38:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.184+0000 7f433d92c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43380c9bc0 con 0x7f433806dfc0 2023-10-08T05:38:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.184+0000 7f43377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 0x7f43380c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.184+0000 7f43377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 0x7f43380c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49164/0 (socket says 172.21.15.70:49164) 2023-10-08T05:38:15.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.184+0000 7f43377fe640 1 -- 172.21.15.70:0/2222664219 learned_addr learned my addr 172.21.15.70:0/2222664219 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:15.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.185+0000 7f43377fe640 1 -- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433806dfc0 msgr2=0x7f43380ca980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:15.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.185+0000 7f43377fe640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433806dfc0 0x7f43380ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.185+0000 7f43377fe640 1 -- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43380afbe0 msgr2=0x7f43380ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:15.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.185+0000 7f43367fc640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43380afbe0 0x7f43380ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.185+0000 7f43377fe640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43380afbe0 0x7f43380ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.185+0000 7f43377fe640 1 -- 172.21.15.70:0/2222664219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43300094d0 con 0x7f43380cb130 2023-10-08T05:38:15.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.185+0000 7f43377fe640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 0x7f43380c8d90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f43300090b0 tx=0x7f4330004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:15.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.185+0000 7f432ffff640 1 -- 172.21.15.70:0/2222664219 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330039a90 con 0x7f43380cb130 2023-10-08T05:38:15.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.185+0000 7f432ffff640 1 -- 172.21.15.70:0/2222664219 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4330039bf0 con 0x7f43380cb130 2023-10-08T05:38:15.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.186+0000 7f433d92c640 1 -- 172.21.15.70:0/2222664219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43380c9ea0 con 0x7f43380cb130 2023-10-08T05:38:15.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.186+0000 7f432ffff640 1 -- 172.21.15.70:0/2222664219 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43300433f0 con 0x7f43380cb130 2023-10-08T05:38:15.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.186+0000 7f433d92c640 1 -- 172.21.15.70:0/2222664219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43380c34f0 con 0x7f43380cb130 2023-10-08T05:38:15.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.187+0000 7f433d92c640 1 -- 172.21.15.70:0/2222664219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433804e160 con 0x7f43380cb130 2023-10-08T05:38:15.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.188+0000 7f432ffff640 1 -- 172.21.15.70:0/2222664219 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f433004c3b0 con 0x7f43380cb130 2023-10-08T05:38:15.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.189+0000 7f432ffff640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f431806b070 0x7f431806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.189+0000 7f432ffff640 1 -- 172.21.15.70:0/2222664219 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f43300ac250 con 0x7f43380cb130 2023-10-08T05:38:15.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.189+0000 7f4336ffd640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f431806b070 0x7f431806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.190+0000 7f4336ffd640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f431806b070 0x7f431806d530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f43280070b0 tx=0x7f4328007040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:15.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.193+0000 7f432ffff640 1 -- 172.21.15.70:0/2222664219 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4330076d00 con 0x7f43380cb130 2023-10-08T05:38:15.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.409+0000 7f433d92c640 1 -- 172.21.15.70:0/2222664219 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f4338070630 con 0x7f431806b070 2023-10-08T05:38:15.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.538+0000 7f432ffff640 1 -- 172.21.15.70:0/2222664219 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4328009bb0 con 0x7f431806b070 2023-10-08T05:38:15.540 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-10-08T05:38:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.541+0000 7f433d92c640 1 -- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f431806b070 msgr2=0x7f431806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.541+0000 7f433d92c640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f431806b070 0x7f431806d530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f43280070b0 tx=0x7f4328007040 comp rx=0 tx=0).stop 2023-10-08T05:38:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.541+0000 7f433d92c640 1 -- 172.21.15.70:0/2222664219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 msgr2=0x7f43380c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.541+0000 7f433d92c640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 0x7f43380c8d90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f43300090b0 tx=0x7f4330004730 comp rx=0 tx=0).stop 2023-10-08T05:38:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.542+0000 7f433d92c640 1 -- 172.21.15.70:0/2222664219 shutdown_connections 2023-10-08T05:38:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.542+0000 7f433d92c640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43380cb130 0x7f43380c8d90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.542+0000 7f433d92c640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43380afbe0 0x7f43380ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.542+0000 7f433d92c640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f431806b070 0x7f431806d530 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.542+0000 7f433d92c640 1 --2- 172.21.15.70:0/2222664219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433806dfc0 0x7f43380ca980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.542+0000 7f433d92c640 1 -- 172.21.15.70:0/2222664219 >> 172.21.15.70:0/2222664219 conn(0x7f433806a5d0 msgr2=0x7f43380cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:15.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.542+0000 7f433d92c640 1 -- 172.21.15.70:0/2222664219 shutdown_connections 2023-10-08T05:38:15.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.542+0000 7f433d92c640 1 -- 172.21.15.70:0/2222664219 wait complete. 2023-10-08T05:38:15.563 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-10-08T05:38:15.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir7 1m 2023-10-08T05:38:15.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.781+0000 7f79f7ac9640 1 Processor -- start 2023-10-08T05:38:15.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.782+0000 7f79f7ac9640 1 -- start start 2023-10-08T05:38:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.783+0000 7f79f7ac9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f79f0070cc0 0x7f79f00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.783+0000 7f79f7ac9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79f00cb450 0x7f79f00b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.783+0000 7f79f7ac9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f0071170 con 0x7f79f0160660 2023-10-08T05:38:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.783+0000 7f79f7ac9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f006e150 con 0x7f79f0070cc0 2023-10-08T05:38:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.783+0000 7f79f7ac9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f006de70 con 0x7f79f00cb450 2023-10-08T05:38:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.783+0000 7f79f5865640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f79f0160660 0x7f79f00c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33244/0 (socket says 172.21.15.70:33244) 2023-10-08T05:38:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.783+0000 7f79f5865640 1 -- 172.21.15.70:0/4070008297 learned_addr learned my addr 172.21.15.70:0/4070008297 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.784+0000 7f79f5064640 1 --2- 172.21.15.70:0/4070008297 >> v2:172.21.15.70:3300/0 conn(0x7f79f0070cc0 0x7f79f00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.784+0000 7f79f6066640 1 --2- 172.21.15.70:0/4070008297 >> v2:172.21.15.181:3300/0 conn(0x7f79f00cb450 0x7f79f00b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.784+0000 7f79f5064640 1 -- 172.21.15.70:0/4070008297 >> v2:172.21.15.181:3300/0 conn(0x7f79f00cb450 msgr2=0x7f79f00b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:15.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.784+0000 7f79f5064640 1 --2- 172.21.15.70:0/4070008297 >> v2:172.21.15.181:3300/0 conn(0x7f79f00cb450 0x7f79f00b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.784+0000 7f79f5064640 1 -- 172.21.15.70:0/4070008297 >> v1:172.21.15.70:6789/0 conn(0x7f79f0160660 legacy=0x7f79f00c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:15.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.784+0000 7f79f5064640 1 -- 172.21.15.70:0/4070008297 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f00c8e30 con 0x7f79f0070cc0 2023-10-08T05:38:15.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.784+0000 7f79f6066640 1 --2- 172.21.15.70:0/4070008297 >> v2:172.21.15.181:3300/0 conn(0x7f79f00cb450 0x7f79f00b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:38:15.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.785+0000 7f79f5064640 1 --2- 172.21.15.70:0/4070008297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0070cc0 0x7f79f00caf10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f79f00b1760 tx=0x7f79e002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acc3b06413fcdb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f4863640 1 -- 172.21.15.70:0/4070008297 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0030710 con 0x7f79f0070cc0 2023-10-08T05:38:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f4863640 1 -- 172.21.15.70:0/4070008297 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e002e050 con 0x7f79f0070cc0 2023-10-08T05:38:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f4863640 1 -- 172.21.15.70:0/4070008297 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0030a90 con 0x7f79f0070cc0 2023-10-08T05:38:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f7ac9640 1 -- 172.21.15.70:0/4070008297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0070cc0 msgr2=0x7f79f00caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f7ac9640 1 --2- 172.21.15.70:0/4070008297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0070cc0 0x7f79f00caf10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f79f00b1760 tx=0x7f79e002b710 comp rx=0 tx=0).stop 2023-10-08T05:38:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f7ac9640 1 -- 172.21.15.70:0/4070008297 shutdown_connections 2023-10-08T05:38:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f7ac9640 1 --2- 172.21.15.70:0/4070008297 >> v2:172.21.15.181:3300/0 conn(0x7f79f00cb450 0x7f79f00b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f7ac9640 1 --2- 172.21.15.70:0/4070008297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0070cc0 0x7f79f00caf10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f7ac9640 1 -- 172.21.15.70:0/4070008297 >> 172.21.15.70:0/4070008297 conn(0x7f79f006a5d0 msgr2=0x7f79f00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:15.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f7ac9640 1 -- 172.21.15.70:0/4070008297 shutdown_connections 2023-10-08T05:38:15.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.786+0000 7f79f7ac9640 1 -- 172.21.15.70:0/4070008297 wait complete. 2023-10-08T05:38:15.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.788+0000 7f79f7ac9640 1 Processor -- start 2023-10-08T05:38:15.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.788+0000 7f79f7ac9640 1 -- start start 2023-10-08T05:38:15.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.789+0000 7f79f7ac9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f0070cc0 0x7f79f00c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.789+0000 7f79f7ac9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79f00cb450 0x7f79f015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.789+0000 7f79f7ac9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0160660 0x7f79f015d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.789+0000 7f79f7ac9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f015c9f0 con 0x7f79f0160660 2023-10-08T05:38:15.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.789+0000 7f79f7ac9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f015bde0 con 0x7f79f0070cc0 2023-10-08T05:38:15.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.789+0000 7f79f7ac9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f015c0c0 con 0x7f79f00cb450 2023-10-08T05:38:15.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f6066640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0160660 0x7f79f015d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f6066640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0160660 0x7f79f015d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49182/0 (socket says 172.21.15.70:49182) 2023-10-08T05:38:15.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f6066640 1 -- 172.21.15.70:0/2795822350 learned_addr learned my addr 172.21.15.70:0/2795822350 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:15.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f5865640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f0070cc0 0x7f79f00c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f5064640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79f00cb450 0x7f79f015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f6066640 1 -- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79f00cb450 msgr2=0x7f79f015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:15.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f6066640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79f00cb450 0x7f79f015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f6066640 1 -- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f0070cc0 msgr2=0x7f79f00c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:15.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f6066640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f0070cc0 0x7f79f00c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:15.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f6066640 1 -- 172.21.15.70:0/2795822350 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0009400 con 0x7f79f0160660 2023-10-08T05:38:15.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f5865640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f0070cc0 0x7f79f00c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:15.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f5064640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79f00cb450 0x7f79f015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:15.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.790+0000 7f79f6066640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0160660 0x7f79f015d320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f79e8007ef0 tx=0x7f79e800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:15.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.791+0000 7f79e67fc640 1 -- 172.21.15.70:0/2795822350 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8003b90 con 0x7f79f0160660 2023-10-08T05:38:15.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.791+0000 7f79e67fc640 1 -- 172.21.15.70:0/2795822350 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e800f040 con 0x7f79f0160660 2023-10-08T05:38:15.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.791+0000 7f79e67fc640 1 -- 172.21.15.70:0/2795822350 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e800de80 con 0x7f79f0160660 2023-10-08T05:38:15.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.791+0000 7f79f7ac9640 1 -- 172.21.15.70:0/2795822350 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f015c3a0 con 0x7f79f0160660 2023-10-08T05:38:15.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.791+0000 7f79f7ac9640 1 -- 172.21.15.70:0/2795822350 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f015d860 con 0x7f79f0160660 2023-10-08T05:38:15.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.792+0000 7f79f7ac9640 1 -- 172.21.15.70:0/2795822350 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f015da50 con 0x7f79f0160660 2023-10-08T05:38:15.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.793+0000 7f79e67fc640 1 -- 172.21.15.70:0/2795822350 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f79e8004030 con 0x7f79f0160660 2023-10-08T05:38:15.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.794+0000 7f79e67fc640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f79d406b070 0x7f79d406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:15.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.795+0000 7f79e67fc640 1 -- 172.21.15.70:0/2795822350 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f79e8089f40 con 0x7f79f0160660 2023-10-08T05:38:15.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.795+0000 7f79f5865640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f79d406b070 0x7f79d406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:15.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.796+0000 7f79f5865640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f79d406b070 0x7f79d406d530 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79ec000f60 tx=0x7f79ec009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:15.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:15.799+0000 7f79e67fc640 1 -- 172.21.15.70:0/2795822350 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79e8054aa0 con 0x7f79f0160660 2023-10-08T05:38:16.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.011+0000 7f79f7ac9640 1 -- 172.21.15.70:0/2795822350 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f79f00c93f0 con 0x7f79d406b070 2023-10-08T05:38:16.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.131+0000 7f79e67fc640 1 -- 172.21.15.70:0/2795822350 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f79ec001610 con 0x7f79d406b070 2023-10-08T05:38:16.133 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-10-08T05:38:16.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 -- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f79d406b070 msgr2=0x7f79d406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f79d406b070 0x7f79d406d530 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79ec000f60 tx=0x7f79ec009040 comp rx=0 tx=0).stop 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 -- 172.21.15.70:0/2795822350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0160660 msgr2=0x7f79f015d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0160660 0x7f79f015d320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f79e8007ef0 tx=0x7f79e800d4a0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 -- 172.21.15.70:0/2795822350 shutdown_connections 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f0160660 0x7f79f015d320 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f79d406b070 0x7f79d406d530 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79f00cb450 0x7f79f015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 --2- 172.21.15.70:0/2795822350 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79f0070cc0 0x7f79f00c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 -- 172.21.15.70:0/2795822350 >> 172.21.15.70:0/2795822350 conn(0x7f79f006a5d0 msgr2=0x7f79f00cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 -- 172.21.15.70:0/2795822350 shutdown_connections 2023-10-08T05:38:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.135+0000 7f79f7ac9640 1 -- 172.21.15.70:0/2795822350 wait complete. 2023-10-08T05:38:16.157 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-10-08T05:38:16.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir8 1m 2023-10-08T05:38:16.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.378+0000 7f5e8fd9e640 1 Processor -- start 2023-10-08T05:38:16.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.378+0000 7f5e8fd9e640 1 -- start start 2023-10-08T05:38:16.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.379+0000 7f5e8fd9e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e880c9cf0 0x7f5e8806db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.379+0000 7f5e8fd9e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8815b310 con 0x7f5e8815aa00 2023-10-08T05:38:16.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.379+0000 7f5e8fd9e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8806d290 con 0x7f5e8806c3a0 2023-10-08T05:38:16.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.379+0000 7f5e8fd9e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8806e0a0 con 0x7f5e880c9cf0 2023-10-08T05:38:16.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8e33b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5e8815aa00 0x7f5e880b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33260/0 (socket says 172.21.15.70:33260) 2023-10-08T05:38:16.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8e33b640 1 -- 172.21.15.70:0/1321261513 learned_addr learned my addr 172.21.15.70:0/1321261513 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:16.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8d339640 1 --2- 172.21.15.70:0/1321261513 >> v2:172.21.15.181:3300/0 conn(0x7f5e880c9cf0 0x7f5e8806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754515018 0 0) 0x7f5e80001490 con 0x7f5e8815aa00 2023-10-08T05:38:16.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e74003520 con 0x7f5e8815aa00 2023-10-08T05:38:16.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2674874968 0 0) 0x7f5e74003520 con 0x7f5e8815aa00 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 >> v2:172.21.15.181:3300/0 conn(0x7f5e880c9cf0 msgr2=0x7f5e8806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8cb38640 1 --2- 172.21.15.70:0/1321261513 >> v2:172.21.15.181:3300/0 conn(0x7f5e880c9cf0 0x7f5e8806db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 >> v1:172.21.15.181:6789/0 conn(0x7f5e8806c3a0 legacy=0x7f5e8806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e880c9860 con 0x7f5e8815aa00 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5e80001b80 con 0x7f5e8815aa00 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 >> v1:172.21.15.70:6789/0 conn(0x7f5e8815aa00 legacy=0x7f5e880b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.380+0000 7f5e8cb38640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e74003970 0x7f5e74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8cb38640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e740047c0 0x7f5e74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8cb38640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e74005180 0x7f5e7400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8806e0a0 con 0x7f5e74005180 2023-10-08T05:38:16.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7400c020 con 0x7f5e74003970 2023-10-08T05:38:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7400c1e0 con 0x7f5e740047c0 2023-10-08T05:38:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8e33b640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e74005180 0x7f5e7400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8db3a640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e74003970 0x7f5e74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8d339640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e740047c0 0x7f5e74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8e33b640 1 -- 172.21.15.70:0/1321261513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e740047c0 msgr2=0x7f5e74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8e33b640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e740047c0 0x7f5e74004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8e33b640 1 -- 172.21.15.70:0/1321261513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e74003970 msgr2=0x7f5e74003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8e33b640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e74003970 0x7f5e74003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.381+0000 7f5e8e33b640 1 -- 172.21.15.70:0/1321261513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74003520 con 0x7f5e74005180 2023-10-08T05:38:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.382+0000 7f5e8e33b640 1 -- 172.21.15.70:0/1321261513 reap_dead start 2023-10-08T05:38:16.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.382+0000 7f5e8e33b640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e74005180 0x7f5e7400b980 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000a800 tx=0x7f5e740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16b2086ba9f9144 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:16.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.383+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80034630 con 0x7f5e74005180 2023-10-08T05:38:16.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.383+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8002f070 con 0x7f5e74005180 2023-10-08T05:38:16.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.383+0000 7f5e8cb38640 1 -- 172.21.15.70:0/1321261513 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800349f0 con 0x7f5e74005180 2023-10-08T05:38:16.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.383+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/1321261513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e74005180 msgr2=0x7f5e7400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.383+0000 7f5e8fd9e640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e74005180 0x7f5e7400b980 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000a800 tx=0x7f5e740057f0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.383+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/1321261513 shutdown_connections 2023-10-08T05:38:16.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.383+0000 7f5e8fd9e640 1 --2- 172.21.15.70:0/1321261513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e74005180 0x7f5e7400b980 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.383+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/1321261513 >> 172.21.15.70:0/1321261513 conn(0x7f5e8806a5d0 msgr2=0x7f5e880b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:16.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.383+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/1321261513 shutdown_connections 2023-10-08T05:38:16.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.383+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/1321261513 wait complete. 2023-10-08T05:38:16.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.385+0000 7f5e8fd9e640 1 Processor -- start 2023-10-08T05:38:16.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.385+0000 7f5e8fd9e640 1 -- start start 2023-10-08T05:38:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.386+0000 7f5e8fd9e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e74005180 0x7f5e8814fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.386+0000 7f5e8fd9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e881596a0 0x7f5e88159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.386+0000 7f5e8fd9e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e880c6b30 0x7f5e880c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.386+0000 7f5e8fd9e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880c3390 con 0x7f5e880c6b30 2023-10-08T05:38:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.386+0000 7f5e8fd9e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8815a060 con 0x7f5e74005180 2023-10-08T05:38:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.386+0000 7f5e8fd9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880c2d80 con 0x7f5e881596a0 2023-10-08T05:38:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8e33b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e880c6b30 0x7f5e880c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8e33b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e880c6b30 0x7f5e880c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49208/0 (socket says 172.21.15.70:49208) 2023-10-08T05:38:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8e33b640 1 -- 172.21.15.70:0/4176909022 learned_addr learned my addr 172.21.15.70:0/4176909022 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:16.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8db3a640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e74005180 0x7f5e8814fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8d339640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e881596a0 0x7f5e88159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8e33b640 1 -- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e881596a0 msgr2=0x7f5e88159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8e33b640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e881596a0 0x7f5e88159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8e33b640 1 -- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e74005180 msgr2=0x7f5e8814fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8e33b640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e74005180 0x7f5e8814fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8e33b640 1 -- 172.21.15.70:0/4176909022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e881502c0 con 0x7f5e880c6b30 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8db3a640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e74005180 0x7f5e8814fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.387+0000 7f5e8e33b640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e880c6b30 0x7f5e880c6fb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5e80000f80 tx=0x7f5e880c5560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.388+0000 7f5e7e7fc640 1 -- 172.21.15.70:0/4176909022 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e80001ad0 con 0x7f5e880c6b30 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.388+0000 7f5e7e7fc640 1 -- 172.21.15.70:0/4176909022 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8002ce90 con 0x7f5e880c6b30 2023-10-08T05:38:16.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.388+0000 7f5e7e7fc640 1 -- 172.21.15.70:0/4176909022 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8003e6a0 con 0x7f5e880c6b30 2023-10-08T05:38:16.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.388+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/4176909022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e880c7860 con 0x7f5e880c6b30 2023-10-08T05:38:16.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.388+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/4176909022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e880c2fb0 con 0x7f5e880c6b30 2023-10-08T05:38:16.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.389+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/4176909022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e880c3170 con 0x7f5e880c6b30 2023-10-08T05:38:16.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.390+0000 7f5e7e7fc640 1 -- 172.21.15.70:0/4176909022 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f5e80031030 con 0x7f5e880c6b30 2023-10-08T05:38:16.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.391+0000 7f5e7e7fc640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f5e6c06b0a0 0x7f5e6c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.391+0000 7f5e7e7fc640 1 -- 172.21.15.70:0/4176909022 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f5e800aad70 con 0x7f5e880c6b30 2023-10-08T05:38:16.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.391+0000 7f5e8db3a640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f5e6c06b0a0 0x7f5e6c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.392+0000 7f5e8db3a640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f5e6c06b0a0 0x7f5e6c06d560 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5e84000c00 tx=0x7f5e84009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:16.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.395+0000 7f5e7e7fc640 1 -- 172.21.15.70:0/4176909022 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e800758c0 con 0x7f5e880c6b30 2023-10-08T05:38:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.607+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/4176909022 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e880c8f30 con 0x7f5e6c06b0a0 2023-10-08T05:38:16.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.732+0000 7f5e7e7fc640 1 -- 172.21.15.70:0/4176909022 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5e8400d050 con 0x7f5e6c06b0a0 2023-10-08T05:38:16.734 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-10-08T05:38:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f5e6c06b0a0 msgr2=0x7f5e6c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f5e6c06b0a0 0x7f5e6c06d560 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5e84000c00 tx=0x7f5e84009040 comp rx=0 tx=0).stop 2023-10-08T05:38:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/4176909022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e880c6b30 msgr2=0x7f5e880c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e880c6b30 0x7f5e880c6fb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5e80000f80 tx=0x7f5e880c5560 comp rx=0 tx=0).stop 2023-10-08T05:38:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/4176909022 shutdown_connections 2023-10-08T05:38:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e880c6b30 0x7f5e880c6fb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f5e6c06b0a0 0x7f5e6c06d560 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e881596a0 0x7f5e88159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 --2- 172.21.15.70:0/4176909022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e74005180 0x7f5e8814fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/4176909022 >> 172.21.15.70:0/4176909022 conn(0x7f5e8806a5d0 msgr2=0x7f5e880b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/4176909022 shutdown_connections 2023-10-08T05:38:16.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.736+0000 7f5e8fd9e640 1 -- 172.21.15.70:0/4176909022 wait complete. 2023-10-08T05:38:16.761 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-10-08T05:38:16.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir9 1m 2023-10-08T05:38:16.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.966+0000 7f841caea640 1 Processor -- start 2023-10-08T05:38:16.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.966+0000 7f841caea640 1 -- start start 2023-10-08T05:38:16.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.967+0000 7f841caea640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84180cb5e0 0x7f84180cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.967+0000 7f841caea640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8418159db0 con 0x7f84180b3c50 2023-10-08T05:38:16.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.967+0000 7f841caea640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8418071fe0 con 0x7f84181599d0 2023-10-08T05:38:16.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.967+0000 7f841caea640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84180b44c0 con 0x7f84180cb5e0 2023-10-08T05:38:16.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.968+0000 7f8416ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84180cb5e0 0x7f84180cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.968+0000 7f8417fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f84180b3c50 0x7f84180cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33266/0 (socket says 172.21.15.70:33266) 2023-10-08T05:38:16.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.968+0000 7f8417fff640 1 -- 172.21.15.70:0/4271978081 learned_addr learned my addr 172.21.15.70:0/4271978081 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:38:16.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.968+0000 7f8416ffd640 1 -- 172.21.15.70:0/4271978081 >> v1:172.21.15.181:6789/0 conn(0x7f84181599d0 legacy=0x7f84180caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:38:16.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.968+0000 7f8416ffd640 1 -- 172.21.15.70:0/4271978081 >> v1:172.21.15.70:6789/0 conn(0x7f84180b3c50 legacy=0x7f84180cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.968+0000 7f8416ffd640 1 -- 172.21.15.70:0/4271978081 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180713d0 con 0x7f84180cb5e0 2023-10-08T05:38:16.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.968+0000 7f84167fc640 1 -- 172.21.15.70:0/4271978081 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163502547 0 0) 0x7f8410001110 con 0x7f84180b3c50 2023-10-08T05:38:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.970+0000 7f8416ffd640 1 --2- 172.21.15.70:0/4271978081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 0x7f84180cdcd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f84000097a0 tx=0x7f840002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7f396843276e8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.970+0000 7f84167fc640 1 -- 172.21.15.70:0/4271978081 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400031830 con 0x7f84180cb5e0 2023-10-08T05:38:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.970+0000 7f84167fc640 1 -- 172.21.15.70:0/4271978081 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8400031990 con 0x7f84180cb5e0 2023-10-08T05:38:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.970+0000 7f84167fc640 1 -- 172.21.15.70:0/4271978081 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400031d20 con 0x7f84180cb5e0 2023-10-08T05:38:16.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.970+0000 7f841caea640 1 -- 172.21.15.70:0/4271978081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 msgr2=0x7f84180cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.970+0000 7f841caea640 1 --2- 172.21.15.70:0/4271978081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 0x7f84180cdcd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f84000097a0 tx=0x7f840002b8d0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.971+0000 7f841caea640 1 -- 172.21.15.70:0/4271978081 shutdown_connections 2023-10-08T05:38:16.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.971+0000 7f841caea640 1 --2- 172.21.15.70:0/4271978081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 0x7f84180cdcd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.971+0000 7f841caea640 1 -- 172.21.15.70:0/4271978081 >> 172.21.15.70:0/4271978081 conn(0x7f841806c050 msgr2=0x7f841806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:16.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.971+0000 7f841caea640 1 -- 172.21.15.70:0/4271978081 shutdown_connections 2023-10-08T05:38:16.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.971+0000 7f841caea640 1 -- 172.21.15.70:0/4271978081 wait complete. 2023-10-08T05:38:16.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.973+0000 7f841caea640 1 Processor -- start 2023-10-08T05:38:16.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.973+0000 7f841caea640 1 -- start start 2023-10-08T05:38:16.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f841caea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84180b3c50 0x7f841815c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f841caea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 0x7f8418153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f841caea640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84181599d0 0x7f84180cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f841caea640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84180a7810 con 0x7f84180cb5e0 2023-10-08T05:38:16.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f841caea640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84180a7af0 con 0x7f84181599d0 2023-10-08T05:38:16.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f841caea640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84180a7dd0 con 0x7f84180b3c50 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f8416ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 0x7f8418153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f8416ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 0x7f8418153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49218/0 (socket says 172.21.15.70:49218) 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f8416ffd640 1 -- 172.21.15.70:0/556804246 learned_addr learned my addr 172.21.15.70:0/556804246 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f84177fe640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84180b3c50 0x7f841815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f8417fff640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84181599d0 0x7f84180cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f8416ffd640 1 -- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84180b3c50 msgr2=0x7f841815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f8416ffd640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84180b3c50 0x7f841815c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.974+0000 7f8416ffd640 1 -- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84181599d0 msgr2=0x7f84180cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.975+0000 7f8416ffd640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84181599d0 0x7f84180cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.975+0000 7f8416ffd640 1 -- 172.21.15.70:0/556804246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400009400 con 0x7f84180cb5e0 2023-10-08T05:38:16.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.975+0000 7f84177fe640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84180b3c50 0x7f841815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:38:16.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.975+0000 7f8416ffd640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 0x7f8418153be0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f84000098c0 tx=0x7f84000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:16.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.975+0000 7f840ffff640 1 -- 172.21.15.70:0/556804246 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840003ba70 con 0x7f84180cb5e0 2023-10-08T05:38:16.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.975+0000 7f840ffff640 1 -- 172.21.15.70:0/556804246 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840003bbd0 con 0x7f84180cb5e0 2023-10-08T05:38:16.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.975+0000 7f840ffff640 1 -- 172.21.15.70:0/556804246 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84000453f0 con 0x7f84180cb5e0 2023-10-08T05:38:16.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.975+0000 7f841caea640 1 -- 172.21.15.70:0/556804246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84180a80b0 con 0x7f84180cb5e0 2023-10-08T05:38:16.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.975+0000 7f841caea640 1 -- 172.21.15.70:0/556804246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841815cd90 con 0x7f84180cb5e0 2023-10-08T05:38:16.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.977+0000 7f841caea640 1 -- 172.21.15.70:0/556804246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8418074150 con 0x7f84180cb5e0 2023-10-08T05:38:16.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.978+0000 7f840ffff640 1 -- 172.21.15.70:0/556804246 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 10) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f840004e450 con 0x7f84180cb5e0 2023-10-08T05:38:16.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.978+0000 7f840ffff640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f83f806b070 0x7f83f806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:38:16.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.979+0000 7f840ffff640 1 -- 172.21.15.70:0/556804246 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f84000abfb0 con 0x7f84180cb5e0 2023-10-08T05:38:16.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.979+0000 7f84177fe640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f83f806b070 0x7f83f806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:38:16.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.981+0000 7f84177fe640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f83f806b070 0x7f83f806d530 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8408007e90 tx=0x7f84080073a0 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:38:16.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:16.984+0000 7f840ffff640 1 -- 172.21.15.70:0/556804246 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8400076a60 con 0x7f84180cb5e0 2023-10-08T05:38:17.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.195+0000 7f841caea640 1 -- 172.21.15.70:0/556804246 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f8418071990 con 0x7f83f806b070 2023-10-08T05:38:17.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.316+0000 7f840ffff640 1 -- 172.21.15.70:0/556804246 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f840800cdb0 con 0x7f83f806b070 2023-10-08T05:38:17.318 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-10-08T05:38:17.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 -- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f83f806b070 msgr2=0x7f83f806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:17.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f83f806b070 0x7f83f806d530 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8408007e90 tx=0x7f84080073a0 comp rx=0 tx=0).stop 2023-10-08T05:38:17.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 -- 172.21.15.70:0/556804246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 msgr2=0x7f8418153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:38:17.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 0x7f8418153be0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f84000098c0 tx=0x7f84000046c0 comp rx=0 tx=0).stop 2023-10-08T05:38:17.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 -- 172.21.15.70:0/556804246 shutdown_connections 2023-10-08T05:38:17.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84181599d0 0x7f84180cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:17.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84180cb5e0 0x7f8418153be0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:17.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f83f806b070 0x7f83f806d530 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:17.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 --2- 172.21.15.70:0/556804246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84180b3c50 0x7f841815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:38:17.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 -- 172.21.15.70:0/556804246 >> 172.21.15.70:0/556804246 conn(0x7f841806c050 msgr2=0x7f84181561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:38:17.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 -- 172.21.15.70:0/556804246 shutdown_connections 2023-10-08T05:38:17.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:38:17.320+0000 7f841caea640 1 -- 172.21.15.70:0/556804246 wait complete. 2023-10-08T05:38:17.341 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-10-08T05:38:35.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:38:35.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:39:00.043 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:39:00.042+0000 7f3adcbd6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:39:00.043 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:39:00.042+0000 7f3adcbd6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:39:00.043 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:39:00.042+0000 7f3adcbd6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:39:00.043 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:39:00.042+0000 7f3adcbd6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:39:00.044 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:39:00.042+0000 7f3adcbd6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:39:05.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:39:05.411 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:39:35.547 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:39:35.557 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:40:05.694 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:40:05.703 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:40:35.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:40:36.148 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:41:06.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:41:06.298 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:41:36.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:41:36.440 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:42:06.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:42:06.599 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:42:36.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:42:36.757 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:43:00.582 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:43:00.581+0000 7f3ade3d9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:43:00.582 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:43:00.581+0000 7f3ade3d9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:43:00.583 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:43:00.581+0000 7f3ade3d9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:43:00.583 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:43:00.581+0000 7f3ade3d9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:43:00.583 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:43:00.581+0000 7f3ade3d9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:43:06.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:43:06.908 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:43:37.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:43:37.057 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:44:07.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:44:07.213 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:44:37.342 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:44:37.351 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:45:07.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:45:07.489 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:45:37.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:45:37.636 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:46:07.760 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:46:07.770 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:46:37.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:46:37.913 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:47:08.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:47:08.060 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:47:38.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:47:38.215 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:48:08.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:48:08.367 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:48:38.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:48:38.503 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:49:08.639 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:49:08.648 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:49:21.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir0 1m 2023-10-08T05:49:21.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.570+0000 7f1d2ba87640 1 Processor -- start 2023-10-08T05:49:21.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.571+0000 7f1d2ba87640 1 -- start start 2023-10-08T05:49:21.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.572+0000 7f1d2ba87640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1d240cb150 0x7f1d240cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:21.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.572+0000 7f1d2ba87640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1d240b1f20 0x7f1d240b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:21.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.572+0000 7f1d2ba87640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2406e1e0 con 0x7f1d24161690 2023-10-08T05:49:21.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.572+0000 7f1d2ba87640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d240cbb40 con 0x7f1d240cb150 2023-10-08T05:49:21.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.572+0000 7f1d2ba87640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2406cc50 con 0x7f1d240b1f20 2023-10-08T05:49:21.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.573+0000 7f1d29022640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1d240cb150 0x7f1d240cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:21.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.573+0000 7f1d29823640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1d24161690 0x7f1d2406dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49348/0 (socket says 172.21.15.70:49348) 2023-10-08T05:49:21.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.573+0000 7f1d29823640 1 -- 172.21.15.70:0/2417203575 learned_addr learned my addr 172.21.15.70:0/2417203575 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:21.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.573+0000 7f1d29022640 1 -- 172.21.15.70:0/2417203575 >> v2:172.21.15.181:3300/0 conn(0x7f1d240b1f20 msgr2=0x7f1d240b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:21.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.573+0000 7f1d29022640 1 --2- 172.21.15.70:0/2417203575 >> v2:172.21.15.181:3300/0 conn(0x7f1d240b1f20 0x7f1d240b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:21.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.573+0000 7f1d29022640 1 -- 172.21.15.70:0/2417203575 >> v1:172.21.15.70:6789/0 conn(0x7f1d24161690 legacy=0x7f1d2406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:21.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.573+0000 7f1d29022640 1 -- 172.21.15.70:0/2417203575 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2406cf30 con 0x7f1d240cb150 2023-10-08T05:49:21.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.574+0000 7f1d29022640 1 --2- 172.21.15.70:0/2417203575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240cb150 0x7f1d240cb600 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1d2406d890 tx=0x7f1d1802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a67b335b2ec99a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:21.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.574+0000 7f1d28821640 1 -- 172.21.15.70:0/2417203575 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18030710 con 0x7f1d240cb150 2023-10-08T05:49:21.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.574+0000 7f1d28821640 1 -- 172.21.15.70:0/2417203575 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1802e050 con 0x7f1d240cb150 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.575+0000 7f1d28821640 1 -- 172.21.15.70:0/2417203575 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18030a90 con 0x7f1d240cb150 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.575+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2417203575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240cb150 msgr2=0x7f1d240cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.575+0000 7f1d2ba87640 1 --2- 172.21.15.70:0/2417203575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240cb150 0x7f1d240cb600 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1d2406d890 tx=0x7f1d1802b710 comp rx=0 tx=0).stop 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.575+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2417203575 shutdown_connections 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.575+0000 7f1d2ba87640 1 --2- 172.21.15.70:0/2417203575 >> v2:172.21.15.181:3300/0 conn(0x7f1d240b1f20 0x7f1d240b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.575+0000 7f1d2ba87640 1 --2- 172.21.15.70:0/2417203575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240cb150 0x7f1d240cb600 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.575+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2417203575 >> 172.21.15.70:0/2417203575 conn(0x7f1d2406a5d0 msgr2=0x7f1d240b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.575+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2417203575 shutdown_connections 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.575+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2417203575 wait complete. 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.577+0000 7f1d2ba87640 1 Processor -- start 2023-10-08T05:49:21.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.577+0000 7f1d2ba87640 1 -- start start 2023-10-08T05:49:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.578+0000 7f1d2ba87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240b1f20 0x7f1d240c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.578+0000 7f1d2ba87640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d240cb150 0x7f1d240c2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.578+0000 7f1d2ba87640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d24161690 0x7f1d240c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.578+0000 7f1d2ba87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240cdcc0 con 0x7f1d240b1f20 2023-10-08T05:49:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.578+0000 7f1d2ba87640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240c4790 con 0x7f1d240cb150 2023-10-08T05:49:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.578+0000 7f1d2ba87640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240c4a70 con 0x7f1d24161690 2023-10-08T05:49:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.578+0000 7f1d29823640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240b1f20 0x7f1d240c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:21.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.578+0000 7f1d29823640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240b1f20 0x7f1d240c40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41442/0 (socket says 172.21.15.70:41442) 2023-10-08T05:49:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.578+0000 7f1d29823640 1 -- 172.21.15.70:0/2828810371 learned_addr learned my addr 172.21.15.70:0/2828810371 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d29022640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d240cb150 0x7f1d240c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d29823640 1 -- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d24161690 msgr2=0x7f1d240c25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d2a024640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d24161690 0x7f1d240c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d29823640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d24161690 0x7f1d240c25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d29823640 1 -- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d240cb150 msgr2=0x7f1d240c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:21.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d29823640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d240cb150 0x7f1d240c2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d29823640 1 -- 172.21.15.70:0/2828810371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d18009400 con 0x7f1d240b1f20 2023-10-08T05:49:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d29022640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d240cb150 0x7f1d240c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d29823640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240b1f20 0x7f1d240c40b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1d14001f80 tx=0x7f1d1400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d227fc640 1 -- 172.21.15.70:0/2828810371 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14012400 con 0x7f1d240b1f20 2023-10-08T05:49:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d227fc640 1 -- 172.21.15.70:0/2828810371 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d14012560 con 0x7f1d240b1f20 2023-10-08T05:49:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d227fc640 1 -- 172.21.15.70:0/2828810371 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14012930 con 0x7f1d240b1f20 2023-10-08T05:49:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.579+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2828810371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d240c4d50 con 0x7f1d240b1f20 2023-10-08T05:49:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.580+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2828810371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d241f5d10 con 0x7f1d240b1f20 2023-10-08T05:49:21.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.581+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2828810371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d2404e160 con 0x7f1d240b1f20 2023-10-08T05:49:21.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.581+0000 7f1d227fc640 1 -- 172.21.15.70:0/2828810371 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f1d14007070 con 0x7f1d240b1f20 2023-10-08T05:49:21.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.582+0000 7f1d227fc640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f1d1006b070 0x7f1d1006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:21.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.583+0000 7f1d227fc640 1 -- 172.21.15.70:0/2828810371 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f1d14089df0 con 0x7f1d240b1f20 2023-10-08T05:49:21.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.583+0000 7f1d29022640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f1d1006b070 0x7f1d1006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:21.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.584+0000 7f1d29022640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f1d1006b070 0x7f1d1006d530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d18034000 tx=0x7f1d18008030 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:21.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.587+0000 7f1d227fc640 1 -- 172.21.15.70:0/2828810371 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d140549b0 con 0x7f1d240b1f20 2023-10-08T05:49:21.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.798+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2828810371 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d2406d4f0 con 0x7f1d1006b070 2023-10-08T05:49:21.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.961+0000 7f1d227fc640 1 -- 172.21.15.70:0/2828810371 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1d180309e0 con 0x7f1d1006b070 2023-10-08T05:49:21.962 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-10-08T05:49:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.965+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f1d1006b070 msgr2=0x7f1d1006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.965+0000 7f1d2ba87640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f1d1006b070 0x7f1d1006d530 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1d18034000 tx=0x7f1d18008030 comp rx=0 tx=0).stop 2023-10-08T05:49:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.965+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2828810371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240b1f20 msgr2=0x7f1d240c40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.965+0000 7f1d2ba87640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240b1f20 0x7f1d240c40b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1d14001f80 tx=0x7f1d1400a430 comp rx=0 tx=0).stop 2023-10-08T05:49:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.965+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2828810371 shutdown_connections 2023-10-08T05:49:21.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.965+0000 7f1d2ba87640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f1d1006b070 0x7f1d1006d530 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:21.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.965+0000 7f1d2ba87640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d24161690 0x7f1d240c25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:21.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.965+0000 7f1d2ba87640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d240cb150 0x7f1d240c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:21.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.965+0000 7f1d2ba87640 1 --2- 172.21.15.70:0/2828810371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d240b1f20 0x7f1d240c40b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:21.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.965+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2828810371 >> 172.21.15.70:0/2828810371 conn(0x7f1d2406a5d0 msgr2=0x7f1d240b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:21.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.966+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2828810371 shutdown_connections 2023-10-08T05:49:21.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:21.966+0000 7f1d2ba87640 1 -- 172.21.15.70:0/2828810371 wait complete. 2023-10-08T05:49:21.988 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-10-08T05:49:21.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir1 1m 2023-10-08T05:49:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.209+0000 7fe54b59e640 1 Processor -- start 2023-10-08T05:49:22.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.209+0000 7fe54b59e640 1 -- start start 2023-10-08T05:49:22.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.211+0000 7fe54b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 0x7fe54c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.211+0000 7fe54b59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe54c0b3c50 0x7fe54c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.211+0000 7fe54b59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54c159db0 con 0x7fe54c1599d0 2023-10-08T05:49:22.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.211+0000 7fe54b59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54c071fe0 con 0x7fe54c0b3c50 2023-10-08T05:49:22.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.211+0000 7fe54b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c0b44c0 con 0x7fe54c0cb5e0 2023-10-08T05:49:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.212+0000 7fe541d9b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 0x7fe54c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.212+0000 7fe541d9b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 0x7fe54c0cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:56090/0 (socket says 172.21.15.70:56090) 2023-10-08T05:49:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.212+0000 7fe541d9b640 1 -- 172.21.15.70:0/2535228954 learned_addr learned my addr 172.21.15.70:0/2535228954 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.212+0000 7fe54ad9d640 1 --2- 172.21.15.70:0/2535228954 >> v2:172.21.15.181:3300/0 conn(0x7fe54c0b3c50 0x7fe54c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.212+0000 7fe549d9b640 1 -- 172.21.15.70:0/2535228954 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376542623 0 0) 0x7fe53c001110 con 0x7fe54c1599d0 2023-10-08T05:49:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.212+0000 7fe549d9b640 1 -- 172.21.15.70:0/2535228954 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe538003520 con 0x7fe54c1599d0 2023-10-08T05:49:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.213+0000 7fe54ad9d640 1 -- 172.21.15.70:0/2535228954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 msgr2=0x7fe54c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.213+0000 7fe54ad9d640 1 --2- 172.21.15.70:0/2535228954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 0x7fe54c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.213+0000 7fe54ad9d640 1 -- 172.21.15.70:0/2535228954 >> v1:172.21.15.181:6789/0 conn(0x7fe54c1599d0 legacy=0x7fe54c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.213+0000 7fe54ad9d640 1 -- 172.21.15.70:0/2535228954 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c0713d0 con 0x7fe54c0b3c50 2023-10-08T05:49:22.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.214+0000 7fe54ad9d640 1 --2- 172.21.15.70:0/2535228954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54c0b3c50 0x7fe54c0cc0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe54400bd40 tx=0x7fe54402de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f526d4173bc522e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:22.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe549d9b640 1 -- 172.21.15.70:0/2535228954 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54402ebf0 con 0x7fe54c0b3c50 2023-10-08T05:49:22.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe549d9b640 1 -- 172.21.15.70:0/2535228954 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54402ed50 con 0x7fe54c0b3c50 2023-10-08T05:49:22.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe549d9b640 1 -- 172.21.15.70:0/2535228954 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544036640 con 0x7fe54c0b3c50 2023-10-08T05:49:22.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe54b59e640 1 -- 172.21.15.70:0/2535228954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54c0b3c50 msgr2=0x7fe54c0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe54b59e640 1 --2- 172.21.15.70:0/2535228954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54c0b3c50 0x7fe54c0cc0c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe54400bd40 tx=0x7fe54402de40 comp rx=0 tx=0).stop 2023-10-08T05:49:22.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe54b59e640 1 -- 172.21.15.70:0/2535228954 shutdown_connections 2023-10-08T05:49:22.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe54b59e640 1 --2- 172.21.15.70:0/2535228954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54c0b3c50 0x7fe54c0cc0c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe54b59e640 1 --2- 172.21.15.70:0/2535228954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 0x7fe54c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe54b59e640 1 -- 172.21.15.70:0/2535228954 >> 172.21.15.70:0/2535228954 conn(0x7fe54c06c050 msgr2=0x7fe54c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:22.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe54b59e640 1 -- 172.21.15.70:0/2535228954 shutdown_connections 2023-10-08T05:49:22.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.215+0000 7fe54b59e640 1 -- 172.21.15.70:0/2535228954 wait complete. 2023-10-08T05:49:22.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.217+0000 7fe54b59e640 1 Processor -- start 2023-10-08T05:49:22.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.217+0000 7fe54b59e640 1 -- start start 2023-10-08T05:49:22.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.218+0000 7fe54b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54c0b3c50 0x7fe54c1472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.218+0000 7fe54b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 0x7fe54c0b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.218+0000 7fe54b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c1599d0 0x7fe54c0a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.218+0000 7fe54b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c1477e0 con 0x7fe54c1599d0 2023-10-08T05:49:22.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.218+0000 7fe54b59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c0b3310 con 0x7fe54c0b3c50 2023-10-08T05:49:22.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.218+0000 7fe54b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c0aa4f0 con 0x7fe54c0cb5e0 2023-10-08T05:49:22.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.218+0000 7fe54ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c1599d0 0x7fe54c0a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.219+0000 7fe54ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c1599d0 0x7fe54c0a9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41452/0 (socket says 172.21.15.70:41452) 2023-10-08T05:49:22.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.219+0000 7fe54ad9d640 1 -- 172.21.15.70:0/1085870055 learned_addr learned my addr 172.21.15.70:0/1085870055 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:22.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.219+0000 7fe54a59c640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54c0b3c50 0x7fe54c1472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.219+0000 7fe541d9b640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 0x7fe54c0b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.219+0000 7fe54ad9d640 1 -- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 msgr2=0x7fe54c0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.219+0000 7fe54ad9d640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 0x7fe54c0b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.219+0000 7fe54ad9d640 1 -- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54c0b3c50 msgr2=0x7fe54c1472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.219+0000 7fe54ad9d640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54c0b3c50 0x7fe54c1472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.219+0000 7fe54ad9d640 1 -- 172.21.15.70:0/1085870055 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54400ba70 con 0x7fe54c1599d0 2023-10-08T05:49:22.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.219+0000 7fe54ad9d640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c1599d0 0x7fe54c0a9fb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe54400be60 tx=0x7fe544004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:22.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.220+0000 7fe5437fe640 1 -- 172.21.15.70:0/1085870055 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544009000 con 0x7fe54c1599d0 2023-10-08T05:49:22.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.220+0000 7fe5437fe640 1 -- 172.21.15.70:0/1085870055 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe544041640 con 0x7fe54c1599d0 2023-10-08T05:49:22.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.220+0000 7fe5437fe640 1 -- 172.21.15.70:0/1085870055 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5440419f0 con 0x7fe54c1599d0 2023-10-08T05:49:22.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.220+0000 7fe54b59e640 1 -- 172.21.15.70:0/1085870055 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54c147b00 con 0x7fe54c1599d0 2023-10-08T05:49:22.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.220+0000 7fe54b59e640 1 -- 172.21.15.70:0/1085870055 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54c1480c0 con 0x7fe54c1599d0 2023-10-08T05:49:22.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.221+0000 7fe54b59e640 1 -- 172.21.15.70:0/1085870055 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54c074150 con 0x7fe54c1599d0 2023-10-08T05:49:22.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.222+0000 7fe5437fe640 1 -- 172.21.15.70:0/1085870055 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fe54403d040 con 0x7fe54c1599d0 2023-10-08T05:49:22.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.223+0000 7fe5437fe640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe52c06b0a0 0x7fe52c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.223+0000 7fe5437fe640 1 -- 172.21.15.70:0/1085870055 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fe5440ac1c0 con 0x7fe54c1599d0 2023-10-08T05:49:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.224+0000 7fe54a59c640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe52c06b0a0 0x7fe52c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.225+0000 7fe54a59c640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe52c06b0a0 0x7fe52c06d560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe53c001370 tx=0x7fe53c009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:22.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.228+0000 7fe5437fe640 1 -- 172.21.15.70:0/1085870055 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe544076f40 con 0x7fe54c1599d0 2023-10-08T05:49:22.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.443+0000 7fe54b59e640 1 -- 172.21.15.70:0/1085870055 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe54c071990 con 0x7fe52c06b0a0 2023-10-08T05:49:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.604+0000 7fe5437fe640 1 -- 172.21.15.70:0/1085870055 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe53c007ac0 con 0x7fe52c06b0a0 2023-10-08T05:49:22.605 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-10-08T05:49:22.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.608+0000 7fe54b59e640 1 -- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe52c06b0a0 msgr2=0x7fe52c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.608+0000 7fe54b59e640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe52c06b0a0 0x7fe52c06d560 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe53c001370 tx=0x7fe53c009040 comp rx=0 tx=0).stop 2023-10-08T05:49:22.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.608+0000 7fe54b59e640 1 -- 172.21.15.70:0/1085870055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c1599d0 msgr2=0x7fe54c0a9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.608+0000 7fe54b59e640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c1599d0 0x7fe54c0a9fb0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe54400be60 tx=0x7fe544004a40 comp rx=0 tx=0).stop 2023-10-08T05:49:22.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.608+0000 7fe54b59e640 1 -- 172.21.15.70:0/1085870055 shutdown_connections 2023-10-08T05:49:22.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.608+0000 7fe54b59e640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe54c1599d0 0x7fe54c0a9fb0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.608+0000 7fe54b59e640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe52c06b0a0 0x7fe52c06d560 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.608+0000 7fe54b59e640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe54c0cb5e0 0x7fe54c0b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.608+0000 7fe54b59e640 1 --2- 172.21.15.70:0/1085870055 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe54c0b3c50 0x7fe54c1472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.608+0000 7fe54b59e640 1 -- 172.21.15.70:0/1085870055 >> 172.21.15.70:0/1085870055 conn(0x7fe54c06c050 msgr2=0x7fe54c0cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.609+0000 7fe54b59e640 1 -- 172.21.15.70:0/1085870055 shutdown_connections 2023-10-08T05:49:22.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.609+0000 7fe54b59e640 1 -- 172.21.15.70:0/1085870055 wait complete. 2023-10-08T05:49:22.629 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-10-08T05:49:22.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir2 1m 2023-10-08T05:49:22.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.850+0000 7f4189dae640 1 Processor -- start 2023-10-08T05:49:22.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.850+0000 7f4189dae640 1 -- start start 2023-10-08T05:49:22.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.852+0000 7f4189dae640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41840c8b40 0x7f41840c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.852+0000 7f4189dae640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41840ccbc0 con 0x7f41840aff70 2023-10-08T05:49:22.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.852+0000 7f4189dae640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41840ce1f0 con 0x7f418415cae0 2023-10-08T05:49:22.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.852+0000 7f4189dae640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41840c94c0 con 0x7f41840c8b40 2023-10-08T05:49:22.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.852+0000 7f41895ad640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f41840aff70 0x7f4184070180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49358/0 (socket says 172.21.15.70:49358) 2023-10-08T05:49:22.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.852+0000 7f41895ad640 1 -- 172.21.15.70:0/3146830170 learned_addr learned my addr 172.21.15.70:0/3146830170 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:22.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f4183fff640 1 --2- 172.21.15.70:0/3146830170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41840c8b40 0x7f41840c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639885020 0 0) 0x7f41840ccbc0 con 0x7f41840aff70 2023-10-08T05:49:22.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4174003520 con 0x7f41840aff70 2023-10-08T05:49:22.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3455867725 0 0) 0x7f4174003520 con 0x7f41840aff70 2023-10-08T05:49:22.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41840c8b40 msgr2=0x7f41840c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:22.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 --2- 172.21.15.70:0/3146830170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41840c8b40 0x7f41840c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 >> v1:172.21.15.181:6789/0 conn(0x7f418415cae0 legacy=0x7f41840cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840c9720 con 0x7f41840aff70 2023-10-08T05:49:22.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f417c0019f0 con 0x7f41840aff70 2023-10-08T05:49:22.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 >> v1:172.21.15.70:6789/0 conn(0x7f41840aff70 legacy=0x7f4184070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 --2- 172.21.15.70:0/3146830170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41740039a0 0x7f4174003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 --2- 172.21.15.70:0/3146830170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41740047c0 0x7f4174004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 --2- 172.21.15.70:0/3146830170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41840c8b40 0x7f4174005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41840c94c0 con 0x7f41740039a0 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417400ba30 con 0x7f41740047c0 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.853+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417400bbf0 con 0x7f41840c8b40 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.854+0000 7f4188dac640 1 --2- 172.21.15.70:0/3146830170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41740039a0 0x7f4174003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.854+0000 7f4188dac640 1 -- 172.21.15.70:0/3146830170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41840c8b40 msgr2=0x7f4174005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.854+0000 7f4188dac640 1 --2- 172.21.15.70:0/3146830170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41840c8b40 0x7f4174005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.854+0000 7f4188dac640 1 -- 172.21.15.70:0/3146830170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41740047c0 msgr2=0x7f4174004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.854+0000 7f4188dac640 1 --2- 172.21.15.70:0/3146830170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41740047c0 0x7f4174004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.854+0000 7f4188dac640 1 -- 172.21.15.70:0/3146830170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4174003520 con 0x7f41740039a0 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.855+0000 7f4188dac640 1 --2- 172.21.15.70:0/3146830170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41740039a0 0x7f4174003dc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4178001720 tx=0x7f41740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6012958949a57c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.855+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41780052d0 con 0x7f41740039a0 2023-10-08T05:49:22.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.855+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417802e050 con 0x7f41740039a0 2023-10-08T05:49:22.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.855+0000 7f41837fe640 1 -- 172.21.15.70:0/3146830170 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41780035f0 con 0x7f41740039a0 2023-10-08T05:49:22.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.856+0000 7f4189dae640 1 -- 172.21.15.70:0/3146830170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41740039a0 msgr2=0x7f4174003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.856+0000 7f4189dae640 1 --2- 172.21.15.70:0/3146830170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41740039a0 0x7f4174003dc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4178001720 tx=0x7f41740057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.856+0000 7f41895ad640 1 -- 172.21.15.70:0/3146830170 reap_dead start 2023-10-08T05:49:22.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.856+0000 7f4189dae640 1 -- 172.21.15.70:0/3146830170 shutdown_connections 2023-10-08T05:49:22.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.856+0000 7f4189dae640 1 -- 172.21.15.70:0/3146830170 >> 172.21.15.70:0/3146830170 conn(0x7f418406c050 msgr2=0x7f418406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:22.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.856+0000 7f4189dae640 1 -- 172.21.15.70:0/3146830170 shutdown_connections 2023-10-08T05:49:22.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.856+0000 7f4189dae640 1 -- 172.21.15.70:0/3146830170 wait complete. 2023-10-08T05:49:22.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.858+0000 7f4189dae640 1 Processor -- start 2023-10-08T05:49:22.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.858+0000 7f4189dae640 1 -- start start 2023-10-08T05:49:22.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.859+0000 7f4189dae640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41740039a0 0x7f4184155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.859+0000 7f4189dae640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4184151fc0 0x7f418414e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.859+0000 7f4189dae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41841523e0 0x7f418414ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.859+0000 7f4189dae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418415fd70 con 0x7f41841523e0 2023-10-08T05:49:22.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.859+0000 7f4189dae640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418414ef70 con 0x7f41740039a0 2023-10-08T05:49:22.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.859+0000 7f4189dae640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418415fa90 con 0x7f4184151fc0 2023-10-08T05:49:22.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.859+0000 7f41895ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41841523e0 0x7f418414ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.859+0000 7f41895ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41841523e0 0x7f418414ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41472/0 (socket says 172.21.15.70:41472) 2023-10-08T05:49:22.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.859+0000 7f41895ad640 1 -- 172.21.15.70:0/395658065 learned_addr learned my addr 172.21.15.70:0/395658065 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f4188dac640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41740039a0 0x7f4184155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f4183fff640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4184151fc0 0x7f418414e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f41895ad640 1 -- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4184151fc0 msgr2=0x7f418414e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f41895ad640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4184151fc0 0x7f418414e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f41895ad640 1 -- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41740039a0 msgr2=0x7f4184155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f41895ad640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41740039a0 0x7f4184155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f41895ad640 1 -- 172.21.15.70:0/395658065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840c30f0 con 0x7f41841523e0 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f4183fff640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4184151fc0 0x7f418414e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f41895ad640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41841523e0 0x7f418414ea30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f417c001560 tx=0x7f417c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f41817fa640 1 -- 172.21.15.70:0/395658065 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c00a6d0 con 0x7f41841523e0 2023-10-08T05:49:22.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f41817fa640 1 -- 172.21.15.70:0/395658065 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c00a830 con 0x7f41841523e0 2023-10-08T05:49:22.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f41817fa640 1 -- 172.21.15.70:0/395658065 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c00ac20 con 0x7f41841523e0 2023-10-08T05:49:22.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f4189dae640 1 -- 172.21.15.70:0/395658065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41840c33d0 con 0x7f41841523e0 2023-10-08T05:49:22.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.860+0000 7f4189dae640 1 -- 172.21.15.70:0/395658065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41840c3bd0 con 0x7f41841523e0 2023-10-08T05:49:22.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.861+0000 7f4189dae640 1 -- 172.21.15.70:0/395658065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41840c38f0 con 0x7f41841523e0 2023-10-08T05:49:22.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.862+0000 7f41817fa640 1 -- 172.21.15.70:0/395658065 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f417c00ad80 con 0x7f41841523e0 2023-10-08T05:49:22.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.863+0000 7f41817fa640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f416806b0a0 0x7f416806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:22.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.863+0000 7f41817fa640 1 -- 172.21.15.70:0/395658065 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f417c089560 con 0x7f41841523e0 2023-10-08T05:49:22.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.864+0000 7f4188dac640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f416806b0a0 0x7f416806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:22.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.865+0000 7f4188dac640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f416806b0a0 0x7f416806d560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4178033000 tx=0x7f4178009030 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:22.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:22.868+0000 7f41817fa640 1 -- 172.21.15.70:0/395658065 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41840c38f0 con 0x7f41841523e0 2023-10-08T05:49:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.082+0000 7f4189dae640 1 -- 172.21.15.70:0/395658065 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f41840cc850 con 0x7f416806b0a0 2023-10-08T05:49:23.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.247+0000 7f41817fa640 1 -- 172.21.15.70:0/395658065 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f41780053e0 con 0x7f416806b0a0 2023-10-08T05:49:23.248 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-10-08T05:49:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 -- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f416806b0a0 msgr2=0x7f416806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f416806b0a0 0x7f416806d560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4178033000 tx=0x7f4178009030 comp rx=0 tx=0).stop 2023-10-08T05:49:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 -- 172.21.15.70:0/395658065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41841523e0 msgr2=0x7f418414ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41841523e0 0x7f418414ea30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f417c001560 tx=0x7f417c00cf00 comp rx=0 tx=0).stop 2023-10-08T05:49:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 -- 172.21.15.70:0/395658065 shutdown_connections 2023-10-08T05:49:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41841523e0 0x7f418414ea30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f416806b0a0 0x7f416806d560 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4184151fc0 0x7f418414e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 --2- 172.21.15.70:0/395658065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f41740039a0 0x7f4184155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 -- 172.21.15.70:0/395658065 >> 172.21.15.70:0/395658065 conn(0x7f418406c050 msgr2=0x7f418406c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:23.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 -- 172.21.15.70:0/395658065 shutdown_connections 2023-10-08T05:49:23.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.251+0000 7f4189dae640 1 -- 172.21.15.70:0/395658065 wait complete. 2023-10-08T05:49:23.272 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-10-08T05:49:23.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir3 1m 2023-10-08T05:49:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.493+0000 7f88793a9640 1 Processor -- start 2023-10-08T05:49:23.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.493+0000 7f88793a9640 1 -- start start 2023-10-08T05:49:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.494+0000 7f88793a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88740afbe0 0x7f88740b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.494+0000 7f88793a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88740cb130 0x7f88740cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.495+0000 7f88793a9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88740b2620 con 0x7f887406dfc0 2023-10-08T05:49:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.495+0000 7f88793a9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88740cbae0 con 0x7f88740cb130 2023-10-08T05:49:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.495+0000 7f88793a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88740b2060 con 0x7f88740afbe0 2023-10-08T05:49:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.495+0000 7f88737fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88740cb130 0x7f88740cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.495+0000 7f88737fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88740cb130 0x7f88740cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41476/0 (socket says 172.21.15.70:41476) 2023-10-08T05:49:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.495+0000 7f88737fe640 1 -- 172.21.15.70:0/1427731248 learned_addr learned my addr 172.21.15.70:0/1427731248 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.495+0000 7f8872ffd640 1 --2- 172.21.15.70:0/1427731248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88740afbe0 0x7f88740b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.495+0000 7f88737fe640 1 -- 172.21.15.70:0/1427731248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88740afbe0 msgr2=0x7f88740b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.495+0000 7f88737fe640 1 --2- 172.21.15.70:0/1427731248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88740afbe0 0x7f88740b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.495+0000 7f88737fe640 1 -- 172.21.15.70:0/1427731248 >> v1:172.21.15.181:6789/0 conn(0x7f887406dfc0 legacy=0x7f8874070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.496+0000 7f88737fe640 1 -- 172.21.15.70:0/1427731248 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740b2340 con 0x7f88740cb130 2023-10-08T05:49:23.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.497+0000 7f88737fe640 1 --2- 172.21.15.70:0/1427731248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88740cb130 0x7f88740cb5a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f886c009870 tx=0x7f886c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ae1553a53e3740 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:23.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.497+0000 7f8871ffb640 1 -- 172.21.15.70:0/1427731248 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c031640 con 0x7f88740cb130 2023-10-08T05:49:23.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.497+0000 7f8871ffb640 1 -- 172.21.15.70:0/1427731248 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c0317a0 con 0x7f88740cb130 2023-10-08T05:49:23.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.497+0000 7f8871ffb640 1 -- 172.21.15.70:0/1427731248 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c031b30 con 0x7f88740cb130 2023-10-08T05:49:23.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.498+0000 7f88793a9640 1 -- 172.21.15.70:0/1427731248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88740cb130 msgr2=0x7f88740cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:23.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.498+0000 7f88793a9640 1 --2- 172.21.15.70:0/1427731248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88740cb130 0x7f88740cb5a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f886c009870 tx=0x7f886c02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.498+0000 7f88793a9640 1 -- 172.21.15.70:0/1427731248 shutdown_connections 2023-10-08T05:49:23.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.498+0000 7f88793a9640 1 --2- 172.21.15.70:0/1427731248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88740cb130 0x7f88740cb5a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.498+0000 7f88793a9640 1 --2- 172.21.15.70:0/1427731248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88740afbe0 0x7f88740b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.498+0000 7f88793a9640 1 -- 172.21.15.70:0/1427731248 >> 172.21.15.70:0/1427731248 conn(0x7f887406a5d0 msgr2=0x7f88740b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:23.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.498+0000 7f88793a9640 1 -- 172.21.15.70:0/1427731248 shutdown_connections 2023-10-08T05:49:23.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.498+0000 7f88793a9640 1 -- 172.21.15.70:0/1427731248 wait complete. 2023-10-08T05:49:23.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.500+0000 7f88793a9640 1 Processor -- start 2023-10-08T05:49:23.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.500+0000 7f88793a9640 1 -- start start 2023-10-08T05:49:23.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.501+0000 7f88793a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887406dfc0 0x7f88741f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:23.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.501+0000 7f88793a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740afbe0 0x7f887415b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:23.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.501+0000 7f88793a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88740cb130 0x7f887414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:23.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.501+0000 7f88793a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88740ca060 con 0x7f887406dfc0 2023-10-08T05:49:23.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.501+0000 7f88793a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887415c340 con 0x7f88740afbe0 2023-10-08T05:49:23.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.501+0000 7f88793a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887415c060 con 0x7f88740cb130 2023-10-08T05:49:23.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.501+0000 7f8872ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887406dfc0 0x7f88741f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:23.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.501+0000 7f8872ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887406dfc0 0x7f88741f2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41486/0 (socket says 172.21.15.70:41486) 2023-10-08T05:49:23.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.501+0000 7f8872ffd640 1 -- 172.21.15.70:0/2355582694 learned_addr learned my addr 172.21.15.70:0/2355582694 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:23.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f88737fe640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88740cb130 0x7f887414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:23.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f88727fc640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740afbe0 0x7f887415b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:23.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f8872ffd640 1 -- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88740cb130 msgr2=0x7f887414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:23.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f8872ffd640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88740cb130 0x7f887414d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f8872ffd640 1 -- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740afbe0 msgr2=0x7f887415b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:23.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f8872ffd640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740afbe0 0x7f887415b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f8872ffd640 1 -- 172.21.15.70:0/2355582694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c0094d0 con 0x7f887406dfc0 2023-10-08T05:49:23.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f88727fc640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740afbe0 0x7f887415b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:23.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f8872ffd640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887406dfc0 0x7f88741f2070 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f886400cd00 tx=0x7f886400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:23.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f886b7fe640 1 -- 172.21.15.70:0/2355582694 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864003990 con 0x7f887406dfc0 2023-10-08T05:49:23.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f886b7fe640 1 -- 172.21.15.70:0/2355582694 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8864003af0 con 0x7f887406dfc0 2023-10-08T05:49:23.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f886b7fe640 1 -- 172.21.15.70:0/2355582694 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864003e80 con 0x7f887406dfc0 2023-10-08T05:49:23.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.502+0000 7f88793a9640 1 -- 172.21.15.70:0/2355582694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88740c9a10 con 0x7f887406dfc0 2023-10-08T05:49:23.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.503+0000 7f88793a9640 1 -- 172.21.15.70:0/2355582694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88741f25b0 con 0x7f887406dfc0 2023-10-08T05:49:23.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.504+0000 7f88793a9640 1 -- 172.21.15.70:0/2355582694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887404e160 con 0x7f887406dfc0 2023-10-08T05:49:23.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.505+0000 7f886b7fe640 1 -- 172.21.15.70:0/2355582694 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f8864004020 con 0x7f887406dfc0 2023-10-08T05:49:23.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.505+0000 7f886b7fe640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f885406b070 0x7f885406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:23.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.506+0000 7f886b7fe640 1 -- 172.21.15.70:0/2355582694 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f886408a130 con 0x7f887406dfc0 2023-10-08T05:49:23.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.506+0000 7f88727fc640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f885406b070 0x7f885406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:23.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.507+0000 7f88727fc640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f885406b070 0x7f885406d530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f88741f5d60 tx=0x7f885c008040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:23.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.510+0000 7f886b7fe640 1 -- 172.21.15.70:0/2355582694 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8864054be0 con 0x7f887406dfc0 2023-10-08T05:49:23.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.722+0000 7f88793a9640 1 -- 172.21.15.70:0/2355582694 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f8874070630 con 0x7f885406b070 2023-10-08T05:49:23.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.888+0000 7f886b7fe640 1 -- 172.21.15.70:0/2355582694 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f885c00ad80 con 0x7f885406b070 2023-10-08T05:49:23.889 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-10-08T05:49:23.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.891+0000 7f88793a9640 1 -- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f885406b070 msgr2=0x7f885406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:23.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.891+0000 7f88793a9640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f885406b070 0x7f885406d530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f88741f5d60 tx=0x7f885c008040 comp rx=0 tx=0).stop 2023-10-08T05:49:23.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.892+0000 7f88793a9640 1 -- 172.21.15.70:0/2355582694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887406dfc0 msgr2=0x7f88741f2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:23.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.892+0000 7f88793a9640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887406dfc0 0x7f88741f2070 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f886400cd00 tx=0x7f886400a5c0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.892+0000 7f88793a9640 1 -- 172.21.15.70:0/2355582694 shutdown_connections 2023-10-08T05:49:23.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.892+0000 7f88793a9640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f885406b070 0x7f885406d530 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.892+0000 7f88793a9640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f88740cb130 0x7f887414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.892+0000 7f88793a9640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88740afbe0 0x7f887415b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.892+0000 7f88793a9640 1 --2- 172.21.15.70:0/2355582694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f887406dfc0 0x7f88741f2070 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:23.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.892+0000 7f88793a9640 1 -- 172.21.15.70:0/2355582694 >> 172.21.15.70:0/2355582694 conn(0x7f887406a5d0 msgr2=0x7f88740c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:23.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.892+0000 7f88793a9640 1 -- 172.21.15.70:0/2355582694 shutdown_connections 2023-10-08T05:49:23.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:23.892+0000 7f88793a9640 1 -- 172.21.15.70:0/2355582694 wait complete. 2023-10-08T05:49:23.912 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-10-08T05:49:23.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir4 1m 2023-10-08T05:49:24.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.125+0000 7f527fbb6640 1 Processor -- start 2023-10-08T05:49:24.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.125+0000 7f527fbb6640 1 -- start start 2023-10-08T05:49:24.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.126+0000 7f527fbb6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 0x7f52780c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.126+0000 7f527fbb6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f527815aa00 0x7f52780b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.126+0000 7f527fbb6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527815b310 con 0x7f527806db60 2023-10-08T05:49:24.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.126+0000 7f527fbb6640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52780ca230 con 0x7f527815aa00 2023-10-08T05:49:24.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.126+0000 7f527fbb6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527806d290 con 0x7f527806c3a0 2023-10-08T05:49:24.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.127+0000 7f527d952640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 0x7f52780c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.127+0000 7f527d952640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 0x7f52780c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:36150/0 (socket says 172.21.15.70:36150) 2023-10-08T05:49:24.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.127+0000 7f527d952640 1 -- 172.21.15.70:0/2685893345 learned_addr learned my addr 172.21.15.70:0/2685893345 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:24.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.127+0000 7f527d151640 1 --1- 172.21.15.70:0/2685893345 >> v1:172.21.15.181:6789/0 conn(0x7f527806db60 0x7f527806c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:35978/0 (socket says 172.21.15.70:35978) 2023-10-08T05:49:24.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.127+0000 7f527e153640 1 --2- 172.21.15.70:0/2685893345 >> v2:172.21.15.181:3300/0 conn(0x7f527815aa00 0x7f52780b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.128+0000 7f527c950640 1 -- 172.21.15.70:0/2685893345 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966203477 0 0) 0x7f5274001110 con 0x7f527806db60 2023-10-08T05:49:24.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.128+0000 7f527d952640 1 -- 172.21.15.70:0/2685893345 >> v2:172.21.15.181:3300/0 conn(0x7f527815aa00 msgr2=0x7f52780b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:24.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.128+0000 7f527d952640 1 --2- 172.21.15.70:0/2685893345 >> v2:172.21.15.181:3300/0 conn(0x7f527815aa00 0x7f52780b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.128+0000 7f527d952640 1 -- 172.21.15.70:0/2685893345 >> v1:172.21.15.181:6789/0 conn(0x7f527806db60 legacy=0x7f527806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:24.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.128+0000 7f527d952640 1 -- 172.21.15.70:0/2685893345 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52780c9860 con 0x7f527806c3a0 2023-10-08T05:49:24.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.130+0000 7f527d952640 1 --2- 172.21.15.70:0/2685893345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 0x7f52780c9cf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f526c009870 tx=0x7f526c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81cc1ba6772840aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:24.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.130+0000 7f527c950640 1 -- 172.21.15.70:0/2685893345 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c030840 con 0x7f527806c3a0 2023-10-08T05:49:24.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.130+0000 7f527c950640 1 -- 172.21.15.70:0/2685893345 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c0309a0 con 0x7f527806c3a0 2023-10-08T05:49:24.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.130+0000 7f527c950640 1 -- 172.21.15.70:0/2685893345 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c030d30 con 0x7f527806c3a0 2023-10-08T05:49:24.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.131+0000 7f527fbb6640 1 -- 172.21.15.70:0/2685893345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 msgr2=0x7f52780c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:24.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.131+0000 7f527fbb6640 1 --2- 172.21.15.70:0/2685893345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 0x7f52780c9cf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f526c009870 tx=0x7f526c02b980 comp rx=0 tx=0).stop 2023-10-08T05:49:24.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.131+0000 7f527fbb6640 1 -- 172.21.15.70:0/2685893345 shutdown_connections 2023-10-08T05:49:24.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.131+0000 7f527fbb6640 1 --2- 172.21.15.70:0/2685893345 >> v2:172.21.15.181:3300/0 conn(0x7f527815aa00 0x7f52780b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.131+0000 7f527fbb6640 1 --2- 172.21.15.70:0/2685893345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 0x7f52780c9cf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.131+0000 7f527fbb6640 1 -- 172.21.15.70:0/2685893345 >> 172.21.15.70:0/2685893345 conn(0x7f527806a5d0 msgr2=0x7f52780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:24.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.131+0000 7f527fbb6640 1 -- 172.21.15.70:0/2685893345 shutdown_connections 2023-10-08T05:49:24.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.131+0000 7f527fbb6640 1 -- 172.21.15.70:0/2685893345 wait complete. 2023-10-08T05:49:24.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.133+0000 7f527fbb6640 1 Processor -- start 2023-10-08T05:49:24.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.133+0000 7f527fbb6640 1 -- start start 2023-10-08T05:49:24.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.134+0000 7f527fbb6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 0x7f52781f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.134+0000 7f527fbb6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527806db60 0x7f527814fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.134+0000 7f527fbb6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527815aa00 0x7f52780c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.134+0000 7f527fbb6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52781f2e00 con 0x7f527815aa00 2023-10-08T05:49:24.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.134+0000 7f527fbb6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527815b4b0 con 0x7f527806db60 2023-10-08T05:49:24.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.134+0000 7f527fbb6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527815c310 con 0x7f527806c3a0 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.134+0000 7f527e153640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527815aa00 0x7f52780c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.134+0000 7f527e153640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527815aa00 0x7f52780c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43864/0 (socket says 172.21.15.70:43864) 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.134+0000 7f527e153640 1 -- 172.21.15.70:0/2283643931 learned_addr learned my addr 172.21.15.70:0/2283643931 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f527d151640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527806db60 0x7f527814fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f527d952640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 0x7f52781f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f527e153640 1 -- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 msgr2=0x7f52781f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f527e153640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 0x7f52781f28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f527e153640 1 -- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527806db60 msgr2=0x7f527814fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f527e153640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527806db60 0x7f527814fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f527e153640 1 -- 172.21.15.70:0/2283643931 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c0094d0 con 0x7f527815aa00 2023-10-08T05:49:24.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f527e153640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527815aa00 0x7f52780c2c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5270009d30 tx=0x7f527000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:24.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f526a7fc640 1 -- 172.21.15.70:0/2283643931 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270009ba0 con 0x7f527815aa00 2023-10-08T05:49:24.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f526a7fc640 1 -- 172.21.15.70:0/2283643931 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527000f040 con 0x7f527815aa00 2023-10-08T05:49:24.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f527fbb6640 1 -- 172.21.15.70:0/2283643931 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527814ff70 con 0x7f527815aa00 2023-10-08T05:49:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.135+0000 7f526a7fc640 1 -- 172.21.15.70:0/2283643931 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52700146c0 con 0x7f527815aa00 2023-10-08T05:49:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.136+0000 7f527fbb6640 1 -- 172.21.15.70:0/2283643931 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527815b9e0 con 0x7f527815aa00 2023-10-08T05:49:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.137+0000 7f527fbb6640 1 -- 172.21.15.70:0/2283643931 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527815c4a0 con 0x7f527815aa00 2023-10-08T05:49:24.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.138+0000 7f526a7fc640 1 -- 172.21.15.70:0/2283643931 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f5270014820 con 0x7f527815aa00 2023-10-08T05:49:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.139+0000 7f526a7fc640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f525c06b0a0 0x7f525c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.139+0000 7f526a7fc640 1 -- 172.21.15.70:0/2283643931 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f527008aa10 con 0x7f527815aa00 2023-10-08T05:49:24.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.140+0000 7f527d952640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f525c06b0a0 0x7f525c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.141+0000 7f527d952640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f525c06b0a0 0x7f525c06d560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f526c036000 tx=0x7f526c03b040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:24.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.144+0000 7f526a7fc640 1 -- 172.21.15.70:0/2283643931 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5270055560 con 0x7f527815aa00 2023-10-08T05:49:24.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.356+0000 7f527fbb6640 1 -- 172.21.15.70:0/2283643931 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f52780c9210 con 0x7f525c06b0a0 2023-10-08T05:49:24.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.524+0000 7f526a7fc640 1 -- 172.21.15.70:0/2283643931 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f526c030b90 con 0x7f525c06b0a0 2023-10-08T05:49:24.525 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-10-08T05:49:24.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.527+0000 7f527fbb6640 1 -- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f525c06b0a0 msgr2=0x7f525c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:24.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.527+0000 7f527fbb6640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f525c06b0a0 0x7f525c06d560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f526c036000 tx=0x7f526c03b040 comp rx=0 tx=0).stop 2023-10-08T05:49:24.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.527+0000 7f527fbb6640 1 -- 172.21.15.70:0/2283643931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527815aa00 msgr2=0x7f52780c2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:24.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.527+0000 7f527fbb6640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527815aa00 0x7f52780c2c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5270009d30 tx=0x7f527000c490 comp rx=0 tx=0).stop 2023-10-08T05:49:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.528+0000 7f527fbb6640 1 -- 172.21.15.70:0/2283643931 shutdown_connections 2023-10-08T05:49:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.528+0000 7f527fbb6640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f527815aa00 0x7f52780c2c80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.528+0000 7f527fbb6640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f527806db60 0x7f527814fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.528+0000 7f527fbb6640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f525c06b0a0 0x7f525c06d560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.528+0000 7f527fbb6640 1 --2- 172.21.15.70:0/2283643931 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f527806c3a0 0x7f52781f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.528+0000 7f527fbb6640 1 -- 172.21.15.70:0/2283643931 >> 172.21.15.70:0/2283643931 conn(0x7f527806a5d0 msgr2=0x7f52780b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.528+0000 7f527fbb6640 1 -- 172.21.15.70:0/2283643931 shutdown_connections 2023-10-08T05:49:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.528+0000 7f527fbb6640 1 -- 172.21.15.70:0/2283643931 wait complete. 2023-10-08T05:49:24.548 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-10-08T05:49:24.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir5 1m 2023-10-08T05:49:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.761+0000 7fa2ad906640 1 Processor -- start 2023-10-08T05:49:24.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.762+0000 7fa2ad906640 1 -- start start 2023-10-08T05:49:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.763+0000 7fa2ad906640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a81599d0 0x7fa2a80cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.763+0000 7fa2ad906640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa2a80cb5e0 0x7fa2a80cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.763+0000 7fa2ad906640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a8159db0 con 0x7fa2a80b3c50 2023-10-08T05:49:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.763+0000 7fa2ad906640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a8071fe0 con 0x7fa2a80cb5e0 2023-10-08T05:49:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.763+0000 7fa2ad906640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80b44c0 con 0x7fa2a81599d0 2023-10-08T05:49:24.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.764+0000 7fa2ad105640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa2a80cb5e0 0x7fa2a80cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.764+0000 7fa2ad105640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa2a80cb5e0 0x7fa2a80cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43878/0 (socket says 172.21.15.70:43878) 2023-10-08T05:49:24.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.764+0000 7fa2ad105640 1 -- 172.21.15.70:0/264314471 learned_addr learned my addr 172.21.15.70:0/264314471 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:24.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.764+0000 7fa2ac904640 1 --2- 172.21.15.70:0/264314471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a81599d0 0x7fa2a80cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.764+0000 7fa2ad105640 1 -- 172.21.15.70:0/264314471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a81599d0 msgr2=0x7fa2a80cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:24.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.764+0000 7fa2ad105640 1 --2- 172.21.15.70:0/264314471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a81599d0 0x7fa2a80cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.764+0000 7fa2ad105640 1 -- 172.21.15.70:0/264314471 >> v1:172.21.15.181:6789/0 conn(0x7fa2a80b3c50 legacy=0x7fa2a80caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:24.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.764+0000 7fa2ad105640 1 -- 172.21.15.70:0/264314471 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80713d0 con 0x7fa2a80cb5e0 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.765+0000 7fa2ad105640 1 --2- 172.21.15.70:0/264314471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80cb5e0 0x7fa2a80cc0c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009e50 tx=0x7fa2a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e93fb451063037 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.765+0000 7fa2a77fe640 1 -- 172.21.15.70:0/264314471 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a002ca70 con 0x7fa2a80cb5e0 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.766+0000 7fa2a77fe640 1 -- 172.21.15.70:0/264314471 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a002cbd0 con 0x7fa2a80cb5e0 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.766+0000 7fa2a77fe640 1 -- 172.21.15.70:0/264314471 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00343f0 con 0x7fa2a80cb5e0 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.766+0000 7fa2ad906640 1 -- 172.21.15.70:0/264314471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80cb5e0 msgr2=0x7fa2a80cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.766+0000 7fa2ad906640 1 --2- 172.21.15.70:0/264314471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80cb5e0 0x7fa2a80cc0c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009e50 tx=0x7fa2a002bed0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.766+0000 7fa2ad906640 1 -- 172.21.15.70:0/264314471 shutdown_connections 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.766+0000 7fa2ad906640 1 --2- 172.21.15.70:0/264314471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80cb5e0 0x7fa2a80cc0c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.766+0000 7fa2ad906640 1 --2- 172.21.15.70:0/264314471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a81599d0 0x7fa2a80cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.766+0000 7fa2ad906640 1 -- 172.21.15.70:0/264314471 >> 172.21.15.70:0/264314471 conn(0x7fa2a806c050 msgr2=0x7fa2a806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:24.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.766+0000 7fa2ad906640 1 -- 172.21.15.70:0/264314471 shutdown_connections 2023-10-08T05:49:24.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.767+0000 7fa2ad906640 1 -- 172.21.15.70:0/264314471 wait complete. 2023-10-08T05:49:24.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.769+0000 7fa2ad906640 1 Processor -- start 2023-10-08T05:49:24.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.769+0000 7fa2ad906640 1 -- start start 2023-10-08T05:49:24.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.770+0000 7fa2ad906640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80b3c50 0x7fa2a80c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.770+0000 7fa2ad906640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a80cb5e0 0x7fa2a80c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.770+0000 7fa2ad906640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a81599d0 0x7fa2a8151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.770+0000 7fa2ad906640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80c26f0 con 0x7fa2a81599d0 2023-10-08T05:49:24.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.770+0000 7fa2ad906640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80c2ee0 con 0x7fa2a80b3c50 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.770+0000 7fa2ad906640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80b18f0 con 0x7fa2a80cb5e0 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.770+0000 7fa2ad105640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a81599d0 0x7fa2a8151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2ad105640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a81599d0 0x7fa2a8151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43892/0 (socket says 172.21.15.70:43892) 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2ad105640 1 -- 172.21.15.70:0/424977518 learned_addr learned my addr 172.21.15.70:0/424977518 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2ac904640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80b3c50 0x7fa2a80c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2a7fff640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a80cb5e0 0x7fa2a80c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2ad105640 1 -- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a80cb5e0 msgr2=0x7fa2a80c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2ad105640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a80cb5e0 0x7fa2a80c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2ad105640 1 -- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80b3c50 msgr2=0x7fa2a80c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2ad105640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80b3c50 0x7fa2a80c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2ad105640 1 -- 172.21.15.70:0/424977518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a0009b30 con 0x7fa2a81599d0 2023-10-08T05:49:24.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2ac904640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80b3c50 0x7fa2a80c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:49:24.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.771+0000 7fa2ad105640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a81599d0 0x7fa2a8151a30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0036040 tx=0x7fa2a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:24.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.772+0000 7fa2a57fa640 1 -- 172.21.15.70:0/424977518 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a003e450 con 0x7fa2a81599d0 2023-10-08T05:49:24.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.772+0000 7fa2a57fa640 1 -- 172.21.15.70:0/424977518 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a003e5d0 con 0x7fa2a81599d0 2023-10-08T05:49:24.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.772+0000 7fa2a57fa640 1 -- 172.21.15.70:0/424977518 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a003e980 con 0x7fa2a81599d0 2023-10-08T05:49:24.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.772+0000 7fa2ad906640 1 -- 172.21.15.70:0/424977518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a8151f70 con 0x7fa2a81599d0 2023-10-08T05:49:24.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.772+0000 7fa2ad906640 1 -- 172.21.15.70:0/424977518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a80c33f0 con 0x7fa2a81599d0 2023-10-08T05:49:24.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.773+0000 7fa2ad906640 1 -- 172.21.15.70:0/424977518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a8074150 con 0x7fa2a81599d0 2023-10-08T05:49:24.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.774+0000 7fa2a57fa640 1 -- 172.21.15.70:0/424977518 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fa2a003eae0 con 0x7fa2a81599d0 2023-10-08T05:49:24.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.775+0000 7fa2a57fa640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa28c06b0a0 0x7fa28c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:24.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.775+0000 7fa2a57fa640 1 -- 172.21.15.70:0/424977518 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fa2a00ac880 con 0x7fa2a81599d0 2023-10-08T05:49:24.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.776+0000 7fa2ac904640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa28c06b0a0 0x7fa28c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:24.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.777+0000 7fa2ac904640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa28c06b0a0 0x7fa28c06d560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa29c006fd0 tx=0x7fa29c008040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:24.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.781+0000 7fa2a57fa640 1 -- 172.21.15.70:0/424977518 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2a00774c0 con 0x7fa2a81599d0 2023-10-08T05:49:24.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:24.992+0000 7fa2ad906640 1 -- 172.21.15.70:0/424977518 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2a8071990 con 0x7fa28c06b0a0 2023-10-08T05:49:25.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.161+0000 7fa2a57fa640 1 -- 172.21.15.70:0/424977518 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fa29c00c050 con 0x7fa28c06b0a0 2023-10-08T05:49:25.161 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-10-08T05:49:25.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.164+0000 7fa2ad906640 1 -- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa28c06b0a0 msgr2=0x7fa28c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:25.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa28c06b0a0 0x7fa28c06d560 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa29c006fd0 tx=0x7fa29c008040 comp rx=0 tx=0).stop 2023-10-08T05:49:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 -- 172.21.15.70:0/424977518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a81599d0 msgr2=0x7fa2a8151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a81599d0 0x7fa2a8151a30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0036040 tx=0x7fa2a0004a40 comp rx=0 tx=0).stop 2023-10-08T05:49:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 -- 172.21.15.70:0/424977518 shutdown_connections 2023-10-08T05:49:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a81599d0 0x7fa2a8151a30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa28c06b0a0 0x7fa28c06d560 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a80cb5e0 0x7fa2a80c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 --2- 172.21.15.70:0/424977518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80b3c50 0x7fa2a80c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 -- 172.21.15.70:0/424977518 >> 172.21.15.70:0/424977518 conn(0x7fa2a806c050 msgr2=0x7fa2a80ca610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 -- 172.21.15.70:0/424977518 shutdown_connections 2023-10-08T05:49:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.165+0000 7fa2ad906640 1 -- 172.21.15.70:0/424977518 wait complete. 2023-10-08T05:49:25.186 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-10-08T05:49:25.186 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir6 1m 2023-10-08T05:49:25.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.404+0000 7f87bfbf5640 1 Processor -- start 2023-10-08T05:49:25.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.404+0000 7f87bfbf5640 1 -- start start 2023-10-08T05:49:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.405+0000 7f87bfbf5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87b80702b0 0x7f87b8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.405+0000 7f87bfbf5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87b80b21f0 0x7f87b80c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bfbf5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b806c3a0 con 0x7f87b80ca1a0 2023-10-08T05:49:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bfbf5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b8070bd0 con 0x7f87b80702b0 2023-10-08T05:49:25.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bfbf5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b806c640 con 0x7f87b80b21f0 2023-10-08T05:49:25.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bd991640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87b80702b0 0x7f87b8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:25.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87be192640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f87b80ca1a0 0x7f87b806d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44346/0 (socket says 172.21.15.70:44346) 2023-10-08T05:49:25.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87be192640 1 -- 172.21.15.70:0/1545066482 learned_addr learned my addr 172.21.15.70:0/1545066482 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bd991640 1 -- 172.21.15.70:0/1545066482 >> v2:172.21.15.181:3300/0 conn(0x7f87b80b21f0 msgr2=0x7f87b80c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bd190640 1 --2- 172.21.15.70:0/1545066482 >> v2:172.21.15.181:3300/0 conn(0x7f87b80b21f0 0x7f87b80c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bd991640 1 --2- 172.21.15.70:0/1545066482 >> v2:172.21.15.181:3300/0 conn(0x7f87b80b21f0 0x7f87b80c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bd991640 1 -- 172.21.15.70:0/1545066482 >> v1:172.21.15.70:6789/0 conn(0x7f87b80ca1a0 legacy=0x7f87b806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bd991640 1 -- 172.21.15.70:0/1545066482 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b806c920 con 0x7f87b80702b0 2023-10-08T05:49:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bd190640 1 --2- 172.21.15.70:0/1545066482 >> v2:172.21.15.181:3300/0 conn(0x7f87b80b21f0 0x7f87b80c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.406+0000 7f87bc98f640 1 -- 172.21.15.70:0/1545066482 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524238016 0 0) 0x7f87b806c3a0 con 0x7f87b80ca1a0 2023-10-08T05:49:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.408+0000 7f87bd991640 1 --2- 172.21.15.70:0/1545066482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80702b0 0x7f87b8070690 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f87b80af860 tx=0x7f87b402cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb928548d2c9a78 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:25.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.408+0000 7f87bc98f640 1 -- 172.21.15.70:0/1545066482 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b402db60 con 0x7f87b80702b0 2023-10-08T05:49:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.408+0000 7f87bc98f640 1 -- 172.21.15.70:0/1545066482 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b402dcc0 con 0x7f87b80702b0 2023-10-08T05:49:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.408+0000 7f87bc98f640 1 -- 172.21.15.70:0/1545066482 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4035660 con 0x7f87b80702b0 2023-10-08T05:49:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.408+0000 7f87bfbf5640 1 -- 172.21.15.70:0/1545066482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80702b0 msgr2=0x7f87b8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.408+0000 7f87bfbf5640 1 --2- 172.21.15.70:0/1545066482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80702b0 0x7f87b8070690 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f87b80af860 tx=0x7f87b402cdb0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.408+0000 7f87bfbf5640 1 -- 172.21.15.70:0/1545066482 shutdown_connections 2023-10-08T05:49:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.408+0000 7f87bfbf5640 1 --2- 172.21.15.70:0/1545066482 >> v2:172.21.15.181:3300/0 conn(0x7f87b80b21f0 0x7f87b80c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.409+0000 7f87bfbf5640 1 --2- 172.21.15.70:0/1545066482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80702b0 0x7f87b8070690 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.409+0000 7f87bfbf5640 1 -- 172.21.15.70:0/1545066482 >> 172.21.15.70:0/1545066482 conn(0x7f87b806a5d0 msgr2=0x7f87b80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.409+0000 7f87bfbf5640 1 -- 172.21.15.70:0/1545066482 shutdown_connections 2023-10-08T05:49:25.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.409+0000 7f87bfbf5640 1 -- 172.21.15.70:0/1545066482 wait complete. 2023-10-08T05:49:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.410+0000 7f87bfbf5640 1 Processor -- start 2023-10-08T05:49:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.410+0000 7f87bfbf5640 1 -- start start 2023-10-08T05:49:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.411+0000 7f87bfbf5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b80702b0 0x7f87b8158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.411+0000 7f87bfbf5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80b21f0 0x7f87b81e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.411+0000 7f87bfbf5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b80ca1a0 0x7f87b81e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.411+0000 7f87bfbf5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b815c5d0 con 0x7f87b80b21f0 2023-10-08T05:49:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.411+0000 7f87bfbf5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b815b9c0 con 0x7f87b80ca1a0 2023-10-08T05:49:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.411+0000 7f87bfbf5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b815bca0 con 0x7f87b80702b0 2023-10-08T05:49:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.412+0000 7f87bd190640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80b21f0 0x7f87b81e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.412+0000 7f87bd190640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80b21f0 0x7f87b81e28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43904/0 (socket says 172.21.15.70:43904) 2023-10-08T05:49:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.412+0000 7f87bd190640 1 -- 172.21.15.70:0/2456525029 learned_addr learned my addr 172.21.15.70:0/2456525029 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.412+0000 7f87be192640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b80ca1a0 0x7f87b81e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.412+0000 7f87bd190640 1 -- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b80702b0 msgr2=0x7f87b8158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.412+0000 7f87bd190640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b80702b0 0x7f87b8158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.412+0000 7f87bd190640 1 -- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b80ca1a0 msgr2=0x7f87b81e52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.412+0000 7f87bd190640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b80ca1a0 0x7f87b81e52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.412+0000 7f87bd190640 1 -- 172.21.15.70:0/2456525029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b400a5a0 con 0x7f87b80b21f0 2023-10-08T05:49:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.412+0000 7f87bd190640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80b21f0 0x7f87b81e28b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c7f0 tx=0x7f87a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.413+0000 7f87ae7fc640 1 -- 172.21.15.70:0/2456525029 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8007a30 con 0x7f87b80b21f0 2023-10-08T05:49:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.413+0000 7f87ae7fc640 1 -- 172.21.15.70:0/2456525029 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a8007b90 con 0x7f87b80b21f0 2023-10-08T05:49:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.413+0000 7f87ae7fc640 1 -- 172.21.15.70:0/2456525029 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a800ce40 con 0x7f87b80b21f0 2023-10-08T05:49:25.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.413+0000 7f87bfbf5640 1 -- 172.21.15.70:0/2456525029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b815bf80 con 0x7f87b80b21f0 2023-10-08T05:49:25.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.413+0000 7f87bfbf5640 1 -- 172.21.15.70:0/2456525029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b8158fb0 con 0x7f87b80b21f0 2023-10-08T05:49:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.414+0000 7f87bfbf5640 1 -- 172.21.15.70:0/2456525029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b804e160 con 0x7f87b80b21f0 2023-10-08T05:49:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.415+0000 7f87ae7fc640 1 -- 172.21.15.70:0/2456525029 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f87a80113f0 con 0x7f87b80b21f0 2023-10-08T05:49:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.416+0000 7f87ae7fc640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f879c06b070 0x7f879c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.416+0000 7f87ae7fc640 1 -- 172.21.15.70:0/2456525029 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f87a8089f00 con 0x7f87b80b21f0 2023-10-08T05:49:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.417+0000 7f87bd991640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f879c06b070 0x7f879c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:25.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.418+0000 7f87bd991640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f879c06b070 0x7f879c06d530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87b4037000 tx=0x7f87b402f000 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:25.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.420+0000 7f87ae7fc640 1 -- 172.21.15.70:0/2456525029 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87a8054920 con 0x7f87b80b21f0 2023-10-08T05:49:25.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.632+0000 7f87bfbf5640 1 -- 172.21.15.70:0/2456525029 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f87b806cee0 con 0x7f879c06b070 2023-10-08T05:49:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.796+0000 7f87ae7fc640 1 -- 172.21.15.70:0/2456525029 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f87b402d960 con 0x7f879c06b070 2023-10-08T05:49:25.797 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-10-08T05:49:25.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.799+0000 7f87bfbf5640 1 -- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f879c06b070 msgr2=0x7f879c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:25.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.799+0000 7f87bfbf5640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f879c06b070 0x7f879c06d530 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87b4037000 tx=0x7f87b402f000 comp rx=0 tx=0).stop 2023-10-08T05:49:25.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.799+0000 7f87bfbf5640 1 -- 172.21.15.70:0/2456525029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80b21f0 msgr2=0x7f87b81e28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:25.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.799+0000 7f87bfbf5640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80b21f0 0x7f87b81e28b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f87a800c7f0 tx=0x7f87a800cbf0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.799+0000 7f87bfbf5640 1 -- 172.21.15.70:0/2456525029 shutdown_connections 2023-10-08T05:49:25.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.799+0000 7f87bfbf5640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87b80ca1a0 0x7f87b81e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.799+0000 7f87bfbf5640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87b80b21f0 0x7f87b81e28b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.799+0000 7f87bfbf5640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f879c06b070 0x7f879c06d530 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.799+0000 7f87bfbf5640 1 --2- 172.21.15.70:0/2456525029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87b80702b0 0x7f87b8158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:25.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.799+0000 7f87bfbf5640 1 -- 172.21.15.70:0/2456525029 >> 172.21.15.70:0/2456525029 conn(0x7f87b806a5d0 msgr2=0x7f87b80ce870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:25.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.800+0000 7f87bfbf5640 1 -- 172.21.15.70:0/2456525029 shutdown_connections 2023-10-08T05:49:25.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:25.800+0000 7f87bfbf5640 1 -- 172.21.15.70:0/2456525029 wait complete. 2023-10-08T05:49:25.820 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-10-08T05:49:25.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir7 1m 2023-10-08T05:49:26.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.031+0000 7fd1a5213640 1 Processor -- start 2023-10-08T05:49:26.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.031+0000 7fd1a5213640 1 -- start start 2023-10-08T05:49:26.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.032+0000 7fd1a5213640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a015ccb0 0x7fd1a00d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.032+0000 7fd1a5213640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd1a00d0b90 0x7fd1a00cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.032+0000 7fd1a5213640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a00d2870 con 0x7fd1a00cff10 2023-10-08T05:49:26.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.032+0000 7fd1a5213640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a00d0380 con 0x7fd1a00d0b90 2023-10-08T05:49:26.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.032+0000 7fd1a5213640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a00d1150 con 0x7fd1a015ccb0 2023-10-08T05:49:26.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.033+0000 7fd19ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a015ccb0 0x7fd1a00d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:26.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.033+0000 7fd19f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd1a00d0b90 0x7fd1a00cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:26.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.033+0000 7fd19ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a015ccb0 0x7fd1a00d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:36202/0 (socket says 172.21.15.70:36202) 2023-10-08T05:49:26.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.033+0000 7fd19ffff640 1 -- 172.21.15.70:0/1388561376 learned_addr learned my addr 172.21.15.70:0/1388561376 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:26.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.034+0000 7fd19effd640 1 -- 172.21.15.70:0/1388561376 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508189289 0 0) 0x7fd1a00d2870 con 0x7fd1a00cff10 2023-10-08T05:49:26.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.034+0000 7fd19ffff640 1 -- 172.21.15.70:0/1388561376 >> v2:172.21.15.181:3300/0 conn(0x7fd1a00d0b90 msgr2=0x7fd1a00cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:26.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.034+0000 7fd19ffff640 1 --2- 172.21.15.70:0/1388561376 >> v2:172.21.15.181:3300/0 conn(0x7fd1a00d0b90 0x7fd1a00cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.034+0000 7fd19ffff640 1 -- 172.21.15.70:0/1388561376 >> v1:172.21.15.181:6789/0 conn(0x7fd1a00cff10 legacy=0x7fd1a00d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:26.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.034+0000 7fd19ffff640 1 -- 172.21.15.70:0/1388561376 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a00d1430 con 0x7fd1a015ccb0 2023-10-08T05:49:26.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.034+0000 7fd19f7fe640 1 --2- 172.21.15.70:0/1388561376 >> v2:172.21.15.181:3300/0 conn(0x7fd1a00d0b90 0x7fd1a00cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:49:26.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.036+0000 7fd19ffff640 1 --2- 172.21.15.70:0/1388561376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a015ccb0 0x7fd1a00d1fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd190009870 tx=0x7fd19002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1a87669cfa281 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:26.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.036+0000 7fd19effd640 1 -- 172.21.15.70:0/1388561376 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190030840 con 0x7fd1a015ccb0 2023-10-08T05:49:26.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.036+0000 7fd19effd640 1 -- 172.21.15.70:0/1388561376 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1900309a0 con 0x7fd1a015ccb0 2023-10-08T05:49:26.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.036+0000 7fd19effd640 1 -- 172.21.15.70:0/1388561376 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190030d30 con 0x7fd1a015ccb0 2023-10-08T05:49:26.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.037+0000 7fd1a5213640 1 -- 172.21.15.70:0/1388561376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a015ccb0 msgr2=0x7fd1a00d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:26.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.037+0000 7fd1a5213640 1 --2- 172.21.15.70:0/1388561376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a015ccb0 0x7fd1a00d1fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd190009870 tx=0x7fd19002b980 comp rx=0 tx=0).stop 2023-10-08T05:49:26.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.037+0000 7fd1a5213640 1 -- 172.21.15.70:0/1388561376 shutdown_connections 2023-10-08T05:49:26.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.037+0000 7fd1a5213640 1 --2- 172.21.15.70:0/1388561376 >> v2:172.21.15.181:3300/0 conn(0x7fd1a00d0b90 0x7fd1a00cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.037+0000 7fd1a5213640 1 --2- 172.21.15.70:0/1388561376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a015ccb0 0x7fd1a00d1fb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.037+0000 7fd1a5213640 1 -- 172.21.15.70:0/1388561376 >> 172.21.15.70:0/1388561376 conn(0x7fd1a0071660 msgr2=0x7fd1a0071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:26.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.037+0000 7fd1a5213640 1 -- 172.21.15.70:0/1388561376 shutdown_connections 2023-10-08T05:49:26.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.037+0000 7fd1a5213640 1 -- 172.21.15.70:0/1388561376 wait complete. 2023-10-08T05:49:26.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.039+0000 7fd1a5213640 1 Processor -- start 2023-10-08T05:49:26.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.039+0000 7fd1a5213640 1 -- start start 2023-10-08T05:49:26.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd1a5213640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1a00cff10 0x7fd1a014efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd1a5213640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a00d0b90 0x7fd1a01f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd1a5213640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1a015ccb0 0x7fd1a00cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd1a5213640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a01f2d40 con 0x7fd1a015ccb0 2023-10-08T05:49:26.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd1a5213640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a01f3020 con 0x7fd1a00cff10 2023-10-08T05:49:26.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd1a5213640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a01f3650 con 0x7fd1a00d0b90 2023-10-08T05:49:26.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd1a4a12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1a015ccb0 0x7fd1a00cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd1a4a12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1a015ccb0 0x7fd1a00cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43920/0 (socket says 172.21.15.70:43920) 2023-10-08T05:49:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd1a4a12640 1 -- 172.21.15.70:0/2838080399 learned_addr learned my addr 172.21.15.70:0/2838080399 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd19f7fe640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a00d0b90 0x7fd1a01f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.041+0000 7fd19ffff640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1a00cff10 0x7fd1a014efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd1a4a12640 1 -- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a00d0b90 msgr2=0x7fd1a01f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd1a4a12640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a00d0b90 0x7fd1a01f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd1a4a12640 1 -- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1a00cff10 msgr2=0x7fd1a014efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd1a4a12640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1a00cff10 0x7fd1a014efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd1a4a12640 1 -- 172.21.15.70:0/2838080399 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188009a70 con 0x7fd1a015ccb0 2023-10-08T05:49:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd1a4a12640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1a015ccb0 0x7fd1a00cbe60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c410 tx=0x7fd19800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:26.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd19cff9640 1 -- 172.21.15.70:0/2838080399 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1980052d0 con 0x7fd1a015ccb0 2023-10-08T05:49:26.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd19cff9640 1 -- 172.21.15.70:0/2838080399 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd198005430 con 0x7fd1a015ccb0 2023-10-08T05:49:26.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd19cff9640 1 -- 172.21.15.70:0/2838080399 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd198003850 con 0x7fd1a015ccb0 2023-10-08T05:49:26.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd1a5213640 1 -- 172.21.15.70:0/2838080399 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1900094d0 con 0x7fd1a015ccb0 2023-10-08T05:49:26.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.042+0000 7fd1a5213640 1 -- 172.21.15.70:0/2838080399 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a014f4f0 con 0x7fd1a015ccb0 2023-10-08T05:49:26.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.043+0000 7fd1a5213640 1 -- 172.21.15.70:0/2838080399 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a01f3b70 con 0x7fd1a015ccb0 2023-10-08T05:49:26.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.045+0000 7fd19cff9640 1 -- 172.21.15.70:0/2838080399 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fd1980039b0 con 0x7fd1a015ccb0 2023-10-08T05:49:26.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.045+0000 7fd19cff9640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fd18006b070 0x7fd18006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.046+0000 7fd19cff9640 1 -- 172.21.15.70:0/2838080399 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fd198089120 con 0x7fd1a015ccb0 2023-10-08T05:49:26.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.046+0000 7fd19ffff640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fd18006b070 0x7fd18006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:26.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.047+0000 7fd19ffff640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fd18006b070 0x7fd18006d530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd190036000 tx=0x7fd190008030 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:26.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.051+0000 7fd19cff9640 1 -- 172.21.15.70:0/2838080399 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1a01f3b70 con 0x7fd1a015ccb0 2023-10-08T05:49:26.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.262+0000 7fd1a5213640 1 -- 172.21.15.70:0/2838080399 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1a00d19f0 con 0x7fd18006b070 2023-10-08T05:49:26.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.430+0000 7fd19cff9640 1 -- 172.21.15.70:0/2838080399 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd19003f040 con 0x7fd18006b070 2023-10-08T05:49:26.430 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-10-08T05:49:26.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 -- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fd18006b070 msgr2=0x7fd18006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:26.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fd18006b070 0x7fd18006d530 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd190036000 tx=0x7fd190008030 comp rx=0 tx=0).stop 2023-10-08T05:49:26.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 -- 172.21.15.70:0/2838080399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1a015ccb0 msgr2=0x7fd1a00cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:26.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1a015ccb0 0x7fd1a00cbe60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c410 tx=0x7fd19800c8d0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 -- 172.21.15.70:0/2838080399 shutdown_connections 2023-10-08T05:49:26.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1a015ccb0 0x7fd1a00cbe60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fd18006b070 0x7fd18006d530 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1a00d0b90 0x7fd1a01f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 --2- 172.21.15.70:0/2838080399 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1a00cff10 0x7fd1a014efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 -- 172.21.15.70:0/2838080399 >> 172.21.15.70:0/2838080399 conn(0x7fd1a0071660 msgr2=0x7fd1a015c330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:26.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 -- 172.21.15.70:0/2838080399 shutdown_connections 2023-10-08T05:49:26.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.433+0000 7fd1a5213640 1 -- 172.21.15.70:0/2838080399 wait complete. 2023-10-08T05:49:26.454 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-10-08T05:49:26.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir8 1m 2023-10-08T05:49:26.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.673+0000 7f555a7cc640 1 Processor -- start 2023-10-08T05:49:26.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.674+0000 7f555a7cc640 1 -- start start 2023-10-08T05:49:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.675+0000 7f555a7cc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55540afbe0 0x7f55540b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.675+0000 7f555a7cc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55540cb130 0x7f55540cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.675+0000 7f555a7cc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55540cbad0 con 0x7f555406dfc0 2023-10-08T05:49:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.675+0000 7f555a7cc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55540b2620 con 0x7f55540cb130 2023-10-08T05:49:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.675+0000 7f555a7cc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55540b2060 con 0x7f55540afbe0 2023-10-08T05:49:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.675+0000 7f5558d69640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55540cb130 0x7f55540cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.675+0000 7f554bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f555406dfc0 0x7f5554070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44362/0 (socket says 172.21.15.70:44362) 2023-10-08T05:49:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.675+0000 7f554bfff640 1 -- 172.21.15.70:0/3222246040 learned_addr learned my addr 172.21.15.70:0/3222246040 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.676+0000 7f5553fff640 1 --2- 172.21.15.70:0/3222246040 >> v2:172.21.15.181:3300/0 conn(0x7f55540afbe0 0x7f55540b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.676+0000 7f5558d69640 1 -- 172.21.15.70:0/3222246040 >> v2:172.21.15.181:3300/0 conn(0x7f55540afbe0 msgr2=0x7f55540b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.676+0000 7f5558d69640 1 --2- 172.21.15.70:0/3222246040 >> v2:172.21.15.181:3300/0 conn(0x7f55540afbe0 0x7f55540b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.676+0000 7f5558d69640 1 -- 172.21.15.70:0/3222246040 >> v1:172.21.15.70:6789/0 conn(0x7f555406dfc0 legacy=0x7f5554070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.676+0000 7f5558d69640 1 -- 172.21.15.70:0/3222246040 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55540b2340 con 0x7f55540cb130 2023-10-08T05:49:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.676+0000 7f5553fff640 1 --2- 172.21.15.70:0/3222246040 >> v2:172.21.15.181:3300/0 conn(0x7f55540afbe0 0x7f55540b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:26.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.677+0000 7f5558d69640 1 --2- 172.21.15.70:0/3222246040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 0x7f55540cb590 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f555406e640 tx=0x7f554c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5cac7ddd10b505 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:26.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.677+0000 7f55537fe640 1 -- 172.21.15.70:0/3222246040 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c02cc70 con 0x7f55540cb130 2023-10-08T05:49:26.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.677+0000 7f55537fe640 1 -- 172.21.15.70:0/3222246040 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c02cdd0 con 0x7f55540cb130 2023-10-08T05:49:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.677+0000 7f55537fe640 1 -- 172.21.15.70:0/3222246040 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c034660 con 0x7f55540cb130 2023-10-08T05:49:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.678+0000 7f555a7cc640 1 -- 172.21.15.70:0/3222246040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 msgr2=0x7f55540cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.678+0000 7f555a7cc640 1 --2- 172.21.15.70:0/3222246040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 0x7f55540cb590 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f555406e640 tx=0x7f554c02bec0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.678+0000 7f555a7cc640 1 -- 172.21.15.70:0/3222246040 shutdown_connections 2023-10-08T05:49:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.678+0000 7f555a7cc640 1 --2- 172.21.15.70:0/3222246040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 0x7f55540cb590 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.678+0000 7f555a7cc640 1 --2- 172.21.15.70:0/3222246040 >> v2:172.21.15.181:3300/0 conn(0x7f55540afbe0 0x7f55540b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.678+0000 7f555a7cc640 1 -- 172.21.15.70:0/3222246040 >> 172.21.15.70:0/3222246040 conn(0x7f555406a5d0 msgr2=0x7f55540b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.678+0000 7f555a7cc640 1 -- 172.21.15.70:0/3222246040 shutdown_connections 2023-10-08T05:49:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.678+0000 7f555a7cc640 1 -- 172.21.15.70:0/3222246040 wait complete. 2023-10-08T05:49:26.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.680+0000 7f555a7cc640 1 Processor -- start 2023-10-08T05:49:26.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.680+0000 7f555a7cc640 1 -- start start 2023-10-08T05:49:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f555a7cc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f555406dfc0 0x7f55541f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f555a7cc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55540afbe0 0x7f55540ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f555a7cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 0x7f55540ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f555a7cc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540c2250 con 0x7f55540cb130 2023-10-08T05:49:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f555a7cc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540c24a0 con 0x7f55540afbe0 2023-10-08T05:49:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f555a7cc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55540c2780 con 0x7f555406dfc0 2023-10-08T05:49:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f5558d69640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 0x7f55540ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f5558d69640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 0x7f55540ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43924/0 (socket says 172.21.15.70:43924) 2023-10-08T05:49:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f5558d69640 1 -- 172.21.15.70:0/2482887894 learned_addr learned my addr 172.21.15.70:0/2482887894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f5558d69640 1 -- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f555406dfc0 msgr2=0x7f55541f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f5558d69640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f555406dfc0 0x7f55541f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f5558d69640 1 -- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55540afbe0 msgr2=0x7f55540ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f5558d69640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55540afbe0 0x7f55540ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.681+0000 7f5558d69640 1 -- 172.21.15.70:0/2482887894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c009a70 con 0x7f55540cb130 2023-10-08T05:49:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.682+0000 7f5558d69640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 0x7f55540ce7a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f554c009e80 tx=0x7f554c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.682+0000 7f55517fa640 1 -- 172.21.15.70:0/2482887894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c040650 con 0x7f55540cb130 2023-10-08T05:49:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.682+0000 7f55517fa640 1 -- 172.21.15.70:0/2482887894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c0407d0 con 0x7f55540cb130 2023-10-08T05:49:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.682+0000 7f555a7cc640 1 -- 172.21.15.70:0/2482887894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55540c2a60 con 0x7f55540cb130 2023-10-08T05:49:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.682+0000 7f55517fa640 1 -- 172.21.15.70:0/2482887894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c040b80 con 0x7f55540cb130 2023-10-08T05:49:26.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.682+0000 7f555a7cc640 1 -- 172.21.15.70:0/2482887894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55540c3020 con 0x7f55540cb130 2023-10-08T05:49:26.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.683+0000 7f555a7cc640 1 -- 172.21.15.70:0/2482887894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555404e160 con 0x7f55540cb130 2023-10-08T05:49:26.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.684+0000 7f55517fa640 1 -- 172.21.15.70:0/2482887894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f554c040ce0 con 0x7f55540cb130 2023-10-08T05:49:26.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.685+0000 7f55517fa640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f553406b070 0x7f553406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:26.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.685+0000 7f55517fa640 1 -- 172.21.15.70:0/2482887894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f554c0aca90 con 0x7f55540cb130 2023-10-08T05:49:26.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.686+0000 7f5553fff640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f553406b070 0x7f553406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:26.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.687+0000 7f5553fff640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f553406b070 0x7f553406d530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5544006fd0 tx=0x7f5544008040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:26.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.690+0000 7f55517fa640 1 -- 172.21.15.70:0/2482887894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f554c077650 con 0x7f55540cb130 2023-10-08T05:49:26.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:26.902+0000 7f555a7cc640 1 -- 172.21.15.70:0/2482887894 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f5554070630 con 0x7f553406b070 2023-10-08T05:49:27.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.071+0000 7f55517fa640 1 -- 172.21.15.70:0/2482887894 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f554400ae40 con 0x7f553406b070 2023-10-08T05:49:27.072 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-10-08T05:49:27.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.074+0000 7f555a7cc640 1 -- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f553406b070 msgr2=0x7f553406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:27.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.074+0000 7f555a7cc640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f553406b070 0x7f553406d530 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5544006fd0 tx=0x7f5544008040 comp rx=0 tx=0).stop 2023-10-08T05:49:27.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.074+0000 7f555a7cc640 1 -- 172.21.15.70:0/2482887894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 msgr2=0x7f55540ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:27.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.074+0000 7f555a7cc640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 0x7f55540ce7a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f554c009e80 tx=0x7f554c004a40 comp rx=0 tx=0).stop 2023-10-08T05:49:27.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.075+0000 7f555a7cc640 1 -- 172.21.15.70:0/2482887894 shutdown_connections 2023-10-08T05:49:27.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.075+0000 7f555a7cc640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55540cb130 0x7f55540ce7a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.075+0000 7f555a7cc640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55540afbe0 0x7f55540ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.075+0000 7f555a7cc640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f553406b070 0x7f553406d530 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.075+0000 7f555a7cc640 1 --2- 172.21.15.70:0/2482887894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f555406dfc0 0x7f55541f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.075+0000 7f555a7cc640 1 -- 172.21.15.70:0/2482887894 >> 172.21.15.70:0/2482887894 conn(0x7f555406a5d0 msgr2=0x7f55540c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:27.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.075+0000 7f555a7cc640 1 -- 172.21.15.70:0/2482887894 shutdown_connections 2023-10-08T05:49:27.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.075+0000 7f555a7cc640 1 -- 172.21.15.70:0/2482887894 wait complete. 2023-10-08T05:49:27.096 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-10-08T05:49:27.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir9 1m 2023-10-08T05:49:27.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.312+0000 7ffbaa21a640 1 Processor -- start 2023-10-08T05:49:27.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.312+0000 7ffbaa21a640 1 -- start start 2023-10-08T05:49:27.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.313+0000 7ffbaa21a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffba406dfc0 0x7ffba4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:27.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.313+0000 7ffbaa21a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba40b1cc0 con 0x7ffba40afbe0 2023-10-08T05:49:27.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.313+0000 7ffbaa21a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba40b2620 con 0x7ffba4071070 2023-10-08T05:49:27.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.313+0000 7ffbaa21a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40b2060 con 0x7ffba406dfc0 2023-10-08T05:49:27.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.314+0000 7ffba37fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ffba40afbe0 0x7ffba40b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44378/0 (socket says 172.21.15.70:44378) 2023-10-08T05:49:27.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.314+0000 7ffba37fe640 1 -- 172.21.15.70:0/4007892875 learned_addr learned my addr 172.21.15.70:0/4007892875 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:27.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.314+0000 7ffba2ffd640 1 --2- 172.21.15.70:0/4007892875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffba406dfc0 0x7ffba4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:27.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.314+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977001346 0 0) 0x7ffba40b1cc0 con 0x7ffba40afbe0 2023-10-08T05:49:27.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.314+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb8c003520 con 0x7ffba40afbe0 2023-10-08T05:49:27.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3720711801 0 0) 0x7ffb8c003520 con 0x7ffba40afbe0 2023-10-08T05:49:27.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffba406dfc0 msgr2=0x7ffba4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:27.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 --2- 172.21.15.70:0/4007892875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffba406dfc0 0x7ffba4070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 >> v1:172.21.15.181:6789/0 conn(0x7ffba4071070 legacy=0x7ffba40cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:27.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba40b2340 con 0x7ffba40afbe0 2023-10-08T05:49:27.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ffb94001670 con 0x7ffba40afbe0 2023-10-08T05:49:27.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 >> v1:172.21.15.70:6789/0 conn(0x7ffba40afbe0 legacy=0x7ffba40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:27.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 --2- 172.21.15.70:0/4007892875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0039a0 0x7ffb8c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:27.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 --2- 172.21.15.70:0/4007892875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0047c0 0x7ffb8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:27.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 --2- 172.21.15.70:0/4007892875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffba406dfc0 0x7ffb8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:27.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40b2060 con 0x7ffb8c0047c0 2023-10-08T05:49:27.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c00ba60 con 0x7ffb8c0039a0 2023-10-08T05:49:27.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c00bc20 con 0x7ffba406dfc0 2023-10-08T05:49:27.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba2ffd640 1 --2- 172.21.15.70:0/4007892875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0047c0 0x7ffb8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:27.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba2ffd640 1 -- 172.21.15.70:0/4007892875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffba406dfc0 msgr2=0x7ffb8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:27.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba2ffd640 1 --2- 172.21.15.70:0/4007892875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffba406dfc0 0x7ffb8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba2ffd640 1 -- 172.21.15.70:0/4007892875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0039a0 msgr2=0x7ffb8c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:27.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba2ffd640 1 --2- 172.21.15.70:0/4007892875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb8c0039a0 0x7ffb8c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.315+0000 7ffba2ffd640 1 -- 172.21.15.70:0/4007892875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c003520 con 0x7ffb8c0047c0 2023-10-08T05:49:27.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.317+0000 7ffba2ffd640 1 --2- 172.21.15.70:0/4007892875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0047c0 0x7ffb8c004c40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ffba406e410 tx=0x7ffb8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa16eb73e51f2731 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:27.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.317+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9802c680 con 0x7ffb8c0047c0 2023-10-08T05:49:27.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.317+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb9802c7e0 con 0x7ffb8c0047c0 2023-10-08T05:49:27.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.317+0000 7ffba27fc640 1 -- 172.21.15.70:0/4007892875 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9802cb90 con 0x7ffb8c0047c0 2023-10-08T05:49:27.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.318+0000 7ffbaa21a640 1 -- 172.21.15.70:0/4007892875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0047c0 msgr2=0x7ffb8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:27.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.318+0000 7ffbaa21a640 1 --2- 172.21.15.70:0/4007892875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb8c0047c0 0x7ffb8c004c40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ffba406e410 tx=0x7ffb8c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.318+0000 7ffba3fff640 1 -- 172.21.15.70:0/4007892875 reap_dead start 2023-10-08T05:49:27.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.318+0000 7ffbaa21a640 1 -- 172.21.15.70:0/4007892875 shutdown_connections 2023-10-08T05:49:27.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.318+0000 7ffbaa21a640 1 -- 172.21.15.70:0/4007892875 >> 172.21.15.70:0/4007892875 conn(0x7ffba406a5d0 msgr2=0x7ffba40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:27.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.318+0000 7ffbaa21a640 1 -- 172.21.15.70:0/4007892875 shutdown_connections 2023-10-08T05:49:27.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.318+0000 7ffbaa21a640 1 -- 172.21.15.70:0/4007892875 wait complete. 2023-10-08T05:49:27.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.320+0000 7ffbaa21a640 1 Processor -- start 2023-10-08T05:49:27.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.321+0000 7ffbaa21a640 1 -- start start 2023-10-08T05:49:27.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffbaa21a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c0047c0 0x7ffba4159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:27.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffbaa21a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffba40c95f0 0x7ffba4149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:27.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffbaa21a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffba40c4bf0 0x7ffba40c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffbaa21a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba414a1d0 con 0x7ffba40c95f0 2023-10-08T05:49:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffbaa21a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba40c3290 con 0x7ffba40c4bf0 2023-10-08T05:49:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffbaa21a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba4159b60 con 0x7ffb8c0047c0 2023-10-08T05:49:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffba2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffba40c95f0 0x7ffba4149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffba2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffba40c95f0 0x7ffba4149c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43944/0 (socket says 172.21.15.70:43944) 2023-10-08T05:49:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffba2ffd640 1 -- 172.21.15.70:0/1725445557 learned_addr learned my addr 172.21.15.70:0/1725445557 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffba37fe640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c0047c0 0x7ffba4159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffba2ffd640 1 -- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c0047c0 msgr2=0x7ffba4159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffba2ffd640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c0047c0 0x7ffba4159480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffba2ffd640 1 -- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffba40c4bf0 msgr2=0x7ffba40c7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffba2ffd640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffba40c4bf0 0x7ffba40c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.322+0000 7ffba2ffd640 1 -- 172.21.15.70:0/1725445557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba4159e40 con 0x7ffba40c95f0 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.323+0000 7ffba2ffd640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffba40c95f0 0x7ffba4149c90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009b80 tx=0x7ffb98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.323+0000 7ffb8bfff640 1 -- 172.21.15.70:0/1725445557 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98034a10 con 0x7ffba40c95f0 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.323+0000 7ffb8bfff640 1 -- 172.21.15.70:0/1725445557 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb98034b70 con 0x7ffba40c95f0 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.323+0000 7ffb8bfff640 1 -- 172.21.15.70:0/1725445557 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9803e400 con 0x7ffba40c95f0 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.323+0000 7ffbaa21a640 1 -- 172.21.15.70:0/1725445557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba415a120 con 0x7ffba40c95f0 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.323+0000 7ffbaa21a640 1 -- 172.21.15.70:0/1725445557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba40c2eb0 con 0x7ffba40c95f0 2023-10-08T05:49:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.324+0000 7ffbaa21a640 1 -- 172.21.15.70:0/1725445557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba40c30a0 con 0x7ffba40c95f0 2023-10-08T05:49:27.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.325+0000 7ffb8bfff640 1 -- 172.21.15.70:0/1725445557 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7ffb9803e560 con 0x7ffba40c95f0 2023-10-08T05:49:27.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.326+0000 7ffb8bfff640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffb9006b070 0x7ffb9006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:27.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.326+0000 7ffb8bfff640 1 -- 172.21.15.70:0/1725445557 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7ffb980abb80 con 0x7ffba40c95f0 2023-10-08T05:49:27.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.327+0000 7ffba37fe640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffb9006b070 0x7ffb9006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:27.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.328+0000 7ffba37fe640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffb9006b070 0x7ffb9006d530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb94007ab0 tx=0x7ffb94009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:27.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.331+0000 7ffb8bfff640 1 -- 172.21.15.70:0/1725445557 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb980767f0 con 0x7ffba40c95f0 2023-10-08T05:49:27.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.544+0000 7ffbaa21a640 1 -- 172.21.15.70:0/1725445557 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7ffba4070630 con 0x7ffb9006b070 2023-10-08T05:49:27.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.720+0000 7ffb8bfff640 1 -- 172.21.15.70:0/1725445557 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ffb9400d890 con 0x7ffb9006b070 2023-10-08T05:49:27.721 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-10-08T05:49:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.723+0000 7ffbaa21a640 1 -- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffb9006b070 msgr2=0x7ffb9006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.723+0000 7ffbaa21a640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffb9006b070 0x7ffb9006d530 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb94007ab0 tx=0x7ffb94009040 comp rx=0 tx=0).stop 2023-10-08T05:49:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.723+0000 7ffbaa21a640 1 -- 172.21.15.70:0/1725445557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffba40c95f0 msgr2=0x7ffba4149c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.723+0000 7ffbaa21a640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffba40c95f0 0x7ffba4149c90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009b80 tx=0x7ffb98004730 comp rx=0 tx=0).stop 2023-10-08T05:49:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.723+0000 7ffbaa21a640 1 -- 172.21.15.70:0/1725445557 shutdown_connections 2023-10-08T05:49:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.723+0000 7ffbaa21a640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffba40c4bf0 0x7ffba40c7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.723+0000 7ffbaa21a640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffba40c95f0 0x7ffba4149c90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.723+0000 7ffbaa21a640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffb9006b070 0x7ffb9006d530 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.723+0000 7ffbaa21a640 1 --2- 172.21.15.70:0/1725445557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb8c0047c0 0x7ffba4159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.724+0000 7ffbaa21a640 1 -- 172.21.15.70:0/1725445557 >> 172.21.15.70:0/1725445557 conn(0x7ffba406a5d0 msgr2=0x7ffba415ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.724+0000 7ffbaa21a640 1 -- 172.21.15.70:0/1725445557 shutdown_connections 2023-10-08T05:49:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:27.724+0000 7ffbaa21a640 1 -- 172.21.15.70:0/1725445557 wait complete. 2023-10-08T05:49:27.743 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-10-08T05:49:27.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:27.744 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-08T05:49:27.803 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:27.803 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:27.803 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:27.803 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_01_UTC 2023-10-08T05:49:27.804 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:27.804 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_01_UTC 2023-10-08T05:49:27.804 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:27.804 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:27.804 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:27.804 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:27.804 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:27.805 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_39_00_UTC', 'scheduled-2023-10-08-05_40_00_UTC', 'scheduled-2023-10-08-05_41_00_UTC', 'scheduled-2023-10-08-05_42_01_UTC', 'scheduled-2023-10-08-05_43_01_UTC', 'scheduled-2023-10-08-05_44_01_UTC', 'scheduled-2023-10-08-05_45_01_UTC', 'scheduled-2023-10-08-05_46_01_UTC', 'scheduled-2023-10-08-05_47_00_UTC', 'scheduled-2023-10-08-05_48_00_UTC', 'scheduled-2023-10-08-05_49_01_UTC'] 2023-10-08T05:49:27.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir0 2023-10-08T05:49:28.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.019+0000 7f36a1060640 1 Processor -- start 2023-10-08T05:49:28.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.020+0000 7f36a1060640 1 -- start start 2023-10-08T05:49:28.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.021+0000 7f36a1060640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.021+0000 7f36a1060640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f369c0ca680 0x7f369c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.021+0000 7f36a1060640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f369c0cab60 con 0x7f369c15ac00 2023-10-08T05:49:28.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.021+0000 7f36a1060640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369c0cb960 con 0x7f369c0ca680 2023-10-08T05:49:28.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.021+0000 7f36a1060640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0cbdf0 con 0x7f369c15a820 2023-10-08T05:49:28.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.022+0000 7f369bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.022+0000 7f369bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:36252/0 (socket says 172.21.15.70:36252) 2023-10-08T05:49:28.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.022+0000 7f369bfff640 1 -- 172.21.15.70:0/3776471031 learned_addr learned my addr 172.21.15.70:0/3776471031 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:28.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.022+0000 7f3693fff640 1 --2- 172.21.15.70:0/3776471031 >> v2:172.21.15.181:3300/0 conn(0x7f369c0ca680 0x7f369c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.023+0000 7f369b7fe640 1 -- 172.21.15.70:0/3776471031 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043620285 0 0) 0x7f369c0cab60 con 0x7f369c15ac00 2023-10-08T05:49:28.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.023+0000 7f369bfff640 1 -- 172.21.15.70:0/3776471031 >> v2:172.21.15.181:3300/0 conn(0x7f369c0ca680 msgr2=0x7f369c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:28.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.023+0000 7f369bfff640 1 --2- 172.21.15.70:0/3776471031 >> v2:172.21.15.181:3300/0 conn(0x7f369c0ca680 0x7f369c0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.023+0000 7f369bfff640 1 -- 172.21.15.70:0/3776471031 >> v1:172.21.15.181:6789/0 conn(0x7f369c15ac00 legacy=0x7f369c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.023+0000 7f369bfff640 1 -- 172.21.15.70:0/3776471031 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c0cb030 con 0x7f369c15a820 2023-10-08T05:49:28.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.024+0000 7f369bfff640 1 --2- 172.21.15.70:0/3776471031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0aff70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f368c009870 tx=0x7f368c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d64545c3659ad1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:28.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.025+0000 7f369b7fe640 1 -- 172.21.15.70:0/3776471031 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c031640 con 0x7f369c15a820 2023-10-08T05:49:28.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.025+0000 7f369b7fe640 1 -- 172.21.15.70:0/3776471031 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c0317a0 con 0x7f369c15a820 2023-10-08T05:49:28.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.025+0000 7f369b7fe640 1 -- 172.21.15.70:0/3776471031 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c031b30 con 0x7f369c15a820 2023-10-08T05:49:28.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.025+0000 7f36a1060640 1 -- 172.21.15.70:0/3776471031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 msgr2=0x7f369c0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.025+0000 7f36a1060640 1 --2- 172.21.15.70:0/3776471031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0aff70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f368c009870 tx=0x7f368c02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.026+0000 7f36a1060640 1 -- 172.21.15.70:0/3776471031 shutdown_connections 2023-10-08T05:49:28.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.026+0000 7f36a1060640 1 --2- 172.21.15.70:0/3776471031 >> v2:172.21.15.181:3300/0 conn(0x7f369c0ca680 0x7f369c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.026+0000 7f36a1060640 1 --2- 172.21.15.70:0/3776471031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0aff70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.026+0000 7f36a1060640 1 -- 172.21.15.70:0/3776471031 >> 172.21.15.70:0/3776471031 conn(0x7f369c06c050 msgr2=0x7f369c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:28.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.026+0000 7f36a1060640 1 -- 172.21.15.70:0/3776471031 shutdown_connections 2023-10-08T05:49:28.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.026+0000 7f36a1060640 1 -- 172.21.15.70:0/3776471031 wait complete. 2023-10-08T05:49:28.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.028+0000 7f36a1060640 1 Processor -- start 2023-10-08T05:49:28.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.028+0000 7f36a1060640 1 -- start start 2023-10-08T05:49:28.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f36a1060640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f369c0ca680 0x7f369c14fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f36a1060640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f36a1060640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f369c15ac00 0x7f369c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f36a1060640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c150300 con 0x7f369c15ac00 2023-10-08T05:49:28.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f36a1060640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0cde30 con 0x7f369c0ca680 2023-10-08T05:49:28.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f36a1060640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0cd010 con 0x7f369c15a820 2023-10-08T05:49:28.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f36a085f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f369c15ac00 0x7f369c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f36a085f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f369c15ac00 0x7f369c0afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43954/0 (socket says 172.21.15.70:43954) 2023-10-08T05:49:28.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f36a085f640 1 -- 172.21.15.70:0/4020407977 learned_addr learned my addr 172.21.15.70:0/4020407977 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:28.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f369bfff640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f369c0ca680 0x7f369c14fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.029+0000 7f3693fff640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36a085f640 1 -- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 msgr2=0x7f369c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36a085f640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36a085f640 1 -- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f369c0ca680 msgr2=0x7f369c14fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36a085f640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f369c0ca680 0x7f369c14fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36a085f640 1 -- 172.21.15.70:0/4020407977 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0094d0 con 0x7f369c15ac00 2023-10-08T05:49:28.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f369bfff640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f369c0ca680 0x7f369c14fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:28.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f3693fff640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:28.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36a085f640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f369c15ac00 0x7f369c0afb60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f369400c730 tx=0x7f369400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:28.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36997fa640 1 -- 172.21.15.70:0/4020407977 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694003da0 con 0x7f369c15ac00 2023-10-08T05:49:28.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36997fa640 1 -- 172.21.15.70:0/4020407977 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369400ce40 con 0x7f369c15ac00 2023-10-08T05:49:28.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36997fa640 1 -- 172.21.15.70:0/4020407977 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694005270 con 0x7f369c15ac00 2023-10-08T05:49:28.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36a1060640 1 -- 172.21.15.70:0/4020407977 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c0cd2b0 con 0x7f369c15ac00 2023-10-08T05:49:28.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.030+0000 7f36a1060640 1 -- 172.21.15.70:0/4020407977 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c0cd870 con 0x7f369c15ac00 2023-10-08T05:49:28.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.031+0000 7f36a1060640 1 -- 172.21.15.70:0/4020407977 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369c074150 con 0x7f369c15ac00 2023-10-08T05:49:28.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.032+0000 7f36997fa640 1 -- 172.21.15.70:0/4020407977 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f3694005490 con 0x7f369c15ac00 2023-10-08T05:49:28.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.033+0000 7f36997fa640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f367c06b070 0x7f367c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.033+0000 7f36997fa640 1 -- 172.21.15.70:0/4020407977 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f3694089ae0 con 0x7f369c15ac00 2023-10-08T05:49:28.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.034+0000 7f369bfff640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f367c06b070 0x7f367c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.035+0000 7f369bfff640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f367c06b070 0x7f367c06d530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f368c036000 tx=0x7f368c008030 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:28.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.037+0000 7f36997fa640 1 -- 172.21.15.70:0/4020407977 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3694054610 con 0x7f369c15ac00 2023-10-08T05:49:28.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.253+0000 7f36a1060640 1 -- 172.21.15.70:0/4020407977 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f369c0cb5f0 con 0x7f367c06b070 2023-10-08T05:49:28.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.273+0000 7f36997fa640 1 -- 172.21.15.70:0/4020407977 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f368c031910 con 0x7f367c06b070 2023-10-08T05:49:28.273 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:28.274 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:05", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:28.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.275+0000 7f36a1060640 1 -- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f367c06b070 msgr2=0x7f367c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.275+0000 7f36a1060640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f367c06b070 0x7f367c06d530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f368c036000 tx=0x7f368c008030 comp rx=0 tx=0).stop 2023-10-08T05:49:28.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.275+0000 7f36a1060640 1 -- 172.21.15.70:0/4020407977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f369c15ac00 msgr2=0x7f369c0afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.275+0000 7f36a1060640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f369c15ac00 0x7f369c0afb60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f369400c730 tx=0x7f369400cbf0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.276+0000 7f36a1060640 1 -- 172.21.15.70:0/4020407977 shutdown_connections 2023-10-08T05:49:28.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.276+0000 7f36a1060640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f369c15ac00 0x7f369c0afb60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.276+0000 7f36a1060640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f367c06b070 0x7f367c06d530 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.276+0000 7f36a1060640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f369c15a820 0x7f369c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.276+0000 7f36a1060640 1 --2- 172.21.15.70:0/4020407977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f369c0ca680 0x7f369c14fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.276+0000 7f36a1060640 1 -- 172.21.15.70:0/4020407977 >> 172.21.15.70:0/4020407977 conn(0x7f369c06c050 msgr2=0x7f369c0c9990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:28.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.276+0000 7f36a1060640 1 -- 172.21.15.70:0/4020407977 shutdown_connections 2023-10-08T05:49:28.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.276+0000 7f36a1060640 1 -- 172.21.15.70:0/4020407977 wait complete. 2023-10-08T05:49:28.295 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:05", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:28.295 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:38:05', 'first': '2023-10-08T05:39:00', 'last': '2023-10-08T05:49:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T05:49:28.295 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T05:49:28.296 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T05:49:28.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:28.296 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-08T05:49:28.353 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:28.354 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:28.354 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_01_UTC 2023-10-08T05:49:28.354 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_01_UTC 2023-10-08T05:49:28.354 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:28.354 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:28.354 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:28.354 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:28.354 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:28.354 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:28.354 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:28.355 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_39_00_UTC', 'scheduled-2023-10-08-05_40_00_UTC', 'scheduled-2023-10-08-05_41_01_UTC', 'scheduled-2023-10-08-05_42_01_UTC', 'scheduled-2023-10-08-05_43_01_UTC', 'scheduled-2023-10-08-05_44_00_UTC', 'scheduled-2023-10-08-05_45_00_UTC', 'scheduled-2023-10-08-05_46_00_UTC', 'scheduled-2023-10-08-05_47_00_UTC', 'scheduled-2023-10-08-05_48_00_UTC', 'scheduled-2023-10-08-05_49_00_UTC'] 2023-10-08T05:49:28.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir1 2023-10-08T05:49:28.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.573+0000 7fb846ae5640 1 Processor -- start 2023-10-08T05:49:28.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.573+0000 7fb846ae5640 1 -- start start 2023-10-08T05:49:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.574+0000 7fb846ae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb84015cd70 0x7fb8401612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.575+0000 7fb846ae5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8400b4e10 0x7fb8400d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.575+0000 7fb846ae5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8400b8c30 con 0x7fb8400cd4e0 2023-10-08T05:49:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.575+0000 7fb846ae5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8400cdd80 con 0x7fb8400b4e10 2023-10-08T05:49:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.575+0000 7fb846ae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400ce060 con 0x7fb84015cd70 2023-10-08T05:49:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.575+0000 7fb83ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb8400cd4e0 0x7fb8400b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44394/0 (socket says 172.21.15.70:44394) 2023-10-08T05:49:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.575+0000 7fb83ffff640 1 -- 172.21.15.70:0/2120776682 learned_addr learned my addr 172.21.15.70:0/2120776682 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:28.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.575+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119745552 0 0) 0x7fb8400b8c30 con 0x7fb8400cd4e0 2023-10-08T05:49:28.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.575+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb82c003520 con 0x7fb8400cd4e0 2023-10-08T05:49:28.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.575+0000 7fb845082640 1 --2- 172.21.15.70:0/2120776682 >> v2:172.21.15.181:3300/0 conn(0x7fb8400b4e10 0x7fb8400d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.575+0000 7fb844881640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb84015cd70 0x7fb8401612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1265783703 0 0) 0x7fb82c003520 con 0x7fb8400cd4e0 2023-10-08T05:49:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb84015cd70 msgr2=0x7fb8401612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb84015cd70 0x7fb8401612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 >> v2:172.21.15.181:3300/0 conn(0x7fb8400b4e10 msgr2=0x7fb8400d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 --2- 172.21.15.70:0/2120776682 >> v2:172.21.15.181:3300/0 conn(0x7fb8400b4e10 0x7fb8400d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8400ce340 con 0x7fb8400cd4e0 2023-10-08T05:49:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fb828001690 con 0x7fb8400cd4e0 2023-10-08T05:49:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 >> v1:172.21.15.70:6789/0 conn(0x7fb8400cd4e0 legacy=0x7fb8400b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb82c003970 0x7fb82c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb82c0047c0 0x7fb82c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb84015cd70 0x7fb82c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400cdd80 con 0x7fb82c0047c0 2023-10-08T05:49:28.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8400ce060 con 0x7fb82c003970 2023-10-08T05:49:28.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c00ba80 con 0x7fb84015cd70 2023-10-08T05:49:28.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb83ffff640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb82c0047c0 0x7fb82c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.576+0000 7fb845082640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb84015cd70 0x7fb82c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.577+0000 7fb844881640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb82c003970 0x7fb82c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.577+0000 7fb83ffff640 1 -- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb84015cd70 msgr2=0x7fb82c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:28.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.577+0000 7fb83ffff640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb84015cd70 0x7fb82c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.577+0000 7fb83ffff640 1 -- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb82c003970 msgr2=0x7fb82c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.577+0000 7fb83ffff640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb82c003970 0x7fb82c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.577+0000 7fb83ffff640 1 -- 172.21.15.70:0/2120776682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c003520 con 0x7fb82c0047c0 2023-10-08T05:49:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.577+0000 7fb844881640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb82c003970 0x7fb82c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.578+0000 7fb83ffff640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb82c0047c0 0x7fb82c004c40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb8400cf460 tx=0x7fb82c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ac598f81a9070f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.578+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb828005ad0 con 0x7fb82c0047c0 2023-10-08T05:49:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.578+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb828005c30 con 0x7fb82c0047c0 2023-10-08T05:49:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.578+0000 7fb83f7fe640 1 -- 172.21.15.70:0/2120776682 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82802c7c0 con 0x7fb82c0047c0 2023-10-08T05:49:28.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.579+0000 7fb846ae5640 1 -- 172.21.15.70:0/2120776682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb82c0047c0 msgr2=0x7fb82c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.579+0000 7fb846ae5640 1 --2- 172.21.15.70:0/2120776682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb82c0047c0 0x7fb82c004c40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb8400cf460 tx=0x7fb82c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.579+0000 7fb845082640 1 -- 172.21.15.70:0/2120776682 reap_dead start 2023-10-08T05:49:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.579+0000 7fb846ae5640 1 -- 172.21.15.70:0/2120776682 shutdown_connections 2023-10-08T05:49:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.579+0000 7fb846ae5640 1 -- 172.21.15.70:0/2120776682 >> 172.21.15.70:0/2120776682 conn(0x7fb8400cf700 msgr2=0x7fb8400cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.579+0000 7fb846ae5640 1 -- 172.21.15.70:0/2120776682 shutdown_connections 2023-10-08T05:49:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.579+0000 7fb846ae5640 1 -- 172.21.15.70:0/2120776682 wait complete. 2023-10-08T05:49:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.581+0000 7fb846ae5640 1 Processor -- start 2023-10-08T05:49:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.581+0000 7fb846ae5640 1 -- start start 2023-10-08T05:49:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb846ae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb82c0047c0 0x7fb840154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb846ae5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8400c8550 0x7fb8401f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb846ae5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840159df0 0x7fb84015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb846ae5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84015a790 con 0x7fb840159df0 2023-10-08T05:49:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb846ae5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8401f6580 con 0x7fb8400c8550 2023-10-08T05:49:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb846ae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8401f1f20 con 0x7fb82c0047c0 2023-10-08T05:49:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb845082640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840159df0 0x7fb84015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb845082640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840159df0 0x7fb84015a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43976/0 (socket says 172.21.15.70:43976) 2023-10-08T05:49:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb845082640 1 -- 172.21.15.70:0/3281030774 learned_addr learned my addr 172.21.15.70:0/3281030774 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb844881640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb82c0047c0 0x7fb840154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb845082640 1 -- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb82c0047c0 msgr2=0x7fb840154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb83ffff640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8400c8550 0x7fb8401f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb845082640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb82c0047c0 0x7fb840154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.582+0000 7fb845082640 1 -- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8400c8550 msgr2=0x7fb8401f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.583+0000 7fb845082640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8400c8550 0x7fb8401f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.583+0000 7fb845082640 1 -- 172.21.15.70:0/3281030774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840158990 con 0x7fb840159df0 2023-10-08T05:49:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.583+0000 7fb844881640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb82c0047c0 0x7fb840154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.583+0000 7fb845082640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840159df0 0x7fb84015a250 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb8400cf430 tx=0x7fb83800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.583+0000 7fb83d7fa640 1 -- 172.21.15.70:0/3281030774 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838004440 con 0x7fb840159df0 2023-10-08T05:49:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.583+0000 7fb83d7fa640 1 -- 172.21.15.70:0/3281030774 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83800fe70 con 0x7fb840159df0 2023-10-08T05:49:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.583+0000 7fb83d7fa640 1 -- 172.21.15.70:0/3281030774 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838003850 con 0x7fb840159df0 2023-10-08T05:49:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.583+0000 7fb846ae5640 1 -- 172.21.15.70:0/3281030774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84015d420 con 0x7fb840159df0 2023-10-08T05:49:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.583+0000 7fb846ae5640 1 -- 172.21.15.70:0/3281030774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84015d9e0 con 0x7fb840159df0 2023-10-08T05:49:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.584+0000 7fb846ae5640 1 -- 172.21.15.70:0/3281030774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84004e160 con 0x7fb840159df0 2023-10-08T05:49:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.585+0000 7fb83d7fa640 1 -- 172.21.15.70:0/3281030774 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fb8380039b0 con 0x7fb840159df0 2023-10-08T05:49:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.586+0000 7fb83d7fa640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb82006b070 0x7fb82006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.586+0000 7fb83d7fa640 1 -- 172.21.15.70:0/3281030774 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fb838089bc0 con 0x7fb840159df0 2023-10-08T05:49:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.587+0000 7fb844881640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb82006b070 0x7fb82006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.588+0000 7fb844881640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb82006b070 0x7fb82006d530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb830009540 tx=0x7fb83000b040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.590+0000 7fb83d7fa640 1 -- 172.21.15.70:0/3281030774 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8380546f0 con 0x7fb840159df0 2023-10-08T05:49:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.803+0000 7fb846ae5640 1 -- 172.21.15.70:0/3281030774 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8400d1130 con 0x7fb82006b070 2023-10-08T05:49:28.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.822+0000 7fb83d7fa640 1 -- 172.21.15.70:0/3281030774 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fb830007e30 con 0x7fb82006b070 2023-10-08T05:49:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:28.823 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:05", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:28.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 -- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb82006b070 msgr2=0x7fb82006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb82006b070 0x7fb82006d530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb830009540 tx=0x7fb83000b040 comp rx=0 tx=0).stop 2023-10-08T05:49:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 -- 172.21.15.70:0/3281030774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840159df0 msgr2=0x7fb84015a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840159df0 0x7fb84015a250 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb8400cf430 tx=0x7fb83800f670 comp rx=0 tx=0).stop 2023-10-08T05:49:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 -- 172.21.15.70:0/3281030774 shutdown_connections 2023-10-08T05:49:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb840159df0 0x7fb84015a250 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8400c8550 0x7fb8401f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb82006b070 0x7fb82006d530 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 --2- 172.21.15.70:0/3281030774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb82c0047c0 0x7fb840154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:28.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 -- 172.21.15.70:0/3281030774 >> 172.21.15.70:0/3281030774 conn(0x7fb8400cf700 msgr2=0x7fb8400d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:28.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.825+0000 7fb846ae5640 1 -- 172.21.15.70:0/3281030774 shutdown_connections 2023-10-08T05:49:28.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:28.826+0000 7fb846ae5640 1 -- 172.21.15.70:0/3281030774 wait complete. 2023-10-08T05:49:28.846 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:05", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:28.846 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:38:05', 'first': '2023-10-08T05:39:00', 'last': '2023-10-08T05:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T05:49:28.846 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T05:49:28.846 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T05:49:28.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:28.847 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-08T05:49:28.900 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:28.901 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_01_UTC 2023-10-08T05:49:28.901 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:28.901 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:28.901 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:28.901 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_01_UTC 2023-10-08T05:49:28.901 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:28.901 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:28.901 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:28.901 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:28.902 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:28.903 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_39_00_UTC', 'scheduled-2023-10-08-05_40_01_UTC', 'scheduled-2023-10-08-05_41_00_UTC', 'scheduled-2023-10-08-05_42_00_UTC', 'scheduled-2023-10-08-05_43_01_UTC', 'scheduled-2023-10-08-05_44_01_UTC', 'scheduled-2023-10-08-05_45_00_UTC', 'scheduled-2023-10-08-05_46_00_UTC', 'scheduled-2023-10-08-05_47_00_UTC', 'scheduled-2023-10-08-05_48_00_UTC', 'scheduled-2023-10-08-05_49_00_UTC'] 2023-10-08T05:49:28.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir2 2023-10-08T05:49:29.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.118+0000 7fa6b9f6a640 1 Processor -- start 2023-10-08T05:49:29.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.119+0000 7fa6b9f6a640 1 -- start start 2023-10-08T05:49:29.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.120+0000 7fa6b9f6a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40b0680 0x7fa6b415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.120+0000 7fa6b9f6a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b40cab60 con 0x7fa6b40ca680 2023-10-08T05:49:29.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.120+0000 7fa6b9f6a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b40cb960 con 0x7fa6b415a820 2023-10-08T05:49:29.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.120+0000 7fa6b9f6a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b40cbdf0 con 0x7fa6b40b0680 2023-10-08T05:49:29.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b9769640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa6b40ca680 0x7fa6b415ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44406/0 (socket says 172.21.15.70:44406) 2023-10-08T05:49:29.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b9769640 1 -- 172.21.15.70:0/4212631257 learned_addr learned my addr 172.21.15.70:0/4212631257 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:29.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b3fff640 1 --2- 172.21.15.70:0/4212631257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40b0680 0x7fa6b415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163418532 0 0) 0x7fa6b40cab60 con 0x7fa6b40ca680 2023-10-08T05:49:29.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a4003520 con 0x7fa6b40ca680 2023-10-08T05:49:29.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2752227515 0 0) 0x7fa6a4003520 con 0x7fa6b40ca680 2023-10-08T05:49:29.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40b0680 msgr2=0x7fa6b415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b37fe640 1 --2- 172.21.15.70:0/4212631257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40b0680 0x7fa6b415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 >> v1:172.21.15.181:6789/0 conn(0x7fa6b415a820 legacy=0x7fa6b40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b40cb030 con 0x7fa6b40ca680 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa6ac0019f0 con 0x7fa6b40ca680 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.121+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 >> v1:172.21.15.70:6789/0 conn(0x7fa6b40ca680 legacy=0x7fa6b415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b37fe640 1 --2- 172.21.15.70:0/4212631257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6a40039a0 0x7fa6a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b37fe640 1 --2- 172.21.15.70:0/4212631257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6a40047c0 0x7fa6a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b37fe640 1 --2- 172.21.15.70:0/4212631257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40b0680 0x7fa6a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b40cbdf0 con 0x7fa6a40039a0 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a400bab0 con 0x7fa6a40047c0 2023-10-08T05:49:29.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a400bc70 con 0x7fa6b40b0680 2023-10-08T05:49:29.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b8f68640 1 --2- 172.21.15.70:0/4212631257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6a40039a0 0x7fa6a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b8f68640 1 -- 172.21.15.70:0/4212631257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40b0680 msgr2=0x7fa6a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:29.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b8f68640 1 --2- 172.21.15.70:0/4212631257 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6b40b0680 0x7fa6a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b8f68640 1 -- 172.21.15.70:0/4212631257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6a40047c0 msgr2=0x7fa6a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:29.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b8f68640 1 --2- 172.21.15.70:0/4212631257 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6a40047c0 0x7fa6a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.122+0000 7fa6b8f68640 1 -- 172.21.15.70:0/4212631257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a4003520 con 0x7fa6a40039a0 2023-10-08T05:49:29.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.123+0000 7fa6b8f68640 1 --2- 172.21.15.70:0/4212631257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6a40039a0 0x7fa6a4003dc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8001750 tx=0x7fa6a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca51aa4845c170d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:29.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.124+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a80052d0 con 0x7fa6a40039a0 2023-10-08T05:49:29.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.124+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a802e050 con 0x7fa6a40039a0 2023-10-08T05:49:29.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.124+0000 7fa6b37fe640 1 -- 172.21.15.70:0/4212631257 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a80035f0 con 0x7fa6a40039a0 2023-10-08T05:49:29.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.124+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/4212631257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6a40039a0 msgr2=0x7fa6a4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.124+0000 7fa6b9f6a640 1 --2- 172.21.15.70:0/4212631257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6a40039a0 0x7fa6a4003dc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8001750 tx=0x7fa6a40057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.124+0000 7fa6b9769640 1 -- 172.21.15.70:0/4212631257 reap_dead start 2023-10-08T05:49:29.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.124+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/4212631257 shutdown_connections 2023-10-08T05:49:29.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.125+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/4212631257 >> 172.21.15.70:0/4212631257 conn(0x7fa6b406c050 msgr2=0x7fa6b406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:29.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.125+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/4212631257 shutdown_connections 2023-10-08T05:49:29.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.125+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/4212631257 wait complete. 2023-10-08T05:49:29.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.126+0000 7fa6b9f6a640 1 Processor -- start 2023-10-08T05:49:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.127+0000 7fa6b9f6a640 1 -- start start 2023-10-08T05:49:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9f6a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6a40039a0 0x7fa6b40c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9f6a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b41eda40 0x7fa6b40c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9f6a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6b41f1410 0x7fa6b41f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9f6a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b40c1de0 con 0x7fa6b41f1410 2023-10-08T05:49:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9f6a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b41f1db0 con 0x7fa6b41eda40 2023-10-08T05:49:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9f6a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b40c20b0 con 0x7fa6a40039a0 2023-10-08T05:49:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9769640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6b41f1410 0x7fa6b41f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9769640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6b41f1410 0x7fa6b41f1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43990/0 (socket says 172.21.15.70:43990) 2023-10-08T05:49:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9769640 1 -- 172.21.15.70:0/358820809 learned_addr learned my addr 172.21.15.70:0/358820809 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:29.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b8f68640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6a40039a0 0x7fa6b40c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9769640 1 -- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6a40039a0 msgr2=0x7fa6b40c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9769640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6a40039a0 0x7fa6b40c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9769640 1 -- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b41eda40 msgr2=0x7fa6b40c18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9769640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b41eda40 0x7fa6b40c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.128+0000 7fa6b9769640 1 -- 172.21.15.70:0/358820809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b40c6900 con 0x7fa6b41f1410 2023-10-08T05:49:29.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.129+0000 7fa6b9769640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6b41f1410 0x7fa6b41f1870 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40caa90 tx=0x7fa6ac00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:29.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.129+0000 7fa6b17fa640 1 -- 172.21.15.70:0/358820809 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00a640 con 0x7fa6b41f1410 2023-10-08T05:49:29.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.129+0000 7fa6b17fa640 1 -- 172.21.15.70:0/358820809 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac00a7a0 con 0x7fa6b41f1410 2023-10-08T05:49:29.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.129+0000 7fa6b17fa640 1 -- 172.21.15.70:0/358820809 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00ab10 con 0x7fa6b41f1410 2023-10-08T05:49:29.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.129+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/358820809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b40c6be0 con 0x7fa6b41f1410 2023-10-08T05:49:29.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.129+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/358820809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b40c71a0 con 0x7fa6b41f1410 2023-10-08T05:49:29.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.130+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/358820809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b4074150 con 0x7fa6b41f1410 2023-10-08T05:49:29.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.131+0000 7fa6b17fa640 1 -- 172.21.15.70:0/358820809 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fa6ac00ac70 con 0x7fa6b41f1410 2023-10-08T05:49:29.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.132+0000 7fa6b17fa640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa69806b070 0x7fa69806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.132+0000 7fa6b17fa640 1 -- 172.21.15.70:0/358820809 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fa6ac0890c0 con 0x7fa6b41f1410 2023-10-08T05:49:29.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.133+0000 7fa6b8f68640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa69806b070 0x7fa69806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.134+0000 7fa6b8f68640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa69806b070 0x7fa69806d530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8033000 tx=0x7fa6a8009030 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:29.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.137+0000 7fa6b17fa640 1 -- 172.21.15.70:0/358820809 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6ac053bf0 con 0x7fa6b41f1410 2023-10-08T05:49:29.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.353+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/358820809 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6b40cb5f0 con 0x7fa69806b070 2023-10-08T05:49:29.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.372+0000 7fa6b17fa640 1 -- 172.21.15.70:0/358820809 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fa6a8005350 con 0x7fa69806b070 2023-10-08T05:49:29.372 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:29.373 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:06", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:29.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.374+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa69806b070 msgr2=0x7fa69806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.374+0000 7fa6b9f6a640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa69806b070 0x7fa69806d530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8033000 tx=0x7fa6a8009030 comp rx=0 tx=0).stop 2023-10-08T05:49:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.374+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/358820809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6b41f1410 msgr2=0x7fa6b41f1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.374+0000 7fa6b9f6a640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6b41f1410 0x7fa6b41f1870 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40caa90 tx=0x7fa6ac00cf00 comp rx=0 tx=0).stop 2023-10-08T05:49:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.375+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/358820809 shutdown_connections 2023-10-08T05:49:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.375+0000 7fa6b9f6a640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6b41f1410 0x7fa6b41f1870 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.375+0000 7fa6b9f6a640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6b41eda40 0x7fa6b40c18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.375+0000 7fa6b9f6a640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa69806b070 0x7fa69806d530 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.375+0000 7fa6b9f6a640 1 --2- 172.21.15.70:0/358820809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6a40039a0 0x7fa6b40c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.375+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/358820809 >> 172.21.15.70:0/358820809 conn(0x7fa6b406c050 msgr2=0x7fa6b41581b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.375+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/358820809 shutdown_connections 2023-10-08T05:49:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.375+0000 7fa6b9f6a640 1 -- 172.21.15.70:0/358820809 wait complete. 2023-10-08T05:49:29.393 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:06", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:29.393 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:38:06', 'first': '2023-10-08T05:39:00', 'last': '2023-10-08T05:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T05:49:29.393 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T05:49:29.393 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T05:49:29.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:29.394 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-08T05:49:29.448 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_01_UTC 2023-10-08T05:49:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_01_UTC 2023-10-08T05:49:29.449 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_01_UTC 2023-10-08T05:49:29.450 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:29.450 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_39_01_UTC', 'scheduled-2023-10-08-05_40_00_UTC', 'scheduled-2023-10-08-05_41_00_UTC', 'scheduled-2023-10-08-05_42_00_UTC', 'scheduled-2023-10-08-05_43_00_UTC', 'scheduled-2023-10-08-05_44_00_UTC', 'scheduled-2023-10-08-05_45_01_UTC', 'scheduled-2023-10-08-05_46_01_UTC', 'scheduled-2023-10-08-05_47_01_UTC', 'scheduled-2023-10-08-05_48_01_UTC', 'scheduled-2023-10-08-05_49_01_UTC'] 2023-10-08T05:49:29.451 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir3 2023-10-08T05:49:29.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.671+0000 7faac51b7640 1 Processor -- start 2023-10-08T05:49:29.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.671+0000 7faac51b7640 1 -- start start 2023-10-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.672+0000 7faac51b7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faac015cd80 0x7faac00b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.672+0000 7faac51b7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00b6730 0x7faac00cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.673+0000 7faac51b7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac015d160 con 0x7faac00cdda0 2023-10-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.673+0000 7faac51b7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac00ce230 con 0x7faac015cd80 2023-10-08T05:49:29.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.673+0000 7faac51b7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac00b7b90 con 0x7faac00b6730 2023-10-08T05:49:29.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.673+0000 7faabf59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7faac00cdda0 0x7faac00b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44420/0 (socket says 172.21.15.70:44420) 2023-10-08T05:49:29.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.673+0000 7faabf59e640 1 -- 172.21.15.70:0/1744245966 learned_addr learned my addr 172.21.15.70:0/1744245966 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:29.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.673+0000 7faabe59c640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00b6730 0x7faac00cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.673+0000 7faabed9d640 1 --2- 172.21.15.70:0/1744245966 >> v2:172.21.15.181:3300/0 conn(0x7faac015cd80 0x7faac00b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.673+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439252074 0 0) 0x7faac015d160 con 0x7faac00cdda0 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.673+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa8003520 con 0x7faac00cdda0 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4074467560 0 0) 0x7faaa8003520 con 0x7faac00cdda0 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00b6730 msgr2=0x7faac00cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00b6730 0x7faac00cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 >> v2:172.21.15.181:3300/0 conn(0x7faac015cd80 msgr2=0x7faac00b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 --2- 172.21.15.70:0/1744245966 >> v2:172.21.15.181:3300/0 conn(0x7faac015cd80 0x7faac00b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac00b8020 con 0x7faac00cdda0 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7faab80019f0 con 0x7faac00cdda0 2023-10-08T05:49:29.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 >> v1:172.21.15.70:6789/0 conn(0x7faac00cdda0 legacy=0x7faac00b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa8003970 0x7faaa8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa80047c0 0x7faaa8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00b6730 0x7faaa8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00ce230 con 0x7faaa80047c0 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00b7b90 con 0x7faaa8003970 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa800b980 con 0x7faac00b6730 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.674+0000 7faabe59c640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa80047c0 0x7faaa8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.675+0000 7faabed9d640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa8003970 0x7faaa8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.675+0000 7faabf59e640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00b6730 0x7faaa8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.675+0000 7faabe59c640 1 -- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00b6730 msgr2=0x7faaa8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.675+0000 7faabe59c640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00b6730 0x7faaa8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.675+0000 7faabe59c640 1 -- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa8003970 msgr2=0x7faaa8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:29.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.675+0000 7faabe59c640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa8003970 0x7faaa8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.675+0000 7faabe59c640 1 -- 172.21.15.70:0/1744245966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa8003520 con 0x7faaa80047c0 2023-10-08T05:49:29.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.675+0000 7faabed9d640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa8003970 0x7faaa8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:29.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.676+0000 7faabe59c640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa80047c0 0x7faaa8004c40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7faab4009ac0 tx=0x7faaa80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb97dd2f702b05e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:29.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.676+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab402c6b0 con 0x7faaa80047c0 2023-10-08T05:49:29.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.676+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab402c810 con 0x7faaa80047c0 2023-10-08T05:49:29.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.676+0000 7faabdd9b640 1 -- 172.21.15.70:0/1744245966 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab402cba0 con 0x7faaa80047c0 2023-10-08T05:49:29.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.677+0000 7faac51b7640 1 -- 172.21.15.70:0/1744245966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa80047c0 msgr2=0x7faaa8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.677+0000 7faac51b7640 1 --2- 172.21.15.70:0/1744245966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa80047c0 0x7faaa8004c40 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7faab4009ac0 tx=0x7faaa80057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.677+0000 7faabf59e640 1 -- 172.21.15.70:0/1744245966 reap_dead start 2023-10-08T05:49:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.677+0000 7faac51b7640 1 -- 172.21.15.70:0/1744245966 shutdown_connections 2023-10-08T05:49:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.677+0000 7faac51b7640 1 -- 172.21.15.70:0/1744245966 >> 172.21.15.70:0/1744245966 conn(0x7faac006b5d0 msgr2=0x7faac006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.677+0000 7faac51b7640 1 -- 172.21.15.70:0/1744245966 shutdown_connections 2023-10-08T05:49:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.677+0000 7faac51b7640 1 -- 172.21.15.70:0/1744245966 wait complete. 2023-10-08T05:49:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.679+0000 7faac51b7640 1 Processor -- start 2023-10-08T05:49:29.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.679+0000 7faac51b7640 1 -- start start 2023-10-08T05:49:29.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.680+0000 7faac51b7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa80047c0 0x7faac01541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.680+0000 7faac51b7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00c9d80 0x7faac01f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.680+0000 7faac51b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ca200 0x7faac01f2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.680+0000 7faac51b7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00cbb70 con 0x7faac00ca200 2023-10-08T05:49:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.680+0000 7faac51b7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00cb120 con 0x7faaa80047c0 2023-10-08T05:49:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.680+0000 7faac51b7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac00cca70 con 0x7faac00c9d80 2023-10-08T05:49:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.680+0000 7faabf59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ca200 0x7faac01f2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.680+0000 7faabf59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ca200 0x7faac01f2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44016/0 (socket says 172.21.15.70:44016) 2023-10-08T05:49:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.680+0000 7faabf59e640 1 -- 172.21.15.70:0/1750539396 learned_addr learned my addr 172.21.15.70:0/1750539396 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.680+0000 7faabed9d640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa80047c0 0x7faac01541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faabe59c640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00c9d80 0x7faac01f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faabf59e640 1 -- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00c9d80 msgr2=0x7faac01f2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faabf59e640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00c9d80 0x7faac01f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faabf59e640 1 -- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa80047c0 msgr2=0x7faac01541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faabf59e640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa80047c0 0x7faac01541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faabf59e640 1 -- 172.21.15.70:0/1750539396 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac00ccd50 con 0x7faac00ca200 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faabf59e640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ca200 0x7faac01f2a50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faac00cfb00 tx=0x7faab800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faaa77fe640 1 -- 172.21.15.70:0/1750539396 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800b8d0 con 0x7faac00ca200 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faaa77fe640 1 -- 172.21.15.70:0/1750539396 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab800ba70 con 0x7faac00ca200 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faaa77fe640 1 -- 172.21.15.70:0/1750539396 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800be20 con 0x7faac00ca200 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faac51b7640 1 -- 172.21.15.70:0/1750539396 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac00cb370 con 0x7faac00ca200 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.681+0000 7faac51b7640 1 -- 172.21.15.70:0/1750539396 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac0154710 con 0x7faac00ca200 2023-10-08T05:49:29.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.683+0000 7faac51b7640 1 -- 172.21.15.70:0/1750539396 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac00d19f0 con 0x7faac00ca200 2023-10-08T05:49:29.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.684+0000 7faaa77fe640 1 -- 172.21.15.70:0/1750539396 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7faab8007110 con 0x7faac00ca200 2023-10-08T05:49:29.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.684+0000 7faaa77fe640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7faaac06b070 0x7faaac06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:29.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.685+0000 7faaa77fe640 1 -- 172.21.15.70:0/1750539396 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7faab808a050 con 0x7faac00ca200 2023-10-08T05:49:29.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.685+0000 7faabed9d640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7faaac06b070 0x7faaac06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:29.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.687+0000 7faabed9d640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7faaac06b070 0x7faaac06d530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faac015c570 tx=0x7faab0008040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:29.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.689+0000 7faaa77fe640 1 -- 172.21.15.70:0/1750539396 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faab8054ba0 con 0x7faac00ca200 2023-10-08T05:49:29.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.898+0000 7faac51b7640 1 -- 172.21.15.70:0/1750539396 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faac00b7260 con 0x7faaac06b070 2023-10-08T05:49:29.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.917+0000 7faaa77fe640 1 -- 172.21.15.70:0/1750539396 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7faab0009850 con 0x7faaac06b070 2023-10-08T05:49:29.917 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:29.917 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:07", "first": "2023-10-08T05:39:01", "last": "2023-10-08T05:49:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 -- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7faaac06b070 msgr2=0x7faaac06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7faaac06b070 0x7faaac06d530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faac015c570 tx=0x7faab0008040 comp rx=0 tx=0).stop 2023-10-08T05:49:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 -- 172.21.15.70:0/1750539396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ca200 msgr2=0x7faac01f2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ca200 0x7faac01f2a50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faac00cfb00 tx=0x7faab800b0d0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 -- 172.21.15.70:0/1750539396 shutdown_connections 2023-10-08T05:49:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac00ca200 0x7faac01f2a50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7faaac06b070 0x7faaac06d530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faac00c9d80 0x7faac01f2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 --2- 172.21.15.70:0/1750539396 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faaa80047c0 0x7faac01541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:29.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 -- 172.21.15.70:0/1750539396 >> 172.21.15.70:0/1750539396 conn(0x7faac006b5d0 msgr2=0x7faac00b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.919+0000 7faac51b7640 1 -- 172.21.15.70:0/1750539396 shutdown_connections 2023-10-08T05:49:29.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:29.920+0000 7faac51b7640 1 -- 172.21.15.70:0/1750539396 wait complete. 2023-10-08T05:49:29.939 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:07", "first": "2023-10-08T05:39:01", "last": "2023-10-08T05:49:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:29.939 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:38:07', 'first': '2023-10-08T05:39:01', 'last': '2023-10-08T05:49:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T05:49:29.939 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T05:49:29.939 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T05:49:29.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:29.939 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-08T05:49:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:29.993 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:29.994 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:29.994 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:29.994 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:29.994 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:29.995 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_39_00_UTC', 'scheduled-2023-10-08-05_40_00_UTC', 'scheduled-2023-10-08-05_41_00_UTC', 'scheduled-2023-10-08-05_42_00_UTC', 'scheduled-2023-10-08-05_43_01_UTC', 'scheduled-2023-10-08-05_44_00_UTC', 'scheduled-2023-10-08-05_45_00_UTC', 'scheduled-2023-10-08-05_46_00_UTC', 'scheduled-2023-10-08-05_47_00_UTC', 'scheduled-2023-10-08-05_48_00_UTC', 'scheduled-2023-10-08-05_49_00_UTC'] 2023-10-08T05:49:29.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir4 2023-10-08T05:49:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.206+0000 7faa1671a640 1 Processor -- start 2023-10-08T05:49:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.207+0000 7faa1671a640 1 -- start start 2023-10-08T05:49:30.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.208+0000 7faa1671a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa100cf690 0x7faa100d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.208+0000 7faa1671a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa100cd520 con 0x7faa100ce510 2023-10-08T05:49:30.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.208+0000 7faa1671a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa100d1910 con 0x7faa100cf2b0 2023-10-08T05:49:30.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.208+0000 7faa1671a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa100ce080 con 0x7faa100cf690 2023-10-08T05:49:30.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.208+0000 7faa15f19640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7faa100ce510 0x7faa1015c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44424/0 (socket says 172.21.15.70:44424) 2023-10-08T05:49:30.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.208+0000 7faa15f19640 1 -- 172.21.15.70:0/595219705 learned_addr learned my addr 172.21.15.70:0/595219705 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.208+0000 7faa14f17640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa100cf690 0x7faa100d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724677018 0 0) 0x7faa100cd520 con 0x7faa100ce510 2023-10-08T05:49:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9fc003520 con 0x7faa100ce510 2023-10-08T05:49:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2451740003 0 0) 0x7fa9fc003520 con 0x7faa100ce510 2023-10-08T05:49:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa100cf690 msgr2=0x7faa100d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa100cf690 0x7faa100d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 >> v1:172.21.15.181:6789/0 conn(0x7faa100cf2b0 legacy=0x7faa100d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa100cd750 con 0x7faa100ce510 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7faa080019f0 con 0x7faa100ce510 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 >> v1:172.21.15.70:6789/0 conn(0x7faa100ce510 legacy=0x7faa1015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9fc0039a0 0x7fa9fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa100cf690 0x7fa9fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9fc004d00 0x7fa9fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100ce080 con 0x7fa9fc0039a0 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc00bc00 con 0x7fa9fc004d00 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc00bdc0 con 0x7faa100cf690 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.209+0000 7faa15718640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9fc0039a0 0x7fa9fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.210+0000 7faa14f17640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa100cf690 0x7fa9fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.210+0000 7faa15f19640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9fc004d00 0x7fa9fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.210+0000 7faa15718640 1 -- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa100cf690 msgr2=0x7fa9fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.210+0000 7faa15718640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa100cf690 0x7fa9fc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.210+0000 7faa15718640 1 -- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9fc004d00 msgr2=0x7fa9fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.210+0000 7faa15718640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9fc004d00 0x7fa9fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.210+0000 7faa15718640 1 -- 172.21.15.70:0/595219705 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc003520 con 0x7fa9fc0039a0 2023-10-08T05:49:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.210+0000 7faa15f19640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9fc004d00 0x7fa9fc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.211+0000 7faa15718640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9fc0039a0 0x7fa9fc003dc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faa0c001750 tx=0x7fa9fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc8c1352ae6feb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:30.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.211+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0052d0 con 0x7fa9fc0039a0 2023-10-08T05:49:30.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.211+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c02e050 con 0x7fa9fc0039a0 2023-10-08T05:49:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.211+0000 7faa07fff640 1 -- 172.21.15.70:0/595219705 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0035f0 con 0x7fa9fc0039a0 2023-10-08T05:49:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.212+0000 7faa1671a640 1 -- 172.21.15.70:0/595219705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9fc0039a0 msgr2=0x7fa9fc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.212+0000 7faa1671a640 1 --2- 172.21.15.70:0/595219705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9fc0039a0 0x7fa9fc003dc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faa0c001750 tx=0x7fa9fc0057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.212+0000 7faa15f19640 1 -- 172.21.15.70:0/595219705 reap_dead start 2023-10-08T05:49:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.212+0000 7faa1671a640 1 -- 172.21.15.70:0/595219705 shutdown_connections 2023-10-08T05:49:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.212+0000 7faa1671a640 1 -- 172.21.15.70:0/595219705 >> 172.21.15.70:0/595219705 conn(0x7faa1009bc40 msgr2=0x7faa100d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.212+0000 7faa1671a640 1 -- 172.21.15.70:0/595219705 shutdown_connections 2023-10-08T05:49:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.212+0000 7faa1671a640 1 -- 172.21.15.70:0/595219705 wait complete. 2023-10-08T05:49:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.214+0000 7faa1671a640 1 Processor -- start 2023-10-08T05:49:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.214+0000 7faa1671a640 1 -- start start 2023-10-08T05:49:30.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa1671a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9fc0039a0 0x7faa100c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa1671a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1015dd30 0x7faa1015e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa1671a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa101f2540 0x7faa101f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa1671a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1015f520 con 0x7faa101f2540 2023-10-08T05:49:30.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa1671a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa100c8a40 con 0x7fa9fc0039a0 2023-10-08T05:49:30.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa1671a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa101f3450 con 0x7faa1015dd30 2023-10-08T05:49:30.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa15f19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa101f2540 0x7faa101f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa15f19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa101f2540 0x7faa101f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44040/0 (socket says 172.21.15.70:44040) 2023-10-08T05:49:30.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa15f19640 1 -- 172.21.15.70:0/419470174 learned_addr learned my addr 172.21.15.70:0/419470174 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:30.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa15718640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9fc0039a0 0x7faa100c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa14f17640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1015dd30 0x7faa1015e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa15f19640 1 -- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1015dd30 msgr2=0x7faa1015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:30.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa15f19640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1015dd30 0x7faa1015e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.215+0000 7faa15f19640 1 -- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9fc0039a0 msgr2=0x7faa100c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.216+0000 7faa15f19640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9fc0039a0 0x7faa100c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.216+0000 7faa15f19640 1 -- 172.21.15.70:0/419470174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa101f2f30 con 0x7faa101f2540 2023-10-08T05:49:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.216+0000 7faa15f19640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa101f2540 0x7faa101f29f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faa08001580 tx=0x7faa0800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.216+0000 7faa05ffb640 1 -- 172.21.15.70:0/419470174 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa08004d60 con 0x7faa101f2540 2023-10-08T05:49:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.216+0000 7faa05ffb640 1 -- 172.21.15.70:0/419470174 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0800bb20 con 0x7faa101f2540 2023-10-08T05:49:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.216+0000 7faa1671a640 1 -- 172.21.15.70:0/419470174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1015e8a0 con 0x7faa101f2540 2023-10-08T05:49:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.216+0000 7faa05ffb640 1 -- 172.21.15.70:0/419470174 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800bed0 con 0x7faa101f2540 2023-10-08T05:49:30.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.216+0000 7faa1671a640 1 -- 172.21.15.70:0/419470174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1015ee60 con 0x7faa101f2540 2023-10-08T05:49:30.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.218+0000 7faa1671a640 1 -- 172.21.15.70:0/419470174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa10074150 con 0x7faa101f2540 2023-10-08T05:49:30.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.218+0000 7faa05ffb640 1 -- 172.21.15.70:0/419470174 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7faa08007110 con 0x7faa101f2540 2023-10-08T05:49:30.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.220+0000 7faa05ffb640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa9f406b0a0 0x7fa9f406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.220+0000 7faa05ffb640 1 -- 172.21.15.70:0/419470174 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7faa08089fc0 con 0x7faa101f2540 2023-10-08T05:49:30.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.220+0000 7faa15718640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa9f406b0a0 0x7fa9f406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.221+0000 7faa15718640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa9f406b0a0 0x7fa9f406d560 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faa0c033000 tx=0x7faa0c009030 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:30.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.224+0000 7faa05ffb640 1 -- 172.21.15.70:0/419470174 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa08054b10 con 0x7faa101f2540 2023-10-08T05:49:30.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.437+0000 7faa1671a640 1 -- 172.21.15.70:0/419470174 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faa100cdd10 con 0x7fa9f406b0a0 2023-10-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.458+0000 7faa05ffb640 1 -- 172.21.15.70:0/419470174 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7faa0c005370 con 0x7fa9f406b0a0 2023-10-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:30.458 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:07", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:30.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 -- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa9f406b0a0 msgr2=0x7fa9f406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:30.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa9f406b0a0 0x7fa9f406d560 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faa0c033000 tx=0x7faa0c009030 comp rx=0 tx=0).stop 2023-10-08T05:49:30.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 -- 172.21.15.70:0/419470174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa101f2540 msgr2=0x7faa101f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:30.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa101f2540 0x7faa101f29f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faa08001580 tx=0x7faa0800b4a0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 -- 172.21.15.70:0/419470174 shutdown_connections 2023-10-08T05:49:30.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa101f2540 0x7faa101f29f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa9f406b0a0 0x7fa9f406d560 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa1015dd30 0x7faa1015e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 --2- 172.21.15.70:0/419470174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9fc0039a0 0x7faa100c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 -- 172.21.15.70:0/419470174 >> 172.21.15.70:0/419470174 conn(0x7faa1009bc40 msgr2=0x7faa100d1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:30.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 -- 172.21.15.70:0/419470174 shutdown_connections 2023-10-08T05:49:30.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.461+0000 7faa1671a640 1 -- 172.21.15.70:0/419470174 wait complete. 2023-10-08T05:49:30.480 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:07", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:30.481 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:38:07', 'first': '2023-10-08T05:39:00', 'last': '2023-10-08T05:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T05:49:30.481 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T05:49:30.481 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T05:49:30.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:30.481 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-08T05:49:30.537 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_01_UTC 2023-10-08T05:49:30.537 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_01_UTC 2023-10-08T05:49:30.537 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:30.537 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:30.538 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:30.538 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_01_UTC 2023-10-08T05:49:30.538 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:30.538 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:30.538 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:30.538 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:30.538 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:30.539 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_39_01_UTC', 'scheduled-2023-10-08-05_40_01_UTC', 'scheduled-2023-10-08-05_41_00_UTC', 'scheduled-2023-10-08-05_42_00_UTC', 'scheduled-2023-10-08-05_43_00_UTC', 'scheduled-2023-10-08-05_44_01_UTC', 'scheduled-2023-10-08-05_45_01_UTC', 'scheduled-2023-10-08-05_46_00_UTC', 'scheduled-2023-10-08-05_47_00_UTC', 'scheduled-2023-10-08-05_48_00_UTC', 'scheduled-2023-10-08-05_49_00_UTC'] 2023-10-08T05:49:30.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir5 2023-10-08T05:49:30.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.756+0000 7f5168943640 1 Processor -- start 2023-10-08T05:49:30.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.756+0000 7f5168943640 1 -- start start 2023-10-08T05:49:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.757+0000 7f5168943640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5164156a10 0x7f51641594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.757+0000 7f5168943640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5164070ac0 0x7f51640aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.757+0000 7f5168943640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5164071d90 con 0x7f51640b04b0 2023-10-08T05:49:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.758+0000 7f5168943640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51640b0900 con 0x7f5164156a10 2023-10-08T05:49:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.758+0000 7f5168943640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5164070f20 con 0x7f5164070ac0 2023-10-08T05:49:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.758+0000 7f51637fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5164156a10 0x7f51641594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.758+0000 7f51637fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5164156a10 0x7f51641594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44046/0 (socket says 172.21.15.70:44046) 2023-10-08T05:49:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.758+0000 7f51637fe640 1 -- 172.21.15.70:0/1898548730 learned_addr learned my addr 172.21.15.70:0/1898548730 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.758+0000 7f51637fe640 1 -- 172.21.15.70:0/1898548730 >> v2:172.21.15.181:3300/0 conn(0x7f5164070ac0 msgr2=0x7f51640aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.758+0000 7f51637fe640 1 --2- 172.21.15.70:0/1898548730 >> v2:172.21.15.181:3300/0 conn(0x7f5164070ac0 0x7f51640aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.758+0000 7f51637fe640 1 -- 172.21.15.70:0/1898548730 >> v1:172.21.15.181:6789/0 conn(0x7f51640b04b0 legacy=0x7f516415cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:30.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.758+0000 7f51637fe640 1 -- 172.21.15.70:0/1898548730 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5164071180 con 0x7f5164156a10 2023-10-08T05:49:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.759+0000 7f51637fe640 1 --2- 172.21.15.70:0/1898548730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164156a10 0x7f51641594e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f516406e1a0 tx=0x7f515402cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c478597a60a7922 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.760+0000 7f5162ffd640 1 -- 172.21.15.70:0/1898548730 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515402dba0 con 0x7f5164156a10 2023-10-08T05:49:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.760+0000 7f5162ffd640 1 -- 172.21.15.70:0/1898548730 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515402dd00 con 0x7f5164156a10 2023-10-08T05:49:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.760+0000 7f5162ffd640 1 -- 172.21.15.70:0/1898548730 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154035660 con 0x7f5164156a10 2023-10-08T05:49:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.760+0000 7f5168943640 1 -- 172.21.15.70:0/1898548730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164156a10 msgr2=0x7f51641594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.760+0000 7f5168943640 1 --2- 172.21.15.70:0/1898548730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164156a10 0x7f51641594e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f516406e1a0 tx=0x7f515402cdf0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.760+0000 7f5168943640 1 -- 172.21.15.70:0/1898548730 shutdown_connections 2023-10-08T05:49:30.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.760+0000 7f5168943640 1 --2- 172.21.15.70:0/1898548730 >> v2:172.21.15.181:3300/0 conn(0x7f5164070ac0 0x7f51640aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.760+0000 7f5168943640 1 --2- 172.21.15.70:0/1898548730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164156a10 0x7f51641594e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.760+0000 7f5168943640 1 -- 172.21.15.70:0/1898548730 >> 172.21.15.70:0/1898548730 conn(0x7f516406c050 msgr2=0x7f516406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:30.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.761+0000 7f5168943640 1 -- 172.21.15.70:0/1898548730 shutdown_connections 2023-10-08T05:49:30.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.761+0000 7f5168943640 1 -- 172.21.15.70:0/1898548730 wait complete. 2023-10-08T05:49:30.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.762+0000 7f5168943640 1 Processor -- start 2023-10-08T05:49:30.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.763+0000 7f5168943640 1 -- start start 2023-10-08T05:49:30.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f5168943640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164070ac0 0x7f51640b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f5168943640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51640b04b0 0x7f51640c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f5168943640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5164156a10 0x7f51641f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f5168943640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51640b1fd0 con 0x7f5164070ac0 2023-10-08T05:49:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f5168943640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51640b13c0 con 0x7f51640b04b0 2023-10-08T05:49:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f5168943640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51640b16a0 con 0x7f5164156a10 2023-10-08T05:49:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f51637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164070ac0 0x7f51640b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f51637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164070ac0 0x7f51640b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44060/0 (socket says 172.21.15.70:44060) 2023-10-08T05:49:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f51637fe640 1 -- 172.21.15.70:0/526373373 learned_addr learned my addr 172.21.15.70:0/526373373 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f51637fe640 1 -- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5164156a10 msgr2=0x7f51641f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f515bfff640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51640b04b0 0x7f51640c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f51637fe640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5164156a10 0x7f51641f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f51637fe640 1 -- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51640b04b0 msgr2=0x7f51640c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f51637fe640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51640b04b0 0x7f51640c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f51637fe640 1 -- 172.21.15.70:0/526373373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515400a670 con 0x7f5164070ac0 2023-10-08T05:49:30.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.764+0000 7f515bfff640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51640b04b0 0x7f51640c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:30.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.765+0000 7f51637fe640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164070ac0 0x7f51640b2460 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f515402ce30 tx=0x7f51540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:30.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.765+0000 7f5160ff9640 1 -- 172.21.15.70:0/526373373 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154040850 con 0x7f5164070ac0 2023-10-08T05:49:30.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.765+0000 7f5160ff9640 1 -- 172.21.15.70:0/526373373 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51540409b0 con 0x7f5164070ac0 2023-10-08T05:49:30.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.765+0000 7f5160ff9640 1 -- 172.21.15.70:0/526373373 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5154040d60 con 0x7f5164070ac0 2023-10-08T05:49:30.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.765+0000 7f5168943640 1 -- 172.21.15.70:0/526373373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51640b1980 con 0x7f5164070ac0 2023-10-08T05:49:30.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.765+0000 7f5168943640 1 -- 172.21.15.70:0/526373373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51640b29a0 con 0x7f5164070ac0 2023-10-08T05:49:30.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.766+0000 7f5168943640 1 -- 172.21.15.70:0/526373373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5164074150 con 0x7f5164070ac0 2023-10-08T05:49:30.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.767+0000 7f5160ff9640 1 -- 172.21.15.70:0/526373373 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f515403c040 con 0x7f5164070ac0 2023-10-08T05:49:30.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.768+0000 7f5160ff9640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f514406b0a0 0x7f514406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:30.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.768+0000 7f5160ff9640 1 -- 172.21.15.70:0/526373373 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f51540ac280 con 0x7f5164070ac0 2023-10-08T05:49:30.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.769+0000 7f515bfff640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f514406b0a0 0x7f514406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:30.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.770+0000 7f515bfff640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f514406b0a0 0x7f514406d560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f51640b4180 tx=0x7f514c008040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:30.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.772+0000 7f5160ff9640 1 -- 172.21.15.70:0/526373373 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5154076e40 con 0x7f5164070ac0 2023-10-08T05:49:30.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.981+0000 7f5168943640 1 -- 172.21.15.70:0/526373373 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5164071740 con 0x7f514406b0a0 2023-10-08T05:49:31.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:30.999+0000 7f5160ff9640 1 -- 172.21.15.70:0/526373373 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f514c00a460 con 0x7f514406b0a0 2023-10-08T05:49:31.000 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:31.000 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:08", "first": "2023-10-08T05:39:01", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:31.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.001+0000 7f5168943640 1 -- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f514406b0a0 msgr2=0x7f514406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.001+0000 7f5168943640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f514406b0a0 0x7f514406d560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f51640b4180 tx=0x7f514c008040 comp rx=0 tx=0).stop 2023-10-08T05:49:31.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.002+0000 7f5168943640 1 -- 172.21.15.70:0/526373373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164070ac0 msgr2=0x7f51640b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.002+0000 7f5168943640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164070ac0 0x7f51640b2460 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f515402ce30 tx=0x7f51540046c0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.002+0000 7f5168943640 1 -- 172.21.15.70:0/526373373 shutdown_connections 2023-10-08T05:49:31.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.002+0000 7f5168943640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f514406b0a0 0x7f514406d560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.002+0000 7f5168943640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5164156a10 0x7f51641f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.002+0000 7f5168943640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51640b04b0 0x7f51640c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.002+0000 7f5168943640 1 --2- 172.21.15.70:0/526373373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5164070ac0 0x7f51640b2460 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.002+0000 7f5168943640 1 -- 172.21.15.70:0/526373373 >> 172.21.15.70:0/526373373 conn(0x7f516406c050 msgr2=0x7f51641557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:31.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.002+0000 7f5168943640 1 -- 172.21.15.70:0/526373373 shutdown_connections 2023-10-08T05:49:31.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.002+0000 7f5168943640 1 -- 172.21.15.70:0/526373373 wait complete. 2023-10-08T05:49:31.020 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:08", "first": "2023-10-08T05:39:01", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:31.021 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:38:08', 'first': '2023-10-08T05:39:01', 'last': '2023-10-08T05:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T05:49:31.021 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T05:49:31.021 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T05:49:31.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:31.021 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-08T05:49:31.073 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:31.074 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:31.074 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:31.074 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:31.074 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:31.074 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:31.074 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:31.074 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:31.074 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_01_UTC 2023-10-08T05:49:31.074 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_01_UTC 2023-10-08T05:49:31.074 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:31.075 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_39_00_UTC', 'scheduled-2023-10-08-05_40_00_UTC', 'scheduled-2023-10-08-05_41_00_UTC', 'scheduled-2023-10-08-05_42_00_UTC', 'scheduled-2023-10-08-05_43_00_UTC', 'scheduled-2023-10-08-05_44_00_UTC', 'scheduled-2023-10-08-05_45_01_UTC', 'scheduled-2023-10-08-05_46_01_UTC', 'scheduled-2023-10-08-05_47_01_UTC', 'scheduled-2023-10-08-05_48_01_UTC', 'scheduled-2023-10-08-05_49_01_UTC'] 2023-10-08T05:49:31.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir6 2023-10-08T05:49:31.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.291+0000 7f11dafd7640 1 Processor -- start 2023-10-08T05:49:31.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.291+0000 7f11dafd7640 1 -- start start 2023-10-08T05:49:31.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.293+0000 7f11dafd7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f11d406dfc0 0x7f11d4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.293+0000 7f11dafd7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11d4071070 0x7f11d40cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.293+0000 7f11dafd7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d40b1cc0 con 0x7f11d40afbe0 2023-10-08T05:49:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.293+0000 7f11dafd7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d40b2620 con 0x7f11d406dfc0 2023-10-08T05:49:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.293+0000 7f11dafd7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40b2060 con 0x7f11d4071070 2023-10-08T05:49:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.293+0000 7f11d8d73640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f11d40afbe0 0x7f11d40b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44426/0 (socket says 172.21.15.70:44426) 2023-10-08T05:49:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.293+0000 7f11d8d73640 1 -- 172.21.15.70:0/2213957207 learned_addr learned my addr 172.21.15.70:0/2213957207 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:31.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.293+0000 7f11d3fff640 1 --2- 172.21.15.70:0/2213957207 >> v2:172.21.15.70:3300/0 conn(0x7f11d406dfc0 0x7f11d4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:31.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.294+0000 7f11d9574640 1 --2- 172.21.15.70:0/2213957207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11d4071070 0x7f11d40cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:31.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.294+0000 7f11d3fff640 1 -- 172.21.15.70:0/2213957207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11d4071070 msgr2=0x7f11d40cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:31.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.294+0000 7f11d3fff640 1 --2- 172.21.15.70:0/2213957207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11d4071070 0x7f11d40cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.294+0000 7f11d3fff640 1 -- 172.21.15.70:0/2213957207 >> v1:172.21.15.70:6789/0 conn(0x7f11d40afbe0 legacy=0x7f11d40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.294+0000 7f11d3fff640 1 -- 172.21.15.70:0/2213957207 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d40b2340 con 0x7f11d406dfc0 2023-10-08T05:49:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.294+0000 7f11d37fe640 1 -- 172.21.15.70:0/2213957207 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354326805 0 0) 0x7f11d40b1cc0 con 0x7f11d40afbe0 2023-10-08T05:49:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.294+0000 7f11d9574640 1 --2- 172.21.15.70:0/2213957207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11d4071070 0x7f11d40cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.295+0000 7f11d3fff640 1 --2- 172.21.15.70:0/2213957207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d406dfc0 0x7f11d4070b30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f11c00097a0 tx=0x7f11c002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3527875a9b3609 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.295+0000 7f11d37fe640 1 -- 172.21.15.70:0/2213957207 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c002d040 con 0x7f11d406dfc0 2023-10-08T05:49:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.295+0000 7f11d37fe640 1 -- 172.21.15.70:0/2213957207 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c0031810 con 0x7f11d406dfc0 2023-10-08T05:49:31.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.295+0000 7f11d37fe640 1 -- 172.21.15.70:0/2213957207 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0031ba0 con 0x7f11d406dfc0 2023-10-08T05:49:31.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.296+0000 7f11dafd7640 1 -- 172.21.15.70:0/2213957207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d406dfc0 msgr2=0x7f11d4070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.296+0000 7f11dafd7640 1 --2- 172.21.15.70:0/2213957207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d406dfc0 0x7f11d4070b30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f11c00097a0 tx=0x7f11c002b8b0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.296+0000 7f11dafd7640 1 -- 172.21.15.70:0/2213957207 shutdown_connections 2023-10-08T05:49:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.296+0000 7f11dafd7640 1 --2- 172.21.15.70:0/2213957207 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11d4071070 0x7f11d40cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.296+0000 7f11dafd7640 1 --2- 172.21.15.70:0/2213957207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d406dfc0 0x7f11d4070b30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.296+0000 7f11dafd7640 1 -- 172.21.15.70:0/2213957207 >> 172.21.15.70:0/2213957207 conn(0x7f11d406a5d0 msgr2=0x7f11d40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.296+0000 7f11dafd7640 1 -- 172.21.15.70:0/2213957207 shutdown_connections 2023-10-08T05:49:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.296+0000 7f11dafd7640 1 -- 172.21.15.70:0/2213957207 wait complete. 2023-10-08T05:49:31.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.298+0000 7f11dafd7640 1 Processor -- start 2023-10-08T05:49:31.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.298+0000 7f11dafd7640 1 -- start start 2023-10-08T05:49:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.299+0000 7f11dafd7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11d406dfc0 0x7f11d41f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.299+0000 7f11dafd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4071070 0x7f11d40ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.299+0000 7f11dafd7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11d40afbe0 0x7f11d415de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.299+0000 7f11dafd7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40cad00 con 0x7f11d4071070 2023-10-08T05:49:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.299+0000 7f11dafd7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40c5150 con 0x7f11d40afbe0 2023-10-08T05:49:31.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.299+0000 7f11dafd7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d415cb00 con 0x7f11d406dfc0 2023-10-08T05:49:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.299+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4071070 0x7f11d40ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.299+0000 7f11d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4071070 0x7f11d40ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44072/0 (socket says 172.21.15.70:44072) 2023-10-08T05:49:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.299+0000 7f11d3fff640 1 -- 172.21.15.70:0/2978927139 learned_addr learned my addr 172.21.15.70:0/2978927139 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11d3fff640 1 -- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11d406dfc0 msgr2=0x7f11d41f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11d3fff640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11d406dfc0 0x7f11d41f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11d9574640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11d40afbe0 0x7f11d415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11d3fff640 1 -- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11d40afbe0 msgr2=0x7f11d415de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11d3fff640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11d40afbe0 0x7f11d415de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11d3fff640 1 -- 172.21.15.70:0/2978927139 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0009400 con 0x7f11d4071070 2023-10-08T05:49:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11d3fff640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4071070 0x7f11d40ca7c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f11c00098c0 tx=0x7f11c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:31.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11d17fa640 1 -- 172.21.15.70:0/2978927139 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c003ad60 con 0x7f11d4071070 2023-10-08T05:49:31.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11d17fa640 1 -- 172.21.15.70:0/2978927139 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c00443f0 con 0x7f11d4071070 2023-10-08T05:49:31.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11d17fa640 1 -- 172.21.15.70:0/2978927139 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c00447a0 con 0x7f11d4071070 2023-10-08T05:49:31.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.300+0000 7f11dafd7640 1 -- 172.21.15.70:0/2978927139 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d415cde0 con 0x7f11d4071070 2023-10-08T05:49:31.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.301+0000 7f11dafd7640 1 -- 172.21.15.70:0/2978927139 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d415d3a0 con 0x7f11d4071070 2023-10-08T05:49:31.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.302+0000 7f11dafd7640 1 -- 172.21.15.70:0/2978927139 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d404e160 con 0x7f11d4071070 2023-10-08T05:49:31.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.302+0000 7f11d17fa640 1 -- 172.21.15.70:0/2978927139 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f11c004d450 con 0x7f11d4071070 2023-10-08T05:49:31.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.303+0000 7f11d17fa640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f11b806b070 0x7f11b806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.303+0000 7f11d17fa640 1 -- 172.21.15.70:0/2978927139 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f11c00ac1f0 con 0x7f11d4071070 2023-10-08T05:49:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.304+0000 7f11d8d73640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f11b806b070 0x7f11b806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:31.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.305+0000 7f11d8d73640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f11b806b070 0x7f11b806d530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f11c8007fd0 tx=0x7f11c8007960 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.308+0000 7f11d17fa640 1 -- 172.21.15.70:0/2978927139 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11c0076c10 con 0x7f11d4071070 2023-10-08T05:49:31.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.522+0000 7f11dafd7640 1 -- 172.21.15.70:0/2978927139 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f11d4070630 con 0x7f11b806b070 2023-10-08T05:49:31.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.544+0000 7f11d17fa640 1 -- 172.21.15.70:0/2978927139 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f11c8001220 con 0x7f11b806b070 2023-10-08T05:49:31.545 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:31.545 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:09", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:31.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.546+0000 7f11dafd7640 1 -- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f11b806b070 msgr2=0x7f11b806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.546+0000 7f11dafd7640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f11b806b070 0x7f11b806d530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f11c8007fd0 tx=0x7f11c8007960 comp rx=0 tx=0).stop 2023-10-08T05:49:31.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.546+0000 7f11dafd7640 1 -- 172.21.15.70:0/2978927139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4071070 msgr2=0x7f11d40ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.546+0000 7f11dafd7640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4071070 0x7f11d40ca7c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f11c00098c0 tx=0x7f11c00046c0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.547+0000 7f11dafd7640 1 -- 172.21.15.70:0/2978927139 shutdown_connections 2023-10-08T05:49:31.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.547+0000 7f11dafd7640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11d40afbe0 0x7f11d415de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.547+0000 7f11dafd7640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d4071070 0x7f11d40ca7c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.547+0000 7f11dafd7640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f11b806b070 0x7f11b806d530 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.547+0000 7f11dafd7640 1 --2- 172.21.15.70:0/2978927139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11d406dfc0 0x7f11d41f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.547+0000 7f11dafd7640 1 -- 172.21.15.70:0/2978927139 >> 172.21.15.70:0/2978927139 conn(0x7f11d406a5d0 msgr2=0x7f11d40c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:31.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.547+0000 7f11dafd7640 1 -- 172.21.15.70:0/2978927139 shutdown_connections 2023-10-08T05:49:31.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.547+0000 7f11dafd7640 1 -- 172.21.15.70:0/2978927139 wait complete. 2023-10-08T05:49:31.566 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:09", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:31.567 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:38:09', 'first': '2023-10-08T05:39:00', 'last': '2023-10-08T05:49:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T05:49:31.567 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T05:49:31.567 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T05:49:31.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:31.567 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-08T05:49:31.619 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:31.620 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:31.620 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:31.620 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:31.620 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:31.620 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:31.620 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:31.620 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:31.621 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_01_UTC 2023-10-08T05:49:31.621 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_01_UTC 2023-10-08T05:49:31.621 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:31.622 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_39_00_UTC', 'scheduled-2023-10-08-05_40_00_UTC', 'scheduled-2023-10-08-05_41_00_UTC', 'scheduled-2023-10-08-05_42_00_UTC', 'scheduled-2023-10-08-05_43_00_UTC', 'scheduled-2023-10-08-05_44_00_UTC', 'scheduled-2023-10-08-05_45_00_UTC', 'scheduled-2023-10-08-05_46_01_UTC', 'scheduled-2023-10-08-05_47_01_UTC', 'scheduled-2023-10-08-05_48_01_UTC', 'scheduled-2023-10-08-05_49_01_UTC'] 2023-10-08T05:49:31.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir7 2023-10-08T05:49:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.838+0000 7f7bbfae5640 1 Processor -- start 2023-10-08T05:49:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.838+0000 7f7bbfae5640 1 -- start start 2023-10-08T05:49:31.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbfae5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bb80afbe0 0x7f7bb80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbfae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bb806dfc0 0x7f7bb8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbfae5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb80b2620 con 0x7f7bb8071070 2023-10-08T05:49:31.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbfae5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb80b2060 con 0x7f7bb80afbe0 2023-10-08T05:49:31.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbfae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80b2340 con 0x7f7bb806dfc0 2023-10-08T05:49:31.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbd881640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bb80afbe0 0x7f7bb80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:31.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbd881640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bb80afbe0 0x7f7bb80b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44076/0 (socket says 172.21.15.70:44076) 2023-10-08T05:49:31.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbd881640 1 -- 172.21.15.70:0/2295968670 learned_addr learned my addr 172.21.15.70:0/2295968670 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:31.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbd881640 1 -- 172.21.15.70:0/2295968670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bb806dfc0 msgr2=0x7f7bb8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:31.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbd881640 1 --2- 172.21.15.70:0/2295968670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bb806dfc0 0x7f7bb8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.840+0000 7f7bbd881640 1 -- 172.21.15.70:0/2295968670 >> v1:172.21.15.181:6789/0 conn(0x7f7bb8071070 legacy=0x7f7bb80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:31.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.841+0000 7f7bbd881640 1 -- 172.21.15.70:0/2295968670 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8070350 con 0x7f7bb80afbe0 2023-10-08T05:49:31.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.842+0000 7f7bbd881640 1 --2- 172.21.15.70:0/2295968670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb80afbe0 0x7f7bb80b15b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009870 tx=0x7f7bb402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbd483274aa1cd9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:31.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.842+0000 7f7bbc87f640 1 -- 172.21.15.70:0/2295968670 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4031640 con 0x7f7bb80afbe0 2023-10-08T05:49:31.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.842+0000 7f7bbc87f640 1 -- 172.21.15.70:0/2295968670 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb40317a0 con 0x7f7bb80afbe0 2023-10-08T05:49:31.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.842+0000 7f7bbc87f640 1 -- 172.21.15.70:0/2295968670 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4031b30 con 0x7f7bb80afbe0 2023-10-08T05:49:31.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.843+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2295968670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb80afbe0 msgr2=0x7f7bb80b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.843+0000 7f7bbfae5640 1 --2- 172.21.15.70:0/2295968670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb80afbe0 0x7f7bb80b15b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009870 tx=0x7f7bb402b8f0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.843+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2295968670 shutdown_connections 2023-10-08T05:49:31.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.843+0000 7f7bbfae5640 1 --2- 172.21.15.70:0/2295968670 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bb806dfc0 0x7f7bb8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.843+0000 7f7bbfae5640 1 --2- 172.21.15.70:0/2295968670 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb80afbe0 0x7f7bb80b15b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.843+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2295968670 >> 172.21.15.70:0/2295968670 conn(0x7f7bb806a5d0 msgr2=0x7f7bb80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:31.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.843+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2295968670 shutdown_connections 2023-10-08T05:49:31.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.843+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2295968670 wait complete. 2023-10-08T05:49:31.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.846+0000 7f7bbfae5640 1 Processor -- start 2023-10-08T05:49:31.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.846+0000 7f7bbfae5640 1 -- start start 2023-10-08T05:49:31.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.847+0000 7f7bbfae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bb806dfc0 0x7f7bb8151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.847+0000 7f7bbfae5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb8071070 0x7f7bb80c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.847+0000 7f7bbfae5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bb80afbe0 0x7f7bb80ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.847+0000 7f7bbfae5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80c3060 con 0x7f7bb8071070 2023-10-08T05:49:31.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.847+0000 7f7bbfae5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80c3340 con 0x7f7bb80afbe0 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.847+0000 7f7bbfae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80c3620 con 0x7f7bb806dfc0 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.848+0000 7f7bbd080640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb8071070 0x7f7bb80c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.848+0000 7f7bbd080640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb8071070 0x7f7bb80c2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44088/0 (socket says 172.21.15.70:44088) 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.848+0000 7f7bbd080640 1 -- 172.21.15.70:0/2710144817 learned_addr learned my addr 172.21.15.70:0/2710144817 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.848+0000 7f7bbd080640 1 -- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bb806dfc0 msgr2=0x7f7bb8151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.848+0000 7f7bbd080640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bb806dfc0 0x7f7bb8151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.848+0000 7f7bbd080640 1 -- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bb80afbe0 msgr2=0x7f7bb80ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.848+0000 7f7bbd080640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bb80afbe0 0x7f7bb80ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.848+0000 7f7bbd080640 1 -- 172.21.15.70:0/2710144817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb40094d0 con 0x7f7bb8071070 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.848+0000 7f7bbd080640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb8071070 0x7f7bb80c2610 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800b410 tx=0x7f7ba800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:31.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.849+0000 7f7bae7fc640 1 -- 172.21.15.70:0/2710144817 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011830 con 0x7f7bb8071070 2023-10-08T05:49:31.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.849+0000 7f7bae7fc640 1 -- 172.21.15.70:0/2710144817 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8011990 con 0x7f7bb8071070 2023-10-08T05:49:31.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.849+0000 7f7bae7fc640 1 -- 172.21.15.70:0/2710144817 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011d40 con 0x7f7bb8071070 2023-10-08T05:49:31.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.849+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2710144817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb80c3900 con 0x7f7bb8071070 2023-10-08T05:49:31.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.849+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2710144817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb80c4600 con 0x7f7bb8071070 2023-10-08T05:49:31.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.850+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2710144817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb804e160 con 0x7f7bb8071070 2023-10-08T05:49:31.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.851+0000 7f7bae7fc640 1 -- 172.21.15.70:0/2710144817 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f7ba801a400 con 0x7f7bb8071070 2023-10-08T05:49:31.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.852+0000 7f7bae7fc640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f7b9c06b070 0x7f7b9c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:31.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.852+0000 7f7bae7fc640 1 -- 172.21.15.70:0/2710144817 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f7ba8089df0 con 0x7f7bb8071070 2023-10-08T05:49:31.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.852+0000 7f7bbd881640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f7b9c06b070 0x7f7b9c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:31.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.853+0000 7f7bbd881640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f7b9c06b070 0x7f7b9c06d530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4036000 tx=0x7f7bb4030000 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:31.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:31.856+0000 7f7bae7fc640 1 -- 172.21.15.70:0/2710144817 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ba8054920 con 0x7f7bb8071070 2023-10-08T05:49:32.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.067+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2710144817 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7bb80affc0 con 0x7f7b9c06b070 2023-10-08T05:49:32.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.086+0000 7f7bae7fc640 1 -- 172.21.15.70:0/2710144817 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f7bb4031a10 con 0x7f7b9c06b070 2023-10-08T05:49:32.086 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:32.086 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:09", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.088+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f7b9c06b070 msgr2=0x7f7b9c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.088+0000 7f7bbfae5640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f7b9c06b070 0x7f7b9c06d530 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4036000 tx=0x7f7bb4030000 comp rx=0 tx=0).stop 2023-10-08T05:49:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.088+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2710144817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb8071070 msgr2=0x7f7bb80c2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.088+0000 7f7bbfae5640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb8071070 0x7f7bb80c2610 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800b410 tx=0x7f7ba800b8d0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.089+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2710144817 shutdown_connections 2023-10-08T05:49:32.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.089+0000 7f7bbfae5640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bb80afbe0 0x7f7bb80ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.089+0000 7f7bbfae5640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bb8071070 0x7f7bb80c2610 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.089+0000 7f7bbfae5640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f7b9c06b070 0x7f7b9c06d530 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.089+0000 7f7bbfae5640 1 --2- 172.21.15.70:0/2710144817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bb806dfc0 0x7f7bb8151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.089+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2710144817 >> 172.21.15.70:0/2710144817 conn(0x7f7bb806a5d0 msgr2=0x7f7bb80c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:32.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.089+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2710144817 shutdown_connections 2023-10-08T05:49:32.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.089+0000 7f7bbfae5640 1 -- 172.21.15.70:0/2710144817 wait complete. 2023-10-08T05:49:32.107 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:09", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:01", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:32.107 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:38:09', 'first': '2023-10-08T05:39:00', 'last': '2023-10-08T05:49:01', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T05:49:32.108 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T05:49:32.108 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T05:49:32.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:32.108 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-08T05:49:32.162 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_01_UTC 2023-10-08T05:49:32.162 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:32.162 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:32.162 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:32.162 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:32.163 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:32.163 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:32.163 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:32.163 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:32.163 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:32.163 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:32.164 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_39_01_UTC', 'scheduled-2023-10-08-05_40_00_UTC', 'scheduled-2023-10-08-05_41_00_UTC', 'scheduled-2023-10-08-05_42_00_UTC', 'scheduled-2023-10-08-05_43_00_UTC', 'scheduled-2023-10-08-05_44_00_UTC', 'scheduled-2023-10-08-05_45_00_UTC', 'scheduled-2023-10-08-05_46_00_UTC', 'scheduled-2023-10-08-05_47_00_UTC', 'scheduled-2023-10-08-05_48_00_UTC', 'scheduled-2023-10-08-05_49_00_UTC'] 2023-10-08T05:49:32.164 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir8 2023-10-08T05:49:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.380+0000 7fedd1df0640 1 Processor -- start 2023-10-08T05:49:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.381+0000 7fedd1df0640 1 -- start start 2023-10-08T05:49:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.382+0000 7fedd1df0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fedcc070cc0 0x7fedcc0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.382+0000 7fedd1df0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc071170 con 0x7fedcc160660 2023-10-08T05:49:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.382+0000 7fedd1df0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc06e150 con 0x7fedcc0cb450 2023-10-08T05:49:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.382+0000 7fedd1df0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc06de70 con 0x7fedcc070cc0 2023-10-08T05:49:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.383+0000 7fedcaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fedcc070cc0 0x7fedcc0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.383+0000 7fedcaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fedcc070cc0 0x7fedcc0caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44090/0 (socket says 172.21.15.70:44090) 2023-10-08T05:49:32.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.383+0000 7fedcb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fedcc160660 0x7fedcc0c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44432/0 (socket says 172.21.15.70:44432) 2023-10-08T05:49:32.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.383+0000 7fedcaffd640 1 -- 172.21.15.70:0/247635400 learned_addr learned my addr 172.21.15.70:0/247635400 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.383+0000 7fedcaffd640 1 -- 172.21.15.70:0/247635400 >> v1:172.21.15.181:6789/0 conn(0x7fedcc0cb450 legacy=0x7fedcc0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.383+0000 7fedcaffd640 1 -- 172.21.15.70:0/247635400 >> v1:172.21.15.70:6789/0 conn(0x7fedcc160660 legacy=0x7fedcc0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.383+0000 7fedcaffd640 1 -- 172.21.15.70:0/247635400 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0c8e30 con 0x7fedcc070cc0 2023-10-08T05:49:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.384+0000 7fedcaffd640 1 --2- 172.21.15.70:0/247635400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc070cc0 0x7fedcc0caf10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fedcc06e340 tx=0x7fedb402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8c29a64dd99ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.385+0000 7fedca7fc640 1 -- 172.21.15.70:0/247635400 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4030840 con 0x7fedcc070cc0 2023-10-08T05:49:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.385+0000 7fedca7fc640 1 -- 172.21.15.70:0/247635400 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb40309a0 con 0x7fedcc070cc0 2023-10-08T05:49:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.385+0000 7fedca7fc640 1 -- 172.21.15.70:0/247635400 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4030d30 con 0x7fedcc070cc0 2023-10-08T05:49:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.385+0000 7fedd1df0640 1 -- 172.21.15.70:0/247635400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc070cc0 msgr2=0x7fedcc0caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.385+0000 7fedd1df0640 1 --2- 172.21.15.70:0/247635400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc070cc0 0x7fedcc0caf10 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fedcc06e340 tx=0x7fedb402b980 comp rx=0 tx=0).stop 2023-10-08T05:49:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.385+0000 7fedd1df0640 1 -- 172.21.15.70:0/247635400 shutdown_connections 2023-10-08T05:49:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.385+0000 7fedd1df0640 1 --2- 172.21.15.70:0/247635400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc070cc0 0x7fedcc0caf10 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.385+0000 7fedd1df0640 1 -- 172.21.15.70:0/247635400 >> 172.21.15.70:0/247635400 conn(0x7fedcc06a5d0 msgr2=0x7fedcc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.385+0000 7fedd1df0640 1 -- 172.21.15.70:0/247635400 shutdown_connections 2023-10-08T05:49:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.386+0000 7fedd1df0640 1 -- 172.21.15.70:0/247635400 wait complete. 2023-10-08T05:49:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.387+0000 7fedd1df0640 1 Processor -- start 2023-10-08T05:49:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.387+0000 7fedd1df0640 1 -- start start 2023-10-08T05:49:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.388+0000 7fedd1df0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc070cc0 0x7fedcc15df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.388+0000 7fedd1df0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0cb450 0x7fedcc0c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.388+0000 7fedd1df0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc160660 0x7fedcc0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.388+0000 7fedd1df0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1f3440 con 0x7fedcc160660 2023-10-08T05:49:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.388+0000 7fedd1df0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc15e5e0 con 0x7fedcc0cb450 2023-10-08T05:49:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.388+0000 7fedd1df0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1f2420 con 0x7fedcc070cc0 2023-10-08T05:49:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc160660 0x7fedcc0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc160660 0x7fedcc0c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44100/0 (socket says 172.21.15.70:44100) 2023-10-08T05:49:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcbfff640 1 -- 172.21.15.70:0/914547402 learned_addr learned my addr 172.21.15.70:0/914547402 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcb7fe640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc070cc0 0x7fedcc15df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcaffd640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0cb450 0x7fedcc0c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcbfff640 1 -- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc070cc0 msgr2=0x7fedcc15df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcbfff640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc070cc0 0x7fedcc15df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcbfff640 1 -- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0cb450 msgr2=0x7fedcc0c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcbfff640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0cb450 0x7fedcc0c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcbfff640 1 -- 172.21.15.70:0/914547402 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb40094d0 con 0x7fedcc160660 2023-10-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.389+0000 7fedcbfff640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc160660 0x7fedcc0c79e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fedc400c440 tx=0x7fedc400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.390+0000 7fedc3fff640 1 -- 172.21.15.70:0/914547402 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4011790 con 0x7fedcc160660 2023-10-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.390+0000 7fedc3fff640 1 -- 172.21.15.70:0/914547402 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc40118f0 con 0x7fedcc160660 2023-10-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.390+0000 7fedc3fff640 1 -- 172.21.15.70:0/914547402 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4011ca0 con 0x7fedcc160660 2023-10-08T05:49:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.390+0000 7fedd1df0640 1 -- 172.21.15.70:0/914547402 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc1f2680 con 0x7fedcc160660 2023-10-08T05:49:32.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.390+0000 7fedd1df0640 1 -- 172.21.15.70:0/914547402 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc1f2c40 con 0x7fedcc160660 2023-10-08T05:49:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.391+0000 7fedd1df0640 1 -- 172.21.15.70:0/914547402 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc04e160 con 0x7fedcc160660 2023-10-08T05:49:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.392+0000 7fedc3fff640 1 -- 172.21.15.70:0/914547402 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fedc401c460 con 0x7fedcc160660 2023-10-08T05:49:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.393+0000 7fedc3fff640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fedac06b070 0x7fedac06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.393+0000 7fedc3fff640 1 -- 172.21.15.70:0/914547402 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fedc4089b20 con 0x7fedcc160660 2023-10-08T05:49:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.393+0000 7fedcb7fe640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fedac06b070 0x7fedac06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.395+0000 7fedcb7fe640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fedac06b070 0x7fedac06d530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0cdc10 tx=0x7fedbc009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:32.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.397+0000 7fedc3fff640 1 -- 172.21.15.70:0/914547402 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fedc40546e0 con 0x7fedcc160660 2023-10-08T05:49:32.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.608+0000 7fedd1df0640 1 -- 172.21.15.70:0/914547402 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fedcc0c93f0 con 0x7fedac06b070 2023-10-08T05:49:32.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.627+0000 7fedc3fff640 1 -- 172.21.15.70:0/914547402 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fedbc001950 con 0x7fedac06b070 2023-10-08T05:49:32.628 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:32.628 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:10", "first": "2023-10-08T05:39:01", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:32.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 -- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fedac06b070 msgr2=0x7fedac06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fedac06b070 0x7fedac06d530 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0cdc10 tx=0x7fedbc009040 comp rx=0 tx=0).stop 2023-10-08T05:49:32.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 -- 172.21.15.70:0/914547402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc160660 msgr2=0x7fedcc0c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc160660 0x7fedcc0c79e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fedc400c440 tx=0x7fedc400c900 comp rx=0 tx=0).stop 2023-10-08T05:49:32.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 -- 172.21.15.70:0/914547402 shutdown_connections 2023-10-08T05:49:32.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedcc160660 0x7fedcc0c79e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fedcc0cb450 0x7fedcc0c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fedac06b070 0x7fedac06d530 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 --2- 172.21.15.70:0/914547402 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fedcc070cc0 0x7fedcc15df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 -- 172.21.15.70:0/914547402 >> 172.21.15.70:0/914547402 conn(0x7fedcc06a5d0 msgr2=0x7fedcc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:32.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 -- 172.21.15.70:0/914547402 shutdown_connections 2023-10-08T05:49:32.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.630+0000 7fedd1df0640 1 -- 172.21.15.70:0/914547402 wait complete. 2023-10-08T05:49:32.650 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:10", "first": "2023-10-08T05:39:01", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:32.650 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:38:10', 'first': '2023-10-08T05:39:01', 'last': '2023-10-08T05:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T05:49:32.651 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T05:49:32.651 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T05:49:32.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:32.651 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-08T05:49:32.702 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:32.702 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_01_UTC 2023-10-08T05:49:32.702 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_01_UTC 2023-10-08T05:49:32.702 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_01_UTC 2023-10-08T05:49:32.702 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:32.703 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:32.703 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:32.703 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:32.703 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:32.703 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:32.703 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:32.704 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_39_00_UTC', 'scheduled-2023-10-08-05_40_01_UTC', 'scheduled-2023-10-08-05_41_01_UTC', 'scheduled-2023-10-08-05_42_01_UTC', 'scheduled-2023-10-08-05_43_00_UTC', 'scheduled-2023-10-08-05_44_00_UTC', 'scheduled-2023-10-08-05_45_00_UTC', 'scheduled-2023-10-08-05_46_00_UTC', 'scheduled-2023-10-08-05_47_00_UTC', 'scheduled-2023-10-08-05_48_00_UTC', 'scheduled-2023-10-08-05_49_00_UTC'] 2023-10-08T05:49:32.704 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir9 2023-10-08T05:49:32.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.918+0000 7fa0e8b8a640 1 Processor -- start 2023-10-08T05:49:32.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.918+0000 7fa0e8b8a640 1 -- start start 2023-10-08T05:49:32.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.919+0000 7fa0e8b8a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0e415ce10 0x7fa0e40ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.919+0000 7fa0e8b8a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa0e40cf3e0 0x7fa0e406b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.919+0000 7fa0e8b8a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa0e406bab0 0x7fa0e40b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0e8b8a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e40cf840 con 0x7fa0e40cf3e0 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0e8b8a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e406bf40 con 0x7fa0e406bab0 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0e8b8a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e406cd90 con 0x7fa0e415ce10 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0d9d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa0e40cf3e0 0x7fa0e406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0d9d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa0e40cf3e0 0x7fa0e406b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44104/0 (socket says 172.21.15.70:44104) 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0d9d9b640 1 -- 172.21.15.70:0/756864593 learned_addr learned my addr 172.21.15.70:0/756864593 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0d9d9b640 1 -- 172.21.15.70:0/756864593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0e415ce10 msgr2=0x7fa0e40ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0d9d9b640 1 --2- 172.21.15.70:0/756864593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0e415ce10 0x7fa0e40ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0d9d9b640 1 -- 172.21.15.70:0/756864593 >> v2:172.21.15.181:3300/0 conn(0x7fa0e406bab0 msgr2=0x7fa0e40b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:32.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0d9d9b640 1 --2- 172.21.15.70:0/756864593 >> v2:172.21.15.181:3300/0 conn(0x7fa0e406bab0 0x7fa0e40b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.920+0000 7fa0d9d9b640 1 -- 172.21.15.70:0/756864593 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e4160ca0 con 0x7fa0e40cf3e0 2023-10-08T05:49:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.921+0000 7fa0d9d9b640 1 --2- 172.21.15.70:0/756864593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e40cf3e0 0x7fa0e406b570 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa0e406f400 tx=0x7fa0cc02b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca623370c0b8d01a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e1d9b640 1 -- 172.21.15.70:0/756864593 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0317f0 con 0x7fa0e40cf3e0 2023-10-08T05:49:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e1d9b640 1 -- 172.21.15.70:0/756864593 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc031950 con 0x7fa0e40cf3e0 2023-10-08T05:49:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e1d9b640 1 -- 172.21.15.70:0/756864593 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc031ce0 con 0x7fa0e40cf3e0 2023-10-08T05:49:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/756864593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e40cf3e0 msgr2=0x7fa0e406b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e8b8a640 1 --2- 172.21.15.70:0/756864593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e40cf3e0 0x7fa0e406b570 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa0e406f400 tx=0x7fa0cc02b890 comp rx=0 tx=0).stop 2023-10-08T05:49:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/756864593 shutdown_connections 2023-10-08T05:49:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e8b8a640 1 --2- 172.21.15.70:0/756864593 >> v2:172.21.15.181:3300/0 conn(0x7fa0e406bab0 0x7fa0e40b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e8b8a640 1 --2- 172.21.15.70:0/756864593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e40cf3e0 0x7fa0e406b570 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e8b8a640 1 --2- 172.21.15.70:0/756864593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0e415ce10 0x7fa0e40ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/756864593 >> 172.21.15.70:0/756864593 conn(0x7fa0e406e630 msgr2=0x7fa0e406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:32.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/756864593 shutdown_connections 2023-10-08T05:49:32.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.922+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/756864593 wait complete. 2023-10-08T05:49:32.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.924+0000 7fa0e8b8a640 1 Processor -- start 2023-10-08T05:49:32.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.924+0000 7fa0e8b8a640 1 -- start start 2023-10-08T05:49:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.925+0000 7fa0e8b8a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e406bab0 0x7fa0e40b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.925+0000 7fa0e8b8a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0e40cf3e0 0x7fa0e4159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.925+0000 7fa0e8b8a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0e415ce10 0x7fa0e40ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.925+0000 7fa0e8b8a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e40b5af0 con 0x7fa0e406bab0 2023-10-08T05:49:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.925+0000 7fa0e8b8a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e40b4510 con 0x7fa0e415ce10 2023-10-08T05:49:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.925+0000 7fa0e8b8a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e40b47f0 con 0x7fa0e40cf3e0 2023-10-08T05:49:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.925+0000 7fa0e259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e406bab0 0x7fa0e40b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.926+0000 7fa0e259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e406bab0 0x7fa0e40b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44112/0 (socket says 172.21.15.70:44112) 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.926+0000 7fa0e259c640 1 -- 172.21.15.70:0/2304901032 learned_addr learned my addr 172.21.15.70:0/2304901032 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.926+0000 7fa0d9d9b640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0e40cf3e0 0x7fa0e4159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.926+0000 7fa0e2d9d640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0e415ce10 0x7fa0e40ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.926+0000 7fa0e259c640 1 -- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0e40cf3e0 msgr2=0x7fa0e4159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.926+0000 7fa0e259c640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0e40cf3e0 0x7fa0e4159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.926+0000 7fa0e259c640 1 -- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0e415ce10 msgr2=0x7fa0e40ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.926+0000 7fa0e259c640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0e415ce10 0x7fa0e40ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.926+0000 7fa0e259c640 1 -- 172.21.15.70:0/2304901032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0094d0 con 0x7fa0e406bab0 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.926+0000 7fa0e259c640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e406bab0 0x7fa0e40b55b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400ba50 tx=0x7fa0d400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.927+0000 7fa0db7fe640 1 -- 172.21.15.70:0/2304901032 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400cbe0 con 0x7fa0e406bab0 2023-10-08T05:49:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.927+0000 7fa0db7fe640 1 -- 172.21.15.70:0/2304901032 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d400cd60 con 0x7fa0e406bab0 2023-10-08T05:49:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.927+0000 7fa0db7fe640 1 -- 172.21.15.70:0/2304901032 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4014680 con 0x7fa0e406bab0 2023-10-08T05:49:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.927+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/2304901032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e40b4ad0 con 0x7fa0e406bab0 2023-10-08T05:49:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.927+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/2304901032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e40b5090 con 0x7fa0e406bab0 2023-10-08T05:49:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.928+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/2304901032 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e40b52b0 con 0x7fa0e406bab0 2023-10-08T05:49:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.929+0000 7fa0db7fe640 1 -- 172.21.15.70:0/2304901032 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fa0d401d3b0 con 0x7fa0e406bab0 2023-10-08T05:49:32.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.930+0000 7fa0db7fe640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa0c406b070 0x7fa0c406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:32.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.930+0000 7fa0db7fe640 1 -- 172.21.15.70:0/2304901032 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fa0d4089bc0 con 0x7fa0e406bab0 2023-10-08T05:49:32.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.930+0000 7fa0d9d9b640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa0c406b070 0x7fa0c406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.932+0000 7fa0d9d9b640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa0c406b070 0x7fa0c406d530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc036000 tx=0x7fa0cc03d040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:32.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:32.934+0000 7fa0db7fe640 1 -- 172.21.15.70:0/2304901032 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0d4054780 con 0x7fa0e406bab0 2023-10-08T05:49:33.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.143+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/2304901032 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0e406c460 con 0x7fa0c406b070 2023-10-08T05:49:33.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.162+0000 7fa0db7fe640 1 -- 172.21.15.70:0/2304901032 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fa0cc031bc0 con 0x7fa0c406b070 2023-10-08T05:49:33.163 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:33.163 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:11", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.164+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa0c406b070 msgr2=0x7fa0c406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.164+0000 7fa0e8b8a640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa0c406b070 0x7fa0c406d530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc036000 tx=0x7fa0cc03d040 comp rx=0 tx=0).stop 2023-10-08T05:49:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.165+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/2304901032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e406bab0 msgr2=0x7fa0e40b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.165+0000 7fa0e8b8a640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e406bab0 0x7fa0e40b55b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400ba50 tx=0x7fa0d400bf10 comp rx=0 tx=0).stop 2023-10-08T05:49:33.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.165+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/2304901032 shutdown_connections 2023-10-08T05:49:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.165+0000 7fa0e8b8a640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0e415ce10 0x7fa0e40ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.165+0000 7fa0e8b8a640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fa0c406b070 0x7fa0c406d530 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.165+0000 7fa0e8b8a640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0e40cf3e0 0x7fa0e4159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.165+0000 7fa0e8b8a640 1 --2- 172.21.15.70:0/2304901032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0e406bab0 0x7fa0e40b55b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.165+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/2304901032 >> 172.21.15.70:0/2304901032 conn(0x7fa0e406e630 msgr2=0x7fa0e40c6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.165+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/2304901032 shutdown_connections 2023-10-08T05:49:33.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.165+0000 7fa0e8b8a640 1 -- 172.21.15.70:0/2304901032 wait complete. 2023-10-08T05:49:33.184 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:38:11", "first": "2023-10-08T05:39:00", "last": "2023-10-08T05:49:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T05:49:33.184 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:38:11', 'first': '2023-10-08T05:39:00', 'last': '2023-10-08T05:49:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T05:49:33.184 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T05:49:33.184 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T05:49:33.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir0 1m 2023-10-08T05:49:33.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.406+0000 7f1828ed7640 1 Processor -- start 2023-10-08T05:49:33.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.406+0000 7f1828ed7640 1 -- start start 2023-10-08T05:49:33.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.408+0000 7f1828ed7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f182406dfc0 0x7f1824070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:33.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.408+0000 7f1828ed7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1824071070 0x7f18240cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:33.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.408+0000 7f1828ed7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18240b1cc0 con 0x7f18240afbe0 2023-10-08T05:49:33.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.408+0000 7f1828ed7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240b2620 con 0x7f1824071070 2023-10-08T05:49:33.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.408+0000 7f1828ed7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18240b2060 con 0x7f182406dfc0 2023-10-08T05:49:33.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.408+0000 7f182259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f18240afbe0 0x7f18240b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44448/0 (socket says 172.21.15.70:44448) 2023-10-08T05:49:33.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.408+0000 7f182259c640 1 -- 172.21.15.70:0/1950189275 learned_addr learned my addr 172.21.15.70:0/1950189275 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:33.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.408+0000 7f1822d9d640 1 --2- 172.21.15.70:0/1950189275 >> v2:172.21.15.70:3300/0 conn(0x7f1824071070 0x7f18240cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:33.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.408+0000 7f1821d9b640 1 --2- 172.21.15.70:0/1950189275 >> v2:172.21.15.181:3300/0 conn(0x7f182406dfc0 0x7f1824070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:33.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.409+0000 7f182159a640 1 -- 172.21.15.70:0/1950189275 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672970367 0 0) 0x7f18240b1cc0 con 0x7f18240afbe0 2023-10-08T05:49:33.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.409+0000 7f182159a640 1 -- 172.21.15.70:0/1950189275 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f180c003520 con 0x7f18240afbe0 2023-10-08T05:49:33.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.409+0000 7f1822d9d640 1 -- 172.21.15.70:0/1950189275 >> v2:172.21.15.181:3300/0 conn(0x7f182406dfc0 msgr2=0x7f1824070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:33.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.409+0000 7f1822d9d640 1 --2- 172.21.15.70:0/1950189275 >> v2:172.21.15.181:3300/0 conn(0x7f182406dfc0 0x7f1824070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.409+0000 7f1822d9d640 1 -- 172.21.15.70:0/1950189275 >> v1:172.21.15.70:6789/0 conn(0x7f18240afbe0 legacy=0x7f18240b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:33.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.409+0000 7f1822d9d640 1 -- 172.21.15.70:0/1950189275 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240b2340 con 0x7f1824071070 2023-10-08T05:49:33.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.410+0000 7f1822d9d640 1 --2- 172.21.15.70:0/1950189275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1824071070 0x7f18240cb130 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f18240b3980 tx=0x7f181c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46a3d21c4017c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.410+0000 7f182159a640 1 -- 172.21.15.70:0/1950189275 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c02cc70 con 0x7f1824071070 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.410+0000 7f182159a640 1 -- 172.21.15.70:0/1950189275 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c02cdd0 con 0x7f1824071070 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.410+0000 7f182159a640 1 -- 172.21.15.70:0/1950189275 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c034660 con 0x7f1824071070 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.411+0000 7f1828ed7640 1 -- 172.21.15.70:0/1950189275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1824071070 msgr2=0x7f18240cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.411+0000 7f1828ed7640 1 --2- 172.21.15.70:0/1950189275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1824071070 0x7f18240cb130 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f18240b3980 tx=0x7f181c02bec0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.411+0000 7f1828ed7640 1 -- 172.21.15.70:0/1950189275 shutdown_connections 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.411+0000 7f1828ed7640 1 --2- 172.21.15.70:0/1950189275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1824071070 0x7f18240cb130 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.411+0000 7f1828ed7640 1 --2- 172.21.15.70:0/1950189275 >> v2:172.21.15.181:3300/0 conn(0x7f182406dfc0 0x7f1824070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.411+0000 7f1828ed7640 1 -- 172.21.15.70:0/1950189275 >> 172.21.15.70:0/1950189275 conn(0x7f182406a5d0 msgr2=0x7f18240b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.411+0000 7f1828ed7640 1 -- 172.21.15.70:0/1950189275 shutdown_connections 2023-10-08T05:49:33.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.411+0000 7f1828ed7640 1 -- 172.21.15.70:0/1950189275 wait complete. 2023-10-08T05:49:33.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.413+0000 7f1828ed7640 1 Processor -- start 2023-10-08T05:49:33.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.413+0000 7f1828ed7640 1 -- start start 2023-10-08T05:49:33.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.414+0000 7f1828ed7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f182406dfc0 0x7f18241f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:33.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.414+0000 7f1828ed7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1824071070 0x7f18241593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:33.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.414+0000 7f1828ed7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18240afbe0 0x7f18241f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:33.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.414+0000 7f1828ed7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18241f48c0 con 0x7f18240afbe0 2023-10-08T05:49:33.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.414+0000 7f1828ed7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182415c6c0 con 0x7f182406dfc0 2023-10-08T05:49:33.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.414+0000 7f1828ed7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182415c9a0 con 0x7f1824071070 2023-10-08T05:49:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.414+0000 7f1822d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18240afbe0 0x7f18241f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.414+0000 7f1822d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18240afbe0 0x7f18241f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44136/0 (socket says 172.21.15.70:44136) 2023-10-08T05:49:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.414+0000 7f1822d9d640 1 -- 172.21.15.70:0/2458694391 learned_addr learned my addr 172.21.15.70:0/2458694391 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.415+0000 7f182259c640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f182406dfc0 0x7f18241f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.415+0000 7f1822d9d640 1 -- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1824071070 msgr2=0x7f18241593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.415+0000 7f1822d9d640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1824071070 0x7f18241593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.415+0000 7f1822d9d640 1 -- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f182406dfc0 msgr2=0x7f18241f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.415+0000 7f1822d9d640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f182406dfc0 0x7f18241f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.415+0000 7f1822d9d640 1 -- 172.21.15.70:0/2458694391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c009a70 con 0x7f18240afbe0 2023-10-08T05:49:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.415+0000 7f1822d9d640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18240afbe0 0x7f18241f1440 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f181c009e80 tx=0x7f181c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.416+0000 7f1812ffd640 1 -- 172.21.15.70:0/2458694391 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c040650 con 0x7f18240afbe0 2023-10-08T05:49:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.416+0000 7f1812ffd640 1 -- 172.21.15.70:0/2458694391 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c0407b0 con 0x7f18240afbe0 2023-10-08T05:49:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.416+0000 7f1812ffd640 1 -- 172.21.15.70:0/2458694391 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c040b60 con 0x7f18240afbe0 2023-10-08T05:49:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.416+0000 7f1828ed7640 1 -- 172.21.15.70:0/2458694391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182415cc80 con 0x7f18240afbe0 2023-10-08T05:49:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.416+0000 7f1828ed7640 1 -- 172.21.15.70:0/2458694391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182415d240 con 0x7f18240afbe0 2023-10-08T05:49:33.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.417+0000 7f1828ed7640 1 -- 172.21.15.70:0/2458694391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182404e160 con 0x7f18240afbe0 2023-10-08T05:49:33.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.418+0000 7f1812ffd640 1 -- 172.21.15.70:0/2458694391 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f181c040cc0 con 0x7f18240afbe0 2023-10-08T05:49:33.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.419+0000 7f1812ffd640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f180406b070 0x7f180406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:33.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.419+0000 7f1812ffd640 1 -- 172.21.15.70:0/2458694391 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f181c0ac9a0 con 0x7f18240afbe0 2023-10-08T05:49:33.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.420+0000 7f182259c640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f180406b070 0x7f180406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:33.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.421+0000 7f182259c640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f180406b070 0x7f180406d530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f18140011e0 tx=0x7f1814009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:33.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.423+0000 7f1812ffd640 1 -- 172.21.15.70:0/2458694391 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f181c0774d0 con 0x7f18240afbe0 2023-10-08T05:49:33.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.633+0000 7f1828ed7640 1 -- 172.21.15.70:0/2458694391 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir0", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f1824070630 con 0x7f180406b070 2023-10-08T05:49:33.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.830+0000 7f1812ffd640 1 -- 172.21.15.70:0/2458694391 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f1814001350 con 0x7f180406b070 2023-10-08T05:49:33.831 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-10-08T05:49:33.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.833+0000 7f1828ed7640 1 -- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f180406b070 msgr2=0x7f180406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:33.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.833+0000 7f1828ed7640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f180406b070 0x7f180406d530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f18140011e0 tx=0x7f1814009040 comp rx=0 tx=0).stop 2023-10-08T05:49:33.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.833+0000 7f1828ed7640 1 -- 172.21.15.70:0/2458694391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18240afbe0 msgr2=0x7f18241f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:33.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.833+0000 7f1828ed7640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18240afbe0 0x7f18241f1440 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f181c009e80 tx=0x7f181c004a40 comp rx=0 tx=0).stop 2023-10-08T05:49:33.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.834+0000 7f1828ed7640 1 -- 172.21.15.70:0/2458694391 shutdown_connections 2023-10-08T05:49:33.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.834+0000 7f1828ed7640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18240afbe0 0x7f18241f1440 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.834+0000 7f1828ed7640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f180406b070 0x7f180406d530 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.834+0000 7f1828ed7640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1824071070 0x7f18241593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.834+0000 7f1828ed7640 1 --2- 172.21.15.70:0/2458694391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f182406dfc0 0x7f18241f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:33.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.834+0000 7f1828ed7640 1 -- 172.21.15.70:0/2458694391 >> 172.21.15.70:0/2458694391 conn(0x7f182406a5d0 msgr2=0x7f18240c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:33.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.834+0000 7f1828ed7640 1 -- 172.21.15.70:0/2458694391 shutdown_connections 2023-10-08T05:49:33.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:33.834+0000 7f1828ed7640 1 -- 172.21.15.70:0/2458694391 wait complete. 2023-10-08T05:49:33.853 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-10-08T05:49:33.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:33.853 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-08T05:49:33.908 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:33.908 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:33.908 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:33.908 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_01_UTC 2023-10-08T05:49:33.909 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:33.909 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_01_UTC 2023-10-08T05:49:33.909 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:33.909 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:33.909 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:33.909 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:33.909 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:33.910 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:33.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:33.910 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_39_00_UTC) 2023-10-08T05:49:33.976 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:33.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:33.977 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_40_00_UTC) 2023-10-08T05:49:34.040 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:34.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:34.040 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_41_00_UTC) 2023-10-08T05:49:34.103 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_42_01_UTC 2023-10-08T05:49:34.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:34.103 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_42_01_UTC) 2023-10-08T05:49:34.176 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:34.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:34.176 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_43_01_UTC) 2023-10-08T05:49:34.247 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_44_01_UTC 2023-10-08T05:49:34.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:34.247 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_44_01_UTC) 2023-10-08T05:49:34.310 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:34.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:34.310 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_45_01_UTC) 2023-10-08T05:49:34.376 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:34.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:34.376 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_46_01_UTC) 2023-10-08T05:49:34.442 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:34.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:34.443 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_47_00_UTC) 2023-10-08T05:49:34.511 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:34.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:34.512 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_48_00_UTC) 2023-10-08T05:49:34.578 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:34.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:34.578 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-05_49_01_UTC) 2023-10-08T05:49:34.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:34.645 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-10-08T05:49:34.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir1 1m 2023-10-08T05:49:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.920+0000 7f02eb6bb640 1 Processor -- start 2023-10-08T05:49:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.920+0000 7f02eb6bb640 1 -- start start 2023-10-08T05:49:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.922+0000 7f02eb6bb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f02e40cb620 0x7f02e40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.922+0000 7f02eb6bb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e4071170 con 0x7f02e4070cc0 2023-10-08T05:49:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.922+0000 7f02eb6bb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e406e150 con 0x7f02e4160660 2023-10-08T05:49:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.922+0000 7f02eb6bb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e406de70 con 0x7f02e40cb620 2023-10-08T05:49:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.922+0000 7f02e8c56640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f02e4070cc0 0x7f02e40caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39974/0 (socket says 172.21.15.70:39974) 2023-10-08T05:49:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.922+0000 7f02e8c56640 1 -- 172.21.15.70:0/2680697879 learned_addr learned my addr 172.21.15.70:0/2680697879 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:34.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.922+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830940787 0 0) 0x7f02d4001110 con 0x7f02e4070cc0 2023-10-08T05:49:34.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d0003520 con 0x7f02e4070cc0 2023-10-08T05:49:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02e9c58640 1 --2- 172.21.15.70:0/2680697879 >> v2:172.21.15.181:3300/0 conn(0x7f02e40cb620 0x7f02e40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4268996260 0 0) 0x7f02d0003520 con 0x7f02e4070cc0 2023-10-08T05:49:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 >> v2:172.21.15.181:3300/0 conn(0x7f02e40cb620 msgr2=0x7f02e40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 --2- 172.21.15.70:0/2680697879 >> v2:172.21.15.181:3300/0 conn(0x7f02e40cb620 0x7f02e40b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 >> v1:172.21.15.181:6789/0 conn(0x7f02e4160660 legacy=0x7f02e40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e40c8e30 con 0x7f02e4070cc0 2023-10-08T05:49:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f02d4001800 con 0x7f02e4070cc0 2023-10-08T05:49:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 >> v1:172.21.15.70:6789/0 conn(0x7f02e4070cc0 legacy=0x7f02e40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d00039c0 0x7f02d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d00047c0 0x7f02d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d00052a0 0x7f02d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e406de70 con 0x7f02d00047c0 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d000c1c0 con 0x7f02d00039c0 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d000c380 con 0x7f02d00052a0 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.923+0000 7f02e8c56640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d00047c0 0x7f02d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.924+0000 7f02e9457640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d00039c0 0x7f02d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.924+0000 7f02e8c56640 1 -- 172.21.15.70:0/2680697879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d00052a0 msgr2=0x7f02d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.924+0000 7f02e8c56640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d00052a0 0x7f02d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.924+0000 7f02e8c56640 1 -- 172.21.15.70:0/2680697879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d00039c0 msgr2=0x7f02d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.924+0000 7f02e8c56640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d00039c0 0x7f02d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.924+0000 7f02e8c56640 1 -- 172.21.15.70:0/2680697879 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d0003520 con 0x7f02d00047c0 2023-10-08T05:49:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.924+0000 7f02e9457640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d00039c0 0x7f02d0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.924+0000 7f02e9c58640 1 -- 172.21.15.70:0/2680697879 reap_dead start 2023-10-08T05:49:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.925+0000 7f02e8c56640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d00047c0 0x7f02d0004c40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f02d400c110 tx=0x7f02d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f84b2f6a4013e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.925+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d4005ad0 con 0x7f02d00047c0 2023-10-08T05:49:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.925+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d4005c30 con 0x7f02d00047c0 2023-10-08T05:49:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.925+0000 7f02dbfff640 1 -- 172.21.15.70:0/2680697879 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d402c7e0 con 0x7f02d00047c0 2023-10-08T05:49:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.926+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2680697879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d00047c0 msgr2=0x7f02d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.926+0000 7f02eb6bb640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d00047c0 0x7f02d0004c40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f02d400c110 tx=0x7f02d00057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.926+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2680697879 shutdown_connections 2023-10-08T05:49:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.926+0000 7f02eb6bb640 1 --2- 172.21.15.70:0/2680697879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d00047c0 0x7f02d0004c40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.926+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2680697879 >> 172.21.15.70:0/2680697879 conn(0x7f02e406a5d0 msgr2=0x7f02e40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.926+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2680697879 shutdown_connections 2023-10-08T05:49:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.926+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2680697879 wait complete. 2023-10-08T05:49:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.928+0000 7f02eb6bb640 1 Processor -- start 2023-10-08T05:49:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.928+0000 7f02eb6bb640 1 -- start start 2023-10-08T05:49:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02eb6bb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d00047c0 0x7f02e41f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02eb6bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4156f80 0x7f02e4157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02eb6bb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e415b6e0 0x7f02e415bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02eb6bb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4157940 con 0x7f02e4156f80 2023-10-08T05:49:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02eb6bb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4149260 con 0x7f02e415b6e0 2023-10-08T05:49:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02eb6bb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4149540 con 0x7f02d00047c0 2023-10-08T05:49:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02e8c56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4156f80 0x7f02e4157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02e8c56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4156f80 0x7f02e4157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57058/0 (socket says 172.21.15.70:57058) 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02e8c56640 1 -- 172.21.15.70:0/2652694990 learned_addr learned my addr 172.21.15.70:0/2652694990 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02e8c56640 1 -- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d00047c0 msgr2=0x7f02e41f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02e8c56640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d00047c0 0x7f02e41f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02e8c56640 1 -- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e415b6e0 msgr2=0x7f02e415bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02e8c56640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e415b6e0 0x7f02e415bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.929+0000 7f02e8c56640 1 -- 172.21.15.70:0/2652694990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e40adcc0 con 0x7f02e4156f80 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.930+0000 7f02e8c56640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4156f80 0x7f02e4157400 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f02d402c380 tx=0x7f02d402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.930+0000 7f02d9ffb640 1 -- 172.21.15.70:0/2652694990 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d403c040 con 0x7f02e4156f80 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.930+0000 7f02d9ffb640 1 -- 172.21.15.70:0/2652694990 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d4001130 con 0x7f02e4156f80 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.930+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2652694990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e40adfa0 con 0x7f02e4156f80 2023-10-08T05:49:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.930+0000 7f02d9ffb640 1 -- 172.21.15.70:0/2652694990 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d402ce50 con 0x7f02e4156f80 2023-10-08T05:49:34.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.930+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2652694990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e414c450 con 0x7f02e4156f80 2023-10-08T05:49:34.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.931+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2652694990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e404e160 con 0x7f02e4156f80 2023-10-08T05:49:34.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.932+0000 7f02d9ffb640 1 -- 172.21.15.70:0/2652694990 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f02d4032070 con 0x7f02e4156f80 2023-10-08T05:49:34.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.933+0000 7f02d9ffb640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f02c806b0a0 0x7f02c806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:34.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.933+0000 7f02d9ffb640 1 -- 172.21.15.70:0/2652694990 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f02d40aaf60 con 0x7f02e4156f80 2023-10-08T05:49:34.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.934+0000 7f02e9457640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f02c806b0a0 0x7f02c806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:34.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.935+0000 7f02e9457640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f02c806b0a0 0x7f02c806d560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02e40684b0 tx=0x7f02e0009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:34.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:34.937+0000 7f02d9ffb640 1 -- 172.21.15.70:0/2652694990 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02d4075bb0 con 0x7f02e4156f80 2023-10-08T05:49:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.145+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2652694990 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir1", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f02e40adb50 con 0x7f02c806b0a0 2023-10-08T05:49:35.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.338+0000 7f02d9ffb640 1 -- 172.21.15.70:0/2652694990 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f02e40adb50 con 0x7f02c806b0a0 2023-10-08T05:49:35.340 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-10-08T05:49:35.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f02c806b0a0 msgr2=0x7f02c806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:35.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f02c806b0a0 0x7f02c806d560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f02e40684b0 tx=0x7f02e0009040 comp rx=0 tx=0).stop 2023-10-08T05:49:35.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2652694990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4156f80 msgr2=0x7f02e4157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:35.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4156f80 0x7f02e4157400 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f02d402c380 tx=0x7f02d402e070 comp rx=0 tx=0).stop 2023-10-08T05:49:35.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2652694990 shutdown_connections 2023-10-08T05:49:35.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e415b6e0 0x7f02e415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:35.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4156f80 0x7f02e4157400 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:35.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f02c806b0a0 0x7f02c806d560 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:35.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 --2- 172.21.15.70:0/2652694990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d00047c0 0x7f02e41f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:35.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2652694990 >> 172.21.15.70:0/2652694990 conn(0x7f02e406a5d0 msgr2=0x7f02e40ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:35.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2652694990 shutdown_connections 2023-10-08T05:49:35.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:35.342+0000 7f02eb6bb640 1 -- 172.21.15.70:0/2652694990 wait complete. 2023-10-08T05:49:35.365 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-10-08T05:49:35.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:35.365 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-08T05:49:35.414 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:35.415 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:35.415 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_01_UTC 2023-10-08T05:49:35.415 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_01_UTC 2023-10-08T05:49:35.415 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:35.415 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:35.415 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:35.415 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:35.415 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:35.415 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:35.416 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:35.416 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:35.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:35.417 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_39_00_UTC) 2023-10-08T05:49:35.482 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:35.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:35.482 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_40_00_UTC) 2023-10-08T05:49:35.551 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_41_01_UTC 2023-10-08T05:49:35.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:35.552 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_41_01_UTC) 2023-10-08T05:49:35.619 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_42_01_UTC 2023-10-08T05:49:35.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:35.620 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_42_01_UTC) 2023-10-08T05:49:35.684 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:35.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:35.685 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_43_01_UTC) 2023-10-08T05:49:35.745 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:35.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:35.745 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_44_00_UTC) 2023-10-08T05:49:35.810 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:35.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:35.810 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_45_00_UTC) 2023-10-08T05:49:35.874 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:35.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:35.875 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_46_00_UTC) 2023-10-08T05:49:35.937 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:35.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:35.937 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_47_00_UTC) 2023-10-08T05:49:35.999 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:35.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:36.000 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_48_00_UTC) 2023-10-08T05:49:36.066 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:36.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:36.067 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-05_49_00_UTC) 2023-10-08T05:49:36.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:36.136 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-10-08T05:49:36.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir2 1m 2023-10-08T05:49:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.451+0000 7f3887fff640 1 Processor -- start 2023-10-08T05:49:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.451+0000 7f3887fff640 1 -- start start 2023-10-08T05:49:36.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.452+0000 7f3887fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f388815a820 0x7f38880aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:36.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.452+0000 7f3887fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388815ac00 0x7f38880b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:36.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.452+0000 7f3887fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38880cab60 con 0x7f38880ca680 2023-10-08T05:49:36.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.452+0000 7f3887fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38880cb960 con 0x7f388815a820 2023-10-08T05:49:36.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.452+0000 7f3887fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38880cbdf0 con 0x7f388815ac00 2023-10-08T05:49:36.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.453+0000 7f3886ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f388815a820 0x7f38880aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:36.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.453+0000 7f38867fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f38880ca680 0x7f38880c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39984/0 (socket says 172.21.15.70:39984) 2023-10-08T05:49:36.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.453+0000 7f38867fc640 1 -- 172.21.15.70:0/3349019808 learned_addr learned my addr 172.21.15.70:0/3349019808 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:36.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.453+0000 7f38877fe640 1 --2- 172.21.15.70:0/3349019808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388815ac00 0x7f38880b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:36.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.453+0000 7f3886ffd640 1 -- 172.21.15.70:0/3349019808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388815ac00 msgr2=0x7f38880b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:36.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.453+0000 7f3886ffd640 1 --2- 172.21.15.70:0/3349019808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388815ac00 0x7f38880b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:36.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.453+0000 7f3886ffd640 1 -- 172.21.15.70:0/3349019808 >> v1:172.21.15.70:6789/0 conn(0x7f38880ca680 legacy=0x7f38880c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:36.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.453+0000 7f3886ffd640 1 -- 172.21.15.70:0/3349019808 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38880cb030 con 0x7f388815a820 2023-10-08T05:49:36.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.453+0000 7f3885ffb640 1 -- 172.21.15.70:0/3349019808 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329374708 0 0) 0x7f38880cab60 con 0x7f38880ca680 2023-10-08T05:49:36.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.454+0000 7f3886ffd640 1 --2- 172.21.15.70:0/3349019808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388815a820 0x7f38880aff70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f38780097a0 tx=0x7f387802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed445da311b13fa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3885ffb640 1 -- 172.21.15.70:0/3349019808 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878031780 con 0x7f388815a820 2023-10-08T05:49:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3885ffb640 1 -- 172.21.15.70:0/3349019808 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38780318e0 con 0x7f388815a820 2023-10-08T05:49:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3885ffb640 1 -- 172.21.15.70:0/3349019808 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878031c70 con 0x7f388815a820 2023-10-08T05:49:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3887fff640 1 -- 172.21.15.70:0/3349019808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388815a820 msgr2=0x7f38880aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3887fff640 1 --2- 172.21.15.70:0/3349019808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388815a820 0x7f38880aff70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f38780097a0 tx=0x7f387802b820 comp rx=0 tx=0).stop 2023-10-08T05:49:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3887fff640 1 -- 172.21.15.70:0/3349019808 shutdown_connections 2023-10-08T05:49:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3887fff640 1 --2- 172.21.15.70:0/3349019808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388815ac00 0x7f38880b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3887fff640 1 --2- 172.21.15.70:0/3349019808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388815a820 0x7f38880aff70 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3887fff640 1 -- 172.21.15.70:0/3349019808 >> 172.21.15.70:0/3349019808 conn(0x7f388806c050 msgr2=0x7f388806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:36.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3887fff640 1 -- 172.21.15.70:0/3349019808 shutdown_connections 2023-10-08T05:49:36.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.455+0000 7f3887fff640 1 -- 172.21.15.70:0/3349019808 wait complete. 2023-10-08T05:49:36.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.457+0000 7f3887fff640 1 Processor -- start 2023-10-08T05:49:36.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.457+0000 7f3887fff640 1 -- start start 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.458+0000 7f3887fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38880ca680 0x7f38881f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.458+0000 7f3887fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388815a820 0x7f38881f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.458+0000 7f3887fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388815ac00 0x7f38880c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.458+0000 7f3887fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38881f22f0 con 0x7f38880ca680 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.458+0000 7f3887fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38881f0e20 con 0x7f388815a820 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.458+0000 7f3887fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38880c4950 con 0x7f388815ac00 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.458+0000 7f3886ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38880ca680 0x7f38881f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.458+0000 7f3886ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38880ca680 0x7f38881f0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57088/0 (socket says 172.21.15.70:57088) 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.458+0000 7f3886ffd640 1 -- 172.21.15.70:0/4282349656 learned_addr learned my addr 172.21.15.70:0/4282349656 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f38867fc640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388815a820 0x7f38881f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:36.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f38877fe640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388815ac00 0x7f38880c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f3886ffd640 1 -- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388815ac00 msgr2=0x7f38880c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f3886ffd640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388815ac00 0x7f38880c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f3886ffd640 1 -- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388815a820 msgr2=0x7f38881f08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f3886ffd640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388815a820 0x7f38881f08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f3886ffd640 1 -- 172.21.15.70:0/4282349656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3878009400 con 0x7f38880ca680 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f3886ffd640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38880ca680 0x7f38881f0fb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3878000c00 tx=0x7f38780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f386f7fe640 1 -- 172.21.15.70:0/4282349656 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878031f10 con 0x7f38880ca680 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f386f7fe640 1 -- 172.21.15.70:0/4282349656 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3878004250 con 0x7f38880ca680 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f386f7fe640 1 -- 172.21.15.70:0/4282349656 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878032620 con 0x7f38880ca680 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.459+0000 7f3887fff640 1 -- 172.21.15.70:0/4282349656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38881f1670 con 0x7f38880ca680 2023-10-08T05:49:36.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.460+0000 7f3887fff640 1 -- 172.21.15.70:0/4282349656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38881f1c30 con 0x7f38880ca680 2023-10-08T05:49:36.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.461+0000 7f3887fff640 1 -- 172.21.15.70:0/4282349656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38881f2430 con 0x7f38880ca680 2023-10-08T05:49:36.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.462+0000 7f386f7fe640 1 -- 172.21.15.70:0/4282349656 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f3878032780 con 0x7f38880ca680 2023-10-08T05:49:36.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.462+0000 7f386f7fe640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f387406b070 0x7f387406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:36.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.463+0000 7f386f7fe640 1 -- 172.21.15.70:0/4282349656 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6852+0+0 (secure 0 0 0) 0x7f38780ac0a0 con 0x7f38880ca680 2023-10-08T05:49:36.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.463+0000 7f38867fc640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f387406b070 0x7f387406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:36.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.464+0000 7f38867fc640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f387406b070 0x7f387406d530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f38880c8810 tx=0x7f387c007c00 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:36.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.467+0000 7f386f7fe640 1 -- 172.21.15.70:0/4282349656 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38881f2430 con 0x7f38880ca680 2023-10-08T05:49:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.759+0000 7f3887fff640 1 -- 172.21.15.70:0/4282349656 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir2", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f38880cb5f0 con 0x7f387406b070 2023-10-08T05:49:36.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.923+0000 7f386f7fe640 1 -- 172.21.15.70:0/4282349656 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f387c007270 con 0x7f387406b070 2023-10-08T05:49:36.925 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-10-08T05:49:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 -- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f387406b070 msgr2=0x7f387406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f387406b070 0x7f387406d530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f38880c8810 tx=0x7f387c007c00 comp rx=0 tx=0).stop 2023-10-08T05:49:36.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 -- 172.21.15.70:0/4282349656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38880ca680 msgr2=0x7f38881f0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38880ca680 0x7f38881f0fb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f3878000c00 tx=0x7f38780046c0 comp rx=0 tx=0).stop 2023-10-08T05:49:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 -- 172.21.15.70:0/4282349656 shutdown_connections 2023-10-08T05:49:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f387406b070 0x7f387406d530 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388815ac00 0x7f38880c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388815a820 0x7f38881f08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 --2- 172.21.15.70:0/4282349656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38880ca680 0x7f38881f0fb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 -- 172.21.15.70:0/4282349656 >> 172.21.15.70:0/4282349656 conn(0x7f388806c050 msgr2=0x7f388815b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 -- 172.21.15.70:0/4282349656 shutdown_connections 2023-10-08T05:49:36.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:36.927+0000 7f3887fff640 1 -- 172.21.15.70:0/4282349656 wait complete. 2023-10-08T05:49:36.949 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-10-08T05:49:36.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:36.949 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-08T05:49:37.006 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:37.006 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_01_UTC 2023-10-08T05:49:37.006 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:37.006 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:37.006 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:37.006 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_01_UTC 2023-10-08T05:49:37.006 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:37.006 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:37.007 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:37.007 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:37.007 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:37.008 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:37.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.008 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_39_00_UTC) 2023-10-08T05:49:37.073 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_40_01_UTC 2023-10-08T05:49:37.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.073 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_40_01_UTC) 2023-10-08T05:49:37.141 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:37.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.141 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_41_00_UTC) 2023-10-08T05:49:37.213 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:37.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.213 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_42_00_UTC) 2023-10-08T05:49:37.313 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:37.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.314 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_43_01_UTC) 2023-10-08T05:49:37.363 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_44_01_UTC 2023-10-08T05:49:37.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.364 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_44_01_UTC) 2023-10-08T05:49:37.428 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:37.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.428 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_45_00_UTC) 2023-10-08T05:49:37.484 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:37.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.484 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_46_00_UTC) 2023-10-08T05:49:37.538 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:37.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.539 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_47_00_UTC) 2023-10-08T05:49:37.601 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:37.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.601 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_48_00_UTC) 2023-10-08T05:49:37.662 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:37.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.663 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-05_49_00_UTC) 2023-10-08T05:49:37.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:37.727 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-10-08T05:49:37.782 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir3 1m 2023-10-08T05:49:37.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.994+0000 7fc9d2b05640 1 Processor -- start 2023-10-08T05:49:37.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.994+0000 7fc9d2b05640 1 -- start start 2023-10-08T05:49:37.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.996+0000 7fc9d2b05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 0x7fc9cc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:37.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.996+0000 7fc9d2b05640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc9cc06db60 0x7fc9cc06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:37.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.996+0000 7fc9d2b05640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc15b310 con 0x7fc9cc15aa00 2023-10-08T05:49:37.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.996+0000 7fc9d2b05640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9cc0ca230 con 0x7fc9cc06db60 2023-10-08T05:49:37.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.996+0000 7fc9d2b05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc06d290 con 0x7fc9cc06c3a0 2023-10-08T05:49:37.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.997+0000 7fc9d08a1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 0x7fc9cc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:37.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.997+0000 7fc9d08a1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 0x7fc9cc0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:33404/0 (socket says 172.21.15.70:33404) 2023-10-08T05:49:37.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.997+0000 7fc9d08a1640 1 -- 172.21.15.70:0/607092556 learned_addr learned my addr 172.21.15.70:0/607092556 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:37.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.997+0000 7fc9cbfff640 1 --2- 172.21.15.70:0/607092556 >> v2:172.21.15.181:3300/0 conn(0x7fc9cc06db60 0x7fc9cc06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:37.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.997+0000 7fc9d08a1640 1 -- 172.21.15.70:0/607092556 >> v2:172.21.15.181:3300/0 conn(0x7fc9cc06db60 msgr2=0x7fc9cc06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:37.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.997+0000 7fc9d08a1640 1 --2- 172.21.15.70:0/607092556 >> v2:172.21.15.181:3300/0 conn(0x7fc9cc06db60 0x7fc9cc06c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:37.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.997+0000 7fc9d08a1640 1 -- 172.21.15.70:0/607092556 >> v1:172.21.15.181:6789/0 conn(0x7fc9cc15aa00 legacy=0x7fc9cc0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:37.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.997+0000 7fc9d08a1640 1 -- 172.21.15.70:0/607092556 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc0c9860 con 0x7fc9cc06c3a0 2023-10-08T05:49:38.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.999+0000 7fc9d08a1640 1 --2- 172.21.15.70:0/607092556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 0x7fc9cc0c9cf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009870 tx=0x7fc9c002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eabff0301367c0ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:38.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.999+0000 7fc9cb7fe640 1 -- 172.21.15.70:0/607092556 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0030840 con 0x7fc9cc06c3a0 2023-10-08T05:49:38.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.999+0000 7fc9cb7fe640 1 -- 172.21.15.70:0/607092556 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c00309a0 con 0x7fc9cc06c3a0 2023-10-08T05:49:38.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:37.999+0000 7fc9cb7fe640 1 -- 172.21.15.70:0/607092556 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0030d30 con 0x7fc9cc06c3a0 2023-10-08T05:49:38.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.000+0000 7fc9d2b05640 1 -- 172.21.15.70:0/607092556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 msgr2=0x7fc9cc0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:38.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.000+0000 7fc9d2b05640 1 --2- 172.21.15.70:0/607092556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 0x7fc9cc0c9cf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009870 tx=0x7fc9c002b980 comp rx=0 tx=0).stop 2023-10-08T05:49:38.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.000+0000 7fc9d2b05640 1 -- 172.21.15.70:0/607092556 shutdown_connections 2023-10-08T05:49:38.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.000+0000 7fc9d2b05640 1 --2- 172.21.15.70:0/607092556 >> v2:172.21.15.181:3300/0 conn(0x7fc9cc06db60 0x7fc9cc06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:38.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.000+0000 7fc9d2b05640 1 --2- 172.21.15.70:0/607092556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 0x7fc9cc0c9cf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:38.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.000+0000 7fc9d2b05640 1 -- 172.21.15.70:0/607092556 >> 172.21.15.70:0/607092556 conn(0x7fc9cc06a5d0 msgr2=0x7fc9cc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:38.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.000+0000 7fc9d2b05640 1 -- 172.21.15.70:0/607092556 shutdown_connections 2023-10-08T05:49:38.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.000+0000 7fc9d2b05640 1 -- 172.21.15.70:0/607092556 wait complete. 2023-10-08T05:49:38.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.002+0000 7fc9d2b05640 1 Processor -- start 2023-10-08T05:49:38.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.002+0000 7fc9d2b05640 1 -- start start 2023-10-08T05:49:38.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.003+0000 7fc9d2b05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 0x7fc9cc0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:38.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.003+0000 7fc9d2b05640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9cc06db60 0x7fc9cc1507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:38.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.003+0000 7fc9d2b05640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc15aa00 0x7fc9cc1f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:38.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.003+0000 7fc9d2b05640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0c37e0 con 0x7fc9cc15aa00 2023-10-08T05:49:38.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.003+0000 7fc9d2b05640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0c3be0 con 0x7fc9cc06db60 2023-10-08T05:49:38.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.003+0000 7fc9d2b05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc15d4c0 con 0x7fc9cc06c3a0 2023-10-08T05:49:38.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.003+0000 7fc9d10a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc15aa00 0x7fc9cc1f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:38.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.003+0000 7fc9d10a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc15aa00 0x7fc9cc1f59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57094/0 (socket says 172.21.15.70:57094) 2023-10-08T05:49:38.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.003+0000 7fc9d10a2640 1 -- 172.21.15.70:0/191875523 learned_addr learned my addr 172.21.15.70:0/191875523 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.004+0000 7fc9d08a1640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 0x7fc9cc0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.004+0000 7fc9cbfff640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9cc06db60 0x7fc9cc1507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.004+0000 7fc9d10a2640 1 -- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 msgr2=0x7fc9cc0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.004+0000 7fc9d10a2640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 0x7fc9cc0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.004+0000 7fc9d10a2640 1 -- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9cc06db60 msgr2=0x7fc9cc1507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.004+0000 7fc9d10a2640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9cc06db60 0x7fc9cc1507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.004+0000 7fc9d10a2640 1 -- 172.21.15.70:0/191875523 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c00094d0 con 0x7fc9cc15aa00 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.004+0000 7fc9d10a2640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc15aa00 0x7fc9cc1f59b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c6b0 tx=0x7fc9c400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.004+0000 7fc9c97fa640 1 -- 172.21.15.70:0/191875523 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4003b70 con 0x7fc9cc15aa00 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.005+0000 7fc9c97fa640 1 -- 172.21.15.70:0/191875523 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4003cf0 con 0x7fc9cc15aa00 2023-10-08T05:49:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.005+0000 7fc9c97fa640 1 -- 172.21.15.70:0/191875523 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4003ea0 con 0x7fc9cc15aa00 2023-10-08T05:49:38.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.005+0000 7fc9d2b05640 1 -- 172.21.15.70:0/191875523 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc0c2ea0 con 0x7fc9cc15aa00 2023-10-08T05:49:38.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.005+0000 7fc9d2b05640 1 -- 172.21.15.70:0/191875523 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc151160 con 0x7fc9cc15aa00 2023-10-08T05:49:38.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.006+0000 7fc9d2b05640 1 -- 172.21.15.70:0/191875523 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc15a240 con 0x7fc9cc15aa00 2023-10-08T05:49:38.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.007+0000 7fc9c97fa640 1 -- 172.21.15.70:0/191875523 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fc9c4005310 con 0x7fc9cc15aa00 2023-10-08T05:49:38.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.008+0000 7fc9c97fa640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fc9b006b070 0x7fc9b006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:38.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.008+0000 7fc9c97fa640 1 -- 172.21.15.70:0/191875523 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6740+0+0 (secure 0 0 0) 0x7fc9c40899d0 con 0x7fc9cc15aa00 2023-10-08T05:49:38.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.009+0000 7fc9d08a1640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fc9b006b070 0x7fc9b006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:38.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.010+0000 7fc9d08a1640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fc9b006b070 0x7fc9b006d530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0036000 tx=0x7fc9c0008030 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:38.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.013+0000 7fc9c97fa640 1 -- 172.21.15.70:0/191875523 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9c40543e0 con 0x7fc9cc15aa00 2023-10-08T05:49:38.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.227+0000 7fc9d2b05640 1 -- 172.21.15.70:0/191875523 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir3", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9cc0c8f30 con 0x7fc9b006b070 2023-10-08T05:49:38.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.407+0000 7fc9c97fa640 1 -- 172.21.15.70:0/191875523 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc9c0044470 con 0x7fc9b006b070 2023-10-08T05:49:38.408 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.411+0000 7fc9d2b05640 1 -- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fc9b006b070 msgr2=0x7fc9b006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.411+0000 7fc9d2b05640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fc9b006b070 0x7fc9b006d530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0036000 tx=0x7fc9c0008030 comp rx=0 tx=0).stop 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.411+0000 7fc9d2b05640 1 -- 172.21.15.70:0/191875523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc15aa00 msgr2=0x7fc9cc1f59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.411+0000 7fc9d2b05640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc15aa00 0x7fc9cc1f59b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c6b0 tx=0x7fc9c400cb70 comp rx=0 tx=0).stop 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.411+0000 7fc9d2b05640 1 -- 172.21.15.70:0/191875523 shutdown_connections 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.411+0000 7fc9d2b05640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc15aa00 0x7fc9cc1f59b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.411+0000 7fc9d2b05640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9cc06db60 0x7fc9cc1507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.411+0000 7fc9d2b05640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fc9b006b070 0x7fc9b006d530 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.411+0000 7fc9d2b05640 1 --2- 172.21.15.70:0/191875523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9cc06c3a0 0x7fc9cc0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.412+0000 7fc9d2b05640 1 -- 172.21.15.70:0/191875523 >> 172.21.15.70:0/191875523 conn(0x7fc9cc06a5d0 msgr2=0x7fc9cc0caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.412+0000 7fc9d2b05640 1 -- 172.21.15.70:0/191875523 shutdown_connections 2023-10-08T05:49:38.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:38.412+0000 7fc9d2b05640 1 -- 172.21.15.70:0/191875523 wait complete. 2023-10-08T05:49:38.431 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-10-08T05:49:38.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:38.432 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-08T05:49:38.481 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_01_UTC 2023-10-08T05:49:38.481 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:38.481 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:38.482 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:38.482 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:38.482 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:38.482 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:38.482 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:38.482 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_01_UTC 2023-10-08T05:49:38.482 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_01_UTC 2023-10-08T05:49:38.482 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:38.483 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_39_01_UTC 2023-10-08T05:49:38.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:38.483 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_39_01_UTC) 2023-10-08T05:49:38.553 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:38.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:38.553 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_40_00_UTC) 2023-10-08T05:49:38.615 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:38.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:38.615 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_41_00_UTC) 2023-10-08T05:49:38.676 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:38.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:38.676 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_42_00_UTC) 2023-10-08T05:49:38.731 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:38.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:38.731 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_43_00_UTC) 2023-10-08T05:49:38.791 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:38.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:38.791 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_44_00_UTC) 2023-10-08T05:49:38.793 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:49:38.849 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:49:38.857 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:38.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:38.858 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_45_01_UTC) 2023-10-08T05:49:38.966 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:38.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:38.966 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_46_01_UTC) 2023-10-08T05:49:39.079 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_47_01_UTC 2023-10-08T05:49:39.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:39.079 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_47_01_UTC) 2023-10-08T05:49:39.145 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_48_01_UTC 2023-10-08T05:49:39.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:39.146 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_48_01_UTC) 2023-10-08T05:49:39.215 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:39.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:39.216 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-05_49_01_UTC) 2023-10-08T05:49:39.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:39.324 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-10-08T05:49:39.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir4 1m 2023-10-08T05:49:39.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.633+0000 7f9d84439640 1 Processor -- start 2023-10-08T05:49:39.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.633+0000 7f9d84439640 1 -- start start 2023-10-08T05:49:39.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.634+0000 7f9d84439640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 0x7f9d7c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:39.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.635+0000 7f9d84439640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7c071170 con 0x7f9d7c070cc0 2023-10-08T05:49:39.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.635+0000 7f9d84439640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7c06e150 con 0x7f9d7c160660 2023-10-08T05:49:39.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.635+0000 7f9d84439640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c06de70 con 0x7f9d7c0cb620 2023-10-08T05:49:39.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.635+0000 7f9d819d4640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9d7c070cc0 0x7f9d7c0caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39996/0 (socket says 172.21.15.70:39996) 2023-10-08T05:49:39.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.635+0000 7f9d819d4640 1 -- 172.21.15.70:0/187548378 learned_addr learned my addr 172.21.15.70:0/187548378 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:39.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.635+0000 7f9d829d6640 1 --2- 172.21.15.70:0/187548378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 0x7f9d7c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:39.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.635+0000 7f9d811d3640 1 -- 172.21.15.70:0/187548378 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309607942 0 0) 0x7f9d6c001110 con 0x7f9d7c070cc0 2023-10-08T05:49:39.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.635+0000 7f9d811d3640 1 -- 172.21.15.70:0/187548378 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d68003520 con 0x7f9d7c070cc0 2023-10-08T05:49:39.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.636+0000 7f9d829d6640 1 -- 172.21.15.70:0/187548378 >> v1:172.21.15.181:6789/0 conn(0x7f9d7c160660 legacy=0x7f9d7c0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:39.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.636+0000 7f9d829d6640 1 -- 172.21.15.70:0/187548378 >> v1:172.21.15.70:6789/0 conn(0x7f9d7c070cc0 legacy=0x7f9d7c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:39.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.636+0000 7f9d829d6640 1 -- 172.21.15.70:0/187548378 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c0c8e30 con 0x7f9d7c0cb620 2023-10-08T05:49:39.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.637+0000 7f9d829d6640 1 --2- 172.21.15.70:0/187548378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 0x7f9d7c0b1990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009dc0 tx=0x7f9d7402bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfdd6fac662af312 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:39.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.637+0000 7f9d811d3640 1 -- 172.21.15.70:0/187548378 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7402ccc0 con 0x7f9d7c0cb620 2023-10-08T05:49:39.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.637+0000 7f9d811d3640 1 -- 172.21.15.70:0/187548378 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7402ce20 con 0x7f9d7c0cb620 2023-10-08T05:49:39.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.637+0000 7f9d811d3640 1 -- 172.21.15.70:0/187548378 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74034660 con 0x7f9d7c0cb620 2023-10-08T05:49:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.638+0000 7f9d84439640 1 -- 172.21.15.70:0/187548378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 msgr2=0x7f9d7c0b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.638+0000 7f9d84439640 1 --2- 172.21.15.70:0/187548378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 0x7f9d7c0b1990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009dc0 tx=0x7f9d7402bf10 comp rx=0 tx=0).stop 2023-10-08T05:49:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.638+0000 7f9d84439640 1 -- 172.21.15.70:0/187548378 shutdown_connections 2023-10-08T05:49:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.638+0000 7f9d84439640 1 --2- 172.21.15.70:0/187548378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 0x7f9d7c0b1990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.638+0000 7f9d84439640 1 -- 172.21.15.70:0/187548378 >> 172.21.15.70:0/187548378 conn(0x7f9d7c06a5d0 msgr2=0x7f9d7c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.638+0000 7f9d84439640 1 -- 172.21.15.70:0/187548378 shutdown_connections 2023-10-08T05:49:39.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.638+0000 7f9d84439640 1 -- 172.21.15.70:0/187548378 wait complete. 2023-10-08T05:49:39.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.640+0000 7f9d84439640 1 Processor -- start 2023-10-08T05:49:39.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.640+0000 7f9d84439640 1 -- start start 2023-10-08T05:49:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.641+0000 7f9d84439640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c070cc0 0x7f9d7c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.641+0000 7f9d84439640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 0x7f9d7c0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.641+0000 7f9d84439640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7c160660 0x7f9d7c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.641+0000 7f9d84439640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c1f2700 con 0x7f9d7c160660 2023-10-08T05:49:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.641+0000 7f9d84439640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0c2250 con 0x7f9d7c070cc0 2023-10-08T05:49:39.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.641+0000 7f9d84439640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c155c60 con 0x7f9d7c0cb620 2023-10-08T05:49:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.641+0000 7f9d829d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7c160660 0x7f9d7c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d829d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7c160660 0x7f9d7c15d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57100/0 (socket says 172.21.15.70:57100) 2023-10-08T05:49:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d829d6640 1 -- 172.21.15.70:0/3834112973 learned_addr learned my addr 172.21.15.70:0/3834112973 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d819d4640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 0x7f9d7c0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d821d5640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c070cc0 0x7f9d7c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d829d6640 1 -- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 msgr2=0x7f9d7c0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d829d6640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 0x7f9d7c0c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d829d6640 1 -- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c070cc0 msgr2=0x7f9d7c1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d829d6640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c070cc0 0x7f9d7c1555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:39.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d829d6640 1 -- 172.21.15.70:0/3834112973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d74009a70 con 0x7f9d7c160660 2023-10-08T05:49:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d821d5640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c070cc0 0x7f9d7c1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.642+0000 7f9d829d6640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7c160660 0x7f9d7c15d960 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009ee0 tx=0x7f9d74004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.643+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3834112973 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74040650 con 0x7f9d7c160660 2023-10-08T05:49:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.643+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3834112973 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d740407b0 con 0x7f9d7c160660 2023-10-08T05:49:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.643+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3834112973 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74040b60 con 0x7f9d7c160660 2023-10-08T05:49:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.643+0000 7f9d84439640 1 -- 172.21.15.70:0/3834112973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c0c84a0 con 0x7f9d7c160660 2023-10-08T05:49:39.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.643+0000 7f9d84439640 1 -- 172.21.15.70:0/3834112973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c0c2780 con 0x7f9d7c160660 2023-10-08T05:49:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.644+0000 7f9d84439640 1 -- 172.21.15.70:0/3834112973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c04e160 con 0x7f9d7c160660 2023-10-08T05:49:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.645+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3834112973 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f9d74040cc0 con 0x7f9d7c160660 2023-10-08T05:49:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.646+0000 7f9d72ffd640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9d6006b070 0x7f9d6006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:39.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.646+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3834112973 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6916+0+0 (secure 0 0 0) 0x7f9d740aca70 con 0x7f9d7c160660 2023-10-08T05:49:39.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.646+0000 7f9d821d5640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9d6006b070 0x7f9d6006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:39.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.647+0000 7f9d821d5640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9d6006b070 0x7f9d6006d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000f60 tx=0x7f9d78009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:39.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.650+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3834112973 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d740773d0 con 0x7f9d7c160660 2023-10-08T05:49:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:39.858+0000 7f9d84439640 1 -- 172.21.15.70:0/3834112973 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir4", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d7c0c93f0 con 0x7f9d6006b070 2023-10-08T05:49:40.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.059+0000 7f9d72ffd640 1 -- 172.21.15.70:0/3834112973 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f9d78001340 con 0x7f9d6006b070 2023-10-08T05:49:40.061 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-10-08T05:49:40.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.064+0000 7f9d84439640 1 -- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9d6006b070 msgr2=0x7f9d6006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:40.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.064+0000 7f9d84439640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9d6006b070 0x7f9d6006d530 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000f60 tx=0x7f9d78009040 comp rx=0 tx=0).stop 2023-10-08T05:49:40.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.064+0000 7f9d84439640 1 -- 172.21.15.70:0/3834112973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7c160660 msgr2=0x7f9d7c15d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:40.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.064+0000 7f9d84439640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7c160660 0x7f9d7c15d960 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009ee0 tx=0x7f9d74004ab0 comp rx=0 tx=0).stop 2023-10-08T05:49:40.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.064+0000 7f9d84439640 1 -- 172.21.15.70:0/3834112973 shutdown_connections 2023-10-08T05:49:40.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.064+0000 7f9d84439640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d7c160660 0x7f9d7c15d960 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:40.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.064+0000 7f9d84439640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9d6006b070 0x7f9d6006d530 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:40.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.064+0000 7f9d84439640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d7c0cb620 0x7f9d7c0c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:40.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.064+0000 7f9d84439640 1 --2- 172.21.15.70:0/3834112973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9d7c070cc0 0x7f9d7c1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:40.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.064+0000 7f9d84439640 1 -- 172.21.15.70:0/3834112973 >> 172.21.15.70:0/3834112973 conn(0x7f9d7c06a5d0 msgr2=0x7f9d7c0caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:40.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.065+0000 7f9d84439640 1 -- 172.21.15.70:0/3834112973 shutdown_connections 2023-10-08T05:49:40.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:40.065+0000 7f9d84439640 1 -- 172.21.15.70:0/3834112973 wait complete. 2023-10-08T05:49:40.091 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-10-08T05:49:40.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.091 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-08T05:49:40.133 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:40.135 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:40.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.136 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_39_00_UTC) 2023-10-08T05:49:40.251 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:40.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.252 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_40_00_UTC) 2023-10-08T05:49:40.371 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:40.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.371 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_41_00_UTC) 2023-10-08T05:49:40.438 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:40.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.439 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_42_00_UTC) 2023-10-08T05:49:40.486 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_43_01_UTC 2023-10-08T05:49:40.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.487 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_43_01_UTC) 2023-10-08T05:49:40.538 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:40.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.538 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_44_00_UTC) 2023-10-08T05:49:40.591 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:40.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.592 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_45_00_UTC) 2023-10-08T05:49:40.650 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:40.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.650 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_46_00_UTC) 2023-10-08T05:49:40.712 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:40.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.712 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_47_00_UTC) 2023-10-08T05:49:40.768 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:40.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.768 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_48_00_UTC) 2023-10-08T05:49:40.832 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:40.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.832 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-05_49_00_UTC) 2023-10-08T05:49:40.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:40.898 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-10-08T05:49:40.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir5 1m 2023-10-08T05:49:41.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.156+0000 7fb2e4409640 1 Processor -- start 2023-10-08T05:49:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.156+0000 7fb2e4409640 1 -- start start 2023-10-08T05:49:44.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.158+0000 7fb2e4409640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc06dfc0 0x7fb2dc070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:44.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.158+0000 7fb2e4409640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc0b1cc0 con 0x7fb2dc071070 2023-10-08T05:49:44.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.158+0000 7fb2e4409640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc0b2620 con 0x7fb2dc0afbe0 2023-10-08T05:49:44.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.158+0000 7fb2e4409640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0b2060 con 0x7fb2dc06dfc0 2023-10-08T05:49:44.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.158+0000 7fb2e29a6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb2dc071070 0x7fb2dc0cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40006/0 (socket says 172.21.15.70:40006) 2023-10-08T05:49:44.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.158+0000 7fb2e29a6640 1 -- 172.21.15.70:0/3700659722 learned_addr learned my addr 172.21.15.70:0/3700659722 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:44.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e19a4640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc06dfc0 0x7fb2dc070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:44.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903363366 0 0) 0x7fb2dc0b1cc0 con 0x7fb2dc071070 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c8003520 con 0x7fb2dc071070 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1238788267 0 0) 0x7fb2c8003520 con 0x7fb2dc071070 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc06dfc0 msgr2=0x7fb2dc070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc06dfc0 0x7fb2dc070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 >> v1:172.21.15.181:6789/0 conn(0x7fb2dc0afbe0 legacy=0x7fb2dc0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc0b2340 con 0x7fb2dc071070 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fb2d40019f0 con 0x7fb2dc071070 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 >> v1:172.21.15.70:6789/0 conn(0x7fb2dc071070 legacy=0x7fb2dc0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc06dfc0 0x7fb2c80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.159+0000 7fb2e11a3640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2c8003ee0 0x7fb2c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:44.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e11a3640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 0x7fb2c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0b2060 con 0x7fb2c8004d00 2023-10-08T05:49:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c800ba30 con 0x7fb2c8003ee0 2023-10-08T05:49:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c800bbf0 con 0x7fb2dc06dfc0 2023-10-08T05:49:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e29a6640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 0x7fb2c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e21a5640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc06dfc0 0x7fb2c80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:44.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e19a4640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2c8003ee0 0x7fb2c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:44.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e29a6640 1 -- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc06dfc0 msgr2=0x7fb2c80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:44.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e29a6640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc06dfc0 0x7fb2c80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:44.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e29a6640 1 -- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2c8003ee0 msgr2=0x7fb2c80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:44.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e29a6640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2c8003ee0 0x7fb2c80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:44.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.160+0000 7fb2e29a6640 1 -- 172.21.15.70:0/3700659722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8003520 con 0x7fb2c8004d00 2023-10-08T05:49:44.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.161+0000 7fb2e29a6640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 0x7fb2c8005180 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400a810 tx=0x7fb2c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437af0c36ac0372 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:44.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.162+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4034630 con 0x7fb2c8004d00 2023-10-08T05:49:44.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.162+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d402f070 con 0x7fb2c8004d00 2023-10-08T05:49:44.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.162+0000 7fb2e11a3640 1 -- 172.21.15.70:0/3700659722 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d40349d0 con 0x7fb2c8004d00 2023-10-08T05:49:44.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.162+0000 7fb2e4409640 1 -- 172.21.15.70:0/3700659722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 msgr2=0x7fb2c8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:44.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.162+0000 7fb2e4409640 1 --2- 172.21.15.70:0/3700659722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 0x7fb2c8005180 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400a810 tx=0x7fb2c80057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:44.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.162+0000 7fb2e29a6640 1 -- 172.21.15.70:0/3700659722 reap_dead start 2023-10-08T05:49:44.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.163+0000 7fb2e4409640 1 -- 172.21.15.70:0/3700659722 shutdown_connections 2023-10-08T05:49:44.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.163+0000 7fb2e4409640 1 -- 172.21.15.70:0/3700659722 >> 172.21.15.70:0/3700659722 conn(0x7fb2dc06a5d0 msgr2=0x7fb2dc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:44.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.163+0000 7fb2e4409640 1 -- 172.21.15.70:0/3700659722 shutdown_connections 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.163+0000 7fb2e4409640 1 -- 172.21.15.70:0/3700659722 wait complete. 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.164+0000 7fb2e4409640 1 Processor -- start 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.165+0000 7fb2e4409640 1 -- start start 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.165+0000 7fb2e4409640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 0x7fb2dc14dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e4409640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2dc15e1f0 0x7fb2dc0c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e4409640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc0c3180 0x7fb2dc0c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e4409640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc15b4d0 con 0x7fb2c8004d00 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e4409640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0c3600 con 0x7fb2dc15e1f0 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e4409640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0c38e0 con 0x7fb2dc0c3180 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e21a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 0x7fb2dc14dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e21a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 0x7fb2dc14dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57116/0 (socket says 172.21.15.70:57116) 2023-10-08T05:49:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e21a5640 1 -- 172.21.15.70:0/2666986271 learned_addr learned my addr 172.21.15.70:0/2666986271 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e21a5640 1 -- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc0c3180 msgr2=0x7fb2dc0c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e21a5640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc0c3180 0x7fb2dc0c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e21a5640 1 -- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2dc15e1f0 msgr2=0x7fb2dc0c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e21a5640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2dc15e1f0 0x7fb2dc0c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e21a5640 1 -- 172.21.15.70:0/2666986271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc15b6e0 con 0x7fb2c8004d00 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.166+0000 7fb2e21a5640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 0x7fb2dc14dc40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800e5d0 tx=0x7fb2d800ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.167+0000 7fb2d2ffd640 1 -- 172.21.15.70:0/2666986271 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80053a0 con 0x7fb2c8004d00 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.167+0000 7fb2d2ffd640 1 -- 172.21.15.70:0/2666986271 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d80035f0 con 0x7fb2c8004d00 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.167+0000 7fb2d2ffd640 1 -- 172.21.15.70:0/2666986271 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80039a0 con 0x7fb2c8004d00 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.167+0000 7fb2e4409640 1 -- 172.21.15.70:0/2666986271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc15b9c0 con 0x7fb2c8004d00 2023-10-08T05:49:44.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.167+0000 7fb2e4409640 1 -- 172.21.15.70:0/2666986271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc14e180 con 0x7fb2c8004d00 2023-10-08T05:49:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.168+0000 7fb2e4409640 1 -- 172.21.15.70:0/2666986271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc0c3aa0 con 0x7fb2c8004d00 2023-10-08T05:49:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.169+0000 7fb2d2ffd640 1 -- 172.21.15.70:0/2666986271 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fb2d8003b00 con 0x7fb2c8004d00 2023-10-08T05:49:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.170+0000 7fb2d2ffd640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb2c006b070 0x7fb2c006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.171+0000 7fb2d2ffd640 1 -- 172.21.15.70:0/2666986271 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6996+0+0 (secure 0 0 0) 0x7fb2d8089d10 con 0x7fb2c8004d00 2023-10-08T05:49:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.171+0000 7fb2e19a4640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb2c006b070 0x7fb2c006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.172+0000 7fb2e19a4640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb2c006b070 0x7fb2c006d530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc15a3c0 tx=0x7fb2cc008040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.175+0000 7fb2d2ffd640 1 -- 172.21.15.70:0/2666986271 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2d8054630 con 0x7fb2c8004d00 2023-10-08T05:49:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.422+0000 7fb2e4409640 1 -- 172.21.15.70:0/2666986271 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir5", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2dc14e370 con 0x7fb2c006b070 2023-10-08T05:49:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.623+0000 7fb2d2ffd640 1 -- 172.21.15.70:0/2666986271 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb2cc004040 con 0x7fb2c006b070 2023-10-08T05:49:44.615 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-10-08T05:49:44.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 -- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb2c006b070 msgr2=0x7fb2c006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:44.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb2c006b070 0x7fb2c006d530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc15a3c0 tx=0x7fb2cc008040 comp rx=0 tx=0).stop 2023-10-08T05:49:44.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 -- 172.21.15.70:0/2666986271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 msgr2=0x7fb2dc14dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 0x7fb2dc14dc40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb2d800e5d0 tx=0x7fb2d800ea90 comp rx=0 tx=0).stop 2023-10-08T05:49:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 -- 172.21.15.70:0/2666986271 shutdown_connections 2023-10-08T05:49:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fb2c006b070 0x7fb2c006d530 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb2dc0c3180 0x7fb2dc0c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb2dc15e1f0 0x7fb2dc0c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 --2- 172.21.15.70:0/2666986271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2c8004d00 0x7fb2dc14dc40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 -- 172.21.15.70:0/2666986271 >> 172.21.15.70:0/2666986271 conn(0x7fb2dc06a5d0 msgr2=0x7fb2dc0c9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 -- 172.21.15.70:0/2666986271 shutdown_connections 2023-10-08T05:49:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:41.626+0000 7fb2e4409640 1 -- 172.21.15.70:0/2666986271 wait complete. 2023-10-08T05:49:44.617 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-10-08T05:49:44.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:44.618 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-08T05:49:44.682 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_01_UTC 2023-10-08T05:49:44.763 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_01_UTC 2023-10-08T05:49:44.763 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:44.763 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:44.763 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:44.763 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_01_UTC 2023-10-08T05:49:44.764 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:44.764 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:44.764 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:44.764 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:44.765 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:44.766 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_39_01_UTC 2023-10-08T05:49:44.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:44.766 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_39_01_UTC) 2023-10-08T05:49:44.840 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_40_01_UTC 2023-10-08T05:49:44.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:44.920 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_40_01_UTC) 2023-10-08T05:49:44.991 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:44.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:44.991 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_41_00_UTC) 2023-10-08T05:49:45.057 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:45.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:45.076 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_42_00_UTC) 2023-10-08T05:49:45.142 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:45.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:45.143 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_43_00_UTC) 2023-10-08T05:49:45.207 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_44_01_UTC 2023-10-08T05:49:45.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:45.238 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_44_01_UTC) 2023-10-08T05:49:45.308 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:45.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:45.309 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_45_01_UTC) 2023-10-08T05:49:45.372 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:45.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:45.373 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_46_00_UTC) 2023-10-08T05:49:45.436 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:45.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:45.437 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_47_00_UTC) 2023-10-08T05:49:45.498 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:45.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:45.498 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_48_00_UTC) 2023-10-08T05:49:45.559 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:45.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:45.560 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-05_49_00_UTC) 2023-10-08T05:49:45.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:45.622 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-10-08T05:49:45.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir6 1m 2023-10-08T05:49:45.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.885+0000 7f507d133640 1 Processor -- start 2023-10-08T05:49:45.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.885+0000 7f507d133640 1 -- start start 2023-10-08T05:49:45.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.887+0000 7f507d133640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f50780cb620 0x7f50780b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:45.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.887+0000 7f507d133640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5078071170 con 0x7f5078070cc0 2023-10-08T05:49:45.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.887+0000 7f507d133640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507806e150 con 0x7f5078160660 2023-10-08T05:49:45.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.887+0000 7f507d133640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507806de70 con 0x7f50780cb620 2023-10-08T05:49:45.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.887+0000 7f507659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5078070cc0 0x7f50780caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44224/0 (socket says 172.21.15.70:44224) 2023-10-08T05:49:45.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.887+0000 7f507659c640 1 -- 172.21.15.70:0/724474142 learned_addr learned my addr 172.21.15.70:0/724474142 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:45.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.887+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127354899 0 0) 0x7f506c001110 con 0x7f5078070cc0 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5060003520 con 0x7f5078070cc0 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f507759e640 1 --2- 172.21.15.70:0/724474142 >> v2:172.21.15.181:3300/0 conn(0x7f50780cb620 0x7f50780b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1279338411 0 0) 0x7f5060003520 con 0x7f5078070cc0 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 >> v2:172.21.15.181:3300/0 conn(0x7f50780cb620 msgr2=0x7f50780b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 --2- 172.21.15.70:0/724474142 >> v2:172.21.15.181:3300/0 conn(0x7f50780cb620 0x7f50780b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 >> v1:172.21.15.181:6789/0 conn(0x7f5078160660 legacy=0x7f50780c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780c8e30 con 0x7f5078070cc0 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f506c001800 con 0x7f5078070cc0 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 >> v1:172.21.15.70:6789/0 conn(0x7f5078070cc0 legacy=0x7f50780caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50600039c0 0x7f5060003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:45.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50600047c0 0x7f5060004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50600052a0 0x7f506000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507806de70 con 0x7f50600047c0 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506000c1f0 con 0x7f50600052a0 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506000c3b0 con 0x7f50600039c0 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.888+0000 7f507659c640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50600047c0 0x7f5060004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.889+0000 7f5076d9d640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50600039c0 0x7f5060003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.889+0000 7f507759e640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50600052a0 0x7f506000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.889+0000 7f507659c640 1 -- 172.21.15.70:0/724474142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50600039c0 msgr2=0x7f5060003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.889+0000 7f507659c640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50600039c0 0x7f5060003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.889+0000 7f507659c640 1 -- 172.21.15.70:0/724474142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50600052a0 msgr2=0x7f506000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:45.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.889+0000 7f507659c640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50600052a0 0x7f506000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:45.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.889+0000 7f507659c640 1 -- 172.21.15.70:0/724474142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5060003520 con 0x7f50600047c0 2023-10-08T05:49:45.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.889+0000 7f507759e640 1 -- 172.21.15.70:0/724474142 reap_dead start 2023-10-08T05:49:45.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.890+0000 7f507659c640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50600047c0 0x7f5060004c40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f506c001eb0 tx=0x7f50600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777a5b1e26044c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:45.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.890+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c005ad0 con 0x7f50600047c0 2023-10-08T05:49:45.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.890+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c005c50 con 0x7f50600047c0 2023-10-08T05:49:45.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.890+0000 7f5075d9b640 1 -- 172.21.15.70:0/724474142 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c02c7c0 con 0x7f50600047c0 2023-10-08T05:49:45.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.891+0000 7f507d133640 1 -- 172.21.15.70:0/724474142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50600047c0 msgr2=0x7f5060004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:45.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.891+0000 7f507d133640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50600047c0 0x7f5060004c40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f506c001eb0 tx=0x7f50600057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:45.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.891+0000 7f507d133640 1 -- 172.21.15.70:0/724474142 shutdown_connections 2023-10-08T05:49:45.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.891+0000 7f507d133640 1 --2- 172.21.15.70:0/724474142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50600047c0 0x7f5060004c40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:45.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.891+0000 7f507d133640 1 -- 172.21.15.70:0/724474142 >> 172.21.15.70:0/724474142 conn(0x7f507806a5d0 msgr2=0x7f50780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:45.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.891+0000 7f507d133640 1 -- 172.21.15.70:0/724474142 shutdown_connections 2023-10-08T05:49:45.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.891+0000 7f507d133640 1 -- 172.21.15.70:0/724474142 wait complete. 2023-10-08T05:49:45.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.893+0000 7f507d133640 1 Processor -- start 2023-10-08T05:49:45.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.893+0000 7f507d133640 1 -- start start 2023-10-08T05:49:45.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.894+0000 7f507d133640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50600047c0 0x7f50781f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.894+0000 7f507d133640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50781f3950 0x7f50781f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.894+0000 7f507d133640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50781f2cd0 0x7f50780c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.894+0000 7f507d133640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780c39e0 con 0x7f50781f2cd0 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.894+0000 7f507d133640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50781f32c0 con 0x7f50600047c0 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.894+0000 7f507d133640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50781f35a0 con 0x7f50781f3950 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.894+0000 7f507759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50781f2cd0 0x7f50780c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.894+0000 7f507759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50781f2cd0 0x7f50780c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42288/0 (socket says 172.21.15.70:42288) 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.894+0000 7f507759e640 1 -- 172.21.15.70:0/2049232667 learned_addr learned my addr 172.21.15.70:0/2049232667 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f507659c640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50781f3950 0x7f50781f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f507759e640 1 -- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50781f3950 msgr2=0x7f50781f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:45.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f507759e640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50781f3950 0x7f50781f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:45.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f507759e640 1 -- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50600047c0 msgr2=0x7f50781f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:45.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f507759e640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50600047c0 0x7f50781f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:45.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f507759e640 1 -- 172.21.15.70:0/2049232667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780c3bf0 con 0x7f50781f2cd0 2023-10-08T05:49:45.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f507759e640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50781f2cd0 0x7f50780c70e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f50780cc050 tx=0x7f5070009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:45.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f505f7fe640 1 -- 172.21.15.70:0/2049232667 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070007c90 con 0x7f50781f2cd0 2023-10-08T05:49:45.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f505f7fe640 1 -- 172.21.15.70:0/2049232667 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5070007df0 con 0x7f50781f2cd0 2023-10-08T05:49:45.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f505f7fe640 1 -- 172.21.15.70:0/2049232667 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070014640 con 0x7f50781f2cd0 2023-10-08T05:49:45.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f507d133640 1 -- 172.21.15.70:0/2049232667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50780c3ed0 con 0x7f50781f2cd0 2023-10-08T05:49:45.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.895+0000 7f507d133640 1 -- 172.21.15.70:0/2049232667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50780c7620 con 0x7f50781f2cd0 2023-10-08T05:49:45.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.897+0000 7f507d133640 1 -- 172.21.15.70:0/2049232667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50781f3760 con 0x7f50781f2cd0 2023-10-08T05:49:45.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.897+0000 7f505f7fe640 1 -- 172.21.15.70:0/2049232667 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f507001d3b0 con 0x7f50781f2cd0 2023-10-08T05:49:45.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.899+0000 7f505f7fe640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f506406b070 0x7f506406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:45.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.899+0000 7f505f7fe640 1 -- 172.21.15.70:0/2049232667 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7168+0+0 (secure 0 0 0) 0x7f5070089ce0 con 0x7f50781f2cd0 2023-10-08T05:49:45.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.900+0000 7f5076d9d640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f506406b070 0x7f506406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:45.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.901+0000 7f5076d9d640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f506406b070 0x7f506406d530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5068007fd0 tx=0x7f5068009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:45.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:45.904+0000 7f505f7fe640 1 -- 172.21.15.70:0/2049232667 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50700545f0 con 0x7f50781f2cd0 2023-10-08T05:49:46.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.116+0000 7f507d133640 1 -- 172.21.15.70:0/2049232667 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir6", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f50780c93f0 con 0x7f506406b070 2023-10-08T05:49:46.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.311+0000 7f505f7fe640 1 -- 172.21.15.70:0/2049232667 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f50680050d0 con 0x7f506406b070 2023-10-08T05:49:46.312 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-10-08T05:49:46.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.314+0000 7f507d133640 1 -- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f506406b070 msgr2=0x7f506406d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:46.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.314+0000 7f507d133640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f506406b070 0x7f506406d530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5068007fd0 tx=0x7f5068009040 comp rx=0 tx=0).stop 2023-10-08T05:49:46.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.314+0000 7f507d133640 1 -- 172.21.15.70:0/2049232667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50781f2cd0 msgr2=0x7f50780c70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:46.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.314+0000 7f507d133640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50781f2cd0 0x7f50780c70e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f50780cc050 tx=0x7f5070009d20 comp rx=0 tx=0).stop 2023-10-08T05:49:46.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.314+0000 7f507d133640 1 -- 172.21.15.70:0/2049232667 shutdown_connections 2023-10-08T05:49:46.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.314+0000 7f507d133640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50781f2cd0 0x7f50780c70e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:46.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.314+0000 7f507d133640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f506406b070 0x7f506406d530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:46.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.314+0000 7f507d133640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50781f3950 0x7f50781f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:46.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.314+0000 7f507d133640 1 --2- 172.21.15.70:0/2049232667 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50600047c0 0x7f50781f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:46.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.314+0000 7f507d133640 1 -- 172.21.15.70:0/2049232667 >> 172.21.15.70:0/2049232667 conn(0x7f507806a5d0 msgr2=0x7f50780caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:46.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.315+0000 7f507d133640 1 -- 172.21.15.70:0/2049232667 shutdown_connections 2023-10-08T05:49:46.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:46.315+0000 7f507d133640 1 -- 172.21.15.70:0/2049232667 wait complete. 2023-10-08T05:49:46.334 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-10-08T05:49:46.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:46.335 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-08T05:49:46.386 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:46.387 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:46.387 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:46.387 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:46.387 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:46.387 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:46.387 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:46.387 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:46.387 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_01_UTC 2023-10-08T05:49:46.387 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_01_UTC 2023-10-08T05:49:46.387 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:46.388 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:46.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:46.389 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_39_00_UTC) 2023-10-08T05:49:46.453 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:46.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:46.453 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_40_00_UTC) 2023-10-08T05:49:46.520 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:46.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:46.520 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_41_00_UTC) 2023-10-08T05:49:46.644 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:46.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:46.644 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_42_00_UTC) 2023-10-08T05:49:46.696 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:46.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:46.697 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_43_00_UTC) 2023-10-08T05:49:46.744 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:46.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:46.744 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_44_00_UTC) 2023-10-08T05:49:46.848 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_45_01_UTC 2023-10-08T05:49:46.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:46.848 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_45_01_UTC) 2023-10-08T05:49:46.913 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:46.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:46.914 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_46_01_UTC) 2023-10-08T05:49:46.962 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_47_01_UTC 2023-10-08T05:49:46.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:46.962 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_47_01_UTC) 2023-10-08T05:49:47.011 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_48_01_UTC 2023-10-08T05:49:47.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:47.012 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_48_01_UTC) 2023-10-08T05:49:47.064 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:47.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:47.065 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-05_49_01_UTC) 2023-10-08T05:49:47.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:47.119 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-10-08T05:49:47.172 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir7 1m 2023-10-08T05:49:47.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.386+0000 7fe613fff640 1 Processor -- start 2023-10-08T05:49:47.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.387+0000 7fe613fff640 1 -- start start 2023-10-08T05:49:47.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.388+0000 7fe613fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140cb5e0 0x7fe6140cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:47.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.388+0000 7fe613fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6140b3c50 0x7fe6140cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:47.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.388+0000 7fe613fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe614159db0 con 0x7fe6141599d0 2023-10-08T05:49:47.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.388+0000 7fe613fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe614071fe0 con 0x7fe6140b3c50 2023-10-08T05:49:47.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.388+0000 7fe613fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140b44c0 con 0x7fe6140cb5e0 2023-10-08T05:49:47.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.388+0000 7fe612ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe6141599d0 0x7fe6140caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44234/0 (socket says 172.21.15.70:44234) 2023-10-08T05:49:47.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe612ffd640 1 -- 172.21.15.70:0/754082224 learned_addr learned my addr 172.21.15.70:0/754082224 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:47.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618615473 0 0) 0x7fe604001490 con 0x7fe6141599d0 2023-10-08T05:49:47.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe600003520 con 0x7fe6141599d0 2023-10-08T05:49:47.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe6137fe640 1 --2- 172.21.15.70:0/754082224 >> v2:172.21.15.181:3300/0 conn(0x7fe6140b3c50 0x7fe6140cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:47.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe6127fc640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140cb5e0 0x7fe6140cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:47.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1538109216 0 0) 0x7fe600003520 con 0x7fe6141599d0 2023-10-08T05:49:47.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140cb5e0 msgr2=0x7fe6140cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:47.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140cb5e0 0x7fe6140cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 >> v2:172.21.15.181:3300/0 conn(0x7fe6140b3c50 msgr2=0x7fe6140cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:47.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 --2- 172.21.15.70:0/754082224 >> v2:172.21.15.181:3300/0 conn(0x7fe6140b3c50 0x7fe6140cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140713d0 con 0x7fe6141599d0 2023-10-08T05:49:47.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe604001b80 con 0x7fe6141599d0 2023-10-08T05:49:47.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 >> v1:172.21.15.70:6789/0 conn(0x7fe6141599d0 legacy=0x7fe6140caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:47.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.389+0000 7fe611ffb640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600003970 0x7fe600003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:47.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe611ffb640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140cb5e0 0x7fe6000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:47.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe611ffb640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600004d00 0x7fe600005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:47.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe614071fe0 con 0x7fe600004d00 2023-10-08T05:49:47.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140b44c0 con 0x7fe600003970 2023-10-08T05:49:47.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe60000ba30 con 0x7fe6140cb5e0 2023-10-08T05:49:47.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe6137fe640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600004d00 0x7fe600005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:47.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe6127fc640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140cb5e0 0x7fe6000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:47.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe612ffd640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600003970 0x7fe600003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:47.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe6137fe640 1 -- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140cb5e0 msgr2=0x7fe6000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:47.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe6137fe640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140cb5e0 0x7fe6000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe6137fe640 1 -- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600003970 msgr2=0x7fe600003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:47.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe6137fe640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600003970 0x7fe600003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe6137fe640 1 -- 172.21.15.70:0/754082224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe600003520 con 0x7fe600004d00 2023-10-08T05:49:47.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.390+0000 7fe6127fc640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140cb5e0 0x7fe6000047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:47.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.391+0000 7fe6137fe640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600004d00 0x7fe600005180 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00ba60 tx=0x7fe6000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208aa0ec586bece5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:47.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.392+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c032680 con 0x7fe600004d00 2023-10-08T05:49:47.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.392+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c0327e0 con 0x7fe600004d00 2023-10-08T05:49:47.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.392+0000 7fe611ffb640 1 -- 172.21.15.70:0/754082224 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c032b70 con 0x7fe600004d00 2023-10-08T05:49:47.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.392+0000 7fe613fff640 1 -- 172.21.15.70:0/754082224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600004d00 msgr2=0x7fe600005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:47.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.392+0000 7fe613fff640 1 --2- 172.21.15.70:0/754082224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600004d00 0x7fe600005180 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00ba60 tx=0x7fe6000057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.392+0000 7fe6137fe640 1 -- 172.21.15.70:0/754082224 reap_dead start 2023-10-08T05:49:47.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.392+0000 7fe613fff640 1 -- 172.21.15.70:0/754082224 shutdown_connections 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.392+0000 7fe613fff640 1 -- 172.21.15.70:0/754082224 >> 172.21.15.70:0/754082224 conn(0x7fe61406c050 msgr2=0x7fe61406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.392+0000 7fe613fff640 1 -- 172.21.15.70:0/754082224 shutdown_connections 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.393+0000 7fe613fff640 1 -- 172.21.15.70:0/754082224 wait complete. 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.394+0000 7fe613fff640 1 Processor -- start 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.394+0000 7fe613fff640 1 -- start start 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.395+0000 7fe613fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600004d00 0x7fe6140c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.395+0000 7fe613fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6140c8010 0x7fe61414e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.395+0000 7fe613fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140c5830 0x7fe6141f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.395+0000 7fe613fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140c8990 con 0x7fe6140c8010 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.395+0000 7fe613fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140c8490 con 0x7fe600004d00 2023-10-08T05:49:47.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.395+0000 7fe613fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140c86b0 con 0x7fe6140c5830 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe6127fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6140c8010 0x7fe61414e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe6127fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6140c8010 0x7fe61414e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42312/0 (socket says 172.21.15.70:42312) 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe6127fc640 1 -- 172.21.15.70:0/2300946910 learned_addr learned my addr 172.21.15.70:0/2300946910 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe612ffd640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600004d00 0x7fe6140c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe6127fc640 1 -- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140c5830 msgr2=0x7fe6141f1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe6127fc640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140c5830 0x7fe6141f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe6127fc640 1 -- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600004d00 msgr2=0x7fe6140c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe6127fc640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600004d00 0x7fe6140c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe6127fc640 1 -- 172.21.15.70:0/2300946910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140b1750 con 0x7fe6140c8010 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe6127fc640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6140c8010 0x7fe61414e350 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00e840 tx=0x7fe5fc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.396+0000 7fe60b7fe640 1 -- 172.21.15.70:0/2300946910 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc00c650 con 0x7fe6140c8010 2023-10-08T05:49:47.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.397+0000 7fe60b7fe640 1 -- 172.21.15.70:0/2300946910 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc00c7b0 con 0x7fe6140c8010 2023-10-08T05:49:47.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.397+0000 7fe60b7fe640 1 -- 172.21.15.70:0/2300946910 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc00cb60 con 0x7fe6140c8010 2023-10-08T05:49:47.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.397+0000 7fe613fff640 1 -- 172.21.15.70:0/2300946910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6140b1a30 con 0x7fe6140c8010 2023-10-08T05:49:47.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.397+0000 7fe613fff640 1 -- 172.21.15.70:0/2300946910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6140c5cb0 con 0x7fe6140c8010 2023-10-08T05:49:47.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.398+0000 7fe613fff640 1 -- 172.21.15.70:0/2300946910 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6140b15b0 con 0x7fe6140c8010 2023-10-08T05:49:47.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.399+0000 7fe60b7fe640 1 -- 172.21.15.70:0/2300946910 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7fe5fc00ccc0 con 0x7fe6140c8010 2023-10-08T05:49:47.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.400+0000 7fe60b7fe640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe5f406b030 0x7fe5f406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:47.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.400+0000 7fe60b7fe640 1 -- 172.21.15.70:0/2300946910 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6960+0+0 (secure 0 0 0) 0x7fe5fc089cc0 con 0x7fe6140c8010 2023-10-08T05:49:47.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.401+0000 7fe612ffd640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe5f406b030 0x7fe5f406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:47.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.402+0000 7fe612ffd640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe5f406b030 0x7fe5f406d4f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe604001710 tx=0x7fe604001250 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:47.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.405+0000 7fe60b7fe640 1 -- 172.21.15.70:0/2300946910 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6140b15b0 con 0x7fe6140c8010 2023-10-08T05:49:47.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.625+0000 7fe613fff640 1 -- 172.21.15.70:0/2300946910 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir7", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fe614071990 con 0x7fe5f406b030 2023-10-08T05:49:47.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.849+0000 7fe60b7fe640 1 -- 172.21.15.70:0/2300946910 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fe60400d050 con 0x7fe5f406b030 2023-10-08T05:49:47.850 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-10-08T05:49:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.851+0000 7fe613fff640 1 -- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe5f406b030 msgr2=0x7fe5f406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.851+0000 7fe613fff640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe5f406b030 0x7fe5f406d4f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe604001710 tx=0x7fe604001250 comp rx=0 tx=0).stop 2023-10-08T05:49:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.851+0000 7fe613fff640 1 -- 172.21.15.70:0/2300946910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6140c8010 msgr2=0x7fe61414e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.851+0000 7fe613fff640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6140c8010 0x7fe61414e350 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc00e840 tx=0x7fe5fc00ed00 comp rx=0 tx=0).stop 2023-10-08T05:49:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.851+0000 7fe613fff640 1 -- 172.21.15.70:0/2300946910 shutdown_connections 2023-10-08T05:49:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.851+0000 7fe613fff640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe5f406b030 0x7fe5f406d4f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.851+0000 7fe613fff640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6140c5830 0x7fe6141f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.851+0000 7fe613fff640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6140c8010 0x7fe61414e350 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.851+0000 7fe613fff640 1 --2- 172.21.15.70:0/2300946910 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe600004d00 0x7fe6140c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:47.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.851+0000 7fe613fff640 1 -- 172.21.15.70:0/2300946910 >> 172.21.15.70:0/2300946910 conn(0x7fe61406c050 msgr2=0x7fe61415a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:47.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.852+0000 7fe613fff640 1 -- 172.21.15.70:0/2300946910 shutdown_connections 2023-10-08T05:49:47.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:47.852+0000 7fe613fff640 1 -- 172.21.15.70:0/2300946910 wait complete. 2023-10-08T05:49:47.872 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-10-08T05:49:47.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:47.872 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-08T05:49:47.930 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:47.930 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:47.930 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:47.931 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:47.931 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:47.931 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:47.931 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:47.931 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:47.931 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_01_UTC 2023-10-08T05:49:47.931 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_01_UTC 2023-10-08T05:49:47.931 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:47.932 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:47.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:47.933 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_39_00_UTC) 2023-10-08T05:49:48.004 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:48.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.004 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_40_00_UTC) 2023-10-08T05:49:48.060 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:48.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.061 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_41_00_UTC) 2023-10-08T05:49:48.116 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:48.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.117 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_42_00_UTC) 2023-10-08T05:49:48.175 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:48.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.176 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_43_00_UTC) 2023-10-08T05:49:48.240 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:48.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.240 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_44_00_UTC) 2023-10-08T05:49:48.308 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:48.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.308 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_45_00_UTC) 2023-10-08T05:49:48.361 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_46_01_UTC 2023-10-08T05:49:48.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.361 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_46_01_UTC) 2023-10-08T05:49:48.408 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_47_01_UTC 2023-10-08T05:49:48.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.409 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_47_01_UTC) 2023-10-08T05:49:48.459 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_48_01_UTC 2023-10-08T05:49:48.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.460 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_48_01_UTC) 2023-10-08T05:49:48.511 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_49_01_UTC 2023-10-08T05:49:48.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.511 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-05_49_01_UTC) 2023-10-08T05:49:48.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:48.568 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-10-08T05:49:48.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir8 1m 2023-10-08T05:49:48.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.824+0000 7f98a90da640 1 Processor -- start 2023-10-08T05:49:48.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.824+0000 7f98a90da640 1 -- start start 2023-10-08T05:49:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.825+0000 7f98a90da640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f98a406db60 0x7f98a40b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.825+0000 7f98a90da640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a415b310 con 0x7f98a40c9cf0 2023-10-08T05:49:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.825+0000 7f98a90da640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a406d290 con 0x7f98a406c3a0 2023-10-08T05:49:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.825+0000 7f98a90da640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a415b110 con 0x7f98a406db60 2023-10-08T05:49:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.826+0000 7f989bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f98a40c9cf0 0x7f98a415aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44244/0 (socket says 172.21.15.70:44244) 2023-10-08T05:49:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.826+0000 7f989bfff640 1 -- 172.21.15.70:0/3760028294 learned_addr learned my addr 172.21.15.70:0/3760028294 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.826+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191748353 0 0) 0x7f988c001110 con 0x7f98a40c9cf0 2023-10-08T05:49:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.826+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9890003520 con 0x7f98a40c9cf0 2023-10-08T05:49:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.826+0000 7f98a359e640 1 --2- 172.21.15.70:0/3760028294 >> v2:172.21.15.181:3300/0 conn(0x7f98a406db60 0x7f98a40b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.826+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3830475717 0 0) 0x7f9890003520 con 0x7f98a40c9cf0 2023-10-08T05:49:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.826+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 >> v2:172.21.15.181:3300/0 conn(0x7f98a406db60 msgr2=0x7f98a40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.826+0000 7f98a259c640 1 --2- 172.21.15.70:0/3760028294 >> v2:172.21.15.181:3300/0 conn(0x7f98a406db60 0x7f98a40b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.826+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 >> v1:172.21.15.181:6789/0 conn(0x7f98a406c3a0 legacy=0x7f98a406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.826+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a40c9860 con 0x7f98a40c9cf0 2023-10-08T05:49:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f988c001800 con 0x7f98a40c9cf0 2023-10-08T05:49:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 >> v1:172.21.15.70:6789/0 conn(0x7f98a40c9cf0 legacy=0x7f98a415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a259c640 1 --2- 172.21.15.70:0/3760028294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98900039c0 0x7f9890003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a259c640 1 --2- 172.21.15.70:0/3760028294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98900047c0 0x7f9890004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a259c640 1 --2- 172.21.15.70:0/3760028294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9890005180 0x7f989000ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a415b110 con 0x7f9890005180 2023-10-08T05:49:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9890010610 con 0x7f98900047c0 2023-10-08T05:49:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98900107d0 con 0x7f98900039c0 2023-10-08T05:49:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a359e640 1 --2- 172.21.15.70:0/3760028294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9890005180 0x7f989000ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f989bfff640 1 --2- 172.21.15.70:0/3760028294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98900047c0 0x7f9890004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a359e640 1 -- 172.21.15.70:0/3760028294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98900039c0 msgr2=0x7f9890003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a359e640 1 --2- 172.21.15.70:0/3760028294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98900039c0 0x7f9890003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.827+0000 7f98a359e640 1 -- 172.21.15.70:0/3760028294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98900047c0 msgr2=0x7f9890004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.828+0000 7f98a359e640 1 --2- 172.21.15.70:0/3760028294 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98900047c0 0x7f9890004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.828+0000 7f98a359e640 1 -- 172.21.15.70:0/3760028294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9890003520 con 0x7f9890005180 2023-10-08T05:49:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.828+0000 7f98a359e640 1 -- 172.21.15.70:0/3760028294 reap_dead start 2023-10-08T05:49:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.829+0000 7f98a359e640 1 --2- 172.21.15.70:0/3760028294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9890005180 0x7f989000ff90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f989c007f60 tx=0x7f98900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e90817fa80c0b22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:48.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.829+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c033600 con 0x7f9890005180 2023-10-08T05:49:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.829+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c02f050 con 0x7f9890005180 2023-10-08T05:49:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.829+0000 7f98a259c640 1 -- 172.21.15.70:0/3760028294 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c033980 con 0x7f9890005180 2023-10-08T05:49:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.830+0000 7f98a90da640 1 -- 172.21.15.70:0/3760028294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9890005180 msgr2=0x7f989000ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.830+0000 7f98a90da640 1 --2- 172.21.15.70:0/3760028294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9890005180 0x7f989000ff90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f989c007f60 tx=0x7f98900057f0 comp rx=0 tx=0).stop 2023-10-08T05:49:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.830+0000 7f98a90da640 1 -- 172.21.15.70:0/3760028294 shutdown_connections 2023-10-08T05:49:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.830+0000 7f98a90da640 1 --2- 172.21.15.70:0/3760028294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9890005180 0x7f989000ff90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.830+0000 7f98a90da640 1 -- 172.21.15.70:0/3760028294 >> 172.21.15.70:0/3760028294 conn(0x7f98a406a5d0 msgr2=0x7f98a40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.830+0000 7f98a90da640 1 -- 172.21.15.70:0/3760028294 shutdown_connections 2023-10-08T05:49:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.830+0000 7f98a90da640 1 -- 172.21.15.70:0/3760028294 wait complete. 2023-10-08T05:49:48.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.832+0000 7f98a90da640 1 Processor -- start 2023-10-08T05:49:48.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.832+0000 7f98a90da640 1 -- start start 2023-10-08T05:49:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.833+0000 7f98a90da640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9890005180 0x7f98a4142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.833+0000 7f98a90da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a40c1970 0x7f98a415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.833+0000 7f98a90da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98a415cad0 0x7f98a41f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.833+0000 7f98a90da640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a415cf80 con 0x7f98a40c1970 2023-10-08T05:49:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.833+0000 7f98a90da640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a41f26b0 con 0x7f9890005180 2023-10-08T05:49:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.833+0000 7f98a90da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a40cdce0 con 0x7f98a415cad0 2023-10-08T05:49:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.833+0000 7f989bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a40c1970 0x7f98a415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.834+0000 7f989bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a40c1970 0x7f98a415c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42326/0 (socket says 172.21.15.70:42326) 2023-10-08T05:49:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.834+0000 7f989bfff640 1 -- 172.21.15.70:0/3232969092 learned_addr learned my addr 172.21.15.70:0/3232969092 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.834+0000 7f98a2d9d640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9890005180 0x7f98a4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.834+0000 7f989bfff640 1 -- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98a415cad0 msgr2=0x7f98a41f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.834+0000 7f989bfff640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98a415cad0 0x7f98a41f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.834+0000 7f989bfff640 1 -- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9890005180 msgr2=0x7f98a4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.834+0000 7f989bfff640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9890005180 0x7f98a4142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.834+0000 7f989bfff640 1 -- 172.21.15.70:0/3232969092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a40cdfc0 con 0x7f98a40c1970 2023-10-08T05:49:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.834+0000 7f989bfff640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a40c1970 0x7f98a415c590 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f988c00d7a0 tx=0x7f988c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.835+0000 7f989b7fe640 1 -- 172.21.15.70:0/3232969092 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0072d0 con 0x7f98a40c1970 2023-10-08T05:49:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.835+0000 7f989b7fe640 1 -- 172.21.15.70:0/3232969092 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c007430 con 0x7f98a40c1970 2023-10-08T05:49:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.835+0000 7f989b7fe640 1 -- 172.21.15.70:0/3232969092 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0047e0 con 0x7f98a40c1970 2023-10-08T05:49:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.835+0000 7f98a90da640 1 -- 172.21.15.70:0/3232969092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a40ce2a0 con 0x7f98a40c1970 2023-10-08T05:49:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.835+0000 7f98a90da640 1 -- 172.21.15.70:0/3232969092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a41f1ec0 con 0x7f98a40c1970 2023-10-08T05:49:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.836+0000 7f98a90da640 1 -- 172.21.15.70:0/3232969092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a40cdb40 con 0x7f98a40c1970 2023-10-08T05:49:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.837+0000 7f989b7fe640 1 -- 172.21.15.70:0/3232969092 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f988c004940 con 0x7f98a40c1970 2023-10-08T05:49:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.838+0000 7f989b7fe640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f988406b0a0 0x7f988406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:48.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.839+0000 7f989b7fe640 1 -- 172.21.15.70:0/3232969092 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6836+0+0 (secure 0 0 0) 0x7f988c0896b0 con 0x7f98a40c1970 2023-10-08T05:49:48.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.839+0000 7f98a2d9d640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f988406b0a0 0x7f988406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:48.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.841+0000 7f98a2d9d640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f988406b0a0 0x7f988406d560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f98a41451a0 tx=0x7f9894009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:48.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:48.844+0000 7f989b7fe640 1 -- 172.21.15.70:0/3232969092 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98a40cdb40 con 0x7f98a40c1970 2023-10-08T05:49:49.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.069+0000 7f98a90da640 1 -- 172.21.15.70:0/3232969092 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir8", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f98a40c1fd0 con 0x7f988406b0a0 2023-10-08T05:49:49.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.289+0000 7f989b7fe640 1 -- 172.21.15.70:0/3232969092 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f98a40c1fd0 con 0x7f988406b0a0 2023-10-08T05:49:49.291 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-10-08T05:49:49.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 -- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f988406b0a0 msgr2=0x7f988406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:49.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f988406b0a0 0x7f988406d560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f98a41451a0 tx=0x7f9894009040 comp rx=0 tx=0).stop 2023-10-08T05:49:49.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 -- 172.21.15.70:0/3232969092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a40c1970 msgr2=0x7f98a415c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:49.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a40c1970 0x7f98a415c590 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f988c00d7a0 tx=0x7f988c00dc60 comp rx=0 tx=0).stop 2023-10-08T05:49:49.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 -- 172.21.15.70:0/3232969092 shutdown_connections 2023-10-08T05:49:49.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f988406b0a0 0x7f988406d560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:49.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98a415cad0 0x7f98a41f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:49.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a40c1970 0x7f98a415c590 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:49.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 --2- 172.21.15.70:0/3232969092 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9890005180 0x7f98a4142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:49.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 -- 172.21.15.70:0/3232969092 >> 172.21.15.70:0/3232969092 conn(0x7f98a406a5d0 msgr2=0x7f98a40c7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:49.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 -- 172.21.15.70:0/3232969092 shutdown_connections 2023-10-08T05:49:49.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:49.293+0000 7f98a90da640 1 -- 172.21.15.70:0/3232969092 wait complete. 2023-10-08T05:49:49.318 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-10-08T05:49:49.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:49.319 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-08T05:49:49.367 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_01_UTC 2023-10-08T05:49:49.368 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:49.368 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:49.368 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:49.368 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:49.368 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:49.369 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:49.369 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:49.369 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:49.369 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:49.369 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:49.370 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_39_01_UTC 2023-10-08T05:49:49.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:49.371 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_39_01_UTC) 2023-10-08T05:49:49.495 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_40_00_UTC 2023-10-08T05:49:49.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:49.495 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_40_00_UTC) 2023-10-08T05:49:49.550 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_41_00_UTC 2023-10-08T05:49:49.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:49.551 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_41_00_UTC) 2023-10-08T05:49:49.619 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_42_00_UTC 2023-10-08T05:49:49.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:49.619 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_42_00_UTC) 2023-10-08T05:49:49.680 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:49.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:49.680 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_43_00_UTC) 2023-10-08T05:49:49.745 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:49.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:49.746 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_44_00_UTC) 2023-10-08T05:49:49.818 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:49.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:49.819 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_45_00_UTC) 2023-10-08T05:49:49.886 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:49.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:49.886 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_46_00_UTC) 2023-10-08T05:49:49.945 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:49.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:49.945 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_47_00_UTC) 2023-10-08T05:49:50.016 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:50.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:50.016 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_48_00_UTC) 2023-10-08T05:49:50.093 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:50.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:50.094 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-05_49_00_UTC) 2023-10-08T05:49:50.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:50.169 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-10-08T05:49:50.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir9 1m 2023-10-08T05:49:50.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.439+0000 7f9e23e7c640 1 Processor -- start 2023-10-08T05:49:50.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.440+0000 7f9e23e7c640 1 -- start start 2023-10-08T05:49:50.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.441+0000 7f9e23e7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e1c0cb150 0x7f9e1c0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:50.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.441+0000 7f9e23e7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0b1f20 0x7f9e1c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:50.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.441+0000 7f9e23e7c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1c06e1e0 con 0x7f9e1c161690 2023-10-08T05:49:50.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.441+0000 7f9e23e7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e1c0cbb40 con 0x7f9e1c0cb150 2023-10-08T05:49:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.441+0000 7f9e23e7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c06cc50 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.441+0000 7f9e21417640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e1c0cb150 0x7f9e1c0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.441+0000 7f9e21417640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e1c0cb150 0x7f9e1c0cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42328/0 (socket says 172.21.15.70:42328) 2023-10-08T05:49:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.441+0000 7f9e21417640 1 -- 172.21.15.70:0/1799564489 learned_addr learned my addr 172.21.15.70:0/1799564489 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.442+0000 7f9e22419640 1 --2- 172.21.15.70:0/1799564489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0b1f20 0x7f9e1c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.442+0000 7f9e21417640 1 -- 172.21.15.70:0/1799564489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0b1f20 msgr2=0x7f9e1c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.442+0000 7f9e21417640 1 --2- 172.21.15.70:0/1799564489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0b1f20 0x7f9e1c0b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.442+0000 7f9e21417640 1 -- 172.21.15.70:0/1799564489 >> v1:172.21.15.181:6789/0 conn(0x7f9e1c161690 legacy=0x7f9e1c06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.442+0000 7f9e21417640 1 -- 172.21.15.70:0/1799564489 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1c06cf30 con 0x7f9e1c0cb150 2023-10-08T05:49:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.442+0000 7f9e22419640 1 --2- 172.21.15.70:0/1799564489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0b1f20 0x7f9e1c0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.443+0000 7f9e21417640 1 --2- 172.21.15.70:0/1799564489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0cb150 0x7f9e1c0cb600 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c06e430 tx=0x7f9e0c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e530e87bc71e236d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.443+0000 7f9e20c16640 1 -- 172.21.15.70:0/1799564489 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c031830 con 0x7f9e1c0cb150 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.443+0000 7f9e20c16640 1 -- 172.21.15.70:0/1799564489 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c031990 con 0x7f9e1c0cb150 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.443+0000 7f9e20c16640 1 -- 172.21.15.70:0/1799564489 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c031d20 con 0x7f9e1c0cb150 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.444+0000 7f9e23e7c640 1 -- 172.21.15.70:0/1799564489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0cb150 msgr2=0x7f9e1c0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.444+0000 7f9e23e7c640 1 --2- 172.21.15.70:0/1799564489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0cb150 0x7f9e1c0cb600 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c06e430 tx=0x7f9e0c02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.444+0000 7f9e23e7c640 1 -- 172.21.15.70:0/1799564489 shutdown_connections 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.444+0000 7f9e23e7c640 1 --2- 172.21.15.70:0/1799564489 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0b1f20 0x7f9e1c0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.444+0000 7f9e23e7c640 1 --2- 172.21.15.70:0/1799564489 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0cb150 0x7f9e1c0cb600 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.444+0000 7f9e23e7c640 1 -- 172.21.15.70:0/1799564489 >> 172.21.15.70:0/1799564489 conn(0x7f9e1c06a5d0 msgr2=0x7f9e1c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.444+0000 7f9e23e7c640 1 -- 172.21.15.70:0/1799564489 shutdown_connections 2023-10-08T05:49:50.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.444+0000 7f9e23e7c640 1 -- 172.21.15.70:0/1799564489 wait complete. 2023-10-08T05:49:50.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.446+0000 7f9e23e7c640 1 Processor -- start 2023-10-08T05:49:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.446+0000 7f9e23e7c640 1 -- start start 2023-10-08T05:49:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.447+0000 7f9e23e7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0b1f20 0x7f9e1c0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.447+0000 7f9e23e7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0cb150 0x7f9e1c15f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.447+0000 7f9e23e7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e1c161690 0x7f9e1c0701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.447+0000 7f9e23e7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c0c2310 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.447+0000 7f9e23e7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c0c4bd0 con 0x7f9e1c161690 2023-10-08T05:49:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.447+0000 7f9e23e7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c0c4eb0 con 0x7f9e1c0cb150 2023-10-08T05:49:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.447+0000 7f9e21c18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0b1f20 0x7f9e1c0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e21c18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0b1f20 0x7f9e1c0c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42344/0 (socket says 172.21.15.70:42344) 2023-10-08T05:49:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e21c18640 1 -- 172.21.15.70:0/498963696 learned_addr learned my addr 172.21.15.70:0/498963696 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:50.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e22419640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e1c161690 0x7f9e1c0701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e21417640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0cb150 0x7f9e1c15f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e21c18640 1 -- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0cb150 msgr2=0x7f9e1c15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e21c18640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0cb150 0x7f9e1c15f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e21c18640 1 -- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e1c161690 msgr2=0x7f9e1c0701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e21c18640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e1c161690 0x7f9e1c0701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e21c18640 1 -- 172.21.15.70:0/498963696 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c0094d0 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e21417640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0cb150 0x7f9e1c15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.448+0000 7f9e21c18640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0b1f20 0x7f9e1c0c42d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800b3f0 tx=0x7f9e1800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.449+0000 7f9e127fc640 1 -- 172.21.15.70:0/498963696 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18011830 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.449+0000 7f9e127fc640 1 -- 172.21.15.70:0/498963696 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e18011990 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.449+0000 7f9e127fc640 1 -- 172.21.15.70:0/498963696 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18011d40 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.449+0000 7f9e23e7c640 1 -- 172.21.15.70:0/498963696 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1c0c2580 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.449+0000 7f9e23e7c640 1 -- 172.21.15.70:0/498963696 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1c0c4810 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.450+0000 7f9e23e7c640 1 -- 172.21.15.70:0/498963696 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1c04e160 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.451+0000 7f9e127fc640 1 -- 172.21.15.70:0/498963696 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f9e1801a400 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.452+0000 7f9e127fc640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9e0006b070 0x7f9e0006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:50.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.452+0000 7f9e127fc640 1 -- 172.21.15.70:0/498963696 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6916+0+0 (secure 0 0 0) 0x7f9e18089cb0 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.452+0000 7f9e21417640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9e0006b070 0x7f9e0006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:50.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.453+0000 7f9e21417640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9e0006b070 0x7f9e0006d530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c036000 tx=0x7f9e0c03c040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:50.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.456+0000 7f9e127fc640 1 -- 172.21.15.70:0/498963696 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e18054610 con 0x7f9e1c0b1f20 2023-10-08T05:49:50.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.697+0000 7f9e23e7c640 1 -- 172.21.15.70:0/498963696 --> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir9", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e1c06d4f0 con 0x7f9e0006b070 2023-10-08T05:49:50.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.903+0000 7f9e127fc640 1 -- 172.21.15.70:0/498963696 <== mgr.4936 v2:172.21.15.181:6832/2692398351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f9e0c031c80 con 0x7f9e0006b070 2023-10-08T05:49:50.904 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-10-08T05:49:50.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 -- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9e0006b070 msgr2=0x7f9e0006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:50.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9e0006b070 0x7f9e0006d530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c036000 tx=0x7f9e0c03c040 comp rx=0 tx=0).stop 2023-10-08T05:49:50.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 -- 172.21.15.70:0/498963696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0b1f20 msgr2=0x7f9e1c0c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:50.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0b1f20 0x7f9e1c0c42d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800b3f0 tx=0x7f9e1800b8b0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 -- 172.21.15.70:0/498963696 shutdown_connections 2023-10-08T05:49:50.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e1c161690 0x7f9e1c0701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f9e0006b070 0x7f9e0006d530 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e1c0cb150 0x7f9e1c15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 --2- 172.21.15.70:0/498963696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e1c0b1f20 0x7f9e1c0c42d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:50.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 -- 172.21.15.70:0/498963696 >> 172.21.15.70:0/498963696 conn(0x7f9e1c06a5d0 msgr2=0x7f9e1c0c7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:50.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 -- 172.21.15.70:0/498963696 shutdown_connections 2023-10-08T05:49:50.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:50.906+0000 7f9e23e7c640 1 -- 172.21.15.70:0/498963696 wait complete. 2023-10-08T05:49:50.936 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-10-08T05:49:50.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:50.936 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-08T05:49:50.983 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:50.983 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_40_01_UTC 2023-10-08T05:49:50.983 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_41_01_UTC 2023-10-08T05:49:50.983 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_42_01_UTC 2023-10-08T05:49:50.983 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:50.983 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:50.983 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:50.983 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:50.983 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:50.984 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:50.984 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:50.985 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_39_00_UTC 2023-10-08T05:49:50.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:50.985 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_39_00_UTC) 2023-10-08T05:49:51.096 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_40_01_UTC 2023-10-08T05:49:51.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.096 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_40_01_UTC) 2023-10-08T05:49:51.165 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_41_01_UTC 2023-10-08T05:49:51.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.166 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_41_01_UTC) 2023-10-08T05:49:51.238 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_42_01_UTC 2023-10-08T05:49:51.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.238 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_42_01_UTC) 2023-10-08T05:49:51.367 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_43_00_UTC 2023-10-08T05:49:51.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.367 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_43_00_UTC) 2023-10-08T05:49:51.419 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_44_00_UTC 2023-10-08T05:49:51.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.419 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_44_00_UTC) 2023-10-08T05:49:51.471 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_45_00_UTC 2023-10-08T05:49:51.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.471 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_45_00_UTC) 2023-10-08T05:49:51.530 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_46_00_UTC 2023-10-08T05:49:51.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.530 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_46_00_UTC) 2023-10-08T05:49:51.610 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_47_00_UTC 2023-10-08T05:49:51.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.611 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_47_00_UTC) 2023-10-08T05:49:51.676 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_48_00_UTC 2023-10-08T05:49:51.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.676 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_48_00_UTC) 2023-10-08T05:49:51.731 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_49_00_UTC 2023-10-08T05:49:51.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.731 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-05_49_00_UTC) 2023-10-08T05:49:51.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:51.790 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-10-08T05:49:51.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T05:49:52.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.069+0000 7f01e265d640 1 Processor -- start 2023-10-08T05:49:52.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.070+0000 7f01e265d640 1 -- start start 2023-10-08T05:49:52.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.072+0000 7f01e265d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f01dc146d10 0x7f01dc162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:52.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.072+0000 7f01e265d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01dc06c9d0 con 0x7f01dc146930 2023-10-08T05:49:52.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.072+0000 7f01e265d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01dc1652e0 con 0x7f01dc144020 2023-10-08T05:49:52.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.072+0000 7f01e265d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01dc165b80 con 0x7f01dc146d10 2023-10-08T05:49:52.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.073+0000 7f01db7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f01dc146d10 0x7f01dc162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:52.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.073+0000 7f01db7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f01dc146d10 0x7f01dc162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42348/0 (socket says 172.21.15.70:42348) 2023-10-08T05:49:52.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.073+0000 7f01db7fe640 1 -- 172.21.15.70:0/1558661076 learned_addr learned my addr 172.21.15.70:0/1558661076 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:52.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.073+0000 7f01db7fe640 1 -- 172.21.15.70:0/1558661076 >> v1:172.21.15.181:6789/0 conn(0x7f01dc144020 legacy=0x7f01dc06ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:52.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.073+0000 7f01db7fe640 1 -- 172.21.15.70:0/1558661076 >> v1:172.21.15.70:6789/0 conn(0x7f01dc146930 legacy=0x7f01dc143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:52.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.073+0000 7f01db7fe640 1 -- 172.21.15.70:0/1558661076 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc1655c0 con 0x7f01dc146d10 2023-10-08T05:49:52.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.073+0000 7f01daffd640 1 -- 172.21.15.70:0/1558661076 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249736796 0 0) 0x7f01dc06c9d0 con 0x7f01dc146930 2023-10-08T05:49:52.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.075+0000 7f01db7fe640 1 --2- 172.21.15.70:0/1558661076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146d10 0x7f01dc162000 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009870 tx=0x7f01cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9367bcd6b1a0bc3f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:52.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.075+0000 7f01daffd640 1 -- 172.21.15.70:0/1558661076 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc030860 con 0x7f01dc146d10 2023-10-08T05:49:52.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.076+0000 7f01daffd640 1 -- 172.21.15.70:0/1558661076 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01cc0309c0 con 0x7f01dc146d10 2023-10-08T05:49:52.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.076+0000 7f01daffd640 1 -- 172.21.15.70:0/1558661076 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc030d50 con 0x7f01dc146d10 2023-10-08T05:49:52.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.077+0000 7f01e265d640 1 -- 172.21.15.70:0/1558661076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146d10 msgr2=0x7f01dc162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:52.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.077+0000 7f01e265d640 1 --2- 172.21.15.70:0/1558661076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146d10 0x7f01dc162000 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009870 tx=0x7f01cc02b9a0 comp rx=0 tx=0).stop 2023-10-08T05:49:52.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.077+0000 7f01e265d640 1 -- 172.21.15.70:0/1558661076 shutdown_connections 2023-10-08T05:49:52.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.077+0000 7f01e265d640 1 --2- 172.21.15.70:0/1558661076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146d10 0x7f01dc162000 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:52.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.077+0000 7f01e265d640 1 -- 172.21.15.70:0/1558661076 >> 172.21.15.70:0/1558661076 conn(0x7f01dc069d50 msgr2=0x7f01dc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:52.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.077+0000 7f01e265d640 1 -- 172.21.15.70:0/1558661076 shutdown_connections 2023-10-08T05:49:52.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.077+0000 7f01e265d640 1 -- 172.21.15.70:0/1558661076 wait complete. 2023-10-08T05:49:52.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.080+0000 7f01e265d640 1 Processor -- start 2023-10-08T05:49:52.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.080+0000 7f01e265d640 1 -- start start 2023-10-08T05:49:52.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.081+0000 7f01e265d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01dc144020 0x7f01dc0ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:52.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.082+0000 7f01e265d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146930 0x7f01dc15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:52.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.082+0000 7f01e265d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01dc146d10 0x7f01dc0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:52.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.082+0000 7f01e265d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc1ff380 con 0x7f01dc146930 2023-10-08T05:49:52.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.082+0000 7f01e265d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc15b3a0 con 0x7f01dc144020 2023-10-08T05:49:52.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.082+0000 7f01e265d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc1fefe0 con 0x7f01dc146d10 2023-10-08T05:49:52.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.082+0000 7f01db7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146930 0x7f01dc15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:52.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.082+0000 7f01db7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146930 0x7f01dc15ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42358/0 (socket says 172.21.15.70:42358) 2023-10-08T05:49:52.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.082+0000 7f01db7fe640 1 -- 172.21.15.70:0/4273933216 learned_addr learned my addr 172.21.15.70:0/4273933216 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:52.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.082+0000 7f01dbfff640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01dc144020 0x7f01dc0ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:52.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.083+0000 7f01db7fe640 1 -- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01dc146d10 msgr2=0x7f01dc0b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.083+0000 7f01db7fe640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01dc146d10 0x7f01dc0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.083+0000 7f01db7fe640 1 -- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01dc144020 msgr2=0x7f01dc0ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.083+0000 7f01db7fe640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01dc144020 0x7f01dc0ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.083+0000 7f01db7fe640 1 -- 172.21.15.70:0/4273933216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc0094d0 con 0x7f01dc146930 2023-10-08T05:49:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.084+0000 7f01db7fe640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146930 0x7f01dc15ae60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0090b0 tx=0x7f01cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.084+0000 7f01d8ff9640 1 -- 172.21.15.70:0/4273933216 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc03e070 con 0x7f01dc146930 2023-10-08T05:49:52.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.084+0000 7f01d8ff9640 1 -- 172.21.15.70:0/4273933216 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01cc0398d0 con 0x7f01dc146930 2023-10-08T05:49:52.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.084+0000 7f01d8ff9640 1 -- 172.21.15.70:0/4273933216 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc039c80 con 0x7f01dc146930 2023-10-08T05:49:52.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.084+0000 7f01e265d640 1 -- 172.21.15.70:0/4273933216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01dc0cafc0 con 0x7f01dc146930 2023-10-08T05:49:52.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.084+0000 7f01e265d640 1 -- 172.21.15.70:0/4273933216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01dc0cb580 con 0x7f01dc146930 2023-10-08T05:49:52.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.086+0000 7f01e265d640 1 -- 172.21.15.70:0/4273933216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01dc1ff4c0 con 0x7f01dc146930 2023-10-08T05:49:52.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.087+0000 7f01d8ff9640 1 -- 172.21.15.70:0/4273933216 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 13) v1 ==== 89417+0+0 (secure 0 0 0) 0x7f01cc042070 con 0x7f01dc146930 2023-10-08T05:49:52.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.087+0000 7f01d8ff9640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f01c806b0c0 0x7f01c806d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:52.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.087+0000 7f01d8ff9640 1 -- 172.21.15.70:0/4273933216 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 7028+0+0 (secure 0 0 0) 0x7f01cc0abf70 con 0x7f01dc146930 2023-10-08T05:49:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.088+0000 7f01dbfff640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f01c806b0c0 0x7f01c806d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.089+0000 7f01dbfff640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f01c806b0c0 0x7f01c806d580 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01d40012e0 tx=0x7f01d4009040 comp rx=0 tx=0).ready entity=mgr.4936 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:52.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.091+0000 7f01d8ff9640 1 -- 172.21.15.70:0/4273933216 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01cc0deca0 con 0x7f01dc146930 2023-10-08T05:49:52.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:52.341+0000 7f01e265d640 1 -- 172.21.15.70:0/4273933216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f01dc04e160 con 0x7f01dc146930 2023-10-08T05:49:53.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.310+0000 7f01d8ff9640 1 -- 172.21.15.70:0/4273933216 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v14) v1 ==== 93+0+0 (secure 0 0 0) 0x7f01cc0340a0 con 0x7f01dc146930 2023-10-08T05:49:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.317+0000 7f01e265d640 1 -- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f01c806b0c0 msgr2=0x7f01c806d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.317+0000 7f01e265d640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f01c806b0c0 0x7f01c806d580 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01d40012e0 tx=0x7f01d4009040 comp rx=0 tx=0).stop 2023-10-08T05:49:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.317+0000 7f01e265d640 1 -- 172.21.15.70:0/4273933216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146930 msgr2=0x7f01dc15ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.317+0000 7f01e265d640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146930 0x7f01dc15ae60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0090b0 tx=0x7f01cc0046c0 comp rx=0 tx=0).stop 2023-10-08T05:49:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.317+0000 7f01e265d640 1 -- 172.21.15.70:0/4273933216 shutdown_connections 2023-10-08T05:49:53.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.317+0000 7f01e265d640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f01c806b0c0 0x7f01c806d580 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f01d40012e0 tx=0x7f01d4009040 comp rx=0 tx=0).stop 2023-10-08T05:49:53.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.317+0000 7f01e265d640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f01dc146d10 0x7f01dc0b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:53.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.317+0000 7f01e265d640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01dc146930 0x7f01dc15ae60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:53.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.317+0000 7f01e265d640 1 --2- 172.21.15.70:0/4273933216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f01dc144020 0x7f01dc0ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:53.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.317+0000 7f01e265d640 1 -- 172.21.15.70:0/4273933216 >> 172.21.15.70:0/4273933216 conn(0x7f01dc069d50 msgr2=0x7f01dc0cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:53.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.319+0000 7f01e265d640 1 -- 172.21.15.70:0/4273933216 shutdown_connections 2023-10-08T05:49:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.320+0000 7f01e265d640 1 -- 172.21.15.70:0/4273933216 wait complete. 2023-10-08T05:49:53.349 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:49:53.349 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:49:53.458 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:49:53.545 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:49:53.546 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:49:53.546 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:49:53.594 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:49:53.595 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:49:53.681 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:49:53.877 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:53.876+0000 7f769f64c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:49:53.888 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:53.887+0000 7fb0bfd53040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:49:53.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.929+0000 7ffa884c9640 1 Processor -- start 2023-10-08T05:49:53.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.929+0000 7ffa884c9640 1 -- start start 2023-10-08T05:49:53.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.930+0000 7ffa884c9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa80159ac0 0x7ffa8006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:53.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.930+0000 7ffa884c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa800709e0 0x7ffa800c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:53.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.930+0000 7ffa884c9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8006dee0 con 0x7ffa800ad790 2023-10-08T05:49:53.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.930+0000 7ffa884c9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8015d850 con 0x7ffa800709e0 2023-10-08T05:49:53.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.930+0000 7ffa884c9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8015db30 con 0x7ffa80159ac0 2023-10-08T05:49:53.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.931+0000 7ffa86a66640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa800709e0 0x7ffa800c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:53.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.931+0000 7ffa85a64640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ffa800ad790 0x7ffa800702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49766/0 (socket says 172.21.15.70:49766) 2023-10-08T05:49:53.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.931+0000 7ffa85a64640 1 -- 172.21.15.70:0/4031108179 learned_addr learned my addr 172.21.15.70:0/4031108179 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:53.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.931+0000 7ffa86265640 1 --2- 172.21.15.70:0/4031108179 >> v2:172.21.15.181:3300/0 conn(0x7ffa80159ac0 0x7ffa8006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:53.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.931+0000 7ffa86a66640 1 -- 172.21.15.70:0/4031108179 >> v2:172.21.15.181:3300/0 conn(0x7ffa80159ac0 msgr2=0x7ffa8006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:53.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.931+0000 7ffa86a66640 1 --2- 172.21.15.70:0/4031108179 >> v2:172.21.15.181:3300/0 conn(0x7ffa80159ac0 0x7ffa8006d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:53.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.931+0000 7ffa86a66640 1 -- 172.21.15.70:0/4031108179 >> v1:172.21.15.70:6789/0 conn(0x7ffa800ad790 legacy=0x7ffa800702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:53.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.931+0000 7ffa85263640 1 -- 172.21.15.70:0/4031108179 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841764745 0 0) 0x7ffa8006dee0 con 0x7ffa800ad790 2023-10-08T05:49:53.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.931+0000 7ffa86a66640 1 -- 172.21.15.70:0/4031108179 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8015de10 con 0x7ffa800709e0 2023-10-08T05:49:53.932 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:53.932+0000 7f769f64c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:49:53.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.933+0000 7ffa86a66640 1 --2- 172.21.15.70:0/4031108179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 0x7ffa800c8dd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffa8006e790 tx=0x7ffa7802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f4507f89d5566 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:53.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.933+0000 7ffa85263640 1 -- 172.21.15.70:0/4031108179 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7802cc80 con 0x7ffa800709e0 2023-10-08T05:49:53.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.933+0000 7ffa85263640 1 -- 172.21.15.70:0/4031108179 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7802cde0 con 0x7ffa800709e0 2023-10-08T05:49:53.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.933+0000 7ffa85263640 1 -- 172.21.15.70:0/4031108179 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78034660 con 0x7ffa800709e0 2023-10-08T05:49:53.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.934+0000 7ffa884c9640 1 -- 172.21.15.70:0/4031108179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 msgr2=0x7ffa800c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:53.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.934+0000 7ffa884c9640 1 --2- 172.21.15.70:0/4031108179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 0x7ffa800c8dd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffa8006e790 tx=0x7ffa7802bed0 comp rx=0 tx=0).stop 2023-10-08T05:49:53.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.934+0000 7ffa884c9640 1 -- 172.21.15.70:0/4031108179 shutdown_connections 2023-10-08T05:49:53.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.934+0000 7ffa884c9640 1 --2- 172.21.15.70:0/4031108179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 0x7ffa800c8dd0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:53.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.934+0000 7ffa884c9640 1 --2- 172.21.15.70:0/4031108179 >> v2:172.21.15.181:3300/0 conn(0x7ffa80159ac0 0x7ffa8006d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:53.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.934+0000 7ffa884c9640 1 -- 172.21.15.70:0/4031108179 >> 172.21.15.70:0/4031108179 conn(0x7ffa8006a380 msgr2=0x7ffa800b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:53.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.934+0000 7ffa884c9640 1 -- 172.21.15.70:0/4031108179 shutdown_connections 2023-10-08T05:49:53.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.934+0000 7ffa884c9640 1 -- 172.21.15.70:0/4031108179 wait complete. 2023-10-08T05:49:53.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.936+0000 7ffa884c9640 1 Processor -- start 2023-10-08T05:49:53.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.936+0000 7ffa884c9640 1 -- start start 2023-10-08T05:49:53.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa884c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 0x7ffa80153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:53.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa884c9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa800ad790 0x7ffa80155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:53.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa884c9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa80159ac0 0x7ffa801f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:53.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa884c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80155e90 con 0x7ffa800709e0 2023-10-08T05:49:53.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa884c9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80156170 con 0x7ffa800ad790 2023-10-08T05:49:53.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa884c9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa80156450 con 0x7ffa80159ac0 2023-10-08T05:49:53.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa86265640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 0x7ffa80153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:53.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa86265640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 0x7ffa80153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50458/0 (socket says 172.21.15.70:50458) 2023-10-08T05:49:53.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa86265640 1 -- 172.21.15.70:0/2952769676 learned_addr learned my addr 172.21.15.70:0/2952769676 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:53.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa86a66640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa80159ac0 0x7ffa801f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:53.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa85a64640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa800ad790 0x7ffa80155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:53.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.937+0000 7ffa86265640 1 -- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa80159ac0 msgr2=0x7ffa801f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:53.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa86265640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa80159ac0 0x7ffa801f1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:53.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa86265640 1 -- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa800ad790 msgr2=0x7ffa80155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:53.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa86265640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa800ad790 0x7ffa80155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:53.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa86265640 1 -- 172.21.15.70:0/2952769676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa78009a70 con 0x7ffa800709e0 2023-10-08T05:49:53.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa85a64640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa800ad790 0x7ffa80155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:53.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa86265640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 0x7ffa80153cf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00d5d0 tx=0x7ffa7c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:53.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa76ffd640 1 -- 172.21.15.70:0/2952769676 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0125f0 con 0x7ffa800709e0 2023-10-08T05:49:53.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa76ffd640 1 -- 172.21.15.70:0/2952769676 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c012770 con 0x7ffa800709e0 2023-10-08T05:49:53.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa76ffd640 1 -- 172.21.15.70:0/2952769676 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c012b20 con 0x7ffa800709e0 2023-10-08T05:49:53.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa884c9640 1 -- 172.21.15.70:0/2952769676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa80156730 con 0x7ffa800709e0 2023-10-08T05:49:53.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.938+0000 7ffa884c9640 1 -- 172.21.15.70:0/2952769676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa80155b60 con 0x7ffa800709e0 2023-10-08T05:49:53.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.939+0000 7ffa884c9640 1 -- 172.21.15.70:0/2952769676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8015e330 con 0x7ffa800709e0 2023-10-08T05:49:53.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.940+0000 7ffa76ffd640 1 -- 172.21.15.70:0/2952769676 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7ffa7c012c80 con 0x7ffa800709e0 2023-10-08T05:49:53.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.941+0000 7ffa76ffd640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffa6406b050 0x7ffa6406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:53.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.941+0000 7ffa76ffd640 1 -- 172.21.15.70:0/2952769676 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6836+0+0 (secure 0 0 0) 0x7ffa7c08a2e0 con 0x7ffa800709e0 2023-10-08T05:49:53.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.941+0000 7ffa85a64640 1 -- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffa6406b050 msgr2=0x7ffa6406d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:53.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.941+0000 7ffa85a64640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffa6406b050 0x7ffa6406d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:49:53.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:53.946+0000 7ffa76ffd640 1 -- 172.21.15.70:0/2952769676 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa7c054c90 con 0x7ffa800709e0 2023-10-08T05:49:53.946 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:53.946+0000 7fb0bfd53040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:49:54.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.134+0000 7ffa884c9640 1 -- 172.21.15.70:0/2952769676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa80067490 con 0x7ffa800709e0 2023-10-08T05:49:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.136+0000 7ffa76ffd640 1 -- 172.21.15.70:0/2952769676 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+18004 (secure 0 0 0) 0x7ffa7c0582b0 con 0x7ffa800709e0 2023-10-08T05:49:54.137 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:54.137 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":57,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:49:53.247400+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:37:41.960853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-08T05:37:43.190325+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/2737793492":"2023-10-09T05:38:04.171728+0000","172.21.15.181:6832/2737793492":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3280597969":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3620192672":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3583094356":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3573082646":"2023-10-09T05:38:04.171728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":9,"length":1},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":24,"length":1},{"begin":30,"length":1},{"begin":38,"length":2},{"begin":45,"length":3},{"begin":53,"length":2},{"begin":57,"length":1},{"begin":62,"length":1},{"begin":64,"length":1},{"begin":66,"length":1},{"begin":73,"length":2},{"begin":79,"length":1},{"begin":86,"length":1},{"begin":88,"length":2},{"begin":94,"length":1},{"begin":96,"length":1},{"begin":99,"length":1},{"begin":107,"length":2},{"begin":193,"length":28}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":30,"length":1},{"begin":39,"length":1},{"begin":47,"length":1},{"begin":57,"length":1},{"begin":62,"length":1},{"begin":74,"length":1},{"begin":88,"length":1},{"begin":96,"length":1},{"begin":213,"length":8}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:49:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.138+0000 7ffa884c9640 1 -- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffa6406b050 msgr2=0x7ffa6406d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:49:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.138+0000 7ffa884c9640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffa6406b050 0x7ffa6406d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.138+0000 7ffa884c9640 1 -- 172.21.15.70:0/2952769676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 msgr2=0x7ffa80153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.138+0000 7ffa884c9640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 0x7ffa80153cf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00d5d0 tx=0x7ffa7c00da90 comp rx=0 tx=0).stop 2023-10-08T05:49:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.138+0000 7ffa884c9640 1 -- 172.21.15.70:0/2952769676 shutdown_connections 2023-10-08T05:49:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.138+0000 7ffa884c9640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7ffa6406b050 0x7ffa6406d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.138+0000 7ffa884c9640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa80159ac0 0x7ffa801f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.138+0000 7ffa884c9640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa800ad790 0x7ffa80155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.138+0000 7ffa884c9640 1 --2- 172.21.15.70:0/2952769676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa800709e0 0x7ffa80153cf0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.138+0000 7ffa884c9640 1 -- 172.21.15.70:0/2952769676 >> 172.21.15.70:0/2952769676 conn(0x7ffa8006a380 msgr2=0x7ffa800ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.139+0000 7ffa884c9640 1 -- 172.21.15.70:0/2952769676 shutdown_connections 2023-10-08T05:49:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.139+0000 7ffa884c9640 1 -- 172.21.15.70:0/2952769676 wait complete. 2023-10-08T05:49:54.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:37:41.960853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:37:43.190325+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 57, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:49:54.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:49:54.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.353+0000 7f56e1d39640 1 Processor -- start 2023-10-08T05:49:54.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.353+0000 7f56e1d39640 1 -- start start 2023-10-08T05:49:54.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.354+0000 7f56e1d39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc1432f0 0x7f56dc0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.354+0000 7f56e1d39640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56dc0b4240 0x7f56dc159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.354+0000 7f56e1d39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56dc0cbb30 con 0x7f56dc0cb6b0 2023-10-08T05:49:54.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.354+0000 7f56e1d39640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56dc1436d0 con 0x7f56dc0b4240 2023-10-08T05:49:54.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.354+0000 7f56e1d39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc15d440 con 0x7f56dc1432f0 2023-10-08T05:49:54.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.355+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc1432f0 0x7f56dc0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.355+0000 7f56dbfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56dc0b4240 0x7f56dc159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.355+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc1432f0 0x7f56dc0cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:52882/0 (socket says 172.21.15.70:52882) 2023-10-08T05:49:54.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.355+0000 7f56db7fe640 1 -- 172.21.15.70:0/2654012680 learned_addr learned my addr 172.21.15.70:0/2654012680 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:54.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.355+0000 7f56d3fff640 1 --1- 172.21.15.70:0/2654012680 >> v1:172.21.15.181:6789/0 conn(0x7f56dc0cb6b0 0x7f56dc0b3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:59868/0 (socket says 172.21.15.70:59868) 2023-10-08T05:49:54.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.356+0000 7f56daffd640 1 -- 172.21.15.70:0/2654012680 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271994017 0 0) 0x7f56dc0cbb30 con 0x7f56dc0cb6b0 2023-10-08T05:49:54.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.356+0000 7f56daffd640 1 -- 172.21.15.70:0/2654012680 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56c8003520 con 0x7f56dc0cb6b0 2023-10-08T05:49:54.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.356+0000 7f56db7fe640 1 -- 172.21.15.70:0/2654012680 >> v2:172.21.15.181:3300/0 conn(0x7f56dc0b4240 msgr2=0x7f56dc159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:54.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.356+0000 7f56db7fe640 1 --2- 172.21.15.70:0/2654012680 >> v2:172.21.15.181:3300/0 conn(0x7f56dc0b4240 0x7f56dc159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.356+0000 7f56db7fe640 1 -- 172.21.15.70:0/2654012680 >> v1:172.21.15.181:6789/0 conn(0x7f56dc0cb6b0 legacy=0x7f56dc0b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:54.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.356+0000 7f56db7fe640 1 -- 172.21.15.70:0/2654012680 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc15d690 con 0x7f56dc1432f0 2023-10-08T05:49:54.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.357+0000 7f56db7fe640 1 --2- 172.21.15.70:0/2654012680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc1432f0 0x7f56dc0cb170 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009870 tx=0x7f56cc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a855d5a696d125 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:54.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.357+0000 7f56daffd640 1 -- 172.21.15.70:0/2654012680 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc031640 con 0x7f56dc1432f0 2023-10-08T05:49:54.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.357+0000 7f56daffd640 1 -- 172.21.15.70:0/2654012680 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc0317a0 con 0x7f56dc1432f0 2023-10-08T05:49:54.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.357+0000 7f56daffd640 1 -- 172.21.15.70:0/2654012680 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc031b10 con 0x7f56dc1432f0 2023-10-08T05:49:54.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.358+0000 7f56e1d39640 1 -- 172.21.15.70:0/2654012680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc1432f0 msgr2=0x7f56dc0cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:54.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.358+0000 7f56e1d39640 1 --2- 172.21.15.70:0/2654012680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc1432f0 0x7f56dc0cb170 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009870 tx=0x7f56cc02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.358+0000 7f56e1d39640 1 -- 172.21.15.70:0/2654012680 shutdown_connections 2023-10-08T05:49:54.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.358+0000 7f56e1d39640 1 --2- 172.21.15.70:0/2654012680 >> v2:172.21.15.181:3300/0 conn(0x7f56dc0b4240 0x7f56dc159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.358+0000 7f56e1d39640 1 --2- 172.21.15.70:0/2654012680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc1432f0 0x7f56dc0cb170 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.358+0000 7f56e1d39640 1 -- 172.21.15.70:0/2654012680 >> 172.21.15.70:0/2654012680 conn(0x7f56dc069d50 msgr2=0x7f56dc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:54.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.358+0000 7f56e1d39640 1 -- 172.21.15.70:0/2654012680 shutdown_connections 2023-10-08T05:49:54.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.358+0000 7f56e1d39640 1 -- 172.21.15.70:0/2654012680 wait complete. 2023-10-08T05:49:54.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.360+0000 7f56e1d39640 1 Processor -- start 2023-10-08T05:49:54.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.360+0000 7f56e1d39640 1 -- start start 2023-10-08T05:49:54.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.361+0000 7f56e1d39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56dc0b4240 0x7f56dc1edb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.361+0000 7f56e1d39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc0cb6b0 0x7f56dc1406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.361+0000 7f56e1d39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc1432f0 0x7f56dc13cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.361+0000 7f56e1d39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc1ee050 con 0x7f56dc0b4240 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.361+0000 7f56e1d39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc1413e0 con 0x7f56dc1432f0 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.361+0000 7f56e1d39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc1416c0 con 0x7f56dc0cb6b0 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.361+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56dc0b4240 0x7f56dc1edb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.361+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56dc0b4240 0x7f56dc1edb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50466/0 (socket says 172.21.15.70:50466) 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.361+0000 7f56db7fe640 1 -- 172.21.15.70:0/1867119088 learned_addr learned my addr 172.21.15.70:0/1867119088 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.361+0000 7f56d3fff640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc0cb6b0 0x7f56dc1406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56dbfff640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc1432f0 0x7f56dc13cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56db7fe640 1 -- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc0cb6b0 msgr2=0x7f56dc1406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56db7fe640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc0cb6b0 0x7f56dc1406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56db7fe640 1 -- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc1432f0 msgr2=0x7f56dc13cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56db7fe640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc1432f0 0x7f56dc13cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56db7fe640 1 -- 172.21.15.70:0/1867119088 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc0094d0 con 0x7f56dc0b4240 2023-10-08T05:49:54.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56dbfff640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc1432f0 0x7f56dc13cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:54.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56db7fe640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56dc0b4240 0x7f56dc1edb10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f56cc000c00 tx=0x7f56cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:54.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56d8ff9640 1 -- 172.21.15.70:0/1867119088 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc03a8b0 con 0x7f56dc0b4240 2023-10-08T05:49:54.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56d8ff9640 1 -- 172.21.15.70:0/1867119088 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc03aa10 con 0x7f56dc0b4240 2023-10-08T05:49:54.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56d8ff9640 1 -- 172.21.15.70:0/1867119088 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc03ada0 con 0x7f56dc0b4240 2023-10-08T05:49:54.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.362+0000 7f56e1d39640 1 -- 172.21.15.70:0/1867119088 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56dc1419a0 con 0x7f56dc0b4240 2023-10-08T05:49:54.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.363+0000 7f56e1d39640 1 -- 172.21.15.70:0/1867119088 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56dc141f60 con 0x7f56dc0b4240 2023-10-08T05:49:54.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.364+0000 7f56e1d39640 1 -- 172.21.15.70:0/1867119088 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56dc04e160 con 0x7f56dc0b4240 2023-10-08T05:49:54.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.365+0000 7f56d8ff9640 1 -- 172.21.15.70:0/1867119088 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f56cc02f050 con 0x7f56dc0b4240 2023-10-08T05:49:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.365+0000 7f56d8ff9640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f56bc06b050 0x7f56bc06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.366+0000 7f56d8ff9640 1 -- 172.21.15.70:0/1867119088 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6836+0+0 (secure 0 0 0) 0x7f56cc0abbd0 con 0x7f56dc0b4240 2023-10-08T05:49:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.366+0000 7f56d3fff640 1 -- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f56bc06b050 msgr2=0x7f56bc06d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.366+0000 7f56d3fff640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f56bc06b050 0x7f56bc06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:49:54.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.370+0000 7f56d8ff9640 1 -- 172.21.15.70:0/1867119088 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56cc076470 con 0x7f56dc0b4240 2023-10-08T05:49:54.402 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:54.401+0000 7fb0bfd53040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:49:54.402 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:54.402+0000 7f769f64c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:49:54.453 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:54.452+0000 7fb0bfd53040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:49:54.554 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:54.502+0000 7fb0bfd53040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:49:54.554 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:54.455+0000 7f769f64c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:49:54.554 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:54.505+0000 7f769f64c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:49:54.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.557+0000 7f56e1d39640 1 -- 172.21.15.70:0/1867119088 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56dc13a7c0 con 0x7f56dc0b4240 2023-10-08T05:49:54.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.558+0000 7f56d8ff9640 1 -- 172.21.15.70:0/1867119088 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56cc079a90 con 0x7f56dc0b4240 2023-10-08T05:49:54.559 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:49:54.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 -- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f56bc06b050 msgr2=0x7f56bc06d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:49:54.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f56bc06b050 0x7f56bc06d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 -- 172.21.15.70:0/1867119088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56dc0b4240 msgr2=0x7f56dc1edb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:54.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56dc0b4240 0x7f56dc1edb10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f56cc000c00 tx=0x7f56cc0046c0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 -- 172.21.15.70:0/1867119088 shutdown_connections 2023-10-08T05:49:54.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56dc1432f0 0x7f56dc13cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f56bc06b050 0x7f56bc06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56dc0cb6b0 0x7f56dc1406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 --2- 172.21.15.70:0/1867119088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56dc0b4240 0x7f56dc1edb10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 -- 172.21.15.70:0/1867119088 >> 172.21.15.70:0/1867119088 conn(0x7f56dc069d50 msgr2=0x7f56dc0ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:54.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 -- 172.21.15.70:0/1867119088 shutdown_connections 2023-10-08T05:49:54.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.560+0000 7f56e1d39640 1 -- 172.21.15.70:0/1867119088 wait complete. 2023-10-08T05:49:54.565 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:54.562+0000 7fb0bfd53040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:49:54.568 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:54.568+0000 7f769f64c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:49:54.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:49:54.612 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:54.611+0000 7fb0bfd53040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:49:54.620 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:54.620+0000 7f769f64c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:49:54.733 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:54.720+0000 7fb0bfd53040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:49:54.737 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:54.736+0000 7f769f64c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:49:54.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.779+0000 7f467e81a640 1 Processor -- start 2023-10-08T05:49:54.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.779+0000 7f467e81a640 1 -- start start 2023-10-08T05:49:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.781+0000 7f467e81a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46780afbe0 0x7f46780b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.781+0000 7f467e81a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f467806dfc0 0x7f4678070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.781+0000 7f467e81a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4678071070 0x7f46780cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.781+0000 7f467e81a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46780b2620 con 0x7f467806dfc0 2023-10-08T05:49:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.781+0000 7f467e81a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46780b2060 con 0x7f4678071070 2023-10-08T05:49:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.781+0000 7f467e81a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46780b2340 con 0x7f46780afbe0 2023-10-08T05:49:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.781+0000 7f466ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f467806dfc0 0x7f4678070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.781+0000 7f466ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f467806dfc0 0x7f4678070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50474/0 (socket says 172.21.15.70:50474) 2023-10-08T05:49:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.781+0000 7f466ffff640 1 -- 172.21.15.70:0/266243142 learned_addr learned my addr 172.21.15.70:0/266243142 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.781+0000 7f4677fff640 1 --2- 172.21.15.70:0/266243142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46780afbe0 0x7f46780b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.782+0000 7f467cdb7640 1 --2- 172.21.15.70:0/266243142 >> v2:172.21.15.181:3300/0 conn(0x7f4678071070 0x7f46780cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.782+0000 7f466ffff640 1 -- 172.21.15.70:0/266243142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46780afbe0 msgr2=0x7f46780b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.782+0000 7f466ffff640 1 --2- 172.21.15.70:0/266243142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46780afbe0 0x7f46780b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.782+0000 7f466ffff640 1 -- 172.21.15.70:0/266243142 >> v2:172.21.15.181:3300/0 conn(0x7f4678071070 msgr2=0x7f46780cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.782+0000 7f466ffff640 1 --2- 172.21.15.70:0/266243142 >> v2:172.21.15.181:3300/0 conn(0x7f4678071070 0x7f46780cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.782+0000 7f466ffff640 1 -- 172.21.15.70:0/266243142 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4678070350 con 0x7f467806dfc0 2023-10-08T05:49:54.784 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:54.783+0000 7fb0bfd53040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:49:54.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.784+0000 7f466ffff640 1 --2- 172.21.15.70:0/266243142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467806dfc0 0x7f4678070b30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f46780b3980 tx=0x7f466002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33462377b66d6095 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:54.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.784+0000 7f46777fe640 1 -- 172.21.15.70:0/266243142 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660031860 con 0x7f467806dfc0 2023-10-08T05:49:54.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.784+0000 7f46777fe640 1 -- 172.21.15.70:0/266243142 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46600319c0 con 0x7f467806dfc0 2023-10-08T05:49:54.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.784+0000 7f46777fe640 1 -- 172.21.15.70:0/266243142 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660031d50 con 0x7f467806dfc0 2023-10-08T05:49:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.785+0000 7f467e81a640 1 -- 172.21.15.70:0/266243142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467806dfc0 msgr2=0x7f4678070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.785+0000 7f467e81a640 1 --2- 172.21.15.70:0/266243142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467806dfc0 0x7f4678070b30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f46780b3980 tx=0x7f466002b900 comp rx=0 tx=0).stop 2023-10-08T05:49:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.785+0000 7f467e81a640 1 -- 172.21.15.70:0/266243142 shutdown_connections 2023-10-08T05:49:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.785+0000 7f467e81a640 1 --2- 172.21.15.70:0/266243142 >> v2:172.21.15.181:3300/0 conn(0x7f4678071070 0x7f46780cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.785+0000 7f467e81a640 1 --2- 172.21.15.70:0/266243142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467806dfc0 0x7f4678070b30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.785+0000 7f467e81a640 1 --2- 172.21.15.70:0/266243142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46780afbe0 0x7f46780b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.785+0000 7f467e81a640 1 -- 172.21.15.70:0/266243142 >> 172.21.15.70:0/266243142 conn(0x7f467806a5d0 msgr2=0x7f46780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.785+0000 7f467e81a640 1 -- 172.21.15.70:0/266243142 shutdown_connections 2023-10-08T05:49:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.785+0000 7f467e81a640 1 -- 172.21.15.70:0/266243142 wait complete. 2023-10-08T05:49:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.787+0000 7f467e81a640 1 Processor -- start 2023-10-08T05:49:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.787+0000 7f467e81a640 1 -- start start 2023-10-08T05:49:54.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f467e81a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467806dfc0 0x7f467814d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f467e81a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4678071070 0x7f467815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f467e81a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46780afbe0 0x7f4678150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f467e81a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46780c2250 con 0x7f4678071070 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f467e81a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46780c24f0 con 0x7f46780afbe0 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f467e81a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467815d1b0 con 0x7f467806dfc0 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f466ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4678071070 0x7f467815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f466ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4678071070 0x7f467815cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50476/0 (socket says 172.21.15.70:50476) 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f466ffff640 1 -- 172.21.15.70:0/2529180423 learned_addr learned my addr 172.21.15.70:0/2529180423 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f4677fff640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467806dfc0 0x7f467814d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f467cdb7640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46780afbe0 0x7f4678150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:54.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f466ffff640 1 -- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467806dfc0 msgr2=0x7f467814d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:54.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f466ffff640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467806dfc0 0x7f467814d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f466ffff640 1 -- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46780afbe0 msgr2=0x7f4678150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:54.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f466ffff640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46780afbe0 0x7f4678150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:54.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.788+0000 7f466ffff640 1 -- 172.21.15.70:0/2529180423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600094d0 con 0x7f4678071070 2023-10-08T05:49:54.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.789+0000 7f466ffff640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4678071070 0x7f467815cbb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f466002b940 tx=0x7f46600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:54.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.789+0000 7f46757fa640 1 -- 172.21.15.70:0/2529180423 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466003ac60 con 0x7f4678071070 2023-10-08T05:49:54.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.789+0000 7f46757fa640 1 -- 172.21.15.70:0/2529180423 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466003ade0 con 0x7f4678071070 2023-10-08T05:49:54.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.789+0000 7f46757fa640 1 -- 172.21.15.70:0/2529180423 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660044630 con 0x7f4678071070 2023-10-08T05:49:54.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.789+0000 7f467e81a640 1 -- 172.21.15.70:0/2529180423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467815d490 con 0x7f4678071070 2023-10-08T05:49:54.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.789+0000 7f467e81a640 1 -- 172.21.15.70:0/2529180423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4678150e50 con 0x7f4678071070 2023-10-08T05:49:54.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.791+0000 7f467e81a640 1 -- 172.21.15.70:0/2529180423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467815d950 con 0x7f4678071070 2023-10-08T05:49:54.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.791+0000 7f46757fa640 1 -- 172.21.15.70:0/2529180423 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f466004d450 con 0x7f4678071070 2023-10-08T05:49:54.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.792+0000 7f46757fa640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f465806b020 0x7f465806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:54.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.792+0000 7f46757fa640 1 -- 172.21.15.70:0/2529180423 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6836+0+0 (secure 0 0 0) 0x7f46600ac420 con 0x7f4678071070 2023-10-08T05:49:54.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.792+0000 7f4677fff640 1 -- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f465806b020 msgr2=0x7f465806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:54.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.792+0000 7f4677fff640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f465806b020 0x7f465806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:49:54.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.796+0000 7f46757fa640 1 -- 172.21.15.70:0/2529180423 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4660076c30 con 0x7f4678071070 2023-10-08T05:49:54.810 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:54.809+0000 7f769f64c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:49:54.835 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:54.834+0000 7fb0bfd53040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:49:54.875 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:54.875+0000 7f769f64c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:49:54.891 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:54.890+0000 7fb0bfd53040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:49:54.933 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:54.932+0000 7f769f64c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:49:54.941 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:54.940+0000 7fb0bfd53040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:49:54.983 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:54.982+0000 7f769f64c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:49:54.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.993+0000 7f4677fff640 1 -- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f465806b020 msgr2=0x7f465806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:54.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:54.993+0000 7f4677fff640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f465806b020 0x7f465806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:49:55.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.012+0000 7f467e81a640 1 -- 172.21.15.70:0/2529180423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4678070600 con 0x7f4678071070 2023-10-08T05:49:55.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.013+0000 7f46757fa640 1 -- 172.21.15.70:0/2529180423 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7f466007a250 con 0x7f4678071070 2023-10-08T05:49:55.013 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:49:55.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 -- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f465806b020 msgr2=0x7f465806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:49:55.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f465806b020 0x7f465806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 -- 172.21.15.70:0/2529180423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4678071070 msgr2=0x7f467815cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4678071070 0x7f467815cbb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f466002b940 tx=0x7f46600046c0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 -- 172.21.15.70:0/2529180423 shutdown_connections 2023-10-08T05:49:55.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46780afbe0 0x7f4678150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4678071070 0x7f467815cbb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f465806b020 0x7f465806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 --2- 172.21.15.70:0/2529180423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467806dfc0 0x7f467814d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 -- 172.21.15.70:0/2529180423 >> 172.21.15.70:0/2529180423 conn(0x7f467806a5d0 msgr2=0x7f46780cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:55.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 -- 172.21.15.70:0/2529180423 shutdown_connections 2023-10-08T05:49:55.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.016+0000 7f467e81a640 1 -- 172.21.15.70:0/2529180423 wait complete. 2023-10-08T05:49:55.033 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:49:55.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.294+0000 7fe6a4839640 1 Processor -- start 2023-10-08T05:49:55.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.294+0000 7fe6a4839640 1 -- start start 2023-10-08T05:49:55.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.295+0000 7fe6a4839640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe6a00b4e10 0x7fe6a01612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.295+0000 7fe6a4839640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a00b8c30 con 0x7fe6a015cd70 2023-10-08T05:49:55.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.296+0000 7fe6a4839640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a00cdd80 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.296+0000 7fe6a4839640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a00ce060 con 0x7fe6a00b4e10 2023-10-08T05:49:55.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.296+0000 7fe69e59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe6a015cd70 0x7fe6a00b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49772/0 (socket says 172.21.15.70:49772) 2023-10-08T05:49:55.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.296+0000 7fe69e59c640 1 -- 172.21.15.70:0/2536889538 learned_addr learned my addr 172.21.15.70:0/2536889538 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:55.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.296+0000 7fe697fff640 1 --2- 172.21.15.70:0/2536889538 >> v2:172.21.15.70:3300/0 conn(0x7fe6a00b4e10 0x7fe6a01612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:55.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.296+0000 7fe69dd9b640 1 -- 172.21.15.70:0/2536889538 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941316351 0 0) 0x7fe6a00b8c30 con 0x7fe6a015cd70 2023-10-08T05:49:55.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.296+0000 7fe69dd9b640 1 -- 172.21.15.70:0/2536889538 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe68c003520 con 0x7fe6a015cd70 2023-10-08T05:49:55.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.296+0000 7fe697fff640 1 -- 172.21.15.70:0/2536889538 >> v1:172.21.15.181:6789/0 conn(0x7fe6a00cd4e0 legacy=0x7fe6a00d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:55.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.296+0000 7fe697fff640 1 -- 172.21.15.70:0/2536889538 >> v1:172.21.15.70:6789/0 conn(0x7fe6a015cd70 legacy=0x7fe6a00b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.296+0000 7fe697fff640 1 -- 172.21.15.70:0/2536889538 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a00ce340 con 0x7fe6a00b4e10 2023-10-08T05:49:55.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.298+0000 7fe697fff640 1 --2- 172.21.15.70:0/2536889538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00b4e10 0x7fe6a01612f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe6880097a0 tx=0x7fe68802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adca6893d008b63d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:55.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.298+0000 7fe69dd9b640 1 -- 172.21.15.70:0/2536889538 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688030610 con 0x7fe6a00b4e10 2023-10-08T05:49:55.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.298+0000 7fe69dd9b640 1 -- 172.21.15.70:0/2536889538 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688030770 con 0x7fe6a00b4e10 2023-10-08T05:49:55.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.298+0000 7fe69dd9b640 1 -- 172.21.15.70:0/2536889538 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688030ae0 con 0x7fe6a00b4e10 2023-10-08T05:49:55.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.299+0000 7fe6a4839640 1 -- 172.21.15.70:0/2536889538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00b4e10 msgr2=0x7fe6a01612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.299+0000 7fe6a4839640 1 --2- 172.21.15.70:0/2536889538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00b4e10 0x7fe6a01612f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe6880097a0 tx=0x7fe68802b960 comp rx=0 tx=0).stop 2023-10-08T05:49:55.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.299+0000 7fe6a4839640 1 -- 172.21.15.70:0/2536889538 shutdown_connections 2023-10-08T05:49:55.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.299+0000 7fe6a4839640 1 --2- 172.21.15.70:0/2536889538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00b4e10 0x7fe6a01612f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.299+0000 7fe6a4839640 1 -- 172.21.15.70:0/2536889538 >> 172.21.15.70:0/2536889538 conn(0x7fe6a00cf700 msgr2=0x7fe6a00cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:55.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.299+0000 7fe6a4839640 1 -- 172.21.15.70:0/2536889538 shutdown_connections 2023-10-08T05:49:55.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.299+0000 7fe6a4839640 1 -- 172.21.15.70:0/2536889538 wait complete. 2023-10-08T05:49:55.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.300+0000 7fe6a4839640 1 Processor -- start 2023-10-08T05:49:55.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.301+0000 7fe6a4839640 1 -- start start 2023-10-08T05:49:55.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.301+0000 7fe6a4839640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a00b4e10 0x7fe6a015ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe6a4839640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00cd4e0 0x7fe6a0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe6a4839640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a015cd70 0x7fe6a00cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe6a4839640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a015e980 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe6a4839640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a0157ac0 con 0x7fe6a00b4e10 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe6a4839640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a015dd70 con 0x7fe6a015cd70 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe697fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00cd4e0 0x7fe6a0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe697fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00cd4e0 0x7fe6a0157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50484/0 (socket says 172.21.15.70:50484) 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe697fff640 1 -- 172.21.15.70:0/3053808614 learned_addr learned my addr 172.21.15.70:0/3053808614 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe69ed9d640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a015cd70 0x7fe6a00cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe69e59c640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a00b4e10 0x7fe6a015ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe697fff640 1 -- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a015cd70 msgr2=0x7fe6a00cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe697fff640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a015cd70 0x7fe6a00cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe697fff640 1 -- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a00b4e10 msgr2=0x7fe6a015ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe697fff640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a00b4e10 0x7fe6a015ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.302+0000 7fe697fff640 1 -- 172.21.15.70:0/3053808614 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe688009400 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.303+0000 7fe697fff640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00cd4e0 0x7fe6a0157580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe6880097a0 tx=0x7fe6880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.303+0000 7fe696ffd640 1 -- 172.21.15.70:0/3053808614 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68803f070 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.303+0000 7fe696ffd640 1 -- 172.21.15.70:0/3053808614 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688006e10 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.303+0000 7fe6a4839640 1 -- 172.21.15.70:0/3053808614 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a015e050 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.303+0000 7fe696ffd640 1 -- 172.21.15.70:0/3053808614 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688030ea0 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.303+0000 7fe6a4839640 1 -- 172.21.15.70:0/3053808614 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a015e610 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.304+0000 7fe6a4839640 1 -- 172.21.15.70:0/3053808614 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a004e160 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.305+0000 7fe696ffd640 1 -- 172.21.15.70:0/3053808614 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fe688003810 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.306+0000 7fe696ffd640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe68006b020 0x7fe68006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.307+0000 7fe696ffd640 1 -- 172.21.15.70:0/3053808614 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6836+0+0 (secure 0 0 0) 0x7fe6880ab780 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.307+0000 7fe69e59c640 1 -- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe68006b020 msgr2=0x7fe68006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:55.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.307+0000 7fe69e59c640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe68006b020 0x7fe68006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:49:55.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.311+0000 7fe696ffd640 1 -- 172.21.15.70:0/3053808614 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6880760d0 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.323 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:55.322+0000 7fb0bfd53040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:49:55.384 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:55.384+0000 7f769f64c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:49:55.482 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:55.481+0000 7fb0bfd53040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:49:55.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.501+0000 7fe6a4839640 1 -- 172.21.15.70:0/3053808614 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6a00d0a00 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.502+0000 7fe696ffd640 1 -- 172.21.15.70:0/3053808614 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6880796f0 con 0x7fe6a00cd4e0 2023-10-08T05:49:55.502 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:49:55.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 -- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe68006b020 msgr2=0x7fe68006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:49:55.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe68006b020 0x7fe68006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 -- 172.21.15.70:0/3053808614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00cd4e0 msgr2=0x7fe6a0157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00cd4e0 0x7fe6a0157580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe6880097a0 tx=0x7fe6880046c0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 -- 172.21.15.70:0/3053808614 shutdown_connections 2023-10-08T05:49:55.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe68006b020 0x7fe68006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a015cd70 0x7fe6a00cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a00cd4e0 0x7fe6a0157580 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 --2- 172.21.15.70:0/3053808614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a00b4e10 0x7fe6a015ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 -- 172.21.15.70:0/3053808614 >> 172.21.15.70:0/3053808614 conn(0x7fe6a00cf700 msgr2=0x7fe6a015b130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:55.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 -- 172.21.15.70:0/3053808614 shutdown_connections 2023-10-08T05:49:55.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.505+0000 7fe6a4839640 1 -- 172.21.15.70:0/3053808614 wait complete. 2023-10-08T05:49:55.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:49:55.546 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:55.546+0000 7f769f64c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:49:55.591 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:55.590+0000 7fb0bfd53040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:49:55.641 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:55.640+0000 7fb0bfd53040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:49:55.660 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:55.660+0000 7f769f64c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:49:55.713 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:55.712+0000 7f769f64c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:49:55.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.732+0000 7f354fe00640 1 Processor -- start 2023-10-08T05:49:55.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.732+0000 7f354fe00640 1 -- start start 2023-10-08T05:49:55.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.734+0000 7f354fe00640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f354815cba0 0x7f35480b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.734+0000 7f354fe00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35480b4d10 0x7f35480d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.734+0000 7f354fe00640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35480b54a0 con 0x7f354815cf80 2023-10-08T05:49:55.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.734+0000 7f354fe00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35480b7130 con 0x7f35480b4d10 2023-10-08T05:49:55.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.734+0000 7f354fe00640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35480b7410 con 0x7f354815cba0 2023-10-08T05:49:55.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.734+0000 7f354e39d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35480b4d10 0x7f35480d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:55.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.734+0000 7f354e39d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35480b4d10 0x7f35480d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50486/0 (socket says 172.21.15.70:50486) 2023-10-08T05:49:55.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.734+0000 7f354e39d640 1 -- 172.21.15.70:0/155350251 learned_addr learned my addr 172.21.15.70:0/155350251 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:55.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.735+0000 7f354db9c640 1 --2- 172.21.15.70:0/155350251 >> v2:172.21.15.181:3300/0 conn(0x7f354815cba0 0x7f35480b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:55.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.735+0000 7f354e39d640 1 -- 172.21.15.70:0/155350251 >> v2:172.21.15.181:3300/0 conn(0x7f354815cba0 msgr2=0x7f35480b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:55.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.735+0000 7f354e39d640 1 --2- 172.21.15.70:0/155350251 >> v2:172.21.15.181:3300/0 conn(0x7f354815cba0 0x7f35480b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.735+0000 7f354e39d640 1 -- 172.21.15.70:0/155350251 >> v1:172.21.15.181:6789/0 conn(0x7f354815cf80 legacy=0x7f35480b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.735+0000 7f354e39d640 1 -- 172.21.15.70:0/155350251 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35480b76f0 con 0x7f35480b4d10 2023-10-08T05:49:55.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.737+0000 7f354e39d640 1 --2- 172.21.15.70:0/155350251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 0x7f35480d0ee0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3540009e50 tx=0x7f354002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adafc2e7258d4b1f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:55.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.737+0000 7f354cb9a640 1 -- 172.21.15.70:0/155350251 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354002ca80 con 0x7f35480b4d10 2023-10-08T05:49:55.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.737+0000 7f354cb9a640 1 -- 172.21.15.70:0/155350251 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354002cbe0 con 0x7f35480b4d10 2023-10-08T05:49:55.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.737+0000 7f354cb9a640 1 -- 172.21.15.70:0/155350251 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35400343f0 con 0x7f35480b4d10 2023-10-08T05:49:55.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.737+0000 7f354fe00640 1 -- 172.21.15.70:0/155350251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 msgr2=0x7f35480d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.737+0000 7f354fe00640 1 --2- 172.21.15.70:0/155350251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 0x7f35480d0ee0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3540009e50 tx=0x7f354002bee0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.738+0000 7f354fe00640 1 -- 172.21.15.70:0/155350251 shutdown_connections 2023-10-08T05:49:55.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.738+0000 7f354fe00640 1 --2- 172.21.15.70:0/155350251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 0x7f35480d0ee0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.738+0000 7f354fe00640 1 --2- 172.21.15.70:0/155350251 >> v2:172.21.15.181:3300/0 conn(0x7f354815cba0 0x7f35480b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.738+0000 7f354fe00640 1 -- 172.21.15.70:0/155350251 >> 172.21.15.70:0/155350251 conn(0x7f35480718a0 msgr2=0x7f3548071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:55.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.738+0000 7f354fe00640 1 -- 172.21.15.70:0/155350251 shutdown_connections 2023-10-08T05:49:55.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.738+0000 7f354fe00640 1 -- 172.21.15.70:0/155350251 wait complete. 2023-10-08T05:49:55.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.739+0000 7f354fe00640 1 Processor -- start 2023-10-08T05:49:55.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.740+0000 7f354fe00640 1 -- start start 2023-10-08T05:49:55.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.740+0000 7f354fe00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 0x7f35480d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.740+0000 7f354fe00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f354815cba0 0x7f35481570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354fe00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f354815cf80 0x7f35480c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354fe00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35480c7c00 con 0x7f35480b4d10 2023-10-08T05:49:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354fe00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35480c7ee0 con 0x7f354815cba0 2023-10-08T05:49:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354fe00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35480c81c0 con 0x7f354815cf80 2023-10-08T05:49:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354db9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 0x7f35480d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354db9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 0x7f35480d2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50502/0 (socket says 172.21.15.70:50502) 2023-10-08T05:49:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354db9c640 1 -- 172.21.15.70:0/2990845351 learned_addr learned my addr 172.21.15.70:0/2990845351 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354db9c640 1 -- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f354815cf80 msgr2=0x7f35480c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354db9c640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f354815cf80 0x7f35480c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354db9c640 1 -- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f354815cba0 msgr2=0x7f35481570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354db9c640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f354815cba0 0x7f35481570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354db9c640 1 -- 172.21.15.70:0/2990845351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3540009b30 con 0x7f35480b4d10 2023-10-08T05:49:55.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.741+0000 7f354db9c640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 0x7f35480d2a30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f354400d800 tx=0x7f354400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:55.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.742+0000 7f353e7fc640 1 -- 172.21.15.70:0/2990845351 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354400b6c0 con 0x7f35480b4d10 2023-10-08T05:49:55.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.742+0000 7f353e7fc640 1 -- 172.21.15.70:0/2990845351 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354400b820 con 0x7f35480b4d10 2023-10-08T05:49:55.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.742+0000 7f353e7fc640 1 -- 172.21.15.70:0/2990845351 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354400bbb0 con 0x7f35480b4d10 2023-10-08T05:49:55.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.742+0000 7f354fe00640 1 -- 172.21.15.70:0/2990845351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35480c84a0 con 0x7f35480b4d10 2023-10-08T05:49:55.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.742+0000 7f354fe00640 1 -- 172.21.15.70:0/2990845351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35480d2f70 con 0x7f35480b4d10 2023-10-08T05:49:55.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.743+0000 7f354fe00640 1 -- 172.21.15.70:0/2990845351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35480d3100 con 0x7f35480b4d10 2023-10-08T05:49:55.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.744+0000 7f353e7fc640 1 -- 172.21.15.70:0/2990845351 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f354400bd10 con 0x7f35480b4d10 2023-10-08T05:49:55.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.745+0000 7f353e7fc640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f352c06b020 0x7f352c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:55.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.745+0000 7f353e7fc640 1 -- 172.21.15.70:0/2990845351 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6836+0+0 (secure 0 0 0) 0x7f354408a3f0 con 0x7f35480b4d10 2023-10-08T05:49:55.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.745+0000 7f354d39b640 1 -- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f352c06b020 msgr2=0x7f352c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:55.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.745+0000 7f354d39b640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f352c06b020 0x7f352c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:49:55.749 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:55.747+0000 7fb0bfd53040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:49:55.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.753+0000 7f353e7fc640 1 -- 172.21.15.70:0/2990845351 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35480d3100 con 0x7f35480b4d10 2023-10-08T05:49:55.829 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:55.829+0000 7f769f64c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:49:55.855 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:55.854+0000 7fb0bfd53040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:49:55.939 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:55.939+0000 7f769f64c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:49:55.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.946+0000 7f354d39b640 1 -- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f352c06b020 msgr2=0x7f352c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:55.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.946+0000 7f354d39b640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f352c06b020 0x7f352c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:49:55.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.958+0000 7f354fe00640 1 -- 172.21.15.70:0/2990845351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f35480d3100 con 0x7f35480b4d10 2023-10-08T05:49:55.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.959+0000 7f353e7fc640 1 -- 172.21.15.70:0/2990845351 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v57) v1 ==== 80+0+360 (secure 0 0 0) 0x7f35480d3100 con 0x7f35480b4d10 2023-10-08T05:49:55.959 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/2737793492 2023-10-09T05:38:04.171728+0000 2023-10-08T05:49:55.959 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/2737793492 2023-10-09T05:38:04.171728+0000 2023-10-08T05:49:55.959 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3280597969 2023-10-09T05:38:04.171728+0000 2023-10-08T05:49:55.959 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3620192672 2023-10-09T05:38:04.171728+0000 2023-10-08T05:49:55.960 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3583094356 2023-10-09T05:38:04.171728+0000 2023-10-08T05:49:55.960 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3573082646 2023-10-09T05:38:04.171728+0000 2023-10-08T05:49:55.960 INFO:teuthology.orchestra.run.smithi070.stderr:listed 6 entries 2023-10-08T05:49:55.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.961+0000 7f354fe00640 1 -- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f352c06b020 msgr2=0x7f352c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:49:55.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.961+0000 7f354fe00640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f352c06b020 0x7f352c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.961+0000 7f354fe00640 1 -- 172.21.15.70:0/2990845351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 msgr2=0x7f35480d2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:55.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.961+0000 7f354fe00640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 0x7f35480d2a30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f354400d800 tx=0x7f354400dcc0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.961+0000 7f354fe00640 1 -- 172.21.15.70:0/2990845351 shutdown_connections 2023-10-08T05:49:55.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.961+0000 7f354fe00640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f352c06b020 0x7f352c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.961+0000 7f354fe00640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f354815cf80 0x7f35480c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.961+0000 7f354fe00640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f354815cba0 0x7f35481570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.961+0000 7f354fe00640 1 --2- 172.21.15.70:0/2990845351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35480b4d10 0x7f35480d2a30 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:55.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.961+0000 7f354fe00640 1 -- 172.21.15.70:0/2990845351 >> 172.21.15.70:0/2990845351 conn(0x7f35480718a0 msgr2=0x7f354806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:55.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.962+0000 7f354fe00640 1 -- 172.21.15.70:0/2990845351 shutdown_connections 2023-10-08T05:49:55.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:55.962+0000 7f354fe00640 1 -- 172.21.15.70:0/2990845351 wait complete. 2023-10-08T05:49:55.979 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T05:49:55.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:55.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T05:49:56.014 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:56.013+0000 7fb0bfd53040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:49:56.062 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:56.062+0000 7fb0bfd53040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:49:56.104 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:56.103+0000 7f769f64c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:49:56.144 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:49:56.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:49:56.145 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T05:49:56.155 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:56.155+0000 7f769f64c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:49:56.184 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:56.183+0000 7fb0bfd53040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:49:56.190 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:49:56.190 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:49:56.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:49:56.246 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:49:56.246 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:49:56.279 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:56.279+0000 7f769f64c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:49:56.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:49:56.301 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:49:56.302 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:49:56.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:49:56.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:49:56.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.567+0000 7f40cf59e640 1 Processor -- start 2023-10-08T05:49:56.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.568+0000 7f40cf59e640 1 -- start start 2023-10-08T05:49:56.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.569+0000 7f40cf59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f40d015ab30 0x7f40d015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.569+0000 7f40cf59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40d00b07d0 0x7f40d006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.569+0000 7f40cf59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d015f7d0 con 0x7f40d00adc70 2023-10-08T05:49:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.569+0000 7f40cf59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d015f210 con 0x7f40d015ab30 2023-10-08T05:49:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.569+0000 7f40cf59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d015f4f0 con 0x7f40d00b07d0 2023-10-08T05:49:56.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.569+0000 7f40ce59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f40d015ab30 0x7f40d015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:56.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.569+0000 7f40ce59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f40d015ab30 0x7f40d015e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50504/0 (socket says 172.21.15.70:50504) 2023-10-08T05:49:56.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.569+0000 7f40ce59c640 1 -- 172.21.15.70:0/624464395 learned_addr learned my addr 172.21.15.70:0/624464395 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:56.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.570+0000 7f40ce59c640 1 -- 172.21.15.70:0/624464395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40d00b07d0 msgr2=0x7f40d006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.570+0000 7f40ced9d640 1 --2- 172.21.15.70:0/624464395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40d00b07d0 0x7f40d006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.570+0000 7f40ce59c640 1 --2- 172.21.15.70:0/624464395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40d00b07d0 0x7f40d006cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.570+0000 7f40ce59c640 1 -- 172.21.15.70:0/624464395 >> v1:172.21.15.70:6789/0 conn(0x7f40d00adc70 legacy=0x7f40d00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.570+0000 7f40ce59c640 1 -- 172.21.15.70:0/624464395 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d006c550 con 0x7f40d015ab30 2023-10-08T05:49:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.571+0000 7f40ce59c640 1 --2- 172.21.15.70:0/624464395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d015ab30 0x7f40d015e8a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009870 tx=0x7f40c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7847cf2b03c6a70d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:56.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.571+0000 7f40cdd9b640 1 -- 172.21.15.70:0/624464395 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0030840 con 0x7f40d015ab30 2023-10-08T05:49:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.571+0000 7f40cdd9b640 1 -- 172.21.15.70:0/624464395 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c00309a0 con 0x7f40d015ab30 2023-10-08T05:49:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.571+0000 7f40cdd9b640 1 -- 172.21.15.70:0/624464395 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0030d30 con 0x7f40d015ab30 2023-10-08T05:49:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.572+0000 7f40cf59e640 1 -- 172.21.15.70:0/624464395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d015ab30 msgr2=0x7f40d015e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.572+0000 7f40cf59e640 1 --2- 172.21.15.70:0/624464395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d015ab30 0x7f40d015e8a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009870 tx=0x7f40c002b980 comp rx=0 tx=0).stop 2023-10-08T05:49:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.572+0000 7f40cf59e640 1 -- 172.21.15.70:0/624464395 shutdown_connections 2023-10-08T05:49:56.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.572+0000 7f40cf59e640 1 --2- 172.21.15.70:0/624464395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40d00b07d0 0x7f40d006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.572+0000 7f40cf59e640 1 --2- 172.21.15.70:0/624464395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d015ab30 0x7f40d015e8a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.572+0000 7f40cf59e640 1 -- 172.21.15.70:0/624464395 >> 172.21.15.70:0/624464395 conn(0x7f40d00cb0c0 msgr2=0x7f40d00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:56.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.572+0000 7f40cf59e640 1 -- 172.21.15.70:0/624464395 shutdown_connections 2023-10-08T05:49:56.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.572+0000 7f40cf59e640 1 -- 172.21.15.70:0/624464395 wait complete. 2023-10-08T05:49:56.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.574+0000 7f40cf59e640 1 Processor -- start 2023-10-08T05:49:56.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.574+0000 7f40cf59e640 1 -- start start 2023-10-08T05:49:56.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40cf59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40d00adc70 0x7f40d0148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:56.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40cf59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d00b07d0 0x7f40d015d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40cf59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d015ab30 0x7f40d00b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40cf59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d0149050 con 0x7f40d00b07d0 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40cf59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00c5710 con 0x7f40d015ab30 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40cf59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d015d540 con 0x7f40d00adc70 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d00b07d0 0x7f40d015d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d00b07d0 0x7f40d015d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50512/0 (socket says 172.21.15.70:50512) 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40c7fff640 1 -- 172.21.15.70:0/153852081 learned_addr learned my addr 172.21.15.70:0/153852081 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40c7fff640 1 -- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40d00adc70 msgr2=0x7f40d0148b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40ce59c640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40d00adc70 0x7f40d0148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40c7fff640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40d00adc70 0x7f40d0148b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40c7fff640 1 -- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d015ab30 msgr2=0x7f40d00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40c7fff640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d015ab30 0x7f40d00b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.575+0000 7f40c7fff640 1 -- 172.21.15.70:0/153852081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00094d0 con 0x7f40d00b07d0 2023-10-08T05:49:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.576+0000 7f40c7fff640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d00b07d0 0x7f40d015d000 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f40b800c400 tx=0x7f40b800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.576+0000 7f40c6ffd640 1 -- 172.21.15.70:0/153852081 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b80052d0 con 0x7f40d00b07d0 2023-10-08T05:49:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.576+0000 7f40c6ffd640 1 -- 172.21.15.70:0/153852081 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b8005450 con 0x7f40d00b07d0 2023-10-08T05:49:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.576+0000 7f40cf59e640 1 -- 172.21.15.70:0/153852081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d015d810 con 0x7f40d00b07d0 2023-10-08T05:49:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.576+0000 7f40c6ffd640 1 -- 172.21.15.70:0/153852081 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8007260 con 0x7f40d00b07d0 2023-10-08T05:49:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.576+0000 7f40cf59e640 1 -- 172.21.15.70:0/153852081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d015ddd0 con 0x7f40d00b07d0 2023-10-08T05:49:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.577+0000 7f40cf59e640 1 -- 172.21.15.70:0/153852081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d004e160 con 0x7f40d00b07d0 2023-10-08T05:49:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.578+0000 7f40c6ffd640 1 -- 172.21.15.70:0/153852081 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7f40b80073c0 con 0x7f40d00b07d0 2023-10-08T05:49:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.580+0000 7f40c6ffd640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f40b006b020 0x7f40b006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.580+0000 7f40c6ffd640 1 -- 172.21.15.70:0/153852081 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6836+0+0 (secure 0 0 0) 0x7f40b8089280 con 0x7f40d00b07d0 2023-10-08T05:49:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.580+0000 7f40ce59c640 1 -- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f40b006b020 msgr2=0x7f40b006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.580+0000 7f40ce59c640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f40b006b020 0x7f40b006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:49:56.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.583+0000 7f40c6ffd640 1 -- 172.21.15.70:0/153852081 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40b8053ba0 con 0x7f40d00b07d0 2023-10-08T05:49:56.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.781+0000 7f40ce59c640 1 -- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f40b006b020 msgr2=0x7f40b006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:56.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.781+0000 7f40ce59c640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f40b006b020 0x7f40b006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:49:56.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.808+0000 7f40cf59e640 1 -- 172.21.15.70:0/153852081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40d015eae0 con 0x7f40d00b07d0 2023-10-08T05:49:56.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.809+0000 7f40c6ffd640 1 -- 172.21.15.70:0/153852081 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f40b80571c0 con 0x7f40d00b07d0 2023-10-08T05:49:56.809 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:56.809 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/809656839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":809656839},{"type":"v1","addr":"172.21.15.70:6835","nonce":809656839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2750508216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2750508216},{"type":"v1","addr":"172.21.15.181:6836","nonce":2750508216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/607266140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":607266140},{"type":"v1","addr":"172.21.15.181:6837","nonce":607266140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:37:44.192492+0000","modified":"2023-10-08T05:37:47.238340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/480502229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":480502229},{"type":"v1","addr":"172.21.15.70:6834","nonce":480502229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-08T05:49:56.810 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 18 2023-10-08T05:49:56.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 -- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f40b006b020 msgr2=0x7f40b006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:49:56.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f40b006b020 0x7f40b006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 -- 172.21.15.70:0/153852081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d00b07d0 msgr2=0x7f40d015d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:56.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d00b07d0 0x7f40d015d000 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f40b800c400 tx=0x7f40b800c8c0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 -- 172.21.15.70:0/153852081 shutdown_connections 2023-10-08T05:49:56.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40d015ab30 0x7f40d00b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d00b07d0 0x7f40d015d000 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7f40b006b020 0x7f40b006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 --2- 172.21.15.70:0/153852081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40d00adc70 0x7f40d0148b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:56.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 -- 172.21.15.70:0/153852081 >> 172.21.15.70:0/153852081 conn(0x7f40d00cb0c0 msgr2=0x7f40d00ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:56.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 -- 172.21.15.70:0/153852081 shutdown_connections 2023-10-08T05:49:56.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:56.811+0000 7f40cf59e640 1 -- 172.21.15.70:0/153852081 wait complete. 2023-10-08T05:49:56.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:49:57.017 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.016+0000 7fb0bfd53040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:49:57.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.019+0000 7fe99b440640 1 Processor -- start 2023-10-08T05:49:57.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.019+0000 7fe99b440640 1 -- start start 2023-10-08T05:49:57.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.021+0000 7fe99b440640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9940af8b0 0x7fe99406ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.021+0000 7fe99b440640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe9940b0ab0 0x7fe9940b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.021+0000 7fe99b440640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9940b0340 con 0x7fe99406b520 2023-10-08T05:49:57.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.021+0000 7fe99b440640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99406b3a0 con 0x7fe9940b0ab0 2023-10-08T05:49:57.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.021+0000 7fe99b440640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9940b0150 con 0x7fe9940af8b0 2023-10-08T05:49:57.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.021+0000 7fe9999dd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe9940b0ab0 0x7fe9940b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.021+0000 7fe9999dd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe9940b0ab0 0x7fe9940b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50522/0 (socket says 172.21.15.70:50522) 2023-10-08T05:49:57.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.021+0000 7fe9999dd640 1 -- 172.21.15.70:0/1852612296 learned_addr learned my addr 172.21.15.70:0/1852612296 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:57.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.022+0000 7fe9989db640 1 --2- 172.21.15.70:0/1852612296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9940af8b0 0x7fe99406ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.022+0000 7fe9999dd640 1 -- 172.21.15.70:0/1852612296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9940af8b0 msgr2=0x7fe99406ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:57.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.022+0000 7fe9999dd640 1 --2- 172.21.15.70:0/1852612296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9940af8b0 0x7fe99406ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.022+0000 7fe9999dd640 1 -- 172.21.15.70:0/1852612296 >> v1:172.21.15.181:6789/0 conn(0x7fe99406b520 legacy=0x7fe99406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:57.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.022+0000 7fe9999dd640 1 -- 172.21.15.70:0/1852612296 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9940cf8f0 con 0x7fe9940b0ab0 2023-10-08T05:49:57.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.023+0000 7fe9999dd640 1 --2- 172.21.15.70:0/1852612296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 0x7fe9940b0f40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009e50 tx=0x7fe98c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f9903390f161df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:57.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.023+0000 7fe98bfff640 1 -- 172.21.15.70:0/1852612296 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c02c9e0 con 0x7fe9940b0ab0 2023-10-08T05:49:57.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.023+0000 7fe98bfff640 1 -- 172.21.15.70:0/1852612296 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c02cb40 con 0x7fe9940b0ab0 2023-10-08T05:49:57.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.023+0000 7fe98bfff640 1 -- 172.21.15.70:0/1852612296 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c0343f0 con 0x7fe9940b0ab0 2023-10-08T05:49:57.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.024+0000 7fe99b440640 1 -- 172.21.15.70:0/1852612296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 msgr2=0x7fe9940b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.024+0000 7fe99b440640 1 --2- 172.21.15.70:0/1852612296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 0x7fe9940b0f40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009e50 tx=0x7fe98c02be40 comp rx=0 tx=0).stop 2023-10-08T05:49:57.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.024+0000 7fe99b440640 1 -- 172.21.15.70:0/1852612296 shutdown_connections 2023-10-08T05:49:57.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.024+0000 7fe99b440640 1 --2- 172.21.15.70:0/1852612296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 0x7fe9940b0f40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.024+0000 7fe99b440640 1 --2- 172.21.15.70:0/1852612296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9940af8b0 0x7fe99406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.024+0000 7fe99b440640 1 -- 172.21.15.70:0/1852612296 >> 172.21.15.70:0/1852612296 conn(0x7fe9940d09e0 msgr2=0x7fe9940d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:57.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.024+0000 7fe99b440640 1 -- 172.21.15.70:0/1852612296 shutdown_connections 2023-10-08T05:49:57.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.024+0000 7fe99b440640 1 -- 172.21.15.70:0/1852612296 wait complete. 2023-10-08T05:49:57.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.026+0000 7fe99b440640 1 Processor -- start 2023-10-08T05:49:57.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.026+0000 7fe99b440640 1 -- start start 2023-10-08T05:49:57.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe99b440640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99406b520 0x7fe994158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe99b440640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9940af8b0 0x7fe99415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe99b440640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 0x7fe99415fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe99b440640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9940c5560 con 0x7fe9940b0ab0 2023-10-08T05:49:57.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe99b440640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe994160160 con 0x7fe99406b520 2023-10-08T05:49:57.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe99b440640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99409e7a0 con 0x7fe9940af8b0 2023-10-08T05:49:57.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9999dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 0x7fe99415fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9999dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 0x7fe99415fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50524/0 (socket says 172.21.15.70:50524) 2023-10-08T05:49:57.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9999dd640 1 -- 172.21.15.70:0/570172052 learned_addr learned my addr 172.21.15.70:0/570172052 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9989db640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9940af8b0 0x7fe99415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9991dc640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99406b520 0x7fe994158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9999dd640 1 -- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9940af8b0 msgr2=0x7fe99415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9999dd640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9940af8b0 0x7fe99415f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9999dd640 1 -- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99406b520 msgr2=0x7fe994158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9999dd640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99406b520 0x7fe994158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9999dd640 1 -- 172.21.15.70:0/570172052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c009b30 con 0x7fe9940b0ab0 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.027+0000 7fe9991dc640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99406b520 0x7fe994158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.028+0000 7fe9999dd640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 0x7fe99415fc20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009f70 tx=0x7fe98c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.028+0000 7fe989ffb640 1 -- 172.21.15.70:0/570172052 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c040670 con 0x7fe9940b0ab0 2023-10-08T05:49:57.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.028+0000 7fe989ffb640 1 -- 172.21.15.70:0/570172052 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c0407f0 con 0x7fe9940b0ab0 2023-10-08T05:49:57.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.028+0000 7fe99b440640 1 -- 172.21.15.70:0/570172052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99415db80 con 0x7fe9940b0ab0 2023-10-08T05:49:57.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.028+0000 7fe989ffb640 1 -- 172.21.15.70:0/570172052 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c040ba0 con 0x7fe9940b0ab0 2023-10-08T05:49:57.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.028+0000 7fe99b440640 1 -- 172.21.15.70:0/570172052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9941605c0 con 0x7fe9940b0ab0 2023-10-08T05:49:57.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.029+0000 7fe99b440640 1 -- 172.21.15.70:0/570172052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9940c56d0 con 0x7fe9940b0ab0 2023-10-08T05:49:57.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.030+0000 7fe989ffb640 1 -- 172.21.15.70:0/570172052 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fe98c040d00 con 0x7fe9940b0ab0 2023-10-08T05:49:57.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.031+0000 7fe989ffb640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe97806b050 0x7fe97806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.031+0000 7fe989ffb640 1 -- 172.21.15.70:0/570172052 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6836+0+0 (secure 0 0 0) 0x7fe98c0ace70 con 0x7fe9940b0ab0 2023-10-08T05:49:57.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.031+0000 7fe9991dc640 1 -- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe97806b050 msgr2=0x7fe97806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:57.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.031+0000 7fe9991dc640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe97806b050 0x7fe97806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:49:57.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.035+0000 7fe989ffb640 1 -- 172.21.15.70:0/570172052 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9940c56d0 con 0x7fe9940b0ab0 2023-10-08T05:49:57.104 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:57.104+0000 7f769f64c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:49:57.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.215+0000 7fe99b440640 1 -- 172.21.15.70:0/570172052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9940c56a0 con 0x7fe9940b0ab0 2023-10-08T05:49:57.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.217+0000 7fe989ffb640 1 -- 172.21.15.70:0/570172052 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+18004 (secure 0 0 0) 0x7fe98c077790 con 0x7fe9940b0ab0 2023-10-08T05:49:57.217 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:57.218 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":57,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:49:53.247400+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:37:41.960853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-08T05:37:43.190325+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/2737793492":"2023-10-09T05:38:04.171728+0000","172.21.15.181:6832/2737793492":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3280597969":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3620192672":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3583094356":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3573082646":"2023-10-09T05:38:04.171728+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":9,"length":1},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":24,"length":1},{"begin":30,"length":1},{"begin":38,"length":2},{"begin":45,"length":3},{"begin":53,"length":2},{"begin":57,"length":1},{"begin":62,"length":1},{"begin":64,"length":1},{"begin":66,"length":1},{"begin":73,"length":2},{"begin":79,"length":1},{"begin":86,"length":1},{"begin":88,"length":2},{"begin":94,"length":1},{"begin":96,"length":1},{"begin":99,"length":1},{"begin":107,"length":2},{"begin":193,"length":28}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":30,"length":1},{"begin":39,"length":1},{"begin":47,"length":1},{"begin":57,"length":1},{"begin":62,"length":1},{"begin":74,"length":1},{"begin":88,"length":1},{"begin":96,"length":1},{"begin":213,"length":8}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:49:57.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.219+0000 7fe99b440640 1 -- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe97806b050 msgr2=0x7fe97806d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:49:57.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.219+0000 7fe99b440640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe97806b050 0x7fe97806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.219+0000 7fe99b440640 1 -- 172.21.15.70:0/570172052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 msgr2=0x7fe99415fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.219+0000 7fe99b440640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 0x7fe99415fc20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009f70 tx=0x7fe98c004a40 comp rx=0 tx=0).stop 2023-10-08T05:49:57.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.219+0000 7fe99b440640 1 -- 172.21.15.70:0/570172052 shutdown_connections 2023-10-08T05:49:57.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.219+0000 7fe99b440640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9940b0ab0 0x7fe99415fc20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.219+0000 7fe99b440640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe97806b050 0x7fe97806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.219+0000 7fe99b440640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9940af8b0 0x7fe99415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.219+0000 7fe99b440640 1 --2- 172.21.15.70:0/570172052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe99406b520 0x7fe994158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.219+0000 7fe99b440640 1 -- 172.21.15.70:0/570172052 >> 172.21.15.70:0/570172052 conn(0x7fe9940d09e0 msgr2=0x7fe99406eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:57.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.220+0000 7fe99b440640 1 -- 172.21.15.70:0/570172052 shutdown_connections 2023-10-08T05:49:57.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.220+0000 7fe99b440640 1 -- 172.21.15.70:0/570172052 wait complete. 2023-10-08T05:49:57.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:37:41.960853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:37:43.190325+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 57, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:49:57.237 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:49:57.314 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.313+0000 7fb0bfd53040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:49:57.374 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.373+0000 7fb0bfd53040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:49:57.414 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:57.414+0000 7f769f64c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:49:57.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.434+0000 7fe409f39640 1 Processor -- start 2023-10-08T05:49:57.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.435+0000 7fe409f39640 1 -- start start 2023-10-08T05:49:57.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe409f39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe40415cc60 0x7fe4041612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe409f39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4040cd3d0 0x7fe4040b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe409f39640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4040b4b30 0x7fe40406b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe409f39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4040b4f80 con 0x7fe4040cd3d0 2023-10-08T05:49:57.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe409f39640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4040cdd70 con 0x7fe4040b4b30 2023-10-08T05:49:57.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe409f39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040ce050 con 0x7fe40415cc60 2023-10-08T05:49:57.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe3fbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4040cd3d0 0x7fe4040b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe3fbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4040cd3d0 0x7fe4040b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50534/0 (socket says 172.21.15.70:50534) 2023-10-08T05:49:57.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe3fbfff640 1 -- 172.21.15.70:0/3075181140 learned_addr learned my addr 172.21.15.70:0/3075181140 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:57.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe403fff640 1 --2- 172.21.15.70:0/3075181140 >> v2:172.21.15.181:3300/0 conn(0x7fe4040b4b30 0x7fe40406b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.436+0000 7fe4037fe640 1 --2- 172.21.15.70:0/3075181140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe40415cc60 0x7fe4041612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.437+0000 7fe3fbfff640 1 -- 172.21.15.70:0/3075181140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe40415cc60 msgr2=0x7fe4041612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:57.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.437+0000 7fe3fbfff640 1 --2- 172.21.15.70:0/3075181140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe40415cc60 0x7fe4041612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.437+0000 7fe3fbfff640 1 -- 172.21.15.70:0/3075181140 >> v2:172.21.15.181:3300/0 conn(0x7fe4040b4b30 msgr2=0x7fe40406b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:57.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.437+0000 7fe3fbfff640 1 --2- 172.21.15.70:0/3075181140 >> v2:172.21.15.181:3300/0 conn(0x7fe4040b4b30 0x7fe40406b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.437+0000 7fe3fbfff640 1 -- 172.21.15.70:0/3075181140 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4040d1010 con 0x7fe4040cd3d0 2023-10-08T05:49:57.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.437+0000 7fe403fff640 1 --2- 172.21.15.70:0/3075181140 >> v2:172.21.15.181:3300/0 conn(0x7fe4040b4b30 0x7fe40406b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:57.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.437+0000 7fe4037fe640 1 --2- 172.21.15.70:0/3075181140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe40415cc60 0x7fe4041612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:57.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.438+0000 7fe3fbfff640 1 --2- 172.21.15.70:0/3075181140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 0x7fe4040b45f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe40408be20 tx=0x7fe3ec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e061f702fddc399 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.438+0000 7fe402ffd640 1 -- 172.21.15.70:0/3075181140 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0317e0 con 0x7fe4040cd3d0 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.438+0000 7fe402ffd640 1 -- 172.21.15.70:0/3075181140 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec031940 con 0x7fe4040cd3d0 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.438+0000 7fe402ffd640 1 -- 172.21.15.70:0/3075181140 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec031cd0 con 0x7fe4040cd3d0 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.439+0000 7fe409f39640 1 -- 172.21.15.70:0/3075181140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 msgr2=0x7fe4040b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.439+0000 7fe409f39640 1 --2- 172.21.15.70:0/3075181140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 0x7fe4040b45f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe40408be20 tx=0x7fe3ec02b880 comp rx=0 tx=0).stop 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.439+0000 7fe409f39640 1 -- 172.21.15.70:0/3075181140 shutdown_connections 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.439+0000 7fe409f39640 1 --2- 172.21.15.70:0/3075181140 >> v2:172.21.15.181:3300/0 conn(0x7fe4040b4b30 0x7fe40406b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.439+0000 7fe409f39640 1 --2- 172.21.15.70:0/3075181140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 0x7fe4040b45f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.439+0000 7fe409f39640 1 --2- 172.21.15.70:0/3075181140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe40415cc60 0x7fe4041612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.439+0000 7fe409f39640 1 -- 172.21.15.70:0/3075181140 >> 172.21.15.70:0/3075181140 conn(0x7fe4040cf5f0 msgr2=0x7fe4040cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.439+0000 7fe409f39640 1 -- 172.21.15.70:0/3075181140 shutdown_connections 2023-10-08T05:49:57.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.439+0000 7fe409f39640 1 -- 172.21.15.70:0/3075181140 wait complete. 2023-10-08T05:49:57.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.441+0000 7fe409f39640 1 Processor -- start 2023-10-08T05:49:57.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.441+0000 7fe409f39640 1 -- start start 2023-10-08T05:49:57.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe409f39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4040b4b30 0x7fe4040caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe409f39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 0x7fe4040c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe409f39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40415cc60 0x7fe404157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe409f39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040cb490 con 0x7fe4040cd3d0 2023-10-08T05:49:57.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe409f39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040c8080 con 0x7fe40415cc60 2023-10-08T05:49:57.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe409f39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4040c8ea0 con 0x7fe4040b4b30 2023-10-08T05:49:57.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe3fbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 0x7fe4040c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe3fbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 0x7fe4040c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50542/0 (socket says 172.21.15.70:50542) 2023-10-08T05:49:57.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe3fbfff640 1 -- 172.21.15.70:0/3931985899 learned_addr learned my addr 172.21.15.70:0/3931985899 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:57.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe4037fe640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4040b4b30 0x7fe4040caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe403fff640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40415cc60 0x7fe404157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe3fbfff640 1 -- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4040b4b30 msgr2=0x7fe4040caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe3fbfff640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4040b4b30 0x7fe4040caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe3fbfff640 1 -- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40415cc60 msgr2=0x7fe404157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe3fbfff640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40415cc60 0x7fe404157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe3fbfff640 1 -- 172.21.15.70:0/3931985899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0094d0 con 0x7fe4040cd3d0 2023-10-08T05:49:57.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.442+0000 7fe403fff640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40415cc60 0x7fe404157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:57.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.443+0000 7fe3fbfff640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 0x7fe4040c7b40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec000c00 tx=0x7fe3ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:57.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.443+0000 7fe400ff9640 1 -- 172.21.15.70:0/3931985899 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec032900 con 0x7fe4040cd3d0 2023-10-08T05:49:57.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.443+0000 7fe400ff9640 1 -- 172.21.15.70:0/3931985899 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec032a60 con 0x7fe4040cd3d0 2023-10-08T05:49:57.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.443+0000 7fe400ff9640 1 -- 172.21.15.70:0/3931985899 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec032e10 con 0x7fe4040cd3d0 2023-10-08T05:49:57.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.443+0000 7fe409f39640 1 -- 172.21.15.70:0/3931985899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4040c8290 con 0x7fe4040cd3d0 2023-10-08T05:49:57.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.443+0000 7fe409f39640 1 -- 172.21.15.70:0/3931985899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4040c8850 con 0x7fe4040cd3d0 2023-10-08T05:49:57.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.444+0000 7fe409f39640 1 -- 172.21.15.70:0/3931985899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40404e160 con 0x7fe4040cd3d0 2023-10-08T05:49:57.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.446+0000 7fe400ff9640 1 -- 172.21.15.70:0/3931985899 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 14) v1 ==== 89400+0+0 (secure 0 0 0) 0x7fe3ec04e450 con 0x7fe4040cd3d0 2023-10-08T05:49:57.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.448+0000 7fe400ff9640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe3e406b020 0x7fe3e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.448+0000 7fe400ff9640 1 -- 172.21.15.70:0/3931985899 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6836+0+0 (secure 0 0 0) 0x7fe3ec0ac150 con 0x7fe4040cd3d0 2023-10-08T05:49:57.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.448+0000 7fe4037fe640 1 -- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe3e406b020 msgr2=0x7fe3e406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2692398351 2023-10-08T05:49:57.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.448+0000 7fe4037fe640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe3e406b020 0x7fe3e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:49:57.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.453+0000 7fe400ff9640 1 -- 172.21.15.70:0/3931985899 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3ec076b40 con 0x7fe4040cd3d0 2023-10-08T05:49:57.476 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:57.476+0000 7f769f64c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:49:57.488 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.487+0000 7fb0bfd53040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:49:57.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.509+0000 7fe400ff9640 1 -- 172.21.15.70:0/3931985899 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 15) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe3ec076f10 con 0x7fe4040cd3d0 2023-10-08T05:49:57.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.510+0000 7fe400ff9640 1 -- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe3e406b020 msgr2=0x7fe3e406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:49:57.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.510+0000 7fe400ff9640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe3e406b020 0x7fe3e406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.588 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.587+0000 7fb044fe0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:49:57.589 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.587+0000 7fb044fe0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:49:57.589 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.587+0000 7fb044fe0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:49:57.589 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.587+0000 7fb044fe0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:49:57.589 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.587+0000 7fb044fe0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:49:57.590 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.588+0000 7fb0427db640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:49:57.590 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.588+0000 7fb0427db640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:49:57.590 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.588+0000 7fb0427db640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:49:57.590 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.588+0000 7fb0427db640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:49:57.590 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:49:57.588+0000 7fb0427db640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:49:57.602 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:49:57.601+0000 7f769f64c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:49:57.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.654+0000 7fe409f39640 1 -- 172.21.15.70:0/3931985899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4040b8710 con 0x7fe4040cd3d0 2023-10-08T05:49:57.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.655+0000 7fe400ff9640 1 -- 172.21.15.70:0/3931985899 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3ec08f5f0 con 0x7fe4040cd3d0 2023-10-08T05:49:57.656 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:49:57.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.658+0000 7fe409f39640 1 -- 172.21.15.70:0/3931985899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 msgr2=0x7fe4040c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.658+0000 7fe409f39640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 0x7fe4040c7b40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec000c00 tx=0x7fe3ec0046c0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.658+0000 7fe409f39640 1 -- 172.21.15.70:0/3931985899 shutdown_connections 2023-10-08T05:49:57.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.658+0000 7fe409f39640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe40415cc60 0x7fe404157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.658+0000 7fe409f39640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4040cd3d0 0x7fe4040c7b40 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.658+0000 7fe409f39640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:6832/2692398351,v1:172.21.15.181:6833/2692398351] conn(0x7fe3e406b020 0x7fe3e406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.658+0000 7fe409f39640 1 --2- 172.21.15.70:0/3931985899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4040b4b30 0x7fe4040caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.658+0000 7fe409f39640 1 -- 172.21.15.70:0/3931985899 >> 172.21.15.70:0/3931985899 conn(0x7fe4040cf5f0 msgr2=0x7fe404160b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:57.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.658+0000 7fe409f39640 1 -- 172.21.15.70:0/3931985899 shutdown_connections 2023-10-08T05:49:57.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.658+0000 7fe409f39640 1 -- 172.21.15.70:0/3931985899 wait complete. 2023-10-08T05:49:57.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:49:57.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.875+0000 7fc092f0f640 1 Processor -- start 2023-10-08T05:49:57.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.875+0000 7fc092f0f640 1 -- start start 2023-10-08T05:49:57.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.877+0000 7fc092f0f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc08c0702b0 0x7fc08c070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.877+0000 7fc092f0f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0b21f0 0x7fc08c0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.877+0000 7fc092f0f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c06c3a0 con 0x7fc08c0ca1a0 2023-10-08T05:49:57.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.877+0000 7fc092f0f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08c070bd0 con 0x7fc08c0702b0 2023-10-08T05:49:57.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.877+0000 7fc092f0f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c06c640 con 0x7fc08c0b21f0 2023-10-08T05:49:57.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.877+0000 7fc090cab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc08c0702b0 0x7fc08c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.877+0000 7fc0914ac640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc08c0ca1a0 0x7fc08c06d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49792/0 (socket says 172.21.15.70:49792) 2023-10-08T05:49:57.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.877+0000 7fc0914ac640 1 -- 172.21.15.70:0/579541643 learned_addr learned my addr 172.21.15.70:0/579541643 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:49:57.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.878+0000 7fc08bfff640 1 --2- 172.21.15.70:0/579541643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0b21f0 0x7fc08c0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.878+0000 7fc090cab640 1 -- 172.21.15.70:0/579541643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0b21f0 msgr2=0x7fc08c0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:57.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.878+0000 7fc090cab640 1 --2- 172.21.15.70:0/579541643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0b21f0 0x7fc08c0c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.878+0000 7fc090cab640 1 -- 172.21.15.70:0/579541643 >> v1:172.21.15.70:6789/0 conn(0x7fc08c0ca1a0 legacy=0x7fc08c06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.878+0000 7fc08b7fe640 1 -- 172.21.15.70:0/579541643 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571488209 0 0) 0x7fc08c06c3a0 con 0x7fc08c0ca1a0 2023-10-08T05:49:57.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.878+0000 7fc090cab640 1 -- 172.21.15.70:0/579541643 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c06c920 con 0x7fc08c0702b0 2023-10-08T05:49:57.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.879+0000 7fc090cab640 1 --2- 172.21.15.70:0/579541643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0702b0 0x7fc08c070690 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc0800097a0 tx=0x7fc08002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8bd1564a8a7a93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.879+0000 7fc08b7fe640 1 -- 172.21.15.70:0/579541643 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080031830 con 0x7fc08c0702b0 2023-10-08T05:49:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.879+0000 7fc08b7fe640 1 -- 172.21.15.70:0/579541643 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc080031990 con 0x7fc08c0702b0 2023-10-08T05:49:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.879+0000 7fc08b7fe640 1 -- 172.21.15.70:0/579541643 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080031d20 con 0x7fc08c0702b0 2023-10-08T05:49:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.880+0000 7fc092f0f640 1 -- 172.21.15.70:0/579541643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0702b0 msgr2=0x7fc08c070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.880+0000 7fc092f0f640 1 --2- 172.21.15.70:0/579541643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0702b0 0x7fc08c070690 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc0800097a0 tx=0x7fc08002b8d0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.880+0000 7fc092f0f640 1 -- 172.21.15.70:0/579541643 shutdown_connections 2023-10-08T05:49:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.880+0000 7fc092f0f640 1 --2- 172.21.15.70:0/579541643 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0b21f0 0x7fc08c0c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.880+0000 7fc092f0f640 1 --2- 172.21.15.70:0/579541643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0702b0 0x7fc08c070690 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.880+0000 7fc092f0f640 1 -- 172.21.15.70:0/579541643 >> 172.21.15.70:0/579541643 conn(0x7fc08c06a5d0 msgr2=0x7fc08c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.880+0000 7fc092f0f640 1 -- 172.21.15.70:0/579541643 shutdown_connections 2023-10-08T05:49:57.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.880+0000 7fc092f0f640 1 -- 172.21.15.70:0/579541643 wait complete. 2023-10-08T05:49:57.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.882+0000 7fc092f0f640 1 Processor -- start 2023-10-08T05:49:57.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.882+0000 7fc092f0f640 1 -- start start 2023-10-08T05:49:57.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc092f0f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0702b0 0x7fc08c154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc092f0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0b21f0 0x7fc08c1e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc092f0f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc08c0ca1a0 0x7fc08c0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:57.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc092f0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c1e2c30 con 0x7fc08c0b21f0 2023-10-08T05:49:57.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc092f0f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c1e2670 con 0x7fc08c0ca1a0 2023-10-08T05:49:57.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc092f0f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c1e2950 con 0x7fc08c0702b0 2023-10-08T05:49:57.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc08bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0b21f0 0x7fc08c1e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc08bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0b21f0 0x7fc08c1e1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50566/0 (socket says 172.21.15.70:50566) 2023-10-08T05:49:57.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc08bfff640 1 -- 172.21.15.70:0/2235283826 learned_addr learned my addr 172.21.15.70:0/2235283826 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:57.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc08bfff640 1 -- 172.21.15.70:0/2235283826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0702b0 msgr2=0x7fc08c154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.883+0000 7fc090cab640 1 --2- 172.21.15.70:0/2235283826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0702b0 0x7fc08c154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc0914ac640 1 --2- 172.21.15.70:0/2235283826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc08c0ca1a0 0x7fc08c0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:57.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc08bfff640 1 --2- 172.21.15.70:0/2235283826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0702b0 0x7fc08c154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc08bfff640 1 -- 172.21.15.70:0/2235283826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc08c0ca1a0 msgr2=0x7fc08c0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:57.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc08bfff640 1 --2- 172.21.15.70:0/2235283826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc08c0ca1a0 0x7fc08c0cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:57.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc08bfff640 1 -- 172.21.15.70:0/2235283826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080009400 con 0x7fc08c0b21f0 2023-10-08T05:49:57.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc08bfff640 1 --2- 172.21.15.70:0/2235283826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0b21f0 0x7fc08c1e1d90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d800 tx=0x7fc07800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:57.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc0897fa640 1 -- 172.21.15.70:0/2235283826 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800b6c0 con 0x7fc08c0b21f0 2023-10-08T05:49:57.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc0897fa640 1 -- 172.21.15.70:0/2235283826 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07800b820 con 0x7fc08c0b21f0 2023-10-08T05:49:57.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc0897fa640 1 -- 172.21.15.70:0/2235283826 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800bbb0 con 0x7fc08c0b21f0 2023-10-08T05:49:57.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc092f0f640 1 -- 172.21.15.70:0/2235283826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c1e35f0 con 0x7fc08c0b21f0 2023-10-08T05:49:57.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.884+0000 7fc092f0f640 1 -- 172.21.15.70:0/2235283826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c1e22d0 con 0x7fc08c0b21f0 2023-10-08T05:49:57.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.885+0000 7fc092f0f640 1 -- 172.21.15.70:0/2235283826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c04e160 con 0x7fc08c0b21f0 2023-10-08T05:49:57.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.886+0000 7fc0897fa640 1 -- 172.21.15.70:0/2235283826 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 15) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fc07800bd10 con 0x7fc08c0b21f0 2023-10-08T05:49:57.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.887+0000 7fc0897fa640 1 -- 172.21.15.70:0/2235283826 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7067+0+0 (secure 0 0 0) 0x7fc078089c40 con 0x7fc08c0b21f0 2023-10-08T05:49:57.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:57.891+0000 7fc0897fa640 1 -- 172.21.15.70:0/2235283826 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc078054b00 con 0x7fc08c0b21f0 2023-10-08T05:49:58.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.077+0000 7fc092f0f640 1 -- 172.21.15.70:0/2235283826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc08c0c4fc0 con 0x7fc08c0b21f0 2023-10-08T05:49:58.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.078+0000 7fc0897fa640 1 -- 172.21.15.70:0/2235283826 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc078058120 con 0x7fc08c0b21f0 2023-10-08T05:49:58.078 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:49:58.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.080+0000 7fc092f0f640 1 -- 172.21.15.70:0/2235283826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0b21f0 msgr2=0x7fc08c1e1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:58.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.080+0000 7fc092f0f640 1 --2- 172.21.15.70:0/2235283826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0b21f0 0x7fc08c1e1d90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc07800d800 tx=0x7fc07800dcc0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.081+0000 7fc092f0f640 1 -- 172.21.15.70:0/2235283826 shutdown_connections 2023-10-08T05:49:58.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.081+0000 7fc092f0f640 1 --2- 172.21.15.70:0/2235283826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc08c0ca1a0 0x7fc08c0cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.081+0000 7fc092f0f640 1 --2- 172.21.15.70:0/2235283826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0b21f0 0x7fc08c1e1d90 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.081+0000 7fc092f0f640 1 --2- 172.21.15.70:0/2235283826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0702b0 0x7fc08c154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.081+0000 7fc092f0f640 1 -- 172.21.15.70:0/2235283826 >> 172.21.15.70:0/2235283826 conn(0x7fc08c06a5d0 msgr2=0x7fc08c0c6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:58.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.081+0000 7fc092f0f640 1 -- 172.21.15.70:0/2235283826 shutdown_connections 2023-10-08T05:49:58.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.081+0000 7fc092f0f640 1 -- 172.21.15.70:0/2235283826 wait complete. 2023-10-08T05:49:58.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:49:58.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.316+0000 7f0b24a19640 1 Processor -- start 2023-10-08T05:49:58.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.317+0000 7f0b24a19640 1 -- start start 2023-10-08T05:49:58.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.318+0000 7f0b24a19640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b2015ccd0 0x7f0b200ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.318+0000 7f0b24a19640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0b2006bcb0 0x7f0b20160da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.318+0000 7f0b24a19640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b200ce9b0 con 0x7f0b201612e0 2023-10-08T05:49:58.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.318+0000 7f0b24a19640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b200cec90 con 0x7f0b2006bcb0 2023-10-08T05:49:58.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.318+0000 7f0b24a19640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200cef70 con 0x7f0b2015ccd0 2023-10-08T05:49:58.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.319+0000 7f0b1ed9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b2015ccd0 0x7f0b200ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.319+0000 7f0b1ed9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b2015ccd0 0x7f0b200ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53010/0 (socket says 172.21.15.70:53010) 2023-10-08T05:49:58.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.319+0000 7f0b1ed9d640 1 -- 172.21.15.70:0/4120212767 learned_addr learned my addr 172.21.15.70:0/4120212767 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:58.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.319+0000 7f0b1e59c640 1 --2- 172.21.15.70:0/4120212767 >> v2:172.21.15.181:3300/0 conn(0x7f0b2006bcb0 0x7f0b20160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.320+0000 7f0b1dd9b640 1 -- 172.21.15.70:0/4120212767 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180274810 0 0) 0x7f0b200ce9b0 con 0x7f0b201612e0 2023-10-08T05:49:58.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.320+0000 7f0b1dd9b640 1 -- 172.21.15.70:0/4120212767 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b08003520 con 0x7f0b201612e0 2023-10-08T05:49:58.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.320+0000 7f0b1ed9d640 1 -- 172.21.15.70:0/4120212767 >> v2:172.21.15.181:3300/0 conn(0x7f0b2006bcb0 msgr2=0x7f0b20160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:58.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.320+0000 7f0b1ed9d640 1 --2- 172.21.15.70:0/4120212767 >> v2:172.21.15.181:3300/0 conn(0x7f0b2006bcb0 0x7f0b20160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.320+0000 7f0b1ed9d640 1 -- 172.21.15.70:0/4120212767 >> v1:172.21.15.181:6789/0 conn(0x7f0b201612e0 legacy=0x7f0b200b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:58.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.320+0000 7f0b1ed9d640 1 -- 172.21.15.70:0/4120212767 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b20160880 con 0x7f0b2015ccd0 2023-10-08T05:49:58.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.320+0000 7f0b1e59c640 1 --2- 172.21.15.70:0/4120212767 >> v2:172.21.15.181:3300/0 conn(0x7f0b2006bcb0 0x7f0b20160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:49:58.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.322+0000 7f0b1ed9d640 1 --2- 172.21.15.70:0/4120212767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b2015ccd0 0x7f0b200ce2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009e50 tx=0x7f0b1002bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5be33017d14993ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:58.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.322+0000 7f0b1dd9b640 1 -- 172.21.15.70:0/4120212767 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1002caf0 con 0x7f0b2015ccd0 2023-10-08T05:49:58.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.322+0000 7f0b1dd9b640 1 -- 172.21.15.70:0/4120212767 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1002cc50 con 0x7f0b2015ccd0 2023-10-08T05:49:58.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.322+0000 7f0b1dd9b640 1 -- 172.21.15.70:0/4120212767 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100343f0 con 0x7f0b2015ccd0 2023-10-08T05:49:58.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.323+0000 7f0b24a19640 1 -- 172.21.15.70:0/4120212767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b2015ccd0 msgr2=0x7f0b200ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:58.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.323+0000 7f0b24a19640 1 --2- 172.21.15.70:0/4120212767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b2015ccd0 0x7f0b200ce2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009e50 tx=0x7f0b1002bf50 comp rx=0 tx=0).stop 2023-10-08T05:49:58.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.323+0000 7f0b24a19640 1 -- 172.21.15.70:0/4120212767 shutdown_connections 2023-10-08T05:49:58.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.323+0000 7f0b24a19640 1 --2- 172.21.15.70:0/4120212767 >> v2:172.21.15.181:3300/0 conn(0x7f0b2006bcb0 0x7f0b20160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.323+0000 7f0b24a19640 1 --2- 172.21.15.70:0/4120212767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b2015ccd0 0x7f0b200ce2d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.323+0000 7f0b24a19640 1 -- 172.21.15.70:0/4120212767 >> 172.21.15.70:0/4120212767 conn(0x7f0b200cd180 msgr2=0x7f0b200cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:58.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.323+0000 7f0b24a19640 1 -- 172.21.15.70:0/4120212767 shutdown_connections 2023-10-08T05:49:58.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.323+0000 7f0b24a19640 1 -- 172.21.15.70:0/4120212767 wait complete. 2023-10-08T05:49:58.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.325+0000 7f0b24a19640 1 Processor -- start 2023-10-08T05:49:58.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.325+0000 7f0b24a19640 1 -- start start 2023-10-08T05:49:58.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.326+0000 7f0b24a19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b2006bcb0 0x7f0b201f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.326+0000 7f0b24a19640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b2015ccd0 0x7f0b2015f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.326+0000 7f0b24a19640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b201612e0 0x7f0b20156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.326+0000 7f0b24a19640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2015ab90 con 0x7f0b2006bcb0 2023-10-08T05:49:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.326+0000 7f0b24a19640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b200b3190 con 0x7f0b2015ccd0 2023-10-08T05:49:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.326+0000 7f0b24a19640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b201f2f50 con 0x7f0b201612e0 2023-10-08T05:49:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.326+0000 7f0b1ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b2006bcb0 0x7f0b201f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.326+0000 7f0b1ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b2006bcb0 0x7f0b201f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50580/0 (socket says 172.21.15.70:50580) 2023-10-08T05:49:58.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.326+0000 7f0b1ed9d640 1 -- 172.21.15.70:0/497011923 learned_addr learned my addr 172.21.15.70:0/497011923 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:58.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.327+0000 7f0b1ed9d640 1 -- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b201612e0 msgr2=0x7f0b20156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:58.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.327+0000 7f0b1ed9d640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b201612e0 0x7f0b20156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.327+0000 7f0b1ed9d640 1 -- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b2015ccd0 msgr2=0x7f0b2015f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:58.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.327+0000 7f0b1ed9d640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b2015ccd0 0x7f0b2015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.327+0000 7f0b1ed9d640 1 -- 172.21.15.70:0/497011923 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b14009400 con 0x7f0b2006bcb0 2023-10-08T05:49:58.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.327+0000 7f0b1ed9d640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b2006bcb0 0x7f0b201f2a10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000f80 tx=0x7f0b10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:58.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.327+0000 7f0b077fe640 1 -- 172.21.15.70:0/497011923 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10009510 con 0x7f0b2006bcb0 2023-10-08T05:49:58.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.327+0000 7f0b077fe640 1 -- 172.21.15.70:0/497011923 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1003e450 con 0x7f0b2006bcb0 2023-10-08T05:49:58.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.327+0000 7f0b077fe640 1 -- 172.21.15.70:0/497011923 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1003e800 con 0x7f0b2006bcb0 2023-10-08T05:49:58.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.328+0000 7f0b24a19640 1 -- 172.21.15.70:0/497011923 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b10009b30 con 0x7f0b2006bcb0 2023-10-08T05:49:58.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.328+0000 7f0b24a19640 1 -- 172.21.15.70:0/497011923 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2015b0c0 con 0x7f0b2006bcb0 2023-10-08T05:49:58.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.329+0000 7f0b24a19640 1 -- 172.21.15.70:0/497011923 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b20074150 con 0x7f0b2006bcb0 2023-10-08T05:49:58.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.330+0000 7f0b077fe640 1 -- 172.21.15.70:0/497011923 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 15) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f0b1003e960 con 0x7f0b2006bcb0 2023-10-08T05:49:58.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.331+0000 7f0b077fe640 1 -- 172.21.15.70:0/497011923 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7067+0+0 (secure 0 0 0) 0x7f0b100ac250 con 0x7f0b2006bcb0 2023-10-08T05:49:58.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.336+0000 7f0b077fe640 1 -- 172.21.15.70:0/497011923 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b10077350 con 0x7f0b2006bcb0 2023-10-08T05:49:58.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.536+0000 7f0b077fe640 1 -- 172.21.15.70:0/497011923 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 16) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f0b10076e00 con 0x7f0b2006bcb0 2023-10-08T05:49:58.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.538+0000 7f0b077fe640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0b0c070f60 0x7f0b0c073350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.539+0000 7f0b24a19640 1 -- 172.21.15.70:0/497011923 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b200cdf40 con 0x7f0b2006bcb0 2023-10-08T05:49:58.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.540+0000 7f0b1e59c640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0b0c070f60 0x7f0b0c073350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.540+0000 7f0b077fe640 1 -- 172.21.15.70:0/497011923 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b10091d50 con 0x7f0b2006bcb0 2023-10-08T05:49:58.541 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:49:58.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.543+0000 7f0b24a19640 1 -- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0b0c070f60 msgr2=0x7f0b0c073350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:58.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.543+0000 7f0b24a19640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0b0c070f60 0x7f0b0c073350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.543+0000 7f0b24a19640 1 -- 172.21.15.70:0/497011923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b2006bcb0 msgr2=0x7f0b201f2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:58.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.543+0000 7f0b24a19640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b2006bcb0 0x7f0b201f2a10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0b10000f80 tx=0x7f0b10004a40 comp rx=0 tx=0).stop 2023-10-08T05:49:58.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.544+0000 7f0b24a19640 1 -- 172.21.15.70:0/497011923 shutdown_connections 2023-10-08T05:49:58.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.544+0000 7f0b24a19640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0b0c070f60 0x7f0b0c073350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.544+0000 7f0b24a19640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b201612e0 0x7f0b20156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.544+0000 7f0b24a19640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b2015ccd0 0x7f0b2015f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.544+0000 7f0b24a19640 1 --2- 172.21.15.70:0/497011923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b2006bcb0 0x7f0b201f2a10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.544+0000 7f0b24a19640 1 -- 172.21.15.70:0/497011923 >> 172.21.15.70:0/497011923 conn(0x7f0b200cd180 msgr2=0x7f0b20073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:58.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.544+0000 7f0b24a19640 1 -- 172.21.15.70:0/497011923 shutdown_connections 2023-10-08T05:49:58.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.544+0000 7f0b24a19640 1 -- 172.21.15.70:0/497011923 wait complete. 2023-10-08T05:49:58.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:49:58.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.764+0000 7f8151ca9640 1 Processor -- start 2023-10-08T05:49:58.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.764+0000 7f8151ca9640 1 -- start start 2023-10-08T05:49:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f8151ca9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 0x7f814c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f8151ca9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f814c0c90b0 0x7f814c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f8151ca9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f814c0cae70 0x7f814c0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f8151ca9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c15bf30 con 0x7f814c0cae70 2023-10-08T05:49:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f8151ca9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c0cb840 con 0x7f814c0c90b0 2023-10-08T05:49:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f8151ca9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c15b2b0 con 0x7f814c15ab40 2023-10-08T05:49:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f814bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f814c0cae70 0x7f814c0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f814bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f814c0cae70 0x7f814c0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50582/0 (socket says 172.21.15.70:50582) 2023-10-08T05:49:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f814bfff640 1 -- 172.21.15.70:0/547623197 learned_addr learned my addr 172.21.15.70:0/547623197 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f814bfff640 1 -- 172.21.15.70:0/547623197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 msgr2=0x7f814c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f814b7fe640 1 --2- 172.21.15.70:0/547623197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 0x7f814c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f8143fff640 1 --2- 172.21.15.70:0/547623197 >> v2:172.21.15.181:3300/0 conn(0x7f814c0c90b0 0x7f814c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f814bfff640 1 --2- 172.21.15.70:0/547623197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 0x7f814c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f814bfff640 1 -- 172.21.15.70:0/547623197 >> v2:172.21.15.181:3300/0 conn(0x7f814c0c90b0 msgr2=0x7f814c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.766+0000 7f814bfff640 1 --2- 172.21.15.70:0/547623197 >> v2:172.21.15.181:3300/0 conn(0x7f814c0c90b0 0x7f814c0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.767+0000 7f814bfff640 1 -- 172.21.15.70:0/547623197 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c15b590 con 0x7f814c0cae70 2023-10-08T05:49:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.767+0000 7f814b7fe640 1 --2- 172.21.15.70:0/547623197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 0x7f814c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:49:58.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.768+0000 7f814bfff640 1 --2- 172.21.15.70:0/547623197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 0x7f814c0cb300 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f814c06e470 tx=0x7f814402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cad253ea41dc958 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:58.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.768+0000 7f814affd640 1 -- 172.21.15.70:0/547623197 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814402ca80 con 0x7f814c0cae70 2023-10-08T05:49:58.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.768+0000 7f814affd640 1 -- 172.21.15.70:0/547623197 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814402cbe0 con 0x7f814c0cae70 2023-10-08T05:49:58.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.768+0000 7f814affd640 1 -- 172.21.15.70:0/547623197 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81440343f0 con 0x7f814c0cae70 2023-10-08T05:49:58.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.769+0000 7f8151ca9640 1 -- 172.21.15.70:0/547623197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 msgr2=0x7f814c0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:58.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.769+0000 7f8151ca9640 1 --2- 172.21.15.70:0/547623197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 0x7f814c0cb300 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f814c06e470 tx=0x7f814402bee0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.769+0000 7f8151ca9640 1 -- 172.21.15.70:0/547623197 shutdown_connections 2023-10-08T05:49:58.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.769+0000 7f8151ca9640 1 --2- 172.21.15.70:0/547623197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 0x7f814c0cb300 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.769+0000 7f8151ca9640 1 --2- 172.21.15.70:0/547623197 >> v2:172.21.15.181:3300/0 conn(0x7f814c0c90b0 0x7f814c0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.769+0000 7f8151ca9640 1 --2- 172.21.15.70:0/547623197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 0x7f814c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.769+0000 7f8151ca9640 1 -- 172.21.15.70:0/547623197 >> 172.21.15.70:0/547623197 conn(0x7f814c06a380 msgr2=0x7f814c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:58.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.769+0000 7f8151ca9640 1 -- 172.21.15.70:0/547623197 shutdown_connections 2023-10-08T05:49:58.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.769+0000 7f8151ca9640 1 -- 172.21.15.70:0/547623197 wait complete. 2023-10-08T05:49:58.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.771+0000 7f8151ca9640 1 Processor -- start 2023-10-08T05:49:58.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.771+0000 7f8151ca9640 1 -- start start 2023-10-08T05:49:58.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.772+0000 7f8151ca9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c0c90b0 0x7f814c150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.772+0000 7f8151ca9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 0x7f814c0c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.772+0000 7f8151ca9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 0x7f814c0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.772+0000 7f8151ca9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0b2510 con 0x7f814c0cae70 2023-10-08T05:49:58.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.772+0000 7f8151ca9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0b29a0 con 0x7f814c0c90b0 2023-10-08T05:49:58.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.772+0000 7f8151ca9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c0b1900 con 0x7f814c15ab40 2023-10-08T05:49:58.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.772+0000 7f8143fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 0x7f814c0c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.772+0000 7f8143fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 0x7f814c0c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50598/0 (socket says 172.21.15.70:50598) 2023-10-08T05:49:58.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.772+0000 7f8143fff640 1 -- 172.21.15.70:0/1357606196 learned_addr learned my addr 172.21.15.70:0/1357606196 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:58.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.772+0000 7f814b7fe640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c0c90b0 0x7f814c150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f814bfff640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 0x7f814c0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8143fff640 1 -- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 msgr2=0x7f814c0c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:58.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8143fff640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 0x7f814c0c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8143fff640 1 -- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c0c90b0 msgr2=0x7f814c150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:58.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8143fff640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c0c90b0 0x7f814c150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:58.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8143fff640 1 -- 172.21.15.70:0/1357606196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8144009b30 con 0x7f814c0cae70 2023-10-08T05:49:58.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8143fff640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 0x7f814c0c5030 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f813400c5a0 tx=0x7f813400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:58.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8148ff9640 1 -- 172.21.15.70:0/1357606196 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134004220 con 0x7f814c0cae70 2023-10-08T05:49:58.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8148ff9640 1 -- 172.21.15.70:0/1357606196 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8134004380 con 0x7f814c0cae70 2023-10-08T05:49:58.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8148ff9640 1 -- 172.21.15.70:0/1357606196 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134003850 con 0x7f814c0cae70 2023-10-08T05:49:58.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8151ca9640 1 -- 172.21.15.70:0/1357606196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c0b1be0 con 0x7f814c0cae70 2023-10-08T05:49:58.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.773+0000 7f8151ca9640 1 -- 172.21.15.70:0/1357606196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c0b21a0 con 0x7f814c0cae70 2023-10-08T05:49:58.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.775+0000 7f8151ca9640 1 -- 172.21.15.70:0/1357606196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814c152590 con 0x7f814c0cae70 2023-10-08T05:49:58.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.775+0000 7f8148ff9640 1 -- 172.21.15.70:0/1357606196 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f81340039b0 con 0x7f814c0cae70 2023-10-08T05:49:58.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.776+0000 7f8148ff9640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f812c06acc0 0x7f812c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:58.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.776+0000 7f8148ff9640 1 -- 172.21.15.70:0/1357606196 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f81340894b0 con 0x7f814c0cae70 2023-10-08T05:49:58.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.777+0000 7f814b7fe640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f812c06acc0 0x7f812c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:58.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.778+0000 7f814b7fe640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f812c06acc0 0x7f812c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f813c00a990 tx=0x7f813c008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:58.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:58.781+0000 7f8148ff9640 1 -- 172.21.15.70:0/1357606196 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81340542d0 con 0x7f814c0cae70 2023-10-08T05:49:59.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.015+0000 7f8151ca9640 1 -- 172.21.15.70:0/1357606196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f814c067490 con 0x7f814c0cae70 2023-10-08T05:49:59.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.016+0000 7f8148ff9640 1 -- 172.21.15.70:0/1357606196 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f81340578f0 con 0x7f814c0cae70 2023-10-08T05:49:59.017 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:59.017 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/809656839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":809656839},{"type":"v1","addr":"172.21.15.70:6835","nonce":809656839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2750508216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2750508216},{"type":"v1","addr":"172.21.15.181:6836","nonce":2750508216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/607266140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":607266140},{"type":"v1","addr":"172.21.15.181:6837","nonce":607266140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:37:44.192492+0000","modified":"2023-10-08T05:37:47.238340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/480502229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":480502229},{"type":"v1","addr":"172.21.15.70:6834","nonce":480502229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-08T05:49:59.017 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 18 2023-10-08T05:49:59.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.018+0000 7f8151ca9640 1 -- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f812c06acc0 msgr2=0x7f812c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.018+0000 7f8151ca9640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f812c06acc0 0x7f812c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f813c00a990 tx=0x7f813c008040 comp rx=0 tx=0).stop 2023-10-08T05:49:59.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.019+0000 7f8151ca9640 1 -- 172.21.15.70:0/1357606196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 msgr2=0x7f814c0c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.019+0000 7f8151ca9640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 0x7f814c0c5030 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f813400c5a0 tx=0x7f813400ca60 comp rx=0 tx=0).stop 2023-10-08T05:49:59.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.019+0000 7f8151ca9640 1 -- 172.21.15.70:0/1357606196 shutdown_connections 2023-10-08T05:49:59.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.019+0000 7f8151ca9640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f812c06acc0 0x7f812c06d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.019+0000 7f8151ca9640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f814c15ab40 0x7f814c0c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.019+0000 7f8151ca9640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f814c0cae70 0x7f814c0c5030 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.019+0000 7f8151ca9640 1 --2- 172.21.15.70:0/1357606196 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f814c0c90b0 0x7f814c150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.019+0000 7f8151ca9640 1 -- 172.21.15.70:0/1357606196 >> 172.21.15.70:0/1357606196 conn(0x7f814c06a380 msgr2=0x7f814c0ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:59.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.019+0000 7f8151ca9640 1 -- 172.21.15.70:0/1357606196 shutdown_connections 2023-10-08T05:49:59.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.019+0000 7f8151ca9640 1 -- 172.21.15.70:0/1357606196 wait complete. 2023-10-08T05:49:59.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:49:59.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.249+0000 7f7393090640 1 Processor -- start 2023-10-08T05:49:59.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.250+0000 7f7393090640 1 -- start start 2023-10-08T05:49:59.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.251+0000 7f7393090640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f738c0add60 0x7f738c162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.251+0000 7f7393090640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c0ae140 con 0x7f738c06d4c0 2023-10-08T05:49:59.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.251+0000 7f7393090640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738c0ccf20 con 0x7f738c0cb150 2023-10-08T05:49:59.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.251+0000 7f7393090640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c161f10 con 0x7f738c0add60 2023-10-08T05:49:59.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.252+0000 7f7390e2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f738c0add60 0x7f738c162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.252+0000 7f7390e2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f738c0add60 0x7f738c162330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50604/0 (socket says 172.21.15.70:50604) 2023-10-08T05:49:59.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.252+0000 7f7390e2c640 1 -- 172.21.15.70:0/3125611888 learned_addr learned my addr 172.21.15.70:0/3125611888 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.252+0000 7f7390e2c640 1 -- 172.21.15.70:0/3125611888 >> v1:172.21.15.181:6789/0 conn(0x7f738c0cb150 legacy=0x7f738c164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.252+0000 7f7390e2c640 1 -- 172.21.15.70:0/3125611888 >> v1:172.21.15.70:6789/0 conn(0x7f738c06d4c0 legacy=0x7f738c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.252+0000 7f7390e2c640 1 -- 172.21.15.70:0/3125611888 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c0cd1c0 con 0x7f738c0add60 2023-10-08T05:49:59.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.253+0000 7f7390e2c640 1 --2- 172.21.15.70:0/3125611888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0add60 0x7f738c162330 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f737c009e50 tx=0x7f737c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959999b95812c11e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:59.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.254+0000 7f738b7fe640 1 -- 172.21.15.70:0/3125611888 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c02ca80 con 0x7f738c0add60 2023-10-08T05:49:59.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.254+0000 7f738b7fe640 1 -- 172.21.15.70:0/3125611888 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c02cbe0 con 0x7f738c0add60 2023-10-08T05:49:59.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.254+0000 7f738b7fe640 1 -- 172.21.15.70:0/3125611888 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c0343f0 con 0x7f738c0add60 2023-10-08T05:49:59.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.254+0000 7f7393090640 1 -- 172.21.15.70:0/3125611888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0add60 msgr2=0x7f738c162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.254+0000 7f7393090640 1 --2- 172.21.15.70:0/3125611888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0add60 0x7f738c162330 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f737c009e50 tx=0x7f737c02bee0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.254+0000 7f7393090640 1 -- 172.21.15.70:0/3125611888 shutdown_connections 2023-10-08T05:49:59.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.254+0000 7f7393090640 1 --2- 172.21.15.70:0/3125611888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0add60 0x7f738c162330 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.254+0000 7f7393090640 1 -- 172.21.15.70:0/3125611888 >> 172.21.15.70:0/3125611888 conn(0x7f738c06a380 msgr2=0x7f738c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:59.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.255+0000 7f7393090640 1 -- 172.21.15.70:0/3125611888 shutdown_connections 2023-10-08T05:49:59.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.255+0000 7f7393090640 1 -- 172.21.15.70:0/3125611888 wait complete. 2023-10-08T05:49:59.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.256+0000 7f7393090640 1 Processor -- start 2023-10-08T05:49:59.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.256+0000 7f7393090640 1 -- start start 2023-10-08T05:49:59.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.257+0000 7f7393090640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c06d4c0 0x7f738c155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.257+0000 7f7393090640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f738c0add60 0x7f738c1570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.257+0000 7f7393090640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0cb150 0x7f738c15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.257+0000 7f7393090640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c15e380 con 0x7f738c0cb150 2023-10-08T05:49:59.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.257+0000 7f7393090640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c15d4b0 con 0x7f738c06d4c0 2023-10-08T05:49:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.257+0000 7f7393090640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c155b90 con 0x7f738c0add60 2023-10-08T05:49:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.258+0000 7f739162d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0cb150 0x7f738c15e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.258+0000 7f739162d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0cb150 0x7f738c15e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50620/0 (socket says 172.21.15.70:50620) 2023-10-08T05:49:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.258+0000 7f739162d640 1 -- 172.21.15.70:0/2480846508 learned_addr learned my addr 172.21.15.70:0/2480846508 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.258+0000 7f7390e2c640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c06d4c0 0x7f738c155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.258+0000 7f739162d640 1 -- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f738c0add60 msgr2=0x7f738c1570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.258+0000 7f738bfff640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f738c0add60 0x7f738c1570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.258+0000 7f739162d640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f738c0add60 0x7f738c1570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.258+0000 7f739162d640 1 -- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c06d4c0 msgr2=0x7f738c155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.258+0000 7f739162d640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c06d4c0 0x7f738c155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.258+0000 7f739162d640 1 -- 172.21.15.70:0/2480846508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c009b30 con 0x7f738c0cb150 2023-10-08T05:49:59.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.259+0000 7f739162d640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0cb150 0x7f738c15e7a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f738400c410 tx=0x7f738400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:59.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.259+0000 7f73897fa640 1 -- 172.21.15.70:0/2480846508 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384011790 con 0x7f738c0cb150 2023-10-08T05:49:59.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.259+0000 7f73897fa640 1 -- 172.21.15.70:0/2480846508 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73840118f0 con 0x7f738c0cb150 2023-10-08T05:49:59.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.259+0000 7f73897fa640 1 -- 172.21.15.70:0/2480846508 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384011ce0 con 0x7f738c0cb150 2023-10-08T05:49:59.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.259+0000 7f7393090640 1 -- 172.21.15.70:0/2480846508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c15d700 con 0x7f738c0cb150 2023-10-08T05:49:59.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.259+0000 7f7393090640 1 -- 172.21.15.70:0/2480846508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c15dcc0 con 0x7f738c0cb150 2023-10-08T05:49:59.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.260+0000 7f7393090640 1 -- 172.21.15.70:0/2480846508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738c15e4c0 con 0x7f738c0cb150 2023-10-08T05:49:59.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.261+0000 7f73897fa640 1 -- 172.21.15.70:0/2480846508 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 16) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f738401c460 con 0x7f738c0cb150 2023-10-08T05:49:59.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.262+0000 7f73897fa640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f737806acc0 0x7f737806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.262+0000 7f73897fa640 1 -- 172.21.15.70:0/2480846508 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f73840894a0 con 0x7f738c0cb150 2023-10-08T05:49:59.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.263+0000 7f7390e2c640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f737806acc0 0x7f737806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.264+0000 7f7390e2c640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f737806acc0 0x7f737806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f737c035000 tx=0x7f737c030000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:59.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.266+0000 7f73897fa640 1 -- 172.21.15.70:0/2480846508 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f738c15e4c0 con 0x7f738c0cb150 2023-10-08T05:49:59.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.457+0000 7f7393090640 1 -- 172.21.15.70:0/2480846508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f738c15e4c0 con 0x7f738c0cb150 2023-10-08T05:49:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.460+0000 7f73897fa640 1 -- 172.21.15.70:0/2480846508 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f738c15e4c0 con 0x7f738c0cb150 2023-10-08T05:49:59.461 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:59.461 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":59,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:49:58.500315+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:37:41.960853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-08T05:37:43.190325+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":59,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/32320998":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/2533276123":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/3573082646":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3583094356":"2023-10-09T05:38:04.171728+0000","172.21.15.181:6832/2737793492":"2023-10-09T05:38:04.171728+0000","172.21.15.181:6833/2737793492":"2023-10-09T05:38:04.171728+0000","172.21.15.181:6832/2692398351":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/3620192672":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3280597969":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/756067864":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/437302488":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/2139016714":"2023-10-09T05:49:57.491954+0000","172.21.15.181:6833/2692398351":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/1816930869":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/995976827":"2023-10-09T05:49:57.491954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":9,"length":1},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":24,"length":1},{"begin":30,"length":1},{"begin":38,"length":2},{"begin":45,"length":3},{"begin":53,"length":2},{"begin":57,"length":1},{"begin":62,"length":1},{"begin":64,"length":1},{"begin":66,"length":1},{"begin":73,"length":2},{"begin":79,"length":1},{"begin":86,"length":1},{"begin":88,"length":2},{"begin":94,"length":1},{"begin":96,"length":1},{"begin":99,"length":1},{"begin":103,"length":1},{"begin":107,"length":2},{"begin":193,"length":29}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:49:59.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 -- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f737806acc0 msgr2=0x7f737806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f737806acc0 0x7f737806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f737c035000 tx=0x7f737c030000 comp rx=0 tx=0).stop 2023-10-08T05:49:59.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 -- 172.21.15.70:0/2480846508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0cb150 msgr2=0x7f738c15e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0cb150 0x7f738c15e7a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f738400c410 tx=0x7f738400c8d0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 -- 172.21.15.70:0/2480846508 shutdown_connections 2023-10-08T05:49:59.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738c0cb150 0x7f738c15e7a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f737806acc0 0x7f737806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f738c0add60 0x7f738c1570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 --2- 172.21.15.70:0/2480846508 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f738c06d4c0 0x7f738c155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 -- 172.21.15.70:0/2480846508 >> 172.21.15.70:0/2480846508 conn(0x7f738c06a380 msgr2=0x7f738c1629d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:59.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 -- 172.21.15.70:0/2480846508 shutdown_connections 2023-10-08T05:49:59.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.463+0000 7f7393090640 1 -- 172.21.15.70:0/2480846508 wait complete. 2023-10-08T05:49:59.483 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T05:49:59.483 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:49:59.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.698+0000 7f3721c4f640 1 Processor -- start 2023-10-08T05:49:59.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.698+0000 7f3721c4f640 1 -- start start 2023-10-08T05:49:59.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.700+0000 7f3721c4f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f371c160940 0x7f371c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.700+0000 7f3721c4f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f371c06c890 0x7f371c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.700+0000 7f3721c4f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f371c0c48f0 con 0x7f371c15cd90 2023-10-08T05:49:59.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.700+0000 7f3721c4f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c0c4d80 con 0x7f371c160940 2023-10-08T05:49:59.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.700+0000 7f3721c4f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f371c0c3ce0 con 0x7f371c06c890 2023-10-08T05:49:59.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.700+0000 7f3713fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f371c160940 0x7f371c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.701+0000 7f3713fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f371c160940 0x7f371c0709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50632/0 (socket says 172.21.15.70:50632) 2023-10-08T05:49:59.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.701+0000 7f3713fff640 1 -- 172.21.15.70:0/608743005 learned_addr learned my addr 172.21.15.70:0/608743005 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:59.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.701+0000 7f371bfff640 1 --2- 172.21.15.70:0/608743005 >> v2:172.21.15.181:3300/0 conn(0x7f371c06c890 0x7f371c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.701+0000 7f3713fff640 1 -- 172.21.15.70:0/608743005 >> v2:172.21.15.181:3300/0 conn(0x7f371c06c890 msgr2=0x7f371c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:59.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.701+0000 7f3713fff640 1 --2- 172.21.15.70:0/608743005 >> v2:172.21.15.181:3300/0 conn(0x7f371c06c890 0x7f371c06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.701+0000 7f3713fff640 1 -- 172.21.15.70:0/608743005 >> v1:172.21.15.181:6789/0 conn(0x7f371c15cd90 legacy=0x7f371c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:49:59.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.701+0000 7f3713fff640 1 -- 172.21.15.70:0/608743005 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c0c3fc0 con 0x7f371c160940 2023-10-08T05:49:59.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.702+0000 7f3713fff640 1 --2- 172.21.15.70:0/608743005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 0x7f371c0709e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f371c0c8d60 tx=0x7f370402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07648e589824ba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:59.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.703+0000 7f371affd640 1 -- 172.21.15.70:0/608743005 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704030860 con 0x7f371c160940 2023-10-08T05:49:59.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.703+0000 7f371affd640 1 -- 172.21.15.70:0/608743005 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37040309c0 con 0x7f371c160940 2023-10-08T05:49:59.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.703+0000 7f371affd640 1 -- 172.21.15.70:0/608743005 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704030d50 con 0x7f371c160940 2023-10-08T05:49:59.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.704+0000 7f3721c4f640 1 -- 172.21.15.70:0/608743005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 msgr2=0x7f371c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.704+0000 7f3721c4f640 1 --2- 172.21.15.70:0/608743005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 0x7f371c0709e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f371c0c8d60 tx=0x7f370402b9a0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.704+0000 7f3721c4f640 1 -- 172.21.15.70:0/608743005 shutdown_connections 2023-10-08T05:49:59.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.704+0000 7f3721c4f640 1 --2- 172.21.15.70:0/608743005 >> v2:172.21.15.181:3300/0 conn(0x7f371c06c890 0x7f371c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.704+0000 7f3721c4f640 1 --2- 172.21.15.70:0/608743005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 0x7f371c0709e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.704+0000 7f3721c4f640 1 -- 172.21.15.70:0/608743005 >> 172.21.15.70:0/608743005 conn(0x7f371c06a380 msgr2=0x7f371c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:59.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.704+0000 7f3721c4f640 1 -- 172.21.15.70:0/608743005 shutdown_connections 2023-10-08T05:49:59.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.704+0000 7f3721c4f640 1 -- 172.21.15.70:0/608743005 wait complete. 2023-10-08T05:49:59.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.706+0000 7f3721c4f640 1 Processor -- start 2023-10-08T05:49:59.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.706+0000 7f3721c4f640 1 -- start start 2023-10-08T05:49:59.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.707+0000 7f3721c4f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f371c06c890 0x7f371c156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.707+0000 7f3721c4f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f371c15cd90 0x7f371c15d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.707+0000 7f3721c4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 0x7f371c0b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.707+0000 7f3721c4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c1568d0 con 0x7f371c160940 2023-10-08T05:49:59.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.707+0000 7f3721c4f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371c1f2e70 con 0x7f371c06c890 2023-10-08T05:49:59.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.707+0000 7f3721c4f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c1f3150 con 0x7f371c15cd90 2023-10-08T05:49:59.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f371bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 0x7f371c0b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f371bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 0x7f371c0b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50638/0 (socket says 172.21.15.70:50638) 2023-10-08T05:49:59.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f371bfff640 1 -- 172.21.15.70:0/199539658 learned_addr learned my addr 172.21.15.70:0/199539658 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:49:59.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f371b7fe640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f371c06c890 0x7f371c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f3713fff640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f371c15cd90 0x7f371c15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f371bfff640 1 -- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f371c15cd90 msgr2=0x7f371c15d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f371bfff640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f371c15cd90 0x7f371c15d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f371bfff640 1 -- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f371c06c890 msgr2=0x7f371c156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f371bfff640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f371c06c890 0x7f371c156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f371bfff640 1 -- 172.21.15.70:0/199539658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37040094d0 con 0x7f371c160940 2023-10-08T05:49:59.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.708+0000 7f371bfff640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 0x7f371c0b33e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3714007e40 tx=0x7f371400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:59.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.709+0000 7f3718ff9640 1 -- 172.21.15.70:0/199539658 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3714014620 con 0x7f371c160940 2023-10-08T05:49:59.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.709+0000 7f3718ff9640 1 -- 172.21.15.70:0/199539658 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371400ee90 con 0x7f371c160940 2023-10-08T05:49:59.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.709+0000 7f3718ff9640 1 -- 172.21.15.70:0/199539658 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37140149e0 con 0x7f371c160940 2023-10-08T05:49:59.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.709+0000 7f3721c4f640 1 -- 172.21.15.70:0/199539658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c1f3430 con 0x7f371c160940 2023-10-08T05:49:59.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.709+0000 7f3721c4f640 1 -- 172.21.15.70:0/199539658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f371c1f39f0 con 0x7f371c160940 2023-10-08T05:49:59.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.710+0000 7f3721c4f640 1 -- 172.21.15.70:0/199539658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371c067490 con 0x7f371c160940 2023-10-08T05:49:59.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.711+0000 7f3718ff9640 1 -- 172.21.15.70:0/199539658 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3714014b40 con 0x7f371c160940 2023-10-08T05:49:59.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.712+0000 7f3718ff9640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f36fc06af00 0x7f36fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:49:59.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.712+0000 7f3718ff9640 1 -- 172.21.15.70:0/199539658 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f371408a6e0 con 0x7f371c160940 2023-10-08T05:49:59.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.712+0000 7f371b7fe640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f36fc06af00 0x7f36fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:49:59.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.714+0000 7f371b7fe640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f36fc06af00 0x7f36fc06d3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f371c0b3f30 tx=0x7f370c009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:49:59.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.716+0000 7f3718ff9640 1 -- 172.21.15.70:0/199539658 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3714055110 con 0x7f371c160940 2023-10-08T05:49:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.948+0000 7f3721c4f640 1 -- 172.21.15.70:0/199539658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f371c0c4270 con 0x7f371c160940 2023-10-08T05:49:59.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.949+0000 7f3718ff9640 1 -- 172.21.15.70:0/199539658 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f3714058730 con 0x7f371c160940 2023-10-08T05:49:59.949 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:49:59.949 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/809656839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":809656839},{"type":"v1","addr":"172.21.15.70:6835","nonce":809656839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2750508216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2750508216},{"type":"v1","addr":"172.21.15.181:6836","nonce":2750508216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/607266140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":607266140},{"type":"v1","addr":"172.21.15.181:6837","nonce":607266140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:37:44.192492+0000","modified":"2023-10-08T05:37:47.238340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/480502229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":480502229},{"type":"v1","addr":"172.21.15.70:6834","nonce":480502229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-08T05:49:59.950 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 18 2023-10-08T05:49:59.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 -- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f36fc06af00 msgr2=0x7f36fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f36fc06af00 0x7f36fc06d3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f371c0b3f30 tx=0x7f370c009040 comp rx=0 tx=0).stop 2023-10-08T05:49:59.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 -- 172.21.15.70:0/199539658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 msgr2=0x7f371c0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:49:59.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 0x7f371c0b33e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3714007e40 tx=0x7f371400e4b0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 -- 172.21.15.70:0/199539658 shutdown_connections 2023-10-08T05:49:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f371c160940 0x7f371c0b33e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f36fc06af00 0x7f36fc06d3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f371c15cd90 0x7f371c15d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 --2- 172.21.15.70:0/199539658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f371c06c890 0x7f371c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:49:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 -- 172.21.15.70:0/199539658 >> 172.21.15.70:0/199539658 conn(0x7f371c06a380 msgr2=0x7f371c160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:49:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 -- 172.21.15.70:0/199539658 shutdown_connections 2023-10-08T05:49:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:49:59.952+0000 7f3721c4f640 1 -- 172.21.15.70:0/199539658 wait complete. 2023-10-08T05:49:59.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:50:00.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.181+0000 7f1ad9701640 1 Processor -- start 2023-10-08T05:50:00.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.181+0000 7f1ad9701640 1 -- start start 2023-10-08T05:50:00.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.183+0000 7f1ad9701640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1ad415ab40 0x7f1ad40ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.183+0000 7f1ad9701640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad40cb580 0x7f1ad40adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.183+0000 7f1ad9701640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad415b2b0 con 0x7f1ad40c90b0 2023-10-08T05:50:00.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.183+0000 7f1ad9701640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad415bf30 con 0x7f1ad415ab40 2023-10-08T05:50:00.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.183+0000 7f1ad9701640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad415f7a0 con 0x7f1ad40cb580 2023-10-08T05:50:00.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.183+0000 7f1aca7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1ad40c90b0 0x7f1ad40cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49818/0 (socket says 172.21.15.70:49818) 2023-10-08T05:50:00.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.183+0000 7f1aca7fc640 1 -- 172.21.15.70:0/3830234268 learned_addr learned my addr 172.21.15.70:0/3830234268 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:00.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.183+0000 7f1ad37fe640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad40cb580 0x7f1ad40adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.183+0000 7f1ad2ffd640 1 --2- 172.21.15.70:0/3830234268 >> v2:172.21.15.181:3300/0 conn(0x7f1ad415ab40 0x7f1ad40ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.183+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413806314 0 0) 0x7f1ad415b2b0 con 0x7f1ad40c90b0 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ac0003520 con 0x7f1ad40c90b0 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3814940385 0 0) 0x7f1ac0003520 con 0x7f1ad40c90b0 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad40cb580 msgr2=0x7f1ad40adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad40cb580 0x7f1ad40adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 >> v2:172.21.15.181:3300/0 conn(0x7f1ad415ab40 msgr2=0x7f1ad40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 --2- 172.21.15.70:0/3830234268 >> v2:172.21.15.181:3300/0 conn(0x7f1ad415ab40 0x7f1ad40ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad415b590 con 0x7f1ad40c90b0 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1abc001670 con 0x7f1ad40c90b0 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 >> v1:172.21.15.70:6789/0 conn(0x7f1ad40c90b0 legacy=0x7f1ad40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ac00039a0 0x7f1ac0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad40cb580 0x7f1ac00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac0004d00 0x7f1ac0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad415bf30 con 0x7f1ac0004d00 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad415f7a0 con 0x7f1ac00039a0 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac000ba00 con 0x7f1ad40cb580 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.184+0000 7f1ad37fe640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac0004d00 0x7f1ac0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.185+0000 7f1aca7fc640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad40cb580 0x7f1ac00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.185+0000 7f1ad37fe640 1 -- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad40cb580 msgr2=0x7f1ac00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.185+0000 7f1ad2ffd640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ac00039a0 0x7f1ac0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.185+0000 7f1ad37fe640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ad40cb580 0x7f1ac00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.185+0000 7f1ad37fe640 1 -- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ac00039a0 msgr2=0x7f1ac0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.185+0000 7f1ad37fe640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ac00039a0 0x7f1ac0003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.185+0000 7f1ad37fe640 1 -- 172.21.15.70:0/3830234268 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac0003520 con 0x7f1ac0004d00 2023-10-08T05:50:00.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.186+0000 7f1ad37fe640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac0004d00 0x7f1ac0005180 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009ac0 tx=0x7f1ac00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a915b8f0fe007d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:00.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.186+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc02c6b0 con 0x7f1ac0004d00 2023-10-08T05:50:00.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.186+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc02c810 con 0x7f1ac0004d00 2023-10-08T05:50:00.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.186+0000 7f1ad27fc640 1 -- 172.21.15.70:0/3830234268 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc02cba0 con 0x7f1ac0004d00 2023-10-08T05:50:00.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.186+0000 7f1ad9701640 1 -- 172.21.15.70:0/3830234268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac0004d00 msgr2=0x7f1ac0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.186+0000 7f1ad9701640 1 --2- 172.21.15.70:0/3830234268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ac0004d00 0x7f1ac0005180 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009ac0 tx=0x7f1ac00057f0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.186+0000 7f1ad37fe640 1 -- 172.21.15.70:0/3830234268 reap_dead start 2023-10-08T05:50:00.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.186+0000 7f1ad9701640 1 -- 172.21.15.70:0/3830234268 shutdown_connections 2023-10-08T05:50:00.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.186+0000 7f1ad9701640 1 -- 172.21.15.70:0/3830234268 >> 172.21.15.70:0/3830234268 conn(0x7f1ad406a380 msgr2=0x7f1ad40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:00.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.187+0000 7f1ad9701640 1 -- 172.21.15.70:0/3830234268 shutdown_connections 2023-10-08T05:50:00.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.187+0000 7f1ad9701640 1 -- 172.21.15.70:0/3830234268 wait complete. 2023-10-08T05:50:00.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.188+0000 7f1ad9701640 1 Processor -- start 2023-10-08T05:50:00.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.188+0000 7f1ad9701640 1 -- start start 2023-10-08T05:50:00.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.189+0000 7f1ad9701640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac0004d00 0x7f1ad41f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.189+0000 7f1ad9701640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad40c40f0 0x7f1ad40c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.189+0000 7f1ad9701640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad40c97d0 0x7f1ad40c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.189+0000 7f1ad9701640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40ca1c0 con 0x7f1ad40c97d0 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.189+0000 7f1ad9701640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad41f62f0 con 0x7f1ad40c40f0 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.189+0000 7f1ad9701640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40c4900 con 0x7f1ac0004d00 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad40c97d0 0x7f1ad40c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad40c97d0 0x7f1ad40c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50648/0 (socket says 172.21.15.70:50648) 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1ad37fe640 1 -- 172.21.15.70:0/1858346432 learned_addr learned my addr 172.21.15.70:0/1858346432 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1ad2ffd640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac0004d00 0x7f1ad41f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1ad37fe640 1 -- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac0004d00 msgr2=0x7f1ad41f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1ad37fe640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac0004d00 0x7f1ad41f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1ad37fe640 1 -- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad40c40f0 msgr2=0x7f1ad40c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1aca7fc640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad40c40f0 0x7f1ad40c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1ad37fe640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad40c40f0 0x7f1ad40c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1ad37fe640 1 -- 172.21.15.70:0/1858346432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad41f6780 con 0x7f1ad40c97d0 2023-10-08T05:50:00.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.190+0000 7f1ad37fe640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad40c97d0 0x7f1ad40c9c80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009ac0 tx=0x7f1acc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:00.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.191+0000 7f1acbfff640 1 -- 172.21.15.70:0/1858346432 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc003810 con 0x7f1ad40c97d0 2023-10-08T05:50:00.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.191+0000 7f1acbfff640 1 -- 172.21.15.70:0/1858346432 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc003970 con 0x7f1ad40c97d0 2023-10-08T05:50:00.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.191+0000 7f1ad9701640 1 -- 172.21.15.70:0/1858346432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad41f59c0 con 0x7f1ad40c97d0 2023-10-08T05:50:00.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.191+0000 7f1acbfff640 1 -- 172.21.15.70:0/1858346432 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc003d20 con 0x7f1ad40c97d0 2023-10-08T05:50:00.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.191+0000 7f1ad9701640 1 -- 172.21.15.70:0/1858346432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad41f5f80 con 0x7f1ad40c97d0 2023-10-08T05:50:00.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.192+0000 7f1ad9701640 1 -- 172.21.15.70:0/1858346432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad41f64a0 con 0x7f1ad40c97d0 2023-10-08T05:50:00.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.193+0000 7f1acbfff640 1 -- 172.21.15.70:0/1858346432 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f1acc03e440 con 0x7f1ad40c97d0 2023-10-08T05:50:00.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.194+0000 7f1acbfff640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f1ab406af30 0x7f1ab406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.194+0000 7f1acbfff640 1 -- 172.21.15.70:0/1858346432 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1acc000e40 con 0x7f1ad40c97d0 2023-10-08T05:50:00.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.195+0000 7f1ad2ffd640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f1ab406af30 0x7f1ab406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.196+0000 7f1ad2ffd640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f1ab406af30 0x7f1ab406d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40cf100 tx=0x7f1ac400b040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:00.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.199+0000 7f1acbfff640 1 -- 172.21.15.70:0/1858346432 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ad41f64a0 con 0x7f1ad40c97d0 2023-10-08T05:50:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.408+0000 7f1ad9701640 1 -- 172.21.15.70:0/1858346432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ad41f64a0 con 0x7f1ad40c97d0 2023-10-08T05:50:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.408+0000 7f1acbfff640 1 -- 172.21.15.70:0/1858346432 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f1ad41f64a0 con 0x7f1ad40c97d0 2023-10-08T05:50:00.409 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:00.409 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/809656839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":809656839},{"type":"v1","addr":"172.21.15.70:6835","nonce":809656839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2750508216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2750508216},{"type":"v1","addr":"172.21.15.181:6836","nonce":2750508216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/607266140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":607266140},{"type":"v1","addr":"172.21.15.181:6837","nonce":607266140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:37:44.192492+0000","modified":"2023-10-08T05:37:47.238340+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/480502229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":480502229},{"type":"v1","addr":"172.21.15.70:6834","nonce":480502229}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-10-08T05:50:00.410 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 18 2023-10-08T05:50:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 -- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f1ab406af30 msgr2=0x7f1ab406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f1ab406af30 0x7f1ab406d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40cf100 tx=0x7f1ac400b040 comp rx=0 tx=0).stop 2023-10-08T05:50:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 -- 172.21.15.70:0/1858346432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad40c97d0 msgr2=0x7f1ad40c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad40c97d0 0x7f1ad40c9c80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009ac0 tx=0x7f1acc004730 comp rx=0 tx=0).stop 2023-10-08T05:50:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 -- 172.21.15.70:0/1858346432 shutdown_connections 2023-10-08T05:50:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ad40c97d0 0x7f1ad40c9c80 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ad40c40f0 0x7f1ad40c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f1ab406af30 0x7f1ab406d3f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 --2- 172.21.15.70:0/1858346432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ac0004d00 0x7f1ad41f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 -- 172.21.15.70:0/1858346432 >> 172.21.15.70:0/1858346432 conn(0x7f1ad406a380 msgr2=0x7f1ad41543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.411+0000 7f1ad9701640 1 -- 172.21.15.70:0/1858346432 shutdown_connections 2023-10-08T05:50:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.412+0000 7f1ad9701640 1 -- 172.21.15.70:0/1858346432 wait complete. 2023-10-08T05:50:00.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:50:00.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.634+0000 7f3f264c2640 1 Processor -- start 2023-10-08T05:50:00.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.634+0000 7f3f264c2640 1 -- start start 2023-10-08T05:50:00.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.635+0000 7f3f264c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f2015ab40 0x7f3f200ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.635+0000 7f3f264c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f200c90b0 0x7f3f200adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.635+0000 7f3f264c2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f200cae70 0x7f3f200cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.635+0000 7f3f264c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2015bf30 con 0x7f3f2015ab40 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.636+0000 7f3f264c2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f200cb840 con 0x7f3f200cae70 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.636+0000 7f3f264c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2015b2b0 con 0x7f3f200c90b0 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.636+0000 7f3f1ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f2015ab40 0x7f3f200ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.636+0000 7f3f1ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f2015ab40 0x7f3f200ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50660/0 (socket says 172.21.15.70:50660) 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.636+0000 7f3f1ffff640 1 -- 172.21.15.70:0/3653616778 learned_addr learned my addr 172.21.15.70:0/3653616778 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.636+0000 7f3f1ffff640 1 -- 172.21.15.70:0/3653616778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f200c90b0 msgr2=0x7f3f200adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.636+0000 7f3f1ffff640 1 --2- 172.21.15.70:0/3653616778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f200c90b0 0x7f3f200adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.636+0000 7f3f1ffff640 1 -- 172.21.15.70:0/3653616778 >> v2:172.21.15.181:3300/0 conn(0x7f3f200cae70 msgr2=0x7f3f200cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.636+0000 7f3f1ffff640 1 --2- 172.21.15.70:0/3653616778 >> v2:172.21.15.181:3300/0 conn(0x7f3f200cae70 0x7f3f200cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.636+0000 7f3f1ffff640 1 -- 172.21.15.70:0/3653616778 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2015b590 con 0x7f3f2015ab40 2023-10-08T05:50:00.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.637+0000 7f3f1ffff640 1 --2- 172.21.15.70:0/3653616778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f2015ab40 0x7f3f200ad720 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3f200c8d50 tx=0x7f3f1002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13a49660e6b2c77 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:00.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f1effd640 1 -- 172.21.15.70:0/3653616778 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10031870 con 0x7f3f2015ab40 2023-10-08T05:50:00.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f1effd640 1 -- 172.21.15.70:0/3653616778 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f100319d0 con 0x7f3f2015ab40 2023-10-08T05:50:00.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f1effd640 1 -- 172.21.15.70:0/3653616778 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10031d60 con 0x7f3f2015ab40 2023-10-08T05:50:00.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f264c2640 1 -- 172.21.15.70:0/3653616778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f2015ab40 msgr2=0x7f3f200ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f264c2640 1 --2- 172.21.15.70:0/3653616778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f2015ab40 0x7f3f200ad720 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3f200c8d50 tx=0x7f3f1002b910 comp rx=0 tx=0).stop 2023-10-08T05:50:00.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f264c2640 1 -- 172.21.15.70:0/3653616778 shutdown_connections 2023-10-08T05:50:00.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f264c2640 1 --2- 172.21.15.70:0/3653616778 >> v2:172.21.15.181:3300/0 conn(0x7f3f200cae70 0x7f3f200cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f264c2640 1 --2- 172.21.15.70:0/3653616778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f200c90b0 0x7f3f200adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f264c2640 1 --2- 172.21.15.70:0/3653616778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f2015ab40 0x7f3f200ad720 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f264c2640 1 -- 172.21.15.70:0/3653616778 >> 172.21.15.70:0/3653616778 conn(0x7f3f2006a380 msgr2=0x7f3f200b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:00.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f264c2640 1 -- 172.21.15.70:0/3653616778 shutdown_connections 2023-10-08T05:50:00.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.638+0000 7f3f264c2640 1 -- 172.21.15.70:0/3653616778 wait complete. 2023-10-08T05:50:00.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.640+0000 7f3f264c2640 1 Processor -- start 2023-10-08T05:50:00.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.640+0000 7f3f264c2640 1 -- start start 2023-10-08T05:50:00.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.641+0000 7f3f264c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f200c90b0 0x7f3f20151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.641+0000 7f3f264c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f200cae70 0x7f3f201450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.641+0000 7f3f264c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f2015ab40 0x7f3f20155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.641+0000 7f3f264c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20152050 con 0x7f3f200c90b0 2023-10-08T05:50:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.641+0000 7f3f264c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2015d2f0 con 0x7f3f200cae70 2023-10-08T05:50:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.641+0000 7f3f264c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f20153830 con 0x7f3f2015ab40 2023-10-08T05:50:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.641+0000 7f3f1ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f200c90b0 0x7f3f20151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.641+0000 7f3f1ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f200c90b0 0x7f3f20151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50666/0 (socket says 172.21.15.70:50666) 2023-10-08T05:50:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.641+0000 7f3f1ffff640 1 -- 172.21.15.70:0/3496498599 learned_addr learned my addr 172.21.15.70:0/3496498599 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f24a5f640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f2015ab40 0x7f3f20155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f1f7fe640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f200cae70 0x7f3f201450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f1ffff640 1 -- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f2015ab40 msgr2=0x7f3f20155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f1ffff640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f2015ab40 0x7f3f20155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f1ffff640 1 -- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f200cae70 msgr2=0x7f3f201450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f1ffff640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f200cae70 0x7f3f201450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f1ffff640 1 -- 172.21.15.70:0/3496498599 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f100094d0 con 0x7f3f200c90b0 2023-10-08T05:50:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f24a5f640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f2015ab40 0x7f3f20155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f1ffff640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f200c90b0 0x7f3f20151b10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3f10000c00 tx=0x7f3f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f1cff9640 1 -- 172.21.15.70:0/3496498599 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10031840 con 0x7f3f200c90b0 2023-10-08T05:50:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f1cff9640 1 -- 172.21.15.70:0/3496498599 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f100443f0 con 0x7f3f200c90b0 2023-10-08T05:50:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.642+0000 7f3f1cff9640 1 -- 172.21.15.70:0/3496498599 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f100447a0 con 0x7f3f200c90b0 2023-10-08T05:50:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.643+0000 7f3f264c2640 1 -- 172.21.15.70:0/3496498599 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f20153b10 con 0x7f3f200c90b0 2023-10-08T05:50:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.643+0000 7f3f264c2640 1 -- 172.21.15.70:0/3496498599 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f201455e0 con 0x7f3f200c90b0 2023-10-08T05:50:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.644+0000 7f3f264c2640 1 -- 172.21.15.70:0/3496498599 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f20147f70 con 0x7f3f200c90b0 2023-10-08T05:50:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.645+0000 7f3f1cff9640 1 -- 172.21.15.70:0/3496498599 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3f1004d450 con 0x7f3f200c90b0 2023-10-08T05:50:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.645+0000 7f3f1cff9640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f3f0c06af00 0x7f3f0c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.646+0000 7f3f1cff9640 1 -- 172.21.15.70:0/3496498599 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3f1002d040 con 0x7f3f200c90b0 2023-10-08T05:50:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.646+0000 7f3f1f7fe640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f3f0c06af00 0x7f3f0c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.647+0000 7f3f1f7fe640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f3f0c06af00 0x7f3f0c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3f20153200 tx=0x7f3f14008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:00.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.650+0000 7f3f1cff9640 1 -- 172.21.15.70:0/3496498599 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f10076f10 con 0x7f3f200c90b0 2023-10-08T05:50:00.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.844+0000 7f3f264c2640 1 -- 172.21.15.70:0/3496498599 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f20067490 con 0x7f3f200c90b0 2023-10-08T05:50:00.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.847+0000 7f3f1cff9640 1 -- 172.21.15.70:0/3496498599 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f3f1007a530 con 0x7f3f200c90b0 2023-10-08T05:50:00.848 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:00.848 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":59,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:49:58.500315+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:37:41.960853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-08T05:37:43.190325+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":59,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1c","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/32320998":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/2533276123":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/3573082646":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3583094356":"2023-10-09T05:38:04.171728+0000","172.21.15.181:6832/2737793492":"2023-10-09T05:38:04.171728+0000","172.21.15.181:6833/2737793492":"2023-10-09T05:38:04.171728+0000","172.21.15.181:6832/2692398351":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/3620192672":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3280597969":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/756067864":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/437302488":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/2139016714":"2023-10-09T05:49:57.491954+0000","172.21.15.181:6833/2692398351":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/1816930869":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/995976827":"2023-10-09T05:49:57.491954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":9,"length":1},{"begin":17,"length":1},{"begin":21,"length":1},{"begin":24,"length":1},{"begin":30,"length":1},{"begin":38,"length":2},{"begin":45,"length":3},{"begin":53,"length":2},{"begin":57,"length":1},{"begin":62,"length":1},{"begin":64,"length":1},{"begin":66,"length":1},{"begin":73,"length":2},{"begin":79,"length":1},{"begin":86,"length":1},{"begin":88,"length":2},{"begin":94,"length":1},{"begin":96,"length":1},{"begin":99,"length":1},{"begin":103,"length":1},{"begin":107,"length":2},{"begin":193,"length":29}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:50:00.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 -- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f3f0c06af00 msgr2=0x7f3f0c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f3f0c06af00 0x7f3f0c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3f20153200 tx=0x7f3f14008040 comp rx=0 tx=0).stop 2023-10-08T05:50:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 -- 172.21.15.70:0/3496498599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f200c90b0 msgr2=0x7f3f20151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f200c90b0 0x7f3f20151b10 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3f10000c00 tx=0x7f3f100046c0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 -- 172.21.15.70:0/3496498599 shutdown_connections 2023-10-08T05:50:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f3f0c06af00 0x7f3f0c06d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f2015ab40 0x7f3f20155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f200cae70 0x7f3f201450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 --2- 172.21.15.70:0/3496498599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f200c90b0 0x7f3f20151b10 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 -- 172.21.15.70:0/3496498599 >> 172.21.15.70:0/3496498599 conn(0x7f3f2006a380 msgr2=0x7f3f200ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 -- 172.21.15.70:0/3496498599 shutdown_connections 2023-10-08T05:50:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:00.850+0000 7f3f264c2640 1 -- 172.21.15.70:0/3496498599 wait complete. 2023-10-08T05:50:00.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T05:50:01.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.087+0000 7f8019d8a640 1 Processor -- start 2023-10-08T05:50:01.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.087+0000 7f8019d8a640 1 -- start start 2023-10-08T05:50:01.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8019d8a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f801406c3a0 0x7f80140c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8019d8a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f801415aa00 0x7f80140b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8019d8a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801415b310 con 0x7f801406db60 2023-10-08T05:50:01.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8019d8a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80140ca230 con 0x7f801415aa00 2023-10-08T05:50:01.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8019d8a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801406d290 con 0x7f801406c3a0 2023-10-08T05:50:01.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8013fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f801415aa00 0x7f80140b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8013fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f801415aa00 0x7f80140b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50678/0 (socket says 172.21.15.70:50678) 2023-10-08T05:50:01.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8013fff640 1 -- 172.21.15.70:0/622324333 learned_addr learned my addr 172.21.15.70:0/622324333 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:01.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8013fff640 1 -- 172.21.15.70:0/622324333 >> v2:172.21.15.181:3300/0 conn(0x7f801406c3a0 msgr2=0x7f80140c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:01.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f80137fe640 1 --2- 172.21.15.70:0/622324333 >> v2:172.21.15.181:3300/0 conn(0x7f801406c3a0 0x7f80140c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8013fff640 1 --2- 172.21.15.70:0/622324333 >> v2:172.21.15.181:3300/0 conn(0x7f801406c3a0 0x7f80140c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8013fff640 1 -- 172.21.15.70:0/622324333 >> v1:172.21.15.70:6789/0 conn(0x7f801406db60 legacy=0x7f801406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:01.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.089+0000 7f8013fff640 1 -- 172.21.15.70:0/622324333 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80140c9860 con 0x7f801415aa00 2023-10-08T05:50:01.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.091+0000 7f8013fff640 1 --2- 172.21.15.70:0/622324333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 0x7f80140b2a90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f80140716b0 tx=0x7f800c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add0755c6a5301b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:01.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.091+0000 7f80127fc640 1 -- 172.21.15.70:0/622324333 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c02cb10 con 0x7f801415aa00 2023-10-08T05:50:01.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.091+0000 7f80127fc640 1 -- 172.21.15.70:0/622324333 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c02cc70 con 0x7f801415aa00 2023-10-08T05:50:01.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.091+0000 7f80127fc640 1 -- 172.21.15.70:0/622324333 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c0343f0 con 0x7f801415aa00 2023-10-08T05:50:01.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.091+0000 7f8019d8a640 1 -- 172.21.15.70:0/622324333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 msgr2=0x7f80140b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:01.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.092+0000 7f8019d8a640 1 --2- 172.21.15.70:0/622324333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 0x7f80140b2a90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f80140716b0 tx=0x7f800c02bf70 comp rx=0 tx=0).stop 2023-10-08T05:50:01.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.092+0000 7f8019d8a640 1 -- 172.21.15.70:0/622324333 shutdown_connections 2023-10-08T05:50:01.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.092+0000 7f8019d8a640 1 --2- 172.21.15.70:0/622324333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 0x7f80140b2a90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.092+0000 7f8019d8a640 1 --2- 172.21.15.70:0/622324333 >> v2:172.21.15.181:3300/0 conn(0x7f801406c3a0 0x7f80140c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.092+0000 7f8019d8a640 1 -- 172.21.15.70:0/622324333 >> 172.21.15.70:0/622324333 conn(0x7f801406a5d0 msgr2=0x7f80140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:01.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.092+0000 7f8019d8a640 1 -- 172.21.15.70:0/622324333 shutdown_connections 2023-10-08T05:50:01.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.092+0000 7f8019d8a640 1 -- 172.21.15.70:0/622324333 wait complete. 2023-10-08T05:50:01.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.093+0000 7f8019d8a640 1 Processor -- start 2023-10-08T05:50:01.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.094+0000 7f8019d8a640 1 -- start start 2023-10-08T05:50:01.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8019d8a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801406c3a0 0x7f8014152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8019d8a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801406db60 0x7f8014156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8019d8a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 0x7f80140c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8019d8a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801415d0d0 con 0x7f801415aa00 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8019d8a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801415c7a0 con 0x7f801406c3a0 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8019d8a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801415ca80 con 0x7f801406db60 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8013fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 0x7f80140c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8013fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 0x7f80140c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50692/0 (socket says 172.21.15.70:50692) 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8013fff640 1 -- 172.21.15.70:0/2841990522 learned_addr learned my addr 172.21.15.70:0/2841990522 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8013fff640 1 -- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801406db60 msgr2=0x7f8014156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8013fff640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801406db60 0x7f8014156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f80137fe640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801406c3a0 0x7f8014152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8013fff640 1 -- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801406c3a0 msgr2=0x7f8014152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8013fff640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801406c3a0 0x7f8014152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.095+0000 7f8013fff640 1 -- 172.21.15.70:0/2841990522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c009b30 con 0x7f801415aa00 2023-10-08T05:50:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.096+0000 7f8013fff640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 0x7f80140c79e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f800c009f40 tx=0x7f800c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.096+0000 7f7ffbfff640 1 -- 172.21.15.70:0/2841990522 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c009510 con 0x7f801415aa00 2023-10-08T05:50:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.096+0000 7f7ffbfff640 1 -- 172.21.15.70:0/2841990522 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c03e450 con 0x7f801415aa00 2023-10-08T05:50:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.096+0000 7f7ffbfff640 1 -- 172.21.15.70:0/2841990522 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c03e7e0 con 0x7f801415aa00 2023-10-08T05:50:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.096+0000 7f8019d8a640 1 -- 172.21.15.70:0/2841990522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801415cd60 con 0x7f801415aa00 2023-10-08T05:50:01.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.097+0000 7f8019d8a640 1 -- 172.21.15.70:0/2841990522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8014155f50 con 0x7f801415aa00 2023-10-08T05:50:01.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.098+0000 7f8019d8a640 1 -- 172.21.15.70:0/2841990522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801415d240 con 0x7f801415aa00 2023-10-08T05:50:01.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.099+0000 7f7ffbfff640 1 -- 172.21.15.70:0/2841990522 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f800c03e940 con 0x7f801415aa00 2023-10-08T05:50:01.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.100+0000 7f7ffbfff640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f800006af00 0x7f800006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.100+0000 7f7ffbfff640 1 -- 172.21.15.70:0/2841990522 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f800c0acab0 con 0x7f801415aa00 2023-10-08T05:50:01.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.100+0000 7f80137fe640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f800006af00 0x7f800006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.101+0000 7f80137fe640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f800006af00 0x7f800006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8004006fd0 tx=0x7f8004008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:01.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.104+0000 7f7ffbfff640 1 -- 172.21.15.70:0/2841990522 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f800c077700 con 0x7f801415aa00 2023-10-08T05:50:01.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.330+0000 7f8019d8a640 1 -- 172.21.15.70:0/2841990522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8014156230 con 0x7f801415aa00 2023-10-08T05:50:01.332 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T05:50:01.332+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T05:50:01.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.335+0000 7f7ffbfff640 1 -- 172.21.15.70:0/2841990522 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f800c043090 con 0x7f801415aa00 2023-10-08T05:50:01.336 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T05:50:01.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.339+0000 7f8019d8a640 1 -- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f800006af00 msgr2=0x7f800006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:01.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.339+0000 7f8019d8a640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f800006af00 0x7f800006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8004006fd0 tx=0x7f8004008040 comp rx=0 tx=0).stop 2023-10-08T05:50:01.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.339+0000 7f8019d8a640 1 -- 172.21.15.70:0/2841990522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 msgr2=0x7f80140c79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:01.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.339+0000 7f8019d8a640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 0x7f80140c79e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f800c009f40 tx=0x7f800c004a40 comp rx=0 tx=0).stop 2023-10-08T05:50:01.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.340+0000 7f8019d8a640 1 -- 172.21.15.70:0/2841990522 shutdown_connections 2023-10-08T05:50:01.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.340+0000 7f8019d8a640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801415aa00 0x7f80140c79e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.340+0000 7f8019d8a640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f800006af00 0x7f800006d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.340+0000 7f8019d8a640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801406db60 0x7f8014156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.340+0000 7f8019d8a640 1 --2- 172.21.15.70:0/2841990522 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801406c3a0 0x7f8014152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.340+0000 7f8019d8a640 1 -- 172.21.15.70:0/2841990522 >> 172.21.15.70:0/2841990522 conn(0x7f801406a5d0 msgr2=0x7f80140cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:01.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.340+0000 7f8019d8a640 1 -- 172.21.15.70:0/2841990522 shutdown_connections 2023-10-08T05:50:01.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.340+0000 7f8019d8a640 1 -- 172.21.15.70:0/2841990522 wait complete. 2023-10-08T05:50:01.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T05:50:01.456 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:50:01.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.556+0000 7f259b8da640 1 Processor -- start 2023-10-08T05:50:01.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.556+0000 7f259b8da640 1 -- start start 2023-10-08T05:50:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f259b8da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259415cdb0 0x7f25940cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f259b8da640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f259415d190 0x7f259406af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f259b8da640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259415d5f0 con 0x7f25940ccb40 2023-10-08T05:50:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f259b8da640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25940b19e0 con 0x7f259415d190 2023-10-08T05:50:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f259b8da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2594069ef0 con 0x7f259415cdb0 2023-10-08T05:50:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f2599e77640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f25940ccb40 0x7f259406cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49856/0 (socket says 172.21.15.70:49856) 2023-10-08T05:50:01.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f2599e77640 1 -- 172.21.15.70:0/301809843 learned_addr learned my addr 172.21.15.70:0/301809843 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:01.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f2599676640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259415cdb0 0x7f25940cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f2598e75640 1 --2- 172.21.15.70:0/301809843 >> v2:172.21.15.181:3300/0 conn(0x7f259415d190 0x7f259406af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362101390 0 0) 0x7f259415d5f0 con 0x7f25940ccb40 2023-10-08T05:50:01.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.558+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2580003520 con 0x7f25940ccb40 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2785137099 0 0) 0x7f2580003520 con 0x7f25940ccb40 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259415cdb0 msgr2=0x7f25940cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259415cdb0 0x7f25940cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 >> v2:172.21.15.181:3300/0 conn(0x7f259415d190 msgr2=0x7f259406af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 --2- 172.21.15.70:0/301809843 >> v2:172.21.15.181:3300/0 conn(0x7f259415d190 0x7f259406af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259406a1d0 con 0x7f25940ccb40 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f258c001670 con 0x7f25940ccb40 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 >> v1:172.21.15.70:6789/0 conn(0x7f25940ccb40 legacy=0x7f259406cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259415cdb0 0x7f2580003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2580003ef0 0x7f25800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2580004d00 0x7f2580005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25940b19e0 con 0x7f2580004d00 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2594069ef0 con 0x7f2580003ef0 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258000b980 con 0x7f259415cdb0 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.559+0000 7f2599e77640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2580004d00 0x7f2580005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.560+0000 7f2599676640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259415cdb0 0x7f2580003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.560+0000 7f2598e75640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2580003ef0 0x7f25800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.560+0000 7f2599e77640 1 -- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259415cdb0 msgr2=0x7f2580003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.560+0000 7f2599e77640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f259415cdb0 0x7f2580003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.560+0000 7f2599e77640 1 -- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2580003ef0 msgr2=0x7f25800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.560+0000 7f2599e77640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2580003ef0 0x7f25800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.560+0000 7f2599e77640 1 -- 172.21.15.70:0/301809843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580003520 con 0x7f2580004d00 2023-10-08T05:50:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.560+0000 7f2598e75640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2580003ef0 0x7f25800047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.561+0000 7f2599e77640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2580004d00 0x7f2580005180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c290 tx=0x7f25800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd6c40c3268d863 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.561+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c005ad0 con 0x7f2580004d00 2023-10-08T05:50:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.561+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c005c30 con 0x7f2580004d00 2023-10-08T05:50:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.561+0000 7f258bfff640 1 -- 172.21.15.70:0/301809843 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c02c8c0 con 0x7f2580004d00 2023-10-08T05:50:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.562+0000 7f259b8da640 1 -- 172.21.15.70:0/301809843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2580004d00 msgr2=0x7f2580005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.562+0000 7f259b8da640 1 --2- 172.21.15.70:0/301809843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2580004d00 0x7f2580005180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c290 tx=0x7f25800057f0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.562+0000 7f2599e77640 1 -- 172.21.15.70:0/301809843 reap_dead start 2023-10-08T05:50:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.562+0000 7f259b8da640 1 -- 172.21.15.70:0/301809843 shutdown_connections 2023-10-08T05:50:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.562+0000 7f259b8da640 1 -- 172.21.15.70:0/301809843 >> 172.21.15.70:0/301809843 conn(0x7f25940b0af0 msgr2=0x7f25940b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:01.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.562+0000 7f259b8da640 1 -- 172.21.15.70:0/301809843 shutdown_connections 2023-10-08T05:50:01.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.562+0000 7f259b8da640 1 -- 172.21.15.70:0/301809843 wait complete. 2023-10-08T05:50:01.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.564+0000 7f259b8da640 1 Processor -- start 2023-10-08T05:50:01.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.564+0000 7f259b8da640 1 -- start start 2023-10-08T05:50:01.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.565+0000 7f259b8da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2580004d00 0x7f25941f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.565+0000 7f259b8da640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25940b1b20 0x7f25940b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.565+0000 7f259b8da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25941f17c0 0x7f25941f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.565+0000 7f259b8da640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25940b24e0 con 0x7f25941f17c0 2023-10-08T05:50:01.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.565+0000 7f259b8da640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25941f6b10 con 0x7f25940b1b20 2023-10-08T05:50:01.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.565+0000 7f259b8da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259415e9d0 con 0x7f2580004d00 2023-10-08T05:50:01.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.565+0000 7f2599e77640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25941f17c0 0x7f25941f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.566+0000 7f2599e77640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25941f17c0 0x7f25941f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50734/0 (socket says 172.21.15.70:50734) 2023-10-08T05:50:01.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.566+0000 7f2599e77640 1 -- 172.21.15.70:0/703445987 learned_addr learned my addr 172.21.15.70:0/703445987 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:01.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.566+0000 7f2598e75640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25940b1b20 0x7f25940b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.566+0000 7f2599676640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2580004d00 0x7f25941f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.566+0000 7f2599e77640 1 -- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2580004d00 msgr2=0x7f25941f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.566+0000 7f2599e77640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2580004d00 0x7f25941f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.566+0000 7f2599e77640 1 -- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25940b1b20 msgr2=0x7f25940b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.566+0000 7f2599e77640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25940b1b20 0x7f25940b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.566+0000 7f2599e77640 1 -- 172.21.15.70:0/703445987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25941f6270 con 0x7f25941f17c0 2023-10-08T05:50:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.566+0000 7f2599e77640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25941f17c0 0x7f25941f1c40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f258c000c00 tx=0x7f258c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.567+0000 7f2589ffb640 1 -- 172.21.15.70:0/703445987 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c03c040 con 0x7f25941f17c0 2023-10-08T05:50:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.567+0000 7f2589ffb640 1 -- 172.21.15.70:0/703445987 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c02ccf0 con 0x7f25941f17c0 2023-10-08T05:50:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.567+0000 7f2589ffb640 1 -- 172.21.15.70:0/703445987 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0072b0 con 0x7f25941f17c0 2023-10-08T05:50:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.567+0000 7f259b8da640 1 -- 172.21.15.70:0/703445987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25941f6550 con 0x7f25941f17c0 2023-10-08T05:50:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.567+0000 7f259b8da640 1 -- 172.21.15.70:0/703445987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25941f2180 con 0x7f25941f17c0 2023-10-08T05:50:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.568+0000 7f259b8da640 1 -- 172.21.15.70:0/703445987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259415e830 con 0x7f25941f17c0 2023-10-08T05:50:01.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.569+0000 7f2589ffb640 1 -- 172.21.15.70:0/703445987 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 17) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f258c032070 con 0x7f25941f17c0 2023-10-08T05:50:01.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.570+0000 7f2589ffb640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f257806af00 0x7f257806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:01.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.570+0000 7f2589ffb640 1 -- 172.21.15.70:0/703445987 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7169+0+0 (secure 0 0 0) 0x7f258c0aa820 con 0x7f25941f17c0 2023-10-08T05:50:01.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.570+0000 7f2599676640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f257806af00 0x7f257806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:01.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.571+0000 7f2599676640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f257806af00 0x7f257806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2590003c00 tx=0x7f259000b040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:01.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.574+0000 7f2589ffb640 1 -- 172.21.15.70:0/703445987 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f259415e830 con 0x7f25941f17c0 2023-10-08T05:50:01.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:01.799+0000 7f259b8da640 1 -- 172.21.15.70:0/703445987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f259415e830 con 0x7f25941f17c0 2023-10-08T05:50:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.339+0000 7f2589ffb640 1 -- 172.21.15.70:0/703445987 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f259415e830 con 0x7f25941f17c0 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 -- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f257806af00 msgr2=0x7f257806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f257806af00 0x7f257806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2590003c00 tx=0x7f259000b040 comp rx=0 tx=0).stop 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 -- 172.21.15.70:0/703445987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25941f17c0 msgr2=0x7f25941f1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25941f17c0 0x7f25941f1c40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f258c000c00 tx=0x7f258c001a00 comp rx=0 tx=0).stop 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 -- 172.21.15.70:0/703445987 shutdown_connections 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25941f17c0 0x7f25941f1c40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25940b1b20 0x7f25940b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f257806af00 0x7f257806d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 --2- 172.21.15.70:0/703445987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2580004d00 0x7f25941f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 -- 172.21.15.70:0/703445987 >> 172.21.15.70:0/703445987 conn(0x7f25940b0af0 msgr2=0x7f25940abd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.343+0000 7f259b8da640 1 -- 172.21.15.70:0/703445987 shutdown_connections 2023-10-08T05:50:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.344+0000 7f259b8da640 1 -- 172.21.15.70:0/703445987 wait complete. 2023-10-08T05:50:02.367 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T05:50:02.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.576+0000 7f49c37fc640 1 Processor -- start 2023-10-08T05:50:02.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.576+0000 7f49c37fc640 1 -- start start 2023-10-08T05:50:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.577+0000 7f49c37fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f49bc15cf50 0x7f49bc0cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.577+0000 7f49c37fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49bc06b070 0x7f49bc06c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.577+0000 7f49c37fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f49bc0cd270 0x7f49bc0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.577+0000 7f49c37fc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49bc06b610 con 0x7f49bc15cf50 2023-10-08T05:50:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.577+0000 7f49c37fc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49bc0cdc40 con 0x7f49bc0cd270 2023-10-08T05:50:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.577+0000 7f49c37fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0cee40 con 0x7f49bc06b070 2023-10-08T05:50:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c1598640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f49bc15cf50 0x7f49bc0cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c1598640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f49bc15cf50 0x7f49bc0cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50736/0 (socket says 172.21.15.70:50736) 2023-10-08T05:50:02.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c1598640 1 -- 172.21.15.70:0/1180019473 learned_addr learned my addr 172.21.15.70:0/1180019473 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c1d99640 1 --2- 172.21.15.70:0/1180019473 >> v2:172.21.15.181:3300/0 conn(0x7f49bc0cd270 0x7f49bc0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c0d97640 1 --2- 172.21.15.70:0/1180019473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49bc06b070 0x7f49bc06c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c1598640 1 -- 172.21.15.70:0/1180019473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49bc06b070 msgr2=0x7f49bc06c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c1598640 1 --2- 172.21.15.70:0/1180019473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49bc06b070 0x7f49bc06c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c1598640 1 -- 172.21.15.70:0/1180019473 >> v2:172.21.15.181:3300/0 conn(0x7f49bc0cd270 msgr2=0x7f49bc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c1598640 1 --2- 172.21.15.70:0/1180019473 >> v2:172.21.15.181:3300/0 conn(0x7f49bc0cd270 0x7f49bc0cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c1598640 1 -- 172.21.15.70:0/1180019473 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc15d330 con 0x7f49bc15cf50 2023-10-08T05:50:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.578+0000 7f49c1d99640 1 --2- 172.21.15.70:0/1180019473 >> v2:172.21.15.181:3300/0 conn(0x7f49bc0cd270 0x7f49bc0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:02.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.579+0000 7f49c1598640 1 --2- 172.21.15.70:0/1180019473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc15cf50 0x7f49bc0cfe80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f49bc09a3a0 tx=0x7f49ac02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35adef6685c9617d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:02.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.580+0000 7f49bbfff640 1 -- 172.21.15.70:0/1180019473 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac02ca60 con 0x7f49bc15cf50 2023-10-08T05:50:02.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.580+0000 7f49bbfff640 1 -- 172.21.15.70:0/1180019473 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac02cbc0 con 0x7f49bc15cf50 2023-10-08T05:50:02.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.580+0000 7f49bbfff640 1 -- 172.21.15.70:0/1180019473 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0343f0 con 0x7f49bc15cf50 2023-10-08T05:50:02.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.580+0000 7f49c37fc640 1 -- 172.21.15.70:0/1180019473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc15cf50 msgr2=0x7f49bc0cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:02.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.580+0000 7f49c37fc640 1 --2- 172.21.15.70:0/1180019473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc15cf50 0x7f49bc0cfe80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f49bc09a3a0 tx=0x7f49ac02bec0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.580+0000 7f49c37fc640 1 -- 172.21.15.70:0/1180019473 shutdown_connections 2023-10-08T05:50:02.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.580+0000 7f49c37fc640 1 --2- 172.21.15.70:0/1180019473 >> v2:172.21.15.181:3300/0 conn(0x7f49bc0cd270 0x7f49bc0cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.580+0000 7f49c37fc640 1 --2- 172.21.15.70:0/1180019473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49bc06b070 0x7f49bc06c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.580+0000 7f49c37fc640 1 --2- 172.21.15.70:0/1180019473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc15cf50 0x7f49bc0cfe80 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.580+0000 7f49c37fc640 1 -- 172.21.15.70:0/1180019473 >> 172.21.15.70:0/1180019473 conn(0x7f49bc099d00 msgr2=0x7f49bc0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:02.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.581+0000 7f49c37fc640 1 -- 172.21.15.70:0/1180019473 shutdown_connections 2023-10-08T05:50:02.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.581+0000 7f49c37fc640 1 -- 172.21.15.70:0/1180019473 wait complete. 2023-10-08T05:50:02.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.582+0000 7f49c37fc640 1 Processor -- start 2023-10-08T05:50:02.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.582+0000 7f49c37fc640 1 -- start start 2023-10-08T05:50:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.583+0000 7f49c37fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc06b070 0x7f49bc1510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.583+0000 7f49c37fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49bc0cd270 0x7f49bc0c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.583+0000 7f49c37fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49bc15cf50 0x7f49bc0c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.583+0000 7f49c37fc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc15e040 con 0x7f49bc06b070 2023-10-08T05:50:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.583+0000 7f49c37fc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc15d6a0 con 0x7f49bc15cf50 2023-10-08T05:50:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.583+0000 7f49c37fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc15d980 con 0x7f49bc0cd270 2023-10-08T05:50:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.584+0000 7f49c1598640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc06b070 0x7f49bc1510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.584+0000 7f49c1598640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc06b070 0x7f49bc1510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50744/0 (socket says 172.21.15.70:50744) 2023-10-08T05:50:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.584+0000 7f49c1598640 1 -- 172.21.15.70:0/627345285 learned_addr learned my addr 172.21.15.70:0/627345285 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.584+0000 7f49c0d97640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49bc0cd270 0x7f49bc0c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.584+0000 7f49c1d99640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49bc15cf50 0x7f49bc0c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.584+0000 7f49c1598640 1 -- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49bc0cd270 msgr2=0x7f49bc0c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.584+0000 7f49c1598640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49bc0cd270 0x7f49bc0c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.584+0000 7f49c1598640 1 -- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49bc15cf50 msgr2=0x7f49bc0c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.584+0000 7f49c1598640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49bc15cf50 0x7f49bc0c4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.584+0000 7f49c1598640 1 -- 172.21.15.70:0/627345285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac009ad0 con 0x7f49bc06b070 2023-10-08T05:50:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.585+0000 7f49c1598640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc06b070 0x7f49bc1510b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f49ac036040 tx=0x7f49ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:02.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.585+0000 7f49b9ffb640 1 -- 172.21.15.70:0/627345285 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac03e450 con 0x7f49bc06b070 2023-10-08T05:50:02.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.585+0000 7f49b9ffb640 1 -- 172.21.15.70:0/627345285 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac03e5b0 con 0x7f49bc06b070 2023-10-08T05:50:02.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.585+0000 7f49b9ffb640 1 -- 172.21.15.70:0/627345285 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac03e960 con 0x7f49bc06b070 2023-10-08T05:50:02.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.585+0000 7f49c37fc640 1 -- 172.21.15.70:0/627345285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc15dc60 con 0x7f49bc06b070 2023-10-08T05:50:02.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.585+0000 7f49c37fc640 1 -- 172.21.15.70:0/627345285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc0b1e20 con 0x7f49bc06b070 2023-10-08T05:50:02.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.586+0000 7f49c37fc640 1 -- 172.21.15.70:0/627345285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc04e160 con 0x7f49bc06b070 2023-10-08T05:50:02.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.587+0000 7f49b9ffb640 1 -- 172.21.15.70:0/627345285 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f49ac03eac0 con 0x7f49bc06b070 2023-10-08T05:50:02.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.588+0000 7f49b9ffb640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f49a806af90 0x7f49a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:02.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.588+0000 7f49b9ffb640 1 -- 172.21.15.70:0/627345285 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7197+0+0 (secure 0 0 0) 0x7f49ac0acd20 con 0x7f49bc06b070 2023-10-08T05:50:02.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.589+0000 7f49c0d97640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f49a806af90 0x7f49a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:02.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.590+0000 7f49c0d97640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f49a806af90 0x7f49a806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0c45d0 tx=0x7f49b0008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:02.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.593+0000 7f49b9ffb640 1 -- 172.21.15.70:0/627345285 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49ac0776e0 con 0x7f49bc06b070 2023-10-08T05:50:02.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:02.782+0000 7f49c37fc640 1 -- 172.21.15.70:0/627345285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f49bc0ce1a0 con 0x7f49bc06b070 2023-10-08T05:50:03.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.570+0000 7f49b9ffb640 1 -- 172.21.15.70:0/627345285 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v62) v1 ==== 181+0+0 (secure 0 0 0) 0x7f49ac07ad00 con 0x7f49bc06b070 2023-10-08T05:50:03.571 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T05:50:03.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 -- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f49a806af90 msgr2=0x7f49a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:03.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f49a806af90 0x7f49a806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0c45d0 tx=0x7f49b0008040 comp rx=0 tx=0).stop 2023-10-08T05:50:03.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 -- 172.21.15.70:0/627345285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc06b070 msgr2=0x7f49bc1510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc06b070 0x7f49bc1510b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f49ac036040 tx=0x7f49ac004a40 comp rx=0 tx=0).stop 2023-10-08T05:50:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 -- 172.21.15.70:0/627345285 shutdown_connections 2023-10-08T05:50:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49bc15cf50 0x7f49bc0c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f49a806af90 0x7f49a806d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49bc0cd270 0x7f49bc0c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 --2- 172.21.15.70:0/627345285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49bc06b070 0x7f49bc1510b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 -- 172.21.15.70:0/627345285 >> 172.21.15.70:0/627345285 conn(0x7f49bc099d00 msgr2=0x7f49bc0ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.575+0000 7f49c37fc640 1 -- 172.21.15.70:0/627345285 shutdown_connections 2023-10-08T05:50:03.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.576+0000 7f49c37fc640 1 -- 172.21.15.70:0/627345285 wait complete. 2023-10-08T05:50:03.599 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T05:50:03.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.810+0000 7f9d0a9f0640 1 Processor -- start 2023-10-08T05:50:03.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.810+0000 7f9d0a9f0640 1 -- start start 2023-10-08T05:50:03.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.811+0000 7f9d0a9f0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9d0415e3c0 0x7f9d0415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.811+0000 7f9d0a9f0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d040b12b0 con 0x7f9d040b1910 2023-10-08T05:50:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.811+0000 7f9d0a9f0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d040b1500 con 0x7f9d0415f6d0 2023-10-08T05:50:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.811+0000 7f9d0a9f0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d040cf1d0 con 0x7f9d0415e3c0 2023-10-08T05:50:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.811+0000 7f9cfb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9d040b1910 0x7f9d0415efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40924/0 (socket says 172.21.15.70:40924) 2023-10-08T05:50:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.811+0000 7f9cfb7fe640 1 -- 172.21.15.70:0/4122368587 learned_addr learned my addr 172.21.15.70:0/4122368587 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d03fff640 1 --2- 172.21.15.70:0/4122368587 >> v2:172.21.15.181:3300/0 conn(0x7f9d0415e3c0 0x7f9d0415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077047138 0 0) 0x7f9d040b12b0 con 0x7f9d040b1910 2023-10-08T05:50:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf0003520 con 0x7f9d040b1910 2023-10-08T05:50:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4277250126 0 0) 0x7f9cf0003520 con 0x7f9d040b1910 2023-10-08T05:50:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 >> v2:172.21.15.181:3300/0 conn(0x7f9d0415e3c0 msgr2=0x7f9d0415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 --2- 172.21.15.70:0/4122368587 >> v2:172.21.15.181:3300/0 conn(0x7f9d0415e3c0 0x7f9d0415ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 >> v1:172.21.15.181:6789/0 conn(0x7f9d0415f6d0 legacy=0x7f9d0415c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d040cf4b0 con 0x7f9d040b1910 2023-10-08T05:50:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9cec001670 con 0x7f9d040b1910 2023-10-08T05:50:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 >> v1:172.21.15.70:6789/0 conn(0x7f9d040b1910 legacy=0x7f9d0415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 --2- 172.21.15.70:0/4122368587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf00039c0 0x7f9cf0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 --2- 172.21.15.70:0/4122368587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf00047c0 0x7f9cf0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 --2- 172.21.15.70:0/4122368587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf00051d0 0x7f9cf000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040cf1d0 con 0x7f9cf00051d0 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf000c120 con 0x7f9cf00047c0 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf000c2e0 con 0x7f9cf00039c0 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.812+0000 7f9d03fff640 1 --2- 172.21.15.70:0/4122368587 >> v2:172.21.15.181:3300/0 conn(0x7f9d0415e3c0 0x7f9d0415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.813+0000 7f9d08f8d640 1 --2- 172.21.15.70:0/4122368587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf00051d0 0x7f9cf000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.813+0000 7f9cfb7fe640 1 --2- 172.21.15.70:0/4122368587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf00047c0 0x7f9cf0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.813+0000 7f9d08f8d640 1 -- 172.21.15.70:0/4122368587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf00039c0 msgr2=0x7f9cf0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.813+0000 7f9d08f8d640 1 --2- 172.21.15.70:0/4122368587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cf00039c0 0x7f9cf0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.813+0000 7f9d08f8d640 1 -- 172.21.15.70:0/4122368587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf00047c0 msgr2=0x7f9cf0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.813+0000 7f9d08f8d640 1 --2- 172.21.15.70:0/4122368587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf00047c0 0x7f9cf0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.813+0000 7f9d08f8d640 1 -- 172.21.15.70:0/4122368587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0003520 con 0x7f9cf00051d0 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.814+0000 7f9d08f8d640 1 -- 172.21.15.70:0/4122368587 reap_dead start 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.814+0000 7f9d08f8d640 1 --2- 172.21.15.70:0/4122368587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf00051d0 0x7f9cf000b980 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc001e50 tx=0x7f9cf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18041196062d31a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.814+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc003b70 con 0x7f9cf00051d0 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.814+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc003cf0 con 0x7f9cf00051d0 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.814+0000 7f9d037fe640 1 -- 172.21.15.70:0/4122368587 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc005310 con 0x7f9cf00051d0 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.815+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/4122368587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf00051d0 msgr2=0x7f9cf000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.815+0000 7f9d0a9f0640 1 --2- 172.21.15.70:0/4122368587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf00051d0 0x7f9cf000b980 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc001e50 tx=0x7f9cf00057f0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.815+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/4122368587 shutdown_connections 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.815+0000 7f9d0a9f0640 1 --2- 172.21.15.70:0/4122368587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cf00051d0 0x7f9cf000b980 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.815+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/4122368587 >> 172.21.15.70:0/4122368587 conn(0x7f9d04099d40 msgr2=0x7f9d040d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.815+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/4122368587 shutdown_connections 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.815+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/4122368587 wait complete. 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.817+0000 7f9d0a9f0640 1 Processor -- start 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.817+0000 7f9d0a9f0640 1 -- start start 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.818+0000 7f9d0a9f0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf00051d0 0x7f9d04153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.818+0000 7f9d0a9f0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d041f5c10 0x7f9d041f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.818+0000 7f9d0a9f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d041f65b0 0x7f9d0406ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.818+0000 7f9d0a9f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041f6a10 con 0x7f9d041f65b0 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.818+0000 7f9d0a9f0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04154180 con 0x7f9cf00051d0 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.818+0000 7f9d0a9f0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d04154460 con 0x7f9d041f5c10 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.818+0000 7f9d08f8d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d041f65b0 0x7f9d0406ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.818+0000 7f9d08f8d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d041f65b0 0x7f9d0406ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39450/0 (socket says 172.21.15.70:39450) 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.818+0000 7f9d08f8d640 1 -- 172.21.15.70:0/2930021490 learned_addr learned my addr 172.21.15.70:0/2930021490 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9cfb7fe640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d041f5c10 0x7f9d041f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d03fff640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf00051d0 0x7f9d04153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d08f8d640 1 -- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d041f5c10 msgr2=0x7f9d041f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d08f8d640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d041f5c10 0x7f9d041f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d08f8d640 1 -- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf00051d0 msgr2=0x7f9d04153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d08f8d640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf00051d0 0x7f9d04153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d08f8d640 1 -- 172.21.15.70:0/2930021490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d04154740 con 0x7f9d041f65b0 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d03fff640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf00051d0 0x7f9d04153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d08f8d640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d041f65b0 0x7f9d0406ae60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc001f70 tx=0x7f9cfc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d017fa640 1 -- 172.21.15.70:0/2930021490 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc006d50 con 0x7f9d041f65b0 2023-10-08T05:50:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d017fa640 1 -- 172.21.15.70:0/2930021490 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc003b80 con 0x7f9d041f65b0 2023-10-08T05:50:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d017fa640 1 -- 172.21.15.70:0/2930021490 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc039680 con 0x7f9d041f65b0 2023-10-08T05:50:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.819+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/2930021490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d04154a20 con 0x7f9d041f65b0 2023-10-08T05:50:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.820+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/2930021490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d04155220 con 0x7f9d041f65b0 2023-10-08T05:50:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.821+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/2930021490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d04154f40 con 0x7f9d041f65b0 2023-10-08T05:50:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.822+0000 7f9d017fa640 1 -- 172.21.15.70:0/2930021490 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9cfc0397e0 con 0x7f9d041f65b0 2023-10-08T05:50:03.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.822+0000 7f9d017fa640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f9ce406af90 0x7f9ce406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:03.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.823+0000 7f9d017fa640 1 -- 172.21.15.70:0/2930021490 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6395+0+0 (secure 0 0 0) 0x7f9cfc0ab920 con 0x7f9d041f65b0 2023-10-08T05:50:03.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.823+0000 7f9d03fff640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f9ce406af90 0x7f9ce406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:03.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.825+0000 7f9d03fff640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f9ce406af90 0x7f9ce406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4006fd0 tx=0x7f9cf4008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:03.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:03.827+0000 7f9d017fa640 1 -- 172.21.15.70:0/2930021490 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d04154f40 con 0x7f9d041f65b0 2023-10-08T05:50:04.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.028+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/2930021490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9d04154f40 con 0x7f9d041f65b0 2023-10-08T05:50:04.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.573+0000 7f9d017fa640 1 -- 172.21.15.70:0/2930021490 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v63) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9d04154f40 con 0x7f9d041f65b0 2023-10-08T05:50:04.574 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T05:50:04.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.579+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f9ce406af90 msgr2=0x7f9ce406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:04.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.579+0000 7f9d0a9f0640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f9ce406af90 0x7f9ce406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4006fd0 tx=0x7f9cf4008040 comp rx=0 tx=0).stop 2023-10-08T05:50:04.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.579+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/2930021490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d041f65b0 msgr2=0x7f9d0406ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:04.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.579+0000 7f9d0a9f0640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d041f65b0 0x7f9d0406ae60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc001f70 tx=0x7f9cfc0014a0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.579+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/2930021490 shutdown_connections 2023-10-08T05:50:04.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.579+0000 7f9d0a9f0640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d041f65b0 0x7f9d0406ae60 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.579+0000 7f9d0a9f0640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f9ce406af90 0x7f9ce406d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.579+0000 7f9d0a9f0640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9d041f5c10 0x7f9d041f6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.579+0000 7f9d0a9f0640 1 --2- 172.21.15.70:0/2930021490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cf00051d0 0x7f9d04153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.579+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/2930021490 >> 172.21.15.70:0/2930021490 conn(0x7f9d04099d40 msgr2=0x7f9d0415ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:04.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.580+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/2930021490 shutdown_connections 2023-10-08T05:50:04.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.580+0000 7f9d0a9f0640 1 -- 172.21.15.70:0/2930021490 wait complete. 2023-10-08T05:50:04.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T05:50:04.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.815+0000 7f160aea9640 1 Processor -- start 2023-10-08T05:50:04.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.816+0000 7f160aea9640 1 -- start start 2023-10-08T05:50:04.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.817+0000 7f160aea9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16041601b0 0x7f16040cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:04.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.817+0000 7f160aea9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f160406a460 0x7f16040ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:04.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.817+0000 7f160aea9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16040ce210 con 0x7f1604071c30 2023-10-08T05:50:04.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.817+0000 7f160aea9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16040cdee0 con 0x7f160406a460 2023-10-08T05:50:04.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.817+0000 7f160aea9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1604161530 con 0x7f16041601b0 2023-10-08T05:50:04.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.817+0000 7f1603fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1604071c30 0x7f1604069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40926/0 (socket says 172.21.15.70:40926) 2023-10-08T05:50:04.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.818+0000 7f1603fff640 1 -- 172.21.15.70:0/3532815506 learned_addr learned my addr 172.21.15.70:0/3532815506 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:04.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.818+0000 7f1609446640 1 --2- 172.21.15.70:0/3532815506 >> v2:172.21.15.70:3300/0 conn(0x7f160406a460 0x7f16040ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:04.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.818+0000 7f1608c45640 1 --2- 172.21.15.70:0/3532815506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16041601b0 0x7f16040cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:04.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.818+0000 7f16037fe640 1 -- 172.21.15.70:0/3532815506 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279354570 0 0) 0x7f16040ce210 con 0x7f1604071c30 2023-10-08T05:50:04.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.818+0000 7f1609446640 1 -- 172.21.15.70:0/3532815506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16041601b0 msgr2=0x7f16040cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:04.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.818+0000 7f1609446640 1 --2- 172.21.15.70:0/3532815506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16041601b0 0x7f16040cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.818+0000 7f1609446640 1 -- 172.21.15.70:0/3532815506 >> v1:172.21.15.70:6789/0 conn(0x7f1604071c30 legacy=0x7f1604069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:04.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.818+0000 7f1609446640 1 -- 172.21.15.70:0/3532815506 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1604161810 con 0x7f160406a460 2023-10-08T05:50:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.819+0000 7f1609446640 1 --2- 172.21.15.70:0/3532815506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f160406a460 0x7f16040ca090 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0097a0 tx=0x7f15fc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6670ed516616bdbc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.819+0000 7f16037fe640 1 -- 172.21.15.70:0/3532815506 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc02d040 con 0x7f160406a460 2023-10-08T05:50:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.820+0000 7f16037fe640 1 -- 172.21.15.70:0/3532815506 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15fc031810 con 0x7f160406a460 2023-10-08T05:50:04.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.820+0000 7f16037fe640 1 -- 172.21.15.70:0/3532815506 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc031ba0 con 0x7f160406a460 2023-10-08T05:50:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.820+0000 7f160aea9640 1 -- 172.21.15.70:0/3532815506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f160406a460 msgr2=0x7f16040ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.820+0000 7f160aea9640 1 --2- 172.21.15.70:0/3532815506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f160406a460 0x7f16040ca090 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0097a0 tx=0x7f15fc02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.820+0000 7f160aea9640 1 -- 172.21.15.70:0/3532815506 shutdown_connections 2023-10-08T05:50:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.820+0000 7f160aea9640 1 --2- 172.21.15.70:0/3532815506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f160406a460 0x7f16040ca090 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.820+0000 7f160aea9640 1 --2- 172.21.15.70:0/3532815506 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16041601b0 0x7f16040cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.820+0000 7f160aea9640 1 -- 172.21.15.70:0/3532815506 >> 172.21.15.70:0/3532815506 conn(0x7f160406d430 msgr2=0x7f160406d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.820+0000 7f160aea9640 1 -- 172.21.15.70:0/3532815506 shutdown_connections 2023-10-08T05:50:04.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.820+0000 7f160aea9640 1 -- 172.21.15.70:0/3532815506 wait complete. 2023-10-08T05:50:04.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.822+0000 7f160aea9640 1 Processor -- start 2023-10-08T05:50:04.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.822+0000 7f160aea9640 1 -- start start 2023-10-08T05:50:04.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.823+0000 7f160aea9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f160406a460 0x7f16040b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:04.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.823+0000 7f160aea9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1604071c30 0x7f16041f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.823+0000 7f160aea9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16041601b0 0x7f16040c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.823+0000 7f160aea9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16040c71b0 con 0x7f1604071c30 2023-10-08T05:50:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.823+0000 7f160aea9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16040c7490 con 0x7f16041601b0 2023-10-08T05:50:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.823+0000 7f160aea9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16040c7770 con 0x7f160406a460 2023-10-08T05:50:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1603fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1604071c30 0x7f16041f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1603fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1604071c30 0x7f16041f3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39480/0 (socket says 172.21.15.70:39480) 2023-10-08T05:50:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1603fff640 1 -- 172.21.15.70:0/3908834759 learned_addr learned my addr 172.21.15.70:0/3908834759 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1608c45640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f160406a460 0x7f16040b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1603fff640 1 -- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f160406a460 msgr2=0x7f16040b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1603fff640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f160406a460 0x7f16040b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1609446640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16041601b0 0x7f16040c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1603fff640 1 -- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16041601b0 msgr2=0x7f16040c3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1603fff640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16041601b0 0x7f16040c3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1603fff640 1 -- 172.21.15.70:0/3908834759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc009400 con 0x7f1604071c30 2023-10-08T05:50:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.824+0000 7f1603fff640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1604071c30 0x7f16041f3970 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f15f800c430 tx=0x7f15f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.825+0000 7f16017fa640 1 -- 172.21.15.70:0/3908834759 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f80074e0 con 0x7f1604071c30 2023-10-08T05:50:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.825+0000 7f16017fa640 1 -- 172.21.15.70:0/3908834759 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f8007660 con 0x7f1604071c30 2023-10-08T05:50:04.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.825+0000 7f16017fa640 1 -- 172.21.15.70:0/3908834759 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8004a20 con 0x7f1604071c30 2023-10-08T05:50:04.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.825+0000 7f160aea9640 1 -- 172.21.15.70:0/3908834759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16040c7a50 con 0x7f1604071c30 2023-10-08T05:50:04.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.825+0000 7f160aea9640 1 -- 172.21.15.70:0/3908834759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16041f3eb0 con 0x7f1604071c30 2023-10-08T05:50:04.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.826+0000 7f160aea9640 1 -- 172.21.15.70:0/3908834759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16040b5550 con 0x7f1604071c30 2023-10-08T05:50:04.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.827+0000 7f16017fa640 1 -- 172.21.15.70:0/3908834759 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f15f8004b80 con 0x7f1604071c30 2023-10-08T05:50:04.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.828+0000 7f16017fa640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f15f006af90 0x7f15f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:04.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.828+0000 7f16017fa640 1 -- 172.21.15.70:0/3908834759 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f15f8089730 con 0x7f1604071c30 2023-10-08T05:50:04.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.829+0000 7f1608c45640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f15f006af90 0x7f15f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:04.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.830+0000 7f1608c45640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f15f006af90 0x7f15f006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f15f40099c0 tx=0x7f15f4003990 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:04.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:04.838+0000 7f16017fa640 1 -- 172.21.15.70:0/3908834759 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15f8054560 con 0x7f1604071c30 2023-10-08T05:50:05.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.026+0000 7f160aea9640 1 -- 172.21.15.70:0/3908834759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f16040c7c90 con 0x7f1604071c30 2023-10-08T05:50:05.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.033+0000 7f16017fa640 1 -- 172.21.15.70:0/3908834759 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f16040c7c90 con 0x7f1604071c30 2023-10-08T05:50:05.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.036+0000 7f160aea9640 1 -- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f15f006af90 msgr2=0x7f15f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.036+0000 7f160aea9640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f15f006af90 0x7f15f006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f15f40099c0 tx=0x7f15f4003990 comp rx=0 tx=0).stop 2023-10-08T05:50:05.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.036+0000 7f160aea9640 1 -- 172.21.15.70:0/3908834759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1604071c30 msgr2=0x7f16041f3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.036+0000 7f160aea9640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1604071c30 0x7f16041f3970 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f15f800c430 tx=0x7f15f800c8f0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.036+0000 7f160aea9640 1 -- 172.21.15.70:0/3908834759 shutdown_connections 2023-10-08T05:50:05.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.036+0000 7f160aea9640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16041601b0 0x7f16040c3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.036+0000 7f160aea9640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1604071c30 0x7f16041f3970 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.036+0000 7f160aea9640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f15f006af90 0x7f15f006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.036+0000 7f160aea9640 1 --2- 172.21.15.70:0/3908834759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f160406a460 0x7f16040b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.036+0000 7f160aea9640 1 -- 172.21.15.70:0/3908834759 >> 172.21.15.70:0/3908834759 conn(0x7f160406d430 msgr2=0x7f1604069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:05.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.037+0000 7f160aea9640 1 -- 172.21.15.70:0/3908834759 shutdown_connections 2023-10-08T05:50:05.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.037+0000 7f160aea9640 1 -- 172.21.15.70:0/3908834759 wait complete. 2023-10-08T05:50:05.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T05:50:05.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.272+0000 7f4422aa9640 1 Processor -- start 2023-10-08T05:50:05.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.272+0000 7f4422aa9640 1 -- start start 2023-10-08T05:50:05.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.274+0000 7f4422aa9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f441c15ac00 0x7f441c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.274+0000 7f4422aa9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f441c0cdb50 0x7f441c0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.274+0000 7f4422aa9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c0ced90 con 0x7f441c0afe90 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.274+0000 7f4422aa9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c0cf070 con 0x7f441c0cdb50 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.274+0000 7f4422aa9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0cf350 con 0x7f441c15ac00 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.274+0000 7f441bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f441c0afe90 0x7f441c0cd440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40934/0 (socket says 172.21.15.70:40934) 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.274+0000 7f441bfff640 1 -- 172.21.15.70:0/3958982722 learned_addr learned my addr 172.21.15.70:0/3958982722 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.274+0000 7f4421046640 1 --2- 172.21.15.70:0/3958982722 >> v2:172.21.15.70:3300/0 conn(0x7f441c0cdb50 0x7f441c0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.274+0000 7f4420845640 1 --2- 172.21.15.70:0/3958982722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f441c15ac00 0x7f441c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.275+0000 7f4421046640 1 -- 172.21.15.70:0/3958982722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f441c15ac00 msgr2=0x7f441c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.275+0000 7f4421046640 1 --2- 172.21.15.70:0/3958982722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f441c15ac00 0x7f441c0d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.275+0000 7f4421046640 1 -- 172.21.15.70:0/3958982722 >> v1:172.21.15.70:6789/0 conn(0x7f441c0afe90 legacy=0x7f441c0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.275+0000 7f4421046640 1 -- 172.21.15.70:0/3958982722 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c0cf630 con 0x7f441c0cdb50 2023-10-08T05:50:05.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.275+0000 7f441b7fe640 1 -- 172.21.15.70:0/3958982722 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94545921 0 0) 0x7f441c0ced90 con 0x7f441c0afe90 2023-10-08T05:50:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.276+0000 7f4421046640 1 --2- 172.21.15.70:0/3958982722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 0x7f441c0b3b10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4414009d90 tx=0x7f441402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112e6c33b8c9f4a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.276+0000 7f441b7fe640 1 -- 172.21.15.70:0/3958982722 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441402cce0 con 0x7f441c0cdb50 2023-10-08T05:50:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.276+0000 7f441b7fe640 1 -- 172.21.15.70:0/3958982722 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441402ce40 con 0x7f441c0cdb50 2023-10-08T05:50:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.276+0000 7f441b7fe640 1 -- 172.21.15.70:0/3958982722 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414034660 con 0x7f441c0cdb50 2023-10-08T05:50:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.277+0000 7f4422aa9640 1 -- 172.21.15.70:0/3958982722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 msgr2=0x7f441c0b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.277+0000 7f4422aa9640 1 --2- 172.21.15.70:0/3958982722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 0x7f441c0b3b10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4414009d90 tx=0x7f441402bf30 comp rx=0 tx=0).stop 2023-10-08T05:50:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.277+0000 7f4422aa9640 1 -- 172.21.15.70:0/3958982722 shutdown_connections 2023-10-08T05:50:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.277+0000 7f4422aa9640 1 --2- 172.21.15.70:0/3958982722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 0x7f441c0b3b10 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.277+0000 7f4422aa9640 1 --2- 172.21.15.70:0/3958982722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f441c15ac00 0x7f441c0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.277+0000 7f4422aa9640 1 -- 172.21.15.70:0/3958982722 >> 172.21.15.70:0/3958982722 conn(0x7f441c06e650 msgr2=0x7f441c06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:05.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.277+0000 7f4422aa9640 1 -- 172.21.15.70:0/3958982722 shutdown_connections 2023-10-08T05:50:05.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.277+0000 7f4422aa9640 1 -- 172.21.15.70:0/3958982722 wait complete. 2023-10-08T05:50:05.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.279+0000 7f4422aa9640 1 Processor -- start 2023-10-08T05:50:05.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.279+0000 7f4422aa9640 1 -- start start 2023-10-08T05:50:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f4422aa9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f441c0afe90 0x7f441c0b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f4422aa9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 0x7f441c157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f4422aa9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f441c15ac00 0x7f441c0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f4422aa9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1584f0 con 0x7f441c0cdb50 2023-10-08T05:50:05.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f4422aa9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0b2c00 con 0x7f441c0afe90 2023-10-08T05:50:05.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f4422aa9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0b1620 con 0x7f441c15ac00 2023-10-08T05:50:05.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f441bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 0x7f441c157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:05.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f441bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 0x7f441c157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39492/0 (socket says 172.21.15.70:39492) 2023-10-08T05:50:05.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f441bfff640 1 -- 172.21.15.70:0/2049909930 learned_addr learned my addr 172.21.15.70:0/2049909930 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f4421046640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f441c15ac00 0x7f441c0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f441bfff640 1 -- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f441c15ac00 msgr2=0x7f441c0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f441bfff640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f441c15ac00 0x7f441c0ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f441bfff640 1 -- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f441c0afe90 msgr2=0x7f441c0b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f4420845640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f441c0afe90 0x7f441c0b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f441bfff640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f441c0afe90 0x7f441c0b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.280+0000 7f441bfff640 1 -- 172.21.15.70:0/2049909930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414009a70 con 0x7f441c0cdb50 2023-10-08T05:50:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.281+0000 7f441bfff640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 0x7f441c157990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f440800c4d0 tx=0x7f440800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.281+0000 7f44197fa640 1 -- 172.21.15.70:0/2049909930 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44080074e0 con 0x7f441c0cdb50 2023-10-08T05:50:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.281+0000 7f44197fa640 1 -- 172.21.15.70:0/2049909930 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4408007660 con 0x7f441c0cdb50 2023-10-08T05:50:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.281+0000 7f44197fa640 1 -- 172.21.15.70:0/2049909930 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408004a20 con 0x7f441c0cdb50 2023-10-08T05:50:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.281+0000 7f4422aa9640 1 -- 172.21.15.70:0/2049909930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c0b1900 con 0x7f441c0cdb50 2023-10-08T05:50:05.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.281+0000 7f4422aa9640 1 -- 172.21.15.70:0/2049909930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c0b1ec0 con 0x7f441c0cdb50 2023-10-08T05:50:05.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.282+0000 7f4422aa9640 1 -- 172.21.15.70:0/2049909930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c04e160 con 0x7f441c0cdb50 2023-10-08T05:50:05.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.283+0000 7f44197fa640 1 -- 172.21.15.70:0/2049909930 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4408004b80 con 0x7f441c0cdb50 2023-10-08T05:50:05.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.284+0000 7f44197fa640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f440006af90 0x7f440006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.284+0000 7f44197fa640 1 -- 172.21.15.70:0/2049909930 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f4408088a70 con 0x7f441c0cdb50 2023-10-08T05:50:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.285+0000 7f4420845640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f440006af90 0x7f440006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:05.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.286+0000 7f4420845640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f440006af90 0x7f440006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4410006fd0 tx=0x7f4410008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:05.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.288+0000 7f44197fa640 1 -- 172.21.15.70:0/2049909930 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4408053920 con 0x7f441c0cdb50 2023-10-08T05:50:05.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.476+0000 7f4422aa9640 1 -- 172.21.15.70:0/2049909930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f441c06ed60 con 0x7f441c0cdb50 2023-10-08T05:50:05.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.484+0000 7f44197fa640 1 -- 172.21.15.70:0/2049909930 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v14) v1 ==== 115+0+0 (secure 0 0 0) 0x7f4408018020 con 0x7f441c0cdb50 2023-10-08T05:50:05.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.486+0000 7f4422aa9640 1 -- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f440006af90 msgr2=0x7f440006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.486+0000 7f4422aa9640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f440006af90 0x7f440006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4410006fd0 tx=0x7f4410008040 comp rx=0 tx=0).stop 2023-10-08T05:50:05.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.486+0000 7f4422aa9640 1 -- 172.21.15.70:0/2049909930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 msgr2=0x7f441c157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.486+0000 7f4422aa9640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 0x7f441c157990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f440800c4d0 tx=0x7f440800c990 comp rx=0 tx=0).stop 2023-10-08T05:50:05.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.486+0000 7f4422aa9640 1 -- 172.21.15.70:0/2049909930 shutdown_connections 2023-10-08T05:50:05.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.486+0000 7f4422aa9640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f440006af90 0x7f440006d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.486+0000 7f4422aa9640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f441c15ac00 0x7f441c0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.486+0000 7f4422aa9640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f441c0cdb50 0x7f441c157990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.486+0000 7f4422aa9640 1 --2- 172.21.15.70:0/2049909930 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f441c0afe90 0x7f441c0b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.486+0000 7f4422aa9640 1 -- 172.21.15.70:0/2049909930 >> 172.21.15.70:0/2049909930 conn(0x7f441c06e650 msgr2=0x7f441c159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:05.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.487+0000 7f4422aa9640 1 -- 172.21.15.70:0/2049909930 shutdown_connections 2023-10-08T05:50:05.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.487+0000 7f4422aa9640 1 -- 172.21.15.70:0/2049909930 wait complete. 2023-10-08T05:50:05.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T05:50:05.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.724+0000 7fcd04b1d640 1 Processor -- start 2023-10-08T05:50:05.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.724+0000 7fcd04b1d640 1 -- start start 2023-10-08T05:50:05.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.725+0000 7fcd04b1d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd0006c3a0 0x7fcd000c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.725+0000 7fcd04b1d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd0015aa00 0x7fcd000b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.726+0000 7fcd04b1d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0015b310 con 0x7fcd0006db60 2023-10-08T05:50:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.726+0000 7fcd04b1d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd000ca230 con 0x7fcd0006c3a0 2023-10-08T05:50:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.726+0000 7fcd04b1d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0006d290 con 0x7fcd0015aa00 2023-10-08T05:50:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.726+0000 7fccfe59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd0006c3a0 0x7fcd000c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.726+0000 7fccfe59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd0006c3a0 0x7fcd000c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39506/0 (socket says 172.21.15.70:39506) 2023-10-08T05:50:05.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.726+0000 7fccfe59c640 1 -- 172.21.15.70:0/3043043712 learned_addr learned my addr 172.21.15.70:0/3043043712 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:05.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.726+0000 7fccfe59c640 1 -- 172.21.15.70:0/3043043712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd0015aa00 msgr2=0x7fcd000b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:05.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.726+0000 7fccfe59c640 1 --2- 172.21.15.70:0/3043043712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd0015aa00 0x7fcd000b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.726+0000 7fccfe59c640 1 -- 172.21.15.70:0/3043043712 >> v1:172.21.15.70:6789/0 conn(0x7fcd0006db60 legacy=0x7fcd0006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.726+0000 7fccfe59c640 1 -- 172.21.15.70:0/3043043712 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd000c9860 con 0x7fcd0006c3a0 2023-10-08T05:50:05.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.727+0000 7fccfe59c640 1 --2- 172.21.15.70:0/3043043712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 0x7fcd000c9cf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fccf000d150 tx=0x7fccf002e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc030fd96a01af6a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:05.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.728+0000 7fccfd59a640 1 -- 172.21.15.70:0/3043043712 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf000b0d0 con 0x7fcd0006c3a0 2023-10-08T05:50:05.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.728+0000 7fccfd59a640 1 -- 172.21.15.70:0/3043043712 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf000b230 con 0x7fcd0006c3a0 2023-10-08T05:50:05.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.728+0000 7fccfd59a640 1 -- 172.21.15.70:0/3043043712 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0009600 con 0x7fcd0006c3a0 2023-10-08T05:50:05.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.728+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3043043712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 msgr2=0x7fcd000c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.728+0000 7fcd04b1d640 1 --2- 172.21.15.70:0/3043043712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 0x7fcd000c9cf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fccf000d150 tx=0x7fccf002e160 comp rx=0 tx=0).stop 2023-10-08T05:50:05.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.728+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3043043712 shutdown_connections 2023-10-08T05:50:05.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.728+0000 7fcd04b1d640 1 --2- 172.21.15.70:0/3043043712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd0015aa00 0x7fcd000b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.728+0000 7fcd04b1d640 1 --2- 172.21.15.70:0/3043043712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 0x7fcd000c9cf0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.728+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3043043712 >> 172.21.15.70:0/3043043712 conn(0x7fcd0006a5d0 msgr2=0x7fcd000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:05.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.729+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3043043712 shutdown_connections 2023-10-08T05:50:05.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.729+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3043043712 wait complete. 2023-10-08T05:50:05.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.730+0000 7fcd04b1d640 1 Processor -- start 2023-10-08T05:50:05.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.730+0000 7fcd04b1d640 1 -- start start 2023-10-08T05:50:05.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.731+0000 7fcd04b1d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 0x7fcd00070810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.731+0000 7fcd04b1d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd0006db60 0x7fcd00154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.731+0000 7fcd04b1d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd0015aa00 0x7fcd00154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.731+0000 7fcd04b1d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd00152cc0 con 0x7fcd0006c3a0 2023-10-08T05:50:05.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.731+0000 7fcd04b1d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0015cf70 con 0x7fcd0015aa00 2023-10-08T05:50:05.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.731+0000 7fcd04b1d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0015d250 con 0x7fcd0006db60 2023-10-08T05:50:05.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.732+0000 7fccfe59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 0x7fcd00070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:05.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.732+0000 7fccfe59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 0x7fcd00070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39518/0 (socket says 172.21.15.70:39518) 2023-10-08T05:50:05.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.732+0000 7fccfe59c640 1 -- 172.21.15.70:0/3085737758 learned_addr learned my addr 172.21.15.70:0/3085737758 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:05.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.732+0000 7fccfe59c640 1 -- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd0006db60 msgr2=0x7fcd00154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.732+0000 7fccfe59c640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd0006db60 0x7fcd00154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.732+0000 7fccfe59c640 1 -- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd0015aa00 msgr2=0x7fcd00154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.732+0000 7fccfe59c640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd0015aa00 0x7fcd00154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.732+0000 7fccfe59c640 1 -- 172.21.15.70:0/3085737758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf000bd40 con 0x7fcd0006c3a0 2023-10-08T05:50:05.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.732+0000 7fccfe59c640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 0x7fcd00070810 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fccf0038040 tx=0x7fccf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:05.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.733+0000 7fccf6ffd640 1 -- 172.21.15.70:0/3085737758 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0036800 con 0x7fcd0006c3a0 2023-10-08T05:50:05.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.733+0000 7fccf6ffd640 1 -- 172.21.15.70:0/3085737758 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf0036980 con 0x7fcd0006c3a0 2023-10-08T05:50:05.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.733+0000 7fccf6ffd640 1 -- 172.21.15.70:0/3085737758 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0036d30 con 0x7fcd0006c3a0 2023-10-08T05:50:05.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.733+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3085737758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0015d530 con 0x7fcd0006c3a0 2023-10-08T05:50:05.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.733+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3085737758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd000b05c0 con 0x7fcd0006c3a0 2023-10-08T05:50:05.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.734+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3085737758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd000b0750 con 0x7fcd0006c3a0 2023-10-08T05:50:05.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.735+0000 7fccf6ffd640 1 -- 172.21.15.70:0/3085737758 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fccf0040420 con 0x7fcd0006c3a0 2023-10-08T05:50:05.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.736+0000 7fccf6ffd640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcce006afc0 0x7fcce006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:05.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.736+0000 7fccf6ffd640 1 -- 172.21.15.70:0/3085737758 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5952+0+0 (secure 0 0 0) 0x7fccf00ac790 con 0x7fcd0006c3a0 2023-10-08T05:50:05.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.736+0000 7fccfdd9b640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcce006afc0 0x7fcce006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:05.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.738+0000 7fccfdd9b640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcce006afc0 0x7fcce006d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcce8000f60 tx=0x7fcce80079b0 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:05.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.740+0000 7fccf6ffd640 1 -- 172.21.15.70:0/3085737758 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd000b0750 con 0x7fcd0006c3a0 2023-10-08T05:50:05.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.930+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3085737758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fcd000b0750 con 0x7fcd0006c3a0 2023-10-08T05:50:05.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.936+0000 7fccf6ffd640 1 -- 172.21.15.70:0/3085737758 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v15) v1 ==== 120+0+0 (secure 0 0 0) 0x7fcd000b0750 con 0x7fcd0006c3a0 2023-10-08T05:50:05.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcce006afc0 msgr2=0x7fcce006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcce006afc0 0x7fcce006d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcce8000f60 tx=0x7fcce80079b0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3085737758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 msgr2=0x7fcd00070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 0x7fcd00070810 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fccf0038040 tx=0x7fccf0004a40 comp rx=0 tx=0).stop 2023-10-08T05:50:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3085737758 shutdown_connections 2023-10-08T05:50:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd0015aa00 0x7fcd00154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcce006afc0 0x7fcce006d480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd0006db60 0x7fcd00154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 --2- 172.21.15.70:0/3085737758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd0006c3a0 0x7fcd00070810 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3085737758 >> 172.21.15.70:0/3085737758 conn(0x7fcd0006a5d0 msgr2=0x7fcd000cb680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3085737758 shutdown_connections 2023-10-08T05:50:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:05.939+0000 7fcd04b1d640 1 -- 172.21.15.70:0/3085737758 wait complete. 2023-10-08T05:50:05.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-08T05:50:06.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.180+0000 7fa23d4fc640 1 Processor -- start 2023-10-08T05:50:06.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.180+0000 7fa23d4fc640 1 -- start start 2023-10-08T05:50:06.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.181+0000 7fa23d4fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa238159ac0 0x7fa23806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.181+0000 7fa23d4fc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23806dee0 con 0x7fa2380709e0 2023-10-08T05:50:06.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.182+0000 7fa23d4fc640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23815d850 con 0x7fa2380ad790 2023-10-08T05:50:06.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.182+0000 7fa23d4fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23815db30 con 0x7fa238159ac0 2023-10-08T05:50:06.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.182+0000 7fa2377fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa2380709e0 0x7fa2380c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40950/0 (socket says 172.21.15.70:40950) 2023-10-08T05:50:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.182+0000 7fa2377fe640 1 -- 172.21.15.70:0/1457166359 learned_addr learned my addr 172.21.15.70:0/1457166359 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:06.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.182+0000 7fa236ffd640 1 --2- 172.21.15.70:0/1457166359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa238159ac0 0x7fa23806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:06.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.182+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616409943 0 0) 0x7fa23806dee0 con 0x7fa2380709e0 2023-10-08T05:50:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.182+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa224003520 con 0x7fa2380709e0 2023-10-08T05:50:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 126752867 0 0) 0x7fa224003520 con 0x7fa2380709e0 2023-10-08T05:50:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa238159ac0 msgr2=0x7fa23806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 --2- 172.21.15.70:0/1457166359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa238159ac0 0x7fa23806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 >> v1:172.21.15.181:6789/0 conn(0x7fa2380ad790 legacy=0x7fa2380702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23815de10 con 0x7fa2380709e0 2023-10-08T05:50:06.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa2300019f0 con 0x7fa2380709e0 2023-10-08T05:50:06.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 >> v1:172.21.15.70:6789/0 conn(0x7fa2380709e0 legacy=0x7fa2380c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 --2- 172.21.15.70:0/1457166359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2240039a0 0x7fa224003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 --2- 172.21.15.70:0/1457166359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa238159ac0 0x7fa2240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 --2- 172.21.15.70:0/1457166359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa224004d00 0x7fa224005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23815db30 con 0x7fa2240039a0 2023-10-08T05:50:06.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22400ba30 con 0x7fa224004d00 2023-10-08T05:50:06.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22400bbf0 con 0x7fa238159ac0 2023-10-08T05:50:06.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.183+0000 7fa236ffd640 1 --2- 172.21.15.70:0/1457166359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2240039a0 0x7fa224003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:06.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.184+0000 7fa236ffd640 1 -- 172.21.15.70:0/1457166359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa238159ac0 msgr2=0x7fa2240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:06.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.184+0000 7fa236ffd640 1 --2- 172.21.15.70:0/1457166359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa238159ac0 0x7fa2240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.184+0000 7fa236ffd640 1 -- 172.21.15.70:0/1457166359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa224004d00 msgr2=0x7fa224005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:06.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.184+0000 7fa236ffd640 1 --2- 172.21.15.70:0/1457166359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa224004d00 0x7fa224005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.184+0000 7fa236ffd640 1 -- 172.21.15.70:0/1457166359 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa224003520 con 0x7fa2240039a0 2023-10-08T05:50:06.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.185+0000 7fa236ffd640 1 --2- 172.21.15.70:0/1457166359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2240039a0 0x7fa224003dc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa228009a90 tx=0x7fa2240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632503ad199644f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:06.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.185+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22802c6d0 con 0x7fa2240039a0 2023-10-08T05:50:06.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.185+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22802c830 con 0x7fa2240039a0 2023-10-08T05:50:06.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.185+0000 7fa2367fc640 1 -- 172.21.15.70:0/1457166359 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22802cbc0 con 0x7fa2240039a0 2023-10-08T05:50:06.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.185+0000 7fa23d4fc640 1 -- 172.21.15.70:0/1457166359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2240039a0 msgr2=0x7fa224003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.185+0000 7fa23d4fc640 1 --2- 172.21.15.70:0/1457166359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2240039a0 0x7fa224003dc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa228009a90 tx=0x7fa2240057f0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.185+0000 7fa2377fe640 1 -- 172.21.15.70:0/1457166359 reap_dead start 2023-10-08T05:50:06.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.186+0000 7fa23d4fc640 1 -- 172.21.15.70:0/1457166359 shutdown_connections 2023-10-08T05:50:06.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.186+0000 7fa23d4fc640 1 -- 172.21.15.70:0/1457166359 >> 172.21.15.70:0/1457166359 conn(0x7fa23806a380 msgr2=0x7fa2380b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:06.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.186+0000 7fa23d4fc640 1 -- 172.21.15.70:0/1457166359 shutdown_connections 2023-10-08T05:50:06.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.186+0000 7fa23d4fc640 1 -- 172.21.15.70:0/1457166359 wait complete. 2023-10-08T05:50:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.187+0000 7fa23d4fc640 1 Processor -- start 2023-10-08T05:50:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.188+0000 7fa23d4fc640 1 -- start start 2023-10-08T05:50:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.188+0000 7fa23d4fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2240039a0 0x7fa2380b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.188+0000 7fa23d4fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2380b19e0 0x7fa2380b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.188+0000 7fa23d4fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2380b3990 0x7fa2380b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.188+0000 7fa23d4fc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380b1e60 con 0x7fa2380b3990 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.188+0000 7fa23d4fc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380cd720 con 0x7fa2380b19e0 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.188+0000 7fa23d4fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2380b4350 con 0x7fa2240039a0 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2380b3990 0x7fa2380b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa2377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2380b3990 0x7fa2380b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39530/0 (socket says 172.21.15.70:39530) 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa2377fe640 1 -- 172.21.15.70:0/919881352 learned_addr learned my addr 172.21.15.70:0/919881352 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa2377fe640 1 -- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2240039a0 msgr2=0x7fa2380b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa236ffd640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2240039a0 0x7fa2380b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa22ffff640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2380b19e0 0x7fa2380b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa2377fe640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2240039a0 0x7fa2380b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa2377fe640 1 -- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2380b19e0 msgr2=0x7fa2380b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa2377fe640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2380b19e0 0x7fa2380b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa2377fe640 1 -- 172.21.15.70:0/919881352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2380ccdf0 con 0x7fa2380b3990 2023-10-08T05:50:06.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.189+0000 7fa2377fe640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2380b3990 0x7fa2380b3e10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa2380ae2f0 tx=0x7fa23000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:06.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.190+0000 7fa22f7fe640 1 -- 172.21.15.70:0/919881352 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23000a5b0 con 0x7fa2380b3990 2023-10-08T05:50:06.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.190+0000 7fa22f7fe640 1 -- 172.21.15.70:0/919881352 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23000a730 con 0x7fa2380b3990 2023-10-08T05:50:06.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.190+0000 7fa22f7fe640 1 -- 172.21.15.70:0/919881352 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23000aae0 con 0x7fa2380b3990 2023-10-08T05:50:06.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.190+0000 7fa23d4fc640 1 -- 172.21.15.70:0/919881352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2380cd0d0 con 0x7fa2380b3990 2023-10-08T05:50:06.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.190+0000 7fa23d4fc640 1 -- 172.21.15.70:0/919881352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa238155430 con 0x7fa2380b3990 2023-10-08T05:50:06.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.191+0000 7fa23d4fc640 1 -- 172.21.15.70:0/919881352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2380c3d80 con 0x7fa2380b3990 2023-10-08T05:50:06.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.192+0000 7fa22f7fe640 1 -- 172.21.15.70:0/919881352 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa23000ac40 con 0x7fa2380b3990 2023-10-08T05:50:06.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.193+0000 7fa22f7fe640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fa21806af90 0x7fa21806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.193+0000 7fa22f7fe640 1 -- 172.21.15.70:0/919881352 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5952+0+0 (secure 0 0 0) 0x7fa230088c30 con 0x7fa2380b3990 2023-10-08T05:50:06.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.193+0000 7fa236ffd640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fa21806af90 0x7fa21806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:06.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.195+0000 7fa236ffd640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fa21806af90 0x7fa21806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa228036000 tx=0x7fa228030000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:06.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.197+0000 7fa22f7fe640 1 -- 172.21.15.70:0/919881352 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa230053ae0 con 0x7fa2380b3990 2023-10-08T05:50:06.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.455+0000 7fa23d4fc640 1 -- 172.21.15.70:0/919881352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fa238067490 con 0x7fa2380b3990 2023-10-08T05:50:06.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.593+0000 7fa22f7fe640 1 -- 172.21.15.70:0/919881352 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v507) v1 ==== 158+0+0 (secure 0 0 0) 0x7fa230057100 con 0x7fa2380b3990 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.596+0000 7fa23d4fc640 1 -- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fa21806af90 msgr2=0x7fa21806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.596+0000 7fa23d4fc640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fa21806af90 0x7fa21806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa228036000 tx=0x7fa228030000 comp rx=0 tx=0).stop 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.596+0000 7fa23d4fc640 1 -- 172.21.15.70:0/919881352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2380b3990 msgr2=0x7fa2380b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.596+0000 7fa23d4fc640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2380b3990 0x7fa2380b3e10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa2380ae2f0 tx=0x7fa23000ccb0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.596+0000 7fa23d4fc640 1 -- 172.21.15.70:0/919881352 shutdown_connections 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.596+0000 7fa23d4fc640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2380b3990 0x7fa2380b3e10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.596+0000 7fa23d4fc640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2380b19e0 0x7fa2380b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.596+0000 7fa23d4fc640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fa21806af90 0x7fa21806d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.596+0000 7fa23d4fc640 1 --2- 172.21.15.70:0/919881352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2240039a0 0x7fa2380b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.596+0000 7fa23d4fc640 1 -- 172.21.15.70:0/919881352 >> 172.21.15.70:0/919881352 conn(0x7fa23806a380 msgr2=0x7fa2380afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.597+0000 7fa23d4fc640 1 -- 172.21.15.70:0/919881352 shutdown_connections 2023-10-08T05:50:06.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.597+0000 7fa23d4fc640 1 -- 172.21.15.70:0/919881352 wait complete. 2023-10-08T05:50:06.616 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-10-08T05:50:06.616 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-10-08T05:50:06.617 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T05:50:06.617 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:50:06.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.830+0000 7fbeb8456640 1 Processor -- start 2023-10-08T05:50:06.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.830+0000 7fbeb8456640 1 -- start start 2023-10-08T05:50:06.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.832+0000 7fbeb8456640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbeb0159ac0 0x7fbeb006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.832+0000 7fbeb8456640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb006dee0 con 0x7fbeb00ad790 2023-10-08T05:50:06.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.832+0000 7fbeb8456640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb015d850 con 0x7fbeb00709e0 2023-10-08T05:50:06.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.832+0000 7fbeb8456640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb015db30 con 0x7fbeb0159ac0 2023-10-08T05:50:06.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.832+0000 7fbeb61f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbeb0159ac0 0x7fbeb006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:06.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.832+0000 7fbeb61f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbeb0159ac0 0x7fbeb006d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39544/0 (socket says 172.21.15.70:39544) 2023-10-08T05:50:06.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.832+0000 7fbeb61f2640 1 -- 172.21.15.70:0/3701245167 learned_addr learned my addr 172.21.15.70:0/3701245167 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:06.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.832+0000 7fbeb59f1640 1 --1- 172.21.15.70:0/3701245167 >> v1:172.21.15.70:6789/0 conn(0x7fbeb00ad790 0x7fbeb00702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40966/0 (socket says 172.21.15.70:40966) 2023-10-08T05:50:06.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.833+0000 7fbeb61f2640 1 -- 172.21.15.70:0/3701245167 >> v1:172.21.15.181:6789/0 conn(0x7fbeb00709e0 legacy=0x7fbeb00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:06.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.833+0000 7fbeb61f2640 1 -- 172.21.15.70:0/3701245167 >> v1:172.21.15.70:6789/0 conn(0x7fbeb00ad790 legacy=0x7fbeb00702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.833+0000 7fbeb61f2640 1 -- 172.21.15.70:0/3701245167 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb015de10 con 0x7fbeb0159ac0 2023-10-08T05:50:06.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.834+0000 7fbeb61f2640 1 --2- 172.21.15.70:0/3701245167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 0x7fbeb006d5e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009870 tx=0x7fbea002dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4262d32f42a36347 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.834+0000 7fbeb51f0640 1 -- 172.21.15.70:0/3701245167 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea002e960 con 0x7fbeb0159ac0 2023-10-08T05:50:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.834+0000 7fbeb51f0640 1 -- 172.21.15.70:0/3701245167 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea002eac0 con 0x7fbeb0159ac0 2023-10-08T05:50:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.834+0000 7fbeb51f0640 1 -- 172.21.15.70:0/3701245167 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea002ee50 con 0x7fbeb0159ac0 2023-10-08T05:50:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.834+0000 7fbeb8456640 1 -- 172.21.15.70:0/3701245167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 msgr2=0x7fbeb006d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.834+0000 7fbeb8456640 1 --2- 172.21.15.70:0/3701245167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 0x7fbeb006d5e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009870 tx=0x7fbea002dbb0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.835+0000 7fbeb8456640 1 -- 172.21.15.70:0/3701245167 shutdown_connections 2023-10-08T05:50:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.835+0000 7fbeb8456640 1 --2- 172.21.15.70:0/3701245167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 0x7fbeb006d5e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.835+0000 7fbeb8456640 1 -- 172.21.15.70:0/3701245167 >> 172.21.15.70:0/3701245167 conn(0x7fbeb006a380 msgr2=0x7fbeb00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.835+0000 7fbeb8456640 1 -- 172.21.15.70:0/3701245167 shutdown_connections 2023-10-08T05:50:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.835+0000 7fbeb8456640 1 -- 172.21.15.70:0/3701245167 wait complete. 2023-10-08T05:50:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.836+0000 7fbeb8456640 1 Processor -- start 2023-10-08T05:50:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.837+0000 7fbeb8456640 1 -- start start 2023-10-08T05:50:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.837+0000 7fbeb8456640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb00709e0 0x7fbeb00b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb8456640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbeb00ad790 0x7fbeb014bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb8456640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 0x7fbeb00af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb8456640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb01f3fd0 con 0x7fbeb0159ac0 2023-10-08T05:50:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb8456640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb014c980 con 0x7fbeb00709e0 2023-10-08T05:50:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb8456640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb00b22f0 con 0x7fbeb00ad790 2023-10-08T05:50:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb69f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 0x7fbeb00af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb69f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 0x7fbeb00af660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39546/0 (socket says 172.21.15.70:39546) 2023-10-08T05:50:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb69f3640 1 -- 172.21.15.70:0/1909117242 learned_addr learned my addr 172.21.15.70:0/1909117242 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb69f3640 1 -- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbeb00ad790 msgr2=0x7fbeb014bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb69f3640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbeb00ad790 0x7fbeb014bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb69f3640 1 -- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb00709e0 msgr2=0x7fbeb00b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb69f3640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb00709e0 0x7fbeb00b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb69f3640 1 -- 172.21.15.70:0/1909117242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00094d0 con 0x7fbeb0159ac0 2023-10-08T05:50:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.838+0000 7fbeb69f3640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 0x7fbeb00af660 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbea800ba10 tx=0x7fbea800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.839+0000 7fbeaeffd640 1 -- 172.21.15.70:0/1909117242 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800cba0 con 0x7fbeb0159ac0 2023-10-08T05:50:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.839+0000 7fbeaeffd640 1 -- 172.21.15.70:0/1909117242 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea800cd00 con 0x7fbeb0159ac0 2023-10-08T05:50:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.839+0000 7fbeaeffd640 1 -- 172.21.15.70:0/1909117242 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8014680 con 0x7fbeb0159ac0 2023-10-08T05:50:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.839+0000 7fbeb8456640 1 -- 172.21.15.70:0/1909117242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb014c490 con 0x7fbeb0159ac0 2023-10-08T05:50:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.839+0000 7fbeb8456640 1 -- 172.21.15.70:0/1909117242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb01f2f60 con 0x7fbeb0159ac0 2023-10-08T05:50:06.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.840+0000 7fbeb8456640 1 -- 172.21.15.70:0/1909117242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb0159ea0 con 0x7fbeb0159ac0 2023-10-08T05:50:06.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.841+0000 7fbeaeffd640 1 -- 172.21.15.70:0/1909117242 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbea80147e0 con 0x7fbeb0159ac0 2023-10-08T05:50:06.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.842+0000 7fbeaeffd640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbe9c06af90 0x7fbe9c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:06.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.842+0000 7fbeaeffd640 1 -- 172.21.15.70:0/1909117242 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5952+0+0 (secure 0 0 0) 0x7fbea80895d0 con 0x7fbeb0159ac0 2023-10-08T05:50:06.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.843+0000 7fbeb61f2640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbe9c06af90 0x7fbe9c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.844+0000 7fbeb61f2640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbe9c06af90 0x7fbe9c06d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbea0038000 tx=0x7fbea0030000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:06.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:06.846+0000 7fbeaeffd640 1 -- 172.21.15.70:0/1909117242 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbea8054490 con 0x7fbeb0159ac0 2023-10-08T05:50:07.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.036+0000 7fbeb8456640 1 -- 172.21.15.70:0/1909117242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbeb014c490 con 0x7fbeb0159ac0 2023-10-08T05:50:07.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.038+0000 7fbeaeffd640 1 -- 172.21.15.70:0/1909117242 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+14199 (secure 0 0 0) 0x7fbeb014c490 con 0x7fbeb0159ac0 2023-10-08T05:50:07.038 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:07.038 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":63,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:50:04.570173+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6832/480502229":"2023-10-09T05:50:01.332157+0000","172.21.15.181:0/32320998":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/2533276123":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/3573082646":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3583094356":"2023-10-09T05:38:04.171728+0000","172.21.15.181:6832/2737793492":"2023-10-09T05:38:04.171728+0000","172.21.15.181:6833/2737793492":"2023-10-09T05:38:04.171728+0000","172.21.15.70:6834/480502229":"2023-10-09T05:50:01.332157+0000","172.21.15.181:6832/2692398351":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/3620192672":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/3280597969":"2023-10-09T05:38:04.171728+0000","172.21.15.181:0/756067864":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/437302488":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/2139016714":"2023-10-09T05:49:57.491954+0000","172.21.15.181:6833/2692398351":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/1816930869":"2023-10-09T05:49:57.491954+0000","172.21.15.181:0/995976827":"2023-10-09T05:49:57.491954+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:50:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 -- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbe9c06af90 msgr2=0x7fbe9c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbe9c06af90 0x7fbe9c06d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbea0038000 tx=0x7fbea0030000 comp rx=0 tx=0).stop 2023-10-08T05:50:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 -- 172.21.15.70:0/1909117242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 msgr2=0x7fbeb00af660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 0x7fbeb00af660 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbea800ba10 tx=0x7fbea800bed0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 -- 172.21.15.70:0/1909117242 shutdown_connections 2023-10-08T05:50:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbeb0159ac0 0x7fbeb00af660 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbe9c06af90 0x7fbe9c06d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbeb00ad790 0x7fbeb014bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 --2- 172.21.15.70:0/1909117242 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbeb00709e0 0x7fbeb00b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 -- 172.21.15.70:0/1909117242 >> 172.21.15.70:0/1909117242 conn(0x7fbeb006a380 msgr2=0x7fbeb014e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.040+0000 7fbeb8456640 1 -- 172.21.15.70:0/1909117242 shutdown_connections 2023-10-08T05:50:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.041+0000 7fbeb8456640 1 -- 172.21.15.70:0/1909117242 wait complete. 2023-10-08T05:50:07.059 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:50:07.059 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:50:07.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.272+0000 7fc603b25640 1 Processor -- start 2023-10-08T05:50:07.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.273+0000 7fc603b25640 1 -- start start 2023-10-08T05:50:07.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.274+0000 7fc603b25640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5fc0b4240 0x7fc5fc1647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.274+0000 7fc603b25640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5fc15d440 con 0x7fc5fc0cb880 2023-10-08T05:50:07.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.274+0000 7fc603b25640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5fc15d690 con 0x7fc5fc1432f0 2023-10-08T05:50:07.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.274+0000 7fc603b25640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5fc1436d0 con 0x7fc5fc0b4240 2023-10-08T05:50:07.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6010c0640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc5fc0cb880 0x7fc5fc0b3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40972/0 (socket says 172.21.15.70:40972) 2023-10-08T05:50:07.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6010c0640 1 -- 172.21.15.70:0/1135704660 learned_addr learned my addr 172.21.15.70:0/1135704660 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:07.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6020c2640 1 --2- 172.21.15.70:0/1135704660 >> v2:172.21.15.181:3300/0 conn(0x7fc5fc0b4240 0x7fc5fc1647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507692964 0 0) 0x7fc5fc15d440 con 0x7fc5fc0cb880 2023-10-08T05:50:07.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5e4003520 con 0x7fc5fc0cb880 2023-10-08T05:50:07.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3759816239 0 0) 0x7fc5e4003520 con 0x7fc5fc0cb880 2023-10-08T05:50:07.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 >> v2:172.21.15.181:3300/0 conn(0x7fc5fc0b4240 msgr2=0x7fc5fc1647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:07.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6008bf640 1 --2- 172.21.15.70:0/1135704660 >> v2:172.21.15.181:3300/0 conn(0x7fc5fc0b4240 0x7fc5fc1647d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 >> v1:172.21.15.181:6789/0 conn(0x7fc5fc1432f0 legacy=0x7fc5fc0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc15d970 con 0x7fc5fc0cb880 2023-10-08T05:50:07.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.275+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc5f0001670 con 0x7fc5fc0cb880 2023-10-08T05:50:07.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 >> v1:172.21.15.70:6789/0 conn(0x7fc5fc0cb880 legacy=0x7fc5fc0b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6008bf640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e4003970 0x7fc5e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6008bf640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e40047c0 0x7fc5e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6008bf640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e4005180 0x7fc5e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc1436d0 con 0x7fc5e40047c0 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e400c120 con 0x7fc5e4003970 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e400c2e0 con 0x7fc5e4005180 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6010c0640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e40047c0 0x7fc5e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6010c0640 1 -- 172.21.15.70:0/1135704660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e4005180 msgr2=0x7fc5e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6010c0640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5e4005180 0x7fc5e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6018c1640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e4003970 0x7fc5e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6010c0640 1 -- 172.21.15.70:0/1135704660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e4003970 msgr2=0x7fc5e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6010c0640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e4003970 0x7fc5e4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6010c0640 1 -- 172.21.15.70:0/1135704660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e4003520 con 0x7fc5e40047c0 2023-10-08T05:50:07.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6020c2640 1 -- 172.21.15.70:0/1135704660 reap_dead start 2023-10-08T05:50:07.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.276+0000 7fc6018c1640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e4003970 0x7fc5e4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:07.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.277+0000 7fc6010c0640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e40047c0 0x7fc5e4004c40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c160 tx=0x7fc5e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce352782cebecce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:07.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.278+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0005ad0 con 0x7fc5e40047c0 2023-10-08T05:50:07.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.278+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f0005c50 con 0x7fc5e40047c0 2023-10-08T05:50:07.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.278+0000 7fc6008bf640 1 -- 172.21.15.70:0/1135704660 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f002c850 con 0x7fc5e40047c0 2023-10-08T05:50:07.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.278+0000 7fc603b25640 1 -- 172.21.15.70:0/1135704660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e40047c0 msgr2=0x7fc5e4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.278+0000 7fc603b25640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e40047c0 0x7fc5e4004c40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c160 tx=0x7fc5e40057f0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.278+0000 7fc603b25640 1 -- 172.21.15.70:0/1135704660 shutdown_connections 2023-10-08T05:50:07.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.278+0000 7fc603b25640 1 --2- 172.21.15.70:0/1135704660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e40047c0 0x7fc5e4004c40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.278+0000 7fc603b25640 1 -- 172.21.15.70:0/1135704660 >> 172.21.15.70:0/1135704660 conn(0x7fc5fc069d50 msgr2=0x7fc5fc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:07.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.279+0000 7fc603b25640 1 -- 172.21.15.70:0/1135704660 shutdown_connections 2023-10-08T05:50:07.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.279+0000 7fc603b25640 1 -- 172.21.15.70:0/1135704660 wait complete. 2023-10-08T05:50:07.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.280+0000 7fc603b25640 1 Processor -- start 2023-10-08T05:50:07.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.280+0000 7fc603b25640 1 -- start start 2023-10-08T05:50:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.281+0000 7fc603b25640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e40047c0 0x7fc5fc1f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.281+0000 7fc603b25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc13dd70 0x7fc5fc13a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.281+0000 7fc603b25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5fc13f4f0 0x7fc5fc140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.281+0000 7fc603b25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc13e550 con 0x7fc5fc13dd70 2023-10-08T05:50:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.281+0000 7fc603b25640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc13adc0 con 0x7fc5e40047c0 2023-10-08T05:50:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.281+0000 7fc603b25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc13e730 con 0x7fc5fc13f4f0 2023-10-08T05:50:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.282+0000 7fc6010c0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc13dd70 0x7fc5fc13a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.282+0000 7fc6010c0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc13dd70 0x7fc5fc13a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39562/0 (socket says 172.21.15.70:39562) 2023-10-08T05:50:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.282+0000 7fc6010c0640 1 -- 172.21.15.70:0/1344378907 learned_addr learned my addr 172.21.15.70:0/1344378907 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.282+0000 7fc6018c1640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e40047c0 0x7fc5fc1f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.282+0000 7fc6010c0640 1 -- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5fc13f4f0 msgr2=0x7fc5fc140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.282+0000 7fc6010c0640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5fc13f4f0 0x7fc5fc140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.282+0000 7fc6010c0640 1 -- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e40047c0 msgr2=0x7fc5fc1f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.282+0000 7fc6010c0640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e40047c0 0x7fc5fc1f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.282+0000 7fc6010c0640 1 -- 172.21.15.70:0/1344378907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc13ea10 con 0x7fc5fc13dd70 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.283+0000 7fc6010c0640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc13dd70 0x7fc5fc13a880 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc5f002c3f0 tx=0x7fc5f0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.283+0000 7fc5fa7fc640 1 -- 172.21.15.70:0/1344378907 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f003c040 con 0x7fc5fc13dd70 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.283+0000 7fc5fa7fc640 1 -- 172.21.15.70:0/1344378907 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f002cc60 con 0x7fc5fc13dd70 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.283+0000 7fc5fa7fc640 1 -- 172.21.15.70:0/1344378907 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0007070 con 0x7fc5fc13dd70 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.283+0000 7fc603b25640 1 -- 172.21.15.70:0/1344378907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5fc13ecf0 con 0x7fc5fc13dd70 2023-10-08T05:50:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.283+0000 7fc603b25640 1 -- 172.21.15.70:0/1344378907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5fc157780 con 0x7fc5fc13dd70 2023-10-08T05:50:07.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.284+0000 7fc603b25640 1 -- 172.21.15.70:0/1344378907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5fc13e870 con 0x7fc5fc13dd70 2023-10-08T05:50:07.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.285+0000 7fc5fa7fc640 1 -- 172.21.15.70:0/1344378907 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc5f0032070 con 0x7fc5fc13dd70 2023-10-08T05:50:07.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.286+0000 7fc5fa7fc640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fc5e806afc0 0x7fc5e806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.286+0000 7fc5fa7fc640 1 -- 172.21.15.70:0/1344378907 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5952+0+0 (secure 0 0 0) 0x7fc5f00aae30 con 0x7fc5fc13dd70 2023-10-08T05:50:07.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.287+0000 7fc6018c1640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fc5e806afc0 0x7fc5e806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.288+0000 7fc6018c1640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fc5e806afc0 0x7fc5e806d480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec007fd0 tx=0x7fc5ec009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.290+0000 7fc5fa7fc640 1 -- 172.21.15.70:0/1344378907 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5fc13e870 con 0x7fc5fc13dd70 2023-10-08T05:50:07.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.481+0000 7fc603b25640 1 -- 172.21.15.70:0/1344378907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5fc13e870 con 0x7fc5fc13dd70 2023-10-08T05:50:07.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.481+0000 7fc5fa7fc640 1 -- 172.21.15.70:0/1344378907 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5fc13e870 con 0x7fc5fc13dd70 2023-10-08T05:50:07.483 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:50:07.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 -- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fc5e806afc0 msgr2=0x7fc5e806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fc5e806afc0 0x7fc5e806d480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec007fd0 tx=0x7fc5ec009040 comp rx=0 tx=0).stop 2023-10-08T05:50:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 -- 172.21.15.70:0/1344378907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc13dd70 msgr2=0x7fc5fc13a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc13dd70 0x7fc5fc13a880 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc5f002c3f0 tx=0x7fc5f0000fe0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 -- 172.21.15.70:0/1344378907 shutdown_connections 2023-10-08T05:50:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fc5e806afc0 0x7fc5e806d480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5fc13f4f0 0x7fc5fc140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc13dd70 0x7fc5fc13a880 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 --2- 172.21.15.70:0/1344378907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5e40047c0 0x7fc5fc1f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 -- 172.21.15.70:0/1344378907 >> 172.21.15.70:0/1344378907 conn(0x7fc5fc069d50 msgr2=0x7fc5fc0b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 -- 172.21.15.70:0/1344378907 shutdown_connections 2023-10-08T05:50:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.484+0000 7fc603b25640 1 -- 172.21.15.70:0/1344378907 wait complete. 2023-10-08T05:50:07.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-08T05:50:07.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.714+0000 7fe1a4ee1640 1 Processor -- start 2023-10-08T05:50:07.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.714+0000 7fe1a4ee1640 1 -- start start 2023-10-08T05:50:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.716+0000 7fe1a4ee1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1a015cd90 0x7fe1a00702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.716+0000 7fe1a4ee1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1a006c890 0x7fe1a006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.716+0000 7fe1a4ee1640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a00c48f0 con 0x7fe1a0160940 2023-10-08T05:50:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.716+0000 7fe1a4ee1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a00c4d80 con 0x7fe1a006c890 2023-10-08T05:50:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.716+0000 7fe1a4ee1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a00c3ce0 con 0x7fe1a015cd90 2023-10-08T05:50:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.716+0000 7fe19ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1a006c890 0x7fe1a006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.717+0000 7fe19ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1a006c890 0x7fe1a006cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39568/0 (socket says 172.21.15.70:39568) 2023-10-08T05:50:07.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.717+0000 7fe19ed9d640 1 -- 172.21.15.70:0/3570373413 learned_addr learned my addr 172.21.15.70:0/3570373413 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.717+0000 7fe19ed9d640 1 -- 172.21.15.70:0/3570373413 >> v2:172.21.15.181:3300/0 conn(0x7fe1a015cd90 msgr2=0x7fe1a00702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.717+0000 7fe19e59c640 1 --2- 172.21.15.70:0/3570373413 >> v2:172.21.15.181:3300/0 conn(0x7fe1a015cd90 0x7fe1a00702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.717+0000 7fe19ed9d640 1 --2- 172.21.15.70:0/3570373413 >> v2:172.21.15.181:3300/0 conn(0x7fe1a015cd90 0x7fe1a00702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.717+0000 7fe19ed9d640 1 -- 172.21.15.70:0/3570373413 >> v1:172.21.15.181:6789/0 conn(0x7fe1a0160940 legacy=0x7fe1a0070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.717+0000 7fe19ed9d640 1 -- 172.21.15.70:0/3570373413 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00c3fc0 con 0x7fe1a006c890 2023-10-08T05:50:07.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.718+0000 7fe19ed9d640 1 --2- 172.21.15.70:0/3570373413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a006c890 0x7fe1a006cd20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe1a006e440 tx=0x7fe19802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12fe17098377524 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.718+0000 7fe19dd9b640 1 -- 172.21.15.70:0/3570373413 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198031870 con 0x7fe1a006c890 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.719+0000 7fe19dd9b640 1 -- 172.21.15.70:0/3570373413 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1980319d0 con 0x7fe1a006c890 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.719+0000 7fe19dd9b640 1 -- 172.21.15.70:0/3570373413 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198031d60 con 0x7fe1a006c890 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.719+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/3570373413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a006c890 msgr2=0x7fe1a006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.719+0000 7fe1a4ee1640 1 --2- 172.21.15.70:0/3570373413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a006c890 0x7fe1a006cd20 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe1a006e440 tx=0x7fe19802b910 comp rx=0 tx=0).stop 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.719+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/3570373413 shutdown_connections 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.719+0000 7fe1a4ee1640 1 --2- 172.21.15.70:0/3570373413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a006c890 0x7fe1a006cd20 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.719+0000 7fe1a4ee1640 1 --2- 172.21.15.70:0/3570373413 >> v2:172.21.15.181:3300/0 conn(0x7fe1a015cd90 0x7fe1a00702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.719+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/3570373413 >> 172.21.15.70:0/3570373413 conn(0x7fe1a006a380 msgr2=0x7fe1a00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.720+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/3570373413 shutdown_connections 2023-10-08T05:50:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.720+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/3570373413 wait complete. 2023-10-08T05:50:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.722+0000 7fe1a4ee1640 1 Processor -- start 2023-10-08T05:50:07.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.722+0000 7fe1a4ee1640 1 -- start start 2023-10-08T05:50:07.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.723+0000 7fe1a4ee1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a006c890 0x7fe1a015b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.723+0000 7fe1a4ee1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a015cd90 0x7fe1a00af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.723+0000 7fe1a4ee1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a0160940 0x7fe1a00af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.723+0000 7fe1a4ee1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a0158ad0 con 0x7fe1a015cd90 2023-10-08T05:50:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.723+0000 7fe1a4ee1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a0158d90 con 0x7fe1a0160940 2023-10-08T05:50:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.723+0000 7fe1a4ee1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a0159070 con 0x7fe1a006c890 2023-10-08T05:50:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.723+0000 7fe195d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a015cd90 0x7fe1a00af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.723+0000 7fe195d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a015cd90 0x7fe1a00af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39574/0 (socket says 172.21.15.70:39574) 2023-10-08T05:50:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.723+0000 7fe195d9b640 1 -- 172.21.15.70:0/1760140894 learned_addr learned my addr 172.21.15.70:0/1760140894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe19ed9d640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a0160940 0x7fe1a00af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe19e59c640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a006c890 0x7fe1a015b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe195d9b640 1 -- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a006c890 msgr2=0x7fe1a015b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe195d9b640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a006c890 0x7fe1a015b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe195d9b640 1 -- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a0160940 msgr2=0x7fe1a00af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe195d9b640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a0160940 0x7fe1a00af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe195d9b640 1 -- 172.21.15.70:0/1760140894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980094d0 con 0x7fe1a015cd90 2023-10-08T05:50:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe19ed9d640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a0160940 0x7fe1a00af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe195d9b640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a015cd90 0x7fe1a00af490 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe18800b3e0 tx=0x7fe18800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe1977fe640 1 -- 172.21.15.70:0/1760140894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188011830 con 0x7fe1a015cd90 2023-10-08T05:50:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe1977fe640 1 -- 172.21.15.70:0/1760140894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1880119b0 con 0x7fe1a015cd90 2023-10-08T05:50:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe1977fe640 1 -- 172.21.15.70:0/1760140894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188011d60 con 0x7fe1a015cd90 2023-10-08T05:50:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.724+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/1760140894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a0159350 con 0x7fe1a015cd90 2023-10-08T05:50:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.725+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/1760140894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a00cb2a0 con 0x7fe1a015cd90 2023-10-08T05:50:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.726+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/1760140894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a0159870 con 0x7fe1a015cd90 2023-10-08T05:50:07.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.727+0000 7fe1977fe640 1 -- 172.21.15.70:0/1760140894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe18801a400 con 0x7fe1a015cd90 2023-10-08T05:50:07.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.727+0000 7fe1977fe640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe18006af90 0x7fe18006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:07.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.728+0000 7fe1977fe640 1 -- 172.21.15.70:0/1760140894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5952+0+0 (secure 0 0 0) 0x7fe188089a70 con 0x7fe1a015cd90 2023-10-08T05:50:07.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.728+0000 7fe19e59c640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe18006af90 0x7fe18006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:07.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.729+0000 7fe19e59c640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe18006af90 0x7fe18006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe190008ef0 tx=0x7fe190008440 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:07.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.732+0000 7fe1977fe640 1 -- 172.21.15.70:0/1760140894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1a0159870 con 0x7fe1a015cd90 2023-10-08T05:50:07.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:07.990+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/1760140894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fe1a0159870 con 0x7fe1a015cd90 2023-10-08T05:50:08.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.597+0000 7fe1977fe640 1 -- 172.21.15.70:0/1760140894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v509) v1 ==== 157+0+0 (secure 0 0 0) 0x7fe1a0159870 con 0x7fe1a015cd90 2023-10-08T05:50:08.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe18006af90 msgr2=0x7fe18006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:08.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe18006af90 0x7fe18006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe190008ef0 tx=0x7fe190008440 comp rx=0 tx=0).stop 2023-10-08T05:50:08.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/1760140894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a015cd90 msgr2=0x7fe1a00af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:08.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a015cd90 0x7fe1a00af490 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe18800b3e0 tx=0x7fe18800b8a0 comp rx=0 tx=0).stop 2023-10-08T05:50:08.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/1760140894 shutdown_connections 2023-10-08T05:50:08.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1a0160940 0x7fe1a00af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:08.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1a015cd90 0x7fe1a00af490 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:08.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe18006af90 0x7fe18006d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:08.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 --2- 172.21.15.70:0/1760140894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1a006c890 0x7fe1a015b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:08.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/1760140894 >> 172.21.15.70:0/1760140894 conn(0x7fe1a006a380 msgr2=0x7fe1a0160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:08.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.601+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/1760140894 shutdown_connections 2023-10-08T05:50:08.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.602+0000 7fe1a4ee1640 1 -- 172.21.15.70:0/1760140894 wait complete. 2023-10-08T05:50:08.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T05:50:08.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.826+0000 7f91e6e5d640 1 Processor -- start 2023-10-08T05:50:08.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.827+0000 7f91e6e5d640 1 -- start start 2023-10-08T05:50:08.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.828+0000 7f91e6e5d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91e0160660 0x7f91e0070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:08.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.828+0000 7f91e6e5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91e00c9910 0x7f91e00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:08.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.828+0000 7f91e6e5d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e006e150 con 0x7f91e00cb450 2023-10-08T05:50:08.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.828+0000 7f91e6e5d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e006de70 con 0x7f91e00c9910 2023-10-08T05:50:08.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.828+0000 7f91e6e5d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e00c8e30 con 0x7f91e0160660 2023-10-08T05:50:08.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.828+0000 7f91dffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91e00c9910 0x7f91e00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:08.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.829+0000 7f91dffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91e00c9910 0x7f91e00caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39582/0 (socket says 172.21.15.70:39582) 2023-10-08T05:50:08.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.829+0000 7f91dffff640 1 -- 172.21.15.70:0/3872287530 learned_addr learned my addr 172.21.15.70:0/3872287530 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:08.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.829+0000 7f91e4bf9640 1 --2- 172.21.15.70:0/3872287530 >> v2:172.21.15.181:3300/0 conn(0x7f91e0160660 0x7f91e0070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:08.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.829+0000 7f91dffff640 1 -- 172.21.15.70:0/3872287530 >> v2:172.21.15.181:3300/0 conn(0x7f91e0160660 msgr2=0x7f91e0070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:08.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.829+0000 7f91dffff640 1 --2- 172.21.15.70:0/3872287530 >> v2:172.21.15.181:3300/0 conn(0x7f91e0160660 0x7f91e0070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:08.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.829+0000 7f91dffff640 1 -- 172.21.15.70:0/3872287530 >> v1:172.21.15.181:6789/0 conn(0x7f91e00cb450 legacy=0x7f91e00b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:08.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.829+0000 7f91dffff640 1 -- 172.21.15.70:0/3872287530 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00c9110 con 0x7f91e00c9910 2023-10-08T05:50:08.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.829+0000 7f91e4bf9640 1 --2- 172.21.15.70:0/3872287530 >> v2:172.21.15.181:3300/0 conn(0x7f91e0160660 0x7f91e0070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:08.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.830+0000 7f91dffff640 1 --2- 172.21.15.70:0/3872287530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 0x7f91e00caf10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f91e00b0d80 tx=0x7f91cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ec7947606bf798 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:08.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.830+0000 7f91df7fe640 1 -- 172.21.15.70:0/3872287530 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc031870 con 0x7f91e00c9910 2023-10-08T05:50:08.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.831+0000 7f91df7fe640 1 -- 172.21.15.70:0/3872287530 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc0319d0 con 0x7f91e00c9910 2023-10-08T05:50:08.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.831+0000 7f91df7fe640 1 -- 172.21.15.70:0/3872287530 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc031d60 con 0x7f91e00c9910 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.831+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3872287530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 msgr2=0x7f91e00caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.831+0000 7f91e6e5d640 1 --2- 172.21.15.70:0/3872287530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 0x7f91e00caf10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f91e00b0d80 tx=0x7f91cc02b910 comp rx=0 tx=0).stop 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.831+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3872287530 shutdown_connections 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.831+0000 7f91e6e5d640 1 --2- 172.21.15.70:0/3872287530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 0x7f91e00caf10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.831+0000 7f91e6e5d640 1 --2- 172.21.15.70:0/3872287530 >> v2:172.21.15.181:3300/0 conn(0x7f91e0160660 0x7f91e0070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.831+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3872287530 >> 172.21.15.70:0/3872287530 conn(0x7f91e006a5d0 msgr2=0x7f91e00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.831+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3872287530 shutdown_connections 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.831+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3872287530 wait complete. 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.833+0000 7f91e6e5d640 1 Processor -- start 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.833+0000 7f91e6e5d640 1 -- start start 2023-10-08T05:50:08.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.834+0000 7f91e6e5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 0x7f91e0156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.834+0000 7f91e6e5d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91e00cb450 0x7f91e0157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.834+0000 7f91e6e5d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91e0160660 0x7f91e00c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.834+0000 7f91e6e5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00c4cc0 con 0x7f91e00c9910 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.834+0000 7f91e6e5d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0157b10 con 0x7f91e0160660 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.834+0000 7f91e6e5d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0157df0 con 0x7f91e00cb450 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.834+0000 7f91e4bf9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 0x7f91e0156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91e4bf9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 0x7f91e0156d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39584/0 (socket says 172.21.15.70:39584) 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91e4bf9640 1 -- 172.21.15.70:0/3586717585 learned_addr learned my addr 172.21.15.70:0/3586717585 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91e53fa640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91e0160660 0x7f91e00c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91e4bf9640 1 -- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91e00cb450 msgr2=0x7f91e0157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:08.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91dffff640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91e00cb450 0x7f91e0157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:08.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91e4bf9640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91e00cb450 0x7f91e0157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:08.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91e4bf9640 1 -- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91e0160660 msgr2=0x7f91e00c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:08.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91e4bf9640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91e0160660 0x7f91e00c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:08.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91e4bf9640 1 -- 172.21.15.70:0/3586717585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0094d0 con 0x7f91e00c9910 2023-10-08T05:50:08.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91e4bf9640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 0x7f91e0156d00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f91d400c7f0 tx=0x7f91d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:08.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.835+0000 7f91dd7fa640 1 -- 172.21.15.70:0/3586717585 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4007820 con 0x7f91e00c9910 2023-10-08T05:50:08.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.836+0000 7f91dd7fa640 1 -- 172.21.15.70:0/3586717585 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d4007980 con 0x7f91e00c9910 2023-10-08T05:50:08.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.836+0000 7f91dd7fa640 1 -- 172.21.15.70:0/3586717585 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4007cf0 con 0x7f91e00c9910 2023-10-08T05:50:08.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.836+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3586717585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e01580d0 con 0x7f91e00c9910 2023-10-08T05:50:08.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.836+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3586717585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e0158690 con 0x7f91e00c9910 2023-10-08T05:50:08.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.837+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3586717585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e004e160 con 0x7f91e00c9910 2023-10-08T05:50:08.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.838+0000 7f91dd7fa640 1 -- 172.21.15.70:0/3586717585 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f91d40113f0 con 0x7f91e00c9910 2023-10-08T05:50:08.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.839+0000 7f91dd7fa640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f91c406af90 0x7f91c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:08.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.839+0000 7f91dd7fa640 1 -- 172.21.15.70:0/3586717585 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f91d4089c10 con 0x7f91e00c9910 2023-10-08T05:50:08.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.839+0000 7f91dffff640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f91c406af90 0x7f91c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:08.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.840+0000 7f91dffff640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f91c406af90 0x7f91c406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91cc036000 tx=0x7f91cc03c040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:08.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:08.843+0000 7f91dd7fa640 1 -- 172.21.15.70:0/3586717585 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91d4054a40 con 0x7f91e00c9910 2023-10-08T05:50:08.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:50:08.991 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:50:09.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.032+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3586717585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f91e0071070 con 0x7f91e00c9910 2023-10-08T05:50:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.036+0000 7f91dd7fa640 1 -- 172.21.15.70:0/3586717585 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91d401f0c0 con 0x7f91e00c9910 2023-10-08T05:50:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f91c406af90 msgr2=0x7f91c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f91c406af90 0x7f91c406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91cc036000 tx=0x7f91cc03c040 comp rx=0 tx=0).stop 2023-10-08T05:50:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3586717585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 msgr2=0x7f91e0156d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 0x7f91e0156d00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f91d400c7f0 tx=0x7f91d400ccb0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3586717585 shutdown_connections 2023-10-08T05:50:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91e0160660 0x7f91e00c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f91c406af90 0x7f91c406d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91e00cb450 0x7f91e0157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 --2- 172.21.15.70:0/3586717585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e00c9910 0x7f91e0156d00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3586717585 >> 172.21.15.70:0/3586717585 conn(0x7f91e006a5d0 msgr2=0x7f91e00c8090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3586717585 shutdown_connections 2023-10-08T05:50:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.039+0000 7f91e6e5d640 1 -- 172.21.15.70:0/3586717585 wait complete. 2023-10-08T05:50:09.058 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:50:09.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:50:09.168 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:50:09.295 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:50:09.295 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:50:09.295 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:50:09.354 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:50:09.354 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:50:09.354 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:50:09.413 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:50:09.414 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:50:09.414 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:50:09.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:50:09.472 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:50:09.472 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:50:09.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:50:09.532 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T05:50:09.532 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:50:09.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.755+0000 7f4262f4c640 1 Processor -- start 2023-10-08T05:50:09.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.755+0000 7f4262f4c640 1 -- start start 2023-10-08T05:50:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.757+0000 7f4262f4c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f425c15ab40 0x7f425c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.757+0000 7f4262f4c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f425c0c90b0 0x7f425c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.757+0000 7f4262f4c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c15b2b0 con 0x7f425c0cae70 2023-10-08T05:50:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.757+0000 7f4262f4c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425c15bf30 con 0x7f425c0c90b0 2023-10-08T05:50:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.757+0000 7f4262f4c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425c15f7a0 con 0x7f425c15ab40 2023-10-08T05:50:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.757+0000 7f425bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f425c0c90b0 0x7f425c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.757+0000 7f42614e9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f425c0cae70 0x7f425c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40980/0 (socket says 172.21.15.70:40980) 2023-10-08T05:50:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.757+0000 7f42614e9640 1 -- 172.21.15.70:0/768207595 learned_addr learned my addr 172.21.15.70:0/768207595 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.757+0000 7f4260ce8640 1 --2- 172.21.15.70:0/768207595 >> v2:172.21.15.181:3300/0 conn(0x7f425c15ab40 0x7f425c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:09.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.758+0000 7f425bfff640 1 -- 172.21.15.70:0/768207595 >> v2:172.21.15.181:3300/0 conn(0x7f425c15ab40 msgr2=0x7f425c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:09.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.758+0000 7f425bfff640 1 --2- 172.21.15.70:0/768207595 >> v2:172.21.15.181:3300/0 conn(0x7f425c15ab40 0x7f425c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.758+0000 7f425bfff640 1 -- 172.21.15.70:0/768207595 >> v1:172.21.15.70:6789/0 conn(0x7f425c0cae70 legacy=0x7f425c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:09.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.758+0000 7f425bfff640 1 -- 172.21.15.70:0/768207595 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c15b590 con 0x7f425c0c90b0 2023-10-08T05:50:09.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.758+0000 7f425b7fe640 1 -- 172.21.15.70:0/768207595 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24993630 0 0) 0x7f425c15b2b0 con 0x7f425c0cae70 2023-10-08T05:50:09.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.759+0000 7f425bfff640 1 --2- 172.21.15.70:0/768207595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 0x7f425c0adc60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f425c06e440 tx=0x7f424802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a3f73bfceaee4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:09.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.759+0000 7f425b7fe640 1 -- 172.21.15.70:0/768207595 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248031830 con 0x7f425c0c90b0 2023-10-08T05:50:09.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.759+0000 7f425b7fe640 1 -- 172.21.15.70:0/768207595 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4248031990 con 0x7f425c0c90b0 2023-10-08T05:50:09.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.759+0000 7f425b7fe640 1 -- 172.21.15.70:0/768207595 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248031d20 con 0x7f425c0c90b0 2023-10-08T05:50:09.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.760+0000 7f4262f4c640 1 -- 172.21.15.70:0/768207595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 msgr2=0x7f425c0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.760+0000 7f4262f4c640 1 --2- 172.21.15.70:0/768207595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 0x7f425c0adc60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f425c06e440 tx=0x7f424802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.760+0000 7f4262f4c640 1 -- 172.21.15.70:0/768207595 shutdown_connections 2023-10-08T05:50:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.760+0000 7f4262f4c640 1 --2- 172.21.15.70:0/768207595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 0x7f425c0adc60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.760+0000 7f4262f4c640 1 --2- 172.21.15.70:0/768207595 >> v2:172.21.15.181:3300/0 conn(0x7f425c15ab40 0x7f425c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.760+0000 7f4262f4c640 1 -- 172.21.15.70:0/768207595 >> 172.21.15.70:0/768207595 conn(0x7f425c06a380 msgr2=0x7f425c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.760+0000 7f4262f4c640 1 -- 172.21.15.70:0/768207595 shutdown_connections 2023-10-08T05:50:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.760+0000 7f4262f4c640 1 -- 172.21.15.70:0/768207595 wait complete. 2023-10-08T05:50:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.762+0000 7f4262f4c640 1 Processor -- start 2023-10-08T05:50:09.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.762+0000 7f4262f4c640 1 -- start start 2023-10-08T05:50:09.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f4262f4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 0x7f425c0acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:09.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f4262f4c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0cae70 0x7f425c0c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:09.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f4262f4c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c15ab40 0x7f425c145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:09.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f4262f4c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0c3620 con 0x7f425c0c90b0 2023-10-08T05:50:09.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f4262f4c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c146520 con 0x7f425c0cae70 2023-10-08T05:50:09.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f4262f4c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c146800 con 0x7f425c15ab40 2023-10-08T05:50:09.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f4260ce8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 0x7f425c0acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:09.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f4260ce8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 0x7f425c0acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39602/0 (socket says 172.21.15.70:39602) 2023-10-08T05:50:09.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f4260ce8640 1 -- 172.21.15.70:0/1094738656 learned_addr learned my addr 172.21.15.70:0/1094738656 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:09.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f425bfff640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0cae70 0x7f425c0c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:09.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f42614e9640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c15ab40 0x7f425c145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:09.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.763+0000 7f4260ce8640 1 -- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c15ab40 msgr2=0x7f425c145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:09.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f4260ce8640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c15ab40 0x7f425c145aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f4260ce8640 1 -- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0cae70 msgr2=0x7f425c0c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:09.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f4260ce8640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0cae70 0x7f425c0c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:09.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f4260ce8640 1 -- 172.21.15.70:0/1094738656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4248009400 con 0x7f425c0c90b0 2023-10-08T05:50:09.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f425bfff640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0cae70 0x7f425c0c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:09.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f4260ce8640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 0x7f425c0acb80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f425000d800 tx=0x7f425000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:09.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f42597fa640 1 -- 172.21.15.70:0/1094738656 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425000b6c0 con 0x7f425c0c90b0 2023-10-08T05:50:09.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f42597fa640 1 -- 172.21.15.70:0/1094738656 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425000b820 con 0x7f425c0c90b0 2023-10-08T05:50:09.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f42597fa640 1 -- 172.21.15.70:0/1094738656 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425000bbb0 con 0x7f425c0c90b0 2023-10-08T05:50:09.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f4262f4c640 1 -- 172.21.15.70:0/1094738656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c0c3870 con 0x7f425c0c90b0 2023-10-08T05:50:09.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.764+0000 7f4262f4c640 1 -- 172.21.15.70:0/1094738656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c149cd0 con 0x7f425c0c90b0 2023-10-08T05:50:09.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.765+0000 7f4262f4c640 1 -- 172.21.15.70:0/1094738656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c067490 con 0x7f425c0c90b0 2023-10-08T05:50:09.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.766+0000 7f42597fa640 1 -- 172.21.15.70:0/1094738656 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f425000bd10 con 0x7f425c0c90b0 2023-10-08T05:50:09.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.767+0000 7f42597fa640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f424006afc0 0x7f424006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:09.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.767+0000 7f42597fa640 1 -- 172.21.15.70:0/1094738656 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f4250089da0 con 0x7f425c0c90b0 2023-10-08T05:50:09.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.768+0000 7f425bfff640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f424006afc0 0x7f424006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:09.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.769+0000 7f425bfff640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f424006afc0 0x7f424006d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4248036000 tx=0x7f4248008030 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:09.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.771+0000 7f42597fa640 1 -- 172.21.15.70:0/1094738656 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4250054c60 con 0x7f425c0c90b0 2023-10-08T05:50:09.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.998+0000 7f4262f4c640 1 -- 172.21.15.70:0/1094738656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f425c0b0ef0 con 0x7f425c0c90b0 2023-10-08T05:50:10.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:09.999+0000 7f42597fa640 1 -- 172.21.15.70:0/1094738656 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f4250058280 con 0x7f425c0c90b0 2023-10-08T05:50:10.001 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:10.001 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/809656839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":809656839},{"type":"v1","addr":"172.21.15.70:6835","nonce":809656839}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4822,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/2750508216","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2750508216},{"type":"v1","addr":"172.21.15.181:6836","nonce":2750508216}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4825,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/607266140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":607266140},{"type":"v1","addr":"172.21.15.181:6837","nonce":607266140}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":6075,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2409457670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2409457670},{"type":"v1","addr":"172.21.15.70:6834","nonce":2409457670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2023-10-08T05:50:10.002 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 20 2023-10-08T05:50:10.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 -- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f424006afc0 msgr2=0x7f424006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:10.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f424006afc0 0x7f424006d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4248036000 tx=0x7f4248008030 comp rx=0 tx=0).stop 2023-10-08T05:50:10.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 -- 172.21.15.70:0/1094738656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 msgr2=0x7f425c0acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:10.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 0x7f425c0acb80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f425000d800 tx=0x7f425000dcc0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 -- 172.21.15.70:0/1094738656 shutdown_connections 2023-10-08T05:50:10.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f424006afc0 0x7f424006d480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f425c15ab40 0x7f425c145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f425c0cae70 0x7f425c0c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 --2- 172.21.15.70:0/1094738656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425c0c90b0 0x7f425c0acb80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 -- 172.21.15.70:0/1094738656 >> 172.21.15.70:0/1094738656 conn(0x7f425c06a380 msgr2=0x7f425c154440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:10.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.002+0000 7f4262f4c640 1 -- 172.21.15.70:0/1094738656 shutdown_connections 2023-10-08T05:50:10.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.003+0000 7f4262f4c640 1 -- 172.21.15.70:0/1094738656 wait complete. 2023-10-08T05:50:10.022 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T05:50:10.022 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T05:50:10.022 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T05:50:10.022 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:50:10.022 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T05:50:10.022 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T05:50:10.023 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T05:50:10.023 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:50:10.023 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T05:50:10.023 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T05:50:10.023 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T05:50:10.023 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:50:10.023 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T05:50:10.024 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T05:50:10.024 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T05:50:10.024 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:50:10.037 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T05:50:10.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T05:50:10.040 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T05:50:10.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T05:50:10.046 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T05:50:10.046 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T05:50:10.053 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T05:50:10.053 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T05:50:10.087 INFO:tasks.ceph.mds.c:Started 2023-10-08T05:50:10.088 DEBUG:teuthology.parallel:result is None 2023-10-08T05:50:10.096 INFO:tasks.ceph.mds.a:Started 2023-10-08T05:50:10.096 DEBUG:teuthology.parallel:result is None 2023-10-08T05:50:10.097 INFO:tasks.ceph.mds.d:Started 2023-10-08T05:50:10.098 DEBUG:teuthology.parallel:result is None 2023-10-08T05:50:10.105 INFO:tasks.ceph.mds.b:Started 2023-10-08T05:50:10.105 DEBUG:teuthology.parallel:result is None 2023-10-08T05:50:10.105 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T05:50:10.248 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:50:10.250 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T05:50:10.263 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:50:10.263 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T05:50:10.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.295+0000 7f5bd11d5640 1 Processor -- start 2023-10-08T05:50:10.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.295+0000 7f5bd11d5640 1 -- start start 2023-10-08T05:50:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.296+0000 7f5bd11d5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc0c8e70 0x7f5bcc0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.296+0000 7f5bd11d5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bcc0cb5a0 0x7f5bcc0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.296+0000 7f5bd11d5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bcc06d490 con 0x7f5bcc0b3ec0 2023-10-08T05:50:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.296+0000 7f5bd11d5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bcc0b42a0 con 0x7f5bcc0cb5a0 2023-10-08T05:50:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.296+0000 7f5bd11d5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc06c880 con 0x7f5bcc0c8e70 2023-10-08T05:50:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bcad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5bcc0b3ec0 0x7f5bcc0cae90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41014/0 (socket says 172.21.15.70:41014) 2023-10-08T05:50:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bcad9d640 1 -- 172.21.15.70:0/776576052 learned_addr learned my addr 172.21.15.70:0/776576052 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143545788 0 0) 0x7f5bcc06d490 con 0x7f5bcc0b3ec0 2023-10-08T05:50:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bb4005730 con 0x7f5bcc0b3ec0 2023-10-08T05:50:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bcb59e640 1 --2- 172.21.15.70:0/776576052 >> v2:172.21.15.181:3300/0 conn(0x7f5bcc0cb5a0 0x7f5bcc0b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bca59c640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc0c8e70 0x7f5bcc0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3523292832 0 0) 0x7f5bb4005730 con 0x7f5bcc0b3ec0 2023-10-08T05:50:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc0c8e70 msgr2=0x7f5bcc0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc0c8e70 0x7f5bcc0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 >> v2:172.21.15.181:3300/0 conn(0x7f5bcc0cb5a0 msgr2=0x7f5bcc0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 --2- 172.21.15.70:0/776576052 >> v2:172.21.15.181:3300/0 conn(0x7f5bcc0cb5a0 0x7f5bcc0b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc06cb60 con 0x7f5bcc0b3ec0 2023-10-08T05:50:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5bbc001670 con 0x7f5bcc0b3ec0 2023-10-08T05:50:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 >> v1:172.21.15.70:6789/0 conn(0x7f5bcc0b3ec0 legacy=0x7f5bcc0cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.297+0000 7f5bc9d9b640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc0c8e70 0x7f5bb4005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bc9d9b640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb4006100 0x7f5bb40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bc9d9b640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bb4006f10 0x7f5bb4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc0b42a0 con 0x7f5bb4006f10 2023-10-08T05:50:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc06c880 con 0x7f5bb4006100 2023-10-08T05:50:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb400dbe0 con 0x7f5bcc0c8e70 2023-10-08T05:50:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bcb59e640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bb4006f10 0x7f5bb4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bcad9d640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc0c8e70 0x7f5bb4005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bca59c640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb4006100 0x7f5bb40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bcb59e640 1 -- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc0c8e70 msgr2=0x7f5bb4005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bcb59e640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc0c8e70 0x7f5bb4005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bcb59e640 1 -- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb4006100 msgr2=0x7f5bb40069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bcb59e640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb4006100 0x7f5bb40069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bcb59e640 1 -- 172.21.15.70:0/776576052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb4005730 con 0x7f5bb4006f10 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.298+0000 7f5bca59c640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bb4006100 0x7f5bb40069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.299+0000 7f5bcb59e640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bb4006f10 0x7f5bb4007390 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009fc0 tx=0x7f5bb4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592a4a59ea46ee13 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.300+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc402cc60 con 0x7f5bb4006f10 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.300+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc400f040 con 0x7f5bb4006f10 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.300+0000 7f5bc9d9b640 1 -- 172.21.15.70:0/776576052 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4034440 con 0x7f5bb4006f10 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.300+0000 7f5bd11d5640 1 -- 172.21.15.70:0/776576052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bb4006f10 msgr2=0x7f5bb4007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.300+0000 7f5bd11d5640 1 --2- 172.21.15.70:0/776576052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bb4006f10 0x7f5bb4007390 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009fc0 tx=0x7f5bb4007a00 comp rx=0 tx=0).stop 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.300+0000 7f5bcb59e640 1 -- 172.21.15.70:0/776576052 reap_dead start 2023-10-08T05:50:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.300+0000 7f5bd11d5640 1 -- 172.21.15.70:0/776576052 shutdown_connections 2023-10-08T05:50:10.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.300+0000 7f5bd11d5640 1 -- 172.21.15.70:0/776576052 >> 172.21.15.70:0/776576052 conn(0x7f5bcc06a5d0 msgr2=0x7f5bcc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:10.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.300+0000 7f5bd11d5640 1 -- 172.21.15.70:0/776576052 shutdown_connections 2023-10-08T05:50:10.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.300+0000 7f5bd11d5640 1 -- 172.21.15.70:0/776576052 wait complete. 2023-10-08T05:50:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.302+0000 7f5bd11d5640 1 Processor -- start 2023-10-08T05:50:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.302+0000 7f5bd11d5640 1 -- start start 2023-10-08T05:50:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.303+0000 7f5bd11d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bcc1f21c0 0x7f5bcc1e46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.303+0000 7f5bd11d5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bcc1f25a0 0x7f5bcc0c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.303+0000 7f5bd11d5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc15a810 0x7f5bcc0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.303+0000 7f5bd11d5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc1501c0 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.303+0000 7f5bd11d5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc1e4fc0 con 0x7f5bcc1f25a0 2023-10-08T05:50:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.303+0000 7f5bd11d5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bcc1e52a0 con 0x7f5bcc15a810 2023-10-08T05:50:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bcad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bcc1f21c0 0x7f5bcc1e46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bcad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bcc1f21c0 0x7f5bcc1e46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39644/0 (socket says 172.21.15.70:39644) 2023-10-08T05:50:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bcad9d640 1 -- 172.21.15.70:0/571633008 learned_addr learned my addr 172.21.15.70:0/571633008 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bcb59e640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc15a810 0x7f5bcc0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:10.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bca59c640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bcc1f25a0 0x7f5bcc0c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:10.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bcad9d640 1 -- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc15a810 msgr2=0x7f5bcc0c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:10.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bcad9d640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc15a810 0x7f5bcc0c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bcad9d640 1 -- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bcc1f25a0 msgr2=0x7f5bcc0c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:10.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bcad9d640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bcc1f25a0 0x7f5bcc0c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:10.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bcad9d640 1 -- 172.21.15.70:0/571633008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc1e5580 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.304+0000 7f5bcad9d640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bcc1f21c0 0x7f5bcc1e46c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc007310 tx=0x7f5bbc0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:10.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.305+0000 7f5bb37fe640 1 -- 172.21.15.70:0/571633008 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00f9f0 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.305+0000 7f5bb37fe640 1 -- 172.21.15.70:0/571633008 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bbc00fb70 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.305+0000 7f5bb37fe640 1 -- 172.21.15.70:0/571633008 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc0133f0 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.305+0000 7f5bd11d5640 1 -- 172.21.15.70:0/571633008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bcc1e5860 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.305+0000 7f5bd11d5640 1 -- 172.21.15.70:0/571633008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bcc1e7100 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.306+0000 7f5bd11d5640 1 -- 172.21.15.70:0/571633008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bcc1e5a20 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.307+0000 7f5bb37fe640 1 -- 172.21.15.70:0/571633008 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5bbc013550 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.308+0000 7f5bb37fe640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f5bb806afc0 0x7f5bb806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.308+0000 7f5bb37fe640 1 -- 172.21.15.70:0/571633008 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5952+0+0 (secure 0 0 0) 0x7f5bbc089a00 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.308+0000 7f5bca59c640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f5bb806afc0 0x7f5bb806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.309+0000 7f5bca59c640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f5bb806afc0 0x7f5bb806d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0ce530 tx=0x7f5bc0008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:10.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.312+0000 7f5bb37fe640 1 -- 172.21.15.70:0/571633008 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5bbc054970 con 0x7f5bcc1f21c0 2023-10-08T05:50:10.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:10.508+0000 7f5bd11d5640 1 -- 172.21.15.70:0/571633008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5bcc04e160 con 0x7f5bcc1f21c0 2023-10-08T05:50:11.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.638+0000 7f5bb37fe640 1 -- 172.21.15.70:0/571633008 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5bbc057f90 con 0x7f5bcc1f21c0 2023-10-08T05:50:11.640 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T05:50:11.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.642+0000 7f5bd11d5640 1 -- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f5bb806afc0 msgr2=0x7f5bb806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:11.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.642+0000 7f5bd11d5640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f5bb806afc0 0x7f5bb806d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0ce530 tx=0x7f5bc0008040 comp rx=0 tx=0).stop 2023-10-08T05:50:11.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.642+0000 7f5bd11d5640 1 -- 172.21.15.70:0/571633008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bcc1f21c0 msgr2=0x7f5bcc1e46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:11.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.642+0000 7f5bd11d5640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bcc1f21c0 0x7f5bcc1e46c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc007310 tx=0x7f5bbc0077d0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.642+0000 7f5bd11d5640 1 -- 172.21.15.70:0/571633008 shutdown_connections 2023-10-08T05:50:11.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.642+0000 7f5bd11d5640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f5bb806afc0 0x7f5bb806d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.642+0000 7f5bd11d5640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bcc15a810 0x7f5bcc0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.642+0000 7f5bd11d5640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bcc1f25a0 0x7f5bcc0c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.642+0000 7f5bd11d5640 1 --2- 172.21.15.70:0/571633008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bcc1f21c0 0x7f5bcc1e46c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.643+0000 7f5bd11d5640 1 -- 172.21.15.70:0/571633008 >> 172.21.15.70:0/571633008 conn(0x7f5bcc06a5d0 msgr2=0x7f5bcc152de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:11.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.643+0000 7f5bd11d5640 1 -- 172.21.15.70:0/571633008 shutdown_connections 2023-10-08T05:50:11.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.643+0000 7f5bd11d5640 1 -- 172.21.15.70:0/571633008 wait complete. 2023-10-08T05:50:11.663 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.861+0000 7f39bb1e7640 1 Processor -- start 2023-10-08T05:50:11.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.862+0000 7f39bb1e7640 1 -- start start 2023-10-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.863+0000 7f39bb1e7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39b40c90b0 0x7f39b40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.863+0000 7f39bb1e7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f39b40cb3b0 0x7f39b415e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.863+0000 7f39bb1e7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b415b2b0 con 0x7f39b415ab40 2023-10-08T05:50:11.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.863+0000 7f39bb1e7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b415bf30 con 0x7f39b40c90b0 2023-10-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.863+0000 7f39bb1e7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b40cb820 con 0x7f39b40cb3b0 2023-10-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.864+0000 7f39b3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39b40c90b0 0x7f39b40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.864+0000 7f39b3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39b40c90b0 0x7f39b40cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39650/0 (socket says 172.21.15.70:39650) 2023-10-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.864+0000 7f39b3fff640 1 -- 172.21.15.70:0/1177440433 learned_addr learned my addr 172.21.15.70:0/1177440433 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.864+0000 7f39b9784640 1 --2- 172.21.15.70:0/1177440433 >> v2:172.21.15.181:3300/0 conn(0x7f39b40cb3b0 0x7f39b415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.864+0000 7f39b3fff640 1 -- 172.21.15.70:0/1177440433 >> v2:172.21.15.181:3300/0 conn(0x7f39b40cb3b0 msgr2=0x7f39b415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.864+0000 7f39b3fff640 1 --2- 172.21.15.70:0/1177440433 >> v2:172.21.15.181:3300/0 conn(0x7f39b40cb3b0 0x7f39b415e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.864+0000 7f39b3fff640 1 -- 172.21.15.70:0/1177440433 >> v1:172.21.15.181:6789/0 conn(0x7f39b415ab40 legacy=0x7f39b40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.864+0000 7f39b3fff640 1 -- 172.21.15.70:0/1177440433 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b415b590 con 0x7f39b40c90b0 2023-10-08T05:50:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.865+0000 7f39b3fff640 1 --2- 172.21.15.70:0/1177440433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 0x7f39b40cae70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f39b40b0ec0 tx=0x7f39a002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d1099832593958 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.865+0000 7f39b37fe640 1 -- 172.21.15.70:0/1177440433 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0030860 con 0x7f39b40c90b0 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.866+0000 7f39b37fe640 1 -- 172.21.15.70:0/1177440433 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a00309c0 con 0x7f39b40c90b0 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.866+0000 7f39b37fe640 1 -- 172.21.15.70:0/1177440433 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0030d50 con 0x7f39b40c90b0 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.866+0000 7f39bb1e7640 1 -- 172.21.15.70:0/1177440433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 msgr2=0x7f39b40cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.866+0000 7f39bb1e7640 1 --2- 172.21.15.70:0/1177440433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 0x7f39b40cae70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f39b40b0ec0 tx=0x7f39a002b9a0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.866+0000 7f39bb1e7640 1 -- 172.21.15.70:0/1177440433 shutdown_connections 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.866+0000 7f39bb1e7640 1 --2- 172.21.15.70:0/1177440433 >> v2:172.21.15.181:3300/0 conn(0x7f39b40cb3b0 0x7f39b415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.866+0000 7f39bb1e7640 1 --2- 172.21.15.70:0/1177440433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 0x7f39b40cae70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.866+0000 7f39bb1e7640 1 -- 172.21.15.70:0/1177440433 >> 172.21.15.70:0/1177440433 conn(0x7f39b406a380 msgr2=0x7f39b40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.866+0000 7f39bb1e7640 1 -- 172.21.15.70:0/1177440433 shutdown_connections 2023-10-08T05:50:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.866+0000 7f39bb1e7640 1 -- 172.21.15.70:0/1177440433 wait complete. 2023-10-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.868+0000 7f39bb1e7640 1 Processor -- start 2023-10-08T05:50:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.868+0000 7f39bb1e7640 1 -- start start 2023-10-08T05:50:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.869+0000 7f39bb1e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 0x7f39b41f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.869+0000 7f39bb1e7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39b40cb3b0 0x7f39b4154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.869+0000 7f39bb1e7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39b415ab40 0x7f39b40ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.869+0000 7f39bb1e7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40b2c60 con 0x7f39b40c90b0 2023-10-08T05:50:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.869+0000 7f39bb1e7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4154840 con 0x7f39b40cb3b0 2023-10-08T05:50:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.869+0000 7f39bb1e7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b40b2980 con 0x7f39b415ab40 2023-10-08T05:50:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.869+0000 7f39b8f83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 0x7f39b41f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.869+0000 7f39b8f83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 0x7f39b41f35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39664/0 (socket says 172.21.15.70:39664) 2023-10-08T05:50:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.869+0000 7f39b8f83640 1 -- 172.21.15.70:0/4266661562 learned_addr learned my addr 172.21.15.70:0/4266661562 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:11.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39b8f83640 1 -- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39b415ab40 msgr2=0x7f39b40ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:11.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39b8f83640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39b415ab40 0x7f39b40ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39b8f83640 1 -- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39b40cb3b0 msgr2=0x7f39b4154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:11.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39b8f83640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39b40cb3b0 0x7f39b4154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:11.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39b8f83640 1 -- 172.21.15.70:0/4266661562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00094d0 con 0x7f39b40c90b0 2023-10-08T05:50:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39b8f83640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 0x7f39b41f35d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f39a800c400 tx=0x7f39a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39b17fa640 1 -- 172.21.15.70:0/4266661562 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a80052d0 con 0x7f39b40c90b0 2023-10-08T05:50:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39b17fa640 1 -- 172.21.15.70:0/4266661562 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a8005430 con 0x7f39b40c90b0 2023-10-08T05:50:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39bb1e7640 1 -- 172.21.15.70:0/4266661562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b40ac960 con 0x7f39b40c90b0 2023-10-08T05:50:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39b17fa640 1 -- 172.21.15.70:0/4266661562 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8003830 con 0x7f39b40c90b0 2023-10-08T05:50:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.870+0000 7f39bb1e7640 1 -- 172.21.15.70:0/4266661562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b40acf20 con 0x7f39b40c90b0 2023-10-08T05:50:11.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.871+0000 7f39bb1e7640 1 -- 172.21.15.70:0/4266661562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b4067490 con 0x7f39b40c90b0 2023-10-08T05:50:11.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.872+0000 7f39b17fa640 1 -- 172.21.15.70:0/4266661562 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f39a8003990 con 0x7f39b40c90b0 2023-10-08T05:50:11.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.873+0000 7f39b17fa640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f399806af90 0x7f399806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:11.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.873+0000 7f39b17fa640 1 -- 172.21.15.70:0/4266661562 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f39a8088b40 con 0x7f39b40c90b0 2023-10-08T05:50:11.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.874+0000 7f39b3fff640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f399806af90 0x7f399806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:11.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.875+0000 7f39b3fff640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f399806af90 0x7f399806d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f39a0036000 tx=0x7f39a003b040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:11.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:11.877+0000 7f39b17fa640 1 -- 172.21.15.70:0/4266661562 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39a8053cb0 con 0x7f39b40c90b0 2023-10-08T05:50:12.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.137+0000 7f39bb1e7640 1 -- 172.21.15.70:0/4266661562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f39b40b0ef0 con 0x7f39b40c90b0 2023-10-08T05:50:12.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.138+0000 7f39b17fa640 1 -- 172.21.15.70:0/4266661562 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f39a8017090 con 0x7f39b40c90b0 2023-10-08T05:50:12.140 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:12.140 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:50:12.140 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T05:50:12.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.140 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T05:50:12.140 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T05:50:12.140 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.140 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:50:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:50:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:50:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T05:50:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T05:50:12.141 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:50:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:50:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:50:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T05:50:12.142 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T05:50:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:50:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:50:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:50:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.143 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T05:50:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T05:50:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:50:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:50:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:50:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T05:50:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T05:50:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:50:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:50:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.145 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T05:50:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T05:50:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:50:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:50:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.146 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T05:50:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T05:50:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:50:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:50:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T05:50:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T05:50:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:50:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:50:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T05:50:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T05:50:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:50:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:50:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T05:50:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T05:50:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:50:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:50:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T05:50:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T05:50:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:50:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:50:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.151 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T05:50:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T05:50:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:50:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:50:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T05:50:12.152 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T05:50:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:50:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T05:50:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T05:50:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T05:50:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T05:50:12.153 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:50:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:50:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:50:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:50:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T05:50:12.154 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T05:50:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:50:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:50:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:50:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:50:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.155 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T05:50:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T05:50:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:50:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:50:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:50:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:50:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.156 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T05:50:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T05:50:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:50:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:50:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:50:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T05:50:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T05:50:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:50:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:50:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:50:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:12.158 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T05:50:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T05:50:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:50:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:50:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:50:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:50:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:12.159 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T05:50:12.159 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:50:12.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 -- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f399806af90 msgr2=0x7f399806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f399806af90 0x7f399806d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f39a0036000 tx=0x7f39a003b040 comp rx=0 tx=0).stop 2023-10-08T05:50:12.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 -- 172.21.15.70:0/4266661562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 msgr2=0x7f39b41f35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 0x7f39b41f35d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f39a800c400 tx=0x7f39a800c8c0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 -- 172.21.15.70:0/4266661562 shutdown_connections 2023-10-08T05:50:12.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f399806af90 0x7f399806d450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39b415ab40 0x7f39b40ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39b40cb3b0 0x7f39b4154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 --2- 172.21.15.70:0/4266661562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b40c90b0 0x7f39b41f35d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 -- 172.21.15.70:0/4266661562 >> 172.21.15.70:0/4266661562 conn(0x7f39b406a380 msgr2=0x7f39b40ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:12.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 -- 172.21.15.70:0/4266661562 shutdown_connections 2023-10-08T05:50:12.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.141+0000 7f39bb1e7640 1 -- 172.21.15.70:0/4266661562 wait complete. 2023-10-08T05:50:12.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:50:12.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.374+0000 7f850ed05640 1 Processor -- start 2023-10-08T05:50:12.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.374+0000 7f850ed05640 1 -- start start 2023-10-08T05:50:12.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.376+0000 7f850ed05640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f850815ab40 0x7f85080ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.376+0000 7f850ed05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85080c90b0 0x7f85080adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.376+0000 7f850ed05640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850815b2b0 con 0x7f85080cae70 2023-10-08T05:50:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.376+0000 7f850ed05640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850815bf30 con 0x7f850815ab40 2023-10-08T05:50:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.376+0000 7f850ed05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850815f7a0 con 0x7f85080c90b0 2023-10-08T05:50:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.377+0000 7f850caa1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f850815ab40 0x7f85080ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.377+0000 7f8507fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85080c90b0 0x7f85080adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.377+0000 7f850caa1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f850815ab40 0x7f85080ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:45064/0 (socket says 172.21.15.70:45064) 2023-10-08T05:50:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.377+0000 7f850caa1640 1 -- 172.21.15.70:0/2774749325 learned_addr learned my addr 172.21.15.70:0/2774749325 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.378+0000 7f850caa1640 1 -- 172.21.15.70:0/2774749325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85080c90b0 msgr2=0x7f85080adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.378+0000 7f850caa1640 1 --2- 172.21.15.70:0/2774749325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85080c90b0 0x7f85080adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.378+0000 7f850caa1640 1 -- 172.21.15.70:0/2774749325 >> v1:172.21.15.181:6789/0 conn(0x7f85080cae70 legacy=0x7f85080702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.378+0000 7f850caa1640 1 -- 172.21.15.70:0/2774749325 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850815b590 con 0x7f850815ab40 2023-10-08T05:50:12.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.380+0000 7f850caa1640 1 --2- 172.21.15.70:0/2774749325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f850815ab40 0x7f85080ad720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009870 tx=0x7f84fc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87de57460d24f52 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:12.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.380+0000 7f85077fe640 1 -- 172.21.15.70:0/2774749325 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc030840 con 0x7f850815ab40 2023-10-08T05:50:12.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.380+0000 7f85077fe640 1 -- 172.21.15.70:0/2774749325 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc0309a0 con 0x7f850815ab40 2023-10-08T05:50:12.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.380+0000 7f85077fe640 1 -- 172.21.15.70:0/2774749325 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc030d30 con 0x7f850815ab40 2023-10-08T05:50:12.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.381+0000 7f850ed05640 1 -- 172.21.15.70:0/2774749325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f850815ab40 msgr2=0x7f85080ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.381+0000 7f850ed05640 1 --2- 172.21.15.70:0/2774749325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f850815ab40 0x7f85080ad720 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009870 tx=0x7f84fc02b980 comp rx=0 tx=0).stop 2023-10-08T05:50:12.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.381+0000 7f850ed05640 1 -- 172.21.15.70:0/2774749325 shutdown_connections 2023-10-08T05:50:12.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.381+0000 7f850ed05640 1 --2- 172.21.15.70:0/2774749325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85080c90b0 0x7f85080adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.381+0000 7f850ed05640 1 --2- 172.21.15.70:0/2774749325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f850815ab40 0x7f85080ad720 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.381+0000 7f850ed05640 1 -- 172.21.15.70:0/2774749325 >> 172.21.15.70:0/2774749325 conn(0x7f850806a380 msgr2=0x7f85080b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:12.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.381+0000 7f850ed05640 1 -- 172.21.15.70:0/2774749325 shutdown_connections 2023-10-08T05:50:12.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.381+0000 7f850ed05640 1 -- 172.21.15.70:0/2774749325 wait complete. 2023-10-08T05:50:12.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.382+0000 7f850ed05640 1 Processor -- start 2023-10-08T05:50:12.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.383+0000 7f850ed05640 1 -- start start 2023-10-08T05:50:12.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850ed05640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85080c90b0 0x7f85081566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850ed05640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85080cae70 0x7f850806cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850ed05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f850815ab40 0x7f850806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850ed05640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85081f3d90 con 0x7f85080c90b0 2023-10-08T05:50:12.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850ed05640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85081f4070 con 0x7f85080cae70 2023-10-08T05:50:12.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850ed05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85081f4350 con 0x7f850815ab40 2023-10-08T05:50:12.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850caa1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85080c90b0 0x7f85081566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850caa1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85080c90b0 0x7f85081566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39680/0 (socket says 172.21.15.70:39680) 2023-10-08T05:50:12.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850caa1640 1 -- 172.21.15.70:0/3135884476 learned_addr learned my addr 172.21.15.70:0/3135884476 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:12.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f8507fff640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85080cae70 0x7f850806cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850d2a2640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f850815ab40 0x7f850806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850caa1640 1 -- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f850815ab40 msgr2=0x7f850806d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850caa1640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f850815ab40 0x7f850806d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850caa1640 1 -- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85080cae70 msgr2=0x7f850806cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850caa1640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85080cae70 0x7f850806cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f850caa1640 1 -- 172.21.15.70:0/3135884476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc0094d0 con 0x7f85080c90b0 2023-10-08T05:50:12.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.384+0000 7f8507fff640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85080cae70 0x7f850806cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:12.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.385+0000 7f850caa1640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85080c90b0 0x7f85081566b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f84fc000c00 tx=0x7f84fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:12.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.385+0000 7f85057fa640 1 -- 172.21.15.70:0/3135884476 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc039c90 con 0x7f85080c90b0 2023-10-08T05:50:12.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.385+0000 7f85057fa640 1 -- 172.21.15.70:0/3135884476 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc039df0 con 0x7f85080c90b0 2023-10-08T05:50:12.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.385+0000 7f85057fa640 1 -- 172.21.15.70:0/3135884476 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc043640 con 0x7f85080c90b0 2023-10-08T05:50:12.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.385+0000 7f850ed05640 1 -- 172.21.15.70:0/3135884476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85081f4630 con 0x7f85080c90b0 2023-10-08T05:50:12.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.385+0000 7f850ed05640 1 -- 172.21.15.70:0/3135884476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85080b1fb0 con 0x7f85080c90b0 2023-10-08T05:50:12.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.387+0000 7f850ed05640 1 -- 172.21.15.70:0/3135884476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8508067490 con 0x7f85080c90b0 2023-10-08T05:50:12.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.388+0000 7f85057fa640 1 -- 172.21.15.70:0/3135884476 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f84fc0437a0 con 0x7f85080c90b0 2023-10-08T05:50:12.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.388+0000 7f85057fa640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f84ec06afc0 0x7f84ec06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.389+0000 7f85057fa640 1 -- 172.21.15.70:0/3135884476 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f84fc03e070 con 0x7f85080c90b0 2023-10-08T05:50:12.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.389+0000 7f8507fff640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f84ec06afc0 0x7f84ec06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.390+0000 7f8507fff640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f84ec06afc0 0x7f84ec06d480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f85080c6da0 tx=0x7f84f4008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:12.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.393+0000 7f85057fa640 1 -- 172.21.15.70:0/3135884476 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84fc077750 con 0x7f85080c90b0 2023-10-08T05:50:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.584+0000 7f850ed05640 1 -- 172.21.15.70:0/3135884476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85080b0ef0 con 0x7f85080c90b0 2023-10-08T05:50:12.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.586+0000 7f85057fa640 1 -- 172.21.15.70:0/3135884476 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+13120 (secure 0 0 0) 0x7f84fc07ad70 con 0x7f85080c90b0 2023-10-08T05:50:12.588 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:12.588 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":68,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:50:11.636333+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:50:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 -- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f84ec06afc0 msgr2=0x7f84ec06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f84ec06afc0 0x7f84ec06d480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f85080c6da0 tx=0x7f84f4008040 comp rx=0 tx=0).stop 2023-10-08T05:50:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 -- 172.21.15.70:0/3135884476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85080c90b0 msgr2=0x7f85081566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85080c90b0 0x7f85081566b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f84fc000c00 tx=0x7f84fc004730 comp rx=0 tx=0).stop 2023-10-08T05:50:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 -- 172.21.15.70:0/3135884476 shutdown_connections 2023-10-08T05:50:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f84ec06afc0 0x7f84ec06d480 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f850815ab40 0x7f850806d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85080cae70 0x7f850806cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 --2- 172.21.15.70:0/3135884476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85080c90b0 0x7f85081566b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 -- 172.21.15.70:0/3135884476 >> 172.21.15.70:0/3135884476 conn(0x7f850806a380 msgr2=0x7f85080cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 -- 172.21.15.70:0/3135884476 shutdown_connections 2023-10-08T05:50:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.589+0000 7f850ed05640 1 -- 172.21.15.70:0/3135884476 wait complete. 2023-10-08T05:50:12.609 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:50:12.609 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:50:12.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.823+0000 7f40f896e640 1 Processor -- start 2023-10-08T05:50:12.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.823+0000 7f40f896e640 1 -- start start 2023-10-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.824+0000 7f40f896e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40f40cb880 0x7f40f40b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.824+0000 7f40f896e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f415d440 con 0x7f40f40b4070 2023-10-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.824+0000 7f40f896e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f415d690 con 0x7f40f41432f0 2023-10-08T05:50:12.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.824+0000 7f40f896e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f41436d0 con 0x7f40f40cb880 2023-10-08T05:50:12.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.825+0000 7f40f2d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f40f40b4070 0x7f40f4159e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41018/0 (socket says 172.21.15.70:41018) 2023-10-08T05:50:12.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.825+0000 7f40e9d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40f40cb880 0x7f40f40b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.825+0000 7f40f2d9d640 1 -- 172.21.15.70:0/871495564 learned_addr learned my addr 172.21.15.70:0/871495564 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:12.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.826+0000 7f40e9d9b640 1 -- 172.21.15.70:0/871495564 >> v1:172.21.15.181:6789/0 conn(0x7f40f41432f0 legacy=0x7f40f40cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.826+0000 7f40e9d9b640 1 -- 172.21.15.70:0/871495564 >> v1:172.21.15.70:6789/0 conn(0x7f40f40b4070 legacy=0x7f40f4159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.826+0000 7f40e9d9b640 1 -- 172.21.15.70:0/871495564 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f415d970 con 0x7f40f40cb880 2023-10-08T05:50:12.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.826+0000 7f40f1d9b640 1 -- 172.21.15.70:0/871495564 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935020971 0 0) 0x7f40f415d440 con 0x7f40f40b4070 2023-10-08T05:50:12.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.827+0000 7f40e9d9b640 1 --2- 172.21.15.70:0/871495564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40cb880 0x7f40f40b3b30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0097a0 tx=0x7f40dc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7758a780dd92e3e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:12.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.828+0000 7f40f1d9b640 1 -- 172.21.15.70:0/871495564 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc031830 con 0x7f40f40cb880 2023-10-08T05:50:12.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.828+0000 7f40f1d9b640 1 -- 172.21.15.70:0/871495564 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc031990 con 0x7f40f40cb880 2023-10-08T05:50:12.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.828+0000 7f40f1d9b640 1 -- 172.21.15.70:0/871495564 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc031d20 con 0x7f40f40cb880 2023-10-08T05:50:12.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.828+0000 7f40f896e640 1 -- 172.21.15.70:0/871495564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40cb880 msgr2=0x7f40f40b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.828+0000 7f40f896e640 1 --2- 172.21.15.70:0/871495564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40cb880 0x7f40f40b3b30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0097a0 tx=0x7f40dc02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.828+0000 7f40f896e640 1 -- 172.21.15.70:0/871495564 shutdown_connections 2023-10-08T05:50:12.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.828+0000 7f40f896e640 1 --2- 172.21.15.70:0/871495564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40cb880 0x7f40f40b3b30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.828+0000 7f40f896e640 1 -- 172.21.15.70:0/871495564 >> 172.21.15.70:0/871495564 conn(0x7f40f4069d50 msgr2=0x7f40f406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.829+0000 7f40f896e640 1 -- 172.21.15.70:0/871495564 shutdown_connections 2023-10-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.829+0000 7f40f896e640 1 -- 172.21.15.70:0/871495564 wait complete. 2023-10-08T05:50:12.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.830+0000 7f40f896e640 1 Processor -- start 2023-10-08T05:50:12.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.831+0000 7f40f896e640 1 -- start start 2023-10-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.831+0000 7f40f896e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40b4070 0x7f40f41f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.832+0000 7f40f896e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40f40cb880 0x7f40f41f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.832+0000 7f40f896e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40f41432f0 0x7f40f4132da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.832+0000 7f40f896e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f41f2090 con 0x7f40f41432f0 2023-10-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.832+0000 7f40f896e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f4133950 con 0x7f40f40b4070 2023-10-08T05:50:12.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.832+0000 7f40f896e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f4133c30 con 0x7f40f40cb880 2023-10-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.832+0000 7f40f259c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40b4070 0x7f40f41f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.832+0000 7f40f259c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40b4070 0x7f40f41f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:45092/0 (socket says 172.21.15.70:45092) 2023-10-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.832+0000 7f40f2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40f41432f0 0x7f40f4132da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.832+0000 7f40f259c640 1 -- 172.21.15.70:0/3971315075 learned_addr learned my addr 172.21.15.70:0/3971315075 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.832+0000 7f40e9d9b640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40f40cb880 0x7f40f41f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.833+0000 7f40f259c640 1 -- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40f40cb880 msgr2=0x7f40f41f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.833+0000 7f40f259c640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40f40cb880 0x7f40f41f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.833+0000 7f40f259c640 1 -- 172.21.15.70:0/3971315075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40f41432f0 msgr2=0x7f40f4132da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.833+0000 7f40f259c640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40f41432f0 0x7f40f4132da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.833+0000 7f40f259c640 1 -- 172.21.15.70:0/3971315075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc009400 con 0x7f40f40b4070 2023-10-08T05:50:12.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.833+0000 7f40f259c640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40b4070 0x7f40f41f6240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f40e400c730 tx=0x7f40e400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:12.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.834+0000 7f40eb7fe640 1 -- 172.21.15.70:0/3971315075 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4003b70 con 0x7f40f40b4070 2023-10-08T05:50:12.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.834+0000 7f40eb7fe640 1 -- 172.21.15.70:0/3971315075 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e4003cf0 con 0x7f40f40b4070 2023-10-08T05:50:12.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.834+0000 7f40eb7fe640 1 -- 172.21.15.70:0/3971315075 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4003ea0 con 0x7f40f40b4070 2023-10-08T05:50:12.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.834+0000 7f40f896e640 1 -- 172.21.15.70:0/3971315075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f41f5760 con 0x7f40f40b4070 2023-10-08T05:50:12.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.834+0000 7f40f896e640 1 -- 172.21.15.70:0/3971315075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f41f5d20 con 0x7f40f40b4070 2023-10-08T05:50:12.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.835+0000 7f40f896e640 1 -- 172.21.15.70:0/3971315075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f404e160 con 0x7f40f40b4070 2023-10-08T05:50:12.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.837+0000 7f40eb7fe640 1 -- 172.21.15.70:0/3971315075 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f40e400aa90 con 0x7f40f40b4070 2023-10-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.838+0000 7f40eb7fe640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f40d406af90 0x7f40d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.838+0000 7f40eb7fe640 1 -- 172.21.15.70:0/3971315075 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f40e4089ca0 con 0x7f40f40b4070 2023-10-08T05:50:12.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.839+0000 7f40e9d9b640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f40d406af90 0x7f40d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:12.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.840+0000 7f40e9d9b640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f40d406af90 0x7f40d406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f40dc036000 tx=0x7f40dc03c040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:12.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:12.853+0000 7f40eb7fe640 1 -- 172.21.15.70:0/3971315075 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40e4054fb0 con 0x7f40f40b4070 2023-10-08T05:50:13.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.047+0000 7f40f896e640 1 -- 172.21.15.70:0/3971315075 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40f406d660 con 0x7f40f40b4070 2023-10-08T05:50:13.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.049+0000 7f40eb7fe640 1 -- 172.21.15.70:0/3971315075 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40e40585d0 con 0x7f40f40b4070 2023-10-08T05:50:13.050 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:50:13.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.051+0000 7f40f896e640 1 -- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f40d406af90 msgr2=0x7f40d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.051+0000 7f40f896e640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f40d406af90 0x7f40d406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f40dc036000 tx=0x7f40dc03c040 comp rx=0 tx=0).stop 2023-10-08T05:50:13.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.051+0000 7f40f896e640 1 -- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40b4070 msgr2=0x7f40f41f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.051+0000 7f40f896e640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40b4070 0x7f40f41f6240 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f40e400c730 tx=0x7f40e400cbf0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.051+0000 7f40f896e640 1 -- 172.21.15.70:0/3971315075 shutdown_connections 2023-10-08T05:50:13.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.051+0000 7f40f896e640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40f41432f0 0x7f40f4132da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.051+0000 7f40f896e640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f40d406af90 0x7f40d406d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.051+0000 7f40f896e640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40f40cb880 0x7f40f41f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.051+0000 7f40f896e640 1 --2- 172.21.15.70:0/3971315075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40f40b4070 0x7f40f41f6240 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.051+0000 7f40f896e640 1 -- 172.21.15.70:0/3971315075 >> 172.21.15.70:0/3971315075 conn(0x7f40f4069d50 msgr2=0x7f40f40c8790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:13.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.052+0000 7f40f896e640 1 -- 172.21.15.70:0/3971315075 shutdown_connections 2023-10-08T05:50:13.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.052+0000 7f40f896e640 1 -- 172.21.15.70:0/3971315075 wait complete. 2023-10-08T05:50:13.072 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T05:50:13.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.285+0000 7f68be2e5640 1 Processor -- start 2023-10-08T05:50:13.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.285+0000 7f68be2e5640 1 -- start start 2023-10-08T05:50:13.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.287+0000 7f68be2e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b80709e0 0x7f68b8164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.287+0000 7f68be2e5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b806dee0 con 0x7f68b8159ac0 2023-10-08T05:50:13.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.287+0000 7f68be2e5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b815d850 con 0x7f68b80ad790 2023-10-08T05:50:13.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.287+0000 7f68be2e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b806dcf0 con 0x7f68b80709e0 2023-10-08T05:50:13.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.287+0000 7f68b7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f68b8159ac0 0x7f68b806d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41030/0 (socket says 172.21.15.70:41030) 2023-10-08T05:50:13.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.287+0000 7f68b7fff640 1 -- 172.21.15.70:0/3502793002 learned_addr learned my addr 172.21.15.70:0/3502793002 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68bc882640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b80709e0 0x7f68b8164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267123276 0 0) 0x7f68b806dee0 con 0x7f68b8159ac0 2023-10-08T05:50:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68a4003520 con 0x7f68b8159ac0 2023-10-08T05:50:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1217357954 0 0) 0x7f68a4003520 con 0x7f68b8159ac0 2023-10-08T05:50:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b80709e0 msgr2=0x7f68b8164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b80709e0 0x7f68b8164a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 >> v1:172.21.15.181:6789/0 conn(0x7f68b80ad790 legacy=0x7f68b80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b815db30 con 0x7f68b8159ac0 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f68a8001670 con 0x7f68b8159ac0 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 >> v1:172.21.15.70:6789/0 conn(0x7f68b8159ac0 legacy=0x7f68b806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68a40039a0 0x7f68a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b80709e0 0x7f68a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a4004d00 0x7f68a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a400dc70 con 0x7f68a4004d00 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a400dde0 con 0x7f68a40039a0 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.288+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a400dfa0 con 0x7f68b80709e0 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.289+0000 7f68bc882640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a4004d00 0x7f68a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.289+0000 7f68b7fff640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68a40039a0 0x7f68a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.289+0000 7f68affff640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b80709e0 0x7f68a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.289+0000 7f68bc882640 1 -- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b80709e0 msgr2=0x7f68a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:13.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.289+0000 7f68bc882640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68b80709e0 0x7f68a40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.289+0000 7f68bc882640 1 -- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68a40039a0 msgr2=0x7f68a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:13.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.289+0000 7f68bc882640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68a40039a0 0x7f68a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.289+0000 7f68bc882640 1 -- 172.21.15.70:0/3502793002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4003520 con 0x7f68a4004d00 2023-10-08T05:50:13.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.290+0000 7f68bc882640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a4004d00 0x7f68a4005180 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f68b80ae2f0 tx=0x7f68a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575e9cb6ab8c521e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:13.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.290+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b002cb40 con 0x7f68a4004d00 2023-10-08T05:50:13.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.291+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b000f040 con 0x7f68a4004d00 2023-10-08T05:50:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.291+0000 7f68b77fe640 1 -- 172.21.15.70:0/3502793002 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b002cea0 con 0x7f68a4004d00 2023-10-08T05:50:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.291+0000 7f68be2e5640 1 -- 172.21.15.70:0/3502793002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a4004d00 msgr2=0x7f68a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.291+0000 7f68be2e5640 1 --2- 172.21.15.70:0/3502793002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68a4004d00 0x7f68a4005180 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f68b80ae2f0 tx=0x7f68a40057f0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.291+0000 7f68bc882640 1 -- 172.21.15.70:0/3502793002 reap_dead start 2023-10-08T05:50:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.291+0000 7f68be2e5640 1 -- 172.21.15.70:0/3502793002 shutdown_connections 2023-10-08T05:50:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.291+0000 7f68be2e5640 1 -- 172.21.15.70:0/3502793002 >> 172.21.15.70:0/3502793002 conn(0x7f68b806a380 msgr2=0x7f68b80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.291+0000 7f68be2e5640 1 -- 172.21.15.70:0/3502793002 shutdown_connections 2023-10-08T05:50:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.291+0000 7f68be2e5640 1 -- 172.21.15.70:0/3502793002 wait complete. 2023-10-08T05:50:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.293+0000 7f68be2e5640 1 Processor -- start 2023-10-08T05:50:13.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.293+0000 7f68be2e5640 1 -- start start 2023-10-08T05:50:13.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.294+0000 7f68be2e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68a4004d00 0x7f68b815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.294+0000 7f68be2e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68b814f1c0 0x7f68b814e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.294+0000 7f68be2e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b806c3c0 0x7f68b806c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.294+0000 7f68be2e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b806cd80 con 0x7f68b814f1c0 2023-10-08T05:50:13.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.294+0000 7f68be2e5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b814ef00 con 0x7f68b806c3c0 2023-10-08T05:50:13.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.294+0000 7f68be2e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b815ad70 con 0x7f68a4004d00 2023-10-08T05:50:13.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68affff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68b814f1c0 0x7f68b814e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68affff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68b814f1c0 0x7f68b814e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39712/0 (socket says 172.21.15.70:39712) 2023-10-08T05:50:13.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68affff640 1 -- 172.21.15.70:0/2845599955 learned_addr learned my addr 172.21.15.70:0/2845599955 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:13.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68b7fff640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68a4004d00 0x7f68b815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68bc882640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b806c3c0 0x7f68b806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68affff640 1 -- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68a4004d00 msgr2=0x7f68b815a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68affff640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68a4004d00 0x7f68b815a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68affff640 1 -- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b806c3c0 msgr2=0x7f68b806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68affff640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b806c3c0 0x7f68b806c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68affff640 1 -- 172.21.15.70:0/2845599955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b815b050 con 0x7f68b814f1c0 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68bc882640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b806c3c0 0x7f68b806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.295+0000 7f68affff640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68b814f1c0 0x7f68b814e9c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f68a000e490 tx=0x7f68a000e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.296+0000 7f68b57fa640 1 -- 172.21.15.70:0/2845599955 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a00052e0 con 0x7f68b814f1c0 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.296+0000 7f68b57fa640 1 -- 172.21.15.70:0/2845599955 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a0005440 con 0x7f68b814f1c0 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.296+0000 7f68b57fa640 1 -- 172.21.15.70:0/2845599955 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0003830 con 0x7f68b814f1c0 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.296+0000 7f68be2e5640 1 -- 172.21.15.70:0/2845599955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b8153090 con 0x7f68b814f1c0 2023-10-08T05:50:13.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.296+0000 7f68be2e5640 1 -- 172.21.15.70:0/2845599955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b8153650 con 0x7f68b814f1c0 2023-10-08T05:50:13.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.297+0000 7f68be2e5640 1 -- 172.21.15.70:0/2845599955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b8153810 con 0x7f68b814f1c0 2023-10-08T05:50:13.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.298+0000 7f68b57fa640 1 -- 172.21.15.70:0/2845599955 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f68a0003990 con 0x7f68b814f1c0 2023-10-08T05:50:13.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.299+0000 7f68b57fa640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f689806af90 0x7f689806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.299+0000 7f68b57fa640 1 -- 172.21.15.70:0/2845599955 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f68a0089420 con 0x7f68b814f1c0 2023-10-08T05:50:13.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.300+0000 7f68b7fff640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f689806af90 0x7f689806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.301+0000 7f68b7fff640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f689806af90 0x7f689806d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f68a8001eb0 tx=0x7f68a8000f50 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.304+0000 7f68b57fa640 1 -- 172.21.15.70:0/2845599955 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68a0054470 con 0x7f68b814f1c0 2023-10-08T05:50:13.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.499+0000 7f68be2e5640 1 -- 172.21.15.70:0/2845599955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f68b8067490 con 0x7f68b814f1c0 2023-10-08T05:50:13.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.499+0000 7f68b57fa640 1 -- 172.21.15.70:0/2845599955 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+65 (secure 0 0 0) 0x7f68a0057a90 con 0x7f68b814f1c0 2023-10-08T05:50:13.501 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:13.501 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:50:13.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:13.501 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T05:50:13.501 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T05:50:13.501 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:13.502 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:50:13.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.502+0000 7f68be2e5640 1 -- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f689806af90 msgr2=0x7f689806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.502+0000 7f68be2e5640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f689806af90 0x7f689806d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f68a8001eb0 tx=0x7f68a8000f50 comp rx=0 tx=0).stop 2023-10-08T05:50:13.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.502+0000 7f68be2e5640 1 -- 172.21.15.70:0/2845599955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68b814f1c0 msgr2=0x7f68b814e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.502+0000 7f68be2e5640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68b814f1c0 0x7f68b814e9c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f68a000e490 tx=0x7f68a000e950 comp rx=0 tx=0).stop 2023-10-08T05:50:13.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.503+0000 7f68be2e5640 1 -- 172.21.15.70:0/2845599955 shutdown_connections 2023-10-08T05:50:13.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.503+0000 7f68be2e5640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f68b806c3c0 0x7f68b806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.503+0000 7f68be2e5640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68b814f1c0 0x7f68b814e9c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.503+0000 7f68be2e5640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f689806af90 0x7f689806d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.503+0000 7f68be2e5640 1 --2- 172.21.15.70:0/2845599955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f68a4004d00 0x7f68b815a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.503+0000 7f68be2e5640 1 -- 172.21.15.70:0/2845599955 >> 172.21.15.70:0/2845599955 conn(0x7f68b806a380 msgr2=0x7f68b8156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:13.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.503+0000 7f68be2e5640 1 -- 172.21.15.70:0/2845599955 shutdown_connections 2023-10-08T05:50:13.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.503+0000 7f68be2e5640 1 -- 172.21.15.70:0/2845599955 wait complete. 2023-10-08T05:50:13.522 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T05:50:13.522 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T05:50:13.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.742+0000 7ff1c9905640 1 Processor -- start 2023-10-08T05:50:13.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.743+0000 7ff1c9905640 1 -- start start 2023-10-08T05:50:13.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.744+0000 7ff1c9905640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff1c406b4a0 0x7ff1c40ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.744+0000 7ff1c9905640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1c40b48c0 0x7ff1c40ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.744+0000 7ff1c9905640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c40ccb80 con 0x7ff1c40cb430 2023-10-08T05:50:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.744+0000 7ff1c9905640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c40cd800 con 0x7ff1c406b4a0 2023-10-08T05:50:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.744+0000 7ff1c9905640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c40cce60 con 0x7ff1c40b48c0 2023-10-08T05:50:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.744+0000 7ff1c2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff1c406b4a0 0x7ff1c40ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.745+0000 7ff1c2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff1c406b4a0 0x7ff1c40ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43488/0 (socket says 172.21.15.70:43488) 2023-10-08T05:50:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.745+0000 7ff1c2ffd640 1 -- 172.21.15.70:0/4073313025 learned_addr learned my addr 172.21.15.70:0/4073313025 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.745+0000 7ff1c27fc640 1 --1- 172.21.15.70:0/4073313025 >> v1:172.21.15.70:6789/0 conn(0x7ff1c40cb430 0x7ff1c4069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52428/0 (socket says 172.21.15.70:52428) 2023-10-08T05:50:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.745+0000 7ff1c2ffd640 1 -- 172.21.15.70:0/4073313025 >> v2:172.21.15.181:3300/0 conn(0x7ff1c40b48c0 msgr2=0x7ff1c40ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.745+0000 7ff1c37fe640 1 --2- 172.21.15.70:0/4073313025 >> v2:172.21.15.181:3300/0 conn(0x7ff1c40b48c0 0x7ff1c40ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.745+0000 7ff1c2ffd640 1 --2- 172.21.15.70:0/4073313025 >> v2:172.21.15.181:3300/0 conn(0x7ff1c40b48c0 0x7ff1c40ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.745+0000 7ff1c1ffb640 1 -- 172.21.15.70:0/4073313025 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069673760 0 0) 0x7ff1c40ccb80 con 0x7ff1c40cb430 2023-10-08T05:50:13.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.745+0000 7ff1c2ffd640 1 -- 172.21.15.70:0/4073313025 >> v1:172.21.15.70:6789/0 conn(0x7ff1c40cb430 legacy=0x7ff1c4069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.745+0000 7ff1c2ffd640 1 -- 172.21.15.70:0/4073313025 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c40cd140 con 0x7ff1c406b4a0 2023-10-08T05:50:13.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.745+0000 7ff1c37fe640 1 --2- 172.21.15.70:0/4073313025 >> v2:172.21.15.181:3300/0 conn(0x7ff1c40b48c0 0x7ff1c40ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:13.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.746+0000 7ff1c2ffd640 1 --2- 172.21.15.70:0/4073313025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c406b4a0 0x7ff1c40ce1f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40c27f0 tx=0x7ff1b402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e19f4d3fbcaa867 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:13.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c1ffb640 1 -- 172.21.15.70:0/4073313025 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40317e0 con 0x7ff1c406b4a0 2023-10-08T05:50:13.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c1ffb640 1 -- 172.21.15.70:0/4073313025 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b4031940 con 0x7ff1c406b4a0 2023-10-08T05:50:13.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c1ffb640 1 -- 172.21.15.70:0/4073313025 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4031cd0 con 0x7ff1c406b4a0 2023-10-08T05:50:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c9905640 1 -- 172.21.15.70:0/4073313025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c406b4a0 msgr2=0x7ff1c40ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c9905640 1 --2- 172.21.15.70:0/4073313025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c406b4a0 0x7ff1c40ce1f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40c27f0 tx=0x7ff1b402b880 comp rx=0 tx=0).stop 2023-10-08T05:50:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c9905640 1 -- 172.21.15.70:0/4073313025 shutdown_connections 2023-10-08T05:50:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c9905640 1 --2- 172.21.15.70:0/4073313025 >> v2:172.21.15.181:3300/0 conn(0x7ff1c40b48c0 0x7ff1c40ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c9905640 1 --2- 172.21.15.70:0/4073313025 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c406b4a0 0x7ff1c40ce1f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c9905640 1 -- 172.21.15.70:0/4073313025 >> 172.21.15.70:0/4073313025 conn(0x7ff1c40cba30 msgr2=0x7ff1c40cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c9905640 1 -- 172.21.15.70:0/4073313025 shutdown_connections 2023-10-08T05:50:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.747+0000 7ff1c9905640 1 -- 172.21.15.70:0/4073313025 wait complete. 2023-10-08T05:50:13.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.749+0000 7ff1c9905640 1 Processor -- start 2023-10-08T05:50:13.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.749+0000 7ff1c9905640 1 -- start start 2023-10-08T05:50:13.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.750+0000 7ff1c9905640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1c40b48c0 0x7ff1c41f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.750+0000 7ff1c9905640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c40cb430 0x7ff1c40c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.750+0000 7ff1c9905640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c40c4b90 0x7ff1c4151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.750+0000 7ff1c9905640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40c8ef0 con 0x7ff1c40cb430 2023-10-08T05:50:13.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.750+0000 7ff1c9905640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40c8370 con 0x7ff1c40c4b90 2023-10-08T05:50:13.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.750+0000 7ff1c9905640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c40c8650 con 0x7ff1c40b48c0 2023-10-08T05:50:13.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c40cb430 0x7ff1c40c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c40cb430 0x7ff1c40c4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43490/0 (socket says 172.21.15.70:43490) 2023-10-08T05:50:13.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c27fc640 1 -- 172.21.15.70:0/3508836136 learned_addr learned my addr 172.21.15.70:0/3508836136 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:13.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c37fe640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c40c4b90 0x7ff1c4151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c2ffd640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1c40b48c0 0x7ff1c41f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c27fc640 1 -- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1c40b48c0 msgr2=0x7ff1c41f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c27fc640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1c40b48c0 0x7ff1c41f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c27fc640 1 -- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c40c4b90 msgr2=0x7ff1c4151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:13.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c27fc640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c40c4b90 0x7ff1c4151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:13.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c27fc640 1 -- 172.21.15.70:0/3508836136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b40094d0 con 0x7ff1c40cb430 2023-10-08T05:50:13.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.751+0000 7ff1c37fe640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c40c4b90 0x7ff1c4151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:13.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.752+0000 7ff1c27fc640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c40cb430 0x7ff1c40c4650 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00c4b0 tx=0x7ff1ac00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:13.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.752+0000 7ff1bbfff640 1 -- 172.21.15.70:0/3508836136 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac0074e0 con 0x7ff1c40cb430 2023-10-08T05:50:13.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.752+0000 7ff1bbfff640 1 -- 172.21.15.70:0/3508836136 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac007660 con 0x7ff1c40cb430 2023-10-08T05:50:13.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.752+0000 7ff1bbfff640 1 -- 172.21.15.70:0/3508836136 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac004a20 con 0x7ff1c40cb430 2023-10-08T05:50:13.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.752+0000 7ff1c9905640 1 -- 172.21.15.70:0/3508836136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c40c8930 con 0x7ff1c40cb430 2023-10-08T05:50:13.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.752+0000 7ff1c9905640 1 -- 172.21.15.70:0/3508836136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c41f2920 con 0x7ff1c40cb430 2023-10-08T05:50:13.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.753+0000 7ff1c9905640 1 -- 172.21.15.70:0/3508836136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c41f5a60 con 0x7ff1c40cb430 2023-10-08T05:50:13.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.754+0000 7ff1bbfff640 1 -- 172.21.15.70:0/3508836136 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff1ac004b80 con 0x7ff1c40cb430 2023-10-08T05:50:13.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.755+0000 7ff1bbfff640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff1a406af90 0x7ff1a406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:13.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.755+0000 7ff1bbfff640 1 -- 172.21.15.70:0/3508836136 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5221+0+0 (secure 0 0 0) 0x7ff1ac088720 con 0x7ff1c40cb430 2023-10-08T05:50:13.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.756+0000 7ff1c2ffd640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff1a406af90 0x7ff1a406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:13.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.757+0000 7ff1c2ffd640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff1a406af90 0x7ff1a406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4036000 tx=0x7ff1b403d040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:13.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.760+0000 7ff1bbfff640 1 -- 172.21.15.70:0/3508836136 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1c41f5a60 con 0x7ff1c40cb430 2023-10-08T05:50:13.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:13.952+0000 7ff1c9905640 1 -- 172.21.15.70:0/3508836136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff1c41f5a60 con 0x7ff1c40cb430 2023-10-08T05:50:14.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.665+0000 7ff1bbfff640 1 -- 172.21.15.70:0/3508836136 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff1c41f5a60 con 0x7ff1c40cb430 2023-10-08T05:50:14.667 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T05:50:14.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 -- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff1a406af90 msgr2=0x7ff1a406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:14.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff1a406af90 0x7ff1a406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4036000 tx=0x7ff1b403d040 comp rx=0 tx=0).stop 2023-10-08T05:50:14.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 -- 172.21.15.70:0/3508836136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c40cb430 msgr2=0x7ff1c40c4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:14.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c40cb430 0x7ff1c40c4650 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00c4b0 tx=0x7ff1ac00c970 comp rx=0 tx=0).stop 2023-10-08T05:50:14.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 -- 172.21.15.70:0/3508836136 shutdown_connections 2023-10-08T05:50:14.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1c40c4b90 0x7ff1c4151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:14.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1c40cb430 0x7ff1c40c4650 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:14.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff1a406af90 0x7ff1a406d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:14.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 --2- 172.21.15.70:0/3508836136 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1c40b48c0 0x7ff1c41f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:14.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 -- 172.21.15.70:0/3508836136 >> 172.21.15.70:0/3508836136 conn(0x7ff1c40cba30 msgr2=0x7ff1c4069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:14.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 -- 172.21.15.70:0/3508836136 shutdown_connections 2023-10-08T05:50:14.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.669+0000 7ff1c9905640 1 -- 172.21.15.70:0/3508836136 wait complete. 2023-10-08T05:50:14.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T05:50:14.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.902+0000 7f6d6ae02640 1 Processor -- start 2023-10-08T05:50:14.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.902+0000 7f6d6ae02640 1 -- start start 2023-10-08T05:50:14.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.903+0000 7f6d6ae02640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d640cb500 0x7f6d640cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:14.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.903+0000 7f6d6ae02640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6d6406cfa0 0x7f6d6415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:14.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.903+0000 7f6d6ae02640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d640c8c60 con 0x7f6d6415b630 2023-10-08T05:50:14.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.903+0000 7f6d6ae02640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d640cbe20 con 0x7f6d6406cfa0 2023-10-08T05:50:14.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.903+0000 7f6d6ae02640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d640b1ac0 con 0x7f6d640cb500 2023-10-08T05:50:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.904+0000 7f6d5bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6d6406cfa0 0x7f6d6415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.904+0000 7f6d5bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6d6406cfa0 0x7f6d6415b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43494/0 (socket says 172.21.15.70:43494) 2023-10-08T05:50:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.904+0000 7f6d5bfff640 1 -- 172.21.15.70:0/2648816072 learned_addr learned my addr 172.21.15.70:0/2648816072 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.904+0000 7f6d5bfff640 1 -- 172.21.15.70:0/2648816072 >> v2:172.21.15.181:3300/0 conn(0x7f6d640cb500 msgr2=0x7f6d640cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.904+0000 7f6d68b9e640 1 --2- 172.21.15.70:0/2648816072 >> v2:172.21.15.181:3300/0 conn(0x7f6d640cb500 0x7f6d640cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.904+0000 7f6d5bfff640 1 --2- 172.21.15.70:0/2648816072 >> v2:172.21.15.181:3300/0 conn(0x7f6d640cb500 0x7f6d640cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:14.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.904+0000 7f6d5bfff640 1 -- 172.21.15.70:0/2648816072 >> v1:172.21.15.181:6789/0 conn(0x7f6d6415b630 legacy=0x7f6d64070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:14.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.904+0000 7f6d5bfff640 1 -- 172.21.15.70:0/2648816072 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640b1da0 con 0x7f6d6406cfa0 2023-10-08T05:50:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.905+0000 7f6d5bfff640 1 --2- 172.21.15.70:0/2648816072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6406cfa0 0x7f6d6415b0f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6d640c58f0 tx=0x7f6d5402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a68f7f47961f6e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.906+0000 7f6d5b7fe640 1 -- 172.21.15.70:0/2648816072 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d540317e0 con 0x7f6d6406cfa0 2023-10-08T05:50:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.906+0000 7f6d5b7fe640 1 -- 172.21.15.70:0/2648816072 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d54031940 con 0x7f6d6406cfa0 2023-10-08T05:50:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.906+0000 7f6d5b7fe640 1 -- 172.21.15.70:0/2648816072 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54031cd0 con 0x7f6d6406cfa0 2023-10-08T05:50:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.906+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2648816072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6406cfa0 msgr2=0x7f6d6415b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.906+0000 7f6d6ae02640 1 --2- 172.21.15.70:0/2648816072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6406cfa0 0x7f6d6415b0f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6d640c58f0 tx=0x7f6d5402b880 comp rx=0 tx=0).stop 2023-10-08T05:50:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.907+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2648816072 shutdown_connections 2023-10-08T05:50:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.907+0000 7f6d6ae02640 1 --2- 172.21.15.70:0/2648816072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6406cfa0 0x7f6d6415b0f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:14.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.907+0000 7f6d6ae02640 1 --2- 172.21.15.70:0/2648816072 >> v2:172.21.15.181:3300/0 conn(0x7f6d640cb500 0x7f6d640cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:14.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.907+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2648816072 >> 172.21.15.70:0/2648816072 conn(0x7f6d6406a140 msgr2=0x7f6d6406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:14.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.907+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2648816072 shutdown_connections 2023-10-08T05:50:14.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.907+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2648816072 wait complete. 2023-10-08T05:50:14.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.908+0000 7f6d6ae02640 1 Processor -- start 2023-10-08T05:50:14.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.909+0000 7f6d6ae02640 1 -- start start 2023-10-08T05:50:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.910+0000 7f6d6ae02640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d6406cfa0 0x7f6d6414a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.910+0000 7f6d6ae02640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d640cb500 0x7f6d6414ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.910+0000 7f6d6ae02640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6415b630 0x7f6d6414cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.910+0000 7f6d6ae02640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d641593f0 con 0x7f6d6415b630 2023-10-08T05:50:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.910+0000 7f6d6ae02640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6414ab90 con 0x7f6d6406cfa0 2023-10-08T05:50:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.910+0000 7f6d6ae02640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6414c470 con 0x7f6d640cb500 2023-10-08T05:50:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.910+0000 7f6d6939f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6415b630 0x7f6d6414cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.910+0000 7f6d6939f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6415b630 0x7f6d6414cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43506/0 (socket says 172.21.15.70:43506) 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.910+0000 7f6d6939f640 1 -- 172.21.15.70:0/2067788617 learned_addr learned my addr 172.21.15.70:0/2067788617 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.911+0000 7f6d68b9e640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d6406cfa0 0x7f6d6414a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.911+0000 7f6d5bfff640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d640cb500 0x7f6d6414ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.911+0000 7f6d6939f640 1 -- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d640cb500 msgr2=0x7f6d6414ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.911+0000 7f6d6939f640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d640cb500 0x7f6d6414ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.911+0000 7f6d6939f640 1 -- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d6406cfa0 msgr2=0x7f6d6414a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.911+0000 7f6d6939f640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d6406cfa0 0x7f6d6414a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.911+0000 7f6d6939f640 1 -- 172.21.15.70:0/2067788617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d540094d0 con 0x7f6d6415b630 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.911+0000 7f6d68b9e640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d6406cfa0 0x7f6d6414a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.911+0000 7f6d6939f640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6415b630 0x7f6d6414cea0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00b4c0 tx=0x7f6d5c00b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:14.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.911+0000 7f6d597fa640 1 -- 172.21.15.70:0/2067788617 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c011830 con 0x7f6d6415b630 2023-10-08T05:50:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.912+0000 7f6d597fa640 1 -- 172.21.15.70:0/2067788617 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c0119b0 con 0x7f6d6415b630 2023-10-08T05:50:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.912+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2067788617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6414b860 con 0x7f6d6415b630 2023-10-08T05:50:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.912+0000 7f6d597fa640 1 -- 172.21.15.70:0/2067788617 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c011d60 con 0x7f6d6415b630 2023-10-08T05:50:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.912+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2067788617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6414be20 con 0x7f6d6415b630 2023-10-08T05:50:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.913+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2067788617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6414c600 con 0x7f6d6415b630 2023-10-08T05:50:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.914+0000 7f6d597fa640 1 -- 172.21.15.70:0/2067788617 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6d5c01a400 con 0x7f6d6415b630 2023-10-08T05:50:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.915+0000 7f6d597fa640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6d5006af90 0x7f6d5006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.915+0000 7f6d597fa640 1 -- 172.21.15.70:0/2067788617 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5591+0+0 (secure 0 0 0) 0x7f6d5c089e30 con 0x7f6d6415b630 2023-10-08T05:50:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.916+0000 7f6d68b9e640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6d5006af90 0x7f6d5006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:14.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.917+0000 7f6d68b9e640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6d5006af90 0x7f6d5006d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6d60007fc0 tx=0x7f6d600075b0 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:14.919+0000 7f6d597fa640 1 -- 172.21.15.70:0/2067788617 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d5c054e20 con 0x7f6d6415b630 2023-10-08T05:50:15.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.109+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2067788617 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6d64071cb0 con 0x7f6d6415b630 2023-10-08T05:50:15.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.680+0000 7f6d597fa640 1 -- 172.21.15.70:0/2067788617 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6d5c0150d0 con 0x7f6d6415b630 2023-10-08T05:50:15.683 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T05:50:15.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6d5006af90 msgr2=0x7f6d5006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:15.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6d5006af90 0x7f6d5006d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6d60007fc0 tx=0x7f6d600075b0 comp rx=0 tx=0).stop 2023-10-08T05:50:15.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2067788617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6415b630 msgr2=0x7f6d6414cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:15.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6415b630 0x7f6d6414cea0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00b4c0 tx=0x7f6d5c00b980 comp rx=0 tx=0).stop 2023-10-08T05:50:15.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2067788617 shutdown_connections 2023-10-08T05:50:15.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6415b630 0x7f6d6414cea0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:15.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6d5006af90 0x7f6d5006d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:15.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d640cb500 0x7f6d6414ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:15.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 --2- 172.21.15.70:0/2067788617 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d6406cfa0 0x7f6d6414a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:15.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2067788617 >> 172.21.15.70:0/2067788617 conn(0x7f6d6406a140 msgr2=0x7f6d640b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:15.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2067788617 shutdown_connections 2023-10-08T05:50:15.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.684+0000 7f6d6ae02640 1 -- 172.21.15.70:0/2067788617 wait complete. 2023-10-08T05:50:15.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T05:50:15.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.909+0000 7fcf5c4a9640 1 Processor -- start 2023-10-08T05:50:15.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.909+0000 7fcf5c4a9640 1 -- start start 2023-10-08T05:50:15.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.911+0000 7fcf5c4a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcf540afbe0 0x7fcf540b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:15.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.911+0000 7fcf5c4a9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf540b2620 con 0x7fcf5406dfc0 2023-10-08T05:50:15.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.911+0000 7fcf5c4a9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf540b2060 con 0x7fcf540cb130 2023-10-08T05:50:15.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.911+0000 7fcf5c4a9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf540b2340 con 0x7fcf540afbe0 2023-10-08T05:50:15.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.911+0000 7fcf5a245640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcf540afbe0 0x7fcf540b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:15.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.911+0000 7fcf59a44640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcf5406dfc0 0x7fcf54070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52434/0 (socket says 172.21.15.70:52434) 2023-10-08T05:50:15.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.911+0000 7fcf59a44640 1 -- 172.21.15.70:0/443930032 learned_addr learned my addr 172.21.15.70:0/443930032 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:15.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.911+0000 7fcf5a245640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcf540afbe0 0x7fcf540b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43508/0 (socket says 172.21.15.70:43508) 2023-10-08T05:50:15.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.912+0000 7fcf5a245640 1 -- 172.21.15.70:0/443930032 >> v1:172.21.15.181:6789/0 conn(0x7fcf540cb130 legacy=0x7fcf5406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:15.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.912+0000 7fcf5a245640 1 -- 172.21.15.70:0/443930032 >> v1:172.21.15.70:6789/0 conn(0x7fcf5406dfc0 legacy=0x7fcf54070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:15.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.912+0000 7fcf59243640 1 -- 172.21.15.70:0/443930032 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848357627 0 0) 0x7fcf540b2620 con 0x7fcf5406dfc0 2023-10-08T05:50:15.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.912+0000 7fcf5a245640 1 -- 172.21.15.70:0/443930032 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54070350 con 0x7fcf540afbe0 2023-10-08T05:50:15.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.913+0000 7fcf5a245640 1 --2- 172.21.15.70:0/443930032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 0x7fcf540b15b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009d90 tx=0x7fcf5002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9579d50e0d4024ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:15.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.913+0000 7fcf59243640 1 -- 172.21.15.70:0/443930032 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5002cc60 con 0x7fcf540afbe0 2023-10-08T05:50:15.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.913+0000 7fcf59243640 1 -- 172.21.15.70:0/443930032 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5002cdc0 con 0x7fcf540afbe0 2023-10-08T05:50:15.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.913+0000 7fcf59243640 1 -- 172.21.15.70:0/443930032 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50034660 con 0x7fcf540afbe0 2023-10-08T05:50:15.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.914+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/443930032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 msgr2=0x7fcf540b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:15.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.914+0000 7fcf5c4a9640 1 --2- 172.21.15.70:0/443930032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 0x7fcf540b15b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009d90 tx=0x7fcf5002beb0 comp rx=0 tx=0).stop 2023-10-08T05:50:15.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.914+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/443930032 shutdown_connections 2023-10-08T05:50:15.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.914+0000 7fcf5c4a9640 1 --2- 172.21.15.70:0/443930032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 0x7fcf540b15b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:15.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.914+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/443930032 >> 172.21.15.70:0/443930032 conn(0x7fcf5406a5d0 msgr2=0x7fcf540b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:15.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.914+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/443930032 shutdown_connections 2023-10-08T05:50:15.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.914+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/443930032 wait complete. 2023-10-08T05:50:15.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.915+0000 7fcf5c4a9640 1 Processor -- start 2023-10-08T05:50:15.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.916+0000 7fcf5c4a9640 1 -- start start 2023-10-08T05:50:15.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf5c4a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5406dfc0 0x7fcf5415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:15.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf5c4a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 0x7fcf540af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:15.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf5c4a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf540cb130 0x7fcf540c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf5c4a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540cd140 con 0x7fcf540afbe0 2023-10-08T05:50:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf5c4a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540af6c0 con 0x7fcf5406dfc0 2023-10-08T05:50:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf5c4a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf540cd390 con 0x7fcf540cb130 2023-10-08T05:50:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf59a44640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 0x7fcf540af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf59a44640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 0x7fcf540af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43522/0 (socket says 172.21.15.70:43522) 2023-10-08T05:50:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf59a44640 1 -- 172.21.15.70:0/2326551148 learned_addr learned my addr 172.21.15.70:0/2326551148 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf59a44640 1 -- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf540cb130 msgr2=0x7fcf540c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf59a44640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf540cb130 0x7fcf540c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf59a44640 1 -- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5406dfc0 msgr2=0x7fcf5415df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:15.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf59a44640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5406dfc0 0x7fcf5415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:15.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.917+0000 7fcf59a44640 1 -- 172.21.15.70:0/2326551148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf50009a70 con 0x7fcf540afbe0 2023-10-08T05:50:15.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.918+0000 7fcf59a44640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 0x7fcf540af180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400c410 tx=0x7fcf4400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:15.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.918+0000 7fcf4affd640 1 -- 172.21.15.70:0/2326551148 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf440074e0 con 0x7fcf540afbe0 2023-10-08T05:50:15.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.918+0000 7fcf4affd640 1 -- 172.21.15.70:0/2326551148 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf44007660 con 0x7fcf540afbe0 2023-10-08T05:50:15.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.918+0000 7fcf4affd640 1 -- 172.21.15.70:0/2326551148 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44004a20 con 0x7fcf540afbe0 2023-10-08T05:50:15.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.918+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/2326551148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf540cd670 con 0x7fcf540afbe0 2023-10-08T05:50:15.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.918+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/2326551148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf540cdc30 con 0x7fcf540afbe0 2023-10-08T05:50:15.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.919+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/2326551148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5404e160 con 0x7fcf540afbe0 2023-10-08T05:50:15.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.920+0000 7fcf4affd640 1 -- 172.21.15.70:0/2326551148 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcf44004b80 con 0x7fcf540afbe0 2023-10-08T05:50:15.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.921+0000 7fcf4affd640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcf3806af90 0x7fcf3806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:15.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.921+0000 7fcf4affd640 1 -- 172.21.15.70:0/2326551148 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5973+0+0 (secure 0 0 0) 0x7fcf44089730 con 0x7fcf540afbe0 2023-10-08T05:50:15.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.922+0000 7fcf5a245640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcf3806af90 0x7fcf3806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:15.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.923+0000 7fcf5a245640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcf3806af90 0x7fcf3806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcf50036000 tx=0x7fcf5002e000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:15.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:15.926+0000 7fcf4affd640 1 -- 172.21.15.70:0/2326551148 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf44058040 con 0x7fcf540afbe0 2023-10-08T05:50:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.141+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/2326551148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcf540c5280 con 0x7fcf540afbe0 2023-10-08T05:50:16.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.694+0000 7fcf4affd640 1 -- 172.21.15.70:0/2326551148 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7fcf44057bd0 con 0x7fcf540afbe0 2023-10-08T05:50:16.700 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 7 and data pool 8 2023-10-08T05:50:16.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.701+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcf3806af90 msgr2=0x7fcf3806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:16.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.701+0000 7fcf5c4a9640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcf3806af90 0x7fcf3806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcf50036000 tx=0x7fcf5002e000 comp rx=0 tx=0).stop 2023-10-08T05:50:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.702+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/2326551148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 msgr2=0x7fcf540af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.702+0000 7fcf5c4a9640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 0x7fcf540af180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fcf4400c410 tx=0x7fcf4400c8d0 comp rx=0 tx=0).stop 2023-10-08T05:50:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.704+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/2326551148 shutdown_connections 2023-10-08T05:50:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.704+0000 7fcf5c4a9640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fcf3806af90 0x7fcf3806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.704+0000 7fcf5c4a9640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf540cb130 0x7fcf540c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.704+0000 7fcf5c4a9640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf540afbe0 0x7fcf540af180 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.704+0000 7fcf5c4a9640 1 --2- 172.21.15.70:0/2326551148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf5406dfc0 0x7fcf5415df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.704+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/2326551148 >> 172.21.15.70:0/2326551148 conn(0x7fcf5406a5d0 msgr2=0x7fcf540c9030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.708+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/2326551148 shutdown_connections 2023-10-08T05:50:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.709+0000 7fcf5c4a9640 1 -- 172.21.15.70:0/2326551148 wait complete. 2023-10-08T05:50:16.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:50:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.922+0000 7f2714168640 1 Processor -- start 2023-10-08T05:50:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.923+0000 7f2714168640 1 -- start start 2023-10-08T05:50:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.924+0000 7f2714168640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f270c06d4c0 0x7f270c162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.924+0000 7f2714168640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f270c0cb150 0x7f270c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.924+0000 7f2714168640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270c0ae140 con 0x7f270c0add60 2023-10-08T05:50:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.924+0000 7f2714168640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270c161f10 con 0x7f270c0cb150 2023-10-08T05:50:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.924+0000 7f2714168640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0cbb20 con 0x7f270c06d4c0 2023-10-08T05:50:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.924+0000 7f2711f04640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f270c0add60 0x7f270c0cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52448/0 (socket says 172.21.15.70:52448) 2023-10-08T05:50:16.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.925+0000 7f2711f04640 1 -- 172.21.15.70:0/2702074638 learned_addr learned my addr 172.21.15.70:0/2702074638 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:16.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.925+0000 7f2712705640 1 --2- 172.21.15.70:0/2702074638 >> v2:172.21.15.70:3300/0 conn(0x7f270c0cb150 0x7f270c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:16.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.925+0000 7f2711703640 1 --2- 172.21.15.70:0/2702074638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f270c06d4c0 0x7f270c162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:16.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.925+0000 7f2710f02640 1 -- 172.21.15.70:0/2702074638 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676650743 0 0) 0x7f270c0ae140 con 0x7f270c0add60 2023-10-08T05:50:16.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.925+0000 7f2710f02640 1 -- 172.21.15.70:0/2702074638 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26f8003520 con 0x7f270c0add60 2023-10-08T05:50:16.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.925+0000 7f2712705640 1 -- 172.21.15.70:0/2702074638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f270c06d4c0 msgr2=0x7f270c162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:16.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.925+0000 7f2712705640 1 --2- 172.21.15.70:0/2702074638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f270c06d4c0 0x7f270c162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:16.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.925+0000 7f2712705640 1 -- 172.21.15.70:0/2702074638 >> v1:172.21.15.70:6789/0 conn(0x7f270c0add60 legacy=0x7f270c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:16.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.925+0000 7f2712705640 1 -- 172.21.15.70:0/2702074638 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c0ccf20 con 0x7f270c0cb150 2023-10-08T05:50:16.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.926+0000 7f2712705640 1 --2- 172.21.15.70:0/2702074638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c0cb150 0x7f270c0cb5e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2704009d90 tx=0x7f270402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c1dc891be59eb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:16.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2710f02640 1 -- 172.21.15.70:0/2702074638 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270402cbd0 con 0x7f270c0cb150 2023-10-08T05:50:16.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2710f02640 1 -- 172.21.15.70:0/2702074638 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270402cd30 con 0x7f270c0cb150 2023-10-08T05:50:16.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2710f02640 1 -- 172.21.15.70:0/2702074638 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270402cea0 con 0x7f270c0cb150 2023-10-08T05:50:16.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2714168640 1 -- 172.21.15.70:0/2702074638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c0cb150 msgr2=0x7f270c0cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:16.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2714168640 1 --2- 172.21.15.70:0/2702074638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c0cb150 0x7f270c0cb5e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f2704009d90 tx=0x7f270402be20 comp rx=0 tx=0).stop 2023-10-08T05:50:16.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2714168640 1 -- 172.21.15.70:0/2702074638 shutdown_connections 2023-10-08T05:50:16.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2714168640 1 --2- 172.21.15.70:0/2702074638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c0cb150 0x7f270c0cb5e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:16.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2714168640 1 --2- 172.21.15.70:0/2702074638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f270c06d4c0 0x7f270c162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:16.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2714168640 1 -- 172.21.15.70:0/2702074638 >> 172.21.15.70:0/2702074638 conn(0x7f270c06a380 msgr2=0x7f270c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:16.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2714168640 1 -- 172.21.15.70:0/2702074638 shutdown_connections 2023-10-08T05:50:16.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.927+0000 7f2714168640 1 -- 172.21.15.70:0/2702074638 wait complete. 2023-10-08T05:50:16.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.929+0000 7f2714168640 1 Processor -- start 2023-10-08T05:50:16.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.929+0000 7f2714168640 1 -- start start 2023-10-08T05:50:16.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.930+0000 7f2714168640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c06d4c0 0x7f270c15f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:16.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.930+0000 7f2714168640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f270c0add60 0x7f270c0c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:16.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.930+0000 7f2714168640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c0cb150 0x7f270c0c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:16.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.930+0000 7f2714168640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0b2f10 con 0x7f270c06d4c0 2023-10-08T05:50:16.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.930+0000 7f2714168640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c15f930 con 0x7f270c0cb150 2023-10-08T05:50:16.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.930+0000 7f2714168640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0b2570 con 0x7f270c0add60 2023-10-08T05:50:16.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f2711f04640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c06d4c0 0x7f270c15f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:16.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f2711f04640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c06d4c0 0x7f270c15f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43534/0 (socket says 172.21.15.70:43534) 2023-10-08T05:50:16.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f2711f04640 1 -- 172.21.15.70:0/229647482 learned_addr learned my addr 172.21.15.70:0/229647482 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:16.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f2711f04640 1 -- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f270c0add60 msgr2=0x7f270c0c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:16.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f2711703640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f270c0add60 0x7f270c0c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:16.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f2711f04640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f270c0add60 0x7f270c0c4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:16.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f2711f04640 1 -- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c0cb150 msgr2=0x7f270c0c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:16.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f2711f04640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c0cb150 0x7f270c0c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:16.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f2711f04640 1 -- 172.21.15.70:0/229647482 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2704009a70 con 0x7f270c06d4c0 2023-10-08T05:50:16.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f2711f04640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c06d4c0 0x7f270c15f3f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2708001eb0 tx=0x7f270800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:16.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.931+0000 7f27027fc640 1 -- 172.21.15.70:0/229647482 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708005ce0 con 0x7f270c06d4c0 2023-10-08T05:50:16.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.932+0000 7f27027fc640 1 -- 172.21.15.70:0/229647482 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2708005e40 con 0x7f270c06d4c0 2023-10-08T05:50:16.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.932+0000 7f27027fc640 1 -- 172.21.15.70:0/229647482 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708007270 con 0x7f270c06d4c0 2023-10-08T05:50:16.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.932+0000 7f2714168640 1 -- 172.21.15.70:0/229647482 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270c0b2850 con 0x7f270c06d4c0 2023-10-08T05:50:16.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.932+0000 7f2714168640 1 -- 172.21.15.70:0/229647482 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270c0ca650 con 0x7f270c06d4c0 2023-10-08T05:50:16.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.933+0000 7f2714168640 1 -- 172.21.15.70:0/229647482 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270c067490 con 0x7f270c06d4c0 2023-10-08T05:50:16.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.934+0000 7f27027fc640 1 -- 172.21.15.70:0/229647482 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f27080073d0 con 0x7f270c06d4c0 2023-10-08T05:50:16.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.934+0000 7f27027fc640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f26f006af90 0x7f26f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:16.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.935+0000 7f27027fc640 1 -- 172.21.15.70:0/229647482 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2708088f80 con 0x7f270c06d4c0 2023-10-08T05:50:16.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.935+0000 7f2711703640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f26f006af90 0x7f26f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:16.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.936+0000 7f2711703640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f26f006af90 0x7f26f006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f270c0cf220 tx=0x7f26fc008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:16.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:16.941+0000 7f27027fc640 1 -- 172.21.15.70:0/229647482 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2708053d10 con 0x7f270c06d4c0 2023-10-08T05:50:17.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.129+0000 7f2714168640 1 -- 172.21.15.70:0/229647482 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f270c0b0ef0 con 0x7f270c06d4c0 2023-10-08T05:50:17.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.132+0000 7f27027fc640 1 -- 172.21.15.70:0/229647482 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16845 (secure 0 0 0) 0x7f2708017070 con 0x7f270c06d4c0 2023-10-08T05:50:17.134 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:17.134 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":71,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:50:16.143679+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:50:14.005619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-08T05:50:15.163925+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:50:17.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.135+0000 7f2714168640 1 -- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f26f006af90 msgr2=0x7f26f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.135+0000 7f2714168640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f26f006af90 0x7f26f006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f270c0cf220 tx=0x7f26fc008040 comp rx=0 tx=0).stop 2023-10-08T05:50:17.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.135+0000 7f2714168640 1 -- 172.21.15.70:0/229647482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c06d4c0 msgr2=0x7f270c15f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.135+0000 7f2714168640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c06d4c0 0x7f270c15f3f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2708001eb0 tx=0x7f270800c960 comp rx=0 tx=0).stop 2023-10-08T05:50:17.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.135+0000 7f2714168640 1 -- 172.21.15.70:0/229647482 shutdown_connections 2023-10-08T05:50:17.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.135+0000 7f2714168640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f270c0cb150 0x7f270c0c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.135+0000 7f2714168640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f26f006af90 0x7f26f006d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.135+0000 7f2714168640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f270c0add60 0x7f270c0c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.135+0000 7f2714168640 1 --2- 172.21.15.70:0/229647482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f270c06d4c0 0x7f270c15f3f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.135+0000 7f2714168640 1 -- 172.21.15.70:0/229647482 >> 172.21.15.70:0/229647482 conn(0x7f270c06a380 msgr2=0x7f270c0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:17.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.136+0000 7f2714168640 1 -- 172.21.15.70:0/229647482 shutdown_connections 2023-10-08T05:50:17.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.136+0000 7f2714168640 1 -- 172.21.15.70:0/229647482 wait complete. 2023-10-08T05:50:17.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:50:17.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.368+0000 7fea283b6640 1 Processor -- start 2023-10-08T05:50:17.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.368+0000 7fea283b6640 1 -- start start 2023-10-08T05:50:17.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.370+0000 7fea283b6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fea20071190 0x7fea2015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.370+0000 7fea283b6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea2006ad00 0x7fea2006b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.370+0000 7fea283b6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea200c9850 con 0x7fea20070db0 2023-10-08T05:50:17.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.370+0000 7fea283b6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2006a870 con 0x7fea2006ad00 2023-10-08T05:50:17.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.370+0000 7fea283b6640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea200cd640 con 0x7fea20071190 2023-10-08T05:50:17.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.370+0000 7fea26953640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea2006ad00 0x7fea2006b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.371+0000 7fea26953640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea2006ad00 0x7fea2006b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43542/0 (socket says 172.21.15.70:43542) 2023-10-08T05:50:17.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.371+0000 7fea26953640 1 -- 172.21.15.70:0/772466684 learned_addr learned my addr 172.21.15.70:0/772466684 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:17.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.371+0000 7fea25951640 1 --2- 172.21.15.70:0/772466684 >> v2:172.21.15.181:3300/0 conn(0x7fea20071190 0x7fea2015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.371+0000 7fea26953640 1 -- 172.21.15.70:0/772466684 >> v2:172.21.15.181:3300/0 conn(0x7fea20071190 msgr2=0x7fea2015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:17.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.371+0000 7fea26953640 1 --2- 172.21.15.70:0/772466684 >> v2:172.21.15.181:3300/0 conn(0x7fea20071190 0x7fea2015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.371+0000 7fea26953640 1 -- 172.21.15.70:0/772466684 >> v1:172.21.15.181:6789/0 conn(0x7fea20070db0 legacy=0x7fea2015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:17.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.371+0000 7fea26953640 1 -- 172.21.15.70:0/772466684 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200cd910 con 0x7fea2006ad00 2023-10-08T05:50:17.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.372+0000 7fea26953640 1 --2- 172.21.15.70:0/772466684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea2006ad00 0x7fea2006b130 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fea2006dc30 tx=0x7fea1802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12b437eda1114d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.372+0000 7fea25150640 1 -- 172.21.15.70:0/772466684 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1802cb20 con 0x7fea2006ad00 2023-10-08T05:50:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.372+0000 7fea25150640 1 -- 172.21.15.70:0/772466684 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1802cc80 con 0x7fea2006ad00 2023-10-08T05:50:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.372+0000 7fea25150640 1 -- 172.21.15.70:0/772466684 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea180343f0 con 0x7fea2006ad00 2023-10-08T05:50:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.373+0000 7fea283b6640 1 -- 172.21.15.70:0/772466684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea2006ad00 msgr2=0x7fea2006b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.373+0000 7fea283b6640 1 --2- 172.21.15.70:0/772466684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea2006ad00 0x7fea2006b130 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fea2006dc30 tx=0x7fea1802bf80 comp rx=0 tx=0).stop 2023-10-08T05:50:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.373+0000 7fea283b6640 1 -- 172.21.15.70:0/772466684 shutdown_connections 2023-10-08T05:50:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.373+0000 7fea283b6640 1 --2- 172.21.15.70:0/772466684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea2006ad00 0x7fea2006b130 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.373+0000 7fea283b6640 1 --2- 172.21.15.70:0/772466684 >> v2:172.21.15.181:3300/0 conn(0x7fea20071190 0x7fea2015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.373+0000 7fea283b6640 1 -- 172.21.15.70:0/772466684 >> 172.21.15.70:0/772466684 conn(0x7fea200b1480 msgr2=0x7fea2006cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:17.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.373+0000 7fea283b6640 1 -- 172.21.15.70:0/772466684 shutdown_connections 2023-10-08T05:50:17.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.373+0000 7fea283b6640 1 -- 172.21.15.70:0/772466684 wait complete. 2023-10-08T05:50:17.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.375+0000 7fea283b6640 1 Processor -- start 2023-10-08T05:50:17.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.375+0000 7fea283b6640 1 -- start start 2023-10-08T05:50:17.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.376+0000 7fea283b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea2006ad00 0x7fea20154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.376+0000 7fea283b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea20070db0 0x7fea200c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.376+0000 7fea283b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20071190 0x7fea200c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.376+0000 7fea283b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea20154b90 con 0x7fea20071190 2023-10-08T05:50:17.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.376+0000 7fea283b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c7ad0 con 0x7fea2006ad00 2023-10-08T05:50:17.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.376+0000 7fea283b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea200c7640 con 0x7fea20070db0 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.376+0000 7fea26953640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20071190 0x7fea200c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.376+0000 7fea26953640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20071190 0x7fea200c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43552/0 (socket says 172.21.15.70:43552) 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.376+0000 7fea26953640 1 -- 172.21.15.70:0/723896081 learned_addr learned my addr 172.21.15.70:0/723896081 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea25951640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea20070db0 0x7fea200c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea26152640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea2006ad00 0x7fea20154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea26953640 1 -- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea20070db0 msgr2=0x7fea200c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea26953640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea20070db0 0x7fea200c6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea26953640 1 -- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea2006ad00 msgr2=0x7fea20154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea26953640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea2006ad00 0x7fea20154650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea26953640 1 -- 172.21.15.70:0/723896081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea18009b30 con 0x7fea20071190 2023-10-08T05:50:17.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea26953640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20071190 0x7fea200c85f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fea1802c590 tx=0x7fea18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:17.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea16ffd640 1 -- 172.21.15.70:0/723896081 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18009510 con 0x7fea20071190 2023-10-08T05:50:17.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea16ffd640 1 -- 172.21.15.70:0/723896081 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1803e450 con 0x7fea20071190 2023-10-08T05:50:17.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea16ffd640 1 -- 172.21.15.70:0/723896081 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1803e7e0 con 0x7fea20071190 2023-10-08T05:50:17.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea283b6640 1 -- 172.21.15.70:0/723896081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea200c6a30 con 0x7fea20071190 2023-10-08T05:50:17.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.377+0000 7fea283b6640 1 -- 172.21.15.70:0/723896081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea200c6ff0 con 0x7fea20071190 2023-10-08T05:50:17.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.378+0000 7fea283b6640 1 -- 172.21.15.70:0/723896081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2006a1a0 con 0x7fea20071190 2023-10-08T05:50:17.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.379+0000 7fea16ffd640 1 -- 172.21.15.70:0/723896081 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fea1803e940 con 0x7fea20071190 2023-10-08T05:50:17.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.380+0000 7fea16ffd640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fea0406af90 0x7fea0406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.380+0000 7fea16ffd640 1 -- 172.21.15.70:0/723896081 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea180ac940 con 0x7fea20071190 2023-10-08T05:50:17.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.381+0000 7fea26152640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fea0406af90 0x7fea0406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.382+0000 7fea26152640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fea0406af90 0x7fea0406d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea1c003d90 tx=0x7fea1c009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:17.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.385+0000 7fea16ffd640 1 -- 172.21.15.70:0/723896081 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea180777f0 con 0x7fea20071190 2023-10-08T05:50:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.573+0000 7fea283b6640 1 -- 172.21.15.70:0/723896081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea20067490 con 0x7fea20071190 2023-10-08T05:50:17.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.575+0000 7fea16ffd640 1 -- 172.21.15.70:0/723896081 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16845 (secure 0 0 0) 0x7fea1807ae10 con 0x7fea20071190 2023-10-08T05:50:17.577 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:17.577 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":71,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:50:16.143679+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:50:14.005619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-08T05:50:15.163925+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:50:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 -- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fea0406af90 msgr2=0x7fea0406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fea0406af90 0x7fea0406d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea1c003d90 tx=0x7fea1c009040 comp rx=0 tx=0).stop 2023-10-08T05:50:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 -- 172.21.15.70:0/723896081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20071190 msgr2=0x7fea200c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20071190 0x7fea200c85f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fea1802c590 tx=0x7fea18004a40 comp rx=0 tx=0).stop 2023-10-08T05:50:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 -- 172.21.15.70:0/723896081 shutdown_connections 2023-10-08T05:50:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea20071190 0x7fea200c85f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fea0406af90 0x7fea0406d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea20070db0 0x7fea200c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 --2- 172.21.15.70:0/723896081 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea2006ad00 0x7fea20154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 -- 172.21.15.70:0/723896081 >> 172.21.15.70:0/723896081 conn(0x7fea200b1480 msgr2=0x7fea200cbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 -- 172.21.15.70:0/723896081 shutdown_connections 2023-10-08T05:50:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.578+0000 7fea283b6640 1 -- 172.21.15.70:0/723896081 wait complete. 2023-10-08T05:50:17.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T05:50:17.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.813+0000 7fb3e5997640 1 Processor -- start 2023-10-08T05:50:17.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.813+0000 7fb3e5997640 1 -- start start 2023-10-08T05:50:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.814+0000 7fb3e5997640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb3e00afbe0 0x7fb3e00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.814+0000 7fb3e5997640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb3e00cb130 0x7fb3e00cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.814+0000 7fb3e5997640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e00cbad0 con 0x7fb3e006dfc0 2023-10-08T05:50:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.814+0000 7fb3e5997640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e00b2620 con 0x7fb3e00cb130 2023-10-08T05:50:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.814+0000 7fb3e5997640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e00b2060 con 0x7fb3e00afbe0 2023-10-08T05:50:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.814+0000 7fb3df7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb3e00cb130 0x7fb3e00cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.814+0000 7fb3df7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb3e00cb130 0x7fb3e00cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43558/0 (socket says 172.21.15.70:43558) 2023-10-08T05:50:17.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.814+0000 7fb3d7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb3e006dfc0 0x7fb3e0070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52456/0 (socket says 172.21.15.70:52456) 2023-10-08T05:50:17.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.814+0000 7fb3df7fe640 1 -- 172.21.15.70:0/235468526 learned_addr learned my addr 172.21.15.70:0/235468526 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:17.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.815+0000 7fb3deffd640 1 --2- 172.21.15.70:0/235468526 >> v2:172.21.15.181:3300/0 conn(0x7fb3e00afbe0 0x7fb3e00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.815+0000 7fb3de7fc640 1 -- 172.21.15.70:0/235468526 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719310317 0 0) 0x7fb3e00cbad0 con 0x7fb3e006dfc0 2023-10-08T05:50:17.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.815+0000 7fb3df7fe640 1 -- 172.21.15.70:0/235468526 >> v2:172.21.15.181:3300/0 conn(0x7fb3e00afbe0 msgr2=0x7fb3e00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:17.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.815+0000 7fb3df7fe640 1 --2- 172.21.15.70:0/235468526 >> v2:172.21.15.181:3300/0 conn(0x7fb3e00afbe0 0x7fb3e00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.815+0000 7fb3df7fe640 1 -- 172.21.15.70:0/235468526 >> v1:172.21.15.70:6789/0 conn(0x7fb3e006dfc0 legacy=0x7fb3e0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.815+0000 7fb3df7fe640 1 -- 172.21.15.70:0/235468526 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e00b2340 con 0x7fb3e00cb130 2023-10-08T05:50:17.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.816+0000 7fb3df7fe640 1 --2- 172.21.15.70:0/235468526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 0x7fb3e00cb590 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb3e006e640 tx=0x7fb3d802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ce5e459fe61ab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:17.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.816+0000 7fb3de7fc640 1 -- 172.21.15.70:0/235468526 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d802cb10 con 0x7fb3e00cb130 2023-10-08T05:50:17.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.816+0000 7fb3de7fc640 1 -- 172.21.15.70:0/235468526 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d802cc70 con 0x7fb3e00cb130 2023-10-08T05:50:17.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.816+0000 7fb3de7fc640 1 -- 172.21.15.70:0/235468526 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d80343f0 con 0x7fb3e00cb130 2023-10-08T05:50:17.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.817+0000 7fb3e5997640 1 -- 172.21.15.70:0/235468526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 msgr2=0x7fb3e00cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.817+0000 7fb3e5997640 1 --2- 172.21.15.70:0/235468526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 0x7fb3e00cb590 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb3e006e640 tx=0x7fb3d802bf70 comp rx=0 tx=0).stop 2023-10-08T05:50:17.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.817+0000 7fb3e5997640 1 -- 172.21.15.70:0/235468526 shutdown_connections 2023-10-08T05:50:17.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.817+0000 7fb3e5997640 1 --2- 172.21.15.70:0/235468526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 0x7fb3e00cb590 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.817+0000 7fb3e5997640 1 --2- 172.21.15.70:0/235468526 >> v2:172.21.15.181:3300/0 conn(0x7fb3e00afbe0 0x7fb3e00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.817+0000 7fb3e5997640 1 -- 172.21.15.70:0/235468526 >> 172.21.15.70:0/235468526 conn(0x7fb3e006a5d0 msgr2=0x7fb3e00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:17.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.817+0000 7fb3e5997640 1 -- 172.21.15.70:0/235468526 shutdown_connections 2023-10-08T05:50:17.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.817+0000 7fb3e5997640 1 -- 172.21.15.70:0/235468526 wait complete. 2023-10-08T05:50:17.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.819+0000 7fb3e5997640 1 Processor -- start 2023-10-08T05:50:17.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.819+0000 7fb3e5997640 1 -- start start 2023-10-08T05:50:17.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.820+0000 7fb3e5997640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e006dfc0 0x7fb3e00c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.820+0000 7fb3e5997640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e00afbe0 0x7fb3e00cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.820+0000 7fb3e5997640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 0x7fb3e0149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.820+0000 7fb3e5997640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00c3f00 con 0x7fb3e00cb130 2023-10-08T05:50:17.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.820+0000 7fb3e5997640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00cf5f0 con 0x7fb3e006dfc0 2023-10-08T05:50:17.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.820+0000 7fb3e5997640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e00ce010 con 0x7fb3e00afbe0 2023-10-08T05:50:17.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.820+0000 7fb3df7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 0x7fb3e0149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.820+0000 7fb3df7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 0x7fb3e0149de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43560/0 (socket says 172.21.15.70:43560) 2023-10-08T05:50:17.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.820+0000 7fb3df7fe640 1 -- 172.21.15.70:0/4174297863 learned_addr learned my addr 172.21.15.70:0/4174297863 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:17.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3deffd640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e006dfc0 0x7fb3e00c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3df7fe640 1 -- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e00afbe0 msgr2=0x7fb3e00cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3d7fff640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e00afbe0 0x7fb3e00cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3df7fe640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e00afbe0 0x7fb3e00cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3df7fe640 1 -- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e006dfc0 msgr2=0x7fb3e00c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:17.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3df7fe640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e006dfc0 0x7fb3e00c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:17.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3df7fe640 1 -- 172.21.15.70:0/4174297863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d8009b30 con 0x7fb3e00cb130 2023-10-08T05:50:17.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3deffd640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e006dfc0 0x7fb3e00c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:17.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3d7fff640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e00afbe0 0x7fb3e00cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:17.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3df7fe640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 0x7fb3e0149de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009f40 tx=0x7fb3d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:17.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3dcff9640 1 -- 172.21.15.70:0/4174297863 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8009510 con 0x7fb3e00cb130 2023-10-08T05:50:17.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3dcff9640 1 -- 172.21.15.70:0/4174297863 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d803e450 con 0x7fb3e00cb130 2023-10-08T05:50:17.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3e5997640 1 -- 172.21.15.70:0/4174297863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e00ce2f0 con 0x7fb3e00cb130 2023-10-08T05:50:17.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.821+0000 7fb3dcff9640 1 -- 172.21.15.70:0/4174297863 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d803e7e0 con 0x7fb3e00cb130 2023-10-08T05:50:17.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.822+0000 7fb3e5997640 1 -- 172.21.15.70:0/4174297863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e00ce8b0 con 0x7fb3e00cb130 2023-10-08T05:50:17.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.822+0000 7fb3e5997640 1 -- 172.21.15.70:0/4174297863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e004e160 con 0x7fb3e00cb130 2023-10-08T05:50:17.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.823+0000 7fb3dcff9640 1 -- 172.21.15.70:0/4174297863 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb3d803e940 con 0x7fb3e00cb130 2023-10-08T05:50:17.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.824+0000 7fb3dcff9640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fb3c006af90 0x7fb3c006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:17.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.824+0000 7fb3dcff9640 1 -- 172.21.15.70:0/4174297863 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb3d80acac0 con 0x7fb3e00cb130 2023-10-08T05:50:17.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.825+0000 7fb3deffd640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fb3c006af90 0x7fb3c006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:17.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.826+0000 7fb3deffd640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fb3c006af90 0x7fb3c006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0006fd0 tx=0x7fb3d0008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:17.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:17.828+0000 7fb3dcff9640 1 -- 172.21.15.70:0/4174297863 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3d8077850 con 0x7fb3e00cb130 2023-10-08T05:50:18.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.061+0000 7fb3e5997640 1 -- 172.21.15.70:0/4174297863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb3e00b0ef0 con 0x7fb3e00cb130 2023-10-08T05:50:18.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.705+0000 7fb3dcff9640 1 -- 172.21.15.70:0/4174297863 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb3d8033e80 con 0x7fb3e00cb130 2023-10-08T05:50:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.708+0000 7fb3e5997640 1 -- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fb3c006af90 msgr2=0x7fb3c006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.708+0000 7fb3e5997640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fb3c006af90 0x7fb3c006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0006fd0 tx=0x7fb3d0008040 comp rx=0 tx=0).stop 2023-10-08T05:50:18.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.708+0000 7fb3e5997640 1 -- 172.21.15.70:0/4174297863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 msgr2=0x7fb3e0149de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:18.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.708+0000 7fb3e5997640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 0x7fb3e0149de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009f40 tx=0x7fb3d8004a40 comp rx=0 tx=0).stop 2023-10-08T05:50:18.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.708+0000 7fb3e5997640 1 -- 172.21.15.70:0/4174297863 shutdown_connections 2023-10-08T05:50:18.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.708+0000 7fb3e5997640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb3e00cb130 0x7fb3e0149de0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:18.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.708+0000 7fb3e5997640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fb3c006af90 0x7fb3c006d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:18.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.708+0000 7fb3e5997640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3e00afbe0 0x7fb3e00cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:18.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.708+0000 7fb3e5997640 1 --2- 172.21.15.70:0/4174297863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb3e006dfc0 0x7fb3e00c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:18.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.708+0000 7fb3e5997640 1 -- 172.21.15.70:0/4174297863 >> 172.21.15.70:0/4174297863 conn(0x7fb3e006a5d0 msgr2=0x7fb3e00b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:18.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.709+0000 7fb3e5997640 1 -- 172.21.15.70:0/4174297863 shutdown_connections 2023-10-08T05:50:18.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.709+0000 7fb3e5997640 1 -- 172.21.15.70:0/4174297863 wait complete. 2023-10-08T05:50:18.730 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T05:50:18.730 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T05:50:18.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.949+0000 7f4fa0197640 1 Processor -- start 2023-10-08T05:50:18.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.950+0000 7f4fa0197640 1 -- start start 2023-10-08T05:50:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.951+0000 7f4fa0197640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f9806dfc0 0x7f4f98070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.951+0000 7f4fa0197640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f98071070 0x7f4f980cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.951+0000 7f4fa0197640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f980b1cc0 con 0x7f4f980afbe0 2023-10-08T05:50:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.951+0000 7f4fa0197640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f980b2620 con 0x7f4f98071070 2023-10-08T05:50:18.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.951+0000 7f4fa0197640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f980b2060 con 0x7f4f9806dfc0 2023-10-08T05:50:18.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.951+0000 7f4f9e734640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f98071070 0x7f4f980cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:18.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.952+0000 7f4f9e734640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f98071070 0x7f4f980cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43564/0 (socket says 172.21.15.70:43564) 2023-10-08T05:50:18.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.952+0000 7f4f9e734640 1 -- 172.21.15.70:0/3359863281 learned_addr learned my addr 172.21.15.70:0/3359863281 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:18.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.952+0000 7f4f9df33640 1 --1- 172.21.15.70:0/3359863281 >> v1:172.21.15.70:6789/0 conn(0x7f4f980afbe0 0x7f4f980b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52468/0 (socket says 172.21.15.70:52468) 2023-10-08T05:50:18.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.952+0000 7f4f9d732640 1 --2- 172.21.15.70:0/3359863281 >> v2:172.21.15.181:3300/0 conn(0x7f4f9806dfc0 0x7f4f98070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:18.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.952+0000 7f4f9e734640 1 -- 172.21.15.70:0/3359863281 >> v2:172.21.15.181:3300/0 conn(0x7f4f9806dfc0 msgr2=0x7f4f98070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:18.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.952+0000 7f4f9e734640 1 --2- 172.21.15.70:0/3359863281 >> v2:172.21.15.181:3300/0 conn(0x7f4f9806dfc0 0x7f4f98070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:18.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.952+0000 7f4f9e734640 1 -- 172.21.15.70:0/3359863281 >> v1:172.21.15.70:6789/0 conn(0x7f4f980afbe0 legacy=0x7f4f980b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:18.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.952+0000 7f4f9e734640 1 -- 172.21.15.70:0/3359863281 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980b2340 con 0x7f4f98071070 2023-10-08T05:50:18.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.952+0000 7f4f9cf31640 1 -- 172.21.15.70:0/3359863281 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837820806 0 0) 0x7f4f980b1cc0 con 0x7f4f980afbe0 2023-10-08T05:50:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.953+0000 7f4f9e734640 1 --2- 172.21.15.70:0/3359863281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f98071070 0x7f4f980cb130 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4f980b3980 tx=0x7f4f9002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c37a91683aee0b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.954+0000 7f4f9cf31640 1 -- 172.21.15.70:0/3359863281 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9002cb20 con 0x7f4f98071070 2023-10-08T05:50:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.954+0000 7f4f9cf31640 1 -- 172.21.15.70:0/3359863281 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9002cc80 con 0x7f4f98071070 2023-10-08T05:50:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.954+0000 7f4f9cf31640 1 -- 172.21.15.70:0/3359863281 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f900343f0 con 0x7f4f98071070 2023-10-08T05:50:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.954+0000 7f4fa0197640 1 -- 172.21.15.70:0/3359863281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f98071070 msgr2=0x7f4f980cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.954+0000 7f4fa0197640 1 --2- 172.21.15.70:0/3359863281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f98071070 0x7f4f980cb130 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4f980b3980 tx=0x7f4f9002bf80 comp rx=0 tx=0).stop 2023-10-08T05:50:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.954+0000 7f4fa0197640 1 -- 172.21.15.70:0/3359863281 shutdown_connections 2023-10-08T05:50:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.954+0000 7f4fa0197640 1 --2- 172.21.15.70:0/3359863281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f98071070 0x7f4f980cb130 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.954+0000 7f4fa0197640 1 --2- 172.21.15.70:0/3359863281 >> v2:172.21.15.181:3300/0 conn(0x7f4f9806dfc0 0x7f4f98070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.954+0000 7f4fa0197640 1 -- 172.21.15.70:0/3359863281 >> 172.21.15.70:0/3359863281 conn(0x7f4f9806a5d0 msgr2=0x7f4f980b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.955+0000 7f4fa0197640 1 -- 172.21.15.70:0/3359863281 shutdown_connections 2023-10-08T05:50:18.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.955+0000 7f4fa0197640 1 -- 172.21.15.70:0/3359863281 wait complete. 2023-10-08T05:50:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.957+0000 7f4fa0197640 1 Processor -- start 2023-10-08T05:50:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.957+0000 7f4fa0197640 1 -- start start 2023-10-08T05:50:18.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.958+0000 7f4fa0197640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9806dfc0 0x7f4f980aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:18.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.958+0000 7f4fa0197640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f98071070 0x7f4f980c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:18.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.958+0000 7f4fa0197640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f980afbe0 0x7f4f980c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:18.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.958+0000 7f4fa0197640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f981f23e0 con 0x7f4f9806dfc0 2023-10-08T05:50:18.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.958+0000 7f4fa0197640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980cad70 con 0x7f4f980afbe0 2023-10-08T05:50:18.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.958+0000 7f4fa0197640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980ca4d0 con 0x7f4f98071070 2023-10-08T05:50:18.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.958+0000 7f4f9df33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9806dfc0 0x7f4f980aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:18.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f9df33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9806dfc0 0x7f4f980aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43580/0 (socket says 172.21.15.70:43580) 2023-10-08T05:50:18.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f9df33640 1 -- 172.21.15.70:0/3087296649 learned_addr learned my addr 172.21.15.70:0/3087296649 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:18.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f9df33640 1 -- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f98071070 msgr2=0x7f4f980c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:18.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f9df33640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f98071070 0x7f4f980c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:18.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f9df33640 1 -- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f980afbe0 msgr2=0x7f4f980c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:18.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f9df33640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f980afbe0 0x7f4f980c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:18.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f9df33640 1 -- 172.21.15.70:0/3087296649 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f90009b30 con 0x7f4f9806dfc0 2023-10-08T05:50:18.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f9df33640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9806dfc0 0x7f4f980aeda0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400c430 tx=0x7f4f9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:18.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f8e7fc640 1 -- 172.21.15.70:0/3087296649 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f940074e0 con 0x7f4f9806dfc0 2023-10-08T05:50:18.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f8e7fc640 1 -- 172.21.15.70:0/3087296649 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f94007660 con 0x7f4f9806dfc0 2023-10-08T05:50:18.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.959+0000 7f4f8e7fc640 1 -- 172.21.15.70:0/3087296649 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94004a20 con 0x7f4f9806dfc0 2023-10-08T05:50:18.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.960+0000 7f4fa0197640 1 -- 172.21.15.70:0/3087296649 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f980ca7b0 con 0x7f4f9806dfc0 2023-10-08T05:50:18.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.960+0000 7f4fa0197640 1 -- 172.21.15.70:0/3087296649 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f980c22f0 con 0x7f4f9806dfc0 2023-10-08T05:50:18.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.961+0000 7f4fa0197640 1 -- 172.21.15.70:0/3087296649 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f980c2510 con 0x7f4f9806dfc0 2023-10-08T05:50:18.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.962+0000 7f4f8e7fc640 1 -- 172.21.15.70:0/3087296649 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4f94004b80 con 0x7f4f9806dfc0 2023-10-08T05:50:18.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.963+0000 7f4f8e7fc640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f4f7c06af90 0x7f4f7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:18.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.963+0000 7f4f8e7fc640 1 -- 172.21.15.70:0/3087296649 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4f94089720 con 0x7f4f9806dfc0 2023-10-08T05:50:18.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.963+0000 7f4f9d732640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f4f7c06af90 0x7f4f7c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:18.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.964+0000 7f4f9d732640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f4f7c06af90 0x7f4f7c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4f980c5470 tx=0x7f4f88008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:18.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:18.967+0000 7f4f8e7fc640 1 -- 172.21.15.70:0/3087296649 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f94054540 con 0x7f4f9806dfc0 2023-10-08T05:50:19.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.192+0000 7f4fa0197640 1 -- 172.21.15.70:0/3087296649 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4f9804e160 con 0x7f4f9806dfc0 2023-10-08T05:50:19.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.716+0000 7f4f8e7fc640 1 -- 172.21.15.70:0/3087296649 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4f94087e60 con 0x7f4f9806dfc0 2023-10-08T05:50:19.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 -- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f4f7c06af90 msgr2=0x7f4f7c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:19.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f4f7c06af90 0x7f4f7c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4f980c5470 tx=0x7f4f88008040 comp rx=0 tx=0).stop 2023-10-08T05:50:19.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 -- 172.21.15.70:0/3087296649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9806dfc0 msgr2=0x7f4f980aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:19.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9806dfc0 0x7f4f980aeda0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400c430 tx=0x7f4f9400c8f0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 -- 172.21.15.70:0/3087296649 shutdown_connections 2023-10-08T05:50:19.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f980afbe0 0x7f4f980c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f4f7c06af90 0x7f4f7c06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f98071070 0x7f4f980c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 --2- 172.21.15.70:0/3087296649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9806dfc0 0x7f4f980aeda0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 -- 172.21.15.70:0/3087296649 >> 172.21.15.70:0/3087296649 conn(0x7f4f9806a5d0 msgr2=0x7f4f980c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:19.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.720+0000 7f4fa0197640 1 -- 172.21.15.70:0/3087296649 shutdown_connections 2023-10-08T05:50:19.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.721+0000 7f4fa0197640 1 -- 172.21.15.70:0/3087296649 wait complete. 2023-10-08T05:50:19.741 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T05:50:19.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:50:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.943+0000 7f3332383640 1 Processor -- start 2023-10-08T05:50:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.943+0000 7f3332383640 1 -- start start 2023-10-08T05:50:19.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.945+0000 7f3332383640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f332c160940 0x7f332c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:19.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.945+0000 7f3332383640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c0c48f0 con 0x7f332c06c890 2023-10-08T05:50:19.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.945+0000 7f3332383640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c0c3ce0 con 0x7f332c15cd90 2023-10-08T05:50:19.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.945+0000 7f3332383640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332c0c4d80 con 0x7f332c160940 2023-10-08T05:50:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.945+0000 7f3330920640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f332c06c890 0x7f332c06cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52484/0 (socket says 172.21.15.70:52484) 2023-10-08T05:50:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.945+0000 7f3330920640 1 -- 172.21.15.70:0/702823370 learned_addr learned my addr 172.21.15.70:0/702823370 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.945+0000 7f3323fff640 1 --2- 172.21.15.70:0/702823370 >> v2:172.21.15.181:3300/0 conn(0x7f332c160940 0x7f332c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363917358 0 0) 0x7f332c0c48f0 con 0x7f332c06c890 2023-10-08T05:50:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3318003520 con 0x7f332c06c890 2023-10-08T05:50:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 229157370 0 0) 0x7f3318003520 con 0x7f332c06c890 2023-10-08T05:50:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 >> v2:172.21.15.181:3300/0 conn(0x7f332c160940 msgr2=0x7f332c0709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 --2- 172.21.15.70:0/702823370 >> v2:172.21.15.181:3300/0 conn(0x7f332c160940 0x7f332c0709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 >> v1:172.21.15.181:6789/0 conn(0x7f332c15cd90 legacy=0x7f332c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c0c3fc0 con 0x7f332c06c890 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f33240019f0 con 0x7f332c06c890 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 >> v1:172.21.15.70:6789/0 conn(0x7f332c06c890 legacy=0x7f332c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33180039c0 0x7f3318003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33180047c0 0x7f3318004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f331800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331800c050 con 0x7f3318005180 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331800c1c0 con 0x7f33180047c0 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.946+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331800c380 con 0x7f33180039c0 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.947+0000 7f3330920640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f331800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.947+0000 7f332bfff640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33180039c0 0x7f3318003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.947+0000 7f3330920640 1 -- 172.21.15.70:0/702823370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33180039c0 msgr2=0x7f3318003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.947+0000 7f3330920640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33180039c0 0x7f3318003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.947+0000 7f3330920640 1 -- 172.21.15.70:0/702823370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33180047c0 msgr2=0x7f3318004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.947+0000 7f3323fff640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33180047c0 0x7f3318004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.947+0000 7f3330920640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33180047c0 0x7f3318004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.947+0000 7f3330920640 1 -- 172.21.15.70:0/702823370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318003520 con 0x7f3318005180 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.948+0000 7f3330920640 1 -- 172.21.15.70:0/702823370 reap_dead start 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.949+0000 7f3330920640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f331800b980 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f332400a760 tx=0x7f33180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb5e602e14a7228 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.949+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324034630 con 0x7f3318005180 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.949+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332402f070 con 0x7f3318005180 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.949+0000 7f332b7fe640 1 -- 172.21.15.70:0/702823370 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324034a10 con 0x7f3318005180 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.950+0000 7f3332383640 1 -- 172.21.15.70:0/702823370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 msgr2=0x7f331800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.950+0000 7f3332383640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f331800b980 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f332400a760 tx=0x7f33180057f0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.950+0000 7f3332383640 1 -- 172.21.15.70:0/702823370 shutdown_connections 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.950+0000 7f3332383640 1 --2- 172.21.15.70:0/702823370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f331800b980 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.950+0000 7f3332383640 1 -- 172.21.15.70:0/702823370 >> 172.21.15.70:0/702823370 conn(0x7f332c06a380 msgr2=0x7f332c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.950+0000 7f3332383640 1 -- 172.21.15.70:0/702823370 shutdown_connections 2023-10-08T05:50:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.950+0000 7f3332383640 1 -- 172.21.15.70:0/702823370 wait complete. 2023-10-08T05:50:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.952+0000 7f3332383640 1 Processor -- start 2023-10-08T05:50:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.952+0000 7f3332383640 1 -- start start 2023-10-08T05:50:19.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f3332383640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f332c1503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f3332383640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f332c0cb9b0 0x7f332c0b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f3332383640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f332c0ca430 0x7f332c1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f3332383640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c0b2060 con 0x7f3318005180 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f3332383640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c0cb520 con 0x7f332c0cb9b0 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f3332383640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c150900 con 0x7f332c0ca430 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f332bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f332c1503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f332bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f332c1503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43598/0 (socket says 172.21.15.70:43598) 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f332bfff640 1 -- 172.21.15.70:0/2436537422 learned_addr learned my addr 172.21.15.70:0/2436537422 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f332bfff640 1 -- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f332c0ca430 msgr2=0x7f332c1550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f332bfff640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f332c0ca430 0x7f332c1550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f332bfff640 1 -- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f332c0cb9b0 msgr2=0x7f332c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:19.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f332bfff640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f332c0cb9b0 0x7f332c0b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:19.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.953+0000 7f332bfff640 1 -- 172.21.15.70:0/2436537422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c0b1aa0 con 0x7f3318005180 2023-10-08T05:50:19.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.954+0000 7f332bfff640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f332c1503c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f331c00e410 tx=0x7f331c00e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:19.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.954+0000 7f33297fa640 1 -- 172.21.15.70:0/2436537422 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c0052e0 con 0x7f3318005180 2023-10-08T05:50:19.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.954+0000 7f33297fa640 1 -- 172.21.15.70:0/2436537422 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331c005440 con 0x7f3318005180 2023-10-08T05:50:19.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.954+0000 7f3332383640 1 -- 172.21.15.70:0/2436537422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c0ca910 con 0x7f3318005180 2023-10-08T05:50:19.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.954+0000 7f33297fa640 1 -- 172.21.15.70:0/2436537422 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c003810 con 0x7f3318005180 2023-10-08T05:50:19.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.954+0000 7f3332383640 1 -- 172.21.15.70:0/2436537422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c0caed0 con 0x7f3318005180 2023-10-08T05:50:19.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.955+0000 7f3332383640 1 -- 172.21.15.70:0/2436537422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c067490 con 0x7f3318005180 2023-10-08T05:50:19.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.956+0000 7f33297fa640 1 -- 172.21.15.70:0/2436537422 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f331c003970 con 0x7f3318005180 2023-10-08T05:50:19.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.957+0000 7f33297fa640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f330c06af90 0x7f330c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:19.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.957+0000 7f33297fa640 1 -- 172.21.15.70:0/2436537422 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f331c089100 con 0x7f3318005180 2023-10-08T05:50:19.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.958+0000 7f3323fff640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f330c06af90 0x7f330c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:19.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.959+0000 7f3323fff640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f330c06af90 0x7f330c06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f332c0cbdb0 tx=0x7f3314008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:19.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:19.962+0000 7f33297fa640 1 -- 172.21.15.70:0/2436537422 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f331c053f50 con 0x7f3318005180 2023-10-08T05:50:20.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.174+0000 7f3332383640 1 -- 172.21.15.70:0/2436537422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f332c158020 con 0x7f3318005180 2023-10-08T05:50:20.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.175+0000 7f33297fa640 1 -- 172.21.15.70:0/2436537422 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f331c019090 con 0x7f3318005180 2023-10-08T05:50:20.177 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:20.177 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1126961048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1126961048},{"type":"v1","addr":"172.21.15.181:6836","nonce":1126961048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3993068621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3993068621},{"type":"v1","addr":"172.21.15.181:6837","nonce":3993068621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/28904251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":28904251},{"type":"v1","addr":"172.21.15.70:6833","nonce":28904251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:50:16.144118+0000","modified":"2023-10-08T05:50:19.194284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1337581673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1337581673},{"type":"v1","addr":"172.21.15.70:6835","nonce":1337581673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-08T05:50:20.177 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 25 2023-10-08T05:50:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 -- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f330c06af90 msgr2=0x7f330c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f330c06af90 0x7f330c06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f332c0cbdb0 tx=0x7f3314008040 comp rx=0 tx=0).stop 2023-10-08T05:50:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 -- 172.21.15.70:0/2436537422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 msgr2=0x7f332c1503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f332c1503c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f331c00e410 tx=0x7f331c00e8d0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 -- 172.21.15.70:0/2436537422 shutdown_connections 2023-10-08T05:50:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f330c06af90 0x7f330c06d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f332c0ca430 0x7f332c1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f332c0cb9b0 0x7f332c0b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 --2- 172.21.15.70:0/2436537422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3318005180 0x7f332c1503c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 -- 172.21.15.70:0/2436537422 >> 172.21.15.70:0/2436537422 conn(0x7f332c06a380 msgr2=0x7f332c160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 -- 172.21.15.70:0/2436537422 shutdown_connections 2023-10-08T05:50:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.178+0000 7f3332383640 1 -- 172.21.15.70:0/2436537422 wait complete. 2023-10-08T05:50:20.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:50:20.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.402+0000 7ff345b8a640 1 Processor -- start 2023-10-08T05:50:20.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.402+0000 7ff345b8a640 1 -- start start 2023-10-08T05:50:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff345b8a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff340159ac0 0x7ff34006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff345b8a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3400ad790 0x7ff3400702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff345b8a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34006dee0 con 0x7ff34015e330 2023-10-08T05:50:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff345b8a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34015d850 con 0x7ff3400ad790 2023-10-08T05:50:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff345b8a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34015db30 con 0x7ff340159ac0 2023-10-08T05:50:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff33effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3400ad790 0x7ff3400702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff33effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3400ad790 0x7ff3400702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43602/0 (socket says 172.21.15.70:43602) 2023-10-08T05:50:20.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff33effd640 1 -- 172.21.15.70:0/2912125307 learned_addr learned my addr 172.21.15.70:0/2912125307 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:20.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff33f7fe640 1 --2- 172.21.15.70:0/2912125307 >> v2:172.21.15.181:3300/0 conn(0x7ff340159ac0 0x7ff34006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:20.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff33effd640 1 -- 172.21.15.70:0/2912125307 >> v2:172.21.15.181:3300/0 conn(0x7ff340159ac0 msgr2=0x7ff34006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:20.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff33effd640 1 --2- 172.21.15.70:0/2912125307 >> v2:172.21.15.181:3300/0 conn(0x7ff340159ac0 0x7ff34006d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff33effd640 1 -- 172.21.15.70:0/2912125307 >> v1:172.21.15.181:6789/0 conn(0x7ff34015e330 legacy=0x7ff34006c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:20.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff33effd640 1 -- 172.21.15.70:0/2912125307 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34015de10 con 0x7ff3400ad790 2023-10-08T05:50:20.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.404+0000 7ff33f7fe640 1 --2- 172.21.15.70:0/2912125307 >> v2:172.21.15.181:3300/0 conn(0x7ff340159ac0 0x7ff34006d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:20.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.406+0000 7ff33effd640 1 --2- 172.21.15.70:0/2912125307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 0x7ff3400702d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff34006e790 tx=0x7ff33402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e3fe47d708ba01 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.406+0000 7ff33e7fc640 1 -- 172.21.15.70:0/2912125307 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334031890 con 0x7ff3400ad790 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.406+0000 7ff33e7fc640 1 -- 172.21.15.70:0/2912125307 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3340319f0 con 0x7ff3400ad790 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.406+0000 7ff33e7fc640 1 -- 172.21.15.70:0/2912125307 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334031d80 con 0x7ff3400ad790 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.407+0000 7ff345b8a640 1 -- 172.21.15.70:0/2912125307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 msgr2=0x7ff3400702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.407+0000 7ff345b8a640 1 --2- 172.21.15.70:0/2912125307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 0x7ff3400702d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff34006e790 tx=0x7ff33402b930 comp rx=0 tx=0).stop 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.407+0000 7ff345b8a640 1 -- 172.21.15.70:0/2912125307 shutdown_connections 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.407+0000 7ff345b8a640 1 --2- 172.21.15.70:0/2912125307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 0x7ff3400702d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.407+0000 7ff345b8a640 1 --2- 172.21.15.70:0/2912125307 >> v2:172.21.15.181:3300/0 conn(0x7ff340159ac0 0x7ff34006d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.407+0000 7ff345b8a640 1 -- 172.21.15.70:0/2912125307 >> 172.21.15.70:0/2912125307 conn(0x7ff34006a380 msgr2=0x7ff3400b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.407+0000 7ff345b8a640 1 -- 172.21.15.70:0/2912125307 shutdown_connections 2023-10-08T05:50:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.407+0000 7ff345b8a640 1 -- 172.21.15.70:0/2912125307 wait complete. 2023-10-08T05:50:20.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.409+0000 7ff345b8a640 1 Processor -- start 2023-10-08T05:50:20.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.409+0000 7ff345b8a640 1 -- start start 2023-10-08T05:50:20.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.410+0000 7ff345b8a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 0x7ff34014cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.410+0000 7ff345b8a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff340159ac0 0x7ff3400b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.410+0000 7ff345b8a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff34015e330 0x7ff3400b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.410+0000 7ff345b8a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401f6910 con 0x7ff3400ad790 2023-10-08T05:50:20.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.410+0000 7ff345b8a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401f2260 con 0x7ff340159ac0 2023-10-08T05:50:20.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.410+0000 7ff345b8a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3401f5d00 con 0x7ff34015e330 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 0x7ff34014cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 0x7ff34014cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43610/0 (socket says 172.21.15.70:43610) 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33f7fe640 1 -- 172.21.15.70:0/3390463929 learned_addr learned my addr 172.21.15.70:0/3390463929 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33effd640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff340159ac0 0x7ff3400b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33f7fe640 1 -- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff34015e330 msgr2=0x7ff3400b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33ffff640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff34015e330 0x7ff3400b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33f7fe640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff34015e330 0x7ff3400b22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33f7fe640 1 -- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff340159ac0 msgr2=0x7ff3400b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33f7fe640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff340159ac0 0x7ff3400b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33f7fe640 1 -- 172.21.15.70:0/3390463929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3340094d0 con 0x7ff3400ad790 2023-10-08T05:50:20.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33effd640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff340159ac0 0x7ff3400b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.411+0000 7ff33f7fe640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 0x7ff34014cc10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c7f0 tx=0x7ff33000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.412+0000 7ff327fff640 1 -- 172.21.15.70:0/3390463929 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330007a30 con 0x7ff3400ad790 2023-10-08T05:50:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.412+0000 7ff327fff640 1 -- 172.21.15.70:0/3390463929 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330007b90 con 0x7ff3400ad790 2023-10-08T05:50:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.412+0000 7ff327fff640 1 -- 172.21.15.70:0/3390463929 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33000ce40 con 0x7ff3400ad790 2023-10-08T05:50:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.412+0000 7ff345b8a640 1 -- 172.21.15.70:0/3390463929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3401f5fe0 con 0x7ff3400ad790 2023-10-08T05:50:20.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.412+0000 7ff345b8a640 1 -- 172.21.15.70:0/3390463929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3401f65a0 con 0x7ff3400ad790 2023-10-08T05:50:20.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.413+0000 7ff345b8a640 1 -- 172.21.15.70:0/3390463929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff340067490 con 0x7ff3400ad790 2023-10-08T05:50:20.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.414+0000 7ff327fff640 1 -- 172.21.15.70:0/3390463929 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff3300113f0 con 0x7ff3400ad790 2023-10-08T05:50:20.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.415+0000 7ff327fff640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff32c06afc0 0x7ff32c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.415+0000 7ff327fff640 1 -- 172.21.15.70:0/3390463929 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff330089c00 con 0x7ff3400ad790 2023-10-08T05:50:20.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.416+0000 7ff33effd640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff32c06afc0 0x7ff32c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:20.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.417+0000 7ff33effd640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff32c06afc0 0x7ff32c06d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff334036000 tx=0x7ff33403c040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:20.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.419+0000 7ff327fff640 1 -- 172.21.15.70:0/3390463929 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff330054880 con 0x7ff3400ad790 2023-10-08T05:50:20.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.609+0000 7ff345b8a640 1 -- 172.21.15.70:0/3390463929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3400b0ff0 con 0x7ff3400ad790 2023-10-08T05:50:20.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.612+0000 7ff327fff640 1 -- 172.21.15.70:0/3390463929 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16832 (secure 0 0 0) 0x7ff33001f090 con 0x7ff3400ad790 2023-10-08T05:50:20.614 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:20.614 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":72,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:50:17.691814+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:50:14.005619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-08T05:50:15.163925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:50:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.614+0000 7ff345b8a640 1 -- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff32c06afc0 msgr2=0x7ff32c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.614+0000 7ff345b8a640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff32c06afc0 0x7ff32c06d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff334036000 tx=0x7ff33403c040 comp rx=0 tx=0).stop 2023-10-08T05:50:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.614+0000 7ff345b8a640 1 -- 172.21.15.70:0/3390463929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 msgr2=0x7ff34014cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.614+0000 7ff345b8a640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 0x7ff34014cc10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c7f0 tx=0x7ff33000cbf0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.614+0000 7ff345b8a640 1 -- 172.21.15.70:0/3390463929 shutdown_connections 2023-10-08T05:50:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.614+0000 7ff345b8a640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ff32c06afc0 0x7ff32c06d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.615+0000 7ff345b8a640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff34015e330 0x7ff3400b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.615+0000 7ff345b8a640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff340159ac0 0x7ff3400b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.615+0000 7ff345b8a640 1 --2- 172.21.15.70:0/3390463929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3400ad790 0x7ff34014cc10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.615+0000 7ff345b8a640 1 -- 172.21.15.70:0/3390463929 >> 172.21.15.70:0/3390463929 conn(0x7ff34006a380 msgr2=0x7ff3400af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:20.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.615+0000 7ff345b8a640 1 -- 172.21.15.70:0/3390463929 shutdown_connections 2023-10-08T05:50:20.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.615+0000 7ff345b8a640 1 -- 172.21.15.70:0/3390463929 wait complete. 2023-10-08T05:50:20.635 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T05:50:20.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:50:20.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.853+0000 7fe98f87c640 1 Processor -- start 2023-10-08T05:50:20.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.854+0000 7fe98f87c640 1 -- start start 2023-10-08T05:50:20.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.855+0000 7fe98f87c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe98815d4e0 0x7fe98806b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.855+0000 7fe98f87c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9881596c0 con 0x7fe9880cc180 2023-10-08T05:50:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.855+0000 7fe98f87c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe98815db00 con 0x7fe9881592e0 2023-10-08T05:50:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.855+0000 7fe98f87c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98815dde0 con 0x7fe98815d4e0 2023-10-08T05:50:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.855+0000 7fe98de19640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe98815d4e0 0x7fe98806b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.855+0000 7fe98de19640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe98815d4e0 0x7fe98806b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43626/0 (socket says 172.21.15.70:43626) 2023-10-08T05:50:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.855+0000 7fe98de19640 1 -- 172.21.15.70:0/855868806 learned_addr learned my addr 172.21.15.70:0/855868806 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.856+0000 7fe98de19640 1 -- 172.21.15.70:0/855868806 >> v1:172.21.15.181:6789/0 conn(0x7fe9881592e0 legacy=0x7fe9880cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:20.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.856+0000 7fe98de19640 1 -- 172.21.15.70:0/855868806 >> v1:172.21.15.70:6789/0 conn(0x7fe9880cc180 legacy=0x7fe98815cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.856+0000 7fe98de19640 1 -- 172.21.15.70:0/855868806 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98806a7a0 con 0x7fe98815d4e0 2023-10-08T05:50:20.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.857+0000 7fe98de19640 1 --2- 172.21.15.70:0/855868806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe98815d4e0 0x7fe98806b280 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe980009e50 tx=0x7fe98002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95410bf3ededb52c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:20.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.857+0000 7fe987fff640 1 -- 172.21.15.70:0/855868806 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98002c9f0 con 0x7fe98815d4e0 2023-10-08T05:50:20.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.857+0000 7fe987fff640 1 -- 172.21.15.70:0/855868806 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98002cb50 con 0x7fe98815d4e0 2023-10-08T05:50:20.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.857+0000 7fe987fff640 1 -- 172.21.15.70:0/855868806 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9800343f0 con 0x7fe98815d4e0 2023-10-08T05:50:20.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.858+0000 7fe98f87c640 1 -- 172.21.15.70:0/855868806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe98815d4e0 msgr2=0x7fe98806b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.858+0000 7fe98f87c640 1 --2- 172.21.15.70:0/855868806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe98815d4e0 0x7fe98806b280 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe980009e50 tx=0x7fe98002be50 comp rx=0 tx=0).stop 2023-10-08T05:50:20.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.858+0000 7fe98f87c640 1 -- 172.21.15.70:0/855868806 shutdown_connections 2023-10-08T05:50:20.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.858+0000 7fe98f87c640 1 --2- 172.21.15.70:0/855868806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe98815d4e0 0x7fe98806b280 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.858+0000 7fe98f87c640 1 -- 172.21.15.70:0/855868806 >> 172.21.15.70:0/855868806 conn(0x7fe9880b1480 msgr2=0x7fe98806cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:20.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.858+0000 7fe98f87c640 1 -- 172.21.15.70:0/855868806 shutdown_connections 2023-10-08T05:50:20.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.858+0000 7fe98f87c640 1 -- 172.21.15.70:0/855868806 wait complete. 2023-10-08T05:50:20.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.860+0000 7fe98f87c640 1 Processor -- start 2023-10-08T05:50:20.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.860+0000 7fe98f87c640 1 -- start start 2023-10-08T05:50:20.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.861+0000 7fe98f87c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9880cc180 0x7fe9880c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.861+0000 7fe98f87c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9881592e0 0x7fe9880c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.861+0000 7fe98f87c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe98815d4e0 0x7fe988158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.861+0000 7fe98f87c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98806ae60 con 0x7fe9880cc180 2023-10-08T05:50:20.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.861+0000 7fe98f87c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98815b400 con 0x7fe9881592e0 2023-10-08T05:50:20.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.861+0000 7fe98f87c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98815bf70 con 0x7fe98815d4e0 2023-10-08T05:50:20.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.861+0000 7fe98d618640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9880cc180 0x7fe9880c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:20.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.861+0000 7fe98d618640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9880cc180 0x7fe9880c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43630/0 (socket says 172.21.15.70:43630) 2023-10-08T05:50:20.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.861+0000 7fe98d618640 1 -- 172.21.15.70:0/817496461 learned_addr learned my addr 172.21.15.70:0/817496461 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:20.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe98d618640 1 -- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe98815d4e0 msgr2=0x7fe988158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe98d618640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe98815d4e0 0x7fe988158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe98d618640 1 -- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9881592e0 msgr2=0x7fe9880c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:20.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe98d618640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9881592e0 0x7fe9880c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:20.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe98d618640 1 -- 172.21.15.70:0/817496461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe980009b30 con 0x7fe9880cc180 2023-10-08T05:50:20.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe98d618640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9880cc180 0x7fe9880c89b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe97800c460 tx=0x7fe97800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:20.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe985ffb640 1 -- 172.21.15.70:0/817496461 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978011790 con 0x7fe9880cc180 2023-10-08T05:50:20.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe985ffb640 1 -- 172.21.15.70:0/817496461 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978011930 con 0x7fe9880cc180 2023-10-08T05:50:20.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe985ffb640 1 -- 172.21.15.70:0/817496461 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978011ce0 con 0x7fe9880cc180 2023-10-08T05:50:20.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe98f87c640 1 -- 172.21.15.70:0/817496461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98815c400 con 0x7fe9880cc180 2023-10-08T05:50:20.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.862+0000 7fe98f87c640 1 -- 172.21.15.70:0/817496461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98815b920 con 0x7fe9880cc180 2023-10-08T05:50:20.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.863+0000 7fe98f87c640 1 -- 172.21.15.70:0/817496461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9880cd3b0 con 0x7fe9880cc180 2023-10-08T05:50:20.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.864+0000 7fe985ffb640 1 -- 172.21.15.70:0/817496461 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe97801c460 con 0x7fe9880cc180 2023-10-08T05:50:20.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.866+0000 7fe985ffb640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe97406af90 0x7fe97406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:20.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.866+0000 7fe985ffb640 1 -- 172.21.15.70:0/817496461 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe978089880 con 0x7fe9880cc180 2023-10-08T05:50:20.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.867+0000 7fe98ce17640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe97406af90 0x7fe97406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:20.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.868+0000 7fe98ce17640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe97406af90 0x7fe97406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9881f5b90 tx=0x7fe97c007950 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:20.871+0000 7fe985ffb640 1 -- 172.21.15.70:0/817496461 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9780546a0 con 0x7fe9880cc180 2023-10-08T05:50:21.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.058+0000 7fe98f87c640 1 -- 172.21.15.70:0/817496461 --> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe98806aa80 con 0x7fe97406af90 2023-10-08T05:50:21.060 INFO:tasks.ceph.mds.a.smithi070.stderr:2023-10-08T05:50:21.059+0000 7f3cb6e5d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:50:21.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.084+0000 7fe985ffb640 1 -- 172.21.15.70:0/817496461 <== mgr.4844 v2:172.21.15.181:6832/992174303 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286424 (secure 0 0 0) 0x7fe97c0050d0 con 0x7fe97406af90 2023-10-08T05:50:21.090 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:21.091 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:50:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.090+0000 7fe98f87c640 1 -- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe97406af90 msgr2=0x7fe97406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.090+0000 7fe98f87c640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe97406af90 0x7fe97406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9881f5b90 tx=0x7fe97c007950 comp rx=0 tx=0).stop 2023-10-08T05:50:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.090+0000 7fe98f87c640 1 -- 172.21.15.70:0/817496461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9880cc180 msgr2=0x7fe9880c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.090+0000 7fe98f87c640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9880cc180 0x7fe9880c89b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe97800c460 tx=0x7fe97800c920 comp rx=0 tx=0).stop 2023-10-08T05:50:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.091+0000 7fe98f87c640 1 -- 172.21.15.70:0/817496461 shutdown_connections 2023-10-08T05:50:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.091+0000 7fe98f87c640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe97406af90 0x7fe97406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.091+0000 7fe98f87c640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe98815d4e0 0x7fe988158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.091+0000 7fe98f87c640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9881592e0 0x7fe9880c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.091+0000 7fe98f87c640 1 --2- 172.21.15.70:0/817496461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9880cc180 0x7fe9880c89b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.091+0000 7fe98f87c640 1 -- 172.21.15.70:0/817496461 >> 172.21.15.70:0/817496461 conn(0x7fe9880b1480 msgr2=0x7fe9880cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:21.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.091+0000 7fe98f87c640 1 -- 172.21.15.70:0/817496461 shutdown_connections 2023-10-08T05:50:21.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.091+0000 7fe98f87c640 1 -- 172.21.15.70:0/817496461 wait complete. 2023-10-08T05:50:21.111 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-08T05:50:19.543992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":318464,"kb_used_data":7940,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754656256,"statfs":{"total":773094113280,"available":772768006144,"internally_reserved":0,"allocated":8130560,"data_stored":4617820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":317836859},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.873958"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019327+0000","last_change":"2023-10-08T05:50:16.717684+0000","last_active":"2023-10-08T05:50:18.019327+0000","last_peered":"2023-10-08T05:50:18.019327+0000","last_clean":"2023-10-08T05:50:18.019327+0000","last_became_active":"2023-10-08T05:50:16.717348+0000","last_became_peered":"2023-10-08T05:50:16.717348+0000","last_unstale":"2023-10-08T05:50:18.019327+0000","last_undegraded":"2023-10-08T05:50:18.019327+0000","last_fullsized":"2023-10-08T05:50:18.019327+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:17.977758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020269+0000","last_change":"2023-10-08T05:50:16.717901+0000","last_active":"2023-10-08T05:50:18.020269+0000","last_peered":"2023-10-08T05:50:18.020269+0000","last_clean":"2023-10-08T05:50:18.020269+0000","last_became_active":"2023-10-08T05:50:16.716713+0000","last_became_peered":"2023-10-08T05:50:16.716713+0000","last_unstale":"2023-10-08T05:50:18.020269+0000","last_undegraded":"2023-10-08T05:50:18.020269+0000","last_fullsized":"2023-10-08T05:50:18.020269+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:31.065492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021002+0000","last_change":"2023-10-08T05:50:16.715005+0000","last_active":"2023-10-08T05:50:18.021002+0000","last_peered":"2023-10-08T05:50:18.021002+0000","last_clean":"2023-10-08T05:50:18.021002+0000","last_became_active":"2023-10-08T05:50:16.713396+0000","last_became_peered":"2023-10-08T05:50:16.713396+0000","last_unstale":"2023-10-08T05:50:18.021002+0000","last_undegraded":"2023-10-08T05:50:18.021002+0000","last_fullsized":"2023-10-08T05:50:18.021002+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:28.711395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019621+0000","last_change":"2023-10-08T05:50:16.714890+0000","last_active":"2023-10-08T05:50:18.019621+0000","last_peered":"2023-10-08T05:50:18.019621+0000","last_clean":"2023-10-08T05:50:18.019621+0000","last_became_active":"2023-10-08T05:50:16.714620+0000","last_became_peered":"2023-10-08T05:50:16.714620+0000","last_unstale":"2023-10-08T05:50:18.019621+0000","last_undegraded":"2023-10-08T05:50:18.019621+0000","last_fullsized":"2023-10-08T05:50:18.019621+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:55.876929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.700958+0000","last_change":"2023-10-08T05:50:16.707702+0000","last_active":"2023-10-08T05:50:17.700958+0000","last_peered":"2023-10-08T05:50:17.700958+0000","last_clean":"2023-10-08T05:50:17.700958+0000","last_became_active":"2023-10-08T05:50:16.707242+0000","last_became_peered":"2023-10-08T05:50:16.707242+0000","last_unstale":"2023-10-08T05:50:17.700958+0000","last_undegraded":"2023-10-08T05:50:17.700958+0000","last_fullsized":"2023-10-08T05:50:17.700958+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:13.456528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018168+0000","last_change":"2023-10-08T05:50:16.709032+0000","last_active":"2023-10-08T05:50:18.018168+0000","last_peered":"2023-10-08T05:50:18.018168+0000","last_clean":"2023-10-08T05:50:18.018168+0000","last_became_active":"2023-10-08T05:50:16.708754+0000","last_became_peered":"2023-10-08T05:50:16.708754+0000","last_unstale":"2023-10-08T05:50:18.018168+0000","last_undegraded":"2023-10-08T05:50:18.018168+0000","last_fullsized":"2023-10-08T05:50:18.018168+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:04:22.053283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020026+0000","last_change":"2023-10-08T05:50:16.713031+0000","last_active":"2023-10-08T05:50:18.020026+0000","last_peered":"2023-10-08T05:50:18.020026+0000","last_clean":"2023-10-08T05:50:18.020026+0000","last_became_active":"2023-10-08T05:50:16.712533+0000","last_became_peered":"2023-10-08T05:50:16.712533+0000","last_unstale":"2023-10-08T05:50:18.020026+0000","last_undegraded":"2023-10-08T05:50:18.020026+0000","last_fullsized":"2023-10-08T05:50:18.020026+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:06.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018741+0000","last_change":"2023-10-08T05:50:16.719950+0000","last_active":"2023-10-08T05:50:18.018741+0000","last_peered":"2023-10-08T05:50:18.018741+0000","last_clean":"2023-10-08T05:50:18.018741+0000","last_became_active":"2023-10-08T05:50:16.719700+0000","last_became_peered":"2023-10-08T05:50:16.719700+0000","last_unstale":"2023-10-08T05:50:18.018741+0000","last_undegraded":"2023-10-08T05:50:18.018741+0000","last_fullsized":"2023-10-08T05:50:18.018741+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:31.065492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019762+0000","last_change":"2023-10-08T05:50:16.723146+0000","last_active":"2023-10-08T05:50:18.019762+0000","last_peered":"2023-10-08T05:50:18.019762+0000","last_clean":"2023-10-08T05:50:18.019762+0000","last_became_active":"2023-10-08T05:50:16.722571+0000","last_became_peered":"2023-10-08T05:50:16.722571+0000","last_unstale":"2023-10-08T05:50:18.019762+0000","last_undegraded":"2023-10-08T05:50:18.019762+0000","last_fullsized":"2023-10-08T05:50:18.019762+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:35:36.557636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019995+0000","last_change":"2023-10-08T05:50:16.704930+0000","last_active":"2023-10-08T05:50:18.019995+0000","last_peered":"2023-10-08T05:50:18.019995+0000","last_clean":"2023-10-08T05:50:18.019995+0000","last_became_active":"2023-10-08T05:50:16.704669+0000","last_became_peered":"2023-10-08T05:50:16.704669+0000","last_unstale":"2023-10-08T05:50:18.019995+0000","last_undegraded":"2023-10-08T05:50:18.019995+0000","last_fullsized":"2023-10-08T05:50:18.019995+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:51:44.615198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.697416+0000","last_change":"2023-10-08T05:50:16.706930+0000","last_active":"2023-10-08T05:50:17.697416+0000","last_peered":"2023-10-08T05:50:17.697416+0000","last_clean":"2023-10-08T05:50:17.697416+0000","last_became_active":"2023-10-08T05:50:16.706623+0000","last_became_peered":"2023-10-08T05:50:16.706623+0000","last_unstale":"2023-10-08T05:50:17.697416+0000","last_undegraded":"2023-10-08T05:50:17.697416+0000","last_fullsized":"2023-10-08T05:50:17.697416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:50.700339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019665+0000","last_change":"2023-10-08T05:50:16.716087+0000","last_active":"2023-10-08T05:50:18.019665+0000","last_peered":"2023-10-08T05:50:18.019665+0000","last_clean":"2023-10-08T05:50:18.019665+0000","last_became_active":"2023-10-08T05:50:16.715804+0000","last_became_peered":"2023-10-08T05:50:16.715804+0000","last_unstale":"2023-10-08T05:50:18.019665+0000","last_undegraded":"2023-10-08T05:50:18.019665+0000","last_fullsized":"2023-10-08T05:50:18.019665+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:14:58.774283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018679+0000","last_change":"2023-10-08T05:50:16.704770+0000","last_active":"2023-10-08T05:50:18.018679+0000","last_peered":"2023-10-08T05:50:18.018679+0000","last_clean":"2023-10-08T05:50:18.018679+0000","last_became_active":"2023-10-08T05:50:16.704416+0000","last_became_peered":"2023-10-08T05:50:16.704416+0000","last_unstale":"2023-10-08T05:50:18.018679+0000","last_undegraded":"2023-10-08T05:50:18.018679+0000","last_fullsized":"2023-10-08T05:50:18.018679+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:54.883271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020461+0000","last_change":"2023-10-08T05:50:16.714033+0000","last_active":"2023-10-08T05:50:18.020461+0000","last_peered":"2023-10-08T05:50:18.020461+0000","last_clean":"2023-10-08T05:50:18.020461+0000","last_became_active":"2023-10-08T05:50:16.713770+0000","last_became_peered":"2023-10-08T05:50:16.713770+0000","last_unstale":"2023-10-08T05:50:18.020461+0000","last_undegraded":"2023-10-08T05:50:18.020461+0000","last_fullsized":"2023-10-08T05:50:18.020461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:36:40.136197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018709+0000","last_change":"2023-10-08T05:50:16.706842+0000","last_active":"2023-10-08T05:50:18.018709+0000","last_peered":"2023-10-08T05:50:18.018709+0000","last_clean":"2023-10-08T05:50:18.018709+0000","last_became_active":"2023-10-08T05:50:16.706512+0000","last_became_peered":"2023-10-08T05:50:16.706512+0000","last_unstale":"2023-10-08T05:50:18.018709+0000","last_undegraded":"2023-10-08T05:50:18.018709+0000","last_fullsized":"2023-10-08T05:50:18.018709+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:33.808383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.017970+0000","last_change":"2023-10-08T05:50:16.705403+0000","last_active":"2023-10-08T05:50:18.017970+0000","last_peered":"2023-10-08T05:50:18.017970+0000","last_clean":"2023-10-08T05:50:18.017970+0000","last_became_active":"2023-10-08T05:50:16.705097+0000","last_became_peered":"2023-10-08T05:50:16.705097+0000","last_unstale":"2023-10-08T05:50:18.017970+0000","last_undegraded":"2023-10-08T05:50:18.017970+0000","last_fullsized":"2023-10-08T05:50:18.017970+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:53.303428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021645+0000","last_change":"2023-10-08T05:50:16.721491+0000","last_active":"2023-10-08T05:50:18.021645+0000","last_peered":"2023-10-08T05:50:18.021645+0000","last_clean":"2023-10-08T05:50:18.021645+0000","last_became_active":"2023-10-08T05:50:16.720867+0000","last_became_peered":"2023-10-08T05:50:16.720867+0000","last_unstale":"2023-10-08T05:50:18.021645+0000","last_undegraded":"2023-10-08T05:50:18.021645+0000","last_fullsized":"2023-10-08T05:50:18.021645+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:09.019345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020630+0000","last_change":"2023-10-08T05:50:16.717361+0000","last_active":"2023-10-08T05:50:18.020630+0000","last_peered":"2023-10-08T05:50:18.020630+0000","last_clean":"2023-10-08T05:50:18.020630+0000","last_became_active":"2023-10-08T05:50:16.717063+0000","last_became_peered":"2023-10-08T05:50:16.717063+0000","last_unstale":"2023-10-08T05:50:18.020630+0000","last_undegraded":"2023-10-08T05:50:18.020630+0000","last_fullsized":"2023-10-08T05:50:18.020630+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:56.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021135+0000","last_change":"2023-10-08T05:50:16.719033+0000","last_active":"2023-10-08T05:50:18.021135+0000","last_peered":"2023-10-08T05:50:18.021135+0000","last_clean":"2023-10-08T05:50:18.021135+0000","last_became_active":"2023-10-08T05:50:16.718726+0000","last_became_peered":"2023-10-08T05:50:16.718726+0000","last_unstale":"2023-10-08T05:50:18.021135+0000","last_undegraded":"2023-10-08T05:50:18.021135+0000","last_fullsized":"2023-10-08T05:50:18.021135+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:15.866750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702722+0000","last_change":"2023-10-08T05:50:16.700178+0000","last_active":"2023-10-08T05:50:17.702722+0000","last_peered":"2023-10-08T05:50:17.702722+0000","last_clean":"2023-10-08T05:50:17.702722+0000","last_became_active":"2023-10-08T05:50:16.699665+0000","last_became_peered":"2023-10-08T05:50:16.699665+0000","last_unstale":"2023-10-08T05:50:17.702722+0000","last_undegraded":"2023-10-08T05:50:17.702722+0000","last_fullsized":"2023-10-08T05:50:17.702722+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:03.935342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702617+0000","last_change":"2023-10-08T05:50:16.701576+0000","last_active":"2023-10-08T05:50:17.702617+0000","last_peered":"2023-10-08T05:50:17.702617+0000","last_clean":"2023-10-08T05:50:17.702617+0000","last_became_active":"2023-10-08T05:50:16.701240+0000","last_became_peered":"2023-10-08T05:50:16.701240+0000","last_unstale":"2023-10-08T05:50:17.702617+0000","last_undegraded":"2023-10-08T05:50:17.702617+0000","last_fullsized":"2023-10-08T05:50:17.702617+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:28.211339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019107+0000","last_change":"2023-10-08T05:50:16.721414+0000","last_active":"2023-10-08T05:50:18.019107+0000","last_peered":"2023-10-08T05:50:18.019107+0000","last_clean":"2023-10-08T05:50:18.019107+0000","last_became_active":"2023-10-08T05:50:16.721141+0000","last_became_peered":"2023-10-08T05:50:16.721141+0000","last_unstale":"2023-10-08T05:50:18.019107+0000","last_undegraded":"2023-10-08T05:50:18.019107+0000","last_fullsized":"2023-10-08T05:50:18.019107+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:05.422082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.696911+0000","last_change":"2023-10-08T05:50:16.705930+0000","last_active":"2023-10-08T05:50:17.696911+0000","last_peered":"2023-10-08T05:50:17.696911+0000","last_clean":"2023-10-08T05:50:17.696911+0000","last_became_active":"2023-10-08T05:50:16.705649+0000","last_became_peered":"2023-10-08T05:50:16.705649+0000","last_unstale":"2023-10-08T05:50:17.696911+0000","last_undegraded":"2023-10-08T05:50:17.696911+0000","last_fullsized":"2023-10-08T05:50:17.696911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:33:44.830896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021830+0000","last_change":"2023-10-08T05:50:16.712850+0000","last_active":"2023-10-08T05:50:18.021830+0000","last_peered":"2023-10-08T05:50:18.021830+0000","last_clean":"2023-10-08T05:50:18.021830+0000","last_became_active":"2023-10-08T05:50:16.712550+0000","last_became_peered":"2023-10-08T05:50:16.712550+0000","last_unstale":"2023-10-08T05:50:18.021830+0000","last_undegraded":"2023-10-08T05:50:18.021830+0000","last_fullsized":"2023-10-08T05:50:18.021830+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:04.188793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021916+0000","last_change":"2023-10-08T05:50:16.717790+0000","last_active":"2023-10-08T05:50:18.021916+0000","last_peered":"2023-10-08T05:50:18.021916+0000","last_clean":"2023-10-08T05:50:18.021916+0000","last_became_active":"2023-10-08T05:50:16.717518+0000","last_became_peered":"2023-10-08T05:50:16.717518+0000","last_unstale":"2023-10-08T05:50:18.021916+0000","last_undegraded":"2023-10-08T05:50:18.021916+0000","last_fullsized":"2023-10-08T05:50:18.021916+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:52.140858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021039+0000","last_change":"2023-10-08T05:50:16.713286+0000","last_active":"2023-10-08T05:50:18.021039+0000","last_peered":"2023-10-08T05:50:18.021039+0000","last_clean":"2023-10-08T05:50:18.021039+0000","last_became_active":"2023-10-08T05:50:16.713028+0000","last_became_peered":"2023-10-08T05:50:16.713028+0000","last_unstale":"2023-10-08T05:50:18.021039+0000","last_undegraded":"2023-10-08T05:50:18.021039+0000","last_fullsized":"2023-10-08T05:50:18.021039+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:35.247977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020424+0000","last_change":"2023-10-08T05:50:16.723253+0000","last_active":"2023-10-08T05:50:18.020424+0000","last_peered":"2023-10-08T05:50:18.020424+0000","last_clean":"2023-10-08T05:50:18.020424+0000","last_became_active":"2023-10-08T05:50:16.722998+0000","last_became_peered":"2023-10-08T05:50:16.722998+0000","last_unstale":"2023-10-08T05:50:18.020424+0000","last_undegraded":"2023-10-08T05:50:18.020424+0000","last_fullsized":"2023-10-08T05:50:18.020424+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:26.006424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.703044+0000","last_change":"2023-10-08T05:50:16.695345+0000","last_active":"2023-10-08T05:50:17.703044+0000","last_peered":"2023-10-08T05:50:17.703044+0000","last_clean":"2023-10-08T05:50:17.703044+0000","last_became_active":"2023-10-08T05:50:16.694357+0000","last_became_peered":"2023-10-08T05:50:16.694357+0000","last_unstale":"2023-10-08T05:50:17.703044+0000","last_undegraded":"2023-10-08T05:50:17.703044+0000","last_fullsized":"2023-10-08T05:50:17.703044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:01:18.816670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020633+0000","last_change":"2023-10-08T05:50:16.720256+0000","last_active":"2023-10-08T05:50:18.020633+0000","last_peered":"2023-10-08T05:50:18.020633+0000","last_clean":"2023-10-08T05:50:18.020633+0000","last_became_active":"2023-10-08T05:50:16.719355+0000","last_became_peered":"2023-10-08T05:50:16.719355+0000","last_unstale":"2023-10-08T05:50:18.020633+0000","last_undegraded":"2023-10-08T05:50:18.020633+0000","last_fullsized":"2023-10-08T05:50:18.020633+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:55:11.211417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018960+0000","last_change":"2023-10-08T05:50:16.710073+0000","last_active":"2023-10-08T05:50:18.018960+0000","last_peered":"2023-10-08T05:50:18.018960+0000","last_clean":"2023-10-08T05:50:18.018960+0000","last_became_active":"2023-10-08T05:50:16.709775+0000","last_became_peered":"2023-10-08T05:50:16.709775+0000","last_unstale":"2023-10-08T05:50:18.018960+0000","last_undegraded":"2023-10-08T05:50:18.018960+0000","last_fullsized":"2023-10-08T05:50:18.018960+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:44.844118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020132+0000","last_change":"2023-10-08T05:50:16.712853+0000","last_active":"2023-10-08T05:50:18.020132+0000","last_peered":"2023-10-08T05:50:18.020132+0000","last_clean":"2023-10-08T05:50:18.020132+0000","last_became_active":"2023-10-08T05:50:16.712209+0000","last_became_peered":"2023-10-08T05:50:16.712209+0000","last_unstale":"2023-10-08T05:50:18.020132+0000","last_undegraded":"2023-10-08T05:50:18.020132+0000","last_fullsized":"2023-10-08T05:50:18.020132+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:43:47.872846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019268+0000","last_change":"2023-10-08T05:50:16.715655+0000","last_active":"2023-10-08T05:50:18.019268+0000","last_peered":"2023-10-08T05:50:18.019268+0000","last_clean":"2023-10-08T05:50:18.019268+0000","last_became_active":"2023-10-08T05:50:16.714985+0000","last_became_peered":"2023-10-08T05:50:16.714985+0000","last_unstale":"2023-10-08T05:50:18.019268+0000","last_undegraded":"2023-10-08T05:50:18.019268+0000","last_fullsized":"2023-10-08T05:50:18.019268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:12.275035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018344+0000","last_change":"2023-10-08T05:50:16.717259+0000","last_active":"2023-10-08T05:50:18.018344+0000","last_peered":"2023-10-08T05:50:18.018344+0000","last_clean":"2023-10-08T05:50:18.018344+0000","last_became_active":"2023-10-08T05:50:16.716886+0000","last_became_peered":"2023-10-08T05:50:16.716886+0000","last_unstale":"2023-10-08T05:50:18.018344+0000","last_undegraded":"2023-10-08T05:50:18.018344+0000","last_fullsized":"2023-10-08T05:50:18.018344+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:22.476967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"71'4","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702855+0000","last_change":"2023-10-08T05:50:15.715619+0000","last_active":"2023-10-08T05:50:17.702855+0000","last_peered":"2023-10-08T05:50:17.702855+0000","last_clean":"2023-10-08T05:50:17.702855+0000","last_became_active":"2023-10-08T05:50:15.715360+0000","last_became_peered":"2023-10-08T05:50:15.715360+0000","last_unstale":"2023-10-08T05:50:17.702855+0000","last_undegraded":"2023-10-08T05:50:17.702855+0000","last_fullsized":"2023-10-08T05:50:17.702855+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:43.493448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019830+0000","last_change":"2023-10-08T05:50:16.715720+0000","last_active":"2023-10-08T05:50:18.019830+0000","last_peered":"2023-10-08T05:50:18.019830+0000","last_clean":"2023-10-08T05:50:18.019830+0000","last_became_active":"2023-10-08T05:50:16.715446+0000","last_became_peered":"2023-10-08T05:50:16.715446+0000","last_unstale":"2023-10-08T05:50:18.019830+0000","last_undegraded":"2023-10-08T05:50:18.019830+0000","last_fullsized":"2023-10-08T05:50:18.019830+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:03.315734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020088+0000","last_change":"2023-10-08T05:50:15.716596+0000","last_active":"2023-10-08T05:50:18.020088+0000","last_peered":"2023-10-08T05:50:18.020088+0000","last_clean":"2023-10-08T05:50:18.020088+0000","last_became_active":"2023-10-08T05:50:15.716306+0000","last_became_peered":"2023-10-08T05:50:15.716306+0000","last_unstale":"2023-10-08T05:50:18.020088+0000","last_undegraded":"2023-10-08T05:50:18.020088+0000","last_fullsized":"2023-10-08T05:50:18.020088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:46.005042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702549+0000","last_change":"2023-10-08T05:50:16.701829+0000","last_active":"2023-10-08T05:50:17.702549+0000","last_peered":"2023-10-08T05:50:17.702549+0000","last_clean":"2023-10-08T05:50:17.702549+0000","last_became_active":"2023-10-08T05:50:16.701356+0000","last_became_peered":"2023-10-08T05:50:16.701356+0000","last_unstale":"2023-10-08T05:50:17.702549+0000","last_undegraded":"2023-10-08T05:50:17.702549+0000","last_fullsized":"2023-10-08T05:50:17.702549+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:12:15.056984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020679+0000","last_change":"2023-10-08T05:50:15.717409+0000","last_active":"2023-10-08T05:50:18.020679+0000","last_peered":"2023-10-08T05:50:18.020679+0000","last_clean":"2023-10-08T05:50:18.020679+0000","last_became_active":"2023-10-08T05:50:15.717162+0000","last_became_peered":"2023-10-08T05:50:15.717162+0000","last_unstale":"2023-10-08T05:50:18.020679+0000","last_undegraded":"2023-10-08T05:50:18.020679+0000","last_fullsized":"2023-10-08T05:50:18.020679+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:23.124649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019115+0000","last_change":"2023-10-08T05:50:16.712211+0000","last_active":"2023-10-08T05:50:18.019115+0000","last_peered":"2023-10-08T05:50:18.019115+0000","last_clean":"2023-10-08T05:50:18.019115+0000","last_became_active":"2023-10-08T05:50:16.711742+0000","last_became_peered":"2023-10-08T05:50:16.711742+0000","last_unstale":"2023-10-08T05:50:18.019115+0000","last_undegraded":"2023-10-08T05:50:18.019115+0000","last_fullsized":"2023-10-08T05:50:18.019115+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:56:31.509751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020310+0000","last_change":"2023-10-08T05:50:15.719653+0000","last_active":"2023-10-08T05:50:18.020310+0000","last_peered":"2023-10-08T05:50:18.020310+0000","last_clean":"2023-10-08T05:50:18.020310+0000","last_became_active":"2023-10-08T05:50:15.719347+0000","last_became_peered":"2023-10-08T05:50:15.719347+0000","last_unstale":"2023-10-08T05:50:18.020310+0000","last_undegraded":"2023-10-08T05:50:18.020310+0000","last_fullsized":"2023-10-08T05:50:18.020310+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:34.232951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701642+0000","last_change":"2023-10-08T05:50:16.719655+0000","last_active":"2023-10-08T05:50:17.701642+0000","last_peered":"2023-10-08T05:50:17.701642+0000","last_clean":"2023-10-08T05:50:17.701642+0000","last_became_active":"2023-10-08T05:50:16.719325+0000","last_became_peered":"2023-10-08T05:50:16.719325+0000","last_unstale":"2023-10-08T05:50:17.701642+0000","last_undegraded":"2023-10-08T05:50:17.701642+0000","last_fullsized":"2023-10-08T05:50:17.701642+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:24.728503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020726+0000","last_change":"2023-10-08T05:50:15.705786+0000","last_active":"2023-10-08T05:50:18.020726+0000","last_peered":"2023-10-08T05:50:18.020726+0000","last_clean":"2023-10-08T05:50:18.020726+0000","last_became_active":"2023-10-08T05:50:15.705333+0000","last_became_peered":"2023-10-08T05:50:15.705333+0000","last_unstale":"2023-10-08T05:50:18.020726+0000","last_undegraded":"2023-10-08T05:50:18.020726+0000","last_fullsized":"2023-10-08T05:50:18.020726+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:57.117517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019437+0000","last_change":"2023-10-08T05:50:16.711800+0000","last_active":"2023-10-08T05:50:18.019437+0000","last_peered":"2023-10-08T05:50:18.019437+0000","last_clean":"2023-10-08T05:50:18.019437+0000","last_became_active":"2023-10-08T05:50:16.710914+0000","last_became_peered":"2023-10-08T05:50:16.710914+0000","last_unstale":"2023-10-08T05:50:18.019437+0000","last_undegraded":"2023-10-08T05:50:18.019437+0000","last_fullsized":"2023-10-08T05:50:18.019437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:40:18.021260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701176+0000","last_change":"2023-10-08T05:50:15.717050+0000","last_active":"2023-10-08T05:50:17.701176+0000","last_peered":"2023-10-08T05:50:17.701176+0000","last_clean":"2023-10-08T05:50:17.701176+0000","last_became_active":"2023-10-08T05:50:15.716779+0000","last_became_peered":"2023-10-08T05:50:15.716779+0000","last_unstale":"2023-10-08T05:50:17.701176+0000","last_undegraded":"2023-10-08T05:50:17.701176+0000","last_fullsized":"2023-10-08T05:50:17.701176+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:51:43.600172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021353+0000","last_change":"2023-10-08T05:50:16.712625+0000","last_active":"2023-10-08T05:50:18.021353+0000","last_peered":"2023-10-08T05:50:18.021353+0000","last_clean":"2023-10-08T05:50:18.021353+0000","last_became_active":"2023-10-08T05:50:16.712354+0000","last_became_peered":"2023-10-08T05:50:16.712354+0000","last_unstale":"2023-10-08T05:50:18.021353+0000","last_undegraded":"2023-10-08T05:50:18.021353+0000","last_fullsized":"2023-10-08T05:50:18.021353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:06.575672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701993+0000","last_change":"2023-10-08T05:50:15.715646+0000","last_active":"2023-10-08T05:50:17.701993+0000","last_peered":"2023-10-08T05:50:17.701993+0000","last_clean":"2023-10-08T05:50:17.701993+0000","last_became_active":"2023-10-08T05:50:15.715329+0000","last_became_peered":"2023-10-08T05:50:15.715329+0000","last_unstale":"2023-10-08T05:50:17.701993+0000","last_undegraded":"2023-10-08T05:50:17.701993+0000","last_fullsized":"2023-10-08T05:50:17.701993+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:22.680763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020008+0000","last_change":"2023-10-08T05:50:16.720577+0000","last_active":"2023-10-08T05:50:18.020008+0000","last_peered":"2023-10-08T05:50:18.020008+0000","last_clean":"2023-10-08T05:50:18.020008+0000","last_became_active":"2023-10-08T05:50:16.720180+0000","last_became_peered":"2023-10-08T05:50:16.720180+0000","last_unstale":"2023-10-08T05:50:18.020008+0000","last_undegraded":"2023-10-08T05:50:18.020008+0000","last_fullsized":"2023-10-08T05:50:18.020008+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:18.809941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021700+0000","last_change":"2023-10-08T05:50:15.704703+0000","last_active":"2023-10-08T05:50:18.021700+0000","last_peered":"2023-10-08T05:50:18.021700+0000","last_clean":"2023-10-08T05:50:18.021700+0000","last_became_active":"2023-10-08T05:50:15.704115+0000","last_became_peered":"2023-10-08T05:50:15.704115+0000","last_unstale":"2023-10-08T05:50:18.021700+0000","last_undegraded":"2023-10-08T05:50:18.021700+0000","last_fullsized":"2023-10-08T05:50:18.021700+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:43.882425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.700469+0000","last_change":"2023-10-08T05:50:16.711773+0000","last_active":"2023-10-08T05:50:17.700469+0000","last_peered":"2023-10-08T05:50:17.700469+0000","last_clean":"2023-10-08T05:50:17.700469+0000","last_became_active":"2023-10-08T05:50:16.711476+0000","last_became_peered":"2023-10-08T05:50:16.711476+0000","last_unstale":"2023-10-08T05:50:17.700469+0000","last_undegraded":"2023-10-08T05:50:17.700469+0000","last_fullsized":"2023-10-08T05:50:17.700469+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:26.006424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020273+0000","last_change":"2023-10-08T05:50:15.718436+0000","last_active":"2023-10-08T05:50:18.020273+0000","last_peered":"2023-10-08T05:50:18.020273+0000","last_clean":"2023-10-08T05:50:18.020273+0000","last_became_active":"2023-10-08T05:50:15.717818+0000","last_became_peered":"2023-10-08T05:50:15.717818+0000","last_unstale":"2023-10-08T05:50:18.020273+0000","last_undegraded":"2023-10-08T05:50:18.020273+0000","last_fullsized":"2023-10-08T05:50:18.020273+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:41:11.950183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020203+0000","last_change":"2023-10-08T05:50:16.714230+0000","last_active":"2023-10-08T05:50:18.020203+0000","last_peered":"2023-10-08T05:50:18.020203+0000","last_clean":"2023-10-08T05:50:18.020203+0000","last_became_active":"2023-10-08T05:50:16.713143+0000","last_became_peered":"2023-10-08T05:50:16.713143+0000","last_unstale":"2023-10-08T05:50:18.020203+0000","last_undegraded":"2023-10-08T05:50:18.020203+0000","last_fullsized":"2023-10-08T05:50:18.020203+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:00.613799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701331+0000","last_change":"2023-10-08T05:50:15.707386+0000","last_active":"2023-10-08T05:50:17.701331+0000","last_peered":"2023-10-08T05:50:17.701331+0000","last_clean":"2023-10-08T05:50:17.701331+0000","last_became_active":"2023-10-08T05:50:15.706885+0000","last_became_peered":"2023-10-08T05:50:15.706885+0000","last_unstale":"2023-10-08T05:50:17.701331+0000","last_undegraded":"2023-10-08T05:50:17.701331+0000","last_fullsized":"2023-10-08T05:50:17.701331+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:16.962732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020751+0000","last_change":"2023-10-08T05:50:16.720163+0000","last_active":"2023-10-08T05:50:18.020751+0000","last_peered":"2023-10-08T05:50:18.020751+0000","last_clean":"2023-10-08T05:50:18.020751+0000","last_became_active":"2023-10-08T05:50:16.719918+0000","last_became_peered":"2023-10-08T05:50:16.719918+0000","last_unstale":"2023-10-08T05:50:18.020751+0000","last_undegraded":"2023-10-08T05:50:18.020751+0000","last_fullsized":"2023-10-08T05:50:18.020751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:37.735709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"71'3","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020743+0000","last_change":"2023-10-08T05:50:15.714177+0000","last_active":"2023-10-08T05:50:18.020743+0000","last_peered":"2023-10-08T05:50:18.020743+0000","last_clean":"2023-10-08T05:50:18.020743+0000","last_became_active":"2023-10-08T05:50:15.713862+0000","last_became_peered":"2023-10-08T05:50:15.713862+0000","last_unstale":"2023-10-08T05:50:18.020743+0000","last_undegraded":"2023-10-08T05:50:18.020743+0000","last_fullsized":"2023-10-08T05:50:18.020743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:54:04.997937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701977+0000","last_change":"2023-10-08T05:50:16.707888+0000","last_active":"2023-10-08T05:50:17.701977+0000","last_peered":"2023-10-08T05:50:17.701977+0000","last_clean":"2023-10-08T05:50:17.701977+0000","last_became_active":"2023-10-08T05:50:16.707606+0000","last_became_peered":"2023-10-08T05:50:16.707606+0000","last_unstale":"2023-10-08T05:50:17.701977+0000","last_undegraded":"2023-10-08T05:50:17.701977+0000","last_fullsized":"2023-10-08T05:50:17.701977+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:10.593532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"71'6","reported_seq":18,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019815+0000","last_change":"2023-10-08T05:50:15.709446+0000","last_active":"2023-10-08T05:50:18.019815+0000","last_peered":"2023-10-08T05:50:18.019815+0000","last_clean":"2023-10-08T05:50:18.019815+0000","last_became_active":"2023-10-08T05:50:15.709144+0000","last_became_peered":"2023-10-08T05:50:15.709144+0000","last_unstale":"2023-10-08T05:50:18.019815+0000","last_undegraded":"2023-10-08T05:50:18.019815+0000","last_fullsized":"2023-10-08T05:50:18.019815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:32.793357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020049+0000","last_change":"2023-10-08T05:50:16.711457+0000","last_active":"2023-10-08T05:50:18.020049+0000","last_peered":"2023-10-08T05:50:18.020049+0000","last_clean":"2023-10-08T05:50:18.020049+0000","last_became_active":"2023-10-08T05:50:16.711176+0000","last_became_peered":"2023-10-08T05:50:16.711176+0000","last_unstale":"2023-10-08T05:50:18.020049+0000","last_undegraded":"2023-10-08T05:50:18.020049+0000","last_fullsized":"2023-10-08T05:50:18.020049+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:59:50.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020818+0000","last_change":"2023-10-08T05:50:15.713971+0000","last_active":"2023-10-08T05:50:18.020818+0000","last_peered":"2023-10-08T05:50:18.020818+0000","last_clean":"2023-10-08T05:50:18.020818+0000","last_became_active":"2023-10-08T05:50:15.713688+0000","last_became_peered":"2023-10-08T05:50:15.713688+0000","last_unstale":"2023-10-08T05:50:18.020818+0000","last_undegraded":"2023-10-08T05:50:18.020818+0000","last_fullsized":"2023-10-08T05:50:18.020818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:31.224857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021260+0000","last_change":"2023-10-08T05:50:16.714837+0000","last_active":"2023-10-08T05:50:18.021260+0000","last_peered":"2023-10-08T05:50:18.021260+0000","last_clean":"2023-10-08T05:50:18.021260+0000","last_became_active":"2023-10-08T05:50:16.713088+0000","last_became_peered":"2023-10-08T05:50:16.713088+0000","last_unstale":"2023-10-08T05:50:18.021260+0000","last_undegraded":"2023-10-08T05:50:18.021260+0000","last_fullsized":"2023-10-08T05:50:18.021260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:02.279284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702458+0000","last_change":"2023-10-08T05:50:15.709182+0000","last_active":"2023-10-08T05:50:17.702458+0000","last_peered":"2023-10-08T05:50:17.702458+0000","last_clean":"2023-10-08T05:50:17.702458+0000","last_became_active":"2023-10-08T05:50:15.708901+0000","last_became_peered":"2023-10-08T05:50:15.708901+0000","last_unstale":"2023-10-08T05:50:17.702458+0000","last_undegraded":"2023-10-08T05:50:17.702458+0000","last_fullsized":"2023-10-08T05:50:17.702458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:40:55.893412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021080+0000","last_change":"2023-10-08T05:50:16.711312+0000","last_active":"2023-10-08T05:50:18.021080+0000","last_peered":"2023-10-08T05:50:18.021080+0000","last_clean":"2023-10-08T05:50:18.021080+0000","last_became_active":"2023-10-08T05:50:16.711040+0000","last_became_peered":"2023-10-08T05:50:16.711040+0000","last_unstale":"2023-10-08T05:50:18.021080+0000","last_undegraded":"2023-10-08T05:50:18.021080+0000","last_fullsized":"2023-10-08T05:50:18.021080+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:12.275035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018978+0000","last_change":"2023-10-08T05:50:15.719279+0000","last_active":"2023-10-08T05:50:18.018978+0000","last_peered":"2023-10-08T05:50:18.018978+0000","last_clean":"2023-10-08T05:50:18.018978+0000","last_became_active":"2023-10-08T05:50:15.718959+0000","last_became_peered":"2023-10-08T05:50:15.718959+0000","last_unstale":"2023-10-08T05:50:18.018978+0000","last_undegraded":"2023-10-08T05:50:18.018978+0000","last_fullsized":"2023-10-08T05:50:18.018978+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:49.346321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021785+0000","last_change":"2023-10-08T05:50:16.721125+0000","last_active":"2023-10-08T05:50:18.021785+0000","last_peered":"2023-10-08T05:50:18.021785+0000","last_clean":"2023-10-08T05:50:18.021785+0000","last_became_active":"2023-10-08T05:50:16.720523+0000","last_became_peered":"2023-10-08T05:50:16.720523+0000","last_unstale":"2023-10-08T05:50:18.021785+0000","last_undegraded":"2023-10-08T05:50:18.021785+0000","last_fullsized":"2023-10-08T05:50:18.021785+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:20.715049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"71'8","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018967+0000","last_change":"2023-10-08T05:50:15.707774+0000","last_active":"2023-10-08T05:50:18.018967+0000","last_peered":"2023-10-08T05:50:18.018967+0000","last_clean":"2023-10-08T05:50:18.018967+0000","last_became_active":"2023-10-08T05:50:15.707447+0000","last_became_peered":"2023-10-08T05:50:15.707447+0000","last_unstale":"2023-10-08T05:50:18.018967+0000","last_undegraded":"2023-10-08T05:50:18.018967+0000","last_fullsized":"2023-10-08T05:50:18.018967+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:17.794915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018874+0000","last_change":"2023-10-08T05:50:16.718007+0000","last_active":"2023-10-08T05:50:18.018874+0000","last_peered":"2023-10-08T05:50:18.018874+0000","last_clean":"2023-10-08T05:50:18.018874+0000","last_became_active":"2023-10-08T05:50:16.717577+0000","last_became_peered":"2023-10-08T05:50:16.717577+0000","last_unstale":"2023-10-08T05:50:18.018874+0000","last_undegraded":"2023-10-08T05:50:18.018874+0000","last_fullsized":"2023-10-08T05:50:18.018874+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:56.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.022005+0000","last_change":"2023-10-08T05:50:15.697449+0000","last_active":"2023-10-08T05:50:18.022005+0000","last_peered":"2023-10-08T05:50:18.022005+0000","last_clean":"2023-10-08T05:50:18.022005+0000","last_became_active":"2023-10-08T05:50:15.696566+0000","last_became_peered":"2023-10-08T05:50:15.696566+0000","last_unstale":"2023-10-08T05:50:18.022005+0000","last_undegraded":"2023-10-08T05:50:18.022005+0000","last_fullsized":"2023-10-08T05:50:18.022005+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:56.179248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.017843+0000","last_change":"2023-10-08T05:50:16.722573+0000","last_active":"2023-10-08T05:50:18.017843+0000","last_peered":"2023-10-08T05:50:18.017843+0000","last_clean":"2023-10-08T05:50:18.017843+0000","last_became_active":"2023-10-08T05:50:16.722189+0000","last_became_peered":"2023-10-08T05:50:16.722189+0000","last_unstale":"2023-10-08T05:50:18.017843+0000","last_undegraded":"2023-10-08T05:50:18.017843+0000","last_fullsized":"2023-10-08T05:50:18.017843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:55.228608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.697459+0000","last_change":"2023-10-08T05:50:15.714591+0000","last_active":"2023-10-08T05:50:17.697459+0000","last_peered":"2023-10-08T05:50:17.697459+0000","last_clean":"2023-10-08T05:50:17.697459+0000","last_became_active":"2023-10-08T05:50:15.714304+0000","last_became_peered":"2023-10-08T05:50:15.714304+0000","last_unstale":"2023-10-08T05:50:17.697459+0000","last_undegraded":"2023-10-08T05:50:17.697459+0000","last_fullsized":"2023-10-08T05:50:17.697459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:11.230748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.700882+0000","last_change":"2023-10-08T05:50:16.720026+0000","last_active":"2023-10-08T05:50:17.700882+0000","last_peered":"2023-10-08T05:50:17.700882+0000","last_clean":"2023-10-08T05:50:17.700882+0000","last_became_active":"2023-10-08T05:50:16.719742+0000","last_became_peered":"2023-10-08T05:50:16.719742+0000","last_unstale":"2023-10-08T05:50:17.700882+0000","last_undegraded":"2023-10-08T05:50:17.700882+0000","last_fullsized":"2023-10-08T05:50:17.700882+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:10.481059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.703014+0000","last_change":"2023-10-08T05:50:15.716363+0000","last_active":"2023-10-08T05:50:17.703014+0000","last_peered":"2023-10-08T05:50:17.703014+0000","last_clean":"2023-10-08T05:50:17.703014+0000","last_became_active":"2023-10-08T05:50:15.715920+0000","last_became_peered":"2023-10-08T05:50:15.715920+0000","last_unstale":"2023-10-08T05:50:17.703014+0000","last_undegraded":"2023-10-08T05:50:17.703014+0000","last_fullsized":"2023-10-08T05:50:17.703014+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:10:14.764591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018503+0000","last_change":"2023-10-08T05:50:16.713948+0000","last_active":"2023-10-08T05:50:18.018503+0000","last_peered":"2023-10-08T05:50:18.018503+0000","last_clean":"2023-10-08T05:50:18.018503+0000","last_became_active":"2023-10-08T05:50:16.713664+0000","last_became_peered":"2023-10-08T05:50:16.713664+0000","last_unstale":"2023-10-08T05:50:18.018503+0000","last_undegraded":"2023-10-08T05:50:18.018503+0000","last_fullsized":"2023-10-08T05:50:18.018503+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:42:15.329615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.703004+0000","last_change":"2023-10-08T05:50:15.715013+0000","last_active":"2023-10-08T05:50:17.703004+0000","last_peered":"2023-10-08T05:50:17.703004+0000","last_clean":"2023-10-08T05:50:17.703004+0000","last_became_active":"2023-10-08T05:50:15.714660+0000","last_became_peered":"2023-10-08T05:50:15.714660+0000","last_unstale":"2023-10-08T05:50:17.703004+0000","last_undegraded":"2023-10-08T05:50:17.703004+0000","last_fullsized":"2023-10-08T05:50:17.703004+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:10.546542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020249+0000","last_change":"2023-10-08T05:50:16.714188+0000","last_active":"2023-10-08T05:50:18.020249+0000","last_peered":"2023-10-08T05:50:18.020249+0000","last_clean":"2023-10-08T05:50:18.020249+0000","last_became_active":"2023-10-08T05:50:16.713873+0000","last_became_peered":"2023-10-08T05:50:16.713873+0000","last_unstale":"2023-10-08T05:50:18.020249+0000","last_undegraded":"2023-10-08T05:50:18.020249+0000","last_fullsized":"2023-10-08T05:50:18.020249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:18:36.163267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018743+0000","last_change":"2023-10-08T05:50:15.714644+0000","last_active":"2023-10-08T05:50:18.018743+0000","last_peered":"2023-10-08T05:50:18.018743+0000","last_clean":"2023-10-08T05:50:18.018743+0000","last_became_active":"2023-10-08T05:50:15.714383+0000","last_became_peered":"2023-10-08T05:50:15.714383+0000","last_unstale":"2023-10-08T05:50:18.018743+0000","last_undegraded":"2023-10-08T05:50:18.018743+0000","last_fullsized":"2023-10-08T05:50:18.018743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:03.007678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.703127+0000","last_change":"2023-10-08T05:50:16.701006+0000","last_active":"2023-10-08T05:50:17.703127+0000","last_peered":"2023-10-08T05:50:17.703127+0000","last_clean":"2023-10-08T05:50:17.703127+0000","last_became_active":"2023-10-08T05:50:16.700540+0000","last_became_peered":"2023-10-08T05:50:16.700540+0000","last_unstale":"2023-10-08T05:50:17.703127+0000","last_undegraded":"2023-10-08T05:50:17.703127+0000","last_fullsized":"2023-10-08T05:50:17.703127+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:19:45.567174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019349+0000","last_change":"2023-10-08T05:50:15.708697+0000","last_active":"2023-10-08T05:50:18.019349+0000","last_peered":"2023-10-08T05:50:18.019349+0000","last_clean":"2023-10-08T05:50:18.019349+0000","last_became_active":"2023-10-08T05:50:15.708426+0000","last_became_peered":"2023-10-08T05:50:15.708426+0000","last_unstale":"2023-10-08T05:50:18.019349+0000","last_undegraded":"2023-10-08T05:50:18.019349+0000","last_fullsized":"2023-10-08T05:50:18.019349+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:49.346321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021653+0000","last_change":"2023-10-08T05:50:16.720028+0000","last_active":"2023-10-08T05:50:18.021653+0000","last_peered":"2023-10-08T05:50:18.021653+0000","last_clean":"2023-10-08T05:50:18.021653+0000","last_became_active":"2023-10-08T05:50:16.719746+0000","last_became_peered":"2023-10-08T05:50:16.719746+0000","last_unstale":"2023-10-08T05:50:18.021653+0000","last_undegraded":"2023-10-08T05:50:18.021653+0000","last_fullsized":"2023-10-08T05:50:18.021653+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:45.849982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020238+0000","last_change":"2023-10-08T05:50:15.712066+0000","last_active":"2023-10-08T05:50:18.020238+0000","last_peered":"2023-10-08T05:50:18.020238+0000","last_clean":"2023-10-08T05:50:18.020238+0000","last_became_active":"2023-10-08T05:50:15.711555+0000","last_became_peered":"2023-10-08T05:50:15.711555+0000","last_unstale":"2023-10-08T05:50:18.020238+0000","last_undegraded":"2023-10-08T05:50:18.020238+0000","last_fullsized":"2023-10-08T05:50:18.020238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:43.529374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019682+0000","last_change":"2023-10-08T05:50:16.707290+0000","last_active":"2023-10-08T05:50:18.019682+0000","last_peered":"2023-10-08T05:50:18.019682+0000","last_clean":"2023-10-08T05:50:18.019682+0000","last_became_active":"2023-10-08T05:50:16.707015+0000","last_became_peered":"2023-10-08T05:50:16.707015+0000","last_unstale":"2023-10-08T05:50:18.019682+0000","last_undegraded":"2023-10-08T05:50:18.019682+0000","last_fullsized":"2023-10-08T05:50:18.019682+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:29.239765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.696815+0000","last_change":"2023-10-08T05:50:15.713684+0000","last_active":"2023-10-08T05:50:17.696815+0000","last_peered":"2023-10-08T05:50:17.696815+0000","last_clean":"2023-10-08T05:50:17.696815+0000","last_became_active":"2023-10-08T05:50:15.713412+0000","last_became_peered":"2023-10-08T05:50:15.713412+0000","last_unstale":"2023-10-08T05:50:17.696815+0000","last_undegraded":"2023-10-08T05:50:17.696815+0000","last_fullsized":"2023-10-08T05:50:17.696815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:46.743198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020068+0000","last_change":"2023-10-08T05:50:16.714910+0000","last_active":"2023-10-08T05:50:18.020068+0000","last_peered":"2023-10-08T05:50:18.020068+0000","last_clean":"2023-10-08T05:50:18.020068+0000","last_became_active":"2023-10-08T05:50:16.714607+0000","last_became_peered":"2023-10-08T05:50:16.714607+0000","last_unstale":"2023-10-08T05:50:18.020068+0000","last_undegraded":"2023-10-08T05:50:18.020068+0000","last_fullsized":"2023-10-08T05:50:18.020068+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:57:41.204748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702971+0000","last_change":"2023-10-08T05:50:15.715614+0000","last_active":"2023-10-08T05:50:17.702971+0000","last_peered":"2023-10-08T05:50:17.702971+0000","last_clean":"2023-10-08T05:50:17.702971+0000","last_became_active":"2023-10-08T05:50:15.715309+0000","last_became_peered":"2023-10-08T05:50:15.715309+0000","last_unstale":"2023-10-08T05:50:17.702971+0000","last_undegraded":"2023-10-08T05:50:17.702971+0000","last_fullsized":"2023-10-08T05:50:17.702971+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:13.374218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018400+0000","last_change":"2023-10-08T05:50:16.689939+0000","last_active":"2023-10-08T05:50:18.018400+0000","last_peered":"2023-10-08T05:50:18.018400+0000","last_clean":"2023-10-08T05:50:18.018400+0000","last_became_active":"2023-10-08T05:50:16.689200+0000","last_became_peered":"2023-10-08T05:50:16.689200+0000","last_unstale":"2023-10-08T05:50:18.018400+0000","last_undegraded":"2023-10-08T05:50:18.018400+0000","last_fullsized":"2023-10-08T05:50:18.018400+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:01.709935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020825+0000","last_change":"2023-10-08T05:50:15.719651+0000","last_active":"2023-10-08T05:50:18.020825+0000","last_peered":"2023-10-08T05:50:18.020825+0000","last_clean":"2023-10-08T05:50:18.020825+0000","last_became_active":"2023-10-08T05:50:15.718971+0000","last_became_peered":"2023-10-08T05:50:15.718971+0000","last_unstale":"2023-10-08T05:50:18.020825+0000","last_undegraded":"2023-10-08T05:50:18.020825+0000","last_fullsized":"2023-10-08T05:50:18.020825+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:56:30.873451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019151+0000","last_change":"2023-10-08T05:50:16.703551+0000","last_active":"2023-10-08T05:50:18.019151+0000","last_peered":"2023-10-08T05:50:18.019151+0000","last_clean":"2023-10-08T05:50:18.019151+0000","last_became_active":"2023-10-08T05:50:16.703259+0000","last_became_peered":"2023-10-08T05:50:16.703259+0000","last_unstale":"2023-10-08T05:50:18.019151+0000","last_undegraded":"2023-10-08T05:50:18.019151+0000","last_fullsized":"2023-10-08T05:50:18.019151+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:26.259393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021070+0000","last_change":"2023-10-08T05:50:15.706889+0000","last_active":"2023-10-08T05:50:18.021070+0000","last_peered":"2023-10-08T05:50:18.021070+0000","last_clean":"2023-10-08T05:50:18.021070+0000","last_became_active":"2023-10-08T05:50:15.706576+0000","last_became_peered":"2023-10-08T05:50:15.706576+0000","last_unstale":"2023-10-08T05:50:18.021070+0000","last_undegraded":"2023-10-08T05:50:18.021070+0000","last_fullsized":"2023-10-08T05:50:18.021070+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:17:29.014240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020854+0000","last_change":"2023-10-08T05:50:16.727005+0000","last_active":"2023-10-08T05:50:18.020854+0000","last_peered":"2023-10-08T05:50:18.020854+0000","last_clean":"2023-10-08T05:50:18.020854+0000","last_became_active":"2023-10-08T05:50:16.726737+0000","last_became_peered":"2023-10-08T05:50:16.726737+0000","last_unstale":"2023-10-08T05:50:18.020854+0000","last_undegraded":"2023-10-08T05:50:18.020854+0000","last_fullsized":"2023-10-08T05:50:18.020854+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:49.632967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019247+0000","last_change":"2023-10-08T05:50:15.708587+0000","last_active":"2023-10-08T05:50:18.019247+0000","last_peered":"2023-10-08T05:50:18.019247+0000","last_clean":"2023-10-08T05:50:18.019247+0000","last_became_active":"2023-10-08T05:50:15.707967+0000","last_became_peered":"2023-10-08T05:50:15.707967+0000","last_unstale":"2023-10-08T05:50:18.019247+0000","last_undegraded":"2023-10-08T05:50:18.019247+0000","last_fullsized":"2023-10-08T05:50:18.019247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:56:30.873451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019722+0000","last_change":"2023-10-08T05:50:16.714941+0000","last_active":"2023-10-08T05:50:18.019722+0000","last_peered":"2023-10-08T05:50:18.019722+0000","last_clean":"2023-10-08T05:50:18.019722+0000","last_became_active":"2023-10-08T05:50:16.714682+0000","last_became_peered":"2023-10-08T05:50:16.714682+0000","last_unstale":"2023-10-08T05:50:18.019722+0000","last_undegraded":"2023-10-08T05:50:18.019722+0000","last_fullsized":"2023-10-08T05:50:18.019722+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:06.575672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019592+0000","last_change":"2023-10-08T05:50:15.709941+0000","last_active":"2023-10-08T05:50:18.019592+0000","last_peered":"2023-10-08T05:50:18.019592+0000","last_clean":"2023-10-08T05:50:18.019592+0000","last_became_active":"2023-10-08T05:50:15.709463+0000","last_became_peered":"2023-10-08T05:50:15.709463+0000","last_unstale":"2023-10-08T05:50:18.019592+0000","last_undegraded":"2023-10-08T05:50:18.019592+0000","last_fullsized":"2023-10-08T05:50:18.019592+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:54:04.997937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021314+0000","last_change":"2023-10-08T05:50:16.716088+0000","last_active":"2023-10-08T05:50:18.021314+0000","last_peered":"2023-10-08T05:50:18.021314+0000","last_clean":"2023-10-08T05:50:18.021314+0000","last_became_active":"2023-10-08T05:50:16.715306+0000","last_became_peered":"2023-10-08T05:50:16.715306+0000","last_unstale":"2023-10-08T05:50:18.021314+0000","last_undegraded":"2023-10-08T05:50:18.021314+0000","last_fullsized":"2023-10-08T05:50:18.021314+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:16.111403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701622+0000","last_change":"2023-10-08T05:50:15.705479+0000","last_active":"2023-10-08T05:50:17.701622+0000","last_peered":"2023-10-08T05:50:17.701622+0000","last_clean":"2023-10-08T05:50:17.701622+0000","last_became_active":"2023-10-08T05:50:15.705052+0000","last_became_peered":"2023-10-08T05:50:15.705052+0000","last_unstale":"2023-10-08T05:50:17.701622+0000","last_undegraded":"2023-10-08T05:50:17.701622+0000","last_fullsized":"2023-10-08T05:50:17.701622+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:53.868245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020095+0000","last_change":"2023-10-08T05:50:16.712022+0000","last_active":"2023-10-08T05:50:18.020095+0000","last_peered":"2023-10-08T05:50:18.020095+0000","last_clean":"2023-10-08T05:50:18.020095+0000","last_became_active":"2023-10-08T05:50:16.711732+0000","last_became_peered":"2023-10-08T05:50:16.711732+0000","last_unstale":"2023-10-08T05:50:18.020095+0000","last_undegraded":"2023-10-08T05:50:18.020095+0000","last_fullsized":"2023-10-08T05:50:18.020095+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:01:14.781042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019580+0000","last_change":"2023-10-08T05:50:15.713100+0000","last_active":"2023-10-08T05:50:18.019580+0000","last_peered":"2023-10-08T05:50:18.019580+0000","last_clean":"2023-10-08T05:50:18.019580+0000","last_became_active":"2023-10-08T05:50:15.712265+0000","last_became_peered":"2023-10-08T05:50:15.712265+0000","last_unstale":"2023-10-08T05:50:18.019580+0000","last_undegraded":"2023-10-08T05:50:18.019580+0000","last_fullsized":"2023-10-08T05:50:18.019580+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:14:57.759257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702031+0000","last_change":"2023-10-08T05:50:16.712877+0000","last_active":"2023-10-08T05:50:17.702031+0000","last_peered":"2023-10-08T05:50:17.702031+0000","last_clean":"2023-10-08T05:50:17.702031+0000","last_became_active":"2023-10-08T05:50:16.712566+0000","last_became_peered":"2023-10-08T05:50:16.712566+0000","last_unstale":"2023-10-08T05:50:17.702031+0000","last_undegraded":"2023-10-08T05:50:17.702031+0000","last_fullsized":"2023-10-08T05:50:17.702031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:10.603639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021866+0000","last_change":"2023-10-08T05:50:15.714591+0000","last_active":"2023-10-08T05:50:18.021866+0000","last_peered":"2023-10-08T05:50:18.021866+0000","last_clean":"2023-10-08T05:50:18.021866+0000","last_became_active":"2023-10-08T05:50:15.714324+0000","last_became_peered":"2023-10-08T05:50:15.714324+0000","last_unstale":"2023-10-08T05:50:18.021866+0000","last_undegraded":"2023-10-08T05:50:18.021866+0000","last_fullsized":"2023-10-08T05:50:18.021866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:45.839989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"59'139","reported_seq":298,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-08T05:50:16.698562+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:50:16.698562+0000","last_peered":"2023-10-08T05:50:16.698562+0000","last_clean":"2023-10-08T05:50:16.698562+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:50:16.698562+0000","last_undegraded":"2023-10-08T05:50:16.698562+0000","last_fullsized":"2023-10-08T05:50:16.698562+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575021,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58972,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312868,"statfs":{"total":96636764160,"available":96576376832,"internally_reserved":0,"allocated":1454080,"data_stored":1015360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.258,"15min":1.2050000000000001},"min":{"1min":0.72599999999999998,"5min":0.68500000000000005,"15min":0.68000000000000005},"max":{"1min":1.6499999999999999,"5min":1.792,"15min":2.2360000000000002},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2230000000000001,"15min":1.1919999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.70699999999999996},"max":{"1min":1.5649999999999999,"5min":1.726,"15min":1.726},"last":1.3300000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.284,"15min":1.236},"min":{"1min":0.83999999999999997,"5min":0.82499999999999996,"15min":0.68600000000000005},"max":{"1min":1.6060000000000001,"5min":1.8360000000000001,"15min":1.972},"last":1.27},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.2769999999999999,"15min":1.256},"min":{"1min":0.86899999999999999,"5min":0.82599999999999996,"15min":0.69199999999999995},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.9379999999999999},"last":1.4199999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2889999999999999,"15min":1.208},"min":{"1min":0.93400000000000005,"5min":0.90200000000000002,"15min":0.65100000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8280000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.335},"min":{"1min":0.82299999999999995,"5min":0.78400000000000003,"15min":0.75600000000000001},"max":{"1min":1.6870000000000001,"5min":1.752,"15min":2.3849999999999998},"last":1.3839999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3069999999999999,"15min":1.276},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6459999999999999,"5min":1.746,"15min":2.2919999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3720000000000001,"15min":1.3799999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.776,"5min":1.8200000000000001,"15min":2.5579999999999998},"last":1.5489999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2310000000000001,"15min":1.2869999999999999},"min":{"1min":0.95799999999999996,"5min":0.78500000000000003,"15min":0.65000000000000002},"max":{"1min":1.7250000000000001,"5min":1.752,"15min":2.4529999999999998},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.329,"15min":1.347},"min":{"1min":0.97899999999999998,"5min":0.89800000000000002,"15min":0.46800000000000003},"max":{"1min":1.909,"5min":1.909,"15min":2.677},"last":1.3640000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3999999999999999,"15min":1.419},"min":{"1min":1.0329999999999999,"5min":0.92800000000000005,"15min":0.54200000000000004},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":3.3319999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.417,"15min":1.4450000000000001},"min":{"1min":1.0820000000000001,"5min":0.89200000000000002,"15min":0.58799999999999997},"max":{"1min":1.7909999999999999,"5min":1.8859999999999999,"15min":2.726},"last":1.6779999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.2569999999999999,"5min":1.028,"15min":0.68300000000000005},"max":{"1min":1.8440000000000001,"5min":2.0030000000000001,"15min":3.5249999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.466,"15min":1.468},"min":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.52000000000000002},"max":{"1min":1.782,"5min":1.9550000000000001,"15min":2.4700000000000002},"last":1.3080000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575020,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":872448,"data_stored":432184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.319,"15min":1.2509999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.639,"5min":1.913,"15min":1.913},"last":1.331},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.292,"15min":1.2929999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68300000000000005},"max":{"1min":1.4990000000000001,"5min":1.8089999999999999,"15min":2.222},"last":0.98499999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.167,"15min":1.159},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.573,"5min":1.681,"15min":1.9530000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.2290000000000001,"15min":1.1930000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5780000000000001,"5min":1.7110000000000001,"15min":1.8029999999999999},"last":1.135}]},{"osd":2,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.24,"15min":1.2010000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.70399999999999996},"max":{"1min":1.7490000000000001,"5min":1.796,"15min":1.796},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.2649999999999999,"15min":1.2589999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.58699999999999997},"max":{"1min":1.3560000000000001,"5min":1.8520000000000001,"15min":1.8620000000000001},"last":1.5900000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.389,"15min":1.3640000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.905,"15min":1.905},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.4570000000000001,"15min":1.446},"min":{"1min":1.046,"5min":1.046,"15min":0.86399999999999999},"max":{"1min":1.679,"5min":1.9450000000000001,"15min":2.056},"last":1.385}]},{"osd":4,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.294,"15min":1.3280000000000001},"min":{"1min":0.86699999999999999,"5min":0.76300000000000001,"15min":0.73499999999999999},"max":{"1min":1.7350000000000001,"5min":1.8169999999999999,"15min":2.278},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3700000000000001,"15min":1.3919999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.64300000000000002},"max":{"1min":1.764,"5min":1.8819999999999999,"15min":2.3399999999999999},"last":1.202}]},{"osd":5,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3160000000000001,"15min":1.383},"min":{"1min":1.0289999999999999,"5min":0.86899999999999999,"15min":0.79600000000000004},"max":{"1min":1.619,"5min":1.833,"15min":4.3099999999999996},"last":1.665},{"interface":"front","average":{"1min":1.407,"5min":1.492,"15min":1.4910000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.871,"5min":1.871,"15min":2.1070000000000002},"last":1.7170000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.464,"15min":1.53},"min":{"1min":1.03,"5min":0.97899999999999998,"15min":0.77200000000000002},"max":{"1min":1.819,"5min":1.9159999999999999,"15min":2.3959999999999999},"last":1.506},{"interface":"front","average":{"1min":1.403,"5min":1.4299999999999999,"15min":1.5069999999999999},"min":{"1min":0.97799999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.726,"5min":1.946,"15min":4.1550000000000002},"last":1.9710000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575020,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":431532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1200000000000001,"15min":1.1299999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.46700000000000003},"max":{"1min":1.363,"5min":1.546,"15min":2.302},"last":1.02},{"interface":"front","average":{"1min":1.236,"5min":1.2170000000000001,"15min":1.181},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.673,"5min":1.758,"15min":3.5529999999999999},"last":1.206}]},{"osd":1,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2110000000000001,"15min":1.2030000000000001},"min":{"1min":0.95799999999999996,"5min":0.79900000000000004,"15min":0.72399999999999998},"max":{"1min":1.845,"5min":1.845,"15min":2.351},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2829999999999999,"15min":1.2689999999999999},"min":{"1min":0.87,"5min":0.84499999999999997,"15min":0.64700000000000002},"max":{"1min":1.657,"5min":1.754,"15min":2.1789999999999998},"last":1.597}]},{"osd":2,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.2210000000000001,"15min":1.246},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.69399999999999995},"max":{"1min":1.5269999999999999,"5min":1.6439999999999999,"15min":3.6739999999999999},"last":1.083},{"interface":"front","average":{"1min":1.268,"5min":1.29,"15min":1.2609999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.79100000000000004},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":2.1110000000000002},"last":1.6639999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.3440000000000001,"15min":1.379},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.63500000000000001},"max":{"1min":1.7450000000000001,"5min":1.8,"15min":6.782},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.393,"15min":1.4339999999999999},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.84799999999999998},"max":{"1min":1.8160000000000001,"5min":2.2280000000000002,"15min":6.6189999999999998},"last":1.387}]},{"osd":4,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2629999999999999,"15min":1.335},"min":{"1min":0.83199999999999996,"5min":0.82899999999999996,"15min":0.53600000000000003},"max":{"1min":1.885,"5min":1.885,"15min":6.1349999999999998},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.484,"5min":1.4159999999999999,"15min":1.4610000000000001},"min":{"1min":1.1120000000000001,"5min":0.95099999999999996,"15min":0.51400000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":7.0069999999999997},"last":1.8480000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3819999999999999,"15min":1.444},"min":{"1min":1.0229999999999999,"5min":0.84299999999999997,"15min":0.57699999999999996},"max":{"1min":1.6100000000000001,"5min":2.359,"15min":7.2569999999999997},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.4510000000000001,"15min":1.494},"min":{"1min":1.2230000000000001,"5min":1.04,"15min":0.59799999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":6.2199999999999998},"last":1.7889999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5229999999999999,"15min":1.5840000000000001},"min":{"1min":1.1679999999999999,"5min":1.018,"15min":0.53900000000000003},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":6.9279999999999999},"last":1.9450000000000001},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.5609999999999999,"15min":1.6000000000000001},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.61699999999999999},"max":{"1min":1.996,"5min":2.4460000000000002,"15min":7.335},"last":1.3049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575020,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":856064,"data_stored":425450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3,"15min":1.2749999999999999},"min":{"1min":0.99299999999999999,"5min":0.80900000000000005,"15min":0.65300000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.863},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4059999999999999,"15min":1.3720000000000001},"min":{"1min":1.038,"5min":1.0069999999999999,"15min":0.78700000000000003},"max":{"1min":1.756,"5min":1.8400000000000001,"15min":2.0680000000000001},"last":1.4730000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":0.81799999999999995,"5min":0.81399999999999995,"15min":0.65500000000000003},"max":{"1min":1.5720000000000001,"5min":1.7050000000000001,"15min":2.3799999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2110000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.556,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.5089999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.24,"15min":1.2210000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.69599999999999995},"max":{"1min":1.548,"5min":1.6759999999999999,"15min":1.77},"last":1.544},{"interface":"front","average":{"1min":1.286,"5min":1.3320000000000001,"15min":1.345},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.748},"max":{"1min":1.5920000000000001,"5min":1.9079999999999999,"15min":2.0710000000000002},"last":1.2669999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2849999999999999,"15min":1.284},"min":{"1min":0.89300000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.5669999999999999,"5min":1.96,"15min":1.96},"last":1.155},{"interface":"front","average":{"1min":1.369,"5min":1.3420000000000001,"15min":1.377},"min":{"1min":0.96399999999999997,"5min":0.94699999999999995,"15min":0.76900000000000002},"max":{"1min":1.7230000000000001,"5min":1.8320000000000001,"15min":2.1080000000000001},"last":1.4259999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.321,"15min":1.3460000000000001},"min":{"1min":0.91900000000000004,"5min":0.89800000000000002,"15min":0.45700000000000002},"max":{"1min":1.7230000000000001,"5min":1.8720000000000001,"15min":2.2370000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.339,"15min":1.387},"min":{"1min":1.0700000000000001,"5min":0.89300000000000002,"15min":0.58199999999999996},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":2.2629999999999999},"last":1.323}]},{"osd":6,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3089999999999999,"15min":1.383},"min":{"1min":0.878,"5min":0.76900000000000002,"15min":0.55000000000000004},"max":{"1min":1.625,"5min":1.821,"15min":2.2080000000000002},"last":1.214},{"interface":"front","average":{"1min":1.427,"5min":1.47,"15min":1.52},"min":{"1min":1.0660000000000001,"5min":1.0409999999999999,"15min":0.70399999999999996},"max":{"1min":1.79,"5min":1.8029999999999999,"15min":2.5150000000000001},"last":1.371}]},{"osd":7,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.448,"15min":1.4850000000000001},"min":{"1min":1,"5min":1,"15min":0.73699999999999999},"max":{"1min":1.6799999999999999,"5min":2.0529999999999999,"15min":2.5539999999999998},"last":1.579},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.502,"15min":1.571},"min":{"1min":1.212,"5min":0.94799999999999995,"15min":0.67400000000000004},"max":{"1min":1.782,"5min":1.782,"15min":2.5489999999999999},"last":0.97199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335636,"statfs":{"total":96636764160,"available":96599691264,"internally_reserved":0,"allocated":880640,"data_stored":433166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.95399999999999996,"15min":0.89900000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.29599999999999999},"max":{"1min":1.5269999999999999,"5min":1.72,"15min":1.72},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.117,"15min":1.0700000000000001},"min":{"1min":0.83399999999999996,"5min":0.61299999999999999,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":1.548,"15min":2.3199999999999998},"last":1.0920000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98599999999999999,"15min":0.94399999999999995},"min":{"1min":0.751,"5min":0.501,"15min":0.373},"max":{"1min":1.1890000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.234,"15min":1.1719999999999999},"min":{"1min":0.94999999999999996,"5min":0.69699999999999995,"15min":0.436},"max":{"1min":1.577,"5min":1.6220000000000001,"15min":1.6619999999999999},"last":1.1879999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.083,"15min":1.038},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.47699999999999998},"max":{"1min":1.631,"5min":1.645,"15min":1.645},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.1990000000000001,"15min":1.113},"min":{"1min":0.874,"5min":0.746,"15min":0.41899999999999998},"max":{"1min":1.6819999999999999,"5min":1.788,"15min":1.788},"last":1.0349999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2350000000000001,"15min":1.206},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.34200000000000003},"max":{"1min":1.5,"5min":1.677,"15min":2.5059999999999998},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3979999999999999,"15min":1.355},"min":{"1min":1.022,"5min":0.93400000000000005,"15min":0.53800000000000003},"max":{"1min":1.5369999999999999,"5min":1.8380000000000001,"15min":1.9930000000000001},"last":1.345}]},{"osd":5,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3069999999999999,"15min":1.2709999999999999},"min":{"1min":1.0409999999999999,"5min":0.73699999999999999,"15min":0.624},"max":{"1min":1.621,"5min":1.8240000000000001,"15min":1.9219999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.3859999999999999,"15min":1.3340000000000001},"min":{"1min":1.212,"5min":0.78500000000000003,"15min":0.59599999999999997},"max":{"1min":2.0219999999999998,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.3759999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3100000000000001,"15min":1.3160000000000001},"min":{"1min":1.0469999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5589999999999999,"5min":1.776,"15min":2.6150000000000002},"last":1.222},{"interface":"front","average":{"1min":1.494,"5min":1.4470000000000001,"15min":1.423},"min":{"1min":1.294,"5min":0.86699999999999999,"15min":0.70099999999999996},"max":{"1min":1.7689999999999999,"5min":2.0489999999999999,"15min":2.1259999999999999},"last":1.3999999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.51,"15min":1.47},"min":{"1min":1.222,"5min":1.0700000000000001,"15min":0.72299999999999998},"max":{"1min":2.085,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.385,"15min":1.399},"min":{"1min":0.99399999999999999,"5min":0.79300000000000004,"15min":0.66900000000000004},"max":{"1min":1.6870000000000001,"5min":1.8200000000000001,"15min":2.7879999999999998},"last":1.3220000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58412,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313428,"statfs":{"total":96636764160,"available":96576950272,"internally_reserved":0,"allocated":876544,"data_stored":433144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.93999999999999995,"15min":0.90600000000000003},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.37},"max":{"1min":1.2549999999999999,"5min":1.5209999999999999,"15min":2.4009999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.155,"5min":1.0760000000000001,"15min":1.0329999999999999},"min":{"1min":0.83299999999999996,"5min":0.71999999999999997,"15min":0.47499999999999998},"max":{"1min":1.5309999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.5529999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.99299999999999999,"15min":0.96899999999999997},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.48099999999999998},"max":{"1min":1.2090000000000001,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.056,"5min":1.1399999999999999,"15min":1.1060000000000001},"min":{"1min":0.83499999999999996,"5min":0.78300000000000003,"15min":0.42999999999999999},"max":{"1min":1.387,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":1.024}]},{"osd":3,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1719999999999999,"15min":1.1439999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.54100000000000004},"max":{"1min":1.4970000000000001,"5min":1.7370000000000001,"15min":1.8120000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.28,"5min":1.2509999999999999,"15min":1.236},"min":{"1min":1.087,"5min":0.81799999999999995,"15min":0.59399999999999997},"max":{"1min":1.4890000000000001,"5min":1.847,"15min":2.3889999999999998},"last":1.1479999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.4059999999999999,"15min":1.3939999999999999},"min":{"1min":1.153,"5min":1.0860000000000001,"15min":0.74199999999999999},"max":{"1min":1.651,"5min":1.9279999999999999,"15min":2.6419999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.2849999999999999,"15min":1.288},"min":{"1min":1.0129999999999999,"5min":0.93300000000000005,"15min":0.63100000000000001},"max":{"1min":1.6619999999999999,"5min":1.8859999999999999,"15min":2.2530000000000001},"last":1.105}]},{"osd":5,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.395,"15min":1.383},"min":{"1min":1.073,"5min":0.96599999999999997,"15min":0.75600000000000001},"max":{"1min":1.8680000000000001,"5min":1.907,"15min":2.3039999999999998},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.5029999999999999,"15min":1.474},"min":{"1min":1.206,"5min":1.2050000000000001,"15min":0.79400000000000004},"max":{"1min":1.9370000000000001,"5min":2.1309999999999998,"15min":2.8170000000000002},"last":1.413}]},{"osd":6,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3859999999999999,"15min":1.373},"min":{"1min":1.1240000000000001,"5min":1.01,"15min":0.77500000000000002},"max":{"1min":1.72,"5min":1.9239999999999999,"15min":2.851},"last":1.391},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5009999999999999,"15min":1.4790000000000001},"min":{"1min":1.145,"5min":1.125,"15min":0.63900000000000001},"max":{"1min":2.044,"5min":2.1829999999999998,"15min":2.556},"last":1.466}]},{"osd":7,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.458,"15min":1.452},"min":{"1min":1.2190000000000001,"5min":1.05,"15min":0.87},"max":{"1min":1.7669999999999999,"5min":2.0369999999999999,"15min":3.0529999999999999},"last":1.296},{"interface":"front","average":{"1min":1.52,"5min":1.5429999999999999,"15min":1.532},"min":{"1min":1.2629999999999999,"5min":1.149,"15min":0.61799999999999999},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.4729999999999999},"last":1.3540000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":872448,"data_stored":431692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.028,"15min":0.95799999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.29799999999999999},"max":{"1min":1.603,"5min":1.796,"15min":1.796},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.133},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.433},"max":{"1min":1.5600000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.04}]},{"osd":2,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.94399999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.34899999999999998},"max":{"1min":1.5009999999999999,"5min":1.526,"15min":1.611},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.145,"15min":1.093},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.38200000000000001},"max":{"1min":1.593,"5min":1.8049999999999999,"15min":2.0750000000000002},"last":1.0880000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.2050000000000001,"15min":1.167},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.41899999999999998},"max":{"1min":1.6699999999999999,"5min":1.6899999999999999,"15min":1.8160000000000001},"last":1.163},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.085,"15min":1.0640000000000001},"min":{"1min":0.76000000000000001,"5min":0.71899999999999997,"15min":0.42799999999999999},"max":{"1min":1.333,"5min":1.788,"15min":1.788},"last":0.76700000000000002}]},{"osd":4,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3400000000000001,"15min":1.2989999999999999},"min":{"1min":1.044,"5min":0.96699999999999997,"15min":0.63900000000000001},"max":{"1min":1.526,"5min":1.966,"15min":1.966},"last":1.119},{"interface":"front","average":{"1min":1.2,"5min":1.24,"15min":1.2170000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.61799999999999999},"max":{"1min":1.712,"5min":1.722,"15min":2.2029999999999998},"last":1.4330000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.349,"15min":1.2989999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.79200000000000004},"max":{"1min":1.857,"5min":1.857,"15min":2.3399999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.427,"5min":1.5029999999999999,"15min":1.4379999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.71599999999999997},"max":{"1min":1.905,"5min":2.077,"15min":2.3069999999999999},"last":1.2330000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.431,"15min":1.415},"min":{"1min":1.0940000000000001,"5min":1.044,"15min":0.65500000000000003},"max":{"1min":1.6970000000000001,"5min":1.714,"15min":2.2490000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.397,"5min":1.448,"15min":1.403},"min":{"1min":1.2110000000000001,"5min":1.1220000000000001,"15min":0.68300000000000005},"max":{"1min":1.764,"5min":1.788,"15min":2.4700000000000002},"last":1.353}]},{"osd":7,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.502,"15min":1.4590000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.79100000000000004},"max":{"1min":1.8149999999999999,"5min":2.0870000000000002,"15min":2.1869999999999998},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4630000000000001,"15min":1.46},"min":{"1min":1.1519999999999999,"5min":1.149,"15min":0.76300000000000001},"max":{"1min":1.7709999999999999,"5min":2.306,"15min":2.4180000000000001},"last":1.385}]}]},{"osd":0,"up_from":13,"seq":55834575021,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1454080,"data_stored":1015292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.93000000000000005,"15min":0.91000000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.38800000000000001},"max":{"1min":1.1040000000000001,"5min":1.524,"15min":1.7130000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":1.079,"5min":1.111,"15min":1.036},"min":{"1min":0.432,"5min":0.432,"15min":0.377},"max":{"1min":1.5189999999999999,"5min":1.675,"15min":2.3500000000000001},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.1160000000000001,"15min":1.071},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.38900000000000001},"max":{"1min":1.4410000000000001,"5min":1.8,"15min":1.8089999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.052,"15min":1.0049999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.42299999999999999},"max":{"1min":1.546,"5min":1.615,"15min":1.78},"last":0.50600000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.23,"15min":1.1930000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.505},"max":{"1min":1.488,"5min":1.5980000000000001,"15min":2.4540000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.131,"15min":1.099},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.439},"max":{"1min":1.5900000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.321,"15min":1.28},"min":{"1min":1.042,"5min":0.94299999999999995,"15min":0.71499999999999997},"max":{"1min":1.718,"5min":1.7470000000000001,"15min":2.496},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3779999999999999,"15min":1.3620000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6799999999999999,"5min":1.7849999999999999,"15min":2.6560000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.379,"15min":1.341},"min":{"1min":1.05,"5min":0.81299999999999994,"15min":0.76000000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":2.2650000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4079999999999999,"15min":1.399},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.69399999999999995},"max":{"1min":1.7430000000000001,"5min":1.802,"15min":2.7069999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.454,"15min":1.4299999999999999},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.71299999999999997},"max":{"1min":1.6990000000000001,"5min":2.1909999999999998,"15min":2.8290000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4179999999999999,"15min":1.423},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.83399999999999996},"max":{"1min":1.768,"5min":1.7809999999999999,"15min":2.4020000000000001},"last":0.83099999999999996}]},{"osd":7,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4630000000000001,"15min":1.458},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.72399999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.746},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.52,"15min":1.49},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.78900000000000003},"max":{"1min":1.8360000000000001,"5min":2.1059999999999999,"15min":2.9300000000000002},"last":0.98199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:50:21.121 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:50:21.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.375+0000 7fe5ead5a640 1 Processor -- start 2023-10-08T05:50:21.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.375+0000 7fe5ead5a640 1 -- start start 2023-10-08T05:50:21.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.376+0000 7fe5ead5a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e4159ac0 0x7fe5e406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.376+0000 7fe5ead5a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5e40ad790 0x7fe5e40702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.376+0000 7fe5ead5a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e406dee0 con 0x7fe5e415e330 2023-10-08T05:50:21.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.376+0000 7fe5ead5a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e415d850 con 0x7fe5e40ad790 2023-10-08T05:50:21.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.376+0000 7fe5ead5a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415db30 con 0x7fe5e4159ac0 2023-10-08T05:50:21.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.377+0000 7fe5dbfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5e40ad790 0x7fe5e40702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.377+0000 7fe5e8af6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e4159ac0 0x7fe5e406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.377+0000 7fe5dbfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5e40ad790 0x7fe5e40702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:43932/0 (socket says 172.21.15.70:43932) 2023-10-08T05:50:21.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.377+0000 7fe5e8af6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e4159ac0 0x7fe5e406d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:36056/0 (socket says 172.21.15.70:36056) 2023-10-08T05:50:21.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.377+0000 7fe5dbfff640 1 -- 172.21.15.70:0/1218471110 learned_addr learned my addr 172.21.15.70:0/1218471110 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:21.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.378+0000 7fe5dbfff640 1 -- 172.21.15.70:0/1218471110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e4159ac0 msgr2=0x7fe5e406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:21.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.378+0000 7fe5dbfff640 1 --2- 172.21.15.70:0/1218471110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e4159ac0 0x7fe5e406d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.378+0000 7fe5dbfff640 1 -- 172.21.15.70:0/1218471110 >> v1:172.21.15.181:6789/0 conn(0x7fe5e415e330 legacy=0x7fe5e406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.378+0000 7fe5dbfff640 1 -- 172.21.15.70:0/1218471110 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e415de10 con 0x7fe5e40ad790 2023-10-08T05:50:21.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.378+0000 7fe5e8af6640 1 --2- 172.21.15.70:0/1218471110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e4159ac0 0x7fe5e406d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:50:21.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.379+0000 7fe5dbfff640 1 --2- 172.21.15.70:0/1218471110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40ad790 0x7fe5e40702d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009870 tx=0x7fe5d402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0c85ccd8189f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:21.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.380+0000 7fe5db7fe640 1 -- 172.21.15.70:0/1218471110 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4030840 con 0x7fe5e40ad790 2023-10-08T05:50:21.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.380+0000 7fe5db7fe640 1 -- 172.21.15.70:0/1218471110 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d40309a0 con 0x7fe5e40ad790 2023-10-08T05:50:21.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.380+0000 7fe5db7fe640 1 -- 172.21.15.70:0/1218471110 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4030d30 con 0x7fe5e40ad790 2023-10-08T05:50:21.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.381+0000 7fe5ead5a640 1 -- 172.21.15.70:0/1218471110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40ad790 msgr2=0x7fe5e40702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.381+0000 7fe5ead5a640 1 --2- 172.21.15.70:0/1218471110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40ad790 0x7fe5e40702d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009870 tx=0x7fe5d402b980 comp rx=0 tx=0).stop 2023-10-08T05:50:21.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.381+0000 7fe5ead5a640 1 -- 172.21.15.70:0/1218471110 shutdown_connections 2023-10-08T05:50:21.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.381+0000 7fe5ead5a640 1 --2- 172.21.15.70:0/1218471110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e40ad790 0x7fe5e40702d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.381+0000 7fe5ead5a640 1 --2- 172.21.15.70:0/1218471110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e4159ac0 0x7fe5e406d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.381+0000 7fe5ead5a640 1 -- 172.21.15.70:0/1218471110 >> 172.21.15.70:0/1218471110 conn(0x7fe5e406a380 msgr2=0x7fe5e40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:21.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.381+0000 7fe5ead5a640 1 -- 172.21.15.70:0/1218471110 shutdown_connections 2023-10-08T05:50:21.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.381+0000 7fe5ead5a640 1 -- 172.21.15.70:0/1218471110 wait complete. 2023-10-08T05:50:21.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.383+0000 7fe5ead5a640 1 Processor -- start 2023-10-08T05:50:21.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.383+0000 7fe5ead5a640 1 -- start start 2023-10-08T05:50:21.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5ead5a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40ad790 0x7fe5e40b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5ead5a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4159ac0 0x7fe5e41517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5ead5a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e415e330 0x7fe5e4151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5ead5a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40b14a0 con 0x7fe5e4159ac0 2023-10-08T05:50:21.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5ead5a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40b2c60 con 0x7fe5e415e330 2023-10-08T05:50:21.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5ead5a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4152c00 con 0x7fe5e40ad790 2023-10-08T05:50:21.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4159ac0 0x7fe5e41517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4159ac0 0x7fe5e41517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43632/0 (socket says 172.21.15.70:43632) 2023-10-08T05:50:21.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5dbfff640 1 -- 172.21.15.70:0/168395494 learned_addr learned my addr 172.21.15.70:0/168395494 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:21.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5e8af6640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40ad790 0x7fe5e40b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.384+0000 7fe5e92f7640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e415e330 0x7fe5e4151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.385+0000 7fe5dbfff640 1 -- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40ad790 msgr2=0x7fe5e40b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.385+0000 7fe5dbfff640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40ad790 0x7fe5e40b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.385+0000 7fe5dbfff640 1 -- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e415e330 msgr2=0x7fe5e4151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.385+0000 7fe5dbfff640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e415e330 0x7fe5e4151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.385+0000 7fe5dbfff640 1 -- 172.21.15.70:0/168395494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e0009400 con 0x7fe5e4159ac0 2023-10-08T05:50:21.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.385+0000 7fe5dbfff640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4159ac0 0x7fe5e41517b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009990 tx=0x7fe5d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:21.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.385+0000 7fe5d97fa640 1 -- 172.21.15.70:0/168395494 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4039c90 con 0x7fe5e4159ac0 2023-10-08T05:50:21.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.386+0000 7fe5d97fa640 1 -- 172.21.15.70:0/168395494 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4039e10 con 0x7fe5e4159ac0 2023-10-08T05:50:21.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.386+0000 7fe5ead5a640 1 -- 172.21.15.70:0/168395494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d40094d0 con 0x7fe5e4159ac0 2023-10-08T05:50:21.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.386+0000 7fe5d97fa640 1 -- 172.21.15.70:0/168395494 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40446c0 con 0x7fe5e4159ac0 2023-10-08T05:50:21.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.386+0000 7fe5ead5a640 1 -- 172.21.15.70:0/168395494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e40b1960 con 0x7fe5e4159ac0 2023-10-08T05:50:21.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.387+0000 7fe5ead5a640 1 -- 172.21.15.70:0/168395494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e4067490 con 0x7fe5e4159ac0 2023-10-08T05:50:21.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.388+0000 7fe5d97fa640 1 -- 172.21.15.70:0/168395494 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe5d404d3b0 con 0x7fe5e4159ac0 2023-10-08T05:50:21.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.389+0000 7fe5d97fa640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe5d006af90 0x7fe5d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.389+0000 7fe5d97fa640 1 -- 172.21.15.70:0/168395494 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe5d403e070 con 0x7fe5e4159ac0 2023-10-08T05:50:21.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.389+0000 7fe5e8af6640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe5d006af90 0x7fe5d006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.390+0000 7fe5e8af6640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe5d006af90 0x7fe5d006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40b2e00 tx=0x7fe5e0008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:21.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.393+0000 7fe5d97fa640 1 -- 172.21.15.70:0/168395494 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5d4077200 con 0x7fe5e4159ac0 2023-10-08T05:50:21.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.578+0000 7fe5ead5a640 1 -- 172.21.15.70:0/168395494 --> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5e4159ea0 con 0x7fe5d006af90 2023-10-08T05:50:21.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.601+0000 7fe5d97fa640 1 -- 172.21.15.70:0/168395494 <== mgr.4844 v2:172.21.15.181:6832/992174303 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286424 (secure 0 0 0) 0x7fe5e0011e90 con 0x7fe5d006af90 2023-10-08T05:50:21.606 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:50:21.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.605+0000 7fe5ead5a640 1 -- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe5d006af90 msgr2=0x7fe5d006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe5d006af90 0x7fe5d006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40b2e00 tx=0x7fe5e0008040 comp rx=0 tx=0).stop 2023-10-08T05:50:21.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 -- 172.21.15.70:0/168395494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4159ac0 msgr2=0x7fe5e41517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4159ac0 0x7fe5e41517b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009990 tx=0x7fe5d4004730 comp rx=0 tx=0).stop 2023-10-08T05:50:21.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 -- 172.21.15.70:0/168395494 shutdown_connections 2023-10-08T05:50:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5e415e330 0x7fe5e4151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e4159ac0 0x7fe5e41517b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fe5d006af90 0x7fe5d006d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 --2- 172.21.15.70:0/168395494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5e40ad790 0x7fe5e40b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 -- 172.21.15.70:0/168395494 >> 172.21.15.70:0/168395494 conn(0x7fe5e406a380 msgr2=0x7fe5e40afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 -- 172.21.15.70:0/168395494 shutdown_connections 2023-10-08T05:50:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.606+0000 7fe5ead5a640 1 -- 172.21.15.70:0/168395494 wait complete. 2023-10-08T05:50:21.627 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-10-08T05:50:21.544932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":318464,"kb_used_data":7940,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754656256,"statfs":{"total":773094113280,"available":772768006144,"internally_reserved":0,"allocated":8130560,"data_stored":4617820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":317836859},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002160"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019327+0000","last_change":"2023-10-08T05:50:16.717684+0000","last_active":"2023-10-08T05:50:18.019327+0000","last_peered":"2023-10-08T05:50:18.019327+0000","last_clean":"2023-10-08T05:50:18.019327+0000","last_became_active":"2023-10-08T05:50:16.717348+0000","last_became_peered":"2023-10-08T05:50:16.717348+0000","last_unstale":"2023-10-08T05:50:18.019327+0000","last_undegraded":"2023-10-08T05:50:18.019327+0000","last_fullsized":"2023-10-08T05:50:18.019327+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:17.977758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020269+0000","last_change":"2023-10-08T05:50:16.717901+0000","last_active":"2023-10-08T05:50:18.020269+0000","last_peered":"2023-10-08T05:50:18.020269+0000","last_clean":"2023-10-08T05:50:18.020269+0000","last_became_active":"2023-10-08T05:50:16.716713+0000","last_became_peered":"2023-10-08T05:50:16.716713+0000","last_unstale":"2023-10-08T05:50:18.020269+0000","last_undegraded":"2023-10-08T05:50:18.020269+0000","last_fullsized":"2023-10-08T05:50:18.020269+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:31.065492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021002+0000","last_change":"2023-10-08T05:50:16.715005+0000","last_active":"2023-10-08T05:50:18.021002+0000","last_peered":"2023-10-08T05:50:18.021002+0000","last_clean":"2023-10-08T05:50:18.021002+0000","last_became_active":"2023-10-08T05:50:16.713396+0000","last_became_peered":"2023-10-08T05:50:16.713396+0000","last_unstale":"2023-10-08T05:50:18.021002+0000","last_undegraded":"2023-10-08T05:50:18.021002+0000","last_fullsized":"2023-10-08T05:50:18.021002+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:28.711395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019621+0000","last_change":"2023-10-08T05:50:16.714890+0000","last_active":"2023-10-08T05:50:18.019621+0000","last_peered":"2023-10-08T05:50:18.019621+0000","last_clean":"2023-10-08T05:50:18.019621+0000","last_became_active":"2023-10-08T05:50:16.714620+0000","last_became_peered":"2023-10-08T05:50:16.714620+0000","last_unstale":"2023-10-08T05:50:18.019621+0000","last_undegraded":"2023-10-08T05:50:18.019621+0000","last_fullsized":"2023-10-08T05:50:18.019621+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:55.876929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.700958+0000","last_change":"2023-10-08T05:50:16.707702+0000","last_active":"2023-10-08T05:50:17.700958+0000","last_peered":"2023-10-08T05:50:17.700958+0000","last_clean":"2023-10-08T05:50:17.700958+0000","last_became_active":"2023-10-08T05:50:16.707242+0000","last_became_peered":"2023-10-08T05:50:16.707242+0000","last_unstale":"2023-10-08T05:50:17.700958+0000","last_undegraded":"2023-10-08T05:50:17.700958+0000","last_fullsized":"2023-10-08T05:50:17.700958+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:13.456528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018168+0000","last_change":"2023-10-08T05:50:16.709032+0000","last_active":"2023-10-08T05:50:18.018168+0000","last_peered":"2023-10-08T05:50:18.018168+0000","last_clean":"2023-10-08T05:50:18.018168+0000","last_became_active":"2023-10-08T05:50:16.708754+0000","last_became_peered":"2023-10-08T05:50:16.708754+0000","last_unstale":"2023-10-08T05:50:18.018168+0000","last_undegraded":"2023-10-08T05:50:18.018168+0000","last_fullsized":"2023-10-08T05:50:18.018168+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:04:22.053283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020026+0000","last_change":"2023-10-08T05:50:16.713031+0000","last_active":"2023-10-08T05:50:18.020026+0000","last_peered":"2023-10-08T05:50:18.020026+0000","last_clean":"2023-10-08T05:50:18.020026+0000","last_became_active":"2023-10-08T05:50:16.712533+0000","last_became_peered":"2023-10-08T05:50:16.712533+0000","last_unstale":"2023-10-08T05:50:18.020026+0000","last_undegraded":"2023-10-08T05:50:18.020026+0000","last_fullsized":"2023-10-08T05:50:18.020026+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:06.255654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018741+0000","last_change":"2023-10-08T05:50:16.719950+0000","last_active":"2023-10-08T05:50:18.018741+0000","last_peered":"2023-10-08T05:50:18.018741+0000","last_clean":"2023-10-08T05:50:18.018741+0000","last_became_active":"2023-10-08T05:50:16.719700+0000","last_became_peered":"2023-10-08T05:50:16.719700+0000","last_unstale":"2023-10-08T05:50:18.018741+0000","last_undegraded":"2023-10-08T05:50:18.018741+0000","last_fullsized":"2023-10-08T05:50:18.018741+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:31.065492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019762+0000","last_change":"2023-10-08T05:50:16.723146+0000","last_active":"2023-10-08T05:50:18.019762+0000","last_peered":"2023-10-08T05:50:18.019762+0000","last_clean":"2023-10-08T05:50:18.019762+0000","last_became_active":"2023-10-08T05:50:16.722571+0000","last_became_peered":"2023-10-08T05:50:16.722571+0000","last_unstale":"2023-10-08T05:50:18.019762+0000","last_undegraded":"2023-10-08T05:50:18.019762+0000","last_fullsized":"2023-10-08T05:50:18.019762+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:35:36.557636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019995+0000","last_change":"2023-10-08T05:50:16.704930+0000","last_active":"2023-10-08T05:50:18.019995+0000","last_peered":"2023-10-08T05:50:18.019995+0000","last_clean":"2023-10-08T05:50:18.019995+0000","last_became_active":"2023-10-08T05:50:16.704669+0000","last_became_peered":"2023-10-08T05:50:16.704669+0000","last_unstale":"2023-10-08T05:50:18.019995+0000","last_undegraded":"2023-10-08T05:50:18.019995+0000","last_fullsized":"2023-10-08T05:50:18.019995+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:51:44.615198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.697416+0000","last_change":"2023-10-08T05:50:16.706930+0000","last_active":"2023-10-08T05:50:17.697416+0000","last_peered":"2023-10-08T05:50:17.697416+0000","last_clean":"2023-10-08T05:50:17.697416+0000","last_became_active":"2023-10-08T05:50:16.706623+0000","last_became_peered":"2023-10-08T05:50:16.706623+0000","last_unstale":"2023-10-08T05:50:17.697416+0000","last_undegraded":"2023-10-08T05:50:17.697416+0000","last_fullsized":"2023-10-08T05:50:17.697416+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:50.700339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019665+0000","last_change":"2023-10-08T05:50:16.716087+0000","last_active":"2023-10-08T05:50:18.019665+0000","last_peered":"2023-10-08T05:50:18.019665+0000","last_clean":"2023-10-08T05:50:18.019665+0000","last_became_active":"2023-10-08T05:50:16.715804+0000","last_became_peered":"2023-10-08T05:50:16.715804+0000","last_unstale":"2023-10-08T05:50:18.019665+0000","last_undegraded":"2023-10-08T05:50:18.019665+0000","last_fullsized":"2023-10-08T05:50:18.019665+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:14:58.774283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018679+0000","last_change":"2023-10-08T05:50:16.704770+0000","last_active":"2023-10-08T05:50:18.018679+0000","last_peered":"2023-10-08T05:50:18.018679+0000","last_clean":"2023-10-08T05:50:18.018679+0000","last_became_active":"2023-10-08T05:50:16.704416+0000","last_became_peered":"2023-10-08T05:50:16.704416+0000","last_unstale":"2023-10-08T05:50:18.018679+0000","last_undegraded":"2023-10-08T05:50:18.018679+0000","last_fullsized":"2023-10-08T05:50:18.018679+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:54.883271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020461+0000","last_change":"2023-10-08T05:50:16.714033+0000","last_active":"2023-10-08T05:50:18.020461+0000","last_peered":"2023-10-08T05:50:18.020461+0000","last_clean":"2023-10-08T05:50:18.020461+0000","last_became_active":"2023-10-08T05:50:16.713770+0000","last_became_peered":"2023-10-08T05:50:16.713770+0000","last_unstale":"2023-10-08T05:50:18.020461+0000","last_undegraded":"2023-10-08T05:50:18.020461+0000","last_fullsized":"2023-10-08T05:50:18.020461+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:36:40.136197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018709+0000","last_change":"2023-10-08T05:50:16.706842+0000","last_active":"2023-10-08T05:50:18.018709+0000","last_peered":"2023-10-08T05:50:18.018709+0000","last_clean":"2023-10-08T05:50:18.018709+0000","last_became_active":"2023-10-08T05:50:16.706512+0000","last_became_peered":"2023-10-08T05:50:16.706512+0000","last_unstale":"2023-10-08T05:50:18.018709+0000","last_undegraded":"2023-10-08T05:50:18.018709+0000","last_fullsized":"2023-10-08T05:50:18.018709+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:33.808383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.017970+0000","last_change":"2023-10-08T05:50:16.705403+0000","last_active":"2023-10-08T05:50:18.017970+0000","last_peered":"2023-10-08T05:50:18.017970+0000","last_clean":"2023-10-08T05:50:18.017970+0000","last_became_active":"2023-10-08T05:50:16.705097+0000","last_became_peered":"2023-10-08T05:50:16.705097+0000","last_unstale":"2023-10-08T05:50:18.017970+0000","last_undegraded":"2023-10-08T05:50:18.017970+0000","last_fullsized":"2023-10-08T05:50:18.017970+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:53.303428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021645+0000","last_change":"2023-10-08T05:50:16.721491+0000","last_active":"2023-10-08T05:50:18.021645+0000","last_peered":"2023-10-08T05:50:18.021645+0000","last_clean":"2023-10-08T05:50:18.021645+0000","last_became_active":"2023-10-08T05:50:16.720867+0000","last_became_peered":"2023-10-08T05:50:16.720867+0000","last_unstale":"2023-10-08T05:50:18.021645+0000","last_undegraded":"2023-10-08T05:50:18.021645+0000","last_fullsized":"2023-10-08T05:50:18.021645+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:09.019345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020630+0000","last_change":"2023-10-08T05:50:16.717361+0000","last_active":"2023-10-08T05:50:18.020630+0000","last_peered":"2023-10-08T05:50:18.020630+0000","last_clean":"2023-10-08T05:50:18.020630+0000","last_became_active":"2023-10-08T05:50:16.717063+0000","last_became_peered":"2023-10-08T05:50:16.717063+0000","last_unstale":"2023-10-08T05:50:18.020630+0000","last_undegraded":"2023-10-08T05:50:18.020630+0000","last_fullsized":"2023-10-08T05:50:18.020630+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:56.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021135+0000","last_change":"2023-10-08T05:50:16.719033+0000","last_active":"2023-10-08T05:50:18.021135+0000","last_peered":"2023-10-08T05:50:18.021135+0000","last_clean":"2023-10-08T05:50:18.021135+0000","last_became_active":"2023-10-08T05:50:16.718726+0000","last_became_peered":"2023-10-08T05:50:16.718726+0000","last_unstale":"2023-10-08T05:50:18.021135+0000","last_undegraded":"2023-10-08T05:50:18.021135+0000","last_fullsized":"2023-10-08T05:50:18.021135+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:15.866750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702722+0000","last_change":"2023-10-08T05:50:16.700178+0000","last_active":"2023-10-08T05:50:17.702722+0000","last_peered":"2023-10-08T05:50:17.702722+0000","last_clean":"2023-10-08T05:50:17.702722+0000","last_became_active":"2023-10-08T05:50:16.699665+0000","last_became_peered":"2023-10-08T05:50:16.699665+0000","last_unstale":"2023-10-08T05:50:17.702722+0000","last_undegraded":"2023-10-08T05:50:17.702722+0000","last_fullsized":"2023-10-08T05:50:17.702722+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:03.935342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702617+0000","last_change":"2023-10-08T05:50:16.701576+0000","last_active":"2023-10-08T05:50:17.702617+0000","last_peered":"2023-10-08T05:50:17.702617+0000","last_clean":"2023-10-08T05:50:17.702617+0000","last_became_active":"2023-10-08T05:50:16.701240+0000","last_became_peered":"2023-10-08T05:50:16.701240+0000","last_unstale":"2023-10-08T05:50:17.702617+0000","last_undegraded":"2023-10-08T05:50:17.702617+0000","last_fullsized":"2023-10-08T05:50:17.702617+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:28.211339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019107+0000","last_change":"2023-10-08T05:50:16.721414+0000","last_active":"2023-10-08T05:50:18.019107+0000","last_peered":"2023-10-08T05:50:18.019107+0000","last_clean":"2023-10-08T05:50:18.019107+0000","last_became_active":"2023-10-08T05:50:16.721141+0000","last_became_peered":"2023-10-08T05:50:16.721141+0000","last_unstale":"2023-10-08T05:50:18.019107+0000","last_undegraded":"2023-10-08T05:50:18.019107+0000","last_fullsized":"2023-10-08T05:50:18.019107+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:05.422082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.696911+0000","last_change":"2023-10-08T05:50:16.705930+0000","last_active":"2023-10-08T05:50:17.696911+0000","last_peered":"2023-10-08T05:50:17.696911+0000","last_clean":"2023-10-08T05:50:17.696911+0000","last_became_active":"2023-10-08T05:50:16.705649+0000","last_became_peered":"2023-10-08T05:50:16.705649+0000","last_unstale":"2023-10-08T05:50:17.696911+0000","last_undegraded":"2023-10-08T05:50:17.696911+0000","last_fullsized":"2023-10-08T05:50:17.696911+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:33:44.830896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021830+0000","last_change":"2023-10-08T05:50:16.712850+0000","last_active":"2023-10-08T05:50:18.021830+0000","last_peered":"2023-10-08T05:50:18.021830+0000","last_clean":"2023-10-08T05:50:18.021830+0000","last_became_active":"2023-10-08T05:50:16.712550+0000","last_became_peered":"2023-10-08T05:50:16.712550+0000","last_unstale":"2023-10-08T05:50:18.021830+0000","last_undegraded":"2023-10-08T05:50:18.021830+0000","last_fullsized":"2023-10-08T05:50:18.021830+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:04.188793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021916+0000","last_change":"2023-10-08T05:50:16.717790+0000","last_active":"2023-10-08T05:50:18.021916+0000","last_peered":"2023-10-08T05:50:18.021916+0000","last_clean":"2023-10-08T05:50:18.021916+0000","last_became_active":"2023-10-08T05:50:16.717518+0000","last_became_peered":"2023-10-08T05:50:16.717518+0000","last_unstale":"2023-10-08T05:50:18.021916+0000","last_undegraded":"2023-10-08T05:50:18.021916+0000","last_fullsized":"2023-10-08T05:50:18.021916+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:52.140858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021039+0000","last_change":"2023-10-08T05:50:16.713286+0000","last_active":"2023-10-08T05:50:18.021039+0000","last_peered":"2023-10-08T05:50:18.021039+0000","last_clean":"2023-10-08T05:50:18.021039+0000","last_became_active":"2023-10-08T05:50:16.713028+0000","last_became_peered":"2023-10-08T05:50:16.713028+0000","last_unstale":"2023-10-08T05:50:18.021039+0000","last_undegraded":"2023-10-08T05:50:18.021039+0000","last_fullsized":"2023-10-08T05:50:18.021039+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:35.247977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020424+0000","last_change":"2023-10-08T05:50:16.723253+0000","last_active":"2023-10-08T05:50:18.020424+0000","last_peered":"2023-10-08T05:50:18.020424+0000","last_clean":"2023-10-08T05:50:18.020424+0000","last_became_active":"2023-10-08T05:50:16.722998+0000","last_became_peered":"2023-10-08T05:50:16.722998+0000","last_unstale":"2023-10-08T05:50:18.020424+0000","last_undegraded":"2023-10-08T05:50:18.020424+0000","last_fullsized":"2023-10-08T05:50:18.020424+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:26.006424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.703044+0000","last_change":"2023-10-08T05:50:16.695345+0000","last_active":"2023-10-08T05:50:17.703044+0000","last_peered":"2023-10-08T05:50:17.703044+0000","last_clean":"2023-10-08T05:50:17.703044+0000","last_became_active":"2023-10-08T05:50:16.694357+0000","last_became_peered":"2023-10-08T05:50:16.694357+0000","last_unstale":"2023-10-08T05:50:17.703044+0000","last_undegraded":"2023-10-08T05:50:17.703044+0000","last_fullsized":"2023-10-08T05:50:17.703044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:01:18.816670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020633+0000","last_change":"2023-10-08T05:50:16.720256+0000","last_active":"2023-10-08T05:50:18.020633+0000","last_peered":"2023-10-08T05:50:18.020633+0000","last_clean":"2023-10-08T05:50:18.020633+0000","last_became_active":"2023-10-08T05:50:16.719355+0000","last_became_peered":"2023-10-08T05:50:16.719355+0000","last_unstale":"2023-10-08T05:50:18.020633+0000","last_undegraded":"2023-10-08T05:50:18.020633+0000","last_fullsized":"2023-10-08T05:50:18.020633+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:55:11.211417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018960+0000","last_change":"2023-10-08T05:50:16.710073+0000","last_active":"2023-10-08T05:50:18.018960+0000","last_peered":"2023-10-08T05:50:18.018960+0000","last_clean":"2023-10-08T05:50:18.018960+0000","last_became_active":"2023-10-08T05:50:16.709775+0000","last_became_peered":"2023-10-08T05:50:16.709775+0000","last_unstale":"2023-10-08T05:50:18.018960+0000","last_undegraded":"2023-10-08T05:50:18.018960+0000","last_fullsized":"2023-10-08T05:50:18.018960+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:44.844118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020132+0000","last_change":"2023-10-08T05:50:16.712853+0000","last_active":"2023-10-08T05:50:18.020132+0000","last_peered":"2023-10-08T05:50:18.020132+0000","last_clean":"2023-10-08T05:50:18.020132+0000","last_became_active":"2023-10-08T05:50:16.712209+0000","last_became_peered":"2023-10-08T05:50:16.712209+0000","last_unstale":"2023-10-08T05:50:18.020132+0000","last_undegraded":"2023-10-08T05:50:18.020132+0000","last_fullsized":"2023-10-08T05:50:18.020132+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:43:47.872846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019268+0000","last_change":"2023-10-08T05:50:16.715655+0000","last_active":"2023-10-08T05:50:18.019268+0000","last_peered":"2023-10-08T05:50:18.019268+0000","last_clean":"2023-10-08T05:50:18.019268+0000","last_became_active":"2023-10-08T05:50:16.714985+0000","last_became_peered":"2023-10-08T05:50:16.714985+0000","last_unstale":"2023-10-08T05:50:18.019268+0000","last_undegraded":"2023-10-08T05:50:18.019268+0000","last_fullsized":"2023-10-08T05:50:18.019268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:12.275035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018344+0000","last_change":"2023-10-08T05:50:16.717259+0000","last_active":"2023-10-08T05:50:18.018344+0000","last_peered":"2023-10-08T05:50:18.018344+0000","last_clean":"2023-10-08T05:50:18.018344+0000","last_became_active":"2023-10-08T05:50:16.716886+0000","last_became_peered":"2023-10-08T05:50:16.716886+0000","last_unstale":"2023-10-08T05:50:18.018344+0000","last_undegraded":"2023-10-08T05:50:18.018344+0000","last_fullsized":"2023-10-08T05:50:18.018344+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:22.476967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"71'4","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702855+0000","last_change":"2023-10-08T05:50:15.715619+0000","last_active":"2023-10-08T05:50:17.702855+0000","last_peered":"2023-10-08T05:50:17.702855+0000","last_clean":"2023-10-08T05:50:17.702855+0000","last_became_active":"2023-10-08T05:50:15.715360+0000","last_became_peered":"2023-10-08T05:50:15.715360+0000","last_unstale":"2023-10-08T05:50:17.702855+0000","last_undegraded":"2023-10-08T05:50:17.702855+0000","last_fullsized":"2023-10-08T05:50:17.702855+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:43.493448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019830+0000","last_change":"2023-10-08T05:50:16.715720+0000","last_active":"2023-10-08T05:50:18.019830+0000","last_peered":"2023-10-08T05:50:18.019830+0000","last_clean":"2023-10-08T05:50:18.019830+0000","last_became_active":"2023-10-08T05:50:16.715446+0000","last_became_peered":"2023-10-08T05:50:16.715446+0000","last_unstale":"2023-10-08T05:50:18.019830+0000","last_undegraded":"2023-10-08T05:50:18.019830+0000","last_fullsized":"2023-10-08T05:50:18.019830+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:03.315734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020088+0000","last_change":"2023-10-08T05:50:15.716596+0000","last_active":"2023-10-08T05:50:18.020088+0000","last_peered":"2023-10-08T05:50:18.020088+0000","last_clean":"2023-10-08T05:50:18.020088+0000","last_became_active":"2023-10-08T05:50:15.716306+0000","last_became_peered":"2023-10-08T05:50:15.716306+0000","last_unstale":"2023-10-08T05:50:18.020088+0000","last_undegraded":"2023-10-08T05:50:18.020088+0000","last_fullsized":"2023-10-08T05:50:18.020088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:46.005042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702549+0000","last_change":"2023-10-08T05:50:16.701829+0000","last_active":"2023-10-08T05:50:17.702549+0000","last_peered":"2023-10-08T05:50:17.702549+0000","last_clean":"2023-10-08T05:50:17.702549+0000","last_became_active":"2023-10-08T05:50:16.701356+0000","last_became_peered":"2023-10-08T05:50:16.701356+0000","last_unstale":"2023-10-08T05:50:17.702549+0000","last_undegraded":"2023-10-08T05:50:17.702549+0000","last_fullsized":"2023-10-08T05:50:17.702549+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:12:15.056984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020679+0000","last_change":"2023-10-08T05:50:15.717409+0000","last_active":"2023-10-08T05:50:18.020679+0000","last_peered":"2023-10-08T05:50:18.020679+0000","last_clean":"2023-10-08T05:50:18.020679+0000","last_became_active":"2023-10-08T05:50:15.717162+0000","last_became_peered":"2023-10-08T05:50:15.717162+0000","last_unstale":"2023-10-08T05:50:18.020679+0000","last_undegraded":"2023-10-08T05:50:18.020679+0000","last_fullsized":"2023-10-08T05:50:18.020679+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:23.124649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019115+0000","last_change":"2023-10-08T05:50:16.712211+0000","last_active":"2023-10-08T05:50:18.019115+0000","last_peered":"2023-10-08T05:50:18.019115+0000","last_clean":"2023-10-08T05:50:18.019115+0000","last_became_active":"2023-10-08T05:50:16.711742+0000","last_became_peered":"2023-10-08T05:50:16.711742+0000","last_unstale":"2023-10-08T05:50:18.019115+0000","last_undegraded":"2023-10-08T05:50:18.019115+0000","last_fullsized":"2023-10-08T05:50:18.019115+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:56:31.509751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020310+0000","last_change":"2023-10-08T05:50:15.719653+0000","last_active":"2023-10-08T05:50:18.020310+0000","last_peered":"2023-10-08T05:50:18.020310+0000","last_clean":"2023-10-08T05:50:18.020310+0000","last_became_active":"2023-10-08T05:50:15.719347+0000","last_became_peered":"2023-10-08T05:50:15.719347+0000","last_unstale":"2023-10-08T05:50:18.020310+0000","last_undegraded":"2023-10-08T05:50:18.020310+0000","last_fullsized":"2023-10-08T05:50:18.020310+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:34.232951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701642+0000","last_change":"2023-10-08T05:50:16.719655+0000","last_active":"2023-10-08T05:50:17.701642+0000","last_peered":"2023-10-08T05:50:17.701642+0000","last_clean":"2023-10-08T05:50:17.701642+0000","last_became_active":"2023-10-08T05:50:16.719325+0000","last_became_peered":"2023-10-08T05:50:16.719325+0000","last_unstale":"2023-10-08T05:50:17.701642+0000","last_undegraded":"2023-10-08T05:50:17.701642+0000","last_fullsized":"2023-10-08T05:50:17.701642+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:24.728503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020726+0000","last_change":"2023-10-08T05:50:15.705786+0000","last_active":"2023-10-08T05:50:18.020726+0000","last_peered":"2023-10-08T05:50:18.020726+0000","last_clean":"2023-10-08T05:50:18.020726+0000","last_became_active":"2023-10-08T05:50:15.705333+0000","last_became_peered":"2023-10-08T05:50:15.705333+0000","last_unstale":"2023-10-08T05:50:18.020726+0000","last_undegraded":"2023-10-08T05:50:18.020726+0000","last_fullsized":"2023-10-08T05:50:18.020726+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:55:57.117517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019437+0000","last_change":"2023-10-08T05:50:16.711800+0000","last_active":"2023-10-08T05:50:18.019437+0000","last_peered":"2023-10-08T05:50:18.019437+0000","last_clean":"2023-10-08T05:50:18.019437+0000","last_became_active":"2023-10-08T05:50:16.710914+0000","last_became_peered":"2023-10-08T05:50:16.710914+0000","last_unstale":"2023-10-08T05:50:18.019437+0000","last_undegraded":"2023-10-08T05:50:18.019437+0000","last_fullsized":"2023-10-08T05:50:18.019437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:40:18.021260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701176+0000","last_change":"2023-10-08T05:50:15.717050+0000","last_active":"2023-10-08T05:50:17.701176+0000","last_peered":"2023-10-08T05:50:17.701176+0000","last_clean":"2023-10-08T05:50:17.701176+0000","last_became_active":"2023-10-08T05:50:15.716779+0000","last_became_peered":"2023-10-08T05:50:15.716779+0000","last_unstale":"2023-10-08T05:50:17.701176+0000","last_undegraded":"2023-10-08T05:50:17.701176+0000","last_fullsized":"2023-10-08T05:50:17.701176+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:51:43.600172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021353+0000","last_change":"2023-10-08T05:50:16.712625+0000","last_active":"2023-10-08T05:50:18.021353+0000","last_peered":"2023-10-08T05:50:18.021353+0000","last_clean":"2023-10-08T05:50:18.021353+0000","last_became_active":"2023-10-08T05:50:16.712354+0000","last_became_peered":"2023-10-08T05:50:16.712354+0000","last_unstale":"2023-10-08T05:50:18.021353+0000","last_undegraded":"2023-10-08T05:50:18.021353+0000","last_fullsized":"2023-10-08T05:50:18.021353+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:06.575672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701993+0000","last_change":"2023-10-08T05:50:15.715646+0000","last_active":"2023-10-08T05:50:17.701993+0000","last_peered":"2023-10-08T05:50:17.701993+0000","last_clean":"2023-10-08T05:50:17.701993+0000","last_became_active":"2023-10-08T05:50:15.715329+0000","last_became_peered":"2023-10-08T05:50:15.715329+0000","last_unstale":"2023-10-08T05:50:17.701993+0000","last_undegraded":"2023-10-08T05:50:17.701993+0000","last_fullsized":"2023-10-08T05:50:17.701993+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:22.680763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020008+0000","last_change":"2023-10-08T05:50:16.720577+0000","last_active":"2023-10-08T05:50:18.020008+0000","last_peered":"2023-10-08T05:50:18.020008+0000","last_clean":"2023-10-08T05:50:18.020008+0000","last_became_active":"2023-10-08T05:50:16.720180+0000","last_became_peered":"2023-10-08T05:50:16.720180+0000","last_unstale":"2023-10-08T05:50:18.020008+0000","last_undegraded":"2023-10-08T05:50:18.020008+0000","last_fullsized":"2023-10-08T05:50:18.020008+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:18.809941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021700+0000","last_change":"2023-10-08T05:50:15.704703+0000","last_active":"2023-10-08T05:50:18.021700+0000","last_peered":"2023-10-08T05:50:18.021700+0000","last_clean":"2023-10-08T05:50:18.021700+0000","last_became_active":"2023-10-08T05:50:15.704115+0000","last_became_peered":"2023-10-08T05:50:15.704115+0000","last_unstale":"2023-10-08T05:50:18.021700+0000","last_undegraded":"2023-10-08T05:50:18.021700+0000","last_fullsized":"2023-10-08T05:50:18.021700+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:43.882425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.700469+0000","last_change":"2023-10-08T05:50:16.711773+0000","last_active":"2023-10-08T05:50:17.700469+0000","last_peered":"2023-10-08T05:50:17.700469+0000","last_clean":"2023-10-08T05:50:17.700469+0000","last_became_active":"2023-10-08T05:50:16.711476+0000","last_became_peered":"2023-10-08T05:50:16.711476+0000","last_unstale":"2023-10-08T05:50:17.700469+0000","last_undegraded":"2023-10-08T05:50:17.700469+0000","last_fullsized":"2023-10-08T05:50:17.700469+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:26.006424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020273+0000","last_change":"2023-10-08T05:50:15.718436+0000","last_active":"2023-10-08T05:50:18.020273+0000","last_peered":"2023-10-08T05:50:18.020273+0000","last_clean":"2023-10-08T05:50:18.020273+0000","last_became_active":"2023-10-08T05:50:15.717818+0000","last_became_peered":"2023-10-08T05:50:15.717818+0000","last_unstale":"2023-10-08T05:50:18.020273+0000","last_undegraded":"2023-10-08T05:50:18.020273+0000","last_fullsized":"2023-10-08T05:50:18.020273+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:41:11.950183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020203+0000","last_change":"2023-10-08T05:50:16.714230+0000","last_active":"2023-10-08T05:50:18.020203+0000","last_peered":"2023-10-08T05:50:18.020203+0000","last_clean":"2023-10-08T05:50:18.020203+0000","last_became_active":"2023-10-08T05:50:16.713143+0000","last_became_peered":"2023-10-08T05:50:16.713143+0000","last_unstale":"2023-10-08T05:50:18.020203+0000","last_undegraded":"2023-10-08T05:50:18.020203+0000","last_fullsized":"2023-10-08T05:50:18.020203+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:00.613799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701331+0000","last_change":"2023-10-08T05:50:15.707386+0000","last_active":"2023-10-08T05:50:17.701331+0000","last_peered":"2023-10-08T05:50:17.701331+0000","last_clean":"2023-10-08T05:50:17.701331+0000","last_became_active":"2023-10-08T05:50:15.706885+0000","last_became_peered":"2023-10-08T05:50:15.706885+0000","last_unstale":"2023-10-08T05:50:17.701331+0000","last_undegraded":"2023-10-08T05:50:17.701331+0000","last_fullsized":"2023-10-08T05:50:17.701331+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:16.962732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020751+0000","last_change":"2023-10-08T05:50:16.720163+0000","last_active":"2023-10-08T05:50:18.020751+0000","last_peered":"2023-10-08T05:50:18.020751+0000","last_clean":"2023-10-08T05:50:18.020751+0000","last_became_active":"2023-10-08T05:50:16.719918+0000","last_became_peered":"2023-10-08T05:50:16.719918+0000","last_unstale":"2023-10-08T05:50:18.020751+0000","last_undegraded":"2023-10-08T05:50:18.020751+0000","last_fullsized":"2023-10-08T05:50:18.020751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:37.735709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"71'3","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020743+0000","last_change":"2023-10-08T05:50:15.714177+0000","last_active":"2023-10-08T05:50:18.020743+0000","last_peered":"2023-10-08T05:50:18.020743+0000","last_clean":"2023-10-08T05:50:18.020743+0000","last_became_active":"2023-10-08T05:50:15.713862+0000","last_became_peered":"2023-10-08T05:50:15.713862+0000","last_unstale":"2023-10-08T05:50:18.020743+0000","last_undegraded":"2023-10-08T05:50:18.020743+0000","last_fullsized":"2023-10-08T05:50:18.020743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:54:04.997937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701977+0000","last_change":"2023-10-08T05:50:16.707888+0000","last_active":"2023-10-08T05:50:17.701977+0000","last_peered":"2023-10-08T05:50:17.701977+0000","last_clean":"2023-10-08T05:50:17.701977+0000","last_became_active":"2023-10-08T05:50:16.707606+0000","last_became_peered":"2023-10-08T05:50:16.707606+0000","last_unstale":"2023-10-08T05:50:17.701977+0000","last_undegraded":"2023-10-08T05:50:17.701977+0000","last_fullsized":"2023-10-08T05:50:17.701977+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:10.593532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"71'6","reported_seq":18,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019815+0000","last_change":"2023-10-08T05:50:15.709446+0000","last_active":"2023-10-08T05:50:18.019815+0000","last_peered":"2023-10-08T05:50:18.019815+0000","last_clean":"2023-10-08T05:50:18.019815+0000","last_became_active":"2023-10-08T05:50:15.709144+0000","last_became_peered":"2023-10-08T05:50:15.709144+0000","last_unstale":"2023-10-08T05:50:18.019815+0000","last_undegraded":"2023-10-08T05:50:18.019815+0000","last_fullsized":"2023-10-08T05:50:18.019815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:32.793357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020049+0000","last_change":"2023-10-08T05:50:16.711457+0000","last_active":"2023-10-08T05:50:18.020049+0000","last_peered":"2023-10-08T05:50:18.020049+0000","last_clean":"2023-10-08T05:50:18.020049+0000","last_became_active":"2023-10-08T05:50:16.711176+0000","last_became_peered":"2023-10-08T05:50:16.711176+0000","last_unstale":"2023-10-08T05:50:18.020049+0000","last_undegraded":"2023-10-08T05:50:18.020049+0000","last_fullsized":"2023-10-08T05:50:18.020049+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:59:50.014852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020818+0000","last_change":"2023-10-08T05:50:15.713971+0000","last_active":"2023-10-08T05:50:18.020818+0000","last_peered":"2023-10-08T05:50:18.020818+0000","last_clean":"2023-10-08T05:50:18.020818+0000","last_became_active":"2023-10-08T05:50:15.713688+0000","last_became_peered":"2023-10-08T05:50:15.713688+0000","last_unstale":"2023-10-08T05:50:18.020818+0000","last_undegraded":"2023-10-08T05:50:18.020818+0000","last_fullsized":"2023-10-08T05:50:18.020818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:31.224857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021260+0000","last_change":"2023-10-08T05:50:16.714837+0000","last_active":"2023-10-08T05:50:18.021260+0000","last_peered":"2023-10-08T05:50:18.021260+0000","last_clean":"2023-10-08T05:50:18.021260+0000","last_became_active":"2023-10-08T05:50:16.713088+0000","last_became_peered":"2023-10-08T05:50:16.713088+0000","last_unstale":"2023-10-08T05:50:18.021260+0000","last_undegraded":"2023-10-08T05:50:18.021260+0000","last_fullsized":"2023-10-08T05:50:18.021260+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:02.279284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702458+0000","last_change":"2023-10-08T05:50:15.709182+0000","last_active":"2023-10-08T05:50:17.702458+0000","last_peered":"2023-10-08T05:50:17.702458+0000","last_clean":"2023-10-08T05:50:17.702458+0000","last_became_active":"2023-10-08T05:50:15.708901+0000","last_became_peered":"2023-10-08T05:50:15.708901+0000","last_unstale":"2023-10-08T05:50:17.702458+0000","last_undegraded":"2023-10-08T05:50:17.702458+0000","last_fullsized":"2023-10-08T05:50:17.702458+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:40:55.893412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021080+0000","last_change":"2023-10-08T05:50:16.711312+0000","last_active":"2023-10-08T05:50:18.021080+0000","last_peered":"2023-10-08T05:50:18.021080+0000","last_clean":"2023-10-08T05:50:18.021080+0000","last_became_active":"2023-10-08T05:50:16.711040+0000","last_became_peered":"2023-10-08T05:50:16.711040+0000","last_unstale":"2023-10-08T05:50:18.021080+0000","last_undegraded":"2023-10-08T05:50:18.021080+0000","last_fullsized":"2023-10-08T05:50:18.021080+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:12.275035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018978+0000","last_change":"2023-10-08T05:50:15.719279+0000","last_active":"2023-10-08T05:50:18.018978+0000","last_peered":"2023-10-08T05:50:18.018978+0000","last_clean":"2023-10-08T05:50:18.018978+0000","last_became_active":"2023-10-08T05:50:15.718959+0000","last_became_peered":"2023-10-08T05:50:15.718959+0000","last_unstale":"2023-10-08T05:50:18.018978+0000","last_undegraded":"2023-10-08T05:50:18.018978+0000","last_fullsized":"2023-10-08T05:50:18.018978+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:49.346321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021785+0000","last_change":"2023-10-08T05:50:16.721125+0000","last_active":"2023-10-08T05:50:18.021785+0000","last_peered":"2023-10-08T05:50:18.021785+0000","last_clean":"2023-10-08T05:50:18.021785+0000","last_became_active":"2023-10-08T05:50:16.720523+0000","last_became_peered":"2023-10-08T05:50:16.720523+0000","last_unstale":"2023-10-08T05:50:18.021785+0000","last_undegraded":"2023-10-08T05:50:18.021785+0000","last_fullsized":"2023-10-08T05:50:18.021785+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:20.715049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"71'8","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018967+0000","last_change":"2023-10-08T05:50:15.707774+0000","last_active":"2023-10-08T05:50:18.018967+0000","last_peered":"2023-10-08T05:50:18.018967+0000","last_clean":"2023-10-08T05:50:18.018967+0000","last_became_active":"2023-10-08T05:50:15.707447+0000","last_became_peered":"2023-10-08T05:50:15.707447+0000","last_unstale":"2023-10-08T05:50:18.018967+0000","last_undegraded":"2023-10-08T05:50:18.018967+0000","last_fullsized":"2023-10-08T05:50:18.018967+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:17.794915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018874+0000","last_change":"2023-10-08T05:50:16.718007+0000","last_active":"2023-10-08T05:50:18.018874+0000","last_peered":"2023-10-08T05:50:18.018874+0000","last_clean":"2023-10-08T05:50:18.018874+0000","last_became_active":"2023-10-08T05:50:16.717577+0000","last_became_peered":"2023-10-08T05:50:16.717577+0000","last_unstale":"2023-10-08T05:50:18.018874+0000","last_undegraded":"2023-10-08T05:50:18.018874+0000","last_fullsized":"2023-10-08T05:50:18.018874+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:56.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.022005+0000","last_change":"2023-10-08T05:50:15.697449+0000","last_active":"2023-10-08T05:50:18.022005+0000","last_peered":"2023-10-08T05:50:18.022005+0000","last_clean":"2023-10-08T05:50:18.022005+0000","last_became_active":"2023-10-08T05:50:15.696566+0000","last_became_peered":"2023-10-08T05:50:15.696566+0000","last_unstale":"2023-10-08T05:50:18.022005+0000","last_undegraded":"2023-10-08T05:50:18.022005+0000","last_fullsized":"2023-10-08T05:50:18.022005+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:56.179248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.017843+0000","last_change":"2023-10-08T05:50:16.722573+0000","last_active":"2023-10-08T05:50:18.017843+0000","last_peered":"2023-10-08T05:50:18.017843+0000","last_clean":"2023-10-08T05:50:18.017843+0000","last_became_active":"2023-10-08T05:50:16.722189+0000","last_became_peered":"2023-10-08T05:50:16.722189+0000","last_unstale":"2023-10-08T05:50:18.017843+0000","last_undegraded":"2023-10-08T05:50:18.017843+0000","last_fullsized":"2023-10-08T05:50:18.017843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:55.228608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.697459+0000","last_change":"2023-10-08T05:50:15.714591+0000","last_active":"2023-10-08T05:50:17.697459+0000","last_peered":"2023-10-08T05:50:17.697459+0000","last_clean":"2023-10-08T05:50:17.697459+0000","last_became_active":"2023-10-08T05:50:15.714304+0000","last_became_peered":"2023-10-08T05:50:15.714304+0000","last_unstale":"2023-10-08T05:50:17.697459+0000","last_undegraded":"2023-10-08T05:50:17.697459+0000","last_fullsized":"2023-10-08T05:50:17.697459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:11.230748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.700882+0000","last_change":"2023-10-08T05:50:16.720026+0000","last_active":"2023-10-08T05:50:17.700882+0000","last_peered":"2023-10-08T05:50:17.700882+0000","last_clean":"2023-10-08T05:50:17.700882+0000","last_became_active":"2023-10-08T05:50:16.719742+0000","last_became_peered":"2023-10-08T05:50:16.719742+0000","last_unstale":"2023-10-08T05:50:17.700882+0000","last_undegraded":"2023-10-08T05:50:17.700882+0000","last_fullsized":"2023-10-08T05:50:17.700882+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:10.481059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.703014+0000","last_change":"2023-10-08T05:50:15.716363+0000","last_active":"2023-10-08T05:50:17.703014+0000","last_peered":"2023-10-08T05:50:17.703014+0000","last_clean":"2023-10-08T05:50:17.703014+0000","last_became_active":"2023-10-08T05:50:15.715920+0000","last_became_peered":"2023-10-08T05:50:15.715920+0000","last_unstale":"2023-10-08T05:50:17.703014+0000","last_undegraded":"2023-10-08T05:50:17.703014+0000","last_fullsized":"2023-10-08T05:50:17.703014+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:10:14.764591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018503+0000","last_change":"2023-10-08T05:50:16.713948+0000","last_active":"2023-10-08T05:50:18.018503+0000","last_peered":"2023-10-08T05:50:18.018503+0000","last_clean":"2023-10-08T05:50:18.018503+0000","last_became_active":"2023-10-08T05:50:16.713664+0000","last_became_peered":"2023-10-08T05:50:16.713664+0000","last_unstale":"2023-10-08T05:50:18.018503+0000","last_undegraded":"2023-10-08T05:50:18.018503+0000","last_fullsized":"2023-10-08T05:50:18.018503+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:42:15.329615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.703004+0000","last_change":"2023-10-08T05:50:15.715013+0000","last_active":"2023-10-08T05:50:17.703004+0000","last_peered":"2023-10-08T05:50:17.703004+0000","last_clean":"2023-10-08T05:50:17.703004+0000","last_became_active":"2023-10-08T05:50:15.714660+0000","last_became_peered":"2023-10-08T05:50:15.714660+0000","last_unstale":"2023-10-08T05:50:17.703004+0000","last_undegraded":"2023-10-08T05:50:17.703004+0000","last_fullsized":"2023-10-08T05:50:17.703004+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:10.546542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020249+0000","last_change":"2023-10-08T05:50:16.714188+0000","last_active":"2023-10-08T05:50:18.020249+0000","last_peered":"2023-10-08T05:50:18.020249+0000","last_clean":"2023-10-08T05:50:18.020249+0000","last_became_active":"2023-10-08T05:50:16.713873+0000","last_became_peered":"2023-10-08T05:50:16.713873+0000","last_unstale":"2023-10-08T05:50:18.020249+0000","last_undegraded":"2023-10-08T05:50:18.020249+0000","last_fullsized":"2023-10-08T05:50:18.020249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:18:36.163267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018743+0000","last_change":"2023-10-08T05:50:15.714644+0000","last_active":"2023-10-08T05:50:18.018743+0000","last_peered":"2023-10-08T05:50:18.018743+0000","last_clean":"2023-10-08T05:50:18.018743+0000","last_became_active":"2023-10-08T05:50:15.714383+0000","last_became_peered":"2023-10-08T05:50:15.714383+0000","last_unstale":"2023-10-08T05:50:18.018743+0000","last_undegraded":"2023-10-08T05:50:18.018743+0000","last_fullsized":"2023-10-08T05:50:18.018743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:03.007678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.703127+0000","last_change":"2023-10-08T05:50:16.701006+0000","last_active":"2023-10-08T05:50:17.703127+0000","last_peered":"2023-10-08T05:50:17.703127+0000","last_clean":"2023-10-08T05:50:17.703127+0000","last_became_active":"2023-10-08T05:50:16.700540+0000","last_became_peered":"2023-10-08T05:50:16.700540+0000","last_unstale":"2023-10-08T05:50:17.703127+0000","last_undegraded":"2023-10-08T05:50:17.703127+0000","last_fullsized":"2023-10-08T05:50:17.703127+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:19:45.567174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019349+0000","last_change":"2023-10-08T05:50:15.708697+0000","last_active":"2023-10-08T05:50:18.019349+0000","last_peered":"2023-10-08T05:50:18.019349+0000","last_clean":"2023-10-08T05:50:18.019349+0000","last_became_active":"2023-10-08T05:50:15.708426+0000","last_became_peered":"2023-10-08T05:50:15.708426+0000","last_unstale":"2023-10-08T05:50:18.019349+0000","last_undegraded":"2023-10-08T05:50:18.019349+0000","last_fullsized":"2023-10-08T05:50:18.019349+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:49.346321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021653+0000","last_change":"2023-10-08T05:50:16.720028+0000","last_active":"2023-10-08T05:50:18.021653+0000","last_peered":"2023-10-08T05:50:18.021653+0000","last_clean":"2023-10-08T05:50:18.021653+0000","last_became_active":"2023-10-08T05:50:16.719746+0000","last_became_peered":"2023-10-08T05:50:16.719746+0000","last_unstale":"2023-10-08T05:50:18.021653+0000","last_undegraded":"2023-10-08T05:50:18.021653+0000","last_fullsized":"2023-10-08T05:50:18.021653+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:45.849982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020238+0000","last_change":"2023-10-08T05:50:15.712066+0000","last_active":"2023-10-08T05:50:18.020238+0000","last_peered":"2023-10-08T05:50:18.020238+0000","last_clean":"2023-10-08T05:50:18.020238+0000","last_became_active":"2023-10-08T05:50:15.711555+0000","last_became_peered":"2023-10-08T05:50:15.711555+0000","last_unstale":"2023-10-08T05:50:18.020238+0000","last_undegraded":"2023-10-08T05:50:18.020238+0000","last_fullsized":"2023-10-08T05:50:18.020238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:43.529374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019682+0000","last_change":"2023-10-08T05:50:16.707290+0000","last_active":"2023-10-08T05:50:18.019682+0000","last_peered":"2023-10-08T05:50:18.019682+0000","last_clean":"2023-10-08T05:50:18.019682+0000","last_became_active":"2023-10-08T05:50:16.707015+0000","last_became_peered":"2023-10-08T05:50:16.707015+0000","last_unstale":"2023-10-08T05:50:18.019682+0000","last_undegraded":"2023-10-08T05:50:18.019682+0000","last_fullsized":"2023-10-08T05:50:18.019682+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:43:29.239765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.696815+0000","last_change":"2023-10-08T05:50:15.713684+0000","last_active":"2023-10-08T05:50:17.696815+0000","last_peered":"2023-10-08T05:50:17.696815+0000","last_clean":"2023-10-08T05:50:17.696815+0000","last_became_active":"2023-10-08T05:50:15.713412+0000","last_became_peered":"2023-10-08T05:50:15.713412+0000","last_unstale":"2023-10-08T05:50:17.696815+0000","last_undegraded":"2023-10-08T05:50:17.696815+0000","last_fullsized":"2023-10-08T05:50:17.696815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:46.743198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020068+0000","last_change":"2023-10-08T05:50:16.714910+0000","last_active":"2023-10-08T05:50:18.020068+0000","last_peered":"2023-10-08T05:50:18.020068+0000","last_clean":"2023-10-08T05:50:18.020068+0000","last_became_active":"2023-10-08T05:50:16.714607+0000","last_became_peered":"2023-10-08T05:50:16.714607+0000","last_unstale":"2023-10-08T05:50:18.020068+0000","last_undegraded":"2023-10-08T05:50:18.020068+0000","last_fullsized":"2023-10-08T05:50:18.020068+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:57:41.204748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702971+0000","last_change":"2023-10-08T05:50:15.715614+0000","last_active":"2023-10-08T05:50:17.702971+0000","last_peered":"2023-10-08T05:50:17.702971+0000","last_clean":"2023-10-08T05:50:17.702971+0000","last_became_active":"2023-10-08T05:50:15.715309+0000","last_became_peered":"2023-10-08T05:50:15.715309+0000","last_unstale":"2023-10-08T05:50:17.702971+0000","last_undegraded":"2023-10-08T05:50:17.702971+0000","last_fullsized":"2023-10-08T05:50:17.702971+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:13.374218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.018400+0000","last_change":"2023-10-08T05:50:16.689939+0000","last_active":"2023-10-08T05:50:18.018400+0000","last_peered":"2023-10-08T05:50:18.018400+0000","last_clean":"2023-10-08T05:50:18.018400+0000","last_became_active":"2023-10-08T05:50:16.689200+0000","last_became_peered":"2023-10-08T05:50:16.689200+0000","last_unstale":"2023-10-08T05:50:18.018400+0000","last_undegraded":"2023-10-08T05:50:18.018400+0000","last_fullsized":"2023-10-08T05:50:18.018400+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:54:01.709935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020825+0000","last_change":"2023-10-08T05:50:15.719651+0000","last_active":"2023-10-08T05:50:18.020825+0000","last_peered":"2023-10-08T05:50:18.020825+0000","last_clean":"2023-10-08T05:50:18.020825+0000","last_became_active":"2023-10-08T05:50:15.718971+0000","last_became_peered":"2023-10-08T05:50:15.718971+0000","last_unstale":"2023-10-08T05:50:18.020825+0000","last_undegraded":"2023-10-08T05:50:18.020825+0000","last_fullsized":"2023-10-08T05:50:18.020825+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:56:30.873451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019151+0000","last_change":"2023-10-08T05:50:16.703551+0000","last_active":"2023-10-08T05:50:18.019151+0000","last_peered":"2023-10-08T05:50:18.019151+0000","last_clean":"2023-10-08T05:50:18.019151+0000","last_became_active":"2023-10-08T05:50:16.703259+0000","last_became_peered":"2023-10-08T05:50:16.703259+0000","last_unstale":"2023-10-08T05:50:18.019151+0000","last_undegraded":"2023-10-08T05:50:18.019151+0000","last_fullsized":"2023-10-08T05:50:18.019151+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:26.259393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021070+0000","last_change":"2023-10-08T05:50:15.706889+0000","last_active":"2023-10-08T05:50:18.021070+0000","last_peered":"2023-10-08T05:50:18.021070+0000","last_clean":"2023-10-08T05:50:18.021070+0000","last_became_active":"2023-10-08T05:50:15.706576+0000","last_became_peered":"2023-10-08T05:50:15.706576+0000","last_unstale":"2023-10-08T05:50:18.021070+0000","last_undegraded":"2023-10-08T05:50:18.021070+0000","last_fullsized":"2023-10-08T05:50:18.021070+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:17:29.014240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020854+0000","last_change":"2023-10-08T05:50:16.727005+0000","last_active":"2023-10-08T05:50:18.020854+0000","last_peered":"2023-10-08T05:50:18.020854+0000","last_clean":"2023-10-08T05:50:18.020854+0000","last_became_active":"2023-10-08T05:50:16.726737+0000","last_became_peered":"2023-10-08T05:50:16.726737+0000","last_unstale":"2023-10-08T05:50:18.020854+0000","last_undegraded":"2023-10-08T05:50:18.020854+0000","last_fullsized":"2023-10-08T05:50:18.020854+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:17:49.632967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019247+0000","last_change":"2023-10-08T05:50:15.708587+0000","last_active":"2023-10-08T05:50:18.019247+0000","last_peered":"2023-10-08T05:50:18.019247+0000","last_clean":"2023-10-08T05:50:18.019247+0000","last_became_active":"2023-10-08T05:50:15.707967+0000","last_became_peered":"2023-10-08T05:50:15.707967+0000","last_unstale":"2023-10-08T05:50:18.019247+0000","last_undegraded":"2023-10-08T05:50:18.019247+0000","last_fullsized":"2023-10-08T05:50:18.019247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:56:30.873451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019722+0000","last_change":"2023-10-08T05:50:16.714941+0000","last_active":"2023-10-08T05:50:18.019722+0000","last_peered":"2023-10-08T05:50:18.019722+0000","last_clean":"2023-10-08T05:50:18.019722+0000","last_became_active":"2023-10-08T05:50:16.714682+0000","last_became_peered":"2023-10-08T05:50:16.714682+0000","last_unstale":"2023-10-08T05:50:18.019722+0000","last_undegraded":"2023-10-08T05:50:18.019722+0000","last_fullsized":"2023-10-08T05:50:18.019722+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:06.575672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019592+0000","last_change":"2023-10-08T05:50:15.709941+0000","last_active":"2023-10-08T05:50:18.019592+0000","last_peered":"2023-10-08T05:50:18.019592+0000","last_clean":"2023-10-08T05:50:18.019592+0000","last_became_active":"2023-10-08T05:50:15.709463+0000","last_became_peered":"2023-10-08T05:50:15.709463+0000","last_unstale":"2023-10-08T05:50:18.019592+0000","last_undegraded":"2023-10-08T05:50:18.019592+0000","last_fullsized":"2023-10-08T05:50:18.019592+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:54:04.997937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021314+0000","last_change":"2023-10-08T05:50:16.716088+0000","last_active":"2023-10-08T05:50:18.021314+0000","last_peered":"2023-10-08T05:50:18.021314+0000","last_clean":"2023-10-08T05:50:18.021314+0000","last_became_active":"2023-10-08T05:50:16.715306+0000","last_became_peered":"2023-10-08T05:50:16.715306+0000","last_unstale":"2023-10-08T05:50:18.021314+0000","last_undegraded":"2023-10-08T05:50:18.021314+0000","last_fullsized":"2023-10-08T05:50:18.021314+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:16.111403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.701622+0000","last_change":"2023-10-08T05:50:15.705479+0000","last_active":"2023-10-08T05:50:17.701622+0000","last_peered":"2023-10-08T05:50:17.701622+0000","last_clean":"2023-10-08T05:50:17.701622+0000","last_became_active":"2023-10-08T05:50:15.705052+0000","last_became_peered":"2023-10-08T05:50:15.705052+0000","last_unstale":"2023-10-08T05:50:17.701622+0000","last_undegraded":"2023-10-08T05:50:17.701622+0000","last_fullsized":"2023-10-08T05:50:17.701622+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:53.868245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.020095+0000","last_change":"2023-10-08T05:50:16.712022+0000","last_active":"2023-10-08T05:50:18.020095+0000","last_peered":"2023-10-08T05:50:18.020095+0000","last_clean":"2023-10-08T05:50:18.020095+0000","last_became_active":"2023-10-08T05:50:16.711732+0000","last_became_peered":"2023-10-08T05:50:16.711732+0000","last_unstale":"2023-10-08T05:50:18.020095+0000","last_undegraded":"2023-10-08T05:50:18.020095+0000","last_fullsized":"2023-10-08T05:50:18.020095+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:01:14.781042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.019580+0000","last_change":"2023-10-08T05:50:15.713100+0000","last_active":"2023-10-08T05:50:18.019580+0000","last_peered":"2023-10-08T05:50:18.019580+0000","last_clean":"2023-10-08T05:50:18.019580+0000","last_became_active":"2023-10-08T05:50:15.712265+0000","last_became_peered":"2023-10-08T05:50:15.712265+0000","last_unstale":"2023-10-08T05:50:18.019580+0000","last_undegraded":"2023-10-08T05:50:18.019580+0000","last_fullsized":"2023-10-08T05:50:18.019580+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:14:57.759257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:17.702031+0000","last_change":"2023-10-08T05:50:16.712877+0000","last_active":"2023-10-08T05:50:17.702031+0000","last_peered":"2023-10-08T05:50:17.702031+0000","last_clean":"2023-10-08T05:50:17.702031+0000","last_became_active":"2023-10-08T05:50:16.712566+0000","last_became_peered":"2023-10-08T05:50:16.712566+0000","last_unstale":"2023-10-08T05:50:17.702031+0000","last_undegraded":"2023-10-08T05:50:17.702031+0000","last_fullsized":"2023-10-08T05:50:17.702031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:15.667144+0000","last_clean_scrub_stamp":"2023-10-08T05:50:15.667144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:10.603639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-10-08T05:50:18.021866+0000","last_change":"2023-10-08T05:50:15.714591+0000","last_active":"2023-10-08T05:50:18.021866+0000","last_peered":"2023-10-08T05:50:18.021866+0000","last_clean":"2023-10-08T05:50:18.021866+0000","last_became_active":"2023-10-08T05:50:15.714324+0000","last_became_peered":"2023-10-08T05:50:15.714324+0000","last_unstale":"2023-10-08T05:50:18.021866+0000","last_undegraded":"2023-10-08T05:50:18.021866+0000","last_fullsized":"2023-10-08T05:50:18.021866+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:50:14.652118+0000","last_clean_scrub_stamp":"2023-10-08T05:50:14.652118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:45.839989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"59'139","reported_seq":298,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-10-08T05:50:16.698562+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:50:16.698562+0000","last_peered":"2023-10-08T05:50:16.698562+0000","last_clean":"2023-10-08T05:50:16.698562+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:50:16.698562+0000","last_undegraded":"2023-10-08T05:50:16.698562+0000","last_fullsized":"2023-10-08T05:50:16.698562+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575021,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58972,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312868,"statfs":{"total":96636764160,"available":96576376832,"internally_reserved":0,"allocated":1454080,"data_stored":1015360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.258,"15min":1.2050000000000001},"min":{"1min":0.72599999999999998,"5min":0.68500000000000005,"15min":0.68000000000000005},"max":{"1min":1.6499999999999999,"5min":1.792,"15min":2.2360000000000002},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2230000000000001,"15min":1.1919999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.70699999999999996},"max":{"1min":1.5649999999999999,"5min":1.726,"15min":1.726},"last":1.3300000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.284,"15min":1.236},"min":{"1min":0.83999999999999997,"5min":0.82499999999999996,"15min":0.68600000000000005},"max":{"1min":1.6060000000000001,"5min":1.8360000000000001,"15min":1.972},"last":1.27},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.2769999999999999,"15min":1.256},"min":{"1min":0.86899999999999999,"5min":0.82599999999999996,"15min":0.69199999999999995},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.9379999999999999},"last":1.4199999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2889999999999999,"15min":1.208},"min":{"1min":0.93400000000000005,"5min":0.90200000000000002,"15min":0.65100000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8280000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.335},"min":{"1min":0.82299999999999995,"5min":0.78400000000000003,"15min":0.75600000000000001},"max":{"1min":1.6870000000000001,"5min":1.752,"15min":2.3849999999999998},"last":1.3839999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3069999999999999,"15min":1.276},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6459999999999999,"5min":1.746,"15min":2.2919999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3720000000000001,"15min":1.3799999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.776,"5min":1.8200000000000001,"15min":2.5579999999999998},"last":1.5489999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2310000000000001,"15min":1.2869999999999999},"min":{"1min":0.95799999999999996,"5min":0.78500000000000003,"15min":0.65000000000000002},"max":{"1min":1.7250000000000001,"5min":1.752,"15min":2.4529999999999998},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.329,"15min":1.347},"min":{"1min":0.97899999999999998,"5min":0.89800000000000002,"15min":0.46800000000000003},"max":{"1min":1.909,"5min":1.909,"15min":2.677},"last":1.3640000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3999999999999999,"15min":1.419},"min":{"1min":1.0329999999999999,"5min":0.92800000000000005,"15min":0.54200000000000004},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":3.3319999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.417,"15min":1.4450000000000001},"min":{"1min":1.0820000000000001,"5min":0.89200000000000002,"15min":0.58799999999999997},"max":{"1min":1.7909999999999999,"5min":1.8859999999999999,"15min":2.726},"last":1.6779999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.5,"15min":1.5369999999999999},"min":{"1min":1.2569999999999999,"5min":1.028,"15min":0.68300000000000005},"max":{"1min":1.8440000000000001,"5min":2.0030000000000001,"15min":3.5249999999999999},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.466,"15min":1.468},"min":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.52000000000000002},"max":{"1min":1.782,"5min":1.9550000000000001,"15min":2.4700000000000002},"last":1.3080000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575020,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":872448,"data_stored":432184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.319,"15min":1.2509999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.639,"5min":1.913,"15min":1.913},"last":1.331},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.292,"15min":1.2929999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68300000000000005},"max":{"1min":1.4990000000000001,"5min":1.8089999999999999,"15min":2.222},"last":0.98499999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.167,"15min":1.159},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.573,"5min":1.681,"15min":1.9530000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.2290000000000001,"15min":1.1930000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5780000000000001,"5min":1.7110000000000001,"15min":1.8029999999999999},"last":1.135}]},{"osd":2,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.24,"15min":1.2010000000000001},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.70399999999999996},"max":{"1min":1.7490000000000001,"5min":1.796,"15min":1.796},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.2649999999999999,"15min":1.2589999999999999},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.58699999999999997},"max":{"1min":1.3560000000000001,"5min":1.8520000000000001,"15min":1.8620000000000001},"last":1.5900000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.389,"15min":1.3640000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.5089999999999999,"5min":1.905,"15min":1.905},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.4570000000000001,"15min":1.446},"min":{"1min":1.046,"5min":1.046,"15min":0.86399999999999999},"max":{"1min":1.679,"5min":1.9450000000000001,"15min":2.056},"last":1.385}]},{"osd":4,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.294,"15min":1.3280000000000001},"min":{"1min":0.86699999999999999,"5min":0.76300000000000001,"15min":0.73499999999999999},"max":{"1min":1.7350000000000001,"5min":1.8169999999999999,"15min":2.278},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3700000000000001,"15min":1.3919999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.64300000000000002},"max":{"1min":1.764,"5min":1.8819999999999999,"15min":2.3399999999999999},"last":1.202}]},{"osd":5,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3160000000000001,"15min":1.383},"min":{"1min":1.0289999999999999,"5min":0.86899999999999999,"15min":0.79600000000000004},"max":{"1min":1.619,"5min":1.833,"15min":4.3099999999999996},"last":1.665},{"interface":"front","average":{"1min":1.407,"5min":1.492,"15min":1.4910000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.871,"5min":1.871,"15min":2.1070000000000002},"last":1.7170000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:49:26 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.464,"15min":1.53},"min":{"1min":1.03,"5min":0.97899999999999998,"15min":0.77200000000000002},"max":{"1min":1.819,"5min":1.9159999999999999,"15min":2.3959999999999999},"last":1.506},{"interface":"front","average":{"1min":1.403,"5min":1.4299999999999999,"15min":1.5069999999999999},"min":{"1min":0.97799999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.726,"5min":1.946,"15min":4.1550000000000002},"last":1.9710000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575020,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":431532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.1200000000000001,"15min":1.1299999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.46700000000000003},"max":{"1min":1.363,"5min":1.546,"15min":2.302},"last":1.02},{"interface":"front","average":{"1min":1.236,"5min":1.2170000000000001,"15min":1.181},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.673,"5min":1.758,"15min":3.5529999999999999},"last":1.206}]},{"osd":1,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2110000000000001,"15min":1.2030000000000001},"min":{"1min":0.95799999999999996,"5min":0.79900000000000004,"15min":0.72399999999999998},"max":{"1min":1.845,"5min":1.845,"15min":2.351},"last":1.8899999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2829999999999999,"15min":1.2689999999999999},"min":{"1min":0.87,"5min":0.84499999999999997,"15min":0.64700000000000002},"max":{"1min":1.657,"5min":1.754,"15min":2.1789999999999998},"last":1.597}]},{"osd":2,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.2210000000000001,"15min":1.246},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.69399999999999995},"max":{"1min":1.5269999999999999,"5min":1.6439999999999999,"15min":3.6739999999999999},"last":1.083},{"interface":"front","average":{"1min":1.268,"5min":1.29,"15min":1.2609999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.79100000000000004},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":2.1110000000000002},"last":1.6639999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.3440000000000001,"15min":1.379},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.63500000000000001},"max":{"1min":1.7450000000000001,"5min":1.8,"15min":6.782},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.393,"15min":1.4339999999999999},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.84799999999999998},"max":{"1min":1.8160000000000001,"5min":2.2280000000000002,"15min":6.6189999999999998},"last":1.387}]},{"osd":4,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2629999999999999,"15min":1.335},"min":{"1min":0.83199999999999996,"5min":0.82899999999999996,"15min":0.53600000000000003},"max":{"1min":1.885,"5min":1.885,"15min":6.1349999999999998},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.484,"5min":1.4159999999999999,"15min":1.4610000000000001},"min":{"1min":1.1120000000000001,"5min":0.95099999999999996,"15min":0.51400000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":7.0069999999999997},"last":1.8480000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3819999999999999,"15min":1.444},"min":{"1min":1.0229999999999999,"5min":0.84299999999999997,"15min":0.57699999999999996},"max":{"1min":1.6100000000000001,"5min":2.359,"15min":7.2569999999999997},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.4510000000000001,"15min":1.494},"min":{"1min":1.2230000000000001,"5min":1.04,"15min":0.59799999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":6.2199999999999998},"last":1.7889999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:49:36 2023","interfaces":[{"interface":"back","average":{"1min":1.5860000000000001,"5min":1.5229999999999999,"15min":1.5840000000000001},"min":{"1min":1.1679999999999999,"5min":1.018,"15min":0.53900000000000003},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":6.9279999999999999},"last":1.9450000000000001},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.5609999999999999,"15min":1.6000000000000001},"min":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.61699999999999999},"max":{"1min":1.996,"5min":2.4460000000000002,"15min":7.335},"last":1.3049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575020,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":856064,"data_stored":425450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3,"15min":1.2749999999999999},"min":{"1min":0.99299999999999999,"5min":0.80900000000000005,"15min":0.65300000000000002},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.863},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4059999999999999,"15min":1.3720000000000001},"min":{"1min":1.038,"5min":1.0069999999999999,"15min":0.78700000000000003},"max":{"1min":1.756,"5min":1.8400000000000001,"15min":2.0680000000000001},"last":1.4730000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.2709999999999999,"15min":1.252},"min":{"1min":0.81799999999999995,"5min":0.81399999999999995,"15min":0.65500000000000003},"max":{"1min":1.5720000000000001,"5min":1.7050000000000001,"15min":2.3799999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.2110000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.556,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.5089999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.24,"15min":1.2210000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.69599999999999995},"max":{"1min":1.548,"5min":1.6759999999999999,"15min":1.77},"last":1.544},{"interface":"front","average":{"1min":1.286,"5min":1.3320000000000001,"15min":1.345},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.748},"max":{"1min":1.5920000000000001,"5min":1.9079999999999999,"15min":2.0710000000000002},"last":1.2669999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2849999999999999,"15min":1.284},"min":{"1min":0.89300000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.5669999999999999,"5min":1.96,"15min":1.96},"last":1.155},{"interface":"front","average":{"1min":1.369,"5min":1.3420000000000001,"15min":1.377},"min":{"1min":0.96399999999999997,"5min":0.94699999999999995,"15min":0.76900000000000002},"max":{"1min":1.7230000000000001,"5min":1.8320000000000001,"15min":2.1080000000000001},"last":1.4259999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.321,"15min":1.3460000000000001},"min":{"1min":0.91900000000000004,"5min":0.89800000000000002,"15min":0.45700000000000002},"max":{"1min":1.7230000000000001,"5min":1.8720000000000001,"15min":2.2370000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.339,"15min":1.387},"min":{"1min":1.0700000000000001,"5min":0.89300000000000002,"15min":0.58199999999999996},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":2.2629999999999999},"last":1.323}]},{"osd":6,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3089999999999999,"15min":1.383},"min":{"1min":0.878,"5min":0.76900000000000002,"15min":0.55000000000000004},"max":{"1min":1.625,"5min":1.821,"15min":2.2080000000000002},"last":1.214},{"interface":"front","average":{"1min":1.427,"5min":1.47,"15min":1.52},"min":{"1min":1.0660000000000001,"5min":1.0409999999999999,"15min":0.70399999999999996},"max":{"1min":1.79,"5min":1.8029999999999999,"15min":2.5150000000000001},"last":1.371}]},{"osd":7,"last update":"Sun Oct 8 05:49:32 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.448,"15min":1.4850000000000001},"min":{"1min":1,"5min":1,"15min":0.73699999999999999},"max":{"1min":1.6799999999999999,"5min":2.0529999999999999,"15min":2.5539999999999998},"last":1.579},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.502,"15min":1.571},"min":{"1min":1.212,"5min":0.94799999999999995,"15min":0.67400000000000004},"max":{"1min":1.782,"5min":1.782,"15min":2.5489999999999999},"last":0.97199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36204,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335636,"statfs":{"total":96636764160,"available":96599691264,"internally_reserved":0,"allocated":880640,"data_stored":433166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.95399999999999996,"15min":0.89900000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.29599999999999999},"max":{"1min":1.5269999999999999,"5min":1.72,"15min":1.72},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.117,"15min":1.0700000000000001},"min":{"1min":0.83399999999999996,"5min":0.61299999999999999,"15min":0.39000000000000001},"max":{"1min":1.5,"5min":1.548,"15min":2.3199999999999998},"last":1.0920000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.98599999999999999,"15min":0.94399999999999995},"min":{"1min":0.751,"5min":0.501,"15min":0.373},"max":{"1min":1.1890000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.234,"15min":1.1719999999999999},"min":{"1min":0.94999999999999996,"5min":0.69699999999999995,"15min":0.436},"max":{"1min":1.577,"5min":1.6220000000000001,"15min":1.6619999999999999},"last":1.1879999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.083,"15min":1.038},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.47699999999999998},"max":{"1min":1.631,"5min":1.645,"15min":1.645},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.1990000000000001,"15min":1.113},"min":{"1min":0.874,"5min":0.746,"15min":0.41899999999999998},"max":{"1min":1.6819999999999999,"5min":1.788,"15min":1.788},"last":1.0349999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2350000000000001,"15min":1.206},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.34200000000000003},"max":{"1min":1.5,"5min":1.677,"15min":2.5059999999999998},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3979999999999999,"15min":1.355},"min":{"1min":1.022,"5min":0.93400000000000005,"15min":0.53800000000000003},"max":{"1min":1.5369999999999999,"5min":1.8380000000000001,"15min":1.9930000000000001},"last":1.345}]},{"osd":5,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3069999999999999,"15min":1.2709999999999999},"min":{"1min":1.0409999999999999,"5min":0.73699999999999999,"15min":0.624},"max":{"1min":1.621,"5min":1.8240000000000001,"15min":1.9219999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.405,"5min":1.3859999999999999,"15min":1.3340000000000001},"min":{"1min":1.212,"5min":0.78500000000000003,"15min":0.59599999999999997},"max":{"1min":2.0219999999999998,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.3759999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3100000000000001,"15min":1.3160000000000001},"min":{"1min":1.0469999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5589999999999999,"5min":1.776,"15min":2.6150000000000002},"last":1.222},{"interface":"front","average":{"1min":1.494,"5min":1.4470000000000001,"15min":1.423},"min":{"1min":1.294,"5min":0.86699999999999999,"15min":0.70099999999999996},"max":{"1min":1.7689999999999999,"5min":2.0489999999999999,"15min":2.1259999999999999},"last":1.3999999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:49:31 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.51,"15min":1.47},"min":{"1min":1.222,"5min":1.0700000000000001,"15min":0.72299999999999998},"max":{"1min":2.085,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.385,"15min":1.399},"min":{"1min":0.99399999999999999,"5min":0.79300000000000004,"15min":0.66900000000000004},"max":{"1min":1.6870000000000001,"5min":1.8200000000000001,"15min":2.7879999999999998},"last":1.3220000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58412,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313428,"statfs":{"total":96636764160,"available":96576950272,"internally_reserved":0,"allocated":876544,"data_stored":433144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.93999999999999995,"15min":0.90600000000000003},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.37},"max":{"1min":1.2549999999999999,"5min":1.5209999999999999,"15min":2.4009999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.155,"5min":1.0760000000000001,"15min":1.0329999999999999},"min":{"1min":0.83299999999999996,"5min":0.71999999999999997,"15min":0.47499999999999998},"max":{"1min":1.5309999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.5529999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.99299999999999999,"15min":0.96899999999999997},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.48099999999999998},"max":{"1min":1.2090000000000001,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.056,"5min":1.1399999999999999,"15min":1.1060000000000001},"min":{"1min":0.83499999999999996,"5min":0.78300000000000003,"15min":0.42999999999999999},"max":{"1min":1.387,"5min":2.1469999999999998,"15min":2.1469999999999998},"last":1.024}]},{"osd":3,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.1719999999999999,"15min":1.1439999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.54100000000000004},"max":{"1min":1.4970000000000001,"5min":1.7370000000000001,"15min":1.8120000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.28,"5min":1.2509999999999999,"15min":1.236},"min":{"1min":1.087,"5min":0.81799999999999995,"15min":0.59399999999999997},"max":{"1min":1.4890000000000001,"5min":1.847,"15min":2.3889999999999998},"last":1.1479999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.4059999999999999,"15min":1.3939999999999999},"min":{"1min":1.153,"5min":1.0860000000000001,"15min":0.74199999999999999},"max":{"1min":1.651,"5min":1.9279999999999999,"15min":2.6419999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.2849999999999999,"15min":1.288},"min":{"1min":1.0129999999999999,"5min":0.93300000000000005,"15min":0.63100000000000001},"max":{"1min":1.6619999999999999,"5min":1.8859999999999999,"15min":2.2530000000000001},"last":1.105}]},{"osd":5,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.395,"15min":1.383},"min":{"1min":1.073,"5min":0.96599999999999997,"15min":0.75600000000000001},"max":{"1min":1.8680000000000001,"5min":1.907,"15min":2.3039999999999998},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.498,"5min":1.5029999999999999,"15min":1.474},"min":{"1min":1.206,"5min":1.2050000000000001,"15min":0.79400000000000004},"max":{"1min":1.9370000000000001,"5min":2.1309999999999998,"15min":2.8170000000000002},"last":1.413}]},{"osd":6,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3859999999999999,"15min":1.373},"min":{"1min":1.1240000000000001,"5min":1.01,"15min":0.77500000000000002},"max":{"1min":1.72,"5min":1.9239999999999999,"15min":2.851},"last":1.391},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5009999999999999,"15min":1.4790000000000001},"min":{"1min":1.145,"5min":1.125,"15min":0.63900000000000001},"max":{"1min":2.044,"5min":2.1829999999999998,"15min":2.556},"last":1.466}]},{"osd":7,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.458,"15min":1.452},"min":{"1min":1.2190000000000001,"5min":1.05,"15min":0.87},"max":{"1min":1.7669999999999999,"5min":2.0369999999999999,"15min":3.0529999999999999},"last":1.296},{"interface":"front","average":{"1min":1.52,"5min":1.5429999999999999,"15min":1.532},"min":{"1min":1.2629999999999999,"5min":1.149,"15min":0.61799999999999999},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.4729999999999999},"last":1.3540000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335644,"statfs":{"total":96636764160,"available":96599699456,"internally_reserved":0,"allocated":872448,"data_stored":431692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.028,"15min":0.95799999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.29799999999999999},"max":{"1min":1.603,"5min":1.796,"15min":1.796},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.133},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.433},"max":{"1min":1.5600000000000001,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.04}]},{"osd":2,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.99099999999999999,"15min":0.94399999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.34899999999999998},"max":{"1min":1.5009999999999999,"5min":1.526,"15min":1.611},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.145,"15min":1.093},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.38200000000000001},"max":{"1min":1.593,"5min":1.8049999999999999,"15min":2.0750000000000002},"last":1.0880000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.2050000000000001,"15min":1.167},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.41899999999999998},"max":{"1min":1.6699999999999999,"5min":1.6899999999999999,"15min":1.8160000000000001},"last":1.163},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.085,"15min":1.0640000000000001},"min":{"1min":0.76000000000000001,"5min":0.71899999999999997,"15min":0.42799999999999999},"max":{"1min":1.333,"5min":1.788,"15min":1.788},"last":0.76700000000000002}]},{"osd":4,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3400000000000001,"15min":1.2989999999999999},"min":{"1min":1.044,"5min":0.96699999999999997,"15min":0.63900000000000001},"max":{"1min":1.526,"5min":1.966,"15min":1.966},"last":1.119},{"interface":"front","average":{"1min":1.2,"5min":1.24,"15min":1.2170000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.61799999999999999},"max":{"1min":1.712,"5min":1.722,"15min":2.2029999999999998},"last":1.4330000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.349,"15min":1.2989999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.79200000000000004},"max":{"1min":1.857,"5min":1.857,"15min":2.3399999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.427,"5min":1.5029999999999999,"15min":1.4379999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.71599999999999997},"max":{"1min":1.905,"5min":2.077,"15min":2.3069999999999999},"last":1.2330000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.431,"15min":1.415},"min":{"1min":1.0940000000000001,"5min":1.044,"15min":0.65500000000000003},"max":{"1min":1.6970000000000001,"5min":1.714,"15min":2.2490000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.397,"5min":1.448,"15min":1.403},"min":{"1min":1.2110000000000001,"5min":1.1220000000000001,"15min":0.68300000000000005},"max":{"1min":1.764,"5min":1.788,"15min":2.4700000000000002},"last":1.353}]},{"osd":7,"last update":"Sun Oct 8 05:49:27 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.502,"15min":1.4590000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.79100000000000004},"max":{"1min":1.8149999999999999,"5min":2.0870000000000002,"15min":2.1869999999999998},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4630000000000001,"15min":1.46},"min":{"1min":1.1519999999999999,"5min":1.149,"15min":0.76300000000000001},"max":{"1min":1.7709999999999999,"5min":2.306,"15min":2.4180000000000001},"last":1.385}]}]},{"osd":0,"up_from":13,"seq":55834575021,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1454080,"data_stored":1015292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.93000000000000005,"15min":0.91000000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.38800000000000001},"max":{"1min":1.1040000000000001,"5min":1.524,"15min":1.7130000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":1.079,"5min":1.111,"15min":1.036},"min":{"1min":0.432,"5min":0.432,"15min":0.377},"max":{"1min":1.5189999999999999,"5min":1.675,"15min":2.3500000000000001},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.1160000000000001,"15min":1.071},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.38900000000000001},"max":{"1min":1.4410000000000001,"5min":1.8,"15min":1.8089999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.052,"15min":1.0049999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.42299999999999999},"max":{"1min":1.546,"5min":1.615,"15min":1.78},"last":0.50600000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.23,"15min":1.1930000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.505},"max":{"1min":1.488,"5min":1.5980000000000001,"15min":2.4540000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.131,"15min":1.099},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.439},"max":{"1min":1.5900000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.321,"15min":1.28},"min":{"1min":1.042,"5min":0.94299999999999995,"15min":0.71499999999999997},"max":{"1min":1.718,"5min":1.7470000000000001,"15min":2.496},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3779999999999999,"15min":1.3620000000000001},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.6799999999999999,"5min":1.7849999999999999,"15min":2.6560000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.379,"15min":1.341},"min":{"1min":1.05,"5min":0.81299999999999994,"15min":0.76000000000000001},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":2.2650000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4079999999999999,"15min":1.399},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.69399999999999995},"max":{"1min":1.7430000000000001,"5min":1.802,"15min":2.7069999999999999},"last":0.73999999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.454,"15min":1.4299999999999999},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.71299999999999997},"max":{"1min":1.6990000000000001,"5min":2.1909999999999998,"15min":2.8290000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4179999999999999,"15min":1.423},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.83399999999999996},"max":{"1min":1.768,"5min":1.7809999999999999,"15min":2.4020000000000001},"last":0.83099999999999996}]},{"osd":7,"last update":"Sun Oct 8 05:49:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4630000000000001,"15min":1.458},"min":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.72399999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.746},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.52,"15min":1.49},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.78900000000000003},"max":{"1min":1.8360000000000001,"5min":2.1059999999999999,"15min":2.9300000000000002},"last":0.98199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:50:21.633 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T05:50:21.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T05:50:21.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.897+0000 7f6e06c56640 1 Processor -- start 2023-10-08T05:50:21.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.898+0000 7f6e06c56640 1 -- start start 2023-10-08T05:50:21.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.899+0000 7f6e06c56640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e000d2bb0 0x7f6e000d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.899+0000 7f6e06c56640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e000d12f0 con 0x7f6e000b17c0 2023-10-08T05:50:21.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.899+0000 7f6e06c56640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e000b5cb0 con 0x7f6e000d07f0 2023-10-08T05:50:21.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.899+0000 7f6e06c56640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e000b54c0 con 0x7f6e000d2bb0 2023-10-08T05:50:21.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.900+0000 7f6e051f3640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6e000b17c0 0x7f6e000b4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52510/0 (socket says 172.21.15.70:52510) 2023-10-08T05:50:21.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.900+0000 7f6e051f3640 1 -- 172.21.15.70:0/780495228 learned_addr learned my addr 172.21.15.70:0/780495228 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:21.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.900+0000 7f6e049f2640 1 --2- 172.21.15.70:0/780495228 >> v2:172.21.15.181:3300/0 conn(0x7f6e000d2bb0 0x7f6e000d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.900+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547903848 0 0) 0x7f6e000d12f0 con 0x7f6e000b17c0 2023-10-08T05:50:21.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.900+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dec003520 con 0x7f6e000b17c0 2023-10-08T05:50:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.900+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2535980226 0 0) 0x7f6dec003520 con 0x7f6e000b17c0 2023-10-08T05:50:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.900+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 >> v2:172.21.15.181:3300/0 conn(0x7f6e000d2bb0 msgr2=0x7f6e000d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.900+0000 7f6dff7fe640 1 --2- 172.21.15.70:0/780495228 >> v2:172.21.15.181:3300/0 conn(0x7f6e000d2bb0 0x7f6e000d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.900+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 >> v1:172.21.15.181:6789/0 conn(0x7f6e000d07f0 legacy=0x7f6e000b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.900+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e000d34d0 con 0x7f6e000b17c0 2023-10-08T05:50:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6df8001670 con 0x7f6e000b17c0 2023-10-08T05:50:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 >> v1:172.21.15.70:6789/0 conn(0x7f6e000b17c0 legacy=0x7f6e000b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6dff7fe640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dec003970 0x7f6dec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6dff7fe640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dec0047c0 0x7f6dec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6dff7fe640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dec005180 0x7f6dec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e000b54c0 con 0x7f6dec005180 2023-10-08T05:50:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec00c020 con 0x7f6dec0047c0 2023-10-08T05:50:21.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec00c1e0 con 0x7f6dec003970 2023-10-08T05:50:21.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6e051f3640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dec005180 0x7f6dec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6e049f2640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dec003970 0x7f6dec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6dfffff640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dec0047c0 0x7f6dec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6e051f3640 1 -- 172.21.15.70:0/780495228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dec003970 msgr2=0x7f6dec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:21.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6e051f3640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6dec003970 0x7f6dec003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6e051f3640 1 -- 172.21.15.70:0/780495228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dec0047c0 msgr2=0x7f6dec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:21.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6e051f3640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dec0047c0 0x7f6dec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6e051f3640 1 -- 172.21.15.70:0/780495228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec003520 con 0x7f6dec005180 2023-10-08T05:50:21.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.901+0000 7f6dfffff640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dec0047c0 0x7f6dec004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:21.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.902+0000 7f6e051f3640 1 -- 172.21.15.70:0/780495228 reap_dead start 2023-10-08T05:50:21.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.903+0000 7f6e051f3640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dec005180 0x7f6dec00b980 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6df800c190 tx=0x7f6dec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1874a1ab4071d8d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:21.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.903+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8005ad0 con 0x7f6dec005180 2023-10-08T05:50:21.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.903+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df8005c50 con 0x7f6dec005180 2023-10-08T05:50:21.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.903+0000 7f6dff7fe640 1 -- 172.21.15.70:0/780495228 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df802c8a0 con 0x7f6dec005180 2023-10-08T05:50:21.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.903+0000 7f6e06c56640 1 -- 172.21.15.70:0/780495228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dec005180 msgr2=0x7f6dec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.903+0000 7f6e06c56640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dec005180 0x7f6dec00b980 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6df800c190 tx=0x7f6dec0057f0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.903+0000 7f6e06c56640 1 -- 172.21.15.70:0/780495228 shutdown_connections 2023-10-08T05:50:21.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.903+0000 7f6e06c56640 1 --2- 172.21.15.70:0/780495228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6dec005180 0x7f6dec00b980 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.903+0000 7f6e06c56640 1 -- 172.21.15.70:0/780495228 >> 172.21.15.70:0/780495228 conn(0x7f6e000718a0 msgr2=0x7f6e00071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:21.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.904+0000 7f6e06c56640 1 -- 172.21.15.70:0/780495228 shutdown_connections 2023-10-08T05:50:21.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.904+0000 7f6e06c56640 1 -- 172.21.15.70:0/780495228 wait complete. 2023-10-08T05:50:21.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.905+0000 7f6e06c56640 1 Processor -- start 2023-10-08T05:50:21.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.906+0000 7f6e06c56640 1 -- start start 2023-10-08T05:50:21.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.906+0000 7f6e06c56640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dec005180 0x7f6e000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.906+0000 7f6e06c56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e000ca6d0 0x7f6e0014d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6e06c56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e0014dea0 0x7f6e000c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6e06c56640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e000b52d0 con 0x7f6e000ca6d0 2023-10-08T05:50:21.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6e06c56640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e000cac70 con 0x7f6dec005180 2023-10-08T05:50:21.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6e06c56640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e000caf50 con 0x7f6e0014dea0 2023-10-08T05:50:21.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6dfffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e000ca6d0 0x7f6e0014d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6dfffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e000ca6d0 0x7f6e0014d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43640/0 (socket says 172.21.15.70:43640) 2023-10-08T05:50:21.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6dfffff640 1 -- 172.21.15.70:0/3058399019 learned_addr learned my addr 172.21.15.70:0/3058399019 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:21.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6e049f2640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dec005180 0x7f6e000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6dfffff640 1 -- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e0014dea0 msgr2=0x7f6e000c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6dfffff640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e0014dea0 0x7f6e000c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6dfffff640 1 -- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dec005180 msgr2=0x7f6e000b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:21.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6dfffff640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dec005180 0x7f6e000b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:21.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6dfffff640 1 -- 172.21.15.70:0/3058399019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0014ce80 con 0x7f6e000ca6d0 2023-10-08T05:50:21.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.907+0000 7f6dfffff640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e000ca6d0 0x7f6e0014d960 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6de800d7a0 tx=0x7f6de800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:21.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.908+0000 7f6dfd7fa640 1 -- 172.21.15.70:0/3058399019 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de8005270 con 0x7f6e000ca6d0 2023-10-08T05:50:21.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.908+0000 7f6dfd7fa640 1 -- 172.21.15.70:0/3058399019 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de80053f0 con 0x7f6e000ca6d0 2023-10-08T05:50:21.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.908+0000 7f6e06c56640 1 -- 172.21.15.70:0/3058399019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0014d160 con 0x7f6e000ca6d0 2023-10-08T05:50:21.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.908+0000 7f6dfd7fa640 1 -- 172.21.15.70:0/3058399019 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de800b8f0 con 0x7f6e000ca6d0 2023-10-08T05:50:21.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.908+0000 7f6e06c56640 1 -- 172.21.15.70:0/3058399019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e000c8d50 con 0x7f6e000ca6d0 2023-10-08T05:50:21.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.909+0000 7f6e06c56640 1 -- 172.21.15.70:0/3058399019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e000c8f70 con 0x7f6e000ca6d0 2023-10-08T05:50:21.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.910+0000 7f6dfd7fa640 1 -- 172.21.15.70:0/3058399019 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6de800ba50 con 0x7f6e000ca6d0 2023-10-08T05:50:21.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.911+0000 7f6dfd7fa640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6de006af90 0x7f6de006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:21.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.911+0000 7f6dfd7fa640 1 -- 172.21.15.70:0/3058399019 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6de8089aa0 con 0x7f6e000ca6d0 2023-10-08T05:50:21.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.911+0000 7f6e049f2640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6de006af90 0x7f6de006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:21.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.913+0000 7f6e049f2640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6de006af90 0x7f6de006d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6df0003a50 tx=0x7f6df000b040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:21.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:21.915+0000 7f6dfd7fa640 1 -- 172.21.15.70:0/3058399019 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6de8054720 con 0x7f6e000ca6d0 2023-10-08T05:50:22.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.176+0000 7f6e06c56640 1 -- 172.21.15.70:0/3058399019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6e0006c590 con 0x7f6e000ca6d0 2023-10-08T05:50:22.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.181+0000 7f6dfd7fa640 1 -- 172.21.15.70:0/3058399019 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6de8019310 con 0x7f6e000ca6d0 2023-10-08T05:50:22.182 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T05:50:22.183 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T05:50:22.183 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T05:50:22.183 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T05:50:22.183 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T05:50:22.183 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T05:50:22.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.184+0000 7f6e06c56640 1 -- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6de006af90 msgr2=0x7f6de006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:22.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.184+0000 7f6e06c56640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6de006af90 0x7f6de006d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6df0003a50 tx=0x7f6df000b040 comp rx=0 tx=0).stop 2023-10-08T05:50:22.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.184+0000 7f6e06c56640 1 -- 172.21.15.70:0/3058399019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e000ca6d0 msgr2=0x7f6e0014d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:22.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.184+0000 7f6e06c56640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e000ca6d0 0x7f6e0014d960 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6de800d7a0 tx=0x7f6de800dc60 comp rx=0 tx=0).stop 2023-10-08T05:50:22.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.184+0000 7f6e06c56640 1 -- 172.21.15.70:0/3058399019 shutdown_connections 2023-10-08T05:50:22.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.184+0000 7f6e06c56640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f6de006af90 0x7f6de006d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.184+0000 7f6e06c56640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e0014dea0 0x7f6e000c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.184+0000 7f6e06c56640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e000ca6d0 0x7f6e0014d960 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.184+0000 7f6e06c56640 1 --2- 172.21.15.70:0/3058399019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6dec005180 0x7f6e000b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.184+0000 7f6e06c56640 1 -- 172.21.15.70:0/3058399019 >> 172.21.15.70:0/3058399019 conn(0x7f6e000718a0 msgr2=0x7f6e000d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:22.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.185+0000 7f6e06c56640 1 -- 172.21.15.70:0/3058399019 shutdown_connections 2023-10-08T05:50:22.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.185+0000 7f6e06c56640 1 -- 172.21.15.70:0/3058399019 wait complete. 2023-10-08T05:50:22.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:50:22.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.418+0000 7f8720968640 1 Processor -- start 2023-10-08T05:50:22.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.418+0000 7f8720968640 1 -- start start 2023-10-08T05:50:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.420+0000 7f8720968640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f871c0ad790 0x7f871c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.420+0000 7f8720968640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871c15e330 0x7f871c070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.420+0000 7f8720968640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c06dee0 con 0x7f871c159ac0 2023-10-08T05:50:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.420+0000 7f8720968640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c06dcf0 con 0x7f871c0ad790 2023-10-08T05:50:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.420+0000 7f8720968640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c15d850 con 0x7f871c15e330 2023-10-08T05:50:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.420+0000 7f8713fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f871c0ad790 0x7f871c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.420+0000 7f8713fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f871c0ad790 0x7f871c0702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43648/0 (socket says 172.21.15.70:43648) 2023-10-08T05:50:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.420+0000 7f8713fff640 1 -- 172.21.15.70:0/2647309825 learned_addr learned my addr 172.21.15.70:0/2647309825 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.420+0000 7f8713fff640 1 -- 172.21.15.70:0/2647309825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871c15e330 msgr2=0x7f871c070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.420+0000 7f871ad9d640 1 --2- 172.21.15.70:0/2647309825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871c15e330 0x7f871c070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.421+0000 7f8713fff640 1 --2- 172.21.15.70:0/2647309825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871c15e330 0x7f871c070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.421+0000 7f8713fff640 1 -- 172.21.15.70:0/2647309825 >> v1:172.21.15.181:6789/0 conn(0x7f871c159ac0 legacy=0x7f871c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.421+0000 7f8713fff640 1 -- 172.21.15.70:0/2647309825 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c15db30 con 0x7f871c0ad790 2023-10-08T05:50:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.421+0000 7f871ad9d640 1 --2- 172.21.15.70:0/2647309825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871c15e330 0x7f871c070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.422+0000 7f8713fff640 1 --2- 172.21.15.70:0/2647309825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 0x7f871c0702d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f870400bb10 tx=0x7f870402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6406ac665427b654 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.422+0000 7f8719d9b640 1 -- 172.21.15.70:0/2647309825 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870402ee70 con 0x7f871c0ad790 2023-10-08T05:50:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.422+0000 7f8719d9b640 1 -- 172.21.15.70:0/2647309825 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8704036440 con 0x7f871c0ad790 2023-10-08T05:50:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.422+0000 7f8719d9b640 1 -- 172.21.15.70:0/2647309825 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87040367d0 con 0x7f871c0ad790 2023-10-08T05:50:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.423+0000 7f8720968640 1 -- 172.21.15.70:0/2647309825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 msgr2=0x7f871c0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.423+0000 7f8720968640 1 --2- 172.21.15.70:0/2647309825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 0x7f871c0702d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f870400bb10 tx=0x7f870402e0c0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.423+0000 7f8720968640 1 -- 172.21.15.70:0/2647309825 shutdown_connections 2023-10-08T05:50:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.423+0000 7f8720968640 1 --2- 172.21.15.70:0/2647309825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871c15e330 0x7f871c070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.423+0000 7f8720968640 1 --2- 172.21.15.70:0/2647309825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 0x7f871c0702d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.423+0000 7f8720968640 1 -- 172.21.15.70:0/2647309825 >> 172.21.15.70:0/2647309825 conn(0x7f871c06a380 msgr2=0x7f871c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.423+0000 7f8720968640 1 -- 172.21.15.70:0/2647309825 shutdown_connections 2023-10-08T05:50:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.423+0000 7f8720968640 1 -- 172.21.15.70:0/2647309825 wait complete. 2023-10-08T05:50:22.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.425+0000 7f8720968640 1 Processor -- start 2023-10-08T05:50:22.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.425+0000 7f8720968640 1 -- start start 2023-10-08T05:50:22.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f8720968640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 0x7f871c1edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:22.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f8720968640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f871c159ac0 0x7f871c158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f8720968640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871c15e330 0x7f871c158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f8720968640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1ee210 con 0x7f871c0ad790 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f8720968640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c158650 con 0x7f871c159ac0 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f8720968640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1573e0 con 0x7f871c15e330 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f871a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 0x7f871c1edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f871a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 0x7f871c1edcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43654/0 (socket says 172.21.15.70:43654) 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f871a59c640 1 -- 172.21.15.70:0/4251941987 learned_addr learned my addr 172.21.15.70:0/4251941987 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f871a59c640 1 -- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871c15e330 msgr2=0x7f871c158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f871a59c640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871c15e330 0x7f871c158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f871a59c640 1 -- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f871c159ac0 msgr2=0x7f871c158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f871a59c640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f871c159ac0 0x7f871c158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.426+0000 7f871a59c640 1 -- 172.21.15.70:0/4251941987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870400b770 con 0x7f871c0ad790 2023-10-08T05:50:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.427+0000 7f871a59c640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 0x7f871c1edcd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c430 tx=0x7f870c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.427+0000 7f8712ffd640 1 -- 172.21.15.70:0/4251941987 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c011790 con 0x7f871c0ad790 2023-10-08T05:50:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.427+0000 7f8712ffd640 1 -- 172.21.15.70:0/4251941987 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c0118f0 con 0x7f871c0ad790 2023-10-08T05:50:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.427+0000 7f8720968640 1 -- 172.21.15.70:0/4251941987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c157c80 con 0x7f871c0ad790 2023-10-08T05:50:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.427+0000 7f8712ffd640 1 -- 172.21.15.70:0/4251941987 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c011ce0 con 0x7f871c0ad790 2023-10-08T05:50:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.427+0000 7f8720968640 1 -- 172.21.15.70:0/4251941987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c157630 con 0x7f871c0ad790 2023-10-08T05:50:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.428+0000 7f8720968640 1 -- 172.21.15.70:0/4251941987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c067490 con 0x7f871c0ad790 2023-10-08T05:50:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.429+0000 7f8712ffd640 1 -- 172.21.15.70:0/4251941987 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f870c01c460 con 0x7f871c0ad790 2023-10-08T05:50:22.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.430+0000 7f8712ffd640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f86fc06af90 0x7f86fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:22.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.430+0000 7f8712ffd640 1 -- 172.21.15.70:0/4251941987 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f870c089830 con 0x7f871c0ad790 2023-10-08T05:50:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.431+0000 7f8713fff640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f86fc06af90 0x7f86fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:22.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.432+0000 7f8713fff640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f86fc06af90 0x7f86fc06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8704038000 tx=0x7f8704030000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:22.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.435+0000 7f8712ffd640 1 -- 172.21.15.70:0/4251941987 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f870c0546f0 con 0x7f871c0ad790 2023-10-08T05:50:22.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.662+0000 7f8720968640 1 -- 172.21.15.70:0/4251941987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f871c1535a0 con 0x7f871c0ad790 2023-10-08T05:50:22.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.663+0000 7f8712ffd640 1 -- 172.21.15.70:0/4251941987 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f870c057d10 con 0x7f871c0ad790 2023-10-08T05:50:22.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:22.664 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1126961048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1126961048},{"type":"v1","addr":"172.21.15.181:6836","nonce":1126961048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3993068621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3993068621},{"type":"v1","addr":"172.21.15.181:6837","nonce":3993068621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/28904251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":28904251},{"type":"v1","addr":"172.21.15.70:6833","nonce":28904251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:50:16.144118+0000","modified":"2023-10-08T05:50:19.194284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1337581673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1337581673},{"type":"v1","addr":"172.21.15.70:6835","nonce":1337581673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-08T05:50:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 25 2023-10-08T05:50:22.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 -- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f86fc06af90 msgr2=0x7f86fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:22.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f86fc06af90 0x7f86fc06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8704038000 tx=0x7f8704030000 comp rx=0 tx=0).stop 2023-10-08T05:50:22.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 -- 172.21.15.70:0/4251941987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 msgr2=0x7f871c1edcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 0x7f871c1edcd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f870c00c430 tx=0x7f870c00c8f0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 -- 172.21.15.70:0/4251941987 shutdown_connections 2023-10-08T05:50:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f86fc06af90 0x7f86fc06d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871c15e330 0x7f871c158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f871c159ac0 0x7f871c158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 --2- 172.21.15.70:0/4251941987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f871c0ad790 0x7f871c1edcd0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 -- 172.21.15.70:0/4251941987 >> 172.21.15.70:0/4251941987 conn(0x7f871c06a380 msgr2=0x7f871c0cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 -- 172.21.15.70:0/4251941987 shutdown_connections 2023-10-08T05:50:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:22.666+0000 7f8720968640 1 -- 172.21.15.70:0/4251941987 wait complete. 2023-10-08T05:50:23.089 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T05:50:16.144118+0000', 'modified': '2023-10-08T05:50:19.194284+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6177': {'gid': 6177, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6835/1337581673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 1337581673}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 1337581673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T05:50:23.089 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T05:50:23.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T05:50:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.313+0000 7fdd5c84f640 1 Processor -- start 2023-10-08T05:50:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.314+0000 7fdd5c84f640 1 -- start start 2023-10-08T05:50:23.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.315+0000 7fdd5c84f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdd5806c3a0 0x7fdd580c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.315+0000 7fdd5c84f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd5806db60 0x7fdd5806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.315+0000 7fdd5c84f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5815b310 con 0x7fdd5815aa00 2023-10-08T05:50:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.315+0000 7fdd5c84f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd580ca230 con 0x7fdd5806c3a0 2023-10-08T05:50:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.315+0000 7fdd5c84f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5806d290 con 0x7fdd5806db60 2023-10-08T05:50:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.315+0000 7fdd5659c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdd5806c3a0 0x7fdd580c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.315+0000 7fdd56d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdd5815aa00 0x7fdd580b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52512/0 (socket says 172.21.15.70:52512) 2023-10-08T05:50:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.316+0000 7fdd56d9d640 1 -- 172.21.15.70:0/767797343 learned_addr learned my addr 172.21.15.70:0/767797343 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.316+0000 7fdd55d9b640 1 --2- 172.21.15.70:0/767797343 >> v2:172.21.15.181:3300/0 conn(0x7fdd5806db60 0x7fdd5806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.316+0000 7fdd5659c640 1 -- 172.21.15.70:0/767797343 >> v2:172.21.15.181:3300/0 conn(0x7fdd5806db60 msgr2=0x7fdd5806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.316+0000 7fdd5659c640 1 --2- 172.21.15.70:0/767797343 >> v2:172.21.15.181:3300/0 conn(0x7fdd5806db60 0x7fdd5806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.316+0000 7fdd5659c640 1 -- 172.21.15.70:0/767797343 >> v1:172.21.15.70:6789/0 conn(0x7fdd5815aa00 legacy=0x7fdd580b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.316+0000 7fdd5659c640 1 -- 172.21.15.70:0/767797343 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580c9860 con 0x7fdd5806c3a0 2023-10-08T05:50:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.316+0000 7fdd5559a640 1 -- 172.21.15.70:0/767797343 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301766348 0 0) 0x7fdd50001490 con 0x7fdd5815aa00 2023-10-08T05:50:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.317+0000 7fdd5659c640 1 --2- 172.21.15.70:0/767797343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5806c3a0 0x7fdd580c9cf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdd580715c0 tx=0x7fdd4802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03e648d3f80efe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.317+0000 7fdd5559a640 1 -- 172.21.15.70:0/767797343 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48031830 con 0x7fdd5806c3a0 2023-10-08T05:50:23.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.318+0000 7fdd5559a640 1 -- 172.21.15.70:0/767797343 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd48031990 con 0x7fdd5806c3a0 2023-10-08T05:50:23.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.318+0000 7fdd5559a640 1 -- 172.21.15.70:0/767797343 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48031d20 con 0x7fdd5806c3a0 2023-10-08T05:50:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.318+0000 7fdd5c84f640 1 -- 172.21.15.70:0/767797343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5806c3a0 msgr2=0x7fdd580c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.318+0000 7fdd5c84f640 1 --2- 172.21.15.70:0/767797343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5806c3a0 0x7fdd580c9cf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdd580715c0 tx=0x7fdd4802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.318+0000 7fdd5c84f640 1 -- 172.21.15.70:0/767797343 shutdown_connections 2023-10-08T05:50:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.318+0000 7fdd5c84f640 1 --2- 172.21.15.70:0/767797343 >> v2:172.21.15.181:3300/0 conn(0x7fdd5806db60 0x7fdd5806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.318+0000 7fdd5c84f640 1 --2- 172.21.15.70:0/767797343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5806c3a0 0x7fdd580c9cf0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.318+0000 7fdd5c84f640 1 -- 172.21.15.70:0/767797343 >> 172.21.15.70:0/767797343 conn(0x7fdd5806a5d0 msgr2=0x7fdd580b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.318+0000 7fdd5c84f640 1 -- 172.21.15.70:0/767797343 shutdown_connections 2023-10-08T05:50:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.318+0000 7fdd5c84f640 1 -- 172.21.15.70:0/767797343 wait complete. 2023-10-08T05:50:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.320+0000 7fdd5c84f640 1 Processor -- start 2023-10-08T05:50:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.320+0000 7fdd5c84f640 1 -- start start 2023-10-08T05:50:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.321+0000 7fdd5c84f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd5806c3a0 0x7fdd58157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.321+0000 7fdd5c84f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd5806db60 0x7fdd580c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.321+0000 7fdd5c84f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5815aa00 0x7fdd580ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.321+0000 7fdd5c84f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5815e450 con 0x7fdd5815aa00 2023-10-08T05:50:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.321+0000 7fdd5c84f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd581f29c0 con 0x7fdd5806c3a0 2023-10-08T05:50:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.321+0000 7fdd5c84f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58158220 con 0x7fdd5806db60 2023-10-08T05:50:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.321+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5815aa00 0x7fdd580ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.322+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5815aa00 0x7fdd580ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43676/0 (socket says 172.21.15.70:43676) 2023-10-08T05:50:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.322+0000 7fdd56d9d640 1 -- 172.21.15.70:0/2634521771 learned_addr learned my addr 172.21.15.70:0/2634521771 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.322+0000 7fdd5659c640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd5806c3a0 0x7fdd58157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.322+0000 7fdd55d9b640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd5806db60 0x7fdd580c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.322+0000 7fdd56d9d640 1 -- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd5806db60 msgr2=0x7fdd580c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.322+0000 7fdd56d9d640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd5806db60 0x7fdd580c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.322+0000 7fdd56d9d640 1 -- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd5806c3a0 msgr2=0x7fdd58157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.322+0000 7fdd56d9d640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd5806c3a0 0x7fdd58157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.322+0000 7fdd56d9d640 1 -- 172.21.15.70:0/2634521771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009400 con 0x7fdd5815aa00 2023-10-08T05:50:23.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.322+0000 7fdd56d9d640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5815aa00 0x7fdd580ca5e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c700 tx=0x7fdd5000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:23.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.323+0000 7fdd46ffd640 1 -- 172.21.15.70:0/2634521771 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5000ce10 con 0x7fdd5815aa00 2023-10-08T05:50:23.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.323+0000 7fdd46ffd640 1 -- 172.21.15.70:0/2634521771 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5000a640 con 0x7fdd5815aa00 2023-10-08T05:50:23.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.323+0000 7fdd5c84f640 1 -- 172.21.15.70:0/2634521771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5815d8d0 con 0x7fdd5815aa00 2023-10-08T05:50:23.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.323+0000 7fdd46ffd640 1 -- 172.21.15.70:0/2634521771 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5000aa70 con 0x7fdd5815aa00 2023-10-08T05:50:23.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.323+0000 7fdd5c84f640 1 -- 172.21.15.70:0/2634521771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5815de90 con 0x7fdd5815aa00 2023-10-08T05:50:23.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.325+0000 7fdd46ffd640 1 -- 172.21.15.70:0/2634521771 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdd5000abd0 con 0x7fdd5815aa00 2023-10-08T05:50:23.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.326+0000 7fdd46ffd640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdd3806af90 0x7fdd3806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.326+0000 7fdd46ffd640 1 -- 172.21.15.70:0/2634521771 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdd50088d70 con 0x7fdd5815aa00 2023-10-08T05:50:23.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.327+0000 7fdd5f3e7740 1 Processor -- start 2023-10-08T05:50:23.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.327+0000 7fdd5659c640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdd3806af90 0x7fdd3806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.327+0000 7fdd5f3e7740 1 -- start start 2023-10-08T05:50:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a903a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a903fa0 0x56340a933e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340a877700 con 0x56340a930b60 2023-10-08T05:50:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340a874df0 con 0x56340a930f40 2023-10-08T05:50:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56340a8706e0 con 0x56340a903fa0 2023-10-08T05:50:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd5659c640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdd3806af90 0x7fdd3806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd48036000 tx=0x7fdd48008030 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a903a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a903a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43688/0 (socket says 172.21.15.70:43688) 2023-10-08T05:50:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd5659c640 1 -- 172.21.15.70:0/3878705080 learned_addr learned my addr 172.21.15.70:0/3878705080 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.328+0000 7fdd55d9b640 1 --2- 172.21.15.70:0/3878705080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd56d9d640 1 --2- 172.21.15.70:0/3878705080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a903fa0 0x56340a933e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd5659c640 1 -- 172.21.15.70:0/3878705080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a903fa0 msgr2=0x56340a933e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd5659c640 1 --2- 172.21.15.70:0/3878705080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a903fa0 0x56340a933e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd5659c640 1 -- 172.21.15.70:0/3878705080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 msgr2=0x56340a9338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd5659c640 1 --2- 172.21.15.70:0/3878705080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd5659c640 1 -- 172.21.15.70:0/3878705080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56340a834910 con 0x56340a930b60 2023-10-08T05:50:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd56d9d640 1 --2- 172.21.15.70:0/3878705080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a903fa0 0x56340a933e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd5659c640 1 --2- 172.21.15.70:0/3878705080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a903a60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009860 tx=0x7fdd48030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ebfba46dfea169 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:23.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd37fff640 1 -- 172.21.15.70:0/3878705080 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48041070 con 0x56340a930b60 2023-10-08T05:50:23.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd37fff640 1 -- 172.21.15.70:0/3878705080 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd48003790 con 0x56340a930b60 2023-10-08T05:50:23.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.329+0000 7fdd37fff640 1 -- 172.21.15.70:0/3878705080 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48003b20 con 0x56340a930b60 2023-10-08T05:50:23.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.330+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/3878705080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 msgr2=0x56340a903a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.330+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/3878705080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a903a60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009860 tx=0x7fdd48030000 comp rx=0 tx=0).stop 2023-10-08T05:50:23.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.330+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/3878705080 shutdown_connections 2023-10-08T05:50:23.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.330+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/3878705080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a903fa0 0x56340a933e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.330+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/3878705080 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.330+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/3878705080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a903a60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.330+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/3878705080 >> 172.21.15.70:0/3878705080 conn(0x56340a905d70 msgr2=0x56340a94e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.330+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/3878705080 shutdown_connections 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.330+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/3878705080 wait complete. 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.331+0000 7fdd5f3e7740 1 -- start start 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.331+0000 7fdd5f3e7740 1 Processor -- start 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.332+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a930b60 0x56340a948f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.332+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9207c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.332+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a949460 0x56340a920d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.332+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340a80e970 con 0x56340a949460 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.332+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340a80e610 con 0x56340a930f40 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.332+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56340a5d4f20 con 0x56340a930b60 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.332+0000 7fdd5f3e7740 10 client.0 _ll_register_callbacks cb 0x56340a84c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.332+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a949460 0x56340a920d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a949460 0x56340a920d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43700/0 (socket says 172.21.15.70:43700) 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd56d9d640 1 -- 172.21.15.70:0/1609336028 learned_addr learned my addr 172.21.15.70:0/1609336028 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd5659c640 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a930b60 0x56340a948f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd55d9b640 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd56d9d640 1 -- 172.21.15.70:0/1609336028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a930b60 msgr2=0x56340a948f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd56d9d640 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a930b60 0x56340a948f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd56d9d640 1 -- 172.21.15.70:0/1609336028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 msgr2=0x56340a9207c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd56d9d640 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9207c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd56d9d640 1 -- 172.21.15.70:0/1609336028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd580c9860 con 0x56340a949460 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd56d9d640 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a949460 0x56340a920d00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdd50086f70 tx=0x7fdd50057e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419c6ebd83e4452d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd357fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.333+0000 7fdd357fa640 1 -- 172.21.15.70:0/1609336028 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5005b890 con 0x56340a949460 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd357fa640 1 -- 172.21.15.70:0/1609336028 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdd5005fa40 con 0x56340a949460 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1609336028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56340a8696c0 con 0x56340a949460 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd357fa640 1 -- 172.21.15.70:0/1609336028 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd5006f070 con 0x56340a949460 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd5f3e7740 10 client.6324 fetch_fsmap learned FSMap version 25 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1609336028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x56340a86ed60 con 0x56340a949460 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd357fa640 1 -- 172.21.15.70:0/1609336028 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56340a8696c0 con 0x56340a949460 2023-10-08T05:50:23.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd357fa640 1 -- 172.21.15.70:0/1609336028 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd50071050 con 0x56340a949460 2023-10-08T05:50:23.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd5f3e7740 10 client.6324 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T05:50:23.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd5f3e7740 10 client.6324 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T05:50:23.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd5f3e7740 20 client.6324 populate_metadata read hostname 'smithi070' 2023-10-08T05:50:23.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.70:6834/1337581673,v1:172.21.15.70:6835/1337581673] conn(0x56340a9219b0 0x56340a935930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd5f3e7740 4 client.6324 mds_command: new command op to 6177 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T05:50:23.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1609336028 --> [v2:172.21.15.70:6834/1337581673,v1:172.21.15.70:6835/1337581673] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56340a8693e0 con 0x56340a9219b0 2023-10-08T05:50:23.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.334+0000 7fdd5659c640 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.70:6834/1337581673,v1:172.21.15.70:6835/1337581673] conn(0x56340a9219b0 0x56340a935930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.335+0000 7fdd5659c640 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.70:6834/1337581673,v1:172.21.15.70:6835/1337581673] conn(0x56340a9219b0 0x56340a935930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35d92b88b20e6b23 server_cookie=afeb4053bbee1446 in_seq=0 out_seq=0 2023-10-08T05:50:23.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.335+0000 7fdd357fa640 10 client.6324 ms_handle_connect on v2:172.21.15.70:6834/1337581673 2023-10-08T05:50:23.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd357fa640 1 -- 172.21.15.70:0/1609336028 <== mds.0 v2:172.21.15.70:6834/1337581673 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56340a8693e0 con 0x56340a9219b0 2023-10-08T05:50:23.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd357fa640 10 client.6324 handle_command_reply: tid=0 2023-10-08T05:50:23.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 client.6324 shutdown 2023-10-08T05:50:23.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1609336028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a949460 msgr2=0x56340a920d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a949460 0x56340a920d00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fdd50086f70 tx=0x7fdd50057e40 comp rx=0 tx=0).stop 2023-10-08T05:50:23.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1609336028 shutdown_connections 2023-10-08T05:50:23.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.70:6834/1337581673,v1:172.21.15.70:6835/1337581673] conn(0x56340a9219b0 0x56340a935930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a949460 0x56340a920d00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9207c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1609336028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a930b60 0x56340a948f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd357fa640 0 client.6324 ms_handle_reset on v2:172.21.15.70:6834/1337581673 2023-10-08T05:50:23.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1609336028 >> 172.21.15.70:0/1609336028 conn(0x56340a903fa0 msgr2=0x56340a907a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:23.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1609336028 shutdown_connections 2023-10-08T05:50:23.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.336+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1609336028 wait complete. 2023-10-08T05:50:23.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.337+0000 7fdd5f3e7740 20 client.6324 trim_cache size 0 max 16384 2023-10-08T05:50:23.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.353+0000 7fdd5f3e7740 1 Processor -- start 2023-10-08T05:50:23.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.353+0000 7fdd5f3e7740 1 -- start start 2023-10-08T05:50:23.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.354+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930b60 0x56340a920bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.354+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930f40 0x56340a92f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.354+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a949460 0x56340a92fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.354+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340a9032b0 con 0x56340a930f40 2023-10-08T05:50:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.354+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340a934390 con 0x56340a930b60 2023-10-08T05:50:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.354+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56340a91e600 con 0x56340a949460 2023-10-08T05:50:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.354+0000 7fdd55d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930f40 0x56340a92f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.355+0000 7fdd55d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930f40 0x56340a92f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43716/0 (socket says 172.21.15.70:43716) 2023-10-08T05:50:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.355+0000 7fdd55d9b640 1 -- 172.21.15.70:0/3104841246 learned_addr learned my addr 172.21.15.70:0/3104841246 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:23.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.355+0000 7fdd55d9b640 1 -- 172.21.15.70:0/3104841246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a949460 msgr2=0x56340a92fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.355+0000 7fdd55d9b640 1 --2- 172.21.15.70:0/3104841246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a949460 0x56340a92fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.355+0000 7fdd55d9b640 1 -- 172.21.15.70:0/3104841246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930b60 msgr2=0x56340a920bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.355+0000 7fdd5659c640 1 --2- 172.21.15.70:0/3104841246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930b60 0x56340a920bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.355+0000 7fdd55d9b640 1 --2- 172.21.15.70:0/3104841246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930b60 0x56340a920bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.355+0000 7fdd55d9b640 1 -- 172.21.15.70:0/3104841246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56340a80db80 con 0x56340a930f40 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.355+0000 7fdd55d9b640 1 --2- 172.21.15.70:0/3104841246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930f40 0x56340a92f7f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c011820 tx=0x7fdd4c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfe17ebc7b1c225 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.355+0000 7fdd44ff9640 1 -- 172.21.15.70:0/3104841246 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004030 con 0x56340a930f40 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd44ff9640 1 -- 172.21.15.70:0/3104841246 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c0041b0 con 0x56340a930f40 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd44ff9640 1 -- 172.21.15.70:0/3104841246 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0138c0 con 0x56340a930f40 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/3104841246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930f40 msgr2=0x56340a92f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/3104841246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930f40 0x56340a92f7f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c011820 tx=0x7fdd4c011ce0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/3104841246 shutdown_connections 2023-10-08T05:50:23.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/3104841246 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a949460 0x56340a92fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/3104841246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930f40 0x56340a92f7f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/3104841246 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930b60 0x56340a920bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/3104841246 >> 172.21.15.70:0/3104841246 conn(0x56340a903fa0 msgr2=0x56340a94a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:23.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/3104841246 shutdown_connections 2023-10-08T05:50:23.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.356+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/3104841246 wait complete. 2023-10-08T05:50:23.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.357+0000 7fdd5f3e7740 1 -- start start 2023-10-08T05:50:23.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.357+0000 7fdd5f3e7740 1 Processor -- start 2023-10-08T05:50:23.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.358+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a92fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.358+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.358+0000 7fdd5f3e7740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a949460 0x56340a944280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.358+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340a936e60 con 0x56340a930b60 2023-10-08T05:50:23.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.358+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56340a961d40 con 0x56340a930f40 2023-10-08T05:50:23.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.358+0000 7fdd5f3e7740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56340a91dbc0 con 0x56340a949460 2023-10-08T05:50:23.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.358+0000 7fdd5f3e7740 10 client.0 _ll_register_callbacks cb 0x7fdd48009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a92fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a92fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43728/0 (socket says 172.21.15.70:43728) 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd5659c640 1 -- 172.21.15.70:0/1366820926 learned_addr learned my addr 172.21.15.70:0/1366820926 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd55d9b640 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd5659c640 1 -- 172.21.15.70:0/1366820926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a949460 msgr2=0x56340a944280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd56d9d640 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a949460 0x56340a944280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd5659c640 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a949460 0x56340a944280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd5659c640 1 -- 172.21.15.70:0/1366820926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 msgr2=0x56340a9b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd5659c640 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd5659c640 1 -- 172.21.15.70:0/1366820926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c00e040 con 0x56340a930b60 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd5659c640 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a92fe20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdd4804f000 tx=0x7fdd48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c3eb87542bebf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:23.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.359+0000 7fdd357fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd357fa640 1 -- 172.21.15.70:0/1366820926 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4805f590 con 0x56340a930b60 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd357fa640 1 -- 172.21.15.70:0/1366820926 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdd48003be0 con 0x56340a930b60 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1366820926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd48003790 con 0x56340a930b60 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd357fa640 1 -- 172.21.15.70:0/1366820926 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd48003da0 con 0x56340a930b60 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd5f3e7740 10 client.6330 fetch_fsmap learned FSMap version 25 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1366820926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x56340a833170 con 0x56340a930b60 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd357fa640 1 -- 172.21.15.70:0/1366820926 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd48003790 con 0x56340a930b60 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd357fa640 1 -- 172.21.15.70:0/1366820926 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd4805f950 con 0x56340a930b60 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd5f3e7740 10 client.6330 fetch_fsmap finished waiting for FSMap version 25 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd5f3e7740 10 client.6330 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T05:50:23.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd5f3e7740 20 client.6330 populate_metadata read hostname 'smithi070' 2023-10-08T05:50:23.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.70:6834/1337581673,v1:172.21.15.70:6835/1337581673] conn(0x56340a960de0 0x56340a9611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd5f3e7740 4 client.6330 mds_command: new command op to 6177 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T05:50:23.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.360+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1366820926 --> [v2:172.21.15.70:6834/1337581673,v1:172.21.15.70:6835/1337581673] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56340a5726c0 con 0x56340a960de0 2023-10-08T05:50:23.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.361+0000 7fdd55d9b640 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.70:6834/1337581673,v1:172.21.15.70:6835/1337581673] conn(0x56340a960de0 0x56340a9611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.361+0000 7fdd55d9b640 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.70:6834/1337581673,v1:172.21.15.70:6835/1337581673] conn(0x56340a960de0 0x56340a9611c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e069677b4e6f92e1 server_cookie=11db45205dc5880 in_seq=0 out_seq=0 2023-10-08T05:50:23.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.361+0000 7fdd357fa640 10 client.6330 ms_handle_connect on v2:172.21.15.70:6834/1337581673 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.361+0000 7fdd357fa640 1 -- 172.21.15.70:0/1366820926 <== mds.0 v2:172.21.15.70:6834/1337581673 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56340a5726c0 con 0x56340a960de0 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.361+0000 7fdd357fa640 10 client.6330 handle_command_reply: tid=0 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.361+0000 7fdd5f3e7740 1 client.6330 shutdown 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1366820926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 msgr2=0x56340a92fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a92fe20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fdd4804f000 tx=0x7fdd48004730 comp rx=0 tx=0).stop 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1366820926 shutdown_connections 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.70:6834/1337581673,v1:172.21.15.70:6835/1337581673] conn(0x56340a960de0 0x56340a9611c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56340a949460 0x56340a944280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56340a930f40 0x56340a9b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 1 --2- 172.21.15.70:0/1366820926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56340a930b60 0x56340a92fe20 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd357fa640 0 client.6330 ms_handle_reset on v2:172.21.15.70:6834/1337581673 2023-10-08T05:50:23.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1366820926 >> 172.21.15.70:0/1366820926 conn(0x56340a903fa0 msgr2=0x56340a94a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:23.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1366820926 shutdown_connections 2023-10-08T05:50:23.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 1 -- 172.21.15.70:0/1366820926 wait complete. 2023-10-08T05:50:23.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.362+0000 7fdd5f3e7740 20 client.6330 trim_cache size 0 max 16384 2023-10-08T05:50:23.368 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:50:23.368 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T05:50:23.368 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T05:50:23.368 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 6177, 2023-10-08T05:50:23.368 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T05:50:23.369 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T05:50:23.369 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T05:50:23.369 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.6645524739999997, 2023-10-08T05:50:23.369 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 25, 2023-10-08T05:50:23.369 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 72, 2023-10-08T05:50:23.369 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 72, 2023-10-08T05:50:23.369 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 13.106574735000001 2023-10-08T05:50:23.369 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:50:23.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 -- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdd3806af90 msgr2=0x7fdd3806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdd3806af90 0x7fdd3806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd48036000 tx=0x7fdd48008030 comp rx=0 tx=0).stop 2023-10-08T05:50:23.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 -- 172.21.15.70:0/2634521771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5815aa00 msgr2=0x7fdd580ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5815aa00 0x7fdd580ca5e0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c700 tx=0x7fdd5000cbc0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 -- 172.21.15.70:0/2634521771 shutdown_connections 2023-10-08T05:50:23.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd5815aa00 0x7fdd580ca5e0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdd3806af90 0x7fdd3806d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd5806db60 0x7fdd580c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 --2- 172.21.15.70:0/2634521771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd5806c3a0 0x7fdd58157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 -- 172.21.15.70:0/2634521771 >> 172.21.15.70:0/2634521771 conn(0x7fdd5806a5d0 msgr2=0x7fdd581584b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:23.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 -- 172.21.15.70:0/2634521771 shutdown_connections 2023-10-08T05:50:23.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.363+0000 7fdd5c84f640 1 -- 172.21.15.70:0/2634521771 wait complete. 2023-10-08T05:50:23.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:50:23.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.596+0000 7fc005a6f640 1 Processor -- start 2023-10-08T05:50:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.596+0000 7fc005a6f640 1 -- start start 2023-10-08T05:50:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.598+0000 7fc005a6f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc00015ab40 0x7fc0000ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.598+0000 7fc005a6f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0000cb580 0x7fc0000adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.598+0000 7fc005a6f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00015b2b0 con 0x7fc0000c90b0 2023-10-08T05:50:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.598+0000 7fc005a6f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00015bf30 con 0x7fc00015ab40 2023-10-08T05:50:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.598+0000 7fc005a6f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00015f7a0 con 0x7fc0000cb580 2023-10-08T05:50:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.598+0000 7fbfff7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc00015ab40 0x7fc0000ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.598+0000 7fbfff7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc00015ab40 0x7fc0000ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46480/0 (socket says 172.21.15.70:46480) 2023-10-08T05:50:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.598+0000 7fbfff7fe640 1 -- 172.21.15.70:0/2730490718 learned_addr learned my addr 172.21.15.70:0/2730490718 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.599+0000 7fbfff7fe640 1 -- 172.21.15.70:0/2730490718 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0000cb580 msgr2=0x7fc0000adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.599+0000 7fbfff7fe640 1 --2- 172.21.15.70:0/2730490718 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0000cb580 0x7fc0000adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.599+0000 7fbfff7fe640 1 -- 172.21.15.70:0/2730490718 >> v1:172.21.15.70:6789/0 conn(0x7fc0000c90b0 legacy=0x7fc0000cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.599+0000 7fbfff7fe640 1 -- 172.21.15.70:0/2730490718 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00015b590 con 0x7fc00015ab40 2023-10-08T05:50:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.599+0000 7fbffeffd640 1 -- 172.21.15.70:0/2730490718 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537871862 0 0) 0x7fc00015b2b0 con 0x7fc0000c90b0 2023-10-08T05:50:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.600+0000 7fbfff7fe640 1 --2- 172.21.15.70:0/2730490718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00015ab40 0x7fc0000ad720 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbff00098a0 tx=0x7fbff002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448eb080d5921390 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:23.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.600+0000 7fbffeffd640 1 -- 172.21.15.70:0/2730490718 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0030840 con 0x7fc00015ab40 2023-10-08T05:50:23.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.600+0000 7fbffeffd640 1 -- 172.21.15.70:0/2730490718 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff00309a0 con 0x7fc00015ab40 2023-10-08T05:50:23.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.600+0000 7fbffeffd640 1 -- 172.21.15.70:0/2730490718 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0030d30 con 0x7fc00015ab40 2023-10-08T05:50:23.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.601+0000 7fc005a6f640 1 -- 172.21.15.70:0/2730490718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00015ab40 msgr2=0x7fc0000ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.601+0000 7fc005a6f640 1 --2- 172.21.15.70:0/2730490718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00015ab40 0x7fc0000ad720 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbff00098a0 tx=0x7fbff002b980 comp rx=0 tx=0).stop 2023-10-08T05:50:23.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.601+0000 7fc005a6f640 1 -- 172.21.15.70:0/2730490718 shutdown_connections 2023-10-08T05:50:23.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.601+0000 7fc005a6f640 1 --2- 172.21.15.70:0/2730490718 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0000cb580 0x7fc0000adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.601+0000 7fc005a6f640 1 --2- 172.21.15.70:0/2730490718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc00015ab40 0x7fc0000ad720 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.601+0000 7fc005a6f640 1 -- 172.21.15.70:0/2730490718 >> 172.21.15.70:0/2730490718 conn(0x7fc00006a380 msgr2=0x7fc0000b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.601+0000 7fc005a6f640 1 -- 172.21.15.70:0/2730490718 shutdown_connections 2023-10-08T05:50:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.601+0000 7fc005a6f640 1 -- 172.21.15.70:0/2730490718 wait complete. 2023-10-08T05:50:23.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.603+0000 7fc005a6f640 1 Processor -- start 2023-10-08T05:50:23.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.603+0000 7fc005a6f640 1 -- start start 2023-10-08T05:50:23.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.604+0000 7fc005a6f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000c90b0 0x7fc0000b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.604+0000 7fc005a6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000cb580 0x7fc0000b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.604+0000 7fc005a6f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.604+0000 7fc005a6f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000b2420 con 0x7fc0000cb580 2023-10-08T05:50:23.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.604+0000 7fc005a6f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000c37a0 con 0x7fc0000c90b0 2023-10-08T05:50:23.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.604+0000 7fc005a6f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0000c3a80 con 0x7fc00015ab40 2023-10-08T05:50:23.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.604+0000 7fbff7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000cb580 0x7fc0000b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.604+0000 7fbff7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000cb580 0x7fc0000b3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46496/0 (socket says 172.21.15.70:46496) 2023-10-08T05:50:23.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.604+0000 7fbff7fff640 1 -- 172.21.15.70:0/3511677914 learned_addr learned my addr 172.21.15.70:0/3511677914 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:23.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fbffffff640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fbff7fff640 1 -- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 msgr2=0x7fc0000c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fbff7fff640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000c62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fbff7fff640 1 -- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000c90b0 msgr2=0x7fc0000b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fbff7fff640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000c90b0 0x7fc0000b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fbff7fff640 1 -- 172.21.15.70:0/3511677914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00094d0 con 0x7fc0000cb580 2023-10-08T05:50:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fbff7fff640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000cb580 0x7fc0000b3bd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800c390 tx=0x7fbfe800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fbffcff9640 1 -- 172.21.15.70:0/3511677914 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe80035f0 con 0x7fc0000cb580 2023-10-08T05:50:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fbffcff9640 1 -- 172.21.15.70:0/3511677914 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe8003750 con 0x7fc0000cb580 2023-10-08T05:50:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fc005a6f640 1 -- 172.21.15.70:0/3511677914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0000c3d60 con 0x7fc0000cb580 2023-10-08T05:50:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fbffcff9640 1 -- 172.21.15.70:0/3511677914 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8003b00 con 0x7fc0000cb580 2023-10-08T05:50:23.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.605+0000 7fc005a6f640 1 -- 172.21.15.70:0/3511677914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0000b16a0 con 0x7fc0000cb580 2023-10-08T05:50:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.606+0000 7fc005a6f640 1 -- 172.21.15.70:0/3511677914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0000c4280 con 0x7fc0000cb580 2023-10-08T05:50:23.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.608+0000 7fbffcff9640 1 -- 172.21.15.70:0/3511677914 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbfe8003c60 con 0x7fc0000cb580 2023-10-08T05:50:23.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.609+0000 7fbffcff9640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbfe006afc0 0x7fbfe006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:23.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.609+0000 7fbffcff9640 1 -- 172.21.15.70:0/3511677914 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbfe8088f70 con 0x7fc0000cb580 2023-10-08T05:50:23.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.609+0000 7fbfff7fe640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbfe006afc0 0x7fbfe006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:23.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.610+0000 7fbfff7fe640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbfe006afc0 0x7fbfe006d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbff0036000 tx=0x7fbff003b040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:23.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.613+0000 7fbffcff9640 1 -- 172.21.15.70:0/3511677914 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0000c4280 con 0x7fc0000cb580 2023-10-08T05:50:23.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.843+0000 7fc005a6f640 1 -- 172.21.15.70:0/3511677914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc0000c4280 con 0x7fc0000cb580 2023-10-08T05:50:23.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.844+0000 7fbffcff9640 1 -- 172.21.15.70:0/3511677914 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc0000c4280 con 0x7fc0000cb580 2023-10-08T05:50:23.845 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T05:50:23.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.846+0000 7fc005a6f640 1 -- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbfe006afc0 msgr2=0x7fbfe006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.846+0000 7fc005a6f640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbfe006afc0 0x7fbfe006d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbff0036000 tx=0x7fbff003b040 comp rx=0 tx=0).stop 2023-10-08T05:50:23.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.846+0000 7fc005a6f640 1 -- 172.21.15.70:0/3511677914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000cb580 msgr2=0x7fc0000b3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:23.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.846+0000 7fc005a6f640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000cb580 0x7fc0000b3bd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800c390 tx=0x7fbfe800c850 comp rx=0 tx=0).stop 2023-10-08T05:50:23.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.847+0000 7fc005a6f640 1 -- 172.21.15.70:0/3511677914 shutdown_connections 2023-10-08T05:50:23.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.847+0000 7fc005a6f640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fbfe006afc0 0x7fbfe006d480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.847+0000 7fc005a6f640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.847+0000 7fc005a6f640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000cb580 0x7fc0000b3bd0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.847+0000 7fc005a6f640 1 --2- 172.21.15.70:0/3511677914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000c90b0 0x7fc0000b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:23.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.847+0000 7fc005a6f640 1 -- 172.21.15.70:0/3511677914 >> 172.21.15.70:0/3511677914 conn(0x7fc00006a380 msgr2=0x7fc0000701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:23.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.847+0000 7fc005a6f640 1 -- 172.21.15.70:0/3511677914 shutdown_connections 2023-10-08T05:50:23.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:23.847+0000 7fc005a6f640 1 -- 172.21.15.70:0/3511677914 wait complete. 2023-10-08T05:50:23.867 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T05:50:23.867 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:50:23.867 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:50:23.867 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T05:50:23.867 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T05:50:23.867 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T05:50:23.867 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T05:50:23.867 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:50:23.867 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:50:23.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:23.868 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:50:23.920 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:50:23.920 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:50:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:50:23.922 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:50:23.922 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:50:23.922 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:50:23.922 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41572sec preferred_lft 41572sec 2023-10-08T05:50:23.922 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:50:23.922 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:50:23.922 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:50:23.922 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:50:23.923 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T05:50:23.923 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:50:23.923 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T05:50:23.923 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:50:23.923 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:50:23.923 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T05:50:23.923 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T05:50:23.923 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:50:23.923 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:50:23.924 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T05:50:23.925 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T05:50:23.925 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:50:23.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:23.926 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:50:24.018 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:50:24.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:24.023 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:50:24.081 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:50:24.081 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:50:24.081 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:50:24.081 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:50:24.082 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:50:24.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:24.082 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:50:24.082 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:50:24.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:50:24.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:50:24.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:50:24.083 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:50:24.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:24.269 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:50:24.318 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T05:50:24.319 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T05:50:24.320 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:50:24.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:24.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T05:50:24.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:50:24.417 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T05:50:24.475 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:50:24.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:50:24.475 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:50:24.529 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T05:50:24.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:24.529 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:50:24.670 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:50:24.670 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:50:24.703 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:50:24.703 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:50:24.703 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:50:24.703 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:50:24.704 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:50:24.704 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T05:50:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:50:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:50:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:50:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:50:24.709 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:50:24.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:24.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:50:24.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:24.789 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:50:24.856 INFO:teuthology.orchestra.run.smithi070.stdout:client4985 2023-10-08T05:50:24.857 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:50:24.857 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:50:24.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T05:50:25.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.160+0000 7f481e10f640 1 Processor -- start 2023-10-08T05:50:25.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.160+0000 7f481e10f640 1 -- start start 2023-10-08T05:50:25.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.161+0000 7f481e10f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48180afbe0 0x7f48180b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.161+0000 7f481e10f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481806dfc0 0x7f4818070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.161+0000 7f481e10f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180b2620 con 0x7f4818071070 2023-10-08T05:50:25.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.161+0000 7f481e10f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48180b2060 con 0x7f48180afbe0 2023-10-08T05:50:25.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.161+0000 7f481e10f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48180b2340 con 0x7f481806dfc0 2023-10-08T05:50:25.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.162+0000 7f48177fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48180afbe0 0x7f48180b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:25.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.162+0000 7f480ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481806dfc0 0x7f4818070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:25.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.162+0000 7f48177fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48180afbe0 0x7f48180b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:34180/0 (socket says 172.21.15.70:34180) 2023-10-08T05:50:25.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.162+0000 7f48177fe640 1 -- 172.21.15.70:0/806879738 learned_addr learned my addr 172.21.15.70:0/806879738 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:25.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.162+0000 7f48177fe640 1 -- 172.21.15.70:0/806879738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481806dfc0 msgr2=0x7f4818070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.162+0000 7f48177fe640 1 --2- 172.21.15.70:0/806879738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481806dfc0 0x7f4818070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.162+0000 7f48177fe640 1 -- 172.21.15.70:0/806879738 >> v1:172.21.15.181:6789/0 conn(0x7f4818071070 legacy=0x7f48180cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.162+0000 7f48177fe640 1 -- 172.21.15.70:0/806879738 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818070350 con 0x7f48180afbe0 2023-10-08T05:50:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.164+0000 7f48177fe640 1 --2- 172.21.15.70:0/806879738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48180afbe0 0x7f48180b15b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4808009870 tx=0x7f480802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9798f51a17a58e8f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.164+0000 7f4816ffd640 1 -- 172.21.15.70:0/806879738 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808030840 con 0x7f48180afbe0 2023-10-08T05:50:25.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.164+0000 7f4816ffd640 1 -- 172.21.15.70:0/806879738 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48080309a0 con 0x7f48180afbe0 2023-10-08T05:50:25.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.164+0000 7f4816ffd640 1 -- 172.21.15.70:0/806879738 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808030d30 con 0x7f48180afbe0 2023-10-08T05:50:25.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.165+0000 7f481e10f640 1 -- 172.21.15.70:0/806879738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48180afbe0 msgr2=0x7f48180b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:25.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.165+0000 7f481e10f640 1 --2- 172.21.15.70:0/806879738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48180afbe0 0x7f48180b15b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4808009870 tx=0x7f480802b980 comp rx=0 tx=0).stop 2023-10-08T05:50:25.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.165+0000 7f481e10f640 1 -- 172.21.15.70:0/806879738 shutdown_connections 2023-10-08T05:50:25.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.165+0000 7f481e10f640 1 --2- 172.21.15.70:0/806879738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f481806dfc0 0x7f4818070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.165+0000 7f481e10f640 1 --2- 172.21.15.70:0/806879738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48180afbe0 0x7f48180b15b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.165+0000 7f481e10f640 1 -- 172.21.15.70:0/806879738 >> 172.21.15.70:0/806879738 conn(0x7f481806a5d0 msgr2=0x7f48180b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:25.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.165+0000 7f481e10f640 1 -- 172.21.15.70:0/806879738 shutdown_connections 2023-10-08T05:50:25.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.165+0000 7f481e10f640 1 -- 172.21.15.70:0/806879738 wait complete. 2023-10-08T05:50:25.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.167+0000 7f481e10f640 1 Processor -- start 2023-10-08T05:50:25.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.167+0000 7f481e10f640 1 -- start start 2023-10-08T05:50:25.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.168+0000 7f481e10f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f481806dfc0 0x7f481815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.168+0000 7f481e10f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4818071070 0x7f48180c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.168+0000 7f481e10f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48180afbe0 0x7f4818151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.168+0000 7f481e10f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481815c770 con 0x7f481806dfc0 2023-10-08T05:50:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.168+0000 7f481e10f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48180c3180 con 0x7f4818071070 2023-10-08T05:50:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.168+0000 7f481e10f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481815bb60 con 0x7f48180afbe0 2023-10-08T05:50:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.168+0000 7f48177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f481806dfc0 0x7f481815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:25.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.168+0000 7f48177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f481806dfc0 0x7f481815b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46498/0 (socket says 172.21.15.70:46498) 2023-10-08T05:50:25.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.168+0000 7f48177fe640 1 -- 172.21.15.70:0/3395740317 learned_addr learned my addr 172.21.15.70:0/3395740317 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:25.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.169+0000 7f480ffff640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4818071070 0x7f48180c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:25.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.169+0000 7f48177fe640 1 -- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48180afbe0 msgr2=0x7f4818151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:25.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.169+0000 7f48177fe640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48180afbe0 0x7f4818151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.169+0000 7f48177fe640 1 -- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4818071070 msgr2=0x7f48180c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.169+0000 7f48177fe640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4818071070 0x7f48180c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.169+0000 7f48177fe640 1 -- 172.21.15.70:0/3395740317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48080094d0 con 0x7f481806dfc0 2023-10-08T05:50:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.169+0000 7f48177fe640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f481806dfc0 0x7f481815b4d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4808000c00 tx=0x7f4808004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.170+0000 7f4814ff9640 1 -- 172.21.15.70:0/3395740317 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808039c90 con 0x7f481806dfc0 2023-10-08T05:50:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.170+0000 7f4814ff9640 1 -- 172.21.15.70:0/3395740317 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4808039df0 con 0x7f481806dfc0 2023-10-08T05:50:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.170+0000 7f4814ff9640 1 -- 172.21.15.70:0/3395740317 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808043640 con 0x7f481806dfc0 2023-10-08T05:50:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.170+0000 7f481e10f640 1 -- 172.21.15.70:0/3395740317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481815be40 con 0x7f481806dfc0 2023-10-08T05:50:25.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.170+0000 7f481e10f640 1 -- 172.21.15.70:0/3395740317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481815c400 con 0x7f481806dfc0 2023-10-08T05:50:25.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.171+0000 7f481e10f640 1 -- 172.21.15.70:0/3395740317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481804e160 con 0x7f481806dfc0 2023-10-08T05:50:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.172+0000 7f4814ff9640 1 -- 172.21.15.70:0/3395740317 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f48080437a0 con 0x7f481806dfc0 2023-10-08T05:50:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.173+0000 7f4814ff9640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f47f806afc0 0x7f47f806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.174+0000 7f4814ff9640 1 -- 172.21.15.70:0/3395740317 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f480803e070 con 0x7f481806dfc0 2023-10-08T05:50:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.174+0000 7f480ffff640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f47f806afc0 0x7f47f806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.175+0000 7f480ffff640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f47f806afc0 0x7f47f806d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4818071830 tx=0x7f4800008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:25.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.178+0000 7f4814ff9640 1 -- 172.21.15.70:0/3395740317 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4808077820 con 0x7f481806dfc0 2023-10-08T05:50:25.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.391+0000 7f481e10f640 1 -- 172.21.15.70:0/3395740317 --> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f48180affc0 con 0x7f47f806afc0 2023-10-08T05:50:25.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.396+0000 7f4814ff9640 1 -- 172.21.15.70:0/3395740317 <== mgr.4844 v2:172.21.15.181:6832/992174303 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f48000096a0 con 0x7f47f806afc0 2023-10-08T05:50:25.397 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:50:25.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:50:25.397 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T05:50:25.397 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:50:25.397 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:50:25.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 -- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f47f806afc0 msgr2=0x7f47f806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:25.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f47f806afc0 0x7f47f806d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4818071830 tx=0x7f4800008040 comp rx=0 tx=0).stop 2023-10-08T05:50:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 -- 172.21.15.70:0/3395740317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f481806dfc0 msgr2=0x7f481815b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f481806dfc0 0x7f481815b4d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4808000c00 tx=0x7f4808004730 comp rx=0 tx=0).stop 2023-10-08T05:50:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 -- 172.21.15.70:0/3395740317 shutdown_connections 2023-10-08T05:50:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f47f806afc0 0x7f47f806d480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48180afbe0 0x7f4818151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4818071070 0x7f48180c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 --2- 172.21.15.70:0/3395740317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f481806dfc0 0x7f481815b4d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 -- 172.21.15.70:0/3395740317 >> 172.21.15.70:0/3395740317 conn(0x7f481806a5d0 msgr2=0x7f48180c6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 -- 172.21.15.70:0/3395740317 shutdown_connections 2023-10-08T05:50:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.399+0000 7f481e10f640 1 -- 172.21.15.70:0/3395740317 wait complete. 2023-10-08T05:50:25.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T05:50:25.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.632+0000 7f088b1b0640 1 Processor -- start 2023-10-08T05:50:25.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.633+0000 7f088b1b0640 1 -- start start 2023-10-08T05:50:25.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.634+0000 7f088b1b0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f08840c9080 0x7f08840c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.634+0000 7f088b1b0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0884070b50 0x7f0884070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.634+0000 7f088b1b0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08840c7ad0 con 0x7f088415a820 2023-10-08T05:50:25.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.634+0000 7f088b1b0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08840c9a40 con 0x7f0884070b50 2023-10-08T05:50:25.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.634+0000 7f088b1b0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088415ac00 con 0x7f08840c9080 2023-10-08T05:50:25.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.634+0000 7f088974d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0884070b50 0x7f0884070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:25.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.634+0000 7f0888f4c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f088415a820 0x7f0884160750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33258/0 (socket says 172.21.15.70:33258) 2023-10-08T05:50:25.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.635+0000 7f088974d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0884070b50 0x7f0884070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46504/0 (socket says 172.21.15.70:46504) 2023-10-08T05:50:25.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.635+0000 7f0888f4c640 1 -- 172.21.15.70:0/41489562 learned_addr learned my addr 172.21.15.70:0/41489562 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:25.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.635+0000 7f0883fff640 1 --2- 172.21.15.70:0/41489562 >> v2:172.21.15.181:3300/0 conn(0x7f08840c9080 0x7f08840c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:25.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.635+0000 7f088974d640 1 -- 172.21.15.70:0/41489562 >> v2:172.21.15.181:3300/0 conn(0x7f08840c9080 msgr2=0x7f08840c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:25.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.635+0000 7f088974d640 1 --2- 172.21.15.70:0/41489562 >> v2:172.21.15.181:3300/0 conn(0x7f08840c9080 0x7f08840c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.635+0000 7f088974d640 1 -- 172.21.15.70:0/41489562 >> v1:172.21.15.70:6789/0 conn(0x7f088415a820 legacy=0x7f0884160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:25.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.635+0000 7f088974d640 1 -- 172.21.15.70:0/41489562 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08840714e0 con 0x7f0884070b50 2023-10-08T05:50:25.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.635+0000 7f0883fff640 1 --2- 172.21.15.70:0/41489562 >> v2:172.21.15.181:3300/0 conn(0x7f08840c9080 0x7f08840c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:25.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.636+0000 7f088974d640 1 --2- 172.21.15.70:0/41489562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 0x7f0884070fa0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f088406ab90 tx=0x7f087c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaad9092a73aee4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:25.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.637+0000 7f08837fe640 1 -- 172.21.15.70:0/41489562 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c02cc70 con 0x7f0884070b50 2023-10-08T05:50:25.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.637+0000 7f08837fe640 1 -- 172.21.15.70:0/41489562 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c02cdd0 con 0x7f0884070b50 2023-10-08T05:50:25.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.637+0000 7f08837fe640 1 -- 172.21.15.70:0/41489562 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c034660 con 0x7f0884070b50 2023-10-08T05:50:25.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.637+0000 7f088b1b0640 1 -- 172.21.15.70:0/41489562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 msgr2=0x7f0884070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:25.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.637+0000 7f088b1b0640 1 --2- 172.21.15.70:0/41489562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 0x7f0884070fa0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f088406ab90 tx=0x7f087c02bec0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.637+0000 7f088b1b0640 1 -- 172.21.15.70:0/41489562 shutdown_connections 2023-10-08T05:50:25.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.637+0000 7f088b1b0640 1 --2- 172.21.15.70:0/41489562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 0x7f0884070fa0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.637+0000 7f088b1b0640 1 --2- 172.21.15.70:0/41489562 >> v2:172.21.15.181:3300/0 conn(0x7f08840c9080 0x7f08840c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.637+0000 7f088b1b0640 1 -- 172.21.15.70:0/41489562 >> 172.21.15.70:0/41489562 conn(0x7f0884069d50 msgr2=0x7f088406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:25.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.638+0000 7f088b1b0640 1 -- 172.21.15.70:0/41489562 shutdown_connections 2023-10-08T05:50:25.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.638+0000 7f088b1b0640 1 -- 172.21.15.70:0/41489562 wait complete. 2023-10-08T05:50:25.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.639+0000 7f088b1b0640 1 Processor -- start 2023-10-08T05:50:25.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.639+0000 7f088b1b0640 1 -- start start 2023-10-08T05:50:25.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.640+0000 7f088b1b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 0x7f0884160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.640+0000 7f088b1b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08840c9080 0x7f08840b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.640+0000 7f088b1b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f088415a820 0x7f08841573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.640+0000 7f088b1b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08840b2be0 con 0x7f0884070b50 2023-10-08T05:50:25.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.640+0000 7f088b1b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08841548c0 con 0x7f088415a820 2023-10-08T05:50:25.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.640+0000 7f088b1b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08840b2e30 con 0x7f08840c9080 2023-10-08T05:50:25.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.641+0000 7f0888f4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 0x7f0884160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.641+0000 7f0888f4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 0x7f0884160040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46518/0 (socket says 172.21.15.70:46518) 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.641+0000 7f0888f4c640 1 -- 172.21.15.70:0/85623651 learned_addr learned my addr 172.21.15.70:0/85623651 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.641+0000 7f0883fff640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08840c9080 0x7f08840b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.641+0000 7f0888f4c640 1 -- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08840c9080 msgr2=0x7f08840b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.641+0000 7f0888f4c640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08840c9080 0x7f08840b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.641+0000 7f0888f4c640 1 -- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f088415a820 msgr2=0x7f08841573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.641+0000 7f0888f4c640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f088415a820 0x7f08841573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.641+0000 7f0888f4c640 1 -- 172.21.15.70:0/85623651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c009a70 con 0x7f0884070b50 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.641+0000 7f0888f4c640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 0x7f0884160040 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f087800c4b0 tx=0x7f087800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.642+0000 7f08817fa640 1 -- 172.21.15.70:0/85623651 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878005ce0 con 0x7f0884070b50 2023-10-08T05:50:25.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.642+0000 7f08817fa640 1 -- 172.21.15.70:0/85623651 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0878005e40 con 0x7f0884070b50 2023-10-08T05:50:25.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.642+0000 7f08817fa640 1 -- 172.21.15.70:0/85623651 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878007270 con 0x7f0884070b50 2023-10-08T05:50:25.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.642+0000 7f088b1b0640 1 -- 172.21.15.70:0/85623651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08840b3110 con 0x7f0884070b50 2023-10-08T05:50:25.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.642+0000 7f088b1b0640 1 -- 172.21.15.70:0/85623651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08840b36d0 con 0x7f0884070b50 2023-10-08T05:50:25.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.643+0000 7f088b1b0640 1 -- 172.21.15.70:0/85623651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088404e160 con 0x7f0884070b50 2023-10-08T05:50:25.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.644+0000 7f08817fa640 1 -- 172.21.15.70:0/85623651 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f08780073d0 con 0x7f0884070b50 2023-10-08T05:50:25.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.645+0000 7f08817fa640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f086806af90 0x7f086806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:25.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.646+0000 7f08817fa640 1 -- 172.21.15.70:0/85623651 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0878088ce0 con 0x7f0884070b50 2023-10-08T05:50:25.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.646+0000 7f0883fff640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f086806af90 0x7f086806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:25.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.647+0000 7f0883fff640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f086806af90 0x7f086806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f08841f54b0 tx=0x7f0870008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:25.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.650+0000 7f08817fa640 1 -- 172.21.15.70:0/85623651 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0878053bb0 con 0x7f0884070b50 2023-10-08T05:50:25.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:25.889+0000 7f088b1b0640 1 -- 172.21.15.70:0/85623651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f08840cdfb0 con 0x7f0884070b50 2023-10-08T05:50:26.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.767+0000 7f08817fa640 1 -- 172.21.15.70:0/85623651 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0878017020 con 0x7f0884070b50 2023-10-08T05:50:26.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.770+0000 7f088b1b0640 1 -- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f086806af90 msgr2=0x7f086806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:26.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.770+0000 7f088b1b0640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f086806af90 0x7f086806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f08841f54b0 tx=0x7f0870008040 comp rx=0 tx=0).stop 2023-10-08T05:50:26.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.770+0000 7f088b1b0640 1 -- 172.21.15.70:0/85623651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 msgr2=0x7f0884160040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:26.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.770+0000 7f088b1b0640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 0x7f0884160040 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f087800c4b0 tx=0x7f087800c970 comp rx=0 tx=0).stop 2023-10-08T05:50:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.771+0000 7f088b1b0640 1 -- 172.21.15.70:0/85623651 shutdown_connections 2023-10-08T05:50:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.771+0000 7f088b1b0640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f088415a820 0x7f08841573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.771+0000 7f088b1b0640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f086806af90 0x7f086806d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.771+0000 7f088b1b0640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08840c9080 0x7f08840b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.771+0000 7f088b1b0640 1 --2- 172.21.15.70:0/85623651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0884070b50 0x7f0884160040 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.771+0000 7f088b1b0640 1 -- 172.21.15.70:0/85623651 >> 172.21.15.70:0/85623651 conn(0x7f0884069d50 msgr2=0x7f08840cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:26.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.771+0000 7f088b1b0640 1 -- 172.21.15.70:0/85623651 shutdown_connections 2023-10-08T05:50:26.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.771+0000 7f088b1b0640 1 -- 172.21.15.70:0/85623651 wait complete. 2023-10-08T05:50:26.794 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T05:50:26.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.990+0000 7fdbcdd9e640 1 Processor -- start 2023-10-08T05:50:26.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.990+0000 7fdbcdd9e640 1 -- start start 2023-10-08T05:50:26.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.992+0000 7fdbcdd9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80cdb40 0x7fdbc80b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:26.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.992+0000 7fdbcdd9e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc80b69d0 con 0x7fdbc815abe0 2023-10-08T05:50:26.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.992+0000 7fdbcdd9e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc80cf770 con 0x7fdbc80d0420 2023-10-08T05:50:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.992+0000 7fdbcdd9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc80b67e0 con 0x7fdbc80cdb40 2023-10-08T05:50:26.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.992+0000 7fdbc77fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdbc815abe0 0x7fdbc80cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33268/0 (socket says 172.21.15.70:33268) 2023-10-08T05:50:26.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.992+0000 7fdbc77fe640 1 -- 172.21.15.70:0/3127246834 learned_addr learned my addr 172.21.15.70:0/3127246834 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:26.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc7fff640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80cdb40 0x7fdbc80b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:26.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834889175 0 0) 0x7fdbc80b69d0 con 0x7fdbc815abe0 2023-10-08T05:50:26.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbb4003520 con 0x7fdbc815abe0 2023-10-08T05:50:26.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3956433241 0 0) 0x7fdbb4003520 con 0x7fdbc815abe0 2023-10-08T05:50:26.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80cdb40 msgr2=0x7fdbc80b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:26.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80cdb40 0x7fdbc80b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:26.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 >> v1:172.21.15.181:6789/0 conn(0x7fdbc80d0420 legacy=0x7fdbc80cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:26.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc80d0880 con 0x7fdbc815abe0 2023-10-08T05:50:26.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fdbb80017f0 con 0x7fdbc815abe0 2023-10-08T05:50:26.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 >> v1:172.21.15.70:6789/0 conn(0x7fdbc815abe0 legacy=0x7fdbc80cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:26.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb40039a0 0x7fdbb4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb4004350 0x7fdbb4004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80cdb40 0x7fdbb4005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb400b980 con 0x7fdbb40039a0 2023-10-08T05:50:27.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb400baf0 con 0x7fdbb4004350 2023-10-08T05:50:27.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.993+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb400bcb0 con 0x7fdbc80cdb40 2023-10-08T05:50:27.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.994+0000 7fdbc77fe640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb40039a0 0x7fdbb4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.994+0000 7fdbc7fff640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80cdb40 0x7fdbb4005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.994+0000 7fdbc77fe640 1 -- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80cdb40 msgr2=0x7fdbb4005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:27.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.994+0000 7fdbc77fe640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80cdb40 0x7fdbb4005080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.994+0000 7fdbc77fe640 1 -- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb4004350 msgr2=0x7fdbb4004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:27.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.994+0000 7fdbbffff640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb4004350 0x7fdbb4004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.994+0000 7fdbc77fe640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb4004350 0x7fdbb4004820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.994+0000 7fdbc77fe640 1 -- 172.21.15.70:0/3127246834 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb4003520 con 0x7fdbb40039a0 2023-10-08T05:50:27.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.995+0000 7fdbc77fe640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb40039a0 0x7fdbb4003d80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c240 tx=0x7fdbb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc437bbe33ea6b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:27.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.995+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb802cad0 con 0x7fdbb40039a0 2023-10-08T05:50:27.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.995+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb802cc30 con 0x7fdbb40039a0 2023-10-08T05:50:27.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.995+0000 7fdbc6ffd640 1 -- 172.21.15.70:0/3127246834 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8007070 con 0x7fdbb40039a0 2023-10-08T05:50:27.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.996+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/3127246834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb40039a0 msgr2=0x7fdbb4003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.996+0000 7fdbcdd9e640 1 --2- 172.21.15.70:0/3127246834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb40039a0 0x7fdbb4003d80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c240 tx=0x7fdbb40057f0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.996+0000 7fdbc7fff640 1 -- 172.21.15.70:0/3127246834 reap_dead start 2023-10-08T05:50:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.996+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/3127246834 shutdown_connections 2023-10-08T05:50:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.996+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/3127246834 >> 172.21.15.70:0/3127246834 conn(0x7fdbc806e650 msgr2=0x7fdbc806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.996+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/3127246834 shutdown_connections 2023-10-08T05:50:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.996+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/3127246834 wait complete. 2023-10-08T05:50:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.998+0000 7fdbcdd9e640 1 Processor -- start 2023-10-08T05:50:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.998+0000 7fdbcdd9e640 1 -- start start 2023-10-08T05:50:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.999+0000 7fdbcdd9e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40039a0 0x7fdbc81529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.999+0000 7fdbcdd9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80c7210 0x7fdbc815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.999+0000 7fdbcdd9e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbc815d300 0x7fdbc80cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.999+0000 7fdbcdd9e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc8152850 con 0x7fdbc815d300 2023-10-08T05:50:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.999+0000 7fdbcdd9e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc8152f30 con 0x7fdbb40039a0 2023-10-08T05:50:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.999+0000 7fdbcdd9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc815d830 con 0x7fdbc80c7210 2023-10-08T05:50:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:26.999+0000 7fdbc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbc815d300 0x7fdbc80cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbc815d300 0x7fdbc80cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46574/0 (socket says 172.21.15.70:46574) 2023-10-08T05:50:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc7fff640 1 -- 172.21.15.70:0/2789002162 learned_addr learned my addr 172.21.15.70:0/2789002162 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbbffff640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80c7210 0x7fdbc815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc77fe640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40039a0 0x7fdbc81529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc7fff640 1 -- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80c7210 msgr2=0x7fdbc815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc7fff640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80c7210 0x7fdbc815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc7fff640 1 -- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40039a0 msgr2=0x7fdbc81529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc7fff640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40039a0 0x7fdbc81529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc7fff640 1 -- 172.21.15.70:0/2789002162 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc815db10 con 0x7fdbc815d300 2023-10-08T05:50:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc77fe640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40039a0 0x7fdbc81529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc7fff640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbc815d300 0x7fdbc80cb190 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8072240 tx=0x7fdbc00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc4ff9640 1 -- 172.21.15.70:0/2789002162 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000e070 con 0x7fdbc815d300 2023-10-08T05:50:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc4ff9640 1 -- 172.21.15.70:0/2789002162 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0011ed0 con 0x7fdbc815d300 2023-10-08T05:50:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbc4ff9640 1 -- 172.21.15.70:0/2789002162 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0003990 con 0x7fdbc815d300 2023-10-08T05:50:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.000+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/2789002162 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc815ddf0 con 0x7fdbc815d300 2023-10-08T05:50:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.001+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/2789002162 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc8155a80 con 0x7fdbc815d300 2023-10-08T05:50:27.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.002+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/2789002162 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc80b1460 con 0x7fdbc815d300 2023-10-08T05:50:27.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.003+0000 7fdbc4ff9640 1 -- 172.21.15.70:0/2789002162 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fdbc0003af0 con 0x7fdbc815d300 2023-10-08T05:50:27.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.004+0000 7fdbc4ff9640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdba806afe0 0x7fdba806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.004+0000 7fdbc4ff9640 1 -- 172.21.15.70:0/2789002162 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdbc008a0e0 con 0x7fdbc815d300 2023-10-08T05:50:27.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.004+0000 7fdbc77fe640 1 -- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdba806afe0 msgr2=0x7fdba806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/992174303 2023-10-08T05:50:27.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.004+0000 7fdbc77fe640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdba806afe0 0x7fdba806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:50:27.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.008+0000 7fdbc4ff9640 1 -- 172.21.15.70:0/2789002162 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdbc80b1460 con 0x7fdbc815d300 2023-10-08T05:50:27.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.197+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/2789002162 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fdbc80b1840 con 0x7fdbc815d300 2023-10-08T05:50:27.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.201+0000 7fdbc4ff9640 1 -- 172.21.15.70:0/2789002162 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdbc80b1840 con 0x7fdbc815d300 2023-10-08T05:50:27.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdba806afe0 msgr2=0x7fdba806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:50:27.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdba806afe0 0x7fdba806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/2789002162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbc815d300 msgr2=0x7fdbc80cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbc815d300 0x7fdbc80cb190 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8072240 tx=0x7fdbc00115a0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/2789002162 shutdown_connections 2023-10-08T05:50:27.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbc815d300 0x7fdbc80cb190 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7fdba806afe0 0x7fdba806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbc80c7210 0x7fdbc815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 --2- 172.21.15.70:0/2789002162 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbb40039a0 0x7fdbc81529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/2789002162 >> 172.21.15.70:0/2789002162 conn(0x7fdbc806e650 msgr2=0x7fdbc80b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:27.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/2789002162 shutdown_connections 2023-10-08T05:50:27.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.204+0000 7fdbcdd9e640 1 -- 172.21.15.70:0/2789002162 wait complete. 2023-10-08T05:50:27.221 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T05:50:27.290 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:27.290+0000 7f9d2a23b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:50:27.292 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:27.291+0000 7ff61d816040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:50:27.344 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:27.343+0000 7ff61d816040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:50:27.346 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:27.346+0000 7f9d2a23b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:50:27.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.434+0000 7ffb07c5d640 1 Processor -- start 2023-10-08T05:50:27.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.434+0000 7ffb07c5d640 1 -- start start 2023-10-08T05:50:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.435+0000 7ffb07c5d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb0015ce00 0x7ffb000cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.435+0000 7ffb07c5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffb000cd470 0x7ffb000cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.435+0000 7ffb07c5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb000cde10 con 0x7ffb00160980 2023-10-08T05:50:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.435+0000 7ffb07c5d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb00161920 con 0x7ffb000cd470 2023-10-08T05:50:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.435+0000 7ffb07c5d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb000cb8b0 con 0x7ffb0015ce00 2023-10-08T05:50:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.436+0000 7ffb051f8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffb000cd470 0x7ffb000cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.436+0000 7ffb051f8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffb000cd470 0x7ffb000cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46588/0 (socket says 172.21.15.70:46588) 2023-10-08T05:50:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.436+0000 7ffb051f8640 1 -- 172.21.15.70:0/1610941754 learned_addr learned my addr 172.21.15.70:0/1610941754 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.436+0000 7ffb059f9640 1 --2- 172.21.15.70:0/1610941754 >> v2:172.21.15.181:3300/0 conn(0x7ffb0015ce00 0x7ffb000cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.436+0000 7ffb051f8640 1 -- 172.21.15.70:0/1610941754 >> v2:172.21.15.181:3300/0 conn(0x7ffb0015ce00 msgr2=0x7ffb000cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.436+0000 7ffb051f8640 1 --2- 172.21.15.70:0/1610941754 >> v2:172.21.15.181:3300/0 conn(0x7ffb0015ce00 0x7ffb000cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.436+0000 7ffb051f8640 1 -- 172.21.15.70:0/1610941754 >> v1:172.21.15.70:6789/0 conn(0x7ffb00160980 legacy=0x7ffb00160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.436+0000 7ffb051f8640 1 -- 172.21.15.70:0/1610941754 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb00161640 con 0x7ffb000cd470 2023-10-08T05:50:27.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.436+0000 7ffb049f7640 1 -- 172.21.15.70:0/1610941754 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267830059 0 0) 0x7ffb000cde10 con 0x7ffb00160980 2023-10-08T05:50:27.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.437+0000 7ffb051f8640 1 --2- 172.21.15.70:0/1610941754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb000cd470 0x7ffb000cd8d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffb00071120 tx=0x7ffaf402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832203594f8dd4c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb049f7640 1 -- 172.21.15.70:0/1610941754 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4031870 con 0x7ffb000cd470 2023-10-08T05:50:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb049f7640 1 -- 172.21.15.70:0/1610941754 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf40319d0 con 0x7ffb000cd470 2023-10-08T05:50:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb049f7640 1 -- 172.21.15.70:0/1610941754 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4031d60 con 0x7ffb000cd470 2023-10-08T05:50:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb07c5d640 1 -- 172.21.15.70:0/1610941754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb000cd470 msgr2=0x7ffb000cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb07c5d640 1 --2- 172.21.15.70:0/1610941754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb000cd470 0x7ffb000cd8d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffb00071120 tx=0x7ffaf402b910 comp rx=0 tx=0).stop 2023-10-08T05:50:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb07c5d640 1 -- 172.21.15.70:0/1610941754 shutdown_connections 2023-10-08T05:50:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb07c5d640 1 --2- 172.21.15.70:0/1610941754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb000cd470 0x7ffb000cd8d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb07c5d640 1 --2- 172.21.15.70:0/1610941754 >> v2:172.21.15.181:3300/0 conn(0x7ffb0015ce00 0x7ffb000cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb07c5d640 1 -- 172.21.15.70:0/1610941754 >> 172.21.15.70:0/1610941754 conn(0x7ffb0006e650 msgr2=0x7ffb0006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb07c5d640 1 -- 172.21.15.70:0/1610941754 shutdown_connections 2023-10-08T05:50:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.438+0000 7ffb07c5d640 1 -- 172.21.15.70:0/1610941754 wait complete. 2023-10-08T05:50:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.440+0000 7ffb07c5d640 1 Processor -- start 2023-10-08T05:50:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.440+0000 7ffb07c5d640 1 -- start start 2023-10-08T05:50:27.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.441+0000 7ffb07c5d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb000cd470 0x7ffb000b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.441+0000 7ffb07c5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb0015ce00 0x7ffb000c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.441+0000 7ffb07c5d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00160980 0x7ffb001f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.441+0000 7ffb07c5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000b0b90 con 0x7ffb0015ce00 2023-10-08T05:50:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.441+0000 7ffb07c5d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000b0e70 con 0x7ffb000cd470 2023-10-08T05:50:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.441+0000 7ffb07c5d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb000b1150 con 0x7ffb00160980 2023-10-08T05:50:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.442+0000 7ffb051f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb0015ce00 0x7ffb000c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.442+0000 7ffb051f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb0015ce00 0x7ffb000c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46596/0 (socket says 172.21.15.70:46596) 2023-10-08T05:50:27.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.442+0000 7ffb051f8640 1 -- 172.21.15.70:0/3248173686 learned_addr learned my addr 172.21.15.70:0/3248173686 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:27.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.442+0000 7ffb059f9640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb000cd470 0x7ffb000b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.442+0000 7ffb051f8640 1 -- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00160980 msgr2=0x7ffb001f5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.442+0000 7ffb051f8640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00160980 0x7ffb001f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.442+0000 7ffb051f8640 1 -- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb000cd470 msgr2=0x7ffb000b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.442+0000 7ffb051f8640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb000cd470 0x7ffb000b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.442+0000 7ffb051f8640 1 -- 172.21.15.70:0/3248173686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40094d0 con 0x7ffb0015ce00 2023-10-08T05:50:27.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.443+0000 7ffb051f8640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb0015ce00 0x7ffb000c65a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4000c00 tx=0x7ffaf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:27.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.443+0000 7ffaf27fc640 1 -- 172.21.15.70:0/3248173686 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4004230 con 0x7ffb0015ce00 2023-10-08T05:50:27.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.443+0000 7ffaf27fc640 1 -- 172.21.15.70:0/3248173686 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf40043b0 con 0x7ffb0015ce00 2023-10-08T05:50:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.443+0000 7ffaf27fc640 1 -- 172.21.15.70:0/3248173686 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4004520 con 0x7ffb0015ce00 2023-10-08T05:50:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.443+0000 7ffb07c5d640 1 -- 172.21.15.70:0/3248173686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb000b1430 con 0x7ffb0015ce00 2023-10-08T05:50:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.443+0000 7ffb07c5d640 1 -- 172.21.15.70:0/3248173686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb001f6520 con 0x7ffb0015ce00 2023-10-08T05:50:27.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.444+0000 7ffb07c5d640 1 -- 172.21.15.70:0/3248173686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb001f6710 con 0x7ffb0015ce00 2023-10-08T05:50:27.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.445+0000 7ffaf27fc640 1 -- 172.21.15.70:0/3248173686 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ffaf4042070 con 0x7ffb0015ce00 2023-10-08T05:50:27.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.446+0000 7ffaf27fc640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ffae406b010 0x7ffae406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.446+0000 7ffaf27fc640 1 -- 172.21.15.70:0/3248173686 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffaf402d040 con 0x7ffb0015ce00 2023-10-08T05:50:27.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.446+0000 7ffb059f9640 1 -- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ffae406b010 msgr2=0x7ffae406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/992174303 2023-10-08T05:50:27.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.446+0000 7ffb059f9640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ffae406b010 0x7ffae406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:50:27.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.450+0000 7ffaf27fc640 1 -- 172.21.15.70:0/3248173686 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffaf4076800 con 0x7ffb0015ce00 2023-10-08T05:50:27.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.639+0000 7ffb07c5d640 1 -- 172.21.15.70:0/3248173686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ffb000b1950 con 0x7ffb0015ce00 2023-10-08T05:50:27.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.644+0000 7ffaf27fc640 1 -- 172.21.15.70:0/3248173686 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7ffb000b1950 con 0x7ffb0015ce00 2023-10-08T05:50:27.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 -- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ffae406b010 msgr2=0x7ffae406d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:50:27.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ffae406b010 0x7ffae406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 -- 172.21.15.70:0/3248173686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb0015ce00 msgr2=0x7ffb000c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb0015ce00 0x7ffb000c65a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4000c00 tx=0x7ffaf40046c0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 -- 172.21.15.70:0/3248173686 shutdown_connections 2023-10-08T05:50:27.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7ffae406b010 0x7ffae406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb00160980 0x7ffb001f5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb0015ce00 0x7ffb000c65a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 --2- 172.21.15.70:0/3248173686 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb000cd470 0x7ffb000b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 -- 172.21.15.70:0/3248173686 >> 172.21.15.70:0/3248173686 conn(0x7ffb0006e650 msgr2=0x7ffb0015c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:27.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 -- 172.21.15.70:0/3248173686 shutdown_connections 2023-10-08T05:50:27.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.647+0000 7ffb07c5d640 1 -- 172.21.15.70:0/3248173686 wait complete. 2023-10-08T05:50:27.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:27.668 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T05:50:27.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-08T05:50:27.804 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:27.803+0000 7ff61d816040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:50:27.819 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:27.818+0000 7f9d2a23b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:50:27.856 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:27.855+0000 7ff61d816040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:50:27.872 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:27.872+0000 7f9d2a23b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:50:27.905 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:27.904+0000 7ff61d816040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:50:27.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.910+0000 7f0bc4239640 1 Processor -- start 2023-10-08T05:50:27.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.910+0000 7f0bc4239640 1 -- start start 2023-10-08T05:50:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.911+0000 7f0bc4239640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0bbc15cba0 0x7f0bbc0b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc4239640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bbc15cf80 0x7f0bbc0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc4239640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bbc0b54a0 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc4239640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bbc0b4fb0 con 0x7f0bbc15cba0 2023-10-08T05:50:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc4239640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0b7130 con 0x7f0bbc15cf80 2023-10-08T05:50:27.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc1fd5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0bbc15cba0 0x7f0bbc0b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc27d6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0bbc0b4b40 0x7f0bbc0b5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33298/0 (socket says 172.21.15.70:33298) 2023-10-08T05:50:27.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc1fd5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0bbc15cba0 0x7f0bbc0b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46612/0 (socket says 172.21.15.70:46612) 2023-10-08T05:50:27.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc27d6640 1 -- 172.21.15.70:0/69282414 learned_addr learned my addr 172.21.15.70:0/69282414 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:27.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc17d4640 1 --2- 172.21.15.70:0/69282414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bbc15cf80 0x7f0bbc0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc1fd5640 1 -- 172.21.15.70:0/69282414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bbc15cf80 msgr2=0x7f0bbc0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:27.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc1fd5640 1 --2- 172.21.15.70:0/69282414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bbc15cf80 0x7f0bbc0b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc1fd5640 1 -- 172.21.15.70:0/69282414 >> v1:172.21.15.70:6789/0 conn(0x7f0bbc0b4b40 legacy=0x7f0bbc0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc1fd5640 1 -- 172.21.15.70:0/69282414 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc0b7410 con 0x7f0bbc15cba0 2023-10-08T05:50:27.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.912+0000 7f0bc0fd3640 1 -- 172.21.15.70:0/69282414 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227816656 0 0) 0x7f0bb4001490 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.914+0000 7f0bc1fd5640 1 --2- 172.21.15.70:0/69282414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bbc15cba0 0x7f0bbc0b84a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80097a0 tx=0x7f0bb802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e267c5c48c3672ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:27.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.914+0000 7f0bc0fd3640 1 -- 172.21.15.70:0/69282414 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb802d040 con 0x7f0bbc15cba0 2023-10-08T05:50:27.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.914+0000 7f0bc0fd3640 1 -- 172.21.15.70:0/69282414 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb8031810 con 0x7f0bbc15cba0 2023-10-08T05:50:27.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.914+0000 7f0bc0fd3640 1 -- 172.21.15.70:0/69282414 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8031ba0 con 0x7f0bbc15cba0 2023-10-08T05:50:27.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.914+0000 7f0bc4239640 1 -- 172.21.15.70:0/69282414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bbc15cba0 msgr2=0x7f0bbc0b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.914+0000 7f0bc4239640 1 --2- 172.21.15.70:0/69282414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bbc15cba0 0x7f0bbc0b84a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0bb80097a0 tx=0x7f0bb802b8b0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.915+0000 7f0bc4239640 1 -- 172.21.15.70:0/69282414 shutdown_connections 2023-10-08T05:50:27.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.915+0000 7f0bc4239640 1 --2- 172.21.15.70:0/69282414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bbc15cf80 0x7f0bbc0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.915+0000 7f0bc4239640 1 --2- 172.21.15.70:0/69282414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bbc15cba0 0x7f0bbc0b84a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.915+0000 7f0bc4239640 1 -- 172.21.15.70:0/69282414 >> 172.21.15.70:0/69282414 conn(0x7f0bbc0718a0 msgr2=0x7f0bbc071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:27.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.915+0000 7f0bc4239640 1 -- 172.21.15.70:0/69282414 shutdown_connections 2023-10-08T05:50:27.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.915+0000 7f0bc4239640 1 -- 172.21.15.70:0/69282414 wait complete. 2023-10-08T05:50:27.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.916+0000 7f0bc4239640 1 Processor -- start 2023-10-08T05:50:27.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.916+0000 7f0bc4239640 1 -- start start 2023-10-08T05:50:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.917+0000 7f0bc4239640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bbc0b4b40 0x7f0bbc1f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.917+0000 7f0bc4239640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bbc15cba0 0x7f0bbc13dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.917+0000 7f0bc4239640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bbc15cf80 0x7f0bbc13fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.917+0000 7f0bc4239640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0c9c50 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.917+0000 7f0bc4239640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc1f33a0 con 0x7f0bbc15cba0 2023-10-08T05:50:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.917+0000 7f0bc4239640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0c9ea0 con 0x7f0bbc15cf80 2023-10-08T05:50:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.918+0000 7f0bc1fd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bbc0b4b40 0x7f0bbc1f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.918+0000 7f0bc1fd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bbc0b4b40 0x7f0bbc1f2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46618/0 (socket says 172.21.15.70:46618) 2023-10-08T05:50:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.918+0000 7f0bc1fd5640 1 -- 172.21.15.70:0/2093566927 learned_addr learned my addr 172.21.15.70:0/2093566927 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.918+0000 7f0bc17d4640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bbc15cba0 0x7f0bbc13dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.918+0000 7f0bc27d6640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bbc15cf80 0x7f0bbc13fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.918+0000 7f0bc1fd5640 1 -- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bbc15cf80 msgr2=0x7f0bbc13fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.918+0000 7f0bc1fd5640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bbc15cf80 0x7f0bbc13fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.918+0000 7f0bc1fd5640 1 -- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bbc15cba0 msgr2=0x7f0bbc13dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.918+0000 7f0bc1fd5640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bbc15cba0 0x7f0bbc13dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:27.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.918+0000 7f0bc1fd5640 1 -- 172.21.15.70:0/2093566927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8009400 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.919+0000 7f0bc1fd5640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bbc0b4b40 0x7f0bbc1f2e60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8000c00 tx=0x7f0bb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:27.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.919+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb803bae0 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.919+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb803bc40 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.919+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8004030 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.919+0000 7f0bc4239640 1 -- 172.21.15.70:0/2093566927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bbc0ca180 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.919+0000 7f0bc4239640 1 -- 172.21.15.70:0/2093566927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bbc0ca740 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.920+0000 7f0bc4239640 1 -- 172.21.15.70:0/2093566927 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bbc0ca930 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.921+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f0bb8043070 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.922+0000 7f0bb27fc640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 0x7f0ba006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:27.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.922+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0bb80ab920 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.922+0000 7f0bc17d4640 1 -- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 msgr2=0x7f0ba006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/992174303 2023-10-08T05:50:27.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.922+0000 7f0bc17d4640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 0x7f0ba006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:50:27.925 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:27.925+0000 7f9d2a23b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:50:27.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:27.926+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0bb80766d0 con 0x7f0bbc0b4b40 2023-10-08T05:50:27.967 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:27.966+0000 7ff61d816040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:50:27.988 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:27.987+0000 7f9d2a23b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:50:28.016 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:28.015+0000 7ff61d816040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:50:28.037 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:28.036+0000 7f9d2a23b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:50:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:28.123+0000 7f0bc17d4640 1 -- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 msgr2=0x7f0ba006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/992174303 2023-10-08T05:50:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:28.123+0000 7f0bc17d4640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 0x7f0ba006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:50:28.125 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:28.124+0000 7ff61d816040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:50:28.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:28.137+0000 7f0bc4239640 1 -- 172.21.15.70:0/2093566927 --> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bbc0b79d0 con 0x7f0ba006afe0 2023-10-08T05:50:28.148 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:28.148+0000 7f9d2a23b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:50:28.187 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:28.186+0000 7ff61d816040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:50:28.210 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:28.210+0000 7f9d2a23b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:50:28.236 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:28.235+0000 7ff61d816040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:50:28.260 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:28.260+0000 7f9d2a23b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:50:28.293 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:28.292+0000 7ff61d816040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:50:28.317 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:28.317+0000 7f9d2a23b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:50:28.344 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:28.343+0000 7ff61d816040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:50:28.368 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:28.367+0000 7f9d2a23b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:50:28.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:28.524+0000 7f0bc17d4640 1 -- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 msgr2=0x7f0ba006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/992174303 2023-10-08T05:50:28.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:28.524+0000 7f0bc17d4640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 0x7f0ba006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:50:28.719 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:28.718+0000 7ff61d816040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:50:28.747 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:28.747+0000 7f9d2a23b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:50:28.877 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:28.876+0000 7ff61d816040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:50:28.904 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:28.904+0000 7f9d2a23b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:50:28.983 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:28.982+0000 7ff61d816040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:50:29.010 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:29.010+0000 7f9d2a23b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:50:29.033 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:29.031+0000 7ff61d816040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:50:29.060 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:29.059+0000 7f9d2a23b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:50:29.139 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:29.138+0000 7ff61d816040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:50:29.166 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:29.166+0000 7f9d2a23b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:50:29.247 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:29.246+0000 7ff61d816040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:50:29.272 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:29.272+0000 7f9d2a23b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:50:29.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:29.325+0000 7f0bc17d4640 1 -- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 msgr2=0x7f0ba006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/992174303 2023-10-08T05:50:29.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:29.325+0000 7f0bc17d4640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 0x7f0ba006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:50:29.405 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:29.403+0000 7ff61d816040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:50:29.428 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:29.427+0000 7f9d2a23b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:50:29.454 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:29.453+0000 7ff61d816040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:50:29.476 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:29.476+0000 7f9d2a23b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:50:29.572 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:29.571+0000 7ff61d816040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:50:29.595 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:29.594+0000 7f9d2a23b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:50:30.366 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:30.365+0000 7ff61d816040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:50:30.406 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:30.406+0000 7f9d2a23b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:50:30.658 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:30.657+0000 7ff61d816040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:50:30.701 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:30.700+0000 7f9d2a23b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:50:30.714 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:30.713+0000 7ff61d816040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:50:30.757 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:30.756+0000 7f9d2a23b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:50:30.827 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:30.826+0000 7ff61d816040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:50:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:30.847+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 20) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f0bb80a9cd0 con 0x7f0bbc0b4b40 2023-10-08T05:50:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:30.847+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 msgr2=0x7f0ba006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:50:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:30.847+0000 7f0bb27fc640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/992174303,v1:172.21.15.181:6833/992174303] conn(0x7f0ba006afe0 0x7f0ba006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:30.876 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:50:30.876+0000 7f9d2a23b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:50:31.019 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:31.017+0000 7ff599fbc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:50:31.019 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:31.017+0000 7ff599fbc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:50:31.019 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:31.017+0000 7ff59d002640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:50:31.019 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:31.017+0000 7ff599fbc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:50:31.019 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:31.017+0000 7ff599fbc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:50:31.020 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:31.017+0000 7ff599fbc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:50:31.020 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:31.017+0000 7ff59d002640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:50:31.020 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:31.017+0000 7ff59d002640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:50:31.020 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:31.017+0000 7ff59d002640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:50:31.020 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:50:31.018+0000 7ff59d002640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:50:31.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:31.857+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 21) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f0bb8076aa0 con 0x7f0bbc0b4b40 2023-10-08T05:50:31.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:31.858+0000 7f0bb27fc640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f0ba0035880 0x7f0ba0075440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:31.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:31.858+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 --> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ba0035c60 con 0x7f0ba0035880 2023-10-08T05:50:31.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:31.862+0000 7f0bc17d4640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f0ba0035880 0x7f0ba0075440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:31.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:31.866+0000 7f0bc17d4640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f0ba0035880 0x7f0ba0075440 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc1533a0 tx=0x7f0bac008040 comp rx=0 tx=0).ready entity=mgr.5380 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:32.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.027+0000 7f0bb27fc640 1 -- 172.21.15.70:0/2093566927 <== mgr.5380 v2:172.21.15.181:6832/3803301866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0ba0035c60 con 0x7f0ba0035880 2023-10-08T05:50:32.028 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1 2023-10-08T05:50:32.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.030+0000 7f0bc4239640 1 -- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f0ba0035880 msgr2=0x7f0ba0075440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.030+0000 7f0bc4239640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f0ba0035880 0x7f0ba0075440 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc1533a0 tx=0x7f0bac008040 comp rx=0 tx=0).stop 2023-10-08T05:50:32.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.030+0000 7f0bc4239640 1 -- 172.21.15.70:0/2093566927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bbc0b4b40 msgr2=0x7f0bbc1f2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.030+0000 7f0bc4239640 1 --2- 172.21.15.70:0/2093566927 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bbc0b4b40 0x7f0bbc1f2e60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8000c00 tx=0x7f0bb80046c0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.030+0000 7f0bc27d6640 1 -- 172.21.15.70:0/2093566927 reap_dead start 2023-10-08T05:50:32.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.031+0000 7f0bc4239640 1 -- 172.21.15.70:0/2093566927 shutdown_connections 2023-10-08T05:50:32.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.031+0000 7f0bc4239640 1 -- 172.21.15.70:0/2093566927 >> 172.21.15.70:0/2093566927 conn(0x7f0bbc0718a0 msgr2=0x7f0bbc15b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:32.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.031+0000 7f0bc4239640 1 -- 172.21.15.70:0/2093566927 shutdown_connections 2023-10-08T05:50:32.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.031+0000 7f0bc4239640 1 -- 172.21.15.70:0/2093566927 wait complete. 2023-10-08T05:50:32.051 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T05:50:32.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 2m 2023-10-08T05:50:32.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.258+0000 7efec036c640 1 Processor -- start 2023-10-08T05:50:32.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.258+0000 7efec036c640 1 -- start start 2023-10-08T05:50:32.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.260+0000 7efec036c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efeb8070cc0 0x7efeb80caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.260+0000 7efec036c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb8071170 con 0x7efeb80cb450 2023-10-08T05:50:32.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.260+0000 7efec036c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb806e150 con 0x7efeb8160660 2023-10-08T05:50:32.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.260+0000 7efec036c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb806de70 con 0x7efeb8070cc0 2023-10-08T05:50:32.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.260+0000 7efebe909640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efeb80cb450 0x7efeb80b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33306/0 (socket says 172.21.15.70:33306) 2023-10-08T05:50:32.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.260+0000 7efebe909640 1 -- 172.21.15.70:0/2795934244 learned_addr learned my addr 172.21.15.70:0/2795934244 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:50:32.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.260+0000 7efebd907640 1 --2- 172.21.15.70:0/2795934244 >> v2:172.21.15.181:3300/0 conn(0x7efeb8070cc0 0x7efeb80caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.261+0000 7efebd907640 1 -- 172.21.15.70:0/2795934244 >> v1:172.21.15.181:6789/0 conn(0x7efeb8160660 legacy=0x7efeb80c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.261+0000 7efebd907640 1 -- 172.21.15.70:0/2795934244 >> v1:172.21.15.70:6789/0 conn(0x7efeb80cb450 legacy=0x7efeb80b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.261+0000 7efebd907640 1 -- 172.21.15.70:0/2795934244 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb80c8e30 con 0x7efeb8070cc0 2023-10-08T05:50:32.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.262+0000 7efebd907640 1 --2- 172.21.15.70:0/2795934244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efeb8070cc0 0x7efeb80caf10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efeb806e340 tx=0x7efea802bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0900be21fbbcdcf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:32.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.267+0000 7efebd106640 1 -- 172.21.15.70:0/2795934244 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea802cce0 con 0x7efeb8070cc0 2023-10-08T05:50:32.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.267+0000 7efebd106640 1 -- 172.21.15.70:0/2795934244 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea802ce40 con 0x7efeb8070cc0 2023-10-08T05:50:32.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.267+0000 7efebd106640 1 -- 172.21.15.70:0/2795934244 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8034660 con 0x7efeb8070cc0 2023-10-08T05:50:32.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.268+0000 7efec036c640 1 -- 172.21.15.70:0/2795934244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efeb8070cc0 msgr2=0x7efeb80caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.268+0000 7efec036c640 1 --2- 172.21.15.70:0/2795934244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efeb8070cc0 0x7efeb80caf10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efeb806e340 tx=0x7efea802bf30 comp rx=0 tx=0).stop 2023-10-08T05:50:32.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.268+0000 7efec036c640 1 -- 172.21.15.70:0/2795934244 shutdown_connections 2023-10-08T05:50:32.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.268+0000 7efec036c640 1 --2- 172.21.15.70:0/2795934244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efeb8070cc0 0x7efeb80caf10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.268+0000 7efec036c640 1 -- 172.21.15.70:0/2795934244 >> 172.21.15.70:0/2795934244 conn(0x7efeb806a5d0 msgr2=0x7efeb80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:32.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.268+0000 7efec036c640 1 -- 172.21.15.70:0/2795934244 shutdown_connections 2023-10-08T05:50:32.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.268+0000 7efec036c640 1 -- 172.21.15.70:0/2795934244 wait complete. 2023-10-08T05:50:32.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.270+0000 7efec036c640 1 Processor -- start 2023-10-08T05:50:32.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.270+0000 7efec036c640 1 -- start start 2023-10-08T05:50:32.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.271+0000 7efec036c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efeb8070cc0 0x7efeb81482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.271+0000 7efec036c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efeb80cb450 0x7efeb815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.271+0000 7efec036c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efeb8160660 0x7efeb80c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.271+0000 7efec036c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb815bf00 con 0x7efeb8160660 2023-10-08T05:50:32.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.271+0000 7efec036c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb815c160 con 0x7efeb8070cc0 2023-10-08T05:50:32.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.271+0000 7efec036c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb815c440 con 0x7efeb80cb450 2023-10-08T05:50:32.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.271+0000 7efebe909640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efeb8160660 0x7efeb80c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.271+0000 7efebe909640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efeb8160660 0x7efeb80c4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46632/0 (socket says 172.21.15.70:46632) 2023-10-08T05:50:32.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.271+0000 7efebe909640 1 -- 172.21.15.70:0/3939106691 learned_addr learned my addr 172.21.15.70:0/3939106691 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:32.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.272+0000 7efebd907640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efeb80cb450 0x7efeb815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.272+0000 7efebe108640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efeb8070cc0 0x7efeb81482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.272+0000 7efebe909640 1 -- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efeb80cb450 msgr2=0x7efeb815b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.272+0000 7efebe909640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efeb80cb450 0x7efeb815b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.272+0000 7efebe909640 1 -- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efeb8070cc0 msgr2=0x7efeb81482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.272+0000 7efebe909640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efeb8070cc0 0x7efeb81482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.272+0000 7efebe909640 1 -- 172.21.15.70:0/3939106691 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea8009a70 con 0x7efeb8160660 2023-10-08T05:50:32.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.272+0000 7efebe909640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efeb8160660 0x7efeb80c4ec0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7efeb000c420 tx=0x7efeb000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:32.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.272+0000 7efeaeffd640 1 -- 172.21.15.70:0/3939106691 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0005ce0 con 0x7efeb8160660 2023-10-08T05:50:32.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.273+0000 7efeaeffd640 1 -- 172.21.15.70:0/3939106691 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb0005e40 con 0x7efeb8160660 2023-10-08T05:50:32.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.273+0000 7efec036c640 1 -- 172.21.15.70:0/3939106691 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb815c720 con 0x7efeb8160660 2023-10-08T05:50:32.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.273+0000 7efeaeffd640 1 -- 172.21.15.70:0/3939106691 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00072d0 con 0x7efeb8160660 2023-10-08T05:50:32.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.273+0000 7efec036c640 1 -- 172.21.15.70:0/3939106691 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb815cce0 con 0x7efeb8160660 2023-10-08T05:50:32.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.274+0000 7efec036c640 1 -- 172.21.15.70:0/3939106691 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb80667d0 con 0x7efeb8160660 2023-10-08T05:50:32.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.275+0000 7efeaeffd640 1 -- 172.21.15.70:0/3939106691 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 89084+0+0 (secure 0 0 0) 0x7efeb00074f0 con 0x7efeb8160660 2023-10-08T05:50:32.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.276+0000 7efeaeffd640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7efe9c06ad10 0x7efe9c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.276+0000 7efeaeffd640 1 -- 172.21.15.70:0/3939106691 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6417+0+0 (secure 0 0 0) 0x7efeb0088890 con 0x7efeb8160660 2023-10-08T05:50:32.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.277+0000 7efebe108640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7efe9c06ad10 0x7efe9c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.278+0000 7efebe108640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7efe9c06ad10 0x7efe9c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efeb4000f60 tx=0x7efeb4009040 comp rx=0 tx=0).ready entity=mgr.5380 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:32.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.281+0000 7efeaeffd640 1 -- 172.21.15.70:0/3939106691 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efeb0053a60 con 0x7efeb8160660 2023-10-08T05:50:32.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.493+0000 7efec036c640 1 -- 172.21.15.70:0/3939106691 --> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "2m", "target": ["mon-mgr", ""]}) v1 -- 0x7efeb80c93f0 con 0x7efe9c06ad10 2023-10-08T05:50:32.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.638+0000 7efeaeffd640 1 -- 172.21.15.70:0/3939106691 <== mgr.5380 v2:172.21.15.181:6832/3803301866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efeb4001360 con 0x7efe9c06ad10 2023-10-08T05:50:32.639 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1 2023-10-08T05:50:32.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.642+0000 7efec036c640 1 -- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7efe9c06ad10 msgr2=0x7efe9c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7efe9c06ad10 0x7efe9c06d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efeb4000f60 tx=0x7efeb4009040 comp rx=0 tx=0).stop 2023-10-08T05:50:32.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 -- 172.21.15.70:0/3939106691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efeb8160660 msgr2=0x7efeb80c4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efeb8160660 0x7efeb80c4ec0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7efeb000c420 tx=0x7efeb000c8e0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 -- 172.21.15.70:0/3939106691 shutdown_connections 2023-10-08T05:50:32.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7efe9c06ad10 0x7efe9c06d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efeb8160660 0x7efeb80c4ec0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efeb80cb450 0x7efeb815b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 --2- 172.21.15.70:0/3939106691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efeb8070cc0 0x7efeb81482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 -- 172.21.15.70:0/3939106691 >> 172.21.15.70:0/3939106691 conn(0x7efeb806a5d0 msgr2=0x7efeb80c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:32.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 -- 172.21.15.70:0/3939106691 shutdown_connections 2023-10-08T05:50:32.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.643+0000 7efec036c640 1 -- 172.21.15.70:0/3939106691 wait complete. 2023-10-08T05:50:32.664 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T05:50:32.664 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-08-05_51_00 in ~28s... 2023-10-08T05:50:32.664 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-08-05_52_00 in ~88s... 2023-10-08T05:50:32.664 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m', '2m'], expected_retention: [] 2023-10-08T05:50:32.665 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-08T05:50:32.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.882+0000 7fcb481b6640 1 Processor -- start 2023-10-08T05:50:32.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.883+0000 7fcb481b6640 1 -- start start 2023-10-08T05:50:32.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.884+0000 7fcb481b6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb400afbe0 0x7fcb400b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.884+0000 7fcb481b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb4006dfc0 0x7fcb40070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.884+0000 7fcb481b6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb400b2620 con 0x7fcb40071070 2023-10-08T05:50:32.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.884+0000 7fcb481b6640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb400b2060 con 0x7fcb400afbe0 2023-10-08T05:50:32.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.884+0000 7fcb481b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb400b2340 con 0x7fcb4006dfc0 2023-10-08T05:50:32.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.885+0000 7fcb45751640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb4006dfc0 0x7fcb40070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.885+0000 7fcb45f52640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcb400afbe0 0x7fcb400b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.885+0000 7fcb45751640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb4006dfc0 0x7fcb40070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:37024/0 (socket says 172.21.15.70:37024) 2023-10-08T05:50:32.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.885+0000 7fcb45751640 1 -- 172.21.15.70:0/3371272248 learned_addr learned my addr 172.21.15.70:0/3371272248 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.886+0000 7fcb45f52640 1 -- 172.21.15.70:0/3371272248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb4006dfc0 msgr2=0x7fcb40070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:50:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.886+0000 7fcb45f52640 1 --2- 172.21.15.70:0/3371272248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb4006dfc0 0x7fcb40070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.886+0000 7fcb45f52640 1 -- 172.21.15.70:0/3371272248 >> v1:172.21.15.181:6789/0 conn(0x7fcb40071070 legacy=0x7fcb400cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.886+0000 7fcb45f52640 1 -- 172.21.15.70:0/3371272248 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40070350 con 0x7fcb400afbe0 2023-10-08T05:50:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.886+0000 7fcb45751640 1 --2- 172.21.15.70:0/3371272248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb4006dfc0 0x7fcb40070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:50:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.886+0000 7fcb44f50640 1 -- 172.21.15.70:0/3371272248 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040244957 0 0) 0x7fcb400b2620 con 0x7fcb40071070 2023-10-08T05:50:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.887+0000 7fcb45f52640 1 --2- 172.21.15.70:0/3371272248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb400afbe0 0x7fcb400b15b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0097a0 tx=0x7fcb3c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec50f1f3bb3ded2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.888+0000 7fcb44f50640 1 -- 172.21.15.70:0/3371272248 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c02d040 con 0x7fcb400afbe0 2023-10-08T05:50:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.888+0000 7fcb44f50640 1 -- 172.21.15.70:0/3371272248 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3c031810 con 0x7fcb400afbe0 2023-10-08T05:50:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.888+0000 7fcb44f50640 1 -- 172.21.15.70:0/3371272248 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c031ba0 con 0x7fcb400afbe0 2023-10-08T05:50:32.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.889+0000 7fcb481b6640 1 -- 172.21.15.70:0/3371272248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb400afbe0 msgr2=0x7fcb400b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.889+0000 7fcb481b6640 1 --2- 172.21.15.70:0/3371272248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb400afbe0 0x7fcb400b15b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c0097a0 tx=0x7fcb3c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.889+0000 7fcb481b6640 1 -- 172.21.15.70:0/3371272248 shutdown_connections 2023-10-08T05:50:32.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.889+0000 7fcb481b6640 1 --2- 172.21.15.70:0/3371272248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb4006dfc0 0x7fcb40070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.889+0000 7fcb481b6640 1 --2- 172.21.15.70:0/3371272248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb400afbe0 0x7fcb400b15b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.889+0000 7fcb481b6640 1 -- 172.21.15.70:0/3371272248 >> 172.21.15.70:0/3371272248 conn(0x7fcb4006a5d0 msgr2=0x7fcb400b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:32.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.889+0000 7fcb481b6640 1 -- 172.21.15.70:0/3371272248 shutdown_connections 2023-10-08T05:50:32.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.889+0000 7fcb481b6640 1 -- 172.21.15.70:0/3371272248 wait complete. 2023-10-08T05:50:32.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.891+0000 7fcb481b6640 1 Processor -- start 2023-10-08T05:50:32.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.891+0000 7fcb481b6640 1 -- start start 2023-10-08T05:50:32.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb481b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb4006dfc0 0x7fcb4014aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb481b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40071070 0x7fcb400c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb481b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb400afbe0 0x7fcb400c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb481b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4015c6f0 con 0x7fcb400afbe0 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb481b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4015b650 con 0x7fcb4006dfc0 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb481b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4015b930 con 0x7fcb40071070 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb46753640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb400afbe0 0x7fcb400c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb46753640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb400afbe0 0x7fcb400c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46638/0 (socket says 172.21.15.70:46638) 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb46753640 1 -- 172.21.15.70:0/1387281576 learned_addr learned my addr 172.21.15.70:0/1387281576 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb45f52640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb4006dfc0 0x7fcb4014aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb45751640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40071070 0x7fcb400c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb46753640 1 -- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40071070 msgr2=0x7fcb400c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb46753640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40071070 0x7fcb400c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb46753640 1 -- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb4006dfc0 msgr2=0x7fcb4014aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:32.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb46753640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb4006dfc0 0x7fcb4014aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:32.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.892+0000 7fcb46753640 1 -- 172.21.15.70:0/1387281576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb300094d0 con 0x7fcb400afbe0 2023-10-08T05:50:32.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.893+0000 7fcb45751640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40071070 0x7fcb400c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:50:32.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.893+0000 7fcb46753640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb400afbe0 0x7fcb400c99d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800c710 tx=0x7fcb3800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:32.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.893+0000 7fcb367fc640 1 -- 172.21.15.70:0/1387281576 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38003da0 con 0x7fcb400afbe0 2023-10-08T05:50:32.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.893+0000 7fcb367fc640 1 -- 172.21.15.70:0/1387281576 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3800ce20 con 0x7fcb400afbe0 2023-10-08T05:50:32.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.893+0000 7fcb481b6640 1 -- 172.21.15.70:0/1387281576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c009400 con 0x7fcb400afbe0 2023-10-08T05:50:32.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.893+0000 7fcb367fc640 1 -- 172.21.15.70:0/1387281576 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38005290 con 0x7fcb400afbe0 2023-10-08T05:50:32.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.893+0000 7fcb481b6640 1 -- 172.21.15.70:0/1387281576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4015bef0 con 0x7fcb400afbe0 2023-10-08T05:50:32.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.895+0000 7fcb481b6640 1 -- 172.21.15.70:0/1387281576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4004e160 con 0x7fcb400afbe0 2023-10-08T05:50:32.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.895+0000 7fcb367fc640 1 -- 172.21.15.70:0/1387281576 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 21) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fcb380053f0 con 0x7fcb400afbe0 2023-10-08T05:50:32.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.896+0000 7fcb367fc640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fcb2406ad40 0x7fcb2406d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:50:32.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.896+0000 7fcb367fc640 1 -- 172.21.15.70:0/1387281576 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fcb38088dc0 con 0x7fcb400afbe0 2023-10-08T05:50:32.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.897+0000 7fcb45f52640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fcb2406ad40 0x7fcb2406d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:50:32.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.899+0000 7fcb45f52640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fcb2406ad40 0x7fcb2406d200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c036000 tx=0x7fcb3c03c040 comp rx=0 tx=0).ready entity=mgr.5380 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:50:32.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:32.901+0000 7fcb367fc640 1 -- 172.21.15.70:0/1387281576 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb38053e60 con 0x7fcb400afbe0 2023-10-08T05:50:33.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.114+0000 7fcb481b6640 1 -- 172.21.15.70:0/1387281576 --> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb400affc0 con 0x7fcb2406ad40 2023-10-08T05:50:33.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.130+0000 7fcb367fc640 1 -- 172.21.15.70:0/1387281576 <== mgr.5380 v2:172.21.15.181:6832/3803301866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7fcb3c031b00 con 0x7fcb2406ad40 2023-10-08T05:50:33.131 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:50:33.131 INFO:teuthology.orchestra.run.smithi070.stdout:{"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2023-10-08T05:50:33.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 -- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fcb2406ad40 msgr2=0x7fcb2406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:33.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fcb2406ad40 0x7fcb2406d200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c036000 tx=0x7fcb3c03c040 comp rx=0 tx=0).stop 2023-10-08T05:50:33.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 -- 172.21.15.70:0/1387281576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb400afbe0 msgr2=0x7fcb400c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:50:33.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb400afbe0 0x7fcb400c99d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800c710 tx=0x7fcb3800cbd0 comp rx=0 tx=0).stop 2023-10-08T05:50:33.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 -- 172.21.15.70:0/1387281576 shutdown_connections 2023-10-08T05:50:33.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fcb2406ad40 0x7fcb2406d200 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:33.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb400afbe0 0x7fcb400c99d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:33.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb40071070 0x7fcb400c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:33.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 --2- 172.21.15.70:0/1387281576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb4006dfc0 0x7fcb4014aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:50:33.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 -- 172.21.15.70:0/1387281576 >> 172.21.15.70:0/1387281576 conn(0x7fcb4006a5d0 msgr2=0x7fcb400ca720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:50:33.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.133+0000 7fcb481b6640 1 -- 172.21.15.70:0/1387281576 shutdown_connections 2023-10-08T05:50:33.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:50:33.134+0000 7fcb481b6640 1 -- 172.21.15.70:0/1387281576 wait complete. 2023-10-08T05:50:33.153 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m", "2m"], "retention": [{}, {}]} 2023-10-08T05:50:33.153 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m', '2m'], 'retention': [{}, {}]} 2023-10-08T05:50:33.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:33.154 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:33.206 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:33.207 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:33.207 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:34.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:34.207 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:34.272 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:34.272 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:34.272 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:35.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:35.273 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:35.334 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:35.335 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:35.335 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:36.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:36.336 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:36.400 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:36.400 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:36.400 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:37.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:37.402 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:37.466 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:37.466 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:37.466 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:38.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:38.468 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:38.531 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:38.532 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:38.532 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:39.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:50:39.140 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:50:39.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:39.533 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:39.595 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:39.595 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:39.595 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:40.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:40.596 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:40.660 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:40.660 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:40.660 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:41.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:41.662 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:41.725 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:41.725 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:41.725 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:42.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:42.726 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:42.790 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:42.790 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:42.790 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:43.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:43.791 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:43.855 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:43.855 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:43.855 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:44.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:44.856 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:44.919 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:44.919 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:44.919 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:45.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:45.920 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:45.983 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:45.983 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:45.983 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:46.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:46.984 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:47.050 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:47.050 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:47.050 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:48.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:48.052 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:48.115 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:48.115 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:48.115 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:49.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:49.116 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:49.177 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:49.177 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:49.177 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:50.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:50.179 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:50.241 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:50.241 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:50.242 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:51.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:51.243 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:51.306 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:51.306 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:51.306 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:52.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:52.308 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:52.374 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:52.374 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:52.374 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:53.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:53.375 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:53.438 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:53.439 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:53.439 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:54.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:54.440 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:54.504 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:54.504 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:54.504 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:55.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:55.505 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:55.568 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:55.568 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:55.568 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:56.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:56.569 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:56.631 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:56.631 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:56.631 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:57.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:57.632 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:57.696 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:57.696 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:57.696 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:58.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:58.697 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:58.760 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:58.760 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:58.760 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:50:59.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:50:59.762 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:50:59.827 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:50:59.827 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:50:59.827 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:00.663 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:51:00.662+0000 7ff58bfa0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:51:00.663 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:51:00.662+0000 7ff58bfa0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:51:00.663 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:51:00.662+0000 7ff58bfa0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:51:00.663 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:51:00.662+0000 7ff58bfa0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:51:00.663 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:51:00.662+0000 7ff58bfa0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:51:00.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:00.828 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:00.888 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:00.889 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:00.889 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:00.889 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:00.889 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-08-05_51_00_UTC'] 2023-10-08T05:51:00.889 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696744232.6645942, now = 1696744260.8897564, timo = 28 2023-10-08T05:51:01.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:01.890 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:01.951 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:01.952 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:01.953 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:01.953 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:02.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:02.954 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:03.014 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:03.016 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:03.016 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:03.016 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:04.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:04.018 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:04.081 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:04.083 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:04.083 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:04.083 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:05.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:05.084 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:05.147 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:05.149 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:05.149 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:05.149 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:06.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:06.150 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:06.211 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:06.212 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:06.212 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:06.212 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:07.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:07.213 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:07.275 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:07.276 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:07.276 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:07.276 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:08.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:08.277 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:08.341 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:08.342 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:08.342 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:08.343 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:09.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:51:09.294 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:51:09.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:09.344 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:09.392 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:09.393 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:09.393 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:09.393 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:10.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:10.394 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:10.456 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:10.457 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:10.457 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:10.457 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:11.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:11.458 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:11.520 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:11.521 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:11.521 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:11.521 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:12.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:12.522 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:12.585 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:12.586 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:12.586 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:12.586 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:13.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:13.587 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:13.650 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:13.651 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:13.651 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:13.651 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:14.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:14.653 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:14.715 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:14.716 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:14.717 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:14.717 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:15.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:15.718 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:15.781 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:15.783 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:15.783 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:15.783 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:16.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:16.784 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:16.847 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:16.849 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:16.849 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:16.849 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:17.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:17.850 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:17.911 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:17.913 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:17.913 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:17.913 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:18.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:18.914 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:18.975 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:18.977 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:18.977 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:18.977 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:19.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:19.978 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:20.039 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:20.040 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:20.040 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:20.041 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:21.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:21.042 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:21.104 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:21.105 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:21.105 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:21.105 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:22.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:22.107 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:22.168 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:22.169 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:22.169 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:22.169 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:23.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:23.171 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:23.232 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:23.233 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:23.233 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:23.233 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:24.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:24.235 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:24.298 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:24.299 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:24.299 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:24.299 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:25.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:25.301 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:25.362 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:25.363 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:25.364 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:25.364 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:26.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:26.365 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:26.427 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:26.429 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:26.429 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:26.429 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:27.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:27.430 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:27.491 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:27.492 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:27.492 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:27.492 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:28.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:28.493 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:28.555 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:28.556 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:28.556 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:28.557 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:29.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:29.557 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:29.621 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:29.623 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:29.623 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:29.623 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:30.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:30.624 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:30.686 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:30.687 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:30.687 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:30.688 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:31.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:31.689 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:31.752 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:31.754 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:31.754 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:31.754 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:32.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:32.760 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:32.823 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:32.824 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:32.824 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:32.824 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:33.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:33.826 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:33.888 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:33.890 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:33.890 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:33.890 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:34.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:34.891 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:34.950 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:34.951 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:34.951 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:34.951 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:35.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:35.953 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:36.016 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:36.017 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:36.017 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:36.017 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:37.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:37.018 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:37.079 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:37.080 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:37.080 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:37.080 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:38.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:38.081 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:38.143 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:38.144 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:38.144 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:38.144 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:39.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:39.145 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:39.207 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:39.208 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:39.208 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:39.208 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:39.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:51:39.442 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:51:40.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:40.210 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:40.271 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:40.272 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:40.273 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:40.273 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:41.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:41.274 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:41.336 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:41.338 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:41.338 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:41.338 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:42.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:42.339 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:42.404 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:42.406 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:42.406 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:42.406 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:43.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:43.407 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:43.471 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:43.472 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:43.472 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:43.473 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:44.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:44.474 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:44.535 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:44.536 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:44.536 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:44.537 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:45.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:45.537 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:45.601 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:45.602 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:45.602 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:45.603 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:46.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:46.604 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:46.666 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:46.667 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:46.667 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:46.667 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:47.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:47.669 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:47.729 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:47.731 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:47.731 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:47.731 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:48.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:48.732 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:48.790 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:48.791 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:48.791 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:48.791 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:49.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:49.792 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:49.855 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:49.856 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:49.856 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:49.856 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:50.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:50.857 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:50.919 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:50.920 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:50.921 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:50.921 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:51.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:51.921 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:51.985 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:51.986 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:51.987 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:51.987 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:52.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:52.988 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:53.050 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:53.052 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:53.052 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:53.052 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:54.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:54.053 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:54.117 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:54.118 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:54.118 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:54.118 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:55.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:55.119 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:55.182 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:55.184 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:55.184 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:55.184 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:56.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:56.185 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:56.246 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:56.248 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:56.248 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:56.248 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:57.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:57.249 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:57.311 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:57.312 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:57.312 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:57.313 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:58.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:58.313 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:58.375 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:58.376 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:58.377 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:58.377 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:51:59.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:51:59.378 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:51:59.439 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:51:59.440 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:51:59.440 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:51:59.440 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:52:00.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:00.442 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:52:00.502 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:52:00.503 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC'} 2023-10-08T05:52:00.504 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:52:00.504 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:52:01.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:01.504 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:52:01.569 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:52:01.569 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_52_00_UTC 2023-10-08T05:52:01.570 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_51_00_UTC', 'scheduled-2023-10-08-05_52_00_UTC'} 2023-10-08T05:52:01.570 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-08-05_52_00_UTC'} 2023-10-08T05:52:01.570 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:52:01.570 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-08-05_52_00_UTC'] 2023-10-08T05:52:01.570 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696744232.6645942, now = 1696744321.570758, timo = 88 2023-10-08T05:52:02.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-08T05:52:02.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.796+0000 7f974cf73640 1 Processor -- start 2023-10-08T05:52:02.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.797+0000 7f974cf73640 1 -- start start 2023-10-08T05:52:02.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.798+0000 7f974cf73640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f97480c9cf0 0x7f974806db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:02.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.798+0000 7f974cf73640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974815aa00 0x7f97480b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:02.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.798+0000 7f974cf73640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974815b310 con 0x7f974806c3a0 2023-10-08T05:52:02.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.798+0000 7f974cf73640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974806e0a0 con 0x7f97480c9cf0 2023-10-08T05:52:02.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.798+0000 7f974cf73640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974806d290 con 0x7f974815aa00 2023-10-08T05:52:02.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.798+0000 7f974659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f974806c3a0 0x7f974806c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57500/0 (socket says 172.21.15.70:57500) 2023-10-08T05:52:02.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.798+0000 7f974659c640 1 -- 172.21.15.70:0/4032775070 learned_addr learned my addr 172.21.15.70:0/4032775070 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f9745d9b640 1 --2- 172.21.15.70:0/4032775070 >> v2:172.21.15.181:3300/0 conn(0x7f97480c9cf0 0x7f974806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f9746d9d640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974815aa00 0x7f97480b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917257467 0 0) 0x7f9738001490 con 0x7f974806c3a0 2023-10-08T05:52:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9734003520 con 0x7f974806c3a0 2023-10-08T05:52:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3297152245 0 0) 0x7f9734003520 con 0x7f974806c3a0 2023-10-08T05:52:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974815aa00 msgr2=0x7f97480b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974815aa00 0x7f97480b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 >> v2:172.21.15.181:3300/0 conn(0x7f97480c9cf0 msgr2=0x7f974806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 --2- 172.21.15.70:0/4032775070 >> v2:172.21.15.181:3300/0 conn(0x7f97480c9cf0 0x7f974806db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480c9860 con 0x7f974806c3a0 2023-10-08T05:52:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9738001b80 con 0x7f974806c3a0 2023-10-08T05:52:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 >> v1:172.21.15.70:6789/0 conn(0x7f974806c3a0 legacy=0x7f974806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.799+0000 7f974559a640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9734003970 0x7f9734003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f974559a640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974815aa00 0x7f97340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f974559a640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9734004d00 0x7f9734005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974806e0a0 con 0x7f9734004d00 2023-10-08T05:52:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974806d290 con 0x7f9734003970 2023-10-08T05:52:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973400b980 con 0x7f974815aa00 2023-10-08T05:52:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f9746d9d640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9734004d00 0x7f9734005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f974659c640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9734003970 0x7f9734003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f9746d9d640 1 -- 172.21.15.70:0/4032775070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974815aa00 msgr2=0x7f97340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f9746d9d640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974815aa00 0x7f97340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f9746d9d640 1 -- 172.21.15.70:0/4032775070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9734003970 msgr2=0x7f9734003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f9746d9d640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9734003970 0x7f9734003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.800+0000 7f9746d9d640 1 -- 172.21.15.70:0/4032775070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734003520 con 0x7f9734004d00 2023-10-08T05:52:02.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.801+0000 7f9746d9d640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9734004d00 0x7f9734005180 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9740009af0 tx=0x7f97340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d8d69e12843d0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:02.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.802+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974002c710 con 0x7f9734004d00 2023-10-08T05:52:02.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.802+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974002c870 con 0x7f9734004d00 2023-10-08T05:52:02.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.802+0000 7f974559a640 1 -- 172.21.15.70:0/4032775070 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974002cc00 con 0x7f9734004d00 2023-10-08T05:52:02.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.802+0000 7f974cf73640 1 -- 172.21.15.70:0/4032775070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9734004d00 msgr2=0x7f9734005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:02.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.802+0000 7f974cf73640 1 --2- 172.21.15.70:0/4032775070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9734004d00 0x7f9734005180 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9740009af0 tx=0x7f97340057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:02.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.802+0000 7f9746d9d640 1 -- 172.21.15.70:0/4032775070 reap_dead start 2023-10-08T05:52:02.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.802+0000 7f974cf73640 1 -- 172.21.15.70:0/4032775070 shutdown_connections 2023-10-08T05:52:02.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.802+0000 7f974cf73640 1 -- 172.21.15.70:0/4032775070 >> 172.21.15.70:0/4032775070 conn(0x7f974806a5d0 msgr2=0x7f97480b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:02.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.802+0000 7f974cf73640 1 -- 172.21.15.70:0/4032775070 shutdown_connections 2023-10-08T05:52:02.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.802+0000 7f974cf73640 1 -- 172.21.15.70:0/4032775070 wait complete. 2023-10-08T05:52:02.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.804+0000 7f974cf73640 1 Processor -- start 2023-10-08T05:52:02.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.804+0000 7f974cf73640 1 -- start start 2023-10-08T05:52:02.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.805+0000 7f974cf73640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9734004d00 0x7f97480c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:02.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.805+0000 7f974cf73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97480c6940 0x7f97480c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:02.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.805+0000 7f974cf73640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97480aee50 0x7f97480ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:02.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.805+0000 7f974cf73640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97480c7af0 con 0x7f97480c6940 2023-10-08T05:52:02.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.805+0000 7f974cf73640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97480c8450 con 0x7f9734004d00 2023-10-08T05:52:02.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.805+0000 7f974cf73640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97480c6e70 con 0x7f97480aee50 2023-10-08T05:52:02.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f9745d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97480c6940 0x7f97480c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:02.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f9745d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97480c6940 0x7f97480c4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44284/0 (socket says 172.21.15.70:44284) 2023-10-08T05:52:02.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f9745d9b640 1 -- 172.21.15.70:0/86294076 learned_addr learned my addr 172.21.15.70:0/86294076 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:02.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f974659c640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9734004d00 0x7f97480c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:02.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f9746d9d640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97480aee50 0x7f97480ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:02.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f9745d9b640 1 -- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97480aee50 msgr2=0x7f97480ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:02.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f9745d9b640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97480aee50 0x7f97480ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:02.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f9745d9b640 1 -- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9734004d00 msgr2=0x7f97480c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:02.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f9745d9b640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9734004d00 0x7f97480c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:02.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f9745d9b640 1 -- 172.21.15.70:0/86294076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480c7150 con 0x7f97480c6940 2023-10-08T05:52:02.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.806+0000 7f9745d9b640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97480c6940 0x7f97480c4b50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f973000e7c0 tx=0x7f973000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:02.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.807+0000 7f973effd640 1 -- 172.21.15.70:0/86294076 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973000eed0 con 0x7f97480c6940 2023-10-08T05:52:02.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.807+0000 7f973effd640 1 -- 172.21.15.70:0/86294076 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973000c940 con 0x7f97480c6940 2023-10-08T05:52:02.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.807+0000 7f973effd640 1 -- 172.21.15.70:0/86294076 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973000ccf0 con 0x7f97480c6940 2023-10-08T05:52:02.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.807+0000 7f974cf73640 1 -- 172.21.15.70:0/86294076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97480c7430 con 0x7f97480c6940 2023-10-08T05:52:02.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.807+0000 7f974cf73640 1 -- 172.21.15.70:0/86294076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97480c5090 con 0x7f97480c6940 2023-10-08T05:52:02.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.808+0000 7f974cf73640 1 -- 172.21.15.70:0/86294076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97480c7c30 con 0x7f97480c6940 2023-10-08T05:52:02.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.809+0000 7f973effd640 1 -- 172.21.15.70:0/86294076 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f9730013470 con 0x7f97480c6940 2023-10-08T05:52:02.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.810+0000 7f973effd640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f972806b100 0x7f972806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:02.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.810+0000 7f973effd640 1 -- 172.21.15.70:0/86294076 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f9730089fc0 con 0x7f97480c6940 2023-10-08T05:52:02.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.810+0000 7f974659c640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f972806b100 0x7f972806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:02.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.812+0000 7f974659c640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f972806b100 0x7f972806d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9738005e70 tx=0x7f9738009040 comp rx=0 tx=0).ready entity=mgr.5380 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:02.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:02.815+0000 7f973effd640 1 -- 172.21.15.70:0/86294076 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97480c7c30 con 0x7f97480c6940 2023-10-08T05:52:03.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.028+0000 7f974cf73640 1 -- 172.21.15.70:0/86294076 --> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f97480c8f30 con 0x7f972806b100 2023-10-08T05:52:03.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.188+0000 7f973effd640 1 -- 172.21.15.70:0/86294076 <== mgr.5380 v2:172.21.15.181:6832/3803301866 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9738004c30 con 0x7f972806b100 2023-10-08T05:52:03.190 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path snap_test_dir1 2023-10-08T05:52:03.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 -- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f972806b100 msgr2=0x7f972806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:03.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f972806b100 0x7f972806d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9738005e70 tx=0x7f9738009040 comp rx=0 tx=0).stop 2023-10-08T05:52:03.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 -- 172.21.15.70:0/86294076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97480c6940 msgr2=0x7f97480c4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:03.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97480c6940 0x7f97480c4b50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f973000e7c0 tx=0x7f973000ec80 comp rx=0 tx=0).stop 2023-10-08T05:52:03.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 -- 172.21.15.70:0/86294076 shutdown_connections 2023-10-08T05:52:03.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f972806b100 0x7f972806d5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97480aee50 0x7f97480ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97480c6940 0x7f97480c4b50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 --2- 172.21.15.70:0/86294076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9734004d00 0x7f97480c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 -- 172.21.15.70:0/86294076 >> 172.21.15.70:0/86294076 conn(0x7f974806a5d0 msgr2=0x7f9748151060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:03.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 -- 172.21.15.70:0/86294076 shutdown_connections 2023-10-08T05:52:03.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.192+0000 7f974cf73640 1 -- 172.21.15.70:0/86294076 wait complete. 2023-10-08T05:52:03.213 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-08T05:52:03.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:03.214 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:52:03.265 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:52:03.265 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_52_00_UTC 2023-10-08T05:52:03.266 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-05_51_00_UTC 2023-10-08T05:52:03.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:03.266 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-05_51_00_UTC) 2023-10-08T05:52:03.321 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-05_52_00_UTC 2023-10-08T05:52:03.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:03.321 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-05_52_00_UTC) 2023-10-08T05:52:03.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:03.378 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T05:52:03.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T05:52:03.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.639+0000 7f1f1f353640 1 Processor -- start 2023-10-08T05:52:03.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.639+0000 7f1f1f353640 1 -- start start 2023-10-08T05:52:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.640+0000 7f1f1f353640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f1815aad0 0x7f1f180b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.640+0000 7f1f1f353640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f180c82b0 0x7f1f180c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1f353640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f1806cd00 0x7f1f1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1f353640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1806d6d0 con 0x7f1f1815aad0 2023-10-08T05:52:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1f353640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1815bf30 con 0x7f1f1806cd00 2023-10-08T05:52:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1f353640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1815b320 con 0x7f1f180c82b0 2023-10-08T05:52:03.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1d0ef640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f1815aad0 0x7f1f180b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:03.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1d0ef640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f1815aad0 0x7f1f180b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43042/0 (socket says 172.21.15.70:43042) 2023-10-08T05:52:03.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1d0ef640 1 -- 172.21.15.70:0/1696990149 learned_addr learned my addr 172.21.15.70:0/1696990149 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:03.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1d0ef640 1 -- 172.21.15.70:0/1696990149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f180c82b0 msgr2=0x7f1f180c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1c8ee640 1 --2- 172.21.15.70:0/1696990149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f180c82b0 0x7f1f180c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1d8f0640 1 --2- 172.21.15.70:0/1696990149 >> v2:172.21.15.181:3300/0 conn(0x7f1f1806cd00 0x7f1f1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1d0ef640 1 --2- 172.21.15.70:0/1696990149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f180c82b0 0x7f1f180c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1d0ef640 1 -- 172.21.15.70:0/1696990149 >> v2:172.21.15.181:3300/0 conn(0x7f1f1806cd00 msgr2=0x7f1f1806d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1d0ef640 1 --2- 172.21.15.70:0/1696990149 >> v2:172.21.15.181:3300/0 conn(0x7f1f1806cd00 0x7f1f1806d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.641+0000 7f1f1d0ef640 1 -- 172.21.15.70:0/1696990149 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1815b600 con 0x7f1f1815aad0 2023-10-08T05:52:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.643+0000 7f1f1d0ef640 1 --2- 172.21.15.70:0/1696990149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 0x7f1f180b1750 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1f180b0c20 tx=0x7f1f1402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b0440c17c8869d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.643+0000 7f1f0ffff640 1 -- 172.21.15.70:0/1696990149 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1402ca70 con 0x7f1f1815aad0 2023-10-08T05:52:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.643+0000 7f1f0ffff640 1 -- 172.21.15.70:0/1696990149 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1402cbd0 con 0x7f1f1815aad0 2023-10-08T05:52:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.643+0000 7f1f0ffff640 1 -- 172.21.15.70:0/1696990149 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140343f0 con 0x7f1f1815aad0 2023-10-08T05:52:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.644+0000 7f1f1f353640 1 -- 172.21.15.70:0/1696990149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 msgr2=0x7f1f180b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.644+0000 7f1f1f353640 1 --2- 172.21.15.70:0/1696990149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 0x7f1f180b1750 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1f180b0c20 tx=0x7f1f1402bed0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.644+0000 7f1f1f353640 1 -- 172.21.15.70:0/1696990149 shutdown_connections 2023-10-08T05:52:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.644+0000 7f1f1f353640 1 --2- 172.21.15.70:0/1696990149 >> v2:172.21.15.181:3300/0 conn(0x7f1f1806cd00 0x7f1f1806d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.644+0000 7f1f1f353640 1 --2- 172.21.15.70:0/1696990149 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f180c82b0 0x7f1f180c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.644+0000 7f1f1f353640 1 --2- 172.21.15.70:0/1696990149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 0x7f1f180b1750 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.644+0000 7f1f1f353640 1 -- 172.21.15.70:0/1696990149 >> 172.21.15.70:0/1696990149 conn(0x7f1f18069d50 msgr2=0x7f1f1806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.644+0000 7f1f1f353640 1 -- 172.21.15.70:0/1696990149 shutdown_connections 2023-10-08T05:52:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.644+0000 7f1f1f353640 1 -- 172.21.15.70:0/1696990149 wait complete. 2023-10-08T05:52:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.646+0000 7f1f1f353640 1 Processor -- start 2023-10-08T05:52:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.646+0000 7f1f1f353640 1 -- start start 2023-10-08T05:52:03.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1f353640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f1806cd00 0x7f1f181432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:03.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1f353640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180c82b0 0x7f1f181507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1f353640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 0x7f1f180c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1f353640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18158f60 con 0x7f1f1815aad0 2023-10-08T05:52:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1f353640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18159240 con 0x7f1f180c82b0 2023-10-08T05:52:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1f353640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18159520 con 0x7f1f1806cd00 2023-10-08T05:52:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1d8f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 0x7f1f180c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1d8f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 0x7f1f180c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43050/0 (socket says 172.21.15.70:43050) 2023-10-08T05:52:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1d8f0640 1 -- 172.21.15.70:0/1639196839 learned_addr learned my addr 172.21.15.70:0/1639196839 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1d8f0640 1 -- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f1806cd00 msgr2=0x7f1f181432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1d0ef640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f1806cd00 0x7f1f181432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1d8f0640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f1806cd00 0x7f1f181432f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1d8f0640 1 -- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180c82b0 msgr2=0x7f1f181507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:03.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1d8f0640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180c82b0 0x7f1f181507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:03.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.647+0000 7f1f1d8f0640 1 -- 172.21.15.70:0/1639196839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f14009b30 con 0x7f1f1815aad0 2023-10-08T05:52:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.648+0000 7f1f1d8f0640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 0x7f1f180c9e20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c570 tx=0x7f1f1000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.648+0000 7f1f0dffb640 1 -- 172.21.15.70:0/1639196839 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10004220 con 0x7f1f1815aad0 2023-10-08T05:52:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.648+0000 7f1f0dffb640 1 -- 172.21.15.70:0/1639196839 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f10004380 con 0x7f1f1815aad0 2023-10-08T05:52:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.648+0000 7f1f0dffb640 1 -- 172.21.15.70:0/1639196839 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10003830 con 0x7f1f1815aad0 2023-10-08T05:52:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.648+0000 7f1f1f353640 1 -- 172.21.15.70:0/1639196839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f18159800 con 0x7f1f1815aad0 2023-10-08T05:52:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.648+0000 7f1f1f353640 1 -- 172.21.15.70:0/1639196839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f18158c30 con 0x7f1f1815aad0 2023-10-08T05:52:03.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.649+0000 7f1f1f353640 1 -- 172.21.15.70:0/1639196839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f18159d20 con 0x7f1f1815aad0 2023-10-08T05:52:03.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.650+0000 7f1f0dffb640 1 -- 172.21.15.70:0/1639196839 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 25) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f1f10003990 con 0x7f1f1815aad0 2023-10-08T05:52:03.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.651+0000 7f1f0dffb640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f1efc06afe0 0x7f1efc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:03.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.651+0000 7f1f0dffb640 1 -- 172.21.15.70:0/1639196839 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f1f10089910 con 0x7f1f1815aad0 2023-10-08T05:52:03.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.652+0000 7f1f1d0ef640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f1efc06afe0 0x7f1efc06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:03.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.653+0000 7f1f1d0ef640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f1efc06afe0 0x7f1efc06d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f14035000 tx=0x7f1f14030000 comp rx=0 tx=0).ready entity=mgr.5380 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:03.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.655+0000 7f1f0dffb640 1 -- 172.21.15.70:0/1639196839 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f18159d20 con 0x7f1f1815aad0 2023-10-08T05:52:03.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:03.894+0000 7f1f1f353640 1 -- 172.21.15.70:0/1639196839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f1f18159d20 con 0x7f1f1815aad0 2023-10-08T05:52:04.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.383+0000 7f1f0dffb640 1 -- 172.21.15.70:0/1639196839 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v26) v1 ==== 93+0+0 (secure 0 0 0) 0x7f1f18159d20 con 0x7f1f1815aad0 2023-10-08T05:52:04.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.387+0000 7f1f1f353640 1 -- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f1efc06afe0 msgr2=0x7f1efc06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:04.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.387+0000 7f1f1f353640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f1efc06afe0 0x7f1efc06d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1f14035000 tx=0x7f1f14030000 comp rx=0 tx=0).stop 2023-10-08T05:52:04.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.387+0000 7f1f1f353640 1 -- 172.21.15.70:0/1639196839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 msgr2=0x7f1f180c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.387+0000 7f1f1f353640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 0x7f1f180c9e20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000c570 tx=0x7f1f1000ca30 comp rx=0 tx=0).stop 2023-10-08T05:52:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.387+0000 7f1f1f353640 1 -- 172.21.15.70:0/1639196839 shutdown_connections 2023-10-08T05:52:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.387+0000 7f1f1f353640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f1efc06afe0 0x7f1efc06d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.388+0000 7f1f1f353640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f1815aad0 0x7f1f180c9e20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.388+0000 7f1f1f353640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f180c82b0 0x7f1f181507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:04.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.388+0000 7f1f1f353640 1 --2- 172.21.15.70:0/1639196839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f1806cd00 0x7f1f181432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:04.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.388+0000 7f1f1f353640 1 -- 172.21.15.70:0/1639196839 >> 172.21.15.70:0/1639196839 conn(0x7f1f18069d50 msgr2=0x7f1f18154c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:04.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.388+0000 7f1f1f353640 1 -- 172.21.15.70:0/1639196839 shutdown_connections 2023-10-08T05:52:04.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.389+0000 7f1f1f353640 1 -- 172.21.15.70:0/1639196839 wait complete. 2023-10-08T05:52:04.419 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:52:04.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:52:04.521 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:52:04.617 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:52:04.618 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:04.618 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:04.670 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:04.670 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:04.762 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:04.916 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:04.915+0000 7fc954a32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:52:04.924 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:04.923+0000 7f189b811040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:52:04.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.962+0000 7f6840a0f640 1 Processor -- start 2023-10-08T05:52:04.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.962+0000 7f6840a0f640 1 -- start start 2023-10-08T05:52:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.963+0000 7f6840a0f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f683c0adc70 0x7f683c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.964+0000 7f6840a0f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f683c15af10 0x7f683c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.964+0000 7f6840a0f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c15f7d0 con 0x7f683c15ab30 2023-10-08T05:52:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.964+0000 7f6840a0f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683c15efb0 con 0x7f683c15af10 2023-10-08T05:52:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.964+0000 7f6840a0f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c15f210 con 0x7f683c0adc70 2023-10-08T05:52:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.964+0000 7f683b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f683c15af10 0x7f683c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.964+0000 7f683b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f683c15af10 0x7f683c0b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43084/0 (socket says 172.21.15.70:43084) 2023-10-08T05:52:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.964+0000 7f683b59e640 1 -- 172.21.15.70:0/1050924746 learned_addr learned my addr 172.21.15.70:0/1050924746 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.964+0000 7f683a59c640 1 --2- 172.21.15.70:0/1050924746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f683c0adc70 0x7f683c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.965+0000 7f683b59e640 1 -- 172.21.15.70:0/1050924746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f683c0adc70 msgr2=0x7f683c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.965+0000 7f683b59e640 1 --2- 172.21.15.70:0/1050924746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f683c0adc70 0x7f683c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.965+0000 7f683b59e640 1 -- 172.21.15.70:0/1050924746 >> v1:172.21.15.181:6789/0 conn(0x7f683c15ab30 legacy=0x7f683c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.965+0000 7f683b59e640 1 -- 172.21.15.70:0/1050924746 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c15f4f0 con 0x7f683c15af10 2023-10-08T05:52:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.966+0000 7f683b59e640 1 --2- 172.21.15.70:0/1050924746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 0x7f683c0b0600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6834009e50 tx=0x7f683402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6874ccfdf8742a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.966+0000 7f6839d9b640 1 -- 172.21.15.70:0/1050924746 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683402cb00 con 0x7f683c15af10 2023-10-08T05:52:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.966+0000 7f6839d9b640 1 -- 172.21.15.70:0/1050924746 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683402cc60 con 0x7f683c15af10 2023-10-08T05:52:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.966+0000 7f6839d9b640 1 -- 172.21.15.70:0/1050924746 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68340343f0 con 0x7f683c15af10 2023-10-08T05:52:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.966+0000 7f6840a0f640 1 -- 172.21.15.70:0/1050924746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 msgr2=0x7f683c0b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.966+0000 7f6840a0f640 1 --2- 172.21.15.70:0/1050924746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 0x7f683c0b0600 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6834009e50 tx=0x7f683402bf60 comp rx=0 tx=0).stop 2023-10-08T05:52:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.967+0000 7f6840a0f640 1 -- 172.21.15.70:0/1050924746 shutdown_connections 2023-10-08T05:52:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.967+0000 7f6840a0f640 1 --2- 172.21.15.70:0/1050924746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 0x7f683c0b0600 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.967+0000 7f6840a0f640 1 --2- 172.21.15.70:0/1050924746 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f683c0adc70 0x7f683c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.967+0000 7f6840a0f640 1 -- 172.21.15.70:0/1050924746 >> 172.21.15.70:0/1050924746 conn(0x7f683c0cb0c0 msgr2=0x7f683c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.967+0000 7f6840a0f640 1 -- 172.21.15.70:0/1050924746 shutdown_connections 2023-10-08T05:52:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.967+0000 7f6840a0f640 1 -- 172.21.15.70:0/1050924746 wait complete. 2023-10-08T05:52:04.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.969+0000 7f6840a0f640 1 Processor -- start 2023-10-08T05:52:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.969+0000 7f6840a0f640 1 -- start start 2023-10-08T05:52:04.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.971+0000 7f6840a0f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f683c0adc70 0x7f683c0c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.971+0000 7f6840a0f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f683c15ab30 0x7f683c0c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.971+0000 7f6840a0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 0x7f683c0c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.971+0000 7f6840a0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0ccc20 con 0x7f683c15af10 2023-10-08T05:52:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.971+0000 7f6840a0f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0c8570 con 0x7f683c0adc70 2023-10-08T05:52:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.971+0000 7f6840a0f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c0c6660 con 0x7f683c15ab30 2023-10-08T05:52:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.971+0000 7f683b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 0x7f683c0c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.971+0000 7f683b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 0x7f683c0c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43100/0 (socket says 172.21.15.70:43100) 2023-10-08T05:52:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.971+0000 7f683b59e640 1 -- 172.21.15.70:0/1654489104 learned_addr learned my addr 172.21.15.70:0/1654489104 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f683a59c640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f683c15ab30 0x7f683c0c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f683ad9d640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f683c0adc70 0x7f683c0c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f683b59e640 1 -- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f683c15ab30 msgr2=0x7f683c0c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f683b59e640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f683c15ab30 0x7f683c0c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f683b59e640 1 -- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f683c0adc70 msgr2=0x7f683c0c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f683b59e640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f683c0adc70 0x7f683c0c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f683b59e640 1 -- 172.21.15.70:0/1654489104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6834009b30 con 0x7f683c15af10 2023-10-08T05:52:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f683ad9d640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f683c0adc70 0x7f683c0c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f683b59e640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 0x7f683c0c8030 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6834009e50 tx=0x7f6834004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f68237fe640 1 -- 172.21.15.70:0/1654489104 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834009510 con 0x7f683c15af10 2023-10-08T05:52:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f68237fe640 1 -- 172.21.15.70:0/1654489104 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683403e450 con 0x7f683c15af10 2023-10-08T05:52:04.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f68237fe640 1 -- 172.21.15.70:0/1654489104 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683403e800 con 0x7f683c15af10 2023-10-08T05:52:04.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.972+0000 7f6840a0f640 1 -- 172.21.15.70:0/1654489104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c0cce70 con 0x7f683c15af10 2023-10-08T05:52:04.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.973+0000 7f6840a0f640 1 -- 172.21.15.70:0/1654489104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c0cd430 con 0x7f683c15af10 2023-10-08T05:52:04.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.974+0000 7f6840a0f640 1 -- 172.21.15.70:0/1654489104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c04e160 con 0x7f683c15af10 2023-10-08T05:52:04.976 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:04.974+0000 7fc954a32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:52:04.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.975+0000 7f68237fe640 1 -- 172.21.15.70:0/1654489104 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f683403e960 con 0x7f683c15af10 2023-10-08T05:52:04.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.976+0000 7f68237fe640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f682806af90 0x7f682806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:04.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.976+0000 7f68237fe640 1 -- 172.21.15.70:0/1654489104 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f68340ac8c0 con 0x7f683c15af10 2023-10-08T05:52:04.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.976+0000 7f683ad9d640 1 -- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f682806af90 msgr2=0x7f682806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:04.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.976+0000 7f683ad9d640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f682806af90 0x7f682806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:04.979 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:04.978+0000 7f189b811040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:52:04.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:04.980+0000 7f68237fe640 1 -- 172.21.15.70:0/1654489104 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68340774c0 con 0x7f683c15af10 2023-10-08T05:52:05.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.172+0000 7f6840a0f640 1 -- 172.21.15.70:0/1654489104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f683c0ada50 con 0x7f683c15af10 2023-10-08T05:52:05.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.174+0000 7f68237fe640 1 -- 172.21.15.70:0/1654489104 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+17431 (secure 0 0 0) 0x7f683407aae0 con 0x7f683c15af10 2023-10-08T05:52:05.176 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:05.176 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":75,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:51:32.320789+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:50:14.005619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-08T05:50:15.163925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1060036340":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/464684516":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/1910914026":"2023-10-09T05:50:30.831312+0000","172.21.15.181:6833/992174303":"2023-10-09T05:50:30.831312+0000","172.21.15.181:6832/992174303":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/596247496":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/368598410":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/3145795928":"2023-10-09T05:50:30.831312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:05.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.176+0000 7f6840a0f640 1 -- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f682806af90 msgr2=0x7f682806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:05.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.176+0000 7f6840a0f640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f682806af90 0x7f682806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.177+0000 7f6840a0f640 1 -- 172.21.15.70:0/1654489104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 msgr2=0x7f683c0c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:05.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.177+0000 7f6840a0f640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 0x7f683c0c8030 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6834009e50 tx=0x7f6834004a40 comp rx=0 tx=0).stop 2023-10-08T05:52:05.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.177+0000 7f6840a0f640 1 -- 172.21.15.70:0/1654489104 shutdown_connections 2023-10-08T05:52:05.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.177+0000 7f6840a0f640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f682806af90 0x7f682806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.177+0000 7f6840a0f640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f683c15af10 0x7f683c0c8030 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.177+0000 7f6840a0f640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f683c15ab30 0x7f683c0c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.177+0000 7f6840a0f640 1 --2- 172.21.15.70:0/1654489104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f683c0adc70 0x7f683c0c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.177+0000 7f6840a0f640 1 -- 172.21.15.70:0/1654489104 >> 172.21.15.70:0/1654489104 conn(0x7f683c0cb0c0 msgr2=0x7f683c159440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:05.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.177+0000 7f6840a0f640 1 -- 172.21.15.70:0/1654489104 shutdown_connections 2023-10-08T05:52:05.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.177+0000 7f6840a0f640 1 -- 172.21.15.70:0/1654489104 wait complete. 2023-10-08T05:52:05.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:50:14.005619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:50:15.163925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:52:05.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:52:05.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.387+0000 7f49872da640 1 Processor -- start 2023-10-08T05:52:05.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.387+0000 7f49872da640 1 -- start start 2023-10-08T05:52:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.389+0000 7f49872da640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f498015d060 0x7f49800caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.389+0000 7f49872da640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f498006b180 0x7f498015d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.389+0000 7f49872da640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49800caa90 con 0x7f498015da80 2023-10-08T05:52:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.389+0000 7f49872da640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498015e3f0 con 0x7f498006b180 2023-10-08T05:52:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.389+0000 7f49872da640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498015def0 con 0x7f498015d060 2023-10-08T05:52:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.389+0000 7f4985ad7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f498006b180 0x7f498015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.389+0000 7f4985ad7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f498006b180 0x7f498015d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43112/0 (socket says 172.21.15.70:43112) 2023-10-08T05:52:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.389+0000 7f4985ad7640 1 -- 172.21.15.70:0/519302338 learned_addr learned my addr 172.21.15.70:0/519302338 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.389+0000 7f49862d8640 1 --2- 172.21.15.70:0/519302338 >> v2:172.21.15.181:3300/0 conn(0x7f498015d060 0x7f49800caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.390+0000 7f4985ad7640 1 -- 172.21.15.70:0/519302338 >> v2:172.21.15.181:3300/0 conn(0x7f498015d060 msgr2=0x7f49800caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.390+0000 7f4985ad7640 1 --2- 172.21.15.70:0/519302338 >> v2:172.21.15.181:3300/0 conn(0x7f498015d060 0x7f49800caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.390+0000 7f4985ad7640 1 -- 172.21.15.70:0/519302338 >> v1:172.21.15.181:6789/0 conn(0x7f498015da80 legacy=0x7f498006c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.390+0000 7f4985ad7640 1 -- 172.21.15.70:0/519302338 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498015e110 con 0x7f498006b180 2023-10-08T05:52:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.391+0000 7f4985ad7640 1 --2- 172.21.15.70:0/519302338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 0x7f498015d540 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4980067390 tx=0x7f497002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a4bd094e5d48c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.391+0000 7f49852d6640 1 -- 172.21.15.70:0/519302338 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700317e0 con 0x7f498006b180 2023-10-08T05:52:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.391+0000 7f49852d6640 1 -- 172.21.15.70:0/519302338 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4970031940 con 0x7f498006b180 2023-10-08T05:52:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.391+0000 7f49852d6640 1 -- 172.21.15.70:0/519302338 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4970031cd0 con 0x7f498006b180 2023-10-08T05:52:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.392+0000 7f49872da640 1 -- 172.21.15.70:0/519302338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 msgr2=0x7f498015d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:05.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.392+0000 7f49872da640 1 --2- 172.21.15.70:0/519302338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 0x7f498015d540 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4980067390 tx=0x7f497002b880 comp rx=0 tx=0).stop 2023-10-08T05:52:05.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.392+0000 7f49872da640 1 -- 172.21.15.70:0/519302338 shutdown_connections 2023-10-08T05:52:05.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.392+0000 7f49872da640 1 --2- 172.21.15.70:0/519302338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 0x7f498015d540 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.392+0000 7f49872da640 1 --2- 172.21.15.70:0/519302338 >> v2:172.21.15.181:3300/0 conn(0x7f498015d060 0x7f49800caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.392+0000 7f49872da640 1 -- 172.21.15.70:0/519302338 >> 172.21.15.70:0/519302338 conn(0x7f49800b08a0 msgr2=0x7f49800b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:05.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.392+0000 7f49872da640 1 -- 172.21.15.70:0/519302338 shutdown_connections 2023-10-08T05:52:05.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.392+0000 7f49872da640 1 -- 172.21.15.70:0/519302338 wait complete. 2023-10-08T05:52:05.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.394+0000 7f49872da640 1 Processor -- start 2023-10-08T05:52:05.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.394+0000 7f49872da640 1 -- start start 2023-10-08T05:52:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49872da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 0x7f49801f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49872da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498015d060 0x7f49800c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49872da640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498015da80 0x7f4980156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49872da640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800c8bd0 con 0x7f498006b180 2023-10-08T05:52:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49872da640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800c8dc0 con 0x7f498015da80 2023-10-08T05:52:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49872da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49801f0530 con 0x7f498015d060 2023-10-08T05:52:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49862d8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 0x7f49801f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49862d8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 0x7f49801f4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43120/0 (socket says 172.21.15.70:43120) 2023-10-08T05:52:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49862d8640 1 -- 172.21.15.70:0/2864967297 learned_addr learned my addr 172.21.15.70:0/2864967297 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f4986ad9640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498015da80 0x7f4980156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f4985ad7640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498015d060 0x7f49800c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49862d8640 1 -- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498015d060 msgr2=0x7f49800c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49862d8640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498015d060 0x7f49800c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49862d8640 1 -- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498015da80 msgr2=0x7f4980156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49862d8640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498015da80 0x7f4980156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f49862d8640 1 -- 172.21.15.70:0/2864967297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700094d0 con 0x7f498006b180 2023-10-08T05:52:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.395+0000 7f4986ad9640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498015da80 0x7f4980156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:52:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.396+0000 7f4985ad7640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498015d060 0x7f49800c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:05.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.396+0000 7f49862d8640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 0x7f49801f4610 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f497c00d800 tx=0x7f497c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:05.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.396+0000 7f4976ffd640 1 -- 172.21.15.70:0/2864967297 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c00b6c0 con 0x7f498006b180 2023-10-08T05:52:05.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.396+0000 7f4976ffd640 1 -- 172.21.15.70:0/2864967297 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c00b820 con 0x7f498006b180 2023-10-08T05:52:05.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.396+0000 7f4976ffd640 1 -- 172.21.15.70:0/2864967297 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c00bbb0 con 0x7f498006b180 2023-10-08T05:52:05.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.396+0000 7f49872da640 1 -- 172.21.15.70:0/2864967297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49801f10a0 con 0x7f498006b180 2023-10-08T05:52:05.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.396+0000 7f49872da640 1 -- 172.21.15.70:0/2864967297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49801f0a50 con 0x7f498006b180 2023-10-08T05:52:05.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.397+0000 7f49872da640 1 -- 172.21.15.70:0/2864967297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49800720a0 con 0x7f498006b180 2023-10-08T05:52:05.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.398+0000 7f4976ffd640 1 -- 172.21.15.70:0/2864967297 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f497c00bd10 con 0x7f498006b180 2023-10-08T05:52:05.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.399+0000 7f4976ffd640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f496406af90 0x7f496406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.399+0000 7f4976ffd640 1 -- 172.21.15.70:0/2864967297 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f497c08a0d0 con 0x7f498006b180 2023-10-08T05:52:05.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.399+0000 7f4985ad7640 1 -- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f496406af90 msgr2=0x7f496406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:05.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.399+0000 7f4985ad7640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f496406af90 0x7f496406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:05.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.403+0000 7f4976ffd640 1 -- 172.21.15.70:0/2864967297 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f497c054cd0 con 0x7f498006b180 2023-10-08T05:52:05.431 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:05.430+0000 7f189b811040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:52:05.447 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:05.446+0000 7fc954a32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:52:05.483 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:05.482+0000 7f189b811040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:52:05.502 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:05.500+0000 7fc954a32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:52:05.533 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:05.532+0000 7f189b811040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:52:05.554 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:05.552+0000 7fc954a32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:52:05.595 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:05.595+0000 7f189b811040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:52:05.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.600+0000 7f4985ad7640 1 -- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f496406af90 msgr2=0x7f496406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:05.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.600+0000 7f4985ad7640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f496406af90 0x7f496406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:05.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.602+0000 7f49872da640 1 -- 172.21.15.70:0/2864967297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4980065fa0 con 0x7f498006b180 2023-10-08T05:52:05.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.603+0000 7f4976ffd640 1 -- 172.21.15.70:0/2864967297 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f497c0582f0 con 0x7f498006b180 2023-10-08T05:52:05.604 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:52:05.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 -- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f496406af90 msgr2=0x7f496406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:05.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f496406af90 0x7f496406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 -- 172.21.15.70:0/2864967297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 msgr2=0x7f49801f4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:05.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 0x7f49801f4610 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f497c00d800 tx=0x7f497c00dcc0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 -- 172.21.15.70:0/2864967297 shutdown_connections 2023-10-08T05:52:05.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f496406af90 0x7f496406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f498015da80 0x7f4980156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f498015d060 0x7f49800c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 --2- 172.21.15.70:0/2864967297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f498006b180 0x7f49801f4610 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 -- 172.21.15.70:0/2864967297 >> 172.21.15.70:0/2864967297 conn(0x7f49800b08a0 msgr2=0x7f49800cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:05.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 -- 172.21.15.70:0/2864967297 shutdown_connections 2023-10-08T05:52:05.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.605+0000 7f49872da640 1 -- 172.21.15.70:0/2864967297 wait complete. 2023-10-08T05:52:05.622 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:05.621+0000 7fc954a32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:52:05.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:52:05.644 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:05.644+0000 7f189b811040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:52:05.674 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:05.674+0000 7fc954a32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:52:05.755 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:05.754+0000 7f189b811040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:52:05.795 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:05.793+0000 7fc954a32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:52:05.817 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:05.816+0000 7f189b811040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:05.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.824+0000 7f57c3656640 1 Processor -- start 2023-10-08T05:52:05.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.824+0000 7f57c3656640 1 -- start start 2023-10-08T05:52:05.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.825+0000 7f57c3656640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f57bc0b21f0 0x7f57bc0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.826+0000 7f57c3656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 0x7f57bc06d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.826+0000 7f57c3656640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57bc06c3a0 con 0x7f57bc0702b0 2023-10-08T05:52:05.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.826+0000 7f57c3656640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57bc06c640 con 0x7f57bc0b21f0 2023-10-08T05:52:05.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.826+0000 7f57c3656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc06c920 con 0x7f57bc0ca1a0 2023-10-08T05:52:05.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.826+0000 7f57c0bf1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f57bc0b21f0 0x7f57bc0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:05.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.826+0000 7f57c13f2640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f57bc0702b0 0x7f57bc070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:52362/0 (socket says 172.21.15.70:52362) 2023-10-08T05:52:05.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.826+0000 7f57c13f2640 1 -- 172.21.15.70:0/1244452851 learned_addr learned my addr 172.21.15.70:0/1244452851 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:05.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.826+0000 7f57c1bf3640 1 --2- 172.21.15.70:0/1244452851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 0x7f57bc06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:05.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.827+0000 7f57b3fff640 1 -- 172.21.15.70:0/1244452851 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148609005 0 0) 0x7f57bc06c3a0 con 0x7f57bc0702b0 2023-10-08T05:52:05.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.827+0000 7f57b3fff640 1 -- 172.21.15.70:0/1244452851 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a8005730 con 0x7f57bc0702b0 2023-10-08T05:52:05.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.827+0000 7f57c1bf3640 1 -- 172.21.15.70:0/1244452851 >> v2:172.21.15.181:3300/0 conn(0x7f57bc0b21f0 msgr2=0x7f57bc0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:05.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.827+0000 7f57c1bf3640 1 --2- 172.21.15.70:0/1244452851 >> v2:172.21.15.181:3300/0 conn(0x7f57bc0b21f0 0x7f57bc0c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.827+0000 7f57c1bf3640 1 -- 172.21.15.70:0/1244452851 >> v1:172.21.15.181:6789/0 conn(0x7f57bc0702b0 legacy=0x7f57bc070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:05.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.827+0000 7f57c1bf3640 1 -- 172.21.15.70:0/1244452851 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc06cc00 con 0x7f57bc0ca1a0 2023-10-08T05:52:05.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.828+0000 7f57c1bf3640 1 --2- 172.21.15.70:0/1244452851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 0x7f57bc06d400 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009d90 tx=0x7f57b402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d868ea423ebdcf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:05.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57b3fff640 1 -- 172.21.15.70:0/1244452851 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b402cc50 con 0x7f57bc0ca1a0 2023-10-08T05:52:05.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57b3fff640 1 -- 172.21.15.70:0/1244452851 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b402cdb0 con 0x7f57bc0ca1a0 2023-10-08T05:52:05.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57b3fff640 1 -- 172.21.15.70:0/1244452851 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4034660 con 0x7f57bc0ca1a0 2023-10-08T05:52:05.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57c3656640 1 -- 172.21.15.70:0/1244452851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 msgr2=0x7f57bc06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:05.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57c3656640 1 --2- 172.21.15.70:0/1244452851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 0x7f57bc06d400 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f57b4009d90 tx=0x7f57b402bea0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57c3656640 1 -- 172.21.15.70:0/1244452851 shutdown_connections 2023-10-08T05:52:05.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57c3656640 1 --2- 172.21.15.70:0/1244452851 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 0x7f57bc06d400 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57c3656640 1 --2- 172.21.15.70:0/1244452851 >> v2:172.21.15.181:3300/0 conn(0x7f57bc0b21f0 0x7f57bc0c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57c3656640 1 -- 172.21.15.70:0/1244452851 >> 172.21.15.70:0/1244452851 conn(0x7f57bc06a5d0 msgr2=0x7f57bc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:05.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57c3656640 1 -- 172.21.15.70:0/1244452851 shutdown_connections 2023-10-08T05:52:05.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.829+0000 7f57c3656640 1 -- 172.21.15.70:0/1244452851 wait complete. 2023-10-08T05:52:05.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.831+0000 7f57c3656640 1 Processor -- start 2023-10-08T05:52:05.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.831+0000 7f57c3656640 1 -- start start 2023-10-08T05:52:05.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.832+0000 7f57c3656640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57bc0702b0 0x7f57bc0c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.832+0000 7f57c3656640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57bc0b21f0 0x7f57bc0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.832+0000 7f57c3656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 0x7f57bc0ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.832+0000 7f57c3656640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0c2f50 con 0x7f57bc0702b0 2023-10-08T05:52:05.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.832+0000 7f57c3656640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0c50b0 con 0x7f57bc0b21f0 2023-10-08T05:52:05.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.832+0000 7f57c3656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc0c22d0 con 0x7f57bc0ca1a0 2023-10-08T05:52:05.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.832+0000 7f57c13f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57bc0702b0 0x7f57bc0c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:05.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.832+0000 7f57c13f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57bc0702b0 0x7f57bc0c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43128/0 (socket says 172.21.15.70:43128) 2023-10-08T05:52:05.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.832+0000 7f57c13f2640 1 -- 172.21.15.70:0/2113554724 learned_addr learned my addr 172.21.15.70:0/2113554724 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:05.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57c0bf1640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57bc0b21f0 0x7f57bc0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:05.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57c1bf3640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 0x7f57bc0ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57c13f2640 1 -- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 msgr2=0x7f57bc0ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57c13f2640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 0x7f57bc0ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57c13f2640 1 -- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57bc0b21f0 msgr2=0x7f57bc0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57c13f2640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57bc0b21f0 0x7f57bc0cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57c13f2640 1 -- 172.21.15.70:0/2113554724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b4009a70 con 0x7f57bc0702b0 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57c0bf1640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57bc0b21f0 0x7f57bc0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57c13f2640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57bc0702b0 0x7f57bc0c4b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f57b800c500 tx=0x7f57b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57b1ffb640 1 -- 172.21.15.70:0/2113554724 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8005ce0 con 0x7f57bc0702b0 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57b1ffb640 1 -- 172.21.15.70:0/2113554724 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b8005e40 con 0x7f57bc0702b0 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57c3656640 1 -- 172.21.15.70:0/2113554724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57bc0c25b0 con 0x7f57bc0702b0 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.833+0000 7f57b1ffb640 1 -- 172.21.15.70:0/2113554724 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8007330 con 0x7f57bc0702b0 2023-10-08T05:52:05.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.834+0000 7f57c3656640 1 -- 172.21.15.70:0/2113554724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57bc0c2b70 con 0x7f57bc0702b0 2023-10-08T05:52:05.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.835+0000 7f57c3656640 1 -- 172.21.15.70:0/2113554724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57bc0c3090 con 0x7f57bc0702b0 2023-10-08T05:52:05.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.835+0000 7f57b1ffb640 1 -- 172.21.15.70:0/2113554724 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f57b8007490 con 0x7f57bc0702b0 2023-10-08T05:52:05.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.837+0000 7f57b1ffb640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f57a006af90 0x7f57a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:05.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.837+0000 7f57b1ffb640 1 -- 172.21.15.70:0/2113554724 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6563+0+0 (secure 0 0 0) 0x7f57b8089190 con 0x7f57bc0702b0 2023-10-08T05:52:05.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.837+0000 7f57c0bf1640 1 -- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f57a006af90 msgr2=0x7f57a006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:05.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.837+0000 7f57c0bf1640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f57a006af90 0x7f57a006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:05.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:05.841+0000 7f57b1ffb640 1 -- 172.21.15.70:0/2113554724 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57bc0c3090 con 0x7f57bc0702b0 2023-10-08T05:52:05.868 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:05.867+0000 7f189b811040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:52:05.874 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:05.873+0000 7fc954a32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:05.925 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:05.924+0000 7f189b811040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:52:05.933 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:05.932+0000 7fc954a32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:52:05.975 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:05.974+0000 7f189b811040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:52:05.994 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:05.993+0000 7fc954a32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:52:06.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.038+0000 7f57c0bf1640 1 -- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f57a006af90 msgr2=0x7f57a006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:06.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.038+0000 7f57c0bf1640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f57a006af90 0x7f57a006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:06.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.042+0000 7f57c3656640 1 -- 172.21.15.70:0/2113554724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57bc0c3090 con 0x7f57bc0702b0 2023-10-08T05:52:06.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.043+0000 7f57b1ffb640 1 -- 172.21.15.70:0/2113554724 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57bc0c3090 con 0x7f57bc0702b0 2023-10-08T05:52:06.044 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:52:06.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.045+0000 7f57c3656640 1 -- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f57a006af90 msgr2=0x7f57a006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:06.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.045+0000 7f57c3656640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f57a006af90 0x7f57a006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.045+0000 7f57c3656640 1 -- 172.21.15.70:0/2113554724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57bc0702b0 msgr2=0x7f57bc0c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:06.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.045+0000 7f57c3656640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57bc0702b0 0x7f57bc0c4b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f57b800c500 tx=0x7f57b800c9c0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.045+0000 7f57c3656640 1 -- 172.21.15.70:0/2113554724 shutdown_connections 2023-10-08T05:52:06.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.045+0000 7f57c3656640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f57a006af90 0x7f57a006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.045+0000 7f57c3656640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57bc0ca1a0 0x7f57bc0ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.045+0000 7f57c3656640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57bc0b21f0 0x7f57bc0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.045+0000 7f57c3656640 1 --2- 172.21.15.70:0/2113554724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57bc0702b0 0x7f57bc0c4b70 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.045+0000 7f57c3656640 1 -- 172.21.15.70:0/2113554724 >> 172.21.15.70:0/2113554724 conn(0x7f57bc06a5d0 msgr2=0x7f57bc15b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:06.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.046+0000 7f57c3656640 1 -- 172.21.15.70:0/2113554724 shutdown_connections 2023-10-08T05:52:06.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.046+0000 7f57c3656640 1 -- 172.21.15.70:0/2113554724 wait complete. 2023-10-08T05:52:06.048 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:06.047+0000 7fc954a32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:52:06.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:52:06.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.264+0000 7fee8824b640 1 Processor -- start 2023-10-08T05:52:06.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.265+0000 7fee8824b640 1 -- start start 2023-10-08T05:52:06.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.266+0000 7fee8824b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fee800afbe0 0x7fee800b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.266+0000 7fee8824b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fee8006dfc0 0x7fee80070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.266+0000 7fee8824b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee80071070 0x7fee800cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.266+0000 7fee8824b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee800b2620 con 0x7fee800afbe0 2023-10-08T05:52:06.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.266+0000 7fee8824b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee800b2060 con 0x7fee8006dfc0 2023-10-08T05:52:06.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.266+0000 7fee8824b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee800b2340 con 0x7fee80071070 2023-10-08T05:52:06.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.266+0000 7fee85fe7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fee800afbe0 0x7fee800b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:06.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.267+0000 7fee85fe7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fee800afbe0 0x7fee800b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43138/0 (socket says 172.21.15.70:43138) 2023-10-08T05:52:06.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.267+0000 7fee85fe7640 1 -- 172.21.15.70:0/1630673736 learned_addr learned my addr 172.21.15.70:0/1630673736 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:06.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.267+0000 7fee857e6640 1 --2- 172.21.15.70:0/1630673736 >> v2:172.21.15.181:3300/0 conn(0x7fee8006dfc0 0x7fee80070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:06.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.267+0000 7fee85fe7640 1 -- 172.21.15.70:0/1630673736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee80071070 msgr2=0x7fee800cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:06.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.267+0000 7fee85fe7640 1 --2- 172.21.15.70:0/1630673736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee80071070 0x7fee800cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.267+0000 7fee85fe7640 1 -- 172.21.15.70:0/1630673736 >> v2:172.21.15.181:3300/0 conn(0x7fee8006dfc0 msgr2=0x7fee80070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:06.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.267+0000 7fee85fe7640 1 --2- 172.21.15.70:0/1630673736 >> v2:172.21.15.181:3300/0 conn(0x7fee8006dfc0 0x7fee80070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.267+0000 7fee85fe7640 1 -- 172.21.15.70:0/1630673736 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee80070350 con 0x7fee800afbe0 2023-10-08T05:52:06.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.268+0000 7fee85fe7640 1 --2- 172.21.15.70:0/1630673736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee800afbe0 0x7fee800b15b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fee800b3980 tx=0x7fee7c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68bde7bf7480c36 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:06.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.269+0000 7fee84fe5640 1 -- 172.21.15.70:0/1630673736 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c031870 con 0x7fee800afbe0 2023-10-08T05:52:06.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.269+0000 7fee84fe5640 1 -- 172.21.15.70:0/1630673736 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c0319d0 con 0x7fee800afbe0 2023-10-08T05:52:06.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.269+0000 7fee84fe5640 1 -- 172.21.15.70:0/1630673736 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c031d60 con 0x7fee800afbe0 2023-10-08T05:52:06.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.269+0000 7fee8824b640 1 -- 172.21.15.70:0/1630673736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee800afbe0 msgr2=0x7fee800b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:06.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.269+0000 7fee8824b640 1 --2- 172.21.15.70:0/1630673736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee800afbe0 0x7fee800b15b0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fee800b3980 tx=0x7fee7c02b910 comp rx=0 tx=0).stop 2023-10-08T05:52:06.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.269+0000 7fee8824b640 1 -- 172.21.15.70:0/1630673736 shutdown_connections 2023-10-08T05:52:06.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.269+0000 7fee8824b640 1 --2- 172.21.15.70:0/1630673736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee80071070 0x7fee800cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.269+0000 7fee8824b640 1 --2- 172.21.15.70:0/1630673736 >> v2:172.21.15.181:3300/0 conn(0x7fee8006dfc0 0x7fee80070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.269+0000 7fee8824b640 1 --2- 172.21.15.70:0/1630673736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee800afbe0 0x7fee800b15b0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.269+0000 7fee8824b640 1 -- 172.21.15.70:0/1630673736 >> 172.21.15.70:0/1630673736 conn(0x7fee8006a5d0 msgr2=0x7fee800b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:06.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.270+0000 7fee8824b640 1 -- 172.21.15.70:0/1630673736 shutdown_connections 2023-10-08T05:52:06.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.270+0000 7fee8824b640 1 -- 172.21.15.70:0/1630673736 wait complete. 2023-10-08T05:52:06.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.271+0000 7fee8824b640 1 Processor -- start 2023-10-08T05:52:06.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.271+0000 7fee8824b640 1 -- start start 2023-10-08T05:52:06.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.272+0000 7fee8824b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee8006dfc0 0x7fee80150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.272+0000 7fee8824b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee80071070 0x7fee800ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.272+0000 7fee8824b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee800afbe0 0x7fee800cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.272+0000 7fee8824b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8015dfe0 con 0x7fee8006dfc0 2023-10-08T05:52:06.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.272+0000 7fee8824b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee801510d0 con 0x7fee800afbe0 2023-10-08T05:52:06.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.272+0000 7fee8824b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8015d360 con 0x7fee80071070 2023-10-08T05:52:06.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee85fe7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee8006dfc0 0x7fee80150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:06.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee85fe7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee8006dfc0 0x7fee80150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43140/0 (socket says 172.21.15.70:43140) 2023-10-08T05:52:06.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee85fe7640 1 -- 172.21.15.70:0/2521248687 learned_addr learned my addr 172.21.15.70:0/2521248687 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:06.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee857e6640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee80071070 0x7fee800ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:06.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee867e8640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee800afbe0 0x7fee800cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:06.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee85fe7640 1 -- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee80071070 msgr2=0x7fee800ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:06.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee85fe7640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee80071070 0x7fee800ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee85fe7640 1 -- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee800afbe0 msgr2=0x7fee800cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:06.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee85fe7640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee800afbe0 0x7fee800cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee85fe7640 1 -- 172.21.15.70:0/2521248687 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c0094d0 con 0x7fee8006dfc0 2023-10-08T05:52:06.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee85fe7640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee8006dfc0 0x7fee80150b90 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fee7c000c00 tx=0x7fee7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:06.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee767fc640 1 -- 172.21.15.70:0/2521248687 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c03abe0 con 0x7fee8006dfc0 2023-10-08T05:52:06.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee767fc640 1 -- 172.21.15.70:0/2521248687 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c03ad40 con 0x7fee8006dfc0 2023-10-08T05:52:06.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee767fc640 1 -- 172.21.15.70:0/2521248687 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c044610 con 0x7fee8006dfc0 2023-10-08T05:52:06.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee8824b640 1 -- 172.21.15.70:0/2521248687 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8015d640 con 0x7fee8006dfc0 2023-10-08T05:52:06.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.274+0000 7fee8824b640 1 -- 172.21.15.70:0/2521248687 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8015dc00 con 0x7fee8006dfc0 2023-10-08T05:52:06.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.276+0000 7fee8824b640 1 -- 172.21.15.70:0/2521248687 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee8004e160 con 0x7fee8006dfc0 2023-10-08T05:52:06.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.276+0000 7fee767fc640 1 -- 172.21.15.70:0/2521248687 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fee7c04d450 con 0x7fee8006dfc0 2023-10-08T05:52:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.278+0000 7fee767fc640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fee6406af90 0x7fee6406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.278+0000 7fee767fc640 1 -- 172.21.15.70:0/2521248687 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6563+0+0 (secure 0 0 0) 0x7fee7c02d040 con 0x7fee8006dfc0 2023-10-08T05:52:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.278+0000 7fee857e6640 1 -- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fee6406af90 msgr2=0x7fee6406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:06.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.278+0000 7fee857e6640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fee6406af90 0x7fee6406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:06.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.282+0000 7fee767fc640 1 -- 172.21.15.70:0/2521248687 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee7c0deca0 con 0x7fee8006dfc0 2023-10-08T05:52:06.366 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:06.365+0000 7f189b811040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:52:06.463 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:06.463+0000 7fc954a32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:52:06.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.476+0000 7fee8824b640 1 -- 172.21.15.70:0/2521248687 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee80151a10 con 0x7fee8006dfc0 2023-10-08T05:52:06.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.477+0000 7fee767fc640 1 -- 172.21.15.70:0/2521248687 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee7c076d10 con 0x7fee8006dfc0 2023-10-08T05:52:06.477 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:52:06.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.479+0000 7fee857e6640 1 -- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fee6406af90 msgr2=0x7fee6406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:06.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee857e6640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fee6406af90 0x7fee6406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:06.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 -- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fee6406af90 msgr2=0x7fee6406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:06.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fee6406af90 0x7fee6406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 -- 172.21.15.70:0/2521248687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee8006dfc0 msgr2=0x7fee80150b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:06.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee8006dfc0 0x7fee80150b90 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fee7c000c00 tx=0x7fee7c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 -- 172.21.15.70:0/2521248687 shutdown_connections 2023-10-08T05:52:06.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7fee6406af90 0x7fee6406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee800afbe0 0x7fee800cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee80071070 0x7fee800ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 --2- 172.21.15.70:0/2521248687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee8006dfc0 0x7fee80150b90 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 -- 172.21.15.70:0/2521248687 >> 172.21.15.70:0/2521248687 conn(0x7fee8006a5d0 msgr2=0x7fee8015ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:06.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 -- 172.21.15.70:0/2521248687 shutdown_connections 2023-10-08T05:52:06.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.480+0000 7fee8824b640 1 -- 172.21.15.70:0/2521248687 wait complete. 2023-10-08T05:52:06.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:52:06.533 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:06.532+0000 7f189b811040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:52:06.631 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:06.631+0000 7fc954a32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:52:06.640 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:06.640+0000 7f189b811040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:52:06.690 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:06.689+0000 7f189b811040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:52:06.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.691+0000 7fabc361a640 1 Processor -- start 2023-10-08T05:52:06.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.691+0000 7fabc361a640 1 -- start start 2023-10-08T05:52:06.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.693+0000 7fabc361a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fabbc0ce560 0x7fabbc06f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.693+0000 7fabc361a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fabbc0cb4d0 0x7fabbc0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.693+0000 7fabc361a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabbc069d50 con 0x7fabbc0cb0f0 2023-10-08T05:52:06.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.693+0000 7fabc361a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabbc0ca8f0 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.693+0000 7fabc361a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabbc06fc00 con 0x7fabbc0ce560 2023-10-08T05:52:06.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.693+0000 7fabc1bb7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fabbc0cb4d0 0x7fabbc0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:06.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.693+0000 7fabc1bb7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fabbc0cb4d0 0x7fabbc0ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43146/0 (socket says 172.21.15.70:43146) 2023-10-08T05:52:06.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.693+0000 7fabc1bb7640 1 -- 172.21.15.70:0/2936881577 learned_addr learned my addr 172.21.15.70:0/2936881577 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:06.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.694+0000 7fabc0bb5640 1 --2- 172.21.15.70:0/2936881577 >> v2:172.21.15.181:3300/0 conn(0x7fabbc0ce560 0x7fabbc06f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:06.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.694+0000 7fabc1bb7640 1 -- 172.21.15.70:0/2936881577 >> v2:172.21.15.181:3300/0 conn(0x7fabbc0ce560 msgr2=0x7fabbc06f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:06.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.694+0000 7fabc1bb7640 1 --2- 172.21.15.70:0/2936881577 >> v2:172.21.15.181:3300/0 conn(0x7fabbc0ce560 0x7fabbc06f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.694+0000 7fabc1bb7640 1 -- 172.21.15.70:0/2936881577 >> v1:172.21.15.181:6789/0 conn(0x7fabbc0cb0f0 legacy=0x7fabbc0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:06.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.694+0000 7fabc1bb7640 1 -- 172.21.15.70:0/2936881577 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc069ff0 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.695+0000 7fabc1bb7640 1 --2- 172.21.15.70:0/2936881577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 0x7fabbc0ca3b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0cd3d0 tx=0x7fabb402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836c2eafd92b9f94 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:06.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.696+0000 7fabb3fff640 1 -- 172.21.15.70:0/2936881577 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb402ca80 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.696+0000 7fabb3fff640 1 -- 172.21.15.70:0/2936881577 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb402cbe0 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.696+0000 7fabb3fff640 1 -- 172.21.15.70:0/2936881577 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb40343f0 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.696+0000 7fabc361a640 1 -- 172.21.15.70:0/2936881577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 msgr2=0x7fabbc0ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:06.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.696+0000 7fabc361a640 1 --2- 172.21.15.70:0/2936881577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 0x7fabbc0ca3b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0cd3d0 tx=0x7fabb402bee0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.696+0000 7fabc361a640 1 -- 172.21.15.70:0/2936881577 shutdown_connections 2023-10-08T05:52:06.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.696+0000 7fabc361a640 1 --2- 172.21.15.70:0/2936881577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 0x7fabbc0ca3b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.696+0000 7fabc361a640 1 --2- 172.21.15.70:0/2936881577 >> v2:172.21.15.181:3300/0 conn(0x7fabbc0ce560 0x7fabbc06f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.696+0000 7fabc361a640 1 -- 172.21.15.70:0/2936881577 >> 172.21.15.70:0/2936881577 conn(0x7fabbc0cc2d0 msgr2=0x7fabbc0cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:06.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.697+0000 7fabc361a640 1 -- 172.21.15.70:0/2936881577 shutdown_connections 2023-10-08T05:52:06.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.697+0000 7fabc361a640 1 -- 172.21.15.70:0/2936881577 wait complete. 2023-10-08T05:52:06.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.698+0000 7fabc361a640 1 Processor -- start 2023-10-08T05:52:06.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.698+0000 7fabc361a640 1 -- start start 2023-10-08T05:52:06.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.699+0000 7fabc361a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc0cb0f0 0x7fabbc0c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.699+0000 7fabc361a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 0x7fabbc0b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.699+0000 7fabc361a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabbc0ce560 0x7fabbc0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.699+0000 7fabc361a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc14d6b0 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.699+0000 7fabc361a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc14ca30 con 0x7fabbc0cb0f0 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.699+0000 7fabc361a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc14cd10 con 0x7fabbc0ce560 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.700+0000 7fabc0bb5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 0x7fabbc0b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.700+0000 7fabc0bb5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 0x7fabbc0b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43158/0 (socket says 172.21.15.70:43158) 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.700+0000 7fabc0bb5640 1 -- 172.21.15.70:0/3659549122 learned_addr learned my addr 172.21.15.70:0/3659549122 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.700+0000 7fabc13b6640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc0cb0f0 0x7fabbc0c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.700+0000 7fabc0bb5640 1 -- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabbc0ce560 msgr2=0x7fabbc0b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.700+0000 7fabc0bb5640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabbc0ce560 0x7fabbc0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.700+0000 7fabc0bb5640 1 -- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc0cb0f0 msgr2=0x7fabbc0c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:06.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.700+0000 7fabc0bb5640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc0cb0f0 0x7fabbc0c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.700+0000 7fabc0bb5640 1 -- 172.21.15.70:0/3659549122 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb4009b30 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.700+0000 7fabc0bb5640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 0x7fabbc0b02c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fabac00d5f0 tx=0x7fabac00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:06.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.701+0000 7fabb1ffb640 1 -- 172.21.15.70:0/3659549122 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac0125f0 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.701+0000 7fabb1ffb640 1 -- 172.21.15.70:0/3659549122 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac012750 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.701+0000 7fabc361a640 1 -- 172.21.15.70:0/3659549122 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabbc14cff0 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.701+0000 7fabb1ffb640 1 -- 172.21.15.70:0/3659549122 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac012b00 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.701+0000 7fabc361a640 1 -- 172.21.15.70:0/3659549122 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabbc0b5170 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.702+0000 7fabc361a640 1 -- 172.21.15.70:0/3659549122 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabbc04e160 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.703+0000 7fabb1ffb640 1 -- 172.21.15.70:0/3659549122 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fabac012c60 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.704+0000 7fabb1ffb640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7faba006af90 0x7faba006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:06.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.704+0000 7fabb1ffb640 1 -- 172.21.15.70:0/3659549122 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fabac089fc0 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.704+0000 7fabc13b6640 1 -- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7faba006af90 msgr2=0x7faba006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:06.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.704+0000 7fabc13b6640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7faba006af90 0x7faba006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:06.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.708+0000 7fabb1ffb640 1 -- 172.21.15.70:0/3659549122 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabac054c00 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.748 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:06.748+0000 7fc954a32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:52:06.799 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:06.798+0000 7f189b811040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:06.800 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:06.800+0000 7fc954a32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:52:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.905+0000 7fabc13b6640 1 -- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7faba006af90 msgr2=0x7faba006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:06.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.905+0000 7fabc13b6640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7faba006af90 0x7faba006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:06.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.907+0000 7fabc361a640 1 -- 172.21.15.70:0/3659549122 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fabbc0c4400 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.908+0000 7fabb1ffb640 1 -- 172.21.15.70:0/3659549122 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v77) v1 ==== 80+0+473 (secure 0 0 0) 0x7fabac058220 con 0x7fabbc0cb4d0 2023-10-08T05:52:06.908 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1060036340 2023-10-09T05:50:30.831312+0000 2023-10-08T05:52:06.908 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/464684516 2023-10-09T05:50:30.831312+0000 2023-10-08T05:52:06.908 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1910914026 2023-10-09T05:50:30.831312+0000 2023-10-08T05:52:06.908 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/992174303 2023-10-09T05:50:30.831312+0000 2023-10-08T05:52:06.908 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/992174303 2023-10-09T05:50:30.831312+0000 2023-10-08T05:52:06.909 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/596247496 2023-10-09T05:50:30.831312+0000 2023-10-08T05:52:06.909 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/368598410 2023-10-09T05:50:30.831312+0000 2023-10-08T05:52:06.909 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3145795928 2023-10-09T05:50:30.831312+0000 2023-10-08T05:52:06.909 INFO:teuthology.orchestra.run.smithi070.stderr:listed 8 entries 2023-10-08T05:52:06.909 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:06.907+0000 7f189b811040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:52:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 -- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7faba006af90 msgr2=0x7faba006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7faba006af90 0x7faba006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 -- 172.21.15.70:0/3659549122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 msgr2=0x7fabbc0b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 0x7fabbc0b02c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fabac00d5f0 tx=0x7fabac00dab0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 -- 172.21.15.70:0/3659549122 shutdown_connections 2023-10-08T05:52:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7faba006af90 0x7faba006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fabbc0ce560 0x7fabbc0b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabbc0cb4d0 0x7fabbc0b02c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 --2- 172.21.15.70:0/3659549122 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fabbc0cb0f0 0x7fabbc0c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 -- 172.21.15.70:0/3659549122 >> 172.21.15.70:0/3659549122 conn(0x7fabbc0cc2d0 msgr2=0x7fabbc15a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 -- 172.21.15.70:0/3659549122 shutdown_connections 2023-10-08T05:52:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:06.910+0000 7fabc361a640 1 -- 172.21.15.70:0/3659549122 wait complete. 2023-10-08T05:52:06.913 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:06.913+0000 7fc954a32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:06.927 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T05:52:06.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:06.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T05:52:07.029 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:52:07.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:07.029 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T05:52:07.038 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:07.038+0000 7fc954a32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:52:07.073 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:52:07.073 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:07.074 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:07.073+0000 7f189b811040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:07.126 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:07.125+0000 7f189b811040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:07.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:52:07.128 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:07.128 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:07.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:52:07.184 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:52:07.184 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:07.205 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:07.205+0000 7fc954a32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:07.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:52:07.240 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:07.250 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:07.249+0000 7f189b811040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:52:07.256 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:07.256+0000 7fc954a32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:07.383 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:07.381+0000 7fc954a32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:52:07.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.445+0000 7f3d61d1d640 1 Processor -- start 2023-10-08T05:52:07.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.445+0000 7f3d61d1d640 1 -- start start 2023-10-08T05:52:07.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.446+0000 7f3d61d1d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d5c15cd20 0x7f3d5c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.446+0000 7f3d61d1d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cde60 0x7f3d5c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.446+0000 7f3d61d1d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5c06b490 con 0x7f3d5c0cb220 2023-10-08T05:52:07.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.446+0000 7f3d61d1d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c06b770 con 0x7f3d5c15cd20 2023-10-08T05:52:07.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.446+0000 7f3d61d1d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c06ba50 con 0x7f3d5c0cde60 2023-10-08T05:52:07.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.447+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cde60 0x7f3d5c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:07.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.447+0000 7f3d5b7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d5c15cd20 0x7f3d5c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:07.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.447+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cde60 0x7f3d5c0cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:47892/0 (socket says 172.21.15.70:47892) 2023-10-08T05:52:07.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.447+0000 7f3d5bfff640 1 -- 172.21.15.70:0/2688553074 learned_addr learned my addr 172.21.15.70:0/2688553074 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:07.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.448+0000 7f3d5b7fe640 1 -- 172.21.15.70:0/2688553074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cde60 msgr2=0x7f3d5c0cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:07.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.448+0000 7f3d5b7fe640 1 --2- 172.21.15.70:0/2688553074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cde60 0x7f3d5c0cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.448+0000 7f3d5b7fe640 1 -- 172.21.15.70:0/2688553074 >> v1:172.21.15.181:6789/0 conn(0x7f3d5c0cb220 legacy=0x7f3d5c0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:07.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.448+0000 7f3d5b7fe640 1 -- 172.21.15.70:0/2688553074 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c06bd30 con 0x7f3d5c15cd20 2023-10-08T05:52:07.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.449+0000 7f3d5b7fe640 1 --2- 172.21.15.70:0/2688553074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5c15cd20 0x7f3d5c0cee40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0097a0 tx=0x7f3d4c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1930e2c34f18c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:07.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.449+0000 7f3d5affd640 1 -- 172.21.15.70:0/2688553074 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c02d040 con 0x7f3d5c15cd20 2023-10-08T05:52:07.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.449+0000 7f3d5affd640 1 -- 172.21.15.70:0/2688553074 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c031810 con 0x7f3d5c15cd20 2023-10-08T05:52:07.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.449+0000 7f3d5affd640 1 -- 172.21.15.70:0/2688553074 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c031ba0 con 0x7f3d5c15cd20 2023-10-08T05:52:07.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.450+0000 7f3d61d1d640 1 -- 172.21.15.70:0/2688553074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5c15cd20 msgr2=0x7f3d5c0cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:07.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.450+0000 7f3d61d1d640 1 --2- 172.21.15.70:0/2688553074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5c15cd20 0x7f3d5c0cee40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0097a0 tx=0x7f3d4c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.450+0000 7f3d61d1d640 1 -- 172.21.15.70:0/2688553074 shutdown_connections 2023-10-08T05:52:07.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.450+0000 7f3d61d1d640 1 --2- 172.21.15.70:0/2688553074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cde60 0x7f3d5c0cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.450+0000 7f3d61d1d640 1 --2- 172.21.15.70:0/2688553074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5c15cd20 0x7f3d5c0cee40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.450+0000 7f3d61d1d640 1 -- 172.21.15.70:0/2688553074 >> 172.21.15.70:0/2688553074 conn(0x7f3d5c069f30 msgr2=0x7f3d5c06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:07.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.450+0000 7f3d61d1d640 1 -- 172.21.15.70:0/2688553074 shutdown_connections 2023-10-08T05:52:07.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.450+0000 7f3d61d1d640 1 -- 172.21.15.70:0/2688553074 wait complete. 2023-10-08T05:52:07.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.452+0000 7f3d61d1d640 1 Processor -- start 2023-10-08T05:52:07.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.452+0000 7f3d61d1d640 1 -- start start 2023-10-08T05:52:07.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d61d1d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cb220 0x7f3d5c14ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d61d1d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5c0cde60 0x7f3d5c0c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d61d1d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c15cd20 0x7f3d5c15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d61d1d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0c9930 con 0x7f3d5c15cd20 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d61d1d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0c9c10 con 0x7f3d5c0cde60 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d61d1d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0c9ef0 con 0x7f3d5c0cb220 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c15cd20 0x7f3d5c15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c15cd20 0x7f3d5c15e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43160/0 (socket says 172.21.15.70:43160) 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d5bfff640 1 -- 172.21.15.70:0/853566926 learned_addr learned my addr 172.21.15.70:0/853566926 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d5b7fe640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cb220 0x7f3d5c14ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.453+0000 7f3d53fff640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5c0cde60 0x7f3d5c0c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d5bfff640 1 -- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cb220 msgr2=0x7f3d5c14ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d5bfff640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cb220 0x7f3d5c14ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d5bfff640 1 -- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5c0cde60 msgr2=0x7f3d5c0c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d5bfff640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5c0cde60 0x7f3d5c0c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d5bfff640 1 -- 172.21.15.70:0/853566926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c009400 con 0x7f3d5c15cd20 2023-10-08T05:52:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d5bfff640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c15cd20 0x7f3d5c15e880 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400efc0 tx=0x7f3d5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d58ff9640 1 -- 172.21.15.70:0/853566926 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5400cd20 con 0x7f3d5c15cd20 2023-10-08T05:52:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d58ff9640 1 -- 172.21.15.70:0/853566926 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5400f040 con 0x7f3d5c15cd20 2023-10-08T05:52:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d61d1d640 1 -- 172.21.15.70:0/853566926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c0ca1d0 con 0x7f3d5c15cd20 2023-10-08T05:52:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d58ff9640 1 -- 172.21.15.70:0/853566926 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d54003c20 con 0x7f3d5c15cd20 2023-10-08T05:52:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.454+0000 7f3d61d1d640 1 -- 172.21.15.70:0/853566926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c0ca790 con 0x7f3d5c15cd20 2023-10-08T05:52:07.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.455+0000 7f3d61d1d640 1 -- 172.21.15.70:0/853566926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c066d80 con 0x7f3d5c15cd20 2023-10-08T05:52:07.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.456+0000 7f3d58ff9640 1 -- 172.21.15.70:0/853566926 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f3d54004030 con 0x7f3d5c15cd20 2023-10-08T05:52:07.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.458+0000 7f3d58ff9640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f3d3c06af90 0x7f3d3c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.458+0000 7f3d58ff9640 1 -- 172.21.15.70:0/853566926 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3d5408a210 con 0x7f3d5c15cd20 2023-10-08T05:52:07.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.458+0000 7f3d5b7fe640 1 -- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f3d3c06af90 msgr2=0x7f3d3c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:07.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.458+0000 7f3d5b7fe640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f3d3c06af90 0x7f3d3c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.462+0000 7f3d58ff9640 1 -- 172.21.15.70:0/853566926 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d54054d40 con 0x7f3d5c15cd20 2023-10-08T05:52:07.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.659+0000 7f3d5b7fe640 1 -- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f3d3c06af90 msgr2=0x7f3d3c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:07.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.659+0000 7f3d5b7fe640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f3d3c06af90 0x7f3d3c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:07.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.687+0000 7f3d61d1d640 1 -- 172.21.15.70:0/853566926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d5c0b6970 con 0x7f3d5c15cd20 2023-10-08T05:52:07.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.688+0000 7f3d58ff9640 1 -- 172.21.15.70:0/853566926 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f3d54058360 con 0x7f3d5c15cd20 2023-10-08T05:52:07.689 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:07.689 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1126961048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1126961048},{"type":"v1","addr":"172.21.15.181:6836","nonce":1126961048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3993068621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3993068621},{"type":"v1","addr":"172.21.15.181:6837","nonce":3993068621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/28904251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":28904251},{"type":"v1","addr":"172.21.15.70:6833","nonce":28904251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:50:16.144118+0000","modified":"2023-10-08T05:50:19.194284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1337581673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1337581673},{"type":"v1","addr":"172.21.15.70:6835","nonce":1337581673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-08T05:52:07.689 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 25 2023-10-08T05:52:07.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 -- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f3d3c06af90 msgr2=0x7f3d3c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:07.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f3d3c06af90 0x7f3d3c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 -- 172.21.15.70:0/853566926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c15cd20 msgr2=0x7f3d5c15e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c15cd20 0x7f3d5c15e880 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400efc0 tx=0x7f3d5400c490 comp rx=0 tx=0).stop 2023-10-08T05:52:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 -- 172.21.15.70:0/853566926 shutdown_connections 2023-10-08T05:52:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f3d3c06af90 0x7f3d3c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d5c15cd20 0x7f3d5c15e880 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d5c0cde60 0x7f3d5c0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 --2- 172.21.15.70:0/853566926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d5c0cb220 0x7f3d5c14ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 -- 172.21.15.70:0/853566926 >> 172.21.15.70:0/853566926 conn(0x7f3d5c069f30 msgr2=0x7f3d5c0b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 -- 172.21.15.70:0/853566926 shutdown_connections 2023-10-08T05:52:07.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.691+0000 7f3d61d1d640 1 -- 172.21.15.70:0/853566926 wait complete. 2023-10-08T05:52:07.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:07.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.953+0000 7feef721a640 1 Processor -- start 2023-10-08T05:52:07.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.953+0000 7feef721a640 1 -- start start 2023-10-08T05:52:07.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.954+0000 7feef721a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feef00c90b0 0x7feef00cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.954+0000 7feef721a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feef00cb3b0 0x7feef015e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.954+0000 7feef721a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef015b2b0 con 0x7feef015ab40 2023-10-08T05:52:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.954+0000 7feef721a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef015bf30 con 0x7feef00c90b0 2023-10-08T05:52:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.954+0000 7feef721a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef00cb820 con 0x7feef00cb3b0 2023-10-08T05:52:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.954+0000 7feeeffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feef00c90b0 0x7feef00cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.954+0000 7feef4fb6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feef015ab40 0x7feef00ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:32862/0 (socket says 172.21.15.70:32862) 2023-10-08T05:52:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.954+0000 7feef4fb6640 1 -- 172.21.15.70:0/4159379276 learned_addr learned my addr 172.21.15.70:0/4159379276 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.955+0000 7feef57b7640 1 --2- 172.21.15.70:0/4159379276 >> v2:172.21.15.181:3300/0 conn(0x7feef00cb3b0 0x7feef015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:07.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.955+0000 7feeeffff640 1 -- 172.21.15.70:0/4159379276 >> v2:172.21.15.181:3300/0 conn(0x7feef00cb3b0 msgr2=0x7feef015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:07.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.955+0000 7feeeffff640 1 --2- 172.21.15.70:0/4159379276 >> v2:172.21.15.181:3300/0 conn(0x7feef00cb3b0 0x7feef015e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.955+0000 7feeeffff640 1 -- 172.21.15.70:0/4159379276 >> v1:172.21.15.70:6789/0 conn(0x7feef015ab40 legacy=0x7feef00ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:07.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.955+0000 7feeeffff640 1 -- 172.21.15.70:0/4159379276 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef015b590 con 0x7feef00c90b0 2023-10-08T05:52:07.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.955+0000 7feef57b7640 1 --2- 172.21.15.70:0/4159379276 >> v2:172.21.15.181:3300/0 conn(0x7feef00cb3b0 0x7feef015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:07.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.956+0000 7feeeffff640 1 --2- 172.21.15.70:0/4159379276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 0x7feef00cae70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7feef00b0ec0 tx=0x7feedc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4bd7f0716eda86 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:07.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.956+0000 7feeef7fe640 1 -- 172.21.15.70:0/4159379276 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc031830 con 0x7feef00c90b0 2023-10-08T05:52:07.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.956+0000 7feeef7fe640 1 -- 172.21.15.70:0/4159379276 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc031990 con 0x7feef00c90b0 2023-10-08T05:52:07.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.956+0000 7feeef7fe640 1 -- 172.21.15.70:0/4159379276 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feedc031d20 con 0x7feef00c90b0 2023-10-08T05:52:07.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.957+0000 7feef721a640 1 -- 172.21.15.70:0/4159379276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 msgr2=0x7feef00cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:07.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.957+0000 7feef721a640 1 --2- 172.21.15.70:0/4159379276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 0x7feef00cae70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7feef00b0ec0 tx=0x7feedc02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.957+0000 7feef721a640 1 -- 172.21.15.70:0/4159379276 shutdown_connections 2023-10-08T05:52:07.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.957+0000 7feef721a640 1 --2- 172.21.15.70:0/4159379276 >> v2:172.21.15.181:3300/0 conn(0x7feef00cb3b0 0x7feef015e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.957+0000 7feef721a640 1 --2- 172.21.15.70:0/4159379276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 0x7feef00cae70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.957+0000 7feef721a640 1 -- 172.21.15.70:0/4159379276 >> 172.21.15.70:0/4159379276 conn(0x7feef006a380 msgr2=0x7feef00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:07.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.957+0000 7feef721a640 1 -- 172.21.15.70:0/4159379276 shutdown_connections 2023-10-08T05:52:07.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.957+0000 7feef721a640 1 -- 172.21.15.70:0/4159379276 wait complete. 2023-10-08T05:52:07.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.960+0000 7feef721a640 1 Processor -- start 2023-10-08T05:52:07.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.960+0000 7feef721a640 1 -- start start 2023-10-08T05:52:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.961+0000 7feef721a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 0x7feef00b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.961+0000 7feef721a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef00cb3b0 0x7feef00b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.961+0000 7feef721a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feef015ab40 0x7feef0159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.961+0000 7feef721a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef0158ce0 con 0x7feef00c90b0 2023-10-08T05:52:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.961+0000 7feef721a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef0158130 con 0x7feef00cb3b0 2023-10-08T05:52:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.961+0000 7feef721a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef0158340 con 0x7feef015ab40 2023-10-08T05:52:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef4fb6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 0x7feef00b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef4fb6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 0x7feef00b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43184/0 (socket says 172.21.15.70:43184) 2023-10-08T05:52:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef4fb6640 1 -- 172.21.15.70:0/3209581538 learned_addr learned my addr 172.21.15.70:0/3209581538 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feeeffff640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef00cb3b0 0x7feef00b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef57b7640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feef015ab40 0x7feef0159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef4fb6640 1 -- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feef015ab40 msgr2=0x7feef0159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef4fb6640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feef015ab40 0x7feef0159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef4fb6640 1 -- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef00cb3b0 msgr2=0x7feef00b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef4fb6640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef00cb3b0 0x7feef00b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef4fb6640 1 -- 172.21.15.70:0/3209581538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc009400 con 0x7feef00c90b0 2023-10-08T05:52:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef57b7640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feef015ab40 0x7feef0159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feef4fb6640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 0x7feef00b2d10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7feee400c480 tx=0x7feee400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.962+0000 7feeed7fa640 1 -- 172.21.15.70:0/3209581538 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4005ce0 con 0x7feef00c90b0 2023-10-08T05:52:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.963+0000 7feeed7fa640 1 -- 172.21.15.70:0/3209581538 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4005e40 con 0x7feef00c90b0 2023-10-08T05:52:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.963+0000 7feef721a640 1 -- 172.21.15.70:0/3209581538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef0158620 con 0x7feef00c90b0 2023-10-08T05:52:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.963+0000 7feeed7fa640 1 -- 172.21.15.70:0/3209581538 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40072f0 con 0x7feef00c90b0 2023-10-08T05:52:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.963+0000 7feef721a640 1 -- 172.21.15.70:0/3209581538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef0159100 con 0x7feef00c90b0 2023-10-08T05:52:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.964+0000 7feef721a640 1 -- 172.21.15.70:0/3209581538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef0158f50 con 0x7feef00c90b0 2023-10-08T05:52:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.965+0000 7feeed7fa640 1 -- 172.21.15.70:0/3209581538 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7feee4007450 con 0x7feef00c90b0 2023-10-08T05:52:07.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.966+0000 7feeed7fa640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7feed406afc0 0x7feed406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:07.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.966+0000 7feeed7fa640 1 -- 172.21.15.70:0/3209581538 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6547+0+0 (secure 0 0 0) 0x7feee4088f00 con 0x7feef00c90b0 2023-10-08T05:52:07.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.966+0000 7feeeffff640 1 -- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7feed406afc0 msgr2=0x7feed406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:07.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.967+0000 7feeeffff640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7feed406afc0 0x7feed406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:07.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:07.970+0000 7feeed7fa640 1 -- 172.21.15.70:0/3209581538 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feee4053c70 con 0x7feef00c90b0 2023-10-08T05:52:08.082 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.081+0000 7f189b811040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:52:08.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.161+0000 7feef721a640 1 -- 172.21.15.70:0/3209581538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feef0067490 con 0x7feef00c90b0 2023-10-08T05:52:08.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.163+0000 7feeed7fa640 1 -- 172.21.15.70:0/3209581538 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+17563 (secure 0 0 0) 0x7feee4017020 con 0x7feef00c90b0 2023-10-08T05:52:08.164 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:08.164 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":77,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:06.414571+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:50:14.005619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-08T05:50:15.163925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1060036340":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/464684516":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/1910914026":"2023-10-09T05:50:30.831312+0000","172.21.15.181:6833/992174303":"2023-10-09T05:50:30.831312+0000","172.21.15.181:6832/992174303":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/596247496":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/368598410":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/3145795928":"2023-10-09T05:50:30.831312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:08.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.165+0000 7feef721a640 1 -- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7feed406afc0 msgr2=0x7feed406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:08.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.165+0000 7feef721a640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7feed406afc0 0x7feed406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.165+0000 7feef721a640 1 -- 172.21.15.70:0/3209581538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 msgr2=0x7feef00b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:08.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.165+0000 7feef721a640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 0x7feef00b2d10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7feee400c480 tx=0x7feee400c940 comp rx=0 tx=0).stop 2023-10-08T05:52:08.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.166+0000 7feef721a640 1 -- 172.21.15.70:0/3209581538 shutdown_connections 2023-10-08T05:52:08.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.166+0000 7feef721a640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7feed406afc0 0x7feed406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.166+0000 7feef721a640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feef015ab40 0x7feef0159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.166+0000 7feef721a640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feef00cb3b0 0x7feef00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.166+0000 7feef721a640 1 --2- 172.21.15.70:0/3209581538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feef00c90b0 0x7feef00b2d10 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.166+0000 7feef721a640 1 -- 172.21.15.70:0/3209581538 >> 172.21.15.70:0/3209581538 conn(0x7feef006a380 msgr2=0x7feef00c9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:08.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.166+0000 7feef721a640 1 -- 172.21.15.70:0/3209581538 shutdown_connections 2023-10-08T05:52:08.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.166+0000 7feef721a640 1 -- 172.21.15.70:0/3209581538 wait complete. 2023-10-08T05:52:08.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:50:14.005619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:50:15.163925+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:52:08.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:52:08.223 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:08.222+0000 7fc954a32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:52:08.378 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.377+0000 7f189b811040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:52:08.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.385+0000 7f49333a5640 1 Processor -- start 2023-10-08T05:52:08.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.386+0000 7f49333a5640 1 -- start start 2023-10-08T05:52:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.387+0000 7f49333a5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f492c0d1120 0x7f492c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.387+0000 7f49333a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c15dcd0 0x7f492c1610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.387+0000 7f49333a5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c0d0a80 con 0x7f492c0d1500 2023-10-08T05:52:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.387+0000 7f49333a5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c161620 con 0x7f492c0d1120 2023-10-08T05:52:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.387+0000 7f49333a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0b1fe0 con 0x7f492c15dcd0 2023-10-08T05:52:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.387+0000 7f49323a3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f492c0d1120 0x7f492c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:08.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.387+0000 7f49323a3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f492c0d1120 0x7f492c0d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43188/0 (socket says 172.21.15.70:43188) 2023-10-08T05:52:08.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.387+0000 7f49323a3640 1 -- 172.21.15.70:0/1694869852 learned_addr learned my addr 172.21.15.70:0/1694869852 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:08.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.388+0000 7f4932ba4640 1 --2- 172.21.15.70:0/1694869852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c15dcd0 0x7f492c1610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:08.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.388+0000 7f49323a3640 1 -- 172.21.15.70:0/1694869852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c15dcd0 msgr2=0x7f492c1610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:08.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.388+0000 7f49323a3640 1 --2- 172.21.15.70:0/1694869852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c15dcd0 0x7f492c1610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.388+0000 7f49323a3640 1 -- 172.21.15.70:0/1694869852 >> v1:172.21.15.181:6789/0 conn(0x7f492c0d1500 legacy=0x7f492c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:08.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.388+0000 7f49323a3640 1 -- 172.21.15.70:0/1694869852 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0b4580 con 0x7f492c0d1120 2023-10-08T05:52:08.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.389+0000 7f49323a3640 1 --2- 172.21.15.70:0/1694869852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0d1120 0x7f492c0d04a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4928009850 tx=0x7f492802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb9b20393c74957 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:08.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.389+0000 7f49313a1640 1 -- 172.21.15.70:0/1694869852 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928004230 con 0x7f492c0d1120 2023-10-08T05:52:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.389+0000 7f49313a1640 1 -- 172.21.15.70:0/1694869852 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928004390 con 0x7f492c0d1120 2023-10-08T05:52:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.389+0000 7f49313a1640 1 -- 172.21.15.70:0/1694869852 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928031830 con 0x7f492c0d1120 2023-10-08T05:52:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.390+0000 7f49333a5640 1 -- 172.21.15.70:0/1694869852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0d1120 msgr2=0x7f492c0d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.390+0000 7f49333a5640 1 --2- 172.21.15.70:0/1694869852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0d1120 0x7f492c0d04a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4928009850 tx=0x7f492802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.390+0000 7f49333a5640 1 -- 172.21.15.70:0/1694869852 shutdown_connections 2023-10-08T05:52:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.390+0000 7f49333a5640 1 --2- 172.21.15.70:0/1694869852 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c15dcd0 0x7f492c1610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.390+0000 7f49333a5640 1 --2- 172.21.15.70:0/1694869852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0d1120 0x7f492c0d04a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.390+0000 7f49333a5640 1 -- 172.21.15.70:0/1694869852 >> 172.21.15.70:0/1694869852 conn(0x7f492c0d2960 msgr2=0x7f492c0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:08.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.390+0000 7f49333a5640 1 -- 172.21.15.70:0/1694869852 shutdown_connections 2023-10-08T05:52:08.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.390+0000 7f49333a5640 1 -- 172.21.15.70:0/1694869852 wait complete. 2023-10-08T05:52:08.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.392+0000 7f49333a5640 1 Processor -- start 2023-10-08T05:52:08.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.392+0000 7f49333a5640 1 -- start start 2023-10-08T05:52:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f49333a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0d1120 0x7f492c15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f49333a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c0d1500 0x7f492c14ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f49333a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c15dcd0 0x7f492c0b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f49333a5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c14f370 con 0x7f492c15dcd0 2023-10-08T05:52:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f49333a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0b63a0 con 0x7f492c0d1500 2023-10-08T05:52:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f49333a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0c7430 con 0x7f492c0d1120 2023-10-08T05:52:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f4932ba4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c15dcd0 0x7f492c0b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f4932ba4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c15dcd0 0x7f492c0b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43204/0 (socket says 172.21.15.70:43204) 2023-10-08T05:52:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f4932ba4640 1 -- 172.21.15.70:0/2812926610 learned_addr learned my addr 172.21.15.70:0/2812926610 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f49323a3640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0d1120 0x7f492c15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.393+0000 7f4931ba2640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c0d1500 0x7f492c14ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f4932ba4640 1 -- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0d1120 msgr2=0x7f492c15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f4932ba4640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0d1120 0x7f492c15b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f4932ba4640 1 -- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c0d1500 msgr2=0x7f492c14ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f4932ba4640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c0d1500 0x7f492c14ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f4932ba4640 1 -- 172.21.15.70:0/2812926610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928009460 con 0x7f492c15dcd0 2023-10-08T05:52:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f4931ba2640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c0d1500 0x7f492c14ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:52:08.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f4932ba4640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c15dcd0 0x7f492c0b5e60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4924007e60 tx=0x7f492400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f4922ffd640 1 -- 172.21.15.70:0/2812926610 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924014420 con 0x7f492c15dcd0 2023-10-08T05:52:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f4922ffd640 1 -- 172.21.15.70:0/2812926610 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49240145a0 con 0x7f492c15dcd0 2023-10-08T05:52:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f49333a5640 1 -- 172.21.15.70:0/2812926610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c0c7710 con 0x7f492c15dcd0 2023-10-08T05:52:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f4922ffd640 1 -- 172.21.15.70:0/2812926610 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924014950 con 0x7f492c15dcd0 2023-10-08T05:52:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.394+0000 7f49333a5640 1 -- 172.21.15.70:0/2812926610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c0c7cd0 con 0x7f492c15dcd0 2023-10-08T05:52:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.396+0000 7f49333a5640 1 -- 172.21.15.70:0/2812926610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c074150 con 0x7f492c15dcd0 2023-10-08T05:52:08.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.396+0000 7f4922ffd640 1 -- 172.21.15.70:0/2812926610 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 26) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f4924014ab0 con 0x7f492c15dcd0 2023-10-08T05:52:08.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.398+0000 7f4922ffd640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f491006af90 0x7f491006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.398+0000 7f4922ffd640 1 -- 172.21.15.70:0/2812926610 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f492408a4d0 con 0x7f492c15dcd0 2023-10-08T05:52:08.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.398+0000 7f49323a3640 1 -- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f491006af90 msgr2=0x7f491006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3803301866 2023-10-08T05:52:08.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.398+0000 7f49323a3640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f491006af90 0x7f491006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:08.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.402+0000 7f4922ffd640 1 -- 172.21.15.70:0/2812926610 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4924055080 con 0x7f492c15dcd0 2023-10-08T05:52:08.436 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.435+0000 7f189b811040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:52:08.536 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:08.536+0000 7fc954a32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:52:08.551 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.550+0000 7f189b811040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:52:08.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.573+0000 7f4922ffd640 1 -- 172.21.15.70:0/2812926610 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 27) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4924055450 con 0x7f492c15dcd0 2023-10-08T05:52:08.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.574+0000 7f4922ffd640 1 -- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f491006af90 msgr2=0x7f491006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:08.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.574+0000 7f4922ffd640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f491006af90 0x7f491006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.592+0000 7f49333a5640 1 -- 172.21.15.70:0/2812926610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f492c06a7a0 con 0x7f492c15dcd0 2023-10-08T05:52:08.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.593+0000 7f4922ffd640 1 -- 172.21.15.70:0/2812926610 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4924087850 con 0x7f492c15dcd0 2023-10-08T05:52:08.593 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:52:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.595+0000 7f49333a5640 1 -- 172.21.15.70:0/2812926610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c15dcd0 msgr2=0x7f492c0b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.595+0000 7f49333a5640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c15dcd0 0x7f492c0b5e60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4924007e60 tx=0x7f492400e4b0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.595+0000 7f49333a5640 1 -- 172.21.15.70:0/2812926610 shutdown_connections 2023-10-08T05:52:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.595+0000 7f49333a5640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:6832/3803301866,v1:172.21.15.181:6833/3803301866] conn(0x7f491006af90 0x7f491006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.595+0000 7f49333a5640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c15dcd0 0x7f492c0b5e60 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.595+0000 7f49333a5640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c0d1500 0x7f492c14ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.595+0000 7f49333a5640 1 --2- 172.21.15.70:0/2812926610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0d1120 0x7f492c15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.595+0000 7f49333a5640 1 -- 172.21.15.70:0/2812926610 >> 172.21.15.70:0/2812926610 conn(0x7f492c0d2960 msgr2=0x7f492c0cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:08.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.596+0000 7f49333a5640 1 -- 172.21.15.70:0/2812926610 shutdown_connections 2023-10-08T05:52:08.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.596+0000 7f49333a5640 1 -- 172.21.15.70:0/2812926610 wait complete. 2023-10-08T05:52:08.603 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:08.603+0000 7fc954a32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:52:08.612 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:52:08.636 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.635+0000 7f181e299640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:08.636 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.635+0000 7f181e299640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:08.636 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.635+0000 7f181e299640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:08.637 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.635+0000 7f181e299640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:08.637 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.635+0000 7f181e299640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:08.637 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.635+0000 7f1820a9e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:08.637 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.635+0000 7f1820a9e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:08.637 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.635+0000 7f1820a9e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:08.637 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.635+0000 7f1820a9e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:08.637 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:08.635+0000 7f1820a9e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:08.735 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:08.734+0000 7fc954a32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:52:08.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.857+0000 7f86dd77c640 1 Processor -- start 2023-10-08T05:52:08.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.858+0000 7f86dd77c640 1 -- start start 2023-10-08T05:52:08.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.859+0000 7f86dd77c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86d80cb150 0x7f86d80cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.859+0000 7f86dd77c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86d80b1f20 0x7f86d80b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.859+0000 7f86dd77c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d806e1e0 con 0x7f86d8161690 2023-10-08T05:52:08.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.859+0000 7f86dd77c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d80cbb10 con 0x7f86d80b1f20 2023-10-08T05:52:08.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.859+0000 7f86dd77c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d806cc50 con 0x7f86d80cb150 2023-10-08T05:52:08.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.860+0000 7f86d6ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f86d8161690 0x7f86d806dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:32864/0 (socket says 172.21.15.70:32864) 2023-10-08T05:52:08.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.860+0000 7f86d6ffd640 1 -- 172.21.15.70:0/2013202845 learned_addr learned my addr 172.21.15.70:0/2013202845 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:08.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.860+0000 7f86d77fe640 1 --2- 172.21.15.70:0/2013202845 >> v2:172.21.15.70:3300/0 conn(0x7f86d80b1f20 0x7f86d80b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:08.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.860+0000 7f86cffff640 1 --2- 172.21.15.70:0/2013202845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86d80cb150 0x7f86d80cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:08.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.860+0000 7f86d67fc640 1 -- 172.21.15.70:0/2013202845 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520229080 0 0) 0x7f86d806e1e0 con 0x7f86d8161690 2023-10-08T05:52:08.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.860+0000 7f86d67fc640 1 -- 172.21.15.70:0/2013202845 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c4003520 con 0x7f86d8161690 2023-10-08T05:52:08.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.860+0000 7f86d77fe640 1 -- 172.21.15.70:0/2013202845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86d80cb150 msgr2=0x7f86d80cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.860+0000 7f86d77fe640 1 --2- 172.21.15.70:0/2013202845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86d80cb150 0x7f86d80cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.860+0000 7f86d77fe640 1 -- 172.21.15.70:0/2013202845 >> v1:172.21.15.70:6789/0 conn(0x7f86d8161690 legacy=0x7f86d806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.860+0000 7f86d77fe640 1 -- 172.21.15.70:0/2013202845 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d806cf30 con 0x7f86d80b1f20 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.861+0000 7f86d77fe640 1 --2- 172.21.15.70:0/2013202845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80b1f20 0x7f86d80b2370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009d90 tx=0x7f86d002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8660246e0a71272d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.861+0000 7f86d67fc640 1 -- 172.21.15.70:0/2013202845 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d002cc50 con 0x7f86d80b1f20 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.861+0000 7f86d67fc640 1 -- 172.21.15.70:0/2013202845 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d002cdb0 con 0x7f86d80b1f20 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.861+0000 7f86d67fc640 1 -- 172.21.15.70:0/2013202845 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0034640 con 0x7f86d80b1f20 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.862+0000 7f86dd77c640 1 -- 172.21.15.70:0/2013202845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80b1f20 msgr2=0x7f86d80b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.862+0000 7f86dd77c640 1 --2- 172.21.15.70:0/2013202845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80b1f20 0x7f86d80b2370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009d90 tx=0x7f86d002bea0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.862+0000 7f86dd77c640 1 -- 172.21.15.70:0/2013202845 shutdown_connections 2023-10-08T05:52:08.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.862+0000 7f86dd77c640 1 --2- 172.21.15.70:0/2013202845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80b1f20 0x7f86d80b2370 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.862+0000 7f86dd77c640 1 --2- 172.21.15.70:0/2013202845 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86d80cb150 0x7f86d80cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.862+0000 7f86dd77c640 1 -- 172.21.15.70:0/2013202845 >> 172.21.15.70:0/2013202845 conn(0x7f86d806a5d0 msgr2=0x7f86d80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:08.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.862+0000 7f86dd77c640 1 -- 172.21.15.70:0/2013202845 shutdown_connections 2023-10-08T05:52:08.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.862+0000 7f86dd77c640 1 -- 172.21.15.70:0/2013202845 wait complete. 2023-10-08T05:52:08.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.864+0000 7f86dd77c640 1 Processor -- start 2023-10-08T05:52:08.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.864+0000 7f86dd77c640 1 -- start start 2023-10-08T05:52:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86dd77c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86d80b1f20 0x7f86d815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86dd77c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80cb150 0x7f86d80c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86dd77c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86d8161690 0x7f86d80c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86dd77c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d8162a40 con 0x7f86d80cb150 2023-10-08T05:52:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86dd77c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d815f5f0 con 0x7f86d8161690 2023-10-08T05:52:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86dd77c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d8161dc0 con 0x7f86d80b1f20 2023-10-08T05:52:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86cffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80cb150 0x7f86d80c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86cffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80cb150 0x7f86d80c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43220/0 (socket says 172.21.15.70:43220) 2023-10-08T05:52:08.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86cffff640 1 -- 172.21.15.70:0/3483708515 learned_addr learned my addr 172.21.15.70:0/3483708515 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86cffff640 1 -- 172.21.15.70:0/3483708515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86d80b1f20 msgr2=0x7f86d815c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86cffff640 1 --2- 172.21.15.70:0/3483708515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86d80b1f20 0x7f86d815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86cffff640 1 -- 172.21.15.70:0/3483708515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86d8161690 msgr2=0x7f86d80c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86cffff640 1 --2- 172.21.15.70:0/3483708515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86d8161690 0x7f86d80c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86cffff640 1 -- 172.21.15.70:0/3483708515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d0009a70 con 0x7f86d80cb150 2023-10-08T05:52:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.865+0000 7f86cffff640 1 --2- 172.21.15.70:0/3483708515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80cb150 0x7f86d80c49b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f86c000d800 tx=0x7f86c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.866+0000 7f86cf7fe640 1 -- 172.21.15.70:0/3483708515 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000b6c0 con 0x7f86d80cb150 2023-10-08T05:52:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.866+0000 7f86cf7fe640 1 -- 172.21.15.70:0/3483708515 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c000b820 con 0x7f86d80cb150 2023-10-08T05:52:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.866+0000 7f86cf7fe640 1 -- 172.21.15.70:0/3483708515 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000bbb0 con 0x7f86d80cb150 2023-10-08T05:52:08.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.866+0000 7f86dd77c640 1 -- 172.21.15.70:0/3483708515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d81620a0 con 0x7f86d80cb150 2023-10-08T05:52:08.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.866+0000 7f86dd77c640 1 -- 172.21.15.70:0/3483708515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d8162660 con 0x7f86d80cb150 2023-10-08T05:52:08.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.867+0000 7f86dd77c640 1 -- 172.21.15.70:0/3483708515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d8162cb0 con 0x7f86d80cb150 2023-10-08T05:52:08.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.868+0000 7f86cf7fe640 1 -- 172.21.15.70:0/3483708515 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 27) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f86c000bd10 con 0x7f86d80cb150 2023-10-08T05:52:08.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.869+0000 7f86cf7fe640 1 -- 172.21.15.70:0/3483708515 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f86c0089b20 con 0x7f86d80cb150 2023-10-08T05:52:08.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:08.874+0000 7f86cf7fe640 1 -- 172.21.15.70:0/3483708515 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f86c0054ac0 con 0x7f86d80cb150 2023-10-08T05:52:09.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.065+0000 7f86dd77c640 1 -- 172.21.15.70:0/3483708515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f86d804e160 con 0x7f86d80cb150 2023-10-08T05:52:09.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.066+0000 7f86cf7fe640 1 -- 172.21.15.70:0/3483708515 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f86c00580e0 con 0x7f86d80cb150 2023-10-08T05:52:09.066 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:52:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.068+0000 7f86dd77c640 1 -- 172.21.15.70:0/3483708515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80cb150 msgr2=0x7f86d80c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.068+0000 7f86dd77c640 1 --2- 172.21.15.70:0/3483708515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80cb150 0x7f86d80c49b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f86c000d800 tx=0x7f86c000dcc0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.068+0000 7f86dd77c640 1 -- 172.21.15.70:0/3483708515 shutdown_connections 2023-10-08T05:52:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.068+0000 7f86dd77c640 1 --2- 172.21.15.70:0/3483708515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86d8161690 0x7f86d80c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.068+0000 7f86dd77c640 1 --2- 172.21.15.70:0/3483708515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d80cb150 0x7f86d80c49b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.068+0000 7f86dd77c640 1 --2- 172.21.15.70:0/3483708515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86d80b1f20 0x7f86d815c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.068+0000 7f86dd77c640 1 -- 172.21.15.70:0/3483708515 >> 172.21.15.70:0/3483708515 conn(0x7f86d806a5d0 msgr2=0x7f86d80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.069+0000 7f86dd77c640 1 -- 172.21.15.70:0/3483708515 shutdown_connections 2023-10-08T05:52:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.069+0000 7f86dd77c640 1 -- 172.21.15.70:0/3483708515 wait complete. 2023-10-08T05:52:09.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:52:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.286+0000 7f73b3fff640 1 Processor -- start 2023-10-08T05:52:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.287+0000 7f73b3fff640 1 -- start start 2023-10-08T05:52:09.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.288+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415a820 0x7f73b40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.288+0000 7f73b3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73b40ca680 0x7f73b40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.288+0000 7f73b3fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b40cab60 con 0x7f73b415ac00 2023-10-08T05:52:09.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.288+0000 7f73b3fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b40cb960 con 0x7f73b40ca680 2023-10-08T05:52:09.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.288+0000 7f73b3fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b40cbdf0 con 0x7f73b415a820 2023-10-08T05:52:09.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.289+0000 7f73abfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73b40ca680 0x7f73b40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.289+0000 7f73abfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73b40ca680 0x7f73b40b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43228/0 (socket says 172.21.15.70:43228) 2023-10-08T05:52:09.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.289+0000 7f73abfff640 1 -- 172.21.15.70:0/320539373 learned_addr learned my addr 172.21.15.70:0/320539373 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:09.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.289+0000 7f73b2ffd640 1 --2- 172.21.15.70:0/320539373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415a820 0x7f73b40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.289+0000 7f73abfff640 1 -- 172.21.15.70:0/320539373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415a820 msgr2=0x7f73b40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:09.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.289+0000 7f73abfff640 1 --2- 172.21.15.70:0/320539373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415a820 0x7f73b40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.289+0000 7f73abfff640 1 -- 172.21.15.70:0/320539373 >> v1:172.21.15.181:6789/0 conn(0x7f73b415ac00 legacy=0x7f73b40c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:09.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.289+0000 7f73abfff640 1 -- 172.21.15.70:0/320539373 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b40cb030 con 0x7f73b40ca680 2023-10-08T05:52:09.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.290+0000 7f73abfff640 1 --2- 172.21.15.70:0/320539373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40ca680 0x7f73b40b04b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f739c009870 tx=0x7f739c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f4f54b945855ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:09.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.291+0000 7f73b27fc640 1 -- 172.21.15.70:0/320539373 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c0317c0 con 0x7f73b40ca680 2023-10-08T05:52:09.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.291+0000 7f73b27fc640 1 -- 172.21.15.70:0/320539373 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c031920 con 0x7f73b40ca680 2023-10-08T05:52:09.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.291+0000 7f73b27fc640 1 -- 172.21.15.70:0/320539373 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c031cb0 con 0x7f73b40ca680 2023-10-08T05:52:09.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.291+0000 7f73b3fff640 1 -- 172.21.15.70:0/320539373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40ca680 msgr2=0x7f73b40b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.291+0000 7f73b3fff640 1 --2- 172.21.15.70:0/320539373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40ca680 0x7f73b40b04b0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f739c009870 tx=0x7f739c02b860 comp rx=0 tx=0).stop 2023-10-08T05:52:09.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.291+0000 7f73b3fff640 1 -- 172.21.15.70:0/320539373 shutdown_connections 2023-10-08T05:52:09.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.291+0000 7f73b3fff640 1 --2- 172.21.15.70:0/320539373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b40ca680 0x7f73b40b04b0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.292+0000 7f73b3fff640 1 --2- 172.21.15.70:0/320539373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415a820 0x7f73b40aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.292+0000 7f73b3fff640 1 -- 172.21.15.70:0/320539373 >> 172.21.15.70:0/320539373 conn(0x7f73b406c050 msgr2=0x7f73b406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:09.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.292+0000 7f73b3fff640 1 -- 172.21.15.70:0/320539373 shutdown_connections 2023-10-08T05:52:09.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.292+0000 7f73b3fff640 1 -- 172.21.15.70:0/320539373 wait complete. 2023-10-08T05:52:09.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.293+0000 7f73b3fff640 1 Processor -- start 2023-10-08T05:52:09.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.293+0000 7f73b3fff640 1 -- start start 2023-10-08T05:52:09.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.294+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b40ca680 0x7f73b40c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.294+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b415a820 0x7f73b40c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.294+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415ac00 0x7f73b40c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.294+0000 7f73b3fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b40c5210 con 0x7f73b415a820 2023-10-08T05:52:09.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.294+0000 7f73b3fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b40ae350 con 0x7f73b40ca680 2023-10-08T05:52:09.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.294+0000 7f73b3fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b40c4970 con 0x7f73b415ac00 2023-10-08T05:52:09.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.294+0000 7f73abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b415a820 0x7f73b40c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b415a820 0x7f73b40c3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43240/0 (socket says 172.21.15.70:43240) 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73abfff640 1 -- 172.21.15.70:0/3507666345 learned_addr learned my addr 172.21.15.70:0/3507666345 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73b37fe640 1 --2- 172.21.15.70:0/3507666345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415ac00 0x7f73b40c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73abfff640 1 -- 172.21.15.70:0/3507666345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415ac00 msgr2=0x7f73b40c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73abfff640 1 --2- 172.21.15.70:0/3507666345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415ac00 0x7f73b40c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73b2ffd640 1 --2- 172.21.15.70:0/3507666345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b40ca680 0x7f73b40c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73abfff640 1 -- 172.21.15.70:0/3507666345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b40ca680 msgr2=0x7f73b40c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73abfff640 1 --2- 172.21.15.70:0/3507666345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b40ca680 0x7f73b40c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73abfff640 1 -- 172.21.15.70:0/3507666345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c0094d0 con 0x7f73b415a820 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73b37fe640 1 --2- 172.21.15.70:0/3507666345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415ac00 0x7f73b40c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73abfff640 1 --2- 172.21.15.70:0/3507666345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b415a820 0x7f73b40c3e20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f739c000c00 tx=0x7f739c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.295+0000 7f73ab7fe640 1 -- 172.21.15.70:0/3507666345 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c032860 con 0x7f73b415a820 2023-10-08T05:52:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.296+0000 7f73ab7fe640 1 -- 172.21.15.70:0/3507666345 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f739c0329c0 con 0x7f73b415a820 2023-10-08T05:52:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.296+0000 7f73b3fff640 1 -- 172.21.15.70:0/3507666345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b40c4c50 con 0x7f73b415a820 2023-10-08T05:52:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.296+0000 7f73ab7fe640 1 -- 172.21.15.70:0/3507666345 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c032d70 con 0x7f73b415a820 2023-10-08T05:52:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.296+0000 7f73b3fff640 1 -- 172.21.15.70:0/3507666345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b40c8650 con 0x7f73b415a820 2023-10-08T05:52:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.297+0000 7f73b3fff640 1 -- 172.21.15.70:0/3507666345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b40ae120 con 0x7f73b415a820 2023-10-08T05:52:09.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.298+0000 7f73ab7fe640 1 -- 172.21.15.70:0/3507666345 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 27) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f739c04d3f0 con 0x7f73b415a820 2023-10-08T05:52:09.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.299+0000 7f73ab7fe640 1 -- 172.21.15.70:0/3507666345 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f739c0ab440 con 0x7f73b415a820 2023-10-08T05:52:09.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.303+0000 7f73ab7fe640 1 -- 172.21.15.70:0/3507666345 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f739c0763e0 con 0x7f73b415a820 2023-10-08T05:52:09.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.481+0000 7f73b3fff640 1 -- 172.21.15.70:0/3507666345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73b4074150 con 0x7f73b415a820 2023-10-08T05:52:09.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.482+0000 7f73ab7fe640 1 -- 172.21.15.70:0/3507666345 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f739c079a00 con 0x7f73b415a820 2023-10-08T05:52:09.482 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:52:09.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.484+0000 7f73b3fff640 1 -- 172.21.15.70:0/3507666345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b415a820 msgr2=0x7f73b40c3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.484+0000 7f73b3fff640 1 --2- 172.21.15.70:0/3507666345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b415a820 0x7f73b40c3e20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f739c000c00 tx=0x7f739c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.485+0000 7f73b3fff640 1 -- 172.21.15.70:0/3507666345 shutdown_connections 2023-10-08T05:52:09.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.485+0000 7f73b3fff640 1 --2- 172.21.15.70:0/3507666345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73b415ac00 0x7f73b40c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.485+0000 7f73b3fff640 1 --2- 172.21.15.70:0/3507666345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73b415a820 0x7f73b40c3e20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.485+0000 7f73b3fff640 1 --2- 172.21.15.70:0/3507666345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73b40ca680 0x7f73b40c8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.485+0000 7f73b3fff640 1 -- 172.21.15.70:0/3507666345 >> 172.21.15.70:0/3507666345 conn(0x7f73b406c050 msgr2=0x7f73b40b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:09.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.485+0000 7f73b3fff640 1 -- 172.21.15.70:0/3507666345 shutdown_connections 2023-10-08T05:52:09.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.485+0000 7f73b3fff640 1 -- 172.21.15.70:0/3507666345 wait complete. 2023-10-08T05:52:09.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:09.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:52:09.577 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:52:09.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.704+0000 7fdc3bae5640 1 Processor -- start 2023-10-08T05:52:09.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.704+0000 7fdc3bae5640 1 -- start start 2023-10-08T05:52:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.705+0000 7fdc3bae5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc3415ab40 0x7fdc340ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.705+0000 7fdc3bae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc340cb580 0x7fdc340adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.705+0000 7fdc3bae5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3415b2b0 con 0x7fdc340c90b0 2023-10-08T05:52:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.705+0000 7fdc3bae5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3415bf30 con 0x7fdc3415ab40 2023-10-08T05:52:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.705+0000 7fdc3bae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3415f7a0 con 0x7fdc340cb580 2023-10-08T05:52:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.706+0000 7fdc39881640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc3415ab40 0x7fdc340ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.706+0000 7fdc39881640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc3415ab40 0x7fdc340ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43248/0 (socket says 172.21.15.70:43248) 2023-10-08T05:52:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.706+0000 7fdc39881640 1 -- 172.21.15.70:0/3875397775 learned_addr learned my addr 172.21.15.70:0/3875397775 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.706+0000 7fdc3a082640 1 --2- 172.21.15.70:0/3875397775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc340cb580 0x7fdc340adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.706+0000 7fdc39881640 1 -- 172.21.15.70:0/3875397775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc340cb580 msgr2=0x7fdc340adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.706+0000 7fdc39881640 1 --2- 172.21.15.70:0/3875397775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc340cb580 0x7fdc340adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.706+0000 7fdc39881640 1 -- 172.21.15.70:0/3875397775 >> v1:172.21.15.70:6789/0 conn(0x7fdc340c90b0 legacy=0x7fdc340cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.706+0000 7fdc39881640 1 -- 172.21.15.70:0/3875397775 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3415b590 con 0x7fdc3415ab40 2023-10-08T05:52:09.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.707+0000 7fdc39881640 1 --2- 172.21.15.70:0/3875397775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 0x7fdc340ad720 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdc300098a0 tx=0x7fdc3002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c50227850f8e2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3887f640 1 -- 172.21.15.70:0/3875397775 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30030840 con 0x7fdc3415ab40 2023-10-08T05:52:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3887f640 1 -- 172.21.15.70:0/3875397775 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc300309a0 con 0x7fdc3415ab40 2023-10-08T05:52:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3887f640 1 -- 172.21.15.70:0/3875397775 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30030d30 con 0x7fdc3415ab40 2023-10-08T05:52:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3bae5640 1 -- 172.21.15.70:0/3875397775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 msgr2=0x7fdc340ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3bae5640 1 --2- 172.21.15.70:0/3875397775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 0x7fdc340ad720 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdc300098a0 tx=0x7fdc3002b980 comp rx=0 tx=0).stop 2023-10-08T05:52:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3bae5640 1 -- 172.21.15.70:0/3875397775 shutdown_connections 2023-10-08T05:52:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3bae5640 1 --2- 172.21.15.70:0/3875397775 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc340cb580 0x7fdc340adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3bae5640 1 --2- 172.21.15.70:0/3875397775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 0x7fdc340ad720 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3bae5640 1 -- 172.21.15.70:0/3875397775 >> 172.21.15.70:0/3875397775 conn(0x7fdc3406a380 msgr2=0x7fdc340b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3bae5640 1 -- 172.21.15.70:0/3875397775 shutdown_connections 2023-10-08T05:52:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.708+0000 7fdc3bae5640 1 -- 172.21.15.70:0/3875397775 wait complete. 2023-10-08T05:52:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.710+0000 7fdc3bae5640 1 Processor -- start 2023-10-08T05:52:09.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.710+0000 7fdc3bae5640 1 -- start start 2023-10-08T05:52:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.711+0000 7fdc3bae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc340c90b0 0x7fdc340c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.711+0000 7fdc3bae5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc340cb580 0x7fdc341f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.711+0000 7fdc3bae5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 0x7fdc340c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.711+0000 7fdc3bae5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340abc10 con 0x7fdc3415ab40 2023-10-08T05:52:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.711+0000 7fdc3bae5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc341f35c0 con 0x7fdc340cb580 2023-10-08T05:52:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.711+0000 7fdc3bae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340c55f0 con 0x7fdc340c90b0 2023-10-08T05:52:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.711+0000 7fdc3a082640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 0x7fdc340c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc3a082640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 0x7fdc340c50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43250/0 (socket says 172.21.15.70:43250) 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc3a082640 1 -- 172.21.15.70:0/838310182 learned_addr learned my addr 172.21.15.70:0/838310182 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc39881640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc340c90b0 0x7fdc340c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc39080640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc340cb580 0x7fdc341f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc3a082640 1 -- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc340c90b0 msgr2=0x7fdc340c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc3a082640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc340c90b0 0x7fdc340c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc3a082640 1 -- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc340cb580 msgr2=0x7fdc341f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc3a082640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc340cb580 0x7fdc341f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc3a082640 1 -- 172.21.15.70:0/838310182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc300094d0 con 0x7fdc3415ab40 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc3a082640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 0x7fdc340c50b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00cd70 tx=0x7fdc2c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc2a7fc640 1 -- 172.21.15.70:0/838310182 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c003b90 con 0x7fdc3415ab40 2023-10-08T05:52:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc2a7fc640 1 -- 172.21.15.70:0/838310182 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c003cf0 con 0x7fdc3415ab40 2023-10-08T05:52:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc3bae5640 1 -- 172.21.15.70:0/838310182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc340c6d60 con 0x7fdc3415ab40 2023-10-08T05:52:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.712+0000 7fdc2a7fc640 1 -- 172.21.15.70:0/838310182 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c004240 con 0x7fdc3415ab40 2023-10-08T05:52:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.713+0000 7fdc3bae5640 1 -- 172.21.15.70:0/838310182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc340c7320 con 0x7fdc3415ab40 2023-10-08T05:52:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.714+0000 7fdc3bae5640 1 -- 172.21.15.70:0/838310182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc34067490 con 0x7fdc3415ab40 2023-10-08T05:52:09.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.715+0000 7fdc2a7fc640 1 -- 172.21.15.70:0/838310182 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 28) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fdc2c0043a0 con 0x7fdc3415ab40 2023-10-08T05:52:09.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.715+0000 7fdc2a7fc640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc1806acc0 0x7fdc1806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:09.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.715+0000 7fdc2a7fc640 1 -- 172.21.15.70:0/838310182 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fdc2c012070 con 0x7fdc3415ab40 2023-10-08T05:52:09.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.716+0000 7fdc39881640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc1806acc0 0x7fdc1806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:09.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.717+0000 7fdc39881640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc1806acc0 0x7fdc1806d180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdc30036000 tx=0x7fdc3003b040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:09.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.720+0000 7fdc2a7fc640 1 -- 172.21.15.70:0/838310182 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc2c055010 con 0x7fdc3415ab40 2023-10-08T05:52:09.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.950+0000 7fdc3bae5640 1 -- 172.21.15.70:0/838310182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc340b0ef0 con 0x7fdc3415ab40 2023-10-08T05:52:09.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.951+0000 7fdc2a7fc640 1 -- 172.21.15.70:0/838310182 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fdc2c058630 con 0x7fdc3415ab40 2023-10-08T05:52:09.952 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:09.952 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1126961048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1126961048},{"type":"v1","addr":"172.21.15.181:6836","nonce":1126961048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3993068621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3993068621},{"type":"v1","addr":"172.21.15.181:6837","nonce":3993068621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/28904251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":28904251},{"type":"v1","addr":"172.21.15.70:6833","nonce":28904251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:50:16.144118+0000","modified":"2023-10-08T05:50:19.194284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1337581673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1337581673},{"type":"v1","addr":"172.21.15.70:6835","nonce":1337581673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-08T05:52:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 25 2023-10-08T05:52:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 -- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc1806acc0 msgr2=0x7fdc1806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc1806acc0 0x7fdc1806d180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdc30036000 tx=0x7fdc3003b040 comp rx=0 tx=0).stop 2023-10-08T05:52:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 -- 172.21.15.70:0/838310182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 msgr2=0x7fdc340c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 0x7fdc340c50b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00cd70 tx=0x7fdc2c00a660 comp rx=0 tx=0).stop 2023-10-08T05:52:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 -- 172.21.15.70:0/838310182 shutdown_connections 2023-10-08T05:52:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc1806acc0 0x7fdc1806d180 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc3415ab40 0x7fdc340c50b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc340cb580 0x7fdc341f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 --2- 172.21.15.70:0/838310182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc340c90b0 0x7fdc340c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 -- 172.21.15.70:0/838310182 >> 172.21.15.70:0/838310182 conn(0x7fdc3406a380 msgr2=0x7fdc3406cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:09.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 -- 172.21.15.70:0/838310182 shutdown_connections 2023-10-08T05:52:09.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:09.954+0000 7fdc3bae5640 1 -- 172.21.15.70:0/838310182 wait complete. 2023-10-08T05:52:09.973 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:10.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.232+0000 7f6e48c39640 1 Processor -- start 2023-10-08T05:52:10.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.232+0000 7f6e48c39640 1 -- start start 2023-10-08T05:52:10.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.234+0000 7f6e48c39640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6e440add60 0x7f6e44162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.234+0000 7f6e48c39640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440ae140 con 0x7f6e440cb150 2023-10-08T05:52:10.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.234+0000 7f6e48c39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440ccf20 con 0x7f6e4406d4c0 2023-10-08T05:52:10.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.234+0000 7f6e48c39640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e44161f10 con 0x7f6e440add60 2023-10-08T05:52:10.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.234+0000 7f6e42d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6e440cb150 0x7f6e44164d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:32876/0 (socket says 172.21.15.70:32876) 2023-10-08T05:52:10.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.234+0000 7f6e42d9d640 1 -- 172.21.15.70:0/1880495571 learned_addr learned my addr 172.21.15.70:0/1880495571 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:10.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.234+0000 7f6e4259c640 1 --2- 172.21.15.70:0/1880495571 >> v2:172.21.15.181:3300/0 conn(0x7f6e440add60 0x7f6e44162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358713769 0 0) 0x7f6e440ae140 con 0x7f6e440cb150 2023-10-08T05:52:10.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e30003520 con 0x7f6e440cb150 2023-10-08T05:52:10.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2545446253 0 0) 0x7f6e30003520 con 0x7f6e440cb150 2023-10-08T05:52:10.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 >> v2:172.21.15.181:3300/0 conn(0x7f6e440add60 msgr2=0x7f6e44162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:10.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 --2- 172.21.15.70:0/1880495571 >> v2:172.21.15.181:3300/0 conn(0x7f6e440add60 0x7f6e44162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 >> v1:172.21.15.181:6789/0 conn(0x7f6e4406d4c0 legacy=0x7f6e440cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440cd1c0 con 0x7f6e440cb150 2023-10-08T05:52:10.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6e3c0019f0 con 0x7f6e440cb150 2023-10-08T05:52:10.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 >> v1:172.21.15.70:6789/0 conn(0x7f6e440cb150 legacy=0x7f6e44164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e30003970 0x7f6e30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e300047c0 0x7f6e30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e30005180 0x7f6e3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44161f10 con 0x7f6e30005180 2023-10-08T05:52:10.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3000c120 con 0x7f6e30003970 2023-10-08T05:52:10.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.235+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3000c2e0 con 0x7f6e300047c0 2023-10-08T05:52:10.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.236+0000 7f6e42d9d640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e30005180 0x7f6e3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.236+0000 7f6e3bfff640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e300047c0 0x7f6e30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.236+0000 7f6e4259c640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e30003970 0x7f6e30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.236+0000 7f6e42d9d640 1 -- 172.21.15.70:0/1880495571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e300047c0 msgr2=0x7f6e30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:10.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.236+0000 7f6e42d9d640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e300047c0 0x7f6e30004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.236+0000 7f6e42d9d640 1 -- 172.21.15.70:0/1880495571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e30003970 msgr2=0x7f6e30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:10.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.236+0000 7f6e42d9d640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e30003970 0x7f6e30003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.236+0000 7f6e42d9d640 1 -- 172.21.15.70:0/1880495571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30003520 con 0x7f6e30005180 2023-10-08T05:52:10.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.237+0000 7f6e42d9d640 1 -- 172.21.15.70:0/1880495571 reap_dead start 2023-10-08T05:52:10.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.237+0000 7f6e42d9d640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e30005180 0x7f6e3000b980 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00a820 tx=0x7f6e300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e11b2a2bfb032c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:10.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.237+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c0072d0 con 0x7f6e30005180 2023-10-08T05:52:10.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.237+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c02f070 con 0x7f6e30005180 2023-10-08T05:52:10.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.237+0000 7f6e41d9b640 1 -- 172.21.15.70:0/1880495571 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c007650 con 0x7f6e30005180 2023-10-08T05:52:10.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.238+0000 7f6e48c39640 1 -- 172.21.15.70:0/1880495571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e30005180 msgr2=0x7f6e3000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.238+0000 7f6e48c39640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e30005180 0x7f6e3000b980 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00a820 tx=0x7f6e300057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.238+0000 7f6e48c39640 1 -- 172.21.15.70:0/1880495571 shutdown_connections 2023-10-08T05:52:10.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.238+0000 7f6e48c39640 1 --2- 172.21.15.70:0/1880495571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e30005180 0x7f6e3000b980 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.238+0000 7f6e48c39640 1 -- 172.21.15.70:0/1880495571 >> 172.21.15.70:0/1880495571 conn(0x7f6e4406a380 msgr2=0x7f6e440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:10.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.238+0000 7f6e48c39640 1 -- 172.21.15.70:0/1880495571 shutdown_connections 2023-10-08T05:52:10.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.238+0000 7f6e48c39640 1 -- 172.21.15.70:0/1880495571 wait complete. 2023-10-08T05:52:10.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.240+0000 7f6e48c39640 1 Processor -- start 2023-10-08T05:52:10.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.240+0000 7f6e48c39640 1 -- start start 2023-10-08T05:52:10.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.241+0000 7f6e48c39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e30005180 0x7f6e4415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.241+0000 7f6e48c39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e441f64d0 0x7f6e441567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.241+0000 7f6e48c39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e44158100 0x7f6e441585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.241+0000 7f6e48c39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44156ce0 con 0x7f6e44158100 2023-10-08T05:52:10.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.241+0000 7f6e48c39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415e820 con 0x7f6e30005180 2023-10-08T05:52:10.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.241+0000 7f6e48c39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415df80 con 0x7f6e441f64d0 2023-10-08T05:52:10.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.241+0000 7f6e42d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e44158100 0x7f6e441585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.241+0000 7f6e42d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e44158100 0x7f6e441585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43278/0 (socket says 172.21.15.70:43278) 2023-10-08T05:52:10.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.241+0000 7f6e42d9d640 1 -- 172.21.15.70:0/1170318501 learned_addr learned my addr 172.21.15.70:0/1170318501 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e42d9d640 1 -- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e441f64d0 msgr2=0x7f6e441567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e42d9d640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e441f64d0 0x7f6e441567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e42d9d640 1 -- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e30005180 msgr2=0x7f6e4415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e42d9d640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e30005180 0x7f6e4415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e42d9d640 1 -- 172.21.15.70:0/1170318501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4415e260 con 0x7f6e44158100 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e42d9d640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e44158100 0x7f6e441585b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c000f80 tx=0x7f6e440cd0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e3affd640 1 -- 172.21.15.70:0/1170318501 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c004d60 con 0x7f6e44158100 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e3affd640 1 -- 172.21.15.70:0/1170318501 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c0070d0 con 0x7f6e44158100 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e3affd640 1 -- 172.21.15.70:0/1170318501 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c007440 con 0x7f6e44158100 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e48c39640 1 -- 172.21.15.70:0/1170318501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4415e540 con 0x7f6e44158100 2023-10-08T05:52:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.242+0000 7f6e48c39640 1 -- 172.21.15.70:0/1170318501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e44158af0 con 0x7f6e44158100 2023-10-08T05:52:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.243+0000 7f6e48c39640 1 -- 172.21.15.70:0/1170318501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e440b20a0 con 0x7f6e44158100 2023-10-08T05:52:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.244+0000 7f6e3affd640 1 -- 172.21.15.70:0/1170318501 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 28) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f6e3c031030 con 0x7f6e44158100 2023-10-08T05:52:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.245+0000 7f6e3affd640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6e2406acf0 0x7f6e2406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.245+0000 7f6e3affd640 1 -- 172.21.15.70:0/1170318501 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f6e3c0ab380 con 0x7f6e44158100 2023-10-08T05:52:10.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.246+0000 7f6e4259c640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6e2406acf0 0x7f6e2406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.247+0000 7f6e4259c640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6e2406acf0 0x7f6e2406d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009950 tx=0x7f6e34008040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:10.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.249+0000 7f6e3affd640 1 -- 172.21.15.70:0/1170318501 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e440b20a0 con 0x7f6e44158100 2023-10-08T05:52:10.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.442+0000 7f6e48c39640 1 -- 172.21.15.70:0/1170318501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e440b20a0 con 0x7f6e44158100 2023-10-08T05:52:10.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.445+0000 7f6e3affd640 1 -- 172.21.15.70:0/1170318501 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18005 (secure 0 0 0) 0x7f6e440b20a0 con 0x7f6e44158100 2023-10-08T05:52:10.446 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:10.446 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":78,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:08.555123+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:50:14.005619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-08T05:50:15.163925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2424526808":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/1956777616":"2023-10-09T05:52:08.555046+0000","172.21.15.181:6833/3803301866":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/602288726":"2023-10-09T05:52:08.555046+0000","172.21.15.181:6832/3803301866":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/2288760234":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/1060036340":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/464684516":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/1910914026":"2023-10-09T05:50:30.831312+0000","172.21.15.181:6833/992174303":"2023-10-09T05:50:30.831312+0000","172.21.15.181:6832/992174303":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/596247496":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/1453825375":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/423662621":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/368598410":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/3145795928":"2023-10-09T05:50:30.831312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:10.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 -- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6e2406acf0 msgr2=0x7f6e2406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6e2406acf0 0x7f6e2406d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009950 tx=0x7f6e34008040 comp rx=0 tx=0).stop 2023-10-08T05:52:10.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 -- 172.21.15.70:0/1170318501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e44158100 msgr2=0x7f6e441585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e44158100 0x7f6e441585b0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c000f80 tx=0x7f6e440cd0b0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 -- 172.21.15.70:0/1170318501 shutdown_connections 2023-10-08T05:52:10.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6e2406acf0 0x7f6e2406d1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e44158100 0x7f6e441585b0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6e441f64d0 0x7f6e441567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 --2- 172.21.15.70:0/1170318501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6e30005180 0x7f6e4415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 -- 172.21.15.70:0/1170318501 >> 172.21.15.70:0/1170318501 conn(0x7f6e4406a380 msgr2=0x7f6e441629d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:10.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 -- 172.21.15.70:0/1170318501 shutdown_connections 2023-10-08T05:52:10.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.448+0000 7f6e48c39640 1 -- 172.21.15.70:0/1170318501 wait complete. 2023-10-08T05:52:10.468 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T05:52:10.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:10.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.685+0000 7f540fb25640 1 Processor -- start 2023-10-08T05:52:10.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.685+0000 7f540fb25640 1 -- start start 2023-10-08T05:52:10.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.686+0000 7f540fb25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f540815ab40 0x7f54080ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.686+0000 7f540fb25640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540815b2b0 con 0x7f54080cb580 2023-10-08T05:52:10.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540fb25640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540815b590 con 0x7f54080c90b0 2023-10-08T05:52:10.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540fb25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540815bf30 con 0x7f540815ab40 2023-10-08T05:52:10.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540d8c1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f540815ab40 0x7f54080ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540e0c2640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f54080cb580 0x7f540815e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:32890/0 (socket says 172.21.15.70:32890) 2023-10-08T05:52:10.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540e0c2640 1 -- 172.21.15.70:0/1218705431 learned_addr learned my addr 172.21.15.70:0/1218705431 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:10.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540c8bf640 1 -- 172.21.15.70:0/1218705431 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171083159 0 0) 0x7f540815b2b0 con 0x7f54080cb580 2023-10-08T05:52:10.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540c8bf640 1 -- 172.21.15.70:0/1218705431 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53f4003520 con 0x7f54080cb580 2023-10-08T05:52:10.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540c8bf640 1 -- 172.21.15.70:0/1218705431 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067598718 0 0) 0x7f540815b590 con 0x7f54080c90b0 2023-10-08T05:52:10.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540d8c1640 1 -- 172.21.15.70:0/1218705431 >> v1:172.21.15.181:6789/0 conn(0x7f54080c90b0 legacy=0x7f54080cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540d8c1640 1 -- 172.21.15.70:0/1218705431 >> v1:172.21.15.70:6789/0 conn(0x7f54080cb580 legacy=0x7f540815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.687+0000 7f540d8c1640 1 -- 172.21.15.70:0/1218705431 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540815b870 con 0x7f540815ab40 2023-10-08T05:52:10.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.689+0000 7f540d8c1640 1 --2- 172.21.15.70:0/1218705431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f540815ab40 0x7f54080ad720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f54040097d0 tx=0x7f540402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540ba02cbe54b0d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:10.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.689+0000 7f540c8bf640 1 -- 172.21.15.70:0/1218705431 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404031890 con 0x7f540815ab40 2023-10-08T05:52:10.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.689+0000 7f540c8bf640 1 -- 172.21.15.70:0/1218705431 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54040319f0 con 0x7f540815ab40 2023-10-08T05:52:10.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.689+0000 7f540c8bf640 1 -- 172.21.15.70:0/1218705431 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5404031d80 con 0x7f540815ab40 2023-10-08T05:52:10.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.690+0000 7f540fb25640 1 -- 172.21.15.70:0/1218705431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f540815ab40 msgr2=0x7f54080ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.690+0000 7f540fb25640 1 --2- 172.21.15.70:0/1218705431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f540815ab40 0x7f54080ad720 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f54040097d0 tx=0x7f540402b930 comp rx=0 tx=0).stop 2023-10-08T05:52:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.690+0000 7f540fb25640 1 -- 172.21.15.70:0/1218705431 shutdown_connections 2023-10-08T05:52:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.690+0000 7f540fb25640 1 --2- 172.21.15.70:0/1218705431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f540815ab40 0x7f54080ad720 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.690+0000 7f540fb25640 1 -- 172.21.15.70:0/1218705431 >> 172.21.15.70:0/1218705431 conn(0x7f540806a380 msgr2=0x7f54080b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.690+0000 7f540fb25640 1 -- 172.21.15.70:0/1218705431 shutdown_connections 2023-10-08T05:52:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.690+0000 7f540fb25640 1 -- 172.21.15.70:0/1218705431 wait complete. 2023-10-08T05:52:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.692+0000 7f540fb25640 1 Processor -- start 2023-10-08T05:52:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.692+0000 7f540fb25640 1 -- start start 2023-10-08T05:52:10.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.693+0000 7f540fb25640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54080c90b0 0x7f54081542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.693+0000 7f540fb25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54080cb580 0x7f54080b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.693+0000 7f540fb25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f540815ab40 0x7f540806c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.693+0000 7f540fb25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54080b4520 con 0x7f540815ab40 2023-10-08T05:52:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.693+0000 7f540fb25640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54081f0590 con 0x7f54080c90b0 2023-10-08T05:52:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.693+0000 7f540fb25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54081f1560 con 0x7f54080cb580 2023-10-08T05:52:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.693+0000 7f540e0c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f540815ab40 0x7f540806c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.693+0000 7f540d8c1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54080c90b0 0x7f54081542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.693+0000 7f540e0c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f540815ab40 0x7f540806c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43282/0 (socket says 172.21.15.70:43282) 2023-10-08T05:52:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.693+0000 7f540e0c2640 1 -- 172.21.15.70:0/682813701 learned_addr learned my addr 172.21.15.70:0/682813701 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.694+0000 7f540d0c0640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54080cb580 0x7f54080b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.694+0000 7f540e0c2640 1 -- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54080cb580 msgr2=0x7f54080b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.694+0000 7f540e0c2640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54080cb580 0x7f54080b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.694+0000 7f540e0c2640 1 -- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54080c90b0 msgr2=0x7f54081542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.694+0000 7f540e0c2640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54080c90b0 0x7f54081542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.694+0000 7f540e0c2640 1 -- 172.21.15.70:0/682813701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5404009400 con 0x7f540815ab40 2023-10-08T05:52:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.694+0000 7f540e0c2640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f540815ab40 0x7f540806c890 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f540000cbe0 tx=0x7f540000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.694+0000 7f53fe7fc640 1 -- 172.21.15.70:0/682813701 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540000ad40 con 0x7f540815ab40 2023-10-08T05:52:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.694+0000 7f53fe7fc640 1 -- 172.21.15.70:0/682813701 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5400012400 con 0x7f540815ab40 2023-10-08T05:52:10.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.694+0000 7f53fe7fc640 1 -- 172.21.15.70:0/682813701 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540000aea0 con 0x7f540815ab40 2023-10-08T05:52:10.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.695+0000 7f540fb25640 1 -- 172.21.15.70:0/682813701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54081f10d0 con 0x7f540815ab40 2023-10-08T05:52:10.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.695+0000 7f540fb25640 1 -- 172.21.15.70:0/682813701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54081f0a80 con 0x7f540815ab40 2023-10-08T05:52:10.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.696+0000 7f540fb25640 1 -- 172.21.15.70:0/682813701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54081f1260 con 0x7f540815ab40 2023-10-08T05:52:10.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.697+0000 7f53fe7fc640 1 -- 172.21.15.70:0/682813701 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 28) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f5400007070 con 0x7f540815ab40 2023-10-08T05:52:10.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.698+0000 7f53fe7fc640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f53ec06acc0 0x7f53ec06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:10.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.698+0000 7f53fe7fc640 1 -- 172.21.15.70:0/682813701 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f5400089560 con 0x7f540815ab40 2023-10-08T05:52:10.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.698+0000 7f540d8c1640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f53ec06acc0 0x7f53ec06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.699+0000 7f540d8c1640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f53ec06acc0 0x7f53ec06d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5404036000 tx=0x7f540403d040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:10.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.702+0000 7f53fe7fc640 1 -- 172.21.15.70:0/682813701 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5400054590 con 0x7f540815ab40 2023-10-08T05:52:10.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.930+0000 7f540fb25640 1 -- 172.21.15.70:0/682813701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5408067490 con 0x7f540815ab40 2023-10-08T05:52:10.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.931+0000 7f53fe7fc640 1 -- 172.21.15.70:0/682813701 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f5400017090 con 0x7f540815ab40 2023-10-08T05:52:10.932 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:10.932 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1126961048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1126961048},{"type":"v1","addr":"172.21.15.181:6836","nonce":1126961048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3993068621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3993068621},{"type":"v1","addr":"172.21.15.181:6837","nonce":3993068621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/28904251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":28904251},{"type":"v1","addr":"172.21.15.70:6833","nonce":28904251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:50:16.144118+0000","modified":"2023-10-08T05:50:19.194284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1337581673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1337581673},{"type":"v1","addr":"172.21.15.70:6835","nonce":1337581673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-08T05:52:10.932 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 25 2023-10-08T05:52:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 -- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f53ec06acc0 msgr2=0x7f53ec06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f53ec06acc0 0x7f53ec06d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5404036000 tx=0x7f540403d040 comp rx=0 tx=0).stop 2023-10-08T05:52:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 -- 172.21.15.70:0/682813701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f540815ab40 msgr2=0x7f540806c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f540815ab40 0x7f540806c890 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f540000cbe0 tx=0x7f540000a430 comp rx=0 tx=0).stop 2023-10-08T05:52:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 -- 172.21.15.70:0/682813701 shutdown_connections 2023-10-08T05:52:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f53ec06acc0 0x7f53ec06d180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f540815ab40 0x7f540806c890 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f54080cb580 0x7f54080b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 --2- 172.21.15.70:0/682813701 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54080c90b0 0x7f54081542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 -- 172.21.15.70:0/682813701 >> 172.21.15.70:0/682813701 conn(0x7f540806a380 msgr2=0x7f54080ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:10.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 -- 172.21.15.70:0/682813701 shutdown_connections 2023-10-08T05:52:10.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:10.934+0000 7f540fb25640 1 -- 172.21.15.70:0/682813701 wait complete. 2023-10-08T05:52:10.953 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:11.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.163+0000 7fd1980e1640 1 Processor -- start 2023-10-08T05:52:11.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.163+0000 7fd1980e1640 1 -- start start 2023-10-08T05:52:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.165+0000 7fd1980e1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd19015ab40 0x7fd1900ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.165+0000 7fd1980e1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd1900c90b0 0x7fd1900adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.165+0000 7fd1980e1640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19015b2b0 con 0x7fd1900cae70 2023-10-08T05:52:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.165+0000 7fd1980e1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19015bf30 con 0x7fd1900c90b0 2023-10-08T05:52:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.165+0000 7fd1980e1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19015f7a0 con 0x7fd19015ab40 2023-10-08T05:52:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.165+0000 7fd19567c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd1900c90b0 0x7fd1900adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.165+0000 7fd19567c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd1900c90b0 0x7fd1900adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43296/0 (socket says 172.21.15.70:43296) 2023-10-08T05:52:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.165+0000 7fd19567c640 1 -- 172.21.15.70:0/1497838827 learned_addr learned my addr 172.21.15.70:0/1497838827 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.165+0000 7fd195e7d640 1 --2- 172.21.15.70:0/1497838827 >> v2:172.21.15.181:3300/0 conn(0x7fd19015ab40 0x7fd1900ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.165+0000 7fd19567c640 1 -- 172.21.15.70:0/1497838827 >> v2:172.21.15.181:3300/0 conn(0x7fd19015ab40 msgr2=0x7fd1900ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:11.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.166+0000 7fd19567c640 1 --2- 172.21.15.70:0/1497838827 >> v2:172.21.15.181:3300/0 conn(0x7fd19015ab40 0x7fd1900ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.166+0000 7fd19567c640 1 -- 172.21.15.70:0/1497838827 >> v1:172.21.15.181:6789/0 conn(0x7fd1900cae70 legacy=0x7fd1900702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:11.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.166+0000 7fd19567c640 1 -- 172.21.15.70:0/1497838827 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19015b590 con 0x7fd1900c90b0 2023-10-08T05:52:11.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.167+0000 7fd19567c640 1 --2- 172.21.15.70:0/1497838827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 0x7fd1900adc60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd19006e440 tx=0x7fd18002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bdb943d2a66832 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:11.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.167+0000 7fd194e7b640 1 -- 172.21.15.70:0/1497838827 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180031870 con 0x7fd1900c90b0 2023-10-08T05:52:11.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.167+0000 7fd194e7b640 1 -- 172.21.15.70:0/1497838827 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1800319d0 con 0x7fd1900c90b0 2023-10-08T05:52:11.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.167+0000 7fd194e7b640 1 -- 172.21.15.70:0/1497838827 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180031d60 con 0x7fd1900c90b0 2023-10-08T05:52:11.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.168+0000 7fd1980e1640 1 -- 172.21.15.70:0/1497838827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 msgr2=0x7fd1900adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:11.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.168+0000 7fd1980e1640 1 --2- 172.21.15.70:0/1497838827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 0x7fd1900adc60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fd19006e440 tx=0x7fd18002b910 comp rx=0 tx=0).stop 2023-10-08T05:52:11.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.168+0000 7fd1980e1640 1 -- 172.21.15.70:0/1497838827 shutdown_connections 2023-10-08T05:52:11.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.168+0000 7fd1980e1640 1 --2- 172.21.15.70:0/1497838827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 0x7fd1900adc60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.168+0000 7fd1980e1640 1 --2- 172.21.15.70:0/1497838827 >> v2:172.21.15.181:3300/0 conn(0x7fd19015ab40 0x7fd1900ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.168+0000 7fd1980e1640 1 -- 172.21.15.70:0/1497838827 >> 172.21.15.70:0/1497838827 conn(0x7fd19006a380 msgr2=0x7fd1900b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:11.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.168+0000 7fd1980e1640 1 -- 172.21.15.70:0/1497838827 shutdown_connections 2023-10-08T05:52:11.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.168+0000 7fd1980e1640 1 -- 172.21.15.70:0/1497838827 wait complete. 2023-10-08T05:52:11.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.170+0000 7fd1980e1640 1 Processor -- start 2023-10-08T05:52:11.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.170+0000 7fd1980e1640 1 -- start start 2023-10-08T05:52:11.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd1980e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 0x7fd190149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd1980e1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1900cae70 0x7fd190150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd1980e1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd19015ab40 0x7fd1900ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd1980e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1901524a0 con 0x7fd1900c90b0 2023-10-08T05:52:11.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd1980e1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1901f6e80 con 0x7fd19015ab40 2023-10-08T05:52:11.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd1980e1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1901f6300 con 0x7fd1900cae70 2023-10-08T05:52:11.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd195e7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 0x7fd190149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:11.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd195e7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 0x7fd190149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43306/0 (socket says 172.21.15.70:43306) 2023-10-08T05:52:11.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd195e7d640 1 -- 172.21.15.70:0/4161680943 learned_addr learned my addr 172.21.15.70:0/4161680943 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:11.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd195e7d640 1 -- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1900cae70 msgr2=0x7fd190150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:11.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd195e7d640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1900cae70 0x7fd190150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd195e7d640 1 -- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd19015ab40 msgr2=0x7fd1900ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:11.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd195e7d640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd19015ab40 0x7fd1900ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.171+0000 7fd195e7d640 1 -- 172.21.15.70:0/4161680943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800094d0 con 0x7fd1900c90b0 2023-10-08T05:52:11.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.172+0000 7fd195e7d640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 0x7fd190149250 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00c820 tx=0x7fd18c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:11.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.172+0000 7fd1867fc640 1 -- 172.21.15.70:0/4161680943 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c004220 con 0x7fd1900c90b0 2023-10-08T05:52:11.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.172+0000 7fd1867fc640 1 -- 172.21.15.70:0/4161680943 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18c0043a0 con 0x7fd1900c90b0 2023-10-08T05:52:11.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.172+0000 7fd1867fc640 1 -- 172.21.15.70:0/4161680943 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18c007a70 con 0x7fd1900c90b0 2023-10-08T05:52:11.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.172+0000 7fd1980e1640 1 -- 172.21.15.70:0/4161680943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1901f65e0 con 0x7fd1900c90b0 2023-10-08T05:52:11.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.172+0000 7fd1980e1640 1 -- 172.21.15.70:0/4161680943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1901f6ba0 con 0x7fd1900c90b0 2023-10-08T05:52:11.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.173+0000 7fd1980e1640 1 -- 172.21.15.70:0/4161680943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd190067490 con 0x7fd1900c90b0 2023-10-08T05:52:11.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.174+0000 7fd1867fc640 1 -- 172.21.15.70:0/4161680943 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 28) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fd18c007bd0 con 0x7fd1900c90b0 2023-10-08T05:52:11.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.175+0000 7fd1867fc640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd17406acc0 0x7fd17406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.176+0000 7fd1867fc640 1 -- 172.21.15.70:0/4161680943 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fd18c0894b0 con 0x7fd1900c90b0 2023-10-08T05:52:11.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.176+0000 7fd19567c640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd17406acc0 0x7fd17406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:11.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.177+0000 7fd19567c640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd17406acc0 0x7fd17406d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd180036000 tx=0x7fd180030000 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:11.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.180+0000 7fd1867fc640 1 -- 172.21.15.70:0/4161680943 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd18c054360 con 0x7fd1900c90b0 2023-10-08T05:52:11.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.411+0000 7fd1980e1640 1 -- 172.21.15.70:0/4161680943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1900afb90 con 0x7fd1900c90b0 2023-10-08T05:52:11.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.411+0000 7fd1867fc640 1 -- 172.21.15.70:0/4161680943 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fd18c057980 con 0x7fd1900c90b0 2023-10-08T05:52:11.412 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:11.412 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1126961048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1126961048},{"type":"v1","addr":"172.21.15.181:6836","nonce":1126961048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3993068621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3993068621},{"type":"v1","addr":"172.21.15.181:6837","nonce":3993068621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/28904251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":28904251},{"type":"v1","addr":"172.21.15.70:6833","nonce":28904251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:50:16.144118+0000","modified":"2023-10-08T05:50:19.194284+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6177":{"gid":6177,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/1337581673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1337581673},{"type":"v1","addr":"172.21.15.70:6835","nonce":1337581673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-10-08T05:52:11.412 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 25 2023-10-08T05:52:11.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 -- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd17406acc0 msgr2=0x7fd17406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd17406acc0 0x7fd17406d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd180036000 tx=0x7fd180030000 comp rx=0 tx=0).stop 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 -- 172.21.15.70:0/4161680943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 msgr2=0x7fd190149250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 0x7fd190149250 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00c820 tx=0x7fd18c00cce0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 -- 172.21.15.70:0/4161680943 shutdown_connections 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd17406acc0 0x7fd17406d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd19015ab40 0x7fd1900ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1900cae70 0x7fd190150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 --2- 172.21.15.70:0/4161680943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1900c90b0 0x7fd190149250 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 -- 172.21.15.70:0/4161680943 >> 172.21.15.70:0/4161680943 conn(0x7fd19006a380 msgr2=0x7fd1900c9500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 -- 172.21.15.70:0/4161680943 shutdown_connections 2023-10-08T05:52:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.414+0000 7fd1980e1640 1 -- 172.21.15.70:0/4161680943 wait complete. 2023-10-08T05:52:11.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:11.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.645+0000 7fdc75275640 1 Processor -- start 2023-10-08T05:52:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.645+0000 7fdc75275640 1 -- start start 2023-10-08T05:52:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.647+0000 7fdc75275640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc7015ab40 0x7fdc700ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.647+0000 7fdc75275640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc700cb580 0x7fdc700adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.647+0000 7fdc75275640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7015b2b0 con 0x7fdc700c90b0 2023-10-08T05:52:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.647+0000 7fdc75275640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7015bf30 con 0x7fdc700cb580 2023-10-08T05:52:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.647+0000 7fdc75275640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7015f7a0 con 0x7fdc7015ab40 2023-10-08T05:52:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.648+0000 7fdc6f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc700cb580 0x7fdc700adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.648+0000 7fdc6f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdc700cb580 0x7fdc700adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43318/0 (socket says 172.21.15.70:43318) 2023-10-08T05:52:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.648+0000 7fdc6f7fe640 1 -- 172.21.15.70:0/4279328774 learned_addr learned my addr 172.21.15.70:0/4279328774 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.648+0000 7fdc6effd640 1 --2- 172.21.15.70:0/4279328774 >> v2:172.21.15.181:3300/0 conn(0x7fdc7015ab40 0x7fdc700ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.648+0000 7fdc6f7fe640 1 -- 172.21.15.70:0/4279328774 >> v2:172.21.15.181:3300/0 conn(0x7fdc7015ab40 msgr2=0x7fdc700ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.648+0000 7fdc6f7fe640 1 --2- 172.21.15.70:0/4279328774 >> v2:172.21.15.181:3300/0 conn(0x7fdc7015ab40 0x7fdc700ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.648+0000 7fdc6f7fe640 1 -- 172.21.15.70:0/4279328774 >> v1:172.21.15.181:6789/0 conn(0x7fdc700c90b0 legacy=0x7fdc700cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.648+0000 7fdc6f7fe640 1 -- 172.21.15.70:0/4279328774 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7015b590 con 0x7fdc700cb580 2023-10-08T05:52:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.649+0000 7fdc6f7fe640 1 --2- 172.21.15.70:0/4279328774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700cb580 0x7fdc700adc60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdc7006e440 tx=0x7fdc6802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9946f2439c7d3c6d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.649+0000 7fdc6dffb640 1 -- 172.21.15.70:0/4279328774 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6802ca80 con 0x7fdc700cb580 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.650+0000 7fdc6dffb640 1 -- 172.21.15.70:0/4279328774 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6802cbe0 con 0x7fdc700cb580 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.650+0000 7fdc6dffb640 1 -- 172.21.15.70:0/4279328774 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc680343f0 con 0x7fdc700cb580 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.650+0000 7fdc75275640 1 -- 172.21.15.70:0/4279328774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700cb580 msgr2=0x7fdc700adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.650+0000 7fdc75275640 1 --2- 172.21.15.70:0/4279328774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700cb580 0x7fdc700adc60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdc7006e440 tx=0x7fdc6802bee0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.650+0000 7fdc75275640 1 -- 172.21.15.70:0/4279328774 shutdown_connections 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.650+0000 7fdc75275640 1 --2- 172.21.15.70:0/4279328774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700cb580 0x7fdc700adc60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.650+0000 7fdc75275640 1 --2- 172.21.15.70:0/4279328774 >> v2:172.21.15.181:3300/0 conn(0x7fdc7015ab40 0x7fdc700ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.650+0000 7fdc75275640 1 -- 172.21.15.70:0/4279328774 >> 172.21.15.70:0/4279328774 conn(0x7fdc7006a380 msgr2=0x7fdc700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.650+0000 7fdc75275640 1 -- 172.21.15.70:0/4279328774 shutdown_connections 2023-10-08T05:52:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.650+0000 7fdc75275640 1 -- 172.21.15.70:0/4279328774 wait complete. 2023-10-08T05:52:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.652+0000 7fdc75275640 1 Processor -- start 2023-10-08T05:52:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.652+0000 7fdc75275640 1 -- start start 2023-10-08T05:52:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.653+0000 7fdc75275640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700c90b0 0x7fdc7014a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.653+0000 7fdc75275640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc700cb580 0x7fdc700c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.653+0000 7fdc75275640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc7015ab40 0x7fdc700c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.653+0000 7fdc75275640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700b2f30 con 0x7fdc700c90b0 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.653+0000 7fdc75275640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc70155500 con 0x7fdc700cb580 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.653+0000 7fdc75275640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc700b23b0 con 0x7fdc7015ab40 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.653+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700c90b0 0x7fdc7014a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700c90b0 0x7fdc7014a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43322/0 (socket says 172.21.15.70:43322) 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6effd640 1 -- 172.21.15.70:0/3747547913 learned_addr learned my addr 172.21.15.70:0/3747547913 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6e7fc640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc700cb580 0x7fdc700c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6f7fe640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc7015ab40 0x7fdc700c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6effd640 1 -- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc7015ab40 msgr2=0x7fdc700c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6effd640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc7015ab40 0x7fdc700c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6effd640 1 -- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc700cb580 msgr2=0x7fdc700c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:11.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6effd640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc700cb580 0x7fdc700c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6effd640 1 -- 172.21.15.70:0/3747547913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc68009b30 con 0x7fdc700c90b0 2023-10-08T05:52:11.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6e7fc640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc700cb580 0x7fdc700c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:11.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.654+0000 7fdc6effd640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700c90b0 0x7fdc7014a2b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c7f0 tx=0x7fdc5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.655+0000 7fdc677fe640 1 -- 172.21.15.70:0/3747547913 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58004220 con 0x7fdc700c90b0 2023-10-08T05:52:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.655+0000 7fdc677fe640 1 -- 172.21.15.70:0/3747547913 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc580043a0 con 0x7fdc700c90b0 2023-10-08T05:52:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.655+0000 7fdc677fe640 1 -- 172.21.15.70:0/3747547913 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58007860 con 0x7fdc700c90b0 2023-10-08T05:52:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.655+0000 7fdc75275640 1 -- 172.21.15.70:0/3747547913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc700b2690 con 0x7fdc700c90b0 2023-10-08T05:52:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.655+0000 7fdc75275640 1 -- 172.21.15.70:0/3747547913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc700b2c50 con 0x7fdc700c90b0 2023-10-08T05:52:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.656+0000 7fdc75275640 1 -- 172.21.15.70:0/3747547913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc701556c0 con 0x7fdc700c90b0 2023-10-08T05:52:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.657+0000 7fdc677fe640 1 -- 172.21.15.70:0/3747547913 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 29) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fdc58007bd0 con 0x7fdc700c90b0 2023-10-08T05:52:11.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.658+0000 7fdc677fe640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc5006af00 0x7fdc5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:11.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.659+0000 7fdc677fe640 1 -- 172.21.15.70:0/3747547913 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fdc58089c30 con 0x7fdc700c90b0 2023-10-08T05:52:11.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.659+0000 7fdc6e7fc640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc5006af00 0x7fdc5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.660+0000 7fdc6e7fc640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc5006af00 0x7fdc5006d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdc700c8890 tx=0x7fdc60008040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:11.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.663+0000 7fdc677fe640 1 -- 172.21.15.70:0/3747547913 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc58054670 con 0x7fdc700c90b0 2023-10-08T05:52:11.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.852+0000 7fdc75275640 1 -- 172.21.15.70:0/3747547913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc70067490 con 0x7fdc700c90b0 2023-10-08T05:52:11.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.853+0000 7fdc677fe640 1 -- 172.21.15.70:0/3747547913 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18005 (secure 0 0 0) 0x7fdc5801e020 con 0x7fdc700c90b0 2023-10-08T05:52:11.854 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:11.854 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":78,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:08.555123+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:50:14.005619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-08T05:50:15.163925+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2424526808":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/1956777616":"2023-10-09T05:52:08.555046+0000","172.21.15.181:6833/3803301866":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/602288726":"2023-10-09T05:52:08.555046+0000","172.21.15.181:6832/3803301866":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/2288760234":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/1060036340":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/464684516":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/1910914026":"2023-10-09T05:50:30.831312+0000","172.21.15.181:6833/992174303":"2023-10-09T05:50:30.831312+0000","172.21.15.181:6832/992174303":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/596247496":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/1453825375":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/423662621":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/368598410":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/3145795928":"2023-10-09T05:50:30.831312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 -- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc5006af00 msgr2=0x7fdc5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc5006af00 0x7fdc5006d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdc700c8890 tx=0x7fdc60008040 comp rx=0 tx=0).stop 2023-10-08T05:52:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 -- 172.21.15.70:0/3747547913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700c90b0 msgr2=0x7fdc7014a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700c90b0 0x7fdc7014a2b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c7f0 tx=0x7fdc5800ccb0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 -- 172.21.15.70:0/3747547913 shutdown_connections 2023-10-08T05:52:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc5006af00 0x7fdc5006d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc7015ab40 0x7fdc700c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc700cb580 0x7fdc700c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 --2- 172.21.15.70:0/3747547913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc700c90b0 0x7fdc7014a2b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:11.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 -- 172.21.15.70:0/3747547913 >> 172.21.15.70:0/3747547913 conn(0x7fdc7006a380 msgr2=0x7fdc700cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:11.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.856+0000 7fdc75275640 1 -- 172.21.15.70:0/3747547913 shutdown_connections 2023-10-08T05:52:11.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:11.857+0000 7fdc75275640 1 -- 172.21.15.70:0/3747547913 wait complete. 2023-10-08T05:52:11.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T05:52:12.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.086+0000 7f346889b640 1 Processor -- start 2023-10-08T05:52:12.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.087+0000 7f346889b640 1 -- start start 2023-10-08T05:52:12.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.088+0000 7f346889b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f34640c9cf0 0x7f346406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.088+0000 7f346889b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f346415aa00 0x7f34640b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.088+0000 7f346889b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f346415b310 con 0x7f346406c3a0 2023-10-08T05:52:12.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.088+0000 7f346889b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346406e0a0 con 0x7f34640c9cf0 2023-10-08T05:52:12.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.088+0000 7f346889b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f346406d290 con 0x7f346415aa00 2023-10-08T05:52:12.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.089+0000 7f346259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f346406c3a0 0x7f346406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:32898/0 (socket says 172.21.15.70:32898) 2023-10-08T05:52:12.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.089+0000 7f346259c640 1 -- 172.21.15.70:0/999581270 learned_addr learned my addr 172.21.15.70:0/999581270 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:12.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.089+0000 7f345bfff640 1 --2- 172.21.15.70:0/999581270 >> v2:172.21.15.70:3300/0 conn(0x7f34640c9cf0 0x7f346406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.089+0000 7f3462d9d640 1 --2- 172.21.15.70:0/999581270 >> v2:172.21.15.181:3300/0 conn(0x7f346415aa00 0x7f34640b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.089+0000 7f3461d9b640 1 -- 172.21.15.70:0/999581270 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617313808 0 0) 0x7f3454001110 con 0x7f346406c3a0 2023-10-08T05:52:12.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.089+0000 7f3461d9b640 1 -- 172.21.15.70:0/999581270 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3450003520 con 0x7f346406c3a0 2023-10-08T05:52:12.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.089+0000 7f345bfff640 1 -- 172.21.15.70:0/999581270 >> v2:172.21.15.181:3300/0 conn(0x7f346415aa00 msgr2=0x7f34640b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:12.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.089+0000 7f345bfff640 1 --2- 172.21.15.70:0/999581270 >> v2:172.21.15.181:3300/0 conn(0x7f346415aa00 0x7f34640b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.089+0000 7f345bfff640 1 -- 172.21.15.70:0/999581270 >> v1:172.21.15.70:6789/0 conn(0x7f346406c3a0 legacy=0x7f346406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:12.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.089+0000 7f345bfff640 1 -- 172.21.15.70:0/999581270 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34640c9860 con 0x7f34640c9cf0 2023-10-08T05:52:12.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.090+0000 7f345bfff640 1 --2- 172.21.15.70:0/999581270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34640c9cf0 0x7f346406db60 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f34640c57a0 tx=0x7f344c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3fd7167f881307 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:12.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.091+0000 7f3461d9b640 1 -- 172.21.15.70:0/999581270 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c031830 con 0x7f34640c9cf0 2023-10-08T05:52:12.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.091+0000 7f3461d9b640 1 -- 172.21.15.70:0/999581270 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c031990 con 0x7f34640c9cf0 2023-10-08T05:52:12.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.091+0000 7f3461d9b640 1 -- 172.21.15.70:0/999581270 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c031d20 con 0x7f34640c9cf0 2023-10-08T05:52:12.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.091+0000 7f346889b640 1 -- 172.21.15.70:0/999581270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34640c9cf0 msgr2=0x7f346406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:12.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.091+0000 7f346889b640 1 --2- 172.21.15.70:0/999581270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34640c9cf0 0x7f346406db60 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f34640c57a0 tx=0x7f344c02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.091+0000 7f346889b640 1 -- 172.21.15.70:0/999581270 shutdown_connections 2023-10-08T05:52:12.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.091+0000 7f346889b640 1 --2- 172.21.15.70:0/999581270 >> v2:172.21.15.181:3300/0 conn(0x7f346415aa00 0x7f34640b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.091+0000 7f346889b640 1 --2- 172.21.15.70:0/999581270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34640c9cf0 0x7f346406db60 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.091+0000 7f346889b640 1 -- 172.21.15.70:0/999581270 >> 172.21.15.70:0/999581270 conn(0x7f346406a5d0 msgr2=0x7f34640b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:12.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.092+0000 7f346889b640 1 -- 172.21.15.70:0/999581270 shutdown_connections 2023-10-08T05:52:12.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.092+0000 7f346889b640 1 -- 172.21.15.70:0/999581270 wait complete. 2023-10-08T05:52:12.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.093+0000 7f346889b640 1 Processor -- start 2023-10-08T05:52:12.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.093+0000 7f346889b640 1 -- start start 2023-10-08T05:52:12.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.094+0000 7f346889b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346406c3a0 0x7f3464155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.094+0000 7f346889b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34640c9cf0 0x7f346413f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.094+0000 7f346889b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f346415aa00 0x7f346413f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.094+0000 7f346889b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346413fde0 con 0x7f346406c3a0 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.094+0000 7f346889b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34641400c0 con 0x7f346415aa00 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.094+0000 7f346889b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3464157b90 con 0x7f34640c9cf0 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f346259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346406c3a0 0x7f3464155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f346259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346406c3a0 0x7f3464155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43342/0 (socket says 172.21.15.70:43342) 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f346259c640 1 -- 172.21.15.70:0/2195624389 learned_addr learned my addr 172.21.15.70:0/2195624389 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f3462d9d640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f346415aa00 0x7f346413f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f346259c640 1 -- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34640c9cf0 msgr2=0x7f346413f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f345bfff640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34640c9cf0 0x7f346413f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f346259c640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34640c9cf0 0x7f346413f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f346259c640 1 -- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f346415aa00 msgr2=0x7f346413f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:12.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f346259c640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f346415aa00 0x7f346413f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f346259c640 1 -- 172.21.15.70:0/2195624389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c009400 con 0x7f346406c3a0 2023-10-08T05:52:12.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.095+0000 7f346259c640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346406c3a0 0x7f3464155830 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f345400c700 tx=0x7f345400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:12.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.096+0000 7f345affd640 1 -- 172.21.15.70:0/2195624389 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454005ce0 con 0x7f346406c3a0 2023-10-08T05:52:12.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.096+0000 7f345affd640 1 -- 172.21.15.70:0/2195624389 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3454005e40 con 0x7f346406c3a0 2023-10-08T05:52:12.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.096+0000 7f345affd640 1 -- 172.21.15.70:0/2195624389 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454007290 con 0x7f346406c3a0 2023-10-08T05:52:12.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.096+0000 7f346889b640 1 -- 172.21.15.70:0/2195624389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3464157e70 con 0x7f346406c3a0 2023-10-08T05:52:12.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.096+0000 7f346889b640 1 -- 172.21.15.70:0/2195624389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3464152ac0 con 0x7f346406c3a0 2023-10-08T05:52:12.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.097+0000 7f346889b640 1 -- 172.21.15.70:0/2195624389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346404e160 con 0x7f346406c3a0 2023-10-08T05:52:12.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.098+0000 7f345affd640 1 -- 172.21.15.70:0/2195624389 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 29) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f34540074b0 con 0x7f346406c3a0 2023-10-08T05:52:12.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.099+0000 7f345affd640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f344406af00 0x7f344406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.099+0000 7f345affd640 1 -- 172.21.15.70:0/2195624389 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f34540890a0 con 0x7f346406c3a0 2023-10-08T05:52:12.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.099+0000 7f345bfff640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f344406af00 0x7f344406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.101+0000 7f345bfff640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f344406af00 0x7f344406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f344c036000 tx=0x7f344c008030 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:12.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.103+0000 7f345affd640 1 -- 172.21.15.70:0/2195624389 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34540575f0 con 0x7f346406c3a0 2023-10-08T05:52:12.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.332+0000 7f346889b640 1 -- 172.21.15.70:0/2195624389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f346415b310 con 0x7f346406c3a0 2023-10-08T05:52:12.334 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T05:52:12.333+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T05:52:12.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.336+0000 7f345affd640 1 -- 172.21.15.70:0/2195624389 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f346415b310 con 0x7f346406c3a0 2023-10-08T05:52:12.337 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T05:52:12.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 -- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f344406af00 msgr2=0x7f344406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:12.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f344406af00 0x7f344406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f344c036000 tx=0x7f344c008030 comp rx=0 tx=0).stop 2023-10-08T05:52:12.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 -- 172.21.15.70:0/2195624389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346406c3a0 msgr2=0x7f3464155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346406c3a0 0x7f3464155830 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f345400c700 tx=0x7f345400cb00 comp rx=0 tx=0).stop 2023-10-08T05:52:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 -- 172.21.15.70:0/2195624389 shutdown_connections 2023-10-08T05:52:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f344406af00 0x7f344406d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f346415aa00 0x7f346413f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34640c9cf0 0x7f346413f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 --2- 172.21.15.70:0/2195624389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346406c3a0 0x7f3464155830 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 -- 172.21.15.70:0/2195624389 >> 172.21.15.70:0/2195624389 conn(0x7f346406a5d0 msgr2=0x7f34640cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 -- 172.21.15.70:0/2195624389 shutdown_connections 2023-10-08T05:52:12.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.341+0000 7f346889b640 1 -- 172.21.15.70:0/2195624389 wait complete. 2023-10-08T05:52:12.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T05:52:12.434 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T05:52:12.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.551+0000 7f9c4064b640 1 Processor -- start 2023-10-08T05:52:12.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.551+0000 7f9c4064b640 1 -- start start 2023-10-08T05:52:12.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.552+0000 7f9c4064b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380afbe0 0x7f9c380b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.552+0000 7f9c4064b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c380cb130 0x7f9c380cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.552+0000 7f9c4064b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c380b2620 con 0x7f9c3806dfc0 2023-10-08T05:52:12.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.552+0000 7f9c4064b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c380cbae0 con 0x7f9c380cb130 2023-10-08T05:52:12.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.552+0000 7f9c4064b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c380b2060 con 0x7f9c380afbe0 2023-10-08T05:52:12.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.553+0000 7f9c3e3e7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380afbe0 0x7f9c380b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.553+0000 7f9c3ebe8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c380cb130 0x7f9c380cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.553+0000 7f9c3e3e7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380afbe0 0x7f9c380b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:48120/0 (socket says 172.21.15.70:48120) 2023-10-08T05:52:12.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.554+0000 7f9c3e3e7640 1 -- 172.21.15.70:0/11685284 learned_addr learned my addr 172.21.15.70:0/11685284 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:12.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.553+0000 7f9c3dbe6640 1 --1- 172.21.15.70:0/11685284 >> v1:172.21.15.181:6789/0 conn(0x7f9c3806dfc0 0x7f9c38070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:52466/0 (socket says 172.21.15.70:52466) 2023-10-08T05:52:12.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.554+0000 7f9c3ebe8640 1 -- 172.21.15.70:0/11685284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380afbe0 msgr2=0x7f9c380b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:12.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.554+0000 7f9c3ebe8640 1 --2- 172.21.15.70:0/11685284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380afbe0 0x7f9c380b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.554+0000 7f9c3ebe8640 1 -- 172.21.15.70:0/11685284 >> v1:172.21.15.181:6789/0 conn(0x7f9c3806dfc0 legacy=0x7f9c38070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:12.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.554+0000 7f9c3ebe8640 1 -- 172.21.15.70:0/11685284 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c380b2340 con 0x7f9c380cb130 2023-10-08T05:52:12.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.556+0000 7f9c3ebe8640 1 --2- 172.21.15.70:0/11685284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c380cb130 0x7f9c380cb5a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009d90 tx=0x7f9c3002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=219f1c812394e613 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:12.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.556+0000 7f9c3d3e5640 1 -- 172.21.15.70:0/11685284 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3002cc50 con 0x7f9c380cb130 2023-10-08T05:52:12.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.556+0000 7f9c3d3e5640 1 -- 172.21.15.70:0/11685284 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3002cdb0 con 0x7f9c380cb130 2023-10-08T05:52:12.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.556+0000 7f9c3d3e5640 1 -- 172.21.15.70:0/11685284 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30034660 con 0x7f9c380cb130 2023-10-08T05:52:12.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.557+0000 7f9c4064b640 1 -- 172.21.15.70:0/11685284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c380cb130 msgr2=0x7f9c380cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:12.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.557+0000 7f9c4064b640 1 --2- 172.21.15.70:0/11685284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c380cb130 0x7f9c380cb5a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009d90 tx=0x7f9c3002bea0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.557+0000 7f9c4064b640 1 -- 172.21.15.70:0/11685284 shutdown_connections 2023-10-08T05:52:12.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.557+0000 7f9c4064b640 1 --2- 172.21.15.70:0/11685284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c380cb130 0x7f9c380cb5a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.557+0000 7f9c4064b640 1 --2- 172.21.15.70:0/11685284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380afbe0 0x7f9c380b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.557+0000 7f9c4064b640 1 -- 172.21.15.70:0/11685284 >> 172.21.15.70:0/11685284 conn(0x7f9c3806a5d0 msgr2=0x7f9c380b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:12.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.557+0000 7f9c4064b640 1 -- 172.21.15.70:0/11685284 shutdown_connections 2023-10-08T05:52:12.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.557+0000 7f9c4064b640 1 -- 172.21.15.70:0/11685284 wait complete. 2023-10-08T05:52:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.559+0000 7f9c4064b640 1 Processor -- start 2023-10-08T05:52:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.559+0000 7f9c4064b640 1 -- start start 2023-10-08T05:52:12.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.560+0000 7f9c4064b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c3806dfc0 0x7f9c380ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.560+0000 7f9c4064b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c380afbe0 0x7f9c380c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.560+0000 7f9c4064b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380cb130 0x7f9c380aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.560+0000 7f9c4064b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3815cd40 con 0x7f9c380afbe0 2023-10-08T05:52:12.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.560+0000 7f9c4064b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c380c96e0 con 0x7f9c3806dfc0 2023-10-08T05:52:12.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.560+0000 7f9c4064b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c380c99c0 con 0x7f9c380cb130 2023-10-08T05:52:12.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.560+0000 7f9c3dbe6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c380afbe0 0x7f9c380c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.560+0000 7f9c3dbe6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c380afbe0 0x7f9c380c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43364/0 (socket says 172.21.15.70:43364) 2023-10-08T05:52:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.560+0000 7f9c3dbe6640 1 -- 172.21.15.70:0/56167759 learned_addr learned my addr 172.21.15.70:0/56167759 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c3e3e7640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c3806dfc0 0x7f9c380ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c3ebe8640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380cb130 0x7f9c380aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c3dbe6640 1 -- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380cb130 msgr2=0x7f9c380aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c3dbe6640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380cb130 0x7f9c380aed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c3dbe6640 1 -- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c3806dfc0 msgr2=0x7f9c380ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c3dbe6640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c3806dfc0 0x7f9c380ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c3dbe6640 1 -- 172.21.15.70:0/56167759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c30009a70 con 0x7f9c380afbe0 2023-10-08T05:52:12.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c3dbe6640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c380afbe0 0x7f9c380c8d90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800c3e0 tx=0x7f9c2800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:12.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c2effd640 1 -- 172.21.15.70:0/56167759 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c280074e0 con 0x7f9c380afbe0 2023-10-08T05:52:12.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c2effd640 1 -- 172.21.15.70:0/56167759 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c28007640 con 0x7f9c380afbe0 2023-10-08T05:52:12.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c2effd640 1 -- 172.21.15.70:0/56167759 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c280049e0 con 0x7f9c380afbe0 2023-10-08T05:52:12.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.561+0000 7f9c4064b640 1 -- 172.21.15.70:0/56167759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c380c9ca0 con 0x7f9c380afbe0 2023-10-08T05:52:12.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.562+0000 7f9c4064b640 1 -- 172.21.15.70:0/56167759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c380cacc0 con 0x7f9c380afbe0 2023-10-08T05:52:12.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.562+0000 7f9c4064b640 1 -- 172.21.15.70:0/56167759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3804e160 con 0x7f9c380afbe0 2023-10-08T05:52:12.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.563+0000 7f9c2effd640 1 -- 172.21.15.70:0/56167759 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 29) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9c28004b40 con 0x7f9c380afbe0 2023-10-08T05:52:12.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.564+0000 7f9c2effd640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9c1c06af00 0x7f9c1c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:12.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.564+0000 7f9c2effd640 1 -- 172.21.15.70:0/56167759 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f9c28089540 con 0x7f9c380afbe0 2023-10-08T05:52:12.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.565+0000 7f9c3e3e7640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9c1c06af00 0x7f9c1c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:12.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.566+0000 7f9c3e3e7640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9c1c06af00 0x7f9c1c06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009ac0 tx=0x7f9c34008040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:12.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.569+0000 7f9c2effd640 1 -- 172.21.15.70:0/56167759 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c280540e0 con 0x7f9c380afbe0 2023-10-08T05:52:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:12.780+0000 7f9c4064b640 1 -- 172.21.15.70:0/56167759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9c380b0ef0 con 0x7f9c380afbe0 2023-10-08T05:52:13.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.340+0000 7f9c2effd640 1 -- 172.21.15.70:0/56167759 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9c28057700 con 0x7f9c380afbe0 2023-10-08T05:52:13.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.343+0000 7f9c4064b640 1 -- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9c1c06af00 msgr2=0x7f9c1c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:13.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.343+0000 7f9c4064b640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9c1c06af00 0x7f9c1c06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009ac0 tx=0x7f9c34008040 comp rx=0 tx=0).stop 2023-10-08T05:52:13.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.343+0000 7f9c4064b640 1 -- 172.21.15.70:0/56167759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c380afbe0 msgr2=0x7f9c380c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:13.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.343+0000 7f9c4064b640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c380afbe0 0x7f9c380c8d90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9c2800c3e0 tx=0x7f9c2800c8a0 comp rx=0 tx=0).stop 2023-10-08T05:52:13.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.343+0000 7f9c4064b640 1 -- 172.21.15.70:0/56167759 shutdown_connections 2023-10-08T05:52:13.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.343+0000 7f9c4064b640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9c1c06af00 0x7f9c1c06d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:13.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.343+0000 7f9c4064b640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c380cb130 0x7f9c380aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:13.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.343+0000 7f9c4064b640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c380afbe0 0x7f9c380c8d90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:13.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.343+0000 7f9c4064b640 1 --2- 172.21.15.70:0/56167759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c3806dfc0 0x7f9c380ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:13.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.343+0000 7f9c4064b640 1 -- 172.21.15.70:0/56167759 >> 172.21.15.70:0/56167759 conn(0x7f9c3806a5d0 msgr2=0x7f9c380c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:13.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.344+0000 7f9c4064b640 1 -- 172.21.15.70:0/56167759 shutdown_connections 2023-10-08T05:52:13.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.344+0000 7f9c4064b640 1 -- 172.21.15.70:0/56167759 wait complete. 2023-10-08T05:52:13.365 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T05:52:13.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.582+0000 7f436124b640 1 Processor -- start 2023-10-08T05:52:13.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.583+0000 7f436124b640 1 -- start start 2023-10-08T05:52:13.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f436124b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f435c06c3a0 0x7f435c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:13.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f436124b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f435c06db60 0x7f435c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:13.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f436124b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c15b310 con 0x7f435c15aa00 2023-10-08T05:52:13.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f436124b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c0ca230 con 0x7f435c06db60 2023-10-08T05:52:13.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f436124b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c06d290 con 0x7f435c06c3a0 2023-10-08T05:52:13.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f435a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f435c06db60 0x7f435c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:13.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f435a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f435c06db60 0x7f435c06c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35360/0 (socket says 172.21.15.70:35360) 2023-10-08T05:52:13.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f435a59c640 1 -- 172.21.15.70:0/2742963821 learned_addr learned my addr 172.21.15.70:0/2742963821 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:13.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f435a59c640 1 -- 172.21.15.70:0/2742963821 >> v2:172.21.15.181:3300/0 conn(0x7f435c06c3a0 msgr2=0x7f435c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:13.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f435a59c640 1 --2- 172.21.15.70:0/2742963821 >> v2:172.21.15.181:3300/0 conn(0x7f435c06c3a0 0x7f435c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:13.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.584+0000 7f435a59c640 1 -- 172.21.15.70:0/2742963821 >> v1:172.21.15.181:6789/0 conn(0x7f435c15aa00 legacy=0x7f435c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:13.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.585+0000 7f435a59c640 1 -- 172.21.15.70:0/2742963821 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0c9860 con 0x7f435c06db60 2023-10-08T05:52:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.586+0000 7f435a59c640 1 --2- 172.21.15.70:0/2742963821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c06db60 0x7f435c06c780 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f435c0716b0 tx=0x7f434402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280113741d4631d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.586+0000 7f4359d9b640 1 -- 172.21.15.70:0/2742963821 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344031870 con 0x7f435c06db60 2023-10-08T05:52:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.586+0000 7f4359d9b640 1 -- 172.21.15.70:0/2742963821 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43440319d0 con 0x7f435c06db60 2023-10-08T05:52:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.586+0000 7f4359d9b640 1 -- 172.21.15.70:0/2742963821 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4344031d60 con 0x7f435c06db60 2023-10-08T05:52:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.586+0000 7f436124b640 1 -- 172.21.15.70:0/2742963821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c06db60 msgr2=0x7f435c06c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.586+0000 7f436124b640 1 --2- 172.21.15.70:0/2742963821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c06db60 0x7f435c06c780 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f435c0716b0 tx=0x7f434402b910 comp rx=0 tx=0).stop 2023-10-08T05:52:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.587+0000 7f436124b640 1 -- 172.21.15.70:0/2742963821 shutdown_connections 2023-10-08T05:52:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.587+0000 7f436124b640 1 --2- 172.21.15.70:0/2742963821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c06db60 0x7f435c06c780 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.587+0000 7f436124b640 1 --2- 172.21.15.70:0/2742963821 >> v2:172.21.15.181:3300/0 conn(0x7f435c06c3a0 0x7f435c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.587+0000 7f436124b640 1 -- 172.21.15.70:0/2742963821 >> 172.21.15.70:0/2742963821 conn(0x7f435c06a5d0 msgr2=0x7f435c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.587+0000 7f436124b640 1 -- 172.21.15.70:0/2742963821 shutdown_connections 2023-10-08T05:52:13.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.587+0000 7f436124b640 1 -- 172.21.15.70:0/2742963821 wait complete. 2023-10-08T05:52:13.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.588+0000 7f436124b640 1 Processor -- start 2023-10-08T05:52:13.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.588+0000 7f436124b640 1 -- start start 2023-10-08T05:52:13.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.589+0000 7f436124b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f435c06c3a0 0x7f435c0c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:13.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.589+0000 7f436124b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f435c06db60 0x7f435c0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:13.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.589+0000 7f436124b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c15aa00 0x7f435c0c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:13.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.589+0000 7f436124b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0c2f60 con 0x7f435c15aa00 2023-10-08T05:52:13.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.589+0000 7f436124b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0c3240 con 0x7f435c06c3a0 2023-10-08T05:52:13.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.589+0000 7f436124b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0c3520 con 0x7f435c06db60 2023-10-08T05:52:13.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f435b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c15aa00 0x7f435c0c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:13.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f435b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c15aa00 0x7f435c0c2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35362/0 (socket says 172.21.15.70:35362) 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f435b59e640 1 -- 172.21.15.70:0/3058802500 learned_addr learned my addr 172.21.15.70:0/3058802500 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f435a59c640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f435c06db60 0x7f435c0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f435b59e640 1 -- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f435c06db60 msgr2=0x7f435c0c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f435b59e640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f435c06db60 0x7f435c0c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f435b59e640 1 -- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f435c06c3a0 msgr2=0x7f435c0c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f435b59e640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f435c06c3a0 0x7f435c0c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f435b59e640 1 -- 172.21.15.70:0/3058802500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43440094d0 con 0x7f435c15aa00 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f435b59e640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c15aa00 0x7f435c0c2270 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f435400b490 tx=0x7f435400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.590+0000 7f43537fe640 1 -- 172.21.15.70:0/3058802500 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354011830 con 0x7f435c15aa00 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.591+0000 7f43537fe640 1 -- 172.21.15.70:0/3058802500 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43540119b0 con 0x7f435c15aa00 2023-10-08T05:52:13.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.591+0000 7f43537fe640 1 -- 172.21.15.70:0/3058802500 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354011d60 con 0x7f435c15aa00 2023-10-08T05:52:13.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.591+0000 7f436124b640 1 -- 172.21.15.70:0/3058802500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c0c3800 con 0x7f435c15aa00 2023-10-08T05:52:13.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.591+0000 7f436124b640 1 -- 172.21.15.70:0/3058802500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c0c1c30 con 0x7f435c15aa00 2023-10-08T05:52:13.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.592+0000 7f436124b640 1 -- 172.21.15.70:0/3058802500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c1f3770 con 0x7f435c15aa00 2023-10-08T05:52:13.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.593+0000 7f43537fe640 1 -- 172.21.15.70:0/3058802500 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f435401a400 con 0x7f435c15aa00 2023-10-08T05:52:13.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.594+0000 7f43537fe640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f433c06af90 0x7f433c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:13.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.594+0000 7f43537fe640 1 -- 172.21.15.70:0/3058802500 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f4354089ed0 con 0x7f435c15aa00 2023-10-08T05:52:13.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.594+0000 7f435ad9d640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f433c06af90 0x7f433c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.595+0000 7f435ad9d640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f433c06af90 0x7f433c06d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f434c00ab90 tx=0x7f434c009250 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:13.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.598+0000 7f43537fe640 1 -- 172.21.15.70:0/3058802500 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f435c1f3770 con 0x7f435c15aa00 2023-10-08T05:52:13.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:13.786+0000 7f436124b640 1 -- 172.21.15.70:0/3058802500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f435c1f3770 con 0x7f435c15aa00 2023-10-08T05:52:14.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.475+0000 7f43537fe640 1 -- 172.21.15.70:0/3058802500 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) v1 ==== 181+0+0 (secure 0 0 0) 0x7f435c1f3770 con 0x7f435c15aa00 2023-10-08T05:52:14.476 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T05:52:14.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.479+0000 7f436124b640 1 -- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f433c06af90 msgr2=0x7f433c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:14.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.479+0000 7f436124b640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f433c06af90 0x7f433c06d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f434c00ab90 tx=0x7f434c009250 comp rx=0 tx=0).stop 2023-10-08T05:52:14.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.479+0000 7f436124b640 1 -- 172.21.15.70:0/3058802500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c15aa00 msgr2=0x7f435c0c2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:14.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.479+0000 7f436124b640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c15aa00 0x7f435c0c2270 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f435400b490 tx=0x7f435400b950 comp rx=0 tx=0).stop 2023-10-08T05:52:14.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.481+0000 7f436124b640 1 -- 172.21.15.70:0/3058802500 shutdown_connections 2023-10-08T05:52:14.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.481+0000 7f436124b640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f433c06af90 0x7f433c06d450 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f434c00ab90 tx=0x7f434c009250 comp rx=0 tx=0).stop 2023-10-08T05:52:14.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.481+0000 7f436124b640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f435c15aa00 0x7f435c0c2270 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:14.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.481+0000 7f436124b640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f435c06db60 0x7f435c0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:14.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.481+0000 7f436124b640 1 --2- 172.21.15.70:0/3058802500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f435c06c3a0 0x7f435c0c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:14.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.481+0000 7f436124b640 1 -- 172.21.15.70:0/3058802500 >> 172.21.15.70:0/3058802500 conn(0x7f435c06a5d0 msgr2=0x7f435c0cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:14.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.481+0000 7f436124b640 1 -- 172.21.15.70:0/3058802500 shutdown_connections 2023-10-08T05:52:14.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.481+0000 7f436124b640 1 -- 172.21.15.70:0/3058802500 wait complete. 2023-10-08T05:52:14.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T05:52:14.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.719+0000 7f0ee717c640 1 Processor -- start 2023-10-08T05:52:14.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.720+0000 7f0ee717c640 1 -- start start 2023-10-08T05:52:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.721+0000 7f0ee717c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0ee01601b0 0x7f0ee00cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.721+0000 7f0ee717c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0ee0071c30 0x7f0ee0069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.721+0000 7f0ee717c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee00ce210 con 0x7f0ee00720c0 2023-10-08T05:52:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.721+0000 7f0ee717c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee00cdee0 con 0x7f0ee0071c30 2023-10-08T05:52:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.721+0000 7f0ee717c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee0161530 con 0x7f0ee01601b0 2023-10-08T05:52:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.722+0000 7f0edffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0ee0071c30 0x7f0ee0069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.722+0000 7f0edffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0ee0071c30 0x7f0ee0069d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35368/0 (socket says 172.21.15.70:35368) 2023-10-08T05:52:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.722+0000 7f0ee5719640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0ee00720c0 0x7f0ee00ca090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60438/0 (socket says 172.21.15.70:60438) 2023-10-08T05:52:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.722+0000 7f0edffff640 1 -- 172.21.15.70:0/1031098433 learned_addr learned my addr 172.21.15.70:0/1031098433 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.722+0000 7f0ee4f18640 1 --2- 172.21.15.70:0/1031098433 >> v2:172.21.15.181:3300/0 conn(0x7f0ee01601b0 0x7f0ee00cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.722+0000 7f0edffff640 1 -- 172.21.15.70:0/1031098433 >> v2:172.21.15.181:3300/0 conn(0x7f0ee01601b0 msgr2=0x7f0ee00cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.722+0000 7f0edffff640 1 --2- 172.21.15.70:0/1031098433 >> v2:172.21.15.181:3300/0 conn(0x7f0ee01601b0 0x7f0ee00cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.722+0000 7f0edffff640 1 -- 172.21.15.70:0/1031098433 >> v1:172.21.15.70:6789/0 conn(0x7f0ee00720c0 legacy=0x7f0ee00ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.722+0000 7f0edffff640 1 -- 172.21.15.70:0/1031098433 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0161810 con 0x7f0ee0071c30 2023-10-08T05:52:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.722+0000 7f0edf7fe640 1 -- 172.21.15.70:0/1031098433 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767553359 0 0) 0x7f0ee00ce210 con 0x7f0ee00720c0 2023-10-08T05:52:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.723+0000 7f0edffff640 1 --2- 172.21.15.70:0/1031098433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee0071c30 0x7f0ee0069d50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0067110 tx=0x7f0ec802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38818baea014a57 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.723+0000 7f0edf7fe640 1 -- 172.21.15.70:0/1031098433 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8031870 con 0x7f0ee0071c30 2023-10-08T05:52:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.724+0000 7f0edf7fe640 1 -- 172.21.15.70:0/1031098433 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec80319d0 con 0x7f0ee0071c30 2023-10-08T05:52:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.724+0000 7f0edf7fe640 1 -- 172.21.15.70:0/1031098433 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8031d60 con 0x7f0ee0071c30 2023-10-08T05:52:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.724+0000 7f0ee717c640 1 -- 172.21.15.70:0/1031098433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee0071c30 msgr2=0x7f0ee0069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.724+0000 7f0ee717c640 1 --2- 172.21.15.70:0/1031098433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee0071c30 0x7f0ee0069d50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0067110 tx=0x7f0ec802b910 comp rx=0 tx=0).stop 2023-10-08T05:52:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.724+0000 7f0ee717c640 1 -- 172.21.15.70:0/1031098433 shutdown_connections 2023-10-08T05:52:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.724+0000 7f0ee717c640 1 --2- 172.21.15.70:0/1031098433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee0071c30 0x7f0ee0069d50 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.724+0000 7f0ee717c640 1 --2- 172.21.15.70:0/1031098433 >> v2:172.21.15.181:3300/0 conn(0x7f0ee01601b0 0x7f0ee00cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.724+0000 7f0ee717c640 1 -- 172.21.15.70:0/1031098433 >> 172.21.15.70:0/1031098433 conn(0x7f0ee006d430 msgr2=0x7f0ee006d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.724+0000 7f0ee717c640 1 -- 172.21.15.70:0/1031098433 shutdown_connections 2023-10-08T05:52:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.724+0000 7f0ee717c640 1 -- 172.21.15.70:0/1031098433 wait complete. 2023-10-08T05:52:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.726+0000 7f0ee717c640 1 Processor -- start 2023-10-08T05:52:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.726+0000 7f0ee717c640 1 -- start start 2023-10-08T05:52:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.727+0000 7f0ee717c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ee0071c30 0x7f0ee01f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.727+0000 7f0ee717c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee00720c0 0x7f0ee01f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.727+0000 7f0ee717c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee01601b0 0x7f0ee00c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.727+0000 7f0ee717c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00c7cc0 con 0x7f0ee00720c0 2023-10-08T05:52:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.727+0000 7f0ee717c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00b3b00 con 0x7f0ee01601b0 2023-10-08T05:52:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.727+0000 7f0ee717c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00c7f10 con 0x7f0ee0071c30 2023-10-08T05:52:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.727+0000 7f0edffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee00720c0 0x7f0ee01f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.727+0000 7f0edffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee00720c0 0x7f0ee01f3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35374/0 (socket says 172.21.15.70:35374) 2023-10-08T05:52:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.727+0000 7f0edffff640 1 -- 172.21.15.70:0/2942268084 learned_addr learned my addr 172.21.15.70:0/2942268084 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0ee5719640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee01601b0 0x7f0ee00c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0edffff640 1 -- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ee0071c30 msgr2=0x7f0ee01f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0edffff640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ee0071c30 0x7f0ee01f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0edffff640 1 -- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee01601b0 msgr2=0x7f0ee00c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0edffff640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee01601b0 0x7f0ee00c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0edffff640 1 -- 172.21.15.70:0/2942268084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec80094d0 con 0x7f0ee00720c0 2023-10-08T05:52:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0edffff640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee00720c0 0x7f0ee01f3240 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0ec802b950 tx=0x7f0ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0edd7fa640 1 -- 172.21.15.70:0/2942268084 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec803ad30 con 0x7f0ee00720c0 2023-10-08T05:52:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0edd7fa640 1 -- 172.21.15.70:0/2942268084 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec80443f0 con 0x7f0ee00720c0 2023-10-08T05:52:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0ee717c640 1 -- 172.21.15.70:0/2942268084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee00c81f0 con 0x7f0ee00720c0 2023-10-08T05:52:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.728+0000 7f0edd7fa640 1 -- 172.21.15.70:0/2942268084 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec80447a0 con 0x7f0ee00720c0 2023-10-08T05:52:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.729+0000 7f0ee717c640 1 -- 172.21.15.70:0/2942268084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee00b40a0 con 0x7f0ee00720c0 2023-10-08T05:52:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.730+0000 7f0ee717c640 1 -- 172.21.15.70:0/2942268084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee00c83e0 con 0x7f0ee00720c0 2023-10-08T05:52:14.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.730+0000 7f0edd7fa640 1 -- 172.21.15.70:0/2942268084 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0ec804d450 con 0x7f0ee00720c0 2023-10-08T05:52:14.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.731+0000 7f0edd7fa640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0ec006afc0 0x7f0ec006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:14.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.731+0000 7f0edd7fa640 1 -- 172.21.15.70:0/2942268084 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f0ec802d040 con 0x7f0ee00720c0 2023-10-08T05:52:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.732+0000 7f0ee4f18640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0ec006afc0 0x7f0ec006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.733+0000 7f0ee4f18640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0ec006afc0 0x7f0ec006d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00070b0 tx=0x7f0ed0007040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:14.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.736+0000 7f0edd7fa640 1 -- 172.21.15.70:0/2942268084 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ec8076ca0 con 0x7f0ee00720c0 2023-10-08T05:52:14.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:14.910+0000 7f0ee717c640 1 -- 172.21.15.70:0/2942268084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0ee004e160 con 0x7f0ee00720c0 2023-10-08T05:52:15.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.487+0000 7f0edd7fa640 1 -- 172.21.15.70:0/2942268084 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0ec8035020 con 0x7f0ee00720c0 2023-10-08T05:52:15.488 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T05:52:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.496+0000 7f0ee717c640 1 -- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0ec006afc0 msgr2=0x7f0ec006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.496+0000 7f0ee717c640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0ec006afc0 0x7f0ec006d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00070b0 tx=0x7f0ed0007040 comp rx=0 tx=0).stop 2023-10-08T05:52:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.496+0000 7f0ee717c640 1 -- 172.21.15.70:0/2942268084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee00720c0 msgr2=0x7f0ee01f3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.496+0000 7f0ee717c640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee00720c0 0x7f0ee01f3240 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0ec802b950 tx=0x7f0ec80046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.497+0000 7f0ee717c640 1 -- 172.21.15.70:0/2942268084 shutdown_connections 2023-10-08T05:52:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.497+0000 7f0ee717c640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0ec006afc0 0x7f0ec006d480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.497+0000 7f0ee717c640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ee01601b0 0x7f0ee00c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.497+0000 7f0ee717c640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ee00720c0 0x7f0ee01f3240 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.497+0000 7f0ee717c640 1 --2- 172.21.15.70:0/2942268084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ee0071c30 0x7f0ee01f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.497+0000 7f0ee717c640 1 -- 172.21.15.70:0/2942268084 >> 172.21.15.70:0/2942268084 conn(0x7f0ee006d430 msgr2=0x7f0ee0069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.497+0000 7f0ee717c640 1 -- 172.21.15.70:0/2942268084 shutdown_connections 2023-10-08T05:52:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.497+0000 7f0ee717c640 1 -- 172.21.15.70:0/2942268084 wait complete. 2023-10-08T05:52:15.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T05:52:15.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.717+0000 7fa373d41640 1 Processor -- start 2023-10-08T05:52:15.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.717+0000 7fa373d41640 1 -- start start 2023-10-08T05:52:15.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.718+0000 7fa373d41640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa36c0702b0 0x7fa36c070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:15.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.718+0000 7fa373d41640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa36c0b21f0 0x7fa36c0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:15.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa373d41640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36c06c3a0 con 0x7fa36c0ca1a0 2023-10-08T05:52:15.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa373d41640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36c070bd0 con 0x7fa36c0702b0 2023-10-08T05:52:15.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa373d41640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36c06c640 con 0x7fa36c0b21f0 2023-10-08T05:52:15.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa371add640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa36c0702b0 0x7fa36c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:15.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa371add640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa36c0702b0 0x7fa36c070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35378/0 (socket says 172.21.15.70:35378) 2023-10-08T05:52:15.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa371add640 1 -- 172.21.15.70:0/3080226929 learned_addr learned my addr 172.21.15.70:0/3080226929 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:15.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa3712dc640 1 --2- 172.21.15.70:0/3080226929 >> v2:172.21.15.181:3300/0 conn(0x7fa36c0b21f0 0x7fa36c0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:15.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa371add640 1 -- 172.21.15.70:0/3080226929 >> v2:172.21.15.181:3300/0 conn(0x7fa36c0b21f0 msgr2=0x7fa36c0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:15.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa371add640 1 --2- 172.21.15.70:0/3080226929 >> v2:172.21.15.181:3300/0 conn(0x7fa36c0b21f0 0x7fa36c0c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa371add640 1 -- 172.21.15.70:0/3080226929 >> v1:172.21.15.181:6789/0 conn(0x7fa36c0ca1a0 legacy=0x7fa36c06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:15.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.719+0000 7fa371add640 1 -- 172.21.15.70:0/3080226929 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c06c920 con 0x7fa36c0702b0 2023-10-08T05:52:15.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.721+0000 7fa371add640 1 --2- 172.21.15.70:0/3080226929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0702b0 0x7fa36c070690 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0af860 tx=0x7fa36802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90eeb3580b01f29 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.721+0000 7fa370adb640 1 -- 172.21.15.70:0/3080226929 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368031870 con 0x7fa36c0702b0 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.721+0000 7fa370adb640 1 -- 172.21.15.70:0/3080226929 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3680319d0 con 0x7fa36c0702b0 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.721+0000 7fa370adb640 1 -- 172.21.15.70:0/3080226929 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368031d60 con 0x7fa36c0702b0 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.722+0000 7fa373d41640 1 -- 172.21.15.70:0/3080226929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0702b0 msgr2=0x7fa36c070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.722+0000 7fa373d41640 1 --2- 172.21.15.70:0/3080226929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0702b0 0x7fa36c070690 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0af860 tx=0x7fa36802b910 comp rx=0 tx=0).stop 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.722+0000 7fa373d41640 1 -- 172.21.15.70:0/3080226929 shutdown_connections 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.722+0000 7fa373d41640 1 --2- 172.21.15.70:0/3080226929 >> v2:172.21.15.181:3300/0 conn(0x7fa36c0b21f0 0x7fa36c0c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.722+0000 7fa373d41640 1 --2- 172.21.15.70:0/3080226929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0702b0 0x7fa36c070690 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.722+0000 7fa373d41640 1 -- 172.21.15.70:0/3080226929 >> 172.21.15.70:0/3080226929 conn(0x7fa36c06a5d0 msgr2=0x7fa36c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.722+0000 7fa373d41640 1 -- 172.21.15.70:0/3080226929 shutdown_connections 2023-10-08T05:52:15.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.722+0000 7fa373d41640 1 -- 172.21.15.70:0/3080226929 wait complete. 2023-10-08T05:52:15.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.724+0000 7fa373d41640 1 Processor -- start 2023-10-08T05:52:15.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.724+0000 7fa373d41640 1 -- start start 2023-10-08T05:52:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa373d41640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa36c0702b0 0x7fa36c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa373d41640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0b21f0 0x7fa36c14c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa373d41640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa36c0ca1a0 0x7fa36c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa373d41640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0cd590 con 0x7fa36c0b21f0 2023-10-08T05:52:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa373d41640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c14e900 con 0x7fa36c0702b0 2023-10-08T05:52:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa373d41640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0cd7f0 con 0x7fa36c0ca1a0 2023-10-08T05:52:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa3712dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0b21f0 0x7fa36c14c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa3712dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0b21f0 0x7fa36c14c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35388/0 (socket says 172.21.15.70:35388) 2023-10-08T05:52:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa3712dc640 1 -- 172.21.15.70:0/3933385779 learned_addr learned my addr 172.21.15.70:0/3933385779 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa3712dc640 1 -- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa36c0ca1a0 msgr2=0x7fa36c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:15.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa3712dc640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa36c0ca1a0 0x7fa36c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa3712dc640 1 -- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa36c0702b0 msgr2=0x7fa36c0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:15.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa3712dc640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa36c0702b0 0x7fa36c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.725+0000 7fa3712dc640 1 -- 172.21.15.70:0/3933385779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3680094d0 con 0x7fa36c0b21f0 2023-10-08T05:52:15.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.726+0000 7fa3712dc640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0b21f0 0x7fa36c14c750 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c7f0 tx=0x7fa35c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:15.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.726+0000 7fa3627fc640 1 -- 172.21.15.70:0/3933385779 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c004220 con 0x7fa36c0b21f0 2023-10-08T05:52:15.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.726+0000 7fa3627fc640 1 -- 172.21.15.70:0/3933385779 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c0043a0 con 0x7fa36c0b21f0 2023-10-08T05:52:15.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.726+0000 7fa373d41640 1 -- 172.21.15.70:0/3933385779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c0cdad0 con 0x7fa36c0b21f0 2023-10-08T05:52:15.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.726+0000 7fa3627fc640 1 -- 172.21.15.70:0/3933385779 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c007860 con 0x7fa36c0b21f0 2023-10-08T05:52:15.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.726+0000 7fa373d41640 1 -- 172.21.15.70:0/3933385779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c0ce090 con 0x7fa36c0b21f0 2023-10-08T05:52:15.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.727+0000 7fa373d41640 1 -- 172.21.15.70:0/3933385779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36c0cf6b0 con 0x7fa36c0b21f0 2023-10-08T05:52:15.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.729+0000 7fa3627fc640 1 -- 172.21.15.70:0/3933385779 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa35c007bd0 con 0x7fa36c0b21f0 2023-10-08T05:52:15.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.730+0000 7fa3627fc640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fa35006af90 0x7fa35006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:15.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.730+0000 7fa3627fc640 1 -- 172.21.15.70:0/3933385779 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fa35c089a90 con 0x7fa36c0b21f0 2023-10-08T05:52:15.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.731+0000 7fa371add640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fa35006af90 0x7fa35006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:15.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.732+0000 7fa371add640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fa35006af90 0x7fa35006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa368036000 tx=0x7fa368030000 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:15.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.735+0000 7fa3627fc640 1 -- 172.21.15.70:0/3933385779 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa35c0546e0 con 0x7fa36c0b21f0 2023-10-08T05:52:15.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.929+0000 7fa373d41640 1 -- 172.21.15.70:0/3933385779 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa36c14eb50 con 0x7fa36c0b21f0 2023-10-08T05:52:15.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.934+0000 7fa3627fc640 1 -- 172.21.15.70:0/3933385779 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa36c14eb50 con 0x7fa36c0b21f0 2023-10-08T05:52:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 -- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fa35006af90 msgr2=0x7fa35006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fa35006af90 0x7fa35006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa368036000 tx=0x7fa368030000 comp rx=0 tx=0).stop 2023-10-08T05:52:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 -- 172.21.15.70:0/3933385779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0b21f0 msgr2=0x7fa36c14c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0b21f0 0x7fa36c14c750 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c7f0 tx=0x7fa35c00ccb0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 -- 172.21.15.70:0/3933385779 shutdown_connections 2023-10-08T05:52:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fa35006af90 0x7fa35006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa36c0ca1a0 0x7fa36c15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa36c0b21f0 0x7fa36c14c750 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 --2- 172.21.15.70:0/3933385779 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa36c0702b0 0x7fa36c0cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:15.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 -- 172.21.15.70:0/3933385779 >> 172.21.15.70:0/3933385779 conn(0x7fa36c06a5d0 msgr2=0x7fa36c153ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:15.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 -- 172.21.15.70:0/3933385779 shutdown_connections 2023-10-08T05:52:15.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:15.938+0000 7fa373d41640 1 -- 172.21.15.70:0/3933385779 wait complete. 2023-10-08T05:52:15.958 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T05:52:16.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.165+0000 7f2362e90640 1 Processor -- start 2023-10-08T05:52:16.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.165+0000 7f2362e90640 1 -- start start 2023-10-08T05:52:16.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.167+0000 7f2362e90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f235c15aad0 0x7f235c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.167+0000 7f2362e90640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f235c06d410 0x7f235c0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.167+0000 7f2362e90640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235c15bf30 con 0x7f235c0c82b0 2023-10-08T05:52:16.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.167+0000 7f2362e90640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f235c15b320 con 0x7f235c06d410 2023-10-08T05:52:16.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.167+0000 7f2362e90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c15b600 con 0x7f235c15aad0 2023-10-08T05:52:16.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.167+0000 7f2353fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f235c0c82b0 0x7f235c06cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60442/0 (socket says 172.21.15.70:60442) 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.167+0000 7f2353fff640 1 -- 172.21.15.70:0/3272202348 learned_addr learned my addr 172.21.15.70:0/3272202348 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f236142d640 1 --2- 172.21.15.70:0/3272202348 >> v2:172.21.15.181:3300/0 conn(0x7f235c06d410 0x7f235c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757585599 0 0) 0x7f235c15bf30 con 0x7f235c0c82b0 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2344003520 con 0x7f235c0c82b0 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f2360c2c640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f235c15aad0 0x7f235c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3563761947 0 0) 0x7f2344003520 con 0x7f235c0c82b0 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f235c15aad0 msgr2=0x7f235c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f235c15aad0 0x7f235c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 >> v2:172.21.15.181:3300/0 conn(0x7f235c06d410 msgr2=0x7f235c0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 --2- 172.21.15.70:0/3272202348 >> v2:172.21.15.181:3300/0 conn(0x7f235c06d410 0x7f235c0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c15b8e0 con 0x7f235c0c82b0 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f234c001670 con 0x7f235c0c82b0 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 >> v1:172.21.15.70:6789/0 conn(0x7f235c0c82b0 legacy=0x7f235c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2344003970 0x7f2344003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23440047c0 0x7f2344004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f235c15aad0 0x7f2344005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c15b320 con 0x7f23440047c0 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c15b600 con 0x7f2344003970 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234400b9d0 con 0x7f235c15aad0 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.168+0000 7f2353fff640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23440047c0 0x7f2344004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.169+0000 7f2360c2c640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2344003970 0x7f2344003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.169+0000 7f2353fff640 1 -- 172.21.15.70:0/3272202348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f235c15aad0 msgr2=0x7f2344005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:16.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.169+0000 7f2353fff640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f235c15aad0 0x7f2344005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.169+0000 7f2353fff640 1 -- 172.21.15.70:0/3272202348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2344003970 msgr2=0x7f2344003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:16.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.169+0000 7f2353fff640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2344003970 0x7f2344003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.169+0000 7f2353fff640 1 -- 172.21.15.70:0/3272202348 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2344003520 con 0x7f23440047c0 2023-10-08T05:52:16.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.170+0000 7f2353fff640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23440047c0 0x7f2344004c40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c190 tx=0x7f23440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd78519d59d77 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:16.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.170+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c005ad0 con 0x7f23440047c0 2023-10-08T05:52:16.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.170+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234c005c50 con 0x7f23440047c0 2023-10-08T05:52:16.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.170+0000 7f23537fe640 1 -- 172.21.15.70:0/3272202348 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234c02c810 con 0x7f23440047c0 2023-10-08T05:52:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.171+0000 7f2362e90640 1 -- 172.21.15.70:0/3272202348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23440047c0 msgr2=0x7f2344004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.171+0000 7f2362e90640 1 --2- 172.21.15.70:0/3272202348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23440047c0 0x7f2344004c40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f234c00c190 tx=0x7f23440057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.171+0000 7f236142d640 1 -- 172.21.15.70:0/3272202348 reap_dead start 2023-10-08T05:52:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.171+0000 7f2362e90640 1 -- 172.21.15.70:0/3272202348 shutdown_connections 2023-10-08T05:52:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.171+0000 7f2362e90640 1 -- 172.21.15.70:0/3272202348 >> 172.21.15.70:0/3272202348 conn(0x7f235c069d50 msgr2=0x7f235c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.171+0000 7f2362e90640 1 -- 172.21.15.70:0/3272202348 shutdown_connections 2023-10-08T05:52:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.171+0000 7f2362e90640 1 -- 172.21.15.70:0/3272202348 wait complete. 2023-10-08T05:52:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.173+0000 7f2362e90640 1 Processor -- start 2023-10-08T05:52:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.173+0000 7f2362e90640 1 -- start start 2023-10-08T05:52:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f2362e90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23440047c0 0x7f235c149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f2362e90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c157920 0x7f235c0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f2362e90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f235c14ac80 0x7f235c156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f2362e90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c14b660 con 0x7f235c14ac80 2023-10-08T05:52:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f2362e90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f235c14a090 con 0x7f235c157920 2023-10-08T05:52:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f2362e90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f235c0b0e10 con 0x7f23440047c0 2023-10-08T05:52:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f236142d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f235c14ac80 0x7f235c156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f236142d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f235c14ac80 0x7f235c156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35406/0 (socket says 172.21.15.70:35406) 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f236142d640 1 -- 172.21.15.70:0/1867043183 learned_addr learned my addr 172.21.15.70:0/1867043183 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f236142d640 1 -- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23440047c0 msgr2=0x7f235c149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f236142d640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23440047c0 0x7f235c149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f236142d640 1 -- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c157920 msgr2=0x7f235c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f236142d640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c157920 0x7f235c0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.174+0000 7f236142d640 1 -- 172.21.15.70:0/1867043183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c0b10f0 con 0x7f235c14ac80 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.175+0000 7f236142d640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f235c14ac80 0x7f235c156c30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f235c15f610 tx=0x7f235400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.175+0000 7f23517fa640 1 -- 172.21.15.70:0/1867043183 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235400c680 con 0x7f235c14ac80 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.175+0000 7f23517fa640 1 -- 172.21.15.70:0/1867043183 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235400c7e0 con 0x7f235c14ac80 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.175+0000 7f2362e90640 1 -- 172.21.15.70:0/1867043183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f235c0b13d0 con 0x7f235c14ac80 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.175+0000 7f23517fa640 1 -- 172.21.15.70:0/1867043183 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235400cb90 con 0x7f235c14ac80 2023-10-08T05:52:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.175+0000 7f2362e90640 1 -- 172.21.15.70:0/1867043183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f235c0c9620 con 0x7f235c14ac80 2023-10-08T05:52:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.176+0000 7f2362e90640 1 -- 172.21.15.70:0/1867043183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f235c0b18f0 con 0x7f235c14ac80 2023-10-08T05:52:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.177+0000 7f23517fa640 1 -- 172.21.15.70:0/1867043183 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f235400ccf0 con 0x7f235c14ac80 2023-10-08T05:52:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.178+0000 7f23517fa640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f234806afc0 0x7f234806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.178+0000 7f23517fa640 1 -- 172.21.15.70:0/1867043183 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f2354089070 con 0x7f235c14ac80 2023-10-08T05:52:16.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.179+0000 7f2360c2c640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f234806afc0 0x7f234806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.180+0000 7f2360c2c640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f234806afc0 0x7f234806d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f23580070b0 tx=0x7f235800b040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.182+0000 7f23517fa640 1 -- 172.21.15.70:0/1867043183 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f235c0b18f0 con 0x7f235c14ac80 2023-10-08T05:52:16.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.371+0000 7f2362e90640 1 -- 172.21.15.70:0/1867043183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f235c0b18f0 con 0x7f235c14ac80 2023-10-08T05:52:16.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.375+0000 7f23517fa640 1 -- 172.21.15.70:0/1867043183 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f235c0b18f0 con 0x7f235c14ac80 2023-10-08T05:52:16.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.378+0000 7f2362e90640 1 -- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f234806afc0 msgr2=0x7f234806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.378+0000 7f2362e90640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f234806afc0 0x7f234806d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f23580070b0 tx=0x7f235800b040 comp rx=0 tx=0).stop 2023-10-08T05:52:16.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.378+0000 7f2362e90640 1 -- 172.21.15.70:0/1867043183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f235c14ac80 msgr2=0x7f235c156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.378+0000 7f2362e90640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f235c14ac80 0x7f235c156c30 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f235c15f610 tx=0x7f235400ec00 comp rx=0 tx=0).stop 2023-10-08T05:52:16.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.379+0000 7f2362e90640 1 -- 172.21.15.70:0/1867043183 shutdown_connections 2023-10-08T05:52:16.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.379+0000 7f2362e90640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f234806afc0 0x7f234806d480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.379+0000 7f2362e90640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f235c14ac80 0x7f235c156c30 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.379+0000 7f2362e90640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f235c157920 0x7f235c0c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.379+0000 7f2362e90640 1 --2- 172.21.15.70:0/1867043183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23440047c0 0x7f235c149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.379+0000 7f2362e90640 1 -- 172.21.15.70:0/1867043183 >> 172.21.15.70:0/1867043183 conn(0x7f235c069d50 msgr2=0x7f235c0cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:16.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.379+0000 7f2362e90640 1 -- 172.21.15.70:0/1867043183 shutdown_connections 2023-10-08T05:52:16.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.379+0000 7f2362e90640 1 -- 172.21.15.70:0/1867043183 wait complete. 2023-10-08T05:52:16.396 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T05:52:16.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.609+0000 7fd902155640 1 Processor -- start 2023-10-08T05:52:16.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.609+0000 7fd902155640 1 -- start start 2023-10-08T05:52:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd902155640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc070cc0 0x7fd8fc0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd902155640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8fc0cb450 0x7fd8fc0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd902155640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc071140 con 0x7fd8fc160660 2023-10-08T05:52:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd902155640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8fc06e150 con 0x7fd8fc0cb450 2023-10-08T05:52:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd902155640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0cb8c0 con 0x7fd8fc070cc0 2023-10-08T05:52:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd8fb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd8fc160660 0x7fd8fc0c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60450/0 (socket says 172.21.15.70:60450) 2023-10-08T05:52:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd8fb7fe640 1 -- 172.21.15.70:0/550619496 learned_addr learned my addr 172.21.15.70:0/550619496 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd8faffd640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc070cc0 0x7fd8fc0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903589057 0 0) 0x7fd8fc071140 con 0x7fd8fc160660 2023-10-08T05:52:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd8fbfff640 1 --2- 172.21.15.70:0/550619496 >> v2:172.21.15.181:3300/0 conn(0x7fd8fc0cb450 0x7fd8fc0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.611+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e4003520 con 0x7fd8fc160660 2023-10-08T05:52:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1829183925 0 0) 0x7fd8e4003520 con 0x7fd8fc160660 2023-10-08T05:52:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc070cc0 msgr2=0x7fd8fc0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc070cc0 0x7fd8fc0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 >> v2:172.21.15.181:3300/0 conn(0x7fd8fc0cb450 msgr2=0x7fd8fc0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 --2- 172.21.15.70:0/550619496 >> v2:172.21.15.181:3300/0 conn(0x7fd8fc0cb450 0x7fd8fc0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc06de70 con 0x7fd8fc160660 2023-10-08T05:52:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd8ec001670 con 0x7fd8fc160660 2023-10-08T05:52:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fbfff640 1 --2- 172.21.15.70:0/550619496 >> v2:172.21.15.181:3300/0 conn(0x7fd8fc0cb450 0x7fd8fc0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:52:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8faffd640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc070cc0 0x7fd8fc0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:52:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 >> v1:172.21.15.70:6789/0 conn(0x7fd8fc160660 legacy=0x7fd8fc0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4003970 0x7fd8e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc070cc0 0x7fd8e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 0x7fd8e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc06e150 con 0x7fd8e4004d00 2023-10-08T05:52:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0cb8c0 con 0x7fd8e4003970 2023-10-08T05:52:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e400ba00 con 0x7fd8fc070cc0 2023-10-08T05:52:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.612+0000 7fd8fbfff640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 0x7fd8e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.613+0000 7fd8fb7fe640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4003970 0x7fd8e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.613+0000 7fd8faffd640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc070cc0 0x7fd8e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.613+0000 7fd8fbfff640 1 -- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc070cc0 msgr2=0x7fd8e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:16.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.613+0000 7fd8fbfff640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc070cc0 0x7fd8e40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.613+0000 7fd8fbfff640 1 -- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4003970 msgr2=0x7fd8e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:16.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.613+0000 7fd8fbfff640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8e4003970 0x7fd8e4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.613+0000 7fd8fbfff640 1 -- 172.21.15.70:0/550619496 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4003520 con 0x7fd8e4004d00 2023-10-08T05:52:16.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.614+0000 7fd8fbfff640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 0x7fd8e4005180 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40073e0 tx=0x7fd8e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424041ad88300f35 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:16.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.614+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f402cdc0 con 0x7fd8e4004d00 2023-10-08T05:52:16.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.614+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f4034440 con 0x7fd8e4004d00 2023-10-08T05:52:16.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.614+0000 7fd8fa7fc640 1 -- 172.21.15.70:0/550619496 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f40347b0 con 0x7fd8e4004d00 2023-10-08T05:52:16.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.615+0000 7fd902155640 1 -- 172.21.15.70:0/550619496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 msgr2=0x7fd8e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.615+0000 7fd902155640 1 --2- 172.21.15.70:0/550619496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 0x7fd8e4005180 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40073e0 tx=0x7fd8e40057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.615+0000 7fd8fbfff640 1 -- 172.21.15.70:0/550619496 reap_dead start 2023-10-08T05:52:16.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.615+0000 7fd902155640 1 -- 172.21.15.70:0/550619496 shutdown_connections 2023-10-08T05:52:16.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.615+0000 7fd902155640 1 -- 172.21.15.70:0/550619496 >> 172.21.15.70:0/550619496 conn(0x7fd8fc06a5d0 msgr2=0x7fd8fc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:16.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.615+0000 7fd902155640 1 -- 172.21.15.70:0/550619496 shutdown_connections 2023-10-08T05:52:16.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.615+0000 7fd902155640 1 -- 172.21.15.70:0/550619496 wait complete. 2023-10-08T05:52:16.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.617+0000 7fd902155640 1 Processor -- start 2023-10-08T05:52:16.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.618+0000 7fd902155640 1 -- start start 2023-10-08T05:52:16.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd902155640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 0x7fd8fc0c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd902155640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc154f60 0x7fd8fc0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd902155640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8fc1f5da0 0x7fd8fc1f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd902155640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc1f6760 con 0x7fd8e4004d00 2023-10-08T05:52:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd902155640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0c34f0 con 0x7fd8fc1f5da0 2023-10-08T05:52:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd902155640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0c82d0 con 0x7fd8fc154f60 2023-10-08T05:52:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd8fb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 0x7fd8fc0c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd8fb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 0x7fd8fc0c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35418/0 (socket says 172.21.15.70:35418) 2023-10-08T05:52:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd8fb7fe640 1 -- 172.21.15.70:0/2369688730 learned_addr learned my addr 172.21.15.70:0/2369688730 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd8fb7fe640 1 -- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc154f60 msgr2=0x7fd8fc0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd8fb7fe640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc154f60 0x7fd8fc0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd8fb7fe640 1 -- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8fc1f5da0 msgr2=0x7fd8fc1f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.619+0000 7fd8fbfff640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8fc1f5da0 0x7fd8fc1f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.620+0000 7fd8fb7fe640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8fc1f5da0 0x7fd8fc1f6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.620+0000 7fd8fb7fe640 1 -- 172.21.15.70:0/2369688730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc1553e0 con 0x7fd8e4004d00 2023-10-08T05:52:16.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.620+0000 7fd8fb7fe640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 0x7fd8fc0c2250 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e760 tx=0x7fd8ec00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:16.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.620+0000 7fd8e3fff640 1 -- 172.21.15.70:0/2369688730 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00ee70 con 0x7fd8e4004d00 2023-10-08T05:52:16.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.620+0000 7fd8e3fff640 1 -- 172.21.15.70:0/2369688730 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec00c8d0 con 0x7fd8e4004d00 2023-10-08T05:52:16.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.620+0000 7fd902155640 1 -- 172.21.15.70:0/2369688730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc0c28e0 con 0x7fd8e4004d00 2023-10-08T05:52:16.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.620+0000 7fd8e3fff640 1 -- 172.21.15.70:0/2369688730 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec00cc80 con 0x7fd8e4004d00 2023-10-08T05:52:16.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.620+0000 7fd902155640 1 -- 172.21.15.70:0/2369688730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc0c2ea0 con 0x7fd8e4004d00 2023-10-08T05:52:16.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.622+0000 7fd902155640 1 -- 172.21.15.70:0/2369688730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc04e160 con 0x7fd8e4004d00 2023-10-08T05:52:16.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.623+0000 7fd8e3fff640 1 -- 172.21.15.70:0/2369688730 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd8ec01d460 con 0x7fd8e4004d00 2023-10-08T05:52:16.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.624+0000 7fd8e3fff640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd8e806af90 0x7fd8e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:16.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.624+0000 7fd8e3fff640 1 -- 172.21.15.70:0/2369688730 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fd8ec089820 con 0x7fd8e4004d00 2023-10-08T05:52:16.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.625+0000 7fd8faffd640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd8e806af90 0x7fd8e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:16.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.626+0000 7fd8faffd640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd8e806af90 0x7fd8e806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0c7b20 tx=0x7fd8f0006df0 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:16.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.629+0000 7fd8e3fff640 1 -- 172.21.15.70:0/2369688730 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8ec0546a0 con 0x7fd8e4004d00 2023-10-08T05:52:16.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.824+0000 7fd902155640 1 -- 172.21.15.70:0/2369688730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd8fc15efd0 con 0x7fd8e4004d00 2023-10-08T05:52:16.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.831+0000 7fd8e3fff640 1 -- 172.21.15.70:0/2369688730 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd8ec057cc0 con 0x7fd8e4004d00 2023-10-08T05:52:16.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.833+0000 7fd902155640 1 -- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd8e806af90 msgr2=0x7fd8e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.833+0000 7fd902155640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd8e806af90 0x7fd8e806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0c7b20 tx=0x7fd8f0006df0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.833+0000 7fd902155640 1 -- 172.21.15.70:0/2369688730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 msgr2=0x7fd8fc0c2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:16.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.833+0000 7fd902155640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 0x7fd8fc0c2250 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00e760 tx=0x7fd8ec00ec20 comp rx=0 tx=0).stop 2023-10-08T05:52:16.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.833+0000 7fd902155640 1 -- 172.21.15.70:0/2369688730 shutdown_connections 2023-10-08T05:52:16.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.833+0000 7fd902155640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd8e806af90 0x7fd8e806d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.833+0000 7fd902155640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8fc1f5da0 0x7fd8fc1f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.833+0000 7fd902155640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8fc154f60 0x7fd8fc0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.833+0000 7fd902155640 1 --2- 172.21.15.70:0/2369688730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8e4004d00 0x7fd8fc0c2250 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:16.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.833+0000 7fd902155640 1 -- 172.21.15.70:0/2369688730 >> 172.21.15.70:0/2369688730 conn(0x7fd8fc06a5d0 msgr2=0x7fd8fc15f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:16.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.834+0000 7fd902155640 1 -- 172.21.15.70:0/2369688730 shutdown_connections 2023-10-08T05:52:16.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:16.834+0000 7fd902155640 1 -- 172.21.15.70:0/2369688730 wait complete. 2023-10-08T05:52:16.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-08T05:52:17.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.067+0000 7f3bcdd33640 1 Processor -- start 2023-10-08T05:52:17.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.067+0000 7f3bcdd33640 1 -- start start 2023-10-08T05:52:17.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.069+0000 7f3bcdd33640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3bc815ab40 0x7f3bc80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.069+0000 7f3bcdd33640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3bc80cb580 0x7f3bc80adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.069+0000 7f3bcdd33640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc815b2b0 con 0x7f3bc80c90b0 2023-10-08T05:52:17.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.069+0000 7f3bcdd33640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc815bf30 con 0x7f3bc815ab40 2023-10-08T05:52:17.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.069+0000 7f3bcdd33640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc815f7a0 con 0x7f3bc80cb580 2023-10-08T05:52:17.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.069+0000 7f3bc77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3bc815ab40 0x7f3bc80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.069+0000 7f3bc77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3bc815ab40 0x7f3bc80ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35432/0 (socket says 172.21.15.70:35432) 2023-10-08T05:52:17.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.069+0000 7f3bc77fe640 1 -- 172.21.15.70:0/1948220834 learned_addr learned my addr 172.21.15.70:0/1948220834 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:17.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.069+0000 7f3bc7fff640 1 --2- 172.21.15.70:0/1948220834 >> v2:172.21.15.181:3300/0 conn(0x7f3bc80cb580 0x7f3bc80adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.070+0000 7f3bc77fe640 1 -- 172.21.15.70:0/1948220834 >> v2:172.21.15.181:3300/0 conn(0x7f3bc80cb580 msgr2=0x7f3bc80adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:17.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.070+0000 7f3bc77fe640 1 --2- 172.21.15.70:0/1948220834 >> v2:172.21.15.181:3300/0 conn(0x7f3bc80cb580 0x7f3bc80adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.070+0000 7f3bc77fe640 1 -- 172.21.15.70:0/1948220834 >> v1:172.21.15.181:6789/0 conn(0x7f3bc80c90b0 legacy=0x7f3bc80cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:17.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.070+0000 7f3bc77fe640 1 -- 172.21.15.70:0/1948220834 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc815b590 con 0x7f3bc815ab40 2023-10-08T05:52:17.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.071+0000 7f3bc77fe640 1 --2- 172.21.15.70:0/1948220834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc815ab40 0x7f3bc80ad720 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80b0ec0 tx=0x7f3bb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb229ab54d64405 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:17.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.071+0000 7f3bc6ffd640 1 -- 172.21.15.70:0/1948220834 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8031870 con 0x7f3bc815ab40 2023-10-08T05:52:17.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.071+0000 7f3bc6ffd640 1 -- 172.21.15.70:0/1948220834 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb80319d0 con 0x7f3bc815ab40 2023-10-08T05:52:17.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.071+0000 7f3bc6ffd640 1 -- 172.21.15.70:0/1948220834 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8031d60 con 0x7f3bc815ab40 2023-10-08T05:52:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.072+0000 7f3bcdd33640 1 -- 172.21.15.70:0/1948220834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc815ab40 msgr2=0x7f3bc80ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.072+0000 7f3bcdd33640 1 --2- 172.21.15.70:0/1948220834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc815ab40 0x7f3bc80ad720 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3bc80b0ec0 tx=0x7f3bb802b910 comp rx=0 tx=0).stop 2023-10-08T05:52:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.072+0000 7f3bcdd33640 1 -- 172.21.15.70:0/1948220834 shutdown_connections 2023-10-08T05:52:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.072+0000 7f3bcdd33640 1 --2- 172.21.15.70:0/1948220834 >> v2:172.21.15.181:3300/0 conn(0x7f3bc80cb580 0x7f3bc80adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.072+0000 7f3bcdd33640 1 --2- 172.21.15.70:0/1948220834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc815ab40 0x7f3bc80ad720 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.072+0000 7f3bcdd33640 1 -- 172.21.15.70:0/1948220834 >> 172.21.15.70:0/1948220834 conn(0x7f3bc806a380 msgr2=0x7f3bc80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.072+0000 7f3bcdd33640 1 -- 172.21.15.70:0/1948220834 shutdown_connections 2023-10-08T05:52:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.072+0000 7f3bcdd33640 1 -- 172.21.15.70:0/1948220834 wait complete. 2023-10-08T05:52:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.074+0000 7f3bcdd33640 1 Processor -- start 2023-10-08T05:52:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.074+0000 7f3bcdd33640 1 -- start start 2023-10-08T05:52:17.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.075+0000 7f3bcdd33640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc80c90b0 0x7f3bc80c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.075+0000 7f3bcdd33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc80cb580 0x7f3bc8140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.075+0000 7f3bcdd33640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc815ab40 0x7f3bc8159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.075+0000 7f3bcdd33640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80c71a0 con 0x7f3bc80cb580 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.075+0000 7f3bcdd33640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80c7ae0 con 0x7f3bc80c90b0 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.075+0000 7f3bcdd33640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80c6520 con 0x7f3bc815ab40 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.075+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc80cb580 0x7f3bc8140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.075+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc80cb580 0x7f3bc8140f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35446/0 (socket says 172.21.15.70:35446) 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.075+0000 7f3bbffff640 1 -- 172.21.15.70:0/962103627 learned_addr learned my addr 172.21.15.70:0/962103627 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.075+0000 7f3bc77fe640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc80c90b0 0x7f3bc80c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bc7fff640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc815ab40 0x7f3bc8159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bbffff640 1 -- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc815ab40 msgr2=0x7f3bc8159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bbffff640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc815ab40 0x7f3bc8159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bbffff640 1 -- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc80c90b0 msgr2=0x7f3bc80c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bbffff640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc80c90b0 0x7f3bc80c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bbffff640 1 -- 172.21.15.70:0/962103627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb80094d0 con 0x7f3bc80cb580 2023-10-08T05:52:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bc77fe640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc80c90b0 0x7f3bc80c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bbffff640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc80cb580 0x7f3bc8140f60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000c400 tx=0x7f3bb000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bc4ff9640 1 -- 172.21.15.70:0/962103627 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00035f0 con 0x7f3bc80cb580 2023-10-08T05:52:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bc4ff9640 1 -- 172.21.15.70:0/962103627 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb0003750 con 0x7f3bc80cb580 2023-10-08T05:52:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bc4ff9640 1 -- 172.21.15.70:0/962103627 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0003b40 con 0x7f3bc80cb580 2023-10-08T05:52:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bcdd33640 1 -- 172.21.15.70:0/962103627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc80c6800 con 0x7f3bc80cb580 2023-10-08T05:52:17.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.076+0000 7f3bcdd33640 1 -- 172.21.15.70:0/962103627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc80c6dc0 con 0x7f3bc80cb580 2023-10-08T05:52:17.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.078+0000 7f3bcdd33640 1 -- 172.21.15.70:0/962103627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc80c7410 con 0x7f3bc80cb580 2023-10-08T05:52:17.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.079+0000 7f3bc4ff9640 1 -- 172.21.15.70:0/962103627 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3bb0003ca0 con 0x7f3bc80cb580 2023-10-08T05:52:17.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.079+0000 7f3bc4ff9640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3ba806af90 0x7f3ba806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.080+0000 7f3bc4ff9640 1 -- 172.21.15.70:0/962103627 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f3bb0088a00 con 0x7f3bc80cb580 2023-10-08T05:52:17.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.080+0000 7f3bc77fe640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3ba806af90 0x7f3ba806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.081+0000 7f3bc77fe640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3ba806af90 0x7f3ba806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8036000 tx=0x7f3bb803c040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:17.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.084+0000 7f3bc4ff9640 1 -- 172.21.15.70:0/962103627 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3bb00539c0 con 0x7f3bc80cb580 2023-10-08T05:52:17.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.355+0000 7f3bcdd33640 1 -- 172.21.15.70:0/962103627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f3bc8067490 con 0x7f3bc80cb580 2023-10-08T05:52:17.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.500+0000 7f3bc4ff9640 1 -- 172.21.15.70:0/962103627 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v591) v1 ==== 154+0+0 (secure 0 0 0) 0x7f3bb00170e0 con 0x7f3bc80cb580 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 -- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3ba806af90 msgr2=0x7f3ba806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3ba806af90 0x7f3ba806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8036000 tx=0x7f3bb803c040 comp rx=0 tx=0).stop 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 -- 172.21.15.70:0/962103627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc80cb580 msgr2=0x7f3bc8140f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc80cb580 0x7f3bc8140f60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000c400 tx=0x7f3bb000c8c0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 -- 172.21.15.70:0/962103627 shutdown_connections 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3ba806af90 0x7f3ba806d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bc815ab40 0x7f3bc8159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bc80cb580 0x7f3bc8140f60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 --2- 172.21.15.70:0/962103627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bc80c90b0 0x7f3bc80c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 -- 172.21.15.70:0/962103627 >> 172.21.15.70:0/962103627 conn(0x7f3bc806a380 msgr2=0x7f3bc80701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 -- 172.21.15.70:0/962103627 shutdown_connections 2023-10-08T05:52:17.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.503+0000 7f3bcdd33640 1 -- 172.21.15.70:0/962103627 wait complete. 2023-10-08T05:52:17.522 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-10-08T05:52:17.523 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-10-08T05:52:17.523 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T05:52:17.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:17.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.737+0000 7f3ea4989640 1 Processor -- start 2023-10-08T05:52:17.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.737+0000 7f3ea4989640 1 -- start start 2023-10-08T05:52:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.739+0000 7f3ea4989640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ea015cd90 0x7f3ea00702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.739+0000 7f3ea4989640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea00c48f0 con 0x7f3ea006c890 2023-10-08T05:52:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.739+0000 7f3ea4989640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea00c3ce0 con 0x7f3ea0160940 2023-10-08T05:52:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.739+0000 7f3ea4989640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea00c4d80 con 0x7f3ea015cd90 2023-10-08T05:52:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.739+0000 7f3e9e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ea015cd90 0x7f3ea00702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.739+0000 7f3e9e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ea015cd90 0x7f3ea00702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35450/0 (socket says 172.21.15.70:35450) 2023-10-08T05:52:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.739+0000 7f3e9ed9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3ea006c890 0x7f3ea006cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60466/0 (socket says 172.21.15.70:60466) 2023-10-08T05:52:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.739+0000 7f3e9e59c640 1 -- 172.21.15.70:0/882845465 learned_addr learned my addr 172.21.15.70:0/882845465 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.740+0000 7f3e9e59c640 1 -- 172.21.15.70:0/882845465 >> v1:172.21.15.181:6789/0 conn(0x7f3ea0160940 legacy=0x7f3ea0070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.740+0000 7f3e9e59c640 1 -- 172.21.15.70:0/882845465 >> v1:172.21.15.70:6789/0 conn(0x7f3ea006c890 legacy=0x7f3ea006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.740+0000 7f3e9e59c640 1 -- 172.21.15.70:0/882845465 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00c3fc0 con 0x7f3ea015cd90 2023-10-08T05:52:17.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.741+0000 7f3e9e59c640 1 --2- 172.21.15.70:0/882845465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea015cd90 0x7f3ea00702d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009870 tx=0x7f3e9002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320f995cf3fda288 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:17.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.741+0000 7f3e9dd9b640 1 -- 172.21.15.70:0/882845465 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90030860 con 0x7f3ea015cd90 2023-10-08T05:52:17.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.741+0000 7f3e9dd9b640 1 -- 172.21.15.70:0/882845465 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e900309c0 con 0x7f3ea015cd90 2023-10-08T05:52:17.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.741+0000 7f3e9dd9b640 1 -- 172.21.15.70:0/882845465 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90030d50 con 0x7f3ea015cd90 2023-10-08T05:52:17.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.742+0000 7f3ea4989640 1 -- 172.21.15.70:0/882845465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea015cd90 msgr2=0x7f3ea00702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.742+0000 7f3ea4989640 1 --2- 172.21.15.70:0/882845465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea015cd90 0x7f3ea00702d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009870 tx=0x7f3e9002b9a0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.742+0000 7f3ea4989640 1 -- 172.21.15.70:0/882845465 shutdown_connections 2023-10-08T05:52:17.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.742+0000 7f3ea4989640 1 --2- 172.21.15.70:0/882845465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea015cd90 0x7f3ea00702d0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.742+0000 7f3ea4989640 1 -- 172.21.15.70:0/882845465 >> 172.21.15.70:0/882845465 conn(0x7f3ea006a380 msgr2=0x7f3ea00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:17.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.742+0000 7f3ea4989640 1 -- 172.21.15.70:0/882845465 shutdown_connections 2023-10-08T05:52:17.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.742+0000 7f3ea4989640 1 -- 172.21.15.70:0/882845465 wait complete. 2023-10-08T05:52:17.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.744+0000 7f3ea4989640 1 Processor -- start 2023-10-08T05:52:17.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.744+0000 7f3ea4989640 1 -- start start 2023-10-08T05:52:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3ea4989640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea006c890 0x7f3ea00b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3ea4989640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea015cd90 0x7f3ea00b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3ea4989640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea0160940 0x7f3ea0148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3ea4989640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00b2f80 con 0x7f3ea0160940 2023-10-08T05:52:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3ea4989640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00b38d0 con 0x7f3ea006c890 2023-10-08T05:52:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3ea4989640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00b2250 con 0x7f3ea015cd90 2023-10-08T05:52:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3e9ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea0160940 0x7f3ea0148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3e9ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea0160940 0x7f3ea0148d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35462/0 (socket says 172.21.15.70:35462) 2023-10-08T05:52:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3e9ed9d640 1 -- 172.21.15.70:0/444270875 learned_addr learned my addr 172.21.15.70:0/444270875 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3e9e59c640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea006c890 0x7f3ea00b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.745+0000 7f3e97fff640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea015cd90 0x7f3ea00b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e9ed9d640 1 -- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea015cd90 msgr2=0x7f3ea00b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e9ed9d640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea015cd90 0x7f3ea00b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e9ed9d640 1 -- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea006c890 msgr2=0x7f3ea00b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e9ed9d640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea006c890 0x7f3ea00b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e9ed9d640 1 -- 172.21.15.70:0/444270875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900094d0 con 0x7f3ea0160940 2023-10-08T05:52:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e9e59c640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea006c890 0x7f3ea00b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e97fff640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea015cd90 0x7f3ea00b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:17.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e9ed9d640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea0160940 0x7f3ea0148d00 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800c780 tx=0x7f3e9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:17.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e96ffd640 1 -- 172.21.15.70:0/444270875 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800ce90 con 0x7f3ea0160940 2023-10-08T05:52:17.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e96ffd640 1 -- 172.21.15.70:0/444270875 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9800a640 con 0x7f3ea0160940 2023-10-08T05:52:17.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3e96ffd640 1 -- 172.21.15.70:0/444270875 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800a9b0 con 0x7f3ea0160940 2023-10-08T05:52:17.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3ea4989640 1 -- 172.21.15.70:0/444270875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea00b19b0 con 0x7f3ea0160940 2023-10-08T05:52:17.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.746+0000 7f3ea4989640 1 -- 172.21.15.70:0/444270875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea00b1d40 con 0x7f3ea0160940 2023-10-08T05:52:17.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.748+0000 7f3ea4989640 1 -- 172.21.15.70:0/444270875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea0067490 con 0x7f3ea0160940 2023-10-08T05:52:17.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.749+0000 7f3e96ffd640 1 -- 172.21.15.70:0/444270875 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3e9800ab10 con 0x7f3ea0160940 2023-10-08T05:52:17.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.750+0000 7f3e96ffd640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3e8006af90 0x7f3e8006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:17.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.750+0000 7f3e96ffd640 1 -- 172.21.15.70:0/444270875 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f3e98088fb0 con 0x7f3ea0160940 2023-10-08T05:52:17.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.751+0000 7f3e9e59c640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3e8006af90 0x7f3e8006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:17.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.752+0000 7f3e9e59c640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3e8006af90 0x7f3e8006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3e90036000 tx=0x7f3e9003b040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:17.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.755+0000 7f3e96ffd640 1 -- 172.21.15.70:0/444270875 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e98053ed0 con 0x7f3ea0160940 2023-10-08T05:52:17.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.946+0000 7f3ea4989640 1 -- 172.21.15.70:0/444270875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ea00b0ef0 con 0x7f3ea0160940 2023-10-08T05:52:17.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.947+0000 7f3e96ffd640 1 -- 172.21.15.70:0/444270875 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+14262 (secure 0 0 0) 0x7f3e98017020 con 0x7f3ea0160940 2023-10-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:17.948 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":82,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:15.476928+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6834/1337581673":"2023-10-09T05:52:12.333949+0000","172.21.15.181:0/2424526808":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/1956777616":"2023-10-09T05:52:08.555046+0000","172.21.15.70:6835/1337581673":"2023-10-09T05:52:12.333949+0000","172.21.15.181:6833/3803301866":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/602288726":"2023-10-09T05:52:08.555046+0000","172.21.15.181:6832/3803301866":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/2288760234":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/1060036340":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/464684516":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/1910914026":"2023-10-09T05:50:30.831312+0000","172.21.15.181:6833/992174303":"2023-10-09T05:50:30.831312+0000","172.21.15.181:6832/992174303":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/596247496":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/1453825375":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/423662621":"2023-10-09T05:52:08.555046+0000","172.21.15.181:0/368598410":"2023-10-09T05:50:30.831312+0000","172.21.15.181:0/3145795928":"2023-10-09T05:50:30.831312+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:17.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 -- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3e8006af90 msgr2=0x7f3e8006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3e8006af90 0x7f3e8006d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3e90036000 tx=0x7f3e9003b040 comp rx=0 tx=0).stop 2023-10-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 -- 172.21.15.70:0/444270875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea0160940 msgr2=0x7f3ea0148d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea0160940 0x7f3ea0148d00 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800c780 tx=0x7f3e9800cc40 comp rx=0 tx=0).stop 2023-10-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 -- 172.21.15.70:0/444270875 shutdown_connections 2023-10-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3e8006af90 0x7f3e8006d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ea0160940 0x7f3ea0148d00 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ea015cd90 0x7f3ea00b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 --2- 172.21.15.70:0/444270875 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ea006c890 0x7f3ea00b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 -- 172.21.15.70:0/444270875 >> 172.21.15.70:0/444270875 conn(0x7f3ea006a380 msgr2=0x7f3ea00af870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 -- 172.21.15.70:0/444270875 shutdown_connections 2023-10-08T05:52:17.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:17.950+0000 7f3ea4989640 1 -- 172.21.15.70:0/444270875 wait complete. 2023-10-08T05:52:17.970 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:52:17.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:52:18.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.188+0000 7f29eea32640 1 Processor -- start 2023-10-08T05:52:18.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.188+0000 7f29eea32640 1 -- start start 2023-10-08T05:52:18.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29eea32640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29e815cca0 0x7f29e80b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29eea32640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e80b6cd0 0x7f29e80cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29eea32640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f29e80cde70 0x7f29e80b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29eea32640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e80b7e70 con 0x7f29e815cca0 2023-10-08T05:52:18.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29eea32640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e80ce300 con 0x7f29e80cde70 2023-10-08T05:52:18.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29eea32640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e815d080 con 0x7f29e80b6cd0 2023-10-08T05:52:18.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29e7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29e815cca0 0x7f29e80b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:18.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29e7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29e815cca0 0x7f29e80b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35468/0 (socket says 172.21.15.70:35468) 2023-10-08T05:52:18.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29e7fff640 1 -- 172.21.15.70:0/3635263353 learned_addr learned my addr 172.21.15.70:0/3635263353 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:18.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29e7fff640 1 -- 172.21.15.70:0/3635263353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e80b6cd0 msgr2=0x7f29e80cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:18.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.190+0000 7f29e7fff640 1 --2- 172.21.15.70:0/3635263353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e80b6cd0 0x7f29e80cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.191+0000 7f29e7fff640 1 -- 172.21.15.70:0/3635263353 >> v2:172.21.15.181:3300/0 conn(0x7f29e80cde70 msgr2=0x7f29e80b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:18.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.191+0000 7f29e7fff640 1 --2- 172.21.15.70:0/3635263353 >> v2:172.21.15.181:3300/0 conn(0x7f29e80cde70 0x7f29e80b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.191+0000 7f29e7fff640 1 -- 172.21.15.70:0/3635263353 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80b7260 con 0x7f29e815cca0 2023-10-08T05:52:18.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.192+0000 7f29e7fff640 1 --2- 172.21.15.70:0/3635263353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e815cca0 0x7f29e80b6790 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f29e8160760 tx=0x7f29d802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8da102e05530219 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:18.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.192+0000 7f29e6ffd640 1 -- 172.21.15.70:0/3635263353 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d802cb00 con 0x7f29e815cca0 2023-10-08T05:52:18.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.192+0000 7f29e6ffd640 1 -- 172.21.15.70:0/3635263353 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d802cc60 con 0x7f29e815cca0 2023-10-08T05:52:18.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.192+0000 7f29e6ffd640 1 -- 172.21.15.70:0/3635263353 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80343f0 con 0x7f29e815cca0 2023-10-08T05:52:18.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.193+0000 7f29eea32640 1 -- 172.21.15.70:0/3635263353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e815cca0 msgr2=0x7f29e80b6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:18.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.193+0000 7f29eea32640 1 --2- 172.21.15.70:0/3635263353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e815cca0 0x7f29e80b6790 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f29e8160760 tx=0x7f29d802bf60 comp rx=0 tx=0).stop 2023-10-08T05:52:18.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.193+0000 7f29eea32640 1 -- 172.21.15.70:0/3635263353 shutdown_connections 2023-10-08T05:52:18.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.193+0000 7f29eea32640 1 --2- 172.21.15.70:0/3635263353 >> v2:172.21.15.181:3300/0 conn(0x7f29e80cde70 0x7f29e80b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.193+0000 7f29eea32640 1 --2- 172.21.15.70:0/3635263353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e80b6cd0 0x7f29e80cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.193+0000 7f29eea32640 1 --2- 172.21.15.70:0/3635263353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e815cca0 0x7f29e80b6790 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.193+0000 7f29eea32640 1 -- 172.21.15.70:0/3635263353 >> 172.21.15.70:0/3635263353 conn(0x7f29e80718a0 msgr2=0x7f29e8071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:18.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.193+0000 7f29eea32640 1 -- 172.21.15.70:0/3635263353 shutdown_connections 2023-10-08T05:52:18.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.193+0000 7f29eea32640 1 -- 172.21.15.70:0/3635263353 wait complete. 2023-10-08T05:52:18.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.195+0000 7f29eea32640 1 Processor -- start 2023-10-08T05:52:18.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.195+0000 7f29eea32640 1 -- start start 2023-10-08T05:52:18.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29eea32640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29e80b6cd0 0x7f29e814f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29eea32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e80cde70 0x7f29e81f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29eea32640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e815cca0 0x7f29e80cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29eea32640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80c77e0 con 0x7f29e80cde70 2023-10-08T05:52:18.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29eea32640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80cada0 con 0x7f29e80b6cd0 2023-10-08T05:52:18.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29eea32640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80c7d30 con 0x7f29e815cca0 2023-10-08T05:52:18.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e80cde70 0x7f29e81f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:18.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e80cde70 0x7f29e81f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35472/0 (socket says 172.21.15.70:35472) 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29e77fe640 1 -- 172.21.15.70:0/3952542110 learned_addr learned my addr 172.21.15.70:0/3952542110 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29e7fff640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29e80b6cd0 0x7f29e814f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29ecfcf640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e815cca0 0x7f29e80cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29e77fe640 1 -- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e815cca0 msgr2=0x7f29e80cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29e77fe640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e815cca0 0x7f29e80cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29e77fe640 1 -- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29e80b6cd0 msgr2=0x7f29e814f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29e77fe640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29e80b6cd0 0x7f29e814f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.196+0000 7f29e77fe640 1 -- 172.21.15.70:0/3952542110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d8009b30 con 0x7f29e80cde70 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.197+0000 7f29e77fe640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e80cde70 0x7f29e81f2af0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00b4d0 tx=0x7f29dc00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.197+0000 7f29e4ff9640 1 -- 172.21.15.70:0/3952542110 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc010610 con 0x7f29e80cde70 2023-10-08T05:52:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.197+0000 7f29e4ff9640 1 -- 172.21.15.70:0/3952542110 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc010770 con 0x7f29e80cde70 2023-10-08T05:52:18.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.197+0000 7f29e4ff9640 1 -- 172.21.15.70:0/3952542110 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc010b20 con 0x7f29e80cde70 2023-10-08T05:52:18.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.197+0000 7f29eea32640 1 -- 172.21.15.70:0/3952542110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e81f3030 con 0x7f29e80cde70 2023-10-08T05:52:18.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.197+0000 7f29eea32640 1 -- 172.21.15.70:0/3952542110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e80caac0 con 0x7f29e80cde70 2023-10-08T05:52:18.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.198+0000 7f29eea32640 1 -- 172.21.15.70:0/3952542110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e804e160 con 0x7f29e80cde70 2023-10-08T05:52:18.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.199+0000 7f29e4ff9640 1 -- 172.21.15.70:0/3952542110 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f29dc010c80 con 0x7f29e80cde70 2023-10-08T05:52:18.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.200+0000 7f29e4ff9640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f29d406af90 0x7f29d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.200+0000 7f29e4ff9640 1 -- 172.21.15.70:0/3952542110 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f29dc089590 con 0x7f29e80cde70 2023-10-08T05:52:18.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.201+0000 7f29e7fff640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f29d406af90 0x7f29d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:18.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.202+0000 7f29e7fff640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f29d406af90 0x7f29d406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f29d8035000 tx=0x7f29d8030000 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:18.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.204+0000 7f29e4ff9640 1 -- 172.21.15.70:0/3952542110 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29dc054370 con 0x7f29e80cde70 2023-10-08T05:52:18.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.394+0000 7f29eea32640 1 -- 172.21.15.70:0/3952542110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29e80c7e70 con 0x7f29e80cde70 2023-10-08T05:52:18.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.395+0000 7f29e4ff9640 1 -- 172.21.15.70:0/3952542110 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29dc057990 con 0x7f29e80cde70 2023-10-08T05:52:18.396 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:52:18.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 -- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f29d406af90 msgr2=0x7f29d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:18.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f29d406af90 0x7f29d406d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f29d8035000 tx=0x7f29d8030000 comp rx=0 tx=0).stop 2023-10-08T05:52:18.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 -- 172.21.15.70:0/3952542110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e80cde70 msgr2=0x7f29e81f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:18.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e80cde70 0x7f29e81f2af0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f29dc00b4d0 tx=0x7f29dc00b990 comp rx=0 tx=0).stop 2023-10-08T05:52:18.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 -- 172.21.15.70:0/3952542110 shutdown_connections 2023-10-08T05:52:18.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f29d406af90 0x7f29d406d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29e815cca0 0x7f29e80cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29e80cde70 0x7f29e81f2af0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 --2- 172.21.15.70:0/3952542110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29e80b6cd0 0x7f29e814f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 -- 172.21.15.70:0/3952542110 >> 172.21.15.70:0/3952542110 conn(0x7f29e80718a0 msgr2=0x7f29e8158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:18.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 -- 172.21.15.70:0/3952542110 shutdown_connections 2023-10-08T05:52:18.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.398+0000 7f29eea32640 1 -- 172.21.15.70:0/3952542110 wait complete. 2023-10-08T05:52:18.416 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-08T05:52:18.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.627+0000 7fc7cf7a5640 1 Processor -- start 2023-10-08T05:52:18.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.628+0000 7fc7cf7a5640 1 -- start start 2023-10-08T05:52:18.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.629+0000 7fc7cf7a5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc7c80af8b0 0x7fc7c806ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.629+0000 7fc7cf7a5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc7c80b0ab0 0x7fc7c80b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.629+0000 7fc7cf7a5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80b0340 con 0x7fc7c806b520 2023-10-08T05:52:18.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.629+0000 7fc7cf7a5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c806b3a0 con 0x7fc7c80af8b0 2023-10-08T05:52:18.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.629+0000 7fc7cf7a5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c80b0150 con 0x7fc7c80b0ab0 2023-10-08T05:52:18.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.629+0000 7fc7ccd40640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc7c80af8b0 0x7fc7c806ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:18.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.629+0000 7fc7ccd40640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc7c80af8b0 0x7fc7c806ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35478/0 (socket says 172.21.15.70:35478) 2023-10-08T05:52:18.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.629+0000 7fc7ccd40640 1 -- 172.21.15.70:0/144429411 learned_addr learned my addr 172.21.15.70:0/144429411 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:18.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.629+0000 7fc7cdd42640 1 --2- 172.21.15.70:0/144429411 >> v2:172.21.15.181:3300/0 conn(0x7fc7c80b0ab0 0x7fc7c80b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.630+0000 7fc7ccd40640 1 -- 172.21.15.70:0/144429411 >> v2:172.21.15.181:3300/0 conn(0x7fc7c80b0ab0 msgr2=0x7fc7c80b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.630+0000 7fc7ccd40640 1 --2- 172.21.15.70:0/144429411 >> v2:172.21.15.181:3300/0 conn(0x7fc7c80b0ab0 0x7fc7c80b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.630+0000 7fc7ccd40640 1 -- 172.21.15.70:0/144429411 >> v1:172.21.15.181:6789/0 conn(0x7fc7c806b520 legacy=0x7fc7c806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:18.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.630+0000 7fc7ccd40640 1 -- 172.21.15.70:0/144429411 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80cf8f0 con 0x7fc7c80af8b0 2023-10-08T05:52:18.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.631+0000 7fc7ccd40640 1 --2- 172.21.15.70:0/144429411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 0x7fc7c806ae60 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80c43d0 tx=0x7fc7b802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a298ed9341eaefd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.631+0000 7fc7bffff640 1 -- 172.21.15.70:0/144429411 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80317e0 con 0x7fc7c80af8b0 2023-10-08T05:52:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.631+0000 7fc7bffff640 1 -- 172.21.15.70:0/144429411 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b8031940 con 0x7fc7c80af8b0 2023-10-08T05:52:18.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.631+0000 7fc7bffff640 1 -- 172.21.15.70:0/144429411 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8031cd0 con 0x7fc7c80af8b0 2023-10-08T05:52:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.631+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/144429411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 msgr2=0x7fc7c806ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.631+0000 7fc7cf7a5640 1 --2- 172.21.15.70:0/144429411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 0x7fc7c806ae60 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80c43d0 tx=0x7fc7b802b880 comp rx=0 tx=0).stop 2023-10-08T05:52:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.632+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/144429411 shutdown_connections 2023-10-08T05:52:18.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.632+0000 7fc7cf7a5640 1 --2- 172.21.15.70:0/144429411 >> v2:172.21.15.181:3300/0 conn(0x7fc7c80b0ab0 0x7fc7c80b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.632+0000 7fc7cf7a5640 1 --2- 172.21.15.70:0/144429411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 0x7fc7c806ae60 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.632+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/144429411 >> 172.21.15.70:0/144429411 conn(0x7fc7c80d09e0 msgr2=0x7fc7c80d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:18.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.632+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/144429411 shutdown_connections 2023-10-08T05:52:18.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.632+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/144429411 wait complete. 2023-10-08T05:52:18.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.633+0000 7fc7cf7a5640 1 Processor -- start 2023-10-08T05:52:18.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.634+0000 7fc7cf7a5640 1 -- start start 2023-10-08T05:52:18.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.634+0000 7fc7cf7a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7c806b520 0x7fc7c8155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7cf7a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 0x7fc7c80c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7cf7a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7c80b0ab0 0x7fc7c80b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7cf7a5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80b4b90 con 0x7fc7c80af8b0 2023-10-08T05:52:18.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7cf7a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80b4d70 con 0x7fc7c80b0ab0 2023-10-08T05:52:18.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7cf7a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80b5050 con 0x7fc7c806b520 2023-10-08T05:52:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7ccd40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 0x7fc7c80c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7ccd40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 0x7fc7c80c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35488/0 (socket says 172.21.15.70:35488) 2023-10-08T05:52:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7ccd40640 1 -- 172.21.15.70:0/3983726620 learned_addr learned my addr 172.21.15.70:0/3983726620 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7cd541640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7c806b520 0x7fc7c8155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7ccd40640 1 -- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7c806b520 msgr2=0x7fc7c8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7ccd40640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7c806b520 0x7fc7c8155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7ccd40640 1 -- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7c80b0ab0 msgr2=0x7fc7c80b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7ccd40640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7c80b0ab0 0x7fc7c80b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.635+0000 7fc7ccd40640 1 -- 172.21.15.70:0/3983726620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b80094d0 con 0x7fc7c80af8b0 2023-10-08T05:52:18.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.636+0000 7fc7ccd40640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 0x7fc7c80c55f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc7b802b8c0 tx=0x7fc7b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.636+0000 7fc7bdffb640 1 -- 172.21.15.70:0/3983726620 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80329c0 con 0x7fc7c80af8b0 2023-10-08T05:52:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.636+0000 7fc7bdffb640 1 -- 172.21.15.70:0/3983726620 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b8032b40 con 0x7fc7c80af8b0 2023-10-08T05:52:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.636+0000 7fc7bdffb640 1 -- 172.21.15.70:0/3983726620 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80453f0 con 0x7fc7c80af8b0 2023-10-08T05:52:18.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.636+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/3983726620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c80b5330 con 0x7fc7c80af8b0 2023-10-08T05:52:18.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.636+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/3983726620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c80b58f0 con 0x7fc7c80af8b0 2023-10-08T05:52:18.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.637+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/3983726620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c80b5ae0 con 0x7fc7c80af8b0 2023-10-08T05:52:18.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.638+0000 7fc7bdffb640 1 -- 172.21.15.70:0/3983726620 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc7b804d3f0 con 0x7fc7c80af8b0 2023-10-08T05:52:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.639+0000 7fc7bdffb640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc7ac06afc0 0x7fc7ac06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.639+0000 7fc7bdffb640 1 -- 172.21.15.70:0/3983726620 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fc7b80ab980 con 0x7fc7c80af8b0 2023-10-08T05:52:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.640+0000 7fc7cd541640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc7ac06afc0 0x7fc7ac06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.641+0000 7fc7cd541640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc7ac06afc0 0x7fc7ac06d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4007dc0 tx=0x7fc7c40072f0 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:18.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.643+0000 7fc7bdffb640 1 -- 172.21.15.70:0/3983726620 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7b8076700 con 0x7fc7c80af8b0 2023-10-08T05:52:18.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:18.884+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/3983726620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fc7c8067490 con 0x7fc7c80af8b0 2023-10-08T05:52:19.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.512+0000 7fc7bdffb640 1 -- 172.21.15.70:0/3983726620 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v593) v1 ==== 159+0+0 (secure 0 0 0) 0x7fc7b8079d20 con 0x7fc7c80af8b0 2023-10-08T05:52:19.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.515+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc7ac06afc0 msgr2=0x7fc7ac06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:19.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.515+0000 7fc7cf7a5640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc7ac06afc0 0x7fc7ac06d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4007dc0 tx=0x7fc7c40072f0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.515+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/3983726620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 msgr2=0x7fc7c80c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:19.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.515+0000 7fc7cf7a5640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 0x7fc7c80c55f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fc7b802b8c0 tx=0x7fc7b80046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.515+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/3983726620 shutdown_connections 2023-10-08T05:52:19.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.515+0000 7fc7cf7a5640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc7ac06afc0 0x7fc7ac06d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.515+0000 7fc7cf7a5640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7c80b0ab0 0x7fc7c80b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.515+0000 7fc7cf7a5640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7c80af8b0 0x7fc7c80c55f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.515+0000 7fc7cf7a5640 1 --2- 172.21.15.70:0/3983726620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7c806b520 0x7fc7c8155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.515+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/3983726620 >> 172.21.15.70:0/3983726620 conn(0x7fc7c80d09e0 msgr2=0x7fc7c815aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.516+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/3983726620 shutdown_connections 2023-10-08T05:52:19.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.516+0000 7fc7cf7a5640 1 -- 172.21.15.70:0/3983726620 wait complete. 2023-10-08T05:52:19.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T05:52:19.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.752+0000 7f75debee640 1 Processor -- start 2023-10-08T05:52:19.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.753+0000 7f75debee640 1 -- start start 2023-10-08T05:52:19.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.754+0000 7f75debee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f75d806c3a0 0x7f75d80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:19.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.754+0000 7f75debee640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d815aa00 0x7f75d80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:19.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.754+0000 7f75debee640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d815b310 con 0x7f75d806db60 2023-10-08T05:52:19.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.754+0000 7f75debee640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d80ca230 con 0x7f75d806c3a0 2023-10-08T05:52:19.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.754+0000 7f75debee640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d806d290 con 0x7f75d815aa00 2023-10-08T05:52:19.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.755+0000 7f75dc98a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f75d806c3a0 0x7f75d80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:19.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.755+0000 7f75dc98a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f75d806c3a0 0x7f75d80c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35496/0 (socket says 172.21.15.70:35496) 2023-10-08T05:52:19.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.755+0000 7f75dc98a640 1 -- 172.21.15.70:0/2399356915 learned_addr learned my addr 172.21.15.70:0/2399356915 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:19.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.755+0000 7f75dd18b640 1 --2- 172.21.15.70:0/2399356915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d815aa00 0x7f75d80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:19.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.755+0000 7f75dc98a640 1 -- 172.21.15.70:0/2399356915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d815aa00 msgr2=0x7f75d80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:19.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.755+0000 7f75dc98a640 1 --2- 172.21.15.70:0/2399356915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d815aa00 0x7f75d80b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.755+0000 7f75dc98a640 1 -- 172.21.15.70:0/2399356915 >> v1:172.21.15.181:6789/0 conn(0x7f75d806db60 legacy=0x7f75d806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:19.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.755+0000 7f75dc98a640 1 -- 172.21.15.70:0/2399356915 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80c9860 con 0x7f75d806c3a0 2023-10-08T05:52:19.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.756+0000 7f75dc98a640 1 --2- 172.21.15.70:0/2399356915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806c3a0 0x7f75d80c9cf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009e50 tx=0x7f75c802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffbb81aadf97bad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:19.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.757+0000 7f75d77fe640 1 -- 172.21.15.70:0/2399356915 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c802cb00 con 0x7f75d806c3a0 2023-10-08T05:52:19.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.757+0000 7f75d77fe640 1 -- 172.21.15.70:0/2399356915 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c802cc60 con 0x7f75d806c3a0 2023-10-08T05:52:19.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.757+0000 7f75d77fe640 1 -- 172.21.15.70:0/2399356915 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c80343f0 con 0x7f75d806c3a0 2023-10-08T05:52:19.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.757+0000 7f75debee640 1 -- 172.21.15.70:0/2399356915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806c3a0 msgr2=0x7f75d80c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:19.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.757+0000 7f75debee640 1 --2- 172.21.15.70:0/2399356915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806c3a0 0x7f75d80c9cf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009e50 tx=0x7f75c802bf60 comp rx=0 tx=0).stop 2023-10-08T05:52:19.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.757+0000 7f75debee640 1 -- 172.21.15.70:0/2399356915 shutdown_connections 2023-10-08T05:52:19.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.757+0000 7f75debee640 1 --2- 172.21.15.70:0/2399356915 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d815aa00 0x7f75d80b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.757+0000 7f75debee640 1 --2- 172.21.15.70:0/2399356915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806c3a0 0x7f75d80c9cf0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.757+0000 7f75debee640 1 -- 172.21.15.70:0/2399356915 >> 172.21.15.70:0/2399356915 conn(0x7f75d806a5d0 msgr2=0x7f75d80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:19.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.757+0000 7f75debee640 1 -- 172.21.15.70:0/2399356915 shutdown_connections 2023-10-08T05:52:19.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.758+0000 7f75debee640 1 -- 172.21.15.70:0/2399356915 wait complete. 2023-10-08T05:52:19.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.759+0000 7f75debee640 1 Processor -- start 2023-10-08T05:52:19.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.759+0000 7f75debee640 1 -- start start 2023-10-08T05:52:19.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.760+0000 7f75debee640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d806c3a0 0x7f75d8153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:19.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.760+0000 7f75debee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806db60 0x7f75d81504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:19.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.760+0000 7f75debee640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d815aa00 0x7f75d81f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:19.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.760+0000 7f75debee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d814d8a0 con 0x7f75d806db60 2023-10-08T05:52:19.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.760+0000 7f75debee640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d80c4ac0 con 0x7f75d806c3a0 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.760+0000 7f75debee640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d80c4da0 con 0x7f75d815aa00 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.760+0000 7f75d7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806db60 0x7f75d81504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806db60 0x7f75d81504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35510/0 (socket says 172.21.15.70:35510) 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d7fff640 1 -- 172.21.15.70:0/3334268982 learned_addr learned my addr 172.21.15.70:0/3334268982 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75dc98a640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d806c3a0 0x7f75d8153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75dd18b640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d815aa00 0x7f75d81f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d7fff640 1 -- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d815aa00 msgr2=0x7f75d81f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d7fff640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d815aa00 0x7f75d81f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d7fff640 1 -- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d806c3a0 msgr2=0x7f75d8153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d7fff640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d806c3a0 0x7f75d8153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d7fff640 1 -- 172.21.15.70:0/3334268982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8009b30 con 0x7f75d806db60 2023-10-08T05:52:19.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75dc98a640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d806c3a0 0x7f75d8153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:19.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d7fff640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806db60 0x7f75d81504c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c450 tx=0x7f75cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:19.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d57fa640 1 -- 172.21.15.70:0/3334268982 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0052d0 con 0x7f75d806db60 2023-10-08T05:52:19.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d57fa640 1 -- 172.21.15.70:0/3334268982 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc005430 con 0x7f75d806db60 2023-10-08T05:52:19.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75d57fa640 1 -- 172.21.15.70:0/3334268982 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc012870 con 0x7f75d806db60 2023-10-08T05:52:19.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.761+0000 7f75debee640 1 -- 172.21.15.70:0/3334268982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d8152c70 con 0x7f75d806db60 2023-10-08T05:52:19.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.762+0000 7f75debee640 1 -- 172.21.15.70:0/3334268982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d80c5080 con 0x7f75d806db60 2023-10-08T05:52:19.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.763+0000 7f75debee640 1 -- 172.21.15.70:0/3334268982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d8153190 con 0x7f75d806db60 2023-10-08T05:52:19.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.764+0000 7f75d57fa640 1 -- 172.21.15.70:0/3334268982 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f75cc0129d0 con 0x7f75d806db60 2023-10-08T05:52:19.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.765+0000 7f75d57fa640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f75c406af90 0x7f75c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:19.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.765+0000 7f75d57fa640 1 -- 172.21.15.70:0/3334268982 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f75cc0891e0 con 0x7f75d806db60 2023-10-08T05:52:19.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.765+0000 7f75dc98a640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f75c406af90 0x7f75c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:19.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.766+0000 7f75dc98a640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f75c406af90 0x7f75c406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f75c8035000 tx=0x7f75c8030000 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:19.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.769+0000 7f75d57fa640 1 -- 172.21.15.70:0/3334268982 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75d8153190 con 0x7f75d806db60 2023-10-08T05:52:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.942+0000 7f75debee640 1 -- 172.21.15.70:0/3334268982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f75d815b310 con 0x7f75d806db60 2023-10-08T05:52:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.947+0000 7f75d57fa640 1 -- 172.21.15.70:0/3334268982 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f75d815b310 con 0x7f75d806db60 2023-10-08T05:52:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 -- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f75c406af90 msgr2=0x7f75c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f75c406af90 0x7f75c406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f75c8035000 tx=0x7f75c8030000 comp rx=0 tx=0).stop 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 -- 172.21.15.70:0/3334268982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806db60 msgr2=0x7f75d81504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806db60 0x7f75d81504c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c450 tx=0x7f75cc00c910 comp rx=0 tx=0).stop 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 -- 172.21.15.70:0/3334268982 shutdown_connections 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f75c406af90 0x7f75c406d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75d815aa00 0x7f75d81f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75d806db60 0x7f75d81504c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 --2- 172.21.15.70:0/3334268982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f75d806c3a0 0x7f75d8153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 -- 172.21.15.70:0/3334268982 >> 172.21.15.70:0/3334268982 conn(0x7f75d806a5d0 msgr2=0x7f75d80ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 -- 172.21.15.70:0/3334268982 shutdown_connections 2023-10-08T05:52:19.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:19.950+0000 7f75debee640 1 -- 172.21.15.70:0/3334268982 wait complete. 2023-10-08T05:52:19.968 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:52:19.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:52:20.075 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:52:20.201 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:52:20.201 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:20.201 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:20.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:52:20.260 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:52:20.260 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:20.320 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:52:20.320 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:20.320 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:20.380 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:52:20.381 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:52:20.381 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:20.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:52:20.442 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T05:52:20.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.662+0000 7f0d8d8da640 1 Processor -- start 2023-10-08T05:52:20.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.662+0000 7f0d8d8da640 1 -- start start 2023-10-08T05:52:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.664+0000 7f0d8d8da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88159ac0 0x7f0d8806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.664+0000 7f0d8d8da640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8806dee0 con 0x7f0d880ad790 2023-10-08T05:52:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.664+0000 7f0d8d8da640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8815d850 con 0x7f0d880709e0 2023-10-08T05:52:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.664+0000 7f0d8d8da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8815db30 con 0x7f0d88159ac0 2023-10-08T05:52:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.664+0000 7f0d7ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0d880ad790 0x7f0d880702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60478/0 (socket says 172.21.15.70:60478) 2023-10-08T05:52:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.664+0000 7f0d7ffff640 1 -- 172.21.15.70:0/3095182602 learned_addr learned my addr 172.21.15.70:0/3095182602 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:20.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d86ffd640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88159ac0 0x7f0d8806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:20.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224756970 0 0) 0x7f0d8806dee0 con 0x7f0d880ad790 2023-10-08T05:52:20.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d74003520 con 0x7f0d880ad790 2023-10-08T05:52:20.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1847825407 0 0) 0x7f0d74003520 con 0x7f0d880ad790 2023-10-08T05:52:20.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88159ac0 msgr2=0x7f0d8806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:20.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88159ac0 0x7f0d8806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:20.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 >> v1:172.21.15.181:6789/0 conn(0x7f0d880709e0 legacy=0x7f0d880c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:20.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8815de10 con 0x7f0d880ad790 2023-10-08T05:52:20.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0d70001670 con 0x7f0d880ad790 2023-10-08T05:52:20.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 >> v1:172.21.15.70:6789/0 conn(0x7f0d880ad790 legacy=0x7f0d880702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:20.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88159ac0 0x7f0d740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d74003f20 0x7f0d740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74004d00 0x7f0d74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8815db30 con 0x7f0d74003f20 2023-10-08T05:52:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7400ba60 con 0x7f0d74004d00 2023-10-08T05:52:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7400bc20 con 0x7f0d88159ac0 2023-10-08T05:52:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.665+0000 7f0d7ffff640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d74003f20 0x7f0d740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.666+0000 7f0d877fe640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74004d00 0x7f0d74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.666+0000 7f0d7ffff640 1 -- 172.21.15.70:0/3095182602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88159ac0 msgr2=0x7f0d740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.666+0000 7f0d7ffff640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88159ac0 0x7f0d740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:20.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.666+0000 7f0d7ffff640 1 -- 172.21.15.70:0/3095182602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74004d00 msgr2=0x7f0d74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.666+0000 7f0d7ffff640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74004d00 0x7f0d74005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:20.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.666+0000 7f0d7ffff640 1 -- 172.21.15.70:0/3095182602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74003520 con 0x7f0d74003f20 2023-10-08T05:52:20.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.667+0000 7f0d7ffff640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d74003f20 0x7f0d740047c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c210 tx=0x7f0d740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412b19cbe4a9e0c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:20.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.667+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70005ad0 con 0x7f0d74003f20 2023-10-08T05:52:20.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.667+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d70005c30 con 0x7f0d74003f20 2023-10-08T05:52:20.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.667+0000 7f0d867fc640 1 -- 172.21.15.70:0/3095182602 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7002cad0 con 0x7f0d74003f20 2023-10-08T05:52:20.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.668+0000 7f0d8d8da640 1 -- 172.21.15.70:0/3095182602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d74003f20 msgr2=0x7f0d740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:20.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.668+0000 7f0d8d8da640 1 --2- 172.21.15.70:0/3095182602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d74003f20 0x7f0d740047c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c210 tx=0x7f0d740057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:20.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.668+0000 7f0d877fe640 1 -- 172.21.15.70:0/3095182602 reap_dead start 2023-10-08T05:52:20.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.668+0000 7f0d8d8da640 1 -- 172.21.15.70:0/3095182602 shutdown_connections 2023-10-08T05:52:20.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.668+0000 7f0d8d8da640 1 -- 172.21.15.70:0/3095182602 >> 172.21.15.70:0/3095182602 conn(0x7f0d8806a380 msgr2=0x7f0d880b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:20.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.668+0000 7f0d8d8da640 1 -- 172.21.15.70:0/3095182602 shutdown_connections 2023-10-08T05:52:20.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.668+0000 7f0d8d8da640 1 -- 172.21.15.70:0/3095182602 wait complete. 2023-10-08T05:52:20.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.669+0000 7f0d8d8da640 1 Processor -- start 2023-10-08T05:52:20.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.670+0000 7f0d8d8da640 1 -- start start 2023-10-08T05:52:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d8d8da640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74003f20 0x7f0d880b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d8d8da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88145e10 0x7f0d88142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d8d8da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d88158430 0x7f0d881588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d8d8da640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8815bc80 con 0x7f0d88158430 2023-10-08T05:52:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d8d8da640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88158e20 con 0x7f0d74003f20 2023-10-08T05:52:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d8d8da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8815b650 con 0x7f0d88145e10 2023-10-08T05:52:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d88158430 0x7f0d881588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:20.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d88158430 0x7f0d881588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35532/0 (socket says 172.21.15.70:35532) 2023-10-08T05:52:20.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d877fe640 1 -- 172.21.15.70:0/2034599111 learned_addr learned my addr 172.21.15.70:0/2034599111 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:20.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d877fe640 1 -- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88145e10 msgr2=0x7f0d88142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:20.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d877fe640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88145e10 0x7f0d88142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:20.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d877fe640 1 -- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74003f20 msgr2=0x7f0d880b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:20.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d877fe640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74003f20 0x7f0d880b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:20.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.671+0000 7f0d877fe640 1 -- 172.21.15.70:0/2034599111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8815cc20 con 0x7f0d88158430 2023-10-08T05:52:20.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.672+0000 7f0d877fe640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d88158430 0x7f0d881588e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0d880ae2f0 tx=0x7f0d8000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:20.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.672+0000 7f0d84ff9640 1 -- 172.21.15.70:0/2034599111 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80003970 con 0x7f0d88158430 2023-10-08T05:52:20.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.672+0000 7f0d84ff9640 1 -- 172.21.15.70:0/2034599111 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d80003ad0 con 0x7f0d88158430 2023-10-08T05:52:20.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.672+0000 7f0d84ff9640 1 -- 172.21.15.70:0/2034599111 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80003e80 con 0x7f0d88158430 2023-10-08T05:52:20.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.672+0000 7f0d8d8da640 1 -- 172.21.15.70:0/2034599111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8815b910 con 0x7f0d88158430 2023-10-08T05:52:20.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.672+0000 7f0d8d8da640 1 -- 172.21.15.70:0/2034599111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8815c7f0 con 0x7f0d88158430 2023-10-08T05:52:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.673+0000 7f0d8d8da640 1 -- 172.21.15.70:0/2034599111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8815c980 con 0x7f0d88158430 2023-10-08T05:52:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.674+0000 7f0d84ff9640 1 -- 172.21.15.70:0/2034599111 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0d800050d0 con 0x7f0d88158430 2023-10-08T05:52:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.675+0000 7f0d84ff9640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0d6806afc0 0x7f0d6806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.675+0000 7f0d84ff9640 1 -- 172.21.15.70:0/2034599111 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f0d800896d0 con 0x7f0d88158430 2023-10-08T05:52:20.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.676+0000 7f0d86ffd640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0d6806afc0 0x7f0d6806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:20.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.677+0000 7f0d86ffd640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0d6806afc0 0x7f0d6806d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009a40 tx=0x7f0d78009500 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:20.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.681+0000 7f0d84ff9640 1 -- 172.21.15.70:0/2034599111 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d800544b0 con 0x7f0d88158430 2023-10-08T05:52:20.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.912+0000 7f0d8d8da640 1 -- 172.21.15.70:0/2034599111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d88067490 con 0x7f0d88158430 2023-10-08T05:52:20.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.912+0000 7f0d84ff9640 1 -- 172.21.15.70:0/2034599111 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f0d80087e60 con 0x7f0d88158430 2023-10-08T05:52:20.913 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:20.913 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/1126961048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1126961048},{"type":"v1","addr":"172.21.15.181:6836","nonce":1126961048}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3993068621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3993068621},{"type":"v1","addr":"172.21.15.181:6837","nonce":3993068621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/28904251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":28904251},{"type":"v1","addr":"172.21.15.70:6833","nonce":28904251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6573,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1171241441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1171241441},{"type":"v1","addr":"172.21.15.70:6835","nonce":1171241441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2023-10-08T05:52:20.913 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 27 2023-10-08T05:52:20.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.915+0000 7f0d8d8da640 1 -- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0d6806afc0 msgr2=0x7f0d6806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:20.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.915+0000 7f0d8d8da640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0d6806afc0 0x7f0d6806d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009a40 tx=0x7f0d78009500 comp rx=0 tx=0).stop 2023-10-08T05:52:20.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.915+0000 7f0d8d8da640 1 -- 172.21.15.70:0/2034599111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d88158430 msgr2=0x7f0d881588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:20.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.915+0000 7f0d8d8da640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d88158430 0x7f0d881588e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0d880ae2f0 tx=0x7f0d8000ec10 comp rx=0 tx=0).stop 2023-10-08T05:52:20.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.915+0000 7f0d8d8da640 1 -- 172.21.15.70:0/2034599111 shutdown_connections 2023-10-08T05:52:20.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.915+0000 7f0d8d8da640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f0d6806afc0 0x7f0d6806d480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:20.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.916+0000 7f0d8d8da640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d88158430 0x7f0d881588e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:20.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.916+0000 7f0d8d8da640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d88145e10 0x7f0d88142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:20.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.916+0000 7f0d8d8da640 1 --2- 172.21.15.70:0/2034599111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d74003f20 0x7f0d880b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:20.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.916+0000 7f0d8d8da640 1 -- 172.21.15.70:0/2034599111 >> 172.21.15.70:0/2034599111 conn(0x7f0d8806a380 msgr2=0x7f0d880c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:20.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.916+0000 7f0d8d8da640 1 -- 172.21.15.70:0/2034599111 shutdown_connections 2023-10-08T05:52:20.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:20.916+0000 7f0d8d8da640 1 -- 172.21.15.70:0/2034599111 wait complete. 2023-10-08T05:52:20.936 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T05:52:20.936 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T05:52:20.936 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T05:52:20.936 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:52:20.936 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T05:52:20.937 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T05:52:20.937 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T05:52:20.937 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:52:20.937 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T05:52:20.937 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T05:52:20.937 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T05:52:20.937 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:52:20.938 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T05:52:20.938 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T05:52:20.938 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T05:52:20.938 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:52:20.952 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T05:52:20.952 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T05:52:20.956 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T05:52:20.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T05:52:20.959 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T05:52:20.959 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T05:52:20.960 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T05:52:20.960 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T05:52:21.004 INFO:tasks.ceph.mds.c:Started 2023-10-08T05:52:21.004 DEBUG:teuthology.parallel:result is None 2023-10-08T05:52:21.009 INFO:tasks.ceph.mds.d:Started 2023-10-08T05:52:21.009 DEBUG:teuthology.parallel:result is None 2023-10-08T05:52:21.011 INFO:tasks.ceph.mds.a:Started 2023-10-08T05:52:21.012 DEBUG:teuthology.parallel:result is None 2023-10-08T05:52:21.016 INFO:tasks.ceph.mds.b:Started 2023-10-08T05:52:21.017 DEBUG:teuthology.parallel:result is None 2023-10-08T05:52:21.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T05:52:21.164 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:52:21.164 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T05:52:21.177 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T05:52:21.179 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:52:21.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.206+0000 7fd3b3be7640 1 Processor -- start 2023-10-08T05:52:21.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.206+0000 7fd3b3be7640 1 -- start start 2023-10-08T05:52:21.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.207+0000 7fd3b3be7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 0x7fd3ac0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.207+0000 7fd3b3be7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3ac06dfc0 0x7fd3ac070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.207+0000 7fd3b3be7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd3ac071070 0x7fd3ac0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.207+0000 7fd3b3be7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ac0b2620 con 0x7fd3ac06dfc0 2023-10-08T05:52:21.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.207+0000 7fd3b3be7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ac0b2060 con 0x7fd3ac071070 2023-10-08T05:52:21.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.207+0000 7fd3b3be7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0b2340 con 0x7fd3ac0afbe0 2023-10-08T05:52:21.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b1182640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3ac06dfc0 0x7fd3ac070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b1182640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3ac06dfc0 0x7fd3ac070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35560/0 (socket says 172.21.15.70:35560) 2023-10-08T05:52:21.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b1182640 1 -- 172.21.15.70:0/3630808618 learned_addr learned my addr 172.21.15.70:0/3630808618 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b2184640 1 --2- 172.21.15.70:0/3630808618 >> v2:172.21.15.181:3300/0 conn(0x7fd3ac071070 0x7fd3ac0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b1983640 1 --2- 172.21.15.70:0/3630808618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 0x7fd3ac0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b1182640 1 -- 172.21.15.70:0/3630808618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 msgr2=0x7fd3ac0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b1182640 1 --2- 172.21.15.70:0/3630808618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 0x7fd3ac0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b1182640 1 -- 172.21.15.70:0/3630808618 >> v2:172.21.15.181:3300/0 conn(0x7fd3ac071070 msgr2=0x7fd3ac0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b1182640 1 --2- 172.21.15.70:0/3630808618 >> v2:172.21.15.181:3300/0 conn(0x7fd3ac071070 0x7fd3ac0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b1182640 1 -- 172.21.15.70:0/3630808618 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac070350 con 0x7fd3ac06dfc0 2023-10-08T05:52:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b2184640 1 --2- 172.21.15.70:0/3630808618 >> v2:172.21.15.181:3300/0 conn(0x7fd3ac071070 0x7fd3ac0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.208+0000 7fd3b1983640 1 --2- 172.21.15.70:0/3630808618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 0x7fd3ac0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.209+0000 7fd3b1182640 1 --2- 172.21.15.70:0/3630808618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac06dfc0 0x7fd3ac070b30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0b3980 tx=0x7fd39c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc393252ed4c7760 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b0981640 1 -- 172.21.15.70:0/3630808618 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c031860 con 0x7fd3ac06dfc0 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b0981640 1 -- 172.21.15.70:0/3630808618 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c0319c0 con 0x7fd3ac06dfc0 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b0981640 1 -- 172.21.15.70:0/3630808618 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c031d50 con 0x7fd3ac06dfc0 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b3be7640 1 -- 172.21.15.70:0/3630808618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac06dfc0 msgr2=0x7fd3ac070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b3be7640 1 --2- 172.21.15.70:0/3630808618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac06dfc0 0x7fd3ac070b30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0b3980 tx=0x7fd39c02b900 comp rx=0 tx=0).stop 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b3be7640 1 -- 172.21.15.70:0/3630808618 shutdown_connections 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b3be7640 1 --2- 172.21.15.70:0/3630808618 >> v2:172.21.15.181:3300/0 conn(0x7fd3ac071070 0x7fd3ac0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b3be7640 1 --2- 172.21.15.70:0/3630808618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac06dfc0 0x7fd3ac070b30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b3be7640 1 --2- 172.21.15.70:0/3630808618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 0x7fd3ac0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b3be7640 1 -- 172.21.15.70:0/3630808618 >> 172.21.15.70:0/3630808618 conn(0x7fd3ac06a5d0 msgr2=0x7fd3ac0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:21.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b3be7640 1 -- 172.21.15.70:0/3630808618 shutdown_connections 2023-10-08T05:52:21.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.210+0000 7fd3b3be7640 1 -- 172.21.15.70:0/3630808618 wait complete. 2023-10-08T05:52:21.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.212+0000 7fd3b3be7640 1 Processor -- start 2023-10-08T05:52:21.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.212+0000 7fd3b3be7640 1 -- start start 2023-10-08T05:52:21.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.213+0000 7fd3b3be7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3ac06dfc0 0x7fd3ac0c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.213+0000 7fd3b3be7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac071070 0x7fd3ac0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.213+0000 7fd3b3be7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 0x7fd3ac151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.213+0000 7fd3b3be7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac152250 con 0x7fd3ac071070 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.213+0000 7fd3b3be7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0ce040 con 0x7fd3ac06dfc0 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.213+0000 7fd3b3be7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0ce320 con 0x7fd3ac0afbe0 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b1182640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac071070 0x7fd3ac0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b1182640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac071070 0x7fd3ac0cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35570/0 (socket says 172.21.15.70:35570) 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b1182640 1 -- 172.21.15.70:0/2932555047 learned_addr learned my addr 172.21.15.70:0/2932555047 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b1983640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3ac06dfc0 0x7fd3ac0c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b2184640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 0x7fd3ac151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b1182640 1 -- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 msgr2=0x7fd3ac151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b1182640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 0x7fd3ac151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b1182640 1 -- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3ac06dfc0 msgr2=0x7fd3ac0c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:21.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b1182640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3ac06dfc0 0x7fd3ac0c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b1182640 1 -- 172.21.15.70:0/2932555047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39c0094d0 con 0x7fd3ac071070 2023-10-08T05:52:21.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.214+0000 7fd3b1182640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac071070 0x7fd3ac0cf0e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd39c02b940 tx=0x7fd39c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:21.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.215+0000 7fd3a27fc640 1 -- 172.21.15.70:0/2932555047 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c031850 con 0x7fd3ac071070 2023-10-08T05:52:21.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.215+0000 7fd3a27fc640 1 -- 172.21.15.70:0/2932555047 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c0443f0 con 0x7fd3ac071070 2023-10-08T05:52:21.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.215+0000 7fd3a27fc640 1 -- 172.21.15.70:0/2932555047 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c0447a0 con 0x7fd3ac071070 2023-10-08T05:52:21.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.215+0000 7fd3b3be7640 1 -- 172.21.15.70:0/2932555047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ac0ce600 con 0x7fd3ac071070 2023-10-08T05:52:21.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.215+0000 7fd3b3be7640 1 -- 172.21.15.70:0/2932555047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ac0cebc0 con 0x7fd3ac071070 2023-10-08T05:52:21.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.216+0000 7fd3b3be7640 1 -- 172.21.15.70:0/2932555047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ac04e160 con 0x7fd3ac071070 2023-10-08T05:52:21.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.217+0000 7fd3a27fc640 1 -- 172.21.15.70:0/2932555047 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd39c04d450 con 0x7fd3ac071070 2023-10-08T05:52:21.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.218+0000 7fd3a27fc640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd39006af90 0x7fd39006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.218+0000 7fd3a27fc640 1 -- 172.21.15.70:0/2932555047 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fd39c0abfa0 con 0x7fd3ac071070 2023-10-08T05:52:21.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.218+0000 7fd3b1983640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd39006af90 0x7fd39006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.219+0000 7fd3b1983640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd39006af90 0x7fd39006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800ab60 tx=0x7fd3a8008040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:21.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.222+0000 7fd3a27fc640 1 -- 172.21.15.70:0/2932555047 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd39c0deca0 con 0x7fd3ac071070 2023-10-08T05:52:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.414+0000 7fd3b3be7640 1 -- 172.21.15.70:0/2932555047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd3ac0b0ef0 con 0x7fd3ac071070 2023-10-08T05:52:21.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.535+0000 7fd3a27fc640 1 -- 172.21.15.70:0/2932555047 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v84) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd39c035070 con 0x7fd3ac071070 2023-10-08T05:52:21.536 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T05:52:21.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.538+0000 7fd3b3be7640 1 -- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd39006af90 msgr2=0x7fd39006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:21.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.538+0000 7fd3b3be7640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd39006af90 0x7fd39006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800ab60 tx=0x7fd3a8008040 comp rx=0 tx=0).stop 2023-10-08T05:52:21.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.538+0000 7fd3b3be7640 1 -- 172.21.15.70:0/2932555047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac071070 msgr2=0x7fd3ac0cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:21.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.538+0000 7fd3b3be7640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac071070 0x7fd3ac0cf0e0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd39c02b940 tx=0x7fd39c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.538+0000 7fd3b3be7640 1 -- 172.21.15.70:0/2932555047 shutdown_connections 2023-10-08T05:52:21.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.538+0000 7fd3b3be7640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fd39006af90 0x7fd39006d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.538+0000 7fd3b3be7640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3ac0afbe0 0x7fd3ac151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.539+0000 7fd3b3be7640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3ac071070 0x7fd3ac0cf0e0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.539+0000 7fd3b3be7640 1 --2- 172.21.15.70:0/2932555047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3ac06dfc0 0x7fd3ac0c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.539+0000 7fd3b3be7640 1 -- 172.21.15.70:0/2932555047 >> 172.21.15.70:0/2932555047 conn(0x7fd3ac06a5d0 msgr2=0x7fd3ac0c91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:21.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.539+0000 7fd3b3be7640 1 -- 172.21.15.70:0/2932555047 shutdown_connections 2023-10-08T05:52:21.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.539+0000 7fd3b3be7640 1 -- 172.21.15.70:0/2932555047 wait complete. 2023-10-08T05:52:21.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T05:52:21.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.769+0000 7f6ce2432640 1 Processor -- start 2023-10-08T05:52:21.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.770+0000 7f6ce2432640 1 -- start start 2023-10-08T05:52:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.771+0000 7f6ce2432640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc159ac0 0x7f6cdc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.771+0000 7f6ce2432640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6cdc0709e0 0x7f6cdc0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.771+0000 7f6ce2432640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc06dee0 con 0x7f6cdc0ad790 2023-10-08T05:52:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.771+0000 7f6ce2432640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc15d850 con 0x7f6cdc0709e0 2023-10-08T05:52:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.771+0000 7f6ce2432640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc15db30 con 0x7f6cdc159ac0 2023-10-08T05:52:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.771+0000 7f6cdb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6cdc0ad790 0x7f6cdc0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60504/0 (socket says 172.21.15.70:60504) 2023-10-08T05:52:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.771+0000 7f6cdb7fe640 1 -- 172.21.15.70:0/1659151653 learned_addr learned my addr 172.21.15.70:0/1659151653 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:21.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519692164 0 0) 0x7f6cdc06dee0 con 0x7f6cdc0ad790 2023-10-08T05:52:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc8005730 con 0x7f6cdc0ad790 2023-10-08T05:52:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdbfff640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc159ac0 0x7f6cdc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6ce09cf640 1 --2- 172.21.15.70:0/1659151653 >> v2:172.21.15.181:3300/0 conn(0x7f6cdc0709e0 0x7f6cdc0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 738339418 0 0) 0x7f6cc8005730 con 0x7f6cdc0ad790 2023-10-08T05:52:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc159ac0 msgr2=0x7f6cdc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc159ac0 0x7f6cdc06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 >> v2:172.21.15.181:3300/0 conn(0x7f6cdc0709e0 msgr2=0x7f6cdc0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 --2- 172.21.15.70:0/1659151653 >> v2:172.21.15.181:3300/0 conn(0x7f6cdc0709e0 0x7f6cdc0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc15de10 con 0x7f6cdc0ad790 2023-10-08T05:52:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6cc4001670 con 0x7f6cdc0ad790 2023-10-08T05:52:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6ce09cf640 1 --2- 172.21.15.70:0/1659151653 >> v2:172.21.15.181:3300/0 conn(0x7f6cdc0709e0 0x7f6cdc0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 >> v1:172.21.15.70:6789/0 conn(0x7f6cdc0ad790 legacy=0x7f6cdc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc159ac0 0x7f6cc8005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc8006200 0x7f6cc80069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.772+0000 7f6cdaffd640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 0x7f6cc8007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc15d850 con 0x7f6cc8006f10 2023-10-08T05:52:21.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc15db30 con 0x7f6cc8006200 2023-10-08T05:52:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc800ddd0 con 0x7f6cdc159ac0 2023-10-08T05:52:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6ce09cf640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 0x7f6cc8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6cdbfff640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc159ac0 0x7f6cc8005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6cdb7fe640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc8006200 0x7f6cc80069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6ce09cf640 1 -- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc159ac0 msgr2=0x7f6cc8005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6ce09cf640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc159ac0 0x7f6cc8005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6ce09cf640 1 -- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc8006200 msgr2=0x7f6cc80069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6ce09cf640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cc8006200 0x7f6cc80069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.773+0000 7f6ce09cf640 1 -- 172.21.15.70:0/1659151653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc8005730 con 0x7f6cc8006f10 2023-10-08T05:52:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.774+0000 7f6ce09cf640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 0x7f6cc8007390 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0caad0 tx=0x7f6cc8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1621016cc94588dc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.775+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd402f640 con 0x7f6cc8006f10 2023-10-08T05:52:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.775+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd402f7a0 con 0x7f6cc8006f10 2023-10-08T05:52:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.775+0000 7f6cdaffd640 1 -- 172.21.15.70:0/1659151653 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd402fb50 con 0x7f6cc8006f10 2023-10-08T05:52:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.775+0000 7f6ce2432640 1 -- 172.21.15.70:0/1659151653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 msgr2=0x7f6cc8007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.775+0000 7f6ce2432640 1 --2- 172.21.15.70:0/1659151653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 0x7f6cc8007390 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0caad0 tx=0x7f6cc8007a00 comp rx=0 tx=0).stop 2023-10-08T05:52:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.775+0000 7f6ce09cf640 1 -- 172.21.15.70:0/1659151653 reap_dead start 2023-10-08T05:52:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.775+0000 7f6ce2432640 1 -- 172.21.15.70:0/1659151653 shutdown_connections 2023-10-08T05:52:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.775+0000 7f6ce2432640 1 -- 172.21.15.70:0/1659151653 >> 172.21.15.70:0/1659151653 conn(0x7f6cdc06a380 msgr2=0x7f6cdc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:21.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.775+0000 7f6ce2432640 1 -- 172.21.15.70:0/1659151653 shutdown_connections 2023-10-08T05:52:21.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.775+0000 7f6ce2432640 1 -- 172.21.15.70:0/1659151653 wait complete. 2023-10-08T05:52:21.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.777+0000 7f6ce2432640 1 Processor -- start 2023-10-08T05:52:21.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.777+0000 7f6ce2432640 1 -- start start 2023-10-08T05:52:21.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.778+0000 7f6ce2432640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 0x7f6cdc0b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.778+0000 7f6ce2432640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc1586a0 0x7f6cdc0b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.778+0000 7f6ce2432640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc158b20 0x7f6cdc0b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.778+0000 7f6ce2432640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0b14a0 con 0x7f6cc8006f10 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.778+0000 7f6ce2432640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc151730 con 0x7f6cdc158b20 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.778+0000 7f6ce2432640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0b16f0 con 0x7f6cdc1586a0 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.778+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 0x7f6cdc0b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.778+0000 7f6cdbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 0x7f6cdc0b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35596/0 (socket says 172.21.15.70:35596) 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.778+0000 7f6cdbfff640 1 -- 172.21.15.70:0/2198654806 learned_addr learned my addr 172.21.15.70:0/2198654806 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6cdb7fe640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc1586a0 0x7f6cdc0b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6cdbfff640 1 -- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc1586a0 msgr2=0x7f6cdc0b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6ce09cf640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc158b20 0x7f6cdc0b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6cdbfff640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc1586a0 0x7f6cdc0b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6cdbfff640 1 -- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc158b20 msgr2=0x7f6cdc0b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:21.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6cdbfff640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc158b20 0x7f6cdc0b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6cdbfff640 1 -- 172.21.15.70:0/2198654806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0b19d0 con 0x7f6cc8006f10 2023-10-08T05:52:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6ce09cf640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc158b20 0x7f6cdc0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6cdbfff640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 0x7f6cdc0b24b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009ce0 tx=0x7f6ccc007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6cd8ff9640 1 -- 172.21.15.70:0/2198654806 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc014610 con 0x7f6cc8006f10 2023-10-08T05:52:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6cd8ff9640 1 -- 172.21.15.70:0/2198654806 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc00f050 con 0x7f6cc8006f10 2023-10-08T05:52:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6cd8ff9640 1 -- 172.21.15.70:0/2198654806 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0149b0 con 0x7f6cc8006f10 2023-10-08T05:52:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.779+0000 7f6ce2432640 1 -- 172.21.15.70:0/2198654806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc0b1cb0 con 0x7f6cc8006f10 2023-10-08T05:52:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.780+0000 7f6ce2432640 1 -- 172.21.15.70:0/2198654806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc143f00 con 0x7f6cc8006f10 2023-10-08T05:52:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.781+0000 7f6ce2432640 1 -- 172.21.15.70:0/2198654806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cdc067490 con 0x7f6cc8006f10 2023-10-08T05:52:21.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.782+0000 7f6cd8ff9640 1 -- 172.21.15.70:0/2198654806 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6ccc014b10 con 0x7f6cc8006f10 2023-10-08T05:52:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.783+0000 7f6cd8ff9640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6cbc06afc0 0x7f6cbc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.783+0000 7f6cd8ff9640 1 -- 172.21.15.70:0/2198654806 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f6ccc089ef0 con 0x7f6cc8006f10 2023-10-08T05:52:21.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.783+0000 7f6cdb7fe640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6cbc06afc0 0x7f6cbc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.785+0000 7f6cdb7fe640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6cbc06afc0 0x7f6cbc06d480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc1594d0 tx=0x7f6cc4009040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:21.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:21.787+0000 7f6cd8ff9640 1 -- 172.21.15.70:0/2198654806 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ccc054ff0 con 0x7f6cc8006f10 2023-10-08T05:52:22.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.040+0000 7f6ce2432640 1 -- 172.21.15.70:0/2198654806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6cdc0b0ff0 con 0x7f6cc8006f10 2023-10-08T05:52:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.041+0000 7f6cd8ff9640 1 -- 172.21.15.70:0/2198654806 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6ccc058610 con 0x7f6cc8006f10 2023-10-08T05:52:22.043 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:22.043 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:52:22.043 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T05:52:22.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.044 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T05:52:22.044 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T05:52:22.044 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.044 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:52:22.044 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:52:22.044 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:52:22.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.044 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.045 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T05:52:22.045 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T05:52:22.045 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.045 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:52:22.045 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:52:22.045 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:52:22.045 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.045 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.046 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T05:52:22.046 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T05:52:22.046 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.046 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:52:22.046 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:52:22.046 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:52:22.046 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.046 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.048 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T05:52:22.049 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.050 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:52:22.050 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:52:22.050 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.050 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T05:52:22.050 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T05:52:22.050 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.050 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:52:22.051 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.052 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T05:52:22.053 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:52:22.054 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:52:22.055 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:52:22.056 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.057 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T05:52:22.057 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T05:52:22.057 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.057 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:52:22.057 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:52:22.057 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:52:22.057 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:52:22.057 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.058 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T05:52:22.058 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T05:52:22.058 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.058 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:52:22.058 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:52:22.058 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:52:22.058 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.059 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.059 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T05:52:22.059 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T05:52:22.059 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.059 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:52:22.059 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:52:22.059 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.059 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:22.060 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T05:52:22.061 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:52:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.044+0000 7f6ce2432640 1 -- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6cbc06afc0 msgr2=0x7f6cbc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.044+0000 7f6ce2432640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6cbc06afc0 0x7f6cbc06d480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc1594d0 tx=0x7f6cc4009040 comp rx=0 tx=0).stop 2023-10-08T05:52:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.044+0000 7f6ce2432640 1 -- 172.21.15.70:0/2198654806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 msgr2=0x7f6cdc0b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.044+0000 7f6ce2432640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 0x7f6cdc0b24b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009ce0 tx=0x7f6ccc007700 comp rx=0 tx=0).stop 2023-10-08T05:52:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.045+0000 7f6ce2432640 1 -- 172.21.15.70:0/2198654806 shutdown_connections 2023-10-08T05:52:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.045+0000 7f6ce2432640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f6cbc06afc0 0x7f6cbc06d480 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.045+0000 7f6ce2432640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6cdc158b20 0x7f6cdc0b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.045+0000 7f6ce2432640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6cdc1586a0 0x7f6cdc0b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.045+0000 7f6ce2432640 1 --2- 172.21.15.70:0/2198654806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cc8006f10 0x7f6cdc0b24b0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.045+0000 7f6ce2432640 1 -- 172.21.15.70:0/2198654806 >> 172.21.15.70:0/2198654806 conn(0x7f6cdc06a380 msgr2=0x7f6cdc0c4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:22.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.045+0000 7f6ce2432640 1 -- 172.21.15.70:0/2198654806 shutdown_connections 2023-10-08T05:52:22.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.045+0000 7f6ce2432640 1 -- 172.21.15.70:0/2198654806 wait complete. 2023-10-08T05:52:22.064 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:22.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.268+0000 7f692b973640 1 Processor -- start 2023-10-08T05:52:22.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.268+0000 7f692b973640 1 -- start start 2023-10-08T05:52:22.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f692b973640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6924159ac0 0x7f692406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f692b973640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69240ad790 0x7f69240702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f692b973640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f692415e330 0x7f6924070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f692b973640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692406dee0 con 0x7f692415e330 2023-10-08T05:52:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f692b973640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692415d850 con 0x7f6924159ac0 2023-10-08T05:52:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f692b973640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692415db30 con 0x7f69240ad790 2023-10-08T05:52:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f6929f10640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f692415e330 0x7f6924070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f6929f10640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f692415e330 0x7f6924070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35604/0 (socket says 172.21.15.70:35604) 2023-10-08T05:52:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f6929f10640 1 -- 172.21.15.70:0/2589715231 learned_addr learned my addr 172.21.15.70:0/2589715231 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f6929f10640 1 -- 172.21.15.70:0/2589715231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69240ad790 msgr2=0x7f69240702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f6929f10640 1 --2- 172.21.15.70:0/2589715231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69240ad790 0x7f69240702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f6929f10640 1 -- 172.21.15.70:0/2589715231 >> v2:172.21.15.181:3300/0 conn(0x7f6924159ac0 msgr2=0x7f692406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:22.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f6929f10640 1 --2- 172.21.15.70:0/2589715231 >> v2:172.21.15.181:3300/0 conn(0x7f6924159ac0 0x7f692406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.270+0000 7f6929f10640 1 -- 172.21.15.70:0/2589715231 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692415de10 con 0x7f692415e330 2023-10-08T05:52:22.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.271+0000 7f6929f10640 1 --2- 172.21.15.70:0/2589715231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f692415e330 0x7f6924070810 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f692406e790 tx=0x7f691c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3b10d182aec1ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:22.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.272+0000 7f691bfff640 1 -- 172.21.15.70:0/2589715231 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c02ca80 con 0x7f692415e330 2023-10-08T05:52:22.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.272+0000 7f691bfff640 1 -- 172.21.15.70:0/2589715231 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c02cbe0 con 0x7f692415e330 2023-10-08T05:52:22.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.272+0000 7f691bfff640 1 -- 172.21.15.70:0/2589715231 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c0343f0 con 0x7f692415e330 2023-10-08T05:52:22.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.272+0000 7f692b973640 1 -- 172.21.15.70:0/2589715231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f692415e330 msgr2=0x7f6924070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.272+0000 7f692b973640 1 --2- 172.21.15.70:0/2589715231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f692415e330 0x7f6924070810 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f692406e790 tx=0x7f691c02bee0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.272+0000 7f692b973640 1 -- 172.21.15.70:0/2589715231 shutdown_connections 2023-10-08T05:52:22.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.272+0000 7f692b973640 1 --2- 172.21.15.70:0/2589715231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f692415e330 0x7f6924070810 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.272+0000 7f692b973640 1 --2- 172.21.15.70:0/2589715231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69240ad790 0x7f69240702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.272+0000 7f692b973640 1 --2- 172.21.15.70:0/2589715231 >> v2:172.21.15.181:3300/0 conn(0x7f6924159ac0 0x7f692406d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.272+0000 7f692b973640 1 -- 172.21.15.70:0/2589715231 >> 172.21.15.70:0/2589715231 conn(0x7f692406a380 msgr2=0x7f69240b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:22.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.273+0000 7f692b973640 1 -- 172.21.15.70:0/2589715231 shutdown_connections 2023-10-08T05:52:22.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.273+0000 7f692b973640 1 -- 172.21.15.70:0/2589715231 wait complete. 2023-10-08T05:52:22.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.274+0000 7f692b973640 1 Processor -- start 2023-10-08T05:52:22.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.274+0000 7f692b973640 1 -- start start 2023-10-08T05:52:22.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.275+0000 7f692b973640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69240ad790 0x7f69240b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.275+0000 7f692b973640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6924159ac0 0x7f6924152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.275+0000 7f692b973640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f692415e330 0x7f6924158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.275+0000 7f692b973640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6924153b30 con 0x7f69240ad790 2023-10-08T05:52:22.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.275+0000 7f692b973640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69240b42a0 con 0x7f692415e330 2023-10-08T05:52:22.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.275+0000 7f692b973640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6924153d40 con 0x7f6924159ac0 2023-10-08T05:52:22.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.276+0000 7f692970f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69240ad790 0x7f69240b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:22.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.276+0000 7f692970f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69240ad790 0x7f69240b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35614/0 (socket says 172.21.15.70:35614) 2023-10-08T05:52:22.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.276+0000 7f692970f640 1 -- 172.21.15.70:0/3100210576 learned_addr learned my addr 172.21.15.70:0/3100210576 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:22.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.276+0000 7f692970f640 1 -- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6924159ac0 msgr2=0x7f6924152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.276+0000 7f692970f640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6924159ac0 0x7f6924152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.276+0000 7f692970f640 1 -- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f692415e330 msgr2=0x7f6924158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.276+0000 7f692970f640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f692415e330 0x7f6924158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.276+0000 7f692970f640 1 -- 172.21.15.70:0/3100210576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c009b30 con 0x7f69240ad790 2023-10-08T05:52:22.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.276+0000 7f692970f640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69240ad790 0x7f69240b3d60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f692000b4a0 tx=0x7f692000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:22.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.277+0000 7f6919ffb640 1 -- 172.21.15.70:0/3100210576 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920010610 con 0x7f69240ad790 2023-10-08T05:52:22.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.277+0000 7f6919ffb640 1 -- 172.21.15.70:0/3100210576 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6920010770 con 0x7f69240ad790 2023-10-08T05:52:22.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.277+0000 7f6919ffb640 1 -- 172.21.15.70:0/3100210576 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920010b20 con 0x7f69240ad790 2023-10-08T05:52:22.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.277+0000 7f692b973640 1 -- 172.21.15.70:0/3100210576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6924154020 con 0x7f69240ad790 2023-10-08T05:52:22.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.277+0000 7f692b973640 1 -- 172.21.15.70:0/3100210576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69241545e0 con 0x7f69240ad790 2023-10-08T05:52:22.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.278+0000 7f692b973640 1 -- 172.21.15.70:0/3100210576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6924067490 con 0x7f69240ad790 2023-10-08T05:52:22.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.279+0000 7f6919ffb640 1 -- 172.21.15.70:0/3100210576 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6920010c80 con 0x7f69240ad790 2023-10-08T05:52:22.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.280+0000 7f6919ffb640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f690806afc0 0x7f690806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.280+0000 7f6919ffb640 1 -- 172.21.15.70:0/3100210576 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f69200891f0 con 0x7f69240ad790 2023-10-08T05:52:22.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.280+0000 7f6928f0e640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f690806afc0 0x7f690806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:22.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.282+0000 7f6928f0e640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f690806afc0 0x7f690806d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69241f5c80 tx=0x7f6914006c90 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:22.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.284+0000 7f6919ffb640 1 -- 172.21.15.70:0/3100210576 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6920054360 con 0x7f69240ad790 2023-10-08T05:52:22.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.473+0000 7f692b973640 1 -- 172.21.15.70:0/3100210576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69240b0ff0 con 0x7f69240ad790 2023-10-08T05:52:22.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.475+0000 7f6919ffb640 1 -- 172.21.15.70:0/3100210576 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13120 (secure 0 0 0) 0x7f6920057980 con 0x7f69240ad790 2023-10-08T05:52:22.476 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:22.476 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":86,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:21.548903+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:22.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 -- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f690806afc0 msgr2=0x7f690806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f690806afc0 0x7f690806d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69241f5c80 tx=0x7f6914006c90 comp rx=0 tx=0).stop 2023-10-08T05:52:22.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 -- 172.21.15.70:0/3100210576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69240ad790 msgr2=0x7f69240b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69240ad790 0x7f69240b3d60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f692000b4a0 tx=0x7f692000b960 comp rx=0 tx=0).stop 2023-10-08T05:52:22.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 -- 172.21.15.70:0/3100210576 shutdown_connections 2023-10-08T05:52:22.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f690806afc0 0x7f690806d480 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f692415e330 0x7f6924158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6924159ac0 0x7f6924152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 --2- 172.21.15.70:0/3100210576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69240ad790 0x7f69240b3d60 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 -- 172.21.15.70:0/3100210576 >> 172.21.15.70:0/3100210576 conn(0x7f692406a380 msgr2=0x7f69240ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.477+0000 7f692b973640 1 -- 172.21.15.70:0/3100210576 shutdown_connections 2023-10-08T05:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.478+0000 7f692b973640 1 -- 172.21.15.70:0/3100210576 wait complete. 2023-10-08T05:52:22.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:52:22.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:52:22.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.712+0000 7fec10e90640 1 Processor -- start 2023-10-08T05:52:22.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.712+0000 7fec10e90640 1 -- start start 2023-10-08T05:52:22.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.713+0000 7fec10e90640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fec0c15cc80 0x7fec0c06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.713+0000 7fec10e90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0c06dab0 0x7fec0c0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec10e90640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0c06a130 con 0x7fec0c06def0 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec10e90640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0c06b710 con 0x7fec0c15cc80 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec10e90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c06a410 con 0x7fec0c06dab0 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec0a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fec0c15cc80 0x7fec0c06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec0a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fec0c15cc80 0x7fec0c06b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35616/0 (socket says 172.21.15.70:35616) 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec0a59c640 1 -- 172.21.15.70:0/1219494415 learned_addr learned my addr 172.21.15.70:0/1219494415 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec03fff640 1 --2- 172.21.15.70:0/1219494415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0c06dab0 0x7fec0c0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec0a59c640 1 -- 172.21.15.70:0/1219494415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0c06dab0 msgr2=0x7fec0c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec0a59c640 1 --2- 172.21.15.70:0/1219494415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0c06dab0 0x7fec0c0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec0a59c640 1 -- 172.21.15.70:0/1219494415 >> v1:172.21.15.181:6789/0 conn(0x7fec0c06def0 legacy=0x7fec0c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec0a59c640 1 -- 172.21.15.70:0/1219494415 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c06a6f0 con 0x7fec0c15cc80 2023-10-08T05:52:22.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.714+0000 7fec03fff640 1 --2- 172.21.15.70:0/1219494415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0c06dab0 0x7fec0c0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:22.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.715+0000 7fec0a59c640 1 --2- 172.21.15.70:0/1219494415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c15cc80 0x7fec0c06b1d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7febfc009870 tx=0x7febfc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f32ed2cb439236 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:22.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.716+0000 7fec09d9b640 1 -- 172.21.15.70:0/1219494415 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc031640 con 0x7fec0c15cc80 2023-10-08T05:52:22.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.716+0000 7fec09d9b640 1 -- 172.21.15.70:0/1219494415 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc0317a0 con 0x7fec0c15cc80 2023-10-08T05:52:22.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.716+0000 7fec09d9b640 1 -- 172.21.15.70:0/1219494415 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc031b30 con 0x7fec0c15cc80 2023-10-08T05:52:22.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.716+0000 7fec10e90640 1 -- 172.21.15.70:0/1219494415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c15cc80 msgr2=0x7fec0c06b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.716+0000 7fec10e90640 1 --2- 172.21.15.70:0/1219494415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c15cc80 0x7fec0c06b1d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7febfc009870 tx=0x7febfc02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.717+0000 7fec10e90640 1 -- 172.21.15.70:0/1219494415 shutdown_connections 2023-10-08T05:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.717+0000 7fec10e90640 1 --2- 172.21.15.70:0/1219494415 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0c06dab0 0x7fec0c0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.717+0000 7fec10e90640 1 --2- 172.21.15.70:0/1219494415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c15cc80 0x7fec0c06b1d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.717+0000 7fec10e90640 1 -- 172.21.15.70:0/1219494415 >> 172.21.15.70:0/1219494415 conn(0x7fec0c0b0ae0 msgr2=0x7fec0c0b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.717+0000 7fec10e90640 1 -- 172.21.15.70:0/1219494415 shutdown_connections 2023-10-08T05:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.717+0000 7fec10e90640 1 -- 172.21.15.70:0/1219494415 wait complete. 2023-10-08T05:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.718+0000 7fec10e90640 1 Processor -- start 2023-10-08T05:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.718+0000 7fec10e90640 1 -- start start 2023-10-08T05:52:22.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.719+0000 7fec10e90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c06dab0 0x7fec0c1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.719+0000 7fec10e90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec0c15cc80 0x7fec0c0c3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.719+0000 7fec10e90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0c0c3d70 0x7fec0c15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.719+0000 7fec10e90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c149a00 con 0x7fec0c06dab0 2023-10-08T05:52:22.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.719+0000 7fec10e90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c15d710 con 0x7fec0c15cc80 2023-10-08T05:52:22.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.719+0000 7fec10e90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c15d100 con 0x7fec0c0c3d70 2023-10-08T05:52:22.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.720+0000 7fec0a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c06dab0 0x7fec0c1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:22.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.720+0000 7fec0a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c06dab0 0x7fec0c1494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35620/0 (socket says 172.21.15.70:35620) 2023-10-08T05:52:22.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.720+0000 7fec0a59c640 1 -- 172.21.15.70:0/1263258168 learned_addr learned my addr 172.21.15.70:0/1263258168 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:22.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.720+0000 7fec0a59c640 1 -- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0c0c3d70 msgr2=0x7fec0c15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.720+0000 7fec0a59c640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0c0c3d70 0x7fec0c15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.720+0000 7fec0a59c640 1 -- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec0c15cc80 msgr2=0x7fec0c0c3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.720+0000 7fec0a59c640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec0c15cc80 0x7fec0c0c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.720+0000 7fec0a59c640 1 -- 172.21.15.70:0/1263258168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc0094d0 con 0x7fec0c06dab0 2023-10-08T05:52:22.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.721+0000 7fec0a59c640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c06dab0 0x7fec0c1494c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7febfc009870 tx=0x7febfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:22.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.721+0000 7fec02ffd640 1 -- 172.21.15.70:0/1263258168 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc03ac60 con 0x7fec0c06dab0 2023-10-08T05:52:22.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.721+0000 7fec02ffd640 1 -- 172.21.15.70:0/1263258168 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc03ade0 con 0x7fec0c06dab0 2023-10-08T05:52:22.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.721+0000 7fec02ffd640 1 -- 172.21.15.70:0/1263258168 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc044630 con 0x7fec0c06dab0 2023-10-08T05:52:22.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.721+0000 7fec10e90640 1 -- 172.21.15.70:0/1263258168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c0c45b0 con 0x7fec0c06dab0 2023-10-08T05:52:22.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.721+0000 7fec10e90640 1 -- 172.21.15.70:0/1263258168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c0c4890 con 0x7fec0c06dab0 2023-10-08T05:52:22.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.722+0000 7fec10e90640 1 -- 172.21.15.70:0/1263258168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c04e160 con 0x7fec0c06dab0 2023-10-08T05:52:22.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.723+0000 7fec02ffd640 1 -- 172.21.15.70:0/1263258168 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7febfc02f050 con 0x7fec0c06dab0 2023-10-08T05:52:22.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.724+0000 7fec02ffd640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7febec06afc0 0x7febec06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:22.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.724+0000 7fec02ffd640 1 -- 172.21.15.70:0/1263258168 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5221+0+0 (secure 0 0 0) 0x7febfc0ab910 con 0x7fec0c06dab0 2023-10-08T05:52:22.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.724+0000 7fec03fff640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7febec06afc0 0x7febec06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:22.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.725+0000 7fec03fff640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7febec06afc0 0x7febec06d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7febf4006fd0 tx=0x7febf4008040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:22.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.728+0000 7fec02ffd640 1 -- 172.21.15.70:0/1263258168 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7febfc0768e0 con 0x7fec0c06dab0 2023-10-08T05:52:22.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.904+0000 7fec10e90640 1 -- 172.21.15.70:0/1263258168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec0c065c60 con 0x7fec0c06dab0 2023-10-08T05:52:22.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.905+0000 7fec02ffd640 1 -- 172.21.15.70:0/1263258168 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7febfc032040 con 0x7fec0c06dab0 2023-10-08T05:52:22.907 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:52:22.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 -- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7febec06afc0 msgr2=0x7febec06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7febec06afc0 0x7febec06d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7febf4006fd0 tx=0x7febf4008040 comp rx=0 tx=0).stop 2023-10-08T05:52:22.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 -- 172.21.15.70:0/1263258168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c06dab0 msgr2=0x7fec0c1494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:22.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c06dab0 0x7fec0c1494c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7febfc009870 tx=0x7febfc0046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 -- 172.21.15.70:0/1263258168 shutdown_connections 2023-10-08T05:52:22.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7febec06afc0 0x7febec06d480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec0c0c3d70 0x7fec0c15dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec0c15cc80 0x7fec0c0c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 --2- 172.21.15.70:0/1263258168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c06dab0 0x7fec0c1494c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:22.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 -- 172.21.15.70:0/1263258168 >> 172.21.15.70:0/1263258168 conn(0x7fec0c0b0ae0 msgr2=0x7fec0c0abfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:22.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 -- 172.21.15.70:0/1263258168 shutdown_connections 2023-10-08T05:52:22.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:22.908+0000 7fec10e90640 1 -- 172.21.15.70:0/1263258168 wait complete. 2023-10-08T05:52:22.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T05:52:23.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.140+0000 7fe3d58df640 1 Processor -- start 2023-10-08T05:52:23.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.140+0000 7fe3d58df640 1 -- start start 2023-10-08T05:52:23.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.141+0000 7fe3d58df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe3d00adc70 0x7fe3d00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.141+0000 7fe3d58df640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3d015af10 0x7fe3d00b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.141+0000 7fe3d58df640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3d015f7d0 con 0x7fe3d015ab30 2023-10-08T05:52:23.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.141+0000 7fe3d58df640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3d015efb0 con 0x7fe3d00adc70 2023-10-08T05:52:23.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.141+0000 7fe3d58df640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d015f210 con 0x7fe3d015af10 2023-10-08T05:52:23.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.142+0000 7fe3cffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe3d00adc70 0x7fe3d00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.142+0000 7fe3d48dd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe3d015ab30 0x7fe3d015e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60520/0 (socket says 172.21.15.70:60520) 2023-10-08T05:52:23.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.142+0000 7fe3d48dd640 1 -- 172.21.15.70:0/3348901422 learned_addr learned my addr 172.21.15.70:0/3348901422 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:23.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.142+0000 7fe3d50de640 1 --2- 172.21.15.70:0/3348901422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3d015af10 0x7fe3d00b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.142+0000 7fe3cffff640 1 -- 172.21.15.70:0/3348901422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3d015af10 msgr2=0x7fe3d00b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.142+0000 7fe3cffff640 1 --2- 172.21.15.70:0/3348901422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3d015af10 0x7fe3d00b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.142+0000 7fe3cffff640 1 -- 172.21.15.70:0/3348901422 >> v1:172.21.15.70:6789/0 conn(0x7fe3d015ab30 legacy=0x7fe3d015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.142+0000 7fe3cffff640 1 -- 172.21.15.70:0/3348901422 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d015f4f0 con 0x7fe3d00adc70 2023-10-08T05:52:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.144+0000 7fe3cffff640 1 --2- 172.21.15.70:0/3348901422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 0x7fe3d00b00c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0097a0 tx=0x7fe3bc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd99f2a895d7f40 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.144+0000 7fe3cf7fe640 1 -- 172.21.15.70:0/3348901422 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc02d040 con 0x7fe3d00adc70 2023-10-08T05:52:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.144+0000 7fe3cf7fe640 1 -- 172.21.15.70:0/3348901422 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc031810 con 0x7fe3d00adc70 2023-10-08T05:52:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.144+0000 7fe3cf7fe640 1 -- 172.21.15.70:0/3348901422 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc031ba0 con 0x7fe3d00adc70 2023-10-08T05:52:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.144+0000 7fe3d58df640 1 -- 172.21.15.70:0/3348901422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 msgr2=0x7fe3d00b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.144+0000 7fe3d58df640 1 --2- 172.21.15.70:0/3348901422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 0x7fe3d00b00c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0097a0 tx=0x7fe3bc02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.145+0000 7fe3d58df640 1 -- 172.21.15.70:0/3348901422 shutdown_connections 2023-10-08T05:52:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.145+0000 7fe3d58df640 1 --2- 172.21.15.70:0/3348901422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3d015af10 0x7fe3d00b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.145+0000 7fe3d58df640 1 --2- 172.21.15.70:0/3348901422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 0x7fe3d00b00c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.145+0000 7fe3d58df640 1 -- 172.21.15.70:0/3348901422 >> 172.21.15.70:0/3348901422 conn(0x7fe3d00cb0c0 msgr2=0x7fe3d00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.145+0000 7fe3d58df640 1 -- 172.21.15.70:0/3348901422 shutdown_connections 2023-10-08T05:52:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.145+0000 7fe3d58df640 1 -- 172.21.15.70:0/3348901422 wait complete. 2023-10-08T05:52:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.146+0000 7fe3d58df640 1 Processor -- start 2023-10-08T05:52:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.147+0000 7fe3d58df640 1 -- start start 2023-10-08T05:52:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.147+0000 7fe3d58df640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 0x7fe3d01e5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.147+0000 7fe3d58df640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d015ab30 0x7fe3d0159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.147+0000 7fe3d58df640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3d015af10 0x7fe3d014e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.147+0000 7fe3d58df640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d015b380 con 0x7fe3d00adc70 2023-10-08T05:52:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.147+0000 7fe3d58df640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01e4bd0 con 0x7fe3d015ab30 2023-10-08T05:52:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.147+0000 7fe3d58df640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01e4eb0 con 0x7fe3d015af10 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.148+0000 7fe3d48dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 0x7fe3d01e5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.148+0000 7fe3d48dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 0x7fe3d01e5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35636/0 (socket says 172.21.15.70:35636) 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.148+0000 7fe3d48dd640 1 -- 172.21.15.70:0/4133631916 learned_addr learned my addr 172.21.15.70:0/4133631916 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.148+0000 7fe3d50de640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3d015af10 0x7fe3d014e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.148+0000 7fe3cffff640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d015ab30 0x7fe3d0159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.148+0000 7fe3d48dd640 1 -- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3d015af10 msgr2=0x7fe3d014e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.148+0000 7fe3d48dd640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3d015af10 0x7fe3d014e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.148+0000 7fe3d48dd640 1 -- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d015ab30 msgr2=0x7fe3d0159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.148+0000 7fe3d48dd640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d015ab30 0x7fe3d0159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.148+0000 7fe3d48dd640 1 -- 172.21.15.70:0/4133631916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc009400 con 0x7fe3d00adc70 2023-10-08T05:52:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.149+0000 7fe3d48dd640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 0x7fe3d01e5ea0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400c450 tx=0x7fe3c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.149+0000 7fe3cd7fa640 1 -- 172.21.15.70:0/4133631916 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4005ce0 con 0x7fe3d00adc70 2023-10-08T05:52:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.149+0000 7fe3cd7fa640 1 -- 172.21.15.70:0/4133631916 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c4005e40 con 0x7fe3d00adc70 2023-10-08T05:52:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.149+0000 7fe3cd7fa640 1 -- 172.21.15.70:0/4133631916 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40072b0 con 0x7fe3d00adc70 2023-10-08T05:52:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.149+0000 7fe3d58df640 1 -- 172.21.15.70:0/4133631916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d01e5190 con 0x7fe3d00adc70 2023-10-08T05:52:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.149+0000 7fe3d58df640 1 -- 172.21.15.70:0/4133631916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d01e63e0 con 0x7fe3d00adc70 2023-10-08T05:52:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.150+0000 7fe3d58df640 1 -- 172.21.15.70:0/4133631916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d01e56b0 con 0x7fe3d00adc70 2023-10-08T05:52:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.151+0000 7fe3cd7fa640 1 -- 172.21.15.70:0/4133631916 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe3c4007410 con 0x7fe3d00adc70 2023-10-08T05:52:23.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.152+0000 7fe3cd7fa640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe3b406afc0 0x7fe3b406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.153+0000 7fe3cd7fa640 1 -- 172.21.15.70:0/4133631916 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fe3c4088b50 con 0x7fe3d00adc70 2023-10-08T05:52:23.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.153+0000 7fe3cffff640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe3b406afc0 0x7fe3b406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.154+0000 7fe3cffff640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe3b406afc0 0x7fe3b406d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc036000 tx=0x7fe3bc03c040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:23.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.157+0000 7fe3cd7fa640 1 -- 172.21.15.70:0/4133631916 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3c4053c30 con 0x7fe3d00adc70 2023-10-08T05:52:23.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.347+0000 7fe3d58df640 1 -- 172.21.15.70:0/4133631916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe3d004e160 con 0x7fe3d00adc70 2023-10-08T05:52:23.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.348+0000 7fe3cd7fa640 1 -- 172.21.15.70:0/4133631916 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe3c4017020 con 0x7fe3d00adc70 2023-10-08T05:52:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:23.350 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:52:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T05:52:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T05:52:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:23.351 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:52:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.351+0000 7fe3d58df640 1 -- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe3b406afc0 msgr2=0x7fe3b406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.351+0000 7fe3d58df640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe3b406afc0 0x7fe3b406d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc036000 tx=0x7fe3bc03c040 comp rx=0 tx=0).stop 2023-10-08T05:52:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.351+0000 7fe3d58df640 1 -- 172.21.15.70:0/4133631916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 msgr2=0x7fe3d01e5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.351+0000 7fe3d58df640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 0x7fe3d01e5ea0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400c450 tx=0x7fe3c400c910 comp rx=0 tx=0).stop 2023-10-08T05:52:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.351+0000 7fe3d58df640 1 -- 172.21.15.70:0/4133631916 shutdown_connections 2023-10-08T05:52:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.351+0000 7fe3d58df640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe3b406afc0 0x7fe3b406d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.351+0000 7fe3d58df640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3d015af10 0x7fe3d014e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.351+0000 7fe3d58df640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3d015ab30 0x7fe3d0159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.351+0000 7fe3d58df640 1 --2- 172.21.15.70:0/4133631916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3d00adc70 0x7fe3d01e5ea0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.351+0000 7fe3d58df640 1 -- 172.21.15.70:0/4133631916 >> 172.21.15.70:0/4133631916 conn(0x7fe3d00cb0c0 msgr2=0x7fe3d00b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:23.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.352+0000 7fe3d58df640 1 -- 172.21.15.70:0/4133631916 shutdown_connections 2023-10-08T05:52:23.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.352+0000 7fe3d58df640 1 -- 172.21.15.70:0/4133631916 wait complete. 2023-10-08T05:52:23.370 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T05:52:23.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T05:52:23.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.587+0000 7ff39570f640 1 Processor -- start 2023-10-08T05:52:23.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.587+0000 7ff39570f640 1 -- start start 2023-10-08T05:52:23.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.589+0000 7ff39570f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3900b1750 0x7ff39015d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.589+0000 7ff39570f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3900b30a0 con 0x7ff3900b40e0 2023-10-08T05:52:23.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.589+0000 7ff39570f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3900cb8b0 con 0x7ff3900702d0 2023-10-08T05:52:23.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.589+0000 7ff39570f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39015dcf0 con 0x7ff3900b1750 2023-10-08T05:52:23.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.589+0000 7ff3867fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff3900b40e0 0x7ff3900cb1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55112/0 (socket says 172.21.15.70:55112) 2023-10-08T05:52:23.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.589+0000 7ff3867fc640 1 -- 172.21.15.70:0/290952001 learned_addr learned my addr 172.21.15.70:0/290952001 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:23.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.589+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710495723 0 0) 0x7ff3900b30a0 con 0x7ff3900b40e0 2023-10-08T05:52:23.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.589+0000 7ff38effd640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3900b1750 0x7ff39015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.589+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff37c003520 con 0x7ff3900b40e0 2023-10-08T05:52:23.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3051490830 0 0) 0x7ff37c003520 con 0x7ff3900b40e0 2023-10-08T05:52:23.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3900b1750 msgr2=0x7ff39015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:23.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3900b1750 0x7ff39015d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 >> v1:172.21.15.181:6789/0 conn(0x7ff3900702d0 legacy=0x7ff3900c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3900b3320 con 0x7ff3900b40e0 2023-10-08T05:52:23.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff378001670 con 0x7ff3900b40e0 2023-10-08T05:52:23.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 >> v1:172.21.15.70:6789/0 conn(0x7ff3900b40e0 legacy=0x7ff3900cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff37c0039a0 0x7ff37c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 0x7ff37c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3900b1750 0x7ff37c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39015dcf0 con 0x7ff37c0047c0 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c0054b0 con 0x7ff37c0039a0 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c005670 con 0x7ff3900b1750 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.590+0000 7ff3867fc640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 0x7ff37c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.591+0000 7ff38effd640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff37c0039a0 0x7ff37c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.591+0000 7ff38f7fe640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3900b1750 0x7ff37c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.591+0000 7ff3867fc640 1 -- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3900b1750 msgr2=0x7ff37c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:23.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.591+0000 7ff3867fc640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3900b1750 0x7ff37c00db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.591+0000 7ff3867fc640 1 -- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff37c0039a0 msgr2=0x7ff37c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:23.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.591+0000 7ff3867fc640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff37c0039a0 0x7ff37c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.591+0000 7ff3867fc640 1 -- 172.21.15.70:0/290952001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c003520 con 0x7ff37c0047c0 2023-10-08T05:52:23.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.591+0000 7ff38f7fe640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3900b1750 0x7ff37c00db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:23.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.591+0000 7ff38effd640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff37c0039a0 0x7ff37c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:23.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.592+0000 7ff3867fc640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 0x7ff37c004c40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c0e0 tx=0x7ff37c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e688fa2b3ba01009 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:23.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.592+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378005ad0 con 0x7ff37c0047c0 2023-10-08T05:52:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.592+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff378005c30 con 0x7ff37c0047c0 2023-10-08T05:52:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.592+0000 7ff38e7fc640 1 -- 172.21.15.70:0/290952001 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37802c810 con 0x7ff37c0047c0 2023-10-08T05:52:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.593+0000 7ff39570f640 1 -- 172.21.15.70:0/290952001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 msgr2=0x7ff37c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.593+0000 7ff39570f640 1 --2- 172.21.15.70:0/290952001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 0x7ff37c004c40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c0e0 tx=0x7ff37c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.593+0000 7ff38f7fe640 1 -- 172.21.15.70:0/290952001 reap_dead start 2023-10-08T05:52:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.593+0000 7ff39570f640 1 -- 172.21.15.70:0/290952001 shutdown_connections 2023-10-08T05:52:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.593+0000 7ff39570f640 1 -- 172.21.15.70:0/290952001 >> 172.21.15.70:0/290952001 conn(0x7ff390069d50 msgr2=0x7ff39006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.593+0000 7ff39570f640 1 -- 172.21.15.70:0/290952001 shutdown_connections 2023-10-08T05:52:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.593+0000 7ff39570f640 1 -- 172.21.15.70:0/290952001 wait complete. 2023-10-08T05:52:23.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.594+0000 7ff39570f640 1 Processor -- start 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.595+0000 7ff39570f640 1 -- start start 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.595+0000 7ff39570f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 0x7ff3901ef710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.595+0000 7ff39570f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff390158960 0x7ff390156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.595+0000 7ff39570f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff390158de0 0x7ff390154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.595+0000 7ff39570f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3901efdf0 con 0x7ff37c0047c0 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff39570f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3901f00d0 con 0x7ff390158960 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff39570f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3901f03b0 con 0x7ff390158de0 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff38effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 0x7ff3901ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff38effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 0x7ff3901ef710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48808/0 (socket says 172.21.15.70:48808) 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff38effd640 1 -- 172.21.15.70:0/2423384856 learned_addr learned my addr 172.21.15.70:0/2423384856 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff3867fc640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff390158960 0x7ff390156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff38effd640 1 -- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff390158de0 msgr2=0x7ff390154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff38effd640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff390158de0 0x7ff390154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff38effd640 1 -- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff390158960 msgr2=0x7ff390156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff38effd640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff390158960 0x7ff390156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff38effd640 1 -- 172.21.15.70:0/2423384856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3901f0690 con 0x7ff37c0047c0 2023-10-08T05:52:23.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.596+0000 7ff38effd640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 0x7ff3901ef710 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff380009fc0 tx=0x7ff38000f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.597+0000 7ff387fff640 1 -- 172.21.15.70:0/2423384856 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3800145f0 con 0x7ff37c0047c0 2023-10-08T05:52:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.597+0000 7ff387fff640 1 -- 172.21.15.70:0/2423384856 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff380014750 con 0x7ff37c0047c0 2023-10-08T05:52:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.597+0000 7ff387fff640 1 -- 172.21.15.70:0/2423384856 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380014b00 con 0x7ff37c0047c0 2023-10-08T05:52:23.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.597+0000 7ff39570f640 1 -- 172.21.15.70:0/2423384856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3901f0970 con 0x7ff37c0047c0 2023-10-08T05:52:23.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.597+0000 7ff39570f640 1 -- 172.21.15.70:0/2423384856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3901570b0 con 0x7ff37c0047c0 2023-10-08T05:52:23.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.598+0000 7ff39570f640 1 -- 172.21.15.70:0/2423384856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff39004e160 con 0x7ff37c0047c0 2023-10-08T05:52:23.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.599+0000 7ff387fff640 1 -- 172.21.15.70:0/2423384856 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff380014c60 con 0x7ff37c0047c0 2023-10-08T05:52:23.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.600+0000 7ff387fff640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff37006af90 0x7ff37006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:23.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.600+0000 7ff387fff640 1 -- 172.21.15.70:0/2423384856 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5221+0+0 (secure 0 0 0) 0x7ff38008a5c0 con 0x7ff37c0047c0 2023-10-08T05:52:23.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.601+0000 7ff3867fc640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff37006af90 0x7ff37006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:23.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.602+0000 7ff3867fc640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff37006af90 0x7ff37006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff378034000 tx=0x7ff378009030 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:23.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.604+0000 7ff387fff640 1 -- 172.21.15.70:0/2423384856 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff380055590 con 0x7ff37c0047c0 2023-10-08T05:52:23.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:23.791+0000 7ff39570f640 1 -- 172.21.15.70:0/2423384856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff3900710b0 con 0x7ff37c0047c0 2023-10-08T05:52:24.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.562+0000 7ff387fff640 1 -- 172.21.15.70:0/2423384856 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff380058bb0 con 0x7ff37c0047c0 2023-10-08T05:52:24.564 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T05:52:24.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.567+0000 7ff39570f640 1 -- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff37006af90 msgr2=0x7ff37006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:24.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.567+0000 7ff39570f640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff37006af90 0x7ff37006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff378034000 tx=0x7ff378009030 comp rx=0 tx=0).stop 2023-10-08T05:52:24.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.567+0000 7ff39570f640 1 -- 172.21.15.70:0/2423384856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 msgr2=0x7ff3901ef710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:24.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.567+0000 7ff39570f640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 0x7ff3901ef710 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff380009fc0 tx=0x7ff38000f5e0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.567+0000 7ff39570f640 1 -- 172.21.15.70:0/2423384856 shutdown_connections 2023-10-08T05:52:24.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.567+0000 7ff39570f640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff37006af90 0x7ff37006d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.567+0000 7ff39570f640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff390158de0 0x7ff390154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.567+0000 7ff39570f640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff390158960 0x7ff390156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.567+0000 7ff39570f640 1 --2- 172.21.15.70:0/2423384856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff37c0047c0 0x7ff3901ef710 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.567+0000 7ff39570f640 1 -- 172.21.15.70:0/2423384856 >> 172.21.15.70:0/2423384856 conn(0x7ff390069d50 msgr2=0x7ff390154160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:24.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.569+0000 7ff39570f640 1 -- 172.21.15.70:0/2423384856 shutdown_connections 2023-10-08T05:52:24.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.569+0000 7ff39570f640 1 -- 172.21.15.70:0/2423384856 wait complete. 2023-10-08T05:52:24.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T05:52:24.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.803+0000 7f5699c21640 1 Processor -- start 2023-10-08T05:52:24.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.804+0000 7f5699c21640 1 -- start start 2023-10-08T05:52:24.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.805+0000 7f5699c21640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f569406e410 0x7f56940b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:24.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.805+0000 7f5699c21640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569406d6a0 con 0x7f56940cb5f0 2023-10-08T05:52:24.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.805+0000 7f5699c21640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56940caad0 con 0x7f56941594e0 2023-10-08T05:52:24.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.805+0000 7f5699c21640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56940cadb0 con 0x7f569406e410 2023-10-08T05:52:24.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.806+0000 7f5693fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f56940cb5f0 0x7f569415cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55122/0 (socket says 172.21.15.70:55122) 2023-10-08T05:52:24.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.806+0000 7f5693fff640 1 -- 172.21.15.70:0/2825727015 learned_addr learned my addr 172.21.15.70:0/2825727015 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:24.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.806+0000 7f56937fe640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f569406e410 0x7f56940b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:24.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.806+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376671062 0 0) 0x7f569406d6a0 con 0x7f56940cb5f0 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.806+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5680003520 con 0x7f56940cb5f0 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118351472 0 0) 0x7f56940caad0 con 0x7f56941594e0 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f569406d6a0 con 0x7f56941594e0 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3739114127 0 0) 0x7f5680003520 con 0x7f56940cb5f0 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f569406e410 msgr2=0x7f56940b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f569406e410 0x7f56940b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 >> v1:172.21.15.181:6789/0 conn(0x7f56941594e0 legacy=0x7f56941569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5694159940 con 0x7f56940cb5f0 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f568c0019f0 con 0x7f56940cb5f0 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 >> v1:172.21.15.70:6789/0 conn(0x7f56940cb5f0 legacy=0x7f569415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:24.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f569406e410 0x7f5680003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5680004090 0x7f56800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 0x7f5680005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56940cadb0 con 0x7f5680004d00 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568000dc70 con 0x7f5680004090 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568000de30 con 0x7f569406e410 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.807+0000 7f5693fff640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 0x7f5680005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.808+0000 7f56937fe640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f569406e410 0x7f5680003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.808+0000 7f5693fff640 1 -- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f569406e410 msgr2=0x7f5680003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.808+0000 7f5693fff640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f569406e410 0x7f5680003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.808+0000 7f568bfff640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5680004090 0x7f56800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:24.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.808+0000 7f5693fff640 1 -- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5680004090 msgr2=0x7f56800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:24.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.808+0000 7f5693fff640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5680004090 0x7f56800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.808+0000 7f5693fff640 1 -- 172.21.15.70:0/2825727015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5680003520 con 0x7f5680004d00 2023-10-08T05:52:24.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.808+0000 7f568bfff640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5680004090 0x7f56800047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:24.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.809+0000 7f5693fff640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 0x7f5680005180 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f568c00a7c0 tx=0x7f56800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e10edfe3ab022b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:24.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.809+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c034630 con 0x7f5680004d00 2023-10-08T05:52:24.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.809+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c02f070 con 0x7f5680004d00 2023-10-08T05:52:24.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.809+0000 7f5692ffd640 1 -- 172.21.15.70:0/2825727015 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c034a30 con 0x7f5680004d00 2023-10-08T05:52:24.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.810+0000 7f5699c21640 1 -- 172.21.15.70:0/2825727015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 msgr2=0x7f5680005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:24.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.810+0000 7f5699c21640 1 --2- 172.21.15.70:0/2825727015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 0x7f5680005180 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f568c00a7c0 tx=0x7f56800057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.810+0000 7f5693fff640 1 -- 172.21.15.70:0/2825727015 reap_dead start 2023-10-08T05:52:24.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.810+0000 7f5699c21640 1 -- 172.21.15.70:0/2825727015 shutdown_connections 2023-10-08T05:52:24.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.810+0000 7f5699c21640 1 -- 172.21.15.70:0/2825727015 >> 172.21.15.70:0/2825727015 conn(0x7f569406a140 msgr2=0x7f569406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:24.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.810+0000 7f5699c21640 1 -- 172.21.15.70:0/2825727015 shutdown_connections 2023-10-08T05:52:24.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.810+0000 7f5699c21640 1 -- 172.21.15.70:0/2825727015 wait complete. 2023-10-08T05:52:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.812+0000 7f5699c21640 1 Processor -- start 2023-10-08T05:52:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.812+0000 7f5699c21640 1 -- start start 2023-10-08T05:52:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.813+0000 7f5699c21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 0x7f5694147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.813+0000 7f5699c21640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56941e3760 0x7f56941e3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.813+0000 7f5699c21640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569415b6a0 0x7f569415bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.813+0000 7f5699c21640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569415c060 con 0x7f5680004d00 2023-10-08T05:52:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.813+0000 7f5699c21640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56940c2710 con 0x7f569415b6a0 2023-10-08T05:52:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.813+0000 7f5699c21640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56940c29f0 con 0x7f56941e3760 2023-10-08T05:52:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.813+0000 7f56937fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 0x7f5694147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.813+0000 7f56937fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 0x7f5694147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48818/0 (socket says 172.21.15.70:48818) 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.813+0000 7f56937fe640 1 -- 172.21.15.70:0/580260094 learned_addr learned my addr 172.21.15.70:0/580260094 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f568bfff640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56941e3760 0x7f56941e3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f5693fff640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569415b6a0 0x7f569415bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f56937fe640 1 -- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56941e3760 msgr2=0x7f56941e3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f56937fe640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56941e3760 0x7f56941e3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f56937fe640 1 -- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569415b6a0 msgr2=0x7f569415bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f56937fe640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569415b6a0 0x7f569415bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f56937fe640 1 -- 172.21.15.70:0/580260094 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56940c2cd0 con 0x7f5680004d00 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f5693fff640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569415b6a0 0x7f569415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f568bfff640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56941e3760 0x7f56941e3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:24.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f56937fe640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 0x7f5694147c30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5684011600 tx=0x7f5684011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:24.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f5690ff9640 1 -- 172.21.15.70:0/580260094 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400f920 con 0x7f5680004d00 2023-10-08T05:52:24.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f5690ff9640 1 -- 172.21.15.70:0/580260094 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568400fa80 con 0x7f5680004d00 2023-10-08T05:52:24.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f5699c21640 1 -- 172.21.15.70:0/580260094 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56940c2fb0 con 0x7f5680004d00 2023-10-08T05:52:24.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.814+0000 7f5690ff9640 1 -- 172.21.15.70:0/580260094 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400fdf0 con 0x7f5680004d00 2023-10-08T05:52:24.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.815+0000 7f5699c21640 1 -- 172.21.15.70:0/580260094 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5694148170 con 0x7f5680004d00 2023-10-08T05:52:24.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.816+0000 7f5699c21640 1 -- 172.21.15.70:0/580260094 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56940667f0 con 0x7f5680004d00 2023-10-08T05:52:24.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.817+0000 7f5690ff9640 1 -- 172.21.15.70:0/580260094 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5684015460 con 0x7f5680004d00 2023-10-08T05:52:24.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.818+0000 7f5690ff9640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f567406af90 0x7f567406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:24.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.818+0000 7f5690ff9640 1 -- 172.21.15.70:0/580260094 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5591+0+0 (secure 0 0 0) 0x7f5684089c30 con 0x7f5680004d00 2023-10-08T05:52:24.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.818+0000 7f568bfff640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f567406af90 0x7f567406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.819+0000 7f568bfff640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f567406af90 0x7f567406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f567c000f60 tx=0x7f567c009040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:24.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:24.822+0000 7f5690ff9640 1 -- 172.21.15.70:0/580260094 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5684054c40 con 0x7f5680004d00 2023-10-08T05:52:25.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.014+0000 7f5699c21640 1 -- 172.21.15.70:0/580260094 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5694071cb0 con 0x7f5680004d00 2023-10-08T05:52:25.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.580+0000 7f5690ff9640 1 -- 172.21.15.70:0/580260094 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7f56840220d0 con 0x7f5680004d00 2023-10-08T05:52:25.582 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T05:52:25.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 -- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f567406af90 msgr2=0x7f567406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:25.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f567406af90 0x7f567406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f567c000f60 tx=0x7f567c009040 comp rx=0 tx=0).stop 2023-10-08T05:52:25.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 -- 172.21.15.70:0/580260094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 msgr2=0x7f5694147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:25.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 0x7f5694147c30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5684011600 tx=0x7f5684011ac0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 -- 172.21.15.70:0/580260094 shutdown_connections 2023-10-08T05:52:25.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f567406af90 0x7f567406d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f569415b6a0 0x7f569415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56941e3760 0x7f56941e3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 --2- 172.21.15.70:0/580260094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5680004d00 0x7f5694147c30 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 -- 172.21.15.70:0/580260094 >> 172.21.15.70:0/580260094 conn(0x7f569406a140 msgr2=0x7f5694155a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:25.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 -- 172.21.15.70:0/580260094 shutdown_connections 2023-10-08T05:52:25.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.584+0000 7f5699c21640 1 -- 172.21.15.70:0/580260094 wait complete. 2023-10-08T05:52:25.618 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T05:52:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.806+0000 7f91d7410640 1 Processor -- start 2023-10-08T05:52:25.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.806+0000 7f91d7410640 1 -- start start 2023-10-08T05:52:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.807+0000 7f91d7410640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91d00afbe0 0x7f91d00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.807+0000 7f91d7410640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91d006dfc0 0x7f91d0070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.807+0000 7f91d7410640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d00b2620 con 0x7f91d0071070 2023-10-08T05:52:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.807+0000 7f91d7410640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d00b2060 con 0x7f91d006dfc0 2023-10-08T05:52:25.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.807+0000 7f91d7410640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d00b2340 con 0x7f91d00afbe0 2023-10-08T05:52:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.808+0000 7f91d49ab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91d006dfc0 0x7f91d0070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.808+0000 7f91d59ad640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f91d0071070 0x7f91d00cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55136/0 (socket says 172.21.15.70:55136) 2023-10-08T05:52:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.808+0000 7f91d59ad640 1 -- 172.21.15.70:0/3704838780 learned_addr learned my addr 172.21.15.70:0/3704838780 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.808+0000 7f91d51ac640 1 --2- 172.21.15.70:0/3704838780 >> v2:172.21.15.181:3300/0 conn(0x7f91d00afbe0 0x7f91d00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:25.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.808+0000 7f91d49ab640 1 -- 172.21.15.70:0/3704838780 >> v2:172.21.15.181:3300/0 conn(0x7f91d00afbe0 msgr2=0x7f91d00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:25.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.808+0000 7f91d49ab640 1 --2- 172.21.15.70:0/3704838780 >> v2:172.21.15.181:3300/0 conn(0x7f91d00afbe0 0x7f91d00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.808+0000 7f91d49ab640 1 -- 172.21.15.70:0/3704838780 >> v1:172.21.15.70:6789/0 conn(0x7f91d0071070 legacy=0x7f91d00cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:25.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.808+0000 7f91c7fff640 1 -- 172.21.15.70:0/3704838780 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940824260 0 0) 0x7f91d00b2620 con 0x7f91d0071070 2023-10-08T05:52:25.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.808+0000 7f91d49ab640 1 -- 172.21.15.70:0/3704838780 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d0070350 con 0x7f91d006dfc0 2023-10-08T05:52:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.810+0000 7f91d49ab640 1 --2- 172.21.15.70:0/3704838780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d006dfc0 0x7f91d0070b30 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f91d006e640 tx=0x7f91c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47a5b67a8281eea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.810+0000 7f91c7fff640 1 -- 172.21.15.70:0/3704838780 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0031830 con 0x7f91d006dfc0 2023-10-08T05:52:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.810+0000 7f91c7fff640 1 -- 172.21.15.70:0/3704838780 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c0031990 con 0x7f91d006dfc0 2023-10-08T05:52:25.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.810+0000 7f91c7fff640 1 -- 172.21.15.70:0/3704838780 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0031d20 con 0x7f91d006dfc0 2023-10-08T05:52:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.810+0000 7f91d7410640 1 -- 172.21.15.70:0/3704838780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d006dfc0 msgr2=0x7f91d0070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.810+0000 7f91d7410640 1 --2- 172.21.15.70:0/3704838780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d006dfc0 0x7f91d0070b30 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f91d006e640 tx=0x7f91c002b8d0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.810+0000 7f91d7410640 1 -- 172.21.15.70:0/3704838780 shutdown_connections 2023-10-08T05:52:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.810+0000 7f91d7410640 1 --2- 172.21.15.70:0/3704838780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d006dfc0 0x7f91d0070b30 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.810+0000 7f91d7410640 1 --2- 172.21.15.70:0/3704838780 >> v2:172.21.15.181:3300/0 conn(0x7f91d00afbe0 0x7f91d00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.810+0000 7f91d7410640 1 -- 172.21.15.70:0/3704838780 >> 172.21.15.70:0/3704838780 conn(0x7f91d006a5d0 msgr2=0x7f91d00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.811+0000 7f91d7410640 1 -- 172.21.15.70:0/3704838780 shutdown_connections 2023-10-08T05:52:25.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.811+0000 7f91d7410640 1 -- 172.21.15.70:0/3704838780 wait complete. 2023-10-08T05:52:25.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.812+0000 7f91d7410640 1 Processor -- start 2023-10-08T05:52:25.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.812+0000 7f91d7410640 1 -- start start 2023-10-08T05:52:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.813+0000 7f91d7410640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d006dfc0 0x7f91d00c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.813+0000 7f91d7410640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d0071070 0x7f91d00c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.813+0000 7f91d7410640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d00afbe0 0x7f91d00c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.813+0000 7f91d7410640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00ce140 con 0x7f91d00afbe0 2023-10-08T05:52:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.813+0000 7f91d7410640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00ce3b0 con 0x7f91d006dfc0 2023-10-08T05:52:25.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.813+0000 7f91d7410640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00ce690 con 0x7f91d0071070 2023-10-08T05:52:25.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.814+0000 7f91d51ac640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d006dfc0 0x7f91d00c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.814+0000 7f91d59ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d00afbe0 0x7f91d00c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.814+0000 7f91d49ab640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d0071070 0x7f91d00c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.814+0000 7f91d51ac640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d006dfc0 0x7f91d00c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:34012/0 (socket says 172.21.15.70:34012) 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.814+0000 7f91d51ac640 1 -- 172.21.15.70:0/1604175494 learned_addr learned my addr 172.21.15.70:0/1604175494 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91d59ad640 1 -- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d0071070 msgr2=0x7f91d00c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91d59ad640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d0071070 0x7f91d00c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91d59ad640 1 -- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d006dfc0 msgr2=0x7f91d00c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91d59ad640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d006dfc0 0x7f91d00c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91d59ad640 1 -- 172.21.15.70:0/1604175494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c0009400 con 0x7f91d00afbe0 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91d59ad640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d00afbe0 0x7f91d00c9550 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f91d015b4e0 tx=0x7f91c800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91c5ffb640 1 -- 172.21.15.70:0/1604175494 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8004d60 con 0x7f91d00afbe0 2023-10-08T05:52:25.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91c5ffb640 1 -- 172.21.15.70:0/1604175494 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c800a610 con 0x7f91d00afbe0 2023-10-08T05:52:25.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91d7410640 1 -- 172.21.15.70:0/1604175494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d00ce970 con 0x7f91d00afbe0 2023-10-08T05:52:25.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91c5ffb640 1 -- 172.21.15.70:0/1604175494 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800a9c0 con 0x7f91d00afbe0 2023-10-08T05:52:25.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.815+0000 7f91d7410640 1 -- 172.21.15.70:0/1604175494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d00c0c80 con 0x7f91d00afbe0 2023-10-08T05:52:25.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.816+0000 7f91d7410640 1 -- 172.21.15.70:0/1604175494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d00cee90 con 0x7f91d00afbe0 2023-10-08T05:52:25.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.818+0000 7f91c5ffb640 1 -- 172.21.15.70:0/1604175494 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f91c800ab20 con 0x7f91d00afbe0 2023-10-08T05:52:25.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.819+0000 7f91c5ffb640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f91b406af90 0x7f91b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:25.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.819+0000 7f91c5ffb640 1 -- 172.21.15.70:0/1604175494 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5973+0+0 (secure 0 0 0) 0x7f91c8088cd0 con 0x7f91d00afbe0 2023-10-08T05:52:25.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.820+0000 7f91d51ac640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f91b406af90 0x7f91b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:25.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.821+0000 7f91d51ac640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f91b406af90 0x7f91b406d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f91cc006fd0 tx=0x7f91cc008040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:25.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:25.824+0000 7f91c5ffb640 1 -- 172.21.15.70:0/1604175494 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91d00cee90 con 0x7f91d00afbe0 2023-10-08T05:52:26.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.050+0000 7f91d7410640 1 -- 172.21.15.70:0/1604175494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f91d00cee90 con 0x7f91d00afbe0 2023-10-08T05:52:26.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.602+0000 7f91c5ffb640 1 -- 172.21.15.70:0/1604175494 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v30) v1 ==== 173+0+0 (secure 0 0 0) 0x7f91d00cee90 con 0x7f91d00afbe0 2023-10-08T05:52:26.604 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 9 and data pool 10 2023-10-08T05:52:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.610+0000 7f91d7410640 1 -- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f91b406af90 msgr2=0x7f91b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.610+0000 7f91d7410640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f91b406af90 0x7f91b406d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f91cc006fd0 tx=0x7f91cc008040 comp rx=0 tx=0).stop 2023-10-08T05:52:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.610+0000 7f91d7410640 1 -- 172.21.15.70:0/1604175494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d00afbe0 msgr2=0x7f91d00c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.611+0000 7f91d7410640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d00afbe0 0x7f91d00c9550 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f91d015b4e0 tx=0x7f91c800cb60 comp rx=0 tx=0).stop 2023-10-08T05:52:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.612+0000 7f91d7410640 1 -- 172.21.15.70:0/1604175494 shutdown_connections 2023-10-08T05:52:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.612+0000 7f91d7410640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f91b406af90 0x7f91b406d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.612+0000 7f91d7410640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d00afbe0 0x7f91d00c9550 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.612+0000 7f91d7410640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91d0071070 0x7f91d00c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.612+0000 7f91d7410640 1 --2- 172.21.15.70:0/1604175494 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91d006dfc0 0x7f91d00c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.612+0000 7f91d7410640 1 -- 172.21.15.70:0/1604175494 >> 172.21.15.70:0/1604175494 conn(0x7f91d006a5d0 msgr2=0x7f91d00c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:26.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.613+0000 7f91d7410640 1 -- 172.21.15.70:0/1604175494 shutdown_connections 2023-10-08T05:52:26.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.613+0000 7f91d7410640 1 -- 172.21.15.70:0/1604175494 wait complete. 2023-10-08T05:52:26.643 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:26.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.825+0000 7f50b39b6640 1 Processor -- start 2023-10-08T05:52:26.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.825+0000 7f50b39b6640 1 -- start start 2023-10-08T05:52:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.827+0000 7f50b39b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac0cbfe0 0x7f50ac15dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.827+0000 7f50b39b6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ac06a730 con 0x7f50ac065eb0 2023-10-08T05:52:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.827+0000 7f50b39b6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ac15e2b0 con 0x7f50ac070b40 2023-10-08T05:52:26.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.827+0000 7f50b39b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0cd7c0 con 0x7f50ac0cbfe0 2023-10-08T05:52:26.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.827+0000 7f50b0f51640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f50ac065eb0 0x7f50ac15e6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55148/0 (socket says 172.21.15.70:55148) 2023-10-08T05:52:26.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.827+0000 7f50b0f51640 1 -- 172.21.15.70:0/2927902823 learned_addr learned my addr 172.21.15.70:0/2927902823 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:26.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50b1752640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac0cbfe0 0x7f50ac15dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473917297 0 0) 0x7f50ac06a730 con 0x7f50ac065eb0 2023-10-08T05:52:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5098003520 con 0x7f50ac065eb0 2023-10-08T05:52:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 864992227 0 0) 0x7f5098003520 con 0x7f50ac065eb0 2023-10-08T05:52:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac0cbfe0 msgr2=0x7f50ac15dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac0cbfe0 0x7f50ac15dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 >> v1:172.21.15.181:6789/0 conn(0x7f50ac070b40 legacy=0x7f50ac156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac0cdaa0 con 0x7f50ac065eb0 2023-10-08T05:52:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f509c001670 con 0x7f50ac065eb0 2023-10-08T05:52:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 >> v1:172.21.15.70:6789/0 conn(0x7f50ac065eb0 legacy=0x7f50ac15e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:26.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980039a0 0x7f5098003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac0cbfe0 0x7f50980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5098004d00 0x7f5098005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0cd7c0 con 0x7f5098004d00 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509800ba30 con 0x7f50980039a0 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509800bbf0 con 0x7f50ac0cbfe0 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.828+0000 7f50b1f53640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5098004d00 0x7f5098005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.829+0000 7f50b0f51640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac0cbfe0 0x7f50980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.829+0000 7f50b1752640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980039a0 0x7f5098003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.829+0000 7f50b1f53640 1 -- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac0cbfe0 msgr2=0x7f50980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.829+0000 7f50b1f53640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac0cbfe0 0x7f50980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:26.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.829+0000 7f50b1f53640 1 -- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980039a0 msgr2=0x7f5098003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:26.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.829+0000 7f50b1f53640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f50980039a0 0x7f5098003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:26.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.829+0000 7f50b1f53640 1 -- 172.21.15.70:0/2927902823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098003520 con 0x7f5098004d00 2023-10-08T05:52:26.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.830+0000 7f50b1f53640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5098004d00 0x7f5098005180 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f50a4001e20 tx=0x7f50980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e76f0fe0bf1c13 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:26.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.830+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4003b70 con 0x7f5098004d00 2023-10-08T05:52:26.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.830+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a4003cd0 con 0x7f5098004d00 2023-10-08T05:52:26.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.830+0000 7f50a3fff640 1 -- 172.21.15.70:0/2927902823 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4005310 con 0x7f5098004d00 2023-10-08T05:52:26.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.831+0000 7f50b39b6640 1 -- 172.21.15.70:0/2927902823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5098004d00 msgr2=0x7f5098005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:26.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.831+0000 7f50b39b6640 1 --2- 172.21.15.70:0/2927902823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5098004d00 0x7f5098005180 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f50a4001e20 tx=0x7f50980057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:26.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.831+0000 7f50b1f53640 1 -- 172.21.15.70:0/2927902823 reap_dead start 2023-10-08T05:52:26.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.831+0000 7f50b39b6640 1 -- 172.21.15.70:0/2927902823 shutdown_connections 2023-10-08T05:52:26.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.831+0000 7f50b39b6640 1 -- 172.21.15.70:0/2927902823 >> 172.21.15.70:0/2927902823 conn(0x7f50ac0b1480 msgr2=0x7f50ac06cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:26.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.831+0000 7f50b39b6640 1 -- 172.21.15.70:0/2927902823 shutdown_connections 2023-10-08T05:52:26.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.831+0000 7f50b39b6640 1 -- 172.21.15.70:0/2927902823 wait complete. 2023-10-08T05:52:26.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.832+0000 7f50b39b6640 1 Processor -- start 2023-10-08T05:52:26.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.833+0000 7f50b39b6640 1 -- start start 2023-10-08T05:52:26.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b39b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5098004d00 0x7f50ac1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:26.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b39b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac1613e0 0x7f50ac0c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:26.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b39b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0b3040 0x7f50ac0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:26.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b39b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0b3a10 con 0x7f50ac0b3040 2023-10-08T05:52:26.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b39b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1f39d0 con 0x7f5098004d00 2023-10-08T05:52:26.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b39b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac156b00 con 0x7f50ac1613e0 2023-10-08T05:52:26.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b1f53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0b3040 0x7f50ac0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:26.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b1f53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0b3040 0x7f50ac0b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48844/0 (socket says 172.21.15.70:48844) 2023-10-08T05:52:26.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b1f53640 1 -- 172.21.15.70:0/1849839152 learned_addr learned my addr 172.21.15.70:0/1849839152 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b1f53640 1 -- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac1613e0 msgr2=0x7f50ac0c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b1f53640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac1613e0 0x7f50ac0c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b1f53640 1 -- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5098004d00 msgr2=0x7f50ac1565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b1f53640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5098004d00 0x7f50ac1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.834+0000 7f50b1f53640 1 -- 172.21.15.70:0/1849839152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac0c7060 con 0x7f50ac0b3040 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.835+0000 7f50b1f53640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0b3040 0x7f50ac0b34d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f50a402c040 tx=0x7f50a40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.835+0000 7f50a1ffb640 1 -- 172.21.15.70:0/1849839152 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4006d50 con 0x7f50ac0b3040 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.835+0000 7f50a1ffb640 1 -- 172.21.15.70:0/1849839152 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a4007db0 con 0x7f50ac0b3040 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.835+0000 7f50a1ffb640 1 -- 172.21.15.70:0/1849839152 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4007270 con 0x7f50ac0b3040 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.835+0000 7f50b39b6640 1 -- 172.21.15.70:0/1849839152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac0c6aa0 con 0x7f50ac0b3040 2023-10-08T05:52:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.835+0000 7f50b39b6640 1 -- 172.21.15.70:0/1849839152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac0c77d0 con 0x7f50ac0b3040 2023-10-08T05:52:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.836+0000 7f50b39b6640 1 -- 172.21.15.70:0/1849839152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac067490 con 0x7f50ac0b3040 2023-10-08T05:52:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.837+0000 7f50a1ffb640 1 -- 172.21.15.70:0/1849839152 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f50a40073d0 con 0x7f50ac0b3040 2023-10-08T05:52:26.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.838+0000 7f50a1ffb640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f509006af90 0x7f509006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:26.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.838+0000 7f50a1ffb640 1 -- 172.21.15.70:0/1849839152 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f50a40ab090 con 0x7f50ac0b3040 2023-10-08T05:52:26.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.839+0000 7f50b1752640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f509006af90 0x7f509006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:26.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.840+0000 7f50b1752640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f509006af90 0x7f509006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009950 tx=0x7f50a8008040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:26.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:26.843+0000 7f50a1ffb640 1 -- 172.21.15.70:0/1849839152 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50a4075e20 con 0x7f50ac0b3040 2023-10-08T05:52:27.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.028+0000 7f50b39b6640 1 -- 172.21.15.70:0/1849839152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50ac0c9500 con 0x7f50ac0b3040 2023-10-08T05:52:27.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.030+0000 7f50a1ffb640 1 -- 172.21.15.70:0/1849839152 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16787 (secure 0 0 0) 0x7f50a4079440 con 0x7f50ac0b3040 2023-10-08T05:52:27.032 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:27.032 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:26.052955+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:52:23.846215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-08T05:52:25.069536+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:27.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 -- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f509006af90 msgr2=0x7f509006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f509006af90 0x7f509006d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009950 tx=0x7f50a8008040 comp rx=0 tx=0).stop 2023-10-08T05:52:27.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 -- 172.21.15.70:0/1849839152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0b3040 msgr2=0x7f50ac0b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0b3040 0x7f50ac0b34d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f50a402c040 tx=0x7f50a40014a0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 -- 172.21.15.70:0/1849839152 shutdown_connections 2023-10-08T05:52:27.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f509006af90 0x7f509006d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f50ac0b3040 0x7f50ac0b34d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f50ac1613e0 0x7f50ac0c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 --2- 172.21.15.70:0/1849839152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5098004d00 0x7f50ac1565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 -- 172.21.15.70:0/1849839152 >> 172.21.15.70:0/1849839152 conn(0x7f50ac0b1480 msgr2=0x7f50ac159390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:27.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 -- 172.21.15.70:0/1849839152 shutdown_connections 2023-10-08T05:52:27.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.033+0000 7f50b39b6640 1 -- 172.21.15.70:0/1849839152 wait complete. 2023-10-08T05:52:27.053 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:27.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.263+0000 7f49fd33c640 1 Processor -- start 2023-10-08T05:52:27.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.264+0000 7f49fd33c640 1 -- start start 2023-10-08T05:52:27.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.265+0000 7f49fd33c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49f80ad790 0x7f49f80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.265+0000 7f49fd33c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f49f815e330 0x7f49f8070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.265+0000 7f49fd33c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f806dee0 con 0x7f49f8159ac0 2023-10-08T05:52:27.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.265+0000 7f49fd33c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f806dcf0 con 0x7f49f815e330 2023-10-08T05:52:27.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.265+0000 7f49fd33c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f815d850 con 0x7f49f80ad790 2023-10-08T05:52:27.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.266+0000 7f49f77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f49f815e330 0x7f49f8070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.266+0000 7f49f77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f49f815e330 0x7f49f8070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48854/0 (socket says 172.21.15.70:48854) 2023-10-08T05:52:27.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.266+0000 7f49f77fe640 1 -- 172.21.15.70:0/2726422115 learned_addr learned my addr 172.21.15.70:0/2726422115 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:27.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.266+0000 7f49f67fc640 1 --2- 172.21.15.70:0/2726422115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49f80ad790 0x7f49f80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.266+0000 7f49f77fe640 1 -- 172.21.15.70:0/2726422115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49f80ad790 msgr2=0x7f49f80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:27.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.266+0000 7f49f77fe640 1 --2- 172.21.15.70:0/2726422115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49f80ad790 0x7f49f80702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.266+0000 7f49f77fe640 1 -- 172.21.15.70:0/2726422115 >> v1:172.21.15.181:6789/0 conn(0x7f49f8159ac0 legacy=0x7f49f806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.266+0000 7f49f77fe640 1 -- 172.21.15.70:0/2726422115 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f815db30 con 0x7f49f815e330 2023-10-08T05:52:27.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.267+0000 7f49f77fe640 1 --2- 172.21.15.70:0/2726422115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f815e330 0x7f49f8070810 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009e50 tx=0x7f49f002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8ac2ab77ba65fa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:27.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49f5ffb640 1 -- 172.21.15.70:0/2726422115 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f002cb00 con 0x7f49f815e330 2023-10-08T05:52:27.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49f5ffb640 1 -- 172.21.15.70:0/2726422115 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f002cc60 con 0x7f49f815e330 2023-10-08T05:52:27.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49f5ffb640 1 -- 172.21.15.70:0/2726422115 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f00343f0 con 0x7f49f815e330 2023-10-08T05:52:27.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49fd33c640 1 -- 172.21.15.70:0/2726422115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f815e330 msgr2=0x7f49f8070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49fd33c640 1 --2- 172.21.15.70:0/2726422115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f815e330 0x7f49f8070810 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009e50 tx=0x7f49f002bf60 comp rx=0 tx=0).stop 2023-10-08T05:52:27.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49fd33c640 1 -- 172.21.15.70:0/2726422115 shutdown_connections 2023-10-08T05:52:27.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49fd33c640 1 --2- 172.21.15.70:0/2726422115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f815e330 0x7f49f8070810 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49fd33c640 1 --2- 172.21.15.70:0/2726422115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49f80ad790 0x7f49f80702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49fd33c640 1 -- 172.21.15.70:0/2726422115 >> 172.21.15.70:0/2726422115 conn(0x7f49f806a380 msgr2=0x7f49f80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:27.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49fd33c640 1 -- 172.21.15.70:0/2726422115 shutdown_connections 2023-10-08T05:52:27.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.268+0000 7f49fd33c640 1 -- 172.21.15.70:0/2726422115 wait complete. 2023-10-08T05:52:27.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.270+0000 7f49fd33c640 1 Processor -- start 2023-10-08T05:52:27.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.270+0000 7f49fd33c640 1 -- start start 2023-10-08T05:52:27.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.271+0000 7f49fd33c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49f80ad790 0x7f49f8149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.271+0000 7f49fd33c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f8159ac0 0x7f49f80aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.271+0000 7f49fd33c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f815e330 0x7f49f80af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.271+0000 7f49fd33c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f815ccb0 con 0x7f49f8159ac0 2023-10-08T05:52:27.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.271+0000 7f49fd33c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8149b40 con 0x7f49f815e330 2023-10-08T05:52:27.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.271+0000 7f49fd33c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f815be20 con 0x7f49f80ad790 2023-10-08T05:52:27.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f8159ac0 0x7f49f80aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f8159ac0 0x7f49f80aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48864/0 (socket says 172.21.15.70:48864) 2023-10-08T05:52:27.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f67fc640 1 -- 172.21.15.70:0/141228033 learned_addr learned my addr 172.21.15.70:0/141228033 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:27.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f6ffd640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49f80ad790 0x7f49f8149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f77fe640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f815e330 0x7f49f80af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f67fc640 1 -- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49f80ad790 msgr2=0x7f49f8149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f67fc640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49f80ad790 0x7f49f8149600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f67fc640 1 -- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f815e330 msgr2=0x7f49f80af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f67fc640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f815e330 0x7f49f80af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f67fc640 1 -- 172.21.15.70:0/141228033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f0009b30 con 0x7f49f8159ac0 2023-10-08T05:52:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.272+0000 7f49f67fc640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f8159ac0 0x7f49f80aeb20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f49e000d860 tx=0x7f49e000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.273+0000 7f49ef7fe640 1 -- 172.21.15.70:0/141228033 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e000de50 con 0x7f49f8159ac0 2023-10-08T05:52:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.273+0000 7f49ef7fe640 1 -- 172.21.15.70:0/141228033 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e000b8d0 con 0x7f49f8159ac0 2023-10-08T05:52:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.273+0000 7f49ef7fe640 1 -- 172.21.15.70:0/141228033 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e000bc80 con 0x7f49f8159ac0 2023-10-08T05:52:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.273+0000 7f49fd33c640 1 -- 172.21.15.70:0/141228033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f815c0a0 con 0x7f49f8159ac0 2023-10-08T05:52:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.273+0000 7f49fd33c640 1 -- 172.21.15.70:0/141228033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f815c660 con 0x7f49f8159ac0 2023-10-08T05:52:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.274+0000 7f49fd33c640 1 -- 172.21.15.70:0/141228033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f815ce60 con 0x7f49f8159ac0 2023-10-08T05:52:27.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.275+0000 7f49ef7fe640 1 -- 172.21.15.70:0/141228033 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f49e001e460 con 0x7f49f8159ac0 2023-10-08T05:52:27.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.276+0000 7f49ef7fe640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f49d806af90 0x7f49d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.276+0000 7f49ef7fe640 1 -- 172.21.15.70:0/141228033 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f49e0089ee0 con 0x7f49f8159ac0 2023-10-08T05:52:27.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.277+0000 7f49f6ffd640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f49d806af90 0x7f49d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.278+0000 7f49f6ffd640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f49d806af90 0x7f49d806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49e8000f60 tx=0x7f49e8009040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:27.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.281+0000 7f49ef7fe640 1 -- 172.21.15.70:0/141228033 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49f815ce60 con 0x7f49f8159ac0 2023-10-08T05:52:27.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.471+0000 7f49fd33c640 1 -- 172.21.15.70:0/141228033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49f815ce60 con 0x7f49f8159ac0 2023-10-08T05:52:27.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.474+0000 7f49ef7fe640 1 -- 172.21.15.70:0/141228033 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16787 (secure 0 0 0) 0x7f49f815ce60 con 0x7f49f8159ac0 2023-10-08T05:52:27.476 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:27.476 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":89,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:26.052955+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:52:23.846215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-08T05:52:25.069536+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:27.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 -- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f49d806af90 msgr2=0x7f49d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f49d806af90 0x7f49d806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49e8000f60 tx=0x7f49e8009040 comp rx=0 tx=0).stop 2023-10-08T05:52:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 -- 172.21.15.70:0/141228033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f8159ac0 msgr2=0x7f49f80aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f8159ac0 0x7f49f80aeb20 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f49e000d860 tx=0x7f49e000dd20 comp rx=0 tx=0).stop 2023-10-08T05:52:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 -- 172.21.15.70:0/141228033 shutdown_connections 2023-10-08T05:52:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f49d806af90 0x7f49d806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49f815e330 0x7f49f80af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49f8159ac0 0x7f49f80aeb20 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 --2- 172.21.15.70:0/141228033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49f80ad790 0x7f49f8149600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 -- 172.21.15.70:0/141228033 >> 172.21.15.70:0/141228033 conn(0x7f49f806a380 msgr2=0x7f49f80c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 -- 172.21.15.70:0/141228033 shutdown_connections 2023-10-08T05:52:27.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.477+0000 7f49fd33c640 1 -- 172.21.15.70:0/141228033 wait complete. 2023-10-08T05:52:27.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T05:52:27.548 INFO:tasks.ceph.mds.b.smithi181.stderr:2023-10-08T05:52:27.547+0000 7f9731f38640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:52:27.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.715+0000 7fdc91bce640 1 Processor -- start 2023-10-08T05:52:27.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.715+0000 7fdc91bce640 1 -- start start 2023-10-08T05:52:27.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.716+0000 7fdc91bce640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0cb130 0x7fdc8c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.717+0000 7fdc91bce640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c0b1cc0 con 0x7fdc8c06dfc0 2023-10-08T05:52:27.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.717+0000 7fdc91bce640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c0cbad0 con 0x7fdc8c0afbe0 2023-10-08T05:52:27.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.717+0000 7fdc91bce640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0b2620 con 0x7fdc8c0cb130 2023-10-08T05:52:27.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.717+0000 7fdc83fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdc8c06dfc0 0x7fdc8c070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55162/0 (socket says 172.21.15.70:55162) 2023-10-08T05:52:27.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.717+0000 7fdc83fff640 1 -- 172.21.15.70:0/3707085038 learned_addr learned my addr 172.21.15.70:0/3707085038 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:27.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.717+0000 7fdc8bfff640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0cb130 0x7fdc8c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.717+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215092264 0 0) 0x7fdc8c0b1cc0 con 0x7fdc8c06dfc0 2023-10-08T05:52:27.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.717+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc78003520 con 0x7fdc8c06dfc0 2023-10-08T05:52:27.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 613965872 0 0) 0x7fdc78003520 con 0x7fdc8c06dfc0 2023-10-08T05:52:27.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0cb130 msgr2=0x7fdc8c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:27.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0cb130 0x7fdc8c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 >> v1:172.21.15.181:6789/0 conn(0x7fdc8c0afbe0 legacy=0x7fdc8c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0b2060 con 0x7fdc8c06dfc0 2023-10-08T05:52:27.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fdc74001670 con 0x7fdc8c06dfc0 2023-10-08T05:52:27.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 >> v1:172.21.15.70:6789/0 conn(0x7fdc8c06dfc0 legacy=0x7fdc8c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 0x7fdc78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc780047c0 0x7fdc78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0cb130 0x7fdc78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0b2620 con 0x7fdc780039a0 2023-10-08T05:52:27.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7800ba90 con 0x7fdc780047c0 2023-10-08T05:52:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7800bc50 con 0x7fdc8c0cb130 2023-10-08T05:52:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.718+0000 7fdc8b7fe640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 0x7fdc78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.719+0000 7fdc8bfff640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0cb130 0x7fdc78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.719+0000 7fdc8b7fe640 1 -- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0cb130 msgr2=0x7fdc78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.719+0000 7fdc83fff640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc780047c0 0x7fdc78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.719+0000 7fdc8b7fe640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c0cb130 0x7fdc78005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.719+0000 7fdc8b7fe640 1 -- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc780047c0 msgr2=0x7fdc78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.719+0000 7fdc8b7fe640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc780047c0 0x7fdc78004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.719+0000 7fdc8b7fe640 1 -- 172.21.15.70:0/3707085038 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc78003520 con 0x7fdc780039a0 2023-10-08T05:52:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.720+0000 7fdc8b7fe640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 0x7fdc78003dc0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c001720 tx=0x7fdc780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ac6b1baca9c10b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.720+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0052d0 con 0x7fdc780039a0 2023-10-08T05:52:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.720+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c02e050 con 0x7fdc780039a0 2023-10-08T05:52:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.720+0000 7fdc8affd640 1 -- 172.21.15.70:0/3707085038 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0035f0 con 0x7fdc780039a0 2023-10-08T05:52:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.721+0000 7fdc91bce640 1 -- 172.21.15.70:0/3707085038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 msgr2=0x7fdc78003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.721+0000 7fdc91bce640 1 --2- 172.21.15.70:0/3707085038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 0x7fdc78003dc0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c001720 tx=0x7fdc780057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.721+0000 7fdc8bfff640 1 -- 172.21.15.70:0/3707085038 reap_dead start 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.721+0000 7fdc91bce640 1 -- 172.21.15.70:0/3707085038 shutdown_connections 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.721+0000 7fdc91bce640 1 -- 172.21.15.70:0/3707085038 >> 172.21.15.70:0/3707085038 conn(0x7fdc8c06a5d0 msgr2=0x7fdc8c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.721+0000 7fdc91bce640 1 -- 172.21.15.70:0/3707085038 shutdown_connections 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.721+0000 7fdc91bce640 1 -- 172.21.15.70:0/3707085038 wait complete. 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.723+0000 7fdc91bce640 1 Processor -- start 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.723+0000 7fdc91bce640 1 -- start start 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc91bce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 0x7fdc8c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc91bce640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c0c36a0 0x7fdc8c0c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc91bce640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c1588e0 0x7fdc8c158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc91bce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c1592a0 con 0x7fdc780039a0 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc91bce640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0c41a0 con 0x7fdc8c0c36a0 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc91bce640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0c4480 con 0x7fdc8c1588e0 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 0x7fdc8c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 0x7fdc8c15d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48868/0 (socket says 172.21.15.70:48868) 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc8b7fe640 1 -- 172.21.15.70:0/591762802 learned_addr learned my addr 172.21.15.70:0/591762802 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc83fff640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c0c36a0 0x7fdc8c0c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc8bfff640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c1588e0 0x7fdc8c158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc8b7fe640 1 -- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c1588e0 msgr2=0x7fdc8c158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc8b7fe640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c1588e0 0x7fdc8c158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.724+0000 7fdc8b7fe640 1 -- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c0c36a0 msgr2=0x7fdc8c0c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:27.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.725+0000 7fdc8b7fe640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c0c36a0 0x7fdc8c0c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:27.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.725+0000 7fdc8b7fe640 1 -- 172.21.15.70:0/591762802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0c4760 con 0x7fdc780039a0 2023-10-08T05:52:27.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.725+0000 7fdc83fff640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c0c36a0 0x7fdc8c0c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:27.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.725+0000 7fdc8b7fe640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 0x7fdc8c15d200 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c008fd0 tx=0x7fdc7c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:27.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.725+0000 7fdc88ff9640 1 -- 172.21.15.70:0/591762802 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0076c0 con 0x7fdc780039a0 2023-10-08T05:52:27.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.725+0000 7fdc88ff9640 1 -- 172.21.15.70:0/591762802 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c0069e0 con 0x7fdc780039a0 2023-10-08T05:52:27.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.725+0000 7fdc88ff9640 1 -- 172.21.15.70:0/591762802 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c006d50 con 0x7fdc780039a0 2023-10-08T05:52:27.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.725+0000 7fdc91bce640 1 -- 172.21.15.70:0/591762802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c0c4a40 con 0x7fdc780039a0 2023-10-08T05:52:27.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.725+0000 7fdc91bce640 1 -- 172.21.15.70:0/591762802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c15d740 con 0x7fdc780039a0 2023-10-08T05:52:27.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.727+0000 7fdc91bce640 1 -- 172.21.15.70:0/591762802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c15d930 con 0x7fdc780039a0 2023-10-08T05:52:27.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.727+0000 7fdc88ff9640 1 -- 172.21.15.70:0/591762802 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdc7c0035f0 con 0x7fdc780039a0 2023-10-08T05:52:27.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.729+0000 7fdc88ff9640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc6c06afc0 0x7fdc6c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:27.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.729+0000 7fdc88ff9640 1 -- 172.21.15.70:0/591762802 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdc7c0ab750 con 0x7fdc780039a0 2023-10-08T05:52:27.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.730+0000 7fdc83fff640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc6c06afc0 0x7fdc6c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:27.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.731+0000 7fdc83fff640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc6c06afc0 0x7fdc6c06d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0c3630 tx=0x7fdc74009040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:27.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.734+0000 7fdc88ff9640 1 -- 172.21.15.70:0/591762802 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc7c076500 con 0x7fdc780039a0 2023-10-08T05:52:27.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:27.965+0000 7fdc91bce640 1 -- 172.21.15.70:0/591762802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdc8c04e160 con 0x7fdc780039a0 2023-10-08T05:52:28.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.607+0000 7fdc88ff9640 1 -- 172.21.15.70:0/591762802 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdc7c079b20 con 0x7fdc780039a0 2023-10-08T05:52:28.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 -- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc6c06afc0 msgr2=0x7fdc6c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:28.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc6c06afc0 0x7fdc6c06d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0c3630 tx=0x7fdc74009040 comp rx=0 tx=0).stop 2023-10-08T05:52:28.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 -- 172.21.15.70:0/591762802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 msgr2=0x7fdc8c15d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:28.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 0x7fdc8c15d200 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c008fd0 tx=0x7fdc7c001120 comp rx=0 tx=0).stop 2023-10-08T05:52:28.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 -- 172.21.15.70:0/591762802 shutdown_connections 2023-10-08T05:52:28.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fdc6c06afc0 0x7fdc6c06d480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:28.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc8c1588e0 0x7fdc8c158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:28.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc8c0c36a0 0x7fdc8c0c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:28.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 --2- 172.21.15.70:0/591762802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc780039a0 0x7fdc8c15d200 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:28.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 -- 172.21.15.70:0/591762802 >> 172.21.15.70:0/591762802 conn(0x7fdc8c06a5d0 msgr2=0x7fdc8c06d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:28.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 -- 172.21.15.70:0/591762802 shutdown_connections 2023-10-08T05:52:28.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.611+0000 7fdc91bce640 1 -- 172.21.15.70:0/591762802 wait complete. 2023-10-08T05:52:28.634 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T05:52:28.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T05:52:28.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.855+0000 7efea1c10640 1 Processor -- start 2023-10-08T05:52:28.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.855+0000 7efea1c10640 1 -- start start 2023-10-08T05:52:28.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.857+0000 7efea1c10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe9c06c3a0 0x7efe9c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:28.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.857+0000 7efea1c10640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efe9c06db60 0x7efe9c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:28.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.857+0000 7efea1c10640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9c15b310 con 0x7efe9c15aa00 2023-10-08T05:52:28.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.857+0000 7efea1c10640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c0ca230 con 0x7efe9c06db60 2023-10-08T05:52:28.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.857+0000 7efea1c10640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c06d290 con 0x7efe9c06c3a0 2023-10-08T05:52:28.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.857+0000 7efe92ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efe9c06db60 0x7efe9c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:28.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.857+0000 7efe9bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efe9c15aa00 0x7efe9c0b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55168/0 (socket says 172.21.15.70:55168) 2023-10-08T05:52:28.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.857+0000 7efe9bfff640 1 -- 172.21.15.70:0/1793192319 learned_addr learned my addr 172.21.15.70:0/1793192319 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:28.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.858+0000 7efe9b7fe640 1 --2- 172.21.15.70:0/1793192319 >> v2:172.21.15.181:3300/0 conn(0x7efe9c06c3a0 0x7efe9c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:28.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.858+0000 7efe92ffd640 1 -- 172.21.15.70:0/1793192319 >> v2:172.21.15.181:3300/0 conn(0x7efe9c06c3a0 msgr2=0x7efe9c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:28.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.858+0000 7efe92ffd640 1 --2- 172.21.15.70:0/1793192319 >> v2:172.21.15.181:3300/0 conn(0x7efe9c06c3a0 0x7efe9c0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:28.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.858+0000 7efe92ffd640 1 -- 172.21.15.70:0/1793192319 >> v1:172.21.15.70:6789/0 conn(0x7efe9c15aa00 legacy=0x7efe9c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:28.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.858+0000 7efe9affd640 1 -- 172.21.15.70:0/1793192319 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519165427 0 0) 0x7efe94001490 con 0x7efe9c15aa00 2023-10-08T05:52:28.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.858+0000 7efe92ffd640 1 -- 172.21.15.70:0/1793192319 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0c9860 con 0x7efe9c06db60 2023-10-08T05:52:28.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.859+0000 7efe92ffd640 1 --2- 172.21.15.70:0/1793192319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 0x7efe9c06c780 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0716b0 tx=0x7efe8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35d143b1a4562f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:28.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.859+0000 7efe9affd640 1 -- 172.21.15.70:0/1793192319 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84031830 con 0x7efe9c06db60 2023-10-08T05:52:28.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.859+0000 7efe9affd640 1 -- 172.21.15.70:0/1793192319 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe84031990 con 0x7efe9c06db60 2023-10-08T05:52:28.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.859+0000 7efe9affd640 1 -- 172.21.15.70:0/1793192319 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84031d20 con 0x7efe9c06db60 2023-10-08T05:52:28.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.860+0000 7efea1c10640 1 -- 172.21.15.70:0/1793192319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 msgr2=0x7efe9c06c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:28.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.860+0000 7efea1c10640 1 --2- 172.21.15.70:0/1793192319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 0x7efe9c06c780 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0716b0 tx=0x7efe8402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:52:28.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.860+0000 7efea1c10640 1 -- 172.21.15.70:0/1793192319 shutdown_connections 2023-10-08T05:52:28.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.860+0000 7efea1c10640 1 --2- 172.21.15.70:0/1793192319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 0x7efe9c06c780 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:28.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.860+0000 7efea1c10640 1 --2- 172.21.15.70:0/1793192319 >> v2:172.21.15.181:3300/0 conn(0x7efe9c06c3a0 0x7efe9c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:28.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.860+0000 7efea1c10640 1 -- 172.21.15.70:0/1793192319 >> 172.21.15.70:0/1793192319 conn(0x7efe9c06a5d0 msgr2=0x7efe9c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:28.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.860+0000 7efea1c10640 1 -- 172.21.15.70:0/1793192319 shutdown_connections 2023-10-08T05:52:28.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.860+0000 7efea1c10640 1 -- 172.21.15.70:0/1793192319 wait complete. 2023-10-08T05:52:28.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.862+0000 7efea1c10640 1 Processor -- start 2023-10-08T05:52:28.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.862+0000 7efea1c10640 1 -- start start 2023-10-08T05:52:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.863+0000 7efea1c10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe9c06c3a0 0x7efe9c0c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.863+0000 7efea1c10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 0x7efe9c156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.863+0000 7efea1c10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9c15aa00 0x7efe9c157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.863+0000 7efea1c10640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0c6250 con 0x7efe9c06db60 2023-10-08T05:52:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.863+0000 7efea1c10640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c157690 con 0x7efe9c06c3a0 2023-10-08T05:52:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.863+0000 7efea1c10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c1f2450 con 0x7efe9c15aa00 2023-10-08T05:52:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.863+0000 7efe92ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 0x7efe9c156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.863+0000 7efe92ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 0x7efe9c156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48894/0 (socket says 172.21.15.70:48894) 2023-10-08T05:52:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.863+0000 7efe92ffd640 1 -- 172.21.15.70:0/1431118653 learned_addr learned my addr 172.21.15.70:0/1431118653 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe9b7fe640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe9c06c3a0 0x7efe9c0c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe9bfff640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9c15aa00 0x7efe9c157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe92ffd640 1 -- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9c15aa00 msgr2=0x7efe9c157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe92ffd640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9c15aa00 0x7efe9c157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe92ffd640 1 -- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe9c06c3a0 msgr2=0x7efe9c0c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe92ffd640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe9c06c3a0 0x7efe9c0c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe92ffd640 1 -- 172.21.15.70:0/1431118653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84009400 con 0x7efe9c06db60 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe9bfff640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9c15aa00 0x7efe9c157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe92ffd640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 0x7efe9c156c10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7efe8402bee0 tx=0x7efe840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe98ff9640 1 -- 172.21.15.70:0/1431118653 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8403ba70 con 0x7efe9c06db60 2023-10-08T05:52:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe98ff9640 1 -- 172.21.15.70:0/1431118653 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8403bbf0 con 0x7efe9c06db60 2023-10-08T05:52:28.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.864+0000 7efe98ff9640 1 -- 172.21.15.70:0/1431118653 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840433f0 con 0x7efe9c06db60 2023-10-08T05:52:28.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.865+0000 7efea1c10640 1 -- 172.21.15.70:0/1431118653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c0c8560 con 0x7efe9c06db60 2023-10-08T05:52:28.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.865+0000 7efea1c10640 1 -- 172.21.15.70:0/1431118653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c0c5b90 con 0x7efe9c06db60 2023-10-08T05:52:28.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.866+0000 7efea1c10640 1 -- 172.21.15.70:0/1431118653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c0c6390 con 0x7efe9c06db60 2023-10-08T05:52:28.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.867+0000 7efe98ff9640 1 -- 172.21.15.70:0/1431118653 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efe84043550 con 0x7efe9c06db60 2023-10-08T05:52:28.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.867+0000 7efe98ff9640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efe7c06af90 0x7efe7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:28.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.868+0000 7efe98ff9640 1 -- 172.21.15.70:0/1431118653 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe840ab7f0 con 0x7efe9c06db60 2023-10-08T05:52:28.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.868+0000 7efe9b7fe640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efe7c06af90 0x7efe7c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.869+0000 7efe9b7fe640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efe7c06af90 0x7efe7c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efe8c006fd0 tx=0x7efe8c008040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:28.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:28.872+0000 7efe98ff9640 1 -- 172.21.15.70:0/1431118653 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe9c0c6390 con 0x7efe9c06db60 2023-10-08T05:52:29.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.101+0000 7efea1c10640 1 -- 172.21.15.70:0/1431118653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efe9c0c6390 con 0x7efe9c06db60 2023-10-08T05:52:29.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.611+0000 7efe98ff9640 1 -- 172.21.15.70:0/1431118653 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7efe9c0c6390 con 0x7efe9c06db60 2023-10-08T05:52:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.614+0000 7efea1c10640 1 -- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efe7c06af90 msgr2=0x7efe7c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.614+0000 7efea1c10640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efe7c06af90 0x7efe7c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efe8c006fd0 tx=0x7efe8c008040 comp rx=0 tx=0).stop 2023-10-08T05:52:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.614+0000 7efea1c10640 1 -- 172.21.15.70:0/1431118653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 msgr2=0x7efe9c156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.614+0000 7efea1c10640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 0x7efe9c156c10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7efe8402bee0 tx=0x7efe840046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.614+0000 7efea1c10640 1 -- 172.21.15.70:0/1431118653 shutdown_connections 2023-10-08T05:52:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.614+0000 7efea1c10640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efe7c06af90 0x7efe7c06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.614+0000 7efea1c10640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe9c15aa00 0x7efe9c157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.614+0000 7efea1c10640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe9c06db60 0x7efe9c156c10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.614+0000 7efea1c10640 1 --2- 172.21.15.70:0/1431118653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe9c06c3a0 0x7efe9c0c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.614+0000 7efea1c10640 1 -- 172.21.15.70:0/1431118653 >> 172.21.15.70:0/1431118653 conn(0x7efe9c06a5d0 msgr2=0x7efe9c0cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:29.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.615+0000 7efea1c10640 1 -- 172.21.15.70:0/1431118653 shutdown_connections 2023-10-08T05:52:29.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.615+0000 7efea1c10640 1 -- 172.21.15.70:0/1431118653 wait complete. 2023-10-08T05:52:29.636 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T05:52:29.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:29.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.853+0000 7ff4604b4640 1 Processor -- start 2023-10-08T05:52:29.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.853+0000 7ff4604b4640 1 -- start start 2023-10-08T05:52:29.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.855+0000 7ff4604b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff458159ac0 0x7ff45806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:29.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.855+0000 7ff4604b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4580ad790 0x7ff4580702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:29.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.855+0000 7ff4604b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45806dee0 con 0x7ff45815e330 2023-10-08T05:52:29.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.855+0000 7ff4604b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45815d850 con 0x7ff458159ac0 2023-10-08T05:52:29.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.855+0000 7ff4604b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45815db30 con 0x7ff4580ad790 2023-10-08T05:52:29.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.855+0000 7ff45ea51640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff45815e330 0x7ff45806c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55182/0 (socket says 172.21.15.70:55182) 2023-10-08T05:52:29.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.855+0000 7ff45ea51640 1 -- 172.21.15.70:0/2856205683 learned_addr learned my addr 172.21.15.70:0/2856205683 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:29.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.855+0000 7ff45e250640 1 --2- 172.21.15.70:0/2856205683 >> v2:172.21.15.70:3300/0 conn(0x7ff458159ac0 0x7ff45806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:29.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.856+0000 7ff45da4f640 1 --2- 172.21.15.70:0/2856205683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4580ad790 0x7ff4580702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:29.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.856+0000 7ff45e250640 1 -- 172.21.15.70:0/2856205683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4580ad790 msgr2=0x7ff4580702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:29.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.856+0000 7ff45e250640 1 --2- 172.21.15.70:0/2856205683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4580ad790 0x7ff4580702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.856+0000 7ff45e250640 1 -- 172.21.15.70:0/2856205683 >> v1:172.21.15.70:6789/0 conn(0x7ff45815e330 legacy=0x7ff45806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:29.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.856+0000 7ff45e250640 1 -- 172.21.15.70:0/2856205683 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45815de10 con 0x7ff458159ac0 2023-10-08T05:52:29.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.857+0000 7ff45e250640 1 --2- 172.21.15.70:0/2856205683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 0x7ff45806d5e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff4540097a0 tx=0x7ff45402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2663845d8580c634 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:29.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.857+0000 7ff45d24e640 1 -- 172.21.15.70:0/2856205683 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45402d040 con 0x7ff458159ac0 2023-10-08T05:52:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.857+0000 7ff45d24e640 1 -- 172.21.15.70:0/2856205683 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff454031810 con 0x7ff458159ac0 2023-10-08T05:52:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.857+0000 7ff45d24e640 1 -- 172.21.15.70:0/2856205683 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff454031ba0 con 0x7ff458159ac0 2023-10-08T05:52:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.858+0000 7ff4604b4640 1 -- 172.21.15.70:0/2856205683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 msgr2=0x7ff45806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.858+0000 7ff4604b4640 1 --2- 172.21.15.70:0/2856205683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 0x7ff45806d5e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff4540097a0 tx=0x7ff45402b8b0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.858+0000 7ff4604b4640 1 -- 172.21.15.70:0/2856205683 shutdown_connections 2023-10-08T05:52:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.858+0000 7ff4604b4640 1 --2- 172.21.15.70:0/2856205683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff4580ad790 0x7ff4580702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.858+0000 7ff4604b4640 1 --2- 172.21.15.70:0/2856205683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 0x7ff45806d5e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.858+0000 7ff4604b4640 1 -- 172.21.15.70:0/2856205683 >> 172.21.15.70:0/2856205683 conn(0x7ff45806a380 msgr2=0x7ff4580b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.858+0000 7ff4604b4640 1 -- 172.21.15.70:0/2856205683 shutdown_connections 2023-10-08T05:52:29.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.858+0000 7ff4604b4640 1 -- 172.21.15.70:0/2856205683 wait complete. 2023-10-08T05:52:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.860+0000 7ff4604b4640 1 Processor -- start 2023-10-08T05:52:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.861+0000 7ff4604b4640 1 -- start start 2023-10-08T05:52:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.862+0000 7ff4604b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4580ad790 0x7ff4581510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.862+0000 7ff4604b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 0x7ff45815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.862+0000 7ff4604b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff45815e330 0x7ff4580b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.862+0000 7ff4604b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45815b400 con 0x7ff458159ac0 2023-10-08T05:52:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.862+0000 7ff4604b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff458151940 con 0x7ff4580ad790 2023-10-08T05:52:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.862+0000 7ff4604b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45815cba0 con 0x7ff45815e330 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.862+0000 7ff45da4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 0x7ff45815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.862+0000 7ff45da4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 0x7ff45815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48920/0 (socket says 172.21.15.70:48920) 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.862+0000 7ff45da4f640 1 -- 172.21.15.70:0/2344373479 learned_addr learned my addr 172.21.15.70:0/2344373479 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff45da4f640 1 -- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff45815e330 msgr2=0x7ff4580b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff45e250640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4580ad790 0x7ff4581510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff45ea51640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff45815e330 0x7ff4580b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff45da4f640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff45815e330 0x7ff4580b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff45da4f640 1 -- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4580ad790 msgr2=0x7ff4581510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff45da4f640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4580ad790 0x7ff4581510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff45da4f640 1 -- 172.21.15.70:0/2344373479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff454009400 con 0x7ff458159ac0 2023-10-08T05:52:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff45e250640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4580ad790 0x7ff4581510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff45da4f640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 0x7ff45815c4a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff44800d800 tx=0x7ff44800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff44effd640 1 -- 172.21.15.70:0/2344373479 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44800b6c0 con 0x7ff458159ac0 2023-10-08T05:52:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff44effd640 1 -- 172.21.15.70:0/2344373479 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44800b820 con 0x7ff458159ac0 2023-10-08T05:52:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff44effd640 1 -- 172.21.15.70:0/2344373479 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44800bbb0 con 0x7ff458159ac0 2023-10-08T05:52:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.863+0000 7ff4604b4640 1 -- 172.21.15.70:0/2344373479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45815aad0 con 0x7ff458159ac0 2023-10-08T05:52:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.864+0000 7ff4604b4640 1 -- 172.21.15.70:0/2344373479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45815b090 con 0x7ff458159ac0 2023-10-08T05:52:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.865+0000 7ff4604b4640 1 -- 172.21.15.70:0/2344373479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45815b5b0 con 0x7ff458159ac0 2023-10-08T05:52:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.866+0000 7ff44effd640 1 -- 172.21.15.70:0/2344373479 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff44800bd10 con 0x7ff458159ac0 2023-10-08T05:52:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.867+0000 7ff44effd640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff43c06afc0 0x7ff43c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.867+0000 7ff44effd640 1 -- 172.21.15.70:0/2344373479 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff448089ed0 con 0x7ff458159ac0 2023-10-08T05:52:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.868+0000 7ff45e250640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff43c06afc0 0x7ff43c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:29.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.869+0000 7ff45e250640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff43c06afc0 0x7ff43c06d480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff454036000 tx=0x7ff45403c040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:29.871+0000 7ff44effd640 1 -- 172.21.15.70:0/2344373479 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff45815b5b0 con 0x7ff458159ac0 2023-10-08T05:52:30.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.100+0000 7ff4604b4640 1 -- 172.21.15.70:0/2344373479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff45815b5b0 con 0x7ff458159ac0 2023-10-08T05:52:30.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.101+0000 7ff44effd640 1 -- 172.21.15.70:0/2344373479 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4920 (secure 0 0 0) 0x7ff45815b5b0 con 0x7ff458159ac0 2023-10-08T05:52:30.102 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:30.103 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1038137327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1038137327},{"type":"v1","addr":"172.21.15.181:6835","nonce":1038137327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6660,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1967783419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1967783419},{"type":"v1","addr":"172.21.15.70:6833","nonce":1967783419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6663,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1304760341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1304760341},{"type":"v1","addr":"172.21.15.70:6835","nonce":1304760341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:52:26.053790+0000","modified":"2023-10-08T05:52:29.103940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6675},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6675":{"gid":6675,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2348955257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2348955257},{"type":"v1","addr":"172.21.15.181:6837","nonce":2348955257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-08T05:52:30.103 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 33 2023-10-08T05:52:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.103+0000 7ff4604b4640 1 -- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff43c06afc0 msgr2=0x7ff43c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.103+0000 7ff4604b4640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff43c06afc0 0x7ff43c06d480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff454036000 tx=0x7ff45403c040 comp rx=0 tx=0).stop 2023-10-08T05:52:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.103+0000 7ff4604b4640 1 -- 172.21.15.70:0/2344373479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 msgr2=0x7ff45815c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.103+0000 7ff4604b4640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 0x7ff45815c4a0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff44800d800 tx=0x7ff44800dcc0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.104+0000 7ff4604b4640 1 -- 172.21.15.70:0/2344373479 shutdown_connections 2023-10-08T05:52:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.104+0000 7ff4604b4640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7ff43c06afc0 0x7ff43c06d480 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.104+0000 7ff4604b4640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff45815e330 0x7ff4580b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.104+0000 7ff4604b4640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff458159ac0 0x7ff45815c4a0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.104+0000 7ff4604b4640 1 --2- 172.21.15.70:0/2344373479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff4580ad790 0x7ff4581510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.104+0000 7ff4604b4640 1 -- 172.21.15.70:0/2344373479 >> 172.21.15.70:0/2344373479 conn(0x7ff45806a380 msgr2=0x7ff4580c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.104+0000 7ff4604b4640 1 -- 172.21.15.70:0/2344373479 shutdown_connections 2023-10-08T05:52:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.104+0000 7ff4604b4640 1 -- 172.21.15.70:0/2344373479 wait complete. 2023-10-08T05:52:30.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:30.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.338+0000 7fbf6ad25640 1 Processor -- start 2023-10-08T05:52:30.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.338+0000 7fbf6ad25640 1 -- start start 2023-10-08T05:52:30.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.340+0000 7fbf6ad25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640cb580 0x7fbf64165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.340+0000 7fbf6ad25640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6415b2b0 con 0x7fbf640c90b0 2023-10-08T05:52:30.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.340+0000 7fbf6ad25640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6415b590 con 0x7fbf6415ab40 2023-10-08T05:52:30.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.340+0000 7fbf6ad25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6415bf30 con 0x7fbf640cb580 2023-10-08T05:52:30.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.341+0000 7fbf63fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbf640c90b0 0x7fbf640cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55186/0 (socket says 172.21.15.70:55186) 2023-10-08T05:52:30.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.341+0000 7fbf63fff640 1 -- 172.21.15.70:0/4087433340 learned_addr learned my addr 172.21.15.70:0/4087433340 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:30.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.341+0000 7fbf692c2640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640cb580 0x7fbf64165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.341+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085287920 0 0) 0x7fbf6415b2b0 con 0x7fbf640c90b0 2023-10-08T05:52:30.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.341+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf50003520 con 0x7fbf640c90b0 2023-10-08T05:52:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554539312 0 0) 0x7fbf6415b590 con 0x7fbf6415ab40 2023-10-08T05:52:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf6415b2b0 con 0x7fbf6415ab40 2023-10-08T05:52:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2810808330 0 0) 0x7fbf50003520 con 0x7fbf640c90b0 2023-10-08T05:52:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640cb580 msgr2=0x7fbf64165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:30.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640cb580 0x7fbf64165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 >> v1:172.21.15.181:6789/0 conn(0x7fbf6415ab40 legacy=0x7fbf640ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6415b870 con 0x7fbf640c90b0 2023-10-08T05:52:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fbf4c001690 con 0x7fbf640c90b0 2023-10-08T05:52:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 >> v1:172.21.15.70:6789/0 conn(0x7fbf640c90b0 legacy=0x7fbf640cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640cb580 0x7fbf50003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf500040e0 0x7fbf500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 0x7fbf50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6415bf30 con 0x7fbf50004d00 2023-10-08T05:52:30.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5000bac0 con 0x7fbf500040e0 2023-10-08T05:52:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5000bc80 con 0x7fbf640cb580 2023-10-08T05:52:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf692c2640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 0x7fbf50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf68ac1640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640cb580 0x7fbf50003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf63fff640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf500040e0 0x7fbf500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf692c2640 1 -- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640cb580 msgr2=0x7fbf50003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:30.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf692c2640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640cb580 0x7fbf50003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf692c2640 1 -- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf500040e0 msgr2=0x7fbf500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:30.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.342+0000 7fbf692c2640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf500040e0 0x7fbf500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.343+0000 7fbf692c2640 1 -- 172.21.15.70:0/4087433340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf50003520 con 0x7fbf50004d00 2023-10-08T05:52:30.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.343+0000 7fbf63fff640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf500040e0 0x7fbf500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:30.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.344+0000 7fbf692c2640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 0x7fbf50005180 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009ac0 tx=0x7fbf500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d990bed5f5821b84 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:30.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.344+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c02c6b0 con 0x7fbf50004d00 2023-10-08T05:52:30.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.344+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c02c810 con 0x7fbf50004d00 2023-10-08T05:52:30.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.344+0000 7fbf637fe640 1 -- 172.21.15.70:0/4087433340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c02cba0 con 0x7fbf50004d00 2023-10-08T05:52:30.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.344+0000 7fbf6ad25640 1 -- 172.21.15.70:0/4087433340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 msgr2=0x7fbf50005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.344+0000 7fbf6ad25640 1 --2- 172.21.15.70:0/4087433340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 0x7fbf50005180 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009ac0 tx=0x7fbf500057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.344+0000 7fbf692c2640 1 -- 172.21.15.70:0/4087433340 reap_dead start 2023-10-08T05:52:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.345+0000 7fbf6ad25640 1 -- 172.21.15.70:0/4087433340 shutdown_connections 2023-10-08T05:52:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.345+0000 7fbf6ad25640 1 -- 172.21.15.70:0/4087433340 >> 172.21.15.70:0/4087433340 conn(0x7fbf6406a380 msgr2=0x7fbf640b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.345+0000 7fbf6ad25640 1 -- 172.21.15.70:0/4087433340 shutdown_connections 2023-10-08T05:52:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.345+0000 7fbf6ad25640 1 -- 172.21.15.70:0/4087433340 wait complete. 2023-10-08T05:52:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.346+0000 7fbf6ad25640 1 Processor -- start 2023-10-08T05:52:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.347+0000 7fbf6ad25640 1 -- start start 2023-10-08T05:52:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf6ad25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 0x7fbf641463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf6ad25640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf64151750 0x7fbf640c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf6ad25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640c4e00 0x7fbf640c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf6ad25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6415f7a0 con 0x7fbf50004d00 2023-10-08T05:52:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf6ad25640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640c5d90 con 0x7fbf64151750 2023-10-08T05:52:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf6ad25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640c4970 con 0x7fbf640c4e00 2023-10-08T05:52:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf68ac1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 0x7fbf641463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf68ac1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 0x7fbf641463b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48946/0 (socket says 172.21.15.70:48946) 2023-10-08T05:52:30.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf68ac1640 1 -- 172.21.15.70:0/3120901209 learned_addr learned my addr 172.21.15.70:0/3120901209 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf68ac1640 1 -- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640c4e00 msgr2=0x7fbf640c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf68ac1640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640c4e00 0x7fbf640c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf68ac1640 1 -- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf64151750 msgr2=0x7fbf640c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf68ac1640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf64151750 0x7fbf640c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.348+0000 7fbf68ac1640 1 -- 172.21.15.70:0/3120901209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640c3a80 con 0x7fbf50004d00 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.349+0000 7fbf68ac1640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 0x7fbf641463b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400eaf0 tx=0x7fbf540020d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.349+0000 7fbf617fa640 1 -- 172.21.15.70:0/3120901209 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54003b80 con 0x7fbf50004d00 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.349+0000 7fbf617fa640 1 -- 172.21.15.70:0/3120901209 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf54003ce0 con 0x7fbf50004d00 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.349+0000 7fbf6ad25640 1 -- 172.21.15.70:0/3120901209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf640c3d60 con 0x7fbf50004d00 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.349+0000 7fbf617fa640 1 -- 172.21.15.70:0/3120901209 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54005310 con 0x7fbf50004d00 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.349+0000 7fbf6ad25640 1 -- 172.21.15.70:0/3120901209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf640c4320 con 0x7fbf50004d00 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.350+0000 7fbf6ad25640 1 -- 172.21.15.70:0/3120901209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf640cdce0 con 0x7fbf50004d00 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.351+0000 7fbf617fa640 1 -- 172.21.15.70:0/3120901209 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbf54005470 con 0x7fbf50004d00 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.352+0000 7fbf617fa640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbf4406af90 0x7fbf4406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.352+0000 7fbf617fa640 1 -- 172.21.15.70:0/3120901209 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbf54089b20 con 0x7fbf50004d00 2023-10-08T05:52:30.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.353+0000 7fbf63fff640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbf4406af90 0x7fbf4406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.354+0000 7fbf63fff640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbf4406af90 0x7fbf4406d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c001010 tx=0x7fbf4c009040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:30.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.356+0000 7fbf617fa640 1 -- 172.21.15.70:0/3120901209 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf54054940 con 0x7fbf50004d00 2023-10-08T05:52:30.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.547+0000 7fbf6ad25640 1 -- 172.21.15.70:0/3120901209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf64067490 con 0x7fbf50004d00 2023-10-08T05:52:30.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.549+0000 7fbf617fa640 1 -- 172.21.15.70:0/3120901209 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16774 (secure 0 0 0) 0x7fbf5401a310 con 0x7fbf50004d00 2023-10-08T05:52:30.551 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:30.551 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":90,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:27.597382+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:52:23.846215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-08T05:52:25.069536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:30.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 -- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbf4406af90 msgr2=0x7fbf4406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbf4406af90 0x7fbf4406d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c001010 tx=0x7fbf4c009040 comp rx=0 tx=0).stop 2023-10-08T05:52:30.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 -- 172.21.15.70:0/3120901209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 msgr2=0x7fbf641463b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 0x7fbf641463b0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400eaf0 tx=0x7fbf540020d0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 -- 172.21.15.70:0/3120901209 shutdown_connections 2023-10-08T05:52:30.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbf4406af90 0x7fbf4406d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf640c4e00 0x7fbf640c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf64151750 0x7fbf640c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 --2- 172.21.15.70:0/3120901209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf50004d00 0x7fbf641463b0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 -- 172.21.15.70:0/3120901209 >> 172.21.15.70:0/3120901209 conn(0x7fbf6406a380 msgr2=0x7fbf640cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:30.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 -- 172.21.15.70:0/3120901209 shutdown_connections 2023-10-08T05:52:30.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.552+0000 7fbf6ad25640 1 -- 172.21.15.70:0/3120901209 wait complete. 2023-10-08T05:52:30.572 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T05:52:30.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:52:30.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.784+0000 7f2dc2f83640 1 Processor -- start 2023-10-08T05:52:30.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.784+0000 7f2dc2f83640 1 -- start start 2023-10-08T05:52:30.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dc2f83640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc159ac0 0x7f2dbc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dc2f83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dbc0ad790 0x7f2dbc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dc2f83640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2dbc15e330 0x7f2dbc070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dc2f83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc06dee0 con 0x7f2dbc0ad790 2023-10-08T05:52:30.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dc2f83640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc15d850 con 0x7f2dbc15e330 2023-10-08T05:52:30.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dc2f83640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc15db30 con 0x7f2dbc159ac0 2023-10-08T05:52:30.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dbbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dbc0ad790 0x7f2dbc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dbbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dbc0ad790 0x7f2dbc0702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48954/0 (socket says 172.21.15.70:48954) 2023-10-08T05:52:30.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dbbfff640 1 -- 172.21.15.70:0/3146650325 learned_addr learned my addr 172.21.15.70:0/3146650325 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:30.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dbbfff640 1 -- 172.21.15.70:0/3146650325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc159ac0 msgr2=0x7f2dbc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:30.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.786+0000 7f2dc0d1f640 1 --2- 172.21.15.70:0/3146650325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc159ac0 0x7f2dbc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.787+0000 7f2dbbfff640 1 --2- 172.21.15.70:0/3146650325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc159ac0 0x7f2dbc06d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.787+0000 7f2dc1520640 1 --2- 172.21.15.70:0/3146650325 >> v2:172.21.15.181:3300/0 conn(0x7f2dbc15e330 0x7f2dbc070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.787+0000 7f2dbbfff640 1 -- 172.21.15.70:0/3146650325 >> v2:172.21.15.181:3300/0 conn(0x7f2dbc15e330 msgr2=0x7f2dbc070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:30.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.787+0000 7f2dbbfff640 1 --2- 172.21.15.70:0/3146650325 >> v2:172.21.15.181:3300/0 conn(0x7f2dbc15e330 0x7f2dbc070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.787+0000 7f2dbbfff640 1 -- 172.21.15.70:0/3146650325 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc15de10 con 0x7f2dbc0ad790 2023-10-08T05:52:30.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.788+0000 7f2dbbfff640 1 --2- 172.21.15.70:0/3146650325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc0ad790 0x7f2dbc0702d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0ce730 tx=0x7f2da402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81cdd4d5d9a1f7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:30.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.788+0000 7f2dbb7fe640 1 -- 172.21.15.70:0/3146650325 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4031870 con 0x7f2dbc0ad790 2023-10-08T05:52:30.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.788+0000 7f2dbb7fe640 1 -- 172.21.15.70:0/3146650325 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da40319d0 con 0x7f2dbc0ad790 2023-10-08T05:52:30.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.788+0000 7f2dbb7fe640 1 -- 172.21.15.70:0/3146650325 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4031d60 con 0x7f2dbc0ad790 2023-10-08T05:52:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.789+0000 7f2dc2f83640 1 -- 172.21.15.70:0/3146650325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc0ad790 msgr2=0x7f2dbc0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.789+0000 7f2dc2f83640 1 --2- 172.21.15.70:0/3146650325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc0ad790 0x7f2dbc0702d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc0ce730 tx=0x7f2da402b910 comp rx=0 tx=0).stop 2023-10-08T05:52:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.789+0000 7f2dc2f83640 1 -- 172.21.15.70:0/3146650325 shutdown_connections 2023-10-08T05:52:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.789+0000 7f2dc2f83640 1 --2- 172.21.15.70:0/3146650325 >> v2:172.21.15.181:3300/0 conn(0x7f2dbc15e330 0x7f2dbc070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.789+0000 7f2dc2f83640 1 --2- 172.21.15.70:0/3146650325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc0ad790 0x7f2dbc0702d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.789+0000 7f2dc2f83640 1 --2- 172.21.15.70:0/3146650325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc159ac0 0x7f2dbc06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.789+0000 7f2dc2f83640 1 -- 172.21.15.70:0/3146650325 >> 172.21.15.70:0/3146650325 conn(0x7f2dbc06a380 msgr2=0x7f2dbc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.789+0000 7f2dc2f83640 1 -- 172.21.15.70:0/3146650325 shutdown_connections 2023-10-08T05:52:30.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.789+0000 7f2dc2f83640 1 -- 172.21.15.70:0/3146650325 wait complete. 2023-10-08T05:52:30.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.791+0000 7f2dc2f83640 1 Processor -- start 2023-10-08T05:52:30.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.792+0000 7f2dc2f83640 1 -- start start 2023-10-08T05:52:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dc2f83640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc0ad790 0x7f2dbc0c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dc2f83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc159ac0 0x7f2dbc15c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dc2f83640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dbc15e330 0x7f2dbc164a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dc2f83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0ce0e0 con 0x7f2dbc159ac0 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dc2f83640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0ce3c0 con 0x7f2dbc15e330 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dc2f83640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc15a810 con 0x7f2dbc0ad790 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc159ac0 0x7f2dbc15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dbbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc159ac0 0x7f2dbc15c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48970/0 (socket says 172.21.15.70:48970) 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dbbfff640 1 -- 172.21.15.70:0/2984819859 learned_addr learned my addr 172.21.15.70:0/2984819859 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dbbfff640 1 -- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc0ad790 msgr2=0x7f2dbc0c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dbbfff640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc0ad790 0x7f2dbc0c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dbbfff640 1 -- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dbc15e330 msgr2=0x7f2dbc164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dbbfff640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dbc15e330 0x7f2dbc164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:30.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.793+0000 7f2dbbfff640 1 -- 172.21.15.70:0/2984819859 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da40094d0 con 0x7f2dbc159ac0 2023-10-08T05:52:30.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.794+0000 7f2dbbfff640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc159ac0 0x7f2dbc15c820 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2da4000c00 tx=0x7f2da40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:30.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.794+0000 7f2db97fa640 1 -- 172.21.15.70:0/2984819859 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4031860 con 0x7f2dbc159ac0 2023-10-08T05:52:30.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.794+0000 7f2db97fa640 1 -- 172.21.15.70:0/2984819859 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da40443f0 con 0x7f2dbc159ac0 2023-10-08T05:52:30.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.794+0000 7f2db97fa640 1 -- 172.21.15.70:0/2984819859 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da40447a0 con 0x7f2dbc159ac0 2023-10-08T05:52:30.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.794+0000 7f2dc2f83640 1 -- 172.21.15.70:0/2984819859 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc15aaf0 con 0x7f2dbc159ac0 2023-10-08T05:52:30.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.794+0000 7f2dc2f83640 1 -- 172.21.15.70:0/2984819859 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc0c9fe0 con 0x7f2dbc159ac0 2023-10-08T05:52:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.795+0000 7f2dc2f83640 1 -- 172.21.15.70:0/2984819859 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc067490 con 0x7f2dbc159ac0 2023-10-08T05:52:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.796+0000 7f2db97fa640 1 -- 172.21.15.70:0/2984819859 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2da4044900 con 0x7f2dbc159ac0 2023-10-08T05:52:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.797+0000 7f2db97fa640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f2d9c06af90 0x7f2d9c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.797+0000 7f2db97fa640 1 -- 172.21.15.70:0/2984819859 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2da402d040 con 0x7f2dbc159ac0 2023-10-08T05:52:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.798+0000 7f2dc0d1f640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f2d9c06af90 0x7f2d9c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:30.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.799+0000 7f2dc0d1f640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f2d9c06af90 0x7f2d9c06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2dac007130 tx=0x7f2dac007310 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.802+0000 7f2db97fa640 1 -- 172.21.15.70:0/2984819859 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2da4077080 con 0x7f2dbc159ac0 2023-10-08T05:52:30.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:30.986+0000 7f2dc2f83640 1 -- 172.21.15.70:0/2984819859 --> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2dbc159ea0 con 0x7f2d9c06af90 2023-10-08T05:52:31.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.015+0000 7f2db97fa640 1 -- 172.21.15.70:0/2984819859 <== mgr.5473 v2:172.21.15.181:6832/3994892546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286630 (secure 0 0 0) 0x7f2dac007e00 con 0x7f2d9c06af90 2023-10-08T05:52:31.020 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:31.021 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:52:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 -- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f2d9c06af90 msgr2=0x7f2d9c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f2d9c06af90 0x7f2d9c06d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2dac007130 tx=0x7f2dac007310 comp rx=0 tx=0).stop 2023-10-08T05:52:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 -- 172.21.15.70:0/2984819859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc159ac0 msgr2=0x7f2dbc15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc159ac0 0x7f2dbc15c820 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f2da4000c00 tx=0x7f2da40046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 -- 172.21.15.70:0/2984819859 shutdown_connections 2023-10-08T05:52:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f2d9c06af90 0x7f2d9c06d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dbc15e330 0x7f2dbc164a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc159ac0 0x7f2dbc15c820 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 --2- 172.21.15.70:0/2984819859 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc0ad790 0x7f2dbc0c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 -- 172.21.15.70:0/2984819859 >> 172.21.15.70:0/2984819859 conn(0x7f2dbc06a380 msgr2=0x7f2dbc0ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 -- 172.21.15.70:0/2984819859 shutdown_connections 2023-10-08T05:52:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.020+0000 7f2dc2f83640 1 -- 172.21.15.70:0/2984819859 wait complete. 2023-10-08T05:52:31.042 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-08T05:52:30.592906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":324456,"kb_used_data":9768,"kb_used_omap":12,"kb_used_meta":314547,"kb_avail":754650264,"statfs":{"total":773094113280,"available":772761870336,"internally_reserved":0,"allocated":10002432,"data_stored":5600108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":322096699},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002180"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.050025+0000","last_change":"2023-10-08T05:52:26.607722+0000","last_active":"2023-10-08T05:52:28.050025+0000","last_peered":"2023-10-08T05:52:28.050025+0000","last_clean":"2023-10-08T05:52:28.050025+0000","last_became_active":"2023-10-08T05:52:26.607448+0000","last_became_peered":"2023-10-08T05:52:26.607448+0000","last_unstale":"2023-10-08T05:52:28.050025+0000","last_undegraded":"2023-10-08T05:52:28.050025+0000","last_fullsized":"2023-10-08T05:52:28.050025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:20:46.060726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.049735+0000","last_change":"2023-10-08T05:52:26.611368+0000","last_active":"2023-10-08T05:52:28.049735+0000","last_peered":"2023-10-08T05:52:28.049735+0000","last_clean":"2023-10-08T05:52:28.049735+0000","last_became_active":"2023-10-08T05:52:26.611115+0000","last_became_peered":"2023-10-08T05:52:26.611115+0000","last_unstale":"2023-10-08T05:52:28.049735+0000","last_undegraded":"2023-10-08T05:52:28.049735+0000","last_fullsized":"2023-10-08T05:52:28.049735+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:59:51.102207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.633432+0000","last_change":"2023-10-08T05:52:26.627434+0000","last_active":"2023-10-08T05:52:27.633432+0000","last_peered":"2023-10-08T05:52:27.633432+0000","last_clean":"2023-10-08T05:52:27.633432+0000","last_became_active":"2023-10-08T05:52:26.626262+0000","last_became_peered":"2023-10-08T05:52:26.626262+0000","last_unstale":"2023-10-08T05:52:27.633432+0000","last_undegraded":"2023-10-08T05:52:27.633432+0000","last_fullsized":"2023-10-08T05:52:27.633432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:48.081753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612692+0000","last_change":"2023-10-08T05:52:26.603872+0000","last_active":"2023-10-08T05:52:27.612692+0000","last_peered":"2023-10-08T05:52:27.612692+0000","last_clean":"2023-10-08T05:52:27.612692+0000","last_became_active":"2023-10-08T05:52:26.603593+0000","last_became_peered":"2023-10-08T05:52:26.603593+0000","last_unstale":"2023-10-08T05:52:27.612692+0000","last_undegraded":"2023-10-08T05:52:27.612692+0000","last_fullsized":"2023-10-08T05:52:27.612692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:40.962951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613819+0000","last_change":"2023-10-08T05:52:26.609920+0000","last_active":"2023-10-08T05:52:27.613819+0000","last_peered":"2023-10-08T05:52:27.613819+0000","last_clean":"2023-10-08T05:52:27.613819+0000","last_became_active":"2023-10-08T05:52:26.608134+0000","last_became_peered":"2023-10-08T05:52:26.608134+0000","last_unstale":"2023-10-08T05:52:27.613819+0000","last_undegraded":"2023-10-08T05:52:27.613819+0000","last_fullsized":"2023-10-08T05:52:27.613819+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:47:02.038317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.048335+0000","last_change":"2023-10-08T05:52:26.622173+0000","last_active":"2023-10-08T05:52:28.048335+0000","last_peered":"2023-10-08T05:52:28.048335+0000","last_clean":"2023-10-08T05:52:28.048335+0000","last_became_active":"2023-10-08T05:52:26.621030+0000","last_became_peered":"2023-10-08T05:52:26.621030+0000","last_unstale":"2023-10-08T05:52:28.048335+0000","last_undegraded":"2023-10-08T05:52:28.048335+0000","last_fullsized":"2023-10-08T05:52:28.048335+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:46.620903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046114+0000","last_change":"2023-10-08T05:52:26.616818+0000","last_active":"2023-10-08T05:52:28.046114+0000","last_peered":"2023-10-08T05:52:28.046114+0000","last_clean":"2023-10-08T05:52:28.046114+0000","last_became_active":"2023-10-08T05:52:26.616534+0000","last_became_peered":"2023-10-08T05:52:26.616534+0000","last_unstale":"2023-10-08T05:52:28.046114+0000","last_undegraded":"2023-10-08T05:52:28.046114+0000","last_fullsized":"2023-10-08T05:52:28.046114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:55.547855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046444+0000","last_change":"2023-10-08T05:52:26.621106+0000","last_active":"2023-10-08T05:52:28.046444+0000","last_peered":"2023-10-08T05:52:28.046444+0000","last_clean":"2023-10-08T05:52:28.046444+0000","last_became_active":"2023-10-08T05:52:26.620787+0000","last_became_peered":"2023-10-08T05:52:26.620787+0000","last_unstale":"2023-10-08T05:52:28.046444+0000","last_undegraded":"2023-10-08T05:52:28.046444+0000","last_fullsized":"2023-10-08T05:52:28.046444+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:26.165323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.633324+0000","last_change":"2023-10-08T05:52:26.630837+0000","last_active":"2023-10-08T05:52:27.633324+0000","last_peered":"2023-10-08T05:52:27.633324+0000","last_clean":"2023-10-08T05:52:27.633324+0000","last_became_active":"2023-10-08T05:52:26.630527+0000","last_became_peered":"2023-10-08T05:52:26.630527+0000","last_unstale":"2023-10-08T05:52:27.633324+0000","last_undegraded":"2023-10-08T05:52:27.633324+0000","last_fullsized":"2023-10-08T05:52:27.633324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:53.711411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.632914+0000","last_change":"2023-10-08T05:52:26.630342+0000","last_active":"2023-10-08T05:52:27.632914+0000","last_peered":"2023-10-08T05:52:27.632914+0000","last_clean":"2023-10-08T05:52:27.632914+0000","last_became_active":"2023-10-08T05:52:26.628804+0000","last_became_peered":"2023-10-08T05:52:26.628804+0000","last_unstale":"2023-10-08T05:52:27.632914+0000","last_undegraded":"2023-10-08T05:52:27.632914+0000","last_fullsized":"2023-10-08T05:52:27.632914+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:32.655131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613460+0000","last_change":"2023-10-08T05:52:26.608390+0000","last_active":"2023-10-08T05:52:27.613460+0000","last_peered":"2023-10-08T05:52:27.613460+0000","last_clean":"2023-10-08T05:52:27.613460+0000","last_became_active":"2023-10-08T05:52:26.608119+0000","last_became_peered":"2023-10-08T05:52:26.608119+0000","last_unstale":"2023-10-08T05:52:27.613460+0000","last_undegraded":"2023-10-08T05:52:27.613460+0000","last_fullsized":"2023-10-08T05:52:27.613460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:15.319541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046342+0000","last_change":"2023-10-08T05:52:26.619890+0000","last_active":"2023-10-08T05:52:28.046342+0000","last_peered":"2023-10-08T05:52:28.046342+0000","last_clean":"2023-10-08T05:52:28.046342+0000","last_became_active":"2023-10-08T05:52:26.619597+0000","last_became_peered":"2023-10-08T05:52:26.619597+0000","last_unstale":"2023-10-08T05:52:28.046342+0000","last_undegraded":"2023-10-08T05:52:28.046342+0000","last_fullsized":"2023-10-08T05:52:28.046342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:48.081753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046674+0000","last_change":"2023-10-08T05:52:26.612317+0000","last_active":"2023-10-08T05:52:28.046674+0000","last_peered":"2023-10-08T05:52:28.046674+0000","last_clean":"2023-10-08T05:52:28.046674+0000","last_became_active":"2023-10-08T05:52:26.612023+0000","last_became_peered":"2023-10-08T05:52:26.612023+0000","last_unstale":"2023-10-08T05:52:28.046674+0000","last_undegraded":"2023-10-08T05:52:28.046674+0000","last_fullsized":"2023-10-08T05:52:28.046674+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:15.670901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.633053+0000","last_change":"2023-10-08T05:52:26.626380+0000","last_active":"2023-10-08T05:52:27.633053+0000","last_peered":"2023-10-08T05:52:27.633053+0000","last_clean":"2023-10-08T05:52:27.633053+0000","last_became_active":"2023-10-08T05:52:26.626027+0000","last_became_peered":"2023-10-08T05:52:26.626027+0000","last_unstale":"2023-10-08T05:52:27.633053+0000","last_undegraded":"2023-10-08T05:52:27.633053+0000","last_fullsized":"2023-10-08T05:52:27.633053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:42.293374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.632428+0000","last_change":"2023-10-08T05:52:26.628459+0000","last_active":"2023-10-08T05:52:27.632428+0000","last_peered":"2023-10-08T05:52:27.632428+0000","last_clean":"2023-10-08T05:52:27.632428+0000","last_became_active":"2023-10-08T05:52:26.628133+0000","last_became_peered":"2023-10-08T05:52:26.628133+0000","last_unstale":"2023-10-08T05:52:27.632428+0000","last_undegraded":"2023-10-08T05:52:27.632428+0000","last_fullsized":"2023-10-08T05:52:27.632428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:33.241545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.045511+0000","last_change":"2023-10-08T05:52:26.611424+0000","last_active":"2023-10-08T05:52:28.045511+0000","last_peered":"2023-10-08T05:52:28.045511+0000","last_clean":"2023-10-08T05:52:28.045511+0000","last_became_active":"2023-10-08T05:52:26.611147+0000","last_became_peered":"2023-10-08T05:52:26.611147+0000","last_unstale":"2023-10-08T05:52:28.045511+0000","last_undegraded":"2023-10-08T05:52:28.045511+0000","last_fullsized":"2023-10-08T05:52:28.045511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:03.695664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.049929+0000","last_change":"2023-10-08T05:52:26.610090+0000","last_active":"2023-10-08T05:52:28.049929+0000","last_peered":"2023-10-08T05:52:28.049929+0000","last_clean":"2023-10-08T05:52:28.049929+0000","last_became_active":"2023-10-08T05:52:26.609811+0000","last_became_peered":"2023-10-08T05:52:26.609811+0000","last_unstale":"2023-10-08T05:52:28.049929+0000","last_undegraded":"2023-10-08T05:52:28.049929+0000","last_fullsized":"2023-10-08T05:52:28.049929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:10.511258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612629+0000","last_change":"2023-10-08T05:52:26.604534+0000","last_active":"2023-10-08T05:52:27.612629+0000","last_peered":"2023-10-08T05:52:27.612629+0000","last_clean":"2023-10-08T05:52:27.612629+0000","last_became_active":"2023-10-08T05:52:26.604267+0000","last_became_peered":"2023-10-08T05:52:26.604267+0000","last_unstale":"2023-10-08T05:52:27.612629+0000","last_undegraded":"2023-10-08T05:52:27.612629+0000","last_fullsized":"2023-10-08T05:52:27.612629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:28.707400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613267+0000","last_change":"2023-10-08T05:52:26.603492+0000","last_active":"2023-10-08T05:52:27.613267+0000","last_peered":"2023-10-08T05:52:27.613267+0000","last_clean":"2023-10-08T05:52:27.613267+0000","last_became_active":"2023-10-08T05:52:26.603075+0000","last_became_peered":"2023-10-08T05:52:26.603075+0000","last_unstale":"2023-10-08T05:52:27.613267+0000","last_undegraded":"2023-10-08T05:52:27.613267+0000","last_fullsized":"2023-10-08T05:52:27.613267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:12:18.916804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.609859+0000","last_change":"2023-10-08T05:52:26.612241+0000","last_active":"2023-10-08T05:52:27.609859+0000","last_peered":"2023-10-08T05:52:27.609859+0000","last_clean":"2023-10-08T05:52:27.609859+0000","last_became_active":"2023-10-08T05:52:26.611918+0000","last_became_peered":"2023-10-08T05:52:26.611918+0000","last_unstale":"2023-10-08T05:52:27.609859+0000","last_undegraded":"2023-10-08T05:52:27.609859+0000","last_fullsized":"2023-10-08T05:52:27.609859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.124027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046626+0000","last_change":"2023-10-08T05:52:26.611297+0000","last_active":"2023-10-08T05:52:28.046626+0000","last_peered":"2023-10-08T05:52:28.046626+0000","last_clean":"2023-10-08T05:52:28.046626+0000","last_became_active":"2023-10-08T05:52:26.611003+0000","last_became_peered":"2023-10-08T05:52:26.611003+0000","last_unstale":"2023-10-08T05:52:28.046626+0000","last_undegraded":"2023-10-08T05:52:28.046626+0000","last_fullsized":"2023-10-08T05:52:28.046626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:18.945623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.609626+0000","last_change":"2023-10-08T05:52:26.621765+0000","last_active":"2023-10-08T05:52:27.609626+0000","last_peered":"2023-10-08T05:52:27.609626+0000","last_clean":"2023-10-08T05:52:27.609626+0000","last_became_active":"2023-10-08T05:52:26.621485+0000","last_became_peered":"2023-10-08T05:52:26.621485+0000","last_unstale":"2023-10-08T05:52:27.609626+0000","last_undegraded":"2023-10-08T05:52:27.609626+0000","last_fullsized":"2023-10-08T05:52:27.609626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:10.253744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.609501+0000","last_change":"2023-10-08T05:52:26.616889+0000","last_active":"2023-10-08T05:52:27.609501+0000","last_peered":"2023-10-08T05:52:27.609501+0000","last_clean":"2023-10-08T05:52:27.609501+0000","last_became_active":"2023-10-08T05:52:26.616530+0000","last_became_peered":"2023-10-08T05:52:26.616530+0000","last_unstale":"2023-10-08T05:52:27.609501+0000","last_undegraded":"2023-10-08T05:52:27.609501+0000","last_fullsized":"2023-10-08T05:52:27.609501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:26.165323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.632794+0000","last_change":"2023-10-08T05:52:26.626580+0000","last_active":"2023-10-08T05:52:27.632794+0000","last_peered":"2023-10-08T05:52:27.632794+0000","last_clean":"2023-10-08T05:52:27.632794+0000","last_became_active":"2023-10-08T05:52:26.625862+0000","last_became_peered":"2023-10-08T05:52:26.625862+0000","last_unstale":"2023-10-08T05:52:27.632794+0000","last_undegraded":"2023-10-08T05:52:27.632794+0000","last_fullsized":"2023-10-08T05:52:27.632794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:54:21.007655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046645+0000","last_change":"2023-10-08T05:52:26.614866+0000","last_active":"2023-10-08T05:52:28.046645+0000","last_peered":"2023-10-08T05:52:28.046645+0000","last_clean":"2023-10-08T05:52:28.046645+0000","last_became_active":"2023-10-08T05:52:26.614592+0000","last_became_peered":"2023-10-08T05:52:26.614592+0000","last_unstale":"2023-10-08T05:52:28.046645+0000","last_undegraded":"2023-10-08T05:52:28.046645+0000","last_fullsized":"2023-10-08T05:52:28.046645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.124027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046762+0000","last_change":"2023-10-08T05:52:26.615840+0000","last_active":"2023-10-08T05:52:28.046762+0000","last_peered":"2023-10-08T05:52:28.046762+0000","last_clean":"2023-10-08T05:52:28.046762+0000","last_became_active":"2023-10-08T05:52:26.615581+0000","last_became_peered":"2023-10-08T05:52:26.615581+0000","last_unstale":"2023-10-08T05:52:28.046762+0000","last_undegraded":"2023-10-08T05:52:28.046762+0000","last_fullsized":"2023-10-08T05:52:28.046762+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:44.005336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613818+0000","last_change":"2023-10-08T05:52:26.606304+0000","last_active":"2023-10-08T05:52:27.613818+0000","last_peered":"2023-10-08T05:52:27.613818+0000","last_clean":"2023-10-08T05:52:27.613818+0000","last_became_active":"2023-10-08T05:52:26.606009+0000","last_became_peered":"2023-10-08T05:52:26.606009+0000","last_unstale":"2023-10-08T05:52:27.613818+0000","last_undegraded":"2023-10-08T05:52:27.613818+0000","last_fullsized":"2023-10-08T05:52:27.613818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:46:36.156852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.048707+0000","last_change":"2023-10-08T05:52:26.611935+0000","last_active":"2023-10-08T05:52:28.048707+0000","last_peered":"2023-10-08T05:52:28.048707+0000","last_clean":"2023-10-08T05:52:28.048707+0000","last_became_active":"2023-10-08T05:52:26.611656+0000","last_became_peered":"2023-10-08T05:52:26.611656+0000","last_unstale":"2023-10-08T05:52:28.048707+0000","last_undegraded":"2023-10-08T05:52:28.048707+0000","last_fullsized":"2023-10-08T05:52:28.048707+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:42:13.213193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.632647+0000","last_change":"2023-10-08T05:52:26.630881+0000","last_active":"2023-10-08T05:52:27.632647+0000","last_peered":"2023-10-08T05:52:27.632647+0000","last_clean":"2023-10-08T05:52:27.632647+0000","last_became_active":"2023-10-08T05:52:26.630557+0000","last_became_peered":"2023-10-08T05:52:26.630557+0000","last_unstale":"2023-10-08T05:52:27.632647+0000","last_undegraded":"2023-10-08T05:52:27.632647+0000","last_fullsized":"2023-10-08T05:52:27.632647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:43.017103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046077+0000","last_change":"2023-10-08T05:52:26.620520+0000","last_active":"2023-10-08T05:52:28.046077+0000","last_peered":"2023-10-08T05:52:28.046077+0000","last_clean":"2023-10-08T05:52:28.046077+0000","last_became_active":"2023-10-08T05:52:26.620057+0000","last_became_peered":"2023-10-08T05:52:26.620057+0000","last_unstale":"2023-10-08T05:52:28.046077+0000","last_undegraded":"2023-10-08T05:52:28.046077+0000","last_fullsized":"2023-10-08T05:52:28.046077+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:53.711411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.611826+0000","last_change":"2023-10-08T05:52:26.598724+0000","last_active":"2023-10-08T05:52:27.611826+0000","last_peered":"2023-10-08T05:52:27.611826+0000","last_clean":"2023-10-08T05:52:27.611826+0000","last_became_active":"2023-10-08T05:52:26.597953+0000","last_became_peered":"2023-10-08T05:52:26.597953+0000","last_unstale":"2023-10-08T05:52:27.611826+0000","last_undegraded":"2023-10-08T05:52:27.611826+0000","last_fullsized":"2023-10-08T05:52:27.611826+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:27.918719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.047826+0000","last_change":"2023-10-08T05:52:26.601603+0000","last_active":"2023-10-08T05:52:28.047826+0000","last_peered":"2023-10-08T05:52:28.047826+0000","last_clean":"2023-10-08T05:52:28.047826+0000","last_became_active":"2023-10-08T05:52:26.601115+0000","last_became_peered":"2023-10-08T05:52:26.601115+0000","last_unstale":"2023-10-08T05:52:28.047826+0000","last_undegraded":"2023-10-08T05:52:28.047826+0000","last_fullsized":"2023-10-08T05:52:28.047826+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:24.840874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613920+0000","last_change":"2023-10-08T05:52:26.612015+0000","last_active":"2023-10-08T05:52:27.613920+0000","last_peered":"2023-10-08T05:52:27.613920+0000","last_clean":"2023-10-08T05:52:27.613920+0000","last_became_active":"2023-10-08T05:52:26.611746+0000","last_became_peered":"2023-10-08T05:52:26.611746+0000","last_unstale":"2023-10-08T05:52:27.613920+0000","last_undegraded":"2023-10-08T05:52:27.613920+0000","last_fullsized":"2023-10-08T05:52:27.613920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:01.282002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"89'4","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.631213+0000","last_change":"2023-10-08T05:52:25.616620+0000","last_active":"2023-10-08T05:52:26.631213+0000","last_peered":"2023-10-08T05:52:26.631213+0000","last_clean":"2023-10-08T05:52:26.631213+0000","last_became_active":"2023-10-08T05:52:25.615871+0000","last_became_peered":"2023-10-08T05:52:25.615871+0000","last_unstale":"2023-10-08T05:52:26.631213+0000","last_undegraded":"2023-10-08T05:52:26.631213+0000","last_fullsized":"2023-10-08T05:52:26.631213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:45.439387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.614004+0000","last_change":"2023-10-08T05:52:26.614195+0000","last_active":"2023-10-08T05:52:27.614004+0000","last_peered":"2023-10-08T05:52:27.614004+0000","last_clean":"2023-10-08T05:52:27.614004+0000","last_became_active":"2023-10-08T05:52:26.613908+0000","last_became_peered":"2023-10-08T05:52:26.613908+0000","last_unstale":"2023-10-08T05:52:27.614004+0000","last_undegraded":"2023-10-08T05:52:27.614004+0000","last_fullsized":"2023-10-08T05:52:27.614004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:17:08.671742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.630388+0000","last_change":"2023-10-08T05:52:25.615974+0000","last_active":"2023-10-08T05:52:26.630388+0000","last_peered":"2023-10-08T05:52:26.630388+0000","last_clean":"2023-10-08T05:52:26.630388+0000","last_became_active":"2023-10-08T05:52:25.615696+0000","last_became_peered":"2023-10-08T05:52:25.615696+0000","last_unstale":"2023-10-08T05:52:26.630388+0000","last_undegraded":"2023-10-08T05:52:26.630388+0000","last_fullsized":"2023-10-08T05:52:26.630388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:47.890265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.049215+0000","last_change":"2023-10-08T05:52:26.596720+0000","last_active":"2023-10-08T05:52:28.049215+0000","last_peered":"2023-10-08T05:52:28.049215+0000","last_clean":"2023-10-08T05:52:28.049215+0000","last_became_active":"2023-10-08T05:52:26.595091+0000","last_became_peered":"2023-10-08T05:52:26.595091+0000","last_unstale":"2023-10-08T05:52:28.049215+0000","last_undegraded":"2023-10-08T05:52:28.049215+0000","last_fullsized":"2023-10-08T05:52:28.049215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:01.346509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.623090+0000","last_change":"2023-10-08T05:52:25.613203+0000","last_active":"2023-10-08T05:52:26.623090+0000","last_peered":"2023-10-08T05:52:26.623090+0000","last_clean":"2023-10-08T05:52:26.623090+0000","last_became_active":"2023-10-08T05:52:25.612769+0000","last_became_peered":"2023-10-08T05:52:25.612769+0000","last_unstale":"2023-10-08T05:52:26.623090+0000","last_undegraded":"2023-10-08T05:52:26.623090+0000","last_fullsized":"2023-10-08T05:52:26.623090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:16.139641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.633365+0000","last_change":"2023-10-08T05:52:26.630623+0000","last_active":"2023-10-08T05:52:27.633365+0000","last_peered":"2023-10-08T05:52:27.633365+0000","last_clean":"2023-10-08T05:52:27.633365+0000","last_became_active":"2023-10-08T05:52:26.630230+0000","last_became_peered":"2023-10-08T05:52:26.630230+0000","last_unstale":"2023-10-08T05:52:27.633365+0000","last_undegraded":"2023-10-08T05:52:27.633365+0000","last_fullsized":"2023-10-08T05:52:27.633365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:26.091385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.624028+0000","last_change":"2023-10-08T05:52:25.609439+0000","last_active":"2023-10-08T05:52:26.624028+0000","last_peered":"2023-10-08T05:52:26.624028+0000","last_clean":"2023-10-08T05:52:26.624028+0000","last_became_active":"2023-10-08T05:52:25.609182+0000","last_became_peered":"2023-10-08T05:52:25.609182+0000","last_unstale":"2023-10-08T05:52:26.624028+0000","last_undegraded":"2023-10-08T05:52:26.624028+0000","last_fullsized":"2023-10-08T05:52:26.624028+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:07.972986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.045859+0000","last_change":"2023-10-08T05:52:26.613321+0000","last_active":"2023-10-08T05:52:28.045859+0000","last_peered":"2023-10-08T05:52:28.045859+0000","last_clean":"2023-10-08T05:52:28.045859+0000","last_became_active":"2023-10-08T05:52:26.613042+0000","last_became_peered":"2023-10-08T05:52:26.613042+0000","last_unstale":"2023-10-08T05:52:28.045859+0000","last_undegraded":"2023-10-08T05:52:28.045859+0000","last_fullsized":"2023-10-08T05:52:28.045859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:38.210900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.624711+0000","last_change":"2023-10-08T05:52:25.602006+0000","last_active":"2023-10-08T05:52:26.624711+0000","last_peered":"2023-10-08T05:52:26.624711+0000","last_clean":"2023-10-08T05:52:26.624711+0000","last_became_active":"2023-10-08T05:52:25.601714+0000","last_became_peered":"2023-10-08T05:52:25.601714+0000","last_unstale":"2023-10-08T05:52:26.624711+0000","last_undegraded":"2023-10-08T05:52:26.624711+0000","last_fullsized":"2023-10-08T05:52:26.624711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:31.639423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.047432+0000","last_change":"2023-10-08T05:52:26.605222+0000","last_active":"2023-10-08T05:52:28.047432+0000","last_peered":"2023-10-08T05:52:28.047432+0000","last_clean":"2023-10-08T05:52:28.047432+0000","last_became_active":"2023-10-08T05:52:26.604900+0000","last_became_peered":"2023-10-08T05:52:26.604900+0000","last_unstale":"2023-10-08T05:52:28.047432+0000","last_undegraded":"2023-10-08T05:52:28.047432+0000","last_fullsized":"2023-10-08T05:52:28.047432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:45.145436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.622381+0000","last_change":"2023-10-08T05:52:25.602167+0000","last_active":"2023-10-08T05:52:26.622381+0000","last_peered":"2023-10-08T05:52:26.622381+0000","last_clean":"2023-10-08T05:52:26.622381+0000","last_became_active":"2023-10-08T05:52:25.601901+0000","last_became_peered":"2023-10-08T05:52:25.601901+0000","last_unstale":"2023-10-08T05:52:26.622381+0000","last_undegraded":"2023-10-08T05:52:26.622381+0000","last_fullsized":"2023-10-08T05:52:26.622381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:25.075678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612521+0000","last_change":"2023-10-08T05:52:26.604564+0000","last_active":"2023-10-08T05:52:27.612521+0000","last_peered":"2023-10-08T05:52:27.612521+0000","last_clean":"2023-10-08T05:52:27.612521+0000","last_became_active":"2023-10-08T05:52:26.604285+0000","last_became_peered":"2023-10-08T05:52:26.604285+0000","last_unstale":"2023-10-08T05:52:27.612521+0000","last_undegraded":"2023-10-08T05:52:27.612521+0000","last_fullsized":"2023-10-08T05:52:27.612521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:12:12.176743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.588261+0000","last_change":"2023-10-08T05:52:25.613579+0000","last_active":"2023-10-08T05:52:26.588261+0000","last_peered":"2023-10-08T05:52:26.588261+0000","last_clean":"2023-10-08T05:52:26.588261+0000","last_became_active":"2023-10-08T05:52:25.613286+0000","last_became_peered":"2023-10-08T05:52:25.613286+0000","last_unstale":"2023-10-08T05:52:26.588261+0000","last_undegraded":"2023-10-08T05:52:26.588261+0000","last_fullsized":"2023-10-08T05:52:26.588261+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:32.225838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.609775+0000","last_change":"2023-10-08T05:52:26.602442+0000","last_active":"2023-10-08T05:52:27.609775+0000","last_peered":"2023-10-08T05:52:27.609775+0000","last_clean":"2023-10-08T05:52:27.609775+0000","last_became_active":"2023-10-08T05:52:26.602110+0000","last_became_peered":"2023-10-08T05:52:26.602110+0000","last_unstale":"2023-10-08T05:52:27.609775+0000","last_undegraded":"2023-10-08T05:52:27.609775+0000","last_fullsized":"2023-10-08T05:52:27.609775+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:18.945623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.592621+0000","last_change":"2023-10-08T05:52:25.618073+0000","last_active":"2023-10-08T05:52:26.592621+0000","last_peered":"2023-10-08T05:52:26.592621+0000","last_clean":"2023-10-08T05:52:26.592621+0000","last_became_active":"2023-10-08T05:52:25.617792+0000","last_became_peered":"2023-10-08T05:52:25.617792+0000","last_unstale":"2023-10-08T05:52:26.592621+0000","last_undegraded":"2023-10-08T05:52:26.592621+0000","last_fullsized":"2023-10-08T05:52:26.592621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:58.514719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.049020+0000","last_change":"2023-10-08T05:52:26.614707+0000","last_active":"2023-10-08T05:52:28.049020+0000","last_peered":"2023-10-08T05:52:28.049020+0000","last_clean":"2023-10-08T05:52:28.049020+0000","last_became_active":"2023-10-08T05:52:26.614374+0000","last_became_peered":"2023-10-08T05:52:26.614374+0000","last_unstale":"2023-10-08T05:52:28.049020+0000","last_undegraded":"2023-10-08T05:52:28.049020+0000","last_fullsized":"2023-10-08T05:52:28.049020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:59.912311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.624630+0000","last_change":"2023-10-08T05:52:25.617521+0000","last_active":"2023-10-08T05:52:26.624630+0000","last_peered":"2023-10-08T05:52:26.624630+0000","last_clean":"2023-10-08T05:52:26.624630+0000","last_became_active":"2023-10-08T05:52:25.617269+0000","last_became_peered":"2023-10-08T05:52:25.617269+0000","last_unstale":"2023-10-08T05:52:26.624630+0000","last_undegraded":"2023-10-08T05:52:26.624630+0000","last_fullsized":"2023-10-08T05:52:26.624630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:36:51.312170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.610091+0000","last_change":"2023-10-08T05:52:26.607854+0000","last_active":"2023-10-08T05:52:27.610091+0000","last_peered":"2023-10-08T05:52:27.610091+0000","last_clean":"2023-10-08T05:52:27.610091+0000","last_became_active":"2023-10-08T05:52:26.607558+0000","last_became_peered":"2023-10-08T05:52:26.607558+0000","last_unstale":"2023-10-08T05:52:27.610091+0000","last_undegraded":"2023-10-08T05:52:27.610091+0000","last_fullsized":"2023-10-08T05:52:27.610091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:15.670901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.584660+0000","last_change":"2023-10-08T05:52:25.610640+0000","last_active":"2023-10-08T05:52:26.584660+0000","last_peered":"2023-10-08T05:52:26.584660+0000","last_clean":"2023-10-08T05:52:26.584660+0000","last_became_active":"2023-10-08T05:52:25.610375+0000","last_became_peered":"2023-10-08T05:52:25.610375+0000","last_unstale":"2023-10-08T05:52:26.584660+0000","last_undegraded":"2023-10-08T05:52:26.584660+0000","last_fullsized":"2023-10-08T05:52:26.584660+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:31.358719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612176+0000","last_change":"2023-10-08T05:52:26.606295+0000","last_active":"2023-10-08T05:52:27.612176+0000","last_peered":"2023-10-08T05:52:27.612176+0000","last_clean":"2023-10-08T05:52:27.612176+0000","last_became_active":"2023-10-08T05:52:26.605212+0000","last_became_peered":"2023-10-08T05:52:26.605212+0000","last_unstale":"2023-10-08T05:52:27.612176+0000","last_undegraded":"2023-10-08T05:52:27.612176+0000","last_fullsized":"2023-10-08T05:52:27.612176+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:25.764209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"89'3","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.636352+0000","last_change":"2023-10-08T05:52:25.619025+0000","last_active":"2023-10-08T05:52:26.636352+0000","last_peered":"2023-10-08T05:52:26.636352+0000","last_clean":"2023-10-08T05:52:26.636352+0000","last_became_active":"2023-10-08T05:52:25.618782+0000","last_became_peered":"2023-10-08T05:52:25.618782+0000","last_unstale":"2023-10-08T05:52:26.636352+0000","last_undegraded":"2023-10-08T05:52:26.636352+0000","last_fullsized":"2023-10-08T05:52:26.636352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:50:07.327419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613788+0000","last_change":"2023-10-08T05:52:26.600651+0000","last_active":"2023-10-08T05:52:27.613788+0000","last_peered":"2023-10-08T05:52:27.613788+0000","last_clean":"2023-10-08T05:52:27.613788+0000","last_became_active":"2023-10-08T05:52:26.593844+0000","last_became_peered":"2023-10-08T05:52:26.593844+0000","last_unstale":"2023-10-08T05:52:27.613788+0000","last_undegraded":"2023-10-08T05:52:27.613788+0000","last_fullsized":"2023-10-08T05:52:27.613788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:58:41.407210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.627996+0000","last_change":"2023-10-08T05:52:25.609882+0000","last_active":"2023-10-08T05:52:26.627996+0000","last_peered":"2023-10-08T05:52:26.627996+0000","last_clean":"2023-10-08T05:52:26.627996+0000","last_became_active":"2023-10-08T05:52:25.608449+0000","last_became_peered":"2023-10-08T05:52:25.608449+0000","last_unstale":"2023-10-08T05:52:26.627996+0000","last_undegraded":"2023-10-08T05:52:26.627996+0000","last_fullsized":"2023-10-08T05:52:26.627996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:14.303833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.611498+0000","last_change":"2023-10-08T05:52:26.622980+0000","last_active":"2023-10-08T05:52:27.611498+0000","last_peered":"2023-10-08T05:52:27.611498+0000","last_clean":"2023-10-08T05:52:27.611498+0000","last_became_active":"2023-10-08T05:52:26.622710+0000","last_became_peered":"2023-10-08T05:52:26.622710+0000","last_unstale":"2023-10-08T05:52:27.611498+0000","last_undegraded":"2023-10-08T05:52:27.611498+0000","last_fullsized":"2023-10-08T05:52:27.611498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:54.741577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.635631+0000","last_change":"2023-10-08T05:52:25.617860+0000","last_active":"2023-10-08T05:52:26.635631+0000","last_peered":"2023-10-08T05:52:26.635631+0000","last_clean":"2023-10-08T05:52:26.635631+0000","last_became_active":"2023-10-08T05:52:25.617594+0000","last_became_peered":"2023-10-08T05:52:25.617594+0000","last_unstale":"2023-10-08T05:52:26.635631+0000","last_undegraded":"2023-10-08T05:52:26.635631+0000","last_fullsized":"2023-10-08T05:52:26.635631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:51.826639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046030+0000","last_change":"2023-10-08T05:52:26.613191+0000","last_active":"2023-10-08T05:52:28.046030+0000","last_peered":"2023-10-08T05:52:28.046030+0000","last_clean":"2023-10-08T05:52:28.046030+0000","last_became_active":"2023-10-08T05:52:26.612899+0000","last_became_peered":"2023-10-08T05:52:26.612899+0000","last_unstale":"2023-10-08T05:52:28.046030+0000","last_undegraded":"2023-10-08T05:52:28.046030+0000","last_fullsized":"2023-10-08T05:52:28.046030+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:12.299867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.598277+0000","last_change":"2023-10-08T05:52:25.609132+0000","last_active":"2023-10-08T05:52:26.598277+0000","last_peered":"2023-10-08T05:52:26.598277+0000","last_clean":"2023-10-08T05:52:26.598277+0000","last_became_active":"2023-10-08T05:52:25.608752+0000","last_became_peered":"2023-10-08T05:52:25.608752+0000","last_unstale":"2023-10-08T05:52:26.598277+0000","last_undegraded":"2023-10-08T05:52:26.598277+0000","last_fullsized":"2023-10-08T05:52:26.598277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:28:11.848537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612713+0000","last_change":"2023-10-08T05:52:26.602288+0000","last_active":"2023-10-08T05:52:27.612713+0000","last_peered":"2023-10-08T05:52:27.612713+0000","last_clean":"2023-10-08T05:52:27.612713+0000","last_became_active":"2023-10-08T05:52:26.601862+0000","last_became_peered":"2023-10-08T05:52:26.601862+0000","last_unstale":"2023-10-08T05:52:27.612713+0000","last_undegraded":"2023-10-08T05:52:27.612713+0000","last_fullsized":"2023-10-08T05:52:27.612713+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:26.008862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.624855+0000","last_change":"2023-10-08T05:52:25.618551+0000","last_active":"2023-10-08T05:52:26.624855+0000","last_peered":"2023-10-08T05:52:26.624855+0000","last_clean":"2023-10-08T05:52:26.624855+0000","last_became_active":"2023-10-08T05:52:25.618294+0000","last_became_peered":"2023-10-08T05:52:25.618294+0000","last_unstale":"2023-10-08T05:52:26.624855+0000","last_undegraded":"2023-10-08T05:52:26.624855+0000","last_fullsized":"2023-10-08T05:52:26.624855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:28:11.848537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.048162+0000","last_change":"2023-10-08T05:52:26.608848+0000","last_active":"2023-10-08T05:52:28.048162+0000","last_peered":"2023-10-08T05:52:28.048162+0000","last_clean":"2023-10-08T05:52:28.048162+0000","last_became_active":"2023-10-08T05:52:26.607816+0000","last_became_peered":"2023-10-08T05:52:26.607816+0000","last_unstale":"2023-10-08T05:52:28.048162+0000","last_undegraded":"2023-10-08T05:52:28.048162+0000","last_fullsized":"2023-10-08T05:52:28.048162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:52:22.172494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.633455+0000","last_change":"2023-10-08T05:52:25.617851+0000","last_active":"2023-10-08T05:52:26.633455+0000","last_peered":"2023-10-08T05:52:26.633455+0000","last_clean":"2023-10-08T05:52:26.633455+0000","last_became_active":"2023-10-08T05:52:25.617547+0000","last_became_peered":"2023-10-08T05:52:25.617547+0000","last_unstale":"2023-10-08T05:52:26.633455+0000","last_undegraded":"2023-10-08T05:52:26.633455+0000","last_fullsized":"2023-10-08T05:52:26.633455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:29.256817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.633242+0000","last_change":"2023-10-08T05:52:26.630834+0000","last_active":"2023-10-08T05:52:27.633242+0000","last_peered":"2023-10-08T05:52:27.633242+0000","last_clean":"2023-10-08T05:52:27.633242+0000","last_became_active":"2023-10-08T05:52:26.630260+0000","last_became_peered":"2023-10-08T05:52:26.630260+0000","last_unstale":"2023-10-08T05:52:27.633242+0000","last_undegraded":"2023-10-08T05:52:27.633242+0000","last_fullsized":"2023-10-08T05:52:27.633242+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:12.299867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.623910+0000","last_change":"2023-10-08T05:52:25.615489+0000","last_active":"2023-10-08T05:52:26.623910+0000","last_peered":"2023-10-08T05:52:26.623910+0000","last_clean":"2023-10-08T05:52:26.623910+0000","last_became_active":"2023-10-08T05:52:25.615202+0000","last_became_peered":"2023-10-08T05:52:25.615202+0000","last_unstale":"2023-10-08T05:52:26.623910+0000","last_undegraded":"2023-10-08T05:52:26.623910+0000","last_fullsized":"2023-10-08T05:52:26.623910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:31.639423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613460+0000","last_change":"2023-10-08T05:52:26.601176+0000","last_active":"2023-10-08T05:52:27.613460+0000","last_peered":"2023-10-08T05:52:27.613460+0000","last_clean":"2023-10-08T05:52:27.613460+0000","last_became_active":"2023-10-08T05:52:26.600520+0000","last_became_peered":"2023-10-08T05:52:26.600520+0000","last_unstale":"2023-10-08T05:52:27.613460+0000","last_undegraded":"2023-10-08T05:52:27.613460+0000","last_fullsized":"2023-10-08T05:52:27.613460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:30.612508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.635622+0000","last_change":"2023-10-08T05:52:25.620400+0000","last_active":"2023-10-08T05:52:26.635622+0000","last_peered":"2023-10-08T05:52:26.635622+0000","last_clean":"2023-10-08T05:52:26.635622+0000","last_became_active":"2023-10-08T05:52:25.620067+0000","last_became_peered":"2023-10-08T05:52:25.620067+0000","last_unstale":"2023-10-08T05:52:26.635622+0000","last_undegraded":"2023-10-08T05:52:26.635622+0000","last_fullsized":"2023-10-08T05:52:26.635622+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:09.438710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613404+0000","last_change":"2023-10-08T05:52:26.615102+0000","last_active":"2023-10-08T05:52:27.613404+0000","last_peered":"2023-10-08T05:52:27.613404+0000","last_clean":"2023-10-08T05:52:27.613404+0000","last_became_active":"2023-10-08T05:52:26.614801+0000","last_became_peered":"2023-10-08T05:52:26.614801+0000","last_unstale":"2023-10-08T05:52:27.613404+0000","last_undegraded":"2023-10-08T05:52:27.613404+0000","last_fullsized":"2023-10-08T05:52:27.613404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:05.974014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.636114+0000","last_change":"2023-10-08T05:52:25.620173+0000","last_active":"2023-10-08T05:52:26.636114+0000","last_peered":"2023-10-08T05:52:26.636114+0000","last_clean":"2023-10-08T05:52:26.636114+0000","last_became_active":"2023-10-08T05:52:25.619012+0000","last_became_peered":"2023-10-08T05:52:25.619012+0000","last_unstale":"2023-10-08T05:52:26.636114+0000","last_undegraded":"2023-10-08T05:52:26.636114+0000","last_fullsized":"2023-10-08T05:52:26.636114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:40.039160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.614191+0000","last_change":"2023-10-08T05:52:26.606637+0000","last_active":"2023-10-08T05:52:27.614191+0000","last_peered":"2023-10-08T05:52:27.614191+0000","last_clean":"2023-10-08T05:52:27.614191+0000","last_became_active":"2023-10-08T05:52:26.605349+0000","last_became_peered":"2023-10-08T05:52:26.605349+0000","last_unstale":"2023-10-08T05:52:27.614191+0000","last_undegraded":"2023-10-08T05:52:27.614191+0000","last_fullsized":"2023-10-08T05:52:27.614191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:17.359233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.590043+0000","last_change":"2023-10-08T05:52:25.613656+0000","last_active":"2023-10-08T05:52:26.590043+0000","last_peered":"2023-10-08T05:52:26.590043+0000","last_clean":"2023-10-08T05:52:26.590043+0000","last_became_active":"2023-10-08T05:52:25.613319+0000","last_became_peered":"2023-10-08T05:52:25.613319+0000","last_unstale":"2023-10-08T05:52:26.590043+0000","last_undegraded":"2023-10-08T05:52:26.590043+0000","last_fullsized":"2023-10-08T05:52:26.590043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:11.284159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.047936+0000","last_change":"2023-10-08T05:52:26.596496+0000","last_active":"2023-10-08T05:52:28.047936+0000","last_peered":"2023-10-08T05:52:28.047936+0000","last_clean":"2023-10-08T05:52:28.047936+0000","last_became_active":"2023-10-08T05:52:26.595929+0000","last_became_peered":"2023-10-08T05:52:26.595929+0000","last_unstale":"2023-10-08T05:52:28.047936+0000","last_undegraded":"2023-10-08T05:52:28.047936+0000","last_fullsized":"2023-10-08T05:52:28.047936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:18:16.856795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.623985+0000","last_change":"2023-10-08T05:52:25.609865+0000","last_active":"2023-10-08T05:52:26.623985+0000","last_peered":"2023-10-08T05:52:26.623985+0000","last_clean":"2023-10-08T05:52:26.623985+0000","last_became_active":"2023-10-08T05:52:25.609580+0000","last_became_peered":"2023-10-08T05:52:25.609580+0000","last_unstale":"2023-10-08T05:52:26.623985+0000","last_undegraded":"2023-10-08T05:52:26.623985+0000","last_fullsized":"2023-10-08T05:52:26.623985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:42.160986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046180+0000","last_change":"2023-10-08T05:52:26.606852+0000","last_active":"2023-10-08T05:52:28.046180+0000","last_peered":"2023-10-08T05:52:28.046180+0000","last_clean":"2023-10-08T05:52:28.046180+0000","last_became_active":"2023-10-08T05:52:26.606540+0000","last_became_peered":"2023-10-08T05:52:26.606540+0000","last_unstale":"2023-10-08T05:52:28.046180+0000","last_undegraded":"2023-10-08T05:52:28.046180+0000","last_fullsized":"2023-10-08T05:52:28.046180+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:26.091385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.598610+0000","last_change":"2023-10-08T05:52:25.609029+0000","last_active":"2023-10-08T05:52:26.598610+0000","last_peered":"2023-10-08T05:52:26.598610+0000","last_clean":"2023-10-08T05:52:26.598610+0000","last_became_active":"2023-10-08T05:52:25.608342+0000","last_became_peered":"2023-10-08T05:52:25.608342+0000","last_unstale":"2023-10-08T05:52:26.598610+0000","last_undegraded":"2023-10-08T05:52:26.598610+0000","last_fullsized":"2023-10-08T05:52:26.598610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:32.225838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612900+0000","last_change":"2023-10-08T05:52:26.604408+0000","last_active":"2023-10-08T05:52:27.612900+0000","last_peered":"2023-10-08T05:52:27.612900+0000","last_clean":"2023-10-08T05:52:27.612900+0000","last_became_active":"2023-10-08T05:52:26.604112+0000","last_became_peered":"2023-10-08T05:52:26.604112+0000","last_unstale":"2023-10-08T05:52:27.612900+0000","last_undegraded":"2023-10-08T05:52:27.612900+0000","last_fullsized":"2023-10-08T05:52:27.612900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:55.747441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.577587+0000","last_change":"2023-10-08T05:52:25.615086+0000","last_active":"2023-10-08T05:52:26.577587+0000","last_peered":"2023-10-08T05:52:26.577587+0000","last_clean":"2023-10-08T05:52:26.577587+0000","last_became_active":"2023-10-08T05:52:25.614831+0000","last_became_peered":"2023-10-08T05:52:25.614831+0000","last_unstale":"2023-10-08T05:52:26.577587+0000","last_undegraded":"2023-10-08T05:52:26.577587+0000","last_fullsized":"2023-10-08T05:52:26.577587+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:22.367245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.048964+0000","last_change":"2023-10-08T05:52:26.598757+0000","last_active":"2023-10-08T05:52:28.048964+0000","last_peered":"2023-10-08T05:52:28.048964+0000","last_clean":"2023-10-08T05:52:28.048964+0000","last_became_active":"2023-10-08T05:52:26.598272+0000","last_became_peered":"2023-10-08T05:52:26.598272+0000","last_unstale":"2023-10-08T05:52:28.048964+0000","last_undegraded":"2023-10-08T05:52:28.048964+0000","last_fullsized":"2023-10-08T05:52:28.048964+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:09.362614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.628873+0000","last_change":"2023-10-08T05:52:25.617759+0000","last_active":"2023-10-08T05:52:26.628873+0000","last_peered":"2023-10-08T05:52:26.628873+0000","last_clean":"2023-10-08T05:52:26.628873+0000","last_became_active":"2023-10-08T05:52:25.617463+0000","last_became_peered":"2023-10-08T05:52:25.617463+0000","last_unstale":"2023-10-08T05:52:26.628873+0000","last_undegraded":"2023-10-08T05:52:26.628873+0000","last_fullsized":"2023-10-08T05:52:26.628873+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:41.277667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046632+0000","last_change":"2023-10-08T05:52:26.605885+0000","last_active":"2023-10-08T05:52:28.046632+0000","last_peered":"2023-10-08T05:52:28.046632+0000","last_clean":"2023-10-08T05:52:28.046632+0000","last_became_active":"2023-10-08T05:52:26.605612+0000","last_became_peered":"2023-10-08T05:52:26.605612+0000","last_unstale":"2023-10-08T05:52:28.046632+0000","last_undegraded":"2023-10-08T05:52:28.046632+0000","last_fullsized":"2023-10-08T05:52:28.046632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:32.304664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.600407+0000","last_change":"2023-10-08T05:52:25.612338+0000","last_active":"2023-10-08T05:52:26.600407+0000","last_peered":"2023-10-08T05:52:26.600407+0000","last_clean":"2023-10-08T05:52:26.600407+0000","last_became_active":"2023-10-08T05:52:25.611999+0000","last_became_peered":"2023-10-08T05:52:25.611999+0000","last_unstale":"2023-10-08T05:52:26.600407+0000","last_undegraded":"2023-10-08T05:52:26.600407+0000","last_fullsized":"2023-10-08T05:52:26.600407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:54:19.991948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046581+0000","last_change":"2023-10-08T05:52:26.612721+0000","last_active":"2023-10-08T05:52:28.046581+0000","last_peered":"2023-10-08T05:52:28.046581+0000","last_clean":"2023-10-08T05:52:28.046581+0000","last_became_active":"2023-10-08T05:52:26.612404+0000","last_became_peered":"2023-10-08T05:52:26.612404+0000","last_unstale":"2023-10-08T05:52:28.046581+0000","last_undegraded":"2023-10-08T05:52:28.046581+0000","last_fullsized":"2023-10-08T05:52:28.046581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:10.253744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.578269+0000","last_change":"2023-10-08T05:52:25.613025+0000","last_active":"2023-10-08T05:52:26.578269+0000","last_peered":"2023-10-08T05:52:26.578269+0000","last_clean":"2023-10-08T05:52:26.578269+0000","last_became_active":"2023-10-08T05:52:25.612435+0000","last_became_peered":"2023-10-08T05:52:25.612435+0000","last_unstale":"2023-10-08T05:52:26.578269+0000","last_undegraded":"2023-10-08T05:52:26.578269+0000","last_fullsized":"2023-10-08T05:52:26.578269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:27.932459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.614294+0000","last_change":"2023-10-08T05:52:26.607665+0000","last_active":"2023-10-08T05:52:27.614294+0000","last_peered":"2023-10-08T05:52:27.614294+0000","last_clean":"2023-10-08T05:52:27.614294+0000","last_became_active":"2023-10-08T05:52:26.607345+0000","last_became_peered":"2023-10-08T05:52:26.607345+0000","last_unstale":"2023-10-08T05:52:27.614294+0000","last_undegraded":"2023-10-08T05:52:27.614294+0000","last_fullsized":"2023-10-08T05:52:27.614294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:53.632360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.598649+0000","last_change":"2023-10-08T05:52:25.608395+0000","last_active":"2023-10-08T05:52:26.598649+0000","last_peered":"2023-10-08T05:52:26.598649+0000","last_clean":"2023-10-08T05:52:26.598649+0000","last_became_active":"2023-10-08T05:52:25.608087+0000","last_became_peered":"2023-10-08T05:52:25.608087+0000","last_unstale":"2023-10-08T05:52:26.598649+0000","last_undegraded":"2023-10-08T05:52:26.598649+0000","last_fullsized":"2023-10-08T05:52:26.598649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:58:40.391503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612177+0000","last_change":"2023-10-08T05:52:26.612662+0000","last_active":"2023-10-08T05:52:27.612177+0000","last_peered":"2023-10-08T05:52:27.612177+0000","last_clean":"2023-10-08T05:52:27.612177+0000","last_became_active":"2023-10-08T05:52:26.612390+0000","last_became_peered":"2023-10-08T05:52:26.612390+0000","last_unstale":"2023-10-08T05:52:27.612177+0000","last_undegraded":"2023-10-08T05:52:27.612177+0000","last_fullsized":"2023-10-08T05:52:27.612177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:38.608854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.623792+0000","last_change":"2023-10-08T05:52:25.618059+0000","last_active":"2023-10-08T05:52:26.623792+0000","last_peered":"2023-10-08T05:52:26.623792+0000","last_clean":"2023-10-08T05:52:26.623792+0000","last_became_active":"2023-10-08T05:52:25.617759+0000","last_became_peered":"2023-10-08T05:52:25.617759+0000","last_unstale":"2023-10-08T05:52:26.623792+0000","last_undegraded":"2023-10-08T05:52:26.623792+0000","last_fullsized":"2023-10-08T05:52:26.623792+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:59.721424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.611943+0000","last_change":"2023-10-08T05:52:26.613844+0000","last_active":"2023-10-08T05:52:27.611943+0000","last_peered":"2023-10-08T05:52:27.611943+0000","last_clean":"2023-10-08T05:52:27.611943+0000","last_became_active":"2023-10-08T05:52:26.613544+0000","last_became_peered":"2023-10-08T05:52:26.613544+0000","last_unstale":"2023-10-08T05:52:27.611943+0000","last_undegraded":"2023-10-08T05:52:27.611943+0000","last_fullsized":"2023-10-08T05:52:27.611943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:05.774388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.591216+0000","last_change":"2023-10-08T05:52:25.608974+0000","last_active":"2023-10-08T05:52:26.591216+0000","last_peered":"2023-10-08T05:52:26.591216+0000","last_clean":"2023-10-08T05:52:26.591216+0000","last_became_active":"2023-10-08T05:52:25.608723+0000","last_became_peered":"2023-10-08T05:52:25.608723+0000","last_unstale":"2023-10-08T05:52:26.591216+0000","last_undegraded":"2023-10-08T05:52:26.591216+0000","last_fullsized":"2023-10-08T05:52:26.591216+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:47.066046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.048624+0000","last_change":"2023-10-08T05:52:26.605412+0000","last_active":"2023-10-08T05:52:28.048624+0000","last_peered":"2023-10-08T05:52:28.048624+0000","last_clean":"2023-10-08T05:52:28.048624+0000","last_became_active":"2023-10-08T05:52:26.605096+0000","last_became_peered":"2023-10-08T05:52:26.605096+0000","last_unstale":"2023-10-08T05:52:28.048624+0000","last_undegraded":"2023-10-08T05:52:28.048624+0000","last_fullsized":"2023-10-08T05:52:28.048624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:16.473131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.588720+0000","last_change":"2023-10-08T05:52:25.613350+0000","last_active":"2023-10-08T05:52:26.588720+0000","last_peered":"2023-10-08T05:52:26.588720+0000","last_clean":"2023-10-08T05:52:26.588720+0000","last_became_active":"2023-10-08T05:52:25.613088+0000","last_became_peered":"2023-10-08T05:52:25.613088+0000","last_unstale":"2023-10-08T05:52:26.588720+0000","last_undegraded":"2023-10-08T05:52:26.588720+0000","last_fullsized":"2023-10-08T05:52:26.588720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:42.001395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046040+0000","last_change":"2023-10-08T05:52:26.610157+0000","last_active":"2023-10-08T05:52:28.046040+0000","last_peered":"2023-10-08T05:52:28.046040+0000","last_clean":"2023-10-08T05:52:28.046040+0000","last_became_active":"2023-10-08T05:52:26.609890+0000","last_became_peered":"2023-10-08T05:52:26.609890+0000","last_unstale":"2023-10-08T05:52:28.046040+0000","last_undegraded":"2023-10-08T05:52:28.046040+0000","last_fullsized":"2023-10-08T05:52:28.046040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:12.964611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.597188+0000","last_change":"2023-10-08T05:52:25.612249+0000","last_active":"2023-10-08T05:52:26.597188+0000","last_peered":"2023-10-08T05:52:26.597188+0000","last_clean":"2023-10-08T05:52:26.597188+0000","last_became_active":"2023-10-08T05:52:25.611988+0000","last_became_peered":"2023-10-08T05:52:25.611988+0000","last_unstale":"2023-10-08T05:52:26.597188+0000","last_undegraded":"2023-10-08T05:52:26.597188+0000","last_fullsized":"2023-10-08T05:52:26.597188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:04.958307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613792+0000","last_change":"2023-10-08T05:52:26.611381+0000","last_active":"2023-10-08T05:52:27.613792+0000","last_peered":"2023-10-08T05:52:27.613792+0000","last_clean":"2023-10-08T05:52:27.613792+0000","last_became_active":"2023-10-08T05:52:26.611115+0000","last_became_peered":"2023-10-08T05:52:26.611115+0000","last_unstale":"2023-10-08T05:52:27.613792+0000","last_undegraded":"2023-10-08T05:52:27.613792+0000","last_fullsized":"2023-10-08T05:52:27.613792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:14.086252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.586840+0000","last_change":"2023-10-08T05:52:25.598181+0000","last_active":"2023-10-08T05:52:26.586840+0000","last_peered":"2023-10-08T05:52:26.586840+0000","last_clean":"2023-10-08T05:52:26.586840+0000","last_became_active":"2023-10-08T05:52:25.597570+0000","last_became_peered":"2023-10-08T05:52:25.597570+0000","last_unstale":"2023-10-08T05:52:26.586840+0000","last_undegraded":"2023-10-08T05:52:26.586840+0000","last_fullsized":"2023-10-08T05:52:26.586840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:36:51.312170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"78'161","reported_seq":393,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.618922+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:52:26.618922+0000","last_peered":"2023-10-08T05:52:26.618922+0000","last_clean":"2023-10-08T05:52:26.618922+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:52:26.618922+0000","last_undegraded":"2023-10-08T05:52:26.618922+0000","last_fullsized":"2023-10-08T05:52:26.618922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575046,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59220,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312620,"statfs":{"total":96636764160,"available":96576122880,"internally_reserved":0,"allocated":1703936,"data_stored":1142963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1859999999999999,"15min":1.2030000000000001},"min":{"1min":0.77800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.2360000000000002},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.196,"15min":1.2030000000000001},"min":{"1min":0.754,"5min":0.44,"15min":0.44},"max":{"1min":1.6890000000000001,"5min":1.726,"15min":1.726},"last":1.4790000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.206,"15min":1.23},"min":{"1min":0.84899999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5509999999999999,"5min":1.6950000000000001,"15min":1.972},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.2749999999999999,"15min":1.2629999999999999},"min":{"1min":0.95699999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.736,"5min":1.7689999999999999,"15min":1.9379999999999999},"last":1.2669999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.2330000000000001,"15min":1.2210000000000001},"min":{"1min":0.90000000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.585,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.3260000000000001,"15min":1.329},"min":{"1min":0.91700000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.609,"5min":1.752,"15min":2.3849999999999998},"last":1.121}]},{"osd":3,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2589999999999999,"15min":1.2769999999999999},"min":{"1min":0.92800000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.639,"5min":1.8799999999999999,"15min":2.2919999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.3360000000000001,"15min":1.365},"min":{"1min":0.93999999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6930000000000001,"5min":1.8200000000000001,"15min":2.5579999999999998},"last":1.4419999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.294,"15min":1.2869999999999999},"min":{"1min":0.97499999999999998,"5min":0.504,"15min":0.504},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.4529999999999998},"last":1.167},{"interface":"front","average":{"1min":1.323,"5min":1.3069999999999999,"15min":1.3340000000000001},"min":{"1min":0.90700000000000003,"5min":0.63600000000000001,"15min":0.46800000000000003},"max":{"1min":1.796,"5min":2.0699999999999998,"15min":2.677},"last":1.4019999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3600000000000001,"15min":1.399},"min":{"1min":0.69499999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":3.3319999999999999},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.425,"15min":1.4279999999999999},"min":{"1min":1.042,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":2.726},"last":1.0669999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.5149999999999999,"15min":1.5249999999999999},"min":{"1min":1.2569999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0680000000000001,"5min":2.2000000000000002,"15min":3.5249999999999999},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.446,"15min":1.46},"min":{"1min":1.0589999999999999,"5min":0.57799999999999996,"15min":0.52000000000000002},"max":{"1min":1.984,"5min":2.0150000000000001,"15min":2.4700000000000002},"last":1.6359999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1097728,"data_stored":551853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.286,"15min":1.2609999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.9119999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.2509999999999999,"15min":1.2969999999999999},"min":{"1min":0.67400000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.8100000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.125,"15min":1.1579999999999999},"min":{"1min":0.73799999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6120000000000001,"5min":1.7789999999999999,"15min":1.9530000000000001},"last":1.393},{"interface":"front","average":{"1min":1.119,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.5820000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1479999999999999,"15min":1.1839999999999999},"min":{"1min":0.86399999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.516,"5min":1.75,"15min":1.796},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.222,"15min":1.2589999999999999},"min":{"1min":0.92700000000000005,"5min":0.78300000000000003,"15min":0.58699999999999997},"max":{"1min":1.752,"5min":1.8520000000000001,"15min":1.8620000000000001},"last":1.0229999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.353,"15min":1.369},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.4019999999999999,"15min":1.4470000000000001},"min":{"1min":1.075,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.222}]},{"osd":4,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.292,"15min":1.3100000000000001},"min":{"1min":0.70399999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.633,"5min":1.8759999999999999,"15min":2.278},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3360000000000001,"15min":1.369},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.64300000000000002},"max":{"1min":1.881,"5min":1.881,"15min":2.3399999999999999},"last":1.629}]},{"osd":5,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.284,"15min":1.339},"min":{"1min":0.79500000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6930000000000001,"5min":1.7689999999999999,"15min":4.3099999999999996},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4710000000000001,"15min":1.476},"min":{"1min":1.3140000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.927,"5min":1.927,"15min":2.1070000000000002},"last":2.012}]},{"osd":7,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.464,"15min":1.504},"min":{"1min":1.1439999999999999,"5min":0.97899999999999998,"15min":0.77200000000000002},"max":{"1min":1.8009999999999999,"5min":1.9159999999999999,"15min":2.3959999999999999},"last":1.956},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4399999999999999,"15min":1.476},"min":{"1min":1.1539999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.843,"5min":1.9710000000000001,"15min":4.1550000000000002},"last":1.7669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":551841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0640000000000001,"15min":1.115},"min":{"1min":0.72699999999999998,"5min":0.58599999999999997,"15min":0.46700000000000003},"max":{"1min":1.4219999999999999,"5min":1.627,"15min":2.302},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.80300000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":3.5529999999999999},"last":1.1950000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2290000000000001,"15min":1.208},"min":{"1min":0.85299999999999998,"5min":0.72999999999999998,"15min":0.72399999999999998},"max":{"1min":1.524,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.651},{"interface":"front","average":{"1min":1.292,"5min":1.2609999999999999,"15min":1.262},"min":{"1min":1.0209999999999999,"5min":0.81399999999999995,"15min":0.64700000000000002},"max":{"1min":1.579,"5min":1.754,"15min":2.1789999999999998},"last":1.0649999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.165,"15min":1.226},"min":{"1min":0.82599999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.698,"5min":1.7430000000000001,"15min":3.6739999999999999},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.29,"5min":1.2929999999999999,"15min":1.272},"min":{"1min":0.96399999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7110000000000001,"5min":3.282,"15min":3.282},"last":1.5369999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3129999999999999,"15min":1.3460000000000001},"min":{"1min":0.97499999999999998,"5min":0.77700000000000002,"15min":0.63500000000000001},"max":{"1min":1.7609999999999999,"5min":2.1259999999999999,"15min":6.782},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.367,"15min":1.399},"min":{"1min":1.159,"5min":0.89700000000000002,"15min":0.84799999999999998},"max":{"1min":1.7230000000000001,"5min":1.8160000000000001,"15min":6.6189999999999998},"last":1.3640000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3029999999999999,"15min":1.3069999999999999},"min":{"1min":0.90300000000000002,"5min":0.82899999999999996,"15min":0.53600000000000003},"max":{"1min":1.619,"5min":4.0179999999999998,"15min":6.1349999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.458,"15min":1.429},"min":{"1min":1.175,"5min":0.71799999999999997,"15min":0.51400000000000001},"max":{"1min":1.9970000000000001,"5min":3.456,"15min":7.0069999999999997},"last":1.609}]},{"osd":6,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.383,"15min":1.4059999999999999},"min":{"1min":1.0289999999999999,"5min":0.83399999999999996,"15min":0.57699999999999996},"max":{"1min":1.663,"5min":1.8089999999999999,"15min":7.2569999999999997},"last":1.889},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4650000000000001,"15min":1.4690000000000001},"min":{"1min":1.149,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7629999999999999,"5min":3.8620000000000001,"15min":6.2199999999999998},"last":1.746}]},{"osd":7,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.5529999999999999,"15min":1.5489999999999999},"min":{"1min":1.2829999999999999,"5min":1.0940000000000001,"15min":0.53900000000000003},"max":{"1min":1.9099999999999999,"5min":3.4079999999999999,"15min":6.9279999999999999},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.53,"15min":1.5580000000000001},"min":{"1min":1.137,"5min":0.91300000000000003,"15min":0.61699999999999999},"max":{"1min":1.806,"5min":1.996,"15min":7.335},"last":1.8260000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1105920,"data_stored":553431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.262,"15min":1.266},"min":{"1min":0.79900000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.7609999999999999,"5min":1.8009999999999999,"15min":1.863},"last":1.151},{"interface":"front","average":{"1min":1.359,"5min":1.3879999999999999,"15min":1.3799999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.0680000000000001},"last":1.403}]},{"osd":1,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.2989999999999999,"15min":1.274},"min":{"1min":0.86099999999999999,"5min":0.79300000000000004,"15min":0.65500000000000003},"max":{"1min":1.673,"5min":2.3519999999999999,"15min":2.3799999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.236,"15min":1.216},"min":{"1min":0.76000000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.544,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.2390000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.236,"15min":1.2170000000000001},"min":{"1min":0.82499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5920000000000001,"5min":1.6759999999999999,"15min":1.77},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3660000000000001,"15min":1.3580000000000001},"min":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.748},"max":{"1min":1.7549999999999999,"5min":2.423,"15min":2.423},"last":1.3080000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.276,"15min":1.276},"min":{"1min":0.872,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.7110000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.079},{"interface":"front","average":{"1min":1.302,"5min":1.343,"15min":1.3520000000000001},"min":{"1min":0.998,"5min":0.90100000000000002,"15min":0.76900000000000002},"max":{"1min":1.875,"5min":1.875,"15min":2.1080000000000001},"last":0.83199999999999996}]},{"osd":5,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.353,"15min":1.341},"min":{"1min":0.91300000000000003,"5min":0.88500000000000001,"15min":0.45700000000000002},"max":{"1min":1.946,"5min":2.48,"15min":2.48},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.375},"min":{"1min":0.94399999999999995,"5min":0.92700000000000005,"15min":0.58199999999999996},"max":{"1min":1.6970000000000001,"5min":2.504,"15min":2.504},"last":1.3560000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.355,"15min":1.3620000000000001},"min":{"1min":0.90400000000000003,"5min":0.877,"15min":0.55000000000000004},"max":{"1min":1.946,"5min":1.9930000000000001,"15min":2.2080000000000002},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.522,"15min":1.514},"min":{"1min":1.2090000000000001,"5min":1.0660000000000001,"15min":0.70399999999999996},"max":{"1min":2.0299999999999998,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.5249999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.508,"15min":1.472},"min":{"1min":1.1719999999999999,"5min":1,"15min":0.73699999999999999},"max":{"1min":1.6950000000000001,"5min":4.125,"15min":4.125},"last":1.49},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5069999999999999,"15min":1.5369999999999999},"min":{"1min":1.0109999999999999,"5min":0.97199999999999998,"15min":0.67400000000000004},"max":{"1min":1.9970000000000001,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.2649999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575047,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1093632,"data_stored":551819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93799999999999994,"15min":0.91300000000000003},"min":{"1min":0.70599999999999996,"5min":0.32100000000000001,"15min":0.29599999999999999},"max":{"1min":1.2729999999999999,"5min":1.5269999999999999,"15min":1.72},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.0760000000000001,"15min":1.0820000000000001},"min":{"1min":0.63700000000000001,"5min":0.45400000000000001,"15min":0.39000000000000001},"max":{"1min":1.4690000000000001,"5min":1.5,"15min":2.3199999999999998},"last":0.94399999999999995}]},{"osd":1,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.93999999999999995,"15min":0.95999999999999996},"min":{"1min":0.70999999999999996,"5min":0.501,"15min":0.373},"max":{"1min":1.387,"5min":1.5629999999999999,"15min":1.6040000000000001},"last":1.329},{"interface":"front","average":{"1min":1.25,"5min":1.1890000000000001,"15min":1.1930000000000001},"min":{"1min":0.66300000000000003,"5min":0.56599999999999995,"15min":0.436},"max":{"1min":1.5429999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.3959999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.069,"15min":1.0629999999999999},"min":{"1min":0.69999999999999996,"5min":0.54500000000000004,"15min":0.47699999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.056},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.1499999999999999,"15min":1.135},"min":{"1min":0.78600000000000003,"5min":0.60499999999999998,"15min":0.41899999999999998},"max":{"1min":1.571,"5min":1.6819999999999999,"15min":1.788},"last":1.232}]},{"osd":4,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2110000000000001,"15min":1.2050000000000001},"min":{"1min":0.86499999999999999,"5min":0.54000000000000004,"15min":0.34200000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.5059999999999998},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.3859999999999999,"15min":1.379},"min":{"1min":1.0529999999999999,"5min":0.72799999999999998,"15min":0.53800000000000003},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":1.5760000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3169999999999999,"15min":1.294},"min":{"1min":0.90400000000000003,"5min":0.73699999999999999,"15min":0.624},"max":{"1min":1.698,"5min":1.883,"15min":1.9219999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.3680000000000001,"15min":1.355},"min":{"1min":0.93500000000000005,"5min":0.78500000000000003,"15min":0.59599999999999997},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.5329999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.296,"15min":1.3160000000000001},"min":{"1min":0.94699999999999995,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.742,"5min":1.776,"15min":2.6150000000000002},"last":1.468},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.4510000000000001,"15min":1.4410000000000001},"min":{"1min":1.1160000000000001,"5min":0.86699999999999999,"15min":0.70099999999999996},"max":{"1min":2.012,"5min":2.0489999999999999,"15min":2.1259999999999999},"last":1.498}]},{"osd":7,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.488,"15min":1.4830000000000001},"min":{"1min":1.026,"5min":0.82399999999999995,"15min":0.72299999999999998},"max":{"1min":1.956,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.619},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.373,"15min":1.3919999999999999},"min":{"1min":0.98399999999999999,"5min":0.77100000000000002,"15min":0.66900000000000004},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":2.7879999999999998},"last":1.363}]}]},{"osd":2,"up_from":13,"seq":55834575045,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58616,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313224,"statfs":{"total":96636764160,"available":96576741376,"internally_reserved":0,"allocated":1093632,"data_stored":551819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.88,"15min":0.90000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.37},"max":{"1min":1.4950000000000001,"5min":1.5209999999999999,"15min":2.4009999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.47499999999999998},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.9470000000000001},"last":0.71799999999999997}]},{"osd":1,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93999999999999995,"15min":0.95499999999999996},"min":{"1min":0.66100000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":2.1070000000000002},"last":1.373},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0840000000000001,"15min":1.0980000000000001},"min":{"1min":0.58399999999999996,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3029999999999999,"5min":1.706,"15min":2.1469999999999998},"last":0.95399999999999996}]},{"osd":3,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1060000000000001,"15min":1.131},"min":{"1min":0.70699999999999996,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6240000000000001,"5min":1.639,"15min":1.8120000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.262,"15min":1.2430000000000001},"min":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.59399999999999997},"max":{"1min":1.552,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":1.181}]},{"osd":4,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.413,"15min":1.3939999999999999},"min":{"1min":0.88400000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.7989999999999999,"5min":4.593,"15min":4.593},"last":1.288},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.286,"15min":1.296},"min":{"1min":0.78600000000000003,"5min":0.63200000000000001,"15min":0.63100000000000001},"max":{"1min":1.881,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.21}]},{"osd":5,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.3839999999999999},"min":{"1min":0.84799999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.819,"5min":6.7140000000000004,"15min":6.7140000000000004},"last":1.429},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4390000000000001,"15min":1.4570000000000001},"min":{"1min":0.92900000000000005,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.72,"5min":2.0529999999999999,"15min":2.8170000000000002},"last":1.3480000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.355,"15min":1.357},"min":{"1min":0.93000000000000005,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8080000000000001,"5min":2.7080000000000002,"15min":2.851},"last":0.998},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.444,"15min":1.4610000000000001},"min":{"1min":0.88600000000000001,"5min":0.74099999999999999,"15min":0.63900000000000001},"max":{"1min":1.6950000000000001,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.2290000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.46,"15min":1.452},"min":{"1min":0.82499999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.8720000000000001,"5min":6.609,"15min":6.609},"last":1.125},{"interface":"front","average":{"1min":1.45,"5min":1.4750000000000001,"15min":1.5089999999999999},"min":{"1min":0.95899999999999996,"5min":0.82299999999999995,"15min":0.61799999999999999},"max":{"1min":1.825,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.323}]}]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335404,"statfs":{"total":96636764160,"available":96599453696,"internally_reserved":0,"allocated":1122304,"data_stored":554195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.439,"5min":0.39700000000000002,"15min":0.29799999999999999},"max":{"1min":1.3740000000000001,"5min":1.796,"15min":1.796},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.149,"5min":1.121,"15min":1.1299999999999999},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.433},"max":{"1min":1.4510000000000001,"5min":1.6060000000000001,"15min":1.8089999999999999},"last":1.0129999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.94199999999999995,"15min":0.95299999999999996},"min":{"1min":0.46899999999999997,"5min":0.39500000000000002,"15min":0.34899999999999998},"max":{"1min":1.3320000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.081,"15min":1.083},"min":{"1min":0.61799999999999999,"5min":0.53400000000000003,"15min":0.38200000000000001},"max":{"1min":1.331,"5min":1.77,"15min":2.0750000000000002},"last":0.81499999999999995}]},{"osd":3,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1439999999999999,"15min":1.1739999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.41899999999999998},"max":{"1min":1.6160000000000001,"5min":1.7549999999999999,"15min":1.8160000000000001},"last":1.242},{"interface":"front","average":{"1min":1.159,"5min":1.0720000000000001,"15min":1.069},"min":{"1min":0.622,"5min":0.46800000000000003,"15min":0.42799999999999999},"max":{"1min":1.585,"5min":1.788,"15min":1.788},"last":0.77100000000000002}]},{"osd":4,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.266,"15min":1.296},"min":{"1min":0.93600000000000005,"5min":0.79800000000000004,"15min":0.63900000000000001},"max":{"1min":1.538,"5min":1.966,"15min":1.966},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.24,"15min":1.2270000000000001},"min":{"1min":0.83399999999999996,"5min":0.72699999999999998,"15min":0.61799999999999999},"max":{"1min":1.5409999999999999,"5min":1.913,"15min":2.2029999999999998},"last":1.101}]},{"osd":5,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3100000000000001,"15min":1.3049999999999999},"min":{"1min":0.95199999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6140000000000001,"5min":1.8680000000000001,"15min":2.3399999999999999},"last":1.393},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4419999999999999,"15min":1.4390000000000001},"min":{"1min":1.04,"5min":0.86799999999999999,"15min":0.71599999999999997},"max":{"1min":1.76,"5min":2.077,"15min":2.3069999999999999},"last":1.375}]},{"osd":6,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.395,"15min":1.419},"min":{"1min":1.0720000000000001,"5min":0.89600000000000002,"15min":0.65500000000000003},"max":{"1min":1.6859999999999999,"5min":1.972,"15min":2.2490000000000001},"last":1.282},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4179999999999999,"15min":1.417},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.68300000000000005},"max":{"1min":1.681,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.169}]},{"osd":7,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.415,"15min":1.4570000000000001},"min":{"1min":0.88500000000000001,"5min":0.82699999999999996,"15min":0.79100000000000004},"max":{"1min":1.893,"5min":1.97,"15min":2.1869999999999998},"last":1.081},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4450000000000001,"15min":1.456},"min":{"1min":1.0089999999999999,"5min":0.98699999999999999,"15min":0.76300000000000001},"max":{"1min":1.698,"5min":2.286,"15min":2.4180000000000001},"last":1.347}]}]},{"osd":0,"up_from":13,"seq":55834575046,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36996,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334844,"statfs":{"total":96636764160,"available":96598880256,"internally_reserved":0,"allocated":1687552,"data_stored":1142187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89200000000000002,"15min":0.90500000000000003},"min":{"1min":0.5,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.7130000000000001},"last":1.002},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.042,"15min":1.026},"min":{"1min":0.58099999999999996,"5min":0.371,"15min":0.371},"max":{"1min":1.6719999999999999,"5min":1.675,"15min":2.3500000000000001},"last":0.78400000000000003}]},{"osd":2,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.073,"15min":1.079},"min":{"1min":0.80800000000000005,"5min":0.39200000000000002,"15min":0.38900000000000001},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.236},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.98799999999999999,"15min":0.995},"min":{"1min":0.54200000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.621,"5min":1.621,"15min":1.78},"last":0.90700000000000003}]},{"osd":3,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.1830000000000001,"15min":1.1950000000000001},"min":{"1min":0.71799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":2.4540000000000002},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.0960000000000001,"15min":1.0820000000000001},"min":{"1min":0.65900000000000003,"5min":0.44700000000000001,"15min":0.439},"max":{"1min":1.7609999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.1899999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.284,"15min":1.268},"min":{"1min":0.94399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":2.496},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.321,"15min":1.3420000000000001},"min":{"1min":0.98499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":2.6560000000000001},"last":1.3080000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.341,"15min":1.333},"min":{"1min":1.155,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.921,"5min":1.921,"15min":2.2650000000000001},"last":1.343},{"interface":"front","average":{"1min":1.45,"5min":1.341,"15min":1.3799999999999999},"min":{"1min":0.94399999999999995,"5min":0.63,"15min":0.63},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.7069999999999999},"last":1.8160000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.405,"15min":1.4099999999999999},"min":{"1min":1.2070000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.97,"5min":2.1909999999999998,"15min":2.8290000000000002},"last":1.399},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.385,"15min":1.3999999999999999},"min":{"1min":0.88900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.4020000000000001},"last":1.4990000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.4239999999999999,"15min":1.4370000000000001},"min":{"1min":0.96399999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.746},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.466,"15min":1.466},"min":{"1min":1.042,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.04,"5min":2.1059999999999999,"15min":2.9300000000000002},"last":1.474}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:52:31.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:52:31.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.300+0000 7f9e764ec640 1 Processor -- start 2023-10-08T05:52:31.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.300+0000 7f9e764ec640 1 -- start start 2023-10-08T05:52:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.302+0000 7f9e764ec640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7015ab40 0x7f9e700ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.302+0000 7f9e764ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e700c90b0 0x7f9e700adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.302+0000 7f9e764ec640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e700cae70 0x7f9e700cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.302+0000 7f9e764ec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7015bf30 con 0x7f9e700c90b0 2023-10-08T05:52:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.302+0000 7f9e764ec640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e700cb840 con 0x7f9e700cae70 2023-10-08T05:52:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.302+0000 7f9e764ec640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7015b2b0 con 0x7f9e7015ab40 2023-10-08T05:52:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.302+0000 7f9e67fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e700c90b0 0x7f9e700adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.303+0000 7f9e67fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e700c90b0 0x7f9e700adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48980/0 (socket says 172.21.15.70:48980) 2023-10-08T05:52:31.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.303+0000 7f9e67fff640 1 -- 172.21.15.70:0/705910003 learned_addr learned my addr 172.21.15.70:0/705910003 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.303+0000 7f9e6ffff640 1 --2- 172.21.15.70:0/705910003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7015ab40 0x7f9e700ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.303+0000 7f9e74a89640 1 --2- 172.21.15.70:0/705910003 >> v2:172.21.15.181:3300/0 conn(0x7f9e700cae70 0x7f9e700cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.303+0000 7f9e67fff640 1 -- 172.21.15.70:0/705910003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7015ab40 msgr2=0x7f9e700ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.303+0000 7f9e67fff640 1 --2- 172.21.15.70:0/705910003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7015ab40 0x7f9e700ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.303+0000 7f9e67fff640 1 -- 172.21.15.70:0/705910003 >> v2:172.21.15.181:3300/0 conn(0x7f9e700cae70 msgr2=0x7f9e700cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.303+0000 7f9e67fff640 1 --2- 172.21.15.70:0/705910003 >> v2:172.21.15.181:3300/0 conn(0x7f9e700cae70 0x7f9e700cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.303+0000 7f9e67fff640 1 -- 172.21.15.70:0/705910003 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7015b590 con 0x7f9e700c90b0 2023-10-08T05:52:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.303+0000 7f9e74a89640 1 --2- 172.21.15.70:0/705910003 >> v2:172.21.15.181:3300/0 conn(0x7f9e700cae70 0x7f9e700cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:31.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.304+0000 7f9e67fff640 1 --2- 172.21.15.70:0/705910003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700c90b0 0x7f9e700adc60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9e700c8d50 tx=0x7f9e5802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23004ec7466741b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e6f7fe640 1 -- 172.21.15.70:0/705910003 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58031860 con 0x7f9e700c90b0 2023-10-08T05:52:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e6f7fe640 1 -- 172.21.15.70:0/705910003 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e580319c0 con 0x7f9e700c90b0 2023-10-08T05:52:31.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e6f7fe640 1 -- 172.21.15.70:0/705910003 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58031d50 con 0x7f9e700c90b0 2023-10-08T05:52:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e764ec640 1 -- 172.21.15.70:0/705910003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700c90b0 msgr2=0x7f9e700adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e764ec640 1 --2- 172.21.15.70:0/705910003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700c90b0 0x7f9e700adc60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9e700c8d50 tx=0x7f9e5802b900 comp rx=0 tx=0).stop 2023-10-08T05:52:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e764ec640 1 -- 172.21.15.70:0/705910003 shutdown_connections 2023-10-08T05:52:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e764ec640 1 --2- 172.21.15.70:0/705910003 >> v2:172.21.15.181:3300/0 conn(0x7f9e700cae70 0x7f9e700cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e764ec640 1 --2- 172.21.15.70:0/705910003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700c90b0 0x7f9e700adc60 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e764ec640 1 --2- 172.21.15.70:0/705910003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e7015ab40 0x7f9e700ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e764ec640 1 -- 172.21.15.70:0/705910003 >> 172.21.15.70:0/705910003 conn(0x7f9e7006a380 msgr2=0x7f9e700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e764ec640 1 -- 172.21.15.70:0/705910003 shutdown_connections 2023-10-08T05:52:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.305+0000 7f9e764ec640 1 -- 172.21.15.70:0/705910003 wait complete. 2023-10-08T05:52:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.307+0000 7f9e764ec640 1 Processor -- start 2023-10-08T05:52:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.307+0000 7f9e764ec640 1 -- start start 2023-10-08T05:52:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.308+0000 7f9e764ec640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e700c90b0 0x7f9e70140780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.308+0000 7f9e764ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700cae70 0x7f9e700af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.308+0000 7f9e764ec640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e7015ab40 0x7f9e700af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.308+0000 7f9e764ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e70140cc0 con 0x7f9e700cae70 2023-10-08T05:52:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.308+0000 7f9e764ec640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e700adfe0 con 0x7f9e7015ab40 2023-10-08T05:52:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.308+0000 7f9e764ec640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e700ada20 con 0x7f9e700c90b0 2023-10-08T05:52:31.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.309+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700cae70 0x7f9e700af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.309+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700cae70 0x7f9e700af460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48988/0 (socket says 172.21.15.70:48988) 2023-10-08T05:52:31.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.309+0000 7f9e67fff640 1 -- 172.21.15.70:0/2031204555 learned_addr learned my addr 172.21.15.70:0/2031204555 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.309+0000 7f9e6ffff640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e700c90b0 0x7f9e70140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.309+0000 7f9e67fff640 1 -- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e700c90b0 msgr2=0x7f9e70140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.309+0000 7f9e67fff640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e700c90b0 0x7f9e70140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.309+0000 7f9e67fff640 1 -- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e7015ab40 msgr2=0x7f9e700af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.309+0000 7f9e67fff640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e7015ab40 0x7f9e700af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.309+0000 7f9e67fff640 1 -- 172.21.15.70:0/2031204555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580094d0 con 0x7f9e700cae70 2023-10-08T05:52:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.310+0000 7f9e67fff640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700cae70 0x7f9e700af460 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9e5802b940 tx=0x7f9e580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.310+0000 7f9e6d7fa640 1 -- 172.21.15.70:0/2031204555 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5803ac60 con 0x7f9e700cae70 2023-10-08T05:52:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.310+0000 7f9e6d7fa640 1 -- 172.21.15.70:0/2031204555 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5803adc0 con 0x7f9e700cae70 2023-10-08T05:52:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.310+0000 7f9e6d7fa640 1 -- 172.21.15.70:0/2031204555 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e580445f0 con 0x7f9e700cae70 2023-10-08T05:52:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.310+0000 7f9e764ec640 1 -- 172.21.15.70:0/2031204555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e700add00 con 0x7f9e700cae70 2023-10-08T05:52:31.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.310+0000 7f9e764ec640 1 -- 172.21.15.70:0/2031204555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e70141360 con 0x7f9e700cae70 2023-10-08T05:52:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.311+0000 7f9e764ec640 1 -- 172.21.15.70:0/2031204555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e70067490 con 0x7f9e700cae70 2023-10-08T05:52:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.312+0000 7f9e6d7fa640 1 -- 172.21.15.70:0/2031204555 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9e5804d450 con 0x7f9e700cae70 2023-10-08T05:52:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.313+0000 7f9e6d7fa640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9e5006af90 0x7f9e5006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.313+0000 7f9e6d7fa640 1 -- 172.21.15.70:0/2031204555 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9e580abf80 con 0x7f9e700cae70 2023-10-08T05:52:31.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.314+0000 7f9e6ffff640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9e5006af90 0x7f9e5006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.315+0000 7f9e6ffff640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9e5006af90 0x7f9e5006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006fd0 tx=0x7f9e60008040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:31.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.318+0000 7f9e6d7fa640 1 -- 172.21.15.70:0/2031204555 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e58076c90 con 0x7f9e700cae70 2023-10-08T05:52:31.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.503+0000 7f9e764ec640 1 -- 172.21.15.70:0/2031204555 --> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e7015bb50 con 0x7f9e5006af90 2023-10-08T05:52:31.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.525+0000 7f9e6d7fa640 1 -- 172.21.15.70:0/2031204555 <== mgr.5473 v2:172.21.15.181:6832/3994892546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286630 (secure 0 0 0) 0x7f9e6000b200 con 0x7f9e5006af90 2023-10-08T05:52:31.528 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:31.532 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:52:31.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 -- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9e5006af90 msgr2=0x7f9e5006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:31.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9e5006af90 0x7f9e5006d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9e60006fd0 tx=0x7f9e60008040 comp rx=0 tx=0).stop 2023-10-08T05:52:31.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 -- 172.21.15.70:0/2031204555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700cae70 msgr2=0x7f9e700af460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:31.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700cae70 0x7f9e700af460 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9e5802b940 tx=0x7f9e580046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 -- 172.21.15.70:0/2031204555 shutdown_connections 2023-10-08T05:52:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f9e5006af90 0x7f9e5006d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e7015ab40 0x7f9e700af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e700cae70 0x7f9e700af460 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 --2- 172.21.15.70:0/2031204555 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e700c90b0 0x7f9e70140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 -- 172.21.15.70:0/2031204555 >> 172.21.15.70:0/2031204555 conn(0x7f9e7006a380 msgr2=0x7f9e700c4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 -- 172.21.15.70:0/2031204555 shutdown_connections 2023-10-08T05:52:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.531+0000 7f9e764ec640 1 -- 172.21.15.70:0/2031204555 wait complete. 2023-10-08T05:52:31.551 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-08T05:52:30.592906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":324456,"kb_used_data":9768,"kb_used_omap":12,"kb_used_meta":314547,"kb_avail":754650264,"statfs":{"total":773094113280,"available":772761870336,"internally_reserved":0,"allocated":10002432,"data_stored":5600108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":322096699},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002180"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.050025+0000","last_change":"2023-10-08T05:52:26.607722+0000","last_active":"2023-10-08T05:52:28.050025+0000","last_peered":"2023-10-08T05:52:28.050025+0000","last_clean":"2023-10-08T05:52:28.050025+0000","last_became_active":"2023-10-08T05:52:26.607448+0000","last_became_peered":"2023-10-08T05:52:26.607448+0000","last_unstale":"2023-10-08T05:52:28.050025+0000","last_undegraded":"2023-10-08T05:52:28.050025+0000","last_fullsized":"2023-10-08T05:52:28.050025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:20:46.060726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.049735+0000","last_change":"2023-10-08T05:52:26.611368+0000","last_active":"2023-10-08T05:52:28.049735+0000","last_peered":"2023-10-08T05:52:28.049735+0000","last_clean":"2023-10-08T05:52:28.049735+0000","last_became_active":"2023-10-08T05:52:26.611115+0000","last_became_peered":"2023-10-08T05:52:26.611115+0000","last_unstale":"2023-10-08T05:52:28.049735+0000","last_undegraded":"2023-10-08T05:52:28.049735+0000","last_fullsized":"2023-10-08T05:52:28.049735+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:59:51.102207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.633432+0000","last_change":"2023-10-08T05:52:26.627434+0000","last_active":"2023-10-08T05:52:27.633432+0000","last_peered":"2023-10-08T05:52:27.633432+0000","last_clean":"2023-10-08T05:52:27.633432+0000","last_became_active":"2023-10-08T05:52:26.626262+0000","last_became_peered":"2023-10-08T05:52:26.626262+0000","last_unstale":"2023-10-08T05:52:27.633432+0000","last_undegraded":"2023-10-08T05:52:27.633432+0000","last_fullsized":"2023-10-08T05:52:27.633432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:48.081753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612692+0000","last_change":"2023-10-08T05:52:26.603872+0000","last_active":"2023-10-08T05:52:27.612692+0000","last_peered":"2023-10-08T05:52:27.612692+0000","last_clean":"2023-10-08T05:52:27.612692+0000","last_became_active":"2023-10-08T05:52:26.603593+0000","last_became_peered":"2023-10-08T05:52:26.603593+0000","last_unstale":"2023-10-08T05:52:27.612692+0000","last_undegraded":"2023-10-08T05:52:27.612692+0000","last_fullsized":"2023-10-08T05:52:27.612692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:40.962951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613819+0000","last_change":"2023-10-08T05:52:26.609920+0000","last_active":"2023-10-08T05:52:27.613819+0000","last_peered":"2023-10-08T05:52:27.613819+0000","last_clean":"2023-10-08T05:52:27.613819+0000","last_became_active":"2023-10-08T05:52:26.608134+0000","last_became_peered":"2023-10-08T05:52:26.608134+0000","last_unstale":"2023-10-08T05:52:27.613819+0000","last_undegraded":"2023-10-08T05:52:27.613819+0000","last_fullsized":"2023-10-08T05:52:27.613819+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:47:02.038317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.048335+0000","last_change":"2023-10-08T05:52:26.622173+0000","last_active":"2023-10-08T05:52:28.048335+0000","last_peered":"2023-10-08T05:52:28.048335+0000","last_clean":"2023-10-08T05:52:28.048335+0000","last_became_active":"2023-10-08T05:52:26.621030+0000","last_became_peered":"2023-10-08T05:52:26.621030+0000","last_unstale":"2023-10-08T05:52:28.048335+0000","last_undegraded":"2023-10-08T05:52:28.048335+0000","last_fullsized":"2023-10-08T05:52:28.048335+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:46.620903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046114+0000","last_change":"2023-10-08T05:52:26.616818+0000","last_active":"2023-10-08T05:52:28.046114+0000","last_peered":"2023-10-08T05:52:28.046114+0000","last_clean":"2023-10-08T05:52:28.046114+0000","last_became_active":"2023-10-08T05:52:26.616534+0000","last_became_peered":"2023-10-08T05:52:26.616534+0000","last_unstale":"2023-10-08T05:52:28.046114+0000","last_undegraded":"2023-10-08T05:52:28.046114+0000","last_fullsized":"2023-10-08T05:52:28.046114+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:55.547855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046444+0000","last_change":"2023-10-08T05:52:26.621106+0000","last_active":"2023-10-08T05:52:28.046444+0000","last_peered":"2023-10-08T05:52:28.046444+0000","last_clean":"2023-10-08T05:52:28.046444+0000","last_became_active":"2023-10-08T05:52:26.620787+0000","last_became_peered":"2023-10-08T05:52:26.620787+0000","last_unstale":"2023-10-08T05:52:28.046444+0000","last_undegraded":"2023-10-08T05:52:28.046444+0000","last_fullsized":"2023-10-08T05:52:28.046444+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:26.165323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.633324+0000","last_change":"2023-10-08T05:52:26.630837+0000","last_active":"2023-10-08T05:52:27.633324+0000","last_peered":"2023-10-08T05:52:27.633324+0000","last_clean":"2023-10-08T05:52:27.633324+0000","last_became_active":"2023-10-08T05:52:26.630527+0000","last_became_peered":"2023-10-08T05:52:26.630527+0000","last_unstale":"2023-10-08T05:52:27.633324+0000","last_undegraded":"2023-10-08T05:52:27.633324+0000","last_fullsized":"2023-10-08T05:52:27.633324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:53.711411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.632914+0000","last_change":"2023-10-08T05:52:26.630342+0000","last_active":"2023-10-08T05:52:27.632914+0000","last_peered":"2023-10-08T05:52:27.632914+0000","last_clean":"2023-10-08T05:52:27.632914+0000","last_became_active":"2023-10-08T05:52:26.628804+0000","last_became_peered":"2023-10-08T05:52:26.628804+0000","last_unstale":"2023-10-08T05:52:27.632914+0000","last_undegraded":"2023-10-08T05:52:27.632914+0000","last_fullsized":"2023-10-08T05:52:27.632914+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:32.655131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613460+0000","last_change":"2023-10-08T05:52:26.608390+0000","last_active":"2023-10-08T05:52:27.613460+0000","last_peered":"2023-10-08T05:52:27.613460+0000","last_clean":"2023-10-08T05:52:27.613460+0000","last_became_active":"2023-10-08T05:52:26.608119+0000","last_became_peered":"2023-10-08T05:52:26.608119+0000","last_unstale":"2023-10-08T05:52:27.613460+0000","last_undegraded":"2023-10-08T05:52:27.613460+0000","last_fullsized":"2023-10-08T05:52:27.613460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:15.319541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046342+0000","last_change":"2023-10-08T05:52:26.619890+0000","last_active":"2023-10-08T05:52:28.046342+0000","last_peered":"2023-10-08T05:52:28.046342+0000","last_clean":"2023-10-08T05:52:28.046342+0000","last_became_active":"2023-10-08T05:52:26.619597+0000","last_became_peered":"2023-10-08T05:52:26.619597+0000","last_unstale":"2023-10-08T05:52:28.046342+0000","last_undegraded":"2023-10-08T05:52:28.046342+0000","last_fullsized":"2023-10-08T05:52:28.046342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:48.081753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046674+0000","last_change":"2023-10-08T05:52:26.612317+0000","last_active":"2023-10-08T05:52:28.046674+0000","last_peered":"2023-10-08T05:52:28.046674+0000","last_clean":"2023-10-08T05:52:28.046674+0000","last_became_active":"2023-10-08T05:52:26.612023+0000","last_became_peered":"2023-10-08T05:52:26.612023+0000","last_unstale":"2023-10-08T05:52:28.046674+0000","last_undegraded":"2023-10-08T05:52:28.046674+0000","last_fullsized":"2023-10-08T05:52:28.046674+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:15.670901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.633053+0000","last_change":"2023-10-08T05:52:26.626380+0000","last_active":"2023-10-08T05:52:27.633053+0000","last_peered":"2023-10-08T05:52:27.633053+0000","last_clean":"2023-10-08T05:52:27.633053+0000","last_became_active":"2023-10-08T05:52:26.626027+0000","last_became_peered":"2023-10-08T05:52:26.626027+0000","last_unstale":"2023-10-08T05:52:27.633053+0000","last_undegraded":"2023-10-08T05:52:27.633053+0000","last_fullsized":"2023-10-08T05:52:27.633053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:42.293374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.632428+0000","last_change":"2023-10-08T05:52:26.628459+0000","last_active":"2023-10-08T05:52:27.632428+0000","last_peered":"2023-10-08T05:52:27.632428+0000","last_clean":"2023-10-08T05:52:27.632428+0000","last_became_active":"2023-10-08T05:52:26.628133+0000","last_became_peered":"2023-10-08T05:52:26.628133+0000","last_unstale":"2023-10-08T05:52:27.632428+0000","last_undegraded":"2023-10-08T05:52:27.632428+0000","last_fullsized":"2023-10-08T05:52:27.632428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:33.241545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.045511+0000","last_change":"2023-10-08T05:52:26.611424+0000","last_active":"2023-10-08T05:52:28.045511+0000","last_peered":"2023-10-08T05:52:28.045511+0000","last_clean":"2023-10-08T05:52:28.045511+0000","last_became_active":"2023-10-08T05:52:26.611147+0000","last_became_peered":"2023-10-08T05:52:26.611147+0000","last_unstale":"2023-10-08T05:52:28.045511+0000","last_undegraded":"2023-10-08T05:52:28.045511+0000","last_fullsized":"2023-10-08T05:52:28.045511+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:03.695664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.049929+0000","last_change":"2023-10-08T05:52:26.610090+0000","last_active":"2023-10-08T05:52:28.049929+0000","last_peered":"2023-10-08T05:52:28.049929+0000","last_clean":"2023-10-08T05:52:28.049929+0000","last_became_active":"2023-10-08T05:52:26.609811+0000","last_became_peered":"2023-10-08T05:52:26.609811+0000","last_unstale":"2023-10-08T05:52:28.049929+0000","last_undegraded":"2023-10-08T05:52:28.049929+0000","last_fullsized":"2023-10-08T05:52:28.049929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:10.511258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612629+0000","last_change":"2023-10-08T05:52:26.604534+0000","last_active":"2023-10-08T05:52:27.612629+0000","last_peered":"2023-10-08T05:52:27.612629+0000","last_clean":"2023-10-08T05:52:27.612629+0000","last_became_active":"2023-10-08T05:52:26.604267+0000","last_became_peered":"2023-10-08T05:52:26.604267+0000","last_unstale":"2023-10-08T05:52:27.612629+0000","last_undegraded":"2023-10-08T05:52:27.612629+0000","last_fullsized":"2023-10-08T05:52:27.612629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:28.707400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613267+0000","last_change":"2023-10-08T05:52:26.603492+0000","last_active":"2023-10-08T05:52:27.613267+0000","last_peered":"2023-10-08T05:52:27.613267+0000","last_clean":"2023-10-08T05:52:27.613267+0000","last_became_active":"2023-10-08T05:52:26.603075+0000","last_became_peered":"2023-10-08T05:52:26.603075+0000","last_unstale":"2023-10-08T05:52:27.613267+0000","last_undegraded":"2023-10-08T05:52:27.613267+0000","last_fullsized":"2023-10-08T05:52:27.613267+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:12:18.916804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.609859+0000","last_change":"2023-10-08T05:52:26.612241+0000","last_active":"2023-10-08T05:52:27.609859+0000","last_peered":"2023-10-08T05:52:27.609859+0000","last_clean":"2023-10-08T05:52:27.609859+0000","last_became_active":"2023-10-08T05:52:26.611918+0000","last_became_peered":"2023-10-08T05:52:26.611918+0000","last_unstale":"2023-10-08T05:52:27.609859+0000","last_undegraded":"2023-10-08T05:52:27.609859+0000","last_fullsized":"2023-10-08T05:52:27.609859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.124027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046626+0000","last_change":"2023-10-08T05:52:26.611297+0000","last_active":"2023-10-08T05:52:28.046626+0000","last_peered":"2023-10-08T05:52:28.046626+0000","last_clean":"2023-10-08T05:52:28.046626+0000","last_became_active":"2023-10-08T05:52:26.611003+0000","last_became_peered":"2023-10-08T05:52:26.611003+0000","last_unstale":"2023-10-08T05:52:28.046626+0000","last_undegraded":"2023-10-08T05:52:28.046626+0000","last_fullsized":"2023-10-08T05:52:28.046626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:18.945623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.609626+0000","last_change":"2023-10-08T05:52:26.621765+0000","last_active":"2023-10-08T05:52:27.609626+0000","last_peered":"2023-10-08T05:52:27.609626+0000","last_clean":"2023-10-08T05:52:27.609626+0000","last_became_active":"2023-10-08T05:52:26.621485+0000","last_became_peered":"2023-10-08T05:52:26.621485+0000","last_unstale":"2023-10-08T05:52:27.609626+0000","last_undegraded":"2023-10-08T05:52:27.609626+0000","last_fullsized":"2023-10-08T05:52:27.609626+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:10.253744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.609501+0000","last_change":"2023-10-08T05:52:26.616889+0000","last_active":"2023-10-08T05:52:27.609501+0000","last_peered":"2023-10-08T05:52:27.609501+0000","last_clean":"2023-10-08T05:52:27.609501+0000","last_became_active":"2023-10-08T05:52:26.616530+0000","last_became_peered":"2023-10-08T05:52:26.616530+0000","last_unstale":"2023-10-08T05:52:27.609501+0000","last_undegraded":"2023-10-08T05:52:27.609501+0000","last_fullsized":"2023-10-08T05:52:27.609501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:26.165323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.632794+0000","last_change":"2023-10-08T05:52:26.626580+0000","last_active":"2023-10-08T05:52:27.632794+0000","last_peered":"2023-10-08T05:52:27.632794+0000","last_clean":"2023-10-08T05:52:27.632794+0000","last_became_active":"2023-10-08T05:52:26.625862+0000","last_became_peered":"2023-10-08T05:52:26.625862+0000","last_unstale":"2023-10-08T05:52:27.632794+0000","last_undegraded":"2023-10-08T05:52:27.632794+0000","last_fullsized":"2023-10-08T05:52:27.632794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:54:21.007655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046645+0000","last_change":"2023-10-08T05:52:26.614866+0000","last_active":"2023-10-08T05:52:28.046645+0000","last_peered":"2023-10-08T05:52:28.046645+0000","last_clean":"2023-10-08T05:52:28.046645+0000","last_became_active":"2023-10-08T05:52:26.614592+0000","last_became_peered":"2023-10-08T05:52:26.614592+0000","last_unstale":"2023-10-08T05:52:28.046645+0000","last_undegraded":"2023-10-08T05:52:28.046645+0000","last_fullsized":"2023-10-08T05:52:28.046645+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:53.124027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046762+0000","last_change":"2023-10-08T05:52:26.615840+0000","last_active":"2023-10-08T05:52:28.046762+0000","last_peered":"2023-10-08T05:52:28.046762+0000","last_clean":"2023-10-08T05:52:28.046762+0000","last_became_active":"2023-10-08T05:52:26.615581+0000","last_became_peered":"2023-10-08T05:52:26.615581+0000","last_unstale":"2023-10-08T05:52:28.046762+0000","last_undegraded":"2023-10-08T05:52:28.046762+0000","last_fullsized":"2023-10-08T05:52:28.046762+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:44.005336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613818+0000","last_change":"2023-10-08T05:52:26.606304+0000","last_active":"2023-10-08T05:52:27.613818+0000","last_peered":"2023-10-08T05:52:27.613818+0000","last_clean":"2023-10-08T05:52:27.613818+0000","last_became_active":"2023-10-08T05:52:26.606009+0000","last_became_peered":"2023-10-08T05:52:26.606009+0000","last_unstale":"2023-10-08T05:52:27.613818+0000","last_undegraded":"2023-10-08T05:52:27.613818+0000","last_fullsized":"2023-10-08T05:52:27.613818+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:46:36.156852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.048707+0000","last_change":"2023-10-08T05:52:26.611935+0000","last_active":"2023-10-08T05:52:28.048707+0000","last_peered":"2023-10-08T05:52:28.048707+0000","last_clean":"2023-10-08T05:52:28.048707+0000","last_became_active":"2023-10-08T05:52:26.611656+0000","last_became_peered":"2023-10-08T05:52:26.611656+0000","last_unstale":"2023-10-08T05:52:28.048707+0000","last_undegraded":"2023-10-08T05:52:28.048707+0000","last_fullsized":"2023-10-08T05:52:28.048707+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:42:13.213193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.632647+0000","last_change":"2023-10-08T05:52:26.630881+0000","last_active":"2023-10-08T05:52:27.632647+0000","last_peered":"2023-10-08T05:52:27.632647+0000","last_clean":"2023-10-08T05:52:27.632647+0000","last_became_active":"2023-10-08T05:52:26.630557+0000","last_became_peered":"2023-10-08T05:52:26.630557+0000","last_unstale":"2023-10-08T05:52:27.632647+0000","last_undegraded":"2023-10-08T05:52:27.632647+0000","last_fullsized":"2023-10-08T05:52:27.632647+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:43.017103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046077+0000","last_change":"2023-10-08T05:52:26.620520+0000","last_active":"2023-10-08T05:52:28.046077+0000","last_peered":"2023-10-08T05:52:28.046077+0000","last_clean":"2023-10-08T05:52:28.046077+0000","last_became_active":"2023-10-08T05:52:26.620057+0000","last_became_peered":"2023-10-08T05:52:26.620057+0000","last_unstale":"2023-10-08T05:52:28.046077+0000","last_undegraded":"2023-10-08T05:52:28.046077+0000","last_fullsized":"2023-10-08T05:52:28.046077+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:53.711411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.611826+0000","last_change":"2023-10-08T05:52:26.598724+0000","last_active":"2023-10-08T05:52:27.611826+0000","last_peered":"2023-10-08T05:52:27.611826+0000","last_clean":"2023-10-08T05:52:27.611826+0000","last_became_active":"2023-10-08T05:52:26.597953+0000","last_became_peered":"2023-10-08T05:52:26.597953+0000","last_unstale":"2023-10-08T05:52:27.611826+0000","last_undegraded":"2023-10-08T05:52:27.611826+0000","last_fullsized":"2023-10-08T05:52:27.611826+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:27.918719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.047826+0000","last_change":"2023-10-08T05:52:26.601603+0000","last_active":"2023-10-08T05:52:28.047826+0000","last_peered":"2023-10-08T05:52:28.047826+0000","last_clean":"2023-10-08T05:52:28.047826+0000","last_became_active":"2023-10-08T05:52:26.601115+0000","last_became_peered":"2023-10-08T05:52:26.601115+0000","last_unstale":"2023-10-08T05:52:28.047826+0000","last_undegraded":"2023-10-08T05:52:28.047826+0000","last_fullsized":"2023-10-08T05:52:28.047826+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:35:24.840874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613920+0000","last_change":"2023-10-08T05:52:26.612015+0000","last_active":"2023-10-08T05:52:27.613920+0000","last_peered":"2023-10-08T05:52:27.613920+0000","last_clean":"2023-10-08T05:52:27.613920+0000","last_became_active":"2023-10-08T05:52:26.611746+0000","last_became_peered":"2023-10-08T05:52:26.611746+0000","last_unstale":"2023-10-08T05:52:27.613920+0000","last_undegraded":"2023-10-08T05:52:27.613920+0000","last_fullsized":"2023-10-08T05:52:27.613920+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:01.282002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"89'4","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.631213+0000","last_change":"2023-10-08T05:52:25.616620+0000","last_active":"2023-10-08T05:52:26.631213+0000","last_peered":"2023-10-08T05:52:26.631213+0000","last_clean":"2023-10-08T05:52:26.631213+0000","last_became_active":"2023-10-08T05:52:25.615871+0000","last_became_peered":"2023-10-08T05:52:25.615871+0000","last_unstale":"2023-10-08T05:52:26.631213+0000","last_undegraded":"2023-10-08T05:52:26.631213+0000","last_fullsized":"2023-10-08T05:52:26.631213+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:45.439387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.614004+0000","last_change":"2023-10-08T05:52:26.614195+0000","last_active":"2023-10-08T05:52:27.614004+0000","last_peered":"2023-10-08T05:52:27.614004+0000","last_clean":"2023-10-08T05:52:27.614004+0000","last_became_active":"2023-10-08T05:52:26.613908+0000","last_became_peered":"2023-10-08T05:52:26.613908+0000","last_unstale":"2023-10-08T05:52:27.614004+0000","last_undegraded":"2023-10-08T05:52:27.614004+0000","last_fullsized":"2023-10-08T05:52:27.614004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:17:08.671742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.630388+0000","last_change":"2023-10-08T05:52:25.615974+0000","last_active":"2023-10-08T05:52:26.630388+0000","last_peered":"2023-10-08T05:52:26.630388+0000","last_clean":"2023-10-08T05:52:26.630388+0000","last_became_active":"2023-10-08T05:52:25.615696+0000","last_became_peered":"2023-10-08T05:52:25.615696+0000","last_unstale":"2023-10-08T05:52:26.630388+0000","last_undegraded":"2023-10-08T05:52:26.630388+0000","last_fullsized":"2023-10-08T05:52:26.630388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:47.890265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.049215+0000","last_change":"2023-10-08T05:52:26.596720+0000","last_active":"2023-10-08T05:52:28.049215+0000","last_peered":"2023-10-08T05:52:28.049215+0000","last_clean":"2023-10-08T05:52:28.049215+0000","last_became_active":"2023-10-08T05:52:26.595091+0000","last_became_peered":"2023-10-08T05:52:26.595091+0000","last_unstale":"2023-10-08T05:52:28.049215+0000","last_undegraded":"2023-10-08T05:52:28.049215+0000","last_fullsized":"2023-10-08T05:52:28.049215+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:01.346509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.623090+0000","last_change":"2023-10-08T05:52:25.613203+0000","last_active":"2023-10-08T05:52:26.623090+0000","last_peered":"2023-10-08T05:52:26.623090+0000","last_clean":"2023-10-08T05:52:26.623090+0000","last_became_active":"2023-10-08T05:52:25.612769+0000","last_became_peered":"2023-10-08T05:52:25.612769+0000","last_unstale":"2023-10-08T05:52:26.623090+0000","last_undegraded":"2023-10-08T05:52:26.623090+0000","last_fullsized":"2023-10-08T05:52:26.623090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:16.139641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.633365+0000","last_change":"2023-10-08T05:52:26.630623+0000","last_active":"2023-10-08T05:52:27.633365+0000","last_peered":"2023-10-08T05:52:27.633365+0000","last_clean":"2023-10-08T05:52:27.633365+0000","last_became_active":"2023-10-08T05:52:26.630230+0000","last_became_peered":"2023-10-08T05:52:26.630230+0000","last_unstale":"2023-10-08T05:52:27.633365+0000","last_undegraded":"2023-10-08T05:52:27.633365+0000","last_fullsized":"2023-10-08T05:52:27.633365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:26.091385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.624028+0000","last_change":"2023-10-08T05:52:25.609439+0000","last_active":"2023-10-08T05:52:26.624028+0000","last_peered":"2023-10-08T05:52:26.624028+0000","last_clean":"2023-10-08T05:52:26.624028+0000","last_became_active":"2023-10-08T05:52:25.609182+0000","last_became_peered":"2023-10-08T05:52:25.609182+0000","last_unstale":"2023-10-08T05:52:26.624028+0000","last_undegraded":"2023-10-08T05:52:26.624028+0000","last_fullsized":"2023-10-08T05:52:26.624028+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:07.972986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.045859+0000","last_change":"2023-10-08T05:52:26.613321+0000","last_active":"2023-10-08T05:52:28.045859+0000","last_peered":"2023-10-08T05:52:28.045859+0000","last_clean":"2023-10-08T05:52:28.045859+0000","last_became_active":"2023-10-08T05:52:26.613042+0000","last_became_peered":"2023-10-08T05:52:26.613042+0000","last_unstale":"2023-10-08T05:52:28.045859+0000","last_undegraded":"2023-10-08T05:52:28.045859+0000","last_fullsized":"2023-10-08T05:52:28.045859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:38.210900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.624711+0000","last_change":"2023-10-08T05:52:25.602006+0000","last_active":"2023-10-08T05:52:26.624711+0000","last_peered":"2023-10-08T05:52:26.624711+0000","last_clean":"2023-10-08T05:52:26.624711+0000","last_became_active":"2023-10-08T05:52:25.601714+0000","last_became_peered":"2023-10-08T05:52:25.601714+0000","last_unstale":"2023-10-08T05:52:26.624711+0000","last_undegraded":"2023-10-08T05:52:26.624711+0000","last_fullsized":"2023-10-08T05:52:26.624711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:31.639423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.047432+0000","last_change":"2023-10-08T05:52:26.605222+0000","last_active":"2023-10-08T05:52:28.047432+0000","last_peered":"2023-10-08T05:52:28.047432+0000","last_clean":"2023-10-08T05:52:28.047432+0000","last_became_active":"2023-10-08T05:52:26.604900+0000","last_became_peered":"2023-10-08T05:52:26.604900+0000","last_unstale":"2023-10-08T05:52:28.047432+0000","last_undegraded":"2023-10-08T05:52:28.047432+0000","last_fullsized":"2023-10-08T05:52:28.047432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:45.145436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.622381+0000","last_change":"2023-10-08T05:52:25.602167+0000","last_active":"2023-10-08T05:52:26.622381+0000","last_peered":"2023-10-08T05:52:26.622381+0000","last_clean":"2023-10-08T05:52:26.622381+0000","last_became_active":"2023-10-08T05:52:25.601901+0000","last_became_peered":"2023-10-08T05:52:25.601901+0000","last_unstale":"2023-10-08T05:52:26.622381+0000","last_undegraded":"2023-10-08T05:52:26.622381+0000","last_fullsized":"2023-10-08T05:52:26.622381+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:25.075678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612521+0000","last_change":"2023-10-08T05:52:26.604564+0000","last_active":"2023-10-08T05:52:27.612521+0000","last_peered":"2023-10-08T05:52:27.612521+0000","last_clean":"2023-10-08T05:52:27.612521+0000","last_became_active":"2023-10-08T05:52:26.604285+0000","last_became_peered":"2023-10-08T05:52:26.604285+0000","last_unstale":"2023-10-08T05:52:27.612521+0000","last_undegraded":"2023-10-08T05:52:27.612521+0000","last_fullsized":"2023-10-08T05:52:27.612521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:12:12.176743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.588261+0000","last_change":"2023-10-08T05:52:25.613579+0000","last_active":"2023-10-08T05:52:26.588261+0000","last_peered":"2023-10-08T05:52:26.588261+0000","last_clean":"2023-10-08T05:52:26.588261+0000","last_became_active":"2023-10-08T05:52:25.613286+0000","last_became_peered":"2023-10-08T05:52:25.613286+0000","last_unstale":"2023-10-08T05:52:26.588261+0000","last_undegraded":"2023-10-08T05:52:26.588261+0000","last_fullsized":"2023-10-08T05:52:26.588261+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:32.225838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.609775+0000","last_change":"2023-10-08T05:52:26.602442+0000","last_active":"2023-10-08T05:52:27.609775+0000","last_peered":"2023-10-08T05:52:27.609775+0000","last_clean":"2023-10-08T05:52:27.609775+0000","last_became_active":"2023-10-08T05:52:26.602110+0000","last_became_peered":"2023-10-08T05:52:26.602110+0000","last_unstale":"2023-10-08T05:52:27.609775+0000","last_undegraded":"2023-10-08T05:52:27.609775+0000","last_fullsized":"2023-10-08T05:52:27.609775+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:18.945623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.592621+0000","last_change":"2023-10-08T05:52:25.618073+0000","last_active":"2023-10-08T05:52:26.592621+0000","last_peered":"2023-10-08T05:52:26.592621+0000","last_clean":"2023-10-08T05:52:26.592621+0000","last_became_active":"2023-10-08T05:52:25.617792+0000","last_became_peered":"2023-10-08T05:52:25.617792+0000","last_unstale":"2023-10-08T05:52:26.592621+0000","last_undegraded":"2023-10-08T05:52:26.592621+0000","last_fullsized":"2023-10-08T05:52:26.592621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:58.514719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.049020+0000","last_change":"2023-10-08T05:52:26.614707+0000","last_active":"2023-10-08T05:52:28.049020+0000","last_peered":"2023-10-08T05:52:28.049020+0000","last_clean":"2023-10-08T05:52:28.049020+0000","last_became_active":"2023-10-08T05:52:26.614374+0000","last_became_peered":"2023-10-08T05:52:26.614374+0000","last_unstale":"2023-10-08T05:52:28.049020+0000","last_undegraded":"2023-10-08T05:52:28.049020+0000","last_fullsized":"2023-10-08T05:52:28.049020+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:59.912311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.624630+0000","last_change":"2023-10-08T05:52:25.617521+0000","last_active":"2023-10-08T05:52:26.624630+0000","last_peered":"2023-10-08T05:52:26.624630+0000","last_clean":"2023-10-08T05:52:26.624630+0000","last_became_active":"2023-10-08T05:52:25.617269+0000","last_became_peered":"2023-10-08T05:52:25.617269+0000","last_unstale":"2023-10-08T05:52:26.624630+0000","last_undegraded":"2023-10-08T05:52:26.624630+0000","last_fullsized":"2023-10-08T05:52:26.624630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:36:51.312170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.610091+0000","last_change":"2023-10-08T05:52:26.607854+0000","last_active":"2023-10-08T05:52:27.610091+0000","last_peered":"2023-10-08T05:52:27.610091+0000","last_clean":"2023-10-08T05:52:27.610091+0000","last_became_active":"2023-10-08T05:52:26.607558+0000","last_became_peered":"2023-10-08T05:52:26.607558+0000","last_unstale":"2023-10-08T05:52:27.610091+0000","last_undegraded":"2023-10-08T05:52:27.610091+0000","last_fullsized":"2023-10-08T05:52:27.610091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:15.670901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.584660+0000","last_change":"2023-10-08T05:52:25.610640+0000","last_active":"2023-10-08T05:52:26.584660+0000","last_peered":"2023-10-08T05:52:26.584660+0000","last_clean":"2023-10-08T05:52:26.584660+0000","last_became_active":"2023-10-08T05:52:25.610375+0000","last_became_peered":"2023-10-08T05:52:25.610375+0000","last_unstale":"2023-10-08T05:52:26.584660+0000","last_undegraded":"2023-10-08T05:52:26.584660+0000","last_fullsized":"2023-10-08T05:52:26.584660+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:31.358719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612176+0000","last_change":"2023-10-08T05:52:26.606295+0000","last_active":"2023-10-08T05:52:27.612176+0000","last_peered":"2023-10-08T05:52:27.612176+0000","last_clean":"2023-10-08T05:52:27.612176+0000","last_became_active":"2023-10-08T05:52:26.605212+0000","last_became_peered":"2023-10-08T05:52:26.605212+0000","last_unstale":"2023-10-08T05:52:27.612176+0000","last_undegraded":"2023-10-08T05:52:27.612176+0000","last_fullsized":"2023-10-08T05:52:27.612176+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:25.764209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"89'3","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.636352+0000","last_change":"2023-10-08T05:52:25.619025+0000","last_active":"2023-10-08T05:52:26.636352+0000","last_peered":"2023-10-08T05:52:26.636352+0000","last_clean":"2023-10-08T05:52:26.636352+0000","last_became_active":"2023-10-08T05:52:25.618782+0000","last_became_peered":"2023-10-08T05:52:25.618782+0000","last_unstale":"2023-10-08T05:52:26.636352+0000","last_undegraded":"2023-10-08T05:52:26.636352+0000","last_fullsized":"2023-10-08T05:52:26.636352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:50:07.327419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613788+0000","last_change":"2023-10-08T05:52:26.600651+0000","last_active":"2023-10-08T05:52:27.613788+0000","last_peered":"2023-10-08T05:52:27.613788+0000","last_clean":"2023-10-08T05:52:27.613788+0000","last_became_active":"2023-10-08T05:52:26.593844+0000","last_became_peered":"2023-10-08T05:52:26.593844+0000","last_unstale":"2023-10-08T05:52:27.613788+0000","last_undegraded":"2023-10-08T05:52:27.613788+0000","last_fullsized":"2023-10-08T05:52:27.613788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:58:41.407210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"89'6","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.627996+0000","last_change":"2023-10-08T05:52:25.609882+0000","last_active":"2023-10-08T05:52:26.627996+0000","last_peered":"2023-10-08T05:52:26.627996+0000","last_clean":"2023-10-08T05:52:26.627996+0000","last_became_active":"2023-10-08T05:52:25.608449+0000","last_became_peered":"2023-10-08T05:52:25.608449+0000","last_unstale":"2023-10-08T05:52:26.627996+0000","last_undegraded":"2023-10-08T05:52:26.627996+0000","last_fullsized":"2023-10-08T05:52:26.627996+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:14.303833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.611498+0000","last_change":"2023-10-08T05:52:26.622980+0000","last_active":"2023-10-08T05:52:27.611498+0000","last_peered":"2023-10-08T05:52:27.611498+0000","last_clean":"2023-10-08T05:52:27.611498+0000","last_became_active":"2023-10-08T05:52:26.622710+0000","last_became_peered":"2023-10-08T05:52:26.622710+0000","last_unstale":"2023-10-08T05:52:27.611498+0000","last_undegraded":"2023-10-08T05:52:27.611498+0000","last_fullsized":"2023-10-08T05:52:27.611498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:54.741577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.635631+0000","last_change":"2023-10-08T05:52:25.617860+0000","last_active":"2023-10-08T05:52:26.635631+0000","last_peered":"2023-10-08T05:52:26.635631+0000","last_clean":"2023-10-08T05:52:26.635631+0000","last_became_active":"2023-10-08T05:52:25.617594+0000","last_became_peered":"2023-10-08T05:52:25.617594+0000","last_unstale":"2023-10-08T05:52:26.635631+0000","last_undegraded":"2023-10-08T05:52:26.635631+0000","last_fullsized":"2023-10-08T05:52:26.635631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:51.826639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046030+0000","last_change":"2023-10-08T05:52:26.613191+0000","last_active":"2023-10-08T05:52:28.046030+0000","last_peered":"2023-10-08T05:52:28.046030+0000","last_clean":"2023-10-08T05:52:28.046030+0000","last_became_active":"2023-10-08T05:52:26.612899+0000","last_became_peered":"2023-10-08T05:52:26.612899+0000","last_unstale":"2023-10-08T05:52:28.046030+0000","last_undegraded":"2023-10-08T05:52:28.046030+0000","last_fullsized":"2023-10-08T05:52:28.046030+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:12.299867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.598277+0000","last_change":"2023-10-08T05:52:25.609132+0000","last_active":"2023-10-08T05:52:26.598277+0000","last_peered":"2023-10-08T05:52:26.598277+0000","last_clean":"2023-10-08T05:52:26.598277+0000","last_became_active":"2023-10-08T05:52:25.608752+0000","last_became_peered":"2023-10-08T05:52:25.608752+0000","last_unstale":"2023-10-08T05:52:26.598277+0000","last_undegraded":"2023-10-08T05:52:26.598277+0000","last_fullsized":"2023-10-08T05:52:26.598277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:28:11.848537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612713+0000","last_change":"2023-10-08T05:52:26.602288+0000","last_active":"2023-10-08T05:52:27.612713+0000","last_peered":"2023-10-08T05:52:27.612713+0000","last_clean":"2023-10-08T05:52:27.612713+0000","last_became_active":"2023-10-08T05:52:26.601862+0000","last_became_peered":"2023-10-08T05:52:26.601862+0000","last_unstale":"2023-10-08T05:52:27.612713+0000","last_undegraded":"2023-10-08T05:52:27.612713+0000","last_fullsized":"2023-10-08T05:52:27.612713+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:26.008862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.624855+0000","last_change":"2023-10-08T05:52:25.618551+0000","last_active":"2023-10-08T05:52:26.624855+0000","last_peered":"2023-10-08T05:52:26.624855+0000","last_clean":"2023-10-08T05:52:26.624855+0000","last_became_active":"2023-10-08T05:52:25.618294+0000","last_became_peered":"2023-10-08T05:52:25.618294+0000","last_unstale":"2023-10-08T05:52:26.624855+0000","last_undegraded":"2023-10-08T05:52:26.624855+0000","last_fullsized":"2023-10-08T05:52:26.624855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:28:11.848537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.048162+0000","last_change":"2023-10-08T05:52:26.608848+0000","last_active":"2023-10-08T05:52:28.048162+0000","last_peered":"2023-10-08T05:52:28.048162+0000","last_clean":"2023-10-08T05:52:28.048162+0000","last_became_active":"2023-10-08T05:52:26.607816+0000","last_became_peered":"2023-10-08T05:52:26.607816+0000","last_unstale":"2023-10-08T05:52:28.048162+0000","last_undegraded":"2023-10-08T05:52:28.048162+0000","last_fullsized":"2023-10-08T05:52:28.048162+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:52:22.172494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"89'8","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.633455+0000","last_change":"2023-10-08T05:52:25.617851+0000","last_active":"2023-10-08T05:52:26.633455+0000","last_peered":"2023-10-08T05:52:26.633455+0000","last_clean":"2023-10-08T05:52:26.633455+0000","last_became_active":"2023-10-08T05:52:25.617547+0000","last_became_peered":"2023-10-08T05:52:25.617547+0000","last_unstale":"2023-10-08T05:52:26.633455+0000","last_undegraded":"2023-10-08T05:52:26.633455+0000","last_fullsized":"2023-10-08T05:52:26.633455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:29.256817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.633242+0000","last_change":"2023-10-08T05:52:26.630834+0000","last_active":"2023-10-08T05:52:27.633242+0000","last_peered":"2023-10-08T05:52:27.633242+0000","last_clean":"2023-10-08T05:52:27.633242+0000","last_became_active":"2023-10-08T05:52:26.630260+0000","last_became_peered":"2023-10-08T05:52:26.630260+0000","last_unstale":"2023-10-08T05:52:27.633242+0000","last_undegraded":"2023-10-08T05:52:27.633242+0000","last_fullsized":"2023-10-08T05:52:27.633242+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:12.299867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.623910+0000","last_change":"2023-10-08T05:52:25.615489+0000","last_active":"2023-10-08T05:52:26.623910+0000","last_peered":"2023-10-08T05:52:26.623910+0000","last_clean":"2023-10-08T05:52:26.623910+0000","last_became_active":"2023-10-08T05:52:25.615202+0000","last_became_peered":"2023-10-08T05:52:25.615202+0000","last_unstale":"2023-10-08T05:52:26.623910+0000","last_undegraded":"2023-10-08T05:52:26.623910+0000","last_fullsized":"2023-10-08T05:52:26.623910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:31.639423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613460+0000","last_change":"2023-10-08T05:52:26.601176+0000","last_active":"2023-10-08T05:52:27.613460+0000","last_peered":"2023-10-08T05:52:27.613460+0000","last_clean":"2023-10-08T05:52:27.613460+0000","last_became_active":"2023-10-08T05:52:26.600520+0000","last_became_peered":"2023-10-08T05:52:26.600520+0000","last_unstale":"2023-10-08T05:52:27.613460+0000","last_undegraded":"2023-10-08T05:52:27.613460+0000","last_fullsized":"2023-10-08T05:52:27.613460+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:30.612508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.635622+0000","last_change":"2023-10-08T05:52:25.620400+0000","last_active":"2023-10-08T05:52:26.635622+0000","last_peered":"2023-10-08T05:52:26.635622+0000","last_clean":"2023-10-08T05:52:26.635622+0000","last_became_active":"2023-10-08T05:52:25.620067+0000","last_became_peered":"2023-10-08T05:52:25.620067+0000","last_unstale":"2023-10-08T05:52:26.635622+0000","last_undegraded":"2023-10-08T05:52:26.635622+0000","last_fullsized":"2023-10-08T05:52:26.635622+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:09.438710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613404+0000","last_change":"2023-10-08T05:52:26.615102+0000","last_active":"2023-10-08T05:52:27.613404+0000","last_peered":"2023-10-08T05:52:27.613404+0000","last_clean":"2023-10-08T05:52:27.613404+0000","last_became_active":"2023-10-08T05:52:26.614801+0000","last_became_peered":"2023-10-08T05:52:26.614801+0000","last_unstale":"2023-10-08T05:52:27.613404+0000","last_undegraded":"2023-10-08T05:52:27.613404+0000","last_fullsized":"2023-10-08T05:52:27.613404+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:05.974014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"89'1","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.636114+0000","last_change":"2023-10-08T05:52:25.620173+0000","last_active":"2023-10-08T05:52:26.636114+0000","last_peered":"2023-10-08T05:52:26.636114+0000","last_clean":"2023-10-08T05:52:26.636114+0000","last_became_active":"2023-10-08T05:52:25.619012+0000","last_became_peered":"2023-10-08T05:52:25.619012+0000","last_unstale":"2023-10-08T05:52:26.636114+0000","last_undegraded":"2023-10-08T05:52:26.636114+0000","last_fullsized":"2023-10-08T05:52:26.636114+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:40.039160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.614191+0000","last_change":"2023-10-08T05:52:26.606637+0000","last_active":"2023-10-08T05:52:27.614191+0000","last_peered":"2023-10-08T05:52:27.614191+0000","last_clean":"2023-10-08T05:52:27.614191+0000","last_became_active":"2023-10-08T05:52:26.605349+0000","last_became_peered":"2023-10-08T05:52:26.605349+0000","last_unstale":"2023-10-08T05:52:27.614191+0000","last_undegraded":"2023-10-08T05:52:27.614191+0000","last_fullsized":"2023-10-08T05:52:27.614191+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:17.359233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.590043+0000","last_change":"2023-10-08T05:52:25.613656+0000","last_active":"2023-10-08T05:52:26.590043+0000","last_peered":"2023-10-08T05:52:26.590043+0000","last_clean":"2023-10-08T05:52:26.590043+0000","last_became_active":"2023-10-08T05:52:25.613319+0000","last_became_peered":"2023-10-08T05:52:25.613319+0000","last_unstale":"2023-10-08T05:52:26.590043+0000","last_undegraded":"2023-10-08T05:52:26.590043+0000","last_fullsized":"2023-10-08T05:52:26.590043+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:11.284159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.047936+0000","last_change":"2023-10-08T05:52:26.596496+0000","last_active":"2023-10-08T05:52:28.047936+0000","last_peered":"2023-10-08T05:52:28.047936+0000","last_clean":"2023-10-08T05:52:28.047936+0000","last_became_active":"2023-10-08T05:52:26.595929+0000","last_became_peered":"2023-10-08T05:52:26.595929+0000","last_unstale":"2023-10-08T05:52:28.047936+0000","last_undegraded":"2023-10-08T05:52:28.047936+0000","last_fullsized":"2023-10-08T05:52:28.047936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:18:16.856795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.623985+0000","last_change":"2023-10-08T05:52:25.609865+0000","last_active":"2023-10-08T05:52:26.623985+0000","last_peered":"2023-10-08T05:52:26.623985+0000","last_clean":"2023-10-08T05:52:26.623985+0000","last_became_active":"2023-10-08T05:52:25.609580+0000","last_became_peered":"2023-10-08T05:52:25.609580+0000","last_unstale":"2023-10-08T05:52:26.623985+0000","last_undegraded":"2023-10-08T05:52:26.623985+0000","last_fullsized":"2023-10-08T05:52:26.623985+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:42.160986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046180+0000","last_change":"2023-10-08T05:52:26.606852+0000","last_active":"2023-10-08T05:52:28.046180+0000","last_peered":"2023-10-08T05:52:28.046180+0000","last_clean":"2023-10-08T05:52:28.046180+0000","last_became_active":"2023-10-08T05:52:26.606540+0000","last_became_peered":"2023-10-08T05:52:26.606540+0000","last_unstale":"2023-10-08T05:52:28.046180+0000","last_undegraded":"2023-10-08T05:52:28.046180+0000","last_fullsized":"2023-10-08T05:52:28.046180+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:44:26.091385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.598610+0000","last_change":"2023-10-08T05:52:25.609029+0000","last_active":"2023-10-08T05:52:26.598610+0000","last_peered":"2023-10-08T05:52:26.598610+0000","last_clean":"2023-10-08T05:52:26.598610+0000","last_became_active":"2023-10-08T05:52:25.608342+0000","last_became_peered":"2023-10-08T05:52:25.608342+0000","last_unstale":"2023-10-08T05:52:26.598610+0000","last_undegraded":"2023-10-08T05:52:26.598610+0000","last_fullsized":"2023-10-08T05:52:26.598610+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:32.225838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612900+0000","last_change":"2023-10-08T05:52:26.604408+0000","last_active":"2023-10-08T05:52:27.612900+0000","last_peered":"2023-10-08T05:52:27.612900+0000","last_clean":"2023-10-08T05:52:27.612900+0000","last_became_active":"2023-10-08T05:52:26.604112+0000","last_became_peered":"2023-10-08T05:52:26.604112+0000","last_unstale":"2023-10-08T05:52:27.612900+0000","last_undegraded":"2023-10-08T05:52:27.612900+0000","last_fullsized":"2023-10-08T05:52:27.612900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:55.747441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.577587+0000","last_change":"2023-10-08T05:52:25.615086+0000","last_active":"2023-10-08T05:52:26.577587+0000","last_peered":"2023-10-08T05:52:26.577587+0000","last_clean":"2023-10-08T05:52:26.577587+0000","last_became_active":"2023-10-08T05:52:25.614831+0000","last_became_peered":"2023-10-08T05:52:25.614831+0000","last_unstale":"2023-10-08T05:52:26.577587+0000","last_undegraded":"2023-10-08T05:52:26.577587+0000","last_fullsized":"2023-10-08T05:52:26.577587+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:22.367245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.048964+0000","last_change":"2023-10-08T05:52:26.598757+0000","last_active":"2023-10-08T05:52:28.048964+0000","last_peered":"2023-10-08T05:52:28.048964+0000","last_clean":"2023-10-08T05:52:28.048964+0000","last_became_active":"2023-10-08T05:52:26.598272+0000","last_became_peered":"2023-10-08T05:52:26.598272+0000","last_unstale":"2023-10-08T05:52:28.048964+0000","last_undegraded":"2023-10-08T05:52:28.048964+0000","last_fullsized":"2023-10-08T05:52:28.048964+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:09.362614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"89'2","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.628873+0000","last_change":"2023-10-08T05:52:25.617759+0000","last_active":"2023-10-08T05:52:26.628873+0000","last_peered":"2023-10-08T05:52:26.628873+0000","last_clean":"2023-10-08T05:52:26.628873+0000","last_became_active":"2023-10-08T05:52:25.617463+0000","last_became_peered":"2023-10-08T05:52:25.617463+0000","last_unstale":"2023-10-08T05:52:26.628873+0000","last_undegraded":"2023-10-08T05:52:26.628873+0000","last_fullsized":"2023-10-08T05:52:26.628873+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:41.277667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046632+0000","last_change":"2023-10-08T05:52:26.605885+0000","last_active":"2023-10-08T05:52:28.046632+0000","last_peered":"2023-10-08T05:52:28.046632+0000","last_clean":"2023-10-08T05:52:28.046632+0000","last_became_active":"2023-10-08T05:52:26.605612+0000","last_became_peered":"2023-10-08T05:52:26.605612+0000","last_unstale":"2023-10-08T05:52:28.046632+0000","last_undegraded":"2023-10-08T05:52:28.046632+0000","last_fullsized":"2023-10-08T05:52:28.046632+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:32.304664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.600407+0000","last_change":"2023-10-08T05:52:25.612338+0000","last_active":"2023-10-08T05:52:26.600407+0000","last_peered":"2023-10-08T05:52:26.600407+0000","last_clean":"2023-10-08T05:52:26.600407+0000","last_became_active":"2023-10-08T05:52:25.611999+0000","last_became_peered":"2023-10-08T05:52:25.611999+0000","last_unstale":"2023-10-08T05:52:26.600407+0000","last_undegraded":"2023-10-08T05:52:26.600407+0000","last_fullsized":"2023-10-08T05:52:26.600407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:54:19.991948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046581+0000","last_change":"2023-10-08T05:52:26.612721+0000","last_active":"2023-10-08T05:52:28.046581+0000","last_peered":"2023-10-08T05:52:28.046581+0000","last_clean":"2023-10-08T05:52:28.046581+0000","last_became_active":"2023-10-08T05:52:26.612404+0000","last_became_peered":"2023-10-08T05:52:26.612404+0000","last_unstale":"2023-10-08T05:52:28.046581+0000","last_undegraded":"2023-10-08T05:52:28.046581+0000","last_fullsized":"2023-10-08T05:52:28.046581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:10.253744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.578269+0000","last_change":"2023-10-08T05:52:25.613025+0000","last_active":"2023-10-08T05:52:26.578269+0000","last_peered":"2023-10-08T05:52:26.578269+0000","last_clean":"2023-10-08T05:52:26.578269+0000","last_became_active":"2023-10-08T05:52:25.612435+0000","last_became_peered":"2023-10-08T05:52:25.612435+0000","last_unstale":"2023-10-08T05:52:26.578269+0000","last_undegraded":"2023-10-08T05:52:26.578269+0000","last_fullsized":"2023-10-08T05:52:26.578269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:27.932459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.614294+0000","last_change":"2023-10-08T05:52:26.607665+0000","last_active":"2023-10-08T05:52:27.614294+0000","last_peered":"2023-10-08T05:52:27.614294+0000","last_clean":"2023-10-08T05:52:27.614294+0000","last_became_active":"2023-10-08T05:52:26.607345+0000","last_became_peered":"2023-10-08T05:52:26.607345+0000","last_unstale":"2023-10-08T05:52:27.614294+0000","last_undegraded":"2023-10-08T05:52:27.614294+0000","last_fullsized":"2023-10-08T05:52:27.614294+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:53.632360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.598649+0000","last_change":"2023-10-08T05:52:25.608395+0000","last_active":"2023-10-08T05:52:26.598649+0000","last_peered":"2023-10-08T05:52:26.598649+0000","last_clean":"2023-10-08T05:52:26.598649+0000","last_became_active":"2023-10-08T05:52:25.608087+0000","last_became_peered":"2023-10-08T05:52:25.608087+0000","last_unstale":"2023-10-08T05:52:26.598649+0000","last_undegraded":"2023-10-08T05:52:26.598649+0000","last_fullsized":"2023-10-08T05:52:26.598649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:58:40.391503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.612177+0000","last_change":"2023-10-08T05:52:26.612662+0000","last_active":"2023-10-08T05:52:27.612177+0000","last_peered":"2023-10-08T05:52:27.612177+0000","last_clean":"2023-10-08T05:52:27.612177+0000","last_became_active":"2023-10-08T05:52:26.612390+0000","last_became_peered":"2023-10-08T05:52:26.612390+0000","last_unstale":"2023-10-08T05:52:27.612177+0000","last_undegraded":"2023-10-08T05:52:27.612177+0000","last_fullsized":"2023-10-08T05:52:27.612177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:38.608854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.623792+0000","last_change":"2023-10-08T05:52:25.618059+0000","last_active":"2023-10-08T05:52:26.623792+0000","last_peered":"2023-10-08T05:52:26.623792+0000","last_clean":"2023-10-08T05:52:26.623792+0000","last_became_active":"2023-10-08T05:52:25.617759+0000","last_became_peered":"2023-10-08T05:52:25.617759+0000","last_unstale":"2023-10-08T05:52:26.623792+0000","last_undegraded":"2023-10-08T05:52:26.623792+0000","last_fullsized":"2023-10-08T05:52:26.623792+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:59.721424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.611943+0000","last_change":"2023-10-08T05:52:26.613844+0000","last_active":"2023-10-08T05:52:27.611943+0000","last_peered":"2023-10-08T05:52:27.611943+0000","last_clean":"2023-10-08T05:52:27.611943+0000","last_became_active":"2023-10-08T05:52:26.613544+0000","last_became_peered":"2023-10-08T05:52:26.613544+0000","last_unstale":"2023-10-08T05:52:27.611943+0000","last_undegraded":"2023-10-08T05:52:27.611943+0000","last_fullsized":"2023-10-08T05:52:27.611943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:05.774388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.591216+0000","last_change":"2023-10-08T05:52:25.608974+0000","last_active":"2023-10-08T05:52:26.591216+0000","last_peered":"2023-10-08T05:52:26.591216+0000","last_clean":"2023-10-08T05:52:26.591216+0000","last_became_active":"2023-10-08T05:52:25.608723+0000","last_became_peered":"2023-10-08T05:52:25.608723+0000","last_unstale":"2023-10-08T05:52:26.591216+0000","last_undegraded":"2023-10-08T05:52:26.591216+0000","last_fullsized":"2023-10-08T05:52:26.591216+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:47.066046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.048624+0000","last_change":"2023-10-08T05:52:26.605412+0000","last_active":"2023-10-08T05:52:28.048624+0000","last_peered":"2023-10-08T05:52:28.048624+0000","last_clean":"2023-10-08T05:52:28.048624+0000","last_became_active":"2023-10-08T05:52:26.605096+0000","last_became_peered":"2023-10-08T05:52:26.605096+0000","last_unstale":"2023-10-08T05:52:28.048624+0000","last_undegraded":"2023-10-08T05:52:28.048624+0000","last_fullsized":"2023-10-08T05:52:28.048624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:16.473131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.588720+0000","last_change":"2023-10-08T05:52:25.613350+0000","last_active":"2023-10-08T05:52:26.588720+0000","last_peered":"2023-10-08T05:52:26.588720+0000","last_clean":"2023-10-08T05:52:26.588720+0000","last_became_active":"2023-10-08T05:52:25.613088+0000","last_became_peered":"2023-10-08T05:52:25.613088+0000","last_unstale":"2023-10-08T05:52:26.588720+0000","last_undegraded":"2023-10-08T05:52:26.588720+0000","last_fullsized":"2023-10-08T05:52:26.588720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:42.001395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:28.046040+0000","last_change":"2023-10-08T05:52:26.610157+0000","last_active":"2023-10-08T05:52:28.046040+0000","last_peered":"2023-10-08T05:52:28.046040+0000","last_clean":"2023-10-08T05:52:28.046040+0000","last_became_active":"2023-10-08T05:52:26.609890+0000","last_became_peered":"2023-10-08T05:52:26.609890+0000","last_unstale":"2023-10-08T05:52:28.046040+0000","last_undegraded":"2023-10-08T05:52:28.046040+0000","last_fullsized":"2023-10-08T05:52:28.046040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:12.964611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.597188+0000","last_change":"2023-10-08T05:52:25.612249+0000","last_active":"2023-10-08T05:52:26.597188+0000","last_peered":"2023-10-08T05:52:26.597188+0000","last_clean":"2023-10-08T05:52:26.597188+0000","last_became_active":"2023-10-08T05:52:25.611988+0000","last_became_peered":"2023-10-08T05:52:25.611988+0000","last_unstale":"2023-10-08T05:52:26.597188+0000","last_undegraded":"2023-10-08T05:52:26.597188+0000","last_fullsized":"2023-10-08T05:52:26.597188+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:04.958307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T05:52:27.613792+0000","last_change":"2023-10-08T05:52:26.611381+0000","last_active":"2023-10-08T05:52:27.613792+0000","last_peered":"2023-10-08T05:52:27.613792+0000","last_clean":"2023-10-08T05:52:27.613792+0000","last_became_active":"2023-10-08T05:52:26.611115+0000","last_became_peered":"2023-10-08T05:52:26.611115+0000","last_unstale":"2023-10-08T05:52:27.613792+0000","last_undegraded":"2023-10-08T05:52:27.613792+0000","last_fullsized":"2023-10-08T05:52:27.613792+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:25.564603+0000","last_clean_scrub_stamp":"2023-10-08T05:52:25.564603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:14.086252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.586840+0000","last_change":"2023-10-08T05:52:25.598181+0000","last_active":"2023-10-08T05:52:26.586840+0000","last_peered":"2023-10-08T05:52:26.586840+0000","last_clean":"2023-10-08T05:52:26.586840+0000","last_became_active":"2023-10-08T05:52:25.597570+0000","last_became_peered":"2023-10-08T05:52:25.597570+0000","last_unstale":"2023-10-08T05:52:26.586840+0000","last_undegraded":"2023-10-08T05:52:26.586840+0000","last_fullsized":"2023-10-08T05:52:26.586840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:52:24.548896+0000","last_clean_scrub_stamp":"2023-10-08T05:52:24.548896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:36:51.312170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"78'161","reported_seq":393,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-08T05:52:26.618922+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:52:26.618922+0000","last_peered":"2023-10-08T05:52:26.618922+0000","last_clean":"2023-10-08T05:52:26.618922+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:52:26.618922+0000","last_undegraded":"2023-10-08T05:52:26.618922+0000","last_fullsized":"2023-10-08T05:52:26.618922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575046,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59220,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312620,"statfs":{"total":96636764160,"available":96576122880,"internally_reserved":0,"allocated":1703936,"data_stored":1142963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1859999999999999,"15min":1.2030000000000001},"min":{"1min":0.77800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.2360000000000002},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.196,"15min":1.2030000000000001},"min":{"1min":0.754,"5min":0.44,"15min":0.44},"max":{"1min":1.6890000000000001,"5min":1.726,"15min":1.726},"last":1.4790000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.206,"15min":1.23},"min":{"1min":0.84899999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5509999999999999,"5min":1.6950000000000001,"15min":1.972},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.2749999999999999,"15min":1.2629999999999999},"min":{"1min":0.95699999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.736,"5min":1.7689999999999999,"15min":1.9379999999999999},"last":1.2669999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.2330000000000001,"15min":1.2210000000000001},"min":{"1min":0.90000000000000002,"5min":0.626,"15min":0.626},"max":{"1min":1.585,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.3260000000000001,"15min":1.329},"min":{"1min":0.91700000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.609,"5min":1.752,"15min":2.3849999999999998},"last":1.121}]},{"osd":3,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2589999999999999,"15min":1.2769999999999999},"min":{"1min":0.92800000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.639,"5min":1.8799999999999999,"15min":2.2919999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.3360000000000001,"15min":1.365},"min":{"1min":0.93999999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6930000000000001,"5min":1.8200000000000001,"15min":2.5579999999999998},"last":1.4419999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.294,"15min":1.2869999999999999},"min":{"1min":0.97499999999999998,"5min":0.504,"15min":0.504},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.4529999999999998},"last":1.167},{"interface":"front","average":{"1min":1.323,"5min":1.3069999999999999,"15min":1.3340000000000001},"min":{"1min":0.90700000000000003,"5min":0.63600000000000001,"15min":0.46800000000000003},"max":{"1min":1.796,"5min":2.0699999999999998,"15min":2.677},"last":1.4019999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3600000000000001,"15min":1.399},"min":{"1min":0.69499999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":2.0339999999999998,"5min":2.0339999999999998,"15min":3.3319999999999999},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.425,"15min":1.4279999999999999},"min":{"1min":1.042,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":2.726},"last":1.0669999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.5149999999999999,"15min":1.5249999999999999},"min":{"1min":1.2569999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.0680000000000001,"5min":2.2000000000000002,"15min":3.5249999999999999},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.446,"15min":1.46},"min":{"1min":1.0589999999999999,"5min":0.57799999999999996,"15min":0.52000000000000002},"max":{"1min":1.984,"5min":2.0150000000000001,"15min":2.4700000000000002},"last":1.6359999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1097728,"data_stored":551853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.286,"15min":1.2609999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.1429999999999998,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.9119999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.2509999999999999,"15min":1.2969999999999999},"min":{"1min":0.67400000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.8100000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.125,"15min":1.1579999999999999},"min":{"1min":0.73799999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6120000000000001,"5min":1.7789999999999999,"15min":1.9530000000000001},"last":1.393},{"interface":"front","average":{"1min":1.119,"5min":1.1699999999999999,"15min":1.177},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.5820000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1479999999999999,"15min":1.1839999999999999},"min":{"1min":0.86399999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.516,"5min":1.75,"15min":1.796},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.222,"15min":1.2589999999999999},"min":{"1min":0.92700000000000005,"5min":0.78300000000000003,"15min":0.58699999999999997},"max":{"1min":1.752,"5min":1.8520000000000001,"15min":1.8620000000000001},"last":1.0229999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.353,"15min":1.369},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.4019999999999999,"15min":1.4470000000000001},"min":{"1min":1.075,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.222}]},{"osd":4,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.292,"15min":1.3100000000000001},"min":{"1min":0.70399999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.633,"5min":1.8759999999999999,"15min":2.278},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3360000000000001,"15min":1.369},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.64300000000000002},"max":{"1min":1.881,"5min":1.881,"15min":2.3399999999999999},"last":1.629}]},{"osd":5,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.284,"15min":1.339},"min":{"1min":0.79500000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6930000000000001,"5min":1.7689999999999999,"15min":4.3099999999999996},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.4710000000000001,"15min":1.476},"min":{"1min":1.3140000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.927,"5min":1.927,"15min":2.1070000000000002},"last":2.012}]},{"osd":7,"last update":"Sun Oct 8 05:51:29 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.464,"15min":1.504},"min":{"1min":1.1439999999999999,"5min":0.97899999999999998,"15min":0.77200000000000002},"max":{"1min":1.8009999999999999,"5min":1.9159999999999999,"15min":2.3959999999999999},"last":1.956},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4399999999999999,"15min":1.476},"min":{"1min":1.1539999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.843,"5min":1.9710000000000001,"15min":4.1550000000000002},"last":1.7669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":551841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0640000000000001,"15min":1.115},"min":{"1min":0.72699999999999998,"5min":0.58599999999999997,"15min":0.46700000000000003},"max":{"1min":1.4219999999999999,"5min":1.627,"15min":2.302},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.80300000000000005,"5min":0.442,"15min":0.442},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":3.5529999999999999},"last":1.1950000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2290000000000001,"15min":1.208},"min":{"1min":0.85299999999999998,"5min":0.72999999999999998,"15min":0.72399999999999998},"max":{"1min":1.524,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.651},{"interface":"front","average":{"1min":1.292,"5min":1.2609999999999999,"15min":1.262},"min":{"1min":1.0209999999999999,"5min":0.81399999999999995,"15min":0.64700000000000002},"max":{"1min":1.579,"5min":1.754,"15min":2.1789999999999998},"last":1.0649999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.165,"15min":1.226},"min":{"1min":0.82599999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.698,"5min":1.7430000000000001,"15min":3.6739999999999999},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.29,"5min":1.2929999999999999,"15min":1.272},"min":{"1min":0.96399999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.7110000000000001,"5min":3.282,"15min":3.282},"last":1.5369999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3129999999999999,"15min":1.3460000000000001},"min":{"1min":0.97499999999999998,"5min":0.77700000000000002,"15min":0.63500000000000001},"max":{"1min":1.7609999999999999,"5min":2.1259999999999999,"15min":6.782},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.367,"15min":1.399},"min":{"1min":1.159,"5min":0.89700000000000002,"15min":0.84799999999999998},"max":{"1min":1.7230000000000001,"5min":1.8160000000000001,"15min":6.6189999999999998},"last":1.3640000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3029999999999999,"15min":1.3069999999999999},"min":{"1min":0.90300000000000002,"5min":0.82899999999999996,"15min":0.53600000000000003},"max":{"1min":1.619,"5min":4.0179999999999998,"15min":6.1349999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.458,"15min":1.429},"min":{"1min":1.175,"5min":0.71799999999999997,"15min":0.51400000000000001},"max":{"1min":1.9970000000000001,"5min":3.456,"15min":7.0069999999999997},"last":1.609}]},{"osd":6,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.383,"15min":1.4059999999999999},"min":{"1min":1.0289999999999999,"5min":0.83399999999999996,"15min":0.57699999999999996},"max":{"1min":1.663,"5min":1.8089999999999999,"15min":7.2569999999999997},"last":1.889},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4650000000000001,"15min":1.4690000000000001},"min":{"1min":1.149,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.7629999999999999,"5min":3.8620000000000001,"15min":6.2199999999999998},"last":1.746}]},{"osd":7,"last update":"Sun Oct 8 05:51:39 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.5529999999999999,"15min":1.5489999999999999},"min":{"1min":1.2829999999999999,"5min":1.0940000000000001,"15min":0.53900000000000003},"max":{"1min":1.9099999999999999,"5min":3.4079999999999999,"15min":6.9279999999999999},"last":1.7869999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.53,"15min":1.5580000000000001},"min":{"1min":1.137,"5min":0.91300000000000003,"15min":0.61699999999999999},"max":{"1min":1.806,"5min":1.996,"15min":7.335},"last":1.8260000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1105920,"data_stored":553431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.262,"15min":1.266},"min":{"1min":0.79900000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.7609999999999999,"5min":1.8009999999999999,"15min":1.863},"last":1.151},{"interface":"front","average":{"1min":1.359,"5min":1.3879999999999999,"15min":1.3799999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.0680000000000001},"last":1.403}]},{"osd":1,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.2989999999999999,"15min":1.274},"min":{"1min":0.86099999999999999,"5min":0.79300000000000004,"15min":0.65500000000000003},"max":{"1min":1.673,"5min":2.3519999999999999,"15min":2.3799999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.236,"15min":1.216},"min":{"1min":0.76000000000000001,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.544,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.2390000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.236,"15min":1.2170000000000001},"min":{"1min":0.82499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5920000000000001,"5min":1.6759999999999999,"15min":1.77},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3660000000000001,"15min":1.3580000000000001},"min":{"1min":0.96799999999999997,"5min":0.85599999999999998,"15min":0.748},"max":{"1min":1.7549999999999999,"5min":2.423,"15min":2.423},"last":1.3080000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.276,"15min":1.276},"min":{"1min":0.872,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.7110000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.079},{"interface":"front","average":{"1min":1.302,"5min":1.343,"15min":1.3520000000000001},"min":{"1min":0.998,"5min":0.90100000000000002,"15min":0.76900000000000002},"max":{"1min":1.875,"5min":1.875,"15min":2.1080000000000001},"last":0.83199999999999996}]},{"osd":5,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.353,"15min":1.341},"min":{"1min":0.91300000000000003,"5min":0.88500000000000001,"15min":0.45700000000000002},"max":{"1min":1.946,"5min":2.48,"15min":2.48},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.375},"min":{"1min":0.94399999999999995,"5min":0.92700000000000005,"15min":0.58199999999999996},"max":{"1min":1.6970000000000001,"5min":2.504,"15min":2.504},"last":1.3560000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.355,"15min":1.3620000000000001},"min":{"1min":0.90400000000000003,"5min":0.877,"15min":0.55000000000000004},"max":{"1min":1.946,"5min":1.9930000000000001,"15min":2.2080000000000002},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.522,"15min":1.514},"min":{"1min":1.2090000000000001,"5min":1.0660000000000001,"15min":0.70399999999999996},"max":{"1min":2.0299999999999998,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.5249999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:51:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.508,"15min":1.472},"min":{"1min":1.1719999999999999,"5min":1,"15min":0.73699999999999999},"max":{"1min":1.6950000000000001,"5min":4.125,"15min":4.125},"last":1.49},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5069999999999999,"15min":1.5369999999999999},"min":{"1min":1.0109999999999999,"5min":0.97199999999999998,"15min":0.67400000000000004},"max":{"1min":1.9970000000000001,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.2649999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575047,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1093632,"data_stored":551819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.93799999999999994,"15min":0.91300000000000003},"min":{"1min":0.70599999999999996,"5min":0.32100000000000001,"15min":0.29599999999999999},"max":{"1min":1.2729999999999999,"5min":1.5269999999999999,"15min":1.72},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.0760000000000001,"15min":1.0820000000000001},"min":{"1min":0.63700000000000001,"5min":0.45400000000000001,"15min":0.39000000000000001},"max":{"1min":1.4690000000000001,"5min":1.5,"15min":2.3199999999999998},"last":0.94399999999999995}]},{"osd":1,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.93999999999999995,"15min":0.95999999999999996},"min":{"1min":0.70999999999999996,"5min":0.501,"15min":0.373},"max":{"1min":1.387,"5min":1.5629999999999999,"15min":1.6040000000000001},"last":1.329},{"interface":"front","average":{"1min":1.25,"5min":1.1890000000000001,"15min":1.1930000000000001},"min":{"1min":0.66300000000000003,"5min":0.56599999999999995,"15min":0.436},"max":{"1min":1.5429999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.3959999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.069,"15min":1.0629999999999999},"min":{"1min":0.69999999999999996,"5min":0.54500000000000004,"15min":0.47699999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.056},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.1499999999999999,"15min":1.135},"min":{"1min":0.78600000000000003,"5min":0.60499999999999998,"15min":0.41899999999999998},"max":{"1min":1.571,"5min":1.6819999999999999,"15min":1.788},"last":1.232}]},{"osd":4,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2110000000000001,"15min":1.2050000000000001},"min":{"1min":0.86499999999999999,"5min":0.54000000000000004,"15min":0.34200000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.5059999999999998},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.3859999999999999,"15min":1.379},"min":{"1min":1.0529999999999999,"5min":0.72799999999999998,"15min":0.53800000000000003},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":1.5760000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3169999999999999,"15min":1.294},"min":{"1min":0.90400000000000003,"5min":0.73699999999999999,"15min":0.624},"max":{"1min":1.698,"5min":1.883,"15min":1.9219999999999999},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.3680000000000001,"15min":1.355},"min":{"1min":0.93500000000000005,"5min":0.78500000000000003,"15min":0.59599999999999997},"max":{"1min":2.0939999999999999,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.5329999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.296,"15min":1.3160000000000001},"min":{"1min":0.94699999999999995,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.742,"5min":1.776,"15min":2.6150000000000002},"last":1.468},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.4510000000000001,"15min":1.4410000000000001},"min":{"1min":1.1160000000000001,"5min":0.86699999999999999,"15min":0.70099999999999996},"max":{"1min":2.012,"5min":2.0489999999999999,"15min":2.1259999999999999},"last":1.498}]},{"osd":7,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.488,"15min":1.4830000000000001},"min":{"1min":1.026,"5min":0.82399999999999995,"15min":0.72299999999999998},"max":{"1min":1.956,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.619},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.373,"15min":1.3919999999999999},"min":{"1min":0.98399999999999999,"5min":0.77100000000000002,"15min":0.66900000000000004},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":2.7879999999999998},"last":1.363}]}]},{"osd":2,"up_from":13,"seq":55834575045,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58616,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313224,"statfs":{"total":96636764160,"available":96576741376,"internally_reserved":0,"allocated":1093632,"data_stored":551819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.88,"15min":0.90000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38,"15min":0.37},"max":{"1min":1.4950000000000001,"5min":1.5209999999999999,"15min":2.4009999999999998},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.0529999999999999,"15min":1.04},"min":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.47499999999999998},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.9470000000000001},"last":0.71799999999999997}]},{"osd":1,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93999999999999995,"15min":0.95499999999999996},"min":{"1min":0.66100000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":2.1070000000000002},"last":1.373},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0840000000000001,"15min":1.0980000000000001},"min":{"1min":0.58399999999999996,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.3029999999999999,"5min":1.706,"15min":2.1469999999999998},"last":0.95399999999999996}]},{"osd":3,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1060000000000001,"15min":1.131},"min":{"1min":0.70699999999999996,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6240000000000001,"5min":1.639,"15min":1.8120000000000001},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.262,"15min":1.2430000000000001},"min":{"1min":0.64900000000000002,"5min":0.59799999999999998,"15min":0.59399999999999997},"max":{"1min":1.552,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":1.181}]},{"osd":4,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.413,"15min":1.3939999999999999},"min":{"1min":0.88400000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.7989999999999999,"5min":4.593,"15min":4.593},"last":1.288},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.286,"15min":1.296},"min":{"1min":0.78600000000000003,"5min":0.63200000000000001,"15min":0.63100000000000001},"max":{"1min":1.881,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.21}]},{"osd":5,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.3839999999999999},"min":{"1min":0.84799999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.819,"5min":6.7140000000000004,"15min":6.7140000000000004},"last":1.429},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4390000000000001,"15min":1.4570000000000001},"min":{"1min":0.92900000000000005,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.72,"5min":2.0529999999999999,"15min":2.8170000000000002},"last":1.3480000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.355,"15min":1.357},"min":{"1min":0.93000000000000005,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8080000000000001,"5min":2.7080000000000002,"15min":2.851},"last":0.998},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.444,"15min":1.4610000000000001},"min":{"1min":0.88600000000000001,"5min":0.74099999999999999,"15min":0.63900000000000001},"max":{"1min":1.6950000000000001,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.2290000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:51:35 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.46,"15min":1.452},"min":{"1min":0.82499999999999996,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.8720000000000001,"5min":6.609,"15min":6.609},"last":1.125},{"interface":"front","average":{"1min":1.45,"5min":1.4750000000000001,"15min":1.5089999999999999},"min":{"1min":0.95899999999999996,"5min":0.82299999999999995,"15min":0.61799999999999999},"max":{"1min":1.825,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.323}]}]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335404,"statfs":{"total":96636764160,"available":96599453696,"internally_reserved":0,"allocated":1122304,"data_stored":554195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.95599999999999996,"15min":0.96399999999999997},"min":{"1min":0.439,"5min":0.39700000000000002,"15min":0.29799999999999999},"max":{"1min":1.3740000000000001,"5min":1.796,"15min":1.796},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.149,"5min":1.121,"15min":1.1299999999999999},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.433},"max":{"1min":1.4510000000000001,"5min":1.6060000000000001,"15min":1.8089999999999999},"last":1.0129999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.94199999999999995,"15min":0.95299999999999996},"min":{"1min":0.46899999999999997,"5min":0.39500000000000002,"15min":0.34899999999999998},"max":{"1min":1.3320000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.081,"15min":1.083},"min":{"1min":0.61799999999999999,"5min":0.53400000000000003,"15min":0.38200000000000001},"max":{"1min":1.331,"5min":1.77,"15min":2.0750000000000002},"last":0.81499999999999995}]},{"osd":3,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1439999999999999,"15min":1.1739999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.41899999999999998},"max":{"1min":1.6160000000000001,"5min":1.7549999999999999,"15min":1.8160000000000001},"last":1.242},{"interface":"front","average":{"1min":1.159,"5min":1.0720000000000001,"15min":1.069},"min":{"1min":0.622,"5min":0.46800000000000003,"15min":0.42799999999999999},"max":{"1min":1.585,"5min":1.788,"15min":1.788},"last":0.77100000000000002}]},{"osd":4,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.266,"15min":1.296},"min":{"1min":0.93600000000000005,"5min":0.79800000000000004,"15min":0.63900000000000001},"max":{"1min":1.538,"5min":1.966,"15min":1.966},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.24,"15min":1.2270000000000001},"min":{"1min":0.83399999999999996,"5min":0.72699999999999998,"15min":0.61799999999999999},"max":{"1min":1.5409999999999999,"5min":1.913,"15min":2.2029999999999998},"last":1.101}]},{"osd":5,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3100000000000001,"15min":1.3049999999999999},"min":{"1min":0.95199999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6140000000000001,"5min":1.8680000000000001,"15min":2.3399999999999999},"last":1.393},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4419999999999999,"15min":1.4390000000000001},"min":{"1min":1.04,"5min":0.86799999999999999,"15min":0.71599999999999997},"max":{"1min":1.76,"5min":2.077,"15min":2.3069999999999999},"last":1.375}]},{"osd":6,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.395,"15min":1.419},"min":{"1min":1.0720000000000001,"5min":0.89600000000000002,"15min":0.65500000000000003},"max":{"1min":1.6859999999999999,"5min":1.972,"15min":2.2490000000000001},"last":1.282},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4179999999999999,"15min":1.417},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.68300000000000005},"max":{"1min":1.681,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.169}]},{"osd":7,"last update":"Sun Oct 8 05:51:32 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.415,"15min":1.4570000000000001},"min":{"1min":0.88500000000000001,"5min":0.82699999999999996,"15min":0.79100000000000004},"max":{"1min":1.893,"5min":1.97,"15min":2.1869999999999998},"last":1.081},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4450000000000001,"15min":1.456},"min":{"1min":1.0089999999999999,"5min":0.98699999999999999,"15min":0.76300000000000001},"max":{"1min":1.698,"5min":2.286,"15min":2.4180000000000001},"last":1.347}]}]},{"osd":0,"up_from":13,"seq":55834575046,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36996,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334844,"statfs":{"total":96636764160,"available":96598880256,"internally_reserved":0,"allocated":1687552,"data_stored":1142187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89200000000000002,"15min":0.90500000000000003},"min":{"1min":0.5,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.7130000000000001},"last":1.002},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.042,"15min":1.026},"min":{"1min":0.58099999999999996,"5min":0.371,"15min":0.371},"max":{"1min":1.6719999999999999,"5min":1.675,"15min":2.3500000000000001},"last":0.78400000000000003}]},{"osd":2,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.073,"15min":1.079},"min":{"1min":0.80800000000000005,"5min":0.39200000000000002,"15min":0.38900000000000001},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.236},{"interface":"front","average":{"1min":1.0389999999999999,"5min":0.98799999999999999,"15min":0.995},"min":{"1min":0.54200000000000004,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.621,"5min":1.621,"15min":1.78},"last":0.90700000000000003}]},{"osd":3,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.1830000000000001,"15min":1.1950000000000001},"min":{"1min":0.71799999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":2.4540000000000002},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.0960000000000001,"15min":1.0820000000000001},"min":{"1min":0.65900000000000003,"5min":0.44700000000000001,"15min":0.439},"max":{"1min":1.7609999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.1899999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.284,"15min":1.268},"min":{"1min":0.94399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":2.496},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.321,"15min":1.3420000000000001},"min":{"1min":0.98499999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":2.6560000000000001},"last":1.3080000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.341,"15min":1.333},"min":{"1min":1.155,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.921,"5min":1.921,"15min":2.2650000000000001},"last":1.343},{"interface":"front","average":{"1min":1.45,"5min":1.341,"15min":1.3799999999999999},"min":{"1min":0.94399999999999995,"5min":0.63,"15min":0.63},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.7069999999999999},"last":1.8160000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.405,"15min":1.4099999999999999},"min":{"1min":1.2070000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.97,"5min":2.1909999999999998,"15min":2.8290000000000002},"last":1.399},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.385,"15min":1.3999999999999999},"min":{"1min":0.88900000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.4020000000000001},"last":1.4990000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:51:37 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.4239999999999999,"15min":1.4370000000000001},"min":{"1min":0.96399999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.746},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.466,"15min":1.466},"min":{"1min":1.042,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.04,"5min":2.1059999999999999,"15min":2.9300000000000002},"last":1.474}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:52:31.559 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T05:52:31.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T05:52:31.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.819+0000 7faa48911640 1 Processor -- start 2023-10-08T05:52:31.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.819+0000 7faa48911640 1 -- start start 2023-10-08T05:52:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.821+0000 7faa48911640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faa440cb970 0x7faa440ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.821+0000 7faa48911640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4415e490 0x7faa4415e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.821+0000 7faa48911640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa440c9c90 con 0x7faa4415de70 2023-10-08T05:52:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.821+0000 7faa48911640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa4415f480 con 0x7faa440cb970 2023-10-08T05:52:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.821+0000 7faa48911640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa4415e250 con 0x7faa4415e490 2023-10-08T05:52:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.821+0000 7faa42ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faa440cb970 0x7faa440ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.821+0000 7faa42ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faa440cb970 0x7faa440ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48996/0 (socket says 172.21.15.70:48996) 2023-10-08T05:52:31.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.821+0000 7faa42ffd640 1 -- 172.21.15.70:0/1907427165 learned_addr learned my addr 172.21.15.70:0/1907427165 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.821+0000 7faa43fff640 1 --2- 172.21.15.70:0/1907427165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4415e490 0x7faa4415e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.822+0000 7faa42ffd640 1 -- 172.21.15.70:0/1907427165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4415e490 msgr2=0x7faa4415e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.822+0000 7faa42ffd640 1 --2- 172.21.15.70:0/1907427165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4415e490 0x7faa4415e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.822+0000 7faa42ffd640 1 -- 172.21.15.70:0/1907427165 >> v1:172.21.15.181:6789/0 conn(0x7faa4415de70 legacy=0x7faa440cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.822+0000 7faa42ffd640 1 -- 172.21.15.70:0/1907427165 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4415f1a0 con 0x7faa440cb970 2023-10-08T05:52:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.822+0000 7faa43fff640 1 --2- 172.21.15.70:0/1907427165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4415e490 0x7faa4415e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:31.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.823+0000 7faa42ffd640 1 --2- 172.21.15.70:0/1907427165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa440cb970 0x7faa440ca9a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009870 tx=0x7faa2c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566a3a11566f6bb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:31.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.823+0000 7faa427fc640 1 -- 172.21.15.70:0/1907427165 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c031640 con 0x7faa440cb970 2023-10-08T05:52:31.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.823+0000 7faa427fc640 1 -- 172.21.15.70:0/1907427165 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c0317a0 con 0x7faa440cb970 2023-10-08T05:52:31.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.823+0000 7faa427fc640 1 -- 172.21.15.70:0/1907427165 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c031b30 con 0x7faa440cb970 2023-10-08T05:52:31.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.824+0000 7faa48911640 1 -- 172.21.15.70:0/1907427165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa440cb970 msgr2=0x7faa440ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:31.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.824+0000 7faa48911640 1 --2- 172.21.15.70:0/1907427165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa440cb970 0x7faa440ca9a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009870 tx=0x7faa2c02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.824+0000 7faa48911640 1 -- 172.21.15.70:0/1907427165 shutdown_connections 2023-10-08T05:52:31.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.824+0000 7faa48911640 1 --2- 172.21.15.70:0/1907427165 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa4415e490 0x7faa4415e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.824+0000 7faa48911640 1 --2- 172.21.15.70:0/1907427165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa440cb970 0x7faa440ca9a0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.824+0000 7faa48911640 1 -- 172.21.15.70:0/1907427165 >> 172.21.15.70:0/1907427165 conn(0x7faa440ce9f0 msgr2=0x7faa440cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:31.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.824+0000 7faa48911640 1 -- 172.21.15.70:0/1907427165 shutdown_connections 2023-10-08T05:52:31.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.824+0000 7faa48911640 1 -- 172.21.15.70:0/1907427165 wait complete. 2023-10-08T05:52:31.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.826+0000 7faa48911640 1 Processor -- start 2023-10-08T05:52:31.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.826+0000 7faa48911640 1 -- start start 2023-10-08T05:52:31.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.827+0000 7faa48911640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa440cb970 0x7faa44158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.827+0000 7faa48911640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4415de70 0x7faa44156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.827+0000 7faa48911640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4415e490 0x7faa441552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.827+0000 7faa48911640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440cc700 con 0x7faa4415de70 2023-10-08T05:52:31.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.827+0000 7faa48911640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa440c4b20 con 0x7faa4415e490 2023-10-08T05:52:31.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.827+0000 7faa48911640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440c4e00 con 0x7faa440cb970 2023-10-08T05:52:31.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.827+0000 7faa42ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4415de70 0x7faa44156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.827+0000 7faa42ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4415de70 0x7faa44156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49012/0 (socket says 172.21.15.70:49012) 2023-10-08T05:52:31.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.827+0000 7faa42ffd640 1 -- 172.21.15.70:0/2378033533 learned_addr learned my addr 172.21.15.70:0/2378033533 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa437fe640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa440cb970 0x7faa44158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa42ffd640 1 -- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa440cb970 msgr2=0x7faa44158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa42ffd640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa440cb970 0x7faa44158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa42ffd640 1 -- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4415e490 msgr2=0x7faa441552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa42ffd640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4415e490 0x7faa441552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa42ffd640 1 -- 172.21.15.70:0/2378033533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c0094d0 con 0x7faa4415de70 2023-10-08T05:52:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa42ffd640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4415de70 0x7faa44156ab0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009870 tx=0x7faa2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa3bfff640 1 -- 172.21.15.70:0/2378033533 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c03ac60 con 0x7faa4415de70 2023-10-08T05:52:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa3bfff640 1 -- 172.21.15.70:0/2378033533 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2c03ade0 con 0x7faa4415de70 2023-10-08T05:52:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa3bfff640 1 -- 172.21.15.70:0/2378033533 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa2c044630 con 0x7faa4415de70 2023-10-08T05:52:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa48911640 1 -- 172.21.15.70:0/2378033533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa440cc320 con 0x7faa4415de70 2023-10-08T05:52:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.828+0000 7faa48911640 1 -- 172.21.15.70:0/2378033533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa44156ff0 con 0x7faa4415de70 2023-10-08T05:52:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.829+0000 7faa48911640 1 -- 172.21.15.70:0/2378033533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa440671b0 con 0x7faa4415de70 2023-10-08T05:52:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.830+0000 7faa3bfff640 1 -- 172.21.15.70:0/2378033533 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7faa2c02f050 con 0x7faa4415de70 2023-10-08T05:52:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.831+0000 7faa3bfff640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7faa2406af90 0x7faa2406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.831+0000 7faa3bfff640 1 -- 172.21.15.70:0/2378033533 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faa2c0abb10 con 0x7faa4415de70 2023-10-08T05:52:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.832+0000 7faa437fe640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7faa2406af90 0x7faa2406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:31.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.833+0000 7faa437fe640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7faa2406af90 0x7faa2406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa34007fd0 tx=0x7faa34009040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:31.835+0000 7faa3bfff640 1 -- 172.21.15.70:0/2378033533 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa2c076830 con 0x7faa4415de70 2023-10-08T05:52:32.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.096+0000 7faa48911640 1 -- 172.21.15.70:0/2378033533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faa440cc840 con 0x7faa4415de70 2023-10-08T05:52:32.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.100+0000 7faa3bfff640 1 -- 172.21.15.70:0/2378033533 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7faa440cc840 con 0x7faa4415de70 2023-10-08T05:52:32.102 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T05:52:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T05:52:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T05:52:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T05:52:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T05:52:32.102 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T05:52:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 -- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7faa2406af90 msgr2=0x7faa2406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:32.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7faa2406af90 0x7faa2406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa34007fd0 tx=0x7faa34009040 comp rx=0 tx=0).stop 2023-10-08T05:52:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 -- 172.21.15.70:0/2378033533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4415de70 msgr2=0x7faa44156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4415de70 0x7faa44156ab0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009870 tx=0x7faa2c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 -- 172.21.15.70:0/2378033533 shutdown_connections 2023-10-08T05:52:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7faa2406af90 0x7faa2406d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa4415e490 0x7faa441552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa4415de70 0x7faa44156ab0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 --2- 172.21.15.70:0/2378033533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa440cb970 0x7faa44158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 -- 172.21.15.70:0/2378033533 >> 172.21.15.70:0/2378033533 conn(0x7faa440ce9f0 msgr2=0x7faa440ac040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 -- 172.21.15.70:0/2378033533 shutdown_connections 2023-10-08T05:52:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.103+0000 7faa48911640 1 -- 172.21.15.70:0/2378033533 wait complete. 2023-10-08T05:52:32.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:32.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.329+0000 7fe9c2052640 1 Processor -- start 2023-10-08T05:52:32.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.329+0000 7fe9c2052640 1 -- start start 2023-10-08T05:52:32.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.331+0000 7fe9c2052640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 0x7fe9bc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:32.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.331+0000 7fe9c2052640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe9bc0cb580 0x7fe9bc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:32.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.331+0000 7fe9c2052640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc15b2b0 con 0x7fe9bc0c90b0 2023-10-08T05:52:32.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.331+0000 7fe9c2052640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc15bf30 con 0x7fe9bc0cb580 2023-10-08T05:52:32.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.331+0000 7fe9c2052640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc15f7a0 con 0x7fe9bc15ab40 2023-10-08T05:52:32.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.331+0000 7fe9bbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe9bc0cb580 0x7fe9bc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:32.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.331+0000 7fe9baffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe9bc0c90b0 0x7fe9bc0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55192/0 (socket says 172.21.15.70:55192) 2023-10-08T05:52:32.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.331+0000 7fe9baffd640 1 -- 172.21.15.70:0/2706677189 learned_addr learned my addr 172.21.15.70:0/2706677189 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:32.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.332+0000 7fe9bbfff640 1 -- 172.21.15.70:0/2706677189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 msgr2=0x7fe9bc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:32.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.332+0000 7fe9bb7fe640 1 --2- 172.21.15.70:0/2706677189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 0x7fe9bc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:32.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.332+0000 7fe9bbfff640 1 --2- 172.21.15.70:0/2706677189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 0x7fe9bc0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.332+0000 7fe9bbfff640 1 -- 172.21.15.70:0/2706677189 >> v1:172.21.15.70:6789/0 conn(0x7fe9bc0c90b0 legacy=0x7fe9bc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:32.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.332+0000 7fe9bbfff640 1 -- 172.21.15.70:0/2706677189 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc15b590 con 0x7fe9bc0cb580 2023-10-08T05:52:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.332+0000 7fe9ba7fc640 1 -- 172.21.15.70:0/2706677189 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303956527 0 0) 0x7fe9bc15b2b0 con 0x7fe9bc0c90b0 2023-10-08T05:52:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.333+0000 7fe9bbfff640 1 --2- 172.21.15.70:0/2706677189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 0x7fe9bc0adc60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009d90 tx=0x7fe9b402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf006d586e3f7c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.333+0000 7fe9ba7fc640 1 -- 172.21.15.70:0/2706677189 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b402cc50 con 0x7fe9bc0cb580 2023-10-08T05:52:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.333+0000 7fe9ba7fc640 1 -- 172.21.15.70:0/2706677189 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b402cdb0 con 0x7fe9bc0cb580 2023-10-08T05:52:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.333+0000 7fe9ba7fc640 1 -- 172.21.15.70:0/2706677189 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4034660 con 0x7fe9bc0cb580 2023-10-08T05:52:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.333+0000 7fe9c2052640 1 -- 172.21.15.70:0/2706677189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 msgr2=0x7fe9bc0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.333+0000 7fe9c2052640 1 --2- 172.21.15.70:0/2706677189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 0x7fe9bc0adc60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009d90 tx=0x7fe9b402bea0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.334+0000 7fe9c2052640 1 -- 172.21.15.70:0/2706677189 shutdown_connections 2023-10-08T05:52:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.334+0000 7fe9c2052640 1 --2- 172.21.15.70:0/2706677189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 0x7fe9bc0adc60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.334+0000 7fe9c2052640 1 --2- 172.21.15.70:0/2706677189 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 0x7fe9bc0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.334+0000 7fe9c2052640 1 -- 172.21.15.70:0/2706677189 >> 172.21.15.70:0/2706677189 conn(0x7fe9bc06a380 msgr2=0x7fe9bc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.334+0000 7fe9c2052640 1 -- 172.21.15.70:0/2706677189 shutdown_connections 2023-10-08T05:52:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.334+0000 7fe9c2052640 1 -- 172.21.15.70:0/2706677189 wait complete. 2023-10-08T05:52:32.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.335+0000 7fe9c2052640 1 Processor -- start 2023-10-08T05:52:32.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.336+0000 7fe9c2052640 1 -- start start 2023-10-08T05:52:32.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.336+0000 7fe9c2052640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9bc0c90b0 0x7fe9bc0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.336+0000 7fe9c2052640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 0x7fe9bc1f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.336+0000 7fe9c2052640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 0x7fe9bc150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.336+0000 7fe9c2052640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc1f5cf0 con 0x7fe9bc0cb580 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.336+0000 7fe9c2052640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc1f5fd0 con 0x7fe9bc0c90b0 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.336+0000 7fe9c2052640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc1f2c80 con 0x7fe9bc15ab40 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 0x7fe9bc1f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 0x7fe9bc1f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49036/0 (socket says 172.21.15.70:49036) 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9baffd640 1 -- 172.21.15.70:0/2063906169 learned_addr learned my addr 172.21.15.70:0/2063906169 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9bb7fe640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9bc0c90b0 0x7fe9bc0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9bbfff640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 0x7fe9bc150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9baffd640 1 -- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 msgr2=0x7fe9bc150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9baffd640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 0x7fe9bc150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9baffd640 1 -- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9bc0c90b0 msgr2=0x7fe9bc0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9baffd640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9bc0c90b0 0x7fe9bc0c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9baffd640 1 -- 172.21.15.70:0/2063906169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b4009a70 con 0x7fe9bc0cb580 2023-10-08T05:52:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.337+0000 7fe9bbfff640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 0x7fe9bc150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.338+0000 7fe9baffd640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 0x7fe9bc1f2740 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000c460 tx=0x7fe9b000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.338+0000 7fe9a3fff640 1 -- 172.21.15.70:0/2063906169 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00074e0 con 0x7fe9bc0cb580 2023-10-08T05:52:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.338+0000 7fe9a3fff640 1 -- 172.21.15.70:0/2063906169 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0007660 con 0x7fe9bc0cb580 2023-10-08T05:52:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.338+0000 7fe9c2052640 1 -- 172.21.15.70:0/2063906169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc1f2f20 con 0x7fe9bc0cb580 2023-10-08T05:52:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.338+0000 7fe9a3fff640 1 -- 172.21.15.70:0/2063906169 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0004a20 con 0x7fe9bc0cb580 2023-10-08T05:52:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.338+0000 7fe9c2052640 1 -- 172.21.15.70:0/2063906169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc1f34e0 con 0x7fe9bc0cb580 2023-10-08T05:52:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.339+0000 7fe9c2052640 1 -- 172.21.15.70:0/2063906169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9bc1f6c00 con 0x7fe9bc0cb580 2023-10-08T05:52:32.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.340+0000 7fe9a3fff640 1 -- 172.21.15.70:0/2063906169 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe9b0004b80 con 0x7fe9bc0cb580 2023-10-08T05:52:32.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.341+0000 7fe9a3fff640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe9a806af90 0x7fe9a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:32.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.341+0000 7fe9a3fff640 1 -- 172.21.15.70:0/2063906169 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe9b0088da0 con 0x7fe9bc0cb580 2023-10-08T05:52:32.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.342+0000 7fe9bb7fe640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe9a806af90 0x7fe9a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:32.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.343+0000 7fe9bb7fe640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe9a806af90 0x7fe9a806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac006fd0 tx=0x7fe9ac008040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:32.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.346+0000 7fe9a3fff640 1 -- 172.21.15.70:0/2063906169 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9b0053bc0 con 0x7fe9bc0cb580 2023-10-08T05:52:32.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.575+0000 7fe9c2052640 1 -- 172.21.15.70:0/2063906169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9bc067490 con 0x7fe9bc0cb580 2023-10-08T05:52:32.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.576+0000 7fe9a3fff640 1 -- 172.21.15.70:0/2063906169 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fe9b0018020 con 0x7fe9bc0cb580 2023-10-08T05:52:32.577 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:32.578 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1038137327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1038137327},{"type":"v1","addr":"172.21.15.181:6835","nonce":1038137327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6660,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1967783419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1967783419},{"type":"v1","addr":"172.21.15.70:6833","nonce":1967783419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6663,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1304760341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1304760341},{"type":"v1","addr":"172.21.15.70:6835","nonce":1304760341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:52:26.053790+0000","modified":"2023-10-08T05:52:29.103940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6675},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6675":{"gid":6675,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2348955257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2348955257},{"type":"v1","addr":"172.21.15.181:6837","nonce":2348955257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-08T05:52:32.578 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 33 2023-10-08T05:52:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 -- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe9a806af90 msgr2=0x7fe9a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe9a806af90 0x7fe9a806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac006fd0 tx=0x7fe9ac008040 comp rx=0 tx=0).stop 2023-10-08T05:52:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 -- 172.21.15.70:0/2063906169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 msgr2=0x7fe9bc1f2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:32.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 0x7fe9bc1f2740 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000c460 tx=0x7fe9b000c920 comp rx=0 tx=0).stop 2023-10-08T05:52:32.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 -- 172.21.15.70:0/2063906169 shutdown_connections 2023-10-08T05:52:32.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fe9a806af90 0x7fe9a806d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9bc15ab40 0x7fe9bc150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9bc0cb580 0x7fe9bc1f2740 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 --2- 172.21.15.70:0/2063906169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9bc0c90b0 0x7fe9bc0c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:32.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 -- 172.21.15.70:0/2063906169 >> 172.21.15.70:0/2063906169 conn(0x7fe9bc06a380 msgr2=0x7fe9bc0adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:32.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 -- 172.21.15.70:0/2063906169 shutdown_connections 2023-10-08T05:52:32.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:32.579+0000 7fe9c2052640 1 -- 172.21.15.70:0/2063906169 wait complete. 2023-10-08T05:52:33.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T05:52:26.053790+0000', 'modified': '2023-10-08T05:52:29.103940+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6675}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6675': {'gid': 6675, 'name': 'b', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6837/2348955257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 2348955257}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 2348955257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T05:52:33.002 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T05:52:33.002 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T05:52:33.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.227+0000 7f3fe5ba9640 1 Processor -- start 2023-10-08T05:52:33.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.227+0000 7f3fe5ba9640 1 -- start start 2023-10-08T05:52:33.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.229+0000 7f3fe5ba9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00afbe0 0x7f3fe00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.229+0000 7f3fe5ba9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00b2620 con 0x7f3fe006dfc0 2023-10-08T05:52:33.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.229+0000 7f3fe5ba9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00b2060 con 0x7f3fe00cb130 2023-10-08T05:52:33.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.229+0000 7f3fe5ba9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00b2340 con 0x7f3fe00afbe0 2023-10-08T05:52:33.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.229+0000 7f3fd7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3fe006dfc0 0x7f3fe0070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55204/0 (socket says 172.21.15.70:55204) 2023-10-08T05:52:33.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.229+0000 7f3fd7fff640 1 -- 172.21.15.70:0/4266700659 learned_addr learned my addr 172.21.15.70:0/4266700659 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:33.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.229+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00afbe0 0x7f3fe00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14963900 0 0) 0x7f3fe00b2620 con 0x7f3fe006dfc0 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fcc003520 con 0x7f3fe006dfc0 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 570432100 0 0) 0x7f3fcc003520 con 0x7f3fe006dfc0 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00afbe0 msgr2=0x7f3fe00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00afbe0 0x7f3fe00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 >> v1:172.21.15.181:6789/0 conn(0x7f3fe00cb130 legacy=0x7f3fe006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe0070350 con 0x7f3fe006dfc0 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3fc8001670 con 0x7f3fe006dfc0 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 >> v1:172.21.15.70:6789/0 conn(0x7f3fe006dfc0 legacy=0x7f3fe0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 0x7f3fcc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00afbe0 0x7f3fcc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc004d00 0x7f3fcc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00b2340 con 0x7f3fcc0039a0 2023-10-08T05:52:33.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc00ba30 con 0x7f3fcc004d00 2023-10-08T05:52:33.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.230+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc00bbf0 con 0x7f3fe00afbe0 2023-10-08T05:52:33.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.231+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 0x7f3fcc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.231+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00afbe0 0x7f3fcc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.231+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/4266700659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00afbe0 msgr2=0x7f3fcc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.231+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00afbe0 0x7f3fcc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.231+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/4266700659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc004d00 msgr2=0x7f3fcc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.231+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fcc004d00 0x7f3fcc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.231+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/4266700659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc003520 con 0x7f3fcc0039a0 2023-10-08T05:52:33.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.232+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 0x7f3fcc003dc0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009a90 tx=0x7f3fcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc8579394bd30c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:33.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.232+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd002c680 con 0x7f3fcc0039a0 2023-10-08T05:52:33.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.232+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd002c7e0 con 0x7f3fcc0039a0 2023-10-08T05:52:33.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.232+0000 7f3fdeffd640 1 -- 172.21.15.70:0/4266700659 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd002cb70 con 0x7f3fcc0039a0 2023-10-08T05:52:33.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.233+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/4266700659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 msgr2=0x7f3fcc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.233+0000 7f3fe5ba9640 1 --2- 172.21.15.70:0/4266700659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 0x7f3fcc003dc0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009a90 tx=0x7f3fcc0057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.233+0000 7f3fdffff640 1 -- 172.21.15.70:0/4266700659 reap_dead start 2023-10-08T05:52:33.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.233+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/4266700659 shutdown_connections 2023-10-08T05:52:33.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.233+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/4266700659 >> 172.21.15.70:0/4266700659 conn(0x7f3fe006a5d0 msgr2=0x7f3fe00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:33.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.233+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/4266700659 shutdown_connections 2023-10-08T05:52:33.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.233+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/4266700659 wait complete. 2023-10-08T05:52:33.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.235+0000 7f3fe5ba9640 1 Processor -- start 2023-10-08T05:52:33.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.235+0000 7f3fe5ba9640 1 -- start start 2023-10-08T05:52:33.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.236+0000 7f3fe5ba9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 0x7f3fe00a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.236+0000 7f3fe5ba9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe00ae810 0x7f3fe00aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.236+0000 7f3fe5ba9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00c3d20 0x7f3fe00c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.236+0000 7f3fe5ba9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00af1d0 con 0x7f3fcc0039a0 2023-10-08T05:52:33.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.236+0000 7f3fe5ba9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00c6b20 con 0x7f3fe00ae810 2023-10-08T05:52:33.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.236+0000 7f3fe5ba9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00c4a50 con 0x7f3fe00c3d20 2023-10-08T05:52:33.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.236+0000 7f3fdf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 0x7f3fe00a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.236+0000 7f3fdf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 0x7f3fe00a7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49064/0 (socket says 172.21.15.70:49064) 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.236+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/2332578990 learned_addr learned my addr 172.21.15.70:0/2332578990 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fdffff640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00c3d20 0x7f3fe00c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe00ae810 0x7f3fe00aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00c3d20 msgr2=0x7f3fe00c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00c3d20 0x7f3fe00c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe00ae810 msgr2=0x7f3fe00aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe00ae810 0x7f3fe00aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/2332578990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00c64d0 con 0x7f3fcc0039a0 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe00ae810 0x7f3fe00aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 0x7f3fe00a7730 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0037040 tx=0x7f3fd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:33.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fdcff9640 1 -- 172.21.15.70:0/2332578990 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd002dea0 con 0x7f3fcc0039a0 2023-10-08T05:52:33.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fdcff9640 1 -- 172.21.15.70:0/2332578990 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd0003810 con 0x7f3fcc0039a0 2023-10-08T05:52:33.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/2332578990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe00c67b0 con 0x7f3fcc0039a0 2023-10-08T05:52:33.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.237+0000 7f3fdcff9640 1 -- 172.21.15.70:0/2332578990 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0003ba0 con 0x7f3fcc0039a0 2023-10-08T05:52:33.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.238+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/2332578990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe00a7c70 con 0x7f3fcc0039a0 2023-10-08T05:52:33.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.240+0000 7f3fdcff9640 1 -- 172.21.15.70:0/2332578990 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3fd0003d00 con 0x7f3fcc0039a0 2023-10-08T05:52:33.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.241+0000 7f3fdcff9640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3fc006afc0 0x7f3fc006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.241+0000 7f3fdcff9640 1 -- 172.21.15.70:0/2332578990 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3fd00ab830 con 0x7f3fcc0039a0 2023-10-08T05:52:33.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.241+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3fc006afc0 0x7f3fc006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.242+0000 7f3fe8814740 1 Processor -- start 2023-10-08T05:52:33.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.242+0000 7f3fe8814740 1 -- start start 2023-10-08T05:52:33.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.243+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3fc006afc0 0x7f3fc006d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8001200 tx=0x7f3fc8009040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:33.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.243+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33b60 0x55af29c06a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.243+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33f40 0x55af29c368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.243+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c06fa0 0x55af29c36e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.243+0000 7f3fe8814740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af29b7a700 con 0x55af29c33f40 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.243+0000 7f3fe8814740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af29b77df0 con 0x55af29c33b60 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.243+0000 7f3fe8814740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af29b736e0 con 0x55af29c06fa0 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.243+0000 7f3fd7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33f40 0x55af29c368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.243+0000 7f3fd7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33f40 0x55af29c368f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49074/0 (socket says 172.21.15.70:49074) 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.243+0000 7f3fd7fff640 1 -- 172.21.15.70:0/862307832 learned_addr learned my addr 172.21.15.70:0/862307832 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/862307832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33b60 0x55af29c06a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fd7fff640 1 -- 172.21.15.70:0/862307832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c06fa0 msgr2=0x55af29c36e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/862307832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c06fa0 0x55af29c36e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fd7fff640 1 -- 172.21.15.70:0/862307832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33b60 msgr2=0x55af29c06a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/862307832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33b60 0x55af29c06a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fd7fff640 1 -- 172.21.15.70:0/862307832 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af29b37910 con 0x55af29c33f40 2023-10-08T05:52:33.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/862307832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33b60 0x55af29c06a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:33.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/862307832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33f40 0x55af29c368f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8005a00 tx=0x7f3fc8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2af46f098233e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:33.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fd57fa640 1 -- 172.21.15.70:0/862307832 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc800c070 con 0x55af29c33f40 2023-10-08T05:52:33.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fd57fa640 1 -- 172.21.15.70:0/862307832 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc800d040 con 0x55af29c33f40 2023-10-08T05:52:33.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.244+0000 7f3fd57fa640 1 -- 172.21.15.70:0/862307832 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8007660 con 0x55af29c33f40 2023-10-08T05:52:33.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.245+0000 7f3fe8814740 1 -- 172.21.15.70:0/862307832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33f40 msgr2=0x55af29c368f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.245+0000 7f3fe8814740 1 --2- 172.21.15.70:0/862307832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33f40 0x55af29c368f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8005a00 tx=0x7f3fc8011000 comp rx=0 tx=0).stop 2023-10-08T05:52:33.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.245+0000 7f3fe8814740 1 -- 172.21.15.70:0/862307832 shutdown_connections 2023-10-08T05:52:33.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.245+0000 7f3fe8814740 1 --2- 172.21.15.70:0/862307832 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c06fa0 0x55af29c36e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.245+0000 7f3fe8814740 1 --2- 172.21.15.70:0/862307832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33f40 0x55af29c368f0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.245+0000 7f3fe8814740 1 --2- 172.21.15.70:0/862307832 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33b60 0x55af29c06a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.245+0000 7f3fe8814740 1 -- 172.21.15.70:0/862307832 >> 172.21.15.70:0/862307832 conn(0x55af29c08d70 msgr2=0x55af29c518f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.245+0000 7f3fe8814740 1 -- 172.21.15.70:0/862307832 shutdown_connections 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.245+0000 7f3fe8814740 1 -- 172.21.15.70:0/862307832 wait complete. 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.246+0000 7f3fe8814740 1 -- start start 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.246+0000 7f3fe8814740 1 Processor -- start 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.247+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c32d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.247+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33f40 0x55af29c39ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.247+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c3a010 0x55af29c23d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.247+0000 7f3fe8814740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af29b11970 con 0x55af29c33b60 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.247+0000 7f3fe8814740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af29b11610 con 0x55af29c3a010 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.247+0000 7f3fe8814740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af298d7f20 con 0x55af29c33f40 2023-10-08T05:52:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.247+0000 7f3fe8814740 10 client.0 _ll_register_callbacks cb 0x55af29b4f610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.247+0000 7f3fdf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c32d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.247+0000 7f3fdf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c32d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49084/0 (socket says 172.21.15.70:49084) 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.247+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/2067693075 learned_addr learned my addr 172.21.15.70:0/2067693075 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/2067693075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33f40 msgr2=0x55af29c39ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33f40 0x55af29c39ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/2067693075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c3a010 msgr2=0x55af29c23d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c3a010 0x55af29c23d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/2067693075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe0070350 con 0x55af29c33b60 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c32d20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f3fd002c2f0 tx=0x7f3fd007aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576aca50ea674eae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:52:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2067693075 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0082580 con 0x55af29c33b60 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2067693075 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3fd0076990 con 0x55af29c33b60 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fe8814740 1 -- 172.21.15.70:0/2067693075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af29b6c6c0 con 0x55af29c33b60 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2067693075 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd0076b50 con 0x55af29c33b60 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fe8814740 10 client.6810 fetch_fsmap learned FSMap version 33 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.248+0000 7f3fe8814740 1 -- 172.21.15.70:0/2067693075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55af29b71d60 con 0x55af29c33b60 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.249+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2067693075 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af29b6c6c0 con 0x55af29c33b60 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.249+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2067693075 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3fd007a4e0 con 0x55af29c33b60 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.249+0000 7f3fe8814740 10 client.6810 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.249+0000 7f3fe8814740 10 client.6810 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.249+0000 7f3fe8814740 20 client.6810 populate_metadata read hostname 'smithi070' 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.249+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.181:6836/2348955257,v1:172.21.15.181:6837/2348955257] conn(0x55af29c232d0 0x55af29c236b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.249+0000 7f3fe8814740 4 client.6810 mds_command: new command op to 6675 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T05:52:33.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.249+0000 7f3fe8814740 1 -- 172.21.15.70:0/2067693075 --> [v2:172.21.15.181:6836/2348955257,v1:172.21.15.181:6837/2348955257] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af29b6c3e0 con 0x55af29c232d0 2023-10-08T05:52:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.250+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.181:6836/2348955257,v1:172.21.15.181:6837/2348955257] conn(0x55af29c232d0 0x55af29c236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.251+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.181:6836/2348955257,v1:172.21.15.181:6837/2348955257] conn(0x55af29c232d0 0x55af29c236b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86b42e0ea3a91168 server_cookie=5d1a323151433ce8 in_seq=0 out_seq=0 2023-10-08T05:52:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.251+0000 7f3fbe7fc640 10 client.6810 ms_handle_connect on v2:172.21.15.181:6836/2348955257 2023-10-08T05:52:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.253+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2067693075 <== mds.0 v2:172.21.15.181:6836/2348955257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af29b6c3e0 con 0x55af29c232d0 2023-10-08T05:52:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.253+0000 7f3fbe7fc640 10 client.6810 handle_command_reply: tid=0 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.253+0000 7f3fe8814740 1 client.6810 shutdown 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.253+0000 7f3fe8814740 1 -- 172.21.15.70:0/2067693075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 msgr2=0x55af29c32d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.253+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c32d20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f3fd002c2f0 tx=0x7f3fd007aa40 comp rx=0 tx=0).stop 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.253+0000 7f3fe8814740 1 -- 172.21.15.70:0/2067693075 shutdown_connections 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.253+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.181:6836/2348955257,v1:172.21.15.181:6837/2348955257] conn(0x55af29c232d0 0x55af29c236b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.253+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c3a010 0x55af29c23d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.253+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33f40 0x55af29c39ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.254+0000 7f3fbe7fc640 0 client.6810 ms_handle_reset on v2:172.21.15.181:6836/2348955257 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.253+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2067693075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c32d20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.254+0000 7f3fe8814740 1 -- 172.21.15.70:0/2067693075 >> 172.21.15.70:0/2067693075 conn(0x55af29c06fa0 msgr2=0x55af29c368e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.254+0000 7f3fe8814740 1 -- 172.21.15.70:0/2067693075 shutdown_connections 2023-10-08T05:52:33.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.254+0000 7f3fe8814740 1 -- 172.21.15.70:0/2067693075 wait complete. 2023-10-08T05:52:33.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.254+0000 7f3fe8814740 20 client.6810 trim_cache size 0 max 16384 2023-10-08T05:52:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.271+0000 7f3fe8814740 1 Processor -- start 2023-10-08T05:52:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.271+0000 7f3fe8814740 1 -- start start 2023-10-08T05:52:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.272+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.272+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33f40 0x55af29c52b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.272+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c3a010 0x55af29c53080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.272+0000 7f3fe8814740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af29c4bf20 con 0x55af29c33b60 2023-10-08T05:52:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.272+0000 7f3fe8814740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af29c542b0 con 0x55af29c3a010 2023-10-08T05:52:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.272+0000 7f3fe8814740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af29c23750 con 0x55af29c33f40 2023-10-08T05:52:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.272+0000 7f3fdf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.272+0000 7f3fdf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49100/0 (socket says 172.21.15.70:49100) 2023-10-08T05:52:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.272+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/1293176161 learned_addr learned my addr 172.21.15.70:0/1293176161 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.273+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/1293176161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33f40 msgr2=0x55af29c52b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.273+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/1293176161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33f40 0x55af29c52b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.273+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/1293176161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c3a010 msgr2=0x55af29c53080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.273+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/1293176161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c3a010 0x55af29c53080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.273+0000 7f3fdf7fe640 1 -- 172.21.15.70:0/1293176161 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af29b10b80 con 0x55af29c33b60 2023-10-08T05:52:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.273+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/1293176161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c521a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0082050 tx=0x7f3fd0041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cec97f4ce5f471 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.273+0000 7f3fd5ffb640 1 -- 172.21.15.70:0/1293176161 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0082580 con 0x55af29c33b60 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.273+0000 7f3fd5ffb640 1 -- 172.21.15.70:0/1293176161 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd007a4e0 con 0x55af29c33b60 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.273+0000 7f3fd5ffb640 1 -- 172.21.15.70:0/1293176161 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd007a650 con 0x55af29c33b60 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.274+0000 7f3fe8814740 1 -- 172.21.15.70:0/1293176161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 msgr2=0x55af29c521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.274+0000 7f3fe8814740 1 --2- 172.21.15.70:0/1293176161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c521a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0082050 tx=0x7f3fd0041000 comp rx=0 tx=0).stop 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.274+0000 7f3fe8814740 1 -- 172.21.15.70:0/1293176161 shutdown_connections 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.274+0000 7f3fe8814740 1 --2- 172.21.15.70:0/1293176161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c3a010 0x55af29c53080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.274+0000 7f3fe8814740 1 --2- 172.21.15.70:0/1293176161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33f40 0x55af29c52b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.274+0000 7f3fe8814740 1 --2- 172.21.15.70:0/1293176161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c33b60 0x55af29c521a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.274+0000 7f3fe8814740 1 -- 172.21.15.70:0/1293176161 >> 172.21.15.70:0/1293176161 conn(0x55af29c06fa0 msgr2=0x55af29c54e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.274+0000 7f3fe8814740 1 -- 172.21.15.70:0/1293176161 shutdown_connections 2023-10-08T05:52:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.274+0000 7f3fe8814740 1 -- 172.21.15.70:0/1293176161 wait complete. 2023-10-08T05:52:33.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.275+0000 7f3fe8814740 1 -- start start 2023-10-08T05:52:33.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.275+0000 7f3fe8814740 1 Processor -- start 2023-10-08T05:52:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33b60 0x55af29cbdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33f40 0x55af29c4c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fe8814740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c3a010 0x55af29c4c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fe8814740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af29cbf900 con 0x55af29c3a010 2023-10-08T05:52:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fe8814740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af29c20f90 con 0x55af29c33f40 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fe8814740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af29c7a070 con 0x55af29c33b60 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fe8814740 10 client.0 _ll_register_callbacks cb 0x7f3fc80057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fdffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c3a010 0x55af29c4c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fdffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c3a010 0x55af29c4c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49116/0 (socket says 172.21.15.70:49116) 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fdffff640 1 -- 172.21.15.70:0/2071187806 learned_addr learned my addr 172.21.15.70:0/2071187806 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.276+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33b60 0x55af29cbdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fd7fff640 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33f40 0x55af29c4c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fdffff640 1 -- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33b60 msgr2=0x55af29cbdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fdffff640 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33b60 0x55af29cbdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fdffff640 1 -- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33f40 msgr2=0x55af29c4c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fdffff640 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33f40 0x55af29c4c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fdffff640 1 -- 172.21.15.70:0/2071187806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af29b71d60 con 0x55af29c3a010 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fdffff640 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c3a010 0x55af29c4c5b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00c8d40 tx=0x7f3fd800e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b70cb9aa6cdd88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2071187806 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8005b40 con 0x55af29c3a010 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2071187806 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3fd80015a0 con 0x55af29c3a010 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fe8814740 1 -- 172.21.15.70:0/2071187806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fc800d040 con 0x55af29c3a010 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2071187806 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fd8005ca0 con 0x55af29c3a010 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fe8814740 10 client.6816 fetch_fsmap learned FSMap version 33 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.277+0000 7f3fe8814740 1 -- 172.21.15.70:0/2071187806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55af29b36170 con 0x55af29c3a010 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.278+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2071187806 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc800d040 con 0x55af29c3a010 2023-10-08T05:52:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.278+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2071187806 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3fd80070d0 con 0x55af29c3a010 2023-10-08T05:52:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.278+0000 7f3fe8814740 10 client.6816 fetch_fsmap finished waiting for FSMap version 33 2023-10-08T05:52:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.278+0000 7f3fe8814740 10 client.6816 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T05:52:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.278+0000 7f3fe8814740 20 client.6816 populate_metadata read hostname 'smithi070' 2023-10-08T05:52:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.278+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:6836/2348955257,v1:172.21.15.181:6837/2348955257] conn(0x55af29c531d0 0x55af29c535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.278+0000 7f3fe8814740 4 client.6816 mds_command: new command op to 6675 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T05:52:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.278+0000 7f3fe8814740 1 -- 172.21.15.70:0/2071187806 --> [v2:172.21.15.181:6836/2348955257,v1:172.21.15.181:6837/2348955257] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55af298756c0 con 0x55af29c531d0 2023-10-08T05:52:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.279+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:6836/2348955257,v1:172.21.15.181:6837/2348955257] conn(0x55af29c531d0 0x55af29c535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.280+0000 7f3fdf7fe640 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:6836/2348955257,v1:172.21.15.181:6837/2348955257] conn(0x55af29c531d0 0x55af29c535b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=664305a95b46cefd server_cookie=7e18e677b123d047 in_seq=0 out_seq=0 2023-10-08T05:52:33.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.280+0000 7f3fbe7fc640 10 client.6816 ms_handle_connect on v2:172.21.15.181:6836/2348955257 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.280+0000 7f3fbe7fc640 1 -- 172.21.15.70:0/2071187806 <== mds.0 v2:172.21.15.181:6836/2348955257 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55af298756c0 con 0x55af29c531d0 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.280+0000 7f3fbe7fc640 10 client.6816 handle_command_reply: tid=0 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.280+0000 7f3fe8814740 1 client.6816 shutdown 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.280+0000 7f3fe8814740 1 -- 172.21.15.70:0/2071187806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c3a010 msgr2=0x55af29c4c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.280+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c3a010 0x55af29c4c5b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00c8d40 tx=0x7f3fd800e5f0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.280+0000 7f3fe8814740 1 -- 172.21.15.70:0/2071187806 shutdown_connections 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.280+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:6836/2348955257,v1:172.21.15.181:6837/2348955257] conn(0x55af29c531d0 0x55af29c535b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55af29c3a010 0x55af29c4c5b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55af29c33f40 0x55af29c4c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fbe7fc640 0 client.6816 ms_handle_reset on v2:172.21.15.181:6836/2348955257 2023-10-08T05:52:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe8814740 1 --2- 172.21.15.70:0/2071187806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55af29c33b60 0x55af29cbdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe8814740 1 -- 172.21.15.70:0/2071187806 >> 172.21.15.70:0/2071187806 conn(0x55af29c06fa0 msgr2=0x55af29c48520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:33.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe8814740 1 -- 172.21.15.70:0/2071187806 shutdown_connections 2023-10-08T05:52:33.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe8814740 1 -- 172.21.15.70:0/2071187806 wait complete. 2023-10-08T05:52:33.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe8814740 20 client.6816 trim_cache size 0 max 16384 2023-10-08T05:52:33.286 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:52:33.286 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T05:52:33.286 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T05:52:33.286 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 6675, 2023-10-08T05:52:33.287 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T05:52:33.287 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T05:52:33.287 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T05:52:33.287 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.6826712739999996, 2023-10-08T05:52:33.287 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 33, 2023-10-08T05:52:33.287 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 90, 2023-10-08T05:52:33.287 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 90, 2023-10-08T05:52:33.287 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 12.096252766999999 2023-10-08T05:52:33.287 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:52:33.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3fc006afc0 msgr2=0x7f3fc006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe5ba9640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3fc006afc0 0x7f3fc006d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8001200 tx=0x7f3fc8009040 comp rx=0 tx=0).stop 2023-10-08T05:52:33.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/2332578990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 msgr2=0x7f3fe00a7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.281+0000 7f3fe5ba9640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 0x7f3fe00a7730 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0037040 tx=0x7f3fd0004730 comp rx=0 tx=0).stop 2023-10-08T05:52:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.282+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/2332578990 shutdown_connections 2023-10-08T05:52:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.282+0000 7f3fe5ba9640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f3fc006afc0 0x7f3fc006d480 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.282+0000 7f3fe5ba9640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fe00c3d20 0x7f3fe00c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.282+0000 7f3fe5ba9640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fe00ae810 0x7f3fe00aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.282+0000 7f3fe5ba9640 1 --2- 172.21.15.70:0/2332578990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fcc0039a0 0x7f3fe00a7730 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.282+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/2332578990 >> 172.21.15.70:0/2332578990 conn(0x7f3fe006a5d0 msgr2=0x7f3fe00b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.282+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/2332578990 shutdown_connections 2023-10-08T05:52:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.282+0000 7f3fe5ba9640 1 -- 172.21.15.70:0/2332578990 wait complete. 2023-10-08T05:52:33.299 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:52:33.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.502+0000 7fc0049c5640 1 Processor -- start 2023-10-08T05:52:33.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.503+0000 7fc0049c5640 1 -- start start 2023-10-08T05:52:33.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.504+0000 7fc0049c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.504+0000 7fc0049c5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0000c90b0 0x7fc0000adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.504+0000 7fc0049c5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc0000cae70 0x7fc0000cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.504+0000 7fc0049c5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00015bf30 con 0x7fc0000c90b0 2023-10-08T05:52:33.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.504+0000 7fc0049c5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0000cb840 con 0x7fc0000cae70 2023-10-08T05:52:33.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.504+0000 7fc0049c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00015b2b0 con 0x7fc00015ab40 2023-10-08T05:52:33.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.504+0000 7fbff5d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0000c90b0 0x7fc0000adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.505+0000 7fbff5d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0000c90b0 0x7fc0000adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49130/0 (socket says 172.21.15.70:49130) 2023-10-08T05:52:33.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.505+0000 7fbff5d9b640 1 -- 172.21.15.70:0/3722037265 learned_addr learned my addr 172.21.15.70:0/3722037265 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:33.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.505+0000 7fbff5d9b640 1 -- 172.21.15.70:0/3722037265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 msgr2=0x7fc0000ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.505+0000 7fbffe59c640 1 --2- 172.21.15.70:0/3722037265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.505+0000 7fbffed9d640 1 --2- 172.21.15.70:0/3722037265 >> v2:172.21.15.181:3300/0 conn(0x7fc0000cae70 0x7fc0000cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.505+0000 7fbff5d9b640 1 --2- 172.21.15.70:0/3722037265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.505+0000 7fbff5d9b640 1 -- 172.21.15.70:0/3722037265 >> v2:172.21.15.181:3300/0 conn(0x7fc0000cae70 msgr2=0x7fc0000cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:33.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.505+0000 7fbff5d9b640 1 --2- 172.21.15.70:0/3722037265 >> v2:172.21.15.181:3300/0 conn(0x7fc0000cae70 0x7fc0000cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.505+0000 7fbff5d9b640 1 -- 172.21.15.70:0/3722037265 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00015b590 con 0x7fc0000c90b0 2023-10-08T05:52:33.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.505+0000 7fbffed9d640 1 --2- 172.21.15.70:0/3722037265 >> v2:172.21.15.181:3300/0 conn(0x7fc0000cae70 0x7fc0000cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:33.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.506+0000 7fbff5d9b640 1 --2- 172.21.15.70:0/3722037265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 0x7fc0000adc60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc0000c8d50 tx=0x7fbfe802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ff7d3ebaf1c65a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:33.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.506+0000 7fbffdd9b640 1 -- 172.21.15.70:0/3722037265 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8031860 con 0x7fc0000c90b0 2023-10-08T05:52:33.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.506+0000 7fbffdd9b640 1 -- 172.21.15.70:0/3722037265 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe80319c0 con 0x7fc0000c90b0 2023-10-08T05:52:33.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.506+0000 7fbffdd9b640 1 -- 172.21.15.70:0/3722037265 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8031d50 con 0x7fc0000c90b0 2023-10-08T05:52:33.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.507+0000 7fc0049c5640 1 -- 172.21.15.70:0/3722037265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 msgr2=0x7fc0000adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.507+0000 7fc0049c5640 1 --2- 172.21.15.70:0/3722037265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 0x7fc0000adc60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc0000c8d50 tx=0x7fbfe802b900 comp rx=0 tx=0).stop 2023-10-08T05:52:33.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.507+0000 7fc0049c5640 1 -- 172.21.15.70:0/3722037265 shutdown_connections 2023-10-08T05:52:33.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.507+0000 7fc0049c5640 1 --2- 172.21.15.70:0/3722037265 >> v2:172.21.15.181:3300/0 conn(0x7fc0000cae70 0x7fc0000cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.507+0000 7fc0049c5640 1 --2- 172.21.15.70:0/3722037265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 0x7fc0000adc60 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.507+0000 7fc0049c5640 1 --2- 172.21.15.70:0/3722037265 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.507+0000 7fc0049c5640 1 -- 172.21.15.70:0/3722037265 >> 172.21.15.70:0/3722037265 conn(0x7fc00006a380 msgr2=0x7fc0000b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.507+0000 7fc0049c5640 1 -- 172.21.15.70:0/3722037265 shutdown_connections 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.507+0000 7fc0049c5640 1 -- 172.21.15.70:0/3722037265 wait complete. 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.509+0000 7fc0049c5640 1 Processor -- start 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.509+0000 7fc0049c5640 1 -- start start 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.510+0000 7fc0049c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 0x7fc0000b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.510+0000 7fc0049c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000cae70 0x7fc000148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.510+0000 7fc0049c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.510+0000 7fc0049c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc000148780 con 0x7fc0000c90b0 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.510+0000 7fc0049c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001f5ea0 con 0x7fc0000cae70 2023-10-08T05:52:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.510+0000 7fc0049c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0001f6180 con 0x7fc00015ab40 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.510+0000 7fbffe59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 0x7fc0000b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.510+0000 7fbffe59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 0x7fc0000b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49134/0 (socket says 172.21.15.70:49134) 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.510+0000 7fbffe59c640 1 -- 172.21.15.70:0/3238760622 learned_addr learned my addr 172.21.15.70:0/3238760622 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbffe59c640 1 -- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 msgr2=0x7fc0000c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbffe59c640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbffe59c640 1 -- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000cae70 msgr2=0x7fc000148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbff5d9b640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000cae70 0x7fc000148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbffe59c640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000cae70 0x7fc000148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbffe59c640 1 -- 172.21.15.70:0/3238760622 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80094d0 con 0x7fc0000c90b0 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbff5d9b640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000cae70 0x7fc000148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbffe59c640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 0x7fc0000b2e50 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbff000ce10 tx=0x7fbff000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbff77fe640 1 -- 172.21.15.70:0/3238760622 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0003b90 con 0x7fc0000c90b0 2023-10-08T05:52:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbff77fe640 1 -- 172.21.15.70:0/3238760622 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0003cf0 con 0x7fc0000c90b0 2023-10-08T05:52:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fbff77fe640 1 -- 172.21.15.70:0/3238760622 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0003ea0 con 0x7fc0000c90b0 2023-10-08T05:52:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.511+0000 7fc0049c5640 1 -- 172.21.15.70:0/3238760622 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0001f6460 con 0x7fc0000c90b0 2023-10-08T05:52:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.512+0000 7fc0049c5640 1 -- 172.21.15.70:0/3238760622 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0001f6a20 con 0x7fc0000c90b0 2023-10-08T05:52:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.513+0000 7fc0049c5640 1 -- 172.21.15.70:0/3238760622 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc000067490 con 0x7fc0000c90b0 2023-10-08T05:52:33.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.514+0000 7fbff77fe640 1 -- 172.21.15.70:0/3238760622 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbff0004240 con 0x7fc0000c90b0 2023-10-08T05:52:33.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.515+0000 7fbff77fe640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbfe006af90 0x7fbfe006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:33.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.515+0000 7fbff77fe640 1 -- 172.21.15.70:0/3238760622 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbff0012070 con 0x7fc0000c90b0 2023-10-08T05:52:33.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.515+0000 7fbff5d9b640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbfe006af90 0x7fbfe006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:33.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.516+0000 7fbff5d9b640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbfe006af90 0x7fbfe006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8036000 tx=0x7fbfe803c040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:33.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.519+0000 7fbff77fe640 1 -- 172.21.15.70:0/3238760622 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbff0055190 con 0x7fc0000c90b0 2023-10-08T05:52:33.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.741+0000 7fc0049c5640 1 -- 172.21.15.70:0/3238760622 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc0000c8cb0 con 0x7fc0000c90b0 2023-10-08T05:52:33.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.741+0000 7fbff77fe640 1 -- 172.21.15.70:0/3238760622 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbff00587b0 con 0x7fc0000c90b0 2023-10-08T05:52:33.743 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T05:52:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 -- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbfe006af90 msgr2=0x7fbfe006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbfe006af90 0x7fbfe006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8036000 tx=0x7fbfe803c040 comp rx=0 tx=0).stop 2023-10-08T05:52:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 -- 172.21.15.70:0/3238760622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 msgr2=0x7fc0000b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 0x7fc0000b2e50 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fbff000ce10 tx=0x7fbff000a660 comp rx=0 tx=0).stop 2023-10-08T05:52:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 -- 172.21.15.70:0/3238760622 shutdown_connections 2023-10-08T05:52:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fbfe006af90 0x7fbfe006d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc00015ab40 0x7fc0000c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0000cae70 0x7fc000148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 --2- 172.21.15.70:0/3238760622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0000c90b0 0x7fc0000b2e50 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 -- 172.21.15.70:0/3238760622 >> 172.21.15.70:0/3238760622 conn(0x7fc00006a380 msgr2=0x7fc0000ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 -- 172.21.15.70:0/3238760622 shutdown_connections 2023-10-08T05:52:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:33.745+0000 7fc0049c5640 1 -- 172.21.15.70:0/3238760622 wait complete. 2023-10-08T05:52:33.763 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T05:52:33.764 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:52:33.764 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:52:33.764 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T05:52:33.764 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T05:52:33.764 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T05:52:33.764 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T05:52:33.764 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:52:33.764 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:52:33.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:33.765 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:52:33.815 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:52:33.815 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:52:33.816 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:52:33.816 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:52:33.816 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:52:33.816 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:52:33.816 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:52:33.816 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:52:33.816 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:52:33.816 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:52:33.816 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:52:33.816 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:52:33.817 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:52:33.817 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:52:33.817 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:52:33.817 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:52:33.817 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:52:33.817 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41443sec preferred_lft 41443sec 2023-10-08T05:52:33.817 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:52:33.817 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:52:33.818 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:52:33.818 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:52:33.818 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T05:52:33.818 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:52:33.818 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T05:52:33.819 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:52:33.819 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:52:33.819 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T05:52:33.819 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T05:52:33.819 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:52:33.820 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:52:33.820 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T05:52:33.820 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T05:52:33.820 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:52:33.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:33.820 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:52:33.912 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:52:33.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:33.916 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:52:33.973 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:52:33.973 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:52:33.973 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:52:33.973 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:52:33.973 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:52:33.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:33.974 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:52:33.974 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:52:33.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:52:33.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:52:33.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:52:33.974 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:52:34.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:34.160 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:52:34.212 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T05:52:34.213 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T05:52:34.213 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:52:34.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:34.213 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T05:52:34.306 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:52:34.307 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T05:52:34.366 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:52:34.366 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:52:34.366 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:52:34.418 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T05:52:34.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:34.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:52:34.559 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:52:34.559 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:52:34.595 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:52:34.595 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:52:34.595 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:52:34.595 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:52:34.595 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:52:34.598 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T05:52:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:52:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:52:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:52:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:52:34.604 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:52:34.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:34.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:52:34.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:34.693 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:52:34.773 INFO:teuthology.orchestra.run.smithi070.stdout:client5647 2023-10-08T05:52:34.773 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:34.773 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:34.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T05:52:35.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.040+0000 7fc5f8010640 1 Processor -- start 2023-10-08T05:52:35.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.040+0000 7fc5f8010640 1 -- start start 2023-10-08T05:52:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.042+0000 7fc5f8010640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f00afbe0 0x7fc5f00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.042+0000 7fc5f8010640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5f006dfc0 0x7fc5f0070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.042+0000 7fc5f8010640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f00b2620 con 0x7fc5f0071070 2023-10-08T05:52:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.042+0000 7fc5f8010640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f00b2060 con 0x7fc5f006dfc0 2023-10-08T05:52:35.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.042+0000 7fc5f8010640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00b2340 con 0x7fc5f00afbe0 2023-10-08T05:52:35.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.043+0000 7fc5f5dac640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f00afbe0 0x7fc5f00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:35.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.043+0000 7fc5f55ab640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5f006dfc0 0x7fc5f0070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:35.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.043+0000 7fc5f5dac640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f00afbe0 0x7fc5f00b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:36014/0 (socket says 172.21.15.70:36014) 2023-10-08T05:52:35.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.043+0000 7fc5f5dac640 1 -- 172.21.15.70:0/58211201 learned_addr learned my addr 172.21.15.70:0/58211201 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:35.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.044+0000 7fc5f5dac640 1 -- 172.21.15.70:0/58211201 >> v2:172.21.15.181:3300/0 conn(0x7fc5f006dfc0 msgr2=0x7fc5f0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:35.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.044+0000 7fc5f5dac640 1 --2- 172.21.15.70:0/58211201 >> v2:172.21.15.181:3300/0 conn(0x7fc5f006dfc0 0x7fc5f0070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.044+0000 7fc5f5dac640 1 -- 172.21.15.70:0/58211201 >> v1:172.21.15.181:6789/0 conn(0x7fc5f0071070 legacy=0x7fc5f00cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:35.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.044+0000 7fc5f5dac640 1 -- 172.21.15.70:0/58211201 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f0070350 con 0x7fc5f00afbe0 2023-10-08T05:52:35.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.045+0000 7fc5f5dac640 1 --2- 172.21.15.70:0/58211201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f00afbe0 0x7fc5f00b15b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009870 tx=0x7fc5ec02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215f9fae945b1979 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:35.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.045+0000 7fc5f4daa640 1 -- 172.21.15.70:0/58211201 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec030840 con 0x7fc5f00afbe0 2023-10-08T05:52:35.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.046+0000 7fc5f4daa640 1 -- 172.21.15.70:0/58211201 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec0309a0 con 0x7fc5f00afbe0 2023-10-08T05:52:35.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.046+0000 7fc5f4daa640 1 -- 172.21.15.70:0/58211201 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec030d30 con 0x7fc5f00afbe0 2023-10-08T05:52:35.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.046+0000 7fc5f8010640 1 -- 172.21.15.70:0/58211201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f00afbe0 msgr2=0x7fc5f00b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:35.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.046+0000 7fc5f8010640 1 --2- 172.21.15.70:0/58211201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f00afbe0 0x7fc5f00b15b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009870 tx=0x7fc5ec02b980 comp rx=0 tx=0).stop 2023-10-08T05:52:35.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.046+0000 7fc5f8010640 1 -- 172.21.15.70:0/58211201 shutdown_connections 2023-10-08T05:52:35.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.046+0000 7fc5f8010640 1 --2- 172.21.15.70:0/58211201 >> v2:172.21.15.181:3300/0 conn(0x7fc5f006dfc0 0x7fc5f0070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.046+0000 7fc5f8010640 1 --2- 172.21.15.70:0/58211201 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f00afbe0 0x7fc5f00b15b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.046+0000 7fc5f8010640 1 -- 172.21.15.70:0/58211201 >> 172.21.15.70:0/58211201 conn(0x7fc5f006a5d0 msgr2=0x7fc5f00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:35.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.046+0000 7fc5f8010640 1 -- 172.21.15.70:0/58211201 shutdown_connections 2023-10-08T05:52:35.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.046+0000 7fc5f8010640 1 -- 172.21.15.70:0/58211201 wait complete. 2023-10-08T05:52:35.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.048+0000 7fc5f8010640 1 Processor -- start 2023-10-08T05:52:35.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.048+0000 7fc5f8010640 1 -- start start 2023-10-08T05:52:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.049+0000 7fc5f8010640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f006dfc0 0x7fc5f01f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.049+0000 7fc5f8010640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f0071070 0x7fc5f014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.049+0000 7fc5f8010640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00afbe0 0x7fc5f00c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.049+0000 7fc5f8010640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00c9b40 con 0x7fc5f0071070 2023-10-08T05:52:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.049+0000 7fc5f8010640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f014f5e0 con 0x7fc5f00afbe0 2023-10-08T05:52:35.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.049+0000 7fc5f8010640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f014fa80 con 0x7fc5f006dfc0 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.050+0000 7fc5f55ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f0071070 0x7fc5f014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.050+0000 7fc5f55ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f0071070 0x7fc5f014f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41856/0 (socket says 172.21.15.70:41856) 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.050+0000 7fc5f55ab640 1 -- 172.21.15.70:0/1617770110 learned_addr learned my addr 172.21.15.70:0/1617770110 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.050+0000 7fc5f65ad640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00afbe0 0x7fc5f00c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.050+0000 7fc5f5dac640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f006dfc0 0x7fc5f01f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.050+0000 7fc5f55ab640 1 -- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f006dfc0 msgr2=0x7fc5f01f25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.050+0000 7fc5f55ab640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f006dfc0 0x7fc5f01f25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.050+0000 7fc5f55ab640 1 -- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00afbe0 msgr2=0x7fc5f00c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.050+0000 7fc5f55ab640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00afbe0 0x7fc5f00c2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.050+0000 7fc5f55ab640 1 -- 172.21.15.70:0/1617770110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec0094d0 con 0x7fc5f0071070 2023-10-08T05:52:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.051+0000 7fc5f55ab640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f0071070 0x7fc5f014f0a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000e8a0 tx=0x7fc5e000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.051+0000 7fc5e67fc640 1 -- 172.21.15.70:0/1617770110 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e00094f0 con 0x7fc5f0071070 2023-10-08T05:52:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.051+0000 7fc5e67fc640 1 -- 172.21.15.70:0/1617770110 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e000c940 con 0x7fc5f0071070 2023-10-08T05:52:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.051+0000 7fc5e67fc640 1 -- 172.21.15.70:0/1617770110 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e000ccf0 con 0x7fc5f0071070 2023-10-08T05:52:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.051+0000 7fc5f8010640 1 -- 172.21.15.70:0/1617770110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f00c8ec0 con 0x7fc5f0071070 2023-10-08T05:52:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.051+0000 7fc5f8010640 1 -- 172.21.15.70:0/1617770110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f00c9480 con 0x7fc5f0071070 2023-10-08T05:52:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.052+0000 7fc5f8010640 1 -- 172.21.15.70:0/1617770110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f004e160 con 0x7fc5f0071070 2023-10-08T05:52:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.053+0000 7fc5e67fc640 1 -- 172.21.15.70:0/1617770110 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc5e0013470 con 0x7fc5f0071070 2023-10-08T05:52:35.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.055+0000 7fc5e67fc640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc5d406af90 0x7fc5d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.055+0000 7fc5e67fc640 1 -- 172.21.15.70:0/1617770110 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc5e0089990 con 0x7fc5f0071070 2023-10-08T05:52:35.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.055+0000 7fc5f5dac640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc5d406af90 0x7fc5d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.056+0000 7fc5f5dac640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc5d406af90 0x7fc5d406d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec036000 tx=0x7fc5ec008030 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:35.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.060+0000 7fc5e67fc640 1 -- 172.21.15.70:0/1617770110 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5e00547b0 con 0x7fc5f0071070 2023-10-08T05:52:35.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.270+0000 7fc5f8010640 1 -- 172.21.15.70:0/1617770110 --> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5f00affc0 con 0x7fc5d406af90 2023-10-08T05:52:35.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.275+0000 7fc5e67fc640 1 -- 172.21.15.70:0/1617770110 <== mgr.5473 v2:172.21.15.181:6832/3994892546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc5ec030b90 con 0x7fc5d406af90 2023-10-08T05:52:35.276 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:52:35.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:52:35.276 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T05:52:35.276 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:52:35.276 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:52:35.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 -- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc5d406af90 msgr2=0x7fc5d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:35.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc5d406af90 0x7fc5d406d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec036000 tx=0x7fc5ec008030 comp rx=0 tx=0).stop 2023-10-08T05:52:35.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 -- 172.21.15.70:0/1617770110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f0071070 msgr2=0x7fc5f014f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:35.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f0071070 0x7fc5f014f0a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000e8a0 tx=0x7fc5e000ed60 comp rx=0 tx=0).stop 2023-10-08T05:52:35.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 -- 172.21.15.70:0/1617770110 shutdown_connections 2023-10-08T05:52:35.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7fc5d406af90 0x7fc5d406d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f00afbe0 0x7fc5f00c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f0071070 0x7fc5f014f0a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 --2- 172.21.15.70:0/1617770110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f006dfc0 0x7fc5f01f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 -- 172.21.15.70:0/1617770110 >> 172.21.15.70:0/1617770110 conn(0x7fc5f006a5d0 msgr2=0x7fc5f01597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:35.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 -- 172.21.15.70:0/1617770110 shutdown_connections 2023-10-08T05:52:35.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.277+0000 7fc5f8010640 1 -- 172.21.15.70:0/1617770110 wait complete. 2023-10-08T05:52:35.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T05:52:35.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.510+0000 7efd560a2640 1 Processor -- start 2023-10-08T05:52:35.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.510+0000 7efd560a2640 1 -- start start 2023-10-08T05:52:35.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.512+0000 7efd560a2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efd500c9080 0x7efd500c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.512+0000 7efd560a2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efd50070b50 0x7efd50070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.512+0000 7efd560a2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd500c7ad0 con 0x7efd5015a820 2023-10-08T05:52:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.512+0000 7efd560a2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd500c9a40 con 0x7efd500c9080 2023-10-08T05:52:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.512+0000 7efd560a2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5015ac00 con 0x7efd50070b50 2023-10-08T05:52:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.512+0000 7efd47fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efd500c9080 0x7efd500c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.512+0000 7efd47fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efd500c9080 0x7efd500c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41872/0 (socket says 172.21.15.70:41872) 2023-10-08T05:52:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.512+0000 7efd47fff640 1 -- 172.21.15.70:0/1493901244 learned_addr learned my addr 172.21.15.70:0/1493901244 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.512+0000 7efd4f7fe640 1 --1- 172.21.15.70:0/1493901244 >> v1:172.21.15.70:6789/0 conn(0x7efd5015a820 0x7efd50160750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38668/0 (socket says 172.21.15.70:38668) 2023-10-08T05:52:35.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.513+0000 7efd47fff640 1 -- 172.21.15.70:0/1493901244 >> v2:172.21.15.181:3300/0 conn(0x7efd50070b50 msgr2=0x7efd50070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:35.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.513+0000 7efd47fff640 1 --2- 172.21.15.70:0/1493901244 >> v2:172.21.15.181:3300/0 conn(0x7efd50070b50 0x7efd50070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.513+0000 7efd47fff640 1 -- 172.21.15.70:0/1493901244 >> v1:172.21.15.70:6789/0 conn(0x7efd5015a820 legacy=0x7efd50160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:35.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.513+0000 7efd47fff640 1 -- 172.21.15.70:0/1493901244 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd500714e0 con 0x7efd500c9080 2023-10-08T05:52:35.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.514+0000 7efd47fff640 1 --2- 172.21.15.70:0/1493901244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd500c9080 0x7efd500c9500 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efd50070120 tx=0x7efd3802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9231d871c42d4875 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:35.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.514+0000 7efd4effd640 1 -- 172.21.15.70:0/1493901244 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38030870 con 0x7efd500c9080 2023-10-08T05:52:35.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.514+0000 7efd4effd640 1 -- 172.21.15.70:0/1493901244 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd380309d0 con 0x7efd500c9080 2023-10-08T05:52:35.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.514+0000 7efd4effd640 1 -- 172.21.15.70:0/1493901244 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38030d60 con 0x7efd500c9080 2023-10-08T05:52:35.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.515+0000 7efd560a2640 1 -- 172.21.15.70:0/1493901244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd500c9080 msgr2=0x7efd500c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:35.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.515+0000 7efd560a2640 1 --2- 172.21.15.70:0/1493901244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd500c9080 0x7efd500c9500 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7efd50070120 tx=0x7efd3802b9b0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.515+0000 7efd560a2640 1 -- 172.21.15.70:0/1493901244 shutdown_connections 2023-10-08T05:52:35.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.515+0000 7efd560a2640 1 --2- 172.21.15.70:0/1493901244 >> v2:172.21.15.181:3300/0 conn(0x7efd50070b50 0x7efd50070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.515+0000 7efd560a2640 1 --2- 172.21.15.70:0/1493901244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd500c9080 0x7efd500c9500 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.515+0000 7efd560a2640 1 -- 172.21.15.70:0/1493901244 >> 172.21.15.70:0/1493901244 conn(0x7efd50069d50 msgr2=0x7efd5006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:35.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.515+0000 7efd560a2640 1 -- 172.21.15.70:0/1493901244 shutdown_connections 2023-10-08T05:52:35.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.515+0000 7efd560a2640 1 -- 172.21.15.70:0/1493901244 wait complete. 2023-10-08T05:52:35.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.517+0000 7efd560a2640 1 Processor -- start 2023-10-08T05:52:35.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.518+0000 7efd560a2640 1 -- start start 2023-10-08T05:52:35.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd560a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd50070b50 0x7efd500b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd560a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd500c9080 0x7efd500c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd560a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd5015a820 0x7efd501f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd560a2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd500b3250 con 0x7efd50070b50 2023-10-08T05:52:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd560a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd500b34f0 con 0x7efd5015a820 2023-10-08T05:52:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd560a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd500b37d0 con 0x7efd500c9080 2023-10-08T05:52:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd4f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd50070b50 0x7efd500b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd4f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd50070b50 0x7efd500b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41878/0 (socket says 172.21.15.70:41878) 2023-10-08T05:52:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd4f7fe640 1 -- 172.21.15.70:0/3244726876 learned_addr learned my addr 172.21.15.70:0/3244726876 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd4f7fe640 1 -- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd500c9080 msgr2=0x7efd500c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:35.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.519+0000 7efd4f7fe640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd500c9080 0x7efd500c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.520+0000 7efd4f7fe640 1 -- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd5015a820 msgr2=0x7efd501f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:35.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.520+0000 7efd4f7fe640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd5015a820 0x7efd501f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:35.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.520+0000 7efd4f7fe640 1 -- 172.21.15.70:0/3244726876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380094d0 con 0x7efd50070b50 2023-10-08T05:52:35.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.520+0000 7efd4f7fe640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd50070b50 0x7efd500b2d10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efd4000c3e0 tx=0x7efd4000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:35.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.520+0000 7efd4cff9640 1 -- 172.21.15.70:0/3244726876 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd400074e0 con 0x7efd50070b50 2023-10-08T05:52:35.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.520+0000 7efd4cff9640 1 -- 172.21.15.70:0/3244726876 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd40007640 con 0x7efd50070b50 2023-10-08T05:52:35.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.520+0000 7efd4cff9640 1 -- 172.21.15.70:0/3244726876 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd400049e0 con 0x7efd50070b50 2023-10-08T05:52:35.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.520+0000 7efd560a2640 1 -- 172.21.15.70:0/3244726876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd500b3ab0 con 0x7efd50070b50 2023-10-08T05:52:35.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.520+0000 7efd560a2640 1 -- 172.21.15.70:0/3244726876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd500b4070 con 0x7efd50070b50 2023-10-08T05:52:35.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.522+0000 7efd560a2640 1 -- 172.21.15.70:0/3244726876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5004e160 con 0x7efd50070b50 2023-10-08T05:52:35.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.522+0000 7efd4cff9640 1 -- 172.21.15.70:0/3244726876 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efd40004b40 con 0x7efd50070b50 2023-10-08T05:52:35.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.523+0000 7efd4cff9640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efd3006af90 0x7efd3006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:35.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.523+0000 7efd4cff9640 1 -- 172.21.15.70:0/3244726876 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efd40089390 con 0x7efd50070b50 2023-10-08T05:52:35.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.524+0000 7efd47fff640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efd3006af90 0x7efd3006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.525+0000 7efd47fff640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efd3006af90 0x7efd3006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd38036000 tx=0x7efd3803b040 comp rx=0 tx=0).ready entity=mgr.5473 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:35.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.528+0000 7efd4cff9640 1 -- 172.21.15.70:0/3244726876 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd40054120 con 0x7efd50070b50 2023-10-08T05:52:35.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:35.765+0000 7efd560a2640 1 -- 172.21.15.70:0/3244726876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7efd50071040 con 0x7efd50070b50 2023-10-08T05:52:36.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.324+0000 7efd4cff9640 1 -- 172.21.15.70:0/3244726876 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v31) v1 ==== 92+0+0 (secure 0 0 0) 0x7efd40057740 con 0x7efd50070b50 2023-10-08T05:52:36.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.327+0000 7efd560a2640 1 -- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efd3006af90 msgr2=0x7efd3006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.327+0000 7efd560a2640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efd3006af90 0x7efd3006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efd38036000 tx=0x7efd3803b040 comp rx=0 tx=0).stop 2023-10-08T05:52:36.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.327+0000 7efd560a2640 1 -- 172.21.15.70:0/3244726876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd50070b50 msgr2=0x7efd500b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.327+0000 7efd560a2640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd50070b50 0x7efd500b2d10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7efd4000c3e0 tx=0x7efd4000c8a0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.328+0000 7efd560a2640 1 -- 172.21.15.70:0/3244726876 shutdown_connections 2023-10-08T05:52:36.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.328+0000 7efd560a2640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7efd3006af90 0x7efd3006d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.328+0000 7efd560a2640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efd5015a820 0x7efd501f5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.328+0000 7efd560a2640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efd500c9080 0x7efd500c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.328+0000 7efd560a2640 1 --2- 172.21.15.70:0/3244726876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd50070b50 0x7efd500b2d10 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.328+0000 7efd560a2640 1 -- 172.21.15.70:0/3244726876 >> 172.21.15.70:0/3244726876 conn(0x7efd50069d50 msgr2=0x7efd5015b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:36.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.328+0000 7efd560a2640 1 -- 172.21.15.70:0/3244726876 shutdown_connections 2023-10-08T05:52:36.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.328+0000 7efd560a2640 1 -- 172.21.15.70:0/3244726876 wait complete. 2023-10-08T05:52:36.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T05:52:36.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.547+0000 7f1d7bdf5640 1 Processor -- start 2023-10-08T05:52:36.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.547+0000 7f1d7bdf5640 1 -- start start 2023-10-08T05:52:36.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d7bdf5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1d7415cce0 0x7f1d740cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d7bdf5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1d7415d0c0 0x7f1d740b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d7bdf5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7406ce20 con 0x7f1d740b30d0 2023-10-08T05:52:36.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d7bdf5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d740cf850 con 0x7f1d7415cce0 2023-10-08T05:52:36.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d7bdf5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7406c060 con 0x7f1d7415d0c0 2023-10-08T05:52:36.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d79b91640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1d7415cce0 0x7f1d740cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d79b91640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1d7415cce0 0x7f1d740cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41910/0 (socket says 172.21.15.70:41910) 2023-10-08T05:52:36.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d79b91640 1 -- 172.21.15.70:0/322884303 learned_addr learned my addr 172.21.15.70:0/322884303 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:36.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d79b91640 1 -- 172.21.15.70:0/322884303 >> v2:172.21.15.181:3300/0 conn(0x7f1d7415d0c0 msgr2=0x7f1d740b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:36.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d79b91640 1 --2- 172.21.15.70:0/322884303 >> v2:172.21.15.181:3300/0 conn(0x7f1d7415d0c0 0x7f1d740b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d79b91640 1 -- 172.21.15.70:0/322884303 >> v1:172.21.15.181:6789/0 conn(0x7f1d740b30d0 legacy=0x7f1d741609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:36.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.549+0000 7f1d79b91640 1 -- 172.21.15.70:0/322884303 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7406c340 con 0x7f1d7415cce0 2023-10-08T05:52:36.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.550+0000 7f1d79b91640 1 --2- 172.21.15.70:0/322884303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415cce0 0x7f1d740cf000 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1d740c4540 tx=0x7f1d7002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989528f16f13d25d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:36.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.551+0000 7f1d78b8f640 1 -- 172.21.15.70:0/322884303 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70031890 con 0x7f1d7415cce0 2023-10-08T05:52:36.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.551+0000 7f1d78b8f640 1 -- 172.21.15.70:0/322884303 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d700319f0 con 0x7f1d7415cce0 2023-10-08T05:52:36.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.551+0000 7f1d78b8f640 1 -- 172.21.15.70:0/322884303 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70031d80 con 0x7f1d7415cce0 2023-10-08T05:52:36.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.551+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/322884303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415cce0 msgr2=0x7f1d740cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.551+0000 7f1d7bdf5640 1 --2- 172.21.15.70:0/322884303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415cce0 0x7f1d740cf000 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1d740c4540 tx=0x7f1d7002b930 comp rx=0 tx=0).stop 2023-10-08T05:52:36.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.551+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/322884303 shutdown_connections 2023-10-08T05:52:36.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.552+0000 7f1d7bdf5640 1 --2- 172.21.15.70:0/322884303 >> v2:172.21.15.181:3300/0 conn(0x7f1d7415d0c0 0x7f1d740b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.552+0000 7f1d7bdf5640 1 --2- 172.21.15.70:0/322884303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415cce0 0x7f1d740cf000 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.552+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/322884303 >> 172.21.15.70:0/322884303 conn(0x7f1d7406e650 msgr2=0x7f1d7406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:36.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.552+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/322884303 shutdown_connections 2023-10-08T05:52:36.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.552+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/322884303 wait complete. 2023-10-08T05:52:36.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.553+0000 7f1d7bdf5640 1 Processor -- start 2023-10-08T05:52:36.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.553+0000 7f1d7bdf5640 1 -- start start 2023-10-08T05:52:36.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.554+0000 7f1d7bdf5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d740b30d0 0x7f1d7415f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.554+0000 7f1d7bdf5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d7415cce0 0x7f1d740c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.554+0000 7f1d7bdf5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415fea0 0x7f1d740c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.554+0000 7f1d7bdf5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740c92f0 con 0x7f1d7415fea0 2023-10-08T05:52:36.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.554+0000 7f1d7bdf5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740c9e30 con 0x7f1d740b30d0 2023-10-08T05:52:36.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.554+0000 7f1d7bdf5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740c9500 con 0x7f1d7415cce0 2023-10-08T05:52:36.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d7a392640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415fea0 0x7f1d740c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d7a392640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415fea0 0x7f1d740c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41926/0 (socket says 172.21.15.70:41926) 2023-10-08T05:52:36.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d7a392640 1 -- 172.21.15.70:0/95409288 learned_addr learned my addr 172.21.15.70:0/95409288 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:36.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d79390640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d7415cce0 0x7f1d740c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d79b91640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d740b30d0 0x7f1d7415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d7a392640 1 -- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d7415cce0 msgr2=0x7f1d740c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d7a392640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d7415cce0 0x7f1d740c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d7a392640 1 -- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d740b30d0 msgr2=0x7f1d7415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d7a392640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d740b30d0 0x7f1d7415f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d7a392640 1 -- 172.21.15.70:0/95409288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d700094d0 con 0x7f1d7415fea0 2023-10-08T05:52:36.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.555+0000 7f1d7a392640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415fea0 0x7f1d740c80c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c780 tx=0x7f1d6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:36.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.556+0000 7f1d6a7fc640 1 -- 172.21.15.70:0/95409288 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c003d80 con 0x7f1d7415fea0 2023-10-08T05:52:36.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.556+0000 7f1d6a7fc640 1 -- 172.21.15.70:0/95409288 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c00ce90 con 0x7f1d7415fea0 2023-10-08T05:52:36.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.556+0000 7f1d6a7fc640 1 -- 172.21.15.70:0/95409288 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0052b0 con 0x7f1d7415fea0 2023-10-08T05:52:36.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.556+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/95409288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d740c97e0 con 0x7f1d7415fea0 2023-10-08T05:52:36.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.556+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/95409288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d740ca2c0 con 0x7f1d7415fea0 2023-10-08T05:52:36.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.557+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/95409288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d740c9fc0 con 0x7f1d7415fea0 2023-10-08T05:52:36.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.558+0000 7f1d6a7fc640 1 -- 172.21.15.70:0/95409288 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f1d6c005410 con 0x7f1d7415fea0 2023-10-08T05:52:36.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.559+0000 7f1d6a7fc640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f1d5806afe0 0x7f1d5806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.559+0000 7f1d6a7fc640 1 -- 172.21.15.70:0/95409288 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1d6c088e20 con 0x7f1d7415fea0 2023-10-08T05:52:36.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.559+0000 7f1d79b91640 1 -- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f1d5806afe0 msgr2=0x7f1d5806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3994892546 2023-10-08T05:52:36.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.559+0000 7f1d79b91640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f1d5806afe0 0x7f1d5806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:36.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.563+0000 7f1d6a7fc640 1 -- 172.21.15.70:0/95409288 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d6c053c40 con 0x7f1d7415fea0 2023-10-08T05:52:36.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.750+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/95409288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f1d7404e160 con 0x7f1d7415fea0 2023-10-08T05:52:36.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.755+0000 7f1d6a7fc640 1 -- 172.21.15.70:0/95409288 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1d6c017020 con 0x7f1d7415fea0 2023-10-08T05:52:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.757+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f1d5806afe0 msgr2=0x7f1d5806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.757+0000 7f1d7bdf5640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f1d5806afe0 0x7f1d5806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.757+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/95409288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415fea0 msgr2=0x7f1d740c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.757+0000 7f1d7bdf5640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415fea0 0x7f1d740c80c0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c780 tx=0x7f1d6c00cc40 comp rx=0 tx=0).stop 2023-10-08T05:52:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.758+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/95409288 shutdown_connections 2023-10-08T05:52:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.758+0000 7f1d7bdf5640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f1d5806afe0 0x7f1d5806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.758+0000 7f1d7bdf5640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d7415fea0 0x7f1d740c80c0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.758+0000 7f1d7bdf5640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d7415cce0 0x7f1d740c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.758+0000 7f1d7bdf5640 1 --2- 172.21.15.70:0/95409288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d740b30d0 0x7f1d7415f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.758+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/95409288 >> 172.21.15.70:0/95409288 conn(0x7f1d7406e650 msgr2=0x7f1d7415af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.758+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/95409288 shutdown_connections 2023-10-08T05:52:36.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.758+0000 7f1d7bdf5640 1 -- 172.21.15.70:0/95409288 wait complete. 2023-10-08T05:52:36.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T05:52:36.841 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:36.841+0000 7f090a410040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:52:36.855 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:36.853+0000 7fc7dc630040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:52:36.893 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:36.893+0000 7f090a410040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:52:36.909 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:36.908+0000 7fc7dc630040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:52:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.975+0000 7f97c7e10640 1 Processor -- start 2023-10-08T05:52:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.975+0000 7f97c7e10640 1 -- start start 2023-10-08T05:52:36.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.977+0000 7f97c7e10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f97c00b4e10 0x7f97c01612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.977+0000 7f97c7e10640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c00b8c30 con 0x7f97c00cd4e0 2023-10-08T05:52:36.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.977+0000 7f97c7e10640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c00cdd80 con 0x7f97c015cd70 2023-10-08T05:52:36.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.977+0000 7f97c7e10640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c00ce060 con 0x7f97c00b4e10 2023-10-08T05:52:36.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c63ad640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f97c00cd4e0 0x7f97c00d1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38686/0 (socket says 172.21.15.70:38686) 2023-10-08T05:52:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c63ad640 1 -- 172.21.15.70:0/4195810530 learned_addr learned my addr 172.21.15.70:0/4195810530 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c53ab640 1 --2- 172.21.15.70:0/4195810530 >> v2:172.21.15.181:3300/0 conn(0x7f97c00b4e10 0x7f97c01612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740756177 0 0) 0x7f97c00b8c30 con 0x7f97c00cd4e0 2023-10-08T05:52:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97ac003520 con 0x7f97c00cd4e0 2023-10-08T05:52:36.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2492382840 0 0) 0x7f97ac003520 con 0x7f97c00cd4e0 2023-10-08T05:52:36.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 >> v2:172.21.15.181:3300/0 conn(0x7f97c00b4e10 msgr2=0x7f97c01612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c4baa640 1 --2- 172.21.15.70:0/4195810530 >> v2:172.21.15.181:3300/0 conn(0x7f97c00b4e10 0x7f97c01612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 >> v1:172.21.15.181:6789/0 conn(0x7f97c015cd70 legacy=0x7f97c00b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c00ce340 con 0x7f97c00cd4e0 2023-10-08T05:52:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.978+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f97b80019f0 con 0x7f97c00cd4e0 2023-10-08T05:52:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 >> v1:172.21.15.70:6789/0 conn(0x7f97c00cd4e0 legacy=0x7f97c00d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c4baa640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ac003970 0x7f97ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c4baa640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97ac0047c0 0x7f97ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c4baa640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97ac005180 0x7f97ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00ce060 con 0x7f97ac003970 2023-10-08T05:52:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac00c020 con 0x7f97ac0047c0 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac00c1e0 con 0x7f97ac005180 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c5bac640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ac003970 0x7f97ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c63ad640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97ac005180 0x7f97ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c53ab640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97ac0047c0 0x7f97ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c5bac640 1 -- 172.21.15.70:0/4195810530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97ac005180 msgr2=0x7f97ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c5bac640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97ac005180 0x7f97ac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c5bac640 1 -- 172.21.15.70:0/4195810530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97ac0047c0 msgr2=0x7f97ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c5bac640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97ac0047c0 0x7f97ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c5bac640 1 -- 172.21.15.70:0/4195810530 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac003520 con 0x7f97ac003970 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c53ab640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97ac0047c0 0x7f97ac004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.979+0000 7f97c63ad640 1 -- 172.21.15.70:0/4195810530 reap_dead start 2023-10-08T05:52:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.981+0000 7f97c5bac640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ac003970 0x7f97ac003d90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f97bc001750 tx=0x7f97ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f14e5abad93d671 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.981+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0052d0 con 0x7f97ac003970 2023-10-08T05:52:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.981+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc02e050 con 0x7f97ac003970 2023-10-08T05:52:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.981+0000 7f97c4baa640 1 -- 172.21.15.70:0/4195810530 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc003830 con 0x7f97ac003970 2023-10-08T05:52:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.981+0000 7f97c7e10640 1 -- 172.21.15.70:0/4195810530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ac003970 msgr2=0x7f97ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.981+0000 7f97c7e10640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ac003970 0x7f97ac003d90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f97bc001750 tx=0x7f97ac0057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.981+0000 7f97c7e10640 1 -- 172.21.15.70:0/4195810530 shutdown_connections 2023-10-08T05:52:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.981+0000 7f97c7e10640 1 --2- 172.21.15.70:0/4195810530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97ac003970 0x7f97ac003d90 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.981+0000 7f97c7e10640 1 -- 172.21.15.70:0/4195810530 >> 172.21.15.70:0/4195810530 conn(0x7f97c00cf700 msgr2=0x7f97c00cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.982+0000 7f97c7e10640 1 -- 172.21.15.70:0/4195810530 shutdown_connections 2023-10-08T05:52:36.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.982+0000 7f97c7e10640 1 -- 172.21.15.70:0/4195810530 wait complete. 2023-10-08T05:52:36.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.983+0000 7f97c7e10640 1 Processor -- start 2023-10-08T05:52:36.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.983+0000 7f97c7e10640 1 -- start start 2023-10-08T05:52:36.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.984+0000 7f97c7e10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97ac003970 0x7f97c015e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.984+0000 7f97c7e10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c00b1150 0x7f97c01f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.984+0000 7f97c7e10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97c00cc280 0x7f97c00cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.984+0000 7f97c7e10640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00ccc70 con 0x7f97c00b1150 2023-10-08T05:52:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.984+0000 7f97c7e10640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c015ed40 con 0x7f97c00cc280 2023-10-08T05:52:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.984+0000 7f97c7e10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c01f6740 con 0x7f97ac003970 2023-10-08T05:52:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c53ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c00b1150 0x7f97c01f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c53ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c00b1150 0x7f97c01f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41936/0 (socket says 172.21.15.70:41936) 2023-10-08T05:52:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c53ab640 1 -- 172.21.15.70:0/2363790826 learned_addr learned my addr 172.21.15.70:0/2363790826 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c63ad640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97c00cc280 0x7f97c00cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c5bac640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97ac003970 0x7f97c015e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c53ab640 1 -- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97ac003970 msgr2=0x7f97c015e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c53ab640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97ac003970 0x7f97c015e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c53ab640 1 -- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97c00cc280 msgr2=0x7f97c00cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c53ab640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97c00cc280 0x7f97c00cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c53ab640 1 -- 172.21.15.70:0/2363790826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c015d760 con 0x7f97c00b1150 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.985+0000 7f97c53ab640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c00b1150 0x7f97c01f6200 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f97b00109d0 tx=0x7f97b0010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.986+0000 7f97b67fc640 1 -- 172.21.15.70:0/2363790826 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b000ec20 con 0x7f97c00b1150 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.986+0000 7f97b67fc640 1 -- 172.21.15.70:0/2363790826 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b000ed80 con 0x7f97c00b1150 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.986+0000 7f97b67fc640 1 -- 172.21.15.70:0/2363790826 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b001c620 con 0x7f97c00b1150 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.986+0000 7f97c7e10640 1 -- 172.21.15.70:0/2363790826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c015da40 con 0x7f97c00b1150 2023-10-08T05:52:36.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.986+0000 7f97c7e10640 1 -- 172.21.15.70:0/2363790826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c015e000 con 0x7f97c00b1150 2023-10-08T05:52:36.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.987+0000 7f97c7e10640 1 -- 172.21.15.70:0/2363790826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c004e160 con 0x7f97c00b1150 2023-10-08T05:52:36.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.988+0000 7f97b67fc640 1 -- 172.21.15.70:0/2363790826 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f97b001c780 con 0x7f97c00b1150 2023-10-08T05:52:36.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.989+0000 7f97b67fc640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f97a406afe0 0x7f97a406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:36.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.989+0000 7f97b67fc640 1 -- 172.21.15.70:0/2363790826 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f97b0089f20 con 0x7f97c00b1150 2023-10-08T05:52:36.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.989+0000 7f97c5bac640 1 -- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f97a406afe0 msgr2=0x7f97a406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3994892546 2023-10-08T05:52:36.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.989+0000 7f97c5bac640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f97a406afe0 0x7f97a406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:36.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:36.993+0000 7f97b67fc640 1 -- 172.21.15.70:0/2363790826 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97b0054e90 con 0x7f97c00b1150 2023-10-08T05:52:37.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.182+0000 7f97c7e10640 1 -- 172.21.15.70:0/2363790826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f97c00ca000 con 0x7f97c00b1150 2023-10-08T05:52:37.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.188+0000 7f97b67fc640 1 -- 172.21.15.70:0/2363790826 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f97b00584b0 con 0x7f97c00b1150 2023-10-08T05:52:37.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.190+0000 7f97c5bac640 1 -- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f97a406afe0 msgr2=0x7f97a406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3994892546 2023-10-08T05:52:37.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.190+0000 7f97c5bac640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f97a406afe0 0x7f97a406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:37.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 -- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f97a406afe0 msgr2=0x7f97a406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:37.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f97a406afe0 0x7f97a406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 -- 172.21.15.70:0/2363790826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c00b1150 msgr2=0x7f97c01f6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:37.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c00b1150 0x7f97c01f6200 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f97b00109d0 tx=0x7f97b0010e90 comp rx=0 tx=0).stop 2023-10-08T05:52:37.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 -- 172.21.15.70:0/2363790826 shutdown_connections 2023-10-08T05:52:37.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7f97a406afe0 0x7f97a406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97c00cc280 0x7f97c00cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97c00b1150 0x7f97c01f6200 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 --2- 172.21.15.70:0/2363790826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97ac003970 0x7f97c015e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 -- 172.21.15.70:0/2363790826 >> 172.21.15.70:0/2363790826 conn(0x7f97c00cf700 msgr2=0x7f97c00b8470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:37.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 -- 172.21.15.70:0/2363790826 shutdown_connections 2023-10-08T05:52:37.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.191+0000 7f97c7e10640 1 -- 172.21.15.70:0/2363790826 wait complete. 2023-10-08T05:52:37.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:37.210 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T05:52:37.265 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-08T05:52:37.343 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:37.343+0000 7f090a410040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:52:37.376 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:37.375+0000 7fc7dc630040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:52:37.395 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:37.395+0000 7f090a410040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:52:37.430 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:37.429+0000 7fc7dc630040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:52:37.445 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:37.445+0000 7f090a410040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:52:37.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.467+0000 7fad02c83640 1 Processor -- start 2023-10-08T05:52:37.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.467+0000 7fad02c83640 1 -- start start 2023-10-08T05:52:37.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.469+0000 7fad02c83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7facfc15cce0 0x7facfc0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:37.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.469+0000 7fad02c83640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7facfc0b2c40 0x7facfc0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:37.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.469+0000 7fad02c83640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facfc06ce20 con 0x7facfc15d0c0 2023-10-08T05:52:37.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.469+0000 7fad02c83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facfc0cf850 con 0x7facfc15cce0 2023-10-08T05:52:37.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.469+0000 7fad02c83640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facfc06c060 con 0x7facfc0b2c40 2023-10-08T05:52:37.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.469+0000 7facfbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7facfc15d0c0 0x7facfc0b2530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38688/0 (socket says 172.21.15.70:38688) 2023-10-08T05:52:37.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.469+0000 7facfbfff640 1 -- 172.21.15.70:0/2648294669 learned_addr learned my addr 172.21.15.70:0/2648294669 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:37.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.469+0000 7fad00a1f640 1 --2- 172.21.15.70:0/2648294669 >> v2:172.21.15.70:3300/0 conn(0x7facfc15cce0 0x7facfc0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:37.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.470+0000 7fad01220640 1 --2- 172.21.15.70:0/2648294669 >> v2:172.21.15.181:3300/0 conn(0x7facfc0b2c40 0x7facfc0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:37.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.470+0000 7fad00a1f640 1 -- 172.21.15.70:0/2648294669 >> v2:172.21.15.181:3300/0 conn(0x7facfc0b2c40 msgr2=0x7facfc0cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:37.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.470+0000 7fad00a1f640 1 --2- 172.21.15.70:0/2648294669 >> v2:172.21.15.181:3300/0 conn(0x7facfc0b2c40 0x7facfc0cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.470+0000 7fad00a1f640 1 -- 172.21.15.70:0/2648294669 >> v1:172.21.15.70:6789/0 conn(0x7facfc15d0c0 legacy=0x7facfc0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:37.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.470+0000 7fad00a1f640 1 -- 172.21.15.70:0/2648294669 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc06c340 con 0x7facfc15cce0 2023-10-08T05:52:37.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.471+0000 7fad00a1f640 1 --2- 172.21.15.70:0/2648294669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facfc15cce0 0x7facfc0cf000 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7facfc0b04b0 tx=0x7facf002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d448bbae9526ae9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:37.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.471+0000 7facfb7fe640 1 -- 172.21.15.70:0/2648294669 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0031830 con 0x7facfc15cce0 2023-10-08T05:52:37.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.471+0000 7facfb7fe640 1 -- 172.21.15.70:0/2648294669 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf0031990 con 0x7facfc15cce0 2023-10-08T05:52:37.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.471+0000 7facfb7fe640 1 -- 172.21.15.70:0/2648294669 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0031d20 con 0x7facfc15cce0 2023-10-08T05:52:37.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.472+0000 7fad02c83640 1 -- 172.21.15.70:0/2648294669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facfc15cce0 msgr2=0x7facfc0cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:37.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.472+0000 7fad02c83640 1 --2- 172.21.15.70:0/2648294669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facfc15cce0 0x7facfc0cf000 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7facfc0b04b0 tx=0x7facf002b8d0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.472+0000 7fad02c83640 1 -- 172.21.15.70:0/2648294669 shutdown_connections 2023-10-08T05:52:37.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.472+0000 7fad02c83640 1 --2- 172.21.15.70:0/2648294669 >> v2:172.21.15.181:3300/0 conn(0x7facfc0b2c40 0x7facfc0cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.472+0000 7fad02c83640 1 --2- 172.21.15.70:0/2648294669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facfc15cce0 0x7facfc0cf000 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.472+0000 7fad02c83640 1 -- 172.21.15.70:0/2648294669 >> 172.21.15.70:0/2648294669 conn(0x7facfc06e650 msgr2=0x7facfc06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:37.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.472+0000 7fad02c83640 1 -- 172.21.15.70:0/2648294669 shutdown_connections 2023-10-08T05:52:37.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.472+0000 7fad02c83640 1 -- 172.21.15.70:0/2648294669 wait complete. 2023-10-08T05:52:37.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.474+0000 7fad02c83640 1 Processor -- start 2023-10-08T05:52:37.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.474+0000 7fad02c83640 1 -- start start 2023-10-08T05:52:37.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7fad02c83640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7facfc0b2c40 0x7facfc0c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:37.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7fad02c83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facfc15cce0 0x7facfc150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:37.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7fad02c83640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc0c5790 0x7facfc0b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:37.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7fad02c83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc151360 con 0x7facfc15cce0 2023-10-08T05:52:37.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7fad02c83640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0c9f70 con 0x7facfc0c5790 2023-10-08T05:52:37.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7fad02c83640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facfc0c5d00 con 0x7facfc0b2c40 2023-10-08T05:52:37.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7facfbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facfc15cce0 0x7facfc150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:37.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7facfbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facfc15cce0 0x7facfc150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41950/0 (socket says 172.21.15.70:41950) 2023-10-08T05:52:37.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7facfbfff640 1 -- 172.21.15.70:0/2942735308 learned_addr learned my addr 172.21.15.70:0/2942735308 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:37.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7fad00a1f640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7facfc0b2c40 0x7facfc0c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:37.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.475+0000 7fad01220640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc0c5790 0x7facfc0b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:37.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7facfbfff640 1 -- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7facfc0b2c40 msgr2=0x7facfc0c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:37.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7facfbfff640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7facfc0b2c40 0x7facfc0c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7facfbfff640 1 -- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc0c5790 msgr2=0x7facfc0b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:37.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7facfbfff640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc0c5790 0x7facfc0b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:37.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7facfbfff640 1 -- 172.21.15.70:0/2942735308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf0009400 con 0x7facfc15cce0 2023-10-08T05:52:37.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7fad01220640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7facfc0c5790 0x7facfc0b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:37.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7facfbfff640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facfc15cce0 0x7facfc150e20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7face800c4d0 tx=0x7face800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:37.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8005ce0 con 0x7facfc15cce0 2023-10-08T05:52:37.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8005e40 con 0x7facfc15cce0 2023-10-08T05:52:37.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7fad02c83640 1 -- 172.21.15.70:0/2942735308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facfc0c5fe0 con 0x7facfc15cce0 2023-10-08T05:52:37.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8007270 con 0x7facfc15cce0 2023-10-08T05:52:37.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.476+0000 7fad02c83640 1 -- 172.21.15.70:0/2942735308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facfc0c65a0 con 0x7facfc15cce0 2023-10-08T05:52:37.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.477+0000 7fad02c83640 1 -- 172.21.15.70:0/2942735308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facfc04e160 con 0x7facfc15cce0 2023-10-08T05:52:37.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.478+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) v1 ==== 89348+0+0 (secure 0 0 0) 0x7face80073d0 con 0x7facfc15cce0 2023-10-08T05:52:37.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.479+0000 7facf97fa640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 0x7face006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:37.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.479+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6073+0+0 (secure 0 0 0) 0x7face8088d70 con 0x7facfc15cce0 2023-10-08T05:52:37.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.479+0000 7fad00a1f640 1 -- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 msgr2=0x7face006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3994892546 2023-10-08T05:52:37.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.479+0000 7fad00a1f640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 0x7face006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:37.487 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:37.483+0000 7fc7dc630040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:52:37.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.484+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7face8053c40 con 0x7facfc15cce0 2023-10-08T05:52:37.507 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:37.506+0000 7f090a410040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:52:37.547 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:37.545+0000 7fc7dc630040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:52:37.555 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:37.556+0000 7f090a410040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:52:37.597 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:37.595+0000 7fc7dc630040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:52:37.666 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:37.666+0000 7f090a410040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:52:37.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.680+0000 7fad00a1f640 1 -- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 msgr2=0x7face006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3994892546 2023-10-08T05:52:37.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.680+0000 7fad00a1f640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 0x7face006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:37.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:37.693+0000 7fad02c83640 1 -- 172.21.15.70:0/2942735308 --> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7facfc0b3f90 con 0x7face006afe0 2023-10-08T05:52:37.709 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:37.708+0000 7fc7dc630040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:52:37.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:37.728+0000 7f090a410040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:37.771 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:37.770+0000 7fc7dc630040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:37.777 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:37.777+0000 7f090a410040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:52:37.821 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:37.819+0000 7fc7dc630040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:52:37.834 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:37.834+0000 7f090a410040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:52:37.877 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:37.876+0000 7fc7dc630040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:52:37.884 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:37.884+0000 7f090a410040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:52:37.926 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:37.925+0000 7fc7dc630040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:52:38.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:38.081+0000 7fad00a1f640 1 -- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 msgr2=0x7face006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3994892546 2023-10-08T05:52:38.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:38.081+0000 7fad00a1f640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 0x7face006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:52:38.261 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:38.261+0000 7f090a410040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:52:38.303 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:38.301+0000 7fc7dc630040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:52:38.419 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:38.419+0000 7f090a410040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:52:38.460 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:38.458+0000 7fc7dc630040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:52:38.525 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:38.525+0000 7f090a410040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:52:38.565 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:38.564+0000 7fc7dc630040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:52:38.574 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:38.574+0000 7f090a410040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:52:38.615 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:38.613+0000 7fc7dc630040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:52:38.681 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:38.680+0000 7f090a410040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:38.721 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:38.719+0000 7fc7dc630040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:38.786 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:38.785+0000 7f090a410040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:52:38.827 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:38.825+0000 7fc7dc630040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:52:38.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:38.882+0000 7fad00a1f640 1 -- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 msgr2=0x7face006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3994892546 2023-10-08T05:52:38.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:38.882+0000 7fad00a1f640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 0x7face006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:52:38.941 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:38.939+0000 7f090a410040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:38.983 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:38.981+0000 7fc7dc630040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:38.989 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:38.989+0000 7f090a410040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:39.031 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:39.030+0000 7fc7dc630040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:39.107 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:39.106+0000 7f090a410040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:52:39.149 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:39.147+0000 7fc7dc630040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:52:39.692 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:52:39.697 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:52:39.927 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:39.926+0000 7f090a410040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:52:39.967 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:39.966+0000 7fc7dc630040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:52:40.219 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.218+0000 7f090a410040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:52:40.262 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:40.261+0000 7fc7dc630040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:52:40.275 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.274+0000 7f090a410040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:52:40.317 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:40.317+0000 7fc7dc630040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:52:40.388 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.387+0000 7f090a410040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:52:40.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:40.409+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 32) v1 ==== 88957+0+0 (secure 0 0 0) 0x7face8054010 con 0x7facfc15cce0 2023-10-08T05:52:40.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:40.409+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 msgr2=0x7face006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:40.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:40.409+0000 7facf97fa640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/3994892546,v1:172.21.15.181:6833/3994892546] conn(0x7face006afe0 0x7face006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:40.441 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:40.440+0000 7fc7dc630040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:52:40.564 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.562+0000 7f0886a87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:40.564 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.562+0000 7f0886a87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:40.564 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.562+0000 7f0886a87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:40.564 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.562+0000 7f0886a87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:40.564 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.562+0000 7f0886a87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:40.564 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.563+0000 7f0889acd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:40.565 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.563+0000 7f0889acd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:40.565 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.563+0000 7f0889acd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:40.565 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.563+0000 7f0889acd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:40.565 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:40.563+0000 7f0889acd640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.424+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 33) v1 ==== 89084+0+0 (secure 0 0 0) 0x7face8057b60 con 0x7facfc15cce0 2023-10-08T05:52:41.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.425+0000 7facf97fa640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7face00355a0 0x7face0035980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:41.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.425+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 --> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7facfc0b3f90 con 0x7face00355a0 2023-10-08T05:52:41.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.427+0000 7fad00a1f640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7face00355a0 0x7face0035980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:41.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.430+0000 7fad00a1f640 1 --2- 172.21.15.70:0/2942735308 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7face00355a0 0x7face0035980 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7facf0036000 tx=0x7facf0008030 comp rx=0 tx=0).ready entity=mgr.5222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:41.452 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:41.452+0000 7f0890adb640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-08T05:52:41.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.453+0000 7facf97fa640 1 -- 172.21.15.70:0/2942735308 <== mgr.5222 v2:172.21.15.181:6832/1709912921 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7facfc0b3f90 con 0x7face00355a0 2023-10-08T05:52:41.453 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-08T05:52:41.473 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T05:52:41.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:41.474 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T05:52:41.532 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T05:52:41.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.745+0000 7ff551c32640 1 Processor -- start 2023-10-08T05:52:41.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.745+0000 7ff551c32640 1 -- start start 2023-10-08T05:52:41.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.747+0000 7ff551c32640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff54c0cb880 0x7ff54c0b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:41.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.747+0000 7ff551c32640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c15d440 con 0x7ff54c1432f0 2023-10-08T05:52:41.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.747+0000 7ff551c32640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c15d690 con 0x7ff54c0b4070 2023-10-08T05:52:41.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.747+0000 7ff551c32640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1436d0 con 0x7ff54c0cb880 2023-10-08T05:52:41.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.747+0000 7ff54b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff54c1432f0 0x7ff54c0cb170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38698/0 (socket says 172.21.15.70:38698) 2023-10-08T05:52:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.747+0000 7ff54b7fe640 1 -- 172.21.15.70:0/2833069118 learned_addr learned my addr 172.21.15.70:0/2833069118 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff543fff640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff54c0cb880 0x7ff54c0b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87071844 0 0) 0x7ff54c15d440 con 0x7ff54c1432f0 2023-10-08T05:52:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff538003520 con 0x7ff54c1432f0 2023-10-08T05:52:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3025566138 0 0) 0x7ff538003520 con 0x7ff54c1432f0 2023-10-08T05:52:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff54c0cb880 msgr2=0x7ff54c0b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff54c0cb880 0x7ff54c0b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 >> v1:172.21.15.181:6789/0 conn(0x7ff54c0b4070 legacy=0x7ff54c159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c15d970 con 0x7ff54c1432f0 2023-10-08T05:52:41.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff53c001670 con 0x7ff54c1432f0 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 >> v1:172.21.15.70:6789/0 conn(0x7ff54c1432f0 legacy=0x7ff54c0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5380039a0 0x7ff538003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff54c0cb880 0x7ff5380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff538004d00 0x7ff538005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1436d0 con 0x7ff5380039a0 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53800dd20 con 0x7ff538004d00 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.748+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53800dee0 con 0x7ff54c0cb880 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.749+0000 7ff54b7fe640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5380039a0 0x7ff538003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.749+0000 7ff54bfff640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff538004d00 0x7ff538005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.749+0000 7ff54b7fe640 1 -- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff54c0cb880 msgr2=0x7ff5380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.749+0000 7ff543fff640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff54c0cb880 0x7ff5380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.749+0000 7ff54b7fe640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff54c0cb880 0x7ff5380047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.749+0000 7ff54b7fe640 1 -- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff538004d00 msgr2=0x7ff538005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.749+0000 7ff54b7fe640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff538004d00 0x7ff538005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.749+0000 7ff54b7fe640 1 -- 172.21.15.70:0/2833069118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff538003520 con 0x7ff5380039a0 2023-10-08T05:52:41.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.750+0000 7ff54b7fe640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5380039a0 0x7ff538003dc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00c0e0 tx=0x7ff5380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3104c6671ce8c45a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.750+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c005ad0 con 0x7ff5380039a0 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.750+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c005c30 con 0x7ff5380039a0 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.750+0000 7ff54affd640 1 -- 172.21.15.70:0/2833069118 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c02c7f0 con 0x7ff5380039a0 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.751+0000 7ff551c32640 1 -- 172.21.15.70:0/2833069118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5380039a0 msgr2=0x7ff538003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.751+0000 7ff551c32640 1 --2- 172.21.15.70:0/2833069118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5380039a0 0x7ff538003dc0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff53c00c0e0 tx=0x7ff5380057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.751+0000 7ff54bfff640 1 -- 172.21.15.70:0/2833069118 reap_dead start 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.751+0000 7ff551c32640 1 -- 172.21.15.70:0/2833069118 shutdown_connections 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.751+0000 7ff551c32640 1 -- 172.21.15.70:0/2833069118 >> 172.21.15.70:0/2833069118 conn(0x7ff54c069d50 msgr2=0x7ff54c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.751+0000 7ff551c32640 1 -- 172.21.15.70:0/2833069118 shutdown_connections 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.751+0000 7ff551c32640 1 -- 172.21.15.70:0/2833069118 wait complete. 2023-10-08T05:52:41.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.753+0000 7ff551c32640 1 Processor -- start 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.753+0000 7ff551c32640 1 -- start start 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff551c32640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5380039a0 0x7ff54c136f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff551c32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff54c138b20 0x7ff54c0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff551c32640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff54c140fc0 0x7ff54c141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff551c32640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c141980 con 0x7ff54c138b20 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff551c32640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0ccc90 con 0x7ff54c140fc0 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff551c32640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0ccf70 con 0x7ff5380039a0 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff543fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff54c138b20 0x7ff54c0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff543fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff54c138b20 0x7ff54c0cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41970/0 (socket says 172.21.15.70:41970) 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff543fff640 1 -- 172.21.15.70:0/2568343124 learned_addr learned my addr 172.21.15.70:0/2568343124 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:41.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff543fff640 1 -- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5380039a0 msgr2=0x7ff54c136f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff543fff640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5380039a0 0x7ff54c136f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff543fff640 1 -- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff54c140fc0 msgr2=0x7ff54c141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff543fff640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff54c140fc0 0x7ff54c141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.754+0000 7ff543fff640 1 -- 172.21.15.70:0/2568343124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c0cd250 con 0x7ff54c138b20 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.755+0000 7ff543fff640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff54c138b20 0x7ff54c0cc2f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff534010860 tx=0x7ff534010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.755+0000 7ff548ff9640 1 -- 172.21.15.70:0/2568343124 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534009610 con 0x7ff54c138b20 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.755+0000 7ff548ff9640 1 -- 172.21.15.70:0/2568343124 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53400eb60 con 0x7ff54c138b20 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.755+0000 7ff551c32640 1 -- 172.21.15.70:0/2568343124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c13b600 con 0x7ff54c138b20 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.755+0000 7ff548ff9640 1 -- 172.21.15.70:0/2568343124 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53401c400 con 0x7ff54c138b20 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.755+0000 7ff551c32640 1 -- 172.21.15.70:0/2568343124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c138fa0 con 0x7ff54c138b20 2023-10-08T05:52:41.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.756+0000 7ff551c32640 1 -- 172.21.15.70:0/2568343124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c13b460 con 0x7ff54c138b20 2023-10-08T05:52:41.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.757+0000 7ff548ff9640 1 -- 172.21.15.70:0/2568343124 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 33) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ff53401c560 con 0x7ff54c138b20 2023-10-08T05:52:41.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.758+0000 7ff548ff9640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7ff52c06ad40 0x7ff52c06d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:41.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.758+0000 7ff548ff9640 1 -- 172.21.15.70:0/2568343124 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7ff534089990 con 0x7ff54c138b20 2023-10-08T05:52:41.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.759+0000 7ff54b7fe640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7ff52c06ad40 0x7ff52c06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:41.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.760+0000 7ff54b7fe640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7ff52c06ad40 0x7ff52c06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff53c034000 tx=0x7ff53c007000 comp rx=0 tx=0).ready entity=mgr.5222 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:41.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.762+0000 7ff548ff9640 1 -- 172.21.15.70:0/2568343124 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff54c13b460 con 0x7ff54c138b20 2023-10-08T05:52:41.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:41.981+0000 7ff551c32640 1 -- 172.21.15.70:0/2568343124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7ff54c13b460 con 0x7ff54c138b20 2023-10-08T05:52:42.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.467+0000 7ff548ff9640 1 -- 172.21.15.70:0/2568343124 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v34) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff54c13b460 con 0x7ff54c138b20 2023-10-08T05:52:42.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.471+0000 7ff551c32640 1 -- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7ff52c06ad40 msgr2=0x7ff52c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.471+0000 7ff551c32640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7ff52c06ad40 0x7ff52c06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff53c034000 tx=0x7ff53c007000 comp rx=0 tx=0).stop 2023-10-08T05:52:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.471+0000 7ff551c32640 1 -- 172.21.15.70:0/2568343124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff54c138b20 msgr2=0x7ff54c0cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.471+0000 7ff551c32640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff54c138b20 0x7ff54c0cc2f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff534010860 tx=0x7ff534010d20 comp rx=0 tx=0).stop 2023-10-08T05:52:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.472+0000 7ff551c32640 1 -- 172.21.15.70:0/2568343124 shutdown_connections 2023-10-08T05:52:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.472+0000 7ff551c32640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7ff52c06ad40 0x7ff52c06d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.472+0000 7ff551c32640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff54c140fc0 0x7ff54c141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.472+0000 7ff551c32640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff54c138b20 0x7ff54c0cc2f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.472+0000 7ff551c32640 1 --2- 172.21.15.70:0/2568343124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5380039a0 0x7ff54c136f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.472+0000 7ff551c32640 1 -- 172.21.15.70:0/2568343124 >> 172.21.15.70:0/2568343124 conn(0x7ff54c069d50 msgr2=0x7ff54c0ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.473+0000 7ff551c32640 1 -- 172.21.15.70:0/2568343124 shutdown_connections 2023-10-08T05:52:42.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:42.473+0000 7ff551c32640 1 -- 172.21.15.70:0/2568343124 wait complete. 2023-10-08T05:52:42.498 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:52:42.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:52:42.610 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:52:42.700 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:52:42.701 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:42.701 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:42.751 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:42.751 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:42.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:43.004 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.003+0000 7f48c922e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:52:43.022 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:43.021+0000 7fb441052040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:52:43.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.051+0000 7fd09ecbb640 1 Processor -- start 2023-10-08T05:52:43.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.052+0000 7fd09ecbb640 1 -- start start 2023-10-08T05:52:43.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.053+0000 7fd09ecbb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd09815ab40 0x7fd0980ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.053+0000 7fd09ecbb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0980c90b0 0x7fd0980adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.053+0000 7fd09ecbb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09815b2b0 con 0x7fd0980cae70 2023-10-08T05:52:43.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.053+0000 7fd09ecbb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09815bf30 con 0x7fd09815ab40 2023-10-08T05:52:43.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.053+0000 7fd09ecbb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09815f7a0 con 0x7fd0980c90b0 2023-10-08T05:52:43.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.054+0000 7fd09ca57640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd09815ab40 0x7fd0980ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.054+0000 7fd09ca57640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd09815ab40 0x7fd0980ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41986/0 (socket says 172.21.15.70:41986) 2023-10-08T05:52:43.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.054+0000 7fd09ca57640 1 -- 172.21.15.70:0/1922632225 learned_addr learned my addr 172.21.15.70:0/1922632225 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:43.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.054+0000 7fd097fff640 1 --2- 172.21.15.70:0/1922632225 >> v2:172.21.15.181:3300/0 conn(0x7fd0980c90b0 0x7fd0980adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.054+0000 7fd09ca57640 1 -- 172.21.15.70:0/1922632225 >> v2:172.21.15.181:3300/0 conn(0x7fd0980c90b0 msgr2=0x7fd0980adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:43.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.054+0000 7fd09ca57640 1 --2- 172.21.15.70:0/1922632225 >> v2:172.21.15.181:3300/0 conn(0x7fd0980c90b0 0x7fd0980adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.054+0000 7fd09ca57640 1 -- 172.21.15.70:0/1922632225 >> v1:172.21.15.181:6789/0 conn(0x7fd0980cae70 legacy=0x7fd0980702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:43.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.054+0000 7fd09ca57640 1 -- 172.21.15.70:0/1922632225 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09815b590 con 0x7fd09815ab40 2023-10-08T05:52:43.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.055+0000 7fd09ca57640 1 --2- 172.21.15.70:0/1922632225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 0x7fd0980ad720 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd09806e560 tx=0x7fd08c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78254d67d1665b8c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:43.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.056+0000 7fd0977fe640 1 -- 172.21.15.70:0/1922632225 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c031870 con 0x7fd09815ab40 2023-10-08T05:52:43.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.056+0000 7fd0977fe640 1 -- 172.21.15.70:0/1922632225 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c0319d0 con 0x7fd09815ab40 2023-10-08T05:52:43.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.056+0000 7fd0977fe640 1 -- 172.21.15.70:0/1922632225 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c031d60 con 0x7fd09815ab40 2023-10-08T05:52:43.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.056+0000 7fd09ecbb640 1 -- 172.21.15.70:0/1922632225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 msgr2=0x7fd0980ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.056+0000 7fd09ecbb640 1 --2- 172.21.15.70:0/1922632225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 0x7fd0980ad720 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd09806e560 tx=0x7fd08c02b910 comp rx=0 tx=0).stop 2023-10-08T05:52:43.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.056+0000 7fd09ecbb640 1 -- 172.21.15.70:0/1922632225 shutdown_connections 2023-10-08T05:52:43.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.056+0000 7fd09ecbb640 1 --2- 172.21.15.70:0/1922632225 >> v2:172.21.15.181:3300/0 conn(0x7fd0980c90b0 0x7fd0980adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.056+0000 7fd09ecbb640 1 --2- 172.21.15.70:0/1922632225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 0x7fd0980ad720 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.056+0000 7fd09ecbb640 1 -- 172.21.15.70:0/1922632225 >> 172.21.15.70:0/1922632225 conn(0x7fd09806a380 msgr2=0x7fd0980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:43.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.057+0000 7fd09ecbb640 1 -- 172.21.15.70:0/1922632225 shutdown_connections 2023-10-08T05:52:43.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.057+0000 7fd09ecbb640 1 -- 172.21.15.70:0/1922632225 wait complete. 2023-10-08T05:52:43.058 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.055+0000 7f48c922e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:52:43.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.058+0000 7fd09ecbb640 1 Processor -- start 2023-10-08T05:52:43.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.058+0000 7fd09ecbb640 1 -- start start 2023-10-08T05:52:43.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.059+0000 7fd09ecbb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0980c90b0 0x7fd0980c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.059+0000 7fd09ecbb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0980cae70 0x7fd0980c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.059+0000 7fd09ecbb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 0x7fd0980c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.059+0000 7fd09ecbb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09815d540 con 0x7fd09815ab40 2023-10-08T05:52:43.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.059+0000 7fd09ecbb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980c6630 con 0x7fd0980c90b0 2023-10-08T05:52:43.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd09ecbb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980c53e0 con 0x7fd0980cae70 2023-10-08T05:52:43.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd09d258640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 0x7fd0980c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd09d258640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 0x7fd0980c5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41994/0 (socket says 172.21.15.70:41994) 2023-10-08T05:52:43.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd09d258640 1 -- 172.21.15.70:0/2839978353 learned_addr learned my addr 172.21.15.70:0/2839978353 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:43.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd09ca57640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0980c90b0 0x7fd0980c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd097fff640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0980cae70 0x7fd0980c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd09d258640 1 -- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0980cae70 msgr2=0x7fd0980c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd09d258640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0980cae70 0x7fd0980c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd09d258640 1 -- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0980c90b0 msgr2=0x7fd0980c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd09d258640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0980c90b0 0x7fd0980c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.060+0000 7fd09d258640 1 -- 172.21.15.70:0/2839978353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c0094d0 con 0x7fd09815ab40 2023-10-08T05:52:43.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.061+0000 7fd09d258640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 0x7fd0980c5d90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd09000c780 tx=0x7fd09000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:43.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.061+0000 7fd0957fa640 1 -- 172.21.15.70:0/2839978353 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09000ce90 con 0x7fd09815ab40 2023-10-08T05:52:43.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.061+0000 7fd0957fa640 1 -- 172.21.15.70:0/2839978353 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd090006b50 con 0x7fd09815ab40 2023-10-08T05:52:43.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.061+0000 7fd0957fa640 1 -- 172.21.15.70:0/2839978353 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09000a850 con 0x7fd09815ab40 2023-10-08T05:52:43.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.061+0000 7fd09ecbb640 1 -- 172.21.15.70:0/2839978353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09815cef0 con 0x7fd09815ab40 2023-10-08T05:52:43.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.061+0000 7fd09ecbb640 1 -- 172.21.15.70:0/2839978353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0980c62d0 con 0x7fd09815ab40 2023-10-08T05:52:43.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.062+0000 7fd09ecbb640 1 -- 172.21.15.70:0/2839978353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09815cd50 con 0x7fd09815ab40 2023-10-08T05:52:43.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.063+0000 7fd0957fa640 1 -- 172.21.15.70:0/2839978353 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fd09000a9b0 con 0x7fd09815ab40 2023-10-08T05:52:43.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.064+0000 7fd0957fa640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fd07c06afc0 0x7fd07c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.064+0000 7fd0957fa640 1 -- 172.21.15.70:0/2839978353 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fd090089b00 con 0x7fd09815ab40 2023-10-08T05:52:43.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.064+0000 7fd09ca57640 1 -- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fd07c06afc0 msgr2=0x7fd07c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:43.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.064+0000 7fd09ca57640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fd07c06afc0 0x7fd07c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:43.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.073+0000 7fd0957fa640 1 -- 172.21.15.70:0/2839978353 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd09815cd50 con 0x7fd09815ab40 2023-10-08T05:52:43.084 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:43.084+0000 7fb441052040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:52:43.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.265+0000 7fd09ca57640 1 -- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fd07c06afc0 msgr2=0x7fd07c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:43.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.265+0000 7fd09ca57640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fd07c06afc0 0x7fd07c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:43.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.268+0000 7fd09ecbb640 1 -- 172.21.15.70:0/2839978353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd09815cd50 con 0x7fd09815ab40 2023-10-08T05:52:43.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.270+0000 7fd0957fa640 1 -- 172.21.15.70:0/2839978353 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+17282 (secure 0 0 0) 0x7fd09815cd50 con 0x7fd09815ab40 2023-10-08T05:52:43.270 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:43.270 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:40.392647+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:52:23.846215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-08T05:52:25.069536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/814792959":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/2758467223":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1699086184":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1378633292":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1217040450":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/2333154107":"2023-10-09T05:52:40.392573+0000","172.21.15.181:6833/3994892546":"2023-10-09T05:52:40.392573+0000","172.21.15.181:6832/3994892546":"2023-10-09T05:52:40.392573+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:43.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 -- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fd07c06afc0 msgr2=0x7fd07c06d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fd07c06afc0 0x7fd07c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 -- 172.21.15.70:0/2839978353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 msgr2=0x7fd0980c5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 0x7fd0980c5d90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd09000c780 tx=0x7fd09000cc40 comp rx=0 tx=0).stop 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 -- 172.21.15.70:0/2839978353 shutdown_connections 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fd07c06afc0 0x7fd07c06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd09815ab40 0x7fd0980c5d90 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0980cae70 0x7fd0980c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 --2- 172.21.15.70:0/2839978353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0980c90b0 0x7fd0980c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 -- 172.21.15.70:0/2839978353 >> 172.21.15.70:0/2839978353 conn(0x7fd09806a380 msgr2=0x7fd0980cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 -- 172.21.15.70:0/2839978353 shutdown_connections 2023-10-08T05:52:43.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.272+0000 7fd09ecbb640 1 -- 172.21.15.70:0/2839978353 wait complete. 2023-10-08T05:52:43.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:52:23.846215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:52:25.069536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:52:43.291 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:52:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.498+0000 7f402dbd0640 1 Processor -- start 2023-10-08T05:52:43.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.498+0000 7f402dbd0640 1 -- start start 2023-10-08T05:52:43.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.500+0000 7f402dbd0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f402815aad0 0x7f40280b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.500+0000 7f402dbd0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402815bf30 con 0x7f40280c82b0 2023-10-08T05:52:43.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.500+0000 7f402dbd0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402815b320 con 0x7f402806d410 2023-10-08T05:52:43.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.500+0000 7f402dbd0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402815b600 con 0x7f402815aad0 2023-10-08T05:52:43.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.500+0000 7f401ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f40280c82b0 0x7f402806cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38714/0 (socket says 172.21.15.70:38714) 2023-10-08T05:52:43.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.500+0000 7f401ffff640 1 -- 172.21.15.70:0/3124894898 learned_addr learned my addr 172.21.15.70:0/3124894898 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:43.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.500+0000 7f40277fe640 1 --2- 172.21.15.70:0/3124894898 >> v2:172.21.15.181:3300/0 conn(0x7f402815aad0 0x7f40280b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195484515 0 0) 0x7f402815bf30 con 0x7f40280c82b0 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4014003520 con 0x7f40280c82b0 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2320236740 0 0) 0x7f4014003520 con 0x7f40280c82b0 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 >> v2:172.21.15.181:3300/0 conn(0x7f402815aad0 msgr2=0x7f40280b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 --2- 172.21.15.70:0/3124894898 >> v2:172.21.15.181:3300/0 conn(0x7f402815aad0 0x7f40280b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 >> v1:172.21.15.181:6789/0 conn(0x7f402806d410 legacy=0x7f4028070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402815b8e0 con 0x7f40280c82b0 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4010001690 con 0x7f40280c82b0 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 >> v1:172.21.15.70:6789/0 conn(0x7f40280c82b0 legacy=0x7f402806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4014003970 0x7f4014003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40140047c0 0x7f4014004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4014005180 0x7f401400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402815b600 con 0x7f4014003970 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401400c000 con 0x7f4014005180 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.501+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401400c1c0 con 0x7f40140047c0 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f40277fe640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4014003970 0x7f4014003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f4027fff640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4014005180 0x7f401400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f401ffff640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40140047c0 0x7f4014004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f40277fe640 1 -- 172.21.15.70:0/3124894898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40140047c0 msgr2=0x7f4014004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f40277fe640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40140047c0 0x7f4014004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f40277fe640 1 -- 172.21.15.70:0/3124894898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4014005180 msgr2=0x7f401400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f40277fe640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4014005180 0x7f401400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f40277fe640 1 -- 172.21.15.70:0/3124894898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4014003520 con 0x7f4014003970 2023-10-08T05:52:43.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f4027fff640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4014005180 0x7f401400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:43.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f4027fff640 1 -- 172.21.15.70:0/3124894898 reap_dead start 2023-10-08T05:52:43.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.502+0000 7f401ffff640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40140047c0 0x7f4014004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:43.505 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.503+0000 7f48c922e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:52:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.503+0000 7f40277fe640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4014003970 0x7f4014003d70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4018009ac0 tx=0x7f40140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634f398f3f5c7e72 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.503+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401802c6d0 con 0x7f4014003970 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.503+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401802c830 con 0x7f4014003970 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.504+0000 7f4026ffd640 1 -- 172.21.15.70:0/3124894898 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401802cbc0 con 0x7f4014003970 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.504+0000 7f402dbd0640 1 -- 172.21.15.70:0/3124894898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4014003970 msgr2=0x7f4014003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.504+0000 7f402dbd0640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4014003970 0x7f4014003d70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f4018009ac0 tx=0x7f40140057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.504+0000 7f402dbd0640 1 -- 172.21.15.70:0/3124894898 shutdown_connections 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.504+0000 7f402dbd0640 1 --2- 172.21.15.70:0/3124894898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4014003970 0x7f4014003d70 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.504+0000 7f402dbd0640 1 -- 172.21.15.70:0/3124894898 >> 172.21.15.70:0/3124894898 conn(0x7f4028069d50 msgr2=0x7f402806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.504+0000 7f402dbd0640 1 -- 172.21.15.70:0/3124894898 shutdown_connections 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.504+0000 7f402dbd0640 1 -- 172.21.15.70:0/3124894898 wait complete. 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.506+0000 7f402dbd0640 1 Processor -- start 2023-10-08T05:52:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.506+0000 7f402dbd0640 1 -- start start 2023-10-08T05:52:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.507+0000 7f402dbd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4014003970 0x7f402814c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.507+0000 7f402dbd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40281ef8e0 0x7f40280cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.507+0000 7f402dbd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40280cdb20 0x7f40281f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.507+0000 7f402dbd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40280cdfa0 con 0x7f40281ef8e0 2023-10-08T05:52:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.507+0000 7f402dbd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402814c5d0 con 0x7f40280cdb20 2023-10-08T05:52:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.507+0000 7f402dbd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40281f4500 con 0x7f4014003970 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.507+0000 7f401ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40281ef8e0 0x7f40280cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.507+0000 7f401ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40281ef8e0 0x7f40280cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42014/0 (socket says 172.21.15.70:42014) 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.507+0000 7f401ffff640 1 -- 172.21.15.70:0/1921623477 learned_addr learned my addr 172.21.15.70:0/1921623477 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f401ffff640 1 -- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4014003970 msgr2=0x7f402814c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f40277fe640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4014003970 0x7f402814c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f4027fff640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40280cdb20 0x7f40281f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f401ffff640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4014003970 0x7f402814c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f401ffff640 1 -- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40280cdb20 msgr2=0x7f40281f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f401ffff640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40280cdb20 0x7f40281f37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f401ffff640 1 -- 172.21.15.70:0/1921623477 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4028150c70 con 0x7f40281ef8e0 2023-10-08T05:52:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f40277fe640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4014003970 0x7f402814c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f401ffff640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40281ef8e0 0x7f40280cd5e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4010000fd0 tx=0x7f401000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f4024ff9640 1 -- 172.21.15.70:0/1921623477 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40100072d0 con 0x7f40281ef8e0 2023-10-08T05:52:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f4024ff9640 1 -- 172.21.15.70:0/1921623477 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4010007450 con 0x7f40281ef8e0 2023-10-08T05:52:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f4024ff9640 1 -- 172.21.15.70:0/1921623477 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40100047e0 con 0x7f40281ef8e0 2023-10-08T05:52:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.508+0000 7f402dbd0640 1 -- 172.21.15.70:0/1921623477 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4028150f50 con 0x7f40281ef8e0 2023-10-08T05:52:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.509+0000 7f402dbd0640 1 -- 172.21.15.70:0/1921623477 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4028151510 con 0x7f40281ef8e0 2023-10-08T05:52:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.510+0000 7f402dbd0640 1 -- 172.21.15.70:0/1921623477 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402804e160 con 0x7f40281ef8e0 2023-10-08T05:52:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.511+0000 7f4024ff9640 1 -- 172.21.15.70:0/1921623477 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f4010004b70 con 0x7f40281ef8e0 2023-10-08T05:52:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.511+0000 7f4024ff9640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f400806af90 0x7f400806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.511+0000 7f4024ff9640 1 -- 172.21.15.70:0/1921623477 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f4010089670 con 0x7f40281ef8e0 2023-10-08T05:52:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.512+0000 7f40277fe640 1 -- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f400806af90 msgr2=0x7f400806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:43.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.512+0000 7f40277fe640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f400806af90 0x7f400806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:43.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.516+0000 7f4024ff9640 1 -- 172.21.15.70:0/1921623477 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4010054190 con 0x7f40281ef8e0 2023-10-08T05:52:43.556 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:43.556+0000 7fb441052040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:52:43.556 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.555+0000 7f48c922e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:52:43.606 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.605+0000 7f48c922e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:52:43.610 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:43.610+0000 7fb441052040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:52:43.662 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:43.662+0000 7fb441052040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:52:43.666 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.665+0000 7f48c922e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:52:43.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.706+0000 7f402dbd0640 1 -- 172.21.15.70:0/1921623477 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f402806eb20 con 0x7f40281ef8e0 2023-10-08T05:52:43.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.707+0000 7f4024ff9640 1 -- 172.21.15.70:0/1921623477 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40100577b0 con 0x7f40281ef8e0 2023-10-08T05:52:43.707 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:52:43.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 -- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f400806af90 msgr2=0x7f400806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:43.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f400806af90 0x7f400806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 -- 172.21.15.70:0/1921623477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40281ef8e0 msgr2=0x7f40280cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40281ef8e0 0x7f40280cd5e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f4010000fd0 tx=0x7f401000db90 comp rx=0 tx=0).stop 2023-10-08T05:52:43.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 -- 172.21.15.70:0/1921623477 shutdown_connections 2023-10-08T05:52:43.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f400806af90 0x7f400806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40280cdb20 0x7f40281f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40281ef8e0 0x7f40280cd5e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 --2- 172.21.15.70:0/1921623477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4014003970 0x7f402814c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 -- 172.21.15.70:0/1921623477 >> 172.21.15.70:0/1921623477 conn(0x7f4028069d50 msgr2=0x7f40280ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:43.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.709+0000 7f402dbd0640 1 -- 172.21.15.70:0/1921623477 shutdown_connections 2023-10-08T05:52:43.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.710+0000 7f402dbd0640 1 -- 172.21.15.70:0/1921623477 wait complete. 2023-10-08T05:52:43.714 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.713+0000 7f48c922e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:52:43.726 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:43.726+0000 7fb441052040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:52:43.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:52:43.778 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:43.777+0000 7fb441052040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:52:43.823 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.822+0000 7f48c922e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:52:43.884 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.883+0000 7f48c922e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:43.894 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:43.894+0000 7fb441052040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:52:43.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.930+0000 7f5349e07640 1 Processor -- start 2023-10-08T05:52:43.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.931+0000 7f5349e07640 1 -- start start 2023-10-08T05:52:43.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.932+0000 7f5349e07640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f53440c9cf0 0x7f534406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.932+0000 7f5349e07640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f534415aa00 0x7f53440b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.932+0000 7f5349e07640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534415b310 con 0x7f534406c3a0 2023-10-08T05:52:43.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.932+0000 7f5349e07640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534406e0a0 con 0x7f53440c9cf0 2023-10-08T05:52:43.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.932+0000 7f5349e07640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534406d290 con 0x7f534415aa00 2023-10-08T05:52:43.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.932+0000 7f53437fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f534406c3a0 0x7f534406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58466/0 (socket says 172.21.15.70:58466) 2023-10-08T05:52:43.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.932+0000 7f53437fe640 1 -- 172.21.15.70:0/3359342573 learned_addr learned my addr 172.21.15.70:0/3359342573 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:43.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.932+0000 7f533affd640 1 --2- 172.21.15.70:0/3359342573 >> v2:172.21.15.70:3300/0 conn(0x7f53440c9cf0 0x7f534406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.933+0000 7f5343fff640 1 --2- 172.21.15.70:0/3359342573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f534415aa00 0x7f53440b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.933+0000 7f533affd640 1 -- 172.21.15.70:0/3359342573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f534415aa00 msgr2=0x7f53440b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.933+0000 7f533affd640 1 --2- 172.21.15.70:0/3359342573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f534415aa00 0x7f53440b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.933+0000 7f533affd640 1 -- 172.21.15.70:0/3359342573 >> v1:172.21.15.70:6789/0 conn(0x7f534406c3a0 legacy=0x7f534406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.933+0000 7f533affd640 1 -- 172.21.15.70:0/3359342573 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53440c9860 con 0x7f53440c9cf0 2023-10-08T05:52:43.935 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.931+0000 7f48c922e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:52:43.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.934+0000 7f533affd640 1 --2- 172.21.15.70:0/3359342573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53440c9cf0 0x7f534406db60 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f532c0097a0 tx=0x7f532c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e5e3b14bfa6fa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.934+0000 7f5342ffd640 1 -- 172.21.15.70:0/3359342573 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c02d040 con 0x7f53440c9cf0 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.934+0000 7f5342ffd640 1 -- 172.21.15.70:0/3359342573 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c031810 con 0x7f53440c9cf0 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.934+0000 7f5342ffd640 1 -- 172.21.15.70:0/3359342573 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c031ba0 con 0x7f53440c9cf0 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.935+0000 7f5349e07640 1 -- 172.21.15.70:0/3359342573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53440c9cf0 msgr2=0x7f534406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.935+0000 7f5349e07640 1 --2- 172.21.15.70:0/3359342573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53440c9cf0 0x7f534406db60 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f532c0097a0 tx=0x7f532c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.935+0000 7f5349e07640 1 -- 172.21.15.70:0/3359342573 shutdown_connections 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.935+0000 7f5349e07640 1 --2- 172.21.15.70:0/3359342573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f534415aa00 0x7f53440b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.935+0000 7f5349e07640 1 --2- 172.21.15.70:0/3359342573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53440c9cf0 0x7f534406db60 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.935+0000 7f5349e07640 1 -- 172.21.15.70:0/3359342573 >> 172.21.15.70:0/3359342573 conn(0x7f534406a5d0 msgr2=0x7f53440b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.935+0000 7f5349e07640 1 -- 172.21.15.70:0/3359342573 shutdown_connections 2023-10-08T05:52:43.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.935+0000 7f5349e07640 1 -- 172.21.15.70:0/3359342573 wait complete. 2023-10-08T05:52:43.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.937+0000 7f5349e07640 1 Processor -- start 2023-10-08T05:52:43.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.937+0000 7f5349e07640 1 -- start start 2023-10-08T05:52:43.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f5349e07640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f534406c3a0 0x7f53441f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f5349e07640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53440c9cf0 0x7f53440c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f5349e07640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f534415aa00 0x7f53440c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f5349e07640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441f34a0 con 0x7f534406c3a0 2023-10-08T05:52:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f5349e07640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441f2c00 con 0x7f534415aa00 2023-10-08T05:52:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f5349e07640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53441f2ee0 con 0x7f53440c9cf0 2023-10-08T05:52:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f53437fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f534406c3a0 0x7f53441f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f53437fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f534406c3a0 0x7f53441f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46182/0 (socket says 172.21.15.70:46182) 2023-10-08T05:52:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f53437fe640 1 -- 172.21.15.70:0/2079496389 learned_addr learned my addr 172.21.15.70:0/2079496389 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f533affd640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53440c9cf0 0x7f53440c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f5343fff640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f534415aa00 0x7f53440c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f53437fe640 1 -- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53440c9cf0 msgr2=0x7f53440c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f53437fe640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53440c9cf0 0x7f53440c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f53437fe640 1 -- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f534415aa00 msgr2=0x7f53440c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.938+0000 7f53437fe640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f534415aa00 0x7f53440c4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:43.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.939+0000 7f53437fe640 1 -- 172.21.15.70:0/2079496389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c009400 con 0x7f534406c3a0 2023-10-08T05:52:43.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.939+0000 7f53437fe640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f534406c3a0 0x7f53441f2520 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f533400cba0 tx=0x7f533400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:43.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.939+0000 7f5340ff9640 1 -- 172.21.15.70:0/2079496389 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533400ad40 con 0x7f534406c3a0 2023-10-08T05:52:43.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.939+0000 7f5340ff9640 1 -- 172.21.15.70:0/2079496389 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5334012400 con 0x7f534406c3a0 2023-10-08T05:52:43.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.939+0000 7f5349e07640 1 -- 172.21.15.70:0/2079496389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53441f31c0 con 0x7f534406c3a0 2023-10-08T05:52:43.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.939+0000 7f5340ff9640 1 -- 172.21.15.70:0/2079496389 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533400aea0 con 0x7f534406c3a0 2023-10-08T05:52:43.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.939+0000 7f5349e07640 1 -- 172.21.15.70:0/2079496389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53440c2e50 con 0x7f534406c3a0 2023-10-08T05:52:43.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.940+0000 7f5349e07640 1 -- 172.21.15.70:0/2079496389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534404e160 con 0x7f534406c3a0 2023-10-08T05:52:43.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.942+0000 7f5340ff9640 1 -- 172.21.15.70:0/2079496389 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f5334007070 con 0x7f534406c3a0 2023-10-08T05:52:43.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.943+0000 7f5340ff9640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f532406af90 0x7f532406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:43.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.943+0000 7f5340ff9640 1 -- 172.21.15.70:0/2079496389 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f5334089770 con 0x7f534406c3a0 2023-10-08T05:52:43.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.943+0000 7f533affd640 1 -- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f532406af90 msgr2=0x7f532406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:43.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.943+0000 7f533affd640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f532406af90 0x7f532406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:43.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:43.948+0000 7f5340ff9640 1 -- 172.21.15.70:0/2079496389 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5334054580 con 0x7f534406c3a0 2023-10-08T05:52:43.963 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:43.963+0000 7fb441052040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:43.990 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:43.989+0000 7f48c922e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:52:44.015 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:44.015+0000 7fb441052040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:52:44.040 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:44.039+0000 7f48c922e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:52:44.074 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:44.074+0000 7fb441052040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:52:44.126 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:44.126+0000 7fb441052040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:52:44.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.136+0000 7f5349e07640 1 -- 172.21.15.70:0/2079496389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f534415b310 con 0x7f534406c3a0 2023-10-08T05:52:44.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.137+0000 7f5340ff9640 1 -- 172.21.15.70:0/2079496389 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f534415b310 con 0x7f534406c3a0 2023-10-08T05:52:44.137 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 -- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f532406af90 msgr2=0x7f532406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f532406af90 0x7f532406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 -- 172.21.15.70:0/2079496389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f534406c3a0 msgr2=0x7f53441f2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f534406c3a0 0x7f53441f2520 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f533400cba0 tx=0x7f533400a430 comp rx=0 tx=0).stop 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 -- 172.21.15.70:0/2079496389 shutdown_connections 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f532406af90 0x7f532406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f534415aa00 0x7f53440c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53440c9cf0 0x7f53440c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 --2- 172.21.15.70:0/2079496389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f534406c3a0 0x7f53441f2520 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 -- 172.21.15.70:0/2079496389 >> 172.21.15.70:0/2079496389 conn(0x7f534406a5d0 msgr2=0x7f534406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:44.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 -- 172.21.15.70:0/2079496389 shutdown_connections 2023-10-08T05:52:44.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.139+0000 7f5349e07640 1 -- 172.21.15.70:0/2079496389 wait complete. 2023-10-08T05:52:44.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:52:44.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.359+0000 7fc3766c9640 1 Processor -- start 2023-10-08T05:52:44.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.359+0000 7fc3766c9640 1 -- start start 2023-10-08T05:52:44.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.361+0000 7fc3766c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc37006db60 0x7fc3700b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.361+0000 7fc3766c9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37015b310 con 0x7fc3700c9cf0 2023-10-08T05:52:44.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.361+0000 7fc3766c9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37006d290 con 0x7fc37006c3a0 2023-10-08T05:52:44.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.361+0000 7fc3766c9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37015b110 con 0x7fc37006db60 2023-10-08T05:52:44.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.361+0000 7fc374c66640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc37006db60 0x7fc3700b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:44.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.361+0000 7fc367fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc3700c9cf0 0x7fc37015aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58482/0 (socket says 172.21.15.70:58482) 2023-10-08T05:52:44.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.361+0000 7fc367fff640 1 -- 172.21.15.70:0/398707251 learned_addr learned my addr 172.21.15.70:0/398707251 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:44.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.361+0000 7fc374c66640 1 -- 172.21.15.70:0/398707251 >> v1:172.21.15.181:6789/0 conn(0x7fc37006c3a0 legacy=0x7fc37006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:44.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.361+0000 7fc36f7fe640 1 -- 172.21.15.70:0/398707251 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936479940 0 0) 0x7fc358001110 con 0x7fc3700c9cf0 2023-10-08T05:52:44.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.361+0000 7fc374c66640 1 -- 172.21.15.70:0/398707251 >> v1:172.21.15.70:6789/0 conn(0x7fc3700c9cf0 legacy=0x7fc37015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:44.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.362+0000 7fc374c66640 1 -- 172.21.15.70:0/398707251 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3700c9860 con 0x7fc37006db60 2023-10-08T05:52:44.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.363+0000 7fc374c66640 1 --2- 172.21.15.70:0/398707251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006db60 0x7fc3700b2a90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc368009d90 tx=0x7fc36802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86f1ccecbdf5d46 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:44.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.363+0000 7fc36f7fe640 1 -- 172.21.15.70:0/398707251 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36802ee70 con 0x7fc37006db60 2023-10-08T05:52:44.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.363+0000 7fc36f7fe640 1 -- 172.21.15.70:0/398707251 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc368036440 con 0x7fc37006db60 2023-10-08T05:52:44.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.363+0000 7fc36f7fe640 1 -- 172.21.15.70:0/398707251 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3680367d0 con 0x7fc37006db60 2023-10-08T05:52:44.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.363+0000 7fc3766c9640 1 -- 172.21.15.70:0/398707251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006db60 msgr2=0x7fc3700b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:44.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.363+0000 7fc3766c9640 1 --2- 172.21.15.70:0/398707251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006db60 0x7fc3700b2a90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc368009d90 tx=0x7fc36802e0c0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.364+0000 7fc3766c9640 1 -- 172.21.15.70:0/398707251 shutdown_connections 2023-10-08T05:52:44.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.364+0000 7fc3766c9640 1 --2- 172.21.15.70:0/398707251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006db60 0x7fc3700b2a90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.364+0000 7fc3766c9640 1 -- 172.21.15.70:0/398707251 >> 172.21.15.70:0/398707251 conn(0x7fc37006a5d0 msgr2=0x7fc3700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:44.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.364+0000 7fc3766c9640 1 -- 172.21.15.70:0/398707251 shutdown_connections 2023-10-08T05:52:44.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.364+0000 7fc3766c9640 1 -- 172.21.15.70:0/398707251 wait complete. 2023-10-08T05:52:44.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.365+0000 7fc3766c9640 1 Processor -- start 2023-10-08T05:52:44.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.366+0000 7fc3766c9640 1 -- start start 2023-10-08T05:52:44.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.366+0000 7fc3766c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006c3a0 0x7fc3700c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.366+0000 7fc3766c9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc37006db60 0x7fc3700c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.366+0000 7fc3766c9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3700c9cf0 0x7fc3700c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.366+0000 7fc3766c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700c8320 con 0x7fc37006c3a0 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.366+0000 7fc3766c9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700c5810 con 0x7fc3700c9cf0 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.366+0000 7fc3766c9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3700c5af0 con 0x7fc37006db60 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.367+0000 7fc36ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006c3a0 0x7fc3700c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.367+0000 7fc36ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006c3a0 0x7fc3700c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46206/0 (socket says 172.21.15.70:46206) 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.367+0000 7fc36ffff640 1 -- 172.21.15.70:0/161757860 learned_addr learned my addr 172.21.15.70:0/161757860 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.367+0000 7fc36ffff640 1 -- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc37006db60 msgr2=0x7fc3700c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.367+0000 7fc36ffff640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc37006db60 0x7fc3700c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.367+0000 7fc36ffff640 1 -- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3700c9cf0 msgr2=0x7fc3700c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.367+0000 7fc36ffff640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3700c9cf0 0x7fc3700c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.367+0000 7fc36ffff640 1 -- 172.21.15.70:0/161757860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368009a70 con 0x7fc37006c3a0 2023-10-08T05:52:44.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.367+0000 7fc36ffff640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006c3a0 0x7fc3700c4670 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc36000c430 tx=0x7fc36000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.368+0000 7fc36d7fa640 1 -- 172.21.15.70:0/161757860 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360011790 con 0x7fc37006c3a0 2023-10-08T05:52:44.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.368+0000 7fc36d7fa640 1 -- 172.21.15.70:0/161757860 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3600118f0 con 0x7fc37006c3a0 2023-10-08T05:52:44.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.368+0000 7fc3766c9640 1 -- 172.21.15.70:0/161757860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3700c7a80 con 0x7fc37006c3a0 2023-10-08T05:52:44.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.368+0000 7fc36d7fa640 1 -- 172.21.15.70:0/161757860 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360011ca0 con 0x7fc37006c3a0 2023-10-08T05:52:44.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.368+0000 7fc3766c9640 1 -- 172.21.15.70:0/161757860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3700c8040 con 0x7fc37006c3a0 2023-10-08T05:52:44.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.369+0000 7fc3766c9640 1 -- 172.21.15.70:0/161757860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37004e160 con 0x7fc37006c3a0 2023-10-08T05:52:44.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.370+0000 7fc36d7fa640 1 -- 172.21.15.70:0/161757860 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fc36001c460 con 0x7fc37006c3a0 2023-10-08T05:52:44.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.371+0000 7fc36d7fa640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fc35006af90 0x7fc35006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.371+0000 7fc36d7fa640 1 -- 172.21.15.70:0/161757860 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fc360089a20 con 0x7fc37006c3a0 2023-10-08T05:52:44.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.371+0000 7fc367fff640 1 -- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fc35006af90 msgr2=0x7fc35006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:44.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.371+0000 7fc367fff640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fc35006af90 0x7fc35006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:44.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.379+0000 7fc36d7fa640 1 -- 172.21.15.70:0/161757860 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc3600546e0 con 0x7fc37006c3a0 2023-10-08T05:52:44.418 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:44.417+0000 7f48c922e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:52:44.533 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:44.533+0000 7fb441052040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:52:44.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.569+0000 7fc3766c9640 1 -- 172.21.15.70:0/161757860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc37015ac10 con 0x7fc37006c3a0 2023-10-08T05:52:44.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.570+0000 7fc36d7fa640 1 -- 172.21.15.70:0/161757860 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc360057d00 con 0x7fc37006c3a0 2023-10-08T05:52:44.571 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:52:44.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.572+0000 7fc367fff640 1 -- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fc35006af90 msgr2=0x7fc35006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:44.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.572+0000 7fc367fff640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fc35006af90 0x7fc35006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:44.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 -- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fc35006af90 msgr2=0x7fc35006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:44.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fc35006af90 0x7fc35006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 -- 172.21.15.70:0/161757860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006c3a0 msgr2=0x7fc3700c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:44.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006c3a0 0x7fc3700c4670 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fc36000c430 tx=0x7fc36000c8f0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 -- 172.21.15.70:0/161757860 shutdown_connections 2023-10-08T05:52:44.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fc35006af90 0x7fc35006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3700c9cf0 0x7fc3700c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc37006db60 0x7fc3700c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 --2- 172.21.15.70:0/161757860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc37006c3a0 0x7fc3700c4670 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 -- 172.21.15.70:0/161757860 >> 172.21.15.70:0/161757860 conn(0x7fc37006a5d0 msgr2=0x7fc37006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:44.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 -- 172.21.15.70:0/161757860 shutdown_connections 2023-10-08T05:52:44.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.573+0000 7fc3766c9640 1 -- 172.21.15.70:0/161757860 wait complete. 2023-10-08T05:52:44.577 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:44.576+0000 7f48c922e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:52:44.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:52:44.684 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:44.682+0000 7f48c922e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:52:44.700 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:44.700+0000 7fb441052040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:52:44.732 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:44.731+0000 7f48c922e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:52:44.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.789+0000 7f9abee7c640 1 Processor -- start 2023-10-08T05:52:44.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.789+0000 7f9abee7c640 1 -- start start 2023-10-08T05:52:44.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.791+0000 7f9abee7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9ab80cd470 0x7f9ab80cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.791+0000 7f9abee7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ab8160980 0x7f9ab8160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.791+0000 7f9abee7c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab80cde10 con 0x7f9ab815ce00 2023-10-08T05:52:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.791+0000 7f9abee7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab8161920 con 0x7f9ab80cd470 2023-10-08T05:52:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.791+0000 7f9abee7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab8161640 con 0x7f9ab8160980 2023-10-08T05:52:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.791+0000 7f9ab7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9ab80cd470 0x7f9ab80cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.791+0000 7f9ab7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9ab80cd470 0x7f9ab80cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46214/0 (socket says 172.21.15.70:46214) 2023-10-08T05:52:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.791+0000 7f9ab7fff640 1 -- 172.21.15.70:0/2560230505 learned_addr learned my addr 172.21.15.70:0/2560230505 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.791+0000 7f9abd419640 1 --2- 172.21.15.70:0/2560230505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ab8160980 0x7f9ab8160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:44.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.792+0000 7f9ab7fff640 1 -- 172.21.15.70:0/2560230505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ab8160980 msgr2=0x7f9ab8160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:44.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.792+0000 7f9ab7fff640 1 --2- 172.21.15.70:0/2560230505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ab8160980 0x7f9ab8160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.792+0000 7f9ab7fff640 1 -- 172.21.15.70:0/2560230505 >> v1:172.21.15.181:6789/0 conn(0x7f9ab815ce00 legacy=0x7f9ab80cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:44.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.792+0000 7f9ab7fff640 1 -- 172.21.15.70:0/2560230505 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab806b690 con 0x7f9ab80cd470 2023-10-08T05:52:44.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.793+0000 7f9ab7fff640 1 --2- 172.21.15.70:0/2560230505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab80cd470 0x7f9ab80cd8d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009870 tx=0x7f9aa402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d772b7647b04de09 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:44.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.793+0000 7f9ab77fe640 1 -- 172.21.15.70:0/2560230505 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40317c0 con 0x7f9ab80cd470 2023-10-08T05:52:44.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.793+0000 7f9ab77fe640 1 -- 172.21.15.70:0/2560230505 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa4031920 con 0x7f9ab80cd470 2023-10-08T05:52:44.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.793+0000 7f9ab77fe640 1 -- 172.21.15.70:0/2560230505 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4031cb0 con 0x7f9ab80cd470 2023-10-08T05:52:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.794+0000 7f9abee7c640 1 -- 172.21.15.70:0/2560230505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab80cd470 msgr2=0x7f9ab80cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.794+0000 7f9abee7c640 1 --2- 172.21.15.70:0/2560230505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab80cd470 0x7f9ab80cd8d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009870 tx=0x7f9aa402b860 comp rx=0 tx=0).stop 2023-10-08T05:52:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.794+0000 7f9abee7c640 1 -- 172.21.15.70:0/2560230505 shutdown_connections 2023-10-08T05:52:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.794+0000 7f9abee7c640 1 --2- 172.21.15.70:0/2560230505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ab8160980 0x7f9ab8160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.794+0000 7f9abee7c640 1 --2- 172.21.15.70:0/2560230505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab80cd470 0x7f9ab80cd8d0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.794+0000 7f9abee7c640 1 -- 172.21.15.70:0/2560230505 >> 172.21.15.70:0/2560230505 conn(0x7f9ab806e650 msgr2=0x7f9ab806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.794+0000 7f9abee7c640 1 -- 172.21.15.70:0/2560230505 shutdown_connections 2023-10-08T05:52:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.794+0000 7f9abee7c640 1 -- 172.21.15.70:0/2560230505 wait complete. 2023-10-08T05:52:44.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.796+0000 7f9abee7c640 1 Processor -- start 2023-10-08T05:52:44.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.796+0000 7f9abee7c640 1 -- start start 2023-10-08T05:52:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.797+0000 7f9abee7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab80cd470 0x7f9ab81550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.797+0000 7f9abee7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab815ce00 0x7f9ab80c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.797+0000 7f9abee7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ab8160980 0x7f9ab80c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.797+0000 7f9abee7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80b5e60 con 0x7f9ab815ce00 2023-10-08T05:52:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.797+0000 7f9abee7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80b6140 con 0x7f9ab80cd470 2023-10-08T05:52:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.797+0000 7f9abee7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80b6420 con 0x7f9ab8160980 2023-10-08T05:52:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.797+0000 7f9ab7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab815ce00 0x7f9ab80c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.797+0000 7f9ab7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab815ce00 0x7f9ab80c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46220/0 (socket says 172.21.15.70:46220) 2023-10-08T05:52:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.797+0000 7f9ab7fff640 1 -- 172.21.15.70:0/916488657 learned_addr learned my addr 172.21.15.70:0/916488657 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.798+0000 7f9abd419640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ab8160980 0x7f9ab80c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:44.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.798+0000 7f9abcc18640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab80cd470 0x7f9ab81550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.798+0000 7f9ab7fff640 1 -- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ab8160980 msgr2=0x7f9ab80c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.798+0000 7f9ab7fff640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ab8160980 0x7f9ab80c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.798+0000 7f9ab7fff640 1 -- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab80cd470 msgr2=0x7f9ab81550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.798+0000 7f9ab7fff640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab80cd470 0x7f9ab81550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.798+0000 7f9ab7fff640 1 -- 172.21.15.70:0/916488657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa40094d0 con 0x7f9ab815ce00 2023-10-08T05:52:44.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.798+0000 7f9ab7fff640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab815ce00 0x7f9ab80c8110 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4000c00 tx=0x7f9aa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:44.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.798+0000 7f9ab57fa640 1 -- 172.21.15.70:0/916488657 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4032860 con 0x7f9ab815ce00 2023-10-08T05:52:44.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.799+0000 7f9ab57fa640 1 -- 172.21.15.70:0/916488657 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa40329c0 con 0x7f9ab815ce00 2023-10-08T05:52:44.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.799+0000 7f9ab57fa640 1 -- 172.21.15.70:0/916488657 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4032d70 con 0x7f9ab815ce00 2023-10-08T05:52:44.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.799+0000 7f9abee7c640 1 -- 172.21.15.70:0/916488657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab80b6700 con 0x7f9ab815ce00 2023-10-08T05:52:44.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.799+0000 7f9abee7c640 1 -- 172.21.15.70:0/916488657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab8155610 con 0x7f9ab815ce00 2023-10-08T05:52:44.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.800+0000 7f9abee7c640 1 -- 172.21.15.70:0/916488657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab80b0740 con 0x7f9ab815ce00 2023-10-08T05:52:44.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.801+0000 7f9ab57fa640 1 -- 172.21.15.70:0/916488657 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f9aa404e450 con 0x7f9ab815ce00 2023-10-08T05:52:44.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.802+0000 7f9ab57fa640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f9a9c06af90 0x7f9a9c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:44.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.802+0000 7f9ab57fa640 1 -- 172.21.15.70:0/916488657 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f9aa40abd00 con 0x7f9ab815ce00 2023-10-08T05:52:44.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.802+0000 7f9abcc18640 1 -- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f9a9c06af90 msgr2=0x7f9a9c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:44.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.802+0000 7f9abcc18640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f9a9c06af90 0x7f9a9c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:44.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:44.806+0000 7f9ab57fa640 1 -- 172.21.15.70:0/916488657 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ab80b0740 con 0x7f9ab815ce00 2023-10-08T05:52:44.815 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:44.815+0000 7fb441052040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:52:44.840 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:44.839+0000 7f48c922e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:44.867 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:44.867+0000 7fb441052040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:52:44.946 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:44.945+0000 7f48c922e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:52:44.978 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:44.978+0000 7fb441052040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:45.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.003+0000 7f9abcc18640 1 -- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f9a9c06af90 msgr2=0x7f9a9c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:45.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.003+0000 7f9abcc18640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f9a9c06af90 0x7f9a9c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:45.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.007+0000 7f9abee7c640 1 -- 172.21.15.70:0/916488657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ab80b0740 con 0x7f9ab815ce00 2023-10-08T05:52:45.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.008+0000 7f9ab57fa640 1 -- 172.21.15.70:0/916488657 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v91) v1 ==== 80+0+477 (secure 0 0 0) 0x7f9ab80b0740 con 0x7f9ab815ce00 2023-10-08T05:52:45.008 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/814792959 2023-10-09T05:52:40.392573+0000 2023-10-08T05:52:45.008 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2758467223 2023-10-09T05:52:40.392573+0000 2023-10-08T05:52:45.008 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1699086184 2023-10-09T05:52:40.392573+0000 2023-10-08T05:52:45.008 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1378633292 2023-10-09T05:52:40.392573+0000 2023-10-08T05:52:45.008 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1217040450 2023-10-09T05:52:40.392573+0000 2023-10-08T05:52:45.009 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2333154107 2023-10-09T05:52:40.392573+0000 2023-10-08T05:52:45.009 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/3994892546 2023-10-09T05:52:40.392573+0000 2023-10-08T05:52:45.009 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/3994892546 2023-10-09T05:52:40.392573+0000 2023-10-08T05:52:45.009 INFO:teuthology.orchestra.run.smithi070.stderr:listed 8 entries 2023-10-08T05:52:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 -- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f9a9c06af90 msgr2=0x7f9a9c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f9a9c06af90 0x7f9a9c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 -- 172.21.15.70:0/916488657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab815ce00 msgr2=0x7f9ab80c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab815ce00 0x7f9ab80c8110 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4000c00 tx=0x7f9aa40046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 -- 172.21.15.70:0/916488657 shutdown_connections 2023-10-08T05:52:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f9a9c06af90 0x7f9a9c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ab8160980 0x7f9ab80c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ab815ce00 0x7f9ab80c8110 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 --2- 172.21.15.70:0/916488657 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9ab80cd470 0x7f9ab81550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 -- 172.21.15.70:0/916488657 >> 172.21.15.70:0/916488657 conn(0x7f9ab806e650 msgr2=0x7f9ab80d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:45.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 -- 172.21.15.70:0/916488657 shutdown_connections 2023-10-08T05:52:45.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.010+0000 7f9abee7c640 1 -- 172.21.15.70:0/916488657 wait complete. 2023-10-08T05:52:45.028 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T05:52:45.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:45.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T05:52:45.093 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:45.093+0000 7fb441052040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:52:45.100 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:45.099+0000 7f48c922e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:45.147 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:52:45.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:52:45.147 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T05:52:45.151 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:45.150+0000 7f48c922e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:45.196 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:52:45.197 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:45.252 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:52:45.252 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:45.252 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:45.263 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:45.263+0000 7fb441052040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:52:45.273 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:45.272+0000 7f48c922e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:52:45.305 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:52:45.305 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:52:45.305 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:45.315 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:45.315+0000 7fb441052040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:52:45.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:52:45.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:45.443 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:45.443+0000 7fb441052040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:52:45.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.575+0000 7f62c457c640 1 Processor -- start 2023-10-08T05:52:45.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.575+0000 7f62c457c640 1 -- start start 2023-10-08T05:52:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.576+0000 7f62c457c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f62bc15ab40 0x7f62bc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.577+0000 7f62c457c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62bc15b2b0 con 0x7f62bc0c90b0 2023-10-08T05:52:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.577+0000 7f62c457c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62bc15b590 con 0x7f62bc0cb580 2023-10-08T05:52:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.577+0000 7f62c457c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62bc15bf30 con 0x7f62bc15ab40 2023-10-08T05:52:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.577+0000 7f62c1b17640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f62bc0c90b0 0x7f62bc0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58484/0 (socket says 172.21.15.70:58484) 2023-10-08T05:52:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.577+0000 7f62c1b17640 1 -- 172.21.15.70:0/977653973 learned_addr learned my addr 172.21.15.70:0/977653973 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.577+0000 7f62c2318640 1 --2- 172.21.15.70:0/977653973 >> v2:172.21.15.70:3300/0 conn(0x7f62bc15ab40 0x7f62bc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.577+0000 7f62c2318640 1 -- 172.21.15.70:0/977653973 >> v1:172.21.15.181:6789/0 conn(0x7f62bc0cb580 legacy=0x7f62bc15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.577+0000 7f62c2318640 1 -- 172.21.15.70:0/977653973 >> v1:172.21.15.70:6789/0 conn(0x7f62bc0c90b0 legacy=0x7f62bc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.577+0000 7f62c2318640 1 -- 172.21.15.70:0/977653973 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc15b870 con 0x7f62bc15ab40 2023-10-08T05:52:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.577+0000 7f62c1316640 1 -- 172.21.15.70:0/977653973 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324852604 0 0) 0x7f62bc15b2b0 con 0x7f62bc0c90b0 2023-10-08T05:52:45.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.579+0000 7f62c2318640 1 --2- 172.21.15.70:0/977653973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc15ab40 0x7f62bc0ad720 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f62b80097a0 tx=0x7f62b802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a7696e2b5b221b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:45.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.579+0000 7f62c1316640 1 -- 172.21.15.70:0/977653973 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8030610 con 0x7f62bc15ab40 2023-10-08T05:52:45.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.579+0000 7f62c1316640 1 -- 172.21.15.70:0/977653973 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b8030770 con 0x7f62bc15ab40 2023-10-08T05:52:45.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.579+0000 7f62c1316640 1 -- 172.21.15.70:0/977653973 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b8030b00 con 0x7f62bc15ab40 2023-10-08T05:52:45.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.580+0000 7f62c457c640 1 -- 172.21.15.70:0/977653973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc15ab40 msgr2=0x7f62bc0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:45.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.580+0000 7f62c457c640 1 --2- 172.21.15.70:0/977653973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc15ab40 0x7f62bc0ad720 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f62b80097a0 tx=0x7f62b802b960 comp rx=0 tx=0).stop 2023-10-08T05:52:45.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.580+0000 7f62c457c640 1 -- 172.21.15.70:0/977653973 shutdown_connections 2023-10-08T05:52:45.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.580+0000 7f62c457c640 1 --2- 172.21.15.70:0/977653973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc15ab40 0x7f62bc0ad720 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.580+0000 7f62c457c640 1 -- 172.21.15.70:0/977653973 >> 172.21.15.70:0/977653973 conn(0x7f62bc06a380 msgr2=0x7f62bc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:45.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.580+0000 7f62c457c640 1 -- 172.21.15.70:0/977653973 shutdown_connections 2023-10-08T05:52:45.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.580+0000 7f62c457c640 1 -- 172.21.15.70:0/977653973 wait complete. 2023-10-08T05:52:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.582+0000 7f62c457c640 1 Processor -- start 2023-10-08T05:52:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.582+0000 7f62c457c640 1 -- start start 2023-10-08T05:52:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c457c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62bc0c90b0 0x7f62bc14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c457c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc0cb580 0x7f62bc0b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c457c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc15ab40 0x7f62bc0c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c457c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc14f3a0 con 0x7f62bc0cb580 2023-10-08T05:52:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c457c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc151af0 con 0x7f62bc15ab40 2023-10-08T05:52:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c457c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62bc151dd0 con 0x7f62bc0c90b0 2023-10-08T05:52:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c1b17640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc0cb580 0x7f62bc0b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:45.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c1b17640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc0cb580 0x7f62bc0b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46238/0 (socket says 172.21.15.70:46238) 2023-10-08T05:52:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c1b17640 1 -- 172.21.15.70:0/2044964440 learned_addr learned my addr 172.21.15.70:0/2044964440 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c2b19640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc15ab40 0x7f62bc0c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.583+0000 7f62c2318640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62bc0c90b0 0x7f62bc14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62c1b17640 1 -- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62bc0c90b0 msgr2=0x7f62bc14ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:45.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62c1b17640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62bc0c90b0 0x7f62bc14ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62c1b17640 1 -- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc15ab40 msgr2=0x7f62bc0c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62c1b17640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc15ab40 0x7f62bc0c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62c1b17640 1 -- 172.21.15.70:0/2044964440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b8009400 con 0x7f62bc0cb580 2023-10-08T05:52:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62c1b17640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc0cb580 0x7f62bc0b13d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00c460 tx=0x7f62ac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62b2ffd640 1 -- 172.21.15.70:0/2044964440 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0074e0 con 0x7f62bc0cb580 2023-10-08T05:52:45.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62b2ffd640 1 -- 172.21.15.70:0/2044964440 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac007680 con 0x7f62bc0cb580 2023-10-08T05:52:45.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62b2ffd640 1 -- 172.21.15.70:0/2044964440 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac004a00 con 0x7f62bc0cb580 2023-10-08T05:52:45.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62c457c640 1 -- 172.21.15.70:0/2044964440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62bc1520b0 con 0x7f62bc0cb580 2023-10-08T05:52:45.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.584+0000 7f62c457c640 1 -- 172.21.15.70:0/2044964440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62bc152670 con 0x7f62bc0cb580 2023-10-08T05:52:45.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.585+0000 7f62c457c640 1 -- 172.21.15.70:0/2044964440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62bc152860 con 0x7f62bc0cb580 2023-10-08T05:52:45.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.586+0000 7f62b2ffd640 1 -- 172.21.15.70:0/2044964440 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f62ac004b60 con 0x7f62bc0cb580 2023-10-08T05:52:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.587+0000 7f62b2ffd640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f62a006af90 0x7f62a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.587+0000 7f62b2ffd640 1 -- 172.21.15.70:0/2044964440 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f62ac088ae0 con 0x7f62bc0cb580 2023-10-08T05:52:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.588+0000 7f62c2318640 1 -- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f62a006af90 msgr2=0x7f62a006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:45.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.588+0000 7f62c2318640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f62a006af90 0x7f62a006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:45.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.595+0000 7f62b2ffd640 1 -- 172.21.15.70:0/2044964440 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62ac0538f0 con 0x7f62bc0cb580 2023-10-08T05:52:45.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.789+0000 7f62c2318640 1 -- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f62a006af90 msgr2=0x7f62a006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:45.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.789+0000 7f62c2318640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f62a006af90 0x7f62a006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:52:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.822+0000 7f62c457c640 1 -- 172.21.15.70:0/2044964440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62bc067490 con 0x7f62bc0cb580 2023-10-08T05:52:45.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.823+0000 7f62b2ffd640 1 -- 172.21.15.70:0/2044964440 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f62ac018020 con 0x7f62bc0cb580 2023-10-08T05:52:45.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:45.824 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1038137327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1038137327},{"type":"v1","addr":"172.21.15.181:6835","nonce":1038137327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6660,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1967783419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1967783419},{"type":"v1","addr":"172.21.15.70:6833","nonce":1967783419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6663,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1304760341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1304760341},{"type":"v1","addr":"172.21.15.70:6835","nonce":1304760341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:52:26.053790+0000","modified":"2023-10-08T05:52:29.103940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6675},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6675":{"gid":6675,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2348955257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2348955257},{"type":"v1","addr":"172.21.15.181:6837","nonce":2348955257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-08T05:52:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 33 2023-10-08T05:52:45.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 -- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f62a006af90 msgr2=0x7f62a006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f62a006af90 0x7f62a006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 -- 172.21.15.70:0/2044964440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc0cb580 msgr2=0x7f62bc0b13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc0cb580 0x7f62bc0b13d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00c460 tx=0x7f62ac00c920 comp rx=0 tx=0).stop 2023-10-08T05:52:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 -- 172.21.15.70:0/2044964440 shutdown_connections 2023-10-08T05:52:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f62a006af90 0x7f62a006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62bc15ab40 0x7f62bc0c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62bc0cb580 0x7f62bc0b13d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 --2- 172.21.15.70:0/2044964440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62bc0c90b0 0x7f62bc14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 -- 172.21.15.70:0/2044964440 >> 172.21.15.70:0/2044964440 conn(0x7f62bc06a380 msgr2=0x7f62bc06da80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 -- 172.21.15.70:0/2044964440 shutdown_connections 2023-10-08T05:52:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:45.826+0000 7f62c457c640 1 -- 172.21.15.70:0/2044964440 wait complete. 2023-10-08T05:52:45.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:46.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.035+0000 7f6762be7640 1 Processor -- start 2023-10-08T05:52:46.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.035+0000 7f6762be7640 1 -- start start 2023-10-08T05:52:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.036+0000 7f6762be7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f675c0cb880 0x7f675c0b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.036+0000 7f6762be7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f675c0cbce0 0x7f675c0b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.036+0000 7f6762be7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675c0ce550 con 0x7f675c069d50 2023-10-08T05:52:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.036+0000 7f6762be7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675c1609d0 con 0x7f675c0cbce0 2023-10-08T05:52:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.036+0000 7f6762be7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675c0cdcf0 con 0x7f675c0cb880 2023-10-08T05:52:46.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.037+0000 7f6761184640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f675c0cbce0 0x7f675c0b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.037+0000 7f6760983640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f675c069d50 0x7f675c06a130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:43662/0 (socket says 172.21.15.70:43662) 2023-10-08T05:52:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.037+0000 7f6761184640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f675c0cbce0 0x7f675c0b1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:56622/0 (socket says 172.21.15.70:56622) 2023-10-08T05:52:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.037+0000 7f6760983640 1 -- 172.21.15.70:0/204023000 learned_addr learned my addr 172.21.15.70:0/204023000 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.037+0000 7f675bfff640 1 --2- 172.21.15.70:0/204023000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f675c0cb880 0x7f675c0b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.037+0000 7f6761184640 1 -- 172.21.15.70:0/204023000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f675c0cb880 msgr2=0x7f675c0b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.037+0000 7f6761184640 1 --2- 172.21.15.70:0/204023000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f675c0cb880 0x7f675c0b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.037+0000 7f6761184640 1 -- 172.21.15.70:0/204023000 >> v1:172.21.15.181:6789/0 conn(0x7f675c069d50 legacy=0x7f675c06a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.037+0000 7f6761184640 1 -- 172.21.15.70:0/204023000 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c0cdf90 con 0x7f675c0cbce0 2023-10-08T05:52:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.038+0000 7f6761184640 1 --2- 172.21.15.70:0/204023000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f675c0cbce0 0x7f675c0b1850 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6754009d90 tx=0x7f675402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2928c413c262c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.039+0000 7f675b7fe640 1 -- 172.21.15.70:0/204023000 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675402cc50 con 0x7f675c0cbce0 2023-10-08T05:52:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.039+0000 7f675b7fe640 1 -- 172.21.15.70:0/204023000 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675402cdb0 con 0x7f675c0cbce0 2023-10-08T05:52:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.039+0000 7f675b7fe640 1 -- 172.21.15.70:0/204023000 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754034660 con 0x7f675c0cbce0 2023-10-08T05:52:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.040+0000 7f6762be7640 1 -- 172.21.15.70:0/204023000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f675c0cbce0 msgr2=0x7f675c0b1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.040+0000 7f6762be7640 1 --2- 172.21.15.70:0/204023000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f675c0cbce0 0x7f675c0b1850 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6754009d90 tx=0x7f675402bea0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.040+0000 7f6762be7640 1 -- 172.21.15.70:0/204023000 shutdown_connections 2023-10-08T05:52:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.040+0000 7f6762be7640 1 --2- 172.21.15.70:0/204023000 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f675c0cbce0 0x7f675c0b1850 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.040+0000 7f6762be7640 1 --2- 172.21.15.70:0/204023000 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f675c0cb880 0x7f675c0b1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.040+0000 7f6762be7640 1 -- 172.21.15.70:0/204023000 >> 172.21.15.70:0/204023000 conn(0x7f675c06d450 msgr2=0x7f675c06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.040+0000 7f6762be7640 1 -- 172.21.15.70:0/204023000 shutdown_connections 2023-10-08T05:52:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.040+0000 7f6762be7640 1 -- 172.21.15.70:0/204023000 wait complete. 2023-10-08T05:52:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.041+0000 7f6762be7640 1 Processor -- start 2023-10-08T05:52:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.042+0000 7f6762be7640 1 -- start start 2023-10-08T05:52:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f6762be7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f675c069d50 0x7f675c0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f6762be7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f675c0cb880 0x7f675c14a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f6762be7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f675c0cbce0 0x7f675c0b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f6762be7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c0ae060 con 0x7f675c0cb880 2023-10-08T05:52:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f6762be7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c14a7d0 con 0x7f675c0cbce0 2023-10-08T05:52:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f6762be7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675c0ad3e0 con 0x7f675c069d50 2023-10-08T05:52:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f675bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f675c0cb880 0x7f675c14a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f675bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f675c0cb880 0x7f675c14a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46242/0 (socket says 172.21.15.70:46242) 2023-10-08T05:52:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f675bfff640 1 -- 172.21.15.70:0/2445955142 learned_addr learned my addr 172.21.15.70:0/2445955142 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f6761184640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f675c0cbce0 0x7f675c0b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f6760983640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f675c069d50 0x7f675c0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f675bfff640 1 -- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f675c069d50 msgr2=0x7f675c0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f675bfff640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f675c069d50 0x7f675c0ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f675bfff640 1 -- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f675c0cbce0 msgr2=0x7f675c0b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f675bfff640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f675c0cbce0 0x7f675c0b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f675bfff640 1 -- 172.21.15.70:0/2445955142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754009a70 con 0x7f675c0cb880 2023-10-08T05:52:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.043+0000 7f6761184640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f675c0cbce0 0x7f675c0b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:52:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.044+0000 7f675bfff640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f675c0cb880 0x7f675c14a290 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f674800e9b0 tx=0x7f674800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.044+0000 7f67597fa640 1 -- 172.21.15.70:0/2445955142 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748009670 con 0x7f675c0cb880 2023-10-08T05:52:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.044+0000 7f67597fa640 1 -- 172.21.15.70:0/2445955142 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674800c9a0 con 0x7f675c0cb880 2023-10-08T05:52:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.044+0000 7f6762be7640 1 -- 172.21.15.70:0/2445955142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c0ad6c0 con 0x7f675c0cb880 2023-10-08T05:52:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.044+0000 7f67597fa640 1 -- 172.21.15.70:0/2445955142 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674800cd50 con 0x7f675c0cb880 2023-10-08T05:52:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.044+0000 7f6762be7640 1 -- 172.21.15.70:0/2445955142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c0adc80 con 0x7f675c0cb880 2023-10-08T05:52:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.045+0000 7f6762be7640 1 -- 172.21.15.70:0/2445955142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675c0ae9c0 con 0x7f675c0cb880 2023-10-08T05:52:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.046+0000 7f67597fa640 1 -- 172.21.15.70:0/2445955142 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f6748013470 con 0x7f675c0cb880 2023-10-08T05:52:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.048+0000 7f67597fa640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f674006af90 0x7f674006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.048+0000 7f67597fa640 1 -- 172.21.15.70:0/2445955142 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f6748089d50 con 0x7f675c0cb880 2023-10-08T05:52:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.048+0000 7f6760983640 1 -- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f674006af90 msgr2=0x7f674006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.048+0000 7f6760983640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f674006af90 0x7f674006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:46.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.051+0000 7f67597fa640 1 -- 172.21.15.70:0/2445955142 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6748054980 con 0x7f675c0cb880 2023-10-08T05:52:46.097 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.094+0000 7f48c922e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:52:46.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.242+0000 7f6762be7640 1 -- 172.21.15.70:0/2445955142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f675c0ae1a0 con 0x7f675c0cb880 2023-10-08T05:52:46.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.244+0000 7f67597fa640 1 -- 172.21.15.70:0/2445955142 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+17282 (secure 0 0 0) 0x7f675c0ae1a0 con 0x7f675c0cb880 2023-10-08T05:52:46.245 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:46.245 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:40.392647+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:52:23.846215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-08T05:52:25.069536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/814792959":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/2758467223":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1699086184":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1378633292":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1217040450":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/2333154107":"2023-10-09T05:52:40.392573+0000","172.21.15.181:6833/3994892546":"2023-10-09T05:52:40.392573+0000","172.21.15.181:6832/3994892546":"2023-10-09T05:52:40.392573+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:46.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 -- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f674006af90 msgr2=0x7f674006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:46.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f674006af90 0x7f674006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 -- 172.21.15.70:0/2445955142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f675c0cb880 msgr2=0x7f675c14a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f675c0cb880 0x7f675c14a290 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f674800e9b0 tx=0x7f674800ee70 comp rx=0 tx=0).stop 2023-10-08T05:52:46.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 -- 172.21.15.70:0/2445955142 shutdown_connections 2023-10-08T05:52:46.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7f674006af90 0x7f674006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f675c0cbce0 0x7f675c0b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f675c0cb880 0x7f675c14a290 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 --2- 172.21.15.70:0/2445955142 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f675c069d50 0x7f675c0ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 -- 172.21.15.70:0/2445955142 >> 172.21.15.70:0/2445955142 conn(0x7f675c06d450 msgr2=0x7f675c0c8570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:46.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 -- 172.21.15.70:0/2445955142 shutdown_connections 2023-10-08T05:52:46.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.247+0000 7f6762be7640 1 -- 172.21.15.70:0/2445955142 wait complete. 2023-10-08T05:52:46.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:52:23.846215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:52:25.069536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:52:46.265 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:52:46.273 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:46.273+0000 7fb441052040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:52:46.390 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.389+0000 7f48c922e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:52:46.447 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.446+0000 7f48c922e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:52:46.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.455+0000 7fa5aad4e640 1 Processor -- start 2023-10-08T05:52:46.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.455+0000 7fa5aad4e640 1 -- start start 2023-10-08T05:52:46.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.457+0000 7fa5aad4e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5a40d01c0 0x7fa5a40cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.457+0000 7fa5aad4e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a40714a0 con 0x7fa5a415c890 2023-10-08T05:52:46.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.457+0000 7fa5aad4e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a4071780 con 0x7fa5a415cc70 2023-10-08T05:52:46.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.457+0000 7fa5aad4e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a4160590 con 0x7fa5a40d01c0 2023-10-08T05:52:46.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.457+0000 7fa5a954b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5a40d01c0 0x7fa5a40cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.457+0000 7fa5a9d4c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa5a415c890 0x7fa5a40cfab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58488/0 (socket says 172.21.15.70:58488) 2023-10-08T05:52:46.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.457+0000 7fa5a9d4c640 1 -- 172.21.15.70:0/1144059956 learned_addr learned my addr 172.21.15.70:0/1144059956 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:46.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.458+0000 7fa5a954b640 1 -- 172.21.15.70:0/1144059956 >> v1:172.21.15.181:6789/0 conn(0x7fa5a415cc70 legacy=0x7fa5a41609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.458+0000 7fa5a954b640 1 -- 172.21.15.70:0/1144059956 >> v1:172.21.15.70:6789/0 conn(0x7fa5a415c890 legacy=0x7fa5a40cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.458+0000 7fa5a954b640 1 -- 172.21.15.70:0/1144059956 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a4071a60 con 0x7fa5a40d01c0 2023-10-08T05:52:46.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.458+0000 7fa5a8d4a640 1 -- 172.21.15.70:0/1144059956 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528595084 0 0) 0x7fa5a40714a0 con 0x7fa5a415c890 2023-10-08T05:52:46.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.460+0000 7fa5a954b640 1 --2- 172.21.15.70:0/1144059956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 0x7fa5a40cca50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa5940097a0 tx=0x7fa59402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663a6d1af2ba0be9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:46.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.460+0000 7fa5a8d4a640 1 -- 172.21.15.70:0/1144059956 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594031830 con 0x7fa5a40d01c0 2023-10-08T05:52:46.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.460+0000 7fa5a8d4a640 1 -- 172.21.15.70:0/1144059956 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa594031990 con 0x7fa5a40d01c0 2023-10-08T05:52:46.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.460+0000 7fa5a8d4a640 1 -- 172.21.15.70:0/1144059956 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594031d20 con 0x7fa5a40d01c0 2023-10-08T05:52:46.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.460+0000 7fa5aad4e640 1 -- 172.21.15.70:0/1144059956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 msgr2=0x7fa5a40cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.460+0000 7fa5aad4e640 1 --2- 172.21.15.70:0/1144059956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 0x7fa5a40cca50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa5940097a0 tx=0x7fa59402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.461+0000 7fa5aad4e640 1 -- 172.21.15.70:0/1144059956 shutdown_connections 2023-10-08T05:52:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.461+0000 7fa5aad4e640 1 --2- 172.21.15.70:0/1144059956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 0x7fa5a40cca50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.461+0000 7fa5aad4e640 1 -- 172.21.15.70:0/1144059956 >> 172.21.15.70:0/1144059956 conn(0x7fa5a406b250 msgr2=0x7fa5a406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.461+0000 7fa5aad4e640 1 -- 172.21.15.70:0/1144059956 shutdown_connections 2023-10-08T05:52:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.461+0000 7fa5aad4e640 1 -- 172.21.15.70:0/1144059956 wait complete. 2023-10-08T05:52:46.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.463+0000 7fa5aad4e640 1 Processor -- start 2023-10-08T05:52:46.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.463+0000 7fa5aad4e640 1 -- start start 2023-10-08T05:52:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.465+0000 7fa5aad4e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 0x7fa5a40c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.465+0000 7fa5aad4e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5a415c890 0x7fa5a4158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.465+0000 7fa5aad4e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a415cc70 0x7fa5a406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.465+0000 7fa5aad4e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40b2fe0 con 0x7fa5a40d01c0 2023-10-08T05:52:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.465+0000 7fa5aad4e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40c6020 con 0x7fa5a415cc70 2023-10-08T05:52:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.465+0000 7fa5aad4e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40b23d0 con 0x7fa5a415c890 2023-10-08T05:52:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.465+0000 7fa5a9d4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 0x7fa5a40c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.465+0000 7fa5a9d4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 0x7fa5a40c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46256/0 (socket says 172.21.15.70:46256) 2023-10-08T05:52:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.465+0000 7fa5a9d4c640 1 -- 172.21.15.70:0/2951884390 learned_addr learned my addr 172.21.15.70:0/2951884390 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.465+0000 7fa5aa54d640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a415cc70 0x7fa5a406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa5a954b640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5a415c890 0x7fa5a4158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa5a9d4c640 1 -- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5a415c890 msgr2=0x7fa5a4158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa5a9d4c640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5a415c890 0x7fa5a4158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa5a9d4c640 1 -- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a415cc70 msgr2=0x7fa5a406d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa5a9d4c640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a415cc70 0x7fa5a406d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa5a9d4c640 1 -- 172.21.15.70:0/2951884390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa594009400 con 0x7fa5a40d01c0 2023-10-08T05:52:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa5aa54d640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a415cc70 0x7fa5a406d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa5a9d4c640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 0x7fa5a40c5ae0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000c4a0 tx=0x7fa5a000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa59a7fc640 1 -- 172.21.15.70:0/2951884390 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00074e0 con 0x7fa5a40d01c0 2023-10-08T05:52:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa59a7fc640 1 -- 172.21.15.70:0/2951884390 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0007640 con 0x7fa5a40d01c0 2023-10-08T05:52:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa5aad4e640 1 -- 172.21.15.70:0/2951884390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a40b26b0 con 0x7fa5a40d01c0 2023-10-08T05:52:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa59a7fc640 1 -- 172.21.15.70:0/2951884390 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0004a00 con 0x7fa5a40d01c0 2023-10-08T05:52:46.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.466+0000 7fa5aad4e640 1 -- 172.21.15.70:0/2951884390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a40b2c70 con 0x7fa5a40d01c0 2023-10-08T05:52:46.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.467+0000 7fa5aad4e640 1 -- 172.21.15.70:0/2951884390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a4074150 con 0x7fa5a40d01c0 2023-10-08T05:52:46.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.468+0000 7fa59a7fc640 1 -- 172.21.15.70:0/2951884390 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 34) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fa5a0004b60 con 0x7fa5a40d01c0 2023-10-08T05:52:46.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.470+0000 7fa59a7fc640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fa58806af90 0x7fa58806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.470+0000 7fa59a7fc640 1 -- 172.21.15.70:0/2951884390 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fa5a0088cb0 con 0x7fa5a40d01c0 2023-10-08T05:52:46.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.470+0000 7fa5a954b640 1 -- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fa58806af90 msgr2=0x7fa58806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1709912921 2023-10-08T05:52:46.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.470+0000 7fa5a954b640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fa58806af90 0x7fa58806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:52:46.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.473+0000 7fa59a7fc640 1 -- 172.21.15.70:0/2951884390 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5a00538e0 con 0x7fa5a40d01c0 2023-10-08T05:52:46.562 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.561+0000 7f48c922e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:52:46.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.583+0000 7fa59a7fc640 1 -- 172.21.15.70:0/2951884390 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 35) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa5a0053cb0 con 0x7fa5a40d01c0 2023-10-08T05:52:46.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.584+0000 7fa59a7fc640 1 -- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fa58806af90 msgr2=0x7fa58806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:52:46.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.584+0000 7fa59a7fc640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fa58806af90 0x7fa58806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.598 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:46.598+0000 7fb441052040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:52:46.653 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.652+0000 7f484dd3a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:46.654 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.652+0000 7f484dd3a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:46.654 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.652+0000 7f484dd3a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:46.654 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.652+0000 7f484dd3a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:46.654 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.652+0000 7f484dd3a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:46.655 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.653+0000 7f484ad34640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:46.655 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.653+0000 7f484ad34640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:46.655 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.653+0000 7f484ad34640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:46.655 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.653+0000 7f484ad34640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:46.655 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:46.653+0000 7f484ad34640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:52:46.661 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:46.661+0000 7fb441052040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:52:46.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.668+0000 7fa5aad4e640 1 -- 172.21.15.70:0/2951884390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5a415f4a0 con 0x7fa5a40d01c0 2023-10-08T05:52:46.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.669+0000 7fa59a7fc640 1 -- 172.21.15.70:0/2951884390 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5a00728d0 con 0x7fa5a40d01c0 2023-10-08T05:52:46.669 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:52:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.671+0000 7fa5aad4e640 1 -- 172.21.15.70:0/2951884390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 msgr2=0x7fa5a40c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.671+0000 7fa5aad4e640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 0x7fa5a40c5ae0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000c4a0 tx=0x7fa5a000c960 comp rx=0 tx=0).stop 2023-10-08T05:52:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.672+0000 7fa5aad4e640 1 -- 172.21.15.70:0/2951884390 shutdown_connections 2023-10-08T05:52:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.672+0000 7fa5aad4e640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:6832/1709912921,v1:172.21.15.181:6833/1709912921] conn(0x7fa58806af90 0x7fa58806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.672+0000 7fa5aad4e640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5a415cc70 0x7fa5a406d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.672+0000 7fa5aad4e640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5a415c890 0x7fa5a4158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.672+0000 7fa5aad4e640 1 --2- 172.21.15.70:0/2951884390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5a40d01c0 0x7fa5a40c5ae0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.672+0000 7fa5aad4e640 1 -- 172.21.15.70:0/2951884390 >> 172.21.15.70:0/2951884390 conn(0x7fa5a406b250 msgr2=0x7fa5a40ca360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:46.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.672+0000 7fa5aad4e640 1 -- 172.21.15.70:0/2951884390 shutdown_connections 2023-10-08T05:52:46.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.672+0000 7fa5aad4e640 1 -- 172.21.15.70:0/2951884390 wait complete. 2023-10-08T05:52:46.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:52:46.780 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:52:46.779+0000 7fb441052040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:52:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.941+0000 7fa162e9b640 1 Processor -- start 2023-10-08T05:52:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.941+0000 7fa162e9b640 1 -- start start 2023-10-08T05:52:46.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.942+0000 7fa162e9b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c15ab80 0x7fa15c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.942+0000 7fa162e9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa15c15af60 0x7fa15c0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.942+0000 7fa162e9b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15c0c9840 con 0x7fa15c06e960 2023-10-08T05:52:46.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.942+0000 7fa162e9b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15c0ca430 con 0x7fa15c15af60 2023-10-08T05:52:46.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.942+0000 7fa162e9b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c0c9a90 con 0x7fa15c15ab80 2023-10-08T05:52:46.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.942+0000 7fa161438640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa15c06e960 0x7fa15c06edd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58492/0 (socket says 172.21.15.70:58492) 2023-10-08T05:52:46.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.942+0000 7fa161438640 1 -- 172.21.15.70:0/745081315 learned_addr learned my addr 172.21.15.70:0/745081315 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:46.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa160c37640 1 --2- 172.21.15.70:0/745081315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c15ab80 0x7fa15c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15bfff640 1 --2- 172.21.15.70:0/745081315 >> v2:172.21.15.181:3300/0 conn(0x7fa15c15af60 0x7fa15c0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261312369 0 0) 0x7fa15c0c9840 con 0x7fa15c06e960 2023-10-08T05:52:46.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa148003520 con 0x7fa15c06e960 2023-10-08T05:52:46.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3783559480 0 0) 0x7fa148003520 con 0x7fa15c06e960 2023-10-08T05:52:46.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c15ab80 msgr2=0x7fa15c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:46.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 --2- 172.21.15.70:0/745081315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c15ab80 0x7fa15c06b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 >> v2:172.21.15.181:3300/0 conn(0x7fa15c15af60 msgr2=0x7fa15c0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 --2- 172.21.15.70:0/745081315 >> v2:172.21.15.181:3300/0 conn(0x7fa15c15af60 0x7fa15c0b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c0c9d70 con 0x7fa15c06e960 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa154001670 con 0x7fa15c06e960 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 >> v1:172.21.15.70:6789/0 conn(0x7fa15c06e960 legacy=0x7fa15c06edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 --2- 172.21.15.70:0/745081315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa148003970 0x7fa148003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 --2- 172.21.15.70:0/745081315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c15ab80 0x7fa1480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 --2- 172.21.15.70:0/745081315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa148004d00 0x7fa148005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c0ca430 con 0x7fa148003970 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c0c9a90 con 0x7fa148004d00 2023-10-08T05:52:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.943+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14800b980 con 0x7fa15c15ab80 2023-10-08T05:52:46.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.944+0000 7fa160c37640 1 --2- 172.21.15.70:0/745081315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa148003970 0x7fa148003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.944+0000 7fa160c37640 1 -- 172.21.15.70:0/745081315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c15ab80 msgr2=0x7fa1480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:46.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.944+0000 7fa160c37640 1 --2- 172.21.15.70:0/745081315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c15ab80 0x7fa1480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.944+0000 7fa160c37640 1 -- 172.21.15.70:0/745081315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa148004d00 msgr2=0x7fa148005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:46.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.944+0000 7fa160c37640 1 --2- 172.21.15.70:0/745081315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa148004d00 0x7fa148005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.944+0000 7fa160c37640 1 -- 172.21.15.70:0/745081315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa148003520 con 0x7fa148003970 2023-10-08T05:52:46.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.945+0000 7fa160c37640 1 --2- 172.21.15.70:0/745081315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa148003970 0x7fa148003d90 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009fc0 tx=0x7fa1480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbe295cd425fdc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:46.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.945+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c02cc30 con 0x7fa148003970 2023-10-08T05:52:46.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.945+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c00f040 con 0x7fa148003970 2023-10-08T05:52:46.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.945+0000 7fa15b7fe640 1 -- 172.21.15.70:0/745081315 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c034440 con 0x7fa148003970 2023-10-08T05:52:46.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.946+0000 7fa162e9b640 1 -- 172.21.15.70:0/745081315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa148003970 msgr2=0x7fa148003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.946+0000 7fa162e9b640 1 --2- 172.21.15.70:0/745081315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa148003970 0x7fa148003d90 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009fc0 tx=0x7fa1480057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.946+0000 7fa161438640 1 -- 172.21.15.70:0/745081315 reap_dead start 2023-10-08T05:52:46.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.946+0000 7fa162e9b640 1 -- 172.21.15.70:0/745081315 shutdown_connections 2023-10-08T05:52:46.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.946+0000 7fa162e9b640 1 -- 172.21.15.70:0/745081315 >> 172.21.15.70:0/745081315 conn(0x7fa15c0cba60 msgr2=0x7fa15c0cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:46.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.946+0000 7fa162e9b640 1 -- 172.21.15.70:0/745081315 shutdown_connections 2023-10-08T05:52:46.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.946+0000 7fa162e9b640 1 -- 172.21.15.70:0/745081315 wait complete. 2023-10-08T05:52:46.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.948+0000 7fa162e9b640 1 Processor -- start 2023-10-08T05:52:46.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.948+0000 7fa162e9b640 1 -- start start 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa162e9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa148003970 0x7fa15c1f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa162e9b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c14cdd0 0x7fa15c14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa162e9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa15c1f1e00 0x7fa15c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa162e9b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c0c6400 con 0x7fa15c1f1e00 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa162e9b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c14b2c0 con 0x7fa148003970 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa162e9b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c14d380 con 0x7fa15c14cdd0 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa161438640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa15c1f1e00 0x7fa15c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa161438640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa15c1f1e00 0x7fa15c069d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46286/0 (socket says 172.21.15.70:46286) 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa161438640 1 -- 172.21.15.70:0/2800420542 learned_addr learned my addr 172.21.15.70:0/2800420542 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa15bfff640 1 --2- 172.21.15.70:0/2800420542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c14cdd0 0x7fa15c14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa160c37640 1 --2- 172.21.15.70:0/2800420542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa148003970 0x7fa15c1f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.949+0000 7fa161438640 1 -- 172.21.15.70:0/2800420542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c14cdd0 msgr2=0x7fa15c14ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.950+0000 7fa161438640 1 --2- 172.21.15.70:0/2800420542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c14cdd0 0x7fa15c14ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.950+0000 7fa161438640 1 -- 172.21.15.70:0/2800420542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa148003970 msgr2=0x7fa15c1f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.950+0000 7fa161438640 1 --2- 172.21.15.70:0/2800420542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa148003970 0x7fa15c1f6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.950+0000 7fa161438640 1 -- 172.21.15.70:0/2800420542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c14d660 con 0x7fa15c1f1e00 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.950+0000 7fa161438640 1 --2- 172.21.15.70:0/2800420542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa15c1f1e00 0x7fa15c069d50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa15c066310 tx=0x7fa15400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.950+0000 7fa1597fa640 1 -- 172.21.15.70:0/2800420542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540074e0 con 0x7fa15c1f1e00 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.950+0000 7fa1597fa640 1 -- 172.21.15.70:0/2800420542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa154007660 con 0x7fa15c1f1e00 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.950+0000 7fa1597fa640 1 -- 172.21.15.70:0/2800420542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154004a20 con 0x7fa15c1f1e00 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.950+0000 7fa162e9b640 1 -- 172.21.15.70:0/2800420542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c14d940 con 0x7fa15c1f1e00 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.950+0000 7fa162e9b640 1 -- 172.21.15.70:0/2800420542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15c0c8150 con 0x7fa15c1f1e00 2023-10-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.951+0000 7fa162e9b640 1 -- 172.21.15.70:0/2800420542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15c04e160 con 0x7fa15c1f1e00 2023-10-08T05:52:46.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.952+0000 7fa1597fa640 1 -- 172.21.15.70:0/2800420542 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 35) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa154004b80 con 0x7fa15c1f1e00 2023-10-08T05:52:46.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.953+0000 7fa1597fa640 1 -- 172.21.15.70:0/2800420542 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7fa1540883b0 con 0x7fa15c1f1e00 2023-10-08T05:52:46.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:46.958+0000 7fa1597fa640 1 -- 172.21.15.70:0/2800420542 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa154053490 con 0x7fa15c1f1e00 2023-10-08T05:52:47.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.131+0000 7fa162e9b640 1 -- 172.21.15.70:0/2800420542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa15c06f0b0 con 0x7fa15c1f1e00 2023-10-08T05:52:47.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.132+0000 7fa1597fa640 1 -- 172.21.15.70:0/2800420542 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa154056ab0 con 0x7fa15c1f1e00 2023-10-08T05:52:47.132 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:52:47.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.134+0000 7fa162e9b640 1 -- 172.21.15.70:0/2800420542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa15c1f1e00 msgr2=0x7fa15c069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:47.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.134+0000 7fa162e9b640 1 --2- 172.21.15.70:0/2800420542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa15c1f1e00 0x7fa15c069d50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa15c066310 tx=0x7fa15400c8b0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.135+0000 7fa162e9b640 1 -- 172.21.15.70:0/2800420542 shutdown_connections 2023-10-08T05:52:47.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.135+0000 7fa162e9b640 1 --2- 172.21.15.70:0/2800420542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa15c1f1e00 0x7fa15c069d50 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.135+0000 7fa162e9b640 1 --2- 172.21.15.70:0/2800420542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa15c14cdd0 0x7fa15c14ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.135+0000 7fa162e9b640 1 --2- 172.21.15.70:0/2800420542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa148003970 0x7fa15c1f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.135+0000 7fa162e9b640 1 -- 172.21.15.70:0/2800420542 >> 172.21.15.70:0/2800420542 conn(0x7fa15c0cba60 msgr2=0x7fa15c0b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:47.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.135+0000 7fa162e9b640 1 -- 172.21.15.70:0/2800420542 shutdown_connections 2023-10-08T05:52:47.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.135+0000 7fa162e9b640 1 -- 172.21.15.70:0/2800420542 wait complete. 2023-10-08T05:52:47.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:52:47.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.419+0000 7f03b60ea640 1 Processor -- start 2023-10-08T05:52:47.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.419+0000 7f03b60ea640 1 -- start start 2023-10-08T05:52:47.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.421+0000 7f03b60ea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b01599d0 0x7f03b00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.421+0000 7f03b60ea640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f03b00b3c50 0x7f03b00caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.421+0000 7f03b60ea640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03b00cb410 0x7f03b00c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.421+0000 7f03b60ea640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b00cb8a0 con 0x7f03b00cb410 2023-10-08T05:52:47.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.421+0000 7f03b60ea640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b0159db0 con 0x7f03b00b3c50 2023-10-08T05:52:47.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.421+0000 7f03b60ea640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b0071fe0 con 0x7f03b01599d0 2023-10-08T05:52:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.421+0000 7f03b58e9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03b00cb410 0x7f03b00c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.421+0000 7f03b58e9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03b00cb410 0x7f03b00c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46290/0 (socket says 172.21.15.70:46290) 2023-10-08T05:52:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.421+0000 7f03b58e9640 1 -- 172.21.15.70:0/361120340 learned_addr learned my addr 172.21.15.70:0/361120340 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.422+0000 7f03b50e8640 1 --2- 172.21.15.70:0/361120340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b01599d0 0x7f03b00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.421+0000 7f03b48e7640 1 --2- 172.21.15.70:0/361120340 >> v2:172.21.15.181:3300/0 conn(0x7f03b00b3c50 0x7f03b00caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.422+0000 7f03b58e9640 1 -- 172.21.15.70:0/361120340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b01599d0 msgr2=0x7f03b00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.422+0000 7f03b58e9640 1 --2- 172.21.15.70:0/361120340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b01599d0 0x7f03b00cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.422+0000 7f03b58e9640 1 -- 172.21.15.70:0/361120340 >> v2:172.21.15.181:3300/0 conn(0x7f03b00b3c50 msgr2=0x7f03b00caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.422+0000 7f03b58e9640 1 --2- 172.21.15.70:0/361120340 >> v2:172.21.15.181:3300/0 conn(0x7f03b00b3c50 0x7f03b00caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.422+0000 7f03b58e9640 1 -- 172.21.15.70:0/361120340 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00b44c0 con 0x7f03b00cb410 2023-10-08T05:52:47.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.422+0000 7f03b48e7640 1 --2- 172.21.15.70:0/361120340 >> v2:172.21.15.181:3300/0 conn(0x7f03b00b3c50 0x7f03b00caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:47.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.422+0000 7f03b50e8640 1 --2- 172.21.15.70:0/361120340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b01599d0 0x7f03b00cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.423+0000 7f03b58e9640 1 --2- 172.21.15.70:0/361120340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b00cb410 0x7f03b00c8b40 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f03b006dad0 tx=0x7f03a802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1122fe5e5c48360c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.423+0000 7f03a7fff640 1 -- 172.21.15.70:0/361120340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a802ca80 con 0x7f03b00cb410 2023-10-08T05:52:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.423+0000 7f03a7fff640 1 -- 172.21.15.70:0/361120340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a802cbe0 con 0x7f03b00cb410 2023-10-08T05:52:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.423+0000 7f03a7fff640 1 -- 172.21.15.70:0/361120340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a80343f0 con 0x7f03b00cb410 2023-10-08T05:52:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.424+0000 7f03b60ea640 1 -- 172.21.15.70:0/361120340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b00cb410 msgr2=0x7f03b00c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.424+0000 7f03b60ea640 1 --2- 172.21.15.70:0/361120340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b00cb410 0x7f03b00c8b40 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f03b006dad0 tx=0x7f03a802bee0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.424+0000 7f03b60ea640 1 -- 172.21.15.70:0/361120340 shutdown_connections 2023-10-08T05:52:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.424+0000 7f03b60ea640 1 --2- 172.21.15.70:0/361120340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b00cb410 0x7f03b00c8b40 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.424+0000 7f03b60ea640 1 --2- 172.21.15.70:0/361120340 >> v2:172.21.15.181:3300/0 conn(0x7f03b00b3c50 0x7f03b00caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.424+0000 7f03b60ea640 1 --2- 172.21.15.70:0/361120340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b01599d0 0x7f03b00cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.424+0000 7f03b60ea640 1 -- 172.21.15.70:0/361120340 >> 172.21.15.70:0/361120340 conn(0x7f03b006c050 msgr2=0x7f03b006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:47.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.424+0000 7f03b60ea640 1 -- 172.21.15.70:0/361120340 shutdown_connections 2023-10-08T05:52:47.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.424+0000 7f03b60ea640 1 -- 172.21.15.70:0/361120340 wait complete. 2023-10-08T05:52:47.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.425+0000 7f03b60ea640 1 Processor -- start 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.426+0000 7f03b60ea640 1 -- start start 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b60ea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b00b3c50 0x7f03b014b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b60ea640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03b00cb410 0x7f03b014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b60ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b01599d0 0x7f03b014c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b60ea640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b014d670 con 0x7f03b01599d0 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b60ea640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b0158210 con 0x7f03b00cb410 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b60ea640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b014c9f0 con 0x7f03b00b3c50 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b58e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b01599d0 0x7f03b014c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b58e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b01599d0 0x7f03b014c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46302/0 (socket says 172.21.15.70:46302) 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b58e9640 1 -- 172.21.15.70:0/2316819900 learned_addr learned my addr 172.21.15.70:0/2316819900 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:47.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b50e8640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b00b3c50 0x7f03b014b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b58e9640 1 -- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b00b3c50 msgr2=0x7f03b014b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:47.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b48e7640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03b00cb410 0x7f03b014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b58e9640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b00b3c50 0x7f03b014b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b58e9640 1 -- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03b00cb410 msgr2=0x7f03b014be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:47.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b58e9640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03b00cb410 0x7f03b014be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.427+0000 7f03b58e9640 1 -- 172.21.15.70:0/2316819900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a8009b30 con 0x7f03b01599d0 2023-10-08T05:52:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.428+0000 7f03b58e9640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b01599d0 0x7f03b014c3c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f03a8036040 tx=0x7f03a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.428+0000 7f03a5ffb640 1 -- 172.21.15.70:0/2316819900 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8009510 con 0x7f03b01599d0 2023-10-08T05:52:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.428+0000 7f03a5ffb640 1 -- 172.21.15.70:0/2316819900 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a803e450 con 0x7f03b01599d0 2023-10-08T05:52:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.428+0000 7f03a5ffb640 1 -- 172.21.15.70:0/2316819900 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a803e800 con 0x7f03b01599d0 2023-10-08T05:52:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.428+0000 7f03b60ea640 1 -- 172.21.15.70:0/2316819900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b014ccd0 con 0x7f03b01599d0 2023-10-08T05:52:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.428+0000 7f03b60ea640 1 -- 172.21.15.70:0/2316819900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b014d290 con 0x7f03b01599d0 2023-10-08T05:52:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.429+0000 7f03b60ea640 1 -- 172.21.15.70:0/2316819900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b014d7b0 con 0x7f03b01599d0 2023-10-08T05:52:47.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.430+0000 7f03a5ffb640 1 -- 172.21.15.70:0/2316819900 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 35) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f03a802cd50 con 0x7f03b01599d0 2023-10-08T05:52:47.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.431+0000 7f03a5ffb640 1 -- 172.21.15.70:0/2316819900 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f03a80ac120 con 0x7f03b01599d0 2023-10-08T05:52:47.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.436+0000 7f03a5ffb640 1 -- 172.21.15.70:0/2316819900 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03b014d7b0 con 0x7f03b01599d0 2023-10-08T05:52:47.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.590+0000 7f03a5ffb640 1 -- 172.21.15.70:0/2316819900 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 36) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f03a8076c70 con 0x7f03b01599d0 2023-10-08T05:52:47.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.591+0000 7f03a5ffb640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f0394070bb0 0x7f0394072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.593+0000 7f03b50e8640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f0394070bb0 0x7f0394072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.597+0000 7f03b50e8640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f0394070bb0 0x7f0394072fa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00a950 tx=0x7f03ac008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:47.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.624+0000 7f03b60ea640 1 -- 172.21.15.70:0/2316819900 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03b014d7b0 con 0x7f03b01599d0 2023-10-08T05:52:47.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.625+0000 7f03a5ffb640 1 -- 172.21.15.70:0/2316819900 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03b014d7b0 con 0x7f03b01599d0 2023-10-08T05:52:47.625 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:52:47.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.627+0000 7f03b60ea640 1 -- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f0394070bb0 msgr2=0x7f0394072fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:47.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.627+0000 7f03b60ea640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f0394070bb0 0x7f0394072fa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00a950 tx=0x7f03ac008040 comp rx=0 tx=0).stop 2023-10-08T05:52:47.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.627+0000 7f03b60ea640 1 -- 172.21.15.70:0/2316819900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b01599d0 msgr2=0x7f03b014c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:47.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.627+0000 7f03b60ea640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b01599d0 0x7f03b014c3c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f03a8036040 tx=0x7f03a8004a40 comp rx=0 tx=0).stop 2023-10-08T05:52:47.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.627+0000 7f03b60ea640 1 -- 172.21.15.70:0/2316819900 shutdown_connections 2023-10-08T05:52:47.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.628+0000 7f03b60ea640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f0394070bb0 0x7f0394072fa0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.628+0000 7f03b60ea640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b01599d0 0x7f03b014c3c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.628+0000 7f03b60ea640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03b00cb410 0x7f03b014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.628+0000 7f03b60ea640 1 --2- 172.21.15.70:0/2316819900 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03b00b3c50 0x7f03b014b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.628+0000 7f03b60ea640 1 -- 172.21.15.70:0/2316819900 >> 172.21.15.70:0/2316819900 conn(0x7f03b006c050 msgr2=0x7f03b00ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:47.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.628+0000 7f03b60ea640 1 -- 172.21.15.70:0/2316819900 shutdown_connections 2023-10-08T05:52:47.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.628+0000 7f03b60ea640 1 -- 172.21.15.70:0/2316819900 wait complete. 2023-10-08T05:52:47.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:47.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.847+0000 7f4033a32640 1 Processor -- start 2023-10-08T05:52:47.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.847+0000 7f4033a32640 1 -- start start 2023-10-08T05:52:47.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.849+0000 7f4033a32640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f402c15cd90 0x7f402c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.849+0000 7f4033a32640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f402c06c890 0x7f402c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.849+0000 7f4033a32640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402c0c48f0 con 0x7f402c160940 2023-10-08T05:52:47.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.849+0000 7f4033a32640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402c0c4d80 con 0x7f402c15cd90 2023-10-08T05:52:47.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.849+0000 7f4033a32640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402c0c3ce0 con 0x7f402c06c890 2023-10-08T05:52:47.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.849+0000 7f4030fcd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f402c160940 0x7f402c070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58496/0 (socket says 172.21.15.70:58496) 2023-10-08T05:52:47.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.849+0000 7f40317ce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f402c15cd90 0x7f402c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.849+0000 7f4030fcd640 1 -- 172.21.15.70:0/3958745228 learned_addr learned my addr 172.21.15.70:0/3958745228 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:47.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.849+0000 7f40317ce640 1 -- 172.21.15.70:0/3958745228 >> v2:172.21.15.181:3300/0 conn(0x7f402c06c890 msgr2=0x7f402c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:47.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.849+0000 7f4031fcf640 1 --2- 172.21.15.70:0/3958745228 >> v2:172.21.15.181:3300/0 conn(0x7f402c06c890 0x7f402c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.850+0000 7f40317ce640 1 --2- 172.21.15.70:0/3958745228 >> v2:172.21.15.181:3300/0 conn(0x7f402c06c890 0x7f402c06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.850+0000 7f40317ce640 1 -- 172.21.15.70:0/3958745228 >> v1:172.21.15.70:6789/0 conn(0x7f402c160940 legacy=0x7f402c070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:47.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.850+0000 7f401ffff640 1 -- 172.21.15.70:0/3958745228 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616227547 0 0) 0x7f402c0c48f0 con 0x7f402c160940 2023-10-08T05:52:47.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.850+0000 7f40317ce640 1 -- 172.21.15.70:0/3958745228 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c0c3fc0 con 0x7f402c15cd90 2023-10-08T05:52:47.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.850+0000 7f4031fcf640 1 --2- 172.21.15.70:0/3958745228 >> v2:172.21.15.181:3300/0 conn(0x7f402c06c890 0x7f402c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:47.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.851+0000 7f40317ce640 1 --2- 172.21.15.70:0/3958745228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 0x7f402c0702d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f402c0c8d60 tx=0x7f402802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d51d10b97a15a43 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.851+0000 7f401ffff640 1 -- 172.21.15.70:0/3958745228 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028031830 con 0x7f402c15cd90 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.851+0000 7f401ffff640 1 -- 172.21.15.70:0/3958745228 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4028031990 con 0x7f402c15cd90 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.851+0000 7f401ffff640 1 -- 172.21.15.70:0/3958745228 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4028031d20 con 0x7f402c15cd90 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.852+0000 7f4033a32640 1 -- 172.21.15.70:0/3958745228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 msgr2=0x7f402c0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.852+0000 7f4033a32640 1 --2- 172.21.15.70:0/3958745228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 0x7f402c0702d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f402c0c8d60 tx=0x7f402802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.852+0000 7f4033a32640 1 -- 172.21.15.70:0/3958745228 shutdown_connections 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.852+0000 7f4033a32640 1 --2- 172.21.15.70:0/3958745228 >> v2:172.21.15.181:3300/0 conn(0x7f402c06c890 0x7f402c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.852+0000 7f4033a32640 1 --2- 172.21.15.70:0/3958745228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 0x7f402c0702d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.852+0000 7f4033a32640 1 -- 172.21.15.70:0/3958745228 >> 172.21.15.70:0/3958745228 conn(0x7f402c06a380 msgr2=0x7f402c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.852+0000 7f4033a32640 1 -- 172.21.15.70:0/3958745228 shutdown_connections 2023-10-08T05:52:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.852+0000 7f4033a32640 1 -- 172.21.15.70:0/3958745228 wait complete. 2023-10-08T05:52:47.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.853+0000 7f4033a32640 1 Processor -- start 2023-10-08T05:52:47.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.854+0000 7f4033a32640 1 -- start start 2023-10-08T05:52:47.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4033a32640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c06c890 0x7f402c15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4033a32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 0x7f402c15fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4033a32640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f402c160940 0x7f402c0b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4033a32640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0c6ed0 con 0x7f402c15cd90 2023-10-08T05:52:47.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4033a32640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1484e0 con 0x7f402c06c890 2023-10-08T05:52:47.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4033a32640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c160260 con 0x7f402c160940 2023-10-08T05:52:47.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4030fcd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 0x7f402c15fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4030fcd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 0x7f402c15fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46308/0 (socket says 172.21.15.70:46308) 2023-10-08T05:52:47.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4030fcd640 1 -- 172.21.15.70:0/3362597435 learned_addr learned my addr 172.21.15.70:0/3362597435 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:47.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4030fcd640 1 -- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f402c160940 msgr2=0x7f402c0b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:47.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4030fcd640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f402c160940 0x7f402c0b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4030fcd640 1 -- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c06c890 msgr2=0x7f402c15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:47.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f40317ce640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c06c890 0x7f402c15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4030fcd640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c06c890 0x7f402c15f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:47.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.855+0000 7f4030fcd640 1 -- 172.21.15.70:0/3362597435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4028009400 con 0x7f402c15cd90 2023-10-08T05:52:47.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.856+0000 7f4030fcd640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 0x7f402c15fd20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f402000c430 tx=0x7f402000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:47.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.856+0000 7f401dffb640 1 -- 172.21.15.70:0/3362597435 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40200074e0 con 0x7f402c15cd90 2023-10-08T05:52:47.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.856+0000 7f401dffb640 1 -- 172.21.15.70:0/3362597435 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4020007660 con 0x7f402c15cd90 2023-10-08T05:52:47.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.856+0000 7f401dffb640 1 -- 172.21.15.70:0/3362597435 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4020004a20 con 0x7f402c15cd90 2023-10-08T05:52:47.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.856+0000 7f4033a32640 1 -- 172.21.15.70:0/3362597435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c0c68a0 con 0x7f402c15cd90 2023-10-08T05:52:47.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.856+0000 7f4033a32640 1 -- 172.21.15.70:0/3362597435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c0b41a0 con 0x7f402c15cd90 2023-10-08T05:52:47.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.857+0000 7f4033a32640 1 -- 172.21.15.70:0/3362597435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402c0c7010 con 0x7f402c15cd90 2023-10-08T05:52:47.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.858+0000 7f401dffb640 1 -- 172.21.15.70:0/3362597435 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4020004b80 con 0x7f402c15cd90 2023-10-08T05:52:47.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.859+0000 7f401dffb640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f401006acc0 0x7f401006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:47.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.859+0000 7f401dffb640 1 -- 172.21.15.70:0/3362597435 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f4020089460 con 0x7f402c15cd90 2023-10-08T05:52:47.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.860+0000 7f40317ce640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f401006acc0 0x7f401006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:47.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.861+0000 7f40317ce640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f401006acc0 0x7f401006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4028036000 tx=0x7f4028008030 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:47.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:47.863+0000 7f401dffb640 1 -- 172.21.15.70:0/3362597435 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f402c0c7010 con 0x7f402c15cd90 2023-10-08T05:52:48.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.089+0000 7f4033a32640 1 -- 172.21.15.70:0/3362597435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f402c0c7010 con 0x7f402c15cd90 2023-10-08T05:52:48.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.090+0000 7f401dffb640 1 -- 172.21.15.70:0/3362597435 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f402c0c7010 con 0x7f402c15cd90 2023-10-08T05:52:48.091 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:48.091 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1038137327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1038137327},{"type":"v1","addr":"172.21.15.181:6835","nonce":1038137327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6660,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1967783419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1967783419},{"type":"v1","addr":"172.21.15.70:6833","nonce":1967783419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6663,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1304760341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1304760341},{"type":"v1","addr":"172.21.15.70:6835","nonce":1304760341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:52:26.053790+0000","modified":"2023-10-08T05:52:29.103940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6675},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6675":{"gid":6675,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2348955257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2348955257},{"type":"v1","addr":"172.21.15.181:6837","nonce":2348955257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-08T05:52:48.091 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 33 2023-10-08T05:52:48.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.093+0000 7f4033a32640 1 -- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f401006acc0 msgr2=0x7f401006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.093+0000 7f4033a32640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f401006acc0 0x7f401006d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4028036000 tx=0x7f4028008030 comp rx=0 tx=0).stop 2023-10-08T05:52:48.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.094+0000 7f4033a32640 1 -- 172.21.15.70:0/3362597435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 msgr2=0x7f402c15fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.094+0000 7f4033a32640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 0x7f402c15fd20 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f402000c430 tx=0x7f402000c8f0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.094+0000 7f4033a32640 1 -- 172.21.15.70:0/3362597435 shutdown_connections 2023-10-08T05:52:48.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.094+0000 7f4033a32640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f401006acc0 0x7f401006d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.094+0000 7f4033a32640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f402c160940 0x7f402c0b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.094+0000 7f4033a32640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f402c15cd90 0x7f402c15fd20 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.094+0000 7f4033a32640 1 --2- 172.21.15.70:0/3362597435 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f402c06c890 0x7f402c15f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.094+0000 7f4033a32640 1 -- 172.21.15.70:0/3362597435 >> 172.21.15.70:0/3362597435 conn(0x7f402c06a380 msgr2=0x7f402c160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:48.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.094+0000 7f4033a32640 1 -- 172.21.15.70:0/3362597435 shutdown_connections 2023-10-08T05:52:48.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.094+0000 7f4033a32640 1 -- 172.21.15.70:0/3362597435 wait complete. 2023-10-08T05:52:48.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:48.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.325+0000 7f097f59e640 1 Processor -- start 2023-10-08T05:52:48.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.325+0000 7f097f59e640 1 -- start start 2023-10-08T05:52:48.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f097f59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f098015ab30 0x7f098015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f097f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09800b07d0 0x7f098006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f097f59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098015f7d0 con 0x7f09800adc70 2023-10-08T05:52:48.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f097f59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098015f210 con 0x7f098015ab30 2023-10-08T05:52:48.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f097f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098015f4f0 con 0x7f09800b07d0 2023-10-08T05:52:48.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f0977fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f09800adc70 0x7f09800b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58512/0 (socket says 172.21.15.70:58512) 2023-10-08T05:52:48.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f0977fff640 1 -- 172.21.15.70:0/2237425515 learned_addr learned my addr 172.21.15.70:0/2237425515 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:48.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631778041 0 0) 0x7f098015f7d0 con 0x7f09800adc70 2023-10-08T05:52:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f097ed9d640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09800b07d0 0x7f098006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f097e59c640 1 --2- 172.21.15.70:0/2237425515 >> v2:172.21.15.181:3300/0 conn(0x7f098015ab30 0x7f098015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.327+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f096c003520 con 0x7f09800adc70 2023-10-08T05:52:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 116091360 0 0) 0x7f096c003520 con 0x7f09800adc70 2023-10-08T05:52:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09800b07d0 msgr2=0x7f098006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09800b07d0 0x7f098006cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 >> v2:172.21.15.181:3300/0 conn(0x7f098015ab30 msgr2=0x7f098015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 --2- 172.21.15.70:0/2237425515 >> v2:172.21.15.181:3300/0 conn(0x7f098015ab30 0x7f098015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098006c550 con 0x7f09800adc70 2023-10-08T05:52:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0968001670 con 0x7f09800adc70 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097ed9d640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09800b07d0 0x7f098006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 >> v1:172.21.15.70:6789/0 conn(0x7f09800adc70 legacy=0x7f09800b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c003970 0x7f096c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09800b07d0 0x7f096c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c004d00 0x7f096c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098015f210 con 0x7f096c004d00 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098015f4f0 con 0x7f096c003970 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c00ba60 con 0x7f09800b07d0 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097e59c640 1 --2- 172.21.15.70:0/2237425515 >> v2:172.21.15.181:3300/0 conn(0x7f098015ab30 0x7f098015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.328+0000 7f097ed9d640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c004d00 0x7f096c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.329+0000 7f0977fff640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09800b07d0 0x7f096c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.329+0000 7f097ed9d640 1 -- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09800b07d0 msgr2=0x7f096c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:48.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.329+0000 7f097ed9d640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09800b07d0 0x7f096c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.329+0000 7f097e59c640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c003970 0x7f096c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.329+0000 7f097ed9d640 1 -- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c003970 msgr2=0x7f096c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:48.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.329+0000 7f097ed9d640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c003970 0x7f096c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.329+0000 7f097ed9d640 1 -- 172.21.15.70:0/2237425515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c003520 con 0x7f096c004d00 2023-10-08T05:52:48.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.329+0000 7f0977fff640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09800b07d0 0x7f096c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.330+0000 7f097ed9d640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c004d00 0x7f096c005180 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f09780073b0 tx=0x7f096c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88243933e3c07c39 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.330+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097802cd30 con 0x7f096c004d00 2023-10-08T05:52:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.330+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097802ce90 con 0x7f096c004d00 2023-10-08T05:52:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.330+0000 7f097dd9b640 1 -- 172.21.15.70:0/2237425515 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978034640 con 0x7f096c004d00 2023-10-08T05:52:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.331+0000 7f097f59e640 1 -- 172.21.15.70:0/2237425515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c004d00 msgr2=0x7f096c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.331+0000 7f097f59e640 1 --2- 172.21.15.70:0/2237425515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c004d00 0x7f096c005180 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f09780073b0 tx=0x7f096c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.331+0000 7f097ed9d640 1 -- 172.21.15.70:0/2237425515 reap_dead start 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.331+0000 7f097f59e640 1 -- 172.21.15.70:0/2237425515 shutdown_connections 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.331+0000 7f097f59e640 1 -- 172.21.15.70:0/2237425515 >> 172.21.15.70:0/2237425515 conn(0x7f09800cb0c0 msgr2=0x7f09800cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.331+0000 7f097f59e640 1 -- 172.21.15.70:0/2237425515 shutdown_connections 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.331+0000 7f097f59e640 1 -- 172.21.15.70:0/2237425515 wait complete. 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.332+0000 7f097f59e640 1 Processor -- start 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.333+0000 7f097f59e640 1 -- start start 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.333+0000 7f097f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c004d00 0x7f09801f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f097f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09800c2680 0x7f098014f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f097f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09801577a0 0x7f0980157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f097f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0980158160 con 0x7f09800c2680 2023-10-08T05:52:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f097f59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801f4bd0 con 0x7f096c004d00 2023-10-08T05:52:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f097f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09801f0590 con 0x7f09801577a0 2023-10-08T05:52:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f0977fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09800c2680 0x7f098014f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f0977fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09800c2680 0x7f098014f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46330/0 (socket says 172.21.15.70:46330) 2023-10-08T05:52:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f0977fff640 1 -- 172.21.15.70:0/3897432245 learned_addr learned my addr 172.21.15.70:0/3897432245 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f097e59c640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c004d00 0x7f09801f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f097ed9d640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09801577a0 0x7f0980157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f0977fff640 1 -- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09801577a0 msgr2=0x7f0980157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f0977fff640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09801577a0 0x7f0980157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f0977fff640 1 -- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c004d00 msgr2=0x7f09801f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f0977fff640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c004d00 0x7f09801f4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f0977fff640 1 -- 172.21.15.70:0/3897432245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800c5ab0 con 0x7f09800c2680 2023-10-08T05:52:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.334+0000 7f097e59c640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c004d00 0x7f09801f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.335+0000 7f0977fff640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09800c2680 0x7f098014f200 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f096800d7e0 tx=0x7f096800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.335+0000 7f0976ffd640 1 -- 172.21.15.70:0/3897432245 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09680074e0 con 0x7f09800c2680 2023-10-08T05:52:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.335+0000 7f0976ffd640 1 -- 172.21.15.70:0/3897432245 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0968007640 con 0x7f09800c2680 2023-10-08T05:52:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.335+0000 7f0976ffd640 1 -- 172.21.15.70:0/3897432245 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09680049e0 con 0x7f09800c2680 2023-10-08T05:52:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.335+0000 7f097f59e640 1 -- 172.21.15.70:0/3897432245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09800c5d90 con 0x7f09800c2680 2023-10-08T05:52:48.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.335+0000 7f097f59e640 1 -- 172.21.15.70:0/3897432245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09800c6350 con 0x7f09800c2680 2023-10-08T05:52:48.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.336+0000 7f097f59e640 1 -- 172.21.15.70:0/3897432245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09800b4070 con 0x7f09800c2680 2023-10-08T05:52:48.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.337+0000 7f0976ffd640 1 -- 172.21.15.70:0/3897432245 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 36) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f0968004b40 con 0x7f09800c2680 2023-10-08T05:52:48.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.338+0000 7f0976ffd640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f096006acc0 0x7f096006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.339+0000 7f0976ffd640 1 -- 172.21.15.70:0/3897432245 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f09680890d0 con 0x7f09800c2680 2023-10-08T05:52:48.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.339+0000 7f097e59c640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f096006acc0 0x7f096006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.340+0000 7f097e59c640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f096006acc0 0x7f096006d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f097000ba40 tx=0x7f0970008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:48.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.343+0000 7f0976ffd640 1 -- 172.21.15.70:0/3897432245 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0968053fd0 con 0x7f09800c2680 2023-10-08T05:52:48.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.532+0000 7f097f59e640 1 -- 172.21.15.70:0/3897432245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f098004e160 con 0x7f09800c2680 2023-10-08T05:52:48.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.534+0000 7f0976ffd640 1 -- 172.21.15.70:0/3897432245 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17791 (secure 0 0 0) 0x7f0968019090 con 0x7f09800c2680 2023-10-08T05:52:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:48.535 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":92,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:46.566144+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:52:23.846215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-08T05:52:25.069536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3148780950":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/999647594":"2023-10-09T05:52:46.566070+0000","172.21.15.181:6833/1709912921":"2023-10-09T05:52:46.566070+0000","172.21.15.181:6832/1709912921":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/3306361152":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/2398424071":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/814792959":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1860309542":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/2758467223":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1699086184":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1378633292":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1217040450":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/3712172314":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/2333154107":"2023-10-09T05:52:40.392573+0000","172.21.15.181:6833/3994892546":"2023-10-09T05:52:40.392573+0000","172.21.15.181:6832/3994892546":"2023-10-09T05:52:40.392573+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:48.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.536+0000 7f097f59e640 1 -- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f096006acc0 msgr2=0x7f096006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.536+0000 7f097f59e640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f096006acc0 0x7f096006d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f097000ba40 tx=0x7f0970008040 comp rx=0 tx=0).stop 2023-10-08T05:52:48.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.536+0000 7f097f59e640 1 -- 172.21.15.70:0/3897432245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09800c2680 msgr2=0x7f098014f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.536+0000 7f097f59e640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09800c2680 0x7f098014f200 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f096800d7e0 tx=0x7f096800dca0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.537+0000 7f097f59e640 1 -- 172.21.15.70:0/3897432245 shutdown_connections 2023-10-08T05:52:48.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.537+0000 7f097f59e640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f096006acc0 0x7f096006d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.537+0000 7f097f59e640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09801577a0 0x7f0980157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.537+0000 7f097f59e640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09800c2680 0x7f098014f200 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.537+0000 7f097f59e640 1 --2- 172.21.15.70:0/3897432245 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c004d00 0x7f09801f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.537+0000 7f097f59e640 1 -- 172.21.15.70:0/3897432245 >> 172.21.15.70:0/3897432245 conn(0x7f09800cb0c0 msgr2=0x7f09800cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:48.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.537+0000 7f097f59e640 1 -- 172.21.15.70:0/3897432245 shutdown_connections 2023-10-08T05:52:48.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.537+0000 7f097f59e640 1 -- 172.21.15.70:0/3897432245 wait complete. 2023-10-08T05:52:48.556 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T05:52:48.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:48.633 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:48.630+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:48.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.769+0000 7ffbdffff640 1 Processor -- start 2023-10-08T05:52:48.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.769+0000 7ffbdffff640 1 -- start start 2023-10-08T05:52:48.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.771+0000 7ffbdffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffbe015aa10 0x7ffbe006c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.771+0000 7ffbdffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffbe006cec0 0x7ffbe00b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.771+0000 7ffbdffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffbe00b48e0 0x7ffbe0070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.771+0000 7ffbdffff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe006d400 con 0x7ffbe006cec0 2023-10-08T05:52:48.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.771+0000 7ffbdffff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe00b4d70 con 0x7ffbe015aa10 2023-10-08T05:52:48.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.771+0000 7ffbdffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe015f4e0 con 0x7ffbe00b48e0 2023-10-08T05:52:48.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.772+0000 7ffbd67fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffbe006cec0 0x7ffbe00b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.772+0000 7ffbd67fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffbe006cec0 0x7ffbe00b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46338/0 (socket says 172.21.15.70:46338) 2023-10-08T05:52:48.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.772+0000 7ffbd67fc640 1 -- 172.21.15.70:0/1190278687 learned_addr learned my addr 172.21.15.70:0/1190278687 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:48.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.772+0000 7ffbdeffd640 1 --2- 172.21.15.70:0/1190278687 >> v2:172.21.15.181:3300/0 conn(0x7ffbe015aa10 0x7ffbe006c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.772+0000 7ffbdf7fe640 1 --2- 172.21.15.70:0/1190278687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffbe00b48e0 0x7ffbe0070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.772+0000 7ffbd67fc640 1 -- 172.21.15.70:0/1190278687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffbe00b48e0 msgr2=0x7ffbe0070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:48.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.772+0000 7ffbd67fc640 1 --2- 172.21.15.70:0/1190278687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffbe00b48e0 0x7ffbe0070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.772+0000 7ffbd67fc640 1 -- 172.21.15.70:0/1190278687 >> v2:172.21.15.181:3300/0 conn(0x7ffbe015aa10 msgr2=0x7ffbe006c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:48.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.772+0000 7ffbd67fc640 1 --2- 172.21.15.70:0/1190278687 >> v2:172.21.15.181:3300/0 conn(0x7ffbe015aa10 0x7ffbe006c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.772+0000 7ffbd67fc640 1 -- 172.21.15.70:0/1190278687 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe0069d50 con 0x7ffbe006cec0 2023-10-08T05:52:48.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.773+0000 7ffbd67fc640 1 --2- 172.21.15.70:0/1190278687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe006cec0 0x7ffbe00b43a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00cfad0 tx=0x7ffbc802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7395685add987a1c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbde7fc640 1 -- 172.21.15.70:0/1190278687 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8031870 con 0x7ffbe006cec0 2023-10-08T05:52:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbde7fc640 1 -- 172.21.15.70:0/1190278687 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc80319d0 con 0x7ffbe006cec0 2023-10-08T05:52:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbde7fc640 1 -- 172.21.15.70:0/1190278687 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8031d60 con 0x7ffbe006cec0 2023-10-08T05:52:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbdffff640 1 -- 172.21.15.70:0/1190278687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe006cec0 msgr2=0x7ffbe00b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbdffff640 1 --2- 172.21.15.70:0/1190278687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe006cec0 0x7ffbe00b43a0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00cfad0 tx=0x7ffbc802b910 comp rx=0 tx=0).stop 2023-10-08T05:52:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbdffff640 1 -- 172.21.15.70:0/1190278687 shutdown_connections 2023-10-08T05:52:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbdffff640 1 --2- 172.21.15.70:0/1190278687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffbe00b48e0 0x7ffbe0070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbdffff640 1 --2- 172.21.15.70:0/1190278687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe006cec0 0x7ffbe00b43a0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbdffff640 1 --2- 172.21.15.70:0/1190278687 >> v2:172.21.15.181:3300/0 conn(0x7ffbe015aa10 0x7ffbe006c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbdffff640 1 -- 172.21.15.70:0/1190278687 >> 172.21.15.70:0/1190278687 conn(0x7ffbe00cede0 msgr2=0x7ffbe00cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:48.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbdffff640 1 -- 172.21.15.70:0/1190278687 shutdown_connections 2023-10-08T05:52:48.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.774+0000 7ffbdffff640 1 -- 172.21.15.70:0/1190278687 wait complete. 2023-10-08T05:52:48.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.776+0000 7ffbdffff640 1 Processor -- start 2023-10-08T05:52:48.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.776+0000 7ffbdffff640 1 -- start start 2023-10-08T05:52:48.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.777+0000 7ffbdffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe006cec0 0x7ffbe015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.777+0000 7ffbdffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffbe00b48e0 0x7ffbe00c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.777+0000 7ffbdffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe015aa10 0x7ffbe014f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.777+0000 7ffbdffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe014bbc0 con 0x7ffbe015aa10 2023-10-08T05:52:48.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.777+0000 7ffbdffff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe014afb0 con 0x7ffbe006cec0 2023-10-08T05:52:48.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.777+0000 7ffbdffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe014b290 con 0x7ffbe00b48e0 2023-10-08T05:52:48.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbdf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe015aa10 0x7ffbe014f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbdf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe015aa10 0x7ffbe014f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46348/0 (socket says 172.21.15.70:46348) 2023-10-08T05:52:48.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbdf7fe640 1 -- 172.21.15.70:0/2655283830 learned_addr learned my addr 172.21.15.70:0/2655283830 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:48.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbdeffd640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe006cec0 0x7ffbe015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbd67fc640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffbe00b48e0 0x7ffbe00c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbdf7fe640 1 -- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffbe00b48e0 msgr2=0x7ffbe00c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbdf7fe640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffbe00b48e0 0x7ffbe00c2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbdf7fe640 1 -- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe006cec0 msgr2=0x7ffbe015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:48.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbdf7fe640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe006cec0 0x7ffbe015d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:48.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbdf7fe640 1 -- 172.21.15.70:0/2655283830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc80094d0 con 0x7ffbe015aa10 2023-10-08T05:52:48.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.778+0000 7ffbdf7fe640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe015aa10 0x7ffbe014f090 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8007e60 tx=0x7ffbd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:48.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.779+0000 7ffbdcff9640 1 -- 172.21.15.70:0/2655283830 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8014620 con 0x7ffbe015aa10 2023-10-08T05:52:48.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.779+0000 7ffbdcff9640 1 -- 172.21.15.70:0/2655283830 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd800ee90 con 0x7ffbe015aa10 2023-10-08T05:52:48.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.779+0000 7ffbdcff9640 1 -- 172.21.15.70:0/2655283830 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd80149e0 con 0x7ffbe015aa10 2023-10-08T05:52:48.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.779+0000 7ffbdffff640 1 -- 172.21.15.70:0/2655283830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe014b570 con 0x7ffbe015aa10 2023-10-08T05:52:48.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.779+0000 7ffbdffff640 1 -- 172.21.15.70:0/2655283830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe015c3c0 con 0x7ffbe015aa10 2023-10-08T05:52:48.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.780+0000 7ffbdffff640 1 -- 172.21.15.70:0/2655283830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe015e6e0 con 0x7ffbe015aa10 2023-10-08T05:52:48.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.781+0000 7ffbdcff9640 1 -- 172.21.15.70:0/2655283830 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ffbd8014b40 con 0x7ffbe015aa10 2023-10-08T05:52:48.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.782+0000 7ffbdcff9640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ffbc006af30 0x7ffbc006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:48.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.783+0000 7ffbdcff9640 1 -- 172.21.15.70:0/2655283830 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7ffbd808a460 con 0x7ffbe015aa10 2023-10-08T05:52:48.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.783+0000 7ffbdeffd640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ffbc006af30 0x7ffbc006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:48.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.784+0000 7ffbdeffd640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ffbc006af30 0x7ffbc006d3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0006fd0 tx=0x7ffbd0008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:48.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:48.787+0000 7ffbdcff9640 1 -- 172.21.15.70:0/2655283830 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffbd8055000 con 0x7ffbe015aa10 2023-10-08T05:52:49.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.017+0000 7ffbdffff640 1 -- 172.21.15.70:0/2655283830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbe004e160 con 0x7ffbe015aa10 2023-10-08T05:52:49.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.018+0000 7ffbdcff9640 1 -- 172.21.15.70:0/2655283830 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4920 (secure 0 0 0) 0x7ffbd8058620 con 0x7ffbe015aa10 2023-10-08T05:52:49.019 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:49.019 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1038137327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1038137327},{"type":"v1","addr":"172.21.15.181:6835","nonce":1038137327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6660,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1967783419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1967783419},{"type":"v1","addr":"172.21.15.70:6833","nonce":1967783419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6663,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1304760341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1304760341},{"type":"v1","addr":"172.21.15.70:6835","nonce":1304760341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:52:26.053790+0000","modified":"2023-10-08T05:52:29.103940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6675},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6675":{"gid":6675,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2348955257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2348955257},{"type":"v1","addr":"172.21.15.181:6837","nonce":2348955257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-08T05:52:49.019 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 33 2023-10-08T05:52:49.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.021+0000 7ffbdffff640 1 -- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ffbc006af30 msgr2=0x7ffbc006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.021+0000 7ffbdffff640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ffbc006af30 0x7ffbc006d3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0006fd0 tx=0x7ffbd0008040 comp rx=0 tx=0).stop 2023-10-08T05:52:49.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.021+0000 7ffbdffff640 1 -- 172.21.15.70:0/2655283830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe015aa10 msgr2=0x7ffbe014f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.021+0000 7ffbdffff640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe015aa10 0x7ffbe014f090 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8007e60 tx=0x7ffbd800e4b0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.021+0000 7ffbdffff640 1 -- 172.21.15.70:0/2655283830 shutdown_connections 2023-10-08T05:52:49.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.021+0000 7ffbdffff640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ffbc006af30 0x7ffbc006d3f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.021+0000 7ffbdffff640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbe015aa10 0x7ffbe014f090 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.021+0000 7ffbdffff640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffbe00b48e0 0x7ffbe00c2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.022+0000 7ffbdffff640 1 --2- 172.21.15.70:0/2655283830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffbe006cec0 0x7ffbe015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.022+0000 7ffbdffff640 1 -- 172.21.15.70:0/2655283830 >> 172.21.15.70:0/2655283830 conn(0x7ffbe00cede0 msgr2=0x7ffbe00b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:49.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.022+0000 7ffbdffff640 1 -- 172.21.15.70:0/2655283830 shutdown_connections 2023-10-08T05:52:49.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.022+0000 7ffbdffff640 1 -- 172.21.15.70:0/2655283830 wait complete. 2023-10-08T05:52:49.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.249+0000 7f7047fff640 1 Processor -- start 2023-10-08T05:52:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.249+0000 7f7047fff640 1 -- start start 2023-10-08T05:52:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.251+0000 7f7047fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7048157480 0x7f704806af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.251+0000 7f7047fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f704815dc60 0x7f704806b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.251+0000 7f7047fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480cde30 con 0x7f70480ce2c0 2023-10-08T05:52:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.251+0000 7f7047fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704815e070 con 0x7f7048157480 2023-10-08T05:52:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.251+0000 7f7047fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7048157860 con 0x7f704815dc60 2023-10-08T05:52:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.251+0000 7f7046ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7048157480 0x7f704806af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.252+0000 7f7046ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7048157480 0x7f704806af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46358/0 (socket says 172.21.15.70:46358) 2023-10-08T05:52:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.252+0000 7f7046ffd640 1 -- 172.21.15.70:0/3771767799 learned_addr learned my addr 172.21.15.70:0/3771767799 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:49.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.252+0000 7f7046ffd640 1 -- 172.21.15.70:0/3771767799 >> v2:172.21.15.181:3300/0 conn(0x7f704815dc60 msgr2=0x7f704806b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:49.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.252+0000 7f7046ffd640 1 --2- 172.21.15.70:0/3771767799 >> v2:172.21.15.181:3300/0 conn(0x7f704815dc60 0x7f704806b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.252+0000 7f7046ffd640 1 -- 172.21.15.70:0/3771767799 >> v1:172.21.15.70:6789/0 conn(0x7f70480ce2c0 legacy=0x7f704815d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.252+0000 7f7046ffd640 1 -- 172.21.15.70:0/3771767799 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704815e2c0 con 0x7f7048157480 2023-10-08T05:52:49.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.253+0000 7f7046ffd640 1 --2- 172.21.15.70:0/3771767799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 0x7f704806af70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7048160930 tx=0x7f703802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1999271445452556 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:49.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.254+0000 7f70467fc640 1 -- 172.21.15.70:0/3771767799 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038031870 con 0x7f7048157480 2023-10-08T05:52:49.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.254+0000 7f70467fc640 1 -- 172.21.15.70:0/3771767799 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70380319d0 con 0x7f7048157480 2023-10-08T05:52:49.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.254+0000 7f70467fc640 1 -- 172.21.15.70:0/3771767799 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038031d60 con 0x7f7048157480 2023-10-08T05:52:49.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.254+0000 7f7047fff640 1 -- 172.21.15.70:0/3771767799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 msgr2=0x7f704806af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.254+0000 7f7047fff640 1 --2- 172.21.15.70:0/3771767799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 0x7f704806af70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7048160930 tx=0x7f703802b910 comp rx=0 tx=0).stop 2023-10-08T05:52:49.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.254+0000 7f7047fff640 1 -- 172.21.15.70:0/3771767799 shutdown_connections 2023-10-08T05:52:49.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.254+0000 7f7047fff640 1 --2- 172.21.15.70:0/3771767799 >> v2:172.21.15.181:3300/0 conn(0x7f704815dc60 0x7f704806b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.254+0000 7f7047fff640 1 --2- 172.21.15.70:0/3771767799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 0x7f704806af70 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.254+0000 7f7047fff640 1 -- 172.21.15.70:0/3771767799 >> 172.21.15.70:0/3771767799 conn(0x7f70480d2ae0 msgr2=0x7f70480d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:49.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.254+0000 7f7047fff640 1 -- 172.21.15.70:0/3771767799 shutdown_connections 2023-10-08T05:52:49.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.255+0000 7f7047fff640 1 -- 172.21.15.70:0/3771767799 wait complete. 2023-10-08T05:52:49.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.257+0000 7f7047fff640 1 Processor -- start 2023-10-08T05:52:49.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.257+0000 7f7047fff640 1 -- start start 2023-10-08T05:52:49.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.258+0000 7f7047fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70480ce2c0 0x7f704815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.258+0000 7f7047fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 0x7f70480ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.258+0000 7f7047fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704815dc60 0x7f704814fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.258+0000 7f7047fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048154d20 con 0x7f7048157480 2023-10-08T05:52:49.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.258+0000 7f7047fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70480cafb0 con 0x7f70480ce2c0 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.258+0000 7f7047fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70480cb290 con 0x7f704815dc60 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.258+0000 7f703ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 0x7f70480ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.258+0000 7f703ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 0x7f70480ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46364/0 (socket says 172.21.15.70:46364) 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.258+0000 7f703ffff640 1 -- 172.21.15.70:0/1544060228 learned_addr learned my addr 172.21.15.70:0/1544060228 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.259+0000 7f7046ffd640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70480ce2c0 0x7f704815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.259+0000 7f70477fe640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704815dc60 0x7f704814fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.259+0000 7f703ffff640 1 -- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704815dc60 msgr2=0x7f704814fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.259+0000 7f703ffff640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704815dc60 0x7f704814fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.259+0000 7f703ffff640 1 -- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70480ce2c0 msgr2=0x7f704815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.259+0000 7f703ffff640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70480ce2c0 0x7f704815c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.259+0000 7f703ffff640 1 -- 172.21.15.70:0/1544060228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70380094d0 con 0x7f7048157480 2023-10-08T05:52:49.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.259+0000 7f703ffff640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 0x7f70480ca4f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f703000c400 tx=0x7f703000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.259+0000 7f703f7fe640 1 -- 172.21.15.70:0/1544060228 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70300052d0 con 0x7f7048157480 2023-10-08T05:52:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.260+0000 7f703f7fe640 1 -- 172.21.15.70:0/1544060228 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7030005430 con 0x7f7048157480 2023-10-08T05:52:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.260+0000 7f703f7fe640 1 -- 172.21.15.70:0/1544060228 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7030003830 con 0x7f7048157480 2023-10-08T05:52:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.260+0000 7f7047fff640 1 -- 172.21.15.70:0/1544060228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70480cb570 con 0x7f7048157480 2023-10-08T05:52:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.260+0000 7f7047fff640 1 -- 172.21.15.70:0/1544060228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70480ca250 con 0x7f7048157480 2023-10-08T05:52:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.261+0000 7f7047fff640 1 -- 172.21.15.70:0/1544060228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7048154ed0 con 0x7f7048157480 2023-10-08T05:52:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.262+0000 7f703f7fe640 1 -- 172.21.15.70:0/1544060228 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f7030003990 con 0x7f7048157480 2023-10-08T05:52:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.263+0000 7f703f7fe640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f702806af30 0x7f702806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.263+0000 7f703f7fe640 1 -- 172.21.15.70:0/1544060228 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f7030088fd0 con 0x7f7048157480 2023-10-08T05:52:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.263+0000 7f7046ffd640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f702806af30 0x7f702806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.265+0000 7f7046ffd640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f702806af30 0x7f702806d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7038036000 tx=0x7f703803c040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:49.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.267+0000 7f703f7fe640 1 -- 172.21.15.70:0/1544060228 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7048154ed0 con 0x7f7048157480 2023-10-08T05:52:49.491 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:49.489+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:49.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.503+0000 7f7047fff640 1 -- 172.21.15.70:0/1544060228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f704804e160 con 0x7f7048157480 2023-10-08T05:52:49.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.503+0000 7f703f7fe640 1 -- 172.21.15.70:0/1544060228 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f7030053c00 con 0x7f7048157480 2023-10-08T05:52:49.504 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:49.504 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1038137327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1038137327},{"type":"v1","addr":"172.21.15.181:6835","nonce":1038137327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6660,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1967783419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1967783419},{"type":"v1","addr":"172.21.15.70:6833","nonce":1967783419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6663,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1304760341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1304760341},{"type":"v1","addr":"172.21.15.70:6835","nonce":1304760341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:52:26.053790+0000","modified":"2023-10-08T05:52:29.103940+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6675},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6675":{"gid":6675,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6837/2348955257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2348955257},{"type":"v1","addr":"172.21.15.181:6837","nonce":2348955257}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-10-08T05:52:49.504 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 33 2023-10-08T05:52:49.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 -- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f702806af30 msgr2=0x7f702806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f702806af30 0x7f702806d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7038036000 tx=0x7f703803c040 comp rx=0 tx=0).stop 2023-10-08T05:52:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 -- 172.21.15.70:0/1544060228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 msgr2=0x7f70480ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 0x7f70480ca4f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f703000c400 tx=0x7f703000c8c0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 -- 172.21.15.70:0/1544060228 shutdown_connections 2023-10-08T05:52:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f702806af30 0x7f702806d3f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704815dc60 0x7f704814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7048157480 0x7f70480ca4f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 --2- 172.21.15.70:0/1544060228 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70480ce2c0 0x7f704815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 -- 172.21.15.70:0/1544060228 >> 172.21.15.70:0/1544060228 conn(0x7f70480d2ae0 msgr2=0x7f70480b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 -- 172.21.15.70:0/1544060228 shutdown_connections 2023-10-08T05:52:49.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.507+0000 7f7047fff640 1 -- 172.21.15.70:0/1544060228 wait complete. 2023-10-08T05:52:49.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:49.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.718+0000 7f655fb6c640 1 Processor -- start 2023-10-08T05:52:49.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.718+0000 7f655fb6c640 1 -- start start 2023-10-08T05:52:49.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.720+0000 7f655fb6c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580ad790 0x7f65580702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.720+0000 7f655fb6c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f655815e330 0x7f6558070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.720+0000 7f655fb6c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655806dee0 con 0x7f6558159ac0 2023-10-08T05:52:49.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.720+0000 7f655fb6c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655806dcf0 con 0x7f655815e330 2023-10-08T05:52:49.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.720+0000 7f655fb6c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655815d850 con 0x7f65580ad790 2023-10-08T05:52:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.720+0000 7f655d908640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6558159ac0 0x7f655806d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58530/0 (socket says 172.21.15.70:58530) 2023-10-08T05:52:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.720+0000 7f655d908640 1 -- 172.21.15.70:0/4113903788 learned_addr learned my addr 172.21.15.70:0/4113903788 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655e109640 1 --2- 172.21.15.70:0/4113903788 >> v2:172.21.15.181:3300/0 conn(0x7f655815e330 0x7f6558070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389819796 0 0) 0x7f655806dee0 con 0x7f6558159ac0 2023-10-08T05:52:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6544003520 con 0x7f6558159ac0 2023-10-08T05:52:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655d107640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580ad790 0x7f65580702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1775612311 0 0) 0x7f6544003520 con 0x7f6558159ac0 2023-10-08T05:52:49.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580ad790 msgr2=0x7f65580702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:49.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580ad790 0x7f65580702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 >> v2:172.21.15.181:3300/0 conn(0x7f655815e330 msgr2=0x7f6558070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:49.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 --2- 172.21.15.70:0/4113903788 >> v2:172.21.15.181:3300/0 conn(0x7f655815e330 0x7f6558070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655815db30 con 0x7f6558159ac0 2023-10-08T05:52:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6554001670 con 0x7f6558159ac0 2023-10-08T05:52:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 >> v1:172.21.15.70:6789/0 conn(0x7f6558159ac0 legacy=0x7f655806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580ad790 0x7f6544003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6544003fa0 0x7f65440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6544004d00 0x7f6544005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655815d850 con 0x7f6544004d00 2023-10-08T05:52:49.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654400dc40 con 0x7f6544003fa0 2023-10-08T05:52:49.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.721+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654400ddd0 con 0x7f65580ad790 2023-10-08T05:52:49.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.722+0000 7f655e109640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6544004d00 0x7f6544005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.722+0000 7f655e109640 1 -- 172.21.15.70:0/4113903788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580ad790 msgr2=0x7f6544003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:49.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.722+0000 7f655d107640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6544003fa0 0x7f65440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.722+0000 7f655e109640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580ad790 0x7f6544003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.722+0000 7f655e109640 1 -- 172.21.15.70:0/4113903788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6544003fa0 msgr2=0x7f65440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:49.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.722+0000 7f655e109640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6544003fa0 0x7f65440047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.722+0000 7f655e109640 1 -- 172.21.15.70:0/4113903788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6544003520 con 0x7f6544004d00 2023-10-08T05:52:49.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.723+0000 7f655e109640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6544004d00 0x7f6544005180 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6550007360 tx=0x7f65440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e49f9db9c6c98b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:49.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.723+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655002ccf0 con 0x7f6544004d00 2023-10-08T05:52:49.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.723+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655002ce50 con 0x7f6544004d00 2023-10-08T05:52:49.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.723+0000 7f655c906640 1 -- 172.21.15.70:0/4113903788 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550034660 con 0x7f6544004d00 2023-10-08T05:52:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.724+0000 7f655fb6c640 1 -- 172.21.15.70:0/4113903788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6544004d00 msgr2=0x7f6544005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.724+0000 7f655fb6c640 1 --2- 172.21.15.70:0/4113903788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6544004d00 0x7f6544005180 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f6550007360 tx=0x7f65440057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.724+0000 7f655e109640 1 -- 172.21.15.70:0/4113903788 reap_dead start 2023-10-08T05:52:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.724+0000 7f655fb6c640 1 -- 172.21.15.70:0/4113903788 shutdown_connections 2023-10-08T05:52:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.724+0000 7f655fb6c640 1 -- 172.21.15.70:0/4113903788 >> 172.21.15.70:0/4113903788 conn(0x7f655806a380 msgr2=0x7f65580b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.724+0000 7f655fb6c640 1 -- 172.21.15.70:0/4113903788 shutdown_connections 2023-10-08T05:52:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.724+0000 7f655fb6c640 1 -- 172.21.15.70:0/4113903788 wait complete. 2023-10-08T05:52:49.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.726+0000 7f655fb6c640 1 Processor -- start 2023-10-08T05:52:49.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.726+0000 7f655fb6c640 1 -- start start 2023-10-08T05:52:49.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655fb6c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6544004d00 0x7f655815ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655fb6c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65581f3220 0x7f6558158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655fb6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6558158aa0 0x7f65581f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655fb6c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65581f2550 con 0x7f6558158aa0 2023-10-08T05:52:49.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655fb6c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6558158f20 con 0x7f6544004d00 2023-10-08T05:52:49.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655fb6c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65581f2d90 con 0x7f65581f3220 2023-10-08T05:52:49.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655e109640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6558158aa0 0x7f65581f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655e109640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6558158aa0 0x7f65581f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46382/0 (socket says 172.21.15.70:46382) 2023-10-08T05:52:49.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655e109640 1 -- 172.21.15.70:0/2408236519 learned_addr learned my addr 172.21.15.70:0/2408236519 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:49.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655e109640 1 -- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65581f3220 msgr2=0x7f6558158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655e109640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65581f3220 0x7f6558158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655e109640 1 -- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6544004d00 msgr2=0x7f655815ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655e109640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6544004d00 0x7f655815ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.727+0000 7f655e109640 1 -- 172.21.15.70:0/2408236519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65581f6870 con 0x7f6558158aa0 2023-10-08T05:52:49.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.728+0000 7f655e109640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6558158aa0 0x7f65581f5b40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f655002dfd0 tx=0x7f6550004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.728+0000 7f654e7fc640 1 -- 172.21.15.70:0/2408236519 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550003b90 con 0x7f6558158aa0 2023-10-08T05:52:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.728+0000 7f654e7fc640 1 -- 172.21.15.70:0/2408236519 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6550003d30 con 0x7f6558158aa0 2023-10-08T05:52:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.728+0000 7f654e7fc640 1 -- 172.21.15.70:0/2408236519 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550003ea0 con 0x7f6558158aa0 2023-10-08T05:52:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.728+0000 7f655fb6c640 1 -- 172.21.15.70:0/2408236519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65581f2740 con 0x7f6558158aa0 2023-10-08T05:52:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.728+0000 7f655fb6c640 1 -- 172.21.15.70:0/2408236519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655815cf50 con 0x7f6558158aa0 2023-10-08T05:52:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.729+0000 7f655fb6c640 1 -- 172.21.15.70:0/2408236519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65580c40c0 con 0x7f6558158aa0 2023-10-08T05:52:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.730+0000 7f654e7fc640 1 -- 172.21.15.70:0/2408236519 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f655003b040 con 0x7f6558158aa0 2023-10-08T05:52:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.731+0000 7f654e7fc640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f653c06af00 0x7f653c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.731+0000 7f654e7fc640 1 -- 172.21.15.70:0/2408236519 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7f65500aaef0 con 0x7f6558158aa0 2023-10-08T05:52:49.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.732+0000 7f655d908640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f653c06af00 0x7f653c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:49.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.733+0000 7f655d908640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f653c06af00 0x7f653c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65580cbcb0 tx=0x7f65540014d0 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:49.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.735+0000 7f654e7fc640 1 -- 172.21.15.70:0/2408236519 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6550075b90 con 0x7f6558158aa0 2023-10-08T05:52:49.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.914+0000 7f655fb6c640 1 -- 172.21.15.70:0/2408236519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65581f2f00 con 0x7f6558158aa0 2023-10-08T05:52:49.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.917+0000 7f654e7fc640 1 -- 172.21.15.70:0/2408236519 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17791 (secure 0 0 0) 0x7f6550045020 con 0x7f6558158aa0 2023-10-08T05:52:49.918 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:49.918 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":92,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:46.566144+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:52:23.846215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-08T05:52:25.069536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3148780950":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/999647594":"2023-10-09T05:52:46.566070+0000","172.21.15.181:6833/1709912921":"2023-10-09T05:52:46.566070+0000","172.21.15.181:6832/1709912921":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/3306361152":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/2398424071":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/814792959":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1860309542":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/2758467223":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1699086184":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1378633292":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1217040450":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/3712172314":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/2333154107":"2023-10-09T05:52:40.392573+0000","172.21.15.181:6833/3994892546":"2023-10-09T05:52:40.392573+0000","172.21.15.181:6832/3994892546":"2023-10-09T05:52:40.392573+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:49.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.919+0000 7f655fb6c640 1 -- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f653c06af00 msgr2=0x7f653c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.919+0000 7f655fb6c640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f653c06af00 0x7f653c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65580cbcb0 tx=0x7f65540014d0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.919+0000 7f655fb6c640 1 -- 172.21.15.70:0/2408236519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6558158aa0 msgr2=0x7f65581f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:49.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.919+0000 7f655fb6c640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6558158aa0 0x7f65581f5b40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f655002dfd0 tx=0x7f6550004ab0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.920+0000 7f655fb6c640 1 -- 172.21.15.70:0/2408236519 shutdown_connections 2023-10-08T05:52:49.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.920+0000 7f655fb6c640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f653c06af00 0x7f653c06d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.920+0000 7f655fb6c640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6558158aa0 0x7f65581f5b40 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.920+0000 7f655fb6c640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65581f3220 0x7f6558158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.920+0000 7f655fb6c640 1 --2- 172.21.15.70:0/2408236519 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6544004d00 0x7f655815ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:49.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.920+0000 7f655fb6c640 1 -- 172.21.15.70:0/2408236519 >> 172.21.15.70:0/2408236519 conn(0x7f655806a380 msgr2=0x7f65580c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:49.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.920+0000 7f655fb6c640 1 -- 172.21.15.70:0/2408236519 shutdown_connections 2023-10-08T05:52:49.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:49.920+0000 7f655fb6c640 1 -- 172.21.15.70:0/2408236519 wait complete. 2023-10-08T05:52:49.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T05:52:50.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.143+0000 7ff2d88fc640 1 Processor -- start 2023-10-08T05:52:50.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.144+0000 7ff2d88fc640 1 -- start start 2023-10-08T05:52:50.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.145+0000 7ff2d88fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff2d415d0c0 0x7ff2d40b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.145+0000 7ff2d88fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff2d40b30d0 0x7ff2d40b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.145+0000 7ff2d88fc640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d406ce20 con 0x7ff2d415cce0 2023-10-08T05:52:50.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.145+0000 7ff2d88fc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d40cf850 con 0x7ff2d40b30d0 2023-10-08T05:52:50.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.145+0000 7ff2d88fc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d406c060 con 0x7ff2d415d0c0 2023-10-08T05:52:50.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.145+0000 7ff2d2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff2d40b30d0 0x7ff2d40b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:50.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.145+0000 7ff2d2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff2d40b30d0 0x7ff2d40b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46388/0 (socket says 172.21.15.70:46388) 2023-10-08T05:52:50.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.145+0000 7ff2d2d9d640 1 -- 172.21.15.70:0/2474708382 learned_addr learned my addr 172.21.15.70:0/2474708382 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:50.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.146+0000 7ff2cbfff640 1 --2- 172.21.15.70:0/2474708382 >> v2:172.21.15.181:3300/0 conn(0x7ff2d415d0c0 0x7ff2d40b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:50.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.146+0000 7ff2d2d9d640 1 -- 172.21.15.70:0/2474708382 >> v2:172.21.15.181:3300/0 conn(0x7ff2d415d0c0 msgr2=0x7ff2d40b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:50.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.146+0000 7ff2d2d9d640 1 --2- 172.21.15.70:0/2474708382 >> v2:172.21.15.181:3300/0 conn(0x7ff2d415d0c0 0x7ff2d40b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.146+0000 7ff2d2d9d640 1 -- 172.21.15.70:0/2474708382 >> v1:172.21.15.181:6789/0 conn(0x7ff2d415cce0 legacy=0x7ff2d40cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:50.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.146+0000 7ff2d2d9d640 1 -- 172.21.15.70:0/2474708382 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d406c340 con 0x7ff2d40b30d0 2023-10-08T05:52:50.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.146+0000 7ff2cbfff640 1 --2- 172.21.15.70:0/2474708382 >> v2:172.21.15.181:3300/0 conn(0x7ff2d415d0c0 0x7ff2d40b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:50.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.147+0000 7ff2d2d9d640 1 --2- 172.21.15.70:0/2474708382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d40b30d0 0x7ff2d40b2a70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40c4540 tx=0x7ff2cc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563f9253764265b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:50.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.147+0000 7ff2d1d9b640 1 -- 172.21.15.70:0/2474708382 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc02ca80 con 0x7ff2d40b30d0 2023-10-08T05:52:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.147+0000 7ff2d1d9b640 1 -- 172.21.15.70:0/2474708382 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc02cbe0 con 0x7ff2d40b30d0 2023-10-08T05:52:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.147+0000 7ff2d1d9b640 1 -- 172.21.15.70:0/2474708382 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0343f0 con 0x7ff2d40b30d0 2023-10-08T05:52:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.148+0000 7ff2d88fc640 1 -- 172.21.15.70:0/2474708382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d40b30d0 msgr2=0x7ff2d40b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.148+0000 7ff2d88fc640 1 --2- 172.21.15.70:0/2474708382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d40b30d0 0x7ff2d40b2a70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40c4540 tx=0x7ff2cc02bee0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.148+0000 7ff2d88fc640 1 -- 172.21.15.70:0/2474708382 shutdown_connections 2023-10-08T05:52:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.148+0000 7ff2d88fc640 1 --2- 172.21.15.70:0/2474708382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d40b30d0 0x7ff2d40b2a70 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.148+0000 7ff2d88fc640 1 --2- 172.21.15.70:0/2474708382 >> v2:172.21.15.181:3300/0 conn(0x7ff2d415d0c0 0x7ff2d40b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.148+0000 7ff2d88fc640 1 -- 172.21.15.70:0/2474708382 >> 172.21.15.70:0/2474708382 conn(0x7ff2d406e650 msgr2=0x7ff2d406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.148+0000 7ff2d88fc640 1 -- 172.21.15.70:0/2474708382 shutdown_connections 2023-10-08T05:52:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.148+0000 7ff2d88fc640 1 -- 172.21.15.70:0/2474708382 wait complete. 2023-10-08T05:52:50.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.150+0000 7ff2d88fc640 1 Processor -- start 2023-10-08T05:52:50.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.150+0000 7ff2d88fc640 1 -- start start 2023-10-08T05:52:50.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.151+0000 7ff2d88fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2d40b30d0 0x7ff2d40b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.151+0000 7ff2d88fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d415cce0 0x7ff2d40b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.151+0000 7ff2d88fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d415f2c0 0x7ff2d415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.151+0000 7ff2d88fc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40b4190 con 0x7ff2d415cce0 2023-10-08T05:52:50.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.151+0000 7ff2d88fc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d415e6c0 con 0x7ff2d415f2c0 2023-10-08T05:52:50.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.151+0000 7ff2d88fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40c4740 con 0x7ff2d40b30d0 2023-10-08T05:52:50.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2cbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d415cce0 0x7ff2d40b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:50.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2cbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d415cce0 0x7ff2d40b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46392/0 (socket says 172.21.15.70:46392) 2023-10-08T05:52:50.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2cbfff640 1 -- 172.21.15.70:0/3739595507 learned_addr learned my addr 172.21.15.70:0/3739595507 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:50.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2d259c640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2d40b30d0 0x7ff2d40b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:50.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2d2d9d640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d415f2c0 0x7ff2d415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2cbfff640 1 -- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2d40b30d0 msgr2=0x7ff2d40b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2cbfff640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2d40b30d0 0x7ff2d40b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2cbfff640 1 -- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d415f2c0 msgr2=0x7ff2d415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2cbfff640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d415f2c0 0x7ff2d415d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2cbfff640 1 -- 172.21.15.70:0/3739595507 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc009b30 con 0x7ff2d415cce0 2023-10-08T05:52:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2d2d9d640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d415f2c0 0x7ff2d415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.152+0000 7ff2cbfff640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d415cce0 0x7ff2d40b0730 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00c820 tx=0x7ff2bc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.153+0000 7ff2caffd640 1 -- 172.21.15.70:0/3739595507 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc007a30 con 0x7ff2d415cce0 2023-10-08T05:52:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.153+0000 7ff2caffd640 1 -- 172.21.15.70:0/3739595507 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc007b90 con 0x7ff2d415cce0 2023-10-08T05:52:50.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.153+0000 7ff2d88fc640 1 -- 172.21.15.70:0/3739595507 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d415de20 con 0x7ff2d415cce0 2023-10-08T05:52:50.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.153+0000 7ff2caffd640 1 -- 172.21.15.70:0/3739595507 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc00ce10 con 0x7ff2d415cce0 2023-10-08T05:52:50.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.153+0000 7ff2d88fc640 1 -- 172.21.15.70:0/3739595507 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d415e3e0 con 0x7ff2d415cce0 2023-10-08T05:52:50.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.154+0000 7ff2d88fc640 1 -- 172.21.15.70:0/3739595507 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d406c8e0 con 0x7ff2d415cce0 2023-10-08T05:52:50.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.155+0000 7ff2caffd640 1 -- 172.21.15.70:0/3739595507 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ff2bc0113f0 con 0x7ff2d415cce0 2023-10-08T05:52:50.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.156+0000 7ff2caffd640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff2b406af00 0x7ff2b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.156+0000 7ff2caffd640 1 -- 172.21.15.70:0/3739595507 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6761+0+0 (secure 0 0 0) 0x7ff2bc089c20 con 0x7ff2d415cce0 2023-10-08T05:52:50.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.157+0000 7ff2d259c640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff2b406af00 0x7ff2b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:50.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.158+0000 7ff2d259c640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff2b406af00 0x7ff2b406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2d415c5b0 tx=0x7ff2c4009040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:50.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.160+0000 7ff2caffd640 1 -- 172.21.15.70:0/3739595507 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2d406c8e0 con 0x7ff2d415cce0 2023-10-08T05:52:50.360 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:50.357+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:50.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.395+0000 7ff2d88fc640 1 -- 172.21.15.70:0/3739595507 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff2d40c45a0 con 0x7ff2d415cce0 2023-10-08T05:52:50.396 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T05:52:50.396+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T05:52:50.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.400+0000 7ff2caffd640 1 -- 172.21.15.70:0/3739595507 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff2d40c45a0 con 0x7ff2d415cce0 2023-10-08T05:52:50.400 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.403+0000 7ff2d88fc640 1 -- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff2b406af00 msgr2=0x7ff2b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.403+0000 7ff2d88fc640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff2b406af00 0x7ff2b406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2d415c5b0 tx=0x7ff2c4009040 comp rx=0 tx=0).stop 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.403+0000 7ff2d88fc640 1 -- 172.21.15.70:0/3739595507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d415cce0 msgr2=0x7ff2d40b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.403+0000 7ff2d88fc640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d415cce0 0x7ff2d40b0730 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00c820 tx=0x7ff2bc00cce0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.403+0000 7ff2d88fc640 1 -- 172.21.15.70:0/3739595507 shutdown_connections 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.403+0000 7ff2d88fc640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff2b406af00 0x7ff2b406d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.403+0000 7ff2d88fc640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2d415f2c0 0x7ff2d415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.403+0000 7ff2d88fc640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2d415cce0 0x7ff2d40b0730 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.403+0000 7ff2d88fc640 1 --2- 172.21.15.70:0/3739595507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2d40b30d0 0x7ff2d40b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.403+0000 7ff2d88fc640 1 -- 172.21.15.70:0/3739595507 >> 172.21.15.70:0/3739595507 conn(0x7ff2d406e650 msgr2=0x7ff2d40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:50.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.404+0000 7ff2d88fc640 1 -- 172.21.15.70:0/3739595507 shutdown_connections 2023-10-08T05:52:50.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.404+0000 7ff2d88fc640 1 -- 172.21.15.70:0/3739595507 wait complete. 2023-10-08T05:52:50.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T05:52:50.430 INFO:tasks.ceph.mds.b.smithi181.stderr: -3723> 2023-10-08T05:52:27.547+0000 7f9731f38640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:52:50.526 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:52:50.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.636+0000 7fc859327640 1 Processor -- start 2023-10-08T05:52:50.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.636+0000 7fc859327640 1 -- start start 2023-10-08T05:52:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.638+0000 7fc859327640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8540cb5e0 0x7fc8540cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.638+0000 7fc859327640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc854159db0 con 0x7fc8541599d0 2023-10-08T05:52:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.638+0000 7fc859327640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc854071fe0 con 0x7fc8540b3c50 2023-10-08T05:52:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.638+0000 7fc859327640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8540b44c0 con 0x7fc8540cb5e0 2023-10-08T05:52:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.638+0000 7fc853fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc8541599d0 0x7fc8540caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58546/0 (socket says 172.21.15.70:58546) 2023-10-08T05:52:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.638+0000 7fc853fff640 1 -- 172.21.15.70:0/215489636 learned_addr learned my addr 172.21.15.70:0/215489636 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.638+0000 7fc84b7fe640 1 --2- 172.21.15.70:0/215489636 >> v2:172.21.15.70:3300/0 conn(0x7fc8540cb5e0 0x7fc8540cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.638+0000 7fc8537fe640 1 -- 172.21.15.70:0/215489636 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560336567 0 0) 0x7fc844001110 con 0x7fc8541599d0 2023-10-08T05:52:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.638+0000 7fc8537fe640 1 -- 172.21.15.70:0/215489636 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc840003520 con 0x7fc8541599d0 2023-10-08T05:52:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.639+0000 7fc84b7fe640 1 -- 172.21.15.70:0/215489636 >> v1:172.21.15.181:6789/0 conn(0x7fc8540b3c50 legacy=0x7fc8540cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:50.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.639+0000 7fc84b7fe640 1 -- 172.21.15.70:0/215489636 >> v1:172.21.15.70:6789/0 conn(0x7fc8541599d0 legacy=0x7fc8540caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:50.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.639+0000 7fc84b7fe640 1 -- 172.21.15.70:0/215489636 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8540713d0 con 0x7fc8540cb5e0 2023-10-08T05:52:50.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.640+0000 7fc84b7fe640 1 --2- 172.21.15.70:0/215489636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8540cb5e0 0x7fc8540cdcd0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0097a0 tx=0x7fc83c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8980523269677e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:50.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.640+0000 7fc8537fe640 1 -- 172.21.15.70:0/215489636 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c031830 con 0x7fc8540cb5e0 2023-10-08T05:52:50.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.640+0000 7fc8537fe640 1 -- 172.21.15.70:0/215489636 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c031990 con 0x7fc8540cb5e0 2023-10-08T05:52:50.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.640+0000 7fc8537fe640 1 -- 172.21.15.70:0/215489636 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c031d20 con 0x7fc8540cb5e0 2023-10-08T05:52:50.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.641+0000 7fc859327640 1 -- 172.21.15.70:0/215489636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8540cb5e0 msgr2=0x7fc8540cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:50.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.641+0000 7fc859327640 1 --2- 172.21.15.70:0/215489636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8540cb5e0 0x7fc8540cdcd0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0097a0 tx=0x7fc83c02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.641+0000 7fc859327640 1 -- 172.21.15.70:0/215489636 shutdown_connections 2023-10-08T05:52:50.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.641+0000 7fc859327640 1 --2- 172.21.15.70:0/215489636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8540cb5e0 0x7fc8540cdcd0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.641+0000 7fc859327640 1 -- 172.21.15.70:0/215489636 >> 172.21.15.70:0/215489636 conn(0x7fc85406c050 msgr2=0x7fc85406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:50.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.641+0000 7fc859327640 1 -- 172.21.15.70:0/215489636 shutdown_connections 2023-10-08T05:52:50.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.641+0000 7fc859327640 1 -- 172.21.15.70:0/215489636 wait complete. 2023-10-08T05:52:50.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.643+0000 7fc859327640 1 Processor -- start 2023-10-08T05:52:50.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.643+0000 7fc859327640 1 -- start start 2023-10-08T05:52:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc859327640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8540b3c50 0x7fc8540c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc859327640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8540cb5e0 0x7fc8541ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc859327640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8541599d0 0x7fc854157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc859327640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540c67b0 con 0x7fc8541599d0 2023-10-08T05:52:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc859327640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540c6a90 con 0x7fc8540cb5e0 2023-10-08T05:52:50.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc859327640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540c6d70 con 0x7fc8540b3c50 2023-10-08T05:52:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc858b26640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8541599d0 0x7fc854157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc858b26640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8541599d0 0x7fc854157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46424/0 (socket says 172.21.15.70:46424) 2023-10-08T05:52:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc858b26640 1 -- 172.21.15.70:0/2141340679 learned_addr learned my addr 172.21.15.70:0/2141340679 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc858b26640 1 -- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8540b3c50 msgr2=0x7fc8540c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc853fff640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8540b3c50 0x7fc8540c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc858b26640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8540b3c50 0x7fc8540c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc858b26640 1 -- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8540cb5e0 msgr2=0x7fc8541ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc858b26640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8540cb5e0 0x7fc8541ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.644+0000 7fc858b26640 1 -- 172.21.15.70:0/2141340679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83c009400 con 0x7fc8541599d0 2023-10-08T05:52:50.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.645+0000 7fc858b26640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8541599d0 0x7fc854157870 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00c780 tx=0x7fc84c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.645+0000 7fc8517fa640 1 -- 172.21.15.70:0/2141340679 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c004b60 con 0x7fc8541599d0 2023-10-08T05:52:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.645+0000 7fc8517fa640 1 -- 172.21.15.70:0/2141340679 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c00ce90 con 0x7fc8541599d0 2023-10-08T05:52:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.645+0000 7fc8517fa640 1 -- 172.21.15.70:0/2141340679 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c00a850 con 0x7fc8541599d0 2023-10-08T05:52:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.645+0000 7fc859327640 1 -- 172.21.15.70:0/2141340679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8540c7050 con 0x7fc8541599d0 2023-10-08T05:52:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.645+0000 7fc859327640 1 -- 172.21.15.70:0/2141340679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8540c7d90 con 0x7fc8541599d0 2023-10-08T05:52:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.646+0000 7fc859327640 1 -- 172.21.15.70:0/2141340679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8540c7570 con 0x7fc8541599d0 2023-10-08T05:52:50.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.647+0000 7fc8517fa640 1 -- 172.21.15.70:0/2141340679 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc84c00a9b0 con 0x7fc8541599d0 2023-10-08T05:52:50.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.648+0000 7fc8517fa640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc83406af00 0x7fc83406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:50.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.648+0000 7fc8517fa640 1 -- 172.21.15.70:0/2141340679 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fc84c089bd0 con 0x7fc8541599d0 2023-10-08T05:52:50.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.649+0000 7fc853fff640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc83406af00 0x7fc83406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:50.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.650+0000 7fc853fff640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc83406af00 0x7fc83406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc844001370 tx=0x7fc844009040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:50.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.653+0000 7fc8517fa640 1 -- 172.21.15.70:0/2141340679 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8540c7570 con 0x7fc8541599d0 2023-10-08T05:52:50.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:50.895+0000 7fc859327640 1 -- 172.21.15.70:0/2141340679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc8540c7570 con 0x7fc8541599d0 2023-10-08T05:52:51.235 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:51.233+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:51.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.403+0000 7fc8517fa640 1 -- 172.21.15.70:0/2141340679 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc8540c7570 con 0x7fc8541599d0 2023-10-08T05:52:51.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 -- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc83406af00 msgr2=0x7fc83406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:51.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc83406af00 0x7fc83406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc844001370 tx=0x7fc844009040 comp rx=0 tx=0).stop 2023-10-08T05:52:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 -- 172.21.15.70:0/2141340679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8541599d0 msgr2=0x7fc854157870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8541599d0 0x7fc854157870 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00c780 tx=0x7fc84c00cc40 comp rx=0 tx=0).stop 2023-10-08T05:52:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 -- 172.21.15.70:0/2141340679 shutdown_connections 2023-10-08T05:52:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc83406af00 0x7fc83406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8541599d0 0x7fc854157870 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8540cb5e0 0x7fc8541ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 --2- 172.21.15.70:0/2141340679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8540b3c50 0x7fc8540c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 -- 172.21.15.70:0/2141340679 >> 172.21.15.70:0/2141340679 conn(0x7fc85406c050 msgr2=0x7fc8540cadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 -- 172.21.15.70:0/2141340679 shutdown_connections 2023-10-08T05:52:51.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.407+0000 7fc859327640 1 -- 172.21.15.70:0/2141340679 wait complete. 2023-10-08T05:52:51.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T05:52:51.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.647+0000 7fa8774d4640 1 Processor -- start 2023-10-08T05:52:51.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.647+0000 7fa8774d4640 1 -- start start 2023-10-08T05:52:51.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.648+0000 7fa8774d4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8700afbe0 0x7fa8700b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:51.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.648+0000 7fa8774d4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa8700cb130 0x7fa8700cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:51.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.648+0000 7fa8774d4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8700cbad0 con 0x7fa87006dfc0 2023-10-08T05:52:51.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.648+0000 7fa8774d4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700b2620 con 0x7fa8700afbe0 2023-10-08T05:52:51.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.648+0000 7fa8774d4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8700b2060 con 0x7fa8700cb130 2023-10-08T05:52:51.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.649+0000 7fa875270640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8700afbe0 0x7fa8700b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:51.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.649+0000 7fa874a6f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa87006dfc0 0x7fa870070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58558/0 (socket says 172.21.15.70:58558) 2023-10-08T05:52:51.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.649+0000 7fa874a6f640 1 -- 172.21.15.70:0/3246154194 learned_addr learned my addr 172.21.15.70:0/3246154194 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:51.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.649+0000 7fa86ffff640 1 -- 172.21.15.70:0/3246154194 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178581115 0 0) 0x7fa8700cbad0 con 0x7fa87006dfc0 2023-10-08T05:52:51.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.649+0000 7fa86ffff640 1 -- 172.21.15.70:0/3246154194 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa858003520 con 0x7fa87006dfc0 2023-10-08T05:52:51.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.649+0000 7fa875a71640 1 --2- 172.21.15.70:0/3246154194 >> v2:172.21.15.181:3300/0 conn(0x7fa8700cb130 0x7fa8700cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:51.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.649+0000 7fa875270640 1 -- 172.21.15.70:0/3246154194 >> v2:172.21.15.181:3300/0 conn(0x7fa8700cb130 msgr2=0x7fa8700cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:51.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.649+0000 7fa875270640 1 --2- 172.21.15.70:0/3246154194 >> v2:172.21.15.181:3300/0 conn(0x7fa8700cb130 0x7fa8700cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:51.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.649+0000 7fa875270640 1 -- 172.21.15.70:0/3246154194 >> v1:172.21.15.70:6789/0 conn(0x7fa87006dfc0 legacy=0x7fa870070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:51.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.649+0000 7fa875270640 1 -- 172.21.15.70:0/3246154194 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700b2340 con 0x7fa8700afbe0 2023-10-08T05:52:51.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.651+0000 7fa875270640 1 --2- 172.21.15.70:0/3246154194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 0x7fa8700b15b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa8700c8d10 tx=0x7fa86002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53621f8650804c82 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:51.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.651+0000 7fa86ffff640 1 -- 172.21.15.70:0/3246154194 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860031830 con 0x7fa8700afbe0 2023-10-08T05:52:51.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.651+0000 7fa86ffff640 1 -- 172.21.15.70:0/3246154194 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860031990 con 0x7fa8700afbe0 2023-10-08T05:52:51.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.651+0000 7fa86ffff640 1 -- 172.21.15.70:0/3246154194 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860031d20 con 0x7fa8700afbe0 2023-10-08T05:52:51.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.652+0000 7fa8774d4640 1 -- 172.21.15.70:0/3246154194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 msgr2=0x7fa8700b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:51.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.652+0000 7fa8774d4640 1 --2- 172.21.15.70:0/3246154194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 0x7fa8700b15b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa8700c8d10 tx=0x7fa86002b8d0 comp rx=0 tx=0).stop 2023-10-08T05:52:51.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.652+0000 7fa8774d4640 1 -- 172.21.15.70:0/3246154194 shutdown_connections 2023-10-08T05:52:51.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.652+0000 7fa8774d4640 1 --2- 172.21.15.70:0/3246154194 >> v2:172.21.15.181:3300/0 conn(0x7fa8700cb130 0x7fa8700cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:51.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.652+0000 7fa8774d4640 1 --2- 172.21.15.70:0/3246154194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 0x7fa8700b15b0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:51.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.652+0000 7fa8774d4640 1 -- 172.21.15.70:0/3246154194 >> 172.21.15.70:0/3246154194 conn(0x7fa87006a5d0 msgr2=0x7fa8700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:51.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.652+0000 7fa8774d4640 1 -- 172.21.15.70:0/3246154194 shutdown_connections 2023-10-08T05:52:51.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.652+0000 7fa8774d4640 1 -- 172.21.15.70:0/3246154194 wait complete. 2023-10-08T05:52:51.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.654+0000 7fa8774d4640 1 Processor -- start 2023-10-08T05:52:51.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.654+0000 7fa8774d4640 1 -- start start 2023-10-08T05:52:51.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.655+0000 7fa8774d4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa87006dfc0 0x7fa8701f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:51.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.655+0000 7fa8774d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 0x7fa87014dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:51.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.655+0000 7fa8774d4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8700cb130 0x7fa8700ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:51.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.655+0000 7fa8774d4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8701f6ea0 con 0x7fa8700afbe0 2023-10-08T05:52:51.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.655+0000 7fa8774d4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8700ce9f0 con 0x7fa87006dfc0 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.655+0000 7fa8774d4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8701f2ed0 con 0x7fa8700cb130 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.655+0000 7fa874a6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 0x7fa87014dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.655+0000 7fa874a6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 0x7fa87014dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46442/0 (socket says 172.21.15.70:46442) 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.655+0000 7fa874a6f640 1 -- 172.21.15.70:0/4197693589 learned_addr learned my addr 172.21.15.70:0/4197693589 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa875a71640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8700cb130 0x7fa8700ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa875270640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa87006dfc0 0x7fa8701f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa874a6f640 1 -- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8700cb130 msgr2=0x7fa8700ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa874a6f640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8700cb130 0x7fa8700ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa874a6f640 1 -- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa87006dfc0 msgr2=0x7fa8701f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa874a6f640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa87006dfc0 0x7fa8701f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa874a6f640 1 -- 172.21.15.70:0/4197693589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860009400 con 0x7fa8700afbe0 2023-10-08T05:52:51.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa875270640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa87006dfc0 0x7fa8701f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:51.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa875a71640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8700cb130 0x7fa8700ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:51.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa874a6f640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 0x7fa87014dab0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c4d0 tx=0x7fa86400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:51.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa86dffb640 1 -- 172.21.15.70:0/4197693589 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864005ce0 con 0x7fa8700afbe0 2023-10-08T05:52:51.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa86dffb640 1 -- 172.21.15.70:0/4197693589 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa864005e40 con 0x7fa8700afbe0 2023-10-08T05:52:51.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa86dffb640 1 -- 172.21.15.70:0/4197693589 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864007270 con 0x7fa8700afbe0 2023-10-08T05:52:51.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.656+0000 7fa8774d4640 1 -- 172.21.15.70:0/4197693589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8701f31b0 con 0x7fa8700afbe0 2023-10-08T05:52:51.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.657+0000 7fa8774d4640 1 -- 172.21.15.70:0/4197693589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8701f3770 con 0x7fa8700afbe0 2023-10-08T05:52:51.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.658+0000 7fa8774d4640 1 -- 172.21.15.70:0/4197693589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87004e160 con 0x7fa8700afbe0 2023-10-08T05:52:51.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.659+0000 7fa86dffb640 1 -- 172.21.15.70:0/4197693589 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa864007490 con 0x7fa8700afbe0 2023-10-08T05:52:51.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.660+0000 7fa86dffb640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa85c06af00 0x7fa85c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:51.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.660+0000 7fa86dffb640 1 -- 172.21.15.70:0/4197693589 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fa864088f90 con 0x7fa8700afbe0 2023-10-08T05:52:51.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.660+0000 7fa875270640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa85c06af00 0x7fa85c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:51.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.662+0000 7fa875270640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa85c06af00 0x7fa85c06d3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa860036000 tx=0x7fa860008030 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:51.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.664+0000 7fa86dffb640 1 -- 172.21.15.70:0/4197693589 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa864053c90 con 0x7fa8700afbe0 2023-10-08T05:52:51.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:51.866+0000 7fa8774d4640 1 -- 172.21.15.70:0/4197693589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa8700b0ef0 con 0x7fa8700afbe0 2023-10-08T05:52:52.138 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:52.136+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.624+0000 7fa86dffb640 1 -- 172.21.15.70:0/4197693589 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa8640572b0 con 0x7fa8700afbe0 2023-10-08T05:52:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T05:52:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.630+0000 7fa8774d4640 1 -- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa85c06af00 msgr2=0x7fa85c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.630+0000 7fa8774d4640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa85c06af00 0x7fa85c06d3c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa860036000 tx=0x7fa860008030 comp rx=0 tx=0).stop 2023-10-08T05:52:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.630+0000 7fa8774d4640 1 -- 172.21.15.70:0/4197693589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 msgr2=0x7fa87014dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.630+0000 7fa8774d4640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 0x7fa87014dab0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c4d0 tx=0x7fa86400c990 comp rx=0 tx=0).stop 2023-10-08T05:52:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.631+0000 7fa8774d4640 1 -- 172.21.15.70:0/4197693589 shutdown_connections 2023-10-08T05:52:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.631+0000 7fa8774d4640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa85c06af00 0x7fa85c06d3c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.631+0000 7fa8774d4640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8700cb130 0x7fa8700ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.631+0000 7fa8774d4640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8700afbe0 0x7fa87014dab0 secure :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c4d0 tx=0x7fa86400c990 comp rx=0 tx=0).stop 2023-10-08T05:52:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.631+0000 7fa8774d4640 1 --2- 172.21.15.70:0/4197693589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa87006dfc0 0x7fa8701f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.631+0000 7fa8774d4640 1 -- 172.21.15.70:0/4197693589 >> 172.21.15.70:0/4197693589 conn(0x7fa87006a5d0 msgr2=0x7fa8700c91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.632+0000 7fa8774d4640 1 -- 172.21.15.70:0/4197693589 shutdown_connections 2023-10-08T05:52:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.633+0000 7fa8774d4640 1 -- 172.21.15.70:0/4197693589 wait complete. 2023-10-08T05:52:52.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T05:52:52.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.856+0000 7f815f5d0640 1 Processor -- start 2023-10-08T05:52:52.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.856+0000 7f815f5d0640 1 -- start start 2023-10-08T05:52:52.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.858+0000 7f815f5d0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f815806dfc0 0x7f8158070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.858+0000 7f815f5d0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8158071070 0x7f81580cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.858+0000 7f815f5d0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81580b1cc0 con 0x7f81580afbe0 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.858+0000 7f815f5d0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81580b2620 con 0x7f815806dfc0 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.858+0000 7f815f5d0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81580b2060 con 0x7f8158071070 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.858+0000 7f815cb6b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f815806dfc0 0x7f8158070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.858+0000 7f815cb6b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f815806dfc0 0x7f8158070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46456/0 (socket says 172.21.15.70:46456) 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.858+0000 7f815cb6b640 1 -- 172.21.15.70:0/1750906214 learned_addr learned my addr 172.21.15.70:0/1750906214 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.858+0000 7f815db6d640 1 --2- 172.21.15.70:0/1750906214 >> v2:172.21.15.181:3300/0 conn(0x7f8158071070 0x7f81580cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.859+0000 7f815cb6b640 1 -- 172.21.15.70:0/1750906214 >> v2:172.21.15.181:3300/0 conn(0x7f8158071070 msgr2=0x7f81580cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.859+0000 7f815cb6b640 1 --2- 172.21.15.70:0/1750906214 >> v2:172.21.15.181:3300/0 conn(0x7f8158071070 0x7f81580cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:52.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.859+0000 7f815cb6b640 1 -- 172.21.15.70:0/1750906214 >> v1:172.21.15.181:6789/0 conn(0x7f81580afbe0 legacy=0x7f81580b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.859+0000 7f815cb6b640 1 -- 172.21.15.70:0/1750906214 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580b2340 con 0x7f815806dfc0 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.860+0000 7f815cb6b640 1 --2- 172.21.15.70:0/1750906214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 0x7f8158070b30 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f81580c8d10 tx=0x7f814802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5a1849367a1883 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.860+0000 7f814ffff640 1 -- 172.21.15.70:0/1750906214 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148030860 con 0x7f815806dfc0 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.860+0000 7f814ffff640 1 -- 172.21.15.70:0/1750906214 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81480309c0 con 0x7f815806dfc0 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.860+0000 7f814ffff640 1 -- 172.21.15.70:0/1750906214 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148030d50 con 0x7f815806dfc0 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.861+0000 7f815f5d0640 1 -- 172.21.15.70:0/1750906214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 msgr2=0x7f8158070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.861+0000 7f815f5d0640 1 --2- 172.21.15.70:0/1750906214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 0x7f8158070b30 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f81580c8d10 tx=0x7f814802b9a0 comp rx=0 tx=0).stop 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.861+0000 7f815f5d0640 1 -- 172.21.15.70:0/1750906214 shutdown_connections 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.861+0000 7f815f5d0640 1 --2- 172.21.15.70:0/1750906214 >> v2:172.21.15.181:3300/0 conn(0x7f8158071070 0x7f81580cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.861+0000 7f815f5d0640 1 --2- 172.21.15.70:0/1750906214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 0x7f8158070b30 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.861+0000 7f815f5d0640 1 -- 172.21.15.70:0/1750906214 >> 172.21.15.70:0/1750906214 conn(0x7f815806a5d0 msgr2=0x7f81580b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:52.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.861+0000 7f815f5d0640 1 -- 172.21.15.70:0/1750906214 shutdown_connections 2023-10-08T05:52:52.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.861+0000 7f815f5d0640 1 -- 172.21.15.70:0/1750906214 wait complete. 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.863+0000 7f815f5d0640 1 Processor -- start 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.863+0000 7f815f5d0640 1 -- start start 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.864+0000 7f815f5d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 0x7f81580c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.864+0000 7f815f5d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8158071070 0x7f81581f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.864+0000 7f815f5d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81580afbe0 0x7f815814e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.864+0000 7f815f5d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81580ce730 con 0x7f815806dfc0 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.864+0000 7f815f5d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815815c600 con 0x7f81580afbe0 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.864+0000 7f815f5d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815815c8e0 con 0x7f8158071070 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.864+0000 7f815d36c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 0x7f81580c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.864+0000 7f815d36c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 0x7f81580c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46464/0 (socket says 172.21.15.70:46464) 2023-10-08T05:52:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.864+0000 7f815d36c640 1 -- 172.21.15.70:0/969685847 learned_addr learned my addr 172.21.15.70:0/969685847 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f815d36c640 1 -- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8158071070 msgr2=0x7f81581f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f815d36c640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8158071070 0x7f81581f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f815d36c640 1 -- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81580afbe0 msgr2=0x7f815814e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f815db6d640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81580afbe0 0x7f815814e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f815d36c640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81580afbe0 0x7f815814e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f815d36c640 1 -- 172.21.15.70:0/969685847 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81480094d0 con 0x7f815806dfc0 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f815d36c640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 0x7f81580c4ee0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f815400c400 tx=0x7f815400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f814dffb640 1 -- 172.21.15.70:0/969685847 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81540052d0 con 0x7f815806dfc0 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f814dffb640 1 -- 172.21.15.70:0/969685847 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8154005430 con 0x7f815806dfc0 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f814dffb640 1 -- 172.21.15.70:0/969685847 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8154003830 con 0x7f815806dfc0 2023-10-08T05:52:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.865+0000 7f815f5d0640 1 -- 172.21.15.70:0/969685847 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815815cbc0 con 0x7f815806dfc0 2023-10-08T05:52:52.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.866+0000 7f815f5d0640 1 -- 172.21.15.70:0/969685847 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81581f26c0 con 0x7f815806dfc0 2023-10-08T05:52:52.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.867+0000 7f815f5d0640 1 -- 172.21.15.70:0/969685847 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815815d0e0 con 0x7f815806dfc0 2023-10-08T05:52:52.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.868+0000 7f814dffb640 1 -- 172.21.15.70:0/969685847 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8154003990 con 0x7f815806dfc0 2023-10-08T05:52:52.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.869+0000 7f814dffb640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f813c06af00 0x7f813c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:52.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.869+0000 7f814dffb640 1 -- 172.21.15.70:0/969685847 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6409+0+0 (secure 0 0 0) 0x7f8154088fe0 con 0x7f815806dfc0 2023-10-08T05:52:52.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.869+0000 7f815cb6b640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f813c06af00 0x7f813c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:52.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.870+0000 7f815cb6b640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f813c06af00 0x7f813c06d3c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8148036000 tx=0x7f814803b040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:52.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:52.873+0000 7f814dffb640 1 -- 172.21.15.70:0/969685847 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81540bbca0 con 0x7f815806dfc0 2023-10-08T05:52:53.054 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:53.052+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:53.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.070+0000 7f815f5d0640 1 -- 172.21.15.70:0/969685847 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f815804e160 con 0x7f815806dfc0 2023-10-08T05:52:53.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.627+0000 7f814dffb640 1 -- 172.21.15.70:0/969685847 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8154017020 con 0x7f815806dfc0 2023-10-08T05:52:53.628 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T05:52:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.632+0000 7f815f5d0640 1 -- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f813c06af00 msgr2=0x7f813c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.632+0000 7f815f5d0640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f813c06af00 0x7f813c06d3c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8148036000 tx=0x7f814803b040 comp rx=0 tx=0).stop 2023-10-08T05:52:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.632+0000 7f815f5d0640 1 -- 172.21.15.70:0/969685847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 msgr2=0x7f81580c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.632+0000 7f815f5d0640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 0x7f81580c4ee0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f815400c400 tx=0x7f815400c8c0 comp rx=0 tx=0).stop 2023-10-08T05:52:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.633+0000 7f815f5d0640 1 -- 172.21.15.70:0/969685847 shutdown_connections 2023-10-08T05:52:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.633+0000 7f815f5d0640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f813c06af00 0x7f813c06d3c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.633+0000 7f815f5d0640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81580afbe0 0x7f815814e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.633+0000 7f815f5d0640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8158071070 0x7f81581f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.633+0000 7f815f5d0640 1 --2- 172.21.15.70:0/969685847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815806dfc0 0x7f81580c4ee0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.633+0000 7f815f5d0640 1 -- 172.21.15.70:0/969685847 >> 172.21.15.70:0/969685847 conn(0x7f815806a5d0 msgr2=0x7f815815e870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.634+0000 7f815f5d0640 1 -- 172.21.15.70:0/969685847 shutdown_connections 2023-10-08T05:52:53.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.634+0000 7f815f5d0640 1 -- 172.21.15.70:0/969685847 wait complete. 2023-10-08T05:52:53.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T05:52:53.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.871+0000 7fdce77c5640 1 Processor -- start 2023-10-08T05:52:53.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.871+0000 7fdce77c5640 1 -- start start 2023-10-08T05:52:53.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.873+0000 7fdce77c5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdce015ce10 0x7fdce00ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:53.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.873+0000 7fdce77c5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdce00cf3e0 0x7fdce006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.873+0000 7fdce77c5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce00cf840 con 0x7fdce006bab0 2023-10-08T05:52:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.873+0000 7fdce77c5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce006bf40 con 0x7fdce00cf3e0 2023-10-08T05:52:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.873+0000 7fdce77c5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce006cd90 con 0x7fdce015ce10 2023-10-08T05:52:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.873+0000 7fdce4d60640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdce00cf3e0 0x7fdce006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.873+0000 7fdce4d60640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdce00cf3e0 0x7fdce006b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54498/0 (socket says 172.21.15.70:54498) 2023-10-08T05:52:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.873+0000 7fdce4d60640 1 -- 172.21.15.70:0/4198927034 learned_addr learned my addr 172.21.15.70:0/4198927034 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.874+0000 7fdce5561640 1 --2- 172.21.15.70:0/4198927034 >> v2:172.21.15.181:3300/0 conn(0x7fdce015ce10 0x7fdce00ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.874+0000 7fdce4d60640 1 -- 172.21.15.70:0/4198927034 >> v2:172.21.15.181:3300/0 conn(0x7fdce015ce10 msgr2=0x7fdce00ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.874+0000 7fdce4d60640 1 --2- 172.21.15.70:0/4198927034 >> v2:172.21.15.181:3300/0 conn(0x7fdce015ce10 0x7fdce00ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:53.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.874+0000 7fdce4d60640 1 -- 172.21.15.70:0/4198927034 >> v1:172.21.15.181:6789/0 conn(0x7fdce006bab0 legacy=0x7fdce00b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:53.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.874+0000 7fdce4d60640 1 -- 172.21.15.70:0/4198927034 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce0160ca0 con 0x7fdce00cf3e0 2023-10-08T05:52:53.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.875+0000 7fdce4d60640 1 --2- 172.21.15.70:0/4198927034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce00cf3e0 0x7fdce006b570 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fdce00675c0 tx=0x7fdcd002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f02955d292f9ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:53.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.875+0000 7fdcd7fff640 1 -- 172.21.15.70:0/4198927034 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd00317e0 con 0x7fdce00cf3e0 2023-10-08T05:52:53.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.875+0000 7fdcd7fff640 1 -- 172.21.15.70:0/4198927034 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0031940 con 0x7fdce00cf3e0 2023-10-08T05:52:53.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.875+0000 7fdcd7fff640 1 -- 172.21.15.70:0/4198927034 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0031cd0 con 0x7fdce00cf3e0 2023-10-08T05:52:53.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.876+0000 7fdce77c5640 1 -- 172.21.15.70:0/4198927034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce00cf3e0 msgr2=0x7fdce006b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:53.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.876+0000 7fdce77c5640 1 --2- 172.21.15.70:0/4198927034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce00cf3e0 0x7fdce006b570 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fdce00675c0 tx=0x7fdcd002b880 comp rx=0 tx=0).stop 2023-10-08T05:52:53.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.876+0000 7fdce77c5640 1 -- 172.21.15.70:0/4198927034 shutdown_connections 2023-10-08T05:52:53.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.876+0000 7fdce77c5640 1 --2- 172.21.15.70:0/4198927034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce00cf3e0 0x7fdce006b570 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:53.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.876+0000 7fdce77c5640 1 --2- 172.21.15.70:0/4198927034 >> v2:172.21.15.181:3300/0 conn(0x7fdce015ce10 0x7fdce00ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:53.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.876+0000 7fdce77c5640 1 -- 172.21.15.70:0/4198927034 >> 172.21.15.70:0/4198927034 conn(0x7fdce006e630 msgr2=0x7fdce006ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:53.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.876+0000 7fdce77c5640 1 -- 172.21.15.70:0/4198927034 shutdown_connections 2023-10-08T05:52:53.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.876+0000 7fdce77c5640 1 -- 172.21.15.70:0/4198927034 wait complete. 2023-10-08T05:52:53.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.878+0000 7fdce77c5640 1 Processor -- start 2023-10-08T05:52:53.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.878+0000 7fdce77c5640 1 -- start start 2023-10-08T05:52:53.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce77c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce006bab0 0x7fdce015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:53.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce77c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce00cf3e0 0x7fdce014cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce77c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce015ce10 0x7fdce014c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce77c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce014c800 con 0x7fdce006bab0 2023-10-08T05:52:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce77c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce014cd10 con 0x7fdce00cf3e0 2023-10-08T05:52:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce77c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce015d7d0 con 0x7fdce015ce10 2023-10-08T05:52:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce5561640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce006bab0 0x7fdce015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce5561640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce006bab0 0x7fdce015d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54510/0 (socket says 172.21.15.70:54510) 2023-10-08T05:52:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce5561640 1 -- 172.21.15.70:0/2859892821 learned_addr learned my addr 172.21.15.70:0/2859892821 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce4d60640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce00cf3e0 0x7fdce014cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce5561640 1 -- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce015ce10 msgr2=0x7fdce014c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce5561640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce015ce10 0x7fdce014c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce5561640 1 -- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce00cf3e0 msgr2=0x7fdce014cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce5561640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce00cf3e0 0x7fdce014cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.880+0000 7fdce5561640 1 -- 172.21.15.70:0/2859892821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd00094d0 con 0x7fdce006bab0 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.881+0000 7fdce5561640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce006bab0 0x7fdce015d1f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc007ef0 tx=0x7fdcdc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.881+0000 7fdcd5ffb640 1 -- 172.21.15.70:0/2859892821 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc014620 con 0x7fdce006bab0 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.881+0000 7fdcd5ffb640 1 -- 172.21.15.70:0/2859892821 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc014780 con 0x7fdce006bab0 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.881+0000 7fdce77c5640 1 -- 172.21.15.70:0/2859892821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce015dab0 con 0x7fdce006bab0 2023-10-08T05:52:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.881+0000 7fdcd5ffb640 1 -- 172.21.15.70:0/2859892821 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc014b30 con 0x7fdce006bab0 2023-10-08T05:52:53.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.881+0000 7fdce77c5640 1 -- 172.21.15.70:0/2859892821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce015e070 con 0x7fdce006bab0 2023-10-08T05:52:53.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.882+0000 7fdce77c5640 1 -- 172.21.15.70:0/2859892821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce004e160 con 0x7fdce006bab0 2023-10-08T05:52:53.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.883+0000 7fdcd5ffb640 1 -- 172.21.15.70:0/2859892821 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fdcdc014c90 con 0x7fdce006bab0 2023-10-08T05:52:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.885+0000 7fdcd5ffb640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fdcc406af00 0x7fdcc406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.885+0000 7fdcd5ffb640 1 -- 172.21.15.70:0/2859892821 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fdcdc08a150 con 0x7fdce006bab0 2023-10-08T05:52:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.885+0000 7fdce4d60640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fdcc406af00 0x7fdcc406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.887+0000 7fdce4d60640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fdcc406af00 0x7fdcc406d3c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0036000 tx=0x7fdcd003d040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:53.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:53.890+0000 7fdcd5ffb640 1 -- 172.21.15.70:0/2859892821 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcdc055180 con 0x7fdce006bab0 2023-10-08T05:52:53.998 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:53.996+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:54.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.088+0000 7fdce77c5640 1 -- 172.21.15.70:0/2859892821 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdce00c60e0 con 0x7fdce006bab0 2023-10-08T05:52:54.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.095+0000 7fdcd5ffb640 1 -- 172.21.15.70:0/2859892821 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdcdc0587a0 con 0x7fdce006bab0 2023-10-08T05:52:54.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.097+0000 7fdce77c5640 1 -- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fdcc406af00 msgr2=0x7fdcc406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.097+0000 7fdce77c5640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fdcc406af00 0x7fdcc406d3c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0036000 tx=0x7fdcd003d040 comp rx=0 tx=0).stop 2023-10-08T05:52:54.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.098+0000 7fdce77c5640 1 -- 172.21.15.70:0/2859892821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce006bab0 msgr2=0x7fdce015d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.098+0000 7fdce77c5640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce006bab0 0x7fdce015d1f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc007ef0 tx=0x7fdcdc00e4b0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.098+0000 7fdce77c5640 1 -- 172.21.15.70:0/2859892821 shutdown_connections 2023-10-08T05:52:54.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.098+0000 7fdce77c5640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fdcc406af00 0x7fdcc406d3c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.098+0000 7fdce77c5640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdce015ce10 0x7fdce014c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.098+0000 7fdce77c5640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdce00cf3e0 0x7fdce014cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.098+0000 7fdce77c5640 1 --2- 172.21.15.70:0/2859892821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce006bab0 0x7fdce015d1f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.098+0000 7fdce77c5640 1 -- 172.21.15.70:0/2859892821 >> 172.21.15.70:0/2859892821 conn(0x7fdce006e630 msgr2=0x7fdce015c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:54.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.098+0000 7fdce77c5640 1 -- 172.21.15.70:0/2859892821 shutdown_connections 2023-10-08T05:52:54.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.098+0000 7fdce77c5640 1 -- 172.21.15.70:0/2859892821 wait complete. 2023-10-08T05:52:54.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T05:52:54.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.322+0000 7f3835026640 1 Processor -- start 2023-10-08T05:52:54.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.322+0000 7f3835026640 1 -- start start 2023-10-08T05:52:54.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.324+0000 7f3835026640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f383015a820 0x7f3830160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.324+0000 7f3835026640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38300c9080 0x7f38300c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.324+0000 7f3835026640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38300c7ad0 con 0x7f3830070b50 2023-10-08T05:52:54.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.324+0000 7f3835026640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38300c9a40 con 0x7f38300c9080 2023-10-08T05:52:54.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.324+0000 7f3835026640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383015ac00 con 0x7f383015a820 2023-10-08T05:52:54.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.324+0000 7f382f59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3830070b50 0x7f38300c7f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48150/0 (socket says 172.21.15.70:48150) 2023-10-08T05:52:54.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.324+0000 7f382f59e640 1 -- 172.21.15.70:0/2784783860 learned_addr learned my addr 172.21.15.70:0/2784783860 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:54.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382ed9d640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f383015a820 0x7f3830160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382e59c640 1 --2- 172.21.15.70:0/2784783860 >> v2:172.21.15.181:3300/0 conn(0x7f38300c9080 0x7f38300c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779561927 0 0) 0x7f38300c7ad0 con 0x7f3830070b50 2023-10-08T05:52:54.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3818003520 con 0x7f3830070b50 2023-10-08T05:52:54.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3866682256 0 0) 0x7f3818003520 con 0x7f3830070b50 2023-10-08T05:52:54.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f383015a820 msgr2=0x7f3830160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382dd9b640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f383015a820 0x7f3830160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 >> v2:172.21.15.181:3300/0 conn(0x7f38300c9080 msgr2=0x7f38300c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382dd9b640 1 --2- 172.21.15.70:0/2784783860 >> v2:172.21.15.181:3300/0 conn(0x7f38300c9080 0x7f38300c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38300cb210 con 0x7f3830070b50 2023-10-08T05:52:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.325+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f38280019f0 con 0x7f3830070b50 2023-10-08T05:52:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 >> v1:172.21.15.70:6789/0 conn(0x7f3830070b50 legacy=0x7f38300c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382dd9b640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f383015a820 0x7f3818003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382dd9b640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3818003f40 0x7f38180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382dd9b640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3818004d00 0x7f3818005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38300c9a40 con 0x7f3818004d00 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383015ac00 con 0x7f3818003f40 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381800b980 con 0x7f383015a820 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382f59e640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3818004d00 0x7f3818005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382e59c640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3818003f40 0x7f38180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382ed9d640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f383015a820 0x7f3818003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382f59e640 1 -- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f383015a820 msgr2=0x7f3818003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382f59e640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f383015a820 0x7f3818003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382f59e640 1 -- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3818003f40 msgr2=0x7f38180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382f59e640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3818003f40 0x7f38180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.326+0000 7f382f59e640 1 -- 172.21.15.70:0/2784783860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3818003520 con 0x7f3818004d00 2023-10-08T05:52:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.327+0000 7f382ed9d640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f383015a820 0x7f3818003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:54.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.328+0000 7f382f59e640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3818004d00 0x7f3818005180 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f382800a7c0 tx=0x7f38180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370184122d066092 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:54.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.328+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828034630 con 0x7f3818004d00 2023-10-08T05:52:54.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.328+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382802f070 con 0x7f3818004d00 2023-10-08T05:52:54.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.328+0000 7f382dd9b640 1 -- 172.21.15.70:0/2784783860 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828034a30 con 0x7f3818004d00 2023-10-08T05:52:54.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.328+0000 7f3835026640 1 -- 172.21.15.70:0/2784783860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3818004d00 msgr2=0x7f3818005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.328+0000 7f3835026640 1 --2- 172.21.15.70:0/2784783860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3818004d00 0x7f3818005180 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f382800a7c0 tx=0x7f38180057f0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.328+0000 7f382f59e640 1 -- 172.21.15.70:0/2784783860 reap_dead start 2023-10-08T05:52:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.329+0000 7f3835026640 1 -- 172.21.15.70:0/2784783860 shutdown_connections 2023-10-08T05:52:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.329+0000 7f3835026640 1 -- 172.21.15.70:0/2784783860 >> 172.21.15.70:0/2784783860 conn(0x7f3830069d50 msgr2=0x7f383006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.329+0000 7f3835026640 1 -- 172.21.15.70:0/2784783860 shutdown_connections 2023-10-08T05:52:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.329+0000 7f3835026640 1 -- 172.21.15.70:0/2784783860 wait complete. 2023-10-08T05:52:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.330+0000 7f3835026640 1 Processor -- start 2023-10-08T05:52:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.331+0000 7f3835026640 1 -- start start 2023-10-08T05:52:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.331+0000 7f3835026640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f383014b2a0 0x7f3830156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.331+0000 7f3835026640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383014b680 0x7f383015b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.331+0000 7f3835026640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3830157040 0x7f383015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f3835026640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38301594d0 con 0x7f383014b2a0 2023-10-08T05:52:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f3835026640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3830158890 con 0x7f383014b680 2023-10-08T05:52:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f3835026640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3830158b30 con 0x7f3830157040 2023-10-08T05:52:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f382ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f383014b2a0 0x7f3830156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f382ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f383014b2a0 0x7f3830156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54530/0 (socket says 172.21.15.70:54530) 2023-10-08T05:52:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f382ed9d640 1 -- 172.21.15.70:0/478405857 learned_addr learned my addr 172.21.15.70:0/478405857 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f382f59e640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3830157040 0x7f383015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f382ed9d640 1 -- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3830157040 msgr2=0x7f383015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f382e59c640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383014b680 0x7f383015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f382ed9d640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3830157040 0x7f383015b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f382ed9d640 1 -- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383014b680 msgr2=0x7f383015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f382ed9d640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383014b680 0x7f383015b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.332+0000 7f382ed9d640 1 -- 172.21.15.70:0/478405857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3830158e10 con 0x7f383014b2a0 2023-10-08T05:52:54.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.333+0000 7f382ed9d640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f383014b2a0 0x7f3830156b00 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f38200108b0 tx=0x7f3820010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:54.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.333+0000 7f38177fe640 1 -- 172.21.15.70:0/478405857 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382000eb60 con 0x7f383014b2a0 2023-10-08T05:52:54.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.333+0000 7f38177fe640 1 -- 172.21.15.70:0/478405857 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382000ecc0 con 0x7f383014b2a0 2023-10-08T05:52:54.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.333+0000 7f38177fe640 1 -- 172.21.15.70:0/478405857 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382001c400 con 0x7f383014b2a0 2023-10-08T05:52:54.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.333+0000 7f3835026640 1 -- 172.21.15.70:0/478405857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38301590f0 con 0x7f383014b2a0 2023-10-08T05:52:54.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.333+0000 7f3835026640 1 -- 172.21.15.70:0/478405857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3830159ad0 con 0x7f383014b2a0 2023-10-08T05:52:54.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.334+0000 7f3835026640 1 -- 172.21.15.70:0/478405857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383006e610 con 0x7f383014b2a0 2023-10-08T05:52:54.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.335+0000 7f38177fe640 1 -- 172.21.15.70:0/478405857 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f382001c560 con 0x7f383014b2a0 2023-10-08T05:52:54.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.336+0000 7f38177fe640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f381c06af00 0x7f381c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.336+0000 7f38177fe640 1 -- 172.21.15.70:0/478405857 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f3820089d90 con 0x7f383014b2a0 2023-10-08T05:52:54.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.337+0000 7f382e59c640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f381c06af00 0x7f381c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.338+0000 7f382e59c640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f381c06af00 0x7f381c06d3c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f383014ba60 tx=0x7f3824008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:54.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.340+0000 7f38177fe640 1 -- 172.21.15.70:0/478405857 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3820054d70 con 0x7f383014b2a0 2023-10-08T05:52:54.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.534+0000 7f3835026640 1 -- 172.21.15.70:0/478405857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f383004e160 con 0x7f383014b2a0 2023-10-08T05:52:54.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.541+0000 7f38177fe640 1 -- 172.21.15.70:0/478405857 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v26) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3820058390 con 0x7f383014b2a0 2023-10-08T05:52:54.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 -- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f381c06af00 msgr2=0x7f381c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f381c06af00 0x7f381c06d3c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f383014ba60 tx=0x7f3824008040 comp rx=0 tx=0).stop 2023-10-08T05:52:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 -- 172.21.15.70:0/478405857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f383014b2a0 msgr2=0x7f3830156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f383014b2a0 0x7f3830156b00 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f38200108b0 tx=0x7f3820010d70 comp rx=0 tx=0).stop 2023-10-08T05:52:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 -- 172.21.15.70:0/478405857 shutdown_connections 2023-10-08T05:52:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f381c06af00 0x7f381c06d3c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3830157040 0x7f383015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f383014b680 0x7f383015b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 --2- 172.21.15.70:0/478405857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f383014b2a0 0x7f3830156b00 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 -- 172.21.15.70:0/478405857 >> 172.21.15.70:0/478405857 conn(0x7f3830069d50 msgr2=0x7f383015c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 -- 172.21.15.70:0/478405857 shutdown_connections 2023-10-08T05:52:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.544+0000 7f3835026640 1 -- 172.21.15.70:0/478405857 wait complete. 2023-10-08T05:52:54.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T05:52:54.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.764+0000 7f6706f4c640 1 Processor -- start 2023-10-08T05:52:54.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.764+0000 7f6706f4c640 1 -- start start 2023-10-08T05:52:54.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f6706f4c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67000afbe0 0x7f67000b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f6706f4c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f670006dfc0 0x7f6700070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f6706f4c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67000b2620 con 0x7f6700071070 2023-10-08T05:52:54.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f6706f4c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000b2060 con 0x7f670006dfc0 2023-10-08T05:52:54.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f6706f4c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000b2340 con 0x7f67000afbe0 2023-10-08T05:52:54.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f66fffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f670006dfc0 0x7f6700070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f67054e9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6700071070 0x7f67000cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48162/0 (socket says 172.21.15.70:48162) 2023-10-08T05:52:54.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f67054e9640 1 -- 172.21.15.70:0/154833672 learned_addr learned my addr 172.21.15.70:0/154833672 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:54.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f66fffff640 1 -- 172.21.15.70:0/154833672 >> v2:172.21.15.181:3300/0 conn(0x7f67000afbe0 msgr2=0x7f67000b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:54.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f66fffff640 1 --2- 172.21.15.70:0/154833672 >> v2:172.21.15.181:3300/0 conn(0x7f67000afbe0 0x7f67000b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.766+0000 7f66fffff640 1 -- 172.21.15.70:0/154833672 >> v1:172.21.15.70:6789/0 conn(0x7f6700071070 legacy=0x7f67000cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.767+0000 7f66fffff640 1 -- 172.21.15.70:0/154833672 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6700070350 con 0x7f670006dfc0 2023-10-08T05:52:54.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.768+0000 7f66fffff640 1 --2- 172.21.15.70:0/154833672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f670006dfc0 0x7f6700070b30 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f670006e640 tx=0x7f66f402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0193ec31272745 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:54.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.768+0000 7f66ff7fe640 1 -- 172.21.15.70:0/154833672 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f402cc70 con 0x7f670006dfc0 2023-10-08T05:52:54.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.768+0000 7f66ff7fe640 1 -- 172.21.15.70:0/154833672 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f402cdd0 con 0x7f670006dfc0 2023-10-08T05:52:54.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.768+0000 7f66ff7fe640 1 -- 172.21.15.70:0/154833672 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4034660 con 0x7f670006dfc0 2023-10-08T05:52:54.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.769+0000 7f6706f4c640 1 -- 172.21.15.70:0/154833672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f670006dfc0 msgr2=0x7f6700070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.769+0000 7f6706f4c640 1 --2- 172.21.15.70:0/154833672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f670006dfc0 0x7f6700070b30 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f670006e640 tx=0x7f66f402bec0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.769+0000 7f6706f4c640 1 -- 172.21.15.70:0/154833672 shutdown_connections 2023-10-08T05:52:54.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.769+0000 7f6706f4c640 1 --2- 172.21.15.70:0/154833672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f670006dfc0 0x7f6700070b30 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.769+0000 7f6706f4c640 1 --2- 172.21.15.70:0/154833672 >> v2:172.21.15.181:3300/0 conn(0x7f67000afbe0 0x7f67000b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.769+0000 7f6706f4c640 1 -- 172.21.15.70:0/154833672 >> 172.21.15.70:0/154833672 conn(0x7f670006a5d0 msgr2=0x7f67000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:54.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.769+0000 7f6706f4c640 1 -- 172.21.15.70:0/154833672 shutdown_connections 2023-10-08T05:52:54.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.769+0000 7f6706f4c640 1 -- 172.21.15.70:0/154833672 wait complete. 2023-10-08T05:52:54.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.770+0000 7f6706f4c640 1 Processor -- start 2023-10-08T05:52:54.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.771+0000 7f6706f4c640 1 -- start start 2023-10-08T05:52:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.771+0000 7f6706f4c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670006dfc0 0x7f67000cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f6706f4c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6700071070 0x7f670015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f6706f4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67000afbe0 0x7f6700146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f6706f4c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000ccb80 con 0x7f67000afbe0 2023-10-08T05:52:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f6706f4c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000cf1a0 con 0x7f670006dfc0 2023-10-08T05:52:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f6706f4c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67000cf480 con 0x7f6700071070 2023-10-08T05:52:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f67054e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67000afbe0 0x7f6700146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f67054e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67000afbe0 0x7f6700146710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54550/0 (socket says 172.21.15.70:54550) 2023-10-08T05:52:54.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f67054e9640 1 -- 172.21.15.70:0/2377241757 learned_addr learned my addr 172.21.15.70:0/2377241757 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f6704ce8640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670006dfc0 0x7f67000cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f67054e9640 1 -- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6700071070 msgr2=0x7f670015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f67054e9640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6700071070 0x7f670015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f67054e9640 1 -- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670006dfc0 msgr2=0x7f67000cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f67054e9640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670006dfc0 0x7f67000cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.772+0000 7f67054e9640 1 -- 172.21.15.70:0/2377241757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4009a70 con 0x7f67000afbe0 2023-10-08T05:52:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.773+0000 7f67054e9640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67000afbe0 0x7f6700146710 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c400 tx=0x7f66f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.773+0000 7f66fd7fa640 1 -- 172.21.15.70:0/2377241757 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8005ce0 con 0x7f67000afbe0 2023-10-08T05:52:54.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.773+0000 7f66fd7fa640 1 -- 172.21.15.70:0/2377241757 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f8005e40 con 0x7f67000afbe0 2023-10-08T05:52:54.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.773+0000 7f66fd7fa640 1 -- 172.21.15.70:0/2377241757 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f80072d0 con 0x7f67000afbe0 2023-10-08T05:52:54.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.773+0000 7f6706f4c640 1 -- 172.21.15.70:0/2377241757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67000c3540 con 0x7f67000afbe0 2023-10-08T05:52:54.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.773+0000 7f6706f4c640 1 -- 172.21.15.70:0/2377241757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67000cee50 con 0x7f67000afbe0 2023-10-08T05:52:54.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.774+0000 7f6706f4c640 1 -- 172.21.15.70:0/2377241757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67000c3a60 con 0x7f67000afbe0 2023-10-08T05:52:54.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.775+0000 7f66fd7fa640 1 -- 172.21.15.70:0/2377241757 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f66f80074f0 con 0x7f67000afbe0 2023-10-08T05:52:54.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.776+0000 7f66fd7fa640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f66ec06af00 0x7f66ec06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:54.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.776+0000 7f66fd7fa640 1 -- 172.21.15.70:0/2377241757 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f66f8088ce0 con 0x7f67000afbe0 2023-10-08T05:52:54.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.777+0000 7f6704ce8640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f66ec06af00 0x7f66ec06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:54.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.778+0000 7f6704ce8640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f66ec06af00 0x7f66ec06d3c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f66f000a810 tx=0x7f66f0008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:54.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.780+0000 7f66fd7fa640 1 -- 172.21.15.70:0/2377241757 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67000c3a60 con 0x7f67000afbe0 2023-10-08T05:52:54.912 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:54.910+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:54.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.983+0000 7f6706f4c640 1 -- 172.21.15.70:0/2377241757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f67000c3a60 con 0x7f67000afbe0 2023-10-08T05:52:54.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.989+0000 7f66fd7fa640 1 -- 172.21.15.70:0/2377241757 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v27) v1 ==== 120+0+0 (secure 0 0 0) 0x7f67000c3a60 con 0x7f67000afbe0 2023-10-08T05:52:54.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.991+0000 7f6706f4c640 1 -- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f66ec06af00 msgr2=0x7f66ec06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.991+0000 7f6706f4c640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f66ec06af00 0x7f66ec06d3c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f66f000a810 tx=0x7f66f0008040 comp rx=0 tx=0).stop 2023-10-08T05:52:54.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.991+0000 7f6706f4c640 1 -- 172.21.15.70:0/2377241757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67000afbe0 msgr2=0x7f6700146710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:54.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.991+0000 7f6706f4c640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67000afbe0 0x7f6700146710 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f66f800c400 tx=0x7f66f800c8c0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.991+0000 7f6706f4c640 1 -- 172.21.15.70:0/2377241757 shutdown_connections 2023-10-08T05:52:54.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.991+0000 7f6706f4c640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f66ec06af00 0x7f66ec06d3c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.991+0000 7f6706f4c640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67000afbe0 0x7f6700146710 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.991+0000 7f6706f4c640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6700071070 0x7f670015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.991+0000 7f6706f4c640 1 --2- 172.21.15.70:0/2377241757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f670006dfc0 0x7f67000cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:54.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.991+0000 7f6706f4c640 1 -- 172.21.15.70:0/2377241757 >> 172.21.15.70:0/2377241757 conn(0x7f670006a5d0 msgr2=0x7f670015dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:54.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.992+0000 7f6706f4c640 1 -- 172.21.15.70:0/2377241757 shutdown_connections 2023-10-08T05:52:54.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:54.992+0000 7f6706f4c640 1 -- 172.21.15.70:0/2377241757 wait complete. 2023-10-08T05:52:55.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-08T05:52:55.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.220+0000 7fc33a2d7640 1 Processor -- start 2023-10-08T05:52:55.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.220+0000 7fc33a2d7640 1 -- start start 2023-10-08T05:52:55.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.222+0000 7fc33a2d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc33406d4c0 0x7fc334162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.222+0000 7fc33a2d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc3340cb150 0x7fc3340cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.222+0000 7fc33a2d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3340ae140 con 0x7fc3340add60 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.222+0000 7fc33a2d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc334161f10 con 0x7fc3340cb150 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.222+0000 7fc33a2d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3340cbb20 con 0x7fc33406d4c0 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.222+0000 7fc333fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc3340add60 0x7fc3340cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48168/0 (socket says 172.21.15.70:48168) 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.222+0000 7fc333fff640 1 -- 172.21.15.70:0/1237768590 learned_addr learned my addr 172.21.15.70:0/1237768590 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.222+0000 7fc338874640 1 --2- 172.21.15.70:0/1237768590 >> v2:172.21.15.70:3300/0 conn(0x7fc3340cb150 0x7fc3340cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.223+0000 7fc3337fe640 1 --2- 172.21.15.70:0/1237768590 >> v2:172.21.15.181:3300/0 conn(0x7fc33406d4c0 0x7fc334162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.223+0000 7fc338874640 1 -- 172.21.15.70:0/1237768590 >> v2:172.21.15.181:3300/0 conn(0x7fc33406d4c0 msgr2=0x7fc334162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.223+0000 7fc338874640 1 --2- 172.21.15.70:0/1237768590 >> v2:172.21.15.181:3300/0 conn(0x7fc33406d4c0 0x7fc334162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.223+0000 7fc338874640 1 -- 172.21.15.70:0/1237768590 >> v1:172.21.15.70:6789/0 conn(0x7fc3340add60 legacy=0x7fc3340cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:55.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.223+0000 7fc338874640 1 -- 172.21.15.70:0/1237768590 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3340ccf20 con 0x7fc3340cb150 2023-10-08T05:52:55.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.224+0000 7fc338874640 1 --2- 172.21.15.70:0/1237768590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 0x7fc3340cb5e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc33406e560 tx=0x7fc32c02b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03b6ab698cba256 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:55.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.224+0000 7fc332ffd640 1 -- 172.21.15.70:0/1237768590 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c030710 con 0x7fc3340cb150 2023-10-08T05:52:55.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.224+0000 7fc332ffd640 1 -- 172.21.15.70:0/1237768590 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c02e050 con 0x7fc3340cb150 2023-10-08T05:52:55.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.224+0000 7fc332ffd640 1 -- 172.21.15.70:0/1237768590 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c030a90 con 0x7fc3340cb150 2023-10-08T05:52:55.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.225+0000 7fc33a2d7640 1 -- 172.21.15.70:0/1237768590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 msgr2=0x7fc3340cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:55.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.225+0000 7fc33a2d7640 1 --2- 172.21.15.70:0/1237768590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 0x7fc3340cb5e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc33406e560 tx=0x7fc32c02b680 comp rx=0 tx=0).stop 2023-10-08T05:52:55.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.225+0000 7fc33a2d7640 1 -- 172.21.15.70:0/1237768590 shutdown_connections 2023-10-08T05:52:55.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.225+0000 7fc33a2d7640 1 --2- 172.21.15.70:0/1237768590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 0x7fc3340cb5e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.225+0000 7fc33a2d7640 1 --2- 172.21.15.70:0/1237768590 >> v2:172.21.15.181:3300/0 conn(0x7fc33406d4c0 0x7fc334162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.225+0000 7fc33a2d7640 1 -- 172.21.15.70:0/1237768590 >> 172.21.15.70:0/1237768590 conn(0x7fc33406a380 msgr2=0x7fc3340b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:55.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.225+0000 7fc33a2d7640 1 -- 172.21.15.70:0/1237768590 shutdown_connections 2023-10-08T05:52:55.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.225+0000 7fc33a2d7640 1 -- 172.21.15.70:0/1237768590 wait complete. 2023-10-08T05:52:55.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.227+0000 7fc33a2d7640 1 Processor -- start 2023-10-08T05:52:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.227+0000 7fc33a2d7640 1 -- start start 2023-10-08T05:52:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc33a2d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc33406d4c0 0x7fc3341565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc33a2d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3340add60 0x7fc3340b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc33a2d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 0x7fc3340c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc33a2d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc334158980 con 0x7fc3340cb150 2023-10-08T05:52:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc33a2d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340b41a0 con 0x7fc3340add60 2023-10-08T05:52:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc33a2d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc334158c20 con 0x7fc33406d4c0 2023-10-08T05:52:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc338874640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 0x7fc3340c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc338874640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 0x7fc3340c4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54562/0 (socket says 172.21.15.70:54562) 2023-10-08T05:52:55.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc338874640 1 -- 172.21.15.70:0/3160699452 learned_addr learned my addr 172.21.15.70:0/3160699452 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:55.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc338874640 1 -- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc33406d4c0 msgr2=0x7fc3341565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:55.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc333fff640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc33406d4c0 0x7fc3341565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc338874640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc33406d4c0 0x7fc3341565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc3337fe640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3340add60 0x7fc3340b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.228+0000 7fc338874640 1 -- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3340add60 msgr2=0x7fc3340b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:55.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.229+0000 7fc338874640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3340add60 0x7fc3340b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.229+0000 7fc338874640 1 -- 172.21.15.70:0/3160699452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c009400 con 0x7fc3340cb150 2023-10-08T05:52:55.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.229+0000 7fc338874640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 0x7fc3340c4ca0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc32c035040 tx=0x7fc32c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:55.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.229+0000 7fc330ff9640 1 -- 172.21.15.70:0/3160699452 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c03b810 con 0x7fc3340cb150 2023-10-08T05:52:55.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.229+0000 7fc330ff9640 1 -- 172.21.15.70:0/3160699452 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c03b990 con 0x7fc3340cb150 2023-10-08T05:52:55.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.229+0000 7fc330ff9640 1 -- 172.21.15.70:0/3160699452 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c03bd40 con 0x7fc3340cb150 2023-10-08T05:52:55.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.229+0000 7fc33a2d7640 1 -- 172.21.15.70:0/3160699452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc334158f00 con 0x7fc3340cb150 2023-10-08T05:52:55.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.229+0000 7fc33a2d7640 1 -- 172.21.15.70:0/3160699452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3341594c0 con 0x7fc3340cb150 2023-10-08T05:52:55.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.230+0000 7fc33a2d7640 1 -- 172.21.15.70:0/3160699452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc334067490 con 0x7fc3340cb150 2023-10-08T05:52:55.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.231+0000 7fc330ff9640 1 -- 172.21.15.70:0/3160699452 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc32c039040 con 0x7fc3340cb150 2023-10-08T05:52:55.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.232+0000 7fc330ff9640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc31406af00 0x7fc31406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.232+0000 7fc330ff9640 1 -- 172.21.15.70:0/3160699452 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fc32c0ab120 con 0x7fc3340cb150 2023-10-08T05:52:55.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.233+0000 7fc333fff640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc31406af00 0x7fc31406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.234+0000 7fc333fff640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc31406af00 0x7fc31406d3c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc3240012e0 tx=0x7fc324009040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:55.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.236+0000 7fc330ff9640 1 -- 172.21.15.70:0/3160699452 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc32c075fb0 con 0x7fc3340cb150 2023-10-08T05:52:55.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.491+0000 7fc33a2d7640 1 -- 172.21.15.70:0/3160699452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fc3340b4990 con 0x7fc3340cb150 2023-10-08T05:52:55.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.652+0000 7fc330ff9640 1 -- 172.21.15.70:0/3160699452 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v625) v1 ==== 156+0+0 (secure 0 0 0) 0x7fc32c04b0e0 con 0x7fc3340cb150 2023-10-08T05:52:55.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 -- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc31406af00 msgr2=0x7fc31406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:55.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc31406af00 0x7fc31406d3c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc3240012e0 tx=0x7fc324009040 comp rx=0 tx=0).stop 2023-10-08T05:52:55.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 -- 172.21.15.70:0/3160699452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 msgr2=0x7fc3340c4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:55.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 0x7fc3340c4ca0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc32c035040 tx=0x7fc32c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 -- 172.21.15.70:0/3160699452 shutdown_connections 2023-10-08T05:52:55.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc31406af00 0x7fc31406d3c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3340cb150 0x7fc3340c4ca0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3340add60 0x7fc3340b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 --2- 172.21.15.70:0/3160699452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc33406d4c0 0x7fc3341565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 -- 172.21.15.70:0/3160699452 >> 172.21.15.70:0/3160699452 conn(0x7fc33406a380 msgr2=0x7fc334164b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 -- 172.21.15.70:0/3160699452 shutdown_connections 2023-10-08T05:52:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.655+0000 7fc33a2d7640 1 -- 172.21.15.70:0/3160699452 wait complete. 2023-10-08T05:52:55.677 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-10-08T05:52:55.677 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-10-08T05:52:55.678 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T05:52:55.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:52:55.839 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:55.837+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:55.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.886+0000 7fa99f6f2640 1 Processor -- start 2023-10-08T05:52:55.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.886+0000 7fa99f6f2640 1 -- start start 2023-10-08T05:52:55.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.887+0000 7fa99f6f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980c90b0 0x7fa9980cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.887+0000 7fa99f6f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9980cb3b0 0x7fa99815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.887+0000 7fa99f6f2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99815b2b0 con 0x7fa99815ab40 2023-10-08T05:52:55.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.887+0000 7fa99f6f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99815bf30 con 0x7fa9980cb3b0 2023-10-08T05:52:55.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.887+0000 7fa99f6f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9980cb840 con 0x7fa9980c90b0 2023-10-08T05:52:55.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.888+0000 7fa99cc8d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980c90b0 0x7fa9980cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.888+0000 7fa99dc8f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9980cb3b0 0x7fa99815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.888+0000 7fa99cc8d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980c90b0 0x7fa9980cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:47982/0 (socket says 172.21.15.70:47982) 2023-10-08T05:52:55.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.888+0000 7fa99dc8f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9980cb3b0 0x7fa99815e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:37308/0 (socket says 172.21.15.70:37308) 2023-10-08T05:52:55.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.888+0000 7fa99cc8d640 1 -- 172.21.15.70:0/2186375151 learned_addr learned my addr 172.21.15.70:0/2186375151 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:55.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.888+0000 7fa99d48e640 1 --1- 172.21.15.70:0/2186375151 >> v1:172.21.15.181:6789/0 conn(0x7fa99815ab40 0x7fa9980ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:57612/0 (socket says 172.21.15.70:57612) 2023-10-08T05:52:55.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.889+0000 7fa99cc8d640 1 -- 172.21.15.70:0/2186375151 >> v2:172.21.15.181:3300/0 conn(0x7fa9980cb3b0 msgr2=0x7fa99815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:55.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.889+0000 7fa99cc8d640 1 --2- 172.21.15.70:0/2186375151 >> v2:172.21.15.181:3300/0 conn(0x7fa9980cb3b0 0x7fa99815e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.889+0000 7fa99cc8d640 1 -- 172.21.15.70:0/2186375151 >> v1:172.21.15.181:6789/0 conn(0x7fa99815ab40 legacy=0x7fa9980ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:55.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.889+0000 7fa99cc8d640 1 -- 172.21.15.70:0/2186375151 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99815b590 con 0x7fa9980c90b0 2023-10-08T05:52:55.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.891+0000 7fa99cc8d640 1 --2- 172.21.15.70:0/2186375151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980c90b0 0x7fa9980cae70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa988009870 tx=0x7fa98802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc46f935fe38afb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:55.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.891+0000 7fa98ffff640 1 -- 172.21.15.70:0/2186375151 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988030840 con 0x7fa9980c90b0 2023-10-08T05:52:55.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.891+0000 7fa98ffff640 1 -- 172.21.15.70:0/2186375151 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9880309a0 con 0x7fa9980c90b0 2023-10-08T05:52:55.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.891+0000 7fa98ffff640 1 -- 172.21.15.70:0/2186375151 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988030d30 con 0x7fa9980c90b0 2023-10-08T05:52:55.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.892+0000 7fa99f6f2640 1 -- 172.21.15.70:0/2186375151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980c90b0 msgr2=0x7fa9980cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:55.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.892+0000 7fa99f6f2640 1 --2- 172.21.15.70:0/2186375151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980c90b0 0x7fa9980cae70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa988009870 tx=0x7fa98802b980 comp rx=0 tx=0).stop 2023-10-08T05:52:55.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.892+0000 7fa99f6f2640 1 -- 172.21.15.70:0/2186375151 shutdown_connections 2023-10-08T05:52:55.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.892+0000 7fa99f6f2640 1 --2- 172.21.15.70:0/2186375151 >> v2:172.21.15.181:3300/0 conn(0x7fa9980cb3b0 0x7fa99815e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.892+0000 7fa99f6f2640 1 --2- 172.21.15.70:0/2186375151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980c90b0 0x7fa9980cae70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.892+0000 7fa99f6f2640 1 -- 172.21.15.70:0/2186375151 >> 172.21.15.70:0/2186375151 conn(0x7fa99806a380 msgr2=0x7fa9980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:55.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.892+0000 7fa99f6f2640 1 -- 172.21.15.70:0/2186375151 shutdown_connections 2023-10-08T05:52:55.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.892+0000 7fa99f6f2640 1 -- 172.21.15.70:0/2186375151 wait complete. 2023-10-08T05:52:55.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.894+0000 7fa99f6f2640 1 Processor -- start 2023-10-08T05:52:55.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.894+0000 7fa99f6f2640 1 -- start start 2023-10-08T05:52:55.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.895+0000 7fa99f6f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980c90b0 0x7fa998144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.895+0000 7fa99f6f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980cb3b0 0x7fa9980ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.895+0000 7fa99f6f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa99815ab40 0x7fa9980cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.895+0000 7fa99f6f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa998147950 con 0x7fa99815ab40 2023-10-08T05:52:55.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.895+0000 7fa99f6f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981f0710 con 0x7fa9980c90b0 2023-10-08T05:52:55.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.895+0000 7fa99f6f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9981f09f0 con 0x7fa9980cb3b0 2023-10-08T05:52:55.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.895+0000 7fa99dc8f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa99815ab40 0x7fa9980cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.895+0000 7fa99dc8f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa99815ab40 0x7fa9980cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54576/0 (socket says 172.21.15.70:54576) 2023-10-08T05:52:55.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.895+0000 7fa99dc8f640 1 -- 172.21.15.70:0/1883208926 learned_addr learned my addr 172.21.15.70:0/1883208926 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:55.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa99d48e640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980c90b0 0x7fa998144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa99cc8d640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980cb3b0 0x7fa9980ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa99dc8f640 1 -- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980cb3b0 msgr2=0x7fa9980ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:55.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa99dc8f640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980cb3b0 0x7fa9980ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa99dc8f640 1 -- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980c90b0 msgr2=0x7fa998144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:55.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa99dc8f640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980c90b0 0x7fa998144300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:55.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa99dc8f640 1 -- 172.21.15.70:0/1883208926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9880094d0 con 0x7fa99815ab40 2023-10-08T05:52:55.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa99cc8d640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980cb3b0 0x7fa9980ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:55.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa99dc8f640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa99815ab40 0x7fa9980cec00 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa99000efc0 tx=0x7fa99000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:55.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa98dffb640 1 -- 172.21.15.70:0/1883208926 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990014480 con 0x7fa99815ab40 2023-10-08T05:52:55.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa98dffb640 1 -- 172.21.15.70:0/1883208926 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99000f040 con 0x7fa99815ab40 2023-10-08T05:52:55.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa99f6f2640 1 -- 172.21.15.70:0/1883208926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9981f0cd0 con 0x7fa99815ab40 2023-10-08T05:52:55.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.896+0000 7fa98dffb640 1 -- 172.21.15.70:0/1883208926 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990014820 con 0x7fa99815ab40 2023-10-08T05:52:55.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.897+0000 7fa99f6f2640 1 -- 172.21.15.70:0/1883208926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9981f14d0 con 0x7fa99815ab40 2023-10-08T05:52:55.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.898+0000 7fa99f6f2640 1 -- 172.21.15.70:0/1883208926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa998067490 con 0x7fa99815ab40 2023-10-08T05:52:55.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.898+0000 7fa98dffb640 1 -- 172.21.15.70:0/1883208926 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa990014980 con 0x7fa99815ab40 2023-10-08T05:52:55.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.899+0000 7fa98dffb640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa97c06af00 0x7fa97c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:55.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.899+0000 7fa98dffb640 1 -- 172.21.15.70:0/1883208926 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fa99008a530 con 0x7fa99815ab40 2023-10-08T05:52:55.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.900+0000 7fa99d48e640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa97c06af00 0x7fa97c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:55.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.901+0000 7fa99d48e640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa97c06af00 0x7fa97c06d3c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9980b0e60 tx=0x7fa994007ce0 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:55.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:55.903+0000 7fa98dffb640 1 -- 172.21.15.70:0/1883208926 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa9900552c0 con 0x7fa99815ab40 2023-10-08T05:52:56.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.101+0000 7fa99f6f2640 1 -- 172.21.15.70:0/1883208926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9980b0ef0 con 0x7fa99815ab40 2023-10-08T05:52:56.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.103+0000 7fa98dffb640 1 -- 172.21.15.70:0/1883208926 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+14270 (secure 0 0 0) 0x7fa9900588e0 con 0x7fa99815ab40 2023-10-08T05:52:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:56.105 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":95,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:52:53.624433+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6836/2348955257":"2023-10-09T05:52:50.396618+0000","172.21.15.181:0/3148780950":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/999647594":"2023-10-09T05:52:46.566070+0000","172.21.15.181:6833/1709912921":"2023-10-09T05:52:46.566070+0000","172.21.15.181:6837/2348955257":"2023-10-09T05:52:50.396618+0000","172.21.15.181:6832/1709912921":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/3306361152":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/2398424071":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/814792959":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1860309542":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/2758467223":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1699086184":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1378633292":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/1217040450":"2023-10-09T05:52:40.392573+0000","172.21.15.181:0/3712172314":"2023-10-09T05:52:46.566070+0000","172.21.15.181:0/2333154107":"2023-10-09T05:52:40.392573+0000","172.21.15.181:6833/3994892546":"2023-10-09T05:52:40.392573+0000","172.21.15.181:6832/3994892546":"2023-10-09T05:52:40.392573+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:52:56.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 -- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa97c06af00 msgr2=0x7fa97c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa97c06af00 0x7fa97c06d3c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa9980b0e60 tx=0x7fa994007ce0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 -- 172.21.15.70:0/1883208926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa99815ab40 msgr2=0x7fa9980cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa99815ab40 0x7fa9980cec00 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa99000efc0 tx=0x7fa99000c490 comp rx=0 tx=0).stop 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 -- 172.21.15.70:0/1883208926 shutdown_connections 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa97c06af00 0x7fa97c06d3c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa99815ab40 0x7fa9980cec00 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9980cb3b0 0x7fa9980ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 --2- 172.21.15.70:0/1883208926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9980c90b0 0x7fa998144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 -- 172.21.15.70:0/1883208926 >> 172.21.15.70:0/1883208926 conn(0x7fa99806a380 msgr2=0x7fa99806c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 -- 172.21.15.70:0/1883208926 shutdown_connections 2023-10-08T05:52:56.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.106+0000 7fa99f6f2640 1 -- 172.21.15.70:0/1883208926 wait complete. 2023-10-08T05:52:56.127 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:52:56.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:52:56.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.333+0000 7fa373fff640 1 Processor -- start 2023-10-08T05:52:56.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.333+0000 7fa373fff640 1 -- start start 2023-10-08T05:52:56.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.335+0000 7fa373fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa37415aab0 0x7fa374070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.335+0000 7fa373fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa3740caef0 0x7fa3740aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.335+0000 7fa373fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740b04a0 0x7fa3740c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.335+0000 7fa373fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3740b0930 con 0x7fa37415aab0 2023-10-08T05:52:56.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.335+0000 7fa373fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37415ae90 con 0x7fa3740caef0 2023-10-08T05:52:56.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.335+0000 7fa373fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740cbdb0 con 0x7fa3740b04a0 2023-10-08T05:52:56.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.335+0000 7fa372ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa37415aab0 0x7fa374070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.336+0000 7fa372ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa37415aab0 0x7fa374070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54592/0 (socket says 172.21.15.70:54592) 2023-10-08T05:52:56.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.336+0000 7fa372ffd640 1 -- 172.21.15.70:0/3193098708 learned_addr learned my addr 172.21.15.70:0/3193098708 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:56.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.336+0000 7fa3737fe640 1 --2- 172.21.15.70:0/3193098708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740b04a0 0x7fa3740c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.336+0000 7fa3727fc640 1 --2- 172.21.15.70:0/3193098708 >> v2:172.21.15.181:3300/0 conn(0x7fa3740caef0 0x7fa3740aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.336+0000 7fa372ffd640 1 -- 172.21.15.70:0/3193098708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740b04a0 msgr2=0x7fa3740c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:56.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.336+0000 7fa372ffd640 1 --2- 172.21.15.70:0/3193098708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740b04a0 0x7fa3740c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.336+0000 7fa372ffd640 1 -- 172.21.15.70:0/3193098708 >> v2:172.21.15.181:3300/0 conn(0x7fa3740caef0 msgr2=0x7fa3740aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:56.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.336+0000 7fa372ffd640 1 --2- 172.21.15.70:0/3193098708 >> v2:172.21.15.181:3300/0 conn(0x7fa3740caef0 0x7fa3740aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.336+0000 7fa372ffd640 1 -- 172.21.15.70:0/3193098708 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3740cb7f0 con 0x7fa37415aab0 2023-10-08T05:52:56.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.337+0000 7fa372ffd640 1 --2- 172.21.15.70:0/3193098708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa37415aab0 0x7fa374070b50 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa364009870 tx=0x7fa36402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58272b8786cfe54 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:56.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.337+0000 7fa371ffb640 1 -- 172.21.15.70:0/3193098708 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3640317c0 con 0x7fa37415aab0 2023-10-08T05:52:56.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.337+0000 7fa371ffb640 1 -- 172.21.15.70:0/3193098708 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa364031920 con 0x7fa37415aab0 2023-10-08T05:52:56.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.337+0000 7fa371ffb640 1 -- 172.21.15.70:0/3193098708 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364031cb0 con 0x7fa37415aab0 2023-10-08T05:52:56.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.338+0000 7fa373fff640 1 -- 172.21.15.70:0/3193098708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa37415aab0 msgr2=0x7fa374070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.338+0000 7fa373fff640 1 --2- 172.21.15.70:0/3193098708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa37415aab0 0x7fa374070b50 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa364009870 tx=0x7fa36402b860 comp rx=0 tx=0).stop 2023-10-08T05:52:56.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.338+0000 7fa373fff640 1 -- 172.21.15.70:0/3193098708 shutdown_connections 2023-10-08T05:52:56.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.338+0000 7fa373fff640 1 --2- 172.21.15.70:0/3193098708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3740b04a0 0x7fa3740c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.338+0000 7fa373fff640 1 --2- 172.21.15.70:0/3193098708 >> v2:172.21.15.181:3300/0 conn(0x7fa3740caef0 0x7fa3740aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.338+0000 7fa373fff640 1 --2- 172.21.15.70:0/3193098708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa37415aab0 0x7fa374070b50 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.338+0000 7fa373fff640 1 -- 172.21.15.70:0/3193098708 >> 172.21.15.70:0/3193098708 conn(0x7fa37406c010 msgr2=0x7fa37406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:56.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.338+0000 7fa373fff640 1 -- 172.21.15.70:0/3193098708 shutdown_connections 2023-10-08T05:52:56.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.338+0000 7fa373fff640 1 -- 172.21.15.70:0/3193098708 wait complete. 2023-10-08T05:52:56.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.340+0000 7fa373fff640 1 Processor -- start 2023-10-08T05:52:56.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.340+0000 7fa373fff640 1 -- start start 2023-10-08T05:52:56.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.341+0000 7fa373fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3740b04a0 0x7fa37414a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.341+0000 7fa373fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3740caef0 0x7fa37415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.341+0000 7fa373fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa37415aab0 0x7fa37415c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.341+0000 7fa373fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37414aef0 con 0x7fa3740b04a0 2023-10-08T05:52:56.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.341+0000 7fa373fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3741f37f0 con 0x7fa3740caef0 2023-10-08T05:52:56.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.341+0000 7fa373fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37415be20 con 0x7fa37415aab0 2023-10-08T05:52:56.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.341+0000 7fa372ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3740b04a0 0x7fa37414a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.341+0000 7fa372ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3740b04a0 0x7fa37414a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54594/0 (socket says 172.21.15.70:54594) 2023-10-08T05:52:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.341+0000 7fa372ffd640 1 -- 172.21.15.70:0/1256465184 learned_addr learned my addr 172.21.15.70:0/1256465184 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.342+0000 7fa3727fc640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3740caef0 0x7fa37415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.342+0000 7fa372ffd640 1 -- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa37415aab0 msgr2=0x7fa37415c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.342+0000 7fa3737fe640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa37415aab0 0x7fa37415c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.342+0000 7fa372ffd640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa37415aab0 0x7fa37415c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.342+0000 7fa372ffd640 1 -- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3740caef0 msgr2=0x7fa37415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.342+0000 7fa372ffd640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3740caef0 0x7fa37415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.342+0000 7fa372ffd640 1 -- 172.21.15.70:0/1256465184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3640094d0 con 0x7fa3740b04a0 2023-10-08T05:52:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.342+0000 7fa3727fc640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3740caef0 0x7fa37415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.342+0000 7fa372ffd640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3740b04a0 0x7fa37414a9b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa364000c00 tx=0x7fa3640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.343+0000 7fa35b7fe640 1 -- 172.21.15.70:0/1256465184 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364032860 con 0x7fa3740b04a0 2023-10-08T05:52:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.343+0000 7fa35b7fe640 1 -- 172.21.15.70:0/1256465184 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3640329e0 con 0x7fa3740b04a0 2023-10-08T05:52:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.343+0000 7fa35b7fe640 1 -- 172.21.15.70:0/1256465184 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364032d90 con 0x7fa3740b04a0 2023-10-08T05:52:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.343+0000 7fa373fff640 1 -- 172.21.15.70:0/1256465184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3741f31a0 con 0x7fa3740b04a0 2023-10-08T05:52:56.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.343+0000 7fa373fff640 1 -- 172.21.15.70:0/1256465184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3741f2ea0 con 0x7fa3740b04a0 2023-10-08T05:52:56.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.344+0000 7fa373fff640 1 -- 172.21.15.70:0/1256465184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa374074150 con 0x7fa3740b04a0 2023-10-08T05:52:56.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.345+0000 7fa35b7fe640 1 -- 172.21.15.70:0/1256465184 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa36404d3f0 con 0x7fa3740b04a0 2023-10-08T05:52:56.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.346+0000 7fa35b7fe640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa36006af30 0x7fa36006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.346+0000 7fa35b7fe640 1 -- 172.21.15.70:0/1256465184 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fa3640ab4f0 con 0x7fa3740b04a0 2023-10-08T05:52:56.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.347+0000 7fa3727fc640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa36006af30 0x7fa36006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.348+0000 7fa3727fc640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa36006af30 0x7fa36006d3f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa37415e1a0 tx=0x7fa368008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:56.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.351+0000 7fa35b7fe640 1 -- 172.21.15.70:0/1256465184 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa364076420 con 0x7fa3740b04a0 2023-10-08T05:52:56.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.540+0000 7fa373fff640 1 -- 172.21.15.70:0/1256465184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3740afe60 con 0x7fa3740b04a0 2023-10-08T05:52:56.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.541+0000 7fa35b7fe640 1 -- 172.21.15.70:0/1256465184 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa364079a40 con 0x7fa3740b04a0 2023-10-08T05:52:56.543 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:52:56.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 -- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa36006af30 msgr2=0x7fa36006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa36006af30 0x7fa36006d3f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa37415e1a0 tx=0x7fa368008040 comp rx=0 tx=0).stop 2023-10-08T05:52:56.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 -- 172.21.15.70:0/1256465184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3740b04a0 msgr2=0x7fa37414a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3740b04a0 0x7fa37414a9b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa364000c00 tx=0x7fa3640046c0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 -- 172.21.15.70:0/1256465184 shutdown_connections 2023-10-08T05:52:56.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fa36006af30 0x7fa36006d3f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa37415aab0 0x7fa37415c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3740caef0 0x7fa37415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 --2- 172.21.15.70:0/1256465184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3740b04a0 0x7fa37414a9b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 -- 172.21.15.70:0/1256465184 >> 172.21.15.70:0/1256465184 conn(0x7fa37406c010 msgr2=0x7fa3740cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:56.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 -- 172.21.15.70:0/1256465184 shutdown_connections 2023-10-08T05:52:56.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.544+0000 7fa373fff640 1 -- 172.21.15.70:0/1256465184 wait complete. 2023-10-08T05:52:56.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-08T05:52:56.772 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:56.770+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:56.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.772+0000 7f22469a7640 1 Processor -- start 2023-10-08T05:52:56.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.773+0000 7f22469a7640 1 -- start start 2023-10-08T05:52:56.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.774+0000 7f22469a7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f224015ab30 0x7f224015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.774+0000 7f22469a7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22400b07d0 0x7f224006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.774+0000 7f22469a7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224015f7d0 con 0x7f22400adc70 2023-10-08T05:52:56.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.774+0000 7f22469a7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f224015f210 con 0x7f22400b07d0 2023-10-08T05:52:56.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.774+0000 7f22469a7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224015f4f0 con 0x7f224015ab30 2023-10-08T05:52:56.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.775+0000 7f22459a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f224015ab30 0x7f224015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.775+0000 7f22461a6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22400b07d0 0x7f224006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.775+0000 7f22459a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f224015ab30 0x7f224015e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:48032/0 (socket says 172.21.15.70:48032) 2023-10-08T05:52:56.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.775+0000 7f22459a5640 1 -- 172.21.15.70:0/2489423879 learned_addr learned my addr 172.21.15.70:0/2489423879 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:56.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.776+0000 7f22459a5640 1 -- 172.21.15.70:0/2489423879 >> v2:172.21.15.181:3300/0 conn(0x7f22400b07d0 msgr2=0x7f224006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:56.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.776+0000 7f22459a5640 1 --2- 172.21.15.70:0/2489423879 >> v2:172.21.15.181:3300/0 conn(0x7f22400b07d0 0x7f224006cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.776+0000 7f22459a5640 1 -- 172.21.15.70:0/2489423879 >> v1:172.21.15.181:6789/0 conn(0x7f22400adc70 legacy=0x7f22400b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.776+0000 7f22459a5640 1 -- 172.21.15.70:0/2489423879 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224006c550 con 0x7f224015ab30 2023-10-08T05:52:56.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.777+0000 7f22459a5640 1 --2- 172.21.15.70:0/2489423879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f224015ab30 0x7f224015e8a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2234009e50 tx=0x7f223402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c4f1f5b4bec58a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:56.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.777+0000 7f22449a3640 1 -- 172.21.15.70:0/2489423879 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223402cb00 con 0x7f224015ab30 2023-10-08T05:52:56.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.777+0000 7f22449a3640 1 -- 172.21.15.70:0/2489423879 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223402cc60 con 0x7f224015ab30 2023-10-08T05:52:56.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.777+0000 7f22449a3640 1 -- 172.21.15.70:0/2489423879 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340343f0 con 0x7f224015ab30 2023-10-08T05:52:56.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.778+0000 7f22469a7640 1 -- 172.21.15.70:0/2489423879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f224015ab30 msgr2=0x7f224015e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.778+0000 7f22469a7640 1 --2- 172.21.15.70:0/2489423879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f224015ab30 0x7f224015e8a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2234009e50 tx=0x7f223402bf60 comp rx=0 tx=0).stop 2023-10-08T05:52:56.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.778+0000 7f22469a7640 1 -- 172.21.15.70:0/2489423879 shutdown_connections 2023-10-08T05:52:56.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.778+0000 7f22469a7640 1 --2- 172.21.15.70:0/2489423879 >> v2:172.21.15.181:3300/0 conn(0x7f22400b07d0 0x7f224006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.778+0000 7f22469a7640 1 --2- 172.21.15.70:0/2489423879 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f224015ab30 0x7f224015e8a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.778+0000 7f22469a7640 1 -- 172.21.15.70:0/2489423879 >> 172.21.15.70:0/2489423879 conn(0x7f22400cb0c0 msgr2=0x7f22400cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:56.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.778+0000 7f22469a7640 1 -- 172.21.15.70:0/2489423879 shutdown_connections 2023-10-08T05:52:56.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.778+0000 7f22469a7640 1 -- 172.21.15.70:0/2489423879 wait complete. 2023-10-08T05:52:56.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.780+0000 7f22469a7640 1 Processor -- start 2023-10-08T05:52:56.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.780+0000 7f22469a7640 1 -- start start 2023-10-08T05:52:56.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.781+0000 7f22469a7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22400adc70 0x7f22400c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.781+0000 7f22469a7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22400b07d0 0x7f22400c2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.781+0000 7f22469a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224015ab30 0x7f224014fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.781+0000 7f22469a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224014ffc0 con 0x7f224015ab30 2023-10-08T05:52:56.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.781+0000 7f22469a7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22400b2a20 con 0x7f22400adc70 2023-10-08T05:52:56.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.781+0000 7f22469a7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22400b2d00 con 0x7f22400b07d0 2023-10-08T05:52:56.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.782+0000 7f22461a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224015ab30 0x7f224014fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.782+0000 7f22461a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224015ab30 0x7f224014fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54600/0 (socket says 172.21.15.70:54600) 2023-10-08T05:52:56.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.782+0000 7f22461a6640 1 -- 172.21.15.70:0/1798999839 learned_addr learned my addr 172.21.15.70:0/1798999839 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:56.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.782+0000 7f22459a5640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22400adc70 0x7f22400c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.782+0000 7f22451a4640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22400b07d0 0x7f22400c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.782+0000 7f22461a6640 1 -- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22400b07d0 msgr2=0x7f22400c2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.782+0000 7f22461a6640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22400b07d0 0x7f22400c2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.782+0000 7f22461a6640 1 -- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22400adc70 msgr2=0x7f22400c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:56.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.782+0000 7f22461a6640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22400adc70 0x7f22400c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:56.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.782+0000 7f22461a6640 1 -- 172.21.15.70:0/1798999839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2234009b30 con 0x7f224015ab30 2023-10-08T05:52:56.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.783+0000 7f22461a6640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224015ab30 0x7f224014fa80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f223800e930 tx=0x7f223800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:56.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.783+0000 7f223e7fc640 1 -- 172.21.15.70:0/1798999839 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238009690 con 0x7f224015ab30 2023-10-08T05:52:56.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.783+0000 7f223e7fc640 1 -- 172.21.15.70:0/1798999839 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223800c980 con 0x7f224015ab30 2023-10-08T05:52:56.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.783+0000 7f22469a7640 1 -- 172.21.15.70:0/1798999839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22400b2fe0 con 0x7f224015ab30 2023-10-08T05:52:56.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.783+0000 7f223e7fc640 1 -- 172.21.15.70:0/1798999839 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223800cd30 con 0x7f224015ab30 2023-10-08T05:52:56.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.783+0000 7f22469a7640 1 -- 172.21.15.70:0/1798999839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22400c3cc0 con 0x7f224015ab30 2023-10-08T05:52:56.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.784+0000 7f22469a7640 1 -- 172.21.15.70:0/1798999839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22400b3220 con 0x7f224015ab30 2023-10-08T05:52:56.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.785+0000 7f223e7fc640 1 -- 172.21.15.70:0/1798999839 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2238013470 con 0x7f224015ab30 2023-10-08T05:52:56.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.786+0000 7f223e7fc640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f222406af00 0x7f222406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:56.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.787+0000 7f223e7fc640 1 -- 172.21.15.70:0/1798999839 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f2238089a60 con 0x7f224015ab30 2023-10-08T05:52:56.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.787+0000 7f22459a5640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f222406af00 0x7f222406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:56.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.788+0000 7f22459a5640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f222406af00 0x7f222406d3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2234035000 tx=0x7f2234030000 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:56.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:56.791+0000 7f223e7fc640 1 -- 172.21.15.70:0/1798999839 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22400b3220 con 0x7f224015ab30 2023-10-08T05:52:57.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.050+0000 7f22469a7640 1 -- 172.21.15.70:0/1798999839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f22400b3220 con 0x7f224015ab30 2023-10-08T05:52:57.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.667+0000 7f223e7fc640 1 -- 172.21.15.70:0/1798999839 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v627) v1 ==== 169+0+0 (secure 0 0 0) 0x7f22400b3220 con 0x7f224015ab30 2023-10-08T05:52:57.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 -- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f222406af00 msgr2=0x7f222406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:57.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f222406af00 0x7f222406d3c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2234035000 tx=0x7f2234030000 comp rx=0 tx=0).stop 2023-10-08T05:52:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 -- 172.21.15.70:0/1798999839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224015ab30 msgr2=0x7f224014fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224015ab30 0x7f224014fa80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f223800e930 tx=0x7f223800edf0 comp rx=0 tx=0).stop 2023-10-08T05:52:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 -- 172.21.15.70:0/1798999839 shutdown_connections 2023-10-08T05:52:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f222406af00 0x7f222406d3c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224015ab30 0x7f224014fa80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22400b07d0 0x7f22400c2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 --2- 172.21.15.70:0/1798999839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22400adc70 0x7f22400c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 -- 172.21.15.70:0/1798999839 >> 172.21.15.70:0/1798999839 conn(0x7f22400cb0c0 msgr2=0x7f2240159280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 -- 172.21.15.70:0/1798999839 shutdown_connections 2023-10-08T05:52:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.671+0000 7f22469a7640 1 -- 172.21.15.70:0/1798999839 wait complete. 2023-10-08T05:52:57.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T05:52:57.733 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:57.731+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:57.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.904+0000 7fbccaa4b640 1 Processor -- start 2023-10-08T05:52:57.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.904+0000 7fbccaa4b640 1 -- start start 2023-10-08T05:52:57.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.906+0000 7fbccaa4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc406c3a0 0x7fbcc40c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:57.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.906+0000 7fbccaa4b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcc415aa00 0x7fbcc40b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:57.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.906+0000 7fbccaa4b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc415b310 con 0x7fbcc406db60 2023-10-08T05:52:57.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.906+0000 7fbccaa4b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc40ca230 con 0x7fbcc415aa00 2023-10-08T05:52:57.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.906+0000 7fbccaa4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc406d290 con 0x7fbcc406c3a0 2023-10-08T05:52:57.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.907+0000 7fbcc8fe8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcc415aa00 0x7fbcc40b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:57.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.907+0000 7fbcc3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc406c3a0 0x7fbcc40c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:57.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.907+0000 7fbcc8fe8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbcc415aa00 0x7fbcc40b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:37370/0 (socket says 172.21.15.70:37370) 2023-10-08T05:52:57.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.907+0000 7fbcc3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc406c3a0 0x7fbcc40c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:48058/0 (socket says 172.21.15.70:48058) 2023-10-08T05:52:57.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.907+0000 7fbcc8fe8640 1 -- 172.21.15.70:0/1014364018 learned_addr learned my addr 172.21.15.70:0/1014364018 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:57.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.908+0000 7fbcc8fe8640 1 -- 172.21.15.70:0/1014364018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc406c3a0 msgr2=0x7fbcc40c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:57.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.908+0000 7fbcc8fe8640 1 --2- 172.21.15.70:0/1014364018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc406c3a0 0x7fbcc40c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:57.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.908+0000 7fbcc8fe8640 1 -- 172.21.15.70:0/1014364018 >> v1:172.21.15.181:6789/0 conn(0x7fbcc406db60 legacy=0x7fbcc406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:57.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.908+0000 7fbcc8fe8640 1 -- 172.21.15.70:0/1014364018 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc40c9860 con 0x7fbcc415aa00 2023-10-08T05:52:57.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.909+0000 7fbcc8fe8640 1 --2- 172.21.15.70:0/1014364018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415aa00 0x7fbcc40b2a90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00bb10 tx=0x7fbcbc02e150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60e6258247ba0937 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:57.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbcc37fe640 1 -- 172.21.15.70:0/1014364018 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc036440 con 0x7fbcc415aa00 2023-10-08T05:52:57.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbcc37fe640 1 -- 172.21.15.70:0/1014364018 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc0365a0 con 0x7fbcc415aa00 2023-10-08T05:52:57.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbcc37fe640 1 -- 172.21.15.70:0/1014364018 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc036930 con 0x7fbcc415aa00 2023-10-08T05:52:57.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbccaa4b640 1 -- 172.21.15.70:0/1014364018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415aa00 msgr2=0x7fbcc40b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbccaa4b640 1 --2- 172.21.15.70:0/1014364018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415aa00 0x7fbcc40b2a90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00bb10 tx=0x7fbcbc02e150 comp rx=0 tx=0).stop 2023-10-08T05:52:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbccaa4b640 1 -- 172.21.15.70:0/1014364018 shutdown_connections 2023-10-08T05:52:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbccaa4b640 1 --2- 172.21.15.70:0/1014364018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc415aa00 0x7fbcc40b2a90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbccaa4b640 1 --2- 172.21.15.70:0/1014364018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc406c3a0 0x7fbcc40c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbccaa4b640 1 -- 172.21.15.70:0/1014364018 >> 172.21.15.70:0/1014364018 conn(0x7fbcc406a5d0 msgr2=0x7fbcc40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbccaa4b640 1 -- 172.21.15.70:0/1014364018 shutdown_connections 2023-10-08T05:52:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.910+0000 7fbccaa4b640 1 -- 172.21.15.70:0/1014364018 wait complete. 2023-10-08T05:52:57.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.912+0000 7fbccaa4b640 1 Processor -- start 2023-10-08T05:52:57.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.912+0000 7fbccaa4b640 1 -- start start 2023-10-08T05:52:57.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.913+0000 7fbccaa4b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc406c3a0 0x7fbcc40c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:57.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbccaa4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc406db60 0x7fbcc40c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbccaa4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc415aa00 0x7fbcc40c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbccaa4b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc40c51f0 con 0x7fbcc406db60 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbccaa4b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc40c4c30 con 0x7fbcc406c3a0 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbccaa4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc40c4f10 con 0x7fbcc415aa00 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbcbb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc406db60 0x7fbcc40c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbcbb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc406db60 0x7fbcc40c40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54604/0 (socket says 172.21.15.70:54604) 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbcbb7fe640 1 -- 172.21.15.70:0/2909198738 learned_addr learned my addr 172.21.15.70:0/2909198738 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbcc3fff640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc406c3a0 0x7fbcc40c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbcc8fe8640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc415aa00 0x7fbcc40c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbcbb7fe640 1 -- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc415aa00 msgr2=0x7fbcc40c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:57.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbcbb7fe640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc415aa00 0x7fbcc40c4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:57.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbcbb7fe640 1 -- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc406c3a0 msgr2=0x7fbcc40c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:57.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbcbb7fe640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc406c3a0 0x7fbcc40c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:57.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.914+0000 7fbcbb7fe640 1 -- 172.21.15.70:0/2909198738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc00b770 con 0x7fbcc406db60 2023-10-08T05:52:57.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.915+0000 7fbcbb7fe640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc406db60 0x7fbcc40c40e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00c450 tx=0x7fbcac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:57.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.915+0000 7fbcc17fa640 1 -- 172.21.15.70:0/2909198738 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0052d0 con 0x7fbcc406db60 2023-10-08T05:52:57.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.915+0000 7fbcc17fa640 1 -- 172.21.15.70:0/2909198738 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac005430 con 0x7fbcc406db60 2023-10-08T05:52:57.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.915+0000 7fbcc17fa640 1 -- 172.21.15.70:0/2909198738 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac012870 con 0x7fbcc406db60 2023-10-08T05:52:57.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.915+0000 7fbccaa4b640 1 -- 172.21.15.70:0/2909198738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc40c6a00 con 0x7fbcc406db60 2023-10-08T05:52:57.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.915+0000 7fbccaa4b640 1 -- 172.21.15.70:0/2909198738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc414b880 con 0x7fbcc406db60 2023-10-08T05:52:57.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.916+0000 7fbccaa4b640 1 -- 172.21.15.70:0/2909198738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc40c6860 con 0x7fbcc406db60 2023-10-08T05:52:57.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.917+0000 7fbcc17fa640 1 -- 172.21.15.70:0/2909198738 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fbcac0129d0 con 0x7fbcc406db60 2023-10-08T05:52:57.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.918+0000 7fbcc17fa640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fbca406af00 0x7fbca406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:57.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.918+0000 7fbcc17fa640 1 -- 172.21.15.70:0/2909198738 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7fbcac089180 con 0x7fbcc406db60 2023-10-08T05:52:57.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.919+0000 7fbcc3fff640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fbca406af00 0x7fbca406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:57.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.920+0000 7fbcc3fff640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fbca406af00 0x7fbca406d3c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40074a0 tx=0x7fbcb400b040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:57.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:57.923+0000 7fbcc17fa640 1 -- 172.21.15.70:0/2909198738 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbcc40c6860 con 0x7fbcc406db60 2023-10-08T05:52:58.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.119+0000 7fbccaa4b640 1 -- 172.21.15.70:0/2909198738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbcc415b310 con 0x7fbcc406db60 2023-10-08T05:52:58.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.126+0000 7fbcc17fa640 1 -- 172.21.15.70:0/2909198738 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbcc415b310 con 0x7fbcc406db60 2023-10-08T05:52:58.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.129+0000 7fbccaa4b640 1 -- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fbca406af00 msgr2=0x7fbca406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:58.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.129+0000 7fbccaa4b640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fbca406af00 0x7fbca406d3c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40074a0 tx=0x7fbcb400b040 comp rx=0 tx=0).stop 2023-10-08T05:52:58.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.129+0000 7fbccaa4b640 1 -- 172.21.15.70:0/2909198738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc406db60 msgr2=0x7fbcc40c40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:58.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.129+0000 7fbccaa4b640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc406db60 0x7fbcc40c40e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00c450 tx=0x7fbcac00c910 comp rx=0 tx=0).stop 2023-10-08T05:52:58.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.129+0000 7fbccaa4b640 1 -- 172.21.15.70:0/2909198738 shutdown_connections 2023-10-08T05:52:58.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.129+0000 7fbccaa4b640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fbca406af00 0x7fbca406d3c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:58.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.129+0000 7fbccaa4b640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbcc415aa00 0x7fbcc40c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:58.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.129+0000 7fbccaa4b640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcc406db60 0x7fbcc40c40e0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:58.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.129+0000 7fbccaa4b640 1 --2- 172.21.15.70:0/2909198738 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbcc406c3a0 0x7fbcc40c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:58.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.129+0000 7fbccaa4b640 1 -- 172.21.15.70:0/2909198738 >> 172.21.15.70:0/2909198738 conn(0x7fbcc406a5d0 msgr2=0x7fbcc40caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:58.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.130+0000 7fbccaa4b640 1 -- 172.21.15.70:0/2909198738 shutdown_connections 2023-10-08T05:52:58.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.130+0000 7fbccaa4b640 1 -- 172.21.15.70:0/2909198738 wait complete. 2023-10-08T05:52:58.148 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:52:58.148 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:52:58.255 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:52:58.390 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:52:58.390 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:58.390 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:58.446 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:52:58.446 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:52:58.446 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:58.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:52:58.506 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:52:58.506 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:58.566 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:52:58.566 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:52:58.566 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:52:58.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:52:58.627 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T05:52:58.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:52:58.644 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:58.643+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:58.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.841+0000 7f7915b1b640 1 Processor -- start 2023-10-08T05:52:58.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.842+0000 7f7915b1b640 1 -- start start 2023-10-08T05:52:58.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.843+0000 7f7915b1b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f791015caf0 0x7f79100cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:58.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.843+0000 7f7915b1b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f79100d0a00 0x7f79100cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:58.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.843+0000 7f7915b1b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f791015ced0 con 0x7f79100d1eb0 2023-10-08T05:52:58.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.843+0000 7f7915b1b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79100d2470 con 0x7f79100d0a00 2023-10-08T05:52:58.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.843+0000 7f7915b1b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79100d2750 con 0x7f791015caf0 2023-10-08T05:52:58.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.843+0000 7f790ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f79100d0a00 0x7f79100cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:58.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.843+0000 7f790ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f79100d0a00 0x7f79100cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54606/0 (socket says 172.21.15.70:54606) 2023-10-08T05:52:58.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.843+0000 7f790ffff640 1 -- 172.21.15.70:0/1948539080 learned_addr learned my addr 172.21.15.70:0/1948539080 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:58.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.843+0000 7f791531a640 1 --1- 172.21.15.70:0/1948539080 >> v1:172.21.15.70:6789/0 conn(0x7f79100d1eb0 0x7f79100b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48184/0 (socket says 172.21.15.70:48184) 2023-10-08T05:52:58.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.844+0000 7f7914b19640 1 --2- 172.21.15.70:0/1948539080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f791015caf0 0x7f79100cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:58.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.844+0000 7f790ffff640 1 -- 172.21.15.70:0/1948539080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f791015caf0 msgr2=0x7f79100cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:58.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.844+0000 7f790ffff640 1 --2- 172.21.15.70:0/1948539080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f791015caf0 0x7f79100cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:58.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.844+0000 7f790ffff640 1 -- 172.21.15.70:0/1948539080 >> v1:172.21.15.70:6789/0 conn(0x7f79100d1eb0 legacy=0x7f79100b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:58.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.844+0000 7f790ffff640 1 -- 172.21.15.70:0/1948539080 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79100d2a30 con 0x7f79100d0a00 2023-10-08T05:52:58.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.844+0000 7f790f7fe640 1 -- 172.21.15.70:0/1948539080 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766067021 0 0) 0x7f791015ced0 con 0x7f79100d1eb0 2023-10-08T05:52:58.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.845+0000 7f790ffff640 1 --2- 172.21.15.70:0/1948539080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79100d0a00 0x7f79100cd710 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7904009870 tx=0x7f790402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad121a564acde9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:58.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.845+0000 7f790f7fe640 1 -- 172.21.15.70:0/1948539080 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904030840 con 0x7f79100d0a00 2023-10-08T05:52:58.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.845+0000 7f790f7fe640 1 -- 172.21.15.70:0/1948539080 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79040309a0 con 0x7f79100d0a00 2023-10-08T05:52:58.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.845+0000 7f790f7fe640 1 -- 172.21.15.70:0/1948539080 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904030d30 con 0x7f79100d0a00 2023-10-08T05:52:58.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.846+0000 7f7915b1b640 1 -- 172.21.15.70:0/1948539080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79100d0a00 msgr2=0x7f79100cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:58.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.846+0000 7f7915b1b640 1 --2- 172.21.15.70:0/1948539080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79100d0a00 0x7f79100cd710 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f7904009870 tx=0x7f790402b980 comp rx=0 tx=0).stop 2023-10-08T05:52:58.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.846+0000 7f7915b1b640 1 -- 172.21.15.70:0/1948539080 shutdown_connections 2023-10-08T05:52:58.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.846+0000 7f7915b1b640 1 --2- 172.21.15.70:0/1948539080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79100d0a00 0x7f79100cd710 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:58.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.846+0000 7f7915b1b640 1 --2- 172.21.15.70:0/1948539080 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f791015caf0 0x7f79100cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:58.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.846+0000 7f7915b1b640 1 -- 172.21.15.70:0/1948539080 >> 172.21.15.70:0/1948539080 conn(0x7f7910071870 msgr2=0x7f7910071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:58.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.846+0000 7f7915b1b640 1 -- 172.21.15.70:0/1948539080 shutdown_connections 2023-10-08T05:52:58.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.846+0000 7f7915b1b640 1 -- 172.21.15.70:0/1948539080 wait complete. 2023-10-08T05:52:58.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.848+0000 7f7915b1b640 1 Processor -- start 2023-10-08T05:52:58.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.848+0000 7f7915b1b640 1 -- start start 2023-10-08T05:52:58.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.849+0000 7f7915b1b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79100d0a00 0x7f79101f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:58.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.849+0000 7f7915b1b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79100d1eb0 0x7f79100ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:58.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.849+0000 7f7915b1b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f791015caf0 0x7f79100abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:58.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.849+0000 7f7915b1b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79100cb3f0 con 0x7f791015caf0 2023-10-08T05:52:58.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.849+0000 7f7915b1b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79100a9bc0 con 0x7f79100d1eb0 2023-10-08T05:52:58.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.849+0000 7f7915b1b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79100cb110 con 0x7f79100d0a00 2023-10-08T05:52:58.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.849+0000 7f791531a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f791015caf0 0x7f79100abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:58.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.849+0000 7f791531a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f791015caf0 0x7f79100abd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54614/0 (socket says 172.21.15.70:54614) 2023-10-08T05:52:58.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.849+0000 7f791531a640 1 -- 172.21.15.70:0/3810793733 learned_addr learned my addr 172.21.15.70:0/3810793733 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:58.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f7914b19640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79100d0a00 0x7f79101f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:58.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f790ffff640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79100d1eb0 0x7f79100ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:58.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f791531a640 1 -- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79100d0a00 msgr2=0x7f79101f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:58.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f791531a640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79100d0a00 0x7f79101f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:58.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f791531a640 1 -- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79100d1eb0 msgr2=0x7f79100ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:58.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f791531a640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79100d1eb0 0x7f79100ab850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:58.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f791531a640 1 -- 172.21.15.70:0/3810793733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79040094d0 con 0x7f791015caf0 2023-10-08T05:52:58.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f7914b19640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79100d0a00 0x7f79101f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:58.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f791531a640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f791015caf0 0x7f79100abd90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f790800c450 tx=0x7f790800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:58.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f790d7fa640 1 -- 172.21.15.70:0/3810793733 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79080074e0 con 0x7f791015caf0 2023-10-08T05:52:58.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f790d7fa640 1 -- 172.21.15.70:0/3810793733 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7908007660 con 0x7f791015caf0 2023-10-08T05:52:58.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f790d7fa640 1 -- 172.21.15.70:0/3810793733 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908004a20 con 0x7f791015caf0 2023-10-08T05:52:58.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.850+0000 7f7915b1b640 1 -- 172.21.15.70:0/3810793733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79100a9500 con 0x7f791015caf0 2023-10-08T05:52:58.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.851+0000 7f7915b1b640 1 -- 172.21.15.70:0/3810793733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79100ca450 con 0x7f791015caf0 2023-10-08T05:52:58.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.852+0000 7f7915b1b640 1 -- 172.21.15.70:0/3810793733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79100a9360 con 0x7f791015caf0 2023-10-08T05:52:58.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.853+0000 7f790d7fa640 1 -- 172.21.15.70:0/3810793733 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f7908004b80 con 0x7f791015caf0 2023-10-08T05:52:58.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.854+0000 7f790d7fa640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f78fc06af00 0x7f78fc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:58.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.854+0000 7f790d7fa640 1 -- 172.21.15.70:0/3810793733 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7f7908088960 con 0x7f791015caf0 2023-10-08T05:52:58.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.854+0000 7f7914b19640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f78fc06af00 0x7f78fc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:58.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.855+0000 7f7914b19640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f78fc06af00 0x7f78fc06d3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7900007330 tx=0x7f790000b040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:58.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:58.858+0000 7f790d7fa640 1 -- 172.21.15.70:0/3810793733 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79100a9360 con 0x7f791015caf0 2023-10-08T05:52:59.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.091+0000 7f7915b1b640 1 -- 172.21.15.70:0/3810793733 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79100a9d00 con 0x7f791015caf0 2023-10-08T05:52:59.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.092+0000 7f790d7fa640 1 -- 172.21.15.70:0/3810793733 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f79100a9d00 con 0x7f791015caf0 2023-10-08T05:52:59.094 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:52:59.094 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5608,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1038137327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1038137327},{"type":"v1","addr":"172.21.15.181:6835","nonce":1038137327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5824,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/273560382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":273560382},{"type":"v1","addr":"172.21.15.181:6837","nonce":273560382}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6660,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1967783419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1967783419},{"type":"v1","addr":"172.21.15.70:6833","nonce":1967783419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6663,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1304760341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1304760341},{"type":"v1","addr":"172.21.15.70:6835","nonce":1304760341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2023-10-08T05:52:59.094 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 35 2023-10-08T05:52:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.095+0000 7f7915b1b640 1 -- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f78fc06af00 msgr2=0x7f78fc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.095+0000 7f7915b1b640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f78fc06af00 0x7f78fc06d3c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7900007330 tx=0x7f790000b040 comp rx=0 tx=0).stop 2023-10-08T05:52:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.095+0000 7f7915b1b640 1 -- 172.21.15.70:0/3810793733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f791015caf0 msgr2=0x7f79100abd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.095+0000 7f7915b1b640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f791015caf0 0x7f79100abd90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f790800c450 tx=0x7f790800c910 comp rx=0 tx=0).stop 2023-10-08T05:52:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.095+0000 7f7915b1b640 1 -- 172.21.15.70:0/3810793733 shutdown_connections 2023-10-08T05:52:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.095+0000 7f7915b1b640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f78fc06af00 0x7f78fc06d3c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.095+0000 7f7915b1b640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f791015caf0 0x7f79100abd90 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.095+0000 7f7915b1b640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79100d1eb0 0x7f79100ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.095+0000 7f7915b1b640 1 --2- 172.21.15.70:0/3810793733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79100d0a00 0x7f79101f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.095+0000 7f7915b1b640 1 -- 172.21.15.70:0/3810793733 >> 172.21.15.70:0/3810793733 conn(0x7f7910071870 msgr2=0x7f79100b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:59.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.096+0000 7f7915b1b640 1 -- 172.21.15.70:0/3810793733 shutdown_connections 2023-10-08T05:52:59.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.096+0000 7f7915b1b640 1 -- 172.21.15.70:0/3810793733 wait complete. 2023-10-08T05:52:59.115 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T05:52:59.115 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T05:52:59.116 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T05:52:59.116 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:52:59.116 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T05:52:59.116 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T05:52:59.116 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T05:52:59.116 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:52:59.117 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T05:52:59.117 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T05:52:59.117 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T05:52:59.117 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:52:59.118 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T05:52:59.118 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T05:52:59.118 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T05:52:59.118 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:52:59.138 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T05:52:59.138 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T05:52:59.143 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T05:52:59.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T05:52:59.154 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T05:52:59.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T05:52:59.159 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T05:52:59.159 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T05:52:59.203 INFO:tasks.ceph.mds.a:Started 2023-10-08T05:52:59.204 DEBUG:teuthology.parallel:result is None 2023-10-08T05:52:59.207 INFO:tasks.ceph.mds.b:Started 2023-10-08T05:52:59.207 DEBUG:teuthology.parallel:result is None 2023-10-08T05:52:59.212 INFO:tasks.ceph.mds.c:Started 2023-10-08T05:52:59.212 DEBUG:teuthology.parallel:result is None 2023-10-08T05:52:59.214 INFO:tasks.ceph.mds.d:Started 2023-10-08T05:52:59.215 DEBUG:teuthology.parallel:result is None 2023-10-08T05:52:59.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T05:52:59.368 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T05:52:59.375 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:52:59.376 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T05:52:59.377 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:52:59.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.412+0000 7ff9b8090640 1 Processor -- start 2023-10-08T05:52:59.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.412+0000 7ff9b8090640 1 -- start start 2023-10-08T05:52:59.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.413+0000 7ff9b8090640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9b00cea30 0x7ff9b015d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:59.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.413+0000 7ff9b8090640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b00ccb40 0x7ff9b00ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:59.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.413+0000 7ff9b8090640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b00cd490 con 0x7ff9b015cdb0 2023-10-08T05:52:59.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.414+0000 7ff9b8090640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b006b6a0 con 0x7ff9b00cea30 2023-10-08T05:52:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.413+0000 7ff9b562b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9b00cea30 0x7ff9b015d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.414+0000 7ff9b8090640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b19e0 con 0x7ff9b00ccb40 2023-10-08T05:52:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.414+0000 7ff9b5e2c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff9b015cdb0 0x7ff9b006af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:57660/0 (socket says 172.21.15.70:57660) 2023-10-08T05:52:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.414+0000 7ff9b5e2c640 1 -- 172.21.15.70:0/3047248935 learned_addr learned my addr 172.21.15.70:0/3047248935 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:52:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.414+0000 7ff9b662d640 1 --2- 172.21.15.70:0/3047248935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b00ccb40 0x7ff9b00ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.414+0000 7ff9b662d640 1 -- 172.21.15.70:0/3047248935 >> v2:172.21.15.70:3300/0 conn(0x7ff9b00cea30 msgr2=0x7ff9b015d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:52:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.414+0000 7ff9b662d640 1 --2- 172.21.15.70:0/3047248935 >> v2:172.21.15.70:3300/0 conn(0x7ff9b00cea30 0x7ff9b015d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.414+0000 7ff9b662d640 1 -- 172.21.15.70:0/3047248935 >> v1:172.21.15.181:6789/0 conn(0x7ff9b015cdb0 legacy=0x7ff9b006af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.414+0000 7ff9b662d640 1 -- 172.21.15.70:0/3047248935 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b0069ef0 con 0x7ff9b00ccb40 2023-10-08T05:52:59.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.414+0000 7ff9b562b640 1 --2- 172.21.15.70:0/3047248935 >> v2:172.21.15.70:3300/0 conn(0x7ff9b00cea30 0x7ff9b015d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:52:59.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.415+0000 7ff9b662d640 1 --2- 172.21.15.70:0/3047248935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b00ccb40 0x7ff9b00ccf50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009d90 tx=0x7ff9a802bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3fc38cf97fe66e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:59.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b4e2a640 1 -- 172.21.15.70:0/3047248935 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a802ca10 con 0x7ff9b00ccb40 2023-10-08T05:52:59.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b4e2a640 1 -- 172.21.15.70:0/3047248935 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a802cb70 con 0x7ff9b00ccb40 2023-10-08T05:52:59.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b4e2a640 1 -- 172.21.15.70:0/3047248935 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8034440 con 0x7ff9b00ccb40 2023-10-08T05:52:59.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b8090640 1 -- 172.21.15.70:0/3047248935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b00ccb40 msgr2=0x7ff9b00ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:59.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b8090640 1 --2- 172.21.15.70:0/3047248935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b00ccb40 0x7ff9b00ccf50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009d90 tx=0x7ff9a802bce0 comp rx=0 tx=0).stop 2023-10-08T05:52:59.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b8090640 1 -- 172.21.15.70:0/3047248935 shutdown_connections 2023-10-08T05:52:59.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b8090640 1 --2- 172.21.15.70:0/3047248935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b00ccb40 0x7ff9b00ccf50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:59.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b8090640 1 --2- 172.21.15.70:0/3047248935 >> v2:172.21.15.70:3300/0 conn(0x7ff9b00cea30 0x7ff9b015d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:59.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b8090640 1 -- 172.21.15.70:0/3047248935 >> 172.21.15.70:0/3047248935 conn(0x7ff9b00b0af0 msgr2=0x7ff9b00b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:52:59.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b8090640 1 -- 172.21.15.70:0/3047248935 shutdown_connections 2023-10-08T05:52:59.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.416+0000 7ff9b8090640 1 -- 172.21.15.70:0/3047248935 wait complete. 2023-10-08T05:52:59.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.418+0000 7ff9b8090640 1 Processor -- start 2023-10-08T05:52:59.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.418+0000 7ff9b8090640 1 -- start start 2023-10-08T05:52:59.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.419+0000 7ff9b8090640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b00ccb40 0x7ff9b00b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:59.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.419+0000 7ff9b8090640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b00cea30 0x7ff9b01f2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:59.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.419+0000 7ff9b8090640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b015cdb0 0x7ff9b00b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:59.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.419+0000 7ff9b8090640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b4a00 con 0x7ff9b00cea30 2023-10-08T05:52:59.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.419+0000 7ff9b8090640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b01f3580 con 0x7ff9b00ccb40 2023-10-08T05:52:59.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.419+0000 7ff9b8090640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b2da0 con 0x7ff9b015cdb0 2023-10-08T05:52:59.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.419+0000 7ff9b562b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b00cea30 0x7ff9b01f2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:59.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.419+0000 7ff9b562b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b00cea30 0x7ff9b01f2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54656/0 (socket says 172.21.15.70:54656) 2023-10-08T05:52:59.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.419+0000 7ff9b562b640 1 -- 172.21.15.70:0/1633531441 learned_addr learned my addr 172.21.15.70:0/1633531441 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:52:59.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b662d640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b015cdb0 0x7ff9b00b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:59.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b5e2c640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b00ccb40 0x7ff9b00b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:59.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b562b640 1 -- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b015cdb0 msgr2=0x7ff9b00b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:59.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b562b640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b015cdb0 0x7ff9b00b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:59.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b562b640 1 -- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b00ccb40 msgr2=0x7ff9b00b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:52:59.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b562b640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b00ccb40 0x7ff9b00b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:52:59.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b562b640 1 -- 172.21.15.70:0/1633531441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0009400 con 0x7ff9b00cea30 2023-10-08T05:52:59.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b662d640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b015cdb0 0x7ff9b00b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:59.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b5e2c640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b00ccb40 0x7ff9b00b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:52:59.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b562b640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b00cea30 0x7ff9b01f2510 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000ebe0 tx=0x7ff9a000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:59.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9a67fc640 1 -- 172.21.15.70:0/1633531441 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0009760 con 0x7ff9b00cea30 2023-10-08T05:52:59.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9a67fc640 1 -- 172.21.15.70:0/1633531441 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a000cc30 con 0x7ff9b00cea30 2023-10-08T05:52:59.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9a67fc640 1 -- 172.21.15.70:0/1633531441 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000eee0 con 0x7ff9b00cea30 2023-10-08T05:52:59.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.420+0000 7ff9b8090640 1 -- 172.21.15.70:0/1633531441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a8009a70 con 0x7ff9b00cea30 2023-10-08T05:52:59.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.421+0000 7ff9b8090640 1 -- 172.21.15.70:0/1633531441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b01f2ce0 con 0x7ff9b00cea30 2023-10-08T05:52:59.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.422+0000 7ff9b8090640 1 -- 172.21.15.70:0/1633531441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b004e160 con 0x7ff9b00cea30 2023-10-08T05:52:59.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.423+0000 7ff9a67fc640 1 -- 172.21.15.70:0/1633531441 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7ff9a0013470 con 0x7ff9b00cea30 2023-10-08T05:52:59.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.424+0000 7ff9a67fc640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff99406af00 0x7ff99406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:52:59.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.424+0000 7ff9a67fc640 1 -- 172.21.15.70:0/1633531441 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5995+0+0 (secure 0 0 0) 0x7ff9a0089a00 con 0x7ff9b00cea30 2023-10-08T05:52:59.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.424+0000 7ff9b5e2c640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff99406af00 0x7ff99406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:52:59.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.425+0000 7ff9b5e2c640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff99406af00 0x7ff99406d3c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000f60 tx=0x7ff9ac009040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:52:59.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.428+0000 7ff9a67fc640 1 -- 172.21.15.70:0/1633531441 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9a00549a0 con 0x7ff9b00cea30 2023-10-08T05:52:59.546 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:52:59.544+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:52:59.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:52:59.636+0000 7ff9b8090640 1 -- 172.21.15.70:0/1633531441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff9b01e84d0 con 0x7ff9b00cea30 2023-10-08T05:53:00.539 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:00.537+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:00.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.708+0000 7ff9a67fc640 1 -- 172.21.15.70:0/1633531441 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff9a0057fc0 con 0x7ff9b00cea30 2023-10-08T05:53:00.710 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T05:53:00.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.711+0000 7ff9b8090640 1 -- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff99406af00 msgr2=0x7ff99406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:00.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.711+0000 7ff9b8090640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff99406af00 0x7ff99406d3c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000f60 tx=0x7ff9ac009040 comp rx=0 tx=0).stop 2023-10-08T05:53:00.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.711+0000 7ff9b8090640 1 -- 172.21.15.70:0/1633531441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b00cea30 msgr2=0x7ff9b01f2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:00.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.711+0000 7ff9b8090640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b00cea30 0x7ff9b01f2510 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000ebe0 tx=0x7ff9a000c520 comp rx=0 tx=0).stop 2023-10-08T05:53:00.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.712+0000 7ff9b8090640 1 -- 172.21.15.70:0/1633531441 shutdown_connections 2023-10-08T05:53:00.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.712+0000 7ff9b8090640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7ff99406af00 0x7ff99406d3c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:00.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.712+0000 7ff9b8090640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9b015cdb0 0x7ff9b00b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:00.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.712+0000 7ff9b8090640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b00cea30 0x7ff9b01f2510 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:00.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.712+0000 7ff9b8090640 1 --2- 172.21.15.70:0/1633531441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9b00ccb40 0x7ff9b00b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:00.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.712+0000 7ff9b8090640 1 -- 172.21.15.70:0/1633531441 >> 172.21.15.70:0/1633531441 conn(0x7ff9b00b0af0 msgr2=0x7ff9b00ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:00.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.712+0000 7ff9b8090640 1 -- 172.21.15.70:0/1633531441 shutdown_connections 2023-10-08T05:53:00.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.712+0000 7ff9b8090640 1 -- 172.21.15.70:0/1633531441 wait complete. 2023-10-08T05:53:00.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T05:53:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.935+0000 7f43da6da640 1 Processor -- start 2023-10-08T05:53:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.936+0000 7f43da6da640 1 -- start start 2023-10-08T05:53:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.937+0000 7f43da6da640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f43d4159ac0 0x7f43d406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.937+0000 7f43da6da640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43d40709e0 0x7f43d40c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.937+0000 7f43da6da640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d406dee0 con 0x7f43d40ad790 2023-10-08T05:53:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.937+0000 7f43da6da640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d415d850 con 0x7f43d4159ac0 2023-10-08T05:53:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.937+0000 7f43da6da640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d415db30 con 0x7f43d40709e0 2023-10-08T05:53:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.938+0000 7f43d3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f43d4159ac0 0x7f43d406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.938+0000 7f43d3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f43d4159ac0 0x7f43d406d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54666/0 (socket says 172.21.15.70:54666) 2023-10-08T05:53:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.938+0000 7f43d3fff640 1 -- 172.21.15.70:0/2696750799 learned_addr learned my addr 172.21.15.70:0/2696750799 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.938+0000 7f43d8c77640 1 --2- 172.21.15.70:0/2696750799 >> v2:172.21.15.181:3300/0 conn(0x7f43d40709e0 0x7f43d40c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.938+0000 7f43d3fff640 1 -- 172.21.15.70:0/2696750799 >> v2:172.21.15.181:3300/0 conn(0x7f43d40709e0 msgr2=0x7f43d40c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.938+0000 7f43d3fff640 1 --2- 172.21.15.70:0/2696750799 >> v2:172.21.15.181:3300/0 conn(0x7f43d40709e0 0x7f43d40c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.938+0000 7f43d3fff640 1 -- 172.21.15.70:0/2696750799 >> v1:172.21.15.181:6789/0 conn(0x7f43d40ad790 legacy=0x7f43d40702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.938+0000 7f43d3fff640 1 -- 172.21.15.70:0/2696750799 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d415de10 con 0x7f43d4159ac0 2023-10-08T05:53:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.939+0000 7f43d3fff640 1 --2- 172.21.15.70:0/2696750799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d4159ac0 0x7f43d406d5e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f43d40caab0 tx=0x7f43c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d665e7369ec925a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.939+0000 7f43d37fe640 1 -- 172.21.15.70:0/2696750799 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4031870 con 0x7f43d4159ac0 2023-10-08T05:53:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.939+0000 7f43d37fe640 1 -- 172.21.15.70:0/2696750799 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c40319d0 con 0x7f43d4159ac0 2023-10-08T05:53:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.940+0000 7f43d37fe640 1 -- 172.21.15.70:0/2696750799 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4031d60 con 0x7f43d4159ac0 2023-10-08T05:53:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.940+0000 7f43da6da640 1 -- 172.21.15.70:0/2696750799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d4159ac0 msgr2=0x7f43d406d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.940+0000 7f43da6da640 1 --2- 172.21.15.70:0/2696750799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d4159ac0 0x7f43d406d5e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f43d40caab0 tx=0x7f43c402b910 comp rx=0 tx=0).stop 2023-10-08T05:53:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.940+0000 7f43da6da640 1 -- 172.21.15.70:0/2696750799 shutdown_connections 2023-10-08T05:53:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.940+0000 7f43da6da640 1 --2- 172.21.15.70:0/2696750799 >> v2:172.21.15.181:3300/0 conn(0x7f43d40709e0 0x7f43d40c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.940+0000 7f43da6da640 1 --2- 172.21.15.70:0/2696750799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d4159ac0 0x7f43d406d5e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.940+0000 7f43da6da640 1 -- 172.21.15.70:0/2696750799 >> 172.21.15.70:0/2696750799 conn(0x7f43d406a380 msgr2=0x7f43d40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.940+0000 7f43da6da640 1 -- 172.21.15.70:0/2696750799 shutdown_connections 2023-10-08T05:53:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.940+0000 7f43da6da640 1 -- 172.21.15.70:0/2696750799 wait complete. 2023-10-08T05:53:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.942+0000 7f43da6da640 1 Processor -- start 2023-10-08T05:53:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.942+0000 7f43da6da640 1 -- start start 2023-10-08T05:53:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.943+0000 7f43da6da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d40709e0 0x7f43d415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.943+0000 7f43da6da640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43d40ad790 0x7f43d4150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.943+0000 7f43da6da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d4159ac0 0x7f43d415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.943+0000 7f43da6da640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d415a560 con 0x7f43d40709e0 2023-10-08T05:53:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.943+0000 7f43da6da640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d415a7e0 con 0x7f43d40ad790 2023-10-08T05:53:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.943+0000 7f43da6da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d415aac0 con 0x7f43d4159ac0 2023-10-08T05:53:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.943+0000 7f43d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d40709e0 0x7f43d415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.944+0000 7f43d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d40709e0 0x7f43d415b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54680/0 (socket says 172.21.15.70:54680) 2023-10-08T05:53:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.944+0000 7f43d3fff640 1 -- 172.21.15.70:0/51600731 learned_addr learned my addr 172.21.15.70:0/51600731 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.944+0000 7f43cbfff640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43d40ad790 0x7f43d4150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.944+0000 7f43d3fff640 1 -- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d4159ac0 msgr2=0x7f43d415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.944+0000 7f43d3fff640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d4159ac0 0x7f43d415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.944+0000 7f43d3fff640 1 -- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43d40ad790 msgr2=0x7f43d4150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.944+0000 7f43d3fff640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43d40ad790 0x7f43d4150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.944+0000 7f43d3fff640 1 -- 172.21.15.70:0/51600731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c40094d0 con 0x7f43d40709e0 2023-10-08T05:53:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.944+0000 7f43d3fff640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d40709e0 0x7f43d415b880 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f43c4000c00 tx=0x7f43c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.945+0000 7f43d17fa640 1 -- 172.21.15.70:0/51600731 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4031840 con 0x7f43d40709e0 2023-10-08T05:53:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.945+0000 7f43d17fa640 1 -- 172.21.15.70:0/51600731 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c40443f0 con 0x7f43d40709e0 2023-10-08T05:53:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.945+0000 7f43da6da640 1 -- 172.21.15.70:0/51600731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d415ada0 con 0x7f43d40709e0 2023-10-08T05:53:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.945+0000 7f43d17fa640 1 -- 172.21.15.70:0/51600731 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c40447a0 con 0x7f43d40709e0 2023-10-08T05:53:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.945+0000 7f43da6da640 1 -- 172.21.15.70:0/51600731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d415b360 con 0x7f43d40709e0 2023-10-08T05:53:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.946+0000 7f43da6da640 1 -- 172.21.15.70:0/51600731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d415bdc0 con 0x7f43d40709e0 2023-10-08T05:53:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.947+0000 7f43d17fa640 1 -- 172.21.15.70:0/51600731 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f43c404d450 con 0x7f43d40709e0 2023-10-08T05:53:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.948+0000 7f43d17fa640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f43b406af00 0x7f43b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.949+0000 7f43d17fa640 1 -- 172.21.15.70:0/51600731 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f43c402d040 con 0x7f43d40709e0 2023-10-08T05:53:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.949+0000 7f43cbfff640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f43b406af00 0x7f43b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:00.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.951+0000 7f43cbfff640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f43b406af00 0x7f43b406d3c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43d41594d0 tx=0x7f43bc009040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:00.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:00.953+0000 7f43d17fa640 1 -- 172.21.15.70:0/51600731 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43c4076e50 con 0x7f43d40709e0 2023-10-08T05:53:01.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.217+0000 7f43da6da640 1 -- 172.21.15.70:0/51600731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f43d4067490 con 0x7f43d40709e0 2023-10-08T05:53:01.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.218+0000 7f43d17fa640 1 -- 172.21.15.70:0/51600731 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f43c40abe60 con 0x7f43d40709e0 2023-10-08T05:53:01.219 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:01.220 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:53:01.220 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T05:53:01.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.220 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T05:53:01.220 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T05:53:01.220 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.220 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:01.220 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:53:01.220 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:53:01.220 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.221 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.221 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T05:53:01.221 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T05:53:01.221 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.221 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:01.221 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:53:01.221 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:53:01.222 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.224 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T05:53:01.225 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.226 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:01.226 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:01.226 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.226 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.226 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T05:53:01.226 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T05:53:01.226 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:01.227 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:01.228 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.228 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.228 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T05:53:01.228 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T05:53:01.228 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.228 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:01.228 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:01.228 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T05:53:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T05:53:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T05:53:01.231 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.232 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:01.232 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:53:01.232 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:53:01.232 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:53:01.232 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.233 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T05:53:01.233 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T05:53:01.233 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.233 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:01.233 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:53:01.233 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:53:01.233 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:53:01.233 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.235 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T05:53:01.236 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T05:53:01.236 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.236 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:53:01.236 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:53:01.236 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.236 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:01.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:01.236 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T05:53:01.236 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T05:53:01.237 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:01.237 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:53:01.237 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:53:01.237 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:01.237 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.237 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:01.237 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T05:53:01.237 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:53:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 -- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f43b406af00 msgr2=0x7f43b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f43b406af00 0x7f43b406d3c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43d41594d0 tx=0x7f43bc009040 comp rx=0 tx=0).stop 2023-10-08T05:53:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 -- 172.21.15.70:0/51600731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d40709e0 msgr2=0x7f43d415b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d40709e0 0x7f43d415b880 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f43c4000c00 tx=0x7f43c40046c0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 -- 172.21.15.70:0/51600731 shutdown_connections 2023-10-08T05:53:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f43b406af00 0x7f43b406d3c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43d4159ac0 0x7f43d415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43d40ad790 0x7f43d4150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 --2- 172.21.15.70:0/51600731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43d40709e0 0x7f43d415b880 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 -- 172.21.15.70:0/51600731 >> 172.21.15.70:0/51600731 conn(0x7f43d406a380 msgr2=0x7f43d40cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:01.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 -- 172.21.15.70:0/51600731 shutdown_connections 2023-10-08T05:53:01.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.221+0000 7f43da6da640 1 -- 172.21.15.70:0/51600731 wait complete. 2023-10-08T05:53:01.240 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.437+0000 7fd03a810640 1 Processor -- start 2023-10-08T05:53:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.437+0000 7fd03a810640 1 -- start start 2023-10-08T05:53:01.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.439+0000 7fd03a810640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd03415b6f0 0x7fd03415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.439+0000 7fd03a810640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03415af50 con 0x7fd0340cc310 2023-10-08T05:53:01.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.439+0000 7fd03a810640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03415c040 con 0x7fd03415ab70 2023-10-08T05:53:01.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.439+0000 7fd03a810640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0340cc900 con 0x7fd03415b6f0 2023-10-08T05:53:01.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.439+0000 7fd02bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd0340cc310 0x7fd0340b13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48194/0 (socket says 172.21.15.70:48194) 2023-10-08T05:53:01.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.439+0000 7fd02bfff640 1 -- 172.21.15.70:0/2238233816 learned_addr learned my addr 172.21.15.70:0/2238233816 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.440+0000 7fd038dad640 1 --2- 172.21.15.70:0/2238233816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd03415b6f0 0x7fd03415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.440+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338063827 0 0) 0x7fd03415af50 con 0x7fd0340cc310 2023-10-08T05:53:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.440+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd020003520 con 0x7fd0340cc310 2023-10-08T05:53:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.440+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1117678110 0 0) 0x7fd020003520 con 0x7fd0340cc310 2023-10-08T05:53:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.440+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd03415b6f0 msgr2=0x7fd03415bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.440+0000 7fd0337fe640 1 --2- 172.21.15.70:0/2238233816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd03415b6f0 0x7fd03415bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.440+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 >> v1:172.21.15.181:6789/0 conn(0x7fd03415ab70 legacy=0x7fd0340aebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.440+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0340ccbe0 con 0x7fd0340cc310 2023-10-08T05:53:01.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.440+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd01c001670 con 0x7fd0340cc310 2023-10-08T05:53:01.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 >> v1:172.21.15.70:6789/0 conn(0x7fd0340cc310 legacy=0x7fd0340b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd0337fe640 1 --2- 172.21.15.70:0/2238233816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0200039a0 0x7fd020003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd0337fe640 1 --2- 172.21.15.70:0/2238233816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0200047c0 0x7fd020004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd0337fe640 1 --2- 172.21.15.70:0/2238233816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd03415b6f0 0x7fd020005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0340cc900 con 0x7fd0200039a0 2023-10-08T05:53:01.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02000bac0 con 0x7fd0200047c0 2023-10-08T05:53:01.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02000bc80 con 0x7fd03415b6f0 2023-10-08T05:53:01.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd033fff640 1 --2- 172.21.15.70:0/2238233816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0200039a0 0x7fd020003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd033fff640 1 -- 172.21.15.70:0/2238233816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd03415b6f0 msgr2=0x7fd020005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:01.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd033fff640 1 --2- 172.21.15.70:0/2238233816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd03415b6f0 0x7fd020005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd033fff640 1 -- 172.21.15.70:0/2238233816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0200047c0 msgr2=0x7fd020004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:01.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd033fff640 1 --2- 172.21.15.70:0/2238233816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0200047c0 0x7fd020004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.441+0000 7fd033fff640 1 -- 172.21.15.70:0/2238233816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd020003520 con 0x7fd0200039a0 2023-10-08T05:53:01.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.442+0000 7fd033fff640 1 --2- 172.21.15.70:0/2238233816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0200039a0 0x7fd020003dc0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd024007fd0 tx=0x7fd0200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e9c959e39fa548 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:01.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.443+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0240052d0 con 0x7fd0200039a0 2023-10-08T05:53:01.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.443+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02402e050 con 0x7fd0200039a0 2023-10-08T05:53:01.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.443+0000 7fd0337fe640 1 -- 172.21.15.70:0/2238233816 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024003640 con 0x7fd0200039a0 2023-10-08T05:53:01.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.443+0000 7fd03a810640 1 -- 172.21.15.70:0/2238233816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0200039a0 msgr2=0x7fd020003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.443+0000 7fd03a810640 1 --2- 172.21.15.70:0/2238233816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0200039a0 0x7fd020003dc0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fd024007fd0 tx=0x7fd0200057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.443+0000 7fd038dad640 1 -- 172.21.15.70:0/2238233816 reap_dead start 2023-10-08T05:53:01.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.443+0000 7fd03a810640 1 -- 172.21.15.70:0/2238233816 shutdown_connections 2023-10-08T05:53:01.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.443+0000 7fd03a810640 1 -- 172.21.15.70:0/2238233816 >> 172.21.15.70:0/2238233816 conn(0x7fd03406d450 msgr2=0x7fd03406d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:01.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.443+0000 7fd03a810640 1 -- 172.21.15.70:0/2238233816 shutdown_connections 2023-10-08T05:53:01.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.443+0000 7fd03a810640 1 -- 172.21.15.70:0/2238233816 wait complete. 2023-10-08T05:53:01.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.445+0000 7fd03a810640 1 Processor -- start 2023-10-08T05:53:01.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.445+0000 7fd03a810640 1 -- start start 2023-10-08T05:53:01.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.446+0000 7fd03a810640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0200039a0 0x7fd0340c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.446+0000 7fd03a810640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd034156ac0 0x7fd034156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.446+0000 7fd03a810640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0340c4c40 0x7fd0340c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.446+0000 7fd03a810640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0340c41a0 con 0x7fd034156ac0 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.446+0000 7fd03a810640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0340c3520 con 0x7fd0340c4c40 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.446+0000 7fd03a810640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0340c3800 con 0x7fd0200039a0 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.446+0000 7fd02bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd034156ac0 0x7fd034156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.446+0000 7fd02bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd034156ac0 0x7fd034156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54690/0 (socket says 172.21.15.70:54690) 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.446+0000 7fd02bfff640 1 -- 172.21.15.70:0/804963151 learned_addr learned my addr 172.21.15.70:0/804963151 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd033fff640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0200039a0 0x7fd0340c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd02bfff640 1 -- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0200039a0 msgr2=0x7fd0340c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd02bfff640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0200039a0 0x7fd0340c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd038dad640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0340c4c40 0x7fd0340c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd02bfff640 1 -- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0340c4c40 msgr2=0x7fd0340c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd02bfff640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0340c4c40 0x7fd0340c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd02bfff640 1 -- 172.21.15.70:0/804963151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0340c3ae0 con 0x7fd034156ac0 2023-10-08T05:53:01.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd02bfff640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd034156ac0 0x7fd034156f40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c730 tx=0x7fd01c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:01.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd0317fa640 1 -- 172.21.15.70:0/804963151 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0072d0 con 0x7fd034156ac0 2023-10-08T05:53:01.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd0317fa640 1 -- 172.21.15.70:0/804963151 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c007450 con 0x7fd034156ac0 2023-10-08T05:53:01.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd03a810640 1 -- 172.21.15.70:0/804963151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0340c3dc0 con 0x7fd034156ac0 2023-10-08T05:53:01.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.447+0000 7fd0317fa640 1 -- 172.21.15.70:0/804963151 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0047e0 con 0x7fd034156ac0 2023-10-08T05:53:01.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.448+0000 7fd03a810640 1 -- 172.21.15.70:0/804963151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0340c45c0 con 0x7fd034156ac0 2023-10-08T05:53:01.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.449+0000 7fd03a810640 1 -- 172.21.15.70:0/804963151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0340c42e0 con 0x7fd034156ac0 2023-10-08T05:53:01.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.449+0000 7fd0317fa640 1 -- 172.21.15.70:0/804963151 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fd01c004960 con 0x7fd034156ac0 2023-10-08T05:53:01.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.451+0000 7fd0317fa640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd01406af00 0x7fd01406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.451+0000 7fd0317fa640 1 -- 172.21.15.70:0/804963151 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fd01c088c10 con 0x7fd034156ac0 2023-10-08T05:53:01.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.451+0000 7fd033fff640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd01406af00 0x7fd01406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.452+0000 7fd033fff640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd01406af00 0x7fd01406d3c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd024033000 tx=0x7fd024009030 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:01.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.455+0000 7fd0317fa640 1 -- 172.21.15.70:0/804963151 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0340c42e0 con 0x7fd034156ac0 2023-10-08T05:53:01.500 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:01.498+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:01.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.651+0000 7fd03a810640 1 -- 172.21.15.70:0/804963151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd034066d80 con 0x7fd034156ac0 2023-10-08T05:53:01.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.652+0000 7fd0317fa640 1 -- 172.21.15.70:0/804963151 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+13122 (secure 0 0 0) 0x7fd01c018020 con 0x7fd034156ac0 2023-10-08T05:53:01.653 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:01.653 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":100,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:00.706275+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:01.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 -- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd01406af00 msgr2=0x7fd01406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd01406af00 0x7fd01406d3c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd024033000 tx=0x7fd024009030 comp rx=0 tx=0).stop 2023-10-08T05:53:01.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 -- 172.21.15.70:0/804963151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd034156ac0 msgr2=0x7fd034156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd034156ac0 0x7fd034156f40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c730 tx=0x7fd01c00cbf0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 -- 172.21.15.70:0/804963151 shutdown_connections 2023-10-08T05:53:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd01406af00 0x7fd01406d3c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0340c4c40 0x7fd0340c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd034156ac0 0x7fd034156f40 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 --2- 172.21.15.70:0/804963151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0200039a0 0x7fd0340c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 -- 172.21.15.70:0/804963151 >> 172.21.15.70:0/804963151 conn(0x7fd03406d450 msgr2=0x7fd03406d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 -- 172.21.15.70:0/804963151 shutdown_connections 2023-10-08T05:53:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.655+0000 7fd03a810640 1 -- 172.21.15.70:0/804963151 wait complete. 2023-10-08T05:53:01.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:53:01.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:53:01.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.888+0000 7fab73ef2640 1 Processor -- start 2023-10-08T05:53:01.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.888+0000 7fab73ef2640 1 -- start start 2023-10-08T05:53:01.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.890+0000 7fab73ef2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fab6c0b1750 0x7fab6c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.890+0000 7fab73ef2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab6c06d240 0x7fab6c070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.890+0000 7fab73ef2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c06d670 con 0x7fab6c15aad0 2023-10-08T05:53:01.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.890+0000 7fab73ef2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c15bf30 con 0x7fab6c06d240 2023-10-08T05:53:01.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.890+0000 7fab73ef2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c0b1bb0 con 0x7fab6c0b1750 2023-10-08T05:53:01.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.890+0000 7fab71c8e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fab6c15aad0 0x7fab6c0c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48202/0 (socket says 172.21.15.70:48202) 2023-10-08T05:53:01.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.890+0000 7fab71c8e640 1 -- 172.21.15.70:0/2844343325 learned_addr learned my addr 172.21.15.70:0/2844343325 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:01.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.890+0000 7fab7248f640 1 --2- 172.21.15.70:0/2844343325 >> v2:172.21.15.70:3300/0 conn(0x7fab6c06d240 0x7fab6c070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.891+0000 7fab7148d640 1 --2- 172.21.15.70:0/2844343325 >> v2:172.21.15.181:3300/0 conn(0x7fab6c0b1750 0x7fab6c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.891+0000 7fab7248f640 1 -- 172.21.15.70:0/2844343325 >> v2:172.21.15.181:3300/0 conn(0x7fab6c0b1750 msgr2=0x7fab6c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:01.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.891+0000 7fab7248f640 1 --2- 172.21.15.70:0/2844343325 >> v2:172.21.15.181:3300/0 conn(0x7fab6c0b1750 0x7fab6c06cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.891+0000 7fab7248f640 1 -- 172.21.15.70:0/2844343325 >> v1:172.21.15.70:6789/0 conn(0x7fab6c15aad0 legacy=0x7fab6c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.891+0000 7fab7248f640 1 -- 172.21.15.70:0/2844343325 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c15b320 con 0x7fab6c06d240 2023-10-08T05:53:01.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.891+0000 7fab70c8c640 1 -- 172.21.15.70:0/2844343325 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087451838 0 0) 0x7fab6c06d670 con 0x7fab6c15aad0 2023-10-08T05:53:01.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.891+0000 7fab7148d640 1 --2- 172.21.15.70:0/2844343325 >> v2:172.21.15.181:3300/0 conn(0x7fab6c0b1750 0x7fab6c06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:01.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.892+0000 7fab7248f640 1 --2- 172.21.15.70:0/2844343325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 0x7fab6c070570 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fab6c06e330 tx=0x7fab6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912f290d19a5e539 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:01.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.892+0000 7fab70c8c640 1 -- 172.21.15.70:0/2844343325 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64031830 con 0x7fab6c06d240 2023-10-08T05:53:01.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.892+0000 7fab70c8c640 1 -- 172.21.15.70:0/2844343325 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab64031990 con 0x7fab6c06d240 2023-10-08T05:53:01.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.892+0000 7fab70c8c640 1 -- 172.21.15.70:0/2844343325 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab64031d20 con 0x7fab6c06d240 2023-10-08T05:53:01.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.893+0000 7fab73ef2640 1 -- 172.21.15.70:0/2844343325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 msgr2=0x7fab6c070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.893+0000 7fab73ef2640 1 --2- 172.21.15.70:0/2844343325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 0x7fab6c070570 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fab6c06e330 tx=0x7fab6402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.893+0000 7fab73ef2640 1 -- 172.21.15.70:0/2844343325 shutdown_connections 2023-10-08T05:53:01.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.893+0000 7fab73ef2640 1 --2- 172.21.15.70:0/2844343325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 0x7fab6c070570 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.893+0000 7fab73ef2640 1 --2- 172.21.15.70:0/2844343325 >> v2:172.21.15.181:3300/0 conn(0x7fab6c0b1750 0x7fab6c06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.893+0000 7fab73ef2640 1 -- 172.21.15.70:0/2844343325 >> 172.21.15.70:0/2844343325 conn(0x7fab6c069d50 msgr2=0x7fab6c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:01.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.893+0000 7fab73ef2640 1 -- 172.21.15.70:0/2844343325 shutdown_connections 2023-10-08T05:53:01.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.893+0000 7fab73ef2640 1 -- 172.21.15.70:0/2844343325 wait complete. 2023-10-08T05:53:01.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.895+0000 7fab73ef2640 1 Processor -- start 2023-10-08T05:53:01.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.895+0000 7fab73ef2640 1 -- start start 2023-10-08T05:53:01.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.896+0000 7fab73ef2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 0x7fab6c15c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.896+0000 7fab73ef2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab6c0b1750 0x7fab6c144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.896+0000 7fab73ef2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab6c15aad0 0x7fab6c145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.896+0000 7fab73ef2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c1e4430 con 0x7fab6c06d240 2023-10-08T05:53:01.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.896+0000 7fab73ef2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c1e46d0 con 0x7fab6c0b1750 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.896+0000 7fab73ef2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c1e49b0 con 0x7fab6c15aad0 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.896+0000 7fab71c8e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 0x7fab6c15c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.896+0000 7fab71c8e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 0x7fab6c15c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54696/0 (socket says 172.21.15.70:54696) 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.896+0000 7fab71c8e640 1 -- 172.21.15.70:0/255093033 learned_addr learned my addr 172.21.15.70:0/255093033 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab7148d640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab6c0b1750 0x7fab6c144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab7248f640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab6c15aad0 0x7fab6c145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab71c8e640 1 -- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab6c15aad0 msgr2=0x7fab6c145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab71c8e640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab6c15aad0 0x7fab6c145320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab71c8e640 1 -- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab6c0b1750 msgr2=0x7fab6c144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab71c8e640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab6c0b1750 0x7fab6c144de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:01.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab71c8e640 1 -- 172.21.15.70:0/255093033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab64009400 con 0x7fab6c06d240 2023-10-08T05:53:01.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab7148d640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab6c0b1750 0x7fab6c144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:01.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab71c8e640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 0x7fab6c15c370 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00cb10 tx=0x7fab5c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:01.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab6a7fc640 1 -- 172.21.15.70:0/255093033 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00ac70 con 0x7fab6c06d240 2023-10-08T05:53:01.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab6a7fc640 1 -- 172.21.15.70:0/255093033 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c00add0 con 0x7fab6c06d240 2023-10-08T05:53:01.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab73ef2640 1 -- 172.21.15.70:0/255093033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c1e4c90 con 0x7fab6c06d240 2023-10-08T05:53:01.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.897+0000 7fab6a7fc640 1 -- 172.21.15.70:0/255093033 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c012620 con 0x7fab6c06d240 2023-10-08T05:53:01.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.898+0000 7fab73ef2640 1 -- 172.21.15.70:0/255093033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c145860 con 0x7fab6c06d240 2023-10-08T05:53:01.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.899+0000 7fab73ef2640 1 -- 172.21.15.70:0/255093033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c1e51b0 con 0x7fab6c06d240 2023-10-08T05:53:01.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.900+0000 7fab6a7fc640 1 -- 172.21.15.70:0/255093033 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fab5c007070 con 0x7fab6c06d240 2023-10-08T05:53:01.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.900+0000 7fab6a7fc640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fab5806af00 0x7fab5806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:01.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.901+0000 7fab6a7fc640 1 -- 172.21.15.70:0/255093033 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fab5c089590 con 0x7fab6c06d240 2023-10-08T05:53:01.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.901+0000 7fab7148d640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fab5806af00 0x7fab5806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:01.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.903+0000 7fab7148d640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fab5806af00 0x7fab5806d3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fab60006fd0 tx=0x7fab60008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:01.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:01.905+0000 7fab6a7fc640 1 -- 172.21.15.70:0/255093033 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab6c1e51b0 con 0x7fab6c06d240 2023-10-08T05:53:02.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.100+0000 7fab73ef2640 1 -- 172.21.15.70:0/255093033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab6c1e51b0 con 0x7fab6c06d240 2023-10-08T05:53:02.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.101+0000 7fab6a7fc640 1 -- 172.21.15.70:0/255093033 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab5c017020 con 0x7fab6c06d240 2023-10-08T05:53:02.102 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:53:02.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 -- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fab5806af00 msgr2=0x7fab5806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fab5806af00 0x7fab5806d3c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fab60006fd0 tx=0x7fab60008040 comp rx=0 tx=0).stop 2023-10-08T05:53:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 -- 172.21.15.70:0/255093033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 msgr2=0x7fab6c15c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 0x7fab6c15c370 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00cb10 tx=0x7fab5c00a060 comp rx=0 tx=0).stop 2023-10-08T05:53:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 -- 172.21.15.70:0/255093033 shutdown_connections 2023-10-08T05:53:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fab5806af00 0x7fab5806d3c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab6c15aad0 0x7fab6c145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab6c0b1750 0x7fab6c144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 --2- 172.21.15.70:0/255093033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab6c06d240 0x7fab6c15c370 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 -- 172.21.15.70:0/255093033 >> 172.21.15.70:0/255093033 conn(0x7fab6c069d50 msgr2=0x7fab6c156440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 -- 172.21.15.70:0/255093033 shutdown_connections 2023-10-08T05:53:02.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.104+0000 7fab73ef2640 1 -- 172.21.15.70:0/255093033 wait complete. 2023-10-08T05:53:02.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T05:53:02.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.333+0000 7f29bbfff640 1 Processor -- start 2023-10-08T05:53:02.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.333+0000 7f29bbfff640 1 -- start start 2023-10-08T05:53:02.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29bbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29bc0cab40 0x7f29bc1614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29bbfff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29bc0c9020 con 0x7f29bc15cca0 2023-10-08T05:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29bbfff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29bc0ccc20 con 0x7f29bc0b07d0 2023-10-08T05:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29bbfff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc0c92e0 con 0x7f29bc0cab40 2023-10-08T05:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29bb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29bc0cab40 0x7f29bc1614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29bb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29bc0cab40 0x7f29bc1614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54706/0 (socket says 172.21.15.70:54706) 2023-10-08T05:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29bb7fe640 1 -- 172.21.15.70:0/1036701609 learned_addr learned my addr 172.21.15.70:0/1036701609 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29baffd640 1 --1- 172.21.15.70:0/1036701609 >> v1:172.21.15.70:6789/0 conn(0x7f29bc15cca0 0x7f29bc0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48214/0 (socket says 172.21.15.70:48214) 2023-10-08T05:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29bb7fe640 1 -- 172.21.15.70:0/1036701609 >> v1:172.21.15.181:6789/0 conn(0x7f29bc0b07d0 legacy=0x7f29bc0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29bb7fe640 1 -- 172.21.15.70:0/1036701609 >> v1:172.21.15.70:6789/0 conn(0x7f29bc15cca0 legacy=0x7f29bc0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.336+0000 7f29bb7fe640 1 -- 172.21.15.70:0/1036701609 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0cc7b0 con 0x7f29bc0cab40 2023-10-08T05:53:02.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.335+0000 7f29ba7fc640 1 -- 172.21.15.70:0/1036701609 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988778440 0 0) 0x7f29bc0c9020 con 0x7f29bc15cca0 2023-10-08T05:53:02.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.337+0000 7f29bb7fe640 1 --2- 172.21.15.70:0/1036701609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0cab40 0x7f29bc1614b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009e50 tx=0x7f29b402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128e50a130660639 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:02.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.337+0000 7f29ba7fc640 1 -- 172.21.15.70:0/1036701609 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b402c9f0 con 0x7f29bc0cab40 2023-10-08T05:53:02.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.337+0000 7f29ba7fc640 1 -- 172.21.15.70:0/1036701609 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b402cb50 con 0x7f29bc0cab40 2023-10-08T05:53:02.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.337+0000 7f29ba7fc640 1 -- 172.21.15.70:0/1036701609 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40343f0 con 0x7f29bc0cab40 2023-10-08T05:53:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.338+0000 7f29bbfff640 1 -- 172.21.15.70:0/1036701609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0cab40 msgr2=0x7f29bc1614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.338+0000 7f29bbfff640 1 --2- 172.21.15.70:0/1036701609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0cab40 0x7f29bc1614b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009e50 tx=0x7f29b402be50 comp rx=0 tx=0).stop 2023-10-08T05:53:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.338+0000 7f29bbfff640 1 -- 172.21.15.70:0/1036701609 shutdown_connections 2023-10-08T05:53:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.338+0000 7f29bbfff640 1 --2- 172.21.15.70:0/1036701609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0cab40 0x7f29bc1614b0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.338+0000 7f29bbfff640 1 -- 172.21.15.70:0/1036701609 >> 172.21.15.70:0/1036701609 conn(0x7f29bc0cb0c0 msgr2=0x7f29bc0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.338+0000 7f29bbfff640 1 -- 172.21.15.70:0/1036701609 shutdown_connections 2023-10-08T05:53:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.338+0000 7f29bbfff640 1 -- 172.21.15.70:0/1036701609 wait complete. 2023-10-08T05:53:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.340+0000 7f29bbfff640 1 Processor -- start 2023-10-08T05:53:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.340+0000 7f29bbfff640 1 -- start start 2023-10-08T05:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.341+0000 7f29bbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0b07d0 0x7f29bc1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.341+0000 7f29bbfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29bc0cab40 0x7f29bc0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.341+0000 7f29bbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29bc15cca0 0x7f29bc0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.341+0000 7f29bbfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0b1390 con 0x7f29bc0b07d0 2023-10-08T05:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.341+0000 7f29bbfff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc15f870 con 0x7f29bc15cca0 2023-10-08T05:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.341+0000 7f29bbfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc159d40 con 0x7f29bc0cab40 2023-10-08T05:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.341+0000 7f29baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0b07d0 0x7f29bc1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.341+0000 7f29baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0b07d0 0x7f29bc1593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54714/0 (socket says 172.21.15.70:54714) 2023-10-08T05:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.341+0000 7f29baffd640 1 -- 172.21.15.70:0/3195684288 learned_addr learned my addr 172.21.15.70:0/3195684288 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29baffd640 1 -- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29bc0cab40 msgr2=0x7f29bc0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29baffd640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29bc0cab40 0x7f29bc0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29baffd640 1 -- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29bc15cca0 msgr2=0x7f29bc0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29bb7fe640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29bc15cca0 0x7f29bc0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29baffd640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29bc15cca0 0x7f29bc0b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29baffd640 1 -- 172.21.15.70:0/3195684288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b4009b30 con 0x7f29bc0b07d0 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29baffd640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0b07d0 0x7f29bc1593f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c500 tx=0x7f29ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29b37fe640 1 -- 172.21.15.70:0/3195684288 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac007560 con 0x7f29bc0b07d0 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29b37fe640 1 -- 172.21.15.70:0/3195684288 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac0047e0 con 0x7f29bc0b07d0 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29b37fe640 1 -- 172.21.15.70:0/3195684288 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac004b90 con 0x7f29bc0b07d0 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29bbfff640 1 -- 172.21.15.70:0/3195684288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc0c7a60 con 0x7f29bc0b07d0 2023-10-08T05:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.342+0000 7f29bbfff640 1 -- 172.21.15.70:0/3195684288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc0c8020 con 0x7f29bc0b07d0 2023-10-08T05:53:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.344+0000 7f29bbfff640 1 -- 172.21.15.70:0/3195684288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc04e160 con 0x7f29bc0b07d0 2023-10-08T05:53:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.344+0000 7f29b37fe640 1 -- 172.21.15.70:0/3195684288 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f29ac004cf0 con 0x7f29bc0b07d0 2023-10-08T05:53:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.345+0000 7f29b37fe640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f299c06af00 0x7f299c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.346+0000 7f29b37fe640 1 -- 172.21.15.70:0/3195684288 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f29ac0885f0 con 0x7f29bc0b07d0 2023-10-08T05:53:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.346+0000 7f29b3fff640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f299c06af00 0x7f299c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.347+0000 7f29b3fff640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f299c06af00 0x7f299c06d3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f29a4006fd0 tx=0x7f29a4006bc0 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.350+0000 7f29b37fe640 1 -- 172.21.15.70:0/3195684288 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29ac053820 con 0x7f29bc0b07d0 2023-10-08T05:53:02.471 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:02.469+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:02.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.548+0000 7f29bbfff640 1 -- 172.21.15.70:0/3195684288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f29bc0b45c0 con 0x7f29bc0b07d0 2023-10-08T05:53:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.549+0000 7f29b37fe640 1 -- 172.21.15.70:0/3195684288 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) v1 ==== 84+0+65 (secure 0 0 0) 0x7f29ac018060 con 0x7f29bc0b07d0 2023-10-08T05:53:02.550 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:02.550 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:53:02.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:02.551 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T05:53:02.551 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T05:53:02.551 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:02.551 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:53:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.551+0000 7f29bbfff640 1 -- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f299c06af00 msgr2=0x7f299c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.551+0000 7f29bbfff640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f299c06af00 0x7f299c06d3c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f29a4006fd0 tx=0x7f29a4006bc0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.551+0000 7f29bbfff640 1 -- 172.21.15.70:0/3195684288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0b07d0 msgr2=0x7f29bc1593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.551+0000 7f29bbfff640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0b07d0 0x7f29bc1593f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c500 tx=0x7f29ac00c9c0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.552+0000 7f29bbfff640 1 -- 172.21.15.70:0/3195684288 shutdown_connections 2023-10-08T05:53:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.552+0000 7f29bbfff640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f299c06af00 0x7f299c06d3c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.552+0000 7f29bbfff640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f29bc15cca0 0x7f29bc0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.552+0000 7f29bbfff640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f29bc0cab40 0x7f29bc0afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.552+0000 7f29bbfff640 1 --2- 172.21.15.70:0/3195684288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc0b07d0 0x7f29bc1593f0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.552+0000 7f29bbfff640 1 -- 172.21.15.70:0/3195684288 >> 172.21.15.70:0/3195684288 conn(0x7f29bc0cb0c0 msgr2=0x7f29bc0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.552+0000 7f29bbfff640 1 -- 172.21.15.70:0/3195684288 shutdown_connections 2023-10-08T05:53:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.552+0000 7f29bbfff640 1 -- 172.21.15.70:0/3195684288 wait complete. 2023-10-08T05:53:02.572 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T05:53:02.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T05:53:02.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.788+0000 7fd5527d7640 1 Processor -- start 2023-10-08T05:53:02.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.789+0000 7fd5527d7640 1 -- start start 2023-10-08T05:53:02.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.790+0000 7fd5527d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd54c0ccea0 0x7fd54c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.790+0000 7fd5527d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54c072450 con 0x7fd54c0b1510 2023-10-08T05:53:02.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.790+0000 7fd5527d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54c0713f0 con 0x7fd54c15cba0 2023-10-08T05:53:02.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.790+0000 7fd5527d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c071690 con 0x7fd54c0ccea0 2023-10-08T05:53:02.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.790+0000 7fd550d74640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd54c0b1510 0x7fd54c0ca010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48220/0 (socket says 172.21.15.70:48220) 2023-10-08T05:53:02.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.790+0000 7fd550d74640 1 -- 172.21.15.70:0/1845884752 learned_addr learned my addr 172.21.15.70:0/1845884752 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:02.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54b7fe640 1 --2- 172.21.15.70:0/1845884752 >> v2:172.21.15.181:3300/0 conn(0x7fd54c0ccea0 0x7fd54c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:02.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403696604 0 0) 0x7fd54c072450 con 0x7fd54c0b1510 2023-10-08T05:53:02.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd538003520 con 0x7fd54c0b1510 2023-10-08T05:53:02.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1077963899 0 0) 0x7fd538003520 con 0x7fd54c0b1510 2023-10-08T05:53:02.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 >> v2:172.21.15.181:3300/0 conn(0x7fd54c0ccea0 msgr2=0x7fd54c0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:02.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 --2- 172.21.15.70:0/1845884752 >> v2:172.21.15.181:3300/0 conn(0x7fd54c0ccea0 0x7fd54c0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 >> v1:172.21.15.181:6789/0 conn(0x7fd54c15cba0 legacy=0x7fd54c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:02.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c071970 con 0x7fd54c0b1510 2023-10-08T05:53:02.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54b7fe640 1 --2- 172.21.15.70:0/1845884752 >> v2:172.21.15.181:3300/0 conn(0x7fd54c0ccea0 0x7fd54c0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:53:02.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd544001670 con 0x7fd54c0b1510 2023-10-08T05:53:02.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 >> v1:172.21.15.70:6789/0 conn(0x7fd54c0b1510 legacy=0x7fd54c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 --2- 172.21.15.70:0/1845884752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd538003970 0x7fd538003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 --2- 172.21.15.70:0/1845884752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5380047c0 0x7fd538004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 --2- 172.21.15.70:0/1845884752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd53800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c071690 con 0x7fd538005180 2023-10-08T05:53:02.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53800c0b0 con 0x7fd5380047c0 2023-10-08T05:53:02.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.791+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53800c270 con 0x7fd538003970 2023-10-08T05:53:02.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.792+0000 7fd550d74640 1 --2- 172.21.15.70:0/1845884752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd53800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:02.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.792+0000 7fd550d74640 1 -- 172.21.15.70:0/1845884752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd538003970 msgr2=0x7fd538003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:02.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.792+0000 7fd550d74640 1 --2- 172.21.15.70:0/1845884752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd538003970 0x7fd538003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.792+0000 7fd550d74640 1 -- 172.21.15.70:0/1845884752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5380047c0 msgr2=0x7fd538004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:02.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.792+0000 7fd550d74640 1 --2- 172.21.15.70:0/1845884752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5380047c0 0x7fd538004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.792+0000 7fd550d74640 1 -- 172.21.15.70:0/1845884752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd538003520 con 0x7fd538005180 2023-10-08T05:53:02.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.793+0000 7fd550d74640 1 -- 172.21.15.70:0/1845884752 reap_dead start 2023-10-08T05:53:02.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.793+0000 7fd550d74640 1 --2- 172.21.15.70:0/1845884752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd53800b980 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd54400c210 tx=0x7fd5380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef64f2efdba5f2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.793+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544005ad0 con 0x7fd538005180 2023-10-08T05:53:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.793+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd544005c30 con 0x7fd538005180 2023-10-08T05:53:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.793+0000 7fd54affd640 1 -- 172.21.15.70:0/1845884752 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54402c890 con 0x7fd538005180 2023-10-08T05:53:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.794+0000 7fd5527d7640 1 -- 172.21.15.70:0/1845884752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 msgr2=0x7fd53800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.794+0000 7fd5527d7640 1 --2- 172.21.15.70:0/1845884752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd53800b980 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd54400c210 tx=0x7fd5380057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.794+0000 7fd5527d7640 1 -- 172.21.15.70:0/1845884752 shutdown_connections 2023-10-08T05:53:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.794+0000 7fd5527d7640 1 --2- 172.21.15.70:0/1845884752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd53800b980 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.794+0000 7fd5527d7640 1 -- 172.21.15.70:0/1845884752 >> 172.21.15.70:0/1845884752 conn(0x7fd54c06b250 msgr2=0x7fd54c06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.794+0000 7fd5527d7640 1 -- 172.21.15.70:0/1845884752 shutdown_connections 2023-10-08T05:53:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.794+0000 7fd5527d7640 1 -- 172.21.15.70:0/1845884752 wait complete. 2023-10-08T05:53:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.796+0000 7fd5527d7640 1 Processor -- start 2023-10-08T05:53:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.796+0000 7fd5527d7640 1 -- start start 2023-10-08T05:53:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.797+0000 7fd5527d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd54c0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.797+0000 7fd5527d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd54c0ad1c0 0x7fd54c14cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.797+0000 7fd5527d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd54c15fd20 0x7fd54c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.797+0000 7fd5527d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0acda0 con 0x7fd538005180 2023-10-08T05:53:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.797+0000 7fd5527d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0ac120 con 0x7fd54c15fd20 2023-10-08T05:53:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.797+0000 7fd5527d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0ac400 con 0x7fd54c0ad1c0 2023-10-08T05:53:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.797+0000 7fd54bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd54c0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.797+0000 7fd54bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd54c0ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54722/0 (socket says 172.21.15.70:54722) 2023-10-08T05:53:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.797+0000 7fd54bfff640 1 -- 172.21.15.70:0/2654777423 learned_addr learned my addr 172.21.15.70:0/2654777423 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd54b7fe640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd54c0ad1c0 0x7fd54c14cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd54bfff640 1 -- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd54c0ad1c0 msgr2=0x7fd54c14cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd54bfff640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd54c0ad1c0 0x7fd54c14cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd54bfff640 1 -- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd54c15fd20 msgr2=0x7fd54c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:02.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd550d74640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd54c15fd20 0x7fd54c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:02.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd54bfff640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd54c15fd20 0x7fd54c069d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:02.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd54bfff640 1 -- 172.21.15.70:0/2654777423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c0ac6e0 con 0x7fd538005180 2023-10-08T05:53:02.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd54bfff640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd54c0ab8d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00c7f0 tx=0x7fd53c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:02.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd548ff9640 1 -- 172.21.15.70:0/2654777423 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c004b60 con 0x7fd538005180 2023-10-08T05:53:02.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd548ff9640 1 -- 172.21.15.70:0/2654777423 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c004cc0 con 0x7fd538005180 2023-10-08T05:53:02.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd548ff9640 1 -- 172.21.15.70:0/2654777423 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00aa90 con 0x7fd538005180 2023-10-08T05:53:02.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.798+0000 7fd5527d7640 1 -- 172.21.15.70:0/2654777423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54c0ac9c0 con 0x7fd538005180 2023-10-08T05:53:02.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.799+0000 7fd5527d7640 1 -- 172.21.15.70:0/2654777423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c14d130 con 0x7fd538005180 2023-10-08T05:53:02.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.799+0000 7fd5527d7640 1 -- 172.21.15.70:0/2654777423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c0ad010 con 0x7fd538005180 2023-10-08T05:53:02.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.801+0000 7fd548ff9640 1 -- 172.21.15.70:0/2654777423 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fd53c00abf0 con 0x7fd538005180 2023-10-08T05:53:02.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.801+0000 7fd548ff9640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd52c06af00 0x7fd52c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:02.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.801+0000 7fd548ff9640 1 -- 172.21.15.70:0/2654777423 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fd53c089760 con 0x7fd538005180 2023-10-08T05:53:02.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.802+0000 7fd54b7fe640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd52c06af00 0x7fd52c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:02.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.803+0000 7fd54b7fe640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd52c06af00 0x7fd52c06d3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd534006fd0 tx=0x7fd534008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:02.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.806+0000 7fd548ff9640 1 -- 172.21.15.70:0/2654777423 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd53c054900 con 0x7fd538005180 2023-10-08T05:53:03.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:02.999+0000 7fd5527d7640 1 -- 172.21.15.70:0/2654777423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd54c04e160 con 0x7fd538005180 2023-10-08T05:53:03.494 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:03.492+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.726+0000 7fd548ff9640 1 -- 172.21.15.70:0/2654777423 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd53c057f20 con 0x7fd538005180 2023-10-08T05:53:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T05:53:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 -- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd52c06af00 msgr2=0x7fd52c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd52c06af00 0x7fd52c06d3c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd534006fd0 tx=0x7fd534008040 comp rx=0 tx=0).stop 2023-10-08T05:53:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 -- 172.21.15.70:0/2654777423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 msgr2=0x7fd54c0ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd54c0ab8d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00c7f0 tx=0x7fd53c00ccb0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 -- 172.21.15.70:0/2654777423 shutdown_connections 2023-10-08T05:53:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fd52c06af00 0x7fd52c06d3c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd54c15fd20 0x7fd54c069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd54c0ad1c0 0x7fd54c14cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 --2- 172.21.15.70:0/2654777423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd538005180 0x7fd54c0ab8d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 -- 172.21.15.70:0/2654777423 >> 172.21.15.70:0/2654777423 conn(0x7fd54c06b250 msgr2=0x7fd54c0b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.730+0000 7fd5527d7640 1 -- 172.21.15.70:0/2654777423 shutdown_connections 2023-10-08T05:53:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.731+0000 7fd5527d7640 1 -- 172.21.15.70:0/2654777423 wait complete. 2023-10-08T05:53:03.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T05:53:03.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.965+0000 7fc08b59e640 1 Processor -- start 2023-10-08T05:53:03.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.965+0000 7fc08b59e640 1 -- start start 2023-10-08T05:53:03.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.967+0000 7fc08b59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc08c0b3150 0x7fc08c0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:03.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.967+0000 7fc08b59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c06cab0 con 0x7fc08c0cbac0 2023-10-08T05:53:03.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.967+0000 7fc08b59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c0add00 con 0x7fc08c0ca740 2023-10-08T05:53:03.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.967+0000 7fc08b59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08c06c180 con 0x7fc08c0b3150 2023-10-08T05:53:03.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.967+0000 7fc08ad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc08c0cbac0 0x7fc08c0c8c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37034/0 (socket says 172.21.15.70:37034) 2023-10-08T05:53:03.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.967+0000 7fc08ad9d640 1 -- 172.21.15.70:0/2859803016 learned_addr learned my addr 172.21.15.70:0/2859803016 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:03.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.967+0000 7fc08a59c640 1 --2- 172.21.15.70:0/2859803016 >> v2:172.21.15.181:3300/0 conn(0x7fc08c0b3150 0x7fc08c0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:03.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271647565 0 0) 0x7fc08c06cab0 con 0x7fc08c0cbac0 2023-10-08T05:53:03.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc078003520 con 0x7fc08c0cbac0 2023-10-08T05:53:03.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1368749356 0 0) 0x7fc078003520 con 0x7fc08c0cbac0 2023-10-08T05:53:03.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 >> v2:172.21.15.181:3300/0 conn(0x7fc08c0b3150 msgr2=0x7fc08c0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:03.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 --2- 172.21.15.70:0/2859803016 >> v2:172.21.15.181:3300/0 conn(0x7fc08c0b3150 0x7fc08c0ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 >> v1:172.21.15.181:6789/0 conn(0x7fc08c0ca740 legacy=0x7fc08c0cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:03.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c06c460 con 0x7fc08c0cbac0 2023-10-08T05:53:03.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc0840019f0 con 0x7fc08c0cbac0 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 >> v1:172.21.15.70:6789/0 conn(0x7fc08c0cbac0 legacy=0x7fc08c0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc078003970 0x7fc078003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0780047c0 0x7fc078004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc078005180 0x7fc07800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c06c180 con 0x7fc078005180 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07800e330 con 0x7fc0780047c0 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.968+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07800e4f0 con 0x7fc078003970 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.969+0000 7fc08ad9d640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc078005180 0x7fc07800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.969+0000 7fc08a59c640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc078003970 0x7fc078003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.969+0000 7fc083fff640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0780047c0 0x7fc078004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:03.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.969+0000 7fc08ad9d640 1 -- 172.21.15.70:0/2859803016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc078003970 msgr2=0x7fc078003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:03.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.969+0000 7fc08ad9d640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc078003970 0x7fc078003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.969+0000 7fc08ad9d640 1 -- 172.21.15.70:0/2859803016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0780047c0 msgr2=0x7fc078004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:03.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.969+0000 7fc08ad9d640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0780047c0 0x7fc078004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.969+0000 7fc08ad9d640 1 -- 172.21.15.70:0/2859803016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078003520 con 0x7fc078005180 2023-10-08T05:53:03.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.969+0000 7fc083fff640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc0780047c0 0x7fc078004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:03.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.970+0000 7fc08ad9d640 1 -- 172.21.15.70:0/2859803016 reap_dead start 2023-10-08T05:53:03.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.970+0000 7fc08ad9d640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc078005180 0x7fc07800db90 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc08400a830 tx=0x7fc0780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3cf5b4fb5c5c97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:03.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.970+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840072d0 con 0x7fc078005180 2023-10-08T05:53:03.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.971+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08402f070 con 0x7fc078005180 2023-10-08T05:53:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.971+0000 7fc089d9b640 1 -- 172.21.15.70:0/2859803016 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840076b0 con 0x7fc078005180 2023-10-08T05:53:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.971+0000 7fc08b59e640 1 -- 172.21.15.70:0/2859803016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc078005180 msgr2=0x7fc07800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.971+0000 7fc08b59e640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc078005180 0x7fc07800db90 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc08400a830 tx=0x7fc0780057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.971+0000 7fc08b59e640 1 -- 172.21.15.70:0/2859803016 shutdown_connections 2023-10-08T05:53:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.971+0000 7fc08b59e640 1 --2- 172.21.15.70:0/2859803016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc078005180 0x7fc07800db90 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.971+0000 7fc08b59e640 1 -- 172.21.15.70:0/2859803016 >> 172.21.15.70:0/2859803016 conn(0x7fc08c0ce9f0 msgr2=0x7fc08c0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.971+0000 7fc08b59e640 1 -- 172.21.15.70:0/2859803016 shutdown_connections 2023-10-08T05:53:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.971+0000 7fc08b59e640 1 -- 172.21.15.70:0/2859803016 wait complete. 2023-10-08T05:53:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.973+0000 7fc08b59e640 1 Processor -- start 2023-10-08T05:53:03.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.973+0000 7fc08b59e640 1 -- start start 2023-10-08T05:53:03.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.974+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc078005180 0x7fc08c0c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:03.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.974+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0ad1f0 0x7fc08c158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:03.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.974+0000 7fc08b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0c47e0 0x7fc08c0c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:03.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.974+0000 7fc08b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0c43c0 con 0x7fc08c0ad1f0 2023-10-08T05:53:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.974+0000 7fc08b59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0c87a0 con 0x7fc078005180 2023-10-08T05:53:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.974+0000 7fc08b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c0c4dd0 con 0x7fc08c0c47e0 2023-10-08T05:53:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc083fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0ad1f0 0x7fc08c158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc083fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0ad1f0 0x7fc08c158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46524/0 (socket says 172.21.15.70:46524) 2023-10-08T05:53:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc083fff640 1 -- 172.21.15.70:0/3785916063 learned_addr learned my addr 172.21.15.70:0/3785916063 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc08a59c640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc078005180 0x7fc08c0c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc083fff640 1 -- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0c47e0 msgr2=0x7fc08c0c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc083fff640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0c47e0 0x7fc08c0c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc083fff640 1 -- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc078005180 msgr2=0x7fc08c0c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc083fff640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc078005180 0x7fc08c0c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc083fff640 1 -- 172.21.15.70:0/3785916063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c06d0c0 con 0x7fc08c0ad1f0 2023-10-08T05:53:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc083fff640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0ad1f0 0x7fc08c158570 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc074000f60 tx=0x7fc07400d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.975+0000 7fc082ffd640 1 -- 172.21.15.70:0/3785916063 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074005480 con 0x7fc08c0ad1f0 2023-10-08T05:53:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.976+0000 7fc082ffd640 1 -- 172.21.15.70:0/3785916063 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc074003610 con 0x7fc08c0ad1f0 2023-10-08T05:53:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.976+0000 7fc082ffd640 1 -- 172.21.15.70:0/3785916063 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0740039c0 con 0x7fc08c0ad1f0 2023-10-08T05:53:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.976+0000 7fc08b59e640 1 -- 172.21.15.70:0/3785916063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c0c3a60 con 0x7fc08c0ad1f0 2023-10-08T05:53:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.976+0000 7fc08b59e640 1 -- 172.21.15.70:0/3785916063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c0c3fe0 con 0x7fc08c0ad1f0 2023-10-08T05:53:03.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.977+0000 7fc08b59e640 1 -- 172.21.15.70:0/3785916063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c0c6d70 con 0x7fc08c0ad1f0 2023-10-08T05:53:03.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.979+0000 7fc082ffd640 1 -- 172.21.15.70:0/3785916063 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc074003b20 con 0x7fc08c0ad1f0 2023-10-08T05:53:03.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.979+0000 7fc082ffd640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc06c06af30 0x7fc06c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:03.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.980+0000 7fc082ffd640 1 -- 172.21.15.70:0/3785916063 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5591+0+0 (secure 0 0 0) 0x7fc07408a040 con 0x7fc08c0ad1f0 2023-10-08T05:53:03.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.980+0000 7fc08a59c640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc06c06af30 0x7fc06c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:03.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.981+0000 7fc08a59c640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc06c06af30 0x7fc06c06d3f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0c12a0 tx=0x7fc07c008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:03.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:03.984+0000 7fc082ffd640 1 -- 172.21.15.70:0/3785916063 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc074055170 con 0x7fc08c0ad1f0 2023-10-08T05:53:04.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.179+0000 7fc08b59e640 1 -- 172.21.15.70:0/3785916063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc08c0c4500 con 0x7fc08c0ad1f0 2023-10-08T05:53:04.537 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:04.534+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:04.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.733+0000 7fc082ffd640 1 -- 172.21.15.70:0/3785916063 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc08c0c4500 con 0x7fc08c0ad1f0 2023-10-08T05:53:04.735 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T05:53:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.737+0000 7fc08b59e640 1 -- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc06c06af30 msgr2=0x7fc06c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.737+0000 7fc08b59e640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc06c06af30 0x7fc06c06d3f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0c12a0 tx=0x7fc07c008040 comp rx=0 tx=0).stop 2023-10-08T05:53:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.737+0000 7fc08b59e640 1 -- 172.21.15.70:0/3785916063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0ad1f0 msgr2=0x7fc08c158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.737+0000 7fc08b59e640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0ad1f0 0x7fc08c158570 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc074000f60 tx=0x7fc07400d9f0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.737+0000 7fc08b59e640 1 -- 172.21.15.70:0/3785916063 shutdown_connections 2023-10-08T05:53:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.737+0000 7fc08b59e640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc06c06af30 0x7fc06c06d3f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.737+0000 7fc08b59e640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc08c0c47e0 0x7fc08c0c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.737+0000 7fc08b59e640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc08c0ad1f0 0x7fc08c158570 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.737+0000 7fc08b59e640 1 --2- 172.21.15.70:0/3785916063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc078005180 0x7fc08c0c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.737+0000 7fc08b59e640 1 -- 172.21.15.70:0/3785916063 >> 172.21.15.70:0/3785916063 conn(0x7fc08c0ce9f0 msgr2=0x7fc08c0b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.738+0000 7fc08b59e640 1 -- 172.21.15.70:0/3785916063 shutdown_connections 2023-10-08T05:53:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.738+0000 7fc08b59e640 1 -- 172.21.15.70:0/3785916063 wait complete. 2023-10-08T05:53:04.773 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T05:53:04.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.961+0000 7f2e3554c640 1 Processor -- start 2023-10-08T05:53:04.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.962+0000 7f2e3554c640 1 -- start start 2023-10-08T05:53:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.963+0000 7f2e3554c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2e3015d0c0 0x7f2e300b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.963+0000 7f2e3554c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e300b30d0 0x7f2e300b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.963+0000 7f2e3554c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3006ce20 con 0x7f2e3015cce0 2023-10-08T05:53:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.963+0000 7f2e3554c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e300cf850 con 0x7f2e3015d0c0 2023-10-08T05:53:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.963+0000 7f2e3554c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3006c060 con 0x7f2e300b30d0 2023-10-08T05:53:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.964+0000 7f2e2effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2e3015cce0 0x7f2e300cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37042/0 (socket says 172.21.15.70:37042) 2023-10-08T05:53:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.964+0000 7f2e2effd640 1 -- 172.21.15.70:0/2776267255 learned_addr learned my addr 172.21.15.70:0/2776267255 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.964+0000 7f2e267fc640 1 --2- 172.21.15.70:0/2776267255 >> v2:172.21.15.181:3300/0 conn(0x7f2e3015d0c0 0x7f2e300b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.964+0000 7f2e2f7fe640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e300b30d0 0x7f2e300b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.964+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422755983 0 0) 0x7f2e3006ce20 con 0x7f2e3015cce0 2023-10-08T05:53:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.964+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e1c003520 con 0x7f2e3015cce0 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 550807189 0 0) 0x7f2e1c003520 con 0x7f2e3015cce0 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e300b30d0 msgr2=0x7f2e300b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e300b30d0 0x7f2e300b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 >> v2:172.21.15.181:3300/0 conn(0x7f2e3015d0c0 msgr2=0x7f2e300b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 --2- 172.21.15.70:0/2776267255 >> v2:172.21.15.181:3300/0 conn(0x7f2e3015d0c0 0x7f2e300b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3006c340 con 0x7f2e3015cce0 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2e20001670 con 0x7f2e3015cce0 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 >> v1:172.21.15.70:6789/0 conn(0x7f2e3015cce0 legacy=0x7f2e300cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c003970 0x7f2e1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e300b30d0 0x7f2e1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 0x7f2e1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300cf850 con 0x7f2e1c004d00 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3006c060 con 0x7f2e1c003970 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c00b980 con 0x7f2e300b30d0 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2f7fe640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 0x7f2e1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e267fc640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e300b30d0 0x7f2e1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.965+0000 7f2e2effd640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c003970 0x7f2e1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.966+0000 7f2e2f7fe640 1 -- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e300b30d0 msgr2=0x7f2e1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.966+0000 7f2e2f7fe640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e300b30d0 0x7f2e1c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.966+0000 7f2e2f7fe640 1 -- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c003970 msgr2=0x7f2e1c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.966+0000 7f2e2f7fe640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e1c003970 0x7f2e1c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.966+0000 7f2e2f7fe640 1 -- 172.21.15.70:0/2776267255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c003520 con 0x7f2e1c004d00 2023-10-08T05:53:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.967+0000 7f2e2f7fe640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 0x7f2e1c005180 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009fc0 tx=0x7f2e1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdfd532b8d03c2e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.967+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2802cc20 con 0x7f2e1c004d00 2023-10-08T05:53:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.967+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2800f040 con 0x7f2e1c004d00 2023-10-08T05:53:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.967+0000 7f2e2e7fc640 1 -- 172.21.15.70:0/2776267255 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28034440 con 0x7f2e1c004d00 2023-10-08T05:53:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.968+0000 7f2e3554c640 1 -- 172.21.15.70:0/2776267255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 msgr2=0x7f2e1c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.968+0000 7f2e3554c640 1 --2- 172.21.15.70:0/2776267255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 0x7f2e1c005180 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009fc0 tx=0x7f2e1c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.968+0000 7f2e2f7fe640 1 -- 172.21.15.70:0/2776267255 reap_dead start 2023-10-08T05:53:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.968+0000 7f2e3554c640 1 -- 172.21.15.70:0/2776267255 shutdown_connections 2023-10-08T05:53:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.968+0000 7f2e3554c640 1 -- 172.21.15.70:0/2776267255 >> 172.21.15.70:0/2776267255 conn(0x7f2e3006e650 msgr2=0x7f2e3006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.968+0000 7f2e3554c640 1 -- 172.21.15.70:0/2776267255 shutdown_connections 2023-10-08T05:53:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.968+0000 7f2e3554c640 1 -- 172.21.15.70:0/2776267255 wait complete. 2023-10-08T05:53:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.970+0000 7f2e3554c640 1 Processor -- start 2023-10-08T05:53:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.970+0000 7f2e3554c640 1 -- start start 2023-10-08T05:53:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e3554c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 0x7f2e300b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e3554c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e301f1f20 0x7f2e300ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e3554c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e301f23a0 0x7f2e300ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e3554c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300b5580 con 0x7f2e1c004d00 2023-10-08T05:53:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e3554c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3015ef70 con 0x7f2e301f1f20 2023-10-08T05:53:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e3554c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3015f250 con 0x7f2e301f23a0 2023-10-08T05:53:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e2effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 0x7f2e300b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e2effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 0x7f2e300b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46546/0 (socket says 172.21.15.70:46546) 2023-10-08T05:53:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e2effd640 1 -- 172.21.15.70:0/2582531266 learned_addr learned my addr 172.21.15.70:0/2582531266 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e267fc640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e301f1f20 0x7f2e300ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.971+0000 7f2e2f7fe640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e301f23a0 0x7f2e300ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e2effd640 1 -- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e301f23a0 msgr2=0x7f2e300ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e2effd640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e301f23a0 0x7f2e300ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e2effd640 1 -- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e301f1f20 msgr2=0x7f2e300ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e2effd640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e301f1f20 0x7f2e300ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e2effd640 1 -- 172.21.15.70:0/2582531266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3015f530 con 0x7f2e1c004d00 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e2effd640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 0x7f2e300b5040 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2e20000fb0 tx=0x7f2e2000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e27fff640 1 -- 172.21.15.70:0/2582531266 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000c840 con 0x7f2e1c004d00 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e27fff640 1 -- 172.21.15.70:0/2582531266 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2000c9c0 con 0x7f2e1c004d00 2023-10-08T05:53:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e27fff640 1 -- 172.21.15.70:0/2582531266 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000cd70 con 0x7f2e1c004d00 2023-10-08T05:53:04.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e3554c640 1 -- 172.21.15.70:0/2582531266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3015f810 con 0x7f2e1c004d00 2023-10-08T05:53:04.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.972+0000 7f2e3554c640 1 -- 172.21.15.70:0/2582531266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3015faf0 con 0x7f2e1c004d00 2023-10-08T05:53:04.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.974+0000 7f2e3554c640 1 -- 172.21.15.70:0/2582531266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3004e160 con 0x7f2e1c004d00 2023-10-08T05:53:04.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.975+0000 7f2e27fff640 1 -- 172.21.15.70:0/2582531266 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2e2001d460 con 0x7f2e1c004d00 2023-10-08T05:53:04.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.976+0000 7f2e27fff640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f2e1006af00 0x7f2e1006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:04.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.977+0000 7f2e27fff640 1 -- 172.21.15.70:0/2582531266 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5973+0+0 (secure 0 0 0) 0x7f2e2008a380 con 0x7f2e1c004d00 2023-10-08T05:53:04.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.977+0000 7f2e267fc640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f2e1006af00 0x7f2e1006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:04.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.979+0000 7f2e267fc640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f2e1006af00 0x7f2e1006d3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e300cad10 tx=0x7f2e18008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:04.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:04.981+0000 7f2e27fff640 1 -- 172.21.15.70:0/2582531266 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e200552b0 con 0x7f2e1c004d00 2023-10-08T05:53:05.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.222+0000 7f2e3554c640 1 -- 172.21.15.70:0/2582531266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2e300b6a30 con 0x7f2e1c004d00 2023-10-08T05:53:05.597 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:05.595+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:05.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.781+0000 7f2e27fff640 1 -- 172.21.15.70:0/2582531266 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v37) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2e20001630 con 0x7f2e1c004d00 2023-10-08T05:53:05.785 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 11 and data pool 12 2023-10-08T05:53:05.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.786+0000 7f2e3554c640 1 -- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f2e1006af00 msgr2=0x7f2e1006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:05.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.786+0000 7f2e3554c640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f2e1006af00 0x7f2e1006d3c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e300cad10 tx=0x7f2e18008040 comp rx=0 tx=0).stop 2023-10-08T05:53:05.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.786+0000 7f2e3554c640 1 -- 172.21.15.70:0/2582531266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 msgr2=0x7f2e300b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:05.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.786+0000 7f2e3554c640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 0x7f2e300b5040 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2e20000fb0 tx=0x7f2e2000ed50 comp rx=0 tx=0).stop 2023-10-08T05:53:05.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.786+0000 7f2e3554c640 1 -- 172.21.15.70:0/2582531266 shutdown_connections 2023-10-08T05:53:05.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.786+0000 7f2e3554c640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f2e1006af00 0x7f2e1006d3c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:05.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.786+0000 7f2e3554c640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e301f23a0 0x7f2e300ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:05.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.786+0000 7f2e3554c640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e301f1f20 0x7f2e300ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:05.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.786+0000 7f2e3554c640 1 --2- 172.21.15.70:0/2582531266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c004d00 0x7f2e300b5040 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:05.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.786+0000 7f2e3554c640 1 -- 172.21.15.70:0/2582531266 >> 172.21.15.70:0/2582531266 conn(0x7f2e3006e650 msgr2=0x7f2e300c71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:05.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.787+0000 7f2e3554c640 1 -- 172.21.15.70:0/2582531266 shutdown_connections 2023-10-08T05:53:05.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:05.787+0000 7f2e3554c640 1 -- 172.21.15.70:0/2582531266 wait complete. 2023-10-08T05:53:05.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:06.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.005+0000 7f63b5839640 1 Processor -- start 2023-10-08T05:53:06.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.005+0000 7f63b5839640 1 -- start start 2023-10-08T05:53:06.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.006+0000 7f63b5839640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b01599d0 0x7f63b00ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.006+0000 7f63b5839640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63b00b07d0 0x7f63b006c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.006+0000 7f63b5839640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b00cacf0 con 0x7f63b00cc210 2023-10-08T05:53:06.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.006+0000 7f63b5839640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b00b10f0 con 0x7f63b00b07d0 2023-10-08T05:53:06.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.006+0000 7f63b5839640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00ca3c0 con 0x7f63b01599d0 2023-10-08T05:53:06.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.007+0000 7f63b4837640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b01599d0 0x7f63b00ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.007+0000 7f63b5038640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63b00b07d0 0x7f63b006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.007+0000 7f63b5038640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63b00b07d0 0x7f63b006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:38034/0 (socket says 172.21.15.70:38034) 2023-10-08T05:53:06.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.007+0000 7f63b5038640 1 -- 172.21.15.70:0/814030867 learned_addr learned my addr 172.21.15.70:0/814030867 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:06.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.008+0000 7f63b4837640 1 -- 172.21.15.70:0/814030867 >> v2:172.21.15.181:3300/0 conn(0x7f63b00b07d0 msgr2=0x7f63b006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:06.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.008+0000 7f63b4837640 1 --2- 172.21.15.70:0/814030867 >> v2:172.21.15.181:3300/0 conn(0x7f63b00b07d0 0x7f63b006c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.008+0000 7f63b4837640 1 -- 172.21.15.70:0/814030867 >> v1:172.21.15.181:6789/0 conn(0x7f63b00cc210 legacy=0x7f63b00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.008+0000 7f63b4837640 1 -- 172.21.15.70:0/814030867 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b00ca6a0 con 0x7f63b01599d0 2023-10-08T05:53:06.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.008+0000 7f63b5038640 1 --2- 172.21.15.70:0/814030867 >> v2:172.21.15.181:3300/0 conn(0x7f63b00b07d0 0x7f63b006c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:53:06.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.009+0000 7f63b4837640 1 --2- 172.21.15.70:0/814030867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b01599d0 0x7f63b00ad730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f63a40097a0 tx=0x7f63a402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c70a3d31c6556ef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:06.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.010+0000 7f63af7fe640 1 -- 172.21.15.70:0/814030867 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a402d040 con 0x7f63b01599d0 2023-10-08T05:53:06.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.010+0000 7f63af7fe640 1 -- 172.21.15.70:0/814030867 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a4031810 con 0x7f63b01599d0 2023-10-08T05:53:06.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.010+0000 7f63af7fe640 1 -- 172.21.15.70:0/814030867 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4031ba0 con 0x7f63b01599d0 2023-10-08T05:53:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.010+0000 7f63b5839640 1 -- 172.21.15.70:0/814030867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b01599d0 msgr2=0x7f63b00ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.010+0000 7f63b5839640 1 --2- 172.21.15.70:0/814030867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b01599d0 0x7f63b00ad730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f63a40097a0 tx=0x7f63a402b8b0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.010+0000 7f63b5839640 1 -- 172.21.15.70:0/814030867 shutdown_connections 2023-10-08T05:53:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.010+0000 7f63b5839640 1 --2- 172.21.15.70:0/814030867 >> v2:172.21.15.181:3300/0 conn(0x7f63b00b07d0 0x7f63b006c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.010+0000 7f63b5839640 1 --2- 172.21.15.70:0/814030867 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b01599d0 0x7f63b00ad730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.010+0000 7f63b5839640 1 -- 172.21.15.70:0/814030867 >> 172.21.15.70:0/814030867 conn(0x7f63b00cb0c0 msgr2=0x7f63b00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.011+0000 7f63b5839640 1 -- 172.21.15.70:0/814030867 shutdown_connections 2023-10-08T05:53:06.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.011+0000 7f63b5839640 1 -- 172.21.15.70:0/814030867 wait complete. 2023-10-08T05:53:06.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.012+0000 7f63b5839640 1 Processor -- start 2023-10-08T05:53:06.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.013+0000 7f63b5839640 1 -- start start 2023-10-08T05:53:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.014+0000 7f63b5839640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b00b07d0 0x7f63b01484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.014+0000 7f63b5839640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00cc210 0x7f63b00ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.014+0000 7f63b5839640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b01599d0 0x7f63b015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.014+0000 7f63b5839640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b01e7150 con 0x7f63b01599d0 2023-10-08T05:53:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.014+0000 7f63b5839640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00ae8c0 con 0x7f63b00cc210 2023-10-08T05:53:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.014+0000 7f63b5839640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00aeba0 con 0x7f63b00b07d0 2023-10-08T05:53:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.014+0000 7f63b5038640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b01599d0 0x7f63b015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.014+0000 7f63b5038640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b01599d0 0x7f63b015c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46552/0 (socket says 172.21.15.70:46552) 2023-10-08T05:53:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.014+0000 7f63b5038640 1 -- 172.21.15.70:0/2056779812 learned_addr learned my addr 172.21.15.70:0/2056779812 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:06.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63b5038640 1 -- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b00b07d0 msgr2=0x7f63b01484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63b5038640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b00b07d0 0x7f63b01484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63b5038640 1 -- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00cc210 msgr2=0x7f63b00ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63affff640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00cc210 0x7f63b00ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63b5038640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00cc210 0x7f63b00ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63b5038640 1 -- 172.21.15.70:0/2056779812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a800bd40 con 0x7f63b01599d0 2023-10-08T05:53:06.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63affff640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00cc210 0x7f63b00ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:06.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63b5038640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b01599d0 0x7f63b015c500 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f63a800e370 tx=0x7f63a800e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:06.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63ad7fa640 1 -- 172.21.15.70:0/2056779812 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8010040 con 0x7f63b01599d0 2023-10-08T05:53:06.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63ad7fa640 1 -- 172.21.15.70:0/2056779812 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a800b2f0 con 0x7f63b01599d0 2023-10-08T05:53:06.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63ad7fa640 1 -- 172.21.15.70:0/2056779812 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8009840 con 0x7f63b01599d0 2023-10-08T05:53:06.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63b5839640 1 -- 172.21.15.70:0/2056779812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a4009400 con 0x7f63b01599d0 2023-10-08T05:53:06.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.015+0000 7f63b5839640 1 -- 172.21.15.70:0/2056779812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b00af160 con 0x7f63b01599d0 2023-10-08T05:53:06.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.017+0000 7f63b5839640 1 -- 172.21.15.70:0/2056779812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b004e160 con 0x7f63b01599d0 2023-10-08T05:53:06.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.018+0000 7f63ad7fa640 1 -- 172.21.15.70:0/2056779812 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f63a80099a0 con 0x7f63b01599d0 2023-10-08T05:53:06.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.019+0000 7f63ad7fa640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f639406af00 0x7f639406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.019+0000 7f63ad7fa640 1 -- 172.21.15.70:0/2056779812 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f63a8089fa0 con 0x7f63b01599d0 2023-10-08T05:53:06.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.019+0000 7f63b4837640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f639406af00 0x7f639406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.020+0000 7f63b4837640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f639406af00 0x7f639406d3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f63a4036000 tx=0x7f63a4030000 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:06.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.024+0000 7f63ad7fa640 1 -- 172.21.15.70:0/2056779812 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63a8054e70 con 0x7f63b01599d0 2023-10-08T05:53:06.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.225+0000 7f63b5839640 1 -- 172.21.15.70:0/2056779812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63b0070550 con 0x7f63b01599d0 2023-10-08T05:53:06.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.227+0000 7f63ad7fa640 1 -- 172.21.15.70:0/2056779812 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16859 (secure 0 0 0) 0x7f63a8058490 con 0x7f63b01599d0 2023-10-08T05:53:06.229 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:06.229 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":103,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:05.224581+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:03.051784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:04.230588+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:06.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 -- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f639406af00 msgr2=0x7f639406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f639406af00 0x7f639406d3c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f63a4036000 tx=0x7f63a4030000 comp rx=0 tx=0).stop 2023-10-08T05:53:06.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 -- 172.21.15.70:0/2056779812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b01599d0 msgr2=0x7f63b015c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b01599d0 0x7f63b015c500 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f63a800e370 tx=0x7f63a800e830 comp rx=0 tx=0).stop 2023-10-08T05:53:06.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 -- 172.21.15.70:0/2056779812 shutdown_connections 2023-10-08T05:53:06.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f639406af00 0x7f639406d3c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b01599d0 0x7f63b015c500 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00cc210 0x7f63b00ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 --2- 172.21.15.70:0/2056779812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b00b07d0 0x7f63b01484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 -- 172.21.15.70:0/2056779812 >> 172.21.15.70:0/2056779812 conn(0x7f63b00cb0c0 msgr2=0x7f63b00acc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:06.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 -- 172.21.15.70:0/2056779812 shutdown_connections 2023-10-08T05:53:06.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.230+0000 7f63b5839640 1 -- 172.21.15.70:0/2056779812 wait complete. 2023-10-08T05:53:06.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:06.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.462+0000 7f52c7fff640 1 Processor -- start 2023-10-08T05:53:06.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.462+0000 7f52c7fff640 1 -- start start 2023-10-08T05:53:06.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f52c815ab30 0x7f52c815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c7fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c80b07d0 0x7f52c806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c7fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c815f7d0 con 0x7f52c80adc70 2023-10-08T05:53:06.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c7fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c815f210 con 0x7f52c815ab30 2023-10-08T05:53:06.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c7fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c815f4f0 con 0x7f52c80b07d0 2023-10-08T05:53:06.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c6ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f52c815ab30 0x7f52c815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c6ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f52c815ab30 0x7f52c815e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46566/0 (socket says 172.21.15.70:46566) 2023-10-08T05:53:06.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c6ffd640 1 -- 172.21.15.70:0/3203394223 learned_addr learned my addr 172.21.15.70:0/3203394223 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:06.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c6ffd640 1 -- 172.21.15.70:0/3203394223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c80b07d0 msgr2=0x7f52c806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:06.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c6ffd640 1 --2- 172.21.15.70:0/3203394223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c80b07d0 0x7f52c806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c6ffd640 1 -- 172.21.15.70:0/3203394223 >> v1:172.21.15.70:6789/0 conn(0x7f52c80adc70 legacy=0x7f52c80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.464+0000 7f52c6ffd640 1 -- 172.21.15.70:0/3203394223 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c806c550 con 0x7f52c815ab30 2023-10-08T05:53:06.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.466+0000 7f52c6ffd640 1 --2- 172.21.15.70:0/3203394223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c815ab30 0x7f52c815e8a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009870 tx=0x7f52b802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178d9f34c7687e22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:06.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.466+0000 7f52c5ffb640 1 -- 172.21.15.70:0/3203394223 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8030840 con 0x7f52c815ab30 2023-10-08T05:53:06.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.466+0000 7f52c5ffb640 1 -- 172.21.15.70:0/3203394223 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b80309a0 con 0x7f52c815ab30 2023-10-08T05:53:06.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.466+0000 7f52c5ffb640 1 -- 172.21.15.70:0/3203394223 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8030d30 con 0x7f52c815ab30 2023-10-08T05:53:06.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.467+0000 7f52c7fff640 1 -- 172.21.15.70:0/3203394223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c815ab30 msgr2=0x7f52c815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.467+0000 7f52c7fff640 1 --2- 172.21.15.70:0/3203394223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c815ab30 0x7f52c815e8a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009870 tx=0x7f52b802b980 comp rx=0 tx=0).stop 2023-10-08T05:53:06.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.467+0000 7f52c7fff640 1 -- 172.21.15.70:0/3203394223 shutdown_connections 2023-10-08T05:53:06.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.467+0000 7f52c7fff640 1 --2- 172.21.15.70:0/3203394223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c80b07d0 0x7f52c806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.467+0000 7f52c7fff640 1 --2- 172.21.15.70:0/3203394223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c815ab30 0x7f52c815e8a0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.467+0000 7f52c7fff640 1 -- 172.21.15.70:0/3203394223 >> 172.21.15.70:0/3203394223 conn(0x7f52c80cb0c0 msgr2=0x7f52c80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:06.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.467+0000 7f52c7fff640 1 -- 172.21.15.70:0/3203394223 shutdown_connections 2023-10-08T05:53:06.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.467+0000 7f52c7fff640 1 -- 172.21.15.70:0/3203394223 wait complete. 2023-10-08T05:53:06.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.469+0000 7f52c7fff640 1 Processor -- start 2023-10-08T05:53:06.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.469+0000 7f52c7fff640 1 -- start start 2023-10-08T05:53:06.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c7fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c80adc70 0x7f52c814f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c80b07d0 0x7f52c80ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c815ab30 0x7f52c80cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c7fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80c6200 con 0x7f52c80b07d0 2023-10-08T05:53:06.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c7fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80cc2b0 con 0x7f52c815ab30 2023-10-08T05:53:06.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c7fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80cc590 con 0x7f52c80adc70 2023-10-08T05:53:06.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c80b07d0 0x7f52c80ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c80b07d0 0x7f52c80ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46576/0 (socket says 172.21.15.70:46576) 2023-10-08T05:53:06.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c67fc640 1 -- 172.21.15.70:0/3695405231 learned_addr learned my addr 172.21.15.70:0/3695405231 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:06.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c67fc640 1 -- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c80adc70 msgr2=0x7f52c814f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c6ffd640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c80adc70 0x7f52c814f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c67fc640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c80adc70 0x7f52c814f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c67fc640 1 -- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c815ab30 msgr2=0x7f52c80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c67fc640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c815ab30 0x7f52c80cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.470+0000 7f52c67fc640 1 -- 172.21.15.70:0/3695405231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b80094d0 con 0x7f52c80b07d0 2023-10-08T05:53:06.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.471+0000 7f52c67fc640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c80b07d0 0x7f52c80ccd90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c450 tx=0x7f52bc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:06.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.471+0000 7f52af7fe640 1 -- 172.21.15.70:0/3695405231 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0052d0 con 0x7f52c80b07d0 2023-10-08T05:53:06.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.471+0000 7f52af7fe640 1 -- 172.21.15.70:0/3695405231 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc005430 con 0x7f52c80b07d0 2023-10-08T05:53:06.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.471+0000 7f52af7fe640 1 -- 172.21.15.70:0/3695405231 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc012870 con 0x7f52c80b07d0 2023-10-08T05:53:06.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.471+0000 7f52c7fff640 1 -- 172.21.15.70:0/3695405231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c80cc870 con 0x7f52c80b07d0 2023-10-08T05:53:06.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.471+0000 7f52c7fff640 1 -- 172.21.15.70:0/3695405231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c80c64e0 con 0x7f52c80b07d0 2023-10-08T05:53:06.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.472+0000 7f52c7fff640 1 -- 172.21.15.70:0/3695405231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c80ccab0 con 0x7f52c80b07d0 2023-10-08T05:53:06.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.473+0000 7f52af7fe640 1 -- 172.21.15.70:0/3695405231 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f52bc0129d0 con 0x7f52c80b07d0 2023-10-08T05:53:06.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.474+0000 7f52af7fe640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f52b406af00 0x7f52b406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.474+0000 7f52af7fe640 1 -- 172.21.15.70:0/3695405231 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f52bc089340 con 0x7f52c80b07d0 2023-10-08T05:53:06.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.475+0000 7f52c6ffd640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f52b406af00 0x7f52b406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.476+0000 7f52c6ffd640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f52b406af00 0x7f52b406d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52b8036000 tx=0x7f52b803b040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:06.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.479+0000 7f52af7fe640 1 -- 172.21.15.70:0/3695405231 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52c80ccab0 con 0x7f52c80b07d0 2023-10-08T05:53:06.634 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:06.631+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:06.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.673+0000 7f52c7fff640 1 -- 172.21.15.70:0/3695405231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52c80ccab0 con 0x7f52c80b07d0 2023-10-08T05:53:06.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.676+0000 7f52af7fe640 1 -- 172.21.15.70:0/3695405231 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16859 (secure 0 0 0) 0x7f52c80ccab0 con 0x7f52c80b07d0 2023-10-08T05:53:06.678 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:06.678 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":103,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:05.224581+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:03.051784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:04.230588+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:06.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.679+0000 7f52c7fff640 1 -- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f52b406af00 msgr2=0x7f52b406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.679+0000 7f52c7fff640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f52b406af00 0x7f52b406d3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52b8036000 tx=0x7f52b803b040 comp rx=0 tx=0).stop 2023-10-08T05:53:06.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.679+0000 7f52c7fff640 1 -- 172.21.15.70:0/3695405231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c80b07d0 msgr2=0x7f52c80ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.679+0000 7f52c7fff640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c80b07d0 0x7f52c80ccd90 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c450 tx=0x7f52bc00c910 comp rx=0 tx=0).stop 2023-10-08T05:53:06.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.680+0000 7f52c7fff640 1 -- 172.21.15.70:0/3695405231 shutdown_connections 2023-10-08T05:53:06.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.680+0000 7f52c7fff640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f52b406af00 0x7f52b406d3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.680+0000 7f52c7fff640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c815ab30 0x7f52c80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.680+0000 7f52c7fff640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c80b07d0 0x7f52c80ccd90 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.680+0000 7f52c7fff640 1 --2- 172.21.15.70:0/3695405231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c80adc70 0x7f52c814f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.680+0000 7f52c7fff640 1 -- 172.21.15.70:0/3695405231 >> 172.21.15.70:0/3695405231 conn(0x7f52c80cb0c0 msgr2=0x7f52c80b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:06.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.680+0000 7f52c7fff640 1 -- 172.21.15.70:0/3695405231 shutdown_connections 2023-10-08T05:53:06.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.680+0000 7f52c7fff640 1 -- 172.21.15.70:0/3695405231 wait complete. 2023-10-08T05:53:06.701 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T05:53:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.911+0000 7fc600687640 1 Processor -- start 2023-10-08T05:53:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.912+0000 7fc600687640 1 -- start start 2023-10-08T05:53:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.913+0000 7fc600687640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5f806dfc0 0x7fc5f8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.913+0000 7fc600687640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5f8071070 0x7fc5f80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.913+0000 7fc600687640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f80b1cc0 con 0x7fc5f80afbe0 2023-10-08T05:53:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.913+0000 7fc600687640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f80b2620 con 0x7fc5f8071070 2023-10-08T05:53:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.913+0000 7fc600687640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f80b2060 con 0x7fc5f806dfc0 2023-10-08T05:53:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.913+0000 7fc5fec24640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5f8071070 0x7fc5f80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.914+0000 7fc5fec24640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5f8071070 0x7fc5f80cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46590/0 (socket says 172.21.15.70:46590) 2023-10-08T05:53:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.914+0000 7fc5fec24640 1 -- 172.21.15.70:0/1660752726 learned_addr learned my addr 172.21.15.70:0/1660752726 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.914+0000 7fc5fdc22640 1 --2- 172.21.15.70:0/1660752726 >> v2:172.21.15.181:3300/0 conn(0x7fc5f806dfc0 0x7fc5f8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.914+0000 7fc5fec24640 1 -- 172.21.15.70:0/1660752726 >> v2:172.21.15.181:3300/0 conn(0x7fc5f806dfc0 msgr2=0x7fc5f8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.914+0000 7fc5fec24640 1 --2- 172.21.15.70:0/1660752726 >> v2:172.21.15.181:3300/0 conn(0x7fc5f806dfc0 0x7fc5f8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.914+0000 7fc5fec24640 1 -- 172.21.15.70:0/1660752726 >> v1:172.21.15.181:6789/0 conn(0x7fc5f80afbe0 legacy=0x7fc5f80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.914+0000 7fc5fec24640 1 -- 172.21.15.70:0/1660752726 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f80b2340 con 0x7fc5f8071070 2023-10-08T05:53:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.914+0000 7fc5fdc22640 1 --2- 172.21.15.70:0/1660752726 >> v2:172.21.15.181:3300/0 conn(0x7fc5f806dfc0 0x7fc5f8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.915+0000 7fc5fec24640 1 --2- 172.21.15.70:0/1660752726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f8071070 0x7fc5f80cb130 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80b3980 tx=0x7fc5f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c34c5b9486f831e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc5fd421640 1 -- 172.21.15.70:0/1660752726 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0031870 con 0x7fc5f8071070 2023-10-08T05:53:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc5fd421640 1 -- 172.21.15.70:0/1660752726 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f00319d0 con 0x7fc5f8071070 2023-10-08T05:53:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc5fd421640 1 -- 172.21.15.70:0/1660752726 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0031d60 con 0x7fc5f8071070 2023-10-08T05:53:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc600687640 1 -- 172.21.15.70:0/1660752726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f8071070 msgr2=0x7fc5f80cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc600687640 1 --2- 172.21.15.70:0/1660752726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f8071070 0x7fc5f80cb130 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80b3980 tx=0x7fc5f002b910 comp rx=0 tx=0).stop 2023-10-08T05:53:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc600687640 1 -- 172.21.15.70:0/1660752726 shutdown_connections 2023-10-08T05:53:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc600687640 1 --2- 172.21.15.70:0/1660752726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f8071070 0x7fc5f80cb130 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc600687640 1 --2- 172.21.15.70:0/1660752726 >> v2:172.21.15.181:3300/0 conn(0x7fc5f806dfc0 0x7fc5f8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc600687640 1 -- 172.21.15.70:0/1660752726 >> 172.21.15.70:0/1660752726 conn(0x7fc5f806a5d0 msgr2=0x7fc5f80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc600687640 1 -- 172.21.15.70:0/1660752726 shutdown_connections 2023-10-08T05:53:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.916+0000 7fc600687640 1 -- 172.21.15.70:0/1660752726 wait complete. 2023-10-08T05:53:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.918+0000 7fc600687640 1 Processor -- start 2023-10-08T05:53:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.918+0000 7fc600687640 1 -- start start 2023-10-08T05:53:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.919+0000 7fc600687640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f806dfc0 0x7fc5f80c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.919+0000 7fc600687640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f8071070 0x7fc5f81f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.919+0000 7fc600687640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f80afbe0 0x7fc5f8150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.919+0000 7fc600687640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c5350 con 0x7fc5f806dfc0 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.919+0000 7fc600687640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80cdae0 con 0x7fc5f80afbe0 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.919+0000 7fc600687640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80cddc0 con 0x7fc5f8071070 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fe423640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f806dfc0 0x7fc5f80c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fe423640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f806dfc0 0x7fc5f80c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46596/0 (socket says 172.21.15.70:46596) 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fe423640 1 -- 172.21.15.70:0/2227016202 learned_addr learned my addr 172.21.15.70:0/2227016202 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fdc22640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f8071070 0x7fc5f81f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fec24640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f80afbe0 0x7fc5f8150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fe423640 1 -- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f8071070 msgr2=0x7fc5f81f2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fe423640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f8071070 0x7fc5f81f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fe423640 1 -- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f80afbe0 msgr2=0x7fc5f8150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fe423640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f80afbe0 0x7fc5f8150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fe423640 1 -- 172.21.15.70:0/2227016202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00094d0 con 0x7fc5f806dfc0 2023-10-08T05:53:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.920+0000 7fc5fe423640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f806dfc0 0x7fc5f80c4e10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800b3f0 tx=0x7fc5e800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.921+0000 7fc5eeffd640 1 -- 172.21.15.70:0/2227016202 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8011830 con 0x7fc5f806dfc0 2023-10-08T05:53:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.921+0000 7fc5eeffd640 1 -- 172.21.15.70:0/2227016202 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e8011990 con 0x7fc5f806dfc0 2023-10-08T05:53:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.921+0000 7fc5eeffd640 1 -- 172.21.15.70:0/2227016202 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8011d40 con 0x7fc5f806dfc0 2023-10-08T05:53:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.921+0000 7fc600687640 1 -- 172.21.15.70:0/2227016202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f80ce0a0 con 0x7fc5f806dfc0 2023-10-08T05:53:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.921+0000 7fc600687640 1 -- 172.21.15.70:0/2227016202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f80ce660 con 0x7fc5f806dfc0 2023-10-08T05:53:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.922+0000 7fc600687640 1 -- 172.21.15.70:0/2227016202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f80ce880 con 0x7fc5f806dfc0 2023-10-08T05:53:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.923+0000 7fc5eeffd640 1 -- 172.21.15.70:0/2227016202 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fc5e801a400 con 0x7fc5f806dfc0 2023-10-08T05:53:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.924+0000 7fc5eeffd640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc5e406af30 0x7fc5e406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.924+0000 7fc5eeffd640 1 -- 172.21.15.70:0/2227016202 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc5e80894c0 con 0x7fc5f806dfc0 2023-10-08T05:53:06.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.925+0000 7fc5fdc22640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc5e406af30 0x7fc5e406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:06.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.926+0000 7fc5fdc22640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc5e406af30 0x7fc5e406d3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5f806e410 tx=0x7fc5f40084f0 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:06.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:06.928+0000 7fc5eeffd640 1 -- 172.21.15.70:0/2227016202 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5e80544a0 con 0x7fc5f806dfc0 2023-10-08T05:53:07.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.157+0000 7fc600687640 1 -- 172.21.15.70:0/2227016202 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc5f804e160 con 0x7fc5f806dfc0 2023-10-08T05:53:07.712 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:07.709+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:07.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.791+0000 7fc5eeffd640 1 -- 172.21.15.70:0/2227016202 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc5e8057ac0 con 0x7fc5f806dfc0 2023-10-08T05:53:07.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.795+0000 7fc600687640 1 -- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc5e406af30 msgr2=0x7fc5e406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:07.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc5e406af30 0x7fc5e406d3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc5f806e410 tx=0x7fc5f40084f0 comp rx=0 tx=0).stop 2023-10-08T05:53:07.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 -- 172.21.15.70:0/2227016202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f806dfc0 msgr2=0x7fc5f80c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:07.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f806dfc0 0x7fc5f80c4e10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800b3f0 tx=0x7fc5e800b8b0 comp rx=0 tx=0).stop 2023-10-08T05:53:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 -- 172.21.15.70:0/2227016202 shutdown_connections 2023-10-08T05:53:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fc5e406af30 0x7fc5e406d3f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5f80afbe0 0x7fc5f8150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5f8071070 0x7fc5f81f2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 --2- 172.21.15.70:0/2227016202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f806dfc0 0x7fc5f80c4e10 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 -- 172.21.15.70:0/2227016202 >> 172.21.15.70:0/2227016202 conn(0x7fc5f806a5d0 msgr2=0x7fc5f80c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 -- 172.21.15.70:0/2227016202 shutdown_connections 2023-10-08T05:53:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:07.796+0000 7fc600687640 1 -- 172.21.15.70:0/2227016202 wait complete. 2023-10-08T05:53:07.818 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T05:53:07.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T05:53:08.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.034+0000 7fca35d0c640 1 Processor -- start 2023-10-08T05:53:08.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.034+0000 7fca35d0c640 1 -- start start 2023-10-08T05:53:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.036+0000 7fca35d0c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca300d00e0 0x7fca300d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.036+0000 7fca35d0c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fca300d0b90 0x7fca300b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.036+0000 7fca35d0c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca300d2870 con 0x7fca3015ccb0 2023-10-08T05:53:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.036+0000 7fca35d0c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca300d1150 con 0x7fca300d0b90 2023-10-08T05:53:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.036+0000 7fca35d0c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca300d1430 con 0x7fca300d00e0 2023-10-08T05:53:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.036+0000 7fca34d0a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fca3015ccb0 0x7fca300cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37060/0 (socket says 172.21.15.70:37060) 2023-10-08T05:53:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.036+0000 7fca34d0a640 1 -- 172.21.15.70:0/1308932120 learned_addr learned my addr 172.21.15.70:0/1308932120 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2ffff640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca300d00e0 0x7fca300d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662509859 0 0) 0x7fca300d2870 con 0x7fca3015ccb0 2023-10-08T05:53:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca3550b640 1 --2- 172.21.15.70:0/1308932120 >> v2:172.21.15.181:3300/0 conn(0x7fca300d0b90 0x7fca300b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca20003520 con 0x7fca3015ccb0 2023-10-08T05:53:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1905261243 0 0) 0x7fca20003520 con 0x7fca3015ccb0 2023-10-08T05:53:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca300d00e0 msgr2=0x7fca300d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca300d00e0 0x7fca300d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 >> v2:172.21.15.181:3300/0 conn(0x7fca300d0b90 msgr2=0x7fca300b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 --2- 172.21.15.70:0/1308932120 >> v2:172.21.15.181:3300/0 conn(0x7fca300d0b90 0x7fca300b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300d1710 con 0x7fca3015ccb0 2023-10-08T05:53:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fca24001670 con 0x7fca3015ccb0 2023-10-08T05:53:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 >> v1:172.21.15.70:6789/0 conn(0x7fca3015ccb0 legacy=0x7fca300cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca20003970 0x7fca20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.037+0000 7fca2f7fe640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca300d00e0 0x7fca200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca2f7fe640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca20004d00 0x7fca20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300d1150 con 0x7fca20004d00 2023-10-08T05:53:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300d1430 con 0x7fca20003970 2023-10-08T05:53:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2000b9b0 con 0x7fca300d00e0 2023-10-08T05:53:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca3550b640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca20004d00 0x7fca20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca34d0a640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca20003970 0x7fca20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca2ffff640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca300d00e0 0x7fca200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca3550b640 1 -- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca300d00e0 msgr2=0x7fca200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca3550b640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca300d00e0 0x7fca200047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca3550b640 1 -- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca20003970 msgr2=0x7fca20003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca3550b640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca20003970 0x7fca20003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.038+0000 7fca3550b640 1 -- 172.21.15.70:0/1308932120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca20003520 con 0x7fca20004d00 2023-10-08T05:53:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.039+0000 7fca3550b640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca20004d00 0x7fca20005180 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fca28007360 tx=0x7fca200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c212c73bb5d910 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.040+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2802ccf0 con 0x7fca20004d00 2023-10-08T05:53:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.040+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2802ce50 con 0x7fca20004d00 2023-10-08T05:53:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.040+0000 7fca2f7fe640 1 -- 172.21.15.70:0/1308932120 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28034660 con 0x7fca20004d00 2023-10-08T05:53:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.040+0000 7fca35d0c640 1 -- 172.21.15.70:0/1308932120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca20004d00 msgr2=0x7fca20005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.040+0000 7fca35d0c640 1 --2- 172.21.15.70:0/1308932120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca20004d00 0x7fca20005180 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fca28007360 tx=0x7fca200057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.040+0000 7fca3550b640 1 -- 172.21.15.70:0/1308932120 reap_dead start 2023-10-08T05:53:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.040+0000 7fca35d0c640 1 -- 172.21.15.70:0/1308932120 shutdown_connections 2023-10-08T05:53:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.040+0000 7fca35d0c640 1 -- 172.21.15.70:0/1308932120 >> 172.21.15.70:0/1308932120 conn(0x7fca30071660 msgr2=0x7fca30071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.041+0000 7fca35d0c640 1 -- 172.21.15.70:0/1308932120 shutdown_connections 2023-10-08T05:53:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.041+0000 7fca35d0c640 1 -- 172.21.15.70:0/1308932120 wait complete. 2023-10-08T05:53:08.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.042+0000 7fca35d0c640 1 Processor -- start 2023-10-08T05:53:08.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.042+0000 7fca35d0c640 1 -- start start 2023-10-08T05:53:08.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.043+0000 7fca35d0c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca20004d00 0x7fca300c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:08.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.043+0000 7fca35d0c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca300b70e0 0x7fca3014dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:08.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.043+0000 7fca35d0c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca300c8500 0x7fca300c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:08.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.043+0000 7fca35d0c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300c7b30 con 0x7fca300c8500 2023-10-08T05:53:08.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.043+0000 7fca35d0c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300c8ef0 con 0x7fca300b70e0 2023-10-08T05:53:08.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.043+0000 7fca35d0c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca300c7640 con 0x7fca20004d00 2023-10-08T05:53:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.044+0000 7fca3550b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca300c8500 0x7fca300c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.044+0000 7fca3550b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca300c8500 0x7fca300c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46616/0 (socket says 172.21.15.70:46616) 2023-10-08T05:53:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.044+0000 7fca3550b640 1 -- 172.21.15.70:0/1520364048 learned_addr learned my addr 172.21.15.70:0/1520364048 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.044+0000 7fca2ffff640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca300b70e0 0x7fca3014dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.044+0000 7fca34d0a640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca20004d00 0x7fca300c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:08.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.044+0000 7fca3550b640 1 -- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca20004d00 msgr2=0x7fca300c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.044+0000 7fca3550b640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca20004d00 0x7fca300c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.044+0000 7fca3550b640 1 -- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca300b70e0 msgr2=0x7fca3014dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.044+0000 7fca3550b640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca300b70e0 0x7fca3014dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.044+0000 7fca3550b640 1 -- 172.21.15.70:0/1520364048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca301600e0 con 0x7fca300c8500 2023-10-08T05:53:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.045+0000 7fca3550b640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca300c8500 0x7fca300c89b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fca2802dfd0 tx=0x7fca28004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:08.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.045+0000 7fca2d7fa640 1 -- 172.21.15.70:0/1520364048 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28044070 con 0x7fca300c8500 2023-10-08T05:53:08.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.045+0000 7fca2d7fa640 1 -- 172.21.15.70:0/1520364048 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2803e400 con 0x7fca300c8500 2023-10-08T05:53:08.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.045+0000 7fca2d7fa640 1 -- 172.21.15.70:0/1520364048 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2803e7d0 con 0x7fca300c8500 2023-10-08T05:53:08.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.045+0000 7fca35d0c640 1 -- 172.21.15.70:0/1520364048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca300c7850 con 0x7fca300c8500 2023-10-08T05:53:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.045+0000 7fca35d0c640 1 -- 172.21.15.70:0/1520364048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3015e560 con 0x7fca300c8500 2023-10-08T05:53:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.046+0000 7fca35d0c640 1 -- 172.21.15.70:0/1520364048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3015e0e0 con 0x7fca300c8500 2023-10-08T05:53:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.047+0000 7fca2d7fa640 1 -- 172.21.15.70:0/1520364048 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fca2803e930 con 0x7fca300c8500 2023-10-08T05:53:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.048+0000 7fca2d7fa640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fca1406af00 0x7fca1406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.048+0000 7fca2d7fa640 1 -- 172.21.15.70:0/1520364048 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fca280aaf40 con 0x7fca300c8500 2023-10-08T05:53:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.049+0000 7fca34d0a640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fca1406af00 0x7fca1406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.050+0000 7fca34d0a640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fca1406af00 0x7fca1406d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca24000c00 tx=0x7fca24009040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.053+0000 7fca2d7fa640 1 -- 172.21.15.70:0/1520364048 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca3015e0e0 con 0x7fca300c8500 2023-10-08T05:53:08.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.289+0000 7fca35d0c640 1 -- 172.21.15.70:0/1520364048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fca3015e0e0 con 0x7fca300c8500 2023-10-08T05:53:08.785 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:08.783+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:08.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.794+0000 7fca2d7fa640 1 -- 172.21.15.70:0/1520364048 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7fca3015e0e0 con 0x7fca300c8500 2023-10-08T05:53:08.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 -- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fca1406af00 msgr2=0x7fca1406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:08.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fca1406af00 0x7fca1406d3c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fca24000c00 tx=0x7fca24009040 comp rx=0 tx=0).stop 2023-10-08T05:53:08.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 -- 172.21.15.70:0/1520364048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca300c8500 msgr2=0x7fca300c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:08.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca300c8500 0x7fca300c89b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fca2802dfd0 tx=0x7fca28004ab0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 -- 172.21.15.70:0/1520364048 shutdown_connections 2023-10-08T05:53:08.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fca1406af00 0x7fca1406d3c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca300c8500 0x7fca300c89b0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fca300b70e0 0x7fca3014dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 --2- 172.21.15.70:0/1520364048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fca20004d00 0x7fca300c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:08.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 -- 172.21.15.70:0/1520364048 >> 172.21.15.70:0/1520364048 conn(0x7fca30071660 msgr2=0x7fca30157d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:08.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 -- 172.21.15.70:0/1520364048 shutdown_connections 2023-10-08T05:53:08.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:08.798+0000 7fca35d0c640 1 -- 172.21.15.70:0/1520364048 wait complete. 2023-10-08T05:53:08.819 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T05:53:08.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:53:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.036+0000 7f61b514c640 1 Processor -- start 2023-10-08T05:53:09.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.036+0000 7f61b514c640 1 -- start start 2023-10-08T05:53:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.037+0000 7f61b514c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f61b0159ac0 0x7f61b006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.037+0000 7f61b514c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b006dee0 con 0x7f61b00709e0 2023-10-08T05:53:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.037+0000 7f61b514c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b015d850 con 0x7f61b00ad790 2023-10-08T05:53:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.037+0000 7f61b514c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b015db30 con 0x7f61b0159ac0 2023-10-08T05:53:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.038+0000 7f61af59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f61b00709e0 0x7f61b00c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37068/0 (socket says 172.21.15.70:37068) 2023-10-08T05:53:09.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.038+0000 7f61af59e640 1 -- 172.21.15.70:0/233162354 learned_addr learned my addr 172.21.15.70:0/233162354 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.038+0000 7f61aed9d640 1 --2- 172.21.15.70:0/233162354 >> v2:172.21.15.181:3300/0 conn(0x7f61b0159ac0 0x7f61b006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.038+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108205218 0 0) 0x7f61b006dee0 con 0x7f61b00709e0 2023-10-08T05:53:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.038+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f619c003520 con 0x7f61b00709e0 2023-10-08T05:53:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3348354486 0 0) 0x7f619c003520 con 0x7f61b00709e0 2023-10-08T05:53:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 >> v2:172.21.15.181:3300/0 conn(0x7f61b0159ac0 msgr2=0x7f61b006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 --2- 172.21.15.70:0/233162354 >> v2:172.21.15.181:3300/0 conn(0x7f61b0159ac0 0x7f61b006d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 >> v1:172.21.15.181:6789/0 conn(0x7f61b00ad790 legacy=0x7f61b00702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b015de10 con 0x7f61b00709e0 2023-10-08T05:53:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f61a80019f0 con 0x7f61b00709e0 2023-10-08T05:53:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 >> v1:172.21.15.70:6789/0 conn(0x7f61b00709e0 legacy=0x7f61b00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f619c003970 0x7f619c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0047c0 0x7f619c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f619c005180 0x7f619c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b015db30 con 0x7f619c0047c0 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c00c020 con 0x7f619c005180 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c00c1e0 con 0x7f619c003970 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61a7fff640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0047c0 0x7f619c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.039+0000 7f61af59e640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f619c005180 0x7f619c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.040+0000 7f61aed9d640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f619c003970 0x7f619c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.040+0000 7f61a7fff640 1 -- 172.21.15.70:0/233162354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f619c003970 msgr2=0x7f619c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.040+0000 7f61a7fff640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f619c003970 0x7f619c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.040+0000 7f61a7fff640 1 -- 172.21.15.70:0/233162354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f619c005180 msgr2=0x7f619c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:09.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.040+0000 7f61a7fff640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f619c005180 0x7f619c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.040+0000 7f61a7fff640 1 -- 172.21.15.70:0/233162354 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c003520 con 0x7f619c0047c0 2023-10-08T05:53:09.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.040+0000 7f61af59e640 1 -- 172.21.15.70:0/233162354 reap_dead start 2023-10-08T05:53:09.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.041+0000 7f61a7fff640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0047c0 0x7f619c004c40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6198001750 tx=0x7f619c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d095c08aed232c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:09.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.041+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61980052d0 con 0x7f619c0047c0 2023-10-08T05:53:09.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.041+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619802e050 con 0x7f619c0047c0 2023-10-08T05:53:09.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.041+0000 7f61ae59c640 1 -- 172.21.15.70:0/233162354 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198003830 con 0x7f619c0047c0 2023-10-08T05:53:09.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.042+0000 7f61b514c640 1 -- 172.21.15.70:0/233162354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0047c0 msgr2=0x7f619c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.042+0000 7f61b514c640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0047c0 0x7f619c004c40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6198001750 tx=0x7f619c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.042+0000 7f61b514c640 1 -- 172.21.15.70:0/233162354 shutdown_connections 2023-10-08T05:53:09.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.042+0000 7f61b514c640 1 --2- 172.21.15.70:0/233162354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f619c0047c0 0x7f619c004c40 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.042+0000 7f61b514c640 1 -- 172.21.15.70:0/233162354 >> 172.21.15.70:0/233162354 conn(0x7f61b006a380 msgr2=0x7f61b00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:09.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.042+0000 7f61b514c640 1 -- 172.21.15.70:0/233162354 shutdown_connections 2023-10-08T05:53:09.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.042+0000 7f61b514c640 1 -- 172.21.15.70:0/233162354 wait complete. 2023-10-08T05:53:09.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.044+0000 7f61b514c640 1 Processor -- start 2023-10-08T05:53:09.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.044+0000 7f61b514c640 1 -- start start 2023-10-08T05:53:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.045+0000 7f61b514c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f619c0047c0 0x7f61b01f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.045+0000 7f61b514c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61b01f3830 0x7f61b0158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.045+0000 7f61b514c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0158970 0x7f61b00b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.045+0000 7f61b514c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0151440 con 0x7f61b01f3830 2023-10-08T05:53:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.045+0000 7f61b514c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0158e20 con 0x7f61b0158970 2023-10-08T05:53:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.045+0000 7f61b514c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b00b21c0 con 0x7f619c0047c0 2023-10-08T05:53:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.045+0000 7f61a7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61b01f3830 0x7f61b0158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.045+0000 7f61a7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61b01f3830 0x7f61b0158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46630/0 (socket says 172.21.15.70:46630) 2023-10-08T05:53:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.045+0000 7f61a7fff640 1 -- 172.21.15.70:0/2280346694 learned_addr learned my addr 172.21.15.70:0/2280346694 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.045+0000 7f61aed9d640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f619c0047c0 0x7f61b01f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61a7fff640 1 -- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f619c0047c0 msgr2=0x7f61b01f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61af59e640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0158970 0x7f61b00b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61a7fff640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f619c0047c0 0x7f61b01f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61a7fff640 1 -- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0158970 msgr2=0x7f61b00b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61a7fff640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0158970 0x7f61b00b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61a7fff640 1 -- 172.21.15.70:0/2280346694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b00b15b0 con 0x7f61b01f3830 2023-10-08T05:53:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61aed9d640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f619c0047c0 0x7f61b01f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61a7fff640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61b01f3830 0x7f61b0158430 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6198005040 tx=0x7f6198001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61a77fe640 1 -- 172.21.15.70:0/2280346694 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61980076c0 con 0x7f61b01f3830 2023-10-08T05:53:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61a77fe640 1 -- 172.21.15.70:0/2280346694 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6198007a30 con 0x7f61b01f3830 2023-10-08T05:53:09.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61b514c640 1 -- 172.21.15.70:0/2280346694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b00b1890 con 0x7f61b01f3830 2023-10-08T05:53:09.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61a77fe640 1 -- 172.21.15.70:0/2280346694 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198006be0 con 0x7f61b01f3830 2023-10-08T05:53:09.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.046+0000 7f61b514c640 1 -- 172.21.15.70:0/2280346694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b00b1e50 con 0x7f61b01f3830 2023-10-08T05:53:09.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.047+0000 7f61b514c640 1 -- 172.21.15.70:0/2280346694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b00b2330 con 0x7f61b01f3830 2023-10-08T05:53:09.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.048+0000 7f61a77fe640 1 -- 172.21.15.70:0/2280346694 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6198006d40 con 0x7f61b01f3830 2023-10-08T05:53:09.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.049+0000 7f61a77fe640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f619006af00 0x7f619006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.049+0000 7f61a77fe640 1 -- 172.21.15.70:0/2280346694 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f61980ab910 con 0x7f61b01f3830 2023-10-08T05:53:09.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.050+0000 7f61aed9d640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f619006af00 0x7f619006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.051+0000 7f61aed9d640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f619006af00 0x7f619006d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f61b004fa60 tx=0x7f61a0008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:09.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.054+0000 7f61a77fe640 1 -- 172.21.15.70:0/2280346694 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6198076760 con 0x7f61b01f3830 2023-10-08T05:53:09.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.292+0000 7f61b514c640 1 -- 172.21.15.70:0/2280346694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61b0067490 con 0x7f61b01f3830 2023-10-08T05:53:09.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.293+0000 7f61a77fe640 1 -- 172.21.15.70:0/2280346694 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f619804b090 con 0x7f61b01f3830 2023-10-08T05:53:09.295 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:09.295 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5384,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3148555214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3148555214},{"type":"v1","addr":"172.21.15.181:6837","nonce":3148555214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5896,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3669053115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3669053115},{"type":"v1","addr":"172.21.15.181:6836","nonce":3669053115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":7122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3775478637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3775478637},{"type":"v1","addr":"172.21.15.70:6833","nonce":3775478637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:05.225421+0000","modified":"2023-10-08T05:53:08.291318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7131},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7131":{"gid":7131,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/24744490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":24744490},{"type":"v1","addr":"172.21.15.70:6835","nonce":24744490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-08T05:53:09.295 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-10-08T05:53:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 -- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f619006af00 msgr2=0x7f619006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f619006af00 0x7f619006d3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f61b004fa60 tx=0x7f61a0008040 comp rx=0 tx=0).stop 2023-10-08T05:53:09.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 -- 172.21.15.70:0/2280346694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61b01f3830 msgr2=0x7f61b0158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61b01f3830 0x7f61b0158430 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6198005040 tx=0x7f6198001120 comp rx=0 tx=0).stop 2023-10-08T05:53:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 -- 172.21.15.70:0/2280346694 shutdown_connections 2023-10-08T05:53:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f619006af00 0x7f619006d3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61b0158970 0x7f61b00b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61b01f3830 0x7f61b0158430 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 --2- 172.21.15.70:0/2280346694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f619c0047c0 0x7f61b01f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 -- 172.21.15.70:0/2280346694 >> 172.21.15.70:0/2280346694 conn(0x7f61b006a380 msgr2=0x7f61b00c4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 -- 172.21.15.70:0/2280346694 shutdown_connections 2023-10-08T05:53:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.296+0000 7f61b514c640 1 -- 172.21.15.70:0/2280346694 wait complete. 2023-10-08T05:53:09.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:09.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.520+0000 7f3c764da640 1 Processor -- start 2023-10-08T05:53:09.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.520+0000 7f3c764da640 1 -- start start 2023-10-08T05:53:09.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.521+0000 7f3c764da640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3c700cbce0 0x7f3c70160e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.521+0000 7f3c764da640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c700ce550 con 0x7f3c700cb880 2023-10-08T05:53:09.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.521+0000 7f3c764da640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c701609d0 con 0x7f3c70069d50 2023-10-08T05:53:09.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.521+0000 7f3c764da640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c700cdcf0 con 0x7f3c700cbce0 2023-10-08T05:53:09.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.522+0000 7f3c6f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3c700cb880 0x7f3c700b1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37082/0 (socket says 172.21.15.70:37082) 2023-10-08T05:53:09.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.522+0000 7f3c6f7fe640 1 -- 172.21.15.70:0/3351044586 learned_addr learned my addr 172.21.15.70:0/3351044586 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:09.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.522+0000 7f3c74a77640 1 --2- 172.21.15.70:0/3351044586 >> v2:172.21.15.181:3300/0 conn(0x7f3c700cbce0 0x7f3c70160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.522+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176103398 0 0) 0x7f3c700ce550 con 0x7f3c700cb880 2023-10-08T05:53:09.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.522+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c5c003520 con 0x7f3c700cb880 2023-10-08T05:53:09.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3967499291 0 0) 0x7f3c5c003520 con 0x7f3c700cb880 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 >> v2:172.21.15.181:3300/0 conn(0x7f3c700cbce0 msgr2=0x7f3c70160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 --2- 172.21.15.70:0/3351044586 >> v2:172.21.15.181:3300/0 conn(0x7f3c700cbce0 0x7f3c70160e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 >> v1:172.21.15.181:6789/0 conn(0x7f3c70069d50 legacy=0x7f3c7006a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c700cdf90 con 0x7f3c700cb880 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3c58001670 con 0x7f3c700cb880 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 >> v1:172.21.15.70:6789/0 conn(0x7f3c700cb880 legacy=0x7f3c700b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c5c0039c0 0x7f3c5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c0047c0 0x7f3c5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c005180 0x7f3c5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c700cdcf0 con 0x7f3c5c005180 2023-10-08T05:53:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c00c080 con 0x7f3c5c0047c0 2023-10-08T05:53:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c00c240 con 0x7f3c5c0039c0 2023-10-08T05:53:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c74a77640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c005180 0x7f3c5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.523+0000 7f3c6f7fe640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c0047c0 0x7f3c5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.524+0000 7f3c74a77640 1 -- 172.21.15.70:0/3351044586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c5c0039c0 msgr2=0x7f3c5c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.524+0000 7f3c6ffff640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c5c0039c0 0x7f3c5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.524+0000 7f3c74a77640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c5c0039c0 0x7f3c5c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.524+0000 7f3c74a77640 1 -- 172.21.15.70:0/3351044586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c0047c0 msgr2=0x7f3c5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.524+0000 7f3c74a77640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c0047c0 0x7f3c5c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.524+0000 7f3c74a77640 1 -- 172.21.15.70:0/3351044586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c003520 con 0x7f3c5c005180 2023-10-08T05:53:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.524+0000 7f3c6f7fe640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c0047c0 0x7f3c5c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:09.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.525+0000 7f3c74a77640 1 -- 172.21.15.70:0/3351044586 reap_dead start 2023-10-08T05:53:09.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.525+0000 7f3c74a77640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c005180 0x7f3c5c00b980 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009fc0 tx=0x7f3c5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6124ca3de858fe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:09.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.525+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6802cc60 con 0x7f3c5c005180 2023-10-08T05:53:09.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.525+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6800f040 con 0x7f3c5c005180 2023-10-08T05:53:09.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.525+0000 7f3c6effd640 1 -- 172.21.15.70:0/3351044586 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c68003990 con 0x7f3c5c005180 2023-10-08T05:53:09.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.525+0000 7f3c764da640 1 -- 172.21.15.70:0/3351044586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c005180 msgr2=0x7f3c5c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.525+0000 7f3c764da640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c005180 0x7f3c5c00b980 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3c68009fc0 tx=0x7f3c5c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.526+0000 7f3c764da640 1 -- 172.21.15.70:0/3351044586 shutdown_connections 2023-10-08T05:53:09.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.526+0000 7f3c764da640 1 --2- 172.21.15.70:0/3351044586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c5c005180 0x7f3c5c00b980 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.526+0000 7f3c764da640 1 -- 172.21.15.70:0/3351044586 >> 172.21.15.70:0/3351044586 conn(0x7f3c7006d450 msgr2=0x7f3c7006d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:09.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.526+0000 7f3c764da640 1 -- 172.21.15.70:0/3351044586 shutdown_connections 2023-10-08T05:53:09.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.526+0000 7f3c764da640 1 -- 172.21.15.70:0/3351044586 wait complete. 2023-10-08T05:53:09.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.527+0000 7f3c764da640 1 Processor -- start 2023-10-08T05:53:09.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.528+0000 7f3c764da640 1 -- start start 2023-10-08T05:53:09.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c764da640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c005180 0x7f3c700c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c764da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c700b4c50 0x7f3c700b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c764da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c700b29f0 0x7f3c700b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c764da640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7015b0a0 con 0x7f3c700b4c50 2023-10-08T05:53:09.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c764da640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c700b50d0 con 0x7f3c5c005180 2023-10-08T05:53:09.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c764da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7015a420 con 0x7f3c700b29f0 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c6f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c700b4c50 0x7f3c700b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c6f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c700b4c50 0x7f3c700b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46644/0 (socket says 172.21.15.70:46644) 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c6f7fe640 1 -- 172.21.15.70:0/3399984359 learned_addr learned my addr 172.21.15.70:0/3399984359 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c6ffff640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c005180 0x7f3c700c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c74a77640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c700b29f0 0x7f3c700b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c6f7fe640 1 -- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c700b29f0 msgr2=0x7f3c700b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c6f7fe640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c700b29f0 0x7f3c700b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c6f7fe640 1 -- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c005180 msgr2=0x7f3c700c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c6f7fe640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c005180 0x7f3c700c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c6f7fe640 1 -- 172.21.15.70:0/3399984359 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7015a700 con 0x7f3c700b4c50 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.529+0000 7f3c6ffff640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c005180 0x7f3c700c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.530+0000 7f3c6f7fe640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c700b4c50 0x7f3c700b24b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800d7a0 tx=0x7f3c5800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:09.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.530+0000 7f3c6cff9640 1 -- 172.21.15.70:0/3399984359 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c580074e0 con 0x7f3c700b4c50 2023-10-08T05:53:09.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.530+0000 7f3c6cff9640 1 -- 172.21.15.70:0/3399984359 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58007640 con 0x7f3c700b4c50 2023-10-08T05:53:09.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.530+0000 7f3c764da640 1 -- 172.21.15.70:0/3399984359 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7015a9e0 con 0x7f3c700b4c50 2023-10-08T05:53:09.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.530+0000 7f3c6cff9640 1 -- 172.21.15.70:0/3399984359 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c580049e0 con 0x7f3c700b4c50 2023-10-08T05:53:09.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.530+0000 7f3c764da640 1 -- 172.21.15.70:0/3399984359 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7015a0d0 con 0x7f3c700b4c50 2023-10-08T05:53:09.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.531+0000 7f3c764da640 1 -- 172.21.15.70:0/3399984359 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c70067490 con 0x7f3c700b4c50 2023-10-08T05:53:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.532+0000 7f3c6cff9640 1 -- 172.21.15.70:0/3399984359 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3c58004b40 con 0x7f3c700b4c50 2023-10-08T05:53:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.534+0000 7f3c6cff9640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f3c5006af00 0x7f3c5006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.534+0000 7f3c6cff9640 1 -- 172.21.15.70:0/3399984359 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3c58089090 con 0x7f3c700b4c50 2023-10-08T05:53:09.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.535+0000 7f3c6ffff640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f3c5006af00 0x7f3c5006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.536+0000 7f3c6ffff640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f3c5006af00 0x7f3c5006d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3c60000c00 tx=0x7f3c60009040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.538+0000 7f3c6cff9640 1 -- 172.21.15.70:0/3399984359 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c58053f60 con 0x7f3c700b4c50 2023-10-08T05:53:09.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.732+0000 7f3c764da640 1 -- 172.21.15.70:0/3399984359 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c700b5810 con 0x7f3c700b4c50 2023-10-08T05:53:09.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.733+0000 7f3c6cff9640 1 -- 172.21.15.70:0/3399984359 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16846 (secure 0 0 0) 0x7f3c58019090 con 0x7f3c700b4c50 2023-10-08T05:53:09.735 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:09.735 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":104,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:06.778728+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:03.051784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:04.230588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 -- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f3c5006af00 msgr2=0x7f3c5006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f3c5006af00 0x7f3c5006d3c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3c60000c00 tx=0x7f3c60009040 comp rx=0 tx=0).stop 2023-10-08T05:53:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 -- 172.21.15.70:0/3399984359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c700b4c50 msgr2=0x7f3c700b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c700b4c50 0x7f3c700b24b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800d7a0 tx=0x7f3c5800dc60 comp rx=0 tx=0).stop 2023-10-08T05:53:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 -- 172.21.15.70:0/3399984359 shutdown_connections 2023-10-08T05:53:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f3c5006af00 0x7f3c5006d3c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3c700b29f0 0x7f3c700b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c700b4c50 0x7f3c700b24b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 --2- 172.21.15.70:0/3399984359 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3c5c005180 0x7f3c700c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 -- 172.21.15.70:0/3399984359 >> 172.21.15.70:0/3399984359 conn(0x7f3c7006d450 msgr2=0x7f3c70151ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 -- 172.21.15.70:0/3399984359 shutdown_connections 2023-10-08T05:53:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.736+0000 7f3c764da640 1 -- 172.21.15.70:0/3399984359 wait complete. 2023-10-08T05:53:09.757 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T05:53:09.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:53:09.820 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:09.817+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:09.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:53:09.848 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:53:09.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.969+0000 7fce4a221640 1 Processor -- start 2023-10-08T05:53:09.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.969+0000 7fce4a221640 1 -- start start 2023-10-08T05:53:09.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.971+0000 7fce4a221640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fce440ad790 0x7fce440702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.971+0000 7fce4a221640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4415e330 0x7fce44070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.971+0000 7fce4a221640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4406dee0 con 0x7fce44159ac0 2023-10-08T05:53:09.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.971+0000 7fce4a221640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4406dcf0 con 0x7fce440ad790 2023-10-08T05:53:09.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.971+0000 7fce4a221640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415d850 con 0x7fce4415e330 2023-10-08T05:53:09.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.972+0000 7fce437fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fce44159ac0 0x7fce4406d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37096/0 (socket says 172.21.15.70:37096) 2023-10-08T05:53:09.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.972+0000 7fce437fe640 1 -- 172.21.15.70:0/1718199546 learned_addr learned my addr 172.21.15.70:0/1718199546 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:09.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.972+0000 7fce43fff640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4415e330 0x7fce44070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.972+0000 7fce3bfff640 1 --2- 172.21.15.70:0/1718199546 >> v2:172.21.15.181:3300/0 conn(0x7fce440ad790 0x7fce440702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.972+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866241024 0 0) 0x7fce4406dee0 con 0x7fce44159ac0 2023-10-08T05:53:09.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.972+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce30005730 con 0x7fce44159ac0 2023-10-08T05:53:09.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.972+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2752744477 0 0) 0x7fce30005730 con 0x7fce44159ac0 2023-10-08T05:53:09.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.972+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4415e330 msgr2=0x7fce44070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4415e330 0x7fce44070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 >> v2:172.21.15.181:3300/0 conn(0x7fce440ad790 msgr2=0x7fce440702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 --2- 172.21.15.70:0/1718199546 >> v2:172.21.15.181:3300/0 conn(0x7fce440ad790 0x7fce440702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4415db30 con 0x7fce44159ac0 2023-10-08T05:53:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fce34001670 con 0x7fce44159ac0 2023-10-08T05:53:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 >> v1:172.21.15.70:6789/0 conn(0x7fce44159ac0 legacy=0x7fce4406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30005b80 0x7fce30005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4415e330 0x7fce300069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 0x7fce30006f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415d850 con 0x7fce300064e0 2023-10-08T05:53:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3000bde0 con 0x7fce30005b80 2023-10-08T05:53:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce30007480 con 0x7fce4415e330 2023-10-08T05:53:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce43fff640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 0x7fce30006f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce3bfff640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4415e330 0x7fce300069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce437fe640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30005b80 0x7fce30005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce43fff640 1 -- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4415e330 msgr2=0x7fce300069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce43fff640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4415e330 0x7fce300069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce43fff640 1 -- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30005b80 msgr2=0x7fce30005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce43fff640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30005b80 0x7fce30005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce43fff640 1 -- 172.21.15.70:0/1718199546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce30005730 con 0x7fce300064e0 2023-10-08T05:53:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.973+0000 7fce437fe640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce30005b80 0x7fce30005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.975+0000 7fce43fff640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 0x7fce30006f10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009fc0 tx=0x7fce30007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6a5923785eb260 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.975+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c02cc20 con 0x7fce300064e0 2023-10-08T05:53:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.975+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c00f040 con 0x7fce300064e0 2023-10-08T05:53:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.975+0000 7fce42ffd640 1 -- 172.21.15.70:0/1718199546 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c034440 con 0x7fce300064e0 2023-10-08T05:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.975+0000 7fce4a221640 1 -- 172.21.15.70:0/1718199546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 msgr2=0x7fce30006f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.975+0000 7fce4a221640 1 --2- 172.21.15.70:0/1718199546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 0x7fce30006f10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009fc0 tx=0x7fce30007a00 comp rx=0 tx=0).stop 2023-10-08T05:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.975+0000 7fce43fff640 1 -- 172.21.15.70:0/1718199546 reap_dead start 2023-10-08T05:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.976+0000 7fce4a221640 1 -- 172.21.15.70:0/1718199546 shutdown_connections 2023-10-08T05:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.976+0000 7fce4a221640 1 -- 172.21.15.70:0/1718199546 >> 172.21.15.70:0/1718199546 conn(0x7fce4406a380 msgr2=0x7fce440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.976+0000 7fce4a221640 1 -- 172.21.15.70:0/1718199546 shutdown_connections 2023-10-08T05:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.976+0000 7fce4a221640 1 -- 172.21.15.70:0/1718199546 wait complete. 2023-10-08T05:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.977+0000 7fce4a221640 1 Processor -- start 2023-10-08T05:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.978+0000 7fce4a221640 1 -- start start 2023-10-08T05:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.978+0000 7fce4a221640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 0x7fce44155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce4a221640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4415a020 0x7fce4415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce4a221640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4413de20 0x7fce4413e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce4a221640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415aac0 con 0x7fce300064e0 2023-10-08T05:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce4a221640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415bf10 con 0x7fce4415a020 2023-10-08T05:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce4a221640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415c1f0 con 0x7fce4413de20 2023-10-08T05:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce437fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 0x7fce44155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce437fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 0x7fce44155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46656/0 (socket says 172.21.15.70:46656) 2023-10-08T05:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce437fe640 1 -- 172.21.15.70:0/2087816481 learned_addr learned my addr 172.21.15.70:0/2087816481 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce43fff640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4413de20 0x7fce4413e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce3bfff640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4415a020 0x7fce4415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce437fe640 1 -- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4413de20 msgr2=0x7fce4413e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce437fe640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4413de20 0x7fce4413e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce437fe640 1 -- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4415a020 msgr2=0x7fce4415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce437fe640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4415a020 0x7fce4415a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce437fe640 1 -- 172.21.15.70:0/2087816481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4415c4d0 con 0x7fce300064e0 2023-10-08T05:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.979+0000 7fce3bfff640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4415a020 0x7fce4415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.980+0000 7fce437fe640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 0x7fce44155390 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fce3400d890 tx=0x7fce3400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.980+0000 7fce40ff9640 1 -- 172.21.15.70:0/2087816481 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34007560 con 0x7fce300064e0 2023-10-08T05:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.980+0000 7fce40ff9640 1 -- 172.21.15.70:0/2087816481 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce340076c0 con 0x7fce300064e0 2023-10-08T05:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.980+0000 7fce40ff9640 1 -- 172.21.15.70:0/2087816481 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce340049e0 con 0x7fce300064e0 2023-10-08T05:53:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.980+0000 7fce4a221640 1 -- 172.21.15.70:0/2087816481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4415c7b0 con 0x7fce300064e0 2023-10-08T05:53:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.980+0000 7fce4a221640 1 -- 172.21.15.70:0/2087816481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4414f6c0 con 0x7fce300064e0 2023-10-08T05:53:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.981+0000 7fce4a221640 1 -- 172.21.15.70:0/2087816481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce44067490 con 0x7fce300064e0 2023-10-08T05:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.982+0000 7fce40ff9640 1 -- 172.21.15.70:0/2087816481 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fce34004b40 con 0x7fce300064e0 2023-10-08T05:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.983+0000 7fce40ff9640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fce2406aed0 0x7fce2406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.983+0000 7fce40ff9640 1 -- 172.21.15.70:0/2087816481 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fce340893d0 con 0x7fce300064e0 2023-10-08T05:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.984+0000 7fce3bfff640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fce2406aed0 0x7fce2406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.985+0000 7fce3bfff640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fce2406aed0 0x7fce2406d390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce44155ed0 tx=0x7fce2c008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:09.987+0000 7fce40ff9640 1 -- 172.21.15.70:0/2087816481 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce34054260 con 0x7fce300064e0 2023-10-08T05:53:10.144 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-10-08T05:53:10.143+0000 7fa8d696f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:53:10.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.179+0000 7fce4a221640 1 -- 172.21.15.70:0/2087816481 --> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fce4415c360 con 0x7fce2406aed0 2023-10-08T05:53:10.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.207+0000 7fce40ff9640 1 -- 172.21.15.70:0/2087816481 <== mgr.5737 v2:172.21.15.181:6832/2060337708 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286930 (secure 0 0 0) 0x7fce4415c360 con 0x7fce2406aed0 2023-10-08T05:53:10.210 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:10.214 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:53:10.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.214+0000 7fce4a221640 1 -- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fce2406aed0 msgr2=0x7fce2406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:10.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.214+0000 7fce4a221640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fce2406aed0 0x7fce2406d390 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fce44155ed0 tx=0x7fce2c008040 comp rx=0 tx=0).stop 2023-10-08T05:53:10.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.214+0000 7fce4a221640 1 -- 172.21.15.70:0/2087816481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 msgr2=0x7fce44155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:10.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.214+0000 7fce4a221640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 0x7fce44155390 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fce3400d890 tx=0x7fce3400dd50 comp rx=0 tx=0).stop 2023-10-08T05:53:10.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.215+0000 7fce4a221640 1 -- 172.21.15.70:0/2087816481 shutdown_connections 2023-10-08T05:53:10.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.215+0000 7fce4a221640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fce2406aed0 0x7fce2406d390 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.215+0000 7fce4a221640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce4413de20 0x7fce4413e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.215+0000 7fce4a221640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce4415a020 0x7fce4415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.215+0000 7fce4a221640 1 --2- 172.21.15.70:0/2087816481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce300064e0 0x7fce44155390 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.215+0000 7fce4a221640 1 -- 172.21.15.70:0/2087816481 >> 172.21.15.70:0/2087816481 conn(0x7fce4406a380 msgr2=0x7fce441586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:10.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.215+0000 7fce4a221640 1 -- 172.21.15.70:0/2087816481 shutdown_connections 2023-10-08T05:53:10.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.215+0000 7fce4a221640 1 -- 172.21.15.70:0/2087816481 wait complete. 2023-10-08T05:53:10.235 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-08T05:53:08.608958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":550,"num_read_kb":1032,"num_write":289,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2196,"ondisk_log_size":2196,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":313316,"kb_used_data":11128,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754661404,"statfs":{"total":773094113280,"available":772773277696,"internally_reserved":0,"allocated":11395072,"data_stored":6313652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":309317179},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.873943"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803698+0000","last_change":"2023-10-08T05:53:05.804151+0000","last_active":"2023-10-08T05:53:06.803698+0000","last_peered":"2023-10-08T05:53:06.803698+0000","last_clean":"2023-10-08T05:53:06.803698+0000","last_became_active":"2023-10-08T05:53:05.803842+0000","last_became_peered":"2023-10-08T05:53:05.803842+0000","last_unstale":"2023-10-08T05:53:06.803698+0000","last_undegraded":"2023-10-08T05:53:06.803698+0000","last_fullsized":"2023-10-08T05:53:06.803698+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:23.414780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793874+0000","last_change":"2023-10-08T05:53:05.796317+0000","last_active":"2023-10-08T05:53:06.793874+0000","last_peered":"2023-10-08T05:53:06.793874+0000","last_clean":"2023-10-08T05:53:06.793874+0000","last_became_active":"2023-10-08T05:53:05.796020+0000","last_became_peered":"2023-10-08T05:53:05.796020+0000","last_unstale":"2023-10-08T05:53:06.793874+0000","last_undegraded":"2023-10-08T05:53:06.793874+0000","last_fullsized":"2023-10-08T05:53:06.793874+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:33.604099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.273901+0000","last_change":"2023-10-08T05:53:05.798782+0000","last_active":"2023-10-08T05:53:07.273901+0000","last_peered":"2023-10-08T05:53:07.273901+0000","last_clean":"2023-10-08T05:53:07.273901+0000","last_became_active":"2023-10-08T05:53:05.798516+0000","last_became_peered":"2023-10-08T05:53:05.798516+0000","last_unstale":"2023-10-08T05:53:07.273901+0000","last_undegraded":"2023-10-08T05:53:07.273901+0000","last_fullsized":"2023-10-08T05:53:07.273901+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:17.434212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271380+0000","last_change":"2023-10-08T05:53:05.798713+0000","last_active":"2023-10-08T05:53:07.271380+0000","last_peered":"2023-10-08T05:53:07.271380+0000","last_clean":"2023-10-08T05:53:07.271380+0000","last_became_active":"2023-10-08T05:53:05.798445+0000","last_became_peered":"2023-10-08T05:53:05.798445+0000","last_unstale":"2023-10-08T05:53:07.271380+0000","last_undegraded":"2023-10-08T05:53:07.271380+0000","last_fullsized":"2023-10-08T05:53:07.271380+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:02.485242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271579+0000","last_change":"2023-10-08T05:53:05.803570+0000","last_active":"2023-10-08T05:53:07.271579+0000","last_peered":"2023-10-08T05:53:07.271579+0000","last_clean":"2023-10-08T05:53:07.271579+0000","last_became_active":"2023-10-08T05:53:05.802131+0000","last_became_peered":"2023-10-08T05:53:05.802131+0000","last_unstale":"2023-10-08T05:53:07.271579+0000","last_undegraded":"2023-10-08T05:53:07.271579+0000","last_fullsized":"2023-10-08T05:53:07.271579+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:47:41.201385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.273334+0000","last_change":"2023-10-08T05:53:05.804643+0000","last_active":"2023-10-08T05:53:07.273334+0000","last_peered":"2023-10-08T05:53:07.273334+0000","last_clean":"2023-10-08T05:53:07.273334+0000","last_became_active":"2023-10-08T05:53:05.804353+0000","last_became_peered":"2023-10-08T05:53:05.804353+0000","last_unstale":"2023-10-08T05:53:07.273334+0000","last_undegraded":"2023-10-08T05:53:07.273334+0000","last_fullsized":"2023-10-08T05:53:07.273334+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:48.735927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.799840+0000","last_change":"2023-10-08T05:53:05.790851+0000","last_active":"2023-10-08T05:53:06.799840+0000","last_peered":"2023-10-08T05:53:06.799840+0000","last_clean":"2023-10-08T05:53:06.799840+0000","last_became_active":"2023-10-08T05:53:05.790473+0000","last_became_peered":"2023-10-08T05:53:05.790473+0000","last_unstale":"2023-10-08T05:53:06.799840+0000","last_undegraded":"2023-10-08T05:53:06.799840+0000","last_fullsized":"2023-10-08T05:53:06.799840+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:42:52.376261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271097+0000","last_change":"2023-10-08T05:53:05.796060+0000","last_active":"2023-10-08T05:53:07.271097+0000","last_peered":"2023-10-08T05:53:07.271097+0000","last_clean":"2023-10-08T05:53:07.271097+0000","last_became_active":"2023-10-08T05:53:05.795791+0000","last_became_peered":"2023-10-08T05:53:05.795791+0000","last_unstale":"2023-10-08T05:53:07.271097+0000","last_undegraded":"2023-10-08T05:53:07.271097+0000","last_fullsized":"2023-10-08T05:53:07.271097+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:53.249320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800993+0000","last_change":"2023-10-08T05:53:05.791575+0000","last_active":"2023-10-08T05:53:06.800993+0000","last_peered":"2023-10-08T05:53:06.800993+0000","last_clean":"2023-10-08T05:53:06.800993+0000","last_became_active":"2023-10-08T05:53:05.790545+0000","last_became_peered":"2023-10-08T05:53:05.790545+0000","last_unstale":"2023-10-08T05:53:06.800993+0000","last_undegraded":"2023-10-08T05:53:06.800993+0000","last_fullsized":"2023-10-08T05:53:06.800993+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:16.125323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271699+0000","last_change":"2023-10-08T05:53:05.802819+0000","last_active":"2023-10-08T05:53:07.271699+0000","last_peered":"2023-10-08T05:53:07.271699+0000","last_clean":"2023-10-08T05:53:07.271699+0000","last_became_active":"2023-10-08T05:53:05.802521+0000","last_became_peered":"2023-10-08T05:53:05.802521+0000","last_unstale":"2023-10-08T05:53:07.271699+0000","last_undegraded":"2023-10-08T05:53:07.271699+0000","last_fullsized":"2023-10-08T05:53:07.271699+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:31.090058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793753+0000","last_change":"2023-10-08T05:53:05.802108+0000","last_active":"2023-10-08T05:53:06.793753+0000","last_peered":"2023-10-08T05:53:06.793753+0000","last_clean":"2023-10-08T05:53:06.793753+0000","last_became_active":"2023-10-08T05:53:05.801691+0000","last_became_peered":"2023-10-08T05:53:05.801691+0000","last_unstale":"2023-10-08T05:53:06.793753+0000","last_undegraded":"2023-10-08T05:53:06.793753+0000","last_fullsized":"2023-10-08T05:53:06.793753+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:18.113815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271466+0000","last_change":"2023-10-08T05:53:05.796469+0000","last_active":"2023-10-08T05:53:07.271466+0000","last_peered":"2023-10-08T05:53:07.271466+0000","last_clean":"2023-10-08T05:53:07.271466+0000","last_became_active":"2023-10-08T05:53:05.796129+0000","last_became_peered":"2023-10-08T05:53:05.796129+0000","last_unstale":"2023-10-08T05:53:07.271466+0000","last_undegraded":"2023-10-08T05:53:07.271466+0000","last_fullsized":"2023-10-08T05:53:07.271466+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:23.414780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271925+0000","last_change":"2023-10-08T05:53:05.809944+0000","last_active":"2023-10-08T05:53:07.271925+0000","last_peered":"2023-10-08T05:53:07.271925+0000","last_clean":"2023-10-08T05:53:07.271925+0000","last_became_active":"2023-10-08T05:53:05.809675+0000","last_became_peered":"2023-10-08T05:53:05.809675+0000","last_unstale":"2023-10-08T05:53:07.271925+0000","last_undegraded":"2023-10-08T05:53:07.271925+0000","last_fullsized":"2023-10-08T05:53:07.271925+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:56.522301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.273718+0000","last_change":"2023-10-08T05:53:05.804069+0000","last_active":"2023-10-08T05:53:07.273718+0000","last_peered":"2023-10-08T05:53:07.273718+0000","last_clean":"2023-10-08T05:53:07.273718+0000","last_became_active":"2023-10-08T05:53:05.800052+0000","last_became_peered":"2023-10-08T05:53:05.800052+0000","last_unstale":"2023-10-08T05:53:07.273718+0000","last_undegraded":"2023-10-08T05:53:07.273718+0000","last_fullsized":"2023-10-08T05:53:07.273718+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:14.797057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793235+0000","last_change":"2023-10-08T05:53:05.790331+0000","last_active":"2023-10-08T05:53:06.793235+0000","last_peered":"2023-10-08T05:53:06.793235+0000","last_clean":"2023-10-08T05:53:06.793235+0000","last_became_active":"2023-10-08T05:53:05.789747+0000","last_became_peered":"2023-10-08T05:53:05.789747+0000","last_unstale":"2023-10-08T05:53:06.793235+0000","last_undegraded":"2023-10-08T05:53:06.793235+0000","last_fullsized":"2023-10-08T05:53:06.793235+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:32.287095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802442+0000","last_change":"2023-10-08T05:53:05.805853+0000","last_active":"2023-10-08T05:53:06.802442+0000","last_peered":"2023-10-08T05:53:06.802442+0000","last_clean":"2023-10-08T05:53:06.802442+0000","last_became_active":"2023-10-08T05:53:05.805566+0000","last_became_peered":"2023-10-08T05:53:05.805566+0000","last_unstale":"2023-10-08T05:53:06.802442+0000","last_undegraded":"2023-10-08T05:53:06.802442+0000","last_fullsized":"2023-10-08T05:53:06.802442+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:14.811048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.275815+0000","last_change":"2023-10-08T05:53:05.795160+0000","last_active":"2023-10-08T05:53:07.275815+0000","last_peered":"2023-10-08T05:53:07.275815+0000","last_clean":"2023-10-08T05:53:07.275815+0000","last_became_active":"2023-10-08T05:53:05.794693+0000","last_became_peered":"2023-10-08T05:53:05.794693+0000","last_unstale":"2023-10-08T05:53:07.275815+0000","last_undegraded":"2023-10-08T05:53:07.275815+0000","last_fullsized":"2023-10-08T05:53:07.275815+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:11:50.876086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274412+0000","last_change":"2023-10-08T05:53:05.794406+0000","last_active":"2023-10-08T05:53:07.274412+0000","last_peered":"2023-10-08T05:53:07.274412+0000","last_clean":"2023-10-08T05:53:07.274412+0000","last_became_active":"2023-10-08T05:53:05.794135+0000","last_became_peered":"2023-10-08T05:53:05.794135+0000","last_unstale":"2023-10-08T05:53:07.274412+0000","last_undegraded":"2023-10-08T05:53:07.274412+0000","last_fullsized":"2023-10-08T05:53:07.274412+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:44.413602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.273588+0000","last_change":"2023-10-08T05:53:05.804227+0000","last_active":"2023-10-08T05:53:07.273588+0000","last_peered":"2023-10-08T05:53:07.273588+0000","last_clean":"2023-10-08T05:53:07.273588+0000","last_became_active":"2023-10-08T05:53:05.803951+0000","last_became_peered":"2023-10-08T05:53:05.803951+0000","last_unstale":"2023-10-08T05:53:07.273588+0000","last_undegraded":"2023-10-08T05:53:07.273588+0000","last_fullsized":"2023-10-08T05:53:07.273588+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:31.090058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272630+0000","last_change":"2023-10-08T05:53:05.791220+0000","last_active":"2023-10-08T05:53:07.272630+0000","last_peered":"2023-10-08T05:53:07.272630+0000","last_clean":"2023-10-08T05:53:07.272630+0000","last_became_active":"2023-10-08T05:53:05.789390+0000","last_became_peered":"2023-10-08T05:53:05.789390+0000","last_unstale":"2023-10-08T05:53:07.272630+0000","last_undegraded":"2023-10-08T05:53:07.272630+0000","last_fullsized":"2023-10-08T05:53:07.272630+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:42.467508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.799329+0000","last_change":"2023-10-08T05:53:05.801975+0000","last_active":"2023-10-08T05:53:06.799329+0000","last_peered":"2023-10-08T05:53:06.799329+0000","last_clean":"2023-10-08T05:53:06.799329+0000","last_became_active":"2023-10-08T05:53:05.801648+0000","last_became_peered":"2023-10-08T05:53:05.801648+0000","last_unstale":"2023-10-08T05:53:06.799329+0000","last_undegraded":"2023-10-08T05:53:06.799329+0000","last_fullsized":"2023-10-08T05:53:06.799329+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:29:08.807994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271775+0000","last_change":"2023-10-08T05:53:05.810741+0000","last_active":"2023-10-08T05:53:07.271775+0000","last_peered":"2023-10-08T05:53:07.271775+0000","last_clean":"2023-10-08T05:53:07.271775+0000","last_became_active":"2023-10-08T05:53:05.810403+0000","last_became_peered":"2023-10-08T05:53:05.810403+0000","last_unstale":"2023-10-08T05:53:07.271775+0000","last_undegraded":"2023-10-08T05:53:07.271775+0000","last_fullsized":"2023-10-08T05:53:07.271775+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:40.445070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800184+0000","last_change":"2023-10-08T05:53:05.793504+0000","last_active":"2023-10-08T05:53:06.800184+0000","last_peered":"2023-10-08T05:53:06.800184+0000","last_clean":"2023-10-08T05:53:06.800184+0000","last_became_active":"2023-10-08T05:53:05.793223+0000","last_became_peered":"2023-10-08T05:53:05.793223+0000","last_unstale":"2023-10-08T05:53:06.800184+0000","last_undegraded":"2023-10-08T05:53:06.800184+0000","last_fullsized":"2023-10-08T05:53:06.800184+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:49.249431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793188+0000","last_change":"2023-10-08T05:53:05.796449+0000","last_active":"2023-10-08T05:53:06.793188+0000","last_peered":"2023-10-08T05:53:06.793188+0000","last_clean":"2023-10-08T05:53:06.793188+0000","last_became_active":"2023-10-08T05:53:05.796144+0000","last_became_peered":"2023-10-08T05:53:05.796144+0000","last_unstale":"2023-10-08T05:53:06.793188+0000","last_undegraded":"2023-10-08T05:53:06.793188+0000","last_fullsized":"2023-10-08T05:53:06.793188+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.108691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793809+0000","last_change":"2023-10-08T05:53:05.788124+0000","last_active":"2023-10-08T05:53:06.793809+0000","last_peered":"2023-10-08T05:53:06.793809+0000","last_clean":"2023-10-08T05:53:06.793809+0000","last_became_active":"2023-10-08T05:53:05.787390+0000","last_became_peered":"2023-10-08T05:53:05.787390+0000","last_unstale":"2023-10-08T05:53:06.793809+0000","last_undegraded":"2023-10-08T05:53:06.793809+0000","last_fullsized":"2023-10-08T05:53:06.793809+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:28:33.332486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271579+0000","last_change":"2023-10-08T05:53:05.802939+0000","last_active":"2023-10-08T05:53:07.271579+0000","last_peered":"2023-10-08T05:53:07.271579+0000","last_clean":"2023-10-08T05:53:07.271579+0000","last_became_active":"2023-10-08T05:53:05.802651+0000","last_became_peered":"2023-10-08T05:53:05.802651+0000","last_unstale":"2023-10-08T05:53:07.271579+0000","last_undegraded":"2023-10-08T05:53:07.271579+0000","last_fullsized":"2023-10-08T05:53:07.271579+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:48.735927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803151+0000","last_change":"2023-10-08T05:53:05.807790+0000","last_active":"2023-10-08T05:53:06.803151+0000","last_peered":"2023-10-08T05:53:06.803151+0000","last_clean":"2023-10-08T05:53:06.803151+0000","last_became_active":"2023-10-08T05:53:05.807502+0000","last_became_peered":"2023-10-08T05:53:05.807502+0000","last_unstale":"2023-10-08T05:53:06.803151+0000","last_undegraded":"2023-10-08T05:53:06.803151+0000","last_fullsized":"2023-10-08T05:53:06.803151+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:35.349762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272953+0000","last_change":"2023-10-08T05:53:05.793705+0000","last_active":"2023-10-08T05:53:07.272953+0000","last_peered":"2023-10-08T05:53:07.272953+0000","last_clean":"2023-10-08T05:53:07.272953+0000","last_became_active":"2023-10-08T05:53:05.793355+0000","last_became_peered":"2023-10-08T05:53:05.793355+0000","last_unstale":"2023-10-08T05:53:07.272953+0000","last_undegraded":"2023-10-08T05:53:07.272953+0000","last_fullsized":"2023-10-08T05:53:07.272953+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:39.356767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793493+0000","last_change":"2023-10-08T05:53:05.784136+0000","last_active":"2023-10-08T05:53:06.793493+0000","last_peered":"2023-10-08T05:53:06.793493+0000","last_clean":"2023-10-08T05:53:06.793493+0000","last_became_active":"2023-10-08T05:53:05.783763+0000","last_became_peered":"2023-10-08T05:53:05.783763+0000","last_unstale":"2023-10-08T05:53:06.793493+0000","last_undegraded":"2023-10-08T05:53:06.793493+0000","last_fullsized":"2023-10-08T05:53:06.793493+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:06:23.168404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800210+0000","last_change":"2023-10-08T05:53:05.813374+0000","last_active":"2023-10-08T05:53:06.800210+0000","last_peered":"2023-10-08T05:53:06.800210+0000","last_clean":"2023-10-08T05:53:06.800210+0000","last_became_active":"2023-10-08T05:53:05.813074+0000","last_became_peered":"2023-10-08T05:53:05.813074+0000","last_unstale":"2023-10-08T05:53:06.800210+0000","last_undegraded":"2023-10-08T05:53:06.800210+0000","last_fullsized":"2023-10-08T05:53:06.800210+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:42:45.243485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272482+0000","last_change":"2023-10-08T05:53:05.785403+0000","last_active":"2023-10-08T05:53:07.272482+0000","last_peered":"2023-10-08T05:53:07.272482+0000","last_clean":"2023-10-08T05:53:07.272482+0000","last_became_active":"2023-10-08T05:53:05.785109+0000","last_became_peered":"2023-10-08T05:53:05.785109+0000","last_unstale":"2023-10-08T05:53:07.272482+0000","last_undegraded":"2023-10-08T05:53:07.272482+0000","last_fullsized":"2023-10-08T05:53:07.272482+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:15.369645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.801135+0000","last_change":"2023-10-08T05:53:05.805691+0000","last_active":"2023-10-08T05:53:06.801135+0000","last_peered":"2023-10-08T05:53:06.801135+0000","last_clean":"2023-10-08T05:53:06.801135+0000","last_became_active":"2023-10-08T05:53:05.805385+0000","last_became_peered":"2023-10-08T05:53:05.805385+0000","last_unstale":"2023-10-08T05:53:06.801135+0000","last_undegraded":"2023-10-08T05:53:06.801135+0000","last_fullsized":"2023-10-08T05:53:06.801135+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:44.044510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271679+0000","last_change":"2023-10-08T05:53:05.792769+0000","last_active":"2023-10-08T05:53:07.271679+0000","last_peered":"2023-10-08T05:53:07.271679+0000","last_clean":"2023-10-08T05:53:07.271679+0000","last_became_active":"2023-10-08T05:53:05.792496+0000","last_became_peered":"2023-10-08T05:53:05.792496+0000","last_unstale":"2023-10-08T05:53:07.271679+0000","last_undegraded":"2023-10-08T05:53:07.271679+0000","last_fullsized":"2023-10-08T05:53:07.271679+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:56.938089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"103'4","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.275279+0000","last_change":"2023-10-08T05:53:04.770249+0000","last_active":"2023-10-08T05:53:07.275279+0000","last_peered":"2023-10-08T05:53:07.275279+0000","last_clean":"2023-10-08T05:53:07.275279+0000","last_became_active":"2023-10-08T05:53:04.769944+0000","last_became_peered":"2023-10-08T05:53:04.769944+0000","last_unstale":"2023-10-08T05:53:07.275279+0000","last_undegraded":"2023-10-08T05:53:07.275279+0000","last_fullsized":"2023-10-08T05:53:07.275279+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:33.897876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793397+0000","last_change":"2023-10-08T05:53:05.796757+0000","last_active":"2023-10-08T05:53:06.793397+0000","last_peered":"2023-10-08T05:53:06.793397+0000","last_clean":"2023-10-08T05:53:06.793397+0000","last_became_active":"2023-10-08T05:53:05.796399+0000","last_became_peered":"2023-10-08T05:53:05.796399+0000","last_unstale":"2023-10-08T05:53:06.793397+0000","last_undegraded":"2023-10-08T05:53:06.793397+0000","last_fullsized":"2023-10-08T05:53:06.793397+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:54.833969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"103'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802834+0000","last_change":"2023-10-08T05:53:04.775265+0000","last_active":"2023-10-08T05:53:06.802834+0000","last_peered":"2023-10-08T05:53:06.802834+0000","last_clean":"2023-10-08T05:53:06.802834+0000","last_became_active":"2023-10-08T05:53:04.774915+0000","last_became_peered":"2023-10-08T05:53:04.774915+0000","last_unstale":"2023-10-08T05:53:06.802834+0000","last_undegraded":"2023-10-08T05:53:06.802834+0000","last_fullsized":"2023-10-08T05:53:06.802834+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:45.638271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800983+0000","last_change":"2023-10-08T05:53:05.792336+0000","last_active":"2023-10-08T05:53:06.800983+0000","last_peered":"2023-10-08T05:53:06.800983+0000","last_clean":"2023-10-08T05:53:06.800983+0000","last_became_active":"2023-10-08T05:53:05.791137+0000","last_became_peered":"2023-10-08T05:53:05.791137+0000","last_unstale":"2023-10-08T05:53:06.800983+0000","last_undegraded":"2023-10-08T05:53:06.800983+0000","last_fullsized":"2023-10-08T05:53:06.800983+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:49.217386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803112+0000","last_change":"2023-10-08T05:53:04.776791+0000","last_active":"2023-10-08T05:53:06.803112+0000","last_peered":"2023-10-08T05:53:06.803112+0000","last_clean":"2023-10-08T05:53:06.803112+0000","last_became_active":"2023-10-08T05:53:04.776525+0000","last_became_peered":"2023-10-08T05:53:04.776525+0000","last_unstale":"2023-10-08T05:53:06.803112+0000","last_undegraded":"2023-10-08T05:53:06.803112+0000","last_fullsized":"2023-10-08T05:53:06.803112+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:02:42.286157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802668+0000","last_change":"2023-10-08T05:53:05.801465+0000","last_active":"2023-10-08T05:53:06.802668+0000","last_peered":"2023-10-08T05:53:06.802668+0000","last_clean":"2023-10-08T05:53:06.802668+0000","last_became_active":"2023-10-08T05:53:05.801162+0000","last_became_peered":"2023-10-08T05:53:05.801162+0000","last_unstale":"2023-10-08T05:53:06.802668+0000","last_undegraded":"2023-10-08T05:53:06.802668+0000","last_fullsized":"2023-10-08T05:53:06.802668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:38:59.232443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.799922+0000","last_change":"2023-10-08T05:53:04.767204+0000","last_active":"2023-10-08T05:53:06.799922+0000","last_peered":"2023-10-08T05:53:06.799922+0000","last_clean":"2023-10-08T05:53:06.799922+0000","last_became_active":"2023-10-08T05:53:04.766876+0000","last_became_peered":"2023-10-08T05:53:04.766876+0000","last_unstale":"2023-10-08T05:53:06.799922+0000","last_undegraded":"2023-10-08T05:53:06.799922+0000","last_fullsized":"2023-10-08T05:53:06.799922+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:17:46.822177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793833+0000","last_change":"2023-10-08T05:53:05.792920+0000","last_active":"2023-10-08T05:53:06.793833+0000","last_peered":"2023-10-08T05:53:06.793833+0000","last_clean":"2023-10-08T05:53:06.793833+0000","last_became_active":"2023-10-08T05:53:05.792647+0000","last_became_peered":"2023-10-08T05:53:05.792647+0000","last_unstale":"2023-10-08T05:53:06.793833+0000","last_undegraded":"2023-10-08T05:53:06.793833+0000","last_fullsized":"2023-10-08T05:53:06.793833+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:13:56.109947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"103'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802741+0000","last_change":"2023-10-08T05:53:04.767204+0000","last_active":"2023-10-08T05:53:06.802741+0000","last_peered":"2023-10-08T05:53:06.802741+0000","last_clean":"2023-10-08T05:53:06.802741+0000","last_became_active":"2023-10-08T05:53:04.766888+0000","last_became_peered":"2023-10-08T05:53:04.766888+0000","last_unstale":"2023-10-08T05:53:06.802741+0000","last_undegraded":"2023-10-08T05:53:06.802741+0000","last_fullsized":"2023-10-08T05:53:06.802741+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:22.214955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793878+0000","last_change":"2023-10-08T05:53:05.801287+0000","last_active":"2023-10-08T05:53:06.793878+0000","last_peered":"2023-10-08T05:53:06.793878+0000","last_clean":"2023-10-08T05:53:06.793878+0000","last_became_active":"2023-10-08T05:53:05.801009+0000","last_became_peered":"2023-10-08T05:53:05.801009+0000","last_unstale":"2023-10-08T05:53:06.793878+0000","last_undegraded":"2023-10-08T05:53:06.793878+0000","last_fullsized":"2023-10-08T05:53:06.793878+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:54.138563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"103'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271056+0000","last_change":"2023-10-08T05:53:04.766762+0000","last_active":"2023-10-08T05:53:07.271056+0000","last_peered":"2023-10-08T05:53:07.271056+0000","last_clean":"2023-10-08T05:53:07.271056+0000","last_became_active":"2023-10-08T05:53:04.766473+0000","last_became_peered":"2023-10-08T05:53:04.766473+0000","last_unstale":"2023-10-08T05:53:07.271056+0000","last_undegraded":"2023-10-08T05:53:07.271056+0000","last_fullsized":"2023-10-08T05:53:07.271056+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:59.014929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.801976+0000","last_change":"2023-10-08T05:53:05.790331+0000","last_active":"2023-10-08T05:53:06.801976+0000","last_peered":"2023-10-08T05:53:06.801976+0000","last_clean":"2023-10-08T05:53:06.801976+0000","last_became_active":"2023-10-08T05:53:05.789869+0000","last_became_peered":"2023-10-08T05:53:05.789869+0000","last_unstale":"2023-10-08T05:53:06.801976+0000","last_undegraded":"2023-10-08T05:53:06.801976+0000","last_fullsized":"2023-10-08T05:53:06.801976+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:42:59.809635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793545+0000","last_change":"2023-10-08T05:53:04.768193+0000","last_active":"2023-10-08T05:53:06.793545+0000","last_peered":"2023-10-08T05:53:06.793545+0000","last_clean":"2023-10-08T05:53:06.793545+0000","last_became_active":"2023-10-08T05:53:04.767487+0000","last_became_peered":"2023-10-08T05:53:04.767487+0000","last_unstale":"2023-10-08T05:53:06.793545+0000","last_undegraded":"2023-10-08T05:53:06.793545+0000","last_fullsized":"2023-10-08T05:53:06.793545+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:05:11.946222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.801631+0000","last_change":"2023-10-08T05:53:05.794277+0000","last_active":"2023-10-08T05:53:06.801631+0000","last_peered":"2023-10-08T05:53:06.801631+0000","last_clean":"2023-10-08T05:53:06.801631+0000","last_became_active":"2023-10-08T05:53:05.793926+0000","last_became_peered":"2023-10-08T05:53:05.793926+0000","last_unstale":"2023-10-08T05:53:06.801631+0000","last_undegraded":"2023-10-08T05:53:06.801631+0000","last_fullsized":"2023-10-08T05:53:06.801631+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:33.771627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271115+0000","last_change":"2023-10-08T05:53:04.771324+0000","last_active":"2023-10-08T05:53:07.271115+0000","last_peered":"2023-10-08T05:53:07.271115+0000","last_clean":"2023-10-08T05:53:07.271115+0000","last_became_active":"2023-10-08T05:53:04.771015+0000","last_became_peered":"2023-10-08T05:53:04.771015+0000","last_unstale":"2023-10-08T05:53:07.271115+0000","last_undegraded":"2023-10-08T05:53:07.271115+0000","last_fullsized":"2023-10-08T05:53:07.271115+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:28.184091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793187+0000","last_change":"2023-10-08T05:53:05.796962+0000","last_active":"2023-10-08T05:53:06.793187+0000","last_peered":"2023-10-08T05:53:06.793187+0000","last_clean":"2023-10-08T05:53:06.793187+0000","last_became_active":"2023-10-08T05:53:05.796608+0000","last_became_peered":"2023-10-08T05:53:05.796608+0000","last_unstale":"2023-10-08T05:53:06.793187+0000","last_undegraded":"2023-10-08T05:53:06.793187+0000","last_fullsized":"2023-10-08T05:53:06.793187+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:49.416812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803302+0000","last_change":"2023-10-08T05:53:04.774772+0000","last_active":"2023-10-08T05:53:06.803302+0000","last_peered":"2023-10-08T05:53:06.803302+0000","last_clean":"2023-10-08T05:53:06.803302+0000","last_became_active":"2023-10-08T05:53:04.774479+0000","last_became_peered":"2023-10-08T05:53:04.774479+0000","last_unstale":"2023-10-08T05:53:06.803302+0000","last_undegraded":"2023-10-08T05:53:06.803302+0000","last_fullsized":"2023-10-08T05:53:06.803302+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:59.014929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271312+0000","last_change":"2023-10-08T05:53:05.800443+0000","last_active":"2023-10-08T05:53:07.271312+0000","last_peered":"2023-10-08T05:53:07.271312+0000","last_clean":"2023-10-08T05:53:07.271312+0000","last_became_active":"2023-10-08T05:53:05.800145+0000","last_became_peered":"2023-10-08T05:53:05.800145+0000","last_unstale":"2023-10-08T05:53:07.271312+0000","last_undegraded":"2023-10-08T05:53:07.271312+0000","last_fullsized":"2023-10-08T05:53:07.271312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:35.349762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271710+0000","last_change":"2023-10-08T05:53:04.776402+0000","last_active":"2023-10-08T05:53:07.271710+0000","last_peered":"2023-10-08T05:53:07.271710+0000","last_clean":"2023-10-08T05:53:07.271710+0000","last_became_active":"2023-10-08T05:53:04.776120+0000","last_became_peered":"2023-10-08T05:53:04.776120+0000","last_unstale":"2023-10-08T05:53:07.271710+0000","last_undegraded":"2023-10-08T05:53:07.271710+0000","last_fullsized":"2023-10-08T05:53:07.271710+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:13.798415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274122+0000","last_change":"2023-10-08T05:53:05.788107+0000","last_active":"2023-10-08T05:53:07.274122+0000","last_peered":"2023-10-08T05:53:07.274122+0000","last_clean":"2023-10-08T05:53:07.274122+0000","last_became_active":"2023-10-08T05:53:05.787822+0000","last_became_peered":"2023-10-08T05:53:05.787822+0000","last_unstale":"2023-10-08T05:53:07.274122+0000","last_undegraded":"2023-10-08T05:53:07.274122+0000","last_fullsized":"2023-10-08T05:53:07.274122+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:24:17.612355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"103'3","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274136+0000","last_change":"2023-10-08T05:53:04.761003+0000","last_active":"2023-10-08T05:53:07.274136+0000","last_peered":"2023-10-08T05:53:07.274136+0000","last_clean":"2023-10-08T05:53:07.274136+0000","last_became_active":"2023-10-08T05:53:04.760697+0000","last_became_peered":"2023-10-08T05:53:04.760697+0000","last_unstale":"2023-10-08T05:53:07.274136+0000","last_undegraded":"2023-10-08T05:53:07.274136+0000","last_fullsized":"2023-10-08T05:53:07.274136+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:12:57.067239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.273525+0000","last_change":"2023-10-08T05:53:05.793564+0000","last_active":"2023-10-08T05:53:07.273525+0000","last_peered":"2023-10-08T05:53:07.273525+0000","last_clean":"2023-10-08T05:53:07.273525+0000","last_became_active":"2023-10-08T05:53:05.793301+0000","last_became_peered":"2023-10-08T05:53:05.793301+0000","last_unstale":"2023-10-08T05:53:07.273525+0000","last_undegraded":"2023-10-08T05:53:07.273525+0000","last_fullsized":"2023-10-08T05:53:07.273525+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:51.408521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"103'6","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800195+0000","last_change":"2023-10-08T05:53:04.767287+0000","last_active":"2023-10-08T05:53:06.800195+0000","last_peered":"2023-10-08T05:53:06.800195+0000","last_clean":"2023-10-08T05:53:06.800195+0000","last_became_active":"2023-10-08T05:53:04.766938+0000","last_became_peered":"2023-10-08T05:53:04.766938+0000","last_unstale":"2023-10-08T05:53:06.800195+0000","last_undegraded":"2023-10-08T05:53:06.800195+0000","last_fullsized":"2023-10-08T05:53:06.800195+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:23.295871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271761+0000","last_change":"2023-10-08T05:53:05.793641+0000","last_active":"2023-10-08T05:53:07.271761+0000","last_peered":"2023-10-08T05:53:07.271761+0000","last_clean":"2023-10-08T05:53:07.271761+0000","last_became_active":"2023-10-08T05:53:05.793002+0000","last_became_peered":"2023-10-08T05:53:05.793002+0000","last_unstale":"2023-10-08T05:53:07.271761+0000","last_undegraded":"2023-10-08T05:53:07.271761+0000","last_fullsized":"2023-10-08T05:53:07.271761+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:14.797057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271054+0000","last_change":"2023-10-08T05:53:04.773332+0000","last_active":"2023-10-08T05:53:07.271054+0000","last_peered":"2023-10-08T05:53:07.271054+0000","last_clean":"2023-10-08T05:53:07.271054+0000","last_became_active":"2023-10-08T05:53:04.773034+0000","last_became_peered":"2023-10-08T05:53:04.773034+0000","last_unstale":"2023-10-08T05:53:07.271054+0000","last_undegraded":"2023-10-08T05:53:07.271054+0000","last_fullsized":"2023-10-08T05:53:07.271054+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:45.638271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800582+0000","last_change":"2023-10-08T05:53:05.786818+0000","last_active":"2023-10-08T05:53:06.800582+0000","last_peered":"2023-10-08T05:53:06.800582+0000","last_clean":"2023-10-08T05:53:06.800582+0000","last_became_active":"2023-10-08T05:53:05.786313+0000","last_became_peered":"2023-10-08T05:53:05.786313+0000","last_unstale":"2023-10-08T05:53:06.800582+0000","last_undegraded":"2023-10-08T05:53:06.800582+0000","last_fullsized":"2023-10-08T05:53:06.800582+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:49.674326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803032+0000","last_change":"2023-10-08T05:53:04.773639+0000","last_active":"2023-10-08T05:53:06.803032+0000","last_peered":"2023-10-08T05:53:06.803032+0000","last_clean":"2023-10-08T05:53:06.803032+0000","last_became_active":"2023-10-08T05:53:04.771979+0000","last_became_peered":"2023-10-08T05:53:04.771979+0000","last_unstale":"2023-10-08T05:53:06.803032+0000","last_undegraded":"2023-10-08T05:53:06.803032+0000","last_fullsized":"2023-10-08T05:53:06.803032+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:44.955390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800009+0000","last_change":"2023-10-08T05:53:05.787263+0000","last_active":"2023-10-08T05:53:06.800009+0000","last_peered":"2023-10-08T05:53:06.800009+0000","last_clean":"2023-10-08T05:53:06.800009+0000","last_became_active":"2023-10-08T05:53:05.786778+0000","last_became_peered":"2023-10-08T05:53:05.786778+0000","last_unstale":"2023-10-08T05:53:06.800009+0000","last_undegraded":"2023-10-08T05:53:06.800009+0000","last_fullsized":"2023-10-08T05:53:06.800009+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:39.075379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.275390+0000","last_change":"2023-10-08T05:53:04.768859+0000","last_active":"2023-10-08T05:53:07.275390+0000","last_peered":"2023-10-08T05:53:07.275390+0000","last_clean":"2023-10-08T05:53:07.275390+0000","last_became_active":"2023-10-08T05:53:04.768545+0000","last_became_peered":"2023-10-08T05:53:04.768545+0000","last_unstale":"2023-10-08T05:53:07.275390+0000","last_undegraded":"2023-10-08T05:53:07.275390+0000","last_fullsized":"2023-10-08T05:53:07.275390+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:20:18.077161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802062+0000","last_change":"2023-10-08T05:53:05.790447+0000","last_active":"2023-10-08T05:53:06.802062+0000","last_peered":"2023-10-08T05:53:06.802062+0000","last_clean":"2023-10-08T05:53:06.802062+0000","last_became_active":"2023-10-08T05:53:05.789296+0000","last_became_peered":"2023-10-08T05:53:05.789296+0000","last_unstale":"2023-10-08T05:53:06.802062+0000","last_undegraded":"2023-10-08T05:53:06.802062+0000","last_fullsized":"2023-10-08T05:53:06.802062+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:15.865170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"103'8","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793306+0000","last_change":"2023-10-08T05:53:04.756043+0000","last_active":"2023-10-08T05:53:06.793306+0000","last_peered":"2023-10-08T05:53:06.793306+0000","last_clean":"2023-10-08T05:53:06.793306+0000","last_became_active":"2023-10-08T05:53:04.755754+0000","last_became_peered":"2023-10-08T05:53:04.755754+0000","last_unstale":"2023-10-08T05:53:06.793306+0000","last_undegraded":"2023-10-08T05:53:06.793306+0000","last_fullsized":"2023-10-08T05:53:06.793306+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:26.232188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271834+0000","last_change":"2023-10-08T05:53:05.799933+0000","last_active":"2023-10-08T05:53:07.271834+0000","last_peered":"2023-10-08T05:53:07.271834+0000","last_clean":"2023-10-08T05:53:07.271834+0000","last_became_active":"2023-10-08T05:53:05.799615+0000","last_became_peered":"2023-10-08T05:53:05.799615+0000","last_unstale":"2023-10-08T05:53:07.271834+0000","last_undegraded":"2023-10-08T05:53:07.271834+0000","last_fullsized":"2023-10-08T05:53:07.271834+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:02.485242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"103'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803533+0000","last_change":"2023-10-08T05:53:04.773136+0000","last_active":"2023-10-08T05:53:06.803533+0000","last_peered":"2023-10-08T05:53:06.803533+0000","last_clean":"2023-10-08T05:53:06.803533+0000","last_became_active":"2023-10-08T05:53:04.772843+0000","last_became_peered":"2023-10-08T05:53:04.772843+0000","last_unstale":"2023-10-08T05:53:06.803533+0000","last_undegraded":"2023-10-08T05:53:06.803533+0000","last_fullsized":"2023-10-08T05:53:06.803533+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:59.541711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.275286+0000","last_change":"2023-10-08T05:53:05.795230+0000","last_active":"2023-10-08T05:53:07.275286+0000","last_peered":"2023-10-08T05:53:07.275286+0000","last_clean":"2023-10-08T05:53:07.275286+0000","last_became_active":"2023-10-08T05:53:05.794943+0000","last_became_peered":"2023-10-08T05:53:05.794943+0000","last_unstale":"2023-10-08T05:53:07.275286+0000","last_undegraded":"2023-10-08T05:53:07.275286+0000","last_fullsized":"2023-10-08T05:53:07.275286+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:02.025736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802225+0000","last_change":"2023-10-08T05:53:04.762301+0000","last_active":"2023-10-08T05:53:06.802225+0000","last_peered":"2023-10-08T05:53:06.802225+0000","last_clean":"2023-10-08T05:53:06.802225+0000","last_became_active":"2023-10-08T05:53:04.760836+0000","last_became_peered":"2023-10-08T05:53:04.760836+0000","last_unstale":"2023-10-08T05:53:06.802225+0000","last_undegraded":"2023-10-08T05:53:06.802225+0000","last_fullsized":"2023-10-08T05:53:06.802225+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:43:32.838450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272125+0000","last_change":"2023-10-08T05:53:05.804317+0000","last_active":"2023-10-08T05:53:07.272125+0000","last_peered":"2023-10-08T05:53:07.272125+0000","last_clean":"2023-10-08T05:53:07.272125+0000","last_became_active":"2023-10-08T05:53:05.804041+0000","last_became_peered":"2023-10-08T05:53:05.804041+0000","last_unstale":"2023-10-08T05:53:07.272125+0000","last_undegraded":"2023-10-08T05:53:07.272125+0000","last_fullsized":"2023-10-08T05:53:07.272125+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:32.795428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"103'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274363+0000","last_change":"2023-10-08T05:53:04.770620+0000","last_active":"2023-10-08T05:53:07.274363+0000","last_peered":"2023-10-08T05:53:07.274363+0000","last_clean":"2023-10-08T05:53:07.274363+0000","last_became_active":"2023-10-08T05:53:04.770350+0000","last_became_peered":"2023-10-08T05:53:04.770350+0000","last_unstale":"2023-10-08T05:53:07.274363+0000","last_undegraded":"2023-10-08T05:53:07.274363+0000","last_fullsized":"2023-10-08T05:53:07.274363+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:08.762943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272615+0000","last_change":"2023-10-08T05:53:05.801697+0000","last_active":"2023-10-08T05:53:07.272615+0000","last_peered":"2023-10-08T05:53:07.272615+0000","last_clean":"2023-10-08T05:53:07.272615+0000","last_became_active":"2023-10-08T05:53:05.801427+0000","last_became_peered":"2023-10-08T05:53:05.801427+0000","last_unstale":"2023-10-08T05:53:07.272615+0000","last_undegraded":"2023-10-08T05:53:07.272615+0000","last_fullsized":"2023-10-08T05:53:07.272615+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:23.414780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272945+0000","last_change":"2023-10-08T05:53:04.761736+0000","last_active":"2023-10-08T05:53:07.272945+0000","last_peered":"2023-10-08T05:53:07.272945+0000","last_clean":"2023-10-08T05:53:07.272945+0000","last_became_active":"2023-10-08T05:53:04.761450+0000","last_became_peered":"2023-10-08T05:53:04.761450+0000","last_unstale":"2023-10-08T05:53:07.272945+0000","last_undegraded":"2023-10-08T05:53:07.272945+0000","last_fullsized":"2023-10-08T05:53:07.272945+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:38:58.219810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.794012+0000","last_change":"2023-10-08T05:53:05.794678+0000","last_active":"2023-10-08T05:53:06.794012+0000","last_peered":"2023-10-08T05:53:06.794012+0000","last_clean":"2023-10-08T05:53:06.794012+0000","last_became_active":"2023-10-08T05:53:05.794374+0000","last_became_peered":"2023-10-08T05:53:05.794374+0000","last_unstale":"2023-10-08T05:53:06.794012+0000","last_undegraded":"2023-10-08T05:53:06.794012+0000","last_fullsized":"2023-10-08T05:53:06.794012+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:25.569321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272788+0000","last_change":"2023-10-08T05:53:04.752567+0000","last_active":"2023-10-08T05:53:07.272788+0000","last_peered":"2023-10-08T05:53:07.272788+0000","last_clean":"2023-10-08T05:53:07.272788+0000","last_became_active":"2023-10-08T05:53:04.752160+0000","last_became_peered":"2023-10-08T05:53:04.752160+0000","last_unstale":"2023-10-08T05:53:07.272788+0000","last_undegraded":"2023-10-08T05:53:07.272788+0000","last_fullsized":"2023-10-08T05:53:07.272788+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:27.133706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.794220+0000","last_change":"2023-10-08T05:53:05.795694+0000","last_active":"2023-10-08T05:53:06.794220+0000","last_peered":"2023-10-08T05:53:06.794220+0000","last_clean":"2023-10-08T05:53:06.794220+0000","last_became_active":"2023-10-08T05:53:05.795344+0000","last_became_peered":"2023-10-08T05:53:05.795344+0000","last_unstale":"2023-10-08T05:53:06.794220+0000","last_undegraded":"2023-10-08T05:53:06.794220+0000","last_fullsized":"2023-10-08T05:53:06.794220+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:39:01.800884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272120+0000","last_change":"2023-10-08T05:53:04.766767+0000","last_active":"2023-10-08T05:53:07.272120+0000","last_peered":"2023-10-08T05:53:07.272120+0000","last_clean":"2023-10-08T05:53:07.272120+0000","last_became_active":"2023-10-08T05:53:04.766498+0000","last_became_peered":"2023-10-08T05:53:04.766498+0000","last_unstale":"2023-10-08T05:53:07.272120+0000","last_undegraded":"2023-10-08T05:53:07.272120+0000","last_fullsized":"2023-10-08T05:53:07.272120+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:45.638271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802428+0000","last_change":"2023-10-08T05:53:05.795323+0000","last_active":"2023-10-08T05:53:06.802428+0000","last_peered":"2023-10-08T05:53:06.802428+0000","last_clean":"2023-10-08T05:53:06.802428+0000","last_became_active":"2023-10-08T05:53:05.794959+0000","last_became_peered":"2023-10-08T05:53:05.794959+0000","last_unstale":"2023-10-08T05:53:06.802428+0000","last_undegraded":"2023-10-08T05:53:06.802428+0000","last_fullsized":"2023-10-08T05:53:06.802428+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:58.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272214+0000","last_change":"2023-10-08T05:53:04.765996+0000","last_active":"2023-10-08T05:53:07.272214+0000","last_peered":"2023-10-08T05:53:07.272214+0000","last_clean":"2023-10-08T05:53:07.272214+0000","last_became_active":"2023-10-08T05:53:04.765694+0000","last_became_peered":"2023-10-08T05:53:04.765694+0000","last_unstale":"2023-10-08T05:53:07.272214+0000","last_undegraded":"2023-10-08T05:53:07.272214+0000","last_fullsized":"2023-10-08T05:53:07.272214+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:17.491464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.799649+0000","last_change":"2023-10-08T05:53:05.786194+0000","last_active":"2023-10-08T05:53:06.799649+0000","last_peered":"2023-10-08T05:53:06.799649+0000","last_clean":"2023-10-08T05:53:06.799649+0000","last_became_active":"2023-10-08T05:53:05.785790+0000","last_became_peered":"2023-10-08T05:53:05.785790+0000","last_unstale":"2023-10-08T05:53:06.799649+0000","last_undegraded":"2023-10-08T05:53:06.799649+0000","last_fullsized":"2023-10-08T05:53:06.799649+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:26.370365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271601+0000","last_change":"2023-10-08T05:53:04.766900+0000","last_active":"2023-10-08T05:53:07.271601+0000","last_peered":"2023-10-08T05:53:07.271601+0000","last_clean":"2023-10-08T05:53:07.271601+0000","last_became_active":"2023-10-08T05:53:04.766635+0000","last_became_peered":"2023-10-08T05:53:04.766635+0000","last_unstale":"2023-10-08T05:53:07.271601+0000","last_undegraded":"2023-10-08T05:53:07.271601+0000","last_fullsized":"2023-10-08T05:53:07.271601+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:01:46.238991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800228+0000","last_change":"2023-10-08T05:53:05.791929+0000","last_active":"2023-10-08T05:53:06.800228+0000","last_peered":"2023-10-08T05:53:06.800228+0000","last_clean":"2023-10-08T05:53:06.800228+0000","last_became_active":"2023-10-08T05:53:05.791631+0000","last_became_peered":"2023-10-08T05:53:05.791631+0000","last_unstale":"2023-10-08T05:53:06.800228+0000","last_undegraded":"2023-10-08T05:53:06.800228+0000","last_fullsized":"2023-10-08T05:53:06.800228+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:30.265275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.794158+0000","last_change":"2023-10-08T05:53:04.763353+0000","last_active":"2023-10-08T05:53:06.794158+0000","last_peered":"2023-10-08T05:53:06.794158+0000","last_clean":"2023-10-08T05:53:06.794158+0000","last_became_active":"2023-10-08T05:53:04.763086+0000","last_became_peered":"2023-10-08T05:53:04.763086+0000","last_unstale":"2023-10-08T05:53:06.794158+0000","last_undegraded":"2023-10-08T05:53:06.794158+0000","last_fullsized":"2023-10-08T05:53:06.794158+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:31.861846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.801343+0000","last_change":"2023-10-08T05:53:05.791920+0000","last_active":"2023-10-08T05:53:06.801343+0000","last_peered":"2023-10-08T05:53:06.801343+0000","last_clean":"2023-10-08T05:53:06.801343+0000","last_became_active":"2023-10-08T05:53:05.791524+0000","last_became_peered":"2023-10-08T05:53:05.791524+0000","last_unstale":"2023-10-08T05:53:06.801343+0000","last_undegraded":"2023-10-08T05:53:06.801343+0000","last_fullsized":"2023-10-08T05:53:06.801343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:05.092018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803355+0000","last_change":"2023-10-08T05:53:04.775537+0000","last_active":"2023-10-08T05:53:06.803355+0000","last_peered":"2023-10-08T05:53:06.803355+0000","last_clean":"2023-10-08T05:53:06.803355+0000","last_became_active":"2023-10-08T05:53:04.775214+0000","last_became_peered":"2023-10-08T05:53:04.775214+0000","last_unstale":"2023-10-08T05:53:06.803355+0000","last_undegraded":"2023-10-08T05:53:06.803355+0000","last_fullsized":"2023-10-08T05:53:06.803355+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:17.491464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793554+0000","last_change":"2023-10-08T05:53:05.803359+0000","last_active":"2023-10-08T05:53:06.793554+0000","last_peered":"2023-10-08T05:53:06.793554+0000","last_clean":"2023-10-08T05:53:06.793554+0000","last_became_active":"2023-10-08T05:53:05.803102+0000","last_became_peered":"2023-10-08T05:53:05.803102+0000","last_unstale":"2023-10-08T05:53:06.793554+0000","last_undegraded":"2023-10-08T05:53:06.793554+0000","last_fullsized":"2023-10-08T05:53:06.793554+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:13.331423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802301+0000","last_change":"2023-10-08T05:53:04.775328+0000","last_active":"2023-10-08T05:53:06.802301+0000","last_peered":"2023-10-08T05:53:06.802301+0000","last_clean":"2023-10-08T05:53:06.802301+0000","last_became_active":"2023-10-08T05:53:04.774980+0000","last_became_peered":"2023-10-08T05:53:04.774980+0000","last_unstale":"2023-10-08T05:53:06.802301+0000","last_undegraded":"2023-10-08T05:53:06.802301+0000","last_fullsized":"2023-10-08T05:53:06.802301+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:27.133706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.799658+0000","last_change":"2023-10-08T05:53:05.791160+0000","last_active":"2023-10-08T05:53:06.799658+0000","last_peered":"2023-10-08T05:53:06.799658+0000","last_clean":"2023-10-08T05:53:06.799658+0000","last_became_active":"2023-10-08T05:53:05.790851+0000","last_became_peered":"2023-10-08T05:53:05.790851+0000","last_unstale":"2023-10-08T05:53:06.799658+0000","last_undegraded":"2023-10-08T05:53:06.799658+0000","last_fullsized":"2023-10-08T05:53:06.799658+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:55.636199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800894+0000","last_change":"2023-10-08T05:53:04.768188+0000","last_active":"2023-10-08T05:53:06.800894+0000","last_peered":"2023-10-08T05:53:06.800894+0000","last_clean":"2023-10-08T05:53:06.800894+0000","last_became_active":"2023-10-08T05:53:04.767893+0000","last_became_peered":"2023-10-08T05:53:04.767893+0000","last_unstale":"2023-10-08T05:53:06.800894+0000","last_undegraded":"2023-10-08T05:53:06.800894+0000","last_fullsized":"2023-10-08T05:53:06.800894+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:02.828936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.801455+0000","last_change":"2023-10-08T05:53:05.797926+0000","last_active":"2023-10-08T05:53:06.801455+0000","last_peered":"2023-10-08T05:53:06.801455+0000","last_clean":"2023-10-08T05:53:06.801455+0000","last_became_active":"2023-10-08T05:53:05.797610+0000","last_became_peered":"2023-10-08T05:53:05.797610+0000","last_unstale":"2023-10-08T05:53:06.801455+0000","last_undegraded":"2023-10-08T05:53:06.801455+0000","last_fullsized":"2023-10-08T05:53:06.801455+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:21:25.223794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272364+0000","last_change":"2023-10-08T05:53:04.769571+0000","last_active":"2023-10-08T05:53:07.272364+0000","last_peered":"2023-10-08T05:53:07.272364+0000","last_clean":"2023-10-08T05:53:07.272364+0000","last_became_active":"2023-10-08T05:53:04.769300+0000","last_became_peered":"2023-10-08T05:53:04.769300+0000","last_unstale":"2023-10-08T05:53:07.272364+0000","last_undegraded":"2023-10-08T05:53:07.272364+0000","last_fullsized":"2023-10-08T05:53:07.272364+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:16.759289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274739+0000","last_change":"2023-10-08T05:53:05.808996+0000","last_active":"2023-10-08T05:53:07.274739+0000","last_peered":"2023-10-08T05:53:07.274739+0000","last_clean":"2023-10-08T05:53:07.274739+0000","last_became_active":"2023-10-08T05:53:05.808706+0000","last_became_peered":"2023-10-08T05:53:05.808706+0000","last_unstale":"2023-10-08T05:53:07.274739+0000","last_undegraded":"2023-10-08T05:53:07.274739+0000","last_fullsized":"2023-10-08T05:53:07.274739+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:02.546020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800446+0000","last_change":"2023-10-08T05:53:04.759641+0000","last_active":"2023-10-08T05:53:06.800446+0000","last_peered":"2023-10-08T05:53:06.800446+0000","last_clean":"2023-10-08T05:53:06.800446+0000","last_became_active":"2023-10-08T05:53:04.759317+0000","last_became_peered":"2023-10-08T05:53:04.759317+0000","last_unstale":"2023-10-08T05:53:06.800446+0000","last_undegraded":"2023-10-08T05:53:06.800446+0000","last_fullsized":"2023-10-08T05:53:06.800446+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:38:24.605530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274995+0000","last_change":"2023-10-08T05:53:05.795306+0000","last_active":"2023-10-08T05:53:07.274995+0000","last_peered":"2023-10-08T05:53:07.274995+0000","last_clean":"2023-10-08T05:53:07.274995+0000","last_became_active":"2023-10-08T05:53:05.795054+0000","last_became_peered":"2023-10-08T05:53:05.795054+0000","last_unstale":"2023-10-08T05:53:07.274995+0000","last_undegraded":"2023-10-08T05:53:07.274995+0000","last_fullsized":"2023-10-08T05:53:07.274995+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:59:11.703415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271902+0000","last_change":"2023-10-08T05:53:04.751604+0000","last_active":"2023-10-08T05:53:07.271902+0000","last_peered":"2023-10-08T05:53:07.271902+0000","last_clean":"2023-10-08T05:53:07.271902+0000","last_became_active":"2023-10-08T05:53:04.751201+0000","last_became_peered":"2023-10-08T05:53:04.751201+0000","last_unstale":"2023-10-08T05:53:07.271902+0000","last_undegraded":"2023-10-08T05:53:07.271902+0000","last_fullsized":"2023-10-08T05:53:07.271902+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:57.910900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803470+0000","last_change":"2023-10-08T05:53:05.811605+0000","last_active":"2023-10-08T05:53:06.803470+0000","last_peered":"2023-10-08T05:53:06.803470+0000","last_clean":"2023-10-08T05:53:06.803470+0000","last_became_active":"2023-10-08T05:53:05.811337+0000","last_became_peered":"2023-10-08T05:53:05.811337+0000","last_unstale":"2023-10-08T05:53:06.803470+0000","last_undegraded":"2023-10-08T05:53:06.803470+0000","last_fullsized":"2023-10-08T05:53:06.803470+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:02.485242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274329+0000","last_change":"2023-10-08T05:53:04.763510+0000","last_active":"2023-10-08T05:53:07.274329+0000","last_peered":"2023-10-08T05:53:07.274329+0000","last_clean":"2023-10-08T05:53:07.274329+0000","last_became_active":"2023-10-08T05:53:04.763227+0000","last_became_peered":"2023-10-08T05:53:04.763227+0000","last_unstale":"2023-10-08T05:53:07.274329+0000","last_undegraded":"2023-10-08T05:53:07.274329+0000","last_fullsized":"2023-10-08T05:53:07.274329+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:43.031877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"104'2158","reported_seq":2432,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.598658+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:53:07.598658+0000","last_peered":"2023-10-08T05:53:07.598658+0000","last_clean":"2023-10-08T05:53:07.598658+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:53:07.598658+0000","last_undegraded":"2023-10-08T05:53:07.598658+0000","last_fullsized":"2023-10-08T05:53:07.598658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2158,"log_dups_size":0,"ondisk_log_size":2158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":550,"num_read_kb":1032,"num_write":245,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":550,"num_read_kb":1032,"num_write":245,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2158,"ondisk_log_size":2158,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575053,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51052,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320788,"statfs":{"total":96636764160,"available":96584486912,"internally_reserved":0,"allocated":1871872,"data_stored":1234734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.171,"15min":1.2010000000000001},"min":{"1min":0.73299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.603,"5min":1.8859999999999999,"15min":1.901},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.2070000000000001},"min":{"1min":0.71499999999999997,"5min":0.44,"15min":0.44},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.726},"last":1.718}]},{"osd":1,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.177,"15min":1.2310000000000001},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.554,"5min":1.6950000000000001,"15min":1.8360000000000001},"last":1.466},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.266},"min":{"1min":0.83299999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6759999999999999,"5min":1.7689999999999999,"15min":1.9379999999999999},"last":1.5469999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.2010000000000001,"15min":1.226},"min":{"1min":0.71899999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.748,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.117},{"interface":"front","average":{"1min":1.288,"5min":1.319,"15min":1.323},"min":{"1min":0.88600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.665,"5min":1.752,"15min":1.7769999999999999},"last":1.052}]},{"osd":3,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.258,"15min":1.28},"min":{"1min":0.85099999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6060000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.351,"15min":1.3640000000000001},"min":{"1min":1.0760000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.6020000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2929999999999999,"15min":1.284},"min":{"1min":0.76200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.837,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3120000000000001,"15min":1.331},"min":{"1min":0.89600000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0550000000000002,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.681}]},{"osd":5,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.375,"15min":1.3959999999999999},"min":{"1min":1.0049999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9810000000000001,"5min":2.0339999999999998,"15min":2.169},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.405,"15min":1.4179999999999999},"min":{"1min":0.93600000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.694,"5min":1.9319999999999999,"15min":2.206},"last":1.8049999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.512,"15min":1.5189999999999999},"min":{"1min":1.0940000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.968,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.4490000000000001,"15min":1.4610000000000001},"min":{"1min":1.04,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.8460000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1286144,"data_stored":645956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3049999999999999,"15min":1.2729999999999999},"min":{"1min":0.80900000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9119999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.484},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.2689999999999999,"15min":1.3100000000000001},"min":{"1min":0.91000000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8100000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.98399999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.157,"15min":1.1679999999999999},"min":{"1min":0.875,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6890000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.196,"15min":1.1850000000000001},"min":{"1min":0.92700000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8169999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.1619999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.143,"15min":1.1919999999999999},"min":{"1min":0.98799999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.774,"5min":1.774,"15min":1.796},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.262},"min":{"1min":0.90700000000000003,"5min":0.78300000000000003,"15min":0.58699999999999997},"max":{"1min":1.617,"5min":1.752,"15min":1.8620000000000001},"last":1.02}]},{"osd":3,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.351,"15min":1.3759999999999999},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.8420000000000001,"5min":2.137,"15min":2.137},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.4119999999999999,"15min":1.458},"min":{"1min":1.006,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.024,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.86799999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.3220000000000001,"15min":1.3089999999999999},"min":{"1min":0.90800000000000003,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.599,"5min":1.375,"15min":1.377},"min":{"1min":1.125,"5min":0.78400000000000003,"15min":0.64300000000000002},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.381}]},{"osd":5,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.319,"15min":1.333},"min":{"1min":0.95599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":2.1970000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.694,"5min":1.5189999999999999,"15min":1.484},"min":{"1min":1.208,"5min":0.747,"15min":0.747},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.6419999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.7290000000000001,"5min":1.4970000000000001,"15min":1.51},"min":{"1min":1.3360000000000001,"5min":0.97899999999999998,"15min":0.77200000000000002},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.629,"5min":1.472,"15min":1.47},"min":{"1min":1.0389999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.3320000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575052,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1265664,"data_stored":637714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0329999999999999,"15min":1.1120000000000001},"min":{"1min":0.76600000000000001,"5min":0.58599999999999997,"15min":0.46700000000000003},"max":{"1min":1.5469999999999999,"5min":1.627,"15min":1.8169999999999999},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.1859999999999999,"15min":1.202},"min":{"1min":0.78900000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.546,"5min":1.7430000000000001,"15min":1.758},"last":0.90100000000000002}]},{"osd":1,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.2050000000000001,"15min":1.204},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.651,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.218,"15min":1.25},"min":{"1min":0.85499999999999998,"5min":0.81399999999999995,"15min":0.64700000000000002},"max":{"1min":1.5349999999999999,"5min":1.657,"15min":2.0339999999999998},"last":1.488}]},{"osd":2,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2,"15min":1.2270000000000001},"min":{"1min":0.875,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2649999999999999,"15min":1.272},"min":{"1min":0.77200000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.5369999999999999,"5min":3.282,"15min":3.282},"last":0.71399999999999997}]},{"osd":3,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3100000000000001,"15min":1.3280000000000001},"min":{"1min":0.88,"5min":0.77700000000000002,"15min":0.63500000000000001},"max":{"1min":1.6759999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.282},{"interface":"front","average":{"1min":1.397,"5min":1.375,"15min":1.3839999999999999},"min":{"1min":1.0029999999999999,"5min":0.89700000000000002,"15min":0.84799999999999998},"max":{"1min":1.919,"5min":1.919,"15min":2.2280000000000002},"last":1.2549999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.321,"15min":1.29},"min":{"1min":0.83499999999999996,"5min":0.82899999999999996,"15min":0.80400000000000005},"max":{"1min":1.681,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4630000000000001,"15min":1.405},"min":{"1min":1.1319999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7569999999999999,"5min":3.456,"15min":3.456},"last":1.762}]},{"osd":6,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.3899999999999999,"15min":1.393},"min":{"1min":0.91900000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":2.359},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.458,"15min":1.454},"min":{"1min":1.0669999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.782,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.04}]},{"osd":7,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.552,"15min":1.5249999999999999},"min":{"1min":1.238,"5min":1.0940000000000001,"15min":0.53900000000000003},"max":{"1min":1.843,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":1.593},{"interface":"front","average":{"1min":1.639,"5min":1.5589999999999999,"15min":1.544},"min":{"1min":1.083,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.157,"5min":2.157,"15min":2.4460000000000002},"last":1.5469999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575052,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":646060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.2230000000000001,"15min":1.26},"min":{"1min":0.80500000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.589,"5min":1.8009999999999999,"15min":1.863},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.3440000000000001,"15min":1.3779999999999999},"min":{"1min":0.76800000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.6439999999999999,"5min":1.8520000000000001,"15min":2.0680000000000001},"last":1.796}]},{"osd":1,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2589999999999999,"15min":1.272},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.538,"5min":2.3519999999999999,"15min":2.3799999999999999},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2030000000000001,"15min":1.21},"min":{"1min":0.84499999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6719999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.3540000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.226,"15min":1.2190000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.107},{"interface":"front","average":{"1min":1.272,"5min":1.3400000000000001,"15min":1.3560000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.6299999999999999,"5min":2.423,"15min":2.423},"last":1.6020000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2509999999999999,"15min":1.264},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6080000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.323,"15min":1.337},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.5329999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.3620000000000001,"15min":1.333},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.7310000000000001,"5min":2.48,"15min":2.48},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.383,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7090000000000001,"5min":2.504,"15min":2.504},"last":1.893}]},{"osd":6,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.3779999999999999,"15min":1.3560000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.55000000000000004},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.512,"15min":1.502},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.8049999999999999,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.929}]},{"osd":7,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4990000000000001,"15min":1.4570000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8029999999999999,"5min":4.125,"15min":4.125},"last":1.996},{"interface":"front","average":{"1min":1.46,"5min":1.5009999999999999,"15min":1.5169999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.4350000000000001,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":2.0339999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575054,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36576,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335264,"statfs":{"total":96636764160,"available":96599310336,"internally_reserved":0,"allocated":1265664,"data_stored":638262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.91900000000000004,"15min":0.91800000000000004},"min":{"1min":0.58299999999999996,"5min":0.32100000000000001,"15min":0.29599999999999999},"max":{"1min":1.194,"5min":1.5269999999999999,"15min":1.72},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.04,"15min":1.083},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3380000000000001,"5min":1.5,"15min":1.6140000000000001},"last":0.95499999999999996}]},{"osd":1,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.92300000000000004,"15min":0.96699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.429,"5min":1.5629999999999999,"15min":1.6040000000000001},"last":1.042},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.198,"15min":1.206},"min":{"1min":0.63700000000000001,"5min":0.56599999999999995,"15min":0.436},"max":{"1min":1.54,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.1759999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.0369999999999999,"15min":1.071},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4350000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.097,"5min":1.1399999999999999,"15min":1.145},"min":{"1min":0.69299999999999995,"5min":0.60499999999999998,"15min":0.41899999999999998},"max":{"1min":1.6040000000000001,"5min":1.6819999999999999,"15min":1.788},"last":1.0720000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1990000000000001,"15min":1.202},"min":{"1min":0.68999999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.429,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3680000000000001,"15min":1.3859999999999999},"min":{"1min":0.90900000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7450000000000001,"5min":2.589,"15min":2.589},"last":1.2130000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.304,"15min":1.302},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.73699999999999999},"max":{"1min":1.659,"5min":1.883,"15min":1.9219999999999999},"last":1.151},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.3400000000000001,"15min":1.3560000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.65200000000000002},"max":{"1min":1.6020000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.129}]},{"osd":6,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3129999999999999},"min":{"1min":0.89100000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.766,"5min":1.766,"15min":1.776},"last":1.26},{"interface":"front","average":{"1min":1.371,"5min":1.4410000000000001,"15min":1.444},"min":{"1min":0.95899999999999996,"5min":0.86799999999999999,"15min":0.86699999999999999},"max":{"1min":1.6220000000000001,"5min":2.012,"15min":2.0489999999999999},"last":1.3560000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.466,"15min":1.486},"min":{"1min":0.879,"5min":0.82399999999999995,"15min":0.72299999999999998},"max":{"1min":1.6599999999999999,"5min":2.085,"15min":2.1480000000000001},"last":1.242},{"interface":"front","average":{"1min":1.329,"5min":1.365,"15min":1.3859999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.704,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.0980000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575052,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58800,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313040,"statfs":{"total":96636764160,"available":96576552960,"internally_reserved":0,"allocated":1273856,"data_stored":638422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.82599999999999996,"15min":0.88900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.2390000000000001,"5min":1.5209999999999999,"15min":2.4009999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.014,"15min":1.036},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4650000000000001,"5min":1.5880000000000001,"15min":1.9470000000000001},"last":1.3560000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.91200000000000003,"15min":0.94699999999999995},"min":{"1min":0.50800000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.373,"5min":1.7609999999999999,"15min":2.1070000000000002},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.036,"15min":1.093},"min":{"1min":0.47799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4550000000000001,"5min":1.639,"15min":2.1469999999999998},"last":0.60999999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0720000000000001,"15min":1.1240000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.472,"5min":1.639,"15min":1.8120000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.2150000000000001,"15min":1.23},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.6399999999999999,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":0.89000000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.375,"15min":1.385},"min":{"1min":0.86899999999999999,"5min":0.622,"15min":0.622},"max":{"1min":1.5660000000000001,"5min":4.593,"15min":4.593},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2669999999999999,"15min":1.2889999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.528,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.0209999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.349,"15min":1.373},"min":{"1min":0.72599999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6160000000000001,"5min":6.7140000000000004,"15min":6.7140000000000004},"last":1.163},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.4059999999999999,"15min":1.448},"min":{"1min":0.71999999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.6910000000000001,"5min":2.0529999999999999,"15min":2.4119999999999999},"last":1.321}]},{"osd":6,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.3140000000000001,"15min":1.337},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.597,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.3919999999999999,"15min":1.4450000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.659,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.292}]},{"osd":7,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.419,"15min":1.4330000000000001},"min":{"1min":0.73699999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7170000000000001,"5min":6.609,"15min":6.609},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.4199999999999999,"15min":1.488},"min":{"1min":0.877,"5min":0.82299999999999995,"15min":0.79800000000000004},"max":{"1min":1.6060000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.391}]}]},{"osd":1,"up_from":13,"seq":55834575052,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36584,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335256,"statfs":{"total":96636764160,"available":96599302144,"internally_reserved":0,"allocated":1269760,"data_stored":637792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.96499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29799999999999999},"max":{"1min":1.2010000000000001,"5min":1.796,"15min":1.796},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0860000000000001,"15min":1.1279999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.433},"max":{"1min":1.431,"5min":1.6060000000000001,"15min":1.8089999999999999},"last":0.626}]},{"osd":2,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92300000000000004,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.39500000000000002,"15min":0.35899999999999999},"max":{"1min":1.3120000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0469999999999999,"15min":1.081},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5860000000000001,"5min":1.77,"15min":1.8049999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1140000000000001,"15min":1.175},"min":{"1min":0.5,"5min":0.5,"15min":0.438},"max":{"1min":1.46,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.012,"5min":1.0609999999999999,"15min":1.0669999999999999},"min":{"1min":0.55300000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.585,"5min":1.788,"15min":1.788},"last":0.872}]},{"osd":4,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2430000000000001,"15min":1.2969999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.63900000000000001},"max":{"1min":1.625,"5min":1.966,"15min":1.966},"last":1.073},{"interface":"front","average":{"1min":1.143,"5min":1.2170000000000001,"15min":1.226},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.61799999999999999},"max":{"1min":1.536,"5min":1.913,"15min":1.913},"last":1.389}]},{"osd":5,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2809999999999999,"15min":1.304},"min":{"1min":0.76500000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5089999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.641},{"interface":"front","average":{"1min":1.383,"5min":1.417,"15min":1.4410000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.755},"max":{"1min":1.663,"5min":2.077,"15min":2.077},"last":1.5469999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.381,"15min":1.4179999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.75,"5min":1.972,"15min":1.9970000000000001},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.399,"15min":1.4199999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6339999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.597}]},{"osd":7,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.4530000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6279999999999999,"5min":1.97,"15min":2.1869999999999998},"last":1.353},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4399999999999999,"15min":1.45},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.82999999999999996},"max":{"1min":1.829,"5min":2.286,"15min":2.306},"last":1.254}]}]},{"osd":0,"up_from":13,"seq":55834575053,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28840,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343000,"statfs":{"total":96636764160,"available":96607232000,"internally_reserved":0,"allocated":1867776,"data_stored":1234712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.88,"15min":0.90800000000000003},"min":{"1min":0.57799999999999996,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.026,"5min":1.0209999999999999,"15min":1.0269999999999999},"min":{"1min":0.52900000000000003,"5min":0.371,"15min":0.371},"max":{"1min":1.514,"5min":1.6719999999999999,"15min":1.675},"last":0.90700000000000003}]},{"osd":2,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.038,"15min":1.081},"min":{"1min":0.56899999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.357,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.94499999999999995,"15min":0.98999999999999999},"min":{"1min":0.52500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.722,"5min":1.722,"15min":1.78},"last":0.85199999999999998}]},{"osd":3,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.143,"15min":1.194},"min":{"1min":0.59799999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6319999999999999,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.071,"15min":1.077},"min":{"1min":0.47999999999999998,"5min":0.44700000000000001,"15min":0.439},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.022}]},{"osd":4,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.246,"15min":1.266},"min":{"1min":0.79200000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.724,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3049999999999999,"15min":1.3380000000000001},"min":{"1min":0.97999999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.571,"5min":1.8939999999999999,"15min":2.2200000000000002},"last":1.24}]},{"osd":5,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.296,"15min":1.325},"min":{"1min":0.83699999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.532,"5min":1.921,"15min":2.0489999999999999},"last":1.415},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3300000000000001,"15min":1.3759999999999999},"min":{"1min":1.0209999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.873,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3919999999999999,"15min":1.405},"min":{"1min":0.91700000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.032,"5min":2.1909999999999998,"15min":2.5600000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.349,"15min":1.3899999999999999},"min":{"1min":0.92600000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6060000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.4830000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3999999999999999,"15min":1.4279999999999999},"min":{"1min":0.96499999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7969999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.444,"15min":1.46},"min":{"1min":0.85799999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8140000000000001,"5min":2.1059999999999999,"15min":2.2759999999999998},"last":1.1140000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:53:10.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:53:10.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.494+0000 7f6d66925640 1 Processor -- start 2023-10-08T05:53:10.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.494+0000 7f6d66925640 1 -- start start 2023-10-08T05:53:10.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.496+0000 7f6d66925640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d600ad790 0x7f6d600702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:10.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.496+0000 7f6d66925640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6d6015e330 0x7f6d60070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:10.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.496+0000 7f6d66925640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6006dee0 con 0x7f6d60159ac0 2023-10-08T05:53:10.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.496+0000 7f6d66925640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6006dcf0 con 0x7f6d6015e330 2023-10-08T05:53:10.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.496+0000 7f6d66925640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6015d850 con 0x7f6d600ad790 2023-10-08T05:53:10.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.496+0000 7f6d5ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6d60159ac0 0x7f6d6006d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37106/0 (socket says 172.21.15.70:37106) 2023-10-08T05:53:10.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.497+0000 7f6d5ffff640 1 -- 172.21.15.70:0/2931701010 learned_addr learned my addr 172.21.15.70:0/2931701010 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:10.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.497+0000 7f6d64ec2640 1 --2- 172.21.15.70:0/2931701010 >> v2:172.21.15.70:3300/0 conn(0x7f6d6015e330 0x7f6d60070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:10.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.497+0000 7f6d57fff640 1 --2- 172.21.15.70:0/2931701010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d600ad790 0x7f6d600702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:10.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.497+0000 7f6d64ec2640 1 -- 172.21.15.70:0/2931701010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d600ad790 msgr2=0x7f6d600702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:10.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.497+0000 7f6d64ec2640 1 --2- 172.21.15.70:0/2931701010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d600ad790 0x7f6d600702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.497+0000 7f6d64ec2640 1 -- 172.21.15.70:0/2931701010 >> v1:172.21.15.70:6789/0 conn(0x7f6d60159ac0 legacy=0x7f6d6006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:10.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.497+0000 7f6d64ec2640 1 -- 172.21.15.70:0/2931701010 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6015db30 con 0x7f6d6015e330 2023-10-08T05:53:10.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.498+0000 7f6d64ec2640 1 --2- 172.21.15.70:0/2931701010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 0x7f6d60070810 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009d90 tx=0x7f6d5802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5411aa7f57917cc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:10.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.499+0000 7f6d5f7fe640 1 -- 172.21.15.70:0/2931701010 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5802cc60 con 0x7f6d6015e330 2023-10-08T05:53:10.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.499+0000 7f6d5f7fe640 1 -- 172.21.15.70:0/2931701010 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5802cdc0 con 0x7f6d6015e330 2023-10-08T05:53:10.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.499+0000 7f6d5f7fe640 1 -- 172.21.15.70:0/2931701010 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58034660 con 0x7f6d6015e330 2023-10-08T05:53:10.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.499+0000 7f6d66925640 1 -- 172.21.15.70:0/2931701010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 msgr2=0x7f6d60070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:10.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.499+0000 7f6d66925640 1 --2- 172.21.15.70:0/2931701010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 0x7f6d60070810 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009d90 tx=0x7f6d5802beb0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.499+0000 7f6d66925640 1 -- 172.21.15.70:0/2931701010 shutdown_connections 2023-10-08T05:53:10.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.499+0000 7f6d66925640 1 --2- 172.21.15.70:0/2931701010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 0x7f6d60070810 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.499+0000 7f6d66925640 1 --2- 172.21.15.70:0/2931701010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d600ad790 0x7f6d600702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.499+0000 7f6d66925640 1 -- 172.21.15.70:0/2931701010 >> 172.21.15.70:0/2931701010 conn(0x7f6d6006a380 msgr2=0x7f6d600b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:10.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.499+0000 7f6d66925640 1 -- 172.21.15.70:0/2931701010 shutdown_connections 2023-10-08T05:53:10.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.500+0000 7f6d66925640 1 -- 172.21.15.70:0/2931701010 wait complete. 2023-10-08T05:53:10.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.501+0000 7f6d66925640 1 Processor -- start 2023-10-08T05:53:10.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.501+0000 7f6d66925640 1 -- start start 2023-10-08T05:53:10.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.502+0000 7f6d66925640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d600ad790 0x7f6d60152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:10.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.502+0000 7f6d66925640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d60159ac0 0x7f6d600cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:10.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.502+0000 7f6d66925640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 0x7f6d600cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:10.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.502+0000 7f6d66925640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d601496b0 con 0x7f6d6015e330 2023-10-08T05:53:10.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.502+0000 7f6d66925640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d60158f30 con 0x7f6d600ad790 2023-10-08T05:53:10.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.502+0000 7f6d66925640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d60159210 con 0x7f6d60159ac0 2023-10-08T05:53:10.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.503+0000 7f6d64ec2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 0x7f6d600cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:10.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.503+0000 7f6d64ec2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 0x7f6d600cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46678/0 (socket says 172.21.15.70:46678) 2023-10-08T05:53:10.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.503+0000 7f6d64ec2640 1 -- 172.21.15.70:0/1946639160 learned_addr learned my addr 172.21.15.70:0/1946639160 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:10.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.503+0000 7f6d57fff640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d60159ac0 0x7f6d600cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:10.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.503+0000 7f6d64ec2640 1 -- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d60159ac0 msgr2=0x7f6d600cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:10.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.503+0000 7f6d64ec2640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d60159ac0 0x7f6d600cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.503+0000 7f6d64ec2640 1 -- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d600ad790 msgr2=0x7f6d60152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:10.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.503+0000 7f6d64ec2640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d600ad790 0x7f6d60152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.503+0000 7f6d64ec2640 1 -- 172.21.15.70:0/1946639160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d58009a70 con 0x7f6d6015e330 2023-10-08T05:53:10.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.504+0000 7f6d64ec2640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 0x7f6d600cbab0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009d90 tx=0x7f6d58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:10.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.504+0000 7f6d5d7fa640 1 -- 172.21.15.70:0/1946639160 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58040650 con 0x7f6d6015e330 2023-10-08T05:53:10.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.504+0000 7f6d5d7fa640 1 -- 172.21.15.70:0/1946639160 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d580407d0 con 0x7f6d6015e330 2023-10-08T05:53:10.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.504+0000 7f6d66925640 1 -- 172.21.15.70:0/1946639160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d60151a40 con 0x7f6d6015e330 2023-10-08T05:53:10.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.504+0000 7f6d5d7fa640 1 -- 172.21.15.70:0/1946639160 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58040b80 con 0x7f6d6015e330 2023-10-08T05:53:10.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.504+0000 7f6d66925640 1 -- 172.21.15.70:0/1946639160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d60152000 con 0x7f6d6015e330 2023-10-08T05:53:10.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.505+0000 7f6d66925640 1 -- 172.21.15.70:0/1946639160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d60067490 con 0x7f6d6015e330 2023-10-08T05:53:10.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.506+0000 7f6d5d7fa640 1 -- 172.21.15.70:0/1946639160 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6d58040ce0 con 0x7f6d6015e330 2023-10-08T05:53:10.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.507+0000 7f6d5d7fa640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6d4006af00 0x7f6d4006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:10.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.507+0000 7f6d5d7fa640 1 -- 172.21.15.70:0/1946639160 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6d580ac6c0 con 0x7f6d6015e330 2023-10-08T05:53:10.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.508+0000 7f6d5ffff640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6d4006af00 0x7f6d4006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:10.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.509+0000 7f6d5ffff640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6d4006af00 0x7f6d4006d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d601f5d00 tx=0x7f6d50007a40 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:10.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.511+0000 7f6d5d7fa640 1 -- 172.21.15.70:0/1946639160 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d58077510 con 0x7f6d6015e330 2023-10-08T05:53:10.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.705+0000 7f6d66925640 1 -- 172.21.15.70:0/1946639160 --> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d60159ea0 con 0x7f6d4006af00 2023-10-08T05:53:10.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.729+0000 7f6d5d7fa640 1 -- 172.21.15.70:0/1946639160 <== mgr.5737 v2:172.21.15.181:6832/2060337708 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286930 (secure 0 0 0) 0x7f6d500058b0 con 0x7f6d4006af00 2023-10-08T05:53:10.734 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:10.736 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:53:10.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.735+0000 7f6d66925640 1 -- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6d4006af00 msgr2=0x7f6d4006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:10.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.735+0000 7f6d66925640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6d4006af00 0x7f6d4006d3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d601f5d00 tx=0x7f6d50007a40 comp rx=0 tx=0).stop 2023-10-08T05:53:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.735+0000 7f6d66925640 1 -- 172.21.15.70:0/1946639160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 msgr2=0x7f6d600cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.735+0000 7f6d66925640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 0x7f6d600cbab0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009d90 tx=0x7f6d58004a40 comp rx=0 tx=0).stop 2023-10-08T05:53:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.736+0000 7f6d66925640 1 -- 172.21.15.70:0/1946639160 shutdown_connections 2023-10-08T05:53:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.736+0000 7f6d66925640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6d4006af00 0x7f6d4006d3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.736+0000 7f6d66925640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d6015e330 0x7f6d600cbab0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.736+0000 7f6d66925640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d60159ac0 0x7f6d600cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.736+0000 7f6d66925640 1 --2- 172.21.15.70:0/1946639160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d600ad790 0x7f6d60152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.736+0000 7f6d66925640 1 -- 172.21.15.70:0/1946639160 >> 172.21.15.70:0/1946639160 conn(0x7f6d6006a380 msgr2=0x7f6d6014d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.736+0000 7f6d66925640 1 -- 172.21.15.70:0/1946639160 shutdown_connections 2023-10-08T05:53:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:10.736+0000 7f6d66925640 1 -- 172.21.15.70:0/1946639160 wait complete. 2023-10-08T05:53:10.756 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-08T05:53:10.609892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":550,"num_read_kb":1032,"num_write":289,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2196,"ondisk_log_size":2196,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":313316,"kb_used_data":11128,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754661404,"statfs":{"total":773094113280,"available":772773277696,"internally_reserved":0,"allocated":11395072,"data_stored":6313652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":309317179},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002172"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803698+0000","last_change":"2023-10-08T05:53:05.804151+0000","last_active":"2023-10-08T05:53:06.803698+0000","last_peered":"2023-10-08T05:53:06.803698+0000","last_clean":"2023-10-08T05:53:06.803698+0000","last_became_active":"2023-10-08T05:53:05.803842+0000","last_became_peered":"2023-10-08T05:53:05.803842+0000","last_unstale":"2023-10-08T05:53:06.803698+0000","last_undegraded":"2023-10-08T05:53:06.803698+0000","last_fullsized":"2023-10-08T05:53:06.803698+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:23.414780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793874+0000","last_change":"2023-10-08T05:53:05.796317+0000","last_active":"2023-10-08T05:53:06.793874+0000","last_peered":"2023-10-08T05:53:06.793874+0000","last_clean":"2023-10-08T05:53:06.793874+0000","last_became_active":"2023-10-08T05:53:05.796020+0000","last_became_peered":"2023-10-08T05:53:05.796020+0000","last_unstale":"2023-10-08T05:53:06.793874+0000","last_undegraded":"2023-10-08T05:53:06.793874+0000","last_fullsized":"2023-10-08T05:53:06.793874+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:33.604099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.273901+0000","last_change":"2023-10-08T05:53:05.798782+0000","last_active":"2023-10-08T05:53:07.273901+0000","last_peered":"2023-10-08T05:53:07.273901+0000","last_clean":"2023-10-08T05:53:07.273901+0000","last_became_active":"2023-10-08T05:53:05.798516+0000","last_became_peered":"2023-10-08T05:53:05.798516+0000","last_unstale":"2023-10-08T05:53:07.273901+0000","last_undegraded":"2023-10-08T05:53:07.273901+0000","last_fullsized":"2023-10-08T05:53:07.273901+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:17.434212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271380+0000","last_change":"2023-10-08T05:53:05.798713+0000","last_active":"2023-10-08T05:53:07.271380+0000","last_peered":"2023-10-08T05:53:07.271380+0000","last_clean":"2023-10-08T05:53:07.271380+0000","last_became_active":"2023-10-08T05:53:05.798445+0000","last_became_peered":"2023-10-08T05:53:05.798445+0000","last_unstale":"2023-10-08T05:53:07.271380+0000","last_undegraded":"2023-10-08T05:53:07.271380+0000","last_fullsized":"2023-10-08T05:53:07.271380+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:02.485242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271579+0000","last_change":"2023-10-08T05:53:05.803570+0000","last_active":"2023-10-08T05:53:07.271579+0000","last_peered":"2023-10-08T05:53:07.271579+0000","last_clean":"2023-10-08T05:53:07.271579+0000","last_became_active":"2023-10-08T05:53:05.802131+0000","last_became_peered":"2023-10-08T05:53:05.802131+0000","last_unstale":"2023-10-08T05:53:07.271579+0000","last_undegraded":"2023-10-08T05:53:07.271579+0000","last_fullsized":"2023-10-08T05:53:07.271579+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:47:41.201385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.273334+0000","last_change":"2023-10-08T05:53:05.804643+0000","last_active":"2023-10-08T05:53:07.273334+0000","last_peered":"2023-10-08T05:53:07.273334+0000","last_clean":"2023-10-08T05:53:07.273334+0000","last_became_active":"2023-10-08T05:53:05.804353+0000","last_became_peered":"2023-10-08T05:53:05.804353+0000","last_unstale":"2023-10-08T05:53:07.273334+0000","last_undegraded":"2023-10-08T05:53:07.273334+0000","last_fullsized":"2023-10-08T05:53:07.273334+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:48.735927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.799840+0000","last_change":"2023-10-08T05:53:05.790851+0000","last_active":"2023-10-08T05:53:06.799840+0000","last_peered":"2023-10-08T05:53:06.799840+0000","last_clean":"2023-10-08T05:53:06.799840+0000","last_became_active":"2023-10-08T05:53:05.790473+0000","last_became_peered":"2023-10-08T05:53:05.790473+0000","last_unstale":"2023-10-08T05:53:06.799840+0000","last_undegraded":"2023-10-08T05:53:06.799840+0000","last_fullsized":"2023-10-08T05:53:06.799840+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:42:52.376261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271097+0000","last_change":"2023-10-08T05:53:05.796060+0000","last_active":"2023-10-08T05:53:07.271097+0000","last_peered":"2023-10-08T05:53:07.271097+0000","last_clean":"2023-10-08T05:53:07.271097+0000","last_became_active":"2023-10-08T05:53:05.795791+0000","last_became_peered":"2023-10-08T05:53:05.795791+0000","last_unstale":"2023-10-08T05:53:07.271097+0000","last_undegraded":"2023-10-08T05:53:07.271097+0000","last_fullsized":"2023-10-08T05:53:07.271097+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:53.249320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800993+0000","last_change":"2023-10-08T05:53:05.791575+0000","last_active":"2023-10-08T05:53:06.800993+0000","last_peered":"2023-10-08T05:53:06.800993+0000","last_clean":"2023-10-08T05:53:06.800993+0000","last_became_active":"2023-10-08T05:53:05.790545+0000","last_became_peered":"2023-10-08T05:53:05.790545+0000","last_unstale":"2023-10-08T05:53:06.800993+0000","last_undegraded":"2023-10-08T05:53:06.800993+0000","last_fullsized":"2023-10-08T05:53:06.800993+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:16.125323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271699+0000","last_change":"2023-10-08T05:53:05.802819+0000","last_active":"2023-10-08T05:53:07.271699+0000","last_peered":"2023-10-08T05:53:07.271699+0000","last_clean":"2023-10-08T05:53:07.271699+0000","last_became_active":"2023-10-08T05:53:05.802521+0000","last_became_peered":"2023-10-08T05:53:05.802521+0000","last_unstale":"2023-10-08T05:53:07.271699+0000","last_undegraded":"2023-10-08T05:53:07.271699+0000","last_fullsized":"2023-10-08T05:53:07.271699+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:31.090058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793753+0000","last_change":"2023-10-08T05:53:05.802108+0000","last_active":"2023-10-08T05:53:06.793753+0000","last_peered":"2023-10-08T05:53:06.793753+0000","last_clean":"2023-10-08T05:53:06.793753+0000","last_became_active":"2023-10-08T05:53:05.801691+0000","last_became_peered":"2023-10-08T05:53:05.801691+0000","last_unstale":"2023-10-08T05:53:06.793753+0000","last_undegraded":"2023-10-08T05:53:06.793753+0000","last_fullsized":"2023-10-08T05:53:06.793753+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:18.113815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271466+0000","last_change":"2023-10-08T05:53:05.796469+0000","last_active":"2023-10-08T05:53:07.271466+0000","last_peered":"2023-10-08T05:53:07.271466+0000","last_clean":"2023-10-08T05:53:07.271466+0000","last_became_active":"2023-10-08T05:53:05.796129+0000","last_became_peered":"2023-10-08T05:53:05.796129+0000","last_unstale":"2023-10-08T05:53:07.271466+0000","last_undegraded":"2023-10-08T05:53:07.271466+0000","last_fullsized":"2023-10-08T05:53:07.271466+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:23.414780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271925+0000","last_change":"2023-10-08T05:53:05.809944+0000","last_active":"2023-10-08T05:53:07.271925+0000","last_peered":"2023-10-08T05:53:07.271925+0000","last_clean":"2023-10-08T05:53:07.271925+0000","last_became_active":"2023-10-08T05:53:05.809675+0000","last_became_peered":"2023-10-08T05:53:05.809675+0000","last_unstale":"2023-10-08T05:53:07.271925+0000","last_undegraded":"2023-10-08T05:53:07.271925+0000","last_fullsized":"2023-10-08T05:53:07.271925+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:56.522301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.273718+0000","last_change":"2023-10-08T05:53:05.804069+0000","last_active":"2023-10-08T05:53:07.273718+0000","last_peered":"2023-10-08T05:53:07.273718+0000","last_clean":"2023-10-08T05:53:07.273718+0000","last_became_active":"2023-10-08T05:53:05.800052+0000","last_became_peered":"2023-10-08T05:53:05.800052+0000","last_unstale":"2023-10-08T05:53:07.273718+0000","last_undegraded":"2023-10-08T05:53:07.273718+0000","last_fullsized":"2023-10-08T05:53:07.273718+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:14.797057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793235+0000","last_change":"2023-10-08T05:53:05.790331+0000","last_active":"2023-10-08T05:53:06.793235+0000","last_peered":"2023-10-08T05:53:06.793235+0000","last_clean":"2023-10-08T05:53:06.793235+0000","last_became_active":"2023-10-08T05:53:05.789747+0000","last_became_peered":"2023-10-08T05:53:05.789747+0000","last_unstale":"2023-10-08T05:53:06.793235+0000","last_undegraded":"2023-10-08T05:53:06.793235+0000","last_fullsized":"2023-10-08T05:53:06.793235+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:32.287095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802442+0000","last_change":"2023-10-08T05:53:05.805853+0000","last_active":"2023-10-08T05:53:06.802442+0000","last_peered":"2023-10-08T05:53:06.802442+0000","last_clean":"2023-10-08T05:53:06.802442+0000","last_became_active":"2023-10-08T05:53:05.805566+0000","last_became_peered":"2023-10-08T05:53:05.805566+0000","last_unstale":"2023-10-08T05:53:06.802442+0000","last_undegraded":"2023-10-08T05:53:06.802442+0000","last_fullsized":"2023-10-08T05:53:06.802442+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:14.811048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.275815+0000","last_change":"2023-10-08T05:53:05.795160+0000","last_active":"2023-10-08T05:53:07.275815+0000","last_peered":"2023-10-08T05:53:07.275815+0000","last_clean":"2023-10-08T05:53:07.275815+0000","last_became_active":"2023-10-08T05:53:05.794693+0000","last_became_peered":"2023-10-08T05:53:05.794693+0000","last_unstale":"2023-10-08T05:53:07.275815+0000","last_undegraded":"2023-10-08T05:53:07.275815+0000","last_fullsized":"2023-10-08T05:53:07.275815+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:11:50.876086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274412+0000","last_change":"2023-10-08T05:53:05.794406+0000","last_active":"2023-10-08T05:53:07.274412+0000","last_peered":"2023-10-08T05:53:07.274412+0000","last_clean":"2023-10-08T05:53:07.274412+0000","last_became_active":"2023-10-08T05:53:05.794135+0000","last_became_peered":"2023-10-08T05:53:05.794135+0000","last_unstale":"2023-10-08T05:53:07.274412+0000","last_undegraded":"2023-10-08T05:53:07.274412+0000","last_fullsized":"2023-10-08T05:53:07.274412+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:44.413602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.273588+0000","last_change":"2023-10-08T05:53:05.804227+0000","last_active":"2023-10-08T05:53:07.273588+0000","last_peered":"2023-10-08T05:53:07.273588+0000","last_clean":"2023-10-08T05:53:07.273588+0000","last_became_active":"2023-10-08T05:53:05.803951+0000","last_became_peered":"2023-10-08T05:53:05.803951+0000","last_unstale":"2023-10-08T05:53:07.273588+0000","last_undegraded":"2023-10-08T05:53:07.273588+0000","last_fullsized":"2023-10-08T05:53:07.273588+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:31.090058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272630+0000","last_change":"2023-10-08T05:53:05.791220+0000","last_active":"2023-10-08T05:53:07.272630+0000","last_peered":"2023-10-08T05:53:07.272630+0000","last_clean":"2023-10-08T05:53:07.272630+0000","last_became_active":"2023-10-08T05:53:05.789390+0000","last_became_peered":"2023-10-08T05:53:05.789390+0000","last_unstale":"2023-10-08T05:53:07.272630+0000","last_undegraded":"2023-10-08T05:53:07.272630+0000","last_fullsized":"2023-10-08T05:53:07.272630+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:42.467508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.799329+0000","last_change":"2023-10-08T05:53:05.801975+0000","last_active":"2023-10-08T05:53:06.799329+0000","last_peered":"2023-10-08T05:53:06.799329+0000","last_clean":"2023-10-08T05:53:06.799329+0000","last_became_active":"2023-10-08T05:53:05.801648+0000","last_became_peered":"2023-10-08T05:53:05.801648+0000","last_unstale":"2023-10-08T05:53:06.799329+0000","last_undegraded":"2023-10-08T05:53:06.799329+0000","last_fullsized":"2023-10-08T05:53:06.799329+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:29:08.807994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271775+0000","last_change":"2023-10-08T05:53:05.810741+0000","last_active":"2023-10-08T05:53:07.271775+0000","last_peered":"2023-10-08T05:53:07.271775+0000","last_clean":"2023-10-08T05:53:07.271775+0000","last_became_active":"2023-10-08T05:53:05.810403+0000","last_became_peered":"2023-10-08T05:53:05.810403+0000","last_unstale":"2023-10-08T05:53:07.271775+0000","last_undegraded":"2023-10-08T05:53:07.271775+0000","last_fullsized":"2023-10-08T05:53:07.271775+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:40.445070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800184+0000","last_change":"2023-10-08T05:53:05.793504+0000","last_active":"2023-10-08T05:53:06.800184+0000","last_peered":"2023-10-08T05:53:06.800184+0000","last_clean":"2023-10-08T05:53:06.800184+0000","last_became_active":"2023-10-08T05:53:05.793223+0000","last_became_peered":"2023-10-08T05:53:05.793223+0000","last_unstale":"2023-10-08T05:53:06.800184+0000","last_undegraded":"2023-10-08T05:53:06.800184+0000","last_fullsized":"2023-10-08T05:53:06.800184+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:49.249431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793188+0000","last_change":"2023-10-08T05:53:05.796449+0000","last_active":"2023-10-08T05:53:06.793188+0000","last_peered":"2023-10-08T05:53:06.793188+0000","last_clean":"2023-10-08T05:53:06.793188+0000","last_became_active":"2023-10-08T05:53:05.796144+0000","last_became_peered":"2023-10-08T05:53:05.796144+0000","last_unstale":"2023-10-08T05:53:06.793188+0000","last_undegraded":"2023-10-08T05:53:06.793188+0000","last_fullsized":"2023-10-08T05:53:06.793188+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.108691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793809+0000","last_change":"2023-10-08T05:53:05.788124+0000","last_active":"2023-10-08T05:53:06.793809+0000","last_peered":"2023-10-08T05:53:06.793809+0000","last_clean":"2023-10-08T05:53:06.793809+0000","last_became_active":"2023-10-08T05:53:05.787390+0000","last_became_peered":"2023-10-08T05:53:05.787390+0000","last_unstale":"2023-10-08T05:53:06.793809+0000","last_undegraded":"2023-10-08T05:53:06.793809+0000","last_fullsized":"2023-10-08T05:53:06.793809+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:28:33.332486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271579+0000","last_change":"2023-10-08T05:53:05.802939+0000","last_active":"2023-10-08T05:53:07.271579+0000","last_peered":"2023-10-08T05:53:07.271579+0000","last_clean":"2023-10-08T05:53:07.271579+0000","last_became_active":"2023-10-08T05:53:05.802651+0000","last_became_peered":"2023-10-08T05:53:05.802651+0000","last_unstale":"2023-10-08T05:53:07.271579+0000","last_undegraded":"2023-10-08T05:53:07.271579+0000","last_fullsized":"2023-10-08T05:53:07.271579+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:48.735927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803151+0000","last_change":"2023-10-08T05:53:05.807790+0000","last_active":"2023-10-08T05:53:06.803151+0000","last_peered":"2023-10-08T05:53:06.803151+0000","last_clean":"2023-10-08T05:53:06.803151+0000","last_became_active":"2023-10-08T05:53:05.807502+0000","last_became_peered":"2023-10-08T05:53:05.807502+0000","last_unstale":"2023-10-08T05:53:06.803151+0000","last_undegraded":"2023-10-08T05:53:06.803151+0000","last_fullsized":"2023-10-08T05:53:06.803151+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:35.349762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272953+0000","last_change":"2023-10-08T05:53:05.793705+0000","last_active":"2023-10-08T05:53:07.272953+0000","last_peered":"2023-10-08T05:53:07.272953+0000","last_clean":"2023-10-08T05:53:07.272953+0000","last_became_active":"2023-10-08T05:53:05.793355+0000","last_became_peered":"2023-10-08T05:53:05.793355+0000","last_unstale":"2023-10-08T05:53:07.272953+0000","last_undegraded":"2023-10-08T05:53:07.272953+0000","last_fullsized":"2023-10-08T05:53:07.272953+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:39.356767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793493+0000","last_change":"2023-10-08T05:53:05.784136+0000","last_active":"2023-10-08T05:53:06.793493+0000","last_peered":"2023-10-08T05:53:06.793493+0000","last_clean":"2023-10-08T05:53:06.793493+0000","last_became_active":"2023-10-08T05:53:05.783763+0000","last_became_peered":"2023-10-08T05:53:05.783763+0000","last_unstale":"2023-10-08T05:53:06.793493+0000","last_undegraded":"2023-10-08T05:53:06.793493+0000","last_fullsized":"2023-10-08T05:53:06.793493+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:06:23.168404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800210+0000","last_change":"2023-10-08T05:53:05.813374+0000","last_active":"2023-10-08T05:53:06.800210+0000","last_peered":"2023-10-08T05:53:06.800210+0000","last_clean":"2023-10-08T05:53:06.800210+0000","last_became_active":"2023-10-08T05:53:05.813074+0000","last_became_peered":"2023-10-08T05:53:05.813074+0000","last_unstale":"2023-10-08T05:53:06.800210+0000","last_undegraded":"2023-10-08T05:53:06.800210+0000","last_fullsized":"2023-10-08T05:53:06.800210+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:42:45.243485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272482+0000","last_change":"2023-10-08T05:53:05.785403+0000","last_active":"2023-10-08T05:53:07.272482+0000","last_peered":"2023-10-08T05:53:07.272482+0000","last_clean":"2023-10-08T05:53:07.272482+0000","last_became_active":"2023-10-08T05:53:05.785109+0000","last_became_peered":"2023-10-08T05:53:05.785109+0000","last_unstale":"2023-10-08T05:53:07.272482+0000","last_undegraded":"2023-10-08T05:53:07.272482+0000","last_fullsized":"2023-10-08T05:53:07.272482+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:15.369645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.801135+0000","last_change":"2023-10-08T05:53:05.805691+0000","last_active":"2023-10-08T05:53:06.801135+0000","last_peered":"2023-10-08T05:53:06.801135+0000","last_clean":"2023-10-08T05:53:06.801135+0000","last_became_active":"2023-10-08T05:53:05.805385+0000","last_became_peered":"2023-10-08T05:53:05.805385+0000","last_unstale":"2023-10-08T05:53:06.801135+0000","last_undegraded":"2023-10-08T05:53:06.801135+0000","last_fullsized":"2023-10-08T05:53:06.801135+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:44.044510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271679+0000","last_change":"2023-10-08T05:53:05.792769+0000","last_active":"2023-10-08T05:53:07.271679+0000","last_peered":"2023-10-08T05:53:07.271679+0000","last_clean":"2023-10-08T05:53:07.271679+0000","last_became_active":"2023-10-08T05:53:05.792496+0000","last_became_peered":"2023-10-08T05:53:05.792496+0000","last_unstale":"2023-10-08T05:53:07.271679+0000","last_undegraded":"2023-10-08T05:53:07.271679+0000","last_fullsized":"2023-10-08T05:53:07.271679+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:56.938089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"103'4","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.275279+0000","last_change":"2023-10-08T05:53:04.770249+0000","last_active":"2023-10-08T05:53:07.275279+0000","last_peered":"2023-10-08T05:53:07.275279+0000","last_clean":"2023-10-08T05:53:07.275279+0000","last_became_active":"2023-10-08T05:53:04.769944+0000","last_became_peered":"2023-10-08T05:53:04.769944+0000","last_unstale":"2023-10-08T05:53:07.275279+0000","last_undegraded":"2023-10-08T05:53:07.275279+0000","last_fullsized":"2023-10-08T05:53:07.275279+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:27:33.897876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793397+0000","last_change":"2023-10-08T05:53:05.796757+0000","last_active":"2023-10-08T05:53:06.793397+0000","last_peered":"2023-10-08T05:53:06.793397+0000","last_clean":"2023-10-08T05:53:06.793397+0000","last_became_active":"2023-10-08T05:53:05.796399+0000","last_became_peered":"2023-10-08T05:53:05.796399+0000","last_unstale":"2023-10-08T05:53:06.793397+0000","last_undegraded":"2023-10-08T05:53:06.793397+0000","last_fullsized":"2023-10-08T05:53:06.793397+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:54.833969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"103'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802834+0000","last_change":"2023-10-08T05:53:04.775265+0000","last_active":"2023-10-08T05:53:06.802834+0000","last_peered":"2023-10-08T05:53:06.802834+0000","last_clean":"2023-10-08T05:53:06.802834+0000","last_became_active":"2023-10-08T05:53:04.774915+0000","last_became_peered":"2023-10-08T05:53:04.774915+0000","last_unstale":"2023-10-08T05:53:06.802834+0000","last_undegraded":"2023-10-08T05:53:06.802834+0000","last_fullsized":"2023-10-08T05:53:06.802834+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:45.638271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800983+0000","last_change":"2023-10-08T05:53:05.792336+0000","last_active":"2023-10-08T05:53:06.800983+0000","last_peered":"2023-10-08T05:53:06.800983+0000","last_clean":"2023-10-08T05:53:06.800983+0000","last_became_active":"2023-10-08T05:53:05.791137+0000","last_became_peered":"2023-10-08T05:53:05.791137+0000","last_unstale":"2023-10-08T05:53:06.800983+0000","last_undegraded":"2023-10-08T05:53:06.800983+0000","last_fullsized":"2023-10-08T05:53:06.800983+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:49.217386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803112+0000","last_change":"2023-10-08T05:53:04.776791+0000","last_active":"2023-10-08T05:53:06.803112+0000","last_peered":"2023-10-08T05:53:06.803112+0000","last_clean":"2023-10-08T05:53:06.803112+0000","last_became_active":"2023-10-08T05:53:04.776525+0000","last_became_peered":"2023-10-08T05:53:04.776525+0000","last_unstale":"2023-10-08T05:53:06.803112+0000","last_undegraded":"2023-10-08T05:53:06.803112+0000","last_fullsized":"2023-10-08T05:53:06.803112+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:02:42.286157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802668+0000","last_change":"2023-10-08T05:53:05.801465+0000","last_active":"2023-10-08T05:53:06.802668+0000","last_peered":"2023-10-08T05:53:06.802668+0000","last_clean":"2023-10-08T05:53:06.802668+0000","last_became_active":"2023-10-08T05:53:05.801162+0000","last_became_peered":"2023-10-08T05:53:05.801162+0000","last_unstale":"2023-10-08T05:53:06.802668+0000","last_undegraded":"2023-10-08T05:53:06.802668+0000","last_fullsized":"2023-10-08T05:53:06.802668+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:38:59.232443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.799922+0000","last_change":"2023-10-08T05:53:04.767204+0000","last_active":"2023-10-08T05:53:06.799922+0000","last_peered":"2023-10-08T05:53:06.799922+0000","last_clean":"2023-10-08T05:53:06.799922+0000","last_became_active":"2023-10-08T05:53:04.766876+0000","last_became_peered":"2023-10-08T05:53:04.766876+0000","last_unstale":"2023-10-08T05:53:06.799922+0000","last_undegraded":"2023-10-08T05:53:06.799922+0000","last_fullsized":"2023-10-08T05:53:06.799922+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:17:46.822177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793833+0000","last_change":"2023-10-08T05:53:05.792920+0000","last_active":"2023-10-08T05:53:06.793833+0000","last_peered":"2023-10-08T05:53:06.793833+0000","last_clean":"2023-10-08T05:53:06.793833+0000","last_became_active":"2023-10-08T05:53:05.792647+0000","last_became_peered":"2023-10-08T05:53:05.792647+0000","last_unstale":"2023-10-08T05:53:06.793833+0000","last_undegraded":"2023-10-08T05:53:06.793833+0000","last_fullsized":"2023-10-08T05:53:06.793833+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:13:56.109947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"103'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802741+0000","last_change":"2023-10-08T05:53:04.767204+0000","last_active":"2023-10-08T05:53:06.802741+0000","last_peered":"2023-10-08T05:53:06.802741+0000","last_clean":"2023-10-08T05:53:06.802741+0000","last_became_active":"2023-10-08T05:53:04.766888+0000","last_became_peered":"2023-10-08T05:53:04.766888+0000","last_unstale":"2023-10-08T05:53:06.802741+0000","last_undegraded":"2023-10-08T05:53:06.802741+0000","last_fullsized":"2023-10-08T05:53:06.802741+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:22.214955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793878+0000","last_change":"2023-10-08T05:53:05.801287+0000","last_active":"2023-10-08T05:53:06.793878+0000","last_peered":"2023-10-08T05:53:06.793878+0000","last_clean":"2023-10-08T05:53:06.793878+0000","last_became_active":"2023-10-08T05:53:05.801009+0000","last_became_peered":"2023-10-08T05:53:05.801009+0000","last_unstale":"2023-10-08T05:53:06.793878+0000","last_undegraded":"2023-10-08T05:53:06.793878+0000","last_fullsized":"2023-10-08T05:53:06.793878+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:54.138563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"103'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271056+0000","last_change":"2023-10-08T05:53:04.766762+0000","last_active":"2023-10-08T05:53:07.271056+0000","last_peered":"2023-10-08T05:53:07.271056+0000","last_clean":"2023-10-08T05:53:07.271056+0000","last_became_active":"2023-10-08T05:53:04.766473+0000","last_became_peered":"2023-10-08T05:53:04.766473+0000","last_unstale":"2023-10-08T05:53:07.271056+0000","last_undegraded":"2023-10-08T05:53:07.271056+0000","last_fullsized":"2023-10-08T05:53:07.271056+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:59.014929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.801976+0000","last_change":"2023-10-08T05:53:05.790331+0000","last_active":"2023-10-08T05:53:06.801976+0000","last_peered":"2023-10-08T05:53:06.801976+0000","last_clean":"2023-10-08T05:53:06.801976+0000","last_became_active":"2023-10-08T05:53:05.789869+0000","last_became_peered":"2023-10-08T05:53:05.789869+0000","last_unstale":"2023-10-08T05:53:06.801976+0000","last_undegraded":"2023-10-08T05:53:06.801976+0000","last_fullsized":"2023-10-08T05:53:06.801976+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:42:59.809635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793545+0000","last_change":"2023-10-08T05:53:04.768193+0000","last_active":"2023-10-08T05:53:06.793545+0000","last_peered":"2023-10-08T05:53:06.793545+0000","last_clean":"2023-10-08T05:53:06.793545+0000","last_became_active":"2023-10-08T05:53:04.767487+0000","last_became_peered":"2023-10-08T05:53:04.767487+0000","last_unstale":"2023-10-08T05:53:06.793545+0000","last_undegraded":"2023-10-08T05:53:06.793545+0000","last_fullsized":"2023-10-08T05:53:06.793545+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:05:11.946222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.801631+0000","last_change":"2023-10-08T05:53:05.794277+0000","last_active":"2023-10-08T05:53:06.801631+0000","last_peered":"2023-10-08T05:53:06.801631+0000","last_clean":"2023-10-08T05:53:06.801631+0000","last_became_active":"2023-10-08T05:53:05.793926+0000","last_became_peered":"2023-10-08T05:53:05.793926+0000","last_unstale":"2023-10-08T05:53:06.801631+0000","last_undegraded":"2023-10-08T05:53:06.801631+0000","last_fullsized":"2023-10-08T05:53:06.801631+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:33.771627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271115+0000","last_change":"2023-10-08T05:53:04.771324+0000","last_active":"2023-10-08T05:53:07.271115+0000","last_peered":"2023-10-08T05:53:07.271115+0000","last_clean":"2023-10-08T05:53:07.271115+0000","last_became_active":"2023-10-08T05:53:04.771015+0000","last_became_peered":"2023-10-08T05:53:04.771015+0000","last_unstale":"2023-10-08T05:53:07.271115+0000","last_undegraded":"2023-10-08T05:53:07.271115+0000","last_fullsized":"2023-10-08T05:53:07.271115+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:28.184091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793187+0000","last_change":"2023-10-08T05:53:05.796962+0000","last_active":"2023-10-08T05:53:06.793187+0000","last_peered":"2023-10-08T05:53:06.793187+0000","last_clean":"2023-10-08T05:53:06.793187+0000","last_became_active":"2023-10-08T05:53:05.796608+0000","last_became_peered":"2023-10-08T05:53:05.796608+0000","last_unstale":"2023-10-08T05:53:06.793187+0000","last_undegraded":"2023-10-08T05:53:06.793187+0000","last_fullsized":"2023-10-08T05:53:06.793187+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:49.416812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803302+0000","last_change":"2023-10-08T05:53:04.774772+0000","last_active":"2023-10-08T05:53:06.803302+0000","last_peered":"2023-10-08T05:53:06.803302+0000","last_clean":"2023-10-08T05:53:06.803302+0000","last_became_active":"2023-10-08T05:53:04.774479+0000","last_became_peered":"2023-10-08T05:53:04.774479+0000","last_unstale":"2023-10-08T05:53:06.803302+0000","last_undegraded":"2023-10-08T05:53:06.803302+0000","last_fullsized":"2023-10-08T05:53:06.803302+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:13:59.014929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271312+0000","last_change":"2023-10-08T05:53:05.800443+0000","last_active":"2023-10-08T05:53:07.271312+0000","last_peered":"2023-10-08T05:53:07.271312+0000","last_clean":"2023-10-08T05:53:07.271312+0000","last_became_active":"2023-10-08T05:53:05.800145+0000","last_became_peered":"2023-10-08T05:53:05.800145+0000","last_unstale":"2023-10-08T05:53:07.271312+0000","last_undegraded":"2023-10-08T05:53:07.271312+0000","last_fullsized":"2023-10-08T05:53:07.271312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:35.349762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271710+0000","last_change":"2023-10-08T05:53:04.776402+0000","last_active":"2023-10-08T05:53:07.271710+0000","last_peered":"2023-10-08T05:53:07.271710+0000","last_clean":"2023-10-08T05:53:07.271710+0000","last_became_active":"2023-10-08T05:53:04.776120+0000","last_became_peered":"2023-10-08T05:53:04.776120+0000","last_unstale":"2023-10-08T05:53:07.271710+0000","last_undegraded":"2023-10-08T05:53:07.271710+0000","last_fullsized":"2023-10-08T05:53:07.271710+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:13.798415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274122+0000","last_change":"2023-10-08T05:53:05.788107+0000","last_active":"2023-10-08T05:53:07.274122+0000","last_peered":"2023-10-08T05:53:07.274122+0000","last_clean":"2023-10-08T05:53:07.274122+0000","last_became_active":"2023-10-08T05:53:05.787822+0000","last_became_peered":"2023-10-08T05:53:05.787822+0000","last_unstale":"2023-10-08T05:53:07.274122+0000","last_undegraded":"2023-10-08T05:53:07.274122+0000","last_fullsized":"2023-10-08T05:53:07.274122+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:24:17.612355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"103'3","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274136+0000","last_change":"2023-10-08T05:53:04.761003+0000","last_active":"2023-10-08T05:53:07.274136+0000","last_peered":"2023-10-08T05:53:07.274136+0000","last_clean":"2023-10-08T05:53:07.274136+0000","last_became_active":"2023-10-08T05:53:04.760697+0000","last_became_peered":"2023-10-08T05:53:04.760697+0000","last_unstale":"2023-10-08T05:53:07.274136+0000","last_undegraded":"2023-10-08T05:53:07.274136+0000","last_fullsized":"2023-10-08T05:53:07.274136+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:12:57.067239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.273525+0000","last_change":"2023-10-08T05:53:05.793564+0000","last_active":"2023-10-08T05:53:07.273525+0000","last_peered":"2023-10-08T05:53:07.273525+0000","last_clean":"2023-10-08T05:53:07.273525+0000","last_became_active":"2023-10-08T05:53:05.793301+0000","last_became_peered":"2023-10-08T05:53:05.793301+0000","last_unstale":"2023-10-08T05:53:07.273525+0000","last_undegraded":"2023-10-08T05:53:07.273525+0000","last_fullsized":"2023-10-08T05:53:07.273525+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:51.408521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"103'6","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800195+0000","last_change":"2023-10-08T05:53:04.767287+0000","last_active":"2023-10-08T05:53:06.800195+0000","last_peered":"2023-10-08T05:53:06.800195+0000","last_clean":"2023-10-08T05:53:06.800195+0000","last_became_active":"2023-10-08T05:53:04.766938+0000","last_became_peered":"2023-10-08T05:53:04.766938+0000","last_unstale":"2023-10-08T05:53:06.800195+0000","last_undegraded":"2023-10-08T05:53:06.800195+0000","last_fullsized":"2023-10-08T05:53:06.800195+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:12:23.295871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271761+0000","last_change":"2023-10-08T05:53:05.793641+0000","last_active":"2023-10-08T05:53:07.271761+0000","last_peered":"2023-10-08T05:53:07.271761+0000","last_clean":"2023-10-08T05:53:07.271761+0000","last_became_active":"2023-10-08T05:53:05.793002+0000","last_became_peered":"2023-10-08T05:53:05.793002+0000","last_unstale":"2023-10-08T05:53:07.271761+0000","last_undegraded":"2023-10-08T05:53:07.271761+0000","last_fullsized":"2023-10-08T05:53:07.271761+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:14.797057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271054+0000","last_change":"2023-10-08T05:53:04.773332+0000","last_active":"2023-10-08T05:53:07.271054+0000","last_peered":"2023-10-08T05:53:07.271054+0000","last_clean":"2023-10-08T05:53:07.271054+0000","last_became_active":"2023-10-08T05:53:04.773034+0000","last_became_peered":"2023-10-08T05:53:04.773034+0000","last_unstale":"2023-10-08T05:53:07.271054+0000","last_undegraded":"2023-10-08T05:53:07.271054+0000","last_fullsized":"2023-10-08T05:53:07.271054+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:45.638271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800582+0000","last_change":"2023-10-08T05:53:05.786818+0000","last_active":"2023-10-08T05:53:06.800582+0000","last_peered":"2023-10-08T05:53:06.800582+0000","last_clean":"2023-10-08T05:53:06.800582+0000","last_became_active":"2023-10-08T05:53:05.786313+0000","last_became_peered":"2023-10-08T05:53:05.786313+0000","last_unstale":"2023-10-08T05:53:06.800582+0000","last_undegraded":"2023-10-08T05:53:06.800582+0000","last_fullsized":"2023-10-08T05:53:06.800582+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:49.674326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803032+0000","last_change":"2023-10-08T05:53:04.773639+0000","last_active":"2023-10-08T05:53:06.803032+0000","last_peered":"2023-10-08T05:53:06.803032+0000","last_clean":"2023-10-08T05:53:06.803032+0000","last_became_active":"2023-10-08T05:53:04.771979+0000","last_became_peered":"2023-10-08T05:53:04.771979+0000","last_unstale":"2023-10-08T05:53:06.803032+0000","last_undegraded":"2023-10-08T05:53:06.803032+0000","last_fullsized":"2023-10-08T05:53:06.803032+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:44.955390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800009+0000","last_change":"2023-10-08T05:53:05.787263+0000","last_active":"2023-10-08T05:53:06.800009+0000","last_peered":"2023-10-08T05:53:06.800009+0000","last_clean":"2023-10-08T05:53:06.800009+0000","last_became_active":"2023-10-08T05:53:05.786778+0000","last_became_peered":"2023-10-08T05:53:05.786778+0000","last_unstale":"2023-10-08T05:53:06.800009+0000","last_undegraded":"2023-10-08T05:53:06.800009+0000","last_fullsized":"2023-10-08T05:53:06.800009+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:02:39.075379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.275390+0000","last_change":"2023-10-08T05:53:04.768859+0000","last_active":"2023-10-08T05:53:07.275390+0000","last_peered":"2023-10-08T05:53:07.275390+0000","last_clean":"2023-10-08T05:53:07.275390+0000","last_became_active":"2023-10-08T05:53:04.768545+0000","last_became_peered":"2023-10-08T05:53:04.768545+0000","last_unstale":"2023-10-08T05:53:07.275390+0000","last_undegraded":"2023-10-08T05:53:07.275390+0000","last_fullsized":"2023-10-08T05:53:07.275390+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:20:18.077161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802062+0000","last_change":"2023-10-08T05:53:05.790447+0000","last_active":"2023-10-08T05:53:06.802062+0000","last_peered":"2023-10-08T05:53:06.802062+0000","last_clean":"2023-10-08T05:53:06.802062+0000","last_became_active":"2023-10-08T05:53:05.789296+0000","last_became_peered":"2023-10-08T05:53:05.789296+0000","last_unstale":"2023-10-08T05:53:06.802062+0000","last_undegraded":"2023-10-08T05:53:06.802062+0000","last_fullsized":"2023-10-08T05:53:06.802062+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:15.865170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"103'8","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793306+0000","last_change":"2023-10-08T05:53:04.756043+0000","last_active":"2023-10-08T05:53:06.793306+0000","last_peered":"2023-10-08T05:53:06.793306+0000","last_clean":"2023-10-08T05:53:06.793306+0000","last_became_active":"2023-10-08T05:53:04.755754+0000","last_became_peered":"2023-10-08T05:53:04.755754+0000","last_unstale":"2023-10-08T05:53:06.793306+0000","last_undegraded":"2023-10-08T05:53:06.793306+0000","last_fullsized":"2023-10-08T05:53:06.793306+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:26.232188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271834+0000","last_change":"2023-10-08T05:53:05.799933+0000","last_active":"2023-10-08T05:53:07.271834+0000","last_peered":"2023-10-08T05:53:07.271834+0000","last_clean":"2023-10-08T05:53:07.271834+0000","last_became_active":"2023-10-08T05:53:05.799615+0000","last_became_peered":"2023-10-08T05:53:05.799615+0000","last_unstale":"2023-10-08T05:53:07.271834+0000","last_undegraded":"2023-10-08T05:53:07.271834+0000","last_fullsized":"2023-10-08T05:53:07.271834+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:02.485242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"103'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803533+0000","last_change":"2023-10-08T05:53:04.773136+0000","last_active":"2023-10-08T05:53:06.803533+0000","last_peered":"2023-10-08T05:53:06.803533+0000","last_clean":"2023-10-08T05:53:06.803533+0000","last_became_active":"2023-10-08T05:53:04.772843+0000","last_became_peered":"2023-10-08T05:53:04.772843+0000","last_unstale":"2023-10-08T05:53:06.803533+0000","last_undegraded":"2023-10-08T05:53:06.803533+0000","last_fullsized":"2023-10-08T05:53:06.803533+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:59.541711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.275286+0000","last_change":"2023-10-08T05:53:05.795230+0000","last_active":"2023-10-08T05:53:07.275286+0000","last_peered":"2023-10-08T05:53:07.275286+0000","last_clean":"2023-10-08T05:53:07.275286+0000","last_became_active":"2023-10-08T05:53:05.794943+0000","last_became_peered":"2023-10-08T05:53:05.794943+0000","last_unstale":"2023-10-08T05:53:07.275286+0000","last_undegraded":"2023-10-08T05:53:07.275286+0000","last_fullsized":"2023-10-08T05:53:07.275286+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:02.025736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802225+0000","last_change":"2023-10-08T05:53:04.762301+0000","last_active":"2023-10-08T05:53:06.802225+0000","last_peered":"2023-10-08T05:53:06.802225+0000","last_clean":"2023-10-08T05:53:06.802225+0000","last_became_active":"2023-10-08T05:53:04.760836+0000","last_became_peered":"2023-10-08T05:53:04.760836+0000","last_unstale":"2023-10-08T05:53:06.802225+0000","last_undegraded":"2023-10-08T05:53:06.802225+0000","last_fullsized":"2023-10-08T05:53:06.802225+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:43:32.838450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272125+0000","last_change":"2023-10-08T05:53:05.804317+0000","last_active":"2023-10-08T05:53:07.272125+0000","last_peered":"2023-10-08T05:53:07.272125+0000","last_clean":"2023-10-08T05:53:07.272125+0000","last_became_active":"2023-10-08T05:53:05.804041+0000","last_became_peered":"2023-10-08T05:53:05.804041+0000","last_unstale":"2023-10-08T05:53:07.272125+0000","last_undegraded":"2023-10-08T05:53:07.272125+0000","last_fullsized":"2023-10-08T05:53:07.272125+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:32.795428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"103'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274363+0000","last_change":"2023-10-08T05:53:04.770620+0000","last_active":"2023-10-08T05:53:07.274363+0000","last_peered":"2023-10-08T05:53:07.274363+0000","last_clean":"2023-10-08T05:53:07.274363+0000","last_became_active":"2023-10-08T05:53:04.770350+0000","last_became_peered":"2023-10-08T05:53:04.770350+0000","last_unstale":"2023-10-08T05:53:07.274363+0000","last_undegraded":"2023-10-08T05:53:07.274363+0000","last_fullsized":"2023-10-08T05:53:07.274363+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:08.762943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272615+0000","last_change":"2023-10-08T05:53:05.801697+0000","last_active":"2023-10-08T05:53:07.272615+0000","last_peered":"2023-10-08T05:53:07.272615+0000","last_clean":"2023-10-08T05:53:07.272615+0000","last_became_active":"2023-10-08T05:53:05.801427+0000","last_became_peered":"2023-10-08T05:53:05.801427+0000","last_unstale":"2023-10-08T05:53:07.272615+0000","last_undegraded":"2023-10-08T05:53:07.272615+0000","last_fullsized":"2023-10-08T05:53:07.272615+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:23.414780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272945+0000","last_change":"2023-10-08T05:53:04.761736+0000","last_active":"2023-10-08T05:53:07.272945+0000","last_peered":"2023-10-08T05:53:07.272945+0000","last_clean":"2023-10-08T05:53:07.272945+0000","last_became_active":"2023-10-08T05:53:04.761450+0000","last_became_peered":"2023-10-08T05:53:04.761450+0000","last_unstale":"2023-10-08T05:53:07.272945+0000","last_undegraded":"2023-10-08T05:53:07.272945+0000","last_fullsized":"2023-10-08T05:53:07.272945+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:38:58.219810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.794012+0000","last_change":"2023-10-08T05:53:05.794678+0000","last_active":"2023-10-08T05:53:06.794012+0000","last_peered":"2023-10-08T05:53:06.794012+0000","last_clean":"2023-10-08T05:53:06.794012+0000","last_became_active":"2023-10-08T05:53:05.794374+0000","last_became_peered":"2023-10-08T05:53:05.794374+0000","last_unstale":"2023-10-08T05:53:06.794012+0000","last_undegraded":"2023-10-08T05:53:06.794012+0000","last_fullsized":"2023-10-08T05:53:06.794012+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:25.569321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272788+0000","last_change":"2023-10-08T05:53:04.752567+0000","last_active":"2023-10-08T05:53:07.272788+0000","last_peered":"2023-10-08T05:53:07.272788+0000","last_clean":"2023-10-08T05:53:07.272788+0000","last_became_active":"2023-10-08T05:53:04.752160+0000","last_became_peered":"2023-10-08T05:53:04.752160+0000","last_unstale":"2023-10-08T05:53:07.272788+0000","last_undegraded":"2023-10-08T05:53:07.272788+0000","last_fullsized":"2023-10-08T05:53:07.272788+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:27.133706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.794220+0000","last_change":"2023-10-08T05:53:05.795694+0000","last_active":"2023-10-08T05:53:06.794220+0000","last_peered":"2023-10-08T05:53:06.794220+0000","last_clean":"2023-10-08T05:53:06.794220+0000","last_became_active":"2023-10-08T05:53:05.795344+0000","last_became_peered":"2023-10-08T05:53:05.795344+0000","last_unstale":"2023-10-08T05:53:06.794220+0000","last_undegraded":"2023-10-08T05:53:06.794220+0000","last_fullsized":"2023-10-08T05:53:06.794220+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:39:01.800884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272120+0000","last_change":"2023-10-08T05:53:04.766767+0000","last_active":"2023-10-08T05:53:07.272120+0000","last_peered":"2023-10-08T05:53:07.272120+0000","last_clean":"2023-10-08T05:53:07.272120+0000","last_became_active":"2023-10-08T05:53:04.766498+0000","last_became_peered":"2023-10-08T05:53:04.766498+0000","last_unstale":"2023-10-08T05:53:07.272120+0000","last_undegraded":"2023-10-08T05:53:07.272120+0000","last_fullsized":"2023-10-08T05:53:07.272120+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:45.638271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802428+0000","last_change":"2023-10-08T05:53:05.795323+0000","last_active":"2023-10-08T05:53:06.802428+0000","last_peered":"2023-10-08T05:53:06.802428+0000","last_clean":"2023-10-08T05:53:06.802428+0000","last_became_active":"2023-10-08T05:53:05.794959+0000","last_became_peered":"2023-10-08T05:53:05.794959+0000","last_unstale":"2023-10-08T05:53:06.802428+0000","last_undegraded":"2023-10-08T05:53:06.802428+0000","last_fullsized":"2023-10-08T05:53:06.802428+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:58.996874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272214+0000","last_change":"2023-10-08T05:53:04.765996+0000","last_active":"2023-10-08T05:53:07.272214+0000","last_peered":"2023-10-08T05:53:07.272214+0000","last_clean":"2023-10-08T05:53:07.272214+0000","last_became_active":"2023-10-08T05:53:04.765694+0000","last_became_peered":"2023-10-08T05:53:04.765694+0000","last_unstale":"2023-10-08T05:53:07.272214+0000","last_undegraded":"2023-10-08T05:53:07.272214+0000","last_fullsized":"2023-10-08T05:53:07.272214+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:17.491464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.799649+0000","last_change":"2023-10-08T05:53:05.786194+0000","last_active":"2023-10-08T05:53:06.799649+0000","last_peered":"2023-10-08T05:53:06.799649+0000","last_clean":"2023-10-08T05:53:06.799649+0000","last_became_active":"2023-10-08T05:53:05.785790+0000","last_became_peered":"2023-10-08T05:53:05.785790+0000","last_unstale":"2023-10-08T05:53:06.799649+0000","last_undegraded":"2023-10-08T05:53:06.799649+0000","last_fullsized":"2023-10-08T05:53:06.799649+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:09:26.370365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"103'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271601+0000","last_change":"2023-10-08T05:53:04.766900+0000","last_active":"2023-10-08T05:53:07.271601+0000","last_peered":"2023-10-08T05:53:07.271601+0000","last_clean":"2023-10-08T05:53:07.271601+0000","last_became_active":"2023-10-08T05:53:04.766635+0000","last_became_peered":"2023-10-08T05:53:04.766635+0000","last_unstale":"2023-10-08T05:53:07.271601+0000","last_undegraded":"2023-10-08T05:53:07.271601+0000","last_fullsized":"2023-10-08T05:53:07.271601+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:01:46.238991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800228+0000","last_change":"2023-10-08T05:53:05.791929+0000","last_active":"2023-10-08T05:53:06.800228+0000","last_peered":"2023-10-08T05:53:06.800228+0000","last_clean":"2023-10-08T05:53:06.800228+0000","last_became_active":"2023-10-08T05:53:05.791631+0000","last_became_peered":"2023-10-08T05:53:05.791631+0000","last_unstale":"2023-10-08T05:53:06.800228+0000","last_undegraded":"2023-10-08T05:53:06.800228+0000","last_fullsized":"2023-10-08T05:53:06.800228+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:30.265275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.794158+0000","last_change":"2023-10-08T05:53:04.763353+0000","last_active":"2023-10-08T05:53:06.794158+0000","last_peered":"2023-10-08T05:53:06.794158+0000","last_clean":"2023-10-08T05:53:06.794158+0000","last_became_active":"2023-10-08T05:53:04.763086+0000","last_became_peered":"2023-10-08T05:53:04.763086+0000","last_unstale":"2023-10-08T05:53:06.794158+0000","last_undegraded":"2023-10-08T05:53:06.794158+0000","last_fullsized":"2023-10-08T05:53:06.794158+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:31.861846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.801343+0000","last_change":"2023-10-08T05:53:05.791920+0000","last_active":"2023-10-08T05:53:06.801343+0000","last_peered":"2023-10-08T05:53:06.801343+0000","last_clean":"2023-10-08T05:53:06.801343+0000","last_became_active":"2023-10-08T05:53:05.791524+0000","last_became_peered":"2023-10-08T05:53:05.791524+0000","last_unstale":"2023-10-08T05:53:06.801343+0000","last_undegraded":"2023-10-08T05:53:06.801343+0000","last_fullsized":"2023-10-08T05:53:06.801343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:05.092018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803355+0000","last_change":"2023-10-08T05:53:04.775537+0000","last_active":"2023-10-08T05:53:06.803355+0000","last_peered":"2023-10-08T05:53:06.803355+0000","last_clean":"2023-10-08T05:53:06.803355+0000","last_became_active":"2023-10-08T05:53:04.775214+0000","last_became_peered":"2023-10-08T05:53:04.775214+0000","last_unstale":"2023-10-08T05:53:06.803355+0000","last_undegraded":"2023-10-08T05:53:06.803355+0000","last_fullsized":"2023-10-08T05:53:06.803355+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:17.491464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.793554+0000","last_change":"2023-10-08T05:53:05.803359+0000","last_active":"2023-10-08T05:53:06.793554+0000","last_peered":"2023-10-08T05:53:06.793554+0000","last_clean":"2023-10-08T05:53:06.793554+0000","last_became_active":"2023-10-08T05:53:05.803102+0000","last_became_peered":"2023-10-08T05:53:05.803102+0000","last_unstale":"2023-10-08T05:53:06.793554+0000","last_undegraded":"2023-10-08T05:53:06.793554+0000","last_fullsized":"2023-10-08T05:53:06.793554+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:13.331423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.802301+0000","last_change":"2023-10-08T05:53:04.775328+0000","last_active":"2023-10-08T05:53:06.802301+0000","last_peered":"2023-10-08T05:53:06.802301+0000","last_clean":"2023-10-08T05:53:06.802301+0000","last_became_active":"2023-10-08T05:53:04.774980+0000","last_became_peered":"2023-10-08T05:53:04.774980+0000","last_unstale":"2023-10-08T05:53:06.802301+0000","last_undegraded":"2023-10-08T05:53:06.802301+0000","last_fullsized":"2023-10-08T05:53:06.802301+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:27.133706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.799658+0000","last_change":"2023-10-08T05:53:05.791160+0000","last_active":"2023-10-08T05:53:06.799658+0000","last_peered":"2023-10-08T05:53:06.799658+0000","last_clean":"2023-10-08T05:53:06.799658+0000","last_became_active":"2023-10-08T05:53:05.790851+0000","last_became_peered":"2023-10-08T05:53:05.790851+0000","last_unstale":"2023-10-08T05:53:06.799658+0000","last_undegraded":"2023-10-08T05:53:06.799658+0000","last_fullsized":"2023-10-08T05:53:06.799658+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:55.636199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800894+0000","last_change":"2023-10-08T05:53:04.768188+0000","last_active":"2023-10-08T05:53:06.800894+0000","last_peered":"2023-10-08T05:53:06.800894+0000","last_clean":"2023-10-08T05:53:06.800894+0000","last_became_active":"2023-10-08T05:53:04.767893+0000","last_became_peered":"2023-10-08T05:53:04.767893+0000","last_unstale":"2023-10-08T05:53:06.800894+0000","last_undegraded":"2023-10-08T05:53:06.800894+0000","last_fullsized":"2023-10-08T05:53:06.800894+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:02.828936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.801455+0000","last_change":"2023-10-08T05:53:05.797926+0000","last_active":"2023-10-08T05:53:06.801455+0000","last_peered":"2023-10-08T05:53:06.801455+0000","last_clean":"2023-10-08T05:53:06.801455+0000","last_became_active":"2023-10-08T05:53:05.797610+0000","last_became_peered":"2023-10-08T05:53:05.797610+0000","last_unstale":"2023-10-08T05:53:06.801455+0000","last_undegraded":"2023-10-08T05:53:06.801455+0000","last_fullsized":"2023-10-08T05:53:06.801455+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:21:25.223794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.272364+0000","last_change":"2023-10-08T05:53:04.769571+0000","last_active":"2023-10-08T05:53:07.272364+0000","last_peered":"2023-10-08T05:53:07.272364+0000","last_clean":"2023-10-08T05:53:07.272364+0000","last_became_active":"2023-10-08T05:53:04.769300+0000","last_became_peered":"2023-10-08T05:53:04.769300+0000","last_unstale":"2023-10-08T05:53:07.272364+0000","last_undegraded":"2023-10-08T05:53:07.272364+0000","last_fullsized":"2023-10-08T05:53:07.272364+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:16.759289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274739+0000","last_change":"2023-10-08T05:53:05.808996+0000","last_active":"2023-10-08T05:53:07.274739+0000","last_peered":"2023-10-08T05:53:07.274739+0000","last_clean":"2023-10-08T05:53:07.274739+0000","last_became_active":"2023-10-08T05:53:05.808706+0000","last_became_peered":"2023-10-08T05:53:05.808706+0000","last_unstale":"2023-10-08T05:53:07.274739+0000","last_undegraded":"2023-10-08T05:53:07.274739+0000","last_fullsized":"2023-10-08T05:53:07.274739+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:02.546020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.800446+0000","last_change":"2023-10-08T05:53:04.759641+0000","last_active":"2023-10-08T05:53:06.800446+0000","last_peered":"2023-10-08T05:53:06.800446+0000","last_clean":"2023-10-08T05:53:06.800446+0000","last_became_active":"2023-10-08T05:53:04.759317+0000","last_became_peered":"2023-10-08T05:53:04.759317+0000","last_unstale":"2023-10-08T05:53:06.800446+0000","last_undegraded":"2023-10-08T05:53:06.800446+0000","last_fullsized":"2023-10-08T05:53:06.800446+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:38:24.605530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274995+0000","last_change":"2023-10-08T05:53:05.795306+0000","last_active":"2023-10-08T05:53:07.274995+0000","last_peered":"2023-10-08T05:53:07.274995+0000","last_clean":"2023-10-08T05:53:07.274995+0000","last_became_active":"2023-10-08T05:53:05.795054+0000","last_became_peered":"2023-10-08T05:53:05.795054+0000","last_unstale":"2023-10-08T05:53:07.274995+0000","last_undegraded":"2023-10-08T05:53:07.274995+0000","last_fullsized":"2023-10-08T05:53:07.274995+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:59:11.703415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.271902+0000","last_change":"2023-10-08T05:53:04.751604+0000","last_active":"2023-10-08T05:53:07.271902+0000","last_peered":"2023-10-08T05:53:07.271902+0000","last_clean":"2023-10-08T05:53:07.271902+0000","last_became_active":"2023-10-08T05:53:04.751201+0000","last_became_peered":"2023-10-08T05:53:04.751201+0000","last_unstale":"2023-10-08T05:53:07.271902+0000","last_undegraded":"2023-10-08T05:53:07.271902+0000","last_fullsized":"2023-10-08T05:53:07.271902+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:57.910900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:06.803470+0000","last_change":"2023-10-08T05:53:05.811605+0000","last_active":"2023-10-08T05:53:06.803470+0000","last_peered":"2023-10-08T05:53:06.803470+0000","last_clean":"2023-10-08T05:53:06.803470+0000","last_became_active":"2023-10-08T05:53:05.811337+0000","last_became_peered":"2023-10-08T05:53:05.811337+0000","last_unstale":"2023-10-08T05:53:06.803470+0000","last_undegraded":"2023-10-08T05:53:06.803470+0000","last_fullsized":"2023-10-08T05:53:06.803470+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:04.727671+0000","last_clean_scrub_stamp":"2023-10-08T05:53:04.727671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:33:02.485242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.274329+0000","last_change":"2023-10-08T05:53:04.763510+0000","last_active":"2023-10-08T05:53:07.274329+0000","last_peered":"2023-10-08T05:53:07.274329+0000","last_clean":"2023-10-08T05:53:07.274329+0000","last_became_active":"2023-10-08T05:53:04.763227+0000","last_became_peered":"2023-10-08T05:53:04.763227+0000","last_unstale":"2023-10-08T05:53:07.274329+0000","last_undegraded":"2023-10-08T05:53:07.274329+0000","last_fullsized":"2023-10-08T05:53:07.274329+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:03.715038+0000","last_clean_scrub_stamp":"2023-10-08T05:53:03.715038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:43.031877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"104'2158","reported_seq":2432,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-10-08T05:53:07.598658+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:53:07.598658+0000","last_peered":"2023-10-08T05:53:07.598658+0000","last_clean":"2023-10-08T05:53:07.598658+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:53:07.598658+0000","last_undegraded":"2023-10-08T05:53:07.598658+0000","last_fullsized":"2023-10-08T05:53:07.598658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2158,"log_dups_size":0,"ondisk_log_size":2158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":550,"num_read_kb":1032,"num_write":245,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":550,"num_read_kb":1032,"num_write":245,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2158,"ondisk_log_size":2158,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575053,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51052,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320788,"statfs":{"total":96636764160,"available":96584486912,"internally_reserved":0,"allocated":1871872,"data_stored":1234734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.171,"15min":1.2010000000000001},"min":{"1min":0.73299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.603,"5min":1.8859999999999999,"15min":1.901},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.1799999999999999,"15min":1.2070000000000001},"min":{"1min":0.71499999999999997,"5min":0.44,"15min":0.44},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.726},"last":1.718}]},{"osd":1,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.177,"15min":1.2310000000000001},"min":{"1min":0.60699999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.554,"5min":1.6950000000000001,"15min":1.8360000000000001},"last":1.466},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.266},"min":{"1min":0.83299999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.6759999999999999,"5min":1.7689999999999999,"15min":1.9379999999999999},"last":1.5469999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.2010000000000001,"15min":1.226},"min":{"1min":0.71899999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.748,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.117},{"interface":"front","average":{"1min":1.288,"5min":1.319,"15min":1.323},"min":{"1min":0.88600000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.665,"5min":1.752,"15min":1.7769999999999999},"last":1.052}]},{"osd":3,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.258,"15min":1.28},"min":{"1min":0.85099999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6060000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.351,"15min":1.3640000000000001},"min":{"1min":1.0760000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.6020000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2929999999999999,"15min":1.284},"min":{"1min":0.76200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":1.837,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3120000000000001,"15min":1.331},"min":{"1min":0.89600000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":2.0550000000000002,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.681}]},{"osd":5,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.375,"15min":1.3959999999999999},"min":{"1min":1.0049999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9810000000000001,"5min":2.0339999999999998,"15min":2.169},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.405,"15min":1.4179999999999999},"min":{"1min":0.93600000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.694,"5min":1.9319999999999999,"15min":2.206},"last":1.8049999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.512,"15min":1.5189999999999999},"min":{"1min":1.0940000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.968,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.4490000000000001,"15min":1.4610000000000001},"min":{"1min":1.04,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.8460000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36596,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335244,"statfs":{"total":96636764160,"available":96599289856,"internally_reserved":0,"allocated":1286144,"data_stored":645956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3049999999999999,"15min":1.2729999999999999},"min":{"1min":0.80900000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9119999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.484},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.2689999999999999,"15min":1.3100000000000001},"min":{"1min":0.91000000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8100000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.98399999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.157,"15min":1.1679999999999999},"min":{"1min":0.875,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6890000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.196,"15min":1.1850000000000001},"min":{"1min":0.92700000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8169999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.1619999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.143,"15min":1.1919999999999999},"min":{"1min":0.98799999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.774,"5min":1.774,"15min":1.796},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2150000000000001,"15min":1.262},"min":{"1min":0.90700000000000003,"5min":0.78300000000000003,"15min":0.58699999999999997},"max":{"1min":1.617,"5min":1.752,"15min":1.8620000000000001},"last":1.02}]},{"osd":3,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.351,"15min":1.3759999999999999},"min":{"1min":0.85799999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.8420000000000001,"5min":2.137,"15min":2.137},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.4119999999999999,"15min":1.458},"min":{"1min":1.006,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.024,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":0.86799999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.3220000000000001,"15min":1.3089999999999999},"min":{"1min":0.90800000000000003,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.599,"5min":1.375,"15min":1.377},"min":{"1min":1.125,"5min":0.78400000000000003,"15min":0.64300000000000002},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.381}]},{"osd":5,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.319,"15min":1.333},"min":{"1min":0.95599999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":2.1970000000000001},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.694,"5min":1.5189999999999999,"15min":1.484},"min":{"1min":1.208,"5min":0.747,"15min":0.747},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.6419999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:52:32 2023","interfaces":[{"interface":"back","average":{"1min":1.7290000000000001,"5min":1.4970000000000001,"15min":1.51},"min":{"1min":1.3360000000000001,"5min":0.97899999999999998,"15min":0.77200000000000002},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.629,"5min":1.472,"15min":1.47},"min":{"1min":1.0389999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.3320000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575052,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1265664,"data_stored":637714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.0329999999999999,"15min":1.1120000000000001},"min":{"1min":0.76600000000000001,"5min":0.58599999999999997,"15min":0.46700000000000003},"max":{"1min":1.5469999999999999,"5min":1.627,"15min":1.8169999999999999},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.1859999999999999,"15min":1.202},"min":{"1min":0.78900000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.546,"5min":1.7430000000000001,"15min":1.758},"last":0.90100000000000002}]},{"osd":1,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.2050000000000001,"15min":1.204},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.651,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.218,"15min":1.25},"min":{"1min":0.85499999999999998,"5min":0.81399999999999995,"15min":0.64700000000000002},"max":{"1min":1.5349999999999999,"5min":1.657,"15min":2.0339999999999998},"last":1.488}]},{"osd":2,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2,"15min":1.2270000000000001},"min":{"1min":0.875,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.8,"5min":1.8,"15min":1.8},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2649999999999999,"15min":1.272},"min":{"1min":0.77200000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.5369999999999999,"5min":3.282,"15min":3.282},"last":0.71399999999999997}]},{"osd":3,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3100000000000001,"15min":1.3280000000000001},"min":{"1min":0.88,"5min":0.77700000000000002,"15min":0.63500000000000001},"max":{"1min":1.6759999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.282},{"interface":"front","average":{"1min":1.397,"5min":1.375,"15min":1.3839999999999999},"min":{"1min":1.0029999999999999,"5min":0.89700000000000002,"15min":0.84799999999999998},"max":{"1min":1.919,"5min":1.919,"15min":2.2280000000000002},"last":1.2549999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.321,"15min":1.29},"min":{"1min":0.83499999999999996,"5min":0.82899999999999996,"15min":0.80400000000000005},"max":{"1min":1.681,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4630000000000001,"15min":1.405},"min":{"1min":1.1319999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.7569999999999999,"5min":3.456,"15min":3.456},"last":1.762}]},{"osd":6,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.3899999999999999,"15min":1.393},"min":{"1min":0.91900000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":2.359},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.458,"15min":1.454},"min":{"1min":1.0669999999999999,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.782,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.04}]},{"osd":7,"last update":"Sun Oct 8 05:52:42 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.552,"15min":1.5249999999999999},"min":{"1min":1.238,"5min":1.0940000000000001,"15min":0.53900000000000003},"max":{"1min":1.843,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":1.593},{"interface":"front","average":{"1min":1.639,"5min":1.5589999999999999,"15min":1.544},"min":{"1min":1.083,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.157,"5min":2.157,"15min":2.4460000000000002},"last":1.5469999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575052,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1294336,"data_stored":646060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.2230000000000001,"15min":1.26},"min":{"1min":0.80500000000000005,"5min":0.498,"15min":0.498},"max":{"1min":1.589,"5min":1.8009999999999999,"15min":1.863},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.26,"5min":1.3440000000000001,"15min":1.3779999999999999},"min":{"1min":0.76800000000000002,"5min":0.75,"15min":0.75},"max":{"1min":1.6439999999999999,"5min":1.8520000000000001,"15min":2.0680000000000001},"last":1.796}]},{"osd":1,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2589999999999999,"15min":1.272},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.538,"5min":2.3519999999999999,"15min":2.3799999999999999},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2030000000000001,"15min":1.21},"min":{"1min":0.84499999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.6719999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.3540000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.226,"15min":1.2190000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.107},{"interface":"front","average":{"1min":1.272,"5min":1.3400000000000001,"15min":1.3560000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.6299999999999999,"5min":2.423,"15min":2.423},"last":1.6020000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2509999999999999,"15min":1.264},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6080000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.323,"15min":1.337},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.1600000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.5329999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.3620000000000001,"15min":1.333},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.7310000000000001,"5min":2.48,"15min":2.48},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.383,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7090000000000001,"5min":2.504,"15min":2.504},"last":1.893}]},{"osd":6,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.3779999999999999,"15min":1.3560000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.55000000000000004},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.512,"15min":1.502},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.8049999999999999,"5min":2.5270000000000001,"15min":2.5270000000000001},"last":1.929}]},{"osd":7,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4990000000000001,"15min":1.4570000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8029999999999999,"5min":4.125,"15min":4.125},"last":1.996},{"interface":"front","average":{"1min":1.46,"5min":1.5009999999999999,"15min":1.5169999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.4350000000000001,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":2.0339999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575054,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36576,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335264,"statfs":{"total":96636764160,"available":96599310336,"internally_reserved":0,"allocated":1265664,"data_stored":638262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.91900000000000004,"15min":0.91800000000000004},"min":{"1min":0.58299999999999996,"5min":0.32100000000000001,"15min":0.29599999999999999},"max":{"1min":1.194,"5min":1.5269999999999999,"15min":1.72},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.04,"15min":1.083},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3380000000000001,"5min":1.5,"15min":1.6140000000000001},"last":0.95499999999999996}]},{"osd":1,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.92300000000000004,"15min":0.96699999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.429,"5min":1.5629999999999999,"15min":1.6040000000000001},"last":1.042},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.198,"15min":1.206},"min":{"1min":0.63700000000000001,"5min":0.56599999999999995,"15min":0.436},"max":{"1min":1.54,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.1759999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":1.0369999999999999,"15min":1.071},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.4350000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.097,"5min":1.1399999999999999,"15min":1.145},"min":{"1min":0.69299999999999995,"5min":0.60499999999999998,"15min":0.41899999999999998},"max":{"1min":1.6040000000000001,"5min":1.6819999999999999,"15min":1.788},"last":1.0720000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1990000000000001,"15min":1.202},"min":{"1min":0.68999999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.429,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3680000000000001,"15min":1.3859999999999999},"min":{"1min":0.90900000000000003,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7450000000000001,"5min":2.589,"15min":2.589},"last":1.2130000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.304,"15min":1.302},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.73699999999999999},"max":{"1min":1.659,"5min":1.883,"15min":1.9219999999999999},"last":1.151},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.3400000000000001,"15min":1.3560000000000001},"min":{"1min":0.745,"5min":0.745,"15min":0.65200000000000002},"max":{"1min":1.6020000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.129}]},{"osd":6,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3129999999999999},"min":{"1min":0.89100000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.766,"5min":1.766,"15min":1.776},"last":1.26},{"interface":"front","average":{"1min":1.371,"5min":1.4410000000000001,"15min":1.444},"min":{"1min":0.95899999999999996,"5min":0.86799999999999999,"15min":0.86699999999999999},"max":{"1min":1.6220000000000001,"5min":2.012,"15min":2.0489999999999999},"last":1.3560000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:52:38 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.466,"15min":1.486},"min":{"1min":0.879,"5min":0.82399999999999995,"15min":0.72299999999999998},"max":{"1min":1.6599999999999999,"5min":2.085,"15min":2.1480000000000001},"last":1.242},{"interface":"front","average":{"1min":1.329,"5min":1.365,"15min":1.3859999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.704,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.0980000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575052,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58800,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313040,"statfs":{"total":96636764160,"available":96576552960,"internally_reserved":0,"allocated":1273856,"data_stored":638422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.82599999999999996,"15min":0.88900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.2390000000000001,"5min":1.5209999999999999,"15min":2.4009999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":1.014,"15min":1.036},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.4650000000000001,"5min":1.5880000000000001,"15min":1.9470000000000001},"last":1.3560000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.91200000000000003,"15min":0.94699999999999995},"min":{"1min":0.50800000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.373,"5min":1.7609999999999999,"15min":2.1070000000000002},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.036,"15min":1.093},"min":{"1min":0.47799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.4550000000000001,"5min":1.639,"15min":2.1469999999999998},"last":0.60999999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0720000000000001,"15min":1.1240000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.472,"5min":1.639,"15min":1.8120000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.2150000000000001,"15min":1.23},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.6399999999999999,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":0.89000000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.375,"15min":1.385},"min":{"1min":0.86899999999999999,"5min":0.622,"15min":0.622},"max":{"1min":1.5660000000000001,"5min":4.593,"15min":4.593},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.2669999999999999,"15min":1.2889999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.528,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.0209999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.349,"15min":1.373},"min":{"1min":0.72599999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6160000000000001,"5min":6.7140000000000004,"15min":6.7140000000000004},"last":1.163},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.4059999999999999,"15min":1.448},"min":{"1min":0.71999999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.6910000000000001,"5min":2.0529999999999999,"15min":2.4119999999999999},"last":1.321}]},{"osd":6,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.3140000000000001,"15min":1.337},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.597,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.3919999999999999,"15min":1.4450000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.659,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.292}]},{"osd":7,"last update":"Sun Oct 8 05:52:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.419,"15min":1.4330000000000001},"min":{"1min":0.73699999999999999,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.7170000000000001,"5min":6.609,"15min":6.609},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.4199999999999999,"15min":1.488},"min":{"1min":0.877,"5min":0.82299999999999995,"15min":0.79800000000000004},"max":{"1min":1.6060000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.391}]}]},{"osd":1,"up_from":13,"seq":55834575052,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36584,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335256,"statfs":{"total":96636764160,"available":96599302144,"internally_reserved":0,"allocated":1269760,"data_stored":637792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.92700000000000005,"15min":0.96499999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.29799999999999999},"max":{"1min":1.2010000000000001,"5min":1.796,"15min":1.796},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0860000000000001,"15min":1.1279999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.433},"max":{"1min":1.431,"5min":1.6060000000000001,"15min":1.8089999999999999},"last":0.626}]},{"osd":2,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.92300000000000004,"15min":0.95699999999999996},"min":{"1min":0.58299999999999996,"5min":0.39500000000000002,"15min":0.35899999999999999},"max":{"1min":1.3120000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0469999999999999,"15min":1.081},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.5860000000000001,"5min":1.77,"15min":1.8049999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.1140000000000001,"15min":1.175},"min":{"1min":0.5,"5min":0.5,"15min":0.438},"max":{"1min":1.46,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.012,"5min":1.0609999999999999,"15min":1.0669999999999999},"min":{"1min":0.55300000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.585,"5min":1.788,"15min":1.788},"last":0.872}]},{"osd":4,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2430000000000001,"15min":1.2969999999999999},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.63900000000000001},"max":{"1min":1.625,"5min":1.966,"15min":1.966},"last":1.073},{"interface":"front","average":{"1min":1.143,"5min":1.2170000000000001,"15min":1.226},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.61799999999999999},"max":{"1min":1.536,"5min":1.913,"15min":1.913},"last":1.389}]},{"osd":5,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.2809999999999999,"15min":1.304},"min":{"1min":0.76500000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5089999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.641},{"interface":"front","average":{"1min":1.383,"5min":1.417,"15min":1.4410000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.755},"max":{"1min":1.663,"5min":2.077,"15min":2.077},"last":1.5469999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.381,"15min":1.4179999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.75,"5min":1.972,"15min":1.9970000000000001},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.399,"15min":1.4199999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6339999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.597}]},{"osd":7,"last update":"Sun Oct 8 05:52:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.4530000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6279999999999999,"5min":1.97,"15min":2.1869999999999998},"last":1.353},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4399999999999999,"15min":1.45},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.82999999999999996},"max":{"1min":1.829,"5min":2.286,"15min":2.306},"last":1.254}]}]},{"osd":0,"up_from":13,"seq":55834575053,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28840,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343000,"statfs":{"total":96636764160,"available":96607232000,"internally_reserved":0,"allocated":1867776,"data_stored":1234712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.88,"15min":0.90800000000000003},"min":{"1min":0.57799999999999996,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.77,"5min":1.77,"15min":1.77},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.026,"5min":1.0209999999999999,"15min":1.0269999999999999},"min":{"1min":0.52900000000000003,"5min":0.371,"15min":0.371},"max":{"1min":1.514,"5min":1.6719999999999999,"15min":1.675},"last":0.90700000000000003}]},{"osd":2,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.038,"15min":1.081},"min":{"1min":0.56899999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.357,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.94499999999999995,"15min":0.98999999999999999},"min":{"1min":0.52500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.722,"5min":1.722,"15min":1.78},"last":0.85199999999999998}]},{"osd":3,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.143,"15min":1.194},"min":{"1min":0.59799999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.6319999999999999,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.071,"15min":1.077},"min":{"1min":0.47999999999999998,"5min":0.44700000000000001,"15min":0.439},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.022}]},{"osd":4,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.246,"15min":1.266},"min":{"1min":0.79200000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.724,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3049999999999999,"15min":1.3380000000000001},"min":{"1min":0.97999999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.571,"5min":1.8939999999999999,"15min":2.2200000000000002},"last":1.24}]},{"osd":5,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.296,"15min":1.325},"min":{"1min":0.83699999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.532,"5min":1.921,"15min":2.0489999999999999},"last":1.415},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3300000000000001,"15min":1.3759999999999999},"min":{"1min":1.0209999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.873,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.3919999999999999,"15min":1.405},"min":{"1min":0.91700000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.032,"5min":2.1909999999999998,"15min":2.5600000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.349,"15min":1.3899999999999999},"min":{"1min":0.92600000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.6060000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.4830000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:52:37 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3999999999999999,"15min":1.4279999999999999},"min":{"1min":0.96499999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7969999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.444,"15min":1.46},"min":{"1min":0.85799999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8140000000000001,"5min":2.1059999999999999,"15min":2.2759999999999998},"last":1.1140000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:53:10.763 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T05:53:10.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T05:53:10.863 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:10.861+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:11.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.022+0000 7f8e45cc2640 1 Processor -- start 2023-10-08T05:53:11.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.023+0000 7f8e45cc2640 1 -- start start 2023-10-08T05:53:11.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.024+0000 7f8e45cc2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8e4006cfa0 0x7f8e4015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.024+0000 7f8e45cc2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e400c8c60 con 0x7f8e4015b630 2023-10-08T05:53:11.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.024+0000 7f8e45cc2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e400b1ac0 con 0x7f8e400cb500 2023-10-08T05:53:11.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.024+0000 7f8e45cc2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e400b1da0 con 0x7f8e4006cfa0 2023-10-08T05:53:11.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.025+0000 7f8e37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8e4006cfa0 0x7f8e4015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.025+0000 7f8e3ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8e4015b630 0x7f8e40070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37114/0 (socket says 172.21.15.70:37114) 2023-10-08T05:53:11.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.025+0000 7f8e3ffff640 1 -- 172.21.15.70:0/964938486 learned_addr learned my addr 172.21.15.70:0/964938486 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.025+0000 7f8e3effd640 1 -- 172.21.15.70:0/964938486 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588432285 0 0) 0x7f8e400c8c60 con 0x7f8e4015b630 2023-10-08T05:53:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.025+0000 7f8e3effd640 1 -- 172.21.15.70:0/964938486 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e2c003520 con 0x7f8e4015b630 2023-10-08T05:53:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.025+0000 7f8e37fff640 1 -- 172.21.15.70:0/964938486 >> v1:172.21.15.181:6789/0 conn(0x7f8e400cb500 legacy=0x7f8e400cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.025+0000 7f8e37fff640 1 -- 172.21.15.70:0/964938486 >> v1:172.21.15.70:6789/0 conn(0x7f8e4015b630 legacy=0x7f8e40070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.025+0000 7f8e37fff640 1 -- 172.21.15.70:0/964938486 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e400b2080 con 0x7f8e4006cfa0 2023-10-08T05:53:11.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.027+0000 7f8e37fff640 1 --2- 172.21.15.70:0/964938486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 0x7f8e4015b0f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8e280097a0 tx=0x7f8e2802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bfd8c32b62d57d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:11.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.027+0000 7f8e3effd640 1 -- 172.21.15.70:0/964938486 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2802e070 con 0x7f8e4006cfa0 2023-10-08T05:53:11.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.027+0000 7f8e3effd640 1 -- 172.21.15.70:0/964938486 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e280315a0 con 0x7f8e4006cfa0 2023-10-08T05:53:11.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.027+0000 7f8e3effd640 1 -- 172.21.15.70:0/964938486 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28031910 con 0x7f8e4006cfa0 2023-10-08T05:53:11.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.028+0000 7f8e45cc2640 1 -- 172.21.15.70:0/964938486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 msgr2=0x7f8e4015b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.028+0000 7f8e45cc2640 1 --2- 172.21.15.70:0/964938486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 0x7f8e4015b0f0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8e280097a0 tx=0x7f8e2802b840 comp rx=0 tx=0).stop 2023-10-08T05:53:11.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.028+0000 7f8e45cc2640 1 -- 172.21.15.70:0/964938486 shutdown_connections 2023-10-08T05:53:11.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.028+0000 7f8e45cc2640 1 --2- 172.21.15.70:0/964938486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 0x7f8e4015b0f0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.028+0000 7f8e45cc2640 1 -- 172.21.15.70:0/964938486 >> 172.21.15.70:0/964938486 conn(0x7f8e4006a140 msgr2=0x7f8e4006a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:11.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.028+0000 7f8e45cc2640 1 -- 172.21.15.70:0/964938486 shutdown_connections 2023-10-08T05:53:11.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.028+0000 7f8e45cc2640 1 -- 172.21.15.70:0/964938486 wait complete. 2023-10-08T05:53:11.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.030+0000 7f8e45cc2640 1 Processor -- start 2023-10-08T05:53:11.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.030+0000 7f8e45cc2640 1 -- start start 2023-10-08T05:53:11.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.031+0000 7f8e45cc2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 0x7f8e40157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.031+0000 7f8e45cc2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e400cb500 0x7f8e400ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.031+0000 7f8e45cc2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e4015b630 0x7f8e40150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.031+0000 7f8e45cc2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e40159040 con 0x7f8e4006cfa0 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.031+0000 7f8e45cc2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e40157fa0 con 0x7f8e400cb500 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.031+0000 7f8e45cc2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e40158280 con 0x7f8e4015b630 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e3f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 0x7f8e40157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e3f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 0x7f8e40157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46700/0 (socket says 172.21.15.70:46700) 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e3f7fe640 1 -- 172.21.15.70:0/2316674497 learned_addr learned my addr 172.21.15.70:0/2316674497 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e37fff640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e400cb500 0x7f8e400ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e3f7fe640 1 -- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e4015b630 msgr2=0x7f8e40150990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e3f7fe640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e4015b630 0x7f8e40150990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e3f7fe640 1 -- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e400cb500 msgr2=0x7f8e400ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e3f7fe640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e400cb500 0x7f8e400ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e3f7fe640 1 -- 172.21.15.70:0/2316674497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e28009400 con 0x7f8e4006cfa0 2023-10-08T05:53:11.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e37fff640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e400cb500 0x7f8e400ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:11.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.032+0000 7f8e3f7fe640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 0x7f8e40157710 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000b3e0 tx=0x7f8e3000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:11.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.033+0000 7f8e3cff9640 1 -- 172.21.15.70:0/2316674497 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30011830 con 0x7f8e4006cfa0 2023-10-08T05:53:11.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.033+0000 7f8e3cff9640 1 -- 172.21.15.70:0/2316674497 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e300119b0 con 0x7f8e4006cfa0 2023-10-08T05:53:11.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.033+0000 7f8e3cff9640 1 -- 172.21.15.70:0/2316674497 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30011d60 con 0x7f8e4006cfa0 2023-10-08T05:53:11.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.033+0000 7f8e45cc2640 1 -- 172.21.15.70:0/2316674497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e40158560 con 0x7f8e4006cfa0 2023-10-08T05:53:11.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.033+0000 7f8e45cc2640 1 -- 172.21.15.70:0/2316674497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e40158b20 con 0x7f8e4006cfa0 2023-10-08T05:53:11.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.034+0000 7f8e45cc2640 1 -- 172.21.15.70:0/2316674497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e40071cb0 con 0x7f8e4006cfa0 2023-10-08T05:53:11.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.035+0000 7f8e3cff9640 1 -- 172.21.15.70:0/2316674497 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8e3001a400 con 0x7f8e4006cfa0 2023-10-08T05:53:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.036+0000 7f8e3cff9640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f8e2006af00 0x7f8e2006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.036+0000 7f8e3cff9640 1 -- 172.21.15.70:0/2316674497 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8e30089970 con 0x7f8e4006cfa0 2023-10-08T05:53:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.036+0000 7f8e37fff640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f8e2006af00 0x7f8e2006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.038+0000 7f8e37fff640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f8e2006af00 0x7f8e2006d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8e28036000 tx=0x7f8e2803d040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:11.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.040+0000 7f8e3cff9640 1 -- 172.21.15.70:0/2316674497 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8e30054850 con 0x7f8e4006cfa0 2023-10-08T05:53:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.309+0000 7f8e45cc2640 1 -- 172.21.15.70:0/2316674497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8e4004e240 con 0x7f8e4006cfa0 2023-10-08T05:53:11.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.313+0000 7f8e3cff9640 1 -- 172.21.15.70:0/2316674497 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8e300150e0 con 0x7f8e4006cfa0 2023-10-08T05:53:11.315 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T05:53:11.315 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T05:53:11.315 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T05:53:11.315 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T05:53:11.315 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T05:53:11.315 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T05:53:11.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.316+0000 7f8e45cc2640 1 -- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f8e2006af00 msgr2=0x7f8e2006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.316+0000 7f8e45cc2640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f8e2006af00 0x7f8e2006d3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8e28036000 tx=0x7f8e2803d040 comp rx=0 tx=0).stop 2023-10-08T05:53:11.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.316+0000 7f8e45cc2640 1 -- 172.21.15.70:0/2316674497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 msgr2=0x7f8e40157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.316+0000 7f8e45cc2640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 0x7f8e40157710 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000b3e0 tx=0x7f8e3000b8a0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.317+0000 7f8e45cc2640 1 -- 172.21.15.70:0/2316674497 shutdown_connections 2023-10-08T05:53:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.317+0000 7f8e45cc2640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f8e2006af00 0x7f8e2006d3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.317+0000 7f8e45cc2640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8e4015b630 0x7f8e40150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.317+0000 7f8e45cc2640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8e400cb500 0x7f8e400ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.317+0000 7f8e45cc2640 1 --2- 172.21.15.70:0/2316674497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e4006cfa0 0x7f8e40157710 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.317+0000 7f8e45cc2640 1 -- 172.21.15.70:0/2316674497 >> 172.21.15.70:0/2316674497 conn(0x7f8e4006a140 msgr2=0x7f8e400cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.317+0000 7f8e45cc2640 1 -- 172.21.15.70:0/2316674497 shutdown_connections 2023-10-08T05:53:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.317+0000 7f8e45cc2640 1 -- 172.21.15.70:0/2316674497 wait complete. 2023-10-08T05:53:11.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:53:11.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.554+0000 7f6a63ae0640 1 Processor -- start 2023-10-08T05:53:11.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.554+0000 7f6a63ae0640 1 -- start start 2023-10-08T05:53:11.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.556+0000 7f6a63ae0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6a5c15cd20 0x7f6a5c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.556+0000 7f6a63ae0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a5c0cb220 0x7f6a5c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.556+0000 7f6a63ae0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c06b490 con 0x7f6a5c0cd750 2023-10-08T05:53:11.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.556+0000 7f6a63ae0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c06b770 con 0x7f6a5c15cd20 2023-10-08T05:53:11.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.556+0000 7f6a63ae0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c06ba50 con 0x7f6a5c0cb220 2023-10-08T05:53:11.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.556+0000 7f6a6207d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6a5c0cd750 0x7f6a5c06c530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37122/0 (socket says 172.21.15.70:37122) 2023-10-08T05:53:11.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.556+0000 7f6a6207d640 1 -- 172.21.15.70:0/55944186 learned_addr learned my addr 172.21.15.70:0/55944186 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:11.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6187c640 1 --2- 172.21.15.70:0/55944186 >> v2:172.21.15.181:3300/0 conn(0x7f6a5c15cd20 0x7f6a5c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6107b640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a5c0cb220 0x7f6a5c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214197142 0 0) 0x7f6a5c06b490 con 0x7f6a5c0cd750 2023-10-08T05:53:11.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a48003520 con 0x7f6a5c0cd750 2023-10-08T05:53:11.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1643989316 0 0) 0x7f6a48003520 con 0x7f6a5c0cd750 2023-10-08T05:53:11.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a5c0cb220 msgr2=0x7f6a5c0cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a5c0cb220 0x7f6a5c0cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 >> v2:172.21.15.181:3300/0 conn(0x7f6a5c15cd20 msgr2=0x7f6a5c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 --2- 172.21.15.70:0/55944186 >> v2:172.21.15.181:3300/0 conn(0x7f6a5c15cd20 0x7f6a5c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c06bd30 con 0x7f6a5c0cd750 2023-10-08T05:53:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6a54001a40 con 0x7f6a5c0cd750 2023-10-08T05:53:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6187c640 1 --2- 172.21.15.70:0/55944186 >> v2:172.21.15.181:3300/0 conn(0x7f6a5c15cd20 0x7f6a5c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:53:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 >> v1:172.21.15.70:6789/0 conn(0x7f6a5c0cd750 legacy=0x7f6a5c06c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a48003970 0x7f6a48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a480047c0 0x7f6a48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a5c0cb220 0x7f6a48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c06b770 con 0x7f6a48003970 2023-10-08T05:53:11.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c06ba50 con 0x7f6a480047c0 2023-10-08T05:53:11.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4800b9d0 con 0x7f6a5c0cb220 2023-10-08T05:53:11.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.557+0000 7f6a6187c640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a48003970 0x7f6a48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.558+0000 7f6a6107b640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a480047c0 0x7f6a48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.558+0000 7f6a6207d640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a5c0cb220 0x7f6a48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.558+0000 7f6a6187c640 1 -- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a5c0cb220 msgr2=0x7f6a48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:11.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.558+0000 7f6a6187c640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a5c0cb220 0x7f6a48005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.558+0000 7f6a6187c640 1 -- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a480047c0 msgr2=0x7f6a48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.558+0000 7f6a6187c640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a480047c0 0x7f6a48004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.558+0000 7f6a6187c640 1 -- 172.21.15.70:0/55944186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a48003520 con 0x7f6a48003970 2023-10-08T05:53:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.558+0000 7f6a6207d640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a5c0cb220 0x7f6a48005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.559+0000 7f6a6187c640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a48003970 0x7f6a48003d90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f6a580098a0 tx=0x7f6a480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382fba68aa336e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.559+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a58004230 con 0x7f6a48003970 2023-10-08T05:53:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.559+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a58004390 con 0x7f6a48003970 2023-10-08T05:53:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.559+0000 7f6a6087a640 1 -- 172.21.15.70:0/55944186 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a58003830 con 0x7f6a48003970 2023-10-08T05:53:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.560+0000 7f6a63ae0640 1 -- 172.21.15.70:0/55944186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a48003970 msgr2=0x7f6a48003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.560+0000 7f6a63ae0640 1 --2- 172.21.15.70:0/55944186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a48003970 0x7f6a48003d90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f6a580098a0 tx=0x7f6a480057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.560+0000 7f6a6207d640 1 -- 172.21.15.70:0/55944186 reap_dead start 2023-10-08T05:53:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.560+0000 7f6a63ae0640 1 -- 172.21.15.70:0/55944186 shutdown_connections 2023-10-08T05:53:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.560+0000 7f6a63ae0640 1 -- 172.21.15.70:0/55944186 >> 172.21.15.70:0/55944186 conn(0x7f6a5c069f30 msgr2=0x7f6a5c06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.560+0000 7f6a63ae0640 1 -- 172.21.15.70:0/55944186 shutdown_connections 2023-10-08T05:53:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.560+0000 7f6a63ae0640 1 -- 172.21.15.70:0/55944186 wait complete. 2023-10-08T05:53:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.562+0000 7f6a63ae0640 1 Processor -- start 2023-10-08T05:53:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.562+0000 7f6a63ae0640 1 -- start start 2023-10-08T05:53:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.563+0000 7f6a63ae0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a48003970 0x7f6a5c14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.563+0000 7f6a63ae0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5c0c7ac0 0x7f6a5c0b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.563+0000 7f6a63ae0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a5c0c7f40 0x7f6a5c0b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.563+0000 7f6a63ae0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0ca750 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.563+0000 7f6a63ae0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0caa30 con 0x7f6a5c0c7f40 2023-10-08T05:53:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.563+0000 7f6a63ae0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0cad10 con 0x7f6a48003970 2023-10-08T05:53:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6107b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5c0c7ac0 0x7f6a5c0b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6107b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5c0c7ac0 0x7f6a5c0b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46710/0 (socket says 172.21.15.70:46710) 2023-10-08T05:53:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6107b640 1 -- 172.21.15.70:0/3274007935 learned_addr learned my addr 172.21.15.70:0/3274007935 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:11.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6187c640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a48003970 0x7f6a5c14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6207d640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a5c0c7f40 0x7f6a5c0b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6107b640 1 -- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a48003970 msgr2=0x7f6a5c14f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6107b640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a48003970 0x7f6a5c14f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6107b640 1 -- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a5c0c7f40 msgr2=0x7f6a5c0b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6107b640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a5c0c7f40 0x7f6a5c0b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6107b640 1 -- 172.21.15.70:0/3274007935 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0c4b50 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6207d640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a5c0c7f40 0x7f6a5c0b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:11.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.564+0000 7f6a6107b640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5c0c7ac0 0x7f6a5c0b4c00 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c011600 tx=0x7f6a4c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:11.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.565+0000 7f6a527fc640 1 -- 172.21.15.70:0/3274007935 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00f920 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.565+0000 7f6a527fc640 1 -- 172.21.15.70:0/3274007935 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c00fa80 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.565+0000 7f6a527fc640 1 -- 172.21.15.70:0/3274007935 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00fdf0 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.565+0000 7f6a63ae0640 1 -- 172.21.15.70:0/3274007935 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c0c4e30 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.565+0000 7f6a63ae0640 1 -- 172.21.15.70:0/3274007935 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c0b5860 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.566+0000 7f6a63ae0640 1 -- 172.21.15.70:0/3274007935 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c066d80 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.567+0000 7f6a527fc640 1 -- 172.21.15.70:0/3274007935 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f6a4c015460 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.568+0000 7f6a527fc640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6a4006af00 0x7f6a4006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.568+0000 7f6a527fc640 1 -- 172.21.15.70:0/3274007935 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6a4c089cb0 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.569+0000 7f6a6187c640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6a4006af00 0x7f6a4006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.570+0000 7f6a6187c640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6a4006af00 0x7f6a4006d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a58035000 tx=0x7f6a58039040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:11.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.572+0000 7f6a527fc640 1 -- 172.21.15.70:0/3274007935 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a4c054c00 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.800+0000 7f6a63ae0640 1 -- 172.21.15.70:0/3274007935 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a5c06a310 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.801+0000 7f6a527fc640 1 -- 172.21.15.70:0/3274007935 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f6a4c022090 con 0x7f6a5c0c7ac0 2023-10-08T05:53:11.803 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:11.803 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5384,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3148555214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3148555214},{"type":"v1","addr":"172.21.15.181:6837","nonce":3148555214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5896,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3669053115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3669053115},{"type":"v1","addr":"172.21.15.181:6836","nonce":3669053115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":7122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3775478637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3775478637},{"type":"v1","addr":"172.21.15.70:6833","nonce":3775478637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:05.225421+0000","modified":"2023-10-08T05:53:08.291318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7131},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7131":{"gid":7131,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/24744490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":24744490},{"type":"v1","addr":"172.21.15.70:6835","nonce":24744490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-08T05:53:11.803 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-10-08T05:53:11.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 -- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6a4006af00 msgr2=0x7f6a4006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6a4006af00 0x7f6a4006d3c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a58035000 tx=0x7f6a58039040 comp rx=0 tx=0).stop 2023-10-08T05:53:11.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 -- 172.21.15.70:0/3274007935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5c0c7ac0 msgr2=0x7f6a5c0b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:11.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5c0c7ac0 0x7f6a5c0b4c00 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c011600 tx=0x7f6a4c011ac0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 -- 172.21.15.70:0/3274007935 shutdown_connections 2023-10-08T05:53:11.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f6a4006af00 0x7f6a4006d3c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6a5c0c7f40 0x7f6a5c0b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a5c0c7ac0 0x7f6a5c0b4c00 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 --2- 172.21.15.70:0/3274007935 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6a48003970 0x7f6a5c14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:11.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 -- 172.21.15.70:0/3274007935 >> 172.21.15.70:0/3274007935 conn(0x7f6a5c069f30 msgr2=0x7f6a5c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:11.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 -- 172.21.15.70:0/3274007935 shutdown_connections 2023-10-08T05:53:11.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:11.804+0000 7f6a63ae0640 1 -- 172.21.15.70:0/3274007935 wait complete. 2023-10-08T05:53:11.945 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:11.943+0000 7f48625e3640 -1 client.5776: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5234:75cb77c4-a226-4634-92ae-0462ea0d891c:v1:172.21.15.181:0/999647594] 2023-10-08T05:53:12.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T05:53:05.225421+0000', 'modified': '2023-10-08T05:53:08.291318+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7131}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7131': {'gid': 7131, 'name': 'c', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6835/24744490', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 24744490}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 24744490}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T05:53:12.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T05:53:12.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T05:53:12.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.455+0000 7f26807b0640 1 Processor -- start 2023-10-08T05:53:12.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.455+0000 7f26807b0640 1 -- start start 2023-10-08T05:53:12.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.457+0000 7f26807b0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f267815ce10 0x7f26780ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.457+0000 7f26807b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f267806bc80 0x7f26780cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.457+0000 7f26807b0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26780cf840 con 0x7f26780cf3e0 2023-10-08T05:53:12.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.457+0000 7f26807b0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267806cd90 con 0x7f267815ce10 2023-10-08T05:53:12.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.457+0000 7f26807b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2678160ca0 con 0x7f267806bc80 2023-10-08T05:53:12.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.457+0000 7f267dd4b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f26780cf3e0 0x7f267806b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37138/0 (socket says 172.21.15.70:37138) 2023-10-08T05:53:12.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.457+0000 7f267dd4b640 1 -- 172.21.15.70:0/2747606593 learned_addr learned my addr 172.21.15.70:0/2747606593 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:12.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.457+0000 7f267e54c640 1 --2- 172.21.15.70:0/2747606593 >> v2:172.21.15.181:3300/0 conn(0x7f267815ce10 0x7f26780ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.457+0000 7f267ed4d640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f267806bc80 0x7f26780cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489978726 0 0) 0x7f26780cf840 con 0x7f26780cf3e0 2023-10-08T05:53:12.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2664003520 con 0x7f26780cf3e0 2023-10-08T05:53:12.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3472897695 0 0) 0x7f2664003520 con 0x7f26780cf3e0 2023-10-08T05:53:12.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f267806bc80 msgr2=0x7f26780cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f267806bc80 0x7f26780cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 >> v2:172.21.15.181:3300/0 conn(0x7f267815ce10 msgr2=0x7f26780ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 --2- 172.21.15.70:0/2747606593 >> v2:172.21.15.181:3300/0 conn(0x7f267815ce10 0x7f26780ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267806c180 con 0x7f26780cf3e0 2023-10-08T05:53:12.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2668001670 con 0x7f26780cf3e0 2023-10-08T05:53:12.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 >> v1:172.21.15.70:6789/0 conn(0x7f26780cf3e0 legacy=0x7f267806b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f267806bc80 0x7f2664003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2664003f40 0x7f26640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2664004d00 0x7f2664005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267806cd90 con 0x7f2664004d00 2023-10-08T05:53:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678160ca0 con 0x7f2664003f40 2023-10-08T05:53:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.458+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266400ba30 con 0x7f267806bc80 2023-10-08T05:53:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.459+0000 7f267ed4d640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2664004d00 0x7f2664005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.459+0000 7f267e54c640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f267806bc80 0x7f2664003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.459+0000 7f267dd4b640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2664003f40 0x7f26640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.459+0000 7f267ed4d640 1 -- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f267806bc80 msgr2=0x7f2664003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.459+0000 7f267ed4d640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f267806bc80 0x7f2664003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.459+0000 7f267ed4d640 1 -- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2664003f40 msgr2=0x7f26640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.459+0000 7f267ed4d640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2664003f40 0x7f26640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.459+0000 7f267ed4d640 1 -- 172.21.15.70:0/2747606593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2664003520 con 0x7f2664004d00 2023-10-08T05:53:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.460+0000 7f267ed4d640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2664004d00 0x7f2664005180 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2670009f90 tx=0x7f26640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91af200b3a0786ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.460+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267002cbd0 con 0x7f2664004d00 2023-10-08T05:53:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.460+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267000f040 con 0x7f2664004d00 2023-10-08T05:53:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.460+0000 7f267d54a640 1 -- 172.21.15.70:0/2747606593 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670034440 con 0x7f2664004d00 2023-10-08T05:53:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.461+0000 7f26807b0640 1 -- 172.21.15.70:0/2747606593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2664004d00 msgr2=0x7f2664005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.461+0000 7f26807b0640 1 --2- 172.21.15.70:0/2747606593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2664004d00 0x7f2664005180 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2670009f90 tx=0x7f26640057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.461+0000 7f267ed4d640 1 -- 172.21.15.70:0/2747606593 reap_dead start 2023-10-08T05:53:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.461+0000 7f26807b0640 1 -- 172.21.15.70:0/2747606593 shutdown_connections 2023-10-08T05:53:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.461+0000 7f26807b0640 1 -- 172.21.15.70:0/2747606593 >> 172.21.15.70:0/2747606593 conn(0x7f267806e630 msgr2=0x7f267806ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.461+0000 7f26807b0640 1 -- 172.21.15.70:0/2747606593 shutdown_connections 2023-10-08T05:53:12.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.461+0000 7f26807b0640 1 -- 172.21.15.70:0/2747606593 wait complete. 2023-10-08T05:53:12.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.463+0000 7f26807b0640 1 Processor -- start 2023-10-08T05:53:12.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.463+0000 7f26807b0640 1 -- start start 2023-10-08T05:53:12.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.464+0000 7f26807b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2664004d00 0x7f26780c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.464+0000 7f26807b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780c7b20 0x7f267814a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.464+0000 7f26807b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f267814ad20 0x7f26780c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.464+0000 7f26807b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780b6030 con 0x7f26780c7b20 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.464+0000 7f26807b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267814b1a0 con 0x7f267814ad20 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.464+0000 7f26807b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26780b54b0 con 0x7f2664004d00 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267dd4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780c7b20 0x7f267814a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267dd4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780c7b20 0x7f267814a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46730/0 (socket says 172.21.15.70:46730) 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267dd4b640 1 -- 172.21.15.70:0/2831890444 learned_addr learned my addr 172.21.15.70:0/2831890444 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267e54c640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2664004d00 0x7f26780c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267ed4d640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f267814ad20 0x7f26780c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267dd4b640 1 -- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2664004d00 msgr2=0x7f26780c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267dd4b640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2664004d00 0x7f26780c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267dd4b640 1 -- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f267814ad20 msgr2=0x7f26780c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267dd4b640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f267814ad20 0x7f26780c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267dd4b640 1 -- 172.21.15.70:0/2831890444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780b5790 con 0x7f26780c7b20 2023-10-08T05:53:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.465+0000 7f267dd4b640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780c7b20 0x7f267814a7e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f266800e820 tx=0x7f266800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.466+0000 7f266effd640 1 -- 172.21.15.70:0/2831890444 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266800c870 con 0x7f26780c7b20 2023-10-08T05:53:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.466+0000 7f266effd640 1 -- 172.21.15.70:0/2831890444 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266800c9f0 con 0x7f26780c7b20 2023-10-08T05:53:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.466+0000 7f26807b0640 1 -- 172.21.15.70:0/2831890444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26780b5a70 con 0x7f26780c7b20 2023-10-08T05:53:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.466+0000 7f266effd640 1 -- 172.21.15.70:0/2831890444 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266800cda0 con 0x7f26780c7b20 2023-10-08T05:53:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.466+0000 7f26807b0640 1 -- 172.21.15.70:0/2831890444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780ca8c0 con 0x7f26780c7b20 2023-10-08T05:53:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.468+0000 7f266effd640 1 -- 172.21.15.70:0/2831890444 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f266801d460 con 0x7f26780c7b20 2023-10-08T05:53:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.469+0000 7f266effd640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f265c06af30 0x7f265c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.470+0000 7f266effd640 1 -- 172.21.15.70:0/2831890444 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2668089880 con 0x7f26780c7b20 2023-10-08T05:53:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.470+0000 7f267e54c640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f265c06af30 0x7f265c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.470+0000 7f268341e740 1 Processor -- start 2023-10-08T05:53:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.471+0000 7f268341e740 1 -- start start 2023-10-08T05:53:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.471+0000 7f267e54c640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f265c06af30 0x7f265c06d3f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2674009a90 tx=0x7f2674008040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f268341e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 0x5649a077ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f268341e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f268341e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a077efa0 0x5649a07aee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f268341e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a06f2700 con 0x5649a07abf40 2023-10-08T05:53:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f268341e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a06efdf0 con 0x5649a077efa0 2023-10-08T05:53:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f268341e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649a06eb6e0 con 0x5649a07abb60 2023-10-08T05:53:12.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267dd4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267dd4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46744/0 (socket says 172.21.15.70:46744) 2023-10-08T05:53:12.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267dd4b640 1 -- 172.21.15.70:0/120092110 learned_addr learned my addr 172.21.15.70:0/120092110 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267ed4d640 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a077efa0 0x5649a07aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267e54c640 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 0x5649a077ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267dd4b640 1 -- 172.21.15.70:0/120092110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 msgr2=0x5649a077ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267dd4b640 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 0x5649a077ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267dd4b640 1 -- 172.21.15.70:0/120092110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a077efa0 msgr2=0x5649a07aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267dd4b640 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a077efa0 0x5649a07aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267dd4b640 1 -- 172.21.15.70:0/120092110 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649a06af910 con 0x5649a07abf40 2023-10-08T05:53:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267e54c640 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 0x5649a077ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.472+0000 7f267ed4d640 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a077efa0 0x5649a07aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.473+0000 7f267dd4b640 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07ae8f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2668087b60 tx=0x7f2668057e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aeafbdc06cebedb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.473+0000 7f265bfff640 1 -- 172.21.15.70:0/120092110 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266805c570 con 0x5649a07abf40 2023-10-08T05:53:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.473+0000 7f265bfff640 1 -- 172.21.15.70:0/120092110 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2668060720 con 0x5649a07abf40 2023-10-08T05:53:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.473+0000 7f265bfff640 1 -- 172.21.15.70:0/120092110 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2668054b30 con 0x5649a07abf40 2023-10-08T05:53:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.474+0000 7f268341e740 1 -- 172.21.15.70:0/120092110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 msgr2=0x5649a07ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.474+0000 7f268341e740 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07ae8f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2668087b60 tx=0x7f2668057e40 comp rx=0 tx=0).stop 2023-10-08T05:53:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.474+0000 7f268341e740 1 -- 172.21.15.70:0/120092110 shutdown_connections 2023-10-08T05:53:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.474+0000 7f268341e740 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a077efa0 0x5649a07aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.474+0000 7f268341e740 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07ae8f0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.474+0000 7f268341e740 1 --2- 172.21.15.70:0/120092110 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 0x5649a077ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.474+0000 7f268341e740 1 -- 172.21.15.70:0/120092110 >> 172.21.15.70:0/120092110 conn(0x5649a0780d70 msgr2=0x5649a07c98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.474+0000 7f268341e740 1 -- 172.21.15.70:0/120092110 shutdown_connections 2023-10-08T05:53:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.474+0000 7f268341e740 1 -- 172.21.15.70:0/120092110 wait complete. 2023-10-08T05:53:12.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.475+0000 7f268341e740 1 -- start start 2023-10-08T05:53:12.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.475+0000 7f268341e740 1 Processor -- start 2023-10-08T05:53:12.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.476+0000 7f268341e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abb60 0x5649a07a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.476+0000 7f268341e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.476+0000 7f268341e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07ae070 0x5649a07b0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.476+0000 7f268341e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a0689970 con 0x5649a07abf40 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.476+0000 7f268341e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a0689610 con 0x5649a07abb60 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.476+0000 7f268341e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649a044ff20 con 0x5649a07ae070 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.476+0000 7f268341e740 10 client.0 _ll_register_callbacks cb 0x5649a06c7610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.476+0000 7f267dd4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.476+0000 7f267dd4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46754/0 (socket says 172.21.15.70:46754) 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.476+0000 7f267dd4b640 1 -- 172.21.15.70:0/1485209928 learned_addr learned my addr 172.21.15.70:0/1485209928 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f267e54c640 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abb60 0x5649a07a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f267dd4b640 1 -- 172.21.15.70:0/1485209928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07ae070 msgr2=0x5649a07b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f267ed4d640 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07ae070 0x5649a07b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f267dd4b640 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07ae070 0x5649a07b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f267dd4b640 1 -- 172.21.15.70:0/1485209928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abb60 msgr2=0x5649a07a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f267dd4b640 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abb60 0x5649a07a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f267dd4b640 1 -- 172.21.15.70:0/1485209928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780ca8c0 con 0x5649a07abf40 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f267dd4b640 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07a51a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2668056770 tx=0x7f2668012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98530faa89312fc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f26597fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f26597fa640 1 -- 172.21.15.70:0/1485209928 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266805c5f0 con 0x5649a07abf40 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f26597fa640 1 -- 172.21.15.70:0/1485209928 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f266805c160 con 0x5649a07abf40 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f268341e740 1 -- 172.21.15.70:0/1485209928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649a06e46c0 con 0x5649a07abf40 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f26597fa640 1 -- 172.21.15.70:0/1485209928 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f266805c360 con 0x5649a07abf40 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f268341e740 10 client.7263 fetch_fsmap learned FSMap version 41 2023-10-08T05:53:12.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.477+0000 7f268341e740 1 -- 172.21.15.70:0/1485209928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x5649a06e9d60 con 0x5649a07abf40 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.478+0000 7f26597fa640 1 -- 172.21.15.70:0/1485209928 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5649a06e46c0 con 0x5649a07abf40 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.478+0000 7f26597fa640 1 -- 172.21.15.70:0/1485209928 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f266805e040 con 0x5649a07abf40 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.478+0000 7f268341e740 10 client.7263 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.478+0000 7f268341e740 10 client.7263 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.478+0000 7f268341e740 20 client.7263 populate_metadata read hostname 'smithi070' 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.478+0000 7f268341e740 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.70:6834/24744490,v1:172.21.15.70:6835/24744490] conn(0x5649a07ae5c0 0x5649a07a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.478+0000 7f268341e740 4 client.7263 mds_command: new command op to 7131 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.478+0000 7f268341e740 1 -- 172.21.15.70:0/1485209928 --> [v2:172.21.15.70:6834/24744490,v1:172.21.15.70:6835/24744490] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649a06e43e0 con 0x5649a07ae5c0 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.478+0000 7f267e54c640 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.70:6834/24744490,v1:172.21.15.70:6835/24744490] conn(0x5649a07ae5c0 0x5649a07a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.479+0000 7f267e54c640 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.70:6834/24744490,v1:172.21.15.70:6835/24744490] conn(0x5649a07ae5c0 0x5649a07a72b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b7a93c6ab34a35 server_cookie=2032b55ac1765ffc in_seq=0 out_seq=0 2023-10-08T05:53:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.479+0000 7f26597fa640 10 client.7263 ms_handle_connect on v2:172.21.15.70:6834/24744490 2023-10-08T05:53:12.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f26597fa640 1 -- 172.21.15.70:0/1485209928 <== mds.0 v2:172.21.15.70:6834/24744490 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5649a06e43e0 con 0x5649a07ae5c0 2023-10-08T05:53:12.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f26597fa640 10 client.7263 handle_command_reply: tid=0 2023-10-08T05:53:12.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 client.7263 shutdown 2023-10-08T05:53:12.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 -- 172.21.15.70:0/1485209928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 msgr2=0x5649a07a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07a51a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2668056770 tx=0x7f2668012000 comp rx=0 tx=0).stop 2023-10-08T05:53:12.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 -- 172.21.15.70:0/1485209928 shutdown_connections 2023-10-08T05:53:12.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.70:6834/24744490,v1:172.21.15.70:6835/24744490] conn(0x5649a07ae5c0 0x5649a07a72b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07ae070 0x5649a07b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a07a51a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 --2- 172.21.15.70:0/1485209928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abb60 0x5649a07a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f26597fa640 0 client.7263 ms_handle_reset on v2:172.21.15.70:6834/24744490 2023-10-08T05:53:12.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 -- 172.21.15.70:0/1485209928 >> 172.21.15.70:0/1485209928 conn(0x5649a077efa0 msgr2=0x5649a0782910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:12.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 -- 172.21.15.70:0/1485209928 shutdown_connections 2023-10-08T05:53:12.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 1 -- 172.21.15.70:0/1485209928 wait complete. 2023-10-08T05:53:12.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.480+0000 7f268341e740 20 client.7263 trim_cache size 0 max 16384 2023-10-08T05:53:12.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.497+0000 7f268341e740 1 Processor -- start 2023-10-08T05:53:12.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.497+0000 7f268341e740 1 -- start start 2023-10-08T05:53:12.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.498+0000 7f268341e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abb60 0x5649a07cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.498+0000 7f268341e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a0782700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.498+0000 7f268341e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07ae070 0x5649a0782c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.498+0000 7f268341e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a077f4f0 con 0x5649a07abf40 2023-10-08T05:53:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.498+0000 7f268341e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a07af8e0 con 0x5649a07abb60 2023-10-08T05:53:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.498+0000 7f268341e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649a07a57b0 con 0x5649a07ae070 2023-10-08T05:53:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.498+0000 7f267dd4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a0782700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.498+0000 7f267dd4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a0782700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46764/0 (socket says 172.21.15.70:46764) 2023-10-08T05:53:12.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.498+0000 7f267dd4b640 1 -- 172.21.15.70:0/3992054920 learned_addr learned my addr 172.21.15.70:0/3992054920 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f267dd4b640 1 -- 172.21.15.70:0/3992054920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07ae070 msgr2=0x5649a0782c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f267ed4d640 1 --2- 172.21.15.70:0/3992054920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07ae070 0x5649a0782c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f267dd4b640 1 --2- 172.21.15.70:0/3992054920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07ae070 0x5649a0782c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f267dd4b640 1 -- 172.21.15.70:0/3992054920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abb60 msgr2=0x5649a07cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f267e54c640 1 --2- 172.21.15.70:0/3992054920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abb60 0x5649a07cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f267dd4b640 1 --2- 172.21.15.70:0/3992054920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abb60 0x5649a07cbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f267dd4b640 1 -- 172.21.15.70:0/3992054920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649a0688b80 con 0x5649a07abf40 2023-10-08T05:53:12.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f267dd4b640 1 --2- 172.21.15.70:0/3992054920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a0782700 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2668087f90 tx=0x7f2668083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69ce16ef0b7af2e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f266cff9640 1 -- 172.21.15.70:0/3992054920 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26680581f0 con 0x5649a07abf40 2023-10-08T05:53:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f266cff9640 1 -- 172.21.15.70:0/3992054920 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266805c570 con 0x5649a07abf40 2023-10-08T05:53:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.499+0000 7f266cff9640 1 -- 172.21.15.70:0/3992054920 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266805c160 con 0x5649a07abf40 2023-10-08T05:53:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.500+0000 7f268341e740 1 -- 172.21.15.70:0/3992054920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 msgr2=0x5649a0782700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.500+0000 7f268341e740 1 --2- 172.21.15.70:0/3992054920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a0782700 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2668087f90 tx=0x7f2668083040 comp rx=0 tx=0).stop 2023-10-08T05:53:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.500+0000 7f268341e740 1 -- 172.21.15.70:0/3992054920 shutdown_connections 2023-10-08T05:53:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.500+0000 7f268341e740 1 --2- 172.21.15.70:0/3992054920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07ae070 0x5649a0782c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.500+0000 7f268341e740 1 --2- 172.21.15.70:0/3992054920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07abf40 0x5649a0782700 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.500+0000 7f268341e740 1 --2- 172.21.15.70:0/3992054920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abb60 0x5649a07cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.500+0000 7f268341e740 1 -- 172.21.15.70:0/3992054920 >> 172.21.15.70:0/3992054920 conn(0x5649a077efa0 msgr2=0x5649a077da40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:12.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.500+0000 7f268341e740 1 -- 172.21.15.70:0/3992054920 shutdown_connections 2023-10-08T05:53:12.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.500+0000 7f268341e740 1 -- 172.21.15.70:0/3992054920 wait complete. 2023-10-08T05:53:12.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.501+0000 7f268341e740 1 -- start start 2023-10-08T05:53:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.501+0000 7f268341e740 1 Processor -- start 2023-10-08T05:53:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.502+0000 7f268341e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 0x5649a081a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.502+0000 7f268341e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abf40 0x5649a07cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.502+0000 7f268341e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07ae070 0x5649a0831ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.502+0000 7f268341e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a0832a00 con 0x5649a07ae070 2023-10-08T05:53:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.502+0000 7f268341e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a07cd730 con 0x5649a07abf40 2023-10-08T05:53:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.502+0000 7f268341e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649a0784080 con 0x5649a07abb60 2023-10-08T05:53:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.502+0000 7f268341e740 10 client.0 _ll_register_callbacks cb 0x7f26680015e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:53:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.502+0000 7f267ed4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07ae070 0x5649a0831ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f267ed4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07ae070 0x5649a0831ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46774/0 (socket says 172.21.15.70:46774) 2023-10-08T05:53:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f267ed4d640 1 -- 172.21.15.70:0/1281376743 learned_addr learned my addr 172.21.15.70:0/1281376743 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f267dd4b640 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abf40 0x5649a07cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f267e54c640 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 0x5649a081a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f267ed4d640 1 -- 172.21.15.70:0/1281376743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 msgr2=0x5649a081a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f267ed4d640 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 0x5649a081a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f267ed4d640 1 -- 172.21.15.70:0/1281376743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abf40 msgr2=0x5649a07cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f267ed4d640 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abf40 0x5649a07cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f267ed4d640 1 -- 172.21.15.70:0/1281376743 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780ca8c0 con 0x5649a07ae070 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f267ed4d640 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07ae070 0x5649a0831ee0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f267000e230 tx=0x7f2670004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765dedef47ea1764 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.503+0000 7f26597fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f26597fa640 1 -- 172.21.15.70:0/1281376743 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670034640 con 0x5649a07ae070 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f26597fa640 1 -- 172.21.15.70:0/1281376743 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f26700347a0 con 0x5649a07ae070 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f268341e740 1 -- 172.21.15.70:0/1281376743 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2668060720 con 0x5649a07ae070 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f26597fa640 1 -- 172.21.15.70:0/1281376743 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2670034960 con 0x5649a07ae070 2023-10-08T05:53:12.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f268341e740 10 client.7269 fetch_fsmap learned FSMap version 41 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f268341e740 1 -- 172.21.15.70:0/1281376743 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x5649a06ae170 con 0x5649a07ae070 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f26597fa640 1 -- 172.21.15.70:0/1281376743 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2668060720 con 0x5649a07ae070 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f26597fa640 1 -- 172.21.15.70:0/1281376743 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f267003e040 con 0x5649a07ae070 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f268341e740 10 client.7269 fetch_fsmap finished waiting for FSMap version 41 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f268341e740 10 client.7269 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f268341e740 20 client.7269 populate_metadata read hostname 'smithi070' 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f268341e740 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.70:6834/24744490,v1:172.21.15.70:6835/24744490] conn(0x5649a07b0bc0 0x5649a07b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f268341e740 4 client.7269 mds_command: new command op to 7131 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.504+0000 7f268341e740 1 -- 172.21.15.70:0/1281376743 --> [v2:172.21.15.70:6834/24744490,v1:172.21.15.70:6835/24744490] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5649a03ed6c0 con 0x5649a07b0bc0 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.505+0000 7f267e54c640 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.70:6834/24744490,v1:172.21.15.70:6835/24744490] conn(0x5649a07b0bc0 0x5649a07b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.505+0000 7f267e54c640 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.70:6834/24744490,v1:172.21.15.70:6835/24744490] conn(0x5649a07b0bc0 0x5649a07b0fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a34bca52fc73a9af server_cookie=a991f08932485dba in_seq=0 out_seq=0 2023-10-08T05:53:12.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.505+0000 7f26597fa640 10 client.7269 ms_handle_connect on v2:172.21.15.70:6834/24744490 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.505+0000 7f26597fa640 1 -- 172.21.15.70:0/1281376743 <== mds.0 v2:172.21.15.70:6834/24744490 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5649a03ed6c0 con 0x5649a07b0bc0 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.505+0000 7f26597fa640 10 client.7269 handle_command_reply: tid=0 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.505+0000 7f268341e740 1 client.7269 shutdown 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 1 -- 172.21.15.70:0/1281376743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07ae070 msgr2=0x5649a0831ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07ae070 0x5649a0831ee0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f267000e230 tx=0x7f2670004ab0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 1 -- 172.21.15.70:0/1281376743 shutdown_connections 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.70:6834/24744490,v1:172.21.15.70:6835/24744490] conn(0x5649a07b0bc0 0x5649a07b0fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5649a07ae070 0x5649a0831ee0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5649a07abf40 0x5649a07cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 1 --2- 172.21.15.70:0/1281376743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5649a07abb60 0x5649a081a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f26597fa640 0 client.7269 ms_handle_reset on v2:172.21.15.70:6834/24744490 2023-10-08T05:53:12.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 1 -- 172.21.15.70:0/1281376743 >> 172.21.15.70:0/1281376743 conn(0x5649a077efa0 msgr2=0x5649a07cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:12.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 1 -- 172.21.15.70:0/1281376743 shutdown_connections 2023-10-08T05:53:12.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 1 -- 172.21.15.70:0/1281376743 wait complete. 2023-10-08T05:53:12.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.506+0000 7f268341e740 20 client.7269 trim_cache size 0 max 16384 2023-10-08T05:53:12.512 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:53:12.512 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T05:53:12.512 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T05:53:12.512 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 7131, 2023-10-08T05:53:12.512 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T05:53:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T05:53:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T05:53:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.7262063349999996, 2023-10-08T05:53:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 41, 2023-10-08T05:53:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 104, 2023-10-08T05:53:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 104, 2023-10-08T05:53:12.513 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 13.125103663000001 2023-10-08T05:53:12.513 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:53:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 -- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f265c06af30 msgr2=0x7f265c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f265c06af30 0x7f265c06d3f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2674009a90 tx=0x7f2674008040 comp rx=0 tx=0).stop 2023-10-08T05:53:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 -- 172.21.15.70:0/2831890444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780c7b20 msgr2=0x7f267814a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780c7b20 0x7f267814a7e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f266800e820 tx=0x7f266800ece0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 -- 172.21.15.70:0/2831890444 shutdown_connections 2023-10-08T05:53:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f265c06af30 0x7f265c06d3f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f267814ad20 0x7f26780c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26780c7b20 0x7f267814a7e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 --2- 172.21.15.70:0/2831890444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2664004d00 0x7f26780c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 -- 172.21.15.70:0/2831890444 >> 172.21.15.70:0/2831890444 conn(0x7f267806e630 msgr2=0x7f267815b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 -- 172.21.15.70:0/2831890444 shutdown_connections 2023-10-08T05:53:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.507+0000 7f26807b0640 1 -- 172.21.15.70:0/2831890444 wait complete. 2023-10-08T05:53:12.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:53:12.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.728+0000 7fcd83c7c640 1 Processor -- start 2023-10-08T05:53:12.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.728+0000 7fcd83c7c640 1 -- start start 2023-10-08T05:53:12.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.730+0000 7fcd83c7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd7c0c90b0 0x7fcd7c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.730+0000 7fcd83c7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd7c0cb3b0 0x7fcd7c15e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.730+0000 7fcd83c7c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c15b2b0 con 0x7fcd7c15ab40 2023-10-08T05:53:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.730+0000 7fcd83c7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c15bf30 con 0x7fcd7c0c90b0 2023-10-08T05:53:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.730+0000 7fcd83c7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0cb820 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.730+0000 7fcd81217640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd7c0c90b0 0x7fcd7c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.730+0000 7fcd81a18640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcd7c15ab40 0x7fcd7c0ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37152/0 (socket says 172.21.15.70:37152) 2023-10-08T05:53:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.730+0000 7fcd81a18640 1 -- 172.21.15.70:0/4146345709 learned_addr learned my addr 172.21.15.70:0/4146345709 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:12.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.731+0000 7fcd82219640 1 --2- 172.21.15.70:0/4146345709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd7c0cb3b0 0x7fcd7c15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.731+0000 7fcd81217640 1 -- 172.21.15.70:0/4146345709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd7c0cb3b0 msgr2=0x7fcd7c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:12.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.731+0000 7fcd81217640 1 --2- 172.21.15.70:0/4146345709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd7c0cb3b0 0x7fcd7c15e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.731+0000 7fcd81217640 1 -- 172.21.15.70:0/4146345709 >> v1:172.21.15.70:6789/0 conn(0x7fcd7c15ab40 legacy=0x7fcd7c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.731+0000 7fcd81217640 1 -- 172.21.15.70:0/4146345709 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c15b590 con 0x7fcd7c0c90b0 2023-10-08T05:53:12.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.732+0000 7fcd81217640 1 --2- 172.21.15.70:0/4146345709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0c90b0 0x7fcd7c0cae70 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0097d0 tx=0x7fcd6c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387d534cf5db3880 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:12.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.732+0000 7fcd80a16640 1 -- 172.21.15.70:0/4146345709 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c02d040 con 0x7fcd7c0c90b0 2023-10-08T05:53:12.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.732+0000 7fcd80a16640 1 -- 172.21.15.70:0/4146345709 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c031810 con 0x7fcd7c0c90b0 2023-10-08T05:53:12.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.732+0000 7fcd80a16640 1 -- 172.21.15.70:0/4146345709 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c031ba0 con 0x7fcd7c0c90b0 2023-10-08T05:53:12.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.733+0000 7fcd83c7c640 1 -- 172.21.15.70:0/4146345709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0c90b0 msgr2=0x7fcd7c0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.733+0000 7fcd83c7c640 1 --2- 172.21.15.70:0/4146345709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0c90b0 0x7fcd7c0cae70 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0097d0 tx=0x7fcd6c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.733+0000 7fcd83c7c640 1 -- 172.21.15.70:0/4146345709 shutdown_connections 2023-10-08T05:53:12.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.733+0000 7fcd83c7c640 1 --2- 172.21.15.70:0/4146345709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd7c0cb3b0 0x7fcd7c15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.733+0000 7fcd83c7c640 1 --2- 172.21.15.70:0/4146345709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0c90b0 0x7fcd7c0cae70 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.733+0000 7fcd83c7c640 1 -- 172.21.15.70:0/4146345709 >> 172.21.15.70:0/4146345709 conn(0x7fcd7c06a380 msgr2=0x7fcd7c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:12.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.733+0000 7fcd83c7c640 1 -- 172.21.15.70:0/4146345709 shutdown_connections 2023-10-08T05:53:12.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.733+0000 7fcd83c7c640 1 -- 172.21.15.70:0/4146345709 wait complete. 2023-10-08T05:53:12.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.735+0000 7fcd83c7c640 1 Processor -- start 2023-10-08T05:53:12.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.735+0000 7fcd83c7c640 1 -- start start 2023-10-08T05:53:12.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.736+0000 7fcd83c7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd7c0c90b0 0x7fcd7c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.736+0000 7fcd83c7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0cb3b0 0x7fcd7c158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.736+0000 7fcd83c7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd7c15ab40 0x7fcd7c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.736+0000 7fcd83c7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1f4410 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.736+0000 7fcd83c7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1f4830 con 0x7fcd7c15ab40 2023-10-08T05:53:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.736+0000 7fcd83c7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c158830 con 0x7fcd7c0c90b0 2023-10-08T05:53:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.736+0000 7fcd81217640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0cb3b0 0x7fcd7c158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.736+0000 7fcd81217640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0cb3b0 0x7fcd7c158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46784/0 (socket says 172.21.15.70:46784) 2023-10-08T05:53:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.736+0000 7fcd81217640 1 -- 172.21.15.70:0/196926895 learned_addr learned my addr 172.21.15.70:0/196926895 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd81217640 1 -- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd7c0c90b0 msgr2=0x7fcd7c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd81217640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd7c0c90b0 0x7fcd7c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd81217640 1 -- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd7c15ab40 msgr2=0x7fcd7c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd81217640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd7c15ab40 0x7fcd7c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd81217640 1 -- 172.21.15.70:0/196926895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c009400 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd81217640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0cb3b0 0x7fcd7c158130 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0097d0 tx=0x7fcd6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd727fc640 1 -- 172.21.15.70:0/196926895 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c03ad60 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd727fc640 1 -- 172.21.15.70:0/196926895 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c0443f0 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd727fc640 1 -- 172.21.15.70:0/196926895 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0447a0 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd83c7c640 1 -- 172.21.15.70:0/196926895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c1f3a70 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.737+0000 7fcd83c7c640 1 -- 172.21.15.70:0/196926895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c1f4030 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.739+0000 7fcd83c7c640 1 -- 172.21.15.70:0/196926895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c1f41c0 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.740+0000 7fcd727fc640 1 -- 172.21.15.70:0/196926895 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fcd6c04d450 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.740+0000 7fcd727fc640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fcd6006af30 0x7fcd6006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.741+0000 7fcd727fc640 1 -- 172.21.15.70:0/196926895 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcd6c0abb00 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.741+0000 7fcd81a18640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fcd6006af30 0x7fcd6006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:12.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.742+0000 7fcd81a18640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fcd6006af30 0x7fcd6006d3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005880 tx=0x7fcd78007a00 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.745+0000 7fcd727fc640 1 -- 172.21.15.70:0/196926895 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd6c076a60 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.976+0000 7fcd83c7c640 1 -- 172.21.15.70:0/196926895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcd7c067490 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.976+0000 7fcd727fc640 1 -- 172.21.15.70:0/196926895 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcd6c035070 con 0x7fcd7c0cb3b0 2023-10-08T05:53:12.978 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T05:53:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 -- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fcd6006af30 msgr2=0x7fcd6006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fcd6006af30 0x7fcd6006d3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd78005880 tx=0x7fcd78007a00 comp rx=0 tx=0).stop 2023-10-08T05:53:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 -- 172.21.15.70:0/196926895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0cb3b0 msgr2=0x7fcd7c158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0cb3b0 0x7fcd7c158130 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0097d0 tx=0x7fcd6c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 -- 172.21.15.70:0/196926895 shutdown_connections 2023-10-08T05:53:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7fcd6006af30 0x7fcd6006d3f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd7c15ab40 0x7fcd7c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7c0cb3b0 0x7fcd7c158130 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 --2- 172.21.15.70:0/196926895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd7c0c90b0 0x7fcd7c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:12.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 -- 172.21.15.70:0/196926895 >> 172.21.15.70:0/196926895 conn(0x7fcd7c06a380 msgr2=0x7fcd7c0c9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:12.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.979+0000 7fcd83c7c640 1 -- 172.21.15.70:0/196926895 shutdown_connections 2023-10-08T05:53:12.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:12.980+0000 7fcd83c7c640 1 -- 172.21.15.70:0/196926895 wait complete. 2023-10-08T05:53:12.998 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T05:53:12.998 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:53:12.998 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:53:12.998 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T05:53:12.998 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T05:53:12.998 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T05:53:12.998 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T05:53:12.998 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:53:12.998 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:53:12.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:12.999 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:53:13.053 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:53:13.054 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:53:13.054 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:53:13.054 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:13.054 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:53:13.054 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:13.054 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:53:13.054 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:53:13.054 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:53:13.054 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41403sec preferred_lft 41403sec 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:13.055 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T05:53:13.056 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T05:53:13.057 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:13.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:13.058 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:53:13.148 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:53:13.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:13.155 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:53:13.210 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:53:13.210 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:53:13.210 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:53:13.210 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:53:13.210 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:53:13.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:13.211 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:53:13.211 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:53:13.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:53:13.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:53:13.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:53:13.211 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:53:13.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:13.400 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:53:13.451 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T05:53:13.451 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T05:53:13.451 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:53:13.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:13.451 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T05:53:13.540 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:53:13.540 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T05:53:13.592 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:53:13.593 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:53:13.593 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:53:13.645 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T05:53:13.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:13.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:53:13.787 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:53:13.788 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:53:13.825 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:53:13.825 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:53:13.826 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:53:13.826 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:53:13.826 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:53:13.827 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T05:53:13.827 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:53:13.828 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:53:13.828 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:53:13.828 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:53:13.831 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:53:13.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:13.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:53:13.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:13.911 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:53:13.977 INFO:teuthology.orchestra.run.smithi070.stdout:client5441 2023-10-08T05:53:13.978 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:53:13.978 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:14.030 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T05:53:14.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.247+0000 7f81356d0640 1 Processor -- start 2023-10-08T05:53:14.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.248+0000 7f81356d0640 1 -- start start 2023-10-08T05:53:14.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.249+0000 7f81356d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813006c3a0 0x7f81300c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.249+0000 7f81356d0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f813015aa00 0x7f81300b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.249+0000 7f81356d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813015b310 con 0x7f813006db60 2023-10-08T05:53:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.249+0000 7f81356d0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81300ca230 con 0x7f813015aa00 2023-10-08T05:53:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.249+0000 7f81356d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813006d290 con 0x7f813006c3a0 2023-10-08T05:53:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.250+0000 7f812e7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f813006db60 0x7f813006c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37790/0 (socket says 172.21.15.70:37790) 2023-10-08T05:53:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.250+0000 7f812e7fc640 1 -- 172.21.15.70:0/294212221 learned_addr learned my addr 172.21.15.70:0/294212221 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.250+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859531776 0 0) 0x7f8118001110 con 0x7f813006db60 2023-10-08T05:53:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.250+0000 7f812f7fe640 1 --2- 172.21.15.70:0/294212221 >> v2:172.21.15.181:3300/0 conn(0x7f813015aa00 0x7f81300b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.250+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f811c003520 con 0x7f813006db60 2023-10-08T05:53:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812effd640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813006c3a0 0x7f81300c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3111426315 0 0) 0x7f811c003520 con 0x7f813006db60 2023-10-08T05:53:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813006c3a0 msgr2=0x7f81300c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813006c3a0 0x7f81300c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 >> v2:172.21.15.181:3300/0 conn(0x7f813015aa00 msgr2=0x7f81300b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 --2- 172.21.15.70:0/294212221 >> v2:172.21.15.181:3300/0 conn(0x7f813015aa00 0x7f81300b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300c9860 con 0x7f813006db60 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f8118001800 con 0x7f813006db60 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 >> v1:172.21.15.70:6789/0 conn(0x7f813006db60 legacy=0x7f813006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813006c3a0 0x7f811c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c003f40 0x7f811c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f811c004d00 0x7f811c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813006d290 con 0x7f811c004d00 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c00b9b0 con 0x7f811c003f40 2023-10-08T05:53:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c00bb70 con 0x7f813006c3a0 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.251+0000 7f812f7fe640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f811c004d00 0x7f811c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.252+0000 7f812e7fc640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c003f40 0x7f811c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.252+0000 7f812f7fe640 1 -- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813006c3a0 msgr2=0x7f811c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.252+0000 7f812effd640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813006c3a0 0x7f811c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.252+0000 7f812f7fe640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f813006c3a0 0x7f811c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.252+0000 7f812f7fe640 1 -- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c003f40 msgr2=0x7f811c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.252+0000 7f812f7fe640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c003f40 0x7f811c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.252+0000 7f812f7fe640 1 -- 172.21.15.70:0/294212221 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c003520 con 0x7f811c004d00 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.252+0000 7f812e7fc640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f811c003f40 0x7f811c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.253+0000 7f812f7fe640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f811c004d00 0x7f811c005180 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8128009ac0 tx=0x7f811c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589f9828490016b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.253+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812802c710 con 0x7f811c004d00 2023-10-08T05:53:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.253+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812802c870 con 0x7f811c004d00 2023-10-08T05:53:14.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.253+0000 7f812dffb640 1 -- 172.21.15.70:0/294212221 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812802cc00 con 0x7f811c004d00 2023-10-08T05:53:14.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.254+0000 7f81356d0640 1 -- 172.21.15.70:0/294212221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f811c004d00 msgr2=0x7f811c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:14.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.254+0000 7f81356d0640 1 --2- 172.21.15.70:0/294212221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f811c004d00 0x7f811c005180 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8128009ac0 tx=0x7f811c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.254+0000 7f812f7fe640 1 -- 172.21.15.70:0/294212221 reap_dead start 2023-10-08T05:53:14.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.254+0000 7f81356d0640 1 -- 172.21.15.70:0/294212221 shutdown_connections 2023-10-08T05:53:14.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.254+0000 7f81356d0640 1 -- 172.21.15.70:0/294212221 >> 172.21.15.70:0/294212221 conn(0x7f813006a5d0 msgr2=0x7f81300b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:14.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.254+0000 7f81356d0640 1 -- 172.21.15.70:0/294212221 shutdown_connections 2023-10-08T05:53:14.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.254+0000 7f81356d0640 1 -- 172.21.15.70:0/294212221 wait complete. 2023-10-08T05:53:14.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.256+0000 7f81356d0640 1 Processor -- start 2023-10-08T05:53:14.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.256+0000 7f81356d0640 1 -- start start 2023-10-08T05:53:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.257+0000 7f81356d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f811c004d00 0x7f81301522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.257+0000 7f81356d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81300c41f0 0x7f81300c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.257+0000 7f81356d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81300c5d50 0x7f81300701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.257+0000 7f81356d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8130152830 con 0x7f81300c5d50 2023-10-08T05:53:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.257+0000 7f81356d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81300c61d0 con 0x7f81300c41f0 2023-10-08T05:53:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.257+0000 7f81356d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81300c64b0 con 0x7f811c004d00 2023-10-08T05:53:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.257+0000 7f812f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81300c5d50 0x7f81300701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.257+0000 7f812f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81300c5d50 0x7f81300701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46532/0 (socket says 172.21.15.70:46532) 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.257+0000 7f812f7fe640 1 -- 172.21.15.70:0/1140766247 learned_addr learned my addr 172.21.15.70:0/1140766247 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f812effd640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f811c004d00 0x7f81301522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f812f7fe640 1 -- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f811c004d00 msgr2=0x7f81301522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f812f7fe640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f811c004d00 0x7f81301522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f812f7fe640 1 -- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81300c41f0 msgr2=0x7f81300c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f812f7fe640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81300c41f0 0x7f81300c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f812f7fe640 1 -- 172.21.15.70:0/1140766247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300c39f0 con 0x7f81300c5d50 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f812f7fe640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81300c5d50 0x7f81300701b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8128037040 tx=0x7f8128004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f81277fe640 1 -- 172.21.15.70:0/1140766247 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812802c710 con 0x7f81300c5d50 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f81277fe640 1 -- 172.21.15.70:0/1140766247 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8128003830 con 0x7f81300c5d50 2023-10-08T05:53:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f81277fe640 1 -- 172.21.15.70:0/1140766247 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128003be0 con 0x7f81300c5d50 2023-10-08T05:53:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.258+0000 7f81356d0640 1 -- 172.21.15.70:0/1140766247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81300c3cd0 con 0x7f81300c5d50 2023-10-08T05:53:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.259+0000 7f81356d0640 1 -- 172.21.15.70:0/1140766247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8130152db0 con 0x7f81300c5d50 2023-10-08T05:53:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.260+0000 7f81356d0640 1 -- 172.21.15.70:0/1140766247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813004e160 con 0x7f81300c5d50 2023-10-08T05:53:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.261+0000 7f81277fe640 1 -- 172.21.15.70:0/1140766247 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8128003d40 con 0x7f81300c5d50 2023-10-08T05:53:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.261+0000 7f81277fe640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f811006af00 0x7f811006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.262+0000 7f81277fe640 1 -- 172.21.15.70:0/1140766247 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f81280aacd0 con 0x7f81300c5d50 2023-10-08T05:53:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.262+0000 7f812effd640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f811006af00 0x7f811006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.263+0000 7f812effd640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f811006af00 0x7f811006d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8120009df0 tx=0x7f812000b040 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:14.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.266+0000 7f81277fe640 1 -- 172.21.15.70:0/1140766247 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8128075c80 con 0x7f81300c5d50 2023-10-08T05:53:14.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.479+0000 7f81356d0640 1 -- 172.21.15.70:0/1140766247 --> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f81300c3870 con 0x7f811006af00 2023-10-08T05:53:14.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.484+0000 7f81277fe640 1 -- 172.21.15.70:0/1140766247 <== mgr.5737 v2:172.21.15.181:6832/2060337708 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f812000f040 con 0x7f811006af00 2023-10-08T05:53:14.484 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:53:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:14.484 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T05:53:14.485 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:14.485 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:53:14.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.486+0000 7f81356d0640 1 -- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f811006af00 msgr2=0x7f811006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:14.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.486+0000 7f81356d0640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f811006af00 0x7f811006d3c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8120009df0 tx=0x7f812000b040 comp rx=0 tx=0).stop 2023-10-08T05:53:14.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.486+0000 7f81356d0640 1 -- 172.21.15.70:0/1140766247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81300c5d50 msgr2=0x7f81300701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:14.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.486+0000 7f81356d0640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81300c5d50 0x7f81300701b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8128037040 tx=0x7f8128004730 comp rx=0 tx=0).stop 2023-10-08T05:53:14.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.487+0000 7f81356d0640 1 -- 172.21.15.70:0/1140766247 shutdown_connections 2023-10-08T05:53:14.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.487+0000 7f81356d0640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f811006af00 0x7f811006d3c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.487+0000 7f81356d0640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81300c5d50 0x7f81300701b0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.487+0000 7f81356d0640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81300c41f0 0x7f81300c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.487+0000 7f81356d0640 1 --2- 172.21.15.70:0/1140766247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f811c004d00 0x7f81301522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.487+0000 7f81356d0640 1 -- 172.21.15.70:0/1140766247 >> 172.21.15.70:0/1140766247 conn(0x7f813006a5d0 msgr2=0x7f81300ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:14.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.487+0000 7f81356d0640 1 -- 172.21.15.70:0/1140766247 shutdown_connections 2023-10-08T05:53:14.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.487+0000 7f81356d0640 1 -- 172.21.15.70:0/1140766247 wait complete. 2023-10-08T05:53:14.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T05:53:14.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.716+0000 7f867759e640 1 Processor -- start 2023-10-08T05:53:14.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.716+0000 7f867759e640 1 -- start start 2023-10-08T05:53:14.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.718+0000 7f867759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780cdc30 0x7f86780cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.718+0000 7f867759e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86780cce90 0x7f8678159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.718+0000 7f867759e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867815ac90 con 0x7f86780caef0 2023-10-08T05:53:14.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.718+0000 7f867759e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867815a6d0 con 0x7f86780cce90 2023-10-08T05:53:14.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.718+0000 7f867759e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867815a9b0 con 0x7f86780cdc30 2023-10-08T05:53:14.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.718+0000 7f867659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f86780caef0 0x7f86780cb2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37806/0 (socket says 172.21.15.70:37806) 2023-10-08T05:53:14.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.718+0000 7f867659c640 1 -- 172.21.15.70:0/592358150 learned_addr learned my addr 172.21.15.70:0/592358150 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:14.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.718+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439045377 0 0) 0x7f867815ac90 con 0x7f86780caef0 2023-10-08T05:53:14.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.718+0000 7f8676d9d640 1 --2- 172.21.15.70:0/592358150 >> v2:172.21.15.181:3300/0 conn(0x7f86780cce90 0x7f8678159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.718+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8660003520 con 0x7f86780caef0 2023-10-08T05:53:14.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f8675d9b640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780cdc30 0x7f86780cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2030767504 0 0) 0x7f8660003520 con 0x7f86780caef0 2023-10-08T05:53:14.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780cdc30 msgr2=0x7f86780cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:14.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780cdc30 0x7f86780cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 >> v2:172.21.15.181:3300/0 conn(0x7f86780cce90 msgr2=0x7f8678159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:14.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 --2- 172.21.15.70:0/592358150 >> v2:172.21.15.181:3300/0 conn(0x7f86780cce90 0x7f8678159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780cc150 con 0x7f86780caef0 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f8668001670 con 0x7f86780caef0 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f8676d9d640 1 --2- 172.21.15.70:0/592358150 >> v2:172.21.15.181:3300/0 conn(0x7f86780cce90 0x7f8678159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 >> v1:172.21.15.70:6789/0 conn(0x7f86780caef0 legacy=0x7f86780cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8660003970 0x7f8660003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86600047c0 0x7f8660004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780cdc30 0x7f8660005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867815a6d0 con 0x7f8660003970 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867815a9b0 con 0x7f86600047c0 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866000bab0 con 0x7f86780cdc30 2023-10-08T05:53:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.719+0000 7f867659c640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8660003970 0x7f8660003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.720+0000 7f8675d9b640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86600047c0 0x7f8660004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.720+0000 7f867659c640 1 -- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780cdc30 msgr2=0x7f8660005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.720+0000 7f8676d9d640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780cdc30 0x7f8660005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.720+0000 7f867659c640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86780cdc30 0x7f8660005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.720+0000 7f867659c640 1 -- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86600047c0 msgr2=0x7f8660004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.720+0000 7f867659c640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86600047c0 0x7f8660004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.720+0000 7f867659c640 1 -- 172.21.15.70:0/592358150 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8660003520 con 0x7f8660003970 2023-10-08T05:53:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.720+0000 7f8675d9b640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86600047c0 0x7f8660004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.721+0000 7f867659c640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8660003970 0x7f8660003d90 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f866800c0e0 tx=0x7f86600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d14542bcef43ecb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:14.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.721+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8668005ad0 con 0x7f8660003970 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.721+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668005c50 con 0x7f8660003970 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.721+0000 7f867559a640 1 -- 172.21.15.70:0/592358150 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866802c7e0 con 0x7f8660003970 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.722+0000 7f867759e640 1 -- 172.21.15.70:0/592358150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8660003970 msgr2=0x7f8660003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.722+0000 7f867759e640 1 --2- 172.21.15.70:0/592358150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8660003970 0x7f8660003d90 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f866800c0e0 tx=0x7f86600057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.722+0000 7f8676d9d640 1 -- 172.21.15.70:0/592358150 reap_dead start 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.722+0000 7f867759e640 1 -- 172.21.15.70:0/592358150 shutdown_connections 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.722+0000 7f867759e640 1 -- 172.21.15.70:0/592358150 >> 172.21.15.70:0/592358150 conn(0x7f867806c010 msgr2=0x7f867806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.722+0000 7f867759e640 1 -- 172.21.15.70:0/592358150 shutdown_connections 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.722+0000 7f867759e640 1 -- 172.21.15.70:0/592358150 wait complete. 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.724+0000 7f867759e640 1 Processor -- start 2023-10-08T05:53:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.724+0000 7f867759e640 1 -- start start 2023-10-08T05:53:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.725+0000 7f867759e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8660003970 0x7f86780c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.725+0000 7f867759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f867814a7f0 0x7f867814ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.725+0000 7f867759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86780c92c0 0x7f86780c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.725+0000 7f867759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867814b1b0 con 0x7f86780c92c0 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.725+0000 7f867759e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c84f0 con 0x7f8660003970 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.725+0000 7f867759e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c9cb0 con 0x7f867814a7f0 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f8676d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86780c92c0 0x7f86780c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f8676d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86780c92c0 0x7f86780c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46548/0 (socket says 172.21.15.70:46548) 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f8676d9d640 1 -- 172.21.15.70:0/2592290462 learned_addr learned my addr 172.21.15.70:0/2592290462 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f867659c640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8660003970 0x7f86780c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f8675d9b640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f867814a7f0 0x7f867814ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f8676d9d640 1 -- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f867814a7f0 msgr2=0x7f867814ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f8676d9d640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f867814a7f0 0x7f867814ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f8676d9d640 1 -- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8660003970 msgr2=0x7f86780c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f8676d9d640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8660003970 0x7f86780c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f8676d9d640 1 -- 172.21.15.70:0/2592290462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8678148a30 con 0x7f86780c92c0 2023-10-08T05:53:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.726+0000 7f8676d9d640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86780c92c0 0x7f86780c9770 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f867815d270 tx=0x7f8670004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.727+0000 7f8666ffd640 1 -- 172.21.15.70:0/2592290462 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670007ca0 con 0x7f86780c92c0 2023-10-08T05:53:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.727+0000 7f8666ffd640 1 -- 172.21.15.70:0/2592290462 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8670007e00 con 0x7f86780c92c0 2023-10-08T05:53:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.727+0000 7f8666ffd640 1 -- 172.21.15.70:0/2592290462 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670014640 con 0x7f86780c92c0 2023-10-08T05:53:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.727+0000 7f867759e640 1 -- 172.21.15.70:0/2592290462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86780c78e0 con 0x7f86780c92c0 2023-10-08T05:53:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.727+0000 7f867759e640 1 -- 172.21.15.70:0/2592290462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86780c7ea0 con 0x7f86780c92c0 2023-10-08T05:53:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.728+0000 7f867759e640 1 -- 172.21.15.70:0/2592290462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8678074150 con 0x7f86780c92c0 2023-10-08T05:53:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.729+0000 7f8666ffd640 1 -- 172.21.15.70:0/2592290462 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 37) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f867001d3b0 con 0x7f86780c92c0 2023-10-08T05:53:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.730+0000 7f8666ffd640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f865806af30 0x7f865806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:14.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.730+0000 7f8666ffd640 1 -- 172.21.15.70:0/2592290462 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f8670089410 con 0x7f86780c92c0 2023-10-08T05:53:14.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.731+0000 7f867659c640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f865806af30 0x7f865806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.732+0000 7f867659c640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f865806af30 0x7f865806d3f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8668034000 tx=0x7f8668007000 comp rx=0 tx=0).ready entity=mgr.5737 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.734+0000 7f8666ffd640 1 -- 172.21.15.70:0/2592290462 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8670054330 con 0x7f86780c92c0 2023-10-08T05:53:14.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:14.972+0000 7f867759e640 1 -- 172.21.15.70:0/2592290462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f86780cbdd0 con 0x7f86780c92c0 2023-10-08T05:53:15.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.358+0000 7f8666ffd640 1 -- 172.21.15.70:0/2592290462 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v38) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8670057950 con 0x7f86780c92c0 2023-10-08T05:53:15.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.362+0000 7f867759e640 1 -- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f865806af30 msgr2=0x7f865806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:15.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.362+0000 7f867759e640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f865806af30 0x7f865806d3f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8668034000 tx=0x7f8668007000 comp rx=0 tx=0).stop 2023-10-08T05:53:15.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.362+0000 7f867759e640 1 -- 172.21.15.70:0/2592290462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86780c92c0 msgr2=0x7f86780c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:15.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.362+0000 7f867759e640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86780c92c0 0x7f86780c9770 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f867815d270 tx=0x7f8670004770 comp rx=0 tx=0).stop 2023-10-08T05:53:15.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.363+0000 7f867759e640 1 -- 172.21.15.70:0/2592290462 shutdown_connections 2023-10-08T05:53:15.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.363+0000 7f867759e640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f865806af30 0x7f865806d3f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.363+0000 7f867759e640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86780c92c0 0x7f86780c9770 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.363+0000 7f867759e640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f867814a7f0 0x7f867814ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.363+0000 7f867759e640 1 --2- 172.21.15.70:0/2592290462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8660003970 0x7f86780c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.363+0000 7f867759e640 1 -- 172.21.15.70:0/2592290462 >> 172.21.15.70:0/2592290462 conn(0x7f867806c010 msgr2=0x7f86780c55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:15.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.363+0000 7f867759e640 1 -- 172.21.15.70:0/2592290462 shutdown_connections 2023-10-08T05:53:15.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.364+0000 7f867759e640 1 -- 172.21.15.70:0/2592290462 wait complete. 2023-10-08T05:53:15.387 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T05:53:15.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.611+0000 7f345c232640 1 Processor -- start 2023-10-08T05:53:15.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.611+0000 7f345c232640 1 -- start start 2023-10-08T05:53:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f345c232640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f345415ce10 0x7f34540ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f345c232640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345406bc80 0x7f34540cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f345c232640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34540cf840 con 0x7f34540cf3e0 2023-10-08T05:53:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f345c232640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345406cd90 con 0x7f345415ce10 2023-10-08T05:53:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f345c232640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3454160ca0 con 0x7f345406bc80 2023-10-08T05:53:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f3459fce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f345415ce10 0x7f34540ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f34597cd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f34540cf3e0 0x7f345406b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37810/0 (socket says 172.21.15.70:37810) 2023-10-08T05:53:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f34597cd640 1 -- 172.21.15.70:0/1671104009 learned_addr learned my addr 172.21.15.70:0/1671104009 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f345a7cf640 1 --2- 172.21.15.70:0/1671104009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345406bc80 0x7f34540cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f3459fce640 1 -- 172.21.15.70:0/1671104009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345406bc80 msgr2=0x7f34540cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f3459fce640 1 --2- 172.21.15.70:0/1671104009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345406bc80 0x7f34540cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f3459fce640 1 -- 172.21.15.70:0/1671104009 >> v1:172.21.15.70:6789/0 conn(0x7f34540cf3e0 legacy=0x7f345406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f3459fce640 1 -- 172.21.15.70:0/1671104009 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345406c180 con 0x7f345415ce10 2023-10-08T05:53:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.613+0000 7f3458fcc640 1 -- 172.21.15.70:0/1671104009 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478723322 0 0) 0x7f34540cf840 con 0x7f34540cf3e0 2023-10-08T05:53:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.614+0000 7f345a7cf640 1 --2- 172.21.15.70:0/1671104009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345406bc80 0x7f34540cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.615+0000 7f3459fce640 1 --2- 172.21.15.70:0/1671104009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345415ce10 0x7f34540ceea0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f34500097a0 tx=0x7f345002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b58c4c0348281c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:15.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.615+0000 7f3458fcc640 1 -- 172.21.15.70:0/1671104009 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450031780 con 0x7f345415ce10 2023-10-08T05:53:15.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.615+0000 7f3458fcc640 1 -- 172.21.15.70:0/1671104009 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34500318e0 con 0x7f345415ce10 2023-10-08T05:53:15.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.615+0000 7f3458fcc640 1 -- 172.21.15.70:0/1671104009 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450031c70 con 0x7f345415ce10 2023-10-08T05:53:15.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.616+0000 7f345c232640 1 -- 172.21.15.70:0/1671104009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345415ce10 msgr2=0x7f34540ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:15.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.616+0000 7f345c232640 1 --2- 172.21.15.70:0/1671104009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345415ce10 0x7f34540ceea0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f34500097a0 tx=0x7f345002b820 comp rx=0 tx=0).stop 2023-10-08T05:53:15.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.616+0000 7f345c232640 1 -- 172.21.15.70:0/1671104009 shutdown_connections 2023-10-08T05:53:15.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.616+0000 7f345c232640 1 --2- 172.21.15.70:0/1671104009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345406bc80 0x7f34540cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.616+0000 7f345c232640 1 --2- 172.21.15.70:0/1671104009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345415ce10 0x7f34540ceea0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.616+0000 7f345c232640 1 -- 172.21.15.70:0/1671104009 >> 172.21.15.70:0/1671104009 conn(0x7f345406e630 msgr2=0x7f345406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:15.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.616+0000 7f345c232640 1 -- 172.21.15.70:0/1671104009 shutdown_connections 2023-10-08T05:53:15.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.616+0000 7f345c232640 1 -- 172.21.15.70:0/1671104009 wait complete. 2023-10-08T05:53:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.617+0000 7f345c232640 1 Processor -- start 2023-10-08T05:53:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.618+0000 7f345c232640 1 -- start start 2023-10-08T05:53:15.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.618+0000 7f345c232640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345406bc80 0x7f34541f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:15.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f345c232640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34540cf3e0 0x7f34540c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:15.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f345c232640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345415ce10 0x7f34541f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:15.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f345c232640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34540c4560 con 0x7f345406bc80 2023-10-08T05:53:15.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f345c232640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3454157800 con 0x7f34540cf3e0 2023-10-08T05:53:15.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f345c232640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34540c47d0 con 0x7f345415ce10 2023-10-08T05:53:15.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f3459fce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345406bc80 0x7f34541f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:15.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f3459fce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345406bc80 0x7f34541f6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46578/0 (socket says 172.21.15.70:46578) 2023-10-08T05:53:15.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f3459fce640 1 -- 172.21.15.70:0/1354302107 learned_addr learned my addr 172.21.15.70:0/1354302107 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:15.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f34597cd640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34540cf3e0 0x7f34540c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:15.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f345a7cf640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345415ce10 0x7f34541f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:15.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f3459fce640 1 -- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345415ce10 msgr2=0x7f34541f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:15.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f3459fce640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345415ce10 0x7f34541f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f3459fce640 1 -- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34540cf3e0 msgr2=0x7f34540c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:15.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f3459fce640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34540cf3e0 0x7f34540c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f3459fce640 1 -- 172.21.15.70:0/1354302107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3450009400 con 0x7f345406bc80 2023-10-08T05:53:15.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f34597cd640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34540cf3e0 0x7f34540c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:15.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.619+0000 7f345a7cf640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345415ce10 0x7f34541f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:15.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.620+0000 7f3459fce640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345406bc80 0x7f34541f6a90 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f34500098c0 tx=0x7f34500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:15.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.620+0000 7f344a7fc640 1 -- 172.21.15.70:0/1354302107 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450004230 con 0x7f345406bc80 2023-10-08T05:53:15.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.620+0000 7f344a7fc640 1 -- 172.21.15.70:0/1354302107 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3450004390 con 0x7f345406bc80 2023-10-08T05:53:15.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.620+0000 7f344a7fc640 1 -- 172.21.15.70:0/1354302107 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450032860 con 0x7f345406bc80 2023-10-08T05:53:15.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.620+0000 7f345c232640 1 -- 172.21.15.70:0/1354302107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34540c61d0 con 0x7f345406bc80 2023-10-08T05:53:15.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.620+0000 7f345c232640 1 -- 172.21.15.70:0/1354302107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34541f31f0 con 0x7f345406bc80 2023-10-08T05:53:15.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.621+0000 7f345c232640 1 -- 172.21.15.70:0/1354302107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345404e160 con 0x7f345406bc80 2023-10-08T05:53:15.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.623+0000 7f344a7fc640 1 -- 172.21.15.70:0/1354302107 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 38) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f34500329c0 con 0x7f345406bc80 2023-10-08T05:53:15.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.624+0000 7f344a7fc640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f343806afe0 0x7f343806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:15.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.625+0000 7f344a7fc640 1 -- 172.21.15.70:0/1354302107 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f34500abd80 con 0x7f345406bc80 2023-10-08T05:53:15.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.625+0000 7f34597cd640 1 -- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f343806afe0 msgr2=0x7f343806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2060337708 2023-10-08T05:53:15.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.625+0000 7f34597cd640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f343806afe0 0x7f343806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:15.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.629+0000 7f344a7fc640 1 -- 172.21.15.70:0/1354302107 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3450076b90 con 0x7f345406bc80 2023-10-08T05:53:15.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.815+0000 7f345c232640 1 -- 172.21.15.70:0/1354302107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f34540b69d0 con 0x7f345406bc80 2023-10-08T05:53:15.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.820+0000 7f344a7fc640 1 -- 172.21.15.70:0/1354302107 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f345007a1b0 con 0x7f345406bc80 2023-10-08T05:53:15.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 -- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f343806afe0 msgr2=0x7f343806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:15.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f343806afe0 0x7f343806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 -- 172.21.15.70:0/1354302107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345406bc80 msgr2=0x7f34541f6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:15.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345406bc80 0x7f34541f6a90 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f34500098c0 tx=0x7f34500046c0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 -- 172.21.15.70:0/1354302107 shutdown_connections 2023-10-08T05:53:15.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f343806afe0 0x7f343806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f345415ce10 0x7f34541f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34540cf3e0 0x7f34540c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 --2- 172.21.15.70:0/1354302107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f345406bc80 0x7f34541f6a90 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:15.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 -- 172.21.15.70:0/1354302107 >> 172.21.15.70:0/1354302107 conn(0x7f345406e630 msgr2=0x7f34540cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:15.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 -- 172.21.15.70:0/1354302107 shutdown_connections 2023-10-08T05:53:15.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:15.824+0000 7f345c232640 1 -- 172.21.15.70:0/1354302107 wait complete. 2023-10-08T05:53:15.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T05:53:15.885 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:15.883+0000 7fce32630040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:53:15.927 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:15.927+0000 7ff759011040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:53:15.936 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:15.935+0000 7fce32630040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:53:15.985 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:15.985+0000 7ff759011040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:53:16.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.044+0000 7f38936af640 1 Processor -- start 2023-10-08T05:53:16.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.044+0000 7f38936af640 1 -- start start 2023-10-08T05:53:16.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.045+0000 7f38936af640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f388c15cd70 0x7f388c1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.045+0000 7f38936af640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f388c0b4e10 0x7f388c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.045+0000 7f38936af640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388c0b8c30 con 0x7f388c0cd4e0 2023-10-08T05:53:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.045+0000 7f38936af640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0cdd80 con 0x7f388c15cd70 2023-10-08T05:53:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.045+0000 7f38936af640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0ce060 con 0x7f388c0b4e10 2023-10-08T05:53:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.046+0000 7f389144b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f388c15cd70 0x7f388c1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.046+0000 7f389144b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f388c15cd70 0x7f388c1612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46580/0 (socket says 172.21.15.70:46580) 2023-10-08T05:53:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.046+0000 7f389144b640 1 -- 172.21.15.70:0/2847690808 learned_addr learned my addr 172.21.15.70:0/2847690808 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.046+0000 7f3891c4c640 1 --2- 172.21.15.70:0/2847690808 >> v2:172.21.15.181:3300/0 conn(0x7f388c0b4e10 0x7f388c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.046+0000 7f389144b640 1 -- 172.21.15.70:0/2847690808 >> v2:172.21.15.181:3300/0 conn(0x7f388c0b4e10 msgr2=0x7f388c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:16.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.046+0000 7f389144b640 1 --2- 172.21.15.70:0/2847690808 >> v2:172.21.15.181:3300/0 conn(0x7f388c0b4e10 0x7f388c0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.046+0000 7f389144b640 1 -- 172.21.15.70:0/2847690808 >> v1:172.21.15.181:6789/0 conn(0x7f388c0cd4e0 legacy=0x7f388c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:16.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.046+0000 7f389144b640 1 -- 172.21.15.70:0/2847690808 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0ce340 con 0x7f388c15cd70 2023-10-08T05:53:16.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.046+0000 7f3891c4c640 1 --2- 172.21.15.70:0/2847690808 >> v2:172.21.15.181:3300/0 conn(0x7f388c0b4e10 0x7f388c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:16.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.047+0000 7f389144b640 1 --2- 172.21.15.70:0/2847690808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c15cd70 0x7f388c1612f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f388c0d07d0 tx=0x7f388802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d390b3f333ed0864 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:16.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.047+0000 7f3883fff640 1 -- 172.21.15.70:0/2847690808 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388802ca80 con 0x7f388c15cd70 2023-10-08T05:53:16.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.048+0000 7f3883fff640 1 -- 172.21.15.70:0/2847690808 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388802cbe0 con 0x7f388c15cd70 2023-10-08T05:53:16.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.048+0000 7f3883fff640 1 -- 172.21.15.70:0/2847690808 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38880343f0 con 0x7f388c15cd70 2023-10-08T05:53:16.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.048+0000 7f38936af640 1 -- 172.21.15.70:0/2847690808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c15cd70 msgr2=0x7f388c1612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:16.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.048+0000 7f38936af640 1 --2- 172.21.15.70:0/2847690808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c15cd70 0x7f388c1612f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f388c0d07d0 tx=0x7f388802bee0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.048+0000 7f38936af640 1 -- 172.21.15.70:0/2847690808 shutdown_connections 2023-10-08T05:53:16.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.048+0000 7f38936af640 1 --2- 172.21.15.70:0/2847690808 >> v2:172.21.15.181:3300/0 conn(0x7f388c0b4e10 0x7f388c0d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.048+0000 7f38936af640 1 --2- 172.21.15.70:0/2847690808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c15cd70 0x7f388c1612f0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.048+0000 7f38936af640 1 -- 172.21.15.70:0/2847690808 >> 172.21.15.70:0/2847690808 conn(0x7f388c0cf700 msgr2=0x7f388c0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:16.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.048+0000 7f38936af640 1 -- 172.21.15.70:0/2847690808 shutdown_connections 2023-10-08T05:53:16.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.048+0000 7f38936af640 1 -- 172.21.15.70:0/2847690808 wait complete. 2023-10-08T05:53:16.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.050+0000 7f38936af640 1 Processor -- start 2023-10-08T05:53:16.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.050+0000 7f38936af640 1 -- start start 2023-10-08T05:53:16.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.051+0000 7f38936af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c0b4e10 0x7f388c145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.051+0000 7f38936af640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c0cd4e0 0x7f388c1461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.051+0000 7f38936af640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388c15cd70 0x7f388c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.051+0000 7f38936af640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c15edc0 con 0x7f388c0b4e10 2023-10-08T05:53:16.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.051+0000 7f38936af640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c15f620 con 0x7f388c0cd4e0 2023-10-08T05:53:16.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.051+0000 7f38936af640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0c9470 con 0x7f388c15cd70 2023-10-08T05:53:16.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.051+0000 7f389144b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c0b4e10 0x7f388c145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.051+0000 7f389144b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c0b4e10 0x7f388c145c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46594/0 (socket says 172.21.15.70:46594) 2023-10-08T05:53:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.051+0000 7f389144b640 1 -- 172.21.15.70:0/480732550 learned_addr learned my addr 172.21.15.70:0/480732550 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.052+0000 7f3890c4a640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c0cd4e0 0x7f388c1461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.052+0000 7f389144b640 1 -- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388c15cd70 msgr2=0x7f388c1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.052+0000 7f389144b640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388c15cd70 0x7f388c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.052+0000 7f389144b640 1 -- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c0cd4e0 msgr2=0x7f388c1461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.052+0000 7f389144b640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c0cd4e0 0x7f388c1461a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.052+0000 7f389144b640 1 -- 172.21.15.70:0/480732550 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3888009b30 con 0x7f388c0b4e10 2023-10-08T05:53:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.052+0000 7f3890c4a640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c0cd4e0 0x7f388c1461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.052+0000 7f389144b640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c0b4e10 0x7f388c145c60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3888036040 tx=0x7f3888004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.052+0000 7f3881ffb640 1 -- 172.21.15.70:0/480732550 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388803e450 con 0x7f388c0b4e10 2023-10-08T05:53:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.053+0000 7f3881ffb640 1 -- 172.21.15.70:0/480732550 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388803e5f0 con 0x7f388c0b4e10 2023-10-08T05:53:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.053+0000 7f3881ffb640 1 -- 172.21.15.70:0/480732550 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388803e9a0 con 0x7f388c0b4e10 2023-10-08T05:53:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.053+0000 7f38936af640 1 -- 172.21.15.70:0/480732550 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c15efd0 con 0x7f388c0b4e10 2023-10-08T05:53:16.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.053+0000 7f38936af640 1 -- 172.21.15.70:0/480732550 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c0c9f90 con 0x7f388c0b4e10 2023-10-08T05:53:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.054+0000 7f38936af640 1 -- 172.21.15.70:0/480732550 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388c04e160 con 0x7f388c0b4e10 2023-10-08T05:53:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.055+0000 7f3881ffb640 1 -- 172.21.15.70:0/480732550 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 38) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f388802cd50 con 0x7f388c0b4e10 2023-10-08T05:53:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.055+0000 7f3881ffb640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f387006afe0 0x7f387006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.056+0000 7f3881ffb640 1 -- 172.21.15.70:0/480732550 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f38880ac650 con 0x7f388c0b4e10 2023-10-08T05:53:16.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.056+0000 7f3890c4a640 1 -- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f387006afe0 msgr2=0x7f387006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2060337708 2023-10-08T05:53:16.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.056+0000 7f3890c4a640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f387006afe0 0x7f387006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:16.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.060+0000 7f3881ffb640 1 -- 172.21.15.70:0/480732550 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38880773b0 con 0x7f388c0b4e10 2023-10-08T05:53:16.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.253+0000 7f38936af640 1 -- 172.21.15.70:0/480732550 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f388c0d0a00 con 0x7f388c0b4e10 2023-10-08T05:53:16.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.257+0000 7f3890c4a640 1 -- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f387006afe0 msgr2=0x7f387006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2060337708 2023-10-08T05:53:16.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.257+0000 7f3890c4a640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f387006afe0 0x7f387006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:53:16.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.258+0000 7f3881ffb640 1 -- 172.21.15.70:0/480732550 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f388807a9d0 con 0x7f388c0b4e10 2023-10-08T05:53:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.261+0000 7f38936af640 1 -- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f387006afe0 msgr2=0x7f387006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.261+0000 7f38936af640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f387006afe0 0x7f387006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.261+0000 7f38936af640 1 -- 172.21.15.70:0/480732550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c0b4e10 msgr2=0x7f388c145c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.261+0000 7f38936af640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c0b4e10 0x7f388c145c60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3888036040 tx=0x7f3888004a40 comp rx=0 tx=0).stop 2023-10-08T05:53:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.261+0000 7f38936af640 1 -- 172.21.15.70:0/480732550 shutdown_connections 2023-10-08T05:53:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.261+0000 7f38936af640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f387006afe0 0x7f387006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.261+0000 7f38936af640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f388c15cd70 0x7f388c1466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.261+0000 7f38936af640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f388c0cd4e0 0x7f388c1461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.261+0000 7f38936af640 1 --2- 172.21.15.70:0/480732550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f388c0b4e10 0x7f388c145c60 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.261+0000 7f38936af640 1 -- 172.21.15.70:0/480732550 >> 172.21.15.70:0/480732550 conn(0x7f388c0cf700 msgr2=0x7f388c0b7650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:16.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.262+0000 7f38936af640 1 -- 172.21.15.70:0/480732550 shutdown_connections 2023-10-08T05:53:16.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.262+0000 7f38936af640 1 -- 172.21.15.70:0/480732550 wait complete. 2023-10-08T05:53:16.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-08T05:53:16.381 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:16.380+0000 7fce32630040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:53:16.432 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:16.431+0000 7fce32630040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:53:16.456 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:16.456+0000 7ff759011040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:53:16.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.478+0000 7f429e652640 1 Processor -- start 2023-10-08T05:53:16.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.479+0000 7f429e652640 1 -- start start 2023-10-08T05:53:16.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.480+0000 7f429e652640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42981601b0 0x7f42980cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.480+0000 7f429e652640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4298071c30 0x7f4298069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.480+0000 7f429e652640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42980ce210 con 0x7f4298072090 2023-10-08T05:53:16.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.480+0000 7f429e652640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42980cdee0 con 0x7f4298071c30 2023-10-08T05:53:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.480+0000 7f429e652640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4298161530 con 0x7f42981601b0 2023-10-08T05:53:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.480+0000 7f428ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4298071c30 0x7f4298069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.480+0000 7f4297fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42981601b0 0x7f42980cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.480+0000 7f428ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4298071c30 0x7f4298069d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:52102/0 (socket says 172.21.15.70:52102) 2023-10-08T05:53:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.481+0000 7f428ffff640 1 -- 172.21.15.70:0/3347735901 learned_addr learned my addr 172.21.15.70:0/3347735901 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.481+0000 7f428ffff640 1 -- 172.21.15.70:0/3347735901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42981601b0 msgr2=0x7f42980cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.481+0000 7f428ffff640 1 --2- 172.21.15.70:0/3347735901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42981601b0 0x7f42980cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.481+0000 7f428ffff640 1 -- 172.21.15.70:0/3347735901 >> v1:172.21.15.181:6789/0 conn(0x7f4298072090 legacy=0x7f42980ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.481+0000 7f428ffff640 1 -- 172.21.15.70:0/3347735901 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4298161810 con 0x7f4298071c30 2023-10-08T05:53:16.483 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:16.482+0000 7fce32630040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:53:16.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.483+0000 7f428ffff640 1 --2- 172.21.15.70:0/3347735901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4298071c30 0x7f4298069d50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4280009870 tx=0x7f428002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f6c5f356a0e44a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:16.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.483+0000 7f42977fe640 1 -- 172.21.15.70:0/3347735901 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280030840 con 0x7f4298071c30 2023-10-08T05:53:16.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.484+0000 7f42977fe640 1 -- 172.21.15.70:0/3347735901 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42800309a0 con 0x7f4298071c30 2023-10-08T05:53:16.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.484+0000 7f42977fe640 1 -- 172.21.15.70:0/3347735901 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280030d30 con 0x7f4298071c30 2023-10-08T05:53:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.484+0000 7f429e652640 1 -- 172.21.15.70:0/3347735901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4298071c30 msgr2=0x7f4298069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.484+0000 7f429e652640 1 --2- 172.21.15.70:0/3347735901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4298071c30 0x7f4298069d50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4280009870 tx=0x7f428002b980 comp rx=0 tx=0).stop 2023-10-08T05:53:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.484+0000 7f429e652640 1 -- 172.21.15.70:0/3347735901 shutdown_connections 2023-10-08T05:53:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.484+0000 7f429e652640 1 --2- 172.21.15.70:0/3347735901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4298071c30 0x7f4298069d50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.484+0000 7f429e652640 1 --2- 172.21.15.70:0/3347735901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42981601b0 0x7f42980cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.484+0000 7f429e652640 1 -- 172.21.15.70:0/3347735901 >> 172.21.15.70:0/3347735901 conn(0x7f429806d430 msgr2=0x7f429806d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.484+0000 7f429e652640 1 -- 172.21.15.70:0/3347735901 shutdown_connections 2023-10-08T05:53:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.484+0000 7f429e652640 1 -- 172.21.15.70:0/3347735901 wait complete. 2023-10-08T05:53:16.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.486+0000 7f429e652640 1 Processor -- start 2023-10-08T05:53:16.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.486+0000 7f429e652640 1 -- start start 2023-10-08T05:53:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.487+0000 7f429e652640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4298071c30 0x7f4298152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.487+0000 7f429e652640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4298072090 0x7f42980c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.487+0000 7f429e652640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42981601b0 0x7f42980b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.487+0000 7f429e652640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298153490 con 0x7f4298072090 2023-10-08T05:53:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.487+0000 7f429e652640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42980c4710 con 0x7f42981601b0 2023-10-08T05:53:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.487+0000 7f429e652640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42980c4960 con 0x7f4298071c30 2023-10-08T05:53:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.487+0000 7f428ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4298072090 0x7f42980c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.488+0000 7f428ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4298072090 0x7f42980c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46604/0 (socket says 172.21.15.70:46604) 2023-10-08T05:53:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.488+0000 7f428ffff640 1 -- 172.21.15.70:0/3265271114 learned_addr learned my addr 172.21.15.70:0/3265271114 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.488+0000 7f4297fff640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4298071c30 0x7f4298152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.488+0000 7f429cbef640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42981601b0 0x7f42980b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.488+0000 7f428ffff640 1 -- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4298071c30 msgr2=0x7f4298152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.488+0000 7f428ffff640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4298071c30 0x7f4298152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.488+0000 7f428ffff640 1 -- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42981601b0 msgr2=0x7f42980b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:16.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.488+0000 7f428ffff640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42981601b0 0x7f42980b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:16.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.488+0000 7f428ffff640 1 -- 172.21.15.70:0/3265271114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42800094d0 con 0x7f4298072090 2023-10-08T05:53:16.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.488+0000 7f428ffff640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4298072090 0x7f42980c3d70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f428002b9c0 tx=0x7f4280004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:16.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.489+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280039a90 con 0x7f4298072090 2023-10-08T05:53:16.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.489+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4280039bf0 con 0x7f4298072090 2023-10-08T05:53:16.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.489+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280043410 con 0x7f4298072090 2023-10-08T05:53:16.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.489+0000 7f429e652640 1 -- 172.21.15.70:0/3265271114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42980c4c40 con 0x7f4298072090 2023-10-08T05:53:16.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.489+0000 7f429e652640 1 -- 172.21.15.70:0/3265271114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42980c5200 con 0x7f4298072090 2023-10-08T05:53:16.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.490+0000 7f429e652640 1 -- 172.21.15.70:0/3265271114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429804e160 con 0x7f4298072090 2023-10-08T05:53:16.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.491+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 38) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f428002f390 con 0x7f4298072090 2023-10-08T05:53:16.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.491+0000 7f42957fa640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 0x7f427806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:16.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.492+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6116+0+0 (secure 0 0 0) 0x7f42800abc50 con 0x7f4298072090 2023-10-08T05:53:16.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.492+0000 7f4297fff640 1 -- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 msgr2=0x7f427806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2060337708 2023-10-08T05:53:16.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.492+0000 7f4297fff640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 0x7f427806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:16.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.496+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42800768a0 con 0x7f4298072090 2023-10-08T05:53:16.513 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:16.512+0000 7ff759011040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:53:16.544 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:16.543+0000 7fce32630040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:53:16.565 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:16.564+0000 7ff759011040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:53:16.592 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:16.591+0000 7fce32630040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:53:16.627 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:16.627+0000 7ff759011040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:53:16.678 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:16.677+0000 7ff759011040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:53:16.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.693+0000 7f4297fff640 1 -- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 msgr2=0x7f427806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2060337708 2023-10-08T05:53:16.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.693+0000 7f4297fff640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 0x7f427806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:53:16.702 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:16.701+0000 7fce32630040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:53:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:16.706+0000 7f429e652640 1 -- 172.21.15.70:0/3265271114 --> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f42980b2ba0 con 0x7f427806afe0 2023-10-08T05:53:16.763 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:16.762+0000 7fce32630040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:16.788 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:16.788+0000 7ff759011040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:53:16.812 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:16.811+0000 7fce32630040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:53:16.851 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:16.851+0000 7ff759011040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:16.869 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:16.868+0000 7fce32630040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:53:16.900 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:16.899+0000 7ff759011040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:53:16.918 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:16.917+0000 7fce32630040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:53:16.956 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:16.956+0000 7ff759011040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:53:17.006 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:17.005+0000 7ff759011040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:53:17.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:17.094+0000 7f4297fff640 1 -- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 msgr2=0x7f427806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2060337708 2023-10-08T05:53:17.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:17.094+0000 7f4297fff640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 0x7f427806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:53:17.297 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:17.296+0000 7fce32630040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:53:17.413 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:17.412+0000 7ff759011040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:53:17.482 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:17.481+0000 7fce32630040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:53:17.572 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:17.572+0000 7ff759011040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:53:17.589 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:17.588+0000 7fce32630040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:53:17.638 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:17.637+0000 7fce32630040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:53:17.680 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:17.680+0000 7ff759011040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:53:17.730 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:17.730+0000 7ff759011040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:53:17.745 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:17.744+0000 7fce32630040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:17.839 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:17.839+0000 7ff759011040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:17.852 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:17.851+0000 7fce32630040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:53:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:17.895+0000 7f4297fff640 1 -- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 msgr2=0x7f427806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2060337708 2023-10-08T05:53:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:17.895+0000 7f4297fff640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 0x7f427806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:53:17.945 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:17.945+0000 7ff759011040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:53:18.007 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:18.005+0000 7fce32630040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:18.055 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:18.054+0000 7fce32630040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:18.100 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:18.099+0000 7ff759011040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:18.148 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:18.148+0000 7ff759011040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:18.174 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:18.173+0000 7fce32630040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:53:18.267 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:18.266+0000 7ff759011040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:53:18.994 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:18.993+0000 7fce32630040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:53:19.072 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:19.071+0000 7ff759011040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:53:19.287 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.286+0000 7fce32630040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:53:19.344 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.343+0000 7fce32630040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:53:19.363 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:19.363+0000 7ff759011040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:53:19.420 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:19.420+0000 7ff759011040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:53:19.458 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.457+0000 7fce32630040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:53:19.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:19.481+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 39) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f4280076c70 con 0x7f4298072090 2023-10-08T05:53:19.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:19.482+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 msgr2=0x7f427806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:19.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:19.482+0000 7f42957fa640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/2060337708,v1:172.21.15.181:6833/2060337708] conn(0x7f427806afe0 0x7f427806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:19.547 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:19.547+0000 7ff759011040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:53:19.665 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.663+0000 7fcdae9a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:19.665 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.663+0000 7fcdae9a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:19.666 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.663+0000 7fcdae9a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:19.666 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.663+0000 7fcdae9a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:19.666 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.663+0000 7fcdae9a1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:19.666 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.664+0000 7fcdb19e7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:19.666 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.664+0000 7fcdb19e7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:19.666 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.664+0000 7fcdb19e7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:19.667 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.664+0000 7fcdb19e7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:19.667 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:19.664+0000 7fcdb19e7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:20.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.487+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 40) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f428008e2b0 con 0x7f4298072090 2023-10-08T05:53:20.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.488+0000 7f42957fa640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f42780358e0 0x7f4278075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:20.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.488+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 --> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f42980b2ba0 con 0x7f42780358e0 2023-10-08T05:53:20.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.499+0000 7f4297fff640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f42780358e0 0x7f4278075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:20.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.508+0000 7f4297fff640 1 --2- 172.21.15.70:0/3265271114 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f42780358e0 0x7f4278075500 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4288009380 tx=0x7f4288009190 comp rx=0 tx=0).ready entity=mgr.5977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:20.527 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:20.526+0000 7fcdb89f5640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-08T05:53:20.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.528+0000 7f42957fa640 1 -- 172.21.15.70:0/3265271114 <== mgr.5977 v2:172.21.15.181:6832/1003988322 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f42980b2ba0 con 0x7f42780358e0 2023-10-08T05:53:20.528 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-08T05:53:20.540 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T05:53:20.541 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T05:53:20.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.746+0000 7f8c3e354640 1 Processor -- start 2023-10-08T05:53:20.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.747+0000 7f8c3e354640 1 -- start start 2023-10-08T05:53:20.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.748+0000 7f8c3e354640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8c380b23d0 0x7f8c3815d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:20.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.748+0000 7f8c3e354640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c3806b330 0x7f8c38160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.748+0000 7f8c3e354640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3815d6b0 con 0x7f8c380b27b0 2023-10-08T05:53:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.748+0000 7f8c3e354640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3815d990 con 0x7f8c3806b330 2023-10-08T05:53:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.748+0000 7f8c3e354640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3815dc70 con 0x7f8c380b23d0 2023-10-08T05:53:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.748+0000 7f8c3db53640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c3806b330 0x7f8c38160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.748+0000 7f8c3db53640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8c3806b330 0x7f8c38160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46618/0 (socket says 172.21.15.70:46618) 2023-10-08T05:53:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.748+0000 7f8c3db53640 1 -- 172.21.15.70:0/2219558061 learned_addr learned my addr 172.21.15.70:0/2219558061 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:20.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.749+0000 7f8c3d352640 1 --2- 172.21.15.70:0/2219558061 >> v2:172.21.15.181:3300/0 conn(0x7f8c380b23d0 0x7f8c3815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:20.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.749+0000 7f8c3db53640 1 -- 172.21.15.70:0/2219558061 >> v2:172.21.15.181:3300/0 conn(0x7f8c380b23d0 msgr2=0x7f8c3815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:20.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.749+0000 7f8c3db53640 1 --2- 172.21.15.70:0/2219558061 >> v2:172.21.15.181:3300/0 conn(0x7f8c380b23d0 0x7f8c3815d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:20.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.749+0000 7f8c3db53640 1 -- 172.21.15.70:0/2219558061 >> v1:172.21.15.70:6789/0 conn(0x7f8c380b27b0 legacy=0x7f8c3806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:20.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.749+0000 7f8c3db53640 1 -- 172.21.15.70:0/2219558061 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c38160710 con 0x7f8c3806b330 2023-10-08T05:53:20.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.750+0000 7f8c3db53640 1 --2- 172.21.15.70:0/2219558061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c3806b330 0x7f8c38160c30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8c380b2e20 tx=0x7f8c3002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580106a7f6b5456e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:20.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c37fff640 1 -- 172.21.15.70:0/2219558061 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30031870 con 0x7f8c3806b330 2023-10-08T05:53:20.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c37fff640 1 -- 172.21.15.70:0/2219558061 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c300319d0 con 0x7f8c3806b330 2023-10-08T05:53:20.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c37fff640 1 -- 172.21.15.70:0/2219558061 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30031d60 con 0x7f8c3806b330 2023-10-08T05:53:20.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c3e354640 1 -- 172.21.15.70:0/2219558061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c3806b330 msgr2=0x7f8c38160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:20.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c3e354640 1 --2- 172.21.15.70:0/2219558061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c3806b330 0x7f8c38160c30 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8c380b2e20 tx=0x7f8c3002b910 comp rx=0 tx=0).stop 2023-10-08T05:53:20.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c3e354640 1 -- 172.21.15.70:0/2219558061 shutdown_connections 2023-10-08T05:53:20.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c3e354640 1 --2- 172.21.15.70:0/2219558061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c3806b330 0x7f8c38160c30 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:20.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c3e354640 1 --2- 172.21.15.70:0/2219558061 >> v2:172.21.15.181:3300/0 conn(0x7f8c380b23d0 0x7f8c3815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:20.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c3e354640 1 -- 172.21.15.70:0/2219558061 >> 172.21.15.70:0/2219558061 conn(0x7f8c380cd160 msgr2=0x7f8c380cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c3e354640 1 -- 172.21.15.70:0/2219558061 shutdown_connections 2023-10-08T05:53:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.751+0000 7f8c3e354640 1 -- 172.21.15.70:0/2219558061 wait complete. 2023-10-08T05:53:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.753+0000 7f8c3e354640 1 Processor -- start 2023-10-08T05:53:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.753+0000 7f8c3e354640 1 -- start start 2023-10-08T05:53:20.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.754+0000 7f8c3e354640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c3806b330 0x7f8c38149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.754+0000 7f8c3e354640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c380b23d0 0x7f8c380c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.754+0000 7f8c3e354640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c380b27b0 0x7f8c3815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.754+0000 7f8c3e354640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c380c9790 con 0x7f8c380b23d0 2023-10-08T05:53:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.754+0000 7f8c3e354640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3814a050 con 0x7f8c380b27b0 2023-10-08T05:53:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.754+0000 7f8c3e354640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c380c8b80 con 0x7f8c3806b330 2023-10-08T05:53:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.754+0000 7f8c3cb51640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c380b23d0 0x7f8c380c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.754+0000 7f8c3cb51640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c380b23d0 0x7f8c380c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46626/0 (socket says 172.21.15.70:46626) 2023-10-08T05:53:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.754+0000 7f8c3cb51640 1 -- 172.21.15.70:0/1171822233 learned_addr learned my addr 172.21.15.70:0/1171822233 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c3db53640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c380b27b0 0x7f8c3815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c3d352640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c3806b330 0x7f8c38149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c3cb51640 1 -- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c3806b330 msgr2=0x7f8c38149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c3cb51640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c3806b330 0x7f8c38149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c3cb51640 1 -- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c380b27b0 msgr2=0x7f8c3815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c3cb51640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c380b27b0 0x7f8c3815ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c3cb51640 1 -- 172.21.15.70:0/1171822233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c300094d0 con 0x7f8c380b23d0 2023-10-08T05:53:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c3cb51640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c380b23d0 0x7f8c380c81c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00c400 tx=0x7f8c2c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c35ffb640 1 -- 172.21.15.70:0/1171822233 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c0052d0 con 0x7f8c380b23d0 2023-10-08T05:53:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c35ffb640 1 -- 172.21.15.70:0/1171822233 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c005430 con 0x7f8c380b23d0 2023-10-08T05:53:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c35ffb640 1 -- 172.21.15.70:0/1171822233 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c003830 con 0x7f8c380b23d0 2023-10-08T05:53:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c3e354640 1 -- 172.21.15.70:0/1171822233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c380c8e60 con 0x7f8c380b23d0 2023-10-08T05:53:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.755+0000 7f8c3e354640 1 -- 172.21.15.70:0/1171822233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c380c9420 con 0x7f8c380b23d0 2023-10-08T05:53:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.756+0000 7f8c3e354640 1 -- 172.21.15.70:0/1171822233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c38074150 con 0x7f8c380b23d0 2023-10-08T05:53:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.758+0000 7f8c35ffb640 1 -- 172.21.15.70:0/1171822233 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 40) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f8c2c003990 con 0x7f8c380b23d0 2023-10-08T05:53:20.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.758+0000 7f8c35ffb640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f8c2406ad40 0x7f8c2406d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:20.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.759+0000 7f8c35ffb640 1 -- 172.21.15.70:0/1171822233 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8c2c088b30 con 0x7f8c380b23d0 2023-10-08T05:53:20.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.759+0000 7f8c3d352640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f8c2406ad40 0x7f8c2406d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:20.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.760+0000 7f8c3d352640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f8c2406ad40 0x7f8c2406d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009f30 tx=0x7f8c2800b040 comp rx=0 tx=0).ready entity=mgr.5977 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:20.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:20.764+0000 7f8c35ffb640 1 -- 172.21.15.70:0/1171822233 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c2c053c30 con 0x7f8c380b23d0 2023-10-08T05:53:21.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.002+0000 7f8c3e354640 1 -- 172.21.15.70:0/1171822233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f8c3806b110 con 0x7f8c380b23d0 2023-10-08T05:53:21.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.534+0000 7f8c35ffb640 1 -- 172.21.15.70:0/1171822233 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v41) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8c2c057250 con 0x7f8c380b23d0 2023-10-08T05:53:21.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.540+0000 7f8c3e354640 1 -- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f8c2406ad40 msgr2=0x7f8c2406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:21.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.540+0000 7f8c3e354640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f8c2406ad40 0x7f8c2406d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009f30 tx=0x7f8c2800b040 comp rx=0 tx=0).stop 2023-10-08T05:53:21.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.540+0000 7f8c3e354640 1 -- 172.21.15.70:0/1171822233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c380b23d0 msgr2=0x7f8c380c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:21.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.540+0000 7f8c3e354640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c380b23d0 0x7f8c380c81c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c00c400 tx=0x7f8c2c00c8c0 comp rx=0 tx=0).stop 2023-10-08T05:53:21.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.540+0000 7f8c3e354640 1 -- 172.21.15.70:0/1171822233 shutdown_connections 2023-10-08T05:53:21.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.540+0000 7f8c3e354640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8c380b27b0 0x7f8c3815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.540+0000 7f8c3e354640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c380b23d0 0x7f8c380c81c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.540+0000 7f8c3e354640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f8c2406ad40 0x7f8c2406d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.540+0000 7f8c3e354640 1 --2- 172.21.15.70:0/1171822233 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8c3806b330 0x7f8c38149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.540+0000 7f8c3e354640 1 -- 172.21.15.70:0/1171822233 >> 172.21.15.70:0/1171822233 conn(0x7f8c380cd160 msgr2=0x7f8c380ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.541+0000 7f8c3e354640 1 -- 172.21.15.70:0/1171822233 shutdown_connections 2023-10-08T05:53:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:21.541+0000 7f8c3e354640 1 -- 172.21.15.70:0/1171822233 wait complete. 2023-10-08T05:53:21.561 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:53:21.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:53:21.665 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:53:21.760 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:53:21.761 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:53:21.761 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:21.817 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:53:21.817 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:21.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:22.065 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:22.065+0000 7f23efa30040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:53:22.065 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:22.065+0000 7fbc4ba4c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:53:22.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.112+0000 7f3ed3516640 1 Processor -- start 2023-10-08T05:53:22.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.113+0000 7f3ed3516640 1 -- start start 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.114+0000 7f3ed3516640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3ecc15ab40 0x7f3ecc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.114+0000 7f3ed3516640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 0x7f3ecc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.114+0000 7f3ed3516640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ecc0cae70 0x7f3ecc0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.114+0000 7f3ed3516640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc15bf30 con 0x7f3ecc0cae70 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.114+0000 7f3ed3516640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc0cb840 con 0x7f3ecc15ab40 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.114+0000 7f3ed3516640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc15b2b0 con 0x7f3ecc0c90b0 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed1ab3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ecc0cae70 0x7f3ecc0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed1ab3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ecc0cae70 0x7f3ecc0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46652/0 (socket says 172.21.15.70:46652) 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed1ab3640 1 -- 172.21.15.70:0/1169221065 learned_addr learned my addr 172.21.15.70:0/1169221065 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed12b2640 1 --2- 172.21.15.70:0/1169221065 >> v2:172.21.15.181:3300/0 conn(0x7f3ecc15ab40 0x7f3ecc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed0ab1640 1 --2- 172.21.15.70:0/1169221065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 0x7f3ecc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed1ab3640 1 -- 172.21.15.70:0/1169221065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 msgr2=0x7f3ecc0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed1ab3640 1 --2- 172.21.15.70:0/1169221065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 0x7f3ecc0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed1ab3640 1 -- 172.21.15.70:0/1169221065 >> v2:172.21.15.181:3300/0 conn(0x7f3ecc15ab40 msgr2=0x7f3ecc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed1ab3640 1 --2- 172.21.15.70:0/1169221065 >> v2:172.21.15.181:3300/0 conn(0x7f3ecc15ab40 0x7f3ecc0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed1ab3640 1 -- 172.21.15.70:0/1169221065 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc15b590 con 0x7f3ecc0cae70 2023-10-08T05:53:22.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.115+0000 7f3ed0ab1640 1 --2- 172.21.15.70:0/1169221065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 0x7f3ecc0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:22.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.116+0000 7f3ed1ab3640 1 --2- 172.21.15.70:0/1169221065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cae70 0x7f3ecc0cb300 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc06e440 tx=0x7f3ec402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a2db9f777768a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:22.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ec3fff640 1 -- 172.21.15.70:0/1169221065 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec402ca80 con 0x7f3ecc0cae70 2023-10-08T05:53:22.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ec3fff640 1 -- 172.21.15.70:0/1169221065 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec402cbe0 con 0x7f3ecc0cae70 2023-10-08T05:53:22.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ec3fff640 1 -- 172.21.15.70:0/1169221065 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40343f0 con 0x7f3ecc0cae70 2023-10-08T05:53:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ed3516640 1 -- 172.21.15.70:0/1169221065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cae70 msgr2=0x7f3ecc0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ed3516640 1 --2- 172.21.15.70:0/1169221065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cae70 0x7f3ecc0cb300 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc06e440 tx=0x7f3ec402bee0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ed3516640 1 -- 172.21.15.70:0/1169221065 shutdown_connections 2023-10-08T05:53:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ed3516640 1 --2- 172.21.15.70:0/1169221065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cae70 0x7f3ecc0cb300 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ed3516640 1 --2- 172.21.15.70:0/1169221065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 0x7f3ecc0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ed3516640 1 --2- 172.21.15.70:0/1169221065 >> v2:172.21.15.181:3300/0 conn(0x7f3ecc15ab40 0x7f3ecc0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ed3516640 1 -- 172.21.15.70:0/1169221065 >> 172.21.15.70:0/1169221065 conn(0x7f3ecc06a380 msgr2=0x7f3ecc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.117+0000 7f3ed3516640 1 -- 172.21.15.70:0/1169221065 shutdown_connections 2023-10-08T05:53:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.118+0000 7f3ed3516640 1 -- 172.21.15.70:0/1169221065 wait complete. 2023-10-08T05:53:22.120 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:22.118+0000 7fbc4ba4c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:53:22.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.119+0000 7f3ed3516640 1 Processor -- start 2023-10-08T05:53:22.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.119+0000 7f3ed3516640 1 -- start start 2023-10-08T05:53:22.120 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:22.120+0000 7f23efa30040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:53:22.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.120+0000 7f3ed3516640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 0x7f3ecc1f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.120+0000 7f3ed3516640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ecc0cae70 0x7f3ecc1f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.120+0000 7f3ed3516640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc15ab40 0x7f3ecc159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.120+0000 7f3ed3516640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc1f6200 con 0x7f3ecc15ab40 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.120+0000 7f3ed3516640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc146f70 con 0x7f3ecc0cae70 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.120+0000 7f3ed3516640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc143b00 con 0x7f3ecc0c90b0 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed1ab3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc15ab40 0x7f3ecc159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed1ab3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc15ab40 0x7f3ecc159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46654/0 (socket says 172.21.15.70:46654) 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed1ab3640 1 -- 172.21.15.70:0/482939759 learned_addr learned my addr 172.21.15.70:0/482939759 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed12b2640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 0x7f3ecc1f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed0ab1640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ecc0cae70 0x7f3ecc1f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed1ab3640 1 -- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 msgr2=0x7f3ecc1f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed1ab3640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 0x7f3ecc1f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed1ab3640 1 -- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ecc0cae70 msgr2=0x7f3ecc1f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed1ab3640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ecc0cae70 0x7f3ecc1f1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed1ab3640 1 -- 172.21.15.70:0/482939759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec4009b30 con 0x7f3ecc15ab40 2023-10-08T05:53:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.121+0000 7f3ed1ab3640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc15ab40 0x7f3ecc159180 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009f40 tx=0x7f3ec4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.122+0000 7f3ec1ffb640 1 -- 172.21.15.70:0/482939759 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4009510 con 0x7f3ecc15ab40 2023-10-08T05:53:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.122+0000 7f3ec1ffb640 1 -- 172.21.15.70:0/482939759 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec403e450 con 0x7f3ecc15ab40 2023-10-08T05:53:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.122+0000 7f3ec1ffb640 1 -- 172.21.15.70:0/482939759 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec403e800 con 0x7f3ecc15ab40 2023-10-08T05:53:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.122+0000 7f3ed3516640 1 -- 172.21.15.70:0/482939759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc143820 con 0x7f3ecc15ab40 2023-10-08T05:53:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.122+0000 7f3ed3516640 1 -- 172.21.15.70:0/482939759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc1f5e20 con 0x7f3ecc15ab40 2023-10-08T05:53:22.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.123+0000 7f3ed3516640 1 -- 172.21.15.70:0/482939759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc1f6340 con 0x7f3ecc15ab40 2023-10-08T05:53:22.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.124+0000 7f3ec1ffb640 1 -- 172.21.15.70:0/482939759 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f3ec402cd50 con 0x7f3ecc15ab40 2023-10-08T05:53:22.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.125+0000 7f3ec1ffb640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f3eb006af90 0x7f3eb006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.125+0000 7f3ec1ffb640 1 -- 172.21.15.70:0/482939759 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3ec40ac8f0 con 0x7f3ecc15ab40 2023-10-08T05:53:22.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.125+0000 7f3ed12b2640 1 -- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f3eb006af90 msgr2=0x7f3eb006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:22.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.125+0000 7f3ed12b2640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f3eb006af90 0x7f3eb006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:22.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.129+0000 7f3ec1ffb640 1 -- 172.21.15.70:0/482939759 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ecc1f6340 con 0x7f3ecc15ab40 2023-10-08T05:53:22.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.319+0000 7f3ed3516640 1 -- 172.21.15.70:0/482939759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ecc1f6340 con 0x7f3ecc15ab40 2023-10-08T05:53:22.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.321+0000 7f3ec1ffb640 1 -- 172.21.15.70:0/482939759 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17414 (secure 0 0 0) 0x7f3ecc1f6340 con 0x7f3ecc15ab40 2023-10-08T05:53:22.321 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:22.322 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":106,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:19.462584+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:03.051784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:04.230588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3846159301":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/3022940786":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/3990555712":"2023-10-09T05:53:19.462537+0000","172.21.15.181:6833/2060337708":"2023-10-09T05:53:19.462537+0000","172.21.15.181:6832/2060337708":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/773795607":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/540392205":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/618139400":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/999647594":"2023-10-08T06:53:12.487289+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:22.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 -- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f3eb006af90 msgr2=0x7f3eb006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:22.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f3eb006af90 0x7f3eb006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 -- 172.21.15.70:0/482939759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc15ab40 msgr2=0x7f3ecc159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc15ab40 0x7f3ecc159180 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009f40 tx=0x7f3ec4004a40 comp rx=0 tx=0).stop 2023-10-08T05:53:22.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 -- 172.21.15.70:0/482939759 shutdown_connections 2023-10-08T05:53:22.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc15ab40 0x7f3ecc159180 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ecc0cae70 0x7f3ecc1f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f3eb006af90 0x7f3eb006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 --2- 172.21.15.70:0/482939759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0c90b0 0x7f3ecc1f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 -- 172.21.15.70:0/482939759 >> 172.21.15.70:0/482939759 conn(0x7f3ecc06a380 msgr2=0x7f3ecc0ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:22.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 -- 172.21.15.70:0/482939759 shutdown_connections 2023-10-08T05:53:22.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.323+0000 7f3ed3516640 1 -- 172.21.15.70:0/482939759 wait complete. 2023-10-08T05:53:22.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:53:03.051784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:53:04.230588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:53:22.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:53:22.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.546+0000 7fb35a1f5640 1 Processor -- start 2023-10-08T05:53:22.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.546+0000 7fb35a1f5640 1 -- start start 2023-10-08T05:53:22.626 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:22.579+0000 7fbc4ba4c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:53:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.547+0000 7fb35a1f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb35415aad0 0x7fb3540b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.547+0000 7fb35a1f5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb35406d410 0x7fb3540c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.547+0000 7fb35a1f5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35415bf30 con 0x7fb3540c82b0 2023-10-08T05:53:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.547+0000 7fb35a1f5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35415b320 con 0x7fb35406d410 2023-10-08T05:53:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.547+0000 7fb35a1f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35415b600 con 0x7fb35415aad0 2023-10-08T05:53:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.548+0000 7fb353fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb35406d410 0x7fb3540c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.548+0000 7fb353fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb35406d410 0x7fb3540c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46658/0 (socket says 172.21.15.70:46658) 2023-10-08T05:53:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.548+0000 7fb353fff640 1 -- 172.21.15.70:0/2817587127 learned_addr learned my addr 172.21.15.70:0/2817587127 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.548+0000 7fb3537fe640 1 --2- 172.21.15.70:0/2817587127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb35415aad0 0x7fb3540b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.548+0000 7fb353fff640 1 -- 172.21.15.70:0/2817587127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb35415aad0 msgr2=0x7fb3540b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.548+0000 7fb353fff640 1 --2- 172.21.15.70:0/2817587127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb35415aad0 0x7fb3540b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.548+0000 7fb353fff640 1 -- 172.21.15.70:0/2817587127 >> v1:172.21.15.70:6789/0 conn(0x7fb3540c82b0 legacy=0x7fb35406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.548+0000 7fb353fff640 1 -- 172.21.15.70:0/2817587127 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35415b8e0 con 0x7fb35406d410 2023-10-08T05:53:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.548+0000 7fb352ffd640 1 -- 172.21.15.70:0/2817587127 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733400310 0 0) 0x7fb35415bf30 con 0x7fb3540c82b0 2023-10-08T05:53:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.549+0000 7fb353fff640 1 --2- 172.21.15.70:0/2817587127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 0x7fb3540c8710 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009e50 tx=0x7fb34c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8749b301d1e9d93f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.549+0000 7fb352ffd640 1 -- 172.21.15.70:0/2817587127 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c02cb00 con 0x7fb35406d410 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.550+0000 7fb352ffd640 1 -- 172.21.15.70:0/2817587127 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c02cc60 con 0x7fb35406d410 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.550+0000 7fb352ffd640 1 -- 172.21.15.70:0/2817587127 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0343f0 con 0x7fb35406d410 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.550+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2817587127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 msgr2=0x7fb3540c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.550+0000 7fb35a1f5640 1 --2- 172.21.15.70:0/2817587127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 0x7fb3540c8710 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009e50 tx=0x7fb34c02bf60 comp rx=0 tx=0).stop 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.550+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2817587127 shutdown_connections 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.550+0000 7fb35a1f5640 1 --2- 172.21.15.70:0/2817587127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 0x7fb3540c8710 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.550+0000 7fb35a1f5640 1 --2- 172.21.15.70:0/2817587127 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb35415aad0 0x7fb3540b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.550+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2817587127 >> 172.21.15.70:0/2817587127 conn(0x7fb354069d50 msgr2=0x7fb35406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.550+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2817587127 shutdown_connections 2023-10-08T05:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.550+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2817587127 wait complete. 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.552+0000 7fb35a1f5640 1 Processor -- start 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.552+0000 7fb35a1f5640 1 -- start start 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.553+0000 7fb35a1f5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 0x7fb3541f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.553+0000 7fb35a1f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3540c82b0 0x7fb354156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.553+0000 7fb35a1f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb35415aad0 0x7fb354155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.553+0000 7fb35a1f5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3541f2170 con 0x7fb35406d410 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.553+0000 7fb35a1f5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb354157d50 con 0x7fb35415aad0 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.553+0000 7fb35a1f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb354157140 con 0x7fb3540c82b0 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.553+0000 7fb3537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 0x7fb3541f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.553+0000 7fb3537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 0x7fb3541f1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46660/0 (socket says 172.21.15.70:46660) 2023-10-08T05:53:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.553+0000 7fb3537fe640 1 -- 172.21.15.70:0/2906318016 learned_addr learned my addr 172.21.15.70:0/2906318016 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:22.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb34bfff640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3540c82b0 0x7fb354156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb353fff640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb35415aad0 0x7fb354155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb3537fe640 1 -- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3540c82b0 msgr2=0x7fb354156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb3537fe640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3540c82b0 0x7fb354156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb3537fe640 1 -- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb35415aad0 msgr2=0x7fb354155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb3537fe640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb35415aad0 0x7fb354155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb3537fe640 1 -- 172.21.15.70:0/2906318016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c009b30 con 0x7fb35406d410 2023-10-08T05:53:22.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb353fff640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb35415aad0 0x7fb354155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:22.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb3537fe640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 0x7fb3541f1c30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb34400d5d0 tx=0x7fb34400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:22.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb350ff9640 1 -- 172.21.15.70:0/2906318016 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3440125f0 con 0x7fb35406d410 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb350ff9640 1 -- 172.21.15.70:0/2906318016 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb344012770 con 0x7fb35406d410 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2906318016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb354157420 con 0x7fb35406d410 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.554+0000 7fb350ff9640 1 -- 172.21.15.70:0/2906318016 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb344012b20 con 0x7fb35406d410 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.555+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2906318016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3541579e0 con 0x7fb35406d410 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.556+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2906318016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35404e160 con 0x7fb35406d410 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.557+0000 7fb350ff9640 1 -- 172.21.15.70:0/2906318016 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fb344012c80 con 0x7fb35406d410 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.557+0000 7fb350ff9640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fb33406af90 0x7fb33406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.558+0000 7fb350ff9640 1 -- 172.21.15.70:0/2906318016 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb344089fa0 con 0x7fb35406d410 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.558+0000 7fb34bfff640 1 -- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fb33406af90 msgr2=0x7fb33406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.558+0000 7fb34bfff640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fb33406af90 0x7fb33406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:22.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.562+0000 7fb350ff9640 1 -- 172.21.15.70:0/2906318016 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb344054c40 con 0x7fb35406d410 2023-10-08T05:53:22.636 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:22.591+0000 7f23efa30040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:53:22.636 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:22.631+0000 7fbc4ba4c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:53:22.645 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:22.645+0000 7f23efa30040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:53:22.682 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:22.681+0000 7fbc4ba4c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:53:22.710 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:22.696+0000 7f23efa30040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:53:22.744 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:22.743+0000 7fbc4ba4c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:53:22.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.756+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2906318016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3540b3d70 con 0x7fb35406d410 2023-10-08T05:53:22.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.757+0000 7fb350ff9640 1 -- 172.21.15.70:0/2906318016 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb344058260 con 0x7fb35406d410 2023-10-08T05:53:22.757 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:53:22.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.759+0000 7fb34bfff640 1 -- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fb33406af90 msgr2=0x7fb33406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:22.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.759+0000 7fb34bfff640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fb33406af90 0x7fb33406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:53:22.759 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:22.759+0000 7f23efa30040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:53:22.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fb33406af90 msgr2=0x7fb33406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:22.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fb33406af90 0x7fb33406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2906318016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 msgr2=0x7fb3541f1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 0x7fb3541f1c30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fb34400d5d0 tx=0x7fb34400da90 comp rx=0 tx=0).stop 2023-10-08T05:53:22.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2906318016 shutdown_connections 2023-10-08T05:53:22.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb35415aad0 0x7fb354155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fb33406af90 0x7fb33406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb3540c82b0 0x7fb354156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 --2- 172.21.15.70:0/2906318016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb35406d410 0x7fb3541f1c30 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2906318016 >> 172.21.15.70:0/2906318016 conn(0x7fb354069d50 msgr2=0x7fb3540cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:22.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.760+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2906318016 shutdown_connections 2023-10-08T05:53:22.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.761+0000 7fb35a1f5640 1 -- 172.21.15.70:0/2906318016 wait complete. 2023-10-08T05:53:22.778 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:53:22.793 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:22.792+0000 7fbc4ba4c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:53:22.814 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:22.814+0000 7f23efa30040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:53:22.905 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:22.904+0000 7fbc4ba4c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:53:22.930 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:22.929+0000 7f23efa30040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:53:22.967 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:22.966+0000 7fbc4ba4c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:22.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.988+0000 7f5396a56640 1 Processor -- start 2023-10-08T05:53:22.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.989+0000 7f5396a56640 1 -- start start 2023-10-08T05:53:22.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.990+0000 7f5396a56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53900afbe0 0x7f53900b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.990+0000 7f5396a56640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f539006dfc0 0x7f5390070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.990+0000 7f5396a56640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53900b2620 con 0x7f5390071070 2023-10-08T05:53:22.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.990+0000 7f5396a56640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53900b2060 con 0x7f539006dfc0 2023-10-08T05:53:22.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.990+0000 7f5396a56640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53900b2340 con 0x7f53900afbe0 2023-10-08T05:53:22.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.991+0000 7f5387fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f539006dfc0 0x7f5390070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.991+0000 7f5387fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f539006dfc0 0x7f5390070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46668/0 (socket says 172.21.15.70:46668) 2023-10-08T05:53:22.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.991+0000 7f5387fff640 1 -- 172.21.15.70:0/1648973089 learned_addr learned my addr 172.21.15.70:0/1648973089 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:22.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.991+0000 7f538ffff640 1 --2- 172.21.15.70:0/1648973089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53900afbe0 0x7f53900b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.991+0000 7f5387fff640 1 -- 172.21.15.70:0/1648973089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53900afbe0 msgr2=0x7f53900b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:22.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.991+0000 7f5387fff640 1 --2- 172.21.15.70:0/1648973089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53900afbe0 0x7f53900b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.991+0000 7f5387fff640 1 -- 172.21.15.70:0/1648973089 >> v1:172.21.15.181:6789/0 conn(0x7f5390071070 legacy=0x7f53900cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.991+0000 7f5387fff640 1 -- 172.21.15.70:0/1648973089 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5390070350 con 0x7f539006dfc0 2023-10-08T05:53:22.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.992+0000 7f5387fff640 1 --2- 172.21.15.70:0/1648973089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 0x7f5390070b30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5378009870 tx=0x7f537802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc20305e1a25881 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:22.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.993+0000 7f538f7fe640 1 -- 172.21.15.70:0/1648973089 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378030840 con 0x7f539006dfc0 2023-10-08T05:53:22.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.993+0000 7f538f7fe640 1 -- 172.21.15.70:0/1648973089 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53780309a0 con 0x7f539006dfc0 2023-10-08T05:53:22.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.993+0000 7f538f7fe640 1 -- 172.21.15.70:0/1648973089 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378030d30 con 0x7f539006dfc0 2023-10-08T05:53:22.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.993+0000 7f5396a56640 1 -- 172.21.15.70:0/1648973089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 msgr2=0x7f5390070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.993+0000 7f5396a56640 1 --2- 172.21.15.70:0/1648973089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 0x7f5390070b30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5378009870 tx=0x7f537802b980 comp rx=0 tx=0).stop 2023-10-08T05:53:22.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.993+0000 7f5396a56640 1 -- 172.21.15.70:0/1648973089 shutdown_connections 2023-10-08T05:53:22.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.993+0000 7f5396a56640 1 --2- 172.21.15.70:0/1648973089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 0x7f5390070b30 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.993+0000 7f5396a56640 1 --2- 172.21.15.70:0/1648973089 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53900afbe0 0x7f53900b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.993+0000 7f5396a56640 1 -- 172.21.15.70:0/1648973089 >> 172.21.15.70:0/1648973089 conn(0x7f539006a5d0 msgr2=0x7f53900b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:22.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.994+0000 7f5396a56640 1 -- 172.21.15.70:0/1648973089 shutdown_connections 2023-10-08T05:53:22.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.994+0000 7f5396a56640 1 -- 172.21.15.70:0/1648973089 wait complete. 2023-10-08T05:53:22.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.995+0000 7f5396a56640 1 Processor -- start 2023-10-08T05:53:22.996 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:22.995+0000 7f23efa30040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:22.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.995+0000 7f5396a56640 1 -- start start 2023-10-08T05:53:22.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.996+0000 7f5396a56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 0x7f53900c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f5396a56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5390071070 0x7f53900cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f5396a56640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900afbe0 0x7f539014fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f5396a56640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900ccb80 con 0x7f539006dfc0 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f5396a56640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900c49c0 con 0x7f53900afbe0 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f5396a56640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53900c4ca0 con 0x7f5390071070 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f538ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 0x7f53900c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f538ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 0x7f53900c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46680/0 (socket says 172.21.15.70:46680) 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f538ffff640 1 -- 172.21.15.70:0/4065456393 learned_addr learned my addr 172.21.15.70:0/4065456393 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f5387fff640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5390071070 0x7f53900cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f5394ff3640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900afbe0 0x7f539014fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:22.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f538ffff640 1 -- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5390071070 msgr2=0x7f53900cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f538ffff640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5390071070 0x7f53900cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f538ffff640 1 -- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900afbe0 msgr2=0x7f539014fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:22.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f538ffff640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900afbe0 0x7f539014fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:22.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f538ffff640 1 -- 172.21.15.70:0/4065456393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53780094d0 con 0x7f539006dfc0 2023-10-08T05:53:22.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.997+0000 7f5394ff3640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900afbe0 0x7f539014fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:22.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.998+0000 7f538ffff640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 0x7f53900c23b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f538000d800 tx=0x7f538000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:23.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.998+0000 7f538d7fa640 1 -- 172.21.15.70:0/4065456393 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538000b6c0 con 0x7f539006dfc0 2023-10-08T05:53:23.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.998+0000 7f538d7fa640 1 -- 172.21.15.70:0/4065456393 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538000b820 con 0x7f539006dfc0 2023-10-08T05:53:23.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.998+0000 7f538d7fa640 1 -- 172.21.15.70:0/4065456393 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538000bbb0 con 0x7f539006dfc0 2023-10-08T05:53:23.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.998+0000 7f5396a56640 1 -- 172.21.15.70:0/4065456393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53900c4f80 con 0x7f539006dfc0 2023-10-08T05:53:23.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.998+0000 7f5396a56640 1 -- 172.21.15.70:0/4065456393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53901503e0 con 0x7f539006dfc0 2023-10-08T05:53:23.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:22.999+0000 7f5396a56640 1 -- 172.21.15.70:0/4065456393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539004e160 con 0x7f539006dfc0 2023-10-08T05:53:23.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.000+0000 7f538d7fa640 1 -- 172.21.15.70:0/4065456393 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f538000bd10 con 0x7f539006dfc0 2023-10-08T05:53:23.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.001+0000 7f538d7fa640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f537006af90 0x7f537006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.001+0000 7f538d7fa640 1 -- 172.21.15.70:0/4065456393 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5380089ff0 con 0x7f539006dfc0 2023-10-08T05:53:23.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.001+0000 7f5387fff640 1 -- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f537006af90 msgr2=0x7f537006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:23.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.002+0000 7f5387fff640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f537006af90 0x7f537006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:23.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.006+0000 7f538d7fa640 1 -- 172.21.15.70:0/4065456393 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53800bcca0 con 0x7f539006dfc0 2023-10-08T05:53:23.019 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:23.018+0000 7fbc4ba4c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:53:23.048 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:23.048+0000 7f23efa30040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:53:23.083 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:23.076+0000 7fbc4ba4c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:53:23.107 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:23.107+0000 7f23efa30040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:53:23.128 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:23.127+0000 7fbc4ba4c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:53:23.159 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:23.158+0000 7f23efa30040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:53:23.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.195+0000 7f5396a56640 1 -- 172.21.15.70:0/4065456393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53900b0ef0 con 0x7f539006dfc0 2023-10-08T05:53:23.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.196+0000 7f538d7fa640 1 -- 172.21.15.70:0/4065456393 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5380054cd0 con 0x7f539006dfc0 2023-10-08T05:53:23.196 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:53:23.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.198+0000 7f5396a56640 1 -- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f537006af90 msgr2=0x7f537006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:23.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.198+0000 7f5396a56640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f537006af90 0x7f537006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.198+0000 7f5396a56640 1 -- 172.21.15.70:0/4065456393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 msgr2=0x7f53900c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.198+0000 7f5396a56640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 0x7f53900c23b0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f538000d800 tx=0x7f538000dcc0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.198+0000 7f5396a56640 1 -- 172.21.15.70:0/4065456393 shutdown_connections 2023-10-08T05:53:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.198+0000 7f5396a56640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900afbe0 0x7f539014fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.198+0000 7f5396a56640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f537006af90 0x7f537006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.198+0000 7f5396a56640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5390071070 0x7f53900cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.198+0000 7f5396a56640 1 --2- 172.21.15.70:0/4065456393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f539006dfc0 0x7f53900c23b0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.199+0000 7f5396a56640 1 -- 172.21.15.70:0/4065456393 >> 172.21.15.70:0/4065456393 conn(0x7f539006a5d0 msgr2=0x7f539006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:23.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.199+0000 7f5396a56640 1 -- 172.21.15.70:0/4065456393 shutdown_connections 2023-10-08T05:53:23.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.199+0000 7f5396a56640 1 -- 172.21.15.70:0/4065456393 wait complete. 2023-10-08T05:53:23.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:53:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.469+0000 7f6293773640 1 Processor -- start 2023-10-08T05:53:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.469+0000 7f6293773640 1 -- start start 2023-10-08T05:53:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.471+0000 7f6293773640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 0x7f628c0cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.471+0000 7f6293773640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f628c0d31a0 0x7f628c0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.471+0000 7f6293773640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c0b8b00 con 0x7f628c0b61f0 2023-10-08T05:53:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.471+0000 7f6293773640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c0b8540 con 0x7f628c0d31a0 2023-10-08T05:53:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.471+0000 7f6293773640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0b8820 con 0x7f628c15cc70 2023-10-08T05:53:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.471+0000 7f6290d0e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f628c0d31a0 0x7f628c0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.471+0000 7f6290d0e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f628c0d31a0 0x7f628c0b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46690/0 (socket says 172.21.15.70:46690) 2023-10-08T05:53:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.471+0000 7f6290d0e640 1 -- 172.21.15.70:0/2541685008 learned_addr learned my addr 172.21.15.70:0/2541685008 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.471+0000 7f629150f640 1 --2- 172.21.15.70:0/2541685008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 0x7f628c0cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.472+0000 7f6290d0e640 1 -- 172.21.15.70:0/2541685008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 msgr2=0x7f628c0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.472+0000 7f6290d0e640 1 --2- 172.21.15.70:0/2541685008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 0x7f628c0cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.472+0000 7f6290d0e640 1 -- 172.21.15.70:0/2541685008 >> v1:172.21.15.181:6789/0 conn(0x7f628c0b61f0 legacy=0x7f628c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.472+0000 7f6290d0e640 1 -- 172.21.15.70:0/2541685008 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0cd270 con 0x7f628c0d31a0 2023-10-08T05:53:23.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.473+0000 7f6290d0e640 1 --2- 172.21.15.70:0/2541685008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 0x7f628c0b5cb0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f627c009870 tx=0x7f627c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6e392be9217e57 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:23.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.473+0000 7f6283fff640 1 -- 172.21.15.70:0/2541685008 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c031640 con 0x7f628c0d31a0 2023-10-08T05:53:23.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.474+0000 7f6283fff640 1 -- 172.21.15.70:0/2541685008 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c0317a0 con 0x7f628c0d31a0 2023-10-08T05:53:23.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.474+0000 7f6283fff640 1 -- 172.21.15.70:0/2541685008 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c031b30 con 0x7f628c0d31a0 2023-10-08T05:53:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.474+0000 7f6293773640 1 -- 172.21.15.70:0/2541685008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 msgr2=0x7f628c0b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.474+0000 7f6293773640 1 --2- 172.21.15.70:0/2541685008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 0x7f628c0b5cb0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f627c009870 tx=0x7f627c02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.475+0000 7f6293773640 1 -- 172.21.15.70:0/2541685008 shutdown_connections 2023-10-08T05:53:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.475+0000 7f6293773640 1 --2- 172.21.15.70:0/2541685008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 0x7f628c0b5cb0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.475+0000 7f6293773640 1 --2- 172.21.15.70:0/2541685008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 0x7f628c0cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.475+0000 7f6293773640 1 -- 172.21.15.70:0/2541685008 >> 172.21.15.70:0/2541685008 conn(0x7f628c0718a0 msgr2=0x7f628c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:23.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.475+0000 7f6293773640 1 -- 172.21.15.70:0/2541685008 shutdown_connections 2023-10-08T05:53:23.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.475+0000 7f6293773640 1 -- 172.21.15.70:0/2541685008 wait complete. 2023-10-08T05:53:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.477+0000 7f6293773640 1 Processor -- start 2023-10-08T05:53:23.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.477+0000 7f6293773640 1 -- start start 2023-10-08T05:53:23.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.479+0000 7f6293773640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628c0b61f0 0x7f628c1f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.479+0000 7f6293773640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 0x7f628c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.479+0000 7f6293773640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 0x7f628c0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.479+0000 7f6293773640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0cbc80 con 0x7f628c0d31a0 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.479+0000 7f6293773640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0cb000 con 0x7f628c0b61f0 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.479+0000 7f6293773640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0cb2e0 con 0x7f628c15cc70 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.479+0000 7f6290d0e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 0x7f628c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.479+0000 7f6290d0e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 0x7f628c0ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46704/0 (socket says 172.21.15.70:46704) 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.479+0000 7f6290d0e640 1 -- 172.21.15.70:0/2311695414 learned_addr learned my addr 172.21.15.70:0/2311695414 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.479+0000 7f629150f640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628c0b61f0 0x7f628c1f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.480+0000 7f6291d10640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 0x7f628c0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.480+0000 7f6290d0e640 1 -- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 msgr2=0x7f628c0c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.480+0000 7f6290d0e640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 0x7f628c0c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.480+0000 7f6290d0e640 1 -- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628c0b61f0 msgr2=0x7f628c1f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.480+0000 7f6290d0e640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628c0b61f0 0x7f628c1f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.480+0000 7f6290d0e640 1 -- 172.21.15.70:0/2311695414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c0094d0 con 0x7f628c0d31a0 2023-10-08T05:53:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.480+0000 7f6291d10640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 0x7f628c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.480+0000 7f6290d0e640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 0x7f628c0ca620 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f627c000c00 tx=0x7f627c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.480+0000 7f6281ffb640 1 -- 172.21.15.70:0/2311695414 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c03ac60 con 0x7f628c0d31a0 2023-10-08T05:53:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.481+0000 7f6281ffb640 1 -- 172.21.15.70:0/2311695414 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c03adc0 con 0x7f628c0d31a0 2023-10-08T05:53:23.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.481+0000 7f6281ffb640 1 -- 172.21.15.70:0/2311695414 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c044630 con 0x7f628c0d31a0 2023-10-08T05:53:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.481+0000 7f6293773640 1 -- 172.21.15.70:0/2311695414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0cb5c0 con 0x7f628c0d31a0 2023-10-08T05:53:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.481+0000 7f6293773640 1 -- 172.21.15.70:0/2311695414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0cc780 con 0x7f628c0d31a0 2023-10-08T05:53:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.482+0000 7f6293773640 1 -- 172.21.15.70:0/2311695414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c0cb420 con 0x7f628c0d31a0 2023-10-08T05:53:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.483+0000 7f6281ffb640 1 -- 172.21.15.70:0/2311695414 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f627c02f050 con 0x7f628c0d31a0 2023-10-08T05:53:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.484+0000 7f6281ffb640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.484+0000 7f6281ffb640 1 -- 172.21.15.70:0/2311695414 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f627c0ab950 con 0x7f628c0d31a0 2023-10-08T05:53:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.484+0000 7f629150f640 1 -- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f627006af90 msgr2=0x7f627006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.484+0000 7f629150f640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.488+0000 7f6281ffb640 1 -- 172.21.15.70:0/2311695414 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f628c0cb420 con 0x7f628c0d31a0 2023-10-08T05:53:23.513 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:23.512+0000 7fbc4ba4c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:53:23.564 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:23.563+0000 7f23efa30040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:53:23.673 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:23.672+0000 7fbc4ba4c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:53:23.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.680+0000 7f6293773640 1 -- 172.21.15.70:0/2311695414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f628c0cb420 con 0x7f628c0d31a0 2023-10-08T05:53:23.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.681+0000 7f6281ffb640 1 -- 172.21.15.70:0/2311695414 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f628c0cb420 con 0x7f628c0d31a0 2023-10-08T05:53:23.681 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:53:23.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.683+0000 7f6293773640 1 -- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f627006af90 msgr2=0x7f627006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:23.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.683+0000 7f6293773640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.684+0000 7f6293773640 1 -- 172.21.15.70:0/2311695414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 msgr2=0x7f628c0ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:23.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.684+0000 7f6293773640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 0x7f628c0ca620 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f627c000c00 tx=0x7f627c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.684+0000 7f6293773640 1 -- 172.21.15.70:0/2311695414 shutdown_connections 2023-10-08T05:53:23.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.684+0000 7f6293773640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.684+0000 7f6293773640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f628c15cc70 0x7f628c0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.684+0000 7f6293773640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f628c0d31a0 0x7f628c0ca620 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.684+0000 7f6293773640 1 --2- 172.21.15.70:0/2311695414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f628c0b61f0 0x7f628c1f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.684+0000 7f6293773640 1 -- 172.21.15.70:0/2311695414 >> 172.21.15.70:0/2311695414 conn(0x7f628c0718a0 msgr2=0x7f628c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:23.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.684+0000 7f6293773640 1 -- 172.21.15.70:0/2311695414 shutdown_connections 2023-10-08T05:53:23.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.684+0000 7f6293773640 1 -- 172.21.15.70:0/2311695414 wait complete. 2023-10-08T05:53:23.702 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:53:23.729 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:23.729+0000 7f23efa30040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:53:23.783 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:23.782+0000 7fbc4ba4c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:53:23.869 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:23.842+0000 7f23efa30040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:53:23.869 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:23.833+0000 7fbc4ba4c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:53:23.894 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:23.894+0000 7f23efa30040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:53:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.900+0000 7f5165426640 1 Processor -- start 2023-10-08T05:53:23.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.901+0000 7f5165426640 1 -- start start 2023-10-08T05:53:23.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.902+0000 7f5165426640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51601601b0 0x7f51600cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.902+0000 7f5165426640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f516006a460 0x7f51600ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.902+0000 7f5165426640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51600ce210 con 0x7f5160071c30 2023-10-08T05:53:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.902+0000 7f5165426640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51600cdee0 con 0x7f51601601b0 2023-10-08T05:53:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.902+0000 7f5165426640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5160161530 con 0x7f516006a460 2023-10-08T05:53:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.903+0000 7f515effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51601601b0 0x7f51600cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.903+0000 7f515effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51601601b0 0x7f51600cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33606/0 (socket says 172.21.15.70:33606) 2023-10-08T05:53:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.903+0000 7f515effd640 1 -- 172.21.15.70:0/1927717298 learned_addr learned my addr 172.21.15.70:0/1927717298 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.903+0000 7f5157fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5160071c30 0x7f5160069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36872/0 (socket says 172.21.15.70:36872) 2023-10-08T05:53:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.903+0000 7f515f7fe640 1 --2- 172.21.15.70:0/1927717298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f516006a460 0x7f51600ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.903+0000 7f515effd640 1 -- 172.21.15.70:0/1927717298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f516006a460 msgr2=0x7f51600ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:23.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.903+0000 7f515effd640 1 --2- 172.21.15.70:0/1927717298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f516006a460 0x7f51600ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.903+0000 7f515effd640 1 -- 172.21.15.70:0/1927717298 >> v1:172.21.15.70:6789/0 conn(0x7f5160071c30 legacy=0x7f5160069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:23.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.903+0000 7f515effd640 1 -- 172.21.15.70:0/1927717298 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160161810 con 0x7f51601601b0 2023-10-08T05:53:23.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.903+0000 7f515e7fc640 1 -- 172.21.15.70:0/1927717298 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470011000 0 0) 0x7f51600ce210 con 0x7f5160071c30 2023-10-08T05:53:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.904+0000 7f515effd640 1 --2- 172.21.15.70:0/1927717298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51601601b0 0x7f51600cd430 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5150009870 tx=0x7f515002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abb9421d707ed8e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f515e7fc640 1 -- 172.21.15.70:0/1927717298 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150030840 con 0x7f51601601b0 2023-10-08T05:53:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f515e7fc640 1 -- 172.21.15.70:0/1927717298 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51500309a0 con 0x7f51601601b0 2023-10-08T05:53:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f515e7fc640 1 -- 172.21.15.70:0/1927717298 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150030d30 con 0x7f51601601b0 2023-10-08T05:53:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f5165426640 1 -- 172.21.15.70:0/1927717298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51601601b0 msgr2=0x7f51600cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f5165426640 1 --2- 172.21.15.70:0/1927717298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51601601b0 0x7f51600cd430 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f5150009870 tx=0x7f515002b980 comp rx=0 tx=0).stop 2023-10-08T05:53:23.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f5165426640 1 -- 172.21.15.70:0/1927717298 shutdown_connections 2023-10-08T05:53:23.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f5165426640 1 --2- 172.21.15.70:0/1927717298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f516006a460 0x7f51600ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f5165426640 1 --2- 172.21.15.70:0/1927717298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51601601b0 0x7f51600cd430 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f5165426640 1 -- 172.21.15.70:0/1927717298 >> 172.21.15.70:0/1927717298 conn(0x7f516006d430 msgr2=0x7f516006d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:23.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f5165426640 1 -- 172.21.15.70:0/1927717298 shutdown_connections 2023-10-08T05:53:23.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.905+0000 7f5165426640 1 -- 172.21.15.70:0/1927717298 wait complete. 2023-10-08T05:53:23.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.907+0000 7f5165426640 1 Processor -- start 2023-10-08T05:53:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.907+0000 7f5165426640 1 -- start start 2023-10-08T05:53:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.908+0000 7f5165426640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f516006a460 0x7f51600ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.908+0000 7f5165426640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5160071c30 0x7f51601f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.908+0000 7f5165426640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51601601b0 0x7f51601f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.908+0000 7f5165426640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51601f5c40 con 0x7f5160071c30 2023-10-08T05:53:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.908+0000 7f5165426640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51600ae200 con 0x7f516006a460 2023-10-08T05:53:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.908+0000 7f5165426640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51600ad620 con 0x7f51601601b0 2023-10-08T05:53:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.908+0000 7f5157fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5160071c30 0x7f51601f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.908+0000 7f5157fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5160071c30 0x7f51601f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33616/0 (socket says 172.21.15.70:33616) 2023-10-08T05:53:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.908+0000 7f5157fff640 1 -- 172.21.15.70:0/1025969712 learned_addr learned my addr 172.21.15.70:0/1025969712 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.908+0000 7f515effd640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f516006a460 0x7f51600ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f515f7fe640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51601601b0 0x7f51601f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:23.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f5157fff640 1 -- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51601601b0 msgr2=0x7f51601f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f5157fff640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51601601b0 0x7f51601f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f5157fff640 1 -- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f516006a460 msgr2=0x7f51600ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f5157fff640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f516006a460 0x7f51600ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f5157fff640 1 -- 172.21.15.70:0/1025969712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51500094d0 con 0x7f5160071c30 2023-10-08T05:53:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f5157fff640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5160071c30 0x7f51601f1df0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f514800c430 tx=0x7f514800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f51577fe640 1 -- 172.21.15.70:0/1025969712 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51480074e0 con 0x7f5160071c30 2023-10-08T05:53:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f51577fe640 1 -- 172.21.15.70:0/1025969712 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5148007660 con 0x7f5160071c30 2023-10-08T05:53:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f5165426640 1 -- 172.21.15.70:0/1025969712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51600ad900 con 0x7f5160071c30 2023-10-08T05:53:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f51577fe640 1 -- 172.21.15.70:0/1025969712 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5148004a20 con 0x7f5160071c30 2023-10-08T05:53:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.909+0000 7f5165426640 1 -- 172.21.15.70:0/1025969712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51601f5e90 con 0x7f5160071c30 2023-10-08T05:53:23.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.911+0000 7f5165426640 1 -- 172.21.15.70:0/1025969712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51601f60b0 con 0x7f5160071c30 2023-10-08T05:53:23.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.912+0000 7f51577fe640 1 -- 172.21.15.70:0/1025969712 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f5148004b80 con 0x7f5160071c30 2023-10-08T05:53:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.912+0000 7f51577fe640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f514006af90 0x7f514006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.912+0000 7f51577fe640 1 -- 172.21.15.70:0/1025969712 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5148089920 con 0x7f5160071c30 2023-10-08T05:53:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.913+0000 7f515effd640 1 -- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f514006af90 msgr2=0x7f514006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.913+0000 7f515effd640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f514006af90 0x7f514006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:23.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:23.920+0000 7f51577fe640 1 -- 172.21.15.70:0/1025969712 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5148054540 con 0x7f5160071c30 2023-10-08T05:53:23.943 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:23.942+0000 7fbc4ba4c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:24.011 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:24.010+0000 7f23efa30040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:24.049 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:24.048+0000 7fbc4ba4c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:53:24.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.114+0000 7f515effd640 1 -- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f514006af90 msgr2=0x7f514006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:24.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.114+0000 7f515effd640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f514006af90 0x7f514006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:53:24.120 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:24.119+0000 7f23efa30040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:53:24.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.122+0000 7f5165426640 1 -- 172.21.15.70:0/1025969712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f516004e160 con 0x7f5160071c30 2023-10-08T05:53:24.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.122+0000 7f51577fe640 1 -- 172.21.15.70:0/1025969712 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 9 entries v106) v1 ==== 80+0+533 (secure 0 0 0) 0x7f5148057b60 con 0x7f5160071c30 2023-10-08T05:53:24.123 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3846159301 2023-10-09T05:53:19.462537+0000 2023-10-08T05:53:24.123 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3022940786 2023-10-09T05:53:19.462537+0000 2023-10-08T05:53:24.123 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3990555712 2023-10-09T05:53:19.462537+0000 2023-10-08T05:53:24.123 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/2060337708 2023-10-09T05:53:19.462537+0000 2023-10-08T05:53:24.123 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/2060337708 2023-10-09T05:53:19.462537+0000 2023-10-08T05:53:24.123 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/773795607 2023-10-09T05:53:19.462537+0000 2023-10-08T05:53:24.123 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/540392205 2023-10-09T05:53:19.462537+0000 2023-10-08T05:53:24.123 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/618139400 2023-10-09T05:53:19.462537+0000 2023-10-08T05:53:24.123 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/999647594 2023-10-08T06:53:12.487289+0000 2023-10-08T05:53:24.124 INFO:teuthology.orchestra.run.smithi070.stderr:listed 9 entries 2023-10-08T05:53:24.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 -- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f514006af90 msgr2=0x7f514006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:24.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f514006af90 0x7f514006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 -- 172.21.15.70:0/1025969712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5160071c30 msgr2=0x7f51601f1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:24.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5160071c30 0x7f51601f1df0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f514800c430 tx=0x7f514800c8f0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 -- 172.21.15.70:0/1025969712 shutdown_connections 2023-10-08T05:53:24.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f514006af90 0x7f514006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51601601b0 0x7f51601f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5160071c30 0x7f51601f1df0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 --2- 172.21.15.70:0/1025969712 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f516006a460 0x7f51600ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 -- 172.21.15.70:0/1025969712 >> 172.21.15.70:0/1025969712 conn(0x7f516006d430 msgr2=0x7f51600b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:24.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 -- 172.21.15.70:0/1025969712 shutdown_connections 2023-10-08T05:53:24.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.125+0000 7f5165426640 1 -- 172.21.15.70:0/1025969712 wait complete. 2023-10-08T05:53:24.142 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T05:53:24.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:24.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T05:53:24.207 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:24.206+0000 7fbc4ba4c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:24.257 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:24.256+0000 7fbc4ba4c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:24.285 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:24.285+0000 7f23efa30040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:24.292 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:53:24.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:24.293 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T05:53:24.336 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:53:24.336 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:24.337 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:24.337+0000 7f23efa30040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:24.382 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:24.381+0000 7fbc4ba4c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:53:24.390 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:53:24.390 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:53:24.390 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:24.444 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:53:24.445 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:53:24.445 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:24.460 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:24.460+0000 7f23efa30040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:53:24.500 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:53:24.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:53:24.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.712+0000 7f38ce121640 1 Processor -- start 2023-10-08T05:53:24.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.713+0000 7f38ce121640 1 -- start start 2023-10-08T05:53:24.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.714+0000 7f38ce121640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c815cdf0 0x7f38c80ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:24.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.714+0000 7f38ce121640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38c80d07c0 0x7f38c80d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:24.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.714+0000 7f38ce121640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38c80d1470 0x7f38c8160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:24.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.714+0000 7f38ce121640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c80d1900 con 0x7f38c80d07c0 2023-10-08T05:53:24.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.714+0000 7f38ce121640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c80ce610 con 0x7f38c80d1470 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.714+0000 7f38ce121640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c815d1d0 con 0x7f38c815cdf0 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.714+0000 7f38cc91e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38c80d07c0 0x7f38c80d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.714+0000 7f38cc91e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38c80d07c0 0x7f38c80d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33622/0 (socket says 172.21.15.70:33622) 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.714+0000 7f38cc91e640 1 -- 172.21.15.70:0/4174889762 learned_addr learned my addr 172.21.15.70:0/4174889762 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.715+0000 7f38cd920640 1 --2- 172.21.15.70:0/4174889762 >> v2:172.21.15.181:3300/0 conn(0x7f38c80d1470 0x7f38c8160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.715+0000 7f38cd11f640 1 --2- 172.21.15.70:0/4174889762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c815cdf0 0x7f38c80ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.715+0000 7f38cc91e640 1 -- 172.21.15.70:0/4174889762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c815cdf0 msgr2=0x7f38c80ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.715+0000 7f38cc91e640 1 --2- 172.21.15.70:0/4174889762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c815cdf0 0x7f38c80ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.715+0000 7f38cc91e640 1 -- 172.21.15.70:0/4174889762 >> v2:172.21.15.181:3300/0 conn(0x7f38c80d1470 msgr2=0x7f38c8160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.715+0000 7f38cc91e640 1 --2- 172.21.15.70:0/4174889762 >> v2:172.21.15.181:3300/0 conn(0x7f38c80d1470 0x7f38c8160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.715+0000 7f38cc91e640 1 -- 172.21.15.70:0/4174889762 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c80ceb00 con 0x7f38c80d07c0 2023-10-08T05:53:24.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.716+0000 7f38cc91e640 1 --2- 172.21.15.70:0/4174889762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 0x7f38c80d0f30 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f38c80d1c10 tx=0x7f38b802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4156298ffa2dbb98 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:24.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.716+0000 7f38bffff640 1 -- 172.21.15.70:0/4174889762 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8031740 con 0x7f38c80d07c0 2023-10-08T05:53:24.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.716+0000 7f38bffff640 1 -- 172.21.15.70:0/4174889762 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b80318a0 con 0x7f38c80d07c0 2023-10-08T05:53:24.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.716+0000 7f38bffff640 1 -- 172.21.15.70:0/4174889762 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8031c30 con 0x7f38c80d07c0 2023-10-08T05:53:24.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.717+0000 7f38ce121640 1 -- 172.21.15.70:0/4174889762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 msgr2=0x7f38c80d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:24.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.717+0000 7f38ce121640 1 --2- 172.21.15.70:0/4174889762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 0x7f38c80d0f30 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f38c80d1c10 tx=0x7f38b802b7e0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.717+0000 7f38ce121640 1 -- 172.21.15.70:0/4174889762 shutdown_connections 2023-10-08T05:53:24.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.717+0000 7f38ce121640 1 --2- 172.21.15.70:0/4174889762 >> v2:172.21.15.181:3300/0 conn(0x7f38c80d1470 0x7f38c8160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.717+0000 7f38ce121640 1 --2- 172.21.15.70:0/4174889762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 0x7f38c80d0f30 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.717+0000 7f38ce121640 1 --2- 172.21.15.70:0/4174889762 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c815cdf0 0x7f38c80ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.717+0000 7f38ce121640 1 -- 172.21.15.70:0/4174889762 >> 172.21.15.70:0/4174889762 conn(0x7f38c809bcb0 msgr2=0x7f38c80d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:24.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.717+0000 7f38ce121640 1 -- 172.21.15.70:0/4174889762 shutdown_connections 2023-10-08T05:53:24.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.717+0000 7f38ce121640 1 -- 172.21.15.70:0/4174889762 wait complete. 2023-10-08T05:53:24.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.719+0000 7f38ce121640 1 Processor -- start 2023-10-08T05:53:24.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.719+0000 7f38ce121640 1 -- start start 2023-10-08T05:53:24.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.720+0000 7f38ce121640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 0x7f38c8155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:24.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.720+0000 7f38ce121640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c80d1470 0x7f38c80c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:24.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.720+0000 7f38ce121640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38c815cdf0 0x7f38c8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:24.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.720+0000 7f38ce121640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c815a1a0 con 0x7f38c80d07c0 2023-10-08T05:53:24.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.720+0000 7f38ce121640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c8159a50 con 0x7f38c815cdf0 2023-10-08T05:53:24.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.720+0000 7f38ce121640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c8156b40 con 0x7f38c80d1470 2023-10-08T05:53:24.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.720+0000 7f38cd11f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 0x7f38c8155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:24.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.720+0000 7f38cd11f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 0x7f38c8155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33638/0 (socket says 172.21.15.70:33638) 2023-10-08T05:53:24.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cd11f640 1 -- 172.21.15.70:0/1503051745 learned_addr learned my addr 172.21.15.70:0/1503051745 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:24.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cd920640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38c815cdf0 0x7f38c8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:24.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cc91e640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c80d1470 0x7f38c80c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:24.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cd11f640 1 -- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c80d1470 msgr2=0x7f38c80c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cd11f640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c80d1470 0x7f38c80c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cd11f640 1 -- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38c815cdf0 msgr2=0x7f38c8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cd11f640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38c815cdf0 0x7f38c8159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cd11f640 1 -- 172.21.15.70:0/1503051745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b80094d0 con 0x7f38c80d07c0 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cc91e640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c80d1470 0x7f38c80c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cd920640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38c815cdf0 0x7f38c8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38cd11f640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 0x7f38c8155c10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c7f0 tx=0x7f38c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38bdffb640 1 -- 172.21.15.70:0/1503051745 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4004220 con 0x7f38c80d07c0 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38bdffb640 1 -- 172.21.15.70:0/1503051745 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c40043a0 con 0x7f38c80d07c0 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38bdffb640 1 -- 172.21.15.70:0/1503051745 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4007860 con 0x7f38c80d07c0 2023-10-08T05:53:24.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.721+0000 7f38ce121640 1 -- 172.21.15.70:0/1503051745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c81562a0 con 0x7f38c80d07c0 2023-10-08T05:53:24.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.722+0000 7f38ce121640 1 -- 172.21.15.70:0/1503051745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c8156860 con 0x7f38c80d07c0 2023-10-08T05:53:24.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.723+0000 7f38ce121640 1 -- 172.21.15.70:0/1503051745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c80b51b0 con 0x7f38c80d07c0 2023-10-08T05:53:24.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.724+0000 7f38bdffb640 1 -- 172.21.15.70:0/1503051745 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f38c4007bd0 con 0x7f38c80d07c0 2023-10-08T05:53:24.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.725+0000 7f38bdffb640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f38ac06af90 0x7f38ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:24.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.725+0000 7f38bdffb640 1 -- 172.21.15.70:0/1503051745 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f38c4089a80 con 0x7f38c80d07c0 2023-10-08T05:53:24.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.725+0000 7f38cc91e640 1 -- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f38ac06af90 msgr2=0x7f38ac06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:24.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.725+0000 7f38cc91e640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f38ac06af90 0x7f38ac06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:24.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.729+0000 7f38bdffb640 1 -- 172.21.15.70:0/1503051745 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38c40546b0 con 0x7f38c80d07c0 2023-10-08T05:53:24.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.926+0000 7f38cc91e640 1 -- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f38ac06af90 msgr2=0x7f38ac06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:24.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.926+0000 7f38cc91e640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f38ac06af90 0x7f38ac06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:53:24.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.956+0000 7f38ce121640 1 -- 172.21.15.70:0/1503051745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38c80b5ef0 con 0x7f38c80d07c0 2023-10-08T05:53:24.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.957+0000 7f38bdffb640 1 -- 172.21.15.70:0/1503051745 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f38c401e020 con 0x7f38c80d07c0 2023-10-08T05:53:24.958 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:24.958 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5384,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3148555214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3148555214},{"type":"v1","addr":"172.21.15.181:6837","nonce":3148555214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5896,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3669053115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3669053115},{"type":"v1","addr":"172.21.15.181:6836","nonce":3669053115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":7122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3775478637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3775478637},{"type":"v1","addr":"172.21.15.70:6833","nonce":3775478637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:05.225421+0000","modified":"2023-10-08T05:53:08.291318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7131},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7131":{"gid":7131,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/24744490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":24744490},{"type":"v1","addr":"172.21.15.70:6835","nonce":24744490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-08T05:53:24.958 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-10-08T05:53:24.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 -- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f38ac06af90 msgr2=0x7f38ac06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:24.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f38ac06af90 0x7f38ac06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 -- 172.21.15.70:0/1503051745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 msgr2=0x7f38c8155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 0x7f38c8155c10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c7f0 tx=0x7f38c400ccb0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 -- 172.21.15.70:0/1503051745 shutdown_connections 2023-10-08T05:53:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38c815cdf0 0x7f38c8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f38ac06af90 0x7f38ac06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38c80d1470 0x7f38c80c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 --2- 172.21.15.70:0/1503051745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38c80d07c0 0x7f38c8155c10 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 -- 172.21.15.70:0/1503051745 >> 172.21.15.70:0/1503051745 conn(0x7f38c809bcb0 msgr2=0x7f38c815f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 -- 172.21.15.70:0/1503051745 shutdown_connections 2023-10-08T05:53:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:24.960+0000 7f38ce121640 1 -- 172.21.15.70:0/1503051745 wait complete. 2023-10-08T05:53:24.977 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.172+0000 7f0acffff640 1 Processor -- start 2023-10-08T05:53:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.172+0000 7f0acffff640 1 -- start start 2023-10-08T05:53:25.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.174+0000 7f0acffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00cd8a0 0x7f0ad006b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.174+0000 7f0acffff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad00cb590 con 0x7f0ad006bf60 2023-10-08T05:53:25.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.174+0000 7f0acffff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad00cfc90 con 0x7f0ad00cf720 2023-10-08T05:53:25.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.174+0000 7f0acffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00cff70 con 0x7f0ad00cd8a0 2023-10-08T05:53:25.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0acf7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0ad006bf60 0x7f0ad00cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36874/0 (socket says 172.21.15.70:36874) 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0acf7fe640 1 -- 172.21.15.70:0/425016950 learned_addr learned my addr 172.21.15.70:0/425016950 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0aceffd640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00cd8a0 0x7f0ad006b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836127426 0 0) 0x7f0ad00cb590 con 0x7f0ad006bf60 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0abc003520 con 0x7f0ad006bf60 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1894582158 0 0) 0x7f0abc003520 con 0x7f0ad006bf60 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00cd8a0 msgr2=0x7f0ad006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00cd8a0 0x7f0ad006b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 >> v1:172.21.15.181:6789/0 conn(0x7f0ad00cf720 legacy=0x7f0ad006b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad00caf40 con 0x7f0ad006bf60 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0ac80019f0 con 0x7f0ad006bf60 2023-10-08T05:53:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 >> v1:172.21.15.70:6789/0 conn(0x7f0ad006bf60 legacy=0x7f0ad00cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc0039a0 0x7f0abc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.175+0000 7f0ace7fc640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0abc0047c0 0x7f0abc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0ace7fc640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00cd8a0 0x7f0abc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00cff70 con 0x7f0abc0039a0 2023-10-08T05:53:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc00ba60 con 0x7f0abc0047c0 2023-10-08T05:53:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc00bc20 con 0x7f0ad00cd8a0 2023-10-08T05:53:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0aceffd640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc0039a0 0x7f0abc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0ac7fff640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0abc0047c0 0x7f0abc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0aceffd640 1 -- 172.21.15.70:0/425016950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00cd8a0 msgr2=0x7f0abc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0aceffd640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ad00cd8a0 0x7f0abc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0aceffd640 1 -- 172.21.15.70:0/425016950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0abc0047c0 msgr2=0x7f0abc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:25.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0aceffd640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0abc0047c0 0x7f0abc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.176+0000 7f0aceffd640 1 -- 172.21.15.70:0/425016950 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc003520 con 0x7f0abc0039a0 2023-10-08T05:53:25.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.177+0000 7f0aceffd640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc0039a0 0x7f0abc003dc0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009a90 tx=0x7f0abc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bb2feea7a4338a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:25.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.177+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac002c6b0 con 0x7f0abc0039a0 2023-10-08T05:53:25.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.177+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac002c810 con 0x7f0abc0039a0 2023-10-08T05:53:25.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.177+0000 7f0ace7fc640 1 -- 172.21.15.70:0/425016950 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac002cba0 con 0x7f0abc0039a0 2023-10-08T05:53:25.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.178+0000 7f0acffff640 1 -- 172.21.15.70:0/425016950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc0039a0 msgr2=0x7f0abc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:25.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.178+0000 7f0acffff640 1 --2- 172.21.15.70:0/425016950 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0abc0039a0 0x7f0abc003dc0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009a90 tx=0x7f0abc0057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.178+0000 7f0acf7fe640 1 -- 172.21.15.70:0/425016950 reap_dead start 2023-10-08T05:53:25.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.178+0000 7f0acffff640 1 -- 172.21.15.70:0/425016950 shutdown_connections 2023-10-08T05:53:25.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.178+0000 7f0acffff640 1 -- 172.21.15.70:0/425016950 >> 172.21.15.70:0/425016950 conn(0x7f0ad0099b60 msgr2=0x7f0ad00d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:25.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.178+0000 7f0acffff640 1 -- 172.21.15.70:0/425016950 shutdown_connections 2023-10-08T05:53:25.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.178+0000 7f0acffff640 1 -- 172.21.15.70:0/425016950 wait complete. 2023-10-08T05:53:25.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.180+0000 7f0acffff640 1 Processor -- start 2023-10-08T05:53:25.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.180+0000 7f0acffff640 1 -- start start 2023-10-08T05:53:25.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.181+0000 7f0acffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0abc0039a0 0x7f0ad0159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.181+0000 7f0acffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0159da0 0x7f0ad015c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.181+0000 7f0acffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad015a770 0x7f0ad015ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.181+0000 7f0acffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad0159c40 con 0x7f0ad0159da0 2023-10-08T05:53:25.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.181+0000 7f0acffff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad015b140 con 0x7f0ad015a770 2023-10-08T05:53:25.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.181+0000 7f0acffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad009e440 con 0x7f0abc0039a0 2023-10-08T05:53:25.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.181+0000 7f0ac7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0159da0 0x7f0ad015c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.181+0000 7f0ac7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0159da0 0x7f0ad015c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33660/0 (socket says 172.21.15.70:33660) 2023-10-08T05:53:25.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.181+0000 7f0ac7fff640 1 -- 172.21.15.70:0/3805510297 learned_addr learned my addr 172.21.15.70:0/3805510297 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0aceffd640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0abc0039a0 0x7f0ad0159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0acf7fe640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad015a770 0x7f0ad015ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0ac7fff640 1 -- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0abc0039a0 msgr2=0x7f0ad0159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0ac7fff640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0abc0039a0 0x7f0ad0159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0ac7fff640 1 -- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad015a770 msgr2=0x7f0ad015ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0ac7fff640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad015a770 0x7f0ad015ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0ac7fff640 1 -- 172.21.15.70:0/3805510297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad006e430 con 0x7f0ad0159da0 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0aceffd640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0abc0039a0 0x7f0ad0159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0acf7fe640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad015a770 0x7f0ad015ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0ac7fff640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0159da0 0x7f0ad015c790 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800e4d0 tx=0x7f0ab800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:25.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0ac77fe640 1 -- 172.21.15.70:0/3805510297 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab80053a0 con 0x7f0ad0159da0 2023-10-08T05:53:25.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0ac77fe640 1 -- 172.21.15.70:0/3805510297 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab80035f0 con 0x7f0ad0159da0 2023-10-08T05:53:25.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0ac77fe640 1 -- 172.21.15.70:0/3805510297 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8003980 con 0x7f0ad0159da0 2023-10-08T05:53:25.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0acffff640 1 -- 172.21.15.70:0/3805510297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad015ccd0 con 0x7f0ad0159da0 2023-10-08T05:53:25.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.182+0000 7f0acffff640 1 -- 172.21.15.70:0/3805510297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad015d250 con 0x7f0ad0159da0 2023-10-08T05:53:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.183+0000 7f0acffff640 1 -- 172.21.15.70:0/3805510297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad006e5c0 con 0x7f0ad0159da0 2023-10-08T05:53:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.184+0000 7f0ac77fe640 1 -- 172.21.15.70:0/3805510297 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f0ab8003ae0 con 0x7f0ad0159da0 2023-10-08T05:53:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.185+0000 7f0ac77fe640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f0ab006afc0 0x7f0ab006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.185+0000 7f0ac77fe640 1 -- 172.21.15.70:0/3805510297 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0ab8089760 con 0x7f0ad0159da0 2023-10-08T05:53:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.185+0000 7f0aceffd640 1 -- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f0ab006afc0 msgr2=0x7f0ab006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.185+0000 7f0aceffd640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f0ab006afc0 0x7f0ab006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:25.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.193+0000 7f0ac77fe640 1 -- 172.21.15.70:0/3805510297 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ab8054400 con 0x7f0ad0159da0 2023-10-08T05:53:25.232 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.231+0000 7fbc4ba4c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:53:25.293 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:25.292+0000 7f23efa30040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:53:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.384+0000 7f0acffff640 1 -- 172.21.15.70:0/3805510297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ad004e160 con 0x7f0ad0159da0 2023-10-08T05:53:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.387+0000 7f0ac77fe640 1 -- 172.21.15.70:0/3805510297 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17414 (secure 0 0 0) 0x7f0ab8057a20 con 0x7f0ad0159da0 2023-10-08T05:53:25.387 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.387+0000 7f0aceffd640 1 -- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f0ab006afc0 msgr2=0x7f0ab006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:25.388 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":106,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:19.462584+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:03.051784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:04.230588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3846159301":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/3022940786":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/3990555712":"2023-10-09T05:53:19.462537+0000","172.21.15.181:6833/2060337708":"2023-10-09T05:53:19.462537+0000","172.21.15.181:6832/2060337708":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/773795607":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/540392205":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/618139400":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/999647594":"2023-10-08T06:53:12.487289+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:25.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.387+0000 7f0aceffd640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f0ab006afc0 0x7f0ab006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:53:25.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 -- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f0ab006afc0 msgr2=0x7f0ab006d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f0ab006afc0 0x7f0ab006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 -- 172.21.15.70:0/3805510297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0159da0 msgr2=0x7f0ad015c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0159da0 0x7f0ad015c790 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800e4d0 tx=0x7f0ab800e990 comp rx=0 tx=0).stop 2023-10-08T05:53:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 -- 172.21.15.70:0/3805510297 shutdown_connections 2023-10-08T05:53:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0ad015a770 0x7f0ad015ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0159da0 0x7f0ad015c790 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7f0ab006afc0 0x7f0ab006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 --2- 172.21.15.70:0/3805510297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0abc0039a0 0x7f0ad0159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 -- 172.21.15.70:0/3805510297 >> 172.21.15.70:0/3805510297 conn(0x7f0ad0099b60 msgr2=0x7f0ad00b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 -- 172.21.15.70:0/3805510297 shutdown_connections 2023-10-08T05:53:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.389+0000 7f0acffff640 1 -- 172.21.15.70:0/3805510297 wait complete. 2023-10-08T05:53:25.408 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:53:03.051784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:53:04.230588+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:53:25.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:53:25.532 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.532+0000 7fbc4ba4c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:53:25.591 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.591+0000 7fbc4ba4c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:53:25.604 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:25.603+0000 7f23efa30040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:53:25.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.611+0000 7fd55359e640 1 Processor -- start 2023-10-08T05:53:25.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.611+0000 7fd55359e640 1 -- start start 2023-10-08T05:53:25.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd55359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5540d1120 0x7fd5540d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd55359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5540d1500 0x7fd5541610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd55359e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5540b5060 0x7fd5540cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd55359e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd554161620 con 0x7fd5540d1500 2023-10-08T05:53:25.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd55359e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5540b1fe0 con 0x7fd5540b5060 2023-10-08T05:53:25.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd55359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540d0a80 con 0x7fd5540d1120 2023-10-08T05:53:25.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd551d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5540d1500 0x7fd5541610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd551d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5540d1500 0x7fd5541610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33676/0 (socket says 172.21.15.70:33676) 2023-10-08T05:53:25.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd551d9b640 1 -- 172.21.15.70:0/849854228 learned_addr learned my addr 172.21.15.70:0/849854228 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:25.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd55259c640 1 --2- 172.21.15.70:0/849854228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5540d1120 0x7fd5540d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd552d9d640 1 --2- 172.21.15.70:0/849854228 >> v2:172.21.15.181:3300/0 conn(0x7fd5540b5060 0x7fd5540cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd551d9b640 1 -- 172.21.15.70:0/849854228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5540d1120 msgr2=0x7fd5540d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:25.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd551d9b640 1 --2- 172.21.15.70:0/849854228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5540d1120 0x7fd5540d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd551d9b640 1 -- 172.21.15.70:0/849854228 >> v2:172.21.15.181:3300/0 conn(0x7fd5540b5060 msgr2=0x7fd5540cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:25.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd551d9b640 1 --2- 172.21.15.70:0/849854228 >> v2:172.21.15.181:3300/0 conn(0x7fd5540b5060 0x7fd5540cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.613+0000 7fd551d9b640 1 -- 172.21.15.70:0/849854228 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540b4580 con 0x7fd5540d1500 2023-10-08T05:53:25.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.615+0000 7fd551d9b640 1 --2- 172.21.15.70:0/849854228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1500 0x7fd5541610e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd55408d0b0 tx=0x7fd54802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b1602ff7806bc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:25.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.615+0000 7fd55159a640 1 -- 172.21.15.70:0/849854228 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548031740 con 0x7fd5540d1500 2023-10-08T05:53:25.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.615+0000 7fd55159a640 1 -- 172.21.15.70:0/849854228 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5480318a0 con 0x7fd5540d1500 2023-10-08T05:53:25.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.615+0000 7fd55159a640 1 -- 172.21.15.70:0/849854228 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548031c30 con 0x7fd5540d1500 2023-10-08T05:53:25.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.616+0000 7fd55359e640 1 -- 172.21.15.70:0/849854228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1500 msgr2=0x7fd5541610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:25.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.616+0000 7fd55359e640 1 --2- 172.21.15.70:0/849854228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1500 0x7fd5541610e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd55408d0b0 tx=0x7fd54802b7e0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.616+0000 7fd55359e640 1 -- 172.21.15.70:0/849854228 shutdown_connections 2023-10-08T05:53:25.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.616+0000 7fd55359e640 1 --2- 172.21.15.70:0/849854228 >> v2:172.21.15.181:3300/0 conn(0x7fd5540b5060 0x7fd5540cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.616+0000 7fd55359e640 1 --2- 172.21.15.70:0/849854228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1500 0x7fd5541610e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.616+0000 7fd55359e640 1 --2- 172.21.15.70:0/849854228 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5540d1120 0x7fd5540d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.616+0000 7fd55359e640 1 -- 172.21.15.70:0/849854228 >> 172.21.15.70:0/849854228 conn(0x7fd5540d2960 msgr2=0x7fd5540d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:25.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.616+0000 7fd55359e640 1 -- 172.21.15.70:0/849854228 shutdown_connections 2023-10-08T05:53:25.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.616+0000 7fd55359e640 1 -- 172.21.15.70:0/849854228 wait complete. 2023-10-08T05:53:25.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.618+0000 7fd55359e640 1 Processor -- start 2023-10-08T05:53:25.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.618+0000 7fd55359e640 1 -- start start 2023-10-08T05:53:25.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd55359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b5060 0x7fd5541e8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd55359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1120 0x7fd5540af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd55359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5540d1500 0x7fd55414f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd55359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540af9f0 con 0x7fd5540d1120 2023-10-08T05:53:25.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd55359e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540b6090 con 0x7fd5540b5060 2023-10-08T05:53:25.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd55359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540b04c0 con 0x7fd5540d1500 2023-10-08T05:53:25.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd551d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1120 0x7fd5540af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd551d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1120 0x7fd5540af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33690/0 (socket says 172.21.15.70:33690) 2023-10-08T05:53:25.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd551d9b640 1 -- 172.21.15.70:0/2648103526 learned_addr learned my addr 172.21.15.70:0/2648103526 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:25.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd55259c640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b5060 0x7fd5541e8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd552d9d640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5540d1500 0x7fd55414f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:25.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd551d9b640 1 -- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5540d1500 msgr2=0x7fd55414f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:25.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd551d9b640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5540d1500 0x7fd55414f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd551d9b640 1 -- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b5060 msgr2=0x7fd5541e8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:25.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd551d9b640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b5060 0x7fd5541e8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.619+0000 7fd551d9b640 1 -- 172.21.15.70:0/2648103526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5480094d0 con 0x7fd5540d1120 2023-10-08T05:53:25.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.620+0000 7fd551d9b640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1120 0x7fd5540af4b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd548000c00 tx=0x7fd5480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:25.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.620+0000 7fd542ffd640 1 -- 172.21.15.70:0/2648103526 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548033c70 con 0x7fd5540d1120 2023-10-08T05:53:25.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.620+0000 7fd542ffd640 1 -- 172.21.15.70:0/2648103526 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd548033dd0 con 0x7fd5540d1120 2023-10-08T05:53:25.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.620+0000 7fd542ffd640 1 -- 172.21.15.70:0/2648103526 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54803d6b0 con 0x7fd5540d1120 2023-10-08T05:53:25.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.620+0000 7fd55359e640 1 -- 172.21.15.70:0/2648103526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5540afc20 con 0x7fd5540d1120 2023-10-08T05:53:25.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.620+0000 7fd55359e640 1 -- 172.21.15.70:0/2648103526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5540b01e0 con 0x7fd5540d1120 2023-10-08T05:53:25.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.621+0000 7fd55359e640 1 -- 172.21.15.70:0/2648103526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd554074150 con 0x7fd5540d1120 2023-10-08T05:53:25.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.622+0000 7fd542ffd640 1 -- 172.21.15.70:0/2648103526 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 41) v1 ==== 89332+0+0 (secure 0 0 0) 0x7fd54803a040 con 0x7fd5540d1120 2023-10-08T05:53:25.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.623+0000 7fd542ffd640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fd53406af90 0x7fd53406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:25.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.623+0000 7fd542ffd640 1 -- 172.21.15.70:0/2648103526 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd5480abac0 con 0x7fd5540d1120 2023-10-08T05:53:25.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.623+0000 7fd55259c640 1 -- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fd53406af90 msgr2=0x7fd53406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1003988322 2023-10-08T05:53:25.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.623+0000 7fd55259c640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fd53406af90 0x7fd53406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:25.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.631+0000 7fd542ffd640 1 -- 172.21.15.70:0/2648103526 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd5480765c0 con 0x7fd5540d1120 2023-10-08T05:53:25.670 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:25.668+0000 7f23efa30040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:53:25.706 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.705+0000 7fbc4ba4c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:53:25.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.725+0000 7fd542ffd640 1 -- 172.21.15.70:0/2648103526 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 42) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fd548076990 con 0x7fd5540d1120 2023-10-08T05:53:25.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.726+0000 7fd542ffd640 1 -- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fd53406af90 msgr2=0x7fd53406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:25.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.726+0000 7fd542ffd640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fd53406af90 0x7fd53406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.792 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:25.792+0000 7f23efa30040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:53:25.795 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.796+0000 7fbbcdd84640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:25.795 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.796+0000 7fbbcdd84640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:25.796 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.796+0000 7fbbcdd84640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:25.796 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.796+0000 7fbbcdd84640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:25.796 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.796+0000 7fbbcdd84640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:25.796 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.796+0000 7fbbd0d8a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:25.796 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.796+0000 7fbbd0d8a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:25.797 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.796+0000 7fbbd0d8a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:25.797 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.796+0000 7fbbd0d8a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:25.797 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:25.796+0000 7fbbd0d8a640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:25.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.829+0000 7fd55359e640 1 -- 172.21.15.70:0/2648103526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5540b5e60 con 0x7fd5540d1120 2023-10-08T05:53:25.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.830+0000 7fd542ffd640 1 -- 172.21.15.70:0/2648103526 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd54807c700 con 0x7fd5540d1120 2023-10-08T05:53:25.830 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:53:25.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.832+0000 7fd55359e640 1 -- 172.21.15.70:0/2648103526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1120 msgr2=0x7fd5540af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:25.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.832+0000 7fd55359e640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1120 0x7fd5540af4b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd548000c00 tx=0x7fd5480046c0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.832+0000 7fd55359e640 1 -- 172.21.15.70:0/2648103526 shutdown_connections 2023-10-08T05:53:25.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.832+0000 7fd55359e640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:6832/1003988322,v1:172.21.15.181:6833/1003988322] conn(0x7fd53406af90 0x7fd53406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.832+0000 7fd55359e640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5540d1500 0x7fd55414f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.832+0000 7fd55359e640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5540d1120 0x7fd5540af4b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.832+0000 7fd55359e640 1 --2- 172.21.15.70:0/2648103526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5540b5060 0x7fd5541e8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:25.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.832+0000 7fd55359e640 1 -- 172.21.15.70:0/2648103526 >> 172.21.15.70:0/2648103526 conn(0x7fd5540d2960 msgr2=0x7fd5540b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:25.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.832+0000 7fd55359e640 1 -- 172.21.15.70:0/2648103526 shutdown_connections 2023-10-08T05:53:25.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:25.832+0000 7fd55359e640 1 -- 172.21.15.70:0/2648103526 wait complete. 2023-10-08T05:53:25.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:53:26.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.049+0000 7fd93535a640 1 Processor -- start 2023-10-08T05:53:26.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.049+0000 7fd93535a640 1 -- start start 2023-10-08T05:53:26.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.051+0000 7fd93535a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 0x7fd9300cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.051+0000 7fd93535a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd930160980 0x7fd930160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.051+0000 7fd93535a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9300cde10 con 0x7fd93015ce00 2023-10-08T05:53:26.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.051+0000 7fd93535a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd930161920 con 0x7fd930160980 2023-10-08T05:53:26.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.051+0000 7fd93535a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd930161640 con 0x7fd9300cd470 2023-10-08T05:53:26.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.051+0000 7fd92effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd93015ce00 0x7fd9300cb370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36878/0 (socket says 172.21.15.70:36878) 2023-10-08T05:53:26.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.051+0000 7fd92effd640 1 -- 172.21.15.70:0/1472274591 learned_addr learned my addr 172.21.15.70:0/1472274591 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:26.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.051+0000 7fd92f7fe640 1 --2- 172.21.15.70:0/1472274591 >> v2:172.21.15.70:3300/0 conn(0x7fd930160980 0x7fd930160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.052+0000 7fd92dffb640 1 -- 172.21.15.70:0/1472274591 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824451948 0 0) 0x7fd9300cde10 con 0x7fd93015ce00 2023-10-08T05:53:26.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.052+0000 7fd92e7fc640 1 --2- 172.21.15.70:0/1472274591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 0x7fd9300cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.052+0000 7fd92dffb640 1 -- 172.21.15.70:0/1472274591 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd918003520 con 0x7fd93015ce00 2023-10-08T05:53:26.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.052+0000 7fd92f7fe640 1 -- 172.21.15.70:0/1472274591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 msgr2=0x7fd9300cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:26.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.052+0000 7fd92f7fe640 1 --2- 172.21.15.70:0/1472274591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 0x7fd9300cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.052+0000 7fd92f7fe640 1 -- 172.21.15.70:0/1472274591 >> v1:172.21.15.70:6789/0 conn(0x7fd93015ce00 legacy=0x7fd9300cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.052+0000 7fd92f7fe640 1 -- 172.21.15.70:0/1472274591 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93006b690 con 0x7fd930160980 2023-10-08T05:53:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.053+0000 7fd92f7fe640 1 --2- 172.21.15.70:0/1472274591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 0x7fd930160e10 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd928009d90 tx=0x7fd92802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4210c96fb0b6d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.053+0000 7fd92dffb640 1 -- 172.21.15.70:0/1472274591 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92802cbd0 con 0x7fd930160980 2023-10-08T05:53:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.053+0000 7fd92dffb640 1 -- 172.21.15.70:0/1472274591 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92802cd30 con 0x7fd930160980 2023-10-08T05:53:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.053+0000 7fd92dffb640 1 -- 172.21.15.70:0/1472274591 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92802cea0 con 0x7fd930160980 2023-10-08T05:53:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.054+0000 7fd93535a640 1 -- 172.21.15.70:0/1472274591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 msgr2=0x7fd930160e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.054+0000 7fd93535a640 1 --2- 172.21.15.70:0/1472274591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 0x7fd930160e10 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd928009d90 tx=0x7fd92802be20 comp rx=0 tx=0).stop 2023-10-08T05:53:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.054+0000 7fd93535a640 1 -- 172.21.15.70:0/1472274591 shutdown_connections 2023-10-08T05:53:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.054+0000 7fd93535a640 1 --2- 172.21.15.70:0/1472274591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 0x7fd930160e10 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.054+0000 7fd93535a640 1 --2- 172.21.15.70:0/1472274591 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 0x7fd9300cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.054+0000 7fd93535a640 1 -- 172.21.15.70:0/1472274591 >> 172.21.15.70:0/1472274591 conn(0x7fd93006e650 msgr2=0x7fd93006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:26.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.054+0000 7fd93535a640 1 -- 172.21.15.70:0/1472274591 shutdown_connections 2023-10-08T05:53:26.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.054+0000 7fd93535a640 1 -- 172.21.15.70:0/1472274591 wait complete. 2023-10-08T05:53:26.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.056+0000 7fd93535a640 1 Processor -- start 2023-10-08T05:53:26.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.056+0000 7fd93535a640 1 -- start start 2023-10-08T05:53:26.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd93535a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 0x7fd9301f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd93535a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd93015ce00 0x7fd9301f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd93535a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 0x7fd9300b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd93535a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300b2110 con 0x7fd930160980 2023-10-08T05:53:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd93535a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300b1b50 con 0x7fd93015ce00 2023-10-08T05:53:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd93535a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300b1e30 con 0x7fd9300cd470 2023-10-08T05:53:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd92f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 0x7fd9300b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd92f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 0x7fd9300b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33694/0 (socket says 172.21.15.70:33694) 2023-10-08T05:53:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd92f7fe640 1 -- 172.21.15.70:0/4110475168 learned_addr learned my addr 172.21.15.70:0/4110475168 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd92e7fc640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd93015ce00 0x7fd9301f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd92f7fe640 1 -- 172.21.15.70:0/4110475168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 msgr2=0x7fd9301f1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd92effd640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 0x7fd9301f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd92f7fe640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 0x7fd9301f1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd92f7fe640 1 -- 172.21.15.70:0/4110475168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd93015ce00 msgr2=0x7fd9301f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.057+0000 7fd92f7fe640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd93015ce00 0x7fd9301f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.058+0000 7fd92f7fe640 1 -- 172.21.15.70:0/4110475168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd928009a70 con 0x7fd930160980 2023-10-08T05:53:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.058+0000 7fd92e7fc640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd93015ce00 0x7fd9301f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.058+0000 7fd92effd640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 0x7fd9301f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.058+0000 7fd92f7fe640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 0x7fd9300b29b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd928009eb0 tx=0x7fd928004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.058+0000 7fd9177fe640 1 -- 172.21.15.70:0/4110475168 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92802cea0 con 0x7fd930160980 2023-10-08T05:53:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.058+0000 7fd9177fe640 1 -- 172.21.15.70:0/4110475168 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92803f420 con 0x7fd930160980 2023-10-08T05:53:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.058+0000 7fd9177fe640 1 -- 172.21.15.70:0/4110475168 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92803f7d0 con 0x7fd930160980 2023-10-08T05:53:26.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.058+0000 7fd93535a640 1 -- 172.21.15.70:0/4110475168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9300b3820 con 0x7fd930160980 2023-10-08T05:53:26.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.058+0000 7fd93535a640 1 -- 172.21.15.70:0/4110475168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9300b2ef0 con 0x7fd930160980 2023-10-08T05:53:26.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.059+0000 7fd93535a640 1 -- 172.21.15.70:0/4110475168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9300b3d40 con 0x7fd930160980 2023-10-08T05:53:26.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.060+0000 7fd9177fe640 1 -- 172.21.15.70:0/4110475168 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 42) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fd92803f930 con 0x7fd930160980 2023-10-08T05:53:26.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.061+0000 7fd9177fe640 1 -- 172.21.15.70:0/4110475168 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6804+0+0 (secure 0 0 0) 0x7fd9280abc80 con 0x7fd930160980 2023-10-08T05:53:26.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.066+0000 7fd9177fe640 1 -- 172.21.15.70:0/4110475168 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9300b3d40 con 0x7fd930160980 2023-10-08T05:53:26.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.254+0000 7fd93535a640 1 -- 172.21.15.70:0/4110475168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9300b3d40 con 0x7fd930160980 2023-10-08T05:53:26.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.255+0000 7fd9177fe640 1 -- 172.21.15.70:0/4110475168 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd928039020 con 0x7fd930160980 2023-10-08T05:53:26.255 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:53:26.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.257+0000 7fd93535a640 1 -- 172.21.15.70:0/4110475168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 msgr2=0x7fd9300b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.257+0000 7fd93535a640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 0x7fd9300b29b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd928009eb0 tx=0x7fd928004a40 comp rx=0 tx=0).stop 2023-10-08T05:53:26.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.257+0000 7fd93535a640 1 -- 172.21.15.70:0/4110475168 shutdown_connections 2023-10-08T05:53:26.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.257+0000 7fd93535a640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd930160980 0x7fd9300b29b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.257+0000 7fd93535a640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd93015ce00 0x7fd9301f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.257+0000 7fd93535a640 1 --2- 172.21.15.70:0/4110475168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9300cd470 0x7fd9301f1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.257+0000 7fd93535a640 1 -- 172.21.15.70:0/4110475168 >> 172.21.15.70:0/4110475168 conn(0x7fd93006e650 msgr2=0x7fd9300afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:26.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.258+0000 7fd93535a640 1 -- 172.21.15.70:0/4110475168 shutdown_connections 2023-10-08T05:53:26.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.258+0000 7fd93535a640 1 -- 172.21.15.70:0/4110475168 wait complete. 2023-10-08T05:53:26.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:53:26.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.489+0000 7f4c2b367640 1 Processor -- start 2023-10-08T05:53:26.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.490+0000 7f4c2b367640 1 -- start start 2023-10-08T05:53:26.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.491+0000 7f4c2b367640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4c240b0680 0x7f4c2415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.491+0000 7f4c2b367640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c240ca680 0x7f4c2415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.491+0000 7f4c2b367640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c240cab60 con 0x7f4c2415a820 2023-10-08T05:53:26.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.491+0000 7f4c2b367640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c240cb960 con 0x7f4c240b0680 2023-10-08T05:53:26.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.491+0000 7f4c2b367640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c240cbdf0 con 0x7f4c240ca680 2023-10-08T05:53:26.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.491+0000 7f4c29b64640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4c240b0680 0x7f4c2415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.491+0000 7f4c29b64640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4c240b0680 0x7f4c2415e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33702/0 (socket says 172.21.15.70:33702) 2023-10-08T05:53:26.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.491+0000 7f4c29b64640 1 -- 172.21.15.70:0/1812923092 learned_addr learned my addr 172.21.15.70:0/1812923092 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:26.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.491+0000 7f4c2a365640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4c2415a820 0x7f4c240aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36886/0 (socket says 172.21.15.70:36886) 2023-10-08T05:53:26.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.492+0000 7f4c2ab66640 1 --2- 172.21.15.70:0/1812923092 >> v2:172.21.15.181:3300/0 conn(0x7f4c240ca680 0x7f4c2415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.492+0000 7f4c29b64640 1 -- 172.21.15.70:0/1812923092 >> v2:172.21.15.181:3300/0 conn(0x7f4c240ca680 msgr2=0x7f4c2415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:26.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.492+0000 7f4c29b64640 1 --2- 172.21.15.70:0/1812923092 >> v2:172.21.15.181:3300/0 conn(0x7f4c240ca680 0x7f4c2415ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.492+0000 7f4c29b64640 1 -- 172.21.15.70:0/1812923092 >> v1:172.21.15.70:6789/0 conn(0x7f4c2415a820 legacy=0x7f4c240aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.492+0000 7f4c29363640 1 -- 172.21.15.70:0/1812923092 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069472326 0 0) 0x7f4c240cab60 con 0x7f4c2415a820 2023-10-08T05:53:26.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.492+0000 7f4c29b64640 1 -- 172.21.15.70:0/1812923092 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240cb030 con 0x7f4c240b0680 2023-10-08T05:53:26.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.493+0000 7f4c29b64640 1 --2- 172.21.15.70:0/1812923092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c240b0680 0x7f4c2415e540 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4c240b0e00 tx=0x7f4c1402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c74dfc3687d513 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:26.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.493+0000 7f4c29363640 1 -- 172.21.15.70:0/1812923092 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14031870 con 0x7f4c240b0680 2023-10-08T05:53:26.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.493+0000 7f4c29363640 1 -- 172.21.15.70:0/1812923092 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c140319d0 con 0x7f4c240b0680 2023-10-08T05:53:26.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.493+0000 7f4c29363640 1 -- 172.21.15.70:0/1812923092 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14031d60 con 0x7f4c240b0680 2023-10-08T05:53:26.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.494+0000 7f4c2b367640 1 -- 172.21.15.70:0/1812923092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c240b0680 msgr2=0x7f4c2415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.494+0000 7f4c2b367640 1 --2- 172.21.15.70:0/1812923092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c240b0680 0x7f4c2415e540 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4c240b0e00 tx=0x7f4c1402b910 comp rx=0 tx=0).stop 2023-10-08T05:53:26.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.494+0000 7f4c2b367640 1 -- 172.21.15.70:0/1812923092 shutdown_connections 2023-10-08T05:53:26.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.494+0000 7f4c2b367640 1 --2- 172.21.15.70:0/1812923092 >> v2:172.21.15.181:3300/0 conn(0x7f4c240ca680 0x7f4c2415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.494+0000 7f4c2b367640 1 --2- 172.21.15.70:0/1812923092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c240b0680 0x7f4c2415e540 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.494+0000 7f4c2b367640 1 -- 172.21.15.70:0/1812923092 >> 172.21.15.70:0/1812923092 conn(0x7f4c2406c050 msgr2=0x7f4c2406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:26.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.494+0000 7f4c2b367640 1 -- 172.21.15.70:0/1812923092 shutdown_connections 2023-10-08T05:53:26.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.494+0000 7f4c2b367640 1 -- 172.21.15.70:0/1812923092 wait complete. 2023-10-08T05:53:26.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.496+0000 7f4c2b367640 1 Processor -- start 2023-10-08T05:53:26.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.496+0000 7f4c2b367640 1 -- start start 2023-10-08T05:53:26.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2b367640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c240b0680 0x7f4c240c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2b367640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ca680 0x7f4c2415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2b367640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2415a820 0x7f4c24159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2b367640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2415aca0 con 0x7f4c2415a820 2023-10-08T05:53:26.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2b367640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2415cc80 con 0x7f4c240ca680 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2b367640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2415cf60 con 0x7f4c240b0680 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2ab66640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2415a820 0x7f4c24159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2ab66640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2415a820 0x7f4c24159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33712/0 (socket says 172.21.15.70:33712) 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2ab66640 1 -- 172.21.15.70:0/1147325839 learned_addr learned my addr 172.21.15.70:0/1147325839 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2ab66640 1 -- 172.21.15.70:0/1147325839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c240b0680 msgr2=0x7f4c240c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2ab66640 1 --2- 172.21.15.70:0/1147325839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c240b0680 0x7f4c240c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2ab66640 1 -- 172.21.15.70:0/1147325839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ca680 msgr2=0x7f4c2415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2ab66640 1 --2- 172.21.15.70:0/1147325839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ca680 0x7f4c2415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.497+0000 7f4c2ab66640 1 -- 172.21.15.70:0/1147325839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c140094d0 con 0x7f4c2415a820 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.498+0000 7f4c2ab66640 1 --2- 172.21.15.70:0/1147325839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2415a820 0x7f4c24159150 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00cd90 tx=0x7f4c1c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.498+0000 7f4c1affd640 1 -- 172.21.15.70:0/1147325839 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c003b90 con 0x7f4c2415a820 2023-10-08T05:53:26.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.498+0000 7f4c1affd640 1 -- 172.21.15.70:0/1147325839 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c003d10 con 0x7f4c2415a820 2023-10-08T05:53:26.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.498+0000 7f4c1affd640 1 -- 172.21.15.70:0/1147325839 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c004260 con 0x7f4c2415a820 2023-10-08T05:53:26.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.498+0000 7f4c2b367640 1 -- 172.21.15.70:0/1147325839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2415af40 con 0x7f4c2415a820 2023-10-08T05:53:26.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.498+0000 7f4c2b367640 1 -- 172.21.15.70:0/1147325839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c240c5220 con 0x7f4c2415a820 2023-10-08T05:53:26.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.499+0000 7f4c2b367640 1 -- 172.21.15.70:0/1147325839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2415b130 con 0x7f4c2415a820 2023-10-08T05:53:26.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.500+0000 7f4c1affd640 1 -- 172.21.15.70:0/1147325839 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 42) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4c1c0043c0 con 0x7f4c2415a820 2023-10-08T05:53:26.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.501+0000 7f4c1affd640 1 -- 172.21.15.70:0/1147325839 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6804+0+0 (secure 0 0 0) 0x7f4c1c012070 con 0x7f4c2415a820 2023-10-08T05:53:26.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.505+0000 7f4c1affd640 1 -- 172.21.15.70:0/1147325839 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c1c054f40 con 0x7f4c2415a820 2023-10-08T05:53:26.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.688+0000 7f4c2b367640 1 -- 172.21.15.70:0/1147325839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c24074150 con 0x7f4c2415a820 2023-10-08T05:53:26.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.689+0000 7f4c1affd640 1 -- 172.21.15.70:0/1147325839 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c1c058560 con 0x7f4c2415a820 2023-10-08T05:53:26.689 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:53:26.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.691+0000 7f4c2b367640 1 -- 172.21.15.70:0/1147325839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2415a820 msgr2=0x7f4c24159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.691+0000 7f4c2b367640 1 --2- 172.21.15.70:0/1147325839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2415a820 0x7f4c24159150 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c00cd90 tx=0x7f4c1c00a660 comp rx=0 tx=0).stop 2023-10-08T05:53:26.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.691+0000 7f4c2b367640 1 -- 172.21.15.70:0/1147325839 shutdown_connections 2023-10-08T05:53:26.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.691+0000 7f4c2b367640 1 --2- 172.21.15.70:0/1147325839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c2415a820 0x7f4c24159150 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.691+0000 7f4c2b367640 1 --2- 172.21.15.70:0/1147325839 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c240ca680 0x7f4c2415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.691+0000 7f4c2b367640 1 --2- 172.21.15.70:0/1147325839 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c240b0680 0x7f4c240c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.691+0000 7f4c2b367640 1 -- 172.21.15.70:0/1147325839 >> 172.21.15.70:0/1147325839 conn(0x7f4c2406c050 msgr2=0x7f4c241e3730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:26.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.692+0000 7f4c2b367640 1 -- 172.21.15.70:0/1147325839 shutdown_connections 2023-10-08T05:53:26.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.692+0000 7f4c2b367640 1 -- 172.21.15.70:0/1147325839 wait complete. 2023-10-08T05:53:26.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:53:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.909+0000 7f8058f39640 1 Processor -- start 2023-10-08T05:53:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.910+0000 7f8058f39640 1 -- start start 2023-10-08T05:53:26.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.911+0000 7f8058f39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f805415ab40 0x7f80540ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.911+0000 7f8058f39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f80540c90b0 0x7f80540adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.911+0000 7f8058f39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805415b2b0 con 0x7f80540cae70 2023-10-08T05:53:26.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.911+0000 7f8058f39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805415bf30 con 0x7f80540c90b0 2023-10-08T05:53:26.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.911+0000 7f8058f39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805415f7a0 con 0x7f805415ab40 2023-10-08T05:53:26.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.911+0000 7f804bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f80540c90b0 0x7f80540adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.911+0000 7f804bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f80540c90b0 0x7f80540adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33728/0 (socket says 172.21.15.70:33728) 2023-10-08T05:53:26.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.911+0000 7f804bfff640 1 -- 172.21.15.70:0/2754352139 learned_addr learned my addr 172.21.15.70:0/2754352139 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:26.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.912+0000 7f804bfff640 1 -- 172.21.15.70:0/2754352139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f805415ab40 msgr2=0x7f80540ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:26.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.912+0000 7f805259c640 1 --2- 172.21.15.70:0/2754352139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f805415ab40 0x7f80540ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.912+0000 7f804bfff640 1 --2- 172.21.15.70:0/2754352139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f805415ab40 0x7f80540ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.912+0000 7f804bfff640 1 -- 172.21.15.70:0/2754352139 >> v1:172.21.15.181:6789/0 conn(0x7f80540cae70 legacy=0x7f80540702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:26.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.912+0000 7f804bfff640 1 -- 172.21.15.70:0/2754352139 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805415b590 con 0x7f80540c90b0 2023-10-08T05:53:26.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.913+0000 7f804bfff640 1 --2- 172.21.15.70:0/2754352139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80540c90b0 0x7f80540adc60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f803c009870 tx=0x7f803c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df15ce9ea50c8c48 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:26.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.913+0000 7f8051d9b640 1 -- 172.21.15.70:0/2754352139 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c031640 con 0x7f80540c90b0 2023-10-08T05:53:26.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.913+0000 7f8051d9b640 1 -- 172.21.15.70:0/2754352139 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c0317a0 con 0x7f80540c90b0 2023-10-08T05:53:26.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.913+0000 7f8051d9b640 1 -- 172.21.15.70:0/2754352139 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c031b30 con 0x7f80540c90b0 2023-10-08T05:53:26.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.914+0000 7f8058f39640 1 -- 172.21.15.70:0/2754352139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80540c90b0 msgr2=0x7f80540adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.914+0000 7f8058f39640 1 --2- 172.21.15.70:0/2754352139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80540c90b0 0x7f80540adc60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f803c009870 tx=0x7f803c02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.914+0000 7f8058f39640 1 -- 172.21.15.70:0/2754352139 shutdown_connections 2023-10-08T05:53:26.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.914+0000 7f8058f39640 1 --2- 172.21.15.70:0/2754352139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80540c90b0 0x7f80540adc60 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.914+0000 7f8058f39640 1 --2- 172.21.15.70:0/2754352139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f805415ab40 0x7f80540ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.914+0000 7f8058f39640 1 -- 172.21.15.70:0/2754352139 >> 172.21.15.70:0/2754352139 conn(0x7f805406a380 msgr2=0x7f80540b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:26.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.914+0000 7f8058f39640 1 -- 172.21.15.70:0/2754352139 shutdown_connections 2023-10-08T05:53:26.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.914+0000 7f8058f39640 1 -- 172.21.15.70:0/2754352139 wait complete. 2023-10-08T05:53:26.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.915+0000 7f8058f39640 1 Processor -- start 2023-10-08T05:53:26.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.916+0000 7f8058f39640 1 -- start start 2023-10-08T05:53:26.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.916+0000 7f8058f39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80540c90b0 0x7f80540c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.916+0000 7f8058f39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80540cae70 0x7f80540c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.916+0000 7f8058f39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f805415ab40 0x7f80541599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.916+0000 7f8058f39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80541595a0 con 0x7f805415ab40 2023-10-08T05:53:26.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.916+0000 7f8058f39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8054158640 con 0x7f80540cae70 2023-10-08T05:53:26.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.916+0000 7f8058f39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8054158920 con 0x7f80540c90b0 2023-10-08T05:53:26.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.917+0000 7f8052d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f805415ab40 0x7f80541599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.917+0000 7f8052d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f805415ab40 0x7f80541599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33738/0 (socket says 172.21.15.70:33738) 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.917+0000 7f8052d9d640 1 -- 172.21.15.70:0/3213897829 learned_addr learned my addr 172.21.15.70:0/3213897829 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.917+0000 7f8052d9d640 1 -- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80540c90b0 msgr2=0x7f80540c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.917+0000 7f8052d9d640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80540c90b0 0x7f80540c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.917+0000 7f8052d9d640 1 -- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80540cae70 msgr2=0x7f80540c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.917+0000 7f8052d9d640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80540cae70 0x7f80540c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.917+0000 7f8052d9d640 1 -- 172.21.15.70:0/3213897829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0094d0 con 0x7f805415ab40 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.917+0000 7f8052d9d640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f805415ab40 0x7f80541599c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f804c00c780 tx=0x7f804c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.918+0000 7f804affd640 1 -- 172.21.15.70:0/3213897829 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c004b60 con 0x7f805415ab40 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.918+0000 7f804affd640 1 -- 172.21.15.70:0/3213897829 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c00ce90 con 0x7f805415ab40 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.918+0000 7f804affd640 1 -- 172.21.15.70:0/3213897829 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c00a850 con 0x7f805415ab40 2023-10-08T05:53:26.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.918+0000 7f8058f39640 1 -- 172.21.15.70:0/3213897829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8054158c00 con 0x7f805415ab40 2023-10-08T05:53:26.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.918+0000 7f8058f39640 1 -- 172.21.15.70:0/3213897829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80541591c0 con 0x7f805415ab40 2023-10-08T05:53:26.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.919+0000 7f8058f39640 1 -- 172.21.15.70:0/3213897829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80541596e0 con 0x7f805415ab40 2023-10-08T05:53:26.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.920+0000 7f804affd640 1 -- 172.21.15.70:0/3213897829 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 43) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f804c00a9b0 con 0x7f805415ab40 2023-10-08T05:53:26.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.921+0000 7f804affd640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f803406acc0 0x7f803406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.921+0000 7f804affd640 1 -- 172.21.15.70:0/3213897829 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6804+0+0 (secure 0 0 0) 0x7f804c089930 con 0x7f805415ab40 2023-10-08T05:53:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.921+0000 7f805259c640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f803406acc0 0x7f803406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.922+0000 7f805259c640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f803406acc0 0x7f803406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8044006fd0 tx=0x7f8044008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:26.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:26.925+0000 7f804affd640 1 -- 172.21.15.70:0/3213897829 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f804c054800 con 0x7f805415ab40 2023-10-08T05:53:27.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.149+0000 7f8058f39640 1 -- 172.21.15.70:0/3213897829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8054067490 con 0x7f805415ab40 2023-10-08T05:53:27.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.150+0000 7f804affd640 1 -- 172.21.15.70:0/3213897829 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f804c057e20 con 0x7f805415ab40 2023-10-08T05:53:27.151 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:27.151 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5384,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3148555214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3148555214},{"type":"v1","addr":"172.21.15.181:6837","nonce":3148555214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5896,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3669053115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3669053115},{"type":"v1","addr":"172.21.15.181:6836","nonce":3669053115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":7122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3775478637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3775478637},{"type":"v1","addr":"172.21.15.70:6833","nonce":3775478637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:05.225421+0000","modified":"2023-10-08T05:53:08.291318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7131},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7131":{"gid":7131,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/24744490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":24744490},{"type":"v1","addr":"172.21.15.70:6835","nonce":24744490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-08T05:53:27.151 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-10-08T05:53:27.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 -- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f803406acc0 msgr2=0x7f803406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f803406acc0 0x7f803406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8044006fd0 tx=0x7f8044008040 comp rx=0 tx=0).stop 2023-10-08T05:53:27.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 -- 172.21.15.70:0/3213897829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f805415ab40 msgr2=0x7f80541599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f805415ab40 0x7f80541599c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f804c00c780 tx=0x7f804c00cc40 comp rx=0 tx=0).stop 2023-10-08T05:53:27.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 -- 172.21.15.70:0/3213897829 shutdown_connections 2023-10-08T05:53:27.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f803406acc0 0x7f803406d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f805415ab40 0x7f80541599c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80540cae70 0x7f80540c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 --2- 172.21.15.70:0/3213897829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80540c90b0 0x7f80540c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 -- 172.21.15.70:0/3213897829 >> 172.21.15.70:0/3213897829 conn(0x7f805406a380 msgr2=0x7f80540701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:27.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 -- 172.21.15.70:0/3213897829 shutdown_connections 2023-10-08T05:53:27.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.153+0000 7f8058f39640 1 -- 172.21.15.70:0/3213897829 wait complete. 2023-10-08T05:53:27.172 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:27.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.384+0000 7f3f11ea9640 1 Processor -- start 2023-10-08T05:53:27.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.385+0000 7f3f11ea9640 1 -- start start 2023-10-08T05:53:27.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.386+0000 7f3f11ea9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f0c159ac0 0x7f3f0c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.386+0000 7f3f11ea9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f0c0ad790 0x7f3f0c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.386+0000 7f3f11ea9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c15e330 0x7f3f0c070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.386+0000 7f3f11ea9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c06dee0 con 0x7f3f0c0ad790 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.386+0000 7f3f11ea9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c15d850 con 0x7f3f0c159ac0 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.386+0000 7f3f11ea9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c15db30 con 0x7f3f0c15e330 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.387+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f0c0ad790 0x7f3f0c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.387+0000 7f3f03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f0c0ad790 0x7f3f0c0702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33740/0 (socket says 172.21.15.70:33740) 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.387+0000 7f3f03fff640 1 -- 172.21.15.70:0/2374690152 learned_addr learned my addr 172.21.15.70:0/2374690152 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.387+0000 7f3f0b7fe640 1 --2- 172.21.15.70:0/2374690152 >> v2:172.21.15.181:3300/0 conn(0x7f3f0c159ac0 0x7f3f0c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.387+0000 7f3f03fff640 1 -- 172.21.15.70:0/2374690152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c15e330 msgr2=0x7f3f0c070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.387+0000 7f3f03fff640 1 --2- 172.21.15.70:0/2374690152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c15e330 0x7f3f0c070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.387+0000 7f3f03fff640 1 -- 172.21.15.70:0/2374690152 >> v2:172.21.15.181:3300/0 conn(0x7f3f0c159ac0 msgr2=0x7f3f0c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.387+0000 7f3f03fff640 1 --2- 172.21.15.70:0/2374690152 >> v2:172.21.15.181:3300/0 conn(0x7f3f0c159ac0 0x7f3f0c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.387+0000 7f3f03fff640 1 -- 172.21.15.70:0/2374690152 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c15de10 con 0x7f3f0c0ad790 2023-10-08T05:53:27.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.387+0000 7f3f0b7fe640 1 --2- 172.21.15.70:0/2374690152 >> v2:172.21.15.181:3300/0 conn(0x7f3f0c159ac0 0x7f3f0c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:27.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.388+0000 7f3f03fff640 1 --2- 172.21.15.70:0/2374690152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 0x7f3f0c0702d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c06e790 tx=0x7f3ef402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bdb7f669fdcad9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:27.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f0affd640 1 -- 172.21.15.70:0/2374690152 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4031870 con 0x7f3f0c0ad790 2023-10-08T05:53:27.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f0affd640 1 -- 172.21.15.70:0/2374690152 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef40319d0 con 0x7f3f0c0ad790 2023-10-08T05:53:27.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f0affd640 1 -- 172.21.15.70:0/2374690152 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4031d60 con 0x7f3f0c0ad790 2023-10-08T05:53:27.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f11ea9640 1 -- 172.21.15.70:0/2374690152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 msgr2=0x7f3f0c0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f11ea9640 1 --2- 172.21.15.70:0/2374690152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 0x7f3f0c0702d0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c06e790 tx=0x7f3ef402b910 comp rx=0 tx=0).stop 2023-10-08T05:53:27.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f11ea9640 1 -- 172.21.15.70:0/2374690152 shutdown_connections 2023-10-08T05:53:27.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f11ea9640 1 --2- 172.21.15.70:0/2374690152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c15e330 0x7f3f0c070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f11ea9640 1 --2- 172.21.15.70:0/2374690152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 0x7f3f0c0702d0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f11ea9640 1 --2- 172.21.15.70:0/2374690152 >> v2:172.21.15.181:3300/0 conn(0x7f3f0c159ac0 0x7f3f0c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f11ea9640 1 -- 172.21.15.70:0/2374690152 >> 172.21.15.70:0/2374690152 conn(0x7f3f0c06a380 msgr2=0x7f3f0c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:27.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f11ea9640 1 -- 172.21.15.70:0/2374690152 shutdown_connections 2023-10-08T05:53:27.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.389+0000 7f3f11ea9640 1 -- 172.21.15.70:0/2374690152 wait complete. 2023-10-08T05:53:27.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.391+0000 7f3f11ea9640 1 Processor -- start 2023-10-08T05:53:27.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.391+0000 7f3f11ea9640 1 -- start start 2023-10-08T05:53:27.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.392+0000 7f3f11ea9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 0x7f3f0c14e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.392+0000 7f3f11ea9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f0c159ac0 0x7f3f0c154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.392+0000 7f3f11ea9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c15e330 0x7f3f0c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.392+0000 7f3f11ea9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c14c360 con 0x7f3f0c0ad790 2023-10-08T05:53:27.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.392+0000 7f3f11ea9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c154c90 con 0x7f3f0c159ac0 2023-10-08T05:53:27.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.392+0000 7f3f11ea9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c154f70 con 0x7f3f0c15e330 2023-10-08T05:53:27.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.392+0000 7f3f0b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 0x7f3f0c14e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.392+0000 7f3f0b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 0x7f3f0c14e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33746/0 (socket says 172.21.15.70:33746) 2023-10-08T05:53:27.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.392+0000 7f3f0b7fe640 1 -- 172.21.15.70:0/318936674 learned_addr learned my addr 172.21.15.70:0/318936674 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:27.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f03fff640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f0c159ac0 0x7f3f0c154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f0b7fe640 1 -- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c15e330 msgr2=0x7f3f0c154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f0b7fe640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c15e330 0x7f3f0c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f0b7fe640 1 -- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f0c159ac0 msgr2=0x7f3f0c154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f0b7fe640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f0c159ac0 0x7f3f0c154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f0b7fe640 1 -- 172.21.15.70:0/318936674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef40094d0 con 0x7f3f0c0ad790 2023-10-08T05:53:27.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f0b7fe640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 0x7f3f0c14e5c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00c7f0 tx=0x7f3efc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:27.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f08ff9640 1 -- 172.21.15.70:0/318936674 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc007a30 con 0x7f3f0c0ad790 2023-10-08T05:53:27.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f08ff9640 1 -- 172.21.15.70:0/318936674 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc007b90 con 0x7f3f0c0ad790 2023-10-08T05:53:27.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f08ff9640 1 -- 172.21.15.70:0/318936674 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00ce40 con 0x7f3f0c0ad790 2023-10-08T05:53:27.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.393+0000 7f3f11ea9640 1 -- 172.21.15.70:0/318936674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c155250 con 0x7f3f0c0ad790 2023-10-08T05:53:27.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.394+0000 7f3f11ea9640 1 -- 172.21.15.70:0/318936674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c155810 con 0x7f3f0c0ad790 2023-10-08T05:53:27.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.395+0000 7f3f11ea9640 1 -- 172.21.15.70:0/318936674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c155a30 con 0x7f3f0c0ad790 2023-10-08T05:53:27.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.396+0000 7f3f08ff9640 1 -- 172.21.15.70:0/318936674 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 43) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f3efc0113f0 con 0x7f3f0c0ad790 2023-10-08T05:53:27.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.396+0000 7f3f08ff9640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3eec06acc0 0x7f3eec06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.397+0000 7f3f08ff9640 1 -- 172.21.15.70:0/318936674 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6804+0+0 (secure 0 0 0) 0x7f3efc089870 con 0x7f3f0c0ad790 2023-10-08T05:53:27.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.397+0000 7f3f03fff640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3eec06acc0 0x7f3eec06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.399+0000 7f3f03fff640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3eec06acc0 0x7f3eec06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4036000 tx=0x7f3ef403c040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:27.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.401+0000 7f3f08ff9640 1 -- 172.21.15.70:0/318936674 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3efc054810 con 0x7f3f0c0ad790 2023-10-08T05:53:27.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.596+0000 7f3f11ea9640 1 -- 172.21.15.70:0/318936674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f0c067490 con 0x7f3f0c0ad790 2023-10-08T05:53:27.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.599+0000 7f3f08ff9640 1 -- 172.21.15.70:0/318936674 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+17923 (secure 0 0 0) 0x7f3efc057e30 con 0x7f3f0c0ad790 2023-10-08T05:53:27.600 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:27.600 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":107,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:25.709807+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:03.051784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:04.230588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2094733819":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/2580364003":"2023-10-09T05:53:25.709759+0000","172.21.15.181:6833/1003988322":"2023-10-09T05:53:25.709759+0000","172.21.15.181:6832/1003988322":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/4283069774":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/3846159301":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/1194629361":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/3022940786":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/3990555712":"2023-10-09T05:53:19.462537+0000","172.21.15.181:6833/2060337708":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/881087078":"2023-10-09T05:53:25.709759+0000","172.21.15.181:6832/2060337708":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/773795607":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/540392205":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/3787865617":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/618139400":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/999647594":"2023-10-08T06:53:12.487289+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:27.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 -- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3eec06acc0 msgr2=0x7f3eec06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3eec06acc0 0x7f3eec06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4036000 tx=0x7f3ef403c040 comp rx=0 tx=0).stop 2023-10-08T05:53:27.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 -- 172.21.15.70:0/318936674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 msgr2=0x7f3f0c14e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 0x7f3f0c14e5c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00c7f0 tx=0x7f3efc00cbf0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 -- 172.21.15.70:0/318936674 shutdown_connections 2023-10-08T05:53:27.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3eec06acc0 0x7f3eec06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f0c15e330 0x7f3f0c154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f0c159ac0 0x7f3f0c154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 --2- 172.21.15.70:0/318936674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f0c0ad790 0x7f3f0c14e5c0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 -- 172.21.15.70:0/318936674 >> 172.21.15.70:0/318936674 conn(0x7f3f0c06a380 msgr2=0x7f3f0c14ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:27.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 -- 172.21.15.70:0/318936674 shutdown_connections 2023-10-08T05:53:27.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.602+0000 7f3f11ea9640 1 -- 172.21.15.70:0/318936674 wait complete. 2023-10-08T05:53:27.622 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T05:53:27.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:53:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.835+0000 7fc679c75640 1 Processor -- start 2023-10-08T05:53:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.835+0000 7fc679c75640 1 -- start start 2023-10-08T05:53:27.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc679c75640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc6740cc210 0x7fc6740b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc679c75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc674159db0 0x7fc6740b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc679c75640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6740cacf0 con 0x7fc6741599d0 2023-10-08T05:53:27.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc679c75640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6740b10f0 con 0x7fc6740cc210 2023-10-08T05:53:27.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc679c75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740ade40 con 0x7fc674159db0 2023-10-08T05:53:27.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc678c73640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc6741599d0 0x7fc6740ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36902/0 (socket says 172.21.15.70:36902) 2023-10-08T05:53:27.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc678c73640 1 -- 172.21.15.70:0/1752320310 learned_addr learned my addr 172.21.15.70:0/1752320310 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:27.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218119480 0 0) 0x7fc6740cacf0 con 0x7fc6741599d0 2023-10-08T05:53:27.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc679474640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc674159db0 0x7fc6740b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc673fff640 1 --2- 172.21.15.70:0/1752320310 >> v2:172.21.15.181:3300/0 conn(0x7fc6740cc210 0x7fc6740b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.837+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc664005730 con 0x7fc6741599d0 2023-10-08T05:53:27.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3887666203 0 0) 0x7fc664005730 con 0x7fc6741599d0 2023-10-08T05:53:27.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc674159db0 msgr2=0x7fc6740b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:27.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc674159db0 0x7fc6740b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 >> v2:172.21.15.181:3300/0 conn(0x7fc6740cc210 msgr2=0x7fc6740b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 --2- 172.21.15.70:0/1752320310 >> v2:172.21.15.181:3300/0 conn(0x7fc6740cc210 0x7fc6740b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6740ca3c0 con 0x7fc6741599d0 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc668001670 con 0x7fc6741599d0 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 >> v1:172.21.15.70:6789/0 conn(0x7fc6741599d0 legacy=0x7fc6740ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc664005b80 0x7fc664005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 0x7fc664006a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc664007170 0x7fc66400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740b10f0 con 0x7fc664006570 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66400e0d0 con 0x7fc664007170 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66400e240 con 0x7fc664005b80 2023-10-08T05:53:27.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc679474640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc674159db0 0x7fc6740b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:53:27.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.838+0000 7fc673fff640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 0x7fc664006a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.839+0000 7fc673fff640 1 -- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc664005b80 msgr2=0x7fc664005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:27.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.839+0000 7fc673fff640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc664005b80 0x7fc664005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.839+0000 7fc679474640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc664007170 0x7fc66400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.839+0000 7fc673fff640 1 -- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc664007170 msgr2=0x7fc66400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:27.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.839+0000 7fc673fff640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc664007170 0x7fc66400db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.839+0000 7fc673fff640 1 -- 172.21.15.70:0/1752320310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc664005730 con 0x7fc664006570 2023-10-08T05:53:27.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.840+0000 7fc673fff640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 0x7fc664006a50 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc660009ac0 tx=0x7fc664007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85627d0174723b4c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:27.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.840+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66002c710 con 0x7fc664006570 2023-10-08T05:53:27.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.840+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66002c870 con 0x7fc664006570 2023-10-08T05:53:27.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.840+0000 7fc6737fe640 1 -- 172.21.15.70:0/1752320310 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66002cc00 con 0x7fc664006570 2023-10-08T05:53:27.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.841+0000 7fc679c75640 1 -- 172.21.15.70:0/1752320310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 msgr2=0x7fc664006a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.841+0000 7fc679c75640 1 --2- 172.21.15.70:0/1752320310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 0x7fc664006a50 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc660009ac0 tx=0x7fc664007a00 comp rx=0 tx=0).stop 2023-10-08T05:53:27.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.841+0000 7fc679474640 1 -- 172.21.15.70:0/1752320310 reap_dead start 2023-10-08T05:53:27.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.841+0000 7fc679c75640 1 -- 172.21.15.70:0/1752320310 shutdown_connections 2023-10-08T05:53:27.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.841+0000 7fc679c75640 1 -- 172.21.15.70:0/1752320310 >> 172.21.15.70:0/1752320310 conn(0x7fc6740cb0c0 msgr2=0x7fc6740cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:27.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.841+0000 7fc679c75640 1 -- 172.21.15.70:0/1752320310 shutdown_connections 2023-10-08T05:53:27.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.841+0000 7fc679c75640 1 -- 172.21.15.70:0/1752320310 wait complete. 2023-10-08T05:53:27.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.843+0000 7fc679c75640 1 Processor -- start 2023-10-08T05:53:27.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.843+0000 7fc679c75640 1 -- start start 2023-10-08T05:53:27.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.844+0000 7fc679c75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 0x7fc6740c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.844+0000 7fc679c75640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6740ad8d0 0x7fc6740af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.844+0000 7fc679c75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc674151700 0x7fc6740acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc679c75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740c5e10 con 0x7fc664006570 2023-10-08T05:53:27.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc679c75640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740c6800 con 0x7fc6740ad8d0 2023-10-08T05:53:27.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc679c75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740ac710 con 0x7fc674151700 2023-10-08T05:53:27.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc678c73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 0x7fc6740c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc678c73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 0x7fc6740c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33760/0 (socket says 172.21.15.70:33760) 2023-10-08T05:53:27.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc678c73640 1 -- 172.21.15.70:0/3637755442 learned_addr learned my addr 172.21.15.70:0/3637755442 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:27.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc673fff640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6740ad8d0 0x7fc6740af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc679474640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc674151700 0x7fc6740acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc678c73640 1 -- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc674151700 msgr2=0x7fc6740acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc678c73640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc674151700 0x7fc6740acba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc678c73640 1 -- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6740ad8d0 msgr2=0x7fc6740af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:27.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc678c73640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6740ad8d0 0x7fc6740af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:27.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc678c73640 1 -- 172.21.15.70:0/3637755442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6740add30 con 0x7fc664006570 2023-10-08T05:53:27.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.845+0000 7fc673fff640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6740ad8d0 0x7fc6740af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:53:27.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.846+0000 7fc678c73640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 0x7fc6740c5710 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc66800c730 tx=0x7fc66800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:27.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.846+0000 7fc6717fa640 1 -- 172.21.15.70:0/3637755442 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6680074e0 con 0x7fc664006570 2023-10-08T05:53:27.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.846+0000 7fc6717fa640 1 -- 172.21.15.70:0/3637755442 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc668007660 con 0x7fc664006570 2023-10-08T05:53:27.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.846+0000 7fc6717fa640 1 -- 172.21.15.70:0/3637755442 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668004a20 con 0x7fc664006570 2023-10-08T05:53:27.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.846+0000 7fc679c75640 1 -- 172.21.15.70:0/3637755442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6740c6240 con 0x7fc664006570 2023-10-08T05:53:27.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.846+0000 7fc679c75640 1 -- 172.21.15.70:0/3637755442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6740af670 con 0x7fc664006570 2023-10-08T05:53:27.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.847+0000 7fc679c75640 1 -- 172.21.15.70:0/3637755442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67406d390 con 0x7fc664006570 2023-10-08T05:53:27.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.848+0000 7fc6717fa640 1 -- 172.21.15.70:0/3637755442 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 43) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fc668004b80 con 0x7fc664006570 2023-10-08T05:53:27.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.849+0000 7fc6717fa640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fc65806acf0 0x7fc65806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:27.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.849+0000 7fc6717fa640 1 -- 172.21.15.70:0/3637755442 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6804+0+0 (secure 0 0 0) 0x7fc668088a20 con 0x7fc664006570 2023-10-08T05:53:27.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.850+0000 7fc673fff640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fc65806acf0 0x7fc65806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:27.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.850+0000 7fc673fff640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fc65806acf0 0x7fc65806d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc660036000 tx=0x7fc660030000 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:27.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:27.854+0000 7fc6717fa640 1 -- 172.21.15.70:0/3637755442 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc6680539c0 con 0x7fc664006570 2023-10-08T05:53:28.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.078+0000 7fc679c75640 1 -- 172.21.15.70:0/3637755442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc67404e160 con 0x7fc664006570 2023-10-08T05:53:28.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.080+0000 7fc6717fa640 1 -- 172.21.15.70:0/3637755442 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fc668018090 con 0x7fc664006570 2023-10-08T05:53:28.080 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:28.080 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5384,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3148555214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3148555214},{"type":"v1","addr":"172.21.15.181:6837","nonce":3148555214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5896,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3669053115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3669053115},{"type":"v1","addr":"172.21.15.181:6836","nonce":3669053115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":7122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3775478637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3775478637},{"type":"v1","addr":"172.21.15.70:6833","nonce":3775478637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:05.225421+0000","modified":"2023-10-08T05:53:08.291318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7131},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7131":{"gid":7131,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/24744490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":24744490},{"type":"v1","addr":"172.21.15.70:6835","nonce":24744490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-08T05:53:28.081 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-10-08T05:53:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 -- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fc65806acf0 msgr2=0x7fc65806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fc65806acf0 0x7fc65806d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc660036000 tx=0x7fc660030000 comp rx=0 tx=0).stop 2023-10-08T05:53:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 -- 172.21.15.70:0/3637755442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 msgr2=0x7fc6740c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 0x7fc6740c5710 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc66800c730 tx=0x7fc66800cbf0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 -- 172.21.15.70:0/3637755442 shutdown_connections 2023-10-08T05:53:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fc65806acf0 0x7fc65806d1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc674151700 0x7fc6740acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc6740ad8d0 0x7fc6740af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 --2- 172.21.15.70:0/3637755442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc664006570 0x7fc6740c5710 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 -- 172.21.15.70:0/3637755442 >> 172.21.15.70:0/3637755442 conn(0x7fc6740cb0c0 msgr2=0x7fc6740b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:28.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 -- 172.21.15.70:0/3637755442 shutdown_connections 2023-10-08T05:53:28.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.082+0000 7fc679c75640 1 -- 172.21.15.70:0/3637755442 wait complete. 2023-10-08T05:53:28.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:53:28.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.313+0000 7f092b95e640 1 Processor -- start 2023-10-08T05:53:28.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.313+0000 7f092b95e640 1 -- start start 2023-10-08T05:53:28.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.315+0000 7f092b95e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240cb580 0x7f0924165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.315+0000 7f092b95e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092415b2b0 con 0x7f092415ab40 2023-10-08T05:53:28.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.315+0000 7f092b95e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092415b590 con 0x7f09240c90b0 2023-10-08T05:53:28.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.315+0000 7f092b95e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092415bf30 con 0x7f09240cb580 2023-10-08T05:53:28.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.315+0000 7f09296fa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f092415ab40 0x7f09240ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36918/0 (socket says 172.21.15.70:36918) 2023-10-08T05:53:28.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.315+0000 7f09296fa640 1 -- 172.21.15.70:0/2353641076 learned_addr learned my addr 172.21.15.70:0/2353641076 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:28.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.315+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828490851 0 0) 0x7f092415b2b0 con 0x7f092415ab40 2023-10-08T05:53:28.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.315+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0910003520 con 0x7f092415ab40 2023-10-08T05:53:28.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f0929efb640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240cb580 0x7f0924165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3016044008 0 0) 0x7f0910003520 con 0x7f092415ab40 2023-10-08T05:53:28.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240cb580 msgr2=0x7f0924165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:28.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240cb580 0x7f0924165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 >> v1:172.21.15.181:6789/0 conn(0x7f09240c90b0 legacy=0x7f09240cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092415b870 con 0x7f092415ab40 2023-10-08T05:53:28.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0920001690 con 0x7f092415ab40 2023-10-08T05:53:28.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 >> v1:172.21.15.70:6789/0 conn(0x7f092415ab40 legacy=0x7f09240ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240cb580 0x7f09100039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0910004020 0x7f09100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0910004d00 0x7f0910005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092415bf30 con 0x7f0910004d00 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091000bbc0 con 0x7f0910004020 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091000bd80 con 0x7f09240cb580 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.316+0000 7f0929efb640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0910004d00 0x7f0910005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.317+0000 7f09296fa640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240cb580 0x7f09100039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.317+0000 7f0928ef9640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0910004020 0x7f09100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.317+0000 7f0929efb640 1 -- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240cb580 msgr2=0x7f09100039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.317+0000 7f0929efb640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240cb580 0x7f09100039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.317+0000 7f0929efb640 1 -- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0910004020 msgr2=0x7f09100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:28.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.317+0000 7f0929efb640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0910004020 0x7f09100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.317+0000 7f0929efb640 1 -- 172.21.15.70:0/2353641076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0910003520 con 0x7f0910004d00 2023-10-08T05:53:28.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.317+0000 7f09296fa640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09240cb580 0x7f09100039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:28.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.318+0000 7f0929efb640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0910004d00 0x7f0910005180 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f091c007360 tx=0x7f09100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d71ae18cef725a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:28.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.318+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c02ccb0 con 0x7f0910004d00 2023-10-08T05:53:28.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.318+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c02ce10 con 0x7f0910004d00 2023-10-08T05:53:28.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.318+0000 7f091bfff640 1 -- 172.21.15.70:0/2353641076 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c034660 con 0x7f0910004d00 2023-10-08T05:53:28.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.319+0000 7f092b95e640 1 -- 172.21.15.70:0/2353641076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0910004d00 msgr2=0x7f0910005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.319+0000 7f092b95e640 1 --2- 172.21.15.70:0/2353641076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0910004d00 0x7f0910005180 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f091c007360 tx=0x7f09100057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.319+0000 7f0929efb640 1 -- 172.21.15.70:0/2353641076 reap_dead start 2023-10-08T05:53:28.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.319+0000 7f092b95e640 1 -- 172.21.15.70:0/2353641076 shutdown_connections 2023-10-08T05:53:28.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.319+0000 7f092b95e640 1 -- 172.21.15.70:0/2353641076 >> 172.21.15.70:0/2353641076 conn(0x7f092406a380 msgr2=0x7f09240b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:28.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.319+0000 7f092b95e640 1 -- 172.21.15.70:0/2353641076 shutdown_connections 2023-10-08T05:53:28.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.319+0000 7f092b95e640 1 -- 172.21.15.70:0/2353641076 wait complete. 2023-10-08T05:53:28.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.321+0000 7f092b95e640 1 Processor -- start 2023-10-08T05:53:28.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.321+0000 7f092b95e640 1 -- start start 2023-10-08T05:53:28.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f092b95e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0910004d00 0x7f09240ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f092b95e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092406d340 0x7f09240b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f092b95e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f092406d7c0 0x7f09240b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f092b95e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092415f7a0 con 0x7f092406d340 2023-10-08T05:53:28.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f092b95e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240b2cc0 con 0x7f0910004d00 2023-10-08T05:53:28.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f092b95e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09240c8380 con 0x7f092406d7c0 2023-10-08T05:53:28.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f0928ef9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092406d340 0x7f09240b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f0928ef9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092406d340 0x7f09240b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33782/0 (socket says 172.21.15.70:33782) 2023-10-08T05:53:28.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f0928ef9640 1 -- 172.21.15.70:0/128603808 learned_addr learned my addr 172.21.15.70:0/128603808 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:28.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f0928ef9640 1 -- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f092406d7c0 msgr2=0x7f09240b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f0928ef9640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f092406d7c0 0x7f09240b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f0928ef9640 1 -- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0910004d00 msgr2=0x7f09240ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f0928ef9640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0910004d00 0x7f09240ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.322+0000 7f0928ef9640 1 -- 172.21.15.70:0/128603808 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09240b2f30 con 0x7f092406d340 2023-10-08T05:53:28.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.323+0000 7f0928ef9640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092406d340 0x7f09240b3730 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f091400e4e0 tx=0x7f091400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.323+0000 7f0919ffb640 1 -- 172.21.15.70:0/128603808 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140052e0 con 0x7f092406d340 2023-10-08T05:53:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.323+0000 7f0919ffb640 1 -- 172.21.15.70:0/128603808 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914005440 con 0x7f092406d340 2023-10-08T05:53:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.323+0000 7f092b95e640 1 -- 172.21.15.70:0/128603808 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09240b3210 con 0x7f092406d340 2023-10-08T05:53:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.323+0000 7f0919ffb640 1 -- 172.21.15.70:0/128603808 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914003830 con 0x7f092406d340 2023-10-08T05:53:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.323+0000 7f092b95e640 1 -- 172.21.15.70:0/128603808 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09240c8660 con 0x7f092406d340 2023-10-08T05:53:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.324+0000 7f092b95e640 1 -- 172.21.15.70:0/128603808 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09240b2b20 con 0x7f092406d340 2023-10-08T05:53:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.325+0000 7f0919ffb640 1 -- 172.21.15.70:0/128603808 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 43) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f0914003990 con 0x7f092406d340 2023-10-08T05:53:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.327+0000 7f0919ffb640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f090806acf0 0x7f090806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.327+0000 7f0919ffb640 1 -- 172.21.15.70:0/128603808 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6804+0+0 (secure 0 0 0) 0x7f09140892f0 con 0x7f092406d340 2023-10-08T05:53:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.327+0000 7f09296fa640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f090806acf0 0x7f090806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.328+0000 7f09296fa640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f090806acf0 0x7f090806d1b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0920000c00 tx=0x7f0920009040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:28.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.331+0000 7f0919ffb640 1 -- 172.21.15.70:0/128603808 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09240b2b20 con 0x7f092406d340 2023-10-08T05:53:28.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.559+0000 7f092b95e640 1 -- 172.21.15.70:0/128603808 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09240b2b20 con 0x7f092406d340 2023-10-08T05:53:28.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.560+0000 7f0919ffb640 1 -- 172.21.15.70:0/128603808 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f09240b2b20 con 0x7f092406d340 2023-10-08T05:53:28.560 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:28.561 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5384,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3148555214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3148555214},{"type":"v1","addr":"172.21.15.181:6837","nonce":3148555214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5896,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3669053115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3669053115},{"type":"v1","addr":"172.21.15.181:6836","nonce":3669053115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":7122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3775478637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3775478637},{"type":"v1","addr":"172.21.15.70:6833","nonce":3775478637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:05.225421+0000","modified":"2023-10-08T05:53:08.291318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7131},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7131":{"gid":7131,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/24744490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":24744490},{"type":"v1","addr":"172.21.15.70:6835","nonce":24744490}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-10-08T05:53:28.561 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-10-08T05:53:28.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 -- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f090806acf0 msgr2=0x7f090806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f090806acf0 0x7f090806d1b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0920000c00 tx=0x7f0920009040 comp rx=0 tx=0).stop 2023-10-08T05:53:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 -- 172.21.15.70:0/128603808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092406d340 msgr2=0x7f09240b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092406d340 0x7f09240b3730 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f091400e4e0 tx=0x7f091400e9a0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 -- 172.21.15.70:0/128603808 shutdown_connections 2023-10-08T05:53:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f090806acf0 0x7f090806d1b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f092406d7c0 0x7f09240b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f092406d340 0x7f09240b3730 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 --2- 172.21.15.70:0/128603808 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0910004d00 0x7f09240ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 -- 172.21.15.70:0/128603808 >> 172.21.15.70:0/128603808 conn(0x7f092406a380 msgr2=0x7f09240701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 -- 172.21.15.70:0/128603808 shutdown_connections 2023-10-08T05:53:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.563+0000 7f092b95e640 1 -- 172.21.15.70:0/128603808 wait complete. 2023-10-08T05:53:28.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:28.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.793+0000 7f9627c7c640 1 Processor -- start 2023-10-08T05:53:28.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.793+0000 7f9627c7c640 1 -- start start 2023-10-08T05:53:28.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.795+0000 7f9627c7c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9620070db0 0x7f962015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.795+0000 7f9627c7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 0x7f962015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.795+0000 7f9627c7c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96200c9850 con 0x7f962006ad00 2023-10-08T05:53:28.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.795+0000 7f9627c7c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962006a870 con 0x7f9620070db0 2023-10-08T05:53:28.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.795+0000 7f9627c7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962015e280 con 0x7f9620071190 2023-10-08T05:53:28.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.796+0000 7f9625217640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 0x7f962015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.796+0000 7f9625a18640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9620070db0 0x7f962015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.796+0000 7f9625a18640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9620070db0 0x7f962015dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:45224/0 (socket says 172.21.15.70:45224) 2023-10-08T05:53:28.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.796+0000 7f9625217640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 0x7f962015e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:40436/0 (socket says 172.21.15.70:40436) 2023-10-08T05:53:28.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.796+0000 7f9625a18640 1 -- 172.21.15.70:0/393419717 learned_addr learned my addr 172.21.15.70:0/393419717 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:28.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.797+0000 7f9625217640 1 -- 172.21.15.70:0/393419717 >> v2:172.21.15.181:3300/0 conn(0x7f9620070db0 msgr2=0x7f962015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:28.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.797+0000 7f9625217640 1 --2- 172.21.15.70:0/393419717 >> v2:172.21.15.181:3300/0 conn(0x7f9620070db0 0x7f962015dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.797+0000 7f9625217640 1 -- 172.21.15.70:0/393419717 >> v1:172.21.15.181:6789/0 conn(0x7f962006ad00 legacy=0x7f962006b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.797+0000 7f9625217640 1 -- 172.21.15.70:0/393419717 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96200cd640 con 0x7f9620071190 2023-10-08T05:53:28.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.798+0000 7f9625217640 1 --2- 172.21.15.70:0/393419717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 0x7f962015e480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f96100097a0 tx=0x7f961002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faa62fb4fe773c8f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:28.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.798+0000 7f9624a16640 1 -- 172.21.15.70:0/393419717 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610030800 con 0x7f9620071190 2023-10-08T05:53:28.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.799+0000 7f9624a16640 1 -- 172.21.15.70:0/393419717 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9610030960 con 0x7f9620071190 2023-10-08T05:53:28.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.799+0000 7f9624a16640 1 -- 172.21.15.70:0/393419717 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610030cf0 con 0x7f9620071190 2023-10-08T05:53:28.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.799+0000 7f9627c7c640 1 -- 172.21.15.70:0/393419717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 msgr2=0x7f962015e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.799+0000 7f9627c7c640 1 --2- 172.21.15.70:0/393419717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 0x7f962015e480 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f96100097a0 tx=0x7f961002b940 comp rx=0 tx=0).stop 2023-10-08T05:53:28.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.799+0000 7f9627c7c640 1 -- 172.21.15.70:0/393419717 shutdown_connections 2023-10-08T05:53:28.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.799+0000 7f9627c7c640 1 --2- 172.21.15.70:0/393419717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 0x7f962015e480 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.799+0000 7f9627c7c640 1 --2- 172.21.15.70:0/393419717 >> v2:172.21.15.181:3300/0 conn(0x7f9620070db0 0x7f962015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.799+0000 7f9627c7c640 1 -- 172.21.15.70:0/393419717 >> 172.21.15.70:0/393419717 conn(0x7f96200b1480 msgr2=0x7f962006cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:28.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.799+0000 7f9627c7c640 1 -- 172.21.15.70:0/393419717 shutdown_connections 2023-10-08T05:53:28.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.799+0000 7f9627c7c640 1 -- 172.21.15.70:0/393419717 wait complete. 2023-10-08T05:53:28.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.801+0000 7f9627c7c640 1 Processor -- start 2023-10-08T05:53:28.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.801+0000 7f9627c7c640 1 -- start start 2023-10-08T05:53:28.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.802+0000 7f9627c7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f962006ad00 0x7f96200ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.802+0000 7f9627c7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9620070db0 0x7f96201f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.802+0000 7f9627c7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 0x7f962014ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.802+0000 7f9627c7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96201f3e00 con 0x7f962006ad00 2023-10-08T05:53:28.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.802+0000 7f9627c7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96200add60 con 0x7f9620070db0 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.802+0000 7f9627c7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96200ac8e0 con 0x7f9620071190 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.802+0000 7f9625a18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f962006ad00 0x7f96200ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.802+0000 7f9625a18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f962006ad00 0x7f96200ac1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33798/0 (socket says 172.21.15.70:33798) 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.802+0000 7f9625a18640 1 -- 172.21.15.70:0/508524537 learned_addr learned my addr 172.21.15.70:0/508524537 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.803+0000 7f9625217640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9620070db0 0x7f96201f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.803+0000 7f9626219640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 0x7f962014ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.803+0000 7f9625a18640 1 -- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 msgr2=0x7f962014ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.803+0000 7f9625a18640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 0x7f962014ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.803+0000 7f9625a18640 1 -- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9620070db0 msgr2=0x7f96201f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.803+0000 7f9625a18640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9620070db0 0x7f96201f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.803+0000 7f9625a18640 1 -- 172.21.15.70:0/508524537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9610009400 con 0x7f962006ad00 2023-10-08T05:53:28.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.803+0000 7f9625a18640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f962006ad00 0x7f96200ac1e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f961c009fd0 tx=0x7f961c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:28.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.804+0000 7f96167fc640 1 -- 172.21.15.70:0/508524537 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c00c940 con 0x7f962006ad00 2023-10-08T05:53:28.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.804+0000 7f96167fc640 1 -- 172.21.15.70:0/508524537 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961c00caa0 con 0x7f962006ad00 2023-10-08T05:53:28.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.804+0000 7f96167fc640 1 -- 172.21.15.70:0/508524537 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c00ce30 con 0x7f962006ad00 2023-10-08T05:53:28.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.804+0000 7f9627c7c640 1 -- 172.21.15.70:0/508524537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96201f3460 con 0x7f962006ad00 2023-10-08T05:53:28.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.804+0000 7f9627c7c640 1 -- 172.21.15.70:0/508524537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96201f3a20 con 0x7f962006ad00 2023-10-08T05:53:28.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.805+0000 7f9627c7c640 1 -- 172.21.15.70:0/508524537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96201f4070 con 0x7f962006ad00 2023-10-08T05:53:28.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.806+0000 7f96167fc640 1 -- 172.21.15.70:0/508524537 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f961c013470 con 0x7f962006ad00 2023-10-08T05:53:28.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.807+0000 7f96167fc640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f960406af00 0x7f960406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:28.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.807+0000 7f96167fc640 1 -- 172.21.15.70:0/508524537 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6804+0+0 (secure 0 0 0) 0x7f961c089c80 con 0x7f962006ad00 2023-10-08T05:53:28.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.808+0000 7f9625217640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f960406af00 0x7f960406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.809+0000 7f9625217640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f960406af00 0x7f960406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9610036000 tx=0x7f961003b040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.811+0000 7f96167fc640 1 -- 172.21.15.70:0/508524537 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f961c054880 con 0x7f962006ad00 2023-10-08T05:53:28.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:28.997+0000 7f9627c7c640 1 -- 172.21.15.70:0/508524537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f962006a1a0 con 0x7f962006ad00 2023-10-08T05:53:29.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.000+0000 7f96167fc640 1 -- 172.21.15.70:0/508524537 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+17923 (secure 0 0 0) 0x7f961c057ea0 con 0x7f962006ad00 2023-10-08T05:53:29.000 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:29.001 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":107,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:25.709807+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:03.051784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:04.230588+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2094733819":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/2580364003":"2023-10-09T05:53:25.709759+0000","172.21.15.181:6833/1003988322":"2023-10-09T05:53:25.709759+0000","172.21.15.181:6832/1003988322":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/4283069774":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/3846159301":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/1194629361":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/3022940786":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/3990555712":"2023-10-09T05:53:19.462537+0000","172.21.15.181:6833/2060337708":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/881087078":"2023-10-09T05:53:25.709759+0000","172.21.15.181:6832/2060337708":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/773795607":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/540392205":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/3787865617":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/618139400":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/999647594":"2023-10-08T06:53:12.487289+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:29.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 -- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f960406af00 msgr2=0x7f960406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f960406af00 0x7f960406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9610036000 tx=0x7f961003b040 comp rx=0 tx=0).stop 2023-10-08T05:53:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 -- 172.21.15.70:0/508524537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f962006ad00 msgr2=0x7f96200ac1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f962006ad00 0x7f96200ac1e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f961c009fd0 tx=0x7f961c00ed40 comp rx=0 tx=0).stop 2023-10-08T05:53:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 -- 172.21.15.70:0/508524537 shutdown_connections 2023-10-08T05:53:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f960406af00 0x7f960406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9620071190 0x7f962014ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9620070db0 0x7f96201f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 --2- 172.21.15.70:0/508524537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f962006ad00 0x7f96200ac1e0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 -- 172.21.15.70:0/508524537 >> 172.21.15.70:0/508524537 conn(0x7f96200b1480 msgr2=0x7f962015e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:29.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 -- 172.21.15.70:0/508524537 shutdown_connections 2023-10-08T05:53:29.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.002+0000 7f9627c7c640 1 -- 172.21.15.70:0/508524537 wait complete. 2023-10-08T05:53:29.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T05:53:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.230+0000 7fd69b275640 1 Processor -- start 2023-10-08T05:53:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.231+0000 7fd69b275640 1 -- start start 2023-10-08T05:53:29.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.232+0000 7fd69b275640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd69406c3a0 0x7fd6940c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.232+0000 7fd69b275640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd69406db60 0x7fd69406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.232+0000 7fd69b275640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69415b310 con 0x7fd69415aa00 2023-10-08T05:53:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.232+0000 7fd69b275640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6940ca230 con 0x7fd69406c3a0 2023-10-08T05:53:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.232+0000 7fd69b275640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69406d290 con 0x7fd69406db60 2023-10-08T05:53:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.232+0000 7fd699011640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd69406c3a0 0x7fd6940c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.232+0000 7fd699812640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd69415aa00 0x7fd6940b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36930/0 (socket says 172.21.15.70:36930) 2023-10-08T05:53:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.232+0000 7fd699812640 1 -- 172.21.15.70:0/353064016 learned_addr learned my addr 172.21.15.70:0/353064016 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.233+0000 7fd699011640 1 -- 172.21.15.70:0/353064016 >> v2:172.21.15.181:3300/0 conn(0x7fd69406db60 msgr2=0x7fd69406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.233+0000 7fd698810640 1 --2- 172.21.15.70:0/353064016 >> v2:172.21.15.181:3300/0 conn(0x7fd69406db60 0x7fd69406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.233+0000 7fd699011640 1 --2- 172.21.15.70:0/353064016 >> v2:172.21.15.181:3300/0 conn(0x7fd69406db60 0x7fd69406c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.233+0000 7fd699011640 1 -- 172.21.15.70:0/353064016 >> v1:172.21.15.70:6789/0 conn(0x7fd69415aa00 legacy=0x7fd6940b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.233+0000 7fd699011640 1 -- 172.21.15.70:0/353064016 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940c9860 con 0x7fd69406c3a0 2023-10-08T05:53:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.233+0000 7fd698810640 1 --2- 172.21.15.70:0/353064016 >> v2:172.21.15.181:3300/0 conn(0x7fd69406db60 0x7fd69406c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.233+0000 7fd693fff640 1 -- 172.21.15.70:0/353064016 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377775715 0 0) 0x7fd68c001490 con 0x7fd69415aa00 2023-10-08T05:53:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.234+0000 7fd699011640 1 --2- 172.21.15.70:0/353064016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69406c3a0 0x7fd6940c9cf0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd6940715c0 tx=0x7fd68402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8a8dd0129ecb00 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.234+0000 7fd693fff640 1 -- 172.21.15.70:0/353064016 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684031830 con 0x7fd69406c3a0 2023-10-08T05:53:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.234+0000 7fd693fff640 1 -- 172.21.15.70:0/353064016 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd684031990 con 0x7fd69406c3a0 2023-10-08T05:53:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.234+0000 7fd693fff640 1 -- 172.21.15.70:0/353064016 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684031d20 con 0x7fd69406c3a0 2023-10-08T05:53:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.235+0000 7fd69b275640 1 -- 172.21.15.70:0/353064016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69406c3a0 msgr2=0x7fd6940c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.235+0000 7fd69b275640 1 --2- 172.21.15.70:0/353064016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69406c3a0 0x7fd6940c9cf0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd6940715c0 tx=0x7fd68402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.235+0000 7fd69b275640 1 -- 172.21.15.70:0/353064016 shutdown_connections 2023-10-08T05:53:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.235+0000 7fd69b275640 1 --2- 172.21.15.70:0/353064016 >> v2:172.21.15.181:3300/0 conn(0x7fd69406db60 0x7fd69406c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.235+0000 7fd69b275640 1 --2- 172.21.15.70:0/353064016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69406c3a0 0x7fd6940c9cf0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.235+0000 7fd69b275640 1 -- 172.21.15.70:0/353064016 >> 172.21.15.70:0/353064016 conn(0x7fd69406a5d0 msgr2=0x7fd6940b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.235+0000 7fd69b275640 1 -- 172.21.15.70:0/353064016 shutdown_connections 2023-10-08T05:53:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.235+0000 7fd69b275640 1 -- 172.21.15.70:0/353064016 wait complete. 2023-10-08T05:53:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.237+0000 7fd69b275640 1 Processor -- start 2023-10-08T05:53:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.237+0000 7fd69b275640 1 -- start start 2023-10-08T05:53:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.238+0000 7fd69b275640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd69406c3a0 0x7fd694156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.238+0000 7fd69b275640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd69406db60 0x7fd6940ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.238+0000 7fd69b275640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69415aa00 0x7fd6940cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.238+0000 7fd69b275640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69415c540 con 0x7fd69415aa00 2023-10-08T05:53:29.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.238+0000 7fd69b275640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd694157070 con 0x7fd69406c3a0 2023-10-08T05:53:29.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.238+0000 7fd69b275640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940c3cf0 con 0x7fd69406db60 2023-10-08T05:53:29.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd699812640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69415aa00 0x7fd6940cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd699812640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69415aa00 0x7fd6940cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33818/0 (socket says 172.21.15.70:33818) 2023-10-08T05:53:29.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd699812640 1 -- 172.21.15.70:0/19540266 learned_addr learned my addr 172.21.15.70:0/19540266 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd699011640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd69406c3a0 0x7fd694156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd698810640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd69406db60 0x7fd6940ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd699812640 1 -- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd69406db60 msgr2=0x7fd6940ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd699812640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd69406db60 0x7fd6940ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd699812640 1 -- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd69406c3a0 msgr2=0x7fd694156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd699812640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd69406c3a0 0x7fd694156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd699812640 1 -- 172.21.15.70:0/19540266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd684009400 con 0x7fd69415aa00 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.239+0000 7fd699812640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69415aa00 0x7fd6940cac00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c780 tx=0x7fd68c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.240+0000 7fd691ffb640 1 -- 172.21.15.70:0/19540266 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00ce90 con 0x7fd69415aa00 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.240+0000 7fd691ffb640 1 -- 172.21.15.70:0/19540266 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c00a640 con 0x7fd69415aa00 2023-10-08T05:53:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.240+0000 7fd69b275640 1 -- 172.21.15.70:0/19540266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69415b9c0 con 0x7fd69415aa00 2023-10-08T05:53:29.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.240+0000 7fd691ffb640 1 -- 172.21.15.70:0/19540266 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c00a9d0 con 0x7fd69415aa00 2023-10-08T05:53:29.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.240+0000 7fd69b275640 1 -- 172.21.15.70:0/19540266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69415bf80 con 0x7fd69415aa00 2023-10-08T05:53:29.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.241+0000 7fd69b275640 1 -- 172.21.15.70:0/19540266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69406cc70 con 0x7fd69415aa00 2023-10-08T05:53:29.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.242+0000 7fd691ffb640 1 -- 172.21.15.70:0/19540266 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fd68c00ab30 con 0x7fd69415aa00 2023-10-08T05:53:29.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.243+0000 7fd691ffb640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd68006af00 0x7fd68006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.243+0000 7fd691ffb640 1 -- 172.21.15.70:0/19540266 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6804+0+0 (secure 0 0 0) 0x7fd68c088e10 con 0x7fd69415aa00 2023-10-08T05:53:29.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.244+0000 7fd699011640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd68006af00 0x7fd68006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.245+0000 7fd699011640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd68006af00 0x7fd68006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd684036000 tx=0x7fd68403c040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:29.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.248+0000 7fd691ffb640 1 -- 172.21.15.70:0/19540266 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd68c053a80 con 0x7fd69415aa00 2023-10-08T05:53:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.475+0000 7fd69b275640 1 -- 172.21.15.70:0/19540266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd6940c3eb0 con 0x7fd69415aa00 2023-10-08T05:53:29.478 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T05:53:29.477+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T05:53:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.481+0000 7fd691ffb640 1 -- 172.21.15.70:0/19540266 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd68c018070 con 0x7fd69415aa00 2023-10-08T05:53:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T05:53:29.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.483+0000 7fd69b275640 1 -- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd68006af00 msgr2=0x7fd68006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.483+0000 7fd69b275640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd68006af00 0x7fd68006d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd684036000 tx=0x7fd68403c040 comp rx=0 tx=0).stop 2023-10-08T05:53:29.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.483+0000 7fd69b275640 1 -- 172.21.15.70:0/19540266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69415aa00 msgr2=0x7fd6940cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.483+0000 7fd69b275640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69415aa00 0x7fd6940cac00 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd68c00c780 tx=0x7fd68c00cc40 comp rx=0 tx=0).stop 2023-10-08T05:53:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.484+0000 7fd69b275640 1 -- 172.21.15.70:0/19540266 shutdown_connections 2023-10-08T05:53:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.484+0000 7fd69b275640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd68006af00 0x7fd68006d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.484+0000 7fd69b275640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd69415aa00 0x7fd6940cac00 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.484+0000 7fd69b275640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd69406db60 0x7fd6940ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.484+0000 7fd69b275640 1 --2- 172.21.15.70:0/19540266 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd69406c3a0 0x7fd694156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.484+0000 7fd69b275640 1 -- 172.21.15.70:0/19540266 >> 172.21.15.70:0/19540266 conn(0x7fd69406a5d0 msgr2=0x7fd6940b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.484+0000 7fd69b275640 1 -- 172.21.15.70:0/19540266 shutdown_connections 2023-10-08T05:53:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.485+0000 7fd69b275640 1 -- 172.21.15.70:0/19540266 wait complete. 2023-10-08T05:53:29.503 INFO:tasks.ceph.mds.c.smithi070.stderr: -2635> 2023-10-08T05:53:10.143+0000 7fa8d696f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:53:29.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T05:53:29.567 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:53:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.700+0000 7f2584209640 1 Processor -- start 2023-10-08T05:53:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.701+0000 7f2584209640 1 -- start start 2023-10-08T05:53:29.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.702+0000 7f2584209640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c06dfc0 0x7f257c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.702+0000 7f2584209640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f257c071070 0x7f257c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.702+0000 7f2584209640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f257c0b1cc0 con 0x7f257c0afbe0 2023-10-08T05:53:29.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.702+0000 7f2584209640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c0b2620 con 0x7f257c071070 2023-10-08T05:53:29.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.702+0000 7f2584209640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0b2060 con 0x7f257c06dfc0 2023-10-08T05:53:29.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.702+0000 7f25827a6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f257c071070 0x7f257c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.702+0000 7f25827a6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f257c071070 0x7f257c0cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33856/0 (socket says 172.21.15.70:33856) 2023-10-08T05:53:29.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.702+0000 7f25827a6640 1 -- 172.21.15.70:0/2928125486 learned_addr learned my addr 172.21.15.70:0/2928125486 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:29.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.703+0000 7f25817a4640 1 --2- 172.21.15.70:0/2928125486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c06dfc0 0x7f257c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.703+0000 7f25827a6640 1 -- 172.21.15.70:0/2928125486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c06dfc0 msgr2=0x7f257c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:29.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.703+0000 7f25827a6640 1 --2- 172.21.15.70:0/2928125486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c06dfc0 0x7f257c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.703+0000 7f25827a6640 1 -- 172.21.15.70:0/2928125486 >> v1:172.21.15.181:6789/0 conn(0x7f257c0afbe0 legacy=0x7f257c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:29.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.703+0000 7f25827a6640 1 -- 172.21.15.70:0/2928125486 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0b2340 con 0x7f257c071070 2023-10-08T05:53:29.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.704+0000 7f25827a6640 1 --2- 172.21.15.70:0/2928125486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c071070 0x7f257c0cb130 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f2574009e50 tx=0x7f257402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df2d0f5891fc07d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:29.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.704+0000 7f2580fa3640 1 -- 172.21.15.70:0/2928125486 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257402ca70 con 0x7f257c071070 2023-10-08T05:53:29.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.704+0000 7f2580fa3640 1 -- 172.21.15.70:0/2928125486 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257402cbd0 con 0x7f257c071070 2023-10-08T05:53:29.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.704+0000 7f2580fa3640 1 -- 172.21.15.70:0/2928125486 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25740343f0 con 0x7f257c071070 2023-10-08T05:53:29.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.705+0000 7f2584209640 1 -- 172.21.15.70:0/2928125486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c071070 msgr2=0x7f257c0cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.705+0000 7f2584209640 1 --2- 172.21.15.70:0/2928125486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c071070 0x7f257c0cb130 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f2574009e50 tx=0x7f257402bed0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.705+0000 7f2584209640 1 -- 172.21.15.70:0/2928125486 shutdown_connections 2023-10-08T05:53:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.705+0000 7f2584209640 1 --2- 172.21.15.70:0/2928125486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c071070 0x7f257c0cb130 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.705+0000 7f2584209640 1 --2- 172.21.15.70:0/2928125486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c06dfc0 0x7f257c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.705+0000 7f2584209640 1 -- 172.21.15.70:0/2928125486 >> 172.21.15.70:0/2928125486 conn(0x7f257c06a5d0 msgr2=0x7f257c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.705+0000 7f2584209640 1 -- 172.21.15.70:0/2928125486 shutdown_connections 2023-10-08T05:53:29.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.705+0000 7f2584209640 1 -- 172.21.15.70:0/2928125486 wait complete. 2023-10-08T05:53:29.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.707+0000 7f2584209640 1 Processor -- start 2023-10-08T05:53:29.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.707+0000 7f2584209640 1 -- start start 2023-10-08T05:53:29.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f2584209640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c06dfc0 0x7f257c0c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f2584209640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c071070 0x7f257c1f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f2584209640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c0afbe0 0x7f257c1f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f2584209640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0c2f40 con 0x7f257c0afbe0 2023-10-08T05:53:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f2584209640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c15ced0 con 0x7f257c071070 2023-10-08T05:53:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f2584209640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c1f25d0 con 0x7f257c06dfc0 2023-10-08T05:53:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f25827a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c0afbe0 0x7f257c1f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f25827a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c0afbe0 0x7f257c1f2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33864/0 (socket says 172.21.15.70:33864) 2023-10-08T05:53:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f25827a6640 1 -- 172.21.15.70:0/3694670535 learned_addr learned my addr 172.21.15.70:0/3694670535 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f25827a6640 1 -- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c06dfc0 msgr2=0x7f257c0c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.708+0000 7f2581fa5640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c06dfc0 0x7f257c0c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.709+0000 7f25827a6640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c06dfc0 0x7f257c0c4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.709+0000 7f25827a6640 1 -- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c071070 msgr2=0x7f257c1f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.709+0000 7f25827a6640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c071070 0x7f257c1f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.709+0000 7f25827a6640 1 -- 172.21.15.70:0/3694670535 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574009b30 con 0x7f257c0afbe0 2023-10-08T05:53:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.709+0000 7f25827a6640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c0afbe0 0x7f257c1f2090 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2574036040 tx=0x7f2574004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.709+0000 7f257a7fc640 1 -- 172.21.15.70:0/3694670535 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574009510 con 0x7f257c0afbe0 2023-10-08T05:53:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.709+0000 7f257a7fc640 1 -- 172.21.15.70:0/3694670535 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257403e450 con 0x7f257c0afbe0 2023-10-08T05:53:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.709+0000 7f257a7fc640 1 -- 172.21.15.70:0/3694670535 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257403e7e0 con 0x7f257c0afbe0 2023-10-08T05:53:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.709+0000 7f2584209640 1 -- 172.21.15.70:0/3694670535 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c15d0d0 con 0x7f257c0afbe0 2023-10-08T05:53:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.709+0000 7f2584209640 1 -- 172.21.15.70:0/3694670535 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c15d690 con 0x7f257c0afbe0 2023-10-08T05:53:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.710+0000 7f2584209640 1 -- 172.21.15.70:0/3694670535 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257c04e160 con 0x7f257c0afbe0 2023-10-08T05:53:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.711+0000 7f257a7fc640 1 -- 172.21.15.70:0/3694670535 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 44) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f257403e940 con 0x7f257c0afbe0 2023-10-08T05:53:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.712+0000 7f257a7fc640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f256806af00 0x7f256806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.712+0000 7f257a7fc640 1 -- 172.21.15.70:0/3694670535 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6890+0+0 (secure 0 0 0) 0x7f25740ac980 con 0x7f257c0afbe0 2023-10-08T05:53:29.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.713+0000 7f2581fa5640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f256806af00 0x7f256806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:29.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.714+0000 7f2581fa5640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f256806af00 0x7f256806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f256c003d90 tx=0x7f256c009040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:29.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.716+0000 7f257a7fc640 1 -- 172.21.15.70:0/3694670535 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25740774a0 con 0x7f257c0afbe0 2023-10-08T05:53:29.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:29.950+0000 7f2584209640 1 -- 172.21.15.70:0/3694670535 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f257c073110 con 0x7f257c0afbe0 2023-10-08T05:53:30.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.557+0000 7f257a7fc640 1 -- 172.21.15.70:0/3694670535 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f257407aac0 con 0x7f257c0afbe0 2023-10-08T05:53:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.561+0000 7f2584209640 1 -- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f256806af00 msgr2=0x7f256806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.561+0000 7f2584209640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f256806af00 0x7f256806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f256c003d90 tx=0x7f256c009040 comp rx=0 tx=0).stop 2023-10-08T05:53:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.561+0000 7f2584209640 1 -- 172.21.15.70:0/3694670535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c0afbe0 msgr2=0x7f257c1f2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.561+0000 7f2584209640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c0afbe0 0x7f257c1f2090 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2574036040 tx=0x7f2574004a40 comp rx=0 tx=0).stop 2023-10-08T05:53:30.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.562+0000 7f2584209640 1 -- 172.21.15.70:0/3694670535 shutdown_connections 2023-10-08T05:53:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.562+0000 7f2584209640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f256806af00 0x7f256806d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.562+0000 7f2584209640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c0afbe0 0x7f257c1f2090 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.562+0000 7f2584209640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f257c071070 0x7f257c1f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.562+0000 7f2584209640 1 --2- 172.21.15.70:0/3694670535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f257c06dfc0 0x7f257c0c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.562+0000 7f2584209640 1 -- 172.21.15.70:0/3694670535 >> 172.21.15.70:0/3694670535 conn(0x7f257c06a5d0 msgr2=0x7f257c0c8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.562+0000 7f2584209640 1 -- 172.21.15.70:0/3694670535 shutdown_connections 2023-10-08T05:53:30.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.562+0000 7f2584209640 1 -- 172.21.15.70:0/3694670535 wait complete. 2023-10-08T05:53:30.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T05:53:30.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.789+0000 7f0322ae7640 1 Processor -- start 2023-10-08T05:53:30.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.789+0000 7f0322ae7640 1 -- start start 2023-10-08T05:53:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.791+0000 7f0322ae7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f031c0cb620 0x7f031c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.791+0000 7f0322ae7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031c071170 con 0x7f031c160660 2023-10-08T05:53:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.791+0000 7f0322ae7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031c06e150 con 0x7f031c070cc0 2023-10-08T05:53:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.791+0000 7f0322ae7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c06de70 con 0x7f031c0cb620 2023-10-08T05:53:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.791+0000 7f0320883640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f031c160660 0x7f031c0c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36950/0 (socket says 172.21.15.70:36950) 2023-10-08T05:53:30.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.791+0000 7f0320883640 1 -- 172.21.15.70:0/1919888720 learned_addr learned my addr 172.21.15.70:0/1919888720 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:30.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.791+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879675288 0 0) 0x7f0310001110 con 0x7f031c160660 2023-10-08T05:53:30.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.791+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f030c003520 con 0x7f031c160660 2023-10-08T05:53:30.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.791+0000 7f0321084640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f031c0cb620 0x7f031c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:30.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1515427905 0 0) 0x7f030c003520 con 0x7f031c160660 2023-10-08T05:53:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f031c0cb620 msgr2=0x7f031c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f031c0cb620 0x7f031c0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 >> v1:172.21.15.181:6789/0 conn(0x7f031c070cc0 legacy=0x7f031c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c0c8e30 con 0x7f031c160660 2023-10-08T05:53:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0310001800 con 0x7f031c160660 2023-10-08T05:53:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 >> v1:172.21.15.70:6789/0 conn(0x7f031c160660 legacy=0x7f031c0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f030c0039a0 0x7f030c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f031c0cb620 0x7f030c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c004d00 0x7f030c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:30.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c06de70 con 0x7f030c004d00 2023-10-08T05:53:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c00ba60 con 0x7f030c0039a0 2023-10-08T05:53:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030c00bc20 con 0x7f031c0cb620 2023-10-08T05:53:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f0321084640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c004d00 0x7f030c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.792+0000 7f0320883640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f030c0039a0 0x7f030c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.793+0000 7f0321084640 1 -- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f031c0cb620 msgr2=0x7f030c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.793+0000 7f031bfff640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f031c0cb620 0x7f030c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.793+0000 7f0321084640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f031c0cb620 0x7f030c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.793+0000 7f0321084640 1 -- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f030c0039a0 msgr2=0x7f030c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.793+0000 7f0321084640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f030c0039a0 0x7f030c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:30.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.793+0000 7f0321084640 1 -- 172.21.15.70:0/1919888720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c003520 con 0x7f030c004d00 2023-10-08T05:53:30.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.793+0000 7f0320883640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f030c0039a0 0x7f030c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.794+0000 7f0321084640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c004d00 0x7f030c005180 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0314009f90 tx=0x7f030c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e342d067e1f8efaa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.794+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031402cbd0 con 0x7f030c004d00 2023-10-08T05:53:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.794+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031400f040 con 0x7f030c004d00 2023-10-08T05:53:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.794+0000 7f031b7fe640 1 -- 172.21.15.70:0/1919888720 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314034440 con 0x7f030c004d00 2023-10-08T05:53:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.794+0000 7f0322ae7640 1 -- 172.21.15.70:0/1919888720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c004d00 msgr2=0x7f030c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.794+0000 7f0322ae7640 1 --2- 172.21.15.70:0/1919888720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f030c004d00 0x7f030c005180 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0314009f90 tx=0x7f030c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:30.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.794+0000 7f0321084640 1 -- 172.21.15.70:0/1919888720 reap_dead start 2023-10-08T05:53:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.795+0000 7f0322ae7640 1 -- 172.21.15.70:0/1919888720 shutdown_connections 2023-10-08T05:53:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.795+0000 7f0322ae7640 1 -- 172.21.15.70:0/1919888720 >> 172.21.15.70:0/1919888720 conn(0x7f031c06a5d0 msgr2=0x7f031c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.795+0000 7f0322ae7640 1 -- 172.21.15.70:0/1919888720 shutdown_connections 2023-10-08T05:53:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.795+0000 7f0322ae7640 1 -- 172.21.15.70:0/1919888720 wait complete. 2023-10-08T05:53:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.796+0000 7f0322ae7640 1 Processor -- start 2023-10-08T05:53:30.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.797+0000 7f0322ae7640 1 -- start start 2023-10-08T05:53:30.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.797+0000 7f0322ae7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f030c004d00 0x7f031c0ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:30.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.797+0000 7f0322ae7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f031c0c76c0 0x7f031c0c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.797+0000 7f0322ae7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f031c1549a0 0x7f031c15c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.797+0000 7f0322ae7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0c2480 con 0x7f031c1549a0 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.797+0000 7f0322ae7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0c4fe0 con 0x7f031c0c76c0 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.797+0000 7f0322ae7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c0acf10 con 0x7f030c004d00 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0321084640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f031c1549a0 0x7f031c15c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0321084640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f031c1549a0 0x7f031c15c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33880/0 (socket says 172.21.15.70:33880) 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0321084640 1 -- 172.21.15.70:0/1213269596 learned_addr learned my addr 172.21.15.70:0/1213269596 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0320883640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f030c004d00 0x7f031c0ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0321084640 1 -- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f030c004d00 msgr2=0x7f031c0ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0321084640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f030c004d00 0x7f031c0ac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0321084640 1 -- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f031c0c76c0 msgr2=0x7f031c0c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0321084640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f031c0c76c0 0x7f031c0c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0321084640 1 -- 172.21.15.70:0/1213269596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c0c4740 con 0x7f031c1549a0 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0320883640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f030c004d00 0x7f031c0ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.798+0000 7f0321084640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f031c1549a0 0x7f031c15c540 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0314009f90 tx=0x7f0314009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.799+0000 7f03197fa640 1 -- 172.21.15.70:0/1213269596 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314043070 con 0x7f031c1549a0 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.799+0000 7f03197fa640 1 -- 172.21.15.70:0/1213269596 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0314034ad0 con 0x7f031c1549a0 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.799+0000 7f0322ae7640 1 -- 172.21.15.70:0/1213269596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031c0c4a20 con 0x7f031c1549a0 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.799+0000 7f03197fa640 1 -- 172.21.15.70:0/1213269596 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314034e80 con 0x7f031c1549a0 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.799+0000 7f0322ae7640 1 -- 172.21.15.70:0/1213269596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031c0c4310 con 0x7f031c1549a0 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.800+0000 7f0322ae7640 1 -- 172.21.15.70:0/1213269596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031c0c44d0 con 0x7f031c1549a0 2023-10-08T05:53:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.801+0000 7f03197fa640 1 -- 172.21.15.70:0/1213269596 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f031403e440 con 0x7f031c1549a0 2023-10-08T05:53:30.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.802+0000 7f03197fa640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f030006af90 0x7f030006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:30.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.803+0000 7f03197fa640 1 -- 172.21.15.70:0/1213269596 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6890+0+0 (secure 0 0 0) 0x7f03140ab760 con 0x7f031c1549a0 2023-10-08T05:53:30.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.803+0000 7f0320883640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f030006af90 0x7f030006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:30.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.804+0000 7f0320883640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f030006af90 0x7f030006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0310001930 tx=0x7f0310009040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:30.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.807+0000 7f03197fa640 1 -- 172.21.15.70:0/1213269596 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0314076250 con 0x7f031c1549a0 2023-10-08T05:53:30.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:30.996+0000 7f0322ae7640 1 -- 172.21.15.70:0/1213269596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f031c158ef0 con 0x7f031c1549a0 2023-10-08T05:53:31.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.579+0000 7f03197fa640 1 -- 172.21.15.70:0/1213269596 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0314079870 con 0x7f031c1549a0 2023-10-08T05:53:31.581 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T05:53:31.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.583+0000 7f0322ae7640 1 -- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f030006af90 msgr2=0x7f030006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:31.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.583+0000 7f0322ae7640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f030006af90 0x7f030006d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0310001930 tx=0x7f0310009040 comp rx=0 tx=0).stop 2023-10-08T05:53:31.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.583+0000 7f0322ae7640 1 -- 172.21.15.70:0/1213269596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f031c1549a0 msgr2=0x7f031c15c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:31.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.583+0000 7f0322ae7640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f031c1549a0 0x7f031c15c540 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0314009f90 tx=0x7f0314009750 comp rx=0 tx=0).stop 2023-10-08T05:53:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.586+0000 7f0322ae7640 1 -- 172.21.15.70:0/1213269596 shutdown_connections 2023-10-08T05:53:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.586+0000 7f0322ae7640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f030006af90 0x7f030006d450 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0310001930 tx=0x7f0310009040 comp rx=0 tx=0).stop 2023-10-08T05:53:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.586+0000 7f0322ae7640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f031c1549a0 0x7f031c15c540 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.586+0000 7f0322ae7640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f031c0c76c0 0x7f031c0c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.586+0000 7f0322ae7640 1 --2- 172.21.15.70:0/1213269596 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f030c004d00 0x7f031c0ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.586+0000 7f0322ae7640 1 -- 172.21.15.70:0/1213269596 >> 172.21.15.70:0/1213269596 conn(0x7f031c06a5d0 msgr2=0x7f031c1587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.588+0000 7f0322ae7640 1 -- 172.21.15.70:0/1213269596 shutdown_connections 2023-10-08T05:53:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.589+0000 7f0322ae7640 1 -- 172.21.15.70:0/1213269596 wait complete. 2023-10-08T05:53:31.613 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T05:53:31.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.825+0000 7febc131d640 1 Processor -- start 2023-10-08T05:53:31.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.826+0000 7febc131d640 1 -- start start 2023-10-08T05:53:31.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.827+0000 7febc131d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7febbc0b05b0 0x7febbc071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:31.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.827+0000 7febc131d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc15d850 con 0x7febbc159e00 2023-10-08T05:53:31.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.827+0000 7febc131d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc15db30 con 0x7febbc0b0990 2023-10-08T05:53:31.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.827+0000 7febc131d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc070a50 con 0x7febbc0b05b0 2023-10-08T05:53:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febbb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7febbc159e00 0x7febbc0c98b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36966/0 (socket says 172.21.15.70:36966) 2023-10-08T05:53:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febbb7fe640 1 -- 172.21.15.70:0/125644131 learned_addr learned my addr 172.21.15.70:0/125644131 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febbaffd640 1 --2- 172.21.15.70:0/125644131 >> v2:172.21.15.181:3300/0 conn(0x7febbc0b05b0 0x7febbc071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22532701 0 0) 0x7febbc15d850 con 0x7febbc159e00 2023-10-08T05:53:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feba8003520 con 0x7febbc159e00 2023-10-08T05:53:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2229987252 0 0) 0x7feba8003520 con 0x7febbc159e00 2023-10-08T05:53:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 >> v2:172.21.15.181:3300/0 conn(0x7febbc0b05b0 msgr2=0x7febbc071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febba7fc640 1 --2- 172.21.15.70:0/125644131 >> v2:172.21.15.181:3300/0 conn(0x7febbc0b05b0 0x7febbc071460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 >> v1:172.21.15.181:6789/0 conn(0x7febbc0b0990 legacy=0x7febbc15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc070c60 con 0x7febbc159e00 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.828+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7febb40019f0 con 0x7febbc159e00 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 >> v1:172.21.15.70:6789/0 conn(0x7febbc159e00 legacy=0x7febbc0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febba7fc640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba80039c0 0x7feba8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febba7fc640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feba80047c0 0x7feba8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febba7fc640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80051d0 0x7feba800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc070a50 con 0x7feba80039c0 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba800e330 con 0x7feba80051d0 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba800e4f0 con 0x7feba80047c0 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febbaffd640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba80039c0 0x7feba8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febbb7fe640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80051d0 0x7feba800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febb3fff640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feba80047c0 0x7feba8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febbaffd640 1 -- 172.21.15.70:0/125644131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feba80047c0 msgr2=0x7feba8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.829+0000 7febbaffd640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feba80047c0 0x7feba8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.830+0000 7febbaffd640 1 -- 172.21.15.70:0/125644131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80051d0 msgr2=0x7feba800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.830+0000 7febbaffd640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80051d0 0x7feba800db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.830+0000 7febbaffd640 1 -- 172.21.15.70:0/125644131 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8003520 con 0x7feba80039c0 2023-10-08T05:53:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.830+0000 7febbb7fe640 1 -- 172.21.15.70:0/125644131 reap_dead start 2023-10-08T05:53:31.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.831+0000 7febbaffd640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba80039c0 0x7feba8003de0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7febac00ba30 tx=0x7feba80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9457a4d8bab3156f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:31.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.831+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac032640 con 0x7feba80039c0 2023-10-08T05:53:31.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.831+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac0327a0 con 0x7feba80039c0 2023-10-08T05:53:31.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.831+0000 7febba7fc640 1 -- 172.21.15.70:0/125644131 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac032b30 con 0x7feba80039c0 2023-10-08T05:53:31.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.832+0000 7febc131d640 1 -- 172.21.15.70:0/125644131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba80039c0 msgr2=0x7feba8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:31.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.832+0000 7febc131d640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba80039c0 0x7feba8003de0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7febac00ba30 tx=0x7feba80057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:31.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.832+0000 7febc131d640 1 -- 172.21.15.70:0/125644131 shutdown_connections 2023-10-08T05:53:31.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.832+0000 7febc131d640 1 --2- 172.21.15.70:0/125644131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feba80039c0 0x7feba8003de0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:31.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.832+0000 7febc131d640 1 -- 172.21.15.70:0/125644131 >> 172.21.15.70:0/125644131 conn(0x7febbc069d50 msgr2=0x7febbc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:31.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.832+0000 7febc131d640 1 -- 172.21.15.70:0/125644131 shutdown_connections 2023-10-08T05:53:31.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.832+0000 7febc131d640 1 -- 172.21.15.70:0/125644131 wait complete. 2023-10-08T05:53:31.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.834+0000 7febc131d640 1 Processor -- start 2023-10-08T05:53:31.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.834+0000 7febc131d640 1 -- start start 2023-10-08T05:53:31.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.835+0000 7febc131d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80039c0 0x7febbc150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.835+0000 7febc131d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febbc0b1260 0x7febbc0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.835+0000 7febc131d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febbc0b3410 0x7febbc158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.835+0000 7febc131d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0b2790 con 0x7febbc0b1260 2023-10-08T05:53:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.835+0000 7febc131d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0b2300 con 0x7feba80039c0 2023-10-08T05:53:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.835+0000 7febc131d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0b16f0 con 0x7febbc0b3410 2023-10-08T05:53:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.835+0000 7febb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febbc0b1260 0x7febbc0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.835+0000 7febb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febbc0b1260 0x7febbc0b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33904/0 (socket says 172.21.15.70:33904) 2023-10-08T05:53:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.835+0000 7febb3fff640 1 -- 172.21.15.70:0/1664337533 learned_addr learned my addr 172.21.15.70:0/1664337533 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febb3fff640 1 -- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febbc0b3410 msgr2=0x7febbc158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febb3fff640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febbc0b3410 0x7febbc158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febb3fff640 1 -- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80039c0 msgr2=0x7febbc150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febbaffd640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80039c0 0x7febbc150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febb3fff640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80039c0 0x7febbc150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febb3fff640 1 -- 172.21.15.70:0/1664337533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc0b19d0 con 0x7febbc0b1260 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febb3fff640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febbc0b1260 0x7febbc0b2ed0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7feba400d400 tx=0x7feba400d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febb37fe640 1 -- 172.21.15.70:0/1664337533 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba400b070 con 0x7febbc0b1260 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febb37fe640 1 -- 172.21.15.70:0/1664337533 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba40054e0 con 0x7febbc0b1260 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febb37fe640 1 -- 172.21.15.70:0/1664337533 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4003830 con 0x7febbc0b1260 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febc131d640 1 -- 172.21.15.70:0/1664337533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc0b1cb0 con 0x7febbc0b1260 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.836+0000 7febc131d640 1 -- 172.21.15.70:0/1664337533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc1587c0 con 0x7febbc0b1260 2023-10-08T05:53:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.837+0000 7febc131d640 1 -- 172.21.15.70:0/1664337533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc04e160 con 0x7febbc0b1260 2023-10-08T05:53:31.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.839+0000 7febb37fe640 1 -- 172.21.15.70:0/1664337533 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feba4003990 con 0x7febbc0b1260 2023-10-08T05:53:31.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.839+0000 7febb37fe640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feb9c06af90 0x7feb9c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:31.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.840+0000 7febb37fe640 1 -- 172.21.15.70:0/1664337533 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6452+0+0 (secure 0 0 0) 0x7feba40893b0 con 0x7febbc0b1260 2023-10-08T05:53:31.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.840+0000 7febbaffd640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feb9c06af90 0x7feb9c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:31.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.841+0000 7febbaffd640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feb9c06af90 0x7feb9c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7febac00ba00 tx=0x7febac03d040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:31.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:31.844+0000 7febb37fe640 1 -- 172.21.15.70:0/1664337533 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feba4053ff0 con 0x7febbc0b1260 2023-10-08T05:53:32.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.037+0000 7febc131d640 1 -- 172.21.15.70:0/1664337533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7febbc070320 con 0x7febbc0b1260 2023-10-08T05:53:32.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.592+0000 7febb37fe640 1 -- 172.21.15.70:0/1664337533 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) v1 ==== 169+0+0 (secure 0 0 0) 0x7feba40180d0 con 0x7febbc0b1260 2023-10-08T05:53:32.594 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T05:53:32.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.597+0000 7febc131d640 1 -- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feb9c06af90 msgr2=0x7feb9c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:32.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.597+0000 7febc131d640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feb9c06af90 0x7feb9c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7febac00ba00 tx=0x7febac03d040 comp rx=0 tx=0).stop 2023-10-08T05:53:32.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.597+0000 7febc131d640 1 -- 172.21.15.70:0/1664337533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febbc0b1260 msgr2=0x7febbc0b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:32.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.597+0000 7febc131d640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febbc0b1260 0x7febbc0b2ed0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7feba400d400 tx=0x7feba400d8c0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.599+0000 7febc131d640 1 -- 172.21.15.70:0/1664337533 shutdown_connections 2023-10-08T05:53:32.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.599+0000 7febc131d640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feb9c06af90 0x7feb9c06d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.599+0000 7febc131d640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7febbc0b3410 0x7febbc158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.599+0000 7febc131d640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febbc0b1260 0x7febbc0b2ed0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.599+0000 7febc131d640 1 --2- 172.21.15.70:0/1664337533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feba80039c0 0x7febbc150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.599+0000 7febc131d640 1 -- 172.21.15.70:0/1664337533 >> 172.21.15.70:0/1664337533 conn(0x7febbc069d50 msgr2=0x7febbc0cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:32.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.601+0000 7febc131d640 1 -- 172.21.15.70:0/1664337533 shutdown_connections 2023-10-08T05:53:32.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.601+0000 7febc131d640 1 -- 172.21.15.70:0/1664337533 wait complete. 2023-10-08T05:53:32.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T05:53:32.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.828+0000 7fdda194e640 1 Processor -- start 2023-10-08T05:53:32.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.829+0000 7fdda194e640 1 -- start start 2023-10-08T05:53:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.830+0000 7fdda194e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd9c0ca370 0x7fdd9c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.830+0000 7fdda194e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9c06c3a0 con 0x7fdd9c0702b0 2023-10-08T05:53:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.830+0000 7fdda194e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9c06c640 con 0x7fdd9c0b21f0 2023-10-08T05:53:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.830+0000 7fdda194e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9c06c920 con 0x7fdd9c0ca370 2023-10-08T05:53:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.831+0000 7fdd9affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdd9c0702b0 0x7fdd9c070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36976/0 (socket says 172.21.15.70:36976) 2023-10-08T05:53:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.831+0000 7fdd9affd640 1 -- 172.21.15.70:0/651353236 learned_addr learned my addr 172.21.15.70:0/651353236 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:32.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.831+0000 7fdd9b7fe640 1 --2- 172.21.15.70:0/651353236 >> v2:172.21.15.181:3300/0 conn(0x7fdd9c0ca370 0x7fdd9c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:32.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.831+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638566195 0 0) 0x7fdd9c06c3a0 con 0x7fdd9c0702b0 2023-10-08T05:53:32.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.831+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd88003520 con 0x7fdd9c0702b0 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.831+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3723263757 0 0) 0x7fdd88003520 con 0x7fdd9c0702b0 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.831+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 >> v2:172.21.15.181:3300/0 conn(0x7fdd9c0ca370 msgr2=0x7fdd9c06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.831+0000 7fdd9a7fc640 1 --2- 172.21.15.70:0/651353236 >> v2:172.21.15.181:3300/0 conn(0x7fdd9c0ca370 0x7fdd9c06d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.831+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 >> v1:172.21.15.181:6789/0 conn(0x7fdd9c0b21f0 legacy=0x7fdd9c0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.831+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c06cc00 con 0x7fdd9c0702b0 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fdd8c001670 con 0x7fdd9c0702b0 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 >> v1:172.21.15.70:6789/0 conn(0x7fdd9c0702b0 legacy=0x7fdd9c070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9a7fc640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd880039c0 0x7fdd88003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9a7fc640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd880047c0 0x7fdd88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9a7fc640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd880052a0 0x7fdd8800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c06c920 con 0x7fdd880047c0 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8800e3d0 con 0x7fdd880039c0 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8800e590 con 0x7fdd880052a0 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd93fff640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd880047c0 0x7fdd88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9affd640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd880039c0 0x7fdd88003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd93fff640 1 -- 172.21.15.70:0/651353236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd880052a0 msgr2=0x7fdd8800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd93fff640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd880052a0 0x7fdd8800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd93fff640 1 -- 172.21.15.70:0/651353236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd880039c0 msgr2=0x7fdd88003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd93fff640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd880039c0 0x7fdd88003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd93fff640 1 -- 172.21.15.70:0/651353236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd88003520 con 0x7fdd880047c0 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9affd640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd880039c0 0x7fdd88003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.832+0000 7fdd9b7fe640 1 -- 172.21.15.70:0/651353236 reap_dead start 2023-10-08T05:53:32.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.834+0000 7fdd93fff640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd880047c0 0x7fdd88004c40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdd84001750 tx=0x7fdd880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3b3a6c6e062f83 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.834+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd840052d0 con 0x7fdd880047c0 2023-10-08T05:53:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.834+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8402e050 con 0x7fdd880047c0 2023-10-08T05:53:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.834+0000 7fdd9a7fc640 1 -- 172.21.15.70:0/651353236 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84003830 con 0x7fdd880047c0 2023-10-08T05:53:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.834+0000 7fdda194e640 1 -- 172.21.15.70:0/651353236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd880047c0 msgr2=0x7fdd88004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.834+0000 7fdda194e640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd880047c0 0x7fdd88004c40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fdd84001750 tx=0x7fdd880057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.835+0000 7fdda194e640 1 -- 172.21.15.70:0/651353236 shutdown_connections 2023-10-08T05:53:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.835+0000 7fdda194e640 1 --2- 172.21.15.70:0/651353236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd880047c0 0x7fdd88004c40 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.835+0000 7fdda194e640 1 -- 172.21.15.70:0/651353236 >> 172.21.15.70:0/651353236 conn(0x7fdd9c06a5d0 msgr2=0x7fdd9c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.835+0000 7fdda194e640 1 -- 172.21.15.70:0/651353236 shutdown_connections 2023-10-08T05:53:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.835+0000 7fdda194e640 1 -- 172.21.15.70:0/651353236 wait complete. 2023-10-08T05:53:32.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.836+0000 7fdda194e640 1 Processor -- start 2023-10-08T05:53:32.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.837+0000 7fdda194e640 1 -- start start 2023-10-08T05:53:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.837+0000 7fdda194e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd880047c0 0x7fdd9c0c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.837+0000 7fdda194e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd9c157650 0x7fdd9c155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.837+0000 7fdda194e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd9c157ac0 0x7fdd9c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.837+0000 7fdda194e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c1563d0 con 0x7fdd9c157ac0 2023-10-08T05:53:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.837+0000 7fdda194e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c15c150 con 0x7fdd9c157650 2023-10-08T05:53:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.837+0000 7fdda194e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c0ac380 con 0x7fdd880047c0 2023-10-08T05:53:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd9c157ac0 0x7fdd9c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd9c157ac0 0x7fdd9c155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33914/0 (socket says 172.21.15.70:33914) 2023-10-08T05:53:32.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd9b7fe640 1 -- 172.21.15.70:0/2136643579 learned_addr learned my addr 172.21.15.70:0/2136643579 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:32.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd9affd640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd880047c0 0x7fdd9c0c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:32.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd93fff640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd9c157650 0x7fdd9c155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:32.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd9b7fe640 1 -- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd880047c0 msgr2=0x7fdd9c0c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:32.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd9b7fe640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd880047c0 0x7fdd9c0c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd9b7fe640 1 -- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd9c157650 msgr2=0x7fdd9c155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:32.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd9b7fe640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd9c157650 0x7fdd9c155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:32.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd9b7fe640 1 -- 172.21.15.70:0/2136643579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c0ac660 con 0x7fdd9c157ac0 2023-10-08T05:53:32.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.838+0000 7fdd9affd640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd880047c0 0x7fdd9c0c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:32.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.839+0000 7fdd9b7fe640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd9c157ac0 0x7fdd9c155e90 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0ca750 tx=0x7fdd9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:32.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.839+0000 7fdd98ff9640 1 -- 172.21.15.70:0/2136643579 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400ed90 con 0x7fdd9c157ac0 2023-10-08T05:53:32.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.839+0000 7fdd98ff9640 1 -- 172.21.15.70:0/2136643579 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd94014420 con 0x7fdd9c157ac0 2023-10-08T05:53:32.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.839+0000 7fdd98ff9640 1 -- 172.21.15.70:0/2136643579 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94014810 con 0x7fdd9c157ac0 2023-10-08T05:53:32.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.839+0000 7fdda194e640 1 -- 172.21.15.70:0/2136643579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9c0ac940 con 0x7fdd9c157ac0 2023-10-08T05:53:32.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.839+0000 7fdda194e640 1 -- 172.21.15.70:0/2136643579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c15bd70 con 0x7fdd9c157ac0 2023-10-08T05:53:32.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.840+0000 7fdda194e640 1 -- 172.21.15.70:0/2136643579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9c15bf00 con 0x7fdd9c157ac0 2023-10-08T05:53:32.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.841+0000 7fdd98ff9640 1 -- 172.21.15.70:0/2136643579 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdd94014970 con 0x7fdd9c157ac0 2023-10-08T05:53:32.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.842+0000 7fdd98ff9640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fdd7c06af90 0x7fdd7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:32.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.842+0000 7fdd98ff9640 1 -- 172.21.15.70:0/2136643579 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6038+0+0 (secure 0 0 0) 0x7fdd94089ed0 con 0x7fdd9c157ac0 2023-10-08T05:53:32.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.843+0000 7fdd9affd640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fdd7c06af90 0x7fdd7c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:32.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.844+0000 7fdd9affd640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fdd7c06af90 0x7fdd7c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c000c00 tx=0x7fdd8c009040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:32.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:32.847+0000 7fdd98ff9640 1 -- 172.21.15.70:0/2136643579 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd94054c80 con 0x7fdd9c157ac0 2023-10-08T05:53:33.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.053+0000 7fdda194e640 1 -- 172.21.15.70:0/2136643579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdd9c04e160 con 0x7fdd9c157ac0 2023-10-08T05:53:33.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.058+0000 7fdd98ff9640 1 -- 172.21.15.70:0/2136643579 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd940582a0 con 0x7fdd9c157ac0 2023-10-08T05:53:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 -- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fdd7c06af90 msgr2=0x7fdd7c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fdd7c06af90 0x7fdd7c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c000c00 tx=0x7fdd8c009040 comp rx=0 tx=0).stop 2023-10-08T05:53:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 -- 172.21.15.70:0/2136643579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd9c157ac0 msgr2=0x7fdd9c155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd9c157ac0 0x7fdd9c155e90 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c0ca750 tx=0x7fdd9400e4b0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 -- 172.21.15.70:0/2136643579 shutdown_connections 2023-10-08T05:53:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fdd7c06af90 0x7fdd7c06d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd9c157ac0 0x7fdd9c155e90 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd9c157650 0x7fdd9c155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 --2- 172.21.15.70:0/2136643579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd880047c0 0x7fdd9c0c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 -- 172.21.15.70:0/2136643579 >> 172.21.15.70:0/2136643579 conn(0x7fdd9c06a5d0 msgr2=0x7fdd9c0c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 -- 172.21.15.70:0/2136643579 shutdown_connections 2023-10-08T05:53:33.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.061+0000 7fdda194e640 1 -- 172.21.15.70:0/2136643579 wait complete. 2023-10-08T05:53:33.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T05:53:33.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.292+0000 7f7e0fcd4640 1 Processor -- start 2023-10-08T05:53:33.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.292+0000 7f7e0fcd4640 1 -- start start 2023-10-08T05:53:33.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.294+0000 7f7e0fcd4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0815aad0 0x7f7e080b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.294+0000 7f7e0fcd4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0815bf30 con 0x7f7e0806d410 2023-10-08T05:53:33.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.294+0000 7f7e0fcd4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0815b320 con 0x7f7e080c82b0 2023-10-08T05:53:33.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.294+0000 7f7e0fcd4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0815b600 con 0x7f7e0815aad0 2023-10-08T05:53:33.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.294+0000 7f7e0e271640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7e0806d410 0x7f7e08070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36986/0 (socket says 172.21.15.70:36986) 2023-10-08T05:53:33.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.294+0000 7f7e0e271640 1 -- 172.21.15.70:0/3703324213 learned_addr learned my addr 172.21.15.70:0/3703324213 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.294+0000 7f7e0da70640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0815aad0 0x7f7e080b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396459821 0 0) 0x7f7e0815bf30 con 0x7f7e0806d410 2023-10-08T05:53:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df4003520 con 0x7f7e0806d410 2023-10-08T05:53:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3807266742 0 0) 0x7f7df4003520 con 0x7f7e0806d410 2023-10-08T05:53:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0815aad0 msgr2=0x7f7e080b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0815aad0 0x7f7e080b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 >> v1:172.21.15.181:6789/0 conn(0x7f7e080c82b0 legacy=0x7f7e0806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0815b8e0 con 0x7f7e0806d410 2023-10-08T05:53:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7e000019f0 con 0x7f7e0806d410 2023-10-08T05:53:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 >> v1:172.21.15.70:6789/0 conn(0x7f7e0806d410 legacy=0x7f7e08070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0815aad0 0x7f7df40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df4003f70 0x7f7df40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4004d00 0x7f7df4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0815b600 con 0x7f7df4003f70 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df400bac0 con 0x7f7df4004d00 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.295+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df400bc80 con 0x7f7e0815aad0 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.296+0000 7f7e0d26f640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df4003f70 0x7f7df40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.296+0000 7f7e0da70640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0815aad0 0x7f7df40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.296+0000 7f7e0e271640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4004d00 0x7f7df4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.296+0000 7f7e0d26f640 1 -- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0815aad0 msgr2=0x7f7df40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:33.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.296+0000 7f7e0d26f640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0815aad0 0x7f7df40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.296+0000 7f7e0d26f640 1 -- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4004d00 msgr2=0x7f7df4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:33.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.296+0000 7f7e0d26f640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4004d00 0x7f7df4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.296+0000 7f7e0d26f640 1 -- 172.21.15.70:0/3703324213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df4003520 con 0x7f7df4003f70 2023-10-08T05:53:33.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.296+0000 7f7e0e271640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4004d00 0x7f7df4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:33.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.297+0000 7f7e0d26f640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df4003f70 0x7f7df40047c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f7df8001750 tx=0x7f7df40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce9a3e74a69eb43 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:33.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.297+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df80052d0 con 0x7f7df4003f70 2023-10-08T05:53:33.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.297+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df802e050 con 0x7f7df4003f70 2023-10-08T05:53:33.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.297+0000 7f7e0ca6e640 1 -- 172.21.15.70:0/3703324213 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8003830 con 0x7f7df4003f70 2023-10-08T05:53:33.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.298+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/3703324213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df4003f70 msgr2=0x7f7df40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.298+0000 7f7e0fcd4640 1 --2- 172.21.15.70:0/3703324213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7df4003f70 0x7f7df40047c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f7df8001750 tx=0x7f7df40057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.298+0000 7f7e0e271640 1 -- 172.21.15.70:0/3703324213 reap_dead start 2023-10-08T05:53:33.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.298+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/3703324213 shutdown_connections 2023-10-08T05:53:33.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.298+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/3703324213 >> 172.21.15.70:0/3703324213 conn(0x7f7e08069d50 msgr2=0x7f7e0806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:33.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.298+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/3703324213 shutdown_connections 2023-10-08T05:53:33.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.298+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/3703324213 wait complete. 2023-10-08T05:53:33.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.300+0000 7f7e0fcd4640 1 Processor -- start 2023-10-08T05:53:33.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.300+0000 7f7e0fcd4640 1 -- start start 2023-10-08T05:53:33.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.301+0000 7f7e0fcd4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4003f70 0x7f7e081f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.301+0000 7f7e0fcd4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0814f880 0x7f7e0814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.301+0000 7f7e0fcd4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e08141e40 0x7f7e081422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.301+0000 7f7e0fcd4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e081f2060 con 0x7f7e08141e40 2023-10-08T05:53:33.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.301+0000 7f7e0fcd4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08150f70 con 0x7f7df4003f70 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.301+0000 7f7e0fcd4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08150360 con 0x7f7e0814f880 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0e271640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e08141e40 0x7f7e081422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0e271640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e08141e40 0x7f7e081422f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33928/0 (socket says 172.21.15.70:33928) 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0e271640 1 -- 172.21.15.70:0/1814353776 learned_addr learned my addr 172.21.15.70:0/1814353776 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0d26f640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0814f880 0x7f7e0814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0da70640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4003f70 0x7f7e081f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0e271640 1 -- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0814f880 msgr2=0x7f7e0814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0e271640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0814f880 0x7f7e0814fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0e271640 1 -- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4003f70 msgr2=0x7f7e081f1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0e271640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4003f70 0x7f7e081f1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0e271640 1 -- 172.21.15.70:0/1814353776 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e08150640 con 0x7f7e08141e40 2023-10-08T05:53:33.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0da70640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4003f70 0x7f7e081f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:33.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.302+0000 7f7e0e271640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e08141e40 0x7f7e081422f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f7e0815f610 tx=0x7f7e0000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:33.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.303+0000 7f7dfe7fc640 1 -- 172.21.15.70:0/1814353776 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000bb20 con 0x7f7e08141e40 2023-10-08T05:53:33.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.303+0000 7f7dfe7fc640 1 -- 172.21.15.70:0/1814353776 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0000bc80 con 0x7f7e08141e40 2023-10-08T05:53:33.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.303+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/1814353776 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e08150920 con 0x7f7e08141e40 2023-10-08T05:53:33.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.303+0000 7f7dfe7fc640 1 -- 172.21.15.70:0/1814353776 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00007070 con 0x7f7e08141e40 2023-10-08T05:53:33.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.303+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/1814353776 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e08151400 con 0x7f7e08141e40 2023-10-08T05:53:33.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.304+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/1814353776 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e081515c0 con 0x7f7e08141e40 2023-10-08T05:53:33.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.305+0000 7f7dfe7fc640 1 -- 172.21.15.70:0/1814353776 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7e000071d0 con 0x7f7e08141e40 2023-10-08T05:53:33.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.306+0000 7f7dfe7fc640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7dec06afc0 0x7f7dec06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.306+0000 7f7dfe7fc640 1 -- 172.21.15.70:0/1814353776 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f7e00089e80 con 0x7f7e08141e40 2023-10-08T05:53:33.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.307+0000 7f7e0da70640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7dec06afc0 0x7f7dec06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.308+0000 7f7e0da70640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7dec06afc0 0x7f7dec06d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009a00 tx=0x7f7e04008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:33.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.311+0000 7f7dfe7fc640 1 -- 172.21.15.70:0/1814353776 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e00054be0 con 0x7f7e08141e40 2023-10-08T05:53:33.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.501+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/1814353776 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f7e0804e160 con 0x7f7e08141e40 2023-10-08T05:53:33.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.507+0000 7f7dfe7fc640 1 -- 172.21.15.70:0/1814353776 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v32) v1 ==== 115+0+0 (secure 0 0 0) 0x7f7e00058200 con 0x7f7e08141e40 2023-10-08T05:53:33.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7dec06afc0 msgr2=0x7f7dec06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7dec06afc0 0x7f7dec06d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009a00 tx=0x7f7e04008040 comp rx=0 tx=0).stop 2023-10-08T05:53:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/1814353776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e08141e40 msgr2=0x7f7e081422f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e08141e40 0x7f7e081422f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f7e0815f610 tx=0x7f7e0000b4a0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/1814353776 shutdown_connections 2023-10-08T05:53:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7dec06afc0 0x7f7dec06d480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e08141e40 0x7f7e081422f0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e0814f880 0x7f7e0814fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 --2- 172.21.15.70:0/1814353776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7df4003f70 0x7f7e081f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/1814353776 >> 172.21.15.70:0/1814353776 conn(0x7f7e08069d50 msgr2=0x7f7e080c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:33.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/1814353776 shutdown_connections 2023-10-08T05:53:33.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.510+0000 7f7e0fcd4640 1 -- 172.21.15.70:0/1814353776 wait complete. 2023-10-08T05:53:33.529 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T05:53:33.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.742+0000 7f56cd7e2640 1 Processor -- start 2023-10-08T05:53:33.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.743+0000 7f56cd7e2640 1 -- start start 2023-10-08T05:53:33.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.744+0000 7f56cd7e2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f56c80afbe0 0x7f56c80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.744+0000 7f56cd7e2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80cb130 0x7f56c80cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.744+0000 7f56cd7e2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c80cbad0 con 0x7f56c806dfc0 2023-10-08T05:53:33.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.744+0000 7f56cd7e2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c80b2620 con 0x7f56c80afbe0 2023-10-08T05:53:33.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.744+0000 7f56cd7e2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80b2060 con 0x7f56c80cb130 2023-10-08T05:53:33.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.745+0000 7f56be7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f56c806dfc0 0x7f56c8070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60320/0 (socket says 172.21.15.70:60320) 2023-10-08T05:53:33.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.745+0000 7f56be7fc640 1 -- 172.21.15.70:0/3681248700 learned_addr learned my addr 172.21.15.70:0/3681248700 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.745+0000 7f56c6ffd640 1 --2- 172.21.15.70:0/3681248700 >> v2:172.21.15.181:3300/0 conn(0x7f56c80afbe0 0x7f56c80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.745+0000 7f56c77fe640 1 --2- 172.21.15.70:0/3681248700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80cb130 0x7f56c80cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.745+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045912220 0 0) 0x7f56c80cbad0 con 0x7f56c806dfc0 2023-10-08T05:53:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.745+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56b4003520 con 0x7f56c806dfc0 2023-10-08T05:53:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.745+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2275554723 0 0) 0x7f56b4003520 con 0x7f56c806dfc0 2023-10-08T05:53:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80cb130 msgr2=0x7f56c80cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 --2- 172.21.15.70:0/3681248700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80cb130 0x7f56c80cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 >> v2:172.21.15.181:3300/0 conn(0x7f56c80afbe0 msgr2=0x7f56c80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 --2- 172.21.15.70:0/3681248700 >> v2:172.21.15.181:3300/0 conn(0x7f56c80afbe0 0x7f56c80b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80b2340 con 0x7f56c806dfc0 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f56b00019f0 con 0x7f56c806dfc0 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c6ffd640 1 --2- 172.21.15.70:0/3681248700 >> v2:172.21.15.181:3300/0 conn(0x7f56c80afbe0 0x7f56c80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 >> v1:172.21.15.70:6789/0 conn(0x7f56c806dfc0 legacy=0x7f56c8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 --2- 172.21.15.70:0/3681248700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b4003970 0x7f56b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 --2- 172.21.15.70:0/3681248700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b40047c0 0x7f56b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 --2- 172.21.15.70:0/3681248700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80cb130 0x7f56b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80b2620 con 0x7f56b4003970 2023-10-08T05:53:33.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80b2060 con 0x7f56b40047c0 2023-10-08T05:53:33.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b400ba60 con 0x7f56c80cb130 2023-10-08T05:53:33.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c6ffd640 1 --2- 172.21.15.70:0/3681248700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b4003970 0x7f56b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c6ffd640 1 -- 172.21.15.70:0/3681248700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80cb130 msgr2=0x7f56b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:33.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c6ffd640 1 --2- 172.21.15.70:0/3681248700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80cb130 0x7f56b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c6ffd640 1 -- 172.21.15.70:0/3681248700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b40047c0 msgr2=0x7f56b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:33.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c6ffd640 1 --2- 172.21.15.70:0/3681248700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b40047c0 0x7f56b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.746+0000 7f56c6ffd640 1 -- 172.21.15.70:0/3681248700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4003520 con 0x7f56b4003970 2023-10-08T05:53:33.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.748+0000 7f56c6ffd640 1 --2- 172.21.15.70:0/3681248700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b4003970 0x7f56b4003d90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006fd0 tx=0x7f56b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7916f81bc23da1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:33.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.748+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8004230 con 0x7f56b4003970 2023-10-08T05:53:33.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.748+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b8004390 con 0x7f56b4003970 2023-10-08T05:53:33.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.748+0000 7f56c67fc640 1 -- 172.21.15.70:0/3681248700 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8003830 con 0x7f56b4003970 2023-10-08T05:53:33.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.748+0000 7f56cd7e2640 1 -- 172.21.15.70:0/3681248700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b4003970 msgr2=0x7f56b4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.748+0000 7f56cd7e2640 1 --2- 172.21.15.70:0/3681248700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56b4003970 0x7f56b4003d90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006fd0 tx=0x7f56b40057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.748+0000 7f56c77fe640 1 -- 172.21.15.70:0/3681248700 reap_dead start 2023-10-08T05:53:33.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.748+0000 7f56cd7e2640 1 -- 172.21.15.70:0/3681248700 shutdown_connections 2023-10-08T05:53:33.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.748+0000 7f56cd7e2640 1 -- 172.21.15.70:0/3681248700 >> 172.21.15.70:0/3681248700 conn(0x7f56c806a5d0 msgr2=0x7f56c80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:33.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.749+0000 7f56cd7e2640 1 -- 172.21.15.70:0/3681248700 shutdown_connections 2023-10-08T05:53:33.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.749+0000 7f56cd7e2640 1 -- 172.21.15.70:0/3681248700 wait complete. 2023-10-08T05:53:33.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.750+0000 7f56cd7e2640 1 Processor -- start 2023-10-08T05:53:33.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.750+0000 7f56cd7e2640 1 -- start start 2023-10-08T05:53:33.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.751+0000 7f56cd7e2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b4003970 0x7f56c814f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.751+0000 7f56cd7e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56c80c91d0 0x7f56c815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.751+0000 7f56cd7e2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80c9650 0x7f56c806cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.751+0000 7f56cd7e2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80c2b20 con 0x7f56c80c91d0 2023-10-08T05:53:33.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.751+0000 7f56cd7e2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80ca9e0 con 0x7f56b4003970 2023-10-08T05:53:33.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.751+0000 7f56cd7e2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c80cacc0 con 0x7f56c80c9650 2023-10-08T05:53:33.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.752+0000 7f56be7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56c80c91d0 0x7f56c815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.752+0000 7f56be7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56c80c91d0 0x7f56c815b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58984/0 (socket says 172.21.15.70:58984) 2023-10-08T05:53:33.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.752+0000 7f56be7fc640 1 -- 172.21.15.70:0/2144701889 learned_addr learned my addr 172.21.15.70:0/2144701889 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:33.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.752+0000 7f56be7fc640 1 -- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80c9650 msgr2=0x7f56c806cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.752+0000 7f56be7fc640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80c9650 0x7f56c806cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.752+0000 7f56be7fc640 1 -- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b4003970 msgr2=0x7f56c814f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.752+0000 7f56be7fc640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b4003970 0x7f56c814f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.752+0000 7f56be7fc640 1 -- 172.21.15.70:0/2144701889 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80c2460 con 0x7f56c80c91d0 2023-10-08T05:53:33.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.752+0000 7f56be7fc640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56c80c91d0 0x7f56c815b510 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f56b0002230 tx=0x7f56b000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:33.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.753+0000 7f56bffff640 1 -- 172.21.15.70:0/2144701889 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b000a7c0 con 0x7f56c80c91d0 2023-10-08T05:53:33.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.753+0000 7f56bffff640 1 -- 172.21.15.70:0/2144701889 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b000a940 con 0x7f56c80c91d0 2023-10-08T05:53:33.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.753+0000 7f56cd7e2640 1 -- 172.21.15.70:0/2144701889 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c80c2740 con 0x7f56c80c91d0 2023-10-08T05:53:33.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.753+0000 7f56bffff640 1 -- 172.21.15.70:0/2144701889 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b000acf0 con 0x7f56c80c91d0 2023-10-08T05:53:33.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.753+0000 7f56cd7e2640 1 -- 172.21.15.70:0/2144701889 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c815ba50 con 0x7f56c80c91d0 2023-10-08T05:53:33.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.754+0000 7f56cd7e2640 1 -- 172.21.15.70:0/2144701889 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c80b4750 con 0x7f56c80c91d0 2023-10-08T05:53:33.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.755+0000 7f56bffff640 1 -- 172.21.15.70:0/2144701889 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f56b00070d0 con 0x7f56c80c91d0 2023-10-08T05:53:33.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.756+0000 7f56bffff640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f56a806af90 0x7f56a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:33.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.756+0000 7f56bffff640 1 -- 172.21.15.70:0/2144701889 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f56b0089340 con 0x7f56c80c91d0 2023-10-08T05:53:33.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.757+0000 7f56c6ffd640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f56a806af90 0x7f56a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:33.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.758+0000 7f56c6ffd640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f56a806af90 0x7f56a806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f56b8035000 tx=0x7f56b8030000 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:33.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.760+0000 7f56bffff640 1 -- 172.21.15.70:0/2144701889 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56b0054220 con 0x7f56c80c91d0 2023-10-08T05:53:33.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.950+0000 7f56cd7e2640 1 -- 172.21.15.70:0/2144701889 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f56c80c2c60 con 0x7f56c80c91d0 2023-10-08T05:53:33.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.957+0000 7f56bffff640 1 -- 172.21.15.70:0/2144701889 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v33) v1 ==== 120+0+0 (secure 0 0 0) 0x7f56c80c2c60 con 0x7f56c80c91d0 2023-10-08T05:53:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 -- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f56a806af90 msgr2=0x7f56a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f56a806af90 0x7f56a806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f56b8035000 tx=0x7f56b8030000 comp rx=0 tx=0).stop 2023-10-08T05:53:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 -- 172.21.15.70:0/2144701889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56c80c91d0 msgr2=0x7f56c815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56c80c91d0 0x7f56c815b510 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f56b0002230 tx=0x7f56b000cf70 comp rx=0 tx=0).stop 2023-10-08T05:53:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 -- 172.21.15.70:0/2144701889 shutdown_connections 2023-10-08T05:53:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f56a806af90 0x7f56a806d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f56c80c9650 0x7f56c806cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56c80c91d0 0x7f56c815b510 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 --2- 172.21.15.70:0/2144701889 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f56b4003970 0x7f56c814f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 -- 172.21.15.70:0/2144701889 >> 172.21.15.70:0/2144701889 conn(0x7f56c806a5d0 msgr2=0x7f56c80cd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 -- 172.21.15.70:0/2144701889 shutdown_connections 2023-10-08T05:53:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:33.960+0000 7f56cd7e2640 1 -- 172.21.15.70:0/2144701889 wait complete. 2023-10-08T05:53:33.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-08T05:53:34.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.185+0000 7fbbe6bce640 1 Processor -- start 2023-10-08T05:53:35.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.185+0000 7fbbe6bce640 1 -- start start 2023-10-08T05:53:35.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbe6bce640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbbe015ab40 0x7fbbe00ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbe6bce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbbe00c90b0 0x7fbbe00adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbe6bce640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe015b2b0 con 0x7fbbe00cae70 2023-10-08T05:53:35.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbe6bce640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe015bf30 con 0x7fbbe00c90b0 2023-10-08T05:53:35.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbe6bce640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe015f7a0 con 0x7fbbe015ab40 2023-10-08T05:53:35.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbdffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbbe00c90b0 0x7fbbe00adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbe516b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbbe00cae70 0x7fbbe00702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60330/0 (socket says 172.21.15.70:60330) 2023-10-08T05:53:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbe516b640 1 -- 172.21.15.70:0/518005892 learned_addr learned my addr 172.21.15.70:0/518005892 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbe496a640 1 --2- 172.21.15.70:0/518005892 >> v2:172.21.15.181:3300/0 conn(0x7fbbe015ab40 0x7fbbe00ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbdffff640 1 -- 172.21.15.70:0/518005892 >> v2:172.21.15.181:3300/0 conn(0x7fbbe015ab40 msgr2=0x7fbbe00ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbdffff640 1 --2- 172.21.15.70:0/518005892 >> v2:172.21.15.181:3300/0 conn(0x7fbbe015ab40 0x7fbbe00ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbdffff640 1 -- 172.21.15.70:0/518005892 >> v1:172.21.15.70:6789/0 conn(0x7fbbe00cae70 legacy=0x7fbbe00702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.187+0000 7fbbdffff640 1 -- 172.21.15.70:0/518005892 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe015b590 con 0x7fbbe00c90b0 2023-10-08T05:53:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.189+0000 7fbbdffff640 1 --2- 172.21.15.70:0/518005892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 0x7fbbe00adc60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fbbe006e440 tx=0x7fbbcc02b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7385f882f5afc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.189+0000 7fbbdf7fe640 1 -- 172.21.15.70:0/518005892 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc030710 con 0x7fbbe00c90b0 2023-10-08T05:53:35.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.189+0000 7fbbdf7fe640 1 -- 172.21.15.70:0/518005892 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbcc02e050 con 0x7fbbe00c90b0 2023-10-08T05:53:35.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.189+0000 7fbbdf7fe640 1 -- 172.21.15.70:0/518005892 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc030a90 con 0x7fbbe00c90b0 2023-10-08T05:53:35.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.189+0000 7fbbe6bce640 1 -- 172.21.15.70:0/518005892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 msgr2=0x7fbbe00adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.189+0000 7fbbe6bce640 1 --2- 172.21.15.70:0/518005892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 0x7fbbe00adc60 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fbbe006e440 tx=0x7fbbcc02b710 comp rx=0 tx=0).stop 2023-10-08T05:53:35.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.190+0000 7fbbe6bce640 1 -- 172.21.15.70:0/518005892 shutdown_connections 2023-10-08T05:53:35.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.190+0000 7fbbe6bce640 1 --2- 172.21.15.70:0/518005892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 0x7fbbe00adc60 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.190+0000 7fbbe6bce640 1 --2- 172.21.15.70:0/518005892 >> v2:172.21.15.181:3300/0 conn(0x7fbbe015ab40 0x7fbbe00ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.190+0000 7fbbe6bce640 1 -- 172.21.15.70:0/518005892 >> 172.21.15.70:0/518005892 conn(0x7fbbe006a380 msgr2=0x7fbbe00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:35.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.190+0000 7fbbe6bce640 1 -- 172.21.15.70:0/518005892 shutdown_connections 2023-10-08T05:53:35.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.190+0000 7fbbe6bce640 1 -- 172.21.15.70:0/518005892 wait complete. 2023-10-08T05:53:35.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.191+0000 7fbbe6bce640 1 Processor -- start 2023-10-08T05:53:35.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.192+0000 7fbbe6bce640 1 -- start start 2023-10-08T05:53:35.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.192+0000 7fbbe6bce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 0x7fbbe01f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.192+0000 7fbbe6bce640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbe00cae70 0x7fbbe0155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe6bce640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe015ab40 0x7fbbe015d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe6bce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe01555f0 con 0x7fbbe00c90b0 2023-10-08T05:53:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe6bce640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00ad7c0 con 0x7fbbe015ab40 2023-10-08T05:53:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe6bce640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0154c50 con 0x7fbbe00cae70 2023-10-08T05:53:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe496a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 0x7fbbe01f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe496a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 0x7fbbe01f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59008/0 (socket says 172.21.15.70:59008) 2023-10-08T05:53:35.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe496a640 1 -- 172.21.15.70:0/741120398 learned_addr learned my addr 172.21.15.70:0/741120398 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe496a640 1 -- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbe00cae70 msgr2=0x7fbbe0155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe496a640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbe00cae70 0x7fbbe0155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe496a640 1 -- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe015ab40 msgr2=0x7fbbe015d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe496a640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe015ab40 0x7fbbe015d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe496a640 1 -- 172.21.15.70:0/741120398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc009400 con 0x7fbbe00c90b0 2023-10-08T05:53:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.193+0000 7fbbe496a640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 0x7fbbe01f1e00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c7f0 tx=0x7fbbd400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.194+0000 7fbbdd7fa640 1 -- 172.21.15.70:0/741120398 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4004220 con 0x7fbbe00c90b0 2023-10-08T05:53:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.194+0000 7fbbdd7fa640 1 -- 172.21.15.70:0/741120398 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd40043a0 con 0x7fbbe00c90b0 2023-10-08T05:53:35.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.194+0000 7fbbdd7fa640 1 -- 172.21.15.70:0/741120398 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4007860 con 0x7fbbe00c90b0 2023-10-08T05:53:35.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.194+0000 7fbbe6bce640 1 -- 172.21.15.70:0/741120398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe0154f30 con 0x7fbbe00c90b0 2023-10-08T05:53:35.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.194+0000 7fbbe6bce640 1 -- 172.21.15.70:0/741120398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe01f2340 con 0x7fbbe00c90b0 2023-10-08T05:53:35.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.195+0000 7fbbe6bce640 1 -- 172.21.15.70:0/741120398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe0155730 con 0x7fbbe00c90b0 2023-10-08T05:53:35.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.196+0000 7fbbdd7fa640 1 -- 172.21.15.70:0/741120398 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbbd4007bd0 con 0x7fbbe00c90b0 2023-10-08T05:53:35.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.197+0000 7fbbdd7fa640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fbbc406af90 0x7fbbc406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.197+0000 7fbbdd7fa640 1 -- 172.21.15.70:0/741120398 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6038+0+0 (secure 0 0 0) 0x7fbbd4089a30 con 0x7fbbe00c90b0 2023-10-08T05:53:35.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.197+0000 7fbbdffff640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fbbc406af90 0x7fbbc406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.199+0000 7fbbdffff640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fbbc406af90 0x7fbbc406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc034000 tx=0x7fbbcc02f000 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:35.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.201+0000 7fbbdd7fa640 1 -- 172.21.15.70:0/741120398 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbe0155730 con 0x7fbbe00c90b0 2023-10-08T05:53:35.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.458+0000 7fbbe6bce640 1 -- 172.21.15.70:0/741120398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7fbbe0155730 con 0x7fbbe00c90b0 2023-10-08T05:53:35.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.606+0000 7fbbdd7fa640 1 -- 172.21.15.70:0/741120398 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v661) v1 ==== 166+0+0 (secure 0 0 0) 0x7fbbd401e020 con 0x7fbbe00c90b0 2023-10-08T05:53:35.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 -- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fbbc406af90 msgr2=0x7fbbc406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fbbc406af90 0x7fbbc406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc034000 tx=0x7fbbcc02f000 comp rx=0 tx=0).stop 2023-10-08T05:53:35.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 -- 172.21.15.70:0/741120398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 msgr2=0x7fbbe01f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 0x7fbbe01f1e00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c7f0 tx=0x7fbbd400ccb0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 -- 172.21.15.70:0/741120398 shutdown_connections 2023-10-08T05:53:35.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fbbc406af90 0x7fbbc406d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbe015ab40 0x7fbbe015d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbe00cae70 0x7fbbe0155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 --2- 172.21.15.70:0/741120398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbe00c90b0 0x7fbbe01f1e00 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 -- 172.21.15.70:0/741120398 >> 172.21.15.70:0/741120398 conn(0x7fbbe006a380 msgr2=0x7fbbe00ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:35.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.611+0000 7fbbe6bce640 1 -- 172.21.15.70:0/741120398 shutdown_connections 2023-10-08T05:53:35.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:34.612+0000 7fbbe6bce640 1 -- 172.21.15.70:0/741120398 wait complete. 2023-10-08T05:53:35.490 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-10-08T05:53:35.490 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-10-08T05:53:35.490 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T05:53:35.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:35.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.718+0000 7f613d839640 1 Processor -- start 2023-10-08T05:53:35.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.719+0000 7f613d839640 1 -- start start 2023-10-08T05:53:35.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.720+0000 7f613d839640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f613815ab40 0x7f61380ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.720+0000 7f613d839640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f61380c90b0 0x7f61380adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.720+0000 7f613d839640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61380cae70 0x7f61380cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.720+0000 7f613d839640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613815bf30 con 0x7f61380c90b0 2023-10-08T05:53:35.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.720+0000 7f613d839640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61380cb840 con 0x7f613815ab40 2023-10-08T05:53:35.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.720+0000 7f613d839640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613815b2b0 con 0x7f61380cae70 2023-10-08T05:53:35.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f612e7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f61380c90b0 0x7f61380adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f612e7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f61380c90b0 0x7f61380adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59014/0 (socket says 172.21.15.70:59014) 2023-10-08T05:53:35.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f612e7fc640 1 -- 172.21.15.70:0/3643010928 learned_addr learned my addr 172.21.15.70:0/3643010928 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f61377fe640 1 --2- 172.21.15.70:0/3643010928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61380cae70 0x7f61380cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f612e7fc640 1 -- 172.21.15.70:0/3643010928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61380cae70 msgr2=0x7f61380cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f612e7fc640 1 --2- 172.21.15.70:0/3643010928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61380cae70 0x7f61380cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f6136ffd640 1 --2- 172.21.15.70:0/3643010928 >> v2:172.21.15.181:3300/0 conn(0x7f613815ab40 0x7f61380ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f612e7fc640 1 -- 172.21.15.70:0/3643010928 >> v2:172.21.15.181:3300/0 conn(0x7f613815ab40 msgr2=0x7f61380ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:35.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f612e7fc640 1 --2- 172.21.15.70:0/3643010928 >> v2:172.21.15.181:3300/0 conn(0x7f613815ab40 0x7f61380ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f612e7fc640 1 -- 172.21.15.70:0/3643010928 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613815b590 con 0x7f61380c90b0 2023-10-08T05:53:35.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.721+0000 7f61377fe640 1 --2- 172.21.15.70:0/3643010928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61380cae70 0x7f61380cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:35.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.722+0000 7f612e7fc640 1 --2- 172.21.15.70:0/3643010928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 0x7f61380adc60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f613806e440 tx=0x7f612002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69138c4ee074afdd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:35.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.723+0000 7f61367fc640 1 -- 172.21.15.70:0/3643010928 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120031870 con 0x7f61380c90b0 2023-10-08T05:53:35.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.723+0000 7f61367fc640 1 -- 172.21.15.70:0/3643010928 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61200319d0 con 0x7f61380c90b0 2023-10-08T05:53:35.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.723+0000 7f61367fc640 1 -- 172.21.15.70:0/3643010928 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6120031d60 con 0x7f61380c90b0 2023-10-08T05:53:35.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.723+0000 7f613d839640 1 -- 172.21.15.70:0/3643010928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 msgr2=0x7f61380adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.723+0000 7f613d839640 1 --2- 172.21.15.70:0/3643010928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 0x7f61380adc60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f613806e440 tx=0x7f612002b910 comp rx=0 tx=0).stop 2023-10-08T05:53:35.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.724+0000 7f613d839640 1 -- 172.21.15.70:0/3643010928 shutdown_connections 2023-10-08T05:53:35.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.724+0000 7f613d839640 1 --2- 172.21.15.70:0/3643010928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61380cae70 0x7f61380cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.724+0000 7f613d839640 1 --2- 172.21.15.70:0/3643010928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 0x7f61380adc60 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.724+0000 7f613d839640 1 --2- 172.21.15.70:0/3643010928 >> v2:172.21.15.181:3300/0 conn(0x7f613815ab40 0x7f61380ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.724+0000 7f613d839640 1 -- 172.21.15.70:0/3643010928 >> 172.21.15.70:0/3643010928 conn(0x7f613806a380 msgr2=0x7f61380b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:35.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.724+0000 7f613d839640 1 -- 172.21.15.70:0/3643010928 shutdown_connections 2023-10-08T05:53:35.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.724+0000 7f613d839640 1 -- 172.21.15.70:0/3643010928 wait complete. 2023-10-08T05:53:35.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.726+0000 7f613d839640 1 Processor -- start 2023-10-08T05:53:35.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.726+0000 7f613d839640 1 -- start start 2023-10-08T05:53:35.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.727+0000 7f613d839640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 0x7f613814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.727+0000 7f613d839640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61380cae70 0x7f61380c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.727+0000 7f613d839640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f613815ab40 0x7f6138159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.727+0000 7f613d839640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61380c5300 con 0x7f61380c90b0 2023-10-08T05:53:35.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.727+0000 7f613d839640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61380c7c30 con 0x7f61380cae70 2023-10-08T05:53:35.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.727+0000 7f613d839640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61380c6b90 con 0x7f613815ab40 2023-10-08T05:53:35.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.727+0000 7f6136ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 0x7f613814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.727+0000 7f6136ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 0x7f613814e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59026/0 (socket says 172.21.15.70:59026) 2023-10-08T05:53:35.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.727+0000 7f6136ffd640 1 -- 172.21.15.70:0/929844280 learned_addr learned my addr 172.21.15.70:0/929844280 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:35.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f61377fe640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f613815ab40 0x7f6138159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f612e7fc640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61380cae70 0x7f61380c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f6136ffd640 1 -- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f613815ab40 msgr2=0x7f6138159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f6136ffd640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f613815ab40 0x7f6138159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f6136ffd640 1 -- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61380cae70 msgr2=0x7f61380c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f6136ffd640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61380cae70 0x7f61380c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f6136ffd640 1 -- 172.21.15.70:0/929844280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61200094d0 con 0x7f61380c90b0 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f6136ffd640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 0x7f613814e5c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f612800c7f0 tx=0x7f612800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f6134ff9640 1 -- 172.21.15.70:0/929844280 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128004220 con 0x7f61380c90b0 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f6134ff9640 1 -- 172.21.15.70:0/929844280 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6128004380 con 0x7f61380c90b0 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f6134ff9640 1 -- 172.21.15.70:0/929844280 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128007840 con 0x7f61380c90b0 2023-10-08T05:53:35.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f613d839640 1 -- 172.21.15.70:0/929844280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61380c6e70 con 0x7f61380c90b0 2023-10-08T05:53:35.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.728+0000 7f613d839640 1 -- 172.21.15.70:0/929844280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61380c7430 con 0x7f61380c90b0 2023-10-08T05:53:35.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.729+0000 7f613d839640 1 -- 172.21.15.70:0/929844280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6138067490 con 0x7f61380c90b0 2023-10-08T05:53:35.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.730+0000 7f6134ff9640 1 -- 172.21.15.70:0/929844280 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6128007bb0 con 0x7f61380c90b0 2023-10-08T05:53:35.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.731+0000 7f6134ff9640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f611806af90 0x7f611806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:35.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.731+0000 7f6134ff9640 1 -- 172.21.15.70:0/929844280 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f61280899f0 con 0x7f61380c90b0 2023-10-08T05:53:35.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.732+0000 7f612e7fc640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f611806af90 0x7f611806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:35.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.733+0000 7f612e7fc640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f611806af90 0x7f611806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6120036000 tx=0x7f612003c040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:35.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.736+0000 7f6134ff9640 1 -- 172.21.15.70:0/929844280 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6128054690 con 0x7f61380c90b0 2023-10-08T05:53:35.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.924+0000 7f613d839640 1 -- 172.21.15.70:0/929844280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61380b0ef0 con 0x7f61380c90b0 2023-10-08T05:53:35.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.926+0000 7f6134ff9640 1 -- 172.21.15.70:0/929844280 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+14333 (secure 0 0 0) 0x7f612801e020 con 0x7f61380c90b0 2023-10-08T05:53:35.928 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:35.928 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":110,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:32.581316+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6834/24744490":"2023-10-09T05:53:29.478048+0000","172.21.15.181:0/2094733819":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/2580364003":"2023-10-09T05:53:25.709759+0000","172.21.15.181:6833/1003988322":"2023-10-09T05:53:25.709759+0000","172.21.15.181:6832/1003988322":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/4283069774":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/3846159301":"2023-10-09T05:53:19.462537+0000","172.21.15.70:6835/24744490":"2023-10-09T05:53:29.478048+0000","172.21.15.181:0/1194629361":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/3022940786":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/3990555712":"2023-10-09T05:53:19.462537+0000","172.21.15.181:6833/2060337708":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/881087078":"2023-10-09T05:53:25.709759+0000","172.21.15.181:6832/2060337708":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/773795607":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/540392205":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/3787865617":"2023-10-09T05:53:25.709759+0000","172.21.15.181:0/618139400":"2023-10-09T05:53:19.462537+0000","172.21.15.181:0/999647594":"2023-10-08T06:53:12.487289+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:35.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.928+0000 7f613d839640 1 -- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f611806af90 msgr2=0x7f611806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.928+0000 7f613d839640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f611806af90 0x7f611806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6120036000 tx=0x7f612003c040 comp rx=0 tx=0).stop 2023-10-08T05:53:35.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.928+0000 7f613d839640 1 -- 172.21.15.70:0/929844280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 msgr2=0x7f613814e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:35.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.928+0000 7f613d839640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 0x7f613814e5c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f612800c7f0 tx=0x7f612800ccb0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.928+0000 7f613d839640 1 -- 172.21.15.70:0/929844280 shutdown_connections 2023-10-08T05:53:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.928+0000 7f613d839640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f611806af90 0x7f611806d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.928+0000 7f613d839640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f613815ab40 0x7f6138159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.928+0000 7f613d839640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f61380cae70 0x7f61380c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.928+0000 7f613d839640 1 --2- 172.21.15.70:0/929844280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61380c90b0 0x7f613814e5c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.928+0000 7f613d839640 1 -- 172.21.15.70:0/929844280 >> 172.21.15.70:0/929844280 conn(0x7f613806a380 msgr2=0x7f61380ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.929+0000 7f613d839640 1 -- 172.21.15.70:0/929844280 shutdown_connections 2023-10-08T05:53:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:35.929+0000 7f613d839640 1 -- 172.21.15.70:0/929844280 wait complete. 2023-10-08T05:53:35.947 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:53:35.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:53:36.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.164+0000 7fd864dce640 1 Processor -- start 2023-10-08T05:53:36.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.164+0000 7fd864dce640 1 -- start start 2023-10-08T05:53:36.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.166+0000 7fd864dce640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8600b1750 0x7fd86006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.166+0000 7fd864dce640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd86006d240 0x7fd860070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.166+0000 7fd864dce640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86015bf30 con 0x7fd86015aad0 2023-10-08T05:53:36.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.166+0000 7fd864dce640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86006d6d0 con 0x7fd86006d240 2023-10-08T05:53:36.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.166+0000 7fd864dce640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600b1bb0 con 0x7fd8600b1750 2023-10-08T05:53:36.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.166+0000 7fd85e59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd86015aad0 0x7fd8600c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60342/0 (socket says 172.21.15.70:60342) 2023-10-08T05:53:36.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.166+0000 7fd85e59c640 1 -- 172.21.15.70:0/400056569 learned_addr learned my addr 172.21.15.70:0/400056569 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:36.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd857fff640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8600b1750 0x7fd86006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85ed9d640 1 --2- 172.21.15.70:0/400056569 >> v2:172.21.15.181:3300/0 conn(0x7fd86006d240 0x7fd860070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984294605 0 0) 0x7fd86015bf30 con 0x7fd86015aad0 2023-10-08T05:53:36.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd84c003520 con 0x7fd86015aad0 2023-10-08T05:53:36.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3419671832 0 0) 0x7fd84c003520 con 0x7fd86015aad0 2023-10-08T05:53:36.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8600b1750 msgr2=0x7fd86006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:36.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8600b1750 0x7fd86006cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 >> v2:172.21.15.181:3300/0 conn(0x7fd86006d240 msgr2=0x7fd860070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 --2- 172.21.15.70:0/400056569 >> v2:172.21.15.181:3300/0 conn(0x7fd86006d240 0x7fd860070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86015b320 con 0x7fd86015aad0 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd850001670 con 0x7fd86015aad0 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 >> v1:172.21.15.70:6789/0 conn(0x7fd86015aad0 legacy=0x7fd8600c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8600b1750 0x7fd84c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c003ef0 0x7fd84c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 0x7fd84c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86006d6d0 con 0x7fd84c004d00 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600b1bb0 con 0x7fd84c003ef0 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.167+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c00fd90 con 0x7fd8600b1750 2023-10-08T05:53:36.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.168+0000 7fd85ed9d640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 0x7fd84c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.168+0000 7fd857fff640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c003ef0 0x7fd84c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.168+0000 7fd85e59c640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8600b1750 0x7fd84c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.168+0000 7fd85ed9d640 1 -- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8600b1750 msgr2=0x7fd84c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:36.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.168+0000 7fd85ed9d640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8600b1750 0x7fd84c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.168+0000 7fd85ed9d640 1 -- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c003ef0 msgr2=0x7fd84c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:36.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.168+0000 7fd85ed9d640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c003ef0 0x7fd84c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.168+0000 7fd85ed9d640 1 -- 172.21.15.70:0/400056569 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c003520 con 0x7fd84c004d00 2023-10-08T05:53:36.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.168+0000 7fd85e59c640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8600b1750 0x7fd84c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:36.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.168+0000 7fd857fff640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd84c003ef0 0x7fd84c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:36.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.169+0000 7fd85ed9d640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 0x7fd84c005180 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fd858009fc0 tx=0x7fd84c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7310a79acc3b2e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:36.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.169+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85802cc40 con 0x7fd84c004d00 2023-10-08T05:53:36.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.169+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85800f040 con 0x7fd84c004d00 2023-10-08T05:53:36.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.169+0000 7fd85dd9b640 1 -- 172.21.15.70:0/400056569 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858034440 con 0x7fd84c004d00 2023-10-08T05:53:36.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.170+0000 7fd864dce640 1 -- 172.21.15.70:0/400056569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 msgr2=0x7fd84c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:36.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.170+0000 7fd864dce640 1 --2- 172.21.15.70:0/400056569 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 0x7fd84c005180 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fd858009fc0 tx=0x7fd84c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.170+0000 7fd85ed9d640 1 -- 172.21.15.70:0/400056569 reap_dead start 2023-10-08T05:53:36.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.170+0000 7fd864dce640 1 -- 172.21.15.70:0/400056569 shutdown_connections 2023-10-08T05:53:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.170+0000 7fd864dce640 1 -- 172.21.15.70:0/400056569 >> 172.21.15.70:0/400056569 conn(0x7fd860069d50 msgr2=0x7fd86006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.170+0000 7fd864dce640 1 -- 172.21.15.70:0/400056569 shutdown_connections 2023-10-08T05:53:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.170+0000 7fd864dce640 1 -- 172.21.15.70:0/400056569 wait complete. 2023-10-08T05:53:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.172+0000 7fd864dce640 1 Processor -- start 2023-10-08T05:53:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.172+0000 7fd864dce640 1 -- start start 2023-10-08T05:53:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.173+0000 7fd864dce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 0x7fd86014f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.173+0000 7fd864dce640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8601427b0 0x7fd86013f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.173+0000 7fd864dce640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd86013fc00 0x7fd860157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.173+0000 7fd864dce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600cce50 con 0x7fd84c004d00 2023-10-08T05:53:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.173+0000 7fd864dce640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86013f6d0 con 0x7fd86013fc00 2023-10-08T05:53:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.173+0000 7fd864dce640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600cc1d0 con 0x7fd8601427b0 2023-10-08T05:53:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.173+0000 7fd85e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 0x7fd86014f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.173+0000 7fd85e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 0x7fd86014f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59056/0 (socket says 172.21.15.70:59056) 2023-10-08T05:53:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.173+0000 7fd85e59c640 1 -- 172.21.15.70:0/3483590782 learned_addr learned my addr 172.21.15.70:0/3483590782 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd85e59c640 1 -- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8601427b0 msgr2=0x7fd86013f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd85e59c640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8601427b0 0x7fd86013f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd85e59c640 1 -- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd86013fc00 msgr2=0x7fd860157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd85ed9d640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd86013fc00 0x7fd860157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd85e59c640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd86013fc00 0x7fd860157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd85e59c640 1 -- 172.21.15.70:0/3483590782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600cc4b0 con 0x7fd84c004d00 2023-10-08T05:53:36.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd85e59c640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 0x7fd86014f780 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fd85000dae0 tx=0x7fd850007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:36.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd856ffd640 1 -- 172.21.15.70:0/3483590782 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500074e0 con 0x7fd84c004d00 2023-10-08T05:53:36.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd856ffd640 1 -- 172.21.15.70:0/3483590782 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd850007640 con 0x7fd84c004d00 2023-10-08T05:53:36.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd856ffd640 1 -- 172.21.15.70:0/3483590782 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500049e0 con 0x7fd84c004d00 2023-10-08T05:53:36.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd864dce640 1 -- 172.21.15.70:0/3483590782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8600cc790 con 0x7fd84c004d00 2023-10-08T05:53:36.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.174+0000 7fd864dce640 1 -- 172.21.15.70:0/3483590782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8600cbe30 con 0x7fd84c004d00 2023-10-08T05:53:36.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.175+0000 7fd864dce640 1 -- 172.21.15.70:0/3483590782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86004e160 con 0x7fd84c004d00 2023-10-08T05:53:36.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.176+0000 7fd856ffd640 1 -- 172.21.15.70:0/3483590782 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd850004b40 con 0x7fd84c004d00 2023-10-08T05:53:36.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.177+0000 7fd856ffd640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd84006afc0 0x7fd84006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.177+0000 7fd856ffd640 1 -- 172.21.15.70:0/3483590782 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6038+0+0 (secure 0 0 0) 0x7fd8500894c0 con 0x7fd84c004d00 2023-10-08T05:53:36.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.178+0000 7fd857fff640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd84006afc0 0x7fd84006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.179+0000 7fd857fff640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd84006afc0 0x7fd84006d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8600b2eb0 tx=0x7fd848008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.181+0000 7fd856ffd640 1 -- 172.21.15.70:0/3483590782 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8500541f0 con 0x7fd84c004d00 2023-10-08T05:53:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.368+0000 7fd864dce640 1 -- 172.21.15.70:0/3483590782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd86015b8b0 con 0x7fd84c004d00 2023-10-08T05:53:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.369+0000 7fd856ffd640 1 -- 172.21.15.70:0/3483590782 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd850057810 con 0x7fd84c004d00 2023-10-08T05:53:36.370 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:53:36.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 -- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd84006afc0 msgr2=0x7fd84006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd84006afc0 0x7fd84006d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd8600b2eb0 tx=0x7fd848008040 comp rx=0 tx=0).stop 2023-10-08T05:53:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 -- 172.21.15.70:0/3483590782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 msgr2=0x7fd86014f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 0x7fd86014f780 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fd85000dae0 tx=0x7fd850007e10 comp rx=0 tx=0).stop 2023-10-08T05:53:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 -- 172.21.15.70:0/3483590782 shutdown_connections 2023-10-08T05:53:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd84006afc0 0x7fd84006d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd86013fc00 0x7fd860157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8601427b0 0x7fd86013f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 --2- 172.21.15.70:0/3483590782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd84c004d00 0x7fd86014f780 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 -- 172.21.15.70:0/3483590782 >> 172.21.15.70:0/3483590782 conn(0x7fd860069d50 msgr2=0x7fd860152390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 -- 172.21.15.70:0/3483590782 shutdown_connections 2023-10-08T05:53:36.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.371+0000 7fd864dce640 1 -- 172.21.15.70:0/3483590782 wait complete. 2023-10-08T05:53:36.392 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-08T05:53:36.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.597+0000 7ff91ac6f640 1 Processor -- start 2023-10-08T05:53:36.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.598+0000 7ff91ac6f640 1 -- start start 2023-10-08T05:53:36.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.599+0000 7ff91ac6f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9140cc130 0x7ff9140cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.599+0000 7ff91ac6f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff914161260 0x7ff914070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.599+0000 7ff91ac6f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91406a7e0 con 0x7ff9140cbd50 2023-10-08T05:53:36.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.599+0000 7ff91ac6f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91415eb30 con 0x7ff9140cc130 2023-10-08T05:53:36.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.599+0000 7ff91ac6f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140cda90 con 0x7ff914161260 2023-10-08T05:53:36.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.600+0000 7ff913fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9140cc130 0x7ff9140cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.600+0000 7ff913fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9140cc130 0x7ff9140cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59062/0 (socket says 172.21.15.70:59062) 2023-10-08T05:53:36.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.600+0000 7ff913fff640 1 -- 172.21.15.70:0/305885444 learned_addr learned my addr 172.21.15.70:0/305885444 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:36.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.600+0000 7ff91920c640 1 --2- 172.21.15.70:0/305885444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff914161260 0x7ff914070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.600+0000 7ff913fff640 1 -- 172.21.15.70:0/305885444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff914161260 msgr2=0x7ff914070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:36.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.600+0000 7ff913fff640 1 --2- 172.21.15.70:0/305885444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff914161260 0x7ff914070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.600+0000 7ff913fff640 1 -- 172.21.15.70:0/305885444 >> v1:172.21.15.181:6789/0 conn(0x7ff9140cbd50 legacy=0x7ff91415e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:36.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.600+0000 7ff913fff640 1 -- 172.21.15.70:0/305885444 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91406a190 con 0x7ff9140cc130 2023-10-08T05:53:36.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.601+0000 7ff913fff640 1 --2- 172.21.15.70:0/305885444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 0x7ff9140cd4b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff908009870 tx=0x7ff90802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b537fc780865370 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:36.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff9137fe640 1 -- 172.21.15.70:0/305885444 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9080317c0 con 0x7ff9140cc130 2023-10-08T05:53:36.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff9137fe640 1 -- 172.21.15.70:0/305885444 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff908031920 con 0x7ff9140cc130 2023-10-08T05:53:36.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff9137fe640 1 -- 172.21.15.70:0/305885444 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908031cb0 con 0x7ff9140cc130 2023-10-08T05:53:36.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff91ac6f640 1 -- 172.21.15.70:0/305885444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 msgr2=0x7ff9140cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:36.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff91ac6f640 1 --2- 172.21.15.70:0/305885444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 0x7ff9140cd4b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff908009870 tx=0x7ff90802b860 comp rx=0 tx=0).stop 2023-10-08T05:53:36.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff91ac6f640 1 -- 172.21.15.70:0/305885444 shutdown_connections 2023-10-08T05:53:36.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff91ac6f640 1 --2- 172.21.15.70:0/305885444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff914161260 0x7ff914070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff91ac6f640 1 --2- 172.21.15.70:0/305885444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 0x7ff9140cd4b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff91ac6f640 1 -- 172.21.15.70:0/305885444 >> 172.21.15.70:0/305885444 conn(0x7ff9140b1480 msgr2=0x7ff91406cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:36.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff91ac6f640 1 -- 172.21.15.70:0/305885444 shutdown_connections 2023-10-08T05:53:36.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.602+0000 7ff91ac6f640 1 -- 172.21.15.70:0/305885444 wait complete. 2023-10-08T05:53:36.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.604+0000 7ff91ac6f640 1 Processor -- start 2023-10-08T05:53:36.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.604+0000 7ff91ac6f640 1 -- start start 2023-10-08T05:53:36.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.605+0000 7ff91ac6f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9140cbd50 0x7ff9140b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.605+0000 7ff91ac6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 0x7ff9140ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.605+0000 7ff91ac6f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff914161260 0x7ff9140ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.605+0000 7ff91ac6f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140ad5d0 con 0x7ff9140cc130 2023-10-08T05:53:36.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.605+0000 7ff91ac6f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140ad770 con 0x7ff9140cbd50 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.605+0000 7ff91ac6f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140ac850 con 0x7ff914161260 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.605+0000 7ff913fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 0x7ff9140ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.606+0000 7ff913fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 0x7ff9140ad090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59066/0 (socket says 172.21.15.70:59066) 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.606+0000 7ff913fff640 1 -- 172.21.15.70:0/4203012368 learned_addr learned my addr 172.21.15.70:0/4203012368 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.606+0000 7ff91920c640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff914161260 0x7ff9140ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.606+0000 7ff918a0b640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9140cbd50 0x7ff9140b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.606+0000 7ff913fff640 1 -- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff914161260 msgr2=0x7ff9140ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.606+0000 7ff913fff640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff914161260 0x7ff9140ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.606+0000 7ff913fff640 1 -- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9140cbd50 msgr2=0x7ff9140b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.606+0000 7ff913fff640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9140cbd50 0x7ff9140b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:36.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.606+0000 7ff913fff640 1 -- 172.21.15.70:0/4203012368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9080094d0 con 0x7ff9140cc130 2023-10-08T05:53:36.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.606+0000 7ff913fff640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 0x7ff9140ad090 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff908000c00 tx=0x7ff9080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:36.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.607+0000 7ff9117fa640 1 -- 172.21.15.70:0/4203012368 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908032860 con 0x7ff9140cc130 2023-10-08T05:53:36.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.607+0000 7ff9117fa640 1 -- 172.21.15.70:0/4203012368 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9080329c0 con 0x7ff9140cc130 2023-10-08T05:53:36.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.607+0000 7ff9117fa640 1 -- 172.21.15.70:0/4203012368 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908032d70 con 0x7ff9140cc130 2023-10-08T05:53:36.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.607+0000 7ff91ac6f640 1 -- 172.21.15.70:0/4203012368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9140acb30 con 0x7ff9140cc130 2023-10-08T05:53:36.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.607+0000 7ff91ac6f640 1 -- 172.21.15.70:0/4203012368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9140adca0 con 0x7ff9140cc130 2023-10-08T05:53:36.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.608+0000 7ff91ac6f640 1 -- 172.21.15.70:0/4203012368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9140ade90 con 0x7ff9140cc130 2023-10-08T05:53:36.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.609+0000 7ff9117fa640 1 -- 172.21.15.70:0/4203012368 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff90804e450 con 0x7ff9140cc130 2023-10-08T05:53:36.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.610+0000 7ff9117fa640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ff90006af90 0x7ff90006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:36.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.610+0000 7ff9117fa640 1 -- 172.21.15.70:0/4203012368 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6038+0+0 (secure 0 0 0) 0x7ff9080abcd0 con 0x7ff9140cc130 2023-10-08T05:53:36.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.611+0000 7ff918a0b640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ff90006af90 0x7ff90006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:36.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.612+0000 7ff918a0b640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ff90006af90 0x7ff90006d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff904009f20 tx=0x7ff9040094d0 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:36.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.615+0000 7ff9117fa640 1 -- 172.21.15.70:0/4203012368 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff908076ba0 con 0x7ff9140cc130 2023-10-08T05:53:36.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:36.876+0000 7ff91ac6f640 1 -- 172.21.15.70:0/4203012368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7ff914067490 con 0x7ff9140cc130 2023-10-08T05:53:37.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.624+0000 7ff9117fa640 1 -- 172.21.15.70:0/4203012368 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v664) v1 ==== 186+0+0 (secure 0 0 0) 0x7ff90807a1c0 con 0x7ff9140cc130 2023-10-08T05:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.628+0000 7ff91ac6f640 1 -- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ff90006af90 msgr2=0x7ff90006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.628+0000 7ff91ac6f640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ff90006af90 0x7ff90006d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff904009f20 tx=0x7ff9040094d0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.628+0000 7ff91ac6f640 1 -- 172.21.15.70:0/4203012368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 msgr2=0x7ff9140ad090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.628+0000 7ff91ac6f640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 0x7ff9140ad090 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff908000c00 tx=0x7ff9080046c0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.628+0000 7ff91ac6f640 1 -- 172.21.15.70:0/4203012368 shutdown_connections 2023-10-08T05:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.628+0000 7ff91ac6f640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ff90006af90 0x7ff90006d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.628+0000 7ff91ac6f640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff914161260 0x7ff9140ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.628+0000 7ff91ac6f640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9140cc130 0x7ff9140ad090 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.628+0000 7ff91ac6f640 1 --2- 172.21.15.70:0/4203012368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9140cbd50 0x7ff9140b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.628+0000 7ff91ac6f640 1 -- 172.21.15.70:0/4203012368 >> 172.21.15.70:0/4203012368 conn(0x7ff9140b1480 msgr2=0x7ff91415fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.629+0000 7ff91ac6f640 1 -- 172.21.15.70:0/4203012368 shutdown_connections 2023-10-08T05:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.629+0000 7ff91ac6f640 1 -- 172.21.15.70:0/4203012368 wait complete. 2023-10-08T05:53:37.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T05:53:37.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.874+0000 7f6307221640 1 Processor -- start 2023-10-08T05:53:37.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.875+0000 7f6307221640 1 -- start start 2023-10-08T05:53:37.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.876+0000 7f6307221640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f630015cce0 0x7f63000cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:37.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.876+0000 7f6307221640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63000b2c40 0x7f63000cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:37.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.876+0000 7f6307221640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630006ce20 con 0x7f630015d0c0 2023-10-08T05:53:37.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.876+0000 7f6307221640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63000cf850 con 0x7f630015cce0 2023-10-08T05:53:37.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.876+0000 7f6307221640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630006c060 con 0x7f63000b2c40 2023-10-08T05:53:37.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.876+0000 7f6304fbd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f630015cce0 0x7f63000cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:37.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.877+0000 7f6304fbd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f630015cce0 0x7f63000cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59070/0 (socket says 172.21.15.70:59070) 2023-10-08T05:53:37.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.877+0000 7f6304fbd640 1 -- 172.21.15.70:0/967761305 learned_addr learned my addr 172.21.15.70:0/967761305 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:37.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.877+0000 7f6304fbd640 1 -- 172.21.15.70:0/967761305 >> v2:172.21.15.181:3300/0 conn(0x7f63000b2c40 msgr2=0x7f63000cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:37.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.877+0000 7f6304fbd640 1 --2- 172.21.15.70:0/967761305 >> v2:172.21.15.181:3300/0 conn(0x7f63000b2c40 0x7f63000cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.877+0000 7f6304fbd640 1 -- 172.21.15.70:0/967761305 >> v1:172.21.15.70:6789/0 conn(0x7f630015d0c0 legacy=0x7f63000b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:37.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.877+0000 7f6304fbd640 1 -- 172.21.15.70:0/967761305 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630006c340 con 0x7f630015cce0 2023-10-08T05:53:37.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.878+0000 7f6304fbd640 1 --2- 172.21.15.70:0/967761305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f630015cce0 0x7f63000cf000 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f63000b04b0 tx=0x7f62f002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c2283264b54452 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:37.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.878+0000 7f62ff7fe640 1 -- 172.21.15.70:0/967761305 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f002cb10 con 0x7f630015cce0 2023-10-08T05:53:37.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.878+0000 7f62ff7fe640 1 -- 172.21.15.70:0/967761305 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f002cc70 con 0x7f630015cce0 2023-10-08T05:53:37.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.878+0000 7f62ff7fe640 1 -- 172.21.15.70:0/967761305 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f00343f0 con 0x7f630015cce0 2023-10-08T05:53:37.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.879+0000 7f6307221640 1 -- 172.21.15.70:0/967761305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f630015cce0 msgr2=0x7f63000cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:37.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.879+0000 7f6307221640 1 --2- 172.21.15.70:0/967761305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f630015cce0 0x7f63000cf000 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f63000b04b0 tx=0x7f62f002bf70 comp rx=0 tx=0).stop 2023-10-08T05:53:37.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.879+0000 7f6307221640 1 -- 172.21.15.70:0/967761305 shutdown_connections 2023-10-08T05:53:37.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.879+0000 7f6307221640 1 --2- 172.21.15.70:0/967761305 >> v2:172.21.15.181:3300/0 conn(0x7f63000b2c40 0x7f63000cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.879+0000 7f6307221640 1 --2- 172.21.15.70:0/967761305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f630015cce0 0x7f63000cf000 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.879+0000 7f6307221640 1 -- 172.21.15.70:0/967761305 >> 172.21.15.70:0/967761305 conn(0x7f630006e650 msgr2=0x7f630006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:37.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.879+0000 7f6307221640 1 -- 172.21.15.70:0/967761305 shutdown_connections 2023-10-08T05:53:37.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.879+0000 7f6307221640 1 -- 172.21.15.70:0/967761305 wait complete. 2023-10-08T05:53:37.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.882+0000 7f6307221640 1 Processor -- start 2023-10-08T05:53:37.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.882+0000 7f6307221640 1 -- start start 2023-10-08T05:53:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.883+0000 7f6307221640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63000b2c40 0x7f63000cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.883+0000 7f6307221640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f630015cce0 0x7f63000b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.883+0000 7f6307221640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63001f5bf0 0x7f6300148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.883+0000 7f6307221640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63001f6080 con 0x7f63001f5bf0 2023-10-08T05:53:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.883+0000 7f6307221640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630014ac80 con 0x7f630015cce0 2023-10-08T05:53:37.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.883+0000 7f6307221640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6300148770 con 0x7f63000b2c40 2023-10-08T05:53:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.883+0000 7f63057be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63001f5bf0 0x7f6300148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.883+0000 7f63057be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63001f5bf0 0x7f6300148a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59074/0 (socket says 172.21.15.70:59074) 2023-10-08T05:53:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.883+0000 7f63057be640 1 -- 172.21.15.70:0/871008327 learned_addr learned my addr 172.21.15.70:0/871008327 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.884+0000 7f6304fbd640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63000b2c40 0x7f63000cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.884+0000 7f62fffff640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f630015cce0 0x7f63000b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.884+0000 7f63057be640 1 -- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63000b2c40 msgr2=0x7f63000cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.884+0000 7f63057be640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63000b2c40 0x7f63000cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.884+0000 7f63057be640 1 -- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f630015cce0 msgr2=0x7f63000b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.884+0000 7f63057be640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f630015cce0 0x7f63000b6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.884+0000 7f63057be640 1 -- 172.21.15.70:0/871008327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f0009b30 con 0x7f63001f5bf0 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.884+0000 7f6304fbd640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63000b2c40 0x7f63000cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.884+0000 7f63057be640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63001f5bf0 0x7f6300148a30 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009fd0 tx=0x7f62f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.884+0000 7f62fd7fa640 1 -- 172.21.15.70:0/871008327 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8011830 con 0x7f63001f5bf0 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.885+0000 7f62fd7fa640 1 -- 172.21.15.70:0/871008327 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f8011990 con 0x7f63001f5bf0 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.885+0000 7f62fd7fa640 1 -- 172.21.15.70:0/871008327 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8011d20 con 0x7f63001f5bf0 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.885+0000 7f6307221640 1 -- 172.21.15.70:0/871008327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630014ee30 con 0x7f63001f5bf0 2023-10-08T05:53:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.885+0000 7f6307221640 1 -- 172.21.15.70:0/871008327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63001f6280 con 0x7f63001f5bf0 2023-10-08T05:53:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.886+0000 7f6307221640 1 -- 172.21.15.70:0/871008327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630014adf0 con 0x7f63001f5bf0 2023-10-08T05:53:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.887+0000 7f62fd7fa640 1 -- 172.21.15.70:0/871008327 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f62f801b460 con 0x7f63001f5bf0 2023-10-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.888+0000 7f62fd7fa640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62ec06af90 0x7f62ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.888+0000 7f62fd7fa640 1 -- 172.21.15.70:0/871008327 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f62f8089620 con 0x7f63001f5bf0 2023-10-08T05:53:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.888+0000 7f6304fbd640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62ec06af90 0x7f62ec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:37.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.890+0000 7f6304fbd640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62ec06af90 0x7f62ec06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f62f0035000 tx=0x7f62f0030000 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:37.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:37.894+0000 7f62fd7fa640 1 -- 172.21.15.70:0/871008327 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62f8054500 con 0x7f63001f5bf0 2023-10-08T05:53:38.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.083+0000 7f6307221640 1 -- 172.21.15.70:0/871008327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f630004e160 con 0x7f63001f5bf0 2023-10-08T05:53:38.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.090+0000 7f62fd7fa640 1 -- 172.21.15.70:0/871008327 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f62f8057b20 con 0x7f63001f5bf0 2023-10-08T05:53:38.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 -- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62ec06af90 msgr2=0x7f62ec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:38.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62ec06af90 0x7f62ec06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f62f0035000 tx=0x7f62f0030000 comp rx=0 tx=0).stop 2023-10-08T05:53:38.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 -- 172.21.15.70:0/871008327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63001f5bf0 msgr2=0x7f6300148a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:38.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63001f5bf0 0x7f6300148a30 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009fd0 tx=0x7f62f800b8e0 comp rx=0 tx=0).stop 2023-10-08T05:53:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 -- 172.21.15.70:0/871008327 shutdown_connections 2023-10-08T05:53:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62ec06af90 0x7f62ec06d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63001f5bf0 0x7f6300148a30 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f630015cce0 0x7f63000b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 --2- 172.21.15.70:0/871008327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63000b2c40 0x7f63000cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 -- 172.21.15.70:0/871008327 >> 172.21.15.70:0/871008327 conn(0x7f630006e650 msgr2=0x7f630015a660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:38.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 -- 172.21.15.70:0/871008327 shutdown_connections 2023-10-08T05:53:38.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.093+0000 7f6307221640 1 -- 172.21.15.70:0/871008327 wait complete. 2023-10-08T05:53:38.112 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:53:38.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:53:38.222 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:53:38.349 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:53:38.349 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:53:38.349 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:38.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:53:38.408 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:53:38.408 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:38.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:53:38.467 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:53:38.467 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:38.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:53:38.529 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:53:38.529 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:38.590 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:53:38.591 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T05:53:38.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:53:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.816+0000 7f1568c52640 1 Processor -- start 2023-10-08T05:53:39.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.817+0000 7f1568c52640 1 -- start start 2023-10-08T05:53:39.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.818+0000 7f1568c52640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15640c90b0 0x7f15640cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.818+0000 7f1568c52640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156415b2b0 con 0x7f15640cb3b0 2023-10-08T05:53:39.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.818+0000 7f1568c52640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156415b590 con 0x7f156415ab40 2023-10-08T05:53:39.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.818+0000 7f1568c52640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156415bf30 con 0x7f15640c90b0 2023-10-08T05:53:39.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.818+0000 7f1562d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f15640cb3b0 0x7f156415e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60360/0 (socket says 172.21.15.70:60360) 2023-10-08T05:53:39.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1562d9d640 1 -- 172.21.15.70:0/3580066581 learned_addr learned my addr 172.21.15.70:0/3580066581 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:39.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f155bfff640 1 --2- 172.21.15.70:0/3580066581 >> v2:172.21.15.181:3300/0 conn(0x7f15640c90b0 0x7f15640cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477845311 0 0) 0x7f156415b2b0 con 0x7f15640cb3b0 2023-10-08T05:53:39.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1550003520 con 0x7f15640cb3b0 2023-10-08T05:53:39.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 310130020 0 0) 0x7f1550003520 con 0x7f15640cb3b0 2023-10-08T05:53:39.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 >> v2:172.21.15.181:3300/0 conn(0x7f15640c90b0 msgr2=0x7f15640cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:39.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1561d9b640 1 --2- 172.21.15.70:0/3580066581 >> v2:172.21.15.181:3300/0 conn(0x7f15640c90b0 0x7f15640cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 >> v1:172.21.15.181:6789/0 conn(0x7f156415ab40 legacy=0x7f15640ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:39.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156415b870 con 0x7f15640cb3b0 2023-10-08T05:53:39.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f155c0019f0 con 0x7f15640cb3b0 2023-10-08T05:53:39.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 >> v1:172.21.15.70:6789/0 conn(0x7f15640cb3b0 legacy=0x7f156415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.819+0000 7f1561d9b640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1550003970 0x7f1550003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1561d9b640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15500047c0 0x7f1550004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1561d9b640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f155000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156415bf30 con 0x7f1550005180 2023-10-08T05:53:39.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155000c020 con 0x7f15500047c0 2023-10-08T05:53:39.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155000c1e0 con 0x7f1550003970 2023-10-08T05:53:39.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1562d9d640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f155000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1562d9d640 1 -- 172.21.15.70:0/3580066581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1550003970 msgr2=0x7f1550003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:39.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f155bfff640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15500047c0 0x7f1550004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f156259c640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1550003970 0x7f1550003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1562d9d640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1550003970 0x7f1550003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1562d9d640 1 -- 172.21.15.70:0/3580066581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15500047c0 msgr2=0x7f1550004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:39.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1562d9d640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15500047c0 0x7f1550004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.820+0000 7f1562d9d640 1 -- 172.21.15.70:0/3580066581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1550003520 con 0x7f1550005180 2023-10-08T05:53:39.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.821+0000 7f1562d9d640 1 -- 172.21.15.70:0/3580066581 reap_dead start 2023-10-08T05:53:39.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.821+0000 7f1562d9d640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f155000b980 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f155c00a830 tx=0x7f15500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb6f89cce22aab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:39.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.822+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c0072d0 con 0x7f1550005180 2023-10-08T05:53:39.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.822+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c02f070 con 0x7f1550005180 2023-10-08T05:53:39.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.822+0000 7f1561d9b640 1 -- 172.21.15.70:0/3580066581 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c007690 con 0x7f1550005180 2023-10-08T05:53:39.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.822+0000 7f1568c52640 1 -- 172.21.15.70:0/3580066581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 msgr2=0x7f155000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.822+0000 7f1568c52640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f155000b980 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f155c00a830 tx=0x7f15500057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.822+0000 7f1568c52640 1 -- 172.21.15.70:0/3580066581 shutdown_connections 2023-10-08T05:53:39.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.822+0000 7f1568c52640 1 --2- 172.21.15.70:0/3580066581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f155000b980 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.822+0000 7f1568c52640 1 -- 172.21.15.70:0/3580066581 >> 172.21.15.70:0/3580066581 conn(0x7f156406a380 msgr2=0x7f15640b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:39.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.822+0000 7f1568c52640 1 -- 172.21.15.70:0/3580066581 shutdown_connections 2023-10-08T05:53:39.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.822+0000 7f1568c52640 1 -- 172.21.15.70:0/3580066581 wait complete. 2023-10-08T05:53:39.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.824+0000 7f1568c52640 1 Processor -- start 2023-10-08T05:53:39.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.824+0000 7f1568c52640 1 -- start start 2023-10-08T05:53:39.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.825+0000 7f1568c52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f15640c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.825+0000 7f1568c52640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15640ad2c0 0x7f1564155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.825+0000 7f1568c52640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15641f6a80 0x7f15640b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.825+0000 7f1568c52640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15641561e0 con 0x7f1550005180 2023-10-08T05:53:39.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.825+0000 7f1568c52640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15641f65f0 con 0x7f15641f6a80 2023-10-08T05:53:39.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.825+0000 7f1568c52640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15640c8230 con 0x7f15640ad2c0 2023-10-08T05:53:39.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.825+0000 7f156259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f15640c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f156259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f15640c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59084/0 (socket says 172.21.15.70:59084) 2023-10-08T05:53:39.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f156259c640 1 -- 172.21.15.70:0/2420082585 learned_addr learned my addr 172.21.15.70:0/2420082585 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:39.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f156259c640 1 -- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15640ad2c0 msgr2=0x7f1564155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f156259c640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15640ad2c0 0x7f1564155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f156259c640 1 -- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15641f6a80 msgr2=0x7f15640b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f156259c640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15641f6a80 0x7f15640b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f156259c640 1 -- 172.21.15.70:0/2420082585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15640c8510 con 0x7f1550005180 2023-10-08T05:53:39.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f156259c640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f15640c7b50 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f155400d5a0 tx=0x7f155400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:39.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f155affd640 1 -- 172.21.15.70:0/2420082585 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554003610 con 0x7f1550005180 2023-10-08T05:53:39.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f155affd640 1 -- 172.21.15.70:0/2420082585 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1554003770 con 0x7f1550005180 2023-10-08T05:53:39.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f155affd640 1 -- 172.21.15.70:0/2420082585 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554003b00 con 0x7f1550005180 2023-10-08T05:53:39.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.826+0000 7f1568c52640 1 -- 172.21.15.70:0/2420082585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15641f6280 con 0x7f1550005180 2023-10-08T05:53:39.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.827+0000 7f1568c52640 1 -- 172.21.15.70:0/2420082585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15641f6f00 con 0x7f1550005180 2023-10-08T05:53:39.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.828+0000 7f1568c52640 1 -- 172.21.15.70:0/2420082585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156415af20 con 0x7f1550005180 2023-10-08T05:53:39.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.829+0000 7f155affd640 1 -- 172.21.15.70:0/2420082585 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1554003c60 con 0x7f1550005180 2023-10-08T05:53:39.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.830+0000 7f155affd640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f154406afc0 0x7f154406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.830+0000 7f155affd640 1 -- 172.21.15.70:0/2420082585 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f1554089490 con 0x7f1550005180 2023-10-08T05:53:39.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.830+0000 7f155bfff640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f154406afc0 0x7f154406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.831+0000 7f155bfff640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f154406afc0 0x7f154406d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f156415d9b0 tx=0x7f154c008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:39.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:38.834+0000 7f155affd640 1 -- 172.21.15.70:0/2420082585 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1554054280 con 0x7f1550005180 2023-10-08T05:53:39.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.042+0000 7f1568c52640 1 -- 172.21.15.70:0/2420082585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1564067490 con 0x7f1550005180 2023-10-08T05:53:39.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.043+0000 7f155affd640 1 -- 172.21.15.70:0/2420082585 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f15540578a0 con 0x7f1550005180 2023-10-08T05:53:39.116 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:39.149 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5384,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3148555214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":3148555214},{"type":"v1","addr":"172.21.15.181:6837","nonce":3148555214}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5896,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3669053115","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3669053115},{"type":"v1","addr":"172.21.15.181:6836","nonce":3669053115}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":7122,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/3775478637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3775478637},{"type":"v1","addr":"172.21.15.70:6833","nonce":3775478637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":7491,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/588726543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":588726543},{"type":"v1","addr":"172.21.15.70:6835","nonce":588726543}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[]} 2023-10-08T05:53:39.150 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 43 2023-10-08T05:53:39.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 -- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f154406afc0 msgr2=0x7f154406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f154406afc0 0x7f154406d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f156415d9b0 tx=0x7f154c008040 comp rx=0 tx=0).stop 2023-10-08T05:53:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 -- 172.21.15.70:0/2420082585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 msgr2=0x7f15640c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f15640c7b50 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f155400d5a0 tx=0x7f155400da60 comp rx=0 tx=0).stop 2023-10-08T05:53:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 -- 172.21.15.70:0/2420082585 shutdown_connections 2023-10-08T05:53:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f154406afc0 0x7f154406d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15641f6a80 0x7f15640b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15640ad2c0 0x7f1564155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 --2- 172.21.15.70:0/2420082585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1550005180 0x7f15640c7b50 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 -- 172.21.15.70:0/2420082585 >> 172.21.15.70:0/2420082585 conn(0x7f156406a380 msgr2=0x7f15640cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:39.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 -- 172.21.15.70:0/2420082585 shutdown_connections 2023-10-08T05:53:39.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.046+0000 7f1568c52640 1 -- 172.21.15.70:0/2420082585 wait complete. 2023-10-08T05:53:39.153 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T05:53:39.153 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T05:53:39.153 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T05:53:39.153 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:53:39.153 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T05:53:39.153 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T05:53:39.154 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T05:53:39.154 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:53:39.154 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T05:53:39.154 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T05:53:39.154 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T05:53:39.154 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:53:39.155 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T05:53:39.155 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T05:53:39.155 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T05:53:39.155 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:53:39.173 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T05:53:39.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T05:53:39.175 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T05:53:39.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T05:53:39.177 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T05:53:39.177 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T05:53:39.179 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T05:53:39.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T05:53:39.226 INFO:tasks.ceph.mds.b:Started 2023-10-08T05:53:39.226 DEBUG:teuthology.parallel:result is None 2023-10-08T05:53:39.227 INFO:tasks.ceph.mds.c:Started 2023-10-08T05:53:39.227 DEBUG:teuthology.parallel:result is None 2023-10-08T05:53:39.234 INFO:tasks.ceph.mds.d:Started 2023-10-08T05:53:39.234 DEBUG:teuthology.parallel:result is None 2023-10-08T05:53:39.235 INFO:tasks.ceph.mds.a:Started 2023-10-08T05:53:39.235 DEBUG:teuthology.parallel:result is None 2023-10-08T05:53:39.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T05:53:39.385 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:53:39.388 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T05:53:39.394 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:53:39.398 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T05:53:39.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.422+0000 7f55d26d1640 1 Processor -- start 2023-10-08T05:53:39.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.422+0000 7f55d26d1640 1 -- start start 2023-10-08T05:53:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.423+0000 7f55d26d1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55cc0cf2b0 0x7f55cc0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.423+0000 7f55d26d1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55cc0cf690 0x7f55cc0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.423+0000 7f55d26d1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55cc0d1100 0x7f55cc15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.423+0000 7f55d26d1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55cc0cd520 con 0x7f55cc0d1100 2023-10-08T05:53:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.423+0000 7f55d26d1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55cc0d1910 con 0x7f55cc0cf690 2023-10-08T05:53:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.423+0000 7f55d26d1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc0ce080 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.424+0000 7f55d1ed0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55cc0d1100 0x7f55cc15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.424+0000 7f55d1ed0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55cc0d1100 0x7f55cc15c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59120/0 (socket says 172.21.15.70:59120) 2023-10-08T05:53:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.424+0000 7f55d1ed0640 1 -- 172.21.15.70:0/2311178173 learned_addr learned my addr 172.21.15.70:0/2311178173 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.424+0000 7f55d16cf640 1 --2- 172.21.15.70:0/2311178173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55cc0cf2b0 0x7f55cc0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.424+0000 7f55d0ece640 1 --2- 172.21.15.70:0/2311178173 >> v2:172.21.15.181:3300/0 conn(0x7f55cc0cf690 0x7f55cc0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.424+0000 7f55d1ed0640 1 -- 172.21.15.70:0/2311178173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55cc0cf2b0 msgr2=0x7f55cc0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.424+0000 7f55d1ed0640 1 --2- 172.21.15.70:0/2311178173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55cc0cf2b0 0x7f55cc0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.424+0000 7f55d1ed0640 1 -- 172.21.15.70:0/2311178173 >> v2:172.21.15.181:3300/0 conn(0x7f55cc0cf690 msgr2=0x7f55cc0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:39.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.424+0000 7f55d1ed0640 1 --2- 172.21.15.70:0/2311178173 >> v2:172.21.15.181:3300/0 conn(0x7f55cc0cf690 0x7f55cc0ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.424+0000 7f55d1ed0640 1 -- 172.21.15.70:0/2311178173 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55cc0cd750 con 0x7f55cc0d1100 2023-10-08T05:53:39.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.425+0000 7f55d1ed0640 1 --2- 172.21.15.70:0/2311178173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0d1100 0x7f55cc15c820 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f55cc08c2b0 tx=0x7f55c402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd466fa3c18521c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:39.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55c3fff640 1 -- 172.21.15.70:0/2311178173 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c402c9f0 con 0x7f55cc0d1100 2023-10-08T05:53:39.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55c3fff640 1 -- 172.21.15.70:0/2311178173 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c402cb50 con 0x7f55cc0d1100 2023-10-08T05:53:39.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55c3fff640 1 -- 172.21.15.70:0/2311178173 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c40343f0 con 0x7f55cc0d1100 2023-10-08T05:53:39.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55d26d1640 1 -- 172.21.15.70:0/2311178173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0d1100 msgr2=0x7f55cc15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55d26d1640 1 --2- 172.21.15.70:0/2311178173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0d1100 0x7f55cc15c820 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f55cc08c2b0 tx=0x7f55c402be50 comp rx=0 tx=0).stop 2023-10-08T05:53:39.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55d26d1640 1 -- 172.21.15.70:0/2311178173 shutdown_connections 2023-10-08T05:53:39.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55d26d1640 1 --2- 172.21.15.70:0/2311178173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0d1100 0x7f55cc15c820 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55d26d1640 1 --2- 172.21.15.70:0/2311178173 >> v2:172.21.15.181:3300/0 conn(0x7f55cc0cf690 0x7f55cc0ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55d26d1640 1 --2- 172.21.15.70:0/2311178173 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55cc0cf2b0 0x7f55cc0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55d26d1640 1 -- 172.21.15.70:0/2311178173 >> 172.21.15.70:0/2311178173 conn(0x7f55cc09bc40 msgr2=0x7f55cc0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:39.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55d26d1640 1 -- 172.21.15.70:0/2311178173 shutdown_connections 2023-10-08T05:53:39.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.426+0000 7f55d26d1640 1 -- 172.21.15.70:0/2311178173 wait complete. 2023-10-08T05:53:39.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.428+0000 7f55d26d1640 1 Processor -- start 2023-10-08T05:53:39.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.428+0000 7f55d26d1640 1 -- start start 2023-10-08T05:53:39.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.429+0000 7f55d26d1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0cf2b0 0x7f55cc09e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.429+0000 7f55d26d1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55cc0cf690 0x7f55cc150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.429+0000 7f55d26d1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55cc0d1100 0x7f55cc09cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.429+0000 7f55d26d1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc09e020 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.429+0000 7f55d26d1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc09d410 con 0x7f55cc0cf690 2023-10-08T05:53:39.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.429+0000 7f55d26d1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc09d6f0 con 0x7f55cc0d1100 2023-10-08T05:53:39.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.429+0000 7f55d16cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0cf2b0 0x7f55cc09e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.429+0000 7f55d16cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0cf2b0 0x7f55cc09e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59136/0 (socket says 172.21.15.70:59136) 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.429+0000 7f55d16cf640 1 -- 172.21.15.70:0/1606607036 learned_addr learned my addr 172.21.15.70:0/1606607036 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55d0ece640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55cc0cf690 0x7f55cc150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55d1ed0640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55cc0d1100 0x7f55cc09cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55d16cf640 1 -- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55cc0d1100 msgr2=0x7f55cc09cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55d16cf640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55cc0d1100 0x7f55cc09cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55d16cf640 1 -- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55cc0cf690 msgr2=0x7f55cc150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55d16cf640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55cc0cf690 0x7f55cc150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55d16cf640 1 -- 172.21.15.70:0/1606607036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c4009b30 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55d0ece640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55cc0cf690 0x7f55cc150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:53:39.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55d16cf640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0cf2b0 0x7f55cc09e4b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f55c800e870 tx=0x7f55c800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:39.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55c1ffb640 1 -- 172.21.15.70:0/1606607036 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c80094f0 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55c1ffb640 1 -- 172.21.15.70:0/1606607036 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c800c940 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55c1ffb640 1 -- 172.21.15.70:0/1606607036 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c800ccf0 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.430+0000 7f55d26d1640 1 -- 172.21.15.70:0/1606607036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc09d9d0 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.431+0000 7f55d26d1640 1 -- 172.21.15.70:0/1606607036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc1510a0 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.432+0000 7f55d26d1640 1 -- 172.21.15.70:0/1606607036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55cc074150 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.432+0000 7f55c1ffb640 1 -- 172.21.15.70:0/1606607036 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f55c8013470 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.433+0000 7f55c1ffb640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f55b006af90 0x7f55b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.433+0000 7f55c1ffb640 1 -- 172.21.15.70:0/1606607036 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6038+0+0 (secure 0 0 0) 0x7f55c8089b30 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.434+0000 7f55d0ece640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f55b006af90 0x7f55b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.435+0000 7f55d0ece640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f55b006af90 0x7f55b006d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0ccf20 tx=0x7f55bc008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:39.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.438+0000 7f55c1ffb640 1 -- 172.21.15.70:0/1606607036 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55c8054850 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.640+0000 7f55d26d1640 1 -- 172.21.15.70:0/1606607036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f55cc0d0070 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.657+0000 7f55c1ffb640 1 -- 172.21.15.70:0/1606607036 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v112) v1 ==== 97+0+0 (secure 0 0 0) 0x7f55c8021020 con 0x7f55cc0cf2b0 2023-10-08T05:53:39.725 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T05:53:39.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.661+0000 7f55d26d1640 1 -- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f55b006af90 msgr2=0x7f55b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.661+0000 7f55d26d1640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f55b006af90 0x7f55b006d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f55cc0ccf20 tx=0x7f55bc008040 comp rx=0 tx=0).stop 2023-10-08T05:53:39.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.661+0000 7f55d26d1640 1 -- 172.21.15.70:0/1606607036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0cf2b0 msgr2=0x7f55cc09e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.661+0000 7f55d26d1640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0cf2b0 0x7f55cc09e4b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f55c800e870 tx=0x7f55c800ed30 comp rx=0 tx=0).stop 2023-10-08T05:53:39.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.661+0000 7f55d26d1640 1 -- 172.21.15.70:0/1606607036 shutdown_connections 2023-10-08T05:53:39.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.661+0000 7f55d26d1640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f55b006af90 0x7f55b006d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.661+0000 7f55d26d1640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55cc0d1100 0x7f55cc09cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.661+0000 7f55d26d1640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55cc0cf690 0x7f55cc150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.661+0000 7f55d26d1640 1 --2- 172.21.15.70:0/1606607036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55cc0cf2b0 0x7f55cc09e4b0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.662+0000 7f55d26d1640 1 -- 172.21.15.70:0/1606607036 >> 172.21.15.70:0/1606607036 conn(0x7f55cc09bc40 msgr2=0x7f55cc15fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:39.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.662+0000 7f55d26d1640 1 -- 172.21.15.70:0/1606607036 shutdown_connections 2023-10-08T05:53:39.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.662+0000 7f55d26d1640 1 -- 172.21.15.70:0/1606607036 wait complete. 2023-10-08T05:53:39.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T05:53:39.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.942+0000 7f7d045b6640 1 Processor -- start 2023-10-08T05:53:39.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.942+0000 7f7d045b6640 1 -- start start 2023-10-08T05:53:39.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.944+0000 7f7d045b6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7cfc159ac0 0x7f7cfc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.944+0000 7f7d045b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc0709e0 0x7f7cfc0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.944+0000 7f7d045b6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc06dee0 con 0x7f7cfc0ad790 2023-10-08T05:53:39.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.944+0000 7f7d045b6640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc15d850 con 0x7f7cfc159ac0 2023-10-08T05:53:39.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.944+0000 7f7d045b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15db30 con 0x7f7cfc0709e0 2023-10-08T05:53:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.944+0000 7f7d02b53640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc0709e0 0x7f7cfc0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.945+0000 7f7d02b53640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc0709e0 0x7f7cfc0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:42264/0 (socket says 172.21.15.70:42264) 2023-10-08T05:53:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.945+0000 7f7d02b53640 1 -- 172.21.15.70:0/1850606306 learned_addr learned my addr 172.21.15.70:0/1850606306 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.945+0000 7f7d02352640 1 --2- 172.21.15.70:0/1850606306 >> v2:172.21.15.181:3300/0 conn(0x7f7cfc159ac0 0x7f7cfc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.945+0000 7f7d02b53640 1 -- 172.21.15.70:0/1850606306 >> v2:172.21.15.181:3300/0 conn(0x7f7cfc159ac0 msgr2=0x7f7cfc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.945+0000 7f7d02b53640 1 --2- 172.21.15.70:0/1850606306 >> v2:172.21.15.181:3300/0 conn(0x7f7cfc159ac0 0x7f7cfc06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.945+0000 7f7d02b53640 1 -- 172.21.15.70:0/1850606306 >> v1:172.21.15.181:6789/0 conn(0x7f7cfc0ad790 legacy=0x7f7cfc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.945+0000 7f7d02b53640 1 -- 172.21.15.70:0/1850606306 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc15de10 con 0x7f7cfc0709e0 2023-10-08T05:53:39.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.945+0000 7f7d01350640 1 -- 172.21.15.70:0/1850606306 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325047993 0 0) 0x7f7cfc06dee0 con 0x7f7cfc0ad790 2023-10-08T05:53:39.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.947+0000 7f7d02b53640 1 --2- 172.21.15.70:0/1850606306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc0709e0 0x7f7cfc0c8dd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009870 tx=0x7f7cf402db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7641ea0bea10d70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:39.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.947+0000 7f7d01350640 1 -- 172.21.15.70:0/1850606306 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf402e940 con 0x7f7cfc0709e0 2023-10-08T05:53:39.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.947+0000 7f7d01350640 1 -- 172.21.15.70:0/1850606306 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf402eaa0 con 0x7f7cfc0709e0 2023-10-08T05:53:39.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.947+0000 7f7d01350640 1 -- 172.21.15.70:0/1850606306 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf402ee30 con 0x7f7cfc0709e0 2023-10-08T05:53:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.948+0000 7f7d045b6640 1 -- 172.21.15.70:0/1850606306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc0709e0 msgr2=0x7f7cfc0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.948+0000 7f7d045b6640 1 --2- 172.21.15.70:0/1850606306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc0709e0 0x7f7cfc0c8dd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009870 tx=0x7f7cf402db90 comp rx=0 tx=0).stop 2023-10-08T05:53:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.948+0000 7f7d045b6640 1 -- 172.21.15.70:0/1850606306 shutdown_connections 2023-10-08T05:53:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.948+0000 7f7d045b6640 1 --2- 172.21.15.70:0/1850606306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc0709e0 0x7f7cfc0c8dd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.948+0000 7f7d045b6640 1 --2- 172.21.15.70:0/1850606306 >> v2:172.21.15.181:3300/0 conn(0x7f7cfc159ac0 0x7f7cfc06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.948+0000 7f7d045b6640 1 -- 172.21.15.70:0/1850606306 >> 172.21.15.70:0/1850606306 conn(0x7f7cfc06a380 msgr2=0x7f7cfc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.948+0000 7f7d045b6640 1 -- 172.21.15.70:0/1850606306 shutdown_connections 2023-10-08T05:53:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.948+0000 7f7d045b6640 1 -- 172.21.15.70:0/1850606306 wait complete. 2023-10-08T05:53:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.950+0000 7f7d045b6640 1 Processor -- start 2023-10-08T05:53:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.950+0000 7f7d045b6640 1 -- start start 2023-10-08T05:53:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.951+0000 7f7d045b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc0709e0 0x7f7cfc13e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.951+0000 7f7d045b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cfc0ad790 0x7f7cfc141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.951+0000 7f7d045b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc159ac0 0x7f7cfc141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.951+0000 7f7d045b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0b4300 con 0x7f7cfc0709e0 2023-10-08T05:53:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.951+0000 7f7d045b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0b4020 con 0x7f7cfc0ad790 2023-10-08T05:53:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.951+0000 7f7d045b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc153c40 con 0x7f7cfc159ac0 2023-10-08T05:53:39.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d02352640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc0709e0 0x7f7cfc13e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d02352640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc0709e0 0x7f7cfc13e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59142/0 (socket says 172.21.15.70:59142) 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d02352640 1 -- 172.21.15.70:0/1743870533 learned_addr learned my addr 172.21.15.70:0/1743870533 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d02b53640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc159ac0 0x7f7cfc141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d01b51640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cfc0ad790 0x7f7cfc141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d02352640 1 -- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc159ac0 msgr2=0x7f7cfc141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d02352640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc159ac0 0x7f7cfc141890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d02352640 1 -- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cfc0ad790 msgr2=0x7f7cfc141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d02352640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cfc0ad790 0x7f7cfc141350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d02352640 1 -- 172.21.15.70:0/1743870533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf40094d0 con 0x7f7cfc0709e0 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.952+0000 7f7d02352640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc0709e0 0x7f7cfc13e8c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800efc0 tx=0x7f7cf800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.953+0000 7f7cf2ffd640 1 -- 172.21.15.70:0/1743870533 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8009b80 con 0x7f7cfc0709e0 2023-10-08T05:53:39.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.953+0000 7f7cf2ffd640 1 -- 172.21.15.70:0/1743870533 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf800f040 con 0x7f7cfc0709e0 2023-10-08T05:53:39.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.953+0000 7f7cf2ffd640 1 -- 172.21.15.70:0/1743870533 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf80146a0 con 0x7f7cfc0709e0 2023-10-08T05:53:39.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.953+0000 7f7d045b6640 1 -- 172.21.15.70:0/1743870533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc153f20 con 0x7f7cfc0709e0 2023-10-08T05:53:39.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.953+0000 7f7d045b6640 1 -- 172.21.15.70:0/1743870533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc13ee00 con 0x7f7cfc0709e0 2023-10-08T05:53:39.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.954+0000 7f7d045b6640 1 -- 172.21.15.70:0/1743870533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc067490 con 0x7f7cfc0709e0 2023-10-08T05:53:39.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.955+0000 7f7cf2ffd640 1 -- 172.21.15.70:0/1743870533 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7cf8014800 con 0x7f7cfc0709e0 2023-10-08T05:53:39.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.956+0000 7f7cf2ffd640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7ce006af90 0x7f7ce006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:39.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.956+0000 7f7cf2ffd640 1 -- 172.21.15.70:0/1743870533 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f7cf808a290 con 0x7f7cfc0709e0 2023-10-08T05:53:39.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.957+0000 7f7d01b51640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7ce006af90 0x7f7ce006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:39.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.958+0000 7f7d01b51640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7ce006af90 0x7f7ce006d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7cec000f60 tx=0x7f7cec009040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:39.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:39.960+0000 7f7cf2ffd640 1 -- 172.21.15.70:0/1743870533 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7cf8055260 con 0x7f7cfc0709e0 2023-10-08T05:53:39.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:53:39.995 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:53:40.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.226+0000 7f7d045b6640 1 -- 172.21.15.70:0/1743870533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7cfc0b0ff0 con 0x7f7cfc0709e0 2023-10-08T05:53:40.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.227+0000 7f7cf2ffd640 1 -- 172.21.15.70:0/1743870533 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7cf8058880 con 0x7f7cfc0709e0 2023-10-08T05:53:40.228 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:40.228 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:53:40.228 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T05:53:40.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.229 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T05:53:40.229 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T05:53:40.229 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.229 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:40.229 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:53:40.229 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:53:40.229 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.230 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T05:53:40.231 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:40.232 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T05:53:40.234 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:40.235 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T05:53:40.237 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T05:53:40.238 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.240 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T05:53:40.241 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:53:40.242 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:40.243 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T05:53:40.244 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:53:40.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 -- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7ce006af90 msgr2=0x7f7ce006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7ce006af90 0x7f7ce006d450 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7cec000f60 tx=0x7f7cec009040 comp rx=0 tx=0).stop 2023-10-08T05:53:40.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 -- 172.21.15.70:0/1743870533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc0709e0 msgr2=0x7f7cfc13e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc0709e0 0x7f7cfc13e8c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800efc0 tx=0x7f7cf800c490 comp rx=0 tx=0).stop 2023-10-08T05:53:40.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 -- 172.21.15.70:0/1743870533 shutdown_connections 2023-10-08T05:53:40.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f7ce006af90 0x7f7ce006d450 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cfc159ac0 0x7f7cfc141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cfc0ad790 0x7f7cfc141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 --2- 172.21.15.70:0/1743870533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc0709e0 0x7f7cfc13e8c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 -- 172.21.15.70:0/1743870533 >> 172.21.15.70:0/1743870533 conn(0x7f7cfc06a380 msgr2=0x7f7cfc0cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:40.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.229+0000 7f7d045b6640 1 -- 172.21.15.70:0/1743870533 shutdown_connections 2023-10-08T05:53:40.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.230+0000 7f7d045b6640 1 -- 172.21.15.70:0/1743870533 wait complete. 2023-10-08T05:53:40.249 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:40.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.462+0000 7fa4d05f0640 1 Processor -- start 2023-10-08T05:53:40.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.462+0000 7fa4d05f0640 1 -- start start 2023-10-08T05:53:40.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.463+0000 7fa4d05f0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4c815ab40 0x7fa4c80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.463+0000 7fa4d05f0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa4c80c90b0 0x7fa4c80adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.463+0000 7fa4d05f0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4c80cae70 0x7fa4c80cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.463+0000 7fa4d05f0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c815bf30 con 0x7fa4c80cae70 2023-10-08T05:53:40.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.463+0000 7fa4d05f0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c80cb840 con 0x7fa4c80c90b0 2023-10-08T05:53:40.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.463+0000 7fa4d05f0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c815b2b0 con 0x7fa4c815ab40 2023-10-08T05:53:40.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.464+0000 7fa4ceb8d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4c80cae70 0x7fa4c80cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.464+0000 7fa4ceb8d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4c80cae70 0x7fa4c80cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59156/0 (socket says 172.21.15.70:59156) 2023-10-08T05:53:40.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.464+0000 7fa4ceb8d640 1 -- 172.21.15.70:0/1812518614 learned_addr learned my addr 172.21.15.70:0/1812518614 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:40.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.464+0000 7fa4ce38c640 1 --2- 172.21.15.70:0/1812518614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4c815ab40 0x7fa4c80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.464+0000 7fa4cdb8b640 1 --2- 172.21.15.70:0/1812518614 >> v2:172.21.15.181:3300/0 conn(0x7fa4c80c90b0 0x7fa4c80adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.464+0000 7fa4ceb8d640 1 -- 172.21.15.70:0/1812518614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4c815ab40 msgr2=0x7fa4c80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:40.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.464+0000 7fa4ceb8d640 1 --2- 172.21.15.70:0/1812518614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4c815ab40 0x7fa4c80ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.464+0000 7fa4ceb8d640 1 -- 172.21.15.70:0/1812518614 >> v2:172.21.15.181:3300/0 conn(0x7fa4c80c90b0 msgr2=0x7fa4c80adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:40.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.464+0000 7fa4ceb8d640 1 --2- 172.21.15.70:0/1812518614 >> v2:172.21.15.181:3300/0 conn(0x7fa4c80c90b0 0x7fa4c80adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.464+0000 7fa4ceb8d640 1 -- 172.21.15.70:0/1812518614 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c815b590 con 0x7fa4c80cae70 2023-10-08T05:53:40.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.465+0000 7fa4ceb8d640 1 --2- 172.21.15.70:0/1812518614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c80cae70 0x7fa4c80cb300 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa4c806e470 tx=0x7fa4c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ea781b873f2ebe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:40.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4cd38a640 1 -- 172.21.15.70:0/1812518614 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c002ca70 con 0x7fa4c80cae70 2023-10-08T05:53:40.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4cd38a640 1 -- 172.21.15.70:0/1812518614 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c002cbd0 con 0x7fa4c80cae70 2023-10-08T05:53:40.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4cd38a640 1 -- 172.21.15.70:0/1812518614 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c00343f0 con 0x7fa4c80cae70 2023-10-08T05:53:40.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1812518614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c80cae70 msgr2=0x7fa4c80cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4d05f0640 1 --2- 172.21.15.70:0/1812518614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c80cae70 0x7fa4c80cb300 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fa4c806e470 tx=0x7fa4c002bed0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1812518614 shutdown_connections 2023-10-08T05:53:40.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4d05f0640 1 --2- 172.21.15.70:0/1812518614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c80cae70 0x7fa4c80cb300 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4d05f0640 1 --2- 172.21.15.70:0/1812518614 >> v2:172.21.15.181:3300/0 conn(0x7fa4c80c90b0 0x7fa4c80adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4d05f0640 1 --2- 172.21.15.70:0/1812518614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4c815ab40 0x7fa4c80ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1812518614 >> 172.21.15.70:0/1812518614 conn(0x7fa4c806a380 msgr2=0x7fa4c80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:40.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1812518614 shutdown_connections 2023-10-08T05:53:40.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.466+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1812518614 wait complete. 2023-10-08T05:53:40.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.468+0000 7fa4d05f0640 1 Processor -- start 2023-10-08T05:53:40.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.468+0000 7fa4d05f0640 1 -- start start 2023-10-08T05:53:40.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.469+0000 7fa4d05f0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4c80c90b0 0x7fa4c80b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.469+0000 7fa4d05f0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4c80cae70 0x7fa4c80b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.469+0000 7fa4d05f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c815ab40 0x7fa4c8154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.469+0000 7fa4d05f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c8147a30 con 0x7fa4c815ab40 2023-10-08T05:53:40.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.469+0000 7fa4d05f0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c74d0 con 0x7fa4c80c90b0 2023-10-08T05:53:40.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.469+0000 7fa4d05f0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c77b0 con 0x7fa4c80cae70 2023-10-08T05:53:40.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.469+0000 7fa4ceb8d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c815ab40 0x7fa4c8154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4ceb8d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c815ab40 0x7fa4c8154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59164/0 (socket says 172.21.15.70:59164) 2023-10-08T05:53:40.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4ceb8d640 1 -- 172.21.15.70:0/1203363384 learned_addr learned my addr 172.21.15.70:0/1203363384 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:40.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4ce38c640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4c80c90b0 0x7fa4c80b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4cdb8b640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4c80cae70 0x7fa4c80b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4ceb8d640 1 -- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4c80cae70 msgr2=0x7fa4c80b3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4ceb8d640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4c80cae70 0x7fa4c80b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4ceb8d640 1 -- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4c80c90b0 msgr2=0x7fa4c80b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4ceb8d640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4c80c90b0 0x7fa4c80b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4ceb8d640 1 -- 172.21.15.70:0/1203363384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c0009b30 con 0x7fa4c815ab40 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4ce38c640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4c80c90b0 0x7fa4c80b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.470+0000 7fa4ceb8d640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c815ab40 0x7fa4c8154bb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009f40 tx=0x7fa4c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.471+0000 7fa4beffd640 1 -- 172.21.15.70:0/1203363384 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c003e450 con 0x7fa4c815ab40 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.471+0000 7fa4beffd640 1 -- 172.21.15.70:0/1203363384 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c003e5b0 con 0x7fa4c815ab40 2023-10-08T05:53:40.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.471+0000 7fa4beffd640 1 -- 172.21.15.70:0/1203363384 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c003e960 con 0x7fa4c815ab40 2023-10-08T05:53:40.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.471+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1203363384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c80c7a90 con 0x7fa4c815ab40 2023-10-08T05:53:40.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.471+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1203363384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c80c8290 con 0x7fa4c815ab40 2023-10-08T05:53:40.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.472+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1203363384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c80c7fb0 con 0x7fa4c815ab40 2023-10-08T05:53:40.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.473+0000 7fa4beffd640 1 -- 172.21.15.70:0/1203363384 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa4c003eac0 con 0x7fa4c815ab40 2023-10-08T05:53:40.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.474+0000 7fa4beffd640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa4ac06af90 0x7fa4ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.474+0000 7fa4beffd640 1 -- 172.21.15.70:0/1203363384 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fa4c00ac730 con 0x7fa4c815ab40 2023-10-08T05:53:40.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.475+0000 7fa4ce38c640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa4ac06af90 0x7fa4ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.476+0000 7fa4ce38c640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa4ac06af90 0x7fa4ac06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4008890 tx=0x7fa4c4008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:40.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.478+0000 7fa4beffd640 1 -- 172.21.15.70:0/1203363384 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa4c80c7fb0 con 0x7fa4c815ab40 2023-10-08T05:53:40.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.672+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1203363384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4c80c7fb0 con 0x7fa4c815ab40 2023-10-08T05:53:40.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.673+0000 7fa4beffd640 1 -- 172.21.15.70:0/1203363384 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+13130 (secure 0 0 0) 0x7fa4c80c7fb0 con 0x7fa4c815ab40 2023-10-08T05:53:40.675 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:40.675 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":114,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:39.671198+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:40.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.676+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa4ac06af90 msgr2=0x7fa4ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.676+0000 7fa4d05f0640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa4ac06af90 0x7fa4ac06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4008890 tx=0x7fa4c4008040 comp rx=0 tx=0).stop 2023-10-08T05:53:40.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.676+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1203363384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c815ab40 msgr2=0x7fa4c8154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.676+0000 7fa4d05f0640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c815ab40 0x7fa4c8154bb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009f40 tx=0x7fa4c0004a40 comp rx=0 tx=0).stop 2023-10-08T05:53:40.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.677+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1203363384 shutdown_connections 2023-10-08T05:53:40.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.677+0000 7fa4d05f0640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa4ac06af90 0x7fa4ac06d450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.677+0000 7fa4d05f0640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4c815ab40 0x7fa4c8154bb0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.677+0000 7fa4d05f0640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa4c80cae70 0x7fa4c80b3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.677+0000 7fa4d05f0640 1 --2- 172.21.15.70:0/1203363384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa4c80c90b0 0x7fa4c80b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.677+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1203363384 >> 172.21.15.70:0/1203363384 conn(0x7fa4c806a380 msgr2=0x7fa4c80ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:40.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.677+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1203363384 shutdown_connections 2023-10-08T05:53:40.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.677+0000 7fa4d05f0640 1 -- 172.21.15.70:0/1203363384 wait complete. 2023-10-08T05:53:40.695 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:53:40.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:53:40.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.904+0000 7f82361d7640 1 Processor -- start 2023-10-08T05:53:40.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.904+0000 7f82361d7640 1 -- start start 2023-10-08T05:53:40.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.905+0000 7f82361d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f82300cb880 0x7f82300b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.905+0000 7f82361d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823015d440 con 0x7f82300b4070 2023-10-08T05:53:40.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.906+0000 7f82361d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823015d690 con 0x7f82301432f0 2023-10-08T05:53:40.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.906+0000 7f82361d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82301436d0 con 0x7f82300cb880 2023-10-08T05:53:40.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.906+0000 7f822ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f82300b4070 0x7f8230159e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60378/0 (socket says 172.21.15.70:60378) 2023-10-08T05:53:40.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.906+0000 7f822ffff640 1 -- 172.21.15.70:0/350332928 learned_addr learned my addr 172.21.15.70:0/350332928 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:40.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.906+0000 7f822effd640 1 --2- 172.21.15.70:0/350332928 >> v2:172.21.15.181:3300/0 conn(0x7f82300cb880 0x7f82300b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.906+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384493140 0 0) 0x7f823015d440 con 0x7f82300b4070 2023-10-08T05:53:40.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.906+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f821c003520 con 0x7f82300b4070 2023-10-08T05:53:40.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4188618044 0 0) 0x7f821c003520 con 0x7f82300b4070 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 >> v2:172.21.15.181:3300/0 conn(0x7f82300cb880 msgr2=0x7f82300b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 --2- 172.21.15.70:0/350332928 >> v2:172.21.15.181:3300/0 conn(0x7f82300cb880 0x7f82300b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 >> v1:172.21.15.181:6789/0 conn(0x7f82301432f0 legacy=0x7f82300cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823015d970 con 0x7f82300b4070 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f8228001670 con 0x7f82300b4070 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 >> v1:172.21.15.70:6789/0 conn(0x7f82300b4070 legacy=0x7f8230159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f821c003970 0x7f821c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f821c005180 0x7f821c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82301436d0 con 0x7f821c0047c0 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c00c120 con 0x7f821c003970 2023-10-08T05:53:40.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c00c2e0 con 0x7f821c005180 2023-10-08T05:53:40.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822effd640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822f7fe640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f821c003970 0x7f821c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822ffff640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f821c005180 0x7f821c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822effd640 1 -- 172.21.15.70:0/350332928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f821c005180 msgr2=0x7f821c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:40.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822effd640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f821c005180 0x7f821c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.907+0000 7f822effd640 1 -- 172.21.15.70:0/350332928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f821c003970 msgr2=0x7f821c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:40.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.908+0000 7f822effd640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f821c003970 0x7f821c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.908+0000 7f822ffff640 1 -- 172.21.15.70:0/350332928 reap_dead start 2023-10-08T05:53:40.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.908+0000 7f822effd640 1 -- 172.21.15.70:0/350332928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c003520 con 0x7f821c0047c0 2023-10-08T05:53:40.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.908+0000 7f822f7fe640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f821c003970 0x7f821c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:40.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.909+0000 7f822effd640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8218009fc0 tx=0x7f821c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0aabdaa306c37ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:40.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.909+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821802cc20 con 0x7f821c0047c0 2023-10-08T05:53:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.909+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821800f040 con 0x7f821c0047c0 2023-10-08T05:53:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.909+0000 7f822e7fc640 1 -- 172.21.15.70:0/350332928 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218034440 con 0x7f821c0047c0 2023-10-08T05:53:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.910+0000 7f82361d7640 1 -- 172.21.15.70:0/350332928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f821c0047c0 msgr2=0x7f821c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.910+0000 7f82361d7640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8218009fc0 tx=0x7f821c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.910+0000 7f82361d7640 1 -- 172.21.15.70:0/350332928 shutdown_connections 2023-10-08T05:53:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.910+0000 7f82361d7640 1 --2- 172.21.15.70:0/350332928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f821c0047c0 0x7f821c004c40 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.910+0000 7f82361d7640 1 -- 172.21.15.70:0/350332928 >> 172.21.15.70:0/350332928 conn(0x7f8230069d50 msgr2=0x7f823006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.910+0000 7f82361d7640 1 -- 172.21.15.70:0/350332928 shutdown_connections 2023-10-08T05:53:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.910+0000 7f82361d7640 1 -- 172.21.15.70:0/350332928 wait complete. 2023-10-08T05:53:40.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.912+0000 7f82361d7640 1 Processor -- start 2023-10-08T05:53:40.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.912+0000 7f82361d7640 1 -- start start 2023-10-08T05:53:40.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f82361d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f821c0047c0 0x7f823015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f82361d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8230159ce0 0x7f823015a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f82361d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82300a8c40 0x7f82300a90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f82361d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823015b4d0 con 0x7f8230159ce0 2023-10-08T05:53:40.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f82361d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82300a9600 con 0x7f821c0047c0 2023-10-08T05:53:40.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f82361d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82301f2200 con 0x7f82300a8c40 2023-10-08T05:53:40.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f822effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8230159ce0 0x7f823015a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f822effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8230159ce0 0x7f823015a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59182/0 (socket says 172.21.15.70:59182) 2023-10-08T05:53:40.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f822effd640 1 -- 172.21.15.70:0/866160576 learned_addr learned my addr 172.21.15.70:0/866160576 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:40.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f822f7fe640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f821c0047c0 0x7f823015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f822effd640 1 -- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82300a8c40 msgr2=0x7f82300a90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f822effd640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82300a8c40 0x7f82300a90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f822effd640 1 -- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f821c0047c0 msgr2=0x7f823015bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:40.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f822effd640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f821c0047c0 0x7f823015bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:40.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.913+0000 7f822effd640 1 -- 172.21.15.70:0/866160576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823015a850 con 0x7f8230159ce0 2023-10-08T05:53:40.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.914+0000 7f822effd640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8230159ce0 0x7f823015a160 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f821800e230 tx=0x7f8218009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:40.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.914+0000 7f8227fff640 1 -- 172.21.15.70:0/866160576 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218034c50 con 0x7f8230159ce0 2023-10-08T05:53:40.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.914+0000 7f8227fff640 1 -- 172.21.15.70:0/866160576 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8218034db0 con 0x7f8230159ce0 2023-10-08T05:53:40.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.914+0000 7f82361d7640 1 -- 172.21.15.70:0/866160576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823015ab30 con 0x7f8230159ce0 2023-10-08T05:53:40.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.914+0000 7f8227fff640 1 -- 172.21.15.70:0/866160576 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8218003bb0 con 0x7f8230159ce0 2023-10-08T05:53:40.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.914+0000 7f82361d7640 1 -- 172.21.15.70:0/866160576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823015b0f0 con 0x7f8230159ce0 2023-10-08T05:53:40.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.915+0000 7f82361d7640 1 -- 172.21.15.70:0/866160576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823015b2b0 con 0x7f8230159ce0 2023-10-08T05:53:40.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.916+0000 7f8227fff640 1 -- 172.21.15.70:0/866160576 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8218040450 con 0x7f8230159ce0 2023-10-08T05:53:40.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.917+0000 7f8227fff640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f821006afc0 0x7f821006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:40.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.917+0000 7f8227fff640 1 -- 172.21.15.70:0/866160576 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f82180abee0 con 0x7f8230159ce0 2023-10-08T05:53:40.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.918+0000 7f822f7fe640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f821006afc0 0x7f821006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:40.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.919+0000 7f822f7fe640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f821006afc0 0x7f821006d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f823004f940 tx=0x7f8220009040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:40.921+0000 7f8227fff640 1 -- 172.21.15.70:0/866160576 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8218076fc0 con 0x7f8230159ce0 2023-10-08T05:53:41.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.113+0000 7f82361d7640 1 -- 172.21.15.70:0/866160576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f823015b610 con 0x7f8230159ce0 2023-10-08T05:53:41.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.114+0000 7f8227fff640 1 -- 172.21.15.70:0/866160576 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f823015b610 con 0x7f8230159ce0 2023-10-08T05:53:41.115 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:53:41.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.116+0000 7f82361d7640 1 -- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f821006afc0 msgr2=0x7f821006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.116+0000 7f82361d7640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f821006afc0 0x7f821006d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f823004f940 tx=0x7f8220009040 comp rx=0 tx=0).stop 2023-10-08T05:53:41.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.116+0000 7f82361d7640 1 -- 172.21.15.70:0/866160576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8230159ce0 msgr2=0x7f823015a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.116+0000 7f82361d7640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8230159ce0 0x7f823015a160 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f821800e230 tx=0x7f8218009750 comp rx=0 tx=0).stop 2023-10-08T05:53:41.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.116+0000 7f82361d7640 1 -- 172.21.15.70:0/866160576 shutdown_connections 2023-10-08T05:53:41.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.116+0000 7f82361d7640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f821006afc0 0x7f821006d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.116+0000 7f82361d7640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82300a8c40 0x7f82300a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.116+0000 7f82361d7640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8230159ce0 0x7f823015a160 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.116+0000 7f82361d7640 1 --2- 172.21.15.70:0/866160576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f821c0047c0 0x7f823015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.116+0000 7f82361d7640 1 -- 172.21.15.70:0/866160576 >> 172.21.15.70:0/866160576 conn(0x7f8230069d50 msgr2=0x7f82300cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.117+0000 7f82361d7640 1 -- 172.21.15.70:0/866160576 shutdown_connections 2023-10-08T05:53:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.117+0000 7f82361d7640 1 -- 172.21.15.70:0/866160576 wait complete. 2023-10-08T05:53:41.137 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T05:53:41.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.347+0000 7fe25e3f0640 1 Processor -- start 2023-10-08T05:53:41.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.347+0000 7fe25e3f0640 1 -- start start 2023-10-08T05:53:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.349+0000 7fe25e3f0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 0x7fe2580ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.349+0000 7fe25e3f0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe2580c90b0 0x7fe2580adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.349+0000 7fe25e3f0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe25815b2b0 con 0x7fe2580cae70 2023-10-08T05:53:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.349+0000 7fe25e3f0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25815bf30 con 0x7fe2580c90b0 2023-10-08T05:53:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.349+0000 7fe25e3f0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25815f7a0 con 0x7fe25815ab40 2023-10-08T05:53:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.349+0000 7fe24ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe2580c90b0 0x7fe2580adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.349+0000 7fe24ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe2580c90b0 0x7fe2580adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59190/0 (socket says 172.21.15.70:59190) 2023-10-08T05:53:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.349+0000 7fe24ffff640 1 -- 172.21.15.70:0/4168741841 learned_addr learned my addr 172.21.15.70:0/4168741841 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.350+0000 7fe257fff640 1 --2- 172.21.15.70:0/4168741841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 0x7fe2580ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.350+0000 7fe24ffff640 1 -- 172.21.15.70:0/4168741841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 msgr2=0x7fe2580ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:41.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.350+0000 7fe24ffff640 1 --2- 172.21.15.70:0/4168741841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 0x7fe2580ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.350+0000 7fe24ffff640 1 -- 172.21.15.70:0/4168741841 >> v1:172.21.15.181:6789/0 conn(0x7fe2580cae70 legacy=0x7fe2580702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.350+0000 7fe24ffff640 1 -- 172.21.15.70:0/4168741841 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25815b590 con 0x7fe2580c90b0 2023-10-08T05:53:41.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.351+0000 7fe24ffff640 1 --2- 172.21.15.70:0/4168741841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 0x7fe2580adc60 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe240009870 tx=0x7fe24002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156b12b1fb84d513 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:41.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.352+0000 7fe2577fe640 1 -- 172.21.15.70:0/4168741841 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240030840 con 0x7fe2580c90b0 2023-10-08T05:53:41.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.352+0000 7fe2577fe640 1 -- 172.21.15.70:0/4168741841 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2400309a0 con 0x7fe2580c90b0 2023-10-08T05:53:41.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.352+0000 7fe2577fe640 1 -- 172.21.15.70:0/4168741841 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240030d30 con 0x7fe2580c90b0 2023-10-08T05:53:41.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.353+0000 7fe25e3f0640 1 -- 172.21.15.70:0/4168741841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 msgr2=0x7fe2580adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.353+0000 7fe25e3f0640 1 --2- 172.21.15.70:0/4168741841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 0x7fe2580adc60 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe240009870 tx=0x7fe24002b980 comp rx=0 tx=0).stop 2023-10-08T05:53:41.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.353+0000 7fe25e3f0640 1 -- 172.21.15.70:0/4168741841 shutdown_connections 2023-10-08T05:53:41.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.353+0000 7fe25e3f0640 1 --2- 172.21.15.70:0/4168741841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 0x7fe2580adc60 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.353+0000 7fe25e3f0640 1 --2- 172.21.15.70:0/4168741841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 0x7fe2580ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.353+0000 7fe25e3f0640 1 -- 172.21.15.70:0/4168741841 >> 172.21.15.70:0/4168741841 conn(0x7fe25806a380 msgr2=0x7fe2580b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:41.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.353+0000 7fe25e3f0640 1 -- 172.21.15.70:0/4168741841 shutdown_connections 2023-10-08T05:53:41.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.353+0000 7fe25e3f0640 1 -- 172.21.15.70:0/4168741841 wait complete. 2023-10-08T05:53:41.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.355+0000 7fe25e3f0640 1 Processor -- start 2023-10-08T05:53:41.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.355+0000 7fe25e3f0640 1 -- start start 2023-10-08T05:53:41.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe25e3f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 0x7fe258159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe25e3f0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2580cae70 0x7fe258158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe25e3f0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 0x7fe25815c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe25e3f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2581f2040 con 0x7fe2580c90b0 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe25e3f0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe258158670 con 0x7fe2580cae70 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe25e3f0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25815cc40 con 0x7fe25815ab40 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe257fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 0x7fe258159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe257fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 0x7fe258159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59200/0 (socket says 172.21.15.70:59200) 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe257fff640 1 -- 172.21.15.70:0/870826810 learned_addr learned my addr 172.21.15.70:0/870826810 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe24ffff640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2580cae70 0x7fe258158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe257fff640 1 -- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 msgr2=0x7fe25815c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.356+0000 7fe25c98d640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 0x7fe25815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe257fff640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 0x7fe25815c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe257fff640 1 -- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2580cae70 msgr2=0x7fe258158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe257fff640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2580cae70 0x7fe258158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe257fff640 1 -- 172.21.15.70:0/870826810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400094d0 con 0x7fe2580c90b0 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe25c98d640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 0x7fe25815c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe24ffff640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2580cae70 0x7fe258158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe257fff640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 0x7fe258159b50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe24800d800 tx=0x7fe24800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe2557fa640 1 -- 172.21.15.70:0/870826810 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24800b6c0 con 0x7fe2580c90b0 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe2557fa640 1 -- 172.21.15.70:0/870826810 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24800b840 con 0x7fe2580c90b0 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe25e3f0640 1 -- 172.21.15.70:0/870826810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2580c3620 con 0x7fe2580c90b0 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe2557fa640 1 -- 172.21.15.70:0/870826810 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24800bbf0 con 0x7fe2580c90b0 2023-10-08T05:53:41.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.357+0000 7fe25e3f0640 1 -- 172.21.15.70:0/870826810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe258158b40 con 0x7fe2580c90b0 2023-10-08T05:53:41.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.358+0000 7fe25e3f0640 1 -- 172.21.15.70:0/870826810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe258067490 con 0x7fe2580c90b0 2023-10-08T05:53:41.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.359+0000 7fe2557fa640 1 -- 172.21.15.70:0/870826810 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe24800bd50 con 0x7fe2580c90b0 2023-10-08T05:53:41.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.360+0000 7fe2557fa640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fe23806af90 0x7fe23806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.360+0000 7fe2557fa640 1 -- 172.21.15.70:0/870826810 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fe248089c20 con 0x7fe2580c90b0 2023-10-08T05:53:41.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.361+0000 7fe24ffff640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fe23806af90 0x7fe23806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.362+0000 7fe24ffff640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fe23806af90 0x7fe23806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe240036000 tx=0x7fe24003b040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:41.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.365+0000 7fe2557fa640 1 -- 172.21.15.70:0/870826810 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe248054d00 con 0x7fe2580c90b0 2023-10-08T05:53:41.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.557+0000 7fe25e3f0640 1 -- 172.21.15.70:0/870826810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe2580b0ef0 con 0x7fe2580c90b0 2023-10-08T05:53:41.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.559+0000 7fe2557fa640 1 -- 172.21.15.70:0/870826810 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe248058320 con 0x7fe2580c90b0 2023-10-08T05:53:41.560 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:41.560 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:53:41.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:41.560 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T05:53:41.560 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T05:53:41.560 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:41.561 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:53:41.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.561+0000 7fe25e3f0640 1 -- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fe23806af90 msgr2=0x7fe23806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.561+0000 7fe25e3f0640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fe23806af90 0x7fe23806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe240036000 tx=0x7fe24003b040 comp rx=0 tx=0).stop 2023-10-08T05:53:41.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.561+0000 7fe25e3f0640 1 -- 172.21.15.70:0/870826810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 msgr2=0x7fe258159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.561+0000 7fe25e3f0640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 0x7fe258159b50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe24800d800 tx=0x7fe24800dcc0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.562+0000 7fe25e3f0640 1 -- 172.21.15.70:0/870826810 shutdown_connections 2023-10-08T05:53:41.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.562+0000 7fe25e3f0640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fe23806af90 0x7fe23806d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.562+0000 7fe25e3f0640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe25815ab40 0x7fe25815c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.562+0000 7fe25e3f0640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2580cae70 0x7fe258158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.562+0000 7fe25e3f0640 1 --2- 172.21.15.70:0/870826810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2580c90b0 0x7fe258159b50 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.562+0000 7fe25e3f0640 1 -- 172.21.15.70:0/870826810 >> 172.21.15.70:0/870826810 conn(0x7fe25806a380 msgr2=0x7fe2580ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:41.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.562+0000 7fe25e3f0640 1 -- 172.21.15.70:0/870826810 shutdown_connections 2023-10-08T05:53:41.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.562+0000 7fe25e3f0640 1 -- 172.21.15.70:0/870826810 wait complete. 2023-10-08T05:53:41.580 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T05:53:41.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T05:53:41.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.791+0000 7f4b1caec640 1 Processor -- start 2023-10-08T05:53:41.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.791+0000 7f4b1caec640 1 -- start start 2023-10-08T05:53:41.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.793+0000 7f4b1caec640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4b180b23d0 0x7f4b1815d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.793+0000 7f4b1caec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1815d6b0 con 0x7f4b1806b330 2023-10-08T05:53:41.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.793+0000 7f4b1caec640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1815d990 con 0x7f4b180b27b0 2023-10-08T05:53:41.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.793+0000 7f4b1caec640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b1815dc70 con 0x7f4b180b23d0 2023-10-08T05:53:41.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.793+0000 7f4b17fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4b1806b330 0x7f4b18160c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60382/0 (socket says 172.21.15.70:60382) 2023-10-08T05:53:41.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.793+0000 7f4b17fff640 1 -- 172.21.15.70:0/1802502377 learned_addr learned my addr 172.21.15.70:0/1802502377 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:41.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b177fe640 1 --2- 172.21.15.70:0/1802502377 >> v2:172.21.15.181:3300/0 conn(0x7f4b180b23d0 0x7f4b1815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678919032 0 0) 0x7f4b1815d6b0 con 0x7f4b1806b330 2023-10-08T05:53:41.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b04003520 con 0x7f4b1806b330 2023-10-08T05:53:41.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652639444 0 0) 0x7f4b04003520 con 0x7f4b1806b330 2023-10-08T05:53:41.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 >> v2:172.21.15.181:3300/0 conn(0x7f4b180b23d0 msgr2=0x7f4b1815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:41.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b16ffd640 1 --2- 172.21.15.70:0/1802502377 >> v2:172.21.15.181:3300/0 conn(0x7f4b180b23d0 0x7f4b1815d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 >> v1:172.21.15.181:6789/0 conn(0x7f4b180b27b0 legacy=0x7f4b1806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18160710 con 0x7f4b1806b330 2023-10-08T05:53:41.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4b100019f0 con 0x7f4b1806b330 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 >> v1:172.21.15.70:6789/0 conn(0x7f4b1806b330 legacy=0x7f4b18160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.794+0000 7f4b16ffd640 1 --2- 172.21.15.70:0/1802502377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b16ffd640 1 --2- 172.21.15.70:0/1802502377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b040047c0 0x7f4b04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b16ffd640 1 --2- 172.21.15.70:0/1802502377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b04005180 0x7f4b0400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1815dc70 con 0x7f4b04003970 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0400c120 con 0x7f4b04005180 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0400c2e0 con 0x7f4b040047c0 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b177fe640 1 --2- 172.21.15.70:0/1802502377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b177fe640 1 -- 172.21.15.70:0/1802502377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b040047c0 msgr2=0x7f4b04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b177fe640 1 --2- 172.21.15.70:0/1802502377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b040047c0 0x7f4b04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b177fe640 1 -- 172.21.15.70:0/1802502377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b04005180 msgr2=0x7f4b0400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:41.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b177fe640 1 --2- 172.21.15.70:0/1802502377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b04005180 0x7f4b0400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b177fe640 1 -- 172.21.15.70:0/1802502377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b04003520 con 0x7f4b04003970 2023-10-08T05:53:41.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.795+0000 7f4b17fff640 1 -- 172.21.15.70:0/1802502377 reap_dead start 2023-10-08T05:53:41.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.796+0000 7f4b177fe640 1 --2- 172.21.15.70:0/1802502377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b04003d90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009ac0 tx=0x7f4b040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e36bed9ff7d7c21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:41.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.796+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0802c6d0 con 0x7f4b04003970 2023-10-08T05:53:41.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.797+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0802c830 con 0x7f4b04003970 2023-10-08T05:53:41.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.797+0000 7f4b16ffd640 1 -- 172.21.15.70:0/1802502377 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0802cbc0 con 0x7f4b04003970 2023-10-08T05:53:41.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.797+0000 7f4b1caec640 1 -- 172.21.15.70:0/1802502377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 msgr2=0x7f4b04003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.797+0000 7f4b1caec640 1 --2- 172.21.15.70:0/1802502377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b04003d90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009ac0 tx=0x7f4b040057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.797+0000 7f4b1caec640 1 -- 172.21.15.70:0/1802502377 shutdown_connections 2023-10-08T05:53:41.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.797+0000 7f4b1caec640 1 --2- 172.21.15.70:0/1802502377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b04003d90 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.797+0000 7f4b1caec640 1 -- 172.21.15.70:0/1802502377 >> 172.21.15.70:0/1802502377 conn(0x7f4b180cd160 msgr2=0x7f4b180cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.797+0000 7f4b1caec640 1 -- 172.21.15.70:0/1802502377 shutdown_connections 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.797+0000 7f4b1caec640 1 -- 172.21.15.70:0/1802502377 wait complete. 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.799+0000 7f4b1caec640 1 Processor -- start 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.799+0000 7f4b1caec640 1 -- start start 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.800+0000 7f4b1caec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b18159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.800+0000 7f4b1caec640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b180c9620 0x7f4b180b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.800+0000 7f4b1caec640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b180b0fa0 0x7f4b180b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.800+0000 7f4b1caec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b180b1960 con 0x7f4b04003970 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.800+0000 7f4b1caec640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b18159560 con 0x7f4b180b0fa0 2023-10-08T05:53:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.800+0000 7f4b1caec640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b180b3a50 con 0x7f4b180c9620 2023-10-08T05:53:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.800+0000 7f4b177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b18159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.800+0000 7f4b177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b18159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59216/0 (socket says 172.21.15.70:59216) 2023-10-08T05:53:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.800+0000 7f4b177fe640 1 -- 172.21.15.70:0/2284025563 learned_addr learned my addr 172.21.15.70:0/2284025563 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b177fe640 1 -- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b180c9620 msgr2=0x7f4b180b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b17fff640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b180b0fa0 0x7f4b180b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b0ffff640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b180c9620 0x7f4b180b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b177fe640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b180c9620 0x7f4b180b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b177fe640 1 -- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b180b0fa0 msgr2=0x7f4b180b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b177fe640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b180b0fa0 0x7f4b180b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:41.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b177fe640 1 -- 172.21.15.70:0/2284025563 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b180b3d30 con 0x7f4b04003970 2023-10-08T05:53:41.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b177fe640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b18159020 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009ac0 tx=0x7f4b08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:41.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b14ff9640 1 -- 172.21.15.70:0/2284025563 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08003810 con 0x7f4b04003970 2023-10-08T05:53:41.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b14ff9640 1 -- 172.21.15.70:0/2284025563 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b08003970 con 0x7f4b04003970 2023-10-08T05:53:41.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b1caec640 1 -- 172.21.15.70:0/2284025563 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b180b4010 con 0x7f4b04003970 2023-10-08T05:53:41.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.801+0000 7f4b14ff9640 1 -- 172.21.15.70:0/2284025563 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b08003d00 con 0x7f4b04003970 2023-10-08T05:53:41.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.802+0000 7f4b1caec640 1 -- 172.21.15.70:0/2284025563 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b181f3900 con 0x7f4b04003970 2023-10-08T05:53:41.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.803+0000 7f4b1caec640 1 -- 172.21.15.70:0/2284025563 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b181f3a90 con 0x7f4b04003970 2023-10-08T05:53:41.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.803+0000 7f4b14ff9640 1 -- 172.21.15.70:0/2284025563 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4b08040450 con 0x7f4b04003970 2023-10-08T05:53:41.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.804+0000 7f4b14ff9640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f4af806af90 0x7f4af806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:41.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.804+0000 7f4b14ff9640 1 -- 172.21.15.70:0/2284025563 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f4b080ab500 con 0x7f4b04003970 2023-10-08T05:53:41.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.805+0000 7f4b0ffff640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f4af806af90 0x7f4af806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:41.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.807+0000 7f4b0ffff640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f4af806af90 0x7f4af806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b181f5bf0 tx=0x7f4b00009040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:41.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.809+0000 7f4b14ff9640 1 -- 172.21.15.70:0/2284025563 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b080765e0 con 0x7f4b04003970 2023-10-08T05:53:41.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:41.996+0000 7f4b1caec640 1 -- 172.21.15.70:0/2284025563 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4b18074150 con 0x7f4b04003970 2023-10-08T05:53:42.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.664+0000 7f4b14ff9640 1 -- 172.21.15.70:0/2284025563 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v115) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4b080044e0 con 0x7f4b04003970 2023-10-08T05:53:42.666 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T05:53:42.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 -- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f4af806af90 msgr2=0x7f4af806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:42.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f4af806af90 0x7f4af806d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b181f5bf0 tx=0x7f4b00009040 comp rx=0 tx=0).stop 2023-10-08T05:53:42.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 -- 172.21.15.70:0/2284025563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 msgr2=0x7f4b18159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:42.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b18159020 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4b08009ac0 tx=0x7f4b08004730 comp rx=0 tx=0).stop 2023-10-08T05:53:42.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 -- 172.21.15.70:0/2284025563 shutdown_connections 2023-10-08T05:53:42.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f4af806af90 0x7f4af806d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:42.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b180b0fa0 0x7f4b180b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:42.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b180c9620 0x7f4b180b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:42.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 --2- 172.21.15.70:0/2284025563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04003970 0x7f4b18159020 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:42.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 -- 172.21.15.70:0/2284025563 >> 172.21.15.70:0/2284025563 conn(0x7f4b180cd160 msgr2=0x7f4b1806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:42.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 -- 172.21.15.70:0/2284025563 shutdown_connections 2023-10-08T05:53:42.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.668+0000 7f4b1caec640 1 -- 172.21.15.70:0/2284025563 wait complete. 2023-10-08T05:53:42.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T05:53:42.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.901+0000 7fa1f15d0640 1 Processor -- start 2023-10-08T05:53:42.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.901+0000 7fa1f15d0640 1 -- start start 2023-10-08T05:53:42.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.902+0000 7fa1f15d0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa1ec15e300 0x7fa1ec0cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:42.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.902+0000 7fa1f15d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec0b2890 con 0x7fa1ec0cba40 2023-10-08T05:53:42.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.902+0000 7fa1f15d0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec15e790 con 0x7fa1ec159d50 2023-10-08T05:53:42.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.902+0000 7fa1f15d0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ec15d8b0 con 0x7fa1ec15e300 2023-10-08T05:53:42.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.903+0000 7fa1eb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa1ec0cba40 0x7fa1ec0c9310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:60390/0 (socket says 172.21.15.70:60390) 2023-10-08T05:53:42.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.903+0000 7fa1eb7fe640 1 -- 172.21.15.70:0/1565331462 learned_addr learned my addr 172.21.15.70:0/1565331462 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:42.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.903+0000 7fa1ea7fc640 1 --2- 172.21.15.70:0/1565331462 >> v2:172.21.15.181:3300/0 conn(0x7fa1ec15e300 0x7fa1ec0cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:42.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.903+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989722857 0 0) 0x7fa1ec0b2890 con 0x7fa1ec0cba40 2023-10-08T05:53:42.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.903+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d4005730 con 0x7fa1ec0cba40 2023-10-08T05:53:42.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2460229516 0 0) 0x7fa1d4005730 con 0x7fa1ec0cba40 2023-10-08T05:53:42.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 >> v2:172.21.15.181:3300/0 conn(0x7fa1ec15e300 msgr2=0x7fa1ec0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:42.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 --2- 172.21.15.70:0/1565331462 >> v2:172.21.15.181:3300/0 conn(0x7fa1ec15e300 0x7fa1ec0cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:42.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 >> v1:172.21.15.181:6789/0 conn(0x7fa1ec159d50 legacy=0x7fa1ec0b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:42.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec0cbe70 con 0x7fa1ec0cba40 2023-10-08T05:53:42.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa1e40019f0 con 0x7fa1ec0cba40 2023-10-08T05:53:42.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 >> v1:172.21.15.70:6789/0 conn(0x7fa1ec0cba40 legacy=0x7fa1ec0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:42.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 --2- 172.21.15.70:0/1565331462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d4005b80 0x7fa1d4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:42.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 --2- 172.21.15.70:0/1565331462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d40069d0 0x7fa1d4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:42.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 --2- 172.21.15.70:0/1565331462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d4007390 0x7fa1d400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:42.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec15d8b0 con 0x7fa1d4005b80 2023-10-08T05:53:42.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d400e230 con 0x7fa1d4007390 2023-10-08T05:53:42.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d400e3f0 con 0x7fa1d40069d0 2023-10-08T05:53:42.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.904+0000 7fa1eaffd640 1 --2- 172.21.15.70:0/1565331462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d4005b80 0x7fa1d4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:42.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.905+0000 7fa1eb7fe640 1 --2- 172.21.15.70:0/1565331462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d4007390 0x7fa1d400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:42.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.905+0000 7fa1eaffd640 1 -- 172.21.15.70:0/1565331462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d40069d0 msgr2=0x7fa1d4006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:42.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.905+0000 7fa1eaffd640 1 --2- 172.21.15.70:0/1565331462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1d40069d0 0x7fa1d4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:42.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.905+0000 7fa1eaffd640 1 -- 172.21.15.70:0/1565331462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d4007390 msgr2=0x7fa1d400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:42.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.905+0000 7fa1eaffd640 1 --2- 172.21.15.70:0/1565331462 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d4007390 0x7fa1d400db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:42.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.905+0000 7fa1eaffd640 1 -- 172.21.15.70:0/1565331462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d4005730 con 0x7fa1d4005b80 2023-10-08T05:53:42.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.905+0000 7fa1eb7fe640 1 -- 172.21.15.70:0/1565331462 reap_dead start 2023-10-08T05:53:42.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.906+0000 7fa1eaffd640 1 --2- 172.21.15.70:0/1565331462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d4005b80 0x7fa1d4005fa0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc001750 tx=0x7fa1d4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da8730bc592d2bb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:42.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.906+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc0052d0 con 0x7fa1d4005b80 2023-10-08T05:53:42.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.906+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1dc02e050 con 0x7fa1d4005b80 2023-10-08T05:53:42.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.906+0000 7fa1e9ffb640 1 -- 172.21.15.70:0/1565331462 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc003830 con 0x7fa1d4005b80 2023-10-08T05:53:42.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.907+0000 7fa1f15d0640 1 -- 172.21.15.70:0/1565331462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d4005b80 msgr2=0x7fa1d4005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:42.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.907+0000 7fa1f15d0640 1 --2- 172.21.15.70:0/1565331462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d4005b80 0x7fa1d4005fa0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc001750 tx=0x7fa1d4007a00 comp rx=0 tx=0).stop 2023-10-08T05:53:42.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.907+0000 7fa1f15d0640 1 -- 172.21.15.70:0/1565331462 shutdown_connections 2023-10-08T05:53:42.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.907+0000 7fa1f15d0640 1 --2- 172.21.15.70:0/1565331462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1d4005b80 0x7fa1d4005fa0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:42.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.907+0000 7fa1f15d0640 1 -- 172.21.15.70:0/1565331462 >> 172.21.15.70:0/1565331462 conn(0x7fa1ec06a140 msgr2=0x7fa1ec06a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:42.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.907+0000 7fa1f15d0640 1 -- 172.21.15.70:0/1565331462 shutdown_connections 2023-10-08T05:53:42.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.907+0000 7fa1f15d0640 1 -- 172.21.15.70:0/1565331462 wait complete. 2023-10-08T05:53:42.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.909+0000 7fa1f15d0640 1 Processor -- start 2023-10-08T05:53:42.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.909+0000 7fa1f15d0640 1 -- start start 2023-10-08T05:53:42.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1f15d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d4005b80 0x7fa1ec152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1f15d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1ec152ec0 0x7fa1ec1f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1f15d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec152970 0x7fa1ec157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1f15d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec15cfc0 con 0x7fa1ec152970 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1f15d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec0c3530 con 0x7fa1d4005b80 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1f15d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec0c3810 con 0x7fa1ec152ec0 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1eb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec152970 0x7fa1ec157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1eb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec152970 0x7fa1ec157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59236/0 (socket says 172.21.15.70:59236) 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1eb7fe640 1 -- 172.21.15.70:0/4002186309 learned_addr learned my addr 172.21.15.70:0/4002186309 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1ea7fc640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1ec152ec0 0x7fa1ec1f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.910+0000 7fa1eaffd640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d4005b80 0x7fa1ec152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:42.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1eb7fe640 1 -- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1ec152ec0 msgr2=0x7fa1ec1f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1eb7fe640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1ec152ec0 0x7fa1ec1f2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1eb7fe640 1 -- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d4005b80 msgr2=0x7fa1ec152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1eb7fe640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d4005b80 0x7fa1ec152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1eb7fe640 1 -- 172.21.15.70:0/4002186309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec0c3af0 con 0x7fa1ec152970 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1eaffd640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d4005b80 0x7fa1ec152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1eb7fe640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec152970 0x7fa1ec157cd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0702b0 tx=0x7fa1e4007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1d37fe640 1 -- 172.21.15.70:0/4002186309 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e400eaa0 con 0x7fa1ec152970 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1d37fe640 1 -- 172.21.15.70:0/4002186309 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e400ec00 con 0x7fa1ec152970 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1d37fe640 1 -- 172.21.15.70:0/4002186309 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4013420 con 0x7fa1ec152970 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1f15d0640 1 -- 172.21.15.70:0/4002186309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec0c3dd0 con 0x7fa1ec152970 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.911+0000 7fa1f15d0640 1 -- 172.21.15.70:0/4002186309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec1526a0 con 0x7fa1ec152970 2023-10-08T05:53:42.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.912+0000 7fa1f15d0640 1 -- 172.21.15.70:0/4002186309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec0c42f0 con 0x7fa1ec152970 2023-10-08T05:53:42.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.914+0000 7fa1d37fe640 1 -- 172.21.15.70:0/4002186309 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa1e4013580 con 0x7fa1ec152970 2023-10-08T05:53:42.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.915+0000 7fa1d37fe640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa1d806afc0 0x7fa1d806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:42.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.916+0000 7fa1d37fe640 1 -- 172.21.15.70:0/4002186309 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5591+0+0 (secure 0 0 0) 0x7fa1e408a400 con 0x7fa1ec152970 2023-10-08T05:53:42.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.916+0000 7fa1eaffd640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa1d806afc0 0x7fa1d806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:42.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.917+0000 7fa1eaffd640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa1d806afc0 0x7fa1d806d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc033000 tx=0x7fa1dc009030 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:42.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:42.920+0000 7fa1d37fe640 1 -- 172.21.15.70:0/4002186309 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1e40553f0 con 0x7fa1ec152970 2023-10-08T05:53:43.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.106+0000 7fa1f15d0640 1 -- 172.21.15.70:0/4002186309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa1ec0c42f0 con 0x7fa1ec152970 2023-10-08T05:53:43.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.672+0000 7fa1d37fe640 1 -- 172.21.15.70:0/4002186309 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v116) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa1ec0c42f0 con 0x7fa1ec152970 2023-10-08T05:53:43.674 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T05:53:43.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.676+0000 7fa1f15d0640 1 -- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa1d806afc0 msgr2=0x7fa1d806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:43.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.676+0000 7fa1f15d0640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa1d806afc0 0x7fa1d806d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc033000 tx=0x7fa1dc009030 comp rx=0 tx=0).stop 2023-10-08T05:53:43.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.677+0000 7fa1f15d0640 1 -- 172.21.15.70:0/4002186309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec152970 msgr2=0x7fa1ec157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:43.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.677+0000 7fa1f15d0640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec152970 0x7fa1ec157cd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0702b0 tx=0x7fa1e4007bc0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.677+0000 7fa1f15d0640 1 -- 172.21.15.70:0/4002186309 shutdown_connections 2023-10-08T05:53:43.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.677+0000 7fa1f15d0640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fa1d806afc0 0x7fa1d806d480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.677+0000 7fa1f15d0640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1ec152970 0x7fa1ec157cd0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.677+0000 7fa1f15d0640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1ec152ec0 0x7fa1ec1f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.677+0000 7fa1f15d0640 1 --2- 172.21.15.70:0/4002186309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1d4005b80 0x7fa1ec152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.677+0000 7fa1f15d0640 1 -- 172.21.15.70:0/4002186309 >> 172.21.15.70:0/4002186309 conn(0x7fa1ec06a140 msgr2=0x7fa1ec0ca870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:43.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.677+0000 7fa1f15d0640 1 -- 172.21.15.70:0/4002186309 shutdown_connections 2023-10-08T05:53:43.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.678+0000 7fa1f15d0640 1 -- 172.21.15.70:0/4002186309 wait complete. 2023-10-08T05:53:43.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T05:53:43.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.895+0000 7fb10e5d0640 1 Processor -- start 2023-10-08T05:53:43.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.895+0000 7fb10e5d0640 1 -- start start 2023-10-08T05:53:43.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.896+0000 7fb10e5d0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb108160660 0x7fb108070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:43.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.896+0000 7fb10e5d0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb1080c9910 0x7fb1080caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:43.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.896+0000 7fb10e5d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10806e150 con 0x7fb1080cb450 2023-10-08T05:53:43.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.896+0000 7fb10e5d0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10806de70 con 0x7fb1080c9910 2023-10-08T05:53:43.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.896+0000 7fb10e5d0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1080c8e30 con 0x7fb108160660 2023-10-08T05:53:43.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.897+0000 7fb0fffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb1080c9910 0x7fb1080caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:43.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.897+0000 7fb0fffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb1080c9910 0x7fb1080caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51448/0 (socket says 172.21.15.70:51448) 2023-10-08T05:53:43.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.897+0000 7fb0fffff640 1 -- 172.21.15.70:0/477808416 learned_addr learned my addr 172.21.15.70:0/477808416 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:43.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.897+0000 7fb10cb6d640 1 --1- 172.21.15.70:0/477808416 >> v1:172.21.15.70:6789/0 conn(0x7fb1080cb450 0x7fb1080b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34020/0 (socket says 172.21.15.70:34020) 2023-10-08T05:53:43.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.897+0000 7fb107fff640 1 --2- 172.21.15.70:0/477808416 >> v2:172.21.15.181:3300/0 conn(0x7fb108160660 0x7fb108070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:43.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.897+0000 7fb0fffff640 1 -- 172.21.15.70:0/477808416 >> v2:172.21.15.181:3300/0 conn(0x7fb108160660 msgr2=0x7fb108070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:43.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.897+0000 7fb0fffff640 1 --2- 172.21.15.70:0/477808416 >> v2:172.21.15.181:3300/0 conn(0x7fb108160660 0x7fb108070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.897+0000 7fb0fffff640 1 -- 172.21.15.70:0/477808416 >> v1:172.21.15.70:6789/0 conn(0x7fb1080cb450 legacy=0x7fb1080b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:43.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.897+0000 7fb0fffff640 1 -- 172.21.15.70:0/477808416 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080c9110 con 0x7fb1080c9910 2023-10-08T05:53:43.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.898+0000 7fb0fffff640 1 --2- 172.21.15.70:0/477808416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1080c9910 0x7fb1080caf10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fb1080b0d80 tx=0x7fb0f002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5950ab9519203a48 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:43.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.898+0000 7fb1077fe640 1 -- 172.21.15.70:0/477808416 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0030870 con 0x7fb1080c9910 2023-10-08T05:53:43.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.899+0000 7fb1077fe640 1 -- 172.21.15.70:0/477808416 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f00309d0 con 0x7fb1080c9910 2023-10-08T05:53:43.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.899+0000 7fb1077fe640 1 -- 172.21.15.70:0/477808416 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0030d60 con 0x7fb1080c9910 2023-10-08T05:53:43.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.899+0000 7fb10e5d0640 1 -- 172.21.15.70:0/477808416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1080c9910 msgr2=0x7fb1080caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:43.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.899+0000 7fb10e5d0640 1 --2- 172.21.15.70:0/477808416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1080c9910 0x7fb1080caf10 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fb1080b0d80 tx=0x7fb0f002b9b0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.899+0000 7fb10e5d0640 1 -- 172.21.15.70:0/477808416 shutdown_connections 2023-10-08T05:53:43.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.899+0000 7fb10e5d0640 1 --2- 172.21.15.70:0/477808416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1080c9910 0x7fb1080caf10 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.899+0000 7fb10e5d0640 1 --2- 172.21.15.70:0/477808416 >> v2:172.21.15.181:3300/0 conn(0x7fb108160660 0x7fb108070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.899+0000 7fb10e5d0640 1 -- 172.21.15.70:0/477808416 >> 172.21.15.70:0/477808416 conn(0x7fb10806a5d0 msgr2=0x7fb1080b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:43.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.899+0000 7fb10e5d0640 1 -- 172.21.15.70:0/477808416 shutdown_connections 2023-10-08T05:53:43.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.899+0000 7fb10e5d0640 1 -- 172.21.15.70:0/477808416 wait complete. 2023-10-08T05:53:43.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.901+0000 7fb10e5d0640 1 Processor -- start 2023-10-08T05:53:43.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.901+0000 7fb10e5d0640 1 -- start start 2023-10-08T05:53:43.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.902+0000 7fb10e5d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1080c9910 0x7fb1080c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:43.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.902+0000 7fb10e5d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1080cb450 0x7fb10815d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:43.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.902+0000 7fb10e5d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb108160660 0x7fb10815d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:43.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.902+0000 7fb10e5d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10815b6e0 con 0x7fb108160660 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.902+0000 7fb10e5d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10815c6f0 con 0x7fb1080cb450 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.902+0000 7fb10e5d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10815dcb0 con 0x7fb1080c9910 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.902+0000 7fb10cb6d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb108160660 0x7fb10815d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.902+0000 7fb10cb6d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb108160660 0x7fb10815d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51450/0 (socket says 172.21.15.70:51450) 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.902+0000 7fb10cb6d640 1 -- 172.21.15.70:0/2928251045 learned_addr learned my addr 172.21.15.70:0/2928251045 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.903+0000 7fb107fff640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1080c9910 0x7fb1080c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.903+0000 7fb0fffff640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1080cb450 0x7fb10815d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.903+0000 7fb10cb6d640 1 -- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1080c9910 msgr2=0x7fb1080c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.903+0000 7fb10cb6d640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1080c9910 0x7fb1080c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.903+0000 7fb10cb6d640 1 -- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1080cb450 msgr2=0x7fb10815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.903+0000 7fb10cb6d640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1080cb450 0x7fb10815d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:43.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.903+0000 7fb10cb6d640 1 -- 172.21.15.70:0/2928251045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00094d0 con 0x7fb108160660 2023-10-08T05:53:43.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.903+0000 7fb10cb6d640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb108160660 0x7fb10815d770 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb10000c780 tx=0x7fb10000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:43.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.903+0000 7fb1057fa640 1 -- 172.21.15.70:0/2928251045 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10000ce90 con 0x7fb108160660 2023-10-08T05:53:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.904+0000 7fb1057fa640 1 -- 172.21.15.70:0/2928251045 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10000a640 con 0x7fb108160660 2023-10-08T05:53:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.904+0000 7fb1057fa640 1 -- 172.21.15.70:0/2928251045 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10000a9f0 con 0x7fb108160660 2023-10-08T05:53:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.904+0000 7fb10e5d0640 1 -- 172.21.15.70:0/2928251045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10815b990 con 0x7fb108160660 2023-10-08T05:53:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.904+0000 7fb10e5d0640 1 -- 172.21.15.70:0/2928251045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10815c1d0 con 0x7fb108160660 2023-10-08T05:53:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.905+0000 7fb10e5d0640 1 -- 172.21.15.70:0/2928251045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10815c3c0 con 0x7fb108160660 2023-10-08T05:53:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.906+0000 7fb1057fa640 1 -- 172.21.15.70:0/2928251045 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb10000ab50 con 0x7fb108160660 2023-10-08T05:53:43.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.907+0000 7fb1057fa640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fb0e806af90 0x7fb0e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:43.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.907+0000 7fb1057fa640 1 -- 172.21.15.70:0/2928251045 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5973+0+0 (secure 0 0 0) 0x7fb100089830 con 0x7fb108160660 2023-10-08T05:53:43.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.908+0000 7fb107fff640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fb0e806af90 0x7fb0e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:43.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.909+0000 7fb107fff640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fb0e806af90 0x7fb0e806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8006fd0 tx=0x7fb0f8008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:43.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:43.911+0000 7fb1057fa640 1 -- 172.21.15.70:0/2928251045 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb100054750 con 0x7fb108160660 2023-10-08T05:53:44.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.122+0000 7fb10e5d0640 1 -- 172.21.15.70:0/2928251045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb10804e160 con 0x7fb108160660 2023-10-08T05:53:44.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.722+0000 7fb1057fa640 1 -- 172.21.15.70:0/2928251045 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb100057d70 con 0x7fb108160660 2023-10-08T05:53:44.724 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 13 and data pool 14 2023-10-08T05:53:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.727+0000 7fb10e5d0640 1 -- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fb0e806af90 msgr2=0x7fb0e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.727+0000 7fb10e5d0640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fb0e806af90 0x7fb0e806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8006fd0 tx=0x7fb0f8008040 comp rx=0 tx=0).stop 2023-10-08T05:53:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.727+0000 7fb10e5d0640 1 -- 172.21.15.70:0/2928251045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb108160660 msgr2=0x7fb10815d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.727+0000 7fb10e5d0640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb108160660 0x7fb10815d770 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb10000c780 tx=0x7fb10000cc40 comp rx=0 tx=0).stop 2023-10-08T05:53:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.728+0000 7fb10e5d0640 1 -- 172.21.15.70:0/2928251045 shutdown_connections 2023-10-08T05:53:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.728+0000 7fb10e5d0640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fb0e806af90 0x7fb0e806d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.728+0000 7fb10e5d0640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb108160660 0x7fb10815d770 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.728+0000 7fb10e5d0640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb1080cb450 0x7fb10815d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.728+0000 7fb10e5d0640 1 --2- 172.21.15.70:0/2928251045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1080c9910 0x7fb1080c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.728+0000 7fb10e5d0640 1 -- 172.21.15.70:0/2928251045 >> 172.21.15.70:0/2928251045 conn(0x7fb10806a5d0 msgr2=0x7fb1080c80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.733+0000 7fb10e5d0640 1 -- 172.21.15.70:0/2928251045 shutdown_connections 2023-10-08T05:53:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.734+0000 7fb10e5d0640 1 -- 172.21.15.70:0/2928251045 wait complete. 2023-10-08T05:53:44.764 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:44.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.937+0000 7f998d2e5640 1 Processor -- start 2023-10-08T05:53:44.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.937+0000 7f998d2e5640 1 -- start start 2023-10-08T05:53:44.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.939+0000 7f998d2e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9988159ac0 0x7f998806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:44.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.939+0000 7f998d2e5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f99880ad790 0x7f99880702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:44.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.939+0000 7f998d2e5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f998815e330 0x7f9988070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:44.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.939+0000 7f998d2e5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998806dee0 con 0x7f998815e330 2023-10-08T05:53:44.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.939+0000 7f998d2e5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998815d850 con 0x7f99880ad790 2023-10-08T05:53:44.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.939+0000 7f998d2e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998815db30 con 0x7f9988159ac0 2023-10-08T05:53:44.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.939+0000 7f99877fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f998815e330 0x7f9988070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:44.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.939+0000 7f99877fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f998815e330 0x7f9988070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51458/0 (socket says 172.21.15.70:51458) 2023-10-08T05:53:44.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.939+0000 7f99877fe640 1 -- 172.21.15.70:0/1210126368 learned_addr learned my addr 172.21.15.70:0/1210126368 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:44.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.940+0000 7f997e7fc640 1 --2- 172.21.15.70:0/1210126368 >> v2:172.21.15.181:3300/0 conn(0x7f99880ad790 0x7f99880702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:44.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.940+0000 7f9986ffd640 1 --2- 172.21.15.70:0/1210126368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9988159ac0 0x7f998806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:44.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.940+0000 7f99877fe640 1 -- 172.21.15.70:0/1210126368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9988159ac0 msgr2=0x7f998806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:44.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.940+0000 7f99877fe640 1 --2- 172.21.15.70:0/1210126368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9988159ac0 0x7f998806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.940+0000 7f99877fe640 1 -- 172.21.15.70:0/1210126368 >> v2:172.21.15.181:3300/0 conn(0x7f99880ad790 msgr2=0x7f99880702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:44.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.940+0000 7f99877fe640 1 --2- 172.21.15.70:0/1210126368 >> v2:172.21.15.181:3300/0 conn(0x7f99880ad790 0x7f99880702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.940+0000 7f99877fe640 1 -- 172.21.15.70:0/1210126368 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998815de10 con 0x7f998815e330 2023-10-08T05:53:44.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.941+0000 7f99877fe640 1 --2- 172.21.15.70:0/1210126368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 0x7f9988070810 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f998806e470 tx=0x7f998002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9b1cec30a59ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:44.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.941+0000 7f99867fc640 1 -- 172.21.15.70:0/1210126368 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998002ca70 con 0x7f998815e330 2023-10-08T05:53:44.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.941+0000 7f99867fc640 1 -- 172.21.15.70:0/1210126368 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998002cbd0 con 0x7f998815e330 2023-10-08T05:53:44.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.941+0000 7f99867fc640 1 -- 172.21.15.70:0/1210126368 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99800343f0 con 0x7f998815e330 2023-10-08T05:53:44.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.942+0000 7f998d2e5640 1 -- 172.21.15.70:0/1210126368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 msgr2=0x7f9988070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:44.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.942+0000 7f998d2e5640 1 --2- 172.21.15.70:0/1210126368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 0x7f9988070810 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f998806e470 tx=0x7f998002bed0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.942+0000 7f998d2e5640 1 -- 172.21.15.70:0/1210126368 shutdown_connections 2023-10-08T05:53:44.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.942+0000 7f998d2e5640 1 --2- 172.21.15.70:0/1210126368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 0x7f9988070810 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.942+0000 7f998d2e5640 1 --2- 172.21.15.70:0/1210126368 >> v2:172.21.15.181:3300/0 conn(0x7f99880ad790 0x7f99880702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.942+0000 7f998d2e5640 1 --2- 172.21.15.70:0/1210126368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9988159ac0 0x7f998806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.942+0000 7f998d2e5640 1 -- 172.21.15.70:0/1210126368 >> 172.21.15.70:0/1210126368 conn(0x7f998806a380 msgr2=0x7f99880b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:44.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.942+0000 7f998d2e5640 1 -- 172.21.15.70:0/1210126368 shutdown_connections 2023-10-08T05:53:44.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.942+0000 7f998d2e5640 1 -- 172.21.15.70:0/1210126368 wait complete. 2023-10-08T05:53:44.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.944+0000 7f998d2e5640 1 Processor -- start 2023-10-08T05:53:44.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.944+0000 7f998d2e5640 1 -- start start 2023-10-08T05:53:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f998d2e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99880ad790 0x7f998814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f998d2e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9988159ac0 0x7f9988151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f998d2e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 0x7f998815c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f998d2e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9988150b00 con 0x7f998815e330 2023-10-08T05:53:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f998d2e5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9988150de0 con 0x7f99880ad790 2023-10-08T05:53:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f998d2e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99881510c0 con 0x7f9988159ac0 2023-10-08T05:53:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f99877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 0x7f998815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f99877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 0x7f998815c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51460/0 (socket says 172.21.15.70:51460) 2023-10-08T05:53:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f99877fe640 1 -- 172.21.15.70:0/3459318790 learned_addr learned my addr 172.21.15.70:0/3459318790 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f99877fe640 1 -- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9988159ac0 msgr2=0x7f9988151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f99877fe640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9988159ac0 0x7f9988151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f99877fe640 1 -- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99880ad790 msgr2=0x7f998814fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f99877fe640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99880ad790 0x7f998814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.945+0000 7f99877fe640 1 -- 172.21.15.70:0/3459318790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9980009b30 con 0x7f998815e330 2023-10-08T05:53:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.946+0000 7f99877fe640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 0x7f998815c060 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9980036040 tx=0x7f9980004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.946+0000 7f997ffff640 1 -- 172.21.15.70:0/3459318790 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980009510 con 0x7f998815e330 2023-10-08T05:53:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.946+0000 7f997ffff640 1 -- 172.21.15.70:0/3459318790 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998003e450 con 0x7f998815e330 2023-10-08T05:53:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.946+0000 7f997ffff640 1 -- 172.21.15.70:0/3459318790 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998003e800 con 0x7f998815e330 2023-10-08T05:53:44.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.946+0000 7f998d2e5640 1 -- 172.21.15.70:0/3459318790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99881513a0 con 0x7f998815e330 2023-10-08T05:53:44.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.946+0000 7f998d2e5640 1 -- 172.21.15.70:0/3459318790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99881520e0 con 0x7f998815e330 2023-10-08T05:53:44.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.947+0000 7f998d2e5640 1 -- 172.21.15.70:0/3459318790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9988151860 con 0x7f998815e330 2023-10-08T05:53:44.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.948+0000 7f997ffff640 1 -- 172.21.15.70:0/3459318790 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f998003e960 con 0x7f998815e330 2023-10-08T05:53:44.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.949+0000 7f997ffff640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f996806af90 0x7f996806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:44.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.949+0000 7f997ffff640 1 -- 172.21.15.70:0/3459318790 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f99800ac850 con 0x7f998815e330 2023-10-08T05:53:44.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.950+0000 7f9986ffd640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f996806af90 0x7f996806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:44.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.951+0000 7f9986ffd640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f996806af90 0x7f996806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9978006fd0 tx=0x7f9978008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:44.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:44.953+0000 7f997ffff640 1 -- 172.21.15.70:0/3459318790 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99800777b0 con 0x7f998815e330 2023-10-08T05:53:45.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.130+0000 7f998d2e5640 1 -- 172.21.15.70:0/3459318790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9988067490 con 0x7f998815e330 2023-10-08T05:53:45.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.133+0000 7f997ffff640 1 -- 172.21.15.70:0/3459318790 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16799 (secure 0 0 0) 0x7f998007add0 con 0x7f998815e330 2023-10-08T05:53:45.135 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:45.135 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":117,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:44.125025+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:42.052166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:43.161991+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:45.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 -- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f996806af90 msgr2=0x7f996806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f996806af90 0x7f996806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9978006fd0 tx=0x7f9978008040 comp rx=0 tx=0).stop 2023-10-08T05:53:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 -- 172.21.15.70:0/3459318790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 msgr2=0x7f998815c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 0x7f998815c060 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9980036040 tx=0x7f9980004a40 comp rx=0 tx=0).stop 2023-10-08T05:53:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 -- 172.21.15.70:0/3459318790 shutdown_connections 2023-10-08T05:53:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f996806af90 0x7f996806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f998815e330 0x7f998815c060 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9988159ac0 0x7f9988151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 --2- 172.21.15.70:0/3459318790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99880ad790 0x7f998814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 -- 172.21.15.70:0/3459318790 >> 172.21.15.70:0/3459318790 conn(0x7f998806a380 msgr2=0x7f99880af720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 -- 172.21.15.70:0/3459318790 shutdown_connections 2023-10-08T05:53:45.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.136+0000 7f998d2e5640 1 -- 172.21.15.70:0/3459318790 wait complete. 2023-10-08T05:53:45.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:45.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.363+0000 7f79cbfff640 1 Processor -- start 2023-10-08T05:53:45.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.363+0000 7f79cbfff640 1 -- start start 2023-10-08T05:53:45.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.364+0000 7f79cbfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc1599d0 0x7f79cc0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.364+0000 7f79cbfff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79cc0cacf0 con 0x7f79cc0b07d0 2023-10-08T05:53:45.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.364+0000 7f79cbfff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79cc0ca3c0 con 0x7f79cc0cc210 2023-10-08T05:53:45.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.364+0000 7f79cbfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc0b10f0 con 0x7f79cc1599d0 2023-10-08T05:53:45.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79cb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f79cc0b07d0 0x7f79cc06c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34036/0 (socket says 172.21.15.70:34036) 2023-10-08T05:53:45.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79cb7fe640 1 -- 172.21.15.70:0/770053844 learned_addr learned my addr 172.21.15.70:0/770053844 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:45.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79caffd640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc1599d0 0x7f79cc0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697311666 0 0) 0x7f79cc0cacf0 con 0x7f79cc0b07d0 2023-10-08T05:53:45.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79b8005730 con 0x7f79cc0b07d0 2023-10-08T05:53:45.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3011369093 0 0) 0x7f79b8005730 con 0x7f79cc0b07d0 2023-10-08T05:53:45.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc1599d0 msgr2=0x7f79cc0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:45.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79c9ffb640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc1599d0 0x7f79cc0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 >> v1:172.21.15.181:6789/0 conn(0x7f79cc0cc210 legacy=0x7f79cc0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc0ca6a0 con 0x7f79cc0b07d0 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.365+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f79c4001670 con 0x7f79cc0b07d0 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 >> v1:172.21.15.70:6789/0 conn(0x7f79cc0b07d0 legacy=0x7f79cc06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79c9ffb640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b8005bb0 0x7f79b8005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79c9ffb640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc1599d0 0x7f79b80069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79c9ffb640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8006f10 0x7f79b8007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc0b10f0 con 0x7f79b8006f10 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b800dc40 con 0x7f79b8005bb0 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b800de00 con 0x7f79cc1599d0 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79cb7fe640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8006f10 0x7f79b8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79caffd640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b8005bb0 0x7f79b8005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79ca7fc640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc1599d0 0x7f79b80069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79cb7fe640 1 -- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc1599d0 msgr2=0x7f79b80069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79cb7fe640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc1599d0 0x7f79b80069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79cb7fe640 1 -- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b8005bb0 msgr2=0x7f79b8005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79cb7fe640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b8005bb0 0x7f79b8005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79cb7fe640 1 -- 172.21.15.70:0/770053844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b8005730 con 0x7f79b8006f10 2023-10-08T05:53:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.366+0000 7f79ca7fc640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc1599d0 0x7f79b80069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.367+0000 7f79cb7fe640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8006f10 0x7f79b8007390 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0cc060 tx=0x7f79b8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33203d764702e63 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.368+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c4005ad0 con 0x7f79b8006f10 2023-10-08T05:53:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.368+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c4005c30 con 0x7f79b8006f10 2023-10-08T05:53:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.368+0000 7f79c9ffb640 1 -- 172.21.15.70:0/770053844 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c402c850 con 0x7f79b8006f10 2023-10-08T05:53:45.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.368+0000 7f79cbfff640 1 -- 172.21.15.70:0/770053844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8006f10 msgr2=0x7f79b8007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.368+0000 7f79cbfff640 1 --2- 172.21.15.70:0/770053844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79b8006f10 0x7f79b8007390 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0cc060 tx=0x7f79b8007a00 comp rx=0 tx=0).stop 2023-10-08T05:53:45.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.368+0000 7f79cb7fe640 1 -- 172.21.15.70:0/770053844 reap_dead start 2023-10-08T05:53:45.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.368+0000 7f79cbfff640 1 -- 172.21.15.70:0/770053844 shutdown_connections 2023-10-08T05:53:45.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.368+0000 7f79cbfff640 1 -- 172.21.15.70:0/770053844 >> 172.21.15.70:0/770053844 conn(0x7f79cc0cb0c0 msgr2=0x7f79cc0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:45.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.368+0000 7f79cbfff640 1 -- 172.21.15.70:0/770053844 shutdown_connections 2023-10-08T05:53:45.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.368+0000 7f79cbfff640 1 -- 172.21.15.70:0/770053844 wait complete. 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.370+0000 7f79cbfff640 1 Processor -- start 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.370+0000 7f79cbfff640 1 -- start start 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.371+0000 7f79cbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b8006f10 0x7f79cc0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.371+0000 7f79cbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79cc0c8760 0x7f79cc0ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.371+0000 7f79cbfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc156660 0x7f79cc0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.371+0000 7f79cbfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc0ac970 con 0x7f79cc0c8760 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.371+0000 7f79cbfff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc0c7370 con 0x7f79b8006f10 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.371+0000 7f79cbfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc1571c0 con 0x7f79cc156660 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.371+0000 7f79ca7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79cc0c8760 0x7f79cc0ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79ca7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79cc0c8760 0x7f79cc0ac2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51478/0 (socket says 172.21.15.70:51478) 2023-10-08T05:53:45.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79ca7fc640 1 -- 172.21.15.70:0/1366503739 learned_addr learned my addr 172.21.15.70:0/1366503739 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79cb7fe640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc156660 0x7f79cc0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79caffd640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b8006f10 0x7f79cc0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79ca7fc640 1 -- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc156660 msgr2=0x7f79cc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79ca7fc640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc156660 0x7f79cc0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79ca7fc640 1 -- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b8006f10 msgr2=0x7f79cc0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79ca7fc640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b8006f10 0x7f79cc0c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79ca7fc640 1 -- 172.21.15.70:0/1366503739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc0c67a0 con 0x7f79cc0c8760 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79caffd640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b8006f10 0x7f79cc0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.372+0000 7f79ca7fc640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79cc0c8760 0x7f79cc0ac2e0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f79b400db40 tx=0x7f79b40020a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.373+0000 7f79c37fe640 1 -- 172.21.15.70:0/1366503739 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4017070 con 0x7f79cc0c8760 2023-10-08T05:53:45.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.373+0000 7f79c37fe640 1 -- 172.21.15.70:0/1366503739 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b4003ba0 con 0x7f79cc0c8760 2023-10-08T05:53:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.373+0000 7f79cbfff640 1 -- 172.21.15.70:0/1366503739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79cc0c6a40 con 0x7f79cc0c8760 2023-10-08T05:53:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.373+0000 7f79c37fe640 1 -- 172.21.15.70:0/1366503739 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4006b50 con 0x7f79cc0c8760 2023-10-08T05:53:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.373+0000 7f79cbfff640 1 -- 172.21.15.70:0/1366503739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79cc0c7000 con 0x7f79cc0c8760 2023-10-08T05:53:45.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.374+0000 7f79cbfff640 1 -- 172.21.15.70:0/1366503739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79cc04e160 con 0x7f79cc0c8760 2023-10-08T05:53:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.375+0000 7f79c37fe640 1 -- 172.21.15.70:0/1366503739 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f79b4006cb0 con 0x7f79cc0c8760 2023-10-08T05:53:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.376+0000 7f79c37fe640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f79ac06af90 0x7f79ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.376+0000 7f79c37fe640 1 -- 172.21.15.70:0/1366503739 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f79b4089760 con 0x7f79cc0c8760 2023-10-08T05:53:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.376+0000 7f79caffd640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f79ac06af90 0x7f79ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.377+0000 7f79caffd640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f79ac06af90 0x7f79ac06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009a00 tx=0x7f79bc008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:45.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.380+0000 7f79c37fe640 1 -- 172.21.15.70:0/1366503739 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79b40543e0 con 0x7f79cc0c8760 2023-10-08T05:53:45.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.569+0000 7f79cbfff640 1 -- 172.21.15.70:0/1366503739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79cc0c74b0 con 0x7f79cc0c8760 2023-10-08T05:53:45.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.572+0000 7f79c37fe640 1 -- 172.21.15.70:0/1366503739 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16799 (secure 0 0 0) 0x7f79b4057a00 con 0x7f79cc0c8760 2023-10-08T05:53:45.573 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:45.573 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":117,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:44.125025+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:42.052166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:43.161991+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.574+0000 7f79cbfff640 1 -- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f79ac06af90 msgr2=0x7f79ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.574+0000 7f79cbfff640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f79ac06af90 0x7f79ac06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009a00 tx=0x7f79bc008040 comp rx=0 tx=0).stop 2023-10-08T05:53:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.574+0000 7f79cbfff640 1 -- 172.21.15.70:0/1366503739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79cc0c8760 msgr2=0x7f79cc0ac2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.574+0000 7f79cbfff640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79cc0c8760 0x7f79cc0ac2e0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f79b400db40 tx=0x7f79b40020a0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.575+0000 7f79cbfff640 1 -- 172.21.15.70:0/1366503739 shutdown_connections 2023-10-08T05:53:45.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.575+0000 7f79cbfff640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f79ac06af90 0x7f79ac06d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.575+0000 7f79cbfff640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79cc156660 0x7f79cc0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.575+0000 7f79cbfff640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79cc0c8760 0x7f79cc0ac2e0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.575+0000 7f79cbfff640 1 --2- 172.21.15.70:0/1366503739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79b8006f10 0x7f79cc0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.575+0000 7f79cbfff640 1 -- 172.21.15.70:0/1366503739 >> 172.21.15.70:0/1366503739 conn(0x7f79cc0cb0c0 msgr2=0x7f79cc0ce5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.575+0000 7f79cbfff640 1 -- 172.21.15.70:0/1366503739 shutdown_connections 2023-10-08T05:53:45.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.575+0000 7f79cbfff640 1 -- 172.21.15.70:0/1366503739 wait complete. 2023-10-08T05:53:45.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T05:53:45.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.818+0000 7f3f5a2cf640 1 Processor -- start 2023-10-08T05:53:45.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.818+0000 7f3f5a2cf640 1 -- start start 2023-10-08T05:53:45.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.819+0000 7f3f5a2cf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f54160660 0x7f3f54070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.819+0000 7f3f5a2cf640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f540c9910 0x7f3f540caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.820+0000 7f3f5a2cf640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5406e150 con 0x7f3f540cb450 2023-10-08T05:53:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.820+0000 7f3f5a2cf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f540cb8c0 con 0x7f3f54160660 2023-10-08T05:53:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.820+0000 7f3f5a2cf640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5406de70 con 0x7f3f540c9910 2023-10-08T05:53:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.820+0000 7f3f53fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f54160660 0x7f3f54070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.820+0000 7f3f53fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f54160660 0x7f3f54070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51488/0 (socket says 172.21.15.70:51488) 2023-10-08T05:53:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.820+0000 7f3f53fff640 1 -- 172.21.15.70:0/585730140 learned_addr learned my addr 172.21.15.70:0/585730140 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.820+0000 7f3f53fff640 1 -- 172.21.15.70:0/585730140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f540c9910 msgr2=0x7f3f540caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.820+0000 7f3f53fff640 1 --2- 172.21.15.70:0/585730140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f540c9910 0x7f3f540caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.820+0000 7f3f53fff640 1 -- 172.21.15.70:0/585730140 >> v1:172.21.15.181:6789/0 conn(0x7f3f540cb450 legacy=0x7f3f540b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:45.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.820+0000 7f3f53fff640 1 -- 172.21.15.70:0/585730140 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540c8e30 con 0x7f3f54160660 2023-10-08T05:53:45.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.821+0000 7f3f53fff640 1 --2- 172.21.15.70:0/585730140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f54160660 0x7f3f54070cc0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009870 tx=0x7f3f4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51721cb86fb010f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f52ffd640 1 -- 172.21.15.70:0/585730140 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44031640 con 0x7f3f54160660 2023-10-08T05:53:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f52ffd640 1 -- 172.21.15.70:0/585730140 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f440317a0 con 0x7f3f54160660 2023-10-08T05:53:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f52ffd640 1 -- 172.21.15.70:0/585730140 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44031b30 con 0x7f3f54160660 2023-10-08T05:53:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/585730140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f54160660 msgr2=0x7f3f54070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f5a2cf640 1 --2- 172.21.15.70:0/585730140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f54160660 0x7f3f54070cc0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009870 tx=0x7f3f4402b8f0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/585730140 shutdown_connections 2023-10-08T05:53:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f5a2cf640 1 --2- 172.21.15.70:0/585730140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f540c9910 0x7f3f540caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f5a2cf640 1 --2- 172.21.15.70:0/585730140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f54160660 0x7f3f54070cc0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/585730140 >> 172.21.15.70:0/585730140 conn(0x7f3f5406a5d0 msgr2=0x7f3f540b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:45.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/585730140 shutdown_connections 2023-10-08T05:53:45.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.822+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/585730140 wait complete. 2023-10-08T05:53:45.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.824+0000 7f3f5a2cf640 1 Processor -- start 2023-10-08T05:53:45.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.824+0000 7f3f5a2cf640 1 -- start start 2023-10-08T05:53:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.825+0000 7f3f5a2cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f540c9910 0x7f3f541f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.825+0000 7f3f5a2cf640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f540cb450 0x7f3f54150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.825+0000 7f3f5a2cf640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f54160660 0x7f3f54156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.825+0000 7f3f5a2cf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541544a0 con 0x7f3f540c9910 2023-10-08T05:53:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.825+0000 7f3f5a2cf640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541f2520 con 0x7f3f540cb450 2023-10-08T05:53:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.825+0000 7f3f5a2cf640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5414d470 con 0x7f3f54160660 2023-10-08T05:53:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.825+0000 7f3f53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f540c9910 0x7f3f541f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.825+0000 7f3f53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f540c9910 0x7f3f541f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51500/0 (socket says 172.21.15.70:51500) 2023-10-08T05:53:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.825+0000 7f3f53fff640 1 -- 172.21.15.70:0/4052786874 learned_addr learned my addr 172.21.15.70:0/4052786874 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:45.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.826+0000 7f3f53fff640 1 -- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f54160660 msgr2=0x7f3f54156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.826+0000 7f3f53fff640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f54160660 0x7f3f54156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.826+0000 7f3f53fff640 1 -- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f540cb450 msgr2=0x7f3f54150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:45.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.826+0000 7f3f53fff640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f540cb450 0x7f3f54150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:45.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.826+0000 7f3f53fff640 1 -- 172.21.15.70:0/4052786874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440094d0 con 0x7f3f540c9910 2023-10-08T05:53:45.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.826+0000 7f3f53fff640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f540c9910 0x7f3f541f1fe0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009870 tx=0x7f3f440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:45.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.826+0000 7f3f50ff9640 1 -- 172.21.15.70:0/4052786874 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4403ac60 con 0x7f3f540c9910 2023-10-08T05:53:45.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.826+0000 7f3f50ff9640 1 -- 172.21.15.70:0/4052786874 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4403adc0 con 0x7f3f540c9910 2023-10-08T05:53:45.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.826+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/4052786874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f54159380 con 0x7f3f540c9910 2023-10-08T05:53:45.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.826+0000 7f3f50ff9640 1 -- 172.21.15.70:0/4052786874 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44044630 con 0x7f3f540c9910 2023-10-08T05:53:45.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.827+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/4052786874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f54153de0 con 0x7f3f540c9910 2023-10-08T05:53:45.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.828+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/4052786874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5404e160 con 0x7f3f540c9910 2023-10-08T05:53:45.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.828+0000 7f3f50ff9640 1 -- 172.21.15.70:0/4052786874 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3f4402f050 con 0x7f3f540c9910 2023-10-08T05:53:45.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.829+0000 7f3f50ff9640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3f4006af90 0x7f3f4006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:45.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.830+0000 7f3f50ff9640 1 -- 172.21.15.70:0/4052786874 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3f440aba70 con 0x7f3f540c9910 2023-10-08T05:53:45.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.830+0000 7f3f537fe640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3f4006af90 0x7f3f4006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:45.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.831+0000 7f3f537fe640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3f4006af90 0x7f3f4006d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3f540c1cb0 tx=0x7f3f48006c90 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:45.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:45.833+0000 7f3f50ff9640 1 -- 172.21.15.70:0/4052786874 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f440766f0 con 0x7f3f540c9910 2023-10-08T05:53:46.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.056+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/4052786874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3f54073590 con 0x7f3f540c9910 2023-10-08T05:53:46.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.736+0000 7f3f50ff9640 1 -- 172.21.15.70:0/4052786874 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3f44031e60 con 0x7f3f540c9910 2023-10-08T05:53:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.739+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3f4006af90 msgr2=0x7f3f4006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.739+0000 7f3f5a2cf640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3f4006af90 0x7f3f4006d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3f540c1cb0 tx=0x7f3f48006c90 comp rx=0 tx=0).stop 2023-10-08T05:53:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.739+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/4052786874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f540c9910 msgr2=0x7f3f541f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.739+0000 7f3f5a2cf640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f540c9910 0x7f3f541f1fe0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009870 tx=0x7f3f440046c0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.739+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/4052786874 shutdown_connections 2023-10-08T05:53:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.739+0000 7f3f5a2cf640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f3f4006af90 0x7f3f4006d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.739+0000 7f3f5a2cf640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f54160660 0x7f3f54156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.739+0000 7f3f5a2cf640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f540cb450 0x7f3f54150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.739+0000 7f3f5a2cf640 1 --2- 172.21.15.70:0/4052786874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f540c9910 0x7f3f541f1fe0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.739+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/4052786874 >> 172.21.15.70:0/4052786874 conn(0x7f3f5406a5d0 msgr2=0x7f3f540ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.740+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/4052786874 shutdown_connections 2023-10-08T05:53:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.740+0000 7f3f5a2cf640 1 -- 172.21.15.70:0/4052786874 wait complete. 2023-10-08T05:53:46.761 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T05:53:46.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T05:53:46.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.975+0000 7fde2ef5a640 1 Processor -- start 2023-10-08T05:53:46.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.976+0000 7fde2ef5a640 1 -- start start 2023-10-08T05:53:46.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.977+0000 7fde2ef5a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 0x7fde280b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:46.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.977+0000 7fde2ef5a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fde2806dfc0 0x7fde28070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:46.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.977+0000 7fde2ef5a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde280b2620 con 0x7fde28071070 2023-10-08T05:53:46.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.977+0000 7fde2ef5a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde280b2060 con 0x7fde2806dfc0 2023-10-08T05:53:46.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.977+0000 7fde2ef5a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde280b2340 con 0x7fde280afbe0 2023-10-08T05:53:46.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.978+0000 7fde2ccf6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 0x7fde280b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:46.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.978+0000 7fde2d4f7640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fde28071070 0x7fde280cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:55892/0 (socket says 172.21.15.70:55892) 2023-10-08T05:53:46.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.978+0000 7fde2d4f7640 1 -- 172.21.15.70:0/2802432294 learned_addr learned my addr 172.21.15.70:0/2802432294 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:46.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.978+0000 7fde27fff640 1 --2- 172.21.15.70:0/2802432294 >> v2:172.21.15.181:3300/0 conn(0x7fde2806dfc0 0x7fde28070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:46.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.979+0000 7fde2ccf6640 1 -- 172.21.15.70:0/2802432294 >> v2:172.21.15.181:3300/0 conn(0x7fde2806dfc0 msgr2=0x7fde28070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:46.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.979+0000 7fde2ccf6640 1 --2- 172.21.15.70:0/2802432294 >> v2:172.21.15.181:3300/0 conn(0x7fde2806dfc0 0x7fde28070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.979+0000 7fde2ccf6640 1 -- 172.21.15.70:0/2802432294 >> v1:172.21.15.181:6789/0 conn(0x7fde28071070 legacy=0x7fde280cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:46.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.979+0000 7fde2ccf6640 1 -- 172.21.15.70:0/2802432294 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde28070350 con 0x7fde280afbe0 2023-10-08T05:53:46.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.979+0000 7fde27fff640 1 --2- 172.21.15.70:0/2802432294 >> v2:172.21.15.181:3300/0 conn(0x7fde2806dfc0 0x7fde28070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:53:46.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.980+0000 7fde2ccf6640 1 --2- 172.21.15.70:0/2802432294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 0x7fde280b15b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0097a0 tx=0x7fde1c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=579585d854b7da85 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:46.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.981+0000 7fde277fe640 1 -- 172.21.15.70:0/2802432294 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c02d040 con 0x7fde280afbe0 2023-10-08T05:53:46.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.981+0000 7fde277fe640 1 -- 172.21.15.70:0/2802432294 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c031810 con 0x7fde280afbe0 2023-10-08T05:53:46.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.981+0000 7fde277fe640 1 -- 172.21.15.70:0/2802432294 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c031ba0 con 0x7fde280afbe0 2023-10-08T05:53:46.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.982+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2802432294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 msgr2=0x7fde280b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:46.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.982+0000 7fde2ef5a640 1 --2- 172.21.15.70:0/2802432294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 0x7fde280b15b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0097a0 tx=0x7fde1c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.982+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2802432294 shutdown_connections 2023-10-08T05:53:46.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.982+0000 7fde2ef5a640 1 --2- 172.21.15.70:0/2802432294 >> v2:172.21.15.181:3300/0 conn(0x7fde2806dfc0 0x7fde28070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.982+0000 7fde2ef5a640 1 --2- 172.21.15.70:0/2802432294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 0x7fde280b15b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.982+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2802432294 >> 172.21.15.70:0/2802432294 conn(0x7fde2806a5d0 msgr2=0x7fde280b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:46.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.982+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2802432294 shutdown_connections 2023-10-08T05:53:46.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.982+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2802432294 wait complete. 2023-10-08T05:53:46.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.984+0000 7fde2ef5a640 1 Processor -- start 2023-10-08T05:53:46.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.984+0000 7fde2ef5a640 1 -- start start 2023-10-08T05:53:46.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.985+0000 7fde2ef5a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde2806dfc0 0x7fde2815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:46.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.985+0000 7fde2ef5a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde28071070 0x7fde280ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:46.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.985+0000 7fde2ef5a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 0x7fde2815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:46.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.985+0000 7fde2ef5a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde280ceb30 con 0x7fde2806dfc0 2023-10-08T05:53:46.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.985+0000 7fde2ef5a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde280cee10 con 0x7fde28071070 2023-10-08T05:53:46.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.985+0000 7fde2ef5a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde280cf0f0 con 0x7fde280afbe0 2023-10-08T05:53:46.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.985+0000 7fde2ccf6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde2806dfc0 0x7fde2815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:46.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.985+0000 7fde2ccf6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde2806dfc0 0x7fde2815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51510/0 (socket says 172.21.15.70:51510) 2023-10-08T05:53:46.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.985+0000 7fde2ccf6640 1 -- 172.21.15.70:0/2829351490 learned_addr learned my addr 172.21.15.70:0/2829351490 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:46.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde2ccf6640 1 -- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 msgr2=0x7fde2815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:46.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde2d4f7640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 0x7fde2815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:46.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde27fff640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde28071070 0x7fde280ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:46.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde2ccf6640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 0x7fde2815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde2ccf6640 1 -- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde28071070 msgr2=0x7fde280ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:46.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde2ccf6640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde28071070 0x7fde280ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:46.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde2ccf6640 1 -- 172.21.15.70:0/2829351490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde18009400 con 0x7fde2806dfc0 2023-10-08T05:53:46.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde2d4f7640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 0x7fde2815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:46.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde2ccf6640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde2806dfc0 0x7fde2815d010 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fde1c000c00 tx=0x7fde1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:46.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde257fa640 1 -- 172.21.15.70:0/2829351490 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c006e10 con 0x7fde2806dfc0 2023-10-08T05:53:46.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.986+0000 7fde257fa640 1 -- 172.21.15.70:0/2829351490 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c03ad60 con 0x7fde2806dfc0 2023-10-08T05:53:46.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.987+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2829351490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c009400 con 0x7fde2806dfc0 2023-10-08T05:53:46.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.987+0000 7fde257fa640 1 -- 172.21.15.70:0/2829351490 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c044630 con 0x7fde2806dfc0 2023-10-08T05:53:46.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.987+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2829351490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde280cd380 con 0x7fde2806dfc0 2023-10-08T05:53:46.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.988+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2829351490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2804e160 con 0x7fde2806dfc0 2023-10-08T05:53:46.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.989+0000 7fde257fa640 1 -- 172.21.15.70:0/2829351490 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fde1c04d450 con 0x7fde2806dfc0 2023-10-08T05:53:46.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.990+0000 7fde257fa640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fde1406af90 0x7fde1406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:46.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.990+0000 7fde257fa640 1 -- 172.21.15.70:0/2829351490 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fde1c0abc50 con 0x7fde2806dfc0 2023-10-08T05:53:46.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.990+0000 7fde27fff640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fde1406af90 0x7fde1406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:46.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.991+0000 7fde27fff640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fde1406af90 0x7fde1406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fde280caf40 tx=0x7fde18008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:46.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:46.996+0000 7fde257fa640 1 -- 172.21.15.70:0/2829351490 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde1c076960 con 0x7fde2806dfc0 2023-10-08T05:53:47.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.203+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2829351490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fde280b0ef0 con 0x7fde2806dfc0 2023-10-08T05:53:47.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.749+0000 7fde257fa640 1 -- 172.21.15.70:0/2829351490 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 122+0+0 (secure 0 0 0) 0x7fde1c079f80 con 0x7fde2806dfc0 2023-10-08T05:53:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.752+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fde1406af90 msgr2=0x7fde1406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.752+0000 7fde2ef5a640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fde1406af90 0x7fde1406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fde280caf40 tx=0x7fde18008040 comp rx=0 tx=0).stop 2023-10-08T05:53:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.752+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2829351490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde2806dfc0 msgr2=0x7fde2815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.752+0000 7fde2ef5a640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde2806dfc0 0x7fde2815d010 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fde1c000c00 tx=0x7fde1c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:53:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.752+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2829351490 shutdown_connections 2023-10-08T05:53:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.752+0000 7fde2ef5a640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fde1406af90 0x7fde1406d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.752+0000 7fde2ef5a640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde280afbe0 0x7fde2815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:47.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.752+0000 7fde2ef5a640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde28071070 0x7fde280ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:47.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.752+0000 7fde2ef5a640 1 --2- 172.21.15.70:0/2829351490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde2806dfc0 0x7fde2815d010 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:47.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.752+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2829351490 >> 172.21.15.70:0/2829351490 conn(0x7fde2806a5d0 msgr2=0x7fde280c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:47.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.753+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2829351490 shutdown_connections 2023-10-08T05:53:47.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.753+0000 7fde2ef5a640 1 -- 172.21.15.70:0/2829351490 wait complete. 2023-10-08T05:53:47.774 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T05:53:47.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:53:47.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.983+0000 7f667a825640 1 Processor -- start 2023-10-08T05:53:47.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.984+0000 7f667a825640 1 -- start start 2023-10-08T05:53:47.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.985+0000 7f667a825640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66740b07d0 0x7f66741614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.985+0000 7f667a825640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66740c9020 con 0x7f66740cab40 2023-10-08T05:53:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.985+0000 7f667a825640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66740ccc20 con 0x7f667415cca0 2023-10-08T05:53:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.985+0000 7f667a825640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66740c92e0 con 0x7f66740b07d0 2023-10-08T05:53:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.985+0000 7f6679022640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66740b07d0 0x7f66741614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.985+0000 7f6679022640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66740b07d0 0x7f66741614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51512/0 (socket says 172.21.15.70:51512) 2023-10-08T05:53:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.985+0000 7f6679022640 1 -- 172.21.15.70:0/3025534763 learned_addr learned my addr 172.21.15.70:0/3025534763 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:47.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.985+0000 7f667a024640 1 --1- 172.21.15.70:0/3025534763 >> v1:172.21.15.70:6789/0 conn(0x7f66740cab40 0x7f66740c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34046/0 (socket says 172.21.15.70:34046) 2023-10-08T05:53:47.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.986+0000 7f6679022640 1 -- 172.21.15.70:0/3025534763 >> v1:172.21.15.181:6789/0 conn(0x7f667415cca0 legacy=0x7f66740b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:47.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.986+0000 7f6679022640 1 -- 172.21.15.70:0/3025534763 >> v1:172.21.15.70:6789/0 conn(0x7f66740cab40 legacy=0x7f66740c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:47.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.986+0000 7f6679022640 1 -- 172.21.15.70:0/3025534763 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66740cc7b0 con 0x7f66740b07d0 2023-10-08T05:53:47.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.986+0000 7f6678821640 1 -- 172.21.15.70:0/3025534763 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136048694 0 0) 0x7f66740c9020 con 0x7f66740cab40 2023-10-08T05:53:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.987+0000 7f6679022640 1 --2- 172.21.15.70:0/3025534763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 0x7f66741614b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f6664009870 tx=0x7f666402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bd6ee4b8819526 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.987+0000 7f6678821640 1 -- 172.21.15.70:0/3025534763 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664031870 con 0x7f66740b07d0 2023-10-08T05:53:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.987+0000 7f6678821640 1 -- 172.21.15.70:0/3025534763 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66640319d0 con 0x7f66740b07d0 2023-10-08T05:53:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.987+0000 7f6678821640 1 -- 172.21.15.70:0/3025534763 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664031d60 con 0x7f66740b07d0 2023-10-08T05:53:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.988+0000 7f667a825640 1 -- 172.21.15.70:0/3025534763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 msgr2=0x7f66741614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.988+0000 7f667a825640 1 --2- 172.21.15.70:0/3025534763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 0x7f66741614b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f6664009870 tx=0x7f666402b910 comp rx=0 tx=0).stop 2023-10-08T05:53:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.988+0000 7f667a825640 1 -- 172.21.15.70:0/3025534763 shutdown_connections 2023-10-08T05:53:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.988+0000 7f667a825640 1 --2- 172.21.15.70:0/3025534763 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 0x7f66741614b0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:47.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.988+0000 7f667a825640 1 -- 172.21.15.70:0/3025534763 >> 172.21.15.70:0/3025534763 conn(0x7f66740cb0c0 msgr2=0x7f66740cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:47.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.988+0000 7f667a825640 1 -- 172.21.15.70:0/3025534763 shutdown_connections 2023-10-08T05:53:47.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.988+0000 7f667a825640 1 -- 172.21.15.70:0/3025534763 wait complete. 2023-10-08T05:53:47.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.990+0000 7f667a825640 1 Processor -- start 2023-10-08T05:53:47.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.990+0000 7f667a825640 1 -- start start 2023-10-08T05:53:47.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.991+0000 7f667a825640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 0x7f667415e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:47.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.991+0000 7f667a825640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66740cab40 0x7f6674070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:47.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.991+0000 7f667a825640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f667415cca0 0x7f66741f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.991+0000 7f667a825640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667415fd40 con 0x7f66740b07d0 2023-10-08T05:53:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.991+0000 7f667a825640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667415f130 con 0x7f667415cca0 2023-10-08T05:53:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.991+0000 7f667a825640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667415f410 con 0x7f66740cab40 2023-10-08T05:53:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.991+0000 7f6679823640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 0x7f667415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.991+0000 7f6679823640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 0x7f667415e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51520/0 (socket says 172.21.15.70:51520) 2023-10-08T05:53:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.991+0000 7f6679823640 1 -- 172.21.15.70:0/4075232154 learned_addr learned my addr 172.21.15.70:0/4075232154 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f6679823640 1 -- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66740cab40 msgr2=0x7f6674070420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f6679823640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66740cab40 0x7f6674070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f6679823640 1 -- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f667415cca0 msgr2=0x7f66741f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f6679823640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f667415cca0 0x7f66741f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f6679823640 1 -- 172.21.15.70:0/4075232154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66640094d0 con 0x7f66740b07d0 2023-10-08T05:53:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f6679823640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 0x7f667415e830 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f667000c450 tx=0x7f667000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:47.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f666a7fc640 1 -- 172.21.15.70:0/4075232154 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66700052d0 con 0x7f66740b07d0 2023-10-08T05:53:47.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f666a7fc640 1 -- 172.21.15.70:0/4075232154 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6670005430 con 0x7f66740b07d0 2023-10-08T05:53:47.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f666a7fc640 1 -- 172.21.15.70:0/4075232154 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670012870 con 0x7f66740b07d0 2023-10-08T05:53:47.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f667a825640 1 -- 172.21.15.70:0/4075232154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667415f6f0 con 0x7f66740b07d0 2023-10-08T05:53:47.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.992+0000 7f667a825640 1 -- 172.21.15.70:0/4075232154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667415ed70 con 0x7f66740b07d0 2023-10-08T05:53:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.993+0000 7f667a825640 1 -- 172.21.15.70:0/4075232154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667415ef00 con 0x7f66740b07d0 2023-10-08T05:53:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.994+0000 7f666a7fc640 1 -- 172.21.15.70:0/4075232154 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f66700129d0 con 0x7f66740b07d0 2023-10-08T05:53:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.995+0000 7f666a7fc640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f665806af90 0x7f665806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.995+0000 7f666a7fc640 1 -- 172.21.15.70:0/4075232154 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6670089570 con 0x7f66740b07d0 2023-10-08T05:53:47.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.996+0000 7f6679022640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f665806af90 0x7f665806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:47.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:47.997+0000 7f6679022640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f665806af90 0x7f665806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6664036000 tx=0x7f6664030000 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:48.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.000+0000 7f666a7fc640 1 -- 172.21.15.70:0/4075232154 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6670054270 con 0x7f66740b07d0 2023-10-08T05:53:48.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.225+0000 7f667a825640 1 -- 172.21.15.70:0/4075232154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f667404e160 con 0x7f66740b07d0 2023-10-08T05:53:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.226+0000 7f666a7fc640 1 -- 172.21.15.70:0/4075232154 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f6670057890 con 0x7f66740b07d0 2023-10-08T05:53:48.226 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:48.226 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/290122525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":290122525},{"type":"v1","addr":"172.21.15.70:6835","nonce":290122525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4017950051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4017950051},{"type":"v1","addr":"172.21.15.181:6837","nonce":4017950051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1329101557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1329101557},{"type":"v1","addr":"172.21.15.70:6834","nonce":1329101557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:44.125639+0000","modified":"2023-10-08T05:53:47.206011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6184":{"gid":6184,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/422082772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":422082772},{"type":"v1","addr":"172.21.15.181:6835","nonce":422082772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-08T05:53:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 50 2023-10-08T05:53:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 -- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f665806af90 msgr2=0x7f665806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f665806af90 0x7f665806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6664036000 tx=0x7f6664030000 comp rx=0 tx=0).stop 2023-10-08T05:53:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 -- 172.21.15.70:0/4075232154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 msgr2=0x7f667415e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 0x7f667415e830 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f667000c450 tx=0x7f667000c910 comp rx=0 tx=0).stop 2023-10-08T05:53:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 -- 172.21.15.70:0/4075232154 shutdown_connections 2023-10-08T05:53:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f665806af90 0x7f665806d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f667415cca0 0x7f66741f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66740cab40 0x7f6674070420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 --2- 172.21.15.70:0/4075232154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66740b07d0 0x7f667415e830 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 -- 172.21.15.70:0/4075232154 >> 172.21.15.70:0/4075232154 conn(0x7f66740cb0c0 msgr2=0x7f667406cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:48.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 -- 172.21.15.70:0/4075232154 shutdown_connections 2023-10-08T05:53:48.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.228+0000 7f667a825640 1 -- 172.21.15.70:0/4075232154 wait complete. 2023-10-08T05:53:48.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:53:48.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.462+0000 7f0b44887640 1 Processor -- start 2023-10-08T05:53:48.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.462+0000 7f0b44887640 1 -- start start 2023-10-08T05:53:48.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.463+0000 7f0b44887640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0b400cb880 0x7f0b400b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.463+0000 7f0b44887640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0b400cbce0 0x7f0b400b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.463+0000 7f0b44887640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b400ce550 con 0x7f0b40069d50 2023-10-08T05:53:48.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.463+0000 7f0b44887640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b401609d0 con 0x7f0b400cbce0 2023-10-08T05:53:48.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.463+0000 7f0b44887640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b400cdcf0 con 0x7f0b400cb880 2023-10-08T05:53:48.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3e59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0b40069d50 0x7f0b4006a130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34050/0 (socket says 172.21.15.70:34050) 2023-10-08T05:53:48.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3e59c640 1 -- 172.21.15.70:0/553390693 learned_addr learned my addr 172.21.15.70:0/553390693 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:48.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3dd9b640 1 --2- 172.21.15.70:0/553390693 >> v2:172.21.15.181:3300/0 conn(0x7f0b400cb880 0x7f0b400b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284107890 0 0) 0x7f0b400ce550 con 0x7f0b40069d50 2023-10-08T05:53:48.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3ed9d640 1 --2- 172.21.15.70:0/553390693 >> v2:172.21.15.70:3300/0 conn(0x7f0b400cbce0 0x7f0b400b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b28003520 con 0x7f0b40069d50 2023-10-08T05:53:48.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1710706277 0 0) 0x7f0b28003520 con 0x7f0b40069d50 2023-10-08T05:53:48.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 >> v2:172.21.15.181:3300/0 conn(0x7f0b400cb880 msgr2=0x7f0b400b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:48.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3d59a640 1 --2- 172.21.15.70:0/553390693 >> v2:172.21.15.181:3300/0 conn(0x7f0b400cb880 0x7f0b400b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 >> v2:172.21.15.70:3300/0 conn(0x7f0b400cbce0 msgr2=0x7f0b400b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:48.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3d59a640 1 --2- 172.21.15.70:0/553390693 >> v2:172.21.15.70:3300/0 conn(0x7f0b400cbce0 0x7f0b400b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400cdf90 con 0x7f0b40069d50 2023-10-08T05:53:48.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0b30001670 con 0x7f0b40069d50 2023-10-08T05:53:48.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.464+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 >> v1:172.21.15.70:6789/0 conn(0x7f0b40069d50 legacy=0x7f0b4006a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3d59a640 1 --2- 172.21.15.70:0/553390693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b28003970 0x7f0b28003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3d59a640 1 --2- 172.21.15.70:0/553390693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b280047c0 0x7f0b28004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3d59a640 1 --2- 172.21.15.70:0/553390693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b28005180 0x7f0b2800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b401609d0 con 0x7f0b280047c0 2023-10-08T05:53:48.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400cdcf0 con 0x7f0b28003970 2023-10-08T05:53:48.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2800c080 con 0x7f0b28005180 2023-10-08T05:53:48.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3dd9b640 1 --2- 172.21.15.70:0/553390693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b280047c0 0x7f0b28004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3dd9b640 1 -- 172.21.15.70:0/553390693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b28005180 msgr2=0x7f0b2800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:48.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3dd9b640 1 --2- 172.21.15.70:0/553390693 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b28005180 0x7f0b2800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3dd9b640 1 -- 172.21.15.70:0/553390693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b28003970 msgr2=0x7f0b28003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:48.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3dd9b640 1 --2- 172.21.15.70:0/553390693 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b28003970 0x7f0b28003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.465+0000 7f0b3dd9b640 1 -- 172.21.15.70:0/553390693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b28003520 con 0x7f0b280047c0 2023-10-08T05:53:48.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.466+0000 7f0b3ed9d640 1 -- 172.21.15.70:0/553390693 reap_dead start 2023-10-08T05:53:48.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.466+0000 7f0b3dd9b640 1 --2- 172.21.15.70:0/553390693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b280047c0 0x7f0b28004c40 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400ba30 tx=0x7f0b280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcda7cb5e4fd041c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:48.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.467+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34032650 con 0x7f0b280047c0 2023-10-08T05:53:48.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.467+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b340327b0 con 0x7f0b280047c0 2023-10-08T05:53:48.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.467+0000 7f0b3d59a640 1 -- 172.21.15.70:0/553390693 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34032b40 con 0x7f0b280047c0 2023-10-08T05:53:48.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.467+0000 7f0b44887640 1 -- 172.21.15.70:0/553390693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b280047c0 msgr2=0x7f0b28004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.467+0000 7f0b44887640 1 --2- 172.21.15.70:0/553390693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b280047c0 0x7f0b28004c40 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400ba30 tx=0x7f0b280057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.467+0000 7f0b44887640 1 -- 172.21.15.70:0/553390693 shutdown_connections 2023-10-08T05:53:48.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.467+0000 7f0b44887640 1 --2- 172.21.15.70:0/553390693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b280047c0 0x7f0b28004c40 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.467+0000 7f0b44887640 1 -- 172.21.15.70:0/553390693 >> 172.21.15.70:0/553390693 conn(0x7f0b4006d450 msgr2=0x7f0b4006d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:48.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.467+0000 7f0b44887640 1 -- 172.21.15.70:0/553390693 shutdown_connections 2023-10-08T05:53:48.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.468+0000 7f0b44887640 1 -- 172.21.15.70:0/553390693 wait complete. 2023-10-08T05:53:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.469+0000 7f0b44887640 1 Processor -- start 2023-10-08T05:53:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.469+0000 7f0b44887640 1 -- start start 2023-10-08T05:53:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.470+0000 7f0b44887640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b280047c0 0x7f0b4014dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.470+0000 7f0b44887640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b4015d2f0 0x7f0b4015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.470+0000 7f0b44887640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b4015d770 0x7f0b4015f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.470+0000 7f0b44887640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4015f8d0 con 0x7f0b4015d2f0 2023-10-08T05:53:48.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.470+0000 7f0b44887640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400b0580 con 0x7f0b280047c0 2023-10-08T05:53:48.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.470+0000 7f0b44887640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400b0860 con 0x7f0b4015d770 2023-10-08T05:53:48.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.471+0000 7f0b3dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b4015d2f0 0x7f0b4015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.471+0000 7f0b3dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b4015d2f0 0x7f0b4015ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51550/0 (socket says 172.21.15.70:51550) 2023-10-08T05:53:48.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.471+0000 7f0b3dd9b640 1 -- 172.21.15.70:0/2326905528 learned_addr learned my addr 172.21.15.70:0/2326905528 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:48.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.471+0000 7f0b3e59c640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b280047c0 0x7f0b4014dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.471+0000 7f0b3ed9d640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b4015d770 0x7f0b4015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.471+0000 7f0b3dd9b640 1 -- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b4015d770 msgr2=0x7f0b4015f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.471+0000 7f0b3dd9b640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b4015d770 0x7f0b4015f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.471+0000 7f0b3dd9b640 1 -- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b280047c0 msgr2=0x7f0b4014dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.471+0000 7f0b3dd9b640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b280047c0 0x7f0b4014dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.471+0000 7f0b3dd9b640 1 -- 172.21.15.70:0/2326905528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400b0b40 con 0x7f0b4015d2f0 2023-10-08T05:53:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.472+0000 7f0b3dd9b640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b4015d2f0 0x7f0b4015ee50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400bb50 tx=0x7f0b34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.472+0000 7f0b2effd640 1 -- 172.21.15.70:0/2326905528 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34032e10 con 0x7f0b4015d2f0 2023-10-08T05:53:48.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.472+0000 7f0b2effd640 1 -- 172.21.15.70:0/2326905528 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3403b630 con 0x7f0b4015d2f0 2023-10-08T05:53:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.472+0000 7f0b44887640 1 -- 172.21.15.70:0/2326905528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b400b2780 con 0x7f0b4015d2f0 2023-10-08T05:53:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.472+0000 7f0b2effd640 1 -- 172.21.15.70:0/2326905528 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3403b9e0 con 0x7f0b4015d2f0 2023-10-08T05:53:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.472+0000 7f0b44887640 1 -- 172.21.15.70:0/2326905528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4015a5a0 con 0x7f0b4015d2f0 2023-10-08T05:53:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.473+0000 7f0b44887640 1 -- 172.21.15.70:0/2326905528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b40067490 con 0x7f0b4015d2f0 2023-10-08T05:53:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.474+0000 7f0b2effd640 1 -- 172.21.15.70:0/2326905528 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0b3402e070 con 0x7f0b4015d2f0 2023-10-08T05:53:48.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.475+0000 7f0b2effd640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f0b2006afc0 0x7f0b2006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.476+0000 7f0b2effd640 1 -- 172.21.15.70:0/2326905528 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0b340ab4a0 con 0x7f0b4015d2f0 2023-10-08T05:53:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.476+0000 7f0b3e59c640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f0b2006afc0 0x7f0b2006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.477+0000 7f0b3e59c640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f0b2006afc0 0x7f0b2006d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b30001eb0 tx=0x7f0b300016a0 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:48.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.480+0000 7f0b2effd640 1 -- 172.21.15.70:0/2326905528 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b34079bb0 con 0x7f0b4015d2f0 2023-10-08T05:53:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.671+0000 7f0b44887640 1 -- 172.21.15.70:0/2326905528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b400b5810 con 0x7f0b4015d2f0 2023-10-08T05:53:48.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.674+0000 7f0b2effd640 1 -- 172.21.15.70:0/2326905528 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16786 (secure 0 0 0) 0x7f0b34006c30 con 0x7f0b4015d2f0 2023-10-08T05:53:48.675 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:48.675 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":118,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:53:45.715955+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:42.052166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:43.161991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:53:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 -- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f0b2006afc0 msgr2=0x7f0b2006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f0b2006afc0 0x7f0b2006d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b30001eb0 tx=0x7f0b300016a0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 -- 172.21.15.70:0/2326905528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b4015d2f0 msgr2=0x7f0b4015ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b4015d2f0 0x7f0b4015ee50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400bb50 tx=0x7f0b34004730 comp rx=0 tx=0).stop 2023-10-08T05:53:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 -- 172.21.15.70:0/2326905528 shutdown_connections 2023-10-08T05:53:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f0b2006afc0 0x7f0b2006d480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b4015d770 0x7f0b4015f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b4015d2f0 0x7f0b4015ee50 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 --2- 172.21.15.70:0/2326905528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b280047c0 0x7f0b4014dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 -- 172.21.15.70:0/2326905528 >> 172.21.15.70:0/2326905528 conn(0x7f0b4006d450 msgr2=0x7f0b400c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 -- 172.21.15.70:0/2326905528 shutdown_connections 2023-10-08T05:53:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.677+0000 7f0b44887640 1 -- 172.21.15.70:0/2326905528 wait complete. 2023-10-08T05:53:48.695 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T05:53:48.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:53:48.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.906+0000 7f41381b0640 1 Processor -- start 2023-10-08T05:53:48.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.906+0000 7f41381b0640 1 -- start start 2023-10-08T05:53:48.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.908+0000 7f41381b0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4130159ac0 0x7f413006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.908+0000 7f41381b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300709e0 0x7f41300c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.908+0000 7f41381b0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413006dee0 con 0x7f41300ad790 2023-10-08T05:53:48.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.908+0000 7f41381b0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413015d850 con 0x7f4130159ac0 2023-10-08T05:53:48.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.908+0000 7f41381b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413015db30 con 0x7f41300709e0 2023-10-08T05:53:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.908+0000 7f413674d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300709e0 0x7f41300c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f413674d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300709e0 0x7f41300c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53200/0 (socket says 172.21.15.70:53200) 2023-10-08T05:53:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f413574b640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f41300ad790 0x7f41300702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:55916/0 (socket says 172.21.15.70:55916) 2023-10-08T05:53:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f413674d640 1 -- 172.21.15.70:0/2915696788 learned_addr learned my addr 172.21.15.70:0/2915696788 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f4135f4c640 1 --2- 172.21.15.70:0/2915696788 >> v2:172.21.15.181:3300/0 conn(0x7f4130159ac0 0x7f413006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f4134f4a640 1 -- 172.21.15.70:0/2915696788 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672510407 0 0) 0x7f413006dee0 con 0x7f41300ad790 2023-10-08T05:53:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f4134f4a640 1 -- 172.21.15.70:0/2915696788 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f411c003520 con 0x7f41300ad790 2023-10-08T05:53:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f4135f4c640 1 -- 172.21.15.70:0/2915696788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300709e0 msgr2=0x7f41300c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f4135f4c640 1 --2- 172.21.15.70:0/2915696788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300709e0 0x7f41300c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f4135f4c640 1 -- 172.21.15.70:0/2915696788 >> v1:172.21.15.181:6789/0 conn(0x7f41300ad790 legacy=0x7f41300702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f4135f4c640 1 -- 172.21.15.70:0/2915696788 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413015de10 con 0x7f4130159ac0 2023-10-08T05:53:48.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.909+0000 7f413674d640 1 --2- 172.21.15.70:0/2915696788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300709e0 0x7f41300c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:53:48.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.911+0000 7f4135f4c640 1 --2- 172.21.15.70:0/2915696788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4130159ac0 0x7f413006d5e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f412c0097a0 tx=0x7f412c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69ddc0ba13ba989a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:48.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.911+0000 7f4134f4a640 1 -- 172.21.15.70:0/2915696788 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c02d040 con 0x7f4130159ac0 2023-10-08T05:53:48.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.911+0000 7f4134f4a640 1 -- 172.21.15.70:0/2915696788 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c031810 con 0x7f4130159ac0 2023-10-08T05:53:48.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.911+0000 7f4134f4a640 1 -- 172.21.15.70:0/2915696788 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c031ba0 con 0x7f4130159ac0 2023-10-08T05:53:48.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.912+0000 7f41381b0640 1 -- 172.21.15.70:0/2915696788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4130159ac0 msgr2=0x7f413006d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.912+0000 7f41381b0640 1 --2- 172.21.15.70:0/2915696788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4130159ac0 0x7f413006d5e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f412c0097a0 tx=0x7f412c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.912+0000 7f41381b0640 1 -- 172.21.15.70:0/2915696788 shutdown_connections 2023-10-08T05:53:48.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.912+0000 7f41381b0640 1 --2- 172.21.15.70:0/2915696788 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300709e0 0x7f41300c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.912+0000 7f41381b0640 1 --2- 172.21.15.70:0/2915696788 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4130159ac0 0x7f413006d5e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.912+0000 7f41381b0640 1 -- 172.21.15.70:0/2915696788 >> 172.21.15.70:0/2915696788 conn(0x7f413006a380 msgr2=0x7f41300b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:48.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.912+0000 7f41381b0640 1 -- 172.21.15.70:0/2915696788 shutdown_connections 2023-10-08T05:53:48.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.912+0000 7f41381b0640 1 -- 172.21.15.70:0/2915696788 wait complete. 2023-10-08T05:53:48.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.914+0000 7f41381b0640 1 Processor -- start 2023-10-08T05:53:48.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.914+0000 7f41381b0640 1 -- start start 2023-10-08T05:53:48.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f41381b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41300709e0 0x7f41300b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f41381b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300ad790 0x7f4130147ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f41381b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4130159ac0 0x7f4130143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f41381b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300b19e0 con 0x7f41300709e0 2023-10-08T05:53:48.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f41381b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300b4660 con 0x7f4130159ac0 2023-10-08T05:53:48.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f41381b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4130153e20 con 0x7f41300ad790 2023-10-08T05:53:48.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f4135f4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41300709e0 0x7f41300b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f4135f4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41300709e0 0x7f41300b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51556/0 (socket says 172.21.15.70:51556) 2023-10-08T05:53:48.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f4135f4c640 1 -- 172.21.15.70:0/3453379086 learned_addr learned my addr 172.21.15.70:0/3453379086 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:48.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f413574b640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300ad790 0x7f4130147ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f413674d640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4130159ac0 0x7f4130143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f4135f4c640 1 -- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300ad790 msgr2=0x7f4130147ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f4135f4c640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300ad790 0x7f4130147ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f4135f4c640 1 -- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4130159ac0 msgr2=0x7f4130143f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:48.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f4135f4c640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4130159ac0 0x7f4130143f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:48.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f4135f4c640 1 -- 172.21.15.70:0/3453379086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412800bd40 con 0x7f41300709e0 2023-10-08T05:53:48.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f413574b640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300ad790 0x7f4130147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:48.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.915+0000 7f413674d640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4130159ac0 0x7f4130143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:48.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.916+0000 7f4135f4c640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41300709e0 0x7f41300b14a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f412c000c00 tx=0x7f412c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:48.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.916+0000 7f41267fc640 1 -- 172.21.15.70:0/3453379086 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c03ad60 con 0x7f41300709e0 2023-10-08T05:53:48.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.916+0000 7f41267fc640 1 -- 172.21.15.70:0/3453379086 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c0443f0 con 0x7f41300709e0 2023-10-08T05:53:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.916+0000 7f41267fc640 1 -- 172.21.15.70:0/3453379086 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c0445a0 con 0x7f41300709e0 2023-10-08T05:53:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.916+0000 7f41381b0640 1 -- 172.21.15.70:0/3453379086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412c009400 con 0x7f41300709e0 2023-10-08T05:53:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.916+0000 7f41381b0640 1 -- 172.21.15.70:0/3453379086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41301543e0 con 0x7f41300709e0 2023-10-08T05:53:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.917+0000 7f41381b0640 1 -- 172.21.15.70:0/3453379086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4130159ea0 con 0x7f41300709e0 2023-10-08T05:53:48.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.918+0000 7f41267fc640 1 -- 172.21.15.70:0/3453379086 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f412c04d450 con 0x7f41300709e0 2023-10-08T05:53:48.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.919+0000 7f41267fc640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f411406af90 0x7f411406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:48.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.919+0000 7f41267fc640 1 -- 172.21.15.70:0/3453379086 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f412c0abbb0 con 0x7f41300709e0 2023-10-08T05:53:48.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.920+0000 7f413574b640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f411406af90 0x7f411406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:48.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.921+0000 7f413574b640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f411406af90 0x7f411406d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4130155830 tx=0x7f4120009040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:48.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:48.923+0000 7f41267fc640 1 -- 172.21.15.70:0/3453379086 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f412c076830 con 0x7f41300709e0 2023-10-08T05:53:49.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.108+0000 7f41381b0640 1 -- 172.21.15.70:0/3453379086 --> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4130067490 con 0x7f411406af90 2023-10-08T05:53:49.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.138+0000 7f41267fc640 1 -- 172.21.15.70:0/3453379086 <== mgr.5513 v2:172.21.15.181:6832/1906631919 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287573 (secure 0 0 0) 0x7f4130067490 con 0x7f411406af90 2023-10-08T05:53:49.139 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:49.143 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:53:49.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 -- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f411406af90 msgr2=0x7f411406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f411406af90 0x7f411406d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4130155830 tx=0x7f4120009040 comp rx=0 tx=0).stop 2023-10-08T05:53:49.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 -- 172.21.15.70:0/3453379086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41300709e0 msgr2=0x7f41300b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41300709e0 0x7f41300b14a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f412c000c00 tx=0x7f412c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 -- 172.21.15.70:0/3453379086 shutdown_connections 2023-10-08T05:53:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f411406af90 0x7f411406d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4130159ac0 0x7f4130143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41300ad790 0x7f4130147ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 --2- 172.21.15.70:0/3453379086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41300709e0 0x7f41300b14a0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 -- 172.21.15.70:0/3453379086 >> 172.21.15.70:0/3453379086 conn(0x7f413006a380 msgr2=0x7f41300af460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.144+0000 7f41381b0640 1 -- 172.21.15.70:0/3453379086 shutdown_connections 2023-10-08T05:53:49.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.145+0000 7f41381b0640 1 -- 172.21.15.70:0/3453379086 wait complete. 2023-10-08T05:53:49.165 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-08T05:53:47.751480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":765,"num_read_kb":1528,"num_write":333,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2679,"ondisk_log_size":2679,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":318968,"kb_used_data":12584,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754655752,"statfs":{"total":773094113280,"available":772767490048,"internally_reserved":0,"allocated":12886016,"data_stored":7071780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":313577019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.080724"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.773202+0000","last_change":"2023-10-08T05:53:44.732254+0000","last_active":"2023-10-08T05:53:45.773202+0000","last_peered":"2023-10-08T05:53:45.773202+0000","last_clean":"2023-10-08T05:53:45.773202+0000","last_became_active":"2023-10-08T05:53:44.731866+0000","last_became_peered":"2023-10-08T05:53:44.731866+0000","last_unstale":"2023-10-08T05:53:45.773202+0000","last_undegraded":"2023-10-08T05:53:45.773202+0000","last_fullsized":"2023-10-08T05:53:45.773202+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:07:40.057790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736686+0000","last_change":"2023-10-08T05:53:44.734557+0000","last_active":"2023-10-08T05:53:45.736686+0000","last_peered":"2023-10-08T05:53:45.736686+0000","last_clean":"2023-10-08T05:53:45.736686+0000","last_became_active":"2023-10-08T05:53:44.734245+0000","last_became_peered":"2023-10-08T05:53:44.734245+0000","last_unstale":"2023-10-08T05:53:45.736686+0000","last_undegraded":"2023-10-08T05:53:45.736686+0000","last_fullsized":"2023-10-08T05:53:45.736686+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:38.558149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772630+0000","last_change":"2023-10-08T05:53:44.751821+0000","last_active":"2023-10-08T05:53:45.772630+0000","last_peered":"2023-10-08T05:53:45.772630+0000","last_clean":"2023-10-08T05:53:45.772630+0000","last_became_active":"2023-10-08T05:53:44.751512+0000","last_became_peered":"2023-10-08T05:53:44.751512+0000","last_unstale":"2023-10-08T05:53:45.772630+0000","last_undegraded":"2023-10-08T05:53:45.772630+0000","last_fullsized":"2023-10-08T05:53:45.772630+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:05:54.492792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768827+0000","last_change":"2023-10-08T05:53:44.740032+0000","last_active":"2023-10-08T05:53:45.768827+0000","last_peered":"2023-10-08T05:53:45.768827+0000","last_clean":"2023-10-08T05:53:45.768827+0000","last_became_active":"2023-10-08T05:53:44.739367+0000","last_became_peered":"2023-10-08T05:53:44.739367+0000","last_unstale":"2023-10-08T05:53:45.768827+0000","last_undegraded":"2023-10-08T05:53:45.768827+0000","last_fullsized":"2023-10-08T05:53:45.768827+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:38:10.428853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769757+0000","last_change":"2023-10-08T05:53:44.737746+0000","last_active":"2023-10-08T05:53:45.769757+0000","last_peered":"2023-10-08T05:53:45.769757+0000","last_clean":"2023-10-08T05:53:45.769757+0000","last_became_active":"2023-10-08T05:53:44.737418+0000","last_became_peered":"2023-10-08T05:53:44.737418+0000","last_unstale":"2023-10-08T05:53:45.769757+0000","last_undegraded":"2023-10-08T05:53:45.769757+0000","last_fullsized":"2023-10-08T05:53:45.769757+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:11.862363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769975+0000","last_change":"2023-10-08T05:53:44.735914+0000","last_active":"2023-10-08T05:53:45.769975+0000","last_peered":"2023-10-08T05:53:45.769975+0000","last_clean":"2023-10-08T05:53:45.769975+0000","last_became_active":"2023-10-08T05:53:44.735446+0000","last_became_peered":"2023-10-08T05:53:44.735446+0000","last_unstale":"2023-10-08T05:53:45.769975+0000","last_undegraded":"2023-10-08T05:53:45.769975+0000","last_fullsized":"2023-10-08T05:53:45.769975+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:31.564091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749893+0000","last_change":"2023-10-08T05:53:44.735531+0000","last_active":"2023-10-08T05:53:45.749893+0000","last_peered":"2023-10-08T05:53:45.749893+0000","last_clean":"2023-10-08T05:53:45.749893+0000","last_became_active":"2023-10-08T05:53:44.733259+0000","last_became_peered":"2023-10-08T05:53:44.733259+0000","last_unstale":"2023-10-08T05:53:45.749893+0000","last_undegraded":"2023-10-08T05:53:45.749893+0000","last_fullsized":"2023-10-08T05:53:45.749893+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:25.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768328+0000","last_change":"2023-10-08T05:53:44.743714+0000","last_active":"2023-10-08T05:53:45.768328+0000","last_peered":"2023-10-08T05:53:45.768328+0000","last_clean":"2023-10-08T05:53:45.768328+0000","last_became_active":"2023-10-08T05:53:44.743190+0000","last_became_peered":"2023-10-08T05:53:44.743190+0000","last_unstale":"2023-10-08T05:53:45.768328+0000","last_undegraded":"2023-10-08T05:53:45.768328+0000","last_fullsized":"2023-10-08T05:53:45.768328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:18.838107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769733+0000","last_change":"2023-10-08T05:53:44.730820+0000","last_active":"2023-10-08T05:53:45.769733+0000","last_peered":"2023-10-08T05:53:45.769733+0000","last_clean":"2023-10-08T05:53:45.769733+0000","last_became_active":"2023-10-08T05:53:44.730554+0000","last_became_peered":"2023-10-08T05:53:44.730554+0000","last_unstale":"2023-10-08T05:53:45.769733+0000","last_undegraded":"2023-10-08T05:53:45.769733+0000","last_fullsized":"2023-10-08T05:53:45.769733+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:13.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.730270+0000","last_change":"2023-10-08T05:53:44.737666+0000","last_active":"2023-10-08T05:53:45.730270+0000","last_peered":"2023-10-08T05:53:45.730270+0000","last_clean":"2023-10-08T05:53:45.730270+0000","last_became_active":"2023-10-08T05:53:44.737405+0000","last_became_peered":"2023-10-08T05:53:44.737405+0000","last_unstale":"2023-10-08T05:53:45.730270+0000","last_undegraded":"2023-10-08T05:53:45.730270+0000","last_fullsized":"2023-10-08T05:53:45.730270+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:03.246411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770006+0000","last_change":"2023-10-08T05:53:44.734979+0000","last_active":"2023-10-08T05:53:45.770006+0000","last_peered":"2023-10-08T05:53:45.770006+0000","last_clean":"2023-10-08T05:53:45.770006+0000","last_became_active":"2023-10-08T05:53:44.734683+0000","last_became_peered":"2023-10-08T05:53:44.734683+0000","last_unstale":"2023-10-08T05:53:45.770006+0000","last_undegraded":"2023-10-08T05:53:45.770006+0000","last_fullsized":"2023-10-08T05:53:45.770006+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:18.838107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.730233+0000","last_change":"2023-10-08T05:53:44.733196+0000","last_active":"2023-10-08T05:53:45.730233+0000","last_peered":"2023-10-08T05:53:45.730233+0000","last_clean":"2023-10-08T05:53:45.730233+0000","last_became_active":"2023-10-08T05:53:44.732929+0000","last_became_peered":"2023-10-08T05:53:44.732929+0000","last_unstale":"2023-10-08T05:53:45.730233+0000","last_undegraded":"2023-10-08T05:53:45.730233+0000","last_fullsized":"2023-10-08T05:53:45.730233+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:42.779477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769219+0000","last_change":"2023-10-08T05:53:44.742293+0000","last_active":"2023-10-08T05:53:45.769219+0000","last_peered":"2023-10-08T05:53:45.769219+0000","last_clean":"2023-10-08T05:53:45.769219+0000","last_became_active":"2023-10-08T05:53:44.742012+0000","last_became_peered":"2023-10-08T05:53:44.742012+0000","last_unstale":"2023-10-08T05:53:45.769219+0000","last_undegraded":"2023-10-08T05:53:45.769219+0000","last_fullsized":"2023-10-08T05:53:45.769219+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:40:51.342521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769458+0000","last_change":"2023-10-08T05:53:44.748135+0000","last_active":"2023-10-08T05:53:45.769458+0000","last_peered":"2023-10-08T05:53:45.769458+0000","last_clean":"2023-10-08T05:53:45.769458+0000","last_became_active":"2023-10-08T05:53:44.747810+0000","last_became_peered":"2023-10-08T05:53:44.747810+0000","last_unstale":"2023-10-08T05:53:45.769458+0000","last_undegraded":"2023-10-08T05:53:45.769458+0000","last_fullsized":"2023-10-08T05:53:45.769458+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:59.155842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770593+0000","last_change":"2023-10-08T05:53:44.736760+0000","last_active":"2023-10-08T05:53:45.770593+0000","last_peered":"2023-10-08T05:53:45.770593+0000","last_clean":"2023-10-08T05:53:45.770593+0000","last_became_active":"2023-10-08T05:53:44.736481+0000","last_became_peered":"2023-10-08T05:53:44.736481+0000","last_unstale":"2023-10-08T05:53:45.770593+0000","last_undegraded":"2023-10-08T05:53:45.770593+0000","last_fullsized":"2023-10-08T05:53:45.770593+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:39.108631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769354+0000","last_change":"2023-10-08T05:53:44.739039+0000","last_active":"2023-10-08T05:53:45.769354+0000","last_peered":"2023-10-08T05:53:45.769354+0000","last_clean":"2023-10-08T05:53:45.769354+0000","last_became_active":"2023-10-08T05:53:44.738624+0000","last_became_peered":"2023-10-08T05:53:44.738624+0000","last_unstale":"2023-10-08T05:53:45.769354+0000","last_undegraded":"2023-10-08T05:53:45.769354+0000","last_fullsized":"2023-10-08T05:53:45.769354+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:35.705136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772315+0000","last_change":"2023-10-08T05:53:44.741912+0000","last_active":"2023-10-08T05:53:45.772315+0000","last_peered":"2023-10-08T05:53:45.772315+0000","last_clean":"2023-10-08T05:53:45.772315+0000","last_became_active":"2023-10-08T05:53:44.740692+0000","last_became_peered":"2023-10-08T05:53:44.740692+0000","last_unstale":"2023-10-08T05:53:45.772315+0000","last_undegraded":"2023-10-08T05:53:45.772315+0000","last_fullsized":"2023-10-08T05:53:45.772315+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:16:13.976999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772859+0000","last_change":"2023-10-08T05:53:44.731955+0000","last_active":"2023-10-08T05:53:45.772859+0000","last_peered":"2023-10-08T05:53:45.772859+0000","last_clean":"2023-10-08T05:53:45.772859+0000","last_became_active":"2023-10-08T05:53:44.731619+0000","last_became_peered":"2023-10-08T05:53:44.731619+0000","last_unstale":"2023-10-08T05:53:45.772859+0000","last_undegraded":"2023-10-08T05:53:45.772859+0000","last_fullsized":"2023-10-08T05:53:45.772859+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:29:19.871040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770222+0000","last_change":"2023-10-08T05:53:44.747971+0000","last_active":"2023-10-08T05:53:45.770222+0000","last_peered":"2023-10-08T05:53:45.770222+0000","last_clean":"2023-10-08T05:53:45.770222+0000","last_became_active":"2023-10-08T05:53:44.746820+0000","last_became_peered":"2023-10-08T05:53:44.746820+0000","last_unstale":"2023-10-08T05:53:45.770222+0000","last_undegraded":"2023-10-08T05:53:45.770222+0000","last_fullsized":"2023-10-08T05:53:45.770222+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:28.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769127+0000","last_change":"2023-10-08T05:53:44.738631+0000","last_active":"2023-10-08T05:53:45.769127+0000","last_peered":"2023-10-08T05:53:45.769127+0000","last_clean":"2023-10-08T05:53:45.769127+0000","last_became_active":"2023-10-08T05:53:44.738297+0000","last_became_peered":"2023-10-08T05:53:44.738297+0000","last_unstale":"2023-10-08T05:53:45.769127+0000","last_undegraded":"2023-10-08T05:53:45.769127+0000","last_fullsized":"2023-10-08T05:53:45.769127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:05:51.896763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769227+0000","last_change":"2023-10-08T05:53:44.737940+0000","last_active":"2023-10-08T05:53:45.769227+0000","last_peered":"2023-10-08T05:53:45.769227+0000","last_clean":"2023-10-08T05:53:45.769227+0000","last_became_active":"2023-10-08T05:53:44.737614+0000","last_became_peered":"2023-10-08T05:53:44.737614+0000","last_unstale":"2023-10-08T05:53:45.769227+0000","last_undegraded":"2023-10-08T05:53:45.769227+0000","last_fullsized":"2023-10-08T05:53:45.769227+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:51:26.444102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769217+0000","last_change":"2023-10-08T05:53:44.743404+0000","last_active":"2023-10-08T05:53:45.769217+0000","last_peered":"2023-10-08T05:53:45.769217+0000","last_clean":"2023-10-08T05:53:45.769217+0000","last_became_active":"2023-10-08T05:53:44.741563+0000","last_became_peered":"2023-10-08T05:53:44.741563+0000","last_unstale":"2023-10-08T05:53:45.769217+0000","last_undegraded":"2023-10-08T05:53:45.769217+0000","last_fullsized":"2023-10-08T05:53:45.769217+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:53.829681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.734832+0000","last_change":"2023-10-08T05:53:44.742183+0000","last_active":"2023-10-08T05:53:45.734832+0000","last_peered":"2023-10-08T05:53:45.734832+0000","last_clean":"2023-10-08T05:53:45.734832+0000","last_became_active":"2023-10-08T05:53:44.741849+0000","last_became_peered":"2023-10-08T05:53:44.741849+0000","last_unstale":"2023-10-08T05:53:45.734832+0000","last_undegraded":"2023-10-08T05:53:45.734832+0000","last_fullsized":"2023-10-08T05:53:45.734832+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:27.089669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772806+0000","last_change":"2023-10-08T05:53:44.736682+0000","last_active":"2023-10-08T05:53:45.772806+0000","last_peered":"2023-10-08T05:53:45.772806+0000","last_clean":"2023-10-08T05:53:45.772806+0000","last_became_active":"2023-10-08T05:53:44.736396+0000","last_became_peered":"2023-10-08T05:53:44.736396+0000","last_unstale":"2023-10-08T05:53:45.772806+0000","last_undegraded":"2023-10-08T05:53:45.772806+0000","last_fullsized":"2023-10-08T05:53:45.772806+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:42.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736826+0000","last_change":"2023-10-08T05:53:44.734308+0000","last_active":"2023-10-08T05:53:45.736826+0000","last_peered":"2023-10-08T05:53:45.736826+0000","last_clean":"2023-10-08T05:53:45.736826+0000","last_became_active":"2023-10-08T05:53:44.733982+0000","last_became_peered":"2023-10-08T05:53:44.733982+0000","last_unstale":"2023-10-08T05:53:45.736826+0000","last_undegraded":"2023-10-08T05:53:45.736826+0000","last_fullsized":"2023-10-08T05:53:45.736826+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:09.123159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769851+0000","last_change":"2023-10-08T05:53:44.743498+0000","last_active":"2023-10-08T05:53:45.769851+0000","last_peered":"2023-10-08T05:53:45.769851+0000","last_clean":"2023-10-08T05:53:45.769851+0000","last_became_active":"2023-10-08T05:53:44.740098+0000","last_became_peered":"2023-10-08T05:53:44.740098+0000","last_unstale":"2023-10-08T05:53:45.769851+0000","last_undegraded":"2023-10-08T05:53:45.769851+0000","last_fullsized":"2023-10-08T05:53:45.769851+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:10.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.730480+0000","last_change":"2023-10-08T05:53:44.729943+0000","last_active":"2023-10-08T05:53:45.730480+0000","last_peered":"2023-10-08T05:53:45.730480+0000","last_clean":"2023-10-08T05:53:45.730480+0000","last_became_active":"2023-10-08T05:53:44.729680+0000","last_became_peered":"2023-10-08T05:53:44.729680+0000","last_unstale":"2023-10-08T05:53:45.730480+0000","last_undegraded":"2023-10-08T05:53:45.730480+0000","last_fullsized":"2023-10-08T05:53:45.730480+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:31.314169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768953+0000","last_change":"2023-10-08T05:53:44.750223+0000","last_active":"2023-10-08T05:53:45.768953+0000","last_peered":"2023-10-08T05:53:45.768953+0000","last_clean":"2023-10-08T05:53:45.768953+0000","last_became_active":"2023-10-08T05:53:44.749962+0000","last_became_peered":"2023-10-08T05:53:44.749962+0000","last_unstale":"2023-10-08T05:53:45.768953+0000","last_undegraded":"2023-10-08T05:53:45.768953+0000","last_fullsized":"2023-10-08T05:53:45.768953+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:30.965219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749806+0000","last_change":"2023-10-08T05:53:44.735327+0000","last_active":"2023-10-08T05:53:45.749806+0000","last_peered":"2023-10-08T05:53:45.749806+0000","last_clean":"2023-10-08T05:53:45.749806+0000","last_became_active":"2023-10-08T05:53:44.734968+0000","last_became_peered":"2023-10-08T05:53:44.734968+0000","last_unstale":"2023-10-08T05:53:45.749806+0000","last_undegraded":"2023-10-08T05:53:45.749806+0000","last_fullsized":"2023-10-08T05:53:45.749806+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:07:40.057790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769824+0000","last_change":"2023-10-08T05:53:44.728034+0000","last_active":"2023-10-08T05:53:45.769824+0000","last_peered":"2023-10-08T05:53:45.769824+0000","last_clean":"2023-10-08T05:53:45.769824+0000","last_became_active":"2023-10-08T05:53:44.727758+0000","last_became_peered":"2023-10-08T05:53:44.727758+0000","last_unstale":"2023-10-08T05:53:45.769824+0000","last_undegraded":"2023-10-08T05:53:45.769824+0000","last_fullsized":"2023-10-08T05:53:45.769824+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:44.157586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735998+0000","last_change":"2023-10-08T05:53:44.747259+0000","last_active":"2023-10-08T05:53:45.735998+0000","last_peered":"2023-10-08T05:53:45.735998+0000","last_clean":"2023-10-08T05:53:45.735998+0000","last_became_active":"2023-10-08T05:53:44.746949+0000","last_became_peered":"2023-10-08T05:53:44.746949+0000","last_unstale":"2023-10-08T05:53:45.735998+0000","last_undegraded":"2023-10-08T05:53:45.735998+0000","last_fullsized":"2023-10-08T05:53:45.735998+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:29:19.871040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769714+0000","last_change":"2023-10-08T05:53:44.739131+0000","last_active":"2023-10-08T05:53:45.769714+0000","last_peered":"2023-10-08T05:53:45.769714+0000","last_clean":"2023-10-08T05:53:45.769714+0000","last_became_active":"2023-10-08T05:53:44.735989+0000","last_became_peered":"2023-10-08T05:53:44.735989+0000","last_unstale":"2023-10-08T05:53:45.769714+0000","last_undegraded":"2023-10-08T05:53:45.769714+0000","last_fullsized":"2023-10-08T05:53:45.769714+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:32:44.279315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769595+0000","last_change":"2023-10-08T05:53:44.751155+0000","last_active":"2023-10-08T05:53:45.769595+0000","last_peered":"2023-10-08T05:53:45.769595+0000","last_clean":"2023-10-08T05:53:45.769595+0000","last_became_active":"2023-10-08T05:53:44.750892+0000","last_became_peered":"2023-10-08T05:53:44.750892+0000","last_unstale":"2023-10-08T05:53:45.769595+0000","last_undegraded":"2023-10-08T05:53:45.769595+0000","last_fullsized":"2023-10-08T05:53:45.769595+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:24.878153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"117'4","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735631+0000","last_change":"2023-10-08T05:53:43.708846+0000","last_active":"2023-10-08T05:53:45.735631+0000","last_peered":"2023-10-08T05:53:45.735631+0000","last_clean":"2023-10-08T05:53:45.735631+0000","last_became_active":"2023-10-08T05:53:43.708567+0000","last_became_peered":"2023-10-08T05:53:43.708567+0000","last_unstale":"2023-10-08T05:53:45.735631+0000","last_undegraded":"2023-10-08T05:53:45.735631+0000","last_fullsized":"2023-10-08T05:53:45.735631+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:22.341562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.771817+0000","last_change":"2023-10-08T05:53:44.734771+0000","last_active":"2023-10-08T05:53:45.771817+0000","last_peered":"2023-10-08T05:53:45.771817+0000","last_clean":"2023-10-08T05:53:45.771817+0000","last_became_active":"2023-10-08T05:53:44.734448+0000","last_became_peered":"2023-10-08T05:53:44.734448+0000","last_unstale":"2023-10-08T05:53:45.771817+0000","last_undegraded":"2023-10-08T05:53:45.771817+0000","last_fullsized":"2023-10-08T05:53:45.771817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:17.047581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770064+0000","last_change":"2023-10-08T05:53:43.716282+0000","last_active":"2023-10-08T05:53:45.770064+0000","last_peered":"2023-10-08T05:53:45.770064+0000","last_clean":"2023-10-08T05:53:45.770064+0000","last_became_active":"2023-10-08T05:53:43.715918+0000","last_became_peered":"2023-10-08T05:53:43.715918+0000","last_unstale":"2023-10-08T05:53:45.770064+0000","last_undegraded":"2023-10-08T05:53:45.770064+0000","last_fullsized":"2023-10-08T05:53:45.770064+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:07.715363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.750110+0000","last_change":"2023-10-08T05:53:44.736217+0000","last_active":"2023-10-08T05:53:45.750110+0000","last_peered":"2023-10-08T05:53:45.750110+0000","last_clean":"2023-10-08T05:53:45.750110+0000","last_became_active":"2023-10-08T05:53:44.735922+0000","last_became_peered":"2023-10-08T05:53:44.735922+0000","last_unstale":"2023-10-08T05:53:45.750110+0000","last_undegraded":"2023-10-08T05:53:45.750110+0000","last_fullsized":"2023-10-08T05:53:45.750110+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:09.123159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749887+0000","last_change":"2023-10-08T05:53:43.712447+0000","last_active":"2023-10-08T05:53:45.749887+0000","last_peered":"2023-10-08T05:53:45.749887+0000","last_clean":"2023-10-08T05:53:45.749887+0000","last_became_active":"2023-10-08T05:53:43.712167+0000","last_became_peered":"2023-10-08T05:53:43.712167+0000","last_unstale":"2023-10-08T05:53:45.749887+0000","last_undegraded":"2023-10-08T05:53:45.749887+0000","last_fullsized":"2023-10-08T05:53:45.749887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:16.037633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772381+0000","last_change":"2023-10-08T05:53:44.741309+0000","last_active":"2023-10-08T05:53:45.772381+0000","last_peered":"2023-10-08T05:53:45.772381+0000","last_clean":"2023-10-08T05:53:45.772381+0000","last_became_active":"2023-10-08T05:53:44.740260+0000","last_became_peered":"2023-10-08T05:53:44.740260+0000","last_unstale":"2023-10-08T05:53:45.772381+0000","last_undegraded":"2023-10-08T05:53:45.772381+0000","last_fullsized":"2023-10-08T05:53:45.772381+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:09.787683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735306+0000","last_change":"2023-10-08T05:53:43.710659+0000","last_active":"2023-10-08T05:53:45.735306+0000","last_peered":"2023-10-08T05:53:45.735306+0000","last_clean":"2023-10-08T05:53:45.735306+0000","last_became_active":"2023-10-08T05:53:43.710395+0000","last_became_peered":"2023-10-08T05:53:43.710395+0000","last_unstale":"2023-10-08T05:53:45.735306+0000","last_undegraded":"2023-10-08T05:53:45.735306+0000","last_fullsized":"2023-10-08T05:53:45.735306+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:29.336481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768504+0000","last_change":"2023-10-08T05:53:44.738694+0000","last_active":"2023-10-08T05:53:45.768504+0000","last_peered":"2023-10-08T05:53:45.768504+0000","last_clean":"2023-10-08T05:53:45.768504+0000","last_became_active":"2023-10-08T05:53:44.738398+0000","last_became_peered":"2023-10-08T05:53:44.738398+0000","last_unstale":"2023-10-08T05:53:45.768504+0000","last_undegraded":"2023-10-08T05:53:45.768504+0000","last_fullsized":"2023-10-08T05:53:45.768504+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:37:27.581656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769991+0000","last_change":"2023-10-08T05:53:43.707124+0000","last_active":"2023-10-08T05:53:45.769991+0000","last_peered":"2023-10-08T05:53:45.769991+0000","last_clean":"2023-10-08T05:53:45.769991+0000","last_became_active":"2023-10-08T05:53:43.706820+0000","last_became_peered":"2023-10-08T05:53:43.706820+0000","last_unstale":"2023-10-08T05:53:45.769991+0000","last_undegraded":"2023-10-08T05:53:45.769991+0000","last_fullsized":"2023-10-08T05:53:45.769991+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:13.519501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.734832+0000","last_change":"2023-10-08T05:53:44.731745+0000","last_active":"2023-10-08T05:53:45.734832+0000","last_peered":"2023-10-08T05:53:45.734832+0000","last_clean":"2023-10-08T05:53:45.734832+0000","last_became_active":"2023-10-08T05:53:44.731465+0000","last_became_peered":"2023-10-08T05:53:44.731465+0000","last_unstale":"2023-10-08T05:53:45.734832+0000","last_undegraded":"2023-10-08T05:53:45.734832+0000","last_fullsized":"2023-10-08T05:53:45.734832+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:01.277669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.771944+0000","last_change":"2023-10-08T05:53:43.707705+0000","last_active":"2023-10-08T05:53:45.771944+0000","last_peered":"2023-10-08T05:53:45.771944+0000","last_clean":"2023-10-08T05:53:45.771944+0000","last_became_active":"2023-10-08T05:53:43.707370+0000","last_became_peered":"2023-10-08T05:53:43.707370+0000","last_unstale":"2023-10-08T05:53:45.771944+0000","last_undegraded":"2023-10-08T05:53:45.771944+0000","last_fullsized":"2023-10-08T05:53:45.771944+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:09.933374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770304+0000","last_change":"2023-10-08T05:53:44.735736+0000","last_active":"2023-10-08T05:53:45.770304+0000","last_peered":"2023-10-08T05:53:45.770304+0000","last_clean":"2023-10-08T05:53:45.770304+0000","last_became_active":"2023-10-08T05:53:44.734877+0000","last_became_peered":"2023-10-08T05:53:44.734877+0000","last_unstale":"2023-10-08T05:53:45.770304+0000","last_undegraded":"2023-10-08T05:53:45.770304+0000","last_fullsized":"2023-10-08T05:53:45.770304+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:58:53.628913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.773326+0000","last_change":"2023-10-08T05:53:43.706120+0000","last_active":"2023-10-08T05:53:45.773326+0000","last_peered":"2023-10-08T05:53:45.773326+0000","last_clean":"2023-10-08T05:53:45.773326+0000","last_became_active":"2023-10-08T05:53:43.705815+0000","last_became_peered":"2023-10-08T05:53:43.705815+0000","last_unstale":"2023-10-08T05:53:45.773326+0000","last_undegraded":"2023-10-08T05:53:45.773326+0000","last_fullsized":"2023-10-08T05:53:45.773326+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:08.777735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769580+0000","last_change":"2023-10-08T05:53:44.739827+0000","last_active":"2023-10-08T05:53:45.769580+0000","last_peered":"2023-10-08T05:53:45.769580+0000","last_clean":"2023-10-08T05:53:45.769580+0000","last_became_active":"2023-10-08T05:53:44.739401+0000","last_became_peered":"2023-10-08T05:53:44.739401+0000","last_unstale":"2023-10-08T05:53:45.769580+0000","last_undegraded":"2023-10-08T05:53:45.769580+0000","last_fullsized":"2023-10-08T05:53:45.769580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:22:01.118078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749480+0000","last_change":"2023-10-08T05:53:43.710254+0000","last_active":"2023-10-08T05:53:45.749480+0000","last_peered":"2023-10-08T05:53:45.749480+0000","last_clean":"2023-10-08T05:53:45.749480+0000","last_became_active":"2023-10-08T05:53:43.709935+0000","last_became_peered":"2023-10-08T05:53:43.709935+0000","last_unstale":"2023-10-08T05:53:45.749480+0000","last_undegraded":"2023-10-08T05:53:45.749480+0000","last_fullsized":"2023-10-08T05:53:45.749480+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:05:53.482845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769714+0000","last_change":"2023-10-08T05:53:44.732406+0000","last_active":"2023-10-08T05:53:45.769714+0000","last_peered":"2023-10-08T05:53:45.769714+0000","last_clean":"2023-10-08T05:53:45.769714+0000","last_became_active":"2023-10-08T05:53:44.732141+0000","last_became_peered":"2023-10-08T05:53:44.732141+0000","last_unstale":"2023-10-08T05:53:45.769714+0000","last_undegraded":"2023-10-08T05:53:45.769714+0000","last_fullsized":"2023-10-08T05:53:45.769714+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:33:04.471656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736941+0000","last_change":"2023-10-08T05:53:43.703549+0000","last_active":"2023-10-08T05:53:45.736941+0000","last_peered":"2023-10-08T05:53:45.736941+0000","last_clean":"2023-10-08T05:53:45.736941+0000","last_became_active":"2023-10-08T05:53:43.703188+0000","last_became_peered":"2023-10-08T05:53:43.703188+0000","last_unstale":"2023-10-08T05:53:45.736941+0000","last_undegraded":"2023-10-08T05:53:45.736941+0000","last_fullsized":"2023-10-08T05:53:45.736941+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:41.290271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749871+0000","last_change":"2023-10-08T05:53:44.750298+0000","last_active":"2023-10-08T05:53:45.749871+0000","last_peered":"2023-10-08T05:53:45.749871+0000","last_clean":"2023-10-08T05:53:45.749871+0000","last_became_active":"2023-10-08T05:53:44.749727+0000","last_became_peered":"2023-10-08T05:53:44.749727+0000","last_unstale":"2023-10-08T05:53:45.749871+0000","last_undegraded":"2023-10-08T05:53:45.749871+0000","last_fullsized":"2023-10-08T05:53:45.749871+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:38.558149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.773139+0000","last_change":"2023-10-08T05:53:43.704891+0000","last_active":"2023-10-08T05:53:45.773139+0000","last_peered":"2023-10-08T05:53:45.773139+0000","last_clean":"2023-10-08T05:53:45.773139+0000","last_became_active":"2023-10-08T05:53:43.704611+0000","last_became_peered":"2023-10-08T05:53:43.704611+0000","last_unstale":"2023-10-08T05:53:45.773139+0000","last_undegraded":"2023-10-08T05:53:45.773139+0000","last_fullsized":"2023-10-08T05:53:45.773139+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:26.253802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.773047+0000","last_change":"2023-10-08T05:53:44.731169+0000","last_active":"2023-10-08T05:53:45.773047+0000","last_peered":"2023-10-08T05:53:45.773047+0000","last_clean":"2023-10-08T05:53:45.773047+0000","last_became_active":"2023-10-08T05:53:44.730888+0000","last_became_peered":"2023-10-08T05:53:44.730888+0000","last_unstale":"2023-10-08T05:53:45.773047+0000","last_undegraded":"2023-10-08T05:53:45.773047+0000","last_fullsized":"2023-10-08T05:53:45.773047+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:25.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"117'3","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770460+0000","last_change":"2023-10-08T05:53:43.711687+0000","last_active":"2023-10-08T05:53:45.770460+0000","last_peered":"2023-10-08T05:53:45.770460+0000","last_clean":"2023-10-08T05:53:45.770460+0000","last_became_active":"2023-10-08T05:53:43.711419+0000","last_became_peered":"2023-10-08T05:53:43.711419+0000","last_unstale":"2023-10-08T05:53:45.770460+0000","last_undegraded":"2023-10-08T05:53:45.770460+0000","last_fullsized":"2023-10-08T05:53:45.770460+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:26.817989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770697+0000","last_change":"2023-10-08T05:53:44.739401+0000","last_active":"2023-10-08T05:53:45.770697+0000","last_peered":"2023-10-08T05:53:45.770697+0000","last_clean":"2023-10-08T05:53:45.770697+0000","last_became_active":"2023-10-08T05:53:44.739123+0000","last_became_peered":"2023-10-08T05:53:44.739123+0000","last_unstale":"2023-10-08T05:53:45.770697+0000","last_undegraded":"2023-10-08T05:53:45.770697+0000","last_fullsized":"2023-10-08T05:53:45.770697+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:14:50.756106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"117'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.771075+0000","last_change":"2023-10-08T05:53:43.711065+0000","last_active":"2023-10-08T05:53:45.771075+0000","last_peered":"2023-10-08T05:53:45.771075+0000","last_clean":"2023-10-08T05:53:45.771075+0000","last_became_active":"2023-10-08T05:53:43.710777+0000","last_became_peered":"2023-10-08T05:53:43.710777+0000","last_unstale":"2023-10-08T05:53:45.771075+0000","last_undegraded":"2023-10-08T05:53:45.771075+0000","last_fullsized":"2023-10-08T05:53:45.771075+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:48.331810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768728+0000","last_change":"2023-10-08T05:53:44.743530+0000","last_active":"2023-10-08T05:53:45.768728+0000","last_peered":"2023-10-08T05:53:45.768728+0000","last_clean":"2023-10-08T05:53:45.768728+0000","last_became_active":"2023-10-08T05:53:44.742774+0000","last_became_peered":"2023-10-08T05:53:44.742774+0000","last_unstale":"2023-10-08T05:53:45.768728+0000","last_undegraded":"2023-10-08T05:53:45.768728+0000","last_fullsized":"2023-10-08T05:53:45.768728+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:28.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749867+0000","last_change":"2023-10-08T05:53:43.711053+0000","last_active":"2023-10-08T05:53:45.749867+0000","last_peered":"2023-10-08T05:53:45.749867+0000","last_clean":"2023-10-08T05:53:45.749867+0000","last_became_active":"2023-10-08T05:53:43.710593+0000","last_became_peered":"2023-10-08T05:53:43.710593+0000","last_unstale":"2023-10-08T05:53:45.749867+0000","last_undegraded":"2023-10-08T05:53:45.749867+0000","last_fullsized":"2023-10-08T05:53:45.749867+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:08.777735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769063+0000","last_change":"2023-10-08T05:53:44.737760+0000","last_active":"2023-10-08T05:53:45.769063+0000","last_peered":"2023-10-08T05:53:45.769063+0000","last_clean":"2023-10-08T05:53:45.769063+0000","last_became_active":"2023-10-08T05:53:44.737449+0000","last_became_peered":"2023-10-08T05:53:44.737449+0000","last_unstale":"2023-10-08T05:53:45.769063+0000","last_undegraded":"2023-10-08T05:53:45.769063+0000","last_fullsized":"2023-10-08T05:53:45.769063+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:06.945445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770544+0000","last_change":"2023-10-08T05:53:43.709722+0000","last_active":"2023-10-08T05:53:45.770544+0000","last_peered":"2023-10-08T05:53:45.770544+0000","last_clean":"2023-10-08T05:53:45.770544+0000","last_became_active":"2023-10-08T05:53:43.709448+0000","last_became_peered":"2023-10-08T05:53:43.709448+0000","last_unstale":"2023-10-08T05:53:45.770544+0000","last_undegraded":"2023-10-08T05:53:45.770544+0000","last_fullsized":"2023-10-08T05:53:45.770544+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:13.277722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.729627+0000","last_change":"2023-10-08T05:53:44.737093+0000","last_active":"2023-10-08T05:53:45.729627+0000","last_peered":"2023-10-08T05:53:45.729627+0000","last_clean":"2023-10-08T05:53:45.729627+0000","last_became_active":"2023-10-08T05:53:44.736820+0000","last_became_peered":"2023-10-08T05:53:44.736820+0000","last_unstale":"2023-10-08T05:53:45.729627+0000","last_undegraded":"2023-10-08T05:53:45.729627+0000","last_fullsized":"2023-10-08T05:53:45.729627+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:39:04.556070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770697+0000","last_change":"2023-10-08T05:53:43.713775+0000","last_active":"2023-10-08T05:53:45.770697+0000","last_peered":"2023-10-08T05:53:45.770697+0000","last_clean":"2023-10-08T05:53:45.770697+0000","last_became_active":"2023-10-08T05:53:43.713507+0000","last_became_peered":"2023-10-08T05:53:43.713507+0000","last_unstale":"2023-10-08T05:53:45.770697+0000","last_undegraded":"2023-10-08T05:53:45.770697+0000","last_fullsized":"2023-10-08T05:53:45.770697+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:23.699939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.730423+0000","last_change":"2023-10-08T05:53:44.727245+0000","last_active":"2023-10-08T05:53:45.730423+0000","last_peered":"2023-10-08T05:53:45.730423+0000","last_clean":"2023-10-08T05:53:45.730423+0000","last_became_active":"2023-10-08T05:53:44.726668+0000","last_became_peered":"2023-10-08T05:53:44.726668+0000","last_unstale":"2023-10-08T05:53:45.730423+0000","last_undegraded":"2023-10-08T05:53:45.730423+0000","last_fullsized":"2023-10-08T05:53:45.730423+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:53:40.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"117'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768598+0000","last_change":"2023-10-08T05:53:43.703262+0000","last_active":"2023-10-08T05:53:45.768598+0000","last_peered":"2023-10-08T05:53:45.768598+0000","last_clean":"2023-10-08T05:53:45.768598+0000","last_became_active":"2023-10-08T05:53:43.702921+0000","last_became_peered":"2023-10-08T05:53:43.702921+0000","last_unstale":"2023-10-08T05:53:45.768598+0000","last_undegraded":"2023-10-08T05:53:45.768598+0000","last_fullsized":"2023-10-08T05:53:45.768598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:05.935497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769961+0000","last_change":"2023-10-08T05:53:44.749980+0000","last_active":"2023-10-08T05:53:45.769961+0000","last_peered":"2023-10-08T05:53:45.769961+0000","last_clean":"2023-10-08T05:53:45.769961+0000","last_became_active":"2023-10-08T05:53:44.749709+0000","last_became_peered":"2023-10-08T05:53:44.749709+0000","last_unstale":"2023-10-08T05:53:45.769961+0000","last_undegraded":"2023-10-08T05:53:45.769961+0000","last_fullsized":"2023-10-08T05:53:45.769961+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:16.945508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735789+0000","last_change":"2023-10-08T05:53:43.708305+0000","last_active":"2023-10-08T05:53:45.735789+0000","last_peered":"2023-10-08T05:53:45.735789+0000","last_clean":"2023-10-08T05:53:45.735789+0000","last_became_active":"2023-10-08T05:53:43.708002+0000","last_became_peered":"2023-10-08T05:53:43.708002+0000","last_unstale":"2023-10-08T05:53:45.735789+0000","last_undegraded":"2023-10-08T05:53:45.735789+0000","last_fullsized":"2023-10-08T05:53:45.735789+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:40.473980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735488+0000","last_change":"2023-10-08T05:53:44.729322+0000","last_active":"2023-10-08T05:53:45.735488+0000","last_peered":"2023-10-08T05:53:45.735488+0000","last_clean":"2023-10-08T05:53:45.735488+0000","last_became_active":"2023-10-08T05:53:44.728678+0000","last_became_peered":"2023-10-08T05:53:44.728678+0000","last_unstale":"2023-10-08T05:53:45.735488+0000","last_undegraded":"2023-10-08T05:53:45.735488+0000","last_fullsized":"2023-10-08T05:53:45.735488+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:47.049141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.750145+0000","last_change":"2023-10-08T05:53:43.707105+0000","last_active":"2023-10-08T05:53:45.750145+0000","last_peered":"2023-10-08T05:53:45.750145+0000","last_clean":"2023-10-08T05:53:45.750145+0000","last_became_active":"2023-10-08T05:53:43.706767+0000","last_became_peered":"2023-10-08T05:53:43.706767+0000","last_unstale":"2023-10-08T05:53:45.750145+0000","last_undegraded":"2023-10-08T05:53:45.750145+0000","last_fullsized":"2023-10-08T05:53:45.750145+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:16:12.967051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770048+0000","last_change":"2023-10-08T05:53:44.717492+0000","last_active":"2023-10-08T05:53:45.770048+0000","last_peered":"2023-10-08T05:53:45.770048+0000","last_clean":"2023-10-08T05:53:45.770048+0000","last_became_active":"2023-10-08T05:53:44.716683+0000","last_became_peered":"2023-10-08T05:53:44.716683+0000","last_unstale":"2023-10-08T05:53:45.770048+0000","last_undegraded":"2023-10-08T05:53:45.770048+0000","last_fullsized":"2023-10-08T05:53:45.770048+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:37:27.581656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770463+0000","last_change":"2023-10-08T05:53:43.709269+0000","last_active":"2023-10-08T05:53:45.770463+0000","last_peered":"2023-10-08T05:53:45.770463+0000","last_clean":"2023-10-08T05:53:45.770463+0000","last_became_active":"2023-10-08T05:53:43.708402+0000","last_became_peered":"2023-10-08T05:53:43.708402+0000","last_unstale":"2023-10-08T05:53:45.770463+0000","last_undegraded":"2023-10-08T05:53:45.770463+0000","last_fullsized":"2023-10-08T05:53:45.770463+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:51:09.614335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770046+0000","last_change":"2023-10-08T05:53:44.734191+0000","last_active":"2023-10-08T05:53:45.770046+0000","last_peered":"2023-10-08T05:53:45.770046+0000","last_clean":"2023-10-08T05:53:45.770046+0000","last_became_active":"2023-10-08T05:53:44.733901+0000","last_became_peered":"2023-10-08T05:53:44.733901+0000","last_unstale":"2023-10-08T05:53:45.770046+0000","last_undegraded":"2023-10-08T05:53:45.770046+0000","last_fullsized":"2023-10-08T05:53:45.770046+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:22.850435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736190+0000","last_change":"2023-10-08T05:53:43.691491+0000","last_active":"2023-10-08T05:53:45.736190+0000","last_peered":"2023-10-08T05:53:45.736190+0000","last_clean":"2023-10-08T05:53:45.736190+0000","last_became_active":"2023-10-08T05:53:43.690758+0000","last_became_peered":"2023-10-08T05:53:43.690758+0000","last_unstale":"2023-10-08T05:53:45.736190+0000","last_undegraded":"2023-10-08T05:53:45.736190+0000","last_fullsized":"2023-10-08T05:53:45.736190+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:58:52.618965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736121+0000","last_change":"2023-10-08T05:53:44.734530+0000","last_active":"2023-10-08T05:53:45.736121+0000","last_peered":"2023-10-08T05:53:45.736121+0000","last_clean":"2023-10-08T05:53:45.736121+0000","last_became_active":"2023-10-08T05:53:44.734222+0000","last_became_peered":"2023-10-08T05:53:44.734222+0000","last_unstale":"2023-10-08T05:53:45.736121+0000","last_undegraded":"2023-10-08T05:53:45.736121+0000","last_fullsized":"2023-10-08T05:53:45.736121+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:25.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.729737+0000","last_change":"2023-10-08T05:53:43.707038+0000","last_active":"2023-10-08T05:53:45.729737+0000","last_peered":"2023-10-08T05:53:45.729737+0000","last_clean":"2023-10-08T05:53:45.729737+0000","last_became_active":"2023-10-08T05:53:43.706743+0000","last_became_peered":"2023-10-08T05:53:43.706743+0000","last_unstale":"2023-10-08T05:53:45.729737+0000","last_undegraded":"2023-10-08T05:53:45.729737+0000","last_fullsized":"2023-10-08T05:53:45.729737+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:45.009747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736397+0000","last_change":"2023-10-08T05:53:44.748584+0000","last_active":"2023-10-08T05:53:45.736397+0000","last_peered":"2023-10-08T05:53:45.736397+0000","last_clean":"2023-10-08T05:53:45.736397+0000","last_became_active":"2023-10-08T05:53:44.748276+0000","last_became_peered":"2023-10-08T05:53:44.748276+0000","last_unstale":"2023-10-08T05:53:45.736397+0000","last_undegraded":"2023-10-08T05:53:45.736397+0000","last_fullsized":"2023-10-08T05:53:45.736397+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:07:40.057790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736163+0000","last_change":"2023-10-08T05:53:43.696874+0000","last_active":"2023-10-08T05:53:45.736163+0000","last_peered":"2023-10-08T05:53:45.736163+0000","last_clean":"2023-10-08T05:53:45.736163+0000","last_became_active":"2023-10-08T05:53:43.696257+0000","last_became_peered":"2023-10-08T05:53:43.696257+0000","last_unstale":"2023-10-08T05:53:45.736163+0000","last_undegraded":"2023-10-08T05:53:45.736163+0000","last_fullsized":"2023-10-08T05:53:45.736163+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:16.037633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770876+0000","last_change":"2023-10-08T05:53:44.730174+0000","last_active":"2023-10-08T05:53:45.770876+0000","last_peered":"2023-10-08T05:53:45.770876+0000","last_clean":"2023-10-08T05:53:45.770876+0000","last_became_active":"2023-10-08T05:53:44.729850+0000","last_became_peered":"2023-10-08T05:53:44.729850+0000","last_unstale":"2023-10-08T05:53:45.770876+0000","last_undegraded":"2023-10-08T05:53:45.770876+0000","last_fullsized":"2023-10-08T05:53:45.770876+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:45.340517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.729895+0000","last_change":"2023-10-08T05:53:43.709948+0000","last_active":"2023-10-08T05:53:45.729895+0000","last_peered":"2023-10-08T05:53:45.729895+0000","last_clean":"2023-10-08T05:53:45.729895+0000","last_became_active":"2023-10-08T05:53:43.709689+0000","last_became_peered":"2023-10-08T05:53:43.709689+0000","last_unstale":"2023-10-08T05:53:45.729895+0000","last_undegraded":"2023-10-08T05:53:45.729895+0000","last_fullsized":"2023-10-08T05:53:45.729895+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:45:22.865416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735255+0000","last_change":"2023-10-08T05:53:44.727792+0000","last_active":"2023-10-08T05:53:45.735255+0000","last_peered":"2023-10-08T05:53:45.735255+0000","last_clean":"2023-10-08T05:53:45.735255+0000","last_became_active":"2023-10-08T05:53:44.727518+0000","last_became_peered":"2023-10-08T05:53:44.727518+0000","last_unstale":"2023-10-08T05:53:45.735255+0000","last_undegraded":"2023-10-08T05:53:45.735255+0000","last_fullsized":"2023-10-08T05:53:45.735255+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:35.256324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.750128+0000","last_change":"2023-10-08T05:53:43.710347+0000","last_active":"2023-10-08T05:53:45.750128+0000","last_peered":"2023-10-08T05:53:45.750128+0000","last_clean":"2023-10-08T05:53:45.750128+0000","last_became_active":"2023-10-08T05:53:43.710043+0000","last_became_peered":"2023-10-08T05:53:43.710043+0000","last_unstale":"2023-10-08T05:53:45.750128+0000","last_undegraded":"2023-10-08T05:53:45.750128+0000","last_fullsized":"2023-10-08T05:53:45.750128+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:44.330569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772355+0000","last_change":"2023-10-08T05:53:44.749972+0000","last_active":"2023-10-08T05:53:45.772355+0000","last_peered":"2023-10-08T05:53:45.772355+0000","last_clean":"2023-10-08T05:53:45.772355+0000","last_became_active":"2023-10-08T05:53:44.749689+0000","last_became_peered":"2023-10-08T05:53:44.749689+0000","last_unstale":"2023-10-08T05:53:45.772355+0000","last_undegraded":"2023-10-08T05:53:45.772355+0000","last_fullsized":"2023-10-08T05:53:45.772355+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:01:02.983379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770229+0000","last_change":"2023-10-08T05:53:43.708320+0000","last_active":"2023-10-08T05:53:45.770229+0000","last_peered":"2023-10-08T05:53:45.770229+0000","last_clean":"2023-10-08T05:53:45.770229+0000","last_became_active":"2023-10-08T05:53:43.707945+0000","last_became_peered":"2023-10-08T05:53:43.707945+0000","last_unstale":"2023-10-08T05:53:45.770229+0000","last_undegraded":"2023-10-08T05:53:45.770229+0000","last_fullsized":"2023-10-08T05:53:45.770229+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:23.699939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772104+0000","last_change":"2023-10-08T05:53:44.728019+0000","last_active":"2023-10-08T05:53:45.772104+0000","last_peered":"2023-10-08T05:53:45.772104+0000","last_clean":"2023-10-08T05:53:45.772104+0000","last_became_active":"2023-10-08T05:53:44.727589+0000","last_became_peered":"2023-10-08T05:53:44.727589+0000","last_unstale":"2023-10-08T05:53:45.772104+0000","last_undegraded":"2023-10-08T05:53:45.772104+0000","last_fullsized":"2023-10-08T05:53:45.772104+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:27.263750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770631+0000","last_change":"2023-10-08T05:53:43.709377+0000","last_active":"2023-10-08T05:53:45.770631+0000","last_peered":"2023-10-08T05:53:45.770631+0000","last_clean":"2023-10-08T05:53:45.770631+0000","last_became_active":"2023-10-08T05:53:43.708973+0000","last_became_peered":"2023-10-08T05:53:43.708973+0000","last_unstale":"2023-10-08T05:53:45.770631+0000","last_undegraded":"2023-10-08T05:53:45.770631+0000","last_fullsized":"2023-10-08T05:53:45.770631+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:48.072808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736875+0000","last_change":"2023-10-08T05:53:44.750141+0000","last_active":"2023-10-08T05:53:45.736875+0000","last_peered":"2023-10-08T05:53:45.736875+0000","last_clean":"2023-10-08T05:53:45.736875+0000","last_became_active":"2023-10-08T05:53:44.749840+0000","last_became_peered":"2023-10-08T05:53:44.749840+0000","last_unstale":"2023-10-08T05:53:45.736875+0000","last_undegraded":"2023-10-08T05:53:45.736875+0000","last_fullsized":"2023-10-08T05:53:45.736875+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:06.420177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735416+0000","last_change":"2023-10-08T05:53:43.707137+0000","last_active":"2023-10-08T05:53:45.735416+0000","last_peered":"2023-10-08T05:53:45.735416+0000","last_clean":"2023-10-08T05:53:45.735416+0000","last_became_active":"2023-10-08T05:53:43.706836+0000","last_became_peered":"2023-10-08T05:53:43.706836+0000","last_unstale":"2023-10-08T05:53:45.735416+0000","last_undegraded":"2023-10-08T05:53:45.735416+0000","last_fullsized":"2023-10-08T05:53:45.735416+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:55.362172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.730423+0000","last_change":"2023-10-08T05:53:44.729709+0000","last_active":"2023-10-08T05:53:45.730423+0000","last_peered":"2023-10-08T05:53:45.730423+0000","last_clean":"2023-10-08T05:53:45.730423+0000","last_became_active":"2023-10-08T05:53:44.729293+0000","last_became_peered":"2023-10-08T05:53:44.729293+0000","last_unstale":"2023-10-08T05:53:45.730423+0000","last_undegraded":"2023-10-08T05:53:45.730423+0000","last_fullsized":"2023-10-08T05:53:45.730423+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:34.574107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770960+0000","last_change":"2023-10-08T05:53:43.714848+0000","last_active":"2023-10-08T05:53:45.770960+0000","last_peered":"2023-10-08T05:53:45.770960+0000","last_clean":"2023-10-08T05:53:45.770960+0000","last_became_active":"2023-10-08T05:53:43.714532+0000","last_became_peered":"2023-10-08T05:53:43.714532+0000","last_unstale":"2023-10-08T05:53:45.770960+0000","last_undegraded":"2023-10-08T05:53:45.770960+0000","last_fullsized":"2023-10-08T05:53:45.770960+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:58.428910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.773398+0000","last_change":"2023-10-08T05:53:44.742773+0000","last_active":"2023-10-08T05:53:45.773398+0000","last_peered":"2023-10-08T05:53:45.773398+0000","last_clean":"2023-10-08T05:53:45.773398+0000","last_became_active":"2023-10-08T05:53:44.742495+0000","last_became_peered":"2023-10-08T05:53:44.742495+0000","last_unstale":"2023-10-08T05:53:45.773398+0000","last_undegraded":"2023-10-08T05:53:45.773398+0000","last_fullsized":"2023-10-08T05:53:45.773398+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:38.558149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736340+0000","last_change":"2023-10-08T05:53:43.691457+0000","last_active":"2023-10-08T05:53:45.736340+0000","last_peered":"2023-10-08T05:53:45.736340+0000","last_clean":"2023-10-08T05:53:45.736340+0000","last_became_active":"2023-10-08T05:53:43.690742+0000","last_became_peered":"2023-10-08T05:53:43.690742+0000","last_unstale":"2023-10-08T05:53:45.736340+0000","last_undegraded":"2023-10-08T05:53:45.736340+0000","last_fullsized":"2023-10-08T05:53:45.736340+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:16:12.967051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.734829+0000","last_change":"2023-10-08T05:53:44.741482+0000","last_active":"2023-10-08T05:53:45.734829+0000","last_peered":"2023-10-08T05:53:45.734829+0000","last_clean":"2023-10-08T05:53:45.734829+0000","last_became_active":"2023-10-08T05:53:44.741197+0000","last_became_peered":"2023-10-08T05:53:44.741197+0000","last_unstale":"2023-10-08T05:53:45.734829+0000","last_undegraded":"2023-10-08T05:53:45.734829+0000","last_fullsized":"2023-10-08T05:53:45.734829+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:59:50.641322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.734980+0000","last_change":"2023-10-08T05:53:43.706023+0000","last_active":"2023-10-08T05:53:45.734980+0000","last_peered":"2023-10-08T05:53:45.734980+0000","last_clean":"2023-10-08T05:53:45.734980+0000","last_became_active":"2023-10-08T05:53:43.705742+0000","last_became_peered":"2023-10-08T05:53:43.705742+0000","last_unstale":"2023-10-08T05:53:45.734980+0000","last_undegraded":"2023-10-08T05:53:45.734980+0000","last_fullsized":"2023-10-08T05:53:45.734980+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:53.297605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.771060+0000","last_change":"2023-10-08T05:53:44.735709+0000","last_active":"2023-10-08T05:53:45.771060+0000","last_peered":"2023-10-08T05:53:45.771060+0000","last_clean":"2023-10-08T05:53:45.771060+0000","last_became_active":"2023-10-08T05:53:44.735392+0000","last_became_peered":"2023-10-08T05:53:44.735392+0000","last_unstale":"2023-10-08T05:53:45.771060+0000","last_undegraded":"2023-10-08T05:53:45.771060+0000","last_fullsized":"2023-10-08T05:53:45.771060+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:18.150326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749586+0000","last_change":"2023-10-08T05:53:43.712535+0000","last_active":"2023-10-08T05:53:45.749586+0000","last_peered":"2023-10-08T05:53:45.749586+0000","last_clean":"2023-10-08T05:53:45.749586+0000","last_became_active":"2023-10-08T05:53:43.712118+0000","last_became_peered":"2023-10-08T05:53:43.712118+0000","last_unstale":"2023-10-08T05:53:45.749586+0000","last_undegraded":"2023-10-08T05:53:45.749586+0000","last_fullsized":"2023-10-08T05:53:45.749586+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:41.290271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.750305+0000","last_change":"2023-10-08T05:53:44.739587+0000","last_active":"2023-10-08T05:53:45.750305+0000","last_peered":"2023-10-08T05:53:45.750305+0000","last_clean":"2023-10-08T05:53:45.750305+0000","last_became_active":"2023-10-08T05:53:44.739316+0000","last_became_peered":"2023-10-08T05:53:44.739316+0000","last_unstale":"2023-10-08T05:53:45.750305+0000","last_undegraded":"2023-10-08T05:53:45.750305+0000","last_fullsized":"2023-10-08T05:53:45.750305+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:06.420177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769038+0000","last_change":"2023-10-08T05:53:43.703413+0000","last_active":"2023-10-08T05:53:45.769038+0000","last_peered":"2023-10-08T05:53:45.769038+0000","last_clean":"2023-10-08T05:53:45.769038+0000","last_became_active":"2023-10-08T05:53:43.703129+0000","last_became_peered":"2023-10-08T05:53:43.703129+0000","last_unstale":"2023-10-08T05:53:45.769038+0000","last_undegraded":"2023-10-08T05:53:45.769038+0000","last_fullsized":"2023-10-08T05:53:45.769038+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:12.115005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"107'2641","reported_seq":2994,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-08T05:53:44.727313+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:53:44.727313+0000","last_peered":"2023-10-08T05:53:44.727313+0000","last_clean":"2023-10-08T05:53:44.727313+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:53:44.727313+0000","last_undegraded":"2023-10-08T05:53:44.727313+0000","last_fullsized":"2023-10-08T05:53:44.727313+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2641,"log_dups_size":0,"ondisk_log_size":2641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":765,"num_read_kb":1528,"num_write":289,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":765,"num_read_kb":1528,"num_write":289,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2641,"ondisk_log_size":2641,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575060,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51244,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320596,"statfs":{"total":96636764160,"available":96584290304,"internally_reserved":0,"allocated":2056192,"data_stored":1326202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.1080000000000001,"15min":1.1879999999999999},"min":{"1min":0.69999999999999996,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2969999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.1439999999999999,"15min":1.21},"min":{"1min":0.63700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.718,"5min":1.7210000000000001,"15min":1.726},"last":1.4119999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.145,"15min":1.2310000000000001},"min":{"1min":0.68400000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5,"5min":1.619,"15min":1.8360000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.252,"15min":1.2689999999999999},"min":{"1min":0.83799999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.591,"5min":1.7689999999999999,"15min":1.9379999999999999},"last":1.298}]},{"osd":2,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1819999999999999,"15min":1.2330000000000001},"min":{"1min":0.71599999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.502,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.294,"15min":1.3180000000000001},"min":{"1min":0.88500000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.714,"5min":1.714,"15min":1.7769999999999999},"last":1.492}]},{"osd":3,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2569999999999999,"15min":1.2869999999999999},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.611,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.3,"15min":1.353},"min":{"1min":0.83199999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6020000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.1220000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3160000000000001,"15min":1.288},"min":{"1min":1.0189999999999999,"5min":0.504,"15min":0.504},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.153},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.355,"15min":1.333},"min":{"1min":0.92600000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.766,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.0740000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3799999999999999,"15min":1.3939999999999999},"min":{"1min":0.94899999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7569999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.387},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4039999999999999,"15min":1.421},"min":{"1min":0.95999999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.3600000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.532,"15min":1.522},"min":{"1min":1.1160000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.476,"15min":1.46},"min":{"1min":0.84299999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.8859999999999999,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.3240000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575059,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":1474560,"data_stored":737994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.246,"15min":1.28},"min":{"1min":0.70899999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.55,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.262,"15min":1.3100000000000001},"min":{"1min":0.82199999999999995,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.6679999999999999,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.5740000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1319999999999999,"15min":1.169},"min":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4370000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.218},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.159,"15min":1.1930000000000001},"min":{"1min":0.69199999999999995,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6299999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.7030000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1499999999999999,"15min":1.1890000000000001},"min":{"1min":0.69099999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7470000000000001,"5min":1.774,"15min":1.796},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.169,"5min":1.1970000000000001,"15min":1.26},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.58699999999999997},"max":{"1min":1.581,"5min":1.752,"15min":1.8620000000000001},"last":0.94799999999999995}]},{"osd":3,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.3100000000000001,"15min":1.371},"min":{"1min":0.80300000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.571,"5min":2.137,"15min":2.137},"last":1.627},{"interface":"front","average":{"1min":1.359,"5min":1.399,"15min":1.45},"min":{"1min":0.86799999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.637,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.3380000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3320000000000001,"15min":1.3100000000000001},"min":{"1min":0.85999999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.724,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.399,"15min":1.3839999999999999},"min":{"1min":1.159,"5min":0.78400000000000003,"15min":0.64300000000000002},"max":{"1min":1.9159999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.742}]},{"osd":5,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.319,"15min":1.3320000000000001},"min":{"1min":0.83599999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.502,"15min":1.492},"min":{"1min":1.034,"5min":0.747,"15min":0.747},"max":{"1min":2.0350000000000001,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.913}]},{"osd":7,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5309999999999999,"15min":1.5149999999999999},"min":{"1min":1.1499999999999999,"5min":0.98299999999999998,"15min":0.77200000000000002},"max":{"1min":2.6070000000000002,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.091},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4950000000000001,"15min":1.474},"min":{"1min":1.0860000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.6659999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575059,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":735778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.0589999999999999,"15min":1.1240000000000001},"min":{"1min":0.70399999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.173,"15min":1.2050000000000001},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.758},"last":1.0760000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.179,"15min":1.1990000000000001},"min":{"1min":0.73699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.756,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.258,"15min":1.2649999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.70199999999999996},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5389999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1859999999999999,"15min":1.2290000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.5629999999999999,"5min":1.8,"15min":1.8},"last":1.341},{"interface":"front","average":{"1min":1.23,"5min":1.258,"15min":1.2829999999999999},"min":{"1min":0.71399999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.3199999999999998,"5min":3.282,"15min":3.282},"last":1.585}]},{"osd":3,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3169999999999999,"15min":1.3420000000000001},"min":{"1min":0.88700000000000001,"5min":0.77700000000000002,"15min":0.63500000000000001},"max":{"1min":2.5710000000000002,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.423},{"interface":"front","average":{"1min":1.337,"5min":1.3759999999999999,"15min":1.3899999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.73,"5min":1.919,"15min":2.2280000000000002},"last":1.73}]},{"osd":4,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.361,"15min":1.2949999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.2919999999999998,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.496,"15min":1.4239999999999999},"min":{"1min":0.94599999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.927,"5min":3.456,"15min":3.456},"last":1.214}]},{"osd":6,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.4279999999999999,"15min":1.4039999999999999},"min":{"1min":0.95299999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.105,"5min":2.105,"15min":2.359},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.5049999999999999,"15min":1.464},"min":{"1min":0.88100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.4390000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.681}]},{"osd":7,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.661,"5min":1.5940000000000001,"15min":1.544},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.018},"max":{"1min":3.0270000000000001,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":1.615},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.5820000000000001,"15min":1.5529999999999999},"min":{"1min":1.0129999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.4460000000000002},"last":1.631}]}]},{"osd":4,"up_from":13,"seq":55834575059,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":735938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1830000000000001,"15min":1.252},"min":{"1min":0.70399999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.5720000000000001,"5min":1.8009999999999999,"15min":1.8149999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3759999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.4059999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.2569999999999999,"15min":1.28},"min":{"1min":0.91800000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.5499999999999998,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.212,"15min":1.216},"min":{"1min":0.70299999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.1499999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":1.2629999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.218,"15min":1.2270000000000001},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6160000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.222},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3380000000000001,"15min":1.355},"min":{"1min":0.75700000000000001,"5min":0.622,"15min":0.622},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.5609999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.2829999999999999,"15min":1.276},"min":{"1min":0.84799999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.3070000000000004,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3080000000000001,"15min":1.335},"min":{"1min":0.83599999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.643,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.5289999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.393,"15min":1.337},"min":{"1min":0.751,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.5},{"interface":"front","average":{"1min":1.599,"5min":1.431,"15min":1.373},"min":{"1min":0.78600000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.4189999999999996,"5min":5.4189999999999996,"15min":5.4189999999999996},"last":1.169}]},{"osd":6,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.4399999999999999,"15min":1.3640000000000001},"min":{"1min":0.72199999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":4.9059999999999997,"5min":4.9059999999999997,"15min":4.9059999999999997},"last":1.306},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.5660000000000001,"15min":1.514},"min":{"1min":0.67300000000000004,"5min":0.433,"15min":0.433},"max":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":1.3320000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.7909999999999999,"5min":1.5669999999999999,"15min":1.47},"min":{"1min":0.94299999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":6.6689999999999996,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.675,"5min":1.532,"15min":1.5209999999999999},"min":{"1min":0.995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.361}]}]},{"osd":3,"up_from":13,"seq":55834575061,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1454080,"data_stored":735778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.89500000000000002,"15min":0.91900000000000004},"min":{"1min":0.41399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1819999999999999,"5min":1.5269999999999999,"15min":1.72},"last":1.123},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.018,"15min":1.091},"min":{"1min":0.53200000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.456,"5min":1.5,"15min":1.6140000000000001},"last":0.74299999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.90200000000000002,"15min":0.96399999999999997},"min":{"1min":0.39200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.998,"5min":1.149,"15min":1.208},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.4610000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.3360000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.006,"15min":1.0649999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7090000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.196},{"interface":"front","average":{"1min":1.016,"5min":1.1200000000000001,"15min":1.1439999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3600000000000001,"5min":1.6819999999999999,"15min":1.788},"last":0.92400000000000004}]},{"osd":4,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1599999999999999,"15min":1.206},"min":{"1min":0.76300000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.494,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.3440000000000001,"15min":1.3839999999999999},"min":{"1min":1.0720000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9590000000000001,"5min":2.589,"15min":2.589},"last":1.3939999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.286,"15min":1.3029999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73699999999999999},"max":{"1min":1.855,"5min":1.883,"15min":1.883},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.3280000000000001,"15min":1.355},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.798,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4370000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.3100000000000001,"15min":1.3200000000000001},"min":{"1min":0.65700000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7390000000000001,"5min":1.766,"15min":1.776},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4379999999999999,"15min":1.4470000000000001},"min":{"1min":0.96699999999999997,"5min":0.86799999999999999,"15min":0.86699999999999999},"max":{"1min":2.016,"5min":2.016,"15min":2.0489999999999999},"last":1.5049999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.462,"15min":1.4910000000000001},"min":{"1min":0.89500000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.913,"5min":2.085,"15min":2.1480000000000001},"last":1.536},{"interface":"front","average":{"1min":1.343,"5min":1.363,"15min":1.391},"min":{"1min":0.80800000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.248}]}]},{"osd":2,"up_from":13,"seq":55834575059,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58976,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312864,"statfs":{"total":96636764160,"available":96576372736,"internally_reserved":0,"allocated":1454080,"data_stored":735778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77800000000000002,"15min":0.874},"min":{"1min":0.32200000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.97099999999999997,"15min":1.03},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3560000000000001,"5min":1.5880000000000001,"15min":1.9470000000000001},"last":1.0349999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.85899999999999999,"15min":0.94499999999999995},"min":{"1min":0.48199999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1839999999999999,"5min":1.7609999999999999,"15min":2.1070000000000002},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96699999999999997,"15min":1.0900000000000001},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.79,"5min":1.79,"15min":2.1469999999999998},"last":1.349}]},{"osd":3,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.024,"15min":1.1200000000000001},"min":{"1min":0.53000000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.631,"5min":1.631,"15min":1.8120000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.149,"15min":1.216},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7350000000000001,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":1.7410000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3169999999999999,"15min":1.3759999999999999},"min":{"1min":0.66000000000000003,"5min":0.622,"15min":0.622},"max":{"1min":1.8280000000000001,"5min":4.593,"15min":4.593},"last":1.246},{"interface":"front","average":{"1min":1.149,"5min":1.2450000000000001,"15min":1.28},"min":{"1min":0.65900000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.6040000000000001,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.296}]},{"osd":5,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.3100000000000001,"15min":1.3680000000000001},"min":{"1min":0.68799999999999994,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.621,"5min":6.7140000000000004,"15min":6.7140000000000004},"last":1.526},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.353,"15min":1.4430000000000001},"min":{"1min":0.78800000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8620000000000001,"5min":1.9370000000000001,"15min":2.4119999999999999},"last":1.5840000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2869999999999999,"15min":1.343},"min":{"1min":0.64600000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.581,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.208,"5min":1.3300000000000001,"15min":1.4390000000000001},"min":{"1min":0.77500000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.619,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.639}]},{"osd":7,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.387,"15min":1.4279999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7450000000000001,"5min":6.609,"15min":6.609},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3700000000000001,"15min":1.488},"min":{"1min":0.98099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.5529999999999999,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.706}]}]},{"osd":1,"up_from":13,"seq":55834575059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":1458176,"data_stored":735848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.86699999999999999,"15min":0.95999999999999996},"min":{"1min":0.35799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2410000000000001,"5min":1.603,"15min":1.796},"last":1.026},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.0049999999999999,"15min":1.125},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.409,"5min":1.5600000000000001,"15min":1.8089999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.871,"15min":0.95699999999999996},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.498},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.96999999999999997,"15min":1.079},"min":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4370000000000001,"5min":1.593,"15min":1.8049999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":1.056,"15min":1.1719999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3879999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.999,"15min":1.0649999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5329999999999999,"5min":1.7669999999999999,"15min":1.788},"last":0.81999999999999995}]},{"osd":4,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1819999999999999,"15min":1.2949999999999999},"min":{"1min":0.67700000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.966},"last":1.234},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1779999999999999,"15min":1.2210000000000001},"min":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6719999999999999,"5min":1.913,"15min":1.913},"last":0.95599999999999996}]},{"osd":5,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.248,"15min":1.304},"min":{"1min":0.73199999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7909999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.113},{"interface":"front","average":{"1min":1.248,"5min":1.355,"15min":1.4470000000000001},"min":{"1min":0.86499999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7090000000000001,"5min":1.905,"15min":2.077},"last":1.2130000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3380000000000001,"15min":1.421},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.843,"5min":1.972,"15min":1.9970000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3640000000000001,"15min":1.4219999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8580000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.056}]},{"osd":7,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.319,"15min":1.4530000000000001},"min":{"1min":0.76600000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6259999999999999,"5min":1.893,"15min":2.1869999999999998},"last":1.093},{"interface":"front","average":{"1min":1.208,"5min":1.3680000000000001,"15min":1.4470000000000001},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":2.306},"last":1.077}]}]},{"osd":0,"up_from":13,"seq":55834575060,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2072576,"data_stored":1328464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.83799999999999997,"15min":0.91000000000000003},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.552,"5min":1.77,"15min":1.77},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.97099999999999997,"15min":1.0369999999999999},"min":{"1min":0.40899999999999997,"5min":0.371,"15min":0.371},"max":{"1min":1.3859999999999999,"5min":1.6719999999999999,"15min":1.675},"last":0.70899999999999996}]},{"osd":2,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.997,"15min":1.0840000000000001},"min":{"1min":0.42599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.361,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.89400000000000002,"15min":0.99099999999999999},"min":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.395,"5min":1.722,"15min":1.78},"last":1.4770000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.087,"15min":1.1970000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6599999999999999,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.036,"15min":1.0880000000000001},"min":{"1min":0.46500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.496,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.93100000000000005}]},{"osd":4,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.206,"15min":1.268},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.8129999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2689999999999999,"15min":1.343},"min":{"1min":0.70599999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6879999999999999,"5min":1.8939999999999999,"15min":2.2200000000000002},"last":1.05}]},{"osd":5,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.282,"15min":1.337},"min":{"1min":0.67700000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":2.0489999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.3049999999999999,"15min":1.377},"min":{"1min":0.71199999999999997,"5min":0.63,"15min":0.63},"max":{"1min":1.776,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.236}]},{"osd":6,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.347,"15min":1.413},"min":{"1min":0.78000000000000003,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.875,"5min":2.032,"15min":2.1909999999999998},"last":1.278},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.329,"15min":1.399},"min":{"1min":0.85599999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0430000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.4299999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.371,"15min":1.4370000000000001},"min":{"1min":0.74399999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4019999999999999,"15min":1.4730000000000001},"min":{"1min":0.78800000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.2759999999999998},"last":1.131}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:53:49.171 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:53:49.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.419+0000 7fd8d2a90640 1 Processor -- start 2023-10-08T05:53:49.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.419+0000 7fd8d2a90640 1 -- start start 2023-10-08T05:53:49.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.420+0000 7fd8d2a90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc15ab30 0x7fd8cc15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.420+0000 7fd8d2a90640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd8cc0b07d0 0x7fd8cc06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.420+0000 7fd8d2a90640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8cc15f7d0 con 0x7fd8cc0adc70 2023-10-08T05:53:49.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.420+0000 7fd8d2a90640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8cc15f210 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.420+0000 7fd8d2a90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc15f4f0 con 0x7fd8cc15ab30 2023-10-08T05:53:49.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.421+0000 7fd8d1a8e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc15ab30 0x7fd8cc15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.421+0000 7fd8d1a8e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc15ab30 0x7fd8cc15e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53220/0 (socket says 172.21.15.70:53220) 2023-10-08T05:53:49.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.421+0000 7fd8d1a8e640 1 -- 172.21.15.70:0/1271102465 learned_addr learned my addr 172.21.15.70:0/1271102465 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:49.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.421+0000 7fd8d228f640 1 --2- 172.21.15.70:0/1271102465 >> v2:172.21.15.181:3300/0 conn(0x7fd8cc0b07d0 0x7fd8cc06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.422+0000 7fd8d0a8c640 1 -- 172.21.15.70:0/1271102465 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739548130 0 0) 0x7fd8cc15f7d0 con 0x7fd8cc0adc70 2023-10-08T05:53:49.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.422+0000 7fd8d0a8c640 1 -- 172.21.15.70:0/1271102465 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b4003520 con 0x7fd8cc0adc70 2023-10-08T05:53:49.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.422+0000 7fd8d1a8e640 1 -- 172.21.15.70:0/1271102465 >> v2:172.21.15.181:3300/0 conn(0x7fd8cc0b07d0 msgr2=0x7fd8cc06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:49.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.422+0000 7fd8d1a8e640 1 --2- 172.21.15.70:0/1271102465 >> v2:172.21.15.181:3300/0 conn(0x7fd8cc0b07d0 0x7fd8cc06cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.422+0000 7fd8d1a8e640 1 -- 172.21.15.70:0/1271102465 >> v1:172.21.15.181:6789/0 conn(0x7fd8cc0adc70 legacy=0x7fd8cc0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.422+0000 7fd8d1a8e640 1 -- 172.21.15.70:0/1271102465 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc06c550 con 0x7fd8cc15ab30 2023-10-08T05:53:49.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.424+0000 7fd8d1a8e640 1 --2- 172.21.15.70:0/1271102465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc15ab30 0x7fd8cc15e8a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009e50 tx=0x7fd8bc02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca63eae5a218b6ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:49.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.424+0000 7fd8d0a8c640 1 -- 172.21.15.70:0/1271102465 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc02cb00 con 0x7fd8cc15ab30 2023-10-08T05:53:49.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.425+0000 7fd8d0a8c640 1 -- 172.21.15.70:0/1271102465 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc02cc60 con 0x7fd8cc15ab30 2023-10-08T05:53:49.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.425+0000 7fd8d0a8c640 1 -- 172.21.15.70:0/1271102465 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0343f0 con 0x7fd8cc15ab30 2023-10-08T05:53:49.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.425+0000 7fd8d2a90640 1 -- 172.21.15.70:0/1271102465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc15ab30 msgr2=0x7fd8cc15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.425+0000 7fd8d2a90640 1 --2- 172.21.15.70:0/1271102465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc15ab30 0x7fd8cc15e8a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009e50 tx=0x7fd8bc02bf60 comp rx=0 tx=0).stop 2023-10-08T05:53:49.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.425+0000 7fd8d2a90640 1 -- 172.21.15.70:0/1271102465 shutdown_connections 2023-10-08T05:53:49.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.425+0000 7fd8d2a90640 1 --2- 172.21.15.70:0/1271102465 >> v2:172.21.15.181:3300/0 conn(0x7fd8cc0b07d0 0x7fd8cc06cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.425+0000 7fd8d2a90640 1 --2- 172.21.15.70:0/1271102465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc15ab30 0x7fd8cc15e8a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.425+0000 7fd8d2a90640 1 -- 172.21.15.70:0/1271102465 >> 172.21.15.70:0/1271102465 conn(0x7fd8cc0cb0c0 msgr2=0x7fd8cc0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:49.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.426+0000 7fd8d2a90640 1 -- 172.21.15.70:0/1271102465 shutdown_connections 2023-10-08T05:53:49.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.426+0000 7fd8d2a90640 1 -- 172.21.15.70:0/1271102465 wait complete. 2023-10-08T05:53:49.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.427+0000 7fd8d2a90640 1 Processor -- start 2023-10-08T05:53:49.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.427+0000 7fd8d2a90640 1 -- start start 2023-10-08T05:53:49.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.428+0000 7fd8d2a90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc0adc70 0x7fd8cc0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.428+0000 7fd8d2a90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8cc0b07d0 0x7fd8cc14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.428+0000 7fd8d2a90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8cc15ab30 0x7fd8cc15c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.428+0000 7fd8d2a90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc15ddd0 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.428+0000 7fd8d2a90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc15b2c0 con 0x7fd8cc15ab30 2023-10-08T05:53:49.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.428+0000 7fd8d2a90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc15d530 con 0x7fd8cc0adc70 2023-10-08T05:53:49.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d128d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8cc0b07d0 0x7fd8cc14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d128d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8cc0b07d0 0x7fd8cc14c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51570/0 (socket says 172.21.15.70:51570) 2023-10-08T05:53:49.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d128d640 1 -- 172.21.15.70:0/2357969914 learned_addr learned my addr 172.21.15.70:0/2357969914 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:49.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d1a8e640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc0adc70 0x7fd8cc0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d228f640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8cc15ab30 0x7fd8cc15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d128d640 1 -- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc0adc70 msgr2=0x7fd8cc0c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d128d640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc0adc70 0x7fd8cc0c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d128d640 1 -- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8cc15ab30 msgr2=0x7fd8cc15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d128d640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8cc15ab30 0x7fd8cc15c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d128d640 1 -- 172.21.15.70:0/2357969914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc009b30 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.429+0000 7fd8d128d640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8cc0b07d0 0x7fd8cc14c890 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000c490 tx=0x7fd8c000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.430+0000 7fd8ca7fc640 1 -- 172.21.15.70:0/2357969914 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00052d0 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.430+0000 7fd8ca7fc640 1 -- 172.21.15.70:0/2357969914 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c0005430 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.430+0000 7fd8d2a90640 1 -- 172.21.15.70:0/2357969914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8cc15d810 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.430+0000 7fd8ca7fc640 1 -- 172.21.15.70:0/2357969914 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0012870 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.430+0000 7fd8d2a90640 1 -- 172.21.15.70:0/2357969914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8cc14cdd0 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.431+0000 7fd8d2a90640 1 -- 172.21.15.70:0/2357969914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8cc0ca260 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.432+0000 7fd8ca7fc640 1 -- 172.21.15.70:0/2357969914 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd8c00129d0 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.433+0000 7fd8ca7fc640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd8b806afc0 0x7fd8b806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.433+0000 7fd8ca7fc640 1 -- 172.21.15.70:0/2357969914 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd8c00894a0 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.434+0000 7fd8d1a8e640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd8b806afc0 0x7fd8b806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.435+0000 7fd8d1a8e640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd8b806afc0 0x7fd8b806d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc035000 tx=0x7fd8bc030000 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:49.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.438+0000 7fd8ca7fc640 1 -- 172.21.15.70:0/2357969914 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8c00542d0 con 0x7fd8cc0b07d0 2023-10-08T05:53:49.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.624+0000 7fd8d2a90640 1 -- 172.21.15.70:0/2357969914 --> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8cc14cf90 con 0x7fd8b806afc0 2023-10-08T05:53:49.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.647+0000 7fd8ca7fc640 1 -- 172.21.15.70:0/2357969914 <== mgr.5513 v2:172.21.15.181:6832/1906631919 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287573 (secure 0 0 0) 0x7fd8cc14cf90 con 0x7fd8b806afc0 2023-10-08T05:53:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:49.653 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:53:49.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.653+0000 7fd8d2a90640 1 -- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd8b806afc0 msgr2=0x7fd8b806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.653+0000 7fd8d2a90640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd8b806afc0 0x7fd8b806d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc035000 tx=0x7fd8bc030000 comp rx=0 tx=0).stop 2023-10-08T05:53:49.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.653+0000 7fd8d2a90640 1 -- 172.21.15.70:0/2357969914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8cc0b07d0 msgr2=0x7fd8cc14c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.653+0000 7fd8d2a90640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8cc0b07d0 0x7fd8cc14c890 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000c490 tx=0x7fd8c000c950 comp rx=0 tx=0).stop 2023-10-08T05:53:49.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.653+0000 7fd8d2a90640 1 -- 172.21.15.70:0/2357969914 shutdown_connections 2023-10-08T05:53:49.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.653+0000 7fd8d2a90640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7fd8b806afc0 0x7fd8b806d480 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.653+0000 7fd8d2a90640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8cc15ab30 0x7fd8cc15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.653+0000 7fd8d2a90640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8cc0b07d0 0x7fd8cc14c890 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.653+0000 7fd8d2a90640 1 --2- 172.21.15.70:0/2357969914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8cc0adc70 0x7fd8cc0c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.653+0000 7fd8d2a90640 1 -- 172.21.15.70:0/2357969914 >> 172.21.15.70:0/2357969914 conn(0x7fd8cc0cb0c0 msgr2=0x7fd8cc0ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:49.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.654+0000 7fd8d2a90640 1 -- 172.21.15.70:0/2357969914 shutdown_connections 2023-10-08T05:53:49.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.654+0000 7fd8d2a90640 1 -- 172.21.15.70:0/2357969914 wait complete. 2023-10-08T05:53:49.673 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-08T05:53:47.751480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":765,"num_read_kb":1528,"num_write":333,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2679,"ondisk_log_size":2679,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":318968,"kb_used_data":12584,"kb_used_omap":12,"kb_used_meta":306227,"kb_avail":754655752,"statfs":{"total":773094113280,"available":772767490048,"internally_reserved":0,"allocated":12886016,"data_stored":7071780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":313577019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.080724"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.773202+0000","last_change":"2023-10-08T05:53:44.732254+0000","last_active":"2023-10-08T05:53:45.773202+0000","last_peered":"2023-10-08T05:53:45.773202+0000","last_clean":"2023-10-08T05:53:45.773202+0000","last_became_active":"2023-10-08T05:53:44.731866+0000","last_became_peered":"2023-10-08T05:53:44.731866+0000","last_unstale":"2023-10-08T05:53:45.773202+0000","last_undegraded":"2023-10-08T05:53:45.773202+0000","last_fullsized":"2023-10-08T05:53:45.773202+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:07:40.057790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736686+0000","last_change":"2023-10-08T05:53:44.734557+0000","last_active":"2023-10-08T05:53:45.736686+0000","last_peered":"2023-10-08T05:53:45.736686+0000","last_clean":"2023-10-08T05:53:45.736686+0000","last_became_active":"2023-10-08T05:53:44.734245+0000","last_became_peered":"2023-10-08T05:53:44.734245+0000","last_unstale":"2023-10-08T05:53:45.736686+0000","last_undegraded":"2023-10-08T05:53:45.736686+0000","last_fullsized":"2023-10-08T05:53:45.736686+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:38.558149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772630+0000","last_change":"2023-10-08T05:53:44.751821+0000","last_active":"2023-10-08T05:53:45.772630+0000","last_peered":"2023-10-08T05:53:45.772630+0000","last_clean":"2023-10-08T05:53:45.772630+0000","last_became_active":"2023-10-08T05:53:44.751512+0000","last_became_peered":"2023-10-08T05:53:44.751512+0000","last_unstale":"2023-10-08T05:53:45.772630+0000","last_undegraded":"2023-10-08T05:53:45.772630+0000","last_fullsized":"2023-10-08T05:53:45.772630+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:05:54.492792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768827+0000","last_change":"2023-10-08T05:53:44.740032+0000","last_active":"2023-10-08T05:53:45.768827+0000","last_peered":"2023-10-08T05:53:45.768827+0000","last_clean":"2023-10-08T05:53:45.768827+0000","last_became_active":"2023-10-08T05:53:44.739367+0000","last_became_peered":"2023-10-08T05:53:44.739367+0000","last_unstale":"2023-10-08T05:53:45.768827+0000","last_undegraded":"2023-10-08T05:53:45.768827+0000","last_fullsized":"2023-10-08T05:53:45.768827+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:38:10.428853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769757+0000","last_change":"2023-10-08T05:53:44.737746+0000","last_active":"2023-10-08T05:53:45.769757+0000","last_peered":"2023-10-08T05:53:45.769757+0000","last_clean":"2023-10-08T05:53:45.769757+0000","last_became_active":"2023-10-08T05:53:44.737418+0000","last_became_peered":"2023-10-08T05:53:44.737418+0000","last_unstale":"2023-10-08T05:53:45.769757+0000","last_undegraded":"2023-10-08T05:53:45.769757+0000","last_fullsized":"2023-10-08T05:53:45.769757+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:11.862363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769975+0000","last_change":"2023-10-08T05:53:44.735914+0000","last_active":"2023-10-08T05:53:45.769975+0000","last_peered":"2023-10-08T05:53:45.769975+0000","last_clean":"2023-10-08T05:53:45.769975+0000","last_became_active":"2023-10-08T05:53:44.735446+0000","last_became_peered":"2023-10-08T05:53:44.735446+0000","last_unstale":"2023-10-08T05:53:45.769975+0000","last_undegraded":"2023-10-08T05:53:45.769975+0000","last_fullsized":"2023-10-08T05:53:45.769975+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:31.564091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749893+0000","last_change":"2023-10-08T05:53:44.735531+0000","last_active":"2023-10-08T05:53:45.749893+0000","last_peered":"2023-10-08T05:53:45.749893+0000","last_clean":"2023-10-08T05:53:45.749893+0000","last_became_active":"2023-10-08T05:53:44.733259+0000","last_became_peered":"2023-10-08T05:53:44.733259+0000","last_unstale":"2023-10-08T05:53:45.749893+0000","last_undegraded":"2023-10-08T05:53:45.749893+0000","last_fullsized":"2023-10-08T05:53:45.749893+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:25.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768328+0000","last_change":"2023-10-08T05:53:44.743714+0000","last_active":"2023-10-08T05:53:45.768328+0000","last_peered":"2023-10-08T05:53:45.768328+0000","last_clean":"2023-10-08T05:53:45.768328+0000","last_became_active":"2023-10-08T05:53:44.743190+0000","last_became_peered":"2023-10-08T05:53:44.743190+0000","last_unstale":"2023-10-08T05:53:45.768328+0000","last_undegraded":"2023-10-08T05:53:45.768328+0000","last_fullsized":"2023-10-08T05:53:45.768328+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:18.838107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769733+0000","last_change":"2023-10-08T05:53:44.730820+0000","last_active":"2023-10-08T05:53:45.769733+0000","last_peered":"2023-10-08T05:53:45.769733+0000","last_clean":"2023-10-08T05:53:45.769733+0000","last_became_active":"2023-10-08T05:53:44.730554+0000","last_became_peered":"2023-10-08T05:53:44.730554+0000","last_unstale":"2023-10-08T05:53:45.769733+0000","last_undegraded":"2023-10-08T05:53:45.769733+0000","last_fullsized":"2023-10-08T05:53:45.769733+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:13.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.730270+0000","last_change":"2023-10-08T05:53:44.737666+0000","last_active":"2023-10-08T05:53:45.730270+0000","last_peered":"2023-10-08T05:53:45.730270+0000","last_clean":"2023-10-08T05:53:45.730270+0000","last_became_active":"2023-10-08T05:53:44.737405+0000","last_became_peered":"2023-10-08T05:53:44.737405+0000","last_unstale":"2023-10-08T05:53:45.730270+0000","last_undegraded":"2023-10-08T05:53:45.730270+0000","last_fullsized":"2023-10-08T05:53:45.730270+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:03.246411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770006+0000","last_change":"2023-10-08T05:53:44.734979+0000","last_active":"2023-10-08T05:53:45.770006+0000","last_peered":"2023-10-08T05:53:45.770006+0000","last_clean":"2023-10-08T05:53:45.770006+0000","last_became_active":"2023-10-08T05:53:44.734683+0000","last_became_peered":"2023-10-08T05:53:44.734683+0000","last_unstale":"2023-10-08T05:53:45.770006+0000","last_undegraded":"2023-10-08T05:53:45.770006+0000","last_fullsized":"2023-10-08T05:53:45.770006+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:18.838107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.730233+0000","last_change":"2023-10-08T05:53:44.733196+0000","last_active":"2023-10-08T05:53:45.730233+0000","last_peered":"2023-10-08T05:53:45.730233+0000","last_clean":"2023-10-08T05:53:45.730233+0000","last_became_active":"2023-10-08T05:53:44.732929+0000","last_became_peered":"2023-10-08T05:53:44.732929+0000","last_unstale":"2023-10-08T05:53:45.730233+0000","last_undegraded":"2023-10-08T05:53:45.730233+0000","last_fullsized":"2023-10-08T05:53:45.730233+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:42.779477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769219+0000","last_change":"2023-10-08T05:53:44.742293+0000","last_active":"2023-10-08T05:53:45.769219+0000","last_peered":"2023-10-08T05:53:45.769219+0000","last_clean":"2023-10-08T05:53:45.769219+0000","last_became_active":"2023-10-08T05:53:44.742012+0000","last_became_peered":"2023-10-08T05:53:44.742012+0000","last_unstale":"2023-10-08T05:53:45.769219+0000","last_undegraded":"2023-10-08T05:53:45.769219+0000","last_fullsized":"2023-10-08T05:53:45.769219+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:40:51.342521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769458+0000","last_change":"2023-10-08T05:53:44.748135+0000","last_active":"2023-10-08T05:53:45.769458+0000","last_peered":"2023-10-08T05:53:45.769458+0000","last_clean":"2023-10-08T05:53:45.769458+0000","last_became_active":"2023-10-08T05:53:44.747810+0000","last_became_peered":"2023-10-08T05:53:44.747810+0000","last_unstale":"2023-10-08T05:53:45.769458+0000","last_undegraded":"2023-10-08T05:53:45.769458+0000","last_fullsized":"2023-10-08T05:53:45.769458+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:59.155842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770593+0000","last_change":"2023-10-08T05:53:44.736760+0000","last_active":"2023-10-08T05:53:45.770593+0000","last_peered":"2023-10-08T05:53:45.770593+0000","last_clean":"2023-10-08T05:53:45.770593+0000","last_became_active":"2023-10-08T05:53:44.736481+0000","last_became_peered":"2023-10-08T05:53:44.736481+0000","last_unstale":"2023-10-08T05:53:45.770593+0000","last_undegraded":"2023-10-08T05:53:45.770593+0000","last_fullsized":"2023-10-08T05:53:45.770593+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:39.108631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769354+0000","last_change":"2023-10-08T05:53:44.739039+0000","last_active":"2023-10-08T05:53:45.769354+0000","last_peered":"2023-10-08T05:53:45.769354+0000","last_clean":"2023-10-08T05:53:45.769354+0000","last_became_active":"2023-10-08T05:53:44.738624+0000","last_became_peered":"2023-10-08T05:53:44.738624+0000","last_unstale":"2023-10-08T05:53:45.769354+0000","last_undegraded":"2023-10-08T05:53:45.769354+0000","last_fullsized":"2023-10-08T05:53:45.769354+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:35.705136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772315+0000","last_change":"2023-10-08T05:53:44.741912+0000","last_active":"2023-10-08T05:53:45.772315+0000","last_peered":"2023-10-08T05:53:45.772315+0000","last_clean":"2023-10-08T05:53:45.772315+0000","last_became_active":"2023-10-08T05:53:44.740692+0000","last_became_peered":"2023-10-08T05:53:44.740692+0000","last_unstale":"2023-10-08T05:53:45.772315+0000","last_undegraded":"2023-10-08T05:53:45.772315+0000","last_fullsized":"2023-10-08T05:53:45.772315+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:16:13.976999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772859+0000","last_change":"2023-10-08T05:53:44.731955+0000","last_active":"2023-10-08T05:53:45.772859+0000","last_peered":"2023-10-08T05:53:45.772859+0000","last_clean":"2023-10-08T05:53:45.772859+0000","last_became_active":"2023-10-08T05:53:44.731619+0000","last_became_peered":"2023-10-08T05:53:44.731619+0000","last_unstale":"2023-10-08T05:53:45.772859+0000","last_undegraded":"2023-10-08T05:53:45.772859+0000","last_fullsized":"2023-10-08T05:53:45.772859+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:29:19.871040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770222+0000","last_change":"2023-10-08T05:53:44.747971+0000","last_active":"2023-10-08T05:53:45.770222+0000","last_peered":"2023-10-08T05:53:45.770222+0000","last_clean":"2023-10-08T05:53:45.770222+0000","last_became_active":"2023-10-08T05:53:44.746820+0000","last_became_peered":"2023-10-08T05:53:44.746820+0000","last_unstale":"2023-10-08T05:53:45.770222+0000","last_undegraded":"2023-10-08T05:53:45.770222+0000","last_fullsized":"2023-10-08T05:53:45.770222+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:28.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769127+0000","last_change":"2023-10-08T05:53:44.738631+0000","last_active":"2023-10-08T05:53:45.769127+0000","last_peered":"2023-10-08T05:53:45.769127+0000","last_clean":"2023-10-08T05:53:45.769127+0000","last_became_active":"2023-10-08T05:53:44.738297+0000","last_became_peered":"2023-10-08T05:53:44.738297+0000","last_unstale":"2023-10-08T05:53:45.769127+0000","last_undegraded":"2023-10-08T05:53:45.769127+0000","last_fullsized":"2023-10-08T05:53:45.769127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:05:51.896763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769227+0000","last_change":"2023-10-08T05:53:44.737940+0000","last_active":"2023-10-08T05:53:45.769227+0000","last_peered":"2023-10-08T05:53:45.769227+0000","last_clean":"2023-10-08T05:53:45.769227+0000","last_became_active":"2023-10-08T05:53:44.737614+0000","last_became_peered":"2023-10-08T05:53:44.737614+0000","last_unstale":"2023-10-08T05:53:45.769227+0000","last_undegraded":"2023-10-08T05:53:45.769227+0000","last_fullsized":"2023-10-08T05:53:45.769227+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:51:26.444102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769217+0000","last_change":"2023-10-08T05:53:44.743404+0000","last_active":"2023-10-08T05:53:45.769217+0000","last_peered":"2023-10-08T05:53:45.769217+0000","last_clean":"2023-10-08T05:53:45.769217+0000","last_became_active":"2023-10-08T05:53:44.741563+0000","last_became_peered":"2023-10-08T05:53:44.741563+0000","last_unstale":"2023-10-08T05:53:45.769217+0000","last_undegraded":"2023-10-08T05:53:45.769217+0000","last_fullsized":"2023-10-08T05:53:45.769217+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:53.829681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.734832+0000","last_change":"2023-10-08T05:53:44.742183+0000","last_active":"2023-10-08T05:53:45.734832+0000","last_peered":"2023-10-08T05:53:45.734832+0000","last_clean":"2023-10-08T05:53:45.734832+0000","last_became_active":"2023-10-08T05:53:44.741849+0000","last_became_peered":"2023-10-08T05:53:44.741849+0000","last_unstale":"2023-10-08T05:53:45.734832+0000","last_undegraded":"2023-10-08T05:53:45.734832+0000","last_fullsized":"2023-10-08T05:53:45.734832+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:27.089669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772806+0000","last_change":"2023-10-08T05:53:44.736682+0000","last_active":"2023-10-08T05:53:45.772806+0000","last_peered":"2023-10-08T05:53:45.772806+0000","last_clean":"2023-10-08T05:53:45.772806+0000","last_became_active":"2023-10-08T05:53:44.736396+0000","last_became_peered":"2023-10-08T05:53:44.736396+0000","last_unstale":"2023-10-08T05:53:45.772806+0000","last_undegraded":"2023-10-08T05:53:45.772806+0000","last_fullsized":"2023-10-08T05:53:45.772806+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:42.300218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736826+0000","last_change":"2023-10-08T05:53:44.734308+0000","last_active":"2023-10-08T05:53:45.736826+0000","last_peered":"2023-10-08T05:53:45.736826+0000","last_clean":"2023-10-08T05:53:45.736826+0000","last_became_active":"2023-10-08T05:53:44.733982+0000","last_became_peered":"2023-10-08T05:53:44.733982+0000","last_unstale":"2023-10-08T05:53:45.736826+0000","last_undegraded":"2023-10-08T05:53:45.736826+0000","last_fullsized":"2023-10-08T05:53:45.736826+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:09.123159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769851+0000","last_change":"2023-10-08T05:53:44.743498+0000","last_active":"2023-10-08T05:53:45.769851+0000","last_peered":"2023-10-08T05:53:45.769851+0000","last_clean":"2023-10-08T05:53:45.769851+0000","last_became_active":"2023-10-08T05:53:44.740098+0000","last_became_peered":"2023-10-08T05:53:44.740098+0000","last_unstale":"2023-10-08T05:53:45.769851+0000","last_undegraded":"2023-10-08T05:53:45.769851+0000","last_fullsized":"2023-10-08T05:53:45.769851+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:10.943322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.730480+0000","last_change":"2023-10-08T05:53:44.729943+0000","last_active":"2023-10-08T05:53:45.730480+0000","last_peered":"2023-10-08T05:53:45.730480+0000","last_clean":"2023-10-08T05:53:45.730480+0000","last_became_active":"2023-10-08T05:53:44.729680+0000","last_became_peered":"2023-10-08T05:53:44.729680+0000","last_unstale":"2023-10-08T05:53:45.730480+0000","last_undegraded":"2023-10-08T05:53:45.730480+0000","last_fullsized":"2023-10-08T05:53:45.730480+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:31.314169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768953+0000","last_change":"2023-10-08T05:53:44.750223+0000","last_active":"2023-10-08T05:53:45.768953+0000","last_peered":"2023-10-08T05:53:45.768953+0000","last_clean":"2023-10-08T05:53:45.768953+0000","last_became_active":"2023-10-08T05:53:44.749962+0000","last_became_peered":"2023-10-08T05:53:44.749962+0000","last_unstale":"2023-10-08T05:53:45.768953+0000","last_undegraded":"2023-10-08T05:53:45.768953+0000","last_fullsized":"2023-10-08T05:53:45.768953+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:29:30.965219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749806+0000","last_change":"2023-10-08T05:53:44.735327+0000","last_active":"2023-10-08T05:53:45.749806+0000","last_peered":"2023-10-08T05:53:45.749806+0000","last_clean":"2023-10-08T05:53:45.749806+0000","last_became_active":"2023-10-08T05:53:44.734968+0000","last_became_peered":"2023-10-08T05:53:44.734968+0000","last_unstale":"2023-10-08T05:53:45.749806+0000","last_undegraded":"2023-10-08T05:53:45.749806+0000","last_fullsized":"2023-10-08T05:53:45.749806+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:07:40.057790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769824+0000","last_change":"2023-10-08T05:53:44.728034+0000","last_active":"2023-10-08T05:53:45.769824+0000","last_peered":"2023-10-08T05:53:45.769824+0000","last_clean":"2023-10-08T05:53:45.769824+0000","last_became_active":"2023-10-08T05:53:44.727758+0000","last_became_peered":"2023-10-08T05:53:44.727758+0000","last_unstale":"2023-10-08T05:53:45.769824+0000","last_undegraded":"2023-10-08T05:53:45.769824+0000","last_fullsized":"2023-10-08T05:53:45.769824+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:44.157586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735998+0000","last_change":"2023-10-08T05:53:44.747259+0000","last_active":"2023-10-08T05:53:45.735998+0000","last_peered":"2023-10-08T05:53:45.735998+0000","last_clean":"2023-10-08T05:53:45.735998+0000","last_became_active":"2023-10-08T05:53:44.746949+0000","last_became_peered":"2023-10-08T05:53:44.746949+0000","last_unstale":"2023-10-08T05:53:45.735998+0000","last_undegraded":"2023-10-08T05:53:45.735998+0000","last_fullsized":"2023-10-08T05:53:45.735998+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:29:19.871040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769714+0000","last_change":"2023-10-08T05:53:44.739131+0000","last_active":"2023-10-08T05:53:45.769714+0000","last_peered":"2023-10-08T05:53:45.769714+0000","last_clean":"2023-10-08T05:53:45.769714+0000","last_became_active":"2023-10-08T05:53:44.735989+0000","last_became_peered":"2023-10-08T05:53:44.735989+0000","last_unstale":"2023-10-08T05:53:45.769714+0000","last_undegraded":"2023-10-08T05:53:45.769714+0000","last_fullsized":"2023-10-08T05:53:45.769714+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:32:44.279315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769595+0000","last_change":"2023-10-08T05:53:44.751155+0000","last_active":"2023-10-08T05:53:45.769595+0000","last_peered":"2023-10-08T05:53:45.769595+0000","last_clean":"2023-10-08T05:53:45.769595+0000","last_became_active":"2023-10-08T05:53:44.750892+0000","last_became_peered":"2023-10-08T05:53:44.750892+0000","last_unstale":"2023-10-08T05:53:45.769595+0000","last_undegraded":"2023-10-08T05:53:45.769595+0000","last_fullsized":"2023-10-08T05:53:45.769595+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:24.878153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"117'4","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735631+0000","last_change":"2023-10-08T05:53:43.708846+0000","last_active":"2023-10-08T05:53:45.735631+0000","last_peered":"2023-10-08T05:53:45.735631+0000","last_clean":"2023-10-08T05:53:45.735631+0000","last_became_active":"2023-10-08T05:53:43.708567+0000","last_became_peered":"2023-10-08T05:53:43.708567+0000","last_unstale":"2023-10-08T05:53:45.735631+0000","last_undegraded":"2023-10-08T05:53:45.735631+0000","last_fullsized":"2023-10-08T05:53:45.735631+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:22.341562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.771817+0000","last_change":"2023-10-08T05:53:44.734771+0000","last_active":"2023-10-08T05:53:45.771817+0000","last_peered":"2023-10-08T05:53:45.771817+0000","last_clean":"2023-10-08T05:53:45.771817+0000","last_became_active":"2023-10-08T05:53:44.734448+0000","last_became_peered":"2023-10-08T05:53:44.734448+0000","last_unstale":"2023-10-08T05:53:45.771817+0000","last_undegraded":"2023-10-08T05:53:45.771817+0000","last_fullsized":"2023-10-08T05:53:45.771817+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:17.047581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770064+0000","last_change":"2023-10-08T05:53:43.716282+0000","last_active":"2023-10-08T05:53:45.770064+0000","last_peered":"2023-10-08T05:53:45.770064+0000","last_clean":"2023-10-08T05:53:45.770064+0000","last_became_active":"2023-10-08T05:53:43.715918+0000","last_became_peered":"2023-10-08T05:53:43.715918+0000","last_unstale":"2023-10-08T05:53:45.770064+0000","last_undegraded":"2023-10-08T05:53:45.770064+0000","last_fullsized":"2023-10-08T05:53:45.770064+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:07.715363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.750110+0000","last_change":"2023-10-08T05:53:44.736217+0000","last_active":"2023-10-08T05:53:45.750110+0000","last_peered":"2023-10-08T05:53:45.750110+0000","last_clean":"2023-10-08T05:53:45.750110+0000","last_became_active":"2023-10-08T05:53:44.735922+0000","last_became_peered":"2023-10-08T05:53:44.735922+0000","last_unstale":"2023-10-08T05:53:45.750110+0000","last_undegraded":"2023-10-08T05:53:45.750110+0000","last_fullsized":"2023-10-08T05:53:45.750110+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:09.123159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749887+0000","last_change":"2023-10-08T05:53:43.712447+0000","last_active":"2023-10-08T05:53:45.749887+0000","last_peered":"2023-10-08T05:53:45.749887+0000","last_clean":"2023-10-08T05:53:45.749887+0000","last_became_active":"2023-10-08T05:53:43.712167+0000","last_became_peered":"2023-10-08T05:53:43.712167+0000","last_unstale":"2023-10-08T05:53:45.749887+0000","last_undegraded":"2023-10-08T05:53:45.749887+0000","last_fullsized":"2023-10-08T05:53:45.749887+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:16.037633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772381+0000","last_change":"2023-10-08T05:53:44.741309+0000","last_active":"2023-10-08T05:53:45.772381+0000","last_peered":"2023-10-08T05:53:45.772381+0000","last_clean":"2023-10-08T05:53:45.772381+0000","last_became_active":"2023-10-08T05:53:44.740260+0000","last_became_peered":"2023-10-08T05:53:44.740260+0000","last_unstale":"2023-10-08T05:53:45.772381+0000","last_undegraded":"2023-10-08T05:53:45.772381+0000","last_fullsized":"2023-10-08T05:53:45.772381+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:09.787683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735306+0000","last_change":"2023-10-08T05:53:43.710659+0000","last_active":"2023-10-08T05:53:45.735306+0000","last_peered":"2023-10-08T05:53:45.735306+0000","last_clean":"2023-10-08T05:53:45.735306+0000","last_became_active":"2023-10-08T05:53:43.710395+0000","last_became_peered":"2023-10-08T05:53:43.710395+0000","last_unstale":"2023-10-08T05:53:45.735306+0000","last_undegraded":"2023-10-08T05:53:45.735306+0000","last_fullsized":"2023-10-08T05:53:45.735306+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:29.336481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768504+0000","last_change":"2023-10-08T05:53:44.738694+0000","last_active":"2023-10-08T05:53:45.768504+0000","last_peered":"2023-10-08T05:53:45.768504+0000","last_clean":"2023-10-08T05:53:45.768504+0000","last_became_active":"2023-10-08T05:53:44.738398+0000","last_became_peered":"2023-10-08T05:53:44.738398+0000","last_unstale":"2023-10-08T05:53:45.768504+0000","last_undegraded":"2023-10-08T05:53:45.768504+0000","last_fullsized":"2023-10-08T05:53:45.768504+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:37:27.581656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769991+0000","last_change":"2023-10-08T05:53:43.707124+0000","last_active":"2023-10-08T05:53:45.769991+0000","last_peered":"2023-10-08T05:53:45.769991+0000","last_clean":"2023-10-08T05:53:45.769991+0000","last_became_active":"2023-10-08T05:53:43.706820+0000","last_became_peered":"2023-10-08T05:53:43.706820+0000","last_unstale":"2023-10-08T05:53:45.769991+0000","last_undegraded":"2023-10-08T05:53:45.769991+0000","last_fullsized":"2023-10-08T05:53:45.769991+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:13.519501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.734832+0000","last_change":"2023-10-08T05:53:44.731745+0000","last_active":"2023-10-08T05:53:45.734832+0000","last_peered":"2023-10-08T05:53:45.734832+0000","last_clean":"2023-10-08T05:53:45.734832+0000","last_became_active":"2023-10-08T05:53:44.731465+0000","last_became_peered":"2023-10-08T05:53:44.731465+0000","last_unstale":"2023-10-08T05:53:45.734832+0000","last_undegraded":"2023-10-08T05:53:45.734832+0000","last_fullsized":"2023-10-08T05:53:45.734832+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:01.277669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.771944+0000","last_change":"2023-10-08T05:53:43.707705+0000","last_active":"2023-10-08T05:53:45.771944+0000","last_peered":"2023-10-08T05:53:45.771944+0000","last_clean":"2023-10-08T05:53:45.771944+0000","last_became_active":"2023-10-08T05:53:43.707370+0000","last_became_peered":"2023-10-08T05:53:43.707370+0000","last_unstale":"2023-10-08T05:53:45.771944+0000","last_undegraded":"2023-10-08T05:53:45.771944+0000","last_fullsized":"2023-10-08T05:53:45.771944+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:47:09.933374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770304+0000","last_change":"2023-10-08T05:53:44.735736+0000","last_active":"2023-10-08T05:53:45.770304+0000","last_peered":"2023-10-08T05:53:45.770304+0000","last_clean":"2023-10-08T05:53:45.770304+0000","last_became_active":"2023-10-08T05:53:44.734877+0000","last_became_peered":"2023-10-08T05:53:44.734877+0000","last_unstale":"2023-10-08T05:53:45.770304+0000","last_undegraded":"2023-10-08T05:53:45.770304+0000","last_fullsized":"2023-10-08T05:53:45.770304+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:58:53.628913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.773326+0000","last_change":"2023-10-08T05:53:43.706120+0000","last_active":"2023-10-08T05:53:45.773326+0000","last_peered":"2023-10-08T05:53:45.773326+0000","last_clean":"2023-10-08T05:53:45.773326+0000","last_became_active":"2023-10-08T05:53:43.705815+0000","last_became_peered":"2023-10-08T05:53:43.705815+0000","last_unstale":"2023-10-08T05:53:45.773326+0000","last_undegraded":"2023-10-08T05:53:45.773326+0000","last_fullsized":"2023-10-08T05:53:45.773326+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:08.777735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769580+0000","last_change":"2023-10-08T05:53:44.739827+0000","last_active":"2023-10-08T05:53:45.769580+0000","last_peered":"2023-10-08T05:53:45.769580+0000","last_clean":"2023-10-08T05:53:45.769580+0000","last_became_active":"2023-10-08T05:53:44.739401+0000","last_became_peered":"2023-10-08T05:53:44.739401+0000","last_unstale":"2023-10-08T05:53:45.769580+0000","last_undegraded":"2023-10-08T05:53:45.769580+0000","last_fullsized":"2023-10-08T05:53:45.769580+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:22:01.118078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749480+0000","last_change":"2023-10-08T05:53:43.710254+0000","last_active":"2023-10-08T05:53:45.749480+0000","last_peered":"2023-10-08T05:53:45.749480+0000","last_clean":"2023-10-08T05:53:45.749480+0000","last_became_active":"2023-10-08T05:53:43.709935+0000","last_became_peered":"2023-10-08T05:53:43.709935+0000","last_unstale":"2023-10-08T05:53:45.749480+0000","last_undegraded":"2023-10-08T05:53:45.749480+0000","last_fullsized":"2023-10-08T05:53:45.749480+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:05:53.482845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769714+0000","last_change":"2023-10-08T05:53:44.732406+0000","last_active":"2023-10-08T05:53:45.769714+0000","last_peered":"2023-10-08T05:53:45.769714+0000","last_clean":"2023-10-08T05:53:45.769714+0000","last_became_active":"2023-10-08T05:53:44.732141+0000","last_became_peered":"2023-10-08T05:53:44.732141+0000","last_unstale":"2023-10-08T05:53:45.769714+0000","last_undegraded":"2023-10-08T05:53:45.769714+0000","last_fullsized":"2023-10-08T05:53:45.769714+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:33:04.471656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736941+0000","last_change":"2023-10-08T05:53:43.703549+0000","last_active":"2023-10-08T05:53:45.736941+0000","last_peered":"2023-10-08T05:53:45.736941+0000","last_clean":"2023-10-08T05:53:45.736941+0000","last_became_active":"2023-10-08T05:53:43.703188+0000","last_became_peered":"2023-10-08T05:53:43.703188+0000","last_unstale":"2023-10-08T05:53:45.736941+0000","last_undegraded":"2023-10-08T05:53:45.736941+0000","last_fullsized":"2023-10-08T05:53:45.736941+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:41.290271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749871+0000","last_change":"2023-10-08T05:53:44.750298+0000","last_active":"2023-10-08T05:53:45.749871+0000","last_peered":"2023-10-08T05:53:45.749871+0000","last_clean":"2023-10-08T05:53:45.749871+0000","last_became_active":"2023-10-08T05:53:44.749727+0000","last_became_peered":"2023-10-08T05:53:44.749727+0000","last_unstale":"2023-10-08T05:53:45.749871+0000","last_undegraded":"2023-10-08T05:53:45.749871+0000","last_fullsized":"2023-10-08T05:53:45.749871+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:38.558149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.773139+0000","last_change":"2023-10-08T05:53:43.704891+0000","last_active":"2023-10-08T05:53:45.773139+0000","last_peered":"2023-10-08T05:53:45.773139+0000","last_clean":"2023-10-08T05:53:45.773139+0000","last_became_active":"2023-10-08T05:53:43.704611+0000","last_became_peered":"2023-10-08T05:53:43.704611+0000","last_unstale":"2023-10-08T05:53:45.773139+0000","last_undegraded":"2023-10-08T05:53:45.773139+0000","last_fullsized":"2023-10-08T05:53:45.773139+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:26.253802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.773047+0000","last_change":"2023-10-08T05:53:44.731169+0000","last_active":"2023-10-08T05:53:45.773047+0000","last_peered":"2023-10-08T05:53:45.773047+0000","last_clean":"2023-10-08T05:53:45.773047+0000","last_became_active":"2023-10-08T05:53:44.730888+0000","last_became_peered":"2023-10-08T05:53:44.730888+0000","last_unstale":"2023-10-08T05:53:45.773047+0000","last_undegraded":"2023-10-08T05:53:45.773047+0000","last_fullsized":"2023-10-08T05:53:45.773047+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:25.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"117'3","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770460+0000","last_change":"2023-10-08T05:53:43.711687+0000","last_active":"2023-10-08T05:53:45.770460+0000","last_peered":"2023-10-08T05:53:45.770460+0000","last_clean":"2023-10-08T05:53:45.770460+0000","last_became_active":"2023-10-08T05:53:43.711419+0000","last_became_peered":"2023-10-08T05:53:43.711419+0000","last_unstale":"2023-10-08T05:53:45.770460+0000","last_undegraded":"2023-10-08T05:53:45.770460+0000","last_fullsized":"2023-10-08T05:53:45.770460+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:26.817989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770697+0000","last_change":"2023-10-08T05:53:44.739401+0000","last_active":"2023-10-08T05:53:45.770697+0000","last_peered":"2023-10-08T05:53:45.770697+0000","last_clean":"2023-10-08T05:53:45.770697+0000","last_became_active":"2023-10-08T05:53:44.739123+0000","last_became_peered":"2023-10-08T05:53:44.739123+0000","last_unstale":"2023-10-08T05:53:45.770697+0000","last_undegraded":"2023-10-08T05:53:45.770697+0000","last_fullsized":"2023-10-08T05:53:45.770697+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:14:50.756106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"117'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.771075+0000","last_change":"2023-10-08T05:53:43.711065+0000","last_active":"2023-10-08T05:53:45.771075+0000","last_peered":"2023-10-08T05:53:45.771075+0000","last_clean":"2023-10-08T05:53:45.771075+0000","last_became_active":"2023-10-08T05:53:43.710777+0000","last_became_peered":"2023-10-08T05:53:43.710777+0000","last_unstale":"2023-10-08T05:53:45.771075+0000","last_undegraded":"2023-10-08T05:53:45.771075+0000","last_fullsized":"2023-10-08T05:53:45.771075+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:48.331810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768728+0000","last_change":"2023-10-08T05:53:44.743530+0000","last_active":"2023-10-08T05:53:45.768728+0000","last_peered":"2023-10-08T05:53:45.768728+0000","last_clean":"2023-10-08T05:53:45.768728+0000","last_became_active":"2023-10-08T05:53:44.742774+0000","last_became_peered":"2023-10-08T05:53:44.742774+0000","last_unstale":"2023-10-08T05:53:45.768728+0000","last_undegraded":"2023-10-08T05:53:45.768728+0000","last_fullsized":"2023-10-08T05:53:45.768728+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:28.555393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749867+0000","last_change":"2023-10-08T05:53:43.711053+0000","last_active":"2023-10-08T05:53:45.749867+0000","last_peered":"2023-10-08T05:53:45.749867+0000","last_clean":"2023-10-08T05:53:45.749867+0000","last_became_active":"2023-10-08T05:53:43.710593+0000","last_became_peered":"2023-10-08T05:53:43.710593+0000","last_unstale":"2023-10-08T05:53:45.749867+0000","last_undegraded":"2023-10-08T05:53:45.749867+0000","last_fullsized":"2023-10-08T05:53:45.749867+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:08.777735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769063+0000","last_change":"2023-10-08T05:53:44.737760+0000","last_active":"2023-10-08T05:53:45.769063+0000","last_peered":"2023-10-08T05:53:45.769063+0000","last_clean":"2023-10-08T05:53:45.769063+0000","last_became_active":"2023-10-08T05:53:44.737449+0000","last_became_peered":"2023-10-08T05:53:44.737449+0000","last_unstale":"2023-10-08T05:53:45.769063+0000","last_undegraded":"2023-10-08T05:53:45.769063+0000","last_fullsized":"2023-10-08T05:53:45.769063+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:06.945445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770544+0000","last_change":"2023-10-08T05:53:43.709722+0000","last_active":"2023-10-08T05:53:45.770544+0000","last_peered":"2023-10-08T05:53:45.770544+0000","last_clean":"2023-10-08T05:53:45.770544+0000","last_became_active":"2023-10-08T05:53:43.709448+0000","last_became_peered":"2023-10-08T05:53:43.709448+0000","last_unstale":"2023-10-08T05:53:45.770544+0000","last_undegraded":"2023-10-08T05:53:45.770544+0000","last_fullsized":"2023-10-08T05:53:45.770544+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:54:13.277722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.729627+0000","last_change":"2023-10-08T05:53:44.737093+0000","last_active":"2023-10-08T05:53:45.729627+0000","last_peered":"2023-10-08T05:53:45.729627+0000","last_clean":"2023-10-08T05:53:45.729627+0000","last_became_active":"2023-10-08T05:53:44.736820+0000","last_became_peered":"2023-10-08T05:53:44.736820+0000","last_unstale":"2023-10-08T05:53:45.729627+0000","last_undegraded":"2023-10-08T05:53:45.729627+0000","last_fullsized":"2023-10-08T05:53:45.729627+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:39:04.556070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770697+0000","last_change":"2023-10-08T05:53:43.713775+0000","last_active":"2023-10-08T05:53:45.770697+0000","last_peered":"2023-10-08T05:53:45.770697+0000","last_clean":"2023-10-08T05:53:45.770697+0000","last_became_active":"2023-10-08T05:53:43.713507+0000","last_became_peered":"2023-10-08T05:53:43.713507+0000","last_unstale":"2023-10-08T05:53:45.770697+0000","last_undegraded":"2023-10-08T05:53:45.770697+0000","last_fullsized":"2023-10-08T05:53:45.770697+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:23.699939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.730423+0000","last_change":"2023-10-08T05:53:44.727245+0000","last_active":"2023-10-08T05:53:45.730423+0000","last_peered":"2023-10-08T05:53:45.730423+0000","last_clean":"2023-10-08T05:53:45.730423+0000","last_became_active":"2023-10-08T05:53:44.726668+0000","last_became_peered":"2023-10-08T05:53:44.726668+0000","last_unstale":"2023-10-08T05:53:45.730423+0000","last_undegraded":"2023-10-08T05:53:45.730423+0000","last_fullsized":"2023-10-08T05:53:45.730423+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:53:40.273469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"117'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.768598+0000","last_change":"2023-10-08T05:53:43.703262+0000","last_active":"2023-10-08T05:53:45.768598+0000","last_peered":"2023-10-08T05:53:45.768598+0000","last_clean":"2023-10-08T05:53:45.768598+0000","last_became_active":"2023-10-08T05:53:43.702921+0000","last_became_peered":"2023-10-08T05:53:43.702921+0000","last_unstale":"2023-10-08T05:53:45.768598+0000","last_undegraded":"2023-10-08T05:53:45.768598+0000","last_fullsized":"2023-10-08T05:53:45.768598+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:05.935497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769961+0000","last_change":"2023-10-08T05:53:44.749980+0000","last_active":"2023-10-08T05:53:45.769961+0000","last_peered":"2023-10-08T05:53:45.769961+0000","last_clean":"2023-10-08T05:53:45.769961+0000","last_became_active":"2023-10-08T05:53:44.749709+0000","last_became_peered":"2023-10-08T05:53:44.749709+0000","last_unstale":"2023-10-08T05:53:45.769961+0000","last_undegraded":"2023-10-08T05:53:45.769961+0000","last_fullsized":"2023-10-08T05:53:45.769961+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:50:16.945508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735789+0000","last_change":"2023-10-08T05:53:43.708305+0000","last_active":"2023-10-08T05:53:45.735789+0000","last_peered":"2023-10-08T05:53:45.735789+0000","last_clean":"2023-10-08T05:53:45.735789+0000","last_became_active":"2023-10-08T05:53:43.708002+0000","last_became_peered":"2023-10-08T05:53:43.708002+0000","last_unstale":"2023-10-08T05:53:45.735789+0000","last_undegraded":"2023-10-08T05:53:45.735789+0000","last_fullsized":"2023-10-08T05:53:45.735789+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:40.473980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735488+0000","last_change":"2023-10-08T05:53:44.729322+0000","last_active":"2023-10-08T05:53:45.735488+0000","last_peered":"2023-10-08T05:53:45.735488+0000","last_clean":"2023-10-08T05:53:45.735488+0000","last_became_active":"2023-10-08T05:53:44.728678+0000","last_became_peered":"2023-10-08T05:53:44.728678+0000","last_unstale":"2023-10-08T05:53:45.735488+0000","last_undegraded":"2023-10-08T05:53:45.735488+0000","last_fullsized":"2023-10-08T05:53:45.735488+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:47.049141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.750145+0000","last_change":"2023-10-08T05:53:43.707105+0000","last_active":"2023-10-08T05:53:45.750145+0000","last_peered":"2023-10-08T05:53:45.750145+0000","last_clean":"2023-10-08T05:53:45.750145+0000","last_became_active":"2023-10-08T05:53:43.706767+0000","last_became_peered":"2023-10-08T05:53:43.706767+0000","last_unstale":"2023-10-08T05:53:45.750145+0000","last_undegraded":"2023-10-08T05:53:45.750145+0000","last_fullsized":"2023-10-08T05:53:45.750145+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:16:12.967051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770048+0000","last_change":"2023-10-08T05:53:44.717492+0000","last_active":"2023-10-08T05:53:45.770048+0000","last_peered":"2023-10-08T05:53:45.770048+0000","last_clean":"2023-10-08T05:53:45.770048+0000","last_became_active":"2023-10-08T05:53:44.716683+0000","last_became_peered":"2023-10-08T05:53:44.716683+0000","last_unstale":"2023-10-08T05:53:45.770048+0000","last_undegraded":"2023-10-08T05:53:45.770048+0000","last_fullsized":"2023-10-08T05:53:45.770048+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:37:27.581656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"117'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770463+0000","last_change":"2023-10-08T05:53:43.709269+0000","last_active":"2023-10-08T05:53:45.770463+0000","last_peered":"2023-10-08T05:53:45.770463+0000","last_clean":"2023-10-08T05:53:45.770463+0000","last_became_active":"2023-10-08T05:53:43.708402+0000","last_became_peered":"2023-10-08T05:53:43.708402+0000","last_unstale":"2023-10-08T05:53:45.770463+0000","last_undegraded":"2023-10-08T05:53:45.770463+0000","last_fullsized":"2023-10-08T05:53:45.770463+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:51:09.614335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770046+0000","last_change":"2023-10-08T05:53:44.734191+0000","last_active":"2023-10-08T05:53:45.770046+0000","last_peered":"2023-10-08T05:53:45.770046+0000","last_clean":"2023-10-08T05:53:45.770046+0000","last_became_active":"2023-10-08T05:53:44.733901+0000","last_became_peered":"2023-10-08T05:53:44.733901+0000","last_unstale":"2023-10-08T05:53:45.770046+0000","last_undegraded":"2023-10-08T05:53:45.770046+0000","last_fullsized":"2023-10-08T05:53:45.770046+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:22.850435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736190+0000","last_change":"2023-10-08T05:53:43.691491+0000","last_active":"2023-10-08T05:53:45.736190+0000","last_peered":"2023-10-08T05:53:45.736190+0000","last_clean":"2023-10-08T05:53:45.736190+0000","last_became_active":"2023-10-08T05:53:43.690758+0000","last_became_peered":"2023-10-08T05:53:43.690758+0000","last_unstale":"2023-10-08T05:53:45.736190+0000","last_undegraded":"2023-10-08T05:53:45.736190+0000","last_fullsized":"2023-10-08T05:53:45.736190+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:58:52.618965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736121+0000","last_change":"2023-10-08T05:53:44.734530+0000","last_active":"2023-10-08T05:53:45.736121+0000","last_peered":"2023-10-08T05:53:45.736121+0000","last_clean":"2023-10-08T05:53:45.736121+0000","last_became_active":"2023-10-08T05:53:44.734222+0000","last_became_peered":"2023-10-08T05:53:44.734222+0000","last_unstale":"2023-10-08T05:53:45.736121+0000","last_undegraded":"2023-10-08T05:53:45.736121+0000","last_fullsized":"2023-10-08T05:53:45.736121+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:25.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.729737+0000","last_change":"2023-10-08T05:53:43.707038+0000","last_active":"2023-10-08T05:53:45.729737+0000","last_peered":"2023-10-08T05:53:45.729737+0000","last_clean":"2023-10-08T05:53:45.729737+0000","last_became_active":"2023-10-08T05:53:43.706743+0000","last_became_peered":"2023-10-08T05:53:43.706743+0000","last_unstale":"2023-10-08T05:53:45.729737+0000","last_undegraded":"2023-10-08T05:53:45.729737+0000","last_fullsized":"2023-10-08T05:53:45.729737+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:45.009747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736397+0000","last_change":"2023-10-08T05:53:44.748584+0000","last_active":"2023-10-08T05:53:45.736397+0000","last_peered":"2023-10-08T05:53:45.736397+0000","last_clean":"2023-10-08T05:53:45.736397+0000","last_became_active":"2023-10-08T05:53:44.748276+0000","last_became_peered":"2023-10-08T05:53:44.748276+0000","last_unstale":"2023-10-08T05:53:45.736397+0000","last_undegraded":"2023-10-08T05:53:45.736397+0000","last_fullsized":"2023-10-08T05:53:45.736397+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:07:40.057790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736163+0000","last_change":"2023-10-08T05:53:43.696874+0000","last_active":"2023-10-08T05:53:45.736163+0000","last_peered":"2023-10-08T05:53:45.736163+0000","last_clean":"2023-10-08T05:53:45.736163+0000","last_became_active":"2023-10-08T05:53:43.696257+0000","last_became_peered":"2023-10-08T05:53:43.696257+0000","last_unstale":"2023-10-08T05:53:45.736163+0000","last_undegraded":"2023-10-08T05:53:45.736163+0000","last_fullsized":"2023-10-08T05:53:45.736163+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:16.037633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770876+0000","last_change":"2023-10-08T05:53:44.730174+0000","last_active":"2023-10-08T05:53:45.770876+0000","last_peered":"2023-10-08T05:53:45.770876+0000","last_clean":"2023-10-08T05:53:45.770876+0000","last_became_active":"2023-10-08T05:53:44.729850+0000","last_became_peered":"2023-10-08T05:53:44.729850+0000","last_unstale":"2023-10-08T05:53:45.770876+0000","last_undegraded":"2023-10-08T05:53:45.770876+0000","last_fullsized":"2023-10-08T05:53:45.770876+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:45.340517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.729895+0000","last_change":"2023-10-08T05:53:43.709948+0000","last_active":"2023-10-08T05:53:45.729895+0000","last_peered":"2023-10-08T05:53:45.729895+0000","last_clean":"2023-10-08T05:53:45.729895+0000","last_became_active":"2023-10-08T05:53:43.709689+0000","last_became_peered":"2023-10-08T05:53:43.709689+0000","last_unstale":"2023-10-08T05:53:45.729895+0000","last_undegraded":"2023-10-08T05:53:45.729895+0000","last_fullsized":"2023-10-08T05:53:45.729895+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:45:22.865416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735255+0000","last_change":"2023-10-08T05:53:44.727792+0000","last_active":"2023-10-08T05:53:45.735255+0000","last_peered":"2023-10-08T05:53:45.735255+0000","last_clean":"2023-10-08T05:53:45.735255+0000","last_became_active":"2023-10-08T05:53:44.727518+0000","last_became_peered":"2023-10-08T05:53:44.727518+0000","last_unstale":"2023-10-08T05:53:45.735255+0000","last_undegraded":"2023-10-08T05:53:45.735255+0000","last_fullsized":"2023-10-08T05:53:45.735255+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:35.256324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"117'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.750128+0000","last_change":"2023-10-08T05:53:43.710347+0000","last_active":"2023-10-08T05:53:45.750128+0000","last_peered":"2023-10-08T05:53:45.750128+0000","last_clean":"2023-10-08T05:53:45.750128+0000","last_became_active":"2023-10-08T05:53:43.710043+0000","last_became_peered":"2023-10-08T05:53:43.710043+0000","last_unstale":"2023-10-08T05:53:45.750128+0000","last_undegraded":"2023-10-08T05:53:45.750128+0000","last_fullsized":"2023-10-08T05:53:45.750128+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:44.330569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772355+0000","last_change":"2023-10-08T05:53:44.749972+0000","last_active":"2023-10-08T05:53:45.772355+0000","last_peered":"2023-10-08T05:53:45.772355+0000","last_clean":"2023-10-08T05:53:45.772355+0000","last_became_active":"2023-10-08T05:53:44.749689+0000","last_became_peered":"2023-10-08T05:53:44.749689+0000","last_unstale":"2023-10-08T05:53:45.772355+0000","last_undegraded":"2023-10-08T05:53:45.772355+0000","last_fullsized":"2023-10-08T05:53:45.772355+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:01:02.983379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770229+0000","last_change":"2023-10-08T05:53:43.708320+0000","last_active":"2023-10-08T05:53:45.770229+0000","last_peered":"2023-10-08T05:53:45.770229+0000","last_clean":"2023-10-08T05:53:45.770229+0000","last_became_active":"2023-10-08T05:53:43.707945+0000","last_became_peered":"2023-10-08T05:53:43.707945+0000","last_unstale":"2023-10-08T05:53:45.770229+0000","last_undegraded":"2023-10-08T05:53:45.770229+0000","last_fullsized":"2023-10-08T05:53:45.770229+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:23.699939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.772104+0000","last_change":"2023-10-08T05:53:44.728019+0000","last_active":"2023-10-08T05:53:45.772104+0000","last_peered":"2023-10-08T05:53:45.772104+0000","last_clean":"2023-10-08T05:53:45.772104+0000","last_became_active":"2023-10-08T05:53:44.727589+0000","last_became_peered":"2023-10-08T05:53:44.727589+0000","last_unstale":"2023-10-08T05:53:45.772104+0000","last_undegraded":"2023-10-08T05:53:45.772104+0000","last_fullsized":"2023-10-08T05:53:45.772104+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:27.263750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770631+0000","last_change":"2023-10-08T05:53:43.709377+0000","last_active":"2023-10-08T05:53:45.770631+0000","last_peered":"2023-10-08T05:53:45.770631+0000","last_clean":"2023-10-08T05:53:45.770631+0000","last_became_active":"2023-10-08T05:53:43.708973+0000","last_became_peered":"2023-10-08T05:53:43.708973+0000","last_unstale":"2023-10-08T05:53:45.770631+0000","last_undegraded":"2023-10-08T05:53:45.770631+0000","last_fullsized":"2023-10-08T05:53:45.770631+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:48.072808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736875+0000","last_change":"2023-10-08T05:53:44.750141+0000","last_active":"2023-10-08T05:53:45.736875+0000","last_peered":"2023-10-08T05:53:45.736875+0000","last_clean":"2023-10-08T05:53:45.736875+0000","last_became_active":"2023-10-08T05:53:44.749840+0000","last_became_peered":"2023-10-08T05:53:44.749840+0000","last_unstale":"2023-10-08T05:53:45.736875+0000","last_undegraded":"2023-10-08T05:53:45.736875+0000","last_fullsized":"2023-10-08T05:53:45.736875+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:06.420177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.735416+0000","last_change":"2023-10-08T05:53:43.707137+0000","last_active":"2023-10-08T05:53:45.735416+0000","last_peered":"2023-10-08T05:53:45.735416+0000","last_clean":"2023-10-08T05:53:45.735416+0000","last_became_active":"2023-10-08T05:53:43.706836+0000","last_became_peered":"2023-10-08T05:53:43.706836+0000","last_unstale":"2023-10-08T05:53:45.735416+0000","last_undegraded":"2023-10-08T05:53:45.735416+0000","last_fullsized":"2023-10-08T05:53:45.735416+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:20:55.362172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.730423+0000","last_change":"2023-10-08T05:53:44.729709+0000","last_active":"2023-10-08T05:53:45.730423+0000","last_peered":"2023-10-08T05:53:45.730423+0000","last_clean":"2023-10-08T05:53:45.730423+0000","last_became_active":"2023-10-08T05:53:44.729293+0000","last_became_peered":"2023-10-08T05:53:44.729293+0000","last_unstale":"2023-10-08T05:53:45.730423+0000","last_undegraded":"2023-10-08T05:53:45.730423+0000","last_fullsized":"2023-10-08T05:53:45.730423+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:34.574107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.770960+0000","last_change":"2023-10-08T05:53:43.714848+0000","last_active":"2023-10-08T05:53:45.770960+0000","last_peered":"2023-10-08T05:53:45.770960+0000","last_clean":"2023-10-08T05:53:45.770960+0000","last_became_active":"2023-10-08T05:53:43.714532+0000","last_became_peered":"2023-10-08T05:53:43.714532+0000","last_unstale":"2023-10-08T05:53:45.770960+0000","last_undegraded":"2023-10-08T05:53:45.770960+0000","last_fullsized":"2023-10-08T05:53:45.770960+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:23:58.428910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.773398+0000","last_change":"2023-10-08T05:53:44.742773+0000","last_active":"2023-10-08T05:53:45.773398+0000","last_peered":"2023-10-08T05:53:45.773398+0000","last_clean":"2023-10-08T05:53:45.773398+0000","last_became_active":"2023-10-08T05:53:44.742495+0000","last_became_peered":"2023-10-08T05:53:44.742495+0000","last_unstale":"2023-10-08T05:53:45.773398+0000","last_undegraded":"2023-10-08T05:53:45.773398+0000","last_fullsized":"2023-10-08T05:53:45.773398+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:38.558149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.736340+0000","last_change":"2023-10-08T05:53:43.691457+0000","last_active":"2023-10-08T05:53:45.736340+0000","last_peered":"2023-10-08T05:53:45.736340+0000","last_clean":"2023-10-08T05:53:45.736340+0000","last_became_active":"2023-10-08T05:53:43.690742+0000","last_became_peered":"2023-10-08T05:53:43.690742+0000","last_unstale":"2023-10-08T05:53:45.736340+0000","last_undegraded":"2023-10-08T05:53:45.736340+0000","last_fullsized":"2023-10-08T05:53:45.736340+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:16:12.967051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.734829+0000","last_change":"2023-10-08T05:53:44.741482+0000","last_active":"2023-10-08T05:53:45.734829+0000","last_peered":"2023-10-08T05:53:45.734829+0000","last_clean":"2023-10-08T05:53:45.734829+0000","last_became_active":"2023-10-08T05:53:44.741197+0000","last_became_peered":"2023-10-08T05:53:44.741197+0000","last_unstale":"2023-10-08T05:53:45.734829+0000","last_undegraded":"2023-10-08T05:53:45.734829+0000","last_fullsized":"2023-10-08T05:53:45.734829+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:59:50.641322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.734980+0000","last_change":"2023-10-08T05:53:43.706023+0000","last_active":"2023-10-08T05:53:45.734980+0000","last_peered":"2023-10-08T05:53:45.734980+0000","last_clean":"2023-10-08T05:53:45.734980+0000","last_became_active":"2023-10-08T05:53:43.705742+0000","last_became_peered":"2023-10-08T05:53:43.705742+0000","last_unstale":"2023-10-08T05:53:45.734980+0000","last_undegraded":"2023-10-08T05:53:45.734980+0000","last_fullsized":"2023-10-08T05:53:45.734980+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:53.297605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.771060+0000","last_change":"2023-10-08T05:53:44.735709+0000","last_active":"2023-10-08T05:53:45.771060+0000","last_peered":"2023-10-08T05:53:45.771060+0000","last_clean":"2023-10-08T05:53:45.771060+0000","last_became_active":"2023-10-08T05:53:44.735392+0000","last_became_peered":"2023-10-08T05:53:44.735392+0000","last_unstale":"2023-10-08T05:53:45.771060+0000","last_undegraded":"2023-10-08T05:53:45.771060+0000","last_fullsized":"2023-10-08T05:53:45.771060+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:18.150326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.749586+0000","last_change":"2023-10-08T05:53:43.712535+0000","last_active":"2023-10-08T05:53:45.749586+0000","last_peered":"2023-10-08T05:53:45.749586+0000","last_clean":"2023-10-08T05:53:45.749586+0000","last_became_active":"2023-10-08T05:53:43.712118+0000","last_became_peered":"2023-10-08T05:53:43.712118+0000","last_unstale":"2023-10-08T05:53:45.749586+0000","last_undegraded":"2023-10-08T05:53:45.749586+0000","last_fullsized":"2023-10-08T05:53:45.749586+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:41.290271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.750305+0000","last_change":"2023-10-08T05:53:44.739587+0000","last_active":"2023-10-08T05:53:45.750305+0000","last_peered":"2023-10-08T05:53:45.750305+0000","last_clean":"2023-10-08T05:53:45.750305+0000","last_became_active":"2023-10-08T05:53:44.739316+0000","last_became_peered":"2023-10-08T05:53:44.739316+0000","last_unstale":"2023-10-08T05:53:45.750305+0000","last_undegraded":"2023-10-08T05:53:45.750305+0000","last_fullsized":"2023-10-08T05:53:45.750305+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:43.665578+0000","last_clean_scrub_stamp":"2023-10-08T05:53:43.665578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:23:06.420177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-08T05:53:45.769038+0000","last_change":"2023-10-08T05:53:43.703413+0000","last_active":"2023-10-08T05:53:45.769038+0000","last_peered":"2023-10-08T05:53:45.769038+0000","last_clean":"2023-10-08T05:53:45.769038+0000","last_became_active":"2023-10-08T05:53:43.703129+0000","last_became_peered":"2023-10-08T05:53:43.703129+0000","last_unstale":"2023-10-08T05:53:45.769038+0000","last_undegraded":"2023-10-08T05:53:45.769038+0000","last_fullsized":"2023-10-08T05:53:45.769038+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:53:42.655631+0000","last_clean_scrub_stamp":"2023-10-08T05:53:42.655631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:12.115005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"107'2641","reported_seq":2994,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-10-08T05:53:44.727313+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:53:44.727313+0000","last_peered":"2023-10-08T05:53:44.727313+0000","last_clean":"2023-10-08T05:53:44.727313+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:53:44.727313+0000","last_undegraded":"2023-10-08T05:53:44.727313+0000","last_fullsized":"2023-10-08T05:53:44.727313+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2641,"log_dups_size":0,"ondisk_log_size":2641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":765,"num_read_kb":1528,"num_write":289,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":765,"num_read_kb":1528,"num_write":289,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2641,"ondisk_log_size":2641,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575060,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51244,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320596,"statfs":{"total":96636764160,"available":96584290304,"internally_reserved":0,"allocated":2056192,"data_stored":1326202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.1080000000000001,"15min":1.1879999999999999},"min":{"1min":0.69999999999999996,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2969999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.1439999999999999,"15min":1.21},"min":{"1min":0.63700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.718,"5min":1.7210000000000001,"15min":1.726},"last":1.4119999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.145,"15min":1.2310000000000001},"min":{"1min":0.68400000000000005,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5,"5min":1.619,"15min":1.8360000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.252,"15min":1.2689999999999999},"min":{"1min":0.83799999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.591,"5min":1.7689999999999999,"15min":1.9379999999999999},"last":1.298}]},{"osd":2,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1819999999999999,"15min":1.2330000000000001},"min":{"1min":0.71599999999999997,"5min":0.626,"15min":0.626},"max":{"1min":1.502,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.21,"5min":1.294,"15min":1.3180000000000001},"min":{"1min":0.88500000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.714,"5min":1.714,"15min":1.7769999999999999},"last":1.492}]},{"osd":3,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2569999999999999,"15min":1.2869999999999999},"min":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.611,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.3,"15min":1.353},"min":{"1min":0.83199999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6020000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.1220000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3160000000000001,"15min":1.288},"min":{"1min":1.0189999999999999,"5min":0.504,"15min":0.504},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.153},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.355,"15min":1.333},"min":{"1min":0.92600000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.766,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.0740000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3799999999999999,"15min":1.3939999999999999},"min":{"1min":0.94899999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.7569999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.387},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4039999999999999,"15min":1.421},"min":{"1min":0.95999999999999996,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.3600000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:53:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.532,"15min":1.522},"min":{"1min":1.1160000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.476,"15min":1.46},"min":{"1min":0.84299999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.8859999999999999,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.3240000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575059,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":1474560,"data_stored":737994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.246,"15min":1.28},"min":{"1min":0.70899999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.55,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.262,"15min":1.3100000000000001},"min":{"1min":0.82199999999999995,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.6679999999999999,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.5740000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1319999999999999,"15min":1.169},"min":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4370000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":1.218},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.159,"15min":1.1930000000000001},"min":{"1min":0.69199999999999995,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6299999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.7030000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1499999999999999,"15min":1.1890000000000001},"min":{"1min":0.69099999999999995,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7470000000000001,"5min":1.774,"15min":1.796},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.169,"5min":1.1970000000000001,"15min":1.26},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.58699999999999997},"max":{"1min":1.581,"5min":1.752,"15min":1.8620000000000001},"last":0.94799999999999995}]},{"osd":3,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.3100000000000001,"15min":1.371},"min":{"1min":0.80300000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.571,"5min":2.137,"15min":2.137},"last":1.627},{"interface":"front","average":{"1min":1.359,"5min":1.399,"15min":1.45},"min":{"1min":0.86799999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.637,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.3380000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3320000000000001,"15min":1.3100000000000001},"min":{"1min":0.85999999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.724,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.399,"15min":1.3839999999999999},"min":{"1min":1.159,"5min":0.78400000000000003,"15min":0.64300000000000002},"max":{"1min":1.9159999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.742}]},{"osd":5,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.319,"15min":1.3320000000000001},"min":{"1min":0.83599999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.502,"15min":1.492},"min":{"1min":1.034,"5min":0.747,"15min":0.747},"max":{"1min":2.0350000000000001,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.913}]},{"osd":7,"last update":"Sun Oct 8 05:53:32 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5309999999999999,"15min":1.5149999999999999},"min":{"1min":1.1499999999999999,"5min":0.98299999999999998,"15min":0.77200000000000002},"max":{"1min":2.6070000000000002,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.091},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4950000000000001,"15min":1.474},"min":{"1min":1.0860000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.6659999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575059,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1454080,"data_stored":735778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.0589999999999999,"15min":1.1240000000000001},"min":{"1min":0.70399999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.6299999999999999,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.173,"15min":1.2050000000000001},"min":{"1min":0.47899999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.758},"last":1.0760000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.179,"15min":1.1990000000000001},"min":{"1min":0.73699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.756,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.258,"15min":1.2649999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.70199999999999996},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5389999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1859999999999999,"15min":1.2290000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.5629999999999999,"5min":1.8,"15min":1.8},"last":1.341},{"interface":"front","average":{"1min":1.23,"5min":1.258,"15min":1.2829999999999999},"min":{"1min":0.71399999999999997,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.3199999999999998,"5min":3.282,"15min":3.282},"last":1.585}]},{"osd":3,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3169999999999999,"15min":1.3420000000000001},"min":{"1min":0.88700000000000001,"5min":0.77700000000000002,"15min":0.63500000000000001},"max":{"1min":2.5710000000000002,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.423},{"interface":"front","average":{"1min":1.337,"5min":1.3759999999999999,"15min":1.3899999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.73,"5min":1.919,"15min":2.2280000000000002},"last":1.73}]},{"osd":4,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.361,"15min":1.2949999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.2919999999999998,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.496,"15min":1.4239999999999999},"min":{"1min":0.94599999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":2.927,"5min":3.456,"15min":3.456},"last":1.214}]},{"osd":6,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.4279999999999999,"15min":1.4039999999999999},"min":{"1min":0.95299999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.105,"5min":2.105,"15min":2.359},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.5049999999999999,"15min":1.464},"min":{"1min":0.88100000000000001,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.4390000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.681}]},{"osd":7,"last update":"Sun Oct 8 05:53:46 2023","interfaces":[{"interface":"back","average":{"1min":1.661,"5min":1.5940000000000001,"15min":1.544},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.018},"max":{"1min":3.0270000000000001,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":1.615},{"interface":"front","average":{"1min":1.6279999999999999,"5min":1.5820000000000001,"15min":1.5529999999999999},"min":{"1min":1.0129999999999999,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":2.4460000000000002},"last":1.631}]}]},{"osd":4,"up_from":13,"seq":55834575059,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":735938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1830000000000001,"15min":1.252},"min":{"1min":0.70399999999999996,"5min":0.498,"15min":0.498},"max":{"1min":1.5720000000000001,"5min":1.8009999999999999,"15min":1.8149999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3759999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.4059999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.2569999999999999,"15min":1.28},"min":{"1min":0.91800000000000004,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.5499999999999998,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.212,"15min":1.216},"min":{"1min":0.70299999999999996,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.1499999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":1.2629999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.218,"15min":1.2270000000000001},"min":{"1min":0.81399999999999995,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6160000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.222},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3380000000000001,"15min":1.355},"min":{"1min":0.75700000000000001,"5min":0.622,"15min":0.622},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.5609999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.2829999999999999,"15min":1.276},"min":{"1min":0.84799999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.3070000000000004,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3080000000000001,"15min":1.335},"min":{"1min":0.83599999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.643,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.5289999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.393,"15min":1.337},"min":{"1min":0.751,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.5},{"interface":"front","average":{"1min":1.599,"5min":1.431,"15min":1.373},"min":{"1min":0.78600000000000003,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":5.4189999999999996,"5min":5.4189999999999996,"15min":5.4189999999999996},"last":1.169}]},{"osd":6,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.4399999999999999,"15min":1.3640000000000001},"min":{"1min":0.72199999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":4.9059999999999997,"5min":4.9059999999999997,"15min":4.9059999999999997},"last":1.306},{"interface":"front","average":{"1min":1.7709999999999999,"5min":1.5660000000000001,"15min":1.514},"min":{"1min":0.67300000000000004,"5min":0.433,"15min":0.433},"max":{"1min":6.0970000000000004,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":1.3320000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.7909999999999999,"5min":1.5669999999999999,"15min":1.47},"min":{"1min":0.94299999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":6.6689999999999996,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.675,"5min":1.532,"15min":1.5209999999999999},"min":{"1min":0.995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":4.1849999999999996,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.361}]}]},{"osd":3,"up_from":13,"seq":55834575061,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36764,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335076,"statfs":{"total":96636764160,"available":96599117824,"internally_reserved":0,"allocated":1454080,"data_stored":735778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.89500000000000002,"15min":0.91900000000000004},"min":{"1min":0.41399999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1819999999999999,"5min":1.5269999999999999,"15min":1.72},"last":1.123},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.018,"15min":1.091},"min":{"1min":0.53200000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.456,"5min":1.5,"15min":1.6140000000000001},"last":0.74299999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.90200000000000002,"15min":0.96399999999999997},"min":{"1min":0.39200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.998,"5min":1.149,"15min":1.208},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.4610000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.3360000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":1.006,"15min":1.0649999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.7090000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.196},{"interface":"front","average":{"1min":1.016,"5min":1.1200000000000001,"15min":1.1439999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3600000000000001,"5min":1.6819999999999999,"15min":1.788},"last":0.92400000000000004}]},{"osd":4,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1599999999999999,"15min":1.206},"min":{"1min":0.76300000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.494,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.3440000000000001,"15min":1.3839999999999999},"min":{"1min":1.0720000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9590000000000001,"5min":2.589,"15min":2.589},"last":1.3939999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.286,"15min":1.3029999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73699999999999999},"max":{"1min":1.855,"5min":1.883,"15min":1.883},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.3280000000000001,"15min":1.355},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.798,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.4370000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.3100000000000001,"15min":1.3200000000000001},"min":{"1min":0.65700000000000003,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7390000000000001,"5min":1.766,"15min":1.776},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4379999999999999,"15min":1.4470000000000001},"min":{"1min":0.96699999999999997,"5min":0.86799999999999999,"15min":0.86699999999999999},"max":{"1min":2.016,"5min":2.016,"15min":2.0489999999999999},"last":1.5049999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:53:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.462,"15min":1.4910000000000001},"min":{"1min":0.89500000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.913,"5min":2.085,"15min":2.1480000000000001},"last":1.536},{"interface":"front","average":{"1min":1.343,"5min":1.363,"15min":1.391},"min":{"1min":0.80800000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.248}]}]},{"osd":2,"up_from":13,"seq":55834575059,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58976,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312864,"statfs":{"total":96636764160,"available":96576372736,"internally_reserved":0,"allocated":1454080,"data_stored":735778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77800000000000002,"15min":0.874},"min":{"1min":0.32200000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.97099999999999997,"15min":1.03},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.3560000000000001,"5min":1.5880000000000001,"15min":1.9470000000000001},"last":1.0349999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.85899999999999999,"15min":0.94499999999999995},"min":{"1min":0.48199999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1839999999999999,"5min":1.7609999999999999,"15min":2.1070000000000002},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.96699999999999997,"15min":1.0900000000000001},"min":{"1min":0.51500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.79,"5min":1.79,"15min":2.1469999999999998},"last":1.349}]},{"osd":3,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.024,"15min":1.1200000000000001},"min":{"1min":0.53000000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.631,"5min":1.631,"15min":1.8120000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.149,"15min":1.216},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.7350000000000001,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":1.7410000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.3169999999999999,"15min":1.3759999999999999},"min":{"1min":0.66000000000000003,"5min":0.622,"15min":0.622},"max":{"1min":1.8280000000000001,"5min":4.593,"15min":4.593},"last":1.246},{"interface":"front","average":{"1min":1.149,"5min":1.2450000000000001,"15min":1.28},"min":{"1min":0.65900000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.6040000000000001,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.296}]},{"osd":5,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.3100000000000001,"15min":1.3680000000000001},"min":{"1min":0.68799999999999994,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.621,"5min":6.7140000000000004,"15min":6.7140000000000004},"last":1.526},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.353,"15min":1.4430000000000001},"min":{"1min":0.78800000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8620000000000001,"5min":1.9370000000000001,"15min":2.4119999999999999},"last":1.5840000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2869999999999999,"15min":1.343},"min":{"1min":0.64600000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.581,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.208,"5min":1.3300000000000001,"15min":1.4390000000000001},"min":{"1min":0.77500000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.619,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.639}]},{"osd":7,"last update":"Sun Oct 8 05:53:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.387,"15min":1.4279999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7450000000000001,"5min":6.609,"15min":6.609},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3700000000000001,"15min":1.488},"min":{"1min":0.98099999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.5529999999999999,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.706}]}]},{"osd":1,"up_from":13,"seq":55834575059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36772,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335068,"statfs":{"total":96636764160,"available":96599109632,"internally_reserved":0,"allocated":1458176,"data_stored":735848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.86699999999999999,"15min":0.95999999999999996},"min":{"1min":0.35799999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.2410000000000001,"5min":1.603,"15min":1.796},"last":1.026},{"interface":"front","average":{"1min":0.82899999999999996,"5min":1.0049999999999999,"15min":1.125},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.409,"5min":1.5600000000000001,"15min":1.8089999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.871,"15min":0.95699999999999996},"min":{"1min":0.45100000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98799999999999999,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.498},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.96999999999999997,"15min":1.079},"min":{"1min":0.45300000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4370000000000001,"5min":1.593,"15min":1.8049999999999999},"last":0.73799999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":1.056,"15min":1.1719999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3879999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.999,"15min":1.0649999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.5329999999999999,"5min":1.7669999999999999,"15min":1.788},"last":0.81999999999999995}]},{"osd":4,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1819999999999999,"15min":1.2949999999999999},"min":{"1min":0.67700000000000005,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.966},"last":1.234},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1779999999999999,"15min":1.2210000000000001},"min":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.6719999999999999,"5min":1.913,"15min":1.913},"last":0.95599999999999996}]},{"osd":5,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.248,"15min":1.304},"min":{"1min":0.73199999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.7909999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.113},{"interface":"front","average":{"1min":1.248,"5min":1.355,"15min":1.4470000000000001},"min":{"1min":0.86499999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.7090000000000001,"5min":1.905,"15min":2.077},"last":1.2130000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3380000000000001,"15min":1.421},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.843,"5min":1.972,"15min":1.9970000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3640000000000001,"15min":1.4219999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8580000000000001,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.056}]},{"osd":7,"last update":"Sun Oct 8 05:53:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.319,"15min":1.4530000000000001},"min":{"1min":0.76600000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.6259999999999999,"5min":1.893,"15min":2.1869999999999998},"last":1.093},{"interface":"front","average":{"1min":1.208,"5min":1.3680000000000001,"15min":1.4470000000000001},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":2.306},"last":1.077}]}]},{"osd":0,"up_from":13,"seq":55834575060,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2072576,"data_stored":1328464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.83799999999999997,"15min":0.91000000000000003},"min":{"1min":0.30199999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.552,"5min":1.77,"15min":1.77},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.97099999999999997,"15min":1.0369999999999999},"min":{"1min":0.40899999999999997,"5min":0.371,"15min":0.371},"max":{"1min":1.3859999999999999,"5min":1.6719999999999999,"15min":1.675},"last":0.70899999999999996}]},{"osd":2,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.997,"15min":1.0840000000000001},"min":{"1min":0.42599999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.361,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.89400000000000002,"15min":0.99099999999999999},"min":{"1min":0.47399999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.395,"5min":1.722,"15min":1.78},"last":1.4770000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.087,"15min":1.1970000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6599999999999999,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.036,"15min":1.0880000000000001},"min":{"1min":0.46500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.496,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.93100000000000005}]},{"osd":4,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.206,"15min":1.268},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.8129999999999999,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2689999999999999,"15min":1.343},"min":{"1min":0.70599999999999996,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.6879999999999999,"5min":1.8939999999999999,"15min":2.2200000000000002},"last":1.05}]},{"osd":5,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.282,"15min":1.337},"min":{"1min":0.67700000000000005,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":2.0489999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.3049999999999999,"15min":1.377},"min":{"1min":0.71199999999999997,"5min":0.63,"15min":0.63},"max":{"1min":1.776,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.236}]},{"osd":6,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.347,"15min":1.413},"min":{"1min":0.78000000000000003,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.875,"5min":2.032,"15min":2.1909999999999998},"last":1.278},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.329,"15min":1.399},"min":{"1min":0.85599999999999998,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0430000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.4299999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:53:39 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.371,"15min":1.4370000000000001},"min":{"1min":0.74399999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4019999999999999,"15min":1.4730000000000001},"min":{"1min":0.78800000000000003,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.0569999999999999,"5min":2.0569999999999999,"15min":2.2759999999999998},"last":1.131}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:53:49.680 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T05:53:49.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T05:53:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.933+0000 7f2171ed7640 1 Processor -- start 2023-10-08T05:53:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.933+0000 7f2171ed7640 1 -- start start 2023-10-08T05:53:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.935+0000 7f2171ed7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0cb500 0x7f216c0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.935+0000 7f2171ed7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f216c070690 0x7f216c070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.935+0000 7f2171ed7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0c8c60 con 0x7f216c06cfa0 2023-10-08T05:53:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.935+0000 7f2171ed7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c0cbe20 con 0x7f216c070690 2023-10-08T05:53:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.935+0000 7f2171ed7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0b1ac0 con 0x7f216c0cb500 2023-10-08T05:53:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.935+0000 7f216bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f216c070690 0x7f216c070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.935+0000 7f216bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f216c070690 0x7f216c070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51574/0 (socket says 172.21.15.70:51574) 2023-10-08T05:53:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.935+0000 7f216bfff640 1 -- 172.21.15.70:0/649108689 learned_addr learned my addr 172.21.15.70:0/649108689 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.935+0000 7f2163fff640 1 --1- 172.21.15.70:0/649108689 >> v1:172.21.15.70:6789/0 conn(0x7f216c06cfa0 0x7f216c15b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34052/0 (socket says 172.21.15.70:34052) 2023-10-08T05:53:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.936+0000 7f216b7fe640 1 --2- 172.21.15.70:0/649108689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0cb500 0x7f216c0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.936+0000 7f216bfff640 1 -- 172.21.15.70:0/649108689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0cb500 msgr2=0x7f216c0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.936+0000 7f216bfff640 1 --2- 172.21.15.70:0/649108689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0cb500 0x7f216c0cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.936+0000 7f216bfff640 1 -- 172.21.15.70:0/649108689 >> v1:172.21.15.70:6789/0 conn(0x7f216c06cfa0 legacy=0x7f216c15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.936+0000 7f216bfff640 1 -- 172.21.15.70:0/649108689 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0b1da0 con 0x7f216c070690 2023-10-08T05:53:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.936+0000 7f216b7fe640 1 --2- 172.21.15.70:0/649108689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0cb500 0x7f216c0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.936+0000 7f216affd640 1 -- 172.21.15.70:0/649108689 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692061710 0 0) 0x7f216c0c8c60 con 0x7f216c06cfa0 2023-10-08T05:53:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.937+0000 7f216bfff640 1 --2- 172.21.15.70:0/649108689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c070690 0x7f216c070af0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2164009e50 tx=0x7f216402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4492225619d13bcb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.937+0000 7f216affd640 1 -- 172.21.15.70:0/649108689 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216402ca70 con 0x7f216c070690 2023-10-08T05:53:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.937+0000 7f216affd640 1 -- 172.21.15.70:0/649108689 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216402cbd0 con 0x7f216c070690 2023-10-08T05:53:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.937+0000 7f216affd640 1 -- 172.21.15.70:0/649108689 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21640343f0 con 0x7f216c070690 2023-10-08T05:53:49.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.938+0000 7f2171ed7640 1 -- 172.21.15.70:0/649108689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c070690 msgr2=0x7f216c070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.938+0000 7f2171ed7640 1 --2- 172.21.15.70:0/649108689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c070690 0x7f216c070af0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2164009e50 tx=0x7f216402bed0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.938+0000 7f2171ed7640 1 -- 172.21.15.70:0/649108689 shutdown_connections 2023-10-08T05:53:49.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.938+0000 7f2171ed7640 1 --2- 172.21.15.70:0/649108689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c070690 0x7f216c070af0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.938+0000 7f2171ed7640 1 --2- 172.21.15.70:0/649108689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c0cb500 0x7f216c0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.938+0000 7f2171ed7640 1 -- 172.21.15.70:0/649108689 >> 172.21.15.70:0/649108689 conn(0x7f216c06a140 msgr2=0x7f216c06a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:49.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.938+0000 7f2171ed7640 1 -- 172.21.15.70:0/649108689 shutdown_connections 2023-10-08T05:53:49.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.938+0000 7f2171ed7640 1 -- 172.21.15.70:0/649108689 wait complete. 2023-10-08T05:53:49.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.940+0000 7f2171ed7640 1 Processor -- start 2023-10-08T05:53:49.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.940+0000 7f2171ed7640 1 -- start start 2023-10-08T05:53:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f2171ed7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c06cfa0 0x7f216c0c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f2171ed7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c070690 0x7f216c0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f2171ed7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c0cb500 0x7f216c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f2171ed7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c146460 con 0x7f216c0cb500 2023-10-08T05:53:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f2171ed7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0c4a50 con 0x7f216c06cfa0 2023-10-08T05:53:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f2171ed7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c148a10 con 0x7f216c070690 2023-10-08T05:53:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f216bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c0cb500 0x7f216c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f216bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c0cb500 0x7f216c0ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51576/0 (socket says 172.21.15.70:51576) 2023-10-08T05:53:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f216bfff640 1 -- 172.21.15.70:0/3393518047 learned_addr learned my addr 172.21.15.70:0/3393518047 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f216bfff640 1 -- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c070690 msgr2=0x7f216c0c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f216bfff640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c070690 0x7f216c0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f216bfff640 1 -- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c06cfa0 msgr2=0x7f216c0c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:49.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f216b7fe640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c06cfa0 0x7f216c0c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f216bfff640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c06cfa0 0x7f216c0c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:49.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.941+0000 7f216bfff640 1 -- 172.21.15.70:0/3393518047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2164009b30 con 0x7f216c0cb500 2023-10-08T05:53:49.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.942+0000 7f216bfff640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c0cb500 0x7f216c0ca480 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f2164036040 tx=0x7f2164004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:49.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.942+0000 7f2168ff9640 1 -- 172.21.15.70:0/3393518047 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164009510 con 0x7f216c0cb500 2023-10-08T05:53:49.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.942+0000 7f2168ff9640 1 -- 172.21.15.70:0/3393518047 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216403e450 con 0x7f216c0cb500 2023-10-08T05:53:49.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.942+0000 7f2168ff9640 1 -- 172.21.15.70:0/3393518047 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216403e800 con 0x7f216c0cb500 2023-10-08T05:53:49.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.942+0000 7f2171ed7640 1 -- 172.21.15.70:0/3393518047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c145b30 con 0x7f216c0cb500 2023-10-08T05:53:49.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.942+0000 7f2171ed7640 1 -- 172.21.15.70:0/3393518047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c1460f0 con 0x7f216c0cb500 2023-10-08T05:53:49.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.943+0000 7f2171ed7640 1 -- 172.21.15.70:0/3393518047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c146610 con 0x7f216c0cb500 2023-10-08T05:53:49.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.944+0000 7f2168ff9640 1 -- 172.21.15.70:0/3393518047 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f216403e960 con 0x7f216c0cb500 2023-10-08T05:53:49.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.946+0000 7f2168ff9640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f214c06af90 0x7f214c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:49.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.946+0000 7f2168ff9640 1 -- 172.21.15.70:0/3393518047 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f21640ac730 con 0x7f216c0cb500 2023-10-08T05:53:49.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.946+0000 7f216b7fe640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f214c06af90 0x7f214c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:49.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.947+0000 7f216b7fe640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f214c06af90 0x7f214c06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f216c1f5e10 tx=0x7f215c008040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:49.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:49.950+0000 7f2168ff9640 1 -- 172.21.15.70:0/3393518047 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f216c146610 con 0x7f216c0cb500 2023-10-08T05:53:50.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.207+0000 7f2171ed7640 1 -- 172.21.15.70:0/3393518047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f216c071cb0 con 0x7f216c0cb500 2023-10-08T05:53:50.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.211+0000 7f2168ff9640 1 -- 172.21.15.70:0/3393518047 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2164077550 con 0x7f216c0cb500 2023-10-08T05:53:50.212 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T05:53:50.212 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T05:53:50.212 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T05:53:50.212 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T05:53:50.212 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T05:53:50.212 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T05:53:50.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 -- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f214c06af90 msgr2=0x7f214c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:50.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f214c06af90 0x7f214c06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f216c1f5e10 tx=0x7f215c008040 comp rx=0 tx=0).stop 2023-10-08T05:53:50.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 -- 172.21.15.70:0/3393518047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c0cb500 msgr2=0x7f216c0ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:50.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c0cb500 0x7f216c0ca480 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f2164036040 tx=0x7f2164004a40 comp rx=0 tx=0).stop 2023-10-08T05:53:50.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 -- 172.21.15.70:0/3393518047 shutdown_connections 2023-10-08T05:53:50.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f214c06af90 0x7f214c06d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f216c0cb500 0x7f216c0ca480 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f216c070690 0x7f216c0c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 --2- 172.21.15.70:0/3393518047 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f216c06cfa0 0x7f216c0c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 -- 172.21.15.70:0/3393518047 >> 172.21.15.70:0/3393518047 conn(0x7f216c06a140 msgr2=0x7f216c0b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:50.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 -- 172.21.15.70:0/3393518047 shutdown_connections 2023-10-08T05:53:50.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.214+0000 7f2171ed7640 1 -- 172.21.15.70:0/3393518047 wait complete. 2023-10-08T05:53:50.233 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:53:50.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.433+0000 7f8fe2ba9640 1 Processor -- start 2023-10-08T05:53:50.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.434+0000 7f8fe2ba9640 1 -- start start 2023-10-08T05:53:50.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.435+0000 7f8fe2ba9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8fdc071190 0x7f8fdc15e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:50.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.435+0000 7f8fe2ba9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fdc0c9850 con 0x7f8fdc070db0 2023-10-08T05:53:50.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.435+0000 7f8fe2ba9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fdc06a870 con 0x7f8fdc06ad00 2023-10-08T05:53:50.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.435+0000 7f8fe2ba9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fdc0cd640 con 0x7f8fdc071190 2023-10-08T05:53:50.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.435+0000 7f8fe0945640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8fdc070db0 0x7f8fdc15dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34064/0 (socket says 172.21.15.70:34064) 2023-10-08T05:53:50.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.435+0000 7f8fe0945640 1 -- 172.21.15.70:0/3493296985 learned_addr learned my addr 172.21.15.70:0/3493296985 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:50.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899106995 0 0) 0x7f8fdc0c9850 con 0x7f8fdc070db0 2023-10-08T05:53:50.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc4003520 con 0x7f8fdc070db0 2023-10-08T05:53:50.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdbfff640 1 --2- 172.21.15.70:0/3493296985 >> v2:172.21.15.181:3300/0 conn(0x7f8fdc071190 0x7f8fdc15e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:50.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3292904590 0 0) 0x7f8fc4003520 con 0x7f8fdc070db0 2023-10-08T05:53:50.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 >> v2:172.21.15.181:3300/0 conn(0x7f8fdc071190 msgr2=0x7f8fdc15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:50.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 --2- 172.21.15.70:0/3493296985 >> v2:172.21.15.181:3300/0 conn(0x7f8fdc071190 0x7f8fdc15e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 >> v1:172.21.15.181:6789/0 conn(0x7f8fdc06ad00 legacy=0x7f8fdc06b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:50.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc0cd910 con 0x7f8fdc070db0 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f8fcc0019f0 con 0x7f8fdc070db0 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 >> v1:172.21.15.70:6789/0 conn(0x7f8fdc070db0 legacy=0x7f8fdc15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fc4003970 0x7f8fc4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fc40047c0 0x7f8fc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc4005180 0x7f8fc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc0cd640 con 0x7f8fc4005180 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc400c0b0 con 0x7f8fc40047c0 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.436+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc400c270 con 0x7f8fc4003970 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.437+0000 7f8fe1146640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc4005180 0x7f8fc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.437+0000 7f8fe0945640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fc4003970 0x7f8fc4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:50.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.437+0000 7f8fdbfff640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fc40047c0 0x7f8fc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.437+0000 7f8fe1146640 1 -- 172.21.15.70:0/3493296985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fc4003970 msgr2=0x7f8fc4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.437+0000 7f8fe1146640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fc4003970 0x7f8fc4003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.437+0000 7f8fe1146640 1 -- 172.21.15.70:0/3493296985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fc40047c0 msgr2=0x7f8fc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.437+0000 7f8fe1146640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fc40047c0 0x7f8fc4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.437+0000 7f8fe1146640 1 -- 172.21.15.70:0/3493296985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc4003520 con 0x7f8fc4005180 2023-10-08T05:53:50.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.437+0000 7f8fdbfff640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fc40047c0 0x7f8fc4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:50.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.438+0000 7f8fe1146640 1 -- 172.21.15.70:0/3493296985 reap_dead start 2023-10-08T05:53:50.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.438+0000 7f8fe1146640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc4005180 0x7f8fc400b980 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4003d80 tx=0x7f8fc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7105464261185b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:50.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.439+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4007220 con 0x7f8fc4005180 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.439+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd402e050 con 0x7f8fc4005180 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.439+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3493296985 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40075e0 con 0x7f8fc4005180 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.439+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/3493296985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc4005180 msgr2=0x7f8fc400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.439+0000 7f8fe2ba9640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc4005180 0x7f8fc400b980 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4003d80 tx=0x7f8fc40057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.439+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/3493296985 shutdown_connections 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.439+0000 7f8fe2ba9640 1 --2- 172.21.15.70:0/3493296985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc4005180 0x7f8fc400b980 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.439+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/3493296985 >> 172.21.15.70:0/3493296985 conn(0x7f8fdc0b1480 msgr2=0x7f8fdc06cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.439+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/3493296985 shutdown_connections 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.439+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/3493296985 wait complete. 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.441+0000 7f8fe2ba9640 1 Processor -- start 2023-10-08T05:53:50.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.441+0000 7f8fe2ba9640 1 -- start start 2023-10-08T05:53:50.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.442+0000 7f8fe2ba9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fc4005180 0x7f8fdc0cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:50.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.442+0000 7f8fe2ba9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fdc0c7b10 0x7f8fdc1f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:50.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.442+0000 7f8fe2ba9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc0b24a0 0x7f8fdc0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.442+0000 7f8fe2ba9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc0cc460 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.442+0000 7f8fe2ba9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc0c9480 con 0x7f8fc4005180 2023-10-08T05:53:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.442+0000 7f8fe2ba9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1f2b90 con 0x7f8fdc0c7b10 2023-10-08T05:53:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fe1146640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc0b24a0 0x7f8fdc0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fe1146640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc0b24a0 0x7f8fdc0b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51600/0 (socket says 172.21.15.70:51600) 2023-10-08T05:53:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fe1146640 1 -- 172.21.15.70:0/1207325214 learned_addr learned my addr 172.21.15.70:0/1207325214 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fe0945640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fc4005180 0x7f8fdc0cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fdbfff640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fdc0c7b10 0x7f8fdc1f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fe1146640 1 -- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fdc0c7b10 msgr2=0x7f8fdc1f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:50.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fe1146640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fdc0c7b10 0x7f8fdc1f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fe1146640 1 -- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fc4005180 msgr2=0x7f8fdc0cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:50.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fe1146640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fc4005180 0x7f8fdc0cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fe1146640 1 -- 172.21.15.70:0/1207325214 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc1f2e70 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fdbfff640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fdc0c7b10 0x7f8fdc1f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:50.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.443+0000 7f8fe1146640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc0b24a0 0x7f8fdc0b2930 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4003ea0 tx=0x7f8fd40036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:50.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.444+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1207325214 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4007a70 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.444+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1207325214 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd403e470 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.444+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1207325214 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd403e800 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.444+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/1207325214 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc1f3150 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.444+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/1207325214 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc0b1f80 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.445+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/1207325214 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fdc0b21a0 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.446+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1207325214 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8fd403e960 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.447+0000 7f8fd97fa640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f8fc806afc0 0x7f8fc806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:50.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.447+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1207325214 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8fd40aba00 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.448+0000 7f8fe0945640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f8fc806afc0 0x7f8fc806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:50.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.449+0000 7f8fe0945640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f8fc806afc0 0x7f8fc806d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0013d0 tx=0x7f8fcc009040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:50.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.451+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1207325214 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8fd40767c0 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.659+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/1207325214 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fdc06a1a0 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.660+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1207325214 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f8fd4079de0 con 0x7f8fdc0b24a0 2023-10-08T05:53:50.660 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:53:50.660 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/290122525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":290122525},{"type":"v1","addr":"172.21.15.70:6835","nonce":290122525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4017950051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4017950051},{"type":"v1","addr":"172.21.15.181:6837","nonce":4017950051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1329101557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1329101557},{"type":"v1","addr":"172.21.15.70:6834","nonce":1329101557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:44.125639+0000","modified":"2023-10-08T05:53:47.206011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6184":{"gid":6184,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/422082772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":422082772},{"type":"v1","addr":"172.21.15.181:6835","nonce":422082772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-08T05:53:50.661 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 50 2023-10-08T05:53:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.662+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f8fc806afc0 msgr2=0x7f8fc806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.662+0000 7f8fe2ba9640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f8fc806afc0 0x7f8fc806d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0013d0 tx=0x7f8fcc009040 comp rx=0 tx=0).stop 2023-10-08T05:53:50.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.662+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/1207325214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc0b24a0 msgr2=0x7f8fdc0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:50.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.662+0000 7f8fe2ba9640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc0b24a0 0x7f8fdc0b2930 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4003ea0 tx=0x7f8fd40036f0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.663+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/1207325214 shutdown_connections 2023-10-08T05:53:50.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.663+0000 7f8fe2ba9640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f8fc806afc0 0x7f8fc806d480 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.663+0000 7f8fe2ba9640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc0b24a0 0x7f8fdc0b2930 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.663+0000 7f8fe2ba9640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8fdc0c7b10 0x7f8fdc1f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.663+0000 7f8fe2ba9640 1 --2- 172.21.15.70:0/1207325214 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8fc4005180 0x7f8fdc0cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:50.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.663+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/1207325214 >> 172.21.15.70:0/1207325214 conn(0x7f8fdc0b1480 msgr2=0x7f8fdc15dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:50.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.663+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/1207325214 shutdown_connections 2023-10-08T05:53:50.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:50.663+0000 7f8fe2ba9640 1 -- 172.21.15.70:0/1207325214 wait complete. 2023-10-08T05:53:50.676 INFO:tasks.ceph.mds.b.smithi181.stderr:2023-10-08T05:53:50.675+0000 7fe9ec7a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:53:51.083 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T05:53:44.125639+0000', 'modified': '2023-10-08T05:53:47.206011+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6184}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6184': {'gid': 6184, 'name': 'b', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.181:6835/422082772', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6834', 'nonce': 422082772}, {'type': 'v1', 'addr': '172.21.15.181:6835', 'nonce': 422082772}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T05:53:51.083 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T05:53:51.084 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T05:53:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.309+0000 7f052b890640 1 Processor -- start 2023-10-08T05:53:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.310+0000 7f052b890640 1 -- start start 2023-10-08T05:53:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.311+0000 7f052b890640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f05240cea30 0x7f052415d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.311+0000 7f052b890640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f05240ccb40 0x7f05240ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.311+0000 7f052b890640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05240cd490 con 0x7f052415cdb0 2023-10-08T05:53:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.311+0000 7f052b890640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052406b6a0 con 0x7f05240cea30 2023-10-08T05:53:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.311+0000 7f052b890640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05240b19e0 con 0x7f05240ccb40 2023-10-08T05:53:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.312+0000 7f0528e2b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f05240cea30 0x7f052415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.312+0000 7f0528e2b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f05240cea30 0x7f052415d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51614/0 (socket says 172.21.15.70:51614) 2023-10-08T05:53:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.312+0000 7f0528e2b640 1 -- 172.21.15.70:0/1936385701 learned_addr learned my addr 172.21.15.70:0/1936385701 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.312+0000 7f0528e2b640 1 -- 172.21.15.70:0/1936385701 >> v2:172.21.15.181:3300/0 conn(0x7f05240ccb40 msgr2=0x7f05240ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.312+0000 7f0528e2b640 1 --2- 172.21.15.70:0/1936385701 >> v2:172.21.15.181:3300/0 conn(0x7f05240ccb40 0x7f05240ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.312+0000 7f0528e2b640 1 -- 172.21.15.70:0/1936385701 >> v1:172.21.15.181:6789/0 conn(0x7f052415cdb0 legacy=0x7f052406af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.312+0000 7f0528e2b640 1 -- 172.21.15.70:0/1936385701 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524069ef0 con 0x7f05240cea30 2023-10-08T05:53:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.313+0000 7f0528e2b640 1 --2- 172.21.15.70:0/1936385701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05240cea30 0x7f052415d190 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f05240cb9c0 tx=0x7f051802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1c1298fda209b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:51.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.314+0000 7f0523fff640 1 -- 172.21.15.70:0/1936385701 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518031870 con 0x7f05240cea30 2023-10-08T05:53:51.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.314+0000 7f0523fff640 1 -- 172.21.15.70:0/1936385701 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05180319d0 con 0x7f05240cea30 2023-10-08T05:53:51.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.314+0000 7f0523fff640 1 -- 172.21.15.70:0/1936385701 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518031d60 con 0x7f05240cea30 2023-10-08T05:53:51.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.314+0000 7f052b890640 1 -- 172.21.15.70:0/1936385701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05240cea30 msgr2=0x7f052415d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.314+0000 7f052b890640 1 --2- 172.21.15.70:0/1936385701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05240cea30 0x7f052415d190 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f05240cb9c0 tx=0x7f051802b910 comp rx=0 tx=0).stop 2023-10-08T05:53:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.315+0000 7f052b890640 1 -- 172.21.15.70:0/1936385701 shutdown_connections 2023-10-08T05:53:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.315+0000 7f052b890640 1 --2- 172.21.15.70:0/1936385701 >> v2:172.21.15.181:3300/0 conn(0x7f05240ccb40 0x7f05240ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.315+0000 7f052b890640 1 --2- 172.21.15.70:0/1936385701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05240cea30 0x7f052415d190 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.315+0000 7f052b890640 1 -- 172.21.15.70:0/1936385701 >> 172.21.15.70:0/1936385701 conn(0x7f05240b0af0 msgr2=0x7f05240b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.315+0000 7f052b890640 1 -- 172.21.15.70:0/1936385701 shutdown_connections 2023-10-08T05:53:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.315+0000 7f052b890640 1 -- 172.21.15.70:0/1936385701 wait complete. 2023-10-08T05:53:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.317+0000 7f052b890640 1 Processor -- start 2023-10-08T05:53:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.317+0000 7f052b890640 1 -- start start 2023-10-08T05:53:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.318+0000 7f052b890640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05240ccb40 0x7f052414e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.318+0000 7f052b890640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05240cea30 0x7f05240b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.318+0000 7f052b890640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f052415cdb0 0x7f05240c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.318+0000 7f052b890640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240b20d0 con 0x7f052415cdb0 2023-10-08T05:53:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.318+0000 7f052b890640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052415b790 con 0x7f05240cea30 2023-10-08T05:53:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.318+0000 7f052b890640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052415ba70 con 0x7f05240ccb40 2023-10-08T05:53:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.318+0000 7f0529e2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f052415cdb0 0x7f05240c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.318+0000 7f0529e2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f052415cdb0 0x7f05240c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51622/0 (socket says 172.21.15.70:51622) 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.318+0000 7f0529e2d640 1 -- 172.21.15.70:0/351930574 learned_addr learned my addr 172.21.15.70:0/351930574 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f052962c640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05240ccb40 0x7f052414e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f0529e2d640 1 -- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05240ccb40 msgr2=0x7f052414e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f0529e2d640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05240ccb40 0x7f052414e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f0529e2d640 1 -- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05240cea30 msgr2=0x7f05240b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f0529e2d640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05240cea30 0x7f05240b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f0529e2d640 1 -- 172.21.15.70:0/351930574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05180094d0 con 0x7f052415cdb0 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f052962c640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05240ccb40 0x7f052414e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f0529e2d640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f052415cdb0 0x7f05240c4520 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f051c009fd0 tx=0x7f051c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f0521ffb640 1 -- 172.21.15.70:0/351930574 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c011830 con 0x7f052415cdb0 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f0521ffb640 1 -- 172.21.15.70:0/351930574 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c011990 con 0x7f052415cdb0 2023-10-08T05:53:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f0521ffb640 1 -- 172.21.15.70:0/351930574 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c011d20 con 0x7f052415cdb0 2023-10-08T05:53:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.319+0000 7f052b890640 1 -- 172.21.15.70:0/351930574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052415bd50 con 0x7f052415cdb0 2023-10-08T05:53:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.320+0000 7f052b890640 1 -- 172.21.15.70:0/351930574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052414ea20 con 0x7f052415cdb0 2023-10-08T05:53:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.322+0000 7f0521ffb640 1 -- 172.21.15.70:0/351930574 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f051c01b460 con 0x7f052415cdb0 2023-10-08T05:53:51.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.323+0000 7f0521ffb640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f051006af90 0x7f051006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.323+0000 7f0521ffb640 1 -- 172.21.15.70:0/351930574 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f051c0896b0 con 0x7f052415cdb0 2023-10-08T05:53:51.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.324+0000 7f052962c640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f051006af90 0x7f051006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.324+0000 7f052e41e740 1 Processor -- start 2023-10-08T05:53:51.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.324+0000 7f052e41e740 1 -- start start 2023-10-08T05:53:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f052962c640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f051006af90 0x7f051006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0514007eb0 tx=0x7f05140073c0 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f052e41e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 0x55ff63289a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f052e41e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632b6f40 0x55ff632b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f052e41e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff63289fa0 0x55ff632b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f052e41e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff631fd700 con 0x55ff632b6f40 2023-10-08T05:53:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f052e41e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff631fadf0 con 0x55ff632b6b60 2023-10-08T05:53:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f052e41e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff631f66e0 con 0x55ff63289fa0 2023-10-08T05:53:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f0528e2b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632b6f40 0x55ff632b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f0528e2b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632b6f40 0x55ff632b98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51638/0 (socket says 172.21.15.70:51638) 2023-10-08T05:53:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f0528e2b640 1 -- 172.21.15.70:0/2435186117 learned_addr learned my addr 172.21.15.70:0/2435186117 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:51.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f0529e2d640 1 --2- 172.21.15.70:0/2435186117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff63289fa0 0x55ff632b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f0528e2b640 1 -- 172.21.15.70:0/2435186117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff63289fa0 msgr2=0x55ff632b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f052962c640 1 --2- 172.21.15.70:0/2435186117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 0x55ff63289a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.325+0000 7f0528e2b640 1 --2- 172.21.15.70:0/2435186117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff63289fa0 0x55ff632b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.326+0000 7f0528e2b640 1 -- 172.21.15.70:0/2435186117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 msgr2=0x55ff63289a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.326+0000 7f0528e2b640 1 --2- 172.21.15.70:0/2435186117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 0x55ff63289a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.326+0000 7f0528e2b640 1 -- 172.21.15.70:0/2435186117 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff631ba910 con 0x55ff632b6f40 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.326+0000 7f0529e2d640 1 --2- 172.21.15.70:0/2435186117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff63289fa0 0x55ff632b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.326+0000 7f0528e2b640 1 --2- 172.21.15.70:0/2435186117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632b6f40 0x55ff632b98f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0518000c00 tx=0x7f051802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e682b5fd8596d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.326+0000 7f050affd640 1 -- 172.21.15.70:0/2435186117 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518031870 con 0x55ff632b6f40 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.326+0000 7f050affd640 1 -- 172.21.15.70:0/2435186117 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0518031d60 con 0x55ff632b6f40 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.326+0000 7f050affd640 1 -- 172.21.15.70:0/2435186117 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051803a660 con 0x55ff632b6f40 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.327+0000 7f052e41e740 1 -- 172.21.15.70:0/2435186117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632b6f40 msgr2=0x55ff632b98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.327+0000 7f052e41e740 1 --2- 172.21.15.70:0/2435186117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632b6f40 0x55ff632b98f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f0518000c00 tx=0x7f051802e000 comp rx=0 tx=0).stop 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.327+0000 7f052e41e740 1 -- 172.21.15.70:0/2435186117 shutdown_connections 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.327+0000 7f052e41e740 1 --2- 172.21.15.70:0/2435186117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff63289fa0 0x55ff632b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.327+0000 7f052e41e740 1 --2- 172.21.15.70:0/2435186117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632b6f40 0x55ff632b98f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.327+0000 7f052e41e740 1 --2- 172.21.15.70:0/2435186117 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 0x55ff63289a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.327+0000 7f052e41e740 1 -- 172.21.15.70:0/2435186117 >> 172.21.15.70:0/2435186117 conn(0x55ff6328bd70 msgr2=0x55ff632d48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:51.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.327+0000 7f052e41e740 1 -- 172.21.15.70:0/2435186117 shutdown_connections 2023-10-08T05:53:51.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.327+0000 7f052e41e740 1 -- 172.21.15.70:0/2435186117 wait complete. 2023-10-08T05:53:51.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.328+0000 7f052e41e740 1 -- start start 2023-10-08T05:53:51.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.328+0000 7f052e41e740 1 Processor -- start 2023-10-08T05:53:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.329+0000 7f052e41e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 0x55ff632a4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.329+0000 7f052e41e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6f40 0x55ff632a7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.329+0000 7f052e41e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.329+0000 7f052e41e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63194970 con 0x55ff632bcfb0 2023-10-08T05:53:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.329+0000 7f052e41e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff63194610 con 0x55ff632b6b60 2023-10-08T05:53:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.329+0000 7f052e41e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff62f5af20 con 0x55ff632b6f40 2023-10-08T05:53:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.329+0000 7f052e41e740 10 client.0 _ll_register_callbacks cb 0x55ff631d2610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:53:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.329+0000 7f0529e2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.329+0000 7f0529e2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632a4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51650/0 (socket says 172.21.15.70:51650) 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.329+0000 7f0529e2d640 1 -- 172.21.15.70:0/2066270920 learned_addr learned my addr 172.21.15.70:0/2066270920 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f0529e2d640 1 -- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6f40 msgr2=0x55ff632a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f0528e2b640 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6f40 0x55ff632a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f0529e2d640 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6f40 0x55ff632a7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f0529e2d640 1 -- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 msgr2=0x55ff632a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f0529e2d640 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 0x55ff632a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f0529e2d640 1 -- 172.21.15.70:0/2066270920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0524069ef0 con 0x55ff632bcfb0 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f0529e2d640 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632a4e10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f051c0581c0 tx=0x7f051c05c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e3af9702b01b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f04fffff640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f04fffff640 1 -- 172.21.15.70:0/2066270920 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c057920 con 0x55ff632bcfb0 2023-10-08T05:53:51.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f04fffff640 1 -- 172.21.15.70:0/2066270920 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f051c087ba0 con 0x55ff632bcfb0 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f052e41e740 1 -- 172.21.15.70:0/2066270920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff631ef6c0 con 0x55ff632bcfb0 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f04fffff640 1 -- 172.21.15.70:0/2066270920 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f051c05c170 con 0x55ff632bcfb0 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f052e41e740 10 client.7716 fetch_fsmap learned FSMap version 50 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.330+0000 7f052e41e740 1 -- 172.21.15.70:0/2066270920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55ff631f4d60 con 0x55ff632bcfb0 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.331+0000 7f04fffff640 1 -- 172.21.15.70:0/2066270920 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff631ef6c0 con 0x55ff632bcfb0 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.331+0000 7f04fffff640 1 -- 172.21.15.70:0/2066270920 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f051c0600a0 con 0x55ff632bcfb0 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.331+0000 7f052e41e740 10 client.7716 fetch_fsmap finished waiting for FSMap version 50 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.331+0000 7f052e41e740 10 client.7716 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.331+0000 7f052e41e740 20 client.7716 populate_metadata read hostname 'smithi070' 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.331+0000 7f052e41e740 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:6834/422082772,v1:172.21.15.181:6835/422082772] conn(0x55ff632d2950 0x55ff632d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.331+0000 7f052e41e740 4 client.7716 mds_command: new command op to 6184 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T05:53:51.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.331+0000 7f052e41e740 1 -- 172.21.15.70:0/2066270920 --> [v2:172.21.15.181:6834/422082772,v1:172.21.15.181:6835/422082772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff631ef3e0 con 0x55ff632d2950 2023-10-08T05:53:51.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.331+0000 7f052962c640 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:6834/422082772,v1:172.21.15.181:6835/422082772] conn(0x55ff632d2950 0x55ff632d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.332+0000 7f052962c640 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:6834/422082772,v1:172.21.15.181:6835/422082772] conn(0x55ff632d2950 0x55ff632d4430 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc44dfa8ea65bbe server_cookie=770fb925764b77b6 in_seq=0 out_seq=0 2023-10-08T05:53:51.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.332+0000 7f04fffff640 10 client.7716 ms_handle_connect on v2:172.21.15.181:6834/422082772 2023-10-08T05:53:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f04fffff640 1 -- 172.21.15.70:0/2066270920 <== mds.0 v2:172.21.15.181:6834/422082772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ff631ef3e0 con 0x55ff632d2950 2023-10-08T05:53:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f04fffff640 10 client.7716 handle_command_reply: tid=0 2023-10-08T05:53:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 client.7716 shutdown 2023-10-08T05:53:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 -- 172.21.15.70:0/2066270920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 msgr2=0x55ff632a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632a4e10 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f051c0581c0 tx=0x7f051c05c4c0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 -- 172.21.15.70:0/2066270920 shutdown_connections 2023-10-08T05:53:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:6834/422082772,v1:172.21.15.181:6835/422082772] conn(0x55ff632d2950 0x55ff632d4430 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632a4e10 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6f40 0x55ff632a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 --2- 172.21.15.70:0/2066270920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 0x55ff632a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 -- 172.21.15.70:0/2066270920 >> 172.21.15.70:0/2066270920 conn(0x55ff63289fa0 msgr2=0x55ff632b98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f04fffff640 0 client.7716 ms_handle_reset on v2:172.21.15.181:6834/422082772 2023-10-08T05:53:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 -- 172.21.15.70:0/2066270920 shutdown_connections 2023-10-08T05:53:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.334+0000 7f052e41e740 1 -- 172.21.15.70:0/2066270920 wait complete. 2023-10-08T05:53:51.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.335+0000 7f052e41e740 20 client.7716 trim_cache size 0 max 16384 2023-10-08T05:53:51.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.351+0000 7f052e41e740 1 Processor -- start 2023-10-08T05:53:51.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.351+0000 7f052e41e740 1 -- start start 2023-10-08T05:53:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f052e41e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 0x55ff632a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f052e41e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6f40 0x55ff632b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f052e41e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f052e41e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff6332e530 con 0x55ff632bcfb0 2023-10-08T05:53:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f052e41e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff632d7c80 con 0x55ff632b6b60 2023-10-08T05:53:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f052e41e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff632b9e20 con 0x55ff632b6f40 2023-10-08T05:53:51.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f0529e2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f0529e2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632baad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51658/0 (socket says 172.21.15.70:51658) 2023-10-08T05:53:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f0529e2d640 1 -- 172.21.15.70:0/1439572578 learned_addr learned my addr 172.21.15.70:0/1439572578 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f0529e2d640 1 -- 172.21.15.70:0/1439572578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6f40 msgr2=0x55ff632b98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f0529e2d640 1 --2- 172.21.15.70:0/1439572578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6f40 0x55ff632b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f0529e2d640 1 -- 172.21.15.70:0/1439572578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 msgr2=0x55ff632a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f0529e2d640 1 --2- 172.21.15.70:0/1439572578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 0x55ff632a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.352+0000 7f0529e2d640 1 -- 172.21.15.70:0/1439572578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff63193b80 con 0x55ff632bcfb0 2023-10-08T05:53:51.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.353+0000 7f0529e2d640 1 --2- 172.21.15.70:0/1439572578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632baad0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f051c0606f0 tx=0x7f051c089050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a248f77dbfbf766e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:51.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.353+0000 7f050b7fe640 1 -- 172.21.15.70:0/1439572578 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c05c170 con 0x55ff632bcfb0 2023-10-08T05:53:51.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.353+0000 7f050b7fe640 1 -- 172.21.15.70:0/1439572578 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c076e90 con 0x55ff632bcfb0 2023-10-08T05:53:51.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.353+0000 7f050b7fe640 1 -- 172.21.15.70:0/1439572578 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00d040 con 0x55ff632bcfb0 2023-10-08T05:53:51.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.353+0000 7f052e41e740 1 -- 172.21.15.70:0/1439572578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 msgr2=0x55ff632baad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.353+0000 7f052e41e740 1 --2- 172.21.15.70:0/1439572578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632baad0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f051c0606f0 tx=0x7f051c089050 comp rx=0 tx=0).stop 2023-10-08T05:53:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.354+0000 7f052e41e740 1 -- 172.21.15.70:0/1439572578 shutdown_connections 2023-10-08T05:53:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.354+0000 7f052e41e740 1 --2- 172.21.15.70:0/1439572578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632baad0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.354+0000 7f052e41e740 1 --2- 172.21.15.70:0/1439572578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6f40 0x55ff632b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.354+0000 7f052e41e740 1 --2- 172.21.15.70:0/1439572578 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6b60 0x55ff632a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.354+0000 7f052e41e740 1 -- 172.21.15.70:0/1439572578 >> 172.21.15.70:0/1439572578 conn(0x55ff63289fa0 msgr2=0x55ff632ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.354+0000 7f052e41e740 1 -- 172.21.15.70:0/1439572578 shutdown_connections 2023-10-08T05:53:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.354+0000 7f052e41e740 1 -- 172.21.15.70:0/1439572578 wait complete. 2023-10-08T05:53:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.354+0000 7f052e41e740 1 -- start start 2023-10-08T05:53:51.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.354+0000 7f052e41e740 1 Processor -- start 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.355+0000 7f052e41e740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6b60 0x55ff632f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.355+0000 7f052e41e740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6f40 0x55ff63328da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.355+0000 7f052e41e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f052e41e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff632a5b70 con 0x55ff632bcfb0 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f052e41e740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff632f1270 con 0x55ff632b6f40 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f052e41e740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff632b98e0 con 0x55ff632b6b60 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f052e41e740 10 client.0 _ll_register_callbacks cb 0x7f05180096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f0529e2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f0529e2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632d4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51662/0 (socket says 172.21.15.70:51662) 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f0529e2d640 1 -- 172.21.15.70:0/668224022 learned_addr learned my addr 172.21.15.70:0/668224022 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:51.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f0529e2d640 1 -- 172.21.15.70:0/668224022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6b60 msgr2=0x55ff632f0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f0529e2d640 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6b60 0x55ff632f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f0529e2d640 1 -- 172.21.15.70:0/668224022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6f40 msgr2=0x55ff63328da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f0529e2d640 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6f40 0x55ff63328da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f0529e2d640 1 -- 172.21.15.70:0/668224022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff631f4d60 con 0x55ff632bcfb0 2023-10-08T05:53:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f0529e2d640 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632d4430 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f051c013010 tx=0x7f051c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6574b1780ac173b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.356+0000 7f04fffff640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:53:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f04fffff640 1 -- 172.21.15.70:0/668224022 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c054590 con 0x55ff632bcfb0 2023-10-08T05:53:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f04fffff640 1 -- 172.21.15.70:0/668224022 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f051c087ba0 con 0x55ff632bcfb0 2023-10-08T05:53:51.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f052e41e740 1 -- 172.21.15.70:0/668224022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0518031d60 con 0x55ff632bcfb0 2023-10-08T05:53:51.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f04fffff640 1 -- 172.21.15.70:0/668224022 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f051c00d040 con 0x55ff632bcfb0 2023-10-08T05:53:51.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f052e41e740 10 client.7722 fetch_fsmap learned FSMap version 50 2023-10-08T05:53:51.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f052e41e740 1 -- 172.21.15.70:0/668224022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55ff631b9170 con 0x55ff632bcfb0 2023-10-08T05:53:51.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f04fffff640 1 -- 172.21.15.70:0/668224022 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0518031d60 con 0x55ff632bcfb0 2023-10-08T05:53:51.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f04fffff640 1 -- 172.21.15.70:0/668224022 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f051c05e130 con 0x55ff632bcfb0 2023-10-08T05:53:51.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f052e41e740 10 client.7722 fetch_fsmap finished waiting for FSMap version 50 2023-10-08T05:53:51.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f052e41e740 10 client.7722 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T05:53:51.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f052e41e740 20 client.7722 populate_metadata read hostname 'smithi070' 2023-10-08T05:53:51.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f052e41e740 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.181:6834/422082772,v1:172.21.15.181:6835/422082772] conn(0x55ff6332d030 0x55ff6332d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f052e41e740 4 client.7722 mds_command: new command op to 6184 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.357+0000 7f052e41e740 1 -- 172.21.15.70:0/668224022 --> [v2:172.21.15.181:6834/422082772,v1:172.21.15.181:6835/422082772] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ff62ef86c0 con 0x55ff6332d030 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.358+0000 7f052962c640 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.181:6834/422082772,v1:172.21.15.181:6835/422082772] conn(0x55ff6332d030 0x55ff6332d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.359+0000 7f04fffff640 10 client.7722 ms_handle_connect on v2:172.21.15.181:6834/422082772 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.359+0000 7f052962c640 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.181:6834/422082772,v1:172.21.15.181:6835/422082772] conn(0x55ff6332d030 0x55ff6332d410 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c511857f6e642ffd server_cookie=4df491acaf3eb7ec in_seq=0 out_seq=0 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f04fffff640 1 -- 172.21.15.70:0/668224022 <== mds.0 v2:172.21.15.181:6834/422082772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ff62ef86c0 con 0x55ff6332d030 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f04fffff640 10 client.7722 handle_command_reply: tid=0 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 client.7722 shutdown 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 -- 172.21.15.70:0/668224022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 msgr2=0x55ff632d4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632d4430 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f051c013010 tx=0x7f051c013040 comp rx=0 tx=0).stop 2023-10-08T05:53:51.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 -- 172.21.15.70:0/668224022 shutdown_connections 2023-10-08T05:53:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.181:6834/422082772,v1:172.21.15.181:6835/422082772] conn(0x55ff6332d030 0x55ff6332d410 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55ff632bcfb0 0x55ff632d4430 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55ff632b6f40 0x55ff63328da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 --2- 172.21.15.70:0/668224022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55ff632b6b60 0x55ff632f0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f04fffff640 0 client.7722 ms_handle_reset on v2:172.21.15.181:6834/422082772 2023-10-08T05:53:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 -- 172.21.15.70:0/668224022 >> 172.21.15.70:0/668224022 conn(0x55ff63289fa0 msgr2=0x55ff632bd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 -- 172.21.15.70:0/668224022 shutdown_connections 2023-10-08T05:53:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 1 -- 172.21.15.70:0/668224022 wait complete. 2023-10-08T05:53:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.360+0000 7f052e41e740 20 client.7722 trim_cache size 0 max 16384 2023-10-08T05:53:51.365 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:53:51.366 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T05:53:51.366 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T05:53:51.366 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 6184, 2023-10-08T05:53:51.366 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T05:53:51.366 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T05:53:51.366 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T05:53:51.366 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.6371889150000003, 2023-10-08T05:53:51.366 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 50, 2023-10-08T05:53:51.366 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 118, 2023-10-08T05:53:51.366 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 118, 2023-10-08T05:53:51.367 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 11.961253973 2023-10-08T05:53:51.367 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:53:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 -- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f051006af90 msgr2=0x7f051006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f051006af90 0x7f051006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0514007eb0 tx=0x7f05140073c0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 -- 172.21.15.70:0/351930574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f052415cdb0 msgr2=0x7f05240c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f052415cdb0 0x7f05240c4520 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f051c009fd0 tx=0x7f051c00b8e0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 -- 172.21.15.70:0/351930574 shutdown_connections 2023-10-08T05:53:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f051006af90 0x7f051006d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f052415cdb0 0x7f05240c4520 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f05240cea30 0x7f05240b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 --2- 172.21.15.70:0/351930574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f05240ccb40 0x7f052414e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 -- 172.21.15.70:0/351930574 >> 172.21.15.70:0/351930574 conn(0x7f05240b0af0 msgr2=0x7f05240ac9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:51.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 -- 172.21.15.70:0/351930574 shutdown_connections 2023-10-08T05:53:51.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.361+0000 7f052b890640 1 -- 172.21.15.70:0/351930574 wait complete. 2023-10-08T05:53:51.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:53:51.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.582+0000 7f930573e640 1 Processor -- start 2023-10-08T05:53:51.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.582+0000 7f930573e640 1 -- start start 2023-10-08T05:53:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.584+0000 7f930573e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 0x7f93000ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.584+0000 7f930573e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f93000c90b0 0x7f93000adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.584+0000 7f930573e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930015b2b0 con 0x7f93000cae70 2023-10-08T05:53:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.584+0000 7f930573e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f930015bf30 con 0x7f93000c90b0 2023-10-08T05:53:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.584+0000 7f930573e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930015f7a0 con 0x7f930015ab40 2023-10-08T05:53:51.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.585+0000 7f92feffd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 0x7f93000ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.585+0000 7f92feffd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 0x7f93000ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53324/0 (socket says 172.21.15.70:53324) 2023-10-08T05:53:51.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.585+0000 7f92feffd640 1 -- 172.21.15.70:0/3752760448 learned_addr learned my addr 172.21.15.70:0/3752760448 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:51.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.585+0000 7f92fe7fc640 1 --2- 172.21.15.70:0/3752760448 >> v2:172.21.15.181:3300/0 conn(0x7f93000c90b0 0x7f93000adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.585+0000 7f92feffd640 1 -- 172.21.15.70:0/3752760448 >> v2:172.21.15.181:3300/0 conn(0x7f93000c90b0 msgr2=0x7f93000adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:51.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.585+0000 7f92feffd640 1 --2- 172.21.15.70:0/3752760448 >> v2:172.21.15.181:3300/0 conn(0x7f93000c90b0 0x7f93000adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.585+0000 7f92feffd640 1 -- 172.21.15.70:0/3752760448 >> v1:172.21.15.181:6789/0 conn(0x7f93000cae70 legacy=0x7f93000702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.585+0000 7f92feffd640 1 -- 172.21.15.70:0/3752760448 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930015b590 con 0x7f930015ab40 2023-10-08T05:53:51.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.587+0000 7f92feffd640 1 --2- 172.21.15.70:0/3752760448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 0x7f93000ad720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009870 tx=0x7f92f002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76881ecdeb5535d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:51.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.587+0000 7f92fdffb640 1 -- 172.21.15.70:0/3752760448 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0030840 con 0x7f930015ab40 2023-10-08T05:53:51.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.587+0000 7f92fdffb640 1 -- 172.21.15.70:0/3752760448 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f00309a0 con 0x7f930015ab40 2023-10-08T05:53:51.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.587+0000 7f92fdffb640 1 -- 172.21.15.70:0/3752760448 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0030d30 con 0x7f930015ab40 2023-10-08T05:53:51.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.588+0000 7f930573e640 1 -- 172.21.15.70:0/3752760448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 msgr2=0x7f93000ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.588+0000 7f930573e640 1 --2- 172.21.15.70:0/3752760448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 0x7f93000ad720 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009870 tx=0x7f92f002b980 comp rx=0 tx=0).stop 2023-10-08T05:53:51.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.588+0000 7f930573e640 1 -- 172.21.15.70:0/3752760448 shutdown_connections 2023-10-08T05:53:51.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.588+0000 7f930573e640 1 --2- 172.21.15.70:0/3752760448 >> v2:172.21.15.181:3300/0 conn(0x7f93000c90b0 0x7f93000adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.588+0000 7f930573e640 1 --2- 172.21.15.70:0/3752760448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 0x7f93000ad720 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.588+0000 7f930573e640 1 -- 172.21.15.70:0/3752760448 >> 172.21.15.70:0/3752760448 conn(0x7f930006a380 msgr2=0x7f93000b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:51.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.588+0000 7f930573e640 1 -- 172.21.15.70:0/3752760448 shutdown_connections 2023-10-08T05:53:51.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.588+0000 7f930573e640 1 -- 172.21.15.70:0/3752760448 wait complete. 2023-10-08T05:53:51.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.590+0000 7f930573e640 1 Processor -- start 2023-10-08T05:53:51.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.590+0000 7f930573e640 1 -- start start 2023-10-08T05:53:51.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f930573e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93000c90b0 0x7f93000af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f930573e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93000cae70 0x7f9300151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f930573e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 0x7f93000ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f930573e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93000c56f0 con 0x7f93000cae70 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f930573e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93000c5410 con 0x7f93000c90b0 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f930573e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9300153a90 con 0x7f930015ab40 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f92fe7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93000cae70 0x7f9300151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f92fe7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93000cae70 0x7f9300151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51668/0 (socket says 172.21.15.70:51668) 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f92fe7fc640 1 -- 172.21.15.70:0/2766982497 learned_addr learned my addr 172.21.15.70:0/2766982497 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f92fe7fc640 1 -- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 msgr2=0x7f93000ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f92fe7fc640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 0x7f93000ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f92feffd640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93000c90b0 0x7f93000af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f92fe7fc640 1 -- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93000c90b0 msgr2=0x7f93000af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.591+0000 7f92fe7fc640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93000c90b0 0x7f93000af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.592+0000 7f92fe7fc640 1 -- 172.21.15.70:0/2766982497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f00094d0 con 0x7f93000cae70 2023-10-08T05:53:51.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.592+0000 7f92fe7fc640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93000cae70 0x7f9300151790 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f92f400d5f0 tx=0x7f92f400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:51.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.592+0000 7f92e77fe640 1 -- 172.21.15.70:0/2766982497 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f40125f0 con 0x7f93000cae70 2023-10-08T05:53:51.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.592+0000 7f92e77fe640 1 -- 172.21.15.70:0/2766982497 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f4012750 con 0x7f93000cae70 2023-10-08T05:53:51.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.592+0000 7f92e77fe640 1 -- 172.21.15.70:0/2766982497 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4012b00 con 0x7f93000cae70 2023-10-08T05:53:51.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.592+0000 7f930573e640 1 -- 172.21.15.70:0/2766982497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9300153d70 con 0x7f93000cae70 2023-10-08T05:53:51.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.592+0000 7f930573e640 1 -- 172.21.15.70:0/2766982497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9300151cd0 con 0x7f93000cae70 2023-10-08T05:53:51.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.594+0000 7f930573e640 1 -- 172.21.15.70:0/2766982497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93001538c0 con 0x7f93000cae70 2023-10-08T05:53:51.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.595+0000 7f92e77fe640 1 -- 172.21.15.70:0/2766982497 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f92f4012c60 con 0x7f93000cae70 2023-10-08T05:53:51.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.596+0000 7f92e77fe640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f92ec06af90 0x7f92ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:51.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.596+0000 7f92e77fe640 1 -- 172.21.15.70:0/2766982497 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f92f4089e00 con 0x7f93000cae70 2023-10-08T05:53:51.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.596+0000 7f92feffd640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f92ec06af90 0x7f92ec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:51.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.598+0000 7f92feffd640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f92ec06af90 0x7f92ec06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92f0036000 tx=0x7f92f003b040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:51.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.600+0000 7f92e77fe640 1 -- 172.21.15.70:0/2766982497 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92f4054c20 con 0x7f93000cae70 2023-10-08T05:53:51.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.834+0000 7f930573e640 1 -- 172.21.15.70:0/2766982497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9300067490 con 0x7f93000cae70 2023-10-08T05:53:51.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.835+0000 7f92e77fe640 1 -- 172.21.15.70:0/2766982497 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f92f4058240 con 0x7f93000cae70 2023-10-08T05:53:51.836 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T05:53:51.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 -- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f92ec06af90 msgr2=0x7f92ec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f92ec06af90 0x7f92ec06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f92f0036000 tx=0x7f92f003b040 comp rx=0 tx=0).stop 2023-10-08T05:53:51.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 -- 172.21.15.70:0/2766982497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93000cae70 msgr2=0x7f9300151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:51.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93000cae70 0x7f9300151790 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f92f400d5f0 tx=0x7f92f400dab0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 -- 172.21.15.70:0/2766982497 shutdown_connections 2023-10-08T05:53:51.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f92ec06af90 0x7f92ec06d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f930015ab40 0x7f93000ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93000cae70 0x7f9300151790 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 --2- 172.21.15.70:0/2766982497 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93000c90b0 0x7f93000af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:51.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 -- 172.21.15.70:0/2766982497 >> 172.21.15.70:0/2766982497 conn(0x7f930006a380 msgr2=0x7f93000ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:51.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 -- 172.21.15.70:0/2766982497 shutdown_connections 2023-10-08T05:53:51.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:51.838+0000 7f930573e640 1 -- 172.21.15.70:0/2766982497 wait complete. 2023-10-08T05:53:51.858 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T05:53:51.858 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:53:51.858 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:53:51.858 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T05:53:51.858 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T05:53:51.858 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T05:53:51.858 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T05:53:51.859 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:53:51.859 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:53:51.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:51.859 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:53:51.911 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:53:51.911 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:53:51.911 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:53:51.911 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:51.911 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:53:51.911 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:53:51.912 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41364sec preferred_lft 41364sec 2023-10-08T05:53:51.913 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:53:51.913 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:51.913 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:53:51.913 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:53:51.913 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T05:53:51.913 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:51.913 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T05:53:51.913 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:51.913 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:53:51.914 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T05:53:51.914 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T05:53:51.914 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:51.914 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:53:51.914 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T05:53:51.914 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T05:53:51.914 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:53:51.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:51.915 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:53:52.006 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:53:52.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:52.010 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:53:52.063 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:53:52.064 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:53:52.064 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:53:52.064 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:53:52.064 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:53:52.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:52.065 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:53:52.065 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:53:52.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:53:52.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:53:52.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:53:52.066 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:53:52.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:52.243 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:53:52.297 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T05:53:52.297 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T05:53:52.298 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:53:52.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:52.298 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T05:53:52.395 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:53:52.395 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T05:53:52.450 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:53:52.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:53:52.451 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:53:52.504 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T05:53:52.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:52.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:53:52.646 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:53:52.646 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:53:52.681 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:53:52.681 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:53:52.681 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:53:52.681 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:53:52.681 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:53:52.683 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T05:53:52.683 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:53:52.684 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:53:52.684 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:53:52.684 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:53:52.687 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:53:52.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:52.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:53:52.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:53:52.770 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:53:52.837 INFO:teuthology.orchestra.run.smithi070.stdout:client6268 2023-10-08T05:53:52.838 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:53:52.838 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:53:52.890 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T05:53:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.089+0000 7f6315c09640 1 Processor -- start 2023-10-08T05:53:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.089+0000 7f6315c09640 1 -- start start 2023-10-08T05:53:53.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.091+0000 7f6315c09640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63100cb130 0x7f63100cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.091+0000 7f6315c09640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63100b1cc0 con 0x7f631006dfc0 2023-10-08T05:53:53.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.091+0000 7f6315c09640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63100cbad0 con 0x7f63100afbe0 2023-10-08T05:53:53.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.091+0000 7f6315c09640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63100b2620 con 0x7f63100cb130 2023-10-08T05:53:53.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.091+0000 7f6307fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f631006dfc0 0x7f6310070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34080/0 (socket says 172.21.15.70:34080) 2023-10-08T05:53:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.091+0000 7f6307fff640 1 -- 172.21.15.70:0/1826974230 learned_addr learned my addr 172.21.15.70:0/1826974230 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630ffff640 1 --2- 172.21.15.70:0/1826974230 >> v2:172.21.15.181:3300/0 conn(0x7f63100cb130 0x7f63100cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975076720 0 0) 0x7f63100b1cc0 con 0x7f631006dfc0 2023-10-08T05:53:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62fc003520 con 0x7f631006dfc0 2023-10-08T05:53:53.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2291489037 0 0) 0x7f62fc003520 con 0x7f631006dfc0 2023-10-08T05:53:53.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 >> v2:172.21.15.181:3300/0 conn(0x7f63100cb130 msgr2=0x7f63100cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:53.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 --2- 172.21.15.70:0/1826974230 >> v2:172.21.15.181:3300/0 conn(0x7f63100cb130 0x7f63100cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 >> v1:172.21.15.181:6789/0 conn(0x7f63100afbe0 legacy=0x7f63100b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63100b2060 con 0x7f631006dfc0 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f62f8001670 con 0x7f631006dfc0 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 >> v1:172.21.15.70:6789/0 conn(0x7f631006dfc0 legacy=0x7f6310070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0039c0 0x7f62fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62fc0047c0 0x7f62fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62fc005180 0x7f62fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100b2620 con 0x7f62fc0039c0 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc00c080 con 0x7f62fc005180 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.092+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc00c240 con 0x7f62fc0047c0 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.093+0000 7f630f7fe640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0039c0 0x7f62fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.093+0000 7f630ffff640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62fc005180 0x7f62fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.093+0000 7f6307fff640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62fc0047c0 0x7f62fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.093+0000 7f630f7fe640 1 -- 172.21.15.70:0/1826974230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62fc0047c0 msgr2=0x7f62fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.093+0000 7f630f7fe640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62fc0047c0 0x7f62fc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.093+0000 7f630f7fe640 1 -- 172.21.15.70:0/1826974230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62fc005180 msgr2=0x7f62fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.093+0000 7f630f7fe640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62fc005180 0x7f62fc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.093+0000 7f630f7fe640 1 -- 172.21.15.70:0/1826974230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc003520 con 0x7f62fc0039c0 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.093+0000 7f630ffff640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62fc005180 0x7f62fc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.093+0000 7f630ffff640 1 -- 172.21.15.70:0/1826974230 reap_dead start 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.094+0000 7f630f7fe640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0039c0 0x7f62fc003de0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6300001750 tx=0x7f62fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ba9eda553f2141 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.094+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63000052d0 con 0x7f62fc0039c0 2023-10-08T05:53:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.095+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630002e050 con 0x7f62fc0039c0 2023-10-08T05:53:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.095+0000 7f630effd640 1 -- 172.21.15.70:0/1826974230 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300003830 con 0x7f62fc0039c0 2023-10-08T05:53:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.095+0000 7f6315c09640 1 -- 172.21.15.70:0/1826974230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0039c0 msgr2=0x7f62fc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.095+0000 7f6315c09640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0039c0 0x7f62fc003de0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f6300001750 tx=0x7f62fc0057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.095+0000 7f6315c09640 1 -- 172.21.15.70:0/1826974230 shutdown_connections 2023-10-08T05:53:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.095+0000 7f6315c09640 1 --2- 172.21.15.70:0/1826974230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0039c0 0x7f62fc003de0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.095+0000 7f6315c09640 1 -- 172.21.15.70:0/1826974230 >> 172.21.15.70:0/1826974230 conn(0x7f631006a5d0 msgr2=0x7f63100b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.095+0000 7f6315c09640 1 -- 172.21.15.70:0/1826974230 shutdown_connections 2023-10-08T05:53:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.095+0000 7f6315c09640 1 -- 172.21.15.70:0/1826974230 wait complete. 2023-10-08T05:53:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.097+0000 7f6315c09640 1 Processor -- start 2023-10-08T05:53:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.097+0000 7f6315c09640 1 -- start start 2023-10-08T05:53:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.098+0000 7f6315c09640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62fc0039c0 0x7f63100ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.098+0000 7f6315c09640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100cabb0 0x7f63100cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.098+0000 7f6315c09640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63100cdc30 0x7f63100c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.098+0000 7f6315c09640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100ce0e0 con 0x7f63100cabb0 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.098+0000 7f6315c09640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100c45a0 con 0x7f62fc0039c0 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.098+0000 7f6315c09640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63100c8d90 con 0x7f63100cdc30 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.098+0000 7f6307fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100cabb0 0x7f63100cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.098+0000 7f6307fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100cabb0 0x7f63100cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51688/0 (socket says 172.21.15.70:51688) 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.098+0000 7f6307fff640 1 -- 172.21.15.70:0/2983959027 learned_addr learned my addr 172.21.15.70:0/2983959027 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f6307fff640 1 -- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63100cdc30 msgr2=0x7f63100c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f630f7fe640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62fc0039c0 0x7f63100ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f6307fff640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63100cdc30 0x7f63100c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f6307fff640 1 -- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62fc0039c0 msgr2=0x7f63100ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f6307fff640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62fc0039c0 0x7f63100ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f6307fff640 1 -- 172.21.15.70:0/2983959027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63100c98d0 con 0x7f63100cabb0 2023-10-08T05:53:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f6307fff640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100cabb0 0x7f63100cd6f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f62f800d720 tx=0x7f62f800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f630cff9640 1 -- 172.21.15.70:0/2983959027 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80072d0 con 0x7f63100cabb0 2023-10-08T05:53:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f630cff9640 1 -- 172.21.15.70:0/2983959027 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f8007430 con 0x7f63100cabb0 2023-10-08T05:53:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f630cff9640 1 -- 172.21.15.70:0/2983959027 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f80047e0 con 0x7f63100cabb0 2023-10-08T05:53:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f6315c09640 1 -- 172.21.15.70:0/2983959027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63100c8fa0 con 0x7f63100cabb0 2023-10-08T05:53:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.099+0000 7f6315c09640 1 -- 172.21.15.70:0/2983959027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63100c9560 con 0x7f63100cabb0 2023-10-08T05:53:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.100+0000 7f6315c09640 1 -- 172.21.15.70:0/2983959027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63100c9a60 con 0x7f63100cabb0 2023-10-08T05:53:53.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.102+0000 7f630cff9640 1 -- 172.21.15.70:0/2983959027 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f62f8004940 con 0x7f63100cabb0 2023-10-08T05:53:53.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.102+0000 7f630cff9640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62f006af90 0x7f62f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.102+0000 7f630cff9640 1 -- 172.21.15.70:0/2983959027 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f62f8089480 con 0x7f63100cabb0 2023-10-08T05:53:53.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.103+0000 7f630f7fe640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62f006af90 0x7f62f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.104+0000 7f630f7fe640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62f006af90 0x7f62f006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6300033000 tx=0x7f6300009030 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:53.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.107+0000 7f630cff9640 1 -- 172.21.15.70:0/2983959027 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62f80541b0 con 0x7f63100cabb0 2023-10-08T05:53:53.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.322+0000 7f6315c09640 1 -- 172.21.15.70:0/2983959027 --> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6310070350 con 0x7f62f006af90 2023-10-08T05:53:53.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.326+0000 7f630cff9640 1 -- 172.21.15.70:0/2983959027 <== mgr.5513 v2:172.21.15.181:6832/1906631919 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6300007040 con 0x7f62f006af90 2023-10-08T05:53:53.326 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:53:53.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:53:53.326 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T05:53:53.327 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:53:53.327 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:53:53.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.328+0000 7f6315c09640 1 -- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62f006af90 msgr2=0x7f62f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.328+0000 7f6315c09640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62f006af90 0x7f62f006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6300033000 tx=0x7f6300009030 comp rx=0 tx=0).stop 2023-10-08T05:53:53.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.328+0000 7f6315c09640 1 -- 172.21.15.70:0/2983959027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100cabb0 msgr2=0x7f63100cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.328+0000 7f6315c09640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100cabb0 0x7f63100cd6f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f62f800d720 tx=0x7f62f800dbe0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.329+0000 7f6315c09640 1 -- 172.21.15.70:0/2983959027 shutdown_connections 2023-10-08T05:53:53.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.329+0000 7f6315c09640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f62f006af90 0x7f62f006d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.329+0000 7f6315c09640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63100cdc30 0x7f63100c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.329+0000 7f6315c09640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100cabb0 0x7f63100cd6f0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.329+0000 7f6315c09640 1 --2- 172.21.15.70:0/2983959027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62fc0039c0 0x7f63100ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.329+0000 7f6315c09640 1 -- 172.21.15.70:0/2983959027 >> 172.21.15.70:0/2983959027 conn(0x7f631006a5d0 msgr2=0x7f6310159f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:53.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.329+0000 7f6315c09640 1 -- 172.21.15.70:0/2983959027 shutdown_connections 2023-10-08T05:53:53.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.329+0000 7f6315c09640 1 -- 172.21.15.70:0/2983959027 wait complete. 2023-10-08T05:53:53.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T05:53:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.563+0000 7f84ef3d7640 1 Processor -- start 2023-10-08T05:53:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.564+0000 7f84ef3d7640 1 -- start start 2023-10-08T05:53:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.565+0000 7f84ef3d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84e80c9080 0x7f84e80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.565+0000 7f84ef3d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f84e8070b50 0x7f84e8070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.565+0000 7f84ef3d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e80c7ad0 con 0x7f84e815a820 2023-10-08T05:53:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.565+0000 7f84ef3d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e80c9a40 con 0x7f84e80c9080 2023-10-08T05:53:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.565+0000 7f84ef3d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e815ac00 con 0x7f84e8070b50 2023-10-08T05:53:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.566+0000 7f84ed173640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f84e815a820 0x7f84e8160750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49976/0 (socket says 172.21.15.70:49976) 2023-10-08T05:53:53.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.566+0000 7f84ed173640 1 -- 172.21.15.70:0/721518526 learned_addr learned my addr 172.21.15.70:0/721518526 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:53.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.566+0000 7f84ec972640 1 --2- 172.21.15.70:0/721518526 >> v2:172.21.15.70:3300/0 conn(0x7f84e80c9080 0x7f84e80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.566+0000 7f84ec972640 1 -- 172.21.15.70:0/721518526 >> v2:172.21.15.181:3300/0 conn(0x7f84e8070b50 msgr2=0x7f84e8070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.566+0000 7f84ed974640 1 --2- 172.21.15.70:0/721518526 >> v2:172.21.15.181:3300/0 conn(0x7f84e8070b50 0x7f84e8070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.566+0000 7f84ec972640 1 --2- 172.21.15.70:0/721518526 >> v2:172.21.15.181:3300/0 conn(0x7f84e8070b50 0x7f84e8070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.566+0000 7f84ec972640 1 -- 172.21.15.70:0/721518526 >> v1:172.21.15.70:6789/0 conn(0x7f84e815a820 legacy=0x7f84e8160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.566+0000 7f84dffff640 1 -- 172.21.15.70:0/721518526 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682128756 0 0) 0x7f84e80c7ad0 con 0x7f84e815a820 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.566+0000 7f84ec972640 1 -- 172.21.15.70:0/721518526 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e80714e0 con 0x7f84e80c9080 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.567+0000 7f84ec972640 1 --2- 172.21.15.70:0/721518526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e80c9080 0x7f84e80c9500 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f84e8070120 tx=0x7f84d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cc176d1cfcdab3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84dffff640 1 -- 172.21.15.70:0/721518526 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8031830 con 0x7f84e80c9080 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84dffff640 1 -- 172.21.15.70:0/721518526 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d8031990 con 0x7f84e80c9080 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84dffff640 1 -- 172.21.15.70:0/721518526 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8031d20 con 0x7f84e80c9080 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84ef3d7640 1 -- 172.21.15.70:0/721518526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e80c9080 msgr2=0x7f84e80c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84ef3d7640 1 --2- 172.21.15.70:0/721518526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e80c9080 0x7f84e80c9500 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f84e8070120 tx=0x7f84d802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84ef3d7640 1 -- 172.21.15.70:0/721518526 shutdown_connections 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84ef3d7640 1 --2- 172.21.15.70:0/721518526 >> v2:172.21.15.181:3300/0 conn(0x7f84e8070b50 0x7f84e8070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84ef3d7640 1 --2- 172.21.15.70:0/721518526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e80c9080 0x7f84e80c9500 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84ef3d7640 1 -- 172.21.15.70:0/721518526 >> 172.21.15.70:0/721518526 conn(0x7f84e8069d50 msgr2=0x7f84e806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84ef3d7640 1 -- 172.21.15.70:0/721518526 shutdown_connections 2023-10-08T05:53:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.568+0000 7f84ef3d7640 1 -- 172.21.15.70:0/721518526 wait complete. 2023-10-08T05:53:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.570+0000 7f84ef3d7640 1 Processor -- start 2023-10-08T05:53:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.570+0000 7f84ef3d7640 1 -- start start 2023-10-08T05:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.571+0000 7f84ef3d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84e8070b50 0x7f84e81460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.571+0000 7f84ef3d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e80c9080 0x7f84e80cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.571+0000 7f84ef3d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e815a820 0x7f84e80cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.571+0000 7f84ef3d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e815df70 con 0x7f84e815a820 2023-10-08T05:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.571+0000 7f84ef3d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e8146b50 con 0x7f84e80c9080 2023-10-08T05:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.571+0000 7f84ef3d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e8146e30 con 0x7f84e8070b50 2023-10-08T05:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ed974640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e815a820 0x7f84e80cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ed974640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e815a820 0x7f84e80cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56314/0 (socket says 172.21.15.70:56314) 2023-10-08T05:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ed974640 1 -- 172.21.15.70:0/2235744137 learned_addr learned my addr 172.21.15.70:0/2235744137 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ed173640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84e8070b50 0x7f84e81460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ed974640 1 -- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84e8070b50 msgr2=0x7f84e81460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ed974640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84e8070b50 0x7f84e81460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ec972640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e80c9080 0x7f84e80cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ed974640 1 -- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e80c9080 msgr2=0x7f84e80cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ed974640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e80c9080 0x7f84e80cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ed974640 1 -- 172.21.15.70:0/2235744137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d8009400 con 0x7f84e815a820 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.572+0000 7f84ed974640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e815a820 0x7f84e80cd650 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f84e000cd70 tx=0x7f84e000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.573+0000 7f84ddffb640 1 -- 172.21.15.70:0/2235744137 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0003b90 con 0x7f84e815a820 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.573+0000 7f84ddffb640 1 -- 172.21.15.70:0/2235744137 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e0003cf0 con 0x7f84e815a820 2023-10-08T05:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.573+0000 7f84ef3d7640 1 -- 172.21.15.70:0/2235744137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e8147110 con 0x7f84e815a820 2023-10-08T05:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.573+0000 7f84ddffb640 1 -- 172.21.15.70:0/2235744137 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0004240 con 0x7f84e815a820 2023-10-08T05:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.573+0000 7f84ef3d7640 1 -- 172.21.15.70:0/2235744137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e8146630 con 0x7f84e815a820 2023-10-08T05:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.574+0000 7f84ef3d7640 1 -- 172.21.15.70:0/2235744137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e8147350 con 0x7f84e815a820 2023-10-08T05:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.575+0000 7f84ddffb640 1 -- 172.21.15.70:0/2235744137 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 45) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f84e00043a0 con 0x7f84e815a820 2023-10-08T05:53:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.576+0000 7f84ddffb640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f84cc06af90 0x7f84cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.576+0000 7f84ddffb640 1 -- 172.21.15.70:0/2235744137 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f84e0012070 con 0x7f84e815a820 2023-10-08T05:53:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.576+0000 7f84ed173640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f84cc06af90 0x7f84cc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.577+0000 7f84ed173640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f84cc06af90 0x7f84cc06d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84e4000f60 tx=0x7f84e4009040 comp rx=0 tx=0).ready entity=mgr.5513 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:53.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.580+0000 7f84ddffb640 1 -- 172.21.15.70:0/2235744137 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84e8147350 con 0x7f84e815a820 2023-10-08T05:53:53.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:53.820+0000 7f84ef3d7640 1 -- 172.21.15.70:0/2235744137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f84e8147350 con 0x7f84e815a820 2023-10-08T05:53:54.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.805+0000 7f84ddffb640 1 -- 172.21.15.70:0/2235744137 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v46) v1 ==== 92+0+0 (secure 0 0 0) 0x7f84e8147350 con 0x7f84e815a820 2023-10-08T05:53:54.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.808+0000 7f84ef3d7640 1 -- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f84cc06af90 msgr2=0x7f84cc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:54.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.808+0000 7f84ef3d7640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f84cc06af90 0x7f84cc06d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84e4000f60 tx=0x7f84e4009040 comp rx=0 tx=0).stop 2023-10-08T05:53:54.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.808+0000 7f84ef3d7640 1 -- 172.21.15.70:0/2235744137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e815a820 msgr2=0x7f84e80cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:54.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.808+0000 7f84ef3d7640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e815a820 0x7f84e80cd650 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f84e000cd70 tx=0x7f84e000a660 comp rx=0 tx=0).stop 2023-10-08T05:53:54.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.809+0000 7f84ef3d7640 1 -- 172.21.15.70:0/2235744137 shutdown_connections 2023-10-08T05:53:54.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.809+0000 7f84ef3d7640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f84cc06af90 0x7f84cc06d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:54.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.809+0000 7f84ef3d7640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e815a820 0x7f84e80cd650 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:54.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.809+0000 7f84ef3d7640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e80c9080 0x7f84e80cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:54.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.809+0000 7f84ef3d7640 1 --2- 172.21.15.70:0/2235744137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84e8070b50 0x7f84e81460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:54.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.809+0000 7f84ef3d7640 1 -- 172.21.15.70:0/2235744137 >> 172.21.15.70:0/2235744137 conn(0x7f84e8069d50 msgr2=0x7f84e8155840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:54.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.809+0000 7f84ef3d7640 1 -- 172.21.15.70:0/2235744137 shutdown_connections 2023-10-08T05:53:54.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:54.809+0000 7f84ef3d7640 1 -- 172.21.15.70:0/2235744137 wait complete. 2023-10-08T05:53:54.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T05:53:55.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.039+0000 7ffa346d4640 1 Processor -- start 2023-10-08T05:53:55.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.039+0000 7ffa346d4640 1 -- start start 2023-10-08T05:53:55.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.041+0000 7ffa346d4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa2c06c3a0 0x7ffa2c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.041+0000 7ffa346d4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa2c06db60 0x7ffa2c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.041+0000 7ffa346d4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 0x7ffa2c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.041+0000 7ffa346d4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c0ca230 con 0x7ffa2c06db60 2023-10-08T05:53:55.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.041+0000 7ffa346d4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c15b310 con 0x7ffa2c06c3a0 2023-10-08T05:53:55.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.041+0000 7ffa346d4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c06d290 con 0x7ffa2c15aa00 2023-10-08T05:53:55.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.041+0000 7ffa31c6f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa2c06db60 0x7ffa2c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.041+0000 7ffa31c6f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa2c06db60 0x7ffa2c06c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56350/0 (socket says 172.21.15.70:56350) 2023-10-08T05:53:55.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.041+0000 7ffa31c6f640 1 -- 172.21.15.70:0/1013658429 learned_addr learned my addr 172.21.15.70:0/1013658429 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:55.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.042+0000 7ffa32470640 1 --2- 172.21.15.70:0/1013658429 >> v2:172.21.15.181:3300/0 conn(0x7ffa2c06c3a0 0x7ffa2c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.042+0000 7ffa32c71640 1 --2- 172.21.15.70:0/1013658429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 0x7ffa2c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.042+0000 7ffa31c6f640 1 -- 172.21.15.70:0/1013658429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 msgr2=0x7ffa2c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.042+0000 7ffa31c6f640 1 --2- 172.21.15.70:0/1013658429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 0x7ffa2c0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.042+0000 7ffa31c6f640 1 -- 172.21.15.70:0/1013658429 >> v2:172.21.15.181:3300/0 conn(0x7ffa2c06c3a0 msgr2=0x7ffa2c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.042+0000 7ffa31c6f640 1 --2- 172.21.15.70:0/1013658429 >> v2:172.21.15.181:3300/0 conn(0x7ffa2c06c3a0 0x7ffa2c0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.042+0000 7ffa31c6f640 1 -- 172.21.15.70:0/1013658429 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c0c9860 con 0x7ffa2c06db60 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.042+0000 7ffa32470640 1 --2- 172.21.15.70:0/1013658429 >> v2:172.21.15.181:3300/0 conn(0x7ffa2c06c3a0 0x7ffa2c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.042+0000 7ffa32c71640 1 --2- 172.21.15.70:0/1013658429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 0x7ffa2c0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.043+0000 7ffa31c6f640 1 --2- 172.21.15.70:0/1013658429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 0x7ffa2c06c780 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0716b0 tx=0x7ffa1c02e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e12e13a5ab87819 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.043+0000 7ffa3146e640 1 -- 172.21.15.70:0/1013658429 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c02ee80 con 0x7ffa2c06db60 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.043+0000 7ffa3146e640 1 -- 172.21.15.70:0/1013658429 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c036440 con 0x7ffa2c06db60 2023-10-08T05:53:55.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.043+0000 7ffa3146e640 1 -- 172.21.15.70:0/1013658429 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c0367b0 con 0x7ffa2c06db60 2023-10-08T05:53:55.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.044+0000 7ffa346d4640 1 -- 172.21.15.70:0/1013658429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 msgr2=0x7ffa2c06c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.044+0000 7ffa346d4640 1 --2- 172.21.15.70:0/1013658429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 0x7ffa2c06c780 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0716b0 tx=0x7ffa1c02e0d0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.044+0000 7ffa346d4640 1 -- 172.21.15.70:0/1013658429 shutdown_connections 2023-10-08T05:53:55.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.044+0000 7ffa346d4640 1 --2- 172.21.15.70:0/1013658429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 0x7ffa2c0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.044+0000 7ffa346d4640 1 --2- 172.21.15.70:0/1013658429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 0x7ffa2c06c780 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.044+0000 7ffa346d4640 1 --2- 172.21.15.70:0/1013658429 >> v2:172.21.15.181:3300/0 conn(0x7ffa2c06c3a0 0x7ffa2c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.044+0000 7ffa346d4640 1 -- 172.21.15.70:0/1013658429 >> 172.21.15.70:0/1013658429 conn(0x7ffa2c06a5d0 msgr2=0x7ffa2c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:55.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.044+0000 7ffa346d4640 1 -- 172.21.15.70:0/1013658429 shutdown_connections 2023-10-08T05:53:55.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.044+0000 7ffa346d4640 1 -- 172.21.15.70:0/1013658429 wait complete. 2023-10-08T05:53:55.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.046+0000 7ffa346d4640 1 Processor -- start 2023-10-08T05:53:55.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.046+0000 7ffa346d4640 1 -- start start 2023-10-08T05:53:55.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.047+0000 7ffa346d4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c06c3a0 0x7ffa2c154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.047+0000 7ffa346d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 0x7ffa2c0c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.047+0000 7ffa346d4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 0x7ffa2c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.047+0000 7ffa346d4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0c78d0 con 0x7ffa2c06db60 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.047+0000 7ffa346d4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c154f30 con 0x7ffa2c06c3a0 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.047+0000 7ffa346d4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c155210 con 0x7ffa2c15aa00 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.047+0000 7ffa31c6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 0x7ffa2c0c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.047+0000 7ffa31c6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 0x7ffa2c0c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56362/0 (socket says 172.21.15.70:56362) 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.047+0000 7ffa31c6f640 1 -- 172.21.15.70:0/2805954288 learned_addr learned my addr 172.21.15.70:0/2805954288 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.048+0000 7ffa32c71640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 0x7ffa2c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.048+0000 7ffa32470640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c06c3a0 0x7ffa2c154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.048+0000 7ffa31c6f640 1 -- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 msgr2=0x7ffa2c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.048+0000 7ffa31c6f640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 0x7ffa2c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.048+0000 7ffa31c6f640 1 -- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c06c3a0 msgr2=0x7ffa2c154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.048+0000 7ffa31c6f640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c06c3a0 0x7ffa2c154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.048+0000 7ffa31c6f640 1 -- 172.21.15.70:0/2805954288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c00b770 con 0x7ffa2c06db60 2023-10-08T05:53:55.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.048+0000 7ffa31c6f640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 0x7ffa2c0c7390 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c02e110 tx=0x7ffa1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:55.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.049+0000 7ffa22ffd640 1 -- 172.21.15.70:0/2805954288 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c034b80 con 0x7ffa2c06db60 2023-10-08T05:53:55.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.049+0000 7ffa22ffd640 1 -- 172.21.15.70:0/2805954288 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c036e90 con 0x7ffa2c06db60 2023-10-08T05:53:55.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.049+0000 7ffa22ffd640 1 -- 172.21.15.70:0/2805954288 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c041460 con 0x7ffa2c06db60 2023-10-08T05:53:55.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.049+0000 7ffa346d4640 1 -- 172.21.15.70:0/2805954288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c1554f0 con 0x7ffa2c06db60 2023-10-08T05:53:55.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.049+0000 7ffa346d4640 1 -- 172.21.15.70:0/2805954288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c155ab0 con 0x7ffa2c06db60 2023-10-08T05:53:55.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.050+0000 7ffa346d4640 1 -- 172.21.15.70:0/2805954288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2c155ca0 con 0x7ffa2c06db60 2023-10-08T05:53:55.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.052+0000 7ffa22ffd640 1 -- 172.21.15.70:0/2805954288 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 46) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ffa1c049360 con 0x7ffa2c06db60 2023-10-08T05:53:55.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.054+0000 7ffa22ffd640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ffa1006afe0 0x7ffa1006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.054+0000 7ffa22ffd640 1 -- 172.21.15.70:0/2805954288 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffa1c0abda0 con 0x7ffa2c06db60 2023-10-08T05:53:55.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.054+0000 7ffa32470640 1 -- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ffa1006afe0 msgr2=0x7ffa1006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1906631919 2023-10-08T05:53:55.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.054+0000 7ffa32470640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ffa1006afe0 0x7ffa1006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:55.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.058+0000 7ffa22ffd640 1 -- 172.21.15.70:0/2805954288 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa1c076c60 con 0x7ffa2c06db60 2023-10-08T05:53:55.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.251+0000 7ffa346d4640 1 -- 172.21.15.70:0/2805954288 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ffa2c155350 con 0x7ffa2c06db60 2023-10-08T05:53:55.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.255+0000 7ffa32470640 1 -- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ffa1006afe0 msgr2=0x7ffa1006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1906631919 2023-10-08T05:53:55.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.255+0000 7ffa32470640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ffa1006afe0 0x7ffa1006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:53:55.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.257+0000 7ffa22ffd640 1 -- 172.21.15.70:0/2805954288 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7ffa2c155350 con 0x7ffa2c06db60 2023-10-08T05:53:55.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.259+0000 7ffa346d4640 1 -- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ffa1006afe0 msgr2=0x7ffa1006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:55.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.259+0000 7ffa346d4640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ffa1006afe0 0x7ffa1006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.259+0000 7ffa346d4640 1 -- 172.21.15.70:0/2805954288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 msgr2=0x7ffa2c0c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.259+0000 7ffa346d4640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 0x7ffa2c0c7390 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c02e110 tx=0x7ffa1c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.259+0000 7ffa346d4640 1 -- 172.21.15.70:0/2805954288 shutdown_connections 2023-10-08T05:53:55.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.259+0000 7ffa346d4640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7ffa1006afe0 0x7ffa1006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.259+0000 7ffa346d4640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa2c15aa00 0x7ffa2c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.259+0000 7ffa346d4640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa2c06db60 0x7ffa2c0c7390 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.259+0000 7ffa346d4640 1 --2- 172.21.15.70:0/2805954288 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa2c06c3a0 0x7ffa2c154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.259+0000 7ffa346d4640 1 -- 172.21.15.70:0/2805954288 >> 172.21.15.70:0/2805954288 conn(0x7ffa2c06a5d0 msgr2=0x7ffa2c0b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:55.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.260+0000 7ffa346d4640 1 -- 172.21.15.70:0/2805954288 shutdown_connections 2023-10-08T05:53:55.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.260+0000 7ffa346d4640 1 -- 172.21.15.70:0/2805954288 wait complete. 2023-10-08T05:53:55.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T05:53:55.342 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:55.342+0000 7f98ed630040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:53:55.384 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:55.383+0000 7f86ec04c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:53:55.397 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:55.397+0000 7f98ed630040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:53:55.442 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:55.441+0000 7f86ec04c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:53:55.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.489+0000 7feebdd89640 1 Processor -- start 2023-10-08T05:53:55.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.490+0000 7feebdd89640 1 -- start start 2023-10-08T05:53:55.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.491+0000 7feebdd89640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feeb806dfc0 0x7feeb8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.491+0000 7feebdd89640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feeb8071070 0x7feeb80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.491+0000 7feebdd89640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb80b1cc0 con 0x7feeb80afbe0 2023-10-08T05:53:55.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.491+0000 7feebdd89640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb80b2620 con 0x7feeb806dfc0 2023-10-08T05:53:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.491+0000 7feebdd89640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb80b2060 con 0x7feeb8071070 2023-10-08T05:53:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.491+0000 7feeb77fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feeb80afbe0 0x7feeb80b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49998/0 (socket says 172.21.15.70:49998) 2023-10-08T05:53:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.491+0000 7feeb77fe640 1 -- 172.21.15.70:0/258726647 learned_addr learned my addr 172.21.15.70:0/258726647 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.491+0000 7feeb6ffd640 1 --2- 172.21.15.70:0/258726647 >> v2:172.21.15.70:3300/0 conn(0x7feeb806dfc0 0x7feeb8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.492+0000 7feeb7fff640 1 --2- 172.21.15.70:0/258726647 >> v2:172.21.15.181:3300/0 conn(0x7feeb8071070 0x7feeb80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.492+0000 7feeb67fc640 1 -- 172.21.15.70:0/258726647 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666349705 0 0) 0x7feeb80b1cc0 con 0x7feeb80afbe0 2023-10-08T05:53:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.492+0000 7feeb67fc640 1 -- 172.21.15.70:0/258726647 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea0003520 con 0x7feeb80afbe0 2023-10-08T05:53:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.492+0000 7feeb6ffd640 1 -- 172.21.15.70:0/258726647 >> v2:172.21.15.181:3300/0 conn(0x7feeb8071070 msgr2=0x7feeb80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.492+0000 7feeb6ffd640 1 --2- 172.21.15.70:0/258726647 >> v2:172.21.15.181:3300/0 conn(0x7feeb8071070 0x7feeb80cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.492+0000 7feeb6ffd640 1 -- 172.21.15.70:0/258726647 >> v1:172.21.15.70:6789/0 conn(0x7feeb80afbe0 legacy=0x7feeb80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.492+0000 7feeb6ffd640 1 -- 172.21.15.70:0/258726647 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb80b2340 con 0x7feeb806dfc0 2023-10-08T05:53:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.493+0000 7feeb6ffd640 1 --2- 172.21.15.70:0/258726647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 0x7feeb8070b30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7feeb80c8d10 tx=0x7feeac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db40a848f6d1717d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:55.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.493+0000 7feeb67fc640 1 -- 172.21.15.70:0/258726647 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac031830 con 0x7feeb806dfc0 2023-10-08T05:53:55.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.493+0000 7feeb67fc640 1 -- 172.21.15.70:0/258726647 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac031990 con 0x7feeb806dfc0 2023-10-08T05:53:55.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.493+0000 7feeb67fc640 1 -- 172.21.15.70:0/258726647 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac031d20 con 0x7feeb806dfc0 2023-10-08T05:53:55.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.494+0000 7feebdd89640 1 -- 172.21.15.70:0/258726647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 msgr2=0x7feeb8070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.494+0000 7feebdd89640 1 --2- 172.21.15.70:0/258726647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 0x7feeb8070b30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7feeb80c8d10 tx=0x7feeac02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.494+0000 7feebdd89640 1 -- 172.21.15.70:0/258726647 shutdown_connections 2023-10-08T05:53:55.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.494+0000 7feebdd89640 1 --2- 172.21.15.70:0/258726647 >> v2:172.21.15.181:3300/0 conn(0x7feeb8071070 0x7feeb80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.494+0000 7feebdd89640 1 --2- 172.21.15.70:0/258726647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 0x7feeb8070b30 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.494+0000 7feebdd89640 1 -- 172.21.15.70:0/258726647 >> 172.21.15.70:0/258726647 conn(0x7feeb806a5d0 msgr2=0x7feeb80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:55.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.494+0000 7feebdd89640 1 -- 172.21.15.70:0/258726647 shutdown_connections 2023-10-08T05:53:55.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.494+0000 7feebdd89640 1 -- 172.21.15.70:0/258726647 wait complete. 2023-10-08T05:53:55.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.496+0000 7feebdd89640 1 Processor -- start 2023-10-08T05:53:55.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.496+0000 7feebdd89640 1 -- start start 2023-10-08T05:53:55.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feebdd89640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 0x7feeb815cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feebdd89640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeb8071070 0x7feeb814bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feebdd89640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeb80afbe0 0x7feeb80c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feebdd89640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb814c410 con 0x7feeb806dfc0 2023-10-08T05:53:55.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feebdd89640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb815baf0 con 0x7feeb80afbe0 2023-10-08T05:53:55.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feebdd89640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb815bdd0 con 0x7feeb8071070 2023-10-08T05:53:55.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feeb77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 0x7feeb815cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feeb77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 0x7feeb815cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56372/0 (socket says 172.21.15.70:56372) 2023-10-08T05:53:55.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feeb77fe640 1 -- 172.21.15.70:0/1224934612 learned_addr learned my addr 172.21.15.70:0/1224934612 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:55.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feeb6ffd640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeb8071070 0x7feeb814bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feeb77fe640 1 -- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeb8071070 msgr2=0x7feeb814bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feeb77fe640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeb8071070 0x7feeb814bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feeb77fe640 1 -- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeb80afbe0 msgr2=0x7feeb80c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feeb77fe640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeb80afbe0 0x7feeb80c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.497+0000 7feeb77fe640 1 -- 172.21.15.70:0/1224934612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac009400 con 0x7feeb806dfc0 2023-10-08T05:53:55.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.498+0000 7feeb77fe640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 0x7feeb815cb90 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7feea800c410 tx=0x7feea800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:55.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.498+0000 7fee9ffff640 1 -- 172.21.15.70:0/1224934612 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea8005ce0 con 0x7feeb806dfc0 2023-10-08T05:53:55.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.498+0000 7fee9ffff640 1 -- 172.21.15.70:0/1224934612 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea8005e40 con 0x7feeb806dfc0 2023-10-08T05:53:55.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.498+0000 7feebdd89640 1 -- 172.21.15.70:0/1224934612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb815c0b0 con 0x7feeb806dfc0 2023-10-08T05:53:55.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.498+0000 7fee9ffff640 1 -- 172.21.15.70:0/1224934612 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea80072f0 con 0x7feeb806dfc0 2023-10-08T05:53:55.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.498+0000 7feebdd89640 1 -- 172.21.15.70:0/1224934612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb815c670 con 0x7feeb806dfc0 2023-10-08T05:53:55.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.499+0000 7feebdd89640 1 -- 172.21.15.70:0/1224934612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb815c860 con 0x7feeb806dfc0 2023-10-08T05:53:55.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.500+0000 7fee9ffff640 1 -- 172.21.15.70:0/1224934612 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 46) v1 ==== 89348+0+0 (secure 0 0 0) 0x7feea8007450 con 0x7feeb806dfc0 2023-10-08T05:53:55.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.502+0000 7fee9ffff640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feea406afe0 0x7feea406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.502+0000 7fee9ffff640 1 -- 172.21.15.70:0/1224934612 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feea8089080 con 0x7feeb806dfc0 2023-10-08T05:53:55.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.502+0000 7feeb6ffd640 1 -- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feea406afe0 msgr2=0x7feea406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1906631919 2023-10-08T05:53:55.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.502+0000 7feeb6ffd640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feea406afe0 0x7feea406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:55.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.505+0000 7fee9ffff640 1 -- 172.21.15.70:0/1224934612 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feea80bbca0 con 0x7feeb806dfc0 2023-10-08T05:53:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.698+0000 7feebdd89640 1 -- 172.21.15.70:0/1224934612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7feeb804e160 con 0x7feeb806dfc0 2023-10-08T05:53:55.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.702+0000 7fee9ffff640 1 -- 172.21.15.70:0/1224934612 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7feea80170c0 con 0x7feeb806dfc0 2023-10-08T05:53:55.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.703+0000 7feeb6ffd640 1 -- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feea406afe0 msgr2=0x7feea406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1906631919 2023-10-08T05:53:55.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.703+0000 7feeb6ffd640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feea406afe0 0x7feea406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:53:55.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 -- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feea406afe0 msgr2=0x7feea406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feea406afe0 0x7feea406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 -- 172.21.15.70:0/1224934612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 msgr2=0x7feeb815cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 0x7feeb815cb90 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7feea800c410 tx=0x7feea800c8d0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 -- 172.21.15.70:0/1224934612 shutdown_connections 2023-10-08T05:53:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7feea406afe0 0x7feea406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeb80afbe0 0x7feeb80c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeb8071070 0x7feeb814bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 --2- 172.21.15.70:0/1224934612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feeb806dfc0 0x7feeb815cb90 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 -- 172.21.15.70:0/1224934612 >> 172.21.15.70:0/1224934612 conn(0x7feeb806a5d0 msgr2=0x7feeb806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:55.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 -- 172.21.15.70:0/1224934612 shutdown_connections 2023-10-08T05:53:55.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.705+0000 7feebdd89640 1 -- 172.21.15.70:0/1224934612 wait complete. 2023-10-08T05:53:55.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /bad-path 1m 2023-10-08T05:53:55.872 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:55.872+0000 7f98ed630040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:53:55.899 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:55.898+0000 7f86ec04c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:53:55.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.916+0000 7f9920f16640 1 Processor -- start 2023-10-08T05:53:55.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.917+0000 7f9920f16640 1 -- start start 2023-10-08T05:53:55.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.918+0000 7f9920f16640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f991c163da0 0x7f991c166190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.918+0000 7f9920f16640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c0b12b0 con 0x7f991c15e3c0 2023-10-08T05:53:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.918+0000 7f9920f16640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c0b1500 con 0x7f991c0b1910 2023-10-08T05:53:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.918+0000 7f9920f16640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c0cf1d0 con 0x7f991c163da0 2023-10-08T05:53:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.918+0000 7f991a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f991c15e3c0 0x7f991c15ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50000/0 (socket says 172.21.15.70:50000) 2023-10-08T05:53:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.918+0000 7f991a59c640 1 -- 172.21.15.70:0/3383353611 learned_addr learned my addr 172.21.15.70:0/3383353611 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:53:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570717637 0 0) 0x7f991c0b12b0 con 0x7f991c15e3c0 2023-10-08T05:53:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f991ad9d640 1 --2- 172.21.15.70:0/3383353611 >> v2:172.21.15.181:3300/0 conn(0x7f991c163da0 0x7f991c166190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9908003520 con 0x7f991c15e3c0 2023-10-08T05:53:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 839073902 0 0) 0x7f9908003520 con 0x7f991c15e3c0 2023-10-08T05:53:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 >> v2:172.21.15.181:3300/0 conn(0x7f991c163da0 msgr2=0x7f991c166190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 --2- 172.21.15.70:0/3383353611 >> v2:172.21.15.181:3300/0 conn(0x7f991c163da0 0x7f991c166190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 >> v1:172.21.15.181:6789/0 conn(0x7f991c0b1910 legacy=0x7f991c15f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c0cf4b0 con 0x7f991c15e3c0 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f990c001670 con 0x7f991c15e3c0 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 >> v1:172.21.15.70:6789/0 conn(0x7f991c15e3c0 legacy=0x7f991c15ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9908003970 0x7f9908003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99080047c0 0x7f9908004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908005180 0x7f990800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0cf1d0 con 0x7f9908005180 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990800c020 con 0x7f9908003970 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.919+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990800c1e0 con 0x7f99080047c0 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.920+0000 7f991ad9d640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908005180 0x7f990800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.920+0000 7f991a59c640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9908003970 0x7f9908003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.920+0000 7f9913fff640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99080047c0 0x7f9908004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.920+0000 7f991ad9d640 1 -- 172.21.15.70:0/3383353611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99080047c0 msgr2=0x7f9908004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.920+0000 7f991ad9d640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99080047c0 0x7f9908004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.920+0000 7f991ad9d640 1 -- 172.21.15.70:0/3383353611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9908003970 msgr2=0x7f9908003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:53:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.920+0000 7f991ad9d640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9908003970 0x7f9908003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.920+0000 7f991ad9d640 1 -- 172.21.15.70:0/3383353611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9908003520 con 0x7f9908005180 2023-10-08T05:53:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.921+0000 7f991ad9d640 1 -- 172.21.15.70:0/3383353611 reap_dead start 2023-10-08T05:53:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.921+0000 7f991ad9d640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908005180 0x7f990800b980 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9914009f90 tx=0x7f99080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd17d692dbb40ea5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.921+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991402cc20 con 0x7f9908005180 2023-10-08T05:53:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.922+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991400f040 con 0x7f9908005180 2023-10-08T05:53:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.922+0000 7f9919d9b640 1 -- 172.21.15.70:0/3383353611 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914003990 con 0x7f9908005180 2023-10-08T05:53:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.922+0000 7f9920f16640 1 -- 172.21.15.70:0/3383353611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908005180 msgr2=0x7f990800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.922+0000 7f9920f16640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908005180 0x7f990800b980 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f9914009f90 tx=0x7f99080057f0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.922+0000 7f9920f16640 1 -- 172.21.15.70:0/3383353611 shutdown_connections 2023-10-08T05:53:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.922+0000 7f9920f16640 1 --2- 172.21.15.70:0/3383353611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9908005180 0x7f990800b980 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.922+0000 7f9920f16640 1 -- 172.21.15.70:0/3383353611 >> 172.21.15.70:0/3383353611 conn(0x7f991c099d40 msgr2=0x7f991c0d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:53:55.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.922+0000 7f9920f16640 1 -- 172.21.15.70:0/3383353611 shutdown_connections 2023-10-08T05:53:55.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.922+0000 7f9920f16640 1 -- 172.21.15.70:0/3383353611 wait complete. 2023-10-08T05:53:55.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.924+0000 7f9920f16640 1 Processor -- start 2023-10-08T05:53:55.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.924+0000 7f9920f16640 1 -- start start 2023-10-08T05:53:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.925+0000 7f9920f16640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9908005180 0x7f991c0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.925+0000 7f9920f16640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991c0b42a0 0x7f991c155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.925+0000 7f9920f16640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f991c0b4720 0x7f991c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.925+0000 7f9920f16640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0cc2f0 con 0x7f991c0b4720 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.925+0000 7f9920f16640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0cc5d0 con 0x7f9908005180 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.925+0000 7f9920f16640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0cc8b0 con 0x7f991c0b42a0 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.925+0000 7f991ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f991c0b4720 0x7f991c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.925+0000 7f991ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f991c0b4720 0x7f991c156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56392/0 (socket says 172.21.15.70:56392) 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.925+0000 7f991ad9d640 1 -- 172.21.15.70:0/2828156148 learned_addr learned my addr 172.21.15.70:0/2828156148 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f991a59c640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9908005180 0x7f991c0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f9913fff640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991c0b42a0 0x7f991c155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f991ad9d640 1 -- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991c0b42a0 msgr2=0x7f991c155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f991ad9d640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f991c0b42a0 0x7f991c155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f991ad9d640 1 -- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9908005180 msgr2=0x7f991c0c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:53:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f991ad9d640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9908005180 0x7f991c0c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f991ad9d640 1 -- 172.21.15.70:0/2828156148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c0ccb90 con 0x7f991c0b4720 2023-10-08T05:53:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f991ad9d640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f991c0b4720 0x7f991c156430 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f991402c690 tx=0x7f9914009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:53:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914035ee0 con 0x7f991c0b4720 2023-10-08T05:53:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9914040470 con 0x7f991c0b4720 2023-10-08T05:53:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914040800 con 0x7f991c0b4720 2023-10-08T05:53:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.926+0000 7f9920f16640 1 -- 172.21.15.70:0/2828156148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c0cce70 con 0x7f991c0b4720 2023-10-08T05:53:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.927+0000 7f9920f16640 1 -- 172.21.15.70:0/2828156148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c0c8080 con 0x7f991c0b4720 2023-10-08T05:53:55.930 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:55.927+0000 7f98ed630040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:53:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.928+0000 7f9920f16640 1 -- 172.21.15.70:0/2828156148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c0cd0b0 con 0x7f991c0b4720 2023-10-08T05:53:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.929+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 46) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f9914040960 con 0x7f991c0b4720 2023-10-08T05:53:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.930+0000 7f9912ffd640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 0x7f98fc06d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:53:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.930+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f99140ac830 con 0x7f991c0b4720 2023-10-08T05:53:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.930+0000 7f991a59c640 1 -- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 msgr2=0x7f98fc06d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1906631919 2023-10-08T05:53:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.930+0000 7f991a59c640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 0x7f98fc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:53:55.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:55.938+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f991c0cd0b0 con 0x7f991c0b4720 2023-10-08T05:53:55.952 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:55.951+0000 7f86ec04c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:53:55.983 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:55.982+0000 7f98ed630040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:53:56.002 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:56.001+0000 7f86ec04c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:53:56.046 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:56.045+0000 7f98ed630040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:53:56.062 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:56.061+0000 7f86ec04c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:53:56.096 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:56.096+0000 7f98ed630040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:53:56.111 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:56.110+0000 7f86ec04c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:53:56.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:56.131+0000 7f991a59c640 1 -- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 msgr2=0x7f98fc06d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1906631919 2023-10-08T05:53:56.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:56.131+0000 7f991a59c640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 0x7f98fc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:53:56.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:56.148+0000 7f9920f16640 1 -- 172.21.15.70:0/2828156148 --> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f991c15e7a0 con 0x7f98fc06b010 2023-10-08T05:53:56.208 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:56.208+0000 7f98ed630040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:53:56.221 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:56.220+0000 7f86ec04c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:53:56.270 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:56.270+0000 7f98ed630040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:56.283 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:56.282+0000 7f86ec04c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:56.319 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:56.319+0000 7f98ed630040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:53:56.333 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:56.332+0000 7f86ec04c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:53:56.375 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:56.375+0000 7f98ed630040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:53:56.389 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:56.388+0000 7f86ec04c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:53:56.425 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:56.424+0000 7f98ed630040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:53:56.439 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:56.438+0000 7f86ec04c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:53:56.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:56.532+0000 7f991a59c640 1 -- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 msgr2=0x7f98fc06d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1906631919 2023-10-08T05:53:56.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:56.532+0000 7f991a59c640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 0x7f98fc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:53:56.804 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:56.804+0000 7f98ed630040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:53:56.818 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:56.817+0000 7f86ec04c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:53:56.962 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:56.961+0000 7f98ed630040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:53:56.976 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:56.974+0000 7f86ec04c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:53:57.070 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:57.069+0000 7f98ed630040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:53:57.084 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:57.083+0000 7f86ec04c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:53:57.119 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:57.119+0000 7f98ed630040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:53:57.134 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:57.133+0000 7f86ec04c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:53:57.226 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:57.226+0000 7f98ed630040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:57.242 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:57.241+0000 7f86ec04c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:57.332 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:57.332+0000 7f98ed630040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:53:57.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:57.333+0000 7f991a59c640 1 -- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 msgr2=0x7f98fc06d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1906631919 2023-10-08T05:53:57.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:57.333+0000 7f991a59c640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 0x7f98fc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:53:57.348 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:57.347+0000 7f86ec04c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:53:57.490 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:57.489+0000 7f98ed630040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:57.505 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:57.504+0000 7f86ec04c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:53:57.540 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:57.539+0000 7f98ed630040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:57.555 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:57.554+0000 7f86ec04c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:53:57.659 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:57.657+0000 7f98ed630040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:53:57.674 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:57.672+0000 7f86ec04c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:53:58.485 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:58.484+0000 7f98ed630040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:53:58.490 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:58.489+0000 7f86ec04c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:53:58.779 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:58.778+0000 7f98ed630040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:53:58.786 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:58.785+0000 7f86ec04c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:53:58.836 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:58.835+0000 7f98ed630040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:53:58.842 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:58.841+0000 7f86ec04c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:53:58.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:58.935+0000 7f991a59c640 1 -- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 msgr2=0x7f98fc06d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1906631919 2023-10-08T05:53:58.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:58.935+0000 7f991a59c640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 0x7f98fc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T05:53:58.949 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:53:58.948+0000 7f98ed630040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:53:58.954 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:58.953+0000 7f86ec04c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:53:58.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:58.972+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 47) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f9914077ab0 con 0x7f991c0b4720 2023-10-08T05:53:58.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:58.973+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 msgr2=0x7f98fc06d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:53:58.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:53:58.973+0000 7f9912ffd640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1906631919,v1:172.21.15.181:6833/1906631919] conn(0x7f98fc06b010 0x7f98fc06d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:53:59.123 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:59.122+0000 7f866bb87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:59.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:59.122+0000 7f866bb87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:59.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:59.122+0000 7f866bb87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:59.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:59.122+0000 7f866bb87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:59.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:59.122+0000 7f866bb87640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:59.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:59.122+0000 7f8668b41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:59.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:59.122+0000 7f8668b41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:59.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:59.122+0000 7f8668b41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:59.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:59.122+0000 7f8668b41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:53:59.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:53:59.122+0000 7f8668b41640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:54:00.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.031+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 48) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f991407f190 con 0x7f991c0b4720 2023-10-08T05:54:00.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.032+0000 7f9912ffd640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f98fc035910 0x7f98fc075470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:54:00.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.032+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 --> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f98fc035cf0 con 0x7f98fc035910 2023-10-08T05:54:00.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.034+0000 7f991a59c640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f98fc035910 0x7f98fc075470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:54:00.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.040+0000 7f991a59c640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f98fc035910 0x7f98fc075470 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f990c001eb0 tx=0x7f990c001de0 comp rx=0 tx=0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:54:00.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.174+0000 7f9912ffd640 1 -- 172.21.15.70:0/2828156148 <== mgr.6280 v2:172.21.15.181:6832/1714252273 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f98fc035cf0 con 0x7f98fc035910 2023-10-08T05:54:00.174 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /bad-path 2023-10-08T05:54:00.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.177+0000 7f9920f16640 1 -- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f98fc035910 msgr2=0x7f98fc075470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:54:00.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.177+0000 7f9920f16640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f98fc035910 0x7f98fc075470 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f990c001eb0 tx=0x7f990c001de0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.177+0000 7f9920f16640 1 -- 172.21.15.70:0/2828156148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f991c0b4720 msgr2=0x7f991c156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:54:00.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.177+0000 7f9920f16640 1 --2- 172.21.15.70:0/2828156148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f991c0b4720 0x7f991c156430 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f991402c690 tx=0x7f9914009750 comp rx=0 tx=0).stop 2023-10-08T05:54:00.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.177+0000 7f991ad9d640 1 -- 172.21.15.70:0/2828156148 reap_dead start 2023-10-08T05:54:00.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.177+0000 7f9920f16640 1 -- 172.21.15.70:0/2828156148 shutdown_connections 2023-10-08T05:54:00.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.177+0000 7f9920f16640 1 -- 172.21.15.70:0/2828156148 >> 172.21.15.70:0/2828156148 conn(0x7f991c099d40 msgr2=0x7f991c157090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:54:00.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.177+0000 7f9920f16640 1 -- 172.21.15.70:0/2828156148 shutdown_connections 2023-10-08T05:54:00.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.177+0000 7f9920f16640 1 -- 172.21.15.70:0/2828156148 wait complete. 2023-10-08T05:54:00.198 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-10-08T05:54:00.198 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-10-08T05:54:00.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.415+0000 7fd28cbc9640 1 Processor -- start 2023-10-08T05:54:00.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.415+0000 7fd28cbc9640 1 -- start start 2023-10-08T05:54:00.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.417+0000 7fd28cbc9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd2880afbe0 0x7fd2880b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:54:00.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.417+0000 7fd28cbc9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd28806dfc0 0x7fd288070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:54:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.417+0000 7fd28cbc9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2880b2620 con 0x7fd288071070 2023-10-08T05:54:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.417+0000 7fd28cbc9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2880b2060 con 0x7fd2880afbe0 2023-10-08T05:54:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.417+0000 7fd28cbc9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880b2340 con 0x7fd28806dfc0 2023-10-08T05:54:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.417+0000 7fd286d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd288071070 0x7fd2880cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50014/0 (socket says 172.21.15.70:50014) 2023-10-08T05:54:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.417+0000 7fd286d9d640 1 -- 172.21.15.70:0/3680789955 learned_addr learned my addr 172.21.15.70:0/3680789955 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:54:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.417+0000 7fd28659c640 1 --2- 172.21.15.70:0/3680789955 >> v2:172.21.15.70:3300/0 conn(0x7fd2880afbe0 0x7fd2880b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:54:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.418+0000 7fd27ffff640 1 --2- 172.21.15.70:0/3680789955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd28806dfc0 0x7fd288070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:54:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.418+0000 7fd285d9b640 1 -- 172.21.15.70:0/3680789955 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243388674 0 0) 0x7fd2880b2620 con 0x7fd288071070 2023-10-08T05:54:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.418+0000 7fd285d9b640 1 -- 172.21.15.70:0/3680789955 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd274003520 con 0x7fd288071070 2023-10-08T05:54:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.418+0000 7fd28659c640 1 -- 172.21.15.70:0/3680789955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd28806dfc0 msgr2=0x7fd288070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:54:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.418+0000 7fd28659c640 1 --2- 172.21.15.70:0/3680789955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd28806dfc0 0x7fd288070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.418+0000 7fd28659c640 1 -- 172.21.15.70:0/3680789955 >> v1:172.21.15.70:6789/0 conn(0x7fd288071070 legacy=0x7fd2880cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:54:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.418+0000 7fd28659c640 1 -- 172.21.15.70:0/3680789955 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd288070350 con 0x7fd2880afbe0 2023-10-08T05:54:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.419+0000 7fd28659c640 1 --2- 172.21.15.70:0/3680789955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 0x7fd2880b15b0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd2780097a0 tx=0x7fd27802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c732da1c9af866 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:54:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.420+0000 7fd285d9b640 1 -- 172.21.15.70:0/3680789955 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27802d040 con 0x7fd2880afbe0 2023-10-08T05:54:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.420+0000 7fd285d9b640 1 -- 172.21.15.70:0/3680789955 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd278031810 con 0x7fd2880afbe0 2023-10-08T05:54:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.420+0000 7fd285d9b640 1 -- 172.21.15.70:0/3680789955 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278031ba0 con 0x7fd2880afbe0 2023-10-08T05:54:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.420+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3680789955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 msgr2=0x7fd2880b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:54:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.420+0000 7fd28cbc9640 1 --2- 172.21.15.70:0/3680789955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 0x7fd2880b15b0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd2780097a0 tx=0x7fd27802b8b0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.420+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3680789955 shutdown_connections 2023-10-08T05:54:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.420+0000 7fd28cbc9640 1 --2- 172.21.15.70:0/3680789955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd28806dfc0 0x7fd288070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.420+0000 7fd28cbc9640 1 --2- 172.21.15.70:0/3680789955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 0x7fd2880b15b0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.420+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3680789955 >> 172.21.15.70:0/3680789955 conn(0x7fd28806a5d0 msgr2=0x7fd2880b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:54:00.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.421+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3680789955 shutdown_connections 2023-10-08T05:54:00.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.421+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3680789955 wait complete. 2023-10-08T05:54:00.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.422+0000 7fd28cbc9640 1 Processor -- start 2023-10-08T05:54:00.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.422+0000 7fd28cbc9640 1 -- start start 2023-10-08T05:54:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.423+0000 7fd28cbc9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd28806dfc0 0x7fd28814e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:54:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.423+0000 7fd28cbc9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288071070 0x7fd2880cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:54:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.423+0000 7fd28cbc9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 0x7fd2880c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:54:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.423+0000 7fd28cbc9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880cc640 con 0x7fd2880afbe0 2023-10-08T05:54:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.423+0000 7fd28cbc9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2881f2640 con 0x7fd288071070 2023-10-08T05:54:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.423+0000 7fd28cbc9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28814eaa0 con 0x7fd28806dfc0 2023-10-08T05:54:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd286d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 0x7fd2880c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:54:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd286d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 0x7fd2880c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56406/0 (socket says 172.21.15.70:56406) 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd286d9d640 1 -- 172.21.15.70:0/3319874487 learned_addr learned my addr 172.21.15.70:0/3319874487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd28659c640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd28806dfc0 0x7fd28814e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd27ffff640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288071070 0x7fd2880cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd286d9d640 1 -- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd28806dfc0 msgr2=0x7fd28814e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd286d9d640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd28806dfc0 0x7fd28814e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd286d9d640 1 -- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288071070 msgr2=0x7fd2880cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd286d9d640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288071070 0x7fd2880cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd286d9d640 1 -- 172.21.15.70:0/3319874487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd278009400 con 0x7fd2880afbe0 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd286d9d640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 0x7fd2880c4210 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd28000caa0 tx=0x7fd28000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.424+0000 7fd27effd640 1 -- 172.21.15.70:0/3319874487 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28000ac00 con 0x7fd2880afbe0 2023-10-08T05:54:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.425+0000 7fd27effd640 1 -- 172.21.15.70:0/3319874487 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28000ad60 con 0x7fd2880afbe0 2023-10-08T05:54:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.425+0000 7fd27effd640 1 -- 172.21.15.70:0/3319874487 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280012620 con 0x7fd2880afbe0 2023-10-08T05:54:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.425+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3319874487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28814fb90 con 0x7fd2880afbe0 2023-10-08T05:54:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.425+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3319874487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2880cc890 con 0x7fd2880afbe0 2023-10-08T05:54:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.426+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3319874487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28804e160 con 0x7fd2880afbe0 2023-10-08T05:54:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.427+0000 7fd27effd640 1 -- 172.21.15.70:0/3319874487 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 48) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fd280007070 con 0x7fd2880afbe0 2023-10-08T05:54:00.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.428+0000 7fd27effd640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fd26806ad10 0x7fd26806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:54:00.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.428+0000 7fd27effd640 1 -- 172.21.15.70:0/3319874487 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fd2800895e0 con 0x7fd2880afbe0 2023-10-08T05:54:00.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.428+0000 7fd28659c640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fd26806ad10 0x7fd26806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:54:00.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.429+0000 7fd28659c640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fd26806ad10 0x7fd26806d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd278036000 tx=0x7fd27803c040 comp rx=0 tx=0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:54:00.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.432+0000 7fd27effd640 1 -- 172.21.15.70:0/3319874487 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd280054630 con 0x7fd2880afbe0 2023-10-08T05:54:00.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.626+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3319874487 --> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2880affc0 con 0x7fd26806ad10 2023-10-08T05:54:00.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.643+0000 7fd27effd640 1 -- 172.21.15.70:0/3319874487 <== mgr.6280 v2:172.21.15.181:6832/1714252273 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7fd278031b00 con 0x7fd26806ad10 2023-10-08T05:54:00.643 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:54:00.643 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:54:00", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-08T05:54:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fd26806ad10 msgr2=0x7fd26806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:54:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fd26806ad10 0x7fd26806d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd278036000 tx=0x7fd27803c040 comp rx=0 tx=0).stop 2023-10-08T05:54:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3319874487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 msgr2=0x7fd2880c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:54:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 0x7fd2880c4210 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd28000caa0 tx=0x7fd28000cf60 comp rx=0 tx=0).stop 2023-10-08T05:54:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3319874487 shutdown_connections 2023-10-08T05:54:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fd26806ad10 0x7fd26806d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2880afbe0 0x7fd2880c4210 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd288071070 0x7fd2880cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 --2- 172.21.15.70:0/3319874487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd28806dfc0 0x7fd28814e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:54:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3319874487 >> 172.21.15.70:0/3319874487 conn(0x7fd28806a5d0 msgr2=0x7fd2880c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:54:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3319874487 shutdown_connections 2023-10-08T05:54:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:54:00.645+0000 7fd28cbc9640 1 -- 172.21.15.70:0/3319874487 wait complete. 2023-10-08T05:54:00.665 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:54:00", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-08T05:54:10.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:54:10.145 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:54:40.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:54:40.291 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:55:00.196 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:00.195+0000 7f865b326640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:00.197 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:00.195+0000 7f865b326640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:00.197 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:00.195+0000 7f865b326640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:00.197 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:00.195+0000 7f865b326640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:00.197 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:00.195+0000 7f865b326640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:00.666 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-10-08T05:55:00.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.894+0000 7f983f2a9640 1 Processor -- start 2023-10-08T05:55:00.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.894+0000 7f983f2a9640 1 -- start start 2023-10-08T05:55:00.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.896+0000 7f983f2a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9838160210 0x7f98380c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:00.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.896+0000 7f983f2a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98380caa80 0x7f98380cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:00.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.896+0000 7f983f2a9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983806d1a0 con 0x7f98381605f0 2023-10-08T05:55:00.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.896+0000 7f983f2a9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98380cc7a0 con 0x7f9838160210 2023-10-08T05:55:00.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.896+0000 7f983f2a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98380ccfb0 con 0x7f98380caa80 2023-10-08T05:55:00.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.896+0000 7f983d045640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9838160210 0x7f98380c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:00.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.896+0000 7f983d045640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9838160210 0x7f98380c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42596/0 (socket says 172.21.15.70:42596) 2023-10-08T05:55:00.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.896+0000 7f983d045640 1 -- 172.21.15.70:0/3963674635 learned_addr learned my addr 172.21.15.70:0/3963674635 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:00.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.897+0000 7f983d045640 1 -- 172.21.15.70:0/3963674635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98380caa80 msgr2=0x7f98380cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:00.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.897+0000 7f983d045640 1 --2- 172.21.15.70:0/3963674635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98380caa80 0x7f98380cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:00.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.897+0000 7f983d045640 1 -- 172.21.15.70:0/3963674635 >> v1:172.21.15.181:6789/0 conn(0x7f98381605f0 legacy=0x7f9838070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:00.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.897+0000 7f983d045640 1 -- 172.21.15.70:0/3963674635 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380c9760 con 0x7f9838160210 2023-10-08T05:55:00.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.898+0000 7f983d045640 1 --2- 172.21.15.70:0/3963674635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9838160210 0x7f98380c9220 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9834009870 tx=0x7f983402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcaef606927c885a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:00.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.898+0000 7f982ffff640 1 -- 172.21.15.70:0/3963674635 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834031640 con 0x7f9838160210 2023-10-08T05:55:00.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.898+0000 7f982ffff640 1 -- 172.21.15.70:0/3963674635 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98340317a0 con 0x7f9838160210 2023-10-08T05:55:00.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.898+0000 7f982ffff640 1 -- 172.21.15.70:0/3963674635 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834031b30 con 0x7f9838160210 2023-10-08T05:55:00.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.899+0000 7f983f2a9640 1 -- 172.21.15.70:0/3963674635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9838160210 msgr2=0x7f98380c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:00.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.899+0000 7f983f2a9640 1 --2- 172.21.15.70:0/3963674635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9838160210 0x7f98380c9220 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f9834009870 tx=0x7f983402b8f0 comp rx=0 tx=0).stop 2023-10-08T05:55:00.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.899+0000 7f983f2a9640 1 -- 172.21.15.70:0/3963674635 shutdown_connections 2023-10-08T05:55:00.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.899+0000 7f983f2a9640 1 --2- 172.21.15.70:0/3963674635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98380caa80 0x7f98380cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:00.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.899+0000 7f983f2a9640 1 --2- 172.21.15.70:0/3963674635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9838160210 0x7f98380c9220 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:00.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.899+0000 7f983f2a9640 1 -- 172.21.15.70:0/3963674635 >> 172.21.15.70:0/3963674635 conn(0x7f98380cebd0 msgr2=0x7f98380cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:00.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.899+0000 7f983f2a9640 1 -- 172.21.15.70:0/3963674635 shutdown_connections 2023-10-08T05:55:00.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.899+0000 7f983f2a9640 1 -- 172.21.15.70:0/3963674635 wait complete. 2023-10-08T05:55:00.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.901+0000 7f983f2a9640 1 Processor -- start 2023-10-08T05:55:00.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.901+0000 7f983f2a9640 1 -- start start 2023-10-08T05:55:00.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.902+0000 7f983f2a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98380caa80 0x7f98381f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:00.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.902+0000 7f983f2a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9838160210 0x7f9838154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:00.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.902+0000 7f983f2a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98381605f0 0x7f98380c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:00.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.902+0000 7f983f2a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9838154690 con 0x7f98381605f0 2023-10-08T05:55:00.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.902+0000 7f983f2a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98381f34d0 con 0x7f98380caa80 2023-10-08T05:55:00.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.902+0000 7f983f2a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98381f2c30 con 0x7f9838160210 2023-10-08T05:55:00.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.902+0000 7f983d846640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98381605f0 0x7f98380c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:00.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.902+0000 7f983d846640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98381605f0 0x7f98380c32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42610/0 (socket says 172.21.15.70:42610) 2023-10-08T05:55:00.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.902+0000 7f983d846640 1 -- 172.21.15.70:0/2982081176 learned_addr learned my addr 172.21.15.70:0/2982081176 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:00.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f983d045640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98380caa80 0x7f98381f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:00.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f983c844640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9838160210 0x7f9838154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:00.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f983d846640 1 -- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9838160210 msgr2=0x7f9838154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:00.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f983d846640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9838160210 0x7f9838154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:00.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f983d846640 1 -- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98380caa80 msgr2=0x7f98381f2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:00.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f983d846640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98380caa80 0x7f98381f2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:00.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f983d846640 1 -- 172.21.15.70:0/2982081176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340094d0 con 0x7f98381605f0 2023-10-08T05:55:00.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f983d846640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98381605f0 0x7f98380c32b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f983000c700 tx=0x7f983000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:00.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f982dffb640 1 -- 172.21.15.70:0/2982081176 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983000aa50 con 0x7f98381605f0 2023-10-08T05:55:00.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f982dffb640 1 -- 172.21.15.70:0/2982081176 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983000abb0 con 0x7f98381605f0 2023-10-08T05:55:00.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f983f2a9640 1 -- 172.21.15.70:0/2982081176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98381f2f10 con 0x7f98381605f0 2023-10-08T05:55:00.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.903+0000 7f982dffb640 1 -- 172.21.15.70:0/2982081176 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983000ce10 con 0x7f98381605f0 2023-10-08T05:55:00.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.904+0000 7f983f2a9640 1 -- 172.21.15.70:0/2982081176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98380b17d0 con 0x7f98381605f0 2023-10-08T05:55:00.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.905+0000 7f983f2a9640 1 -- 172.21.15.70:0/2982081176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98380b19f0 con 0x7f98381605f0 2023-10-08T05:55:00.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.906+0000 7f982dffb640 1 -- 172.21.15.70:0/2982081176 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 50) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f9830012420 con 0x7f98381605f0 2023-10-08T05:55:00.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.906+0000 7f982dffb640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f981c06b070 0x7f981c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:00.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.907+0000 7f982dffb640 1 -- 172.21.15.70:0/2982081176 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f9830089f40 con 0x7f98381605f0 2023-10-08T05:55:00.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.907+0000 7f983d045640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f981c06b070 0x7f981c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:00.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.908+0000 7f983d045640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f981c06b070 0x7f981c06d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9834036000 tx=0x7f983403c040 comp rx=0 tx=0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:00.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:00.911+0000 7f982dffb640 1 -- 172.21.15.70:0/2982081176 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9830054990 con 0x7f98381605f0 2023-10-08T05:55:01.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.122+0000 7f983f2a9640 1 -- 172.21.15.70:0/2982081176 --> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98380cccd0 con 0x7f981c06b070 2023-10-08T05:55:01.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.139+0000 7f982dffb640 1 -- 172.21.15.70:0/2982081176 <== mgr.6280 v2:172.21.15.181:6832/1714252273 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f9834031a90 con 0x7f981c06b070 2023-10-08T05:55:01.139 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:01.139 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:54:00", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-08T05:55:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 -- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f981c06b070 msgr2=0x7f981c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f981c06b070 0x7f981c06d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9834036000 tx=0x7f983403c040 comp rx=0 tx=0).stop 2023-10-08T05:55:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 -- 172.21.15.70:0/2982081176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98381605f0 msgr2=0x7f98380c32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98381605f0 0x7f98380c32b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f983000c700 tx=0x7f983000cbc0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 -- 172.21.15.70:0/2982081176 shutdown_connections 2023-10-08T05:55:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f981c06b070 0x7f981c06d530 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98381605f0 0x7f98380c32b0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9838160210 0x7f9838154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 --2- 172.21.15.70:0/2982081176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98380caa80 0x7f98381f2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 -- 172.21.15.70:0/2982081176 >> 172.21.15.70:0/2982081176 conn(0x7f98380cebd0 msgr2=0x7f98380ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 -- 172.21.15.70:0/2982081176 shutdown_connections 2023-10-08T05:55:01.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.141+0000 7f983f2a9640 1 -- 172.21.15.70:0/2982081176 wait complete. 2023-10-08T05:55:01.161 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:54:00", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-08T05:55:01.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /bad-path 2023-10-08T05:55:01.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.378+0000 7fe015b90640 1 Processor -- start 2023-10-08T05:55:01.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.378+0000 7fe015b90640 1 -- start start 2023-10-08T05:55:01.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.379+0000 7fe015b90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe01006dfc0 0x7fe010070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:01.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.379+0000 7fe015b90640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0100b1cc0 con 0x7fe0100afbe0 2023-10-08T05:55:01.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.379+0000 7fe015b90640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0100b2620 con 0x7fe010071070 2023-10-08T05:55:01.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.379+0000 7fe015b90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100b2060 con 0x7fe01006dfc0 2023-10-08T05:55:01.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.380+0000 7fe00f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe0100afbe0 0x7fe0100b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46932/0 (socket says 172.21.15.70:46932) 2023-10-08T05:55:01.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.380+0000 7fe00f7fe640 1 -- 172.21.15.70:0/1565847198 learned_addr learned my addr 172.21.15.70:0/1565847198 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:01.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.380+0000 7fe00effd640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe01006dfc0 0x7fe010070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:01.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.380+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591321277 0 0) 0x7fe0100b1cc0 con 0x7fe0100afbe0 2023-10-08T05:55:01.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.380+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdff8003520 con 0x7fe0100afbe0 2023-10-08T05:55:01.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.380+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1647516195 0 0) 0x7fdff8003520 con 0x7fe0100afbe0 2023-10-08T05:55:01.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.380+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe01006dfc0 msgr2=0x7fe010070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.380+0000 7fe00e7fc640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe01006dfc0 0x7fe010070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.380+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 >> v1:172.21.15.181:6789/0 conn(0x7fe010071070 legacy=0x7fe0100cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100b2340 con 0x7fe0100afbe0 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe000001670 con 0x7fe0100afbe0 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 >> v1:172.21.15.70:6789/0 conn(0x7fe0100afbe0 legacy=0x7fe0100b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00e7fc640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe01006dfc0 0x7fdff80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00e7fc640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 0x7fdff80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00e7fc640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff8004d00 0x7fdff8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100b2060 con 0x7fdff8003fd0 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff800bb40 con 0x7fdff8004d00 2023-10-08T05:55:01.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff800bd00 con 0x7fe01006dfc0 2023-10-08T05:55:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00effd640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 0x7fdff80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00f7fe640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe01006dfc0 0x7fdff80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00ffff640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff8004d00 0x7fdff8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00effd640 1 -- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe01006dfc0 msgr2=0x7fdff80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00effd640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe01006dfc0 0x7fdff80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00effd640 1 -- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff8004d00 msgr2=0x7fdff8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00effd640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff8004d00 0x7fdff8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.381+0000 7fe00effd640 1 -- 172.21.15.70:0/1565847198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff8003520 con 0x7fdff8003fd0 2023-10-08T05:55:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.382+0000 7fe00f7fe640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe01006dfc0 0x7fdff80039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.382+0000 7fe00ffff640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdff8004d00 0x7fdff8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.383+0000 7fe00effd640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 0x7fdff80047c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe00400ba10 tx=0x7fdff80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb23eb99fa11aff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.383+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004032630 con 0x7fdff8003fd0 2023-10-08T05:55:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.383+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe004032790 con 0x7fdff8003fd0 2023-10-08T05:55:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.383+0000 7fe00e7fc640 1 -- 172.21.15.70:0/1565847198 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe004032b20 con 0x7fdff8003fd0 2023-10-08T05:55:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.384+0000 7fe015b90640 1 -- 172.21.15.70:0/1565847198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 msgr2=0x7fdff80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.384+0000 7fe015b90640 1 --2- 172.21.15.70:0/1565847198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 0x7fdff80047c0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe00400ba10 tx=0x7fdff80057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.384+0000 7fe00ffff640 1 -- 172.21.15.70:0/1565847198 reap_dead start 2023-10-08T05:55:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.384+0000 7fe015b90640 1 -- 172.21.15.70:0/1565847198 shutdown_connections 2023-10-08T05:55:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.384+0000 7fe015b90640 1 -- 172.21.15.70:0/1565847198 >> 172.21.15.70:0/1565847198 conn(0x7fe01006a5d0 msgr2=0x7fe0100b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:01.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.384+0000 7fe015b90640 1 -- 172.21.15.70:0/1565847198 shutdown_connections 2023-10-08T05:55:01.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.384+0000 7fe015b90640 1 -- 172.21.15.70:0/1565847198 wait complete. 2023-10-08T05:55:01.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.386+0000 7fe015b90640 1 Processor -- start 2023-10-08T05:55:01.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.386+0000 7fe015b90640 1 -- start start 2023-10-08T05:55:01.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe015b90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 0x7fe0100c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:01.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe015b90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0100c4f30 0x7fe0101f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:01.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe015b90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0100cd300 0x7fe010159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe015b90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe010158590 con 0x7fdff8003fd0 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe015b90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0101f26e0 con 0x7fe0100c4f30 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe015b90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0101f29c0 con 0x7fe0100cd300 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe00f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 0x7fe0100c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe00f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 0x7fe0100c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42622/0 (socket says 172.21.15.70:42622) 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe00f7fe640 1 -- 172.21.15.70:0/2237182644 learned_addr learned my addr 172.21.15.70:0/2237182644 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe00f7fe640 1 -- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0100cd300 msgr2=0x7fe010159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe00f7fe640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0100cd300 0x7fe010159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe00f7fe640 1 -- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0100c4f30 msgr2=0x7fe0101f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe00f7fe640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0100c4f30 0x7fe0101f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.387+0000 7fe00f7fe640 1 -- 172.21.15.70:0/2237182644 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe010158830 con 0x7fdff8003fd0 2023-10-08T05:55:01.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.388+0000 7fe00f7fe640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 0x7fe0100c49f0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fe00000d8b0 tx=0x7fe00000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:01.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.388+0000 7fdff7fff640 1 -- 172.21.15.70:0/2237182644 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0000074e0 con 0x7fdff8003fd0 2023-10-08T05:55:01.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.388+0000 7fdff7fff640 1 -- 172.21.15.70:0/2237182644 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe000007640 con 0x7fdff8003fd0 2023-10-08T05:55:01.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.388+0000 7fdff7fff640 1 -- 172.21.15.70:0/2237182644 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0000049e0 con 0x7fdff8003fd0 2023-10-08T05:55:01.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.388+0000 7fe015b90640 1 -- 172.21.15.70:0/2237182644 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe010158b10 con 0x7fdff8003fd0 2023-10-08T05:55:01.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.388+0000 7fe015b90640 1 -- 172.21.15.70:0/2237182644 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0101f2180 con 0x7fdff8003fd0 2023-10-08T05:55:01.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.389+0000 7fe015b90640 1 -- 172.21.15.70:0/2237182644 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01004e160 con 0x7fdff8003fd0 2023-10-08T05:55:01.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.390+0000 7fdff7fff640 1 -- 172.21.15.70:0/2237182644 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 50) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fe000004b40 con 0x7fdff8003fd0 2023-10-08T05:55:01.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.391+0000 7fdff7fff640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fdffc06b070 0x7fdffc06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:01.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.391+0000 7fdff7fff640 1 -- 172.21.15.70:0/2237182644 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fe0000899f0 con 0x7fdff8003fd0 2023-10-08T05:55:01.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.392+0000 7fe00effd640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fdffc06b070 0x7fdffc06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:01.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.393+0000 7fe00effd640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fdffc06b070 0x7fdffc06d530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe01006e410 tx=0x7fe004032000 comp rx=0 tx=0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:01.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.396+0000 7fdff7fff640 1 -- 172.21.15.70:0/2237182644 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe000054510 con 0x7fdff8003fd0 2023-10-08T05:55:01.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.579+0000 7fdff7fff640 1 -- 172.21.15.70:0/2237182644 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 51) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe0000548e0 con 0x7fdff8003fd0 2023-10-08T05:55:01.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.611+0000 7fe015b90640 1 -- 172.21.15.70:0/2237182644 --> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7fe010070630 con 0x7fdffc06b070 2023-10-08T05:55:01.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.774+0000 7fdff7fff640 1 -- 172.21.15.70:0/2237182644 <== mgr.6280 v2:172.21.15.181:6832/1714252273 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7fe004032550 con 0x7fdffc06b070 2023-10-08T05:55:01.775 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /bad-path 2023-10-08T05:55:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.778+0000 7fe015b90640 1 -- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fdffc06b070 msgr2=0x7fdffc06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.778+0000 7fe015b90640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fdffc06b070 0x7fdffc06d530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe01006e410 tx=0x7fe004032000 comp rx=0 tx=0).stop 2023-10-08T05:55:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.778+0000 7fe015b90640 1 -- 172.21.15.70:0/2237182644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 msgr2=0x7fe0100c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.778+0000 7fe015b90640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 0x7fe0100c49f0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fe00000d8b0 tx=0x7fe00000dd70 comp rx=0 tx=0).stop 2023-10-08T05:55:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.778+0000 7fe015b90640 1 -- 172.21.15.70:0/2237182644 shutdown_connections 2023-10-08T05:55:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.778+0000 7fe015b90640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fdffc06b070 0x7fdffc06d530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.778+0000 7fe015b90640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0100cd300 0x7fe010159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.778+0000 7fe015b90640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0100c4f30 0x7fe0101f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.778+0000 7fe015b90640 1 --2- 172.21.15.70:0/2237182644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdff8003fd0 0x7fe0100c49f0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:01.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.778+0000 7fe015b90640 1 -- 172.21.15.70:0/2237182644 >> 172.21.15.70:0/2237182644 conn(0x7fe01006a5d0 msgr2=0x7fe0100c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:01.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.779+0000 7fe015b90640 1 -- 172.21.15.70:0/2237182644 shutdown_connections 2023-10-08T05:55:01.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:01.779+0000 7fe015b90640 1 -- 172.21.15.70:0/2237182644 wait complete. 2023-10-08T05:55:01.799 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-10-08T05:55:01.799 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T05:55:02.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.013+0000 7f704b07c640 1 Processor -- start 2023-10-08T05:55:02.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.013+0000 7f704b07c640 1 -- start start 2023-10-08T05:55:02.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.015+0000 7f704b07c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f704415aad0 0x7f70440b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:02.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.015+0000 7f704b07c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70440c82b0 0x7f70440c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:02.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.015+0000 7f704b07c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f704406cd00 0x7f704406d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:02.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.015+0000 7f704b07c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704406d6d0 con 0x7f704406cd00 2023-10-08T05:55:02.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.015+0000 7f704b07c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704415bf30 con 0x7f704415aad0 2023-10-08T05:55:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.015+0000 7f704b07c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704415b320 con 0x7f70440c82b0 2023-10-08T05:55:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.015+0000 7f7049619640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f704406cd00 0x7f704406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.015+0000 7f7049619640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f704406cd00 0x7f704406d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42626/0 (socket says 172.21.15.70:42626) 2023-10-08T05:55:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.015+0000 7f7049619640 1 -- 172.21.15.70:0/2398148114 learned_addr learned my addr 172.21.15.70:0/2398148114 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.015+0000 7f7043fff640 1 --2- 172.21.15.70:0/2398148114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70440c82b0 0x7f70440c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.016+0000 7f7048e18640 1 --2- 172.21.15.70:0/2398148114 >> v2:172.21.15.181:3300/0 conn(0x7f704415aad0 0x7f70440b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.016+0000 7f7049619640 1 -- 172.21.15.70:0/2398148114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70440c82b0 msgr2=0x7f70440c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.016+0000 7f7049619640 1 --2- 172.21.15.70:0/2398148114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70440c82b0 0x7f70440c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.016+0000 7f7049619640 1 -- 172.21.15.70:0/2398148114 >> v2:172.21.15.181:3300/0 conn(0x7f704415aad0 msgr2=0x7f70440b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:02.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.016+0000 7f7049619640 1 --2- 172.21.15.70:0/2398148114 >> v2:172.21.15.181:3300/0 conn(0x7f704415aad0 0x7f70440b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.016+0000 7f7049619640 1 -- 172.21.15.70:0/2398148114 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704415b600 con 0x7f704406cd00 2023-10-08T05:55:02.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.016+0000 7f7048e18640 1 --2- 172.21.15.70:0/2398148114 >> v2:172.21.15.181:3300/0 conn(0x7f704415aad0 0x7f70440b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:02.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.017+0000 7f7049619640 1 --2- 172.21.15.70:0/2398148114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704406cd00 0x7f704406d190 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f704406e6a0 tx=0x7f703c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a304b2ce8956b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:02.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.017+0000 7f70437fe640 1 -- 172.21.15.70:0/2398148114 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c02ca80 con 0x7f704406cd00 2023-10-08T05:55:02.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.017+0000 7f70437fe640 1 -- 172.21.15.70:0/2398148114 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c02cbe0 con 0x7f704406cd00 2023-10-08T05:55:02.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.017+0000 7f70437fe640 1 -- 172.21.15.70:0/2398148114 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c0343f0 con 0x7f704406cd00 2023-10-08T05:55:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.018+0000 7f704b07c640 1 -- 172.21.15.70:0/2398148114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704406cd00 msgr2=0x7f704406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.018+0000 7f704b07c640 1 --2- 172.21.15.70:0/2398148114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704406cd00 0x7f704406d190 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f704406e6a0 tx=0x7f703c02bee0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.018+0000 7f704b07c640 1 -- 172.21.15.70:0/2398148114 shutdown_connections 2023-10-08T05:55:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.018+0000 7f704b07c640 1 --2- 172.21.15.70:0/2398148114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704406cd00 0x7f704406d190 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.018+0000 7f704b07c640 1 --2- 172.21.15.70:0/2398148114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70440c82b0 0x7f70440c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.018+0000 7f704b07c640 1 --2- 172.21.15.70:0/2398148114 >> v2:172.21.15.181:3300/0 conn(0x7f704415aad0 0x7f70440b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.018+0000 7f704b07c640 1 -- 172.21.15.70:0/2398148114 >> 172.21.15.70:0/2398148114 conn(0x7f7044069d50 msgr2=0x7f704406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.018+0000 7f704b07c640 1 -- 172.21.15.70:0/2398148114 shutdown_connections 2023-10-08T05:55:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.018+0000 7f704b07c640 1 -- 172.21.15.70:0/2398148114 wait complete. 2023-10-08T05:55:02.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.020+0000 7f704b07c640 1 Processor -- start 2023-10-08T05:55:02.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.020+0000 7f704b07c640 1 -- start start 2023-10-08T05:55:02.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.021+0000 7f704b07c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704406cd00 0x7f704414a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:02.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.021+0000 7f704b07c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70440c82b0 0x7f704414be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:02.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.021+0000 7f704b07c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704415aad0 0x7f704414c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:02.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.021+0000 7f704b07c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704414d660 con 0x7f704415aad0 2023-10-08T05:55:02.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.021+0000 7f704b07c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704414a7a0 con 0x7f70440c82b0 2023-10-08T05:55:02.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.021+0000 7f704b07c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704414da80 con 0x7f704406cd00 2023-10-08T05:55:02.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.021+0000 7f7049619640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704415aad0 0x7f704414c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:02.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.021+0000 7f7049619640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704415aad0 0x7f704414c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42640/0 (socket says 172.21.15.70:42640) 2023-10-08T05:55:02.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.021+0000 7f7049619640 1 -- 172.21.15.70:0/1753955866 learned_addr learned my addr 172.21.15.70:0/1753955866 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:02.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.021+0000 7f7048e18640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704406cd00 0x7f704414a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:02.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.022+0000 7f7049619640 1 -- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704406cd00 msgr2=0x7f704414a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:02.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.022+0000 7f7049619640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704406cd00 0x7f704414a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.022+0000 7f7049619640 1 -- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70440c82b0 msgr2=0x7f704414be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:02.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.022+0000 7f7049619640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70440c82b0 0x7f704414be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.022+0000 7f7049619640 1 -- 172.21.15.70:0/1753955866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c009b30 con 0x7f704415aad0 2023-10-08T05:55:02.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.022+0000 7f7049619640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704415aad0 0x7f704414c3d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f703c036040 tx=0x7f703c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:02.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.022+0000 7f70417fa640 1 -- 172.21.15.70:0/1753955866 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c009510 con 0x7f704415aad0 2023-10-08T05:55:02.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.022+0000 7f70417fa640 1 -- 172.21.15.70:0/1753955866 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c03e450 con 0x7f704415aad0 2023-10-08T05:55:02.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.022+0000 7f704b07c640 1 -- 172.21.15.70:0/1753955866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704414c9e0 con 0x7f704415aad0 2023-10-08T05:55:02.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.022+0000 7f70417fa640 1 -- 172.21.15.70:0/1753955866 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c03e7e0 con 0x7f704415aad0 2023-10-08T05:55:02.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.023+0000 7f704b07c640 1 -- 172.21.15.70:0/1753955866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704414cfa0 con 0x7f704415aad0 2023-10-08T05:55:02.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.024+0000 7f704b07c640 1 -- 172.21.15.70:0/1753955866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704414d7a0 con 0x7f704415aad0 2023-10-08T05:55:02.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.025+0000 7f70417fa640 1 -- 172.21.15.70:0/1753955866 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 51) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f703c02cd50 con 0x7f704415aad0 2023-10-08T05:55:02.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.026+0000 7f70417fa640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f702806b100 0x7f702806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:02.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.026+0000 7f70417fa640 1 -- 172.21.15.70:0/1753955866 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f703c0ac9b0 con 0x7f704415aad0 2023-10-08T05:55:02.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.026+0000 7f7048e18640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f702806b100 0x7f702806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:02.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.028+0000 7f7048e18640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f702806b100 0x7f702806d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f703800a950 tx=0x7f7038008040 comp rx=0 tx=0).ready entity=mgr.6280 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:02.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.031+0000 7f70417fa640 1 -- 172.21.15.70:0/1753955866 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f704414d7a0 con 0x7f704415aad0 2023-10-08T05:55:02.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.268+0000 7f704b07c640 1 -- 172.21.15.70:0/1753955866 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f704414d7a0 con 0x7f704415aad0 2023-10-08T05:55:02.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.579+0000 7f70417fa640 1 -- 172.21.15.70:0/1753955866 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v52) v1 ==== 93+0+0 (secure 0 0 0) 0x7f704414d7a0 con 0x7f704415aad0 2023-10-08T05:55:02.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.582+0000 7f704b07c640 1 -- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f702806b100 msgr2=0x7f702806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.582+0000 7f704b07c640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f702806b100 0x7f702806d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f703800a950 tx=0x7f7038008040 comp rx=0 tx=0).stop 2023-10-08T05:55:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.582+0000 7f704b07c640 1 -- 172.21.15.70:0/1753955866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704415aad0 msgr2=0x7f704414c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.582+0000 7f704b07c640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704415aad0 0x7f704414c3d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f703c036040 tx=0x7f703c004a40 comp rx=0 tx=0).stop 2023-10-08T05:55:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.582+0000 7f704b07c640 1 -- 172.21.15.70:0/1753955866 shutdown_connections 2023-10-08T05:55:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.583+0000 7f704b07c640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f702806b100 0x7f702806d5c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.583+0000 7f704b07c640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704415aad0 0x7f704414c3d0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.583+0000 7f704b07c640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70440c82b0 0x7f704414be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.583+0000 7f704b07c640 1 --2- 172.21.15.70:0/1753955866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704406cd00 0x7f704414a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.583+0000 7f704b07c640 1 -- 172.21.15.70:0/1753955866 >> 172.21.15.70:0/1753955866 conn(0x7f7044069d50 msgr2=0x7f704406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.583+0000 7f704b07c640 1 -- 172.21.15.70:0/1753955866 shutdown_connections 2023-10-08T05:55:02.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:02.583+0000 7f704b07c640 1 -- 172.21.15.70:0/1753955866 wait complete. 2023-10-08T05:55:02.607 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:55:02.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:55:02.718 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:55:02.817 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:55:02.817 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:55:02.817 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:55:02.870 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:55:02.870 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:55:02.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:55:03.096 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:03.095+0000 7fa25dc3f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:55:03.118 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:03.117+0000 7f32afc2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:55:03.151 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:03.151+0000 7fa25dc3f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:55:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.163+0000 7f656359e640 1 Processor -- start 2023-10-08T05:55:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.163+0000 7f656359e640 1 -- start start 2023-10-08T05:55:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.164+0000 7f656359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656415d1a0 0x7f65640b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.164+0000 7f656359e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65640b6490 0x7f65640cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.164+0000 7f656359e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65640d0fd0 con 0x7f656415cdc0 2023-10-08T05:55:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.164+0000 7f656359e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6564161410 con 0x7f65640b6490 2023-10-08T05:55:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.164+0000 7f656359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65641616f0 con 0x7f656415d1a0 2023-10-08T05:55:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f656259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f656415cdc0 0x7f6564160bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46962/0 (socket says 172.21.15.70:46962) 2023-10-08T05:55:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f656259c640 1 -- 172.21.15.70:0/1996661776 learned_addr learned my addr 172.21.15.70:0/1996661776 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6562d9d640 1 --2- 172.21.15.70:0/1996661776 >> v2:172.21.15.181:3300/0 conn(0x7f65640b6490 0x7f65640cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f655bfff640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656415d1a0 0x7f65640b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288975095 0 0) 0x7f65640d0fd0 con 0x7f656415cdc0 2023-10-08T05:55:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6550003520 con 0x7f656415cdc0 2023-10-08T05:55:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3398916731 0 0) 0x7f6550003520 con 0x7f656415cdc0 2023-10-08T05:55:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656415d1a0 msgr2=0x7f65640b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656415d1a0 0x7f65640b5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 >> v2:172.21.15.181:3300/0 conn(0x7f65640b6490 msgr2=0x7f65640cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 --2- 172.21.15.70:0/1996661776 >> v2:172.21.15.181:3300/0 conn(0x7f65640b6490 0x7f65640cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65641619d0 con 0x7f656415cdc0 2023-10-08T05:55:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6554001670 con 0x7f656415cdc0 2023-10-08T05:55:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6562d9d640 1 --2- 172.21.15.70:0/1996661776 >> v2:172.21.15.181:3300/0 conn(0x7f65640b6490 0x7f65640cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:55:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 >> v1:172.21.15.70:6789/0 conn(0x7f656415cdc0 legacy=0x7f6564160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.165+0000 7f6561d9b640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656415d1a0 0x7f6550003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6561d9b640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6550003fa0 0x7f65500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6561d9b640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6550004d00 0x7f6550005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6564161410 con 0x7f6550004d00 2023-10-08T05:55:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65641616f0 con 0x7f6550003fa0 2023-10-08T05:55:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655000ba80 con 0x7f656415d1a0 2023-10-08T05:55:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6562d9d640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6550004d00 0x7f6550005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f656259c640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656415d1a0 0x7f6550003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f655bfff640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6550003fa0 0x7f65500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6562d9d640 1 -- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656415d1a0 msgr2=0x7f6550003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6562d9d640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f656415d1a0 0x7f6550003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6562d9d640 1 -- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6550003fa0 msgr2=0x7f65500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6562d9d640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6550003fa0 0x7f65500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.166+0000 7f6562d9d640 1 -- 172.21.15.70:0/1996661776 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6550003520 con 0x7f6550004d00 2023-10-08T05:55:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.167+0000 7f6562d9d640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6550004d00 0x7f6550005180 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f655c0073b0 tx=0x7f65500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d60e46232e4b4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.168+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c02cd90 con 0x7f6550004d00 2023-10-08T05:55:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.168+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c003990 con 0x7f6550004d00 2023-10-08T05:55:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.168+0000 7f6561d9b640 1 -- 172.21.15.70:0/1996661776 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c003d40 con 0x7f6550004d00 2023-10-08T05:55:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.168+0000 7f656359e640 1 -- 172.21.15.70:0/1996661776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6550004d00 msgr2=0x7f6550005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.168+0000 7f656359e640 1 --2- 172.21.15.70:0/1996661776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6550004d00 0x7f6550005180 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f655c0073b0 tx=0x7f65500057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.168+0000 7f6562d9d640 1 -- 172.21.15.70:0/1996661776 reap_dead start 2023-10-08T05:55:03.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.168+0000 7f656359e640 1 -- 172.21.15.70:0/1996661776 shutdown_connections 2023-10-08T05:55:03.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.168+0000 7f656359e640 1 -- 172.21.15.70:0/1996661776 >> 172.21.15.70:0/1996661776 conn(0x7f6564071b10 msgr2=0x7f656406b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:03.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.168+0000 7f656359e640 1 -- 172.21.15.70:0/1996661776 shutdown_connections 2023-10-08T05:55:03.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.168+0000 7f656359e640 1 -- 172.21.15.70:0/1996661776 wait complete. 2023-10-08T05:55:03.174 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:03.170+0000 7f32afc2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:55:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.170+0000 7f656359e640 1 Processor -- start 2023-10-08T05:55:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.170+0000 7f656359e640 1 -- start start 2023-10-08T05:55:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.171+0000 7f656359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6550004d00 0x7f656415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.171+0000 7f656359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65640b4d00 0x7f65640c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.171+0000 7f656359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65640b5180 0x7f65640cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.171+0000 7f656359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640b3c60 con 0x7f65640b5180 2023-10-08T05:55:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.171+0000 7f656359e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65640b3f40 con 0x7f65640b4d00 2023-10-08T05:55:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.171+0000 7f656359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65640b4220 con 0x7f6550004d00 2023-10-08T05:55:03.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f6562d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65640b5180 0x7f65640cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f6562d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65640b5180 0x7f65640cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42676/0 (socket says 172.21.15.70:42676) 2023-10-08T05:55:03.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f6562d9d640 1 -- 172.21.15.70:0/552609220 learned_addr learned my addr 172.21.15.70:0/552609220 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:03.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f655bfff640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65640b4d00 0x7f65640c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f656259c640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6550004d00 0x7f656415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f6562d9d640 1 -- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6550004d00 msgr2=0x7f656415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f6562d9d640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6550004d00 0x7f656415a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f6562d9d640 1 -- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65640b4d00 msgr2=0x7f65640c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f6562d9d640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65640b4d00 0x7f65640c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f6562d9d640 1 -- 172.21.15.70:0/552609220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65640b4500 con 0x7f65640b5180 2023-10-08T05:55:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.172+0000 7f6562d9d640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65640b5180 0x7f65640cf2e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f655c0073b0 tx=0x7f655c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:03.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.173+0000 7f655affd640 1 -- 172.21.15.70:0/552609220 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c004500 con 0x7f65640b5180 2023-10-08T05:55:03.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.173+0000 7f655affd640 1 -- 172.21.15.70:0/552609220 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c004660 con 0x7f65640b5180 2023-10-08T05:55:03.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.173+0000 7f655affd640 1 -- 172.21.15.70:0/552609220 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c040470 con 0x7f65640b5180 2023-10-08T05:55:03.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.173+0000 7f656359e640 1 -- 172.21.15.70:0/552609220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65640b47e0 con 0x7f65640b5180 2023-10-08T05:55:03.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.173+0000 7f656359e640 1 -- 172.21.15.70:0/552609220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65640c6da0 con 0x7f65640b5180 2023-10-08T05:55:03.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.174+0000 7f656359e640 1 -- 172.21.15.70:0/552609220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65640b49d0 con 0x7f65640b5180 2023-10-08T05:55:03.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.175+0000 7f655affd640 1 -- 172.21.15.70:0/552609220 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 52) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f655c0405f0 con 0x7f65640b5180 2023-10-08T05:55:03.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.176+0000 7f655affd640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f654406b0b0 0x7f654406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.176+0000 7f655affd640 1 -- 172.21.15.70:0/552609220 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f655c0ac4c0 con 0x7f65640b5180 2023-10-08T05:55:03.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.176+0000 7f656259c640 1 -- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f654406b0b0 msgr2=0x7f654406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:03.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.176+0000 7f656259c640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f654406b0b0 0x7f654406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:03.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.180+0000 7f655affd640 1 -- 172.21.15.70:0/552609220 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f655c076ed0 con 0x7f65640b5180 2023-10-08T05:55:03.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.371+0000 7f656359e640 1 -- 172.21.15.70:0/552609220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f656404e160 con 0x7f65640b5180 2023-10-08T05:55:03.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.373+0000 7f655affd640 1 -- 172.21.15.70:0/552609220 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17487 (secure 0 0 0) 0x7f655c07a4f0 con 0x7f65640b5180 2023-10-08T05:55:03.374 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:03.374 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":121,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:00.541234+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:42.052166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:43.161991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/1906631919":"2023-10-09T05:53:58.959064+0000","172.21.15.181:6832/1906631919":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3209826772":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/1283682257":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/2590863547":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3077797034":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3436033120":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/168987985":"2023-10-09T05:53:58.959064+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:55:03.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.375+0000 7f656359e640 1 -- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f654406b0b0 msgr2=0x7f654406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:03.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.375+0000 7f656359e640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f654406b0b0 0x7f654406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.375+0000 7f656359e640 1 -- 172.21.15.70:0/552609220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65640b5180 msgr2=0x7f65640cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.375+0000 7f656359e640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65640b5180 0x7f65640cf2e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f655c0073b0 tx=0x7f655c004ab0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.376+0000 7f656359e640 1 -- 172.21.15.70:0/552609220 shutdown_connections 2023-10-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.376+0000 7f656359e640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f654406b0b0 0x7f654406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.376+0000 7f656359e640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65640b5180 0x7f65640cf2e0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.376+0000 7f656359e640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65640b4d00 0x7f65640c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.376+0000 7f656359e640 1 --2- 172.21.15.70:0/552609220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6550004d00 0x7f656415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.376+0000 7f656359e640 1 -- 172.21.15.70:0/552609220 >> 172.21.15.70:0/552609220 conn(0x7f6564071b10 msgr2=0x7f656406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.376+0000 7f656359e640 1 -- 172.21.15.70:0/552609220 shutdown_connections 2023-10-08T05:55:03.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.376+0000 7f656359e640 1 -- 172.21.15.70:0/552609220 wait complete. 2023-10-08T05:55:03.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:53:42.052166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:53:43.161991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:55:03.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:55:03.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.599+0000 7feeb31a9640 1 Processor -- start 2023-10-08T05:55:03.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.600+0000 7feeb31a9640 1 -- start start 2023-10-08T05:55:03.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.601+0000 7feeb31a9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feeac0cd6f0 0x7feeac071b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.602+0000 7feeb31a9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac0cf440 con 0x7feeac0cdad0 2023-10-08T05:55:03.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.602+0000 7feeb31a9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac0cf690 con 0x7feeac159ed0 2023-10-08T05:55:03.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.602+0000 7feeb31a9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac0cf970 con 0x7feeac0cd6f0 2023-10-08T05:55:03.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.602+0000 7feeabfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feeac0cdad0 0x7feeac1597c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41892/0 (socket says 172.21.15.70:41892) 2023-10-08T05:55:03.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.602+0000 7feeabfff640 1 -- 172.21.15.70:0/1621966365 learned_addr learned my addr 172.21.15.70:0/1621966365 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:03.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.602+0000 7feeb0f45640 1 --2- 172.21.15.70:0/1621966365 >> v2:172.21.15.181:3300/0 conn(0x7feeac0cd6f0 0x7feeac071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.602+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688146511 0 0) 0x7feeac0cf440 con 0x7feeac0cdad0 2023-10-08T05:55:03.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.602+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee9c003520 con 0x7feeac0cdad0 2023-10-08T05:55:03.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3740523771 0 0) 0x7fee9c003520 con 0x7feeac0cdad0 2023-10-08T05:55:03.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 >> v2:172.21.15.181:3300/0 conn(0x7feeac0cd6f0 msgr2=0x7feeac071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:03.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 --2- 172.21.15.70:0/1621966365 >> v2:172.21.15.181:3300/0 conn(0x7feeac0cd6f0 0x7feeac071b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 >> v1:172.21.15.181:6789/0 conn(0x7feeac159ed0 legacy=0x7feeac0ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0cfc50 con 0x7feeac0cdad0 2023-10-08T05:55:03.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fee98001670 con 0x7feeac0cdad0 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 >> v1:172.21.15.70:6789/0 conn(0x7feeac0cdad0 legacy=0x7feeac1597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 --2- 172.21.15.70:0/1621966365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee9c0039c0 0x7fee9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 --2- 172.21.15.70:0/1621966365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee9c0047c0 0x7fee9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 --2- 172.21.15.70:0/1621966365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7fee9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0cf970 con 0x7fee9c005180 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c00c080 con 0x7fee9c0047c0 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c00c240 con 0x7fee9c0039c0 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeb1746640 1 --2- 172.21.15.70:0/1621966365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7fee9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeb1746640 1 -- 172.21.15.70:0/1621966365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee9c0039c0 msgr2=0x7fee9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeabfff640 1 --2- 172.21.15.70:0/1621966365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee9c0047c0 0x7fee9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeb1746640 1 --2- 172.21.15.70:0/1621966365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee9c0039c0 0x7fee9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.603+0000 7feeb1746640 1 -- 172.21.15.70:0/1621966365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee9c0047c0 msgr2=0x7fee9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:03.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.604+0000 7feeb1746640 1 --2- 172.21.15.70:0/1621966365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee9c0047c0 0x7fee9c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.604+0000 7feeb1746640 1 -- 172.21.15.70:0/1621966365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c003520 con 0x7fee9c005180 2023-10-08T05:55:03.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.604+0000 7feeb1746640 1 -- 172.21.15.70:0/1621966365 reap_dead start 2023-10-08T05:55:03.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.605+0000 7feeb1746640 1 --2- 172.21.15.70:0/1621966365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7fee9c00b980 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7feea4001e50 tx=0x7fee9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35e51807cfb0569 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:03.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.605+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4003b70 con 0x7fee9c005180 2023-10-08T05:55:03.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.605+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea4003cf0 con 0x7fee9c005180 2023-10-08T05:55:03.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.605+0000 7feeab7fe640 1 -- 172.21.15.70:0/1621966365 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4005310 con 0x7fee9c005180 2023-10-08T05:55:03.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.606+0000 7feeb31a9640 1 -- 172.21.15.70:0/1621966365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 msgr2=0x7fee9c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.606+0000 7feeb31a9640 1 --2- 172.21.15.70:0/1621966365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7fee9c00b980 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7feea4001e50 tx=0x7fee9c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.606+0000 7feeb31a9640 1 -- 172.21.15.70:0/1621966365 shutdown_connections 2023-10-08T05:55:03.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.606+0000 7feeb31a9640 1 --2- 172.21.15.70:0/1621966365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7fee9c00b980 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.606+0000 7feeb31a9640 1 -- 172.21.15.70:0/1621966365 >> 172.21.15.70:0/1621966365 conn(0x7feeac06ef30 msgr2=0x7feeac06f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:03.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.606+0000 7feeb31a9640 1 -- 172.21.15.70:0/1621966365 shutdown_connections 2023-10-08T05:55:03.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.606+0000 7feeb31a9640 1 -- 172.21.15.70:0/1621966365 wait complete. 2023-10-08T05:55:03.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.608+0000 7feeb31a9640 1 Processor -- start 2023-10-08T05:55:03.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.608+0000 7feeb31a9640 1 -- start start 2023-10-08T05:55:03.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb31a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7feeac1f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb31a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0c6eb0 0x7feeac15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb31a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac0c7310 0x7feeac15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb31a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac1516a0 con 0x7fee9c005180 2023-10-08T05:55:03.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb31a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac1f3190 con 0x7feeac0c7310 2023-10-08T05:55:03.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb31a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac1f3410 con 0x7feeac0c6eb0 2023-10-08T05:55:03.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb0f45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7feeac1f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb0f45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7feeac1f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43714/0 (socket says 172.21.15.70:43714) 2023-10-08T05:55:03.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb0f45640 1 -- 172.21.15.70:0/556796651 learned_addr learned my addr 172.21.15.70:0/556796651 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb0f45640 1 -- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0c6eb0 msgr2=0x7feeac15a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb0f45640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0c6eb0 0x7feeac15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb1746640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac0c7310 0x7feeac15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb0f45640 1 -- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac0c7310 msgr2=0x7feeac15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb0f45640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac0c7310 0x7feeac15ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.609+0000 7feeb0f45640 1 -- 172.21.15.70:0/556796651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac1f36f0 con 0x7fee9c005180 2023-10-08T05:55:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.610+0000 7feeb0f45640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7feeac1f2c50 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7feea000e7a0 tx=0x7feea000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.610+0000 7feea97fa640 1 -- 172.21.15.70:0/556796651 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea000eeb0 con 0x7fee9c005180 2023-10-08T05:55:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.610+0000 7feea97fa640 1 -- 172.21.15.70:0/556796651 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea000c940 con 0x7fee9c005180 2023-10-08T05:55:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.610+0000 7feea97fa640 1 -- 172.21.15.70:0/556796651 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea000cd10 con 0x7fee9c005180 2023-10-08T05:55:03.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.610+0000 7feeb31a9640 1 -- 172.21.15.70:0/556796651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac1f39d0 con 0x7fee9c005180 2023-10-08T05:55:03.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.610+0000 7feeb31a9640 1 -- 172.21.15.70:0/556796651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac1f3f90 con 0x7fee9c005180 2023-10-08T05:55:03.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.611+0000 7feeb31a9640 1 -- 172.21.15.70:0/556796651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac04e160 con 0x7fee9c005180 2023-10-08T05:55:03.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.612+0000 7feea97fa640 1 -- 172.21.15.70:0/556796651 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 52) v1 ==== 89445+0+0 (secure 0 0 0) 0x7feea0013470 con 0x7fee9c005180 2023-10-08T05:55:03.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.613+0000 7feea97fa640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fee9006b0b0 0x7fee9006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:03.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.613+0000 7feea97fa640 1 -- 172.21.15.70:0/556796651 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7feea0089fa0 con 0x7fee9c005180 2023-10-08T05:55:03.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.613+0000 7feeabfff640 1 -- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fee9006b0b0 msgr2=0x7fee9006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:03.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.613+0000 7feeabfff640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fee9006b0b0 0x7fee9006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:03.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.617+0000 7feea97fa640 1 -- 172.21.15.70:0/556796651 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feea0054880 con 0x7fee9c005180 2023-10-08T05:55:03.620 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:03.619+0000 7f32afc2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:55:03.626 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:03.626+0000 7fa25dc3f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:55:03.672 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:03.671+0000 7f32afc2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:55:03.680 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:03.680+0000 7fa25dc3f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:55:03.725 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:03.724+0000 7f32afc2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:55:03.735 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:03.734+0000 7fa25dc3f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:55:03.787 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:03.786+0000 7f32afc2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:55:03.798 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:03.798+0000 7fa25dc3f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:55:03.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.808+0000 7feeb31a9640 1 -- 172.21.15.70:0/556796651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feeac0723f0 con 0x7fee9c005180 2023-10-08T05:55:03.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.809+0000 7feea97fa640 1 -- 172.21.15.70:0/556796651 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7feea00210a0 con 0x7fee9c005180 2023-10-08T05:55:03.809 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:55:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 -- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fee9006b0b0 msgr2=0x7fee9006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fee9006b0b0 0x7fee9006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 -- 172.21.15.70:0/556796651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 msgr2=0x7feeac1f2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7feeac1f2c50 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7feea000e7a0 tx=0x7feea000ec60 comp rx=0 tx=0).stop 2023-10-08T05:55:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 -- 172.21.15.70:0/556796651 shutdown_connections 2023-10-08T05:55:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fee9006b0b0 0x7fee9006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feeac0c7310 0x7feeac15ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feeac0c6eb0 0x7feeac15a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 --2- 172.21.15.70:0/556796651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee9c005180 0x7feeac1f2c50 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 -- 172.21.15.70:0/556796651 >> 172.21.15.70:0/556796651 conn(0x7feeac06ef30 msgr2=0x7feeac069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 -- 172.21.15.70:0/556796651 shutdown_connections 2023-10-08T05:55:03.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:03.811+0000 7feeb31a9640 1 -- 172.21.15.70:0/556796651 wait complete. 2023-10-08T05:55:03.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:55:03.835 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:03.834+0000 7f32afc2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:55:03.851 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:03.851+0000 7fa25dc3f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:55:03.946 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:03.945+0000 7f32afc2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:55:03.968 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:03.968+0000 7fa25dc3f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:55:04.009 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:04.008+0000 7f32afc2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:55:04.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.031+0000 7f3b230b4640 1 Processor -- start 2023-10-08T05:55:04.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.032+0000 7f3b230b4640 1 -- start start 2023-10-08T05:55:04.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.033+0000 7f3b230b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3b1c15cce0 0x7f3b1c0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.033+0000 7f3b230b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b1c15d0c0 0x7f3b1c0b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.033+0000 7f3b230b4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3b1c0b30d0 0x7f3b1c0b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.033+0000 7f3b230b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c06ce20 con 0x7f3b1c15cce0 2023-10-08T05:55:04.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.033+0000 7f3b230b4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c0cf850 con 0x7f3b1c0b30d0 2023-10-08T05:55:04.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.033+0000 7f3b230b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c06c060 con 0x7f3b1c15d0c0 2023-10-08T05:55:04.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.033+0000 7f3b20e50640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3b1c15cce0 0x7f3b1c0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.033+0000 7f3b20e50640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3b1c15cce0 0x7f3b1c0cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43724/0 (socket says 172.21.15.70:43724) 2023-10-08T05:55:04.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.033+0000 7f3b20e50640 1 -- 172.21.15.70:0/3004964756 learned_addr learned my addr 172.21.15.70:0/3004964756 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:04.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.034+0000 7f3b21651640 1 --2- 172.21.15.70:0/3004964756 >> v2:172.21.15.181:3300/0 conn(0x7f3b1c0b30d0 0x7f3b1c0b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.034+0000 7f3b1bfff640 1 --2- 172.21.15.70:0/3004964756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b1c15d0c0 0x7f3b1c0b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.034+0000 7f3b20e50640 1 -- 172.21.15.70:0/3004964756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b1c15d0c0 msgr2=0x7f3b1c0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:04.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.034+0000 7f3b20e50640 1 --2- 172.21.15.70:0/3004964756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b1c15d0c0 0x7f3b1c0b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.034+0000 7f3b20e50640 1 -- 172.21.15.70:0/3004964756 >> v2:172.21.15.181:3300/0 conn(0x7f3b1c0b30d0 msgr2=0x7f3b1c0b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:04.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.034+0000 7f3b20e50640 1 --2- 172.21.15.70:0/3004964756 >> v2:172.21.15.181:3300/0 conn(0x7f3b1c0b30d0 0x7f3b1c0b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.034+0000 7f3b20e50640 1 -- 172.21.15.70:0/3004964756 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c06c340 con 0x7f3b1c15cce0 2023-10-08T05:55:04.036 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:04.034+0000 7fa25dc3f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:55:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.035+0000 7f3b20e50640 1 --2- 172.21.15.70:0/3004964756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c15cce0 0x7f3b1c0cf000 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c06fa40 tx=0x7f3b1002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ed133d71626ea2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.035+0000 7f3b1b7fe640 1 -- 172.21.15.70:0/3004964756 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10031870 con 0x7f3b1c15cce0 2023-10-08T05:55:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.035+0000 7f3b1b7fe640 1 -- 172.21.15.70:0/3004964756 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b100319d0 con 0x7f3b1c15cce0 2023-10-08T05:55:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.035+0000 7f3b1b7fe640 1 -- 172.21.15.70:0/3004964756 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10031d60 con 0x7f3b1c15cce0 2023-10-08T05:55:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.036+0000 7f3b230b4640 1 -- 172.21.15.70:0/3004964756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c15cce0 msgr2=0x7f3b1c0cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.036+0000 7f3b230b4640 1 --2- 172.21.15.70:0/3004964756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c15cce0 0x7f3b1c0cf000 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c06fa40 tx=0x7f3b1002b910 comp rx=0 tx=0).stop 2023-10-08T05:55:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.036+0000 7f3b230b4640 1 -- 172.21.15.70:0/3004964756 shutdown_connections 2023-10-08T05:55:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.036+0000 7f3b230b4640 1 --2- 172.21.15.70:0/3004964756 >> v2:172.21.15.181:3300/0 conn(0x7f3b1c0b30d0 0x7f3b1c0b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.036+0000 7f3b230b4640 1 --2- 172.21.15.70:0/3004964756 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b1c15d0c0 0x7f3b1c0b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.036+0000 7f3b230b4640 1 --2- 172.21.15.70:0/3004964756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c15cce0 0x7f3b1c0cf000 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.036+0000 7f3b230b4640 1 -- 172.21.15.70:0/3004964756 >> 172.21.15.70:0/3004964756 conn(0x7f3b1c06e650 msgr2=0x7f3b1c06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.036+0000 7f3b230b4640 1 -- 172.21.15.70:0/3004964756 shutdown_connections 2023-10-08T05:55:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.036+0000 7f3b230b4640 1 -- 172.21.15.70:0/3004964756 wait complete. 2023-10-08T05:55:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.038+0000 7f3b230b4640 1 Processor -- start 2023-10-08T05:55:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.038+0000 7f3b230b4640 1 -- start start 2023-10-08T05:55:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b230b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b1c0b30d0 0x7f3b1c15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b230b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b1c15cce0 0x7f3b1c15fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b230b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c0c8970 0x7f3b1c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b230b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c160120 con 0x7f3b1c0c8970 2023-10-08T05:55:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b230b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c15e5c0 con 0x7f3b1c15cce0 2023-10-08T05:55:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b230b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c15e8a0 con 0x7f3b1c0b30d0 2023-10-08T05:55:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b21651640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c0c8970 0x7f3b1c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b21651640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c0c8970 0x7f3b1c0c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43728/0 (socket says 172.21.15.70:43728) 2023-10-08T05:55:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b21651640 1 -- 172.21.15.70:0/2857100001 learned_addr learned my addr 172.21.15.70:0/2857100001 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b1bfff640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b1c15cce0 0x7f3b1c15fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.039+0000 7f3b20e50640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b1c0b30d0 0x7f3b1c15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b21651640 1 -- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b1c0b30d0 msgr2=0x7f3b1c15dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b21651640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b1c0b30d0 0x7f3b1c15dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b21651640 1 -- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b1c15cce0 msgr2=0x7f3b1c15fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b21651640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b1c15cce0 0x7f3b1c15fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b21651640 1 -- 172.21.15.70:0/2857100001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b100094d0 con 0x7f3b1c0c8970 2023-10-08T05:55:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b21651640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c0c8970 0x7f3b1c0c8e00 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3b14007d10 tx=0x7f3b1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b197fa640 1 -- 172.21.15.70:0/2857100001 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14014420 con 0x7f3b1c0c8970 2023-10-08T05:55:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b197fa640 1 -- 172.21.15.70:0/2857100001 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b14014580 con 0x7f3b1c0c8970 2023-10-08T05:55:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b230b4640 1 -- 172.21.15.70:0/2857100001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c15eb80 con 0x7f3b1c0c8970 2023-10-08T05:55:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b197fa640 1 -- 172.21.15.70:0/2857100001 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14014930 con 0x7f3b1c0c8970 2023-10-08T05:55:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.040+0000 7f3b230b4640 1 -- 172.21.15.70:0/2857100001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c0c45a0 con 0x7f3b1c0c8970 2023-10-08T05:55:04.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.041+0000 7f3b230b4640 1 -- 172.21.15.70:0/2857100001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c04e160 con 0x7f3b1c0c8970 2023-10-08T05:55:04.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.042+0000 7f3b197fa640 1 -- 172.21.15.70:0/2857100001 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 52) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f3b14014a90 con 0x7f3b1c0c8970 2023-10-08T05:55:04.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.043+0000 7f3b197fa640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3b0006b0b0 0x7f3b0006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.043+0000 7f3b197fa640 1 -- 172.21.15.70:0/2857100001 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f3b1408a560 con 0x7f3b1c0c8970 2023-10-08T05:55:04.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.043+0000 7f3b20e50640 1 -- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3b0006b0b0 msgr2=0x7f3b0006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:04.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.043+0000 7f3b20e50640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3b0006b0b0 0x7f3b0006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:04.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.047+0000 7f3b197fa640 1 -- 172.21.15.70:0/2857100001 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b14055020 con 0x7f3b1c0c8970 2023-10-08T05:55:04.059 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:04.058+0000 7f32afc2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:55:04.087 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:04.087+0000 7fa25dc3f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:55:04.116 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:04.115+0000 7f32afc2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:55:04.146 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:04.146+0000 7fa25dc3f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:55:04.165 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:04.165+0000 7f32afc2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:55:04.198 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:04.198+0000 7fa25dc3f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:55:04.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.234+0000 7f3b230b4640 1 -- 172.21.15.70:0/2857100001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b1c0cb740 con 0x7f3b1c0c8970 2023-10-08T05:55:04.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.235+0000 7f3b197fa640 1 -- 172.21.15.70:0/2857100001 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b140098c0 con 0x7f3b1c0c8970 2023-10-08T05:55:04.236 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:55:04.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 -- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3b0006b0b0 msgr2=0x7f3b0006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:04.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3b0006b0b0 0x7f3b0006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 -- 172.21.15.70:0/2857100001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c0c8970 msgr2=0x7f3b1c0c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c0c8970 0x7f3b1c0c8e00 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3b14007d10 tx=0x7f3b1400e4b0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 -- 172.21.15.70:0/2857100001 shutdown_connections 2023-10-08T05:55:04.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3b0006b0b0 0x7f3b0006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b1c0c8970 0x7f3b1c0c8e00 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b1c15cce0 0x7f3b1c15fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 --2- 172.21.15.70:0/2857100001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b1c0b30d0 0x7f3b1c15dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 -- 172.21.15.70:0/2857100001 >> 172.21.15.70:0/2857100001 conn(0x7f3b1c06e650 msgr2=0x7f3b1c0cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:04.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 -- 172.21.15.70:0/2857100001 shutdown_connections 2023-10-08T05:55:04.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.239+0000 7f3b230b4640 1 -- 172.21.15.70:0/2857100001 wait complete. 2023-10-08T05:55:04.256 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:55:04.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.512+0000 7f67acb6c640 1 Processor -- start 2023-10-08T05:55:04.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.513+0000 7f67acb6c640 1 -- start start 2023-10-08T05:55:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.514+0000 7f67acb6c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67a806c3a0 0x7f67a80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.514+0000 7f67acb6c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a806db60 0x7f67a806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.514+0000 7f67acb6c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a815b310 con 0x7f67a815aa00 2023-10-08T05:55:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.514+0000 7f67acb6c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a80ca230 con 0x7f67a806c3a0 2023-10-08T05:55:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.514+0000 7f67acb6c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a806d290 con 0x7f67a806db60 2023-10-08T05:55:04.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.515+0000 7f679ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a806db60 0x7f67a806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.515+0000 7f67a659c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67a806c3a0 0x7f67a80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.515+0000 7f679ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a806db60 0x7f67a806c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:57248/0 (socket says 172.21.15.70:57248) 2023-10-08T05:55:04.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.515+0000 7f67a659c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f67a806c3a0 0x7f67a80c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:46864/0 (socket says 172.21.15.70:46864) 2023-10-08T05:55:04.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.515+0000 7f679ffff640 1 -- 172.21.15.70:0/1146007578 learned_addr learned my addr 172.21.15.70:0/1146007578 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:04.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.515+0000 7f679ffff640 1 -- 172.21.15.70:0/1146007578 >> v2:172.21.15.181:3300/0 conn(0x7f67a806c3a0 msgr2=0x7f67a80c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:04.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.515+0000 7f679ffff640 1 --2- 172.21.15.70:0/1146007578 >> v2:172.21.15.181:3300/0 conn(0x7f67a806c3a0 0x7f67a80c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.515+0000 7f679ffff640 1 -- 172.21.15.70:0/1146007578 >> v1:172.21.15.181:6789/0 conn(0x7f67a815aa00 legacy=0x7f67a80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.515+0000 7f679ffff640 1 -- 172.21.15.70:0/1146007578 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a80c9860 con 0x7f67a806db60 2023-10-08T05:55:04.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.517+0000 7f679ffff640 1 --2- 172.21.15.70:0/1146007578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a806db60 0x7f67a806c780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6790009870 tx=0x7f679002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7429655ac2d7e50 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:04.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.517+0000 7f67a5d9b640 1 -- 172.21.15.70:0/1146007578 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790030840 con 0x7f67a806db60 2023-10-08T05:55:04.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.517+0000 7f67a5d9b640 1 -- 172.21.15.70:0/1146007578 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67900309a0 con 0x7f67a806db60 2023-10-08T05:55:04.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.517+0000 7f67a5d9b640 1 -- 172.21.15.70:0/1146007578 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790030d30 con 0x7f67a806db60 2023-10-08T05:55:04.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.518+0000 7f67acb6c640 1 -- 172.21.15.70:0/1146007578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a806db60 msgr2=0x7f67a806c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.518+0000 7f67acb6c640 1 --2- 172.21.15.70:0/1146007578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a806db60 0x7f67a806c780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6790009870 tx=0x7f679002b980 comp rx=0 tx=0).stop 2023-10-08T05:55:04.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.518+0000 7f67acb6c640 1 -- 172.21.15.70:0/1146007578 shutdown_connections 2023-10-08T05:55:04.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.518+0000 7f67acb6c640 1 --2- 172.21.15.70:0/1146007578 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a806db60 0x7f67a806c780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.518+0000 7f67acb6c640 1 --2- 172.21.15.70:0/1146007578 >> v2:172.21.15.181:3300/0 conn(0x7f67a806c3a0 0x7f67a80c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.518+0000 7f67acb6c640 1 -- 172.21.15.70:0/1146007578 >> 172.21.15.70:0/1146007578 conn(0x7f67a806a5d0 msgr2=0x7f67a80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:04.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.518+0000 7f67acb6c640 1 -- 172.21.15.70:0/1146007578 shutdown_connections 2023-10-08T05:55:04.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.518+0000 7f67acb6c640 1 -- 172.21.15.70:0/1146007578 wait complete. 2023-10-08T05:55:04.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.520+0000 7f67acb6c640 1 Processor -- start 2023-10-08T05:55:04.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.520+0000 7f67acb6c640 1 -- start start 2023-10-08T05:55:04.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.521+0000 7f67acb6c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a806c3a0 0x7f67a80c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.521+0000 7f67acb6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a806db60 0x7f67a80c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.521+0000 7f67acb6c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815aa00 0x7f67a80c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.521+0000 7f67acb6c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a81585e0 con 0x7f67a806db60 2023-10-08T05:55:04.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.521+0000 7f67acb6c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80c7110 con 0x7f67a806c3a0 2023-10-08T05:55:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.521+0000 7f67acb6c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a80c6220 con 0x7f67a815aa00 2023-10-08T05:55:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.521+0000 7f679ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a806db60 0x7f67a80c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.521+0000 7f679ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a806db60 0x7f67a80c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43732/0 (socket says 172.21.15.70:43732) 2023-10-08T05:55:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.521+0000 7f679ffff640 1 -- 172.21.15.70:0/3835769430 learned_addr learned my addr 172.21.15.70:0/3835769430 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f67a6d9d640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815aa00 0x7f67a80c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f67a659c640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a806c3a0 0x7f67a80c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f679ffff640 1 -- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815aa00 msgr2=0x7f67a80c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f679ffff640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815aa00 0x7f67a80c3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f679ffff640 1 -- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a806c3a0 msgr2=0x7f67a80c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f679ffff640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a806c3a0 0x7f67a80c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f679ffff640 1 -- 172.21.15.70:0/3835769430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900094d0 con 0x7f67a806db60 2023-10-08T05:55:04.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f67a6d9d640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815aa00 0x7f67a80c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:04.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f679ffff640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a806db60 0x7f67a80c35b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f6790000c00 tx=0x7f6790004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:04.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f679effd640 1 -- 172.21.15.70:0/3835769430 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790039c90 con 0x7f67a806db60 2023-10-08T05:55:04.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f679effd640 1 -- 172.21.15.70:0/3835769430 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790039df0 con 0x7f67a806db60 2023-10-08T05:55:04.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f679effd640 1 -- 172.21.15.70:0/3835769430 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790043640 con 0x7f67a806db60 2023-10-08T05:55:04.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.522+0000 7f67acb6c640 1 -- 172.21.15.70:0/3835769430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a80c6500 con 0x7f67a806db60 2023-10-08T05:55:04.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.523+0000 7f67acb6c640 1 -- 172.21.15.70:0/3835769430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a80c6ac0 con 0x7f67a806db60 2023-10-08T05:55:04.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.524+0000 7f67acb6c640 1 -- 172.21.15.70:0/3835769430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a80c8ed0 con 0x7f67a806db60 2023-10-08T05:55:04.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.525+0000 7f679effd640 1 -- 172.21.15.70:0/3835769430 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 52) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f67900437a0 con 0x7f67a806db60 2023-10-08T05:55:04.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.525+0000 7f679effd640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f678806b0b0 0x7f678806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.526+0000 7f679effd640 1 -- 172.21.15.70:0/3835769430 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f679003e070 con 0x7f67a806db60 2023-10-08T05:55:04.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.526+0000 7f67a659c640 1 -- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f678806b0b0 msgr2=0x7f678806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:04.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.526+0000 7f67a659c640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f678806b0b0 0x7f678806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:04.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.531+0000 7f679effd640 1 -- 172.21.15.70:0/3835769430 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67900778a0 con 0x7f67a806db60 2023-10-08T05:55:04.545 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:04.544+0000 7f32afc2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:55:04.601 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:04.600+0000 7fa25dc3f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:55:04.704 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:04.703+0000 7f32afc2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:55:04.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.725+0000 7f67acb6c640 1 -- 172.21.15.70:0/3835769430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67a804e160 con 0x7f67a806db60 2023-10-08T05:55:04.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.726+0000 7f679effd640 1 -- 172.21.15.70:0/3835769430 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6790034020 con 0x7f67a806db60 2023-10-08T05:55:04.727 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:55:04.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.727+0000 7f67a659c640 1 -- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f678806b0b0 msgr2=0x7f678806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:04.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.727+0000 7f67a659c640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f678806b0b0 0x7f678806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:55:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.728+0000 7f67acb6c640 1 -- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f678806b0b0 msgr2=0x7f678806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f678806b0b0 0x7f678806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 -- 172.21.15.70:0/3835769430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a806db60 msgr2=0x7f67a80c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a806db60 0x7f67a80c35b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f6790000c00 tx=0x7f6790004730 comp rx=0 tx=0).stop 2023-10-08T05:55:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 -- 172.21.15.70:0/3835769430 shutdown_connections 2023-10-08T05:55:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f678806b0b0 0x7f678806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67a815aa00 0x7f67a80c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67a806db60 0x7f67a80c35b0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 --2- 172.21.15.70:0/3835769430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f67a806c3a0 0x7f67a80c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 -- 172.21.15.70:0/3835769430 >> 172.21.15.70:0/3835769430 conn(0x7f67a806a5d0 msgr2=0x7f67a80ca700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 -- 172.21.15.70:0/3835769430 shutdown_connections 2023-10-08T05:55:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.729+0000 7f67acb6c640 1 -- 172.21.15.70:0/3835769430 wait complete. 2023-10-08T05:55:04.749 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:55:04.772 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:04.772+0000 7fa25dc3f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:55:04.817 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:04.816+0000 7f32afc2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:55:04.867 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:04.866+0000 7f32afc2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:55:04.888 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:04.888+0000 7fa25dc3f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:55:04.940 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:04.939+0000 7fa25dc3f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:55:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.954+0000 7f3a3db73640 1 Processor -- start 2023-10-08T05:55:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.955+0000 7f3a3db73640 1 -- start start 2023-10-08T05:55:04.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.956+0000 7f3a3db73640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3a38160660 0x7f3a38070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.956+0000 7f3a3db73640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3a380c9910 0x7f3a380caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.956+0000 7f3a3db73640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a380cb450 0x7f3a380b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.956+0000 7f3a3db73640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3806e150 con 0x7f3a380c9910 2023-10-08T05:55:04.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.956+0000 7f3a3db73640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a380cb8c0 con 0x7f3a38160660 2023-10-08T05:55:04.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.956+0000 7f3a3db73640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3806de70 con 0x7f3a380cb450 2023-10-08T05:55:04.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.956+0000 7f3a2ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3a380c9910 0x7f3a380caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.956+0000 7f3a2ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3a380c9910 0x7f3a380caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43734/0 (socket says 172.21.15.70:43734) 2023-10-08T05:55:04.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.956+0000 7f3a2ffff640 1 -- 172.21.15.70:0/2342759439 learned_addr learned my addr 172.21.15.70:0/2342759439 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:04.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.957+0000 7f3a37fff640 1 --2- 172.21.15.70:0/2342759439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a380cb450 0x7f3a380b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.957+0000 7f3a377fe640 1 --2- 172.21.15.70:0/2342759439 >> v2:172.21.15.181:3300/0 conn(0x7f3a38160660 0x7f3a38070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.957+0000 7f3a2ffff640 1 -- 172.21.15.70:0/2342759439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a380cb450 msgr2=0x7f3a380b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:04.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.957+0000 7f3a2ffff640 1 --2- 172.21.15.70:0/2342759439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a380cb450 0x7f3a380b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.957+0000 7f3a2ffff640 1 -- 172.21.15.70:0/2342759439 >> v2:172.21.15.181:3300/0 conn(0x7f3a38160660 msgr2=0x7f3a38070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:04.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.957+0000 7f3a2ffff640 1 --2- 172.21.15.70:0/2342759439 >> v2:172.21.15.181:3300/0 conn(0x7f3a38160660 0x7f3a38070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.957+0000 7f3a2ffff640 1 -- 172.21.15.70:0/2342759439 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a380c8e30 con 0x7f3a380c9910 2023-10-08T05:55:04.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.958+0000 7f3a2ffff640 1 --2- 172.21.15.70:0/2342759439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a380c9910 0x7f3a380caf10 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3a380b0d80 tx=0x7f3a2002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09a3399f4fe818 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:04.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.958+0000 7f3a36ffd640 1 -- 172.21.15.70:0/2342759439 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20031860 con 0x7f3a380c9910 2023-10-08T05:55:04.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.958+0000 7f3a36ffd640 1 -- 172.21.15.70:0/2342759439 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a200319c0 con 0x7f3a380c9910 2023-10-08T05:55:04.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.958+0000 7f3a36ffd640 1 -- 172.21.15.70:0/2342759439 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20031d50 con 0x7f3a380c9910 2023-10-08T05:55:04.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.959+0000 7f3a3db73640 1 -- 172.21.15.70:0/2342759439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a380c9910 msgr2=0x7f3a380caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.959+0000 7f3a3db73640 1 --2- 172.21.15.70:0/2342759439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a380c9910 0x7f3a380caf10 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3a380b0d80 tx=0x7f3a2002b900 comp rx=0 tx=0).stop 2023-10-08T05:55:04.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.959+0000 7f3a3db73640 1 -- 172.21.15.70:0/2342759439 shutdown_connections 2023-10-08T05:55:04.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.959+0000 7f3a3db73640 1 --2- 172.21.15.70:0/2342759439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a380cb450 0x7f3a380b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.959+0000 7f3a3db73640 1 --2- 172.21.15.70:0/2342759439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a380c9910 0x7f3a380caf10 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.959+0000 7f3a3db73640 1 --2- 172.21.15.70:0/2342759439 >> v2:172.21.15.181:3300/0 conn(0x7f3a38160660 0x7f3a38070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.959+0000 7f3a3db73640 1 -- 172.21.15.70:0/2342759439 >> 172.21.15.70:0/2342759439 conn(0x7f3a3806a5d0 msgr2=0x7f3a380b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:04.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.959+0000 7f3a3db73640 1 -- 172.21.15.70:0/2342759439 shutdown_connections 2023-10-08T05:55:04.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.959+0000 7f3a3db73640 1 -- 172.21.15.70:0/2342759439 wait complete. 2023-10-08T05:55:04.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.961+0000 7f3a3db73640 1 Processor -- start 2023-10-08T05:55:04.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.961+0000 7f3a3db73640 1 -- start start 2023-10-08T05:55:04.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.962+0000 7f3a3db73640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a380c9910 0x7f3a380c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.962+0000 7f3a3db73640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a380cb450 0x7f3a3814cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.962+0000 7f3a3db73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a38160660 0x7f3a381f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.962+0000 7f3a3db73640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3814d170 con 0x7f3a38160660 2023-10-08T05:55:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.962+0000 7f3a3db73640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380c7800 con 0x7f3a380cb450 2023-10-08T05:55:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.962+0000 7f3a3db73640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a380c6bf0 con 0x7f3a380c9910 2023-10-08T05:55:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a37fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a38160660 0x7f3a381f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a37fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a38160660 0x7f3a381f2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43738/0 (socket says 172.21.15.70:43738) 2023-10-08T05:55:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a37fff640 1 -- 172.21.15.70:0/472102058 learned_addr learned my addr 172.21.15.70:0/472102058 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a377fe640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a380c9910 0x7f3a380c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a2ffff640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a380cb450 0x7f3a3814cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a37fff640 1 -- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a380c9910 msgr2=0x7f3a380c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a37fff640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a380c9910 0x7f3a380c3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a37fff640 1 -- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a380cb450 msgr2=0x7f3a3814cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a37fff640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a380cb450 0x7f3a3814cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a37fff640 1 -- 172.21.15.70:0/472102058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a200094d0 con 0x7f3a38160660 2023-10-08T05:55:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.963+0000 7f3a37fff640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a38160660 0x7f3a381f2ae0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f3a30007e60 tx=0x7f3a3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.964+0000 7f3a34ff9640 1 -- 172.21.15.70:0/472102058 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a30014620 con 0x7f3a38160660 2023-10-08T05:55:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.964+0000 7f3a34ff9640 1 -- 172.21.15.70:0/472102058 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3000ee90 con 0x7f3a38160660 2023-10-08T05:55:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.964+0000 7f3a34ff9640 1 -- 172.21.15.70:0/472102058 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300149e0 con 0x7f3a38160660 2023-10-08T05:55:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.964+0000 7f3a3db73640 1 -- 172.21.15.70:0/472102058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a380c6ed0 con 0x7f3a38160660 2023-10-08T05:55:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.964+0000 7f3a3db73640 1 -- 172.21.15.70:0/472102058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a380c7490 con 0x7f3a38160660 2023-10-08T05:55:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.965+0000 7f3a3db73640 1 -- 172.21.15.70:0/472102058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a380c7970 con 0x7f3a38160660 2023-10-08T05:55:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.966+0000 7f3a34ff9640 1 -- 172.21.15.70:0/472102058 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 52) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f3a30014b40 con 0x7f3a38160660 2023-10-08T05:55:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.967+0000 7f3a34ff9640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3a1806b0b0 0x7f3a1806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.967+0000 7f3a34ff9640 1 -- 172.21.15.70:0/472102058 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f3a3008a620 con 0x7f3a38160660 2023-10-08T05:55:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.967+0000 7f3a377fe640 1 -- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3a1806b0b0 msgr2=0x7f3a1806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:04.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.967+0000 7f3a377fe640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3a1806b0b0 0x7f3a1806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:04.971+0000 7f3a34ff9640 1 -- 172.21.15.70:0/472102058 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a30055100 con 0x7f3a38160660 2023-10-08T05:55:04.974 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:04.974+0000 7f32afc2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:55:05.055 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:05.054+0000 7fa25dc3f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:55:05.081 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:05.080+0000 7f32afc2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:55:05.165 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:05.165+0000 7fa25dc3f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:55:05.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.168+0000 7f3a3db73640 1 -- 172.21.15.70:0/472102058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3a3804e160 con 0x7f3a38160660 2023-10-08T05:55:05.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.168+0000 7f3a377fe640 1 -- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3a1806b0b0 msgr2=0x7f3a1806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:05.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.168+0000 7f3a377fe640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3a1806b0b0 0x7f3a1806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:55:05.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.168+0000 7f3a34ff9640 1 -- 172.21.15.70:0/472102058 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v121) v1 ==== 80+0+477 (secure 0 0 0) 0x7f3a30058720 con 0x7f3a38160660 2023-10-08T05:55:05.169 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/1906631919 2023-10-09T05:53:58.959064+0000 2023-10-08T05:55:05.169 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/1906631919 2023-10-09T05:53:58.959064+0000 2023-10-08T05:55:05.169 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3209826772 2023-10-09T05:53:58.959064+0000 2023-10-08T05:55:05.169 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1283682257 2023-10-09T05:53:58.959064+0000 2023-10-08T05:55:05.170 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2590863547 2023-10-09T05:53:58.959064+0000 2023-10-08T05:55:05.170 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3077797034 2023-10-09T05:53:58.959064+0000 2023-10-08T05:55:05.170 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3436033120 2023-10-09T05:53:58.959064+0000 2023-10-08T05:55:05.170 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/168987985 2023-10-09T05:53:58.959064+0000 2023-10-08T05:55:05.170 INFO:teuthology.orchestra.run.smithi070.stderr:listed 8 entries 2023-10-08T05:55:05.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 -- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3a1806b0b0 msgr2=0x7f3a1806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:05.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3a1806b0b0 0x7f3a1806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 -- 172.21.15.70:0/472102058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a38160660 msgr2=0x7f3a381f2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:05.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a38160660 0x7f3a381f2ae0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f3a30007e60 tx=0x7f3a3000e4b0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 -- 172.21.15.70:0/472102058 shutdown_connections 2023-10-08T05:55:05.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f3a1806b0b0 0x7f3a1806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a38160660 0x7f3a381f2ae0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3a380cb450 0x7f3a3814cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 --2- 172.21.15.70:0/472102058 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3a380c9910 0x7f3a380c3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 -- 172.21.15.70:0/472102058 >> 172.21.15.70:0/472102058 conn(0x7f3a3806a5d0 msgr2=0x7f3a380b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 -- 172.21.15.70:0/472102058 shutdown_connections 2023-10-08T05:55:05.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.171+0000 7f3a3db73640 1 -- 172.21.15.70:0/472102058 wait complete. 2023-10-08T05:55:05.189 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T05:55:05.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:05.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T05:55:05.236 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:05.235+0000 7f32afc2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:55:05.285 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:05.284+0000 7f32afc2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:55:05.333 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:05.333+0000 7fa25dc3f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:55:05.346 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:55:05.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:05.346 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T05:55:05.385 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:05.385+0000 7fa25dc3f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:55:05.387 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:55:05.387 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:55:05.405 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:05.405+0000 7f32afc2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:55:05.443 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:55:05.443 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:55:05.443 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:55:05.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:55:05.497 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:55:05.497 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:55:05.509 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:05.509+0000 7fa25dc3f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:55:05.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:55:05.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:55:05.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.766+0000 7f8574a14640 1 Processor -- start 2023-10-08T05:55:05.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.766+0000 7f8574a14640 1 -- start start 2023-10-08T05:55:05.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.768+0000 7f8574a14640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f857015cc60 0x7f85700cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:05.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.768+0000 7f8574a14640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857015d040 con 0x7f85700ce6e0 2023-10-08T05:55:05.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.768+0000 7f8574a14640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85700d0650 con 0x7f85700d16f0 2023-10-08T05:55:05.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.768+0000 7f8574a14640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85700d0930 con 0x7f857015cc60 2023-10-08T05:55:05.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.768+0000 7f856f59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f85700ce6e0 0x7f8570166aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41904/0 (socket says 172.21.15.70:41904) 2023-10-08T05:55:05.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.768+0000 7f856f59e640 1 -- 172.21.15.70:0/1368075911 learned_addr learned my addr 172.21.15.70:0/1368075911 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:05.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.768+0000 7f856ed9d640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f857015cc60 0x7f85700cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:05.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.768+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289310253 0 0) 0x7f857015d040 con 0x7f85700ce6e0 2023-10-08T05:55:05.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.768+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f855c003520 con 0x7f85700ce6e0 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190593702 0 0) 0x7f855c003520 con 0x7f85700ce6e0 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f857015cc60 msgr2=0x7f85700cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f857015cc60 0x7f85700cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 >> v1:172.21.15.181:6789/0 conn(0x7f85700d16f0 legacy=0x7f85700cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85700d0c10 con 0x7f85700ce6e0 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f85680019f0 con 0x7f85700ce6e0 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 >> v1:172.21.15.70:6789/0 conn(0x7f85700ce6e0 legacy=0x7f8570166aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f855c0039a0 0x7f855c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c0047c0 0x7f855c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f857015cc60 0x7f855c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85700d0930 con 0x7f855c0039a0 2023-10-08T05:55:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c00ba60 con 0x7f855c0047c0 2023-10-08T05:55:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855c00bc20 con 0x7f857015cc60 2023-10-08T05:55:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.769+0000 7f856ed9d640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f855c0039a0 0x7f855c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.770+0000 7f8567fff640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c0047c0 0x7f855c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.770+0000 7f856ed9d640 1 -- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f857015cc60 msgr2=0x7f855c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.770+0000 7f856ed9d640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f857015cc60 0x7f855c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.770+0000 7f856ed9d640 1 -- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c0047c0 msgr2=0x7f855c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.770+0000 7f856ed9d640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c0047c0 0x7f855c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.770+0000 7f856ed9d640 1 -- 172.21.15.70:0/1368075911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c003520 con 0x7f855c0039a0 2023-10-08T05:55:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.770+0000 7f8567fff640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c0047c0 0x7f855c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:05.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.771+0000 7f856ed9d640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f855c0039a0 0x7f855c003dc0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8560009a80 tx=0x7f855c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889912f850f6bb58 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:05.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.771+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560005990 con 0x7f855c0039a0 2023-10-08T05:55:05.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.771+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560005af0 con 0x7f855c0039a0 2023-10-08T05:55:05.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.771+0000 7f856e59c640 1 -- 172.21.15.70:0/1368075911 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560005ea0 con 0x7f855c0039a0 2023-10-08T05:55:05.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.771+0000 7f8574a14640 1 -- 172.21.15.70:0/1368075911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f855c0039a0 msgr2=0x7f855c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:05.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.771+0000 7f8574a14640 1 --2- 172.21.15.70:0/1368075911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f855c0039a0 0x7f855c003dc0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8560009a80 tx=0x7f855c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.772+0000 7f856f59e640 1 -- 172.21.15.70:0/1368075911 reap_dead start 2023-10-08T05:55:05.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.772+0000 7f8574a14640 1 -- 172.21.15.70:0/1368075911 shutdown_connections 2023-10-08T05:55:05.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.772+0000 7f8574a14640 1 -- 172.21.15.70:0/1368075911 >> 172.21.15.70:0/1368075911 conn(0x7f857009bcf0 msgr2=0x7f85700d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:05.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.772+0000 7f8574a14640 1 -- 172.21.15.70:0/1368075911 shutdown_connections 2023-10-08T05:55:05.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.772+0000 7f8574a14640 1 -- 172.21.15.70:0/1368075911 wait complete. 2023-10-08T05:55:05.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.773+0000 7f8574a14640 1 Processor -- start 2023-10-08T05:55:05.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.774+0000 7f8574a14640 1 -- start start 2023-10-08T05:55:05.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.774+0000 7f8574a14640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c0039a0 0x7f857015ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:05.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.774+0000 7f8574a14640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f857015ef00 0x7f857015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:05.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.774+0000 7f8574a14640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85700c6c40 0x7f85700c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.774+0000 7f8574a14640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85700ca8f0 con 0x7f857015ef00 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.774+0000 7f8574a14640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85700c9a40 con 0x7f855c0039a0 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.774+0000 7f8574a14640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85700cc2f0 con 0x7f85700c6c40 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f8567fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f857015ef00 0x7f857015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f8567fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f857015ef00 0x7f857015f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43756/0 (socket says 172.21.15.70:43756) 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f8567fff640 1 -- 172.21.15.70:0/15478984 learned_addr learned my addr 172.21.15.70:0/15478984 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f856ed9d640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c0039a0 0x7f857015ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f856f59e640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85700c6c40 0x7f85700c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f8567fff640 1 -- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85700c6c40 msgr2=0x7f85700c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f8567fff640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85700c6c40 0x7f85700c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f8567fff640 1 -- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c0039a0 msgr2=0x7f857015ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:05.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f8567fff640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c0039a0 0x7f857015ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:05.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f8567fff640 1 -- 172.21.15.70:0/15478984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85700c9ce0 con 0x7f857015ef00 2023-10-08T05:55:05.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.775+0000 7f8567fff640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f857015ef00 0x7f857015f380 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f855800d7a0 tx=0x7f855800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:05.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.776+0000 7f85677fe640 1 -- 172.21.15.70:0/15478984 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558017070 con 0x7f857015ef00 2023-10-08T05:55:05.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.776+0000 7f85677fe640 1 -- 172.21.15.70:0/15478984 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8558005070 con 0x7f857015ef00 2023-10-08T05:55:05.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.776+0000 7f85677fe640 1 -- 172.21.15.70:0/15478984 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558005420 con 0x7f857015ef00 2023-10-08T05:55:05.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.776+0000 7f8574a14640 1 -- 172.21.15.70:0/15478984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85700c9fc0 con 0x7f857015ef00 2023-10-08T05:55:05.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.776+0000 7f8574a14640 1 -- 172.21.15.70:0/15478984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85700ca580 con 0x7f857015ef00 2023-10-08T05:55:05.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.777+0000 7f8574a14640 1 -- 172.21.15.70:0/15478984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857004e160 con 0x7f857015ef00 2023-10-08T05:55:05.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.779+0000 7f85677fe640 1 -- 172.21.15.70:0/15478984 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 52) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f8558007030 con 0x7f857015ef00 2023-10-08T05:55:05.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.780+0000 7f85677fe640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f855006b0b0 0x7f855006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:05.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.780+0000 7f85677fe640 1 -- 172.21.15.70:0/15478984 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f8558089810 con 0x7f857015ef00 2023-10-08T05:55:05.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.780+0000 7f856ed9d640 1 -- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f855006b0b0 msgr2=0x7f855006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:05.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.780+0000 7f856ed9d640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f855006b0b0 0x7f855006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:05.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.785+0000 7f85677fe640 1 -- 172.21.15.70:0/15478984 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8558054110 con 0x7f857015ef00 2023-10-08T05:55:05.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.981+0000 7f856ed9d640 1 -- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f855006b0b0 msgr2=0x7f855006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:05.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:05.981+0000 7f856ed9d640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f855006b0b0 0x7f855006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:55:06.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.008+0000 7f8574a14640 1 -- 172.21.15.70:0/15478984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85700d3080 con 0x7f857015ef00 2023-10-08T05:55:06.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.008+0000 7f85677fe640 1 -- 172.21.15.70:0/15478984 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f8558057730 con 0x7f857015ef00 2023-10-08T05:55:06.009 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:06.009 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/290122525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":290122525},{"type":"v1","addr":"172.21.15.70:6835","nonce":290122525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4017950051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4017950051},{"type":"v1","addr":"172.21.15.181:6837","nonce":4017950051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1329101557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1329101557},{"type":"v1","addr":"172.21.15.70:6834","nonce":1329101557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:44.125639+0000","modified":"2023-10-08T05:53:47.206011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6184":{"gid":6184,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/422082772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":422082772},{"type":"v1","addr":"172.21.15.181:6835","nonce":422082772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-08T05:55:06.009 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 50 2023-10-08T05:55:06.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 -- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f855006b0b0 msgr2=0x7f855006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:06.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f855006b0b0 0x7f855006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 -- 172.21.15.70:0/15478984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f857015ef00 msgr2=0x7f857015f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f857015ef00 0x7f857015f380 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f855800d7a0 tx=0x7f855800dc60 comp rx=0 tx=0).stop 2023-10-08T05:55:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 -- 172.21.15.70:0/15478984 shutdown_connections 2023-10-08T05:55:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7f855006b0b0 0x7f855006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85700c6c40 0x7f85700c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f857015ef00 0x7f857015f380 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 --2- 172.21.15.70:0/15478984 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f855c0039a0 0x7f857015ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 -- 172.21.15.70:0/15478984 >> 172.21.15.70:0/15478984 conn(0x7f857009bcf0 msgr2=0x7f85700cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 -- 172.21.15.70:0/15478984 shutdown_connections 2023-10-08T05:55:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.011+0000 7f8574a14640 1 -- 172.21.15.70:0/15478984 wait complete. 2023-10-08T05:55:06.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:55:06.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.211+0000 7ff1f8590640 1 Processor -- start 2023-10-08T05:55:06.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.211+0000 7ff1f8590640 1 -- start start 2023-10-08T05:55:06.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.213+0000 7ff1f8590640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1f00ad790 0x7ff1f00702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.213+0000 7ff1f8590640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f015e330 0x7ff1f0070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.213+0000 7ff1f8590640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f006dee0 con 0x7ff1f0159ac0 2023-10-08T05:55:06.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.213+0000 7ff1f8590640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f006dcf0 con 0x7ff1f00ad790 2023-10-08T05:55:06.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.213+0000 7ff1f8590640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f015d850 con 0x7ff1f015e330 2023-10-08T05:55:06.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.214+0000 7ff1f5b2b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1f00ad790 0x7ff1f00702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.214+0000 7ff1f6b2d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f015e330 0x7ff1f0070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.214+0000 7ff1f5b2b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1f00ad790 0x7ff1f00702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:46916/0 (socket says 172.21.15.70:46916) 2023-10-08T05:55:06.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.214+0000 7ff1f632c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7ff1f0159ac0 0x7ff1f006d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:35470/0 (socket says 172.21.15.70:35470) 2023-10-08T05:55:06.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.214+0000 7ff1f5b2b640 1 -- 172.21.15.70:0/4128388172 learned_addr learned my addr 172.21.15.70:0/4128388172 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:06.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.215+0000 7ff1f5b2b640 1 -- 172.21.15.70:0/4128388172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f015e330 msgr2=0x7ff1f0070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:06.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.215+0000 7ff1f5b2b640 1 --2- 172.21.15.70:0/4128388172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f015e330 0x7ff1f0070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.215+0000 7ff1f5b2b640 1 -- 172.21.15.70:0/4128388172 >> v1:172.21.15.181:6789/0 conn(0x7ff1f0159ac0 legacy=0x7ff1f006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.215+0000 7ff1f5b2b640 1 -- 172.21.15.70:0/4128388172 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f015db30 con 0x7ff1f00ad790 2023-10-08T05:55:06.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.215+0000 7ff1f6b2d640 1 --2- 172.21.15.70:0/4128388172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f015e330 0x7ff1f0070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:55:06.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.216+0000 7ff1f5b2b640 1 --2- 172.21.15.70:0/4128388172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f00ad790 0x7ff1f00702d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009870 tx=0x7ff1e002db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90373041edd05b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:06.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.217+0000 7ff1f532a640 1 -- 172.21.15.70:0/4128388172 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e002e940 con 0x7ff1f00ad790 2023-10-08T05:55:06.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.217+0000 7ff1f532a640 1 -- 172.21.15.70:0/4128388172 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e002eaa0 con 0x7ff1f00ad790 2023-10-08T05:55:06.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.217+0000 7ff1f532a640 1 -- 172.21.15.70:0/4128388172 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e002ee30 con 0x7ff1f00ad790 2023-10-08T05:55:06.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.217+0000 7ff1f8590640 1 -- 172.21.15.70:0/4128388172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f00ad790 msgr2=0x7ff1f00702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.217+0000 7ff1f8590640 1 --2- 172.21.15.70:0/4128388172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f00ad790 0x7ff1f00702d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0009870 tx=0x7ff1e002db90 comp rx=0 tx=0).stop 2023-10-08T05:55:06.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.217+0000 7ff1f8590640 1 -- 172.21.15.70:0/4128388172 shutdown_connections 2023-10-08T05:55:06.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.217+0000 7ff1f8590640 1 --2- 172.21.15.70:0/4128388172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f015e330 0x7ff1f0070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.217+0000 7ff1f8590640 1 --2- 172.21.15.70:0/4128388172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f00ad790 0x7ff1f00702d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.217+0000 7ff1f8590640 1 -- 172.21.15.70:0/4128388172 >> 172.21.15.70:0/4128388172 conn(0x7ff1f006a380 msgr2=0x7ff1f00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.218+0000 7ff1f8590640 1 -- 172.21.15.70:0/4128388172 shutdown_connections 2023-10-08T05:55:06.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.218+0000 7ff1f8590640 1 -- 172.21.15.70:0/4128388172 wait complete. 2023-10-08T05:55:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.220+0000 7ff1f8590640 1 Processor -- start 2023-10-08T05:55:06.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.220+0000 7ff1f8590640 1 -- start start 2023-10-08T05:55:06.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.221+0000 7ff1f8590640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f00ad790 0x7ff1f0152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.221+0000 7ff1f8590640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f0159ac0 0x7ff1f0154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.221+0000 7ff1f8590640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1f015e330 0x7ff1f01552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.221+0000 7ff1f8590640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f01529b0 con 0x7ff1f015e330 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.221+0000 7ff1f8590640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00cc680 con 0x7ff1f0159ac0 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.221+0000 7ff1f8590640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f00cc960 con 0x7ff1f00ad790 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.221+0000 7ff1f6b2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1f015e330 0x7ff1f01552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.222+0000 7ff1f6b2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1f015e330 0x7ff1f01552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43768/0 (socket says 172.21.15.70:43768) 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.222+0000 7ff1f6b2d640 1 -- 172.21.15.70:0/1681394639 learned_addr learned my addr 172.21.15.70:0/1681394639 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.222+0000 7ff1f5b2b640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f0159ac0 0x7ff1f0154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.222+0000 7ff1f632c640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f00ad790 0x7ff1f0152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.222+0000 7ff1f6b2d640 1 -- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f00ad790 msgr2=0x7ff1f0152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.222+0000 7ff1f6b2d640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f00ad790 0x7ff1f0152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.222+0000 7ff1f6b2d640 1 -- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f0159ac0 msgr2=0x7ff1f0154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.222+0000 7ff1f6b2d640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f0159ac0 0x7ff1f0154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.222+0000 7ff1f6b2d640 1 -- 172.21.15.70:0/1681394639 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e8009a70 con 0x7ff1f015e330 2023-10-08T05:55:06.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.222+0000 7ff1f6b2d640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1f015e330 0x7ff1f01552f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800efc0 tx=0x7ff1e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:06.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.223+0000 7ff1e6ffd640 1 -- 172.21.15.70:0/1681394639 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8009cd0 con 0x7ff1f015e330 2023-10-08T05:55:06.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.223+0000 7ff1e6ffd640 1 -- 172.21.15.70:0/1681394639 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e800f040 con 0x7ff1f015e330 2023-10-08T05:55:06.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.223+0000 7ff1f8590640 1 -- 172.21.15.70:0/1681394639 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e00094d0 con 0x7ff1f015e330 2023-10-08T05:55:06.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.223+0000 7ff1e6ffd640 1 -- 172.21.15.70:0/1681394639 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8003990 con 0x7ff1f015e330 2023-10-08T05:55:06.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.223+0000 7ff1f8590640 1 -- 172.21.15.70:0/1681394639 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f00ccf20 con 0x7ff1f015e330 2023-10-08T05:55:06.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.224+0000 7ff1f8590640 1 -- 172.21.15.70:0/1681394639 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f015b9f0 con 0x7ff1f015e330 2023-10-08T05:55:06.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.225+0000 7ff1e6ffd640 1 -- 172.21.15.70:0/1681394639 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 52) v1 ==== 89445+0+0 (secure 0 0 0) 0x7ff1e8014480 con 0x7ff1f015e330 2023-10-08T05:55:06.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.226+0000 7ff1e6ffd640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7ff1d406b0e0 0x7ff1d406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.227+0000 7ff1e6ffd640 1 -- 172.21.15.70:0/1681394639 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7ff1e808a980 con 0x7ff1f015e330 2023-10-08T05:55:06.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.227+0000 7ff1f632c640 1 -- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7ff1d406b0e0 msgr2=0x7ff1d406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:06.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.227+0000 7ff1f632c640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7ff1d406b0e0 0x7ff1d406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:06.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.231+0000 7ff1e6ffd640 1 -- 172.21.15.70:0/1681394639 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1e8055320 con 0x7ff1f015e330 2023-10-08T05:55:06.239 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.237+0000 7f32afc2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:55:06.340 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:06.340+0000 7fa25dc3f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:55:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.422+0000 7ff1f8590640 1 -- 172.21.15.70:0/1681394639 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1f0067490 con 0x7ff1f015e330 2023-10-08T05:55:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.423+0000 7ff1e6ffd640 1 -- 172.21.15.70:0/1681394639 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+17487 (secure 0 0 0) 0x7ff1e8058940 con 0x7ff1f015e330 2023-10-08T05:55:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:06.424 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":121,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:00.541234+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:42.052166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:43.161991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/1906631919":"2023-10-09T05:53:58.959064+0000","172.21.15.181:6832/1906631919":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3209826772":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/1283682257":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/2590863547":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3077797034":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3436033120":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/168987985":"2023-10-09T05:53:58.959064+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:55:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 -- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7ff1d406b0e0 msgr2=0x7ff1d406d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7ff1d406b0e0 0x7ff1d406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 -- 172.21.15.70:0/1681394639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1f015e330 msgr2=0x7ff1f01552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1f015e330 0x7ff1f01552f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800efc0 tx=0x7ff1e800c490 comp rx=0 tx=0).stop 2023-10-08T05:55:06.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 -- 172.21.15.70:0/1681394639 shutdown_connections 2023-10-08T05:55:06.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7ff1d406b0e0 0x7ff1d406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1f015e330 0x7ff1f01552f0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1f0159ac0 0x7ff1f0154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 --2- 172.21.15.70:0/1681394639 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1f00ad790 0x7ff1f0152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 -- 172.21.15.70:0/1681394639 >> 172.21.15.70:0/1681394639 conn(0x7ff1f006a380 msgr2=0x7ff1f00c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:06.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 -- 172.21.15.70:0/1681394639 shutdown_connections 2023-10-08T05:55:06.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.426+0000 7ff1f8590640 1 -- 172.21.15.70:0/1681394639 wait complete. 2023-10-08T05:55:06.443 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:53:42.052166+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:53:43.161991+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:55:06.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:55:06.534 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.533+0000 7f32afc2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:55:06.591 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.590+0000 7f32afc2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:55:06.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.643+0000 7fa25759e640 1 Processor -- start 2023-10-08T05:55:06.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.643+0000 7fa25759e640 1 -- start start 2023-10-08T05:55:06.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.644+0000 7fa25759e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa2580d1120 0x7fa2580d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.644+0000 7fa25759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580d1500 0x7fa2581610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.644+0000 7fa25759e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2580d0a80 con 0x7fa2580b5060 2023-10-08T05:55:06.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.645+0000 7fa25759e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa258161620 con 0x7fa2580d1120 2023-10-08T05:55:06.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.645+0000 7fa25759e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580b1fe0 con 0x7fa2580d1500 2023-10-08T05:55:06.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.645+0000 7fa256d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa2580b5060 0x7fa2580cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41916/0 (socket says 172.21.15.70:41916) 2023-10-08T05:55:06.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.645+0000 7fa256d9d640 1 -- 172.21.15.70:0/228551551 learned_addr learned my addr 172.21.15.70:0/228551551 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:06.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.645+0000 7fa25659c640 1 --2- 172.21.15.70:0/228551551 >> v2:172.21.15.181:3300/0 conn(0x7fa2580d1120 0x7fa2580d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.645+0000 7fa255d9b640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580d1500 0x7fa2581610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.645+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641540724 0 0) 0x7fa2580d0a80 con 0x7fa2580b5060 2023-10-08T05:55:06.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.645+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa240003520 con 0x7fa2580b5060 2023-10-08T05:55:06.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1957012687 0 0) 0x7fa240003520 con 0x7fa2580b5060 2023-10-08T05:55:06.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580d1500 msgr2=0x7fa2581610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:06.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580d1500 0x7fa2581610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 >> v2:172.21.15.181:3300/0 conn(0x7fa2580d1120 msgr2=0x7fa2580d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 --2- 172.21.15.70:0/228551551 >> v2:172.21.15.181:3300/0 conn(0x7fa2580d1120 0x7fa2580d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2580b4580 con 0x7fa2580b5060 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa2500019f0 con 0x7fa2580b5060 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 >> v1:172.21.15.70:6789/0 conn(0x7fa2580b5060 legacy=0x7fa2580cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa240003970 0x7fa240003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2400047c0 0x7fa240004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa240005180 0x7fa24000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa258161620 con 0x7fa240003970 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580b1fe0 con 0x7fa240005180 2023-10-08T05:55:06.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24000c020 con 0x7fa2400047c0 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa255d9b640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580d1500 0x7fa2581610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa25659c640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa240003970 0x7fa240003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.646+0000 7fa255d9b640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2400047c0 0x7fa240004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.647+0000 7fa256d9d640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa240005180 0x7fa24000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.647+0000 7fa25659c640 1 -- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2400047c0 msgr2=0x7fa240004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.647+0000 7fa25659c640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2400047c0 0x7fa240004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.647+0000 7fa25659c640 1 -- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa240005180 msgr2=0x7fa24000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.647+0000 7fa25659c640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa240005180 0x7fa24000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.647+0000 7fa25659c640 1 -- 172.21.15.70:0/228551551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa240003520 con 0x7fa240003970 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.647+0000 7fa256d9d640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa240005180 0x7fa24000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:06.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.648+0000 7fa25659c640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa240003970 0x7fa240003d90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa248009ab0 tx=0x7fa2400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12db3a6b2243f8ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:06.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.648+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248005990 con 0x7fa240003970 2023-10-08T05:55:06.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.648+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa248005af0 con 0x7fa240003970 2023-10-08T05:55:06.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.648+0000 7fa25559a640 1 -- 172.21.15.70:0/228551551 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa248005ea0 con 0x7fa240003970 2023-10-08T05:55:06.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.648+0000 7fa25759e640 1 -- 172.21.15.70:0/228551551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa240003970 msgr2=0x7fa240003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.648+0000 7fa25759e640 1 --2- 172.21.15.70:0/228551551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa240003970 0x7fa240003d90 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa248009ab0 tx=0x7fa2400057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.649+0000 7fa256d9d640 1 -- 172.21.15.70:0/228551551 reap_dead start 2023-10-08T05:55:06.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.649+0000 7fa25759e640 1 -- 172.21.15.70:0/228551551 shutdown_connections 2023-10-08T05:55:06.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.649+0000 7fa25759e640 1 -- 172.21.15.70:0/228551551 >> 172.21.15.70:0/228551551 conn(0x7fa2580d2960 msgr2=0x7fa2580d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.649+0000 7fa25759e640 1 -- 172.21.15.70:0/228551551 shutdown_connections 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.649+0000 7fa25759e640 1 -- 172.21.15.70:0/228551551 wait complete. 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.650+0000 7fa25759e640 1 Processor -- start 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.651+0000 7fa25759e640 1 -- start start 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.651+0000 7fa25759e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa240003970 0x7fa25815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.651+0000 7fa25759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580b7650 0x7fa258150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.651+0000 7fa25759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa258160490 0x7fa258160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa25759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2580cae30 con 0x7fa258160490 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa25759e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa258160e80 con 0x7fa240003970 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa25759e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25815bd60 con 0x7fa2580b7650 2023-10-08T05:55:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa256d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa258160490 0x7fa258160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa256d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa258160490 0x7fa258160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43782/0 (socket says 172.21.15.70:43782) 2023-10-08T05:55:06.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa256d9d640 1 -- 172.21.15.70:0/1713472134 learned_addr learned my addr 172.21.15.70:0/1713472134 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:06.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa25659c640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa240003970 0x7fa25815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa255d9b640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580b7650 0x7fa258150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:06.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa256d9d640 1 -- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580b7650 msgr2=0x7fa258150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa256d9d640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580b7650 0x7fa258150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa256d9d640 1 -- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa240003970 msgr2=0x7fa25815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa256d9d640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa240003970 0x7fa25815b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa256d9d640 1 -- 172.21.15.70:0/1713472134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2581505a0 con 0x7fa258160490 2023-10-08T05:55:06.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa255d9b640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580b7650 0x7fa258150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:06.656 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:06.652+0000 7fa25dc3f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:55:06.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.653+0000 7fa256d9d640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa258160490 0x7fa258160940 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa2580d1ac0 tx=0x7fa25000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:06.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.653+0000 7fa246ffd640 1 -- 172.21.15.70:0/1713472134 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250004d60 con 0x7fa258160490 2023-10-08T05:55:06.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.653+0000 7fa246ffd640 1 -- 172.21.15.70:0/1713472134 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25000bb80 con 0x7fa258160490 2023-10-08T05:55:06.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.653+0000 7fa25759e640 1 -- 172.21.15.70:0/1713472134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2580b7e10 con 0x7fa258160490 2023-10-08T05:55:06.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.653+0000 7fa246ffd640 1 -- 172.21.15.70:0/1713472134 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250007070 con 0x7fa258160490 2023-10-08T05:55:06.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.653+0000 7fa25759e640 1 -- 172.21.15.70:0/1713472134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2580b83d0 con 0x7fa258160490 2023-10-08T05:55:06.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.655+0000 7fa25759e640 1 -- 172.21.15.70:0/1713472134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2580caf70 con 0x7fa258160490 2023-10-08T05:55:06.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.655+0000 7fa246ffd640 1 -- 172.21.15.70:0/1713472134 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 52) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fa2500071d0 con 0x7fa258160490 2023-10-08T05:55:06.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.656+0000 7fa246ffd640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fa23806b0e0 0x7fa23806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:06.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.656+0000 7fa246ffd640 1 -- 172.21.15.70:0/1713472134 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fa25008a140 con 0x7fa258160490 2023-10-08T05:55:06.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.656+0000 7fa25659c640 1 -- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fa23806b0e0 msgr2=0x7fa23806d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1714252273 2023-10-08T05:55:06.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.656+0000 7fa25659c640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fa23806b0e0 0x7fa23806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:06.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.660+0000 7fa246ffd640 1 -- 172.21.15.70:0/1713472134 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2580caf70 con 0x7fa258160490 2023-10-08T05:55:06.708 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.707+0000 7f32afc2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:55:06.727 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:06.727+0000 7fa25dc3f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:55:06.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.728+0000 7fa246ffd640 1 -- 172.21.15.70:0/1713472134 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 53) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa250054df0 con 0x7fa258160490 2023-10-08T05:55:06.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.728+0000 7fa246ffd640 1 -- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fa23806b0e0 msgr2=0x7fa23806d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:06.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.728+0000 7fa246ffd640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fa23806b0e0 0x7fa23806d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.794 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.793+0000 7f3234814640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:06.794 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.793+0000 7f3234814640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:06.794 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.793+0000 7f3234814640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:06.794 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.793+0000 7f3234814640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:06.794 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.793+0000 7f3234814640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:06.795 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.794+0000 7f323100d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:06.795 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.794+0000 7f323100d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:06.795 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.794+0000 7f323100d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:06.795 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.794+0000 7f323100d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:06.795 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:06.794+0000 7f323100d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:06.851 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:06.851+0000 7fa25dc3f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:55:06.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.866+0000 7fa25759e640 1 -- 172.21.15.70:0/1713472134 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa258074150 con 0x7fa258160490 2023-10-08T05:55:06.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.867+0000 7fa246ffd640 1 -- 172.21.15.70:0/1713472134 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa250073660 con 0x7fa258160490 2023-10-08T05:55:06.867 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:55:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.870+0000 7fa25759e640 1 -- 172.21.15.70:0/1713472134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa258160490 msgr2=0x7fa258160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.870+0000 7fa25759e640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa258160490 0x7fa258160940 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa2580d1ac0 tx=0x7fa25000b4a0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.871+0000 7fa25759e640 1 -- 172.21.15.70:0/1713472134 shutdown_connections 2023-10-08T05:55:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.871+0000 7fa25759e640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:6832/1714252273,v1:172.21.15.181:6833/1714252273] conn(0x7fa23806b0e0 0x7fa23806d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.871+0000 7fa25759e640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa258160490 0x7fa258160940 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.871+0000 7fa25759e640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2580b7650 0x7fa258150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.871+0000 7fa25759e640 1 --2- 172.21.15.70:0/1713472134 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa240003970 0x7fa25815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:06.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.871+0000 7fa25759e640 1 -- 172.21.15.70:0/1713472134 >> 172.21.15.70:0/1713472134 conn(0x7fa2580d2960 msgr2=0x7fa25806c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:06.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.871+0000 7fa25759e640 1 -- 172.21.15.70:0/1713472134 shutdown_connections 2023-10-08T05:55:06.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:06.871+0000 7fa25759e640 1 -- 172.21.15.70:0/1713472134 wait complete. 2023-10-08T05:55:06.887 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:55:07.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.088+0000 7fe0d5901640 1 Processor -- start 2023-10-08T05:55:07.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.088+0000 7fe0d5901640 1 -- start start 2023-10-08T05:55:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.090+0000 7fe0d5901640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0d00afbe0 0x7fe0d00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.090+0000 7fe0d5901640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0d00cb130 0x7fe0d00cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.090+0000 7fe0d5901640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d00cbad0 con 0x7fe0d006dfc0 2023-10-08T05:55:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.090+0000 7fe0d5901640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d00b2620 con 0x7fe0d00afbe0 2023-10-08T05:55:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.090+0000 7fe0d5901640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d00b2060 con 0x7fe0d00cb130 2023-10-08T05:55:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.090+0000 7fe0ceffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0d00afbe0 0x7fe0d00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.090+0000 7fe0ceffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0d00afbe0 0x7fe0d00b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43792/0 (socket says 172.21.15.70:43792) 2023-10-08T05:55:07.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.090+0000 7fe0ceffd640 1 -- 172.21.15.70:0/1898754066 learned_addr learned my addr 172.21.15.70:0/1898754066 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.091+0000 7fe0ceffd640 1 -- 172.21.15.70:0/1898754066 >> v2:172.21.15.181:3300/0 conn(0x7fe0d00cb130 msgr2=0x7fe0d00cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.091+0000 7fe0ceffd640 1 --2- 172.21.15.70:0/1898754066 >> v2:172.21.15.181:3300/0 conn(0x7fe0d00cb130 0x7fe0d00cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.091+0000 7fe0ceffd640 1 -- 172.21.15.70:0/1898754066 >> v1:172.21.15.70:6789/0 conn(0x7fe0d006dfc0 legacy=0x7fe0d0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.091+0000 7fe0ceffd640 1 -- 172.21.15.70:0/1898754066 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d00b2340 con 0x7fe0d00afbe0 2023-10-08T05:55:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.092+0000 7fe0ceffd640 1 --2- 172.21.15.70:0/1898754066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d00afbe0 0x7fe0d00b15b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00c8d10 tx=0x7fe0c002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152606459233b7ef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.092+0000 7fe0ce7fc640 1 -- 172.21.15.70:0/1898754066 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0030870 con 0x7fe0d00afbe0 2023-10-08T05:55:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.092+0000 7fe0ce7fc640 1 -- 172.21.15.70:0/1898754066 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c00309d0 con 0x7fe0d00afbe0 2023-10-08T05:55:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.092+0000 7fe0ce7fc640 1 -- 172.21.15.70:0/1898754066 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0030d60 con 0x7fe0d00afbe0 2023-10-08T05:55:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.093+0000 7fe0d5901640 1 -- 172.21.15.70:0/1898754066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d00afbe0 msgr2=0x7fe0d00b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.093+0000 7fe0d5901640 1 --2- 172.21.15.70:0/1898754066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d00afbe0 0x7fe0d00b15b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00c8d10 tx=0x7fe0c002b9b0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.093+0000 7fe0d5901640 1 -- 172.21.15.70:0/1898754066 shutdown_connections 2023-10-08T05:55:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.093+0000 7fe0d5901640 1 --2- 172.21.15.70:0/1898754066 >> v2:172.21.15.181:3300/0 conn(0x7fe0d00cb130 0x7fe0d00cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.093+0000 7fe0d5901640 1 --2- 172.21.15.70:0/1898754066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d00afbe0 0x7fe0d00b15b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.093+0000 7fe0d5901640 1 -- 172.21.15.70:0/1898754066 >> 172.21.15.70:0/1898754066 conn(0x7fe0d006a5d0 msgr2=0x7fe0d00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.093+0000 7fe0d5901640 1 -- 172.21.15.70:0/1898754066 shutdown_connections 2023-10-08T05:55:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.093+0000 7fe0d5901640 1 -- 172.21.15.70:0/1898754066 wait complete. 2023-10-08T05:55:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.095+0000 7fe0d5901640 1 Processor -- start 2023-10-08T05:55:07.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.095+0000 7fe0d5901640 1 -- start start 2023-10-08T05:55:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0d5901640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d006dfc0 0x7fe0d015c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0d5901640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0d00afbe0 0x7fe0d0144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0d5901640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00cb130 0x7fe0d00ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0d5901640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00c77c0 con 0x7fe0d006dfc0 2023-10-08T05:55:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0d5901640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d015cea0 con 0x7fe0d00afbe0 2023-10-08T05:55:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0d5901640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d015d0d0 con 0x7fe0d00cb130 2023-10-08T05:55:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d006dfc0 0x7fe0d015c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d006dfc0 0x7fe0d015c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43802/0 (socket says 172.21.15.70:43802) 2023-10-08T05:55:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0ceffd640 1 -- 172.21.15.70:0/2787108949 learned_addr learned my addr 172.21.15.70:0/2787108949 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0ceffd640 1 -- 172.21.15.70:0/2787108949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00cb130 msgr2=0x7fe0d00ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0ceffd640 1 --2- 172.21.15.70:0/2787108949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00cb130 0x7fe0d00ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0ceffd640 1 -- 172.21.15.70:0/2787108949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0d00afbe0 msgr2=0x7fe0d0144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0ceffd640 1 --2- 172.21.15.70:0/2787108949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0d00afbe0 0x7fe0d0144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.096+0000 7fe0ceffd640 1 -- 172.21.15.70:0/2787108949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00094d0 con 0x7fe0d006dfc0 2023-10-08T05:55:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.097+0000 7fe0ceffd640 1 --2- 172.21.15.70:0/2787108949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d006dfc0 0x7fe0d015c960 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0000c00 tx=0x7fe0c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.097+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2787108949 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0030840 con 0x7fe0d006dfc0 2023-10-08T05:55:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.097+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2787108949 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c0043400 con 0x7fe0d006dfc0 2023-10-08T05:55:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.097+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2787108949 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00437d0 con 0x7fe0d006dfc0 2023-10-08T05:55:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.097+0000 7fe0d5901640 1 -- 172.21.15.70:0/2787108949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d015d3b0 con 0x7fe0d006dfc0 2023-10-08T05:55:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.097+0000 7fe0d5901640 1 -- 172.21.15.70:0/2787108949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d015d970 con 0x7fe0d006dfc0 2023-10-08T05:55:07.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.098+0000 7fe0d5901640 1 -- 172.21.15.70:0/2787108949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d004e160 con 0x7fe0d006dfc0 2023-10-08T05:55:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.099+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2787108949 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 53) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe0c0043930 con 0x7fe0d006dfc0 2023-10-08T05:55:07.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.100+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2787108949 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6963+0+0 (secure 0 0 0) 0x7fe0c003e070 con 0x7fe0d006dfc0 2023-10-08T05:55:07.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.104+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2787108949 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe0c0077a80 con 0x7fe0d006dfc0 2023-10-08T05:55:07.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.292+0000 7fe0d5901640 1 -- 172.21.15.70:0/2787108949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe0d0159280 con 0x7fe0d006dfc0 2023-10-08T05:55:07.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.293+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2787108949 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe0c007b0a0 con 0x7fe0d006dfc0 2023-10-08T05:55:07.293 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:55:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.295+0000 7fe0d5901640 1 -- 172.21.15.70:0/2787108949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d006dfc0 msgr2=0x7fe0d015c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.295+0000 7fe0d5901640 1 --2- 172.21.15.70:0/2787108949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d006dfc0 0x7fe0d015c960 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0000c00 tx=0x7fe0c0004730 comp rx=0 tx=0).stop 2023-10-08T05:55:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.295+0000 7fe0d5901640 1 -- 172.21.15.70:0/2787108949 shutdown_connections 2023-10-08T05:55:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.295+0000 7fe0d5901640 1 --2- 172.21.15.70:0/2787108949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0d00cb130 0x7fe0d00ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.295+0000 7fe0d5901640 1 --2- 172.21.15.70:0/2787108949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0d00afbe0 0x7fe0d0144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.295+0000 7fe0d5901640 1 --2- 172.21.15.70:0/2787108949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d006dfc0 0x7fe0d015c960 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.295+0000 7fe0d5901640 1 -- 172.21.15.70:0/2787108949 >> 172.21.15.70:0/2787108949 conn(0x7fe0d006a5d0 msgr2=0x7fe0d0073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.295+0000 7fe0d5901640 1 -- 172.21.15.70:0/2787108949 shutdown_connections 2023-10-08T05:55:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.295+0000 7fe0d5901640 1 -- 172.21.15.70:0/2787108949 wait complete. 2023-10-08T05:55:07.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:55:07.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.520+0000 7f48e424f640 1 Processor -- start 2023-10-08T05:55:07.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.520+0000 7f48e424f640 1 -- start start 2023-10-08T05:55:07.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.521+0000 7f48e424f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f48dc070cc0 0x7f48dc0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.522+0000 7f48e424f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48dc0cb450 0x7f48dc0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.522+0000 7f48e424f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48dc071170 con 0x7f48dc160660 2023-10-08T05:55:07.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.522+0000 7f48e424f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48dc06e150 con 0x7f48dc0cb450 2023-10-08T05:55:07.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.522+0000 7f48e424f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48dc06de70 con 0x7f48dc070cc0 2023-10-08T05:55:07.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.522+0000 7f48e1feb640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f48dc160660 0x7f48dc0c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41922/0 (socket says 172.21.15.70:41922) 2023-10-08T05:55:07.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.522+0000 7f48e1feb640 1 -- 172.21.15.70:0/2355874645 learned_addr learned my addr 172.21.15.70:0/2355874645 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:07.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.522+0000 7f48e27ec640 1 --2- 172.21.15.70:0/2355874645 >> v2:172.21.15.70:3300/0 conn(0x7f48dc0cb450 0x7f48dc0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.522+0000 7f48e17ea640 1 --2- 172.21.15.70:0/2355874645 >> v2:172.21.15.181:3300/0 conn(0x7f48dc070cc0 0x7f48dc0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.522+0000 7f48e0fe9640 1 -- 172.21.15.70:0/2355874645 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949176459 0 0) 0x7f48d8001110 con 0x7f48dc160660 2023-10-08T05:55:07.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.522+0000 7f48e0fe9640 1 -- 172.21.15.70:0/2355874645 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48c8003520 con 0x7f48dc160660 2023-10-08T05:55:07.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.523+0000 7f48e27ec640 1 -- 172.21.15.70:0/2355874645 >> v2:172.21.15.181:3300/0 conn(0x7f48dc070cc0 msgr2=0x7f48dc0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:07.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.523+0000 7f48e27ec640 1 --2- 172.21.15.70:0/2355874645 >> v2:172.21.15.181:3300/0 conn(0x7f48dc070cc0 0x7f48dc0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.523+0000 7f48e27ec640 1 -- 172.21.15.70:0/2355874645 >> v1:172.21.15.70:6789/0 conn(0x7f48dc160660 legacy=0x7f48dc0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.523+0000 7f48e0fe9640 1 -- 172.21.15.70:0/2355874645 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 414350576 0 0) 0x7f48c8003520 con 0x7f48dc160660 2023-10-08T05:55:07.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.523+0000 7f48e27ec640 1 -- 172.21.15.70:0/2355874645 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48dc0c8e30 con 0x7f48dc0cb450 2023-10-08T05:55:07.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.524+0000 7f48e27ec640 1 --2- 172.21.15.70:0/2355874645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 0x7f48dc0b1990 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0b0d50 tx=0x7f48d402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec21b16d7f8c1680 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:07.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.524+0000 7f48e0fe9640 1 -- 172.21.15.70:0/2355874645 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8001800 con 0x7f48dc0cb450 2023-10-08T05:55:07.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.524+0000 7f48e0fe9640 1 -- 172.21.15.70:0/2355874645 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d402c8c0 con 0x7f48dc0cb450 2023-10-08T05:55:07.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.524+0000 7f48e0fe9640 1 -- 172.21.15.70:0/2355874645 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d402cc50 con 0x7f48dc0cb450 2023-10-08T05:55:07.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.525+0000 7f48e424f640 1 -- 172.21.15.70:0/2355874645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 msgr2=0x7f48dc0b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.525+0000 7f48e424f640 1 --2- 172.21.15.70:0/2355874645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 0x7f48dc0b1990 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f48dc0b0d50 tx=0x7f48d402bd90 comp rx=0 tx=0).stop 2023-10-08T05:55:07.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.525+0000 7f48e424f640 1 -- 172.21.15.70:0/2355874645 shutdown_connections 2023-10-08T05:55:07.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.525+0000 7f48e424f640 1 --2- 172.21.15.70:0/2355874645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 0x7f48dc0b1990 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.525+0000 7f48e424f640 1 --2- 172.21.15.70:0/2355874645 >> v2:172.21.15.181:3300/0 conn(0x7f48dc070cc0 0x7f48dc0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.525+0000 7f48e424f640 1 -- 172.21.15.70:0/2355874645 >> 172.21.15.70:0/2355874645 conn(0x7f48dc06a5d0 msgr2=0x7f48dc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:07.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.525+0000 7f48e424f640 1 -- 172.21.15.70:0/2355874645 shutdown_connections 2023-10-08T05:55:07.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.525+0000 7f48e424f640 1 -- 172.21.15.70:0/2355874645 wait complete. 2023-10-08T05:55:07.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.527+0000 7f48e424f640 1 Processor -- start 2023-10-08T05:55:07.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.527+0000 7f48e424f640 1 -- start start 2023-10-08T05:55:07.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.527+0000 7f48e424f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48dc070cc0 0x7f48dc15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e424f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 0x7f48dc14cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e424f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48dc160660 0x7f48dc15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e424f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc14d100 con 0x7f48dc0cb450 2023-10-08T05:55:07.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e424f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc0abbc0 con 0x7f48dc070cc0 2023-10-08T05:55:07.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e424f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48dc15c2a0 con 0x7f48dc160660 2023-10-08T05:55:07.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e17ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 0x7f48dc14cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e17ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 0x7f48dc14cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43818/0 (socket says 172.21.15.70:43818) 2023-10-08T05:55:07.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e17ea640 1 -- 172.21.15.70:0/1926501730 learned_addr learned my addr 172.21.15.70:0/1926501730 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:07.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e17ea640 1 -- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48dc160660 msgr2=0x7f48dc15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e17ea640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48dc160660 0x7f48dc15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e17ea640 1 -- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48dc070cc0 msgr2=0x7f48dc15cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e17ea640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48dc070cc0 0x7f48dc15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.528+0000 7f48e17ea640 1 -- 172.21.15.70:0/1926501730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4009a70 con 0x7f48dc0cb450 2023-10-08T05:55:07.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.529+0000 7f48e17ea640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 0x7f48dc14cbc0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00e9b0 tx=0x7f48cc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:07.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.529+0000 7f48d27fc640 1 -- 172.21.15.70:0/1926501730 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc009670 con 0x7f48dc0cb450 2023-10-08T05:55:07.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.529+0000 7f48d27fc640 1 -- 172.21.15.70:0/1926501730 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc00c9a0 con 0x7f48dc0cb450 2023-10-08T05:55:07.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.529+0000 7f48e424f640 1 -- 172.21.15.70:0/1926501730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48dc15c580 con 0x7f48dc0cb450 2023-10-08T05:55:07.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.529+0000 7f48d27fc640 1 -- 172.21.15.70:0/1926501730 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00cd50 con 0x7f48dc0cb450 2023-10-08T05:55:07.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.529+0000 7f48e424f640 1 -- 172.21.15.70:0/1926501730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48dc0ae910 con 0x7f48dc0cb450 2023-10-08T05:55:07.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.530+0000 7f48e424f640 1 -- 172.21.15.70:0/1926501730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48dc0ae490 con 0x7f48dc0cb450 2023-10-08T05:55:07.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.531+0000 7f48d27fc640 1 -- 172.21.15.70:0/1926501730 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 53) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f48cc013470 con 0x7f48dc0cb450 2023-10-08T05:55:07.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.532+0000 7f48d27fc640 1 -- 172.21.15.70:0/1926501730 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6963+0+0 (secure 0 0 0) 0x7f48cc089930 con 0x7f48dc0cb450 2023-10-08T05:55:07.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.537+0000 7f48d27fc640 1 -- 172.21.15.70:0/1926501730 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48dc0ae490 con 0x7f48dc0cb450 2023-10-08T05:55:07.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.732+0000 7f48e424f640 1 -- 172.21.15.70:0/1926501730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48dc0ae490 con 0x7f48dc0cb450 2023-10-08T05:55:07.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.734+0000 7f48d27fc640 1 -- 172.21.15.70:0/1926501730 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 54) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f48cc054300 con 0x7f48dc0cb450 2023-10-08T05:55:07.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.735+0000 7f48d27fc640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f48c0070e30 0x7f48c0073220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.735+0000 7f48d27fc640 1 -- 172.21.15.70:0/1926501730 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48cc070580 con 0x7f48dc0cb450 2023-10-08T05:55:07.736 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:55:07.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.737+0000 7f48e1feb640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f48c0070e30 0x7f48c0073220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 -- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f48c0070e30 msgr2=0x7f48c0073220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f48c0070e30 0x7f48c0073220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 -- 172.21.15.70:0/1926501730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 msgr2=0x7f48dc14cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 0x7f48dc14cbc0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00e9b0 tx=0x7f48cc00ee70 comp rx=0 tx=0).stop 2023-10-08T05:55:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 -- 172.21.15.70:0/1926501730 shutdown_connections 2023-10-08T05:55:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48dc160660 0x7f48dc15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f48c0070e30 0x7f48c0073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48dc0cb450 0x7f48dc14cbc0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 --2- 172.21.15.70:0/1926501730 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48dc070cc0 0x7f48dc15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 -- 172.21.15.70:0/1926501730 >> 172.21.15.70:0/1926501730 conn(0x7f48dc06a5d0 msgr2=0x7f48dc0ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:07.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.738+0000 7f48e424f640 1 -- 172.21.15.70:0/1926501730 shutdown_connections 2023-10-08T05:55:07.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.739+0000 7f48e424f640 1 -- 172.21.15.70:0/1926501730 wait complete. 2023-10-08T05:55:07.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:55:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.964+0000 7f0fe3fff640 1 Processor -- start 2023-10-08T05:55:07.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.964+0000 7f0fe3fff640 1 -- start start 2023-10-08T05:55:07.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.965+0000 7f0fe3fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0fe4071010 0x7f0fe415ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.965+0000 7f0fe3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415d380 0x7f0fe40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.965+0000 7f0fe3fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe415d7d0 con 0x7f0fe4150440 2023-10-08T05:55:07.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.965+0000 7f0fe3fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe40714a0 con 0x7f0fe4071010 2023-10-08T05:55:07.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.965+0000 7f0fe3fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40c9f90 con 0x7f0fe415d380 2023-10-08T05:55:07.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.966+0000 7f0fdbfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0fe4071010 0x7f0fe415ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.966+0000 7f0fe37fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415d380 0x7f0fe40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.966+0000 7f0fdbfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0fe4071010 0x7f0fe415ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:47008/0 (socket says 172.21.15.70:47008) 2023-10-08T05:55:07.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.966+0000 7f0fdbfff640 1 -- 172.21.15.70:0/1672723358 learned_addr learned my addr 172.21.15.70:0/1672723358 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:07.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.967+0000 7f0fdbfff640 1 -- 172.21.15.70:0/1672723358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415d380 msgr2=0x7f0fe40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:07.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.967+0000 7f0fdbfff640 1 --2- 172.21.15.70:0/1672723358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415d380 0x7f0fe40cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.967+0000 7f0fdbfff640 1 -- 172.21.15.70:0/1672723358 >> v1:172.21.15.181:6789/0 conn(0x7f0fe4150440 legacy=0x7f0fe40c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.967+0000 7f0fdbfff640 1 -- 172.21.15.70:0/1672723358 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40ca270 con 0x7f0fe4071010 2023-10-08T05:55:07.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.968+0000 7f0fdbfff640 1 --2- 172.21.15.70:0/1672723358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe4071010 0x7f0fe415ce40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00aa10 tx=0x7f0fcc02ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f7de8c30883c228 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:07.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe27fc640 1 -- 172.21.15.70:0/1672723358 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc02dc10 con 0x7f0fe4071010 2023-10-08T05:55:07.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe27fc640 1 -- 172.21.15.70:0/1672723358 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fcc02dd70 con 0x7f0fe4071010 2023-10-08T05:55:07.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe27fc640 1 -- 172.21.15.70:0/1672723358 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc035660 con 0x7f0fe4071010 2023-10-08T05:55:07.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe3fff640 1 -- 172.21.15.70:0/1672723358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe4071010 msgr2=0x7f0fe415ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe3fff640 1 --2- 172.21.15.70:0/1672723358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe4071010 0x7f0fe415ce40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00aa10 tx=0x7f0fcc02ce60 comp rx=0 tx=0).stop 2023-10-08T05:55:07.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe3fff640 1 -- 172.21.15.70:0/1672723358 shutdown_connections 2023-10-08T05:55:07.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe3fff640 1 --2- 172.21.15.70:0/1672723358 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe415d380 0x7f0fe40cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe3fff640 1 --2- 172.21.15.70:0/1672723358 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe4071010 0x7f0fe415ce40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe3fff640 1 -- 172.21.15.70:0/1672723358 >> 172.21.15.70:0/1672723358 conn(0x7f0fe40cb0c0 msgr2=0x7f0fe40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:07.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe3fff640 1 -- 172.21.15.70:0/1672723358 shutdown_connections 2023-10-08T05:55:07.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.969+0000 7f0fe3fff640 1 -- 172.21.15.70:0/1672723358 wait complete. 2023-10-08T05:55:07.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.971+0000 7f0fe3fff640 1 Processor -- start 2023-10-08T05:55:07.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.971+0000 7f0fe3fff640 1 -- start start 2023-10-08T05:55:07.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.972+0000 7f0fe3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe4071010 0x7f0fe40c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.972+0000 7f0fe3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe4150440 0x7f0fe40c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.972+0000 7f0fe3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe415d380 0x7f0fe40c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.972+0000 7f0fe3fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40c32b0 con 0x7f0fe4150440 2023-10-08T05:55:07.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.972+0000 7f0fe3fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe4155d10 con 0x7f0fe415d380 2023-10-08T05:55:07.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.972+0000 7f0fe3fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe40c2630 con 0x7f0fe4071010 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe4150440 0x7f0fe40c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe4150440 0x7f0fe40c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43832/0 (socket says 172.21.15.70:43832) 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fdbfff640 1 -- 172.21.15.70:0/662916810 learned_addr learned my addr 172.21.15.70:0/662916810 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fe2ffd640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe4071010 0x7f0fe40c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fe37fe640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe415d380 0x7f0fe40c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fdbfff640 1 -- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe4071010 msgr2=0x7f0fe40c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fdbfff640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe4071010 0x7f0fe40c36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fdbfff640 1 -- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe415d380 msgr2=0x7f0fe40c4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fdbfff640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe415d380 0x7f0fe40c4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fdbfff640 1 -- 172.21.15.70:0/662916810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc00a670 con 0x7f0fe4150440 2023-10-08T05:55:07.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.973+0000 7f0fdbfff640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe4150440 0x7f0fe40c45f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc02cea0 tx=0x7f0fcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:07.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.974+0000 7f0fe0ff9640 1 -- 172.21.15.70:0/662916810 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc03f600 con 0x7f0fe4150440 2023-10-08T05:55:07.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.974+0000 7f0fe0ff9640 1 -- 172.21.15.70:0/662916810 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fcc03f780 con 0x7f0fe4150440 2023-10-08T05:55:07.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.974+0000 7f0fe3fff640 1 -- 172.21.15.70:0/662916810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe40c2910 con 0x7f0fe4150440 2023-10-08T05:55:07.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.974+0000 7f0fe0ff9640 1 -- 172.21.15.70:0/662916810 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc03fb30 con 0x7f0fe4150440 2023-10-08T05:55:07.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.974+0000 7f0fe3fff640 1 -- 172.21.15.70:0/662916810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe40c2ed0 con 0x7f0fe4150440 2023-10-08T05:55:07.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.975+0000 7f0fe3fff640 1 -- 172.21.15.70:0/662916810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe40c33f0 con 0x7f0fe4150440 2023-10-08T05:55:07.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.976+0000 7f0fe0ff9640 1 -- 172.21.15.70:0/662916810 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 54) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f0fcc03fc90 con 0x7f0fe4150440 2023-10-08T05:55:07.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.977+0000 7f0fe0ff9640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0fc406acc0 0x7f0fc406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:07.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.977+0000 7f0fe0ff9640 1 -- 172.21.15.70:0/662916810 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6963+0+0 (secure 0 0 0) 0x7f0fcc0abca0 con 0x7f0fe4150440 2023-10-08T05:55:07.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.978+0000 7f0fe2ffd640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0fc406acc0 0x7f0fc406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:07.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.979+0000 7f0fe2ffd640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0fc406acc0 0x7f0fc406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4007fd0 tx=0x7f0fd4009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:07.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:07.982+0000 7f0fe0ff9640 1 -- 172.21.15.70:0/662916810 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0fe40c33f0 con 0x7f0fe4150440 2023-10-08T05:55:08.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.208+0000 7f0fe3fff640 1 -- 172.21.15.70:0/662916810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fe40c33f0 con 0x7f0fe4150440 2023-10-08T05:55:08.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.209+0000 7f0fe0ff9640 1 -- 172.21.15.70:0/662916810 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f0fe40c33f0 con 0x7f0fe4150440 2023-10-08T05:55:08.209 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:08.209 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/290122525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":290122525},{"type":"v1","addr":"172.21.15.70:6835","nonce":290122525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4017950051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4017950051},{"type":"v1","addr":"172.21.15.181:6837","nonce":4017950051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1329101557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1329101557},{"type":"v1","addr":"172.21.15.70:6834","nonce":1329101557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:44.125639+0000","modified":"2023-10-08T05:53:47.206011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6184":{"gid":6184,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/422082772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":422082772},{"type":"v1","addr":"172.21.15.181:6835","nonce":422082772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-08T05:55:08.210 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 50 2023-10-08T05:55:08.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 -- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0fc406acc0 msgr2=0x7f0fc406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0fc406acc0 0x7f0fc406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4007fd0 tx=0x7f0fd4009040 comp rx=0 tx=0).stop 2023-10-08T05:55:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 -- 172.21.15.70:0/662916810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe4150440 msgr2=0x7f0fe40c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe4150440 0x7f0fe40c45f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc02cea0 tx=0x7f0fcc0046c0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 -- 172.21.15.70:0/662916810 shutdown_connections 2023-10-08T05:55:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fe415d380 0x7f0fe40c4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0fc406acc0 0x7f0fc406d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe4150440 0x7f0fe40c45f0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 --2- 172.21.15.70:0/662916810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fe4071010 0x7f0fe40c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 -- 172.21.15.70:0/662916810 >> 172.21.15.70:0/662916810 conn(0x7f0fe40cb0c0 msgr2=0x7f0fe40c97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 -- 172.21.15.70:0/662916810 shutdown_connections 2023-10-08T05:55:08.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.211+0000 7f0fe3fff640 1 -- 172.21.15.70:0/662916810 wait complete. 2023-10-08T05:55:08.230 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.438+0000 7f2a970a2640 1 Processor -- start 2023-10-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.438+0000 7f2a970a2640 1 -- start start 2023-10-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.439+0000 7f2a970a2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2a90159ac0 0x7f2a9006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a970a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a900709e0 0x7f2a900c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a970a2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9006dee0 con 0x7f2a900ad790 2023-10-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a970a2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9015d850 con 0x7f2a90159ac0 2023-10-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a970a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9015db30 con 0x7f2a900709e0 2023-10-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a94e3e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2a90159ac0 0x7f2a9006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a94e3e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2a90159ac0 0x7f2a9006d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43840/0 (socket says 172.21.15.70:43840) 2023-10-08T05:55:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a94e3e640 1 -- 172.21.15.70:0/2724593936 learned_addr learned my addr 172.21.15.70:0/2724593936 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a94e3e640 1 -- 172.21.15.70:0/2724593936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a900709e0 msgr2=0x7f2a900c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a94e3e640 1 --2- 172.21.15.70:0/2724593936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a900709e0 0x7f2a900c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a94e3e640 1 -- 172.21.15.70:0/2724593936 >> v1:172.21.15.181:6789/0 conn(0x7f2a900ad790 legacy=0x7f2a900702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:08.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.440+0000 7f2a94e3e640 1 -- 172.21.15.70:0/2724593936 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9015de10 con 0x7f2a90159ac0 2023-10-08T05:55:08.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.441+0000 7f2a94e3e640 1 --2- 172.21.15.70:0/2724593936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a90159ac0 0x7f2a9006d5e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2a9006e790 tx=0x7f2a8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44481edc24a4cc3b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:08.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.442+0000 7f2a8f7fe640 1 -- 172.21.15.70:0/2724593936 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84031870 con 0x7f2a90159ac0 2023-10-08T05:55:08.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.442+0000 7f2a8f7fe640 1 -- 172.21.15.70:0/2724593936 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a840319d0 con 0x7f2a90159ac0 2023-10-08T05:55:08.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.442+0000 7f2a8f7fe640 1 -- 172.21.15.70:0/2724593936 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84031d60 con 0x7f2a90159ac0 2023-10-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.442+0000 7f2a970a2640 1 -- 172.21.15.70:0/2724593936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a90159ac0 msgr2=0x7f2a9006d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.442+0000 7f2a970a2640 1 --2- 172.21.15.70:0/2724593936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a90159ac0 0x7f2a9006d5e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2a9006e790 tx=0x7f2a8402b910 comp rx=0 tx=0).stop 2023-10-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.442+0000 7f2a970a2640 1 -- 172.21.15.70:0/2724593936 shutdown_connections 2023-10-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.442+0000 7f2a970a2640 1 --2- 172.21.15.70:0/2724593936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a900709e0 0x7f2a900c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.442+0000 7f2a970a2640 1 --2- 172.21.15.70:0/2724593936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a90159ac0 0x7f2a9006d5e0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.442+0000 7f2a970a2640 1 -- 172.21.15.70:0/2724593936 >> 172.21.15.70:0/2724593936 conn(0x7f2a9006a380 msgr2=0x7f2a900b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.443+0000 7f2a970a2640 1 -- 172.21.15.70:0/2724593936 shutdown_connections 2023-10-08T05:55:08.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.443+0000 7f2a970a2640 1 -- 172.21.15.70:0/2724593936 wait complete. 2023-10-08T05:55:08.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.444+0000 7f2a970a2640 1 Processor -- start 2023-10-08T05:55:08.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.444+0000 7f2a970a2640 1 -- start start 2023-10-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.445+0000 7f2a970a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a900709e0 0x7f2a9015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.445+0000 7f2a970a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a900ad790 0x7f2a90166040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.445+0000 7f2a970a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a90159ac0 0x7f2a901422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.445+0000 7f2a970a2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90166580 con 0x7f2a900ad790 2023-10-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.445+0000 7f2a970a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90166820 con 0x7f2a900709e0 2023-10-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.445+0000 7f2a970a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a90166b00 con 0x7f2a90159ac0 2023-10-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a900ad790 0x7f2a90166040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a900ad790 0x7f2a90166040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43856/0 (socket says 172.21.15.70:43856) 2023-10-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a8ffff640 1 -- 172.21.15.70:0/2900612826 learned_addr learned my addr 172.21.15.70:0/2900612826 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a94e3e640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a900709e0 0x7f2a9015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a9563f640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a90159ac0 0x7f2a901422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a8ffff640 1 -- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a90159ac0 msgr2=0x7f2a901422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a8ffff640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a90159ac0 0x7f2a901422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a8ffff640 1 -- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a900709e0 msgr2=0x7f2a9015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a8ffff640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a900709e0 0x7f2a9015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a8ffff640 1 -- 172.21.15.70:0/2900612826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a840094d0 con 0x7f2a900ad790 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.446+0000 7f2a8ffff640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a900ad790 0x7f2a90166040 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c410 tx=0x7f2a7c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.447+0000 7f2a8d7fa640 1 -- 172.21.15.70:0/2900612826 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c0035f0 con 0x7f2a900ad790 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.447+0000 7f2a8d7fa640 1 -- 172.21.15.70:0/2900612826 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c003750 con 0x7f2a900ad790 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.447+0000 7f2a8d7fa640 1 -- 172.21.15.70:0/2900612826 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c003b40 con 0x7f2a900ad790 2023-10-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.447+0000 7f2a970a2640 1 -- 172.21.15.70:0/2900612826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a90166de0 con 0x7f2a900ad790 2023-10-08T05:55:08.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.447+0000 7f2a970a2640 1 -- 172.21.15.70:0/2900612826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a901673a0 con 0x7f2a900ad790 2023-10-08T05:55:08.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.448+0000 7f2a970a2640 1 -- 172.21.15.70:0/2900612826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a90167590 con 0x7f2a900ad790 2023-10-08T05:55:08.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.449+0000 7f2a8d7fa640 1 -- 172.21.15.70:0/2900612826 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 54) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f2a7c003ca0 con 0x7f2a900ad790 2023-10-08T05:55:08.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.450+0000 7f2a8d7fa640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a7406acf0 0x7f2a7406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.450+0000 7f2a8d7fa640 1 -- 172.21.15.70:0/2900612826 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6963+0+0 (secure 0 0 0) 0x7f2a7c088ac0 con 0x7f2a900ad790 2023-10-08T05:55:08.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.451+0000 7f2a94e3e640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a7406acf0 0x7f2a7406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:08.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.452+0000 7f2a94e3e640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a7406acf0 0x7f2a7406d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a84036000 tx=0x7f2a8403c040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:08.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.454+0000 7f2a8d7fa640 1 -- 172.21.15.70:0/2900612826 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a7c053860 con 0x7f2a900ad790 2023-10-08T05:55:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.647+0000 7f2a970a2640 1 -- 172.21.15.70:0/2900612826 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a90067490 con 0x7f2a900ad790 2023-10-08T05:55:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.651+0000 7f2a8d7fa640 1 -- 172.21.15.70:0/2900612826 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+18123 (secure 0 0 0) 0x7f2a7c056e80 con 0x7f2a900ad790 2023-10-08T05:55:08.652 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:08.652 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":122,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:06.712251+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:42.052166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:43.161991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/1714252273":"2023-10-09T05:55:06.712118+0000","172.21.15.181:6832/1714252273":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1721597867":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1614829749":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/2367948918":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/3835712371":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/2025529952":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1108570452":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1700228937":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/3426004759":"2023-10-09T05:55:06.712118+0000","172.21.15.181:6833/1906631919":"2023-10-09T05:53:58.959064+0000","172.21.15.181:6832/1906631919":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3209826772":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/1283682257":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/2590863547":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3077797034":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3436033120":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/168987985":"2023-10-09T05:53:58.959064+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:55:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 -- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a7406acf0 msgr2=0x7f2a7406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a7406acf0 0x7f2a7406d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a84036000 tx=0x7f2a8403c040 comp rx=0 tx=0).stop 2023-10-08T05:55:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 -- 172.21.15.70:0/2900612826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a900ad790 msgr2=0x7f2a90166040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a900ad790 0x7f2a90166040 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c410 tx=0x7f2a7c00c8d0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 -- 172.21.15.70:0/2900612826 shutdown_connections 2023-10-08T05:55:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a90159ac0 0x7f2a901422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a7406acf0 0x7f2a7406d1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a900ad790 0x7f2a90166040 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 --2- 172.21.15.70:0/2900612826 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a900709e0 0x7f2a9015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 -- 172.21.15.70:0/2900612826 >> 172.21.15.70:0/2900612826 conn(0x7f2a9006a380 msgr2=0x7f2a900aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 -- 172.21.15.70:0/2900612826 shutdown_connections 2023-10-08T05:55:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.653+0000 7f2a970a2640 1 -- 172.21.15.70:0/2900612826 wait complete. 2023-10-08T05:55:08.673 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T05:55:08.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:55:08.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.881+0000 7f591b7b6640 1 Processor -- start 2023-10-08T05:55:08.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.881+0000 7f591b7b6640 1 -- start start 2023-10-08T05:55:08.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.882+0000 7f591b7b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f591415ab40 0x7f59140ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.882+0000 7f591b7b6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59140cb580 0x7f59140adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.882+0000 7f591b7b6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591415b2b0 con 0x7f59140c90b0 2023-10-08T05:55:08.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.882+0000 7f591b7b6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591415bf30 con 0x7f59140cb580 2023-10-08T05:55:08.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.882+0000 7f591b7b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591415f7a0 con 0x7f591415ab40 2023-10-08T05:55:08.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.883+0000 7f5919d53640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59140cb580 0x7f59140adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:08.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.883+0000 7f5919d53640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59140cb580 0x7f59140adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43864/0 (socket says 172.21.15.70:43864) 2023-10-08T05:55:08.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.883+0000 7f5919d53640 1 -- 172.21.15.70:0/1768962125 learned_addr learned my addr 172.21.15.70:0/1768962125 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:08.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.883+0000 7f5919552640 1 --2- 172.21.15.70:0/1768962125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f591415ab40 0x7f59140ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:08.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.883+0000 7f5919d53640 1 -- 172.21.15.70:0/1768962125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f591415ab40 msgr2=0x7f59140ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:08.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.883+0000 7f5919d53640 1 --2- 172.21.15.70:0/1768962125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f591415ab40 0x7f59140ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.883+0000 7f5919d53640 1 -- 172.21.15.70:0/1768962125 >> v1:172.21.15.70:6789/0 conn(0x7f59140c90b0 legacy=0x7f59140cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.883+0000 7f5919d53640 1 -- 172.21.15.70:0/1768962125 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591415b590 con 0x7f59140cb580 2023-10-08T05:55:08.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.884+0000 7f5919d53640 1 --2- 172.21.15.70:0/1768962125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140cb580 0x7f59140adc60 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f590c009e50 tx=0x7f590c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2da180ccd5c5f68 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:08.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.885+0000 7f590bfff640 1 -- 172.21.15.70:0/1768962125 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c02cb00 con 0x7f59140cb580 2023-10-08T05:55:08.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.885+0000 7f590bfff640 1 -- 172.21.15.70:0/1768962125 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c02cc60 con 0x7f59140cb580 2023-10-08T05:55:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.885+0000 7f590bfff640 1 -- 172.21.15.70:0/1768962125 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c0343f0 con 0x7f59140cb580 2023-10-08T05:55:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.885+0000 7f591b7b6640 1 -- 172.21.15.70:0/1768962125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140cb580 msgr2=0x7f59140adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.885+0000 7f591b7b6640 1 --2- 172.21.15.70:0/1768962125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140cb580 0x7f59140adc60 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f590c009e50 tx=0x7f590c02bf60 comp rx=0 tx=0).stop 2023-10-08T05:55:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.885+0000 7f591b7b6640 1 -- 172.21.15.70:0/1768962125 shutdown_connections 2023-10-08T05:55:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.885+0000 7f591b7b6640 1 --2- 172.21.15.70:0/1768962125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140cb580 0x7f59140adc60 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.885+0000 7f591b7b6640 1 --2- 172.21.15.70:0/1768962125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f591415ab40 0x7f59140ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.885+0000 7f591b7b6640 1 -- 172.21.15.70:0/1768962125 >> 172.21.15.70:0/1768962125 conn(0x7f591406a380 msgr2=0x7f59140b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.886+0000 7f591b7b6640 1 -- 172.21.15.70:0/1768962125 shutdown_connections 2023-10-08T05:55:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.886+0000 7f591b7b6640 1 -- 172.21.15.70:0/1768962125 wait complete. 2023-10-08T05:55:08.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.887+0000 7f591b7b6640 1 Processor -- start 2023-10-08T05:55:08.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.887+0000 7f591b7b6640 1 -- start start 2023-10-08T05:55:08.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.888+0000 7f591b7b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140c90b0 0x7f59141f0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.888+0000 7f591b7b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59140cb580 0x7f591414de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.888+0000 7f591b7b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f591415ab40 0x7f591415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.888+0000 7f591b7b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59140c7ed0 con 0x7f59140c90b0 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.888+0000 7f591b7b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591406da50 con 0x7f591415ab40 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.888+0000 7f591b7b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591406df40 con 0x7f59140cb580 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5919552640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140c90b0 0x7f59141f0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5919552640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140c90b0 0x7f59141f0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43868/0 (socket says 172.21.15.70:43868) 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5919552640 1 -- 172.21.15.70:0/2421186854 learned_addr learned my addr 172.21.15.70:0/2421186854 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5918d51640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59140cb580 0x7f591414de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5919552640 1 -- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59140cb580 msgr2=0x7f591414de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5919552640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59140cb580 0x7f591414de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5919552640 1 -- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f591415ab40 msgr2=0x7f591415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5919552640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f591415ab40 0x7f591415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5919552640 1 -- 172.21.15.70:0/2421186854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c009b30 con 0x7f59140c90b0 2023-10-08T05:55:08.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5918d51640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59140cb580 0x7f591414de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:08.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5919552640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140c90b0 0x7f59141f0020 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f591000d830 tx=0x7f591000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.889+0000 7f5909ffb640 1 -- 172.21.15.70:0/2421186854 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591000b6c0 con 0x7f59140c90b0 2023-10-08T05:55:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.890+0000 7f5909ffb640 1 -- 172.21.15.70:0/2421186854 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591000b820 con 0x7f59140c90b0 2023-10-08T05:55:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.890+0000 7f5909ffb640 1 -- 172.21.15.70:0/2421186854 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591000bbb0 con 0x7f59140c90b0 2023-10-08T05:55:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.890+0000 7f591b7b6640 1 -- 172.21.15.70:0/2421186854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59141f0560 con 0x7f59140c90b0 2023-10-08T05:55:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.890+0000 7f591b7b6640 1 -- 172.21.15.70:0/2421186854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591406dc60 con 0x7f59140c90b0 2023-10-08T05:55:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.891+0000 7f591b7b6640 1 -- 172.21.15.70:0/2421186854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59140cdaa0 con 0x7f59140c90b0 2023-10-08T05:55:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.892+0000 7f5909ffb640 1 -- 172.21.15.70:0/2421186854 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 55) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f591001e460 con 0x7f59140c90b0 2023-10-08T05:55:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.892+0000 7f5909ffb640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f58f806af00 0x7f58f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.893+0000 7f5909ffb640 1 -- 172.21.15.70:0/2421186854 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6963+0+0 (secure 0 0 0) 0x7f591008a320 con 0x7f59140c90b0 2023-10-08T05:55:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.893+0000 7f5918d51640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f58f806af00 0x7f58f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:08.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.894+0000 7f5918d51640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f58f806af00 0x7f58f806d3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f591406d9a0 tx=0x7f5904009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:08.897+0000 7f5909ffb640 1 -- 172.21.15.70:0/2421186854 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5910054e70 con 0x7f59140c90b0 2023-10-08T05:55:09.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.125+0000 7f591b7b6640 1 -- 172.21.15.70:0/2421186854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5914067490 con 0x7f59140c90b0 2023-10-08T05:55:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.126+0000 7f5909ffb640 1 -- 172.21.15.70:0/2421186854 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f5910058490 con 0x7f59140c90b0 2023-10-08T05:55:09.127 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:09.127 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/290122525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":290122525},{"type":"v1","addr":"172.21.15.70:6835","nonce":290122525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4017950051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4017950051},{"type":"v1","addr":"172.21.15.181:6837","nonce":4017950051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1329101557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1329101557},{"type":"v1","addr":"172.21.15.70:6834","nonce":1329101557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:44.125639+0000","modified":"2023-10-08T05:53:47.206011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6184":{"gid":6184,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/422082772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":422082772},{"type":"v1","addr":"172.21.15.181:6835","nonce":422082772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-08T05:55:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 50 2023-10-08T05:55:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 -- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f58f806af00 msgr2=0x7f58f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f58f806af00 0x7f58f806d3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f591406d9a0 tx=0x7f5904009040 comp rx=0 tx=0).stop 2023-10-08T05:55:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 -- 172.21.15.70:0/2421186854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140c90b0 msgr2=0x7f59141f0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140c90b0 0x7f59141f0020 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f591000d830 tx=0x7f591000dcf0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 -- 172.21.15.70:0/2421186854 shutdown_connections 2023-10-08T05:55:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f591415ab40 0x7f591415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59140cb580 0x7f591414de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f58f806af00 0x7f58f806d3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 --2- 172.21.15.70:0/2421186854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59140c90b0 0x7f59141f0020 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 -- 172.21.15.70:0/2421186854 >> 172.21.15.70:0/2421186854 conn(0x7f591406a380 msgr2=0x7f59140cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:09.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 -- 172.21.15.70:0/2421186854 shutdown_connections 2023-10-08T05:55:09.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.129+0000 7f591b7b6640 1 -- 172.21.15.70:0/2421186854 wait complete. 2023-10-08T05:55:09.148 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:55:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.365+0000 7f871359e640 1 Processor -- start 2023-10-08T05:55:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.365+0000 7f871359e640 1 -- start start 2023-10-08T05:55:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.366+0000 7f871359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871415ab30 0x7f871415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.366+0000 7f871359e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87140adc70 0x7f87140b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.366+0000 7f871359e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871415f7d0 con 0x7f871415af10 2023-10-08T05:55:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.366+0000 7f871359e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871415f210 con 0x7f87140adc70 2023-10-08T05:55:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.366+0000 7f871359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871415f4f0 con 0x7f871415ab30 2023-10-08T05:55:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8712d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f871415af10 0x7f87140701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41942/0 (socket says 172.21.15.70:41942) 2023-10-08T05:55:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8712d9d640 1 -- 172.21.15.70:0/2435384518 learned_addr learned my addr 172.21.15.70:0/2435384518 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f871259c640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871415ab30 0x7f871415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f870bfff640 1 --2- 172.21.15.70:0/2435384518 >> v2:172.21.15.181:3300/0 conn(0x7f87140adc70 0x7f87140b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632053492 0 0) 0x7f871415f7d0 con 0x7f871415af10 2023-10-08T05:55:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8700003520 con 0x7f871415af10 2023-10-08T05:55:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3733477126 0 0) 0x7f8700003520 con 0x7f871415af10 2023-10-08T05:55:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871415ab30 msgr2=0x7f871415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8711d9b640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871415ab30 0x7f871415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 >> v2:172.21.15.181:3300/0 conn(0x7f87140adc70 msgr2=0x7f87140b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8711d9b640 1 --2- 172.21.15.70:0/2435384518 >> v2:172.21.15.181:3300/0 conn(0x7f87140adc70 0x7f87140b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871406c550 con 0x7f871415af10 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.367+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f870c001a10 con 0x7f871415af10 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 >> v1:172.21.15.70:6789/0 conn(0x7f871415af10 legacy=0x7f87140701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8711d9b640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871415ab30 0x7f8700003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8711d9b640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8700003ef0 0x7f87000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8711d9b640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 0x7f8700005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871415f210 con 0x7f8700004d00 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871415f4f0 con 0x7f8700003ef0 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870000b980 con 0x7f871415ab30 2023-10-08T05:55:09.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8712d9d640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 0x7f8700005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f870bfff640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8700003ef0 0x7f87000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8712d9d640 1 -- 172.21.15.70:0/2435384518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871415ab30 msgr2=0x7f8700003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8712d9d640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871415ab30 0x7f8700003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8712d9d640 1 -- 172.21.15.70:0/2435384518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8700003ef0 msgr2=0x7f87000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8712d9d640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8700003ef0 0x7f87000047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.368+0000 7f8712d9d640 1 -- 172.21.15.70:0/2435384518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8700003520 con 0x7f8700004d00 2023-10-08T05:55:09.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.369+0000 7f8712d9d640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 0x7f8700005180 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f870c00a850 tx=0x7f87000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14901651ad3d93c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.370+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c02cea0 con 0x7f8700004d00 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.370+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c02f070 con 0x7f8700004d00 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.370+0000 7f8711d9b640 1 -- 172.21.15.70:0/2435384518 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c034870 con 0x7f8700004d00 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.370+0000 7f871359e640 1 -- 172.21.15.70:0/2435384518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 msgr2=0x7f8700005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.370+0000 7f871359e640 1 --2- 172.21.15.70:0/2435384518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 0x7f8700005180 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f870c00a850 tx=0x7f87000057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.370+0000 7f8712d9d640 1 -- 172.21.15.70:0/2435384518 reap_dead start 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.370+0000 7f871359e640 1 -- 172.21.15.70:0/2435384518 shutdown_connections 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.370+0000 7f871359e640 1 -- 172.21.15.70:0/2435384518 >> 172.21.15.70:0/2435384518 conn(0x7f87140cb0c0 msgr2=0x7f87140cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.370+0000 7f871359e640 1 -- 172.21.15.70:0/2435384518 shutdown_connections 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.370+0000 7f871359e640 1 -- 172.21.15.70:0/2435384518 wait complete. 2023-10-08T05:55:09.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.372+0000 7f871359e640 1 Processor -- start 2023-10-08T05:55:09.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.372+0000 7f871359e640 1 -- start start 2023-10-08T05:55:09.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.373+0000 7f871359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 0x7f87140ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.373+0000 7f871359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871414fc30 0x7f87140b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.373+0000 7f871359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8714154aa0 0x7f8714154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.373+0000 7f871359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8714155460 con 0x7f8700004d00 2023-10-08T05:55:09.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.373+0000 7f871359e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87140ab970 con 0x7f8714154aa0 2023-10-08T05:55:09.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.373+0000 7f871359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87140abc50 con 0x7f871414fc30 2023-10-08T05:55:09.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.374+0000 7f871259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 0x7f87140ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.374+0000 7f871259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 0x7f87140ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43896/0 (socket says 172.21.15.70:43896) 2023-10-08T05:55:09.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.374+0000 7f871259c640 1 -- 172.21.15.70:0/2272265048 learned_addr learned my addr 172.21.15.70:0/2272265048 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:09.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.374+0000 7f870bfff640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871414fc30 0x7f87140b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.374+0000 7f871259c640 1 -- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871414fc30 msgr2=0x7f87140b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.374+0000 7f871259c640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871414fc30 0x7f87140b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.374+0000 7f871259c640 1 -- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8714154aa0 msgr2=0x7f8714154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.374+0000 7f871259c640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8714154aa0 0x7f8714154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.374+0000 7f871259c640 1 -- 172.21.15.70:0/2272265048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87140abf30 con 0x7f8700004d00 2023-10-08T05:55:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.374+0000 7f871259c640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 0x7f87140ae710 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f870400e7f0 tx=0x7f870400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.375+0000 7f870affd640 1 -- 172.21.15.70:0/2272265048 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870400eea0 con 0x7f8700004d00 2023-10-08T05:55:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.375+0000 7f870affd640 1 -- 172.21.15.70:0/2272265048 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870400c8c0 con 0x7f8700004d00 2023-10-08T05:55:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.375+0000 7f870affd640 1 -- 172.21.15.70:0/2272265048 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870400cc50 con 0x7f8700004d00 2023-10-08T05:55:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.375+0000 7f871359e640 1 -- 172.21.15.70:0/2272265048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87140ac210 con 0x7f8700004d00 2023-10-08T05:55:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.375+0000 7f871359e640 1 -- 172.21.15.70:0/2272265048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87140aec50 con 0x7f8700004d00 2023-10-08T05:55:09.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.376+0000 7f871359e640 1 -- 172.21.15.70:0/2272265048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87140aee40 con 0x7f8700004d00 2023-10-08T05:55:09.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.377+0000 7f870affd640 1 -- 172.21.15.70:0/2272265048 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 55) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f8704013470 con 0x7f8700004d00 2023-10-08T05:55:09.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.378+0000 7f870affd640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f86f406af30 0x7f86f406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.378+0000 7f870affd640 1 -- 172.21.15.70:0/2272265048 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6963+0+0 (secure 0 0 0) 0x7f8704089ad0 con 0x7f8700004d00 2023-10-08T05:55:09.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.379+0000 7f870bfff640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f86f406af30 0x7f86f406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.380+0000 7f870bfff640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f86f406af30 0x7f86f406d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f87141f6290 tx=0x7f86fc008040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:09.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.383+0000 7f870affd640 1 -- 172.21.15.70:0/2272265048 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87140aee40 con 0x7f8700004d00 2023-10-08T05:55:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.610+0000 7f871359e640 1 -- 172.21.15.70:0/2272265048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87140aee40 con 0x7f8700004d00 2023-10-08T05:55:09.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.611+0000 7f870affd640 1 -- 172.21.15.70:0/2272265048 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f87140aee40 con 0x7f8700004d00 2023-10-08T05:55:09.612 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:09.612 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":50,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/290122525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":290122525},{"type":"v1","addr":"172.21.15.70:6835","nonce":290122525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4017950051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4017950051},{"type":"v1","addr":"172.21.15.181:6837","nonce":4017950051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1329101557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1329101557},{"type":"v1","addr":"172.21.15.70:6834","nonce":1329101557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:53:44.125639+0000","modified":"2023-10-08T05:53:47.206011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6184":{"gid":6184,"name":"b","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.181:6835/422082772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":422082772},{"type":"v1","addr":"172.21.15.181:6835","nonce":422082772}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-10-08T05:55:09.612 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 50 2023-10-08T05:55:09.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.614+0000 7f871359e640 1 -- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f86f406af30 msgr2=0x7f86f406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.614+0000 7f871359e640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f86f406af30 0x7f86f406d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f87141f6290 tx=0x7f86fc008040 comp rx=0 tx=0).stop 2023-10-08T05:55:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.614+0000 7f871359e640 1 -- 172.21.15.70:0/2272265048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 msgr2=0x7f87140ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.614+0000 7f871359e640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 0x7f87140ae710 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f870400e7f0 tx=0x7f870400ecb0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.615+0000 7f871359e640 1 -- 172.21.15.70:0/2272265048 shutdown_connections 2023-10-08T05:55:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.615+0000 7f871359e640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8714154aa0 0x7f8714154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.615+0000 7f871359e640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f871414fc30 0x7f87140b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.615+0000 7f871359e640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f86f406af30 0x7f86f406d3f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.615+0000 7f871359e640 1 --2- 172.21.15.70:0/2272265048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8700004d00 0x7f87140ae710 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.615+0000 7f871359e640 1 -- 172.21.15.70:0/2272265048 >> 172.21.15.70:0/2272265048 conn(0x7f87140cb0c0 msgr2=0x7f87140ce560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.615+0000 7f871359e640 1 -- 172.21.15.70:0/2272265048 shutdown_connections 2023-10-08T05:55:09.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.615+0000 7f871359e640 1 -- 172.21.15.70:0/2272265048 wait complete. 2023-10-08T05:55:09.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:55:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.848+0000 7fab08b9e640 1 Processor -- start 2023-10-08T05:55:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.848+0000 7fab08b9e640 1 -- start start 2023-10-08T05:55:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.850+0000 7fab08b9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab0415ab40 0x7fab040ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.850+0000 7fab08b9e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fab040c90b0 0x7fab040adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.850+0000 7fab08b9e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab040cae70 0x7fab040cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.850+0000 7fab08b9e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0415bf30 con 0x7fab040cae70 2023-10-08T05:55:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.850+0000 7fab08b9e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab040cb840 con 0x7fab040c90b0 2023-10-08T05:55:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.850+0000 7fab08b9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0415b2b0 con 0x7fab0415ab40 2023-10-08T05:55:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.850+0000 7fab02d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab040cae70 0x7fab040cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.851+0000 7fab02d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab040cae70 0x7fab040cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43904/0 (socket says 172.21.15.70:43904) 2023-10-08T05:55:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.851+0000 7fab02d9d640 1 -- 172.21.15.70:0/185799215 learned_addr learned my addr 172.21.15.70:0/185799215 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.851+0000 7fab01d9b640 1 --2- 172.21.15.70:0/185799215 >> v2:172.21.15.181:3300/0 conn(0x7fab040c90b0 0x7fab040adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.851+0000 7fab0259c640 1 --2- 172.21.15.70:0/185799215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab0415ab40 0x7fab040ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.851+0000 7fab02d9d640 1 -- 172.21.15.70:0/185799215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab0415ab40 msgr2=0x7fab040ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:09.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.851+0000 7fab02d9d640 1 --2- 172.21.15.70:0/185799215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab0415ab40 0x7fab040ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.851+0000 7fab02d9d640 1 -- 172.21.15.70:0/185799215 >> v2:172.21.15.181:3300/0 conn(0x7fab040c90b0 msgr2=0x7fab040adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:09.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.851+0000 7fab02d9d640 1 --2- 172.21.15.70:0/185799215 >> v2:172.21.15.181:3300/0 conn(0x7fab040c90b0 0x7fab040adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.851+0000 7fab02d9d640 1 -- 172.21.15.70:0/185799215 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0415b590 con 0x7fab040cae70 2023-10-08T05:55:09.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.852+0000 7fab02d9d640 1 --2- 172.21.15.70:0/185799215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab040cae70 0x7fab040cb300 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fab0406e470 tx=0x7faafc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d30a8a4567567a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:09.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.852+0000 7fab0159a640 1 -- 172.21.15.70:0/185799215 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc031860 con 0x7fab040cae70 2023-10-08T05:55:09.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.852+0000 7fab0159a640 1 -- 172.21.15.70:0/185799215 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc0319c0 con 0x7fab040cae70 2023-10-08T05:55:09.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.852+0000 7fab0159a640 1 -- 172.21.15.70:0/185799215 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc031d50 con 0x7fab040cae70 2023-10-08T05:55:09.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.853+0000 7fab08b9e640 1 -- 172.21.15.70:0/185799215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab040cae70 msgr2=0x7fab040cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.853+0000 7fab08b9e640 1 --2- 172.21.15.70:0/185799215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab040cae70 0x7fab040cb300 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fab0406e470 tx=0x7faafc02b900 comp rx=0 tx=0).stop 2023-10-08T05:55:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.853+0000 7fab08b9e640 1 -- 172.21.15.70:0/185799215 shutdown_connections 2023-10-08T05:55:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.853+0000 7fab08b9e640 1 --2- 172.21.15.70:0/185799215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab040cae70 0x7fab040cb300 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.853+0000 7fab08b9e640 1 --2- 172.21.15.70:0/185799215 >> v2:172.21.15.181:3300/0 conn(0x7fab040c90b0 0x7fab040adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.853+0000 7fab08b9e640 1 --2- 172.21.15.70:0/185799215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab0415ab40 0x7fab040ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.853+0000 7fab08b9e640 1 -- 172.21.15.70:0/185799215 >> 172.21.15.70:0/185799215 conn(0x7fab0406a380 msgr2=0x7fab040b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.853+0000 7fab08b9e640 1 -- 172.21.15.70:0/185799215 shutdown_connections 2023-10-08T05:55:09.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.853+0000 7fab08b9e640 1 -- 172.21.15.70:0/185799215 wait complete. 2023-10-08T05:55:09.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.855+0000 7fab08b9e640 1 Processor -- start 2023-10-08T05:55:09.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.855+0000 7fab08b9e640 1 -- start start 2023-10-08T05:55:09.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.856+0000 7fab08b9e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab040c90b0 0x7fab040b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.856+0000 7fab08b9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab040cae70 0x7fab0414b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.856+0000 7fab08b9e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0415ab40 0x7fab04158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.856+0000 7fab08b9e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040b3be0 con 0x7fab0415ab40 2023-10-08T05:55:09.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.856+0000 7fab08b9e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab040c5620 con 0x7fab040c90b0 2023-10-08T05:55:09.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.856+0000 7fab08b9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab040b1b40 con 0x7fab040cae70 2023-10-08T05:55:09.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.856+0000 7fab02d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0415ab40 0x7fab04158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.856+0000 7fab02d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0415ab40 0x7fab04158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43908/0 (socket says 172.21.15.70:43908) 2023-10-08T05:55:09.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.856+0000 7fab02d9d640 1 -- 172.21.15.70:0/4060440753 learned_addr learned my addr 172.21.15.70:0/4060440753 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:09.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7fab02d9d640 1 -- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab040cae70 msgr2=0x7fab0414b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7fab02d9d640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab040cae70 0x7fab0414b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7fab0259c640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab040c90b0 0x7fab040b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7fab02d9d640 1 -- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab040c90b0 msgr2=0x7fab040b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:09.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7fab02d9d640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab040c90b0 0x7fab040b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:09.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7fab02d9d640 1 -- 172.21.15.70:0/4060440753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc0094d0 con 0x7fab0415ab40 2023-10-08T05:55:09.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7fab02d9d640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0415ab40 0x7fab04158fc0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7faafc0090b0 tx=0x7faafc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:09.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7faafaffd640 1 -- 172.21.15.70:0/4060440753 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc03ac60 con 0x7fab0415ab40 2023-10-08T05:55:09.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7faafaffd640 1 -- 172.21.15.70:0/4060440753 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc03adc0 con 0x7fab0415ab40 2023-10-08T05:55:09.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7faafaffd640 1 -- 172.21.15.70:0/4060440753 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc044630 con 0x7fab0415ab40 2023-10-08T05:55:09.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.857+0000 7fab08b9e640 1 -- 172.21.15.70:0/4060440753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab040b1910 con 0x7fab0415ab40 2023-10-08T05:55:09.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.858+0000 7fab08b9e640 1 -- 172.21.15.70:0/4060440753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0414b8b0 con 0x7fab0415ab40 2023-10-08T05:55:09.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.859+0000 7fab08b9e640 1 -- 172.21.15.70:0/4060440753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab040b3d20 con 0x7fab0415ab40 2023-10-08T05:55:09.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.859+0000 7faafaffd640 1 -- 172.21.15.70:0/4060440753 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 55) v1 ==== 89275+0+0 (secure 0 0 0) 0x7faafc04d450 con 0x7fab0415ab40 2023-10-08T05:55:09.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.861+0000 7faafaffd640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7faae406af00 0x7faae406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:09.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.861+0000 7faafaffd640 1 -- 172.21.15.70:0/4060440753 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6963+0+0 (secure 0 0 0) 0x7faafc0ac280 con 0x7fab0415ab40 2023-10-08T05:55:09.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.862+0000 7fab0259c640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7faae406af00 0x7faae406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:09.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.863+0000 7fab0259c640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7faae406af00 0x7faae406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009f30 tx=0x7faaf400b040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:09.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:09.865+0000 7faafaffd640 1 -- 172.21.15.70:0/4060440753 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab040b3d20 con 0x7fab0415ab40 2023-10-08T05:55:10.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.055+0000 7fab08b9e640 1 -- 172.21.15.70:0/4060440753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab040b3d20 con 0x7fab0415ab40 2023-10-08T05:55:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.057+0000 7faafaffd640 1 -- 172.21.15.70:0/4060440753 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+18123 (secure 0 0 0) 0x7fab040b3d20 con 0x7fab0415ab40 2023-10-08T05:55:10.058 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:10.059 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":122,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:06.712251+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:53:42.052166+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-08T05:53:43.161991+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":2}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.3c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/1714252273":"2023-10-09T05:55:06.712118+0000","172.21.15.181:6832/1714252273":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1721597867":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1614829749":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/2367948918":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/3835712371":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/2025529952":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1108570452":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1700228937":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/3426004759":"2023-10-09T05:55:06.712118+0000","172.21.15.181:6833/1906631919":"2023-10-09T05:53:58.959064+0000","172.21.15.181:6832/1906631919":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3209826772":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/1283682257":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/2590863547":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3077797034":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3436033120":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/168987985":"2023-10-09T05:53:58.959064+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:55:10.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.060+0000 7fab08b9e640 1 -- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7faae406af00 msgr2=0x7faae406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.060+0000 7fab08b9e640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7faae406af00 0x7faae406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009f30 tx=0x7faaf400b040 comp rx=0 tx=0).stop 2023-10-08T05:55:10.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.060+0000 7fab08b9e640 1 -- 172.21.15.70:0/4060440753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0415ab40 msgr2=0x7fab04158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.060+0000 7fab08b9e640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0415ab40 0x7fab04158fc0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7faafc0090b0 tx=0x7faafc0046c0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.060+0000 7fab08b9e640 1 -- 172.21.15.70:0/4060440753 shutdown_connections 2023-10-08T05:55:10.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.060+0000 7fab08b9e640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7faae406af00 0x7faae406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.060+0000 7fab08b9e640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0415ab40 0x7fab04158fc0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.060+0000 7fab08b9e640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab040cae70 0x7fab0414b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.060+0000 7fab08b9e640 1 --2- 172.21.15.70:0/4060440753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab040c90b0 0x7fab040b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.060+0000 7fab08b9e640 1 -- 172.21.15.70:0/4060440753 >> 172.21.15.70:0/4060440753 conn(0x7fab0406a380 msgr2=0x7fab040c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:10.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.061+0000 7fab08b9e640 1 -- 172.21.15.70:0/4060440753 shutdown_connections 2023-10-08T05:55:10.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.061+0000 7fab08b9e640 1 -- 172.21.15.70:0/4060440753 wait complete. 2023-10-08T05:55:10.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T05:55:10.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.300+0000 7fa2ae0d7640 1 Processor -- start 2023-10-08T05:55:10.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.301+0000 7fa2ae0d7640 1 -- start start 2023-10-08T05:55:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.302+0000 7fa2ae0d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa2a80afbe0 0x7fa2a80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.302+0000 7fa2ae0d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a806dfc0 0x7fa2a8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.302+0000 7fa2ae0d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa2a8071070 0x7fa2a80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.302+0000 7fa2ae0d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a80b2620 con 0x7fa2a8071070 2023-10-08T05:55:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.302+0000 7fa2ae0d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a80b2060 con 0x7fa2a80afbe0 2023-10-08T05:55:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.302+0000 7fa2ae0d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80b2340 con 0x7fa2a806dfc0 2023-10-08T05:55:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.302+0000 7fa2a7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa2a8071070 0x7fa2a80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.302+0000 7fa2a7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa2a8071070 0x7fa2a80cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43910/0 (socket says 172.21.15.70:43910) 2023-10-08T05:55:10.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.302+0000 7fa2a7fff640 1 -- 172.21.15.70:0/1279143217 learned_addr learned my addr 172.21.15.70:0/1279143217 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.303+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/1279143217 >> v2:172.21.15.181:3300/0 conn(0x7fa2a80afbe0 0x7fa2a80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.303+0000 7fa29ffff640 1 --2- 172.21.15.70:0/1279143217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a806dfc0 0x7fa2a8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.303+0000 7fa2a7fff640 1 -- 172.21.15.70:0/1279143217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a806dfc0 msgr2=0x7fa2a8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.303+0000 7fa2a7fff640 1 --2- 172.21.15.70:0/1279143217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a806dfc0 0x7fa2a8070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.303+0000 7fa2a7fff640 1 -- 172.21.15.70:0/1279143217 >> v2:172.21.15.181:3300/0 conn(0x7fa2a80afbe0 msgr2=0x7fa2a80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.303+0000 7fa2a7fff640 1 --2- 172.21.15.70:0/1279143217 >> v2:172.21.15.181:3300/0 conn(0x7fa2a80afbe0 0x7fa2a80b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.303+0000 7fa2a7fff640 1 -- 172.21.15.70:0/1279143217 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a8070350 con 0x7fa2a8071070 2023-10-08T05:55:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.303+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/1279143217 >> v2:172.21.15.181:3300/0 conn(0x7fa2a80afbe0 0x7fa2a80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.304+0000 7fa2a7fff640 1 --2- 172.21.15.70:0/1279143217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a8071070 0x7fa2a80cb130 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa2a806e640 tx=0x7fa2a002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e99a637389f2a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.304+0000 7fa2a6ffd640 1 -- 172.21.15.70:0/1279143217 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a002ca80 con 0x7fa2a8071070 2023-10-08T05:55:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.304+0000 7fa2a6ffd640 1 -- 172.21.15.70:0/1279143217 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a002cbe0 con 0x7fa2a8071070 2023-10-08T05:55:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.304+0000 7fa2a6ffd640 1 -- 172.21.15.70:0/1279143217 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00343f0 con 0x7fa2a8071070 2023-10-08T05:55:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.305+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/1279143217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a8071070 msgr2=0x7fa2a80cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.305+0000 7fa2ae0d7640 1 --2- 172.21.15.70:0/1279143217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a8071070 0x7fa2a80cb130 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fa2a806e640 tx=0x7fa2a002bee0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.305+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/1279143217 shutdown_connections 2023-10-08T05:55:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.305+0000 7fa2ae0d7640 1 --2- 172.21.15.70:0/1279143217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a8071070 0x7fa2a80cb130 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.305+0000 7fa2ae0d7640 1 --2- 172.21.15.70:0/1279143217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a806dfc0 0x7fa2a8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.305+0000 7fa2ae0d7640 1 --2- 172.21.15.70:0/1279143217 >> v2:172.21.15.181:3300/0 conn(0x7fa2a80afbe0 0x7fa2a80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.305+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/1279143217 >> 172.21.15.70:0/1279143217 conn(0x7fa2a806a5d0 msgr2=0x7fa2a80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.305+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/1279143217 shutdown_connections 2023-10-08T05:55:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.305+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/1279143217 wait complete. 2023-10-08T05:55:10.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.307+0000 7fa2ae0d7640 1 Processor -- start 2023-10-08T05:55:10.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.307+0000 7fa2ae0d7640 1 -- start start 2023-10-08T05:55:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.308+0000 7fa2ae0d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a806dfc0 0x7fa2a814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.308+0000 7fa2ae0d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a8071070 0x7fa2a80ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.308+0000 7fa2ae0d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80afbe0 0x7fa2a80cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.308+0000 7fa2ae0d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a815c230 con 0x7fa2a806dfc0 2023-10-08T05:55:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.308+0000 7fa2ae0d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a814e330 con 0x7fa2a80afbe0 2023-10-08T05:55:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.308+0000 7fa2ae0d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80c4190 con 0x7fa2a8071070 2023-10-08T05:55:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.308+0000 7fa2a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a806dfc0 0x7fa2a814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.308+0000 7fa2a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a806dfc0 0x7fa2a814ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43918/0 (socket says 172.21.15.70:43918) 2023-10-08T05:55:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.308+0000 7fa2a77fe640 1 -- 172.21.15.70:0/2798638842 learned_addr learned my addr 172.21.15.70:0/2798638842 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa29ffff640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a8071070 0x7fa2a80ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2a7fff640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80afbe0 0x7fa2a80cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2a77fe640 1 -- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a8071070 msgr2=0x7fa2a80ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a8071070 0x7fa2a80ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2a77fe640 1 -- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80afbe0 msgr2=0x7fa2a80cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80afbe0 0x7fa2a80cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2a77fe640 1 -- 172.21.15.70:0/2798638842 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a0009b30 con 0x7fa2a806dfc0 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a806dfc0 0x7fa2a814ddf0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fa29800c7f0 tx=0x7fa29800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2a4ff9640 1 -- 172.21.15.70:0/2798638842 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298007a30 con 0x7fa2a806dfc0 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2a4ff9640 1 -- 172.21.15.70:0/2798638842 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa298007b90 con 0x7fa2a806dfc0 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/2798638842 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a80c4470 con 0x7fa2a806dfc0 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2a4ff9640 1 -- 172.21.15.70:0/2798638842 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29800ce40 con 0x7fa2a806dfc0 2023-10-08T05:55:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.309+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/2798638842 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a80c4a30 con 0x7fa2a806dfc0 2023-10-08T05:55:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.311+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/2798638842 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a804e160 con 0x7fa2a806dfc0 2023-10-08T05:55:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.311+0000 7fa2a4ff9640 1 -- 172.21.15.70:0/2798638842 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 55) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa2980113f0 con 0x7fa2a806dfc0 2023-10-08T05:55:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.312+0000 7fa2a4ff9640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa28806af30 0x7fa28806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.312+0000 7fa2a4ff9640 1 -- 172.21.15.70:0/2798638842 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6963+0+0 (secure 0 0 0) 0x7fa298089d80 con 0x7fa2a806dfc0 2023-10-08T05:55:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.313+0000 7fa29ffff640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa28806af30 0x7fa28806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.314+0000 7fa29ffff640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa28806af30 0x7fa28806d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa2a815bfb0 tx=0x7fa290008040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.316+0000 7fa2a4ff9640 1 -- 172.21.15.70:0/2798638842 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2980547d0 con 0x7fa2a806dfc0 2023-10-08T05:55:10.441 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:55:10.449 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:55:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.544+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/2798638842 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa2a80b0ef0 con 0x7fa2a806dfc0 2023-10-08T05:55:10.546 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T05:55:10.545+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T05:55:10.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.547+0000 7fa2a4ff9640 1 -- 172.21.15.70:0/2798638842 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa29801f020 con 0x7fa2a806dfc0 2023-10-08T05:55:10.549 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T05:55:10.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.550+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa28806af30 msgr2=0x7fa28806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.550+0000 7fa2ae0d7640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa28806af30 0x7fa28806d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa2a815bfb0 tx=0x7fa290008040 comp rx=0 tx=0).stop 2023-10-08T05:55:10.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.550+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/2798638842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a806dfc0 msgr2=0x7fa2a814ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.550+0000 7fa2ae0d7640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a806dfc0 0x7fa2a814ddf0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fa29800c7f0 tx=0x7fa29800cbf0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.551+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/2798638842 shutdown_connections 2023-10-08T05:55:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.551+0000 7fa2ae0d7640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a80afbe0 0x7fa2a80cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.551+0000 7fa2ae0d7640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a8071070 0x7fa2a80ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.551+0000 7fa2ae0d7640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa28806af30 0x7fa28806d3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.551+0000 7fa2ae0d7640 1 --2- 172.21.15.70:0/2798638842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a806dfc0 0x7fa2a814ddf0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.551+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/2798638842 >> 172.21.15.70:0/2798638842 conn(0x7fa2a806a5d0 msgr2=0x7fa2a80c72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.551+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/2798638842 shutdown_connections 2023-10-08T05:55:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.551+0000 7fa2ae0d7640 1 -- 172.21.15.70:0/2798638842 wait complete. 2023-10-08T05:55:10.569 INFO:tasks.ceph.mds.b.smithi181.stderr: -6832> 2023-10-08T05:53:50.675+0000 7fe9ec7a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:55:10.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T05:55:10.637 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:55:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.831+0000 7f201b8da640 1 Processor -- start 2023-10-08T05:55:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.831+0000 7f201b8da640 1 -- start start 2023-10-08T05:55:10.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.833+0000 7f201b8da640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20140afbe0 0x7f20140b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.833+0000 7f201b8da640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20140b2620 con 0x7f201406dfc0 2023-10-08T05:55:10.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.833+0000 7f201b8da640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20140b2060 con 0x7f20140cb130 2023-10-08T05:55:10.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.833+0000 7f201b8da640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20140b2340 con 0x7f20140afbe0 2023-10-08T05:55:10.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.833+0000 7f2018e75640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f201406dfc0 0x7f2014070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41952/0 (socket says 172.21.15.70:41952) 2023-10-08T05:55:10.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.833+0000 7f2018e75640 1 -- 172.21.15.70:0/2032325219 learned_addr learned my addr 172.21.15.70:0/2032325219 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:10.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f2019676640 1 --2- 172.21.15.70:0/2032325219 >> v2:172.21.15.181:3300/0 conn(0x7f20140afbe0 0x7f20140b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3937955765 0 0) 0x7f20140b2620 con 0x7f201406dfc0 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2000003520 con 0x7f201406dfc0 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1094628498 0 0) 0x7f2000003520 con 0x7f201406dfc0 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 >> v2:172.21.15.181:3300/0 conn(0x7f20140afbe0 msgr2=0x7f20140b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 --2- 172.21.15.70:0/2032325219 >> v2:172.21.15.181:3300/0 conn(0x7f20140afbe0 0x7f20140b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 >> v1:172.21.15.181:6789/0 conn(0x7f20140cb130 legacy=0x7f201406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014070350 con 0x7f201406dfc0 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2004001670 con 0x7f201406dfc0 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 >> v1:172.21.15.70:6789/0 conn(0x7f201406dfc0 legacy=0x7f2014070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 --2- 172.21.15.70:0/2032325219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2000003970 0x7f2000003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 --2- 172.21.15.70:0/2032325219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20000047c0 0x7f2000004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 --2- 172.21.15.70:0/2032325219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2000005180 0x7f200000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20140b2340 con 0x7f2000003970 2023-10-08T05:55:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200000c020 con 0x7f2000005180 2023-10-08T05:55:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.834+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200000c1e0 con 0x7f20000047c0 2023-10-08T05:55:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.835+0000 7f2019676640 1 --2- 172.21.15.70:0/2032325219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2000003970 0x7f2000003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.835+0000 7f2019676640 1 -- 172.21.15.70:0/2032325219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20000047c0 msgr2=0x7f2000004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.835+0000 7f2019676640 1 --2- 172.21.15.70:0/2032325219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20000047c0 0x7f2000004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.835+0000 7f2019676640 1 -- 172.21.15.70:0/2032325219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2000005180 msgr2=0x7f200000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.835+0000 7f2019676640 1 --2- 172.21.15.70:0/2032325219 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2000005180 0x7f200000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.835+0000 7f2019676640 1 -- 172.21.15.70:0/2032325219 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000003520 con 0x7f2000003970 2023-10-08T05:55:10.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.835+0000 7f2019e77640 1 -- 172.21.15.70:0/2032325219 reap_dead start 2023-10-08T05:55:10.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.836+0000 7f2019676640 1 --2- 172.21.15.70:0/2032325219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2000003970 0x7f2000003d90 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2010009ac0 tx=0x7f20000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec0cc662c570e85 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:10.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.836+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201002c710 con 0x7f2000003970 2023-10-08T05:55:10.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.836+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201002c870 con 0x7f2000003970 2023-10-08T05:55:10.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.836+0000 7f200bfff640 1 -- 172.21.15.70:0/2032325219 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201002cc00 con 0x7f2000003970 2023-10-08T05:55:10.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.837+0000 7f201b8da640 1 -- 172.21.15.70:0/2032325219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2000003970 msgr2=0x7f2000003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.837+0000 7f201b8da640 1 --2- 172.21.15.70:0/2032325219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2000003970 0x7f2000003d90 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2010009ac0 tx=0x7f20000057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.837+0000 7f201b8da640 1 -- 172.21.15.70:0/2032325219 shutdown_connections 2023-10-08T05:55:10.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.837+0000 7f201b8da640 1 --2- 172.21.15.70:0/2032325219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2000003970 0x7f2000003d90 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.837+0000 7f201b8da640 1 -- 172.21.15.70:0/2032325219 >> 172.21.15.70:0/2032325219 conn(0x7f201406a5d0 msgr2=0x7f20140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:10.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.837+0000 7f201b8da640 1 -- 172.21.15.70:0/2032325219 shutdown_connections 2023-10-08T05:55:10.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.837+0000 7f201b8da640 1 -- 172.21.15.70:0/2032325219 wait complete. 2023-10-08T05:55:10.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.839+0000 7f201b8da640 1 Processor -- start 2023-10-08T05:55:10.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.839+0000 7f201b8da640 1 -- start start 2023-10-08T05:55:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.840+0000 7f201b8da640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2000003970 0x7f2014159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.840+0000 7f201b8da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140c6570 0x7f201415db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.840+0000 7f201b8da640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2014154fb0 0x7f20140c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.840+0000 7f201b8da640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201415a130 con 0x7f20140c6570 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.840+0000 7f201b8da640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2014158e30 con 0x7f2000003970 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.840+0000 7f201b8da640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2014159110 con 0x7f2014154fb0 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.840+0000 7f2018e75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140c6570 0x7f201415db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.840+0000 7f2018e75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140c6570 0x7f201415db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43938/0 (socket says 172.21.15.70:43938) 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.840+0000 7f2018e75640 1 -- 172.21.15.70:0/3694404478 learned_addr learned my addr 172.21.15.70:0/3694404478 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f2019676640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2000003970 0x7f2014159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f2018e75640 1 -- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2014154fb0 msgr2=0x7f20140c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f2018e75640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2014154fb0 0x7f20140c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f2018e75640 1 -- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2000003970 msgr2=0x7f2014159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f2018e75640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2000003970 0x7f2014159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f2018e75640 1 -- 172.21.15.70:0/3694404478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20141593f0 con 0x7f20140c6570 2023-10-08T05:55:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f2018e75640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140c6570 0x7f201415db20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f200400c700 tx=0x7f200400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f2009ffb640 1 -- 172.21.15.70:0/3694404478 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20040072d0 con 0x7f20140c6570 2023-10-08T05:55:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f2009ffb640 1 -- 172.21.15.70:0/3694404478 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2004007450 con 0x7f20140c6570 2023-10-08T05:55:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f2009ffb640 1 -- 172.21.15.70:0/3694404478 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20040047e0 con 0x7f20140c6570 2023-10-08T05:55:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.841+0000 7f201b8da640 1 -- 172.21.15.70:0/3694404478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20141596d0 con 0x7f20140c6570 2023-10-08T05:55:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.842+0000 7f201b8da640 1 -- 172.21.15.70:0/3694404478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201415e350 con 0x7f20140c6570 2023-10-08T05:55:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.842+0000 7f201b8da640 1 -- 172.21.15.70:0/3694404478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201404e160 con 0x7f20140c6570 2023-10-08T05:55:10.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.844+0000 7f2009ffb640 1 -- 172.21.15.70:0/3694404478 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 55) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f2004004960 con 0x7f20140c6570 2023-10-08T05:55:10.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.845+0000 7f2009ffb640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f1ff806af00 0x7f1ff806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:10.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.845+0000 7f2009ffb640 1 -- 172.21.15.70:0/3694404478 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7049+0+0 (secure 0 0 0) 0x7f2004088fd0 con 0x7f20140c6570 2023-10-08T05:55:10.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.845+0000 7f2019676640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f1ff806af00 0x7f1ff806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:10.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.846+0000 7f2019676640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f1ff806af00 0x7f1ff806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2010036000 tx=0x7f2010030000 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:10.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:10.849+0000 7f2009ffb640 1 -- 172.21.15.70:0/3694404478 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2004053b50 con 0x7f20140c6570 2023-10-08T05:55:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.079+0000 7f201b8da640 1 -- 172.21.15.70:0/3694404478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f20140b0ef0 con 0x7f20140c6570 2023-10-08T05:55:11.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.561+0000 7f2009ffb640 1 -- 172.21.15.70:0/3694404478 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2004057170 con 0x7f20140c6570 2023-10-08T05:55:11.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.565+0000 7f201b8da640 1 -- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f1ff806af00 msgr2=0x7f1ff806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:11.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.565+0000 7f201b8da640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f1ff806af00 0x7f1ff806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2010036000 tx=0x7f2010030000 comp rx=0 tx=0).stop 2023-10-08T05:55:11.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.565+0000 7f201b8da640 1 -- 172.21.15.70:0/3694404478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140c6570 msgr2=0x7f201415db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:11.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.565+0000 7f201b8da640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140c6570 0x7f201415db20 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f200400c700 tx=0x7f200400cbc0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.566+0000 7f201b8da640 1 -- 172.21.15.70:0/3694404478 shutdown_connections 2023-10-08T05:55:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.566+0000 7f201b8da640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2014154fb0 0x7f20140c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.566+0000 7f201b8da640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f1ff806af00 0x7f1ff806d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.566+0000 7f201b8da640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20140c6570 0x7f201415db20 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.566+0000 7f201b8da640 1 --2- 172.21.15.70:0/3694404478 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2000003970 0x7f2014159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.566+0000 7f201b8da640 1 -- 172.21.15.70:0/3694404478 >> 172.21.15.70:0/3694404478 conn(0x7f201406a5d0 msgr2=0x7f20140cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.566+0000 7f201b8da640 1 -- 172.21.15.70:0/3694404478 shutdown_connections 2023-10-08T05:55:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.566+0000 7f201b8da640 1 -- 172.21.15.70:0/3694404478 wait complete. 2023-10-08T05:55:11.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T05:55:11.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.808+0000 7f905c675640 1 Processor -- start 2023-10-08T05:55:11.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.809+0000 7f905c675640 1 -- start start 2023-10-08T05:55:11.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.810+0000 7f905c675640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f90540c8e70 0x7f90540ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:11.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.810+0000 7f905c675640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f90540cb5a0 0x7f90540b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:11.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.810+0000 7f905c675640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905406d490 con 0x7f90540b3ec0 2023-10-08T05:55:11.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.810+0000 7f905c675640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90540b42a0 con 0x7f90540cb5a0 2023-10-08T05:55:11.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.810+0000 7f905c675640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905406c880 con 0x7f90540c8e70 2023-10-08T05:55:11.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.810+0000 7f905ac12640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f90540cb5a0 0x7f90540b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:11.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.810+0000 7f905ac12640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f90540cb5a0 0x7f90540b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43948/0 (socket says 172.21.15.70:43948) 2023-10-08T05:55:11.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.810+0000 7f905ac12640 1 -- 172.21.15.70:0/3286076816 learned_addr learned my addr 172.21.15.70:0/3286076816 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:11.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.811+0000 7f9059c10640 1 --2- 172.21.15.70:0/3286076816 >> v2:172.21.15.181:3300/0 conn(0x7f90540c8e70 0x7f90540ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:11.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.811+0000 7f905ac12640 1 -- 172.21.15.70:0/3286076816 >> v2:172.21.15.181:3300/0 conn(0x7f90540c8e70 msgr2=0x7f90540ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:11.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.811+0000 7f905ac12640 1 --2- 172.21.15.70:0/3286076816 >> v2:172.21.15.181:3300/0 conn(0x7f90540c8e70 0x7f90540ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.811+0000 7f905ac12640 1 -- 172.21.15.70:0/3286076816 >> v1:172.21.15.181:6789/0 conn(0x7f90540b3ec0 legacy=0x7f90540cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:11.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.811+0000 7f905ac12640 1 -- 172.21.15.70:0/3286076816 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905406cb60 con 0x7f90540cb5a0 2023-10-08T05:55:11.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.813+0000 7f905ac12640 1 --2- 172.21.15.70:0/3286076816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 0x7f90540b2190 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f90540c2220 tx=0x7f904c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f3bfae678e7dd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:11.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.813+0000 7f905940f640 1 -- 172.21.15.70:0/3286076816 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c030860 con 0x7f90540cb5a0 2023-10-08T05:55:11.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.813+0000 7f905940f640 1 -- 172.21.15.70:0/3286076816 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c0309c0 con 0x7f90540cb5a0 2023-10-08T05:55:11.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.813+0000 7f905940f640 1 -- 172.21.15.70:0/3286076816 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c030d50 con 0x7f90540cb5a0 2023-10-08T05:55:11.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.814+0000 7f905c675640 1 -- 172.21.15.70:0/3286076816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 msgr2=0x7f90540b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:11.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.814+0000 7f905c675640 1 --2- 172.21.15.70:0/3286076816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 0x7f90540b2190 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f90540c2220 tx=0x7f904c02b9a0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.814+0000 7f905c675640 1 -- 172.21.15.70:0/3286076816 shutdown_connections 2023-10-08T05:55:11.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.814+0000 7f905c675640 1 --2- 172.21.15.70:0/3286076816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 0x7f90540b2190 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.814+0000 7f905c675640 1 --2- 172.21.15.70:0/3286076816 >> v2:172.21.15.181:3300/0 conn(0x7f90540c8e70 0x7f90540ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.814+0000 7f905c675640 1 -- 172.21.15.70:0/3286076816 >> 172.21.15.70:0/3286076816 conn(0x7f905406a5d0 msgr2=0x7f90540b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:11.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.814+0000 7f905c675640 1 -- 172.21.15.70:0/3286076816 shutdown_connections 2023-10-08T05:55:11.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.814+0000 7f905c675640 1 -- 172.21.15.70:0/3286076816 wait complete. 2023-10-08T05:55:11.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.816+0000 7f905c675640 1 Processor -- start 2023-10-08T05:55:11.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.816+0000 7f905c675640 1 -- start start 2023-10-08T05:55:11.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.817+0000 7f905c675640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90540b3ec0 0x7f90541676c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.817+0000 7f905c675640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90540c8e70 0x7f90540cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.817+0000 7f905c675640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 0x7f9054158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.817+0000 7f905c675640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905415b780 con 0x7f90540cb5a0 2023-10-08T05:55:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.817+0000 7f905c675640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90541651e0 con 0x7f90540b3ec0 2023-10-08T05:55:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.817+0000 7f905c675640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90541654c0 con 0x7f90540c8e70 2023-10-08T05:55:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.817+0000 7f905ac12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 0x7f9054158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.817+0000 7f905ac12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 0x7f9054158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43956/0 (socket says 172.21.15.70:43956) 2023-10-08T05:55:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.817+0000 7f905ac12640 1 -- 172.21.15.70:0/2243031547 learned_addr learned my addr 172.21.15.70:0/2243031547 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:11.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.817+0000 7f905a411640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90540b3ec0 0x7f90541676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:11.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f9059c10640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90540c8e70 0x7f90540cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:11.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f905ac12640 1 -- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90540c8e70 msgr2=0x7f90540cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f905ac12640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90540c8e70 0x7f90540cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f905ac12640 1 -- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90540b3ec0 msgr2=0x7f90541676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f905ac12640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90540b3ec0 0x7f90541676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f905ac12640 1 -- 172.21.15.70:0/2243031547 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c0094d0 con 0x7f90540cb5a0 2023-10-08T05:55:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f905ac12640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 0x7f9054158b10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f904c02b9e0 tx=0x7f904c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f9052ffd640 1 -- 172.21.15.70:0/2243031547 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c039a90 con 0x7f90540cb5a0 2023-10-08T05:55:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f9052ffd640 1 -- 172.21.15.70:0/2243031547 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c039c10 con 0x7f90540cb5a0 2023-10-08T05:55:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f905c675640 1 -- 172.21.15.70:0/2243031547 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9054165bc0 con 0x7f90540cb5a0 2023-10-08T05:55:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.818+0000 7f9052ffd640 1 -- 172.21.15.70:0/2243031547 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c043410 con 0x7f90540cb5a0 2023-10-08T05:55:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.819+0000 7f905c675640 1 -- 172.21.15.70:0/2243031547 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90540ccb80 con 0x7f90540cb5a0 2023-10-08T05:55:11.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.820+0000 7f905c675640 1 -- 172.21.15.70:0/2243031547 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9054165a20 con 0x7f90540cb5a0 2023-10-08T05:55:11.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.821+0000 7f9052ffd640 1 -- 172.21.15.70:0/2243031547 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f904c04c3b0 con 0x7f90540cb5a0 2023-10-08T05:55:11.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.822+0000 7f9052ffd640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f904006afc0 0x7f904006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:11.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.822+0000 7f9052ffd640 1 -- 172.21.15.70:0/2243031547 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 7049+0+0 (secure 0 0 0) 0x7f904c0ac310 con 0x7f90540cb5a0 2023-10-08T05:55:11.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.823+0000 7f905a411640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f904006afc0 0x7f904006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:11.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.824+0000 7f905a411640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f904006afc0 0x7f904006d480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9054150c10 tx=0x7f9044009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:11.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:11.827+0000 7f9052ffd640 1 -- 172.21.15.70:0/2243031547 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9054165a20 con 0x7f90540cb5a0 2023-10-08T05:55:12.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.015+0000 7f905c675640 1 -- 172.21.15.70:0/2243031547 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9054165a20 con 0x7f90540cb5a0 2023-10-08T05:55:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.771+0000 7f9052ffd640 1 -- 172.21.15.70:0/2243031547 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v124) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9054165a20 con 0x7f90540cb5a0 2023-10-08T05:55:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T05:55:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.776+0000 7f905c675640 1 -- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f904006afc0 msgr2=0x7f904006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.776+0000 7f905c675640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f904006afc0 0x7f904006d480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9054150c10 tx=0x7f9044009040 comp rx=0 tx=0).stop 2023-10-08T05:55:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.776+0000 7f905c675640 1 -- 172.21.15.70:0/2243031547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 msgr2=0x7f9054158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:12.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.776+0000 7f905c675640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 0x7f9054158b10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f904c02b9e0 tx=0x7f904c004730 comp rx=0 tx=0).stop 2023-10-08T05:55:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.778+0000 7f905c675640 1 -- 172.21.15.70:0/2243031547 shutdown_connections 2023-10-08T05:55:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.778+0000 7f905c675640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f904006afc0 0x7f904006d480 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.778+0000 7f905c675640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90540cb5a0 0x7f9054158b10 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.778+0000 7f905c675640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90540c8e70 0x7f90540cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.778+0000 7f905c675640 1 --2- 172.21.15.70:0/2243031547 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f90540b3ec0 0x7f90541676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.778+0000 7f905c675640 1 -- 172.21.15.70:0/2243031547 >> 172.21.15.70:0/2243031547 conn(0x7f905406a5d0 msgr2=0x7f905415bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.779+0000 7f905c675640 1 -- 172.21.15.70:0/2243031547 shutdown_connections 2023-10-08T05:55:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:12.780+0000 7f905c675640 1 -- 172.21.15.70:0/2243031547 wait complete. 2023-10-08T05:55:12.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T05:55:13.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.006+0000 7fc309755640 1 Processor -- start 2023-10-08T05:55:13.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.007+0000 7fc309755640 1 -- start start 2023-10-08T05:55:13.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.008+0000 7fc309755640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc30415d0c0 0x7fc3040b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:13.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.008+0000 7fc309755640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc3040b30d0 0x7fc3040b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:13.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.008+0000 7fc309755640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30406ce20 con 0x7fc30415cce0 2023-10-08T05:55:13.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.008+0000 7fc309755640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3040cf850 con 0x7fc30415d0c0 2023-10-08T05:55:13.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.008+0000 7fc309755640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30406c060 con 0x7fc3040b30d0 2023-10-08T05:55:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.008+0000 7fc3027fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc30415d0c0 0x7fc3040b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.009+0000 7fc3027fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc30415d0c0 0x7fc3040b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43972/0 (socket says 172.21.15.70:43972) 2023-10-08T05:55:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.009+0000 7fc3027fc640 1 -- 172.21.15.70:0/1960970509 learned_addr learned my addr 172.21.15.70:0/1960970509 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.009+0000 7fc3037fe640 1 --2- 172.21.15.70:0/1960970509 >> v2:172.21.15.181:3300/0 conn(0x7fc3040b30d0 0x7fc3040b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.009+0000 7fc3027fc640 1 -- 172.21.15.70:0/1960970509 >> v2:172.21.15.181:3300/0 conn(0x7fc3040b30d0 msgr2=0x7fc3040b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.009+0000 7fc3027fc640 1 --2- 172.21.15.70:0/1960970509 >> v2:172.21.15.181:3300/0 conn(0x7fc3040b30d0 0x7fc3040b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.009+0000 7fc3027fc640 1 -- 172.21.15.70:0/1960970509 >> v1:172.21.15.70:6789/0 conn(0x7fc30415cce0 legacy=0x7fc3040cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:13.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.009+0000 7fc3027fc640 1 -- 172.21.15.70:0/1960970509 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30406c340 con 0x7fc30415d0c0 2023-10-08T05:55:13.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.010+0000 7fc3027fc640 1 --2- 172.21.15.70:0/1960970509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415d0c0 0x7fc3040b2530 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc3040b04b0 tx=0x7fc2ec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06c6db5bb7efd3e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.010+0000 7fc301ffb640 1 -- 172.21.15.70:0/1960970509 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec030860 con 0x7fc30415d0c0 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.010+0000 7fc301ffb640 1 -- 172.21.15.70:0/1960970509 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec0309c0 con 0x7fc30415d0c0 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.010+0000 7fc301ffb640 1 -- 172.21.15.70:0/1960970509 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec030d50 con 0x7fc30415d0c0 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.011+0000 7fc309755640 1 -- 172.21.15.70:0/1960970509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415d0c0 msgr2=0x7fc3040b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.011+0000 7fc309755640 1 --2- 172.21.15.70:0/1960970509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415d0c0 0x7fc3040b2530 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fc3040b04b0 tx=0x7fc2ec02b9a0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.011+0000 7fc309755640 1 -- 172.21.15.70:0/1960970509 shutdown_connections 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.011+0000 7fc309755640 1 --2- 172.21.15.70:0/1960970509 >> v2:172.21.15.181:3300/0 conn(0x7fc3040b30d0 0x7fc3040b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.011+0000 7fc309755640 1 --2- 172.21.15.70:0/1960970509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415d0c0 0x7fc3040b2530 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.011+0000 7fc309755640 1 -- 172.21.15.70:0/1960970509 >> 172.21.15.70:0/1960970509 conn(0x7fc30406e650 msgr2=0x7fc30406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.011+0000 7fc309755640 1 -- 172.21.15.70:0/1960970509 shutdown_connections 2023-10-08T05:55:13.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.011+0000 7fc309755640 1 -- 172.21.15.70:0/1960970509 wait complete. 2023-10-08T05:55:13.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.013+0000 7fc309755640 1 Processor -- start 2023-10-08T05:55:13.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.013+0000 7fc309755640 1 -- start start 2023-10-08T05:55:13.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.014+0000 7fc309755640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3040b30d0 0x7fc30415e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:13.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.014+0000 7fc309755640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc30415cce0 0x7fc30414cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:13.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.014+0000 7fc309755640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415b520 0x7fc3040b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:13.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.014+0000 7fc309755640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3040c7c90 con 0x7fc30415b520 2023-10-08T05:55:13.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.014+0000 7fc309755640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30415f960 con 0x7fc30415cce0 2023-10-08T05:55:13.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.014+0000 7fc309755640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30415ed50 con 0x7fc3040b30d0 2023-10-08T05:55:13.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.014+0000 7fc3037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415b520 0x7fc3040b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.014+0000 7fc3037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415b520 0x7fc3040b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43988/0 (socket says 172.21.15.70:43988) 2023-10-08T05:55:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.014+0000 7fc3037fe640 1 -- 172.21.15.70:0/2005370645 learned_addr learned my addr 172.21.15.70:0/2005370645 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc3027fc640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc30415cce0 0x7fc30414cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc302ffd640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3040b30d0 0x7fc30415e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc3037fe640 1 -- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3040b30d0 msgr2=0x7fc30415e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc3037fe640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3040b30d0 0x7fc30415e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc3037fe640 1 -- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc30415cce0 msgr2=0x7fc30414cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc3037fe640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc30415cce0 0x7fc30414cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc3037fe640 1 -- 172.21.15.70:0/2005370645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0094d0 con 0x7fc30415b520 2023-10-08T05:55:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc302ffd640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3040b30d0 0x7fc30415e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc3027fc640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc30415cce0 0x7fc30414cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc3037fe640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415b520 0x7fc3040b0440 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00d800 tx=0x7fc2fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc2fb7fe640 1 -- 172.21.15.70:0/2005370645 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00b6c0 con 0x7fc30415b520 2023-10-08T05:55:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc2fb7fe640 1 -- 172.21.15.70:0/2005370645 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc00b840 con 0x7fc30415b520 2023-10-08T05:55:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc2fb7fe640 1 -- 172.21.15.70:0/2005370645 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00bbf0 con 0x7fc30415b520 2023-10-08T05:55:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.015+0000 7fc309755640 1 -- 172.21.15.70:0/2005370645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30415f030 con 0x7fc30415b520 2023-10-08T05:55:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.016+0000 7fc309755640 1 -- 172.21.15.70:0/2005370645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30415f5f0 con 0x7fc30415b520 2023-10-08T05:55:13.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.017+0000 7fc309755640 1 -- 172.21.15.70:0/2005370645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30415f7b0 con 0x7fc30415b520 2023-10-08T05:55:13.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.018+0000 7fc2fb7fe640 1 -- 172.21.15.70:0/2005370645 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc2fc00bd50 con 0x7fc30415b520 2023-10-08T05:55:13.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.019+0000 7fc2fb7fe640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc2e406af90 0x7fc2e406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:13.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.019+0000 7fc2fb7fe640 1 -- 172.21.15.70:0/2005370645 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6611+0+0 (secure 0 0 0) 0x7fc2fc08a140 con 0x7fc30415b520 2023-10-08T05:55:13.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.019+0000 7fc302ffd640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc2e406af90 0x7fc2e406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:13.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.020+0000 7fc302ffd640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc2e406af90 0x7fc2e406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40012e0 tx=0x7fc2f4009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:13.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.023+0000 7fc2fb7fe640 1 -- 172.21.15.70:0/2005370645 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2fc054cc0 con 0x7fc30415b520 2023-10-08T05:55:13.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.219+0000 7fc309755640 1 -- 172.21.15.70:0/2005370645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc30404e160 con 0x7fc30415b520 2023-10-08T05:55:13.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.777+0000 7fc2fb7fe640 1 -- 172.21.15.70:0/2005370645 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v125) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc2fc0582e0 con 0x7fc30415b520 2023-10-08T05:55:13.779 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T05:55:13.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 -- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc2e406af90 msgr2=0x7fc2e406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc2e406af90 0x7fc2e406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40012e0 tx=0x7fc2f4009040 comp rx=0 tx=0).stop 2023-10-08T05:55:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 -- 172.21.15.70:0/2005370645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415b520 msgr2=0x7fc3040b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415b520 0x7fc3040b0440 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00d800 tx=0x7fc2fc00dcc0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 -- 172.21.15.70:0/2005370645 shutdown_connections 2023-10-08T05:55:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc2e406af90 0x7fc2e406d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc30415b520 0x7fc3040b0440 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc30415cce0 0x7fc30414cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 --2- 172.21.15.70:0/2005370645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc3040b30d0 0x7fc30415e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 -- 172.21.15.70:0/2005370645 >> 172.21.15.70:0/2005370645 conn(0x7fc30406e650 msgr2=0x7fc3040c6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 -- 172.21.15.70:0/2005370645 shutdown_connections 2023-10-08T05:55:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:13.781+0000 7fc309755640 1 -- 172.21.15.70:0/2005370645 wait complete. 2023-10-08T05:55:13.808 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T05:55:14.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.011+0000 7fa6ff59e640 1 Processor -- start 2023-10-08T05:55:14.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.012+0000 7fa6ff59e640 1 -- start start 2023-10-08T05:55:14.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.013+0000 7fa6ff59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa7000b3c50 0x7fa7000cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.013+0000 7fa6ff59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa700159db0 con 0x7fa7000cb5e0 2023-10-08T05:55:14.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.013+0000 7fa6ff59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa700071fe0 con 0x7fa7001599d0 2023-10-08T05:55:14.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.013+0000 7fa6ff59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7000b44c0 con 0x7fa7000b3c50 2023-10-08T05:55:14.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.013+0000 7fa6f7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa7000cb5e0 0x7fa7000cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41030/0 (socket says 172.21.15.70:41030) 2023-10-08T05:55:14.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.013+0000 7fa6f7fff640 1 -- 172.21.15.70:0/3598985710 learned_addr learned my addr 172.21.15.70:0/3598985710 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:14.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fed9d640 1 --2- 172.21.15.70:0/3598985710 >> v2:172.21.15.181:3300/0 conn(0x7fa7000b3c50 0x7fa7000cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671632405 0 0) 0x7fa6e8001110 con 0x7fa7000cb5e0 2023-10-08T05:55:14.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6ec003520 con 0x7fa7000cb5e0 2023-10-08T05:55:14.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2666830777 0 0) 0x7fa6ec003520 con 0x7fa7000cb5e0 2023-10-08T05:55:14.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 >> v2:172.21.15.181:3300/0 conn(0x7fa7000b3c50 msgr2=0x7fa7000cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:14.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 --2- 172.21.15.70:0/3598985710 >> v2:172.21.15.181:3300/0 conn(0x7fa7000b3c50 0x7fa7000cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 >> v1:172.21.15.181:6789/0 conn(0x7fa7001599d0 legacy=0x7fa7000caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7000713d0 con 0x7fa7000cb5e0 2023-10-08T05:55:14.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa6e8001800 con 0x7fa7000cb5e0 2023-10-08T05:55:14.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 >> v1:172.21.15.70:6789/0 conn(0x7fa7000cb5e0 legacy=0x7fa7000cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6ec003970 0x7fa6ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.014+0000 7fa6fdd9b640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6ec0047c0 0x7fa6ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fdd9b640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ec005180 0x7fa6ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000b44c0 con 0x7fa6ec005180 2023-10-08T05:55:14.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec00c020 con 0x7fa6ec003970 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec00c1e0 con 0x7fa6ec0047c0 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fed9d640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ec005180 0x7fa6ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fe59c640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6ec003970 0x7fa6ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6f7fff640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6ec0047c0 0x7fa6ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fed9d640 1 -- 172.21.15.70:0/3598985710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6ec0047c0 msgr2=0x7fa6ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fed9d640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6ec0047c0 0x7fa6ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fed9d640 1 -- 172.21.15.70:0/3598985710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6ec003970 msgr2=0x7fa6ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fed9d640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6ec003970 0x7fa6ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fed9d640 1 -- 172.21.15.70:0/3598985710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec003520 con 0x7fa6ec005180 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6fe59c640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6ec003970 0x7fa6ec003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:55:14.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.015+0000 7fa6f7fff640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6ec0047c0 0x7fa6ec004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.017+0000 7fa6fed9d640 1 -- 172.21.15.70:0/3598985710 reap_dead start 2023-10-08T05:55:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.017+0000 7fa6fed9d640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ec005180 0x7fa6ec00b980 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009fc0 tx=0x7fa6ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6913ff091b4b07a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.017+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f802cc60 con 0x7fa6ec005180 2023-10-08T05:55:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.017+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f800f040 con 0x7fa6ec005180 2023-10-08T05:55:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.017+0000 7fa6fdd9b640 1 -- 172.21.15.70:0/3598985710 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8003990 con 0x7fa6ec005180 2023-10-08T05:55:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.018+0000 7fa6ff59e640 1 -- 172.21.15.70:0/3598985710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ec005180 msgr2=0x7fa6ec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.018+0000 7fa6ff59e640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ec005180 0x7fa6ec00b980 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009fc0 tx=0x7fa6ec0057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.018+0000 7fa6ff59e640 1 -- 172.21.15.70:0/3598985710 shutdown_connections 2023-10-08T05:55:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.018+0000 7fa6ff59e640 1 --2- 172.21.15.70:0/3598985710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6ec005180 0x7fa6ec00b980 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.018+0000 7fa6ff59e640 1 -- 172.21.15.70:0/3598985710 >> 172.21.15.70:0/3598985710 conn(0x7fa70006c050 msgr2=0x7fa70006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:14.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.018+0000 7fa6ff59e640 1 -- 172.21.15.70:0/3598985710 shutdown_connections 2023-10-08T05:55:14.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.018+0000 7fa6ff59e640 1 -- 172.21.15.70:0/3598985710 wait complete. 2023-10-08T05:55:14.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.020+0000 7fa6ff59e640 1 Processor -- start 2023-10-08T05:55:14.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.020+0000 7fa6ff59e640 1 -- start start 2023-10-08T05:55:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6ff59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6ec005180 0x7fa7000c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6ff59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7000b0cf0 0x7fa7000b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6ff59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa700151950 0x7fa700153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6ff59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000c88f0 con 0x7fa7000b0cf0 2023-10-08T05:55:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6ff59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000c6860 con 0x7fa700151950 2023-10-08T05:55:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6ff59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000c8230 con 0x7fa6ec005180 2023-10-08T05:55:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7000b0cf0 0x7fa7000b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7000b0cf0 0x7fa7000b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42586/0 (socket says 172.21.15.70:42586) 2023-10-08T05:55:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6f7fff640 1 -- 172.21.15.70:0/734728887 learned_addr learned my addr 172.21.15.70:0/734728887 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6f7fff640 1 -- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6ec005180 msgr2=0x7fa7000c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6f7fff640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6ec005180 0x7fa7000c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6f7fff640 1 -- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa700151950 msgr2=0x7fa700153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6f7fff640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa700151950 0x7fa700153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.021+0000 7fa6f7fff640 1 -- 172.21.15.70:0/734728887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7000c8510 con 0x7fa7000b0cf0 2023-10-08T05:55:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.022+0000 7fa6f7fff640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7000b0cf0 0x7fa7000b16a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800d7f0 tx=0x7fa6e800dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.022+0000 7fa6f6ffd640 1 -- 172.21.15.70:0/734728887 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80074e0 con 0x7fa7000b0cf0 2023-10-08T05:55:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.022+0000 7fa6f6ffd640 1 -- 172.21.15.70:0/734728887 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8007640 con 0x7fa7000b0cf0 2023-10-08T05:55:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.022+0000 7fa6f6ffd640 1 -- 172.21.15.70:0/734728887 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8004a00 con 0x7fa7000b0cf0 2023-10-08T05:55:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.022+0000 7fa6ff59e640 1 -- 172.21.15.70:0/734728887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7000c7ee0 con 0x7fa7000b0cf0 2023-10-08T05:55:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.022+0000 7fa6ff59e640 1 -- 172.21.15.70:0/734728887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7000b1be0 con 0x7fa7000b0cf0 2023-10-08T05:55:14.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.024+0000 7fa6ff59e640 1 -- 172.21.15.70:0/734728887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7000c8b60 con 0x7fa7000b0cf0 2023-10-08T05:55:14.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.024+0000 7fa6f6ffd640 1 -- 172.21.15.70:0/734728887 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa6e8004b60 con 0x7fa7000b0cf0 2023-10-08T05:55:14.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.025+0000 7fa6f6ffd640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa6e006af90 0x7fa6e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.026+0000 7fa6f6ffd640 1 -- 172.21.15.70:0/734728887 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fa6e8089160 con 0x7fa7000b0cf0 2023-10-08T05:55:14.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.026+0000 7fa6fe59c640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa6e006af90 0x7fa6e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.027+0000 7fa6fe59c640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa6e006af90 0x7fa6e006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0007fd0 tx=0x7fa6f0009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:14.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.030+0000 7fa6f6ffd640 1 -- 172.21.15.70:0/734728887 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6e8057990 con 0x7fa7000b0cf0 2023-10-08T05:55:14.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.216+0000 7fa6ff59e640 1 -- 172.21.15.70:0/734728887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa700159db0 con 0x7fa7000b0cf0 2023-10-08T05:55:14.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.221+0000 7fa6f6ffd640 1 -- 172.21.15.70:0/734728887 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa700159db0 con 0x7fa7000b0cf0 2023-10-08T05:55:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.223+0000 7fa6ff59e640 1 -- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa6e006af90 msgr2=0x7fa6e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.223+0000 7fa6ff59e640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa6e006af90 0x7fa6e006d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0007fd0 tx=0x7fa6f0009040 comp rx=0 tx=0).stop 2023-10-08T05:55:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.223+0000 7fa6ff59e640 1 -- 172.21.15.70:0/734728887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7000b0cf0 msgr2=0x7fa7000b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.223+0000 7fa6ff59e640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7000b0cf0 0x7fa7000b16a0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800d7f0 tx=0x7fa6e800dbf0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.224+0000 7fa6ff59e640 1 -- 172.21.15.70:0/734728887 shutdown_connections 2023-10-08T05:55:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.224+0000 7fa6ff59e640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa700151950 0x7fa700153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.224+0000 7fa6ff59e640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fa6e006af90 0x7fa6e006d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.224+0000 7fa6ff59e640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7000b0cf0 0x7fa7000b16a0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.224+0000 7fa6ff59e640 1 --2- 172.21.15.70:0/734728887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6ec005180 0x7fa7000c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.224+0000 7fa6ff59e640 1 -- 172.21.15.70:0/734728887 >> 172.21.15.70:0/734728887 conn(0x7fa70006c050 msgr2=0x7fa700158c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.224+0000 7fa6ff59e640 1 -- 172.21.15.70:0/734728887 shutdown_connections 2023-10-08T05:55:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.224+0000 7fa6ff59e640 1 -- 172.21.15.70:0/734728887 wait complete. 2023-10-08T05:55:14.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T05:55:14.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.456+0000 7fbda41d0640 1 Processor -- start 2023-10-08T05:55:14.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.457+0000 7fbda41d0640 1 -- start start 2023-10-08T05:55:14.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.458+0000 7fbda41d0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbd9c146930 0x7fbd9c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.458+0000 7fbda41d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c144020 0x7fbd9c164b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.458+0000 7fbda41d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c1650b0 con 0x7fbd9c146d10 2023-10-08T05:55:14.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.458+0000 7fbda41d0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c165b80 con 0x7fbd9c146930 2023-10-08T05:55:14.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.458+0000 7fbda41d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c162540 con 0x7fbd9c144020 2023-10-08T05:55:14.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.459+0000 7fbda176b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbd9c146d10 0x7fbd9c143910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41038/0 (socket says 172.21.15.70:41038) 2023-10-08T05:55:14.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.459+0000 7fbda176b640 1 -- 172.21.15.70:0/4082197322 learned_addr learned my addr 172.21.15.70:0/4082197322 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:14.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.459+0000 7fbda1f6c640 1 --2- 172.21.15.70:0/4082197322 >> v2:172.21.15.181:3300/0 conn(0x7fbd9c146930 0x7fbd9c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.459+0000 7fbda276d640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c144020 0x7fbd9c164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.459+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563249147 0 0) 0x7fbd9c1650b0 con 0x7fbd9c146d10 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.459+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd88003520 con 0x7fbd9c146d10 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3495560089 0 0) 0x7fbd88003520 con 0x7fbd9c146d10 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c144020 msgr2=0x7fbd9c164b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c144020 0x7fbd9c164b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 >> v2:172.21.15.181:3300/0 conn(0x7fbd9c146930 msgr2=0x7fbd9c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 --2- 172.21.15.70:0/4082197322 >> v2:172.21.15.181:3300/0 conn(0x7fbd9c146930 0x7fbd9c162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c06c9d0 con 0x7fbd9c146d10 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fbd8c001670 con 0x7fbd9c146d10 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 >> v1:172.21.15.70:6789/0 conn(0x7fbd9c146d10 legacy=0x7fbd9c143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 0x7fbd88003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c144020 0x7fbd880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd88004d00 0x7fbd88005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c165b80 con 0x7fbd88003970 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c162540 con 0x7fbd88004d00 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8800b980 con 0x7fbd9c144020 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda1f6c640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 0x7fbd88003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda276d640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd88004d00 0x7fbd88005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda1f6c640 1 -- 172.21.15.70:0/4082197322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c144020 msgr2=0x7fbd880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda1f6c640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c144020 0x7fbd880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda1f6c640 1 -- 172.21.15.70:0/4082197322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd88004d00 msgr2=0x7fbd88005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda1f6c640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd88004d00 0x7fbd88005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.460+0000 7fbda1f6c640 1 -- 172.21.15.70:0/4082197322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd88003520 con 0x7fbd88003970 2023-10-08T05:55:14.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.461+0000 7fbda1f6c640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 0x7fbd88003d90 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009ac0 tx=0x7fbd880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6eb636e941a066 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.462+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9802c710 con 0x7fbd88003970 2023-10-08T05:55:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.462+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9802c870 con 0x7fbd88003970 2023-10-08T05:55:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.462+0000 7fbda0f6a640 1 -- 172.21.15.70:0/4082197322 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9802cc00 con 0x7fbd88003970 2023-10-08T05:55:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.462+0000 7fbda41d0640 1 -- 172.21.15.70:0/4082197322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 msgr2=0x7fbd88003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.462+0000 7fbda41d0640 1 --2- 172.21.15.70:0/4082197322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 0x7fbd88003d90 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009ac0 tx=0x7fbd880057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.462+0000 7fbda276d640 1 -- 172.21.15.70:0/4082197322 reap_dead start 2023-10-08T05:55:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.462+0000 7fbda41d0640 1 -- 172.21.15.70:0/4082197322 shutdown_connections 2023-10-08T05:55:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.462+0000 7fbda41d0640 1 -- 172.21.15.70:0/4082197322 >> 172.21.15.70:0/4082197322 conn(0x7fbd9c069d50 msgr2=0x7fbd9c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.462+0000 7fbda41d0640 1 -- 172.21.15.70:0/4082197322 shutdown_connections 2023-10-08T05:55:14.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.462+0000 7fbda41d0640 1 -- 172.21.15.70:0/4082197322 wait complete. 2023-10-08T05:55:14.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.464+0000 7fbda41d0640 1 Processor -- start 2023-10-08T05:55:14.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.464+0000 7fbda41d0640 1 -- start start 2023-10-08T05:55:14.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.465+0000 7fbda41d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 0x7fbd9c160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.465+0000 7fbda41d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd9c1f79f0 0x7fbd9c0cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.465+0000 7fbda41d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c15c2e0 0x7fbd9c15c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.465+0000 7fbda41d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c1f7890 con 0x7fbd88003970 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.465+0000 7fbda41d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c15bec0 con 0x7fbd9c1f79f0 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.465+0000 7fbda41d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c1f7e70 con 0x7fbd9c15c2e0 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.465+0000 7fbda1f6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 0x7fbd9c160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.466+0000 7fbda1f6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 0x7fbd9c160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42612/0 (socket says 172.21.15.70:42612) 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.466+0000 7fbda1f6c640 1 -- 172.21.15.70:0/3573040526 learned_addr learned my addr 172.21.15.70:0/3573040526 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.466+0000 7fbda176b640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd9c1f79f0 0x7fbd9c0cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.466+0000 7fbda1f6c640 1 -- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c15c2e0 msgr2=0x7fbd9c15c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.466+0000 7fbda276d640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c15c2e0 0x7fbd9c15c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.466+0000 7fbda1f6c640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c15c2e0 0x7fbd9c15c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.466+0000 7fbda1f6c640 1 -- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd9c1f79f0 msgr2=0x7fbd9c0cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.466+0000 7fbda1f6c640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd9c1f79f0 0x7fbd9c0cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.466+0000 7fbda1f6c640 1 -- 172.21.15.70:0/3573040526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c1f6ff0 con 0x7fbd88003970 2023-10-08T05:55:14.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.466+0000 7fbda1f6c640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 0x7fbd9c160790 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fbd9802c2b0 tx=0x7fbd98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:14.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.467+0000 7fbd927fc640 1 -- 172.21.15.70:0/3573040526 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98003810 con 0x7fbd88003970 2023-10-08T05:55:14.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.467+0000 7fbd927fc640 1 -- 172.21.15.70:0/3573040526 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9802de90 con 0x7fbd88003970 2023-10-08T05:55:14.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.467+0000 7fbd927fc640 1 -- 172.21.15.70:0/3573040526 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd98003b70 con 0x7fbd88003970 2023-10-08T05:55:14.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.467+0000 7fbda41d0640 1 -- 172.21.15.70:0/3573040526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c1f72d0 con 0x7fbd88003970 2023-10-08T05:55:14.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.467+0000 7fbda41d0640 1 -- 172.21.15.70:0/3573040526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c0cbae0 con 0x7fbd88003970 2023-10-08T05:55:14.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.468+0000 7fbda41d0640 1 -- 172.21.15.70:0/3573040526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c15bc20 con 0x7fbd88003970 2023-10-08T05:55:14.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.469+0000 7fbd927fc640 1 -- 172.21.15.70:0/3573040526 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbd98003cd0 con 0x7fbd88003970 2023-10-08T05:55:14.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.469+0000 7fbd927fc640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd8006af90 0x7fbd8006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.470+0000 7fbd927fc640 1 -- 172.21.15.70:0/3573040526 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fbd980ab770 con 0x7fbd88003970 2023-10-08T05:55:14.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.470+0000 7fbda176b640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd8006af90 0x7fbd8006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.471+0000 7fbda176b640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd8006af90 0x7fbd8006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c000f60 tx=0x7fbd8c009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:14.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.474+0000 7fbd927fc640 1 -- 172.21.15.70:0/3573040526 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd9c15bc20 con 0x7fbd88003970 2023-10-08T05:55:14.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.662+0000 7fbda41d0640 1 -- 172.21.15.70:0/3573040526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fbd9c15bc20 con 0x7fbd88003970 2023-10-08T05:55:14.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.669+0000 7fbd927fc640 1 -- 172.21.15.70:0/3573040526 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v38) v1 ==== 115+0+0 (secure 0 0 0) 0x7fbd9c15bc20 con 0x7fbd88003970 2023-10-08T05:55:14.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.671+0000 7fbda41d0640 1 -- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd8006af90 msgr2=0x7fbd8006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.671+0000 7fbda41d0640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd8006af90 0x7fbd8006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c000f60 tx=0x7fbd8c009040 comp rx=0 tx=0).stop 2023-10-08T05:55:14.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.671+0000 7fbda41d0640 1 -- 172.21.15.70:0/3573040526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 msgr2=0x7fbd9c160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.671+0000 7fbda41d0640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 0x7fbd9c160790 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fbd9802c2b0 tx=0x7fbd98004730 comp rx=0 tx=0).stop 2023-10-08T05:55:14.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.672+0000 7fbda41d0640 1 -- 172.21.15.70:0/3573040526 shutdown_connections 2023-10-08T05:55:14.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.672+0000 7fbda41d0640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd9c15c2e0 0x7fbd9c15c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.672+0000 7fbda41d0640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd9c1f79f0 0x7fbd9c0cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.672+0000 7fbda41d0640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd8006af90 0x7fbd8006d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.672+0000 7fbda41d0640 1 --2- 172.21.15.70:0/3573040526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd88003970 0x7fbd9c160790 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.672+0000 7fbda41d0640 1 -- 172.21.15.70:0/3573040526 >> 172.21.15.70:0/3573040526 conn(0x7fbd9c069d50 msgr2=0x7fbd9c0cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:14.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.672+0000 7fbda41d0640 1 -- 172.21.15.70:0/3573040526 shutdown_connections 2023-10-08T05:55:14.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.672+0000 7fbda41d0640 1 -- 172.21.15.70:0/3573040526 wait complete. 2023-10-08T05:55:14.692 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T05:55:14.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.909+0000 7f9b1a690640 1 Processor -- start 2023-10-08T05:55:14.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.909+0000 7f9b1a690640 1 -- start start 2023-10-08T05:55:14.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.911+0000 7f9b1a690640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b140afbe0 0x7f9b140b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.911+0000 7f9b1a690640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9b1406dfc0 0x7f9b14070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.911+0000 7f9b1a690640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9b14071070 0x7f9b140cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.911+0000 7f9b1a690640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b140b2620 con 0x7f9b14071070 2023-10-08T05:55:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.911+0000 7f9b1a690640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b140b2060 con 0x7f9b1406dfc0 2023-10-08T05:55:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.911+0000 7f9b1a690640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140b2340 con 0x7f9b140afbe0 2023-10-08T05:55:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.911+0000 7f9b18c2d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9b14071070 0x7f9b140cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.911+0000 7f9b18c2d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9b14071070 0x7f9b140cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42614/0 (socket says 172.21.15.70:42614) 2023-10-08T05:55:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.911+0000 7f9b18c2d640 1 -- 172.21.15.70:0/431593341 learned_addr learned my addr 172.21.15.70:0/431593341 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.912+0000 7f9b13fff640 1 --2- 172.21.15.70:0/431593341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b140afbe0 0x7f9b140b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.912+0000 7f9b18c2d640 1 -- 172.21.15.70:0/431593341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b140afbe0 msgr2=0x7f9b140b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.912+0000 7f9b18c2d640 1 --2- 172.21.15.70:0/431593341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b140afbe0 0x7f9b140b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.912+0000 7f9b18c2d640 1 -- 172.21.15.70:0/431593341 >> v2:172.21.15.181:3300/0 conn(0x7f9b1406dfc0 msgr2=0x7f9b14070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.912+0000 7f9b18c2d640 1 --2- 172.21.15.70:0/431593341 >> v2:172.21.15.181:3300/0 conn(0x7f9b1406dfc0 0x7f9b14070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.912+0000 7f9b18c2d640 1 -- 172.21.15.70:0/431593341 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b14070350 con 0x7f9b14071070 2023-10-08T05:55:14.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.912+0000 7f9b13fff640 1 --2- 172.21.15.70:0/431593341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b140afbe0 0x7f9b140b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.913+0000 7f9b18c2d640 1 --2- 172.21.15.70:0/431593341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b14071070 0x7f9b140cb130 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f9b140cb0b0 tx=0x7f9b0c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa6633feae324ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.913+0000 7f9b137fe640 1 -- 172.21.15.70:0/431593341 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c02ca80 con 0x7f9b14071070 2023-10-08T05:55:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.913+0000 7f9b137fe640 1 -- 172.21.15.70:0/431593341 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c02cbe0 con 0x7f9b14071070 2023-10-08T05:55:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.913+0000 7f9b137fe640 1 -- 172.21.15.70:0/431593341 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0343f0 con 0x7f9b14071070 2023-10-08T05:55:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.914+0000 7f9b1a690640 1 -- 172.21.15.70:0/431593341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b14071070 msgr2=0x7f9b140cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.914+0000 7f9b1a690640 1 --2- 172.21.15.70:0/431593341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b14071070 0x7f9b140cb130 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f9b140cb0b0 tx=0x7f9b0c02bee0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.914+0000 7f9b1a690640 1 -- 172.21.15.70:0/431593341 shutdown_connections 2023-10-08T05:55:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.914+0000 7f9b1a690640 1 --2- 172.21.15.70:0/431593341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b14071070 0x7f9b140cb130 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.914+0000 7f9b1a690640 1 --2- 172.21.15.70:0/431593341 >> v2:172.21.15.181:3300/0 conn(0x7f9b1406dfc0 0x7f9b14070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.914+0000 7f9b1a690640 1 --2- 172.21.15.70:0/431593341 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b140afbe0 0x7f9b140b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.914+0000 7f9b1a690640 1 -- 172.21.15.70:0/431593341 >> 172.21.15.70:0/431593341 conn(0x7f9b1406a5d0 msgr2=0x7f9b140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.914+0000 7f9b1a690640 1 -- 172.21.15.70:0/431593341 shutdown_connections 2023-10-08T05:55:14.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.914+0000 7f9b1a690640 1 -- 172.21.15.70:0/431593341 wait complete. 2023-10-08T05:55:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.915+0000 7f9b1a690640 1 Processor -- start 2023-10-08T05:55:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.916+0000 7f9b1a690640 1 -- start start 2023-10-08T05:55:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.916+0000 7f9b1a690640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b1406dfc0 0x7f9b140c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b1a690640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b14071070 0x7f9b140ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b1a690640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b140afbe0 0x7f9b141f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b1a690640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140caa60 con 0x7f9b1406dfc0 2023-10-08T05:55:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b1a690640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140cacc0 con 0x7f9b140afbe0 2023-10-08T05:55:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b1a690640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b141f5e00 con 0x7f9b14071070 2023-10-08T05:55:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b13fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b1406dfc0 0x7f9b140c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b13fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b1406dfc0 0x7f9b140c4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42628/0 (socket says 172.21.15.70:42628) 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b13fff640 1 -- 172.21.15.70:0/3566254501 learned_addr learned my addr 172.21.15.70:0/3566254501 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b13fff640 1 -- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b14071070 msgr2=0x7f9b140ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b13fff640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b14071070 0x7f9b140ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b13fff640 1 -- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b140afbe0 msgr2=0x7f9b141f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b13fff640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b140afbe0 0x7f9b141f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.917+0000 7f9b13fff640 1 -- 172.21.15.70:0/3566254501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c009b30 con 0x7f9b1406dfc0 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.918+0000 7f9b13fff640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b1406dfc0 0x7f9b140c4910 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400c7f0 tx=0x7f9b0400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.918+0000 7f9b117fa640 1 -- 172.21.15.70:0/3566254501 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04007a30 con 0x7f9b1406dfc0 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.918+0000 7f9b117fa640 1 -- 172.21.15.70:0/3566254501 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b04007b90 con 0x7f9b1406dfc0 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.918+0000 7f9b1a690640 1 -- 172.21.15.70:0/3566254501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b141f60e0 con 0x7f9b1406dfc0 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.918+0000 7f9b117fa640 1 -- 172.21.15.70:0/3566254501 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0400ce40 con 0x7f9b1406dfc0 2023-10-08T05:55:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.918+0000 7f9b1a690640 1 -- 172.21.15.70:0/3566254501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b141f66a0 con 0x7f9b1406dfc0 2023-10-08T05:55:14.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.919+0000 7f9b1a690640 1 -- 172.21.15.70:0/3566254501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b141f68c0 con 0x7f9b1406dfc0 2023-10-08T05:55:14.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.920+0000 7f9b117fa640 1 -- 172.21.15.70:0/3566254501 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9b040113f0 con 0x7f9b1406dfc0 2023-10-08T05:55:14.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.921+0000 7f9b117fa640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9af406af90 0x7f9af406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:14.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.921+0000 7f9b117fa640 1 -- 172.21.15.70:0/3566254501 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f9b04089c50 con 0x7f9b1406dfc0 2023-10-08T05:55:14.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.922+0000 7f9b0b7fe640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9af406af90 0x7f9af406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:14.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.923+0000 7f9b0b7fe640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9af406af90 0x7f9af406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9b140cf790 tx=0x7f9afc006bf0 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:14.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:14.925+0000 7f9b117fa640 1 -- 172.21.15.70:0/3566254501 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b04054950 con 0x7f9b1406dfc0 2023-10-08T05:55:15.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.113+0000 7f9b1a690640 1 -- 172.21.15.70:0/3566254501 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f9b140705e0 con 0x7f9b1406dfc0 2023-10-08T05:55:15.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.120+0000 7f9b117fa640 1 -- 172.21.15.70:0/3566254501 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v39) v1 ==== 120+0+0 (secure 0 0 0) 0x7f9b04057f70 con 0x7f9b1406dfc0 2023-10-08T05:55:15.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.122+0000 7f9b1a690640 1 -- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9af406af90 msgr2=0x7f9af406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:15.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.122+0000 7f9b1a690640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9af406af90 0x7f9af406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9b140cf790 tx=0x7f9afc006bf0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.122+0000 7f9b1a690640 1 -- 172.21.15.70:0/3566254501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b1406dfc0 msgr2=0x7f9b140c4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:15.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.122+0000 7f9b1a690640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b1406dfc0 0x7f9b140c4910 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400c7f0 tx=0x7f9b0400cbf0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.122+0000 7f9b1a690640 1 -- 172.21.15.70:0/3566254501 shutdown_connections 2023-10-08T05:55:15.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.122+0000 7f9b1a690640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9b140afbe0 0x7f9b141f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.122+0000 7f9b1a690640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9b14071070 0x7f9b140ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.122+0000 7f9b1a690640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9af406af90 0x7f9af406d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.122+0000 7f9b1a690640 1 --2- 172.21.15.70:0/3566254501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b1406dfc0 0x7f9b140c4910 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.123+0000 7f9b1a690640 1 -- 172.21.15.70:0/3566254501 >> 172.21.15.70:0/3566254501 conn(0x7f9b1406a5d0 msgr2=0x7f9b140c7190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:15.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.123+0000 7f9b1a690640 1 -- 172.21.15.70:0/3566254501 shutdown_connections 2023-10-08T05:55:15.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.123+0000 7f9b1a690640 1 -- 172.21.15.70:0/3566254501 wait complete. 2023-10-08T05:55:15.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-08T05:55:15.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.355+0000 7f457bb39640 1 Processor -- start 2023-10-08T05:55:15.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.355+0000 7f457bb39640 1 -- start start 2023-10-08T05:55:15.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.357+0000 7f457bb39640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f457415ab40 0x7f45740ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:15.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.357+0000 7f457bb39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f45740cb580 0x7f45740adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:15.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.357+0000 7f457bb39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f457415b2b0 con 0x7f45740c90b0 2023-10-08T05:55:15.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.357+0000 7f457bb39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457415bf30 con 0x7f45740cb580 2023-10-08T05:55:15.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.357+0000 7f457bb39640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457415f7a0 con 0x7f457415ab40 2023-10-08T05:55:15.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.357+0000 7f457a0d6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f45740cb580 0x7f45740adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:15.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.357+0000 7f457a0d6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f45740cb580 0x7f45740adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42634/0 (socket says 172.21.15.70:42634) 2023-10-08T05:55:15.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.357+0000 7f457a0d6640 1 -- 172.21.15.70:0/4190969742 learned_addr learned my addr 172.21.15.70:0/4190969742 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:15.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.358+0000 7f45798d5640 1 --2- 172.21.15.70:0/4190969742 >> v2:172.21.15.181:3300/0 conn(0x7f457415ab40 0x7f45740ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:15.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.358+0000 7f457a0d6640 1 -- 172.21.15.70:0/4190969742 >> v2:172.21.15.181:3300/0 conn(0x7f457415ab40 msgr2=0x7f45740ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:15.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.358+0000 7f457a0d6640 1 --2- 172.21.15.70:0/4190969742 >> v2:172.21.15.181:3300/0 conn(0x7f457415ab40 0x7f45740ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.358+0000 7f457a0d6640 1 -- 172.21.15.70:0/4190969742 >> v1:172.21.15.181:6789/0 conn(0x7f45740c90b0 legacy=0x7f45740cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:15.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.358+0000 7f457a0d6640 1 -- 172.21.15.70:0/4190969742 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457415b590 con 0x7f45740cb580 2023-10-08T05:55:15.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.359+0000 7f457a0d6640 1 --2- 172.21.15.70:0/4190969742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 0x7f45740adc60 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f457406e440 tx=0x7f456c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9f12e44ca8e029 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:15.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.359+0000 7f45788d3640 1 -- 172.21.15.70:0/4190969742 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c02cb10 con 0x7f45740cb580 2023-10-08T05:55:15.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.360+0000 7f45788d3640 1 -- 172.21.15.70:0/4190969742 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c02cc70 con 0x7f45740cb580 2023-10-08T05:55:15.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.360+0000 7f45788d3640 1 -- 172.21.15.70:0/4190969742 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c0343f0 con 0x7f45740cb580 2023-10-08T05:55:15.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.360+0000 7f457bb39640 1 -- 172.21.15.70:0/4190969742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 msgr2=0x7f45740adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:15.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.360+0000 7f457bb39640 1 --2- 172.21.15.70:0/4190969742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 0x7f45740adc60 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f457406e440 tx=0x7f456c02bf70 comp rx=0 tx=0).stop 2023-10-08T05:55:15.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.360+0000 7f457bb39640 1 -- 172.21.15.70:0/4190969742 shutdown_connections 2023-10-08T05:55:15.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.360+0000 7f457bb39640 1 --2- 172.21.15.70:0/4190969742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 0x7f45740adc60 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.360+0000 7f457bb39640 1 --2- 172.21.15.70:0/4190969742 >> v2:172.21.15.181:3300/0 conn(0x7f457415ab40 0x7f45740ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.360+0000 7f457bb39640 1 -- 172.21.15.70:0/4190969742 >> 172.21.15.70:0/4190969742 conn(0x7f457406a380 msgr2=0x7f45740b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:15.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.360+0000 7f457bb39640 1 -- 172.21.15.70:0/4190969742 shutdown_connections 2023-10-08T05:55:15.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.361+0000 7f457bb39640 1 -- 172.21.15.70:0/4190969742 wait complete. 2023-10-08T05:55:15.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.362+0000 7f457bb39640 1 Processor -- start 2023-10-08T05:55:15.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.362+0000 7f457bb39640 1 -- start start 2023-10-08T05:55:15.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.363+0000 7f457bb39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45740c90b0 0x7f4574153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:15.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.363+0000 7f457bb39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 0x7f45740abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:15.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.363+0000 7f457bb39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457415ab40 0x7f45740b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:15.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.363+0000 7f457bb39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45740b2450 con 0x7f45740cb580 2023-10-08T05:55:15.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.363+0000 7f457bb39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45740ac4f0 con 0x7f45740c90b0 2023-10-08T05:55:15.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.363+0000 7f457bb39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45740ad070 con 0x7f457415ab40 2023-10-08T05:55:15.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f45790d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 0x7f45740abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:15.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f45790d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 0x7f45740abe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42644/0 (socket says 172.21.15.70:42644) 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f45790d4640 1 -- 172.21.15.70:0/2461620597 learned_addr learned my addr 172.21.15.70:0/2461620597 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f45798d5640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45740c90b0 0x7f4574153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f457a0d6640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457415ab40 0x7f45740b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f45790d4640 1 -- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457415ab40 msgr2=0x7f45740b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f45790d4640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457415ab40 0x7f45740b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f45790d4640 1 -- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45740c90b0 msgr2=0x7f4574153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f45790d4640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45740c90b0 0x7f4574153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f45790d4640 1 -- 172.21.15.70:0/2461620597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c009b30 con 0x7f45740cb580 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.364+0000 7f45798d5640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45740c90b0 0x7f4574153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.365+0000 7f45790d4640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 0x7f45740abe00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f456400c3c0 tx=0x7f456400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.365+0000 7f456a7fc640 1 -- 172.21.15.70:0/2461620597 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45640052d0 con 0x7f45740cb580 2023-10-08T05:55:15.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.365+0000 7f456a7fc640 1 -- 172.21.15.70:0/2461620597 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4564005450 con 0x7f45740cb580 2023-10-08T05:55:15.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.365+0000 7f456a7fc640 1 -- 172.21.15.70:0/2461620597 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564003830 con 0x7f45740cb580 2023-10-08T05:55:15.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.365+0000 7f457bb39640 1 -- 172.21.15.70:0/2461620597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45740b21b0 con 0x7f45740cb580 2023-10-08T05:55:15.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.365+0000 7f457bb39640 1 -- 172.21.15.70:0/2461620597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45740aca20 con 0x7f45740cb580 2023-10-08T05:55:15.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.366+0000 7f457bb39640 1 -- 172.21.15.70:0/2461620597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45740ac340 con 0x7f45740cb580 2023-10-08T05:55:15.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.367+0000 7f456a7fc640 1 -- 172.21.15.70:0/2461620597 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4564003990 con 0x7f45740cb580 2023-10-08T05:55:15.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.368+0000 7f456a7fc640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f455806af90 0x7f455806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:15.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.368+0000 7f456a7fc640 1 -- 172.21.15.70:0/2461620597 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f4564088eb0 con 0x7f45740cb580 2023-10-08T05:55:15.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.369+0000 7f45798d5640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f455806af90 0x7f455806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:15.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.370+0000 7f45798d5640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f455806af90 0x7f455806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4570009ac0 tx=0x7f4570008040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:15.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.372+0000 7f456a7fc640 1 -- 172.21.15.70:0/2461620597 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4564053c30 con 0x7f45740cb580 2023-10-08T05:55:15.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.632+0000 7f457bb39640 1 -- 172.21.15.70:0/2461620597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f457415bb20 con 0x7f45740cb580 2023-10-08T05:55:15.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.800+0000 7f456a7fc640 1 -- 172.21.15.70:0/2461620597 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v730) v1 ==== 183+0+0 (secure 0 0 0) 0x7f45640170f0 con 0x7f45740cb580 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 -- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f455806af90 msgr2=0x7f455806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f455806af90 0x7f455806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4570009ac0 tx=0x7f4570008040 comp rx=0 tx=0).stop 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 -- 172.21.15.70:0/2461620597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 msgr2=0x7f45740abe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 0x7f45740abe00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f456400c3c0 tx=0x7f456400c880 comp rx=0 tx=0).stop 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 -- 172.21.15.70:0/2461620597 shutdown_connections 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f457415ab40 0x7f45740b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f455806af90 0x7f455806d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f45740cb580 0x7f45740abe00 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 --2- 172.21.15.70:0/2461620597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f45740c90b0 0x7f4574153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 -- 172.21.15.70:0/2461620597 >> 172.21.15.70:0/2461620597 conn(0x7f457406a380 msgr2=0x7f45740ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:15.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 -- 172.21.15.70:0/2461620597 shutdown_connections 2023-10-08T05:55:15.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:15.803+0000 7f457bb39640 1 -- 172.21.15.70:0/2461620597 wait complete. 2023-10-08T05:55:15.826 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-10-08T05:55:15.827 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-10-08T05:55:15.827 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T05:55:15.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:55:16.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.035+0000 7f77fa4aa640 1 Processor -- start 2023-10-08T05:55:16.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.035+0000 7f77fa4aa640 1 -- start start 2023-10-08T05:55:16.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.036+0000 7f77fa4aa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77f40cc210 0x7f77f40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.036+0000 7f77fa4aa640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f77f4159db0 0x7f77f40b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.036+0000 7f77fa4aa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f40cacf0 con 0x7f77f41599d0 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.036+0000 7f77fa4aa640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f40b10f0 con 0x7f77f40cc210 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.036+0000 7f77fa4aa640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f40ade40 con 0x7f77f4159db0 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.037+0000 7f77f94a8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f77f41599d0 0x7f77f40ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41048/0 (socket says 172.21.15.70:41048) 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.037+0000 7f77f94a8640 1 -- 172.21.15.70:0/245351695 learned_addr learned my addr 172.21.15.70:0/245351695 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.037+0000 7f77f8ca7640 1 --2- 172.21.15.70:0/245351695 >> v2:172.21.15.70:3300/0 conn(0x7f77f40cc210 0x7f77f40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.037+0000 7f77f9ca9640 1 --2- 172.21.15.70:0/245351695 >> v2:172.21.15.181:3300/0 conn(0x7f77f4159db0 0x7f77f40b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.037+0000 7f77f8ca7640 1 -- 172.21.15.70:0/245351695 >> v2:172.21.15.181:3300/0 conn(0x7f77f4159db0 msgr2=0x7f77f40b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.037+0000 7f77f8ca7640 1 --2- 172.21.15.70:0/245351695 >> v2:172.21.15.181:3300/0 conn(0x7f77f4159db0 0x7f77f40b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.037+0000 7f77f8ca7640 1 -- 172.21.15.70:0/245351695 >> v1:172.21.15.70:6789/0 conn(0x7f77f41599d0 legacy=0x7f77f40ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.037+0000 7f77f8ca7640 1 -- 172.21.15.70:0/245351695 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f40ca3c0 con 0x7f77f40cc210 2023-10-08T05:55:16.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.037+0000 7f77ebfff640 1 -- 172.21.15.70:0/245351695 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725112801 0 0) 0x7f77f40cacf0 con 0x7f77f41599d0 2023-10-08T05:55:16.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.038+0000 7f77f8ca7640 1 --2- 172.21.15.70:0/245351695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f40cc210 0x7f77f40b00c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f77f40cb070 tx=0x7f77e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849dcb4d94a7c07a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:16.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77ebfff640 1 -- 172.21.15.70:0/245351695 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4031830 con 0x7f77f40cc210 2023-10-08T05:55:16.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77ebfff640 1 -- 172.21.15.70:0/245351695 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e4031990 con 0x7f77f40cc210 2023-10-08T05:55:16.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77ebfff640 1 -- 172.21.15.70:0/245351695 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e4031d20 con 0x7f77f40cc210 2023-10-08T05:55:16.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77fa4aa640 1 -- 172.21.15.70:0/245351695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f40cc210 msgr2=0x7f77f40b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77fa4aa640 1 --2- 172.21.15.70:0/245351695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f40cc210 0x7f77f40b00c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f77f40cb070 tx=0x7f77e402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77fa4aa640 1 -- 172.21.15.70:0/245351695 shutdown_connections 2023-10-08T05:55:16.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77fa4aa640 1 --2- 172.21.15.70:0/245351695 >> v2:172.21.15.181:3300/0 conn(0x7f77f4159db0 0x7f77f40b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77fa4aa640 1 --2- 172.21.15.70:0/245351695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f40cc210 0x7f77f40b00c0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77fa4aa640 1 -- 172.21.15.70:0/245351695 >> 172.21.15.70:0/245351695 conn(0x7f77f40cb0c0 msgr2=0x7f77f40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:16.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77fa4aa640 1 -- 172.21.15.70:0/245351695 shutdown_connections 2023-10-08T05:55:16.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.039+0000 7f77fa4aa640 1 -- 172.21.15.70:0/245351695 wait complete. 2023-10-08T05:55:16.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.041+0000 7f77fa4aa640 1 Processor -- start 2023-10-08T05:55:16.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.041+0000 7f77fa4aa640 1 -- start start 2023-10-08T05:55:16.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.042+0000 7f77fa4aa640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40cc210 0x7f77f40aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.042+0000 7f77fa4aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f41599d0 0x7f77f40a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.042+0000 7f77fa4aa640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4159db0 0x7f77f40b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.042+0000 7f77fa4aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40ab440 con 0x7f77f41599d0 2023-10-08T05:55:16.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.042+0000 7f77fa4aa640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40aaba0 con 0x7f77f40cc210 2023-10-08T05:55:16.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.042+0000 7f77fa4aa640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40aae80 con 0x7f77f4159db0 2023-10-08T05:55:16.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f8ca7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f41599d0 0x7f77f40a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f8ca7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f41599d0 0x7f77f40a74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42662/0 (socket says 172.21.15.70:42662) 2023-10-08T05:55:16.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f8ca7640 1 -- 172.21.15.70:0/2540341446 learned_addr learned my addr 172.21.15.70:0/2540341446 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:16.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f94a8640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40cc210 0x7f77f40aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f8ca7640 1 -- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4159db0 msgr2=0x7f77f40b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f9ca9640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4159db0 0x7f77f40b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f8ca7640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4159db0 0x7f77f40b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f8ca7640 1 -- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40cc210 msgr2=0x7f77f40aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f8ca7640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40cc210 0x7f77f40aa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f8ca7640 1 -- 172.21.15.70:0/2540341446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e4009400 con 0x7f77f41599d0 2023-10-08T05:55:16.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.043+0000 7f77f94a8640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40cc210 0x7f77f40aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:16.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.044+0000 7f77f8ca7640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f41599d0 0x7f77f40a74c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009890 tx=0x7f77e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.044+0000 7f77e9ffb640 1 -- 172.21.15.70:0/2540341446 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e403ba70 con 0x7f77f41599d0 2023-10-08T05:55:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.044+0000 7f77e9ffb640 1 -- 172.21.15.70:0/2540341446 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e403bbd0 con 0x7f77f41599d0 2023-10-08T05:55:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.044+0000 7f77e9ffb640 1 -- 172.21.15.70:0/2540341446 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e40453f0 con 0x7f77f41599d0 2023-10-08T05:55:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.044+0000 7f77fa4aa640 1 -- 172.21.15.70:0/2540341446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f40ab160 con 0x7f77f41599d0 2023-10-08T05:55:16.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.044+0000 7f77fa4aa640 1 -- 172.21.15.70:0/2540341446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f41581f0 con 0x7f77f41599d0 2023-10-08T05:55:16.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.045+0000 7f77fa4aa640 1 -- 172.21.15.70:0/2540341446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f40a8c80 con 0x7f77f41599d0 2023-10-08T05:55:16.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.046+0000 7f77e9ffb640 1 -- 172.21.15.70:0/2540341446 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f77e4045550 con 0x7f77f41599d0 2023-10-08T05:55:16.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.047+0000 7f77e9ffb640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f77d806af90 0x7f77d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.047+0000 7f77e9ffb640 1 -- 172.21.15.70:0/2540341446 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f77e40abff0 con 0x7f77f41599d0 2023-10-08T05:55:16.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.048+0000 7f77f94a8640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f77d806af90 0x7f77d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.049+0000 7f77f94a8640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f77d806af90 0x7f77d806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77f0000f60 tx=0x7f77f0009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:16.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.051+0000 7f77e9ffb640 1 -- 172.21.15.70:0/2540341446 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77f40a8c80 con 0x7f77f41599d0 2023-10-08T05:55:16.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.242+0000 7f77fa4aa640 1 -- 172.21.15.70:0/2540341446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77f40a8c80 con 0x7f77f41599d0 2023-10-08T05:55:16.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.244+0000 7f77e9ffb640 1 -- 172.21.15.70:0/2540341446 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+14404 (secure 0 0 0) 0x7f77f40a8c80 con 0x7f77f41599d0 2023-10-08T05:55:16.246 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:16.246 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":125,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:13.772240+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6835/422082772":"2023-10-09T05:55:10.546508+0000","172.21.15.181:6834/422082772":"2023-10-09T05:55:10.546508+0000","172.21.15.181:6833/1714252273":"2023-10-09T05:55:06.712118+0000","172.21.15.181:6832/1714252273":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1721597867":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1614829749":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/2367948918":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/3835712371":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/2025529952":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1108570452":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/1700228937":"2023-10-09T05:55:06.712118+0000","172.21.15.181:0/3426004759":"2023-10-09T05:55:06.712118+0000","172.21.15.181:6833/1906631919":"2023-10-09T05:53:58.959064+0000","172.21.15.181:6832/1906631919":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3209826772":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/1283682257":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/2590863547":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3077797034":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/3436033120":"2023-10-09T05:53:58.959064+0000","172.21.15.181:0/168987985":"2023-10-09T05:53:58.959064+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:55:16.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 -- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f77d806af90 msgr2=0x7f77d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f77d806af90 0x7f77d806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77f0000f60 tx=0x7f77f0009040 comp rx=0 tx=0).stop 2023-10-08T05:55:16.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 -- 172.21.15.70:0/2540341446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f41599d0 msgr2=0x7f77f40a74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f41599d0 0x7f77f40a74c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009890 tx=0x7f77e40046c0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 -- 172.21.15.70:0/2540341446 shutdown_connections 2023-10-08T05:55:16.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77f4159db0 0x7f77f40b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f77d806af90 0x7f77d806d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77f41599d0 0x7f77f40a74c0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 --2- 172.21.15.70:0/2540341446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77f40cc210 0x7f77f40aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 -- 172.21.15.70:0/2540341446 >> 172.21.15.70:0/2540341446 conn(0x7f77f40cb0c0 msgr2=0x7f77f40cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:16.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 -- 172.21.15.70:0/2540341446 shutdown_connections 2023-10-08T05:55:16.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.247+0000 7f77fa4aa640 1 -- 172.21.15.70:0/2540341446 wait complete. 2023-10-08T05:55:16.265 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:55:16.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:55:16.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.475+0000 7f9c6478a640 1 Processor -- start 2023-10-08T05:55:16.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.475+0000 7f9c6478a640 1 -- start start 2023-10-08T05:55:16.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.477+0000 7f9c6478a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c5c15aad0 0x7f9c5c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.477+0000 7f9c6478a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c15bf30 con 0x7f9c5c06d410 2023-10-08T05:55:16.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.477+0000 7f9c6478a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c15b320 con 0x7f9c5c0c82b0 2023-10-08T05:55:16.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.477+0000 7f9c6478a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c15b600 con 0x7f9c5c15aad0 2023-10-08T05:55:16.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.477+0000 7f9c62d27640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9c5c06d410 0x7f9c5c070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41056/0 (socket says 172.21.15.70:41056) 2023-10-08T05:55:16.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.477+0000 7f9c62d27640 1 -- 172.21.15.70:0/2245391969 learned_addr learned my addr 172.21.15.70:0/2245391969 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.477+0000 7f9c62526640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c5c15aad0 0x7f9c5c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663389317 0 0) 0x7f9c5c15bf30 con 0x7f9c5c06d410 2023-10-08T05:55:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c48003520 con 0x7f9c5c06d410 2023-10-08T05:55:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2681612995 0 0) 0x7f9c48003520 con 0x7f9c5c06d410 2023-10-08T05:55:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c5c15aad0 msgr2=0x7f9c5c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c5c15aad0 0x7f9c5c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 >> v1:172.21.15.181:6789/0 conn(0x7f9c5c0c82b0 legacy=0x7f9c5c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c15b8e0 con 0x7f9c5c06d410 2023-10-08T05:55:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9c540019f0 con 0x7f9c5c06d410 2023-10-08T05:55:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 >> v1:172.21.15.70:6789/0 conn(0x7f9c5c06d410 legacy=0x7f9c5c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c480039a0 0x7f9c48003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c480047c0 0x7f9c48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c5c15aad0 0x7f9c48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c15b600 con 0x7f9c480039a0 2023-10-08T05:55:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4800ba30 con 0x7f9c480047c0 2023-10-08T05:55:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.478+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4800bbf0 con 0x7f9c5c15aad0 2023-10-08T05:55:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.479+0000 7f9c62526640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c480039a0 0x7f9c48003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.479+0000 7f9c62d27640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c5c15aad0 0x7f9c48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.479+0000 7f9c61d25640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c480047c0 0x7f9c48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.479+0000 7f9c62526640 1 -- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c5c15aad0 msgr2=0x7f9c48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:16.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.479+0000 7f9c62526640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c5c15aad0 0x7f9c48005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.479+0000 7f9c62526640 1 -- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c480047c0 msgr2=0x7f9c48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:16.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.479+0000 7f9c62526640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c480047c0 0x7f9c48004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.479+0000 7f9c62526640 1 -- 172.21.15.70:0/2245391969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c48003520 con 0x7f9c480039a0 2023-10-08T05:55:16.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.479+0000 7f9c61d25640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c480047c0 0x7f9c48004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:16.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.480+0000 7f9c62526640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c480039a0 0x7f9c48003dc0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009a90 tx=0x7f9c480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0133cc33dd1dab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:16.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.480+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5802c660 con 0x7f9c480039a0 2023-10-08T05:55:16.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.481+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5802c7c0 con 0x7f9c480039a0 2023-10-08T05:55:16.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.481+0000 7f9c61524640 1 -- 172.21.15.70:0/2245391969 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5802cb50 con 0x7f9c480039a0 2023-10-08T05:55:16.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.481+0000 7f9c6478a640 1 -- 172.21.15.70:0/2245391969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c480039a0 msgr2=0x7f9c48003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.481+0000 7f9c6478a640 1 --2- 172.21.15.70:0/2245391969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c480039a0 0x7f9c48003dc0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009a90 tx=0x7f9c480057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.481+0000 7f9c62d27640 1 -- 172.21.15.70:0/2245391969 reap_dead start 2023-10-08T05:55:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.481+0000 7f9c6478a640 1 -- 172.21.15.70:0/2245391969 shutdown_connections 2023-10-08T05:55:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.481+0000 7f9c6478a640 1 -- 172.21.15.70:0/2245391969 >> 172.21.15.70:0/2245391969 conn(0x7f9c5c069d50 msgr2=0x7f9c5c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.481+0000 7f9c6478a640 1 -- 172.21.15.70:0/2245391969 shutdown_connections 2023-10-08T05:55:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.481+0000 7f9c6478a640 1 -- 172.21.15.70:0/2245391969 wait complete. 2023-10-08T05:55:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.483+0000 7f9c6478a640 1 Processor -- start 2023-10-08T05:55:16.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.483+0000 7f9c6478a640 1 -- start start 2023-10-08T05:55:16.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.484+0000 7f9c6478a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c480039a0 0x7f9c5c148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.484+0000 7f9c6478a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c5c0cc590 0x7f9c5c0cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.484+0000 7f9c6478a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c0ccf50 0x7f9c5c1edb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.484+0000 7f9c6478a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0b31b0 con 0x7f9c5c0cc590 2023-10-08T05:55:16.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.484+0000 7f9c6478a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0cd3b0 con 0x7f9c5c0ccf50 2023-10-08T05:55:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.484+0000 7f9c6478a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0b2630 con 0x7f9c480039a0 2023-10-08T05:55:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.484+0000 7f9c61d25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c5c0cc590 0x7f9c5c0cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.484+0000 7f9c61d25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c5c0cc590 0x7f9c5c0cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42690/0 (socket says 172.21.15.70:42690) 2023-10-08T05:55:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.484+0000 7f9c61d25640 1 -- 172.21.15.70:0/1078531679 learned_addr learned my addr 172.21.15.70:0/1078531679 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c62526640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c480039a0 0x7f9c5c148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c62d27640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c0ccf50 0x7f9c5c1edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c61d25640 1 -- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c480039a0 msgr2=0x7f9c5c148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c61d25640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c480039a0 0x7f9c5c148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c61d25640 1 -- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c0ccf50 msgr2=0x7f9c5c1edb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c61d25640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c0ccf50 0x7f9c5c1edb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c61d25640 1 -- 172.21.15.70:0/1078531679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c0b2910 con 0x7f9c5c0cc590 2023-10-08T05:55:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c62d27640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c0ccf50 0x7f9c5c1edb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c61d25640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c5c0cc590 0x7f9c5c0cca10 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00d7a0 tx=0x7f9c4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c52ffd640 1 -- 172.21.15.70:0/1078531679 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c017070 con 0x7f9c5c0cc590 2023-10-08T05:55:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c52ffd640 1 -- 172.21.15.70:0/1078531679 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c005070 con 0x7f9c5c0cc590 2023-10-08T05:55:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c52ffd640 1 -- 172.21.15.70:0/1078531679 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c005420 con 0x7f9c5c0cc590 2023-10-08T05:55:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.485+0000 7f9c6478a640 1 -- 172.21.15.70:0/1078531679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c0b2bf0 con 0x7f9c5c0cc590 2023-10-08T05:55:16.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.486+0000 7f9c6478a640 1 -- 172.21.15.70:0/1078531679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c0b22f0 con 0x7f9c5c0cc590 2023-10-08T05:55:16.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.487+0000 7f9c6478a640 1 -- 172.21.15.70:0/1078531679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c04e160 con 0x7f9c5c0cc590 2023-10-08T05:55:16.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.487+0000 7f9c52ffd640 1 -- 172.21.15.70:0/1078531679 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9c4c007030 con 0x7f9c5c0cc590 2023-10-08T05:55:16.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.488+0000 7f9c52ffd640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9c4006af90 0x7f9c4006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.488+0000 7f9c52ffd640 1 -- 172.21.15.70:0/1078531679 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f9c4c089340 con 0x7f9c5c0cc590 2023-10-08T05:55:16.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.489+0000 7f9c62526640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9c4006af90 0x7f9c4006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.490+0000 7f9c62526640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9c4006af90 0x7f9c4006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9c58036000 tx=0x7f9c58030000 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:16.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.493+0000 7f9c52ffd640 1 -- 172.21.15.70:0/1078531679 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c4c054030 con 0x7f9c5c0cc590 2023-10-08T05:55:16.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.679+0000 7f9c6478a640 1 -- 172.21.15.70:0/1078531679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c5c15c1a0 con 0x7f9c5c0cc590 2023-10-08T05:55:16.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.680+0000 7f9c52ffd640 1 -- 172.21.15.70:0/1078531679 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c4c057650 con 0x7f9c5c0cc590 2023-10-08T05:55:16.681 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:55:16.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.682+0000 7f9c6478a640 1 -- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9c4006af90 msgr2=0x7f9c4006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.682+0000 7f9c6478a640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9c4006af90 0x7f9c4006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9c58036000 tx=0x7f9c58030000 comp rx=0 tx=0).stop 2023-10-08T05:55:16.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.682+0000 7f9c6478a640 1 -- 172.21.15.70:0/1078531679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c5c0cc590 msgr2=0x7f9c5c0cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.682+0000 7f9c6478a640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c5c0cc590 0x7f9c5c0cca10 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00d7a0 tx=0x7f9c4c00dc60 comp rx=0 tx=0).stop 2023-10-08T05:55:16.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.682+0000 7f9c6478a640 1 -- 172.21.15.70:0/1078531679 shutdown_connections 2023-10-08T05:55:16.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.682+0000 7f9c6478a640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c5c0ccf50 0x7f9c5c1edb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.682+0000 7f9c6478a640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f9c4006af90 0x7f9c4006d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.682+0000 7f9c6478a640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c5c0cc590 0x7f9c5c0cca10 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.682+0000 7f9c6478a640 1 --2- 172.21.15.70:0/1078531679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c480039a0 0x7f9c5c148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.682+0000 7f9c6478a640 1 -- 172.21.15.70:0/1078531679 >> 172.21.15.70:0/1078531679 conn(0x7f9c5c069d50 msgr2=0x7f9c5c0ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.683+0000 7f9c6478a640 1 -- 172.21.15.70:0/1078531679 shutdown_connections 2023-10-08T05:55:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.683+0000 7f9c6478a640 1 -- 172.21.15.70:0/1078531679 wait complete. 2023-10-08T05:55:16.702 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-08T05:55:16.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.912+0000 7f039e2a3640 1 Processor -- start 2023-10-08T05:55:16.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.912+0000 7f039e2a3640 1 -- start start 2023-10-08T05:55:16.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.913+0000 7f039e2a3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb450 0x7f03980ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.913+0000 7f039e2a3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03980cb830 0x7f03980c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.913+0000 7f039e2a3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f039815d060 0x7f039806d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.913+0000 7f039e2a3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03980c9f50 con 0x7f03980cb830 2023-10-08T05:55:16.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.913+0000 7f039e2a3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039806caa0 con 0x7f039815d060 2023-10-08T05:55:16.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.913+0000 7f039e2a3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039815d970 con 0x7f03980cb450 2023-10-08T05:55:16.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.914+0000 7f039caa0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03980cb830 0x7f03980c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.914+0000 7f039caa0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03980cb830 0x7f03980c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42700/0 (socket says 172.21.15.70:42700) 2023-10-08T05:55:16.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.914+0000 7f039caa0640 1 -- 172.21.15.70:0/1052377831 learned_addr learned my addr 172.21.15.70:0/1052377831 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:16.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.914+0000 7f039caa0640 1 -- 172.21.15.70:0/1052377831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb450 msgr2=0x7f03980ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:16.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.914+0000 7f039d2a1640 1 --2- 172.21.15.70:0/1052377831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb450 0x7f03980ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.914+0000 7f039caa0640 1 --2- 172.21.15.70:0/1052377831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb450 0x7f03980ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.914+0000 7f039daa2640 1 --2- 172.21.15.70:0/1052377831 >> v2:172.21.15.181:3300/0 conn(0x7f039815d060 0x7f039806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.914+0000 7f039caa0640 1 -- 172.21.15.70:0/1052377831 >> v2:172.21.15.181:3300/0 conn(0x7f039815d060 msgr2=0x7f039806d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:16.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.914+0000 7f039caa0640 1 --2- 172.21.15.70:0/1052377831 >> v2:172.21.15.181:3300/0 conn(0x7f039815d060 0x7f039806d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.914+0000 7f039caa0640 1 -- 172.21.15.70:0/1052377831 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039815dc50 con 0x7f03980cb830 2023-10-08T05:55:16.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.915+0000 7f039caa0640 1 --2- 172.21.15.70:0/1052377831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb830 0x7f03980c8c00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f039806c960 tx=0x7f038802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736a81bfdd308022 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:16.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f038ffff640 1 -- 172.21.15.70:0/1052377831 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388031870 con 0x7f03980cb830 2023-10-08T05:55:16.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f038ffff640 1 -- 172.21.15.70:0/1052377831 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03880319d0 con 0x7f03980cb830 2023-10-08T05:55:16.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f038ffff640 1 -- 172.21.15.70:0/1052377831 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388031d60 con 0x7f03980cb830 2023-10-08T05:55:16.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f039e2a3640 1 -- 172.21.15.70:0/1052377831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb830 msgr2=0x7f03980c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f039e2a3640 1 --2- 172.21.15.70:0/1052377831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb830 0x7f03980c8c00 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f039806c960 tx=0x7f038802b910 comp rx=0 tx=0).stop 2023-10-08T05:55:16.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f039e2a3640 1 -- 172.21.15.70:0/1052377831 shutdown_connections 2023-10-08T05:55:16.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f039e2a3640 1 --2- 172.21.15.70:0/1052377831 >> v2:172.21.15.181:3300/0 conn(0x7f039815d060 0x7f039806d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f039e2a3640 1 --2- 172.21.15.70:0/1052377831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb830 0x7f03980c8c00 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f039e2a3640 1 --2- 172.21.15.70:0/1052377831 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb450 0x7f03980ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f039e2a3640 1 -- 172.21.15.70:0/1052377831 >> 172.21.15.70:0/1052377831 conn(0x7f03980ce9d0 msgr2=0x7f03980cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:16.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f039e2a3640 1 -- 172.21.15.70:0/1052377831 shutdown_connections 2023-10-08T05:55:16.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.916+0000 7f039e2a3640 1 -- 172.21.15.70:0/1052377831 wait complete. 2023-10-08T05:55:16.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.918+0000 7f039e2a3640 1 Processor -- start 2023-10-08T05:55:16.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.918+0000 7f039e2a3640 1 -- start start 2023-10-08T05:55:16.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.919+0000 7f039e2a3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03980cb450 0x7f0398159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.919+0000 7f039e2a3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb830 0x7f03980c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.919+0000 7f039e2a3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f039815d060 0x7f0398071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.919+0000 7f039e2a3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0398159e80 con 0x7f039815d060 2023-10-08T05:55:16.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.919+0000 7f039e2a3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03980c6040 con 0x7f03980cb450 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.919+0000 7f039e2a3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03980c6320 con 0x7f03980cb830 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039daa2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f039815d060 0x7f0398071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039daa2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f039815d060 0x7f0398071090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42710/0 (socket says 172.21.15.70:42710) 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039daa2640 1 -- 172.21.15.70:0/3237572128 learned_addr learned my addr 172.21.15.70:0/3237572128 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039caa0640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb830 0x7f03980c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039d2a1640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03980cb450 0x7f0398159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039daa2640 1 -- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb830 msgr2=0x7f03980c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039daa2640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb830 0x7f03980c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039daa2640 1 -- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03980cb450 msgr2=0x7f0398159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039daa2640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03980cb450 0x7f0398159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:16.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039daa2640 1 -- 172.21.15.70:0/3237572128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03880094d0 con 0x7f039815d060 2023-10-08T05:55:16.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039caa0640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb830 0x7f03980c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:16.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f039daa2640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f039815d060 0x7f0398071090 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f039000cd70 tx=0x7f039000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:16.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.920+0000 7f038dffb640 1 -- 172.21.15.70:0/3237572128 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390003b90 con 0x7f039815d060 2023-10-08T05:55:16.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.921+0000 7f038dffb640 1 -- 172.21.15.70:0/3237572128 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390003cf0 con 0x7f039815d060 2023-10-08T05:55:16.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.921+0000 7f038dffb640 1 -- 172.21.15.70:0/3237572128 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390004240 con 0x7f039815d060 2023-10-08T05:55:16.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.921+0000 7f039e2a3640 1 -- 172.21.15.70:0/3237572128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03980c6600 con 0x7f039815d060 2023-10-08T05:55:16.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.921+0000 7f039e2a3640 1 -- 172.21.15.70:0/3237572128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03980c6bc0 con 0x7f039815d060 2023-10-08T05:55:16.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.922+0000 7f039e2a3640 1 -- 172.21.15.70:0/3237572128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03980b33e0 con 0x7f039815d060 2023-10-08T05:55:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.923+0000 7f038dffb640 1 -- 172.21.15.70:0/3237572128 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f03900043a0 con 0x7f039815d060 2023-10-08T05:55:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.923+0000 7f038dffb640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f037c06af90 0x7f037c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.924+0000 7f038dffb640 1 -- 172.21.15.70:0/3237572128 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f0390012070 con 0x7f039815d060 2023-10-08T05:55:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.924+0000 7f039d2a1640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f037c06af90 0x7f037c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:16.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.926+0000 7f039d2a1640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f037c06af90 0x7f037c06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f039400b740 tx=0x7f0394008040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:16.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:16.927+0000 7f038dffb640 1 -- 172.21.15.70:0/3237572128 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0390055160 con 0x7f039815d060 2023-10-08T05:55:17.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.165+0000 7f039e2a3640 1 -- 172.21.15.70:0/3237572128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f039804e160 con 0x7f039815d060 2023-10-08T05:55:17.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.812+0000 7f038dffb640 1 -- 172.21.15.70:0/3237572128 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v732) v1 ==== 151+0+0 (secure 0 0 0) 0x7f0390058780 con 0x7f039815d060 2023-10-08T05:55:17.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 -- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f037c06af90 msgr2=0x7f037c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:17.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f037c06af90 0x7f037c06d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f039400b740 tx=0x7f0394008040 comp rx=0 tx=0).stop 2023-10-08T05:55:17.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 -- 172.21.15.70:0/3237572128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f039815d060 msgr2=0x7f0398071090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:17.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f039815d060 0x7f0398071090 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f039000cd70 tx=0x7f039000a660 comp rx=0 tx=0).stop 2023-10-08T05:55:17.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 -- 172.21.15.70:0/3237572128 shutdown_connections 2023-10-08T05:55:17.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f037c06af90 0x7f037c06d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:17.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f039815d060 0x7f0398071090 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:17.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb830 0x7f03980c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:17.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 --2- 172.21.15.70:0/3237572128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03980cb450 0x7f0398159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:17.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 -- 172.21.15.70:0/3237572128 >> 172.21.15.70:0/3237572128 conn(0x7f03980ce9d0 msgr2=0x7f03980af410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:17.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 -- 172.21.15.70:0/3237572128 shutdown_connections 2023-10-08T05:55:17.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:17.816+0000 7f039e2a3640 1 -- 172.21.15.70:0/3237572128 wait complete. 2023-10-08T05:55:17.838 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T05:55:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.053+0000 7f7bc6490640 1 Processor -- start 2023-10-08T05:55:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.053+0000 7f7bc6490640 1 -- start start 2023-10-08T05:55:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.054+0000 7f7bc6490640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7bc015cc70 0x7f7bc00cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.054+0000 7f7bc6490640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bc00d31a0 0x7f7bc00b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.054+0000 7f7bc6490640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 0x7f7bc00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.054+0000 7f7bc6490640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc00b8b00 con 0x7f7bc00d31a0 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.054+0000 7f7bc6490640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc00b8540 con 0x7f7bc015cc70 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.054+0000 7f7bc6490640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc00b8820 con 0x7f7bc00b61f0 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.054+0000 7f7bb7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bc00d31a0 0x7f7bc00b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bb7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bc00d31a0 0x7f7bc00b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42724/0 (socket says 172.21.15.70:42724) 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bb7fff640 1 -- 172.21.15.70:0/142705404 learned_addr learned my addr 172.21.15.70:0/142705404 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bbffff640 1 --2- 172.21.15.70:0/142705404 >> v2:172.21.15.181:3300/0 conn(0x7f7bc015cc70 0x7f7bc00cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bb7fff640 1 -- 172.21.15.70:0/142705404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 msgr2=0x7f7bc00b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bc4a2d640 1 --2- 172.21.15.70:0/142705404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 0x7f7bc00b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bb7fff640 1 --2- 172.21.15.70:0/142705404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 0x7f7bc00b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bb7fff640 1 -- 172.21.15.70:0/142705404 >> v2:172.21.15.181:3300/0 conn(0x7f7bc015cc70 msgr2=0x7f7bc00cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:18.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bb7fff640 1 --2- 172.21.15.70:0/142705404 >> v2:172.21.15.181:3300/0 conn(0x7f7bc015cc70 0x7f7bc00cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bb7fff640 1 -- 172.21.15.70:0/142705404 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc00cd270 con 0x7f7bc00d31a0 2023-10-08T05:55:18.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bc4a2d640 1 --2- 172.21.15.70:0/142705404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 0x7f7bc00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:18.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.055+0000 7f7bbffff640 1 --2- 172.21.15.70:0/142705404 >> v2:172.21.15.181:3300/0 conn(0x7f7bc015cc70 0x7f7bc00cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:18.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.056+0000 7f7bb7fff640 1 --2- 172.21.15.70:0/142705404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc00d31a0 0x7f7bc00b5cb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00d0090 tx=0x7f7ba802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeeef4c8585922ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:18.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.056+0000 7f7bbf7fe640 1 -- 172.21.15.70:0/142705404 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80317e0 con 0x7f7bc00d31a0 2023-10-08T05:55:18.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.056+0000 7f7bbf7fe640 1 -- 172.21.15.70:0/142705404 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8031940 con 0x7f7bc00d31a0 2023-10-08T05:55:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.057+0000 7f7bbf7fe640 1 -- 172.21.15.70:0/142705404 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8031cd0 con 0x7f7bc00d31a0 2023-10-08T05:55:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.057+0000 7f7bc6490640 1 -- 172.21.15.70:0/142705404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc00d31a0 msgr2=0x7f7bc00b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.057+0000 7f7bc6490640 1 --2- 172.21.15.70:0/142705404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc00d31a0 0x7f7bc00b5cb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00d0090 tx=0x7f7ba802b880 comp rx=0 tx=0).stop 2023-10-08T05:55:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.057+0000 7f7bc6490640 1 -- 172.21.15.70:0/142705404 shutdown_connections 2023-10-08T05:55:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.057+0000 7f7bc6490640 1 --2- 172.21.15.70:0/142705404 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 0x7f7bc00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.057+0000 7f7bc6490640 1 --2- 172.21.15.70:0/142705404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc00d31a0 0x7f7bc00b5cb0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.057+0000 7f7bc6490640 1 --2- 172.21.15.70:0/142705404 >> v2:172.21.15.181:3300/0 conn(0x7f7bc015cc70 0x7f7bc00cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.057+0000 7f7bc6490640 1 -- 172.21.15.70:0/142705404 >> 172.21.15.70:0/142705404 conn(0x7f7bc00718a0 msgr2=0x7f7bc0071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.057+0000 7f7bc6490640 1 -- 172.21.15.70:0/142705404 shutdown_connections 2023-10-08T05:55:18.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.057+0000 7f7bc6490640 1 -- 172.21.15.70:0/142705404 wait complete. 2023-10-08T05:55:18.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.059+0000 7f7bc6490640 1 Processor -- start 2023-10-08T05:55:18.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.059+0000 7f7bc6490640 1 -- start start 2023-10-08T05:55:18.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.060+0000 7f7bc6490640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 0x7f7bc015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.060+0000 7f7bc6490640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bc00d31a0 0x7f7bc00d1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.060+0000 7f7bc6490640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc015cc70 0x7f7bc00d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.060+0000 7f7bc6490640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc015f090 con 0x7f7bc015cc70 2023-10-08T05:55:18.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.060+0000 7f7bc6490640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc015e410 con 0x7f7bc00d31a0 2023-10-08T05:55:18.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.060+0000 7f7bc6490640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc015e6f0 con 0x7f7bc00b61f0 2023-10-08T05:55:18.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.060+0000 7f7bc4a2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc015cc70 0x7f7bc00d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:18.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.060+0000 7f7bc4a2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc015cc70 0x7f7bc00d1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42726/0 (socket says 172.21.15.70:42726) 2023-10-08T05:55:18.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.060+0000 7f7bc4a2d640 1 -- 172.21.15.70:0/3275331653 learned_addr learned my addr 172.21.15.70:0/3275331653 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:18.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bbffff640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 0x7f7bc015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:18.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bc4a2d640 1 -- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 msgr2=0x7f7bc015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:18.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bb7fff640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bc00d31a0 0x7f7bc00d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:18.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bc4a2d640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 0x7f7bc015ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bc4a2d640 1 -- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bc00d31a0 msgr2=0x7f7bc00d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:18.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bc4a2d640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bc00d31a0 0x7f7bc00d1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bc4a2d640 1 -- 172.21.15.70:0/3275331653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba80094d0 con 0x7f7bc015cc70 2023-10-08T05:55:18.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bc4a2d640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc015cc70 0x7f7bc00d1db0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800cef0 tx=0x7f7bb800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:18.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bbd7fa640 1 -- 172.21.15.70:0/3275331653 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800d840 con 0x7f7bc015cc70 2023-10-08T05:55:18.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bbd7fa640 1 -- 172.21.15.70:0/3275331653 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb800d9c0 con 0x7f7bc015cc70 2023-10-08T05:55:18.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bbd7fa640 1 -- 172.21.15.70:0/3275331653 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800dd70 con 0x7f7bc015cc70 2023-10-08T05:55:18.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bc6490640 1 -- 172.21.15.70:0/3275331653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc015e9d0 con 0x7f7bc015cc70 2023-10-08T05:55:18.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.061+0000 7f7bc6490640 1 -- 172.21.15.70:0/3275331653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc015b320 con 0x7f7bc015cc70 2023-10-08T05:55:18.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.062+0000 7f7bc6490640 1 -- 172.21.15.70:0/3275331653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc015f1d0 con 0x7f7bc015cc70 2023-10-08T05:55:18.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.063+0000 7f7bbd7fa640 1 -- 172.21.15.70:0/3275331653 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7bb801d360 con 0x7f7bc015cc70 2023-10-08T05:55:18.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.064+0000 7f7bbd7fa640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f7ba006af90 0x7f7ba006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.064+0000 7f7bbd7fa640 1 -- 172.21.15.70:0/3275331653 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f7bb808a3f0 con 0x7f7bc015cc70 2023-10-08T05:55:18.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.065+0000 7f7bbffff640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f7ba006af90 0x7f7ba006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:18.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.066+0000 7f7bbffff640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f7ba006af90 0x7f7ba006d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00d25b0 tx=0x7f7bb0008040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:18.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.069+0000 7f7bbd7fa640 1 -- 172.21.15.70:0/3275331653 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7bc015f1d0 con 0x7f7bc015cc70 2023-10-08T05:55:18.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.260+0000 7f7bc6490640 1 -- 172.21.15.70:0/3275331653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7bc015f1d0 con 0x7f7bc015cc70 2023-10-08T05:55:18.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.265+0000 7f7bbd7fa640 1 -- 172.21.15.70:0/3275331653 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7bc015f1d0 con 0x7f7bc015cc70 2023-10-08T05:55:18.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 -- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f7ba006af90 msgr2=0x7f7ba006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:18.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f7ba006af90 0x7f7ba006d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00d25b0 tx=0x7f7bb0008040 comp rx=0 tx=0).stop 2023-10-08T05:55:18.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 -- 172.21.15.70:0/3275331653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc015cc70 msgr2=0x7f7bc00d1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:18.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc015cc70 0x7f7bc00d1db0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800cef0 tx=0x7f7bb800a710 comp rx=0 tx=0).stop 2023-10-08T05:55:18.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 -- 172.21.15.70:0/3275331653 shutdown_connections 2023-10-08T05:55:18.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f7ba006af90 0x7f7ba006d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc015cc70 0x7f7bc00d1db0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bc00d31a0 0x7f7bc00d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 --2- 172.21.15.70:0/3275331653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bc00b61f0 0x7f7bc015ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 -- 172.21.15.70:0/3275331653 >> 172.21.15.70:0/3275331653 conn(0x7f7bc00718a0 msgr2=0x7f7bc00b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:18.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 -- 172.21.15.70:0/3275331653 shutdown_connections 2023-10-08T05:55:18.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.268+0000 7f7bc6490640 1 -- 172.21.15.70:0/3275331653 wait complete. 2023-10-08T05:55:18.289 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:55:18.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:55:18.398 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:55:18.521 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:55:18.522 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:55:18.522 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:55:18.582 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:55:18.582 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:55:18.582 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:55:18.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:55:18.642 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:55:18.642 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:55:18.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:55:18.703 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:55:18.703 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:55:18.763 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:55:18.763 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T05:55:18.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:55:18.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.988+0000 7fb05c5a4640 1 Processor -- start 2023-10-08T05:55:18.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.988+0000 7fb05c5a4640 1 -- start start 2023-10-08T05:55:18.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.990+0000 7fb05c5a4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb05415cd90 0x7fb0540702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.990+0000 7fb05c5a4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb054160940 0x7fb054070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.990+0000 7fb05c5a4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0540c48f0 con 0x7fb054070d50 2023-10-08T05:55:18.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.990+0000 7fb05c5a4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0540c4d80 con 0x7fb054160940 2023-10-08T05:55:18.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.990+0000 7fb05c5a4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0540c3ce0 con 0x7fb05415cd90 2023-10-08T05:55:18.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.990+0000 7fb059b3f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb054160940 0x7fb054070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:18.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.990+0000 7fb059b3f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb054160940 0x7fb054070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42740/0 (socket says 172.21.15.70:42740) 2023-10-08T05:55:18.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.990+0000 7fb059b3f640 1 -- 172.21.15.70:0/1599177477 learned_addr learned my addr 172.21.15.70:0/1599177477 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:18.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.991+0000 7fb05a340640 1 --2- 172.21.15.70:0/1599177477 >> v2:172.21.15.181:3300/0 conn(0x7fb05415cd90 0x7fb0540702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:18.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.991+0000 7fb059b3f640 1 -- 172.21.15.70:0/1599177477 >> v2:172.21.15.181:3300/0 conn(0x7fb05415cd90 msgr2=0x7fb0540702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:18.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.991+0000 7fb059b3f640 1 --2- 172.21.15.70:0/1599177477 >> v2:172.21.15.181:3300/0 conn(0x7fb05415cd90 0x7fb0540702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.991+0000 7fb059b3f640 1 -- 172.21.15.70:0/1599177477 >> v1:172.21.15.181:6789/0 conn(0x7fb054070d50 legacy=0x7fb0540cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.991+0000 7fb059b3f640 1 -- 172.21.15.70:0/1599177477 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540c3fc0 con 0x7fb054160940 2023-10-08T05:55:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.991+0000 7fb05a340640 1 --2- 172.21.15.70:0/1599177477 >> v2:172.21.15.181:3300/0 conn(0x7fb05415cd90 0x7fb0540702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.992+0000 7fb059b3f640 1 --2- 172.21.15.70:0/1599177477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054160940 0x7fb054070810 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fb05406e440 tx=0x7fb04402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee7a5b7b8c3d303 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.992+0000 7fb05933e640 1 -- 172.21.15.70:0/1599177477 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044031870 con 0x7fb054160940 2023-10-08T05:55:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.992+0000 7fb05933e640 1 -- 172.21.15.70:0/1599177477 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0440319d0 con 0x7fb054160940 2023-10-08T05:55:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.992+0000 7fb05933e640 1 -- 172.21.15.70:0/1599177477 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb044031d60 con 0x7fb054160940 2023-10-08T05:55:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.993+0000 7fb05c5a4640 1 -- 172.21.15.70:0/1599177477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054160940 msgr2=0x7fb054070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.993+0000 7fb05c5a4640 1 --2- 172.21.15.70:0/1599177477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054160940 0x7fb054070810 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fb05406e440 tx=0x7fb04402b910 comp rx=0 tx=0).stop 2023-10-08T05:55:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.993+0000 7fb05c5a4640 1 -- 172.21.15.70:0/1599177477 shutdown_connections 2023-10-08T05:55:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.993+0000 7fb05c5a4640 1 --2- 172.21.15.70:0/1599177477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054160940 0x7fb054070810 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.993+0000 7fb05c5a4640 1 --2- 172.21.15.70:0/1599177477 >> v2:172.21.15.181:3300/0 conn(0x7fb05415cd90 0x7fb0540702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.993+0000 7fb05c5a4640 1 -- 172.21.15.70:0/1599177477 >> 172.21.15.70:0/1599177477 conn(0x7fb05406a380 msgr2=0x7fb0540b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.993+0000 7fb05c5a4640 1 -- 172.21.15.70:0/1599177477 shutdown_connections 2023-10-08T05:55:18.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.993+0000 7fb05c5a4640 1 -- 172.21.15.70:0/1599177477 wait complete. 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.995+0000 7fb05c5a4640 1 Processor -- start 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.995+0000 7fb05c5a4640 1 -- start start 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.996+0000 7fb05c5a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054070d50 0x7fb0540b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.996+0000 7fb05c5a4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05415cd90 0x7fb054154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.996+0000 7fb05c5a4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb054160940 0x7fb0540cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.996+0000 7fb05c5a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b31f0 con 0x7fb054070d50 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.996+0000 7fb05c5a4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b1970 con 0x7fb05415cd90 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.996+0000 7fb05c5a4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0540b1c50 con 0x7fb054160940 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.996+0000 7fb05a340640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054070d50 0x7fb0540b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.996+0000 7fb05a340640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054070d50 0x7fb0540b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42756/0 (socket says 172.21.15.70:42756) 2023-10-08T05:55:18.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.996+0000 7fb05a340640 1 -- 172.21.15.70:0/2527912155 learned_addr learned my addr 172.21.15.70:0/2527912155 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.996+0000 7fb059b3f640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05415cd90 0x7fb054154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb05ab41640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb054160940 0x7fb0540cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb05a340640 1 -- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb054160940 msgr2=0x7fb0540cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb05a340640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb054160940 0x7fb0540cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb05a340640 1 -- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05415cd90 msgr2=0x7fb054154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb05a340640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05415cd90 0x7fb054154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb05a340640 1 -- 172.21.15.70:0/2527912155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0440094d0 con 0x7fb054070d50 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb05ab41640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb054160940 0x7fb0540cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb05a340640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054070d50 0x7fb0540b2cb0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c7f0 tx=0x7fb05000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb04affd640 1 -- 172.21.15.70:0/2527912155 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050007a30 con 0x7fb054070d50 2023-10-08T05:55:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb04affd640 1 -- 172.21.15.70:0/2527912155 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb050007b90 con 0x7fb054070d50 2023-10-08T05:55:19.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb04affd640 1 -- 172.21.15.70:0/2527912155 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05000ce40 con 0x7fb054070d50 2023-10-08T05:55:19.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.997+0000 7fb05c5a4640 1 -- 172.21.15.70:0/2527912155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0540b1f30 con 0x7fb054070d50 2023-10-08T05:55:19.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.998+0000 7fb05c5a4640 1 -- 172.21.15.70:0/2527912155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0541548b0 con 0x7fb054070d50 2023-10-08T05:55:19.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:18.999+0000 7fb05c5a4640 1 -- 172.21.15.70:0/2527912155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb054067490 con 0x7fb054070d50 2023-10-08T05:55:19.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.000+0000 7fb04affd640 1 -- 172.21.15.70:0/2527912155 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb0500113f0 con 0x7fb054070d50 2023-10-08T05:55:19.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.001+0000 7fb04affd640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb03806afc0 0x7fb03806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:19.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.001+0000 7fb04affd640 1 -- 172.21.15.70:0/2527912155 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fb050089c00 con 0x7fb054070d50 2023-10-08T05:55:19.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.001+0000 7fb059b3f640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb03806afc0 0x7fb03806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:19.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.002+0000 7fb059b3f640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb03806afc0 0x7fb03806d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb044036000 tx=0x7fb04403c040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:19.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.005+0000 7fb04affd640 1 -- 172.21.15.70:0/2527912155 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb050054900 con 0x7fb054070d50 2023-10-08T05:55:19.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.231+0000 7fb05c5a4640 1 -- 172.21.15.70:0/2527912155 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0540aee60 con 0x7fb054070d50 2023-10-08T05:55:19.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.232+0000 7fb04affd640 1 -- 172.21.15.70:0/2527912155 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fb05001f090 con 0x7fb054070d50 2023-10-08T05:55:19.234 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:19.234 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":52,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/290122525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":290122525},{"type":"v1","addr":"172.21.15.70:6835","nonce":290122525}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5624,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4017950051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4017950051},{"type":"v1","addr":"172.21.15.181:6837","nonce":4017950051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5843,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/3423368130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3423368130},{"type":"v1","addr":"172.21.15.181:6835","nonce":3423368130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6172,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1329101557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1329101557},{"type":"v1","addr":"172.21.15.70:6834","nonce":1329101557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[]} 2023-10-08T05:55:19.234 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 52 2023-10-08T05:55:19.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 -- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb03806afc0 msgr2=0x7fb03806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:19.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb03806afc0 0x7fb03806d480 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb044036000 tx=0x7fb04403c040 comp rx=0 tx=0).stop 2023-10-08T05:55:19.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 -- 172.21.15.70:0/2527912155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054070d50 msgr2=0x7fb0540b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:19.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054070d50 0x7fb0540b2cb0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c7f0 tx=0x7fb05000cbf0 comp rx=0 tx=0).stop 2023-10-08T05:55:19.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 -- 172.21.15.70:0/2527912155 shutdown_connections 2023-10-08T05:55:19.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb054160940 0x7fb0540cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:19.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05415cd90 0x7fb054154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:19.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb03806afc0 0x7fb03806d480 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:19.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 --2- 172.21.15.70:0/2527912155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb054070d50 0x7fb0540b2cb0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:19.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 -- 172.21.15.70:0/2527912155 >> 172.21.15.70:0/2527912155 conn(0x7fb05406a380 msgr2=0x7fb0540ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:19.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 -- 172.21.15.70:0/2527912155 shutdown_connections 2023-10-08T05:55:19.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.235+0000 7fb05c5a4640 1 -- 172.21.15.70:0/2527912155 wait complete. 2023-10-08T05:55:19.255 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T05:55:19.255 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T05:55:19.256 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T05:55:19.256 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:55:19.256 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T05:55:19.256 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T05:55:19.256 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T05:55:19.256 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:55:19.257 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T05:55:19.257 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T05:55:19.257 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T05:55:19.257 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:55:19.257 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T05:55:19.257 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T05:55:19.258 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T05:55:19.258 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:55:19.274 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T05:55:19.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T05:55:19.277 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T05:55:19.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T05:55:19.280 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T05:55:19.280 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T05:55:19.285 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T05:55:19.286 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T05:55:19.325 INFO:tasks.ceph.mds.c:Started 2023-10-08T05:55:19.325 DEBUG:teuthology.parallel:result is None 2023-10-08T05:55:19.333 INFO:tasks.ceph.mds.a:Started 2023-10-08T05:55:19.333 DEBUG:teuthology.parallel:result is None 2023-10-08T05:55:19.334 INFO:tasks.ceph.mds.d:Started 2023-10-08T05:55:19.335 DEBUG:teuthology.parallel:result is None 2023-10-08T05:55:19.342 INFO:tasks.ceph.mds.b:Started 2023-10-08T05:55:19.342 DEBUG:teuthology.parallel:result is None 2023-10-08T05:55:19.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T05:55:19.484 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:55:19.488 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T05:55:19.498 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T05:55:19.498 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:55:19.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.533+0000 7f54269b0640 1 Processor -- start 2023-10-08T05:55:19.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.533+0000 7f54269b0640 1 -- start start 2023-10-08T05:55:19.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.534+0000 7f54269b0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f542006db60 0x7f54200b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:19.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.534+0000 7f54269b0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542015b310 con 0x7f54200c9cf0 2023-10-08T05:55:19.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.534+0000 7f54269b0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542006d290 con 0x7f542006c3a0 2023-10-08T05:55:19.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.534+0000 7f54269b0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542015b110 con 0x7f542006db60 2023-10-08T05:55:19.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.535+0000 7f54177fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f54200c9cf0 0x7f542015aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41058/0 (socket says 172.21.15.70:41058) 2023-10-08T05:55:19.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.535+0000 7f54177fe640 1 -- 172.21.15.70:0/3687676678 learned_addr learned my addr 172.21.15.70:0/3687676678 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:19.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.535+0000 7f5424f4d640 1 --2- 172.21.15.70:0/3687676678 >> v2:172.21.15.181:3300/0 conn(0x7f542006db60 0x7f54200b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:19.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.535+0000 7f541f7fe640 1 -- 172.21.15.70:0/3687676678 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300343269 0 0) 0x7f5408001110 con 0x7f54200c9cf0 2023-10-08T05:55:19.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.535+0000 7f541f7fe640 1 -- 172.21.15.70:0/3687676678 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f540c003520 con 0x7f54200c9cf0 2023-10-08T05:55:19.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.535+0000 7f5424f4d640 1 -- 172.21.15.70:0/3687676678 >> v1:172.21.15.181:6789/0 conn(0x7f542006c3a0 legacy=0x7f542006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:19.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.535+0000 7f541f7fe640 1 -- 172.21.15.70:0/3687676678 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222799797 0 0) 0x7f542006d290 con 0x7f542006c3a0 2023-10-08T05:55:19.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.535+0000 7f5424f4d640 1 -- 172.21.15.70:0/3687676678 >> v1:172.21.15.70:6789/0 conn(0x7f54200c9cf0 legacy=0x7f542015aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:19.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.535+0000 7f5424f4d640 1 -- 172.21.15.70:0/3687676678 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200c9860 con 0x7f542006db60 2023-10-08T05:55:19.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.537+0000 7f5424f4d640 1 --2- 172.21.15.70:0/3687676678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f542006db60 0x7f54200b2a90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5418009d90 tx=0x7f541802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47845f8169242fe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:19.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.537+0000 7f541f7fe640 1 -- 172.21.15.70:0/3687676678 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541802cd00 con 0x7f542006db60 2023-10-08T05:55:19.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.537+0000 7f541f7fe640 1 -- 172.21.15.70:0/3687676678 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541802ce60 con 0x7f542006db60 2023-10-08T05:55:19.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.537+0000 7f541f7fe640 1 -- 172.21.15.70:0/3687676678 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5418034640 con 0x7f542006db60 2023-10-08T05:55:19.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.538+0000 7f54269b0640 1 -- 172.21.15.70:0/3687676678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f542006db60 msgr2=0x7f54200b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:19.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.538+0000 7f54269b0640 1 --2- 172.21.15.70:0/3687676678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f542006db60 0x7f54200b2a90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5418009d90 tx=0x7f541802bf50 comp rx=0 tx=0).stop 2023-10-08T05:55:19.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.538+0000 7f54269b0640 1 -- 172.21.15.70:0/3687676678 shutdown_connections 2023-10-08T05:55:19.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.538+0000 7f54269b0640 1 --2- 172.21.15.70:0/3687676678 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f542006db60 0x7f54200b2a90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:19.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.538+0000 7f54269b0640 1 -- 172.21.15.70:0/3687676678 >> 172.21.15.70:0/3687676678 conn(0x7f542006a5d0 msgr2=0x7f54200b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:19.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.538+0000 7f54269b0640 1 -- 172.21.15.70:0/3687676678 shutdown_connections 2023-10-08T05:55:19.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.538+0000 7f54269b0640 1 -- 172.21.15.70:0/3687676678 wait complete. 2023-10-08T05:55:19.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.539+0000 7f54269b0640 1 Processor -- start 2023-10-08T05:55:19.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.540+0000 7f54269b0640 1 -- start start 2023-10-08T05:55:19.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54269b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f542006c3a0 0x7f54200c0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:19.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54269b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f542006db60 0x7f54201e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:19.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54269b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200c9cf0 0x7f54201e4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:19.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54269b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420157940 con 0x7f542006db60 2023-10-08T05:55:19.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54269b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5420157c20 con 0x7f54200c9cf0 2023-10-08T05:55:19.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54269b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5420157f00 con 0x7f542006c3a0 2023-10-08T05:55:19.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f542006db60 0x7f54201e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:19.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f542006db60 0x7f54201e48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42798/0 (socket says 172.21.15.70:42798) 2023-10-08T05:55:19.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54177fe640 1 -- 172.21.15.70:0/2404411293 learned_addr learned my addr 172.21.15.70:0/2404411293 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:19.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f541ffff640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f542006c3a0 0x7f54200c0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:19.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f5424f4d640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200c9cf0 0x7f54201e4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:19.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54177fe640 1 -- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f542006c3a0 msgr2=0x7f54200c0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:19.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54177fe640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f542006c3a0 0x7f54200c0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:19.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54177fe640 1 -- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200c9cf0 msgr2=0x7f54201e4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:19.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54177fe640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200c9cf0 0x7f54201e4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:19.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f54177fe640 1 -- 172.21.15.70:0/2404411293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5418009a70 con 0x7f542006db60 2023-10-08T05:55:19.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.541+0000 7f5424f4d640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200c9cf0 0x7f54201e4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:55:19.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.542+0000 7f54177fe640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f542006db60 0x7f54201e48c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f540800c390 tx=0x7f540800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:19.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.542+0000 7f541d7fa640 1 -- 172.21.15.70:0/2404411293 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408005ce0 con 0x7f542006db60 2023-10-08T05:55:19.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.542+0000 7f541d7fa640 1 -- 172.21.15.70:0/2404411293 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5408005e60 con 0x7f542006db60 2023-10-08T05:55:19.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.542+0000 7f541d7fa640 1 -- 172.21.15.70:0/2404411293 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408007310 con 0x7f542006db60 2023-10-08T05:55:19.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.542+0000 7f54269b0640 1 -- 172.21.15.70:0/2404411293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54201581e0 con 0x7f542006db60 2023-10-08T05:55:19.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.542+0000 7f54269b0640 1 -- 172.21.15.70:0/2404411293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54200c11f0 con 0x7f542006db60 2023-10-08T05:55:19.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.543+0000 7f54269b0640 1 -- 172.21.15.70:0/2404411293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54200c5af0 con 0x7f542006db60 2023-10-08T05:55:19.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.544+0000 7f541d7fa640 1 -- 172.21.15.70:0/2404411293 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5408007470 con 0x7f542006db60 2023-10-08T05:55:19.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.545+0000 7f541d7fa640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f540006af90 0x7f540006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:19.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.545+0000 7f541d7fa640 1 -- 172.21.15.70:0/2404411293 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f5408088d40 con 0x7f542006db60 2023-10-08T05:55:19.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.546+0000 7f541ffff640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f540006af90 0x7f540006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:19.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.547+0000 7f541ffff640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f540006af90 0x7f540006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5410000f60 tx=0x7f5410009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:19.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.550+0000 7f541d7fa640 1 -- 172.21.15.70:0/2404411293 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54200c5af0 con 0x7f542006db60 2023-10-08T05:55:19.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:19.750+0000 7f54269b0640 1 -- 172.21.15.70:0/2404411293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f54200c5af0 con 0x7f542006db60 2023-10-08T05:55:20.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.851+0000 7f541d7fa640 1 -- 172.21.15.70:0/2404411293 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v130) v1 ==== 97+0+0 (secure 0 0 0) 0x7f54200c5af0 con 0x7f542006db60 2023-10-08T05:55:20.853 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T05:55:20.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 -- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f540006af90 msgr2=0x7f540006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:20.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f540006af90 0x7f540006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5410000f60 tx=0x7f5410009040 comp rx=0 tx=0).stop 2023-10-08T05:55:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 -- 172.21.15.70:0/2404411293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f542006db60 msgr2=0x7f54201e48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f542006db60 0x7f54201e48c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f540800c390 tx=0x7f540800c850 comp rx=0 tx=0).stop 2023-10-08T05:55:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 -- 172.21.15.70:0/2404411293 shutdown_connections 2023-10-08T05:55:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f54200c9cf0 0x7f54201e4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:20.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f540006af90 0x7f540006d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:20.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f542006db60 0x7f54201e48c0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:20.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 --2- 172.21.15.70:0/2404411293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f542006c3a0 0x7f54200c0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:20.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 -- 172.21.15.70:0/2404411293 >> 172.21.15.70:0/2404411293 conn(0x7f542006a5d0 msgr2=0x7f54200af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:20.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.855+0000 7f54269b0640 1 -- 172.21.15.70:0/2404411293 shutdown_connections 2023-10-08T05:55:20.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:20.856+0000 7f54269b0640 1 -- 172.21.15.70:0/2404411293 wait complete. 2023-10-08T05:55:20.877 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T05:55:21.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.090+0000 7f02ff41a640 1 Processor -- start 2023-10-08T05:55:21.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.090+0000 7f02ff41a640 1 -- start start 2023-10-08T05:55:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.092+0000 7f02ff41a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f8160940 0x7f02f80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.092+0000 7f02ff41a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f80c48f0 con 0x7f02f806c890 2023-10-08T05:55:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.092+0000 7f02ff41a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f80c3ce0 con 0x7f02f815cd90 2023-10-08T05:55:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.092+0000 7f02ff41a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80c4d80 con 0x7f02f8160940 2023-10-08T05:55:21.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.092+0000 7f02fd9b7640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f02f806c890 0x7f02f806cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41068/0 (socket says 172.21.15.70:41068) 2023-10-08T05:55:21.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.092+0000 7f02fd9b7640 1 -- 172.21.15.70:0/3588766801 learned_addr learned my addr 172.21.15.70:0/3588766801 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:21.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02fc9b5640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f8160940 0x7f02f80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592973064 0 0) 0x7f02f80c48f0 con 0x7f02f806c890 2023-10-08T05:55:21.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02e4003520 con 0x7f02f806c890 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4216417610 0 0) 0x7f02e4003520 con 0x7f02f806c890 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f8160940 msgr2=0x7f02f80709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f8160940 0x7f02f80709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 >> v1:172.21.15.181:6789/0 conn(0x7f02f815cd90 legacy=0x7f02f80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f80c3fc0 con 0x7f02f806c890 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f02f00019f0 con 0x7f02f806c890 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 >> v1:172.21.15.70:6789/0 conn(0x7f02f806c890 legacy=0x7f02f806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e40039a0 0x7f02e4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f8160940 0x7f02e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4004d00 0x7f02e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e400ba30 con 0x7f02e4004d00 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e400bb70 con 0x7f02e40039a0 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.093+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e400bd30 con 0x7f02f8160940 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.094+0000 7f02fd9b7640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4004d00 0x7f02e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.094+0000 7f02fd1b6640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e40039a0 0x7f02e4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.094+0000 7f02fc9b5640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f8160940 0x7f02e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.094+0000 7f02fd9b7640 1 -- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f8160940 msgr2=0x7f02e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.094+0000 7f02fd9b7640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f8160940 0x7f02e40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.094+0000 7f02fd9b7640 1 -- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e40039a0 msgr2=0x7f02e4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.094+0000 7f02fd9b7640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e40039a0 0x7f02e4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.094+0000 7f02fd9b7640 1 -- 172.21.15.70:0/3588766801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e4003520 con 0x7f02e4004d00 2023-10-08T05:55:21.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.094+0000 7f02fd1b6640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e40039a0 0x7f02e4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:21.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.095+0000 7f02fd9b7640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4004d00 0x7f02e4005180 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f02f000a7f0 tx=0x7f02e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc80b3f05963403 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:21.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.095+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00072d0 con 0x7f02e4004d00 2023-10-08T05:55:21.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.095+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f002f070 con 0x7f02e4004d00 2023-10-08T05:55:21.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.095+0000 7f02effff640 1 -- 172.21.15.70:0/3588766801 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0007690 con 0x7f02e4004d00 2023-10-08T05:55:21.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.096+0000 7f02ff41a640 1 -- 172.21.15.70:0/3588766801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4004d00 msgr2=0x7f02e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.096+0000 7f02ff41a640 1 --2- 172.21.15.70:0/3588766801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e4004d00 0x7f02e4005180 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f02f000a7f0 tx=0x7f02e40057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.096+0000 7f02fd9b7640 1 -- 172.21.15.70:0/3588766801 reap_dead start 2023-10-08T05:55:21.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.096+0000 7f02ff41a640 1 -- 172.21.15.70:0/3588766801 shutdown_connections 2023-10-08T05:55:21.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.096+0000 7f02ff41a640 1 -- 172.21.15.70:0/3588766801 >> 172.21.15.70:0/3588766801 conn(0x7f02f806a380 msgr2=0x7f02f80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:21.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.096+0000 7f02ff41a640 1 -- 172.21.15.70:0/3588766801 shutdown_connections 2023-10-08T05:55:21.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.096+0000 7f02ff41a640 1 -- 172.21.15.70:0/3588766801 wait complete. 2023-10-08T05:55:21.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.098+0000 7f02ff41a640 1 Processor -- start 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.098+0000 7f02ff41a640 1 -- start start 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.099+0000 7f02ff41a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e4004d00 0x7f02f80b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.099+0000 7f02ff41a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f815d350 0x7f02f815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.099+0000 7f02ff41a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02f80acb80 0x7f02f814e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.099+0000 7f02ff41a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f815d7d0 con 0x7f02f80acb80 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.099+0000 7f02ff41a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f815da60 con 0x7f02e4004d00 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.099+0000 7f02ff41a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f815dd40 con 0x7f02f815d350 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.099+0000 7f02fd9b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02f80acb80 0x7f02f814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.099+0000 7f02fd9b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02f80acb80 0x7f02f814e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42820/0 (socket says 172.21.15.70:42820) 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.099+0000 7f02fd9b7640 1 -- 172.21.15.70:0/4048392856 learned_addr learned my addr 172.21.15.70:0/4048392856 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:21.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.100+0000 7f02fd1b6640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e4004d00 0x7f02f80b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.100+0000 7f02fc9b5640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f815d350 0x7f02f815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.100+0000 7f02fd9b7640 1 -- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f815d350 msgr2=0x7f02f815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.100+0000 7f02fd9b7640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f815d350 0x7f02f815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.100+0000 7f02fd9b7640 1 -- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e4004d00 msgr2=0x7f02f80b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.100+0000 7f02fd9b7640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e4004d00 0x7f02f80b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.100+0000 7f02fd9b7640 1 -- 172.21.15.70:0/4048392856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f815e020 con 0x7f02f80acb80 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.100+0000 7f02fc9b5640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f815d350 0x7f02f815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.100+0000 7f02fd1b6640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e4004d00 0x7f02f80b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.100+0000 7f02fd9b7640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02f80acb80 0x7f02f814e230 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f02f000a7f0 tx=0x7f02f80c4860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.101+0000 7f02edffb640 1 -- 172.21.15.70:0/4048392856 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00019b0 con 0x7f02f80acb80 2023-10-08T05:55:21.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.101+0000 7f02edffb640 1 -- 172.21.15.70:0/4048392856 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f0004e90 con 0x7f02f80acb80 2023-10-08T05:55:21.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.101+0000 7f02edffb640 1 -- 172.21.15.70:0/4048392856 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00072f0 con 0x7f02f80acb80 2023-10-08T05:55:21.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.101+0000 7f02ff41a640 1 -- 172.21.15.70:0/4048392856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f815e300 con 0x7f02f80acb80 2023-10-08T05:55:21.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.101+0000 7f02ff41a640 1 -- 172.21.15.70:0/4048392856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f815f040 con 0x7f02f80acb80 2023-10-08T05:55:21.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.102+0000 7f02ff41a640 1 -- 172.21.15.70:0/4048392856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f815f040 con 0x7f02f80acb80 2023-10-08T05:55:21.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.103+0000 7f02edffb640 1 -- 172.21.15.70:0/4048392856 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f02f0031030 con 0x7f02f80acb80 2023-10-08T05:55:21.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.104+0000 7f02edffb640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f02dc06af90 0x7f02dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.104+0000 7f02edffb640 1 -- 172.21.15.70:0/4048392856 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f02f00ab220 con 0x7f02f80acb80 2023-10-08T05:55:21.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.105+0000 7f02fd1b6640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f02dc06af90 0x7f02dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.106+0000 7f02fd1b6640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f02dc06af90 0x7f02dc06d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f02f40016f0 tx=0x7f02f4009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:21.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.108+0000 7f02edffb640 1 -- 172.21.15.70:0/4048392856 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02f00763a0 con 0x7f02f80acb80 2023-10-08T05:55:21.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.368+0000 7f02ff41a640 1 -- 172.21.15.70:0/4048392856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f02f815f040 con 0x7f02f80acb80 2023-10-08T05:55:21.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.369+0000 7f02edffb640 1 -- 172.21.15.70:0/4048392856 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f02f00799c0 con 0x7f02f80acb80 2023-10-08T05:55:21.371 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:21.371 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:55:21.371 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T05:55:21.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.371 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T05:55:21.371 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T05:55:21.371 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.371 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:55:21.371 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:55:21.371 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:55:21.372 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.372 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.372 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T05:55:21.372 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T05:55:21.372 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.372 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:55:21.372 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:55:21.373 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:55:21.373 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.373 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.373 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T05:55:21.373 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T05:55:21.374 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.374 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:55:21.374 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:55:21.374 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:55:21.374 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.375 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T05:55:21.375 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T05:55:21.376 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.376 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:55:21.376 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:55:21.376 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:55:21.376 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.376 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.377 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T05:55:21.377 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T05:55:21.377 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.377 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:55:21.377 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.378 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T05:55:21.380 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T05:55:21.381 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.381 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:55:21.381 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:55:21.381 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.381 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.381 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T05:55:21.381 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T05:55:21.381 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:55:21.382 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:55:21.383 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.383 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.383 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T05:55:21.383 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T05:55:21.383 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.383 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:55:21.383 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:55:21.384 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.384 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T05:55:21.384 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T05:55:21.384 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.385 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:55:21.385 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T05:55:21.385 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T05:55:21.385 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.386 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T05:55:21.386 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T05:55:21.386 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.386 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:55:21.386 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:55:21.386 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:55:21.387 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:55:21.387 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.387 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T05:55:21.387 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T05:55:21.387 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.388 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:55:21.388 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:55:21.388 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:55:21.388 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:55:21.388 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.389 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T05:55:21.389 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T05:55:21.389 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.389 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:55:21.389 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:55:21.390 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:55:21.390 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:55:21.390 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.390 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T05:55:21.391 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T05:55:21.391 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.391 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:55:21.391 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:55:21.391 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:55:21.391 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.391 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.392 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T05:55:21.392 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T05:55:21.392 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.392 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:55:21.393 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:55:21.393 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:55:21.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:21.393 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T05:55:21.393 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T05:55:21.393 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:55:21.394 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:55:21.394 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:55:21.394 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:55:21.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:21.394 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T05:55:21.394 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:55:21.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 -- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f02dc06af90 msgr2=0x7f02dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f02dc06af90 0x7f02dc06d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f02f40016f0 tx=0x7f02f4009040 comp rx=0 tx=0).stop 2023-10-08T05:55:21.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 -- 172.21.15.70:0/4048392856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02f80acb80 msgr2=0x7f02f814e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02f80acb80 0x7f02f814e230 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f02f000a7f0 tx=0x7f02f80c4860 comp rx=0 tx=0).stop 2023-10-08T05:55:21.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 -- 172.21.15.70:0/4048392856 shutdown_connections 2023-10-08T05:55:21.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f02dc06af90 0x7f02dc06d450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02f80acb80 0x7f02f814e230 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02f815d350 0x7f02f815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 --2- 172.21.15.70:0/4048392856 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02e4004d00 0x7f02f80b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 -- 172.21.15.70:0/4048392856 >> 172.21.15.70:0/4048392856 conn(0x7f02f806a380 msgr2=0x7f02f80ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:21.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 -- 172.21.15.70:0/4048392856 shutdown_connections 2023-10-08T05:55:21.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.372+0000 7f02ff41a640 1 -- 172.21.15.70:0/4048392856 wait complete. 2023-10-08T05:55:21.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:55:21.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.599+0000 7f0e85a1a640 1 Processor -- start 2023-10-08T05:55:21.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.599+0000 7f0e85a1a640 1 -- start start 2023-10-08T05:55:21.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e85a1a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e801599d0 0x7f0e800ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e85a1a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e800b07d0 0x7f0e8006c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e85a1a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e800cacf0 con 0x7f0e800cc210 2023-10-08T05:55:21.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e85a1a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e800b10f0 con 0x7f0e800b07d0 2023-10-08T05:55:21.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e85a1a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800ca3c0 con 0x7f0e801599d0 2023-10-08T05:55:21.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e85219640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e800b07d0 0x7f0e8006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e85219640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e800b07d0 0x7f0e8006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42834/0 (socket says 172.21.15.70:42834) 2023-10-08T05:55:21.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e85219640 1 -- 172.21.15.70:0/1043125633 learned_addr learned my addr 172.21.15.70:0/1043125633 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:21.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e84a18640 1 --2- 172.21.15.70:0/1043125633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e801599d0 0x7f0e800ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e85219640 1 -- 172.21.15.70:0/1043125633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e801599d0 msgr2=0x7f0e800ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:21.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.601+0000 7f0e85219640 1 --2- 172.21.15.70:0/1043125633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e801599d0 0x7f0e800ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.602+0000 7f0e85219640 1 -- 172.21.15.70:0/1043125633 >> v1:172.21.15.70:6789/0 conn(0x7f0e800cc210 legacy=0x7f0e800b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.602+0000 7f0e85219640 1 -- 172.21.15.70:0/1043125633 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800ca6a0 con 0x7f0e800b07d0 2023-10-08T05:55:21.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.603+0000 7f0e85219640 1 --2- 172.21.15.70:0/1043125633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e800b07d0 0x7f0e8006c310 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800d150 tx=0x7f0e7802e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b26c0a00dea872 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:21.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.603+0000 7f0e7f7fe640 1 -- 172.21.15.70:0/1043125633 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800b0d0 con 0x7f0e800b07d0 2023-10-08T05:55:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.603+0000 7f0e7f7fe640 1 -- 172.21.15.70:0/1043125633 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7800b230 con 0x7f0e800b07d0 2023-10-08T05:55:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.603+0000 7f0e7f7fe640 1 -- 172.21.15.70:0/1043125633 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78009600 con 0x7f0e800b07d0 2023-10-08T05:55:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.603+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1043125633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e800b07d0 msgr2=0x7f0e8006c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.603+0000 7f0e85a1a640 1 --2- 172.21.15.70:0/1043125633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e800b07d0 0x7f0e8006c310 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800d150 tx=0x7f0e7802e160 comp rx=0 tx=0).stop 2023-10-08T05:55:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.604+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1043125633 shutdown_connections 2023-10-08T05:55:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.604+0000 7f0e85a1a640 1 --2- 172.21.15.70:0/1043125633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e800b07d0 0x7f0e8006c310 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.604+0000 7f0e85a1a640 1 --2- 172.21.15.70:0/1043125633 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e801599d0 0x7f0e800ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.604+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1043125633 >> 172.21.15.70:0/1043125633 conn(0x7f0e800cb0c0 msgr2=0x7f0e800cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.604+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1043125633 shutdown_connections 2023-10-08T05:55:21.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.604+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1043125633 wait complete. 2023-10-08T05:55:21.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.605+0000 7f0e85a1a640 1 Processor -- start 2023-10-08T05:55:21.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.605+0000 7f0e85a1a640 1 -- start start 2023-10-08T05:55:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.606+0000 7f0e85a1a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e800b07d0 0x7f0e8015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.606+0000 7f0e85a1a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e800cc210 0x7f0e800cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.606+0000 7f0e85a1a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e801599d0 0x7f0e80143f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.606+0000 7f0e85a1a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800cf290 con 0x7f0e801599d0 2023-10-08T05:55:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.606+0000 7f0e85a1a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800cee00 con 0x7f0e800b07d0 2023-10-08T05:55:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.606+0000 7f0e85a1a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800ce1f0 con 0x7f0e800cc210 2023-10-08T05:55:21.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e85219640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e801599d0 0x7f0e80143f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e85219640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e801599d0 0x7f0e80143f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42840/0 (socket says 172.21.15.70:42840) 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e85219640 1 -- 172.21.15.70:0/1457675729 learned_addr learned my addr 172.21.15.70:0/1457675729 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e84a18640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e800b07d0 0x7f0e8015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e7ffff640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e800cc210 0x7f0e800cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e85219640 1 -- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e800cc210 msgr2=0x7f0e800cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e85219640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e800cc210 0x7f0e800cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e85219640 1 -- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e800b07d0 msgr2=0x7f0e8015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e85219640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e800b07d0 0x7f0e8015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e85219640 1 -- 172.21.15.70:0/1457675729 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7800bd40 con 0x7f0e801599d0 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e7ffff640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e800cc210 0x7f0e800cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:21.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e84a18640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e800b07d0 0x7f0e8015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:21.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.607+0000 7f0e85219640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e801599d0 0x7f0e80143f30 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800d150 tx=0x7f0e78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:21.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.608+0000 7f0e7d7fa640 1 -- 172.21.15.70:0/1457675729 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800b720 con 0x7f0e801599d0 2023-10-08T05:55:21.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.608+0000 7f0e7d7fa640 1 -- 172.21.15.70:0/1457675729 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e78040660 con 0x7f0e801599d0 2023-10-08T05:55:21.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.608+0000 7f0e7d7fa640 1 -- 172.21.15.70:0/1457675729 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e780409f0 con 0x7f0e801599d0 2023-10-08T05:55:21.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.608+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1457675729 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e800ce4d0 con 0x7f0e801599d0 2023-10-08T05:55:21.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.608+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1457675729 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e800cea90 con 0x7f0e801599d0 2023-10-08T05:55:21.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.609+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1457675729 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e800cef90 con 0x7f0e801599d0 2023-10-08T05:55:21.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.610+0000 7f0e7d7fa640 1 -- 172.21.15.70:0/1457675729 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0e78040b50 con 0x7f0e801599d0 2023-10-08T05:55:21.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.611+0000 7f0e7d7fa640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0e6406af90 0x7f0e6406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:21.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.612+0000 7f0e7d7fa640 1 -- 172.21.15.70:0/1457675729 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f0e780ae750 con 0x7f0e801599d0 2023-10-08T05:55:21.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.612+0000 7f0e84a18640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0e6406af90 0x7f0e6406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:21.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.613+0000 7f0e84a18640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0e6406af90 0x7f0e6406d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e74008890 tx=0x7f0e74008040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:21.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.616+0000 7f0e7d7fa640 1 -- 172.21.15.70:0/1457675729 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e78079a00 con 0x7f0e801599d0 2023-10-08T05:55:21.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.795+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1457675729 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e8004e160 con 0x7f0e801599d0 2023-10-08T05:55:21.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.796+0000 7f0e7d7fa640 1 -- 172.21.15.70:0/1457675729 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+13130 (secure 0 0 0) 0x7f0e7807d020 con 0x7f0e801599d0 2023-10-08T05:55:21.798 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:21.798 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":130,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:20.849747+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:55:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0e6406af90 msgr2=0x7f0e6406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0e6406af90 0x7f0e6406d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e74008890 tx=0x7f0e74008040 comp rx=0 tx=0).stop 2023-10-08T05:55:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1457675729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e801599d0 msgr2=0x7f0e80143f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e801599d0 0x7f0e80143f30 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800d150 tx=0x7f0e78004a40 comp rx=0 tx=0).stop 2023-10-08T05:55:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1457675729 shutdown_connections 2023-10-08T05:55:21.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0e6406af90 0x7f0e6406d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e801599d0 0x7f0e80143f30 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e800cc210 0x7f0e800cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 --2- 172.21.15.70:0/1457675729 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e800b07d0 0x7f0e8015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:21.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1457675729 >> 172.21.15.70:0/1457675729 conn(0x7f0e800cb0c0 msgr2=0x7f0e800cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:21.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1457675729 shutdown_connections 2023-10-08T05:55:21.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:21.799+0000 7f0e85a1a640 1 -- 172.21.15.70:0/1457675729 wait complete. 2023-10-08T05:55:21.818 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:55:21.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:55:22.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.033+0000 7f0ac62a3640 1 Processor -- start 2023-10-08T05:55:22.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.033+0000 7f0ac62a3640 1 -- start start 2023-10-08T05:55:22.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.034+0000 7f0ac62a3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0ac00cdc30 0x7f0ac00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.034+0000 7f0ac62a3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac015ac90 con 0x7f0ac00caef0 2023-10-08T05:55:22.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.034+0000 7f0ac62a3640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac015a6d0 con 0x7f0ac00cce90 2023-10-08T05:55:22.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.034+0000 7f0ac62a3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac015a9b0 con 0x7f0ac00cdc30 2023-10-08T05:55:22.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.035+0000 7f0ac52a1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0ac00caef0 0x7f0ac00cb2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41090/0 (socket says 172.21.15.70:41090) 2023-10-08T05:55:22.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.035+0000 7f0ac52a1640 1 -- 172.21.15.70:0/3691967309 learned_addr learned my addr 172.21.15.70:0/3691967309 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:22.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.035+0000 7f0ac4aa0640 1 --2- 172.21.15.70:0/3691967309 >> v2:172.21.15.181:3300/0 conn(0x7f0ac00cdc30 0x7f0ac00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.035+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394102986 0 0) 0x7f0ac015ac90 con 0x7f0ac00caef0 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.035+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aac003520 con 0x7f0ac00caef0 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2126572239 0 0) 0x7f0aac003520 con 0x7f0ac00caef0 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 >> v2:172.21.15.181:3300/0 conn(0x7f0ac00cdc30 msgr2=0x7f0ac00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 --2- 172.21.15.70:0/3691967309 >> v2:172.21.15.181:3300/0 conn(0x7f0ac00cdc30 0x7f0ac00cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 >> v1:172.21.15.181:6789/0 conn(0x7f0ac00cce90 legacy=0x7f0ac0159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00cc150 con 0x7f0ac00caef0 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0abc001670 con 0x7f0ac00caef0 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 >> v1:172.21.15.70:6789/0 conn(0x7f0ac00caef0 legacy=0x7f0ac00cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aac0039c0 0x7f0aac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0047c0 0x7f0aac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0aac005180 0x7f0aac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac015a9b0 con 0x7f0aac0039c0 2023-10-08T05:55:22.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac00c050 con 0x7f0aac0047c0 2023-10-08T05:55:22.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac00c210 con 0x7f0aac005180 2023-10-08T05:55:22.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.036+0000 7f0ac52a1640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aac0039c0 0x7f0aac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.037+0000 7f0ac4aa0640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0047c0 0x7f0aac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.037+0000 7f0ac52a1640 1 -- 172.21.15.70:0/3691967309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0aac005180 msgr2=0x7f0aac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:22.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.037+0000 7f0ac5aa2640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0aac005180 0x7f0aac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.037+0000 7f0ac52a1640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0aac005180 0x7f0aac00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.037+0000 7f0ac52a1640 1 -- 172.21.15.70:0/3691967309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0047c0 msgr2=0x7f0aac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:22.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.037+0000 7f0ac52a1640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0047c0 0x7f0aac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.037+0000 7f0ac52a1640 1 -- 172.21.15.70:0/3691967309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac003520 con 0x7f0aac0039c0 2023-10-08T05:55:22.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.037+0000 7f0ac5aa2640 1 -- 172.21.15.70:0/3691967309 reap_dead start 2023-10-08T05:55:22.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.037+0000 7f0ac4aa0640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0047c0 0x7f0aac004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:22.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.038+0000 7f0ac52a1640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aac0039c0 0x7f0aac003dc0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f0abc007c00 tx=0x7f0aac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca56ecd2f87fe99 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:22.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.038+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc02ec40 con 0x7f0aac0039c0 2023-10-08T05:55:22.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.038+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc02eda0 con 0x7f0aac0039c0 2023-10-08T05:55:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.038+0000 7f0ab7fff640 1 -- 172.21.15.70:0/3691967309 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc005b10 con 0x7f0aac0039c0 2023-10-08T05:55:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.039+0000 7f0ac62a3640 1 -- 172.21.15.70:0/3691967309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aac0039c0 msgr2=0x7f0aac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.039+0000 7f0ac62a3640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aac0039c0 0x7f0aac003dc0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f0abc007c00 tx=0x7f0aac0057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.039+0000 7f0ac62a3640 1 -- 172.21.15.70:0/3691967309 shutdown_connections 2023-10-08T05:55:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.039+0000 7f0ac62a3640 1 --2- 172.21.15.70:0/3691967309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0aac0039c0 0x7f0aac003dc0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.039+0000 7f0ac62a3640 1 -- 172.21.15.70:0/3691967309 >> 172.21.15.70:0/3691967309 conn(0x7f0ac006c010 msgr2=0x7f0ac006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.039+0000 7f0ac62a3640 1 -- 172.21.15.70:0/3691967309 shutdown_connections 2023-10-08T05:55:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.039+0000 7f0ac62a3640 1 -- 172.21.15.70:0/3691967309 wait complete. 2023-10-08T05:55:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.040+0000 7f0ac62a3640 1 Processor -- start 2023-10-08T05:55:22.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.041+0000 7f0ac62a3640 1 -- start start 2023-10-08T05:55:22.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.041+0000 7f0ac62a3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0039c0 0x7f0ac0155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac62a3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ac00c7da0 0x7f0ac00c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac62a3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ac00ab8d0 0x7f0ac00abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac62a3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00c8b40 con 0x7f0ac00c7da0 2023-10-08T05:55:22.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac62a3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00c8e20 con 0x7f0aac0039c0 2023-10-08T05:55:22.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac62a3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00c9100 con 0x7f0ac00ab8d0 2023-10-08T05:55:22.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac4aa0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ac00c7da0 0x7f0ac00c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac4aa0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ac00c7da0 0x7f0ac00c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42856/0 (socket says 172.21.15.70:42856) 2023-10-08T05:55:22.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac4aa0640 1 -- 172.21.15.70:0/2207728112 learned_addr learned my addr 172.21.15.70:0/2207728112 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:22.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac52a1640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0039c0 0x7f0ac0155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac5aa2640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ac00ab8d0 0x7f0ac00abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac4aa0640 1 -- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ac00ab8d0 msgr2=0x7f0ac00abd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac4aa0640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ac00ab8d0 0x7f0ac00abd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac4aa0640 1 -- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0039c0 msgr2=0x7f0ac0155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac4aa0640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0039c0 0x7f0ac0155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac4aa0640 1 -- 172.21.15.70:0/2207728112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00c93e0 con 0x7f0ac00c7da0 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.042+0000 7f0ac52a1640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0039c0 0x7f0ac0155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.043+0000 7f0ac4aa0640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ac00c7da0 0x7f0ac00c8220 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00107c0 tx=0x7f0ab0010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.043+0000 7f0ab5ffb640 1 -- 172.21.15.70:0/2207728112 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000eb60 con 0x7f0ac00c7da0 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.043+0000 7f0ab5ffb640 1 -- 172.21.15.70:0/2207728112 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab0010ed0 con 0x7f0ac00c7da0 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.043+0000 7f0ab5ffb640 1 -- 172.21.15.70:0/2207728112 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab001c400 con 0x7f0ac00c7da0 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.043+0000 7f0ac62a3640 1 -- 172.21.15.70:0/2207728112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac00c96c0 con 0x7f0ac00c7da0 2023-10-08T05:55:22.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.043+0000 7f0ac62a3640 1 -- 172.21.15.70:0/2207728112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac00c8760 con 0x7f0ac00c7da0 2023-10-08T05:55:22.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.044+0000 7f0ac62a3640 1 -- 172.21.15.70:0/2207728112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac0074150 con 0x7f0ac00c7da0 2023-10-08T05:55:22.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.045+0000 7f0ab5ffb640 1 -- 172.21.15.70:0/2207728112 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0ab001c560 con 0x7f0ac00c7da0 2023-10-08T05:55:22.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.046+0000 7f0ab5ffb640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0aa406af90 0x7f0aa406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.046+0000 7f0ab5ffb640 1 -- 172.21.15.70:0/2207728112 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5221+0+0 (secure 0 0 0) 0x7f0ab0089a70 con 0x7f0ac00c7da0 2023-10-08T05:55:22.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.047+0000 7f0ac52a1640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0aa406af90 0x7f0aa406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.048+0000 7f0ac52a1640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0aa406af90 0x7f0aa406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0abc036000 tx=0x7f0abc009030 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:22.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.050+0000 7f0ab5ffb640 1 -- 172.21.15.70:0/2207728112 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ab0054c00 con 0x7f0ac00c7da0 2023-10-08T05:55:22.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.239+0000 7f0ac62a3640 1 -- 172.21.15.70:0/2207728112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0ac00cb7e0 con 0x7f0ac00c7da0 2023-10-08T05:55:22.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.240+0000 7f0ab5ffb640 1 -- 172.21.15.70:0/2207728112 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ab0058220 con 0x7f0ac00c7da0 2023-10-08T05:55:22.241 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:55:22.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.242+0000 7f0ac62a3640 1 -- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0aa406af90 msgr2=0x7f0aa406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.242+0000 7f0ac62a3640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0aa406af90 0x7f0aa406d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0abc036000 tx=0x7f0abc009030 comp rx=0 tx=0).stop 2023-10-08T05:55:22.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.242+0000 7f0ac62a3640 1 -- 172.21.15.70:0/2207728112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ac00c7da0 msgr2=0x7f0ac00c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.242+0000 7f0ac62a3640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ac00c7da0 0x7f0ac00c8220 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00107c0 tx=0x7f0ab0010c80 comp rx=0 tx=0).stop 2023-10-08T05:55:22.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.242+0000 7f0ac62a3640 1 -- 172.21.15.70:0/2207728112 shutdown_connections 2023-10-08T05:55:22.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.242+0000 7f0ac62a3640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ac00ab8d0 0x7f0ac00abd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.242+0000 7f0ac62a3640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f0aa406af90 0x7f0aa406d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.242+0000 7f0ac62a3640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ac00c7da0 0x7f0ac00c8220 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.242+0000 7f0ac62a3640 1 --2- 172.21.15.70:0/2207728112 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0aac0039c0 0x7f0ac0155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.242+0000 7f0ac62a3640 1 -- 172.21.15.70:0/2207728112 >> 172.21.15.70:0/2207728112 conn(0x7f0ac006c010 msgr2=0x7f0ac00c55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:22.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.243+0000 7f0ac62a3640 1 -- 172.21.15.70:0/2207728112 shutdown_connections 2023-10-08T05:55:22.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.243+0000 7f0ac62a3640 1 -- 172.21.15.70:0/2207728112 wait complete. 2023-10-08T05:55:22.262 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T05:55:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.478+0000 7fccbd010640 1 Processor -- start 2023-10-08T05:55:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.479+0000 7fccbd010640 1 -- start start 2023-10-08T05:55:22.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.480+0000 7fccbd010640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fccb8157000 0x7fccb80cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.480+0000 7fccbd010640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fccb80cb190 0x7fccb80cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.480+0000 7fccbd010640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb80adfc0 con 0x7fccb81597d0 2023-10-08T05:55:22.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.480+0000 7fccbd010640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb80cbb40 con 0x7fccb8157000 2023-10-08T05:55:22.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.480+0000 7fccbd010640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb80ccce0 con 0x7fccb80cb190 2023-10-08T05:55:22.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.480+0000 7fccb6d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fccb8157000 0x7fccb80cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.480+0000 7fccb6d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fccb8157000 0x7fccb80cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42864/0 (socket says 172.21.15.70:42864) 2023-10-08T05:55:22.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.480+0000 7fccb6d9d640 1 -- 172.21.15.70:0/2417887117 learned_addr learned my addr 172.21.15.70:0/2417887117 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:22.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.481+0000 7fccb6d9d640 1 -- 172.21.15.70:0/2417887117 >> v2:172.21.15.181:3300/0 conn(0x7fccb80cb190 msgr2=0x7fccb80cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:22.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.481+0000 7fccb6d9d640 1 --2- 172.21.15.70:0/2417887117 >> v2:172.21.15.181:3300/0 conn(0x7fccb80cb190 0x7fccb80cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.481+0000 7fccb6d9d640 1 -- 172.21.15.70:0/2417887117 >> v1:172.21.15.181:6789/0 conn(0x7fccb81597d0 legacy=0x7fccb806ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:22.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.481+0000 7fccb6d9d640 1 -- 172.21.15.70:0/2417887117 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8159cb0 con 0x7fccb8157000 2023-10-08T05:55:22.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.482+0000 7fccb6d9d640 1 --2- 172.21.15.70:0/2417887117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb8157000 0x7fccb80cc7a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fccb80b1450 tx=0x7fcca802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645d71b1d4326100 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.482+0000 7fccb5d9b640 1 -- 172.21.15.70:0/2417887117 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca802ca80 con 0x7fccb8157000 2023-10-08T05:55:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.482+0000 7fccb5d9b640 1 -- 172.21.15.70:0/2417887117 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca802cbe0 con 0x7fccb8157000 2023-10-08T05:55:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.482+0000 7fccb5d9b640 1 -- 172.21.15.70:0/2417887117 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca80343f0 con 0x7fccb8157000 2023-10-08T05:55:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.483+0000 7fccbd010640 1 -- 172.21.15.70:0/2417887117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb8157000 msgr2=0x7fccb80cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.483+0000 7fccbd010640 1 --2- 172.21.15.70:0/2417887117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb8157000 0x7fccb80cc7a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fccb80b1450 tx=0x7fcca802bee0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.483+0000 7fccbd010640 1 -- 172.21.15.70:0/2417887117 shutdown_connections 2023-10-08T05:55:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.483+0000 7fccbd010640 1 --2- 172.21.15.70:0/2417887117 >> v2:172.21.15.181:3300/0 conn(0x7fccb80cb190 0x7fccb80cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.483+0000 7fccbd010640 1 --2- 172.21.15.70:0/2417887117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb8157000 0x7fccb80cc7a0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.483+0000 7fccbd010640 1 -- 172.21.15.70:0/2417887117 >> 172.21.15.70:0/2417887117 conn(0x7fccb806a380 msgr2=0x7fccb80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.483+0000 7fccbd010640 1 -- 172.21.15.70:0/2417887117 shutdown_connections 2023-10-08T05:55:22.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.483+0000 7fccbd010640 1 -- 172.21.15.70:0/2417887117 wait complete. 2023-10-08T05:55:22.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.484+0000 7fccbd010640 1 Processor -- start 2023-10-08T05:55:22.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.485+0000 7fccbd010640 1 -- start start 2023-10-08T05:55:22.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.485+0000 7fccbd010640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb80cb190 0x7fccb8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.485+0000 7fccbd010640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb81597d0 0x7fccb81495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.485+0000 7fccbd010640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccb8158f70 0x7fccb80c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.485+0000 7fccbd010640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb81594c0 con 0x7fccb80cb190 2023-10-08T05:55:22.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.485+0000 7fccbd010640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8158740 con 0x7fccb81597d0 2023-10-08T05:55:22.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.485+0000 7fccbd010640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb81582b0 con 0x7fccb8158f70 2023-10-08T05:55:22.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.486+0000 7fccb6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb80cb190 0x7fccb8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.486+0000 7fccb6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb80cb190 0x7fccb8150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42876/0 (socket says 172.21.15.70:42876) 2023-10-08T05:55:22.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.486+0000 7fccb6d9d640 1 -- 172.21.15.70:0/1107998016 learned_addr learned my addr 172.21.15.70:0/1107998016 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:22.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.486+0000 7fccb6d9d640 1 -- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccb8158f70 msgr2=0x7fccb80c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.486+0000 7fccb6d9d640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccb8158f70 0x7fccb80c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.486+0000 7fccb6d9d640 1 -- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb81597d0 msgr2=0x7fccb81495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.486+0000 7fccb659c640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb81597d0 0x7fccb81495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.486+0000 7fccb6d9d640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb81597d0 0x7fccb81495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.486+0000 7fccb6d9d640 1 -- 172.21.15.70:0/1107998016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8009b30 con 0x7fccb80cb190 2023-10-08T05:55:22.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.487+0000 7fccb6d9d640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb80cb190 0x7fccb8150280 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fcca8036040 tx=0x7fcca8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:22.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.487+0000 7fcc9f7fe640 1 -- 172.21.15.70:0/1107998016 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca803e450 con 0x7fccb80cb190 2023-10-08T05:55:22.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.487+0000 7fcc9f7fe640 1 -- 172.21.15.70:0/1107998016 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca803e5b0 con 0x7fccb80cb190 2023-10-08T05:55:22.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.487+0000 7fcc9f7fe640 1 -- 172.21.15.70:0/1107998016 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca803e960 con 0x7fccb80cb190 2023-10-08T05:55:22.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.487+0000 7fccbd010640 1 -- 172.21.15.70:0/1107998016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb814f630 con 0x7fccb80cb190 2023-10-08T05:55:22.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.487+0000 7fccbd010640 1 -- 172.21.15.70:0/1107998016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb8157c60 con 0x7fccb80cb190 2023-10-08T05:55:22.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.488+0000 7fccbd010640 1 -- 172.21.15.70:0/1107998016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb8067490 con 0x7fccb80cb190 2023-10-08T05:55:22.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.489+0000 7fcc9f7fe640 1 -- 172.21.15.70:0/1107998016 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcca802cd50 con 0x7fccb80cb190 2023-10-08T05:55:22.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.490+0000 7fcc9f7fe640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcca406afc0 0x7fcca406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.490+0000 7fcc9f7fe640 1 -- 172.21.15.70:0/1107998016 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fcca80ac360 con 0x7fccb80cb190 2023-10-08T05:55:22.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.491+0000 7fccb659c640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcca406afc0 0x7fcca406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.492+0000 7fccb659c640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcca406afc0 0x7fcca406d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fccb814f8d0 tx=0x7fccac008040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:22.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.495+0000 7fcc9f7fe640 1 -- 172.21.15.70:0/1107998016 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcca8077560 con 0x7fccb80cb190 2023-10-08T05:55:22.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.672+0000 7fccbd010640 1 -- 172.21.15.70:0/1107998016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fccb806c6d0 con 0x7fccb80cb190 2023-10-08T05:55:22.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.673+0000 7fcc9f7fe640 1 -- 172.21.15.70:0/1107998016 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v130) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcca807ab80 con 0x7fccb80cb190 2023-10-08T05:55:22.675 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:22.675 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:55:22.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:22.675 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T05:55:22.675 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T05:55:22.675 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:22.676 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:55:22.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 -- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcca406afc0 msgr2=0x7fcca406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcca406afc0 0x7fcca406d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fccb814f8d0 tx=0x7fccac008040 comp rx=0 tx=0).stop 2023-10-08T05:55:22.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 -- 172.21.15.70:0/1107998016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb80cb190 msgr2=0x7fccb8150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb80cb190 0x7fccb8150280 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fcca8036040 tx=0x7fcca8004a40 comp rx=0 tx=0).stop 2023-10-08T05:55:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 -- 172.21.15.70:0/1107998016 shutdown_connections 2023-10-08T05:55:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fccb8158f70 0x7fccb80c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fccb81597d0 0x7fccb81495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcca406afc0 0x7fcca406d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 --2- 172.21.15.70:0/1107998016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccb80cb190 0x7fccb8150280 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 -- 172.21.15.70:0/1107998016 >> 172.21.15.70:0/1107998016 conn(0x7fccb806a380 msgr2=0x7fccb80ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 -- 172.21.15.70:0/1107998016 shutdown_connections 2023-10-08T05:55:22.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.676+0000 7fccbd010640 1 -- 172.21.15.70:0/1107998016 wait complete. 2023-10-08T05:55:22.695 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T05:55:22.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T05:55:22.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.908+0000 7fe034a32640 1 Processor -- start 2023-10-08T05:55:22.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.908+0000 7fe034a32640 1 -- start start 2023-10-08T05:55:22.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe034a32640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe0300702d0 0x7fe03015d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe034a32640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0300b1750 0x7fe0300b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe034a32640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0300b30a0 con 0x7fe0300706b0 2023-10-08T05:55:22.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe034a32640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03015dcf0 con 0x7fe0300b1750 2023-10-08T05:55:22.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe034a32640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0300b3320 con 0x7fe0300702d0 2023-10-08T05:55:22.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe025d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0300b1750 0x7fe0300b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe025d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0300b1750 0x7fe0300b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42884/0 (socket says 172.21.15.70:42884) 2023-10-08T05:55:22.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe025d9b640 1 -- 172.21.15.70:0/886102318 learned_addr learned my addr 172.21.15.70:0/886102318 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:22.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe025d9b640 1 -- 172.21.15.70:0/886102318 >> v2:172.21.15.181:3300/0 conn(0x7fe0300702d0 msgr2=0x7fe03015d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:22.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe025d9b640 1 --2- 172.21.15.70:0/886102318 >> v2:172.21.15.181:3300/0 conn(0x7fe0300702d0 0x7fe03015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe025d9b640 1 -- 172.21.15.70:0/886102318 >> v1:172.21.15.181:6789/0 conn(0x7fe0300706b0 legacy=0x7fe0300c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:22.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.910+0000 7fe025d9b640 1 -- 172.21.15.70:0/886102318 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300b3600 con 0x7fe0300b1750 2023-10-08T05:55:22.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.911+0000 7fe025d9b640 1 --2- 172.21.15.70:0/886102318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300b1750 0x7fe0300b40e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe03006ab40 tx=0x7fe01802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e357e2a13063a0df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:22.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.912+0000 7fe02dd9b640 1 -- 172.21.15.70:0/886102318 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018031800 con 0x7fe0300b1750 2023-10-08T05:55:22.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.912+0000 7fe02dd9b640 1 -- 172.21.15.70:0/886102318 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe018031960 con 0x7fe0300b1750 2023-10-08T05:55:22.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.912+0000 7fe02dd9b640 1 -- 172.21.15.70:0/886102318 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018031cf0 con 0x7fe0300b1750 2023-10-08T05:55:22.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.912+0000 7fe034a32640 1 -- 172.21.15.70:0/886102318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300b1750 msgr2=0x7fe0300b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.912+0000 7fe034a32640 1 --2- 172.21.15.70:0/886102318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300b1750 0x7fe0300b40e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe03006ab40 tx=0x7fe01802b8a0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.912+0000 7fe034a32640 1 -- 172.21.15.70:0/886102318 shutdown_connections 2023-10-08T05:55:22.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.912+0000 7fe034a32640 1 --2- 172.21.15.70:0/886102318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300b1750 0x7fe0300b40e0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.912+0000 7fe034a32640 1 --2- 172.21.15.70:0/886102318 >> v2:172.21.15.181:3300/0 conn(0x7fe0300702d0 0x7fe03015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.912+0000 7fe034a32640 1 -- 172.21.15.70:0/886102318 >> 172.21.15.70:0/886102318 conn(0x7fe030069d50 msgr2=0x7fe03006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:22.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.913+0000 7fe034a32640 1 -- 172.21.15.70:0/886102318 shutdown_connections 2023-10-08T05:55:22.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.913+0000 7fe034a32640 1 -- 172.21.15.70:0/886102318 wait complete. 2023-10-08T05:55:22.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.914+0000 7fe034a32640 1 Processor -- start 2023-10-08T05:55:22.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.914+0000 7fe034a32640 1 -- start start 2023-10-08T05:55:22.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.915+0000 7fe034a32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300702d0 0x7fe030155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.915+0000 7fe034a32640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0300706b0 0x7fe030153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.915+0000 7fe034a32640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0300b1750 0x7fe0301540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.915+0000 7fe034a32640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0301534a0 con 0x7fe0300702d0 2023-10-08T05:55:22.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.915+0000 7fe034a32640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03015bcf0 con 0x7fe0300b1750 2023-10-08T05:55:22.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.915+0000 7fe034a32640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03015b6c0 con 0x7fe0300706b0 2023-10-08T05:55:22.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.915+0000 7fe02e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300702d0 0x7fe030155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe02e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300702d0 0x7fe030155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42888/0 (socket says 172.21.15.70:42888) 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe02e59c640 1 -- 172.21.15.70:0/2627102283 learned_addr learned my addr 172.21.15.70:0/2627102283 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe02e59c640 1 -- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0300706b0 msgr2=0x7fe030153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe025d9b640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0300706b0 0x7fe030153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe02e59c640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0300706b0 0x7fe030153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe02e59c640 1 -- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0300b1750 msgr2=0x7fe0301540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe02ed9d640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0300b1750 0x7fe0301540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe02e59c640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0300b1750 0x7fe0301540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe02e59c640 1 -- 172.21.15.70:0/2627102283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0180094d0 con 0x7fe0300702d0 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe02e59c640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300702d0 0x7fe030155920 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe02000b4a0 tx=0x7fe02000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.916+0000 7fe0277fe640 1 -- 172.21.15.70:0/2627102283 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020010610 con 0x7fe0300702d0 2023-10-08T05:55:22.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.917+0000 7fe0277fe640 1 -- 172.21.15.70:0/2627102283 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020010770 con 0x7fe0300702d0 2023-10-08T05:55:22.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.917+0000 7fe0277fe640 1 -- 172.21.15.70:0/2627102283 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe020010b20 con 0x7fe0300702d0 2023-10-08T05:55:22.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.917+0000 7fe034a32640 1 -- 172.21.15.70:0/2627102283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe030148810 con 0x7fe0300702d0 2023-10-08T05:55:22.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.917+0000 7fe034a32640 1 -- 172.21.15.70:0/2627102283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03015b910 con 0x7fe0300702d0 2023-10-08T05:55:22.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.918+0000 7fe034a32640 1 -- 172.21.15.70:0/2627102283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03004e160 con 0x7fe0300702d0 2023-10-08T05:55:22.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.919+0000 7fe0277fe640 1 -- 172.21.15.70:0/2627102283 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe020010c80 con 0x7fe0300702d0 2023-10-08T05:55:22.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.919+0000 7fe0277fe640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe01006af90 0x7fe01006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:22.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.920+0000 7fe0277fe640 1 -- 172.21.15.70:0/2627102283 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5221+0+0 (secure 0 0 0) 0x7fe020089180 con 0x7fe0300702d0 2023-10-08T05:55:22.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.920+0000 7fe025d9b640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe01006af90 0x7fe01006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:22.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.921+0000 7fe025d9b640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe01006af90 0x7fe01006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe018036000 tx=0x7fe01803c040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:22.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:22.924+0000 7fe0277fe640 1 -- 172.21.15.70:0/2627102283 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe020054390 con 0x7fe0300702d0 2023-10-08T05:55:23.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.099+0000 7fe034a32640 1 -- 172.21.15.70:0/2627102283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe0300710b0 con 0x7fe0300702d0 2023-10-08T05:55:23.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.872+0000 7fe0277fe640 1 -- 172.21.15.70:0/2627102283 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v131) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe0200579b0 con 0x7fe0300702d0 2023-10-08T05:55:23.874 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T05:55:23.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 -- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe01006af90 msgr2=0x7fe01006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:23.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe01006af90 0x7fe01006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe018036000 tx=0x7fe01803c040 comp rx=0 tx=0).stop 2023-10-08T05:55:23.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 -- 172.21.15.70:0/2627102283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300702d0 msgr2=0x7fe030155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:23.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300702d0 0x7fe030155920 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe02000b4a0 tx=0x7fe02000b960 comp rx=0 tx=0).stop 2023-10-08T05:55:23.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 -- 172.21.15.70:0/2627102283 shutdown_connections 2023-10-08T05:55:23.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0300b1750 0x7fe0301540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:23.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe0300706b0 0x7fe030153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:23.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe01006af90 0x7fe01006d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:23.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 --2- 172.21.15.70:0/2627102283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0300702d0 0x7fe030155920 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:23.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 -- 172.21.15.70:0/2627102283 >> 172.21.15.70:0/2627102283 conn(0x7fe030069d50 msgr2=0x7fe0300c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:23.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 -- 172.21.15.70:0/2627102283 shutdown_connections 2023-10-08T05:55:23.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:23.876+0000 7fe034a32640 1 -- 172.21.15.70:0/2627102283 wait complete. 2023-10-08T05:55:23.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T05:55:24.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.098+0000 7fbd38d00640 1 Processor -- start 2023-10-08T05:55:24.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.099+0000 7fbd38d00640 1 -- start start 2023-10-08T05:55:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.100+0000 7fbd38d00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbd340cb500 0x7fbd340cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.100+0000 7fbd38d00640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbd3406cfa0 0x7fbd3415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.100+0000 7fbd38d00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd340c8c60 con 0x7fbd3415b630 2023-10-08T05:55:24.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.100+0000 7fbd38d00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd340cbe20 con 0x7fbd340cb500 2023-10-08T05:55:24.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.100+0000 7fbd38d00640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd340b1ac0 con 0x7fbd3406cfa0 2023-10-08T05:55:24.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.100+0000 7fbd3259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbd340cb500 0x7fbd340cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.101+0000 7fbd32d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbd3415b630 0x7fbd34070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54736/0 (socket says 172.21.15.70:54736) 2023-10-08T05:55:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.101+0000 7fbd32d9d640 1 -- 172.21.15.70:0/3718833576 learned_addr learned my addr 172.21.15.70:0/3718833576 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.101+0000 7fbd2bfff640 1 --2- 172.21.15.70:0/3718833576 >> v2:172.21.15.181:3300/0 conn(0x7fbd3406cfa0 0x7fbd3415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.101+0000 7fbd3259c640 1 -- 172.21.15.70:0/3718833576 >> v2:172.21.15.181:3300/0 conn(0x7fbd3406cfa0 msgr2=0x7fbd3415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.101+0000 7fbd3259c640 1 --2- 172.21.15.70:0/3718833576 >> v2:172.21.15.181:3300/0 conn(0x7fbd3406cfa0 0x7fbd3415b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.101+0000 7fbd3259c640 1 -- 172.21.15.70:0/3718833576 >> v1:172.21.15.70:6789/0 conn(0x7fbd3415b630 legacy=0x7fbd34070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.101+0000 7fbd31d9b640 1 -- 172.21.15.70:0/3718833576 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415605088 0 0) 0x7fbd340c8c60 con 0x7fbd3415b630 2023-10-08T05:55:24.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.101+0000 7fbd3259c640 1 -- 172.21.15.70:0/3718833576 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd340b1da0 con 0x7fbd340cb500 2023-10-08T05:55:24.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.102+0000 7fbd3259c640 1 --2- 172.21.15.70:0/3718833576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd340cb500 0x7fbd340cb8e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fbd340b0690 tx=0x7fbd2402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc30acc54eb3686 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.102+0000 7fbd31d9b640 1 -- 172.21.15.70:0/3718833576 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2402e070 con 0x7fbd340cb500 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.102+0000 7fbd31d9b640 1 -- 172.21.15.70:0/3718833576 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd240315a0 con 0x7fbd340cb500 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.102+0000 7fbd31d9b640 1 -- 172.21.15.70:0/3718833576 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24031930 con 0x7fbd340cb500 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.103+0000 7fbd38d00640 1 -- 172.21.15.70:0/3718833576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd340cb500 msgr2=0x7fbd340cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.103+0000 7fbd38d00640 1 --2- 172.21.15.70:0/3718833576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd340cb500 0x7fbd340cb8e0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fbd340b0690 tx=0x7fbd2402b840 comp rx=0 tx=0).stop 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.103+0000 7fbd38d00640 1 -- 172.21.15.70:0/3718833576 shutdown_connections 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.103+0000 7fbd38d00640 1 --2- 172.21.15.70:0/3718833576 >> v2:172.21.15.181:3300/0 conn(0x7fbd3406cfa0 0x7fbd3415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.103+0000 7fbd38d00640 1 --2- 172.21.15.70:0/3718833576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd340cb500 0x7fbd340cb8e0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.103+0000 7fbd38d00640 1 -- 172.21.15.70:0/3718833576 >> 172.21.15.70:0/3718833576 conn(0x7fbd3406a140 msgr2=0x7fbd3406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.103+0000 7fbd38d00640 1 -- 172.21.15.70:0/3718833576 shutdown_connections 2023-10-08T05:55:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.103+0000 7fbd38d00640 1 -- 172.21.15.70:0/3718833576 wait complete. 2023-10-08T05:55:24.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.105+0000 7fbd38d00640 1 Processor -- start 2023-10-08T05:55:24.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.105+0000 7fbd38d00640 1 -- start start 2023-10-08T05:55:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.106+0000 7fbd38d00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd3406cfa0 0x7fbd341f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.106+0000 7fbd38d00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd340cb500 0x7fbd34157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.106+0000 7fbd38d00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3415b630 0x7fbd341f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.106+0000 7fbd38d00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd341f6390 con 0x7fbd3415b630 2023-10-08T05:55:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.106+0000 7fbd38d00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd341f6670 con 0x7fbd340cb500 2023-10-08T05:55:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.106+0000 7fbd38d00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd341f6950 con 0x7fbd3406cfa0 2023-10-08T05:55:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.106+0000 7fbd32d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3415b630 0x7fbd341f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.106+0000 7fbd32d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3415b630 0x7fbd341f2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54760/0 (socket says 172.21.15.70:54760) 2023-10-08T05:55:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.106+0000 7fbd32d9d640 1 -- 172.21.15.70:0/2825924674 learned_addr learned my addr 172.21.15.70:0/2825924674 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:24.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd3259c640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd3406cfa0 0x7fbd341f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd2bfff640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd340cb500 0x7fbd34157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd32d9d640 1 -- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd3406cfa0 msgr2=0x7fbd341f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd32d9d640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd3406cfa0 0x7fbd341f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd32d9d640 1 -- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd340cb500 msgr2=0x7fbd34157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd32d9d640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd340cb500 0x7fbd34157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd32d9d640 1 -- 172.21.15.70:0/2825924674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd24009400 con 0x7fbd3415b630 2023-10-08T05:55:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd2bfff640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd340cb500 0x7fbd34157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd32d9d640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3415b630 0x7fbd341f2d80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00c700 tx=0x7fbd2c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd2affd640 1 -- 172.21.15.70:0/2825924674 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00ce10 con 0x7fbd3415b630 2023-10-08T05:55:24.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd2affd640 1 -- 172.21.15.70:0/2825924674 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2c00a640 con 0x7fbd3415b630 2023-10-08T05:55:24.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd38d00640 1 -- 172.21.15.70:0/2825924674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd341f2860 con 0x7fbd3415b630 2023-10-08T05:55:24.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.107+0000 7fbd2affd640 1 -- 172.21.15.70:0/2825924674 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c00aa70 con 0x7fbd3415b630 2023-10-08T05:55:24.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.108+0000 7fbd38d00640 1 -- 172.21.15.70:0/2825924674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd341f6080 con 0x7fbd3415b630 2023-10-08T05:55:24.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.108+0000 7fbd38d00640 1 -- 172.21.15.70:0/2825924674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd341f2aa0 con 0x7fbd3415b630 2023-10-08T05:55:24.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.110+0000 7fbd2affd640 1 -- 172.21.15.70:0/2825924674 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbd2c00abd0 con 0x7fbd3415b630 2023-10-08T05:55:24.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.111+0000 7fbd2affd640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd1406af90 0x7fbd1406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:24.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.111+0000 7fbd2affd640 1 -- 172.21.15.70:0/2825924674 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5591+0+0 (secure 0 0 0) 0x7fbd2c088f60 con 0x7fbd3415b630 2023-10-08T05:55:24.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.112+0000 7fbd3259c640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd1406af90 0x7fbd1406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:24.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.113+0000 7fbd3259c640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd1406af90 0x7fbd1406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbd24036000 tx=0x7fbd24008030 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:24.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.115+0000 7fbd2affd640 1 -- 172.21.15.70:0/2825924674 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd2c053ee0 con 0x7fbd3415b630 2023-10-08T05:55:24.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.303+0000 7fbd38d00640 1 -- 172.21.15.70:0/2825924674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbd340b16b0 con 0x7fbd3415b630 2023-10-08T05:55:24.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.889+0000 7fbd2affd640 1 -- 172.21.15.70:0/2825924674 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v132) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbd2c018030 con 0x7fbd3415b630 2023-10-08T05:55:24.891 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T05:55:24.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 -- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd1406af90 msgr2=0x7fbd1406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd1406af90 0x7fbd1406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbd24036000 tx=0x7fbd24008030 comp rx=0 tx=0).stop 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 -- 172.21.15.70:0/2825924674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3415b630 msgr2=0x7fbd341f2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3415b630 0x7fbd341f2d80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00c700 tx=0x7fbd2c00cbc0 comp rx=0 tx=0).stop 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 -- 172.21.15.70:0/2825924674 shutdown_connections 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbd1406af90 0x7fbd1406d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd3415b630 0x7fbd341f2d80 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbd340cb500 0x7fbd34157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 --2- 172.21.15.70:0/2825924674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbd3406cfa0 0x7fbd341f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 -- 172.21.15.70:0/2825924674 >> 172.21.15.70:0/2825924674 conn(0x7fbd3406a140 msgr2=0x7fbd340b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 -- 172.21.15.70:0/2825924674 shutdown_connections 2023-10-08T05:55:24.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:24.894+0000 7fbd38d00640 1 -- 172.21.15.70:0/2825924674 wait complete. 2023-10-08T05:55:24.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T05:55:25.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.116+0000 7f779da75640 1 Processor -- start 2023-10-08T05:55:25.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.117+0000 7f779da75640 1 -- start start 2023-10-08T05:55:25.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.118+0000 7f779da75640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77980cf2b0 0x7f77980d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:25.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.118+0000 7f779da75640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77980cd520 con 0x7f77980ce510 2023-10-08T05:55:25.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.118+0000 7f779da75640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77980d1910 con 0x7f77980cf690 2023-10-08T05:55:25.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.118+0000 7f779da75640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77980ce080 con 0x7f77980cf2b0 2023-10-08T05:55:25.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.118+0000 7f779ca73640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77980cf2b0 0x7f77980d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:25.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.118+0000 7f779ca73640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77980cf2b0 0x7f77980d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54770/0 (socket says 172.21.15.70:54770) 2023-10-08T05:55:25.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.119+0000 7f779ca73640 1 -- 172.21.15.70:0/4167996713 learned_addr learned my addr 172.21.15.70:0/4167996713 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:25.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.119+0000 7f779ca73640 1 -- 172.21.15.70:0/4167996713 >> v1:172.21.15.181:6789/0 conn(0x7f77980cf690 legacy=0x7f77980d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:25.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.119+0000 7f779ca73640 1 -- 172.21.15.70:0/4167996713 >> v1:172.21.15.70:6789/0 conn(0x7f77980ce510 legacy=0x7f779815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:25.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.119+0000 7f779ca73640 1 -- 172.21.15.70:0/4167996713 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980cd750 con 0x7f77980cf2b0 2023-10-08T05:55:25.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.120+0000 7f779ca73640 1 --2- 172.21.15.70:0/4167996713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 0x7f77980d34d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7788009870 tx=0x7f778802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a64599b1added2e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:25.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.120+0000 7f77977fe640 1 -- 172.21.15.70:0/4167996713 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788031870 con 0x7f77980cf2b0 2023-10-08T05:55:25.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.120+0000 7f77977fe640 1 -- 172.21.15.70:0/4167996713 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77880319d0 con 0x7f77980cf2b0 2023-10-08T05:55:25.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.120+0000 7f77977fe640 1 -- 172.21.15.70:0/4167996713 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788031d60 con 0x7f77980cf2b0 2023-10-08T05:55:25.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.121+0000 7f779da75640 1 -- 172.21.15.70:0/4167996713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 msgr2=0x7f77980d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:25.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.121+0000 7f779da75640 1 --2- 172.21.15.70:0/4167996713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 0x7f77980d34d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f7788009870 tx=0x7f778802b910 comp rx=0 tx=0).stop 2023-10-08T05:55:25.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.121+0000 7f779da75640 1 -- 172.21.15.70:0/4167996713 shutdown_connections 2023-10-08T05:55:25.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.121+0000 7f779da75640 1 --2- 172.21.15.70:0/4167996713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 0x7f77980d34d0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:25.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.121+0000 7f779da75640 1 -- 172.21.15.70:0/4167996713 >> 172.21.15.70:0/4167996713 conn(0x7f779809bc40 msgr2=0x7f77980d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:25.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.121+0000 7f779da75640 1 -- 172.21.15.70:0/4167996713 shutdown_connections 2023-10-08T05:55:25.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.121+0000 7f779da75640 1 -- 172.21.15.70:0/4167996713 wait complete. 2023-10-08T05:55:25.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.123+0000 7f779da75640 1 Processor -- start 2023-10-08T05:55:25.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.123+0000 7f779da75640 1 -- start start 2023-10-08T05:55:25.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.124+0000 7f779da75640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980ce510 0x7f7798154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:25.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.124+0000 7f779da75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 0x7f77980cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:25.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.124+0000 7f779da75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77980cf690 0x7f77980cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:25.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.124+0000 7f779da75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77980ca940 con 0x7f77980cf2b0 2023-10-08T05:55:25.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.124+0000 7f779da75640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77980c9dc0 con 0x7f77980ce510 2023-10-08T05:55:25.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.124+0000 7f779da75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77980ca0a0 con 0x7f77980cf690 2023-10-08T05:55:25.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.124+0000 7f7797fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 0x7f77980cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:25.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.124+0000 7f7797fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 0x7f77980cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54776/0 (socket says 172.21.15.70:54776) 2023-10-08T05:55:25.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.124+0000 7f7797fff640 1 -- 172.21.15.70:0/2029301570 learned_addr learned my addr 172.21.15.70:0/2029301570 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:25.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.124+0000 7f779ca73640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980ce510 0x7f7798154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f7797fff640 1 -- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77980cf690 msgr2=0x7f77980cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f779d274640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77980cf690 0x7f77980cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f7797fff640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77980cf690 0x7f77980cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f7797fff640 1 -- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980ce510 msgr2=0x7f7798154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f7797fff640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980ce510 0x7f7798154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f7797fff640 1 -- 172.21.15.70:0/2029301570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77880094d0 con 0x7f77980cf2b0 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f779ca73640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980ce510 0x7f7798154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f7797fff640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 0x7f77980cbb80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f778000ba50 tx=0x7f778000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f77957fa640 1 -- 172.21.15.70:0/2029301570 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778000cc10 con 0x7f77980cf2b0 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f77957fa640 1 -- 172.21.15.70:0/2029301570 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778000cd70 con 0x7f77980cf2b0 2023-10-08T05:55:25.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f77957fa640 1 -- 172.21.15.70:0/2029301570 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780014680 con 0x7f77980cf2b0 2023-10-08T05:55:25.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.125+0000 7f779da75640 1 -- 172.21.15.70:0/2029301570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77980ca380 con 0x7f77980cf2b0 2023-10-08T05:55:25.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.126+0000 7f779da75640 1 -- 172.21.15.70:0/2029301570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77981552a0 con 0x7f77980cf2b0 2023-10-08T05:55:25.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.127+0000 7f779da75640 1 -- 172.21.15.70:0/2029301570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7798074150 con 0x7f77980cf2b0 2023-10-08T05:55:25.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.129+0000 7f77957fa640 1 -- 172.21.15.70:0/2029301570 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f778001d3b0 con 0x7f77980cf2b0 2023-10-08T05:55:25.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.130+0000 7f77957fa640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f777806af90 0x7f777806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:25.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.130+0000 7f77957fa640 1 -- 172.21.15.70:0/2029301570 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5973+0+0 (secure 0 0 0) 0x7f7780089ca0 con 0x7f77980cf2b0 2023-10-08T05:55:25.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.131+0000 7f779ca73640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f777806af90 0x7f777806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:25.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.132+0000 7f779ca73640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f777806af90 0x7f777806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7788036000 tx=0x7f778803c040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:25.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.133+0000 7f77957fa640 1 -- 172.21.15.70:0/2029301570 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7780054ba0 con 0x7f77980cf2b0 2023-10-08T05:55:25.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.360+0000 7f779da75640 1 -- 172.21.15.70:0/2029301570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f77980d0070 con 0x7f77980cf2b0 2023-10-08T05:55:25.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.893+0000 7f77957fa640 1 -- 172.21.15.70:0/2029301570 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7780019020 con 0x7f77980cf2b0 2023-10-08T05:55:25.895 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 15 and data pool 16 2023-10-08T05:55:25.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.900+0000 7f779da75640 1 -- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f777806af90 msgr2=0x7f777806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:25.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.900+0000 7f779da75640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f777806af90 0x7f777806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7788036000 tx=0x7f778803c040 comp rx=0 tx=0).stop 2023-10-08T05:55:25.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.900+0000 7f779da75640 1 -- 172.21.15.70:0/2029301570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 msgr2=0x7f77980cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:25.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.900+0000 7f779da75640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 0x7f77980cbb80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f778000ba50 tx=0x7f778000bf10 comp rx=0 tx=0).stop 2023-10-08T05:55:25.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.902+0000 7f779da75640 1 -- 172.21.15.70:0/2029301570 shutdown_connections 2023-10-08T05:55:25.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.902+0000 7f779da75640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f77980cf690 0x7f77980cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.902+0000 7f779da75640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f777806af90 0x7f777806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.902+0000 7f779da75640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77980cf2b0 0x7f77980cbb80 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.902+0000 7f779da75640 1 --2- 172.21.15.70:0/2029301570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f77980ce510 0x7f7798154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:25.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.902+0000 7f779da75640 1 -- 172.21.15.70:0/2029301570 >> 172.21.15.70:0/2029301570 conn(0x7f779809bc40 msgr2=0x7f77980c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:25.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.904+0000 7f779da75640 1 -- 172.21.15.70:0/2029301570 shutdown_connections 2023-10-08T05:55:25.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:25.906+0000 7f779da75640 1 -- 172.21.15.70:0/2029301570 wait complete. 2023-10-08T05:55:25.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:55:26.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.128+0000 7fae23ef2640 1 Processor -- start 2023-10-08T05:55:26.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.129+0000 7fae23ef2640 1 -- start start 2023-10-08T05:55:26.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.130+0000 7fae23ef2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae1c15ab40 0x7fae1c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.130+0000 7fae23ef2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fae1c0c90b0 0x7fae1c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.130+0000 7fae23ef2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c15b2b0 con 0x7fae1c0cae70 2023-10-08T05:55:26.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.130+0000 7fae23ef2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c15bf30 con 0x7fae1c15ab40 2023-10-08T05:55:26.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.130+0000 7fae23ef2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c15f7a0 con 0x7fae1c0c90b0 2023-10-08T05:55:26.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.130+0000 7fae21c8e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae1c15ab40 0x7fae1c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:26.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.131+0000 7fae21c8e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae1c15ab40 0x7fae1c0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54784/0 (socket says 172.21.15.70:54784) 2023-10-08T05:55:26.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.131+0000 7fae21c8e640 1 -- 172.21.15.70:0/979545851 learned_addr learned my addr 172.21.15.70:0/979545851 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:26.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.131+0000 7fae2248f640 1 --1- 172.21.15.70:0/979545851 >> v1:172.21.15.70:6789/0 conn(0x7fae1c0cae70 0x7fae1c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54762/0 (socket says 172.21.15.70:54762) 2023-10-08T05:55:26.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.131+0000 7fae2148d640 1 --2- 172.21.15.70:0/979545851 >> v2:172.21.15.181:3300/0 conn(0x7fae1c0c90b0 0x7fae1c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:26.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.131+0000 7fae21c8e640 1 -- 172.21.15.70:0/979545851 >> v2:172.21.15.181:3300/0 conn(0x7fae1c0c90b0 msgr2=0x7fae1c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:26.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.131+0000 7fae21c8e640 1 --2- 172.21.15.70:0/979545851 >> v2:172.21.15.181:3300/0 conn(0x7fae1c0c90b0 0x7fae1c0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.131+0000 7fae21c8e640 1 -- 172.21.15.70:0/979545851 >> v1:172.21.15.70:6789/0 conn(0x7fae1c0cae70 legacy=0x7fae1c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.131+0000 7fae21c8e640 1 -- 172.21.15.70:0/979545851 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c15b590 con 0x7fae1c15ab40 2023-10-08T05:55:26.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.131+0000 7fae20c8c640 1 -- 172.21.15.70:0/979545851 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451989049 0 0) 0x7fae1c15b2b0 con 0x7fae1c0cae70 2023-10-08T05:55:26.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.132+0000 7fae21c8e640 1 --2- 172.21.15.70:0/979545851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 0x7fae1c0ad720 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fae1c06e560 tx=0x7fae1802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0e1c27607d085b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:26.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.132+0000 7fae20c8c640 1 -- 172.21.15.70:0/979545851 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18030860 con 0x7fae1c15ab40 2023-10-08T05:55:26.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.132+0000 7fae20c8c640 1 -- 172.21.15.70:0/979545851 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae180309c0 con 0x7fae1c15ab40 2023-10-08T05:55:26.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.132+0000 7fae20c8c640 1 -- 172.21.15.70:0/979545851 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18030d50 con 0x7fae1c15ab40 2023-10-08T05:55:26.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.133+0000 7fae23ef2640 1 -- 172.21.15.70:0/979545851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 msgr2=0x7fae1c0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.133+0000 7fae23ef2640 1 --2- 172.21.15.70:0/979545851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 0x7fae1c0ad720 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fae1c06e560 tx=0x7fae1802b9a0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.133+0000 7fae23ef2640 1 -- 172.21.15.70:0/979545851 shutdown_connections 2023-10-08T05:55:26.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.133+0000 7fae23ef2640 1 --2- 172.21.15.70:0/979545851 >> v2:172.21.15.181:3300/0 conn(0x7fae1c0c90b0 0x7fae1c0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.133+0000 7fae23ef2640 1 --2- 172.21.15.70:0/979545851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 0x7fae1c0ad720 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.133+0000 7fae23ef2640 1 -- 172.21.15.70:0/979545851 >> 172.21.15.70:0/979545851 conn(0x7fae1c06a380 msgr2=0x7fae1c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:26.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.133+0000 7fae23ef2640 1 -- 172.21.15.70:0/979545851 shutdown_connections 2023-10-08T05:55:26.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.133+0000 7fae23ef2640 1 -- 172.21.15.70:0/979545851 wait complete. 2023-10-08T05:55:26.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.135+0000 7fae23ef2640 1 Processor -- start 2023-10-08T05:55:26.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.135+0000 7fae23ef2640 1 -- start start 2023-10-08T05:55:26.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.136+0000 7fae23ef2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae1c0c90b0 0x7fae1c154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.136+0000 7fae23ef2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae1c0cae70 0x7fae1c153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.136+0000 7fae23ef2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 0x7fae1c151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.136+0000 7fae23ef2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1533a0 con 0x7fae1c15ab40 2023-10-08T05:55:26.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.136+0000 7fae23ef2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0c80d0 con 0x7fae1c0c90b0 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.136+0000 7fae23ef2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c152460 con 0x7fae1c0cae70 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.136+0000 7fae2248f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 0x7fae1c151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.136+0000 7fae2248f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 0x7fae1c151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54786/0 (socket says 172.21.15.70:54786) 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.136+0000 7fae2248f640 1 -- 172.21.15.70:0/3948046395 learned_addr learned my addr 172.21.15.70:0/3948046395 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae21c8e640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae1c0c90b0 0x7fae1c154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae2248f640 1 -- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae1c0cae70 msgr2=0x7fae1c153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae2248f640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae1c0cae70 0x7fae1c153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae2248f640 1 -- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae1c0c90b0 msgr2=0x7fae1c154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae2248f640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae1c0c90b0 0x7fae1c154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae2248f640 1 -- 172.21.15.70:0/3948046395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180094d0 con 0x7fae1c15ab40 2023-10-08T05:55:26.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae21c8e640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae1c0c90b0 0x7fae1c154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:26.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae2248f640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 0x7fae1c151f20 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fae1400c7a0 tx=0x7fae1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:26.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae127fc640 1 -- 172.21.15.70:0/3948046395 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1400a640 con 0x7fae1c15ab40 2023-10-08T05:55:26.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae127fc640 1 -- 172.21.15.70:0/3948046395 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1400a7a0 con 0x7fae1c15ab40 2023-10-08T05:55:26.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae127fc640 1 -- 172.21.15.70:0/3948046395 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1400abd0 con 0x7fae1c15ab40 2023-10-08T05:55:26.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.137+0000 7fae23ef2640 1 -- 172.21.15.70:0/3948046395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c0c7df0 con 0x7fae1c15ab40 2023-10-08T05:55:26.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.138+0000 7fae23ef2640 1 -- 172.21.15.70:0/3948046395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c153030 con 0x7fae1c15ab40 2023-10-08T05:55:26.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.139+0000 7fae23ef2640 1 -- 172.21.15.70:0/3948046395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c153550 con 0x7fae1c15ab40 2023-10-08T05:55:26.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.140+0000 7fae127fc640 1 -- 172.21.15.70:0/3948046395 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fae1400ad30 con 0x7fae1c15ab40 2023-10-08T05:55:26.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.141+0000 7fae127fc640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fae0006afc0 0x7fae0006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.141+0000 7fae127fc640 1 -- 172.21.15.70:0/3948046395 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fae14088de0 con 0x7fae1c15ab40 2023-10-08T05:55:26.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.141+0000 7fae21c8e640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fae0006afc0 0x7fae0006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:26.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.143+0000 7fae21c8e640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fae0006afc0 0x7fae0006d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae18036000 tx=0x7fae1803b040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:26.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.145+0000 7fae127fc640 1 -- 172.21.15.70:0/3948046395 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae1c153550 con 0x7fae1c15ab40 2023-10-08T05:55:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.325+0000 7fae23ef2640 1 -- 172.21.15.70:0/3948046395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae1c067490 con 0x7fae1c15ab40 2023-10-08T05:55:26.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.328+0000 7fae127fc640 1 -- 172.21.15.70:0/3948046395 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16787 (secure 0 0 0) 0x7fae14018020 con 0x7fae1c15ab40 2023-10-08T05:55:26.330 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:26.330 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":133,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:25.362334+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:55:23.150004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-08T05:55:24.358581+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:55:26.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.330+0000 7fae23ef2640 1 -- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fae0006afc0 msgr2=0x7fae0006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.330+0000 7fae23ef2640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fae0006afc0 0x7fae0006d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae18036000 tx=0x7fae1803b040 comp rx=0 tx=0).stop 2023-10-08T05:55:26.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.330+0000 7fae23ef2640 1 -- 172.21.15.70:0/3948046395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 msgr2=0x7fae1c151f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.330+0000 7fae23ef2640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 0x7fae1c151f20 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fae1400c7a0 tx=0x7fae1400cc60 comp rx=0 tx=0).stop 2023-10-08T05:55:26.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.331+0000 7fae23ef2640 1 -- 172.21.15.70:0/3948046395 shutdown_connections 2023-10-08T05:55:26.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.331+0000 7fae23ef2640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fae0006afc0 0x7fae0006d480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.331+0000 7fae23ef2640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae1c15ab40 0x7fae1c151f20 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.331+0000 7fae23ef2640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae1c0cae70 0x7fae1c153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.331+0000 7fae23ef2640 1 --2- 172.21.15.70:0/3948046395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae1c0c90b0 0x7fae1c154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.331+0000 7fae23ef2640 1 -- 172.21.15.70:0/3948046395 >> 172.21.15.70:0/3948046395 conn(0x7fae1c06a380 msgr2=0x7fae1c155100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:26.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.331+0000 7fae23ef2640 1 -- 172.21.15.70:0/3948046395 shutdown_connections 2023-10-08T05:55:26.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.331+0000 7fae23ef2640 1 -- 172.21.15.70:0/3948046395 wait complete. 2023-10-08T05:55:26.349 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:55:26.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.559+0000 7f16c533c640 1 Processor -- start 2023-10-08T05:55:26.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.559+0000 7f16c533c640 1 -- start start 2023-10-08T05:55:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.560+0000 7f16c533c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f16c015ab30 0x7f16c015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.560+0000 7f16c533c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16c00b07d0 0x7f16c006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.560+0000 7f16c533c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c015f7d0 con 0x7f16c00adc70 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.560+0000 7f16c533c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c015f210 con 0x7f16c00b07d0 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.560+0000 7f16c533c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c015f4f0 con 0x7f16c015ab30 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.561+0000 7f16c4b3b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16c00b07d0 0x7f16c006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.561+0000 7f16b7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f16c00adc70 0x7f16c00b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54768/0 (socket says 172.21.15.70:54768) 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.561+0000 7f16b7fff640 1 -- 172.21.15.70:0/844971978 learned_addr learned my addr 172.21.15.70:0/844971978 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.561+0000 7f16bffff640 1 --2- 172.21.15.70:0/844971978 >> v2:172.21.15.181:3300/0 conn(0x7f16c015ab30 0x7f16c015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.561+0000 7f16c4b3b640 1 -- 172.21.15.70:0/844971978 >> v2:172.21.15.181:3300/0 conn(0x7f16c015ab30 msgr2=0x7f16c015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.561+0000 7f16c4b3b640 1 --2- 172.21.15.70:0/844971978 >> v2:172.21.15.181:3300/0 conn(0x7f16c015ab30 0x7f16c015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.561+0000 7f16c4b3b640 1 -- 172.21.15.70:0/844971978 >> v1:172.21.15.70:6789/0 conn(0x7f16c00adc70 legacy=0x7f16c00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.561+0000 7f16c4b3b640 1 -- 172.21.15.70:0/844971978 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c006c550 con 0x7f16c00b07d0 2023-10-08T05:55:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.561+0000 7f16bffff640 1 --2- 172.21.15.70:0/844971978 >> v2:172.21.15.181:3300/0 conn(0x7f16c015ab30 0x7f16c015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.561+0000 7f16bf7fe640 1 -- 172.21.15.70:0/844971978 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496994798 0 0) 0x7f16c015f7d0 con 0x7f16c00adc70 2023-10-08T05:55:26.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.562+0000 7f16c4b3b640 1 --2- 172.21.15.70:0/844971978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00b07d0 0x7f16c006cd50 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f16c006ada0 tx=0x7f16b802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133d37ec63508540 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:26.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16bf7fe640 1 -- 172.21.15.70:0/844971978 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b802cc70 con 0x7f16c00b07d0 2023-10-08T05:55:26.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16bf7fe640 1 -- 172.21.15.70:0/844971978 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b802cdd0 con 0x7f16c00b07d0 2023-10-08T05:55:26.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16bf7fe640 1 -- 172.21.15.70:0/844971978 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8034660 con 0x7f16c00b07d0 2023-10-08T05:55:26.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16c533c640 1 -- 172.21.15.70:0/844971978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00b07d0 msgr2=0x7f16c006cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16c533c640 1 --2- 172.21.15.70:0/844971978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00b07d0 0x7f16c006cd50 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f16c006ada0 tx=0x7f16b802bec0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16c533c640 1 -- 172.21.15.70:0/844971978 shutdown_connections 2023-10-08T05:55:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16c533c640 1 --2- 172.21.15.70:0/844971978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00b07d0 0x7f16c006cd50 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16c533c640 1 --2- 172.21.15.70:0/844971978 >> v2:172.21.15.181:3300/0 conn(0x7f16c015ab30 0x7f16c015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16c533c640 1 -- 172.21.15.70:0/844971978 >> 172.21.15.70:0/844971978 conn(0x7f16c00cb0c0 msgr2=0x7f16c00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16c533c640 1 -- 172.21.15.70:0/844971978 shutdown_connections 2023-10-08T05:55:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.563+0000 7f16c533c640 1 -- 172.21.15.70:0/844971978 wait complete. 2023-10-08T05:55:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.565+0000 7f16c533c640 1 Processor -- start 2023-10-08T05:55:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.565+0000 7f16c533c640 1 -- start start 2023-10-08T05:55:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.566+0000 7f16c533c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c00adc70 0x7f16c00acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.566+0000 7f16c533c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c00b07d0 0x7f16c00ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.566+0000 7f16c533c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015ab30 0x7f16c01f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.566+0000 7f16c533c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00ac850 con 0x7f16c015ab30 2023-10-08T05:55:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.566+0000 7f16c533c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00abc40 con 0x7f16c00adc70 2023-10-08T05:55:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.566+0000 7f16c533c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00abf20 con 0x7f16c00b07d0 2023-10-08T05:55:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.566+0000 7f16c4b3b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015ab30 0x7f16c01f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.566+0000 7f16c4b3b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015ab30 0x7f16c01f6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54810/0 (socket says 172.21.15.70:54810) 2023-10-08T05:55:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.566+0000 7f16c4b3b640 1 -- 172.21.15.70:0/1990414993 learned_addr learned my addr 172.21.15.70:0/1990414993 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.567+0000 7f16c4b3b640 1 -- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c00b07d0 msgr2=0x7f16c00ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.567+0000 7f16c4b3b640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c00b07d0 0x7f16c00ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.567+0000 7f16c4b3b640 1 -- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c00adc70 msgr2=0x7f16c00acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.567+0000 7f16c4b3b640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c00adc70 0x7f16c00acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.567+0000 7f16c4b3b640 1 -- 172.21.15.70:0/1990414993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8009a70 con 0x7f16c015ab30 2023-10-08T05:55:26.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.567+0000 7f16c4b3b640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015ab30 0x7f16c01f6190 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009e80 tx=0x7f16b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:26.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.567+0000 7f16bd7fa640 1 -- 172.21.15.70:0/1990414993 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8040650 con 0x7f16c015ab30 2023-10-08T05:55:26.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.567+0000 7f16bd7fa640 1 -- 172.21.15.70:0/1990414993 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b80407b0 con 0x7f16c015ab30 2023-10-08T05:55:26.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.567+0000 7f16bd7fa640 1 -- 172.21.15.70:0/1990414993 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8040b60 con 0x7f16c015ab30 2023-10-08T05:55:26.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.567+0000 7f16c533c640 1 -- 172.21.15.70:0/1990414993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c00ac200 con 0x7f16c015ab30 2023-10-08T05:55:26.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.568+0000 7f16c533c640 1 -- 172.21.15.70:0/1990414993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c00ad220 con 0x7f16c015ab30 2023-10-08T05:55:26.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.568+0000 7f16c533c640 1 -- 172.21.15.70:0/1990414993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c004e160 con 0x7f16c015ab30 2023-10-08T05:55:26.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.569+0000 7f16bd7fa640 1 -- 172.21.15.70:0/1990414993 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f16b8040cc0 con 0x7f16c015ab30 2023-10-08T05:55:26.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.570+0000 7f16bd7fa640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f16a006afc0 0x7f16a006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:26.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.570+0000 7f16bd7fa640 1 -- 172.21.15.70:0/1990414993 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f16b80ac7e0 con 0x7f16c015ab30 2023-10-08T05:55:26.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.571+0000 7f16bffff640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f16a006afc0 0x7f16a006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:26.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.572+0000 7f16bffff640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f16a006afc0 0x7f16a006d480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16b0006fd0 tx=0x7f16b0008040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:26.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.575+0000 7f16bd7fa640 1 -- 172.21.15.70:0/1990414993 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f16b80775f0 con 0x7f16c015ab30 2023-10-08T05:55:26.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.768+0000 7f16c533c640 1 -- 172.21.15.70:0/1990414993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16c00ac990 con 0x7f16c015ab30 2023-10-08T05:55:26.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.771+0000 7f16bd7fa640 1 -- 172.21.15.70:0/1990414993 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+16787 (secure 0 0 0) 0x7f16b807ac10 con 0x7f16c015ab30 2023-10-08T05:55:26.773 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:26.773 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":133,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:25.362334+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:55:23.150004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-08T05:55:24.358581+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:55:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 -- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f16a006afc0 msgr2=0x7f16a006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f16a006afc0 0x7f16a006d480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16b0006fd0 tx=0x7f16b0008040 comp rx=0 tx=0).stop 2023-10-08T05:55:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 -- 172.21.15.70:0/1990414993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015ab30 msgr2=0x7f16c01f6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015ab30 0x7f16c01f6190 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009e80 tx=0x7f16b8004a40 comp rx=0 tx=0).stop 2023-10-08T05:55:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 -- 172.21.15.70:0/1990414993 shutdown_connections 2023-10-08T05:55:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f16a006afc0 0x7f16a006d480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015ab30 0x7f16c01f6190 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c00b07d0 0x7f16c00ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 --2- 172.21.15.70:0/1990414993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c00adc70 0x7f16c00acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:26.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 -- 172.21.15.70:0/1990414993 >> 172.21.15.70:0/1990414993 conn(0x7f16c00cb0c0 msgr2=0x7f16c00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:26.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 -- 172.21.15.70:0/1990414993 shutdown_connections 2023-10-08T05:55:26.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:26.774+0000 7f16c533c640 1 -- 172.21.15.70:0/1990414993 wait complete. 2023-10-08T05:55:26.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T05:55:27.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.013+0000 7f2acd8c2640 1 Processor -- start 2023-10-08T05:55:27.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.013+0000 7f2acd8c2640 1 -- start start 2023-10-08T05:55:27.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.015+0000 7f2acd8c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80702b0 0x7f2ac8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:27.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.015+0000 7f2acd8c2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2ac80b21f0 0x7f2ac80c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:27.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.015+0000 7f2acd8c2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac806c3a0 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.015+0000 7f2acd8c2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac8070bd0 con 0x7f2ac80b21f0 2023-10-08T05:55:27.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.015+0000 7f2acd8c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac806c640 con 0x7f2ac80702b0 2023-10-08T05:55:27.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.016+0000 7f2ac67fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2ac80b21f0 0x7f2ac80c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:27.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.016+0000 7f2ac6ffd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80702b0 0x7f2ac8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:27.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.016+0000 7f2ac67fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2ac80b21f0 0x7f2ac80c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:35898/0 (socket says 172.21.15.70:35898) 2023-10-08T05:55:27.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.016+0000 7f2ac67fc640 1 -- 172.21.15.70:0/2140001153 learned_addr learned my addr 172.21.15.70:0/2140001153 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:27.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.016+0000 7f2ac6ffd640 1 -- 172.21.15.70:0/2140001153 >> v2:172.21.15.181:3300/0 conn(0x7f2ac80b21f0 msgr2=0x7f2ac80c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:27.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.016+0000 7f2ac6ffd640 1 --2- 172.21.15.70:0/2140001153 >> v2:172.21.15.181:3300/0 conn(0x7f2ac80b21f0 0x7f2ac80c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:27.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.016+0000 7f2ac6ffd640 1 -- 172.21.15.70:0/2140001153 >> v1:172.21.15.181:6789/0 conn(0x7f2ac80ca1a0 legacy=0x7f2ac806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:27.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.016+0000 7f2ac6ffd640 1 -- 172.21.15.70:0/2140001153 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac806c920 con 0x7f2ac80702b0 2023-10-08T05:55:27.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.018+0000 7f2ac6ffd640 1 --2- 172.21.15.70:0/2140001153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80702b0 0x7f2ac8070690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009d90 tx=0x7f2ab802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70df8a76047a2422 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:27.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.018+0000 7f2ac5ffb640 1 -- 172.21.15.70:0/2140001153 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab802cc50 con 0x7f2ac80702b0 2023-10-08T05:55:27.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.018+0000 7f2ac5ffb640 1 -- 172.21.15.70:0/2140001153 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab802cdb0 con 0x7f2ac80702b0 2023-10-08T05:55:27.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.018+0000 7f2ac5ffb640 1 -- 172.21.15.70:0/2140001153 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8034660 con 0x7f2ac80702b0 2023-10-08T05:55:27.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.019+0000 7f2acd8c2640 1 -- 172.21.15.70:0/2140001153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80702b0 msgr2=0x7f2ac8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:27.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.019+0000 7f2acd8c2640 1 --2- 172.21.15.70:0/2140001153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80702b0 0x7f2ac8070690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8009d90 tx=0x7f2ab802bea0 comp rx=0 tx=0).stop 2023-10-08T05:55:27.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.019+0000 7f2acd8c2640 1 -- 172.21.15.70:0/2140001153 shutdown_connections 2023-10-08T05:55:27.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.019+0000 7f2acd8c2640 1 --2- 172.21.15.70:0/2140001153 >> v2:172.21.15.181:3300/0 conn(0x7f2ac80b21f0 0x7f2ac80c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:27.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.019+0000 7f2acd8c2640 1 --2- 172.21.15.70:0/2140001153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80702b0 0x7f2ac8070690 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:27.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.019+0000 7f2acd8c2640 1 -- 172.21.15.70:0/2140001153 >> 172.21.15.70:0/2140001153 conn(0x7f2ac806a5d0 msgr2=0x7f2ac80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:27.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.019+0000 7f2acd8c2640 1 -- 172.21.15.70:0/2140001153 shutdown_connections 2023-10-08T05:55:27.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.019+0000 7f2acd8c2640 1 -- 172.21.15.70:0/2140001153 wait complete. 2023-10-08T05:55:27.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.020+0000 7f2acd8c2640 1 Processor -- start 2023-10-08T05:55:27.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.021+0000 7f2acd8c2640 1 -- start start 2023-10-08T05:55:27.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.021+0000 7f2acd8c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ac80702b0 0x7f2ac80c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:27.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.021+0000 7f2acd8c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80b21f0 0x7f2ac8147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:27.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.021+0000 7f2acd8c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ac80ca1a0 0x7f2ac8147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.021+0000 7f2acd8c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac81e4720 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.021+0000 7f2acd8c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac8145870 con 0x7f2ac80702b0 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.021+0000 7f2acd8c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac8145b50 con 0x7f2ac80b21f0 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ac80ca1a0 0x7f2ac8147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ac80ca1a0 0x7f2ac8147da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54818/0 (socket says 172.21.15.70:54818) 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac77fe640 1 -- 172.21.15.70:0/4045699303 learned_addr learned my addr 172.21.15.70:0/4045699303 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac6ffd640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ac80702b0 0x7f2ac80c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac67fc640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80b21f0 0x7f2ac8147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac77fe640 1 -- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80b21f0 msgr2=0x7f2ac8147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac77fe640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80b21f0 0x7f2ac8147860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:27.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac77fe640 1 -- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ac80702b0 msgr2=0x7f2ac80c43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:27.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac77fe640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ac80702b0 0x7f2ac80c43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:27.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac77fe640 1 -- 172.21.15.70:0/4045699303 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab8009a70 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac6ffd640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ac80702b0 0x7f2ac80c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:27.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.022+0000 7f2ac77fe640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ac80ca1a0 0x7f2ac8147da0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c390 tx=0x7f2ac000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:27.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.023+0000 7f2abffff640 1 -- 172.21.15.70:0/4045699303 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00052d0 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.023+0000 7f2abffff640 1 -- 172.21.15.70:0/4045699303 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac0005430 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.023+0000 7f2acd8c2640 1 -- 172.21.15.70:0/4045699303 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac8145e30 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.023+0000 7f2abffff640 1 -- 172.21.15.70:0/4045699303 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0003810 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.023+0000 7f2acd8c2640 1 -- 172.21.15.70:0/4045699303 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac815a180 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.024+0000 7f2acd8c2640 1 -- 172.21.15.70:0/4045699303 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac8146350 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.025+0000 7f2abffff640 1 -- 172.21.15.70:0/4045699303 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2ac0003970 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.026+0000 7f2abffff640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2aa806af90 0x7f2aa806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:27.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.026+0000 7f2abffff640 1 -- 172.21.15.70:0/4045699303 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2ac0088d90 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.027+0000 7f2ac6ffd640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2aa806af90 0x7f2aa806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:27.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.028+0000 7f2ac6ffd640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2aa806af90 0x7f2aa806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8036000 tx=0x7f2ab8030000 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:27.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.031+0000 7f2abffff640 1 -- 172.21.15.70:0/4045699303 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ac8146350 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.239+0000 7f2acd8c2640 1 -- 172.21.15.70:0/4045699303 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2ac8146350 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.924+0000 7f2abffff640 1 -- 172.21.15.70:0/4045699303 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2ac8146350 con 0x7f2ac80ca1a0 2023-10-08T05:55:27.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.928+0000 7f2acd8c2640 1 -- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2aa806af90 msgr2=0x7f2aa806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:27.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.928+0000 7f2acd8c2640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2aa806af90 0x7f2aa806d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8036000 tx=0x7f2ab8030000 comp rx=0 tx=0).stop 2023-10-08T05:55:27.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.928+0000 7f2acd8c2640 1 -- 172.21.15.70:0/4045699303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ac80ca1a0 msgr2=0x7f2ac8147da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:27.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.928+0000 7f2acd8c2640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ac80ca1a0 0x7f2ac8147da0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c390 tx=0x7f2ac000c850 comp rx=0 tx=0).stop 2023-10-08T05:55:27.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.928+0000 7f2acd8c2640 1 -- 172.21.15.70:0/4045699303 shutdown_connections 2023-10-08T05:55:27.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.928+0000 7f2acd8c2640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2aa806af90 0x7f2aa806d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:27.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.928+0000 7f2acd8c2640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ac80ca1a0 0x7f2ac8147da0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:27.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.928+0000 7f2acd8c2640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ac80b21f0 0x7f2ac8147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:27.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.928+0000 7f2acd8c2640 1 --2- 172.21.15.70:0/4045699303 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ac80702b0 0x7f2ac80c43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:27.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.928+0000 7f2acd8c2640 1 -- 172.21.15.70:0/4045699303 >> 172.21.15.70:0/4045699303 conn(0x7f2ac806a5d0 msgr2=0x7f2ac80cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:27.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.929+0000 7f2acd8c2640 1 -- 172.21.15.70:0/4045699303 shutdown_connections 2023-10-08T05:55:27.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:27.929+0000 7f2acd8c2640 1 -- 172.21.15.70:0/4045699303 wait complete. 2023-10-08T05:55:27.950 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T05:55:27.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T05:55:28.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.167+0000 7f3d8e859640 1 Processor -- start 2023-10-08T05:55:28.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.167+0000 7f3d8e859640 1 -- start start 2023-10-08T05:55:28.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.168+0000 7f3d8e859640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d881585a0 0x7f3d8815e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:28.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.168+0000 7f3d8e859640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d8815b4b0 0x7f3d8815b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:28.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.168+0000 7f3d8e859640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d880b63e0 con 0x7f3d880cb6f0 2023-10-08T05:55:28.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.168+0000 7f3d8e859640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8815be00 con 0x7f3d881585a0 2023-10-08T05:55:28.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.168+0000 7f3d8e859640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d880cbba0 con 0x7f3d8815b4b0 2023-10-08T05:55:28.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.168+0000 7f3d8d857640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d881585a0 0x7f3d8815e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:28.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.169+0000 7f3d8d857640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d881585a0 0x7f3d8815e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54834/0 (socket says 172.21.15.70:54834) 2023-10-08T05:55:28.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.169+0000 7f3d8d857640 1 -- 172.21.15.70:0/1791464119 learned_addr learned my addr 172.21.15.70:0/1791464119 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:28.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.169+0000 7f3d8e058640 1 --2- 172.21.15.70:0/1791464119 >> v2:172.21.15.181:3300/0 conn(0x7f3d8815b4b0 0x7f3d8815b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:28.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.169+0000 7f3d8d857640 1 -- 172.21.15.70:0/1791464119 >> v2:172.21.15.181:3300/0 conn(0x7f3d8815b4b0 msgr2=0x7f3d8815b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:28.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.169+0000 7f3d8d857640 1 --2- 172.21.15.70:0/1791464119 >> v2:172.21.15.181:3300/0 conn(0x7f3d8815b4b0 0x7f3d8815b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:28.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.169+0000 7f3d8d857640 1 -- 172.21.15.70:0/1791464119 >> v1:172.21.15.181:6789/0 conn(0x7f3d880cb6f0 legacy=0x7f3d880ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:28.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.169+0000 7f3d8d857640 1 -- 172.21.15.70:0/1791464119 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8815b020 con 0x7f3d881585a0 2023-10-08T05:55:28.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.171+0000 7f3d8d857640 1 --2- 172.21.15.70:0/1791464119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 0x7f3d8815e400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3d880afb90 tx=0x7f3d8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da28db0df8040de server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:28.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.171+0000 7f3d8c855640 1 -- 172.21.15.70:0/1791464119 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84031870 con 0x7f3d881585a0 2023-10-08T05:55:28.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.171+0000 7f3d8c855640 1 -- 172.21.15.70:0/1791464119 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d840319d0 con 0x7f3d881585a0 2023-10-08T05:55:28.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.171+0000 7f3d8c855640 1 -- 172.21.15.70:0/1791464119 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84031d60 con 0x7f3d881585a0 2023-10-08T05:55:28.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.172+0000 7f3d8e859640 1 -- 172.21.15.70:0/1791464119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 msgr2=0x7f3d8815e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:28.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.172+0000 7f3d8e859640 1 --2- 172.21.15.70:0/1791464119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 0x7f3d8815e400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f3d880afb90 tx=0x7f3d8402b910 comp rx=0 tx=0).stop 2023-10-08T05:55:28.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.172+0000 7f3d8e859640 1 -- 172.21.15.70:0/1791464119 shutdown_connections 2023-10-08T05:55:28.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.172+0000 7f3d8e859640 1 --2- 172.21.15.70:0/1791464119 >> v2:172.21.15.181:3300/0 conn(0x7f3d8815b4b0 0x7f3d8815b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:28.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.172+0000 7f3d8e859640 1 --2- 172.21.15.70:0/1791464119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 0x7f3d8815e400 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:28.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.172+0000 7f3d8e859640 1 -- 172.21.15.70:0/1791464119 >> 172.21.15.70:0/1791464119 conn(0x7f3d880cd180 msgr2=0x7f3d880cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:28.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.172+0000 7f3d8e859640 1 -- 172.21.15.70:0/1791464119 shutdown_connections 2023-10-08T05:55:28.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.172+0000 7f3d8e859640 1 -- 172.21.15.70:0/1791464119 wait complete. 2023-10-08T05:55:28.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.174+0000 7f3d8e859640 1 Processor -- start 2023-10-08T05:55:28.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.174+0000 7f3d8e859640 1 -- start start 2023-10-08T05:55:28.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.175+0000 7f3d8e859640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d880cb6f0 0x7f3d88150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:28.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.175+0000 7f3d8e859640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 0x7f3d880cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:28.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.175+0000 7f3d8e859640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d8815b4b0 0x7f3d880c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:28.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.175+0000 7f3d8e859640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880c5200 con 0x7f3d881585a0 2023-10-08T05:55:28.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.175+0000 7f3d8e859640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880ce470 con 0x7f3d880cb6f0 2023-10-08T05:55:28.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.175+0000 7f3d8e859640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880ce750 con 0x7f3d8815b4b0 2023-10-08T05:55:28.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.175+0000 7f3d8d056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 0x7f3d880cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:28.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.175+0000 7f3d8d056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 0x7f3d880cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54838/0 (socket says 172.21.15.70:54838) 2023-10-08T05:55:28.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.175+0000 7f3d8d056640 1 -- 172.21.15.70:0/3277441881 learned_addr learned my addr 172.21.15.70:0/3277441881 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:28.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d8d857640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d880cb6f0 0x7f3d88150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:28.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d8d056640 1 -- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d8815b4b0 msgr2=0x7f3d880c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:28.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d8d056640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d8815b4b0 0x7f3d880c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:28.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d8d056640 1 -- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d880cb6f0 msgr2=0x7f3d88150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:28.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d8d056640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d880cb6f0 0x7f3d88150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:28.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d8d056640 1 -- 172.21.15.70:0/3277441881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d840094d0 con 0x7f3d881585a0 2023-10-08T05:55:28.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d8d056640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 0x7f3d880cf510 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800c410 tx=0x7f3d7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d7e7fc640 1 -- 172.21.15.70:0/3277441881 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d780035f0 con 0x7f3d881585a0 2023-10-08T05:55:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d7e7fc640 1 -- 172.21.15.70:0/3277441881 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d78003750 con 0x7f3d881585a0 2023-10-08T05:55:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d7e7fc640 1 -- 172.21.15.70:0/3277441881 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78003b40 con 0x7f3d881585a0 2023-10-08T05:55:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d8e859640 1 -- 172.21.15.70:0/3277441881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d880cea30 con 0x7f3d881585a0 2023-10-08T05:55:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.176+0000 7f3d8e859640 1 -- 172.21.15.70:0/3277441881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d880ceff0 con 0x7f3d881585a0 2023-10-08T05:55:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.178+0000 7f3d8e859640 1 -- 172.21.15.70:0/3277441881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d88074150 con 0x7f3d881585a0 2023-10-08T05:55:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.179+0000 7f3d7e7fc640 1 -- 172.21.15.70:0/3277441881 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3d78003ca0 con 0x7f3d881585a0 2023-10-08T05:55:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.180+0000 7f3d7e7fc640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f3d6c06af90 0x7f3d6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.180+0000 7f3d7e7fc640 1 -- 172.21.15.70:0/3277441881 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3d78088b40 con 0x7f3d881585a0 2023-10-08T05:55:28.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.180+0000 7f3d8d857640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f3d6c06af90 0x7f3d6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:28.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.181+0000 7f3d8d857640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f3d6c06af90 0x7f3d6c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d84036000 tx=0x7f3d8403c040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:28.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.184+0000 7f3d7e7fc640 1 -- 172.21.15.70:0/3277441881 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3d78053970 con 0x7f3d881585a0 2023-10-08T05:55:28.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.412+0000 7f3d8e859640 1 -- 172.21.15.70:0/3277441881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3d880cf1b0 con 0x7f3d881585a0 2023-10-08T05:55:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.942+0000 7f3d7e7fc640 1 -- 172.21.15.70:0/3277441881 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3d78056f90 con 0x7f3d881585a0 2023-10-08T05:55:28.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 -- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f3d6c06af90 msgr2=0x7f3d6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f3d6c06af90 0x7f3d6c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d84036000 tx=0x7f3d8403c040 comp rx=0 tx=0).stop 2023-10-08T05:55:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 -- 172.21.15.70:0/3277441881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 msgr2=0x7f3d880cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 0x7f3d880cf510 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800c410 tx=0x7f3d7800c8d0 comp rx=0 tx=0).stop 2023-10-08T05:55:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 -- 172.21.15.70:0/3277441881 shutdown_connections 2023-10-08T05:55:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d8815b4b0 0x7f3d880c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f3d6c06af90 0x7f3d6c06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d881585a0 0x7f3d880cf510 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 --2- 172.21.15.70:0/3277441881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d880cb6f0 0x7f3d88150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 -- 172.21.15.70:0/3277441881 >> 172.21.15.70:0/3277441881 conn(0x7f3d880cd180 msgr2=0x7f3d8806ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 -- 172.21.15.70:0/3277441881 shutdown_connections 2023-10-08T05:55:28.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:28.946+0000 7f3d8e859640 1 -- 172.21.15.70:0/3277441881 wait complete. 2023-10-08T05:55:28.967 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T05:55:28.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:55:29.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.174+0000 7fbf2ffff640 1 Processor -- start 2023-10-08T05:55:29.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.174+0000 7fbf2ffff640 1 -- start start 2023-10-08T05:55:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf2ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf300cc210 0x7fbf300b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf2ffff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf300cacf0 con 0x7fbf30159db0 2023-10-08T05:55:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf2ffff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf300ca3c0 con 0x7fbf301599d0 2023-10-08T05:55:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf2ffff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf300b10f0 con 0x7fbf300cc210 2023-10-08T05:55:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf27fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf300cc210 0x7fbf300b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf27fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf300cc210 0x7fbf300b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54854/0 (socket says 172.21.15.70:54854) 2023-10-08T05:55:29.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf27fff640 1 -- 172.21.15.70:0/1506960703 learned_addr learned my addr 172.21.15.70:0/1506960703 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:29.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf2f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbf30159db0 0x7fbf3006c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54782/0 (socket says 172.21.15.70:54782) 2023-10-08T05:55:29.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf27fff640 1 -- 172.21.15.70:0/1506960703 >> v1:172.21.15.181:6789/0 conn(0x7fbf301599d0 legacy=0x7fbf300ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:29.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf27fff640 1 -- 172.21.15.70:0/1506960703 >> v1:172.21.15.70:6789/0 conn(0x7fbf30159db0 legacy=0x7fbf3006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf27fff640 1 -- 172.21.15.70:0/1506960703 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300ca6a0 con 0x7fbf300cc210 2023-10-08T05:55:29.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.176+0000 7fbf2e7fc640 1 -- 172.21.15.70:0/1506960703 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003866432 0 0) 0x7fbf300cacf0 con 0x7fbf30159db0 2023-10-08T05:55:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.178+0000 7fbf27fff640 1 --2- 172.21.15.70:0/1506960703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf300cc210 0x7fbf300b00c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009870 tx=0x7fbf1802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6598f139a1a7d794 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.178+0000 7fbf2e7fc640 1 -- 172.21.15.70:0/1506960703 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18030850 con 0x7fbf300cc210 2023-10-08T05:55:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.178+0000 7fbf2e7fc640 1 -- 172.21.15.70:0/1506960703 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf180309b0 con 0x7fbf300cc210 2023-10-08T05:55:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.178+0000 7fbf2e7fc640 1 -- 172.21.15.70:0/1506960703 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18030d40 con 0x7fbf300cc210 2023-10-08T05:55:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.178+0000 7fbf2ffff640 1 -- 172.21.15.70:0/1506960703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf300cc210 msgr2=0x7fbf300b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.178+0000 7fbf2ffff640 1 --2- 172.21.15.70:0/1506960703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf300cc210 0x7fbf300b00c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009870 tx=0x7fbf1802b990 comp rx=0 tx=0).stop 2023-10-08T05:55:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.179+0000 7fbf2ffff640 1 -- 172.21.15.70:0/1506960703 shutdown_connections 2023-10-08T05:55:29.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.179+0000 7fbf2ffff640 1 --2- 172.21.15.70:0/1506960703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf300cc210 0x7fbf300b00c0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.179+0000 7fbf2ffff640 1 -- 172.21.15.70:0/1506960703 >> 172.21.15.70:0/1506960703 conn(0x7fbf300cb0c0 msgr2=0x7fbf300cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.179+0000 7fbf2ffff640 1 -- 172.21.15.70:0/1506960703 shutdown_connections 2023-10-08T05:55:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.179+0000 7fbf2ffff640 1 -- 172.21.15.70:0/1506960703 wait complete. 2023-10-08T05:55:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.180+0000 7fbf2ffff640 1 Processor -- start 2023-10-08T05:55:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.181+0000 7fbf2ffff640 1 -- start start 2023-10-08T05:55:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.181+0000 7fbf2ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf300cc210 0x7fbf3014f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.181+0000 7fbf2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf301599d0 0x7fbf301f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.181+0000 7fbf2ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf30159db0 0x7fbf301f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.181+0000 7fbf2ffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf301f6690 con 0x7fbf301599d0 2023-10-08T05:55:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf2ffff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3014fd90 con 0x7fbf300cc210 2023-10-08T05:55:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf2ffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300b33f0 con 0x7fbf30159db0 2023-10-08T05:55:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf301599d0 0x7fbf301f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf301599d0 0x7fbf301f5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54858/0 (socket says 172.21.15.70:54858) 2023-10-08T05:55:29.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf27fff640 1 -- 172.21.15.70:0/4229502446 learned_addr learned my addr 172.21.15.70:0/4229502446 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:29.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf2effd640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf300cc210 0x7fbf3014f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf2f7fe640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf30159db0 0x7fbf301f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf27fff640 1 -- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf30159db0 msgr2=0x7fbf301f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf27fff640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf30159db0 0x7fbf301f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf27fff640 1 -- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf300cc210 msgr2=0x7fbf3014f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf27fff640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf300cc210 0x7fbf3014f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf27fff640 1 -- 172.21.15.70:0/4229502446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180094d0 con 0x7fbf301599d0 2023-10-08T05:55:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.182+0000 7fbf2f7fe640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf30159db0 0x7fbf301f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.183+0000 7fbf27fff640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf301599d0 0x7fbf301f5810 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009870 tx=0x7fbf18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.183+0000 7fbf277fe640 1 -- 172.21.15.70:0/4229502446 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18030800 con 0x7fbf301599d0 2023-10-08T05:55:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.183+0000 7fbf277fe640 1 -- 172.21.15.70:0/4229502446 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf18039d90 con 0x7fbf301599d0 2023-10-08T05:55:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.183+0000 7fbf277fe640 1 -- 172.21.15.70:0/4229502446 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18043630 con 0x7fbf301599d0 2023-10-08T05:55:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.183+0000 7fbf2ffff640 1 -- 172.21.15.70:0/4229502446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf300b2b50 con 0x7fbf301599d0 2023-10-08T05:55:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.183+0000 7fbf2ffff640 1 -- 172.21.15.70:0/4229502446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf300b3110 con 0x7fbf301599d0 2023-10-08T05:55:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.184+0000 7fbf2ffff640 1 -- 172.21.15.70:0/4229502446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3004e160 con 0x7fbf301599d0 2023-10-08T05:55:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.185+0000 7fbf277fe640 1 -- 172.21.15.70:0/4229502446 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbf1802e070 con 0x7fbf301599d0 2023-10-08T05:55:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.186+0000 7fbf277fe640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbf1006af90 0x7fbf1006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.186+0000 7fbf277fe640 1 -- 172.21.15.70:0/4229502446 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbf180abba0 con 0x7fbf301599d0 2023-10-08T05:55:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.187+0000 7fbf2effd640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbf1006af90 0x7fbf1006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.188+0000 7fbf2effd640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbf1006af90 0x7fbf1006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbf30155c40 tx=0x7fbf20009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.191+0000 7fbf277fe640 1 -- 172.21.15.70:0/4229502446 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf180768b0 con 0x7fbf301599d0 2023-10-08T05:55:29.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.419+0000 7fbf2ffff640 1 -- 172.21.15.70:0/4229502446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf30070830 con 0x7fbf301599d0 2023-10-08T05:55:29.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.420+0000 7fbf277fe640 1 -- 172.21.15.70:0/4229502446 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fbf18034090 con 0x7fbf301599d0 2023-10-08T05:55:29.421 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:29.421 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3983030529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3983030529},{"type":"v1","addr":"172.21.15.181:6836","nonce":3983030529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5873,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1573936162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1573936162},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573936162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1979922916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1979922916},{"type":"v1","addr":"172.21.15.70:6835","nonce":1979922916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:55:25.363123+0000","modified":"2023-10-08T05:55:28.414414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3020884355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3020884355},{"type":"v1","addr":"172.21.15.70:6833","nonce":3020884355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-08T05:55:29.421 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 58 2023-10-08T05:55:29.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 -- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbf1006af90 msgr2=0x7fbf1006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbf1006af90 0x7fbf1006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbf30155c40 tx=0x7fbf20009040 comp rx=0 tx=0).stop 2023-10-08T05:55:29.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 -- 172.21.15.70:0/4229502446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf301599d0 msgr2=0x7fbf301f5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf301599d0 0x7fbf301f5810 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009870 tx=0x7fbf18004730 comp rx=0 tx=0).stop 2023-10-08T05:55:29.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 -- 172.21.15.70:0/4229502446 shutdown_connections 2023-10-08T05:55:29.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf30159db0 0x7fbf301f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbf1006af90 0x7fbf1006d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf301599d0 0x7fbf301f5810 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 --2- 172.21.15.70:0/4229502446 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf300cc210 0x7fbf3014f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 -- 172.21.15.70:0/4229502446 >> 172.21.15.70:0/4229502446 conn(0x7fbf300cb0c0 msgr2=0x7fbf300cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:29.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 -- 172.21.15.70:0/4229502446 shutdown_connections 2023-10-08T05:55:29.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.423+0000 7fbf2ffff640 1 -- 172.21.15.70:0/4229502446 wait complete. 2023-10-08T05:55:29.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:55:29.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.650+0000 7fd0034d7640 1 Processor -- start 2023-10-08T05:55:29.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.650+0000 7fd0034d7640 1 -- start start 2023-10-08T05:55:29.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.652+0000 7fd0034d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcffc0c90b0 0x7fcffc0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.652+0000 7fd0034d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cb3b0 0x7fcffc15e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.652+0000 7fd0034d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcffc15b2b0 con 0x7fcffc15ab40 2023-10-08T05:55:29.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.652+0000 7fd0034d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcffc15bf30 con 0x7fcffc0c90b0 2023-10-08T05:55:29.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.652+0000 7fd0034d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0cb820 con 0x7fcffc0cb3b0 2023-10-08T05:55:29.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.652+0000 7fd001273640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcffc15ab40 0x7fcffc0ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54784/0 (socket says 172.21.15.70:54784) 2023-10-08T05:55:29.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.652+0000 7fd001273640 1 -- 172.21.15.70:0/4057090816 learned_addr learned my addr 172.21.15.70:0/4057090816 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:29.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.652+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754214883 0 0) 0x7fcffc15b2b0 con 0x7fcffc15ab40 2023-10-08T05:55:29.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.652+0000 7fd000a72640 1 --2- 172.21.15.70:0/4057090816 >> v2:172.21.15.181:3300/0 conn(0x7fcffc0c90b0 0x7fcffc0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe8003520 con 0x7fcffc15ab40 2023-10-08T05:55:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fd001a74640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cb3b0 0x7fcffc15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694040840 0 0) 0x7fcfe8003520 con 0x7fcffc15ab40 2023-10-08T05:55:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cb3b0 msgr2=0x7fcffc15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cb3b0 0x7fcffc15e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 >> v2:172.21.15.181:3300/0 conn(0x7fcffc0c90b0 msgr2=0x7fcffc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 --2- 172.21.15.70:0/4057090816 >> v2:172.21.15.181:3300/0 conn(0x7fcffc0c90b0 0x7fcffc0cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc15b590 con 0x7fcffc15ab40 2023-10-08T05:55:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fcff8001690 con 0x7fcffc15ab40 2023-10-08T05:55:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 >> v1:172.21.15.70:6789/0 conn(0x7fcffc15ab40 legacy=0x7fcffc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cb3b0 0x7fcfe80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe8003f20 0x7fcfe80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfe8004d00 0x7fcfe8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc15bf30 con 0x7fcfe8004d00 2023-10-08T05:55:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0cb820 con 0x7fcfe8003f20 2023-10-08T05:55:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe800ba00 con 0x7fcffc0cb3b0 2023-10-08T05:55:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.653+0000 7fd001a74640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfe8004d00 0x7fcfe8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.654+0000 7fd001273640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cb3b0 0x7fcfe80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.654+0000 7fd000a72640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe8003f20 0x7fcfe80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.654+0000 7fd001a74640 1 -- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cb3b0 msgr2=0x7fcfe80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:29.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.654+0000 7fd001a74640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cb3b0 0x7fcfe80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.654+0000 7fd001a74640 1 -- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe8003f20 msgr2=0x7fcfe80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:29.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.654+0000 7fd001a74640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe8003f20 0x7fcfe80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.654+0000 7fd001a74640 1 -- 172.21.15.70:0/4057090816 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe8003520 con 0x7fcfe8004d00 2023-10-08T05:55:29.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.654+0000 7fd001273640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc0cb3b0 0x7fcfe80039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:29.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.655+0000 7fd001a74640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfe8004d00 0x7fcfe8005180 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009fc0 tx=0x7fcfe80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b717e1d225901d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:29.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.655+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff402cbf0 con 0x7fcfe8004d00 2023-10-08T05:55:29.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.655+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff400f040 con 0x7fcfe8004d00 2023-10-08T05:55:29.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.655+0000 7fcff3fff640 1 -- 172.21.15.70:0/4057090816 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4034440 con 0x7fcfe8004d00 2023-10-08T05:55:29.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.656+0000 7fd0034d7640 1 -- 172.21.15.70:0/4057090816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfe8004d00 msgr2=0x7fcfe8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.656+0000 7fd0034d7640 1 --2- 172.21.15.70:0/4057090816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfe8004d00 0x7fcfe8005180 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009fc0 tx=0x7fcfe80057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.656+0000 7fd001a74640 1 -- 172.21.15.70:0/4057090816 reap_dead start 2023-10-08T05:55:29.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.656+0000 7fd0034d7640 1 -- 172.21.15.70:0/4057090816 shutdown_connections 2023-10-08T05:55:29.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.656+0000 7fd0034d7640 1 -- 172.21.15.70:0/4057090816 >> 172.21.15.70:0/4057090816 conn(0x7fcffc06a380 msgr2=0x7fcffc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:29.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.656+0000 7fd0034d7640 1 -- 172.21.15.70:0/4057090816 shutdown_connections 2023-10-08T05:55:29.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.656+0000 7fd0034d7640 1 -- 172.21.15.70:0/4057090816 wait complete. 2023-10-08T05:55:29.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.658+0000 7fd0034d7640 1 Processor -- start 2023-10-08T05:55:29.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.658+0000 7fd0034d7640 1 -- start start 2023-10-08T05:55:29.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd0034d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe8004d00 0x7fcffc142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd0034d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc153c80 0x7fcffc15cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd0034d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc15d260 0x7fcffc0c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd0034d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc15dcf0 con 0x7fcffc15d260 2023-10-08T05:55:29.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd0034d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc15d730 con 0x7fcfe8004d00 2023-10-08T05:55:29.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd0034d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc15da10 con 0x7fcffc153c80 2023-10-08T05:55:29.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd001a74640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc15d260 0x7fcffc0c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd001a74640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc15d260 0x7fcffc0c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54868/0 (socket says 172.21.15.70:54868) 2023-10-08T05:55:29.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd001a74640 1 -- 172.21.15.70:0/665658051 learned_addr learned my addr 172.21.15.70:0/665658051 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:29.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd001a74640 1 -- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc153c80 msgr2=0x7fcffc15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd001a74640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc153c80 0x7fcffc15cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd001a74640 1 -- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe8004d00 msgr2=0x7fcffc142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd001a74640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe8004d00 0x7fcffc142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.659+0000 7fd001a74640 1 -- 172.21.15.70:0/665658051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc0aba60 con 0x7fcffc15d260 2023-10-08T05:55:29.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.660+0000 7fd001a74640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc15d260 0x7fcffc0c5850 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fcff4002230 tx=0x7fcff4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:29.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.660+0000 7fcff1ffb640 1 -- 172.21.15.70:0/665658051 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4043070 con 0x7fcffc15d260 2023-10-08T05:55:29.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.660+0000 7fcff1ffb640 1 -- 172.21.15.70:0/665658051 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff402de80 con 0x7fcffc15d260 2023-10-08T05:55:29.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.660+0000 7fcff1ffb640 1 -- 172.21.15.70:0/665658051 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4034d10 con 0x7fcffc15d260 2023-10-08T05:55:29.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.660+0000 7fd0034d7640 1 -- 172.21.15.70:0/665658051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc0abd40 con 0x7fcffc15d260 2023-10-08T05:55:29.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.660+0000 7fd0034d7640 1 -- 172.21.15.70:0/665658051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc0c5f70 con 0x7fcffc15d260 2023-10-08T05:55:29.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.661+0000 7fd0034d7640 1 -- 172.21.15.70:0/665658051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcffc0ab8c0 con 0x7fcffc15d260 2023-10-08T05:55:29.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.662+0000 7fcff1ffb640 1 -- 172.21.15.70:0/665658051 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcff403e440 con 0x7fcffc15d260 2023-10-08T05:55:29.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.664+0000 7fcff1ffb640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcfe006af90 0x7fcfe006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:29.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.664+0000 7fcff1ffb640 1 -- 172.21.15.70:0/665658051 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcff40ab490 con 0x7fcffc15d260 2023-10-08T05:55:29.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.664+0000 7fd001273640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcfe006af90 0x7fcfe006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:29.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.665+0000 7fd001273640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcfe006af90 0x7fcfe006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0c8d80 tx=0x7fcff8009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:29.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.668+0000 7fcff1ffb640 1 -- 172.21.15.70:0/665658051 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcffc0ab8c0 con 0x7fcffc15d260 2023-10-08T05:55:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.861+0000 7fd0034d7640 1 -- 172.21.15.70:0/665658051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcffc0ab8c0 con 0x7fcffc15d260 2023-10-08T05:55:29.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.863+0000 7fcff1ffb640 1 -- 172.21.15.70:0/665658051 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+16774 (secure 0 0 0) 0x7fcffc0ab8c0 con 0x7fcffc15d260 2023-10-08T05:55:29.864 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:29.864 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":134,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:26.913100+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:55:23.150004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-08T05:55:24.358581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:55:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 -- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcfe006af90 msgr2=0x7fcfe006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcfe006af90 0x7fcfe006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0c8d80 tx=0x7fcff8009040 comp rx=0 tx=0).stop 2023-10-08T05:55:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 -- 172.21.15.70:0/665658051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc15d260 msgr2=0x7fcffc0c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc15d260 0x7fcffc0c5850 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fcff4002230 tx=0x7fcff4009750 comp rx=0 tx=0).stop 2023-10-08T05:55:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 -- 172.21.15.70:0/665658051 shutdown_connections 2023-10-08T05:55:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fcfe006af90 0x7fcfe006d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcffc15d260 0x7fcffc0c5850 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcffc153c80 0x7fcffc15cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 --2- 172.21.15.70:0/665658051 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcfe8004d00 0x7fcffc142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 -- 172.21.15.70:0/665658051 >> 172.21.15.70:0/665658051 conn(0x7fcffc06a380 msgr2=0x7fcffc0af940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 -- 172.21.15.70:0/665658051 shutdown_connections 2023-10-08T05:55:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:29.866+0000 7fd0034d7640 1 -- 172.21.15.70:0/665658051 wait complete. 2023-10-08T05:55:29.884 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T05:55:29.884 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:55:30.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.093+0000 7f2a35fb8640 1 Processor -- start 2023-10-08T05:55:30.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.093+0000 7f2a35fb8640 1 -- start start 2023-10-08T05:55:30.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.094+0000 7f2a35fb8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2a3015ab30 0x7f2a3015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.094+0000 7f2a35fb8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a300adc70 0x7f2a300b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.094+0000 7f2a35fb8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3015f7d0 con 0x7f2a3015af10 2023-10-08T05:55:30.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.094+0000 7f2a35fb8640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3015f210 con 0x7f2a3015ab30 2023-10-08T05:55:30.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.094+0000 7f2a35fb8640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015f4f0 con 0x7f2a300adc70 2023-10-08T05:55:30.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.095+0000 7f2a357b7640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2a3015af10 0x7f2a300701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54796/0 (socket says 172.21.15.70:54796) 2023-10-08T05:55:30.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.095+0000 7f2a357b7640 1 -- 172.21.15.70:0/1142931291 learned_addr learned my addr 172.21.15.70:0/1142931291 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.095+0000 7f2a2ffff640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a300adc70 0x7f2a300b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.095+0000 7f2a34fb6640 1 --2- 172.21.15.70:0/1142931291 >> v2:172.21.15.181:3300/0 conn(0x7f2a3015ab30 0x7f2a3015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.095+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253970512 0 0) 0x7f2a3015f7d0 con 0x7f2a3015af10 2023-10-08T05:55:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.095+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a20003520 con 0x7f2a3015af10 2023-10-08T05:55:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.095+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2193108695 0 0) 0x7f2a20003520 con 0x7f2a3015af10 2023-10-08T05:55:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.095+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a300adc70 msgr2=0x7f2a300b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.095+0000 7f2a2f7fe640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a300adc70 0x7f2a300b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 >> v2:172.21.15.181:3300/0 conn(0x7f2a3015ab30 msgr2=0x7f2a3015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 --2- 172.21.15.70:0/1142931291 >> v2:172.21.15.181:3300/0 conn(0x7f2a3015ab30 0x7f2a3015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3006c550 con 0x7f2a3015af10 2023-10-08T05:55:30.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2a28001a10 con 0x7f2a3015af10 2023-10-08T05:55:30.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 >> v1:172.21.15.70:6789/0 conn(0x7f2a3015af10 legacy=0x7f2a300701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a20003970 0x7f2a20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a200047c0 0x7f2a20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a300adc70 0x7f2a20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015f210 con 0x7f2a200047c0 2023-10-08T05:55:30.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3015f4f0 con 0x7f2a20003970 2023-10-08T05:55:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2000b9b0 con 0x7f2a300adc70 2023-10-08T05:55:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2ffff640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a200047c0 0x7f2a20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a34fb6640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a20003970 0x7f2a20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a357b7640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a300adc70 0x7f2a20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2ffff640 1 -- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a300adc70 msgr2=0x7f2a20005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2ffff640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a300adc70 0x7f2a20005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2ffff640 1 -- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a20003970 msgr2=0x7f2a20003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.096+0000 7f2a2ffff640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a20003970 0x7f2a20003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.097+0000 7f2a2ffff640 1 -- 172.21.15.70:0/1142931291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a20003520 con 0x7f2a200047c0 2023-10-08T05:55:30.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.098+0000 7f2a2ffff640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a200047c0 0x7f2a20004c40 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009ac0 tx=0x7f2a200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0d0ac4664b820e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.098+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c02c6b0 con 0x7f2a200047c0 2023-10-08T05:55:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.098+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c02c810 con 0x7f2a200047c0 2023-10-08T05:55:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.098+0000 7f2a2f7fe640 1 -- 172.21.15.70:0/1142931291 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c02cba0 con 0x7f2a200047c0 2023-10-08T05:55:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.098+0000 7f2a35fb8640 1 -- 172.21.15.70:0/1142931291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a200047c0 msgr2=0x7f2a20004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.098+0000 7f2a35fb8640 1 --2- 172.21.15.70:0/1142931291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a200047c0 0x7f2a20004c40 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c009ac0 tx=0x7f2a200057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.098+0000 7f2a357b7640 1 -- 172.21.15.70:0/1142931291 reap_dead start 2023-10-08T05:55:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.099+0000 7f2a35fb8640 1 -- 172.21.15.70:0/1142931291 shutdown_connections 2023-10-08T05:55:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.099+0000 7f2a35fb8640 1 -- 172.21.15.70:0/1142931291 >> 172.21.15.70:0/1142931291 conn(0x7f2a300cb0c0 msgr2=0x7f2a300cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.099+0000 7f2a35fb8640 1 -- 172.21.15.70:0/1142931291 shutdown_connections 2023-10-08T05:55:30.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.099+0000 7f2a35fb8640 1 -- 172.21.15.70:0/1142931291 wait complete. 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.100+0000 7f2a35fb8640 1 Processor -- start 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.101+0000 7f2a35fb8640 1 -- start start 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a35fb8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a200047c0 0x7f2a300ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a35fb8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a3014eab0 0x7f2a3014d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a35fb8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300cdac0 0x7f2a300af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a35fb8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3014f220 con 0x7f2a300cdac0 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a35fb8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3014ef40 con 0x7f2a3014eab0 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a35fb8640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a300c6f00 con 0x7f2a200047c0 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a357b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300cdac0 0x7f2a300af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a357b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300cdac0 0x7f2a300af1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54894/0 (socket says 172.21.15.70:54894) 2023-10-08T05:55:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a357b7640 1 -- 172.21.15.70:0/4019294040 learned_addr learned my addr 172.21.15.70:0/4019294040 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a34fb6640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a200047c0 0x7f2a300ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a357b7640 1 -- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a200047c0 msgr2=0x7f2a300ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a2ffff640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a3014eab0 0x7f2a3014d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a357b7640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a200047c0 0x7f2a300ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a357b7640 1 -- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a3014eab0 msgr2=0x7f2a3014d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a357b7640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a3014eab0 0x7f2a3014d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a357b7640 1 -- 172.21.15.70:0/4019294040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a300c71e0 con 0x7f2a300cdac0 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.102+0000 7f2a34fb6640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a200047c0 0x7f2a300ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.103+0000 7f2a357b7640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300cdac0 0x7f2a300af1c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2a28002230 tx=0x7f2a2800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.103+0000 7f2a2d7fa640 1 -- 172.21.15.70:0/4019294040 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28007ae0 con 0x7f2a300cdac0 2023-10-08T05:55:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.103+0000 7f2a2d7fa640 1 -- 172.21.15.70:0/4019294040 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2800edc0 con 0x7f2a300cdac0 2023-10-08T05:55:30.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.103+0000 7f2a35fb8640 1 -- 172.21.15.70:0/4019294040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a300c74c0 con 0x7f2a300cdac0 2023-10-08T05:55:30.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.103+0000 7f2a2d7fa640 1 -- 172.21.15.70:0/4019294040 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a28013660 con 0x7f2a300cdac0 2023-10-08T05:55:30.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.103+0000 7f2a35fb8640 1 -- 172.21.15.70:0/4019294040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3014dd50 con 0x7f2a300cdac0 2023-10-08T05:55:30.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.104+0000 7f2a35fb8640 1 -- 172.21.15.70:0/4019294040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a300c6d60 con 0x7f2a300cdac0 2023-10-08T05:55:30.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.105+0000 7f2a2d7fa640 1 -- 172.21.15.70:0/4019294040 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2a280137c0 con 0x7f2a300cdac0 2023-10-08T05:55:30.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.106+0000 7f2a2d7fa640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a1406af90 0x7f2a1406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.106+0000 7f2a2d7fa640 1 -- 172.21.15.70:0/4019294040 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2a28089f40 con 0x7f2a300cdac0 2023-10-08T05:55:30.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.107+0000 7f2a34fb6640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a1406af90 0x7f2a1406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.108+0000 7f2a34fb6640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a1406af90 0x7f2a1406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2a24006fd0 tx=0x7f2a24008040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:30.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.110+0000 7f2a2d7fa640 1 -- 172.21.15.70:0/4019294040 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a300c6d60 con 0x7f2a300cdac0 2023-10-08T05:55:30.288 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-10-08T05:55:30.288+0000 7f77969a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:55:30.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.294+0000 7f2a35fb8640 1 -- 172.21.15.70:0/4019294040 --> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a3006c830 con 0x7f2a1406af90 2023-10-08T05:55:30.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.324+0000 7f2a2d7fa640 1 -- 172.21.15.70:0/4019294040 <== mgr.5747 v2:172.21.15.181:6832/3142216408 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287153 (secure 0 0 0) 0x7f2a2400b650 con 0x7f2a1406af90 2023-10-08T05:55:30.330 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:30.332 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:55:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 -- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a1406af90 msgr2=0x7f2a1406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a1406af90 0x7f2a1406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2a24006fd0 tx=0x7f2a24008040 comp rx=0 tx=0).stop 2023-10-08T05:55:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 -- 172.21.15.70:0/4019294040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300cdac0 msgr2=0x7f2a300af1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300cdac0 0x7f2a300af1c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2a28002230 tx=0x7f2a2800e530 comp rx=0 tx=0).stop 2023-10-08T05:55:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 -- 172.21.15.70:0/4019294040 shutdown_connections 2023-10-08T05:55:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f2a1406af90 0x7f2a1406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a300cdac0 0x7f2a300af1c0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2a3014eab0 0x7f2a3014d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 --2- 172.21.15.70:0/4019294040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2a200047c0 0x7f2a300ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 -- 172.21.15.70:0/4019294040 >> 172.21.15.70:0/4019294040 conn(0x7f2a300cb0c0 msgr2=0x7f2a30159550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:30.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 -- 172.21.15.70:0/4019294040 shutdown_connections 2023-10-08T05:55:30.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.329+0000 7f2a35fb8640 1 -- 172.21.15.70:0/4019294040 wait complete. 2023-10-08T05:55:30.349 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-08T05:55:28.749629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":957,"num_read_kb":1940,"num_write":373,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2701,"ondisk_log_size":2701,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":324668,"kb_used_data":14148,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754650052,"statfs":{"total":773094113280,"available":772761653248,"internally_reserved":0,"allocated":14487552,"data_stored":7914628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":317836859},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.873808"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284592+0000","last_change":"2023-10-08T05:55:25.916876+0000","last_active":"2023-10-08T05:55:27.284592+0000","last_peered":"2023-10-08T05:55:27.284592+0000","last_clean":"2023-10-08T05:55:27.284592+0000","last_became_active":"2023-10-08T05:55:25.916557+0000","last_became_peered":"2023-10-08T05:55:25.916557+0000","last_unstale":"2023-10-08T05:55:27.284592+0000","last_undegraded":"2023-10-08T05:55:27.284592+0000","last_fullsized":"2023-10-08T05:55:27.284592+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:45.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503739+0000","last_change":"2023-10-08T05:55:25.928225+0000","last_active":"2023-10-08T05:55:27.503739+0000","last_peered":"2023-10-08T05:55:27.503739+0000","last_clean":"2023-10-08T05:55:27.503739+0000","last_became_active":"2023-10-08T05:55:25.927889+0000","last_became_peered":"2023-10-08T05:55:25.927889+0000","last_unstale":"2023-10-08T05:55:27.503739+0000","last_undegraded":"2023-10-08T05:55:27.503739+0000","last_fullsized":"2023-10-08T05:55:27.503739+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:21:23.883548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284430+0000","last_change":"2023-10-08T05:55:25.939516+0000","last_active":"2023-10-08T05:55:27.284430+0000","last_peered":"2023-10-08T05:55:27.284430+0000","last_clean":"2023-10-08T05:55:27.284430+0000","last_became_active":"2023-10-08T05:55:25.938748+0000","last_became_peered":"2023-10-08T05:55:25.938748+0000","last_unstale":"2023-10-08T05:55:27.284430+0000","last_undegraded":"2023-10-08T05:55:27.284430+0000","last_fullsized":"2023-10-08T05:55:27.284430+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:29:57.765972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.925973+0000","last_change":"2023-10-08T05:55:25.927966+0000","last_active":"2023-10-08T05:55:26.925973+0000","last_peered":"2023-10-08T05:55:26.925973+0000","last_clean":"2023-10-08T05:55:26.925973+0000","last_became_active":"2023-10-08T05:55:25.927714+0000","last_became_peered":"2023-10-08T05:55:25.927714+0000","last_unstale":"2023-10-08T05:55:26.925973+0000","last_undegraded":"2023-10-08T05:55:26.925973+0000","last_fullsized":"2023-10-08T05:55:26.925973+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:16.285768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503598+0000","last_change":"2023-10-08T05:55:25.921650+0000","last_active":"2023-10-08T05:55:27.503598+0000","last_peered":"2023-10-08T05:55:27.503598+0000","last_clean":"2023-10-08T05:55:27.503598+0000","last_became_active":"2023-10-08T05:55:25.921345+0000","last_became_peered":"2023-10-08T05:55:25.921345+0000","last_unstale":"2023-10-08T05:55:27.503598+0000","last_undegraded":"2023-10-08T05:55:27.503598+0000","last_fullsized":"2023-10-08T05:55:27.503598+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:50:43.904942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502181+0000","last_change":"2023-10-08T05:55:25.927148+0000","last_active":"2023-10-08T05:55:27.502181+0000","last_peered":"2023-10-08T05:55:27.502181+0000","last_clean":"2023-10-08T05:55:27.502181+0000","last_became_active":"2023-10-08T05:55:25.926795+0000","last_became_peered":"2023-10-08T05:55:25.926795+0000","last_unstale":"2023-10-08T05:55:27.502181+0000","last_undegraded":"2023-10-08T05:55:27.502181+0000","last_fullsized":"2023-10-08T05:55:27.502181+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:21.947765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501958+0000","last_change":"2023-10-08T05:55:25.926748+0000","last_active":"2023-10-08T05:55:27.501958+0000","last_peered":"2023-10-08T05:55:27.501958+0000","last_clean":"2023-10-08T05:55:27.501958+0000","last_became_active":"2023-10-08T05:55:25.926427+0000","last_became_peered":"2023-10-08T05:55:25.926427+0000","last_unstale":"2023-10-08T05:55:27.501958+0000","last_undegraded":"2023-10-08T05:55:27.501958+0000","last_fullsized":"2023-10-08T05:55:27.501958+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.716202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.500842+0000","last_change":"2023-10-08T05:55:25.921911+0000","last_active":"2023-10-08T05:55:27.500842+0000","last_peered":"2023-10-08T05:55:27.500842+0000","last_clean":"2023-10-08T05:55:27.500842+0000","last_became_active":"2023-10-08T05:55:25.921677+0000","last_became_peered":"2023-10-08T05:55:25.921677+0000","last_unstale":"2023-10-08T05:55:27.500842+0000","last_undegraded":"2023-10-08T05:55:27.500842+0000","last_fullsized":"2023-10-08T05:55:27.500842+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:07.653076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502733+0000","last_change":"2023-10-08T05:55:25.927783+0000","last_active":"2023-10-08T05:55:27.502733+0000","last_peered":"2023-10-08T05:55:27.502733+0000","last_clean":"2023-10-08T05:55:27.502733+0000","last_became_active":"2023-10-08T05:55:25.927506+0000","last_became_peered":"2023-10-08T05:55:25.927506+0000","last_unstale":"2023-10-08T05:55:27.502733+0000","last_undegraded":"2023-10-08T05:55:27.502733+0000","last_fullsized":"2023-10-08T05:55:27.502733+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:36:57.120294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504173+0000","last_change":"2023-10-08T05:55:25.929596+0000","last_active":"2023-10-08T05:55:27.504173+0000","last_peered":"2023-10-08T05:55:27.504173+0000","last_clean":"2023-10-08T05:55:27.504173+0000","last_became_active":"2023-10-08T05:55:25.929300+0000","last_became_peered":"2023-10-08T05:55:25.929300+0000","last_unstale":"2023-10-08T05:55:27.504173+0000","last_undegraded":"2023-10-08T05:55:27.504173+0000","last_fullsized":"2023-10-08T05:55:27.504173+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:16.914110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503041+0000","last_change":"2023-10-08T05:55:25.935202+0000","last_active":"2023-10-08T05:55:27.503041+0000","last_peered":"2023-10-08T05:55:27.503041+0000","last_clean":"2023-10-08T05:55:27.503041+0000","last_became_active":"2023-10-08T05:55:25.934864+0000","last_became_peered":"2023-10-08T05:55:25.934864+0000","last_unstale":"2023-10-08T05:55:27.503041+0000","last_undegraded":"2023-10-08T05:55:27.503041+0000","last_fullsized":"2023-10-08T05:55:27.503041+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:45.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.931011+0000","last_change":"2023-10-08T05:55:25.925077+0000","last_active":"2023-10-08T05:55:26.931011+0000","last_peered":"2023-10-08T05:55:26.931011+0000","last_clean":"2023-10-08T05:55:26.931011+0000","last_became_active":"2023-10-08T05:55:25.924802+0000","last_became_peered":"2023-10-08T05:55:25.924802+0000","last_unstale":"2023-10-08T05:55:26.931011+0000","last_undegraded":"2023-10-08T05:55:26.931011+0000","last_fullsized":"2023-10-08T05:55:26.931011+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:55.785568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926061+0000","last_change":"2023-10-08T05:55:25.937597+0000","last_active":"2023-10-08T05:55:26.926061+0000","last_peered":"2023-10-08T05:55:26.926061+0000","last_clean":"2023-10-08T05:55:26.926061+0000","last_became_active":"2023-10-08T05:55:25.935914+0000","last_became_peered":"2023-10-08T05:55:25.935914+0000","last_unstale":"2023-10-08T05:55:26.926061+0000","last_undegraded":"2023-10-08T05:55:26.926061+0000","last_fullsized":"2023-10-08T05:55:26.926061+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:59.379656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.928756+0000","last_change":"2023-10-08T05:55:25.921912+0000","last_active":"2023-10-08T05:55:26.928756+0000","last_peered":"2023-10-08T05:55:26.928756+0000","last_clean":"2023-10-08T05:55:26.928756+0000","last_became_active":"2023-10-08T05:55:25.921560+0000","last_became_peered":"2023-10-08T05:55:25.921560+0000","last_unstale":"2023-10-08T05:55:26.928756+0000","last_undegraded":"2023-10-08T05:55:26.928756+0000","last_fullsized":"2023-10-08T05:55:26.928756+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:29:57.765972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.928582+0000","last_change":"2023-10-08T05:55:25.916817+0000","last_active":"2023-10-08T05:55:26.928582+0000","last_peered":"2023-10-08T05:55:26.928582+0000","last_clean":"2023-10-08T05:55:26.928582+0000","last_became_active":"2023-10-08T05:55:25.916513+0000","last_became_peered":"2023-10-08T05:55:25.916513+0000","last_unstale":"2023-10-08T05:55:26.928582+0000","last_undegraded":"2023-10-08T05:55:26.928582+0000","last_fullsized":"2023-10-08T05:55:26.928582+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:51:00.669382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.928409+0000","last_change":"2023-10-08T05:55:25.920621+0000","last_active":"2023-10-08T05:55:26.928409+0000","last_peered":"2023-10-08T05:55:26.928409+0000","last_clean":"2023-10-08T05:55:26.928409+0000","last_became_active":"2023-10-08T05:55:25.920298+0000","last_became_peered":"2023-10-08T05:55:25.920298+0000","last_unstale":"2023-10-08T05:55:26.928409+0000","last_undegraded":"2023-10-08T05:55:26.928409+0000","last_fullsized":"2023-10-08T05:55:26.928409+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:20.072607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284677+0000","last_change":"2023-10-08T05:55:25.939572+0000","last_active":"2023-10-08T05:55:27.284677+0000","last_peered":"2023-10-08T05:55:27.284677+0000","last_clean":"2023-10-08T05:55:27.284677+0000","last_became_active":"2023-10-08T05:55:25.939279+0000","last_became_peered":"2023-10-08T05:55:25.939279+0000","last_unstale":"2023-10-08T05:55:27.284677+0000","last_undegraded":"2023-10-08T05:55:27.284677+0000","last_fullsized":"2023-10-08T05:55:27.284677+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:19:06.059847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504051+0000","last_change":"2023-10-08T05:55:25.928178+0000","last_active":"2023-10-08T05:55:27.504051+0000","last_peered":"2023-10-08T05:55:27.504051+0000","last_clean":"2023-10-08T05:55:27.504051+0000","last_became_active":"2023-10-08T05:55:25.927795+0000","last_became_peered":"2023-10-08T05:55:25.927795+0000","last_unstale":"2023-10-08T05:55:27.504051+0000","last_undegraded":"2023-10-08T05:55:27.504051+0000","last_fullsized":"2023-10-08T05:55:27.504051+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:45.993171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284188+0000","last_change":"2023-10-08T05:55:25.929932+0000","last_active":"2023-10-08T05:55:27.284188+0000","last_peered":"2023-10-08T05:55:27.284188+0000","last_clean":"2023-10-08T05:55:27.284188+0000","last_became_active":"2023-10-08T05:55:25.929677+0000","last_became_peered":"2023-10-08T05:55:25.929677+0000","last_unstale":"2023-10-08T05:55:27.284188+0000","last_undegraded":"2023-10-08T05:55:27.284188+0000","last_fullsized":"2023-10-08T05:55:27.284188+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:59.379656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.931241+0000","last_change":"2023-10-08T05:55:25.924986+0000","last_active":"2023-10-08T05:55:26.931241+0000","last_peered":"2023-10-08T05:55:26.931241+0000","last_clean":"2023-10-08T05:55:26.931241+0000","last_became_active":"2023-10-08T05:55:25.923671+0000","last_became_peered":"2023-10-08T05:55:25.923671+0000","last_unstale":"2023-10-08T05:55:26.931241+0000","last_undegraded":"2023-10-08T05:55:26.931241+0000","last_fullsized":"2023-10-08T05:55:26.931241+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:12:34.832429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502246+0000","last_change":"2023-10-08T05:55:25.927866+0000","last_active":"2023-10-08T05:55:27.502246+0000","last_peered":"2023-10-08T05:55:27.502246+0000","last_clean":"2023-10-08T05:55:27.502246+0000","last_became_active":"2023-10-08T05:55:25.927611+0000","last_became_peered":"2023-10-08T05:55:25.927611+0000","last_unstale":"2023-10-08T05:55:27.502246+0000","last_undegraded":"2023-10-08T05:55:27.502246+0000","last_fullsized":"2023-10-08T05:55:27.502246+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:51.637826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503507+0000","last_change":"2023-10-08T05:55:25.926358+0000","last_active":"2023-10-08T05:55:27.503507+0000","last_peered":"2023-10-08T05:55:27.503507+0000","last_clean":"2023-10-08T05:55:27.503507+0000","last_became_active":"2023-10-08T05:55:25.924421+0000","last_became_peered":"2023-10-08T05:55:25.924421+0000","last_unstale":"2023-10-08T05:55:27.503507+0000","last_undegraded":"2023-10-08T05:55:27.503507+0000","last_fullsized":"2023-10-08T05:55:27.503507+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:30.550731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.283984+0000","last_change":"2023-10-08T05:55:25.919782+0000","last_active":"2023-10-08T05:55:27.283984+0000","last_peered":"2023-10-08T05:55:27.283984+0000","last_clean":"2023-10-08T05:55:27.283984+0000","last_became_active":"2023-10-08T05:55:25.918562+0000","last_became_peered":"2023-10-08T05:55:25.918562+0000","last_unstale":"2023-10-08T05:55:27.283984+0000","last_undegraded":"2023-10-08T05:55:27.283984+0000","last_fullsized":"2023-10-08T05:55:27.283984+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:59.764889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502769+0000","last_change":"2023-10-08T05:55:25.919390+0000","last_active":"2023-10-08T05:55:27.502769+0000","last_peered":"2023-10-08T05:55:27.502769+0000","last_clean":"2023-10-08T05:55:27.502769+0000","last_became_active":"2023-10-08T05:55:25.919114+0000","last_became_peered":"2023-10-08T05:55:25.919114+0000","last_unstale":"2023-10-08T05:55:27.502769+0000","last_undegraded":"2023-10-08T05:55:27.502769+0000","last_fullsized":"2023-10-08T05:55:27.502769+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:55.785568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930113+0000","last_change":"2023-10-08T05:55:25.938070+0000","last_active":"2023-10-08T05:55:26.930113+0000","last_peered":"2023-10-08T05:55:26.930113+0000","last_clean":"2023-10-08T05:55:26.930113+0000","last_became_active":"2023-10-08T05:55:25.937803+0000","last_became_peered":"2023-10-08T05:55:25.937803+0000","last_unstale":"2023-10-08T05:55:26.930113+0000","last_undegraded":"2023-10-08T05:55:26.930113+0000","last_fullsized":"2023-10-08T05:55:26.930113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:51.236939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930771+0000","last_change":"2023-10-08T05:55:25.929131+0000","last_active":"2023-10-08T05:55:26.930771+0000","last_peered":"2023-10-08T05:55:26.930771+0000","last_clean":"2023-10-08T05:55:26.930771+0000","last_became_active":"2023-10-08T05:55:25.928857+0000","last_became_peered":"2023-10-08T05:55:25.928857+0000","last_unstale":"2023-10-08T05:55:26.930771+0000","last_undegraded":"2023-10-08T05:55:26.930771+0000","last_fullsized":"2023-10-08T05:55:26.930771+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:40.647831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503107+0000","last_change":"2023-10-08T05:55:25.924553+0000","last_active":"2023-10-08T05:55:27.503107+0000","last_peered":"2023-10-08T05:55:27.503107+0000","last_clean":"2023-10-08T05:55:27.503107+0000","last_became_active":"2023-10-08T05:55:25.919977+0000","last_became_peered":"2023-10-08T05:55:25.919977+0000","last_unstale":"2023-10-08T05:55:27.503107+0000","last_undegraded":"2023-10-08T05:55:27.503107+0000","last_fullsized":"2023-10-08T05:55:27.503107+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:19:06.059847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503245+0000","last_change":"2023-10-08T05:55:25.939665+0000","last_active":"2023-10-08T05:55:27.503245+0000","last_peered":"2023-10-08T05:55:27.503245+0000","last_clean":"2023-10-08T05:55:27.503245+0000","last_became_active":"2023-10-08T05:55:25.939387+0000","last_became_peered":"2023-10-08T05:55:25.939387+0000","last_unstale":"2023-10-08T05:55:27.503245+0000","last_undegraded":"2023-10-08T05:55:27.503245+0000","last_fullsized":"2023-10-08T05:55:27.503245+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:58.693510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501890+0000","last_change":"2023-10-08T05:55:25.925332+0000","last_active":"2023-10-08T05:55:27.501890+0000","last_peered":"2023-10-08T05:55:27.501890+0000","last_clean":"2023-10-08T05:55:27.501890+0000","last_became_active":"2023-10-08T05:55:25.925065+0000","last_became_peered":"2023-10-08T05:55:25.925065+0000","last_unstale":"2023-10-08T05:55:27.501890+0000","last_undegraded":"2023-10-08T05:55:27.501890+0000","last_fullsized":"2023-10-08T05:55:27.501890+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:52.152295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930235+0000","last_change":"2023-10-08T05:55:25.922604+0000","last_active":"2023-10-08T05:55:26.930235+0000","last_peered":"2023-10-08T05:55:26.930235+0000","last_clean":"2023-10-08T05:55:26.930235+0000","last_became_active":"2023-10-08T05:55:25.922304+0000","last_became_peered":"2023-10-08T05:55:25.922304+0000","last_unstale":"2023-10-08T05:55:26.930235+0000","last_undegraded":"2023-10-08T05:55:26.930235+0000","last_fullsized":"2023-10-08T05:55:26.930235+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:34.943938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502312+0000","last_change":"2023-10-08T05:55:25.928946+0000","last_active":"2023-10-08T05:55:27.502312+0000","last_peered":"2023-10-08T05:55:27.502312+0000","last_clean":"2023-10-08T05:55:27.502312+0000","last_became_active":"2023-10-08T05:55:25.928670+0000","last_became_peered":"2023-10-08T05:55:25.928670+0000","last_unstale":"2023-10-08T05:55:27.502312+0000","last_undegraded":"2023-10-08T05:55:27.502312+0000","last_fullsized":"2023-10-08T05:55:27.502312+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:14.777452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501088+0000","last_change":"2023-10-08T05:55:25.922881+0000","last_active":"2023-10-08T05:55:27.501088+0000","last_peered":"2023-10-08T05:55:27.501088+0000","last_clean":"2023-10-08T05:55:27.501088+0000","last_became_active":"2023-10-08T05:55:25.922606+0000","last_became_peered":"2023-10-08T05:55:25.922606+0000","last_unstale":"2023-10-08T05:55:27.501088+0000","last_undegraded":"2023-10-08T05:55:27.501088+0000","last_fullsized":"2023-10-08T05:55:27.501088+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:40.364816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284414+0000","last_change":"2023-10-08T05:55:25.936959+0000","last_active":"2023-10-08T05:55:27.284414+0000","last_peered":"2023-10-08T05:55:27.284414+0000","last_clean":"2023-10-08T05:55:27.284414+0000","last_became_active":"2023-10-08T05:55:25.936308+0000","last_became_peered":"2023-10-08T05:55:25.936308+0000","last_unstale":"2023-10-08T05:55:27.284414+0000","last_undegraded":"2023-10-08T05:55:27.284414+0000","last_fullsized":"2023-10-08T05:55:27.284414+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:20.072607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"133'4","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926146+0000","last_change":"2023-10-08T05:55:24.921548+0000","last_active":"2023-10-08T05:55:26.926146+0000","last_peered":"2023-10-08T05:55:26.926146+0000","last_clean":"2023-10-08T05:55:26.926146+0000","last_became_active":"2023-10-08T05:55:24.921266+0000","last_became_peered":"2023-10-08T05:55:24.921266+0000","last_unstale":"2023-10-08T05:55:26.926146+0000","last_undegraded":"2023-10-08T05:55:26.926146+0000","last_fullsized":"2023-10-08T05:55:26.926146+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:58.749374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501133+0000","last_change":"2023-10-08T05:55:25.922570+0000","last_active":"2023-10-08T05:55:27.501133+0000","last_peered":"2023-10-08T05:55:27.501133+0000","last_clean":"2023-10-08T05:55:27.501133+0000","last_became_active":"2023-10-08T05:55:25.922307+0000","last_became_peered":"2023-10-08T05:55:25.922307+0000","last_unstale":"2023-10-08T05:55:27.501133+0000","last_undegraded":"2023-10-08T05:55:27.501133+0000","last_fullsized":"2023-10-08T05:55:27.501133+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:45.680629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"133'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501732+0000","last_change":"2023-10-08T05:55:24.924415+0000","last_active":"2023-10-08T05:55:27.501732+0000","last_peered":"2023-10-08T05:55:27.501732+0000","last_clean":"2023-10-08T05:55:27.501732+0000","last_became_active":"2023-10-08T05:55:24.924175+0000","last_became_peered":"2023-10-08T05:55:24.924175+0000","last_unstale":"2023-10-08T05:55:27.501732+0000","last_undegraded":"2023-10-08T05:55:27.501732+0000","last_fullsized":"2023-10-08T05:55:27.501732+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:56.312969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930341+0000","last_change":"2023-10-08T05:55:25.925746+0000","last_active":"2023-10-08T05:55:26.930341+0000","last_peered":"2023-10-08T05:55:26.930341+0000","last_clean":"2023-10-08T05:55:26.930341+0000","last_became_active":"2023-10-08T05:55:25.925454+0000","last_became_peered":"2023-10-08T05:55:25.925454+0000","last_unstale":"2023-10-08T05:55:26.930341+0000","last_undegraded":"2023-10-08T05:55:26.930341+0000","last_fullsized":"2023-10-08T05:55:26.930341+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:37.759236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930642+0000","last_change":"2023-10-08T05:55:24.925389+0000","last_active":"2023-10-08T05:55:26.930642+0000","last_peered":"2023-10-08T05:55:26.930642+0000","last_clean":"2023-10-08T05:55:26.930642+0000","last_became_active":"2023-10-08T05:55:24.925096+0000","last_became_peered":"2023-10-08T05:55:24.925096+0000","last_unstale":"2023-10-08T05:55:26.930642+0000","last_undegraded":"2023-10-08T05:55:26.930642+0000","last_fullsized":"2023-10-08T05:55:26.930642+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:03.175876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504059+0000","last_change":"2023-10-08T05:55:25.924368+0000","last_active":"2023-10-08T05:55:27.504059+0000","last_peered":"2023-10-08T05:55:27.504059+0000","last_clean":"2023-10-08T05:55:27.504059+0000","last_became_active":"2023-10-08T05:55:25.924039+0000","last_became_peered":"2023-10-08T05:55:25.924039+0000","last_unstale":"2023-10-08T05:55:27.504059+0000","last_undegraded":"2023-10-08T05:55:27.504059+0000","last_fullsized":"2023-10-08T05:55:27.504059+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:08.047146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504057+0000","last_change":"2023-10-08T05:55:24.923839+0000","last_active":"2023-10-08T05:55:27.504057+0000","last_peered":"2023-10-08T05:55:27.504057+0000","last_clean":"2023-10-08T05:55:27.504057+0000","last_became_active":"2023-10-08T05:55:24.923549+0000","last_became_peered":"2023-10-08T05:55:24.923549+0000","last_unstale":"2023-10-08T05:55:27.504057+0000","last_undegraded":"2023-10-08T05:55:27.504057+0000","last_fullsized":"2023-10-08T05:55:27.504057+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:36.743721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926050+0000","last_change":"2023-10-08T05:55:25.925785+0000","last_active":"2023-10-08T05:55:26.926050+0000","last_peered":"2023-10-08T05:55:26.926050+0000","last_clean":"2023-10-08T05:55:26.926050+0000","last_became_active":"2023-10-08T05:55:25.925496+0000","last_became_peered":"2023-10-08T05:55:25.925496+0000","last_unstale":"2023-10-08T05:55:26.926050+0000","last_undegraded":"2023-10-08T05:55:26.926050+0000","last_fullsized":"2023-10-08T05:55:26.926050+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:19.783496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"133'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.928678+0000","last_change":"2023-10-08T05:55:24.923723+0000","last_active":"2023-10-08T05:55:26.928678+0000","last_peered":"2023-10-08T05:55:26.928678+0000","last_clean":"2023-10-08T05:55:26.928678+0000","last_became_active":"2023-10-08T05:55:24.923386+0000","last_became_peered":"2023-10-08T05:55:24.923386+0000","last_unstale":"2023-10-08T05:55:26.928678+0000","last_undegraded":"2023-10-08T05:55:26.928678+0000","last_fullsized":"2023-10-08T05:55:26.928678+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:58.364141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284539+0000","last_change":"2023-10-08T05:55:25.929326+0000","last_active":"2023-10-08T05:55:27.284539+0000","last_peered":"2023-10-08T05:55:27.284539+0000","last_clean":"2023-10-08T05:55:27.284539+0000","last_became_active":"2023-10-08T05:55:25.929051+0000","last_became_peered":"2023-10-08T05:55:25.929051+0000","last_unstale":"2023-10-08T05:55:27.284539+0000","last_undegraded":"2023-10-08T05:55:27.284539+0000","last_fullsized":"2023-10-08T05:55:27.284539+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:23.640733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"133'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502351+0000","last_change":"2023-10-08T05:55:24.922495+0000","last_active":"2023-10-08T05:55:27.502351+0000","last_peered":"2023-10-08T05:55:27.502351+0000","last_clean":"2023-10-08T05:55:27.502351+0000","last_became_active":"2023-10-08T05:55:24.921502+0000","last_became_peered":"2023-10-08T05:55:24.921502+0000","last_unstale":"2023-10-08T05:55:27.502351+0000","last_undegraded":"2023-10-08T05:55:27.502351+0000","last_fullsized":"2023-10-08T05:55:27.502351+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:50:59.653866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504170+0000","last_change":"2023-10-08T05:55:25.940045+0000","last_active":"2023-10-08T05:55:27.504170+0000","last_peered":"2023-10-08T05:55:27.504170+0000","last_clean":"2023-10-08T05:55:27.504170+0000","last_became_active":"2023-10-08T05:55:25.939782+0000","last_became_peered":"2023-10-08T05:55:25.939782+0000","last_unstale":"2023-10-08T05:55:27.504170+0000","last_undegraded":"2023-10-08T05:55:27.504170+0000","last_fullsized":"2023-10-08T05:55:27.504170+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:06.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929762+0000","last_change":"2023-10-08T05:55:24.920669+0000","last_active":"2023-10-08T05:55:26.929762+0000","last_peered":"2023-10-08T05:55:26.929762+0000","last_clean":"2023-10-08T05:55:26.929762+0000","last_became_active":"2023-10-08T05:55:24.920369+0000","last_became_peered":"2023-10-08T05:55:24.920369+0000","last_unstale":"2023-10-08T05:55:26.929762+0000","last_undegraded":"2023-10-08T05:55:26.929762+0000","last_fullsized":"2023-10-08T05:55:26.929762+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:35:21.616608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926045+0000","last_change":"2023-10-08T05:55:25.918315+0000","last_active":"2023-10-08T05:55:26.926045+0000","last_peered":"2023-10-08T05:55:26.926045+0000","last_clean":"2023-10-08T05:55:26.926045+0000","last_became_active":"2023-10-08T05:55:25.918032+0000","last_became_peered":"2023-10-08T05:55:25.918032+0000","last_unstale":"2023-10-08T05:55:26.926045+0000","last_undegraded":"2023-10-08T05:55:26.926045+0000","last_fullsized":"2023-10-08T05:55:26.926045+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:09:49.886504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.925630+0000","last_change":"2023-10-08T05:55:24.918727+0000","last_active":"2023-10-08T05:55:26.925630+0000","last_peered":"2023-10-08T05:55:26.925630+0000","last_clean":"2023-10-08T05:55:26.925630+0000","last_became_active":"2023-10-08T05:55:24.918439+0000","last_became_peered":"2023-10-08T05:55:24.918439+0000","last_unstale":"2023-10-08T05:55:26.925630+0000","last_undegraded":"2023-10-08T05:55:26.925630+0000","last_fullsized":"2023-10-08T05:55:26.925630+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:28.648179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284113+0000","last_change":"2023-10-08T05:55:25.937157+0000","last_active":"2023-10-08T05:55:27.284113+0000","last_peered":"2023-10-08T05:55:27.284113+0000","last_clean":"2023-10-08T05:55:27.284113+0000","last_became_active":"2023-10-08T05:55:25.936672+0000","last_became_peered":"2023-10-08T05:55:25.936672+0000","last_unstale":"2023-10-08T05:55:27.284113+0000","last_undegraded":"2023-10-08T05:55:27.284113+0000","last_fullsized":"2023-10-08T05:55:27.284113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:09:49.886504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930364+0000","last_change":"2023-10-08T05:55:24.924797+0000","last_active":"2023-10-08T05:55:26.930364+0000","last_peered":"2023-10-08T05:55:26.930364+0000","last_clean":"2023-10-08T05:55:26.930364+0000","last_became_active":"2023-10-08T05:55:24.922092+0000","last_became_peered":"2023-10-08T05:55:24.922092+0000","last_unstale":"2023-10-08T05:55:26.930364+0000","last_undegraded":"2023-10-08T05:55:26.930364+0000","last_fullsized":"2023-10-08T05:55:26.930364+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:47:42.546146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503078+0000","last_change":"2023-10-08T05:55:25.923194+0000","last_active":"2023-10-08T05:55:27.503078+0000","last_peered":"2023-10-08T05:55:27.503078+0000","last_clean":"2023-10-08T05:55:27.503078+0000","last_became_active":"2023-10-08T05:55:25.922940+0000","last_became_peered":"2023-10-08T05:55:25.922940+0000","last_unstale":"2023-10-08T05:55:27.503078+0000","last_undegraded":"2023-10-08T05:55:27.503078+0000","last_fullsized":"2023-10-08T05:55:27.503078+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:12:34.832429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284158+0000","last_change":"2023-10-08T05:55:24.916097+0000","last_active":"2023-10-08T05:55:27.284158+0000","last_peered":"2023-10-08T05:55:27.284158+0000","last_clean":"2023-10-08T05:55:27.284158+0000","last_became_active":"2023-10-08T05:55:24.915832+0000","last_became_peered":"2023-10-08T05:55:24.915832+0000","last_unstale":"2023-10-08T05:55:27.284158+0000","last_undegraded":"2023-10-08T05:55:27.284158+0000","last_fullsized":"2023-10-08T05:55:27.284158+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:15:28.229277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501536+0000","last_change":"2023-10-08T05:55:25.923432+0000","last_active":"2023-10-08T05:55:27.501536+0000","last_peered":"2023-10-08T05:55:27.501536+0000","last_clean":"2023-10-08T05:55:27.501536+0000","last_became_active":"2023-10-08T05:55:25.923160+0000","last_became_peered":"2023-10-08T05:55:25.923160+0000","last_unstale":"2023-10-08T05:55:27.501536+0000","last_undegraded":"2023-10-08T05:55:27.501536+0000","last_fullsized":"2023-10-08T05:55:27.501536+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:44:53.071337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"133'3","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929072+0000","last_change":"2023-10-08T05:55:24.924531+0000","last_active":"2023-10-08T05:55:26.929072+0000","last_peered":"2023-10-08T05:55:26.929072+0000","last_clean":"2023-10-08T05:55:26.929072+0000","last_became_active":"2023-10-08T05:55:24.924220+0000","last_became_peered":"2023-10-08T05:55:24.924220+0000","last_unstale":"2023-10-08T05:55:26.929072+0000","last_undegraded":"2023-10-08T05:55:26.929072+0000","last_fullsized":"2023-10-08T05:55:26.929072+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:50.957382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503972+0000","last_change":"2023-10-08T05:55:25.920338+0000","last_active":"2023-10-08T05:55:27.503972+0000","last_peered":"2023-10-08T05:55:27.503972+0000","last_clean":"2023-10-08T05:55:27.503972+0000","last_became_active":"2023-10-08T05:55:25.920031+0000","last_became_peered":"2023-10-08T05:55:25.920031+0000","last_unstale":"2023-10-08T05:55:27.503972+0000","last_undegraded":"2023-10-08T05:55:27.503972+0000","last_fullsized":"2023-10-08T05:55:27.503972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:04.369670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"133'6","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503235+0000","last_change":"2023-10-08T05:55:24.912893+0000","last_active":"2023-10-08T05:55:27.503235+0000","last_peered":"2023-10-08T05:55:27.503235+0000","last_clean":"2023-10-08T05:55:27.503235+0000","last_became_active":"2023-10-08T05:55:24.912587+0000","last_became_peered":"2023-10-08T05:55:24.912587+0000","last_unstale":"2023-10-08T05:55:27.503235+0000","last_undegraded":"2023-10-08T05:55:27.503235+0000","last_fullsized":"2023-10-08T05:55:27.503235+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:52.463852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503974+0000","last_change":"2023-10-08T05:55:25.924248+0000","last_active":"2023-10-08T05:55:27.503974+0000","last_peered":"2023-10-08T05:55:27.503974+0000","last_clean":"2023-10-08T05:55:27.503974+0000","last_became_active":"2023-10-08T05:55:25.923353+0000","last_became_peered":"2023-10-08T05:55:25.923353+0000","last_unstale":"2023-10-08T05:55:27.503974+0000","last_undegraded":"2023-10-08T05:55:27.503974+0000","last_fullsized":"2023-10-08T05:55:27.503974+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:25.180936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504039+0000","last_change":"2023-10-08T05:55:24.925089+0000","last_active":"2023-10-08T05:55:27.504039+0000","last_peered":"2023-10-08T05:55:27.504039+0000","last_clean":"2023-10-08T05:55:27.504039+0000","last_became_active":"2023-10-08T05:55:24.924351+0000","last_became_peered":"2023-10-08T05:55:24.924351+0000","last_unstale":"2023-10-08T05:55:27.504039+0000","last_undegraded":"2023-10-08T05:55:27.504039+0000","last_fullsized":"2023-10-08T05:55:27.504039+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:33.928422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503617+0000","last_change":"2023-10-08T05:55:25.927556+0000","last_active":"2023-10-08T05:55:27.503617+0000","last_peered":"2023-10-08T05:55:27.503617+0000","last_clean":"2023-10-08T05:55:27.503617+0000","last_became_active":"2023-10-08T05:55:25.927039+0000","last_became_peered":"2023-10-08T05:55:25.927039+0000","last_unstale":"2023-10-08T05:55:27.503617+0000","last_undegraded":"2023-10-08T05:55:27.503617+0000","last_fullsized":"2023-10-08T05:55:27.503617+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:49:57.883740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.925815+0000","last_change":"2023-10-08T05:55:24.917361+0000","last_active":"2023-10-08T05:55:26.925815+0000","last_peered":"2023-10-08T05:55:26.925815+0000","last_clean":"2023-10-08T05:55:26.925815+0000","last_became_active":"2023-10-08T05:55:24.917096+0000","last_became_peered":"2023-10-08T05:55:24.917096+0000","last_unstale":"2023-10-08T05:55:26.925815+0000","last_undegraded":"2023-10-08T05:55:26.925815+0000","last_fullsized":"2023-10-08T05:55:26.925815+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:16.529378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501752+0000","last_change":"2023-10-08T05:55:25.923432+0000","last_active":"2023-10-08T05:55:27.501752+0000","last_peered":"2023-10-08T05:55:27.501752+0000","last_clean":"2023-10-08T05:55:27.501752+0000","last_became_active":"2023-10-08T05:55:25.923174+0000","last_became_peered":"2023-10-08T05:55:25.923174+0000","last_unstale":"2023-10-08T05:55:27.501752+0000","last_undegraded":"2023-10-08T05:55:27.501752+0000","last_fullsized":"2023-10-08T05:55:27.501752+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:25.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501277+0000","last_change":"2023-10-08T05:55:24.922690+0000","last_active":"2023-10-08T05:55:27.501277+0000","last_peered":"2023-10-08T05:55:27.501277+0000","last_clean":"2023-10-08T05:55:27.501277+0000","last_became_active":"2023-10-08T05:55:24.922446+0000","last_became_peered":"2023-10-08T05:55:24.922446+0000","last_unstale":"2023-10-08T05:55:27.501277+0000","last_undegraded":"2023-10-08T05:55:27.501277+0000","last_fullsized":"2023-10-08T05:55:27.501277+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:38:23.135308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284768+0000","last_change":"2023-10-08T05:55:25.937349+0000","last_active":"2023-10-08T05:55:27.284768+0000","last_peered":"2023-10-08T05:55:27.284768+0000","last_clean":"2023-10-08T05:55:27.284768+0000","last_became_active":"2023-10-08T05:55:25.937058+0000","last_became_peered":"2023-10-08T05:55:25.937058+0000","last_unstale":"2023-10-08T05:55:27.284768+0000","last_undegraded":"2023-10-08T05:55:27.284768+0000","last_fullsized":"2023-10-08T05:55:27.284768+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:46.526640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"133'8","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929702+0000","last_change":"2023-10-08T05:55:24.920588+0000","last_active":"2023-10-08T05:55:26.929702+0000","last_peered":"2023-10-08T05:55:26.929702+0000","last_clean":"2023-10-08T05:55:26.929702+0000","last_became_active":"2023-10-08T05:55:24.920236+0000","last_became_peered":"2023-10-08T05:55:24.920236+0000","last_unstale":"2023-10-08T05:55:26.929702+0000","last_undegraded":"2023-10-08T05:55:26.929702+0000","last_fullsized":"2023-10-08T05:55:26.929702+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:09.794855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501186+0000","last_change":"2023-10-08T05:55:25.913187+0000","last_active":"2023-10-08T05:55:27.501186+0000","last_peered":"2023-10-08T05:55:27.501186+0000","last_clean":"2023-10-08T05:55:27.501186+0000","last_became_active":"2023-10-08T05:55:25.911155+0000","last_became_peered":"2023-10-08T05:55:25.911155+0000","last_unstale":"2023-10-08T05:55:27.501186+0000","last_undegraded":"2023-10-08T05:55:27.501186+0000","last_fullsized":"2023-10-08T05:55:27.501186+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:28:54.333926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"133'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502718+0000","last_change":"2023-10-08T05:55:24.916202+0000","last_active":"2023-10-08T05:55:27.502718+0000","last_peered":"2023-10-08T05:55:27.502718+0000","last_clean":"2023-10-08T05:55:27.502718+0000","last_became_active":"2023-10-08T05:55:24.915868+0000","last_became_peered":"2023-10-08T05:55:24.915868+0000","last_unstale":"2023-10-08T05:55:27.502718+0000","last_undegraded":"2023-10-08T05:55:27.502718+0000","last_fullsized":"2023-10-08T05:55:27.502718+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:13.269929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502543+0000","last_change":"2023-10-08T05:55:25.933722+0000","last_active":"2023-10-08T05:55:27.502543+0000","last_peered":"2023-10-08T05:55:27.502543+0000","last_clean":"2023-10-08T05:55:27.502543+0000","last_became_active":"2023-10-08T05:55:25.933409+0000","last_became_peered":"2023-10-08T05:55:25.933409+0000","last_unstale":"2023-10-08T05:55:27.502543+0000","last_undegraded":"2023-10-08T05:55:27.502543+0000","last_fullsized":"2023-10-08T05:55:27.502543+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:43:12.993926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"133'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929213+0000","last_change":"2023-10-08T05:55:24.924200+0000","last_active":"2023-10-08T05:55:26.929213+0000","last_peered":"2023-10-08T05:55:26.929213+0000","last_clean":"2023-10-08T05:55:26.929213+0000","last_became_active":"2023-10-08T05:55:24.922782+0000","last_became_peered":"2023-10-08T05:55:24.922782+0000","last_unstale":"2023-10-08T05:55:26.929213+0000","last_undegraded":"2023-10-08T05:55:26.929213+0000","last_fullsized":"2023-10-08T05:55:26.929213+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:58.749374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503803+0000","last_change":"2023-10-08T05:55:25.923884+0000","last_active":"2023-10-08T05:55:27.503803+0000","last_peered":"2023-10-08T05:55:27.503803+0000","last_clean":"2023-10-08T05:55:27.503803+0000","last_became_active":"2023-10-08T05:55:25.923617+0000","last_became_peered":"2023-10-08T05:55:25.923617+0000","last_unstale":"2023-10-08T05:55:27.503803+0000","last_undegraded":"2023-10-08T05:55:27.503803+0000","last_fullsized":"2023-10-08T05:55:27.503803+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:07.078151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503383+0000","last_change":"2023-10-08T05:55:24.928566+0000","last_active":"2023-10-08T05:55:27.503383+0000","last_peered":"2023-10-08T05:55:27.503383+0000","last_clean":"2023-10-08T05:55:27.503383+0000","last_became_active":"2023-10-08T05:55:24.928183+0000","last_became_peered":"2023-10-08T05:55:24.928183+0000","last_unstale":"2023-10-08T05:55:27.503383+0000","last_undegraded":"2023-10-08T05:55:27.503383+0000","last_fullsized":"2023-10-08T05:55:27.503383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:58.364141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284416+0000","last_change":"2023-10-08T05:55:25.908644+0000","last_active":"2023-10-08T05:55:27.284416+0000","last_peered":"2023-10-08T05:55:27.284416+0000","last_clean":"2023-10-08T05:55:27.284416+0000","last_became_active":"2023-10-08T05:55:25.908346+0000","last_became_peered":"2023-10-08T05:55:25.908346+0000","last_unstale":"2023-10-08T05:55:27.284416+0000","last_undegraded":"2023-10-08T05:55:27.284416+0000","last_fullsized":"2023-10-08T05:55:27.284416+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:51:00.669382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502145+0000","last_change":"2023-10-08T05:55:24.924397+0000","last_active":"2023-10-08T05:55:27.502145+0000","last_peered":"2023-10-08T05:55:27.502145+0000","last_clean":"2023-10-08T05:55:27.502145+0000","last_became_active":"2023-10-08T05:55:24.923977+0000","last_became_peered":"2023-10-08T05:55:24.923977+0000","last_unstale":"2023-10-08T05:55:27.502145+0000","last_undegraded":"2023-10-08T05:55:27.502145+0000","last_fullsized":"2023-10-08T05:55:27.502145+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.138903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284377+0000","last_change":"2023-10-08T05:55:25.939606+0000","last_active":"2023-10-08T05:55:27.284377+0000","last_peered":"2023-10-08T05:55:27.284377+0000","last_clean":"2023-10-08T05:55:27.284377+0000","last_became_active":"2023-10-08T05:55:25.939194+0000","last_became_peered":"2023-10-08T05:55:25.939194+0000","last_unstale":"2023-10-08T05:55:27.284377+0000","last_undegraded":"2023-10-08T05:55:27.284377+0000","last_fullsized":"2023-10-08T05:55:27.284377+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:19.783496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501063+0000","last_change":"2023-10-08T05:55:24.920985+0000","last_active":"2023-10-08T05:55:27.501063+0000","last_peered":"2023-10-08T05:55:27.501063+0000","last_clean":"2023-10-08T05:55:27.501063+0000","last_became_active":"2023-10-08T05:55:24.920649+0000","last_became_peered":"2023-10-08T05:55:24.920649+0000","last_unstale":"2023-10-08T05:55:27.501063+0000","last_undegraded":"2023-10-08T05:55:27.501063+0000","last_fullsized":"2023-10-08T05:55:27.501063+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:58:04.772705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926149+0000","last_change":"2023-10-08T05:55:25.934860+0000","last_active":"2023-10-08T05:55:26.926149+0000","last_peered":"2023-10-08T05:55:26.926149+0000","last_clean":"2023-10-08T05:55:26.926149+0000","last_became_active":"2023-10-08T05:55:25.931500+0000","last_became_peered":"2023-10-08T05:55:25.931500+0000","last_unstale":"2023-10-08T05:55:26.926149+0000","last_undegraded":"2023-10-08T05:55:26.926149+0000","last_fullsized":"2023-10-08T05:55:26.926149+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:20.072607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502232+0000","last_change":"2023-10-08T05:55:24.919252+0000","last_active":"2023-10-08T05:55:27.502232+0000","last_peered":"2023-10-08T05:55:27.502232+0000","last_clean":"2023-10-08T05:55:27.502232+0000","last_became_active":"2023-10-08T05:55:24.918968+0000","last_became_peered":"2023-10-08T05:55:24.918968+0000","last_unstale":"2023-10-08T05:55:27.502232+0000","last_undegraded":"2023-10-08T05:55:27.502232+0000","last_fullsized":"2023-10-08T05:55:27.502232+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:08.548178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502015+0000","last_change":"2023-10-08T05:55:25.921321+0000","last_active":"2023-10-08T05:55:27.502015+0000","last_peered":"2023-10-08T05:55:27.502015+0000","last_clean":"2023-10-08T05:55:27.502015+0000","last_became_active":"2023-10-08T05:55:25.921043+0000","last_became_peered":"2023-10-08T05:55:25.921043+0000","last_unstale":"2023-10-08T05:55:27.502015+0000","last_undegraded":"2023-10-08T05:55:27.502015+0000","last_fullsized":"2023-10-08T05:55:27.502015+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:08.790630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501322+0000","last_change":"2023-10-08T05:55:24.913364+0000","last_active":"2023-10-08T05:55:27.501322+0000","last_peered":"2023-10-08T05:55:27.501322+0000","last_clean":"2023-10-08T05:55:27.501322+0000","last_became_active":"2023-10-08T05:55:24.912918+0000","last_became_peered":"2023-10-08T05:55:24.912918+0000","last_unstale":"2023-10-08T05:55:27.501322+0000","last_undegraded":"2023-10-08T05:55:27.501322+0000","last_fullsized":"2023-10-08T05:55:27.501322+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:44.915337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284071+0000","last_change":"2023-10-08T05:55:25.911692+0000","last_active":"2023-10-08T05:55:27.284071+0000","last_peered":"2023-10-08T05:55:27.284071+0000","last_clean":"2023-10-08T05:55:27.284071+0000","last_became_active":"2023-10-08T05:55:25.911404+0000","last_became_peered":"2023-10-08T05:55:25.911404+0000","last_unstale":"2023-10-08T05:55:27.284071+0000","last_undegraded":"2023-10-08T05:55:27.284071+0000","last_fullsized":"2023-10-08T05:55:27.284071+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:16.285768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504049+0000","last_change":"2023-10-08T05:55:24.924205+0000","last_active":"2023-10-08T05:55:27.504049+0000","last_peered":"2023-10-08T05:55:27.504049+0000","last_clean":"2023-10-08T05:55:27.504049+0000","last_became_active":"2023-10-08T05:55:24.923923+0000","last_became_peered":"2023-10-08T05:55:24.923923+0000","last_unstale":"2023-10-08T05:55:27.504049+0000","last_undegraded":"2023-10-08T05:55:27.504049+0000","last_fullsized":"2023-10-08T05:55:27.504049+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:50.221424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929569+0000","last_change":"2023-10-08T05:55:25.924284+0000","last_active":"2023-10-08T05:55:26.929569+0000","last_peered":"2023-10-08T05:55:26.929569+0000","last_clean":"2023-10-08T05:55:26.929569+0000","last_became_active":"2023-10-08T05:55:25.923897+0000","last_became_peered":"2023-10-08T05:55:25.923897+0000","last_unstale":"2023-10-08T05:55:26.929569+0000","last_undegraded":"2023-10-08T05:55:26.929569+0000","last_fullsized":"2023-10-08T05:55:26.929569+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:08.882808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284281+0000","last_change":"2023-10-08T05:55:24.919675+0000","last_active":"2023-10-08T05:55:27.284281+0000","last_peered":"2023-10-08T05:55:27.284281+0000","last_clean":"2023-10-08T05:55:27.284281+0000","last_became_active":"2023-10-08T05:55:24.919401+0000","last_became_peered":"2023-10-08T05:55:24.919401+0000","last_unstale":"2023-10-08T05:55:27.284281+0000","last_undegraded":"2023-10-08T05:55:27.284281+0000","last_fullsized":"2023-10-08T05:55:27.284281+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:43.054211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926284+0000","last_change":"2023-10-08T05:55:25.930439+0000","last_active":"2023-10-08T05:55:26.926284+0000","last_peered":"2023-10-08T05:55:26.926284+0000","last_clean":"2023-10-08T05:55:26.926284+0000","last_became_active":"2023-10-08T05:55:25.930184+0000","last_became_peered":"2023-10-08T05:55:25.930184+0000","last_unstale":"2023-10-08T05:55:26.926284+0000","last_undegraded":"2023-10-08T05:55:26.926284+0000","last_fullsized":"2023-10-08T05:55:26.926284+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:51:00.669382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503850+0000","last_change":"2023-10-08T05:55:24.913568+0000","last_active":"2023-10-08T05:55:27.503850+0000","last_peered":"2023-10-08T05:55:27.503850+0000","last_clean":"2023-10-08T05:55:27.503850+0000","last_became_active":"2023-10-08T05:55:24.913279+0000","last_became_peered":"2023-10-08T05:55:24.913279+0000","last_unstale":"2023-10-08T05:55:27.503850+0000","last_undegraded":"2023-10-08T05:55:27.503850+0000","last_fullsized":"2023-10-08T05:55:27.503850+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:52.735465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503703+0000","last_change":"2023-10-08T05:55:25.931818+0000","last_active":"2023-10-08T05:55:27.503703+0000","last_peered":"2023-10-08T05:55:27.503703+0000","last_clean":"2023-10-08T05:55:27.503703+0000","last_became_active":"2023-10-08T05:55:25.925729+0000","last_became_peered":"2023-10-08T05:55:25.925729+0000","last_unstale":"2023-10-08T05:55:27.503703+0000","last_undegraded":"2023-10-08T05:55:27.503703+0000","last_fullsized":"2023-10-08T05:55:27.503703+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:46.526640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503815+0000","last_change":"2023-10-08T05:55:24.923066+0000","last_active":"2023-10-08T05:55:27.503815+0000","last_peered":"2023-10-08T05:55:27.503815+0000","last_clean":"2023-10-08T05:55:27.503815+0000","last_became_active":"2023-10-08T05:55:24.922810+0000","last_became_peered":"2023-10-08T05:55:24.922810+0000","last_unstale":"2023-10-08T05:55:27.503815+0000","last_undegraded":"2023-10-08T05:55:27.503815+0000","last_fullsized":"2023-10-08T05:55:27.503815+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:39.632316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502185+0000","last_change":"2023-10-08T05:55:25.926674+0000","last_active":"2023-10-08T05:55:27.502185+0000","last_peered":"2023-10-08T05:55:27.502185+0000","last_clean":"2023-10-08T05:55:27.502185+0000","last_became_active":"2023-10-08T05:55:25.926376+0000","last_became_peered":"2023-10-08T05:55:25.926376+0000","last_unstale":"2023-10-08T05:55:27.502185+0000","last_undegraded":"2023-10-08T05:55:27.502185+0000","last_fullsized":"2023-10-08T05:55:27.502185+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:09.764366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930397+0000","last_change":"2023-10-08T05:55:24.920312+0000","last_active":"2023-10-08T05:55:26.930397+0000","last_peered":"2023-10-08T05:55:26.930397+0000","last_clean":"2023-10-08T05:55:26.930397+0000","last_became_active":"2023-10-08T05:55:24.920005+0000","last_became_peered":"2023-10-08T05:55:24.920005+0000","last_unstale":"2023-10-08T05:55:26.930397+0000","last_undegraded":"2023-10-08T05:55:26.930397+0000","last_fullsized":"2023-10-08T05:55:26.930397+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:04.374851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501919+0000","last_change":"2023-10-08T05:55:25.920684+0000","last_active":"2023-10-08T05:55:27.501919+0000","last_peered":"2023-10-08T05:55:27.501919+0000","last_clean":"2023-10-08T05:55:27.501919+0000","last_became_active":"2023-10-08T05:55:25.920401+0000","last_became_peered":"2023-10-08T05:55:25.920401+0000","last_unstale":"2023-10-08T05:55:27.501919+0000","last_undegraded":"2023-10-08T05:55:27.501919+0000","last_fullsized":"2023-10-08T05:55:27.501919+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:00.047081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502129+0000","last_change":"2023-10-08T05:55:24.926301+0000","last_active":"2023-10-08T05:55:27.502129+0000","last_peered":"2023-10-08T05:55:27.502129+0000","last_clean":"2023-10-08T05:55:27.502129+0000","last_became_active":"2023-10-08T05:55:24.926045+0000","last_became_peered":"2023-10-08T05:55:24.926045+0000","last_unstale":"2023-10-08T05:55:27.502129+0000","last_undegraded":"2023-10-08T05:55:27.502129+0000","last_fullsized":"2023-10-08T05:55:27.502129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:59.640943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929598+0000","last_change":"2023-10-08T05:55:25.938224+0000","last_active":"2023-10-08T05:55:26.929598+0000","last_peered":"2023-10-08T05:55:26.929598+0000","last_clean":"2023-10-08T05:55:26.929598+0000","last_became_active":"2023-10-08T05:55:25.937958+0000","last_became_peered":"2023-10-08T05:55:25.937958+0000","last_unstale":"2023-10-08T05:55:26.929598+0000","last_undegraded":"2023-10-08T05:55:26.929598+0000","last_fullsized":"2023-10-08T05:55:26.929598+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:23.640733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502491+0000","last_change":"2023-10-08T05:55:24.915109+0000","last_active":"2023-10-08T05:55:27.502491+0000","last_peered":"2023-10-08T05:55:27.502491+0000","last_clean":"2023-10-08T05:55:27.502491+0000","last_became_active":"2023-10-08T05:55:24.914040+0000","last_became_peered":"2023-10-08T05:55:24.914040+0000","last_unstale":"2023-10-08T05:55:27.502491+0000","last_undegraded":"2023-10-08T05:55:27.502491+0000","last_fullsized":"2023-10-08T05:55:27.502491+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:24.459757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503775+0000","last_change":"2023-10-08T05:55:25.935038+0000","last_active":"2023-10-08T05:55:27.503775+0000","last_peered":"2023-10-08T05:55:27.503775+0000","last_clean":"2023-10-08T05:55:27.503775+0000","last_became_active":"2023-10-08T05:55:25.934749+0000","last_became_peered":"2023-10-08T05:55:25.934749+0000","last_unstale":"2023-10-08T05:55:27.503775+0000","last_undegraded":"2023-10-08T05:55:27.503775+0000","last_fullsized":"2023-10-08T05:55:27.503775+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:56.567240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501933+0000","last_change":"2023-10-08T05:55:24.922088+0000","last_active":"2023-10-08T05:55:27.501933+0000","last_peered":"2023-10-08T05:55:27.501933+0000","last_clean":"2023-10-08T05:55:27.501933+0000","last_became_active":"2023-10-08T05:55:24.921811+0000","last_became_peered":"2023-10-08T05:55:24.921811+0000","last_unstale":"2023-10-08T05:55:27.501933+0000","last_undegraded":"2023-10-08T05:55:27.501933+0000","last_fullsized":"2023-10-08T05:55:27.501933+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:58:04.903345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502273+0000","last_change":"2023-10-08T05:55:25.932100+0000","last_active":"2023-10-08T05:55:27.502273+0000","last_peered":"2023-10-08T05:55:27.502273+0000","last_clean":"2023-10-08T05:55:27.502273+0000","last_became_active":"2023-10-08T05:55:25.923270+0000","last_became_peered":"2023-10-08T05:55:25.923270+0000","last_unstale":"2023-10-08T05:55:27.502273+0000","last_undegraded":"2023-10-08T05:55:27.502273+0000","last_fullsized":"2023-10-08T05:55:27.502273+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:23.640733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501978+0000","last_change":"2023-10-08T05:55:24.914839+0000","last_active":"2023-10-08T05:55:27.501978+0000","last_peered":"2023-10-08T05:55:27.501978+0000","last_clean":"2023-10-08T05:55:27.501978+0000","last_became_active":"2023-10-08T05:55:24.914297+0000","last_became_peered":"2023-10-08T05:55:24.914297+0000","last_unstale":"2023-10-08T05:55:27.501978+0000","last_undegraded":"2023-10-08T05:55:27.501978+0000","last_fullsized":"2023-10-08T05:55:27.501978+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:08.021395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"122'2663","reported_seq":3086,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-08T05:55:25.913206+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:55:25.913206+0000","last_peered":"2023-10-08T05:55:25.913206+0000","last_clean":"2023-10-08T05:55:25.913206+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:55:25.913206+0000","last_undegraded":"2023-10-08T05:55:25.913206+0000","last_fullsized":"2023-10-08T05:55:25.913206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2663,"log_dups_size":0,"ondisk_log_size":2663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":1940,"num_write":329,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":1940,"num_write":329,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2663,"ondisk_log_size":2663,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575079,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55600,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316240,"statfs":{"total":96636764160,"available":96579829760,"internally_reserved":0,"allocated":2260992,"data_stored":1434161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.125,"15min":1.2030000000000001},"min":{"1min":0.85299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.728,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.153,"15min":1.216},"min":{"1min":0.74399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.7030000000000001,"5min":1.7210000000000001,"15min":1.726},"last":1.534}]},{"osd":1,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.194,"15min":1.244},"min":{"1min":1.044,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8360000000000001},"last":1.093},{"interface":"front","average":{"1min":1.367,"5min":1.2629999999999999,"15min":1.276},"min":{"1min":0.753,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.9379999999999999},"last":1.762}]},{"osd":2,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.177,"15min":1.24},"min":{"1min":0.746,"5min":0.626,"15min":0.626},"max":{"1min":1.6160000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.419,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":1.052,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.2010000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2569999999999999,"15min":1.2849999999999999},"min":{"1min":0.93999999999999995,"5min":0.71499999999999997,"15min":0.69999999999999996},"max":{"1min":1.6060000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.3460000000000001,"15min":1.3660000000000001},"min":{"1min":1.1220000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.8660000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.3109999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3400000000000001,"15min":1.2889999999999999},"min":{"1min":0.91300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.718,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.661},{"interface":"front","average":{"1min":1.339,"5min":1.351,"15min":1.3280000000000001},"min":{"1min":0.81599999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.667,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.258}]},{"osd":5,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.421,"15min":1.4039999999999999},"min":{"1min":0.98099999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9630000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.7989999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.427,"15min":1.423},"min":{"1min":1.1120000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.9139999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.887}]},{"osd":6,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":1.5489999999999999,"15min":1.524},"min":{"1min":1.1779999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8160000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.8380000000000001},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.49,"15min":1.464},"min":{"1min":1.0800000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7789999999999999,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.7170000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1667072,"data_stored":839185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.252,"15min":1.2809999999999999},"min":{"1min":0.625,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.6759999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2829999999999999,"15min":1.3129999999999999},"min":{"1min":0.69799999999999995,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.73,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.121}]},{"osd":1,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.151,"15min":1.1699999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4530000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.67800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7030000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.2949999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1359999999999999,"15min":1.1839999999999999},"min":{"1min":0.78800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.498,"5min":1.774,"15min":1.796},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.212,"15min":1.2509999999999999},"min":{"1min":0.76500000000000001,"5min":0.76400000000000001,"15min":0.58699999999999997},"max":{"1min":1.5900000000000001,"5min":1.752,"15min":1.8520000000000001},"last":0.72899999999999998}]},{"osd":3,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3069999999999999,"15min":1.367},"min":{"1min":0.73299999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.7889999999999999,"5min":2.137,"15min":2.137},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.4039999999999999,"15min":1.446},"min":{"1min":0.89700000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8380000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.5529999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.339,"15min":1.3129999999999999},"min":{"1min":0.72099999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8560000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.4019999999999999,"15min":1.3839999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.64300000000000002},"max":{"1min":1.742,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.609}]},{"osd":5,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.325,"15min":1.3320000000000001},"min":{"1min":0.88600000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.647,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.454,"5min":1.5109999999999999,"15min":1.49},"min":{"1min":0.87,"5min":0.747,"15min":0.747},"max":{"1min":1.913,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.6419999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.542,"15min":1.5089999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.77200000000000002},"max":{"1min":1.7450000000000001,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.464},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5149999999999999,"15min":1.4810000000000001},"min":{"1min":0.85999999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.8700000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.242}]}]},{"osd":5,"up_from":13,"seq":55834575078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1658880,"data_stored":843633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.0940000000000001,"15min":1.1319999999999999},"min":{"1min":0.76900000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.7490000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.02},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.163,"15min":1.202},"min":{"1min":0.83499999999999996,"5min":0.47899999999999998,"15min":0.442},"max":{"1min":1.637,"5min":1.7509999999999999,"15min":1.758},"last":1.409}]},{"osd":1,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.127,"15min":1.194},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.323,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.25,"15min":1.2669999999999999},"min":{"1min":0.85099999999999998,"5min":0.81299999999999994,"15min":0.70199999999999996},"max":{"1min":1.639,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.3100000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.228,"15min":1.2330000000000001},"min":{"1min":0.91700000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6779999999999999,"5min":1.8,"15min":1.8},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2470000000000001,"15min":1.282},"min":{"1min":0.82399999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.486,"5min":3.282,"15min":3.282},"last":1.282}]},{"osd":3,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3140000000000001,"15min":1.3380000000000001},"min":{"1min":0.88200000000000001,"5min":0.88,"15min":0.63500000000000001},"max":{"1min":1.655,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.367,"15min":1.3899999999999999},"min":{"1min":0.84499999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.712,"5min":1.919,"15min":2.2280000000000002},"last":1.5069999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.361,"15min":1.3},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.7210000000000001,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.47,"15min":1.423},"min":{"1min":0.85799999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.736,"5min":3.456,"15min":3.456},"last":1.246}]},{"osd":6,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.446,"15min":1.4019999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.782,"5min":2.105,"15min":2.359},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.4970000000000001,"15min":1.466},"min":{"1min":0.623,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.909,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.212}]},{"osd":7,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.569,"15min":1.5389999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.9470000000000001,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5720000000000001,"15min":1.554},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.877,"5min":2.2589999999999999,"15min":2.4460000000000002},"last":1.3360000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":843725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.1679999999999999,"15min":1.2529999999999999},"min":{"1min":0.78100000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.665,"5min":1.7609999999999999,"15min":1.8149999999999999},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.3120000000000001,"15min":1.365},"min":{"1min":0.86599999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5660000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.532}]},{"osd":1,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.286,"15min":1.28},"min":{"1min":0.76300000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.681,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.292},{"interface":"front","average":{"1min":1.304,"5min":1.2330000000000001,"15min":1.2210000000000001},"min":{"1min":0.98999999999999999,"5min":0.70299999999999996,"15min":0.60099999999999998},"max":{"1min":1.6839999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":1.2629999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2210000000000001,"15min":1.238},"min":{"1min":0.80900000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.659,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.198},{"interface":"front","average":{"1min":1.276,"5min":1.3360000000000001,"15min":1.345},"min":{"1min":0.96699999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.766,"5min":2.431,"15min":2.431},"last":0.95399999999999996}]},{"osd":3,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.28,"15min":1.2749999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5600000000000001,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.127},{"interface":"front","average":{"1min":1.413,"5min":1.3169999999999999,"15min":1.343},"min":{"1min":1.0169999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.764,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.5860000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.395,"15min":1.337},"min":{"1min":0.95299999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7629999999999999,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.423,"15min":1.365},"min":{"1min":0.96799999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.6830000000000001,"5min":5.4189999999999996,"15min":5.4189999999999996},"last":1.371}]},{"osd":6,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.448,"15min":1.361},"min":{"1min":0.91600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.966,"5min":4.9059999999999997,"15min":4.9059999999999997},"last":1.716},{"interface":"front","average":{"1min":1.494,"5min":1.579,"15min":1.5109999999999999},"min":{"1min":1.0109999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.829,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":1.8759999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.5600000000000001,"15min":1.468},"min":{"1min":1.1539999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8160000000000001,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":1.5},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5389999999999999,"15min":1.5209999999999999},"min":{"1min":1.0329999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2000000000000002,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.8320000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575080,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36964,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334876,"statfs":{"total":96636764160,"available":96598913024,"internally_reserved":0,"allocated":1662976,"data_stored":844215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.88900000000000001,"15min":0.91800000000000004},"min":{"1min":0.64300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3049999999999999,"5min":1.355,"15min":1.72},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.0429999999999999,"15min":1.099},"min":{"1min":0.74299999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0760000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.89900000000000002,"15min":0.96099999999999997},"min":{"1min":0.60799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.302,"5min":1.605,"15min":1.605},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1279999999999999,"15min":1.202},"min":{"1min":0.67700000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.5569999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.2250000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.99199999999999999,"15min":1.0600000000000001},"min":{"1min":0.72699999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3720000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.083,"15min":1.1399999999999999},"min":{"1min":0.85099999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.599,"5min":1.6040000000000001,"15min":1.788},"last":1.026}]},{"osd":4,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.163,"15min":1.2070000000000001},"min":{"1min":0.97999999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.458,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.173},{"interface":"front","average":{"1min":1.331,"5min":1.3380000000000001,"15min":1.379},"min":{"1min":0.82699999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5549999999999999,"5min":2.589,"15min":2.589},"last":1.7350000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.284,"15min":1.3009999999999999},"min":{"1min":1.0449999999999999,"5min":0.73899999999999999,"15min":0.73699999999999999},"max":{"1min":1.605,"5min":1.883,"15min":1.883},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.3140000000000001,"15min":1.3500000000000001},"min":{"1min":1.1599999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.532,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.389}]},{"osd":6,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.323,"15min":1.3200000000000001},"min":{"1min":1.0409999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":1.677},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.425,"15min":1.4419999999999999},"min":{"1min":1.0840000000000001,"5min":0.86799999999999999,"15min":0.86699999999999999},"max":{"1min":1.6859999999999999,"5min":2.016,"15min":2.0489999999999999},"last":1.585}]},{"osd":7,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4450000000000001,"15min":1.4870000000000001},"min":{"1min":1.25,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.661,"5min":2.0529999999999999,"15min":2.1480000000000001},"last":1.982},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.393},"min":{"1min":0.92700000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8799999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.5209999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575078,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59176,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312664,"statfs":{"total":96636764160,"available":96576167936,"internally_reserved":0,"allocated":1662976,"data_stored":843667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.78900000000000003,"15min":0.871},"min":{"1min":0.63400000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.5249999999999999,"5min":1.629,"15min":1.629},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.95399999999999996,"15min":1.036},"min":{"1min":0.64400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5660000000000001,"5min":1.5880000000000001,"15min":1.9470000000000001},"last":1.145}]},{"osd":1,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.86299999999999999,"15min":0.94399999999999995},"min":{"1min":0.69999999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3560000000000001,"5min":1.7609999999999999,"15min":2.1070000000000002},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.093,"5min":0.97399999999999998,"15min":1.0860000000000001},"min":{"1min":0.79400000000000004,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6890000000000001,"5min":1.79,"15min":2.1469999999999998},"last":1.417}]},{"osd":3,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.0069999999999999,"15min":1.1200000000000001},"min":{"1min":0.72999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.617,"5min":1.631,"15min":1.8120000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.147,"15min":1.216},"min":{"1min":0.89900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.8859999999999999,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":1.516}]},{"osd":4,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3109999999999999,"15min":1.3759999999999999},"min":{"1min":1.117,"5min":0.622,"15min":0.622},"max":{"1min":1.6679999999999999,"5min":4.593,"15min":4.593},"last":1.204},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2569999999999999,"15min":1.282},"min":{"1min":0.92600000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.629,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.2929999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.298,"15min":1.3700000000000001},"min":{"1min":1.0760000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.702,"5min":6.7140000000000004,"15min":6.7140000000000004},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.3420000000000001,"15min":1.4390000000000001},"min":{"1min":1.121,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":2.4119999999999999},"last":1.6160000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.29,"15min":1.3440000000000001},"min":{"1min":1.0109999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.8799999999999999,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.323,"15min":1.4399999999999999},"min":{"1min":1.1319999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.8400000000000001,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.0589999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3759999999999999,"15min":1.4279999999999999},"min":{"1min":1.1160000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.73,"5min":6.609,"15min":6.609},"last":1.375},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.3580000000000001,"15min":1.4850000000000001},"min":{"1min":1.1259999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9370000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.3440000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36960,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334880,"statfs":{"total":96636764160,"available":96598917120,"internally_reserved":0,"allocated":1650688,"data_stored":837003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":0.88700000000000001,"15min":0.96699999999999997},"min":{"1min":0.66100000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.796},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.98599999999999999,"15min":1.1180000000000001},"min":{"1min":0.51600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4119999999999999,"5min":1.4510000000000001,"15min":1.8089999999999999},"last":1.089}]},{"osd":2,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.871,"15min":0.95599999999999996},"min":{"1min":0.498,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.363,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.163,"5min":0.99299999999999999,"15min":1.085},"min":{"1min":0.73799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.474,"5min":1.5860000000000001,"15min":1.8049999999999999},"last":1.5349999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.0580000000000001,"15min":1.171},"min":{"1min":0.66500000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.637,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0169999999999999,"15min":1.0680000000000001},"min":{"1min":0.81899999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.6120000000000001,"5min":1.7669999999999999,"15min":1.788},"last":0.97399999999999998}]},{"osd":4,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.1799999999999999,"15min":1.2909999999999999},"min":{"1min":0.90700000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5780000000000001,"5min":1.7529999999999999,"15min":1.966},"last":1.49},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.1850000000000001,"15min":1.222},"min":{"1min":0.90600000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.671,"5min":1.913,"15min":1.913},"last":1.1339999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2569999999999999,"15min":1.3089999999999999},"min":{"1min":1.0509999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6799999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.393},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.375,"15min":1.45},"min":{"1min":1.2130000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.641}]},{"osd":6,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.343,"15min":1.4159999999999999},"min":{"1min":1.0409999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9970000000000001},"last":1.339},{"interface":"front","average":{"1min":1.389,"5min":1.3620000000000001,"15min":1.4179999999999999},"min":{"1min":1.0009999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.796,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.298}]},{"osd":7,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.3520000000000001,"15min":1.454},"min":{"1min":1.0860000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.377,"15min":1.444},"min":{"1min":1.077,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.306},"last":1.6000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575079,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33384,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338456,"statfs":{"total":96636764160,"available":96602578944,"internally_reserved":0,"allocated":2256896,"data_stored":1429039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85999999999999999,"15min":0.91100000000000003},"min":{"1min":0.65200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.3380000000000001,"5min":1.77,"15min":1.77},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.94999999999999996,"15min":1.0369999999999999},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.3109999999999999,"5min":1.6719999999999999,"15min":1.675},"last":0.82899999999999996}]},{"osd":2,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1,"15min":1.0840000000000001},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.5649999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.876,"15min":0.98699999999999999},"min":{"1min":0.55100000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4770000000000001,"5min":1.722,"15min":1.78},"last":0.874}]},{"osd":3,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0940000000000001,"15min":1.1919999999999999},"min":{"1min":0.49299999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.448,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.093},"min":{"1min":0.56399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4239999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.0249999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1779999999999999,"15min":1.262},"min":{"1min":0.71399999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.571,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.381},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2689999999999999,"15min":1.3440000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.669,"5min":1.8939999999999999,"15min":2.2200000000000002},"last":1.4279999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2549999999999999,"15min":1.3300000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.599,"5min":1.9410000000000001,"15min":2.0489999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.2969999999999999,"15min":1.3720000000000001},"min":{"1min":0.84699999999999998,"5min":0.63,"15min":0.63},"max":{"1min":1.677,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.512}]},{"osd":6,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3400000000000001,"15min":1.4119999999999999},"min":{"1min":0.68700000000000006,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.742,"5min":2.032,"15min":2.1909999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3160000000000001,"15min":1.393},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6870000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.4650000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.359,"15min":1.4319999999999999},"min":{"1min":0.755,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7709999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.782},{"interface":"front","average":{"1min":1.411,"5min":1.395,"15min":1.4750000000000001},"min":{"1min":0.73699999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8340000000000001,"5min":2.0569999999999999,"15min":2.2759999999999998},"last":1.7649999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:55:30.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:55:30.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.613+0000 7fb492db8640 1 Processor -- start 2023-10-08T05:55:30.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.613+0000 7fb492db8640 1 -- start start 2023-10-08T05:55:30.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.614+0000 7fb492db8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb48c161320 0x7fb48c0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.615+0000 7fb492db8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb48c164f70 0x7fb48c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.615+0000 7fb492db8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c165470 con 0x7fb48c06c710 2023-10-08T05:55:30.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.615+0000 7fb492db8640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c165750 con 0x7fb48c164f70 2023-10-08T05:55:30.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.615+0000 7fb492db8640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c165a30 con 0x7fb48c161320 2023-10-08T05:55:30.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.615+0000 7fb4925b7640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb48c06c710 0x7fb48c0c95b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54800/0 (socket says 172.21.15.70:54800) 2023-10-08T05:55:30.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.615+0000 7fb4925b7640 1 -- 172.21.15.70:0/1465599111 learned_addr learned my addr 172.21.15.70:0/1465599111 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.615+0000 7fb491db6640 1 --2- 172.21.15.70:0/1465599111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb48c161320 0x7fb48c0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb4915b5640 1 --2- 172.21.15.70:0/1465599111 >> v2:172.21.15.181:3300/0 conn(0x7fb48c164f70 0x7fb48c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111284330 0 0) 0x7fb48c165470 con 0x7fb48c06c710 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb478003520 con 0x7fb48c06c710 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 536395567 0 0) 0x7fb478003520 con 0x7fb48c06c710 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb48c161320 msgr2=0x7fb48c0cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 --2- 172.21.15.70:0/1465599111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb48c161320 0x7fb48c0cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 >> v2:172.21.15.181:3300/0 conn(0x7fb48c164f70 msgr2=0x7fb48c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 --2- 172.21.15.70:0/1465599111 >> v2:172.21.15.181:3300/0 conn(0x7fb48c164f70 0x7fb48c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c165d10 con 0x7fb48c06c710 2023-10-08T05:55:30.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fb4840019f0 con 0x7fb48c06c710 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 >> v1:172.21.15.70:6789/0 conn(0x7fb48c06c710 legacy=0x7fb48c0c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 --2- 172.21.15.70:0/1465599111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb478003970 0x7fb478003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 --2- 172.21.15.70:0/1465599111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4780047c0 0x7fb478004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 --2- 172.21.15.70:0/1465599111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb48c161320 0x7fb478005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c165750 con 0x7fb478003970 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c165a30 con 0x7fb4780047c0 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.616+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb47800b980 con 0x7fb48c161320 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.617+0000 7fb491db6640 1 --2- 172.21.15.70:0/1465599111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb478003970 0x7fb478003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.617+0000 7fb491db6640 1 -- 172.21.15.70:0/1465599111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb48c161320 msgr2=0x7fb478005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.617+0000 7fb491db6640 1 --2- 172.21.15.70:0/1465599111 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb48c161320 0x7fb478005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.617+0000 7fb491db6640 1 -- 172.21.15.70:0/1465599111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4780047c0 msgr2=0x7fb478004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:30.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.617+0000 7fb491db6640 1 --2- 172.21.15.70:0/1465599111 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4780047c0 0x7fb478004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.617+0000 7fb491db6640 1 -- 172.21.15.70:0/1465599111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb478003520 con 0x7fb478003970 2023-10-08T05:55:30.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.618+0000 7fb491db6640 1 --2- 172.21.15.70:0/1465599111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb478003970 0x7fb478003d90 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb488009ac0 tx=0x7fb4780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483bef598ccf4d10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:30.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.618+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48802c710 con 0x7fb478003970 2023-10-08T05:55:30.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.618+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48802c870 con 0x7fb478003970 2023-10-08T05:55:30.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.618+0000 7fb490db4640 1 -- 172.21.15.70:0/1465599111 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48802cc00 con 0x7fb478003970 2023-10-08T05:55:30.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.619+0000 7fb492db8640 1 -- 172.21.15.70:0/1465599111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb478003970 msgr2=0x7fb478003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.619+0000 7fb492db8640 1 --2- 172.21.15.70:0/1465599111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb478003970 0x7fb478003d90 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb488009ac0 tx=0x7fb4780057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.619+0000 7fb4925b7640 1 -- 172.21.15.70:0/1465599111 reap_dead start 2023-10-08T05:55:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.619+0000 7fb492db8640 1 -- 172.21.15.70:0/1465599111 shutdown_connections 2023-10-08T05:55:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.619+0000 7fb492db8640 1 -- 172.21.15.70:0/1465599111 >> 172.21.15.70:0/1465599111 conn(0x7fb48c0cb0c0 msgr2=0x7fb48c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.619+0000 7fb492db8640 1 -- 172.21.15.70:0/1465599111 shutdown_connections 2023-10-08T05:55:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.619+0000 7fb492db8640 1 -- 172.21.15.70:0/1465599111 wait complete. 2023-10-08T05:55:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.621+0000 7fb492db8640 1 Processor -- start 2023-10-08T05:55:30.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.621+0000 7fb492db8640 1 -- start start 2023-10-08T05:55:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.622+0000 7fb492db8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb478003970 0x7fb48c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.622+0000 7fb492db8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb48c1640e0 0x7fb48c1f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.622+0000 7fb492db8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb48c1f6d00 0x7fb48c1f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.622+0000 7fb492db8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c1f76c0 con 0x7fb48c1640e0 2023-10-08T05:55:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.622+0000 7fb492db8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0b3450 con 0x7fb48c1f6d00 2023-10-08T05:55:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.622+0000 7fb492db8640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c0b3170 con 0x7fb478003970 2023-10-08T05:55:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.622+0000 7fb4915b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb48c1640e0 0x7fb48c1f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.622+0000 7fb4915b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb48c1640e0 0x7fb48c1f67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54908/0 (socket says 172.21.15.70:54908) 2023-10-08T05:55:30.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.622+0000 7fb4915b5640 1 -- 172.21.15.70:0/2023525167 learned_addr learned my addr 172.21.15.70:0/2023525167 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb4915b5640 1 -- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb478003970 msgr2=0x7fb48c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb4915b5640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb478003970 0x7fb48c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb4915b5640 1 -- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb48c1f6d00 msgr2=0x7fb48c1f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb4925b7640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb48c1f6d00 0x7fb48c1f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb4915b5640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb48c1f6d00 0x7fb48c1f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb4915b5640 1 -- 172.21.15.70:0/2023525167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c161830 con 0x7fb48c1640e0 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb4915b5640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb48c1640e0 0x7fb48c1f67c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00e890 tx=0x7fb47c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb4827fc640 1 -- 172.21.15.70:0/2023525167 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c00c8e0 con 0x7fb48c1640e0 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb4827fc640 1 -- 172.21.15.70:0/2023525167 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c00ca60 con 0x7fb48c1640e0 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb4827fc640 1 -- 172.21.15.70:0/2023525167 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c00ce10 con 0x7fb48c1640e0 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.623+0000 7fb492db8640 1 -- 172.21.15.70:0/2023525167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c161b10 con 0x7fb48c1640e0 2023-10-08T05:55:30.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.624+0000 7fb492db8640 1 -- 172.21.15.70:0/2023525167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c1f7e10 con 0x7fb48c1640e0 2023-10-08T05:55:30.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.625+0000 7fb492db8640 1 -- 172.21.15.70:0/2023525167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c0c8fe0 con 0x7fb48c1640e0 2023-10-08T05:55:30.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.626+0000 7fb4827fc640 1 -- 172.21.15.70:0/2023525167 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb47c013470 con 0x7fb48c1640e0 2023-10-08T05:55:30.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.626+0000 7fb4827fc640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb47006afc0 0x7fb47006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:30.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.627+0000 7fb4827fc640 1 -- 172.21.15.70:0/2023525167 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb47c089870 con 0x7fb48c1640e0 2023-10-08T05:55:30.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.627+0000 7fb491db6640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb47006afc0 0x7fb47006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:30.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.628+0000 7fb491db6640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb47006afc0 0x7fb47006d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb488036000 tx=0x7fb48802e000 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:30.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.631+0000 7fb4827fc640 1 -- 172.21.15.70:0/2023525167 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb47c054730 con 0x7fb48c1640e0 2023-10-08T05:55:30.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.816+0000 7fb492db8640 1 -- 172.21.15.70:0/2023525167 --> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb48c04e160 con 0x7fb47006afc0 2023-10-08T05:55:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.839+0000 7fb4827fc640 1 -- 172.21.15.70:0/2023525167 <== mgr.5747 v2:172.21.15.181:6832/3142216408 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287153 (secure 0 0 0) 0x7fb48c04e160 con 0x7fb47006afc0 2023-10-08T05:55:30.843 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:30.844 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:55:30.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 -- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb47006afc0 msgr2=0x7fb47006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb47006afc0 0x7fb47006d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb488036000 tx=0x7fb48802e000 comp rx=0 tx=0).stop 2023-10-08T05:55:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 -- 172.21.15.70:0/2023525167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb48c1640e0 msgr2=0x7fb48c1f67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb48c1640e0 0x7fb48c1f67c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00e890 tx=0x7fb47c00ed50 comp rx=0 tx=0).stop 2023-10-08T05:55:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 -- 172.21.15.70:0/2023525167 shutdown_connections 2023-10-08T05:55:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb48c1f6d00 0x7fb48c1f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fb47006afc0 0x7fb47006d480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb48c1640e0 0x7fb48c1f67c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 --2- 172.21.15.70:0/2023525167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb478003970 0x7fb48c0b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 -- 172.21.15.70:0/2023525167 >> 172.21.15.70:0/2023525167 conn(0x7fb48c0cb0c0 msgr2=0x7fb48c0ae0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:30.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 -- 172.21.15.70:0/2023525167 shutdown_connections 2023-10-08T05:55:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:30.844+0000 7fb492db8640 1 -- 172.21.15.70:0/2023525167 wait complete. 2023-10-08T05:55:30.864 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-08T05:55:30.750909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":957,"num_read_kb":1940,"num_write":373,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2701,"ondisk_log_size":2701,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":324668,"kb_used_data":14148,"kb_used_omap":12,"kb_used_meta":310387,"kb_avail":754650052,"statfs":{"total":773094113280,"available":772761653248,"internally_reserved":0,"allocated":14487552,"data_stored":7914628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":317836859},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002741"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284592+0000","last_change":"2023-10-08T05:55:25.916876+0000","last_active":"2023-10-08T05:55:27.284592+0000","last_peered":"2023-10-08T05:55:27.284592+0000","last_clean":"2023-10-08T05:55:27.284592+0000","last_became_active":"2023-10-08T05:55:25.916557+0000","last_became_peered":"2023-10-08T05:55:25.916557+0000","last_unstale":"2023-10-08T05:55:27.284592+0000","last_undegraded":"2023-10-08T05:55:27.284592+0000","last_fullsized":"2023-10-08T05:55:27.284592+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:45.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503739+0000","last_change":"2023-10-08T05:55:25.928225+0000","last_active":"2023-10-08T05:55:27.503739+0000","last_peered":"2023-10-08T05:55:27.503739+0000","last_clean":"2023-10-08T05:55:27.503739+0000","last_became_active":"2023-10-08T05:55:25.927889+0000","last_became_peered":"2023-10-08T05:55:25.927889+0000","last_unstale":"2023-10-08T05:55:27.503739+0000","last_undegraded":"2023-10-08T05:55:27.503739+0000","last_fullsized":"2023-10-08T05:55:27.503739+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:21:23.883548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284430+0000","last_change":"2023-10-08T05:55:25.939516+0000","last_active":"2023-10-08T05:55:27.284430+0000","last_peered":"2023-10-08T05:55:27.284430+0000","last_clean":"2023-10-08T05:55:27.284430+0000","last_became_active":"2023-10-08T05:55:25.938748+0000","last_became_peered":"2023-10-08T05:55:25.938748+0000","last_unstale":"2023-10-08T05:55:27.284430+0000","last_undegraded":"2023-10-08T05:55:27.284430+0000","last_fullsized":"2023-10-08T05:55:27.284430+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:29:57.765972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.925973+0000","last_change":"2023-10-08T05:55:25.927966+0000","last_active":"2023-10-08T05:55:26.925973+0000","last_peered":"2023-10-08T05:55:26.925973+0000","last_clean":"2023-10-08T05:55:26.925973+0000","last_became_active":"2023-10-08T05:55:25.927714+0000","last_became_peered":"2023-10-08T05:55:25.927714+0000","last_unstale":"2023-10-08T05:55:26.925973+0000","last_undegraded":"2023-10-08T05:55:26.925973+0000","last_fullsized":"2023-10-08T05:55:26.925973+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:16.285768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503598+0000","last_change":"2023-10-08T05:55:25.921650+0000","last_active":"2023-10-08T05:55:27.503598+0000","last_peered":"2023-10-08T05:55:27.503598+0000","last_clean":"2023-10-08T05:55:27.503598+0000","last_became_active":"2023-10-08T05:55:25.921345+0000","last_became_peered":"2023-10-08T05:55:25.921345+0000","last_unstale":"2023-10-08T05:55:27.503598+0000","last_undegraded":"2023-10-08T05:55:27.503598+0000","last_fullsized":"2023-10-08T05:55:27.503598+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:50:43.904942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502181+0000","last_change":"2023-10-08T05:55:25.927148+0000","last_active":"2023-10-08T05:55:27.502181+0000","last_peered":"2023-10-08T05:55:27.502181+0000","last_clean":"2023-10-08T05:55:27.502181+0000","last_became_active":"2023-10-08T05:55:25.926795+0000","last_became_peered":"2023-10-08T05:55:25.926795+0000","last_unstale":"2023-10-08T05:55:27.502181+0000","last_undegraded":"2023-10-08T05:55:27.502181+0000","last_fullsized":"2023-10-08T05:55:27.502181+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:41:21.947765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501958+0000","last_change":"2023-10-08T05:55:25.926748+0000","last_active":"2023-10-08T05:55:27.501958+0000","last_peered":"2023-10-08T05:55:27.501958+0000","last_clean":"2023-10-08T05:55:27.501958+0000","last_became_active":"2023-10-08T05:55:25.926427+0000","last_became_peered":"2023-10-08T05:55:25.926427+0000","last_unstale":"2023-10-08T05:55:27.501958+0000","last_undegraded":"2023-10-08T05:55:27.501958+0000","last_fullsized":"2023-10-08T05:55:27.501958+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:45.716202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.500842+0000","last_change":"2023-10-08T05:55:25.921911+0000","last_active":"2023-10-08T05:55:27.500842+0000","last_peered":"2023-10-08T05:55:27.500842+0000","last_clean":"2023-10-08T05:55:27.500842+0000","last_became_active":"2023-10-08T05:55:25.921677+0000","last_became_peered":"2023-10-08T05:55:25.921677+0000","last_unstale":"2023-10-08T05:55:27.500842+0000","last_undegraded":"2023-10-08T05:55:27.500842+0000","last_fullsized":"2023-10-08T05:55:27.500842+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:07.653076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502733+0000","last_change":"2023-10-08T05:55:25.927783+0000","last_active":"2023-10-08T05:55:27.502733+0000","last_peered":"2023-10-08T05:55:27.502733+0000","last_clean":"2023-10-08T05:55:27.502733+0000","last_became_active":"2023-10-08T05:55:25.927506+0000","last_became_peered":"2023-10-08T05:55:25.927506+0000","last_unstale":"2023-10-08T05:55:27.502733+0000","last_undegraded":"2023-10-08T05:55:27.502733+0000","last_fullsized":"2023-10-08T05:55:27.502733+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:36:57.120294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504173+0000","last_change":"2023-10-08T05:55:25.929596+0000","last_active":"2023-10-08T05:55:27.504173+0000","last_peered":"2023-10-08T05:55:27.504173+0000","last_clean":"2023-10-08T05:55:27.504173+0000","last_became_active":"2023-10-08T05:55:25.929300+0000","last_became_peered":"2023-10-08T05:55:25.929300+0000","last_unstale":"2023-10-08T05:55:27.504173+0000","last_undegraded":"2023-10-08T05:55:27.504173+0000","last_fullsized":"2023-10-08T05:55:27.504173+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:16.914110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503041+0000","last_change":"2023-10-08T05:55:25.935202+0000","last_active":"2023-10-08T05:55:27.503041+0000","last_peered":"2023-10-08T05:55:27.503041+0000","last_clean":"2023-10-08T05:55:27.503041+0000","last_became_active":"2023-10-08T05:55:25.934864+0000","last_became_peered":"2023-10-08T05:55:25.934864+0000","last_unstale":"2023-10-08T05:55:27.503041+0000","last_undegraded":"2023-10-08T05:55:27.503041+0000","last_fullsized":"2023-10-08T05:55:27.503041+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:45.557259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.931011+0000","last_change":"2023-10-08T05:55:25.925077+0000","last_active":"2023-10-08T05:55:26.931011+0000","last_peered":"2023-10-08T05:55:26.931011+0000","last_clean":"2023-10-08T05:55:26.931011+0000","last_became_active":"2023-10-08T05:55:25.924802+0000","last_became_peered":"2023-10-08T05:55:25.924802+0000","last_unstale":"2023-10-08T05:55:26.931011+0000","last_undegraded":"2023-10-08T05:55:26.931011+0000","last_fullsized":"2023-10-08T05:55:26.931011+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:55.785568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926061+0000","last_change":"2023-10-08T05:55:25.937597+0000","last_active":"2023-10-08T05:55:26.926061+0000","last_peered":"2023-10-08T05:55:26.926061+0000","last_clean":"2023-10-08T05:55:26.926061+0000","last_became_active":"2023-10-08T05:55:25.935914+0000","last_became_peered":"2023-10-08T05:55:25.935914+0000","last_unstale":"2023-10-08T05:55:26.926061+0000","last_undegraded":"2023-10-08T05:55:26.926061+0000","last_fullsized":"2023-10-08T05:55:26.926061+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:59.379656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.928756+0000","last_change":"2023-10-08T05:55:25.921912+0000","last_active":"2023-10-08T05:55:26.928756+0000","last_peered":"2023-10-08T05:55:26.928756+0000","last_clean":"2023-10-08T05:55:26.928756+0000","last_became_active":"2023-10-08T05:55:25.921560+0000","last_became_peered":"2023-10-08T05:55:25.921560+0000","last_unstale":"2023-10-08T05:55:26.928756+0000","last_undegraded":"2023-10-08T05:55:26.928756+0000","last_fullsized":"2023-10-08T05:55:26.928756+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:29:57.765972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.928582+0000","last_change":"2023-10-08T05:55:25.916817+0000","last_active":"2023-10-08T05:55:26.928582+0000","last_peered":"2023-10-08T05:55:26.928582+0000","last_clean":"2023-10-08T05:55:26.928582+0000","last_became_active":"2023-10-08T05:55:25.916513+0000","last_became_peered":"2023-10-08T05:55:25.916513+0000","last_unstale":"2023-10-08T05:55:26.928582+0000","last_undegraded":"2023-10-08T05:55:26.928582+0000","last_fullsized":"2023-10-08T05:55:26.928582+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:51:00.669382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.928409+0000","last_change":"2023-10-08T05:55:25.920621+0000","last_active":"2023-10-08T05:55:26.928409+0000","last_peered":"2023-10-08T05:55:26.928409+0000","last_clean":"2023-10-08T05:55:26.928409+0000","last_became_active":"2023-10-08T05:55:25.920298+0000","last_became_peered":"2023-10-08T05:55:25.920298+0000","last_unstale":"2023-10-08T05:55:26.928409+0000","last_undegraded":"2023-10-08T05:55:26.928409+0000","last_fullsized":"2023-10-08T05:55:26.928409+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:20.072607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284677+0000","last_change":"2023-10-08T05:55:25.939572+0000","last_active":"2023-10-08T05:55:27.284677+0000","last_peered":"2023-10-08T05:55:27.284677+0000","last_clean":"2023-10-08T05:55:27.284677+0000","last_became_active":"2023-10-08T05:55:25.939279+0000","last_became_peered":"2023-10-08T05:55:25.939279+0000","last_unstale":"2023-10-08T05:55:27.284677+0000","last_undegraded":"2023-10-08T05:55:27.284677+0000","last_fullsized":"2023-10-08T05:55:27.284677+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:19:06.059847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504051+0000","last_change":"2023-10-08T05:55:25.928178+0000","last_active":"2023-10-08T05:55:27.504051+0000","last_peered":"2023-10-08T05:55:27.504051+0000","last_clean":"2023-10-08T05:55:27.504051+0000","last_became_active":"2023-10-08T05:55:25.927795+0000","last_became_peered":"2023-10-08T05:55:25.927795+0000","last_unstale":"2023-10-08T05:55:27.504051+0000","last_undegraded":"2023-10-08T05:55:27.504051+0000","last_fullsized":"2023-10-08T05:55:27.504051+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:45.993171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284188+0000","last_change":"2023-10-08T05:55:25.929932+0000","last_active":"2023-10-08T05:55:27.284188+0000","last_peered":"2023-10-08T05:55:27.284188+0000","last_clean":"2023-10-08T05:55:27.284188+0000","last_became_active":"2023-10-08T05:55:25.929677+0000","last_became_peered":"2023-10-08T05:55:25.929677+0000","last_unstale":"2023-10-08T05:55:27.284188+0000","last_undegraded":"2023-10-08T05:55:27.284188+0000","last_fullsized":"2023-10-08T05:55:27.284188+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:59.379656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.931241+0000","last_change":"2023-10-08T05:55:25.924986+0000","last_active":"2023-10-08T05:55:26.931241+0000","last_peered":"2023-10-08T05:55:26.931241+0000","last_clean":"2023-10-08T05:55:26.931241+0000","last_became_active":"2023-10-08T05:55:25.923671+0000","last_became_peered":"2023-10-08T05:55:25.923671+0000","last_unstale":"2023-10-08T05:55:26.931241+0000","last_undegraded":"2023-10-08T05:55:26.931241+0000","last_fullsized":"2023-10-08T05:55:26.931241+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:12:34.832429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502246+0000","last_change":"2023-10-08T05:55:25.927866+0000","last_active":"2023-10-08T05:55:27.502246+0000","last_peered":"2023-10-08T05:55:27.502246+0000","last_clean":"2023-10-08T05:55:27.502246+0000","last_became_active":"2023-10-08T05:55:25.927611+0000","last_became_peered":"2023-10-08T05:55:25.927611+0000","last_unstale":"2023-10-08T05:55:27.502246+0000","last_undegraded":"2023-10-08T05:55:27.502246+0000","last_fullsized":"2023-10-08T05:55:27.502246+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:51.637826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503507+0000","last_change":"2023-10-08T05:55:25.926358+0000","last_active":"2023-10-08T05:55:27.503507+0000","last_peered":"2023-10-08T05:55:27.503507+0000","last_clean":"2023-10-08T05:55:27.503507+0000","last_became_active":"2023-10-08T05:55:25.924421+0000","last_became_peered":"2023-10-08T05:55:25.924421+0000","last_unstale":"2023-10-08T05:55:27.503507+0000","last_undegraded":"2023-10-08T05:55:27.503507+0000","last_fullsized":"2023-10-08T05:55:27.503507+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:30.550731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.283984+0000","last_change":"2023-10-08T05:55:25.919782+0000","last_active":"2023-10-08T05:55:27.283984+0000","last_peered":"2023-10-08T05:55:27.283984+0000","last_clean":"2023-10-08T05:55:27.283984+0000","last_became_active":"2023-10-08T05:55:25.918562+0000","last_became_peered":"2023-10-08T05:55:25.918562+0000","last_unstale":"2023-10-08T05:55:27.283984+0000","last_undegraded":"2023-10-08T05:55:27.283984+0000","last_fullsized":"2023-10-08T05:55:27.283984+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:59.764889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502769+0000","last_change":"2023-10-08T05:55:25.919390+0000","last_active":"2023-10-08T05:55:27.502769+0000","last_peered":"2023-10-08T05:55:27.502769+0000","last_clean":"2023-10-08T05:55:27.502769+0000","last_became_active":"2023-10-08T05:55:25.919114+0000","last_became_peered":"2023-10-08T05:55:25.919114+0000","last_unstale":"2023-10-08T05:55:27.502769+0000","last_undegraded":"2023-10-08T05:55:27.502769+0000","last_fullsized":"2023-10-08T05:55:27.502769+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:55.785568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930113+0000","last_change":"2023-10-08T05:55:25.938070+0000","last_active":"2023-10-08T05:55:26.930113+0000","last_peered":"2023-10-08T05:55:26.930113+0000","last_clean":"2023-10-08T05:55:26.930113+0000","last_became_active":"2023-10-08T05:55:25.937803+0000","last_became_peered":"2023-10-08T05:55:25.937803+0000","last_unstale":"2023-10-08T05:55:26.930113+0000","last_undegraded":"2023-10-08T05:55:26.930113+0000","last_fullsized":"2023-10-08T05:55:26.930113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:51.236939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930771+0000","last_change":"2023-10-08T05:55:25.929131+0000","last_active":"2023-10-08T05:55:26.930771+0000","last_peered":"2023-10-08T05:55:26.930771+0000","last_clean":"2023-10-08T05:55:26.930771+0000","last_became_active":"2023-10-08T05:55:25.928857+0000","last_became_peered":"2023-10-08T05:55:25.928857+0000","last_unstale":"2023-10-08T05:55:26.930771+0000","last_undegraded":"2023-10-08T05:55:26.930771+0000","last_fullsized":"2023-10-08T05:55:26.930771+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:40.647831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503107+0000","last_change":"2023-10-08T05:55:25.924553+0000","last_active":"2023-10-08T05:55:27.503107+0000","last_peered":"2023-10-08T05:55:27.503107+0000","last_clean":"2023-10-08T05:55:27.503107+0000","last_became_active":"2023-10-08T05:55:25.919977+0000","last_became_peered":"2023-10-08T05:55:25.919977+0000","last_unstale":"2023-10-08T05:55:27.503107+0000","last_undegraded":"2023-10-08T05:55:27.503107+0000","last_fullsized":"2023-10-08T05:55:27.503107+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:19:06.059847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503245+0000","last_change":"2023-10-08T05:55:25.939665+0000","last_active":"2023-10-08T05:55:27.503245+0000","last_peered":"2023-10-08T05:55:27.503245+0000","last_clean":"2023-10-08T05:55:27.503245+0000","last_became_active":"2023-10-08T05:55:25.939387+0000","last_became_peered":"2023-10-08T05:55:25.939387+0000","last_unstale":"2023-10-08T05:55:27.503245+0000","last_undegraded":"2023-10-08T05:55:27.503245+0000","last_fullsized":"2023-10-08T05:55:27.503245+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:00:58.693510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501890+0000","last_change":"2023-10-08T05:55:25.925332+0000","last_active":"2023-10-08T05:55:27.501890+0000","last_peered":"2023-10-08T05:55:27.501890+0000","last_clean":"2023-10-08T05:55:27.501890+0000","last_became_active":"2023-10-08T05:55:25.925065+0000","last_became_peered":"2023-10-08T05:55:25.925065+0000","last_unstale":"2023-10-08T05:55:27.501890+0000","last_undegraded":"2023-10-08T05:55:27.501890+0000","last_fullsized":"2023-10-08T05:55:27.501890+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:52.152295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930235+0000","last_change":"2023-10-08T05:55:25.922604+0000","last_active":"2023-10-08T05:55:26.930235+0000","last_peered":"2023-10-08T05:55:26.930235+0000","last_clean":"2023-10-08T05:55:26.930235+0000","last_became_active":"2023-10-08T05:55:25.922304+0000","last_became_peered":"2023-10-08T05:55:25.922304+0000","last_unstale":"2023-10-08T05:55:26.930235+0000","last_undegraded":"2023-10-08T05:55:26.930235+0000","last_fullsized":"2023-10-08T05:55:26.930235+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:34.943938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502312+0000","last_change":"2023-10-08T05:55:25.928946+0000","last_active":"2023-10-08T05:55:27.502312+0000","last_peered":"2023-10-08T05:55:27.502312+0000","last_clean":"2023-10-08T05:55:27.502312+0000","last_became_active":"2023-10-08T05:55:25.928670+0000","last_became_peered":"2023-10-08T05:55:25.928670+0000","last_unstale":"2023-10-08T05:55:27.502312+0000","last_undegraded":"2023-10-08T05:55:27.502312+0000","last_fullsized":"2023-10-08T05:55:27.502312+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:14.777452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501088+0000","last_change":"2023-10-08T05:55:25.922881+0000","last_active":"2023-10-08T05:55:27.501088+0000","last_peered":"2023-10-08T05:55:27.501088+0000","last_clean":"2023-10-08T05:55:27.501088+0000","last_became_active":"2023-10-08T05:55:25.922606+0000","last_became_peered":"2023-10-08T05:55:25.922606+0000","last_unstale":"2023-10-08T05:55:27.501088+0000","last_undegraded":"2023-10-08T05:55:27.501088+0000","last_fullsized":"2023-10-08T05:55:27.501088+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:40.364816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284414+0000","last_change":"2023-10-08T05:55:25.936959+0000","last_active":"2023-10-08T05:55:27.284414+0000","last_peered":"2023-10-08T05:55:27.284414+0000","last_clean":"2023-10-08T05:55:27.284414+0000","last_became_active":"2023-10-08T05:55:25.936308+0000","last_became_peered":"2023-10-08T05:55:25.936308+0000","last_unstale":"2023-10-08T05:55:27.284414+0000","last_undegraded":"2023-10-08T05:55:27.284414+0000","last_fullsized":"2023-10-08T05:55:27.284414+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:20.072607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"133'4","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926146+0000","last_change":"2023-10-08T05:55:24.921548+0000","last_active":"2023-10-08T05:55:26.926146+0000","last_peered":"2023-10-08T05:55:26.926146+0000","last_clean":"2023-10-08T05:55:26.926146+0000","last_became_active":"2023-10-08T05:55:24.921266+0000","last_became_peered":"2023-10-08T05:55:24.921266+0000","last_unstale":"2023-10-08T05:55:26.926146+0000","last_undegraded":"2023-10-08T05:55:26.926146+0000","last_fullsized":"2023-10-08T05:55:26.926146+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:58.749374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501133+0000","last_change":"2023-10-08T05:55:25.922570+0000","last_active":"2023-10-08T05:55:27.501133+0000","last_peered":"2023-10-08T05:55:27.501133+0000","last_clean":"2023-10-08T05:55:27.501133+0000","last_became_active":"2023-10-08T05:55:25.922307+0000","last_became_peered":"2023-10-08T05:55:25.922307+0000","last_unstale":"2023-10-08T05:55:27.501133+0000","last_undegraded":"2023-10-08T05:55:27.501133+0000","last_fullsized":"2023-10-08T05:55:27.501133+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:45.680629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"133'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501732+0000","last_change":"2023-10-08T05:55:24.924415+0000","last_active":"2023-10-08T05:55:27.501732+0000","last_peered":"2023-10-08T05:55:27.501732+0000","last_clean":"2023-10-08T05:55:27.501732+0000","last_became_active":"2023-10-08T05:55:24.924175+0000","last_became_peered":"2023-10-08T05:55:24.924175+0000","last_unstale":"2023-10-08T05:55:27.501732+0000","last_undegraded":"2023-10-08T05:55:27.501732+0000","last_fullsized":"2023-10-08T05:55:27.501732+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:56.312969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930341+0000","last_change":"2023-10-08T05:55:25.925746+0000","last_active":"2023-10-08T05:55:26.930341+0000","last_peered":"2023-10-08T05:55:26.930341+0000","last_clean":"2023-10-08T05:55:26.930341+0000","last_became_active":"2023-10-08T05:55:25.925454+0000","last_became_peered":"2023-10-08T05:55:25.925454+0000","last_unstale":"2023-10-08T05:55:26.930341+0000","last_undegraded":"2023-10-08T05:55:26.930341+0000","last_fullsized":"2023-10-08T05:55:26.930341+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:37.759236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930642+0000","last_change":"2023-10-08T05:55:24.925389+0000","last_active":"2023-10-08T05:55:26.930642+0000","last_peered":"2023-10-08T05:55:26.930642+0000","last_clean":"2023-10-08T05:55:26.930642+0000","last_became_active":"2023-10-08T05:55:24.925096+0000","last_became_peered":"2023-10-08T05:55:24.925096+0000","last_unstale":"2023-10-08T05:55:26.930642+0000","last_undegraded":"2023-10-08T05:55:26.930642+0000","last_fullsized":"2023-10-08T05:55:26.930642+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:03.175876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504059+0000","last_change":"2023-10-08T05:55:25.924368+0000","last_active":"2023-10-08T05:55:27.504059+0000","last_peered":"2023-10-08T05:55:27.504059+0000","last_clean":"2023-10-08T05:55:27.504059+0000","last_became_active":"2023-10-08T05:55:25.924039+0000","last_became_peered":"2023-10-08T05:55:25.924039+0000","last_unstale":"2023-10-08T05:55:27.504059+0000","last_undegraded":"2023-10-08T05:55:27.504059+0000","last_fullsized":"2023-10-08T05:55:27.504059+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:08.047146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504057+0000","last_change":"2023-10-08T05:55:24.923839+0000","last_active":"2023-10-08T05:55:27.504057+0000","last_peered":"2023-10-08T05:55:27.504057+0000","last_clean":"2023-10-08T05:55:27.504057+0000","last_became_active":"2023-10-08T05:55:24.923549+0000","last_became_peered":"2023-10-08T05:55:24.923549+0000","last_unstale":"2023-10-08T05:55:27.504057+0000","last_undegraded":"2023-10-08T05:55:27.504057+0000","last_fullsized":"2023-10-08T05:55:27.504057+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:36.743721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926050+0000","last_change":"2023-10-08T05:55:25.925785+0000","last_active":"2023-10-08T05:55:26.926050+0000","last_peered":"2023-10-08T05:55:26.926050+0000","last_clean":"2023-10-08T05:55:26.926050+0000","last_became_active":"2023-10-08T05:55:25.925496+0000","last_became_peered":"2023-10-08T05:55:25.925496+0000","last_unstale":"2023-10-08T05:55:26.926050+0000","last_undegraded":"2023-10-08T05:55:26.926050+0000","last_fullsized":"2023-10-08T05:55:26.926050+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:19.783496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"133'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.928678+0000","last_change":"2023-10-08T05:55:24.923723+0000","last_active":"2023-10-08T05:55:26.928678+0000","last_peered":"2023-10-08T05:55:26.928678+0000","last_clean":"2023-10-08T05:55:26.928678+0000","last_became_active":"2023-10-08T05:55:24.923386+0000","last_became_peered":"2023-10-08T05:55:24.923386+0000","last_unstale":"2023-10-08T05:55:26.928678+0000","last_undegraded":"2023-10-08T05:55:26.928678+0000","last_fullsized":"2023-10-08T05:55:26.928678+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:58.364141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284539+0000","last_change":"2023-10-08T05:55:25.929326+0000","last_active":"2023-10-08T05:55:27.284539+0000","last_peered":"2023-10-08T05:55:27.284539+0000","last_clean":"2023-10-08T05:55:27.284539+0000","last_became_active":"2023-10-08T05:55:25.929051+0000","last_became_peered":"2023-10-08T05:55:25.929051+0000","last_unstale":"2023-10-08T05:55:27.284539+0000","last_undegraded":"2023-10-08T05:55:27.284539+0000","last_fullsized":"2023-10-08T05:55:27.284539+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:23.640733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"133'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502351+0000","last_change":"2023-10-08T05:55:24.922495+0000","last_active":"2023-10-08T05:55:27.502351+0000","last_peered":"2023-10-08T05:55:27.502351+0000","last_clean":"2023-10-08T05:55:27.502351+0000","last_became_active":"2023-10-08T05:55:24.921502+0000","last_became_peered":"2023-10-08T05:55:24.921502+0000","last_unstale":"2023-10-08T05:55:27.502351+0000","last_undegraded":"2023-10-08T05:55:27.502351+0000","last_fullsized":"2023-10-08T05:55:27.502351+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:50:59.653866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504170+0000","last_change":"2023-10-08T05:55:25.940045+0000","last_active":"2023-10-08T05:55:27.504170+0000","last_peered":"2023-10-08T05:55:27.504170+0000","last_clean":"2023-10-08T05:55:27.504170+0000","last_became_active":"2023-10-08T05:55:25.939782+0000","last_became_peered":"2023-10-08T05:55:25.939782+0000","last_unstale":"2023-10-08T05:55:27.504170+0000","last_undegraded":"2023-10-08T05:55:27.504170+0000","last_fullsized":"2023-10-08T05:55:27.504170+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:06.114904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929762+0000","last_change":"2023-10-08T05:55:24.920669+0000","last_active":"2023-10-08T05:55:26.929762+0000","last_peered":"2023-10-08T05:55:26.929762+0000","last_clean":"2023-10-08T05:55:26.929762+0000","last_became_active":"2023-10-08T05:55:24.920369+0000","last_became_peered":"2023-10-08T05:55:24.920369+0000","last_unstale":"2023-10-08T05:55:26.929762+0000","last_undegraded":"2023-10-08T05:55:26.929762+0000","last_fullsized":"2023-10-08T05:55:26.929762+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:35:21.616608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926045+0000","last_change":"2023-10-08T05:55:25.918315+0000","last_active":"2023-10-08T05:55:26.926045+0000","last_peered":"2023-10-08T05:55:26.926045+0000","last_clean":"2023-10-08T05:55:26.926045+0000","last_became_active":"2023-10-08T05:55:25.918032+0000","last_became_peered":"2023-10-08T05:55:25.918032+0000","last_unstale":"2023-10-08T05:55:26.926045+0000","last_undegraded":"2023-10-08T05:55:26.926045+0000","last_fullsized":"2023-10-08T05:55:26.926045+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:09:49.886504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.925630+0000","last_change":"2023-10-08T05:55:24.918727+0000","last_active":"2023-10-08T05:55:26.925630+0000","last_peered":"2023-10-08T05:55:26.925630+0000","last_clean":"2023-10-08T05:55:26.925630+0000","last_became_active":"2023-10-08T05:55:24.918439+0000","last_became_peered":"2023-10-08T05:55:24.918439+0000","last_unstale":"2023-10-08T05:55:26.925630+0000","last_undegraded":"2023-10-08T05:55:26.925630+0000","last_fullsized":"2023-10-08T05:55:26.925630+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:28.648179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284113+0000","last_change":"2023-10-08T05:55:25.937157+0000","last_active":"2023-10-08T05:55:27.284113+0000","last_peered":"2023-10-08T05:55:27.284113+0000","last_clean":"2023-10-08T05:55:27.284113+0000","last_became_active":"2023-10-08T05:55:25.936672+0000","last_became_peered":"2023-10-08T05:55:25.936672+0000","last_unstale":"2023-10-08T05:55:27.284113+0000","last_undegraded":"2023-10-08T05:55:27.284113+0000","last_fullsized":"2023-10-08T05:55:27.284113+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:09:49.886504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930364+0000","last_change":"2023-10-08T05:55:24.924797+0000","last_active":"2023-10-08T05:55:26.930364+0000","last_peered":"2023-10-08T05:55:26.930364+0000","last_clean":"2023-10-08T05:55:26.930364+0000","last_became_active":"2023-10-08T05:55:24.922092+0000","last_became_peered":"2023-10-08T05:55:24.922092+0000","last_unstale":"2023-10-08T05:55:26.930364+0000","last_undegraded":"2023-10-08T05:55:26.930364+0000","last_fullsized":"2023-10-08T05:55:26.930364+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:47:42.546146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503078+0000","last_change":"2023-10-08T05:55:25.923194+0000","last_active":"2023-10-08T05:55:27.503078+0000","last_peered":"2023-10-08T05:55:27.503078+0000","last_clean":"2023-10-08T05:55:27.503078+0000","last_became_active":"2023-10-08T05:55:25.922940+0000","last_became_peered":"2023-10-08T05:55:25.922940+0000","last_unstale":"2023-10-08T05:55:27.503078+0000","last_undegraded":"2023-10-08T05:55:27.503078+0000","last_fullsized":"2023-10-08T05:55:27.503078+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:12:34.832429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284158+0000","last_change":"2023-10-08T05:55:24.916097+0000","last_active":"2023-10-08T05:55:27.284158+0000","last_peered":"2023-10-08T05:55:27.284158+0000","last_clean":"2023-10-08T05:55:27.284158+0000","last_became_active":"2023-10-08T05:55:24.915832+0000","last_became_peered":"2023-10-08T05:55:24.915832+0000","last_unstale":"2023-10-08T05:55:27.284158+0000","last_undegraded":"2023-10-08T05:55:27.284158+0000","last_fullsized":"2023-10-08T05:55:27.284158+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:15:28.229277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501536+0000","last_change":"2023-10-08T05:55:25.923432+0000","last_active":"2023-10-08T05:55:27.501536+0000","last_peered":"2023-10-08T05:55:27.501536+0000","last_clean":"2023-10-08T05:55:27.501536+0000","last_became_active":"2023-10-08T05:55:25.923160+0000","last_became_peered":"2023-10-08T05:55:25.923160+0000","last_unstale":"2023-10-08T05:55:27.501536+0000","last_undegraded":"2023-10-08T05:55:27.501536+0000","last_fullsized":"2023-10-08T05:55:27.501536+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:44:53.071337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"133'3","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929072+0000","last_change":"2023-10-08T05:55:24.924531+0000","last_active":"2023-10-08T05:55:26.929072+0000","last_peered":"2023-10-08T05:55:26.929072+0000","last_clean":"2023-10-08T05:55:26.929072+0000","last_became_active":"2023-10-08T05:55:24.924220+0000","last_became_peered":"2023-10-08T05:55:24.924220+0000","last_unstale":"2023-10-08T05:55:26.929072+0000","last_undegraded":"2023-10-08T05:55:26.929072+0000","last_fullsized":"2023-10-08T05:55:26.929072+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:50.957382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503972+0000","last_change":"2023-10-08T05:55:25.920338+0000","last_active":"2023-10-08T05:55:27.503972+0000","last_peered":"2023-10-08T05:55:27.503972+0000","last_clean":"2023-10-08T05:55:27.503972+0000","last_became_active":"2023-10-08T05:55:25.920031+0000","last_became_peered":"2023-10-08T05:55:25.920031+0000","last_unstale":"2023-10-08T05:55:27.503972+0000","last_undegraded":"2023-10-08T05:55:27.503972+0000","last_fullsized":"2023-10-08T05:55:27.503972+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:53:04.369670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"133'6","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503235+0000","last_change":"2023-10-08T05:55:24.912893+0000","last_active":"2023-10-08T05:55:27.503235+0000","last_peered":"2023-10-08T05:55:27.503235+0000","last_clean":"2023-10-08T05:55:27.503235+0000","last_became_active":"2023-10-08T05:55:24.912587+0000","last_became_peered":"2023-10-08T05:55:24.912587+0000","last_unstale":"2023-10-08T05:55:27.503235+0000","last_undegraded":"2023-10-08T05:55:27.503235+0000","last_fullsized":"2023-10-08T05:55:27.503235+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:52.463852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503974+0000","last_change":"2023-10-08T05:55:25.924248+0000","last_active":"2023-10-08T05:55:27.503974+0000","last_peered":"2023-10-08T05:55:27.503974+0000","last_clean":"2023-10-08T05:55:27.503974+0000","last_became_active":"2023-10-08T05:55:25.923353+0000","last_became_peered":"2023-10-08T05:55:25.923353+0000","last_unstale":"2023-10-08T05:55:27.503974+0000","last_undegraded":"2023-10-08T05:55:27.503974+0000","last_fullsized":"2023-10-08T05:55:27.503974+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:25.180936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504039+0000","last_change":"2023-10-08T05:55:24.925089+0000","last_active":"2023-10-08T05:55:27.504039+0000","last_peered":"2023-10-08T05:55:27.504039+0000","last_clean":"2023-10-08T05:55:27.504039+0000","last_became_active":"2023-10-08T05:55:24.924351+0000","last_became_peered":"2023-10-08T05:55:24.924351+0000","last_unstale":"2023-10-08T05:55:27.504039+0000","last_undegraded":"2023-10-08T05:55:27.504039+0000","last_fullsized":"2023-10-08T05:55:27.504039+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:33.928422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503617+0000","last_change":"2023-10-08T05:55:25.927556+0000","last_active":"2023-10-08T05:55:27.503617+0000","last_peered":"2023-10-08T05:55:27.503617+0000","last_clean":"2023-10-08T05:55:27.503617+0000","last_became_active":"2023-10-08T05:55:25.927039+0000","last_became_peered":"2023-10-08T05:55:25.927039+0000","last_unstale":"2023-10-08T05:55:27.503617+0000","last_undegraded":"2023-10-08T05:55:27.503617+0000","last_fullsized":"2023-10-08T05:55:27.503617+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:49:57.883740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.925815+0000","last_change":"2023-10-08T05:55:24.917361+0000","last_active":"2023-10-08T05:55:26.925815+0000","last_peered":"2023-10-08T05:55:26.925815+0000","last_clean":"2023-10-08T05:55:26.925815+0000","last_became_active":"2023-10-08T05:55:24.917096+0000","last_became_peered":"2023-10-08T05:55:24.917096+0000","last_unstale":"2023-10-08T05:55:26.925815+0000","last_undegraded":"2023-10-08T05:55:26.925815+0000","last_fullsized":"2023-10-08T05:55:26.925815+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:16.529378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501752+0000","last_change":"2023-10-08T05:55:25.923432+0000","last_active":"2023-10-08T05:55:27.501752+0000","last_peered":"2023-10-08T05:55:27.501752+0000","last_clean":"2023-10-08T05:55:27.501752+0000","last_became_active":"2023-10-08T05:55:25.923174+0000","last_became_peered":"2023-10-08T05:55:25.923174+0000","last_unstale":"2023-10-08T05:55:27.501752+0000","last_undegraded":"2023-10-08T05:55:27.501752+0000","last_fullsized":"2023-10-08T05:55:27.501752+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:25.366560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501277+0000","last_change":"2023-10-08T05:55:24.922690+0000","last_active":"2023-10-08T05:55:27.501277+0000","last_peered":"2023-10-08T05:55:27.501277+0000","last_clean":"2023-10-08T05:55:27.501277+0000","last_became_active":"2023-10-08T05:55:24.922446+0000","last_became_peered":"2023-10-08T05:55:24.922446+0000","last_unstale":"2023-10-08T05:55:27.501277+0000","last_undegraded":"2023-10-08T05:55:27.501277+0000","last_fullsized":"2023-10-08T05:55:27.501277+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:38:23.135308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284768+0000","last_change":"2023-10-08T05:55:25.937349+0000","last_active":"2023-10-08T05:55:27.284768+0000","last_peered":"2023-10-08T05:55:27.284768+0000","last_clean":"2023-10-08T05:55:27.284768+0000","last_became_active":"2023-10-08T05:55:25.937058+0000","last_became_peered":"2023-10-08T05:55:25.937058+0000","last_unstale":"2023-10-08T05:55:27.284768+0000","last_undegraded":"2023-10-08T05:55:27.284768+0000","last_fullsized":"2023-10-08T05:55:27.284768+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:46.526640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"133'8","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929702+0000","last_change":"2023-10-08T05:55:24.920588+0000","last_active":"2023-10-08T05:55:26.929702+0000","last_peered":"2023-10-08T05:55:26.929702+0000","last_clean":"2023-10-08T05:55:26.929702+0000","last_became_active":"2023-10-08T05:55:24.920236+0000","last_became_peered":"2023-10-08T05:55:24.920236+0000","last_unstale":"2023-10-08T05:55:26.929702+0000","last_undegraded":"2023-10-08T05:55:26.929702+0000","last_fullsized":"2023-10-08T05:55:26.929702+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:09.794855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501186+0000","last_change":"2023-10-08T05:55:25.913187+0000","last_active":"2023-10-08T05:55:27.501186+0000","last_peered":"2023-10-08T05:55:27.501186+0000","last_clean":"2023-10-08T05:55:27.501186+0000","last_became_active":"2023-10-08T05:55:25.911155+0000","last_became_peered":"2023-10-08T05:55:25.911155+0000","last_unstale":"2023-10-08T05:55:27.501186+0000","last_undegraded":"2023-10-08T05:55:27.501186+0000","last_fullsized":"2023-10-08T05:55:27.501186+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:28:54.333926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"133'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502718+0000","last_change":"2023-10-08T05:55:24.916202+0000","last_active":"2023-10-08T05:55:27.502718+0000","last_peered":"2023-10-08T05:55:27.502718+0000","last_clean":"2023-10-08T05:55:27.502718+0000","last_became_active":"2023-10-08T05:55:24.915868+0000","last_became_peered":"2023-10-08T05:55:24.915868+0000","last_unstale":"2023-10-08T05:55:27.502718+0000","last_undegraded":"2023-10-08T05:55:27.502718+0000","last_fullsized":"2023-10-08T05:55:27.502718+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:13.269929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502543+0000","last_change":"2023-10-08T05:55:25.933722+0000","last_active":"2023-10-08T05:55:27.502543+0000","last_peered":"2023-10-08T05:55:27.502543+0000","last_clean":"2023-10-08T05:55:27.502543+0000","last_became_active":"2023-10-08T05:55:25.933409+0000","last_became_peered":"2023-10-08T05:55:25.933409+0000","last_unstale":"2023-10-08T05:55:27.502543+0000","last_undegraded":"2023-10-08T05:55:27.502543+0000","last_fullsized":"2023-10-08T05:55:27.502543+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:43:12.993926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"133'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929213+0000","last_change":"2023-10-08T05:55:24.924200+0000","last_active":"2023-10-08T05:55:26.929213+0000","last_peered":"2023-10-08T05:55:26.929213+0000","last_clean":"2023-10-08T05:55:26.929213+0000","last_became_active":"2023-10-08T05:55:24.922782+0000","last_became_peered":"2023-10-08T05:55:24.922782+0000","last_unstale":"2023-10-08T05:55:26.929213+0000","last_undegraded":"2023-10-08T05:55:26.929213+0000","last_fullsized":"2023-10-08T05:55:26.929213+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:58.749374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503803+0000","last_change":"2023-10-08T05:55:25.923884+0000","last_active":"2023-10-08T05:55:27.503803+0000","last_peered":"2023-10-08T05:55:27.503803+0000","last_clean":"2023-10-08T05:55:27.503803+0000","last_became_active":"2023-10-08T05:55:25.923617+0000","last_became_peered":"2023-10-08T05:55:25.923617+0000","last_unstale":"2023-10-08T05:55:27.503803+0000","last_undegraded":"2023-10-08T05:55:27.503803+0000","last_fullsized":"2023-10-08T05:55:27.503803+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:07.078151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503383+0000","last_change":"2023-10-08T05:55:24.928566+0000","last_active":"2023-10-08T05:55:27.503383+0000","last_peered":"2023-10-08T05:55:27.503383+0000","last_clean":"2023-10-08T05:55:27.503383+0000","last_became_active":"2023-10-08T05:55:24.928183+0000","last_became_peered":"2023-10-08T05:55:24.928183+0000","last_unstale":"2023-10-08T05:55:27.503383+0000","last_undegraded":"2023-10-08T05:55:27.503383+0000","last_fullsized":"2023-10-08T05:55:27.503383+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:58.364141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284416+0000","last_change":"2023-10-08T05:55:25.908644+0000","last_active":"2023-10-08T05:55:27.284416+0000","last_peered":"2023-10-08T05:55:27.284416+0000","last_clean":"2023-10-08T05:55:27.284416+0000","last_became_active":"2023-10-08T05:55:25.908346+0000","last_became_peered":"2023-10-08T05:55:25.908346+0000","last_unstale":"2023-10-08T05:55:27.284416+0000","last_undegraded":"2023-10-08T05:55:27.284416+0000","last_fullsized":"2023-10-08T05:55:27.284416+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:51:00.669382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502145+0000","last_change":"2023-10-08T05:55:24.924397+0000","last_active":"2023-10-08T05:55:27.502145+0000","last_peered":"2023-10-08T05:55:27.502145+0000","last_clean":"2023-10-08T05:55:27.502145+0000","last_became_active":"2023-10-08T05:55:24.923977+0000","last_became_peered":"2023-10-08T05:55:24.923977+0000","last_unstale":"2023-10-08T05:55:27.502145+0000","last_undegraded":"2023-10-08T05:55:27.502145+0000","last_fullsized":"2023-10-08T05:55:27.502145+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.138903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284377+0000","last_change":"2023-10-08T05:55:25.939606+0000","last_active":"2023-10-08T05:55:27.284377+0000","last_peered":"2023-10-08T05:55:27.284377+0000","last_clean":"2023-10-08T05:55:27.284377+0000","last_became_active":"2023-10-08T05:55:25.939194+0000","last_became_peered":"2023-10-08T05:55:25.939194+0000","last_unstale":"2023-10-08T05:55:27.284377+0000","last_undegraded":"2023-10-08T05:55:27.284377+0000","last_fullsized":"2023-10-08T05:55:27.284377+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:19.783496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501063+0000","last_change":"2023-10-08T05:55:24.920985+0000","last_active":"2023-10-08T05:55:27.501063+0000","last_peered":"2023-10-08T05:55:27.501063+0000","last_clean":"2023-10-08T05:55:27.501063+0000","last_became_active":"2023-10-08T05:55:24.920649+0000","last_became_peered":"2023-10-08T05:55:24.920649+0000","last_unstale":"2023-10-08T05:55:27.501063+0000","last_undegraded":"2023-10-08T05:55:27.501063+0000","last_fullsized":"2023-10-08T05:55:27.501063+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:58:04.772705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926149+0000","last_change":"2023-10-08T05:55:25.934860+0000","last_active":"2023-10-08T05:55:26.926149+0000","last_peered":"2023-10-08T05:55:26.926149+0000","last_clean":"2023-10-08T05:55:26.926149+0000","last_became_active":"2023-10-08T05:55:25.931500+0000","last_became_peered":"2023-10-08T05:55:25.931500+0000","last_unstale":"2023-10-08T05:55:26.926149+0000","last_undegraded":"2023-10-08T05:55:26.926149+0000","last_fullsized":"2023-10-08T05:55:26.926149+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:20.072607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502232+0000","last_change":"2023-10-08T05:55:24.919252+0000","last_active":"2023-10-08T05:55:27.502232+0000","last_peered":"2023-10-08T05:55:27.502232+0000","last_clean":"2023-10-08T05:55:27.502232+0000","last_became_active":"2023-10-08T05:55:24.918968+0000","last_became_peered":"2023-10-08T05:55:24.918968+0000","last_unstale":"2023-10-08T05:55:27.502232+0000","last_undegraded":"2023-10-08T05:55:27.502232+0000","last_fullsized":"2023-10-08T05:55:27.502232+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:08.548178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502015+0000","last_change":"2023-10-08T05:55:25.921321+0000","last_active":"2023-10-08T05:55:27.502015+0000","last_peered":"2023-10-08T05:55:27.502015+0000","last_clean":"2023-10-08T05:55:27.502015+0000","last_became_active":"2023-10-08T05:55:25.921043+0000","last_became_peered":"2023-10-08T05:55:25.921043+0000","last_unstale":"2023-10-08T05:55:27.502015+0000","last_undegraded":"2023-10-08T05:55:27.502015+0000","last_fullsized":"2023-10-08T05:55:27.502015+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:08.790630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501322+0000","last_change":"2023-10-08T05:55:24.913364+0000","last_active":"2023-10-08T05:55:27.501322+0000","last_peered":"2023-10-08T05:55:27.501322+0000","last_clean":"2023-10-08T05:55:27.501322+0000","last_became_active":"2023-10-08T05:55:24.912918+0000","last_became_peered":"2023-10-08T05:55:24.912918+0000","last_unstale":"2023-10-08T05:55:27.501322+0000","last_undegraded":"2023-10-08T05:55:27.501322+0000","last_fullsized":"2023-10-08T05:55:27.501322+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:44.915337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284071+0000","last_change":"2023-10-08T05:55:25.911692+0000","last_active":"2023-10-08T05:55:27.284071+0000","last_peered":"2023-10-08T05:55:27.284071+0000","last_clean":"2023-10-08T05:55:27.284071+0000","last_became_active":"2023-10-08T05:55:25.911404+0000","last_became_peered":"2023-10-08T05:55:25.911404+0000","last_unstale":"2023-10-08T05:55:27.284071+0000","last_undegraded":"2023-10-08T05:55:27.284071+0000","last_fullsized":"2023-10-08T05:55:27.284071+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:16.285768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.504049+0000","last_change":"2023-10-08T05:55:24.924205+0000","last_active":"2023-10-08T05:55:27.504049+0000","last_peered":"2023-10-08T05:55:27.504049+0000","last_clean":"2023-10-08T05:55:27.504049+0000","last_became_active":"2023-10-08T05:55:24.923923+0000","last_became_peered":"2023-10-08T05:55:24.923923+0000","last_unstale":"2023-10-08T05:55:27.504049+0000","last_undegraded":"2023-10-08T05:55:27.504049+0000","last_fullsized":"2023-10-08T05:55:27.504049+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:50.221424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929569+0000","last_change":"2023-10-08T05:55:25.924284+0000","last_active":"2023-10-08T05:55:26.929569+0000","last_peered":"2023-10-08T05:55:26.929569+0000","last_clean":"2023-10-08T05:55:26.929569+0000","last_became_active":"2023-10-08T05:55:25.923897+0000","last_became_peered":"2023-10-08T05:55:25.923897+0000","last_unstale":"2023-10-08T05:55:26.929569+0000","last_undegraded":"2023-10-08T05:55:26.929569+0000","last_fullsized":"2023-10-08T05:55:26.929569+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:08.882808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.284281+0000","last_change":"2023-10-08T05:55:24.919675+0000","last_active":"2023-10-08T05:55:27.284281+0000","last_peered":"2023-10-08T05:55:27.284281+0000","last_clean":"2023-10-08T05:55:27.284281+0000","last_became_active":"2023-10-08T05:55:24.919401+0000","last_became_peered":"2023-10-08T05:55:24.919401+0000","last_unstale":"2023-10-08T05:55:27.284281+0000","last_undegraded":"2023-10-08T05:55:27.284281+0000","last_fullsized":"2023-10-08T05:55:27.284281+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:43.054211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.926284+0000","last_change":"2023-10-08T05:55:25.930439+0000","last_active":"2023-10-08T05:55:26.926284+0000","last_peered":"2023-10-08T05:55:26.926284+0000","last_clean":"2023-10-08T05:55:26.926284+0000","last_became_active":"2023-10-08T05:55:25.930184+0000","last_became_peered":"2023-10-08T05:55:25.930184+0000","last_unstale":"2023-10-08T05:55:26.926284+0000","last_undegraded":"2023-10-08T05:55:26.926284+0000","last_fullsized":"2023-10-08T05:55:26.926284+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:51:00.669382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503850+0000","last_change":"2023-10-08T05:55:24.913568+0000","last_active":"2023-10-08T05:55:27.503850+0000","last_peered":"2023-10-08T05:55:27.503850+0000","last_clean":"2023-10-08T05:55:27.503850+0000","last_became_active":"2023-10-08T05:55:24.913279+0000","last_became_peered":"2023-10-08T05:55:24.913279+0000","last_unstale":"2023-10-08T05:55:27.503850+0000","last_undegraded":"2023-10-08T05:55:27.503850+0000","last_fullsized":"2023-10-08T05:55:27.503850+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:51:52.735465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503703+0000","last_change":"2023-10-08T05:55:25.931818+0000","last_active":"2023-10-08T05:55:27.503703+0000","last_peered":"2023-10-08T05:55:27.503703+0000","last_clean":"2023-10-08T05:55:27.503703+0000","last_became_active":"2023-10-08T05:55:25.925729+0000","last_became_peered":"2023-10-08T05:55:25.925729+0000","last_unstale":"2023-10-08T05:55:27.503703+0000","last_undegraded":"2023-10-08T05:55:27.503703+0000","last_fullsized":"2023-10-08T05:55:27.503703+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:46.526640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503815+0000","last_change":"2023-10-08T05:55:24.923066+0000","last_active":"2023-10-08T05:55:27.503815+0000","last_peered":"2023-10-08T05:55:27.503815+0000","last_clean":"2023-10-08T05:55:27.503815+0000","last_became_active":"2023-10-08T05:55:24.922810+0000","last_became_peered":"2023-10-08T05:55:24.922810+0000","last_unstale":"2023-10-08T05:55:27.503815+0000","last_undegraded":"2023-10-08T05:55:27.503815+0000","last_fullsized":"2023-10-08T05:55:27.503815+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:39.632316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502185+0000","last_change":"2023-10-08T05:55:25.926674+0000","last_active":"2023-10-08T05:55:27.502185+0000","last_peered":"2023-10-08T05:55:27.502185+0000","last_clean":"2023-10-08T05:55:27.502185+0000","last_became_active":"2023-10-08T05:55:25.926376+0000","last_became_peered":"2023-10-08T05:55:25.926376+0000","last_unstale":"2023-10-08T05:55:27.502185+0000","last_undegraded":"2023-10-08T05:55:27.502185+0000","last_fullsized":"2023-10-08T05:55:27.502185+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:09.764366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.930397+0000","last_change":"2023-10-08T05:55:24.920312+0000","last_active":"2023-10-08T05:55:26.930397+0000","last_peered":"2023-10-08T05:55:26.930397+0000","last_clean":"2023-10-08T05:55:26.930397+0000","last_became_active":"2023-10-08T05:55:24.920005+0000","last_became_peered":"2023-10-08T05:55:24.920005+0000","last_unstale":"2023-10-08T05:55:26.930397+0000","last_undegraded":"2023-10-08T05:55:26.930397+0000","last_fullsized":"2023-10-08T05:55:26.930397+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:04.374851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501919+0000","last_change":"2023-10-08T05:55:25.920684+0000","last_active":"2023-10-08T05:55:27.501919+0000","last_peered":"2023-10-08T05:55:27.501919+0000","last_clean":"2023-10-08T05:55:27.501919+0000","last_became_active":"2023-10-08T05:55:25.920401+0000","last_became_peered":"2023-10-08T05:55:25.920401+0000","last_unstale":"2023-10-08T05:55:27.501919+0000","last_undegraded":"2023-10-08T05:55:27.501919+0000","last_fullsized":"2023-10-08T05:55:27.501919+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:00.047081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502129+0000","last_change":"2023-10-08T05:55:24.926301+0000","last_active":"2023-10-08T05:55:27.502129+0000","last_peered":"2023-10-08T05:55:27.502129+0000","last_clean":"2023-10-08T05:55:27.502129+0000","last_became_active":"2023-10-08T05:55:24.926045+0000","last_became_peered":"2023-10-08T05:55:24.926045+0000","last_unstale":"2023-10-08T05:55:27.502129+0000","last_undegraded":"2023-10-08T05:55:27.502129+0000","last_fullsized":"2023-10-08T05:55:27.502129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:59.640943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:26.929598+0000","last_change":"2023-10-08T05:55:25.938224+0000","last_active":"2023-10-08T05:55:26.929598+0000","last_peered":"2023-10-08T05:55:26.929598+0000","last_clean":"2023-10-08T05:55:26.929598+0000","last_became_active":"2023-10-08T05:55:25.937958+0000","last_became_peered":"2023-10-08T05:55:25.937958+0000","last_unstale":"2023-10-08T05:55:26.929598+0000","last_undegraded":"2023-10-08T05:55:26.929598+0000","last_fullsized":"2023-10-08T05:55:26.929598+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:23.640733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502491+0000","last_change":"2023-10-08T05:55:24.915109+0000","last_active":"2023-10-08T05:55:27.502491+0000","last_peered":"2023-10-08T05:55:27.502491+0000","last_clean":"2023-10-08T05:55:27.502491+0000","last_became_active":"2023-10-08T05:55:24.914040+0000","last_became_peered":"2023-10-08T05:55:24.914040+0000","last_unstale":"2023-10-08T05:55:27.502491+0000","last_undegraded":"2023-10-08T05:55:27.502491+0000","last_fullsized":"2023-10-08T05:55:27.502491+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:24.459757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.503775+0000","last_change":"2023-10-08T05:55:25.935038+0000","last_active":"2023-10-08T05:55:27.503775+0000","last_peered":"2023-10-08T05:55:27.503775+0000","last_clean":"2023-10-08T05:55:27.503775+0000","last_became_active":"2023-10-08T05:55:25.934749+0000","last_became_peered":"2023-10-08T05:55:25.934749+0000","last_unstale":"2023-10-08T05:55:27.503775+0000","last_undegraded":"2023-10-08T05:55:27.503775+0000","last_fullsized":"2023-10-08T05:55:27.503775+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:56.567240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501933+0000","last_change":"2023-10-08T05:55:24.922088+0000","last_active":"2023-10-08T05:55:27.501933+0000","last_peered":"2023-10-08T05:55:27.501933+0000","last_clean":"2023-10-08T05:55:27.501933+0000","last_became_active":"2023-10-08T05:55:24.921811+0000","last_became_peered":"2023-10-08T05:55:24.921811+0000","last_unstale":"2023-10-08T05:55:27.501933+0000","last_undegraded":"2023-10-08T05:55:27.501933+0000","last_fullsized":"2023-10-08T05:55:27.501933+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:58:04.903345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.502273+0000","last_change":"2023-10-08T05:55:25.932100+0000","last_active":"2023-10-08T05:55:27.502273+0000","last_peered":"2023-10-08T05:55:27.502273+0000","last_clean":"2023-10-08T05:55:27.502273+0000","last_became_active":"2023-10-08T05:55:25.923270+0000","last_became_peered":"2023-10-08T05:55:25.923270+0000","last_unstale":"2023-10-08T05:55:27.502273+0000","last_undegraded":"2023-10-08T05:55:27.502273+0000","last_fullsized":"2023-10-08T05:55:27.502273+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:24.874552+0000","last_clean_scrub_stamp":"2023-10-08T05:55:24.874552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:23.640733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"133'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T05:55:27.501978+0000","last_change":"2023-10-08T05:55:24.914839+0000","last_active":"2023-10-08T05:55:27.501978+0000","last_peered":"2023-10-08T05:55:27.501978+0000","last_clean":"2023-10-08T05:55:27.501978+0000","last_became_active":"2023-10-08T05:55:24.914297+0000","last_became_peered":"2023-10-08T05:55:24.914297+0000","last_unstale":"2023-10-08T05:55:27.501978+0000","last_undegraded":"2023-10-08T05:55:27.501978+0000","last_fullsized":"2023-10-08T05:55:27.501978+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:55:23.859037+0000","last_clean_scrub_stamp":"2023-10-08T05:55:23.859037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:40:08.021395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"122'2663","reported_seq":3086,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-08T05:55:25.913206+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:55:25.913206+0000","last_peered":"2023-10-08T05:55:25.913206+0000","last_clean":"2023-10-08T05:55:25.913206+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:55:25.913206+0000","last_undegraded":"2023-10-08T05:55:25.913206+0000","last_fullsized":"2023-10-08T05:55:25.913206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2663,"log_dups_size":0,"ondisk_log_size":2663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":1940,"num_write":329,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":957,"num_read_kb":1940,"num_write":329,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2663,"ondisk_log_size":2663,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575079,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55600,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316240,"statfs":{"total":96636764160,"available":96579829760,"internally_reserved":0,"allocated":2260992,"data_stored":1434161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.125,"15min":1.2030000000000001},"min":{"1min":0.85299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.728,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.153,"15min":1.216},"min":{"1min":0.74399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.7030000000000001,"5min":1.7210000000000001,"15min":1.726},"last":1.534}]},{"osd":1,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.194,"15min":1.244},"min":{"1min":1.044,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8360000000000001},"last":1.093},{"interface":"front","average":{"1min":1.367,"5min":1.2629999999999999,"15min":1.276},"min":{"1min":0.753,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.9379999999999999},"last":1.762}]},{"osd":2,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.177,"15min":1.24},"min":{"1min":0.746,"5min":0.626,"15min":0.626},"max":{"1min":1.6160000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.419,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":1.052,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.2010000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2569999999999999,"15min":1.2849999999999999},"min":{"1min":0.93999999999999995,"5min":0.71499999999999997,"15min":0.69999999999999996},"max":{"1min":1.6060000000000001,"5min":1.8799999999999999,"15min":1.8799999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.3460000000000001,"15min":1.3660000000000001},"min":{"1min":1.1220000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.8660000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.3109999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3400000000000001,"15min":1.2889999999999999},"min":{"1min":0.91300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.718,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.661},{"interface":"front","average":{"1min":1.339,"5min":1.351,"15min":1.3280000000000001},"min":{"1min":0.81599999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.667,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.258}]},{"osd":5,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.421,"15min":1.4039999999999999},"min":{"1min":0.98099999999999998,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.9630000000000001,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.7989999999999999},{"interface":"front","average":{"1min":1.534,"5min":1.427,"15min":1.423},"min":{"1min":1.1120000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.9139999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.887}]},{"osd":6,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5880000000000001,"5min":1.5489999999999999,"15min":1.524},"min":{"1min":1.1779999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8160000000000001,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.8380000000000001},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.49,"15min":1.464},"min":{"1min":1.0800000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.7789999999999999,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.7170000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1667072,"data_stored":839185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.252,"15min":1.2809999999999999},"min":{"1min":0.625,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.6759999999999999,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.2829999999999999,"15min":1.3129999999999999},"min":{"1min":0.69799999999999995,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.73,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.121}]},{"osd":1,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.151,"15min":1.1699999999999999},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.4530000000000001,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.153,"15min":1.1899999999999999},"min":{"1min":0.67800000000000005,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.7030000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.2949999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.1359999999999999,"15min":1.1839999999999999},"min":{"1min":0.78800000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.498,"5min":1.774,"15min":1.796},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.218,"5min":1.212,"15min":1.2509999999999999},"min":{"1min":0.76500000000000001,"5min":0.76400000000000001,"15min":0.58699999999999997},"max":{"1min":1.5900000000000001,"5min":1.752,"15min":1.8520000000000001},"last":0.72899999999999998}]},{"osd":3,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3069999999999999,"15min":1.367},"min":{"1min":0.73299999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.7889999999999999,"5min":2.137,"15min":2.137},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.4039999999999999,"15min":1.446},"min":{"1min":0.89700000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.8380000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.5529999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.339,"15min":1.3129999999999999},"min":{"1min":0.72099999999999997,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.8560000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.4019999999999999,"15min":1.3839999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.64300000000000002},"max":{"1min":1.742,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.609}]},{"osd":5,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.325,"15min":1.3320000000000001},"min":{"1min":0.88600000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.647,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.454,"5min":1.5109999999999999,"15min":1.49},"min":{"1min":0.87,"5min":0.747,"15min":0.747},"max":{"1min":1.913,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.6419999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.542,"15min":1.5089999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.77200000000000002},"max":{"1min":1.7450000000000001,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.464},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5149999999999999,"15min":1.4810000000000001},"min":{"1min":0.85999999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.8700000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.242}]}]},{"osd":5,"up_from":13,"seq":55834575078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1658880,"data_stored":843633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.0940000000000001,"15min":1.1319999999999999},"min":{"1min":0.76900000000000002,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.7490000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.02},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.163,"15min":1.202},"min":{"1min":0.83499999999999996,"5min":0.47899999999999998,"15min":0.442},"max":{"1min":1.637,"5min":1.7509999999999999,"15min":1.758},"last":1.409}]},{"osd":1,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.127,"15min":1.194},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.323,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.25,"15min":1.2669999999999999},"min":{"1min":0.85099999999999998,"5min":0.81299999999999994,"15min":0.70199999999999996},"max":{"1min":1.639,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.3100000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.228,"15min":1.2330000000000001},"min":{"1min":0.91700000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6779999999999999,"5min":1.8,"15min":1.8},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2470000000000001,"15min":1.282},"min":{"1min":0.82399999999999995,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.486,"5min":3.282,"15min":3.282},"last":1.282}]},{"osd":3,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3140000000000001,"15min":1.3380000000000001},"min":{"1min":0.88200000000000001,"5min":0.88,"15min":0.63500000000000001},"max":{"1min":1.655,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.367,"15min":1.3899999999999999},"min":{"1min":0.84499999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.712,"5min":1.919,"15min":2.2280000000000002},"last":1.5069999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.361,"15min":1.3},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.7210000000000001,"5min":4.0179999999999998,"15min":4.0179999999999998},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.47,"15min":1.423},"min":{"1min":0.85799999999999998,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.736,"5min":3.456,"15min":3.456},"last":1.246}]},{"osd":6,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.446,"15min":1.4019999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.782,"5min":2.105,"15min":2.359},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.4970000000000001,"15min":1.466},"min":{"1min":0.623,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.909,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.212}]},{"osd":7,"last update":"Sun Oct 8 05:54:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.569,"15min":1.5389999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.9470000000000001,"5min":3.4079999999999999,"15min":3.4079999999999999},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.5529999999999999,"5min":1.5720000000000001,"15min":1.554},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.877,"5min":2.2589999999999999,"15min":2.4460000000000002},"last":1.3360000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1667072,"data_stored":843725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.1679999999999999,"15min":1.2529999999999999},"min":{"1min":0.78100000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.665,"5min":1.7609999999999999,"15min":1.8149999999999999},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.3120000000000001,"15min":1.365},"min":{"1min":0.86599999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5660000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.532}]},{"osd":1,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.286,"15min":1.28},"min":{"1min":0.76300000000000001,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.681,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.292},{"interface":"front","average":{"1min":1.304,"5min":1.2330000000000001,"15min":1.2210000000000001},"min":{"1min":0.98999999999999999,"5min":0.70299999999999996,"15min":0.60099999999999998},"max":{"1min":1.6839999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":1.2629999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2210000000000001,"15min":1.238},"min":{"1min":0.80900000000000005,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.659,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.198},{"interface":"front","average":{"1min":1.276,"5min":1.3360000000000001,"15min":1.345},"min":{"1min":0.96699999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.766,"5min":2.431,"15min":2.431},"last":0.95399999999999996}]},{"osd":3,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.28,"15min":1.2749999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5600000000000001,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.127},{"interface":"front","average":{"1min":1.413,"5min":1.3169999999999999,"15min":1.343},"min":{"1min":1.0169999999999999,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.764,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.5860000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.395,"15min":1.337},"min":{"1min":0.95299999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7629999999999999,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.423,"15min":1.365},"min":{"1min":0.96799999999999997,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.6830000000000001,"5min":5.4189999999999996,"15min":5.4189999999999996},"last":1.371}]},{"osd":6,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.448,"15min":1.361},"min":{"1min":0.91600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.966,"5min":4.9059999999999997,"15min":4.9059999999999997},"last":1.716},{"interface":"front","average":{"1min":1.494,"5min":1.579,"15min":1.5109999999999999},"min":{"1min":1.0109999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.829,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":1.8759999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.421,"5min":1.5600000000000001,"15min":1.468},"min":{"1min":1.1539999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.8160000000000001,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":1.5},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5389999999999999,"15min":1.5209999999999999},"min":{"1min":1.0329999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2000000000000002,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.8320000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575080,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36964,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334876,"statfs":{"total":96636764160,"available":96598913024,"internally_reserved":0,"allocated":1662976,"data_stored":844215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.88900000000000001,"15min":0.91800000000000004},"min":{"1min":0.64300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3049999999999999,"5min":1.355,"15min":1.72},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.0429999999999999,"15min":1.099},"min":{"1min":0.74299999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0760000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.89900000000000002,"15min":0.96099999999999997},"min":{"1min":0.60799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.302,"5min":1.605,"15min":1.605},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1279999999999999,"15min":1.202},"min":{"1min":0.67700000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.5569999999999999,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.2250000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":0.99199999999999999,"15min":1.0600000000000001},"min":{"1min":0.72699999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3720000000000001,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.083,"15min":1.1399999999999999},"min":{"1min":0.85099999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.599,"5min":1.6040000000000001,"15min":1.788},"last":1.026}]},{"osd":4,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.163,"15min":1.2070000000000001},"min":{"1min":0.97999999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.458,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.173},{"interface":"front","average":{"1min":1.331,"5min":1.3380000000000001,"15min":1.379},"min":{"1min":0.82699999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5549999999999999,"5min":2.589,"15min":2.589},"last":1.7350000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.284,"15min":1.3009999999999999},"min":{"1min":1.0449999999999999,"5min":0.73899999999999999,"15min":0.73699999999999999},"max":{"1min":1.605,"5min":1.883,"15min":1.883},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.3140000000000001,"15min":1.3500000000000001},"min":{"1min":1.1599999999999999,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.532,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.389}]},{"osd":6,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.323,"15min":1.3200000000000001},"min":{"1min":1.0409999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.841,"5min":1.841,"15min":1.841},"last":1.677},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.425,"15min":1.4419999999999999},"min":{"1min":1.0840000000000001,"5min":0.86799999999999999,"15min":0.86699999999999999},"max":{"1min":1.6859999999999999,"5min":2.016,"15min":2.0489999999999999},"last":1.585}]},{"osd":7,"last update":"Sun Oct 8 05:54:42 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4450000000000001,"15min":1.4870000000000001},"min":{"1min":1.25,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.661,"5min":2.0529999999999999,"15min":2.1480000000000001},"last":1.982},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.393},"min":{"1min":0.92700000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8799999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.5209999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575078,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59176,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312664,"statfs":{"total":96636764160,"available":96576167936,"internally_reserved":0,"allocated":1662976,"data_stored":843667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.78900000000000003,"15min":0.871},"min":{"1min":0.63400000000000001,"5min":0.255,"15min":0.255},"max":{"1min":1.5249999999999999,"5min":1.629,"15min":1.629},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.0720000000000001,"5min":0.95399999999999996,"15min":1.036},"min":{"1min":0.64400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5660000000000001,"5min":1.5880000000000001,"15min":1.9470000000000001},"last":1.145}]},{"osd":1,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.86299999999999999,"15min":0.94399999999999995},"min":{"1min":0.69999999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3560000000000001,"5min":1.7609999999999999,"15min":2.1070000000000002},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.093,"5min":0.97399999999999998,"15min":1.0860000000000001},"min":{"1min":0.79400000000000004,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6890000000000001,"5min":1.79,"15min":2.1469999999999998},"last":1.417}]},{"osd":3,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.0069999999999999,"15min":1.1200000000000001},"min":{"1min":0.72999999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.617,"5min":1.631,"15min":1.8120000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.147,"15min":1.216},"min":{"1min":0.89900000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.8859999999999999,"5min":7.5890000000000004,"15min":7.5890000000000004},"last":1.516}]},{"osd":4,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3109999999999999,"15min":1.3759999999999999},"min":{"1min":1.117,"5min":0.622,"15min":0.622},"max":{"1min":1.6679999999999999,"5min":4.593,"15min":4.593},"last":1.204},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.2569999999999999,"15min":1.282},"min":{"1min":0.92600000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.629,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.2929999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.298,"15min":1.3700000000000001},"min":{"1min":1.0760000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.702,"5min":6.7140000000000004,"15min":6.7140000000000004},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.3420000000000001,"15min":1.4390000000000001},"min":{"1min":1.121,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":2.4119999999999999},"last":1.6160000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.29,"15min":1.3440000000000001},"min":{"1min":1.0109999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.8799999999999999,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.323,"15min":1.4399999999999999},"min":{"1min":1.1319999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.8400000000000001,"5min":2.8839999999999999,"15min":2.8839999999999999},"last":1.0589999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:54:45 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.3759999999999999,"15min":1.4279999999999999},"min":{"1min":1.1160000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.73,"5min":6.609,"15min":6.609},"last":1.375},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.3580000000000001,"15min":1.4850000000000001},"min":{"1min":1.1259999999999999,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.9370000000000001,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.3440000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36960,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334880,"statfs":{"total":96636764160,"available":96598917120,"internally_reserved":0,"allocated":1650688,"data_stored":837003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":0.88700000000000001,"15min":0.96699999999999997},"min":{"1min":0.66100000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.583,"5min":1.583,"15min":1.796},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.98599999999999999,"15min":1.1180000000000001},"min":{"1min":0.51600000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.4119999999999999,"5min":1.4510000000000001,"15min":1.8089999999999999},"last":1.089}]},{"osd":2,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.871,"15min":0.95599999999999996},"min":{"1min":0.498,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.363,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.163,"5min":0.99299999999999999,"15min":1.085},"min":{"1min":0.73799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.474,"5min":1.5860000000000001,"15min":1.8049999999999999},"last":1.5349999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.0580000000000001,"15min":1.171},"min":{"1min":0.66500000000000004,"5min":0.498,"15min":0.498},"max":{"1min":1.637,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0169999999999999,"15min":1.0680000000000001},"min":{"1min":0.81899999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.6120000000000001,"5min":1.7669999999999999,"15min":1.788},"last":0.97399999999999998}]},{"osd":4,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.1799999999999999,"15min":1.2909999999999999},"min":{"1min":0.90700000000000003,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.5780000000000001,"5min":1.7529999999999999,"15min":1.966},"last":1.49},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.1850000000000001,"15min":1.222},"min":{"1min":0.90600000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.671,"5min":1.913,"15min":1.913},"last":1.1339999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.2569999999999999,"15min":1.3089999999999999},"min":{"1min":1.0509999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6799999999999999,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.393},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.375,"15min":1.45},"min":{"1min":1.2130000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":1.641}]},{"osd":6,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.343,"15min":1.4159999999999999},"min":{"1min":1.0409999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9970000000000001},"last":1.339},{"interface":"front","average":{"1min":1.389,"5min":1.3620000000000001,"15min":1.4179999999999999},"min":{"1min":1.0009999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.796,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.298}]},{"osd":7,"last update":"Sun Oct 8 05:54:37 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.3520000000000001,"15min":1.454},"min":{"1min":1.0860000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.377,"15min":1.444},"min":{"1min":1.077,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.306},"last":1.6000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575079,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33384,"kb_used_data":2204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338456,"statfs":{"total":96636764160,"available":96602578944,"internally_reserved":0,"allocated":2256896,"data_stored":1429039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85999999999999999,"15min":0.91100000000000003},"min":{"1min":0.65200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.3380000000000001,"5min":1.77,"15min":1.77},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.94999999999999996,"15min":1.0369999999999999},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.3109999999999999,"5min":1.6719999999999999,"15min":1.675},"last":0.82899999999999996}]},{"osd":2,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1,"15min":1.0840000000000001},"min":{"1min":0.44600000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.5649999999999999,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.876,"15min":0.98699999999999999},"min":{"1min":0.55100000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.4770000000000001,"5min":1.722,"15min":1.78},"last":0.874}]},{"osd":3,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.0940000000000001,"15min":1.1919999999999999},"min":{"1min":0.49299999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.448,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.093},"min":{"1min":0.56399999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4239999999999999,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.0249999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1779999999999999,"15min":1.262},"min":{"1min":0.71399999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.571,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.381},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2689999999999999,"15min":1.3440000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.669,"5min":1.8939999999999999,"15min":2.2200000000000002},"last":1.4279999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2549999999999999,"15min":1.3300000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.599,"5min":1.9410000000000001,"15min":2.0489999999999999},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.2969999999999999,"15min":1.3720000000000001},"min":{"1min":0.84699999999999998,"5min":0.63,"15min":0.63},"max":{"1min":1.677,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.512}]},{"osd":6,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3400000000000001,"15min":1.4119999999999999},"min":{"1min":0.68700000000000006,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.742,"5min":2.032,"15min":2.1909999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3160000000000001,"15min":1.393},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6870000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.4650000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:54:43 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.359,"15min":1.4319999999999999},"min":{"1min":0.755,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7709999999999999,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.782},{"interface":"front","average":{"1min":1.411,"5min":1.395,"15min":1.4750000000000001},"min":{"1min":0.73699999999999999,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.8340000000000001,"5min":2.0569999999999999,"15min":2.2759999999999998},"last":1.7649999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:55:30.871 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T05:55:30.871 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T05:55:31.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.137+0000 7fe6c5c10640 1 Processor -- start 2023-10-08T05:55:31.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.138+0000 7fe6c5c10640 1 -- start start 2023-10-08T05:55:31.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.139+0000 7fe6c5c10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe6c00cb500 0x7fe6c00cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.139+0000 7fe6c5c10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6c0070690 0x7fe6c0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.139+0000 7fe6c5c10640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c00c8c60 con 0x7fe6c006cfa0 2023-10-08T05:55:31.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.139+0000 7fe6c5c10640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c00cbe20 con 0x7fe6c00cb500 2023-10-08T05:55:31.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.139+0000 7fe6c5c10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00b1ac0 con 0x7fe6c0070690 2023-10-08T05:55:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.140+0000 7fe6beffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe6c006cfa0 0x7fe6c015b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54810/0 (socket says 172.21.15.70:54810) 2023-10-08T05:55:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.140+0000 7fe6beffd640 1 -- 172.21.15.70:0/3743214841 learned_addr learned my addr 172.21.15.70:0/3743214841 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.140+0000 7fe6bf7fe640 1 --2- 172.21.15.70:0/3743214841 >> v2:172.21.15.181:3300/0 conn(0x7fe6c00cb500 0x7fe6c00cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.140+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354224630 0 0) 0x7fe6c00c8c60 con 0x7fe6c006cfa0 2023-10-08T05:55:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.140+0000 7fe6bffff640 1 --2- 172.21.15.70:0/3743214841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6c0070690 0x7fe6c0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.140+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a8003520 con 0x7fe6c006cfa0 2023-10-08T05:55:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2558974044 0 0) 0x7fe6a8003520 con 0x7fe6c006cfa0 2023-10-08T05:55:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6c0070690 msgr2=0x7fe6c0070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:31.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 --2- 172.21.15.70:0/3743214841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6c0070690 0x7fe6c0070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 >> v2:172.21.15.181:3300/0 conn(0x7fe6c00cb500 msgr2=0x7fe6c00cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 --2- 172.21.15.70:0/3743214841 >> v2:172.21.15.181:3300/0 conn(0x7fe6c00cb500 0x7fe6c00cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c00b1da0 con 0x7fe6c006cfa0 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe6b4001670 con 0x7fe6c006cfa0 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 >> v1:172.21.15.70:6789/0 conn(0x7fe6c006cfa0 legacy=0x7fe6c015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 --2- 172.21.15.70:0/3743214841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a8003970 0x7fe6a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 --2- 172.21.15.70:0/3743214841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a80047c0 0x7fe6a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 --2- 172.21.15.70:0/3743214841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6c0070690 0x7fe6a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00cbe20 con 0x7fe6a80047c0 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00b1ac0 con 0x7fe6a8003970 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a800bab0 con 0x7fe6c0070690 2023-10-08T05:55:31.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6bf7fe640 1 --2- 172.21.15.70:0/3743214841 >> v2:172.21.15.181:3300/0 conn(0x7fe6c00cb500 0x7fe6c00cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:55:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.141+0000 7fe6beffd640 1 --2- 172.21.15.70:0/3743214841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a80047c0 0x7fe6a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.142+0000 7fe6beffd640 1 -- 172.21.15.70:0/3743214841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6c0070690 msgr2=0x7fe6a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.142+0000 7fe6beffd640 1 --2- 172.21.15.70:0/3743214841 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6c0070690 0x7fe6a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.142+0000 7fe6beffd640 1 -- 172.21.15.70:0/3743214841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a8003970 msgr2=0x7fe6a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.142+0000 7fe6beffd640 1 --2- 172.21.15.70:0/3743214841 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6a8003970 0x7fe6a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.142+0000 7fe6beffd640 1 -- 172.21.15.70:0/3743214841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a8003520 con 0x7fe6a80047c0 2023-10-08T05:55:31.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.143+0000 7fe6beffd640 1 --2- 172.21.15.70:0/3743214841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a80047c0 0x7fe6a8004c40 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe6c006e5a0 tx=0x7fe6a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1592c15e555f9899 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:31.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.143+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4005ad0 con 0x7fe6a80047c0 2023-10-08T05:55:31.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.143+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b4005c30 con 0x7fe6a80047c0 2023-10-08T05:55:31.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.143+0000 7fe6be7fc640 1 -- 172.21.15.70:0/3743214841 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b402c7a0 con 0x7fe6a80047c0 2023-10-08T05:55:31.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.143+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3743214841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a80047c0 msgr2=0x7fe6a8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.143+0000 7fe6c5c10640 1 --2- 172.21.15.70:0/3743214841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a80047c0 0x7fe6a8004c40 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe6c006e5a0 tx=0x7fe6a80057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.143+0000 7fe6bffff640 1 -- 172.21.15.70:0/3743214841 reap_dead start 2023-10-08T05:55:31.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.143+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3743214841 shutdown_connections 2023-10-08T05:55:31.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.143+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3743214841 >> 172.21.15.70:0/3743214841 conn(0x7fe6c006a140 msgr2=0x7fe6c006a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:31.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.143+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3743214841 shutdown_connections 2023-10-08T05:55:31.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.144+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3743214841 wait complete. 2023-10-08T05:55:31.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.145+0000 7fe6c5c10640 1 Processor -- start 2023-10-08T05:55:31.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.145+0000 7fe6c5c10640 1 -- start start 2023-10-08T05:55:31.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.146+0000 7fe6c5c10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a80047c0 0x7fe6c01f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.146+0000 7fe6c5c10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c01f6620 0x7fe6c00adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.146+0000 7fe6c5c10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01f1c90 0x7fe6c00cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.146+0000 7fe6c5c10640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00ae510 con 0x7fe6c01f6620 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.146+0000 7fe6c5c10640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c01f2570 con 0x7fe6c01f1c90 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.146+0000 7fe6c5c10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c01f2850 con 0x7fe6a80047c0 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.147+0000 7fe6beffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c01f6620 0x7fe6c00adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.147+0000 7fe6beffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c01f6620 0x7fe6c00adce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54922/0 (socket says 172.21.15.70:54922) 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.147+0000 7fe6beffd640 1 -- 172.21.15.70:0/3532514481 learned_addr learned my addr 172.21.15.70:0/3532514481 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.147+0000 7fe6beffd640 1 -- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a80047c0 msgr2=0x7fe6c01f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.147+0000 7fe6beffd640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a80047c0 0x7fe6c01f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.147+0000 7fe6beffd640 1 -- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01f1c90 msgr2=0x7fe6c00cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.147+0000 7fe6beffd640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01f1c90 0x7fe6c00cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.147+0000 7fe6beffd640 1 -- 172.21.15.70:0/3532514481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c01f2b30 con 0x7fe6c01f6620 2023-10-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.147+0000 7fe6beffd640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c01f6620 0x7fe6c00adce0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fe6b402c340 tx=0x7fe6b4001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.148+0000 7fe6a7fff640 1 -- 172.21.15.70:0/3532514481 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b402cbb0 con 0x7fe6c01f6620 2023-10-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.148+0000 7fe6a7fff640 1 -- 172.21.15.70:0/3532514481 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b402cd10 con 0x7fe6c01f6620 2023-10-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.148+0000 7fe6a7fff640 1 -- 172.21.15.70:0/3532514481 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40072b0 con 0x7fe6c01f6620 2023-10-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.148+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3532514481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c00ae220 con 0x7fe6c01f6620 2023-10-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.148+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3532514481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c01f2110 con 0x7fe6c01f6620 2023-10-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.149+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3532514481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c006d990 con 0x7fe6c01f6620 2023-10-08T05:55:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.150+0000 7fe6a7fff640 1 -- 172.21.15.70:0/3532514481 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe6b4032070 con 0x7fe6c01f6620 2023-10-08T05:55:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.151+0000 7fe6a7fff640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe6ac06afc0 0x7fe6ac06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.151+0000 7fe6a7fff640 1 -- 172.21.15.70:0/3532514481 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe6b403c040 con 0x7fe6c01f6620 2023-10-08T05:55:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.151+0000 7fe6bf7fe640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe6ac06afc0 0x7fe6ac06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.152+0000 7fe6bf7fe640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe6ac06afc0 0x7fe6ac06d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009f00 tx=0x7fe6b00094e0 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:31.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.155+0000 7fe6a7fff640 1 -- 172.21.15.70:0/3532514481 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6b4075820 con 0x7fe6c01f6620 2023-10-08T05:55:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.413+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3532514481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe6c01f2cc0 con 0x7fe6c01f6620 2023-10-08T05:55:31.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.417+0000 7fe6a7fff640 1 -- 172.21.15.70:0/3532514481 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe6b40450b0 con 0x7fe6c01f6620 2023-10-08T05:55:31.418 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T05:55:31.418 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T05:55:31.418 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T05:55:31.418 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T05:55:31.418 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T05:55:31.418 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T05:55:31.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe6ac06afc0 msgr2=0x7fe6ac06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe6ac06afc0 0x7fe6ac06d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009f00 tx=0x7fe6b00094e0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3532514481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c01f6620 msgr2=0x7fe6c00adce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c01f6620 0x7fe6c00adce0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fe6b402c340 tx=0x7fe6b4001930 comp rx=0 tx=0).stop 2023-10-08T05:55:31.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3532514481 shutdown_connections 2023-10-08T05:55:31.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6c01f1c90 0x7fe6c00cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fe6ac06afc0 0x7fe6ac06d480 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6c01f6620 0x7fe6c00adce0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 --2- 172.21.15.70:0/3532514481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe6a80047c0 0x7fe6c01f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3532514481 >> 172.21.15.70:0/3532514481 conn(0x7fe6c006a140 msgr2=0x7fe6c00ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:31.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3532514481 shutdown_connections 2023-10-08T05:55:31.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.420+0000 7fe6c5c10640 1 -- 172.21.15.70:0/3532514481 wait complete. 2023-10-08T05:55:31.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:55:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.649+0000 7f91a04f2640 1 Processor -- start 2023-10-08T05:55:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.649+0000 7f91a04f2640 1 -- start start 2023-10-08T05:55:31.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.651+0000 7f91a04f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f919815ab40 0x7f91980ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.651+0000 7f91a04f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91980c90b0 0x7f91980adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.651+0000 7f91a04f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f91980cae70 0x7f91980cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.651+0000 7f91a04f2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919815bf30 con 0x7f91980c90b0 2023-10-08T05:55:31.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.651+0000 7f91a04f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91980cb840 con 0x7f91980cae70 2023-10-08T05:55:31.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.651+0000 7f91a04f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919815b2b0 con 0x7f919815ab40 2023-10-08T05:55:31.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.651+0000 7f919da8d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91980c90b0 0x7f91980adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:31.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.652+0000 7f919da8d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91980c90b0 0x7f91980adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54930/0 (socket says 172.21.15.70:54930) 2023-10-08T05:55:31.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.652+0000 7f919da8d640 1 -- 172.21.15.70:0/3848184571 learned_addr learned my addr 172.21.15.70:0/3848184571 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:31.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.652+0000 7f919ea8f640 1 --2- 172.21.15.70:0/3848184571 >> v2:172.21.15.181:3300/0 conn(0x7f91980cae70 0x7f91980cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:31.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.652+0000 7f919e28e640 1 --2- 172.21.15.70:0/3848184571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f919815ab40 0x7f91980ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:31.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.652+0000 7f919da8d640 1 -- 172.21.15.70:0/3848184571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f919815ab40 msgr2=0x7f91980ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:31.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.652+0000 7f919da8d640 1 --2- 172.21.15.70:0/3848184571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f919815ab40 0x7f91980ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.652+0000 7f919da8d640 1 -- 172.21.15.70:0/3848184571 >> v2:172.21.15.181:3300/0 conn(0x7f91980cae70 msgr2=0x7f91980cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:31.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.652+0000 7f919da8d640 1 --2- 172.21.15.70:0/3848184571 >> v2:172.21.15.181:3300/0 conn(0x7f91980cae70 0x7f91980cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.652+0000 7f919da8d640 1 -- 172.21.15.70:0/3848184571 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919815b590 con 0x7f91980c90b0 2023-10-08T05:55:31.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.653+0000 7f919da8d640 1 --2- 172.21.15.70:0/3848184571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 0x7f91980adc60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f91980c8d50 tx=0x7f918802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b7dae7902c76a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:31.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.653+0000 7f919d28c640 1 -- 172.21.15.70:0/3848184571 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188031860 con 0x7f91980c90b0 2023-10-08T05:55:31.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.653+0000 7f919d28c640 1 -- 172.21.15.70:0/3848184571 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91880319c0 con 0x7f91980c90b0 2023-10-08T05:55:31.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.653+0000 7f919d28c640 1 -- 172.21.15.70:0/3848184571 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9188031d50 con 0x7f91980c90b0 2023-10-08T05:55:31.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.654+0000 7f91a04f2640 1 -- 172.21.15.70:0/3848184571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 msgr2=0x7f91980adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.654+0000 7f91a04f2640 1 --2- 172.21.15.70:0/3848184571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 0x7f91980adc60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f91980c8d50 tx=0x7f918802b900 comp rx=0 tx=0).stop 2023-10-08T05:55:31.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.654+0000 7f91a04f2640 1 -- 172.21.15.70:0/3848184571 shutdown_connections 2023-10-08T05:55:31.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.654+0000 7f91a04f2640 1 --2- 172.21.15.70:0/3848184571 >> v2:172.21.15.181:3300/0 conn(0x7f91980cae70 0x7f91980cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.654+0000 7f91a04f2640 1 --2- 172.21.15.70:0/3848184571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 0x7f91980adc60 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.654+0000 7f91a04f2640 1 --2- 172.21.15.70:0/3848184571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f919815ab40 0x7f91980ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.654+0000 7f91a04f2640 1 -- 172.21.15.70:0/3848184571 >> 172.21.15.70:0/3848184571 conn(0x7f919806a380 msgr2=0x7f91980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:31.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.654+0000 7f91a04f2640 1 -- 172.21.15.70:0/3848184571 shutdown_connections 2023-10-08T05:55:31.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.654+0000 7f91a04f2640 1 -- 172.21.15.70:0/3848184571 wait complete. 2023-10-08T05:55:31.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.656+0000 7f91a04f2640 1 Processor -- start 2023-10-08T05:55:31.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.656+0000 7f91a04f2640 1 -- start start 2023-10-08T05:55:31.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.657+0000 7f91a04f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 0x7f91981470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.657+0000 7f91a04f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91980cae70 0x7f91980c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.657+0000 7f91a04f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f919815ab40 0x7f91980c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.657+0000 7f91a04f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9198158d70 con 0x7f91980c90b0 2023-10-08T05:55:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.657+0000 7f91a04f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91981581c0 con 0x7f919815ab40 2023-10-08T05:55:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.657+0000 7f91a04f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91980b2d40 con 0x7f91980cae70 2023-10-08T05:55:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.657+0000 7f919e28e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 0x7f91981470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.657+0000 7f919e28e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 0x7f91981470e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54940/0 (socket says 172.21.15.70:54940) 2023-10-08T05:55:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.657+0000 7f919e28e640 1 -- 172.21.15.70:0/2941182342 learned_addr learned my addr 172.21.15.70:0/2941182342 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f919e28e640 1 -- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91980cae70 msgr2=0x7f91980c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f919e28e640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91980cae70 0x7f91980c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f919e28e640 1 -- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f919815ab40 msgr2=0x7f91980c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f919e28e640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f919815ab40 0x7f91980c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f919e28e640 1 -- 172.21.15.70:0/2941182342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880094d0 con 0x7f91980c90b0 2023-10-08T05:55:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f919e28e640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 0x7f91981470e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f919400d830 tx=0x7f919400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f918effd640 1 -- 172.21.15.70:0/2941182342 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919400b6c0 con 0x7f91980c90b0 2023-10-08T05:55:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f918effd640 1 -- 172.21.15.70:0/2941182342 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919400b820 con 0x7f91980c90b0 2023-10-08T05:55:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f918effd640 1 -- 172.21.15.70:0/2941182342 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919400bbb0 con 0x7f91980c90b0 2023-10-08T05:55:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f91a04f2640 1 -- 172.21.15.70:0/2941182342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91981583d0 con 0x7f91980c90b0 2023-10-08T05:55:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.658+0000 7f91a04f2640 1 -- 172.21.15.70:0/2941182342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9198158990 con 0x7f91980c90b0 2023-10-08T05:55:31.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.659+0000 7f91a04f2640 1 -- 172.21.15.70:0/2941182342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9198067490 con 0x7f91980c90b0 2023-10-08T05:55:31.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.660+0000 7f918effd640 1 -- 172.21.15.70:0/2941182342 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f919401e460 con 0x7f91980c90b0 2023-10-08T05:55:31.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.661+0000 7f918effd640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f917c06af90 0x7f917c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:31.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.661+0000 7f918effd640 1 -- 172.21.15.70:0/2941182342 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f919408a080 con 0x7f91980c90b0 2023-10-08T05:55:31.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.662+0000 7f919da8d640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f917c06af90 0x7f917c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:31.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.663+0000 7f919da8d640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f917c06af90 0x7f917c06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9188036000 tx=0x7f9188030000 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:31.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.666+0000 7f918effd640 1 -- 172.21.15.70:0/2941182342 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9194054eb0 con 0x7f91980c90b0 2023-10-08T05:55:31.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.887+0000 7f91a04f2640 1 -- 172.21.15.70:0/2941182342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91980b0ef0 con 0x7f91980c90b0 2023-10-08T05:55:31.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.888+0000 7f918effd640 1 -- 172.21.15.70:0/2941182342 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f91940584d0 con 0x7f91980c90b0 2023-10-08T05:55:31.889 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:31.889 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3983030529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3983030529},{"type":"v1","addr":"172.21.15.181:6836","nonce":3983030529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5873,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1573936162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1573936162},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573936162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1979922916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1979922916},{"type":"v1","addr":"172.21.15.70:6835","nonce":1979922916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:55:25.363123+0000","modified":"2023-10-08T05:55:28.414414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3020884355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3020884355},{"type":"v1","addr":"172.21.15.70:6833","nonce":3020884355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-08T05:55:31.889 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 58 2023-10-08T05:55:31.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 -- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f917c06af90 msgr2=0x7f917c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f917c06af90 0x7f917c06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9188036000 tx=0x7f9188030000 comp rx=0 tx=0).stop 2023-10-08T05:55:31.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 -- 172.21.15.70:0/2941182342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 msgr2=0x7f91981470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:31.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 0x7f91981470e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f919400d830 tx=0x7f919400dcf0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 -- 172.21.15.70:0/2941182342 shutdown_connections 2023-10-08T05:55:31.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f919815ab40 0x7f91980c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f91980cae70 0x7f91980c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f917c06af90 0x7f917c06d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 --2- 172.21.15.70:0/2941182342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91980c90b0 0x7f91981470e0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:31.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 -- 172.21.15.70:0/2941182342 >> 172.21.15.70:0/2941182342 conn(0x7f919806a380 msgr2=0x7f91980ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:31.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 -- 172.21.15.70:0/2941182342 shutdown_connections 2023-10-08T05:55:31.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:31.891+0000 7f91a04f2640 1 -- 172.21.15.70:0/2941182342 wait complete. 2023-10-08T05:55:32.312 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T05:55:25.363123+0000', 'modified': '2023-10-08T05:55:28.414414+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8052}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8052': {'gid': 8052, 'name': 'c', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/3020884355', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 3020884355}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 3020884355}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T05:55:32.312 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T05:55:32.312 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T05:55:32.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.540+0000 7f6550083640 1 Processor -- start 2023-10-08T05:55:32.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.541+0000 7f6550083640 1 -- start start 2023-10-08T05:55:32.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.542+0000 7f6550083640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480c9cf0 0x7f654806db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.542+0000 7f6550083640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f654815aa00 0x7f65480b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.542+0000 7f6550083640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654815b310 con 0x7f654806c3a0 2023-10-08T05:55:32.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.542+0000 7f6550083640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654806e0a0 con 0x7f654815aa00 2023-10-08T05:55:32.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.542+0000 7f6550083640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654806d290 con 0x7f65480c9cf0 2023-10-08T05:55:32.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.542+0000 7f654e620640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f654815aa00 0x7f65480b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.542+0000 7f654e620640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f654815aa00 0x7f65480b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54952/0 (socket says 172.21.15.70:54952) 2023-10-08T05:55:32.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.542+0000 7f654e620640 1 -- 172.21.15.70:0/1596055744 learned_addr learned my addr 172.21.15.70:0/1596055744 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:32.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.543+0000 7f654d61e640 1 --2- 172.21.15.70:0/1596055744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480c9cf0 0x7f654806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.543+0000 7f654e620640 1 -- 172.21.15.70:0/1596055744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480c9cf0 msgr2=0x7f654806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.543+0000 7f654e620640 1 --2- 172.21.15.70:0/1596055744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480c9cf0 0x7f654806db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.543+0000 7f654e620640 1 -- 172.21.15.70:0/1596055744 >> v1:172.21.15.181:6789/0 conn(0x7f654806c3a0 legacy=0x7f654806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.543+0000 7f654e620640 1 -- 172.21.15.70:0/1596055744 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480c9860 con 0x7f654815aa00 2023-10-08T05:55:32.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.544+0000 7f654e620640 1 --2- 172.21.15.70:0/1596055744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 0x7f65480b2a90 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6540009e50 tx=0x7f654002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929a6336e16834d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.544+0000 7f654ce1d640 1 -- 172.21.15.70:0/1596055744 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654002caf0 con 0x7f654815aa00 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.544+0000 7f654ce1d640 1 -- 172.21.15.70:0/1596055744 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654002cc50 con 0x7f654815aa00 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.544+0000 7f654ce1d640 1 -- 172.21.15.70:0/1596055744 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65400343f0 con 0x7f654815aa00 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.545+0000 7f6550083640 1 -- 172.21.15.70:0/1596055744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 msgr2=0x7f65480b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.545+0000 7f6550083640 1 --2- 172.21.15.70:0/1596055744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 0x7f65480b2a90 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6540009e50 tx=0x7f654002bf50 comp rx=0 tx=0).stop 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.545+0000 7f6550083640 1 -- 172.21.15.70:0/1596055744 shutdown_connections 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.545+0000 7f6550083640 1 --2- 172.21.15.70:0/1596055744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 0x7f65480b2a90 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.545+0000 7f6550083640 1 --2- 172.21.15.70:0/1596055744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480c9cf0 0x7f654806db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.545+0000 7f6550083640 1 -- 172.21.15.70:0/1596055744 >> 172.21.15.70:0/1596055744 conn(0x7f654806a5d0 msgr2=0x7f65480b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.545+0000 7f6550083640 1 -- 172.21.15.70:0/1596055744 shutdown_connections 2023-10-08T05:55:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.545+0000 7f6550083640 1 -- 172.21.15.70:0/1596055744 wait complete. 2023-10-08T05:55:32.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.547+0000 7f6550083640 1 Processor -- start 2023-10-08T05:55:32.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.547+0000 7f6550083640 1 -- start start 2023-10-08T05:55:32.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.548+0000 7f6550083640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654806c3a0 0x7f6548152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.548+0000 7f6550083640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480c9cf0 0x7f65480c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.548+0000 7f6550083640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 0x7f65480c2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.548+0000 7f6550083640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65480cb160 con 0x7f654815aa00 2023-10-08T05:55:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.548+0000 7f6550083640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6548158d50 con 0x7f654806c3a0 2023-10-08T05:55:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.548+0000 7f6550083640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6548159030 con 0x7f65480c9cf0 2023-10-08T05:55:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.548+0000 7f654e620640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 0x7f65480c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.548+0000 7f654e620640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 0x7f65480c2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54966/0 (socket says 172.21.15.70:54966) 2023-10-08T05:55:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.548+0000 7f654e620640 1 -- 172.21.15.70:0/231668307 learned_addr learned my addr 172.21.15.70:0/231668307 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.549+0000 7f654d61e640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480c9cf0 0x7f65480c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.549+0000 7f654de1f640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654806c3a0 0x7f6548152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.549+0000 7f654e620640 1 -- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480c9cf0 msgr2=0x7f65480c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.549+0000 7f654e620640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480c9cf0 0x7f65480c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.549+0000 7f654e620640 1 -- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654806c3a0 msgr2=0x7f6548152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.549+0000 7f654e620640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654806c3a0 0x7f6548152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.549+0000 7f654e620640 1 -- 172.21.15.70:0/231668307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6540009b30 con 0x7f654815aa00 2023-10-08T05:55:32.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.549+0000 7f654e620640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 0x7f65480c2350 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6540036040 tx=0x7f6540004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:32.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.550+0000 7f653e7fc640 1 -- 172.21.15.70:0/231668307 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540040670 con 0x7f654815aa00 2023-10-08T05:55:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.550+0000 7f653e7fc640 1 -- 172.21.15.70:0/231668307 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6540040810 con 0x7f654815aa00 2023-10-08T05:55:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.550+0000 7f653e7fc640 1 -- 172.21.15.70:0/231668307 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540040bc0 con 0x7f654815aa00 2023-10-08T05:55:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.550+0000 7f6550083640 1 -- 172.21.15.70:0/231668307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6548159310 con 0x7f654815aa00 2023-10-08T05:55:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.550+0000 7f6550083640 1 -- 172.21.15.70:0/231668307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654815e690 con 0x7f654815aa00 2023-10-08T05:55:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.552+0000 7f653e7fc640 1 -- 172.21.15.70:0/231668307 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6540040d20 con 0x7f654815aa00 2023-10-08T05:55:32.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.553+0000 7f653e7fc640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f652c06af90 0x7f652c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.553+0000 7f653e7fc640 1 -- 172.21.15.70:0/231668307 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f65400ac990 con 0x7f654815aa00 2023-10-08T05:55:32.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.554+0000 7f654de1f640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f652c06af90 0x7f652c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.554+0000 7f6552c17740 1 Processor -- start 2023-10-08T05:55:32.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.554+0000 7f6552c17740 1 -- start start 2023-10-08T05:55:32.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.555+0000 7f654de1f640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f652c06af90 0x7f652c06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6544000f60 tx=0x7f6544009040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:32.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.555+0000 7f6552c17740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.555+0000 7f6552c17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c5198f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.555+0000 7f6552c17740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c4e9fa0 0x55aa2c519e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.555+0000 7f6552c17740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c45d700 con 0x55aa2c516b60 2023-10-08T05:55:32.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.555+0000 7f6552c17740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c45adf0 con 0x55aa2c516f40 2023-10-08T05:55:32.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.555+0000 7f6552c17740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c4566e0 con 0x55aa2c4e9fa0 2023-10-08T05:55:32.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.555+0000 7f654de1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.555+0000 7f654de1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4e9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54978/0 (socket says 172.21.15.70:54978) 2023-10-08T05:55:32.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.555+0000 7f654de1f640 1 -- 172.21.15.70:0/2363090720 learned_addr learned my addr 172.21.15.70:0/2363090720 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:32.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.556+0000 7f654d61e640 1 --2- 172.21.15.70:0/2363090720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c5198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.556+0000 7f654de1f640 1 -- 172.21.15.70:0/2363090720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c4e9fa0 msgr2=0x55aa2c519e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.556+0000 7f654de1f640 1 --2- 172.21.15.70:0/2363090720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c4e9fa0 0x55aa2c519e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.556+0000 7f654de1f640 1 -- 172.21.15.70:0/2363090720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 msgr2=0x55aa2c5198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.556+0000 7f654de1f640 1 --2- 172.21.15.70:0/2363090720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c5198f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.556+0000 7f654de1f640 1 -- 172.21.15.70:0/2363090720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa2c41a910 con 0x55aa2c516b60 2023-10-08T05:55:32.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.556+0000 7f654de1f640 1 --2- 172.21.15.70:0/2363090720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4e9a60 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f65440014c0 tx=0x7f6544011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6890ba564d1d88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:32.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.556+0000 7f652b7fe640 1 -- 172.21.15.70:0/2363090720 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65440035f0 con 0x55aa2c516b60 2023-10-08T05:55:32.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.556+0000 7f652b7fe640 1 -- 172.21.15.70:0/2363090720 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6544003750 con 0x55aa2c516b60 2023-10-08T05:55:32.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.556+0000 7f652b7fe640 1 -- 172.21.15.70:0/2363090720 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544003ae0 con 0x55aa2c516b60 2023-10-08T05:55:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.557+0000 7f6552c17740 1 -- 172.21.15.70:0/2363090720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 msgr2=0x55aa2c4e9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.557+0000 7f6552c17740 1 --2- 172.21.15.70:0/2363090720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4e9a60 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f65440014c0 tx=0x7f6544011000 comp rx=0 tx=0).stop 2023-10-08T05:55:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.557+0000 7f6552c17740 1 -- 172.21.15.70:0/2363090720 shutdown_connections 2023-10-08T05:55:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.557+0000 7f6552c17740 1 --2- 172.21.15.70:0/2363090720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c4e9fa0 0x55aa2c519e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.557+0000 7f6552c17740 1 --2- 172.21.15.70:0/2363090720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c5198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.557+0000 7f6552c17740 1 --2- 172.21.15.70:0/2363090720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4e9a60 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.557+0000 7f6552c17740 1 -- 172.21.15.70:0/2363090720 >> 172.21.15.70:0/2363090720 conn(0x55aa2c4ebd70 msgr2=0x55aa2c5348f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.557+0000 7f6552c17740 1 -- 172.21.15.70:0/2363090720 shutdown_connections 2023-10-08T05:55:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.557+0000 7f6552c17740 1 -- 172.21.15.70:0/2363090720 wait complete. 2023-10-08T05:55:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.558+0000 7f6552c17740 1 -- start start 2023-10-08T05:55:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.558+0000 7f6552c17740 1 Processor -- start 2023-10-08T05:55:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f6552c17740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c516b60 0x55aa2c508220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f6552c17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c548a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f6552c17740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c51be90 0x55aa2c51c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f6552c17740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c3f4970 con 0x55aa2c51be90 2023-10-08T05:55:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f6552c17740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c3f4610 con 0x55aa2c516f40 2023-10-08T05:55:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f6552c17740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c1baf20 con 0x55aa2c516b60 2023-10-08T05:55:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f6552c17740 10 client.0 _ll_register_callbacks cb 0x55aa2c432610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:55:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f654e620640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c51be90 0x55aa2c51c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f654e620640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c51be90 0x55aa2c51c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54986/0 (socket says 172.21.15.70:54986) 2023-10-08T05:55:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f654e620640 1 -- 172.21.15.70:0/2803840188 learned_addr learned my addr 172.21.15.70:0/2803840188 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f654e620640 1 -- 172.21.15.70:0/2803840188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c516b60 msgr2=0x55aa2c508220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f654e620640 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c516b60 0x55aa2c508220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f654e620640 1 -- 172.21.15.70:0/2803840188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 msgr2=0x55aa2c548a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.559+0000 7f654e620640 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c548a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.560+0000 7f654e620640 1 -- 172.21.15.70:0/2803840188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654400f040 con 0x55aa2c51be90 2023-10-08T05:55:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.560+0000 7f654e620640 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c51be90 0x55aa2c51c310 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6540034fd0 tx=0x7f6540009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bd77add5066c66 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.560+0000 7f6528ff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.560+0000 7f6528ff9640 1 -- 172.21.15.70:0/2803840188 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654007abf0 con 0x55aa2c51be90 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.560+0000 7f6528ff9640 1 -- 172.21.15.70:0/2803840188 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f654007f650 con 0x55aa2c51be90 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.560+0000 7f6552c17740 1 -- 172.21.15.70:0/2803840188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa2c44f6c0 con 0x55aa2c51be90 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.560+0000 7f6528ff9640 1 -- 172.21.15.70:0/2803840188 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6540083800 con 0x55aa2c51be90 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.560+0000 7f6552c17740 10 client.8199 fetch_fsmap learned FSMap version 58 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.560+0000 7f6552c17740 1 -- 172.21.15.70:0/2803840188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55aa2c454d60 con 0x55aa2c51be90 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.560+0000 7f6528ff9640 1 -- 172.21.15.70:0/2803840188 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa2c44f6c0 con 0x55aa2c51be90 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.561+0000 7f6528ff9640 1 -- 172.21.15.70:0/2803840188 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6540050020 con 0x55aa2c51be90 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.561+0000 7f6552c17740 10 client.8199 fetch_fsmap finished waiting for FSMap version 58 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.561+0000 7f6552c17740 10 client.8199 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T05:55:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.561+0000 7f6552c17740 20 client.8199 populate_metadata read hostname 'smithi070' 2023-10-08T05:55:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.561+0000 7f6552c17740 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.70:6832/3020884355,v1:172.21.15.70:6833/3020884355] conn(0x55aa2c52f8c0 0x55aa2c52b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.561+0000 7f6552c17740 4 client.8199 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T05:55:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.561+0000 7f6552c17740 1 -- 172.21.15.70:0/2803840188 --> [v2:172.21.15.70:6832/3020884355,v1:172.21.15.70:6833/3020884355] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa2c44f3e0 con 0x55aa2c52f8c0 2023-10-08T05:55:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.561+0000 7f654de1f640 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.70:6832/3020884355,v1:172.21.15.70:6833/3020884355] conn(0x55aa2c52f8c0 0x55aa2c52b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.561+0000 7f654de1f640 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.70:6832/3020884355,v1:172.21.15.70:6833/3020884355] conn(0x55aa2c52f8c0 0x55aa2c52b410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b209fb4492847a0 server_cookie=cecb96e1f512a7f0 in_seq=0 out_seq=0 2023-10-08T05:55:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.561+0000 7f6528ff9640 10 client.8199 ms_handle_connect on v2:172.21.15.70:6832/3020884355 2023-10-08T05:55:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.562+0000 7f6528ff9640 1 -- 172.21.15.70:0/2803840188 <== mds.0 v2:172.21.15.70:6832/3020884355 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa2c44f3e0 con 0x55aa2c52f8c0 2023-10-08T05:55:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.562+0000 7f6528ff9640 10 client.8199 handle_command_reply: tid=0 2023-10-08T05:55:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.562+0000 7f6552c17740 1 client.8199 shutdown 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.562+0000 7f6552c17740 1 -- 172.21.15.70:0/2803840188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c51be90 msgr2=0x55aa2c51c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.562+0000 7f6552c17740 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c51be90 0x55aa2c51c310 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f6540034fd0 tx=0x7f6540009510 comp rx=0 tx=0).stop 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.562+0000 7f6552c17740 1 -- 172.21.15.70:0/2803840188 shutdown_connections 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.562+0000 7f6552c17740 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.70:6832/3020884355,v1:172.21.15.70:6833/3020884355] conn(0x55aa2c52f8c0 0x55aa2c52b410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.563+0000 7f6552c17740 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c51be90 0x55aa2c51c310 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.563+0000 7f6552c17740 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c548a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.563+0000 7f6552c17740 1 --2- 172.21.15.70:0/2803840188 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c516b60 0x55aa2c508220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.563+0000 7f6528ff9640 0 client.8199 ms_handle_reset on v2:172.21.15.70:6832/3020884355 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.563+0000 7f6552c17740 1 -- 172.21.15.70:0/2803840188 >> 172.21.15.70:0/2803840188 conn(0x55aa2c4e9fa0 msgr2=0x55aa2c4ed9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.563+0000 7f6552c17740 1 -- 172.21.15.70:0/2803840188 shutdown_connections 2023-10-08T05:55:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.563+0000 7f6552c17740 1 -- 172.21.15.70:0/2803840188 wait complete. 2023-10-08T05:55:32.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.563+0000 7f6552c17740 20 client.8199 trim_cache size 0 max 16384 2023-10-08T05:55:32.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.579+0000 7f6552c17740 1 Processor -- start 2023-10-08T05:55:32.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.579+0000 7f6552c17740 1 -- start start 2023-10-08T05:55:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.580+0000 7f6552c17740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4ef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.580+0000 7f6552c17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c4efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.580+0000 7f6552c17740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c51be90 0x55aa2c52b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.580+0000 7f6552c17740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c458660 con 0x55aa2c516b60 2023-10-08T05:55:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.580+0000 7f6552c17740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c51c5d0 con 0x55aa2c516f40 2023-10-08T05:55:32.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.580+0000 7f6552c17740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c4eb7d0 con 0x55aa2c51be90 2023-10-08T05:55:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.580+0000 7f654de1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.580+0000 7f654de1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4ef880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54988/0 (socket says 172.21.15.70:54988) 2023-10-08T05:55:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.580+0000 7f654de1f640 1 -- 172.21.15.70:0/174095037 learned_addr learned my addr 172.21.15.70:0/174095037 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.581+0000 7f654de1f640 1 -- 172.21.15.70:0/174095037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c51be90 msgr2=0x55aa2c52b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.581+0000 7f654de1f640 1 --2- 172.21.15.70:0/174095037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c51be90 0x55aa2c52b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.581+0000 7f654de1f640 1 -- 172.21.15.70:0/174095037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 msgr2=0x55aa2c4efdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.581+0000 7f654de1f640 1 --2- 172.21.15.70:0/174095037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c4efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.581+0000 7f654de1f640 1 -- 172.21.15.70:0/174095037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa2c52ed40 con 0x55aa2c516b60 2023-10-08T05:55:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.581+0000 7f654de1f640 1 --2- 172.21.15.70:0/174095037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4ef880 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f6544001490 tx=0x7f6544005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7b0fda28b180b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:32.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.581+0000 7f652bfff640 1 -- 172.21.15.70:0/174095037 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544028040 con 0x55aa2c516b60 2023-10-08T05:55:32.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.581+0000 7f652bfff640 1 -- 172.21.15.70:0/174095037 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6544019070 con 0x55aa2c516b60 2023-10-08T05:55:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.581+0000 7f652bfff640 1 -- 172.21.15.70:0/174095037 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654402d5a0 con 0x55aa2c516b60 2023-10-08T05:55:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.582+0000 7f6552c17740 1 -- 172.21.15.70:0/174095037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 msgr2=0x55aa2c4ef880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.582+0000 7f6552c17740 1 --2- 172.21.15.70:0/174095037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4ef880 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f6544001490 tx=0x7f6544005000 comp rx=0 tx=0).stop 2023-10-08T05:55:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.582+0000 7f6552c17740 1 -- 172.21.15.70:0/174095037 shutdown_connections 2023-10-08T05:55:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.582+0000 7f6552c17740 1 --2- 172.21.15.70:0/174095037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c51be90 0x55aa2c52b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.582+0000 7f6552c17740 1 --2- 172.21.15.70:0/174095037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c4efdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.582+0000 7f6552c17740 1 --2- 172.21.15.70:0/174095037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4ef880 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.582+0000 7f6552c17740 1 -- 172.21.15.70:0/174095037 >> 172.21.15.70:0/174095037 conn(0x55aa2c4e9fa0 msgr2=0x55aa2c5302f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.582+0000 7f6552c17740 1 -- 172.21.15.70:0/174095037 shutdown_connections 2023-10-08T05:55:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.582+0000 7f6552c17740 1 -- 172.21.15.70:0/174095037 wait complete. 2023-10-08T05:55:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.583+0000 7f6552c17740 1 -- start start 2023-10-08T05:55:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.583+0000 7f6552c17740 1 Processor -- start 2023-10-08T05:55:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f6552c17740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4eff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f6552c17740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c5a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f6552c17740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c51be90 0x55aa2c5483f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f6552c17740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c54bea0 con 0x55aa2c516b60 2023-10-08T05:55:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f6552c17740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c51abf0 con 0x55aa2c516f40 2023-10-08T05:55:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f6552c17740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa2c51ae40 con 0x55aa2c51be90 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f6552c17740 10 client.0 _ll_register_callbacks cb 0x7f6544001d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f654de1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f654de1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4eff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55004/0 (socket says 172.21.15.70:55004) 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f654de1f640 1 -- 172.21.15.70:0/4228154817 learned_addr learned my addr 172.21.15.70:0/4228154817 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f654de1f640 1 -- 172.21.15.70:0/4228154817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c51be90 msgr2=0x55aa2c5483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f654de1f640 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c51be90 0x55aa2c5483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f654de1f640 1 -- 172.21.15.70:0/4228154817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 msgr2=0x55aa2c5a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f654de1f640 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c5a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.584+0000 7f654de1f640 1 -- 172.21.15.70:0/4228154817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa2c41a910 con 0x55aa2c516b60 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.585+0000 7f654de1f640 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4eff10 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6544017040 tx=0x7f6544000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c85a937443abfdd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.585+0000 7f6528ff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.585+0000 7f6528ff9640 1 -- 172.21.15.70:0/4228154817 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544028040 con 0x55aa2c516b60 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.585+0000 7f6528ff9640 1 -- 172.21.15.70:0/4228154817 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f654402db10 con 0x55aa2c516b60 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.585+0000 7f6552c17740 1 -- 172.21.15.70:0/4228154817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6544003750 con 0x55aa2c516b60 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.585+0000 7f6528ff9640 1 -- 172.21.15.70:0/4228154817 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f654402dcd0 con 0x55aa2c516b60 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.585+0000 7f6552c17740 10 client.8205 fetch_fsmap learned FSMap version 58 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.585+0000 7f6552c17740 1 -- 172.21.15.70:0/4228154817 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x55aa2c3f3b80 con 0x55aa2c516b60 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.585+0000 7f6528ff9640 1 -- 172.21.15.70:0/4228154817 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6544003750 con 0x55aa2c516b60 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.585+0000 7f6528ff9640 1 -- 172.21.15.70:0/4228154817 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 58) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6544035070 con 0x55aa2c516b60 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.586+0000 7f6552c17740 10 client.8205 fetch_fsmap finished waiting for FSMap version 58 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.586+0000 7f6552c17740 10 client.8205 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T05:55:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.586+0000 7f6552c17740 20 client.8205 populate_metadata read hostname 'smithi070' 2023-10-08T05:55:32.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.586+0000 7f6552c17740 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.70:6832/3020884355,v1:172.21.15.70:6833/3020884355] conn(0x55aa2c52a610 0x55aa2c52a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.586+0000 7f6552c17740 4 client.8205 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T05:55:32.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.586+0000 7f6552c17740 1 -- 172.21.15.70:0/4228154817 --> [v2:172.21.15.70:6832/3020884355,v1:172.21.15.70:6833/3020884355] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa2c1586c0 con 0x55aa2c52a610 2023-10-08T05:55:32.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.586+0000 7f654d61e640 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.70:6832/3020884355,v1:172.21.15.70:6833/3020884355] conn(0x55aa2c52a610 0x55aa2c52a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.586+0000 7f6528ff9640 10 client.8205 ms_handle_connect on v2:172.21.15.70:6832/3020884355 2023-10-08T05:55:32.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.586+0000 7f654d61e640 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.70:6832/3020884355,v1:172.21.15.70:6833/3020884355] conn(0x55aa2c52a610 0x55aa2c52a9f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf26ddce24ea6dc server_cookie=b8a53cdcedeb7979 in_seq=0 out_seq=0 2023-10-08T05:55:32.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6528ff9640 1 -- 172.21.15.70:0/4228154817 <== mds.0 v2:172.21.15.70:6832/3020884355 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55aa2c1586c0 con 0x55aa2c52a610 2023-10-08T05:55:32.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6528ff9640 10 client.8205 handle_command_reply: tid=0 2023-10-08T05:55:32.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 client.8205 shutdown 2023-10-08T05:55:32.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 -- 172.21.15.70:0/4228154817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 msgr2=0x55aa2c4eff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4eff10 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f6544017040 tx=0x7f6544000ed0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 -- 172.21.15.70:0/4228154817 shutdown_connections 2023-10-08T05:55:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.70:6832/3020884355,v1:172.21.15.70:6833/3020884355] conn(0x55aa2c52a610 0x55aa2c52a9f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55aa2c51be90 0x55aa2c5483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55aa2c516f40 0x55aa2c5a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6528ff9640 0 client.8205 ms_handle_reset on v2:172.21.15.70:6832/3020884355 2023-10-08T05:55:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 --2- 172.21.15.70:0/4228154817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55aa2c516b60 0x55aa2c4eff10 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 -- 172.21.15.70:0/4228154817 >> 172.21.15.70:0/4228154817 conn(0x55aa2c4e9fa0 msgr2=0x55aa2c51cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 -- 172.21.15.70:0/4228154817 shutdown_connections 2023-10-08T05:55:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 1 -- 172.21.15.70:0/4228154817 wait complete. 2023-10-08T05:55:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.587+0000 7f6552c17740 20 client.8205 trim_cache size 0 max 16384 2023-10-08T05:55:32.596 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:55:32.596 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T05:55:32.596 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T05:55:32.596 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 8052, 2023-10-08T05:55:32.596 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T05:55:32.596 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T05:55:32.596 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T05:55:32.597 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.6913357250000001, 2023-10-08T05:55:32.597 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 58, 2023-10-08T05:55:32.597 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 134, 2023-10-08T05:55:32.597 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 134, 2023-10-08T05:55:32.597 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 13.098750826 2023-10-08T05:55:32.597 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:55:32.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 -- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f652c06af90 msgr2=0x7f652c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f652c06af90 0x7f652c06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6544000f60 tx=0x7f6544009040 comp rx=0 tx=0).stop 2023-10-08T05:55:32.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 -- 172.21.15.70:0/231668307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 msgr2=0x7f65480c2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 0x7f65480c2350 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f6540036040 tx=0x7f6540004a40 comp rx=0 tx=0).stop 2023-10-08T05:55:32.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 -- 172.21.15.70:0/231668307 shutdown_connections 2023-10-08T05:55:32.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f652c06af90 0x7f652c06d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f654815aa00 0x7f65480c2350 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480c9cf0 0x7f65480c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 --2- 172.21.15.70:0/231668307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654806c3a0 0x7f6548152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 -- 172.21.15.70:0/231668307 >> 172.21.15.70:0/231668307 conn(0x7f654806a5d0 msgr2=0x7f6548153670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:32.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 -- 172.21.15.70:0/231668307 shutdown_connections 2023-10-08T05:55:32.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.588+0000 7f6550083640 1 -- 172.21.15.70:0/231668307 wait complete. 2023-10-08T05:55:32.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:55:32.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.814+0000 7fdb64cda640 1 Processor -- start 2023-10-08T05:55:32.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.815+0000 7fdb64cda640 1 -- start start 2023-10-08T05:55:32.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.816+0000 7fdb64cda640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdb600c90b0 0x7fdb600cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.816+0000 7fdb64cda640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6015b2b0 con 0x7fdb6015ab40 2023-10-08T05:55:32.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.816+0000 7fdb64cda640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6015b590 con 0x7fdb600cb3b0 2023-10-08T05:55:32.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.816+0000 7fdb64cda640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6015bf30 con 0x7fdb600c90b0 2023-10-08T05:55:32.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.816+0000 7fdb5e59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdb6015ab40 0x7fdb600ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54818/0 (socket says 172.21.15.70:54818) 2023-10-08T05:55:32.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.816+0000 7fdb5e59c640 1 -- 172.21.15.70:0/97382766 learned_addr learned my addr 172.21.15.70:0/97382766 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:32.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.817+0000 7fdb57fff640 1 --2- 172.21.15.70:0/97382766 >> v2:172.21.15.181:3300/0 conn(0x7fdb600c90b0 0x7fdb600cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.817+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119658438 0 0) 0x7fdb6015b2b0 con 0x7fdb6015ab40 2023-10-08T05:55:32.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.817+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb4c003520 con 0x7fdb6015ab40 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.817+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2768704 0 0) 0x7fdb4c003520 con 0x7fdb6015ab40 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.817+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 >> v2:172.21.15.181:3300/0 conn(0x7fdb600c90b0 msgr2=0x7fdb600cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.817+0000 7fdb5dd9b640 1 --2- 172.21.15.70:0/97382766 >> v2:172.21.15.181:3300/0 conn(0x7fdb600c90b0 0x7fdb600cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.817+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 >> v1:172.21.15.181:6789/0 conn(0x7fdb600cb3b0 legacy=0x7fdb6015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.817+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6015b870 con 0x7fdb6015ab40 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.817+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fdb50001670 con 0x7fdb6015ab40 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.817+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 >> v1:172.21.15.70:6789/0 conn(0x7fdb6015ab40 legacy=0x7fdb600ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb5dd9b640 1 --2- 172.21.15.70:0/97382766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb4c003970 0x7fdb4c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb5dd9b640 1 --2- 172.21.15.70:0/97382766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb5dd9b640 1 --2- 172.21.15.70:0/97382766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb4c005180 0x7fdb4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6015bf30 con 0x7fdb4c0047c0 2023-10-08T05:55:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c00c020 con 0x7fdb4c005180 2023-10-08T05:55:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c00c1e0 con 0x7fdb4c003970 2023-10-08T05:55:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb57fff640 1 --2- 172.21.15.70:0/97382766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb57fff640 1 -- 172.21.15.70:0/97382766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb4c003970 msgr2=0x7fdb4c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb57fff640 1 --2- 172.21.15.70:0/97382766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb4c003970 0x7fdb4c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb57fff640 1 -- 172.21.15.70:0/97382766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb4c005180 msgr2=0x7fdb4c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb57fff640 1 --2- 172.21.15.70:0/97382766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb4c005180 0x7fdb4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb57fff640 1 -- 172.21.15.70:0/97382766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c003520 con 0x7fdb4c0047c0 2023-10-08T05:55:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.818+0000 7fdb5ed9d640 1 -- 172.21.15.70:0/97382766 reap_dead start 2023-10-08T05:55:32.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.819+0000 7fdb57fff640 1 --2- 172.21.15.70:0/97382766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb4c004c40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009a90 tx=0x7fdb4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490cfb86c277985 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.820+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4802c6b0 con 0x7fdb4c0047c0 2023-10-08T05:55:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.820+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4802c810 con 0x7fdb4c0047c0 2023-10-08T05:55:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.820+0000 7fdb5dd9b640 1 -- 172.21.15.70:0/97382766 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4802cba0 con 0x7fdb4c0047c0 2023-10-08T05:55:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.820+0000 7fdb64cda640 1 -- 172.21.15.70:0/97382766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 msgr2=0x7fdb4c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.820+0000 7fdb64cda640 1 --2- 172.21.15.70:0/97382766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb4c004c40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009a90 tx=0x7fdb4c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.820+0000 7fdb64cda640 1 -- 172.21.15.70:0/97382766 shutdown_connections 2023-10-08T05:55:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.820+0000 7fdb64cda640 1 --2- 172.21.15.70:0/97382766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb4c004c40 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.820+0000 7fdb64cda640 1 -- 172.21.15.70:0/97382766 >> 172.21.15.70:0/97382766 conn(0x7fdb6006a380 msgr2=0x7fdb600b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.821+0000 7fdb64cda640 1 -- 172.21.15.70:0/97382766 shutdown_connections 2023-10-08T05:55:32.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.821+0000 7fdb64cda640 1 -- 172.21.15.70:0/97382766 wait complete. 2023-10-08T05:55:32.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.822+0000 7fdb64cda640 1 Processor -- start 2023-10-08T05:55:32.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.822+0000 7fdb64cda640 1 -- start start 2023-10-08T05:55:32.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.823+0000 7fdb64cda640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb600c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.823+0000 7fdb64cda640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb601466e0 0x7fdb600c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.823+0000 7fdb64cda640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb600c4b80 0x7fdb600c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.823+0000 7fdb64cda640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600c5540 con 0x7fdb4c0047c0 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.823+0000 7fdb64cda640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb60146b60 con 0x7fdb600c4b80 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.823+0000 7fdb64cda640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600c77d0 con 0x7fdb601466e0 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb5e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb600c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb5e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb600c71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55010/0 (socket says 172.21.15.70:55010) 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb5e59c640 1 -- 172.21.15.70:0/1125427905 learned_addr learned my addr 172.21.15.70:0/1125427905 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb57fff640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb601466e0 0x7fdb600c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb5e59c640 1 -- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb601466e0 msgr2=0x7fdb600c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb5e59c640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb601466e0 0x7fdb600c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb5e59c640 1 -- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb600c4b80 msgr2=0x7fdb600c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb5e59c640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb600c4b80 0x7fdb600c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb5e59c640 1 -- 172.21.15.70:0/1125427905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600c7ab0 con 0x7fdb4c0047c0 2023-10-08T05:55:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb5e59c640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb600c71f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c730 tx=0x7fdb5000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb56ffd640 1 -- 172.21.15.70:0/1125427905 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb500072d0 con 0x7fdb4c0047c0 2023-10-08T05:55:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb56ffd640 1 -- 172.21.15.70:0/1125427905 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb50007430 con 0x7fdb4c0047c0 2023-10-08T05:55:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.824+0000 7fdb64cda640 1 -- 172.21.15.70:0/1125427905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb600c7d90 con 0x7fdb4c0047c0 2023-10-08T05:55:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.825+0000 7fdb56ffd640 1 -- 172.21.15.70:0/1125427905 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb500047e0 con 0x7fdb4c0047c0 2023-10-08T05:55:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.825+0000 7fdb64cda640 1 -- 172.21.15.70:0/1125427905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb600c8350 con 0x7fdb4c0047c0 2023-10-08T05:55:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.826+0000 7fdb64cda640 1 -- 172.21.15.70:0/1125427905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb600c8540 con 0x7fdb4c0047c0 2023-10-08T05:55:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.827+0000 7fdb56ffd640 1 -- 172.21.15.70:0/1125427905 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdb50004940 con 0x7fdb4c0047c0 2023-10-08T05:55:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.828+0000 7fdb56ffd640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fdb4006af80 0x7fdb4006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.828+0000 7fdb56ffd640 1 -- 172.21.15.70:0/1125427905 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdb50089170 con 0x7fdb4c0047c0 2023-10-08T05:55:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.828+0000 7fdb57fff640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fdb4006af80 0x7fdb4006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:32.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.830+0000 7fdb57fff640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fdb4006af80 0x7fdb4006d440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdb48036000 tx=0x7fdb48030000 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:32.832+0000 7fdb56ffd640 1 -- 172.21.15.70:0/1125427905 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb50053fa0 con 0x7fdb4c0047c0 2023-10-08T05:55:33.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.062+0000 7fdb64cda640 1 -- 172.21.15.70:0/1125427905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb60067490 con 0x7fdb4c0047c0 2023-10-08T05:55:33.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.063+0000 7fdb56ffd640 1 -- 172.21.15.70:0/1125427905 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdb500575c0 con 0x7fdb4c0047c0 2023-10-08T05:55:33.063 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T05:55:33.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.065+0000 7fdb64cda640 1 -- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fdb4006af80 msgr2=0x7fdb4006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:33.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.065+0000 7fdb64cda640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fdb4006af80 0x7fdb4006d440 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdb48036000 tx=0x7fdb48030000 comp rx=0 tx=0).stop 2023-10-08T05:55:33.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.065+0000 7fdb64cda640 1 -- 172.21.15.70:0/1125427905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 msgr2=0x7fdb600c71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:33.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.065+0000 7fdb64cda640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb600c71f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c730 tx=0x7fdb5000cbf0 comp rx=0 tx=0).stop 2023-10-08T05:55:33.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.066+0000 7fdb64cda640 1 -- 172.21.15.70:0/1125427905 shutdown_connections 2023-10-08T05:55:33.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.066+0000 7fdb64cda640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb600c4b80 0x7fdb600c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:33.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.066+0000 7fdb64cda640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb601466e0 0x7fdb600c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:33.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.066+0000 7fdb64cda640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fdb4006af80 0x7fdb4006d440 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:33.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.066+0000 7fdb64cda640 1 --2- 172.21.15.70:0/1125427905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb4c0047c0 0x7fdb600c71f0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:33.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.066+0000 7fdb64cda640 1 -- 172.21.15.70:0/1125427905 >> 172.21.15.70:0/1125427905 conn(0x7fdb6006a380 msgr2=0x7fdb6006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:33.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.066+0000 7fdb64cda640 1 -- 172.21.15.70:0/1125427905 shutdown_connections 2023-10-08T05:55:33.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:33.066+0000 7fdb64cda640 1 -- 172.21.15.70:0/1125427905 wait complete. 2023-10-08T05:55:33.083 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T05:55:33.083 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:55:33.083 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:55:33.083 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T05:55:33.083 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T05:55:33.083 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T05:55:33.083 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T05:55:33.083 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:55:33.083 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:55:33.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:33.084 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:55:33.136 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:55:33.136 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:55:33.136 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:55:33.136 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:55:33.136 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:55:33.136 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:55:33.136 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:55:33.136 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41263sec preferred_lft 41263sec 2023-10-08T05:55:33.137 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:55:33.138 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:55:33.138 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:55:33.138 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:55:33.138 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T05:55:33.138 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:55:33.138 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T05:55:33.138 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:55:33.138 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:55:33.138 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T05:55:33.139 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T05:55:33.139 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:55:33.139 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:55:33.139 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T05:55:33.139 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T05:55:33.139 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:55:33.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:33.139 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:55:33.226 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:55:33.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:33.230 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:55:33.285 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:55:33.285 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:55:33.285 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:55:33.285 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:55:33.285 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:55:33.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:33.286 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:55:33.286 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:55:33.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:55:33.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:55:33.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:55:33.286 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:55:33.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:33.475 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:55:33.527 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T05:55:33.527 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T05:55:33.527 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:55:33.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:33.527 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T05:55:33.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:55:33.623 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T05:55:33.678 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:55:33.678 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:55:33.678 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:55:33.731 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T05:55:33.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:33.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:55:33.874 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:55:33.874 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:55:33.907 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:55:33.907 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:55:33.908 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:55:33.908 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:55:33.908 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:55:33.910 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T05:55:33.910 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:55:33.910 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:55:33.910 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:55:33.910 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:55:33.916 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:55:33.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:33.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:55:33.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:33.998 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:55:34.057 INFO:teuthology.orchestra.run.smithi070.stdout:client5909 2023-10-08T05:55:34.057 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:55:34.058 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:55:34.107 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T05:55:34.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.373+0000 7fbfd3c00640 1 Processor -- start 2023-10-08T05:55:34.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.373+0000 7fbfd3c00640 1 -- start start 2023-10-08T05:55:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.374+0000 7fbfd3c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfcc0afbe0 0x7fbfcc0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.374+0000 7fbfd3c00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbfcc06dfc0 0x7fbfcc070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.374+0000 7fbfd3c00640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfcc071070 0x7fbfcc0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.374+0000 7fbfd3c00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfcc0b2620 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.374+0000 7fbfd3c00640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfcc0b2060 con 0x7fbfcc071070 2023-10-08T05:55:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.374+0000 7fbfd3c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc0b2340 con 0x7fbfcc0afbe0 2023-10-08T05:55:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.374+0000 7fbfd119b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbfcc06dfc0 0x7fbfcc070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.374+0000 7fbfd119b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbfcc06dfc0 0x7fbfcc070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39446/0 (socket says 172.21.15.70:39446) 2023-10-08T05:55:34.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.374+0000 7fbfd119b640 1 -- 172.21.15.70:0/2906584600 learned_addr learned my addr 172.21.15.70:0/2906584600 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:34.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.375+0000 7fbfd199c640 1 --2- 172.21.15.70:0/2906584600 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfcc0afbe0 0x7fbfcc0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.375+0000 7fbfd219d640 1 --2- 172.21.15.70:0/2906584600 >> v2:172.21.15.181:3300/0 conn(0x7fbfcc071070 0x7fbfcc0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.375+0000 7fbfd119b640 1 -- 172.21.15.70:0/2906584600 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfcc0afbe0 msgr2=0x7fbfcc0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:34.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.375+0000 7fbfd119b640 1 --2- 172.21.15.70:0/2906584600 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfcc0afbe0 0x7fbfcc0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.375+0000 7fbfd119b640 1 -- 172.21.15.70:0/2906584600 >> v2:172.21.15.181:3300/0 conn(0x7fbfcc071070 msgr2=0x7fbfcc0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:34.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.375+0000 7fbfd119b640 1 --2- 172.21.15.70:0/2906584600 >> v2:172.21.15.181:3300/0 conn(0x7fbfcc071070 0x7fbfcc0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.375+0000 7fbfd119b640 1 -- 172.21.15.70:0/2906584600 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfcc070350 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.376+0000 7fbfd119b640 1 --2- 172.21.15.70:0/2906584600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 0x7fbfcc070b30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0b3980 tx=0x7fbfc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f69fe02f1a2bda server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:34.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.376+0000 7fbfd099a640 1 -- 172.21.15.70:0/2906584600 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0031870 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.376+0000 7fbfd099a640 1 -- 172.21.15.70:0/2906584600 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc00319d0 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.376+0000 7fbfd099a640 1 -- 172.21.15.70:0/2906584600 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0031d60 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.377+0000 7fbfd3c00640 1 -- 172.21.15.70:0/2906584600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 msgr2=0x7fbfcc070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:34.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.377+0000 7fbfd3c00640 1 --2- 172.21.15.70:0/2906584600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 0x7fbfcc070b30 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0b3980 tx=0x7fbfc002b910 comp rx=0 tx=0).stop 2023-10-08T05:55:34.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.377+0000 7fbfd3c00640 1 -- 172.21.15.70:0/2906584600 shutdown_connections 2023-10-08T05:55:34.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.377+0000 7fbfd3c00640 1 --2- 172.21.15.70:0/2906584600 >> v2:172.21.15.181:3300/0 conn(0x7fbfcc071070 0x7fbfcc0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.377+0000 7fbfd3c00640 1 --2- 172.21.15.70:0/2906584600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 0x7fbfcc070b30 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.377+0000 7fbfd3c00640 1 --2- 172.21.15.70:0/2906584600 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfcc0afbe0 0x7fbfcc0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.377+0000 7fbfd3c00640 1 -- 172.21.15.70:0/2906584600 >> 172.21.15.70:0/2906584600 conn(0x7fbfcc06a5d0 msgr2=0x7fbfcc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:34.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.377+0000 7fbfd3c00640 1 -- 172.21.15.70:0/2906584600 shutdown_connections 2023-10-08T05:55:34.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.377+0000 7fbfd3c00640 1 -- 172.21.15.70:0/2906584600 wait complete. 2023-10-08T05:55:34.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.379+0000 7fbfd3c00640 1 Processor -- start 2023-10-08T05:55:34.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.379+0000 7fbfd3c00640 1 -- start start 2023-10-08T05:55:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.380+0000 7fbfd3c00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 0x7fbfcc1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.380+0000 7fbfd3c00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfcc071070 0x7fbfcc0c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.380+0000 7fbfd3c00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfcc0afbe0 0x7fbfcc0c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.380+0000 7fbfd3c00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc0c2810 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.380+0000 7fbfd3c00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc0c2af0 con 0x7fbfcc071070 2023-10-08T05:55:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.380+0000 7fbfd3c00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfcc0c2dd0 con 0x7fbfcc0afbe0 2023-10-08T05:55:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.380+0000 7fbfd199c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 0x7fbfcc1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.380+0000 7fbfd199c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 0x7fbfcc1534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39452/0 (socket says 172.21.15.70:39452) 2023-10-08T05:55:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.380+0000 7fbfd199c640 1 -- 172.21.15.70:0/3418036487 learned_addr learned my addr 172.21.15.70:0/3418036487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfd119b640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfcc071070 0x7fbfcc0c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfd199c640 1 -- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfcc0afbe0 msgr2=0x7fbfcc0c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfd199c640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfcc0afbe0 0x7fbfcc0c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfd199c640 1 -- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfcc071070 msgr2=0x7fbfcc0c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfd199c640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfcc071070 0x7fbfcc0c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfd199c640 1 -- 172.21.15.70:0/3418036487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00094d0 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfd199c640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 0x7fbfcc1534f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007ef0 tx=0x7fbfbc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:34.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfca7fc640 1 -- 172.21.15.70:0/3418036487 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc014620 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfca7fc640 1 -- 172.21.15.70:0/3418036487 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc014780 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfca7fc640 1 -- 172.21.15.70:0/3418036487 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc014b30 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.381+0000 7fbfd3c00640 1 -- 172.21.15.70:0/3418036487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfcc0c30b0 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.382+0000 7fbfd3c00640 1 -- 172.21.15.70:0/3418036487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfcc0c4f20 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.383+0000 7fbfd3c00640 1 -- 172.21.15.70:0/3418036487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfcc04e160 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.384+0000 7fbfca7fc640 1 -- 172.21.15.70:0/3418036487 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbfbc014c90 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.385+0000 7fbfca7fc640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbfb806af90 0x7fbfb806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.385+0000 7fbfca7fc640 1 -- 172.21.15.70:0/3418036487 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbfbc08a300 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.385+0000 7fbfd119b640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbfb806af90 0x7fbfb806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.386+0000 7fbfd119b640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbfb806af90 0x7fbfb806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0036000 tx=0x7fbfc003c040 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:34.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.389+0000 7fbfca7fc640 1 -- 172.21.15.70:0/3418036487 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbfbc055120 con 0x7fbfcc06dfc0 2023-10-08T05:55:34.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.604+0000 7fbfd3c00640 1 -- 172.21.15.70:0/3418036487 --> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfcc0c2c60 con 0x7fbfb806af90 2023-10-08T05:55:34.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.608+0000 7fbfca7fc640 1 -- 172.21.15.70:0/3418036487 <== mgr.5747 v2:172.21.15.181:6832/3142216408 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbfcc0c2c60 con 0x7fbfb806af90 2023-10-08T05:55:34.608 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:55:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:55:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T05:55:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:55:34.609 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:55:34.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.610+0000 7fbfd3c00640 1 -- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbfb806af90 msgr2=0x7fbfb806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:34.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.610+0000 7fbfd3c00640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbfb806af90 0x7fbfb806d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0036000 tx=0x7fbfc003c040 comp rx=0 tx=0).stop 2023-10-08T05:55:34.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.610+0000 7fbfd3c00640 1 -- 172.21.15.70:0/3418036487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 msgr2=0x7fbfcc1534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:34.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.610+0000 7fbfd3c00640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 0x7fbfcc1534f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc007ef0 tx=0x7fbfbc00e4b0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.610+0000 7fbfd3c00640 1 -- 172.21.15.70:0/3418036487 shutdown_connections 2023-10-08T05:55:34.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.610+0000 7fbfd3c00640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfcc0afbe0 0x7fbfcc0c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.610+0000 7fbfd3c00640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfcc071070 0x7fbfcc0c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.610+0000 7fbfd3c00640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fbfb806af90 0x7fbfb806d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.610+0000 7fbfd3c00640 1 --2- 172.21.15.70:0/3418036487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfcc06dfc0 0x7fbfcc1534f0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.611+0000 7fbfd3c00640 1 -- 172.21.15.70:0/3418036487 >> 172.21.15.70:0/3418036487 conn(0x7fbfcc06a5d0 msgr2=0x7fbfcc0732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:34.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.611+0000 7fbfd3c00640 1 -- 172.21.15.70:0/3418036487 shutdown_connections 2023-10-08T05:55:34.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.611+0000 7fbfd3c00640 1 -- 172.21.15.70:0/3418036487 wait complete. 2023-10-08T05:55:34.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T05:55:34.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.851+0000 7f95ee3f5640 1 Processor -- start 2023-10-08T05:55:34.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.852+0000 7f95ee3f5640 1 -- start start 2023-10-08T05:55:34.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.853+0000 7f95ee3f5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f95e80cb880 0x7f95e80b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.853+0000 7f95ee3f5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e815d440 con 0x7f95e81432f0 2023-10-08T05:55:34.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.853+0000 7f95ee3f5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e815d690 con 0x7f95e80b4070 2023-10-08T05:55:34.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.853+0000 7f95ee3f5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e81436d0 con 0x7f95e80cb880 2023-10-08T05:55:34.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.853+0000 7f95e7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f95e81432f0 0x7f95e80cb170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52200/0 (socket says 172.21.15.70:52200) 2023-10-08T05:55:34.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.853+0000 7f95e7fff640 1 -- 172.21.15.70:0/3995039728 learned_addr learned my addr 172.21.15.70:0/3995039728 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:34.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276810318 0 0) 0x7f95e815d440 con 0x7f95e81432f0 2023-10-08T05:55:34.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d4003520 con 0x7f95e81432f0 2023-10-08T05:55:34.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e77fe640 1 --2- 172.21.15.70:0/3995039728 >> v2:172.21.15.181:3300/0 conn(0x7f95e80cb880 0x7f95e80b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464162609 0 0) 0x7f95d4003520 con 0x7f95e81432f0 2023-10-08T05:55:34.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 >> v2:172.21.15.181:3300/0 conn(0x7f95e80cb880 msgr2=0x7f95e80b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:34.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 --2- 172.21.15.70:0/3995039728 >> v2:172.21.15.181:3300/0 conn(0x7f95e80cb880 0x7f95e80b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 >> v1:172.21.15.181:6789/0 conn(0x7f95e80b4070 legacy=0x7f95e8159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e815d970 con 0x7f95e81432f0 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f95d80019f0 con 0x7f95e81432f0 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 >> v1:172.21.15.70:6789/0 conn(0x7f95e81432f0 legacy=0x7f95e80cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95d4003970 0x7f95d4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95d40052a0 0x7f95d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e81436d0 con 0x7f95d40047c0 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d400c150 con 0x7f95d4003970 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.854+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d400c310 con 0x7f95d40052a0 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.855+0000 7f95e77fe640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.855+0000 7f95e7fff640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95d4003970 0x7f95d4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.855+0000 7f95ec992640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95d40052a0 0x7f95d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.855+0000 7f95e77fe640 1 -- 172.21.15.70:0/3995039728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95d40052a0 msgr2=0x7f95d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:34.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.855+0000 7f95e77fe640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95d40052a0 0x7f95d400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.855+0000 7f95e77fe640 1 -- 172.21.15.70:0/3995039728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95d4003970 msgr2=0x7f95d4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:34.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.855+0000 7f95e77fe640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95d4003970 0x7f95d4003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.855+0000 7f95e77fe640 1 -- 172.21.15.70:0/3995039728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d4003520 con 0x7f95d40047c0 2023-10-08T05:55:34.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.855+0000 7f95ec992640 1 -- 172.21.15.70:0/3995039728 reap_dead start 2023-10-08T05:55:34.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.856+0000 7f95e77fe640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95d4004c40 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f95d000ba00 tx=0x7f95d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13c9fd221515b48 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:34.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.856+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0033680 con 0x7f95d40047c0 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.857+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d00337e0 con 0x7f95d40047c0 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.857+0000 7f95e6ffd640 1 -- 172.21.15.70:0/3995039728 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0033b70 con 0x7f95d40047c0 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.857+0000 7f95ee3f5640 1 -- 172.21.15.70:0/3995039728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 msgr2=0x7f95d4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.857+0000 7f95ee3f5640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95d4004c40 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f95d000ba00 tx=0x7f95d40057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.857+0000 7f95ee3f5640 1 -- 172.21.15.70:0/3995039728 shutdown_connections 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.857+0000 7f95ee3f5640 1 --2- 172.21.15.70:0/3995039728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95d4004c40 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.857+0000 7f95ee3f5640 1 -- 172.21.15.70:0/3995039728 >> 172.21.15.70:0/3995039728 conn(0x7f95e8069d50 msgr2=0x7f95e806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.857+0000 7f95ee3f5640 1 -- 172.21.15.70:0/3995039728 shutdown_connections 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.857+0000 7f95ee3f5640 1 -- 172.21.15.70:0/3995039728 wait complete. 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.859+0000 7f95ee3f5640 1 Processor -- start 2023-10-08T05:55:34.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.859+0000 7f95ee3f5640 1 -- start start 2023-10-08T05:55:34.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.860+0000 7f95ee3f5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95e8156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.860+0000 7f95ee3f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95e80caf50 0x7f95e81393f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.860+0000 7f95ee3f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e8137a60 0x7f95e80cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.860+0000 7f95ee3f5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e8139930 con 0x7f95d40047c0 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.860+0000 7f95ee3f5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80cb4f0 con 0x7f95e8137a60 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.860+0000 7f95ee3f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e8136ca0 con 0x7f95e80caf50 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.860+0000 7f95e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95e8156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.860+0000 7f95e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95e8156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39474/0 (socket says 172.21.15.70:39474) 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.860+0000 7f95e7fff640 1 -- 172.21.15.70:0/1220115718 learned_addr learned my addr 172.21.15.70:0/1220115718 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95e77fe640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95e80caf50 0x7f95e81393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95e7fff640 1 -- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95e80caf50 msgr2=0x7f95e81393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95ec992640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e8137a60 0x7f95e80cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95e7fff640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95e80caf50 0x7f95e81393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95e7fff640 1 -- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e8137a60 msgr2=0x7f95e80cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:34.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95e7fff640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e8137a60 0x7f95e80cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:34.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95e7fff640 1 -- 172.21.15.70:0/1220115718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e8136f80 con 0x7f95d40047c0 2023-10-08T05:55:34.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95e7fff640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95e8156770 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f95d8007b30 tx=0x7f95d800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:34.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95e4ff9640 1 -- 172.21.15.70:0/1220115718 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8013420 con 0x7f95d40047c0 2023-10-08T05:55:34.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95e4ff9640 1 -- 172.21.15.70:0/1220115718 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d80135a0 con 0x7f95d40047c0 2023-10-08T05:55:34.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95e4ff9640 1 -- 172.21.15.70:0/1220115718 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8013950 con 0x7f95d40047c0 2023-10-08T05:55:34.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.861+0000 7f95ee3f5640 1 -- 172.21.15.70:0/1220115718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e8137260 con 0x7f95d40047c0 2023-10-08T05:55:34.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.862+0000 7f95ee3f5640 1 -- 172.21.15.70:0/1220115718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e8137ee0 con 0x7f95d40047c0 2023-10-08T05:55:34.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.863+0000 7f95ee3f5640 1 -- 172.21.15.70:0/1220115718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e8137780 con 0x7f95d40047c0 2023-10-08T05:55:34.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.864+0000 7f95e4ff9640 1 -- 172.21.15.70:0/1220115718 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 56) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f95d8013ab0 con 0x7f95d40047c0 2023-10-08T05:55:34.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.865+0000 7f95e4ff9640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f95c806afb0 0x7f95c806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:34.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.865+0000 7f95e4ff9640 1 -- 172.21.15.70:0/1220115718 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f95d8089ff0 con 0x7f95d40047c0 2023-10-08T05:55:34.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.866+0000 7f95e77fe640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f95c806afb0 0x7f95c806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:34.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.867+0000 7f95e77fe640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f95c806afb0 0x7f95c806d470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f95d40057a0 tx=0x7f95d0030000 comp rx=0 tx=0).ready entity=mgr.5747 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:34.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:34.869+0000 7f95e4ff9640 1 -- 172.21.15.70:0/1220115718 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95e8137780 con 0x7f95d40047c0 2023-10-08T05:55:35.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.107+0000 7f95ee3f5640 1 -- 172.21.15.70:0/1220115718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f95e8137780 con 0x7f95d40047c0 2023-10-08T05:55:35.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.416+0000 7f95e4ff9640 1 -- 172.21.15.70:0/1220115718 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v57) v1 ==== 92+0+0 (secure 0 0 0) 0x7f95e8137780 con 0x7f95d40047c0 2023-10-08T05:55:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.419+0000 7f95ee3f5640 1 -- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f95c806afb0 msgr2=0x7f95c806d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.419+0000 7f95ee3f5640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f95c806afb0 0x7f95c806d470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f95d40057a0 tx=0x7f95d0030000 comp rx=0 tx=0).stop 2023-10-08T05:55:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.419+0000 7f95ee3f5640 1 -- 172.21.15.70:0/1220115718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 msgr2=0x7f95e8156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.419+0000 7f95ee3f5640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95e8156770 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f95d8007b30 tx=0x7f95d800e530 comp rx=0 tx=0).stop 2023-10-08T05:55:35.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.419+0000 7f95ee3f5640 1 -- 172.21.15.70:0/1220115718 shutdown_connections 2023-10-08T05:55:35.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.419+0000 7f95ee3f5640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95e8137a60 0x7f95e80cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:35.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.419+0000 7f95ee3f5640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95e80caf50 0x7f95e81393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:35.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.419+0000 7f95ee3f5640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f95c806afb0 0x7f95c806d470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:35.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.419+0000 7f95ee3f5640 1 --2- 172.21.15.70:0/1220115718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95d40047c0 0x7f95e8156770 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:35.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.419+0000 7f95ee3f5640 1 -- 172.21.15.70:0/1220115718 >> 172.21.15.70:0/1220115718 conn(0x7f95e8069d50 msgr2=0x7f95e80b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:35.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.420+0000 7f95ee3f5640 1 -- 172.21.15.70:0/1220115718 shutdown_connections 2023-10-08T05:55:35.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.420+0000 7f95ee3f5640 1 -- 172.21.15.70:0/1220115718 wait complete. 2023-10-08T05:55:35.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T05:55:35.924 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:35.923+0000 7fd5eb02e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:55:35.938 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:35.938+0000 7fe6cf634040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:55:35.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.961+0000 7f392c355640 1 Processor -- start 2023-10-08T05:55:35.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.961+0000 7f392c355640 1 -- start start 2023-10-08T05:55:35.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.962+0000 7f392c355640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f392415cd70 0x7f39241612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:35.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.962+0000 7f392c355640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39240b4e10 0x7f39240d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:35.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.963+0000 7f392c355640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39240b8c30 con 0x7f39240cd4e0 2023-10-08T05:55:35.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.963+0000 7f392c355640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39240cdd80 con 0x7f392415cd70 2023-10-08T05:55:35.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.963+0000 7f392c355640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39240ce060 con 0x7f39240b4e10 2023-10-08T05:55:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.963+0000 7f392a0f1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f392415cd70 0x7f39241612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.963+0000 7f392a0f1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f392415cd70 0x7f39241612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:59586/0 (socket says 172.21.15.70:59586) 2023-10-08T05:55:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.963+0000 7f392a0f1640 1 -- 172.21.15.70:0/3966851685 learned_addr learned my addr 172.21.15.70:0/3966851685 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.963+0000 7f392a8f2640 1 --2- 172.21.15.70:0/3966851685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39240b4e10 0x7f39240d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.964+0000 7f392a0f1640 1 -- 172.21.15.70:0/3966851685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39240b4e10 msgr2=0x7f39240d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:35.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.964+0000 7f392a0f1640 1 --2- 172.21.15.70:0/3966851685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39240b4e10 0x7f39240d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:35.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.964+0000 7f392a0f1640 1 -- 172.21.15.70:0/3966851685 >> v1:172.21.15.181:6789/0 conn(0x7f39240cd4e0 legacy=0x7f39240b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:35.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.964+0000 7f392a0f1640 1 -- 172.21.15.70:0/3966851685 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39240ce340 con 0x7f392415cd70 2023-10-08T05:55:35.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.965+0000 7f392a0f1640 1 --2- 172.21.15.70:0/3966851685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f392415cd70 0x7f39241612f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3920009870 tx=0x7f392002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6a3c98d3ba9daf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:35.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.965+0000 7f39290ef640 1 -- 172.21.15.70:0/3966851685 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920030840 con 0x7f392415cd70 2023-10-08T05:55:35.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.965+0000 7f39290ef640 1 -- 172.21.15.70:0/3966851685 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39200309a0 con 0x7f392415cd70 2023-10-08T05:55:35.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.965+0000 7f39290ef640 1 -- 172.21.15.70:0/3966851685 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920030d30 con 0x7f392415cd70 2023-10-08T05:55:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.966+0000 7f392c355640 1 -- 172.21.15.70:0/3966851685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f392415cd70 msgr2=0x7f39241612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.966+0000 7f392c355640 1 --2- 172.21.15.70:0/3966851685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f392415cd70 0x7f39241612f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3920009870 tx=0x7f392002b980 comp rx=0 tx=0).stop 2023-10-08T05:55:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.966+0000 7f392c355640 1 -- 172.21.15.70:0/3966851685 shutdown_connections 2023-10-08T05:55:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.966+0000 7f392c355640 1 --2- 172.21.15.70:0/3966851685 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39240b4e10 0x7f39240d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.966+0000 7f392c355640 1 --2- 172.21.15.70:0/3966851685 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f392415cd70 0x7f39241612f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.966+0000 7f392c355640 1 -- 172.21.15.70:0/3966851685 >> 172.21.15.70:0/3966851685 conn(0x7f39240cf700 msgr2=0x7f39240cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.966+0000 7f392c355640 1 -- 172.21.15.70:0/3966851685 shutdown_connections 2023-10-08T05:55:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.966+0000 7f392c355640 1 -- 172.21.15.70:0/3966851685 wait complete. 2023-10-08T05:55:35.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.968+0000 7f392c355640 1 Processor -- start 2023-10-08T05:55:35.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.968+0000 7f392c355640 1 -- start start 2023-10-08T05:55:35.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.969+0000 7f392c355640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39240b4e10 0x7f39240c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:35.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.969+0000 7f392c355640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39240cd4e0 0x7f39240c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:35.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.969+0000 7f392c355640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f392415cd70 0x7f39240c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:35.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.969+0000 7f392c355640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39240c8010 con 0x7f39240b4e10 2023-10-08T05:55:35.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.969+0000 7f392c355640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241e6560 con 0x7f39240cd4e0 2023-10-08T05:55:35.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.969+0000 7f392c355640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39240c7390 con 0x7f392415cd70 2023-10-08T05:55:35.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.969+0000 7f392a0f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39240b4e10 0x7f39240c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:35.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.969+0000 7f392a0f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39240b4e10 0x7f39240c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39504/0 (socket says 172.21.15.70:39504) 2023-10-08T05:55:35.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.969+0000 7f392a0f1640 1 -- 172.21.15.70:0/2109939597 learned_addr learned my addr 172.21.15.70:0/2109939597 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:35.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.969+0000 7f39298f0640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39240cd4e0 0x7f39240c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:35.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f392a8f2640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f392415cd70 0x7f39240c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:35.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f392a0f1640 1 -- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f392415cd70 msgr2=0x7f39240c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:35.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f392a0f1640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f392415cd70 0x7f39240c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f392a0f1640 1 -- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39240cd4e0 msgr2=0x7f39240c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f392a0f1640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39240cd4e0 0x7f39240c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f392a0f1640 1 -- 172.21.15.70:0/2109939597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39200094d0 con 0x7f39240b4e10 2023-10-08T05:55:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f392a0f1640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39240b4e10 0x7f39240c8f30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f392002b9c0 tx=0x7f3920004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f391affd640 1 -- 172.21.15.70:0/2109939597 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920039c90 con 0x7f39240b4e10 2023-10-08T05:55:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f391affd640 1 -- 172.21.15.70:0/2109939597 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3920039df0 con 0x7f39240b4e10 2023-10-08T05:55:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f391affd640 1 -- 172.21.15.70:0/2109939597 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200446c0 con 0x7f39240b4e10 2023-10-08T05:55:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.970+0000 7f392c355640 1 -- 172.21.15.70:0/2109939597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39240c7670 con 0x7f39240b4e10 2023-10-08T05:55:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.971+0000 7f392c355640 1 -- 172.21.15.70:0/2109939597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39240c7c30 con 0x7f39240b4e10 2023-10-08T05:55:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.972+0000 7f392c355640 1 -- 172.21.15.70:0/2109939597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392404e160 con 0x7f39240b4e10 2023-10-08T05:55:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.973+0000 7f391affd640 1 -- 172.21.15.70:0/2109939597 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 57) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f392002f390 con 0x7f39240b4e10 2023-10-08T05:55:35.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.974+0000 7f391affd640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f390806afe0 0x7f390806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:35.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.975+0000 7f391affd640 1 -- 172.21.15.70:0/2109939597 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f392003e070 con 0x7f39240b4e10 2023-10-08T05:55:35.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.975+0000 7f39298f0640 1 -- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f390806afe0 msgr2=0x7f390806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3142216408 2023-10-08T05:55:35.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.975+0000 7f39298f0640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f390806afe0 0x7f390806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:35.977 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:35.976+0000 7fd5eb02e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:55:35.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:35.980+0000 7f391affd640 1 -- 172.21.15.70:0/2109939597 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3920077170 con 0x7f39240b4e10 2023-10-08T05:55:35.996 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:35.996+0000 7fe6cf634040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:55:36.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.170+0000 7f392c355640 1 -- 172.21.15.70:0/2109939597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f39240b4760 con 0x7f39240b4e10 2023-10-08T05:55:36.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.174+0000 7f391affd640 1 -- 172.21.15.70:0/2109939597 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f392007a790 con 0x7f39240b4e10 2023-10-08T05:55:36.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.176+0000 7f39298f0640 1 -- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f390806afe0 msgr2=0x7f390806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3142216408 2023-10-08T05:55:36.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.176+0000 7f39298f0640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f390806afe0 0x7f390806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:55:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 -- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f390806afe0 msgr2=0x7f390806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:36.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f390806afe0 0x7f390806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 -- 172.21.15.70:0/2109939597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39240b4e10 msgr2=0x7f39240c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39240b4e10 0x7f39240c8f30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f392002b9c0 tx=0x7f3920004730 comp rx=0 tx=0).stop 2023-10-08T05:55:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 -- 172.21.15.70:0/2109939597 shutdown_connections 2023-10-08T05:55:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f392415cd70 0x7f39240c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39240cd4e0 0x7f39240c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f390806afe0 0x7f390806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 --2- 172.21.15.70:0/2109939597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39240b4e10 0x7f39240c8f30 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 -- 172.21.15.70:0/2109939597 >> 172.21.15.70:0/2109939597 conn(0x7f39240cf700 msgr2=0x7f39240b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 -- 172.21.15.70:0/2109939597 shutdown_connections 2023-10-08T05:55:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.177+0000 7f392c355640 1 -- 172.21.15.70:0/2109939597 wait complete. 2023-10-08T05:55:36.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T05:55:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.399+0000 7fc7f8eec640 1 Processor -- start 2023-10-08T05:55:36.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.399+0000 7fc7f8eec640 1 -- start start 2023-10-08T05:55:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.400+0000 7fc7f8eec640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc7f40cf2b0 0x7fc7f40d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.400+0000 7fc7f8eec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc7f40ce510 0x7fc7f415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.400+0000 7fc7f8eec640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f40cd520 con 0x7fc7f40cf690 2023-10-08T05:55:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.400+0000 7fc7f8eec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f40d1910 con 0x7fc7f40ce510 2023-10-08T05:55:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.400+0000 7fc7f8eec640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f40ce080 con 0x7fc7f40cf2b0 2023-10-08T05:55:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.401+0000 7fc7f3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc7f40ce510 0x7fc7f415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.401+0000 7fc7f3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc7f40ce510 0x7fc7f415c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39506/0 (socket says 172.21.15.70:39506) 2023-10-08T05:55:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.401+0000 7fc7f3fff640 1 -- 172.21.15.70:0/1488944291 learned_addr learned my addr 172.21.15.70:0/1488944291 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.401+0000 7fc7f37fe640 1 --2- 172.21.15.70:0/1488944291 >> v2:172.21.15.181:3300/0 conn(0x7fc7f40cf2b0 0x7fc7f40d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:36.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.401+0000 7fc7f3fff640 1 -- 172.21.15.70:0/1488944291 >> v2:172.21.15.181:3300/0 conn(0x7fc7f40cf2b0 msgr2=0x7fc7f40d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.401+0000 7fc7f3fff640 1 --2- 172.21.15.70:0/1488944291 >> v2:172.21.15.181:3300/0 conn(0x7fc7f40cf2b0 0x7fc7f40d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.401+0000 7fc7f3fff640 1 -- 172.21.15.70:0/1488944291 >> v1:172.21.15.181:6789/0 conn(0x7fc7f40cf690 legacy=0x7fc7f40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.401+0000 7fc7f3fff640 1 -- 172.21.15.70:0/1488944291 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f40cd750 con 0x7fc7f40ce510 2023-10-08T05:55:36.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.403+0000 7fc7f3fff640 1 --2- 172.21.15.70:0/1488944291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40ce510 0x7fc7f415c820 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40b54b0 tx=0x7fc7ec02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4de015be2f6c9d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.403+0000 7fc7f27fc640 1 -- 172.21.15.70:0/1488944291 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec02c9f0 con 0x7fc7f40ce510 2023-10-08T05:55:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.403+0000 7fc7f27fc640 1 -- 172.21.15.70:0/1488944291 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec02cb50 con 0x7fc7f40ce510 2023-10-08T05:55:36.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.403+0000 7fc7f27fc640 1 -- 172.21.15.70:0/1488944291 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0343f0 con 0x7fc7f40ce510 2023-10-08T05:55:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.404+0000 7fc7f8eec640 1 -- 172.21.15.70:0/1488944291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40ce510 msgr2=0x7fc7f415c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.404+0000 7fc7f8eec640 1 --2- 172.21.15.70:0/1488944291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40ce510 0x7fc7f415c820 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40b54b0 tx=0x7fc7ec02be50 comp rx=0 tx=0).stop 2023-10-08T05:55:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.404+0000 7fc7f8eec640 1 -- 172.21.15.70:0/1488944291 shutdown_connections 2023-10-08T05:55:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.404+0000 7fc7f8eec640 1 --2- 172.21.15.70:0/1488944291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40ce510 0x7fc7f415c820 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.404+0000 7fc7f8eec640 1 --2- 172.21.15.70:0/1488944291 >> v2:172.21.15.181:3300/0 conn(0x7fc7f40cf2b0 0x7fc7f40d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.404+0000 7fc7f8eec640 1 -- 172.21.15.70:0/1488944291 >> 172.21.15.70:0/1488944291 conn(0x7fc7f409bc40 msgr2=0x7fc7f40d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.404+0000 7fc7f8eec640 1 -- 172.21.15.70:0/1488944291 shutdown_connections 2023-10-08T05:55:36.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.404+0000 7fc7f8eec640 1 -- 172.21.15.70:0/1488944291 wait complete. 2023-10-08T05:55:36.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.407+0000 7fc7f8eec640 1 Processor -- start 2023-10-08T05:55:36.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.407+0000 7fc7f8eec640 1 -- start start 2023-10-08T05:55:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.408+0000 7fc7f8eec640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7f40ce510 0x7fc7f40a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.408+0000 7fc7f8eec640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7f40cf2b0 0x7fc7f409e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.408+0000 7fc7f8eec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40cf690 0x7fc7f415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.408+0000 7fc7f8eec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f409d5b0 con 0x7fc7f40cf690 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.408+0000 7fc7f8eec640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40a0560 con 0x7fc7f40cf2b0 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.408+0000 7fc7f8eec640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f409c930 con 0x7fc7f40ce510 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.408+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40cf690 0x7fc7f415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.408+0000 7fc7f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40cf690 0x7fc7f415baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39512/0 (socket says 172.21.15.70:39512) 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.408+0000 7fc7f3fff640 1 -- 172.21.15.70:0/3245183883 learned_addr learned my addr 172.21.15.70:0/3245183883 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.409+0000 7fc7f2ffd640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7f40cf2b0 0x7fc7f409e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.409+0000 7fc7f37fe640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7f40ce510 0x7fc7f40a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.409+0000 7fc7f3fff640 1 -- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7f40ce510 msgr2=0x7fc7f40a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.409+0000 7fc7f3fff640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7f40ce510 0x7fc7f40a0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.409+0000 7fc7f3fff640 1 -- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7f40cf2b0 msgr2=0x7fc7f409e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:36.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.409+0000 7fc7f3fff640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7f40cf2b0 0x7fc7f409e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.409+0000 7fc7f3fff640 1 -- 172.21.15.70:0/3245183883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec009b30 con 0x7fc7f40cf690 2023-10-08T05:55:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.409+0000 7fc7f2ffd640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7f40cf2b0 0x7fc7f409e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:55:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.409+0000 7fc7f3fff640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40cf690 0x7fc7f415baa0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009f40 tx=0x7fc7ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.409+0000 7fc7dbfff640 1 -- 172.21.15.70:0/3245183883 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec009510 con 0x7fc7f40cf690 2023-10-08T05:55:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.410+0000 7fc7dbfff640 1 -- 172.21.15.70:0/3245183883 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec03e450 con 0x7fc7f40cf690 2023-10-08T05:55:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.410+0000 7fc7dbfff640 1 -- 172.21.15.70:0/3245183883 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec03e7e0 con 0x7fc7f40cf690 2023-10-08T05:55:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.410+0000 7fc7f8eec640 1 -- 172.21.15.70:0/3245183883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f409cc10 con 0x7fc7f40cf690 2023-10-08T05:55:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.410+0000 7fc7f8eec640 1 -- 172.21.15.70:0/3245183883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f409d1d0 con 0x7fc7f40cf690 2023-10-08T05:55:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.411+0000 7fc7f8eec640 1 -- 172.21.15.70:0/3245183883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f409d6f0 con 0x7fc7f40cf690 2023-10-08T05:55:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.412+0000 7fc7dbfff640 1 -- 172.21.15.70:0/3245183883 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 57) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fc7ec03e940 con 0x7fc7f40cf690 2023-10-08T05:55:36.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.413+0000 7fc7dbfff640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc7e006afe0 0x7fc7e006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.413+0000 7fc7dbfff640 1 -- 172.21.15.70:0/3245183883 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc7ec0ac400 con 0x7fc7f40cf690 2023-10-08T05:55:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.413+0000 7fc7f37fe640 1 -- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc7e006afe0 msgr2=0x7fc7e006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3142216408 2023-10-08T05:55:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.413+0000 7fc7f37fe640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc7e006afe0 0x7fc7e006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.417+0000 7fc7dbfff640 1 -- 172.21.15.70:0/3245183883 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7f409d6f0 con 0x7fc7f40cf690 2023-10-08T05:55:36.422 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:36.421+0000 7fd5eb02e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:55:36.463 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:36.463+0000 7fe6cf634040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:55:36.474 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:36.473+0000 7fd5eb02e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:55:36.518 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:36.518+0000 7fe6cf634040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:55:36.525 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:36.525+0000 7fd5eb02e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:55:36.570 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:36.570+0000 7fe6cf634040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:55:36.586 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:36.585+0000 7fd5eb02e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:55:36.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.607+0000 7fc7f8eec640 1 -- 172.21.15.70:0/3245183883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fc7f409d6f0 con 0x7fc7f40cf690 2023-10-08T05:55:36.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.614+0000 7fc7f37fe640 1 -- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc7e006afe0 msgr2=0x7fc7e006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3142216408 2023-10-08T05:55:36.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.614+0000 7fc7f37fe640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc7e006afe0 0x7fc7e006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:55:36.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.614+0000 7fc7dbfff640 1 -- 172.21.15.70:0/3245183883 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc7f409d6f0 con 0x7fc7f40cf690 2023-10-08T05:55:36.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.616+0000 7fc7f8eec640 1 -- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc7e006afe0 msgr2=0x7fc7e006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:36.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.616+0000 7fc7f8eec640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc7e006afe0 0x7fc7e006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.617+0000 7fc7f8eec640 1 -- 172.21.15.70:0/3245183883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40cf690 msgr2=0x7fc7f415baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.617+0000 7fc7f8eec640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40cf690 0x7fc7f415baa0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009f40 tx=0x7fc7ec004a40 comp rx=0 tx=0).stop 2023-10-08T05:55:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.617+0000 7fc7f8eec640 1 -- 172.21.15.70:0/3245183883 shutdown_connections 2023-10-08T05:55:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.617+0000 7fc7f8eec640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7fc7e006afe0 0x7fc7e006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.617+0000 7fc7f8eec640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7f40cf690 0x7fc7f415baa0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.617+0000 7fc7f8eec640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc7f40cf2b0 0x7fc7f409e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.617+0000 7fc7f8eec640 1 --2- 172.21.15.70:0/3245183883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc7f40ce510 0x7fc7f40a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.617+0000 7fc7f8eec640 1 -- 172.21.15.70:0/3245183883 >> 172.21.15.70:0/3245183883 conn(0x7fc7f409bc40 msgr2=0x7fc7f40d25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.617+0000 7fc7f8eec640 1 -- 172.21.15.70:0/3245183883 shutdown_connections 2023-10-08T05:55:36.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.617+0000 7fc7f8eec640 1 -- 172.21.15.70:0/3245183883 wait complete. 2023-10-08T05:55:36.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:36.634 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T05:55:36.635 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:36.634+0000 7fe6cf634040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:55:36.636 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:36.634+0000 7fd5eb02e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:55:36.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-08T05:55:36.687 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:36.686+0000 7fe6cf634040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:55:36.746 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:36.745+0000 7fd5eb02e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:55:36.803 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:36.803+0000 7fe6cf634040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:55:36.809 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:36.808+0000 7fd5eb02e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:55:36.858 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:36.857+0000 7fd5eb02e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:55:36.869 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:36.868+0000 7fe6cf634040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:55:36.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.884+0000 7f0101ae5640 1 Processor -- start 2023-10-08T05:55:36.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.884+0000 7f0101ae5640 1 -- start start 2023-10-08T05:55:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.886+0000 7f0101ae5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00fc15cf80 0x7f00fc0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.886+0000 7f0101ae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc0b4b40 0x7f00fc0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.886+0000 7f0101ae5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc0b54a0 con 0x7f00fc15cba0 2023-10-08T05:55:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.886+0000 7f0101ae5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc0b4ff0 con 0x7f00fc15cf80 2023-10-08T05:55:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.886+0000 7f0101ae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0b8bb0 con 0x7f00fc0b4b40 2023-10-08T05:55:36.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.887+0000 7f00fbfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc0b4b40 0x7f00fc0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:36.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.887+0000 7f00fbfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc0b4b40 0x7f00fc0d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:52816/0 (socket says 172.21.15.70:52816) 2023-10-08T05:55:36.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.887+0000 7f00fbfff640 1 -- 172.21.15.70:0/4066099507 learned_addr learned my addr 172.21.15.70:0/4066099507 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:36.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.887+0000 7f00f3fff640 1 --2- 172.21.15.70:0/4066099507 >> v2:172.21.15.181:3300/0 conn(0x7f00fc15cf80 0x7f00fc0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:36.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.887+0000 7f00faffd640 1 -- 172.21.15.70:0/4066099507 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284689057 0 0) 0x7f00ec001110 con 0x7f00fc15cba0 2023-10-08T05:55:36.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.887+0000 7f00faffd640 1 -- 172.21.15.70:0/4066099507 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e8003520 con 0x7f00fc15cba0 2023-10-08T05:55:36.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.888+0000 7f00fbfff640 1 -- 172.21.15.70:0/4066099507 >> v2:172.21.15.181:3300/0 conn(0x7f00fc15cf80 msgr2=0x7f00fc0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:36.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.888+0000 7f00fbfff640 1 --2- 172.21.15.70:0/4066099507 >> v2:172.21.15.181:3300/0 conn(0x7f00fc15cf80 0x7f00fc0b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.888+0000 7f00fbfff640 1 -- 172.21.15.70:0/4066099507 >> v1:172.21.15.181:6789/0 conn(0x7f00fc15cba0 legacy=0x7f00fc0b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:36.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.888+0000 7f00fbfff640 1 -- 172.21.15.70:0/4066099507 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0b7130 con 0x7f00fc0b4b40 2023-10-08T05:55:36.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.890+0000 7f00fbfff640 1 --2- 172.21.15.70:0/4066099507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc0b4b40 0x7f00fc0d0ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009e80 tx=0x7f00f402bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e32a17fd34752148 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:36.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.890+0000 7f00faffd640 1 -- 172.21.15.70:0/4066099507 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f402caf0 con 0x7f00fc0b4b40 2023-10-08T05:55:36.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.890+0000 7f00faffd640 1 -- 172.21.15.70:0/4066099507 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f402cc50 con 0x7f00fc0b4b40 2023-10-08T05:55:36.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.891+0000 7f00faffd640 1 -- 172.21.15.70:0/4066099507 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f40343f0 con 0x7f00fc0b4b40 2023-10-08T05:55:36.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.891+0000 7f0101ae5640 1 -- 172.21.15.70:0/4066099507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc0b4b40 msgr2=0x7f00fc0d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:36.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.891+0000 7f0101ae5640 1 --2- 172.21.15.70:0/4066099507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc0b4b40 0x7f00fc0d0ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009e80 tx=0x7f00f402bf50 comp rx=0 tx=0).stop 2023-10-08T05:55:36.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.891+0000 7f0101ae5640 1 -- 172.21.15.70:0/4066099507 shutdown_connections 2023-10-08T05:55:36.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.891+0000 7f0101ae5640 1 --2- 172.21.15.70:0/4066099507 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc0b4b40 0x7f00fc0d0ee0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.891+0000 7f0101ae5640 1 --2- 172.21.15.70:0/4066099507 >> v2:172.21.15.181:3300/0 conn(0x7f00fc15cf80 0x7f00fc0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.891+0000 7f0101ae5640 1 -- 172.21.15.70:0/4066099507 >> 172.21.15.70:0/4066099507 conn(0x7f00fc0718a0 msgr2=0x7f00fc071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:36.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.891+0000 7f0101ae5640 1 -- 172.21.15.70:0/4066099507 shutdown_connections 2023-10-08T05:55:36.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.891+0000 7f0101ae5640 1 -- 172.21.15.70:0/4066099507 wait complete. 2023-10-08T05:55:36.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.893+0000 7f0101ae5640 1 Processor -- start 2023-10-08T05:55:36.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.893+0000 7f0101ae5640 1 -- start start 2023-10-08T05:55:36.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.894+0000 7f0101ae5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc0b4b40 0x7f00fc146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.894+0000 7f0101ae5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00fc15cba0 0x7f00fc145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.894+0000 7f0101ae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc15cf80 0x7f00fc1e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.894+0000 7f0101ae5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc147040 con 0x7f00fc0b4b40 2023-10-08T05:55:36.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.894+0000 7f0101ae5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc145a60 con 0x7f00fc15cba0 2023-10-08T05:55:36.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.894+0000 7f0101ae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc145d40 con 0x7f00fc15cf80 2023-10-08T05:55:36.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.894+0000 7f00fb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc0b4b40 0x7f00fc146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.894+0000 7f00fb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc0b4b40 0x7f00fc146b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39524/0 (socket says 172.21.15.70:39524) 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.894+0000 7f00fb7fe640 1 -- 172.21.15.70:0/1694518176 learned_addr learned my addr 172.21.15.70:0/1694518176 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00f3fff640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00fc15cba0 0x7f00fc145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00fbfff640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc15cf80 0x7f00fc1e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00fb7fe640 1 -- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc15cf80 msgr2=0x7f00fc1e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00fb7fe640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00fc15cf80 0x7f00fc1e78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00fb7fe640 1 -- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00fc15cba0 msgr2=0x7f00fc145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00fb7fe640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00fc15cba0 0x7f00fc145210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00fb7fe640 1 -- 172.21.15.70:0/1694518176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4009b30 con 0x7f00fc0b4b40 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00fb7fe640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc0b4b40 0x7f00fc146b00 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00c730 tx=0x7f00ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:36.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0052d0 con 0x7f00fc0b4b40 2023-10-08T05:55:36.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec005450 con 0x7f00fc0b4b40 2023-10-08T05:55:36.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec00a880 con 0x7f00fc0b4b40 2023-10-08T05:55:36.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.895+0000 7f0101ae5640 1 -- 172.21.15.70:0/1694518176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc146020 con 0x7f00fc0b4b40 2023-10-08T05:55:36.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.896+0000 7f0101ae5640 1 -- 172.21.15.70:0/1694518176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc1465e0 con 0x7f00fc0b4b40 2023-10-08T05:55:36.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.897+0000 7f0101ae5640 1 -- 172.21.15.70:0/1694518176 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc04e160 con 0x7f00fc0b4b40 2023-10-08T05:55:36.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.898+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 57) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f00ec00a9e0 con 0x7f00fc0b4b40 2023-10-08T05:55:36.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.898+0000 7f00f8ff9640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 0x7f00dc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:36.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.899+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f00ec088cc0 con 0x7f00fc0b4b40 2023-10-08T05:55:36.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.899+0000 7f00f3fff640 1 -- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 msgr2=0x7f00dc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3142216408 2023-10-08T05:55:36.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.899+0000 7f00f3fff640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 0x7f00dc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:55:36.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:36.903+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f00ec08f080 con 0x7f00fc0b4b40 2023-10-08T05:55:36.918 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:36.915+0000 7fd5eb02e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:55:36.923 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:36.923+0000 7fe6cf634040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:55:36.966 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:36.965+0000 7fd5eb02e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:55:36.980 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:36.980+0000 7fe6cf634040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:55:37.031 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:37.030+0000 7fe6cf634040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:55:37.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:37.100+0000 7f00f3fff640 1 -- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 msgr2=0x7f00dc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3142216408 2023-10-08T05:55:37.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:37.100+0000 7f00f3fff640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 0x7f00dc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:55:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:37.115+0000 7f0101ae5640 1 -- 172.21.15.70:0/1694518176 --> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f00fc0b76f0 con 0x7f00dc06afe0 2023-10-08T05:55:37.347 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:37.346+0000 7fd5eb02e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:55:37.414 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:37.413+0000 7fe6cf634040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:55:37.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:37.501+0000 7f00f3fff640 1 -- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 msgr2=0x7f00dc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3142216408 2023-10-08T05:55:37.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:37.501+0000 7f00f3fff640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 0x7f00dc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:55:37.505 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:37.504+0000 7fd5eb02e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:55:37.572 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:37.571+0000 7fe6cf634040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:55:37.613 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:37.612+0000 7fd5eb02e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:55:37.665 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:37.664+0000 7fd5eb02e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:55:37.681 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:37.681+0000 7fe6cf634040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:55:37.731 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:37.731+0000 7fe6cf634040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:55:37.771 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:37.770+0000 7fd5eb02e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:55:37.838 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:37.837+0000 7fe6cf634040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:55:37.877 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:37.876+0000 7fd5eb02e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:55:37.945 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:37.945+0000 7fe6cf634040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:55:38.033 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:38.031+0000 7fd5eb02e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:55:38.081 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:38.080+0000 7fd5eb02e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:55:38.101 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:38.101+0000 7fe6cf634040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:55:38.150 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:38.150+0000 7fe6cf634040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:55:38.200 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:38.198+0000 7fd5eb02e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:55:38.270 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:38.269+0000 7fe6cf634040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:55:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:38.302+0000 7f00f3fff640 1 -- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 msgr2=0x7f00dc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3142216408 2023-10-08T05:55:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:38.303+0000 7f00f3fff640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 0x7f00dc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:55:39.008 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.007+0000 7fd5eb02e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:55:39.094 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:39.094+0000 7fe6cf634040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:55:39.299 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.298+0000 7fd5eb02e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:55:39.356 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.355+0000 7fd5eb02e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:55:39.389 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:39.389+0000 7fe6cf634040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:55:39.447 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:39.446+0000 7fe6cf634040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:55:39.470 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.469+0000 7fd5eb02e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:55:39.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:39.494+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 58) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f00ec053ed0 con 0x7f00fc0b4b40 2023-10-08T05:55:39.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:39.494+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 msgr2=0x7f00dc06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:55:39.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:39.495+0000 7f00f8ff9640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/3142216408,v1:172.21.15.181:6833/3142216408] conn(0x7f00dc06afe0 0x7f00dc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:39.575 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:55:39.575+0000 7fe6cf634040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:55:39.642 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.641+0000 7fd5693bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:39.642 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.641+0000 7fd5693bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:39.642 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.641+0000 7fd5693bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:39.643 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.641+0000 7fd5693bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:39.643 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.641+0000 7fd5693bc640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:39.643 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.641+0000 7fd567378640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:39.643 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.641+0000 7fd567378640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:39.643 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.641+0000 7fd567378640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:39.643 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.641+0000 7fd567378640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:39.643 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:55:39.641+0000 7fd567378640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:55:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.504+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 59) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f00ec057a20 con 0x7f00fc0b4b40 2023-10-08T05:55:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.505+0000 7f00f8ff9640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f00dc0355a0 0x7f00dc035980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.505+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 --> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f00dc035ec0 con 0x7f00dc0355a0 2023-10-08T05:55:40.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.508+0000 7f00f3fff640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f00dc0355a0 0x7f00dc035980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:40.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.517+0000 7f00f3fff640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f00dc0355a0 0x7f00dc035980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0d0f50 tx=0x7f00e4009190 comp rx=0 tx=0).ready entity=mgr.5918 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:40.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:55:40.597 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:55:40.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.644+0000 7f00f8ff9640 1 -- 172.21.15.70:0/1694518176 <== mgr.5918 v2:172.21.15.181:6832/1786188476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f00dc035ec0 con 0x7f00dc0355a0 2023-10-08T05:55:40.645 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1 2023-10-08T05:55:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.647+0000 7f0101ae5640 1 -- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f00dc0355a0 msgr2=0x7f00dc035980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.647+0000 7f0101ae5640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f00dc0355a0 0x7f00dc035980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f00fc0d0f50 tx=0x7f00e4009190 comp rx=0 tx=0).stop 2023-10-08T05:55:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.647+0000 7f0101ae5640 1 -- 172.21.15.70:0/1694518176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc0b4b40 msgr2=0x7f00fc146b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.647+0000 7f0101ae5640 1 --2- 172.21.15.70:0/1694518176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00fc0b4b40 0x7f00fc146b00 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00c730 tx=0x7f00ec00cbf0 comp rx=0 tx=0).stop 2023-10-08T05:55:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.647+0000 7f00fbfff640 1 -- 172.21.15.70:0/1694518176 reap_dead start 2023-10-08T05:55:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.647+0000 7f0101ae5640 1 -- 172.21.15.70:0/1694518176 shutdown_connections 2023-10-08T05:55:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.647+0000 7f0101ae5640 1 -- 172.21.15.70:0/1694518176 >> 172.21.15.70:0/1694518176 conn(0x7f00fc0718a0 msgr2=0x7f00fc159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:40.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.647+0000 7f0101ae5640 1 -- 172.21.15.70:0/1694518176 shutdown_connections 2023-10-08T05:55:40.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.648+0000 7f0101ae5640 1 -- 172.21.15.70:0/1694518176 wait complete. 2023-10-08T05:55:40.667 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T05:55:40.667 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-08-05_56_00 in ~20s... 2023-10-08T05:55:40.668 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2023-10-08T05:55:40.668 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-08T05:55:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.863+0000 7f782bfff640 1 Processor -- start 2023-10-08T05:55:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.863+0000 7f782bfff640 1 -- start start 2023-10-08T05:55:40.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.865+0000 7f782bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c1599d0 0x7f782c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:40.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.865+0000 7f782bfff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c159db0 con 0x7f782c0b3c50 2023-10-08T05:55:40.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.865+0000 7f782bfff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c071fe0 con 0x7f782c0cb5e0 2023-10-08T05:55:40.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.865+0000 7f782bfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0b44c0 con 0x7f782c1599d0 2023-10-08T05:55:40.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.865+0000 7f7823fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f782c0b3c50 0x7f782c0caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52210/0 (socket says 172.21.15.70:52210) 2023-10-08T05:55:40.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.865+0000 7f7823fff640 1 -- 172.21.15.70:0/861818386 learned_addr learned my addr 172.21.15.70:0/861818386 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:55:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.865+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341296084 0 0) 0x7f7814001110 con 0x7f782c0b3c50 2023-10-08T05:55:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.865+0000 7f782affd640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c1599d0 0x7f782c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7818003520 con 0x7f782c0b3c50 2023-10-08T05:55:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3015022174 0 0) 0x7f7818003520 con 0x7f782c0b3c50 2023-10-08T05:55:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c1599d0 msgr2=0x7f782c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c1599d0 0x7f782c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 >> v1:172.21.15.181:6789/0 conn(0x7f782c0cb5e0 legacy=0x7f782c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c0713d0 con 0x7f782c0b3c50 2023-10-08T05:55:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7814001800 con 0x7f782c0b3c50 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782affd640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c1599d0 0x7f782c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 >> v1:172.21.15.70:6789/0 conn(0x7f782c0b3c50 legacy=0x7f782c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c1599d0 0x7f78180039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 0x7f78180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7818004d00 0x7f7818005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0b44c0 con 0x7f7818003fe0 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781800bb40 con 0x7f7818004d00 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781800bd00 con 0x7f782c1599d0 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.866+0000 7f7823fff640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 0x7f78180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.867+0000 7f782affd640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c1599d0 0x7f78180039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.867+0000 7f7823fff640 1 -- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c1599d0 msgr2=0x7f78180039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:40.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.867+0000 7f7823fff640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c1599d0 0x7f78180039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:40.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.867+0000 7f7823fff640 1 -- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7818004d00 msgr2=0x7f7818005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:55:40.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.867+0000 7f782b7fe640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7818004d00 0x7f7818005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:40.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.867+0000 7f7823fff640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7818004d00 0x7f7818005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:40.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.867+0000 7f7823fff640 1 -- 172.21.15.70:0/861818386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7818003520 con 0x7f7818003fe0 2023-10-08T05:55:40.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.868+0000 7f7823fff640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 0x7f78180047c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7814007fc0 tx=0x7f78180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695f71e24ce1114 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:40.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.868+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7814005ad0 con 0x7f7818003fe0 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.868+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7814005c50 con 0x7f7818003fe0 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.868+0000 7f782a7fc640 1 -- 172.21.15.70:0/861818386 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781402c7e0 con 0x7f7818003fe0 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.869+0000 7f782bfff640 1 -- 172.21.15.70:0/861818386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 msgr2=0x7f78180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.869+0000 7f782bfff640 1 --2- 172.21.15.70:0/861818386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 0x7f78180047c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7814007fc0 tx=0x7f78180057f0 comp rx=0 tx=0).stop 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.869+0000 7f782b7fe640 1 -- 172.21.15.70:0/861818386 reap_dead start 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.869+0000 7f782bfff640 1 -- 172.21.15.70:0/861818386 shutdown_connections 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.869+0000 7f782bfff640 1 -- 172.21.15.70:0/861818386 >> 172.21.15.70:0/861818386 conn(0x7f782c06c050 msgr2=0x7f782c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.869+0000 7f782bfff640 1 -- 172.21.15.70:0/861818386 shutdown_connections 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.869+0000 7f782bfff640 1 -- 172.21.15.70:0/861818386 wait complete. 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.871+0000 7f782bfff640 1 Processor -- start 2023-10-08T05:55:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.871+0000 7f782bfff640 1 -- start start 2023-10-08T05:55:40.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.872+0000 7f782bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 0x7f782c14cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:40.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.872+0000 7f782bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f782c0b3420 0x7f782c0c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:40.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.872+0000 7f782bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c168000 0x7f782c168490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:40.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.872+0000 7f782bfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c1689d0 con 0x7f7818003fe0 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.872+0000 7f782bfff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0c4bb0 con 0x7f782c0b3420 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.872+0000 7f782bfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c14d240 con 0x7f782c168000 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.872+0000 7f782affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 0x7f782c14cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.872+0000 7f782affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 0x7f782c14cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39548/0 (socket says 172.21.15.70:39548) 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.872+0000 7f782affd640 1 -- 172.21.15.70:0/3063784039 learned_addr learned my addr 172.21.15.70:0/3063784039 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f7823fff640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f782c0b3420 0x7f782c0c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f782affd640 1 -- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c168000 msgr2=0x7f782c168490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f782affd640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c168000 0x7f782c168490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f782affd640 1 -- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f782c0b3420 msgr2=0x7f782c0c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f782affd640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f782c0b3420 0x7f782c0c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:40.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f782affd640 1 -- 172.21.15.70:0/3063784039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c0c3b10 con 0x7f7818003fe0 2023-10-08T05:55:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f782affd640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 0x7f782c14cd00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f781c00f510 tx=0x7f781c00f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f7828ff9640 1 -- 172.21.15.70:0/3063784039 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c014890 con 0x7f7818003fe0 2023-10-08T05:55:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f7828ff9640 1 -- 172.21.15.70:0/3063784039 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c00c030 con 0x7f7818003fe0 2023-10-08T05:55:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f782bfff640 1 -- 172.21.15.70:0/3063784039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c0c3df0 con 0x7f7818003fe0 2023-10-08T05:55:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.873+0000 7f7828ff9640 1 -- 172.21.15.70:0/3063784039 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c014c30 con 0x7f7818003fe0 2023-10-08T05:55:40.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.874+0000 7f782bfff640 1 -- 172.21.15.70:0/3063784039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c0c43b0 con 0x7f7818003fe0 2023-10-08T05:55:40.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.875+0000 7f782bfff640 1 -- 172.21.15.70:0/3063784039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782c074150 con 0x7f7818003fe0 2023-10-08T05:55:40.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.875+0000 7f7828ff9640 1 -- 172.21.15.70:0/3063784039 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 59) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f781c0133e0 con 0x7f7818003fe0 2023-10-08T05:55:40.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.876+0000 7f7828ff9640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f780c06ad40 0x7f780c06d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:55:40.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.876+0000 7f7828ff9640 1 -- 172.21.15.70:0/3063784039 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f781c089930 con 0x7f7818003fe0 2023-10-08T05:55:40.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.877+0000 7f7823fff640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f780c06ad40 0x7f780c06d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:55:40.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.878+0000 7f7823fff640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f780c06ad40 0x7f780c06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7814034000 tx=0x7f7814009030 comp rx=0 tx=0).ready entity=mgr.5918 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:55:40.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:40.881+0000 7f7828ff9640 1 -- 172.21.15.70:0/3063784039 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f781c054980 con 0x7f7818003fe0 2023-10-08T05:55:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.083+0000 7f782bfff640 1 -- 172.21.15.70:0/3063784039 --> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f782c071990 con 0x7f780c06ad40 2023-10-08T05:55:41.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.101+0000 7f7828ff9640 1 -- 172.21.15.70:0/3063784039 <== mgr.5918 v2:172.21.15.181:6832/1786188476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f7814005870 con 0x7f780c06ad40 2023-10-08T05:55:41.101 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:55:41.101 INFO:teuthology.orchestra.run.smithi070.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 -- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f780c06ad40 msgr2=0x7f780c06d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f780c06ad40 0x7f780c06d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7814034000 tx=0x7f7814009030 comp rx=0 tx=0).stop 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 -- 172.21.15.70:0/3063784039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 msgr2=0x7f782c14cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 0x7f782c14cd00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f781c00f510 tx=0x7f781c00f9d0 comp rx=0 tx=0).stop 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 -- 172.21.15.70:0/3063784039 shutdown_connections 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f780c06ad40 0x7f780c06d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f782c168000 0x7f782c168490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f782c0b3420 0x7f782c0c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 --2- 172.21.15.70:0/3063784039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7818003fe0 0x7f782c14cd00 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 -- 172.21.15.70:0/3063784039 >> 172.21.15.70:0/3063784039 conn(0x7f782c06c050 msgr2=0x7f782c15bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:55:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 -- 172.21.15.70:0/3063784039 shutdown_connections 2023-10-08T05:55:41.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:55:41.103+0000 7f782bfff640 1 -- 172.21.15.70:0/3063784039 wait complete. 2023-10-08T05:55:41.121 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{}]} 2023-10-08T05:55:41.121 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{}]} 2023-10-08T05:55:41.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:41.122 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:41.170 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:41.170 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:41.170 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:42.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:42.171 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:42.233 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:42.234 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:42.234 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:43.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:43.235 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:43.298 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:43.298 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:43.299 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:44.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:44.299 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:44.362 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:44.363 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:44.363 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:45.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:45.364 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:45.426 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:45.426 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:45.426 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:46.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:46.427 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:46.490 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:46.503 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:46.503 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:47.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:47.504 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:47.569 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:47.569 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:47.569 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:48.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:48.571 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:48.635 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:48.635 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:48.635 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:49.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:49.636 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:49.700 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:49.700 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:49.700 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:50.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:50.701 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:50.765 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:50.765 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:50.765 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:51.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:51.767 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:51.829 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:51.829 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:51.829 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:52.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:52.831 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:52.893 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:52.893 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:52.893 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:53.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:53.894 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:53.959 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:53.959 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:53.959 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:54.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:54.961 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:55.024 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:55.024 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:55.024 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:56.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:56.025 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:56.088 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:56.088 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:56.088 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:57.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:57.090 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:57.155 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:57.156 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:57.156 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:58.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:58.158 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:58.223 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:58.223 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:58.223 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:55:59.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:55:59.224 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:55:59.286 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:55:59.286 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:55:59.286 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:56:00.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:00.287 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:56:00.350 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T05:56:00.350 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T05:56:00.350 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:56:00.670 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:00.670+0000 7fd559b5d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:00.670 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:00.670+0000 7fd559b5d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:00.670 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:00.670+0000 7fd559b5d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:00.670 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:00.670+0000 7fd559b5d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:00.670 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:00.670+0000 7fd559b5d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:01.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:01.351 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:56:01.412 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_56_00_UTC 2023-10-08T05:56:01.413 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-05_56_00_UTC'} 2023-10-08T05:56:01.413 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-08-05_56_00_UTC'} 2023-10-08T05:56:01.413 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T05:56:01.413 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-08-05_56_00_UTC'] 2023-10-08T05:56:01.413 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696744540.6679163, now = 1696744561.4136276, timo = 20 2023-10-08T05:56:02.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-08T05:56:02.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.641+0000 7fea61939640 1 Processor -- start 2023-10-08T05:56:02.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.642+0000 7fea61939640 1 -- start start 2023-10-08T05:56:02.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.643+0000 7fea61939640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c06c3a0 0x7fea5c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:02.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.643+0000 7fea61939640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea5c15aa00 0x7fea5c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:02.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.643+0000 7fea61939640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5c15b310 con 0x7fea5c06db60 2023-10-08T05:56:02.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.643+0000 7fea61939640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5c0ca230 con 0x7fea5c15aa00 2023-10-08T05:56:02.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.643+0000 7fea61939640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c06d290 con 0x7fea5c06c3a0 2023-10-08T05:56:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.644+0000 7fea5b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea5c15aa00 0x7fea5c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.644+0000 7fea5b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea5c15aa00 0x7fea5c0b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46466/0 (socket says 172.21.15.70:46466) 2023-10-08T05:56:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.644+0000 7fea5b7fe640 1 -- 172.21.15.70:0/3007950922 learned_addr learned my addr 172.21.15.70:0/3007950922 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.644+0000 7fea5affd640 1 --2- 172.21.15.70:0/3007950922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c06c3a0 0x7fea5c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.644+0000 7fea5b7fe640 1 -- 172.21.15.70:0/3007950922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c06c3a0 msgr2=0x7fea5c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.644+0000 7fea5b7fe640 1 --2- 172.21.15.70:0/3007950922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c06c3a0 0x7fea5c0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.644+0000 7fea5b7fe640 1 -- 172.21.15.70:0/3007950922 >> v1:172.21.15.181:6789/0 conn(0x7fea5c06db60 legacy=0x7fea5c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.644+0000 7fea5b7fe640 1 -- 172.21.15.70:0/3007950922 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c0c9860 con 0x7fea5c15aa00 2023-10-08T05:56:02.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.645+0000 7fea5b7fe640 1 --2- 172.21.15.70:0/3007950922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c15aa00 0x7fea5c0b2a90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fea54009870 tx=0x7fea5402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fcac38312c7421 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:02.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.646+0000 7fea59ffb640 1 -- 172.21.15.70:0/3007950922 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54030840 con 0x7fea5c15aa00 2023-10-08T05:56:02.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.646+0000 7fea59ffb640 1 -- 172.21.15.70:0/3007950922 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea540309a0 con 0x7fea5c15aa00 2023-10-08T05:56:02.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.646+0000 7fea59ffb640 1 -- 172.21.15.70:0/3007950922 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54030d30 con 0x7fea5c15aa00 2023-10-08T05:56:02.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.646+0000 7fea61939640 1 -- 172.21.15.70:0/3007950922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c15aa00 msgr2=0x7fea5c0b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:02.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.646+0000 7fea61939640 1 --2- 172.21.15.70:0/3007950922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c15aa00 0x7fea5c0b2a90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fea54009870 tx=0x7fea5402b980 comp rx=0 tx=0).stop 2023-10-08T05:56:02.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.647+0000 7fea61939640 1 -- 172.21.15.70:0/3007950922 shutdown_connections 2023-10-08T05:56:02.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.647+0000 7fea61939640 1 --2- 172.21.15.70:0/3007950922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c15aa00 0x7fea5c0b2a90 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:02.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.647+0000 7fea61939640 1 --2- 172.21.15.70:0/3007950922 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c06c3a0 0x7fea5c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:02.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.647+0000 7fea61939640 1 -- 172.21.15.70:0/3007950922 >> 172.21.15.70:0/3007950922 conn(0x7fea5c06a5d0 msgr2=0x7fea5c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:02.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.647+0000 7fea61939640 1 -- 172.21.15.70:0/3007950922 shutdown_connections 2023-10-08T05:56:02.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.647+0000 7fea61939640 1 -- 172.21.15.70:0/3007950922 wait complete. 2023-10-08T05:56:02.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.649+0000 7fea61939640 1 Processor -- start 2023-10-08T05:56:02.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.649+0000 7fea61939640 1 -- start start 2023-10-08T05:56:02.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.650+0000 7fea61939640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5c06c3a0 0x7fea5c15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.650+0000 7fea61939640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c06db60 0x7fea5c0c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.650+0000 7fea61939640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c15aa00 0x7fea5c0c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.650+0000 7fea61939640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c0c30a0 con 0x7fea5c06db60 2023-10-08T05:56:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.650+0000 7fea61939640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c15b4b0 con 0x7fea5c06c3a0 2023-10-08T05:56:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.650+0000 7fea61939640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c1575a0 con 0x7fea5c15aa00 2023-10-08T05:56:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.650+0000 7fea5a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c06db60 0x7fea5c0c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.650+0000 7fea5a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c06db60 0x7fea5c0c2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46472/0 (socket says 172.21.15.70:46472) 2023-10-08T05:56:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.650+0000 7fea5a7fc640 1 -- 172.21.15.70:0/66626386 learned_addr learned my addr 172.21.15.70:0/66626386 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea5affd640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5c06c3a0 0x7fea5c15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea5b7fe640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c15aa00 0x7fea5c0c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea5a7fc640 1 -- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c15aa00 msgr2=0x7fea5c0c27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea5a7fc640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c15aa00 0x7fea5c0c27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea5a7fc640 1 -- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5c06c3a0 msgr2=0x7fea5c15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea5a7fc640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5c06c3a0 0x7fea5c15c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea5a7fc640 1 -- 172.21.15.70:0/66626386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea540094d0 con 0x7fea5c06db60 2023-10-08T05:56:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea5affd640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5c06c3a0 0x7fea5c15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea5b7fe640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c15aa00 0x7fea5c0c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea5a7fc640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c06db60 0x7fea5c0c2270 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fea4400c450 tx=0x7fea4400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea53fff640 1 -- 172.21.15.70:0/66626386 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea440052d0 con 0x7fea5c06db60 2023-10-08T05:56:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea53fff640 1 -- 172.21.15.70:0/66626386 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44005430 con 0x7fea5c06db60 2023-10-08T05:56:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea53fff640 1 -- 172.21.15.70:0/66626386 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44012870 con 0x7fea5c06db60 2023-10-08T05:56:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea61939640 1 -- 172.21.15.70:0/66626386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c0cdb90 con 0x7fea5c06db60 2023-10-08T05:56:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.651+0000 7fea61939640 1 -- 172.21.15.70:0/66626386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5c15b6c0 con 0x7fea5c06db60 2023-10-08T05:56:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.653+0000 7fea61939640 1 -- 172.21.15.70:0/66626386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5c04e160 con 0x7fea5c06db60 2023-10-08T05:56:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.654+0000 7fea53fff640 1 -- 172.21.15.70:0/66626386 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 62) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fea440129d0 con 0x7fea5c06db60 2023-10-08T05:56:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.654+0000 7fea53fff640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fea3c06b100 0x7fea3c06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.655+0000 7fea53fff640 1 -- 172.21.15.70:0/66626386 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fea440896b0 con 0x7fea5c06db60 2023-10-08T05:56:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.655+0000 7fea5affd640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fea3c06b100 0x7fea3c06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:02.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.656+0000 7fea5affd640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fea3c06b100 0x7fea3c06d5c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fea4c007330 tx=0x7fea4c00b040 comp rx=0 tx=0).ready entity=mgr.5918 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:02.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.659+0000 7fea53fff640 1 -- 172.21.15.70:0/66626386 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea440541c0 con 0x7fea5c06db60 2023-10-08T05:56:02.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:02.870+0000 7fea61939640 1 -- 172.21.15.70:0/66626386 --> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fea5c0c9210 con 0x7fea3c06b100 2023-10-08T05:56:03.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.050+0000 7fea53fff640 1 -- 172.21.15.70:0/66626386 <== mgr.5918 v2:172.21.15.181:6832/1786188476 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fea4c003990 con 0x7fea3c06b100 2023-10-08T05:56:03.051 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path snap_test_dir1 2023-10-08T05:56:03.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.053+0000 7fea61939640 1 -- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fea3c06b100 msgr2=0x7fea3c06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:03.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.053+0000 7fea61939640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fea3c06b100 0x7fea3c06d5c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fea4c007330 tx=0x7fea4c00b040 comp rx=0 tx=0).stop 2023-10-08T05:56:03.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.053+0000 7fea61939640 1 -- 172.21.15.70:0/66626386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c06db60 msgr2=0x7fea5c0c2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:03.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.053+0000 7fea61939640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c06db60 0x7fea5c0c2270 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fea4400c450 tx=0x7fea4400c910 comp rx=0 tx=0).stop 2023-10-08T05:56:03.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.053+0000 7fea61939640 1 -- 172.21.15.70:0/66626386 shutdown_connections 2023-10-08T05:56:03.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.053+0000 7fea61939640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fea3c06b100 0x7fea3c06d5c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:03.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.053+0000 7fea61939640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea5c15aa00 0x7fea5c0c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:03.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.053+0000 7fea61939640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea5c06db60 0x7fea5c0c2270 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:03.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.053+0000 7fea61939640 1 --2- 172.21.15.70:0/66626386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea5c06c3a0 0x7fea5c15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:03.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.053+0000 7fea61939640 1 -- 172.21.15.70:0/66626386 >> 172.21.15.70:0/66626386 conn(0x7fea5c06a5d0 msgr2=0x7fea5c070270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:03.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.054+0000 7fea61939640 1 -- 172.21.15.70:0/66626386 shutdown_connections 2023-10-08T05:56:03.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.054+0000 7fea61939640 1 -- 172.21.15.70:0/66626386 wait complete. 2023-10-08T05:56:03.073 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-08T05:56:03.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:03.073 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T05:56:03.126 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_56_00_UTC 2023-10-08T05:56:03.127 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-05_56_00_UTC 2023-10-08T05:56:03.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:03.128 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-05_56_00_UTC) 2023-10-08T05:56:03.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:03.188 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T05:56:03.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T05:56:03.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.464+0000 7f2dda980640 1 Processor -- start 2023-10-08T05:56:03.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.465+0000 7f2dda980640 1 -- start start 2023-10-08T05:56:03.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.466+0000 7f2dda980640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2dd40c8fb0 0x7f2dd40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:03.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.466+0000 7f2dda980640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dd415c630 0x7f2dd415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:03.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.466+0000 7f2dda980640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd40b0260 con 0x7f2dd40c9390 2023-10-08T05:56:03.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.466+0000 7f2dda980640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd415cfc0 con 0x7f2dd415c630 2023-10-08T05:56:03.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.466+0000 7f2dda980640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd41612d0 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.467+0000 7f2dda17f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dd415c630 0x7f2dd415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.467+0000 7f2dda17f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dd415c630 0x7f2dd415ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46482/0 (socket says 172.21.15.70:46482) 2023-10-08T05:56:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.467+0000 7f2dda17f640 1 -- 172.21.15.70:0/3119131783 learned_addr learned my addr 172.21.15.70:0/3119131783 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.467+0000 7f2dd997e640 1 --2- 172.21.15.70:0/3119131783 >> v2:172.21.15.181:3300/0 conn(0x7f2dd40c8fb0 0x7f2dd40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.467+0000 7f2dda17f640 1 -- 172.21.15.70:0/3119131783 >> v2:172.21.15.181:3300/0 conn(0x7f2dd40c8fb0 msgr2=0x7f2dd40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.467+0000 7f2dda17f640 1 --2- 172.21.15.70:0/3119131783 >> v2:172.21.15.181:3300/0 conn(0x7f2dd40c8fb0 0x7f2dd40cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.467+0000 7f2dda17f640 1 -- 172.21.15.70:0/3119131783 >> v1:172.21.15.181:6789/0 conn(0x7f2dd40c9390 legacy=0x7f2dd4160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.467+0000 7f2dda17f640 1 -- 172.21.15.70:0/3119131783 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd4161590 con 0x7f2dd415c630 2023-10-08T05:56:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.468+0000 7f2dda17f640 1 --2- 172.21.15.70:0/3119131783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd415c630 0x7f2dd415ca80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f2dd406e180 tx=0x7f2dcc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ba68be697d13c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.469+0000 7f2dd897c640 1 -- 172.21.15.70:0/3119131783 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc030860 con 0x7f2dd415c630 2023-10-08T05:56:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.469+0000 7f2dd897c640 1 -- 172.21.15.70:0/3119131783 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc0309c0 con 0x7f2dd415c630 2023-10-08T05:56:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.469+0000 7f2dd897c640 1 -- 172.21.15.70:0/3119131783 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc030d50 con 0x7f2dd415c630 2023-10-08T05:56:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.469+0000 7f2dda980640 1 -- 172.21.15.70:0/3119131783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd415c630 msgr2=0x7f2dd415ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.469+0000 7f2dda980640 1 --2- 172.21.15.70:0/3119131783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd415c630 0x7f2dd415ca80 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f2dd406e180 tx=0x7f2dcc02b9a0 comp rx=0 tx=0).stop 2023-10-08T05:56:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.469+0000 7f2dda980640 1 -- 172.21.15.70:0/3119131783 shutdown_connections 2023-10-08T05:56:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.469+0000 7f2dda980640 1 --2- 172.21.15.70:0/3119131783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd415c630 0x7f2dd415ca80 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.469+0000 7f2dda980640 1 --2- 172.21.15.70:0/3119131783 >> v2:172.21.15.181:3300/0 conn(0x7f2dd40c8fb0 0x7f2dd40cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.469+0000 7f2dda980640 1 -- 172.21.15.70:0/3119131783 >> 172.21.15.70:0/3119131783 conn(0x7f2dd406c010 msgr2=0x7f2dd406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.470+0000 7f2dda980640 1 -- 172.21.15.70:0/3119131783 shutdown_connections 2023-10-08T05:56:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.470+0000 7f2dda980640 1 -- 172.21.15.70:0/3119131783 wait complete. 2023-10-08T05:56:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.471+0000 7f2dda980640 1 Processor -- start 2023-10-08T05:56:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.471+0000 7f2dda980640 1 -- start start 2023-10-08T05:56:03.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.472+0000 7f2dda980640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd40c8fb0 0x7f2dd4159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.472+0000 7f2dda980640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd40c9390 0x7f2dd41f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.472+0000 7f2dda980640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd415c630 0x7f2dd415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.472+0000 7f2dda980640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd41f2dd0 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.472+0000 7f2dda980640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd41f30b0 con 0x7f2dd40c9390 2023-10-08T05:56:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.472+0000 7f2dda980640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd41f3390 con 0x7f2dd415c630 2023-10-08T05:56:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd997e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd40c8fb0 0x7f2dd4159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd997e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd40c8fb0 0x7f2dd4159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46486/0 (socket says 172.21.15.70:46486) 2023-10-08T05:56:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd997e640 1 -- 172.21.15.70:0/2495150796 learned_addr learned my addr 172.21.15.70:0/2495150796 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd917d640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd40c9390 0x7f2dd41f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dda17f640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd415c630 0x7f2dd415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd997e640 1 -- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd415c630 msgr2=0x7f2dd415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd997e640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd415c630 0x7f2dd415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd997e640 1 -- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd40c9390 msgr2=0x7f2dd41f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd997e640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd40c9390 0x7f2dd41f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd997e640 1 -- 172.21.15.70:0/2495150796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0094d0 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd917d640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd40c9390 0x7f2dd41f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.473+0000 7f2dd997e640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd40c8fb0 0x7f2dd4159d50 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000d800 tx=0x7f2dd000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:03.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.474+0000 7f2dca7fc640 1 -- 172.21.15.70:0/2495150796 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000b6c0 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.474+0000 7f2dca7fc640 1 -- 172.21.15.70:0/2495150796 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd000b820 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.474+0000 7f2dca7fc640 1 -- 172.21.15.70:0/2495150796 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000bbb0 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.474+0000 7f2dda980640 1 -- 172.21.15.70:0/2495150796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd40b28d0 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.474+0000 7f2dda980640 1 -- 172.21.15.70:0/2495150796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd415a290 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.475+0000 7f2dda980640 1 -- 172.21.15.70:0/2495150796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd40ca2e0 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.476+0000 7f2dca7fc640 1 -- 172.21.15.70:0/2495150796 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 62) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f2dd000bd10 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.477+0000 7f2dca7fc640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f2db806b100 0x7f2db806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:03.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.478+0000 7f2dca7fc640 1 -- 172.21.15.70:0/2495150796 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f2dd008a3c0 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.478+0000 7f2dd917d640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f2db806b100 0x7f2db806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:03.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.479+0000 7f2dd917d640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f2db806b100 0x7f2db806d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2dd415a8d0 tx=0x7f2dc4009040 comp rx=0 tx=0).ready entity=mgr.5918 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.482+0000 7f2dca7fc640 1 -- 172.21.15.70:0/2495150796 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2dd0054da0 con 0x7f2dd40c8fb0 2023-10-08T05:56:03.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:03.723+0000 7f2dda980640 1 -- 172.21.15.70:0/2495150796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f2dd4074150 con 0x7f2dd40c8fb0 2023-10-08T05:56:04.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.602+0000 7f2dca7fc640 1 -- 172.21.15.70:0/2495150796 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v63) v1 ==== 93+0+0 (secure 0 0 0) 0x7f2dd00583c0 con 0x7f2dd40c8fb0 2023-10-08T05:56:04.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 -- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f2db806b100 msgr2=0x7f2db806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:04.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f2db806b100 0x7f2db806d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2dd415a8d0 tx=0x7f2dc4009040 comp rx=0 tx=0).stop 2023-10-08T05:56:04.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 -- 172.21.15.70:0/2495150796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd40c8fb0 msgr2=0x7f2dd4159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:04.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd40c8fb0 0x7f2dd4159d50 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000d800 tx=0x7f2dd000dcc0 comp rx=0 tx=0).stop 2023-10-08T05:56:04.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 -- 172.21.15.70:0/2495150796 shutdown_connections 2023-10-08T05:56:04.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f2db806b100 0x7f2db806d5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:04.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dd415c630 0x7f2dd415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:04.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dd40c9390 0x7f2dd41f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:04.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 --2- 172.21.15.70:0/2495150796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dd40c8fb0 0x7f2dd4159d50 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:04.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 -- 172.21.15.70:0/2495150796 >> 172.21.15.70:0/2495150796 conn(0x7f2dd406c010 msgr2=0x7f2dd40717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:04.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 -- 172.21.15.70:0/2495150796 shutdown_connections 2023-10-08T05:56:04.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:04.608+0000 7f2dda980640 1 -- 172.21.15.70:0/2495150796 wait complete. 2023-10-08T05:56:04.628 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:56:04.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:56:04.746 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:56:04.838 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:56:04.838 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:56:04.838 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:04.890 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:56:04.891 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:04.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:05.134 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:05.134+0000 7f6459e2e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:56:05.139 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:05.138+0000 7f909322c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:56:05.189 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:05.189+0000 7f6459e2e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:56:05.192 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:05.191+0000 7f909322c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:56:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.231+0000 7f02dbee5640 1 Processor -- start 2023-10-08T05:56:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.231+0000 7f02dbee5640 1 -- start start 2023-10-08T05:56:05.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.233+0000 7f02dbee5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d40b1d90 0x7f02d40d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.233+0000 7f02dbee5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d40b8bf0 con 0x7f02d40cd8d0 2023-10-08T05:56:05.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.233+0000 7f02dbee5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d40b8630 con 0x7f02d40b1840 2023-10-08T05:56:05.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.233+0000 7f02dbee5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40b8910 con 0x7f02d40b1d90 2023-10-08T05:56:05.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.233+0000 7f02d9c81640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f02d40cd8d0 0x7f02d40cdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42200/0 (socket says 172.21.15.70:42200) 2023-10-08T05:56:05.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.233+0000 7f02d9c81640 1 -- 172.21.15.70:0/3238590019 learned_addr learned my addr 172.21.15.70:0/3238590019 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:05.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02da482640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d40b1d90 0x7f02d40d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416938113 0 0) 0x7f02d40b8bf0 con 0x7f02d40cd8d0 2023-10-08T05:56:05.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c0003520 con 0x7f02d40cd8d0 2023-10-08T05:56:05.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3314365896 0 0) 0x7f02c0003520 con 0x7f02d40cd8d0 2023-10-08T05:56:05.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d40b1d90 msgr2=0x7f02d40d2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:05.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d40b1d90 0x7f02d40d2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 >> v1:172.21.15.181:6789/0 conn(0x7f02d40b1840 legacy=0x7f02d40d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40d1100 con 0x7f02d40cd8d0 2023-10-08T05:56:05.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f02d0001670 con 0x7f02d40cd8d0 2023-10-08T05:56:05.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 >> v1:172.21.15.70:6789/0 conn(0x7f02d40cd8d0 legacy=0x7f02d40cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d40b1d90 0x7f02c00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c0003f20 0x7f02c00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0004d00 0x7f02c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d40b8910 con 0x7f02c0003f20 2023-10-08T05:56:05.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c000ba60 con 0x7f02c0004d00 2023-10-08T05:56:05.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.234+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c000bc20 con 0x7f02d40b1d90 2023-10-08T05:56:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.235+0000 7f02d9480640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c0003f20 0x7f02c00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.235+0000 7f02da482640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0004d00 0x7f02c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.235+0000 7f02d9480640 1 -- 172.21.15.70:0/3238590019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d40b1d90 msgr2=0x7f02c00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.235+0000 7f02d9480640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d40b1d90 0x7f02c00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.235+0000 7f02d9480640 1 -- 172.21.15.70:0/3238590019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0004d00 msgr2=0x7f02c0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.235+0000 7f02d9480640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0004d00 0x7f02c0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.235+0000 7f02d9480640 1 -- 172.21.15.70:0/3238590019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0003520 con 0x7f02c0003f20 2023-10-08T05:56:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.236+0000 7f02d9480640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c0003f20 0x7f02c00047c0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f02c4001750 tx=0x7f02c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486f6d307516b26b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.236+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c40052d0 con 0x7f02c0003f20 2023-10-08T05:56:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.236+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c402e050 con 0x7f02c0003f20 2023-10-08T05:56:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.236+0000 7f02d8c7f640 1 -- 172.21.15.70:0/3238590019 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4003830 con 0x7f02c0003f20 2023-10-08T05:56:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.237+0000 7f02dbee5640 1 -- 172.21.15.70:0/3238590019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c0003f20 msgr2=0x7f02c00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.237+0000 7f02dbee5640 1 --2- 172.21.15.70:0/3238590019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c0003f20 0x7f02c00047c0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f02c4001750 tx=0x7f02c00057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.237+0000 7f02da482640 1 -- 172.21.15.70:0/3238590019 reap_dead start 2023-10-08T05:56:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.237+0000 7f02dbee5640 1 -- 172.21.15.70:0/3238590019 shutdown_connections 2023-10-08T05:56:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.237+0000 7f02dbee5640 1 -- 172.21.15.70:0/3238590019 >> 172.21.15.70:0/3238590019 conn(0x7f02d406b6a0 msgr2=0x7f02d406bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.237+0000 7f02dbee5640 1 -- 172.21.15.70:0/3238590019 shutdown_connections 2023-10-08T05:56:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.237+0000 7f02dbee5640 1 -- 172.21.15.70:0/3238590019 wait complete. 2023-10-08T05:56:05.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.239+0000 7f02dbee5640 1 Processor -- start 2023-10-08T05:56:05.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.239+0000 7f02dbee5640 1 -- start start 2023-10-08T05:56:05.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.240+0000 7f02dbee5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0003f20 0x7f02d4158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.240+0000 7f02dbee5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d41e3740 0x7f02d41f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.240+0000 7f02dbee5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d41f6310 0x7f02d4071480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.240+0000 7f02dbee5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d41e5c80 con 0x7f02d41e3740 2023-10-08T05:56:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.240+0000 7f02dbee5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d41e5650 con 0x7f02c0003f20 2023-10-08T05:56:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.240+0000 7f02dbee5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d41591f0 con 0x7f02d41f6310 2023-10-08T05:56:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.240+0000 7f02d9480640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d41e3740 0x7f02d41f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.240+0000 7f02d9480640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d41e3740 0x7f02d41f5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40776/0 (socket says 172.21.15.70:40776) 2023-10-08T05:56:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.240+0000 7f02d9480640 1 -- 172.21.15.70:0/3592342894 learned_addr learned my addr 172.21.15.70:0/3592342894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02d9c81640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0003f20 0x7f02d4158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02da482640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d41f6310 0x7f02d4071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02d9480640 1 -- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d41f6310 msgr2=0x7f02d4071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02d9480640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d41f6310 0x7f02d4071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02d9480640 1 -- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0003f20 msgr2=0x7f02d4158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02d9480640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0003f20 0x7f02d4158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02d9480640 1 -- 172.21.15.70:0/3592342894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d41f6890 con 0x7f02d41e3740 2023-10-08T05:56:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02da482640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d41f6310 0x7f02d4071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:05.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02d9480640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d41e3740 0x7f02d41f5dd0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f02c4008fd0 tx=0x7f02c4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:05.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02ca7fc640 1 -- 172.21.15.70:0/3592342894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c40076c0 con 0x7f02d41e3740 2023-10-08T05:56:05.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02ca7fc640 1 -- 172.21.15.70:0/3592342894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c4007a30 con 0x7f02d41e3740 2023-10-08T05:56:05.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02ca7fc640 1 -- 172.21.15.70:0/3592342894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4006c00 con 0x7f02d41e3740 2023-10-08T05:56:05.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.241+0000 7f02dbee5640 1 -- 172.21.15.70:0/3592342894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d41f6b70 con 0x7f02d41e3740 2023-10-08T05:56:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.242+0000 7f02dbee5640 1 -- 172.21.15.70:0/3592342894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d41595f0 con 0x7f02d41e3740 2023-10-08T05:56:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.243+0000 7f02dbee5640 1 -- 172.21.15.70:0/3592342894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d40b54f0 con 0x7f02d41e3740 2023-10-08T05:56:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.243+0000 7f02ca7fc640 1 -- 172.21.15.70:0/3592342894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 63) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f02c40035f0 con 0x7f02d41e3740 2023-10-08T05:56:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.244+0000 7f02ca7fc640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 0x7f02b806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.244+0000 7f02ca7fc640 1 -- 172.21.15.70:0/3592342894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f02c40abf50 con 0x7f02d41e3740 2023-10-08T05:56:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.244+0000 7f02d9c81640 1 -- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 msgr2=0x7f02b806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1786188476 2023-10-08T05:56:05.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.245+0000 7f02d9c81640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 0x7f02b806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:05.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.248+0000 7f02ca7fc640 1 -- 172.21.15.70:0/3592342894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02c407a530 con 0x7f02d41e3740 2023-10-08T05:56:05.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.440+0000 7f02dbee5640 1 -- 172.21.15.70:0/3592342894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02d40ccf30 con 0x7f02d41e3740 2023-10-08T05:56:05.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.441+0000 7f02ca7fc640 1 -- 172.21.15.70:0/3592342894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+17282 (secure 0 0 0) 0x7f02c404b090 con 0x7f02d41e3740 2023-10-08T05:56:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:05.443 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":135,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:39.474029+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:55:23.150004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-08T05:55:24.358581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1556900853":"2023-10-09T05:55:39.473892+0000","172.21.15.181:6832/3142216408":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/4173673684":"2023-10-09T05:55:39.473892+0000","172.21.15.181:6833/3142216408":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/1464588158":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/791649123":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/2668737378":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/3044665941":"2023-10-09T05:55:39.473892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:05.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 -- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 msgr2=0x7f02b806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:05.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 0x7f02b806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 -- 172.21.15.70:0/3592342894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d41e3740 msgr2=0x7f02d41f5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d41e3740 0x7f02d41f5dd0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f02c4008fd0 tx=0x7f02c4001120 comp rx=0 tx=0).stop 2023-10-08T05:56:05.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 -- 172.21.15.70:0/3592342894 shutdown_connections 2023-10-08T05:56:05.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 0x7f02b806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d41f6310 0x7f02d4071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d41e3740 0x7f02d41f5dd0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 --2- 172.21.15.70:0/3592342894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0003f20 0x7f02d4158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 -- 172.21.15.70:0/3592342894 >> 172.21.15.70:0/3592342894 conn(0x7f02d406b6a0 msgr2=0x7f02d40b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 -- 172.21.15.70:0/3592342894 shutdown_connections 2023-10-08T05:56:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.444+0000 7f02dbee5640 1 -- 172.21.15.70:0/3592342894 wait complete. 2023-10-08T05:56:05.464 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:55:23.150004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:55:24.358581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:56:05.465 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:56:05.641 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:05.640+0000 7f909322c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:56:05.657 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:05.657+0000 7f6459e2e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:56:05.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.671+0000 7fa8eddb0640 1 Processor -- start 2023-10-08T05:56:05.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.671+0000 7fa8eddb0640 1 -- start start 2023-10-08T05:56:05.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.672+0000 7fa8eddb0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e81432f0 0x7fa8e80cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.672+0000 7fa8eddb0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa8e80b4240 0x7fa8e8159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.672+0000 7fa8eddb0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e80cbb30 con 0x7fa8e80cb6b0 2023-10-08T05:56:05.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.673+0000 7fa8eddb0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e81436d0 con 0x7fa8e80b4240 2023-10-08T05:56:05.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.673+0000 7fa8eddb0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e815d440 con 0x7fa8e81432f0 2023-10-08T05:56:05.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.673+0000 7fa8e6ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa8e80cb6b0 0x7fa8e80b3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42208/0 (socket says 172.21.15.70:42208) 2023-10-08T05:56:05.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.673+0000 7fa8e6ffd640 1 -- 172.21.15.70:0/1437864921 learned_addr learned my addr 172.21.15.70:0/1437864921 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:05.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.673+0000 7fa8e7fff640 1 --2- 172.21.15.70:0/1437864921 >> v2:172.21.15.181:3300/0 conn(0x7fa8e80b4240 0x7fa8e8159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.673+0000 7fa8e77fe640 1 --2- 172.21.15.70:0/1437864921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e81432f0 0x7fa8e80cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.673+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242403569 0 0) 0x7fa8e80cbb30 con 0x7fa8e80cb6b0 2023-10-08T05:56:05.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.673+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8d0003520 con 0x7fa8e80cb6b0 2023-10-08T05:56:05.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 285134423 0 0) 0x7fa8d0003520 con 0x7fa8e80cb6b0 2023-10-08T05:56:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e81432f0 msgr2=0x7fa8e80cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 --2- 172.21.15.70:0/1437864921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e81432f0 0x7fa8e80cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 >> v2:172.21.15.181:3300/0 conn(0x7fa8e80b4240 msgr2=0x7fa8e8159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 --2- 172.21.15.70:0/1437864921 >> v2:172.21.15.181:3300/0 conn(0x7fa8e80b4240 0x7fa8e8159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e815d690 con 0x7fa8e80cb6b0 2023-10-08T05:56:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa8dc001670 con 0x7fa8e80cb6b0 2023-10-08T05:56:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 >> v1:172.21.15.70:6789/0 conn(0x7fa8e80cb6b0 legacy=0x7fa8e80b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 --2- 172.21.15.70:0/1437864921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8d0003970 0x7fa8d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 --2- 172.21.15.70:0/1437864921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e81432f0 0x7fa8d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 --2- 172.21.15.70:0/1437864921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8d0004d00 0x7fa8d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e81436d0 con 0x7fa8d0003970 2023-10-08T05:56:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e815d440 con 0x7fa8d0004d00 2023-10-08T05:56:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d000b980 con 0x7fa8e81432f0 2023-10-08T05:56:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e77fe640 1 --2- 172.21.15.70:0/1437864921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8d0003970 0x7fa8d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e77fe640 1 -- 172.21.15.70:0/1437864921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e81432f0 msgr2=0x7fa8d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e77fe640 1 --2- 172.21.15.70:0/1437864921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e81432f0 0x7fa8d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e77fe640 1 -- 172.21.15.70:0/1437864921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8d0004d00 msgr2=0x7fa8d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e77fe640 1 --2- 172.21.15.70:0/1437864921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8d0004d00 0x7fa8d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.674+0000 7fa8e77fe640 1 -- 172.21.15.70:0/1437864921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0003520 con 0x7fa8d0003970 2023-10-08T05:56:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.676+0000 7fa8e77fe640 1 --2- 172.21.15.70:0/1437864921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8d0003970 0x7fa8d0003d90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009650 tx=0x7fa8d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf8eba24a8b1487 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:05.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.676+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d802e920 con 0x7fa8d0003970 2023-10-08T05:56:05.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.676+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d802ea80 con 0x7fa8d0003970 2023-10-08T05:56:05.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.676+0000 7fa8e67fc640 1 -- 172.21.15.70:0/1437864921 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d802ee10 con 0x7fa8d0003970 2023-10-08T05:56:05.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.676+0000 7fa8eddb0640 1 -- 172.21.15.70:0/1437864921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8d0003970 msgr2=0x7fa8d0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.676+0000 7fa8eddb0640 1 --2- 172.21.15.70:0/1437864921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8d0003970 0x7fa8d0003d90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009650 tx=0x7fa8d00057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.676+0000 7fa8e7fff640 1 -- 172.21.15.70:0/1437864921 reap_dead start 2023-10-08T05:56:05.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.677+0000 7fa8eddb0640 1 -- 172.21.15.70:0/1437864921 shutdown_connections 2023-10-08T05:56:05.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.677+0000 7fa8eddb0640 1 -- 172.21.15.70:0/1437864921 >> 172.21.15.70:0/1437864921 conn(0x7fa8e8069d50 msgr2=0x7fa8e806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:05.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.677+0000 7fa8eddb0640 1 -- 172.21.15.70:0/1437864921 shutdown_connections 2023-10-08T05:56:05.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.677+0000 7fa8eddb0640 1 -- 172.21.15.70:0/1437864921 wait complete. 2023-10-08T05:56:05.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.678+0000 7fa8eddb0640 1 Processor -- start 2023-10-08T05:56:05.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.679+0000 7fa8eddb0640 1 -- start start 2023-10-08T05:56:05.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.679+0000 7fa8eddb0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8d0003970 0x7fa8e81572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.679+0000 7fa8eddb0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e815aae0 0x7fa8e8140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.679+0000 7fa8eddb0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e8140dd0 0x7fa8e813bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.679+0000 7fa8eddb0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e8141280 con 0x7fa8e8140dd0 2023-10-08T05:56:05.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.679+0000 7fa8eddb0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e813c9a0 con 0x7fa8d0003970 2023-10-08T05:56:05.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8eddb0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e813e6a0 con 0x7fa8e815aae0 2023-10-08T05:56:05.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e8140dd0 0x7fa8e813bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e8140dd0 0x7fa8e813bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40794/0 (socket says 172.21.15.70:40794) 2023-10-08T05:56:05.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e7fff640 1 -- 172.21.15.70:0/2420459728 learned_addr learned my addr 172.21.15.70:0/2420459728 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:05.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e6ffd640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e815aae0 0x7fa8e8140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e77fe640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8d0003970 0x7fa8e81572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:05.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e7fff640 1 -- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e815aae0 msgr2=0x7fa8e8140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e7fff640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e815aae0 0x7fa8e8140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e7fff640 1 -- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8d0003970 msgr2=0x7fa8e81572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e7fff640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8d0003970 0x7fa8e81572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e7fff640 1 -- 172.21.15.70:0/2420459728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e813e980 con 0x7fa8e8140dd0 2023-10-08T05:56:05.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.680+0000 7fa8e6ffd640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e815aae0 0x7fa8e8140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:05.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.681+0000 7fa8e7fff640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e8140dd0 0x7fa8e813bc20 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80cc1a0 tx=0x7fa8e000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.681+0000 7fa8cffff640 1 -- 172.21.15.70:0/2420459728 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0010040 con 0x7fa8e8140dd0 2023-10-08T05:56:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.681+0000 7fa8cffff640 1 -- 172.21.15.70:0/2420459728 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e000bae0 con 0x7fa8e8140dd0 2023-10-08T05:56:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.681+0000 7fa8eddb0640 1 -- 172.21.15.70:0/2420459728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e813ec60 con 0x7fa8e8140dd0 2023-10-08T05:56:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.681+0000 7fa8cffff640 1 -- 172.21.15.70:0/2420459728 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000be70 con 0x7fa8e8140dd0 2023-10-08T05:56:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.681+0000 7fa8eddb0640 1 -- 172.21.15.70:0/2420459728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e81577e0 con 0x7fa8e8140dd0 2023-10-08T05:56:05.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.682+0000 7fa8eddb0640 1 -- 172.21.15.70:0/2420459728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e813f180 con 0x7fa8e8140dd0 2023-10-08T05:56:05.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.683+0000 7fa8cffff640 1 -- 172.21.15.70:0/2420459728 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 63) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fa8e001d400 con 0x7fa8e8140dd0 2023-10-08T05:56:05.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.684+0000 7fa8cffff640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa8d406b0e0 0x7fa8d406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:05.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.684+0000 7fa8e77fe640 1 -- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa8d406b0e0 msgr2=0x7fa8d406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1786188476 2023-10-08T05:56:05.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.684+0000 7fa8cffff640 1 -- 172.21.15.70:0/2420459728 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fa8e008a3f0 con 0x7fa8e8140dd0 2023-10-08T05:56:05.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.684+0000 7fa8e77fe640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa8d406b0e0 0x7fa8d406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:05.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.689+0000 7fa8cffff640 1 -- 172.21.15.70:0/2420459728 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8e813f180 con 0x7fa8e8140dd0 2023-10-08T05:56:05.693 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:05.692+0000 7f909322c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:56:05.713 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:05.713+0000 7f6459e2e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:56:05.744 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:05.743+0000 7f909322c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:56:05.765 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:05.764+0000 7f6459e2e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:56:05.805 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:05.804+0000 7f909322c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:56:05.826 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:05.826+0000 7f6459e2e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:56:05.854 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:05.853+0000 7f909322c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:56:05.876 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:05.875+0000 7f6459e2e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:56:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.875+0000 7fa8eddb0640 1 -- 172.21.15.70:0/2420459728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8e813f180 con 0x7fa8e8140dd0 2023-10-08T05:56:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.876+0000 7fa8cffff640 1 -- 172.21.15.70:0/2420459728 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa8e813f180 con 0x7fa8e8140dd0 2023-10-08T05:56:05.877 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:56:05.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 -- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa8d406b0e0 msgr2=0x7fa8d406d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa8d406b0e0 0x7fa8d406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 -- 172.21.15.70:0/2420459728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e8140dd0 msgr2=0x7fa8e813bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e8140dd0 0x7fa8e813bc20 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80cc1a0 tx=0x7fa8e000b480 comp rx=0 tx=0).stop 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 -- 172.21.15.70:0/2420459728 shutdown_connections 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa8d406b0e0 0x7fa8d406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e8140dd0 0x7fa8e813bc20 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa8e815aae0 0x7fa8e8140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 --2- 172.21.15.70:0/2420459728 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa8d0003970 0x7fa8e81572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 -- 172.21.15.70:0/2420459728 >> 172.21.15.70:0/2420459728 conn(0x7fa8e8069d50 msgr2=0x7fa8e8157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 -- 172.21.15.70:0/2420459728 shutdown_connections 2023-10-08T05:56:05.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:05.879+0000 7fa8eddb0640 1 -- 172.21.15.70:0/2420459728 wait complete. 2023-10-08T05:56:05.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:56:05.970 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:05.969+0000 7f909322c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:56:05.997 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:05.997+0000 7f6459e2e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:56:06.032 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:06.031+0000 7f909322c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:06.063 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:06.062+0000 7f6459e2e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:06.081 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:06.080+0000 7f909322c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:56:06.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.102+0000 7fbac620d640 1 Processor -- start 2023-10-08T05:56:06.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.102+0000 7fbac620d640 1 -- start start 2023-10-08T05:56:06.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.104+0000 7fbac620d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac0159db0 0x7fbac00cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.104+0000 7fbac620d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac0072250 con 0x7fbac00cd840 2023-10-08T05:56:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.104+0000 7fbac620d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbac015e200 con 0x7fbac0071dc0 2023-10-08T05:56:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.104+0000 7fbac620d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac015e4e0 con 0x7fbac0159db0 2023-10-08T05:56:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.104+0000 7fbac5a0c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbac00cd840 0x7fbac00b1070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42214/0 (socket says 172.21.15.70:42214) 2023-10-08T05:56:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.104+0000 7fbac5a0c640 1 -- 172.21.15.70:0/2005150227 learned_addr learned my addr 172.21.15.70:0/2005150227 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbac520b640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac0159db0 0x7fbac00cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564145540 0 0) 0x7fbac0072250 con 0x7fbac00cd840 2023-10-08T05:56:06.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaac003520 con 0x7fbac00cd840 2023-10-08T05:56:06.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3778237134 0 0) 0x7fbaac003520 con 0x7fbac00cd840 2023-10-08T05:56:06.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac0159db0 msgr2=0x7fbac00cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:06.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac0159db0 0x7fbac00cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 >> v1:172.21.15.181:6789/0 conn(0x7fbac0071dc0 legacy=0x7fbac00cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac00cb180 con 0x7fbac00cd840 2023-10-08T05:56:06.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fbab80019f0 con 0x7fbac00cd840 2023-10-08T05:56:06.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 >> v1:172.21.15.70:6789/0 conn(0x7fbac00cd840 legacy=0x7fbac00b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaac0039a0 0x7fbaac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac0159db0 0x7fbaac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac004d00 0x7fbaac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac015e4e0 con 0x7fbaac0039a0 2023-10-08T05:56:06.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac00ba30 con 0x7fbaac004d00 2023-10-08T05:56:06.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.105+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaac00bbf0 con 0x7fbac0159db0 2023-10-08T05:56:06.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.106+0000 7fbac520b640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaac0039a0 0x7fbaac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.106+0000 7fbac4a0a640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac0159db0 0x7fbaac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.106+0000 7fbac5a0c640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac004d00 0x7fbaac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.106+0000 7fbac520b640 1 -- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac0159db0 msgr2=0x7fbaac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:06.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.106+0000 7fbac520b640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbac0159db0 0x7fbaac0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.106+0000 7fbac520b640 1 -- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac004d00 msgr2=0x7fbaac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:06.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.106+0000 7fbac520b640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac004d00 0x7fbaac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.106+0000 7fbac520b640 1 -- 172.21.15.70:0/2005150227 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac003520 con 0x7fbaac0039a0 2023-10-08T05:56:06.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.106+0000 7fbac5a0c640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbaac004d00 0x7fbaac005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:06.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.107+0000 7fbac520b640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaac0039a0 0x7fbaac003dc0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009a90 tx=0x7fbaac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dc431316018338 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:06.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.107+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc02c680 con 0x7fbaac0039a0 2023-10-08T05:56:06.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.107+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbabc02c7e0 con 0x7fbaac0039a0 2023-10-08T05:56:06.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.107+0000 7fbab7fff640 1 -- 172.21.15.70:0/2005150227 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbabc02cb70 con 0x7fbaac0039a0 2023-10-08T05:56:06.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.108+0000 7fbac620d640 1 -- 172.21.15.70:0/2005150227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaac0039a0 msgr2=0x7fbaac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.108+0000 7fbac620d640 1 --2- 172.21.15.70:0/2005150227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbaac0039a0 0x7fbaac003dc0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009a90 tx=0x7fbaac0057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.108+0000 7fbac5a0c640 1 -- 172.21.15.70:0/2005150227 reap_dead start 2023-10-08T05:56:06.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.108+0000 7fbac620d640 1 -- 172.21.15.70:0/2005150227 shutdown_connections 2023-10-08T05:56:06.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.108+0000 7fbac620d640 1 -- 172.21.15.70:0/2005150227 >> 172.21.15.70:0/2005150227 conn(0x7fbac00cf5f0 msgr2=0x7fbac00cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:06.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.108+0000 7fbac620d640 1 -- 172.21.15.70:0/2005150227 shutdown_connections 2023-10-08T05:56:06.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.108+0000 7fbac620d640 1 -- 172.21.15.70:0/2005150227 wait complete. 2023-10-08T05:56:06.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.110+0000 7fbac620d640 1 Processor -- start 2023-10-08T05:56:06.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.110+0000 7fbac620d640 1 -- start start 2023-10-08T05:56:06.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.111+0000 7fbac620d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac0039a0 0x7fbac00ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.111+0000 7fbac620d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00ae070 0x7fbac014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.111+0000 7fbac620d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbac00cc150 0x7fbac00cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.111+0000 7fbac620d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac01f38c0 con 0x7fbac00cc150 2023-10-08T05:56:06.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.111+0000 7fbac620d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac00ccb10 con 0x7fbac00ae070 2023-10-08T05:56:06.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.111+0000 7fbac620d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbac01f2830 con 0x7fbaac0039a0 2023-10-08T05:56:06.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.111+0000 7fbac5a0c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbac00cc150 0x7fbac00cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.111+0000 7fbac5a0c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbac00cc150 0x7fbac00cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40816/0 (socket says 172.21.15.70:40816) 2023-10-08T05:56:06.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.111+0000 7fbac5a0c640 1 -- 172.21.15.70:0/2265515898 learned_addr learned my addr 172.21.15.70:0/2265515898 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:06.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.111+0000 7fbac520b640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac0039a0 0x7fbac00ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbac4a0a640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00ae070 0x7fbac014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbac5a0c640 1 -- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac0039a0 msgr2=0x7fbac00ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbac5a0c640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac0039a0 0x7fbac00ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbac5a0c640 1 -- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00ae070 msgr2=0x7fbac014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbac5a0c640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00ae070 0x7fbac014ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbac5a0c640 1 -- 172.21.15.70:0/2265515898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac01f2b00 con 0x7fbac00cc150 2023-10-08T05:56:06.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbac4a0a640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00ae070 0x7fbac014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:06.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbac5a0c640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbac00cc150 0x7fbac00cc5d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fbac00d0270 tx=0x7fbab800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:06.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbab5ffb640 1 -- 172.21.15.70:0/2265515898 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8004d60 con 0x7fbac00cc150 2023-10-08T05:56:06.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbab5ffb640 1 -- 172.21.15.70:0/2265515898 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab800bb80 con 0x7fbac00cc150 2023-10-08T05:56:06.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbab5ffb640 1 -- 172.21.15.70:0/2265515898 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab8007070 con 0x7fbac00cc150 2023-10-08T05:56:06.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbac620d640 1 -- 172.21.15.70:0/2265515898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac01f2de0 con 0x7fbac00cc150 2023-10-08T05:56:06.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.112+0000 7fbac620d640 1 -- 172.21.15.70:0/2265515898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac01f33a0 con 0x7fbac00cc150 2023-10-08T05:56:06.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.113+0000 7fbac620d640 1 -- 172.21.15.70:0/2265515898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac0065e20 con 0x7fbac00cc150 2023-10-08T05:56:06.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.115+0000 7fbab5ffb640 1 -- 172.21.15.70:0/2265515898 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 63) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fbab80071d0 con 0x7fbac00cc150 2023-10-08T05:56:06.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.115+0000 7fbab5ffb640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fbaa406b0b0 0x7fbaa406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.116+0000 7fbab5ffb640 1 -- 172.21.15.70:0/2265515898 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fbab808a240 con 0x7fbac00cc150 2023-10-08T05:56:06.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.116+0000 7fbac520b640 1 -- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fbaa406b0b0 msgr2=0x7fbaa406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1786188476 2023-10-08T05:56:06.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.116+0000 7fbac520b640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fbaa406b0b0 0x7fbaa406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:06.122 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:06.117+0000 7f6459e2e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:56:06.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.123+0000 7fbab5ffb640 1 -- 172.21.15.70:0/2265515898 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbab8054b10 con 0x7fbac00cc150 2023-10-08T05:56:06.140 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:06.138+0000 7f909322c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:56:06.179 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:06.178+0000 7f6459e2e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:56:06.189 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:06.188+0000 7f909322c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:56:06.231 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:06.230+0000 7f6459e2e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:56:06.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.312+0000 7fbac620d640 1 -- 172.21.15.70:0/2265515898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbac0074150 con 0x7fbac00cc150 2023-10-08T05:56:06.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.313+0000 7fbab5ffb640 1 -- 172.21.15.70:0/2265515898 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v135) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbab8058130 con 0x7fbac00cc150 2023-10-08T05:56:06.313 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:56:06.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 -- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fbaa406b0b0 msgr2=0x7fbaa406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fbaa406b0b0 0x7fbaa406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 -- 172.21.15.70:0/2265515898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbac00cc150 msgr2=0x7fbac00cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbac00cc150 0x7fbac00cc5d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fbac00d0270 tx=0x7fbab800b4a0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 -- 172.21.15.70:0/2265515898 shutdown_connections 2023-10-08T05:56:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fbaa406b0b0 0x7fbaa406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbac00cc150 0x7fbac00cc5d0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbac00ae070 0x7fbac014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 --2- 172.21.15.70:0/2265515898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbaac0039a0 0x7fbac00ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 -- 172.21.15.70:0/2265515898 >> 172.21.15.70:0/2265515898 conn(0x7fbac00cf5f0 msgr2=0x7fbac00b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 -- 172.21.15.70:0/2265515898 shutdown_connections 2023-10-08T05:56:06.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.315+0000 7fbac620d640 1 -- 172.21.15.70:0/2265515898 wait complete. 2023-10-08T05:56:06.333 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:56:06.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.537+0000 7f3521116640 1 Processor -- start 2023-10-08T05:56:06.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.538+0000 7f3521116640 1 -- start start 2023-10-08T05:56:06.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.539+0000 7f3521116640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c15ac80 0x7f351c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.539+0000 7f3521116640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f351c072220 0x7f351c06a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.539+0000 7f3521116640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c0b37c0 con 0x7f351c0ca0c0 2023-10-08T05:56:06.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.539+0000 7f3521116640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351c0aefd0 con 0x7f351c072220 2023-10-08T05:56:06.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.539+0000 7f3521116640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351c071460 con 0x7f351c15ac80 2023-10-08T05:56:06.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.539+0000 7f351a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f351c072220 0x7f351c06a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.539+0000 7f351a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f351c072220 0x7f351c06a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40830/0 (socket says 172.21.15.70:40830) 2023-10-08T05:56:06.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.539+0000 7f351a59c640 1 -- 172.21.15.70:0/2999822311 learned_addr learned my addr 172.21.15.70:0/2999822311 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:06.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.540+0000 7f351ad9d640 1 --2- 172.21.15.70:0/2999822311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c15ac80 0x7f351c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.540+0000 7f351a59c640 1 -- 172.21.15.70:0/2999822311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c15ac80 msgr2=0x7f351c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:06.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.540+0000 7f351a59c640 1 --2- 172.21.15.70:0/2999822311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c15ac80 0x7f351c069d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.540+0000 7f351a59c640 1 -- 172.21.15.70:0/2999822311 >> v1:172.21.15.181:6789/0 conn(0x7f351c0ca0c0 legacy=0x7f351c0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:06.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.540+0000 7f351a59c640 1 -- 172.21.15.70:0/2999822311 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c071740 con 0x7f351c072220 2023-10-08T05:56:06.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.541+0000 7f351a59c640 1 --2- 172.21.15.70:0/2999822311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 0x7f351c06a290 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3510009870 tx=0x7f351002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56415a40604ead7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:06.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.541+0000 7f3519d9b640 1 -- 172.21.15.70:0/2999822311 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35100317c0 con 0x7f351c072220 2023-10-08T05:56:06.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.541+0000 7f3519d9b640 1 -- 172.21.15.70:0/2999822311 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3510031920 con 0x7f351c072220 2023-10-08T05:56:06.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.541+0000 7f3519d9b640 1 -- 172.21.15.70:0/2999822311 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3510031cb0 con 0x7f351c072220 2023-10-08T05:56:06.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.542+0000 7f3521116640 1 -- 172.21.15.70:0/2999822311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 msgr2=0x7f351c06a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.542+0000 7f3521116640 1 --2- 172.21.15.70:0/2999822311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 0x7f351c06a290 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f3510009870 tx=0x7f351002b860 comp rx=0 tx=0).stop 2023-10-08T05:56:06.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.542+0000 7f3521116640 1 -- 172.21.15.70:0/2999822311 shutdown_connections 2023-10-08T05:56:06.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.542+0000 7f3521116640 1 --2- 172.21.15.70:0/2999822311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 0x7f351c06a290 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.542+0000 7f3521116640 1 --2- 172.21.15.70:0/2999822311 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c15ac80 0x7f351c069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.542+0000 7f3521116640 1 -- 172.21.15.70:0/2999822311 >> 172.21.15.70:0/2999822311 conn(0x7f351c06d430 msgr2=0x7f351c06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.542+0000 7f3521116640 1 -- 172.21.15.70:0/2999822311 shutdown_connections 2023-10-08T05:56:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.542+0000 7f3521116640 1 -- 172.21.15.70:0/2999822311 wait complete. 2023-10-08T05:56:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.544+0000 7f3521116640 1 Processor -- start 2023-10-08T05:56:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.544+0000 7f3521116640 1 -- start start 2023-10-08T05:56:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.545+0000 7f3521116640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 0x7f351c0b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.545+0000 7f3521116640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c0ca0c0 0x7f351c0c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.545+0000 7f3521116640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351c15ac80 0x7f351c0c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.545+0000 7f3521116640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0a7560 con 0x7f351c072220 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.545+0000 7f3521116640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0a7840 con 0x7f351c15ac80 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.545+0000 7f3521116640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0a7b20 con 0x7f351c0ca0c0 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.545+0000 7f351ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 0x7f351c0b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.545+0000 7f351ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 0x7f351c0b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40836/0 (socket says 172.21.15.70:40836) 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.545+0000 7f351ad9d640 1 -- 172.21.15.70:0/7162994 learned_addr learned my addr 172.21.15.70:0/7162994 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.546+0000 7f351a59c640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c0ca0c0 0x7f351c0c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.546+0000 7f351b59e640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351c15ac80 0x7f351c0c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.546+0000 7f351ad9d640 1 -- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c0ca0c0 msgr2=0x7f351c0c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.546+0000 7f351ad9d640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c0ca0c0 0x7f351c0c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.546+0000 7f351ad9d640 1 -- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351c15ac80 msgr2=0x7f351c0c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.546+0000 7f351ad9d640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351c15ac80 0x7f351c0c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.546+0000 7f351ad9d640 1 -- 172.21.15.70:0/7162994 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35100094d0 con 0x7f351c072220 2023-10-08T05:56:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.546+0000 7f351a59c640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c0ca0c0 0x7f351c0c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.546+0000 7f351ad9d640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 0x7f351c0b0a80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c7f0 tx=0x7f350c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.546+0000 7f35037fe640 1 -- 172.21.15.70:0/7162994 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c004220 con 0x7f351c072220 2023-10-08T05:56:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.547+0000 7f35037fe640 1 -- 172.21.15.70:0/7162994 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c0043a0 con 0x7f351c072220 2023-10-08T05:56:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.547+0000 7f35037fe640 1 -- 172.21.15.70:0/7162994 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c007860 con 0x7f351c072220 2023-10-08T05:56:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.547+0000 7f3521116640 1 -- 172.21.15.70:0/7162994 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c0c8a30 con 0x7f351c072220 2023-10-08T05:56:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.547+0000 7f3521116640 1 -- 172.21.15.70:0/7162994 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c157b40 con 0x7f351c072220 2023-10-08T05:56:06.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.548+0000 7f3521116640 1 -- 172.21.15.70:0/7162994 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351c04e160 con 0x7f351c072220 2023-10-08T05:56:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.549+0000 7f35037fe640 1 -- 172.21.15.70:0/7162994 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 63) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f350c007bd0 con 0x7f351c072220 2023-10-08T05:56:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.550+0000 7f35037fe640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f350806b0b0 0x7f350806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.550+0000 7f35037fe640 1 -- 172.21.15.70:0/7162994 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f350c089db0 con 0x7f351c072220 2023-10-08T05:56:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.550+0000 7f351a59c640 1 -- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f350806b0b0 msgr2=0x7f350806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1786188476 2023-10-08T05:56:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.550+0000 7f351a59c640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f350806b0b0 0x7f350806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:06.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.558+0000 7f35037fe640 1 -- 172.21.15.70:0/7162994 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f350c054720 con 0x7f351c072220 2023-10-08T05:56:06.574 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:06.573+0000 7f909322c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:56:06.636 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:06.635+0000 7f6459e2e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:56:06.734 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:06.733+0000 7f909322c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:56:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.747+0000 7f3521116640 1 -- 172.21.15.70:0/7162994 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f351c0a76a0 con 0x7f351c072220 2023-10-08T05:56:06.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.748+0000 7f35037fe640 1 -- 172.21.15.70:0/7162994 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v135) v1 ==== 100+0+11 (secure 0 0 0) 0x7f351c0a76a0 con 0x7f351c072220 2023-10-08T05:56:06.748 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:56:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 -- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f350806b0b0 msgr2=0x7f350806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:06.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f350806b0b0 0x7f350806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 -- 172.21.15.70:0/7162994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 msgr2=0x7f351c0b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 0x7f351c0b0a80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c7f0 tx=0x7f350c00ccb0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 -- 172.21.15.70:0/7162994 shutdown_connections 2023-10-08T05:56:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f350806b0b0 0x7f350806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f351c15ac80 0x7f351c0c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f351c0ca0c0 0x7f351c0c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 --2- 172.21.15.70:0/7162994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f351c072220 0x7f351c0b0a80 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 -- 172.21.15.70:0/7162994 >> 172.21.15.70:0/7162994 conn(0x7f351c06d430 msgr2=0x7f351c069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 -- 172.21.15.70:0/7162994 shutdown_connections 2023-10-08T05:56:06.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.751+0000 7f3521116640 1 -- 172.21.15.70:0/7162994 wait complete. 2023-10-08T05:56:06.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:56:06.801 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:06.801+0000 7f6459e2e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:56:06.841 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:06.840+0000 7f909322c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:56:06.891 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:06.889+0000 7f909322c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:56:06.915 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:06.914+0000 7f6459e2e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:56:06.966 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:06.966+0000 7f6459e2e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:56:06.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.970+0000 7fa9a5354640 1 Processor -- start 2023-10-08T05:56:06.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.970+0000 7fa9a5354640 1 -- start start 2023-10-08T05:56:06.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.971+0000 7fa9a5354640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9a00cf2b0 0x7fa9a00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.971+0000 7fa9a5354640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9a00cf690 0x7fa9a00ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.971+0000 7fa9a5354640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a00cd520 con 0x7fa9a00d1100 2023-10-08T05:56:06.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.971+0000 7fa9a5354640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a00d1910 con 0x7fa9a00cf690 2023-10-08T05:56:06.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.971+0000 7fa9a5354640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a00ce080 con 0x7fa9a00cf2b0 2023-10-08T05:56:06.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.972+0000 7fa99f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9a00cf690 0x7fa9a00ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.972+0000 7fa99f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9a00cf690 0x7fa9a00ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:44300/0 (socket says 172.21.15.70:44300) 2023-10-08T05:56:06.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.972+0000 7fa99f7fe640 1 -- 172.21.15.70:0/3123431806 learned_addr learned my addr 172.21.15.70:0/3123431806 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:06.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.972+0000 7fa99ffff640 1 --2- 172.21.15.70:0/3123431806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9a00cf2b0 0x7fa9a00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.972+0000 7fa99f7fe640 1 -- 172.21.15.70:0/3123431806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9a00cf2b0 msgr2=0x7fa9a00d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:06.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.972+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/3123431806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9a00cf2b0 0x7fa9a00d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.972+0000 7fa99f7fe640 1 -- 172.21.15.70:0/3123431806 >> v1:172.21.15.181:6789/0 conn(0x7fa9a00d1100 legacy=0x7fa9a015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:06.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.972+0000 7fa99f7fe640 1 -- 172.21.15.70:0/3123431806 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a00cd750 con 0x7fa9a00cf690 2023-10-08T05:56:06.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.974+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/3123431806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a00cf690 0x7fa9a00ce510 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa988009870 tx=0x7fa98802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f877bf57998ae6cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:06.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.974+0000 7fa99effd640 1 -- 172.21.15.70:0/3123431806 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988031640 con 0x7fa9a00cf690 2023-10-08T05:56:06.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.974+0000 7fa99effd640 1 -- 172.21.15.70:0/3123431806 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9880317a0 con 0x7fa9a00cf690 2023-10-08T05:56:06.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.974+0000 7fa99effd640 1 -- 172.21.15.70:0/3123431806 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988031b30 con 0x7fa9a00cf690 2023-10-08T05:56:06.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.975+0000 7fa9a5354640 1 -- 172.21.15.70:0/3123431806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a00cf690 msgr2=0x7fa9a00ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.975+0000 7fa9a5354640 1 --2- 172.21.15.70:0/3123431806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a00cf690 0x7fa9a00ce510 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa988009870 tx=0x7fa98802b8f0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.975+0000 7fa9a5354640 1 -- 172.21.15.70:0/3123431806 shutdown_connections 2023-10-08T05:56:06.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.975+0000 7fa9a5354640 1 --2- 172.21.15.70:0/3123431806 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a00cf690 0x7fa9a00ce510 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.975+0000 7fa9a5354640 1 --2- 172.21.15.70:0/3123431806 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9a00cf2b0 0x7fa9a00d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.975+0000 7fa9a5354640 1 -- 172.21.15.70:0/3123431806 >> 172.21.15.70:0/3123431806 conn(0x7fa9a009bc40 msgr2=0x7fa9a00d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.975+0000 7fa9a5354640 1 -- 172.21.15.70:0/3123431806 shutdown_connections 2023-10-08T05:56:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.975+0000 7fa9a5354640 1 -- 172.21.15.70:0/3123431806 wait complete. 2023-10-08T05:56:06.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.976+0000 7fa9a5354640 1 Processor -- start 2023-10-08T05:56:06.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.977+0000 7fa9a5354640 1 -- start start 2023-10-08T05:56:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.977+0000 7fa9a5354640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a00cf2b0 0x7fa9a015b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.977+0000 7fa9a5354640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9a00cf690 0x7fa9a009fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.977+0000 7fa9a5354640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a00d1100 0x7fa9a00b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.977+0000 7fa9a5354640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a009de20 con 0x7fa9a00d1100 2023-10-08T05:56:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa9a5354640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a009cb00 con 0x7fa9a00cf2b0 2023-10-08T05:56:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa9a5354640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a009cde0 con 0x7fa9a00cf690 2023-10-08T05:56:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa9a4b53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a00d1100 0x7fa9a00b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa9a4b53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a00d1100 0x7fa9a00b6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40842/0 (socket says 172.21.15.70:40842) 2023-10-08T05:56:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa9a4b53640 1 -- 172.21.15.70:0/3299423758 learned_addr learned my addr 172.21.15.70:0/3299423758 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa99ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a00cf2b0 0x7fa9a015b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9a00cf690 0x7fa9a009fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:06.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa9a4b53640 1 -- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9a00cf690 msgr2=0x7fa9a009fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa9a4b53640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9a00cf690 0x7fa9a009fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa9a4b53640 1 -- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a00cf2b0 msgr2=0x7fa9a015b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa9a4b53640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a00cf2b0 0x7fa9a015b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.978+0000 7fa9a4b53640 1 -- 172.21.15.70:0/3299423758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9880094d0 con 0x7fa9a00d1100 2023-10-08T05:56:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.979+0000 7fa9a4b53640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a00d1100 0x7fa9a00b6580 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa99800c7a0 tx=0x7fa99800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.979+0000 7fa99cff9640 1 -- 172.21.15.70:0/3299423758 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998003b70 con 0x7fa9a00d1100 2023-10-08T05:56:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.979+0000 7fa99cff9640 1 -- 172.21.15.70:0/3299423758 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa998003cf0 con 0x7fa9a00d1100 2023-10-08T05:56:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.979+0000 7fa99cff9640 1 -- 172.21.15.70:0/3299423758 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa998003ea0 con 0x7fa9a00d1100 2023-10-08T05:56:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.979+0000 7fa9a5354640 1 -- 172.21.15.70:0/3299423758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a009d0c0 con 0x7fa9a00d1100 2023-10-08T05:56:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.979+0000 7fa9a5354640 1 -- 172.21.15.70:0/3299423758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a015bae0 con 0x7fa9a00d1100 2023-10-08T05:56:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.980+0000 7fa9a5354640 1 -- 172.21.15.70:0/3299423758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a0074150 con 0x7fa9a00d1100 2023-10-08T05:56:06.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.981+0000 7fa99cff9640 1 -- 172.21.15.70:0/3299423758 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 63) v1 ==== 89445+0+0 (secure 0 0 0) 0x7fa99800aa90 con 0x7fa9a00d1100 2023-10-08T05:56:06.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.982+0000 7fa99cff9640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa98006b0b0 0x7fa98006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:06.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.982+0000 7fa99cff9640 1 -- 172.21.15.70:0/3299423758 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7fa998089ab0 con 0x7fa9a00d1100 2023-10-08T05:56:06.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.982+0000 7fa99ffff640 1 -- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa98006b0b0 msgr2=0x7fa98006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1786188476 2023-10-08T05:56:06.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.982+0000 7fa99ffff640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa98006b0b0 0x7fa98006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:06.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:06.990+0000 7fa99cff9640 1 -- 172.21.15.70:0/3299423758 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa998054520 con 0x7fa9a00d1100 2023-10-08T05:56:07.001 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:07.000+0000 7f909322c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:07.080 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:07.080+0000 7f6459e2e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:07.109 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:07.108+0000 7f909322c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:56:07.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.183+0000 7fa99ffff640 1 -- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa98006b0b0 msgr2=0x7fa98006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1786188476 2023-10-08T05:56:07.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.183+0000 7fa99ffff640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa98006b0b0 0x7fa98006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:56:07.188 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:07.188+0000 7f6459e2e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:56:07.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.194+0000 7fa9a5354640 1 -- 172.21.15.70:0/3299423758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa9a00cda00 con 0x7fa9a00d1100 2023-10-08T05:56:07.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.195+0000 7fa99cff9640 1 -- 172.21.15.70:0/3299423758 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v135) v1 ==== 80+0+477 (secure 0 0 0) 0x7fa998057b40 con 0x7fa9a00d1100 2023-10-08T05:56:07.195 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1556900853 2023-10-09T05:55:39.473892+0000 2023-10-08T05:56:07.195 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/3142216408 2023-10-09T05:55:39.473892+0000 2023-10-08T05:56:07.196 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/4173673684 2023-10-09T05:55:39.473892+0000 2023-10-08T05:56:07.196 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/3142216408 2023-10-09T05:55:39.473892+0000 2023-10-08T05:56:07.196 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1464588158 2023-10-09T05:55:39.473892+0000 2023-10-08T05:56:07.196 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/791649123 2023-10-09T05:55:39.473892+0000 2023-10-08T05:56:07.196 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2668737378 2023-10-09T05:55:39.473892+0000 2023-10-08T05:56:07.196 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3044665941 2023-10-09T05:55:39.473892+0000 2023-10-08T05:56:07.196 INFO:teuthology.orchestra.run.smithi070.stderr:listed 8 entries 2023-10-08T05:56:07.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 -- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa98006b0b0 msgr2=0x7fa98006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:07.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa98006b0b0 0x7fa98006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 -- 172.21.15.70:0/3299423758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a00d1100 msgr2=0x7fa9a00b6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:07.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a00d1100 0x7fa9a00b6580 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fa99800c7a0 tx=0x7fa99800cc60 comp rx=0 tx=0).stop 2023-10-08T05:56:07.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 -- 172.21.15.70:0/3299423758 shutdown_connections 2023-10-08T05:56:07.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7fa98006b0b0 0x7fa98006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a00d1100 0x7fa9a00b6580 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9a00cf690 0x7fa9a009fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 --2- 172.21.15.70:0/3299423758 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a00cf2b0 0x7fa9a015b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 -- 172.21.15.70:0/3299423758 >> 172.21.15.70:0/3299423758 conn(0x7fa9a009bc40 msgr2=0x7fa9a00cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:07.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.197+0000 7fa9a5354640 1 -- 172.21.15.70:0/3299423758 shutdown_connections 2023-10-08T05:56:07.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.198+0000 7fa9a5354640 1 -- 172.21.15.70:0/3299423758 wait complete. 2023-10-08T05:56:07.216 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T05:56:07.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:07.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T05:56:07.266 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:07.264+0000 7f909322c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:07.314 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:07.313+0000 7f909322c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:07.360 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:07.359+0000 7f6459e2e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:07.382 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:56:07.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:07.383 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T05:56:07.414 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:07.413+0000 7f6459e2e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:07.429 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:56:07.430 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:07.439 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:07.434+0000 7f909322c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:56:07.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:56:07.483 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:56:07.483 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:07.536 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:56:07.537 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:56:07.537 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:07.543 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:07.543+0000 7f6459e2e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:56:07.592 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:56:07.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:56:07.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.798+0000 7f02dcd00640 1 Processor -- start 2023-10-08T05:56:07.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.798+0000 7f02dcd00640 1 -- start start 2023-10-08T05:56:07.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.800+0000 7f02dcd00640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f02d815cd70 0x7f02d80d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:07.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.800+0000 7f02dcd00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d80d0290 0x7f02d80b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:07.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.800+0000 7f02dcd00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d80d2910 con 0x7f02d80b8530 2023-10-08T05:56:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.800+0000 7f02dcd00640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d815d150 con 0x7f02d815cd70 2023-10-08T05:56:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.800+0000 7f02dcd00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80d0c90 con 0x7f02d80d0290 2023-10-08T05:56:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.800+0000 7f02d5d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f02d80b8530 0x7f02d80cfb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42218/0 (socket says 172.21.15.70:42218) 2023-10-08T05:56:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.800+0000 7f02d5d9b640 1 -- 172.21.15.70:0/1700748735 learned_addr learned my addr 172.21.15.70:0/1700748735 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d659c640 1 --2- 172.21.15.70:0/1700748735 >> v2:172.21.15.181:3300/0 conn(0x7f02d815cd70 0x7f02d80d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d6d9d640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d80d0290 0x7f02d80b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078308391 0 0) 0x7f02d80d2910 con 0x7f02d80b8530 2023-10-08T05:56:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02c0003520 con 0x7f02d80b8530 2023-10-08T05:56:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2400983401 0 0) 0x7f02c0003520 con 0x7f02d80b8530 2023-10-08T05:56:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d80d0290 msgr2=0x7f02d80b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d80d0290 0x7f02d80b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 >> v2:172.21.15.181:3300/0 conn(0x7f02d815cd70 msgr2=0x7f02d80d2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 --2- 172.21.15.70:0/1700748735 >> v2:172.21.15.181:3300/0 conn(0x7f02d815cd70 0x7f02d80d2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80d0f70 con 0x7f02d80b8530 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f02cc001670 con 0x7f02d80b8530 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 >> v1:172.21.15.70:6789/0 conn(0x7f02d80b8530 legacy=0x7f02d80cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0003970 0x7f02c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d80d0290 0x7f02c00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c0004d00 0x7f02c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d815d150 con 0x7f02c0004d00 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80d0c90 con 0x7f02c0003970 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.801+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c000b980 con 0x7f02d80d0290 2023-10-08T05:56:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.802+0000 7f02d6d9d640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c0004d00 0x7f02c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.802+0000 7f02d659c640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0003970 0x7f02c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.802+0000 7f02d5d9b640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d80d0290 0x7f02c00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.802+0000 7f02d6d9d640 1 -- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d80d0290 msgr2=0x7f02c00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.802+0000 7f02d6d9d640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02d80d0290 0x7f02c00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.802+0000 7f02d6d9d640 1 -- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0003970 msgr2=0x7f02c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.802+0000 7f02d6d9d640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02c0003970 0x7f02c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.802+0000 7f02d6d9d640 1 -- 172.21.15.70:0/1700748735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0003520 con 0x7f02c0004d00 2023-10-08T05:56:07.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.803+0000 7f02d6d9d640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c0004d00 0x7f02c0005180 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f02d000e170 tx=0x7f02c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dc61cea5c35153 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:07.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.803+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d002cbf0 con 0x7f02c0004d00 2023-10-08T05:56:07.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.803+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d000f040 con 0x7f02c0004d00 2023-10-08T05:56:07.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.803+0000 7f02d559a640 1 -- 172.21.15.70:0/1700748735 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0034440 con 0x7f02c0004d00 2023-10-08T05:56:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.804+0000 7f02dcd00640 1 -- 172.21.15.70:0/1700748735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c0004d00 msgr2=0x7f02c0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.804+0000 7f02dcd00640 1 --2- 172.21.15.70:0/1700748735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c0004d00 0x7f02c0005180 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f02d000e170 tx=0x7f02c00057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.804+0000 7f02d6d9d640 1 -- 172.21.15.70:0/1700748735 reap_dead start 2023-10-08T05:56:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.804+0000 7f02dcd00640 1 -- 172.21.15.70:0/1700748735 shutdown_connections 2023-10-08T05:56:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.804+0000 7f02dcd00640 1 -- 172.21.15.70:0/1700748735 >> 172.21.15.70:0/1700748735 conn(0x7f02d80718a0 msgr2=0x7f02d8071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.804+0000 7f02dcd00640 1 -- 172.21.15.70:0/1700748735 shutdown_connections 2023-10-08T05:56:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.804+0000 7f02dcd00640 1 -- 172.21.15.70:0/1700748735 wait complete. 2023-10-08T05:56:07.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.807+0000 7f02dcd00640 1 Processor -- start 2023-10-08T05:56:07.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.807+0000 7f02dcd00640 1 -- start start 2023-10-08T05:56:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.809+0000 7f02dcd00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c0004d00 0x7f02d8153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.809+0000 7f02dcd00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d81579f0 0x7f02d8158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.809+0000 7f02dcd00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d80cc420 0x7f02d80d2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.809+0000 7f02dcd00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80cb380 con 0x7f02d81579f0 2023-10-08T05:56:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.809+0000 7f02dcd00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80cb660 con 0x7f02d80cc420 2023-10-08T05:56:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.809+0000 7f02dcd00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80cb940 con 0x7f02c0004d00 2023-10-08T05:56:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.809+0000 7f02d5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d81579f0 0x7f02d8158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.809+0000 7f02d5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d81579f0 0x7f02d8158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40860/0 (socket says 172.21.15.70:40860) 2023-10-08T05:56:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.809+0000 7f02d5d9b640 1 -- 172.21.15.70:0/3178039074 learned_addr learned my addr 172.21.15.70:0/3178039074 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02d659c640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c0004d00 0x7f02d8153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02d6d9d640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d80cc420 0x7f02d80d2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02d5d9b640 1 -- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c0004d00 msgr2=0x7f02d8153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02d5d9b640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c0004d00 0x7f02d8153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02d5d9b640 1 -- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d80cc420 msgr2=0x7f02d80d2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02d5d9b640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d80cc420 0x7f02d80d2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02d5d9b640 1 -- 172.21.15.70:0/3178039074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80cbc20 con 0x7f02d81579f0 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02d5d9b640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d81579f0 0x7f02d8158860 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00e8c0 tx=0x7f02cc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02c6ffd640 1 -- 172.21.15.70:0/3178039074 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00c8d0 con 0x7f02d81579f0 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02c6ffd640 1 -- 172.21.15.70:0/3178039074 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc00ca30 con 0x7f02d81579f0 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02dcd00640 1 -- 172.21.15.70:0/3178039074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d80cbf00 con 0x7f02d81579f0 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02c6ffd640 1 -- 172.21.15.70:0/3178039074 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00cde0 con 0x7f02d81579f0 2023-10-08T05:56:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.810+0000 7f02dcd00640 1 -- 172.21.15.70:0/3178039074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d80d30e0 con 0x7f02d81579f0 2023-10-08T05:56:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.812+0000 7f02dcd00640 1 -- 172.21.15.70:0/3178039074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d81e7700 con 0x7f02d81579f0 2023-10-08T05:56:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.813+0000 7f02c6ffd640 1 -- 172.21.15.70:0/3178039074 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 63) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f02cc01d460 con 0x7f02d81579f0 2023-10-08T05:56:07.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.814+0000 7f02c6ffd640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 0x7f02b806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:07.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.814+0000 7f02c6ffd640 1 -- 172.21.15.70:0/3178039074 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f02cc089d00 con 0x7f02d81579f0 2023-10-08T05:56:07.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.814+0000 7f02d659c640 1 -- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 msgr2=0x7f02b806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1786188476 2023-10-08T05:56:07.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.814+0000 7f02d659c640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 0x7f02b806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:07.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:07.818+0000 7f02c6ffd640 1 -- 172.21.15.70:0/3178039074 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f02cc0548c0 con 0x7f02d81579f0 2023-10-08T05:56:08.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.015+0000 7f02d659c640 1 -- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 msgr2=0x7f02b806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1786188476 2023-10-08T05:56:08.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.015+0000 7f02d659c640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 0x7f02b806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:56:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.043+0000 7f02dcd00640 1 -- 172.21.15.70:0/3178039074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02d80cc0f0 con 0x7f02d81579f0 2023-10-08T05:56:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.044+0000 7f02c6ffd640 1 -- 172.21.15.70:0/3178039074 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f02cc001610 con 0x7f02d81579f0 2023-10-08T05:56:08.045 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:08.045 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3983030529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3983030529},{"type":"v1","addr":"172.21.15.181:6836","nonce":3983030529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5873,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1573936162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1573936162},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573936162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1979922916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1979922916},{"type":"v1","addr":"172.21.15.70:6835","nonce":1979922916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:55:25.363123+0000","modified":"2023-10-08T05:55:28.414414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3020884355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3020884355},{"type":"v1","addr":"172.21.15.70:6833","nonce":3020884355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-08T05:56:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 58 2023-10-08T05:56:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 -- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 msgr2=0x7f02b806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 0x7f02b806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 -- 172.21.15.70:0/3178039074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d81579f0 msgr2=0x7f02d8158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d81579f0 0x7f02d8158860 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00e8c0 tx=0x7f02cc00ed80 comp rx=0 tx=0).stop 2023-10-08T05:56:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 -- 172.21.15.70:0/3178039074 shutdown_connections 2023-10-08T05:56:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f02b806b0b0 0x7f02b806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f02d80cc420 0x7f02d80d2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02d81579f0 0x7f02d8158860 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 --2- 172.21.15.70:0/3178039074 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f02c0004d00 0x7f02d8153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 -- 172.21.15.70:0/3178039074 >> 172.21.15.70:0/3178039074 conn(0x7f02d80718a0 msgr2=0x7f02d8156320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 -- 172.21.15.70:0/3178039074 shutdown_connections 2023-10-08T05:56:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.047+0000 7f02dcd00640 1 -- 172.21.15.70:0/3178039074 wait complete. 2023-10-08T05:56:08.064 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:08.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.252+0000 7f84e90a2640 1 Processor -- start 2023-10-08T05:56:08.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.252+0000 7f84e90a2640 1 -- start start 2023-10-08T05:56:08.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.253+0000 7f84e90a2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84e40ad790 0x7f84e40702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.254+0000 7f84e90a2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e406dee0 con 0x7f84e4159ac0 2023-10-08T05:56:08.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.254+0000 7f84e90a2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e415d850 con 0x7f84e415e330 2023-10-08T05:56:08.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.254+0000 7f84e90a2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e406dcf0 con 0x7f84e40ad790 2023-10-08T05:56:08.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.254+0000 7f84e2d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f84e4159ac0 0x7f84e406d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42220/0 (socket says 172.21.15.70:42220) 2023-10-08T05:56:08.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.254+0000 7f84e259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84e40ad790 0x7f84e40702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:08.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.254+0000 7f84e2d9d640 1 -- 172.21.15.70:0/20600278 learned_addr learned my addr 172.21.15.70:0/20600278 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:08.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.254+0000 7f84e259c640 1 -- 172.21.15.70:0/20600278 >> v1:172.21.15.181:6789/0 conn(0x7f84e415e330 legacy=0x7f84e40c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:08.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.254+0000 7f84e259c640 1 -- 172.21.15.70:0/20600278 >> v1:172.21.15.70:6789/0 conn(0x7f84e4159ac0 legacy=0x7f84e406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:08.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.254+0000 7f84e259c640 1 -- 172.21.15.70:0/20600278 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e415db30 con 0x7f84e40ad790 2023-10-08T05:56:08.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.255+0000 7f84e259c640 1 --2- 172.21.15.70:0/20600278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e40ad790 0x7f84e40702d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f84d80097a0 tx=0x7f84d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2fc312cca1302cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:08.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.256+0000 7f84e1d9b640 1 -- 172.21.15.70:0/20600278 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8031830 con 0x7f84e40ad790 2023-10-08T05:56:08.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.256+0000 7f84e1d9b640 1 -- 172.21.15.70:0/20600278 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d8031990 con 0x7f84e40ad790 2023-10-08T05:56:08.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.256+0000 7f84e1d9b640 1 -- 172.21.15.70:0/20600278 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8031d20 con 0x7f84e40ad790 2023-10-08T05:56:08.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.256+0000 7f84e90a2640 1 -- 172.21.15.70:0/20600278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e40ad790 msgr2=0x7f84e40702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:08.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.256+0000 7f84e90a2640 1 --2- 172.21.15.70:0/20600278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e40ad790 0x7f84e40702d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f84d80097a0 tx=0x7f84d802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.257+0000 7f84e90a2640 1 -- 172.21.15.70:0/20600278 shutdown_connections 2023-10-08T05:56:08.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.257+0000 7f84e90a2640 1 --2- 172.21.15.70:0/20600278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e40ad790 0x7f84e40702d0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.257+0000 7f84e90a2640 1 -- 172.21.15.70:0/20600278 >> 172.21.15.70:0/20600278 conn(0x7f84e406a380 msgr2=0x7f84e40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:08.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.257+0000 7f84e90a2640 1 -- 172.21.15.70:0/20600278 shutdown_connections 2023-10-08T05:56:08.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.257+0000 7f84e90a2640 1 -- 172.21.15.70:0/20600278 wait complete. 2023-10-08T05:56:08.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.259+0000 7f84e90a2640 1 Processor -- start 2023-10-08T05:56:08.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.259+0000 7f84e90a2640 1 -- start start 2023-10-08T05:56:08.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.260+0000 7f84e90a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84e40ad790 0x7f84e414f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.260+0000 7f84e90a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e4159ac0 0x7f84e414f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.260+0000 7f84e90a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e415e330 0x7f84e40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.260+0000 7f84e90a2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e415ca90 con 0x7f84e415e330 2023-10-08T05:56:08.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.260+0000 7f84e90a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40b2020 con 0x7f84e4159ac0 2023-10-08T05:56:08.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.260+0000 7f84e90a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e414f5c0 con 0x7f84e40ad790 2023-10-08T05:56:08.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.260+0000 7f84e359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e415e330 0x7f84e40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:08.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.260+0000 7f84e359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e415e330 0x7f84e40b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40886/0 (socket says 172.21.15.70:40886) 2023-10-08T05:56:08.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.260+0000 7f84e359e640 1 -- 172.21.15.70:0/2495086780 learned_addr learned my addr 172.21.15.70:0/2495086780 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:08.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.261+0000 7f84e259c640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e4159ac0 0x7f84e414f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:08.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.261+0000 7f84e2d9d640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84e40ad790 0x7f84e414f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:08.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.261+0000 7f84e359e640 1 -- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84e40ad790 msgr2=0x7f84e414f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:08.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.261+0000 7f84e359e640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84e40ad790 0x7f84e414f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.261+0000 7f84e359e640 1 -- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e4159ac0 msgr2=0x7f84e414f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:08.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.261+0000 7f84e359e640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e4159ac0 0x7f84e414f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.261+0000 7f84e359e640 1 -- 172.21.15.70:0/2495086780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d8009400 con 0x7f84e415e330 2023-10-08T05:56:08.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.261+0000 7f84e259c640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e4159ac0 0x7f84e414f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:56:08.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.261+0000 7f84e359e640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e415e330 0x7f84e40b3ac0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00b970 tx=0x7f84dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:08.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.262+0000 7f84cb7fe640 1 -- 172.21.15.70:0/2495086780 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc00cae0 con 0x7f84e415e330 2023-10-08T05:56:08.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.262+0000 7f84cb7fe640 1 -- 172.21.15.70:0/2495086780 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc00cc40 con 0x7f84e415e330 2023-10-08T05:56:08.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.262+0000 7f84e90a2640 1 -- 172.21.15.70:0/2495086780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e40b1680 con 0x7f84e415e330 2023-10-08T05:56:08.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.262+0000 7f84cb7fe640 1 -- 172.21.15.70:0/2495086780 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc014440 con 0x7f84e415e330 2023-10-08T05:56:08.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.262+0000 7f84e90a2640 1 -- 172.21.15.70:0/2495086780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e40b1c40 con 0x7f84e415e330 2023-10-08T05:56:08.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.263+0000 7f84e90a2640 1 -- 172.21.15.70:0/2495086780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e40b2160 con 0x7f84e415e330 2023-10-08T05:56:08.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.264+0000 7f84cb7fe640 1 -- 172.21.15.70:0/2495086780 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 63) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f84dc01d3b0 con 0x7f84e415e330 2023-10-08T05:56:08.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.265+0000 7f84cb7fe640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f84d006b0b0 0x7f84d006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.265+0000 7f84cb7fe640 1 -- 172.21.15.70:0/2495086780 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f84dc089b40 con 0x7f84e415e330 2023-10-08T05:56:08.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.265+0000 7f84e2d9d640 1 -- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f84d006b0b0 msgr2=0x7f84d006d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1786188476 2023-10-08T05:56:08.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.265+0000 7f84e2d9d640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f84d006b0b0 0x7f84d006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:08.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.269+0000 7f84cb7fe640 1 -- 172.21.15.70:0/2495086780 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84e40b2160 con 0x7f84e415e330 2023-10-08T05:56:08.279 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.278+0000 7f909322c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:56:08.376 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:08.376+0000 7f6459e2e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:56:08.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.450+0000 7f84e90a2640 1 -- 172.21.15.70:0/2495086780 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84e40b2160 con 0x7f84e415e330 2023-10-08T05:56:08.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.451+0000 7f84cb7fe640 1 -- 172.21.15.70:0/2495086780 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+17282 (secure 0 0 0) 0x7f84e40b2160 con 0x7f84e415e330 2023-10-08T05:56:08.452 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:08.452 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":135,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:55:39.474029+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:55:23.150004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-08T05:55:24.358581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1556900853":"2023-10-09T05:55:39.473892+0000","172.21.15.181:6832/3142216408":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/4173673684":"2023-10-09T05:55:39.473892+0000","172.21.15.181:6833/3142216408":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/1464588158":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/791649123":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/2668737378":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/3044665941":"2023-10-09T05:55:39.473892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:08.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 -- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f84d006b0b0 msgr2=0x7f84d006d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:08.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f84d006b0b0 0x7f84d006d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 -- 172.21.15.70:0/2495086780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e415e330 msgr2=0x7f84e40b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:08.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e415e330 0x7f84e40b3ac0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00b970 tx=0x7f84dc00be30 comp rx=0 tx=0).stop 2023-10-08T05:56:08.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 -- 172.21.15.70:0/2495086780 shutdown_connections 2023-10-08T05:56:08.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f84d006b0b0 0x7f84d006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84e415e330 0x7f84e40b3ac0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84e4159ac0 0x7f84e414f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 --2- 172.21.15.70:0/2495086780 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84e40ad790 0x7f84e414f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 -- 172.21.15.70:0/2495086780 >> 172.21.15.70:0/2495086780 conn(0x7f84e406a380 msgr2=0x7f84e4155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:08.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 -- 172.21.15.70:0/2495086780 shutdown_connections 2023-10-08T05:56:08.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.454+0000 7f84e90a2640 1 -- 172.21.15.70:0/2495086780 wait complete. 2023-10-08T05:56:08.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:55:23.150004+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:55:24.358581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:56:08.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:56:08.574 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.572+0000 7f909322c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:56:08.632 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.631+0000 7f909322c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:56:08.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.673+0000 7f8bfdf97640 1 Processor -- start 2023-10-08T05:56:08.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.673+0000 7f8bfdf97640 1 -- start start 2023-10-08T05:56:08.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.675+0000 7f8bfdf97640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8bf815aad0 0x7f8bf80b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.675+0000 7f8bfdf97640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8bf80c82b0 0x7f8bf80c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.675+0000 7f8bfdf97640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf806cd00 0x7f8bf806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.675+0000 7f8bfdf97640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf806d6b0 con 0x7f8bf80c82b0 2023-10-08T05:56:08.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.675+0000 7f8bfdf97640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf815bf30 con 0x7f8bf815aad0 2023-10-08T05:56:08.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.675+0000 7f8bfdf97640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf815b320 con 0x7f8bf806cd00 2023-10-08T05:56:08.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.675+0000 7f8bf6ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8bf80c82b0 0x7f8bf80c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:08.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.675+0000 7f8bf6ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8bf80c82b0 0x7f8bf80c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40890/0 (socket says 172.21.15.70:40890) 2023-10-08T05:56:08.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.675+0000 7f8bf6ffd640 1 -- 172.21.15.70:0/3717097260 learned_addr learned my addr 172.21.15.70:0/3717097260 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:08.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.675+0000 7f8bf7fff640 1 --2- 172.21.15.70:0/3717097260 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf806cd00 0x7f8bf806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:08.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.676+0000 7f8bf6ffd640 1 -- 172.21.15.70:0/3717097260 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf806cd00 msgr2=0x7f8bf806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:08.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.676+0000 7f8bf6ffd640 1 --2- 172.21.15.70:0/3717097260 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf806cd00 0x7f8bf806d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.676+0000 7f8bf6ffd640 1 -- 172.21.15.70:0/3717097260 >> v2:172.21.15.181:3300/0 conn(0x7f8bf815aad0 msgr2=0x7f8bf80b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:08.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.676+0000 7f8bf6ffd640 1 --2- 172.21.15.70:0/3717097260 >> v2:172.21.15.181:3300/0 conn(0x7f8bf815aad0 0x7f8bf80b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.676+0000 7f8bf6ffd640 1 -- 172.21.15.70:0/3717097260 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf815b600 con 0x7f8bf80c82b0 2023-10-08T05:56:08.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.676+0000 7f8bf7fff640 1 --2- 172.21.15.70:0/3717097260 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf806cd00 0x7f8bf806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:08.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.677+0000 7f8bf6ffd640 1 --2- 172.21.15.70:0/3717097260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf80c82b0 0x7f8bf80c8710 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f8bf806e6a0 tx=0x7f8bec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37298e95b5fab53e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:08.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.677+0000 7f8bf67fc640 1 -- 172.21.15.70:0/3717097260 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec031870 con 0x7f8bf80c82b0 2023-10-08T05:56:08.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.677+0000 7f8bf67fc640 1 -- 172.21.15.70:0/3717097260 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec0319d0 con 0x7f8bf80c82b0 2023-10-08T05:56:08.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.677+0000 7f8bf67fc640 1 -- 172.21.15.70:0/3717097260 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec031d60 con 0x7f8bf80c82b0 2023-10-08T05:56:08.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.678+0000 7f8bfdf97640 1 -- 172.21.15.70:0/3717097260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf80c82b0 msgr2=0x7f8bf80c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:08.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.678+0000 7f8bfdf97640 1 --2- 172.21.15.70:0/3717097260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf80c82b0 0x7f8bf80c8710 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f8bf806e6a0 tx=0x7f8bec02b910 comp rx=0 tx=0).stop 2023-10-08T05:56:08.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.678+0000 7f8bfdf97640 1 -- 172.21.15.70:0/3717097260 shutdown_connections 2023-10-08T05:56:08.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.678+0000 7f8bfdf97640 1 --2- 172.21.15.70:0/3717097260 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf806cd00 0x7f8bf806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.678+0000 7f8bfdf97640 1 --2- 172.21.15.70:0/3717097260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf80c82b0 0x7f8bf80c8710 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.678+0000 7f8bfdf97640 1 --2- 172.21.15.70:0/3717097260 >> v2:172.21.15.181:3300/0 conn(0x7f8bf815aad0 0x7f8bf80b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.678+0000 7f8bfdf97640 1 -- 172.21.15.70:0/3717097260 >> 172.21.15.70:0/3717097260 conn(0x7f8bf8069d50 msgr2=0x7f8bf806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:08.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.678+0000 7f8bfdf97640 1 -- 172.21.15.70:0/3717097260 shutdown_connections 2023-10-08T05:56:08.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.678+0000 7f8bfdf97640 1 -- 172.21.15.70:0/3717097260 wait complete. 2023-10-08T05:56:08.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.680+0000 7f8bfdf97640 1 Processor -- start 2023-10-08T05:56:08.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.681+0000 7f8bfdf97640 1 -- start start 2023-10-08T05:56:08.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.682+0000 7f8bfdf97640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf806cd00 0x7f8bf81430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.682+0000 7f8bfdf97640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80c82b0 0x7f8bf81516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.682+0000 7f8bfdf97640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf815aad0 0x7f8bf8151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.682+0000 7f8bfdf97640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8157c30 con 0x7f8bf815aad0 2023-10-08T05:56:08.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.682+0000 7f8bfdf97640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf81539a0 con 0x7f8bf806cd00 2023-10-08T05:56:08.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.682+0000 7f8bfdf97640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8157ef0 con 0x7f8bf80c82b0 2023-10-08T05:56:08.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf815aad0 0x7f8bf8151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:08.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf815aad0 0x7f8bf8151c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40896/0 (socket says 172.21.15.70:40896) 2023-10-08T05:56:08.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf7fff640 1 -- 172.21.15.70:0/1610943259 learned_addr learned my addr 172.21.15.70:0/1610943259 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:08.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf77fe640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf806cd00 0x7f8bf81430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:08.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf6ffd640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80c82b0 0x7f8bf81516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:08.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf7fff640 1 -- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80c82b0 msgr2=0x7f8bf81516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:08.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf7fff640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80c82b0 0x7f8bf81516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf7fff640 1 -- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf806cd00 msgr2=0x7f8bf81430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:08.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf7fff640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf806cd00 0x7f8bf81430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf7fff640 1 -- 172.21.15.70:0/1610943259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec0094d0 con 0x7f8bf815aad0 2023-10-08T05:56:08.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.683+0000 7f8bf7fff640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf815aad0 0x7f8bf8151c00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000cd50 tx=0x7f8bf000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:08.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.684+0000 7f8bdffff640 1 -- 172.21.15.70:0/1610943259 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000f680 con 0x7f8bf815aad0 2023-10-08T05:56:08.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.684+0000 7f8bdffff640 1 -- 172.21.15.70:0/1610943259 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf000f800 con 0x7f8bf815aad0 2023-10-08T05:56:08.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.684+0000 7f8bdffff640 1 -- 172.21.15.70:0/1610943259 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000fbb0 con 0x7f8bf815aad0 2023-10-08T05:56:08.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.684+0000 7f8bfdf97640 1 -- 172.21.15.70:0/1610943259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf81581d0 con 0x7f8bf815aad0 2023-10-08T05:56:08.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.684+0000 7f8bfdf97640 1 -- 172.21.15.70:0/1610943259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf8143600 con 0x7f8bf815aad0 2023-10-08T05:56:08.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.685+0000 7f8bfdf97640 1 -- 172.21.15.70:0/1610943259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf8143820 con 0x7f8bf815aad0 2023-10-08T05:56:08.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.686+0000 7f8bdffff640 1 -- 172.21.15.70:0/1610943259 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 63) v1 ==== 89445+0+0 (secure 0 0 0) 0x7f8bf000b040 con 0x7f8bf815aad0 2023-10-08T05:56:08.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.687+0000 7f8bdffff640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f8be406b0b0 0x7f8be406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:08.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.687+0000 7f8bdffff640 1 -- 172.21.15.70:0/1610943259 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6417+0+0 (secure 0 0 0) 0x7f8bf008a840 con 0x7f8bf815aad0 2023-10-08T05:56:08.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.687+0000 7f8bf77fe640 1 -- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f8be406b0b0 msgr2=0x7f8be406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1786188476 2023-10-08T05:56:08.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.688+0000 7f8bf77fe640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f8be406b0b0 0x7f8be406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:08.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.692+0000 7f8bdffff640 1 -- 172.21.15.70:0/1610943259 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bf8143820 con 0x7f8bf815aad0 2023-10-08T05:56:08.693 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:08.693+0000 7f6459e2e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:56:08.746 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.745+0000 7f909322c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:56:08.753 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:08.753+0000 7f6459e2e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:56:08.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.766+0000 7f8bdffff640 1 -- 172.21.15.70:0/1610943259 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 64) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f8bf000fd10 con 0x7f8bf815aad0 2023-10-08T05:56:08.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.767+0000 7f8bdffff640 1 -- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f8be406b0b0 msgr2=0x7f8be406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:08.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.767+0000 7f8bdffff640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f8be406b0b0 0x7f8be406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.840 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.839+0000 7f9014dec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:08.840 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.839+0000 7f9014dec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:08.840 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.839+0000 7f9014dec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:08.841 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.839+0000 7f9014dec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:08.841 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.839+0000 7f9014dec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:08.841 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.840+0000 7f9017df2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:08.841 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.840+0000 7f9017df2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:08.842 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.840+0000 7f9017df2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:08.842 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.840+0000 7f9017df2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:08.842 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:08.840+0000 7f9017df2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:08.882 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:08.881+0000 7f6459e2e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:56:08.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.883+0000 7f8bfdf97640 1 -- 172.21.15.70:0/1610943259 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bf8143820 con 0x7f8bf815aad0 2023-10-08T05:56:08.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.884+0000 7f8bdffff640 1 -- 172.21.15.70:0/1610943259 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bf8143820 con 0x7f8bf815aad0 2023-10-08T05:56:08.884 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:56:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.886+0000 7f8bfdf97640 1 -- 172.21.15.70:0/1610943259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf815aad0 msgr2=0x7f8bf8151c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.886+0000 7f8bfdf97640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf815aad0 0x7f8bf8151c00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000cd50 tx=0x7f8bf000a650 comp rx=0 tx=0).stop 2023-10-08T05:56:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.886+0000 7f8bfdf97640 1 -- 172.21.15.70:0/1610943259 shutdown_connections 2023-10-08T05:56:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.886+0000 7f8bfdf97640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:6832/1786188476,v1:172.21.15.181:6833/1786188476] conn(0x7f8be406b0b0 0x7f8be406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.886+0000 7f8bfdf97640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bf815aad0 0x7f8bf8151c00 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.886+0000 7f8bfdf97640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bf80c82b0 0x7f8bf81516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.886+0000 7f8bfdf97640 1 --2- 172.21.15.70:0/1610943259 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bf806cd00 0x7f8bf81430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:08.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.886+0000 7f8bfdf97640 1 -- 172.21.15.70:0/1610943259 >> 172.21.15.70:0/1610943259 conn(0x7f8bf8069d50 msgr2=0x7f8bf815e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:08.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.886+0000 7f8bfdf97640 1 -- 172.21.15.70:0/1610943259 shutdown_connections 2023-10-08T05:56:08.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:08.886+0000 7f8bfdf97640 1 -- 172.21.15.70:0/1610943259 wait complete. 2023-10-08T05:56:08.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:56:09.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.093+0000 7f406d6af640 1 Processor -- start 2023-10-08T05:56:09.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.094+0000 7f406d6af640 1 -- start start 2023-10-08T05:56:09.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.095+0000 7f406d6af640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f406815cdb0 0x7f40680cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.095+0000 7f406d6af640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f40680ccb40 0x7f40680ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.095+0000 7f406d6af640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406815d620 con 0x7f406815d190 2023-10-08T05:56:09.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.095+0000 7f406d6af640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40680cd490 con 0x7f406815cdb0 2023-10-08T05:56:09.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.095+0000 7f406d6af640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406806b6a0 con 0x7f40680ccb40 2023-10-08T05:56:09.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.095+0000 7f4066ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f406815cdb0 0x7f40680cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.095+0000 7f4066ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f406815cdb0 0x7f40680cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40904/0 (socket says 172.21.15.70:40904) 2023-10-08T05:56:09.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.095+0000 7f4066ffd640 1 -- 172.21.15.70:0/2338257797 learned_addr learned my addr 172.21.15.70:0/2338257797 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:09.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.096+0000 7f4066ffd640 1 -- 172.21.15.70:0/2338257797 >> v2:172.21.15.181:3300/0 conn(0x7f40680ccb40 msgr2=0x7f40680ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:09.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.096+0000 7f40677fe640 1 --2- 172.21.15.70:0/2338257797 >> v2:172.21.15.181:3300/0 conn(0x7f40680ccb40 0x7f40680ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.096+0000 7f4066ffd640 1 --2- 172.21.15.70:0/2338257797 >> v2:172.21.15.181:3300/0 conn(0x7f40680ccb40 0x7f40680ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.096+0000 7f4066ffd640 1 -- 172.21.15.70:0/2338257797 >> v1:172.21.15.181:6789/0 conn(0x7f406815d190 legacy=0x7f406806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:09.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.096+0000 7f4066ffd640 1 -- 172.21.15.70:0/2338257797 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40680b19e0 con 0x7f406815cdb0 2023-10-08T05:56:09.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.097+0000 7f4066ffd640 1 --2- 172.21.15.70:0/2338257797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 0x7f40680cea30 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f40680cefa0 tx=0x7f405802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284c9bae59cfb6ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:09.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.097+0000 7f4065ffb640 1 -- 172.21.15.70:0/2338257797 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405802ca80 con 0x7f406815cdb0 2023-10-08T05:56:09.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.097+0000 7f4065ffb640 1 -- 172.21.15.70:0/2338257797 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405802cbe0 con 0x7f406815cdb0 2023-10-08T05:56:09.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.097+0000 7f4065ffb640 1 -- 172.21.15.70:0/2338257797 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40580343f0 con 0x7f406815cdb0 2023-10-08T05:56:09.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.098+0000 7f406d6af640 1 -- 172.21.15.70:0/2338257797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 msgr2=0x7f40680cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.098+0000 7f406d6af640 1 --2- 172.21.15.70:0/2338257797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 0x7f40680cea30 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f40680cefa0 tx=0x7f405802bee0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.098+0000 7f406d6af640 1 -- 172.21.15.70:0/2338257797 shutdown_connections 2023-10-08T05:56:09.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.098+0000 7f406d6af640 1 --2- 172.21.15.70:0/2338257797 >> v2:172.21.15.181:3300/0 conn(0x7f40680ccb40 0x7f40680ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.098+0000 7f406d6af640 1 --2- 172.21.15.70:0/2338257797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 0x7f40680cea30 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.098+0000 7f406d6af640 1 -- 172.21.15.70:0/2338257797 >> 172.21.15.70:0/2338257797 conn(0x7f40680b0af0 msgr2=0x7f40680b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:09.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.098+0000 7f406d6af640 1 -- 172.21.15.70:0/2338257797 shutdown_connections 2023-10-08T05:56:09.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.098+0000 7f406d6af640 1 -- 172.21.15.70:0/2338257797 wait complete. 2023-10-08T05:56:09.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.099+0000 7f406d6af640 1 Processor -- start 2023-10-08T05:56:09.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.100+0000 7f406d6af640 1 -- start start 2023-10-08T05:56:09.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.100+0000 7f406d6af640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40680ccb40 0x7f40681f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.100+0000 7f406d6af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 0x7f40680b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f406d6af640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815d190 0x7f40680b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f406d6af640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406815c610 con 0x7f406815cdb0 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f406d6af640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406815ba90 con 0x7f40680ccb40 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f406d6af640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406815bd70 con 0x7f406815d190 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f40667fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 0x7f40680b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f40667fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 0x7f40680b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40914/0 (socket says 172.21.15.70:40914) 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f40667fc640 1 -- 172.21.15.70:0/1071076987 learned_addr learned my addr 172.21.15.70:0/1071076987 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f4066ffd640 1 --2- 172.21.15.70:0/1071076987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40680ccb40 0x7f40681f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f40667fc640 1 -- 172.21.15.70:0/1071076987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815d190 msgr2=0x7f40680b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f40667fc640 1 --2- 172.21.15.70:0/1071076987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815d190 0x7f40680b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f40667fc640 1 -- 172.21.15.70:0/1071076987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40680ccb40 msgr2=0x7f40681f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f40667fc640 1 --2- 172.21.15.70:0/1071076987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40680ccb40 0x7f40681f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f40667fc640 1 -- 172.21.15.70:0/1071076987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4058009b30 con 0x7f406815cdb0 2023-10-08T05:56:09.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.101+0000 7f40667fc640 1 --2- 172.21.15.70:0/1071076987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 0x7f40680b3df0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f405000c410 tx=0x7f405000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:09.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.102+0000 7f405f7fe640 1 -- 172.21.15.70:0/1071076987 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40500035f0 con 0x7f406815cdb0 2023-10-08T05:56:09.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.102+0000 7f405f7fe640 1 -- 172.21.15.70:0/1071076987 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4050003750 con 0x7f406815cdb0 2023-10-08T05:56:09.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.102+0000 7f405f7fe640 1 -- 172.21.15.70:0/1071076987 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050003b40 con 0x7f406815cdb0 2023-10-08T05:56:09.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.102+0000 7f406d6af640 1 -- 172.21.15.70:0/1071076987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406815c050 con 0x7f406815cdb0 2023-10-08T05:56:09.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.102+0000 7f406d6af640 1 -- 172.21.15.70:0/1071076987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40681f21a0 con 0x7f406815cdb0 2023-10-08T05:56:09.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.103+0000 7f406d6af640 1 -- 172.21.15.70:0/1071076987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40680c2b30 con 0x7f406815cdb0 2023-10-08T05:56:09.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.104+0000 7f405f7fe640 1 -- 172.21.15.70:0/1071076987 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 64) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f4050003ca0 con 0x7f406815cdb0 2023-10-08T05:56:09.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.105+0000 7f405f7fe640 1 -- 172.21.15.70:0/1071076987 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f4050088840 con 0x7f406815cdb0 2023-10-08T05:56:09.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.109+0000 7f405f7fe640 1 -- 172.21.15.70:0/1071076987 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f40680c2b30 con 0x7f406815cdb0 2023-10-08T05:56:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.282+0000 7f406d6af640 1 -- 172.21.15.70:0/1071076987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40680c2b30 con 0x7f406815cdb0 2023-10-08T05:56:09.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.283+0000 7f405f7fe640 1 -- 172.21.15.70:0/1071076987 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v136) v1 ==== 104+0+11 (secure 0 0 0) 0x7f40680c2b30 con 0x7f406815cdb0 2023-10-08T05:56:09.286 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:56:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.286+0000 7f406d6af640 1 -- 172.21.15.70:0/1071076987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 msgr2=0x7f40680b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.286+0000 7f406d6af640 1 --2- 172.21.15.70:0/1071076987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 0x7f40680b3df0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f405000c410 tx=0x7f405000c8d0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.286+0000 7f406d6af640 1 -- 172.21.15.70:0/1071076987 shutdown_connections 2023-10-08T05:56:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.286+0000 7f406d6af640 1 --2- 172.21.15.70:0/1071076987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f406815d190 0x7f40680b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.286+0000 7f406d6af640 1 --2- 172.21.15.70:0/1071076987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f406815cdb0 0x7f40680b3df0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.286+0000 7f406d6af640 1 --2- 172.21.15.70:0/1071076987 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40680ccb40 0x7f40681f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.286+0000 7f406d6af640 1 -- 172.21.15.70:0/1071076987 >> 172.21.15.70:0/1071076987 conn(0x7f40680b0af0 msgr2=0x7f40680abe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:09.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.286+0000 7f406d6af640 1 -- 172.21.15.70:0/1071076987 shutdown_connections 2023-10-08T05:56:09.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.286+0000 7f406d6af640 1 -- 172.21.15.70:0/1071076987 wait complete. 2023-10-08T05:56:09.304 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:56:09.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.508+0000 7fa778490640 1 Processor -- start 2023-10-08T05:56:09.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.508+0000 7fa778490640 1 -- start start 2023-10-08T05:56:09.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.510+0000 7fa778490640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa770160660 0x7fa770070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.510+0000 7fa778490640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7700c9910 0x7fa7700caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.510+0000 7fa778490640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77006e150 con 0x7fa7700cb450 2023-10-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.510+0000 7fa778490640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7700cb8c0 con 0x7fa770160660 2023-10-08T05:56:09.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.510+0000 7fa778490640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77006de70 con 0x7fa7700c9910 2023-10-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.511+0000 7fa775a2b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7700c9910 0x7fa7700caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.511+0000 7fa77622c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa770160660 0x7fa770070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.511+0000 7fa775a2b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7700c9910 0x7fa7700caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:37028/0 (socket says 172.21.15.70:37028) 2023-10-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.511+0000 7fa77622c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa770160660 0x7fa770070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:44374/0 (socket says 172.21.15.70:44374) 2023-10-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.511+0000 7fa775a2b640 1 -- 172.21.15.70:0/681909797 learned_addr learned my addr 172.21.15.70:0/681909797 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.512+0000 7fa77622c640 1 -- 172.21.15.70:0/681909797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7700c9910 msgr2=0x7fa7700caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.512+0000 7fa77622c640 1 --2- 172.21.15.70:0/681909797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7700c9910 0x7fa7700caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.512+0000 7fa77622c640 1 -- 172.21.15.70:0/681909797 >> v1:172.21.15.181:6789/0 conn(0x7fa7700cb450 legacy=0x7fa7700b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.512+0000 7fa77622c640 1 -- 172.21.15.70:0/681909797 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700c8e30 con 0x7fa770160660 2023-10-08T05:56:09.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.513+0000 7fa77622c640 1 --2- 172.21.15.70:0/681909797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa770160660 0x7fa770070cc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0097a0 tx=0x7fa76c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=542ff4dc602afa31 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:09.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.513+0000 7fa77522a640 1 -- 172.21.15.70:0/681909797 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c02d040 con 0x7fa770160660 2023-10-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.513+0000 7fa77522a640 1 -- 172.21.15.70:0/681909797 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c031810 con 0x7fa770160660 2023-10-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.513+0000 7fa77522a640 1 -- 172.21.15.70:0/681909797 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c031ba0 con 0x7fa770160660 2023-10-08T05:56:09.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.514+0000 7fa778490640 1 -- 172.21.15.70:0/681909797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa770160660 msgr2=0x7fa770070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.514+0000 7fa778490640 1 --2- 172.21.15.70:0/681909797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa770160660 0x7fa770070cc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0097a0 tx=0x7fa76c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.514+0000 7fa778490640 1 -- 172.21.15.70:0/681909797 shutdown_connections 2023-10-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.514+0000 7fa778490640 1 --2- 172.21.15.70:0/681909797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7700c9910 0x7fa7700caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.514+0000 7fa778490640 1 --2- 172.21.15.70:0/681909797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa770160660 0x7fa770070cc0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.514+0000 7fa778490640 1 -- 172.21.15.70:0/681909797 >> 172.21.15.70:0/681909797 conn(0x7fa77006a5d0 msgr2=0x7fa7700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.514+0000 7fa778490640 1 -- 172.21.15.70:0/681909797 shutdown_connections 2023-10-08T05:56:09.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.514+0000 7fa778490640 1 -- 172.21.15.70:0/681909797 wait complete. 2023-10-08T05:56:09.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.516+0000 7fa778490640 1 Processor -- start 2023-10-08T05:56:09.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.516+0000 7fa778490640 1 -- start start 2023-10-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.517+0000 7fa778490640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7700c9910 0x7fa770152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.517+0000 7fa778490640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7700cb450 0x7fa770154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.517+0000 7fa778490640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa770160660 0x7fa770156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.517+0000 7fa778490640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770153e70 con 0x7fa7700c9910 2023-10-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.517+0000 7fa778490640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770153410 con 0x7fa770160660 2023-10-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.517+0000 7fa778490640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa770158320 con 0x7fa7700cb450 2023-10-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.517+0000 7fa77622c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7700c9910 0x7fa770152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.517+0000 7fa77622c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7700c9910 0x7fa770152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40918/0 (socket says 172.21.15.70:40918) 2023-10-08T05:56:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.517+0000 7fa77622c640 1 -- 172.21.15.70:0/2773819145 learned_addr learned my addr 172.21.15.70:0/2773819145 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa77622c640 1 -- 172.21.15.70:0/2773819145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7700cb450 msgr2=0x7fa770154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa77622c640 1 --2- 172.21.15.70:0/2773819145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7700cb450 0x7fa770154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa776a2d640 1 --2- 172.21.15.70:0/2773819145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa770160660 0x7fa770156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa77622c640 1 -- 172.21.15.70:0/2773819145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa770160660 msgr2=0x7fa770156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa77622c640 1 --2- 172.21.15.70:0/2773819145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa770160660 0x7fa770156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa77622c640 1 -- 172.21.15.70:0/2773819145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c009400 con 0x7fa7700c9910 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa77622c640 1 --2- 172.21.15.70:0/2773819145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7700c9910 0x7fa770152ed0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0098c0 tx=0x7fa76c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa762ffd640 1 -- 172.21.15.70:0/2773819145 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c03ad60 con 0x7fa7700c9910 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa762ffd640 1 -- 172.21.15.70:0/2773819145 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76c0423f0 con 0x7fa7700c9910 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa762ffd640 1 -- 172.21.15.70:0/2773819145 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c0427a0 con 0x7fa7700c9910 2023-10-08T05:56:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.518+0000 7fa778490640 1 -- 172.21.15.70:0/2773819145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa770154080 con 0x7fa7700c9910 2023-10-08T05:56:09.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.519+0000 7fa778490640 1 -- 172.21.15.70:0/2773819145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa770154640 con 0x7fa7700c9910 2023-10-08T05:56:09.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.520+0000 7fa778490640 1 -- 172.21.15.70:0/2773819145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77004e160 con 0x7fa7700c9910 2023-10-08T05:56:09.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.520+0000 7fa762ffd640 1 -- 172.21.15.70:0/2773819145 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 64) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fa76c0393b0 con 0x7fa7700c9910 2023-10-08T05:56:09.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.521+0000 7fa762ffd640 1 -- 172.21.15.70:0/2773819145 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fa76c0ab210 con 0x7fa7700c9910 2023-10-08T05:56:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.525+0000 7fa762ffd640 1 -- 172.21.15.70:0/2773819145 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa76c0762a0 con 0x7fa7700c9910 2023-10-08T05:56:09.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.701+0000 7fa778490640 1 -- 172.21.15.70:0/2773819145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa770154800 con 0x7fa7700c9910 2023-10-08T05:56:09.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.702+0000 7fa762ffd640 1 -- 172.21.15.70:0/2773819145 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa76c0798c0 con 0x7fa7700c9910 2023-10-08T05:56:09.703 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:56:09.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.705+0000 7fa778490640 1 -- 172.21.15.70:0/2773819145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7700c9910 msgr2=0x7fa770152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.705+0000 7fa778490640 1 --2- 172.21.15.70:0/2773819145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7700c9910 0x7fa770152ed0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0098c0 tx=0x7fa76c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.705+0000 7fa778490640 1 -- 172.21.15.70:0/2773819145 shutdown_connections 2023-10-08T05:56:09.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.705+0000 7fa778490640 1 --2- 172.21.15.70:0/2773819145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa770160660 0x7fa770156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.705+0000 7fa778490640 1 --2- 172.21.15.70:0/2773819145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7700cb450 0x7fa770154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.705+0000 7fa778490640 1 --2- 172.21.15.70:0/2773819145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7700c9910 0x7fa770152ed0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.705+0000 7fa778490640 1 -- 172.21.15.70:0/2773819145 >> 172.21.15.70:0/2773819145 conn(0x7fa77006a5d0 msgr2=0x7fa7700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.705+0000 7fa778490640 1 -- 172.21.15.70:0/2773819145 shutdown_connections 2023-10-08T05:56:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.705+0000 7fa778490640 1 -- 172.21.15.70:0/2773819145 wait complete. 2023-10-08T05:56:09.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:56:09.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.945+0000 7fcebed05640 1 Processor -- start 2023-10-08T05:56:09.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.945+0000 7fcebed05640 1 -- start start 2023-10-08T05:56:09.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.947+0000 7fcebed05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb8158710 0x7fceb815f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.947+0000 7fcebed05640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb8158af0 con 0x7fceb80b1280 2023-10-08T05:56:09.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.947+0000 7fcebed05640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb80ca370 con 0x7fceb815bad0 2023-10-08T05:56:09.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.947+0000 7fcebed05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb815ee70 con 0x7fceb8158710 2023-10-08T05:56:09.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.947+0000 7fcebd2a2640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fceb80b1280 0x7fceb8069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42232/0 (socket says 172.21.15.70:42232) 2023-10-08T05:56:09.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.947+0000 7fcebd2a2640 1 -- 172.21.15.70:0/544721805 learned_addr learned my addr 172.21.15.70:0/544721805 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:09.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.947+0000 7fcebcaa1640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb8158710 0x7fceb815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726996976 0 0) 0x7fceb8158af0 con 0x7fceb80b1280 2023-10-08T05:56:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea8003520 con 0x7fceb80b1280 2023-10-08T05:56:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 709549055 0 0) 0x7fcea8003520 con 0x7fceb80b1280 2023-10-08T05:56:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb8158710 msgr2=0x7fceb815f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb8158710 0x7fceb815f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 >> v1:172.21.15.181:6789/0 conn(0x7fceb815bad0 legacy=0x7fceb80b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb815e7b0 con 0x7fceb80b1280 2023-10-08T05:56:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fceb0001a10 con 0x7fceb80b1280 2023-10-08T05:56:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 >> v1:172.21.15.70:6789/0 conn(0x7fceb80b1280 legacy=0x7fceb8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea80039a0 0x7fcea8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 0x7fcea8004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcea8005100 0x7fcea800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb815ee70 con 0x7fcea8004390 2023-10-08T05:56:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea800c020 con 0x7fcea8005100 2023-10-08T05:56:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.948+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea800c1e0 con 0x7fcea80039a0 2023-10-08T05:56:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.949+0000 7fceb7fff640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 0x7fcea8004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.949+0000 7fcebcaa1640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea80039a0 0x7fcea8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.949+0000 7fcebd2a2640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcea8005100 0x7fcea800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.949+0000 7fceb7fff640 1 -- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea80039a0 msgr2=0x7fcea8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.949+0000 7fceb7fff640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcea80039a0 0x7fcea8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.949+0000 7fceb7fff640 1 -- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcea8005100 msgr2=0x7fcea800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.949+0000 7fceb7fff640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcea8005100 0x7fcea800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.949+0000 7fceb7fff640 1 -- 172.21.15.70:0/544721805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea8003520 con 0x7fcea8004390 2023-10-08T05:56:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.950+0000 7fceb7fff640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 0x7fcea8004860 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcea4001750 tx=0x7fcea80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cd2c7d09163198 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.951+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea40052d0 con 0x7fcea8004390 2023-10-08T05:56:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.951+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea402e050 con 0x7fcea8004390 2023-10-08T05:56:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.951+0000 7fceb77fe640 1 -- 172.21.15.70:0/544721805 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea4003830 con 0x7fcea8004390 2023-10-08T05:56:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.951+0000 7fcebed05640 1 -- 172.21.15.70:0/544721805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 msgr2=0x7fcea8004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.951+0000 7fcebed05640 1 --2- 172.21.15.70:0/544721805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 0x7fcea8004860 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcea4001750 tx=0x7fcea80057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.951+0000 7fcebd2a2640 1 -- 172.21.15.70:0/544721805 reap_dead start 2023-10-08T05:56:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.951+0000 7fcebed05640 1 -- 172.21.15.70:0/544721805 shutdown_connections 2023-10-08T05:56:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.951+0000 7fcebed05640 1 -- 172.21.15.70:0/544721805 >> 172.21.15.70:0/544721805 conn(0x7fceb80cf890 msgr2=0x7fceb80cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.951+0000 7fcebed05640 1 -- 172.21.15.70:0/544721805 shutdown_connections 2023-10-08T05:56:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.951+0000 7fcebed05640 1 -- 172.21.15.70:0/544721805 wait complete. 2023-10-08T05:56:09.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.953+0000 7fcebed05640 1 Processor -- start 2023-10-08T05:56:09.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.953+0000 7fcebed05640 1 -- start start 2023-10-08T05:56:09.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.954+0000 7fcebed05640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 0x7fceb80c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.954+0000 7fcebed05640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb80c2a90 0x7fceb80c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.954+0000 7fcebed05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb80ad1e0 0x7fceb80ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.954+0000 7fcebed05640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb80acdc0 con 0x7fcea8004390 2023-10-08T05:56:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.954+0000 7fcebed05640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb80c3450 con 0x7fceb80c2a90 2023-10-08T05:56:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.954+0000 7fcebed05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb80c59e0 con 0x7fceb80ad1e0 2023-10-08T05:56:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.955+0000 7fcebcaa1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 0x7fceb80c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.955+0000 7fcebcaa1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 0x7fceb80c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40942/0 (socket says 172.21.15.70:40942) 2023-10-08T05:56:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.955+0000 7fcebcaa1640 1 -- 172.21.15.70:0/2567656977 learned_addr learned my addr 172.21.15.70:0/2567656977 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.955+0000 7fcebcaa1640 1 -- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb80ad1e0 msgr2=0x7fceb80ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.955+0000 7fcebcaa1640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb80ad1e0 0x7fceb80ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.955+0000 7fcebcaa1640 1 -- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb80c2a90 msgr2=0x7fceb80c2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.955+0000 7fceb7fff640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb80c2a90 0x7fceb80c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.955+0000 7fcebcaa1640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb80c2a90 0x7fceb80c2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.955+0000 7fcebcaa1640 1 -- 172.21.15.70:0/2567656977 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb80ac140 con 0x7fcea8004390 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.955+0000 7fcebcaa1640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 0x7fceb80c8850 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fceac011890 tx=0x7fceac011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.956+0000 7fceb57fa640 1 -- 172.21.15.70:0/2567656977 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac004240 con 0x7fcea8004390 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.956+0000 7fceb57fa640 1 -- 172.21.15.70:0/2567656977 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceac0043a0 con 0x7fcea8004390 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.956+0000 7fceb57fa640 1 -- 172.21.15.70:0/2567656977 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac00fa20 con 0x7fcea8004390 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.956+0000 7fcebed05640 1 -- 172.21.15.70:0/2567656977 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb80ac420 con 0x7fcea8004390 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.956+0000 7fcebed05640 1 -- 172.21.15.70:0/2567656977 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb80ac9e0 con 0x7fcea8004390 2023-10-08T05:56:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.957+0000 7fcebed05640 1 -- 172.21.15.70:0/2567656977 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb80ad030 con 0x7fcea8004390 2023-10-08T05:56:09.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.958+0000 7fceb57fa640 1 -- 172.21.15.70:0/2567656977 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 65) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fceac00fb80 con 0x7fcea8004390 2023-10-08T05:56:09.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.959+0000 7fceb57fa640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fce9c06acc0 0x7fce9c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:09.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.959+0000 7fceb57fa640 1 -- 172.21.15.70:0/2567656977 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6903+0+0 (secure 0 0 0) 0x7fceac089da0 con 0x7fcea8004390 2023-10-08T05:56:09.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.959+0000 7fceb7fff640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fce9c06acc0 0x7fce9c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:09.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.961+0000 7fceb7fff640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fce9c06acc0 0x7fce9c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcea4033000 tx=0x7fcea4009030 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:09.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:09.964+0000 7fceb57fa640 1 -- 172.21.15.70:0/2567656977 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fceac054c10 con 0x7fcea8004390 2023-10-08T05:56:10.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.193+0000 7fcebed05640 1 -- 172.21.15.70:0/2567656977 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fceb8066d80 con 0x7fcea8004390 2023-10-08T05:56:10.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.194+0000 7fceb57fa640 1 -- 172.21.15.70:0/2567656977 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fceac023090 con 0x7fcea8004390 2023-10-08T05:56:10.196 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:10.197 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3983030529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3983030529},{"type":"v1","addr":"172.21.15.181:6836","nonce":3983030529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5873,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1573936162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1573936162},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573936162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1979922916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1979922916},{"type":"v1","addr":"172.21.15.70:6835","nonce":1979922916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:55:25.363123+0000","modified":"2023-10-08T05:55:28.414414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3020884355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3020884355},{"type":"v1","addr":"172.21.15.70:6833","nonce":3020884355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-08T05:56:10.197 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 58 2023-10-08T05:56:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 -- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fce9c06acc0 msgr2=0x7fce9c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fce9c06acc0 0x7fce9c06d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcea4033000 tx=0x7fcea4009030 comp rx=0 tx=0).stop 2023-10-08T05:56:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 -- 172.21.15.70:0/2567656977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 msgr2=0x7fceb80c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 0x7fceb80c8850 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fceac011890 tx=0x7fceac011d50 comp rx=0 tx=0).stop 2023-10-08T05:56:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 -- 172.21.15.70:0/2567656977 shutdown_connections 2023-10-08T05:56:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fce9c06acc0 0x7fce9c06d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fceb80ad1e0 0x7fceb80ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fceb80c2a90 0x7fceb80c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 --2- 172.21.15.70:0/2567656977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcea8004390 0x7fceb80c8850 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 -- 172.21.15.70:0/2567656977 >> 172.21.15.70:0/2567656977 conn(0x7fceb80cf890 msgr2=0x7fceb806aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:10.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 -- 172.21.15.70:0/2567656977 shutdown_connections 2023-10-08T05:56:10.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.197+0000 7fcebed05640 1 -- 172.21.15.70:0/2567656977 wait complete. 2023-10-08T05:56:10.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:10.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.428+0000 7f4fbe439640 1 Processor -- start 2023-10-08T05:56:10.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.428+0000 7f4fbe439640 1 -- start start 2023-10-08T05:56:10.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.430+0000 7f4fbe439640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb8159ac0 0x7f4fb806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.430+0000 7f4fbe439640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fb80ad790 0x7f4fb80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.430+0000 7f4fbe439640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb806dee0 con 0x7f4fb815e330 2023-10-08T05:56:10.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.430+0000 7f4fbe439640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb815d850 con 0x7f4fb80ad790 2023-10-08T05:56:10.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.430+0000 7f4fbe439640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb815db30 con 0x7f4fb8159ac0 2023-10-08T05:56:10.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.430+0000 7f4fbc9d6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4fb815e330 0x7f4fb806c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42248/0 (socket says 172.21.15.70:42248) 2023-10-08T05:56:10.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.430+0000 7f4fbc9d6640 1 -- 172.21.15.70:0/3469397418 learned_addr learned my addr 172.21.15.70:0/3469397418 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb7fff640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb8159ac0 0x7f4fb806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4faf7fe640 1 --2- 172.21.15.70:0/3469397418 >> v2:172.21.15.181:3300/0 conn(0x7f4fb80ad790 0x7f4fb80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999758161 0 0) 0x7f4fb806dee0 con 0x7f4fb815e330 2023-10-08T05:56:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa4003420 con 0x7f4fb815e330 2023-10-08T05:56:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1809499877 0 0) 0x7f4fa4003420 con 0x7f4fb815e330 2023-10-08T05:56:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb8159ac0 msgr2=0x7f4fb806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb8159ac0 0x7f4fb806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 >> v2:172.21.15.181:3300/0 conn(0x7f4fb80ad790 msgr2=0x7f4fb80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 --2- 172.21.15.70:0/3469397418 >> v2:172.21.15.181:3300/0 conn(0x7f4fb80ad790 0x7f4fb80702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb815de10 con 0x7f4fb815e330 2023-10-08T05:56:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4fb0001670 con 0x7f4fb815e330 2023-10-08T05:56:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 >> v1:172.21.15.70:6789/0 conn(0x7f4fb815e330 legacy=0x7f4fb806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4004a50 0x7f4fa4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb8159ac0 0x7f4fa4003660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa4003ba0 0x7f4fa4004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb815d850 con 0x7f4fa4003ba0 2023-10-08T05:56:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb815db30 con 0x7f4fa4004a50 2023-10-08T05:56:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.431+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa400fd50 con 0x7f4fb8159ac0 2023-10-08T05:56:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.432+0000 7f4fbc9d6640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa4003ba0 0x7f4fa4004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.432+0000 7f4faf7fe640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb8159ac0 0x7f4fa4003660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.432+0000 7f4fbc9d6640 1 -- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb8159ac0 msgr2=0x7f4fa4003660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.432+0000 7f4fb7fff640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4004a50 0x7f4fa4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.432+0000 7f4fbc9d6640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb8159ac0 0x7f4fa4003660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.432+0000 7f4fbc9d6640 1 -- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4004a50 msgr2=0x7f4fa4004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.432+0000 7f4fbc9d6640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa4004a50 0x7f4fa4004e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.432+0000 7f4fbc9d6640 1 -- 172.21.15.70:0/3469397418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4003420 con 0x7f4fa4003ba0 2023-10-08T05:56:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.433+0000 7f4fbc9d6640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa4003ba0 0x7f4fa4004020 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c140 tx=0x7f4fa40046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf6c2b1faf81b09 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.433+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0005ad0 con 0x7f4fa4003ba0 2023-10-08T05:56:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.433+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb0005c50 con 0x7f4fa4003ba0 2023-10-08T05:56:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.433+0000 7f4fb77fe640 1 -- 172.21.15.70:0/3469397418 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb002c810 con 0x7f4fa4003ba0 2023-10-08T05:56:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.434+0000 7f4fbe439640 1 -- 172.21.15.70:0/3469397418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa4003ba0 msgr2=0x7f4fa4004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.434+0000 7f4fbe439640 1 --2- 172.21.15.70:0/3469397418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa4003ba0 0x7f4fa4004020 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c140 tx=0x7f4fa40046e0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.434+0000 7f4fbc9d6640 1 -- 172.21.15.70:0/3469397418 reap_dead start 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.434+0000 7f4fbe439640 1 -- 172.21.15.70:0/3469397418 shutdown_connections 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.434+0000 7f4fbe439640 1 -- 172.21.15.70:0/3469397418 >> 172.21.15.70:0/3469397418 conn(0x7f4fb806a380 msgr2=0x7f4fb80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.434+0000 7f4fbe439640 1 -- 172.21.15.70:0/3469397418 shutdown_connections 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.434+0000 7f4fbe439640 1 -- 172.21.15.70:0/3469397418 wait complete. 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.436+0000 7f4fbe439640 1 Processor -- start 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.436+0000 7f4fbe439640 1 -- start start 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.437+0000 7f4fbe439640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa4003ba0 0x7f4fb81eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.437+0000 7f4fbe439640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb815c3a0 0x7f4fb815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.437+0000 7f4fbe439640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815c820 0x7f4fb815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.437+0000 7f4fbe439640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8153530 con 0x7f4fb815c820 2023-10-08T05:56:10.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.437+0000 7f4fbe439640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8153250 con 0x7f4fb815c3a0 2023-10-08T05:56:10.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.437+0000 7f4fbe439640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb815b8c0 con 0x7f4fa4003ba0 2023-10-08T05:56:10.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.437+0000 7f4fbc9d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815c820 0x7f4fb815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.437+0000 7f4fbc9d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815c820 0x7f4fb815a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40950/0 (socket says 172.21.15.70:40950) 2023-10-08T05:56:10.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.437+0000 7f4fbc9d6640 1 -- 172.21.15.70:0/3925782827 learned_addr learned my addr 172.21.15.70:0/3925782827 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:10.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.437+0000 7f4fb7fff640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa4003ba0 0x7f4fb81eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4faf7fe640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb815c3a0 0x7f4fb815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4fbc9d6640 1 -- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa4003ba0 msgr2=0x7f4fb81eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4fbc9d6640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa4003ba0 0x7f4fb81eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4fbc9d6640 1 -- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb815c3a0 msgr2=0x7f4fb815a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4fbc9d6640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb815c3a0 0x7f4fb815a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4fbc9d6640 1 -- 172.21.15.70:0/3925782827 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb815bba0 con 0x7f4fb815c820 2023-10-08T05:56:10.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4fbc9d6640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815c820 0x7f4fb815a560 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c0e0 tx=0x7f4fb00019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:10.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4fb57fa640 1 -- 172.21.15.70:0/3925782827 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb003c040 con 0x7f4fb815c820 2023-10-08T05:56:10.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4fb57fa640 1 -- 172.21.15.70:0/3925782827 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb002cc20 con 0x7f4fb815c820 2023-10-08T05:56:10.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4fb57fa640 1 -- 172.21.15.70:0/3925782827 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0001b80 con 0x7f4fb815c820 2023-10-08T05:56:10.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.438+0000 7f4fbe439640 1 -- 172.21.15.70:0/3925782827 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb815be80 con 0x7f4fb815c820 2023-10-08T05:56:10.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.439+0000 7f4fbe439640 1 -- 172.21.15.70:0/3925782827 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb8153e00 con 0x7f4fb815c820 2023-10-08T05:56:10.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.440+0000 7f4fbe439640 1 -- 172.21.15.70:0/3925782827 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb815c070 con 0x7f4fb815c820 2023-10-08T05:56:10.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.441+0000 7f4fb57fa640 1 -- 172.21.15.70:0/3925782827 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 65) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4fb0032070 con 0x7f4fb815c820 2023-10-08T05:56:10.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.441+0000 7f4fb57fa640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f9806acf0 0x7f4f9806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.442+0000 7f4fb57fa640 1 -- 172.21.15.70:0/3925782827 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f4fb00aa730 con 0x7f4fb815c820 2023-10-08T05:56:10.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.442+0000 7f4fb7fff640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f9806acf0 0x7f4f9806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.443+0000 7f4fb7fff640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f9806acf0 0x7f4f9806d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4fb815b480 tx=0x7f4fa800d040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:10.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.446+0000 7f4fb57fa640 1 -- 172.21.15.70:0/3925782827 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fb0075670 con 0x7f4fb815c820 2023-10-08T05:56:10.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.637+0000 7f4fbe439640 1 -- 172.21.15.70:0/3925782827 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fb8067490 con 0x7f4fb815c820 2023-10-08T05:56:10.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.640+0000 7f4fb57fa640 1 -- 172.21.15.70:0/3925782827 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+18025 (secure 0 0 0) 0x7f4fb0078c90 con 0x7f4fb815c820 2023-10-08T05:56:10.641 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:10.641 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":137,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:09.758097+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:55:23.150004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-08T05:55:24.358581+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3688966653":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1080150247":"2023-10-09T05:56:08.750606+0000","172.21.15.181:6833/1786188476":"2023-10-09T05:56:08.750606+0000","172.21.15.181:6832/1786188476":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1214793362":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/497155213":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/67726193":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1353868644":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/74854201":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/2395612444":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1556900853":"2023-10-09T05:55:39.473892+0000","172.21.15.181:6832/3142216408":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/4173673684":"2023-10-09T05:55:39.473892+0000","172.21.15.181:6833/3142216408":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/1464588158":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/791649123":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/2668737378":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/3044665941":"2023-10-09T05:55:39.473892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:10.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 -- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f9806acf0 msgr2=0x7f4f9806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f9806acf0 0x7f4f9806d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4fb815b480 tx=0x7f4fa800d040 comp rx=0 tx=0).stop 2023-10-08T05:56:10.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 -- 172.21.15.70:0/3925782827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815c820 msgr2=0x7f4fb815a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815c820 0x7f4fb815a560 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f4fb000c0e0 tx=0x7f4fb00019a0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 -- 172.21.15.70:0/3925782827 shutdown_connections 2023-10-08T05:56:10.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f9806acf0 0x7f4f9806d1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815c820 0x7f4fb815a560 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb815c3a0 0x7f4fb815a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 --2- 172.21.15.70:0/3925782827 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fa4003ba0 0x7f4fb81eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 -- 172.21.15.70:0/3925782827 >> 172.21.15.70:0/3925782827 conn(0x7f4fb806a380 msgr2=0x7f4fb806d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:10.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 -- 172.21.15.70:0/3925782827 shutdown_connections 2023-10-08T05:56:10.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.643+0000 7f4fbe439640 1 -- 172.21.15.70:0/3925782827 wait complete. 2023-10-08T05:56:10.662 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T05:56:10.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:56:10.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:56:10.720 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:56:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.864+0000 7f2ecc7a9640 1 Processor -- start 2023-10-08T05:56:10.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.864+0000 7f2ecc7a9640 1 -- start start 2023-10-08T05:56:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.866+0000 7f2ecc7a9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2ec40c90b0 0x7f2ec40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.866+0000 7f2ecc7a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ec40cb3b0 0x7f2ec415e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.866+0000 7f2ecc7a9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec415b2b0 con 0x7f2ec415ab40 2023-10-08T05:56:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.866+0000 7f2ecc7a9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec415bf30 con 0x7f2ec40cb3b0 2023-10-08T05:56:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.866+0000 7f2ecc7a9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec40cb820 con 0x7f2ec40c90b0 2023-10-08T05:56:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.866+0000 7f2ecad46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ec40cb3b0 0x7f2ec415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.866+0000 7f2ecad46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ec40cb3b0 0x7f2ec415e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40960/0 (socket says 172.21.15.70:40960) 2023-10-08T05:56:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.866+0000 7f2ecad46640 1 -- 172.21.15.70:0/817032938 learned_addr learned my addr 172.21.15.70:0/817032938 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.866+0000 7f2ec9d44640 1 --2- 172.21.15.70:0/817032938 >> v2:172.21.15.181:3300/0 conn(0x7f2ec40c90b0 0x7f2ec40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.867+0000 7f2ecad46640 1 -- 172.21.15.70:0/817032938 >> v2:172.21.15.181:3300/0 conn(0x7f2ec40c90b0 msgr2=0x7f2ec40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.867+0000 7f2ecad46640 1 --2- 172.21.15.70:0/817032938 >> v2:172.21.15.181:3300/0 conn(0x7f2ec40c90b0 0x7f2ec40cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.867+0000 7f2ecad46640 1 -- 172.21.15.70:0/817032938 >> v1:172.21.15.181:6789/0 conn(0x7f2ec415ab40 legacy=0x7f2ec40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.867+0000 7f2ecad46640 1 -- 172.21.15.70:0/817032938 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec415b590 con 0x7f2ec40cb3b0 2023-10-08T05:56:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.867+0000 7f2ec9d44640 1 --2- 172.21.15.70:0/817032938 >> v2:172.21.15.181:3300/0 conn(0x7f2ec40c90b0 0x7f2ec40cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.868+0000 7f2ecad46640 1 --2- 172.21.15.70:0/817032938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec40cb3b0 0x7f2ec415e790 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2ec406e560 tx=0x7f2ebc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad851180406e4e2d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.868+0000 7f2ec9543640 1 -- 172.21.15.70:0/817032938 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc02ca80 con 0x7f2ec40cb3b0 2023-10-08T05:56:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.868+0000 7f2ec9543640 1 -- 172.21.15.70:0/817032938 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc02cbe0 con 0x7f2ec40cb3b0 2023-10-08T05:56:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.868+0000 7f2ec9543640 1 -- 172.21.15.70:0/817032938 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0343f0 con 0x7f2ec40cb3b0 2023-10-08T05:56:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.869+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/817032938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec40cb3b0 msgr2=0x7f2ec415e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.869+0000 7f2ecc7a9640 1 --2- 172.21.15.70:0/817032938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec40cb3b0 0x7f2ec415e790 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f2ec406e560 tx=0x7f2ebc02bee0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.869+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/817032938 shutdown_connections 2023-10-08T05:56:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.869+0000 7f2ecc7a9640 1 --2- 172.21.15.70:0/817032938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec40cb3b0 0x7f2ec415e790 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.869+0000 7f2ecc7a9640 1 --2- 172.21.15.70:0/817032938 >> v2:172.21.15.181:3300/0 conn(0x7f2ec40c90b0 0x7f2ec40cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.869+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/817032938 >> 172.21.15.70:0/817032938 conn(0x7f2ec406a380 msgr2=0x7f2ec40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:10.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.869+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/817032938 shutdown_connections 2023-10-08T05:56:10.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.869+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/817032938 wait complete. 2023-10-08T05:56:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.871+0000 7f2ecc7a9640 1 Processor -- start 2023-10-08T05:56:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.871+0000 7f2ecc7a9640 1 -- start start 2023-10-08T05:56:10.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.872+0000 7f2ecc7a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ec40c90b0 0x7f2ec4159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.872+0000 7f2ecc7a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ec40cb3b0 0x7f2ec413d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.872+0000 7f2ecc7a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec415ab40 0x7f2ec40c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.872+0000 7f2ecc7a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec415a400 con 0x7f2ec415ab40 2023-10-08T05:56:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.872+0000 7f2ecc7a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec4159a30 con 0x7f2ec40cb3b0 2023-10-08T05:56:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.872+0000 7f2ecc7a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec4158b40 con 0x7f2ec40c90b0 2023-10-08T05:56:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.872+0000 7f2ecad46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec415ab40 0x7f2ec40c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.872+0000 7f2ecad46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec415ab40 0x7f2ec40c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40974/0 (socket says 172.21.15.70:40974) 2023-10-08T05:56:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.872+0000 7f2ecad46640 1 -- 172.21.15.70:0/106914769 learned_addr learned my addr 172.21.15.70:0/106914769 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.872+0000 7f2eca545640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ec40c90b0 0x7f2ec4159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.873+0000 7f2ec9d44640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ec40cb3b0 0x7f2ec413d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.873+0000 7f2ecad46640 1 -- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ec40c90b0 msgr2=0x7f2ec4159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.873+0000 7f2ecad46640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ec40c90b0 0x7f2ec4159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.873+0000 7f2ecad46640 1 -- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ec40cb3b0 msgr2=0x7f2ec413d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.873+0000 7f2ecad46640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ec40cb3b0 0x7f2ec413d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.873+0000 7f2ecad46640 1 -- 172.21.15.70:0/106914769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc009b30 con 0x7f2ec415ab40 2023-10-08T05:56:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.873+0000 7f2ecad46640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec415ab40 0x7f2ec40c6940 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc036040 tx=0x7f2ebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.874+0000 7f2ebaffd640 1 -- 172.21.15.70:0/106914769 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc03e450 con 0x7f2ec415ab40 2023-10-08T05:56:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.874+0000 7f2ebaffd640 1 -- 172.21.15.70:0/106914769 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc03e5d0 con 0x7f2ec415ab40 2023-10-08T05:56:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.874+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/106914769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec4158e20 con 0x7f2ec415ab40 2023-10-08T05:56:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.874+0000 7f2ebaffd640 1 -- 172.21.15.70:0/106914769 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc03e980 con 0x7f2ec415ab40 2023-10-08T05:56:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.874+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/106914769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec41593e0 con 0x7f2ec415ab40 2023-10-08T05:56:10.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.875+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/106914769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec4159be0 con 0x7f2ec415ab40 2023-10-08T05:56:10.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.876+0000 7f2ebaffd640 1 -- 172.21.15.70:0/106914769 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 65) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f2ebc02cd50 con 0x7f2ec415ab40 2023-10-08T05:56:10.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.877+0000 7f2ebaffd640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2ea806acf0 0x7f2ea806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:10.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.877+0000 7f2ebaffd640 1 -- 172.21.15.70:0/106914769 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f2ebc0ac590 con 0x7f2ec415ab40 2023-10-08T05:56:10.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.877+0000 7f2eca545640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2ea806acf0 0x7f2ea806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:10.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.878+0000 7f2eca545640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2ea806acf0 0x7f2ea806d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40ae540 tx=0x7f2ec0007e30 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:10.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:10.881+0000 7f2ebaffd640 1 -- 172.21.15.70:0/106914769 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ec4159be0 con 0x7f2ec415ab40 2023-10-08T05:56:11.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.107+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/106914769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ec4159ba0 con 0x7f2ec415ab40 2023-10-08T05:56:11.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.108+0000 7f2ebaffd640 1 -- 172.21.15.70:0/106914769 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f2ebc0774b0 con 0x7f2ec415ab40 2023-10-08T05:56:11.108 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:11.108 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3983030529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3983030529},{"type":"v1","addr":"172.21.15.181:6836","nonce":3983030529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5873,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1573936162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1573936162},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573936162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1979922916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1979922916},{"type":"v1","addr":"172.21.15.70:6835","nonce":1979922916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:55:25.363123+0000","modified":"2023-10-08T05:55:28.414414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3020884355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3020884355},{"type":"v1","addr":"172.21.15.70:6833","nonce":3020884355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-08T05:56:11.109 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 58 2023-10-08T05:56:11.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.110+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2ea806acf0 msgr2=0x7f2ea806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.110+0000 7f2ecc7a9640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2ea806acf0 0x7f2ea806d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40ae540 tx=0x7f2ec0007e30 comp rx=0 tx=0).stop 2023-10-08T05:56:11.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.110+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/106914769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec415ab40 msgr2=0x7f2ec40c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.110+0000 7f2ecc7a9640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec415ab40 0x7f2ec40c6940 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc036040 tx=0x7f2ebc004a40 comp rx=0 tx=0).stop 2023-10-08T05:56:11.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.111+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/106914769 shutdown_connections 2023-10-08T05:56:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.111+0000 7f2ecc7a9640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2ea806acf0 0x7f2ea806d1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.111+0000 7f2ecc7a9640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ec415ab40 0x7f2ec40c6940 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.111+0000 7f2ecc7a9640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ec40cb3b0 0x7f2ec413d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.111+0000 7f2ecc7a9640 1 --2- 172.21.15.70:0/106914769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2ec40c90b0 0x7f2ec4159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.111+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/106914769 >> 172.21.15.70:0/106914769 conn(0x7f2ec406a380 msgr2=0x7f2ec406ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.111+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/106914769 shutdown_connections 2023-10-08T05:56:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.111+0000 7f2ecc7a9640 1 -- 172.21.15.70:0/106914769 wait complete. 2023-10-08T05:56:11.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:56:11.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.387+0000 7f4d7e06b640 1 Processor -- start 2023-10-08T05:56:11.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.387+0000 7f4d7e06b640 1 -- start start 2023-10-08T05:56:11.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.388+0000 7f4d7e06b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d780c90b0 0x7f4d780cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.388+0000 7f4d7e06b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4d780cb3b0 0x7f4d7815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.388+0000 7f4d7e06b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7815b2b0 con 0x7f4d7815ab40 2023-10-08T05:56:11.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.388+0000 7f4d7e06b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7815bf30 con 0x7f4d780cb3b0 2023-10-08T05:56:11.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.388+0000 7f4d7e06b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d780cb840 con 0x7f4d780c90b0 2023-10-08T05:56:11.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.389+0000 7f4d77fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4d780cb3b0 0x7f4d7815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.389+0000 7f4d777fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4d7815ab40 0x7f4d780ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42254/0 (socket says 172.21.15.70:42254) 2023-10-08T05:56:11.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.389+0000 7f4d777fe640 1 -- 172.21.15.70:0/2368375672 learned_addr learned my addr 172.21.15.70:0/2368375672 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:11.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.389+0000 7f4d77fff640 1 -- 172.21.15.70:0/2368375672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d780c90b0 msgr2=0x7f4d780cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:11.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.389+0000 7f4d77fff640 1 --2- 172.21.15.70:0/2368375672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d780c90b0 0x7f4d780cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.389+0000 7f4d77fff640 1 -- 172.21.15.70:0/2368375672 >> v1:172.21.15.70:6789/0 conn(0x7f4d7815ab40 legacy=0x7f4d780ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.389+0000 7f4d77fff640 1 -- 172.21.15.70:0/2368375672 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7815b590 con 0x7f4d780cb3b0 2023-10-08T05:56:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.390+0000 7f4d77fff640 1 --2- 172.21.15.70:0/2368375672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d780cb3b0 0x7f4d7815e790 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009d90 tx=0x7f4d7002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e73cc7d6f353cf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d76ffd640 1 -- 172.21.15.70:0/2368375672 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7002cc60 con 0x7f4d780cb3b0 2023-10-08T05:56:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d76ffd640 1 -- 172.21.15.70:0/2368375672 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7002cdc0 con 0x7f4d780cb3b0 2023-10-08T05:56:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d76ffd640 1 -- 172.21.15.70:0/2368375672 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70034660 con 0x7f4d780cb3b0 2023-10-08T05:56:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2368375672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d780cb3b0 msgr2=0x7f4d7815e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d7e06b640 1 --2- 172.21.15.70:0/2368375672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d780cb3b0 0x7f4d7815e790 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009d90 tx=0x7f4d7002beb0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2368375672 shutdown_connections 2023-10-08T05:56:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d7e06b640 1 --2- 172.21.15.70:0/2368375672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d780cb3b0 0x7f4d7815e790 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d7e06b640 1 --2- 172.21.15.70:0/2368375672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d780c90b0 0x7f4d780cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2368375672 >> 172.21.15.70:0/2368375672 conn(0x7f4d7806a380 msgr2=0x7f4d780b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2368375672 shutdown_connections 2023-10-08T05:56:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.391+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2368375672 wait complete. 2023-10-08T05:56:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.393+0000 7f4d7e06b640 1 Processor -- start 2023-10-08T05:56:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.393+0000 7f4d7e06b640 1 -- start start 2023-10-08T05:56:11.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.394+0000 7f4d7e06b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d780c90b0 0x7f4d78150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.394+0000 7f4d7e06b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d780cb3b0 0x7f4d7815d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.394+0000 7f4d7e06b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d7815ab40 0x7f4d7815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.394+0000 7f4d7e06b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d78151ee0 con 0x7f4d7815ab40 2023-10-08T05:56:11.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.394+0000 7f4d7e06b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d781521c0 con 0x7f4d780c90b0 2023-10-08T05:56:11.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.394+0000 7f4d7e06b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d781524a0 con 0x7f4d780cb3b0 2023-10-08T05:56:11.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.394+0000 7f4d77fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d7815ab40 0x7f4d7815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d77fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d7815ab40 0x7f4d7815d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40992/0 (socket says 172.21.15.70:40992) 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d77fff640 1 -- 172.21.15.70:0/2500497990 learned_addr learned my addr 172.21.15.70:0/2500497990 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d6ffff640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d780cb3b0 0x7f4d7815d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d777fe640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d780c90b0 0x7f4d78150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d77fff640 1 -- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d780cb3b0 msgr2=0x7f4d7815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d77fff640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d780cb3b0 0x7f4d7815d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d77fff640 1 -- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d780c90b0 msgr2=0x7f4d78150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d77fff640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d780c90b0 0x7f4d78150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d77fff640 1 -- 172.21.15.70:0/2500497990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d70009a70 con 0x7f4d7815ab40 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d77fff640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d7815ab40 0x7f4d7815d8d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009d90 tx=0x7f4d70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.395+0000 7f4d74ff9640 1 -- 172.21.15.70:0/2500497990 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70040450 con 0x7f4d7815ab40 2023-10-08T05:56:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.396+0000 7f4d74ff9640 1 -- 172.21.15.70:0/2500497990 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d700405d0 con 0x7f4d7815ab40 2023-10-08T05:56:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.396+0000 7f4d74ff9640 1 -- 172.21.15.70:0/2500497990 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70040980 con 0x7f4d7815ab40 2023-10-08T05:56:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.396+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2500497990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d78152780 con 0x7f4d7815ab40 2023-10-08T05:56:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.396+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2500497990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d78159020 con 0x7f4d7815ab40 2023-10-08T05:56:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.397+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2500497990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d781529a0 con 0x7f4d7815ab40 2023-10-08T05:56:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.398+0000 7f4d74ff9640 1 -- 172.21.15.70:0/2500497990 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 65) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f4d70040ae0 con 0x7f4d7815ab40 2023-10-08T05:56:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.398+0000 7f4d74ff9640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4d5806acc0 0x7f4d5806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.399+0000 7f4d74ff9640 1 -- 172.21.15.70:0/2500497990 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f4d700ac2a0 con 0x7f4d7815ab40 2023-10-08T05:56:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.399+0000 7f4d777fe640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4d5806acc0 0x7f4d5806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.400+0000 7f4d777fe640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4d5806acc0 0x7f4d5806d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4d68005880 tx=0x7f4d68009040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:11.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.403+0000 7f4d74ff9640 1 -- 172.21.15.70:0/2500497990 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d70077260 con 0x7f4d7815ab40 2023-10-08T05:56:11.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.629+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2500497990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d78152020 con 0x7f4d7815ab40 2023-10-08T05:56:11.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.630+0000 7f4d74ff9640 1 -- 172.21.15.70:0/2500497990 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f4d78152020 con 0x7f4d7815ab40 2023-10-08T05:56:11.631 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:11.631 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":58,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3983030529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3983030529},{"type":"v1","addr":"172.21.15.181:6836","nonce":3983030529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5873,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1573936162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1573936162},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573936162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1979922916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1979922916},{"type":"v1","addr":"172.21.15.70:6835","nonce":1979922916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:55:25.363123+0000","modified":"2023-10-08T05:55:28.414414+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/3020884355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3020884355},{"type":"v1","addr":"172.21.15.70:6833","nonce":3020884355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-10-08T05:56:11.632 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 58 2023-10-08T05:56:11.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4d5806acc0 msgr2=0x7f4d5806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4d5806acc0 0x7f4d5806d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4d68005880 tx=0x7f4d68009040 comp rx=0 tx=0).stop 2023-10-08T05:56:11.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2500497990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d7815ab40 msgr2=0x7f4d7815d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d7815ab40 0x7f4d7815d8d0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009d90 tx=0x7f4d70004a40 comp rx=0 tx=0).stop 2023-10-08T05:56:11.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2500497990 shutdown_connections 2023-10-08T05:56:11.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4d5806acc0 0x7f4d5806d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d7815ab40 0x7f4d7815d8d0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4d780cb3b0 0x7f4d7815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 --2- 172.21.15.70:0/2500497990 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4d780c90b0 0x7f4d78150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2500497990 >> 172.21.15.70:0/2500497990 conn(0x7f4d7806a380 msgr2=0x7f4d780caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:11.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2500497990 shutdown_connections 2023-10-08T05:56:11.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.633+0000 7f4d7e06b640 1 -- 172.21.15.70:0/2500497990 wait complete. 2023-10-08T05:56:11.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.858+0000 7f8795602640 1 Processor -- start 2023-10-08T05:56:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.858+0000 7f8795602640 1 -- start start 2023-10-08T05:56:11.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.860+0000 7f8795602640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87900ad790 0x7f87900702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.860+0000 7f8795602640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f879015e330 0x7f8790070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.860+0000 7f8795602640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879006dee0 con 0x7f8790159ac0 2023-10-08T05:56:11.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.860+0000 7f8795602640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879006dcf0 con 0x7f87900ad790 2023-10-08T05:56:11.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.860+0000 7f8795602640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879015d850 con 0x7f879015e330 2023-10-08T05:56:11.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.860+0000 7f878f7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f879015e330 0x7f8790070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.860+0000 7f878f7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f879015e330 0x7f8790070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:37138/0 (socket says 172.21.15.70:37138) 2023-10-08T05:56:11.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.860+0000 7f878f7fe640 1 -- 172.21.15.70:0/820581379 learned_addr learned my addr 172.21.15.70:0/820581379 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:11.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.861+0000 7f878e7fc640 1 --2- 172.21.15.70:0/820581379 >> v2:172.21.15.181:3300/0 conn(0x7f87900ad790 0x7f87900702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.861+0000 7f878f7fe640 1 -- 172.21.15.70:0/820581379 >> v2:172.21.15.181:3300/0 conn(0x7f87900ad790 msgr2=0x7f87900702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:11.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.861+0000 7f878f7fe640 1 --2- 172.21.15.70:0/820581379 >> v2:172.21.15.181:3300/0 conn(0x7f87900ad790 0x7f87900702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.861+0000 7f878f7fe640 1 -- 172.21.15.70:0/820581379 >> v1:172.21.15.181:6789/0 conn(0x7f8790159ac0 legacy=0x7f879006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.861+0000 7f878f7fe640 1 -- 172.21.15.70:0/820581379 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879015db30 con 0x7f879015e330 2023-10-08T05:56:11.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.862+0000 7f878f7fe640 1 --2- 172.21.15.70:0/820581379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f879015e330 0x7f8790070810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8788009e50 tx=0x7f878802e170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67dcaefaf356b856 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:11.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.863+0000 7f878dffb640 1 -- 172.21.15.70:0/820581379 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878802ed10 con 0x7f879015e330 2023-10-08T05:56:11.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.863+0000 7f878dffb640 1 -- 172.21.15.70:0/820581379 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878802ee70 con 0x7f879015e330 2023-10-08T05:56:11.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.863+0000 7f878dffb640 1 -- 172.21.15.70:0/820581379 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87880365f0 con 0x7f879015e330 2023-10-08T05:56:11.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.863+0000 7f8795602640 1 -- 172.21.15.70:0/820581379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f879015e330 msgr2=0x7f8790070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.863+0000 7f8795602640 1 --2- 172.21.15.70:0/820581379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f879015e330 0x7f8790070810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8788009e50 tx=0x7f878802e170 comp rx=0 tx=0).stop 2023-10-08T05:56:11.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.863+0000 7f8795602640 1 -- 172.21.15.70:0/820581379 shutdown_connections 2023-10-08T05:56:11.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.863+0000 7f8795602640 1 --2- 172.21.15.70:0/820581379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f879015e330 0x7f8790070810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.863+0000 7f8795602640 1 --2- 172.21.15.70:0/820581379 >> v2:172.21.15.181:3300/0 conn(0x7f87900ad790 0x7f87900702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.863+0000 7f8795602640 1 -- 172.21.15.70:0/820581379 >> 172.21.15.70:0/820581379 conn(0x7f879006a380 msgr2=0x7f87900b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:11.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.864+0000 7f8795602640 1 -- 172.21.15.70:0/820581379 shutdown_connections 2023-10-08T05:56:11.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.864+0000 7f8795602640 1 -- 172.21.15.70:0/820581379 wait complete. 2023-10-08T05:56:11.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.865+0000 7f8795602640 1 Processor -- start 2023-10-08T05:56:11.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.865+0000 7f8795602640 1 -- start start 2023-10-08T05:56:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.866+0000 7f8795602640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900ad790 0x7f87900cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.866+0000 7f8795602640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8790159ac0 0x7f87900ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.866+0000 7f8795602640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f879015e330 0x7f879014f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.866+0000 7f8795602640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8790150470 con 0x7f87900ad790 2023-10-08T05:56:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.866+0000 7f8795602640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8790150750 con 0x7f879015e330 2023-10-08T05:56:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.866+0000 7f8795602640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8790150a30 con 0x7f8790159ac0 2023-10-08T05:56:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.866+0000 7f878effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900ad790 0x7f87900cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900ad790 0x7f87900cc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41006/0 (socket says 172.21.15.70:41006) 2023-10-08T05:56:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878effd640 1 -- 172.21.15.70:0/1097223172 learned_addr learned my addr 172.21.15.70:0/1097223172 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878f7fe640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f879015e330 0x7f879014f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878e7fc640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8790159ac0 0x7f87900ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878effd640 1 -- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8790159ac0 msgr2=0x7f87900ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878effd640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8790159ac0 0x7f87900ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878effd640 1 -- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f879015e330 msgr2=0x7f879014f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878effd640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f879015e330 0x7f879014f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878effd640 1 -- 172.21.15.70:0/1097223172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788009b30 con 0x7f87900ad790 2023-10-08T05:56:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878e7fc640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8790159ac0 0x7f87900ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.867+0000 7f878effd640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900ad790 0x7f87900cc890 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f878000c400 tx=0x7f878000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.868+0000 7f87777fe640 1 -- 172.21.15.70:0/1097223172 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87800052d0 con 0x7f87900ad790 2023-10-08T05:56:11.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.868+0000 7f87777fe640 1 -- 172.21.15.70:0/1097223172 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8780005450 con 0x7f87900ad790 2023-10-08T05:56:11.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.868+0000 7f8795602640 1 -- 172.21.15.70:0/1097223172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8790153cb0 con 0x7f87900ad790 2023-10-08T05:56:11.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.868+0000 7f87777fe640 1 -- 172.21.15.70:0/1097223172 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8780007260 con 0x7f87900ad790 2023-10-08T05:56:11.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.868+0000 7f8795602640 1 -- 172.21.15.70:0/1097223172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87901477a0 con 0x7f87900ad790 2023-10-08T05:56:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.869+0000 7f8795602640 1 -- 172.21.15.70:0/1097223172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8790150c70 con 0x7f87900ad790 2023-10-08T05:56:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.870+0000 7f87777fe640 1 -- 172.21.15.70:0/1097223172 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 66) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f87800073c0 con 0x7f87900ad790 2023-10-08T05:56:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.871+0000 7f87777fe640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f877c06af00 0x7f877c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.871+0000 7f87777fe640 1 -- 172.21.15.70:0/1097223172 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f8780088ff0 con 0x7f87900ad790 2023-10-08T05:56:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.871+0000 7f878e7fc640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f877c06af00 0x7f877c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.872+0000 7f878e7fc640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f877c06af00 0x7f877c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8790155c00 tx=0x7f8784008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:11.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:11.875+0000 7f87777fe640 1 -- 172.21.15.70:0/1097223172 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87800bbca0 con 0x7f87900ad790 2023-10-08T05:56:12.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.062+0000 7f8795602640 1 -- 172.21.15.70:0/1097223172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8790067490 con 0x7f87900ad790 2023-10-08T05:56:12.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.065+0000 7f87777fe640 1 -- 172.21.15.70:0/1097223172 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+18025 (secure 0 0 0) 0x7f8790150c70 con 0x7f87900ad790 2023-10-08T05:56:12.065 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:12.065 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":137,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:09.758097+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:55:23.150004+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-08T05:55:24.358581+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3688966653":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1080150247":"2023-10-09T05:56:08.750606+0000","172.21.15.181:6833/1786188476":"2023-10-09T05:56:08.750606+0000","172.21.15.181:6832/1786188476":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1214793362":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/497155213":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/67726193":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1353868644":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/74854201":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/2395612444":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1556900853":"2023-10-09T05:55:39.473892+0000","172.21.15.181:6832/3142216408":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/4173673684":"2023-10-09T05:55:39.473892+0000","172.21.15.181:6833/3142216408":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/1464588158":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/791649123":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/2668737378":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/3044665941":"2023-10-09T05:55:39.473892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:12.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 -- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f877c06af00 msgr2=0x7f877c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f877c06af00 0x7f877c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8790155c00 tx=0x7f8784008040 comp rx=0 tx=0).stop 2023-10-08T05:56:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 -- 172.21.15.70:0/1097223172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900ad790 msgr2=0x7f87900cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900ad790 0x7f87900cc890 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f878000c400 tx=0x7f878000c8c0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 -- 172.21.15.70:0/1097223172 shutdown_connections 2023-10-08T05:56:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f877c06af00 0x7f877c06d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f879015e330 0x7f879014f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8790159ac0 0x7f87900ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 --2- 172.21.15.70:0/1097223172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87900ad790 0x7f87900cc890 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 -- 172.21.15.70:0/1097223172 >> 172.21.15.70:0/1097223172 conn(0x7f879006a380 msgr2=0x7f87900701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 -- 172.21.15.70:0/1097223172 shutdown_connections 2023-10-08T05:56:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.068+0000 7f8795602640 1 -- 172.21.15.70:0/1097223172 wait complete. 2023-10-08T05:56:12.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T05:56:12.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.305+0000 7f70689f5640 1 Processor -- start 2023-10-08T05:56:12.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.305+0000 7f70689f5640 1 -- start start 2023-10-08T05:56:12.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.306+0000 7f70689f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70640afbe0 0x7f70640b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.306+0000 7f70689f5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f706406dfc0 0x7f7064070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.306+0000 7f70689f5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70640b2620 con 0x7f7064071070 2023-10-08T05:56:12.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.306+0000 7f70689f5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70640b2060 con 0x7f706406dfc0 2023-10-08T05:56:12.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.306+0000 7f70689f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70640b2340 con 0x7f70640afbe0 2023-10-08T05:56:12.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.307+0000 7f706259c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70640afbe0 0x7f70640b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.307+0000 7f705bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f706406dfc0 0x7f7064070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.307+0000 7f7062d9d640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f7064071070 0x7f70640cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:57046/0 (socket says 172.21.15.70:57046) 2023-10-08T05:56:12.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.307+0000 7f7062d9d640 1 -- 172.21.15.70:0/1969231680 learned_addr learned my addr 172.21.15.70:0/1969231680 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:12.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.308+0000 7f705bfff640 1 -- 172.21.15.70:0/1969231680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70640afbe0 msgr2=0x7f70640b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:12.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.308+0000 7f705bfff640 1 --2- 172.21.15.70:0/1969231680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70640afbe0 0x7f70640b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.308+0000 7f705bfff640 1 -- 172.21.15.70:0/1969231680 >> v1:172.21.15.181:6789/0 conn(0x7f7064071070 legacy=0x7f70640cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.308+0000 7f705bfff640 1 -- 172.21.15.70:0/1969231680 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7064070350 con 0x7f706406dfc0 2023-10-08T05:56:12.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.309+0000 7f705bfff640 1 --2- 172.21.15.70:0/1969231680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f706406dfc0 0x7f7064070b30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f704c0097a0 tx=0x7f704c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ac94bc5602ecfd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:12.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.310+0000 7f7061d9b640 1 -- 172.21.15.70:0/1969231680 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c02d040 con 0x7f706406dfc0 2023-10-08T05:56:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.310+0000 7f7061d9b640 1 -- 172.21.15.70:0/1969231680 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c031810 con 0x7f706406dfc0 2023-10-08T05:56:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.310+0000 7f7061d9b640 1 -- 172.21.15.70:0/1969231680 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c031ba0 con 0x7f706406dfc0 2023-10-08T05:56:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.310+0000 7f70689f5640 1 -- 172.21.15.70:0/1969231680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f706406dfc0 msgr2=0x7f7064070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.310+0000 7f70689f5640 1 --2- 172.21.15.70:0/1969231680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f706406dfc0 0x7f7064070b30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f704c0097a0 tx=0x7f704c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.311+0000 7f70689f5640 1 -- 172.21.15.70:0/1969231680 shutdown_connections 2023-10-08T05:56:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.311+0000 7f70689f5640 1 --2- 172.21.15.70:0/1969231680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f706406dfc0 0x7f7064070b30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.311+0000 7f70689f5640 1 --2- 172.21.15.70:0/1969231680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70640afbe0 0x7f70640b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.311+0000 7f70689f5640 1 -- 172.21.15.70:0/1969231680 >> 172.21.15.70:0/1969231680 conn(0x7f706406a5d0 msgr2=0x7f70640b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.311+0000 7f70689f5640 1 -- 172.21.15.70:0/1969231680 shutdown_connections 2023-10-08T05:56:12.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.311+0000 7f70689f5640 1 -- 172.21.15.70:0/1969231680 wait complete. 2023-10-08T05:56:12.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.312+0000 7f70689f5640 1 Processor -- start 2023-10-08T05:56:12.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.313+0000 7f70689f5640 1 -- start start 2023-10-08T05:56:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f70689f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f706406dfc0 0x7f706414cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f70689f5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7064071070 0x7f70640cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f70689f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70640afbe0 0x7f70640ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f70689f5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706414d0e0 con 0x7f7064071070 2023-10-08T05:56:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f70689f5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70640cd090 con 0x7f70640afbe0 2023-10-08T05:56:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f70689f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70640cd370 con 0x7f706406dfc0 2023-10-08T05:56:12.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f705bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7064071070 0x7f70640cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f705bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7064071070 0x7f70640cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41010/0 (socket says 172.21.15.70:41010) 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f705bfff640 1 -- 172.21.15.70:0/1045853209 learned_addr learned my addr 172.21.15.70:0/1045853209 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f7062d9d640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70640afbe0 0x7f70640ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f706259c640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f706406dfc0 0x7f706414cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f705bfff640 1 -- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f706406dfc0 msgr2=0x7f706414cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f705bfff640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f706406dfc0 0x7f706414cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f705bfff640 1 -- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70640afbe0 msgr2=0x7f70640ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.314+0000 7f705bfff640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70640afbe0 0x7f70640ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.315+0000 7f705bfff640 1 -- 172.21.15.70:0/1045853209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c009400 con 0x7f7064071070 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.315+0000 7f706259c640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f706406dfc0 0x7f706414cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:12.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.315+0000 7f705bfff640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7064071070 0x7f70640cc640 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f704c000c00 tx=0x7f704c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.315+0000 7f705affd640 1 -- 172.21.15.70:0/1045853209 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c006e10 con 0x7f7064071070 2023-10-08T05:56:12.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.315+0000 7f705affd640 1 -- 172.21.15.70:0/1045853209 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c03ad60 con 0x7f7064071070 2023-10-08T05:56:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.315+0000 7f705affd640 1 -- 172.21.15.70:0/1045853209 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c044630 con 0x7f7064071070 2023-10-08T05:56:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.315+0000 7f70689f5640 1 -- 172.21.15.70:0/1045853209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70640cd650 con 0x7f7064071070 2023-10-08T05:56:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.315+0000 7f70689f5640 1 -- 172.21.15.70:0/1045853209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70640c2250 con 0x7f7064071070 2023-10-08T05:56:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.317+0000 7f70689f5640 1 -- 172.21.15.70:0/1045853209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706404e160 con 0x7f7064071070 2023-10-08T05:56:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.317+0000 7f705affd640 1 -- 172.21.15.70:0/1045853209 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 66) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f704c04d450 con 0x7f7064071070 2023-10-08T05:56:12.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.318+0000 7f705affd640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f704806af00 0x7f704806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.318+0000 7f705affd640 1 -- 172.21.15.70:0/1045853209 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f704c0abed0 con 0x7f7064071070 2023-10-08T05:56:12.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.319+0000 7f706259c640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f704806af00 0x7f704806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.320+0000 7f706259c640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f704806af00 0x7f704806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7054009ac0 tx=0x7f7054008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:12.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.323+0000 7f705affd640 1 -- 172.21.15.70:0/1045853209 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f704c0768d0 con 0x7f7064071070 2023-10-08T05:56:12.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.549+0000 7f70689f5640 1 -- 172.21.15.70:0/1045853209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f70640c5280 con 0x7f7064071070 2023-10-08T05:56:12.550 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T05:56:12.550+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T05:56:12.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.554+0000 7f705affd640 1 -- 172.21.15.70:0/1045853209 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7f704c035020 con 0x7f7064071070 2023-10-08T05:56:12.554 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T05:56:12.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.557+0000 7f70689f5640 1 -- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f704806af00 msgr2=0x7f704806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.557+0000 7f70689f5640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f704806af00 0x7f704806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7054009ac0 tx=0x7f7054008040 comp rx=0 tx=0).stop 2023-10-08T05:56:12.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.557+0000 7f70689f5640 1 -- 172.21.15.70:0/1045853209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7064071070 msgr2=0x7f70640cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.557+0000 7f70689f5640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7064071070 0x7f70640cc640 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f704c000c00 tx=0x7f704c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.558+0000 7f70689f5640 1 -- 172.21.15.70:0/1045853209 shutdown_connections 2023-10-08T05:56:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.558+0000 7f70689f5640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f704806af00 0x7f704806d3c0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7054009ac0 tx=0x7f7054008040 comp rx=0 tx=0).stop 2023-10-08T05:56:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.558+0000 7f70689f5640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70640afbe0 0x7f70640ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.558+0000 7f70689f5640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7064071070 0x7f70640cc640 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.558+0000 7f70689f5640 1 --2- 172.21.15.70:0/1045853209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f706406dfc0 0x7f706414cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.558+0000 7f70689f5640 1 -- 172.21.15.70:0/1045853209 >> 172.21.15.70:0/1045853209 conn(0x7f706406a5d0 msgr2=0x7f706406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:12.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.559+0000 7f70689f5640 1 -- 172.21.15.70:0/1045853209 shutdown_connections 2023-10-08T05:56:12.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.559+0000 7f70689f5640 1 -- 172.21.15.70:0/1045853209 wait complete. 2023-10-08T05:56:12.576 INFO:tasks.ceph.mds.c.smithi070.stderr: -7091> 2023-10-08T05:55:30.288+0000 7f77969a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:56:12.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T05:56:12.633 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:56:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.779+0000 7f9619b39640 1 Processor -- start 2023-10-08T05:56:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.780+0000 7f9619b39640 1 -- start start 2023-10-08T05:56:12.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.781+0000 7f9619b39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f961406dfc0 0x7f9614070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.781+0000 7f9619b39640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140b1cc0 con 0x7f96140afbe0 2023-10-08T05:56:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.781+0000 7f9619b39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140b2620 con 0x7f9614071070 2023-10-08T05:56:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.781+0000 7f9619b39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140b2060 con 0x7f961406dfc0 2023-10-08T05:56:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.781+0000 7f96137fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f96140afbe0 0x7f96140b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42262/0 (socket says 172.21.15.70:42262) 2023-10-08T05:56:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.781+0000 7f96137fe640 1 -- 172.21.15.70:0/1476062307 learned_addr learned my addr 172.21.15.70:0/1476062307 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601871287 0 0) 0x7f96140b1cc0 con 0x7f96140afbe0 2023-10-08T05:56:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9600003520 con 0x7f96140afbe0 2023-10-08T05:56:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f960affd640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f961406dfc0 0x7f9614070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 471489669 0 0) 0x7f9600003520 con 0x7f96140afbe0 2023-10-08T05:56:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f961406dfc0 msgr2=0x7f9614070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f961406dfc0 0x7f9614070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 >> v1:172.21.15.181:6789/0 conn(0x7f9614071070 legacy=0x7f96140cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140b2340 con 0x7f96140afbe0 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9604001670 con 0x7f96140afbe0 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 >> v1:172.21.15.70:6789/0 conn(0x7f96140afbe0 legacy=0x7f96140b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96000039a0 0x7f9600003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96000047c0 0x7f9600004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f961406dfc0 0x7f9600005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140b2060 con 0x7f96000039a0 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960000bb10 con 0x7f96000047c0 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.782+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960000bcd0 con 0x7f961406dfc0 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.783+0000 7f96137fe640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96000039a0 0x7f9600003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.783+0000 7f960affd640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96000047c0 0x7f9600004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.783+0000 7f9613fff640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f961406dfc0 0x7f9600005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.783+0000 7f96137fe640 1 -- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f961406dfc0 msgr2=0x7f9600005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.783+0000 7f96137fe640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f961406dfc0 0x7f9600005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.783+0000 7f96137fe640 1 -- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96000047c0 msgr2=0x7f9600004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.783+0000 7f96137fe640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96000047c0 0x7f9600004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.783+0000 7f96137fe640 1 -- 172.21.15.70:0/1476062307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600003520 con 0x7f96000039a0 2023-10-08T05:56:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.783+0000 7f9613fff640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f961406dfc0 0x7f9600005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.783+0000 7f960affd640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96000047c0 0x7f9600004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.784+0000 7f96137fe640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96000039a0 0x7f9600003dc0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f960400c240 tx=0x7f96000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937f0a82a51c340f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.784+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604005ad0 con 0x7f96000039a0 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.784+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9604005c30 con 0x7f96000039a0 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.784+0000 7f9612ffd640 1 -- 172.21.15.70:0/1476062307 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960402c8b0 con 0x7f96000039a0 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.785+0000 7f9619b39640 1 -- 172.21.15.70:0/1476062307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96000039a0 msgr2=0x7f9600003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.785+0000 7f9619b39640 1 --2- 172.21.15.70:0/1476062307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96000039a0 0x7f9600003dc0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f960400c240 tx=0x7f96000057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.785+0000 7f9613fff640 1 -- 172.21.15.70:0/1476062307 reap_dead start 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.785+0000 7f9619b39640 1 -- 172.21.15.70:0/1476062307 shutdown_connections 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.785+0000 7f9619b39640 1 -- 172.21.15.70:0/1476062307 >> 172.21.15.70:0/1476062307 conn(0x7f961406a5d0 msgr2=0x7f96140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.785+0000 7f9619b39640 1 -- 172.21.15.70:0/1476062307 shutdown_connections 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.785+0000 7f9619b39640 1 -- 172.21.15.70:0/1476062307 wait complete. 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.787+0000 7f9619b39640 1 Processor -- start 2023-10-08T05:56:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.787+0000 7f9619b39640 1 -- start start 2023-10-08T05:56:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f9619b39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96000039a0 0x7f961414dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f9619b39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b4d0 0x7f961415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f9619b39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96140c2f60 0x7f96140c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f9619b39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140c2230 con 0x7f961415b4d0 2023-10-08T05:56:12.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f9619b39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96141f3e00 con 0x7f96140c2f60 2023-10-08T05:56:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f9619b39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140c2480 con 0x7f96000039a0 2023-10-08T05:56:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f960affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b4d0 0x7f961415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f960affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b4d0 0x7f961415b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41034/0 (socket says 172.21.15.70:41034) 2023-10-08T05:56:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f960affd640 1 -- 172.21.15.70:0/3231159363 learned_addr learned my addr 172.21.15.70:0/3231159363 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f960affd640 1 -- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96000039a0 msgr2=0x7f961414dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f960affd640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96000039a0 0x7f961414dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f960affd640 1 -- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96140c2f60 msgr2=0x7f96140c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f960affd640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96140c2f60 0x7f96140c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.788+0000 7f960affd640 1 -- 172.21.15.70:0/3231159363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140c2760 con 0x7f961415b4d0 2023-10-08T05:56:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.789+0000 7f960affd640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b4d0 0x7f961415b950 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00e810 tx=0x7f95fc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.789+0000 7f9610ff9640 1 -- 172.21.15.70:0/3231159363 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00c6d0 con 0x7f961415b4d0 2023-10-08T05:56:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.789+0000 7f9610ff9640 1 -- 172.21.15.70:0/3231159363 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc00c830 con 0x7f961415b4d0 2023-10-08T05:56:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.789+0000 7f9619b39640 1 -- 172.21.15.70:0/3231159363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96140c2a40 con 0x7f961415b4d0 2023-10-08T05:56:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.789+0000 7f9610ff9640 1 -- 172.21.15.70:0/3231159363 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc00cbe0 con 0x7f961415b4d0 2023-10-08T05:56:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.789+0000 7f9619b39640 1 -- 172.21.15.70:0/3231159363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961415da50 con 0x7f961415b4d0 2023-10-08T05:56:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.790+0000 7f9619b39640 1 -- 172.21.15.70:0/3231159363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961404e160 con 0x7f961415b4d0 2023-10-08T05:56:12.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.791+0000 7f9610ff9640 1 -- 172.21.15.70:0/3231159363 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 66) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f95fc00cd40 con 0x7f961415b4d0 2023-10-08T05:56:12.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.792+0000 7f9610ff9640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f95f406af00 0x7f95f406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:12.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.792+0000 7f9610ff9640 1 -- 172.21.15.70:0/3231159363 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6961+0+0 (secure 0 0 0) 0x7f95fc089960 con 0x7f961415b4d0 2023-10-08T05:56:12.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.793+0000 7f96137fe640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f95f406af00 0x7f95f406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:12.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.794+0000 7f96137fe640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f95f406af00 0x7f95f406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9604034000 tx=0x7f9604007000 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:12.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:12.796+0000 7f9610ff9640 1 -- 172.21.15.70:0/3231159363 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95fc0543b0 con 0x7f961415b4d0 2023-10-08T05:56:13.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.020+0000 7f9619b39640 1 -- 172.21.15.70:0/3231159363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f96140b0ef0 con 0x7f961415b4d0 2023-10-08T05:56:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.638+0000 7f9610ff9640 1 -- 172.21.15.70:0/3231159363 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7f95fc0579d0 con 0x7f961415b4d0 2023-10-08T05:56:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.641+0000 7f9619b39640 1 -- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f95f406af00 msgr2=0x7f95f406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.641+0000 7f9619b39640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f95f406af00 0x7f95f406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9604034000 tx=0x7f9604007000 comp rx=0 tx=0).stop 2023-10-08T05:56:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.641+0000 7f9619b39640 1 -- 172.21.15.70:0/3231159363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b4d0 msgr2=0x7f961415b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.641+0000 7f9619b39640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b4d0 0x7f961415b950 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f95fc00e810 tx=0x7f95fc00ecd0 comp rx=0 tx=0).stop 2023-10-08T05:56:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.641+0000 7f9619b39640 1 -- 172.21.15.70:0/3231159363 shutdown_connections 2023-10-08T05:56:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.641+0000 7f9619b39640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f95f406af00 0x7f95f406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:13.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.641+0000 7f9619b39640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96140c2f60 0x7f96140c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.641+0000 7f9619b39640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f961415b4d0 0x7f961415b950 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.641+0000 7f9619b39640 1 --2- 172.21.15.70:0/3231159363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96000039a0 0x7f961414dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.641+0000 7f9619b39640 1 -- 172.21.15.70:0/3231159363 >> 172.21.15.70:0/3231159363 conn(0x7f961406a5d0 msgr2=0x7f961415f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.642+0000 7f9619b39640 1 -- 172.21.15.70:0/3231159363 shutdown_connections 2023-10-08T05:56:13.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.642+0000 7f9619b39640 1 -- 172.21.15.70:0/3231159363 wait complete. 2023-10-08T05:56:13.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T05:56:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.924+0000 7fe85216c640 1 Processor -- start 2023-10-08T05:56:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.924+0000 7fe85216c640 1 -- start start 2023-10-08T05:56:13.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.925+0000 7fe85216c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe84c0702b0 0x7fe84c070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:13.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.925+0000 7fe85216c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe84c0ca370 0x7fe84c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:13.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.925+0000 7fe85216c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84c06c3a0 con 0x7fe84c0b21f0 2023-10-08T05:56:13.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.925+0000 7fe85216c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe84c070bd0 con 0x7fe84c0ca370 2023-10-08T05:56:13.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.925+0000 7fe85216c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c06c640 con 0x7fe84c0702b0 2023-10-08T05:56:13.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.926+0000 7fe84bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe84c0ca370 0x7fe84c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:13.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.926+0000 7fe84affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe84c0b21f0 0x7fe84c0c9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53508/0 (socket says 172.21.15.70:53508) 2023-10-08T05:56:13.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.926+0000 7fe84affd640 1 -- 172.21.15.70:0/2263401844 learned_addr learned my addr 172.21.15.70:0/2263401844 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:13.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.926+0000 7fe84b7fe640 1 --2- 172.21.15.70:0/2263401844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe84c0702b0 0x7fe84c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:13.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.926+0000 7fe84bfff640 1 -- 172.21.15.70:0/2263401844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe84c0702b0 msgr2=0x7fe84c070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:13.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.926+0000 7fe84bfff640 1 --2- 172.21.15.70:0/2263401844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe84c0702b0 0x7fe84c070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:13.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.926+0000 7fe84bfff640 1 -- 172.21.15.70:0/2263401844 >> v1:172.21.15.70:6789/0 conn(0x7fe84c0b21f0 legacy=0x7fe84c0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:13.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.926+0000 7fe84bfff640 1 -- 172.21.15.70:0/2263401844 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c06c920 con 0x7fe84c0ca370 2023-10-08T05:56:13.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.928+0000 7fe84bfff640 1 --2- 172.21.15.70:0/2263401844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0ca370 0x7fe84c06d400 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fe844009d90 tx=0x7fe84402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d02c2734c4daf4c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:13.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.928+0000 7fe84a7fc640 1 -- 172.21.15.70:0/2263401844 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84402cbd0 con 0x7fe84c0ca370 2023-10-08T05:56:13.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.928+0000 7fe84a7fc640 1 -- 172.21.15.70:0/2263401844 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84402cd30 con 0x7fe84c0ca370 2023-10-08T05:56:13.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.928+0000 7fe84a7fc640 1 -- 172.21.15.70:0/2263401844 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84402cea0 con 0x7fe84c0ca370 2023-10-08T05:56:13.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.929+0000 7fe85216c640 1 -- 172.21.15.70:0/2263401844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0ca370 msgr2=0x7fe84c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:13.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.929+0000 7fe85216c640 1 --2- 172.21.15.70:0/2263401844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0ca370 0x7fe84c06d400 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fe844009d90 tx=0x7fe84402be20 comp rx=0 tx=0).stop 2023-10-08T05:56:13.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.929+0000 7fe85216c640 1 -- 172.21.15.70:0/2263401844 shutdown_connections 2023-10-08T05:56:13.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.929+0000 7fe85216c640 1 --2- 172.21.15.70:0/2263401844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0ca370 0x7fe84c06d400 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:13.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.929+0000 7fe85216c640 1 --2- 172.21.15.70:0/2263401844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe84c0702b0 0x7fe84c070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:13.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.929+0000 7fe85216c640 1 -- 172.21.15.70:0/2263401844 >> 172.21.15.70:0/2263401844 conn(0x7fe84c06a5d0 msgr2=0x7fe84c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:13.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.929+0000 7fe85216c640 1 -- 172.21.15.70:0/2263401844 shutdown_connections 2023-10-08T05:56:13.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.929+0000 7fe85216c640 1 -- 172.21.15.70:0/2263401844 wait complete. 2023-10-08T05:56:13.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.931+0000 7fe85216c640 1 Processor -- start 2023-10-08T05:56:13.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.931+0000 7fe85216c640 1 -- start start 2023-10-08T05:56:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.932+0000 7fe85216c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe84c0702b0 0x7fe84c14d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.932+0000 7fe85216c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0b21f0 0x7fe84c14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.932+0000 7fe85216c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe84c0ca370 0x7fe84c1563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.932+0000 7fe85216c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c1544c0 con 0x7fe84c0b21f0 2023-10-08T05:56:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.932+0000 7fe85216c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c153f00 con 0x7fe84c0702b0 2023-10-08T05:56:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.932+0000 7fe85216c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c1541e0 con 0x7fe84c0ca370 2023-10-08T05:56:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.932+0000 7fe84affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0b21f0 0x7fe84c14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.932+0000 7fe84affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0b21f0 0x7fe84c14dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34612/0 (socket says 172.21.15.70:34612) 2023-10-08T05:56:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.932+0000 7fe84affd640 1 -- 172.21.15.70:0/2111379925 learned_addr learned my addr 172.21.15.70:0/2111379925 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe84b7fe640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe84c0702b0 0x7fe84c14d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe84bfff640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe84c0ca370 0x7fe84c1563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe84affd640 1 -- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe84c0ca370 msgr2=0x7fe84c1563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe84affd640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe84c0ca370 0x7fe84c1563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe84affd640 1 -- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe84c0702b0 msgr2=0x7fe84c14d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe84affd640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe84c0702b0 0x7fe84c14d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe84affd640 1 -- 172.21.15.70:0/2111379925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844009a70 con 0x7fe84c0b21f0 2023-10-08T05:56:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe84affd640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0b21f0 0x7fe84c14dc00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fe84000c4a0 tx=0x7fe84000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe833fff640 1 -- 172.21.15.70:0/2111379925 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840005ce0 con 0x7fe84c0b21f0 2023-10-08T05:56:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe833fff640 1 -- 172.21.15.70:0/2111379925 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe840005e40 con 0x7fe84c0b21f0 2023-10-08T05:56:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe85216c640 1 -- 172.21.15.70:0/2111379925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c0c6a50 con 0x7fe84c0b21f0 2023-10-08T05:56:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.933+0000 7fe833fff640 1 -- 172.21.15.70:0/2111379925 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe840007270 con 0x7fe84c0b21f0 2023-10-08T05:56:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.934+0000 7fe85216c640 1 -- 172.21.15.70:0/2111379925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c0cc640 con 0x7fe84c0b21f0 2023-10-08T05:56:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.935+0000 7fe85216c640 1 -- 172.21.15.70:0/2111379925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84c0c68b0 con 0x7fe84c0b21f0 2023-10-08T05:56:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.936+0000 7fe833fff640 1 -- 172.21.15.70:0/2111379925 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe8400073d0 con 0x7fe84c0b21f0 2023-10-08T05:56:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.937+0000 7fe833fff640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe83806afc0 0x7fe83806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.937+0000 7fe833fff640 1 -- 172.21.15.70:0/2111379925 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6961+0+0 (secure 0 0 0) 0x7fe840089160 con 0x7fe84c0b21f0 2023-10-08T05:56:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.937+0000 7fe84b7fe640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe83806afc0 0x7fe83806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:13.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.939+0000 7fe84b7fe640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe83806afc0 0x7fe83806d480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe83c006fd0 tx=0x7fe83c008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:13.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:13.941+0000 7fe833fff640 1 -- 172.21.15.70:0/2111379925 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe84c0c68b0 con 0x7fe84c0b21f0 2023-10-08T05:56:14.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.131+0000 7fe85216c640 1 -- 172.21.15.70:0/2111379925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe84c0c68b0 con 0x7fe84c0b21f0 2023-10-08T05:56:14.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.650+0000 7fe833fff640 1 -- 172.21.15.70:0/2111379925 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v139) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe84c0c68b0 con 0x7fe84c0b21f0 2023-10-08T05:56:14.651 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T05:56:14.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.656+0000 7fe85216c640 1 -- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe83806afc0 msgr2=0x7fe83806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:14.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.657+0000 7fe85216c640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe83806afc0 0x7fe83806d480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe83c006fd0 tx=0x7fe83c008040 comp rx=0 tx=0).stop 2023-10-08T05:56:14.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.657+0000 7fe85216c640 1 -- 172.21.15.70:0/2111379925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0b21f0 msgr2=0x7fe84c14dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:14.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.657+0000 7fe85216c640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0b21f0 0x7fe84c14dc00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fe84000c4a0 tx=0x7fe84000c960 comp rx=0 tx=0).stop 2023-10-08T05:56:14.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.657+0000 7fe85216c640 1 -- 172.21.15.70:0/2111379925 shutdown_connections 2023-10-08T05:56:14.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.657+0000 7fe85216c640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe83806afc0 0x7fe83806d480 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:14.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.657+0000 7fe85216c640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe84c0ca370 0x7fe84c1563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:14.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.657+0000 7fe85216c640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe84c0b21f0 0x7fe84c14dc00 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:14.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.657+0000 7fe85216c640 1 --2- 172.21.15.70:0/2111379925 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe84c0702b0 0x7fe84c14d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:14.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.657+0000 7fe85216c640 1 -- 172.21.15.70:0/2111379925 >> 172.21.15.70:0/2111379925 conn(0x7fe84c06a5d0 msgr2=0x7fe84c0cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:14.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.658+0000 7fe85216c640 1 -- 172.21.15.70:0/2111379925 shutdown_connections 2023-10-08T05:56:14.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.658+0000 7fe85216c640 1 -- 172.21.15.70:0/2111379925 wait complete. 2023-10-08T05:56:14.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T05:56:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.883+0000 7f4f86368640 1 Processor -- start 2023-10-08T05:56:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.883+0000 7f4f86368640 1 -- start start 2023-10-08T05:56:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.884+0000 7f4f86368640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f800ca370 0x7f4f8006d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.884+0000 7f4f86368640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8006c3a0 con 0x7f4f800b21f0 2023-10-08T05:56:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.885+0000 7f4f86368640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f8006c640 con 0x7f4f800702b0 2023-10-08T05:56:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.885+0000 7f4f86368640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8006c920 con 0x7f4f800ca370 2023-10-08T05:56:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.885+0000 7f4f84905640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f800ca370 0x7f4f8006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.885+0000 7f4f7f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4f800b21f0 0x7f4f800c9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53522/0 (socket says 172.21.15.70:53522) 2023-10-08T05:56:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.885+0000 7f4f7f7fe640 1 -- 172.21.15.70:0/3144823289 learned_addr learned my addr 172.21.15.70:0/3144823289 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.886+0000 7f4f84905640 1 -- 172.21.15.70:0/3144823289 >> v1:172.21.15.181:6789/0 conn(0x7f4f800702b0 legacy=0x7f4f80070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.886+0000 7f4f84905640 1 -- 172.21.15.70:0/3144823289 >> v1:172.21.15.70:6789/0 conn(0x7f4f800b21f0 legacy=0x7f4f800c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.886+0000 7f4f84905640 1 -- 172.21.15.70:0/3144823289 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8006cc00 con 0x7f4f800ca370 2023-10-08T05:56:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.887+0000 7f4f84905640 1 --2- 172.21.15.70:0/3144823289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800ca370 0x7f4f8006d400 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009d90 tx=0x7f4f7802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c503575ba8c3b66 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.887+0000 7f4f7effd640 1 -- 172.21.15.70:0/3144823289 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7802cc60 con 0x7f4f800ca370 2023-10-08T05:56:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.887+0000 7f4f7effd640 1 -- 172.21.15.70:0/3144823289 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7802cdc0 con 0x7f4f800ca370 2023-10-08T05:56:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.887+0000 7f4f7effd640 1 -- 172.21.15.70:0/3144823289 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f78034660 con 0x7f4f800ca370 2023-10-08T05:56:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.888+0000 7f4f86368640 1 -- 172.21.15.70:0/3144823289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800ca370 msgr2=0x7f4f8006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.888+0000 7f4f86368640 1 --2- 172.21.15.70:0/3144823289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800ca370 0x7f4f8006d400 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009d90 tx=0x7f4f7802beb0 comp rx=0 tx=0).stop 2023-10-08T05:56:14.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.888+0000 7f4f86368640 1 -- 172.21.15.70:0/3144823289 shutdown_connections 2023-10-08T05:56:14.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.888+0000 7f4f86368640 1 --2- 172.21.15.70:0/3144823289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800ca370 0x7f4f8006d400 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:14.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.888+0000 7f4f86368640 1 -- 172.21.15.70:0/3144823289 >> 172.21.15.70:0/3144823289 conn(0x7f4f8006a5d0 msgr2=0x7f4f800b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:14.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.888+0000 7f4f86368640 1 -- 172.21.15.70:0/3144823289 shutdown_connections 2023-10-08T05:56:14.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.888+0000 7f4f86368640 1 -- 172.21.15.70:0/3144823289 wait complete. 2023-10-08T05:56:14.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.890+0000 7f4f86368640 1 Processor -- start 2023-10-08T05:56:14.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.890+0000 7f4f86368640 1 -- start start 2023-10-08T05:56:14.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f86368640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800702b0 0x7f4f80149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:14.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f86368640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f800b21f0 0x7f4f80149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:14.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f86368640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f800ca370 0x7f4f8015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:14.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f86368640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80146e90 con 0x7f4f800702b0 2023-10-08T05:56:14.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f86368640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80147170 con 0x7f4f800ca370 2023-10-08T05:56:14.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f86368640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80147450 con 0x7f4f800b21f0 2023-10-08T05:56:14.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f7ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800702b0 0x7f4f80149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:14.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f7ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800702b0 0x7f4f80149400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34636/0 (socket says 172.21.15.70:34636) 2023-10-08T05:56:14.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f7ffff640 1 -- 172.21.15.70:0/4241786735 learned_addr learned my addr 172.21.15.70:0/4241786735 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f7ffff640 1 -- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f800b21f0 msgr2=0x7f4f80149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f7ffff640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f800b21f0 0x7f4f80149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f7ffff640 1 -- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f800ca370 msgr2=0x7f4f8015a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f7ffff640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f800ca370 0x7f4f8015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.891+0000 7f4f7ffff640 1 -- 172.21.15.70:0/4241786735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f78009a70 con 0x7f4f800702b0 2023-10-08T05:56:14.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.892+0000 7f4f7ffff640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800702b0 0x7f4f80149400 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000c450 tx=0x7f4f7000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.892+0000 7f4f7cff9640 1 -- 172.21.15.70:0/4241786735 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f700052d0 con 0x7f4f800702b0 2023-10-08T05:56:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.892+0000 7f4f7cff9640 1 -- 172.21.15.70:0/4241786735 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f70005430 con 0x7f4f800702b0 2023-10-08T05:56:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.892+0000 7f4f7cff9640 1 -- 172.21.15.70:0/4241786735 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70012870 con 0x7f4f800702b0 2023-10-08T05:56:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.892+0000 7f4f86368640 1 -- 172.21.15.70:0/4241786735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f80147730 con 0x7f4f800702b0 2023-10-08T05:56:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.892+0000 7f4f86368640 1 -- 172.21.15.70:0/4241786735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8015af00 con 0x7f4f800702b0 2023-10-08T05:56:14.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.893+0000 7f4f86368640 1 -- 172.21.15.70:0/4241786735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f80147c50 con 0x7f4f800702b0 2023-10-08T05:56:14.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.894+0000 7f4f7cff9640 1 -- 172.21.15.70:0/4241786735 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4f700129d0 con 0x7f4f800702b0 2023-10-08T05:56:14.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.895+0000 7f4f7cff9640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f6c06af90 0x7f4f6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:14.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.895+0000 7f4f7cff9640 1 -- 172.21.15.70:0/4241786735 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6523+0+0 (secure 0 0 0) 0x7f4f700896a0 con 0x7f4f800702b0 2023-10-08T05:56:14.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.896+0000 7f4f7f7fe640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f6c06af90 0x7f4f6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:14.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.897+0000 7f4f7f7fe640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f6c06af90 0x7f4f6c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4f8014db00 tx=0x7f4f74007a40 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:14.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:14.900+0000 7f4f7cff9640 1 -- 172.21.15.70:0/4241786735 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f70054270 con 0x7f4f800702b0 2023-10-08T05:56:15.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.092+0000 7f4f86368640 1 -- 172.21.15.70:0/4241786735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4f8004e160 con 0x7f4f800702b0 2023-10-08T05:56:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.661+0000 7f4f7cff9640 1 -- 172.21.15.70:0/4241786735 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v140) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4f70057890 con 0x7f4f800702b0 2023-10-08T05:56:15.662 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T05:56:15.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.666+0000 7f4f86368640 1 -- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f6c06af90 msgr2=0x7f4f6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:15.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.666+0000 7f4f86368640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f6c06af90 0x7f4f6c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4f8014db00 tx=0x7f4f74007a40 comp rx=0 tx=0).stop 2023-10-08T05:56:15.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.666+0000 7f4f86368640 1 -- 172.21.15.70:0/4241786735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800702b0 msgr2=0x7f4f80149400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:15.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.666+0000 7f4f86368640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800702b0 0x7f4f80149400 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000c450 tx=0x7f4f7000c910 comp rx=0 tx=0).stop 2023-10-08T05:56:15.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.667+0000 7f4f86368640 1 -- 172.21.15.70:0/4241786735 shutdown_connections 2023-10-08T05:56:15.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.667+0000 7f4f86368640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4f6c06af90 0x7f4f6c06d450 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4f8014db00 tx=0x7f4f74007a40 comp rx=0 tx=0).stop 2023-10-08T05:56:15.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.667+0000 7f4f86368640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f800ca370 0x7f4f8015a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:15.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.667+0000 7f4f86368640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f800b21f0 0x7f4f80149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:15.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.667+0000 7f4f86368640 1 --2- 172.21.15.70:0/4241786735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f800702b0 0x7f4f80149400 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:15.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.667+0000 7f4f86368640 1 -- 172.21.15.70:0/4241786735 >> 172.21.15.70:0/4241786735 conn(0x7f4f8006a5d0 msgr2=0x7f4f80159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.667+0000 7f4f86368640 1 -- 172.21.15.70:0/4241786735 shutdown_connections 2023-10-08T05:56:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.668+0000 7f4f86368640 1 -- 172.21.15.70:0/4241786735 wait complete. 2023-10-08T05:56:15.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T05:56:15.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.898+0000 7fd4516fc640 1 Processor -- start 2023-10-08T05:56:15.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.898+0000 7fd4516fc640 1 -- start start 2023-10-08T05:56:15.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.900+0000 7fd4516fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd44c070cc0 0x7fd44c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:15.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.900+0000 7fd4516fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 0x7fd44c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:15.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.900+0000 7fd4516fc640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c071170 con 0x7fd44c160660 2023-10-08T05:56:15.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.900+0000 7fd4516fc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44c06e150 con 0x7fd44c070cc0 2023-10-08T05:56:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.900+0000 7fd4516fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0cb8c0 con 0x7fd44c0cb450 2023-10-08T05:56:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.901+0000 7fd44b7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 0x7fd44c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.901+0000 7fd44b7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 0x7fd44c0b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:43432/0 (socket says 172.21.15.70:43432) 2023-10-08T05:56:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.901+0000 7fd44b7fe640 1 -- 172.21.15.70:0/3191331203 learned_addr learned my addr 172.21.15.70:0/3191331203 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:15.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.901+0000 7fd443fff640 1 --2- 172.21.15.70:0/3191331203 >> v2:172.21.15.181:3300/0 conn(0x7fd44c070cc0 0x7fd44c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:15.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.902+0000 7fd443fff640 1 -- 172.21.15.70:0/3191331203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 msgr2=0x7fd44c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:15.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.902+0000 7fd443fff640 1 --2- 172.21.15.70:0/3191331203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 0x7fd44c0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:15.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.902+0000 7fd443fff640 1 -- 172.21.15.70:0/3191331203 >> v1:172.21.15.181:6789/0 conn(0x7fd44c160660 legacy=0x7fd44c0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:15.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.902+0000 7fd443fff640 1 -- 172.21.15.70:0/3191331203 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c06de70 con 0x7fd44c070cc0 2023-10-08T05:56:15.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.902+0000 7fd44b7fe640 1 --2- 172.21.15.70:0/3191331203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 0x7fd44c0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:56:15.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.903+0000 7fd443fff640 1 --2- 172.21.15.70:0/3191331203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c070cc0 0x7fd44c0caf10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd4340097a0 tx=0x7fd43402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d8769061404868 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:15.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.903+0000 7fd44a7fc640 1 -- 172.21.15.70:0/3191331203 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43402d040 con 0x7fd44c070cc0 2023-10-08T05:56:15.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.903+0000 7fd44a7fc640 1 -- 172.21.15.70:0/3191331203 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434031810 con 0x7fd44c070cc0 2023-10-08T05:56:15.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.903+0000 7fd44a7fc640 1 -- 172.21.15.70:0/3191331203 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434031ba0 con 0x7fd44c070cc0 2023-10-08T05:56:15.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.904+0000 7fd4516fc640 1 -- 172.21.15.70:0/3191331203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c070cc0 msgr2=0x7fd44c0caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:15.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.904+0000 7fd4516fc640 1 --2- 172.21.15.70:0/3191331203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c070cc0 0x7fd44c0caf10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd4340097a0 tx=0x7fd43402b8b0 comp rx=0 tx=0).stop 2023-10-08T05:56:15.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.904+0000 7fd4516fc640 1 -- 172.21.15.70:0/3191331203 shutdown_connections 2023-10-08T05:56:15.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.904+0000 7fd4516fc640 1 --2- 172.21.15.70:0/3191331203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 0x7fd44c0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:15.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.904+0000 7fd4516fc640 1 --2- 172.21.15.70:0/3191331203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c070cc0 0x7fd44c0caf10 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:15.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.904+0000 7fd4516fc640 1 -- 172.21.15.70:0/3191331203 >> 172.21.15.70:0/3191331203 conn(0x7fd44c06a5d0 msgr2=0x7fd44c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:15.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.904+0000 7fd4516fc640 1 -- 172.21.15.70:0/3191331203 shutdown_connections 2023-10-08T05:56:15.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.904+0000 7fd4516fc640 1 -- 172.21.15.70:0/3191331203 wait complete. 2023-10-08T05:56:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.906+0000 7fd4516fc640 1 Processor -- start 2023-10-08T05:56:15.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.906+0000 7fd4516fc640 1 -- start start 2023-10-08T05:56:15.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.907+0000 7fd4516fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c070cc0 0x7fd44c150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:15.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.907+0000 7fd4516fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 0x7fd44c0c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:15.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.907+0000 7fd4516fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c160660 0x7fd44c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:15.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.907+0000 7fd4516fc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0c64c0 con 0x7fd44c070cc0 2023-10-08T05:56:15.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.907+0000 7fd4516fc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0c58b0 con 0x7fd44c160660 2023-10-08T05:56:15.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.907+0000 7fd4516fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0c5b90 con 0x7fd44c0cb450 2023-10-08T05:56:15.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.907+0000 7fd44affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c070cc0 0x7fd44c150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:15.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.907+0000 7fd44affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c070cc0 0x7fd44c150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34638/0 (socket says 172.21.15.70:34638) 2023-10-08T05:56:15.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.907+0000 7fd44affd640 1 -- 172.21.15.70:0/1828489343 learned_addr learned my addr 172.21.15.70:0/1828489343 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:15.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.908+0000 7fd443fff640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 0x7fd44c0c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:15.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.908+0000 7fd44b7fe640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c160660 0x7fd44c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:15.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.908+0000 7fd44affd640 1 -- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 msgr2=0x7fd44c0c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:15.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.908+0000 7fd44affd640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 0x7fd44c0c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:15.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.908+0000 7fd44affd640 1 -- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c160660 msgr2=0x7fd44c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:15.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.908+0000 7fd44affd640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c160660 0x7fd44c15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:15.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.908+0000 7fd44affd640 1 -- 172.21.15.70:0/1828489343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd444009b30 con 0x7fd44c070cc0 2023-10-08T05:56:15.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.908+0000 7fd44b7fe640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c160660 0x7fd44c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:15.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.908+0000 7fd44affd640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c070cc0 0x7fd44c150eb0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00c400 tx=0x7fd43c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:15.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.909+0000 7fd4437fe640 1 -- 172.21.15.70:0/1828489343 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0052d0 con 0x7fd44c070cc0 2023-10-08T05:56:15.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.909+0000 7fd4437fe640 1 -- 172.21.15.70:0/1828489343 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c005430 con 0x7fd44c070cc0 2023-10-08T05:56:15.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.909+0000 7fd4437fe640 1 -- 172.21.15.70:0/1828489343 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c003830 con 0x7fd44c070cc0 2023-10-08T05:56:15.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.909+0000 7fd4516fc640 1 -- 172.21.15.70:0/1828489343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd434009400 con 0x7fd44c070cc0 2023-10-08T05:56:15.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.909+0000 7fd4516fc640 1 -- 172.21.15.70:0/1828489343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c0c6150 con 0x7fd44c070cc0 2023-10-08T05:56:15.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.910+0000 7fd4516fc640 1 -- 172.21.15.70:0/1828489343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c04e160 con 0x7fd44c070cc0 2023-10-08T05:56:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.911+0000 7fd4437fe640 1 -- 172.21.15.70:0/1828489343 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd43c003990 con 0x7fd44c070cc0 2023-10-08T05:56:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.912+0000 7fd4437fe640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd42c06af90 0x7fd42c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.912+0000 7fd4437fe640 1 -- 172.21.15.70:0/1828489343 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fd43c088da0 con 0x7fd44c070cc0 2023-10-08T05:56:15.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.913+0000 7fd443fff640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd42c06af90 0x7fd42c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:15.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.914+0000 7fd443fff640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd42c06af90 0x7fd42c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd434036000 tx=0x7fd43403c040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:15.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:15.916+0000 7fd4437fe640 1 -- 172.21.15.70:0/1828489343 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd43c053b10 con 0x7fd44c070cc0 2023-10-08T05:56:16.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.108+0000 7fd4516fc640 1 -- 172.21.15.70:0/1828489343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd44c071170 con 0x7fd44c070cc0 2023-10-08T05:56:16.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.115+0000 7fd4437fe640 1 -- 172.21.15.70:0/1828489343 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd44c071170 con 0x7fd44c070cc0 2023-10-08T05:56:16.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.118+0000 7fd4516fc640 1 -- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd42c06af90 msgr2=0x7fd42c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.118+0000 7fd4516fc640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd42c06af90 0x7fd42c06d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd434036000 tx=0x7fd43403c040 comp rx=0 tx=0).stop 2023-10-08T05:56:16.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.118+0000 7fd4516fc640 1 -- 172.21.15.70:0/1828489343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c070cc0 msgr2=0x7fd44c150eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.118+0000 7fd4516fc640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c070cc0 0x7fd44c150eb0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd43c00c400 tx=0x7fd43c00c8c0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.118+0000 7fd4516fc640 1 -- 172.21.15.70:0/1828489343 shutdown_connections 2023-10-08T05:56:16.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.118+0000 7fd4516fc640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd42c06af90 0x7fd42c06d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.118+0000 7fd4516fc640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd44c160660 0x7fd44c15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.118+0000 7fd4516fc640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd44c0cb450 0x7fd44c0c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.118+0000 7fd4516fc640 1 --2- 172.21.15.70:0/1828489343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c070cc0 0x7fd44c150eb0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.118+0000 7fd4516fc640 1 -- 172.21.15.70:0/1828489343 >> 172.21.15.70:0/1828489343 conn(0x7fd44c06a5d0 msgr2=0x7fd44c0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:16.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.119+0000 7fd4516fc640 1 -- 172.21.15.70:0/1828489343 shutdown_connections 2023-10-08T05:56:16.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.119+0000 7fd4516fc640 1 -- 172.21.15.70:0/1828489343 wait complete. 2023-10-08T05:56:16.138 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T05:56:16.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.353+0000 7fcf3cb87640 1 Processor -- start 2023-10-08T05:56:16.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.353+0000 7fcf3cb87640 1 -- start start 2023-10-08T05:56:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.354+0000 7fcf3cb87640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcf3815aab0 0x7fcf38070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.354+0000 7fcf3cb87640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcf380caef0 0x7fcf380aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.354+0000 7fcf3cb87640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3815ae90 con 0x7fcf380b04a0 2023-10-08T05:56:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.354+0000 7fcf3cb87640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf380b0930 con 0x7fcf3815aab0 2023-10-08T05:56:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.354+0000 7fcf3cb87640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf380cbdb0 con 0x7fcf380caef0 2023-10-08T05:56:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.354+0000 7fcf377fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcf3815aab0 0x7fcf38070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.355+0000 7fcf377fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcf3815aab0 0x7fcf38070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34648/0 (socket says 172.21.15.70:34648) 2023-10-08T05:56:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.355+0000 7fcf377fe640 1 -- 172.21.15.70:0/3071386483 learned_addr learned my addr 172.21.15.70:0/3071386483 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:16.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.355+0000 7fcf36ffd640 1 --2- 172.21.15.70:0/3071386483 >> v2:172.21.15.181:3300/0 conn(0x7fcf380caef0 0x7fcf380aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.355+0000 7fcf377fe640 1 -- 172.21.15.70:0/3071386483 >> v2:172.21.15.181:3300/0 conn(0x7fcf380caef0 msgr2=0x7fcf380aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:16.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.355+0000 7fcf377fe640 1 --2- 172.21.15.70:0/3071386483 >> v2:172.21.15.181:3300/0 conn(0x7fcf380caef0 0x7fcf380aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.355+0000 7fcf377fe640 1 -- 172.21.15.70:0/3071386483 >> v1:172.21.15.181:6789/0 conn(0x7fcf380b04a0 legacy=0x7fcf380c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:16.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.355+0000 7fcf377fe640 1 -- 172.21.15.70:0/3071386483 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf380cb7f0 con 0x7fcf3815aab0 2023-10-08T05:56:16.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.355+0000 7fcf36ffd640 1 --2- 172.21.15.70:0/3071386483 >> v2:172.21.15.181:3300/0 conn(0x7fcf380caef0 0x7fcf380aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:16.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.356+0000 7fcf377fe640 1 --2- 172.21.15.70:0/3071386483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf3815aab0 0x7fcf38070b50 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009870 tx=0x7fcf2802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c7a0fabd16592b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:16.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.357+0000 7fcf367fc640 1 -- 172.21.15.70:0/3071386483 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280317e0 con 0x7fcf3815aab0 2023-10-08T05:56:16.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.357+0000 7fcf367fc640 1 -- 172.21.15.70:0/3071386483 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf28031940 con 0x7fcf3815aab0 2023-10-08T05:56:16.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.357+0000 7fcf367fc640 1 -- 172.21.15.70:0/3071386483 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28031cd0 con 0x7fcf3815aab0 2023-10-08T05:56:16.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.357+0000 7fcf3cb87640 1 -- 172.21.15.70:0/3071386483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf3815aab0 msgr2=0x7fcf38070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.357+0000 7fcf3cb87640 1 --2- 172.21.15.70:0/3071386483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf3815aab0 0x7fcf38070b50 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009870 tx=0x7fcf2802b880 comp rx=0 tx=0).stop 2023-10-08T05:56:16.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.357+0000 7fcf3cb87640 1 -- 172.21.15.70:0/3071386483 shutdown_connections 2023-10-08T05:56:16.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.357+0000 7fcf3cb87640 1 --2- 172.21.15.70:0/3071386483 >> v2:172.21.15.181:3300/0 conn(0x7fcf380caef0 0x7fcf380aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.358+0000 7fcf3cb87640 1 --2- 172.21.15.70:0/3071386483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf3815aab0 0x7fcf38070b50 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.358+0000 7fcf3cb87640 1 -- 172.21.15.70:0/3071386483 >> 172.21.15.70:0/3071386483 conn(0x7fcf3806c010 msgr2=0x7fcf3806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:16.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.358+0000 7fcf3cb87640 1 -- 172.21.15.70:0/3071386483 shutdown_connections 2023-10-08T05:56:16.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.358+0000 7fcf3cb87640 1 -- 172.21.15.70:0/3071386483 wait complete. 2023-10-08T05:56:16.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.359+0000 7fcf3cb87640 1 Processor -- start 2023-10-08T05:56:16.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.359+0000 7fcf3cb87640 1 -- start start 2023-10-08T05:56:16.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.360+0000 7fcf3cb87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf380b04a0 0x7fcf380afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.360+0000 7fcf3cb87640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf380caef0 0x7fcf38150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.360+0000 7fcf3cb87640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3815aab0 0x7fcf3815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.360+0000 7fcf3cb87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf380c55b0 con 0x7fcf380b04a0 2023-10-08T05:56:16.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.360+0000 7fcf3cb87640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3815bd20 con 0x7fcf3815aab0 2023-10-08T05:56:16.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.360+0000 7fcf3cb87640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3815c000 con 0x7fcf380caef0 2023-10-08T05:56:16.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.361+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf380b04a0 0x7fcf380afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.361+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf380b04a0 0x7fcf380afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34658/0 (socket says 172.21.15.70:34658) 2023-10-08T05:56:16.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.361+0000 7fcf377fe640 1 -- 172.21.15.70:0/1157607539 learned_addr learned my addr 172.21.15.70:0/1157607539 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:16.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.361+0000 7fcf36ffd640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf380caef0 0x7fcf38150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.361+0000 7fcf37fff640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3815aab0 0x7fcf3815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.361+0000 7fcf377fe640 1 -- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf380caef0 msgr2=0x7fcf38150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.361+0000 7fcf377fe640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf380caef0 0x7fcf38150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.361+0000 7fcf377fe640 1 -- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3815aab0 msgr2=0x7fcf3815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.361+0000 7fcf377fe640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3815aab0 0x7fcf3815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.361+0000 7fcf377fe640 1 -- 172.21.15.70:0/1157607539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf280094d0 con 0x7fcf380b04a0 2023-10-08T05:56:16.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.362+0000 7fcf377fe640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf380b04a0 0x7fcf380afee0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fcf2802b8c0 tx=0x7fcf280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:16.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.362+0000 7fcf1ffff640 1 -- 172.21.15.70:0/1157607539 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280329c0 con 0x7fcf380b04a0 2023-10-08T05:56:16.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.362+0000 7fcf1ffff640 1 -- 172.21.15.70:0/1157607539 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf28032b20 con 0x7fcf380b04a0 2023-10-08T05:56:16.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.362+0000 7fcf1ffff640 1 -- 172.21.15.70:0/1157607539 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280453f0 con 0x7fcf380b04a0 2023-10-08T05:56:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.362+0000 7fcf3cb87640 1 -- 172.21.15.70:0/1157607539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf380c2af0 con 0x7fcf380b04a0 2023-10-08T05:56:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.362+0000 7fcf3cb87640 1 -- 172.21.15.70:0/1157607539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf380c30b0 con 0x7fcf380b04a0 2023-10-08T05:56:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.363+0000 7fcf3cb87640 1 -- 172.21.15.70:0/1157607539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf38074150 con 0x7fcf380b04a0 2023-10-08T05:56:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.364+0000 7fcf1ffff640 1 -- 172.21.15.70:0/1157607539 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcf2804e450 con 0x7fcf380b04a0 2023-10-08T05:56:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.365+0000 7fcf1ffff640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fcf2406af90 0x7fcf2406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.365+0000 7fcf1ffff640 1 -- 172.21.15.70:0/1157607539 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fcf280abbb0 con 0x7fcf380b04a0 2023-10-08T05:56:16.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.366+0000 7fcf36ffd640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fcf2406af90 0x7fcf2406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.367+0000 7fcf36ffd640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fcf2406af90 0x7fcf2406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcf380c5030 tx=0x7fcf2c008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:16.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.369+0000 7fcf1ffff640 1 -- 172.21.15.70:0/1157607539 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf28076810 con 0x7fcf380b04a0 2023-10-08T05:56:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.560+0000 7fcf3cb87640 1 -- 172.21.15.70:0/1157607539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fcf3815ae90 con 0x7fcf380b04a0 2023-10-08T05:56:16.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.567+0000 7fcf1ffff640 1 -- 172.21.15.70:0/1157607539 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v44) v1 ==== 115+0+0 (secure 0 0 0) 0x7fcf3815ae90 con 0x7fcf380b04a0 2023-10-08T05:56:16.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.569+0000 7fcf3cb87640 1 -- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fcf2406af90 msgr2=0x7fcf2406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.569+0000 7fcf3cb87640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fcf2406af90 0x7fcf2406d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcf380c5030 tx=0x7fcf2c008040 comp rx=0 tx=0).stop 2023-10-08T05:56:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.569+0000 7fcf3cb87640 1 -- 172.21.15.70:0/1157607539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf380b04a0 msgr2=0x7fcf380afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.569+0000 7fcf3cb87640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf380b04a0 0x7fcf380afee0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fcf2802b8c0 tx=0x7fcf280046c0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.570+0000 7fcf3cb87640 1 -- 172.21.15.70:0/1157607539 shutdown_connections 2023-10-08T05:56:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.570+0000 7fcf3cb87640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fcf2406af90 0x7fcf2406d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.570+0000 7fcf3cb87640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcf3815aab0 0x7fcf3815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.570+0000 7fcf3cb87640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcf380caef0 0x7fcf38150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.570+0000 7fcf3cb87640 1 --2- 172.21.15.70:0/1157607539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf380b04a0 0x7fcf380afee0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.570+0000 7fcf3cb87640 1 -- 172.21.15.70:0/1157607539 >> 172.21.15.70:0/1157607539 conn(0x7fcf3806c010 msgr2=0x7fcf380cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.570+0000 7fcf3cb87640 1 -- 172.21.15.70:0/1157607539 shutdown_connections 2023-10-08T05:56:16.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.570+0000 7fcf3cb87640 1 -- 172.21.15.70:0/1157607539 wait complete. 2023-10-08T05:56:16.589 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T05:56:16.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.806+0000 7fdf5a675640 1 Processor -- start 2023-10-08T05:56:16.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.806+0000 7fdf5a675640 1 -- start start 2023-10-08T05:56:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.807+0000 7fdf5a675640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf5406db60 0x7fdf540b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.808+0000 7fdf5a675640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5415b310 con 0x7fdf5406c3a0 2023-10-08T05:56:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.808+0000 7fdf5a675640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5406d290 con 0x7fdf540c9cf0 2023-10-08T05:56:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.808+0000 7fdf5a675640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5415b110 con 0x7fdf5406db60 2023-10-08T05:56:16.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.808+0000 7fdf53fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdf5406c3a0 0x7fdf5406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53538/0 (socket says 172.21.15.70:53538) 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.808+0000 7fdf53fff640 1 -- 172.21.15.70:0/4185607801 learned_addr learned my addr 172.21.15.70:0/4185607801 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.808+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971680922 0 0) 0x7fdf44001110 con 0x7fdf5406c3a0 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.808+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf40005730 con 0x7fdf5406c3a0 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.808+0000 7fdf58c12640 1 --2- 172.21.15.70:0/4185607801 >> v2:172.21.15.181:3300/0 conn(0x7fdf5406db60 0x7fdf540b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3282107781 0 0) 0x7fdf40005730 con 0x7fdf5406c3a0 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 >> v2:172.21.15.181:3300/0 conn(0x7fdf5406db60 msgr2=0x7fdf540b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 --2- 172.21.15.70:0/4185607801 >> v2:172.21.15.181:3300/0 conn(0x7fdf5406db60 0x7fdf540b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 >> v1:172.21.15.181:6789/0 conn(0x7fdf540c9cf0 legacy=0x7fdf5415aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540c9860 con 0x7fdf5406c3a0 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fdf44001800 con 0x7fdf5406c3a0 2023-10-08T05:56:16.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 >> v1:172.21.15.70:6789/0 conn(0x7fdf5406c3a0 legacy=0x7fdf5406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 --2- 172.21.15.70:0/4185607801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf40005bd0 0x7fdf40005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 --2- 172.21.15.70:0/4185607801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf400069d0 0x7fdf40006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 --2- 172.21.15.70:0/4185607801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf400073e0 0x7fdf4000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5415b110 con 0x7fdf400069d0 2023-10-08T05:56:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4000e330 con 0x7fdf40005bd0 2023-10-08T05:56:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4000e4f0 con 0x7fdf400073e0 2023-10-08T05:56:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf4bfff640 1 --2- 172.21.15.70:0/4185607801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf400069d0 0x7fdf40006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf53fff640 1 --2- 172.21.15.70:0/4185607801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf40005bd0 0x7fdf40005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf4bfff640 1 -- 172.21.15.70:0/4185607801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf400073e0 msgr2=0x7fdf4000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:16.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf4bfff640 1 --2- 172.21.15.70:0/4185607801 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf400073e0 0x7fdf4000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf4bfff640 1 -- 172.21.15.70:0/4185607801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf40005bd0 msgr2=0x7fdf40005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:16.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.809+0000 7fdf4bfff640 1 --2- 172.21.15.70:0/4185607801 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf40005bd0 0x7fdf40005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.810+0000 7fdf4bfff640 1 -- 172.21.15.70:0/4185607801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf40005730 con 0x7fdf400069d0 2023-10-08T05:56:16.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.810+0000 7fdf58c12640 1 -- 172.21.15.70:0/4185607801 reap_dead start 2023-10-08T05:56:16.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.811+0000 7fdf4bfff640 1 --2- 172.21.15.70:0/4185607801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf400069d0 0x7fdf40006e50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c001750 tx=0x7fdf40007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9a15a5a79ef679 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:16.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.811+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0052d0 con 0x7fdf400069d0 2023-10-08T05:56:16.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.811+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c02e050 con 0x7fdf400069d0 2023-10-08T05:56:16.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.811+0000 7fdf537fe640 1 -- 172.21.15.70:0/4185607801 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c003830 con 0x7fdf400069d0 2023-10-08T05:56:16.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.811+0000 7fdf5a675640 1 -- 172.21.15.70:0/4185607801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf400069d0 msgr2=0x7fdf40006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.811+0000 7fdf5a675640 1 --2- 172.21.15.70:0/4185607801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf400069d0 0x7fdf40006e50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c001750 tx=0x7fdf40007a00 comp rx=0 tx=0).stop 2023-10-08T05:56:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.812+0000 7fdf5a675640 1 -- 172.21.15.70:0/4185607801 shutdown_connections 2023-10-08T05:56:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.812+0000 7fdf5a675640 1 --2- 172.21.15.70:0/4185607801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf400069d0 0x7fdf40006e50 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.812+0000 7fdf5a675640 1 -- 172.21.15.70:0/4185607801 >> 172.21.15.70:0/4185607801 conn(0x7fdf5406a5d0 msgr2=0x7fdf540b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.812+0000 7fdf5a675640 1 -- 172.21.15.70:0/4185607801 shutdown_connections 2023-10-08T05:56:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.812+0000 7fdf5a675640 1 -- 172.21.15.70:0/4185607801 wait complete. 2023-10-08T05:56:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.813+0000 7fdf5a675640 1 Processor -- start 2023-10-08T05:56:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.814+0000 7fdf5a675640 1 -- start start 2023-10-08T05:56:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.814+0000 7fdf5a675640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf400069d0 0x7fdf540cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.814+0000 7fdf5a675640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5414bcb0 0x7fdf540c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.814+0000 7fdf5a675640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf540c4730 0x7fdf540c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.814+0000 7fdf5a675640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540c5120 con 0x7fdf5414bcb0 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.814+0000 7fdf5a675640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540c3c30 con 0x7fdf400069d0 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.814+0000 7fdf5a675640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5414c130 con 0x7fdf540c4730 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf4bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5414bcb0 0x7fdf540c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf4bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5414bcb0 0x7fdf540c36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34664/0 (socket says 172.21.15.70:34664) 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf4bfff640 1 -- 172.21.15.70:0/2124689513 learned_addr learned my addr 172.21.15.70:0/2124689513 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf53fff640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf400069d0 0x7fdf540cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf58c12640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf540c4730 0x7fdf540c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf4bfff640 1 -- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf540c4730 msgr2=0x7fdf540c4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf4bfff640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf540c4730 0x7fdf540c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf4bfff640 1 -- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf400069d0 msgr2=0x7fdf540cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:16.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf4bfff640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf400069d0 0x7fdf540cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:16.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf4bfff640 1 -- 172.21.15.70:0/2124689513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540c2650 con 0x7fdf5414bcb0 2023-10-08T05:56:16.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf53fff640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf400069d0 0x7fdf540cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:16.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.815+0000 7fdf4bfff640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5414bcb0 0x7fdf540c36f0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c000c00 tx=0x7fdf3c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:16.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.816+0000 7fdf517fa640 1 -- 172.21.15.70:0/2124689513 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c003e80 con 0x7fdf5414bcb0 2023-10-08T05:56:16.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.816+0000 7fdf517fa640 1 -- 172.21.15.70:0/2124689513 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c007a30 con 0x7fdf5414bcb0 2023-10-08T05:56:16.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.816+0000 7fdf5a675640 1 -- 172.21.15.70:0/2124689513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf540c2930 con 0x7fdf5414bcb0 2023-10-08T05:56:16.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.816+0000 7fdf517fa640 1 -- 172.21.15.70:0/2124689513 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c03d630 con 0x7fdf5414bcb0 2023-10-08T05:56:16.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.816+0000 7fdf5a675640 1 -- 172.21.15.70:0/2124689513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf540c2ef0 con 0x7fdf5414bcb0 2023-10-08T05:56:16.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.817+0000 7fdf5a675640 1 -- 172.21.15.70:0/2124689513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf540c7060 con 0x7fdf5414bcb0 2023-10-08T05:56:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.818+0000 7fdf517fa640 1 -- 172.21.15.70:0/2124689513 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdf3c03d790 con 0x7fdf5414bcb0 2023-10-08T05:56:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.819+0000 7fdf517fa640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdf3406afc0 0x7fdf3406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.820+0000 7fdf517fa640 1 -- 172.21.15.70:0/2124689513 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fdf3c0ab710 con 0x7fdf5414bcb0 2023-10-08T05:56:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.820+0000 7fdf53fff640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdf3406afc0 0x7fdf3406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.821+0000 7fdf53fff640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdf3406afc0 0x7fdf3406d480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdf44000c00 tx=0x7fdf44001dd0 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:16.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:16.824+0000 7fdf517fa640 1 -- 172.21.15.70:0/2124689513 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf3c0765c0 con 0x7fdf5414bcb0 2023-10-08T05:56:17.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.015+0000 7fdf5a675640 1 -- 172.21.15.70:0/2124689513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fdf5404e160 con 0x7fdf5414bcb0 2023-10-08T05:56:17.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.021+0000 7fdf517fa640 1 -- 172.21.15.70:0/2124689513 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v45) v1 ==== 120+0+0 (secure 0 0 0) 0x7fdf3c079be0 con 0x7fdf5414bcb0 2023-10-08T05:56:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 -- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdf3406afc0 msgr2=0x7fdf3406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdf3406afc0 0x7fdf3406d480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdf44000c00 tx=0x7fdf44001dd0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 -- 172.21.15.70:0/2124689513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5414bcb0 msgr2=0x7fdf540c36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5414bcb0 0x7fdf540c36f0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c000c00 tx=0x7fdf3c001120 comp rx=0 tx=0).stop 2023-10-08T05:56:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 -- 172.21.15.70:0/2124689513 shutdown_connections 2023-10-08T05:56:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdf3406afc0 0x7fdf3406d480 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf540c4730 0x7fdf540c4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5414bcb0 0x7fdf540c36f0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 --2- 172.21.15.70:0/2124689513 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf400069d0 0x7fdf540cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 -- 172.21.15.70:0/2124689513 >> 172.21.15.70:0/2124689513 conn(0x7fdf5406a5d0 msgr2=0x7fdf540c6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 -- 172.21.15.70:0/2124689513 shutdown_connections 2023-10-08T05:56:17.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.024+0000 7fdf5a675640 1 -- 172.21.15.70:0/2124689513 wait complete. 2023-10-08T05:56:17.044 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-08T05:56:17.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.253+0000 7f5f5b983640 1 Processor -- start 2023-10-08T05:56:17.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.253+0000 7f5f5b983640 1 -- start start 2023-10-08T05:56:17.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.255+0000 7f5f5b983640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5f5415ab40 0x7f5f540ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.255+0000 7f5f5b983640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f540c90b0 0x7f5f540adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.255+0000 7f5f5b983640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f540cae70 0x7f5f540cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.255+0000 7f5f5b983640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5415bf30 con 0x7f5f5415ab40 2023-10-08T05:56:17.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.255+0000 7f5f5b983640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f540cb840 con 0x7f5f540c90b0 2023-10-08T05:56:17.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.255+0000 7f5f5b983640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5415b2b0 con 0x7f5f540cae70 2023-10-08T05:56:17.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.255+0000 7f5f5971f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5f5415ab40 0x7f5f540ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.255+0000 7f5f5971f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5f5415ab40 0x7f5f540ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34678/0 (socket says 172.21.15.70:34678) 2023-10-08T05:56:17.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.255+0000 7f5f5971f640 1 -- 172.21.15.70:0/3110972219 learned_addr learned my addr 172.21.15.70:0/3110972219 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:17.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.256+0000 7f5f58f1e640 1 --2- 172.21.15.70:0/3110972219 >> v2:172.21.15.181:3300/0 conn(0x7f5f540c90b0 0x7f5f540adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.256+0000 7f5f59f20640 1 --2- 172.21.15.70:0/3110972219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f540cae70 0x7f5f540cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.256+0000 7f5f5971f640 1 -- 172.21.15.70:0/3110972219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f540cae70 msgr2=0x7f5f540cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:17.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.256+0000 7f5f5971f640 1 --2- 172.21.15.70:0/3110972219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f540cae70 0x7f5f540cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.256+0000 7f5f5971f640 1 -- 172.21.15.70:0/3110972219 >> v2:172.21.15.181:3300/0 conn(0x7f5f540c90b0 msgr2=0x7f5f540adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:17.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.256+0000 7f5f5971f640 1 --2- 172.21.15.70:0/3110972219 >> v2:172.21.15.181:3300/0 conn(0x7f5f540c90b0 0x7f5f540adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.256+0000 7f5f5971f640 1 -- 172.21.15.70:0/3110972219 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5415b590 con 0x7f5f5415ab40 2023-10-08T05:56:17.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.256+0000 7f5f59f20640 1 --2- 172.21.15.70:0/3110972219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f540cae70 0x7f5f540cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:17.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.257+0000 7f5f5971f640 1 --2- 172.21.15.70:0/3110972219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5415ab40 0x7f5f540ad720 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f5f5406e560 tx=0x7f5f5002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf1e3435b379a54 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:17.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.257+0000 7f5f4bfff640 1 -- 172.21.15.70:0/3110972219 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5002ca80 con 0x7f5f5415ab40 2023-10-08T05:56:17.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.257+0000 7f5f4bfff640 1 -- 172.21.15.70:0/3110972219 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5002cbe0 con 0x7f5f5415ab40 2023-10-08T05:56:17.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.257+0000 7f5f4bfff640 1 -- 172.21.15.70:0/3110972219 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f500343f0 con 0x7f5f5415ab40 2023-10-08T05:56:17.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.258+0000 7f5f5b983640 1 -- 172.21.15.70:0/3110972219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5415ab40 msgr2=0x7f5f540ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.258+0000 7f5f5b983640 1 --2- 172.21.15.70:0/3110972219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5415ab40 0x7f5f540ad720 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f5f5406e560 tx=0x7f5f5002bee0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.258+0000 7f5f5b983640 1 -- 172.21.15.70:0/3110972219 shutdown_connections 2023-10-08T05:56:17.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.258+0000 7f5f5b983640 1 --2- 172.21.15.70:0/3110972219 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f540cae70 0x7f5f540cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.258+0000 7f5f5b983640 1 --2- 172.21.15.70:0/3110972219 >> v2:172.21.15.181:3300/0 conn(0x7f5f540c90b0 0x7f5f540adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.258+0000 7f5f5b983640 1 --2- 172.21.15.70:0/3110972219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5415ab40 0x7f5f540ad720 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.258+0000 7f5f5b983640 1 -- 172.21.15.70:0/3110972219 >> 172.21.15.70:0/3110972219 conn(0x7f5f5406a380 msgr2=0x7f5f540b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:17.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.258+0000 7f5f5b983640 1 -- 172.21.15.70:0/3110972219 shutdown_connections 2023-10-08T05:56:17.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.258+0000 7f5f5b983640 1 -- 172.21.15.70:0/3110972219 wait complete. 2023-10-08T05:56:17.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.260+0000 7f5f5b983640 1 Processor -- start 2023-10-08T05:56:17.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.260+0000 7f5f5b983640 1 -- start start 2023-10-08T05:56:17.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5b983640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f540c90b0 0x7f5f541558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5b983640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f540cae70 0x7f5f540b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5b983640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f5415ab40 0x7f5f540aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5b983640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540c5850 con 0x7f5f540c90b0 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5b983640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540c5b30 con 0x7f5f540cae70 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5b983640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540c5e10 con 0x7f5f5415ab40 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5971f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f540c90b0 0x7f5f541558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5971f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f540c90b0 0x7f5f541558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34684/0 (socket says 172.21.15.70:34684) 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5971f640 1 -- 172.21.15.70:0/3314023912 learned_addr learned my addr 172.21.15.70:0/3314023912 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5971f640 1 -- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f5415ab40 msgr2=0x7f5f540aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5971f640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f5415ab40 0x7f5f540aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5971f640 1 -- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f540cae70 msgr2=0x7f5f540b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5971f640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f540cae70 0x7f5f540b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.261+0000 7f5f5971f640 1 -- 172.21.15.70:0/3314023912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f50009b30 con 0x7f5f540c90b0 2023-10-08T05:56:17.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.262+0000 7f5f5971f640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f540c90b0 0x7f5f541558d0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f5f50000f80 tx=0x7f5f50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:17.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.262+0000 7f5f49ffb640 1 -- 172.21.15.70:0/3314023912 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50009510 con 0x7f5f540c90b0 2023-10-08T05:56:17.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.262+0000 7f5f49ffb640 1 -- 172.21.15.70:0/3314023912 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5003e450 con 0x7f5f540c90b0 2023-10-08T05:56:17.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.262+0000 7f5f5b983640 1 -- 172.21.15.70:0/3314023912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f540c60f0 con 0x7f5f540c90b0 2023-10-08T05:56:17.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.262+0000 7f5f49ffb640 1 -- 172.21.15.70:0/3314023912 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5003e800 con 0x7f5f540c90b0 2023-10-08T05:56:17.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.262+0000 7f5f5b983640 1 -- 172.21.15.70:0/3314023912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f54155e10 con 0x7f5f540c90b0 2023-10-08T05:56:17.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.263+0000 7f5f5b983640 1 -- 172.21.15.70:0/3314023912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f540c65b0 con 0x7f5f540c90b0 2023-10-08T05:56:17.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.264+0000 7f5f49ffb640 1 -- 172.21.15.70:0/3314023912 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5f5002cd50 con 0x7f5f540c90b0 2023-10-08T05:56:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.265+0000 7f5f49ffb640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f5f3806af60 0x7f5f3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.265+0000 7f5f49ffb640 1 -- 172.21.15.70:0/3314023912 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f5f500ac540 con 0x7f5f540c90b0 2023-10-08T05:56:17.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.266+0000 7f5f58f1e640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f5f3806af60 0x7f5f3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.267+0000 7f5f58f1e640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f5f3806af60 0x7f5f3806d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f540b3c10 tx=0x7f5f44008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:17.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.269+0000 7f5f49ffb640 1 -- 172.21.15.70:0/3314023912 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f500772b0 con 0x7f5f540c90b0 2023-10-08T05:56:17.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.528+0000 7f5f5b983640 1 -- 172.21.15.70:0/3314023912 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f5f54067490 con 0x7f5f540c90b0 2023-10-08T05:56:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.673+0000 7f5f49ffb640 1 -- 172.21.15.70:0/3314023912 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v781) v1 ==== 148+0+0 (secure 0 0 0) 0x7f5f5003e960 con 0x7f5f540c90b0 2023-10-08T05:56:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 -- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f5f3806af60 msgr2=0x7f5f3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f5f3806af60 0x7f5f3806d420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5f540b3c10 tx=0x7f5f44008040 comp rx=0 tx=0).stop 2023-10-08T05:56:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 -- 172.21.15.70:0/3314023912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f540c90b0 msgr2=0x7f5f541558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f540c90b0 0x7f5f541558d0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f5f50000f80 tx=0x7f5f50004a40 comp rx=0 tx=0).stop 2023-10-08T05:56:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 -- 172.21.15.70:0/3314023912 shutdown_connections 2023-10-08T05:56:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f5f3806af60 0x7f5f3806d420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f5415ab40 0x7f5f540aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f540cae70 0x7f5f540b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 --2- 172.21.15.70:0/3314023912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f540c90b0 0x7f5f541558d0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 -- 172.21.15.70:0/3314023912 >> 172.21.15.70:0/3314023912 conn(0x7f5f5406a380 msgr2=0x7f5f540ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 -- 172.21.15.70:0/3314023912 shutdown_connections 2023-10-08T05:56:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.676+0000 7f5f5b983640 1 -- 172.21.15.70:0/3314023912 wait complete. 2023-10-08T05:56:17.695 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-10-08T05:56:17.695 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T05:56:17.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:17.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.902+0000 7efe63283640 1 Processor -- start 2023-10-08T05:56:17.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.903+0000 7efe63283640 1 -- start start 2023-10-08T05:56:17.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.904+0000 7efe63283640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe5c15cd90 0x7efe5c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.904+0000 7efe63283640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe5c160940 0x7efe5c070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.904+0000 7efe63283640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe5c0c48f0 con 0x7efe5c070d50 2023-10-08T05:56:17.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.904+0000 7efe63283640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe5c0c4d80 con 0x7efe5c15cd90 2023-10-08T05:56:17.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.904+0000 7efe63283640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0c3ce0 con 0x7efe5c160940 2023-10-08T05:56:17.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe61820640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efe5c070d50 0x7efe5c0cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53544/0 (socket says 172.21.15.70:53544) 2023-10-08T05:56:17.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe61820640 1 -- 172.21.15.70:0/4264097144 learned_addr learned my addr 172.21.15.70:0/4264097144 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:17.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe6101f640 1 --2- 172.21.15.70:0/4264097144 >> v2:172.21.15.181:3300/0 conn(0x7efe5c15cd90 0x7efe5c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe6081e640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe5c160940 0x7efe5c070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006386220 0 0) 0x7efe5c0c48f0 con 0x7efe5c070d50 2023-10-08T05:56:17.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe48003520 con 0x7efe5c070d50 2023-10-08T05:56:17.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 722881865 0 0) 0x7efe48003520 con 0x7efe5c070d50 2023-10-08T05:56:17.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe5c160940 msgr2=0x7efe5c070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:17.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe53fff640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe5c160940 0x7efe5c070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 >> v2:172.21.15.181:3300/0 conn(0x7efe5c15cd90 msgr2=0x7efe5c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:17.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe53fff640 1 --2- 172.21.15.70:0/4264097144 >> v2:172.21.15.181:3300/0 conn(0x7efe5c15cd90 0x7efe5c0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0c3fc0 con 0x7efe5c070d50 2023-10-08T05:56:17.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.905+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7efe540019f0 con 0x7efe5c070d50 2023-10-08T05:56:17.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 >> v1:172.21.15.70:6789/0 conn(0x7efe5c070d50 legacy=0x7efe5c0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe53fff640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe5c160940 0x7efe48003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe53fff640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48003f40 0x7efe480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe53fff640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe48004d00 0x7efe48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0c3ce0 con 0x7efe48003f40 2023-10-08T05:56:17.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4800b9b0 con 0x7efe48004d00 2023-10-08T05:56:17.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4800bb70 con 0x7efe5c160940 2023-10-08T05:56:17.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe6081e640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48003f40 0x7efe480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe6101f640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe5c160940 0x7efe48003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe6081e640 1 -- 172.21.15.70:0/4264097144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe5c160940 msgr2=0x7efe48003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:17.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe6081e640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe5c160940 0x7efe48003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe6081e640 1 -- 172.21.15.70:0/4264097144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe48004d00 msgr2=0x7efe48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:17.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe6081e640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe48004d00 0x7efe48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.906+0000 7efe6081e640 1 -- 172.21.15.70:0/4264097144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe48003520 con 0x7efe48003f40 2023-10-08T05:56:17.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.907+0000 7efe6081e640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48003f40 0x7efe480047c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00ba30 tx=0x7efe480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49916cb3aeaa9872 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:17.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.908+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c032640 con 0x7efe48003f40 2023-10-08T05:56:17.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.908+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c0327a0 con 0x7efe48003f40 2023-10-08T05:56:17.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.908+0000 7efe53fff640 1 -- 172.21.15.70:0/4264097144 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c032b30 con 0x7efe48003f40 2023-10-08T05:56:17.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.908+0000 7efe63283640 1 -- 172.21.15.70:0/4264097144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48003f40 msgr2=0x7efe480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.908+0000 7efe63283640 1 --2- 172.21.15.70:0/4264097144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48003f40 0x7efe480047c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00ba30 tx=0x7efe480057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.908+0000 7efe61820640 1 -- 172.21.15.70:0/4264097144 reap_dead start 2023-10-08T05:56:17.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.908+0000 7efe63283640 1 -- 172.21.15.70:0/4264097144 shutdown_connections 2023-10-08T05:56:17.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.908+0000 7efe63283640 1 -- 172.21.15.70:0/4264097144 >> 172.21.15.70:0/4264097144 conn(0x7efe5c06a380 msgr2=0x7efe5c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:17.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.908+0000 7efe63283640 1 -- 172.21.15.70:0/4264097144 shutdown_connections 2023-10-08T05:56:17.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.908+0000 7efe63283640 1 -- 172.21.15.70:0/4264097144 wait complete. 2023-10-08T05:56:17.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.910+0000 7efe63283640 1 Processor -- start 2023-10-08T05:56:17.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.910+0000 7efe63283640 1 -- start start 2023-10-08T05:56:17.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.911+0000 7efe63283640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe48003f40 0x7efe5c15fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.911+0000 7efe63283640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c1f6be0 0x7efe5c0b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.911+0000 7efe63283640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe5c1f6040 0x7efe5c1f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.911+0000 7efe63283640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c1f6a00 con 0x7efe5c1f6040 2023-10-08T05:56:17.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.911+0000 7efe63283640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0b3a90 con 0x7efe5c1f6be0 2023-10-08T05:56:17.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.911+0000 7efe63283640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0b3d30 con 0x7efe48003f40 2023-10-08T05:56:17.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.911+0000 7efe61820640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe5c1f6040 0x7efe5c1f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.911+0000 7efe61820640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe5c1f6040 0x7efe5c1f64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34706/0 (socket says 172.21.15.70:34706) 2023-10-08T05:56:17.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.911+0000 7efe61820640 1 -- 172.21.15.70:0/731475255 learned_addr learned my addr 172.21.15.70:0/731475255 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:17.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe6081e640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c1f6be0 0x7efe5c0b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe6101f640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe48003f40 0x7efe5c15fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe61820640 1 -- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe48003f40 msgr2=0x7efe5c15fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe61820640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe48003f40 0x7efe5c15fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe61820640 1 -- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c1f6be0 msgr2=0x7efe5c0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:17.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe61820640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c1f6be0 0x7efe5c0b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:17.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe61820640 1 -- 172.21.15.70:0/731475255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0b4010 con 0x7efe5c1f6040 2023-10-08T05:56:17.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe61820640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe5c1f6040 0x7efe5c1f64c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7efe5c06abe0 tx=0x7efe5400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:17.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe51ffb640 1 -- 172.21.15.70:0/731475255 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5400cea0 con 0x7efe5c1f6040 2023-10-08T05:56:17.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe51ffb640 1 -- 172.21.15.70:0/731475255 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5400a610 con 0x7efe5c1f6040 2023-10-08T05:56:17.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe51ffb640 1 -- 172.21.15.70:0/731475255 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5400a9c0 con 0x7efe5c1f6040 2023-10-08T05:56:17.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.912+0000 7efe63283640 1 -- 172.21.15.70:0/731475255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c0b42f0 con 0x7efe5c1f6040 2023-10-08T05:56:17.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.913+0000 7efe63283640 1 -- 172.21.15.70:0/731475255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c0b1910 con 0x7efe5c1f6040 2023-10-08T05:56:17.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.914+0000 7efe63283640 1 -- 172.21.15.70:0/731475255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5c067490 con 0x7efe5c1f6040 2023-10-08T05:56:17.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.915+0000 7efe51ffb640 1 -- 172.21.15.70:0/731475255 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efe5400ab20 con 0x7efe5c1f6040 2023-10-08T05:56:17.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.916+0000 7efe51ffb640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efe4006afc0 0x7efe4006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:17.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.916+0000 7efe51ffb640 1 -- 172.21.15.70:0/731475255 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6109+0+0 (secure 0 0 0) 0x7efe54089110 con 0x7efe5c1f6040 2023-10-08T05:56:17.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.917+0000 7efe6101f640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efe4006afc0 0x7efe4006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:17.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.918+0000 7efe6101f640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efe4006afc0 0x7efe4006d480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe58009a90 tx=0x7efe58008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:17.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:17.921+0000 7efe51ffb640 1 -- 172.21.15.70:0/731475255 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe54053f10 con 0x7efe5c1f6040 2023-10-08T05:56:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.093+0000 7efe63283640 1 -- 172.21.15.70:0/731475255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe5c0cc830 con 0x7efe5c1f6040 2023-10-08T05:56:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.094+0000 7efe51ffb640 1 -- 172.21.15.70:0/731475255 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+14444 (secure 0 0 0) 0x7efe5401a090 con 0x7efe5c1f6040 2023-10-08T05:56:18.096 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:18.097 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":140,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:15.651658+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3688966653":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1080150247":"2023-10-09T05:56:08.750606+0000","172.21.15.181:6833/1786188476":"2023-10-09T05:56:08.750606+0000","172.21.15.181:6832/1786188476":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1214793362":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/497155213":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/67726193":"2023-10-09T05:56:08.750606+0000","172.21.15.70:6832/3020884355":"2023-10-09T05:56:12.550608+0000","172.21.15.181:0/1353868644":"2023-10-09T05:56:08.750606+0000","172.21.15.70:6833/3020884355":"2023-10-09T05:56:12.550608+0000","172.21.15.181:0/74854201":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/2395612444":"2023-10-09T05:56:08.750606+0000","172.21.15.181:0/1556900853":"2023-10-09T05:55:39.473892+0000","172.21.15.181:6832/3142216408":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/4173673684":"2023-10-09T05:55:39.473892+0000","172.21.15.181:6833/3142216408":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/1464588158":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/791649123":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/2668737378":"2023-10-09T05:55:39.473892+0000","172.21.15.181:0/3044665941":"2023-10-09T05:55:39.473892+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:18.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 -- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efe4006afc0 msgr2=0x7efe4006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efe4006afc0 0x7efe4006d480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe58009a90 tx=0x7efe58008040 comp rx=0 tx=0).stop 2023-10-08T05:56:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 -- 172.21.15.70:0/731475255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe5c1f6040 msgr2=0x7efe5c1f64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe5c1f6040 0x7efe5c1f64c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7efe5c06abe0 tx=0x7efe5400cc50 comp rx=0 tx=0).stop 2023-10-08T05:56:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 -- 172.21.15.70:0/731475255 shutdown_connections 2023-10-08T05:56:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efe4006afc0 0x7efe4006d480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe5c1f6040 0x7efe5c1f64c0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe5c1f6be0 0x7efe5c0b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 --2- 172.21.15.70:0/731475255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe48003f40 0x7efe5c15fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 -- 172.21.15.70:0/731475255 >> 172.21.15.70:0/731475255 conn(0x7efe5c06a380 msgr2=0x7efe5c0ab950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 -- 172.21.15.70:0/731475255 shutdown_connections 2023-10-08T05:56:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.097+0000 7efe63283640 1 -- 172.21.15.70:0/731475255 wait complete. 2023-10-08T05:56:18.116 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:56:18.116 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:56:18.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.332+0000 7fe08acbb640 1 Processor -- start 2023-10-08T05:56:18.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.332+0000 7fe08acbb640 1 -- start start 2023-10-08T05:56:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.334+0000 7fe08acbb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe084070b50 0x7fe0840aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.334+0000 7fe08acbb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08415ae90 con 0x7fe08415aab0 2023-10-08T05:56:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.334+0000 7fe08acbb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0840cbdb0 con 0x7fe0840b04a0 2023-10-08T05:56:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.334+0000 7fe08acbb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0840cb600 con 0x7fe084070b50 2023-10-08T05:56:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.334+0000 7fe0894b8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe084070b50 0x7fe0840aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.334+0000 7fe0894b8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe084070b50 0x7fe0840aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34710/0 (socket says 172.21.15.70:34710) 2023-10-08T05:56:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.334+0000 7fe0894b8640 1 -- 172.21.15.70:0/18750074 learned_addr learned my addr 172.21.15.70:0/18750074 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.334+0000 7fe089cb9640 1 --1- 172.21.15.70:0/18750074 >> v1:172.21.15.70:6789/0 conn(0x7fe08415aab0 0x7fe0840caef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53550/0 (socket says 172.21.15.70:53550) 2023-10-08T05:56:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.335+0000 7fe0894b8640 1 -- 172.21.15.70:0/18750074 >> v1:172.21.15.181:6789/0 conn(0x7fe0840b04a0 legacy=0x7fe0840b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.335+0000 7fe0894b8640 1 -- 172.21.15.70:0/18750074 >> v1:172.21.15.70:6789/0 conn(0x7fe08415aab0 legacy=0x7fe0840caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.335+0000 7fe0894b8640 1 -- 172.21.15.70:0/18750074 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840cb7f0 con 0x7fe084070b50 2023-10-08T05:56:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.336+0000 7fe0894b8640 1 --2- 172.21.15.70:0/18750074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe084070b50 0x7fe0840aff60 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe0740098a0 tx=0x7fe07402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e767d8970b35e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.336+0000 7fe088cb7640 1 -- 172.21.15.70:0/18750074 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074031640 con 0x7fe084070b50 2023-10-08T05:56:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.336+0000 7fe088cb7640 1 -- 172.21.15.70:0/18750074 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0740317a0 con 0x7fe084070b50 2023-10-08T05:56:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.336+0000 7fe088cb7640 1 -- 172.21.15.70:0/18750074 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074031b30 con 0x7fe084070b50 2023-10-08T05:56:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.337+0000 7fe08acbb640 1 -- 172.21.15.70:0/18750074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe084070b50 msgr2=0x7fe0840aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.337+0000 7fe08acbb640 1 --2- 172.21.15.70:0/18750074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe084070b50 0x7fe0840aff60 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe0740098a0 tx=0x7fe07402b8f0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.337+0000 7fe08acbb640 1 -- 172.21.15.70:0/18750074 shutdown_connections 2023-10-08T05:56:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.337+0000 7fe08acbb640 1 --2- 172.21.15.70:0/18750074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe084070b50 0x7fe0840aff60 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.337+0000 7fe08acbb640 1 -- 172.21.15.70:0/18750074 >> 172.21.15.70:0/18750074 conn(0x7fe08406c010 msgr2=0x7fe08406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.337+0000 7fe08acbb640 1 -- 172.21.15.70:0/18750074 shutdown_connections 2023-10-08T05:56:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.337+0000 7fe08acbb640 1 -- 172.21.15.70:0/18750074 wait complete. 2023-10-08T05:56:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.339+0000 7fe08acbb640 1 Processor -- start 2023-10-08T05:56:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.339+0000 7fe08acbb640 1 -- start start 2023-10-08T05:56:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.340+0000 7fe08acbb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe084070b50 0x7fe0840c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.340+0000 7fe08acbb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0840b04a0 0x7fe084148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.340+0000 7fe08acbb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08415aab0 0x7fe0840ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.340+0000 7fe08acbb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084148d60 con 0x7fe08415aab0 2023-10-08T05:56:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.340+0000 7fe08acbb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840c60f0 con 0x7fe0840b04a0 2023-10-08T05:56:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.340+0000 7fe08acbb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840ad780 con 0x7fe084070b50 2023-10-08T05:56:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.340+0000 7fe08a4ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08415aab0 0x7fe0840ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.340+0000 7fe08a4ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08415aab0 0x7fe0840ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34714/0 (socket says 172.21.15.70:34714) 2023-10-08T05:56:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.340+0000 7fe08a4ba640 1 -- 172.21.15.70:0/321902422 learned_addr learned my addr 172.21.15.70:0/321902422 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.340+0000 7fe0894b8640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0840b04a0 0x7fe084148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe089cb9640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe084070b50 0x7fe0840c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe08a4ba640 1 -- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe084070b50 msgr2=0x7fe0840c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe08a4ba640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe084070b50 0x7fe0840c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe08a4ba640 1 -- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0840b04a0 msgr2=0x7fe084148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe08a4ba640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0840b04a0 0x7fe084148820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe08a4ba640 1 -- 172.21.15.70:0/321902422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0740094d0 con 0x7fe08415aab0 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe089cb9640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe084070b50 0x7fe0840c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe08a4ba640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08415aab0 0x7fe0840ac6a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00c7b0 tx=0x7fe07c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe07a7fc640 1 -- 172.21.15.70:0/321902422 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c004b60 con 0x7fe08415aab0 2023-10-08T05:56:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe07a7fc640 1 -- 172.21.15.70:0/321902422 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c004cc0 con 0x7fe08415aab0 2023-10-08T05:56:18.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe07a7fc640 1 -- 172.21.15.70:0/321902422 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00aa90 con 0x7fe08415aab0 2023-10-08T05:56:18.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe08acbb640 1 -- 172.21.15.70:0/321902422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0840c6320 con 0x7fe08415aab0 2023-10-08T05:56:18.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.341+0000 7fe08acbb640 1 -- 172.21.15.70:0/321902422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0840ad1c0 con 0x7fe08415aab0 2023-10-08T05:56:18.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.342+0000 7fe08acbb640 1 -- 172.21.15.70:0/321902422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe084074150 con 0x7fe08415aab0 2023-10-08T05:56:18.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.344+0000 7fe07a7fc640 1 -- 172.21.15.70:0/321902422 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe07c00abf0 con 0x7fe08415aab0 2023-10-08T05:56:18.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.345+0000 7fe07a7fc640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe06806af90 0x7fe06806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.345+0000 7fe07a7fc640 1 -- 172.21.15.70:0/321902422 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fe07c089af0 con 0x7fe08415aab0 2023-10-08T05:56:18.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.345+0000 7fe089cb9640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe06806af90 0x7fe06806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:18.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.346+0000 7fe089cb9640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe06806af90 0x7fe06806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe080000f60 tx=0x7fe080009040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.349+0000 7fe07a7fc640 1 -- 172.21.15.70:0/321902422 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe07c054870 con 0x7fe08415aab0 2023-10-08T05:56:18.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.536+0000 7fe08acbb640 1 -- 172.21.15.70:0/321902422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0840c64b0 con 0x7fe08415aab0 2023-10-08T05:56:18.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.537+0000 7fe07a7fc640 1 -- 172.21.15.70:0/321902422 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe07c057e90 con 0x7fe08415aab0 2023-10-08T05:56:18.538 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:56:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 -- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe06806af90 msgr2=0x7fe06806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe06806af90 0x7fe06806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe080000f60 tx=0x7fe080009040 comp rx=0 tx=0).stop 2023-10-08T05:56:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 -- 172.21.15.70:0/321902422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08415aab0 msgr2=0x7fe0840ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08415aab0 0x7fe0840ac6a0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00c7b0 tx=0x7fe07c00cc70 comp rx=0 tx=0).stop 2023-10-08T05:56:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 -- 172.21.15.70:0/321902422 shutdown_connections 2023-10-08T05:56:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe06806af90 0x7fe06806d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08415aab0 0x7fe0840ac6a0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0840b04a0 0x7fe084148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 --2- 172.21.15.70:0/321902422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe084070b50 0x7fe0840c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 -- 172.21.15.70:0/321902422 >> 172.21.15.70:0/321902422 conn(0x7fe08406c010 msgr2=0x7fe084159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.539+0000 7fe08acbb640 1 -- 172.21.15.70:0/321902422 shutdown_connections 2023-10-08T05:56:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.540+0000 7fe08acbb640 1 -- 172.21.15.70:0/321902422 wait complete. 2023-10-08T05:56:18.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2023-10-08T05:56:18.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.774+0000 7fa6d6ec2640 1 Processor -- start 2023-10-08T05:56:18.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.774+0000 7fa6d6ec2640 1 -- start start 2023-10-08T05:56:18.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.775+0000 7fa6d6ec2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa6d006d4c0 0x7fa6d0162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.775+0000 7fa6d6ec2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa6d00cb150 0x7fa6d00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.776+0000 7fa6d6ec2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d00ae140 con 0x7fa6d00add60 2023-10-08T05:56:18.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.776+0000 7fa6d6ec2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d0161f10 con 0x7fa6d00cb150 2023-10-08T05:56:18.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.776+0000 7fa6d6ec2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d00cbb20 con 0x7fa6d006d4c0 2023-10-08T05:56:18.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.776+0000 7fa6d545f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa6d00cb150 0x7fa6d00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:18.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.776+0000 7fa6d545f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa6d00cb150 0x7fa6d00cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34716/0 (socket says 172.21.15.70:34716) 2023-10-08T05:56:18.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.776+0000 7fa6d545f640 1 -- 172.21.15.70:0/1454534661 learned_addr learned my addr 172.21.15.70:0/1454534661 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:18.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.776+0000 7fa6d545f640 1 -- 172.21.15.70:0/1454534661 >> v2:172.21.15.181:3300/0 conn(0x7fa6d006d4c0 msgr2=0x7fa6d0162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:18.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.776+0000 7fa6d545f640 1 --2- 172.21.15.70:0/1454534661 >> v2:172.21.15.181:3300/0 conn(0x7fa6d006d4c0 0x7fa6d0162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.776+0000 7fa6d545f640 1 -- 172.21.15.70:0/1454534661 >> v1:172.21.15.181:6789/0 conn(0x7fa6d00add60 legacy=0x7fa6d00cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:18.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.776+0000 7fa6d545f640 1 -- 172.21.15.70:0/1454534661 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d00ccf20 con 0x7fa6d00cb150 2023-10-08T05:56:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.777+0000 7fa6d545f640 1 --2- 172.21.15.70:0/1454534661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 0x7fa6d00cb5e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa6d006e560 tx=0x7fa6c802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b6d251027af075 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.778+0000 7fa6cf7fe640 1 -- 172.21.15.70:0/1454534661 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80317e0 con 0x7fa6d00cb150 2023-10-08T05:56:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.778+0000 7fa6cf7fe640 1 -- 172.21.15.70:0/1454534661 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c8031940 con 0x7fa6d00cb150 2023-10-08T05:56:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.778+0000 7fa6cf7fe640 1 -- 172.21.15.70:0/1454534661 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8031cd0 con 0x7fa6d00cb150 2023-10-08T05:56:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.778+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/1454534661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 msgr2=0x7fa6d00cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.778+0000 7fa6d6ec2640 1 --2- 172.21.15.70:0/1454534661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 0x7fa6d00cb5e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa6d006e560 tx=0x7fa6c802b880 comp rx=0 tx=0).stop 2023-10-08T05:56:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.778+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/1454534661 shutdown_connections 2023-10-08T05:56:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.778+0000 7fa6d6ec2640 1 --2- 172.21.15.70:0/1454534661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 0x7fa6d00cb5e0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.778+0000 7fa6d6ec2640 1 --2- 172.21.15.70:0/1454534661 >> v2:172.21.15.181:3300/0 conn(0x7fa6d006d4c0 0x7fa6d0162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.778+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/1454534661 >> 172.21.15.70:0/1454534661 conn(0x7fa6d006a380 msgr2=0x7fa6d00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:18.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.778+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/1454534661 shutdown_connections 2023-10-08T05:56:18.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.779+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/1454534661 wait complete. 2023-10-08T05:56:18.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.780+0000 7fa6d6ec2640 1 Processor -- start 2023-10-08T05:56:18.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.780+0000 7fa6d6ec2640 1 -- start start 2023-10-08T05:56:18.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.781+0000 7fa6d6ec2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d4c0 0x7fa6d01f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.781+0000 7fa6d6ec2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d00add60 0x7fa6d01f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.781+0000 7fa6d6ec2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 0x7fa6d01fa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.781+0000 7fa6d6ec2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015aaa0 con 0x7fa6d00cb150 2023-10-08T05:56:18.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.781+0000 7fa6d6ec2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015b2b0 con 0x7fa6d00add60 2023-10-08T05:56:18.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.781+0000 7fa6d6ec2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015b590 con 0x7fa6d006d4c0 2023-10-08T05:56:18.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d545f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 0x7fa6d01fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:18.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d545f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 0x7fa6d01fa390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34726/0 (socket says 172.21.15.70:34726) 2023-10-08T05:56:18.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d545f640 1 -- 172.21.15.70:0/839068726 learned_addr learned my addr 172.21.15.70:0/839068726 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:18.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6cffff640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d00add60 0x7fa6d01f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:18.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d4c5e640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d4c0 0x7fa6d01f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:18.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d545f640 1 -- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d4c0 msgr2=0x7fa6d01f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d545f640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d4c0 0x7fa6d01f7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d545f640 1 -- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d00add60 msgr2=0x7fa6d01f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:18.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d545f640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d00add60 0x7fa6d01f9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d545f640 1 -- 172.21.15.70:0/839068726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c80094d0 con 0x7fa6d00cb150 2023-10-08T05:56:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d4c5e640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d4c0 0x7fa6d01f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.782+0000 7fa6d545f640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 0x7fa6d01fa390 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80090b0 tx=0x7fa6c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.783+0000 7fa6cd7fa640 1 -- 172.21.15.70:0/839068726 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8032940 con 0x7fa6d00cb150 2023-10-08T05:56:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.783+0000 7fa6cd7fa640 1 -- 172.21.15.70:0/839068726 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c8032ac0 con 0x7fa6d00cb150 2023-10-08T05:56:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.783+0000 7fa6cd7fa640 1 -- 172.21.15.70:0/839068726 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8032e70 con 0x7fa6d00cb150 2023-10-08T05:56:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.783+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/839068726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d015b870 con 0x7fa6d00cb150 2023-10-08T05:56:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.783+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/839068726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d015afa0 con 0x7fa6d00cb150 2023-10-08T05:56:18.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.784+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/839068726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d015bd90 con 0x7fa6d00cb150 2023-10-08T05:56:18.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.785+0000 7fa6cd7fa640 1 -- 172.21.15.70:0/839068726 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa6c804d3f0 con 0x7fa6d00cb150 2023-10-08T05:56:18.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.786+0000 7fa6cd7fa640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6bc06af90 0x7fa6bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:18.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.786+0000 7fa6cd7fa640 1 -- 172.21.15.70:0/839068726 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fa6c80ab920 con 0x7fa6d00cb150 2023-10-08T05:56:18.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.787+0000 7fa6d4c5e640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6bc06af90 0x7fa6bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:18.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.788+0000 7fa6d4c5e640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6bc06af90 0x7fa6bc06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0007fd0 tx=0x7fa6c0009040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:18.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:18.790+0000 7fa6cd7fa640 1 -- 172.21.15.70:0/839068726 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6d015bd90 con 0x7fa6d00cb150 2023-10-08T05:56:19.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.047+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/839068726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) v1 -- 0x7fa6d015bd90 con 0x7fa6d00cb150 2023-10-08T05:56:19.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.687+0000 7fa6cd7fa640 1 -- 172.21.15.70:0/839068726 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v783) v1 ==== 163+0+0 (secure 0 0 0) 0x7fa6d015bd90 con 0x7fa6d00cb150 2023-10-08T05:56:19.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6bc06af90 msgr2=0x7fa6bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:19.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6bc06af90 0x7fa6bc06d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0007fd0 tx=0x7fa6c0009040 comp rx=0 tx=0).stop 2023-10-08T05:56:19.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/839068726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 msgr2=0x7fa6d01fa390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:19.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 0x7fa6d01fa390 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80090b0 tx=0x7fa6c80046c0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/839068726 shutdown_connections 2023-10-08T05:56:19.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6bc06af90 0x7fa6bc06d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6d00cb150 0x7fa6d01fa390 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa6d00add60 0x7fa6d01f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 --2- 172.21.15.70:0/839068726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa6d006d4c0 0x7fa6d01f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/839068726 >> 172.21.15.70:0/839068726 conn(0x7fa6d006a380 msgr2=0x7fa6d015fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:19.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/839068726 shutdown_connections 2023-10-08T05:56:19.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.690+0000 7fa6d6ec2640 1 -- 172.21.15.70:0/839068726 wait complete. 2023-10-08T05:56:19.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T05:56:19.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.934+0000 7f9c69d2c640 1 Processor -- start 2023-10-08T05:56:19.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.934+0000 7f9c69d2c640 1 -- start start 2023-10-08T05:56:19.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.936+0000 7f9c69d2c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c6415cdb0 0x7f9c640cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:19.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.936+0000 7f9c69d2c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6415d190 0x7f9c6406af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:19.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.936+0000 7f9c69d2c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6415d5f0 con 0x7f9c640ccb40 2023-10-08T05:56:19.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.936+0000 7f9c69d2c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c640b19e0 con 0x7f9c6415cdb0 2023-10-08T05:56:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.936+0000 7f9c69d2c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64069ef0 con 0x7f9c6415d190 2023-10-08T05:56:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.936+0000 7f9c63fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9c640ccb40 0x7f9c6406cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53552/0 (socket says 172.21.15.70:53552) 2023-10-08T05:56:19.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.936+0000 7f9c63fff640 1 -- 172.21.15.70:0/3919750440 learned_addr learned my addr 172.21.15.70:0/3919750440 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.936+0000 7f9c62ffd640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6415d190 0x7f9c6406af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.936+0000 7f9c637fe640 1 --2- 172.21.15.70:0/3919750440 >> v2:172.21.15.181:3300/0 conn(0x7f9c6415cdb0 0x7f9c640cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573192201 0 0) 0x7f9c6415d5f0 con 0x7f9c640ccb40 2023-10-08T05:56:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c4c003520 con 0x7f9c640ccb40 2023-10-08T05:56:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165698280 0 0) 0x7f9c4c003520 con 0x7f9c640ccb40 2023-10-08T05:56:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6415d190 msgr2=0x7f9c6406af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6415d190 0x7f9c6406af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 >> v2:172.21.15.181:3300/0 conn(0x7f9c6415cdb0 msgr2=0x7f9c640cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:19.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 --2- 172.21.15.70:0/3919750440 >> v2:172.21.15.181:3300/0 conn(0x7f9c6415cdb0 0x7f9c640cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6406a1d0 con 0x7f9c640ccb40 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9c5c0019f0 con 0x7f9c640ccb40 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 >> v1:172.21.15.70:6789/0 conn(0x7f9c640ccb40 legacy=0x7f9c6406cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6415d190 0x7f9c4c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c003ef0 0x7f9c4c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c4c004d00 0x7f9c4c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b19e0 con 0x7f9c4c004d00 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64069ef0 con 0x7f9c4c003ef0 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.937+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c00b980 con 0x7f9c6415d190 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.938+0000 7f9c63fff640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c4c004d00 0x7f9c4c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.938+0000 7f9c637fe640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6415d190 0x7f9c4c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:19.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.938+0000 7f9c62ffd640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c003ef0 0x7f9c4c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.938+0000 7f9c63fff640 1 -- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6415d190 msgr2=0x7f9c4c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.938+0000 7f9c63fff640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6415d190 0x7f9c4c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.938+0000 7f9c63fff640 1 -- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c003ef0 msgr2=0x7f9c4c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.938+0000 7f9c63fff640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c4c003ef0 0x7f9c4c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.938+0000 7f9c63fff640 1 -- 172.21.15.70:0/3919750440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c003520 con 0x7f9c4c004d00 2023-10-08T05:56:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.939+0000 7f9c63fff640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c4c004d00 0x7f9c4c005180 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00a8d0 tx=0x7f9c4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edcc283e35a2777 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.939+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c034630 con 0x7f9c4c004d00 2023-10-08T05:56:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.939+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c02f070 con 0x7f9c4c004d00 2023-10-08T05:56:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.939+0000 7f9c627fc640 1 -- 172.21.15.70:0/3919750440 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0349d0 con 0x7f9c4c004d00 2023-10-08T05:56:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.940+0000 7f9c69d2c640 1 -- 172.21.15.70:0/3919750440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c4c004d00 msgr2=0x7f9c4c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.940+0000 7f9c69d2c640 1 --2- 172.21.15.70:0/3919750440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c4c004d00 0x7f9c4c005180 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00a8d0 tx=0x7f9c4c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.940+0000 7f9c63fff640 1 -- 172.21.15.70:0/3919750440 reap_dead start 2023-10-08T05:56:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.940+0000 7f9c69d2c640 1 -- 172.21.15.70:0/3919750440 shutdown_connections 2023-10-08T05:56:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.940+0000 7f9c69d2c640 1 -- 172.21.15.70:0/3919750440 >> 172.21.15.70:0/3919750440 conn(0x7f9c640b0af0 msgr2=0x7f9c640b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:19.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.940+0000 7f9c69d2c640 1 -- 172.21.15.70:0/3919750440 shutdown_connections 2023-10-08T05:56:19.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.940+0000 7f9c69d2c640 1 -- 172.21.15.70:0/3919750440 wait complete. 2023-10-08T05:56:19.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.942+0000 7f9c69d2c640 1 Processor -- start 2023-10-08T05:56:19.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.942+0000 7f9c69d2c640 1 -- start start 2023-10-08T05:56:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.943+0000 7f9c69d2c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4c004d00 0x7f9c640b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.943+0000 7f9c69d2c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c640b1b20 0x7f9c641f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.943+0000 7f9c69d2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c641f5250 0x7f9c6415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.943+0000 7f9c69d2c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b2c10 con 0x7f9c641f5250 2023-10-08T05:56:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.943+0000 7f9c69d2c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c641f5700 con 0x7f9c640b1b20 2023-10-08T05:56:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.943+0000 7f9c69d2c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640b2000 con 0x7f9c4c004d00 2023-10-08T05:56:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.943+0000 7f9c63fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c641f5250 0x7f9c6415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.943+0000 7f9c63fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c641f5250 0x7f9c6415f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34752/0 (socket says 172.21.15.70:34752) 2023-10-08T05:56:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.943+0000 7f9c63fff640 1 -- 172.21.15.70:0/205043213 learned_addr learned my addr 172.21.15.70:0/205043213 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:19.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.944+0000 7f9c62ffd640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c640b1b20 0x7f9c641f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.944+0000 7f9c637fe640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4c004d00 0x7f9c640b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.944+0000 7f9c63fff640 1 -- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4c004d00 msgr2=0x7f9c640b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.944+0000 7f9c63fff640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4c004d00 0x7f9c640b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.944+0000 7f9c63fff640 1 -- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c640b1b20 msgr2=0x7f9c641f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.944+0000 7f9c63fff640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c640b1b20 0x7f9c641f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.944+0000 7f9c63fff640 1 -- 172.21.15.70:0/205043213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c640b22e0 con 0x7f9c641f5250 2023-10-08T05:56:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.944+0000 7f9c63fff640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c641f5250 0x7f9c6415f9b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c02ccb0 tx=0x7f9c5c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.944+0000 7f9c4bfff640 1 -- 172.21.15.70:0/205043213 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c034640 con 0x7f9c641f5250 2023-10-08T05:56:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.945+0000 7f9c4bfff640 1 -- 172.21.15.70:0/205043213 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c0347c0 con 0x7f9c641f5250 2023-10-08T05:56:19.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.945+0000 7f9c69d2c640 1 -- 172.21.15.70:0/205043213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c640b25c0 con 0x7f9c641f5250 2023-10-08T05:56:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.945+0000 7f9c4bfff640 1 -- 172.21.15.70:0/205043213 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c034b70 con 0x7f9c641f5250 2023-10-08T05:56:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.945+0000 7f9c69d2c640 1 -- 172.21.15.70:0/205043213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c640ad960 con 0x7f9c641f5250 2023-10-08T05:56:19.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.946+0000 7f9c69d2c640 1 -- 172.21.15.70:0/205043213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c640b2dc0 con 0x7f9c641f5250 2023-10-08T05:56:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.947+0000 7f9c4bfff640 1 -- 172.21.15.70:0/205043213 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9c5c031030 con 0x7f9c641f5250 2023-10-08T05:56:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.948+0000 7f9c4bfff640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f9c5006b010 0x7f9c5006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.948+0000 7f9c4bfff640 1 -- 172.21.15.70:0/205043213 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f9c5c0abe40 con 0x7f9c641f5250 2023-10-08T05:56:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.948+0000 7f9c637fe640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f9c5006b010 0x7f9c5006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:19.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.949+0000 7f9c637fe640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f9c5006b010 0x7f9c5006d4d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c6406b740 tx=0x7f9c54008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:19.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:19.952+0000 7f9c4bfff640 1 -- 172.21.15.70:0/205043213 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c640b2dc0 con 0x7f9c641f5250 2023-10-08T05:56:20.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.146+0000 7f9c69d2c640 1 -- 172.21.15.70:0/205043213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9c640b2dc0 con 0x7f9c641f5250 2023-10-08T05:56:20.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.152+0000 7f9c4bfff640 1 -- 172.21.15.70:0/205043213 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9c640b2dc0 con 0x7f9c641f5250 2023-10-08T05:56:20.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 -- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f9c5006b010 msgr2=0x7f9c5006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:20.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f9c5006b010 0x7f9c5006d4d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c6406b740 tx=0x7f9c54008040 comp rx=0 tx=0).stop 2023-10-08T05:56:20.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 -- 172.21.15.70:0/205043213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c641f5250 msgr2=0x7f9c6415f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:20.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c641f5250 0x7f9c6415f9b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c02ccb0 tx=0x7f9c5c001360 comp rx=0 tx=0).stop 2023-10-08T05:56:20.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 -- 172.21.15.70:0/205043213 shutdown_connections 2023-10-08T05:56:20.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f9c5006b010 0x7f9c5006d4d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c641f5250 0x7f9c6415f9b0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c640b1b20 0x7f9c641f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 --2- 172.21.15.70:0/205043213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c4c004d00 0x7f9c640b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 -- 172.21.15.70:0/205043213 >> 172.21.15.70:0/205043213 conn(0x7f9c640b0af0 msgr2=0x7f9c640acd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:20.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 -- 172.21.15.70:0/205043213 shutdown_connections 2023-10-08T05:56:20.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.155+0000 7f9c69d2c640 1 -- 172.21.15.70:0/205043213 wait complete. 2023-10-08T05:56:20.176 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:56:20.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:56:20.291 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:56:20.414 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:56:20.415 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:56:20.415 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:20.476 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:56:20.477 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:56:20.477 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:20.537 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:56:20.538 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:56:20.538 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:20.598 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:56:20.598 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:56:20.598 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:20.657 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:56:20.657 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T05:56:20.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:56:20.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.879+0000 7f7366010640 1 Processor -- start 2023-10-08T05:56:20.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.879+0000 7f7366010640 1 -- start start 2023-10-08T05:56:20.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.881+0000 7f7366010640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73601597d0 0x7f736006ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.881+0000 7f7366010640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f73600cb190 0x7f73600cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.881+0000 7f7366010640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73600adfc0 con 0x7f7360157000 2023-10-08T05:56:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.881+0000 7f7366010640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73600cbb80 con 0x7f73600cb190 2023-10-08T05:56:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.881+0000 7f7366010640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736006d380 con 0x7f73601597d0 2023-10-08T05:56:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.881+0000 7f735f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7360157000 0x7f73600cc7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53568/0 (socket says 172.21.15.70:53568) 2023-10-08T05:56:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.881+0000 7f735f7fe640 1 -- 172.21.15.70:0/3578967724 learned_addr learned my addr 172.21.15.70:0/3578967724 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754626178 0 0) 0x7f73600adfc0 con 0x7f7360157000 2023-10-08T05:56:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735ffff640 1 --2- 172.21.15.70:0/3578967724 >> v2:172.21.15.181:3300/0 conn(0x7f73600cb190 0x7f73600cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f7357fff640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73601597d0 0x7f736006ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f734c003520 con 0x7f7360157000 2023-10-08T05:56:20.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3137295204 0 0) 0x7f734c003520 con 0x7f7360157000 2023-10-08T05:56:20.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73601597d0 msgr2=0x7f736006ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:20.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73601597d0 0x7f736006ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 >> v2:172.21.15.181:3300/0 conn(0x7f73600cb190 msgr2=0x7f73600cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 --2- 172.21.15.70:0/3578967724 >> v2:172.21.15.181:3300/0 conn(0x7f73600cb190 0x7f73600cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360159cb0 con 0x7f7360157000 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7350001670 con 0x7f7360157000 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 >> v1:172.21.15.70:6789/0 conn(0x7f7360157000 legacy=0x7f73600cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f734c003970 0x7f734c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c0047c0 0x7f734c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c005180 0x7f734c005950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73600cbb80 con 0x7f734c005180 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736006d380 con 0x7f734c003970 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c006040 con 0x7f734c0047c0 2023-10-08T05:56:20.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f7357fff640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73601597d0 0x7f736006ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:56:20.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735ffff640 1 --2- 172.21.15.70:0/3578967724 >> v2:172.21.15.181:3300/0 conn(0x7f73600cb190 0x7f73600cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:56:20.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.882+0000 7f735ffff640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c005180 0x7f734c005950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:20.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.883+0000 7f735f7fe640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f734c003970 0x7f734c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:20.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.883+0000 7f7357fff640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c0047c0 0x7f734c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:20.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.883+0000 7f735ffff640 1 -- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c0047c0 msgr2=0x7f734c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:20.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.883+0000 7f735ffff640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c0047c0 0x7f734c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.883+0000 7f735ffff640 1 -- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f734c003970 msgr2=0x7f734c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:20.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.883+0000 7f735ffff640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f734c003970 0x7f734c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.883+0000 7f735ffff640 1 -- 172.21.15.70:0/3578967724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c003520 con 0x7f734c005180 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.883+0000 7f735f7fe640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f734c003970 0x7f734c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.884+0000 7f735ffff640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c005180 0x7f734c005950 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f73580073e0 tx=0x7f734c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653e365819cc6d88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.884+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735802cdc0 con 0x7f734c005180 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.884+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7358034440 con 0x7f734c005180 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.884+0000 7f735effd640 1 -- 172.21.15.70:0/3578967724 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73580347b0 con 0x7f734c005180 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.885+0000 7f7366010640 1 -- 172.21.15.70:0/3578967724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c005180 msgr2=0x7f734c005950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.885+0000 7f7366010640 1 --2- 172.21.15.70:0/3578967724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c005180 0x7f734c005950 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f73580073e0 tx=0x7f734c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.885+0000 7f735ffff640 1 -- 172.21.15.70:0/3578967724 reap_dead start 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.885+0000 7f7366010640 1 -- 172.21.15.70:0/3578967724 shutdown_connections 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.885+0000 7f7366010640 1 -- 172.21.15.70:0/3578967724 >> 172.21.15.70:0/3578967724 conn(0x7f736006a380 msgr2=0x7f73600b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:20.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.885+0000 7f7366010640 1 -- 172.21.15.70:0/3578967724 shutdown_connections 2023-10-08T05:56:20.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.885+0000 7f7366010640 1 -- 172.21.15.70:0/3578967724 wait complete. 2023-10-08T05:56:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.887+0000 7f7366010640 1 Processor -- start 2023-10-08T05:56:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.887+0000 7f7366010640 1 -- start start 2023-10-08T05:56:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f7366010640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c005180 0x7f73601eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f7366010640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7360152bc0 0x7f73601ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f7366010640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73601f1420 0x7f73601f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f7366010640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73601f1de0 con 0x7f73601f1420 2023-10-08T05:56:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f7366010640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7360153040 con 0x7f7360152bc0 2023-10-08T05:56:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f7366010640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7360153e90 con 0x7f734c005180 2023-10-08T05:56:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f735ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73601f1420 0x7f73601f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:20.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f735ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73601f1420 0x7f73601f18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34770/0 (socket says 172.21.15.70:34770) 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f735ffff640 1 -- 172.21.15.70:0/1899588708 learned_addr learned my addr 172.21.15.70:0/1899588708 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f735ffff640 1 -- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c005180 msgr2=0x7f73601eb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f735ffff640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c005180 0x7f73601eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f735ffff640 1 -- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7360152bc0 msgr2=0x7f73601ee8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f735ffff640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7360152bc0 0x7f73601ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.888+0000 7f735ffff640 1 -- 172.21.15.70:0/1899588708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73601eedf0 con 0x7f73601f1420 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.889+0000 7f735ffff640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73601f1420 0x7f73601f18a0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f7358037040 tx=0x7f7358004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.889+0000 7f735cff9640 1 -- 172.21.15.70:0/1899588708 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735803e440 con 0x7f73601f1420 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.889+0000 7f735cff9640 1 -- 172.21.15.70:0/1899588708 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735803e5e0 con 0x7f73601f1420 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.889+0000 7f735cff9640 1 -- 172.21.15.70:0/1899588708 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735803e990 con 0x7f73601f1420 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.889+0000 7f7366010640 1 -- 172.21.15.70:0/1899588708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7360153280 con 0x7f73601f1420 2023-10-08T05:56:20.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.889+0000 7f7366010640 1 -- 172.21.15.70:0/1899588708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7360153840 con 0x7f73601f1420 2023-10-08T05:56:20.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.891+0000 7f7366010640 1 -- 172.21.15.70:0/1899588708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7360067490 con 0x7f73601f1420 2023-10-08T05:56:20.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.891+0000 7f735cff9640 1 -- 172.21.15.70:0/1899588708 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f735803eaf0 con 0x7f73601f1420 2023-10-08T05:56:20.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.892+0000 7f735cff9640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f734006afc0 0x7f734006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:20.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.892+0000 7f735cff9640 1 -- 172.21.15.70:0/1899588708 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f73580ab950 con 0x7f73601f1420 2023-10-08T05:56:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.893+0000 7f735f7fe640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f734006afc0 0x7f734006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.894+0000 7f735f7fe640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f734006afc0 0x7f734006d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7350001200 tx=0x7f7350009040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:20.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:20.897+0000 7f735cff9640 1 -- 172.21.15.70:0/1899588708 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73580766c0 con 0x7f73601f1420 2023-10-08T05:56:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.126+0000 7f7366010640 1 -- 172.21.15.70:0/1899588708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73600cd1b0 con 0x7f73601f1420 2023-10-08T05:56:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.127+0000 7f735cff9640 1 -- 172.21.15.70:0/1899588708 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f7358039090 con 0x7f73601f1420 2023-10-08T05:56:21.129 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:21.129 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/3983030529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":3983030529},{"type":"v1","addr":"172.21.15.181:6836","nonce":3983030529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":5873,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1573936162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1573936162},{"type":"v1","addr":"172.21.15.181:6837","nonce":1573936162}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6514,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1979922916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1979922916},{"type":"v1","addr":"172.21.15.70:6835","nonce":1979922916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/2828841554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2828841554},{"type":"v1","addr":"172.21.15.70:6833","nonce":2828841554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[]} 2023-10-08T05:56:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 60 2023-10-08T05:56:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 -- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f734006afc0 msgr2=0x7f734006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f734006afc0 0x7f734006d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7350001200 tx=0x7f7350009040 comp rx=0 tx=0).stop 2023-10-08T05:56:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 -- 172.21.15.70:0/1899588708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73601f1420 msgr2=0x7f73601f18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73601f1420 0x7f73601f18a0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f7358037040 tx=0x7f7358004ab0 comp rx=0 tx=0).stop 2023-10-08T05:56:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 -- 172.21.15.70:0/1899588708 shutdown_connections 2023-10-08T05:56:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f734006afc0 0x7f734006d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73601f1420 0x7f73601f18a0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7360152bc0 0x7f73601ee8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 --2- 172.21.15.70:0/1899588708 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f734c005180 0x7f73601eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 -- 172.21.15.70:0/1899588708 >> 172.21.15.70:0/1899588708 conn(0x7f736006a380 msgr2=0x7f73600c4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:21.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.130+0000 7f7366010640 1 -- 172.21.15.70:0/1899588708 shutdown_connections 2023-10-08T05:56:21.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:21.131+0000 7f7366010640 1 -- 172.21.15.70:0/1899588708 wait complete. 2023-10-08T05:56:21.152 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T05:56:21.152 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T05:56:21.152 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T05:56:21.152 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:56:21.153 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T05:56:21.153 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T05:56:21.153 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T05:56:21.153 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:56:21.153 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T05:56:21.153 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T05:56:21.153 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T05:56:21.153 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:56:21.154 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T05:56:21.154 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T05:56:21.154 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T05:56:21.154 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:56:21.171 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T05:56:21.171 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T05:56:21.173 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T05:56:21.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T05:56:21.175 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T05:56:21.175 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T05:56:21.221 INFO:tasks.ceph.mds.c:Started 2023-10-08T05:56:21.221 DEBUG:teuthology.parallel:result is None 2023-10-08T05:56:21.225 INFO:tasks.ceph.mds.d:Started 2023-10-08T05:56:21.225 DEBUG:teuthology.parallel:result is None 2023-10-08T05:56:21.228 INFO:tasks.ceph.mds.a:Started 2023-10-08T05:56:21.229 DEBUG:teuthology.parallel:result is None 2023-10-08T05:56:21.380 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T05:56:21.381 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:56:21.410 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T05:56:23.740 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-08T05:56:27.175 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T05:56:27.176 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T05:56:27.185 INFO:tasks.ceph.mds.b:Started 2023-10-08T05:56:27.186 DEBUG:teuthology.parallel:result is None 2023-10-08T05:56:27.186 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T05:56:27.375 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:56:27.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.405+0000 7f2f4ff00640 1 Processor -- start 2023-10-08T05:56:27.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.406+0000 7f2f4ff00640 1 -- start start 2023-10-08T05:56:27.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.407+0000 7f2f4ff00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f4806dfc0 0x7f2f48070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:27.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.407+0000 7f2f4ff00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480b1cc0 con 0x7f2f48071070 2023-10-08T05:56:27.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.407+0000 7f2f4ff00640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480b2620 con 0x7f2f480afbe0 2023-10-08T05:56:27.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.407+0000 7f2f4ff00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480b2060 con 0x7f2f4806dfc0 2023-10-08T05:56:27.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.407+0000 7f2f4e49d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2f48071070 0x7f2f480cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57856/0 (socket says 172.21.15.70:57856) 2023-10-08T05:56:27.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.408+0000 7f2f4e49d640 1 -- 172.21.15.70:0/967664158 learned_addr learned my addr 172.21.15.70:0/967664158 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:27.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.408+0000 7f2f4d49b640 1 --2- 172.21.15.70:0/967664158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f4806dfc0 0x7f2f48070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:27.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.408+0000 7f2f4cc9a640 1 -- 172.21.15.70:0/967664158 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007256467 0 0) 0x7f2f480b1cc0 con 0x7f2f48071070 2023-10-08T05:56:27.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.408+0000 7f2f4cc9a640 1 -- 172.21.15.70:0/967664158 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f34003520 con 0x7f2f48071070 2023-10-08T05:56:27.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.408+0000 7f2f4cc9a640 1 -- 172.21.15.70:0/967664158 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189384501 0 0) 0x7f2f480b2620 con 0x7f2f480afbe0 2023-10-08T05:56:27.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.408+0000 7f2f4cc9a640 1 -- 172.21.15.70:0/967664158 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f480b1cc0 con 0x7f2f480afbe0 2023-10-08T05:56:27.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.408+0000 7f2f4d49b640 1 -- 172.21.15.70:0/967664158 >> v1:172.21.15.181:6789/0 conn(0x7f2f480afbe0 legacy=0x7f2f480b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:27.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.408+0000 7f2f4d49b640 1 -- 172.21.15.70:0/967664158 >> v1:172.21.15.70:6789/0 conn(0x7f2f48071070 legacy=0x7f2f480cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:27.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.408+0000 7f2f4d49b640 1 -- 172.21.15.70:0/967664158 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480b2340 con 0x7f2f4806dfc0 2023-10-08T05:56:27.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.410+0000 7f2f4d49b640 1 --2- 172.21.15.70:0/967664158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f4806dfc0 0x7f2f48070b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f380097d0 tx=0x7f2f3802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c4988d3ce4d548 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:27.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.410+0000 7f2f4cc9a640 1 -- 172.21.15.70:0/967664158 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38031890 con 0x7f2f4806dfc0 2023-10-08T05:56:27.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.410+0000 7f2f4cc9a640 1 -- 172.21.15.70:0/967664158 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f380319f0 con 0x7f2f4806dfc0 2023-10-08T05:56:27.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.410+0000 7f2f4cc9a640 1 -- 172.21.15.70:0/967664158 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38031d60 con 0x7f2f4806dfc0 2023-10-08T05:56:27.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.411+0000 7f2f4ff00640 1 -- 172.21.15.70:0/967664158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f4806dfc0 msgr2=0x7f2f48070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:27.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.411+0000 7f2f4ff00640 1 --2- 172.21.15.70:0/967664158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f4806dfc0 0x7f2f48070b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2f380097d0 tx=0x7f2f3802b930 comp rx=0 tx=0).stop 2023-10-08T05:56:27.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.411+0000 7f2f4ff00640 1 -- 172.21.15.70:0/967664158 shutdown_connections 2023-10-08T05:56:27.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.411+0000 7f2f4ff00640 1 --2- 172.21.15.70:0/967664158 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f4806dfc0 0x7f2f48070b30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:27.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.411+0000 7f2f4ff00640 1 -- 172.21.15.70:0/967664158 >> 172.21.15.70:0/967664158 conn(0x7f2f4806a5d0 msgr2=0x7f2f480b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:27.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.411+0000 7f2f4ff00640 1 -- 172.21.15.70:0/967664158 shutdown_connections 2023-10-08T05:56:27.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.411+0000 7f2f4ff00640 1 -- 172.21.15.70:0/967664158 wait complete. 2023-10-08T05:56:27.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.414+0000 7f2f4ff00640 1 Processor -- start 2023-10-08T05:56:27.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.414+0000 7f2f4ff00640 1 -- start start 2023-10-08T05:56:27.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.415+0000 7f2f4ff00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f4806dfc0 0x7f2f4815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:27.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.415+0000 7f2f4ff00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f48071070 0x7f2f480c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:27.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.415+0000 7f2f4ff00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f480afbe0 0x7f2f480c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:27.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.415+0000 7f2f4ff00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4815cb40 con 0x7f2f480afbe0 2023-10-08T05:56:27.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.415+0000 7f2f4ff00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4815d940 con 0x7f2f4806dfc0 2023-10-08T05:56:27.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.415+0000 7f2f4ff00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4815cd30 con 0x7f2f48071070 2023-10-08T05:56:27.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.415+0000 7f2f4e49d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f480afbe0 0x7f2f480c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:27.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.415+0000 7f2f4e49d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f480afbe0 0x7f2f480c2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37830/0 (socket says 172.21.15.70:37830) 2023-10-08T05:56:27.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f4e49d640 1 -- 172.21.15.70:0/796641048 learned_addr learned my addr 172.21.15.70:0/796641048 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:27.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f4dc9c640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f4806dfc0 0x7f2f4815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f4d49b640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f48071070 0x7f2f480c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f4e49d640 1 -- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f48071070 msgr2=0x7f2f480c2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f4e49d640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f48071070 0x7f2f480c2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f4e49d640 1 -- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f4806dfc0 msgr2=0x7f2f4815c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f4e49d640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f4806dfc0 0x7f2f4815c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f4e49d640 1 -- 172.21.15.70:0/796641048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38009400 con 0x7f2f480afbe0 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f4dc9c640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f4806dfc0 0x7f2f4815c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f4e49d640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f480afbe0 0x7f2f480c2790 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000cb20 tx=0x7f2f4000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.416+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/796641048 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000aca0 con 0x7f2f480afbe0 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.417+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/796641048 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4000ae00 con 0x7f2f480afbe0 2023-10-08T05:56:27.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.417+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/796641048 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40012600 con 0x7f2f480afbe0 2023-10-08T05:56:27.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.417+0000 7f2f4ff00640 1 -- 172.21.15.70:0/796641048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4815d010 con 0x7f2f480afbe0 2023-10-08T05:56:27.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.417+0000 7f2f4ff00640 1 -- 172.21.15.70:0/796641048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4815d5d0 con 0x7f2f480afbe0 2023-10-08T05:56:27.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.418+0000 7f2f4ff00640 1 -- 172.21.15.70:0/796641048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4815daf0 con 0x7f2f480afbe0 2023-10-08T05:56:27.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.419+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/796641048 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2f40007070 con 0x7f2f480afbe0 2023-10-08T05:56:27.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.420+0000 7f2f3e7fc640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2f2c06af90 0x7f2f2c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:27.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.420+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/796641048 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f2f400897b0 con 0x7f2f480afbe0 2023-10-08T05:56:27.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.420+0000 7f2f4dc9c640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2f2c06af90 0x7f2f2c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:27.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.421+0000 7f2f4dc9c640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2f2c06af90 0x7f2f2c06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f440011e0 tx=0x7f2f44009040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:27.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.424+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/796641048 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f4815daf0 con 0x7f2f480afbe0 2023-10-08T05:56:27.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:27.625+0000 7f2f4ff00640 1 -- 172.21.15.70:0/796641048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2f4815daf0 con 0x7f2f480afbe0 2023-10-08T05:56:28.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.415+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/796641048 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v145) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2f4815daf0 con 0x7f2f480afbe0 2023-10-08T05:56:28.417 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T05:56:28.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 -- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2f2c06af90 msgr2=0x7f2f2c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:28.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2f2c06af90 0x7f2f2c06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2f440011e0 tx=0x7f2f44009040 comp rx=0 tx=0).stop 2023-10-08T05:56:28.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 -- 172.21.15.70:0/796641048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f480afbe0 msgr2=0x7f2f480c2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:28.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f480afbe0 0x7f2f480c2790 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000cb20 tx=0x7f2f4000a060 comp rx=0 tx=0).stop 2023-10-08T05:56:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 -- 172.21.15.70:0/796641048 shutdown_connections 2023-10-08T05:56:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f2f2c06af90 0x7f2f2c06d450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f480afbe0 0x7f2f480c2790 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2f48071070 0x7f2f480c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 --2- 172.21.15.70:0/796641048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2f4806dfc0 0x7f2f4815c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 -- 172.21.15.70:0/796641048 >> 172.21.15.70:0/796641048 conn(0x7f2f4806a5d0 msgr2=0x7f2f4815e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 -- 172.21.15.70:0/796641048 shutdown_connections 2023-10-08T05:56:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.419+0000 7f2f4ff00640 1 -- 172.21.15.70:0/796641048 wait complete. 2023-10-08T05:56:28.440 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T05:56:28.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.646+0000 7fbe29075640 1 Processor -- start 2023-10-08T05:56:28.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.646+0000 7fbe29075640 1 -- start start 2023-10-08T05:56:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.648+0000 7fbe29075640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe240c90b0 0x7fbe240cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.648+0000 7fbe29075640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe240cb3b0 0x7fbe2415e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.648+0000 7fbe29075640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe2415b2b0 con 0x7fbe2415ab40 2023-10-08T05:56:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.648+0000 7fbe29075640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe2415bf30 con 0x7fbe240c90b0 2023-10-08T05:56:28.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.648+0000 7fbe29075640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240cb820 con 0x7fbe240cb3b0 2023-10-08T05:56:28.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.648+0000 7fbe22d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbe2415ab40 0x7fbe240ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57862/0 (socket says 172.21.15.70:57862) 2023-10-08T05:56:28.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.648+0000 7fbe22d9d640 1 -- 172.21.15.70:0/1786841237 learned_addr learned my addr 172.21.15.70:0/1786841237 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe2359e640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe240cb3b0 0x7fbe2415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe2259c640 1 --2- 172.21.15.70:0/1786841237 >> v2:172.21.15.181:3300/0 conn(0x7fbe240c90b0 0x7fbe240cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242498570 0 0) 0x7fbe2415b2b0 con 0x7fbe2415ab40 2023-10-08T05:56:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe0c003520 con 0x7fbe2415ab40 2023-10-08T05:56:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 248871658 0 0) 0x7fbe0c003520 con 0x7fbe2415ab40 2023-10-08T05:56:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe240cb3b0 msgr2=0x7fbe2415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe240cb3b0 0x7fbe2415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 >> v2:172.21.15.181:3300/0 conn(0x7fbe240c90b0 msgr2=0x7fbe240cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 --2- 172.21.15.70:0/1786841237 >> v2:172.21.15.181:3300/0 conn(0x7fbe240c90b0 0x7fbe240cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2415b590 con 0x7fbe2415ab40 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fbe14001690 con 0x7fbe2415ab40 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 >> v1:172.21.15.70:6789/0 conn(0x7fbe2415ab40 legacy=0x7fbe240ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe0c0039a0 0x7fbe0c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe240cb3b0 0x7fbe0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c004d00 0x7fbe0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2415bf30 con 0x7fbe0c0039a0 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240cb820 con 0x7fbe0c004d00 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.649+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c00b980 con 0x7fbe240cb3b0 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.650+0000 7fbe22d9d640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe0c0039a0 0x7fbe0c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.650+0000 7fbe2359e640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c004d00 0x7fbe0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:28.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.650+0000 7fbe2259c640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe240cb3b0 0x7fbe0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:28.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.650+0000 7fbe22d9d640 1 -- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe240cb3b0 msgr2=0x7fbe0c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:28.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.650+0000 7fbe22d9d640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe240cb3b0 0x7fbe0c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.650+0000 7fbe22d9d640 1 -- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c004d00 msgr2=0x7fbe0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:28.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.650+0000 7fbe22d9d640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe0c004d00 0x7fbe0c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.650+0000 7fbe22d9d640 1 -- 172.21.15.70:0/1786841237 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c003520 con 0x7fbe0c0039a0 2023-10-08T05:56:28.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.651+0000 7fbe22d9d640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe0c0039a0 0x7fbe0c003dc0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c0e0 tx=0x7fbe0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2362c307c4b4e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:28.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.652+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14005ad0 con 0x7fbe0c0039a0 2023-10-08T05:56:28.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.652+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe14005c30 con 0x7fbe0c0039a0 2023-10-08T05:56:28.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.652+0000 7fbe21d9b640 1 -- 172.21.15.70:0/1786841237 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1402c790 con 0x7fbe0c0039a0 2023-10-08T05:56:28.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.652+0000 7fbe29075640 1 -- 172.21.15.70:0/1786841237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe0c0039a0 msgr2=0x7fbe0c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:28.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.652+0000 7fbe29075640 1 --2- 172.21.15.70:0/1786841237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe0c0039a0 0x7fbe0c003dc0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c0e0 tx=0x7fbe0c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.652+0000 7fbe2359e640 1 -- 172.21.15.70:0/1786841237 reap_dead start 2023-10-08T05:56:28.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.652+0000 7fbe29075640 1 -- 172.21.15.70:0/1786841237 shutdown_connections 2023-10-08T05:56:28.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.652+0000 7fbe29075640 1 -- 172.21.15.70:0/1786841237 >> 172.21.15.70:0/1786841237 conn(0x7fbe2406a380 msgr2=0x7fbe240b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:28.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.652+0000 7fbe29075640 1 -- 172.21.15.70:0/1786841237 shutdown_connections 2023-10-08T05:56:28.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.652+0000 7fbe29075640 1 -- 172.21.15.70:0/1786841237 wait complete. 2023-10-08T05:56:28.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.654+0000 7fbe29075640 1 Processor -- start 2023-10-08T05:56:28.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.654+0000 7fbe29075640 1 -- start start 2023-10-08T05:56:28.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.655+0000 7fbe29075640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe0c0039a0 0x7fbe241537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:28.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.655+0000 7fbe29075640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe240b33e0 0x7fbe240b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:28.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.655+0000 7fbe29075640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2415c900 0x7fbe2415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:28.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.655+0000 7fbe29075640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe240b3da0 con 0x7fbe240b33e0 2023-10-08T05:56:28.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.655+0000 7fbe29075640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe24154ce0 con 0x7fbe2415c900 2023-10-08T05:56:28.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.655+0000 7fbe29075640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe241540d0 con 0x7fbe0c0039a0 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe2259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe240b33e0 0x7fbe240b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe2259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe240b33e0 0x7fbe240b3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37842/0 (socket says 172.21.15.70:37842) 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe2259c640 1 -- 172.21.15.70:0/656098083 learned_addr learned my addr 172.21.15.70:0/656098083 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe22d9d640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe0c0039a0 0x7fbe241537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe2359e640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2415c900 0x7fbe2415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe2259c640 1 -- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe0c0039a0 msgr2=0x7fbe241537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe2259c640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe0c0039a0 0x7fbe241537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe2259c640 1 -- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2415c900 msgr2=0x7fbe2415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe2259c640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2415c900 0x7fbe2415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe2259c640 1 -- 172.21.15.70:0/656098083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe241543b0 con 0x7fbe240b33e0 2023-10-08T05:56:28.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.656+0000 7fbe2259c640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe240b33e0 0x7fbe240b3860 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fbe18011560 tx=0x7fbe18011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:28.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.657+0000 7fbe0b7fe640 1 -- 172.21.15.70:0/656098083 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18004220 con 0x7fbe240b33e0 2023-10-08T05:56:28.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.657+0000 7fbe0b7fe640 1 -- 172.21.15.70:0/656098083 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe18004380 con 0x7fbe240b33e0 2023-10-08T05:56:28.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.657+0000 7fbe0b7fe640 1 -- 172.21.15.70:0/656098083 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18015870 con 0x7fbe240b33e0 2023-10-08T05:56:28.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.657+0000 7fbe29075640 1 -- 172.21.15.70:0/656098083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe24154690 con 0x7fbe240b33e0 2023-10-08T05:56:28.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.657+0000 7fbe29075640 1 -- 172.21.15.70:0/656098083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe24153d30 con 0x7fbe240b33e0 2023-10-08T05:56:28.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.658+0000 7fbe29075640 1 -- 172.21.15.70:0/656098083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe24067490 con 0x7fbe240b33e0 2023-10-08T05:56:28.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.659+0000 7fbe0b7fe640 1 -- 172.21.15.70:0/656098083 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbe180159d0 con 0x7fbe240b33e0 2023-10-08T05:56:28.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.660+0000 7fbe0b7fe640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe1006af90 0x7fbe1006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:28.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.660+0000 7fbe0b7fe640 1 -- 172.21.15.70:0/656098083 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbe180896f0 con 0x7fbe240b33e0 2023-10-08T05:56:28.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.661+0000 7fbe22d9d640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe1006af90 0x7fbe1006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:28.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.662+0000 7fbe22d9d640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe1006af90 0x7fbe1006d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe14034000 tx=0x7fbe14037040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:28.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.664+0000 7fbe0b7fe640 1 -- 172.21.15.70:0/656098083 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe180547b0 con 0x7fbe240b33e0 2023-10-08T05:56:28.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.922+0000 7fbe29075640 1 -- 172.21.15.70:0/656098083 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbe240b0ef0 con 0x7fbe240b33e0 2023-10-08T05:56:28.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.923+0000 7fbe0b7fe640 1 -- 172.21.15.70:0/656098083 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbe18057dd0 con 0x7fbe240b33e0 2023-10-08T05:56:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:28.924 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:56:28.924 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.926 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T05:56:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T05:56:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:56:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:56:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:56:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.927 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T05:56:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T05:56:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:56:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:56:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:56:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T05:56:28.929 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T05:56:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:56:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:56:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T05:56:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T05:56:28.930 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:56:28.931 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T05:56:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T05:56:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:56:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:56:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.934 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T05:56:28.935 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:56:28.936 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:56:28.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:56:28.938 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T05:56:28.940 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T05:56:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:56:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:56:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:56:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:56:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:28.941 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T05:56:28.941 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:56:28.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.925+0000 7fbe29075640 1 -- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe1006af90 msgr2=0x7fbe1006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:28.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.925+0000 7fbe29075640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe1006af90 0x7fbe1006d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe14034000 tx=0x7fbe14037040 comp rx=0 tx=0).stop 2023-10-08T05:56:28.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.926+0000 7fbe29075640 1 -- 172.21.15.70:0/656098083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe240b33e0 msgr2=0x7fbe240b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:28.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.926+0000 7fbe29075640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe240b33e0 0x7fbe240b3860 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fbe18011560 tx=0x7fbe18011a20 comp rx=0 tx=0).stop 2023-10-08T05:56:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.926+0000 7fbe29075640 1 -- 172.21.15.70:0/656098083 shutdown_connections 2023-10-08T05:56:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.926+0000 7fbe29075640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe1006af90 0x7fbe1006d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.926+0000 7fbe29075640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2415c900 0x7fbe2415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.926+0000 7fbe29075640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe240b33e0 0x7fbe240b3860 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.926+0000 7fbe29075640 1 --2- 172.21.15.70:0/656098083 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe0c0039a0 0x7fbe241537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.926+0000 7fbe29075640 1 -- 172.21.15.70:0/656098083 >> 172.21.15.70:0/656098083 conn(0x7fbe2406a380 msgr2=0x7fbe240ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.926+0000 7fbe29075640 1 -- 172.21.15.70:0/656098083 shutdown_connections 2023-10-08T05:56:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:28.926+0000 7fbe29075640 1 -- 172.21.15.70:0/656098083 wait complete. 2023-10-08T05:56:28.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:29.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.158+0000 7f4a82ea9640 1 Processor -- start 2023-10-08T05:56:29.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.158+0000 7f4a82ea9640 1 -- start start 2023-10-08T05:56:29.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.160+0000 7f4a82ea9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4a7c06b520 0x7f4a7c06ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.160+0000 7f4a82ea9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0af8b0 0x7f4a7c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.160+0000 7f4a82ea9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a7c0b0340 con 0x7f4a7c1608e0 2023-10-08T05:56:29.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.160+0000 7f4a82ea9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a7c06b3a0 con 0x7f4a7c06b520 2023-10-08T05:56:29.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.160+0000 7f4a82ea9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0b0150 con 0x7f4a7c0af8b0 2023-10-08T05:56:29.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.160+0000 7f4a81446640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4a7c1608e0 0x7f4a7c160d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57864/0 (socket says 172.21.15.70:57864) 2023-10-08T05:56:29.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.160+0000 7f4a81446640 1 -- 172.21.15.70:0/2663396573 learned_addr learned my addr 172.21.15.70:0/2663396573 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:29.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7bfff640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0af8b0 0x7f4a7c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a80c45640 1 --2- 172.21.15.70:0/2663396573 >> v2:172.21.15.181:3300/0 conn(0x7f4a7c06b520 0x7f4a7c06ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076426020 0 0) 0x7f4a7c0b0340 con 0x7f4a7c1608e0 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a6c003520 con 0x7f4a7c1608e0 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3860950789 0 0) 0x7f4a6c003520 con 0x7f4a7c1608e0 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0af8b0 msgr2=0x7f4a7c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0af8b0 0x7f4a7c06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 >> v2:172.21.15.181:3300/0 conn(0x7f4a7c06b520 msgr2=0x7f4a7c06ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 --2- 172.21.15.70:0/2663396573 >> v2:172.21.15.181:3300/0 conn(0x7f4a7c06b520 0x7f4a7c06ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c06be40 con 0x7f4a7c1608e0 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4a740019f0 con 0x7f4a7c1608e0 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 >> v1:172.21.15.70:6789/0 conn(0x7f4a7c1608e0 legacy=0x7f4a7c160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 0x7f4a6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0af8b0 0x7f4a6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a6c004d00 0x7f4a6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c06b3a0 con 0x7f4a6c003970 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0b0150 con 0x7f4a6c004d00 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.161+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c00ba30 con 0x7f4a7c0af8b0 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.162+0000 7f4a80c45640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 0x7f4a6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.162+0000 7f4a7bfff640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0af8b0 0x7f4a6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.162+0000 7f4a81446640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a6c004d00 0x7f4a6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.162+0000 7f4a80c45640 1 -- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0af8b0 msgr2=0x7f4a6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.162+0000 7f4a80c45640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0af8b0 0x7f4a6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.162+0000 7f4a80c45640 1 -- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a6c004d00 msgr2=0x7f4a6c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:29.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.162+0000 7f4a80c45640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a6c004d00 0x7f4a6c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.162+0000 7f4a80c45640 1 -- 172.21.15.70:0/2663396573 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c003520 con 0x7f4a6c003970 2023-10-08T05:56:29.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.162+0000 7f4a81446640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a6c004d00 0x7f4a6c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:29.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.164+0000 7f4a80c45640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 0x7f4a6c003d90 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000ba10 tx=0x7f4a6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737c62d60889d290 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:29.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.164+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70005b90 con 0x7f4a6c003970 2023-10-08T05:56:29.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.164+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a70005cf0 con 0x7f4a6c003970 2023-10-08T05:56:29.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.164+0000 7f4a7b7fe640 1 -- 172.21.15.70:0/2663396573 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70006270 con 0x7f4a6c003970 2023-10-08T05:56:29.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.164+0000 7f4a82ea9640 1 -- 172.21.15.70:0/2663396573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 msgr2=0x7f4a6c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.164+0000 7f4a82ea9640 1 --2- 172.21.15.70:0/2663396573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 0x7f4a6c003d90 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000ba10 tx=0x7f4a6c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.165+0000 7f4a81446640 1 -- 172.21.15.70:0/2663396573 reap_dead start 2023-10-08T05:56:29.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.165+0000 7f4a82ea9640 1 -- 172.21.15.70:0/2663396573 shutdown_connections 2023-10-08T05:56:29.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.165+0000 7f4a82ea9640 1 -- 172.21.15.70:0/2663396573 >> 172.21.15.70:0/2663396573 conn(0x7f4a7c0d09e0 msgr2=0x7f4a7c0d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:29.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.165+0000 7f4a82ea9640 1 -- 172.21.15.70:0/2663396573 shutdown_connections 2023-10-08T05:56:29.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.165+0000 7f4a82ea9640 1 -- 172.21.15.70:0/2663396573 wait complete. 2023-10-08T05:56:29.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.167+0000 7f4a82ea9640 1 Processor -- start 2023-10-08T05:56:29.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.167+0000 7f4a82ea9640 1 -- start start 2023-10-08T05:56:29.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a82ea9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 0x7f4a7c15fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a82ea9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a7c0b5e30 0x7f4a7c0c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a82ea9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0b62b0 0x7f4a7c0c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a82ea9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c14ef40 con 0x7f4a6c003970 2023-10-08T05:56:29.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a82ea9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c148860 con 0x7f4a7c0b5e30 2023-10-08T05:56:29.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a82ea9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c0b4d90 con 0x7f4a7c0b62b0 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a80c45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 0x7f4a7c15fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a80c45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 0x7f4a7c15fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37854/0 (socket says 172.21.15.70:37854) 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a80c45640 1 -- 172.21.15.70:0/4019031898 learned_addr learned my addr 172.21.15.70:0/4019031898 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a7bfff640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a7c0b5e30 0x7f4a7c0c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a81446640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0b62b0 0x7f4a7c0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a80c45640 1 -- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0b62b0 msgr2=0x7f4a7c0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a80c45640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0b62b0 0x7f4a7c0c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a80c45640 1 -- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a7c0b5e30 msgr2=0x7f4a7c0c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a80c45640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a7c0b5e30 0x7f4a7c0c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a80c45640 1 -- 172.21.15.70:0/4019031898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7c0b5070 con 0x7f4a6c003970 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.169+0000 7f4a7bfff640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a7c0b5e30 0x7f4a7c0c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:29.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.170+0000 7f4a80c45640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 0x7f4a7c15fae0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000bb30 tx=0x7f4a70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:29.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.170+0000 7f4a797fa640 1 -- 172.21.15.70:0/4019031898 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7003a650 con 0x7f4a6c003970 2023-10-08T05:56:29.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.170+0000 7f4a797fa640 1 -- 172.21.15.70:0/4019031898 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7003a7b0 con 0x7f4a6c003970 2023-10-08T05:56:29.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.170+0000 7f4a797fa640 1 -- 172.21.15.70:0/4019031898 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7003ab20 con 0x7f4a6c003970 2023-10-08T05:56:29.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.170+0000 7f4a82ea9640 1 -- 172.21.15.70:0/4019031898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7c0b5350 con 0x7f4a6c003970 2023-10-08T05:56:29.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.171+0000 7f4a82ea9640 1 -- 172.21.15.70:0/4019031898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7c0b5910 con 0x7f4a6c003970 2023-10-08T05:56:29.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.172+0000 7f4a82ea9640 1 -- 172.21.15.70:0/4019031898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a7c0b5ad0 con 0x7f4a6c003970 2023-10-08T05:56:29.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.173+0000 7f4a797fa640 1 -- 172.21.15.70:0/4019031898 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4a7003ac80 con 0x7f4a6c003970 2023-10-08T05:56:29.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.174+0000 7f4a797fa640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4a6006af90 0x7f4a6006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.174+0000 7f4a797fa640 1 -- 172.21.15.70:0/4019031898 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4a700abe50 con 0x7f4a6c003970 2023-10-08T05:56:29.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.174+0000 7f4a7bfff640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4a6006af90 0x7f4a6006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.176+0000 7f4a7bfff640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4a6006af90 0x7f4a6006d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0b2c40 tx=0x7f4a68008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:29.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.179+0000 7f4a797fa640 1 -- 172.21.15.70:0/4019031898 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a70076eb0 con 0x7f4a6c003970 2023-10-08T05:56:29.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.369+0000 7f4a82ea9640 1 -- 172.21.15.70:0/4019031898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a7c0cfe80 con 0x7f4a6c003970 2023-10-08T05:56:29.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.371+0000 7f4a797fa640 1 -- 172.21.15.70:0/4019031898 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+13175 (secure 0 0 0) 0x7f4a70049090 con 0x7f4a6c003970 2023-10-08T05:56:29.373 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:29.373 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":145,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:28.413313+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":132,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.374+0000 7f4a82ea9640 1 -- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4a6006af90 msgr2=0x7f4a6006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.374+0000 7f4a82ea9640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4a6006af90 0x7f4a6006d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0b2c40 tx=0x7f4a68008040 comp rx=0 tx=0).stop 2023-10-08T05:56:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.374+0000 7f4a82ea9640 1 -- 172.21.15.70:0/4019031898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 msgr2=0x7f4a7c15fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.374+0000 7f4a82ea9640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 0x7f4a7c15fae0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000bb30 tx=0x7f4a70004730 comp rx=0 tx=0).stop 2023-10-08T05:56:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.374+0000 7f4a82ea9640 1 -- 172.21.15.70:0/4019031898 shutdown_connections 2023-10-08T05:56:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.374+0000 7f4a82ea9640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4a6006af90 0x7f4a6006d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.374+0000 7f4a82ea9640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a7c0b62b0 0x7f4a7c0c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.374+0000 7f4a82ea9640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4a7c0b5e30 0x7f4a7c0c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.374+0000 7f4a82ea9640 1 --2- 172.21.15.70:0/4019031898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6c003970 0x7f4a7c15fae0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.374+0000 7f4a82ea9640 1 -- 172.21.15.70:0/4019031898 >> 172.21.15.70:0/4019031898 conn(0x7f4a7c0d09e0 msgr2=0x7f4a7c0b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:29.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.375+0000 7f4a82ea9640 1 -- 172.21.15.70:0/4019031898 shutdown_connections 2023-10-08T05:56:29.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.375+0000 7f4a82ea9640 1 -- 172.21.15.70:0/4019031898 wait complete. 2023-10-08T05:56:29.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:56:29.395 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:56:29.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.613+0000 7fe011f2c640 1 Processor -- start 2023-10-08T05:56:29.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.613+0000 7fe011f2c640 1 -- start start 2023-10-08T05:56:29.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.615+0000 7fe011f2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe00c0b1750 0x7fe00c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.615+0000 7fe011f2c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe00c06d240 0x7fe00c070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.615+0000 7fe011f2c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe00c06d670 con 0x7fe00c15aad0 2023-10-08T05:56:29.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.615+0000 7fe011f2c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00c15bf30 con 0x7fe00c0b1750 2023-10-08T05:56:29.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.615+0000 7fe011f2c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00c0b1bb0 con 0x7fe00c06d240 2023-10-08T05:56:29.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.615+0000 7fe00affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe00c0b1750 0x7fe00c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.615+0000 7fe00affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe00c0b1750 0x7fe00c06cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37870/0 (socket says 172.21.15.70:37870) 2023-10-08T05:56:29.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.615+0000 7fe00affd640 1 -- 172.21.15.70:0/4273623350 learned_addr learned my addr 172.21.15.70:0/4273623350 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:29.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.616+0000 7fe00bfff640 1 --2- 172.21.15.70:0/4273623350 >> v2:172.21.15.181:3300/0 conn(0x7fe00c06d240 0x7fe00c070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.616+0000 7fe00affd640 1 -- 172.21.15.70:0/4273623350 >> v2:172.21.15.181:3300/0 conn(0x7fe00c06d240 msgr2=0x7fe00c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:29.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.616+0000 7fe00affd640 1 --2- 172.21.15.70:0/4273623350 >> v2:172.21.15.181:3300/0 conn(0x7fe00c06d240 0x7fe00c070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.616+0000 7fe00affd640 1 -- 172.21.15.70:0/4273623350 >> v1:172.21.15.181:6789/0 conn(0x7fe00c15aad0 legacy=0x7fe00c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:29.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.616+0000 7fe00affd640 1 -- 172.21.15.70:0/4273623350 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c15b320 con 0x7fe00c0b1750 2023-10-08T05:56:29.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.616+0000 7fe00bfff640 1 --2- 172.21.15.70:0/4273623350 >> v2:172.21.15.181:3300/0 conn(0x7fe00c06d240 0x7fe00c070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:29.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.617+0000 7fe00affd640 1 --2- 172.21.15.70:0/4273623350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 0x7fe00c06cd00 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe00c06e330 tx=0x7fe00002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0da9b087c89f326 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:29.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.617+0000 7fe00a7fc640 1 -- 172.21.15.70:0/4273623350 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000031860 con 0x7fe00c0b1750 2023-10-08T05:56:29.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.617+0000 7fe00a7fc640 1 -- 172.21.15.70:0/4273623350 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0000319c0 con 0x7fe00c0b1750 2023-10-08T05:56:29.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.617+0000 7fe00a7fc640 1 -- 172.21.15.70:0/4273623350 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000031d50 con 0x7fe00c0b1750 2023-10-08T05:56:29.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.618+0000 7fe011f2c640 1 -- 172.21.15.70:0/4273623350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 msgr2=0x7fe00c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.618+0000 7fe011f2c640 1 --2- 172.21.15.70:0/4273623350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 0x7fe00c06cd00 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fe00c06e330 tx=0x7fe00002b900 comp rx=0 tx=0).stop 2023-10-08T05:56:29.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.618+0000 7fe011f2c640 1 -- 172.21.15.70:0/4273623350 shutdown_connections 2023-10-08T05:56:29.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.618+0000 7fe011f2c640 1 --2- 172.21.15.70:0/4273623350 >> v2:172.21.15.181:3300/0 conn(0x7fe00c06d240 0x7fe00c070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.618+0000 7fe011f2c640 1 --2- 172.21.15.70:0/4273623350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 0x7fe00c06cd00 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.618+0000 7fe011f2c640 1 -- 172.21.15.70:0/4273623350 >> 172.21.15.70:0/4273623350 conn(0x7fe00c069d50 msgr2=0x7fe00c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:29.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.618+0000 7fe011f2c640 1 -- 172.21.15.70:0/4273623350 shutdown_connections 2023-10-08T05:56:29.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.618+0000 7fe011f2c640 1 -- 172.21.15.70:0/4273623350 wait complete. 2023-10-08T05:56:29.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.620+0000 7fe011f2c640 1 Processor -- start 2023-10-08T05:56:29.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.620+0000 7fe011f2c640 1 -- start start 2023-10-08T05:56:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.621+0000 7fe011f2c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00c06d240 0x7fe00c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.621+0000 7fe011f2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 0x7fe00c1f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.621+0000 7fe011f2c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe00c15aad0 0x7fe00c1ef670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.621+0000 7fe011f2c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0cd470 con 0x7fe00c0b1750 2023-10-08T05:56:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.621+0000 7fe011f2c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c1f1020 con 0x7fe00c15aad0 2023-10-08T05:56:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.621+0000 7fe011f2c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0cc8f0 con 0x7fe00c06d240 2023-10-08T05:56:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.621+0000 7fe00affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 0x7fe00c1f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.621+0000 7fe00affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 0x7fe00c1f0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37884/0 (socket says 172.21.15.70:37884) 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.621+0000 7fe00affd640 1 -- 172.21.15.70:0/488084595 learned_addr learned my addr 172.21.15.70:0/488084595 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.622+0000 7fe00affd640 1 -- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00c06d240 msgr2=0x7fe00c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.622+0000 7fe00affd640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00c06d240 0x7fe00c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.622+0000 7fe00affd640 1 -- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe00c15aad0 msgr2=0x7fe00c1ef670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.622+0000 7fe00affd640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe00c15aad0 0x7fe00c1ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.622+0000 7fe00affd640 1 -- 172.21.15.70:0/488084595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000094d0 con 0x7fe00c0b1750 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.622+0000 7fe00affd640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 0x7fe00c1f0ae0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe000000c00 tx=0x7fe0000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.622+0000 7fdff3fff640 1 -- 172.21.15.70:0/488084595 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000031830 con 0x7fe00c0b1750 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.622+0000 7fdff3fff640 1 -- 172.21.15.70:0/488084595 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0000443f0 con 0x7fe00c0b1750 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.622+0000 7fdff3fff640 1 -- 172.21.15.70:0/488084595 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0000447c0 con 0x7fe00c0b1750 2023-10-08T05:56:29.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.622+0000 7fe011f2c640 1 -- 172.21.15.70:0/488084595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c0ccbd0 con 0x7fe00c0b1750 2023-10-08T05:56:29.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.623+0000 7fe011f2c640 1 -- 172.21.15.70:0/488084595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c0cd190 con 0x7fe00c0b1750 2023-10-08T05:56:29.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.624+0000 7fe011f2c640 1 -- 172.21.15.70:0/488084595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00c04e160 con 0x7fe00c0b1750 2023-10-08T05:56:29.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.625+0000 7fdff3fff640 1 -- 172.21.15.70:0/488084595 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe00004d450 con 0x7fe00c0b1750 2023-10-08T05:56:29.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.626+0000 7fdff3fff640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdff806af90 0x7fdff806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:29.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.626+0000 7fdff3fff640 1 -- 172.21.15.70:0/488084595 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe0000abdb0 con 0x7fe00c0b1750 2023-10-08T05:56:29.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.626+0000 7fe00b7fe640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdff806af90 0x7fdff806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:29.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.627+0000 7fe00b7fe640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdff806af90 0x7fdff806d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009650 tx=0x7fdffc006ba0 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:29.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.630+0000 7fdff3fff640 1 -- 172.21.15.70:0/488084595 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe000076e90 con 0x7fe00c0b1750 2023-10-08T05:56:29.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.821+0000 7fe011f2c640 1 -- 172.21.15.70:0/488084595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe00c15d8e0 con 0x7fe00c0b1750 2023-10-08T05:56:29.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.822+0000 7fdff3fff640 1 -- 172.21.15.70:0/488084595 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe000035020 con 0x7fe00c0b1750 2023-10-08T05:56:29.823 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:56:29.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 -- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdff806af90 msgr2=0x7fdff806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdff806af90 0x7fdff806d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009650 tx=0x7fdffc006ba0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 -- 172.21.15.70:0/488084595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 msgr2=0x7fe00c1f0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 0x7fe00c1f0ae0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe000000c00 tx=0x7fe0000046c0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 -- 172.21.15.70:0/488084595 shutdown_connections 2023-10-08T05:56:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fdff806af90 0x7fdff806d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe00c15aad0 0x7fe00c1ef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe00c0b1750 0x7fe00c1f0ae0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 --2- 172.21.15.70:0/488084595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe00c06d240 0x7fe00c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 -- 172.21.15.70:0/488084595 >> 172.21.15.70:0/488084595 conn(0x7fe00c069d50 msgr2=0x7fe00c0c81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 -- 172.21.15.70:0/488084595 shutdown_connections 2023-10-08T05:56:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:29.825+0000 7fe011f2c640 1 -- 172.21.15.70:0/488084595 wait complete. 2023-10-08T05:56:29.845 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T05:56:30.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.059+0000 7fe19310f640 1 Processor -- start 2023-10-08T05:56:30.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.059+0000 7fe19310f640 1 -- start start 2023-10-08T05:56:30.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.061+0000 7fe19310f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe18c0cb580 0x7fe18c165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.061+0000 7fe19310f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c15b2b0 con 0x7fe18c0c90b0 2023-10-08T05:56:30.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.061+0000 7fe19310f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c15b590 con 0x7fe18c15ab40 2023-10-08T05:56:30.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.061+0000 7fe19310f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18c15bf30 con 0x7fe18c0cb580 2023-10-08T05:56:30.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.061+0000 7fe18bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe18c0c90b0 0x7fe18c0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57866/0 (socket says 172.21.15.70:57866) 2023-10-08T05:56:30.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.061+0000 7fe18bfff640 1 -- 172.21.15.70:0/2807692577 learned_addr learned my addr 172.21.15.70:0/2807692577 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:30.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe1916ac640 1 --2- 172.21.15.70:0/2807692577 >> v2:172.21.15.181:3300/0 conn(0x7fe18c0cb580 0x7fe18c165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717608646 0 0) 0x7fe18c15b2b0 con 0x7fe18c0c90b0 2023-10-08T05:56:30.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17c003520 con 0x7fe18c0c90b0 2023-10-08T05:56:30.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2347454298 0 0) 0x7fe17c003520 con 0x7fe18c0c90b0 2023-10-08T05:56:30.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 >> v2:172.21.15.181:3300/0 conn(0x7fe18c0cb580 msgr2=0x7fe18c165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:30.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 --2- 172.21.15.70:0/2807692577 >> v2:172.21.15.181:3300/0 conn(0x7fe18c0cb580 0x7fe18c165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 >> v1:172.21.15.181:6789/0 conn(0x7fe18c15ab40 legacy=0x7fe18c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c15b870 con 0x7fe18c0c90b0 2023-10-08T05:56:30.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe178001670 con 0x7fe18c0c90b0 2023-10-08T05:56:30.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 >> v1:172.21.15.70:6789/0 conn(0x7fe18c0c90b0 legacy=0x7fe18c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe17c0039c0 0x7fe17c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe17c0047c0 0x7fe17c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c005180 0x7fe17c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15bf30 con 0x7fe17c0047c0 2023-10-08T05:56:30.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c00c080 con 0x7fe17c005180 2023-10-08T05:56:30.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.062+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c00c240 con 0x7fe17c0039c0 2023-10-08T05:56:30.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.063+0000 7fe18bfff640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe17c0047c0 0x7fe17c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.063+0000 7fe1916ac640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c005180 0x7fe17c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.063+0000 7fe18bfff640 1 -- 172.21.15.70:0/2807692577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe17c0039c0 msgr2=0x7fe17c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:30.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.063+0000 7fe18bfff640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe17c0039c0 0x7fe17c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.063+0000 7fe18bfff640 1 -- 172.21.15.70:0/2807692577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c005180 msgr2=0x7fe17c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:30.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.063+0000 7fe18bfff640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c005180 0x7fe17c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.063+0000 7fe18bfff640 1 -- 172.21.15.70:0/2807692577 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c003520 con 0x7fe17c0047c0 2023-10-08T05:56:30.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.063+0000 7fe1916ac640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe17c005180 0x7fe17c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:30.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.063+0000 7fe1916ac640 1 -- 172.21.15.70:0/2807692577 reap_dead start 2023-10-08T05:56:30.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.064+0000 7fe18bfff640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe17c0047c0 0x7fe17c004c40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe17800c160 tx=0x7fe17c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef33b9bc9d02ed7c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:30.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.065+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178005ad0 con 0x7fe17c0047c0 2023-10-08T05:56:30.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.065+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe178005c50 con 0x7fe17c0047c0 2023-10-08T05:56:30.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.065+0000 7fe18b7fe640 1 -- 172.21.15.70:0/2807692577 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17802c7c0 con 0x7fe17c0047c0 2023-10-08T05:56:30.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.065+0000 7fe19310f640 1 -- 172.21.15.70:0/2807692577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe17c0047c0 msgr2=0x7fe17c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.065+0000 7fe19310f640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe17c0047c0 0x7fe17c004c40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe17800c160 tx=0x7fe17c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.065+0000 7fe19310f640 1 -- 172.21.15.70:0/2807692577 shutdown_connections 2023-10-08T05:56:30.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.065+0000 7fe19310f640 1 --2- 172.21.15.70:0/2807692577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe17c0047c0 0x7fe17c004c40 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.065+0000 7fe19310f640 1 -- 172.21.15.70:0/2807692577 >> 172.21.15.70:0/2807692577 conn(0x7fe18c06a380 msgr2=0x7fe18c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:30.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.065+0000 7fe19310f640 1 -- 172.21.15.70:0/2807692577 shutdown_connections 2023-10-08T05:56:30.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.065+0000 7fe19310f640 1 -- 172.21.15.70:0/2807692577 wait complete. 2023-10-08T05:56:30.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.067+0000 7fe19310f640 1 Processor -- start 2023-10-08T05:56:30.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.067+0000 7fe19310f640 1 -- start start 2023-10-08T05:56:30.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.068+0000 7fe19310f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe17c0047c0 0x7fe18c0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.068+0000 7fe19310f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe18c0ad4f0 0x7fe18c0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.068+0000 7fe19310f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe18c0b2000 0x7fe18c155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.068+0000 7fe19310f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15f7a0 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.068+0000 7fe19310f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0b1be0 con 0x7fe18c0b2000 2023-10-08T05:56:30.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.068+0000 7fe19310f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0ae020 con 0x7fe17c0047c0 2023-10-08T05:56:30.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe18bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe18c0ad4f0 0x7fe18c0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe18bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe18c0ad4f0 0x7fe18c0ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37912/0 (socket says 172.21.15.70:37912) 2023-10-08T05:56:30.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe18bfff640 1 -- 172.21.15.70:0/2151504119 learned_addr learned my addr 172.21.15.70:0/2151504119 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:30.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe1916ac640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe18c0b2000 0x7fe18c155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe190eab640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe17c0047c0 0x7fe18c0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe18bfff640 1 -- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe17c0047c0 msgr2=0x7fe18c0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe18bfff640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe17c0047c0 0x7fe18c0c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe18bfff640 1 -- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe18c0b2000 msgr2=0x7fe18c155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe18bfff640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe18c0b2000 0x7fe18c155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe18bfff640 1 -- 172.21.15.70:0/2151504119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c0add40 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe1916ac640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe18c0b2000 0x7fe18c155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:30.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.069+0000 7fe18bfff640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe18c0ad4f0 0x7fe18c0ce830 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe17800c280 tx=0x7fe178001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:30.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.070+0000 7fe1897fa640 1 -- 172.21.15.70:0/2151504119 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17802cbd0 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.070+0000 7fe1897fa640 1 -- 172.21.15.70:0/2151504119 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17802cd30 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.070+0000 7fe19310f640 1 -- 172.21.15.70:0/2151504119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c0b1520 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.070+0000 7fe1897fa640 1 -- 172.21.15.70:0/2151504119 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17802cea0 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.070+0000 7fe19310f640 1 -- 172.21.15.70:0/2151504119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c0ad970 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.071+0000 7fe19310f640 1 -- 172.21.15.70:0/2151504119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c067490 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.072+0000 7fe1897fa640 1 -- 172.21.15.70:0/2151504119 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe178032070 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.073+0000 7fe1897fa640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe17006af90 0x7fe17006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.073+0000 7fe1897fa640 1 -- 172.21.15.70:0/2151504119 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe17803c040 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.074+0000 7fe190eab640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe17006af90 0x7fe17006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.075+0000 7fe190eab640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe17006af90 0x7fe17006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe180000f60 tx=0x7fe180009040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:30.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.077+0000 7fe1897fa640 1 -- 172.21.15.70:0/2151504119 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe178075b30 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.272+0000 7fe19310f640 1 -- 172.21.15.70:0/2151504119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe18c0b0ef0 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.273+0000 7fe1897fa640 1 -- 172.21.15.70:0/2151504119 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v145) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe178079150 con 0x7fe18c0ad4f0 2023-10-08T05:56:30.275 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:30.275 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:56:30.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:30.275 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T05:56:30.275 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T05:56:30.275 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:30.275 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:56:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 -- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe17006af90 msgr2=0x7fe17006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe17006af90 0x7fe17006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe180000f60 tx=0x7fe180009040 comp rx=0 tx=0).stop 2023-10-08T05:56:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 -- 172.21.15.70:0/2151504119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe18c0ad4f0 msgr2=0x7fe18c0ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe18c0ad4f0 0x7fe18c0ce830 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe17800c280 tx=0x7fe178001930 comp rx=0 tx=0).stop 2023-10-08T05:56:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 -- 172.21.15.70:0/2151504119 shutdown_connections 2023-10-08T05:56:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fe17006af90 0x7fe17006d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe18c0b2000 0x7fe18c155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe18c0ad4f0 0x7fe18c0ce830 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 --2- 172.21.15.70:0/2151504119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe17c0047c0 0x7fe18c0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 -- 172.21.15.70:0/2151504119 >> 172.21.15.70:0/2151504119 conn(0x7fe18c06a380 msgr2=0x7fe18c06cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:30.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 -- 172.21.15.70:0/2151504119 shutdown_connections 2023-10-08T05:56:30.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.276+0000 7fe19310f640 1 -- 172.21.15.70:0/2151504119 wait complete. 2023-10-08T05:56:30.295 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T05:56:30.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T05:56:30.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-08T05:56:30.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.512+0000 7fbe453a4640 1 Processor -- start 2023-10-08T05:56:30.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.513+0000 7fbe453a4640 1 -- start start 2023-10-08T05:56:30.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.514+0000 7fbe453a4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe4015aad0 0x7fbe400b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.514+0000 7fbe453a4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbe4006d410 0x7fbe400c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.514+0000 7fbe453a4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4015bf30 con 0x7fbe400c82b0 2023-10-08T05:56:30.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.514+0000 7fbe453a4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4015b320 con 0x7fbe4006d410 2023-10-08T05:56:30.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.514+0000 7fbe453a4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4015b600 con 0x7fbe4015aad0 2023-10-08T05:56:30.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe367fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbe400c82b0 0x7fbe4006cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57870/0 (socket says 172.21.15.70:57870) 2023-10-08T05:56:30.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe367fc640 1 -- 172.21.15.70:0/4166615409 learned_addr learned my addr 172.21.15.70:0/4166615409 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:30.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3f7fe640 1 --2- 172.21.15.70:0/4166615409 >> v2:172.21.15.181:3300/0 conn(0x7fbe4006d410 0x7fbe400c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090688995 0 0) 0x7fbe4015bf30 con 0x7fbe400c82b0 2023-10-08T05:56:30.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3effd640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe4015aad0 0x7fbe400b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe2c003520 con 0x7fbe400c82b0 2023-10-08T05:56:30.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4208526908 0 0) 0x7fbe2c003520 con 0x7fbe400c82b0 2023-10-08T05:56:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe4015aad0 msgr2=0x7fbe400b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3e7fc640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe4015aad0 0x7fbe400b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 >> v2:172.21.15.181:3300/0 conn(0x7fbe4006d410 msgr2=0x7fbe400c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3e7fc640 1 --2- 172.21.15.70:0/4166615409 >> v2:172.21.15.181:3300/0 conn(0x7fbe4006d410 0x7fbe400c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4015b8e0 con 0x7fbe400c82b0 2023-10-08T05:56:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fbe28001670 con 0x7fbe400c82b0 2023-10-08T05:56:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.515+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 >> v1:172.21.15.70:6789/0 conn(0x7fbe400c82b0 legacy=0x7fbe4006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe3e7fc640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c003970 0x7fbe2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe3e7fc640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 0x7fbe2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe3e7fc640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe4015aad0 0x7fbe2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4015b320 con 0x7fbe2c0047c0 2023-10-08T05:56:30.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4015b600 con 0x7fbe2c003970 2023-10-08T05:56:30.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c00b980 con 0x7fbe4015aad0 2023-10-08T05:56:30.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe367fc640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 0x7fbe2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe3effd640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c003970 0x7fbe2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe367fc640 1 -- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe4015aad0 msgr2=0x7fbe2c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:30.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe3f7fe640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe4015aad0 0x7fbe2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe367fc640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe4015aad0 0x7fbe2c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe367fc640 1 -- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c003970 msgr2=0x7fbe2c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:30.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe367fc640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe2c003970 0x7fbe2c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.516+0000 7fbe367fc640 1 -- 172.21.15.70:0/4166615409 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c003520 con 0x7fbe2c0047c0 2023-10-08T05:56:30.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.518+0000 7fbe367fc640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 0x7fbe2c004c40 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c190 tx=0x7fbe2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f873f8dd2d0ae3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:30.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.518+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28005ad0 con 0x7fbe2c0047c0 2023-10-08T05:56:30.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.518+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28005c50 con 0x7fbe2c0047c0 2023-10-08T05:56:30.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.518+0000 7fbe3e7fc640 1 -- 172.21.15.70:0/4166615409 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2802c810 con 0x7fbe2c0047c0 2023-10-08T05:56:30.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.518+0000 7fbe453a4640 1 -- 172.21.15.70:0/4166615409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 msgr2=0x7fbe2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.518+0000 7fbe453a4640 1 --2- 172.21.15.70:0/4166615409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 0x7fbe2c004c40 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c190 tx=0x7fbe2c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.518+0000 7fbe3f7fe640 1 -- 172.21.15.70:0/4166615409 reap_dead start 2023-10-08T05:56:30.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.519+0000 7fbe453a4640 1 -- 172.21.15.70:0/4166615409 shutdown_connections 2023-10-08T05:56:30.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.519+0000 7fbe453a4640 1 -- 172.21.15.70:0/4166615409 >> 172.21.15.70:0/4166615409 conn(0x7fbe40069d50 msgr2=0x7fbe4006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:30.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.519+0000 7fbe453a4640 1 -- 172.21.15.70:0/4166615409 shutdown_connections 2023-10-08T05:56:30.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.519+0000 7fbe453a4640 1 -- 172.21.15.70:0/4166615409 wait complete. 2023-10-08T05:56:30.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.520+0000 7fbe453a4640 1 Processor -- start 2023-10-08T05:56:30.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.520+0000 7fbe453a4640 1 -- start start 2023-10-08T05:56:30.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.521+0000 7fbe453a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 0x7fbe4013ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.521+0000 7fbe453a4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe40158c90 0x7fbe401590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.521+0000 7fbe453a4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe40153630 0x7fbe40153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.521+0000 7fbe453a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40140460 con 0x7fbe2c0047c0 2023-10-08T05:56:30.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.521+0000 7fbe453a4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe401543b0 con 0x7fbe40153630 2023-10-08T05:56:30.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.521+0000 7fbe453a4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40159630 con 0x7fbe40158c90 2023-10-08T05:56:30.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.522+0000 7fbe3effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 0x7fbe4013ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.522+0000 7fbe3effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 0x7fbe4013ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37924/0 (socket says 172.21.15.70:37924) 2023-10-08T05:56:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.522+0000 7fbe3effd640 1 -- 172.21.15.70:0/1101597453 learned_addr learned my addr 172.21.15.70:0/1101597453 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.522+0000 7fbe3f7fe640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe40153630 0x7fbe40153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.522+0000 7fbe3effd640 1 -- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe40158c90 msgr2=0x7fbe401590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.522+0000 7fbe3effd640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe40158c90 0x7fbe401590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.522+0000 7fbe3effd640 1 -- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe40153630 msgr2=0x7fbe40153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.522+0000 7fbe3effd640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe40153630 0x7fbe40153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.522+0000 7fbe3effd640 1 -- 172.21.15.70:0/1101597453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe40140780 con 0x7fbe2c0047c0 2023-10-08T05:56:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.522+0000 7fbe3effd640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 0x7fbe4013ff20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fbe30007f30 tx=0x7fbe30011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.523+0000 7fbe37fff640 1 -- 172.21.15.70:0/1101597453 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000e030 con 0x7fbe2c0047c0 2023-10-08T05:56:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.523+0000 7fbe37fff640 1 -- 172.21.15.70:0/1101597453 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe30016470 con 0x7fbe2c0047c0 2023-10-08T05:56:30.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.523+0000 7fbe37fff640 1 -- 172.21.15.70:0/1101597453 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30016820 con 0x7fbe2c0047c0 2023-10-08T05:56:30.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.523+0000 7fbe453a4640 1 -- 172.21.15.70:0/1101597453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe40140a60 con 0x7fbe2c0047c0 2023-10-08T05:56:30.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.523+0000 7fbe453a4640 1 -- 172.21.15.70:0/1101597453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe40141020 con 0x7fbe2c0047c0 2023-10-08T05:56:30.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.524+0000 7fbe453a4640 1 -- 172.21.15.70:0/1101597453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4004e160 con 0x7fbe2c0047c0 2023-10-08T05:56:30.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.525+0000 7fbe37fff640 1 -- 172.21.15.70:0/1101597453 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbe30016980 con 0x7fbe2c0047c0 2023-10-08T05:56:30.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.526+0000 7fbe37fff640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe2006af90 0x7fbe2006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:30.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.526+0000 7fbe37fff640 1 -- 172.21.15.70:0/1101597453 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbe3000f070 con 0x7fbe2c0047c0 2023-10-08T05:56:30.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.526+0000 7fbe367fc640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe2006af90 0x7fbe2006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:30.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.528+0000 7fbe367fc640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe2006af90 0x7fbe2006d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe28034000 tx=0x7fbe2802e000 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:30.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.530+0000 7fbe37fff640 1 -- 172.21.15.70:0/1101597453 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe30055650 con 0x7fbe2c0047c0 2023-10-08T05:56:30.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.720+0000 7fbe453a4640 1 -- 172.21.15.70:0/1101597453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbe400b3360 con 0x7fbe2c0047c0 2023-10-08T05:56:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.881+0000 7fbe37fff640 1 -- 172.21.15.70:0/1101597453 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v146) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbe30058c70 con 0x7fbe2c0047c0 2023-10-08T05:56:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T05:56:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.884+0000 7fbe453a4640 1 -- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe2006af90 msgr2=0x7fbe2006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.884+0000 7fbe453a4640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe2006af90 0x7fbe2006d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbe28034000 tx=0x7fbe2802e000 comp rx=0 tx=0).stop 2023-10-08T05:56:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.885+0000 7fbe453a4640 1 -- 172.21.15.70:0/1101597453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 msgr2=0x7fbe4013ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.885+0000 7fbe453a4640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 0x7fbe4013ff20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fbe30007f30 tx=0x7fbe30011660 comp rx=0 tx=0).stop 2023-10-08T05:56:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.885+0000 7fbe453a4640 1 -- 172.21.15.70:0/1101597453 shutdown_connections 2023-10-08T05:56:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.885+0000 7fbe453a4640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fbe2006af90 0x7fbe2006d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.885+0000 7fbe453a4640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbe40153630 0x7fbe40153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.885+0000 7fbe453a4640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbe40158c90 0x7fbe401590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.885+0000 7fbe453a4640 1 --2- 172.21.15.70:0/1101597453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe2c0047c0 0x7fbe4013ff20 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.885+0000 7fbe453a4640 1 -- 172.21.15.70:0/1101597453 >> 172.21.15.70:0/1101597453 conn(0x7fbe40069d50 msgr2=0x7fbe400cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.885+0000 7fbe453a4640 1 -- 172.21.15.70:0/1101597453 shutdown_connections 2023-10-08T05:56:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:30.885+0000 7fbe453a4640 1 -- 172.21.15.70:0/1101597453 wait complete. 2023-10-08T05:56:30.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T05:56:31.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.118+0000 7efc69bb0640 1 Processor -- start 2023-10-08T05:56:31.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.118+0000 7efc69bb0640 1 -- start start 2023-10-08T05:56:31.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.119+0000 7efc69bb0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc6415e300 0x7efc640cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:31.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.119+0000 7efc69bb0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc640cba40 0x7efc640c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:31.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.119+0000 7efc69bb0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc640b2890 con 0x7efc64159d50 2023-10-08T05:56:31.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.119+0000 7efc69bb0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc640cbeb0 con 0x7efc6415e300 2023-10-08T05:56:31.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.119+0000 7efc69bb0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6415d8b0 con 0x7efc640cba40 2023-10-08T05:56:31.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.120+0000 7efc5bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc6415e300 0x7efc640cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:31.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.120+0000 7efc63fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc640cba40 0x7efc640c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:31.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.120+0000 7efc5bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efc6415e300 0x7efc640cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:32974/0 (socket says 172.21.15.70:32974) 2023-10-08T05:56:31.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.120+0000 7efc63fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc640cba40 0x7efc640c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53600/0 (socket says 172.21.15.70:53600) 2023-10-08T05:56:31.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.120+0000 7efc5bfff640 1 -- 172.21.15.70:0/3431414043 learned_addr learned my addr 172.21.15.70:0/3431414043 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:31.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.121+0000 7efc63fff640 1 -- 172.21.15.70:0/3431414043 >> v2:172.21.15.181:3300/0 conn(0x7efc6415e300 msgr2=0x7efc640cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:31.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.121+0000 7efc63fff640 1 --2- 172.21.15.70:0/3431414043 >> v2:172.21.15.181:3300/0 conn(0x7efc6415e300 0x7efc640cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:31.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.121+0000 7efc63fff640 1 -- 172.21.15.70:0/3431414043 >> v1:172.21.15.181:6789/0 conn(0x7efc64159d50 legacy=0x7efc640b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:31.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.121+0000 7efc63fff640 1 -- 172.21.15.70:0/3431414043 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6415e790 con 0x7efc640cba40 2023-10-08T05:56:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.122+0000 7efc63fff640 1 --2- 172.21.15.70:0/3431414043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc640cba40 0x7efc640c9310 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009d90 tx=0x7efc5c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e42993903a8b13cc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.123+0000 7efc62ffd640 1 -- 172.21.15.70:0/3431414043 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c02cc50 con 0x7efc640cba40 2023-10-08T05:56:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.123+0000 7efc62ffd640 1 -- 172.21.15.70:0/3431414043 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5c02cdb0 con 0x7efc640cba40 2023-10-08T05:56:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.123+0000 7efc62ffd640 1 -- 172.21.15.70:0/3431414043 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c034660 con 0x7efc640cba40 2023-10-08T05:56:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.124+0000 7efc69bb0640 1 -- 172.21.15.70:0/3431414043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc640cba40 msgr2=0x7efc640c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.124+0000 7efc69bb0640 1 --2- 172.21.15.70:0/3431414043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc640cba40 0x7efc640c9310 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efc5c009d90 tx=0x7efc5c02bea0 comp rx=0 tx=0).stop 2023-10-08T05:56:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.124+0000 7efc69bb0640 1 -- 172.21.15.70:0/3431414043 shutdown_connections 2023-10-08T05:56:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.124+0000 7efc69bb0640 1 --2- 172.21.15.70:0/3431414043 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc640cba40 0x7efc640c9310 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:31.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.124+0000 7efc69bb0640 1 --2- 172.21.15.70:0/3431414043 >> v2:172.21.15.181:3300/0 conn(0x7efc6415e300 0x7efc640cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:31.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.124+0000 7efc69bb0640 1 -- 172.21.15.70:0/3431414043 >> 172.21.15.70:0/3431414043 conn(0x7efc6406a140 msgr2=0x7efc6406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:31.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.124+0000 7efc69bb0640 1 -- 172.21.15.70:0/3431414043 shutdown_connections 2023-10-08T05:56:31.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.124+0000 7efc69bb0640 1 -- 172.21.15.70:0/3431414043 wait complete. 2023-10-08T05:56:31.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.126+0000 7efc69bb0640 1 Processor -- start 2023-10-08T05:56:31.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.126+0000 7efc69bb0640 1 -- start start 2023-10-08T05:56:31.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.127+0000 7efc69bb0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640cba40 0x7efc64153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:31.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc69bb0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc64159d50 0x7efc6415cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:31.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc69bb0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6415e300 0x7efc64153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:31.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc69bb0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc640c37a0 con 0x7efc640cba40 2023-10-08T05:56:31.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc69bb0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6415b2b0 con 0x7efc64159d50 2023-10-08T05:56:31.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc69bb0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc640c34c0 con 0x7efc6415e300 2023-10-08T05:56:31.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640cba40 0x7efc64153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:31.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640cba40 0x7efc64153600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37936/0 (socket says 172.21.15.70:37936) 2023-10-08T05:56:31.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc637fe640 1 -- 172.21.15.70:0/2746117560 learned_addr learned my addr 172.21.15.70:0/2746117560 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:31.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc5bfff640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc64159d50 0x7efc6415cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:31.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc63fff640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6415e300 0x7efc64153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:31.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc637fe640 1 -- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6415e300 msgr2=0x7efc64153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:31.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc637fe640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6415e300 0x7efc64153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:31.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc637fe640 1 -- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc64159d50 msgr2=0x7efc6415cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:31.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc637fe640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc64159d50 0x7efc6415cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:31.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.128+0000 7efc637fe640 1 -- 172.21.15.70:0/2746117560 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c009a70 con 0x7efc640cba40 2023-10-08T05:56:31.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.129+0000 7efc5bfff640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc64159d50 0x7efc6415cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:31.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.129+0000 7efc637fe640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640cba40 0x7efc64153600 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7efc5400c450 tx=0x7efc5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:31.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.129+0000 7efc60ff9640 1 -- 172.21.15.70:0/2746117560 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc540052d0 con 0x7efc640cba40 2023-10-08T05:56:31.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.129+0000 7efc60ff9640 1 -- 172.21.15.70:0/2746117560 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc54005430 con 0x7efc640cba40 2023-10-08T05:56:31.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.129+0000 7efc69bb0640 1 -- 172.21.15.70:0/2746117560 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6415b550 con 0x7efc640cba40 2023-10-08T05:56:31.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.129+0000 7efc60ff9640 1 -- 172.21.15.70:0/2746117560 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc54012870 con 0x7efc640cba40 2023-10-08T05:56:31.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.129+0000 7efc69bb0640 1 -- 172.21.15.70:0/2746117560 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc64153df0 con 0x7efc640cba40 2023-10-08T05:56:31.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.131+0000 7efc69bb0640 1 -- 172.21.15.70:0/2746117560 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc6415ba70 con 0x7efc640cba40 2023-10-08T05:56:31.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.132+0000 7efc60ff9640 1 -- 172.21.15.70:0/2746117560 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efc540129d0 con 0x7efc640cba40 2023-10-08T05:56:31.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.132+0000 7efc60ff9640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efc4406afc0 0x7efc4406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:31.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.133+0000 7efc60ff9640 1 -- 172.21.15.70:0/2746117560 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5619+0+0 (secure 0 0 0) 0x7efc54089180 con 0x7efc640cba40 2023-10-08T05:56:31.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.133+0000 7efc5bfff640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efc4406afc0 0x7efc4406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:31.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.135+0000 7efc5bfff640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efc4406afc0 0x7efc4406d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc640cb5a0 tx=0x7efc4c008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:31.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.138+0000 7efc60ff9640 1 -- 172.21.15.70:0/2746117560 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc6415ba70 con 0x7efc640cba40 2023-10-08T05:56:31.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.326+0000 7efc69bb0640 1 -- 172.21.15.70:0/2746117560 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efc6415ba70 con 0x7efc640cba40 2023-10-08T05:56:31.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.898+0000 7efc60ff9640 1 -- 172.21.15.70:0/2746117560 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v147) v1 ==== 170+0+0 (secure 0 0 0) 0x7efc6415ba70 con 0x7efc640cba40 2023-10-08T05:56:31.900 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T05:56:31.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.903+0000 7efc69bb0640 1 -- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efc4406afc0 msgr2=0x7efc4406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:31.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.904+0000 7efc69bb0640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efc4406afc0 0x7efc4406d480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc640cb5a0 tx=0x7efc4c008040 comp rx=0 tx=0).stop 2023-10-08T05:56:31.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.904+0000 7efc69bb0640 1 -- 172.21.15.70:0/2746117560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640cba40 msgr2=0x7efc64153600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:31.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.904+0000 7efc69bb0640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640cba40 0x7efc64153600 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7efc5400c450 tx=0x7efc5400c910 comp rx=0 tx=0).stop 2023-10-08T05:56:31.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.904+0000 7efc69bb0640 1 -- 172.21.15.70:0/2746117560 shutdown_connections 2023-10-08T05:56:31.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.904+0000 7efc69bb0640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7efc4406afc0 0x7efc4406d480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:31.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.904+0000 7efc69bb0640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efc6415e300 0x7efc64153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:31.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.904+0000 7efc69bb0640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efc64159d50 0x7efc6415cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:31.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.904+0000 7efc69bb0640 1 --2- 172.21.15.70:0/2746117560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc640cba40 0x7efc64153600 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:31.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.904+0000 7efc69bb0640 1 -- 172.21.15.70:0/2746117560 >> 172.21.15.70:0/2746117560 conn(0x7efc6406a140 msgr2=0x7efc640c7950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:31.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.905+0000 7efc69bb0640 1 -- 172.21.15.70:0/2746117560 shutdown_connections 2023-10-08T05:56:31.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:31.905+0000 7efc69bb0640 1 -- 172.21.15.70:0/2746117560 wait complete. 2023-10-08T05:56:31.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T05:56:32.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.129+0000 7fd5c9f24640 1 Processor -- start 2023-10-08T05:56:32.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.130+0000 7fd5c9f24640 1 -- start start 2023-10-08T05:56:32.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.131+0000 7fd5c9f24640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd5c4070cc0 0x7fd5c40caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:32.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.131+0000 7fd5c9f24640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c4071170 con 0x7fd5c40cb450 2023-10-08T05:56:32.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.131+0000 7fd5c9f24640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c406e150 con 0x7fd5c4160660 2023-10-08T05:56:32.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.131+0000 7fd5c9f24640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c406de70 con 0x7fd5c4070cc0 2023-10-08T05:56:32.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd5c40cb450 0x7fd5c40b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57880/0 (socket says 172.21.15.70:57880) 2023-10-08T05:56:32.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c3fff640 1 -- 172.21.15.70:0/314997163 learned_addr learned my addr 172.21.15.70:0/314997163 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5bbfff640 1 --2- 172.21.15.70:0/314997163 >> v2:172.21.15.181:3300/0 conn(0x7fd5c4070cc0 0x7fd5c40caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309274550 0 0) 0x7fd5bc001490 con 0x7fd5c40cb450 2023-10-08T05:56:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5b0003520 con 0x7fd5c40cb450 2023-10-08T05:56:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1520143823 0 0) 0x7fd5b0003520 con 0x7fd5c40cb450 2023-10-08T05:56:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 >> v2:172.21.15.181:3300/0 conn(0x7fd5c4070cc0 msgr2=0x7fd5c40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c2ffd640 1 --2- 172.21.15.70:0/314997163 >> v2:172.21.15.181:3300/0 conn(0x7fd5c4070cc0 0x7fd5c40caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 >> v1:172.21.15.181:6789/0 conn(0x7fd5c4160660 legacy=0x7fd5c40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c40c8e30 con 0x7fd5c40cb450 2023-10-08T05:56:32.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd5bc001b80 con 0x7fd5c40cb450 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.132+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 >> v1:172.21.15.70:6789/0 conn(0x7fd5c40cb450 legacy=0x7fd5c40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5c2ffd640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b0003970 0x7fd5b0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5c2ffd640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5c2ffd640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5b0005180 0x7fd5b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c406de70 con 0x7fd5b00047c0 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b000c050 con 0x7fd5b0003970 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b000c210 con 0x7fd5b0005180 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5bbfff640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5c37fe640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b0003970 0x7fd5b0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5bbfff640 1 -- 172.21.15.70:0/314997163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5b0005180 msgr2=0x7fd5b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5bbfff640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5b0005180 0x7fd5b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5bbfff640 1 -- 172.21.15.70:0/314997163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b0003970 msgr2=0x7fd5b0003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5bbfff640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b0003970 0x7fd5b0003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5bbfff640 1 -- 172.21.15.70:0/314997163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b0003520 con 0x7fd5b00047c0 2023-10-08T05:56:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5c37fe640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5b0003970 0x7fd5b0003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.133+0000 7fd5c3fff640 1 -- 172.21.15.70:0/314997163 reap_dead start 2023-10-08T05:56:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.135+0000 7fd5bbfff640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5b0004c40 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009ac0 tx=0x7fd5b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e6ddf86e530ce5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.135+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac02c6b0 con 0x7fd5b00047c0 2023-10-08T05:56:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.135+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ac02c810 con 0x7fd5b00047c0 2023-10-08T05:56:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.135+0000 7fd5c2ffd640 1 -- 172.21.15.70:0/314997163 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac02cba0 con 0x7fd5b00047c0 2023-10-08T05:56:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.135+0000 7fd5c9f24640 1 -- 172.21.15.70:0/314997163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 msgr2=0x7fd5b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.135+0000 7fd5c9f24640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5b0004c40 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009ac0 tx=0x7fd5b00057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.136+0000 7fd5c9f24640 1 -- 172.21.15.70:0/314997163 shutdown_connections 2023-10-08T05:56:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.136+0000 7fd5c9f24640 1 --2- 172.21.15.70:0/314997163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5b0004c40 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.136+0000 7fd5c9f24640 1 -- 172.21.15.70:0/314997163 >> 172.21.15.70:0/314997163 conn(0x7fd5c406a5d0 msgr2=0x7fd5c40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.136+0000 7fd5c9f24640 1 -- 172.21.15.70:0/314997163 shutdown_connections 2023-10-08T05:56:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.136+0000 7fd5c9f24640 1 -- 172.21.15.70:0/314997163 wait complete. 2023-10-08T05:56:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.137+0000 7fd5c9f24640 1 Processor -- start 2023-10-08T05:56:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.138+0000 7fd5c9f24640 1 -- start start 2023-10-08T05:56:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c9f24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5c41f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c9f24640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c41565f0 0x7fd5c41f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c9f24640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5c4153de0 0x7fd5c415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c9f24640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c41f2b00 con 0x7fd5b00047c0 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c9f24640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c41f2800 con 0x7fd5c41565f0 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c9f24640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c4156ad0 con 0x7fd5c4153de0 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5c41f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5c41f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37950/0 (socket says 172.21.15.70:37950) 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c37fe640 1 -- 172.21.15.70:0/83082921 learned_addr learned my addr 172.21.15.70:0/83082921 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5bbfff640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c41565f0 0x7fd5c41f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c3fff640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5c4153de0 0x7fd5c415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c37fe640 1 -- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5c4153de0 msgr2=0x7fd5c415b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c37fe640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5c4153de0 0x7fd5c415b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c37fe640 1 -- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c41565f0 msgr2=0x7fd5c41f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c37fe640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c41565f0 0x7fd5c41f2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c37fe640 1 -- 172.21.15.70:0/83082921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c4156db0 con 0x7fd5b00047c0 2023-10-08T05:56:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.139+0000 7fd5c3fff640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5c4153de0 0x7fd5c415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.140+0000 7fd5c37fe640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5c41f1fe0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400d4a0 tx=0x7fd5b400d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.140+0000 7fd5c0ff9640 1 -- 172.21.15.70:0/83082921 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4005480 con 0x7fd5b00047c0 2023-10-08T05:56:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.140+0000 7fd5c0ff9640 1 -- 172.21.15.70:0/83082921 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b4007030 con 0x7fd5b00047c0 2023-10-08T05:56:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.140+0000 7fd5c0ff9640 1 -- 172.21.15.70:0/83082921 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4007400 con 0x7fd5b00047c0 2023-10-08T05:56:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.140+0000 7fd5c9f24640 1 -- 172.21.15.70:0/83082921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c41f2520 con 0x7fd5b00047c0 2023-10-08T05:56:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.140+0000 7fd5c9f24640 1 -- 172.21.15.70:0/83082921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c41f34d0 con 0x7fd5b00047c0 2023-10-08T05:56:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.141+0000 7fd5c9f24640 1 -- 172.21.15.70:0/83082921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c404e160 con 0x7fd5b00047c0 2023-10-08T05:56:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.143+0000 7fd5c0ff9640 1 -- 172.21.15.70:0/83082921 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd5b4007560 con 0x7fd5b00047c0 2023-10-08T05:56:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.143+0000 7fd5c0ff9640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd5a406af90 0x7fd5a406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.144+0000 7fd5c0ff9640 1 -- 172.21.15.70:0/83082921 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd5b4089310 con 0x7fd5b00047c0 2023-10-08T05:56:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.144+0000 7fd5bbfff640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd5a406af90 0x7fd5a406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:32.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.145+0000 7fd5bbfff640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd5a406af90 0x7fd5a406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009ac0 tx=0x7fd5ac030000 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:32.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.147+0000 7fd5c0ff9640 1 -- 172.21.15.70:0/83082921 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd5b4054100 con 0x7fd5b00047c0 2023-10-08T05:56:32.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.374+0000 7fd5c9f24640 1 -- 172.21.15.70:0/83082921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd5c40cc900 con 0x7fd5b00047c0 2023-10-08T05:56:32.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.911+0000 7fd5c0ff9640 1 -- 172.21.15.70:0/83082921 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v64) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd5b4057720 con 0x7fd5b00047c0 2023-10-08T05:56:32.912 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 17 and data pool 18 2023-10-08T05:56:32.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.914+0000 7fd5c9f24640 1 -- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd5a406af90 msgr2=0x7fd5a406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.914+0000 7fd5c9f24640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd5a406af90 0x7fd5a406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009ac0 tx=0x7fd5ac030000 comp rx=0 tx=0).stop 2023-10-08T05:56:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.914+0000 7fd5c9f24640 1 -- 172.21.15.70:0/83082921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 msgr2=0x7fd5c41f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.914+0000 7fd5c9f24640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5c41f1fe0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400d4a0 tx=0x7fd5b400d960 comp rx=0 tx=0).stop 2023-10-08T05:56:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.914+0000 7fd5c9f24640 1 -- 172.21.15.70:0/83082921 shutdown_connections 2023-10-08T05:56:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.914+0000 7fd5c9f24640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd5a406af90 0x7fd5a406d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.914+0000 7fd5c9f24640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd5c4153de0 0x7fd5c415b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.914+0000 7fd5c9f24640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd5c41565f0 0x7fd5c41f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.914+0000 7fd5c9f24640 1 --2- 172.21.15.70:0/83082921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5b00047c0 0x7fd5c41f1fe0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:32.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.914+0000 7fd5c9f24640 1 -- 172.21.15.70:0/83082921 >> 172.21.15.70:0/83082921 conn(0x7fd5c406a5d0 msgr2=0x7fd5c40c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:32.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.915+0000 7fd5c9f24640 1 -- 172.21.15.70:0/83082921 shutdown_connections 2023-10-08T05:56:32.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:32.915+0000 7fd5c9f24640 1 -- 172.21.15.70:0/83082921 wait complete. 2023-10-08T05:56:32.940 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:33.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.125+0000 7f03c1dd0640 1 Processor -- start 2023-10-08T05:56:33.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.126+0000 7f03c1dd0640 1 -- start start 2023-10-08T05:56:33.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.127+0000 7f03c1dd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc159ac0 0x7f03bc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.127+0000 7f03c1dd0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc06dee0 con 0x7f03bc0ad790 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.127+0000 7f03c1dd0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03bc15d850 con 0x7f03bc0709e0 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.127+0000 7f03c1dd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc15db30 con 0x7f03bc159ac0 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03baffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f03bc0ad790 0x7f03bc0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57892/0 (socket says 172.21.15.70:57892) 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03baffd640 1 -- 172.21.15.70:0/508002008 learned_addr learned my addr 172.21.15.70:0/508002008 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03bb7fe640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc159ac0 0x7f03bc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458024562 0 0) 0x7f03bc06dee0 con 0x7f03bc0ad790 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a8005730 con 0x7f03bc0ad790 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2315012541 0 0) 0x7f03a8005730 con 0x7f03bc0ad790 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc159ac0 msgr2=0x7f03bc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03ba7fc640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc159ac0 0x7f03bc06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 >> v1:172.21.15.181:6789/0 conn(0x7f03bc0709e0 legacy=0x7f03bc0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc15de10 con 0x7f03bc0ad790 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f03a4001670 con 0x7f03bc0ad790 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 >> v1:172.21.15.70:6789/0 conn(0x7f03bc0ad790 legacy=0x7f03bc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.128+0000 7f03ba7fc640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03a8005bb0 0x7f03a8005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03ba7fc640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc159ac0 0x7f03a80069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03ba7fc640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 0x7f03a8007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc15db30 con 0x7f03a8006f10 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a800dc40 con 0x7f03a8005bb0 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a800de00 con 0x7f03bc159ac0 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03bbfff640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 0x7f03a8007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03bb7fe640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03a8005bb0 0x7f03a8005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03baffd640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc159ac0 0x7f03a80069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03bbfff640 1 -- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc159ac0 msgr2=0x7f03a80069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:33.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03bbfff640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc159ac0 0x7f03a80069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03bbfff640 1 -- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03a8005bb0 msgr2=0x7f03a8005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:33.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03bbfff640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03a8005bb0 0x7f03a8005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.129+0000 7f03bbfff640 1 -- 172.21.15.70:0/508002008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a8005730 con 0x7f03a8006f10 2023-10-08T05:56:33.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.130+0000 7f03bbfff640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 0x7f03a8007390 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0ae2f0 tx=0x7f03a8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bc0194f3286d76 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:33.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.131+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40052d0 con 0x7f03a8006f10 2023-10-08T05:56:33.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.131+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b402e050 con 0x7f03a8006f10 2023-10-08T05:56:33.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.131+0000 7f03ba7fc640 1 -- 172.21.15.70:0/508002008 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40035f0 con 0x7f03a8006f10 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.131+0000 7f03c1dd0640 1 -- 172.21.15.70:0/508002008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 msgr2=0x7f03a8007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.131+0000 7f03c1dd0640 1 --2- 172.21.15.70:0/508002008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 0x7f03a8007390 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0ae2f0 tx=0x7f03a8007a00 comp rx=0 tx=0).stop 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.131+0000 7f03bbfff640 1 -- 172.21.15.70:0/508002008 reap_dead start 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.131+0000 7f03c1dd0640 1 -- 172.21.15.70:0/508002008 shutdown_connections 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.131+0000 7f03c1dd0640 1 -- 172.21.15.70:0/508002008 >> 172.21.15.70:0/508002008 conn(0x7f03bc06a380 msgr2=0x7f03bc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.131+0000 7f03c1dd0640 1 -- 172.21.15.70:0/508002008 shutdown_connections 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.132+0000 7f03c1dd0640 1 -- 172.21.15.70:0/508002008 wait complete. 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.133+0000 7f03c1dd0640 1 Processor -- start 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.133+0000 7f03c1dd0640 1 -- start start 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.134+0000 7f03c1dd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 0x7f03bc14ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.134+0000 7f03c1dd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03bc14b320 0x7f03bc0cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.134+0000 7f03c1dd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc15b060 0x7f03bc15b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.134+0000 7f03c1dd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc0b1a70 con 0x7f03a8006f10 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.134+0000 7f03c1dd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc14bbc0 con 0x7f03bc14b320 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.134+0000 7f03c1dd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc15bf70 con 0x7f03bc15b060 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.135+0000 7f03bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 0x7f03bc14ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.135+0000 7f03bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 0x7f03bc14ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37968/0 (socket says 172.21.15.70:37968) 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.135+0000 7f03bb7fe640 1 -- 172.21.15.70:0/852404797 learned_addr learned my addr 172.21.15.70:0/852404797 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.135+0000 7f03bb7fe640 1 -- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc15b060 msgr2=0x7f03bc15b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.135+0000 7f03bb7fe640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc15b060 0x7f03bc15b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.135+0000 7f03bb7fe640 1 -- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03bc14b320 msgr2=0x7f03bc0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.135+0000 7f03bb7fe640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03bc14b320 0x7f03bc0cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.135+0000 7f03bb7fe640 1 -- 172.21.15.70:0/852404797 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc15c250 con 0x7f03a8006f10 2023-10-08T05:56:33.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.135+0000 7f03bb7fe640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 0x7f03bc14ad00 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f03ac007f30 tx=0x7f03ac011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:33.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.136+0000 7f03b3fff640 1 -- 172.21.15.70:0/852404797 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00e030 con 0x7f03a8006f10 2023-10-08T05:56:33.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.136+0000 7f03b3fff640 1 -- 172.21.15.70:0/852404797 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac016470 con 0x7f03a8006f10 2023-10-08T05:56:33.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.136+0000 7f03b3fff640 1 -- 172.21.15.70:0/852404797 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac016820 con 0x7f03a8006f10 2023-10-08T05:56:33.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.136+0000 7f03c1dd0640 1 -- 172.21.15.70:0/852404797 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc15c530 con 0x7f03a8006f10 2023-10-08T05:56:33.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.136+0000 7f03c1dd0640 1 -- 172.21.15.70:0/852404797 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc15bc00 con 0x7f03a8006f10 2023-10-08T05:56:33.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.137+0000 7f03c1dd0640 1 -- 172.21.15.70:0/852404797 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc067490 con 0x7f03a8006f10 2023-10-08T05:56:33.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.138+0000 7f03b3fff640 1 -- 172.21.15.70:0/852404797 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f03ac016980 con 0x7f03a8006f10 2023-10-08T05:56:33.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.139+0000 7f03b3fff640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f039c06af90 0x7f039c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.139+0000 7f03b3fff640 1 -- 172.21.15.70:0/852404797 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03ac00f070 con 0x7f03a8006f10 2023-10-08T05:56:33.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.139+0000 7f03baffd640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f039c06af90 0x7f039c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.141+0000 7f03baffd640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f039c06af90 0x7f039c06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f03a4001eb0 tx=0x7f03a4001d60 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:33.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.144+0000 7f03b3fff640 1 -- 172.21.15.70:0/852404797 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03ac0556b0 con 0x7f03a8006f10 2023-10-08T05:56:33.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.329+0000 7f03c1dd0640 1 -- 172.21.15.70:0/852404797 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03bc159ea0 con 0x7f03a8006f10 2023-10-08T05:56:33.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.331+0000 7f03b3fff640 1 -- 172.21.15.70:0/852404797 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16900 (secure 0 0 0) 0x7f03ac058cd0 con 0x7f03a8006f10 2023-10-08T05:56:33.332 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:33.333 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":148,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:32.377072+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:56:30.775239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-08T05:56:31.378327+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:33.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 -- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f039c06af90 msgr2=0x7f039c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f039c06af90 0x7f039c06d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f03a4001eb0 tx=0x7f03a4001d60 comp rx=0 tx=0).stop 2023-10-08T05:56:33.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 -- 172.21.15.70:0/852404797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 msgr2=0x7f03bc14ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 0x7f03bc14ad00 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f03ac007f30 tx=0x7f03ac011660 comp rx=0 tx=0).stop 2023-10-08T05:56:33.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 -- 172.21.15.70:0/852404797 shutdown_connections 2023-10-08T05:56:33.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f039c06af90 0x7f039c06d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03bc15b060 0x7f03bc15b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03bc14b320 0x7f03bc0cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 --2- 172.21.15.70:0/852404797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a8006f10 0x7f03bc14ad00 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 -- 172.21.15.70:0/852404797 >> 172.21.15.70:0/852404797 conn(0x7f03bc06a380 msgr2=0x7f03bc156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:33.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.333+0000 7f03c1dd0640 1 -- 172.21.15.70:0/852404797 shutdown_connections 2023-10-08T05:56:33.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.334+0000 7f03c1dd0640 1 -- 172.21.15.70:0/852404797 wait complete. 2023-10-08T05:56:33.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:33.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.557+0000 7fd1054d7640 1 Processor -- start 2023-10-08T05:56:33.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.557+0000 7fd1054d7640 1 -- start start 2023-10-08T05:56:33.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.559+0000 7fd1054d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd1000d16f0 0x7fd1000cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.559+0000 7fd1054d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd1000cfa50 0x7fd1000ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.559+0000 7fd1054d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1000cea50 con 0x7fd10015cc60 2023-10-08T05:56:33.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.559+0000 7fd1054d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10015d040 con 0x7fd1000cfa50 2023-10-08T05:56:33.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.559+0000 7fd1054d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1000d0650 con 0x7fd1000d16f0 2023-10-08T05:56:33.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.559+0000 7fd0fffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd10015cc60 0x7fd1000cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35272/0 (socket says 172.21.15.70:35272) 2023-10-08T05:56:33.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.559+0000 7fd0fffff640 1 -- 172.21.15.70:0/3180088118 learned_addr learned my addr 172.21.15.70:0/3180088118 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:33.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.559+0000 7fd104cd6640 1 --2- 172.21.15.70:0/3180088118 >> v2:172.21.15.70:3300/0 conn(0x7fd1000cfa50 0x7fd1000ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.560+0000 7fd0ff7fe640 1 --2- 172.21.15.70:0/3180088118 >> v2:172.21.15.181:3300/0 conn(0x7fd1000d16f0 0x7fd1000cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.560+0000 7fd0feffd640 1 -- 172.21.15.70:0/3180088118 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194364747 0 0) 0x7fd1000cea50 con 0x7fd10015cc60 2023-10-08T05:56:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.560+0000 7fd0feffd640 1 -- 172.21.15.70:0/3180088118 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0e8003520 con 0x7fd10015cc60 2023-10-08T05:56:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.560+0000 7fd104cd6640 1 -- 172.21.15.70:0/3180088118 >> v2:172.21.15.181:3300/0 conn(0x7fd1000d16f0 msgr2=0x7fd1000cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.560+0000 7fd104cd6640 1 --2- 172.21.15.70:0/3180088118 >> v2:172.21.15.181:3300/0 conn(0x7fd1000d16f0 0x7fd1000cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.560+0000 7fd104cd6640 1 -- 172.21.15.70:0/3180088118 >> v1:172.21.15.70:6789/0 conn(0x7fd10015cc60 legacy=0x7fd1000cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.560+0000 7fd104cd6640 1 -- 172.21.15.70:0/3180088118 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000d0930 con 0x7fd1000cfa50 2023-10-08T05:56:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.561+0000 7fd104cd6640 1 --2- 172.21.15.70:0/3180088118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 0x7fd1000ce510 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fd1000b5560 tx=0x7fd0f802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3e8f800b849e59 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.561+0000 7fd0feffd640 1 -- 172.21.15.70:0/3180088118 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f802cc80 con 0x7fd1000cfa50 2023-10-08T05:56:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.561+0000 7fd0feffd640 1 -- 172.21.15.70:0/3180088118 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f802cde0 con 0x7fd1000cfa50 2023-10-08T05:56:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.561+0000 7fd0feffd640 1 -- 172.21.15.70:0/3180088118 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8034660 con 0x7fd1000cfa50 2023-10-08T05:56:33.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.562+0000 7fd1054d7640 1 -- 172.21.15.70:0/3180088118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 msgr2=0x7fd1000ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.562+0000 7fd1054d7640 1 --2- 172.21.15.70:0/3180088118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 0x7fd1000ce510 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fd1000b5560 tx=0x7fd0f802bed0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.562+0000 7fd1054d7640 1 -- 172.21.15.70:0/3180088118 shutdown_connections 2023-10-08T05:56:33.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.562+0000 7fd1054d7640 1 --2- 172.21.15.70:0/3180088118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 0x7fd1000ce510 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.562+0000 7fd1054d7640 1 --2- 172.21.15.70:0/3180088118 >> v2:172.21.15.181:3300/0 conn(0x7fd1000d16f0 0x7fd1000cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.562+0000 7fd1054d7640 1 -- 172.21.15.70:0/3180088118 >> 172.21.15.70:0/3180088118 conn(0x7fd10009bcf0 msgr2=0x7fd1000d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:33.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.562+0000 7fd1054d7640 1 -- 172.21.15.70:0/3180088118 shutdown_connections 2023-10-08T05:56:33.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.562+0000 7fd1054d7640 1 -- 172.21.15.70:0/3180088118 wait complete. 2023-10-08T05:56:33.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.563+0000 7fd1054d7640 1 Processor -- start 2023-10-08T05:56:33.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.564+0000 7fd1054d7640 1 -- start start 2023-10-08T05:56:33.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.564+0000 7fd1054d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 0x7fd1000c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.564+0000 7fd1054d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1000d16f0 0x7fd1000cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.564+0000 7fd1054d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd10015cc60 0x7fd1000cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.564+0000 7fd1054d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000b3f80 con 0x7fd1000cfa50 2023-10-08T05:56:33.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd1054d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000b4230 con 0x7fd10015cc60 2023-10-08T05:56:33.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd1054d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000b4510 con 0x7fd1000d16f0 2023-10-08T05:56:33.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd0fffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 0x7fd1000c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd0fffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 0x7fd1000c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41334/0 (socket says 172.21.15.70:41334) 2023-10-08T05:56:33.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd0fffff640 1 -- 172.21.15.70:0/3695281619 learned_addr learned my addr 172.21.15.70:0/3695281619 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:33.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd104cd6640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd10015cc60 0x7fd1000cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd0fffff640 1 -- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1000d16f0 msgr2=0x7fd1000cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd0fffff640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1000d16f0 0x7fd1000cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd0fffff640 1 -- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd10015cc60 msgr2=0x7fd1000cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd0fffff640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd10015cc60 0x7fd1000cbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.565+0000 7fd0fffff640 1 -- 172.21.15.70:0/3695281619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f8009a70 con 0x7fd1000cfa50 2023-10-08T05:56:33.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.566+0000 7fd0fffff640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 0x7fd1000c8f40 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0001260 tx=0x7fd0f000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:33.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.566+0000 7fd0fcff9640 1 -- 172.21.15.70:0/3695281619 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0007480 con 0x7fd1000cfa50 2023-10-08T05:56:33.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.566+0000 7fd0fcff9640 1 -- 172.21.15.70:0/3695281619 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f00075e0 con 0x7fd1000cfa50 2023-10-08T05:56:33.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.566+0000 7fd0fcff9640 1 -- 172.21.15.70:0/3695281619 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00059c0 con 0x7fd1000cfa50 2023-10-08T05:56:33.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.566+0000 7fd1054d7640 1 -- 172.21.15.70:0/3695281619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1000b47f0 con 0x7fd1000cfa50 2023-10-08T05:56:33.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.566+0000 7fd1054d7640 1 -- 172.21.15.70:0/3695281619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1000b4db0 con 0x7fd1000cfa50 2023-10-08T05:56:33.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.567+0000 7fd1054d7640 1 -- 172.21.15.70:0/3695281619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1000b4fd0 con 0x7fd1000cfa50 2023-10-08T05:56:33.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.568+0000 7fd0fcff9640 1 -- 172.21.15.70:0/3695281619 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd0f0005b20 con 0x7fd1000cfa50 2023-10-08T05:56:33.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.569+0000 7fd0fcff9640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd0ec06af90 0x7fd0ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:33.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.569+0000 7fd0fcff9640 1 -- 172.21.15.70:0/3695281619 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0f0088e70 con 0x7fd1000cfa50 2023-10-08T05:56:33.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.570+0000 7fd0ff7fe640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd0ec06af90 0x7fd0ec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:33.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.571+0000 7fd0ff7fe640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd0ec06af90 0x7fd0ec06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40098f0 tx=0x7fd0f4008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:33.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.573+0000 7fd0fcff9640 1 -- 172.21.15.70:0/3695281619 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0f0053d10 con 0x7fd1000cfa50 2023-10-08T05:56:33.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.762+0000 7fd1054d7640 1 -- 172.21.15.70:0/3695281619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd10004e160 con 0x7fd1000cfa50 2023-10-08T05:56:33.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.765+0000 7fd0fcff9640 1 -- 172.21.15.70:0/3695281619 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16887 (secure 0 0 0) 0x7fd0f0017070 con 0x7fd1000cfa50 2023-10-08T05:56:33.767 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:33.767 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":149,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:33.417285+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:56:30.775239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-08T05:56:31.378327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:33.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 -- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd0ec06af90 msgr2=0x7fd0ec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd0ec06af90 0x7fd0ec06d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40098f0 tx=0x7fd0f4008040 comp rx=0 tx=0).stop 2023-10-08T05:56:33.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 -- 172.21.15.70:0/3695281619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 msgr2=0x7fd1000c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:33.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 0x7fd1000c8f40 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0001260 tx=0x7fd0f000c960 comp rx=0 tx=0).stop 2023-10-08T05:56:33.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 -- 172.21.15.70:0/3695281619 shutdown_connections 2023-10-08T05:56:33.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fd0ec06af90 0x7fd0ec06d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd10015cc60 0x7fd1000cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1000d16f0 0x7fd1000cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 --2- 172.21.15.70:0/3695281619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1000cfa50 0x7fd1000c8f40 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:33.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 -- 172.21.15.70:0/3695281619 >> 172.21.15.70:0/3695281619 conn(0x7fd10009bcf0 msgr2=0x7fd10015d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:33.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 -- 172.21.15.70:0/3695281619 shutdown_connections 2023-10-08T05:56:33.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:33.768+0000 7fd1054d7640 1 -- 172.21.15.70:0/3695281619 wait complete. 2023-10-08T05:56:33.789 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T05:56:34.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.008+0000 7f63cb397640 1 Processor -- start 2023-10-08T05:56:34.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.008+0000 7f63cb397640 1 -- start start 2023-10-08T05:56:34.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.009+0000 7f63cb397640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63c40cb130 0x7f63c40cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:34.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.009+0000 7f63cb397640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c40b1cc0 con 0x7f63c40afbe0 2023-10-08T05:56:34.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.009+0000 7f63cb397640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c40cbad0 con 0x7f63c406dfc0 2023-10-08T05:56:34.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.009+0000 7f63cb397640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40b2620 con 0x7f63c40cb130 2023-10-08T05:56:34.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.010+0000 7f63c9133640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f63c40afbe0 0x7f63c40b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35288/0 (socket says 172.21.15.70:35288) 2023-10-08T05:56:34.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.010+0000 7f63c9133640 1 -- 172.21.15.70:0/344078683 learned_addr learned my addr 172.21.15.70:0/344078683 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:34.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.010+0000 7f63c9934640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63c40cb130 0x7f63c40cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:34.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.010+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905880685 0 0) 0x7f63c40b1cc0 con 0x7f63c40afbe0 2023-10-08T05:56:34.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.010+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63b0003520 con 0x7f63c40afbe0 2023-10-08T05:56:34.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.010+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3979948114 0 0) 0x7f63b0003520 con 0x7f63c40afbe0 2023-10-08T05:56:34.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.010+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63c40cb130 msgr2=0x7f63c40cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:34.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.010+0000 7f63bbfff640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63c40cb130 0x7f63c40cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:34.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.010+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 >> v1:172.21.15.181:6789/0 conn(0x7f63c406dfc0 legacy=0x7f63c4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:34.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c40b2060 con 0x7f63c40afbe0 2023-10-08T05:56:34.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f63c0001670 con 0x7f63c40afbe0 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 >> v1:172.21.15.70:6789/0 conn(0x7f63c40afbe0 legacy=0x7f63c40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63bbfff640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b00039a0 0x7f63b0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63bbfff640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00047c0 0x7f63b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63bbfff640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63c40cb130 0x7f63b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40b2620 con 0x7f63b00039a0 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b000ba90 con 0x7f63b00047c0 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b000bc50 con 0x7f63c40cb130 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63c9133640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b00039a0 0x7f63b0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63c8932640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00047c0 0x7f63b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63c9934640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63c40cb130 0x7f63b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63c9133640 1 -- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63c40cb130 msgr2=0x7f63b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:34.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63c9133640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63c40cb130 0x7f63b0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:34.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63c9133640 1 -- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00047c0 msgr2=0x7f63b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:34.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63c9133640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00047c0 0x7f63b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:34.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63c9133640 1 -- 172.21.15.70:0/344078683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b0003520 con 0x7f63b00039a0 2023-10-08T05:56:34.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63c9934640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63c40cb130 0x7f63b0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:34.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.011+0000 7f63c8932640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63b00047c0 0x7f63b0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:34.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.012+0000 7f63c9133640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b00039a0 0x7f63b0003dc0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f63c0001eb0 tx=0x7f63b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703ffb9130734d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:34.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.013+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0005ad0 con 0x7f63b00039a0 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.013+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c0005c30 con 0x7f63b00039a0 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.013+0000 7f63bbfff640 1 -- 172.21.15.70:0/344078683 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c002c7a0 con 0x7f63b00039a0 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.013+0000 7f63cb397640 1 -- 172.21.15.70:0/344078683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b00039a0 msgr2=0x7f63b0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.013+0000 7f63cb397640 1 --2- 172.21.15.70:0/344078683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63b00039a0 0x7f63b0003dc0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f63c0001eb0 tx=0x7f63b00057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.013+0000 7f63c9934640 1 -- 172.21.15.70:0/344078683 reap_dead start 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.013+0000 7f63cb397640 1 -- 172.21.15.70:0/344078683 shutdown_connections 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.013+0000 7f63cb397640 1 -- 172.21.15.70:0/344078683 >> 172.21.15.70:0/344078683 conn(0x7f63c406a5d0 msgr2=0x7f63c40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.013+0000 7f63cb397640 1 -- 172.21.15.70:0/344078683 shutdown_connections 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.013+0000 7f63cb397640 1 -- 172.21.15.70:0/344078683 wait complete. 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.015+0000 7f63cb397640 1 Processor -- start 2023-10-08T05:56:34.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.015+0000 7f63cb397640 1 -- start start 2023-10-08T05:56:34.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.016+0000 7f63cb397640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b00039a0 0x7f63c40ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:34.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.016+0000 7f63cb397640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63c415c080 0x7f63c414dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:34.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.016+0000 7f63cb397640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63c40c4710 0x7f63c40c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:34.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.016+0000 7f63cb397640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40c3270 con 0x7f63c415c080 2023-10-08T05:56:34.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.016+0000 7f63cb397640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40c5100 con 0x7f63c40c4710 2023-10-08T05:56:34.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.016+0000 7f63cb397640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c40ca9b0 con 0x7f63b00039a0 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c8932640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63c415c080 0x7f63c414dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c8932640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63c415c080 0x7f63c414dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41352/0 (socket says 172.21.15.70:41352) 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c8932640 1 -- 172.21.15.70:0/1802641684 learned_addr learned my addr 172.21.15.70:0/1802641684 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c9133640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b00039a0 0x7f63c40ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c8932640 1 -- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b00039a0 msgr2=0x7f63c40ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c8932640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b00039a0 0x7f63c40ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c8932640 1 -- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63c40c4710 msgr2=0x7f63c40c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c9934640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63c40c4710 0x7f63c40c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c8932640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63c40c4710 0x7f63c40c4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c8932640 1 -- 172.21.15.70:0/1802641684 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c414e180 con 0x7f63c415c080 2023-10-08T05:56:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.017+0000 7f63c8932640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63c415c080 0x7f63c414dc40 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f63b400d7d0 tx=0x7f63b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.018+0000 7f63b9ffb640 1 -- 172.21.15.70:0/1802641684 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4017070 con 0x7f63c415c080 2023-10-08T05:56:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.018+0000 7f63b9ffb640 1 -- 172.21.15.70:0/1802641684 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b4005070 con 0x7f63c415c080 2023-10-08T05:56:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.018+0000 7f63b9ffb640 1 -- 172.21.15.70:0/1802641684 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4005400 con 0x7f63c415c080 2023-10-08T05:56:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.018+0000 7f63cb397640 1 -- 172.21.15.70:0/1802641684 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c40c93d0 con 0x7f63c415c080 2023-10-08T05:56:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.018+0000 7f63cb397640 1 -- 172.21.15.70:0/1802641684 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c40c9990 con 0x7f63c415c080 2023-10-08T05:56:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.019+0000 7f63cb397640 1 -- 172.21.15.70:0/1802641684 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c404e160 con 0x7f63c415c080 2023-10-08T05:56:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.020+0000 7f63b9ffb640 1 -- 172.21.15.70:0/1802641684 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f63b4007030 con 0x7f63c415c080 2023-10-08T05:56:34.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.021+0000 7f63b9ffb640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f63a806afc0 0x7f63a806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:34.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.021+0000 7f63b9ffb640 1 -- 172.21.15.70:0/1802641684 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63b4089100 con 0x7f63c415c080 2023-10-08T05:56:34.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.022+0000 7f63c9133640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f63a806afc0 0x7f63a806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:34.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.023+0000 7f63c9133640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f63a806afc0 0x7f63a806d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63c0034000 tx=0x7f63c0009030 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:34.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.025+0000 7f63b9ffb640 1 -- 172.21.15.70:0/1802641684 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63b4053e90 con 0x7f63c415c080 2023-10-08T05:56:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.235+0000 7f63cb397640 1 -- 172.21.15.70:0/1802641684 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f63c40b0ef0 con 0x7f63c415c080 2023-10-08T05:56:34.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.937+0000 7f63b9ffb640 1 -- 172.21.15.70:0/1802641684 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v67) v1 ==== 118+0+0 (secure 0 0 0) 0x7f63b40574b0 con 0x7f63c415c080 2023-10-08T05:56:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 -- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f63a806afc0 msgr2=0x7f63a806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f63a806afc0 0x7f63a806d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f63c0034000 tx=0x7f63c0009030 comp rx=0 tx=0).stop 2023-10-08T05:56:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 -- 172.21.15.70:0/1802641684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63c415c080 msgr2=0x7f63c414dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63c415c080 0x7f63c414dc40 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f63b400d7d0 tx=0x7f63b400dc90 comp rx=0 tx=0).stop 2023-10-08T05:56:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 -- 172.21.15.70:0/1802641684 shutdown_connections 2023-10-08T05:56:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f63a806afc0 0x7f63a806d480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63c40c4710 0x7f63c40c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:34.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63c415c080 0x7f63c414dc40 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:34.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 --2- 172.21.15.70:0/1802641684 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63b00039a0 0x7f63c40ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:34.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 -- 172.21.15.70:0/1802641684 >> 172.21.15.70:0/1802641684 conn(0x7f63c406a5d0 msgr2=0x7f63c40c7310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:34.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 -- 172.21.15.70:0/1802641684 shutdown_connections 2023-10-08T05:56:34.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:34.941+0000 7f63cb397640 1 -- 172.21.15.70:0/1802641684 wait complete. 2023-10-08T05:56:34.961 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T05:56:34.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T05:56:35.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.175+0000 7f30789d0640 1 Processor -- start 2023-10-08T05:56:35.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.175+0000 7f30789d0640 1 -- start start 2023-10-08T05:56:35.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.177+0000 7f30789d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 0x7f30740c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:35.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.177+0000 7f30789d0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f307415aa00 0x7f30740b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:35.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.177+0000 7f30789d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307415b310 con 0x7f307406db60 2023-10-08T05:56:35.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.177+0000 7f30789d0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30740ca230 con 0x7f307415aa00 2023-10-08T05:56:35.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.177+0000 7f30789d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307406d290 con 0x7f307406c3a0 2023-10-08T05:56:35.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.177+0000 7f306bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f307406db60 0x7f307406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35296/0 (socket says 172.21.15.70:35296) 2023-10-08T05:56:35.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.177+0000 7f306bfff640 1 -- 172.21.15.70:0/3019010013 learned_addr learned my addr 172.21.15.70:0/3019010013 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:35.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.178+0000 7f3072d9d640 1 --2- 172.21.15.70:0/3019010013 >> v2:172.21.15.70:3300/0 conn(0x7f307415aa00 0x7f30740b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:35.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.178+0000 7f307259c640 1 --2- 172.21.15.70:0/3019010013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 0x7f30740c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:35.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.178+0000 7f3071d9b640 1 -- 172.21.15.70:0/3019010013 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244607134 0 0) 0x7f305c001110 con 0x7f307406db60 2023-10-08T05:56:35.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.178+0000 7f3071d9b640 1 -- 172.21.15.70:0/3019010013 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3060005730 con 0x7f307406db60 2023-10-08T05:56:35.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.178+0000 7f3072d9d640 1 -- 172.21.15.70:0/3019010013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 msgr2=0x7f30740c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:35.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.178+0000 7f3072d9d640 1 --2- 172.21.15.70:0/3019010013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 0x7f30740c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:35.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.178+0000 7f3072d9d640 1 -- 172.21.15.70:0/3019010013 >> v1:172.21.15.70:6789/0 conn(0x7f307406db60 legacy=0x7f307406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:35.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.178+0000 7f3072d9d640 1 -- 172.21.15.70:0/3019010013 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740c9860 con 0x7f307415aa00 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.179+0000 7f3072d9d640 1 --2- 172.21.15.70:0/3019010013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 0x7f30740b2a90 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f306c009d90 tx=0x7f306c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41839d7fc9a50f27 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.179+0000 7f3071d9b640 1 -- 172.21.15.70:0/3019010013 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c02cc60 con 0x7f307415aa00 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.179+0000 7f3071d9b640 1 -- 172.21.15.70:0/3019010013 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c02cdc0 con 0x7f307415aa00 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.179+0000 7f3071d9b640 1 -- 172.21.15.70:0/3019010013 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c034640 con 0x7f307415aa00 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.180+0000 7f30789d0640 1 -- 172.21.15.70:0/3019010013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 msgr2=0x7f30740b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.180+0000 7f30789d0640 1 --2- 172.21.15.70:0/3019010013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 0x7f30740b2a90 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f306c009d90 tx=0x7f306c02beb0 comp rx=0 tx=0).stop 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.180+0000 7f30789d0640 1 -- 172.21.15.70:0/3019010013 shutdown_connections 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.180+0000 7f30789d0640 1 --2- 172.21.15.70:0/3019010013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 0x7f30740b2a90 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.180+0000 7f30789d0640 1 --2- 172.21.15.70:0/3019010013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 0x7f30740c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.180+0000 7f30789d0640 1 -- 172.21.15.70:0/3019010013 >> 172.21.15.70:0/3019010013 conn(0x7f307406a5d0 msgr2=0x7f30740b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:35.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.180+0000 7f30789d0640 1 -- 172.21.15.70:0/3019010013 shutdown_connections 2023-10-08T05:56:35.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.180+0000 7f30789d0640 1 -- 172.21.15.70:0/3019010013 wait complete. 2023-10-08T05:56:35.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.182+0000 7f30789d0640 1 Processor -- start 2023-10-08T05:56:35.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.182+0000 7f30789d0640 1 -- start start 2023-10-08T05:56:35.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f30789d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 0x7f3074158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:35.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f30789d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f307406db60 0x7f307415c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:35.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f30789d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 0x7f307415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:35.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f30789d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307415b7e0 con 0x7f307415aa00 2023-10-08T05:56:35.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f30789d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3074159110 con 0x7f307406db60 2023-10-08T05:56:35.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f30789d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307415cfb0 con 0x7f307406c3a0 2023-10-08T05:56:35.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f3072d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 0x7f307415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:35.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f3072d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 0x7f307415c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41372/0 (socket says 172.21.15.70:41372) 2023-10-08T05:56:35.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f3072d9d640 1 -- 172.21.15.70:0/2826261327 learned_addr learned my addr 172.21.15.70:0/2826261327 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:35.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f307259c640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 0x7f3074158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:35.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f306bfff640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f307406db60 0x7f307415c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:35.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f3072d9d640 1 -- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 msgr2=0x7f3074158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:35.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f3072d9d640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 0x7f3074158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:35.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f3072d9d640 1 -- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f307406db60 msgr2=0x7f307415c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:35.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f3072d9d640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f307406db60 0x7f307415c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:35.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f3072d9d640 1 -- 172.21.15.70:0/2826261327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c009a70 con 0x7f307415aa00 2023-10-08T05:56:35.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.183+0000 7f307259c640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 0x7f3074158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:35.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.184+0000 7f3072d9d640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 0x7f307415c910 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f306c02c4c0 tx=0x7f306c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:35.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.184+0000 7f306affd640 1 -- 172.21.15.70:0/2826261327 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c03e610 con 0x7f307415aa00 2023-10-08T05:56:35.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.184+0000 7f306affd640 1 -- 172.21.15.70:0/2826261327 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c03e790 con 0x7f307415aa00 2023-10-08T05:56:35.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.184+0000 7f306affd640 1 -- 172.21.15.70:0/2826261327 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c03eb40 con 0x7f307415aa00 2023-10-08T05:56:35.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.184+0000 7f30789d0640 1 -- 172.21.15.70:0/2826261327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307415d290 con 0x7f307415aa00 2023-10-08T05:56:35.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.184+0000 7f30789d0640 1 -- 172.21.15.70:0/2826261327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307415d850 con 0x7f307415aa00 2023-10-08T05:56:35.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.185+0000 7f30789d0640 1 -- 172.21.15.70:0/2826261327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307404e160 con 0x7f307415aa00 2023-10-08T05:56:35.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.186+0000 7f306affd640 1 -- 172.21.15.70:0/2826261327 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f306c03b050 con 0x7f307415aa00 2023-10-08T05:56:35.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.187+0000 7f306affd640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f305406af90 0x7f305406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:35.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.187+0000 7f306affd640 1 -- 172.21.15.70:0/2826261327 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f306c0ac290 con 0x7f307415aa00 2023-10-08T05:56:35.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.188+0000 7f307259c640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f305406af90 0x7f305406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:35.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.189+0000 7f307259c640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f305406af90 0x7f305406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3064006fd0 tx=0x7f3064008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:35.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.191+0000 7f306affd640 1 -- 172.21.15.70:0/2826261327 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f306c077000 con 0x7f307415aa00 2023-10-08T05:56:35.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.417+0000 7f30789d0640 1 -- 172.21.15.70:0/2826261327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f307406d6b0 con 0x7f307415aa00 2023-10-08T05:56:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.941+0000 7f306affd640 1 -- 172.21.15.70:0/2826261327 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v68) v1 ==== 122+0+0 (secure 0 0 0) 0x7f306c009450 con 0x7f307415aa00 2023-10-08T05:56:35.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 -- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f305406af90 msgr2=0x7f305406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:35.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f305406af90 0x7f305406d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3064006fd0 tx=0x7f3064008040 comp rx=0 tx=0).stop 2023-10-08T05:56:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 -- 172.21.15.70:0/2826261327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 msgr2=0x7f307415c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 0x7f307415c910 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f306c02c4c0 tx=0x7f306c004a40 comp rx=0 tx=0).stop 2023-10-08T05:56:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 -- 172.21.15.70:0/2826261327 shutdown_connections 2023-10-08T05:56:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f305406af90 0x7f305406d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f307415aa00 0x7f307415c910 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f307406db60 0x7f307415c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 --2- 172.21.15.70:0/2826261327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f307406c3a0 0x7f3074158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 -- 172.21.15.70:0/2826261327 >> 172.21.15.70:0/2826261327 conn(0x7f307406a5d0 msgr2=0x7f30740ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 -- 172.21.15.70:0/2826261327 shutdown_connections 2023-10-08T05:56:35.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:35.945+0000 7f30789d0640 1 -- 172.21.15.70:0/2826261327 wait complete. 2023-10-08T05:56:35.966 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T05:56:35.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:56:36.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.178+0000 7f44cbb8a640 1 Processor -- start 2023-10-08T05:56:36.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.178+0000 7f44cbb8a640 1 -- start start 2023-10-08T05:56:36.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.180+0000 7f44cbb8a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c415cd90 0x7f44c40702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.180+0000 7f44cbb8a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44c406c890 0x7f44c406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.180+0000 7f44cbb8a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c40c48f0 con 0x7f44c4160940 2023-10-08T05:56:36.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.180+0000 7f44cbb8a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c40c4d80 con 0x7f44c406c890 2023-10-08T05:56:36.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.180+0000 7f44cbb8a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40c3ce0 con 0x7f44c415cd90 2023-10-08T05:56:36.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.180+0000 7f44c9125640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f44c4160940 0x7f44c4070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35304/0 (socket says 172.21.15.70:35304) 2023-10-08T05:56:36.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.180+0000 7f44c9125640 1 -- 172.21.15.70:0/1251397208 learned_addr learned my addr 172.21.15.70:0/1251397208 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:36.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.180+0000 7f44ca127640 1 --2- 172.21.15.70:0/1251397208 >> v2:172.21.15.181:3300/0 conn(0x7f44c406c890 0x7f44c406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.180+0000 7f44c9926640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c415cd90 0x7f44c40702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819101822 0 0) 0x7f44c40c48f0 con 0x7f44c4160940 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b0003520 con 0x7f44c4160940 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3688602449 0 0) 0x7f44b0003520 con 0x7f44c4160940 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c415cd90 msgr2=0x7f44c40702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c415cd90 0x7f44c40702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 >> v2:172.21.15.181:3300/0 conn(0x7f44c406c890 msgr2=0x7f44c406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 --2- 172.21.15.70:0/1251397208 >> v2:172.21.15.181:3300/0 conn(0x7f44c406c890 0x7f44c406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c40c3fc0 con 0x7f44c4160940 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f44b4001670 con 0x7f44c4160940 2023-10-08T05:56:36.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 >> v1:172.21.15.70:6789/0 conn(0x7f44c4160940 legacy=0x7f44c4070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b0003970 0x7f44b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b00047c0 0x7f44b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c415cd90 0x7f44b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40c3ce0 con 0x7f44b00047c0 2023-10-08T05:56:36.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b000b980 con 0x7f44b0003970 2023-10-08T05:56:36.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b000bac0 con 0x7f44c415cd90 2023-10-08T05:56:36.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.181+0000 7f44c9125640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b00047c0 0x7f44b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.182+0000 7f44c9926640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b0003970 0x7f44b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.182+0000 7f44ca127640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c415cd90 0x7f44b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.182+0000 7f44c9125640 1 -- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c415cd90 msgr2=0x7f44b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:36.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.182+0000 7f44c9125640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c415cd90 0x7f44b0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.182+0000 7f44c9125640 1 -- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b0003970 msgr2=0x7f44b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:36.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.182+0000 7f44c9125640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b0003970 0x7f44b0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.182+0000 7f44c9125640 1 -- 172.21.15.70:0/1251397208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0003520 con 0x7f44b00047c0 2023-10-08T05:56:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.183+0000 7f44c9125640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b00047c0 0x7f44b0004c40 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f44b400c0e0 tx=0x7f44b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59e6f5452c7d789 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.183+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4005ad0 con 0x7f44b00047c0 2023-10-08T05:56:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.183+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4005c30 con 0x7f44b00047c0 2023-10-08T05:56:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.183+0000 7f44c8924640 1 -- 172.21.15.70:0/1251397208 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b402c7f0 con 0x7f44b00047c0 2023-10-08T05:56:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.184+0000 7f44cbb8a640 1 -- 172.21.15.70:0/1251397208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b00047c0 msgr2=0x7f44b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.184+0000 7f44cbb8a640 1 --2- 172.21.15.70:0/1251397208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44b00047c0 0x7f44b0004c40 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f44b400c0e0 tx=0x7f44b00057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.184+0000 7f44ca127640 1 -- 172.21.15.70:0/1251397208 reap_dead start 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.184+0000 7f44cbb8a640 1 -- 172.21.15.70:0/1251397208 shutdown_connections 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.184+0000 7f44cbb8a640 1 -- 172.21.15.70:0/1251397208 >> 172.21.15.70:0/1251397208 conn(0x7f44c406a380 msgr2=0x7f44c40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.184+0000 7f44cbb8a640 1 -- 172.21.15.70:0/1251397208 shutdown_connections 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.184+0000 7f44cbb8a640 1 -- 172.21.15.70:0/1251397208 wait complete. 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.186+0000 7f44cbb8a640 1 Processor -- start 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.186+0000 7f44cbb8a640 1 -- start start 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.187+0000 7f44cbb8a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b00047c0 0x7f44c40c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.187+0000 7f44cbb8a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c40b1d80 0x7f44c40ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.187+0000 7f44cbb8a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c40ae860 0x7f44c41f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.187+0000 7f44cbb8a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40aece0 con 0x7f44c40b1d80 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.187+0000 7f44cbb8a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40b2a90 con 0x7f44b00047c0 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.187+0000 7f44cbb8a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40b3520 con 0x7f44c40ae860 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.187+0000 7f44c9125640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c40b1d80 0x7f44c40ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.187+0000 7f44c9125640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c40b1d80 0x7f44c40ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41390/0 (socket says 172.21.15.70:41390) 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.187+0000 7f44c9125640 1 -- 172.21.15.70:0/4108773037 learned_addr learned my addr 172.21.15.70:0/4108773037 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44c9926640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b00047c0 0x7f44c40c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44c9125640 1 -- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c40ae860 msgr2=0x7f44c41f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44ca127640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c40ae860 0x7f44c41f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44c9125640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c40ae860 0x7f44c41f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44c9125640 1 -- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b00047c0 msgr2=0x7f44c40c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44c9125640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b00047c0 0x7f44c40c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44c9125640 1 -- 172.21.15.70:0/4108773037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c40b3800 con 0x7f44c40b1d80 2023-10-08T05:56:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44c9926640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b00047c0 0x7f44c40c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44ca127640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c40ae860 0x7f44c41f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44c9125640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c40b1d80 0x7f44c40ae320 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f44b400c200 tx=0x7f44c40c4860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44ba7fc640 1 -- 172.21.15.70:0/4108773037 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b402cc00 con 0x7f44c40b1d80 2023-10-08T05:56:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.188+0000 7f44ba7fc640 1 -- 172.21.15.70:0/4108773037 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b402cd60 con 0x7f44c40b1d80 2023-10-08T05:56:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.189+0000 7f44ba7fc640 1 -- 172.21.15.70:0/4108773037 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b40072b0 con 0x7f44c40b1d80 2023-10-08T05:56:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.189+0000 7f44cbb8a640 1 -- 172.21.15.70:0/4108773037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c40b3ae0 con 0x7f44c40b1d80 2023-10-08T05:56:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.189+0000 7f44cbb8a640 1 -- 172.21.15.70:0/4108773037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c40c85f0 con 0x7f44c40b1d80 2023-10-08T05:56:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.190+0000 7f44cbb8a640 1 -- 172.21.15.70:0/4108773037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c4067490 con 0x7f44c40b1d80 2023-10-08T05:56:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.191+0000 7f44ba7fc640 1 -- 172.21.15.70:0/4108773037 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f44b4032070 con 0x7f44c40b1d80 2023-10-08T05:56:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.192+0000 7f44ba7fc640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 0x7f44a806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.192+0000 7f44ba7fc640 1 -- 172.21.15.70:0/4108773037 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f44b403c040 con 0x7f44c40b1d80 2023-10-08T05:56:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.192+0000 7f44c9926640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 0x7f44a806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.193+0000 7f44c9926640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 0x7f44a806d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009890 tx=0x7f44c0008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:36.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.196+0000 7f44ba7fc640 1 -- 172.21.15.70:0/4108773037 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44b4075870 con 0x7f44c40b1d80 2023-10-08T05:56:36.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.424+0000 7f44cbb8a640 1 -- 172.21.15.70:0/4108773037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44c40b0ef0 con 0x7f44c40b1d80 2023-10-08T05:56:36.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.425+0000 7f44ba7fc640 1 -- 172.21.15.70:0/4108773037 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f44b4045020 con 0x7f44c40b1d80 2023-10-08T05:56:36.426 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:36.426 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1820637663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1820637663},{"type":"v1","addr":"172.21.15.181:6835","nonce":1820637663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1666874980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1666874980},{"type":"v1","addr":"172.21.15.181:6837","nonce":1666874980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2918183907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2918183907},{"type":"v1","addr":"172.21.15.70:6834","nonce":2918183907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:56:32.378187+0000","modified":"2023-10-08T05:56:35.419118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6835/3869116018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3869116018},{"type":"v1","addr":"172.21.15.70:6835","nonce":3869116018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-08T05:56:36.426 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 68 2023-10-08T05:56:36.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 -- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 msgr2=0x7f44a806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 0x7f44a806d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009890 tx=0x7f44c0008040 comp rx=0 tx=0).stop 2023-10-08T05:56:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 -- 172.21.15.70:0/4108773037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c40b1d80 msgr2=0x7f44c40ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c40b1d80 0x7f44c40ae320 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f44b400c200 tx=0x7f44c40c4860 comp rx=0 tx=0).stop 2023-10-08T05:56:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 -- 172.21.15.70:0/4108773037 shutdown_connections 2023-10-08T05:56:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 0x7f44a806d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c40ae860 0x7f44c41f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c40b1d80 0x7f44c40ae320 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 --2- 172.21.15.70:0/4108773037 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44b00047c0 0x7f44c40c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 -- 172.21.15.70:0/4108773037 >> 172.21.15.70:0/4108773037 conn(0x7f44c406a380 msgr2=0x7f44c40cc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 -- 172.21.15.70:0/4108773037 shutdown_connections 2023-10-08T05:56:36.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.428+0000 7f44cbb8a640 1 -- 172.21.15.70:0/4108773037 wait complete. 2023-10-08T05:56:36.448 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:36.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.662+0000 7f3fad19e640 1 Processor -- start 2023-10-08T05:56:36.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.662+0000 7f3fad19e640 1 -- start start 2023-10-08T05:56:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.663+0000 7f3fad19e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa8159ac0 0x7f3fa806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.664+0000 7f3fad19e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3fa80709e0 0x7f3fa80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.664+0000 7f3fad19e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa806dee0 con 0x7f3fa80ad790 2023-10-08T05:56:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.664+0000 7f3fad19e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa815d850 con 0x7f3fa80709e0 2023-10-08T05:56:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.664+0000 7f3fad19e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa815db30 con 0x7f3fa8159ac0 2023-10-08T05:56:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.664+0000 7f3fa759e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3fa80709e0 0x7f3fa80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.664+0000 7f3fa759e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3fa80709e0 0x7f3fa80c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39706/0 (socket says 172.21.15.70:39706) 2023-10-08T05:56:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.664+0000 7f3fa6d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa8159ac0 0x7f3fa806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.664+0000 7f3fa759e640 1 -- 172.21.15.70:0/3149144871 learned_addr learned my addr 172.21.15.70:0/3149144871 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:36.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.665+0000 7f3fa759e640 1 -- 172.21.15.70:0/3149144871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa8159ac0 msgr2=0x7f3fa806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:36.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.665+0000 7f3fa759e640 1 --2- 172.21.15.70:0/3149144871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa8159ac0 0x7f3fa806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.665+0000 7f3fa759e640 1 -- 172.21.15.70:0/3149144871 >> v1:172.21.15.181:6789/0 conn(0x7f3fa80ad790 legacy=0x7f3fa80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.665+0000 7f3fa759e640 1 -- 172.21.15.70:0/3149144871 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa815de10 con 0x7f3fa80709e0 2023-10-08T05:56:36.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.666+0000 7f3fa759e640 1 --2- 172.21.15.70:0/3149144871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa80709e0 0x7f3fa80c8dd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009870 tx=0x7f3fa002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c3d565c8cedb173 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:36.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.667+0000 7f3fa5d9b640 1 -- 172.21.15.70:0/3149144871 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0030840 con 0x7f3fa80709e0 2023-10-08T05:56:36.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.667+0000 7f3fa5d9b640 1 -- 172.21.15.70:0/3149144871 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa00309a0 con 0x7f3fa80709e0 2023-10-08T05:56:36.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.667+0000 7f3fa5d9b640 1 -- 172.21.15.70:0/3149144871 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0030d30 con 0x7f3fa80709e0 2023-10-08T05:56:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.667+0000 7f3fad19e640 1 -- 172.21.15.70:0/3149144871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa80709e0 msgr2=0x7f3fa80c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.667+0000 7f3fad19e640 1 --2- 172.21.15.70:0/3149144871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa80709e0 0x7f3fa80c8dd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009870 tx=0x7f3fa002b980 comp rx=0 tx=0).stop 2023-10-08T05:56:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.668+0000 7f3fad19e640 1 -- 172.21.15.70:0/3149144871 shutdown_connections 2023-10-08T05:56:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.668+0000 7f3fad19e640 1 --2- 172.21.15.70:0/3149144871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa80709e0 0x7f3fa80c8dd0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.668+0000 7f3fad19e640 1 --2- 172.21.15.70:0/3149144871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa8159ac0 0x7f3fa806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.668+0000 7f3fad19e640 1 -- 172.21.15.70:0/3149144871 >> 172.21.15.70:0/3149144871 conn(0x7f3fa806a380 msgr2=0x7f3fa80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:36.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.668+0000 7f3fad19e640 1 -- 172.21.15.70:0/3149144871 shutdown_connections 2023-10-08T05:56:36.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.668+0000 7f3fad19e640 1 -- 172.21.15.70:0/3149144871 wait complete. 2023-10-08T05:56:36.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.669+0000 7f3fad19e640 1 Processor -- start 2023-10-08T05:56:36.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.670+0000 7f3fad19e640 1 -- start start 2023-10-08T05:56:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.670+0000 7f3fad19e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa80709e0 0x7f3fa8145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fad19e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa80ad790 0x7f3fa80cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fad19e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa8159ac0 0x7f3fa80b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fad19e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa80ced00 con 0x7f3fa8159ac0 2023-10-08T05:56:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fad19e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa815adc0 con 0x7f3fa80709e0 2023-10-08T05:56:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fad19e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa80ce740 con 0x7f3fa80ad790 2023-10-08T05:56:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fa759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa8159ac0 0x7f3fa80b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fa759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa8159ac0 0x7f3fa80b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41392/0 (socket says 172.21.15.70:41392) 2023-10-08T05:56:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fa759e640 1 -- 172.21.15.70:0/3391161881 learned_addr learned my addr 172.21.15.70:0/3391161881 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:36.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fa6d9d640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa80709e0 0x7f3fa8145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fa659c640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa80ad790 0x7f3fa80cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fa759e640 1 -- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa80ad790 msgr2=0x7f3fa80cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fa759e640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa80ad790 0x7f3fa80cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fa759e640 1 -- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa80709e0 msgr2=0x7f3fa8145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.671+0000 7f3fa759e640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa80709e0 0x7f3fa8145ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.672+0000 7f3fa759e640 1 -- 172.21.15.70:0/3391161881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00094d0 con 0x7f3fa8159ac0 2023-10-08T05:56:36.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.672+0000 7f3fa759e640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa8159ac0 0x7f3fa80b14a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009870 tx=0x7f3fa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:36.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.672+0000 7f3f8f7fe640 1 -- 172.21.15.70:0/3391161881 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0039c90 con 0x7f3fa8159ac0 2023-10-08T05:56:36.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.672+0000 7f3f8f7fe640 1 -- 172.21.15.70:0/3391161881 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0039e10 con 0x7f3fa8159ac0 2023-10-08T05:56:36.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.672+0000 7f3fad19e640 1 -- 172.21.15.70:0/3391161881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa80cea20 con 0x7f3fa8159ac0 2023-10-08T05:56:36.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.672+0000 7f3f8f7fe640 1 -- 172.21.15.70:0/3391161881 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0043640 con 0x7f3fa8159ac0 2023-10-08T05:56:36.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.672+0000 7f3fad19e640 1 -- 172.21.15.70:0/3391161881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa8153440 con 0x7f3fa8159ac0 2023-10-08T05:56:36.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.673+0000 7f3fad19e640 1 -- 172.21.15.70:0/3391161881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa815a8b0 con 0x7f3fa8159ac0 2023-10-08T05:56:36.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.675+0000 7f3f8f7fe640 1 -- 172.21.15.70:0/3391161881 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3fa00437a0 con 0x7f3fa8159ac0 2023-10-08T05:56:36.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.675+0000 7f3f8f7fe640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3f9406af90 0x7f3f9406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:36.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.676+0000 7f3f8f7fe640 1 -- 172.21.15.70:0/3391161881 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3fa003e070 con 0x7f3fa8159ac0 2023-10-08T05:56:36.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.676+0000 7f3fa6d9d640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3f9406af90 0x7f3f9406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:36.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.677+0000 7f3fa6d9d640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3f9406af90 0x7f3f9406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009a30 tx=0x7f3f98008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:36.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.680+0000 7f3f8f7fe640 1 -- 172.21.15.70:0/3391161881 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fa815a8b0 con 0x7f3fa8159ac0 2023-10-08T05:56:36.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.868+0000 7f3fad19e640 1 -- 172.21.15.70:0/3391161881 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fa815a8b0 con 0x7f3fa8159ac0 2023-10-08T05:56:36.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.871+0000 7f3f8f7fe640 1 -- 172.21.15.70:0/3391161881 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16887 (secure 0 0 0) 0x7f3fa815a8b0 con 0x7f3fa8159ac0 2023-10-08T05:56:36.872 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:36.872 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":149,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:33.417285+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:56:30.775239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-08T05:56:31.378327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:36.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 -- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3f9406af90 msgr2=0x7f3f9406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3f9406af90 0x7f3f9406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009a30 tx=0x7f3f98008040 comp rx=0 tx=0).stop 2023-10-08T05:56:36.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 -- 172.21.15.70:0/3391161881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa8159ac0 msgr2=0x7f3fa80b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:36.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa8159ac0 0x7f3fa80b14a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009870 tx=0x7f3fa0004730 comp rx=0 tx=0).stop 2023-10-08T05:56:36.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 -- 172.21.15.70:0/3391161881 shutdown_connections 2023-10-08T05:56:36.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3f9406af90 0x7f3f9406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3fa8159ac0 0x7f3fa80b14a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3fa80ad790 0x7f3fa80cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 --2- 172.21.15.70:0/3391161881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3fa80709e0 0x7f3fa8145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:36.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 -- 172.21.15.70:0/3391161881 >> 172.21.15.70:0/3391161881 conn(0x7f3fa806a380 msgr2=0x7f3fa80c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:36.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 -- 172.21.15.70:0/3391161881 shutdown_connections 2023-10-08T05:56:36.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:36.874+0000 7f3fad19e640 1 -- 172.21.15.70:0/3391161881 wait complete. 2023-10-08T05:56:36.893 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T05:56:36.893 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:56:37.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.106+0000 7f3e3f59e640 1 Processor -- start 2023-10-08T05:56:37.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.107+0000 7f3e3f59e640 1 -- start start 2023-10-08T05:56:37.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.108+0000 7f3e3f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e4015ab30 0x7f3e4015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.108+0000 7f3e3f59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e400b07d0 0x7f3e4006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.108+0000 7f3e3f59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4015f7d0 con 0x7f3e400adc70 2023-10-08T05:56:37.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.108+0000 7f3e3f59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4015f210 con 0x7f3e400b07d0 2023-10-08T05:56:37.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.108+0000 7f3e3f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4015f4f0 con 0x7f3e4015ab30 2023-10-08T05:56:37.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.108+0000 7f3e37fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3e400adc70 0x7f3e400b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35312/0 (socket says 172.21.15.70:35312) 2023-10-08T05:56:37.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.108+0000 7f3e37fff640 1 -- 172.21.15.70:0/3847507709 learned_addr learned my addr 172.21.15.70:0/3847507709 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:37.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267753409 0 0) 0x7f3e4015f7d0 con 0x7f3e400adc70 2023-10-08T05:56:37.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e2c003520 con 0x7f3e400adc70 2023-10-08T05:56:37.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3ed9d640 1 --2- 172.21.15.70:0/3847507709 >> v2:172.21.15.181:3300/0 conn(0x7f3e400b07d0 0x7f3e4006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3e59c640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e4015ab30 0x7f3e4015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1927909156 0 0) 0x7f3e2c003520 con 0x7f3e400adc70 2023-10-08T05:56:37.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e4015ab30 msgr2=0x7f3e4015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e4015ab30 0x7f3e4015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 >> v2:172.21.15.181:3300/0 conn(0x7f3e400b07d0 msgr2=0x7f3e4006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 --2- 172.21.15.70:0/3847507709 >> v2:172.21.15.181:3300/0 conn(0x7f3e400b07d0 0x7f3e4006cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4006c550 con 0x7f3e400adc70 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3e28001670 con 0x7f3e400adc70 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3ed9d640 1 --2- 172.21.15.70:0/3847507709 >> v2:172.21.15.181:3300/0 conn(0x7f3e400b07d0 0x7f3e4006cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 >> v1:172.21.15.70:6789/0 conn(0x7f3e400adc70 legacy=0x7f3e400b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 0x7f3e2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e2c0047c0 0x7f3e2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.109+0000 7f3e3dd9b640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e4015ab30 0x7f3e2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.110+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4015f210 con 0x7f3e2c003970 2023-10-08T05:56:37.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.110+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4015f4f0 con 0x7f3e2c0047c0 2023-10-08T05:56:37.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.110+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c00ba80 con 0x7f3e4015ab30 2023-10-08T05:56:37.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.110+0000 7f3e3e59c640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 0x7f3e2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.110+0000 7f3e37fff640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e2c0047c0 0x7f3e2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.110+0000 7f3e3e59c640 1 -- 172.21.15.70:0/3847507709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e4015ab30 msgr2=0x7f3e2c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:37.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.110+0000 7f3e3e59c640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e4015ab30 0x7f3e2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.110+0000 7f3e3e59c640 1 -- 172.21.15.70:0/3847507709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e2c0047c0 msgr2=0x7f3e2c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:37.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.110+0000 7f3e3e59c640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e2c0047c0 0x7f3e2c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.110+0000 7f3e3e59c640 1 -- 172.21.15.70:0/3847507709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c003520 con 0x7f3e2c003970 2023-10-08T05:56:37.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.111+0000 7f3e3e59c640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 0x7f3e2c003d90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000ba00 tx=0x7f3e2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6114e52bc1512ce6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.111+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30032630 con 0x7f3e2c003970 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.112+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e30032790 con 0x7f3e2c003970 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.112+0000 7f3e3dd9b640 1 -- 172.21.15.70:0/3847507709 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30032b20 con 0x7f3e2c003970 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.112+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3847507709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 msgr2=0x7f3e2c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.112+0000 7f3e3f59e640 1 --2- 172.21.15.70:0/3847507709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 0x7f3e2c003d90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000ba00 tx=0x7f3e2c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.112+0000 7f3e3ed9d640 1 -- 172.21.15.70:0/3847507709 reap_dead start 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.112+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3847507709 shutdown_connections 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.112+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3847507709 >> 172.21.15.70:0/3847507709 conn(0x7f3e400cb0c0 msgr2=0x7f3e400cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.112+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3847507709 shutdown_connections 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.112+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3847507709 wait complete. 2023-10-08T05:56:37.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.114+0000 7f3e3f59e640 1 Processor -- start 2023-10-08T05:56:37.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.114+0000 7f3e3f59e640 1 -- start start 2023-10-08T05:56:37.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.115+0000 7f3e3f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 0x7f3e400c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.115+0000 7f3e3f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e400c6df0 0x7f3e400c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.115+0000 7f3e3f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e400b2850 0x7f3e400b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.115+0000 7f3e3f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e401f1440 con 0x7f3e2c003970 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.115+0000 7f3e3f59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400b3210 con 0x7f3e400c6df0 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.115+0000 7f3e3f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e401f08c0 con 0x7f3e400b2850 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.115+0000 7f3e3e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 0x7f3e400c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.115+0000 7f3e3e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 0x7f3e400c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41410/0 (socket says 172.21.15.70:41410) 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.115+0000 7f3e3e59c640 1 -- 172.21.15.70:0/3806928691 learned_addr learned my addr 172.21.15.70:0/3806928691 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.116+0000 7f3e37fff640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e400c6df0 0x7f3e400c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.116+0000 7f3e3ed9d640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e400b2850 0x7f3e400b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.116+0000 7f3e3e59c640 1 -- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e400b2850 msgr2=0x7f3e400b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.116+0000 7f3e3e59c640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e400b2850 0x7f3e400b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.116+0000 7f3e3e59c640 1 -- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e400c6df0 msgr2=0x7f3e400c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.116+0000 7f3e3e59c640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e400c6df0 0x7f3e400c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.116+0000 7f3e3e59c640 1 -- 172.21.15.70:0/3806928691 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e401f0ba0 con 0x7f3e2c003970 2023-10-08T05:56:37.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.116+0000 7f3e3e59c640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 0x7f3e400c5710 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f3e3002beb0 tx=0x7f3e30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:37.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.117+0000 7f3e36ffd640 1 -- 172.21.15.70:0/3806928691 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30004220 con 0x7f3e2c003970 2023-10-08T05:56:37.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.117+0000 7f3e36ffd640 1 -- 172.21.15.70:0/3806928691 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e300043a0 con 0x7f3e2c003970 2023-10-08T05:56:37.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.117+0000 7f3e36ffd640 1 -- 172.21.15.70:0/3806928691 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3003bab0 con 0x7f3e2c003970 2023-10-08T05:56:37.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.117+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3806928691 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e401f0e80 con 0x7f3e2c003970 2023-10-08T05:56:37.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.117+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3806928691 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e400c5c50 con 0x7f3e2c003970 2023-10-08T05:56:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.118+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3806928691 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e400c5e10 con 0x7f3e2c003970 2023-10-08T05:56:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.119+0000 7f3e36ffd640 1 -- 172.21.15.70:0/3806928691 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3e3003bc10 con 0x7f3e2c003970 2023-10-08T05:56:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.120+0000 7f3e36ffd640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3e2006afc0 0x7f3e2006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.120+0000 7f3e36ffd640 1 -- 172.21.15.70:0/3806928691 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3e300ac340 con 0x7f3e2c003970 2023-10-08T05:56:37.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.121+0000 7f3e37fff640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3e2006afc0 0x7f3e2006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.122+0000 7f3e37fff640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3e2006afc0 0x7f3e2006d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e400c4e80 tx=0x7f3e28009040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:37.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.125+0000 7f3e36ffd640 1 -- 172.21.15.70:0/3806928691 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e30076f10 con 0x7f3e2c003970 2023-10-08T05:56:37.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.312+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3806928691 --> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e4006c830 con 0x7f3e2006afc0 2023-10-08T05:56:37.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.342+0000 7f3e36ffd640 1 -- 172.21.15.70:0/3806928691 <== mgr.6685 v2:172.21.15.181:6832/3857845516 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287236 (secure 0 0 0) 0x7f3e2800d070 con 0x7f3e2006afc0 2023-10-08T05:56:37.344 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:37.347 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:56:37.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3e2006afc0 msgr2=0x7f3e2006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3e2006afc0 0x7f3e2006d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e400c4e80 tx=0x7f3e28009040 comp rx=0 tx=0).stop 2023-10-08T05:56:37.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3806928691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 msgr2=0x7f3e400c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 0x7f3e400c5710 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f3e3002beb0 tx=0x7f3e30004730 comp rx=0 tx=0).stop 2023-10-08T05:56:37.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3806928691 shutdown_connections 2023-10-08T05:56:37.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f3e2006afc0 0x7f3e2006d480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e400b2850 0x7f3e400b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e400c6df0 0x7f3e400c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 --2- 172.21.15.70:0/3806928691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e2c003970 0x7f3e400c5710 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3806928691 >> 172.21.15.70:0/3806928691 conn(0x7f3e400cb0c0 msgr2=0x7f3e400ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:37.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3806928691 shutdown_connections 2023-10-08T05:56:37.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.349+0000 7f3e3f59e640 1 -- 172.21.15.70:0/3806928691 wait complete. 2023-10-08T05:56:37.369 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-08T05:56:36.797763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1149,"num_read_kb":2352,"num_write":413,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2723,"ondisk_log_size":2723,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":352576,"kb_used_data":15688,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754622144,"statfs":{"total":773094113280,"available":772733075456,"internally_reserved":0,"allocated":16064512,"data_stored":8744436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":344837691},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.908100"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713788+0000","last_change":"2023-10-08T05:56:32.447348+0000","last_active":"2023-10-08T05:56:33.713788+0000","last_peered":"2023-10-08T05:56:33.713788+0000","last_clean":"2023-10-08T05:56:33.713788+0000","last_became_active":"2023-10-08T05:56:32.446278+0000","last_became_peered":"2023-10-08T05:56:32.446278+0000","last_unstale":"2023-10-08T05:56:33.713788+0000","last_undegraded":"2023-10-08T05:56:33.713788+0000","last_fullsized":"2023-10-08T05:56:33.713788+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:16.771665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.431107+0000","last_change":"2023-10-08T05:56:32.457690+0000","last_active":"2023-10-08T05:56:33.431107+0000","last_peered":"2023-10-08T05:56:33.431107+0000","last_clean":"2023-10-08T05:56:33.431107+0000","last_became_active":"2023-10-08T05:56:32.457391+0000","last_became_peered":"2023-10-08T05:56:32.457391+0000","last_unstale":"2023-10-08T05:56:33.431107+0000","last_undegraded":"2023-10-08T05:56:33.431107+0000","last_fullsized":"2023-10-08T05:56:33.431107+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:36.163475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.431170+0000","last_change":"2023-10-08T05:56:32.450562+0000","last_active":"2023-10-08T05:56:33.431170+0000","last_peered":"2023-10-08T05:56:33.431170+0000","last_clean":"2023-10-08T05:56:33.431170+0000","last_became_active":"2023-10-08T05:56:32.450240+0000","last_became_peered":"2023-10-08T05:56:32.450240+0000","last_unstale":"2023-10-08T05:56:33.431170+0000","last_undegraded":"2023-10-08T05:56:33.431170+0000","last_fullsized":"2023-10-08T05:56:33.431170+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:11.783751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712182+0000","last_change":"2023-10-08T05:56:32.454874+0000","last_active":"2023-10-08T05:56:33.712182+0000","last_peered":"2023-10-08T05:56:33.712182+0000","last_clean":"2023-10-08T05:56:33.712182+0000","last_became_active":"2023-10-08T05:56:32.454535+0000","last_became_peered":"2023-10-08T05:56:32.454535+0000","last_unstale":"2023-10-08T05:56:33.712182+0000","last_undegraded":"2023-10-08T05:56:33.712182+0000","last_fullsized":"2023-10-08T05:56:33.712182+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:04.579220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713158+0000","last_change":"2023-10-08T05:56:32.451872+0000","last_active":"2023-10-08T05:56:33.713158+0000","last_peered":"2023-10-08T05:56:33.713158+0000","last_clean":"2023-10-08T05:56:33.713158+0000","last_became_active":"2023-10-08T05:56:32.451580+0000","last_became_peered":"2023-10-08T05:56:32.451580+0000","last_unstale":"2023-10-08T05:56:33.713158+0000","last_undegraded":"2023-10-08T05:56:33.713158+0000","last_fullsized":"2023-10-08T05:56:33.713158+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:45:33.173494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712563+0000","last_change":"2023-10-08T05:56:32.443961+0000","last_active":"2023-10-08T05:56:33.712563+0000","last_peered":"2023-10-08T05:56:33.712563+0000","last_clean":"2023-10-08T05:56:33.712563+0000","last_became_active":"2023-10-08T05:56:32.442742+0000","last_became_peered":"2023-10-08T05:56:32.442742+0000","last_unstale":"2023-10-08T05:56:33.712563+0000","last_undegraded":"2023-10-08T05:56:33.712563+0000","last_fullsized":"2023-10-08T05:56:33.712563+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:20.206129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711916+0000","last_change":"2023-10-08T05:56:32.456145+0000","last_active":"2023-10-08T05:56:33.711916+0000","last_peered":"2023-10-08T05:56:33.711916+0000","last_clean":"2023-10-08T05:56:33.711916+0000","last_became_active":"2023-10-08T05:56:32.455861+0000","last_became_peered":"2023-10-08T05:56:32.455861+0000","last_unstale":"2023-10-08T05:56:33.711916+0000","last_undegraded":"2023-10-08T05:56:33.711916+0000","last_fullsized":"2023-10-08T05:56:33.711916+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:32.678293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430299+0000","last_change":"2023-10-08T05:56:32.451302+0000","last_active":"2023-10-08T05:56:33.430299+0000","last_peered":"2023-10-08T05:56:33.430299+0000","last_clean":"2023-10-08T05:56:33.430299+0000","last_became_active":"2023-10-08T05:56:32.451002+0000","last_became_peered":"2023-10-08T05:56:32.451002+0000","last_unstale":"2023-10-08T05:56:33.430299+0000","last_undegraded":"2023-10-08T05:56:33.430299+0000","last_fullsized":"2023-10-08T05:56:33.430299+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:35:27.034365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713202+0000","last_change":"2023-10-08T05:56:32.454949+0000","last_active":"2023-10-08T05:56:33.713202+0000","last_peered":"2023-10-08T05:56:33.713202+0000","last_clean":"2023-10-08T05:56:33.713202+0000","last_became_active":"2023-10-08T05:56:32.454657+0000","last_became_peered":"2023-10-08T05:56:32.454657+0000","last_unstale":"2023-10-08T05:56:33.713202+0000","last_undegraded":"2023-10-08T05:56:33.713202+0000","last_fullsized":"2023-10-08T05:56:33.713202+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:57:17.481858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556303+0000","last_change":"2023-10-08T05:56:32.444306+0000","last_active":"2023-10-08T05:56:33.556303+0000","last_peered":"2023-10-08T05:56:33.556303+0000","last_clean":"2023-10-08T05:56:33.556303+0000","last_became_active":"2023-10-08T05:56:32.444031+0000","last_became_peered":"2023-10-08T05:56:32.444031+0000","last_unstale":"2023-10-08T05:56:33.556303+0000","last_undegraded":"2023-10-08T05:56:33.556303+0000","last_fullsized":"2023-10-08T05:56:33.556303+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:19.099231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713800+0000","last_change":"2023-10-08T05:56:32.449336+0000","last_active":"2023-10-08T05:56:33.713800+0000","last_peered":"2023-10-08T05:56:33.713800+0000","last_clean":"2023-10-08T05:56:33.713800+0000","last_became_active":"2023-10-08T05:56:32.449046+0000","last_became_peered":"2023-10-08T05:56:32.449046+0000","last_unstale":"2023-10-08T05:56:33.713800+0000","last_undegraded":"2023-10-08T05:56:33.713800+0000","last_fullsized":"2023-10-08T05:56:33.713800+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:40.177643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712916+0000","last_change":"2023-10-08T05:56:32.453391+0000","last_active":"2023-10-08T05:56:33.712916+0000","last_peered":"2023-10-08T05:56:33.712916+0000","last_clean":"2023-10-08T05:56:33.712916+0000","last_became_active":"2023-10-08T05:56:32.453124+0000","last_became_peered":"2023-10-08T05:56:32.453124+0000","last_unstale":"2023-10-08T05:56:33.712916+0000","last_undegraded":"2023-10-08T05:56:33.712916+0000","last_fullsized":"2023-10-08T05:56:33.712916+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:52.048323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556039+0000","last_change":"2023-10-08T05:56:32.451548+0000","last_active":"2023-10-08T05:56:33.556039+0000","last_peered":"2023-10-08T05:56:33.556039+0000","last_clean":"2023-10-08T05:56:33.556039+0000","last_became_active":"2023-10-08T05:56:32.451108+0000","last_became_peered":"2023-10-08T05:56:32.451108+0000","last_unstale":"2023-10-08T05:56:33.556039+0000","last_undegraded":"2023-10-08T05:56:33.556039+0000","last_fullsized":"2023-10-08T05:56:33.556039+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:54.769675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430367+0000","last_change":"2023-10-08T05:56:32.449688+0000","last_active":"2023-10-08T05:56:33.430367+0000","last_peered":"2023-10-08T05:56:33.430367+0000","last_clean":"2023-10-08T05:56:33.430367+0000","last_became_active":"2023-10-08T05:56:32.449429+0000","last_became_peered":"2023-10-08T05:56:32.449429+0000","last_unstale":"2023-10-08T05:56:33.430367+0000","last_undegraded":"2023-10-08T05:56:33.430367+0000","last_fullsized":"2023-10-08T05:56:33.430367+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:57:17.481858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711938+0000","last_change":"2023-10-08T05:56:32.445147+0000","last_active":"2023-10-08T05:56:33.711938+0000","last_peered":"2023-10-08T05:56:33.711938+0000","last_clean":"2023-10-08T05:56:33.711938+0000","last_became_active":"2023-10-08T05:56:32.444764+0000","last_became_peered":"2023-10-08T05:56:32.444764+0000","last_unstale":"2023-10-08T05:56:33.711938+0000","last_undegraded":"2023-10-08T05:56:33.711938+0000","last_fullsized":"2023-10-08T05:56:33.711938+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:02.069891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711890+0000","last_change":"2023-10-08T05:56:32.441172+0000","last_active":"2023-10-08T05:56:33.711890+0000","last_peered":"2023-10-08T05:56:33.711890+0000","last_clean":"2023-10-08T05:56:33.711890+0000","last_became_active":"2023-10-08T05:56:32.440481+0000","last_became_peered":"2023-10-08T05:56:32.440481+0000","last_unstale":"2023-10-08T05:56:33.711890+0000","last_undegraded":"2023-10-08T05:56:33.711890+0000","last_fullsized":"2023-10-08T05:56:33.711890+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:16.824353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556383+0000","last_change":"2023-10-08T05:56:32.452293+0000","last_active":"2023-10-08T05:56:33.556383+0000","last_peered":"2023-10-08T05:56:33.556383+0000","last_clean":"2023-10-08T05:56:33.556383+0000","last_became_active":"2023-10-08T05:56:32.451996+0000","last_became_peered":"2023-10-08T05:56:32.451996+0000","last_unstale":"2023-10-08T05:56:33.556383+0000","last_undegraded":"2023-10-08T05:56:33.556383+0000","last_fullsized":"2023-10-08T05:56:33.556383+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:32.454202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.431436+0000","last_change":"2023-10-08T05:56:32.457330+0000","last_active":"2023-10-08T05:56:33.431436+0000","last_peered":"2023-10-08T05:56:33.431436+0000","last_clean":"2023-10-08T05:56:33.431436+0000","last_became_active":"2023-10-08T05:56:32.457074+0000","last_became_peered":"2023-10-08T05:56:32.457074+0000","last_unstale":"2023-10-08T05:56:33.431436+0000","last_undegraded":"2023-10-08T05:56:33.431436+0000","last_fullsized":"2023-10-08T05:56:33.431436+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:16.824353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.432944+0000","last_change":"2023-10-08T05:56:32.459958+0000","last_active":"2023-10-08T05:56:33.432944+0000","last_peered":"2023-10-08T05:56:33.432944+0000","last_clean":"2023-10-08T05:56:33.432944+0000","last_became_active":"2023-10-08T05:56:32.459617+0000","last_became_peered":"2023-10-08T05:56:32.459617+0000","last_unstale":"2023-10-08T05:56:33.432944+0000","last_undegraded":"2023-10-08T05:56:33.432944+0000","last_fullsized":"2023-10-08T05:56:33.432944+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:26.776339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711696+0000","last_change":"2023-10-08T05:56:32.437161+0000","last_active":"2023-10-08T05:56:33.711696+0000","last_peered":"2023-10-08T05:56:33.711696+0000","last_clean":"2023-10-08T05:56:33.711696+0000","last_became_active":"2023-10-08T05:56:32.436827+0000","last_became_peered":"2023-10-08T05:56:32.436827+0000","last_unstale":"2023-10-08T05:56:33.711696+0000","last_undegraded":"2023-10-08T05:56:33.711696+0000","last_fullsized":"2023-10-08T05:56:33.711696+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:22:17.819587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712525+0000","last_change":"2023-10-08T05:56:32.461871+0000","last_active":"2023-10-08T05:56:33.712525+0000","last_peered":"2023-10-08T05:56:33.712525+0000","last_clean":"2023-10-08T05:56:33.712525+0000","last_became_active":"2023-10-08T05:56:32.461515+0000","last_became_peered":"2023-10-08T05:56:32.461515+0000","last_unstale":"2023-10-08T05:56:33.712525+0000","last_undegraded":"2023-10-08T05:56:33.712525+0000","last_fullsized":"2023-10-08T05:56:33.712525+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:13.134824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713206+0000","last_change":"2023-10-08T05:56:32.450561+0000","last_active":"2023-10-08T05:56:33.713206+0000","last_peered":"2023-10-08T05:56:33.713206+0000","last_clean":"2023-10-08T05:56:33.713206+0000","last_became_active":"2023-10-08T05:56:32.450024+0000","last_became_peered":"2023-10-08T05:56:32.450024+0000","last_unstale":"2023-10-08T05:56:33.713206+0000","last_undegraded":"2023-10-08T05:56:33.713206+0000","last_fullsized":"2023-10-08T05:56:33.713206+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:23:09.866243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712887+0000","last_change":"2023-10-08T05:56:32.453905+0000","last_active":"2023-10-08T05:56:33.712887+0000","last_peered":"2023-10-08T05:56:33.712887+0000","last_clean":"2023-10-08T05:56:33.712887+0000","last_became_active":"2023-10-08T05:56:32.453612+0000","last_became_peered":"2023-10-08T05:56:32.453612+0000","last_unstale":"2023-10-08T05:56:33.712887+0000","last_undegraded":"2023-10-08T05:56:33.712887+0000","last_fullsized":"2023-10-08T05:56:33.712887+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:03.088990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433225+0000","last_change":"2023-10-08T05:56:32.449674+0000","last_active":"2023-10-08T05:56:33.433225+0000","last_peered":"2023-10-08T05:56:33.433225+0000","last_clean":"2023-10-08T05:56:33.433225+0000","last_became_active":"2023-10-08T05:56:32.449340+0000","last_became_peered":"2023-10-08T05:56:32.449340+0000","last_unstale":"2023-10-08T05:56:33.433225+0000","last_undegraded":"2023-10-08T05:56:33.433225+0000","last_fullsized":"2023-10-08T05:56:33.433225+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:15.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.435023+0000","last_change":"2023-10-08T05:56:32.448292+0000","last_active":"2023-10-08T05:56:33.435023+0000","last_peered":"2023-10-08T05:56:33.435023+0000","last_clean":"2023-10-08T05:56:33.435023+0000","last_became_active":"2023-10-08T05:56:32.447958+0000","last_became_peered":"2023-10-08T05:56:32.447958+0000","last_unstale":"2023-10-08T05:56:33.435023+0000","last_undegraded":"2023-10-08T05:56:33.435023+0000","last_fullsized":"2023-10-08T05:56:33.435023+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:36.163475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711398+0000","last_change":"2023-10-08T05:56:32.445105+0000","last_active":"2023-10-08T05:56:33.711398+0000","last_peered":"2023-10-08T05:56:33.711398+0000","last_clean":"2023-10-08T05:56:33.711398+0000","last_became_active":"2023-10-08T05:56:32.444833+0000","last_became_peered":"2023-10-08T05:56:32.444833+0000","last_unstale":"2023-10-08T05:56:33.711398+0000","last_undegraded":"2023-10-08T05:56:33.711398+0000","last_fullsized":"2023-10-08T05:56:33.711398+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:27.710441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433316+0000","last_change":"2023-10-08T05:56:32.458454+0000","last_active":"2023-10-08T05:56:33.433316+0000","last_peered":"2023-10-08T05:56:33.433316+0000","last_clean":"2023-10-08T05:56:33.433316+0000","last_became_active":"2023-10-08T05:56:32.456358+0000","last_became_peered":"2023-10-08T05:56:32.456358+0000","last_unstale":"2023-10-08T05:56:33.433316+0000","last_undegraded":"2023-10-08T05:56:33.433316+0000","last_fullsized":"2023-10-08T05:56:33.433316+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:43:57.341349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712864+0000","last_change":"2023-10-08T05:56:32.450843+0000","last_active":"2023-10-08T05:56:33.712864+0000","last_peered":"2023-10-08T05:56:33.712864+0000","last_clean":"2023-10-08T05:56:33.712864+0000","last_became_active":"2023-10-08T05:56:32.450577+0000","last_became_peered":"2023-10-08T05:56:32.450577+0000","last_unstale":"2023-10-08T05:56:33.712864+0000","last_undegraded":"2023-10-08T05:56:33.712864+0000","last_fullsized":"2023-10-08T05:56:33.712864+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:57.980273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711928+0000","last_change":"2023-10-08T05:56:32.436021+0000","last_active":"2023-10-08T05:56:33.711928+0000","last_peered":"2023-10-08T05:56:33.711928+0000","last_clean":"2023-10-08T05:56:33.711928+0000","last_became_active":"2023-10-08T05:56:32.435639+0000","last_became_peered":"2023-10-08T05:56:32.435639+0000","last_unstale":"2023-10-08T05:56:33.711928+0000","last_undegraded":"2023-10-08T05:56:33.711928+0000","last_fullsized":"2023-10-08T05:56:33.711928+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:12.399582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430651+0000","last_change":"2023-10-08T05:56:32.447378+0000","last_active":"2023-10-08T05:56:33.430651+0000","last_peered":"2023-10-08T05:56:33.430651+0000","last_clean":"2023-10-08T05:56:33.430651+0000","last_became_active":"2023-10-08T05:56:32.447126+0000","last_became_peered":"2023-10-08T05:56:32.447126+0000","last_unstale":"2023-10-08T05:56:33.430651+0000","last_undegraded":"2023-10-08T05:56:33.430651+0000","last_fullsized":"2023-10-08T05:56:33.430651+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:48.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433862+0000","last_change":"2023-10-08T05:56:32.455410+0000","last_active":"2023-10-08T05:56:33.433862+0000","last_peered":"2023-10-08T05:56:33.433862+0000","last_clean":"2023-10-08T05:56:33.433862+0000","last_became_active":"2023-10-08T05:56:32.455124+0000","last_became_peered":"2023-10-08T05:56:32.455124+0000","last_unstale":"2023-10-08T05:56:33.433862+0000","last_undegraded":"2023-10-08T05:56:33.433862+0000","last_fullsized":"2023-10-08T05:56:33.433862+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:57:17.481858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433499+0000","last_change":"2023-10-08T05:56:32.449365+0000","last_active":"2023-10-08T05:56:33.433499+0000","last_peered":"2023-10-08T05:56:33.433499+0000","last_clean":"2023-10-08T05:56:33.433499+0000","last_became_active":"2023-10-08T05:56:32.448986+0000","last_became_peered":"2023-10-08T05:56:32.448986+0000","last_unstale":"2023-10-08T05:56:33.433499+0000","last_undegraded":"2023-10-08T05:56:33.433499+0000","last_fullsized":"2023-10-08T05:56:33.433499+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:35:27.034365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556621+0000","last_change":"2023-10-08T05:56:32.446842+0000","last_active":"2023-10-08T05:56:33.556621+0000","last_peered":"2023-10-08T05:56:33.556621+0000","last_clean":"2023-10-08T05:56:33.556621+0000","last_became_active":"2023-10-08T05:56:32.446553+0000","last_became_peered":"2023-10-08T05:56:32.446553+0000","last_unstale":"2023-10-08T05:56:33.556621+0000","last_undegraded":"2023-10-08T05:56:33.556621+0000","last_fullsized":"2023-10-08T05:56:33.556621+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:57:57.911763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"148'4","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433585+0000","last_change":"2023-10-08T05:56:31.928279+0000","last_active":"2023-10-08T05:56:33.433585+0000","last_peered":"2023-10-08T05:56:33.433585+0000","last_clean":"2023-10-08T05:56:33.433585+0000","last_became_active":"2023-10-08T05:56:31.927975+0000","last_became_peered":"2023-10-08T05:56:31.927975+0000","last_unstale":"2023-10-08T05:56:33.433585+0000","last_undegraded":"2023-10-08T05:56:33.433585+0000","last_fullsized":"2023-10-08T05:56:33.433585+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:32:07.072724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712381+0000","last_change":"2023-10-08T05:56:32.442579+0000","last_active":"2023-10-08T05:56:33.712381+0000","last_peered":"2023-10-08T05:56:33.712381+0000","last_clean":"2023-10-08T05:56:33.712381+0000","last_became_active":"2023-10-08T05:56:32.442277+0000","last_became_peered":"2023-10-08T05:56:32.442277+0000","last_unstale":"2023-10-08T05:56:33.712381+0000","last_undegraded":"2023-10-08T05:56:33.712381+0000","last_fullsized":"2023-10-08T05:56:33.712381+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:04.987920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"148'1","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556528+0000","last_change":"2023-10-08T05:56:31.925313+0000","last_active":"2023-10-08T05:56:33.556528+0000","last_peered":"2023-10-08T05:56:33.556528+0000","last_clean":"2023-10-08T05:56:33.556528+0000","last_became_active":"2023-10-08T05:56:31.925025+0000","last_became_peered":"2023-10-08T05:56:31.925025+0000","last_unstale":"2023-10-08T05:56:33.556528+0000","last_undegraded":"2023-10-08T05:56:33.556528+0000","last_fullsized":"2023-10-08T05:56:33.556528+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:45:32.156988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434868+0000","last_change":"2023-10-08T05:56:32.453328+0000","last_active":"2023-10-08T05:56:33.434868+0000","last_peered":"2023-10-08T05:56:33.434868+0000","last_clean":"2023-10-08T05:56:33.434868+0000","last_became_active":"2023-10-08T05:56:32.453020+0000","last_became_peered":"2023-10-08T05:56:32.453020+0000","last_unstale":"2023-10-08T05:56:33.434868+0000","last_undegraded":"2023-10-08T05:56:33.434868+0000","last_fullsized":"2023-10-08T05:56:33.434868+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:48.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712559+0000","last_change":"2023-10-08T05:56:31.932314+0000","last_active":"2023-10-08T05:56:33.712559+0000","last_peered":"2023-10-08T05:56:33.712559+0000","last_clean":"2023-10-08T05:56:33.712559+0000","last_became_active":"2023-10-08T05:56:31.932025+0000","last_became_peered":"2023-10-08T05:56:31.932025+0000","last_unstale":"2023-10-08T05:56:33.712559+0000","last_undegraded":"2023-10-08T05:56:33.712559+0000","last_fullsized":"2023-10-08T05:56:33.712559+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:55.552305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713330+0000","last_change":"2023-10-08T05:56:32.458979+0000","last_active":"2023-10-08T05:56:33.713330+0000","last_peered":"2023-10-08T05:56:33.713330+0000","last_clean":"2023-10-08T05:56:33.713330+0000","last_became_active":"2023-10-08T05:56:32.458714+0000","last_became_peered":"2023-10-08T05:56:32.458714+0000","last_unstale":"2023-10-08T05:56:33.713330+0000","last_undegraded":"2023-10-08T05:56:33.713330+0000","last_fullsized":"2023-10-08T05:56:33.713330+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:17:44.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556583+0000","last_change":"2023-10-08T05:56:31.925179+0000","last_active":"2023-10-08T05:56:33.556583+0000","last_peered":"2023-10-08T05:56:33.556583+0000","last_clean":"2023-10-08T05:56:33.556583+0000","last_became_active":"2023-10-08T05:56:31.924703+0000","last_became_peered":"2023-10-08T05:56:31.924703+0000","last_unstale":"2023-10-08T05:56:33.556583+0000","last_undegraded":"2023-10-08T05:56:33.556583+0000","last_fullsized":"2023-10-08T05:56:33.556583+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:34.435368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556265+0000","last_change":"2023-10-08T05:56:32.452605+0000","last_active":"2023-10-08T05:56:33.556265+0000","last_peered":"2023-10-08T05:56:33.556265+0000","last_clean":"2023-10-08T05:56:33.556265+0000","last_became_active":"2023-10-08T05:56:32.452337+0000","last_became_peered":"2023-10-08T05:56:32.452337+0000","last_unstale":"2023-10-08T05:56:33.556265+0000","last_undegraded":"2023-10-08T05:56:33.556265+0000","last_fullsized":"2023-10-08T05:56:33.556265+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:49:42.893750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"148'1","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712589+0000","last_change":"2023-10-08T05:56:31.931099+0000","last_active":"2023-10-08T05:56:33.712589+0000","last_peered":"2023-10-08T05:56:33.712589+0000","last_clean":"2023-10-08T05:56:33.712589+0000","last_became_active":"2023-10-08T05:56:31.930830+0000","last_became_peered":"2023-10-08T05:56:31.930830+0000","last_unstale":"2023-10-08T05:56:33.712589+0000","last_undegraded":"2023-10-08T05:56:33.712589+0000","last_fullsized":"2023-10-08T05:56:33.712589+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:17.810142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713299+0000","last_change":"2023-10-08T05:56:32.446454+0000","last_active":"2023-10-08T05:56:33.713299+0000","last_peered":"2023-10-08T05:56:33.713299+0000","last_clean":"2023-10-08T05:56:33.713299+0000","last_became_active":"2023-10-08T05:56:32.446133+0000","last_became_peered":"2023-10-08T05:56:32.446133+0000","last_unstale":"2023-10-08T05:56:33.713299+0000","last_undegraded":"2023-10-08T05:56:33.713299+0000","last_fullsized":"2023-10-08T05:56:33.713299+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:35.451874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"148'1","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712444+0000","last_change":"2023-10-08T05:56:31.929046+0000","last_active":"2023-10-08T05:56:33.712444+0000","last_peered":"2023-10-08T05:56:33.712444+0000","last_clean":"2023-10-08T05:56:33.712444+0000","last_became_active":"2023-10-08T05:56:31.927539+0000","last_became_peered":"2023-10-08T05:56:31.927539+0000","last_unstale":"2023-10-08T05:56:33.712444+0000","last_undegraded":"2023-10-08T05:56:33.712444+0000","last_fullsized":"2023-10-08T05:56:33.712444+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:02:13.137236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430512+0000","last_change":"2023-10-08T05:56:32.446226+0000","last_active":"2023-10-08T05:56:33.430512+0000","last_peered":"2023-10-08T05:56:33.430512+0000","last_clean":"2023-10-08T05:56:33.430512+0000","last_became_active":"2023-10-08T05:56:32.445940+0000","last_became_peered":"2023-10-08T05:56:32.445940+0000","last_unstale":"2023-10-08T05:56:33.430512+0000","last_undegraded":"2023-10-08T05:56:33.430512+0000","last_fullsized":"2023-10-08T05:56:33.430512+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:17:44.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433852+0000","last_change":"2023-10-08T05:56:31.930229+0000","last_active":"2023-10-08T05:56:33.433852+0000","last_peered":"2023-10-08T05:56:33.433852+0000","last_clean":"2023-10-08T05:56:33.433852+0000","last_became_active":"2023-10-08T05:56:31.929712+0000","last_became_peered":"2023-10-08T05:56:31.929712+0000","last_unstale":"2023-10-08T05:56:33.433852+0000","last_undegraded":"2023-10-08T05:56:33.433852+0000","last_fullsized":"2023-10-08T05:56:33.433852+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:03.892658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556408+0000","last_change":"2023-10-08T05:56:32.451357+0000","last_active":"2023-10-08T05:56:33.556408+0000","last_peered":"2023-10-08T05:56:33.556408+0000","last_clean":"2023-10-08T05:56:33.556408+0000","last_became_active":"2023-10-08T05:56:32.450738+0000","last_became_peered":"2023-10-08T05:56:32.450738+0000","last_unstale":"2023-10-08T05:56:33.556408+0000","last_undegraded":"2023-10-08T05:56:33.556408+0000","last_fullsized":"2023-10-08T05:56:33.556408+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:13.991802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712142+0000","last_change":"2023-10-08T05:56:31.930275+0000","last_active":"2023-10-08T05:56:33.712142+0000","last_peered":"2023-10-08T05:56:33.712142+0000","last_clean":"2023-10-08T05:56:33.712142+0000","last_became_active":"2023-10-08T05:56:31.929949+0000","last_became_peered":"2023-10-08T05:56:31.929949+0000","last_unstale":"2023-10-08T05:56:33.712142+0000","last_undegraded":"2023-10-08T05:56:33.712142+0000","last_fullsized":"2023-10-08T05:56:33.712142+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:17.131473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433517+0000","last_change":"2023-10-08T05:56:32.454749+0000","last_active":"2023-10-08T05:56:33.433517+0000","last_peered":"2023-10-08T05:56:33.433517+0000","last_clean":"2023-10-08T05:56:33.433517+0000","last_became_active":"2023-10-08T05:56:32.454483+0000","last_became_peered":"2023-10-08T05:56:32.454483+0000","last_unstale":"2023-10-08T05:56:33.433517+0000","last_undegraded":"2023-10-08T05:56:33.433517+0000","last_fullsized":"2023-10-08T05:56:33.433517+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:54.638366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433819+0000","last_change":"2023-10-08T05:56:31.926933+0000","last_active":"2023-10-08T05:56:33.433819+0000","last_peered":"2023-10-08T05:56:33.433819+0000","last_clean":"2023-10-08T05:56:33.433819+0000","last_became_active":"2023-10-08T05:56:31.926638+0000","last_became_peered":"2023-10-08T05:56:31.926638+0000","last_unstale":"2023-10-08T05:56:33.433819+0000","last_undegraded":"2023-10-08T05:56:33.433819+0000","last_fullsized":"2023-10-08T05:56:33.433819+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:29.501903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434916+0000","last_change":"2023-10-08T05:56:32.446591+0000","last_active":"2023-10-08T05:56:33.434916+0000","last_peered":"2023-10-08T05:56:33.434916+0000","last_clean":"2023-10-08T05:56:33.434916+0000","last_became_active":"2023-10-08T05:56:32.446177+0000","last_became_peered":"2023-10-08T05:56:32.446177+0000","last_unstale":"2023-10-08T05:56:33.434916+0000","last_undegraded":"2023-10-08T05:56:33.434916+0000","last_fullsized":"2023-10-08T05:56:33.434916+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:18.147979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434654+0000","last_change":"2023-10-08T05:56:31.924766+0000","last_active":"2023-10-08T05:56:33.434654+0000","last_peered":"2023-10-08T05:56:33.434654+0000","last_clean":"2023-10-08T05:56:33.434654+0000","last_became_active":"2023-10-08T05:56:31.924139+0000","last_became_peered":"2023-10-08T05:56:31.924139+0000","last_unstale":"2023-10-08T05:56:33.434654+0000","last_undegraded":"2023-10-08T05:56:33.434654+0000","last_fullsized":"2023-10-08T05:56:33.434654+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:11.170516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.431554+0000","last_change":"2023-10-08T05:56:32.450421+0000","last_active":"2023-10-08T05:56:33.431554+0000","last_peered":"2023-10-08T05:56:33.431554+0000","last_clean":"2023-10-08T05:56:33.431554+0000","last_became_active":"2023-10-08T05:56:32.450091+0000","last_became_peered":"2023-10-08T05:56:32.450091+0000","last_unstale":"2023-10-08T05:56:33.431554+0000","last_undegraded":"2023-10-08T05:56:33.431554+0000","last_fullsized":"2023-10-08T05:56:33.431554+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:32.678293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"148'3","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.435212+0000","last_change":"2023-10-08T05:56:31.934321+0000","last_active":"2023-10-08T05:56:33.435212+0000","last_peered":"2023-10-08T05:56:33.435212+0000","last_clean":"2023-10-08T05:56:33.435212+0000","last_became_active":"2023-10-08T05:56:31.933973+0000","last_became_peered":"2023-10-08T05:56:31.933973+0000","last_unstale":"2023-10-08T05:56:33.435212+0000","last_undegraded":"2023-10-08T05:56:33.435212+0000","last_fullsized":"2023-10-08T05:56:33.435212+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:03.562715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.435148+0000","last_change":"2023-10-08T05:56:32.452863+0000","last_active":"2023-10-08T05:56:33.435148+0000","last_peered":"2023-10-08T05:56:33.435148+0000","last_clean":"2023-10-08T05:56:33.435148+0000","last_became_active":"2023-10-08T05:56:32.452570+0000","last_became_peered":"2023-10-08T05:56:32.452570+0000","last_unstale":"2023-10-08T05:56:33.435148+0000","last_undegraded":"2023-10-08T05:56:33.435148+0000","last_fullsized":"2023-10-08T05:56:33.435148+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:11.783751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"148'6","reported_seq":18,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430539+0000","last_change":"2023-10-08T05:56:31.934739+0000","last_active":"2023-10-08T05:56:33.430539+0000","last_peered":"2023-10-08T05:56:33.430539+0000","last_clean":"2023-10-08T05:56:33.430539+0000","last_became_active":"2023-10-08T05:56:31.934463+0000","last_became_peered":"2023-10-08T05:56:31.934463+0000","last_unstale":"2023-10-08T05:56:33.430539+0000","last_undegraded":"2023-10-08T05:56:33.430539+0000","last_fullsized":"2023-10-08T05:56:33.430539+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:45.159920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.432791+0000","last_change":"2023-10-08T05:56:32.446763+0000","last_active":"2023-10-08T05:56:33.432791+0000","last_peered":"2023-10-08T05:56:33.432791+0000","last_clean":"2023-10-08T05:56:33.432791+0000","last_became_active":"2023-10-08T05:56:32.446305+0000","last_became_peered":"2023-10-08T05:56:32.446305+0000","last_unstale":"2023-10-08T05:56:33.432791+0000","last_undegraded":"2023-10-08T05:56:33.432791+0000","last_fullsized":"2023-10-08T05:56:33.432791+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:28.275980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430667+0000","last_change":"2023-10-08T05:56:31.930410+0000","last_active":"2023-10-08T05:56:33.430667+0000","last_peered":"2023-10-08T05:56:33.430667+0000","last_clean":"2023-10-08T05:56:33.430667+0000","last_became_active":"2023-10-08T05:56:31.930156+0000","last_became_peered":"2023-10-08T05:56:31.930156+0000","last_unstale":"2023-10-08T05:56:33.430667+0000","last_undegraded":"2023-10-08T05:56:33.430667+0000","last_fullsized":"2023-10-08T05:56:33.430667+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:02.072485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556187+0000","last_change":"2023-10-08T05:56:32.455106+0000","last_active":"2023-10-08T05:56:33.556187+0000","last_peered":"2023-10-08T05:56:33.556187+0000","last_clean":"2023-10-08T05:56:33.556187+0000","last_became_active":"2023-10-08T05:56:32.454813+0000","last_became_peered":"2023-10-08T05:56:32.454813+0000","last_unstale":"2023-10-08T05:56:33.556187+0000","last_undegraded":"2023-10-08T05:56:33.556187+0000","last_fullsized":"2023-10-08T05:56:33.556187+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:51.004877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430871+0000","last_change":"2023-10-08T05:56:31.935623+0000","last_active":"2023-10-08T05:56:33.430871+0000","last_peered":"2023-10-08T05:56:33.430871+0000","last_clean":"2023-10-08T05:56:33.430871+0000","last_became_active":"2023-10-08T05:56:31.935353+0000","last_became_peered":"2023-10-08T05:56:31.935353+0000","last_unstale":"2023-10-08T05:56:33.430871+0000","last_undegraded":"2023-10-08T05:56:33.430871+0000","last_fullsized":"2023-10-08T05:56:33.430871+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:03.562715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711259+0000","last_change":"2023-10-08T05:56:32.437421+0000","last_active":"2023-10-08T05:56:33.711259+0000","last_peered":"2023-10-08T05:56:33.711259+0000","last_clean":"2023-10-08T05:56:33.711259+0000","last_became_active":"2023-10-08T05:56:32.437105+0000","last_became_peered":"2023-10-08T05:56:32.437105+0000","last_unstale":"2023-10-08T05:56:33.711259+0000","last_undegraded":"2023-10-08T05:56:33.711259+0000","last_fullsized":"2023-10-08T05:56:33.711259+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:13.807001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430597+0000","last_change":"2023-10-08T05:56:31.933863+0000","last_active":"2023-10-08T05:56:33.430597+0000","last_peered":"2023-10-08T05:56:33.430597+0000","last_clean":"2023-10-08T05:56:33.430597+0000","last_became_active":"2023-10-08T05:56:31.933568+0000","last_became_peered":"2023-10-08T05:56:31.933568+0000","last_unstale":"2023-10-08T05:56:33.430597+0000","last_undegraded":"2023-10-08T05:56:33.430597+0000","last_fullsized":"2023-10-08T05:56:33.430597+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:03.892658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434579+0000","last_change":"2023-10-08T05:56:32.452597+0000","last_active":"2023-10-08T05:56:33.434579+0000","last_peered":"2023-10-08T05:56:33.434579+0000","last_clean":"2023-10-08T05:56:33.434579+0000","last_became_active":"2023-10-08T05:56:32.452250+0000","last_became_peered":"2023-10-08T05:56:32.452250+0000","last_unstale":"2023-10-08T05:56:33.434579+0000","last_undegraded":"2023-10-08T05:56:33.434579+0000","last_fullsized":"2023-10-08T05:56:33.434579+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:02:14.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"148'1","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556596+0000","last_change":"2023-10-08T05:56:31.926357+0000","last_active":"2023-10-08T05:56:33.556596+0000","last_peered":"2023-10-08T05:56:33.556596+0000","last_clean":"2023-10-08T05:56:33.556596+0000","last_became_active":"2023-10-08T05:56:31.926101+0000","last_became_peered":"2023-10-08T05:56:31.926101+0000","last_unstale":"2023-10-08T05:56:33.556596+0000","last_undegraded":"2023-10-08T05:56:33.556596+0000","last_fullsized":"2023-10-08T05:56:33.556596+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:23:08.849738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713874+0000","last_change":"2023-10-08T05:56:32.448049+0000","last_active":"2023-10-08T05:56:33.713874+0000","last_peered":"2023-10-08T05:56:33.713874+0000","last_clean":"2023-10-08T05:56:33.713874+0000","last_became_active":"2023-10-08T05:56:32.447747+0000","last_became_peered":"2023-10-08T05:56:32.447747+0000","last_unstale":"2023-10-08T05:56:33.713874+0000","last_undegraded":"2023-10-08T05:56:33.713874+0000","last_fullsized":"2023-10-08T05:56:33.713874+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:54.769675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711508+0000","last_change":"2023-10-08T05:56:31.918523+0000","last_active":"2023-10-08T05:56:33.711508+0000","last_peered":"2023-10-08T05:56:33.711508+0000","last_clean":"2023-10-08T05:56:33.711508+0000","last_became_active":"2023-10-08T05:56:31.918035+0000","last_became_peered":"2023-10-08T05:56:31.918035+0000","last_unstale":"2023-10-08T05:56:33.711508+0000","last_undegraded":"2023-10-08T05:56:33.711508+0000","last_fullsized":"2023-10-08T05:56:33.711508+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:54.285931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712758+0000","last_change":"2023-10-08T05:56:32.449279+0000","last_active":"2023-10-08T05:56:33.712758+0000","last_peered":"2023-10-08T05:56:33.712758+0000","last_clean":"2023-10-08T05:56:33.712758+0000","last_became_active":"2023-10-08T05:56:32.448971+0000","last_became_peered":"2023-10-08T05:56:32.448971+0000","last_unstale":"2023-10-08T05:56:33.712758+0000","last_undegraded":"2023-10-08T05:56:33.712758+0000","last_fullsized":"2023-10-08T05:56:33.712758+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:46:03.339371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"148'1","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712753+0000","last_change":"2023-10-08T05:56:31.935961+0000","last_active":"2023-10-08T05:56:33.712753+0000","last_peered":"2023-10-08T05:56:33.712753+0000","last_clean":"2023-10-08T05:56:33.712753+0000","last_became_active":"2023-10-08T05:56:31.935636+0000","last_became_peered":"2023-10-08T05:56:31.935636+0000","last_unstale":"2023-10-08T05:56:33.712753+0000","last_undegraded":"2023-10-08T05:56:33.712753+0000","last_fullsized":"2023-10-08T05:56:33.712753+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:03.015840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.435492+0000","last_change":"2023-10-08T05:56:32.462843+0000","last_active":"2023-10-08T05:56:33.435492+0000","last_peered":"2023-10-08T05:56:33.435492+0000","last_clean":"2023-10-08T05:56:33.435492+0000","last_became_active":"2023-10-08T05:56:32.462424+0000","last_became_peered":"2023-10-08T05:56:32.462424+0000","last_unstale":"2023-10-08T05:56:33.435492+0000","last_undegraded":"2023-10-08T05:56:33.435492+0000","last_fullsized":"2023-10-08T05:56:33.435492+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:16.824353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712417+0000","last_change":"2023-10-08T05:56:31.934213+0000","last_active":"2023-10-08T05:56:33.712417+0000","last_peered":"2023-10-08T05:56:33.712417+0000","last_clean":"2023-10-08T05:56:33.712417+0000","last_became_active":"2023-10-08T05:56:31.933893+0000","last_became_peered":"2023-10-08T05:56:31.933893+0000","last_unstale":"2023-10-08T05:56:33.712417+0000","last_undegraded":"2023-10-08T05:56:33.712417+0000","last_fullsized":"2023-10-08T05:56:33.712417+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:30.797452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712253+0000","last_change":"2023-10-08T05:56:32.448139+0000","last_active":"2023-10-08T05:56:33.712253+0000","last_peered":"2023-10-08T05:56:33.712253+0000","last_clean":"2023-10-08T05:56:33.712253+0000","last_became_active":"2023-10-08T05:56:32.447863+0000","last_became_peered":"2023-10-08T05:56:32.447863+0000","last_unstale":"2023-10-08T05:56:33.712253+0000","last_undegraded":"2023-10-08T05:56:33.712253+0000","last_fullsized":"2023-10-08T05:56:33.712253+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:10.698762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711546+0000","last_change":"2023-10-08T05:56:31.927544+0000","last_active":"2023-10-08T05:56:33.711546+0000","last_peered":"2023-10-08T05:56:33.711546+0000","last_clean":"2023-10-08T05:56:33.711546+0000","last_became_active":"2023-10-08T05:56:31.926241+0000","last_became_peered":"2023-10-08T05:56:31.926241+0000","last_unstale":"2023-10-08T05:56:33.711546+0000","last_undegraded":"2023-10-08T05:56:33.711546+0000","last_fullsized":"2023-10-08T05:56:33.711546+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:35.146970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430299+0000","last_change":"2023-10-08T05:56:32.446323+0000","last_active":"2023-10-08T05:56:33.430299+0000","last_peered":"2023-10-08T05:56:33.430299+0000","last_clean":"2023-10-08T05:56:33.430299+0000","last_became_active":"2023-10-08T05:56:32.445929+0000","last_became_peered":"2023-10-08T05:56:32.445929+0000","last_unstale":"2023-10-08T05:56:33.430299+0000","last_undegraded":"2023-10-08T05:56:33.430299+0000","last_fullsized":"2023-10-08T05:56:33.430299+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:15.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712074+0000","last_change":"2023-10-08T05:56:31.934037+0000","last_active":"2023-10-08T05:56:33.712074+0000","last_peered":"2023-10-08T05:56:33.712074+0000","last_clean":"2023-10-08T05:56:33.712074+0000","last_became_active":"2023-10-08T05:56:31.933728+0000","last_became_peered":"2023-10-08T05:56:31.933728+0000","last_unstale":"2023-10-08T05:56:33.712074+0000","last_undegraded":"2023-10-08T05:56:33.712074+0000","last_fullsized":"2023-10-08T05:56:33.712074+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:23.859616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430740+0000","last_change":"2023-10-08T05:56:32.448871+0000","last_active":"2023-10-08T05:56:33.430740+0000","last_peered":"2023-10-08T05:56:33.430740+0000","last_clean":"2023-10-08T05:56:33.430740+0000","last_became_active":"2023-10-08T05:56:32.448603+0000","last_became_peered":"2023-10-08T05:56:32.448603+0000","last_unstale":"2023-10-08T05:56:33.430740+0000","last_undegraded":"2023-10-08T05:56:33.430740+0000","last_fullsized":"2023-10-08T05:56:33.430740+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:18.147979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430855+0000","last_change":"2023-10-08T05:56:31.923001+0000","last_active":"2023-10-08T05:56:33.430855+0000","last_peered":"2023-10-08T05:56:33.430855+0000","last_clean":"2023-10-08T05:56:33.430855+0000","last_became_active":"2023-10-08T05:56:31.922613+0000","last_became_peered":"2023-10-08T05:56:31.922613+0000","last_unstale":"2023-10-08T05:56:33.430855+0000","last_undegraded":"2023-10-08T05:56:33.430855+0000","last_fullsized":"2023-10-08T05:56:33.430855+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:29.417867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434421+0000","last_change":"2023-10-08T05:56:32.453683+0000","last_active":"2023-10-08T05:56:33.434421+0000","last_peered":"2023-10-08T05:56:33.434421+0000","last_clean":"2023-10-08T05:56:33.434421+0000","last_became_active":"2023-10-08T05:56:32.453413+0000","last_became_peered":"2023-10-08T05:56:32.453413+0000","last_unstale":"2023-10-08T05:56:33.434421+0000","last_undegraded":"2023-10-08T05:56:33.434421+0000","last_fullsized":"2023-10-08T05:56:33.434421+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:17:44.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711501+0000","last_change":"2023-10-08T05:56:31.931739+0000","last_active":"2023-10-08T05:56:33.711501+0000","last_peered":"2023-10-08T05:56:33.711501+0000","last_clean":"2023-10-08T05:56:33.711501+0000","last_became_active":"2023-10-08T05:56:31.931458+0000","last_became_peered":"2023-10-08T05:56:31.931458+0000","last_unstale":"2023-10-08T05:56:33.711501+0000","last_undegraded":"2023-10-08T05:56:33.711501+0000","last_fullsized":"2023-10-08T05:56:33.711501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:29.417867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713238+0000","last_change":"2023-10-08T05:56:32.449255+0000","last_active":"2023-10-08T05:56:33.713238+0000","last_peered":"2023-10-08T05:56:33.713238+0000","last_clean":"2023-10-08T05:56:33.713238+0000","last_became_active":"2023-10-08T05:56:32.448887+0000","last_became_peered":"2023-10-08T05:56:32.448887+0000","last_unstale":"2023-10-08T05:56:33.713238+0000","last_undegraded":"2023-10-08T05:56:33.713238+0000","last_fullsized":"2023-10-08T05:56:33.713238+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:17.632008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712079+0000","last_change":"2023-10-08T05:56:31.935501+0000","last_active":"2023-10-08T05:56:33.712079+0000","last_peered":"2023-10-08T05:56:33.712079+0000","last_clean":"2023-10-08T05:56:33.712079+0000","last_became_active":"2023-10-08T05:56:31.935207+0000","last_became_peered":"2023-10-08T05:56:31.935207+0000","last_unstale":"2023-10-08T05:56:33.712079+0000","last_undegraded":"2023-10-08T05:56:33.712079+0000","last_fullsized":"2023-10-08T05:56:33.712079+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:10.767246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556202+0000","last_change":"2023-10-08T05:56:32.453536+0000","last_active":"2023-10-08T05:56:33.556202+0000","last_peered":"2023-10-08T05:56:33.556202+0000","last_clean":"2023-10-08T05:56:33.556202+0000","last_became_active":"2023-10-08T05:56:32.452803+0000","last_became_peered":"2023-10-08T05:56:32.452803+0000","last_unstale":"2023-10-08T05:56:33.556202+0000","last_undegraded":"2023-10-08T05:56:33.556202+0000","last_fullsized":"2023-10-08T05:56:33.556202+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:53.404295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434674+0000","last_change":"2023-10-08T05:56:31.926227+0000","last_active":"2023-10-08T05:56:33.434674+0000","last_peered":"2023-10-08T05:56:33.434674+0000","last_clean":"2023-10-08T05:56:33.434674+0000","last_became_active":"2023-10-08T05:56:31.925924+0000","last_became_peered":"2023-10-08T05:56:31.925924+0000","last_unstale":"2023-10-08T05:56:33.434674+0000","last_undegraded":"2023-10-08T05:56:33.434674+0000","last_fullsized":"2023-10-08T05:56:33.434674+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:41.694476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712780+0000","last_change":"2023-10-08T05:56:32.460761+0000","last_active":"2023-10-08T05:56:33.712780+0000","last_peered":"2023-10-08T05:56:33.712780+0000","last_clean":"2023-10-08T05:56:33.712780+0000","last_became_active":"2023-10-08T05:56:32.460457+0000","last_became_peered":"2023-10-08T05:56:32.460457+0000","last_unstale":"2023-10-08T05:56:33.712780+0000","last_undegraded":"2023-10-08T05:56:33.712780+0000","last_fullsized":"2023-10-08T05:56:33.712780+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:15.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433544+0000","last_change":"2023-10-08T05:56:31.926931+0000","last_active":"2023-10-08T05:56:33.433544+0000","last_peered":"2023-10-08T05:56:33.433544+0000","last_clean":"2023-10-08T05:56:33.433544+0000","last_became_active":"2023-10-08T05:56:31.926630+0000","last_became_peered":"2023-10-08T05:56:31.926630+0000","last_unstale":"2023-10-08T05:56:33.433544+0000","last_undegraded":"2023-10-08T05:56:33.433544+0000","last_fullsized":"2023-10-08T05:56:33.433544+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:12.316588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711895+0000","last_change":"2023-10-08T05:56:32.447806+0000","last_active":"2023-10-08T05:56:33.711895+0000","last_peered":"2023-10-08T05:56:33.711895+0000","last_clean":"2023-10-08T05:56:33.711895+0000","last_became_active":"2023-10-08T05:56:32.447519+0000","last_became_peered":"2023-10-08T05:56:32.447519+0000","last_unstale":"2023-10-08T05:56:33.711895+0000","last_undegraded":"2023-10-08T05:56:33.711895+0000","last_fullsized":"2023-10-08T05:56:33.711895+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:14.407721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713067+0000","last_change":"2023-10-08T05:56:31.932972+0000","last_active":"2023-10-08T05:56:33.713067+0000","last_peered":"2023-10-08T05:56:33.713067+0000","last_clean":"2023-10-08T05:56:33.713067+0000","last_became_active":"2023-10-08T05:56:31.932442+0000","last_became_peered":"2023-10-08T05:56:31.932442+0000","last_unstale":"2023-10-08T05:56:33.713067+0000","last_undegraded":"2023-10-08T05:56:33.713067+0000","last_fullsized":"2023-10-08T05:56:33.713067+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:47.910691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712047+0000","last_change":"2023-10-08T05:56:32.446571+0000","last_active":"2023-10-08T05:56:33.712047+0000","last_peered":"2023-10-08T05:56:33.712047+0000","last_clean":"2023-10-08T05:56:33.712047+0000","last_became_active":"2023-10-08T05:56:32.446300+0000","last_became_peered":"2023-10-08T05:56:32.446300+0000","last_unstale":"2023-10-08T05:56:33.712047+0000","last_undegraded":"2023-10-08T05:56:33.712047+0000","last_fullsized":"2023-10-08T05:56:33.712047+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:29.549230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712849+0000","last_change":"2023-10-08T05:56:31.930616+0000","last_active":"2023-10-08T05:56:33.712849+0000","last_peered":"2023-10-08T05:56:33.712849+0000","last_clean":"2023-10-08T05:56:33.712849+0000","last_became_active":"2023-10-08T05:56:31.930331+0000","last_became_peered":"2023-10-08T05:56:31.930331+0000","last_unstale":"2023-10-08T05:56:33.712849+0000","last_undegraded":"2023-10-08T05:56:33.712849+0000","last_fullsized":"2023-10-08T05:56:33.712849+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:55.180154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430635+0000","last_change":"2023-10-08T05:56:32.452869+0000","last_active":"2023-10-08T05:56:33.430635+0000","last_peered":"2023-10-08T05:56:33.430635+0000","last_clean":"2023-10-08T05:56:33.430635+0000","last_became_active":"2023-10-08T05:56:32.452595+0000","last_became_peered":"2023-10-08T05:56:32.452595+0000","last_unstale":"2023-10-08T05:56:33.430635+0000","last_undegraded":"2023-10-08T05:56:33.430635+0000","last_fullsized":"2023-10-08T05:56:33.430635+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:02:14.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713217+0000","last_change":"2023-10-08T05:56:31.937769+0000","last_active":"2023-10-08T05:56:33.713217+0000","last_peered":"2023-10-08T05:56:33.713217+0000","last_clean":"2023-10-08T05:56:33.713217+0000","last_became_active":"2023-10-08T05:56:31.937473+0000","last_became_peered":"2023-10-08T05:56:31.937473+0000","last_unstale":"2023-10-08T05:56:33.713217+0000","last_undegraded":"2023-10-08T05:56:33.713217+0000","last_fullsized":"2023-10-08T05:56:33.713217+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:30.482050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711809+0000","last_change":"2023-10-08T05:56:32.461354+0000","last_active":"2023-10-08T05:56:33.711809+0000","last_peered":"2023-10-08T05:56:33.711809+0000","last_clean":"2023-10-08T05:56:33.711809+0000","last_became_active":"2023-10-08T05:56:32.460957+0000","last_became_peered":"2023-10-08T05:56:32.460957+0000","last_unstale":"2023-10-08T05:56:33.711809+0000","last_undegraded":"2023-10-08T05:56:33.711809+0000","last_fullsized":"2023-10-08T05:56:33.711809+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:12.187021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712236+0000","last_change":"2023-10-08T05:56:31.928922+0000","last_active":"2023-10-08T05:56:33.712236+0000","last_peered":"2023-10-08T05:56:33.712236+0000","last_clean":"2023-10-08T05:56:33.712236+0000","last_became_active":"2023-10-08T05:56:31.928654+0000","last_became_peered":"2023-10-08T05:56:31.928654+0000","last_unstale":"2023-10-08T05:56:33.712236+0000","last_undegraded":"2023-10-08T05:56:33.712236+0000","last_fullsized":"2023-10-08T05:56:33.712236+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:57:16.465352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556415+0000","last_change":"2023-10-08T05:56:32.455032+0000","last_active":"2023-10-08T05:56:33.556415+0000","last_peered":"2023-10-08T05:56:33.556415+0000","last_clean":"2023-10-08T05:56:33.556415+0000","last_became_active":"2023-10-08T05:56:32.454758+0000","last_became_peered":"2023-10-08T05:56:32.454758+0000","last_unstale":"2023-10-08T05:56:33.556415+0000","last_undegraded":"2023-10-08T05:56:33.556415+0000","last_fullsized":"2023-10-08T05:56:33.556415+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:49.796506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711901+0000","last_change":"2023-10-08T05:56:31.933166+0000","last_active":"2023-10-08T05:56:33.711901+0000","last_peered":"2023-10-08T05:56:33.711901+0000","last_clean":"2023-10-08T05:56:33.711901+0000","last_became_active":"2023-10-08T05:56:31.932722+0000","last_became_peered":"2023-10-08T05:56:31.932722+0000","last_unstale":"2023-10-08T05:56:33.711901+0000","last_undegraded":"2023-10-08T05:56:33.711901+0000","last_fullsized":"2023-10-08T05:56:33.711901+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:55:03.176910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712881+0000","last_change":"2023-10-08T05:56:32.462277+0000","last_active":"2023-10-08T05:56:33.712881+0000","last_peered":"2023-10-08T05:56:33.712881+0000","last_clean":"2023-10-08T05:56:33.712881+0000","last_became_active":"2023-10-08T05:56:32.461935+0000","last_became_peered":"2023-10-08T05:56:32.461935+0000","last_unstale":"2023-10-08T05:56:33.712881+0000","last_undegraded":"2023-10-08T05:56:33.712881+0000","last_fullsized":"2023-10-08T05:56:33.712881+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:35:27.034365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"148'8","reported_seq":20,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430145+0000","last_change":"2023-10-08T05:56:31.931125+0000","last_active":"2023-10-08T05:56:33.430145+0000","last_peered":"2023-10-08T05:56:33.430145+0000","last_clean":"2023-10-08T05:56:33.430145+0000","last_became_active":"2023-10-08T05:56:31.930852+0000","last_became_peered":"2023-10-08T05:56:31.930852+0000","last_unstale":"2023-10-08T05:56:33.430145+0000","last_undegraded":"2023-10-08T05:56:33.430145+0000","last_fullsized":"2023-10-08T05:56:33.430145+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:03.015840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"137'2685","reported_seq":3180,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-08T05:56:32.442658+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:56:32.442658+0000","last_peered":"2023-10-08T05:56:32.442658+0000","last_clean":"2023-10-08T05:56:32.442658+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:56:32.442658+0000","last_undegraded":"2023-10-08T05:56:32.442658+0000","last_fullsized":"2023-10-08T05:56:32.442658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2685,"log_dups_size":0,"ondisk_log_size":2685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1149,"num_read_kb":2352,"num_write":369,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1149,"num_read_kb":2352,"num_write":369,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2685,"ondisk_log_size":2685,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575091,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55792,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316048,"statfs":{"total":96636764160,"available":96579633152,"internally_reserved":0,"allocated":2461696,"data_stored":1536004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.123,"15min":1.1919999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.36199999999999999},"max":{"1min":1.629,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.165},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.1950000000000001,"15min":1.216},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.44},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.2050000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.2030000000000001,"15min":1.236},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.49199999999999999},"max":{"1min":1.536,"5min":1.8140000000000001,"15min":1.8360000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.2909999999999999,"15min":1.276},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.47099999999999997},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.425}]},{"osd":2,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.194,"15min":1.2330000000000001},"min":{"1min":0.745,"5min":0.71599999999999997,"15min":0.626},"max":{"1min":1.597,"5min":1.748,"15min":1.8939999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3140000000000001,"15min":1.323},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.45400000000000001},"max":{"1min":1.736,"5min":1.97,"15min":1.97},"last":1.46}]},{"osd":3,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.266,"15min":1.2789999999999999},"min":{"1min":0.84899999999999998,"5min":0.72399999999999998,"15min":0.69999999999999996},"max":{"1min":1.6140000000000001,"5min":1.639,"15min":1.8799999999999999},"last":1.008},{"interface":"front","average":{"1min":1.339,"5min":1.371,"15min":1.365},"min":{"1min":0.63,"5min":0.63,"15min":0.60199999999999998},"max":{"1min":1.746,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.82299999999999995}]},{"osd":4,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.363,"15min":1.3009999999999999},"min":{"1min":0.84399999999999997,"5min":0.76200000000000001,"15min":0.504},"max":{"1min":3.6200000000000001,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":1.079},{"interface":"front","average":{"1min":1.373,"5min":1.361,"15min":1.335},"min":{"1min":0.90600000000000003,"5min":0.81599999999999995,"15min":0.63600000000000001},"max":{"1min":1.954,"5min":2.0550000000000002,"15min":2.0699999999999998},"last":1.1200000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4510000000000001,"15min":1.413},"min":{"1min":1.056,"5min":0.69499999999999995,"15min":0.53400000000000003},"max":{"1min":1.7989999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.327},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.4550000000000001,"15min":1.423},"min":{"1min":1.0009999999999999,"5min":0.93600000000000005,"15min":0.39000000000000001},"max":{"1min":1.887,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.3069999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.5660000000000001,"15min":1.5269999999999999},"min":{"1min":1.24,"5min":1.0940000000000001,"15min":0.54400000000000004},"max":{"1min":1.9199999999999999,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4910000000000001,"15min":1.4630000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.355}]}]},{"osd":6,"up_from":13,"seq":55834575090,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37160,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1855488,"data_stored":945442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.2410000000000001,"15min":1.264},"min":{"1min":0.61899999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.486,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.306,"15min":1.302},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.58399999999999996},"max":{"1min":1.5880000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.3009999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1579999999999999,"15min":1.1599999999999999},"min":{"1min":0.60699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7789999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.181,"15min":1.1830000000000001},"min":{"1min":0.78800000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6020000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.448}]},{"osd":2,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.1739999999999999,"15min":1.1799999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5840000000000001,"5min":1.774,"15min":1.796},"last":1.8959999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.2110000000000001,"15min":1.2390000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.58699999999999997},"max":{"1min":1.661,"5min":1.752,"15min":1.8520000000000001},"last":1.1299999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3100000000000001,"15min":1.361},"min":{"1min":0.75600000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.607,"5min":2.137,"15min":2.137},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.419,"15min":1.4379999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.70499999999999996},"max":{"1min":1.6599999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.6850000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3400000000000001,"15min":1.321},"min":{"1min":0.97399999999999998,"5min":0.70399999999999996,"15min":0.60099999999999998},"max":{"1min":1.9399999999999999,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.601},{"interface":"front","average":{"1min":1.349,"5min":1.411,"15min":1.3839999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.64300000000000002},"max":{"1min":1.724,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.8240000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.349,"15min":1.327},"min":{"1min":0.92200000000000004,"5min":0.79500000000000004,"15min":0.69999999999999996},"max":{"1min":1.889,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.389},{"interface":"front","average":{"1min":1.417,"5min":1.512,"15min":1.4830000000000001},"min":{"1min":1.1200000000000001,"5min":0.87,"15min":0.747},"max":{"1min":1.6419999999999999,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.7370000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.546,"15min":1.5029999999999999},"min":{"1min":0.91200000000000003,"5min":0.82399999999999995,"15min":0.77200000000000002},"max":{"1min":2.024,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.9590000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.5069999999999999,"15min":1.4770000000000001},"min":{"1min":0.96499999999999997,"5min":0.85999999999999999,"15min":0.81899999999999995},"max":{"1min":1.9850000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.5089999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575090,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1859584,"data_stored":945020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.125,"15min":1.1259999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.52,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.212,"15min":1.204},"min":{"1min":0.84699999999999998,"5min":0.47899999999999998,"15min":0.442},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.83199999999999996}]},{"osd":1,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1080000000000001,"15min":1.1930000000000001},"min":{"1min":0.746,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":3.2000000000000002},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2490000000000001,"15min":1.2589999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6319999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.371}]},{"osd":2,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.2609999999999999,"15min":1.234},"min":{"1min":0.90500000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.24,"15min":1.2809999999999999},"min":{"1min":0.88800000000000001,"5min":0.71399999999999997,"15min":0.65000000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":3.282},"last":1.075}]},{"osd":3,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3140000000000001,"15min":1.3360000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.63500000000000001},"max":{"1min":1.526,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.633},{"interface":"front","average":{"1min":1.427,"5min":1.389,"15min":1.3959999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":2.2280000000000002},"last":1.583}]},{"osd":4,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.323,"15min":1.306},"min":{"1min":0.70799999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":4.0179999999999998},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.4330000000000001,"15min":1.4299999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8440000000000001,"5min":2.927,"15min":3.456},"last":1.764}]},{"osd":6,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.444,"15min":1.4059999999999999},"min":{"1min":0.79000000000000004,"5min":0.751,"15min":0.751},"max":{"1min":2.149,"5min":2.149,"15min":2.359},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.484,"15min":1.4690000000000001},"min":{"1min":0.77300000000000002,"5min":0.623,"15min":0.58099999999999996},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":3.8620000000000001},"last":1.673}]},{"osd":7,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.538,"15min":1.54},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.829,"5min":3.0270000000000001,"15min":3.4079999999999999},"last":1.845},{"interface":"front","average":{"1min":1.599,"5min":1.5960000000000001,"15min":1.5609999999999999},"min":{"1min":0.73999999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.8169999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575090,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37160,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1863680,"data_stored":946472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2090000000000001,"15min":1.26},"min":{"1min":0.73299999999999998,"5min":0.70399999999999996,"15min":0.498},"max":{"1min":1.6140000000000001,"5min":1.7609999999999999,"15min":1.8149999999999999},"last":1.76},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3129999999999999,"15min":1.361},"min":{"1min":0.84999999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6990000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.165}]},{"osd":1,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.274,"15min":1.2749999999999999},"min":{"1min":0.82099999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6559999999999999,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.60099999999999998},"max":{"1min":1.6559999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.93300000000000005}]},{"osd":2,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.2170000000000001,"15min":1.228},"min":{"1min":0.748,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.542,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.018},{"interface":"front","average":{"1min":1.204,"5min":1.2949999999999999,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.7569999999999999,"5min":2.431,"15min":2.431},"last":1.284}]},{"osd":3,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2669999999999999,"15min":1.276},"min":{"1min":0.81000000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7290000000000001,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.222},{"interface":"front","average":{"1min":1.381,"5min":1.3320000000000001,"15min":1.3400000000000001},"min":{"1min":1.091,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.7470000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.341}]},{"osd":5,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3720000000000001,"15min":1.341},"min":{"1min":0.97699999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6910000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.393},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.385,"15min":1.363},"min":{"1min":0.99199999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.786,"5min":5.4189999999999996,"15min":5.4189999999999996},"last":1.075}]},{"osd":6,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.4419999999999999,"15min":1.363},"min":{"1min":0.83999999999999997,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8069999999999999,"5min":4.9059999999999997,"15min":4.9059999999999997},"last":1.9239999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5680000000000001,"15min":1.5109999999999999},"min":{"1min":1.1559999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.8759999999999999,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":1.51}]},{"osd":7,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.518,"15min":1.4770000000000001},"min":{"1min":1.133,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.792,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":1.446},{"interface":"front","average":{"1min":1.597,"5min":1.5569999999999999,"15min":1.522},"min":{"1min":1.0669999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.8899999999999999,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":2.016}]}]},{"osd":3,"up_from":13,"seq":55834575092,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41324,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330516,"statfs":{"total":96636764160,"available":96594448384,"internally_reserved":0,"allocated":1867776,"data_stored":946506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88300000000000001,"15min":0.91700000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.615,"5min":1.615,"15min":1.72},"last":1.8089999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.0549999999999999,"15min":1.093},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.494,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.772}]},{"osd":1,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.57099999999999995,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.349,"5min":1.605,"15min":1.605},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.161,"5min":1.1519999999999999,"15min":1.1970000000000001},"min":{"1min":0.56100000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.7210000000000001},"last":1.397}]},{"osd":2,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0069999999999999,"15min":1.0529999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.736,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.149},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0900000000000001,"15min":1.1319999999999999},"min":{"1min":0.499,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1899999999999999,"15min":1.2110000000000001},"min":{"1min":0.76200000000000001,"5min":0.68999999999999995,"15min":0.54000000000000004},"max":{"1min":1.647,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3620000000000001,"15min":1.379},"min":{"1min":0.96399999999999997,"5min":0.82699999999999996,"15min":0.72799999999999998},"max":{"1min":1.8460000000000001,"5min":2.589,"15min":2.589},"last":1.575}]},{"osd":5,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2889999999999999,"15min":1.2949999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3109999999999999,"15min":1.343},"min":{"1min":0.81100000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.6850000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.5369999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3420000000000001,"15min":1.3220000000000001},"min":{"1min":0.871,"5min":0.65700000000000003,"15min":0.61799999999999999},"max":{"1min":1.7490000000000001,"5min":1.841,"15min":1.841},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4330000000000001,"15min":1.4379999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.86699999999999999},"max":{"1min":2.0150000000000001,"5min":2.016,"15min":2.0489999999999999},"last":1.694}]},{"osd":7,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.458,"15min":1.482},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.982,"5min":1.982,"15min":2.1480000000000001},"last":1.659},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.379,"15min":1.3859999999999999},"min":{"1min":0.92800000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8380000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.6200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575090,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59364,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312476,"statfs":{"total":96636764160,"available":96575975424,"internally_reserved":0,"allocated":1851392,"data_stored":944860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.79700000000000004,"15min":0.86899999999999999},"min":{"1min":0.48399999999999999,"5min":0.255,"15min":0.255},"max":{"1min":1.526,"5min":1.629,"15min":1.629},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.96199999999999997,"15min":1.0249999999999999},"min":{"1min":0.52300000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.363,"5min":1.5880000000000001,"15min":1.9470000000000001},"last":0.57299999999999995}]},{"osd":1,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.86899999999999999,"15min":0.93600000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.42199999999999999},"max":{"1min":1.173,"5min":1.7609999999999999,"15min":2.1070000000000002},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98699999999999999,"15min":1.0680000000000001},"min":{"1min":0.54400000000000004,"5min":0.47799999999999998,"15min":0.35099999999999998},"max":{"1min":1.417,"5min":1.79,"15min":2.1469999999999998},"last":0.73499999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0449999999999999,"15min":1.1080000000000001},"min":{"1min":0.55200000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.349,"5min":1.631,"15min":1.8120000000000001},"last":1.002},{"interface":"front","average":{"1min":1.153,"5min":1.1020000000000001,"15min":1.206},"min":{"1min":0.66300000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.847,"5min":1.8859999999999999,"15min":7.5890000000000004},"last":1.048}]},{"osd":4,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.2929999999999999,"15min":1.367},"min":{"1min":0.95799999999999996,"5min":0.66000000000000003,"15min":0.622},"max":{"1min":1.7470000000000001,"5min":1.8280000000000001,"15min":4.593},"last":1.145},{"interface":"front","average":{"1min":1.244,"5min":1.2310000000000001,"15min":1.28},"min":{"1min":0.96799999999999997,"5min":0.624,"15min":0.624},"max":{"1min":1.869,"5min":1.881,"15min":6.6879999999999997},"last":1.1100000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.258,"15min":1.359},"min":{"1min":0.88400000000000001,"5min":0.68799999999999994,"15min":0.63900000000000001},"max":{"1min":1.885,"5min":1.885,"15min":6.7140000000000004},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.361,"15min":1.4339999999999999},"min":{"1min":0.88800000000000001,"5min":0.71999999999999997,"15min":0.56599999999999995},"max":{"1min":1.8120000000000001,"5min":1.9359999999999999,"15min":2.4119999999999999},"last":1.226}]},{"osd":6,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.3280000000000001,"15min":1.3460000000000001},"min":{"1min":0.93000000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.7080000000000002},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3300000000000001,"15min":1.431},"min":{"1min":0.91400000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7929999999999999,"5min":1.8400000000000001,"15min":2.8839999999999999},"last":1.2869999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.355,"15min":1.425},"min":{"1min":0.995,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.956,"5min":1.956,"15min":6.609},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.379,"15min":1.478},"min":{"1min":0.91800000000000004,"5min":0.877,"15min":0.82299999999999995},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.9950000000000001},"last":1.3680000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575090,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37152,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334688,"statfs":{"total":96636764160,"available":96598720512,"internally_reserved":0,"allocated":1855488,"data_stored":944882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.91400000000000003,"15min":0.96899999999999997},"min":{"1min":0.48499999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.0349999999999999,"15min":1.1140000000000001},"min":{"1min":0.65500000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.2370000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.873,"15min":0.94099999999999995},"min":{"1min":0.52500000000000002,"5min":0.45100000000000001,"15min":0.39500000000000002},"max":{"1min":1.377,"5min":1.377,"15min":1.8240000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.3,"5min":1.0629999999999999,"15min":1.1000000000000001},"min":{"1min":0.50900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.087,"15min":1.169},"min":{"1min":0.60299999999999998,"5min":0.498,"15min":0.498},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.032,"5min":1.026,"15min":1.071},"min":{"1min":0.58799999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.5469999999999999,"5min":1.6120000000000001,"15min":1.788},"last":1.0880000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.22,"15min":1.2889999999999999},"min":{"1min":0.874,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.891,"5min":1.891,"15min":1.966},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1839999999999999,"15min":1.22},"min":{"1min":0.78200000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.3180000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2669999999999999,"15min":1.304},"min":{"1min":0.80400000000000005,"5min":0.73199999999999998,"15min":0.70399999999999996},"max":{"1min":1.786,"5min":1.7909999999999999,"15min":1.8680000000000001},"last":1.395},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.411,"15min":1.452},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.613}]},{"osd":6,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.361,"15min":1.4139999999999999},"min":{"1min":1.0589999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.833},{"interface":"front","average":{"1min":1.365,"5min":1.357,"15min":1.4099999999999999},"min":{"1min":1.032,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6839999999999999,"5min":1.8580000000000001,"15min":2.8559999999999999},"last":1.5129999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.395,"15min":1.4590000000000001},"min":{"1min":0.92400000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.365},{"interface":"front","average":{"1min":1.488,"5min":1.3939999999999999,"15min":1.4470000000000001},"min":{"1min":1.119,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.893,"5min":2.0819999999999999,"15min":2.306},"last":1.9339999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575091,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33580,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338260,"statfs":{"total":96636764160,"available":96602378240,"internally_reserved":0,"allocated":2449408,"data_stored":1535250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.876,"15min":0.90500000000000003},"min":{"1min":0.38700000000000001,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":1.665,"5min":1.77,"15min":1.77},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.879,"5min":0.98199999999999998,"15min":1.0229999999999999},"min":{"1min":0.41199999999999998,"5min":0.40899999999999997,"15min":0.371},"max":{"1min":1.337,"5min":1.6719999999999999,"15min":1.675},"last":0.753}]},{"osd":2,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0249999999999999,"15min":1.0740000000000001},"min":{"1min":0.435,"5min":0.42599999999999999,"15min":0.39200000000000002},"max":{"1min":1.5740000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.022},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92200000000000004,"15min":0.97799999999999998},"min":{"1min":0.47599999999999998,"5min":0.47399999999999998,"15min":0.40400000000000003},"max":{"1min":1.5880000000000001,"5min":1.722,"15min":1.78},"last":1.175}]},{"osd":3,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.133,"15min":1.1819999999999999},"min":{"1min":0.52300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.823,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0429999999999999,"15min":1.083},"min":{"1min":0.51600000000000001,"5min":0.46500000000000002,"15min":0.44700000000000001},"max":{"1min":1.734,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.121}]},{"osd":4,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2210000000000001,"15min":1.256},"min":{"1min":0.78100000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5860000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.298},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3220000000000001,"15min":1.337},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.962,"5min":1.962,"15min":2.2200000000000002},"last":1.397}]},{"osd":5,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.2929999999999999,"15min":1.3260000000000001},"min":{"1min":0.61199999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8100000000000001,"5min":1.9410000000000001,"15min":2.0489999999999999},"last":1.488},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.365,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.71199999999999997,"15min":0.63},"max":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":1.0649999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3720000000000001,"15min":1.405},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.58499999999999996},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.353,"15min":1.3819999999999999},"min":{"1min":0.73399999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.7050000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.369}]},{"osd":7,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3939999999999999,"15min":1.4219999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.63800000000000001},"max":{"1min":1.8300000000000001,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.4299999999999999,"15min":1.4650000000000001},"min":{"1min":0.76500000000000001,"5min":0.73699999999999999,"15min":0.54700000000000004},"max":{"1min":1.7649999999999999,"5min":2.0569999999999999,"15min":2.2759999999999998},"last":1.655}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:56:37.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:56:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.630+0000 7f94a9787640 1 Processor -- start 2023-10-08T05:56:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.630+0000 7f94a9787640 1 -- start start 2023-10-08T05:56:37.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.631+0000 7f94a9787640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94a40adc70 0x7f94a40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.632+0000 7f94a9787640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a415f7d0 con 0x7f94a415ab30 2023-10-08T05:56:37.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.632+0000 7f94a9787640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a415efb0 con 0x7f94a415af10 2023-10-08T05:56:37.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.632+0000 7f94a9787640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a415f210 con 0x7f94a40adc70 2023-10-08T05:56:37.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.632+0000 7f94a3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f94a415ab30 0x7f94a415e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35326/0 (socket says 172.21.15.70:35326) 2023-10-08T05:56:37.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.632+0000 7f94a3fff640 1 -- 172.21.15.70:0/3264937403 learned_addr learned my addr 172.21.15.70:0/3264937403 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:37.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.632+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492304701 0 0) 0x7f94a415f7d0 con 0x7f94a415ab30 2023-10-08T05:56:37.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.632+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9490003520 con 0x7f94a415ab30 2023-10-08T05:56:37.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656052736 0 0) 0x7f9490003520 con 0x7f94a415ab30 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94a40adc70 msgr2=0x7f94a40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a37fe640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94a40adc70 0x7f94a40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94a40adc70 0x7f94a40b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 >> v1:172.21.15.181:6789/0 conn(0x7f94a415af10 legacy=0x7f94a406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a415f4f0 con 0x7f94a415ab30 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9494001a10 con 0x7f94a415ab30 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 >> v1:172.21.15.70:6789/0 conn(0x7f94a415ab30 legacy=0x7f94a415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94900039a0 0x7f9490003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94900047c0 0x7f9490004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94a40adc70 0x7f949000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a415f210 con 0x7f94900047c0 2023-10-08T05:56:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490005510 con 0x7f94900039a0 2023-10-08T05:56:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949000bec0 con 0x7f94a40adc70 2023-10-08T05:56:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.633+0000 7f94a37fe640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94900047c0 0x7f9490004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.634+0000 7f94a3fff640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94900039a0 0x7f9490003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.634+0000 7f94a8f86640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94a40adc70 0x7f949000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.634+0000 7f94a37fe640 1 -- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94a40adc70 msgr2=0x7f949000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.634+0000 7f94a37fe640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94a40adc70 0x7f949000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.634+0000 7f94a37fe640 1 -- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94900039a0 msgr2=0x7f9490003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.634+0000 7f94a37fe640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94900039a0 0x7f9490003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.634+0000 7f94a37fe640 1 -- 172.21.15.70:0/3264937403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9490003520 con 0x7f94900047c0 2023-10-08T05:56:37.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.635+0000 7f94a37fe640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94900047c0 0x7f9490004c40 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f948c00a9a0 tx=0x7f94900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb74bde6ed28dbd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:37.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.635+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c030400 con 0x7f94900047c0 2023-10-08T05:56:37.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.635+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c030560 con 0x7f94900047c0 2023-10-08T05:56:37.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.635+0000 7f94a2ffd640 1 -- 172.21.15.70:0/3264937403 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0308d0 con 0x7f94900047c0 2023-10-08T05:56:37.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.636+0000 7f94a9787640 1 -- 172.21.15.70:0/3264937403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94900047c0 msgr2=0x7f9490004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.636+0000 7f94a9787640 1 --2- 172.21.15.70:0/3264937403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94900047c0 0x7f9490004c40 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f948c00a9a0 tx=0x7f94900057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.636+0000 7f94a8f86640 1 -- 172.21.15.70:0/3264937403 reap_dead start 2023-10-08T05:56:37.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.636+0000 7f94a9787640 1 -- 172.21.15.70:0/3264937403 shutdown_connections 2023-10-08T05:56:37.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.636+0000 7f94a9787640 1 -- 172.21.15.70:0/3264937403 >> 172.21.15.70:0/3264937403 conn(0x7f94a40cb0c0 msgr2=0x7f94a40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:37.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.636+0000 7f94a9787640 1 -- 172.21.15.70:0/3264937403 shutdown_connections 2023-10-08T05:56:37.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.636+0000 7f94a9787640 1 -- 172.21.15.70:0/3264937403 wait complete. 2023-10-08T05:56:37.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.638+0000 7f94a9787640 1 Processor -- start 2023-10-08T05:56:37.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.638+0000 7f94a9787640 1 -- start start 2023-10-08T05:56:37.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.639+0000 7f94a9787640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94900047c0 0x7f94a415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.639+0000 7f94a9787640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a4154e60 0x7f94a40c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.639+0000 7f94a9787640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94a4156d80 0x7f94a4157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.639+0000 7f94a9787640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a4157740 con 0x7f94a4156d80 2023-10-08T05:56:37.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.639+0000 7f94a9787640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a40cc210 con 0x7f94a4154e60 2023-10-08T05:56:37.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.639+0000 7f94a9787640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a40cc4f0 con 0x7f94900047c0 2023-10-08T05:56:37.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.639+0000 7f94a8f86640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94a4156d80 0x7f94a4157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.639+0000 7f94a8f86640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94a4156d80 0x7f94a4157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41426/0 (socket says 172.21.15.70:41426) 2023-10-08T05:56:37.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.639+0000 7f94a8f86640 1 -- 172.21.15.70:0/61786340 learned_addr learned my addr 172.21.15.70:0/61786340 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:37.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a37fe640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a4154e60 0x7f94a40c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a8f86640 1 -- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94900047c0 msgr2=0x7f94a415b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a3fff640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94900047c0 0x7f94a415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a8f86640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94900047c0 0x7f94a415b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a8f86640 1 -- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a4154e60 msgr2=0x7f94a40c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a8f86640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a4154e60 0x7f94a40c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a8f86640 1 -- 172.21.15.70:0/61786340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a40cc7d0 con 0x7f94a4156d80 2023-10-08T05:56:37.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a37fe640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a4154e60 0x7f94a40c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:37.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a8f86640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94a4156d80 0x7f94a4157200 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f949c000c00 tx=0x7f949c00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:37.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a0ff9640 1 -- 172.21.15.70:0/61786340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c0052e0 con 0x7f94a4156d80 2023-10-08T05:56:37.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a0ff9640 1 -- 172.21.15.70:0/61786340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c005440 con 0x7f94a4156d80 2023-10-08T05:56:37.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a0ff9640 1 -- 172.21.15.70:0/61786340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c003830 con 0x7f94a4156d80 2023-10-08T05:56:37.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.640+0000 7f94a9787640 1 -- 172.21.15.70:0/61786340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a40ccab0 con 0x7f94a4156d80 2023-10-08T05:56:37.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.641+0000 7f94a9787640 1 -- 172.21.15.70:0/61786340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a40cd070 con 0x7f94a4156d80 2023-10-08T05:56:37.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.642+0000 7f94a9787640 1 -- 172.21.15.70:0/61786340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a404e160 con 0x7f94a4156d80 2023-10-08T05:56:37.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.643+0000 7f94a0ff9640 1 -- 172.21.15.70:0/61786340 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f949c003990 con 0x7f94a4156d80 2023-10-08T05:56:37.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.643+0000 7f94a0ff9640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f948406afc0 0x7f948406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:37.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.644+0000 7f94a0ff9640 1 -- 172.21.15.70:0/61786340 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f949c088f00 con 0x7f94a4156d80 2023-10-08T05:56:37.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.644+0000 7f94a3fff640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f948406afc0 0x7f948406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:37.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.645+0000 7f94a3fff640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f948406afc0 0x7f948406d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9494000f80 tx=0x7f9494006040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:37.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.648+0000 7f94a0ff9640 1 -- 172.21.15.70:0/61786340 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f949c053da0 con 0x7f94a4156d80 2023-10-08T05:56:37.735 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-10-08T05:56:37.735+0000 7fc553112640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:56:37.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.836+0000 7f94a9787640 1 -- 172.21.15.70:0/61786340 --> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94a406c830 con 0x7f948406afc0 2023-10-08T05:56:37.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.859+0000 7f94a0ff9640 1 -- 172.21.15.70:0/61786340 <== mgr.6685 v2:172.21.15.181:6832/3857845516 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287236 (secure 0 0 0) 0x7f9494007730 con 0x7f948406afc0 2023-10-08T05:56:37.863 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:56:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 -- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f948406afc0 msgr2=0x7f948406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f948406afc0 0x7f948406d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9494000f80 tx=0x7f9494006040 comp rx=0 tx=0).stop 2023-10-08T05:56:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 -- 172.21.15.70:0/61786340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94a4156d80 msgr2=0x7f94a4157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94a4156d80 0x7f94a4157200 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f949c000c00 tx=0x7f949c00e8a0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 -- 172.21.15.70:0/61786340 shutdown_connections 2023-10-08T05:56:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f948406afc0 0x7f948406d480 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94a4156d80 0x7f94a4157200 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94a4154e60 0x7f94a40c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 --2- 172.21.15.70:0/61786340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94900047c0 0x7f94a415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 -- 172.21.15.70:0/61786340 >> 172.21.15.70:0/61786340 conn(0x7f94a40cb0c0 msgr2=0x7f94a40c9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 -- 172.21.15.70:0/61786340 shutdown_connections 2023-10-08T05:56:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:37.865+0000 7f94a9787640 1 -- 172.21.15.70:0/61786340 wait complete. 2023-10-08T05:56:37.883 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-08T05:56:36.797763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1149,"num_read_kb":2352,"num_write":413,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2723,"ondisk_log_size":2723,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":352576,"kb_used_data":15688,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754622144,"statfs":{"total":773094113280,"available":772733075456,"internally_reserved":0,"allocated":16064512,"data_stored":8744436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":344837691},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.908100"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713788+0000","last_change":"2023-10-08T05:56:32.447348+0000","last_active":"2023-10-08T05:56:33.713788+0000","last_peered":"2023-10-08T05:56:33.713788+0000","last_clean":"2023-10-08T05:56:33.713788+0000","last_became_active":"2023-10-08T05:56:32.446278+0000","last_became_peered":"2023-10-08T05:56:32.446278+0000","last_unstale":"2023-10-08T05:56:33.713788+0000","last_undegraded":"2023-10-08T05:56:33.713788+0000","last_fullsized":"2023-10-08T05:56:33.713788+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:16.771665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.431107+0000","last_change":"2023-10-08T05:56:32.457690+0000","last_active":"2023-10-08T05:56:33.431107+0000","last_peered":"2023-10-08T05:56:33.431107+0000","last_clean":"2023-10-08T05:56:33.431107+0000","last_became_active":"2023-10-08T05:56:32.457391+0000","last_became_peered":"2023-10-08T05:56:32.457391+0000","last_unstale":"2023-10-08T05:56:33.431107+0000","last_undegraded":"2023-10-08T05:56:33.431107+0000","last_fullsized":"2023-10-08T05:56:33.431107+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:36.163475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.431170+0000","last_change":"2023-10-08T05:56:32.450562+0000","last_active":"2023-10-08T05:56:33.431170+0000","last_peered":"2023-10-08T05:56:33.431170+0000","last_clean":"2023-10-08T05:56:33.431170+0000","last_became_active":"2023-10-08T05:56:32.450240+0000","last_became_peered":"2023-10-08T05:56:32.450240+0000","last_unstale":"2023-10-08T05:56:33.431170+0000","last_undegraded":"2023-10-08T05:56:33.431170+0000","last_fullsized":"2023-10-08T05:56:33.431170+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:11.783751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712182+0000","last_change":"2023-10-08T05:56:32.454874+0000","last_active":"2023-10-08T05:56:33.712182+0000","last_peered":"2023-10-08T05:56:33.712182+0000","last_clean":"2023-10-08T05:56:33.712182+0000","last_became_active":"2023-10-08T05:56:32.454535+0000","last_became_peered":"2023-10-08T05:56:32.454535+0000","last_unstale":"2023-10-08T05:56:33.712182+0000","last_undegraded":"2023-10-08T05:56:33.712182+0000","last_fullsized":"2023-10-08T05:56:33.712182+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:04.579220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713158+0000","last_change":"2023-10-08T05:56:32.451872+0000","last_active":"2023-10-08T05:56:33.713158+0000","last_peered":"2023-10-08T05:56:33.713158+0000","last_clean":"2023-10-08T05:56:33.713158+0000","last_became_active":"2023-10-08T05:56:32.451580+0000","last_became_peered":"2023-10-08T05:56:32.451580+0000","last_unstale":"2023-10-08T05:56:33.713158+0000","last_undegraded":"2023-10-08T05:56:33.713158+0000","last_fullsized":"2023-10-08T05:56:33.713158+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:45:33.173494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712563+0000","last_change":"2023-10-08T05:56:32.443961+0000","last_active":"2023-10-08T05:56:33.712563+0000","last_peered":"2023-10-08T05:56:33.712563+0000","last_clean":"2023-10-08T05:56:33.712563+0000","last_became_active":"2023-10-08T05:56:32.442742+0000","last_became_peered":"2023-10-08T05:56:32.442742+0000","last_unstale":"2023-10-08T05:56:33.712563+0000","last_undegraded":"2023-10-08T05:56:33.712563+0000","last_fullsized":"2023-10-08T05:56:33.712563+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:20.206129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711916+0000","last_change":"2023-10-08T05:56:32.456145+0000","last_active":"2023-10-08T05:56:33.711916+0000","last_peered":"2023-10-08T05:56:33.711916+0000","last_clean":"2023-10-08T05:56:33.711916+0000","last_became_active":"2023-10-08T05:56:32.455861+0000","last_became_peered":"2023-10-08T05:56:32.455861+0000","last_unstale":"2023-10-08T05:56:33.711916+0000","last_undegraded":"2023-10-08T05:56:33.711916+0000","last_fullsized":"2023-10-08T05:56:33.711916+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:32.678293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430299+0000","last_change":"2023-10-08T05:56:32.451302+0000","last_active":"2023-10-08T05:56:33.430299+0000","last_peered":"2023-10-08T05:56:33.430299+0000","last_clean":"2023-10-08T05:56:33.430299+0000","last_became_active":"2023-10-08T05:56:32.451002+0000","last_became_peered":"2023-10-08T05:56:32.451002+0000","last_unstale":"2023-10-08T05:56:33.430299+0000","last_undegraded":"2023-10-08T05:56:33.430299+0000","last_fullsized":"2023-10-08T05:56:33.430299+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:35:27.034365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713202+0000","last_change":"2023-10-08T05:56:32.454949+0000","last_active":"2023-10-08T05:56:33.713202+0000","last_peered":"2023-10-08T05:56:33.713202+0000","last_clean":"2023-10-08T05:56:33.713202+0000","last_became_active":"2023-10-08T05:56:32.454657+0000","last_became_peered":"2023-10-08T05:56:32.454657+0000","last_unstale":"2023-10-08T05:56:33.713202+0000","last_undegraded":"2023-10-08T05:56:33.713202+0000","last_fullsized":"2023-10-08T05:56:33.713202+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:57:17.481858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556303+0000","last_change":"2023-10-08T05:56:32.444306+0000","last_active":"2023-10-08T05:56:33.556303+0000","last_peered":"2023-10-08T05:56:33.556303+0000","last_clean":"2023-10-08T05:56:33.556303+0000","last_became_active":"2023-10-08T05:56:32.444031+0000","last_became_peered":"2023-10-08T05:56:32.444031+0000","last_unstale":"2023-10-08T05:56:33.556303+0000","last_undegraded":"2023-10-08T05:56:33.556303+0000","last_fullsized":"2023-10-08T05:56:33.556303+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:19.099231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713800+0000","last_change":"2023-10-08T05:56:32.449336+0000","last_active":"2023-10-08T05:56:33.713800+0000","last_peered":"2023-10-08T05:56:33.713800+0000","last_clean":"2023-10-08T05:56:33.713800+0000","last_became_active":"2023-10-08T05:56:32.449046+0000","last_became_peered":"2023-10-08T05:56:32.449046+0000","last_unstale":"2023-10-08T05:56:33.713800+0000","last_undegraded":"2023-10-08T05:56:33.713800+0000","last_fullsized":"2023-10-08T05:56:33.713800+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:40.177643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712916+0000","last_change":"2023-10-08T05:56:32.453391+0000","last_active":"2023-10-08T05:56:33.712916+0000","last_peered":"2023-10-08T05:56:33.712916+0000","last_clean":"2023-10-08T05:56:33.712916+0000","last_became_active":"2023-10-08T05:56:32.453124+0000","last_became_peered":"2023-10-08T05:56:32.453124+0000","last_unstale":"2023-10-08T05:56:33.712916+0000","last_undegraded":"2023-10-08T05:56:33.712916+0000","last_fullsized":"2023-10-08T05:56:33.712916+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:19:52.048323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556039+0000","last_change":"2023-10-08T05:56:32.451548+0000","last_active":"2023-10-08T05:56:33.556039+0000","last_peered":"2023-10-08T05:56:33.556039+0000","last_clean":"2023-10-08T05:56:33.556039+0000","last_became_active":"2023-10-08T05:56:32.451108+0000","last_became_peered":"2023-10-08T05:56:32.451108+0000","last_unstale":"2023-10-08T05:56:33.556039+0000","last_undegraded":"2023-10-08T05:56:33.556039+0000","last_fullsized":"2023-10-08T05:56:33.556039+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:54.769675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430367+0000","last_change":"2023-10-08T05:56:32.449688+0000","last_active":"2023-10-08T05:56:33.430367+0000","last_peered":"2023-10-08T05:56:33.430367+0000","last_clean":"2023-10-08T05:56:33.430367+0000","last_became_active":"2023-10-08T05:56:32.449429+0000","last_became_peered":"2023-10-08T05:56:32.449429+0000","last_unstale":"2023-10-08T05:56:33.430367+0000","last_undegraded":"2023-10-08T05:56:33.430367+0000","last_fullsized":"2023-10-08T05:56:33.430367+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:57:17.481858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711938+0000","last_change":"2023-10-08T05:56:32.445147+0000","last_active":"2023-10-08T05:56:33.711938+0000","last_peered":"2023-10-08T05:56:33.711938+0000","last_clean":"2023-10-08T05:56:33.711938+0000","last_became_active":"2023-10-08T05:56:32.444764+0000","last_became_peered":"2023-10-08T05:56:32.444764+0000","last_unstale":"2023-10-08T05:56:33.711938+0000","last_undegraded":"2023-10-08T05:56:33.711938+0000","last_fullsized":"2023-10-08T05:56:33.711938+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:02.069891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711890+0000","last_change":"2023-10-08T05:56:32.441172+0000","last_active":"2023-10-08T05:56:33.711890+0000","last_peered":"2023-10-08T05:56:33.711890+0000","last_clean":"2023-10-08T05:56:33.711890+0000","last_became_active":"2023-10-08T05:56:32.440481+0000","last_became_peered":"2023-10-08T05:56:32.440481+0000","last_unstale":"2023-10-08T05:56:33.711890+0000","last_undegraded":"2023-10-08T05:56:33.711890+0000","last_fullsized":"2023-10-08T05:56:33.711890+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:16.824353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556383+0000","last_change":"2023-10-08T05:56:32.452293+0000","last_active":"2023-10-08T05:56:33.556383+0000","last_peered":"2023-10-08T05:56:33.556383+0000","last_clean":"2023-10-08T05:56:33.556383+0000","last_became_active":"2023-10-08T05:56:32.451996+0000","last_became_peered":"2023-10-08T05:56:32.451996+0000","last_unstale":"2023-10-08T05:56:33.556383+0000","last_undegraded":"2023-10-08T05:56:33.556383+0000","last_fullsized":"2023-10-08T05:56:33.556383+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:32.454202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.431436+0000","last_change":"2023-10-08T05:56:32.457330+0000","last_active":"2023-10-08T05:56:33.431436+0000","last_peered":"2023-10-08T05:56:33.431436+0000","last_clean":"2023-10-08T05:56:33.431436+0000","last_became_active":"2023-10-08T05:56:32.457074+0000","last_became_peered":"2023-10-08T05:56:32.457074+0000","last_unstale":"2023-10-08T05:56:33.431436+0000","last_undegraded":"2023-10-08T05:56:33.431436+0000","last_fullsized":"2023-10-08T05:56:33.431436+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:16.824353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.432944+0000","last_change":"2023-10-08T05:56:32.459958+0000","last_active":"2023-10-08T05:56:33.432944+0000","last_peered":"2023-10-08T05:56:33.432944+0000","last_clean":"2023-10-08T05:56:33.432944+0000","last_became_active":"2023-10-08T05:56:32.459617+0000","last_became_peered":"2023-10-08T05:56:32.459617+0000","last_unstale":"2023-10-08T05:56:33.432944+0000","last_undegraded":"2023-10-08T05:56:33.432944+0000","last_fullsized":"2023-10-08T05:56:33.432944+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:26.776339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711696+0000","last_change":"2023-10-08T05:56:32.437161+0000","last_active":"2023-10-08T05:56:33.711696+0000","last_peered":"2023-10-08T05:56:33.711696+0000","last_clean":"2023-10-08T05:56:33.711696+0000","last_became_active":"2023-10-08T05:56:32.436827+0000","last_became_peered":"2023-10-08T05:56:32.436827+0000","last_unstale":"2023-10-08T05:56:33.711696+0000","last_undegraded":"2023-10-08T05:56:33.711696+0000","last_fullsized":"2023-10-08T05:56:33.711696+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:22:17.819587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712525+0000","last_change":"2023-10-08T05:56:32.461871+0000","last_active":"2023-10-08T05:56:33.712525+0000","last_peered":"2023-10-08T05:56:33.712525+0000","last_clean":"2023-10-08T05:56:33.712525+0000","last_became_active":"2023-10-08T05:56:32.461515+0000","last_became_peered":"2023-10-08T05:56:32.461515+0000","last_unstale":"2023-10-08T05:56:33.712525+0000","last_undegraded":"2023-10-08T05:56:33.712525+0000","last_fullsized":"2023-10-08T05:56:33.712525+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:13.134824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713206+0000","last_change":"2023-10-08T05:56:32.450561+0000","last_active":"2023-10-08T05:56:33.713206+0000","last_peered":"2023-10-08T05:56:33.713206+0000","last_clean":"2023-10-08T05:56:33.713206+0000","last_became_active":"2023-10-08T05:56:32.450024+0000","last_became_peered":"2023-10-08T05:56:32.450024+0000","last_unstale":"2023-10-08T05:56:33.713206+0000","last_undegraded":"2023-10-08T05:56:33.713206+0000","last_fullsized":"2023-10-08T05:56:33.713206+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:23:09.866243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712887+0000","last_change":"2023-10-08T05:56:32.453905+0000","last_active":"2023-10-08T05:56:33.712887+0000","last_peered":"2023-10-08T05:56:33.712887+0000","last_clean":"2023-10-08T05:56:33.712887+0000","last_became_active":"2023-10-08T05:56:32.453612+0000","last_became_peered":"2023-10-08T05:56:32.453612+0000","last_unstale":"2023-10-08T05:56:33.712887+0000","last_undegraded":"2023-10-08T05:56:33.712887+0000","last_fullsized":"2023-10-08T05:56:33.712887+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:03.088990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433225+0000","last_change":"2023-10-08T05:56:32.449674+0000","last_active":"2023-10-08T05:56:33.433225+0000","last_peered":"2023-10-08T05:56:33.433225+0000","last_clean":"2023-10-08T05:56:33.433225+0000","last_became_active":"2023-10-08T05:56:32.449340+0000","last_became_peered":"2023-10-08T05:56:32.449340+0000","last_unstale":"2023-10-08T05:56:33.433225+0000","last_undegraded":"2023-10-08T05:56:33.433225+0000","last_fullsized":"2023-10-08T05:56:33.433225+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:15.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.435023+0000","last_change":"2023-10-08T05:56:32.448292+0000","last_active":"2023-10-08T05:56:33.435023+0000","last_peered":"2023-10-08T05:56:33.435023+0000","last_clean":"2023-10-08T05:56:33.435023+0000","last_became_active":"2023-10-08T05:56:32.447958+0000","last_became_peered":"2023-10-08T05:56:32.447958+0000","last_unstale":"2023-10-08T05:56:33.435023+0000","last_undegraded":"2023-10-08T05:56:33.435023+0000","last_fullsized":"2023-10-08T05:56:33.435023+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:36.163475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711398+0000","last_change":"2023-10-08T05:56:32.445105+0000","last_active":"2023-10-08T05:56:33.711398+0000","last_peered":"2023-10-08T05:56:33.711398+0000","last_clean":"2023-10-08T05:56:33.711398+0000","last_became_active":"2023-10-08T05:56:32.444833+0000","last_became_peered":"2023-10-08T05:56:32.444833+0000","last_unstale":"2023-10-08T05:56:33.711398+0000","last_undegraded":"2023-10-08T05:56:33.711398+0000","last_fullsized":"2023-10-08T05:56:33.711398+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:27.710441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433316+0000","last_change":"2023-10-08T05:56:32.458454+0000","last_active":"2023-10-08T05:56:33.433316+0000","last_peered":"2023-10-08T05:56:33.433316+0000","last_clean":"2023-10-08T05:56:33.433316+0000","last_became_active":"2023-10-08T05:56:32.456358+0000","last_became_peered":"2023-10-08T05:56:32.456358+0000","last_unstale":"2023-10-08T05:56:33.433316+0000","last_undegraded":"2023-10-08T05:56:33.433316+0000","last_fullsized":"2023-10-08T05:56:33.433316+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:43:57.341349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712864+0000","last_change":"2023-10-08T05:56:32.450843+0000","last_active":"2023-10-08T05:56:33.712864+0000","last_peered":"2023-10-08T05:56:33.712864+0000","last_clean":"2023-10-08T05:56:33.712864+0000","last_became_active":"2023-10-08T05:56:32.450577+0000","last_became_peered":"2023-10-08T05:56:32.450577+0000","last_unstale":"2023-10-08T05:56:33.712864+0000","last_undegraded":"2023-10-08T05:56:33.712864+0000","last_fullsized":"2023-10-08T05:56:33.712864+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:57.980273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711928+0000","last_change":"2023-10-08T05:56:32.436021+0000","last_active":"2023-10-08T05:56:33.711928+0000","last_peered":"2023-10-08T05:56:33.711928+0000","last_clean":"2023-10-08T05:56:33.711928+0000","last_became_active":"2023-10-08T05:56:32.435639+0000","last_became_peered":"2023-10-08T05:56:32.435639+0000","last_unstale":"2023-10-08T05:56:33.711928+0000","last_undegraded":"2023-10-08T05:56:33.711928+0000","last_fullsized":"2023-10-08T05:56:33.711928+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:12.399582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430651+0000","last_change":"2023-10-08T05:56:32.447378+0000","last_active":"2023-10-08T05:56:33.430651+0000","last_peered":"2023-10-08T05:56:33.430651+0000","last_clean":"2023-10-08T05:56:33.430651+0000","last_became_active":"2023-10-08T05:56:32.447126+0000","last_became_peered":"2023-10-08T05:56:32.447126+0000","last_unstale":"2023-10-08T05:56:33.430651+0000","last_undegraded":"2023-10-08T05:56:33.430651+0000","last_fullsized":"2023-10-08T05:56:33.430651+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:48.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433862+0000","last_change":"2023-10-08T05:56:32.455410+0000","last_active":"2023-10-08T05:56:33.433862+0000","last_peered":"2023-10-08T05:56:33.433862+0000","last_clean":"2023-10-08T05:56:33.433862+0000","last_became_active":"2023-10-08T05:56:32.455124+0000","last_became_peered":"2023-10-08T05:56:32.455124+0000","last_unstale":"2023-10-08T05:56:33.433862+0000","last_undegraded":"2023-10-08T05:56:33.433862+0000","last_fullsized":"2023-10-08T05:56:33.433862+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:57:17.481858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433499+0000","last_change":"2023-10-08T05:56:32.449365+0000","last_active":"2023-10-08T05:56:33.433499+0000","last_peered":"2023-10-08T05:56:33.433499+0000","last_clean":"2023-10-08T05:56:33.433499+0000","last_became_active":"2023-10-08T05:56:32.448986+0000","last_became_peered":"2023-10-08T05:56:32.448986+0000","last_unstale":"2023-10-08T05:56:33.433499+0000","last_undegraded":"2023-10-08T05:56:33.433499+0000","last_fullsized":"2023-10-08T05:56:33.433499+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:35:27.034365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556621+0000","last_change":"2023-10-08T05:56:32.446842+0000","last_active":"2023-10-08T05:56:33.556621+0000","last_peered":"2023-10-08T05:56:33.556621+0000","last_clean":"2023-10-08T05:56:33.556621+0000","last_became_active":"2023-10-08T05:56:32.446553+0000","last_became_peered":"2023-10-08T05:56:32.446553+0000","last_unstale":"2023-10-08T05:56:33.556621+0000","last_undegraded":"2023-10-08T05:56:33.556621+0000","last_fullsized":"2023-10-08T05:56:33.556621+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:57:57.911763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"148'4","reported_seq":16,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433585+0000","last_change":"2023-10-08T05:56:31.928279+0000","last_active":"2023-10-08T05:56:33.433585+0000","last_peered":"2023-10-08T05:56:33.433585+0000","last_clean":"2023-10-08T05:56:33.433585+0000","last_became_active":"2023-10-08T05:56:31.927975+0000","last_became_peered":"2023-10-08T05:56:31.927975+0000","last_unstale":"2023-10-08T05:56:33.433585+0000","last_undegraded":"2023-10-08T05:56:33.433585+0000","last_fullsized":"2023-10-08T05:56:33.433585+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:32:07.072724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712381+0000","last_change":"2023-10-08T05:56:32.442579+0000","last_active":"2023-10-08T05:56:33.712381+0000","last_peered":"2023-10-08T05:56:33.712381+0000","last_clean":"2023-10-08T05:56:33.712381+0000","last_became_active":"2023-10-08T05:56:32.442277+0000","last_became_peered":"2023-10-08T05:56:32.442277+0000","last_unstale":"2023-10-08T05:56:33.712381+0000","last_undegraded":"2023-10-08T05:56:33.712381+0000","last_fullsized":"2023-10-08T05:56:33.712381+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:04.987920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"148'1","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556528+0000","last_change":"2023-10-08T05:56:31.925313+0000","last_active":"2023-10-08T05:56:33.556528+0000","last_peered":"2023-10-08T05:56:33.556528+0000","last_clean":"2023-10-08T05:56:33.556528+0000","last_became_active":"2023-10-08T05:56:31.925025+0000","last_became_peered":"2023-10-08T05:56:31.925025+0000","last_unstale":"2023-10-08T05:56:33.556528+0000","last_undegraded":"2023-10-08T05:56:33.556528+0000","last_fullsized":"2023-10-08T05:56:33.556528+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:45:32.156988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434868+0000","last_change":"2023-10-08T05:56:32.453328+0000","last_active":"2023-10-08T05:56:33.434868+0000","last_peered":"2023-10-08T05:56:33.434868+0000","last_clean":"2023-10-08T05:56:33.434868+0000","last_became_active":"2023-10-08T05:56:32.453020+0000","last_became_peered":"2023-10-08T05:56:32.453020+0000","last_unstale":"2023-10-08T05:56:33.434868+0000","last_undegraded":"2023-10-08T05:56:33.434868+0000","last_fullsized":"2023-10-08T05:56:33.434868+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:48.927196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712559+0000","last_change":"2023-10-08T05:56:31.932314+0000","last_active":"2023-10-08T05:56:33.712559+0000","last_peered":"2023-10-08T05:56:33.712559+0000","last_clean":"2023-10-08T05:56:33.712559+0000","last_became_active":"2023-10-08T05:56:31.932025+0000","last_became_peered":"2023-10-08T05:56:31.932025+0000","last_unstale":"2023-10-08T05:56:33.712559+0000","last_undegraded":"2023-10-08T05:56:33.712559+0000","last_fullsized":"2023-10-08T05:56:33.712559+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:55.552305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713330+0000","last_change":"2023-10-08T05:56:32.458979+0000","last_active":"2023-10-08T05:56:33.713330+0000","last_peered":"2023-10-08T05:56:33.713330+0000","last_clean":"2023-10-08T05:56:33.713330+0000","last_became_active":"2023-10-08T05:56:32.458714+0000","last_became_peered":"2023-10-08T05:56:32.458714+0000","last_unstale":"2023-10-08T05:56:33.713330+0000","last_undegraded":"2023-10-08T05:56:33.713330+0000","last_fullsized":"2023-10-08T05:56:33.713330+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:17:44.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556583+0000","last_change":"2023-10-08T05:56:31.925179+0000","last_active":"2023-10-08T05:56:33.556583+0000","last_peered":"2023-10-08T05:56:33.556583+0000","last_clean":"2023-10-08T05:56:33.556583+0000","last_became_active":"2023-10-08T05:56:31.924703+0000","last_became_peered":"2023-10-08T05:56:31.924703+0000","last_unstale":"2023-10-08T05:56:33.556583+0000","last_undegraded":"2023-10-08T05:56:33.556583+0000","last_fullsized":"2023-10-08T05:56:33.556583+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:34.435368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556265+0000","last_change":"2023-10-08T05:56:32.452605+0000","last_active":"2023-10-08T05:56:33.556265+0000","last_peered":"2023-10-08T05:56:33.556265+0000","last_clean":"2023-10-08T05:56:33.556265+0000","last_became_active":"2023-10-08T05:56:32.452337+0000","last_became_peered":"2023-10-08T05:56:32.452337+0000","last_unstale":"2023-10-08T05:56:33.556265+0000","last_undegraded":"2023-10-08T05:56:33.556265+0000","last_fullsized":"2023-10-08T05:56:33.556265+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:49:42.893750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"148'1","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712589+0000","last_change":"2023-10-08T05:56:31.931099+0000","last_active":"2023-10-08T05:56:33.712589+0000","last_peered":"2023-10-08T05:56:33.712589+0000","last_clean":"2023-10-08T05:56:33.712589+0000","last_became_active":"2023-10-08T05:56:31.930830+0000","last_became_peered":"2023-10-08T05:56:31.930830+0000","last_unstale":"2023-10-08T05:56:33.712589+0000","last_undegraded":"2023-10-08T05:56:33.712589+0000","last_fullsized":"2023-10-08T05:56:33.712589+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:17.810142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713299+0000","last_change":"2023-10-08T05:56:32.446454+0000","last_active":"2023-10-08T05:56:33.713299+0000","last_peered":"2023-10-08T05:56:33.713299+0000","last_clean":"2023-10-08T05:56:33.713299+0000","last_became_active":"2023-10-08T05:56:32.446133+0000","last_became_peered":"2023-10-08T05:56:32.446133+0000","last_unstale":"2023-10-08T05:56:33.713299+0000","last_undegraded":"2023-10-08T05:56:33.713299+0000","last_fullsized":"2023-10-08T05:56:33.713299+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:35.451874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"148'1","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712444+0000","last_change":"2023-10-08T05:56:31.929046+0000","last_active":"2023-10-08T05:56:33.712444+0000","last_peered":"2023-10-08T05:56:33.712444+0000","last_clean":"2023-10-08T05:56:33.712444+0000","last_became_active":"2023-10-08T05:56:31.927539+0000","last_became_peered":"2023-10-08T05:56:31.927539+0000","last_unstale":"2023-10-08T05:56:33.712444+0000","last_undegraded":"2023-10-08T05:56:33.712444+0000","last_fullsized":"2023-10-08T05:56:33.712444+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:02:13.137236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430512+0000","last_change":"2023-10-08T05:56:32.446226+0000","last_active":"2023-10-08T05:56:33.430512+0000","last_peered":"2023-10-08T05:56:33.430512+0000","last_clean":"2023-10-08T05:56:33.430512+0000","last_became_active":"2023-10-08T05:56:32.445940+0000","last_became_peered":"2023-10-08T05:56:32.445940+0000","last_unstale":"2023-10-08T05:56:33.430512+0000","last_undegraded":"2023-10-08T05:56:33.430512+0000","last_fullsized":"2023-10-08T05:56:33.430512+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:17:44.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433852+0000","last_change":"2023-10-08T05:56:31.930229+0000","last_active":"2023-10-08T05:56:33.433852+0000","last_peered":"2023-10-08T05:56:33.433852+0000","last_clean":"2023-10-08T05:56:33.433852+0000","last_became_active":"2023-10-08T05:56:31.929712+0000","last_became_peered":"2023-10-08T05:56:31.929712+0000","last_unstale":"2023-10-08T05:56:33.433852+0000","last_undegraded":"2023-10-08T05:56:33.433852+0000","last_fullsized":"2023-10-08T05:56:33.433852+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:03.892658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556408+0000","last_change":"2023-10-08T05:56:32.451357+0000","last_active":"2023-10-08T05:56:33.556408+0000","last_peered":"2023-10-08T05:56:33.556408+0000","last_clean":"2023-10-08T05:56:33.556408+0000","last_became_active":"2023-10-08T05:56:32.450738+0000","last_became_peered":"2023-10-08T05:56:32.450738+0000","last_unstale":"2023-10-08T05:56:33.556408+0000","last_undegraded":"2023-10-08T05:56:33.556408+0000","last_fullsized":"2023-10-08T05:56:33.556408+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:13.991802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712142+0000","last_change":"2023-10-08T05:56:31.930275+0000","last_active":"2023-10-08T05:56:33.712142+0000","last_peered":"2023-10-08T05:56:33.712142+0000","last_clean":"2023-10-08T05:56:33.712142+0000","last_became_active":"2023-10-08T05:56:31.929949+0000","last_became_peered":"2023-10-08T05:56:31.929949+0000","last_unstale":"2023-10-08T05:56:33.712142+0000","last_undegraded":"2023-10-08T05:56:33.712142+0000","last_fullsized":"2023-10-08T05:56:33.712142+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:17.131473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433517+0000","last_change":"2023-10-08T05:56:32.454749+0000","last_active":"2023-10-08T05:56:33.433517+0000","last_peered":"2023-10-08T05:56:33.433517+0000","last_clean":"2023-10-08T05:56:33.433517+0000","last_became_active":"2023-10-08T05:56:32.454483+0000","last_became_peered":"2023-10-08T05:56:32.454483+0000","last_unstale":"2023-10-08T05:56:33.433517+0000","last_undegraded":"2023-10-08T05:56:33.433517+0000","last_fullsized":"2023-10-08T05:56:33.433517+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:54.638366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433819+0000","last_change":"2023-10-08T05:56:31.926933+0000","last_active":"2023-10-08T05:56:33.433819+0000","last_peered":"2023-10-08T05:56:33.433819+0000","last_clean":"2023-10-08T05:56:33.433819+0000","last_became_active":"2023-10-08T05:56:31.926638+0000","last_became_peered":"2023-10-08T05:56:31.926638+0000","last_unstale":"2023-10-08T05:56:33.433819+0000","last_undegraded":"2023-10-08T05:56:33.433819+0000","last_fullsized":"2023-10-08T05:56:33.433819+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:29.501903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434916+0000","last_change":"2023-10-08T05:56:32.446591+0000","last_active":"2023-10-08T05:56:33.434916+0000","last_peered":"2023-10-08T05:56:33.434916+0000","last_clean":"2023-10-08T05:56:33.434916+0000","last_became_active":"2023-10-08T05:56:32.446177+0000","last_became_peered":"2023-10-08T05:56:32.446177+0000","last_unstale":"2023-10-08T05:56:33.434916+0000","last_undegraded":"2023-10-08T05:56:33.434916+0000","last_fullsized":"2023-10-08T05:56:33.434916+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:18.147979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434654+0000","last_change":"2023-10-08T05:56:31.924766+0000","last_active":"2023-10-08T05:56:33.434654+0000","last_peered":"2023-10-08T05:56:33.434654+0000","last_clean":"2023-10-08T05:56:33.434654+0000","last_became_active":"2023-10-08T05:56:31.924139+0000","last_became_peered":"2023-10-08T05:56:31.924139+0000","last_unstale":"2023-10-08T05:56:33.434654+0000","last_undegraded":"2023-10-08T05:56:33.434654+0000","last_fullsized":"2023-10-08T05:56:33.434654+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:11.170516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.431554+0000","last_change":"2023-10-08T05:56:32.450421+0000","last_active":"2023-10-08T05:56:33.431554+0000","last_peered":"2023-10-08T05:56:33.431554+0000","last_clean":"2023-10-08T05:56:33.431554+0000","last_became_active":"2023-10-08T05:56:32.450091+0000","last_became_peered":"2023-10-08T05:56:32.450091+0000","last_unstale":"2023-10-08T05:56:33.431554+0000","last_undegraded":"2023-10-08T05:56:33.431554+0000","last_fullsized":"2023-10-08T05:56:33.431554+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:32.678293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"148'3","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.435212+0000","last_change":"2023-10-08T05:56:31.934321+0000","last_active":"2023-10-08T05:56:33.435212+0000","last_peered":"2023-10-08T05:56:33.435212+0000","last_clean":"2023-10-08T05:56:33.435212+0000","last_became_active":"2023-10-08T05:56:31.933973+0000","last_became_peered":"2023-10-08T05:56:31.933973+0000","last_unstale":"2023-10-08T05:56:33.435212+0000","last_undegraded":"2023-10-08T05:56:33.435212+0000","last_fullsized":"2023-10-08T05:56:33.435212+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:03.562715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.435148+0000","last_change":"2023-10-08T05:56:32.452863+0000","last_active":"2023-10-08T05:56:33.435148+0000","last_peered":"2023-10-08T05:56:33.435148+0000","last_clean":"2023-10-08T05:56:33.435148+0000","last_became_active":"2023-10-08T05:56:32.452570+0000","last_became_peered":"2023-10-08T05:56:32.452570+0000","last_unstale":"2023-10-08T05:56:33.435148+0000","last_undegraded":"2023-10-08T05:56:33.435148+0000","last_fullsized":"2023-10-08T05:56:33.435148+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:11.783751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"148'6","reported_seq":18,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430539+0000","last_change":"2023-10-08T05:56:31.934739+0000","last_active":"2023-10-08T05:56:33.430539+0000","last_peered":"2023-10-08T05:56:33.430539+0000","last_clean":"2023-10-08T05:56:33.430539+0000","last_became_active":"2023-10-08T05:56:31.934463+0000","last_became_peered":"2023-10-08T05:56:31.934463+0000","last_unstale":"2023-10-08T05:56:33.430539+0000","last_undegraded":"2023-10-08T05:56:33.430539+0000","last_fullsized":"2023-10-08T05:56:33.430539+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:38:45.159920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.432791+0000","last_change":"2023-10-08T05:56:32.446763+0000","last_active":"2023-10-08T05:56:33.432791+0000","last_peered":"2023-10-08T05:56:33.432791+0000","last_clean":"2023-10-08T05:56:33.432791+0000","last_became_active":"2023-10-08T05:56:32.446305+0000","last_became_peered":"2023-10-08T05:56:32.446305+0000","last_unstale":"2023-10-08T05:56:33.432791+0000","last_undegraded":"2023-10-08T05:56:33.432791+0000","last_fullsized":"2023-10-08T05:56:33.432791+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:28.275980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430667+0000","last_change":"2023-10-08T05:56:31.930410+0000","last_active":"2023-10-08T05:56:33.430667+0000","last_peered":"2023-10-08T05:56:33.430667+0000","last_clean":"2023-10-08T05:56:33.430667+0000","last_became_active":"2023-10-08T05:56:31.930156+0000","last_became_peered":"2023-10-08T05:56:31.930156+0000","last_unstale":"2023-10-08T05:56:33.430667+0000","last_undegraded":"2023-10-08T05:56:33.430667+0000","last_fullsized":"2023-10-08T05:56:33.430667+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:02.072485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556187+0000","last_change":"2023-10-08T05:56:32.455106+0000","last_active":"2023-10-08T05:56:33.556187+0000","last_peered":"2023-10-08T05:56:33.556187+0000","last_clean":"2023-10-08T05:56:33.556187+0000","last_became_active":"2023-10-08T05:56:32.454813+0000","last_became_peered":"2023-10-08T05:56:32.454813+0000","last_unstale":"2023-10-08T05:56:33.556187+0000","last_undegraded":"2023-10-08T05:56:33.556187+0000","last_fullsized":"2023-10-08T05:56:33.556187+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:51.004877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430871+0000","last_change":"2023-10-08T05:56:31.935623+0000","last_active":"2023-10-08T05:56:33.430871+0000","last_peered":"2023-10-08T05:56:33.430871+0000","last_clean":"2023-10-08T05:56:33.430871+0000","last_became_active":"2023-10-08T05:56:31.935353+0000","last_became_peered":"2023-10-08T05:56:31.935353+0000","last_unstale":"2023-10-08T05:56:33.430871+0000","last_undegraded":"2023-10-08T05:56:33.430871+0000","last_fullsized":"2023-10-08T05:56:33.430871+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:03.562715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711259+0000","last_change":"2023-10-08T05:56:32.437421+0000","last_active":"2023-10-08T05:56:33.711259+0000","last_peered":"2023-10-08T05:56:33.711259+0000","last_clean":"2023-10-08T05:56:33.711259+0000","last_became_active":"2023-10-08T05:56:32.437105+0000","last_became_peered":"2023-10-08T05:56:32.437105+0000","last_unstale":"2023-10-08T05:56:33.711259+0000","last_undegraded":"2023-10-08T05:56:33.711259+0000","last_fullsized":"2023-10-08T05:56:33.711259+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:13.807001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430597+0000","last_change":"2023-10-08T05:56:31.933863+0000","last_active":"2023-10-08T05:56:33.430597+0000","last_peered":"2023-10-08T05:56:33.430597+0000","last_clean":"2023-10-08T05:56:33.430597+0000","last_became_active":"2023-10-08T05:56:31.933568+0000","last_became_peered":"2023-10-08T05:56:31.933568+0000","last_unstale":"2023-10-08T05:56:33.430597+0000","last_undegraded":"2023-10-08T05:56:33.430597+0000","last_fullsized":"2023-10-08T05:56:33.430597+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:03.892658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434579+0000","last_change":"2023-10-08T05:56:32.452597+0000","last_active":"2023-10-08T05:56:33.434579+0000","last_peered":"2023-10-08T05:56:33.434579+0000","last_clean":"2023-10-08T05:56:33.434579+0000","last_became_active":"2023-10-08T05:56:32.452250+0000","last_became_peered":"2023-10-08T05:56:32.452250+0000","last_unstale":"2023-10-08T05:56:33.434579+0000","last_undegraded":"2023-10-08T05:56:33.434579+0000","last_fullsized":"2023-10-08T05:56:33.434579+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:02:14.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"148'1","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556596+0000","last_change":"2023-10-08T05:56:31.926357+0000","last_active":"2023-10-08T05:56:33.556596+0000","last_peered":"2023-10-08T05:56:33.556596+0000","last_clean":"2023-10-08T05:56:33.556596+0000","last_became_active":"2023-10-08T05:56:31.926101+0000","last_became_peered":"2023-10-08T05:56:31.926101+0000","last_unstale":"2023-10-08T05:56:33.556596+0000","last_undegraded":"2023-10-08T05:56:33.556596+0000","last_fullsized":"2023-10-08T05:56:33.556596+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:23:08.849738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713874+0000","last_change":"2023-10-08T05:56:32.448049+0000","last_active":"2023-10-08T05:56:33.713874+0000","last_peered":"2023-10-08T05:56:33.713874+0000","last_clean":"2023-10-08T05:56:33.713874+0000","last_became_active":"2023-10-08T05:56:32.447747+0000","last_became_peered":"2023-10-08T05:56:32.447747+0000","last_unstale":"2023-10-08T05:56:33.713874+0000","last_undegraded":"2023-10-08T05:56:33.713874+0000","last_fullsized":"2023-10-08T05:56:33.713874+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:54.769675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711508+0000","last_change":"2023-10-08T05:56:31.918523+0000","last_active":"2023-10-08T05:56:33.711508+0000","last_peered":"2023-10-08T05:56:33.711508+0000","last_clean":"2023-10-08T05:56:33.711508+0000","last_became_active":"2023-10-08T05:56:31.918035+0000","last_became_peered":"2023-10-08T05:56:31.918035+0000","last_unstale":"2023-10-08T05:56:33.711508+0000","last_undegraded":"2023-10-08T05:56:33.711508+0000","last_fullsized":"2023-10-08T05:56:33.711508+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:54.285931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712758+0000","last_change":"2023-10-08T05:56:32.449279+0000","last_active":"2023-10-08T05:56:33.712758+0000","last_peered":"2023-10-08T05:56:33.712758+0000","last_clean":"2023-10-08T05:56:33.712758+0000","last_became_active":"2023-10-08T05:56:32.448971+0000","last_became_peered":"2023-10-08T05:56:32.448971+0000","last_unstale":"2023-10-08T05:56:33.712758+0000","last_undegraded":"2023-10-08T05:56:33.712758+0000","last_fullsized":"2023-10-08T05:56:33.712758+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:46:03.339371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"148'1","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712753+0000","last_change":"2023-10-08T05:56:31.935961+0000","last_active":"2023-10-08T05:56:33.712753+0000","last_peered":"2023-10-08T05:56:33.712753+0000","last_clean":"2023-10-08T05:56:33.712753+0000","last_became_active":"2023-10-08T05:56:31.935636+0000","last_became_peered":"2023-10-08T05:56:31.935636+0000","last_unstale":"2023-10-08T05:56:33.712753+0000","last_undegraded":"2023-10-08T05:56:33.712753+0000","last_fullsized":"2023-10-08T05:56:33.712753+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:03.015840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.435492+0000","last_change":"2023-10-08T05:56:32.462843+0000","last_active":"2023-10-08T05:56:33.435492+0000","last_peered":"2023-10-08T05:56:33.435492+0000","last_clean":"2023-10-08T05:56:33.435492+0000","last_became_active":"2023-10-08T05:56:32.462424+0000","last_became_peered":"2023-10-08T05:56:32.462424+0000","last_unstale":"2023-10-08T05:56:33.435492+0000","last_undegraded":"2023-10-08T05:56:33.435492+0000","last_fullsized":"2023-10-08T05:56:33.435492+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:16.824353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712417+0000","last_change":"2023-10-08T05:56:31.934213+0000","last_active":"2023-10-08T05:56:33.712417+0000","last_peered":"2023-10-08T05:56:33.712417+0000","last_clean":"2023-10-08T05:56:33.712417+0000","last_became_active":"2023-10-08T05:56:31.933893+0000","last_became_peered":"2023-10-08T05:56:31.933893+0000","last_unstale":"2023-10-08T05:56:33.712417+0000","last_undegraded":"2023-10-08T05:56:33.712417+0000","last_fullsized":"2023-10-08T05:56:33.712417+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:30.797452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712253+0000","last_change":"2023-10-08T05:56:32.448139+0000","last_active":"2023-10-08T05:56:33.712253+0000","last_peered":"2023-10-08T05:56:33.712253+0000","last_clean":"2023-10-08T05:56:33.712253+0000","last_became_active":"2023-10-08T05:56:32.447863+0000","last_became_peered":"2023-10-08T05:56:32.447863+0000","last_unstale":"2023-10-08T05:56:33.712253+0000","last_undegraded":"2023-10-08T05:56:33.712253+0000","last_fullsized":"2023-10-08T05:56:33.712253+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:10.698762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711546+0000","last_change":"2023-10-08T05:56:31.927544+0000","last_active":"2023-10-08T05:56:33.711546+0000","last_peered":"2023-10-08T05:56:33.711546+0000","last_clean":"2023-10-08T05:56:33.711546+0000","last_became_active":"2023-10-08T05:56:31.926241+0000","last_became_peered":"2023-10-08T05:56:31.926241+0000","last_unstale":"2023-10-08T05:56:33.711546+0000","last_undegraded":"2023-10-08T05:56:33.711546+0000","last_fullsized":"2023-10-08T05:56:33.711546+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:35.146970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430299+0000","last_change":"2023-10-08T05:56:32.446323+0000","last_active":"2023-10-08T05:56:33.430299+0000","last_peered":"2023-10-08T05:56:33.430299+0000","last_clean":"2023-10-08T05:56:33.430299+0000","last_became_active":"2023-10-08T05:56:32.445929+0000","last_became_peered":"2023-10-08T05:56:32.445929+0000","last_unstale":"2023-10-08T05:56:33.430299+0000","last_undegraded":"2023-10-08T05:56:33.430299+0000","last_fullsized":"2023-10-08T05:56:33.430299+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:15.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712074+0000","last_change":"2023-10-08T05:56:31.934037+0000","last_active":"2023-10-08T05:56:33.712074+0000","last_peered":"2023-10-08T05:56:33.712074+0000","last_clean":"2023-10-08T05:56:33.712074+0000","last_became_active":"2023-10-08T05:56:31.933728+0000","last_became_peered":"2023-10-08T05:56:31.933728+0000","last_unstale":"2023-10-08T05:56:33.712074+0000","last_undegraded":"2023-10-08T05:56:33.712074+0000","last_fullsized":"2023-10-08T05:56:33.712074+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:23.859616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430740+0000","last_change":"2023-10-08T05:56:32.448871+0000","last_active":"2023-10-08T05:56:33.430740+0000","last_peered":"2023-10-08T05:56:33.430740+0000","last_clean":"2023-10-08T05:56:33.430740+0000","last_became_active":"2023-10-08T05:56:32.448603+0000","last_became_peered":"2023-10-08T05:56:32.448603+0000","last_unstale":"2023-10-08T05:56:33.430740+0000","last_undegraded":"2023-10-08T05:56:33.430740+0000","last_fullsized":"2023-10-08T05:56:33.430740+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:18.147979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430855+0000","last_change":"2023-10-08T05:56:31.923001+0000","last_active":"2023-10-08T05:56:33.430855+0000","last_peered":"2023-10-08T05:56:33.430855+0000","last_clean":"2023-10-08T05:56:33.430855+0000","last_became_active":"2023-10-08T05:56:31.922613+0000","last_became_peered":"2023-10-08T05:56:31.922613+0000","last_unstale":"2023-10-08T05:56:33.430855+0000","last_undegraded":"2023-10-08T05:56:33.430855+0000","last_fullsized":"2023-10-08T05:56:33.430855+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:29.417867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434421+0000","last_change":"2023-10-08T05:56:32.453683+0000","last_active":"2023-10-08T05:56:33.434421+0000","last_peered":"2023-10-08T05:56:33.434421+0000","last_clean":"2023-10-08T05:56:33.434421+0000","last_became_active":"2023-10-08T05:56:32.453413+0000","last_became_peered":"2023-10-08T05:56:32.453413+0000","last_unstale":"2023-10-08T05:56:33.434421+0000","last_undegraded":"2023-10-08T05:56:33.434421+0000","last_fullsized":"2023-10-08T05:56:33.434421+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:17:44.483556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711501+0000","last_change":"2023-10-08T05:56:31.931739+0000","last_active":"2023-10-08T05:56:33.711501+0000","last_peered":"2023-10-08T05:56:33.711501+0000","last_clean":"2023-10-08T05:56:33.711501+0000","last_became_active":"2023-10-08T05:56:31.931458+0000","last_became_peered":"2023-10-08T05:56:31.931458+0000","last_unstale":"2023-10-08T05:56:33.711501+0000","last_undegraded":"2023-10-08T05:56:33.711501+0000","last_fullsized":"2023-10-08T05:56:33.711501+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:29.417867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713238+0000","last_change":"2023-10-08T05:56:32.449255+0000","last_active":"2023-10-08T05:56:33.713238+0000","last_peered":"2023-10-08T05:56:33.713238+0000","last_clean":"2023-10-08T05:56:33.713238+0000","last_became_active":"2023-10-08T05:56:32.448887+0000","last_became_peered":"2023-10-08T05:56:32.448887+0000","last_unstale":"2023-10-08T05:56:33.713238+0000","last_undegraded":"2023-10-08T05:56:33.713238+0000","last_fullsized":"2023-10-08T05:56:33.713238+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:17.632008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"148'2","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712079+0000","last_change":"2023-10-08T05:56:31.935501+0000","last_active":"2023-10-08T05:56:33.712079+0000","last_peered":"2023-10-08T05:56:33.712079+0000","last_clean":"2023-10-08T05:56:33.712079+0000","last_became_active":"2023-10-08T05:56:31.935207+0000","last_became_peered":"2023-10-08T05:56:31.935207+0000","last_unstale":"2023-10-08T05:56:33.712079+0000","last_undegraded":"2023-10-08T05:56:33.712079+0000","last_fullsized":"2023-10-08T05:56:33.712079+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:10.767246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556202+0000","last_change":"2023-10-08T05:56:32.453536+0000","last_active":"2023-10-08T05:56:33.556202+0000","last_peered":"2023-10-08T05:56:33.556202+0000","last_clean":"2023-10-08T05:56:33.556202+0000","last_became_active":"2023-10-08T05:56:32.452803+0000","last_became_peered":"2023-10-08T05:56:32.452803+0000","last_unstale":"2023-10-08T05:56:33.556202+0000","last_undegraded":"2023-10-08T05:56:33.556202+0000","last_fullsized":"2023-10-08T05:56:33.556202+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:53.404295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.434674+0000","last_change":"2023-10-08T05:56:31.926227+0000","last_active":"2023-10-08T05:56:33.434674+0000","last_peered":"2023-10-08T05:56:33.434674+0000","last_clean":"2023-10-08T05:56:33.434674+0000","last_became_active":"2023-10-08T05:56:31.925924+0000","last_became_peered":"2023-10-08T05:56:31.925924+0000","last_unstale":"2023-10-08T05:56:33.434674+0000","last_undegraded":"2023-10-08T05:56:33.434674+0000","last_fullsized":"2023-10-08T05:56:33.434674+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:41.694476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712780+0000","last_change":"2023-10-08T05:56:32.460761+0000","last_active":"2023-10-08T05:56:33.712780+0000","last_peered":"2023-10-08T05:56:33.712780+0000","last_clean":"2023-10-08T05:56:33.712780+0000","last_became_active":"2023-10-08T05:56:32.460457+0000","last_became_peered":"2023-10-08T05:56:32.460457+0000","last_unstale":"2023-10-08T05:56:33.712780+0000","last_undegraded":"2023-10-08T05:56:33.712780+0000","last_fullsized":"2023-10-08T05:56:33.712780+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:02:15.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.433544+0000","last_change":"2023-10-08T05:56:31.926931+0000","last_active":"2023-10-08T05:56:33.433544+0000","last_peered":"2023-10-08T05:56:33.433544+0000","last_clean":"2023-10-08T05:56:33.433544+0000","last_became_active":"2023-10-08T05:56:31.926630+0000","last_became_peered":"2023-10-08T05:56:31.926630+0000","last_unstale":"2023-10-08T05:56:33.433544+0000","last_undegraded":"2023-10-08T05:56:33.433544+0000","last_fullsized":"2023-10-08T05:56:33.433544+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:12.316588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711895+0000","last_change":"2023-10-08T05:56:32.447806+0000","last_active":"2023-10-08T05:56:33.711895+0000","last_peered":"2023-10-08T05:56:33.711895+0000","last_clean":"2023-10-08T05:56:33.711895+0000","last_became_active":"2023-10-08T05:56:32.447519+0000","last_became_peered":"2023-10-08T05:56:32.447519+0000","last_unstale":"2023-10-08T05:56:33.711895+0000","last_undegraded":"2023-10-08T05:56:33.711895+0000","last_fullsized":"2023-10-08T05:56:33.711895+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:14.407721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713067+0000","last_change":"2023-10-08T05:56:31.932972+0000","last_active":"2023-10-08T05:56:33.713067+0000","last_peered":"2023-10-08T05:56:33.713067+0000","last_clean":"2023-10-08T05:56:33.713067+0000","last_became_active":"2023-10-08T05:56:31.932442+0000","last_became_peered":"2023-10-08T05:56:31.932442+0000","last_unstale":"2023-10-08T05:56:33.713067+0000","last_undegraded":"2023-10-08T05:56:33.713067+0000","last_fullsized":"2023-10-08T05:56:33.713067+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:47.910691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712047+0000","last_change":"2023-10-08T05:56:32.446571+0000","last_active":"2023-10-08T05:56:33.712047+0000","last_peered":"2023-10-08T05:56:33.712047+0000","last_clean":"2023-10-08T05:56:33.712047+0000","last_became_active":"2023-10-08T05:56:32.446300+0000","last_became_peered":"2023-10-08T05:56:32.446300+0000","last_unstale":"2023-10-08T05:56:33.712047+0000","last_undegraded":"2023-10-08T05:56:33.712047+0000","last_fullsized":"2023-10-08T05:56:33.712047+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:29.549230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712849+0000","last_change":"2023-10-08T05:56:31.930616+0000","last_active":"2023-10-08T05:56:33.712849+0000","last_peered":"2023-10-08T05:56:33.712849+0000","last_clean":"2023-10-08T05:56:33.712849+0000","last_became_active":"2023-10-08T05:56:31.930331+0000","last_became_peered":"2023-10-08T05:56:31.930331+0000","last_unstale":"2023-10-08T05:56:33.712849+0000","last_undegraded":"2023-10-08T05:56:33.712849+0000","last_fullsized":"2023-10-08T05:56:33.712849+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:55.180154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430635+0000","last_change":"2023-10-08T05:56:32.452869+0000","last_active":"2023-10-08T05:56:33.430635+0000","last_peered":"2023-10-08T05:56:33.430635+0000","last_clean":"2023-10-08T05:56:33.430635+0000","last_became_active":"2023-10-08T05:56:32.452595+0000","last_became_peered":"2023-10-08T05:56:32.452595+0000","last_unstale":"2023-10-08T05:56:33.430635+0000","last_undegraded":"2023-10-08T05:56:33.430635+0000","last_fullsized":"2023-10-08T05:56:33.430635+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:02:14.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.713217+0000","last_change":"2023-10-08T05:56:31.937769+0000","last_active":"2023-10-08T05:56:33.713217+0000","last_peered":"2023-10-08T05:56:33.713217+0000","last_clean":"2023-10-08T05:56:33.713217+0000","last_became_active":"2023-10-08T05:56:31.937473+0000","last_became_peered":"2023-10-08T05:56:31.937473+0000","last_unstale":"2023-10-08T05:56:33.713217+0000","last_undegraded":"2023-10-08T05:56:33.713217+0000","last_fullsized":"2023-10-08T05:56:33.713217+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:30.482050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711809+0000","last_change":"2023-10-08T05:56:32.461354+0000","last_active":"2023-10-08T05:56:33.711809+0000","last_peered":"2023-10-08T05:56:33.711809+0000","last_clean":"2023-10-08T05:56:33.711809+0000","last_became_active":"2023-10-08T05:56:32.460957+0000","last_became_peered":"2023-10-08T05:56:32.460957+0000","last_unstale":"2023-10-08T05:56:33.711809+0000","last_undegraded":"2023-10-08T05:56:33.711809+0000","last_fullsized":"2023-10-08T05:56:33.711809+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:12.187021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712236+0000","last_change":"2023-10-08T05:56:31.928922+0000","last_active":"2023-10-08T05:56:33.712236+0000","last_peered":"2023-10-08T05:56:33.712236+0000","last_clean":"2023-10-08T05:56:33.712236+0000","last_became_active":"2023-10-08T05:56:31.928654+0000","last_became_peered":"2023-10-08T05:56:31.928654+0000","last_unstale":"2023-10-08T05:56:33.712236+0000","last_undegraded":"2023-10-08T05:56:33.712236+0000","last_fullsized":"2023-10-08T05:56:33.712236+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:57:16.465352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.556415+0000","last_change":"2023-10-08T05:56:32.455032+0000","last_active":"2023-10-08T05:56:33.556415+0000","last_peered":"2023-10-08T05:56:33.556415+0000","last_clean":"2023-10-08T05:56:33.556415+0000","last_became_active":"2023-10-08T05:56:32.454758+0000","last_became_peered":"2023-10-08T05:56:32.454758+0000","last_unstale":"2023-10-08T05:56:33.556415+0000","last_undegraded":"2023-10-08T05:56:33.556415+0000","last_fullsized":"2023-10-08T05:56:33.556415+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:49.796506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.711901+0000","last_change":"2023-10-08T05:56:31.933166+0000","last_active":"2023-10-08T05:56:33.711901+0000","last_peered":"2023-10-08T05:56:33.711901+0000","last_clean":"2023-10-08T05:56:33.711901+0000","last_became_active":"2023-10-08T05:56:31.932722+0000","last_became_peered":"2023-10-08T05:56:31.932722+0000","last_unstale":"2023-10-08T05:56:33.711901+0000","last_undegraded":"2023-10-08T05:56:33.711901+0000","last_fullsized":"2023-10-08T05:56:33.711901+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:55:03.176910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.712881+0000","last_change":"2023-10-08T05:56:32.462277+0000","last_active":"2023-10-08T05:56:33.712881+0000","last_peered":"2023-10-08T05:56:33.712881+0000","last_clean":"2023-10-08T05:56:33.712881+0000","last_became_active":"2023-10-08T05:56:32.461935+0000","last_became_peered":"2023-10-08T05:56:32.461935+0000","last_unstale":"2023-10-08T05:56:33.712881+0000","last_undegraded":"2023-10-08T05:56:33.712881+0000","last_fullsized":"2023-10-08T05:56:33.712881+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:31.883768+0000","last_clean_scrub_stamp":"2023-10-08T05:56:31.883768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:35:27.034365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"148'8","reported_seq":20,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-10-08T05:56:33.430145+0000","last_change":"2023-10-08T05:56:31.931125+0000","last_active":"2023-10-08T05:56:33.430145+0000","last_peered":"2023-10-08T05:56:33.430145+0000","last_clean":"2023-10-08T05:56:33.430145+0000","last_became_active":"2023-10-08T05:56:31.930852+0000","last_became_peered":"2023-10-08T05:56:31.930852+0000","last_unstale":"2023-10-08T05:56:33.430145+0000","last_undegraded":"2023-10-08T05:56:33.430145+0000","last_fullsized":"2023-10-08T05:56:33.430145+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:56:30.867263+0000","last_clean_scrub_stamp":"2023-10-08T05:56:30.867263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:19:03.015840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"137'2685","reported_seq":3180,"reported_epoch":148,"state":"active+clean","last_fresh":"2023-10-08T05:56:32.442658+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:56:32.442658+0000","last_peered":"2023-10-08T05:56:32.442658+0000","last_clean":"2023-10-08T05:56:32.442658+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:56:32.442658+0000","last_undegraded":"2023-10-08T05:56:32.442658+0000","last_fullsized":"2023-10-08T05:56:32.442658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2685,"log_dups_size":0,"ondisk_log_size":2685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1149,"num_read_kb":2352,"num_write":369,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1149,"num_read_kb":2352,"num_write":369,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2685,"ondisk_log_size":2685,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575091,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55792,"kb_used_data":2404,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316048,"statfs":{"total":96636764160,"available":96579633152,"internally_reserved":0,"allocated":2461696,"data_stored":1536004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.123,"15min":1.1919999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.36199999999999999},"max":{"1min":1.629,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.165},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.1950000000000001,"15min":1.216},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.44},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.2050000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.2030000000000001,"15min":1.236},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.49199999999999999},"max":{"1min":1.536,"5min":1.8140000000000001,"15min":1.8360000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.2909999999999999,"15min":1.276},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.47099999999999997},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.425}]},{"osd":2,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.194,"15min":1.2330000000000001},"min":{"1min":0.745,"5min":0.71599999999999997,"15min":0.626},"max":{"1min":1.597,"5min":1.748,"15min":1.8939999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3140000000000001,"15min":1.323},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.45400000000000001},"max":{"1min":1.736,"5min":1.97,"15min":1.97},"last":1.46}]},{"osd":3,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.266,"15min":1.2789999999999999},"min":{"1min":0.84899999999999998,"5min":0.72399999999999998,"15min":0.69999999999999996},"max":{"1min":1.6140000000000001,"5min":1.639,"15min":1.8799999999999999},"last":1.008},{"interface":"front","average":{"1min":1.339,"5min":1.371,"15min":1.365},"min":{"1min":0.63,"5min":0.63,"15min":0.60199999999999998},"max":{"1min":1.746,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.82299999999999995}]},{"osd":4,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.363,"15min":1.3009999999999999},"min":{"1min":0.84399999999999997,"5min":0.76200000000000001,"15min":0.504},"max":{"1min":3.6200000000000001,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":1.079},{"interface":"front","average":{"1min":1.373,"5min":1.361,"15min":1.335},"min":{"1min":0.90600000000000003,"5min":0.81599999999999995,"15min":0.63600000000000001},"max":{"1min":1.954,"5min":2.0550000000000002,"15min":2.0699999999999998},"last":1.1200000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.4510000000000001,"15min":1.413},"min":{"1min":1.056,"5min":0.69499999999999995,"15min":0.53400000000000003},"max":{"1min":1.7989999999999999,"5min":2.0339999999999998,"15min":2.0339999999999998},"last":1.327},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.4550000000000001,"15min":1.423},"min":{"1min":1.0009999999999999,"5min":0.93600000000000005,"15min":0.39000000000000001},"max":{"1min":1.887,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.3069999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.5660000000000001,"15min":1.5269999999999999},"min":{"1min":1.24,"5min":1.0940000000000001,"15min":0.54400000000000004},"max":{"1min":1.9199999999999999,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4910000000000001,"15min":1.4630000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.57799999999999996},"max":{"1min":1.7230000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.355}]}]},{"osd":6,"up_from":13,"seq":55834575090,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37160,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1855488,"data_stored":945442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.2410000000000001,"15min":1.264},"min":{"1min":0.61899999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.486,"5min":2.1429999999999998,"15min":2.1429999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.306,"15min":1.302},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.58399999999999996},"max":{"1min":1.5880000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.3009999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1579999999999999,"15min":1.1599999999999999},"min":{"1min":0.60699999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7789999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.181,"15min":1.1830000000000001},"min":{"1min":0.78800000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6020000000000001,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.448}]},{"osd":2,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.1739999999999999,"15min":1.1799999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.5840000000000001,"5min":1.774,"15min":1.796},"last":1.8959999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.2110000000000001,"15min":1.2390000000000001},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.58699999999999997},"max":{"1min":1.661,"5min":1.752,"15min":1.8520000000000001},"last":1.1299999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3100000000000001,"15min":1.361},"min":{"1min":0.75600000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.607,"5min":2.137,"15min":2.137},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.419,"15min":1.4379999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.70499999999999996},"max":{"1min":1.6599999999999999,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.6850000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3400000000000001,"15min":1.321},"min":{"1min":0.97399999999999998,"5min":0.70399999999999996,"15min":0.60099999999999998},"max":{"1min":1.9399999999999999,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.601},{"interface":"front","average":{"1min":1.349,"5min":1.411,"15min":1.3839999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.64300000000000002},"max":{"1min":1.724,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.8240000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.349,"15min":1.327},"min":{"1min":0.92200000000000004,"5min":0.79500000000000004,"15min":0.69999999999999996},"max":{"1min":1.889,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.389},{"interface":"front","average":{"1min":1.417,"5min":1.512,"15min":1.4830000000000001},"min":{"1min":1.1200000000000001,"5min":0.87,"15min":0.747},"max":{"1min":1.6419999999999999,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.7370000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:55:36 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.546,"15min":1.5029999999999999},"min":{"1min":0.91200000000000003,"5min":0.82399999999999995,"15min":0.77200000000000002},"max":{"1min":2.024,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.9590000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.5069999999999999,"15min":1.4770000000000001},"min":{"1min":0.96499999999999997,"5min":0.85999999999999999,"15min":0.81899999999999995},"max":{"1min":1.9850000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.5089999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575090,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":1859584,"data_stored":945020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.125,"15min":1.1259999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.52,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.212,"15min":1.204},"min":{"1min":0.84699999999999998,"5min":0.47899999999999998,"15min":0.442},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.83199999999999996}]},{"osd":1,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.1080000000000001,"15min":1.1930000000000001},"min":{"1min":0.746,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":2.5009999999999999,"5min":2.5009999999999999,"15min":3.2000000000000002},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2490000000000001,"15min":1.2589999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6319999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.371}]},{"osd":2,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.2609999999999999,"15min":1.234},"min":{"1min":0.90500000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.24,"15min":1.2809999999999999},"min":{"1min":0.88800000000000001,"5min":0.71399999999999997,"15min":0.65000000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":3.282},"last":1.075}]},{"osd":3,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3140000000000001,"15min":1.3360000000000001},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.63500000000000001},"max":{"1min":1.526,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.633},{"interface":"front","average":{"1min":1.427,"5min":1.389,"15min":1.3959999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.9370000000000001,"5min":1.9370000000000001,"15min":2.2280000000000002},"last":1.583}]},{"osd":4,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.323,"15min":1.306},"min":{"1min":0.70799999999999996,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":4.0179999999999998},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.4330000000000001,"15min":1.4299999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8440000000000001,"5min":2.927,"15min":3.456},"last":1.764}]},{"osd":6,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.444,"15min":1.4059999999999999},"min":{"1min":0.79000000000000004,"5min":0.751,"15min":0.751},"max":{"1min":2.149,"5min":2.149,"15min":2.359},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.484,"15min":1.4690000000000001},"min":{"1min":0.77300000000000002,"5min":0.623,"15min":0.58099999999999996},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":3.8620000000000001},"last":1.673}]},{"osd":7,"last update":"Sun Oct 8 05:55:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.538,"15min":1.54},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.829,"5min":3.0270000000000001,"15min":3.4079999999999999},"last":1.845},{"interface":"front","average":{"1min":1.599,"5min":1.5960000000000001,"15min":1.5609999999999999},"min":{"1min":0.73999999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.8169999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575090,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37160,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334680,"statfs":{"total":96636764160,"available":96598712320,"internally_reserved":0,"allocated":1863680,"data_stored":946472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2090000000000001,"15min":1.26},"min":{"1min":0.73299999999999998,"5min":0.70399999999999996,"15min":0.498},"max":{"1min":1.6140000000000001,"5min":1.7609999999999999,"15min":1.8149999999999999},"last":1.76},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3129999999999999,"15min":1.361},"min":{"1min":0.84999999999999998,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.6990000000000001,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.165}]},{"osd":1,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.274,"15min":1.2749999999999999},"min":{"1min":0.82099999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.6559999999999999,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.60099999999999998},"max":{"1min":1.6559999999999999,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":0.93300000000000005}]},{"osd":2,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.2170000000000001,"15min":1.228},"min":{"1min":0.748,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.542,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.018},{"interface":"front","average":{"1min":1.204,"5min":1.2949999999999999,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.622,"15min":0.622},"max":{"1min":1.7569999999999999,"5min":2.431,"15min":2.431},"last":1.284}]},{"osd":3,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2669999999999999,"15min":1.276},"min":{"1min":0.81000000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7290000000000001,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.222},{"interface":"front","average":{"1min":1.381,"5min":1.3320000000000001,"15min":1.3400000000000001},"min":{"1min":1.091,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.7470000000000001,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.341}]},{"osd":5,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3720000000000001,"15min":1.341},"min":{"1min":0.97699999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6910000000000001,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.393},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.385,"15min":1.363},"min":{"1min":0.99199999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.786,"5min":5.4189999999999996,"15min":5.4189999999999996},"last":1.075}]},{"osd":6,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.4419999999999999,"15min":1.363},"min":{"1min":0.83999999999999997,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8069999999999999,"5min":4.9059999999999997,"15min":4.9059999999999997},"last":1.9239999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5680000000000001,"15min":1.5109999999999999},"min":{"1min":1.1559999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.8759999999999999,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":1.51}]},{"osd":7,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.518,"15min":1.4770000000000001},"min":{"1min":1.133,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.792,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":1.446},{"interface":"front","average":{"1min":1.597,"5min":1.5569999999999999,"15min":1.522},"min":{"1min":1.0669999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.8899999999999999,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":2.016}]}]},{"osd":3,"up_from":13,"seq":55834575092,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41324,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330516,"statfs":{"total":96636764160,"available":96594448384,"internally_reserved":0,"allocated":1867776,"data_stored":946506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88300000000000001,"15min":0.91700000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.615,"5min":1.615,"15min":1.72},"last":1.8089999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.0549999999999999,"15min":1.093},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.494,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.772}]},{"osd":1,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92000000000000004,"15min":0.95799999999999996},"min":{"1min":0.57099999999999995,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.349,"5min":1.605,"15min":1.605},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.161,"5min":1.1519999999999999,"15min":1.1970000000000001},"min":{"1min":0.56100000000000005,"5min":0.433,"15min":0.433},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.7210000000000001},"last":1.397}]},{"osd":2,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0069999999999999,"15min":1.0529999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.736,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.149},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0900000000000001,"15min":1.1319999999999999},"min":{"1min":0.499,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.1040000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1899999999999999,"15min":1.2110000000000001},"min":{"1min":0.76200000000000001,"5min":0.68999999999999995,"15min":0.54000000000000004},"max":{"1min":1.647,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3620000000000001,"15min":1.379},"min":{"1min":0.96399999999999997,"5min":0.82699999999999996,"15min":0.72799999999999998},"max":{"1min":1.8460000000000001,"5min":2.589,"15min":2.589},"last":1.575}]},{"osd":5,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.2889999999999999,"15min":1.2949999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.3109999999999999,"15min":1.343},"min":{"1min":0.81100000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.6850000000000001,"5min":2.0939999999999999,"15min":2.0939999999999999},"last":1.5369999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3420000000000001,"15min":1.3220000000000001},"min":{"1min":0.871,"5min":0.65700000000000003,"15min":0.61799999999999999},"max":{"1min":1.7490000000000001,"5min":1.841,"15min":1.841},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4330000000000001,"15min":1.4379999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.86699999999999999},"max":{"1min":2.0150000000000001,"5min":2.016,"15min":2.0489999999999999},"last":1.694}]},{"osd":7,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.458,"15min":1.482},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.982,"5min":1.982,"15min":2.1480000000000001},"last":1.659},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.379,"15min":1.3859999999999999},"min":{"1min":0.92800000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8380000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.6200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575090,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59364,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312476,"statfs":{"total":96636764160,"available":96575975424,"internally_reserved":0,"allocated":1851392,"data_stored":944860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.79700000000000004,"15min":0.86899999999999999},"min":{"1min":0.48399999999999999,"5min":0.255,"15min":0.255},"max":{"1min":1.526,"5min":1.629,"15min":1.629},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.96199999999999997,"15min":1.0249999999999999},"min":{"1min":0.52300000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.363,"5min":1.5880000000000001,"15min":1.9470000000000001},"last":0.57299999999999995}]},{"osd":1,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.86899999999999999,"15min":0.93600000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.42199999999999999},"max":{"1min":1.173,"5min":1.7609999999999999,"15min":2.1070000000000002},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.98699999999999999,"15min":1.0680000000000001},"min":{"1min":0.54400000000000004,"5min":0.47799999999999998,"15min":0.35099999999999998},"max":{"1min":1.417,"5min":1.79,"15min":2.1469999999999998},"last":0.73499999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.0449999999999999,"15min":1.1080000000000001},"min":{"1min":0.55200000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.349,"5min":1.631,"15min":1.8120000000000001},"last":1.002},{"interface":"front","average":{"1min":1.153,"5min":1.1020000000000001,"15min":1.206},"min":{"1min":0.66300000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.847,"5min":1.8859999999999999,"15min":7.5890000000000004},"last":1.048}]},{"osd":4,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.2929999999999999,"15min":1.367},"min":{"1min":0.95799999999999996,"5min":0.66000000000000003,"15min":0.622},"max":{"1min":1.7470000000000001,"5min":1.8280000000000001,"15min":4.593},"last":1.145},{"interface":"front","average":{"1min":1.244,"5min":1.2310000000000001,"15min":1.28},"min":{"1min":0.96799999999999997,"5min":0.624,"15min":0.624},"max":{"1min":1.869,"5min":1.881,"15min":6.6879999999999997},"last":1.1100000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.258,"15min":1.359},"min":{"1min":0.88400000000000001,"5min":0.68799999999999994,"15min":0.63900000000000001},"max":{"1min":1.885,"5min":1.885,"15min":6.7140000000000004},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.361,"15min":1.4339999999999999},"min":{"1min":0.88800000000000001,"5min":0.71999999999999997,"15min":0.56599999999999995},"max":{"1min":1.8120000000000001,"5min":1.9359999999999999,"15min":2.4119999999999999},"last":1.226}]},{"osd":6,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.3280000000000001,"15min":1.3460000000000001},"min":{"1min":0.93000000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.7080000000000002},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3300000000000001,"15min":1.431},"min":{"1min":0.91400000000000003,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7929999999999999,"5min":1.8400000000000001,"15min":2.8839999999999999},"last":1.2869999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:55:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.355,"15min":1.425},"min":{"1min":0.995,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.956,"5min":1.956,"15min":6.609},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.379,"15min":1.478},"min":{"1min":0.91800000000000004,"5min":0.877,"15min":0.82299999999999995},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.9950000000000001},"last":1.3680000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575090,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37152,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334688,"statfs":{"total":96636764160,"available":96598720512,"internally_reserved":0,"allocated":1855488,"data_stored":944882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.91400000000000003,"15min":0.96899999999999997},"min":{"1min":0.48499999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.3010000000000002,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.0349999999999999,"15min":1.1140000000000001},"min":{"1min":0.65500000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.2370000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.873,"15min":0.94099999999999995},"min":{"1min":0.52500000000000002,"5min":0.45100000000000001,"15min":0.39500000000000002},"max":{"1min":1.377,"5min":1.377,"15min":1.8240000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.3,"5min":1.0629999999999999,"15min":1.1000000000000001},"min":{"1min":0.50900000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.0030000000000001,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.087,"15min":1.169},"min":{"1min":0.60299999999999998,"5min":0.498,"15min":0.498},"max":{"1min":2.6629999999999998,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.032,"5min":1.026,"15min":1.071},"min":{"1min":0.58799999999999997,"5min":0.432,"15min":0.432},"max":{"1min":1.5469999999999999,"5min":1.6120000000000001,"15min":1.788},"last":1.0880000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.22,"15min":1.2889999999999999},"min":{"1min":0.874,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.891,"5min":1.891,"15min":1.966},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1839999999999999,"15min":1.22},"min":{"1min":0.78200000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":1.3180000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2669999999999999,"15min":1.304},"min":{"1min":0.80400000000000005,"5min":0.73199999999999998,"15min":0.70399999999999996},"max":{"1min":1.786,"5min":1.7909999999999999,"15min":1.8680000000000001},"last":1.395},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.411,"15min":1.452},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.226,"5min":2.226,"15min":2.226},"last":1.613}]},{"osd":6,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.361,"15min":1.4139999999999999},"min":{"1min":1.0589999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.833},{"interface":"front","average":{"1min":1.365,"5min":1.357,"15min":1.4099999999999999},"min":{"1min":1.032,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6839999999999999,"5min":1.8580000000000001,"15min":2.8559999999999999},"last":1.5129999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:55:39 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.395,"15min":1.4590000000000001},"min":{"1min":0.92400000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.365},{"interface":"front","average":{"1min":1.488,"5min":1.3939999999999999,"15min":1.4470000000000001},"min":{"1min":1.119,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.893,"5min":2.0819999999999999,"15min":2.306},"last":1.9339999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575091,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33580,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338260,"statfs":{"total":96636764160,"available":96602378240,"internally_reserved":0,"allocated":2449408,"data_stored":1535250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.876,"15min":0.90500000000000003},"min":{"1min":0.38700000000000001,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":1.665,"5min":1.77,"15min":1.77},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.879,"5min":0.98199999999999998,"15min":1.0229999999999999},"min":{"1min":0.41199999999999998,"5min":0.40899999999999997,"15min":0.371},"max":{"1min":1.337,"5min":1.6719999999999999,"15min":1.675},"last":0.753}]},{"osd":2,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0249999999999999,"15min":1.0740000000000001},"min":{"1min":0.435,"5min":0.42599999999999999,"15min":0.39200000000000002},"max":{"1min":1.5740000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.022},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.92200000000000004,"15min":0.97799999999999998},"min":{"1min":0.47599999999999998,"5min":0.47399999999999998,"15min":0.40400000000000003},"max":{"1min":1.5880000000000001,"5min":1.722,"15min":1.78},"last":1.175}]},{"osd":3,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.133,"15min":1.1819999999999999},"min":{"1min":0.52300000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.823,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0429999999999999,"15min":1.083},"min":{"1min":0.51600000000000001,"5min":0.46500000000000002,"15min":0.44700000000000001},"max":{"1min":1.734,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.121}]},{"osd":4,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2210000000000001,"15min":1.256},"min":{"1min":0.78100000000000003,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.5860000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.298},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3220000000000001,"15min":1.337},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.962,"5min":1.962,"15min":2.2200000000000002},"last":1.397}]},{"osd":5,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.2929999999999999,"15min":1.3260000000000001},"min":{"1min":0.61199999999999999,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8100000000000001,"5min":1.9410000000000001,"15min":2.0489999999999999},"last":1.488},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.365,"15min":1.3720000000000001},"min":{"1min":0.74099999999999999,"5min":0.71199999999999997,"15min":0.63},"max":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":1.0649999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3720000000000001,"15min":1.405},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.58499999999999996},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.353,"15min":1.3819999999999999},"min":{"1min":0.73399999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.7050000000000001,"5min":2.1099999999999999,"15min":2.1099999999999999},"last":1.369}]},{"osd":7,"last update":"Sun Oct 8 05:55:47 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3939999999999999,"15min":1.4219999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.63800000000000001},"max":{"1min":1.8300000000000001,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.4299999999999999,"15min":1.4650000000000001},"min":{"1min":0.76500000000000001,"5min":0.73699999999999999,"15min":0.54700000000000004},"max":{"1min":1.7649999999999999,"5min":2.0569999999999999,"15min":2.2759999999999998},"last":1.655}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:56:37.890 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T05:56:37.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T05:56:38.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.153+0000 7f55cc4a2640 1 Processor -- start 2023-10-08T05:56:38.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.153+0000 7f55cc4a2640 1 -- start start 2023-10-08T05:56:38.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55cc4a2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55c406cfa0 0x7f55c415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55cc4a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415b630 0x7f55c4070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55cc4a2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c40c8c60 con 0x7f55c40cb500 2023-10-08T05:56:38.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55cc4a2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c40b1ac0 con 0x7f55c406cfa0 2023-10-08T05:56:38.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55cc4a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40b1da0 con 0x7f55c415b630 2023-10-08T05:56:38.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55c9a3d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55c406cfa0 0x7f55c415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55c9a3d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f55c406cfa0 0x7f55c415b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41436/0 (socket says 172.21.15.70:41436) 2023-10-08T05:56:38.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55c9a3d640 1 -- 172.21.15.70:0/3819988594 learned_addr learned my addr 172.21.15.70:0/3819988594 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:38.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55caa3f640 1 --2- 172.21.15.70:0/3819988594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415b630 0x7f55c4070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55c9a3d640 1 -- 172.21.15.70:0/3819988594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415b630 msgr2=0x7f55c4070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:38.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55c9a3d640 1 --2- 172.21.15.70:0/3819988594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415b630 0x7f55c4070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.155+0000 7f55c9a3d640 1 -- 172.21.15.70:0/3819988594 >> v1:172.21.15.181:6789/0 conn(0x7f55c40cb500 legacy=0x7f55c40cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:38.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.156+0000 7f55c9a3d640 1 -- 172.21.15.70:0/3819988594 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c40b2080 con 0x7f55c406cfa0 2023-10-08T05:56:38.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.157+0000 7f55c9a3d640 1 --2- 172.21.15.70:0/3819988594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c406cfa0 0x7f55c415b0f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009870 tx=0x7f55b402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c8072edf812f9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:38.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.157+0000 7f55c923c640 1 -- 172.21.15.70:0/3819988594 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40317c0 con 0x7f55c406cfa0 2023-10-08T05:56:38.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.157+0000 7f55c923c640 1 -- 172.21.15.70:0/3819988594 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b4031920 con 0x7f55c406cfa0 2023-10-08T05:56:38.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.157+0000 7f55c923c640 1 -- 172.21.15.70:0/3819988594 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b4031cb0 con 0x7f55c406cfa0 2023-10-08T05:56:38.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.158+0000 7f55cc4a2640 1 -- 172.21.15.70:0/3819988594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c406cfa0 msgr2=0x7f55c415b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.158+0000 7f55cc4a2640 1 --2- 172.21.15.70:0/3819988594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c406cfa0 0x7f55c415b0f0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009870 tx=0x7f55b402b860 comp rx=0 tx=0).stop 2023-10-08T05:56:38.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.158+0000 7f55cc4a2640 1 -- 172.21.15.70:0/3819988594 shutdown_connections 2023-10-08T05:56:38.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.158+0000 7f55cc4a2640 1 --2- 172.21.15.70:0/3819988594 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c415b630 0x7f55c4070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.158+0000 7f55cc4a2640 1 --2- 172.21.15.70:0/3819988594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c406cfa0 0x7f55c415b0f0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.158+0000 7f55cc4a2640 1 -- 172.21.15.70:0/3819988594 >> 172.21.15.70:0/3819988594 conn(0x7f55c406a140 msgr2=0x7f55c406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:38.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.158+0000 7f55cc4a2640 1 -- 172.21.15.70:0/3819988594 shutdown_connections 2023-10-08T05:56:38.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.158+0000 7f55cc4a2640 1 -- 172.21.15.70:0/3819988594 wait complete. 2023-10-08T05:56:38.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.160+0000 7f55cc4a2640 1 Processor -- start 2023-10-08T05:56:38.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.160+0000 7f55cc4a2640 1 -- start start 2023-10-08T05:56:38.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.161+0000 7f55cc4a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c406cfa0 0x7f55c4153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.161+0000 7f55cc4a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c40cb500 0x7f55c40ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.161+0000 7f55cc4a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c415b630 0x7f55c40aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.161+0000 7f55cc4a2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40b4db0 con 0x7f55c415b630 2023-10-08T05:56:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.161+0000 7f55cc4a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40af580 con 0x7f55c406cfa0 2023-10-08T05:56:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.161+0000 7f55cc4a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c40af860 con 0x7f55c40cb500 2023-10-08T05:56:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.161+0000 7f55caa3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c415b630 0x7f55c40aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.161+0000 7f55caa3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c415b630 0x7f55c40aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41446/0 (socket says 172.21.15.70:41446) 2023-10-08T05:56:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.161+0000 7f55caa3f640 1 -- 172.21.15.70:0/1230668744 learned_addr learned my addr 172.21.15.70:0/1230668744 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55c9a3d640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c40cb500 0x7f55c40ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55ca23e640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c406cfa0 0x7f55c4153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55caa3f640 1 -- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c40cb500 msgr2=0x7f55c40ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55caa3f640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c40cb500 0x7f55c40ae930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55caa3f640 1 -- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c406cfa0 msgr2=0x7f55c4153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55caa3f640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c406cfa0 0x7f55c4153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55caa3f640 1 -- 172.21.15.70:0/1230668744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b40094d0 con 0x7f55c415b630 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55ca23e640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c406cfa0 0x7f55c4153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55caa3f640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c415b630 0x7f55c40aee70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00cc30 tx=0x7f55bc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55baffd640 1 -- 172.21.15.70:0/1230668744 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00ce40 con 0x7f55c415b630 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55baffd640 1 -- 172.21.15.70:0/1230668744 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55bc003990 con 0x7f55c415b630 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55baffd640 1 -- 172.21.15.70:0/1230668744 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc003d20 con 0x7f55c415b630 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55cc4a2640 1 -- 172.21.15.70:0/1230668744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c40afb40 con 0x7f55c415b630 2023-10-08T05:56:38.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.162+0000 7f55cc4a2640 1 -- 172.21.15.70:0/1230668744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c40b0100 con 0x7f55c415b630 2023-10-08T05:56:38.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.164+0000 7f55cc4a2640 1 -- 172.21.15.70:0/1230668744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c40b02f0 con 0x7f55c415b630 2023-10-08T05:56:38.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.165+0000 7f55baffd640 1 -- 172.21.15.70:0/1230668744 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f55bc003e80 con 0x7f55c415b630 2023-10-08T05:56:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.165+0000 7f55baffd640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f55a806af90 0x7f55a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.166+0000 7f55baffd640 1 -- 172.21.15.70:0/1230668744 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55bc089d90 con 0x7f55c415b630 2023-10-08T05:56:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.166+0000 7f55ca23e640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f55a806af90 0x7f55a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.167+0000 7f55ca23e640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f55a806af90 0x7f55a806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f55c0003eb0 tx=0x7f55c0009040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:38.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.170+0000 7f55baffd640 1 -- 172.21.15.70:0/1230668744 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55bc054b00 con 0x7f55c415b630 2023-10-08T05:56:38.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.425+0000 7f55cc4a2640 1 -- 172.21.15.70:0/1230668744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f55c4071cb0 con 0x7f55c415b630 2023-10-08T05:56:38.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.429+0000 7f55baffd640 1 -- 172.21.15.70:0/1230668744 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7f55bc01d070 con 0x7f55c415b630 2023-10-08T05:56:38.430 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T05:56:38.430 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T05:56:38.430 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T05:56:38.430 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T05:56:38.430 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T05:56:38.431 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T05:56:38.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 -- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f55a806af90 msgr2=0x7f55a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f55a806af90 0x7f55a806d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f55c0003eb0 tx=0x7f55c0009040 comp rx=0 tx=0).stop 2023-10-08T05:56:38.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 -- 172.21.15.70:0/1230668744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c415b630 msgr2=0x7f55c40aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c415b630 0x7f55c40aee70 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00cc30 tx=0x7f55bc00a5c0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 -- 172.21.15.70:0/1230668744 shutdown_connections 2023-10-08T05:56:38.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f55a806af90 0x7f55a806d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55c415b630 0x7f55c40aee70 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55c40cb500 0x7f55c40ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 --2- 172.21.15.70:0/1230668744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55c406cfa0 0x7f55c4153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 -- 172.21.15.70:0/1230668744 >> 172.21.15.70:0/1230668744 conn(0x7f55c406a140 msgr2=0x7f55c40cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 -- 172.21.15.70:0/1230668744 shutdown_connections 2023-10-08T05:56:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.432+0000 7f55cc4a2640 1 -- 172.21.15.70:0/1230668744 wait complete. 2023-10-08T05:56:38.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:56:38.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.655+0000 7fedacd53640 1 Processor -- start 2023-10-08T05:56:38.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.655+0000 7fedacd53640 1 -- start start 2023-10-08T05:56:38.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.656+0000 7fedacd53640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feda8159ac0 0x7feda806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.656+0000 7fedacd53640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda80709e0 0x7feda80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.656+0000 7fedacd53640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda806dee0 con 0x7feda80ad790 2023-10-08T05:56:38.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.656+0000 7fedacd53640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda815d850 con 0x7feda8159ac0 2023-10-08T05:56:38.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.656+0000 7fedacd53640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda815db30 con 0x7feda80709e0 2023-10-08T05:56:38.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.657+0000 7feda659c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feda8159ac0 0x7feda806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.657+0000 7fed9ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feda80ad790 0x7feda80702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35330/0 (socket says 172.21.15.70:35330) 2023-10-08T05:56:38.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.657+0000 7fed9ffff640 1 -- 172.21.15.70:0/384030132 learned_addr learned my addr 172.21.15.70:0/384030132 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:38.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.657+0000 7feda6d9d640 1 --2- 172.21.15.70:0/384030132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda80709e0 0x7feda80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.657+0000 7feda5d9b640 1 -- 172.21.15.70:0/384030132 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808899899 0 0) 0x7feda806dee0 con 0x7feda80ad790 2023-10-08T05:56:38.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.657+0000 7feda659c640 1 -- 172.21.15.70:0/384030132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda80709e0 msgr2=0x7feda80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:38.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.657+0000 7feda659c640 1 --2- 172.21.15.70:0/384030132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda80709e0 0x7feda80c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.657+0000 7feda659c640 1 -- 172.21.15.70:0/384030132 >> v1:172.21.15.70:6789/0 conn(0x7feda80ad790 legacy=0x7feda80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.657+0000 7feda659c640 1 -- 172.21.15.70:0/384030132 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda815de10 con 0x7feda8159ac0 2023-10-08T05:56:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.658+0000 7feda659c640 1 --2- 172.21.15.70:0/384030132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda8159ac0 0x7feda806d5e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fed980097a0 tx=0x7fed9802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45c5d1b865b565b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7feda5d9b640 1 -- 172.21.15.70:0/384030132 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9802d040 con 0x7feda8159ac0 2023-10-08T05:56:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7feda5d9b640 1 -- 172.21.15.70:0/384030132 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed98031810 con 0x7feda8159ac0 2023-10-08T05:56:38.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7feda5d9b640 1 -- 172.21.15.70:0/384030132 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed98031ba0 con 0x7feda8159ac0 2023-10-08T05:56:38.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7fedacd53640 1 -- 172.21.15.70:0/384030132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda8159ac0 msgr2=0x7feda806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7fedacd53640 1 --2- 172.21.15.70:0/384030132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda8159ac0 0x7feda806d5e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fed980097a0 tx=0x7fed9802b8b0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7fedacd53640 1 -- 172.21.15.70:0/384030132 shutdown_connections 2023-10-08T05:56:38.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7fedacd53640 1 --2- 172.21.15.70:0/384030132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda80709e0 0x7feda80c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7fedacd53640 1 --2- 172.21.15.70:0/384030132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda8159ac0 0x7feda806d5e0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7fedacd53640 1 -- 172.21.15.70:0/384030132 >> 172.21.15.70:0/384030132 conn(0x7feda806a380 msgr2=0x7feda80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:38.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7fedacd53640 1 -- 172.21.15.70:0/384030132 shutdown_connections 2023-10-08T05:56:38.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.659+0000 7fedacd53640 1 -- 172.21.15.70:0/384030132 wait complete. 2023-10-08T05:56:38.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.661+0000 7fedacd53640 1 Processor -- start 2023-10-08T05:56:38.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.661+0000 7fedacd53640 1 -- start start 2023-10-08T05:56:38.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.662+0000 7fedacd53640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feda80709e0 0x7feda80b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.662+0000 7fedacd53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda80ad790 0x7feda8141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.662+0000 7fedacd53640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda8159ac0 0x7feda815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.662+0000 7fedacd53640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda81461a0 con 0x7feda80ad790 2023-10-08T05:56:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.662+0000 7fedacd53640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda80b15d0 con 0x7feda80709e0 2023-10-08T05:56:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.662+0000 7fedacd53640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda8145900 con 0x7feda8159ac0 2023-10-08T05:56:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.662+0000 7fed9ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda80ad790 0x7feda8141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7fed9ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda80ad790 0x7feda8141260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41456/0 (socket says 172.21.15.70:41456) 2023-10-08T05:56:38.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7fed9ffff640 1 -- 172.21.15.70:0/94460576 learned_addr learned my addr 172.21.15.70:0/94460576 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:38.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7feda659c640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feda80709e0 0x7feda80b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7feda6d9d640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda8159ac0 0x7feda815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7fed9ffff640 1 -- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda8159ac0 msgr2=0x7feda815a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7fed9ffff640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda8159ac0 0x7feda815a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7fed9ffff640 1 -- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feda80709e0 msgr2=0x7feda80b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7fed9ffff640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feda80709e0 0x7feda80b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7fed9ffff640 1 -- 172.21.15.70:0/94460576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed98009400 con 0x7feda80ad790 2023-10-08T05:56:38.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7feda659c640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feda80709e0 0x7feda80b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7fed9ffff640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda80ad790 0x7feda8141260 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fed9000c4a0 tx=0x7fed9000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.663+0000 7fed9effd640 1 -- 172.21.15.70:0/94460576 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90005ce0 con 0x7feda80ad790 2023-10-08T05:56:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.664+0000 7fed9effd640 1 -- 172.21.15.70:0/94460576 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed90005e40 con 0x7feda80ad790 2023-10-08T05:56:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.664+0000 7fed9effd640 1 -- 172.21.15.70:0/94460576 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed900072b0 con 0x7feda80ad790 2023-10-08T05:56:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.664+0000 7fedacd53640 1 -- 172.21.15.70:0/94460576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda8145be0 con 0x7feda80ad790 2023-10-08T05:56:38.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.664+0000 7fedacd53640 1 -- 172.21.15.70:0/94460576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda80b3960 con 0x7feda80ad790 2023-10-08T05:56:38.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.665+0000 7fedacd53640 1 -- 172.21.15.70:0/94460576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda80b17d0 con 0x7feda80ad790 2023-10-08T05:56:38.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.666+0000 7fed9effd640 1 -- 172.21.15.70:0/94460576 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fed90007410 con 0x7feda80ad790 2023-10-08T05:56:38.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.666+0000 7fed9effd640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fed8806af90 0x7fed8806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:38.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.667+0000 7fed9effd640 1 -- 172.21.15.70:0/94460576 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed90088e70 con 0x7feda80ad790 2023-10-08T05:56:38.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.667+0000 7feda659c640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fed8806af90 0x7fed8806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:38.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.668+0000 7feda659c640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fed8806af90 0x7fed8806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fed98036000 tx=0x7fed9803c040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:38.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.671+0000 7fed9effd640 1 -- 172.21.15.70:0/94460576 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feda80b17d0 con 0x7feda80ad790 2023-10-08T05:56:38.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.895+0000 7fedacd53640 1 -- 172.21.15.70:0/94460576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feda80b17d0 con 0x7feda80ad790 2023-10-08T05:56:38.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.896+0000 7fed9effd640 1 -- 172.21.15.70:0/94460576 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4921 (secure 0 0 0) 0x7feda80b17d0 con 0x7feda80ad790 2023-10-08T05:56:38.897 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:38.897 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1820637663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1820637663},{"type":"v1","addr":"172.21.15.181:6835","nonce":1820637663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1666874980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1666874980},{"type":"v1","addr":"172.21.15.181:6837","nonce":1666874980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2918183907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2918183907},{"type":"v1","addr":"172.21.15.70:6834","nonce":2918183907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:56:32.378187+0000","modified":"2023-10-08T05:56:35.419118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6835/3869116018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3869116018},{"type":"v1","addr":"172.21.15.70:6835","nonce":3869116018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-08T05:56:38.898 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 68 2023-10-08T05:56:38.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 -- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fed8806af90 msgr2=0x7fed8806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fed8806af90 0x7fed8806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fed98036000 tx=0x7fed9803c040 comp rx=0 tx=0).stop 2023-10-08T05:56:38.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 -- 172.21.15.70:0/94460576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda80ad790 msgr2=0x7feda8141260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:38.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda80ad790 0x7feda8141260 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fed9000c4a0 tx=0x7fed9000c960 comp rx=0 tx=0).stop 2023-10-08T05:56:38.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 -- 172.21.15.70:0/94460576 shutdown_connections 2023-10-08T05:56:38.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fed8806af90 0x7fed8806d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feda8159ac0 0x7feda815a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feda80ad790 0x7feda8141260 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 --2- 172.21.15.70:0/94460576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feda80709e0 0x7feda80b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:38.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 -- 172.21.15.70:0/94460576 >> 172.21.15.70:0/94460576 conn(0x7feda806a380 msgr2=0x7feda80af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:38.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 -- 172.21.15.70:0/94460576 shutdown_connections 2023-10-08T05:56:38.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:38.899+0000 7fedacd53640 1 -- 172.21.15.70:0/94460576 wait complete. 2023-10-08T05:56:39.319 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T05:56:32.378187+0000', 'modified': '2023-10-08T05:56:35.419118+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8526}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8526': {'gid': 8526, 'name': 'c', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.70:6835/3869116018', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6833', 'nonce': 3869116018}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 3869116018}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T05:56:39.319 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T05:56:39.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T05:56:39.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.545+0000 7f4c9c8f2640 1 Processor -- start 2023-10-08T05:56:39.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.546+0000 7f4c9c8f2640 1 -- start start 2023-10-08T05:56:39.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.547+0000 7f4c9c8f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4c9806db60 0x7f4c980b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.547+0000 7f4c9c8f2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9815b310 con 0x7f4c980c9cf0 2023-10-08T05:56:39.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.547+0000 7f4c9c8f2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9806d290 con 0x7f4c9806c3a0 2023-10-08T05:56:39.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.547+0000 7f4c9c8f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9815b110 con 0x7f4c9806db60 2023-10-08T05:56:39.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c95d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4c980c9cf0 0x7f4c9815aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35346/0 (socket says 172.21.15.70:35346) 2023-10-08T05:56:39.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c95d9b640 1 -- 172.21.15.70:0/565674012 learned_addr learned my addr 172.21.15.70:0/565674012 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:39.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c96d9d640 1 --2- 172.21.15.70:0/565674012 >> v2:172.21.15.181:3300/0 conn(0x7f4c9806db60 0x7f4c980b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957066761 0 0) 0x7f4c80001110 con 0x7f4c980c9cf0 2023-10-08T05:56:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c84003520 con 0x7f4c980c9cf0 2023-10-08T05:56:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1886711537 0 0) 0x7f4c84003520 con 0x7f4c980c9cf0 2023-10-08T05:56:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 >> v2:172.21.15.181:3300/0 conn(0x7f4c9806db60 msgr2=0x7f4c980b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c9559a640 1 --2- 172.21.15.70:0/565674012 >> v2:172.21.15.181:3300/0 conn(0x7f4c9806db60 0x7f4c980b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 >> v1:172.21.15.181:6789/0 conn(0x7f4c9806c3a0 legacy=0x7f4c9806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c980c9860 con 0x7f4c980c9cf0 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4c80001800 con 0x7f4c980c9cf0 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 >> v1:172.21.15.70:6789/0 conn(0x7f4c980c9cf0 legacy=0x7f4c9815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.548+0000 7f4c9559a640 1 --2- 172.21.15.70:0/565674012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c840039c0 0x7f4c84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c9559a640 1 --2- 172.21.15.70:0/565674012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c840047c0 0x7f4c84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c9559a640 1 --2- 172.21.15.70:0/565674012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9815b110 con 0x7f4c84005180 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8400c080 con 0x7f4c840039c0 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8400c240 con 0x7f4c840047c0 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c96d9d640 1 --2- 172.21.15.70:0/565674012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c9659c640 1 --2- 172.21.15.70:0/565674012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c840039c0 0x7f4c84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c96d9d640 1 -- 172.21.15.70:0/565674012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c840047c0 msgr2=0x7f4c84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c96d9d640 1 --2- 172.21.15.70:0/565674012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c840047c0 0x7f4c84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c96d9d640 1 -- 172.21.15.70:0/565674012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c840039c0 msgr2=0x7f4c84003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c96d9d640 1 --2- 172.21.15.70:0/565674012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c840039c0 0x7f4c84003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.549+0000 7f4c96d9d640 1 -- 172.21.15.70:0/565674012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c84003520 con 0x7f4c84005180 2023-10-08T05:56:39.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.550+0000 7f4c96d9d640 1 -- 172.21.15.70:0/565674012 reap_dead start 2023-10-08T05:56:39.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.550+0000 7f4c96d9d640 1 --2- 172.21.15.70:0/565674012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c8400b980 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009ac0 tx=0x7f4c840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23651bdf787e14b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:39.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.550+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90004020 con 0x7f4c84005180 2023-10-08T05:56:39.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.550+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90004180 con 0x7f4c84005180 2023-10-08T05:56:39.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.550+0000 7f4c9559a640 1 -- 172.21.15.70:0/565674012 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9002c710 con 0x7f4c84005180 2023-10-08T05:56:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.551+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/565674012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 msgr2=0x7f4c8400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.551+0000 7f4c9c8f2640 1 --2- 172.21.15.70:0/565674012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c8400b980 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009ac0 tx=0x7f4c840057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.551+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/565674012 shutdown_connections 2023-10-08T05:56:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.551+0000 7f4c9c8f2640 1 --2- 172.21.15.70:0/565674012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c8400b980 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.551+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/565674012 >> 172.21.15.70:0/565674012 conn(0x7f4c9806a5d0 msgr2=0x7f4c980b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.551+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/565674012 shutdown_connections 2023-10-08T05:56:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.551+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/565674012 wait complete. 2023-10-08T05:56:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.553+0000 7f4c9c8f2640 1 Processor -- start 2023-10-08T05:56:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.553+0000 7f4c9c8f2640 1 -- start start 2023-10-08T05:56:39.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.554+0000 7f4c9c8f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c980a8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.554+0000 7f4c9c8f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9814cd20 0x7f4c980cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.554+0000 7f4c9c8f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c980c4440 0x7f4c980c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.554+0000 7f4c9c8f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980c4e00 con 0x7f4c84005180 2023-10-08T05:56:39.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.554+0000 7f4c9c8f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980cf280 con 0x7f4c9814cd20 2023-10-08T05:56:39.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.554+0000 7f4c9c8f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980ce670 con 0x7f4c980c4440 2023-10-08T05:56:39.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.554+0000 7f4c9659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c980a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.554+0000 7f4c9659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c980a8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41466/0 (socket says 172.21.15.70:41466) 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.554+0000 7f4c9659c640 1 -- 172.21.15.70:0/1204706465 learned_addr learned my addr 172.21.15.70:0/1204706465 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c9659c640 1 -- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c980c4440 msgr2=0x7f4c980c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c9659c640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c980c4440 0x7f4c980c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c9659c640 1 -- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9814cd20 msgr2=0x7f4c980cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c9659c640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9814cd20 0x7f4c980cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c9659c640 1 -- 172.21.15.70:0/1204706465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c980ce950 con 0x7f4c84005180 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c9659c640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c980a8240 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800d910 tx=0x7f4c8800ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c8effd640 1 -- 172.21.15.70:0/1204706465 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88003da0 con 0x7f4c84005180 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c8effd640 1 -- 172.21.15.70:0/1204706465 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8800b710 con 0x7f4c84005180 2023-10-08T05:56:39.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c8effd640 1 -- 172.21.15.70:0/1204706465 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800ba80 con 0x7f4c84005180 2023-10-08T05:56:39.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/1204706465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c980cec30 con 0x7f4c84005180 2023-10-08T05:56:39.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.555+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/1204706465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9814d1a0 con 0x7f4c84005180 2023-10-08T05:56:39.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.558+0000 7f4c8effd640 1 -- 172.21.15.70:0/1204706465 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4c8800bbe0 con 0x7f4c84005180 2023-10-08T05:56:39.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.558+0000 7f4c8effd640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4c7806afc0 0x7f4c7806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.559+0000 7f4c8effd640 1 -- 172.21.15.70:0/1204706465 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c88089d30 con 0x7f4c84005180 2023-10-08T05:56:39.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.559+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4c7806afc0 0x7f4c7806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.559+0000 7f4c9f480740 1 Processor -- start 2023-10-08T05:56:39.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.559+0000 7f4c9f480740 1 -- start start 2023-10-08T05:56:39.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.560+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4c7806afc0 0x7f4c7806d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c980c4f90 tx=0x7f4c80001190 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:39.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.560+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63eb60 0x563a8f611a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.560+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63ef40 0x563a8f6418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.560+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f611fa0 0x563a8f641e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.560+0000 7f4c9f480740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8f585700 con 0x563a8f63ef40 2023-10-08T05:56:39.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.560+0000 7f4c9f480740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8f582df0 con 0x563a8f611fa0 2023-10-08T05:56:39.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.560+0000 7f4c9f480740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a8f57e6e0 con 0x563a8f63eb60 2023-10-08T05:56:39.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.561+0000 7f4c95d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63ef40 0x563a8f6418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.561+0000 7f4c95d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63ef40 0x563a8f6418f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41474/0 (socket says 172.21.15.70:41474) 2023-10-08T05:56:39.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.561+0000 7f4c95d9b640 1 -- 172.21.15.70:0/1814686407 learned_addr learned my addr 172.21.15.70:0/1814686407 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:39.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.561+0000 7f4c95d9b640 1 -- 172.21.15.70:0/1814686407 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63eb60 msgr2=0x563a8f611a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.561+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/1814686407 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63eb60 0x563a8f611a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.561+0000 7f4c95d9b640 1 -- 172.21.15.70:0/1814686407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f611fa0 msgr2=0x563a8f641e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.561+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/1814686407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f611fa0 0x563a8f641e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.561+0000 7f4c95d9b640 1 -- 172.21.15.70:0/1814686407 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a8f542910 con 0x563a8f63ef40 2023-10-08T05:56:39.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.561+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/1814686407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63ef40 0x563a8f6418f0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4c80007200 tx=0x7f4c8000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcbd8706caf88d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.561+0000 7f4c77fff640 1 -- 172.21.15.70:0/1814686407 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8001b070 con 0x563a8f63ef40 2023-10-08T05:56:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c77fff640 1 -- 172.21.15.70:0/1814686407 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80005e80 con 0x563a8f63ef40 2023-10-08T05:56:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c77fff640 1 -- 172.21.15.70:0/1814686407 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8000c960 con 0x563a8f63ef40 2023-10-08T05:56:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c9f480740 1 -- 172.21.15.70:0/1814686407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63ef40 msgr2=0x563a8f6418f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c9f480740 1 --2- 172.21.15.70:0/1814686407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63ef40 0x563a8f6418f0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f4c80007200 tx=0x7f4c8000f000 comp rx=0 tx=0).stop 2023-10-08T05:56:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c9f480740 1 -- 172.21.15.70:0/1814686407 shutdown_connections 2023-10-08T05:56:39.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c9f480740 1 --2- 172.21.15.70:0/1814686407 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f611fa0 0x563a8f641e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c9f480740 1 --2- 172.21.15.70:0/1814686407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63ef40 0x563a8f6418f0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c9f480740 1 --2- 172.21.15.70:0/1814686407 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63eb60 0x563a8f611a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c9f480740 1 -- 172.21.15.70:0/1814686407 >> 172.21.15.70:0/1814686407 conn(0x563a8f613d70 msgr2=0x563a8f65c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c9f480740 1 -- 172.21.15.70:0/1814686407 shutdown_connections 2023-10-08T05:56:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.562+0000 7f4c9f480740 1 -- 172.21.15.70:0/1814686407 wait complete. 2023-10-08T05:56:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.563+0000 7f4c9f480740 1 -- start start 2023-10-08T05:56:39.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.563+0000 7f4c9f480740 1 Processor -- start 2023-10-08T05:56:39.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.564+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f6543b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.564+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63ef40 0x563a8f645060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.564+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63bb50 0x563a8f63bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.564+0000 7f4c9f480740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8f51c970 con 0x563a8f63eb60 2023-10-08T05:56:39.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.564+0000 7f4c9f480740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8f51c610 con 0x563a8f63ef40 2023-10-08T05:56:39.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.564+0000 7f4c9f480740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a8f2e2f20 con 0x563a8f63bb50 2023-10-08T05:56:39.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.564+0000 7f4c9f480740 10 client.0 _ll_register_callbacks cb 0x563a8f55a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:56:39.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.564+0000 7f4c9659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f6543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.564+0000 7f4c9659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f6543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41482/0 (socket says 172.21.15.70:41482) 2023-10-08T05:56:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c9659c640 1 -- 172.21.15.70:0/2098878006 learned_addr learned my addr 172.21.15.70:0/2098878006 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c96d9d640 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63bb50 0x563a8f63bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c9659c640 1 -- 172.21.15.70:0/2098878006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63bb50 msgr2=0x563a8f63bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63ef40 0x563a8f645060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c9659c640 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63bb50 0x563a8f63bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c9659c640 1 -- 172.21.15.70:0/2098878006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63ef40 msgr2=0x563a8f645060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c9659c640 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63ef40 0x563a8f645060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c9659c640 1 -- 172.21.15.70:0/2098878006 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c980c9860 con 0x563a8f63eb60 2023-10-08T05:56:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c9659c640 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f6543b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f4c88058350 tx=0x7f4c88058e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418d0d4a2093ff3a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c757fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:56:39.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c757fa640 1 -- 172.21.15.70:0/2098878006 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88057d40 con 0x563a8f63eb60 2023-10-08T05:56:39.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c757fa640 1 -- 172.21.15.70:0/2098878006 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c88060950 con 0x563a8f63eb60 2023-10-08T05:56:39.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.565+0000 7f4c9f480740 1 -- 172.21.15.70:0/2098878006 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a8f5773e0 con 0x563a8f63eb60 2023-10-08T05:56:39.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c757fa640 1 -- 172.21.15.70:0/2098878006 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c880604c0 con 0x563a8f63eb60 2023-10-08T05:56:39.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c9f480740 10 client.8655 fetch_fsmap learned FSMap version 68 2023-10-08T05:56:39.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c9f480740 1 -- 172.21.15.70:0/2098878006 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x563a8f57cd60 con 0x563a8f63eb60 2023-10-08T05:56:39.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c757fa640 1 -- 172.21.15.70:0/2098878006 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a8f5773e0 con 0x563a8f63eb60 2023-10-08T05:56:39.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c757fa640 1 -- 172.21.15.70:0/2098878006 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4c88078070 con 0x563a8f63eb60 2023-10-08T05:56:39.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c9f480740 10 client.8655 fetch_fsmap finished waiting for FSMap version 68 2023-10-08T05:56:39.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c9f480740 10 client.8655 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T05:56:39.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c9f480740 20 client.8655 populate_metadata read hostname 'smithi070' 2023-10-08T05:56:39.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.70:6833/3869116018,v1:172.21.15.70:6835/3869116018] conn(0x563a8f670670 0x563a8f679ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c9f480740 4 client.8655 mds_command: new command op to 8526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T05:56:39.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c9f480740 1 -- 172.21.15.70:0/2098878006 --> [v2:172.21.15.70:6833/3869116018,v1:172.21.15.70:6835/3869116018] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a8f2806c0 con 0x563a8f670670 2023-10-08T05:56:39.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.566+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.70:6833/3869116018,v1:172.21.15.70:6835/3869116018] conn(0x563a8f670670 0x563a8f679ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.567+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.70:6833/3869116018,v1:172.21.15.70:6835/3869116018] conn(0x563a8f670670 0x563a8f679ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f061780d5f4e24 server_cookie=30ca9041425fecc8 in_seq=0 out_seq=0 2023-10-08T05:56:39.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.567+0000 7f4c757fa640 10 client.8655 ms_handle_connect on v2:172.21.15.70:6833/3869116018 2023-10-08T05:56:39.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c757fa640 1 -- 172.21.15.70:0/2098878006 <== mds.0 v2:172.21.15.70:6833/3869116018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563a8f2806c0 con 0x563a8f670670 2023-10-08T05:56:39.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c757fa640 10 client.8655 handle_command_reply: tid=0 2023-10-08T05:56:39.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 client.8655 shutdown 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 -- 172.21.15.70:0/2098878006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 msgr2=0x563a8f6543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f6543b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f4c88058350 tx=0x7f4c88058e10 comp rx=0 tx=0).stop 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 -- 172.21.15.70:0/2098878006 shutdown_connections 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.70:6833/3869116018,v1:172.21.15.70:6835/3869116018] conn(0x563a8f670670 0x563a8f679ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63bb50 0x563a8f63bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63ef40 0x563a8f645060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c757fa640 0 client.8655 ms_handle_reset on v2:172.21.15.70:6833/3869116018 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2098878006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f6543b0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 -- 172.21.15.70:0/2098878006 >> 172.21.15.70:0/2098878006 conn(0x563a8f611fa0 msgr2=0x563a8f6418e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 -- 172.21.15.70:0/2098878006 shutdown_connections 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 1 -- 172.21.15.70:0/2098878006 wait complete. 2023-10-08T05:56:39.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.568+0000 7f4c9f480740 20 client.8655 trim_cache size 0 max 16384 2023-10-08T05:56:39.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.584+0000 7f4c9f480740 1 Processor -- start 2023-10-08T05:56:39.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.584+0000 7f4c9f480740 1 -- start start 2023-10-08T05:56:39.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.585+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63bb50 0x563a8f67e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.585+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f65fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.585+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63ef40 0x563a8f660340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.585+0000 7f4c9f480740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8f580660 con 0x563a8f63eb60 2023-10-08T05:56:39.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.585+0000 7f4c9f480740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8f6113e0 con 0x563a8f63ef40 2023-10-08T05:56:39.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.585+0000 7f4c9f480740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a8f63c110 con 0x563a8f63bb50 2023-10-08T05:56:39.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.585+0000 7f4c95d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f65fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.585+0000 7f4c95d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f65fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41496/0 (socket says 172.21.15.70:41496) 2023-10-08T05:56:39.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.585+0000 7f4c95d9b640 1 -- 172.21.15.70:0/988890964 learned_addr learned my addr 172.21.15.70:0/988890964 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:39.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.586+0000 7f4c95d9b640 1 -- 172.21.15.70:0/988890964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63bb50 msgr2=0x563a8f67e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.586+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/988890964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63bb50 0x563a8f67e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.586+0000 7f4c95d9b640 1 -- 172.21.15.70:0/988890964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63ef40 msgr2=0x563a8f660340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.586+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/988890964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63ef40 0x563a8f660340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.586+0000 7f4c95d9b640 1 -- 172.21.15.70:0/988890964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a8f51bb80 con 0x563a8f63eb60 2023-10-08T05:56:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.586+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/988890964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f65fe00 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f4c80001370 tx=0x7f4c80007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647d5480ae79ddf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.586+0000 7f4c8cff9640 1 -- 172.21.15.70:0/988890964 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8001b070 con 0x563a8f63eb60 2023-10-08T05:56:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.586+0000 7f4c8cff9640 1 -- 172.21.15.70:0/988890964 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8002a5a0 con 0x563a8f63eb60 2023-10-08T05:56:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.586+0000 7f4c8cff9640 1 -- 172.21.15.70:0/988890964 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80026040 con 0x563a8f63eb60 2023-10-08T05:56:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.587+0000 7f4c9f480740 1 -- 172.21.15.70:0/988890964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 msgr2=0x563a8f65fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.587+0000 7f4c9f480740 1 --2- 172.21.15.70:0/988890964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f65fe00 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f4c80001370 tx=0x7f4c80007000 comp rx=0 tx=0).stop 2023-10-08T05:56:39.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.587+0000 7f4c9f480740 1 -- 172.21.15.70:0/988890964 shutdown_connections 2023-10-08T05:56:39.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.587+0000 7f4c9f480740 1 --2- 172.21.15.70:0/988890964 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63ef40 0x563a8f660340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.587+0000 7f4c9f480740 1 --2- 172.21.15.70:0/988890964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f65fe00 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.587+0000 7f4c9f480740 1 --2- 172.21.15.70:0/988890964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63bb50 0x563a8f67e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.587+0000 7f4c9f480740 1 -- 172.21.15.70:0/988890964 >> 172.21.15.70:0/988890964 conn(0x563a8f611fa0 msgr2=0x563a8f644e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:39.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.587+0000 7f4c9f480740 1 -- 172.21.15.70:0/988890964 shutdown_connections 2023-10-08T05:56:39.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.587+0000 7f4c9f480740 1 -- 172.21.15.70:0/988890964 wait complete. 2023-10-08T05:56:39.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.588+0000 7f4c9f480740 1 -- start start 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.588+0000 7f4c9f480740 1 Processor -- start 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63bb50 0x563a8f65fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f660340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c9f480740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63ef40 0x563a8f63cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c9f480740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8f616120 con 0x563a8f63eb60 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c9f480740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a8f676ba0 con 0x563a8f63bb50 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c9f480740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a8f6c5c40 con 0x563a8f63ef40 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c9f480740 10 client.0 _ll_register_callbacks cb 0x7f4c80001c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c95d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f660340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c95d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f660340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41510/0 (socket says 172.21.15.70:41510) 2023-10-08T05:56:39.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c95d9b640 1 -- 172.21.15.70:0/2056787598 learned_addr learned my addr 172.21.15.70:0/2056787598 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c95d9b640 1 -- 172.21.15.70:0/2056787598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63ef40 msgr2=0x563a8f63cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.589+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63ef40 0x563a8f63cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c95d9b640 1 -- 172.21.15.70:0/2056787598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63bb50 msgr2=0x563a8f65fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63bb50 0x563a8f65fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c95d9b640 1 -- 172.21.15.70:0/2056787598 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a8f542910 con 0x563a8f63eb60 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c95d9b640 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f660340 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f4c80001370 tx=0x7f4c800012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f460741400e93a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c757fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c757fa640 1 -- 172.21.15.70:0/2056787598 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80026070 con 0x563a8f63eb60 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c757fa640 1 -- 172.21.15.70:0/2056787598 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c8002a3a0 con 0x563a8f63eb60 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c9f480740 1 -- 172.21.15.70:0/2056787598 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a8f5776c0 con 0x563a8f63eb60 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c757fa640 1 -- 172.21.15.70:0/2056787598 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c8001b070 con 0x563a8f63eb60 2023-10-08T05:56:39.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c9f480740 10 client.8661 fetch_fsmap learned FSMap version 68 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.590+0000 7f4c9f480740 1 -- 172.21.15.70:0/2056787598 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x563a8f541170 con 0x563a8f63eb60 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c757fa640 1 -- 172.21.15.70:0/2056787598 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a8f5776c0 con 0x563a8f63eb60 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c757fa640 1 -- 172.21.15.70:0/2056787598 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4c80029070 con 0x563a8f63eb60 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c9f480740 10 client.8661 fetch_fsmap finished waiting for FSMap version 68 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c9f480740 10 client.8661 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c9f480740 20 client.8661 populate_metadata read hostname 'smithi070' 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.70:6833/3869116018,v1:172.21.15.70:6835/3869116018] conn(0x563a8f63d120 0x563a8f63d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c9f480740 4 client.8661 mds_command: new command op to 8526 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c9f480740 1 -- 172.21.15.70:0/2056787598 --> [v2:172.21.15.70:6833/3869116018,v1:172.21.15.70:6835/3869116018] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563a8f2d3d10 con 0x563a8f63d120 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c9659c640 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.70:6833/3869116018,v1:172.21.15.70:6835/3869116018] conn(0x563a8f63d120 0x563a8f63d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c757fa640 10 client.8661 ms_handle_connect on v2:172.21.15.70:6833/3869116018 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.591+0000 7f4c9659c640 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.70:6833/3869116018,v1:172.21.15.70:6835/3869116018] conn(0x563a8f63d120 0x563a8f63d500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=485b2f9093b40216 server_cookie=6908660aa8a69284 in_seq=0 out_seq=0 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c757fa640 1 -- 172.21.15.70:0/2056787598 <== mds.0 v2:172.21.15.70:6833/3869116018 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x563a8f2d3d10 con 0x563a8f63d120 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c757fa640 10 client.8661 handle_command_reply: tid=0 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 client.8661 shutdown 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 -- 172.21.15.70:0/2056787598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 msgr2=0x563a8f660340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f660340 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f4c80001370 tx=0x7f4c800012e0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 -- 172.21.15.70:0/2056787598 shutdown_connections 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.70:6833/3869116018,v1:172.21.15.70:6835/3869116018] conn(0x563a8f63d120 0x563a8f63d500 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563a8f63ef40 0x563a8f63cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563a8f63eb60 0x563a8f660340 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 --2- 172.21.15.70:0/2056787598 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563a8f63bb50 0x563a8f65fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c757fa640 0 client.8661 ms_handle_reset on v2:172.21.15.70:6833/3869116018 2023-10-08T05:56:39.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 -- 172.21.15.70:0/2056787598 >> 172.21.15.70:0/2056787598 conn(0x563a8f611fa0 msgr2=0x563a8f67e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:39.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 -- 172.21.15.70:0/2056787598 shutdown_connections 2023-10-08T05:56:39.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 1 -- 172.21.15.70:0/2056787598 wait complete. 2023-10-08T05:56:39.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.592+0000 7f4c9f480740 20 client.8661 trim_cache size 0 max 16384 2023-10-08T05:56:39.597 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:56:39.597 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T05:56:39.597 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T05:56:39.598 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 8526, 2023-10-08T05:56:39.598 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T05:56:39.598 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T05:56:39.598 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T05:56:39.598 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.6782741999999997, 2023-10-08T05:56:39.598 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 68, 2023-10-08T05:56:39.598 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 149, 2023-10-08T05:56:39.598 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 149, 2023-10-08T05:56:39.598 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 18.206811726000002 2023-10-08T05:56:39.598 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:56:39.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4c7806afc0 msgr2=0x7f4c7806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4c7806afc0 0x7f4c7806d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4c980c4f90 tx=0x7f4c80001190 comp rx=0 tx=0).stop 2023-10-08T05:56:39.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/1204706465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 msgr2=0x7f4c980a8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c980a8240 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800d910 tx=0x7f4c8800ddd0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/1204706465 shutdown_connections 2023-10-08T05:56:39.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f4c7806afc0 0x7f4c7806d480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c980c4440 0x7f4c980c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c9814cd20 0x7f4c980cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 --2- 172.21.15.70:0/1204706465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c84005180 0x7f4c980a8240 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/1204706465 >> 172.21.15.70:0/1204706465 conn(0x7f4c9806a5d0 msgr2=0x7f4c9815b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:39.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/1204706465 shutdown_connections 2023-10-08T05:56:39.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.593+0000 7f4c9c8f2640 1 -- 172.21.15.70:0/1204706465 wait complete. 2023-10-08T05:56:39.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:56:39.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.804+0000 7f8df7939640 1 Processor -- start 2023-10-08T05:56:39.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.804+0000 7f8df7939640 1 -- start start 2023-10-08T05:56:39.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.805+0000 7f8df7939640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8df015ab40 0x7f8df00ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.805+0000 7f8df7939640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8df00cb580 0x7f8df00adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df7939640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df015b2b0 con 0x7f8df00c90b0 2023-10-08T05:56:39.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df7939640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df015bf30 con 0x7f8df00cb580 2023-10-08T05:56:39.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df7939640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df015f7a0 con 0x7f8df015ab40 2023-10-08T05:56:39.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df5ed6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8df00cb580 0x7f8df00adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df5ed6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8df00cb580 0x7f8df00adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41522/0 (socket says 172.21.15.70:41522) 2023-10-08T05:56:39.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df5ed6640 1 -- 172.21.15.70:0/98822210 learned_addr learned my addr 172.21.15.70:0/98822210 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:39.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df56d5640 1 --2- 172.21.15.70:0/98822210 >> v2:172.21.15.181:3300/0 conn(0x7f8df015ab40 0x7f8df00ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df5ed6640 1 -- 172.21.15.70:0/98822210 >> v2:172.21.15.181:3300/0 conn(0x7f8df015ab40 msgr2=0x7f8df00ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df5ed6640 1 --2- 172.21.15.70:0/98822210 >> v2:172.21.15.181:3300/0 conn(0x7f8df015ab40 0x7f8df00ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df5ed6640 1 -- 172.21.15.70:0/98822210 >> v1:172.21.15.181:6789/0 conn(0x7f8df00c90b0 legacy=0x7f8df00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:39.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.806+0000 7f8df5ed6640 1 -- 172.21.15.70:0/98822210 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df015b590 con 0x7f8df00cb580 2023-10-08T05:56:39.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.807+0000 7f8df5ed6640 1 --2- 172.21.15.70:0/98822210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 0x7f8df00adc60 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f8df006e440 tx=0x7f8de802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6a5962767e503 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:39.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8de7fff640 1 -- 172.21.15.70:0/98822210 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de802ca80 con 0x7f8df00cb580 2023-10-08T05:56:39.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8de7fff640 1 -- 172.21.15.70:0/98822210 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de802cbe0 con 0x7f8df00cb580 2023-10-08T05:56:39.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8de7fff640 1 -- 172.21.15.70:0/98822210 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de80343f0 con 0x7f8df00cb580 2023-10-08T05:56:39.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8df7939640 1 -- 172.21.15.70:0/98822210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 msgr2=0x7f8df00adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8df7939640 1 --2- 172.21.15.70:0/98822210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 0x7f8df00adc60 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f8df006e440 tx=0x7f8de802bee0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8df7939640 1 -- 172.21.15.70:0/98822210 shutdown_connections 2023-10-08T05:56:39.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8df7939640 1 --2- 172.21.15.70:0/98822210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 0x7f8df00adc60 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8df7939640 1 --2- 172.21.15.70:0/98822210 >> v2:172.21.15.181:3300/0 conn(0x7f8df015ab40 0x7f8df00ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8df7939640 1 -- 172.21.15.70:0/98822210 >> 172.21.15.70:0/98822210 conn(0x7f8df006a380 msgr2=0x7f8df00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:39.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8df7939640 1 -- 172.21.15.70:0/98822210 shutdown_connections 2023-10-08T05:56:39.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.808+0000 7f8df7939640 1 -- 172.21.15.70:0/98822210 wait complete. 2023-10-08T05:56:39.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.810+0000 7f8df7939640 1 Processor -- start 2023-10-08T05:56:39.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.810+0000 7f8df7939640 1 -- start start 2023-10-08T05:56:39.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.811+0000 7f8df7939640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df00c90b0 0x7f8df01f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.811+0000 7f8df7939640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 0x7f8df014ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.811+0000 7f8df7939640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df015ab40 0x7f8df01f1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.811+0000 7f8df7939640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df015c6f0 con 0x7f8df00cb580 2023-10-08T05:56:39.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.811+0000 7f8df7939640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df015c9d0 con 0x7f8df00c90b0 2023-10-08T05:56:39.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.811+0000 7f8df7939640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df015ccb0 con 0x7f8df015ab40 2023-10-08T05:56:39.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.811+0000 7f8df4ed4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 0x7f8df014ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.811+0000 7f8df4ed4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 0x7f8df014ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41528/0 (socket says 172.21.15.70:41528) 2023-10-08T05:56:39.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.811+0000 7f8df4ed4640 1 -- 172.21.15.70:0/2825797696 learned_addr learned my addr 172.21.15.70:0/2825797696 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:39.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8df56d5640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df00c90b0 0x7f8df01f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8df5ed6640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df015ab40 0x7f8df01f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8df4ed4640 1 -- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df015ab40 msgr2=0x7f8df01f1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8df4ed4640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df015ab40 0x7f8df01f1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8df4ed4640 1 -- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df00c90b0 msgr2=0x7f8df01f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8df4ed4640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df00c90b0 0x7f8df01f1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8df4ed4640 1 -- 172.21.15.70:0/2825797696 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de8009b30 con 0x7f8df00cb580 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8df4ed4640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 0x7f8df014ae20 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f8de000b3c0 tx=0x7f8de000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8de5ffb640 1 -- 172.21.15.70:0/2825797696 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0011830 con 0x7f8df00cb580 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8de5ffb640 1 -- 172.21.15.70:0/2825797696 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0011990 con 0x7f8df00cb580 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8de5ffb640 1 -- 172.21.15.70:0/2825797696 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0011d60 con 0x7f8df00cb580 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.812+0000 7f8df7939640 1 -- 172.21.15.70:0/2825797696 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df015cf90 con 0x7f8df00cb580 2023-10-08T05:56:39.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.813+0000 7f8df7939640 1 -- 172.21.15.70:0/2825797696 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df01f1850 con 0x7f8df00cb580 2023-10-08T05:56:39.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.814+0000 7f8df7939640 1 -- 172.21.15.70:0/2825797696 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df015a3f0 con 0x7f8df00cb580 2023-10-08T05:56:39.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.815+0000 7f8de5ffb640 1 -- 172.21.15.70:0/2825797696 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8de00193b0 con 0x7f8df00cb580 2023-10-08T05:56:39.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.815+0000 7f8de5ffb640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f8dd406af90 0x7f8dd406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:39.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.816+0000 7f8de5ffb640 1 -- 172.21.15.70:0/2825797696 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8de00896b0 con 0x7f8df00cb580 2023-10-08T05:56:39.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.816+0000 7f8df56d5640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f8dd406af90 0x7f8dd406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:39.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.817+0000 7f8df56d5640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f8dd406af90 0x7f8dd406d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00a970 tx=0x7f8dec008040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:39.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:39.820+0000 7f8de5ffb640 1 -- 172.21.15.70:0/2825797696 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8de0054550 con 0x7f8df00cb580 2023-10-08T05:56:40.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.036+0000 7f8df7939640 1 -- 172.21.15.70:0/2825797696 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8df015d4b0 con 0x7f8df00cb580 2023-10-08T05:56:40.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.037+0000 7f8de5ffb640 1 -- 172.21.15.70:0/2825797696 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8df015d4b0 con 0x7f8df00cb580 2023-10-08T05:56:40.037 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T05:56:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 -- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f8dd406af90 msgr2=0x7f8dd406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:40.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f8dd406af90 0x7f8dd406d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8dec00a970 tx=0x7f8dec008040 comp rx=0 tx=0).stop 2023-10-08T05:56:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 -- 172.21.15.70:0/2825797696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 msgr2=0x7f8df014ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 0x7f8df014ae20 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f8de000b3c0 tx=0x7f8de000b880 comp rx=0 tx=0).stop 2023-10-08T05:56:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 -- 172.21.15.70:0/2825797696 shutdown_connections 2023-10-08T05:56:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f8dd406af90 0x7f8dd406d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8df015ab40 0x7f8df01f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8df00cb580 0x7f8df014ae20 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 --2- 172.21.15.70:0/2825797696 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8df00c90b0 0x7f8df01f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 -- 172.21.15.70:0/2825797696 >> 172.21.15.70:0/2825797696 conn(0x7f8df006a380 msgr2=0x7f8df00ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 -- 172.21.15.70:0/2825797696 shutdown_connections 2023-10-08T05:56:40.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:40.040+0000 7f8df7939640 1 -- 172.21.15.70:0/2825797696 wait complete. 2023-10-08T05:56:40.057 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T05:56:40.057 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:56:40.057 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:56:40.058 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T05:56:40.058 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T05:56:40.058 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T05:56:40.058 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T05:56:40.058 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:56:40.058 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:56:40.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:40.058 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:56:40.110 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:56:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:56:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:56:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:56:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:56:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:56:40.110 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:56:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:56:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:56:40.111 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:56:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:56:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:56:40.111 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:56:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:56:40.111 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:56:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:56:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:56:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41196sec preferred_lft 41196sec 2023-10-08T05:56:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:56:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:56:40.112 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:56:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:56:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T05:56:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:56:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T05:56:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:56:40.113 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:56:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T05:56:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T05:56:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:56:40.113 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:56:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T05:56:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T05:56:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:56:40.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:40.114 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:56:40.202 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:56:40.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:40.206 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:56:40.259 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:56:40.259 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:56:40.259 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:56:40.259 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:56:40.259 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:56:40.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:40.260 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:56:40.260 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:56:40.260 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:56:40.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:56:40.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:56:40.261 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:56:40.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:40.453 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:56:40.500 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T05:56:40.501 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T05:56:40.501 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:56:40.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:40.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T05:56:40.588 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:56:40.588 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T05:56:40.639 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:56:40.640 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:56:40.640 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:56:40.691 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T05:56:40.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:40.692 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:56:40.826 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:56:40.826 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:56:40.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:56:40.859 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:56:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:56:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:56:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:56:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:56:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:56:40.861 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T05:56:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:56:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:56:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:56:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:56:40.865 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:56:40.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:40.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:56:40.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:40.987 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:56:41.100 INFO:teuthology.orchestra.run.smithi070.stdout:client8670 2023-10-08T05:56:41.100 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:56:41.100 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:41.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T05:56:41.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.361+0000 7f44cdce1640 1 Processor -- start 2023-10-08T05:56:41.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.361+0000 7f44cdce1640 1 -- start start 2023-10-08T05:56:41.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.362+0000 7f44cdce1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44c806c3a0 0x7f44c80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.362+0000 7f44cdce1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815aa00 0x7f44c80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.362+0000 7f44cdce1640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c815b310 con 0x7f44c806db60 2023-10-08T05:56:41.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.362+0000 7f44cdce1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c80ca230 con 0x7f44c806c3a0 2023-10-08T05:56:41.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.362+0000 7f44cdce1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c806d290 con 0x7f44c815aa00 2023-10-08T05:56:41.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.363+0000 7f44c77fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44c806c3a0 0x7f44c80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.363+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815aa00 0x7f44c80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.363+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815aa00 0x7f44c80b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:43998/0 (socket says 172.21.15.70:43998) 2023-10-08T05:56:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.363+0000 7f44c77fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f44c806c3a0 0x7f44c80c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39890/0 (socket says 172.21.15.70:39890) 2023-10-08T05:56:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.363+0000 7f44c7fff640 1 -- 172.21.15.70:0/999622056 learned_addr learned my addr 172.21.15.70:0/999622056 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.364+0000 7f44c7fff640 1 -- 172.21.15.70:0/999622056 >> v2:172.21.15.181:3300/0 conn(0x7f44c806c3a0 msgr2=0x7f44c80c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.364+0000 7f44c7fff640 1 --2- 172.21.15.70:0/999622056 >> v2:172.21.15.181:3300/0 conn(0x7f44c806c3a0 0x7f44c80c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.364+0000 7f44c7fff640 1 -- 172.21.15.70:0/999622056 >> v1:172.21.15.181:6789/0 conn(0x7f44c806db60 legacy=0x7f44c806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:41.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.364+0000 7f44c7fff640 1 -- 172.21.15.70:0/999622056 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c80c9860 con 0x7f44c815aa00 2023-10-08T05:56:41.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.365+0000 7f44c7fff640 1 --2- 172.21.15.70:0/999622056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815aa00 0x7f44c80b2a90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009870 tx=0x7f44c002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed74ff0b82586188 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:41.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.366+0000 7f44c67fc640 1 -- 172.21.15.70:0/999622056 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0030840 con 0x7f44c815aa00 2023-10-08T05:56:41.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.366+0000 7f44c67fc640 1 -- 172.21.15.70:0/999622056 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c00309a0 con 0x7f44c815aa00 2023-10-08T05:56:41.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.366+0000 7f44c67fc640 1 -- 172.21.15.70:0/999622056 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0030d30 con 0x7f44c815aa00 2023-10-08T05:56:41.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.367+0000 7f44cdce1640 1 -- 172.21.15.70:0/999622056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815aa00 msgr2=0x7f44c80b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:41.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.367+0000 7f44cdce1640 1 --2- 172.21.15.70:0/999622056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815aa00 0x7f44c80b2a90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009870 tx=0x7f44c002b980 comp rx=0 tx=0).stop 2023-10-08T05:56:41.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.367+0000 7f44cdce1640 1 -- 172.21.15.70:0/999622056 shutdown_connections 2023-10-08T05:56:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.367+0000 7f44cdce1640 1 --2- 172.21.15.70:0/999622056 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c815aa00 0x7f44c80b2a90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.367+0000 7f44cdce1640 1 --2- 172.21.15.70:0/999622056 >> v2:172.21.15.181:3300/0 conn(0x7f44c806c3a0 0x7f44c80c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.367+0000 7f44cdce1640 1 -- 172.21.15.70:0/999622056 >> 172.21.15.70:0/999622056 conn(0x7f44c806a5d0 msgr2=0x7f44c80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.367+0000 7f44cdce1640 1 -- 172.21.15.70:0/999622056 shutdown_connections 2023-10-08T05:56:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.367+0000 7f44cdce1640 1 -- 172.21.15.70:0/999622056 wait complete. 2023-10-08T05:56:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.369+0000 7f44cdce1640 1 Processor -- start 2023-10-08T05:56:41.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.369+0000 7f44cdce1640 1 -- start start 2023-10-08T05:56:41.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44cdce1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c806c3a0 0x7f44c815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44cdce1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c806db60 0x7f44c8070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44cdce1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c815aa00 0x7f44c80707b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44cdce1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c80c2380 con 0x7f44c806db60 2023-10-08T05:56:41.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44cdce1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c81f6d60 con 0x7f44c815aa00 2023-10-08T05:56:41.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44cdce1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c81f29c0 con 0x7f44c806c3a0 2023-10-08T05:56:41.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c806db60 0x7f44c8070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c806db60 0x7f44c8070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41540/0 (socket says 172.21.15.70:41540) 2023-10-08T05:56:41.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c6ffd640 1 -- 172.21.15.70:0/4121267954 learned_addr learned my addr 172.21.15.70:0/4121267954 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:41.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c7fff640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c815aa00 0x7f44c80707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c77fe640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c806c3a0 0x7f44c815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c6ffd640 1 -- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c806c3a0 msgr2=0x7f44c815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c6ffd640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c806c3a0 0x7f44c815cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c6ffd640 1 -- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c815aa00 msgr2=0x7f44c80707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c6ffd640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c815aa00 0x7f44c80707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c6ffd640 1 -- 172.21.15.70:0/4121267954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c00094d0 con 0x7f44c806db60 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.370+0000 7f44c7fff640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c815aa00 0x7f44c80707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.371+0000 7f44c6ffd640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c806db60 0x7f44c8070270 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f44b000c450 tx=0x7f44b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.371+0000 7f44bffff640 1 -- 172.21.15.70:0/4121267954 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b00052d0 con 0x7f44c806db60 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.371+0000 7f44bffff640 1 -- 172.21.15.70:0/4121267954 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b0005430 con 0x7f44c806db60 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.371+0000 7f44bffff640 1 -- 172.21.15.70:0/4121267954 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0012870 con 0x7f44c806db60 2023-10-08T05:56:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.371+0000 7f44cdce1640 1 -- 172.21.15.70:0/4121267954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c81f2ca0 con 0x7f44c806db60 2023-10-08T05:56:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.371+0000 7f44cdce1640 1 -- 172.21.15.70:0/4121267954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c81f3260 con 0x7f44c806db60 2023-10-08T05:56:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.372+0000 7f44cdce1640 1 -- 172.21.15.70:0/4121267954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c804e160 con 0x7f44c806db60 2023-10-08T05:56:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.373+0000 7f44bffff640 1 -- 172.21.15.70:0/4121267954 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f44b00129d0 con 0x7f44c806db60 2023-10-08T05:56:41.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.374+0000 7f44bffff640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 0x7f44a806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.374+0000 7f44bffff640 1 -- 172.21.15.70:0/4121267954 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f44b0089300 con 0x7f44c806db60 2023-10-08T05:56:41.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.375+0000 7f44c77fe640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 0x7f44a806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.376+0000 7f44c77fe640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 0x7f44a806d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f44c81f04a0 tx=0x7f44b800b040 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:41.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.379+0000 7f44bffff640 1 -- 172.21.15.70:0/4121267954 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44b0054070 con 0x7f44c806db60 2023-10-08T05:56:41.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.591+0000 7f44cdce1640 1 -- 172.21.15.70:0/4121267954 --> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f44c80c9210 con 0x7f44a806afc0 2023-10-08T05:56:41.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.595+0000 7f44bffff640 1 -- 172.21.15.70:0/4121267954 <== mgr.6685 v2:172.21.15.181:6832/3857845516 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f44b8016420 con 0x7f44a806afc0 2023-10-08T05:56:41.595 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:56:41.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:56:41.595 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T05:56:41.596 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:56:41.596 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:56:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.597+0000 7f44cdce1640 1 -- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 msgr2=0x7f44a806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.597+0000 7f44cdce1640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 0x7f44a806d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f44c81f04a0 tx=0x7f44b800b040 comp rx=0 tx=0).stop 2023-10-08T05:56:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.597+0000 7f44cdce1640 1 -- 172.21.15.70:0/4121267954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c806db60 msgr2=0x7f44c8070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.597+0000 7f44cdce1640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c806db60 0x7f44c8070270 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f44b000c450 tx=0x7f44b000c910 comp rx=0 tx=0).stop 2023-10-08T05:56:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.597+0000 7f44cdce1640 1 -- 172.21.15.70:0/4121267954 shutdown_connections 2023-10-08T05:56:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.597+0000 7f44cdce1640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f44a806afc0 0x7f44a806d480 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.597+0000 7f44cdce1640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44c815aa00 0x7f44c80707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.597+0000 7f44cdce1640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44c806db60 0x7f44c8070270 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.597+0000 7f44cdce1640 1 --2- 172.21.15.70:0/4121267954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44c806c3a0 0x7f44c815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.597+0000 7f44cdce1640 1 -- 172.21.15.70:0/4121267954 >> 172.21.15.70:0/4121267954 conn(0x7f44c806a5d0 msgr2=0x7f44c80c1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:41.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.598+0000 7f44cdce1640 1 -- 172.21.15.70:0/4121267954 shutdown_connections 2023-10-08T05:56:41.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.598+0000 7f44cdce1640 1 -- 172.21.15.70:0/4121267954 wait complete. 2023-10-08T05:56:41.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T05:56:41.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.823+0000 7f3162593640 1 Processor -- start 2023-10-08T05:56:41.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.823+0000 7f3162593640 1 -- start start 2023-10-08T05:56:41.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3162593640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f315c15cd80 0x7f315c15dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3162593640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f315c0d06c0 0x7f315c15e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3162593640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f315c0d0ec0 0x7f315c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3162593640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315c15d160 con 0x7f315c15cd80 2023-10-08T05:56:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3162593640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f315c0d3800 con 0x7f315c0d0ec0 2023-10-08T05:56:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3162593640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315c0d1850 con 0x7f315c0d06c0 2023-10-08T05:56:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3161591640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f315c15cd80 0x7f315c15dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3161591640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f315c15cd80 0x7f315c15dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41556/0 (socket says 172.21.15.70:41556) 2023-10-08T05:56:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3161591640 1 -- 172.21.15.70:0/9064628 learned_addr learned my addr 172.21.15.70:0/9064628 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:41.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3161591640 1 -- 172.21.15.70:0/9064628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f315c0d06c0 msgr2=0x7f315c15e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:41.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3161591640 1 --2- 172.21.15.70:0/9064628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f315c0d06c0 0x7f315c15e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3161591640 1 -- 172.21.15.70:0/9064628 >> v2:172.21.15.181:3300/0 conn(0x7f315c0d0ec0 msgr2=0x7f315c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:41.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3161591640 1 --2- 172.21.15.70:0/9064628 >> v2:172.21.15.181:3300/0 conn(0x7f315c0d0ec0 0x7f315c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.825+0000 7f3161591640 1 -- 172.21.15.70:0/9064628 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c0ce370 con 0x7f315c15cd80 2023-10-08T05:56:41.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.826+0000 7f3161591640 1 --2- 172.21.15.70:0/9064628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c15cd80 0x7f315c15dcf0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f315c0ccec0 tx=0x7f315802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e891cfb5d9fe84 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:41.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.827+0000 7f3153fff640 1 -- 172.21.15.70:0/9064628 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315802e070 con 0x7f315c15cd80 2023-10-08T05:56:41.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.827+0000 7f3153fff640 1 -- 172.21.15.70:0/9064628 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3158004030 con 0x7f315c15cd80 2023-10-08T05:56:41.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.827+0000 7f3153fff640 1 -- 172.21.15.70:0/9064628 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3158004420 con 0x7f315c15cd80 2023-10-08T05:56:41.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.827+0000 7f3162593640 1 -- 172.21.15.70:0/9064628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c15cd80 msgr2=0x7f315c15dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:41.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.827+0000 7f3162593640 1 --2- 172.21.15.70:0/9064628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c15cd80 0x7f315c15dcf0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f315c0ccec0 tx=0x7f315802b840 comp rx=0 tx=0).stop 2023-10-08T05:56:41.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.827+0000 7f3162593640 1 -- 172.21.15.70:0/9064628 shutdown_connections 2023-10-08T05:56:41.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.827+0000 7f3162593640 1 --2- 172.21.15.70:0/9064628 >> v2:172.21.15.181:3300/0 conn(0x7f315c0d0ec0 0x7f315c0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.827+0000 7f3162593640 1 --2- 172.21.15.70:0/9064628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f315c0d06c0 0x7f315c15e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.827+0000 7f3162593640 1 --2- 172.21.15.70:0/9064628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c15cd80 0x7f315c15dcf0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.827+0000 7f3162593640 1 -- 172.21.15.70:0/9064628 >> 172.21.15.70:0/9064628 conn(0x7f315c09bc40 msgr2=0x7f315c0d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:41.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.828+0000 7f3162593640 1 -- 172.21.15.70:0/9064628 shutdown_connections 2023-10-08T05:56:41.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.828+0000 7f3162593640 1 -- 172.21.15.70:0/9064628 wait complete. 2023-10-08T05:56:41.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.829+0000 7f3162593640 1 Processor -- start 2023-10-08T05:56:41.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.830+0000 7f3162593640 1 -- start start 2023-10-08T05:56:41.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3162593640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f315c0d06c0 0x7f315c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3162593640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c0d0ec0 0x7f315c157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3162593640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f315c15cd80 0x7f315c1f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3162593640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315c0b7d30 con 0x7f315c0d0ec0 2023-10-08T05:56:41.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3162593640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315c156d60 con 0x7f315c15cd80 2023-10-08T05:56:41.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3162593640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315c157040 con 0x7f315c0d06c0 2023-10-08T05:56:41.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3160d90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c0d0ec0 0x7f315c157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3160d90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c0d0ec0 0x7f315c157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41566/0 (socket says 172.21.15.70:41566) 2023-10-08T05:56:41.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3160d90640 1 -- 172.21.15.70:0/869543620 learned_addr learned my addr 172.21.15.70:0/869543620 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:41.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3161591640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f315c0d06c0 0x7f315c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.831+0000 7f3161d92640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f315c15cd80 0x7f315c1f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3160d90640 1 -- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f315c0d06c0 msgr2=0x7f315c0b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:41.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3160d90640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f315c0d06c0 0x7f315c0b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3160d90640 1 -- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f315c15cd80 msgr2=0x7f315c1f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:41.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3160d90640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f315c15cd80 0x7f315c1f5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:41.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3160d90640 1 -- 172.21.15.70:0/869543620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3158009460 con 0x7f315c0d0ec0 2023-10-08T05:56:41.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3161d92640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f315c15cd80 0x7f315c1f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:41.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3160d90640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c0d0ec0 0x7f315c157e00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f314c00b4a0 tx=0x7f314c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:41.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3151ffb640 1 -- 172.21.15.70:0/869543620 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c010610 con 0x7f315c0d0ec0 2023-10-08T05:56:41.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3151ffb640 1 -- 172.21.15.70:0/869543620 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c010770 con 0x7f315c0d0ec0 2023-10-08T05:56:41.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3162593640 1 -- 172.21.15.70:0/869543620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f315c157320 con 0x7f315c0d0ec0 2023-10-08T05:56:41.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3151ffb640 1 -- 172.21.15.70:0/869543620 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c010b20 con 0x7f315c0d0ec0 2023-10-08T05:56:41.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.832+0000 7f3162593640 1 -- 172.21.15.70:0/869543620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315c1578e0 con 0x7f315c0d0ec0 2023-10-08T05:56:41.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.834+0000 7f3162593640 1 -- 172.21.15.70:0/869543620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f315c157ad0 con 0x7f315c0d0ec0 2023-10-08T05:56:41.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.835+0000 7f3151ffb640 1 -- 172.21.15.70:0/869543620 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 67) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f314c010c80 con 0x7f315c0d0ec0 2023-10-08T05:56:41.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.836+0000 7f3151ffb640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f314006af90 0x7f314006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:41.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.836+0000 7f3151ffb640 1 -- 172.21.15.70:0/869543620 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f314c089590 con 0x7f315c0d0ec0 2023-10-08T05:56:41.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.836+0000 7f3161591640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f314006af90 0x7f314006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:41.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.838+0000 7f3161591640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f314006af90 0x7f314006d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3158034000 tx=0x7f3158008030 comp rx=0 tx=0).ready entity=mgr.6685 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:41.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:41.840+0000 7f3151ffb640 1 -- 172.21.15.70:0/869543620 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f314c0543a0 con 0x7f315c0d0ec0 2023-10-08T05:56:42.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.085+0000 7f3162593640 1 -- 172.21.15.70:0/869543620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f315c074150 con 0x7f315c0d0ec0 2023-10-08T05:56:42.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.451+0000 7f3151ffb640 1 -- 172.21.15.70:0/869543620 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v68) v1 ==== 92+0+0 (secure 0 0 0) 0x7f314c0579c0 con 0x7f315c0d0ec0 2023-10-08T05:56:42.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 -- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f314006af90 msgr2=0x7f314006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:42.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f314006af90 0x7f314006d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3158034000 tx=0x7f3158008030 comp rx=0 tx=0).stop 2023-10-08T05:56:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 -- 172.21.15.70:0/869543620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c0d0ec0 msgr2=0x7f315c157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c0d0ec0 0x7f315c157e00 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f314c00b4a0 tx=0x7f314c00b960 comp rx=0 tx=0).stop 2023-10-08T05:56:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 -- 172.21.15.70:0/869543620 shutdown_connections 2023-10-08T05:56:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f314006af90 0x7f314006d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f315c15cd80 0x7f315c1f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f315c0d0ec0 0x7f315c157e00 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 --2- 172.21.15.70:0/869543620 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f315c0d06c0 0x7f315c0b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 -- 172.21.15.70:0/869543620 >> 172.21.15.70:0/869543620 conn(0x7f315c09bc40 msgr2=0x7f315c154760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:42.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.455+0000 7f3162593640 1 -- 172.21.15.70:0/869543620 shutdown_connections 2023-10-08T05:56:42.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.456+0000 7f3162593640 1 -- 172.21.15.70:0/869543620 wait complete. 2023-10-08T05:56:42.478 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T05:56:42.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.684+0000 7f7b4cdf5640 1 Processor -- start 2023-10-08T05:56:42.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.684+0000 7f7b4cdf5640 1 -- start start 2023-10-08T05:56:42.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.685+0000 7f7b4cdf5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b4815cba0 0x7f7b480b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:42.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.685+0000 7f7b4cdf5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7b4815cf80 0x7f7b480b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:42.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.685+0000 7f7b4cdf5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b480b54a0 con 0x7f7b480b4b40 2023-10-08T05:56:42.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.685+0000 7f7b4cdf5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b480b4fb0 con 0x7f7b4815cf80 2023-10-08T05:56:42.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.685+0000 7f7b4cdf5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480b7130 con 0x7f7b4815cba0 2023-10-08T05:56:42.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.686+0000 7f7b46d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7b480b4b40 0x7f7b480b5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35356/0 (socket says 172.21.15.70:35356) 2023-10-08T05:56:42.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.686+0000 7f7b46d9d640 1 -- 172.21.15.70:0/2826415576 learned_addr learned my addr 172.21.15.70:0/2826415576 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:42.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.686+0000 7f7b4659c640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b4815cba0 0x7f7b480b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:42.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.686+0000 7f7b45d9b640 1 --2- 172.21.15.70:0/2826415576 >> v2:172.21.15.181:3300/0 conn(0x7f7b4815cf80 0x7f7b480b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:42.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.686+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75036222 0 0) 0x7f7b40001490 con 0x7f7b480b4b40 2023-10-08T05:56:42.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.686+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b30003520 con 0x7f7b480b4b40 2023-10-08T05:56:42.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 995412095 0 0) 0x7f7b30003520 con 0x7f7b480b4b40 2023-10-08T05:56:42.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b4815cba0 msgr2=0x7f7b480b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b4815cba0 0x7f7b480b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 >> v2:172.21.15.181:3300/0 conn(0x7f7b4815cf80 msgr2=0x7f7b480b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 --2- 172.21.15.70:0/2826415576 >> v2:172.21.15.181:3300/0 conn(0x7f7b4815cf80 0x7f7b480b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b480b7410 con 0x7f7b480b4b40 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7b40001b80 con 0x7f7b480b4b40 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 >> v1:172.21.15.70:6789/0 conn(0x7f7b480b4b40 legacy=0x7f7b480b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b4815cba0 0x7f7b30003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b30003ef0 0x7f7b300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b30004d00 0x7f7b30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480b4fb0 con 0x7f7b30003ef0 2023-10-08T05:56:42.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480b7130 con 0x7f7b30004d00 2023-10-08T05:56:42.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3000b980 con 0x7f7b4815cba0 2023-10-08T05:56:42.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b45d9b640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b30003ef0 0x7f7b300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:42.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b4659c640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b4815cba0 0x7f7b30003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:42.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.687+0000 7f7b46d9d640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b30004d00 0x7f7b30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:42.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.688+0000 7f7b45d9b640 1 -- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b4815cba0 msgr2=0x7f7b30003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:42.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.688+0000 7f7b45d9b640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b4815cba0 0x7f7b30003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.688+0000 7f7b45d9b640 1 -- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b30004d00 msgr2=0x7f7b30005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:42.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.688+0000 7f7b45d9b640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b30004d00 0x7f7b30005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.688+0000 7f7b45d9b640 1 -- 172.21.15.70:0/2826415576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b30003520 con 0x7f7b30003ef0 2023-10-08T05:56:42.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.689+0000 7f7b45d9b640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b30003ef0 0x7f7b300047c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009ac0 tx=0x7f7b300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd2ee67aefba680 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:42.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.689+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c02c6d0 con 0x7f7b30003ef0 2023-10-08T05:56:42.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.689+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c02c830 con 0x7f7b30003ef0 2023-10-08T05:56:42.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.689+0000 7f7b4559a640 1 -- 172.21.15.70:0/2826415576 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c02cbc0 con 0x7f7b30003ef0 2023-10-08T05:56:42.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.690+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/2826415576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b30003ef0 msgr2=0x7f7b300047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:42.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.690+0000 7f7b4cdf5640 1 --2- 172.21.15.70:0/2826415576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b30003ef0 0x7f7b300047c0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009ac0 tx=0x7f7b300057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.690+0000 7f7b46d9d640 1 -- 172.21.15.70:0/2826415576 reap_dead start 2023-10-08T05:56:42.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.690+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/2826415576 shutdown_connections 2023-10-08T05:56:42.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.690+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/2826415576 >> 172.21.15.70:0/2826415576 conn(0x7f7b480718a0 msgr2=0x7f7b48071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:42.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.690+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/2826415576 shutdown_connections 2023-10-08T05:56:42.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.690+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/2826415576 wait complete. 2023-10-08T05:56:42.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.692+0000 7f7b4cdf5640 1 Processor -- start 2023-10-08T05:56:42.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.692+0000 7f7b4cdf5640 1 -- start start 2023-10-08T05:56:42.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b4cdf5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b30003ef0 0x7f7b480c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:42.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b4cdf5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b480cbf10 0x7f7b480cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:42.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b4cdf5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b481e3dc0 0x7f7b481e4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:42.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b4cdf5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480c95b0 con 0x7f7b481e3dc0 2023-10-08T05:56:42.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b4cdf5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b481e4780 con 0x7f7b480cbf10 2023-10-08T05:56:42.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b4cdf5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b481f56b0 con 0x7f7b30003ef0 2023-10-08T05:56:42.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b46d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b481e3dc0 0x7f7b481e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:42.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b46d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b481e3dc0 0x7f7b481e4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41614/0 (socket says 172.21.15.70:41614) 2023-10-08T05:56:42.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b46d9d640 1 -- 172.21.15.70:0/1393463238 learned_addr learned my addr 172.21.15.70:0/1393463238 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b4659c640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b30003ef0 0x7f7b480c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.693+0000 7f7b45d9b640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b480cbf10 0x7f7b480cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b46d9d640 1 -- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b30003ef0 msgr2=0x7f7b480c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b46d9d640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b30003ef0 0x7f7b480c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b46d9d640 1 -- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b480cbf10 msgr2=0x7f7b480cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b46d9d640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b480cbf10 0x7f7b480cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b46d9d640 1 -- 172.21.15.70:0/1393463238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b481f5990 con 0x7f7b481e3dc0 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b46d9d640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b481e3dc0 0x7f7b481e4240 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f7b480cf560 tx=0x7f7b40007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b36ffd640 1 -- 172.21.15.70:0/1393463238 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4000eb00 con 0x7f7b481e3dc0 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b36ffd640 1 -- 172.21.15.70:0/1393463238 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4000ec60 con 0x7f7b481e3dc0 2023-10-08T05:56:42.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/1393463238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b481f5c70 con 0x7f7b481e3dc0 2023-10-08T05:56:42.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b36ffd640 1 -- 172.21.15.70:0/1393463238 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b40013420 con 0x7f7b481e3dc0 2023-10-08T05:56:42.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.694+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/1393463238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b481f6230 con 0x7f7b481e3dc0 2023-10-08T05:56:42.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.696+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/1393463238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4804e160 con 0x7f7b481e3dc0 2023-10-08T05:56:42.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.697+0000 7f7b36ffd640 1 -- 172.21.15.70:0/1393463238 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 68) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f7b40013580 con 0x7f7b481e3dc0 2023-10-08T05:56:42.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.697+0000 7f7b36ffd640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f7b2806afe0 0x7f7b2806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:42.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.698+0000 7f7b36ffd640 1 -- 172.21.15.70:0/1393463238 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b40089bb0 con 0x7f7b481e3dc0 2023-10-08T05:56:42.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.698+0000 7f7b4659c640 1 -- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f7b2806afe0 msgr2=0x7f7b2806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3857845516 2023-10-08T05:56:42.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.698+0000 7f7b4659c640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f7b2806afe0 0x7f7b2806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:42.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.702+0000 7f7b36ffd640 1 -- 172.21.15.70:0/1393463238 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b400549c0 con 0x7f7b481e3dc0 2023-10-08T05:56:42.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.899+0000 7f7b4659c640 1 -- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f7b2806afe0 msgr2=0x7f7b2806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3857845516 2023-10-08T05:56:42.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.899+0000 7f7b4659c640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f7b2806afe0 0x7f7b2806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:56:42.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.926+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/1393463238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7b480b5470 con 0x7f7b481e3dc0 2023-10-08T05:56:42.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.931+0000 7f7b36ffd640 1 -- 172.21.15.70:0/1393463238 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7b40057fe0 con 0x7f7b481e3dc0 2023-10-08T05:56:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f7b2806afe0 msgr2=0x7f7b2806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:42.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f7b2806afe0 0x7f7b2806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/1393463238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b481e3dc0 msgr2=0x7f7b481e4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b481e3dc0 0x7f7b481e4240 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f7b480cf560 tx=0x7f7b40007bc0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/1393463238 shutdown_connections 2023-10-08T05:56:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f7b2806afe0 0x7f7b2806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b481e3dc0 0x7f7b481e4240 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b480cbf10 0x7f7b480cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 --2- 172.21.15.70:0/1393463238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b30003ef0 0x7f7b480c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/1393463238 >> 172.21.15.70:0/1393463238 conn(0x7f7b480718a0 msgr2=0x7f7b480d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/1393463238 shutdown_connections 2023-10-08T05:56:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:42.934+0000 7f7b4cdf5640 1 -- 172.21.15.70:0/1393463238 wait complete. 2023-10-08T05:56:42.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T05:56:42.992 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:42.991+0000 7f5371e0d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:56:43.034 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:43.034+0000 7feea282b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:56:43.045 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:43.044+0000 7f5371e0d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:56:43.089 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:43.089+0000 7feea282b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:56:43.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.159+0000 7f72fddc4640 1 Processor -- start 2023-10-08T05:56:43.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.159+0000 7f72fddc4640 1 -- start start 2023-10-08T05:56:43.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.160+0000 7f72fddc4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72f80cf2b0 0x7f72f80d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.161+0000 7f72fddc4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f80cd520 con 0x7f72f80cf690 2023-10-08T05:56:43.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.161+0000 7f72fddc4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f80d1910 con 0x7f72f80ce510 2023-10-08T05:56:43.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.161+0000 7f72fddc4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f80ce080 con 0x7f72f80cf2b0 2023-10-08T05:56:43.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.161+0000 7f72effff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f72f80cf690 0x7f72f80d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35370/0 (socket says 172.21.15.70:35370) 2023-10-08T05:56:43.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.161+0000 7f72fcdc2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72f80cf2b0 0x7f72f80d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:43.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.161+0000 7f72effff640 1 -- 172.21.15.70:0/686176608 learned_addr learned my addr 172.21.15.70:0/686176608 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:43.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.161+0000 7f72fcdc2640 1 -- 172.21.15.70:0/686176608 >> v1:172.21.15.181:6789/0 conn(0x7f72f80ce510 legacy=0x7f72f815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:43.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.161+0000 7f72fcdc2640 1 -- 172.21.15.70:0/686176608 >> v1:172.21.15.70:6789/0 conn(0x7f72f80cf690 legacy=0x7f72f80d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:43.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.161+0000 7f72fcdc2640 1 -- 172.21.15.70:0/686176608 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f80cd750 con 0x7f72f80cf2b0 2023-10-08T05:56:43.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.161+0000 7f72ef7fe640 1 -- 172.21.15.70:0/686176608 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618261384 0 0) 0x7f72f80cd520 con 0x7f72f80cf690 2023-10-08T05:56:43.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.162+0000 7f72fcdc2640 1 --2- 172.21.15.70:0/686176608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80cf2b0 0x7f72f80d34d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f72f40097a0 tx=0x7f72f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3f193c92294d67 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.163+0000 7f72ef7fe640 1 -- 172.21.15.70:0/686176608 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f402e070 con 0x7f72f80cf2b0 2023-10-08T05:56:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.163+0000 7f72ef7fe640 1 -- 172.21.15.70:0/686176608 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f40315a0 con 0x7f72f80cf2b0 2023-10-08T05:56:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.163+0000 7f72ef7fe640 1 -- 172.21.15.70:0/686176608 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4031930 con 0x7f72f80cf2b0 2023-10-08T05:56:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.163+0000 7f72fddc4640 1 -- 172.21.15.70:0/686176608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80cf2b0 msgr2=0x7f72f80d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.163+0000 7f72fddc4640 1 --2- 172.21.15.70:0/686176608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80cf2b0 0x7f72f80d34d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f72f40097a0 tx=0x7f72f402b840 comp rx=0 tx=0).stop 2023-10-08T05:56:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.163+0000 7f72fddc4640 1 -- 172.21.15.70:0/686176608 shutdown_connections 2023-10-08T05:56:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.163+0000 7f72fddc4640 1 --2- 172.21.15.70:0/686176608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80cf2b0 0x7f72f80d34d0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.163+0000 7f72fddc4640 1 -- 172.21.15.70:0/686176608 >> 172.21.15.70:0/686176608 conn(0x7f72f809bc40 msgr2=0x7f72f80d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.163+0000 7f72fddc4640 1 -- 172.21.15.70:0/686176608 shutdown_connections 2023-10-08T05:56:43.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.163+0000 7f72fddc4640 1 -- 172.21.15.70:0/686176608 wait complete. 2023-10-08T05:56:43.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.165+0000 7f72fddc4640 1 Processor -- start 2023-10-08T05:56:43.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.165+0000 7f72fddc4640 1 -- start start 2023-10-08T05:56:43.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.166+0000 7f72fddc4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80ce510 0x7f72f815b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.166+0000 7f72fddc4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72f80cf2b0 0x7f72f809dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.166+0000 7f72fddc4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f80cf690 0x7f72f809f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.166+0000 7f72fddc4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f809d970 con 0x7f72f80ce510 2023-10-08T05:56:43.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.166+0000 7f72fddc4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f809ccf0 con 0x7f72f80cf2b0 2023-10-08T05:56:43.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.166+0000 7f72fddc4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f809cfd0 con 0x7f72f80cf690 2023-10-08T05:56:43.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.166+0000 7f72fcdc2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80ce510 0x7f72f815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:43.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.166+0000 7f72fcdc2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80ce510 0x7f72f815b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41626/0 (socket says 172.21.15.70:41626) 2023-10-08T05:56:43.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.166+0000 7f72fcdc2640 1 -- 172.21.15.70:0/3971568248 learned_addr learned my addr 172.21.15.70:0/3971568248 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:43.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72effff640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72f80cf2b0 0x7f72f809dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:43.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72fd5c3640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f80cf690 0x7f72f809f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:43.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72fcdc2640 1 -- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f80cf690 msgr2=0x7f72f809f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:43.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72fcdc2640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f80cf690 0x7f72f809f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72fcdc2640 1 -- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72f80cf2b0 msgr2=0x7f72f809dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:43.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72fcdc2640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72f80cf2b0 0x7f72f809dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72fcdc2640 1 -- 172.21.15.70:0/3971568248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f4009400 con 0x7f72f80ce510 2023-10-08T05:56:43.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72fcdc2640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80ce510 0x7f72f815b050 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f72f4034040 tx=0x7f72f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:43.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72ed7fa640 1 -- 172.21.15.70:0/3971568248 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4032900 con 0x7f72f80ce510 2023-10-08T05:56:43.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72ed7fa640 1 -- 172.21.15.70:0/3971568248 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f4032a60 con 0x7f72f80ce510 2023-10-08T05:56:43.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.167+0000 7f72ed7fa640 1 -- 172.21.15.70:0/3971568248 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4032e10 con 0x7f72f80ce510 2023-10-08T05:56:43.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.168+0000 7f72fddc4640 1 -- 172.21.15.70:0/3971568248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f809d2b0 con 0x7f72f80ce510 2023-10-08T05:56:43.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.168+0000 7f72fddc4640 1 -- 172.21.15.70:0/3971568248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f809c910 con 0x7f72f80ce510 2023-10-08T05:56:43.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.169+0000 7f72fddc4640 1 -- 172.21.15.70:0/3971568248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f809dbe0 con 0x7f72f80ce510 2023-10-08T05:56:43.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.170+0000 7f72ed7fa640 1 -- 172.21.15.70:0/3971568248 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 68) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f72f403b3f0 con 0x7f72f80ce510 2023-10-08T05:56:43.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.171+0000 7f72ed7fa640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f72e406afe0 0x7f72e406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.171+0000 7f72ed7fa640 1 -- 172.21.15.70:0/3971568248 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72f40abb60 con 0x7f72f80ce510 2023-10-08T05:56:43.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.171+0000 7f72effff640 1 -- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f72e406afe0 msgr2=0x7f72e406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3857845516 2023-10-08T05:56:43.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.171+0000 7f72effff640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f72e406afe0 0x7f72e406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:43.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.175+0000 7f72ed7fa640 1 -- 172.21.15.70:0/3971568248 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72f4076860 con 0x7f72f80ce510 2023-10-08T05:56:43.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.364+0000 7f72fddc4640 1 -- 172.21.15.70:0/3971568248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f72f8074150 con 0x7f72f80ce510 2023-10-08T05:56:43.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.369+0000 7f72ed7fa640 1 -- 172.21.15.70:0/3971568248 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7f72f4079e80 con 0x7f72f80ce510 2023-10-08T05:56:43.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 -- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f72e406afe0 msgr2=0x7f72e406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f72e406afe0 0x7f72e406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 -- 172.21.15.70:0/3971568248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80ce510 msgr2=0x7f72f815b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80ce510 0x7f72f815b050 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f72f4034040 tx=0x7f72f40046c0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 -- 172.21.15.70:0/3971568248 shutdown_connections 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7f72e406afe0 0x7f72e406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72f80cf690 0x7f72f809f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72f80cf2b0 0x7f72f809dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 --2- 172.21.15.70:0/3971568248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72f80ce510 0x7f72f815b050 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 -- 172.21.15.70:0/3971568248 >> 172.21.15.70:0/3971568248 conn(0x7f72f809bc40 msgr2=0x7f72f815fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 -- 172.21.15.70:0/3971568248 shutdown_connections 2023-10-08T05:56:43.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.372+0000 7f72fddc4640 1 -- 172.21.15.70:0/3971568248 wait complete. 2023-10-08T05:56:43.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:43.391 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/minutes) 2023-10-08T05:56:43.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/minutes 1m 2023-10-08T05:56:43.494 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:43.493+0000 7f5371e0d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:56:43.545 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:43.544+0000 7f5371e0d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:56:43.561 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:43.560+0000 7feea282b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:56:43.596 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:43.595+0000 7f5371e0d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:56:43.616 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:43.615+0000 7feea282b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:56:43.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.641+0000 7fa70999e640 1 Processor -- start 2023-10-08T05:56:43.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.641+0000 7fa70999e640 1 -- start start 2023-10-08T05:56:43.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.642+0000 7fa70999e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa70415cea0 0x7fa7040cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.643+0000 7fa70999e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa7040b2c90 0x7fa7040cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.643+0000 7fa70999e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7040cf930 con 0x7fa70406bf70 2023-10-08T05:56:43.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.643+0000 7fa70999e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70415d280 con 0x7fa70415cea0 2023-10-08T05:56:43.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.643+0000 7fa70999e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70406c710 con 0x7fa7040b2c90 2023-10-08T05:56:43.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.643+0000 7fa702ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa70415cea0 0x7fa7040cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:43.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.643+0000 7fa702ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa70415cea0 0x7fa7040cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49340/0 (socket says 172.21.15.70:49340) 2023-10-08T05:56:43.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.643+0000 7fa702ffd640 1 -- 172.21.15.70:0/3968113907 learned_addr learned my addr 172.21.15.70:0/3968113907 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:43.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.643+0000 7fa7037fe640 1 --2- 172.21.15.70:0/3968113907 >> v2:172.21.15.181:3300/0 conn(0x7fa7040b2c90 0x7fa7040cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:43.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.644+0000 7fa702ffd640 1 -- 172.21.15.70:0/3968113907 >> v2:172.21.15.181:3300/0 conn(0x7fa7040b2c90 msgr2=0x7fa7040cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:43.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.644+0000 7fa702ffd640 1 --2- 172.21.15.70:0/3968113907 >> v2:172.21.15.181:3300/0 conn(0x7fa7040b2c90 0x7fa7040cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.644+0000 7fa702ffd640 1 -- 172.21.15.70:0/3968113907 >> v1:172.21.15.181:6789/0 conn(0x7fa70406bf70 legacy=0x7fa7040b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:43.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.644+0000 7fa702ffd640 1 -- 172.21.15.70:0/3968113907 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70406c9f0 con 0x7fa70415cea0 2023-10-08T05:56:43.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.645+0000 7fa702ffd640 1 --2- 172.21.15.70:0/3968113907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa70415cea0 0x7fa7040cf3f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa7040721d0 tx=0x7fa6f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f280ef4193b77851 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:43.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.645+0000 7fa701ffb640 1 -- 172.21.15.70:0/3968113907 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4031870 con 0x7fa70415cea0 2023-10-08T05:56:43.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.645+0000 7fa701ffb640 1 -- 172.21.15.70:0/3968113907 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40319d0 con 0x7fa70415cea0 2023-10-08T05:56:43.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.645+0000 7fa701ffb640 1 -- 172.21.15.70:0/3968113907 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4031d60 con 0x7fa70415cea0 2023-10-08T05:56:43.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.646+0000 7fa70999e640 1 -- 172.21.15.70:0/3968113907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa70415cea0 msgr2=0x7fa7040cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:43.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.646+0000 7fa70999e640 1 --2- 172.21.15.70:0/3968113907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa70415cea0 0x7fa7040cf3f0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa7040721d0 tx=0x7fa6f402b910 comp rx=0 tx=0).stop 2023-10-08T05:56:43.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.646+0000 7fa70999e640 1 -- 172.21.15.70:0/3968113907 shutdown_connections 2023-10-08T05:56:43.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.646+0000 7fa70999e640 1 --2- 172.21.15.70:0/3968113907 >> v2:172.21.15.181:3300/0 conn(0x7fa7040b2c90 0x7fa7040cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.646+0000 7fa70999e640 1 --2- 172.21.15.70:0/3968113907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa70415cea0 0x7fa7040cf3f0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.646+0000 7fa70999e640 1 -- 172.21.15.70:0/3968113907 >> 172.21.15.70:0/3968113907 conn(0x7fa70406e670 msgr2=0x7fa70406ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:43.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.646+0000 7fa70999e640 1 -- 172.21.15.70:0/3968113907 shutdown_connections 2023-10-08T05:56:43.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.646+0000 7fa70999e640 1 -- 172.21.15.70:0/3968113907 wait complete. 2023-10-08T05:56:43.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.647+0000 7fa70999e640 1 Processor -- start 2023-10-08T05:56:43.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.648+0000 7fa70999e640 1 -- start start 2023-10-08T05:56:43.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.648+0000 7fa70999e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa70406bf70 0x7fa7040b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa70999e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7040b2c90 0x7fa7040c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa70999e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa70415cea0 0x7fa70415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa70999e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7040c8110 con 0x7fa70406bf70 2023-10-08T05:56:43.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa70999e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7040c8340 con 0x7fa70415cea0 2023-10-08T05:56:43.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa70999e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7040c8620 con 0x7fa7040b2c90 2023-10-08T05:56:43.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa702ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa70406bf70 0x7fa7040b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:43.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa702ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa70406bf70 0x7fa7040b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49352/0 (socket says 172.21.15.70:49352) 2023-10-08T05:56:43.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa702ffd640 1 -- 172.21.15.70:0/515028253 learned_addr learned my addr 172.21.15.70:0/515028253 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:43.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa7027fc640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7040b2c90 0x7fa7040c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:43.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa702ffd640 1 -- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7040b2c90 msgr2=0x7fa7040c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:43.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa7037fe640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa70415cea0 0x7fa70415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:43.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa702ffd640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7040b2c90 0x7fa7040c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa702ffd640 1 -- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa70415cea0 msgr2=0x7fa70415a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:43.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa702ffd640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa70415cea0 0x7fa70415a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:43.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa702ffd640 1 -- 172.21.15.70:0/515028253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f40094d0 con 0x7fa70406bf70 2023-10-08T05:56:43.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.649+0000 7fa7027fc640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa7040b2c90 0x7fa7040c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:43.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.650+0000 7fa702ffd640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa70406bf70 0x7fa7040b4280 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4000c00 tx=0x7fa6f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:43.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.650+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f403abe0 con 0x7fa70406bf70 2023-10-08T05:56:43.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.650+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f403ad40 con 0x7fa70406bf70 2023-10-08T05:56:43.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.650+0000 7fa70999e640 1 -- 172.21.15.70:0/515028253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7040c8900 con 0x7fa70406bf70 2023-10-08T05:56:43.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.650+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4044610 con 0x7fa70406bf70 2023-10-08T05:56:43.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.650+0000 7fa70999e640 1 -- 172.21.15.70:0/515028253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7040c8ec0 con 0x7fa70406bf70 2023-10-08T05:56:43.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.651+0000 7fa70999e640 1 -- 172.21.15.70:0/515028253 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70406c3e0 con 0x7fa70406bf70 2023-10-08T05:56:43.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.652+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 68) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fa6f404d450 con 0x7fa70406bf70 2023-10-08T05:56:43.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.653+0000 7fa6ebfff640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 0x7fa6f006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:43.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.653+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa6f402d040 con 0x7fa70406bf70 2023-10-08T05:56:43.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.653+0000 7fa7027fc640 1 -- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 msgr2=0x7fa6f006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3857845516 2023-10-08T05:56:43.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.653+0000 7fa7027fc640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 0x7fa6f006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:43.657 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:43.656+0000 7f5371e0d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:56:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.657+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa70406c3e0 con 0x7fa70406bf70 2023-10-08T05:56:43.669 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:43.669+0000 7feea282b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:56:43.706 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:43.705+0000 7f5371e0d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:56:43.731 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:43.730+0000 7feea282b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:56:43.780 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:43.780+0000 7feea282b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:56:43.817 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:43.816+0000 7f5371e0d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:56:43.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.854+0000 7fa7027fc640 1 -- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 msgr2=0x7fa6f006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3857845516 2023-10-08T05:56:43.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.854+0000 7fa7027fc640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 0x7fa6f006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:56:43.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:43.868+0000 7fa70999e640 1 -- 172.21.15.70:0/515028253 --> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7040cb7b0 con 0x7fa6f006afe0 2023-10-08T05:56:43.879 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:43.878+0000 7f5371e0d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:43.893 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:43.892+0000 7feea282b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:56:43.928 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:43.927+0000 7f5371e0d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:56:43.957 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:43.956+0000 7feea282b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:43.985 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:43.985+0000 7f5371e0d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:56:44.006 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:44.005+0000 7feea282b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:56:44.035 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:44.034+0000 7f5371e0d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:56:44.061 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:44.061+0000 7feea282b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:56:44.111 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:44.111+0000 7feea282b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:56:44.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:44.255+0000 7fa7027fc640 1 -- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 msgr2=0x7fa6f006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3857845516 2023-10-08T05:56:44.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:44.255+0000 7fa7027fc640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 0x7fa6f006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:56:44.414 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:44.413+0000 7f5371e0d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:56:44.490 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:44.489+0000 7feea282b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:56:44.573 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:44.572+0000 7f5371e0d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:56:44.648 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:44.648+0000 7feea282b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:56:44.681 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:44.680+0000 7f5371e0d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:56:44.730 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:44.729+0000 7f5371e0d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:56:44.755 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:44.754+0000 7feea282b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:56:44.804 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:44.804+0000 7feea282b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:56:44.837 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:44.836+0000 7f5371e0d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:44.910 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:44.910+0000 7feea282b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:44.944 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:44.943+0000 7f5371e0d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:56:45.018 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:45.017+0000 7feea282b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:56:45.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:45.056+0000 7fa7027fc640 1 -- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 msgr2=0x7fa6f006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3857845516 2023-10-08T05:56:45.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:45.056+0000 7fa7027fc640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 0x7fa6f006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:56:45.102 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:45.100+0000 7f5371e0d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:45.151 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:45.150+0000 7f5371e0d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:45.172 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:45.172+0000 7feea282b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:45.221 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:45.221+0000 7feea282b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:45.268 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:45.267+0000 7f5371e0d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:56:45.338 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:45.338+0000 7feea282b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:56:46.097 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.096+0000 7f5371e0d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:56:46.157 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:46.157+0000 7feea282b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:56:46.392 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.391+0000 7f5371e0d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:56:46.448 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.447+0000 7f5371e0d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:56:46.452 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:46.451+0000 7feea282b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:56:46.508 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:46.508+0000 7feea282b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:56:46.561 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.560+0000 7f5371e0d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:56:46.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:46.584+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 69) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fa6f40aa300 con 0x7fa70406bf70 2023-10-08T05:56:46.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:46.584+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 msgr2=0x7fa6f006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:46.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:46.584+0000 7fa6ebfff640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/3857845516,v1:172.21.15.181:6833/3857845516] conn(0x7fa6f006afe0 0x7fa6f006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:46.629 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:46.628+0000 7feea282b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:56:46.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.726+0000 7f52ee5a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:46.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.726+0000 7f52ee5a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:46.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.726+0000 7f52ee5a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:46.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.726+0000 7f52ee5a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:46.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.726+0000 7f52ee5a6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:46.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.726+0000 7f52f15ec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:46.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.726+0000 7f52f15ec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:46.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.726+0000 7f52f15ec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:46.728 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.726+0000 7f52f15ec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:46.729 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:46.726+0000 7f52f15ec640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:47.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.596+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 70) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fa6f4076fb0 con 0x7fa70406bf70 2023-10-08T05:56:47.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.597+0000 7fa6ebfff640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa6f0035880 0x7fa6f00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:47.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.597+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/minutes", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6f0035c60 con 0x7fa6f0035880 2023-10-08T05:56:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.604+0000 7fa7027fc640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa6f0035880 0x7fa6f00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:47.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.610+0000 7fa7027fc640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa6f0035880 0x7fa6f00754a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa7040b4b20 tx=0x7fa6f8007db0 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:47.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.760+0000 7fa6ebfff640 1 -- 172.21.15.70:0/515028253 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7fa6f0035c60 con 0x7fa6f0035880 2023-10-08T05:56:47.761 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1/minutes 2023-10-08T05:56:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.763+0000 7fa70999e640 1 -- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa6f0035880 msgr2=0x7fa6f00754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.763+0000 7fa70999e640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa6f0035880 0x7fa6f00754a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa7040b4b20 tx=0x7fa6f8007db0 comp rx=0 tx=0).stop 2023-10-08T05:56:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.763+0000 7fa70999e640 1 -- 172.21.15.70:0/515028253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa70406bf70 msgr2=0x7fa7040b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.763+0000 7fa70999e640 1 --2- 172.21.15.70:0/515028253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa70406bf70 0x7fa7040b4280 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4000c00 tx=0x7fa6f40046c0 comp rx=0 tx=0).stop 2023-10-08T05:56:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.763+0000 7fa7037fe640 1 -- 172.21.15.70:0/515028253 reap_dead start 2023-10-08T05:56:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.763+0000 7fa70999e640 1 -- 172.21.15.70:0/515028253 shutdown_connections 2023-10-08T05:56:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.763+0000 7fa70999e640 1 -- 172.21.15.70:0/515028253 >> 172.21.15.70:0/515028253 conn(0x7fa70406e670 msgr2=0x7fa7040b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:47.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.763+0000 7fa70999e640 1 -- 172.21.15.70:0/515028253 shutdown_connections 2023-10-08T05:56:47.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:47.764+0000 7fa70999e640 1 -- 172.21.15.70:0/515028253 wait complete. 2023-10-08T05:56:47.785 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/minutes 2023-10-08T05:56:47.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:47.785 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/hourly) 2023-10-08T05:56:47.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/hourly 1h 2023-10-08T05:56:48.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.059+0000 7f0b437e7640 1 Processor -- start 2023-10-08T05:56:48.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.059+0000 7f0b437e7640 1 -- start start 2023-10-08T05:56:48.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.061+0000 7f0b437e7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0b3c15aad0 0x7f0b3c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.061+0000 7f0b437e7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c15bf30 con 0x7f0b3c0c82b0 2023-10-08T05:56:48.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.061+0000 7f0b437e7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c15b320 con 0x7f0b3c06d410 2023-10-08T05:56:48.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.061+0000 7f0b437e7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c15b600 con 0x7f0b3c15aad0 2023-10-08T05:56:48.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.061+0000 7f0b40d82640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0b3c0c82b0 0x7f0b3c06cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43574/0 (socket says 172.21.15.70:43574) 2023-10-08T05:56:48.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.061+0000 7f0b41583640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0b3c15aad0 0x7f0b3c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:48.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.061+0000 7f0b40d82640 1 -- 172.21.15.70:0/1805786157 learned_addr learned my addr 172.21.15.70:0/1805786157 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:48.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.062+0000 7f0b41583640 1 -- 172.21.15.70:0/1805786157 >> v1:172.21.15.181:6789/0 conn(0x7f0b3c06d410 legacy=0x7f0b3c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:48.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.062+0000 7f0b41583640 1 -- 172.21.15.70:0/1805786157 >> v1:172.21.15.70:6789/0 conn(0x7f0b3c0c82b0 legacy=0x7f0b3c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:48.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.062+0000 7f0b41583640 1 -- 172.21.15.70:0/1805786157 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c15b8e0 con 0x7f0b3c15aad0 2023-10-08T05:56:48.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.063+0000 7f0b41583640 1 --2- 172.21.15.70:0/1805786157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c15aad0 0x7f0b3c0b1750 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f0b380097a0 tx=0x7f0b3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958c89bf251ef327 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.063+0000 7f0b33fff640 1 -- 172.21.15.70:0/1805786157 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38031830 con 0x7f0b3c15aad0 2023-10-08T05:56:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.063+0000 7f0b33fff640 1 -- 172.21.15.70:0/1805786157 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38031990 con 0x7f0b3c15aad0 2023-10-08T05:56:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.063+0000 7f0b33fff640 1 -- 172.21.15.70:0/1805786157 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38031d20 con 0x7f0b3c15aad0 2023-10-08T05:56:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.064+0000 7f0b437e7640 1 -- 172.21.15.70:0/1805786157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c15aad0 msgr2=0x7f0b3c0b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.064+0000 7f0b437e7640 1 --2- 172.21.15.70:0/1805786157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c15aad0 0x7f0b3c0b1750 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f0b380097a0 tx=0x7f0b3802b8d0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.064+0000 7f0b437e7640 1 -- 172.21.15.70:0/1805786157 shutdown_connections 2023-10-08T05:56:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.064+0000 7f0b437e7640 1 --2- 172.21.15.70:0/1805786157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c15aad0 0x7f0b3c0b1750 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.064+0000 7f0b437e7640 1 -- 172.21.15.70:0/1805786157 >> 172.21.15.70:0/1805786157 conn(0x7f0b3c069d50 msgr2=0x7f0b3c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:48.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.064+0000 7f0b437e7640 1 -- 172.21.15.70:0/1805786157 shutdown_connections 2023-10-08T05:56:48.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.064+0000 7f0b437e7640 1 -- 172.21.15.70:0/1805786157 wait complete. 2023-10-08T05:56:48.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.066+0000 7f0b437e7640 1 Processor -- start 2023-10-08T05:56:48.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.066+0000 7f0b437e7640 1 -- start start 2023-10-08T05:56:48.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.067+0000 7f0b437e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c06d410 0x7f0b3c0cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.067+0000 7f0b437e7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b3c0c82b0 0x7f0b3c14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.067+0000 7f0b437e7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b3c15aad0 0x7f0b3c14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.067+0000 7f0b437e7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0cd240 con 0x7f0b3c06d410 2023-10-08T05:56:48.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.067+0000 7f0b437e7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c1f2690 con 0x7f0b3c0c82b0 2023-10-08T05:56:48.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.067+0000 7f0b437e7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0cdba0 con 0x7f0b3c15aad0 2023-10-08T05:56:48.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.067+0000 7f0b41583640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c06d410 0x7f0b3c0cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:48.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.067+0000 7f0b41583640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c06d410 0x7f0b3c0cd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49376/0 (socket says 172.21.15.70:49376) 2023-10-08T05:56:48.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.067+0000 7f0b41583640 1 -- 172.21.15.70:0/3095992658 learned_addr learned my addr 172.21.15.70:0/3095992658 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:48.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.068+0000 7f0b40d82640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b3c0c82b0 0x7f0b3c14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:48.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.068+0000 7f0b41583640 1 -- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b3c15aad0 msgr2=0x7f0b3c14e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:48.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.068+0000 7f0b41583640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b3c15aad0 0x7f0b3c14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.068+0000 7f0b41583640 1 -- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b3c0c82b0 msgr2=0x7f0b3c14e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:48.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.068+0000 7f0b41583640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b3c0c82b0 0x7f0b3c14e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.068+0000 7f0b41583640 1 -- 172.21.15.70:0/3095992658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b38009400 con 0x7f0b3c06d410 2023-10-08T05:56:48.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.068+0000 7f0b41583640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c06d410 0x7f0b3c0cd660 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0b38000c00 tx=0x7f0b380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:48.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.068+0000 7f0b31ffb640 1 -- 172.21.15.70:0/3095992658 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3803aa60 con 0x7f0b3c06d410 2023-10-08T05:56:48.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.069+0000 7f0b31ffb640 1 -- 172.21.15.70:0/3095992658 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3803abc0 con 0x7f0b3c06d410 2023-10-08T05:56:48.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.069+0000 7f0b31ffb640 1 -- 172.21.15.70:0/3095992658 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b380443f0 con 0x7f0b3c06d410 2023-10-08T05:56:48.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.069+0000 7f0b437e7640 1 -- 172.21.15.70:0/3095992658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c0cc5c0 con 0x7f0b3c06d410 2023-10-08T05:56:48.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.069+0000 7f0b437e7640 1 -- 172.21.15.70:0/3095992658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c0ccb80 con 0x7f0b3c06d410 2023-10-08T05:56:48.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.070+0000 7f0b437e7640 1 -- 172.21.15.70:0/3095992658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c04e160 con 0x7f0b3c06d410 2023-10-08T05:56:48.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.071+0000 7f0b31ffb640 1 -- 172.21.15.70:0/3095992658 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 70) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f0b3804d450 con 0x7f0b3c06d410 2023-10-08T05:56:48.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.072+0000 7f0b31ffb640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f0b2006ad10 0x7f0b2006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.072+0000 7f0b31ffb640 1 -- 172.21.15.70:0/3095992658 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f0b380ab760 con 0x7f0b3c06d410 2023-10-08T05:56:48.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.072+0000 7f0b40d82640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f0b2006ad10 0x7f0b2006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:48.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.074+0000 7f0b40d82640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f0b2006ad10 0x7f0b2006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c1f5c50 tx=0x7f0b2c009040 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:48.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.076+0000 7f0b31ffb640 1 -- 172.21.15.70:0/3095992658 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b38076870 con 0x7f0b3c06d410 2023-10-08T05:56:48.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.286+0000 7f0b437e7640 1 -- 172.21.15.70:0/3095992658 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/hourly", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b3c15bbc0 con 0x7f0b2006ad10 2023-10-08T05:56:48.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.439+0000 7f0b31ffb640 1 -- 172.21.15.70:0/3095992658 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7f0b2c001320 con 0x7f0b2006ad10 2023-10-08T05:56:48.440 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1/hourly 2023-10-08T05:56:48.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.442+0000 7f0b437e7640 1 -- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f0b2006ad10 msgr2=0x7f0b2006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:48.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.442+0000 7f0b437e7640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f0b2006ad10 0x7f0b2006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c1f5c50 tx=0x7f0b2c009040 comp rx=0 tx=0).stop 2023-10-08T05:56:48.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.442+0000 7f0b437e7640 1 -- 172.21.15.70:0/3095992658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c06d410 msgr2=0x7f0b3c0cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:48.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.442+0000 7f0b437e7640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c06d410 0x7f0b3c0cd660 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0b38000c00 tx=0x7f0b380046c0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.443+0000 7f0b437e7640 1 -- 172.21.15.70:0/3095992658 shutdown_connections 2023-10-08T05:56:48.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.443+0000 7f0b437e7640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f0b2006ad10 0x7f0b2006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.443+0000 7f0b437e7640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b3c15aad0 0x7f0b3c14e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.443+0000 7f0b437e7640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b3c0c82b0 0x7f0b3c14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.443+0000 7f0b437e7640 1 --2- 172.21.15.70:0/3095992658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b3c06d410 0x7f0b3c0cd660 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.443+0000 7f0b437e7640 1 -- 172.21.15.70:0/3095992658 >> 172.21.15.70:0/3095992658 conn(0x7f0b3c069d50 msgr2=0x7f0b3c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:48.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.443+0000 7f0b437e7640 1 -- 172.21.15.70:0/3095992658 shutdown_connections 2023-10-08T05:56:48.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.443+0000 7f0b437e7640 1 -- 172.21.15.70:0/3095992658 wait complete. 2023-10-08T05:56:48.466 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/hourly 2023-10-08T05:56:48.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:48.466 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/daily) 2023-10-08T05:56:48.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/daily 1d 2023-10-08T05:56:48.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.735+0000 7f65b3fb0640 1 Processor -- start 2023-10-08T05:56:48.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.735+0000 7f65b3fb0640 1 -- start start 2023-10-08T05:56:48.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.737+0000 7f65b3fb0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65ac0cb880 0x7f65ac0b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.737+0000 7f65b3fb0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65ac0b4070 0x7f65ac159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.737+0000 7f65b3fb0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ac15d440 con 0x7f65ac1432f0 2023-10-08T05:56:48.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.737+0000 7f65b3fb0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac1436d0 con 0x7f65ac0cb880 2023-10-08T05:56:48.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.737+0000 7f65b3fb0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac15d690 con 0x7f65ac0b4070 2023-10-08T05:56:48.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.737+0000 7f65b1d4c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f65ac1432f0 0x7f65ac0cb170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43588/0 (socket says 172.21.15.70:43588) 2023-10-08T05:56:48.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.737+0000 7f65b154b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65ac0cb880 0x7f65ac0b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:48.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.737+0000 7f65b1d4c640 1 -- 172.21.15.70:0/4055396507 learned_addr learned my addr 172.21.15.70:0/4055396507 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:48.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.738+0000 7f65b254d640 1 --2- 172.21.15.70:0/4055396507 >> v2:172.21.15.181:3300/0 conn(0x7f65ac0b4070 0x7f65ac159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:48.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.738+0000 7f65b154b640 1 -- 172.21.15.70:0/4055396507 >> v2:172.21.15.181:3300/0 conn(0x7f65ac0b4070 msgr2=0x7f65ac159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:48.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.738+0000 7f65b154b640 1 --2- 172.21.15.70:0/4055396507 >> v2:172.21.15.181:3300/0 conn(0x7f65ac0b4070 0x7f65ac159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.738+0000 7f65b154b640 1 -- 172.21.15.70:0/4055396507 >> v1:172.21.15.70:6789/0 conn(0x7f65ac1432f0 legacy=0x7f65ac0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:48.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.738+0000 7f65b154b640 1 -- 172.21.15.70:0/4055396507 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac15d970 con 0x7f65ac0cb880 2023-10-08T05:56:48.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.738+0000 7f65b0d4a640 1 -- 172.21.15.70:0/4055396507 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136799304 0 0) 0x7f65ac15d440 con 0x7f65ac1432f0 2023-10-08T05:56:48.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.738+0000 7f65b254d640 1 --2- 172.21.15.70:0/4055396507 >> v2:172.21.15.181:3300/0 conn(0x7f65ac0b4070 0x7f65ac159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:56:48.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.739+0000 7f65b154b640 1 --2- 172.21.15.70:0/4055396507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0cb880 0x7f65ac0b3b30 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f65ac157a20 tx=0x7f659c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c86921fdbae653a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:48.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.739+0000 7f65b0d4a640 1 -- 172.21.15.70:0/4055396507 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c02e070 con 0x7f65ac0cb880 2023-10-08T05:56:48.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.739+0000 7f65b0d4a640 1 -- 172.21.15.70:0/4055396507 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c0315a0 con 0x7f65ac0cb880 2023-10-08T05:56:48.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.739+0000 7f65b0d4a640 1 -- 172.21.15.70:0/4055396507 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c031930 con 0x7f65ac0cb880 2023-10-08T05:56:48.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.740+0000 7f65b3fb0640 1 -- 172.21.15.70:0/4055396507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0cb880 msgr2=0x7f65ac0b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:48.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.740+0000 7f65b3fb0640 1 --2- 172.21.15.70:0/4055396507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0cb880 0x7f65ac0b3b30 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f65ac157a20 tx=0x7f659c02b840 comp rx=0 tx=0).stop 2023-10-08T05:56:48.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.740+0000 7f65b3fb0640 1 -- 172.21.15.70:0/4055396507 shutdown_connections 2023-10-08T05:56:48.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.740+0000 7f65b3fb0640 1 --2- 172.21.15.70:0/4055396507 >> v2:172.21.15.181:3300/0 conn(0x7f65ac0b4070 0x7f65ac159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.740+0000 7f65b3fb0640 1 --2- 172.21.15.70:0/4055396507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0cb880 0x7f65ac0b3b30 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.740+0000 7f65b3fb0640 1 -- 172.21.15.70:0/4055396507 >> 172.21.15.70:0/4055396507 conn(0x7f65ac069d50 msgr2=0x7f65ac06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:48.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.740+0000 7f65b3fb0640 1 -- 172.21.15.70:0/4055396507 shutdown_connections 2023-10-08T05:56:48.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.740+0000 7f65b3fb0640 1 -- 172.21.15.70:0/4055396507 wait complete. 2023-10-08T05:56:48.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.742+0000 7f65b3fb0640 1 Processor -- start 2023-10-08T05:56:48.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.742+0000 7f65b3fb0640 1 -- start start 2023-10-08T05:56:48.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.743+0000 7f65b3fb0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0b4070 0x7f65ac157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.743+0000 7f65b3fb0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65ac0cb880 0x7f65ac134090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.743+0000 7f65b3fb0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac1432f0 0x7f65ac159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.743+0000 7f65b3fb0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac137990 con 0x7f65ac0b4070 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.743+0000 7f65b3fb0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac157890 con 0x7f65ac1432f0 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.743+0000 7f65b3fb0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac136d80 con 0x7f65ac0cb880 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.743+0000 7f65b1d4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0b4070 0x7f65ac157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.744+0000 7f65b1d4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0b4070 0x7f65ac157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49382/0 (socket says 172.21.15.70:49382) 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.744+0000 7f65b1d4c640 1 -- 172.21.15.70:0/2171342344 learned_addr learned my addr 172.21.15.70:0/2171342344 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.744+0000 7f65b254d640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac1432f0 0x7f65ac159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.744+0000 7f65b154b640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65ac0cb880 0x7f65ac134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.744+0000 7f65b1d4c640 1 -- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65ac0cb880 msgr2=0x7f65ac134090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.744+0000 7f65b1d4c640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65ac0cb880 0x7f65ac134090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.744+0000 7f65b1d4c640 1 -- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac1432f0 msgr2=0x7f65ac159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:48.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.744+0000 7f65b1d4c640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac1432f0 0x7f65ac159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:48.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.744+0000 7f65b1d4c640 1 -- 172.21.15.70:0/2171342344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c009400 con 0x7f65ac0b4070 2023-10-08T05:56:48.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.744+0000 7f65b1d4c640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0b4070 0x7f65ac157350 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f65a800c430 tx=0x7f65a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:48.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.745+0000 7f65a27fc640 1 -- 172.21.15.70:0/2171342344 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a80074e0 con 0x7f65ac0b4070 2023-10-08T05:56:48.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.745+0000 7f65a27fc640 1 -- 172.21.15.70:0/2171342344 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a8007660 con 0x7f65ac0b4070 2023-10-08T05:56:48.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.745+0000 7f65a27fc640 1 -- 172.21.15.70:0/2171342344 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8004a20 con 0x7f65ac0b4070 2023-10-08T05:56:48.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.745+0000 7f65b3fb0640 1 -- 172.21.15.70:0/2171342344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ac137060 con 0x7f65ac0b4070 2023-10-08T05:56:48.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.745+0000 7f65b3fb0640 1 -- 172.21.15.70:0/2171342344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ac137620 con 0x7f65ac0b4070 2023-10-08T05:56:48.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.746+0000 7f65b3fb0640 1 -- 172.21.15.70:0/2171342344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ac04e160 con 0x7f65ac0b4070 2023-10-08T05:56:48.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.747+0000 7f65a27fc640 1 -- 172.21.15.70:0/2171342344 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 71) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f65a8004b80 con 0x7f65ac0b4070 2023-10-08T05:56:48.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.749+0000 7f65a27fc640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f659006afe0 0x7f659006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:48.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.749+0000 7f65a27fc640 1 -- 172.21.15.70:0/2171342344 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f65a80897c0 con 0x7f65ac0b4070 2023-10-08T05:56:48.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.749+0000 7f65b154b640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f659006afe0 0x7f659006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:48.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.750+0000 7f65b154b640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f659006afe0 0x7f659006d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f659c036000 tx=0x7f659c03d040 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:48.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.753+0000 7f65a27fc640 1 -- 172.21.15.70:0/2171342344 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65a8054520 con 0x7f65ac0b4070 2023-10-08T05:56:48.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:48.966+0000 7f65b3fb0640 1 -- 172.21.15.70:0/2171342344 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/daily", "snap_schedule": "1d", "target": ["mon-mgr", ""]}) v1 -- 0x7f65ac15df30 con 0x7f659006afe0 2023-10-08T05:56:49.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.126+0000 7f65a27fc640 1 -- 172.21.15.70:0/2171342344 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f659c0319f0 con 0x7f659006afe0 2023-10-08T05:56:49.127 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1/daily 2023-10-08T05:56:49.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 -- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f659006afe0 msgr2=0x7f659006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:49.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f659006afe0 0x7f659006d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f659c036000 tx=0x7f659c03d040 comp rx=0 tx=0).stop 2023-10-08T05:56:49.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 -- 172.21.15.70:0/2171342344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0b4070 msgr2=0x7f65ac157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:49.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0b4070 0x7f65ac157350 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f65a800c430 tx=0x7f65a800c8f0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 -- 172.21.15.70:0/2171342344 shutdown_connections 2023-10-08T05:56:49.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f659006afe0 0x7f659006d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65ac1432f0 0x7f65ac159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65ac0cb880 0x7f65ac134090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 --2- 172.21.15.70:0/2171342344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65ac0b4070 0x7f65ac157350 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 -- 172.21.15.70:0/2171342344 >> 172.21.15.70:0/2171342344 conn(0x7f65ac069d50 msgr2=0x7f65ac0701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:49.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 -- 172.21.15.70:0/2171342344 shutdown_connections 2023-10-08T05:56:49.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.130+0000 7f65b3fb0640 1 -- 172.21.15.70:0/2171342344 wait complete. 2023-10-08T05:56:49.152 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/daily 2023-10-08T05:56:49.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:49.153 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/weekly) 2023-10-08T05:56:49.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/weekly 1w 2023-10-08T05:56:49.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.427+0000 7fc603b0f640 1 Processor -- start 2023-10-08T05:56:49.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.428+0000 7fc603b0f640 1 -- start start 2023-10-08T05:56:49.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.429+0000 7fc603b0f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc5fc15aad0 0x7fc5fc0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.429+0000 7fc603b0f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5fc06d410 0x7fc5fc0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc603b0f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5fc15bf30 con 0x7fc5fc0c82b0 2023-10-08T05:56:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc603b0f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5fc15b320 con 0x7fc5fc06d410 2023-10-08T05:56:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc603b0f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5fc15b600 con 0x7fc5fc15aad0 2023-10-08T05:56:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc6010aa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc5fc0c82b0 0x7fc5fc06cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43592/0 (socket says 172.21.15.70:43592) 2023-10-08T05:56:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc6010aa640 1 -- 172.21.15.70:0/1828025781 learned_addr learned my addr 172.21.15.70:0/1828025781 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc6020ac640 1 --2- 172.21.15.70:0/1828025781 >> v2:172.21.15.70:3300/0 conn(0x7fc5fc06d410 0x7fc5fc0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc6018ab640 1 --2- 172.21.15.70:0/1828025781 >> v2:172.21.15.181:3300/0 conn(0x7fc5fc15aad0 0x7fc5fc0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc6008a9640 1 -- 172.21.15.70:0/1828025781 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218640200 0 0) 0x7fc5fc15bf30 con 0x7fc5fc0c82b0 2023-10-08T05:56:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc6008a9640 1 -- 172.21.15.70:0/1828025781 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5e8003520 con 0x7fc5fc0c82b0 2023-10-08T05:56:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc6020ac640 1 -- 172.21.15.70:0/1828025781 >> v2:172.21.15.181:3300/0 conn(0x7fc5fc15aad0 msgr2=0x7fc5fc0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:49.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.430+0000 7fc6020ac640 1 --2- 172.21.15.70:0/1828025781 >> v2:172.21.15.181:3300/0 conn(0x7fc5fc15aad0 0x7fc5fc0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.431+0000 7fc6020ac640 1 -- 172.21.15.70:0/1828025781 >> v1:172.21.15.70:6789/0 conn(0x7fc5fc0c82b0 legacy=0x7fc5fc06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:49.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.431+0000 7fc6020ac640 1 -- 172.21.15.70:0/1828025781 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5fc15b8e0 con 0x7fc5fc06d410 2023-10-08T05:56:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.432+0000 7fc6020ac640 1 --2- 172.21.15.70:0/1828025781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 0x7fc5fc0c8710 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc06e6a0 tx=0x7fc5f402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e84544f897033b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.432+0000 7fc6008a9640 1 -- 172.21.15.70:0/1828025781 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f402cc70 con 0x7fc5fc06d410 2023-10-08T05:56:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.432+0000 7fc6008a9640 1 -- 172.21.15.70:0/1828025781 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f402cdd0 con 0x7fc5fc06d410 2023-10-08T05:56:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.432+0000 7fc6008a9640 1 -- 172.21.15.70:0/1828025781 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4034660 con 0x7fc5fc06d410 2023-10-08T05:56:49.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.433+0000 7fc603b0f640 1 -- 172.21.15.70:0/1828025781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 msgr2=0x7fc5fc0c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:49.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.433+0000 7fc603b0f640 1 --2- 172.21.15.70:0/1828025781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 0x7fc5fc0c8710 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc06e6a0 tx=0x7fc5f402bec0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.433+0000 7fc603b0f640 1 -- 172.21.15.70:0/1828025781 shutdown_connections 2023-10-08T05:56:49.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.433+0000 7fc603b0f640 1 --2- 172.21.15.70:0/1828025781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 0x7fc5fc0c8710 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.433+0000 7fc603b0f640 1 --2- 172.21.15.70:0/1828025781 >> v2:172.21.15.181:3300/0 conn(0x7fc5fc15aad0 0x7fc5fc0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.433+0000 7fc603b0f640 1 -- 172.21.15.70:0/1828025781 >> 172.21.15.70:0/1828025781 conn(0x7fc5fc069d50 msgr2=0x7fc5fc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.433+0000 7fc603b0f640 1 -- 172.21.15.70:0/1828025781 shutdown_connections 2023-10-08T05:56:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.433+0000 7fc603b0f640 1 -- 172.21.15.70:0/1828025781 wait complete. 2023-10-08T05:56:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.434+0000 7fc603b0f640 1 Processor -- start 2023-10-08T05:56:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.435+0000 7fc603b0f640 1 -- start start 2023-10-08T05:56:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc603b0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 0x7fc5fc0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:49.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc603b0f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5fc0c82b0 0x7fc5fc151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc603b0f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5fc15aad0 0x7fc5fc0c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc603b0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc14ae90 con 0x7fc5fc06d410 2023-10-08T05:56:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc603b0f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc14a280 con 0x7fc5fc0c82b0 2023-10-08T05:56:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc603b0f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5fc14a560 con 0x7fc5fc15aad0 2023-10-08T05:56:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc6018ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 0x7fc5fc0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:49.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc6018ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 0x7fc5fc0cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49404/0 (socket says 172.21.15.70:49404) 2023-10-08T05:56:49.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc6018ab640 1 -- 172.21.15.70:0/3008954128 learned_addr learned my addr 172.21.15.70:0/3008954128 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:49.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc6018ab640 1 -- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5fc15aad0 msgr2=0x7fc5fc0c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:49.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc6018ab640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5fc15aad0 0x7fc5fc0c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc6018ab640 1 -- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5fc0c82b0 msgr2=0x7fc5fc151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:49.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc6018ab640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5fc0c82b0 0x7fc5fc151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.436+0000 7fc6018ab640 1 -- 172.21.15.70:0/3008954128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f4009a70 con 0x7fc5fc06d410 2023-10-08T05:56:49.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.437+0000 7fc6018ab640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 0x7fc5fc0cbb30 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800e870 tx=0x7fc5f800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:49.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.437+0000 7fc5f27fc640 1 -- 172.21.15.70:0/3008954128 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80094f0 con 0x7fc5fc06d410 2023-10-08T05:56:49.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.437+0000 7fc5f27fc640 1 -- 172.21.15.70:0/3008954128 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f800c940 con 0x7fc5fc06d410 2023-10-08T05:56:49.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.437+0000 7fc603b0f640 1 -- 172.21.15.70:0/3008954128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5fc14a840 con 0x7fc5fc06d410 2023-10-08T05:56:49.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.437+0000 7fc5f27fc640 1 -- 172.21.15.70:0/3008954128 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800ccf0 con 0x7fc5fc06d410 2023-10-08T05:56:49.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.437+0000 7fc603b0f640 1 -- 172.21.15.70:0/3008954128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5fc0cc070 con 0x7fc5fc06d410 2023-10-08T05:56:49.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.438+0000 7fc603b0f640 1 -- 172.21.15.70:0/3008954128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5fc04e160 con 0x7fc5fc06d410 2023-10-08T05:56:49.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.439+0000 7fc5f27fc640 1 -- 172.21.15.70:0/3008954128 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 71) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fc5f8013470 con 0x7fc5fc06d410 2023-10-08T05:56:49.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.440+0000 7fc5f27fc640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc5e006b010 0x7fc5e006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:49.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.440+0000 7fc5f27fc640 1 -- 172.21.15.70:0/3008954128 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fc5f8089b80 con 0x7fc5fc06d410 2023-10-08T05:56:49.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.441+0000 7fc6010aa640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc5e006b010 0x7fc5e006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:49.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.442+0000 7fc6010aa640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc5e006b010 0x7fc5e006d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0ce0c0 tx=0x7fc5ec007930 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:49.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.444+0000 7fc5f27fc640 1 -- 172.21.15.70:0/3008954128 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5f8054840 con 0x7fc5fc06d410 2023-10-08T05:56:49.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.655+0000 7fc603b0f640 1 -- 172.21.15.70:0/3008954128 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/weekly", "snap_schedule": "1w", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5fc15bbc0 con 0x7fc5e006b010 2023-10-08T05:56:49.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.808+0000 7fc5f27fc640 1 -- 172.21.15.70:0/3008954128 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7fc5ec0058b0 con 0x7fc5e006b010 2023-10-08T05:56:49.810 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1/weekly 2023-10-08T05:56:49.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 -- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc5e006b010 msgr2=0x7fc5e006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:49.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc5e006b010 0x7fc5e006d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0ce0c0 tx=0x7fc5ec007930 comp rx=0 tx=0).stop 2023-10-08T05:56:49.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 -- 172.21.15.70:0/3008954128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 msgr2=0x7fc5fc0cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:49.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 0x7fc5fc0cbb30 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800e870 tx=0x7fc5f800ed30 comp rx=0 tx=0).stop 2023-10-08T05:56:49.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 -- 172.21.15.70:0/3008954128 shutdown_connections 2023-10-08T05:56:49.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc5e006b010 0x7fc5e006d4d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc5fc15aad0 0x7fc5fc0c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc5fc0c82b0 0x7fc5fc151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 --2- 172.21.15.70:0/3008954128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc06d410 0x7fc5fc0cbb30 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:49.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 -- 172.21.15.70:0/3008954128 >> 172.21.15.70:0/3008954128 conn(0x7fc5fc069d50 msgr2=0x7fc5fc0b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:49.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 -- 172.21.15.70:0/3008954128 shutdown_connections 2023-10-08T05:56:49.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:49.812+0000 7fc603b0f640 1 -- 172.21.15.70:0/3008954128 wait complete. 2023-10-08T05:56:49.836 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/weekly 2023-10-08T05:56:49.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:49.836 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/monthly) 2023-10-08T05:56:49.896 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/monthly 1M 2023-10-08T05:56:50.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.114+0000 7fe8e78c2640 1 Processor -- start 2023-10-08T05:56:50.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.114+0000 7fe8e78c2640 1 -- start start 2023-10-08T05:56:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.116+0000 7fe8e78c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01432f0 0x7fe8e00cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.116+0000 7fe8e78c2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe8e00b4240 0x7fe8e0159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.116+0000 7fe8e78c2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e00cbb30 con 0x7fe8e00cb6b0 2023-10-08T05:56:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.116+0000 7fe8e78c2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e01436d0 con 0x7fe8e00b4240 2023-10-08T05:56:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.116+0000 7fe8e78c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e015d440 con 0x7fe8e01432f0 2023-10-08T05:56:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.116+0000 7fe8e4e5d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe8e00cb6b0 0x7fe8e00b3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43598/0 (socket says 172.21.15.70:43598) 2023-10-08T05:56:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8e4e5d640 1 -- 172.21.15.70:0/1230800713 learned_addr learned my addr 172.21.15.70:0/1230800713 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:50.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8e5e5f640 1 --2- 172.21.15.70:0/1230800713 >> v2:172.21.15.181:3300/0 conn(0x7fe8e00b4240 0x7fe8e0159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8e565e640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01432f0 0x7fe8e00cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001751413 0 0) 0x7fe8e00cbb30 con 0x7fe8e00cb6b0 2023-10-08T05:56:50.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8cc003520 con 0x7fe8e00cb6b0 2023-10-08T05:56:50.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2421852930 0 0) 0x7fe8cc003520 con 0x7fe8e00cb6b0 2023-10-08T05:56:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01432f0 msgr2=0x7fe8e00cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01432f0 0x7fe8e00cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 >> v2:172.21.15.181:3300/0 conn(0x7fe8e00b4240 msgr2=0x7fe8e0159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 --2- 172.21.15.70:0/1230800713 >> v2:172.21.15.181:3300/0 conn(0x7fe8e00b4240 0x7fe8e0159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e015d690 con 0x7fe8e00cb6b0 2023-10-08T05:56:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe8d0001670 con 0x7fe8e00cb6b0 2023-10-08T05:56:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 >> v1:172.21.15.70:6789/0 conn(0x7fe8e00cb6b0 legacy=0x7fe8e00b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.117+0000 7fe8d7fff640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8cc003970 0x7fe8cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8d7fff640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 0x7fe8cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8d7fff640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01432f0 0x7fe8cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e01436d0 con 0x7fe8cc0047c0 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e015d440 con 0x7fe8cc003970 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc00dcf0 con 0x7fe8e01432f0 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8e4e5d640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 0x7fe8cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8e565e640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8cc003970 0x7fe8cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8e5e5f640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01432f0 0x7fe8cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8e4e5d640 1 -- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01432f0 msgr2=0x7fe8cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8e4e5d640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01432f0 0x7fe8cc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8e4e5d640 1 -- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8cc003970 msgr2=0x7fe8cc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8e4e5d640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8cc003970 0x7fe8cc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.118+0000 7fe8e4e5d640 1 -- 172.21.15.70:0/1230800713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc003520 con 0x7fe8cc0047c0 2023-10-08T05:56:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.119+0000 7fe8e4e5d640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 0x7fe8cc004c40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c210 tx=0x7fe8cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302958f7147ad119 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:50.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.119+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0005ad0 con 0x7fe8cc0047c0 2023-10-08T05:56:50.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.119+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d0005c30 con 0x7fe8cc0047c0 2023-10-08T05:56:50.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.119+0000 7fe8d7fff640 1 -- 172.21.15.70:0/1230800713 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d002c8a0 con 0x7fe8cc0047c0 2023-10-08T05:56:50.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.120+0000 7fe8e78c2640 1 -- 172.21.15.70:0/1230800713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 msgr2=0x7fe8cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.120+0000 7fe8e78c2640 1 --2- 172.21.15.70:0/1230800713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 0x7fe8cc004c40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c210 tx=0x7fe8cc0057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.120+0000 7fe8e5e5f640 1 -- 172.21.15.70:0/1230800713 reap_dead start 2023-10-08T05:56:50.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.120+0000 7fe8e78c2640 1 -- 172.21.15.70:0/1230800713 shutdown_connections 2023-10-08T05:56:50.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.120+0000 7fe8e78c2640 1 -- 172.21.15.70:0/1230800713 >> 172.21.15.70:0/1230800713 conn(0x7fe8e0069d50 msgr2=0x7fe8e006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:50.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.120+0000 7fe8e78c2640 1 -- 172.21.15.70:0/1230800713 shutdown_connections 2023-10-08T05:56:50.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.120+0000 7fe8e78c2640 1 -- 172.21.15.70:0/1230800713 wait complete. 2023-10-08T05:56:50.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.122+0000 7fe8e78c2640 1 Processor -- start 2023-10-08T05:56:50.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.122+0000 7fe8e78c2640 1 -- start start 2023-10-08T05:56:50.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.123+0000 7fe8e78c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 0x7fe8e0157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.123+0000 7fe8e78c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01f31f0 0x7fe8e01f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.123+0000 7fe8e78c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e00c7ad0 0x7fe8e00c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.123+0000 7fe8e78c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e0157870 con 0x7fe8cc0047c0 2023-10-08T05:56:50.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.123+0000 7fe8e78c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e00c84a0 con 0x7fe8e00c7ad0 2023-10-08T05:56:50.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.123+0000 7fe8e78c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e013baf0 con 0x7fe8e01f31f0 2023-10-08T05:56:50.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.123+0000 7fe8e565e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 0x7fe8e0157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.123+0000 7fe8e565e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 0x7fe8e0157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49420/0 (socket says 172.21.15.70:49420) 2023-10-08T05:56:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.123+0000 7fe8e565e640 1 -- 172.21.15.70:0/3802355352 learned_addr learned my addr 172.21.15.70:0/3802355352 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.124+0000 7fe8e4e5d640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01f31f0 0x7fe8e01f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.124+0000 7fe8e5e5f640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e00c7ad0 0x7fe8e00c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.124+0000 7fe8e565e640 1 -- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01f31f0 msgr2=0x7fe8e01f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.124+0000 7fe8e565e640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01f31f0 0x7fe8e01f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.124+0000 7fe8e565e640 1 -- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e00c7ad0 msgr2=0x7fe8e00c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.124+0000 7fe8e565e640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e00c7ad0 0x7fe8e00c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.124+0000 7fe8e565e640 1 -- 172.21.15.70:0/3802355352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e01f6a90 con 0x7fe8cc0047c0 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.124+0000 7fe8e5e5f640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e00c7ad0 0x7fe8e00c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.124+0000 7fe8e565e640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 0x7fe8e0157330 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007e60 tx=0x7fe8dc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.124+0000 7fe8d5ffb640 1 -- 172.21.15.70:0/3802355352 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0145f0 con 0x7fe8cc0047c0 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.125+0000 7fe8d5ffb640 1 -- 172.21.15.70:0/3802355352 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc014750 con 0x7fe8cc0047c0 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.125+0000 7fe8d5ffb640 1 -- 172.21.15.70:0/3802355352 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc014ae0 con 0x7fe8cc0047c0 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.125+0000 7fe8e78c2640 1 -- 172.21.15.70:0/3802355352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e013c070 con 0x7fe8cc0047c0 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.125+0000 7fe8e78c2640 1 -- 172.21.15.70:0/3802355352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e01f67b0 con 0x7fe8cc0047c0 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.126+0000 7fe8e78c2640 1 -- 172.21.15.70:0/3802355352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e004e160 con 0x7fe8cc0047c0 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.127+0000 7fe8d5ffb640 1 -- 172.21.15.70:0/3802355352 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 71) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fe8dc014c40 con 0x7fe8cc0047c0 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.128+0000 7fe8d5ffb640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe8c406afe0 0x7fe8c406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.128+0000 7fe8d5ffb640 1 -- 172.21.15.70:0/3802355352 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fe8dc08a8a0 con 0x7fe8cc0047c0 2023-10-08T05:56:50.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.129+0000 7fe8e4e5d640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe8c406afe0 0x7fe8c406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.130+0000 7fe8e4e5d640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe8c406afe0 0x7fe8c406d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0034000 tx=0x7fe8d0007000 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.132+0000 7fe8d5ffb640 1 -- 172.21.15.70:0/3802355352 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8dc055450 con 0x7fe8cc0047c0 2023-10-08T05:56:50.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.347+0000 7fe8e78c2640 1 -- 172.21.15.70:0/3802355352 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/monthly", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8e015dc50 con 0x7fe8c406afe0 2023-10-08T05:56:50.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.500+0000 7fe8d5ffb640 1 -- 172.21.15.70:0/3802355352 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+45 (secure 0 0 0) 0x7fe8d00014b0 con 0x7fe8c406afe0 2023-10-08T05:56:50.500 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1/monthly 2023-10-08T05:56:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.503+0000 7fe8e78c2640 1 -- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe8c406afe0 msgr2=0x7fe8c406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.503+0000 7fe8e78c2640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe8c406afe0 0x7fe8c406d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0034000 tx=0x7fe8d0007000 comp rx=0 tx=0).stop 2023-10-08T05:56:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.503+0000 7fe8e78c2640 1 -- 172.21.15.70:0/3802355352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 msgr2=0x7fe8e0157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.503+0000 7fe8e78c2640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 0x7fe8e0157330 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc007e60 tx=0x7fe8dc00f520 comp rx=0 tx=0).stop 2023-10-08T05:56:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.503+0000 7fe8e78c2640 1 -- 172.21.15.70:0/3802355352 shutdown_connections 2023-10-08T05:56:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.503+0000 7fe8e78c2640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe8c406afe0 0x7fe8c406d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.503+0000 7fe8e78c2640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8e00c7ad0 0x7fe8e00c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.503+0000 7fe8e78c2640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8e01f31f0 0x7fe8e01f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.503+0000 7fe8e78c2640 1 --2- 172.21.15.70:0/3802355352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8cc0047c0 0x7fe8e0157330 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.503+0000 7fe8e78c2640 1 -- 172.21.15.70:0/3802355352 >> 172.21.15.70:0/3802355352 conn(0x7fe8e0069d50 msgr2=0x7fe8e0141880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:50.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.504+0000 7fe8e78c2640 1 -- 172.21.15.70:0/3802355352 shutdown_connections 2023-10-08T05:56:50.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.504+0000 7fe8e78c2640 1 -- 172.21.15.70:0/3802355352 wait complete. 2023-10-08T05:56:50.523 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/monthly 2023-10-08T05:56:50.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:50.524 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/yearly) 2023-10-08T05:56:50.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/yearly 1Y 2023-10-08T05:56:50.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.795+0000 7f1542864640 1 Processor -- start 2023-10-08T05:56:50.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.795+0000 7f1542864640 1 -- start start 2023-10-08T05:56:50.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.797+0000 7f1542864640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f153c15aad0 0x7f153c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.797+0000 7f1542864640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c15bf30 con 0x7f153c0c82b0 2023-10-08T05:56:50.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.797+0000 7f1542864640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c15b320 con 0x7f153c06d410 2023-10-08T05:56:50.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.797+0000 7f1542864640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c15b600 con 0x7f153c15aad0 2023-10-08T05:56:50.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.797+0000 7f153b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f153c0c82b0 0x7f153c06cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43608/0 (socket says 172.21.15.70:43608) 2023-10-08T05:56:50.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.797+0000 7f153b7fe640 1 -- 172.21.15.70:0/349857982 learned_addr learned my addr 172.21.15.70:0/349857982 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:50.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153bfff640 1 --2- 172.21.15.70:0/349857982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f153c15aad0 0x7f153c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810213749 0 0) 0x7f153c15bf30 con 0x7f153c0c82b0 2023-10-08T05:56:50.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1524003520 con 0x7f153c0c82b0 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 420308855 0 0) 0x7f1524003520 con 0x7f153c0c82b0 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f153c15aad0 msgr2=0x7f153c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153affd640 1 --2- 172.21.15.70:0/349857982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f153c15aad0 0x7f153c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 >> v1:172.21.15.181:6789/0 conn(0x7f153c06d410 legacy=0x7f153c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c15b8e0 con 0x7f153c0c82b0 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1530001670 con 0x7f153c0c82b0 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 >> v1:172.21.15.70:6789/0 conn(0x7f153c0c82b0 legacy=0x7f153c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.798+0000 7f153affd640 1 --2- 172.21.15.70:0/349857982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15240039a0 0x7f1524003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153affd640 1 --2- 172.21.15.70:0/349857982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15240047c0 0x7f1524004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153affd640 1 --2- 172.21.15.70:0/349857982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f153c15aad0 0x7f1524005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c15b600 con 0x7f15240039a0 2023-10-08T05:56:50.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152400ba60 con 0x7f15240047c0 2023-10-08T05:56:50.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152400bc20 con 0x7f153c15aad0 2023-10-08T05:56:50.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153bfff640 1 --2- 172.21.15.70:0/349857982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15240039a0 0x7f1524003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153bfff640 1 -- 172.21.15.70:0/349857982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f153c15aad0 msgr2=0x7f1524005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:50.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153bfff640 1 --2- 172.21.15.70:0/349857982 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f153c15aad0 0x7f1524005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153bfff640 1 -- 172.21.15.70:0/349857982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15240047c0 msgr2=0x7f1524004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:50.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153bfff640 1 --2- 172.21.15.70:0/349857982 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15240047c0 0x7f1524004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.799+0000 7f153bfff640 1 -- 172.21.15.70:0/349857982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1524003520 con 0x7f15240039a0 2023-10-08T05:56:50.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.800+0000 7f153bfff640 1 --2- 172.21.15.70:0/349857982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15240039a0 0x7f1524003dc0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f152c009a90 tx=0x7f15240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabf5d24c7fd5662 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:50.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.800+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c02c6b0 con 0x7f15240039a0 2023-10-08T05:56:50.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.800+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c02c810 con 0x7f15240039a0 2023-10-08T05:56:50.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.800+0000 7f153affd640 1 -- 172.21.15.70:0/349857982 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c02cba0 con 0x7f15240039a0 2023-10-08T05:56:50.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.801+0000 7f1542864640 1 -- 172.21.15.70:0/349857982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15240039a0 msgr2=0x7f1524003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.801+0000 7f1542864640 1 --2- 172.21.15.70:0/349857982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15240039a0 0x7f1524003dc0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f152c009a90 tx=0x7f15240057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.801+0000 7f1540e01640 1 -- 172.21.15.70:0/349857982 reap_dead start 2023-10-08T05:56:50.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.801+0000 7f1542864640 1 -- 172.21.15.70:0/349857982 shutdown_connections 2023-10-08T05:56:50.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.801+0000 7f1542864640 1 -- 172.21.15.70:0/349857982 >> 172.21.15.70:0/349857982 conn(0x7f153c069d50 msgr2=0x7f153c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:50.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.801+0000 7f1542864640 1 -- 172.21.15.70:0/349857982 shutdown_connections 2023-10-08T05:56:50.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.801+0000 7f1542864640 1 -- 172.21.15.70:0/349857982 wait complete. 2023-10-08T05:56:50.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.803+0000 7f1542864640 1 Processor -- start 2023-10-08T05:56:50.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.803+0000 7f1542864640 1 -- start start 2023-10-08T05:56:50.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.804+0000 7f1542864640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15240039a0 0x7f153c0cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.804+0000 7f1542864640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f153c0cded0 0x7f153c150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.804+0000 7f1542864640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f153c149a30 0x7f153c151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.804+0000 7f1542864640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c1edd10 con 0x7f153c149a30 2023-10-08T05:56:50.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.804+0000 7f1542864640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c1eeb00 con 0x7f153c0cded0 2023-10-08T05:56:50.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.804+0000 7f1542864640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c1edef0 con 0x7f15240039a0 2023-10-08T05:56:50.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f1540e01640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f153c149a30 0x7f153c151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f1540e01640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f153c149a30 0x7f153c151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49440/0 (socket says 172.21.15.70:49440) 2023-10-08T05:56:50.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f1540e01640 1 -- 172.21.15.70:0/2982307078 learned_addr learned my addr 172.21.15.70:0/2982307078 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:50.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f153bfff640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15240039a0 0x7f153c0cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f153b7fe640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f153c0cded0 0x7f153c150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f1540e01640 1 -- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15240039a0 msgr2=0x7f153c0cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f1540e01640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15240039a0 0x7f153c0cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f1540e01640 1 -- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f153c0cded0 msgr2=0x7f153c150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:50.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f1540e01640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f153c0cded0 0x7f153c150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:50.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f1540e01640 1 -- 172.21.15.70:0/2982307078 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c1ee1d0 con 0x7f153c149a30 2023-10-08T05:56:50.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.805+0000 7f1540e01640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f153c149a30 0x7f153c151350 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f153c071680 tx=0x7f153400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:50.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.806+0000 7f1538ff9640 1 -- 172.21.15.70:0/2982307078 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534005d40 con 0x7f153c149a30 2023-10-08T05:56:50.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.806+0000 7f1538ff9640 1 -- 172.21.15.70:0/2982307078 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153400ced0 con 0x7f153c149a30 2023-10-08T05:56:50.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.806+0000 7f1542864640 1 -- 172.21.15.70:0/2982307078 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c1ee4b0 con 0x7f153c149a30 2023-10-08T05:56:50.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.806+0000 7f1538ff9640 1 -- 172.21.15.70:0/2982307078 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400a850 con 0x7f153c149a30 2023-10-08T05:56:50.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.806+0000 7f1542864640 1 -- 172.21.15.70:0/2982307078 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c14cec0 con 0x7f153c149a30 2023-10-08T05:56:50.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.807+0000 7f1542864640 1 -- 172.21.15.70:0/2982307078 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153c04e160 con 0x7f153c149a30 2023-10-08T05:56:50.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.808+0000 7f1538ff9640 1 -- 172.21.15.70:0/2982307078 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 72) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f153400a9b0 con 0x7f153c149a30 2023-10-08T05:56:50.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.809+0000 7f1538ff9640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f152806b070 0x7f152806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:50.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.809+0000 7f1538ff9640 1 -- 172.21.15.70:0/2982307078 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f1534089060 con 0x7f153c149a30 2023-10-08T05:56:50.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.810+0000 7f153bfff640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f152806b070 0x7f152806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:50.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.811+0000 7f153bfff640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f152806b070 0x7f152806d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f152c036000 tx=0x7f152c030000 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:50.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:50.813+0000 7f1538ff9640 1 -- 172.21.15.70:0/2982307078 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15340bbca0 con 0x7f153c149a30 2023-10-08T05:56:51.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.024+0000 7f1542864640 1 -- 172.21.15.70:0/2982307078 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/yearly", "snap_schedule": "1Y", "target": ["mon-mgr", ""]}) v1 -- 0x7f153c15bbc0 con 0x7f152806b070 2023-10-08T05:56:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.172+0000 7f1538ff9640 1 -- 172.21.15.70:0/2982307078 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+44 (secure 0 0 0) 0x7f152c004040 con 0x7f152806b070 2023-10-08T05:56:51.173 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1/yearly 2023-10-08T05:56:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.175+0000 7f1542864640 1 -- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f152806b070 msgr2=0x7f152806d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.175+0000 7f1542864640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f152806b070 0x7f152806d530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f152c036000 tx=0x7f152c030000 comp rx=0 tx=0).stop 2023-10-08T05:56:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.175+0000 7f1542864640 1 -- 172.21.15.70:0/2982307078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f153c149a30 msgr2=0x7f153c151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.175+0000 7f1542864640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f153c149a30 0x7f153c151350 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f153c071680 tx=0x7f153400cb90 comp rx=0 tx=0).stop 2023-10-08T05:56:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.176+0000 7f1542864640 1 -- 172.21.15.70:0/2982307078 shutdown_connections 2023-10-08T05:56:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.176+0000 7f1542864640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f152806b070 0x7f152806d530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.176+0000 7f1542864640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f153c149a30 0x7f153c151350 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.176+0000 7f1542864640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f153c0cded0 0x7f153c150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.176+0000 7f1542864640 1 --2- 172.21.15.70:0/2982307078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15240039a0 0x7f153c0cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.176+0000 7f1542864640 1 -- 172.21.15.70:0/2982307078 >> 172.21.15.70:0/2982307078 conn(0x7f153c069d50 msgr2=0x7f153c0c8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.176+0000 7f1542864640 1 -- 172.21.15.70:0/2982307078 shutdown_connections 2023-10-08T05:56:51.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.176+0000 7f1542864640 1 -- 172.21.15.70:0/2982307078 wait complete. 2023-10-08T05:56:51.196 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1/yearly 2023-10-08T05:56:51.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:51.197 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/bad_period_spec) 2023-10-08T05:56:51.251 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1X 2023-10-08T05:56:51.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.469+0000 7f593ec9b640 1 Processor -- start 2023-10-08T05:56:51.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.469+0000 7f593ec9b640 1 -- start start 2023-10-08T05:56:51.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.471+0000 7f593ec9b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f593815aad0 0x7f59380b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.471+0000 7f593ec9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f593806d410 0x7f59380c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.471+0000 7f593ec9b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593815bf30 con 0x7f59380c82b0 2023-10-08T05:56:51.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.471+0000 7f593ec9b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f593815b320 con 0x7f593806d410 2023-10-08T05:56:51.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.471+0000 7f593ec9b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593815b600 con 0x7f593815aad0 2023-10-08T05:56:51.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.472+0000 7f593d238640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f593806d410 0x7f59380c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.472+0000 7f593ca37640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f593815aad0 0x7f59380b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.472+0000 7f5937fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f59380c82b0 0x7f593806cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:46886/0 (socket says 172.21.15.70:46886) 2023-10-08T05:56:51.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.472+0000 7f593d238640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f593806d410 0x7f59380c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:55436/0 (socket says 172.21.15.70:55436) 2023-10-08T05:56:51.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.472+0000 7f5937fff640 1 -- 172.21.15.70:0/1047310528 learned_addr learned my addr 172.21.15.70:0/1047310528 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:51.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.472+0000 7f593d238640 1 -- 172.21.15.70:0/1047310528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f593815aad0 msgr2=0x7f59380b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:51.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.472+0000 7f593d238640 1 --2- 172.21.15.70:0/1047310528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f593815aad0 0x7f59380b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.472+0000 7f593d238640 1 -- 172.21.15.70:0/1047310528 >> v1:172.21.15.181:6789/0 conn(0x7f59380c82b0 legacy=0x7f593806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:51.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.472+0000 7f593d238640 1 -- 172.21.15.70:0/1047310528 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593815b8e0 con 0x7f593806d410 2023-10-08T05:56:51.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.472+0000 7f593ca37640 1 --2- 172.21.15.70:0/1047310528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f593815aad0 0x7f59380b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:56:51.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.474+0000 7f593d238640 1 --2- 172.21.15.70:0/1047310528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f593806d410 0x7f59380c8710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f59300097a0 tx=0x7f593002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8f117de34ea3aa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:51.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.474+0000 7f59377fe640 1 -- 172.21.15.70:0/1047310528 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593002d040 con 0x7f593806d410 2023-10-08T05:56:51.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.474+0000 7f59377fe640 1 -- 172.21.15.70:0/1047310528 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5930031810 con 0x7f593806d410 2023-10-08T05:56:51.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.474+0000 7f59377fe640 1 -- 172.21.15.70:0/1047310528 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930031ba0 con 0x7f593806d410 2023-10-08T05:56:51.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.475+0000 7f593ec9b640 1 -- 172.21.15.70:0/1047310528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f593806d410 msgr2=0x7f59380c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:51.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.475+0000 7f593ec9b640 1 --2- 172.21.15.70:0/1047310528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f593806d410 0x7f59380c8710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f59300097a0 tx=0x7f593002b8b0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.475+0000 7f593ec9b640 1 -- 172.21.15.70:0/1047310528 shutdown_connections 2023-10-08T05:56:51.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.475+0000 7f593ec9b640 1 --2- 172.21.15.70:0/1047310528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f593806d410 0x7f59380c8710 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.475+0000 7f593ec9b640 1 --2- 172.21.15.70:0/1047310528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f593815aad0 0x7f59380b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.475+0000 7f593ec9b640 1 -- 172.21.15.70:0/1047310528 >> 172.21.15.70:0/1047310528 conn(0x7f5938069d50 msgr2=0x7f593806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:51.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.475+0000 7f593ec9b640 1 -- 172.21.15.70:0/1047310528 shutdown_connections 2023-10-08T05:56:51.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.475+0000 7f593ec9b640 1 -- 172.21.15.70:0/1047310528 wait complete. 2023-10-08T05:56:51.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.477+0000 7f593ec9b640 1 Processor -- start 2023-10-08T05:56:51.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.477+0000 7f593ec9b640 1 -- start start 2023-10-08T05:56:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.478+0000 7f593ec9b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f593806d410 0x7f59381445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.478+0000 7f593ec9b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59380c82b0 0x7f59380cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.478+0000 7f593ec9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593815aad0 0x7f59380cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.478+0000 7f593ec9b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5938144af0 con 0x7f593815aad0 2023-10-08T05:56:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.478+0000 7f593ec9b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59381468b0 con 0x7f59380c82b0 2023-10-08T05:56:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.478+0000 7f593ec9b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5938145ca0 con 0x7f593806d410 2023-10-08T05:56:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.478+0000 7f593d238640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593815aad0 0x7f59380cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.478+0000 7f593d238640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593815aad0 0x7f59380cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49454/0 (socket says 172.21.15.70:49454) 2023-10-08T05:56:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.478+0000 7f593d238640 1 -- 172.21.15.70:0/2559983152 learned_addr learned my addr 172.21.15.70:0/2559983152 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.479+0000 7f593ca37640 1 --2- 172.21.15.70:0/2559983152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f593806d410 0x7f59381445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.479+0000 7f5937fff640 1 --2- 172.21.15.70:0/2559983152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59380c82b0 0x7f59380cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.479+0000 7f593d238640 1 -- 172.21.15.70:0/2559983152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f593806d410 msgr2=0x7f59381445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:51.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.479+0000 7f593d238640 1 --2- 172.21.15.70:0/2559983152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f593806d410 0x7f59381445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.479+0000 7f593d238640 1 -- 172.21.15.70:0/2559983152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59380c82b0 msgr2=0x7f59380cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:51.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.479+0000 7f593d238640 1 --2- 172.21.15.70:0/2559983152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f59380c82b0 0x7f59380cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.479+0000 7f593d238640 1 -- 172.21.15.70:0/2559983152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5928009a70 con 0x7f593815aad0 2023-10-08T05:56:51.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.479+0000 7f593d238640 1 --2- 172.21.15.70:0/2559983152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593815aad0 0x7f59380cb690 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f593002b8f0 tx=0x7f59300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:51.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.480+0000 7f59357fa640 1 -- 172.21.15.70:0/2559983152 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930006e10 con 0x7f593815aad0 2023-10-08T05:56:51.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.480+0000 7f59357fa640 1 -- 172.21.15.70:0/2559983152 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593003ace0 con 0x7f593815aad0 2023-10-08T05:56:51.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.480+0000 7f593ec9b640 1 -- 172.21.15.70:0/2559983152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5930009400 con 0x7f593815aad0 2023-10-08T05:56:51.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.480+0000 7f59357fa640 1 -- 172.21.15.70:0/2559983152 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5930044610 con 0x7f593815aad0 2023-10-08T05:56:51.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.480+0000 7f593ec9b640 1 -- 172.21.15.70:0/2559983152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5938146260 con 0x7f593815aad0 2023-10-08T05:56:51.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.481+0000 7f593ec9b640 1 -- 172.21.15.70:0/2559983152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5938146a60 con 0x7f593815aad0 2023-10-08T05:56:51.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.482+0000 7f59357fa640 1 -- 172.21.15.70:0/2559983152 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 72) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f5930044770 con 0x7f593815aad0 2023-10-08T05:56:51.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.483+0000 7f59357fa640 1 --2- 172.21.15.70:0/2559983152 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f592406b070 0x7f592406d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.484+0000 7f59357fa640 1 -- 172.21.15.70:0/2559983152 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f59300abff0 con 0x7f593815aad0 2023-10-08T05:56:51.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.484+0000 7f593ca37640 1 --2- 172.21.15.70:0/2559983152 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f592406b070 0x7f592406d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.485+0000 7f593ca37640 1 --2- 172.21.15.70:0/2559983152 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f592406b070 0x7f592406d530 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5928007560 tx=0x7f592800b040 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:51.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.488+0000 7f59357fa640 1 -- 172.21.15.70:0/2559983152 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5938146a60 con 0x7f593815aad0 2023-10-08T05:56:51.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.730+0000 7f593ec9b640 1 -- 172.21.15.70:0/2559983152 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1X", "target": ["mon-mgr", ""]}) v1 -- 0x7f593815bbc0 con 0x7f592406b070 2023-10-08T05:56:51.733 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:51.733+0000 7f52f85fa640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - multiplier should be one of h,d,w,M,Y 2023-10-08T05:56:51.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.733+0000 7f59357fa640 1 -- 172.21.15.70:0/2559983152 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - multiplier should be one of h,d,w,M,Y) v1 ==== 74+0+0 (secure 0 0 0) 0x7f592800f050 con 0x7f592406b070 2023-10-08T05:56:51.734 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENOENT: invalid schedule specified - multiplier should be one of h,d,w,M,Y 2023-10-08T05:56:51.747 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T05:56:51.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1MM 2023-10-08T05:56:51.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.958+0000 7f9c73b18640 1 Processor -- start 2023-10-08T05:56:51.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.959+0000 7f9c73b18640 1 -- start start 2023-10-08T05:56:51.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.960+0000 7f9c73b18640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c6c0b1750 0x7f9c6c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.960+0000 7f9c73b18640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c6c06d240 0x7f9c6c070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.960+0000 7f9c73b18640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6c06d670 con 0x7f9c6c15aad0 2023-10-08T05:56:51.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.960+0000 7f9c73b18640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c6c15bf30 con 0x7f9c6c0b1750 2023-10-08T05:56:51.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.960+0000 7f9c73b18640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c6c0b1bb0 con 0x7f9c6c06d240 2023-10-08T05:56:51.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.961+0000 7f9c710b3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c6c0b1750 0x7f9c6c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.961+0000 7f9c710b3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c6c0b1750 0x7f9c6c06cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49458/0 (socket says 172.21.15.70:49458) 2023-10-08T05:56:51.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.961+0000 7f9c710b3640 1 -- 172.21.15.70:0/3583391682 learned_addr learned my addr 172.21.15.70:0/3583391682 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:51.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.961+0000 7f9c720b5640 1 --2- 172.21.15.70:0/3583391682 >> v2:172.21.15.181:3300/0 conn(0x7f9c6c06d240 0x7f9c6c070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.961+0000 7f9c710b3640 1 -- 172.21.15.70:0/3583391682 >> v2:172.21.15.181:3300/0 conn(0x7f9c6c06d240 msgr2=0x7f9c6c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:51.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.961+0000 7f9c710b3640 1 --2- 172.21.15.70:0/3583391682 >> v2:172.21.15.181:3300/0 conn(0x7f9c6c06d240 0x7f9c6c070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.961+0000 7f9c710b3640 1 -- 172.21.15.70:0/3583391682 >> v1:172.21.15.181:6789/0 conn(0x7f9c6c15aad0 legacy=0x7f9c6c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:51.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.961+0000 7f9c710b3640 1 -- 172.21.15.70:0/3583391682 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c15b320 con 0x7f9c6c0b1750 2023-10-08T05:56:51.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.961+0000 7f9c720b5640 1 --2- 172.21.15.70:0/3583391682 >> v2:172.21.15.181:3300/0 conn(0x7f9c6c06d240 0x7f9c6c070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:51.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.963+0000 7f9c710b3640 1 --2- 172.21.15.70:0/3583391682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6c0b1750 0x7f9c6c06cd00 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c06e330 tx=0x7f9c6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73679dce61b607d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.963+0000 7f9c708b2640 1 -- 172.21.15.70:0/3583391682 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60031640 con 0x7f9c6c0b1750 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.963+0000 7f9c708b2640 1 -- 172.21.15.70:0/3583391682 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c600317a0 con 0x7f9c6c0b1750 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.963+0000 7f9c708b2640 1 -- 172.21.15.70:0/3583391682 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c60031b30 con 0x7f9c6c0b1750 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.964+0000 7f9c73b18640 1 -- 172.21.15.70:0/3583391682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6c0b1750 msgr2=0x7f9c6c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.964+0000 7f9c73b18640 1 --2- 172.21.15.70:0/3583391682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6c0b1750 0x7f9c6c06cd00 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c06e330 tx=0x7f9c6002b8f0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.964+0000 7f9c73b18640 1 -- 172.21.15.70:0/3583391682 shutdown_connections 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.964+0000 7f9c73b18640 1 --2- 172.21.15.70:0/3583391682 >> v2:172.21.15.181:3300/0 conn(0x7f9c6c06d240 0x7f9c6c070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.964+0000 7f9c73b18640 1 --2- 172.21.15.70:0/3583391682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6c0b1750 0x7f9c6c06cd00 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.964+0000 7f9c73b18640 1 -- 172.21.15.70:0/3583391682 >> 172.21.15.70:0/3583391682 conn(0x7f9c6c069d50 msgr2=0x7f9c6c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.964+0000 7f9c73b18640 1 -- 172.21.15.70:0/3583391682 shutdown_connections 2023-10-08T05:56:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.964+0000 7f9c73b18640 1 -- 172.21.15.70:0/3583391682 wait complete. 2023-10-08T05:56:51.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.966+0000 7f9c73b18640 1 Processor -- start 2023-10-08T05:56:51.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.966+0000 7f9c73b18640 1 -- start start 2023-10-08T05:56:51.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c73b18640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6c06d240 0x7f9c6c151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c73b18640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c6c0b1750 0x7f9c6c0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c73b18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6c15aad0 0x7f9c6c0b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c73b18640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c0cc310 con 0x7f9c6c15aad0 2023-10-08T05:56:51.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c73b18640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c152340 con 0x7f9c6c0b1750 2023-10-08T05:56:51.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c73b18640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c159b50 con 0x7f9c6c06d240 2023-10-08T05:56:51.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c720b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6c15aad0 0x7f9c6c0b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c720b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6c15aad0 0x7f9c6c0b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49466/0 (socket says 172.21.15.70:49466) 2023-10-08T05:56:51.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c720b5640 1 -- 172.21.15.70:0/872288020 learned_addr learned my addr 172.21.15.70:0/872288020 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:51.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c710b3640 1 --2- 172.21.15.70:0/872288020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c6c0b1750 0x7f9c6c0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c720b5640 1 -- 172.21.15.70:0/872288020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6c06d240 msgr2=0x7f9c6c151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:51.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c718b4640 1 --2- 172.21.15.70:0/872288020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6c06d240 0x7f9c6c151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c720b5640 1 --2- 172.21.15.70:0/872288020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c6c06d240 0x7f9c6c151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c720b5640 1 -- 172.21.15.70:0/872288020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c6c0b1750 msgr2=0x7f9c6c0ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:51.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c720b5640 1 --2- 172.21.15.70:0/872288020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c6c0b1750 0x7f9c6c0ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:51.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c720b5640 1 -- 172.21.15.70:0/872288020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600094d0 con 0x7f9c6c15aad0 2023-10-08T05:56:51.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.968+0000 7f9c710b3640 1 --2- 172.21.15.70:0/872288020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9c6c0b1750 0x7f9c6c0ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:51.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.969+0000 7f9c720b5640 1 --2- 172.21.15.70:0/872288020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6c15aad0 0x7f9c6c0b26e0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c7f0 tx=0x7f9c6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:51.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.969+0000 7f9c6a7fc640 1 -- 172.21.15.70:0/872288020 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64007820 con 0x7f9c6c15aad0 2023-10-08T05:56:51.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.969+0000 7f9c6a7fc640 1 -- 172.21.15.70:0/872288020 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c64007980 con 0x7f9c6c15aad0 2023-10-08T05:56:51.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.969+0000 7f9c6a7fc640 1 -- 172.21.15.70:0/872288020 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64007d70 con 0x7f9c6c15aad0 2023-10-08T05:56:51.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.969+0000 7f9c73b18640 1 -- 172.21.15.70:0/872288020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6c0cc560 con 0x7f9c6c15aad0 2023-10-08T05:56:51.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.969+0000 7f9c73b18640 1 -- 172.21.15.70:0/872288020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c0ccb20 con 0x7f9c6c15aad0 2023-10-08T05:56:51.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.971+0000 7f9c73b18640 1 -- 172.21.15.70:0/872288020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6c0ccd10 con 0x7f9c6c15aad0 2023-10-08T05:56:51.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.972+0000 7f9c6a7fc640 1 -- 172.21.15.70:0/872288020 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 72) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f9c640113f0 con 0x7f9c6c15aad0 2023-10-08T05:56:51.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.972+0000 7f9c6a7fc640 1 --2- 172.21.15.70:0/872288020 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f9c5806b070 0x7f9c5806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:51.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.973+0000 7f9c6a7fc640 1 -- 172.21.15.70:0/872288020 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f9c6408a170 con 0x7f9c6c15aad0 2023-10-08T05:56:51.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.973+0000 7f9c718b4640 1 --2- 172.21.15.70:0/872288020 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f9c5806b070 0x7f9c5806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:51.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.974+0000 7f9c718b4640 1 --2- 172.21.15.70:0/872288020 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f9c5806b070 0x7f9c5806d530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c1f57d0 tx=0x7f9c5c0072f0 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:51.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:51.977+0000 7f9c6a7fc640 1 -- 172.21.15.70:0/872288020 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c64054ba0 con 0x7f9c6c15aad0 2023-10-08T05:56:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.186+0000 7f9c73b18640 1 -- 172.21.15.70:0/872288020 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1MM", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c6c15b8e0 con 0x7f9c5806b070 2023-10-08T05:56:52.190 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:52.190+0000 7f52f85fa640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-08T05:56:52.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.190+0000 7f9c6a7fc640 1 -- 172.21.15.70:0/872288020 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9c5c0079b0 con 0x7f9c5806b070 2023-10-08T05:56:52.191 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-08T05:56:52.206 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T05:56:52.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec 1 2023-10-08T05:56:52.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.428+0000 7fdb77fff640 1 Processor -- start 2023-10-08T05:56:52.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.429+0000 7fdb77fff640 1 -- start start 2023-10-08T05:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.430+0000 7fdb77fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdb780d06c0 0x7fdb7815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.430+0000 7fdb77fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdb780d0ec0 0x7fdb780d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.430+0000 7fdb77fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb780d3800 con 0x7fdb7815cd80 2023-10-08T05:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.430+0000 7fdb77fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb7815d160 con 0x7fdb780d0ec0 2023-10-08T05:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.430+0000 7fdb77fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb780ce370 con 0x7fdb780d06c0 2023-10-08T05:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.431+0000 7fdb777fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdb780d0ec0 0x7fdb780d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.431+0000 7fdb777fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdb780d0ec0 0x7fdb780d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49478/0 (socket says 172.21.15.70:49478) 2023-10-08T05:56:52.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.431+0000 7fdb777fe640 1 -- 172.21.15.70:0/2791810995 learned_addr learned my addr 172.21.15.70:0/2791810995 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:52.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.431+0000 7fdb777fe640 1 -- 172.21.15.70:0/2791810995 >> v2:172.21.15.181:3300/0 conn(0x7fdb780d06c0 msgr2=0x7fdb7815dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:52.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.431+0000 7fdb777fe640 1 --2- 172.21.15.70:0/2791810995 >> v2:172.21.15.181:3300/0 conn(0x7fdb780d06c0 0x7fdb7815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.431+0000 7fdb777fe640 1 -- 172.21.15.70:0/2791810995 >> v1:172.21.15.181:6789/0 conn(0x7fdb7815cd80 legacy=0x7fdb7815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:52.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.431+0000 7fdb777fe640 1 -- 172.21.15.70:0/2791810995 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7815ecf0 con 0x7fdb780d0ec0 2023-10-08T05:56:52.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.432+0000 7fdb777fe640 1 --2- 172.21.15.70:0/2791810995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb780d0ec0 0x7fdb780d12c0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fdb7808b190 tx=0x7fdb7002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c45acc7b3c4f59 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.432+0000 7fdb75ffb640 1 -- 172.21.15.70:0/2791810995 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb700317e0 con 0x7fdb780d0ec0 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.433+0000 7fdb75ffb640 1 -- 172.21.15.70:0/2791810995 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb70031940 con 0x7fdb780d0ec0 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.433+0000 7fdb75ffb640 1 -- 172.21.15.70:0/2791810995 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70031cd0 con 0x7fdb780d0ec0 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.433+0000 7fdb77fff640 1 -- 172.21.15.70:0/2791810995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb780d0ec0 msgr2=0x7fdb780d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.433+0000 7fdb77fff640 1 --2- 172.21.15.70:0/2791810995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb780d0ec0 0x7fdb780d12c0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fdb7808b190 tx=0x7fdb7002b880 comp rx=0 tx=0).stop 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.433+0000 7fdb77fff640 1 -- 172.21.15.70:0/2791810995 shutdown_connections 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.433+0000 7fdb77fff640 1 --2- 172.21.15.70:0/2791810995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb780d0ec0 0x7fdb780d12c0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.433+0000 7fdb77fff640 1 --2- 172.21.15.70:0/2791810995 >> v2:172.21.15.181:3300/0 conn(0x7fdb780d06c0 0x7fdb7815dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.433+0000 7fdb77fff640 1 -- 172.21.15.70:0/2791810995 >> 172.21.15.70:0/2791810995 conn(0x7fdb7809bc40 msgr2=0x7fdb780d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.433+0000 7fdb77fff640 1 -- 172.21.15.70:0/2791810995 shutdown_connections 2023-10-08T05:56:52.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.433+0000 7fdb77fff640 1 -- 172.21.15.70:0/2791810995 wait complete. 2023-10-08T05:56:52.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.435+0000 7fdb77fff640 1 Processor -- start 2023-10-08T05:56:52.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.435+0000 7fdb77fff640 1 -- start start 2023-10-08T05:56:52.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.436+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb780d06c0 0x7fdb781540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.436+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb780d0ec0 0x7fdb780b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.436+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb7815cd80 0x7fdb780d2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.436+0000 7fdb77fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb780b78a0 con 0x7fdb7815cd80 2023-10-08T05:56:52.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.436+0000 7fdb77fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb78157ee0 con 0x7fdb780d0ec0 2023-10-08T05:56:52.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.436+0000 7fdb77fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb780b6d20 con 0x7fdb780d06c0 2023-10-08T05:56:52.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb7815cd80 0x7fdb780d2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb7815cd80 0x7fdb780d2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49484/0 (socket says 172.21.15.70:49484) 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb777fe640 1 -- 172.21.15.70:0/3974947929 learned_addr learned my addr 172.21.15.70:0/3974947929 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb767fc640 1 --2- 172.21.15.70:0/3974947929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb780d0ec0 0x7fdb780b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb76ffd640 1 --2- 172.21.15.70:0/3974947929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb780d06c0 0x7fdb781540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb777fe640 1 -- 172.21.15.70:0/3974947929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb780d06c0 msgr2=0x7fdb781540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb777fe640 1 --2- 172.21.15.70:0/3974947929 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdb780d06c0 0x7fdb781540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb777fe640 1 -- 172.21.15.70:0/3974947929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb780d0ec0 msgr2=0x7fdb780b6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb777fe640 1 --2- 172.21.15.70:0/3974947929 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdb780d0ec0 0x7fdb780b6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb777fe640 1 -- 172.21.15.70:0/3974947929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb700094d0 con 0x7fdb7815cd80 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.437+0000 7fdb777fe640 1 --2- 172.21.15.70:0/3974947929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb7815cd80 0x7fdb780d2970 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fdb700090b0 tx=0x7fdb700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.438+0000 7fdb6f7fe640 1 -- 172.21.15.70:0/3974947929 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70032940 con 0x7fdb7815cd80 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.438+0000 7fdb6f7fe640 1 -- 172.21.15.70:0/3974947929 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb70032ac0 con 0x7fdb7815cd80 2023-10-08T05:56:52.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.438+0000 7fdb77fff640 1 -- 172.21.15.70:0/3974947929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb780b7000 con 0x7fdb7815cd80 2023-10-08T05:56:52.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.438+0000 7fdb6f7fe640 1 -- 172.21.15.70:0/3974947929 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70032e70 con 0x7fdb7815cd80 2023-10-08T05:56:52.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.438+0000 7fdb77fff640 1 -- 172.21.15.70:0/3974947929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb780b75c0 con 0x7fdb7815cd80 2023-10-08T05:56:52.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.439+0000 7fdb77fff640 1 -- 172.21.15.70:0/3974947929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb781580a0 con 0x7fdb7815cd80 2023-10-08T05:56:52.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.440+0000 7fdb6f7fe640 1 -- 172.21.15.70:0/3974947929 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 72) v1 ==== 89405+0+0 (secure 0 0 0) 0x7fdb7004d3f0 con 0x7fdb7815cd80 2023-10-08T05:56:52.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.441+0000 7fdb6f7fe640 1 --2- 172.21.15.70:0/3974947929 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fdb5806b070 0x7fdb5806d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.441+0000 7fdb6f7fe640 1 -- 172.21.15.70:0/3974947929 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fdb700abc50 con 0x7fdb7815cd80 2023-10-08T05:56:52.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.442+0000 7fdb76ffd640 1 --2- 172.21.15.70:0/3974947929 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fdb5806b070 0x7fdb5806d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.443+0000 7fdb76ffd640 1 --2- 172.21.15.70:0/3974947929 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fdb5806b070 0x7fdb5806d530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb780ca890 tx=0x7fdb68008040 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:52.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.445+0000 7fdb6f7fe640 1 -- 172.21.15.70:0/3974947929 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb700765f0 con 0x7fdb7815cd80 2023-10-08T05:56:52.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.657+0000 7fdb77fff640 1 -- 172.21.15.70:0/3974947929 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb780d1c00 con 0x7fdb5806b070 2023-10-08T05:56:52.661 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:52.661+0000 7f52f85fa640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-08T05:56:52.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.660+0000 7fdb6f7fe640 1 -- 172.21.15.70:0/3974947929 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdb780d1c00 con 0x7fdb5806b070 2023-10-08T05:56:52.662 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-08T05:56:52.675 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T05:56:52.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec M 2023-10-08T05:56:52.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.884+0000 7f6f05efc640 1 Processor -- start 2023-10-08T05:56:52.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.884+0000 7f6f05efc640 1 -- start start 2023-10-08T05:56:52.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.886+0000 7f6f05efc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f0015aad0 0x7f6f000b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.886+0000 7f6f05efc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f000c82b0 0x7f6f000c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.886+0000 7f6f05efc640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0015bf30 con 0x7f6f0006cd00 2023-10-08T05:56:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.886+0000 7f6f05efc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0015b320 con 0x7f6f000c82b0 2023-10-08T05:56:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.886+0000 7f6f05efc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0015b600 con 0x7f6f0015aad0 2023-10-08T05:56:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.887+0000 7f6eff7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f0015aad0 0x7f6f000b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.887+0000 7f6efeffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6f000c82b0 0x7f6f000c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.887+0000 7f6efffff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f6f0006cd00 0x7f6f00070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:46926/0 (socket says 172.21.15.70:46926) 2023-10-08T05:56:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.887+0000 7f6efffff640 1 -- 172.21.15.70:0/650316472 learned_addr learned my addr 172.21.15.70:0/650316472 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:52.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.888+0000 7f6efe7fc640 1 -- 172.21.15.70:0/650316472 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472065912 0 0) 0x7f6f0015bf30 con 0x7f6f0006cd00 2023-10-08T05:56:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.888+0000 7f6efe7fc640 1 -- 172.21.15.70:0/650316472 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee8003520 con 0x7f6f0006cd00 2023-10-08T05:56:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.888+0000 7f6eff7fe640 1 -- 172.21.15.70:0/650316472 >> v2:172.21.15.181:3300/0 conn(0x7f6f000c82b0 msgr2=0x7f6f000c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.888+0000 7f6eff7fe640 1 --2- 172.21.15.70:0/650316472 >> v2:172.21.15.181:3300/0 conn(0x7f6f000c82b0 0x7f6f000c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.888+0000 7f6eff7fe640 1 -- 172.21.15.70:0/650316472 >> v1:172.21.15.181:6789/0 conn(0x7f6f0006cd00 legacy=0x7f6f00070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.888+0000 7f6eff7fe640 1 -- 172.21.15.70:0/650316472 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0015b8e0 con 0x7f6f0015aad0 2023-10-08T05:56:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.888+0000 7f6efeffd640 1 --2- 172.21.15.70:0/650316472 >> v2:172.21.15.181:3300/0 conn(0x7f6f000c82b0 0x7f6f000c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:56:52.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.889+0000 7f6eff7fe640 1 --2- 172.21.15.70:0/650316472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f0015aad0 0x7f6f000b1750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00097a0 tx=0x7f6ef002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a73803ea0056720 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.890+0000 7f6efe7fc640 1 -- 172.21.15.70:0/650316472 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef002d040 con 0x7f6f0015aad0 2023-10-08T05:56:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.890+0000 7f6efe7fc640 1 -- 172.21.15.70:0/650316472 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef0031810 con 0x7f6f0015aad0 2023-10-08T05:56:52.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.890+0000 7f6efe7fc640 1 -- 172.21.15.70:0/650316472 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0031ba0 con 0x7f6f0015aad0 2023-10-08T05:56:52.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.891+0000 7f6f05efc640 1 -- 172.21.15.70:0/650316472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f0015aad0 msgr2=0x7f6f000b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:52.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.891+0000 7f6f05efc640 1 --2- 172.21.15.70:0/650316472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f0015aad0 0x7f6f000b1750 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00097a0 tx=0x7f6ef002b8b0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.891+0000 7f6f05efc640 1 -- 172.21.15.70:0/650316472 shutdown_connections 2023-10-08T05:56:52.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.891+0000 7f6f05efc640 1 --2- 172.21.15.70:0/650316472 >> v2:172.21.15.181:3300/0 conn(0x7f6f000c82b0 0x7f6f000c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.891+0000 7f6f05efc640 1 --2- 172.21.15.70:0/650316472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f0015aad0 0x7f6f000b1750 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.891+0000 7f6f05efc640 1 -- 172.21.15.70:0/650316472 >> 172.21.15.70:0/650316472 conn(0x7f6f00069d50 msgr2=0x7f6f0006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:52.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.891+0000 7f6f05efc640 1 -- 172.21.15.70:0/650316472 shutdown_connections 2023-10-08T05:56:52.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.891+0000 7f6f05efc640 1 -- 172.21.15.70:0/650316472 wait complete. 2023-10-08T05:56:52.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.893+0000 7f6f05efc640 1 Processor -- start 2023-10-08T05:56:52.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.893+0000 7f6f05efc640 1 -- start start 2023-10-08T05:56:52.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6f05efc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f0006cd00 0x7f6f00151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6f05efc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f000c82b0 0x7f6f00152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6f05efc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f0015aad0 0x7f6f001532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6f05efc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0014b090 con 0x7f6f000c82b0 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6f05efc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f00152a30 con 0x7f6f0015aad0 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6f05efc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f001f14e0 con 0x7f6f0006cd00 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6efeffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f000c82b0 0x7f6f00152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6efeffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f000c82b0 0x7f6f00152d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49490/0 (socket says 172.21.15.70:49490) 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6efeffd640 1 -- 172.21.15.70:0/1666066767 learned_addr learned my addr 172.21.15.70:0/1666066767 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6eff7fe640 1 --2- 172.21.15.70:0/1666066767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f0006cd00 0x7f6f00151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.894+0000 7f6efffff640 1 --2- 172.21.15.70:0/1666066767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f0015aad0 0x7f6f001532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6efeffd640 1 -- 172.21.15.70:0/1666066767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f0006cd00 msgr2=0x7f6f00151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6efeffd640 1 --2- 172.21.15.70:0/1666066767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6f0006cd00 0x7f6f00151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6efeffd640 1 -- 172.21.15.70:0/1666066767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f0015aad0 msgr2=0x7f6f001532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6efeffd640 1 --2- 172.21.15.70:0/1666066767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f0015aad0 0x7f6f001532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6efeffd640 1 -- 172.21.15.70:0/1666066767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4009400 con 0x7f6f000c82b0 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6efffff640 1 --2- 172.21.15.70:0/1666066767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6f0015aad0 0x7f6f001532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6efeffd640 1 --2- 172.21.15.70:0/1666066767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f000c82b0 0x7f6f00152d80 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400ebe0 tx=0x7f6ef400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6ee7fff640 1 -- 172.21.15.70:0/1666066767 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4009760 con 0x7f6f000c82b0 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6ee7fff640 1 -- 172.21.15.70:0/1666066767 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef400cc30 con 0x7f6f000c82b0 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6f05efc640 1 -- 172.21.15.70:0/1666066767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef0009400 con 0x7f6f000c82b0 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6ee7fff640 1 -- 172.21.15.70:0/1666066767 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400eee0 con 0x7f6f000c82b0 2023-10-08T05:56:52.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.895+0000 7f6f05efc640 1 -- 172.21.15.70:0/1666066767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f001f1aa0 con 0x7f6f000c82b0 2023-10-08T05:56:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.896+0000 7f6f05efc640 1 -- 172.21.15.70:0/1666066767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0004e160 con 0x7f6f000c82b0 2023-10-08T05:56:52.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.897+0000 7f6ee7fff640 1 -- 172.21.15.70:0/1666066767 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 72) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f6ef4013470 con 0x7f6f000c82b0 2023-10-08T05:56:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.898+0000 7f6ee7fff640 1 --2- 172.21.15.70:0/1666066767 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f6eec06b070 0x7f6eec06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.899+0000 7f6ee7fff640 1 -- 172.21.15.70:0/1666066767 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f6ef4089fe0 con 0x7f6f000c82b0 2023-10-08T05:56:52.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.899+0000 7f6eff7fe640 1 --2- 172.21.15.70:0/1666066767 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f6eec06b070 0x7f6eec06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:52.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.900+0000 7f6eff7fe640 1 --2- 172.21.15.70:0/1666066767 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f6eec06b070 0x7f6eec06d530 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0036000 tx=0x7f6ef003c040 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:52.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:52.903+0000 7f6ee7fff640 1 -- 172.21.15.70:0/1666066767 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ef4054ab0 con 0x7f6f000c82b0 2023-10-08T05:56:53.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.103+0000 7f6f05efc640 1 -- 172.21.15.70:0/1666066767 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f0015bbc0 con 0x7f6eec06b070 2023-10-08T05:56:53.107 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:53.107+0000 7f52f85fa640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-08T05:56:53.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.106+0000 7f6ee7fff640 1 -- 172.21.15.70:0/1666066767 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6ef0031b00 con 0x7f6eec06b070 2023-10-08T05:56:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-08T05:56:53.121 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T05:56:53.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec -1m 2023-10-08T05:56:53.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.330+0000 7f3235bd7640 1 Processor -- start 2023-10-08T05:56:53.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.330+0000 7f3235bd7640 1 -- start start 2023-10-08T05:56:53.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.331+0000 7f3235bd7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f323015aad0 0x7f32300b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.331+0000 7f3235bd7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323015bf30 con 0x7f323006d410 2023-10-08T05:56:53.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.331+0000 7f3235bd7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323015b320 con 0x7f32300c82b0 2023-10-08T05:56:53.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.331+0000 7f3235bd7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323015b600 con 0x7f323015aad0 2023-10-08T05:56:53.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.331+0000 7f322ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f323006d410 0x7f3230070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43622/0 (socket says 172.21.15.70:43622) 2023-10-08T05:56:53.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.331+0000 7f322ffff640 1 -- 172.21.15.70:0/1777941308 learned_addr learned my addr 172.21.15.70:0/1777941308 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322f7fe640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f323015aad0 0x7f32300b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654221517 0 0) 0x7f323015bf30 con 0x7f323006d410 2023-10-08T05:56:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f321c003520 con 0x7f323006d410 2023-10-08T05:56:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1490585949 0 0) 0x7f321c003520 con 0x7f323006d410 2023-10-08T05:56:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f323015aad0 msgr2=0x7f32300b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f323015aad0 0x7f32300b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 >> v1:172.21.15.181:6789/0 conn(0x7f32300c82b0 legacy=0x7f323006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323015b8e0 con 0x7f323006d410 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f32280019f0 con 0x7f323006d410 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 >> v1:172.21.15.70:6789/0 conn(0x7f323006d410 legacy=0x7f3230070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0039a0 0x7f321c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.332+0000 7f322effd640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c0047c0 0x7f321c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f322effd640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f323015aad0 0x7f321c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323015b600 con 0x7f321c0047c0 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c00ba30 con 0x7f321c0039a0 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c00bbf0 con 0x7f323015aad0 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f3227fff640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c0047c0 0x7f321c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f322f7fe640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0039a0 0x7f321c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f3227fff640 1 -- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f323015aad0 msgr2=0x7f321c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f3227fff640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f323015aad0 0x7f321c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f3227fff640 1 -- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0039a0 msgr2=0x7f321c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f3227fff640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0039a0 0x7f321c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f3227fff640 1 -- 172.21.15.70:0/1777941308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c003520 con 0x7f321c0047c0 2023-10-08T05:56:53.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.333+0000 7f322f7fe640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0039a0 0x7f321c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:53.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.334+0000 7f3227fff640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c0047c0 0x7f321c004c40 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f3218001720 tx=0x7f321c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f730bbe7e441d8c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:53.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.335+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32180052d0 con 0x7f321c0047c0 2023-10-08T05:56:53.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.335+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321802e050 con 0x7f321c0047c0 2023-10-08T05:56:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.335+0000 7f322effd640 1 -- 172.21.15.70:0/1777941308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32180035f0 con 0x7f321c0047c0 2023-10-08T05:56:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.335+0000 7f3235bd7640 1 -- 172.21.15.70:0/1777941308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c0047c0 msgr2=0x7f321c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.335+0000 7f3235bd7640 1 --2- 172.21.15.70:0/1777941308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c0047c0 0x7f321c004c40 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f3218001720 tx=0x7f321c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.335+0000 7f322ffff640 1 -- 172.21.15.70:0/1777941308 reap_dead start 2023-10-08T05:56:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.335+0000 7f3235bd7640 1 -- 172.21.15.70:0/1777941308 shutdown_connections 2023-10-08T05:56:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.335+0000 7f3235bd7640 1 -- 172.21.15.70:0/1777941308 >> 172.21.15.70:0/1777941308 conn(0x7f3230069d50 msgr2=0x7f323006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.335+0000 7f3235bd7640 1 -- 172.21.15.70:0/1777941308 shutdown_connections 2023-10-08T05:56:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.335+0000 7f3235bd7640 1 -- 172.21.15.70:0/1777941308 wait complete. 2023-10-08T05:56:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.337+0000 7f3235bd7640 1 Processor -- start 2023-10-08T05:56:53.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.337+0000 7f3235bd7640 1 -- start start 2023-10-08T05:56:53.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.338+0000 7f3235bd7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0047c0 0x7f323014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.338+0000 7f3235bd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32301f4720 0x7f32300c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.338+0000 7f3235bd7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32301f3aa0 0x7f32301f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.338+0000 7f3235bd7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323014a550 con 0x7f32301f4720 2023-10-08T05:56:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.338+0000 7f3235bd7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32301f4460 con 0x7f321c0047c0 2023-10-08T05:56:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.338+0000 7f3235bd7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323013c330 con 0x7f32301f3aa0 2023-10-08T05:56:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f3227fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32301f4720 0x7f32300c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f3227fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32301f4720 0x7f32300c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49516/0 (socket says 172.21.15.70:49516) 2023-10-08T05:56:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f3227fff640 1 -- 172.21.15.70:0/2832843574 learned_addr learned my addr 172.21.15.70:0/2832843574 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f322f7fe640 1 --2- 172.21.15.70:0/2832843574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0047c0 0x7f323014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f322ffff640 1 --2- 172.21.15.70:0/2832843574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32301f3aa0 0x7f32301f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f3227fff640 1 -- 172.21.15.70:0/2832843574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32301f3aa0 msgr2=0x7f32301f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f3227fff640 1 --2- 172.21.15.70:0/2832843574 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32301f3aa0 0x7f32301f3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f3227fff640 1 -- 172.21.15.70:0/2832843574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0047c0 msgr2=0x7f323014a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f3227fff640 1 --2- 172.21.15.70:0/2832843574 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0047c0 0x7f323014a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f3227fff640 1 -- 172.21.15.70:0/2832843574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323013c580 con 0x7f32301f4720 2023-10-08T05:56:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.339+0000 7f3227fff640 1 --2- 172.21.15.70:0/2832843574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32301f4720 0x7f32300c71c0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f3218008fd0 tx=0x7f3218001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.340+0000 7f322cff9640 1 -- 172.21.15.70:0/2832843574 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32180076c0 con 0x7f32301f4720 2023-10-08T05:56:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.340+0000 7f322cff9640 1 -- 172.21.15.70:0/2832843574 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3218007a30 con 0x7f32301f4720 2023-10-08T05:56:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.340+0000 7f322cff9640 1 -- 172.21.15.70:0/2832843574 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218006be0 con 0x7f32301f4720 2023-10-08T05:56:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.340+0000 7f3235bd7640 1 -- 172.21.15.70:0/2832843574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323013c860 con 0x7f32301f4720 2023-10-08T05:56:53.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.340+0000 7f3235bd7640 1 -- 172.21.15.70:0/2832843574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323013ce20 con 0x7f32301f4720 2023-10-08T05:56:53.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.341+0000 7f3235bd7640 1 -- 172.21.15.70:0/2832843574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323004e160 con 0x7f32301f4720 2023-10-08T05:56:53.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.342+0000 7f322cff9640 1 -- 172.21.15.70:0/2832843574 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 72) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f3218006d40 con 0x7f32301f4720 2023-10-08T05:56:53.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.343+0000 7f322cff9640 1 --2- 172.21.15.70:0/2832843574 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f321006b070 0x7f321006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.343+0000 7f322cff9640 1 -- 172.21.15.70:0/2832843574 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f32180abf40 con 0x7f32301f4720 2023-10-08T05:56:53.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.343+0000 7f322f7fe640 1 --2- 172.21.15.70:0/2832843574 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f321006b070 0x7f321006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.345+0000 7f322f7fe640 1 --2- 172.21.15.70:0/2832843574 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f321006b070 0x7f321006d530 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3220009aa0 tx=0x7f3220008040 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:53.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.347+0000 7f322cff9640 1 -- 172.21.15.70:0/2832843574 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32180768e0 con 0x7f32301f4720 2023-10-08T05:56:53.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.561+0000 7f3235bd7640 1 -- 172.21.15.70:0/2832843574 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "-1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f323015bbc0 con 0x7f321006b070 2023-10-08T05:56:53.565 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:53.564+0000 7f52f85fa640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2023-10-08T05:56:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.565+0000 7f322cff9640 1 -- 172.21.15.70:0/2832843574 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc.) v1 ==== 96+0+0 (secure 0 0 0) 0x7f32200129b0 con 0x7f321006b070 2023-10-08T05:56:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENOENT: invalid schedule specified - period must be a non-zero positive value e.g. 1h or 4d etc. 2023-10-08T05:56:53.579 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T05:56:53.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1/bad_period_spec '' 2023-10-08T05:56:53.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.796+0000 7feb9670f640 1 Processor -- start 2023-10-08T05:56:53.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.796+0000 7feb9670f640 1 -- start start 2023-10-08T05:56:53.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.798+0000 7feb9670f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feb900b1750 0x7feb9006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.798+0000 7feb9670f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9006d670 con 0x7feb9015aad0 2023-10-08T05:56:53.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.798+0000 7feb9670f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9015bf30 con 0x7feb9006d240 2023-10-08T05:56:53.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.798+0000 7feb9670f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb900b1bb0 con 0x7feb900b1750 2023-10-08T05:56:53.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.798+0000 7feb8ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feb9015aad0 0x7feb900c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33238/0 (socket says 172.21.15.70:33238) 2023-10-08T05:56:53.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.798+0000 7feb8ffff640 1 -- 172.21.15.70:0/2701247044 learned_addr learned my addr 172.21.15.70:0/2701247044 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:53.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.799+0000 7feb8f7fe640 1 --2- 172.21.15.70:0/2701247044 >> v2:172.21.15.70:3300/0 conn(0x7feb900b1750 0x7feb9006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.799+0000 7feb8effd640 1 -- 172.21.15.70:0/2701247044 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430684585 0 0) 0x7feb9006d670 con 0x7feb9015aad0 2023-10-08T05:56:53.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.799+0000 7feb8effd640 1 -- 172.21.15.70:0/2701247044 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb7c003520 con 0x7feb9015aad0 2023-10-08T05:56:53.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.799+0000 7feb8f7fe640 1 -- 172.21.15.70:0/2701247044 >> v1:172.21.15.181:6789/0 conn(0x7feb9006d240 legacy=0x7feb90070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:53.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.799+0000 7feb8f7fe640 1 -- 172.21.15.70:0/2701247044 >> v1:172.21.15.70:6789/0 conn(0x7feb9015aad0 legacy=0x7feb900c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:53.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.799+0000 7feb8f7fe640 1 -- 172.21.15.70:0/2701247044 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9015b320 con 0x7feb900b1750 2023-10-08T05:56:53.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.800+0000 7feb8f7fe640 1 --2- 172.21.15.70:0/2701247044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb900b1750 0x7feb9006cd00 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7feb780097a0 tx=0x7feb7802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d27041fe464f766 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:53.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.801+0000 7feb8effd640 1 -- 172.21.15.70:0/2701247044 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78030610 con 0x7feb900b1750 2023-10-08T05:56:53.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.801+0000 7feb8effd640 1 -- 172.21.15.70:0/2701247044 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb78030770 con 0x7feb900b1750 2023-10-08T05:56:53.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.801+0000 7feb8effd640 1 -- 172.21.15.70:0/2701247044 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78030ae0 con 0x7feb900b1750 2023-10-08T05:56:53.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.801+0000 7feb9670f640 1 -- 172.21.15.70:0/2701247044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb900b1750 msgr2=0x7feb9006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:53.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.801+0000 7feb9670f640 1 --2- 172.21.15.70:0/2701247044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb900b1750 0x7feb9006cd00 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7feb780097a0 tx=0x7feb7802b960 comp rx=0 tx=0).stop 2023-10-08T05:56:53.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.801+0000 7feb9670f640 1 -- 172.21.15.70:0/2701247044 shutdown_connections 2023-10-08T05:56:53.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.801+0000 7feb9670f640 1 --2- 172.21.15.70:0/2701247044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb900b1750 0x7feb9006cd00 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:53.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.801+0000 7feb9670f640 1 -- 172.21.15.70:0/2701247044 >> 172.21.15.70:0/2701247044 conn(0x7feb90069d50 msgr2=0x7feb9006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:53.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.801+0000 7feb9670f640 1 -- 172.21.15.70:0/2701247044 shutdown_connections 2023-10-08T05:56:53.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.802+0000 7feb9670f640 1 -- 172.21.15.70:0/2701247044 wait complete. 2023-10-08T05:56:53.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.804+0000 7feb9670f640 1 Processor -- start 2023-10-08T05:56:53.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.804+0000 7feb9670f640 1 -- start start 2023-10-08T05:56:53.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.805+0000 7feb9670f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9006d240 0x7feb900b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.805+0000 7feb9670f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb900b1750 0x7feb9014bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.805+0000 7feb9670f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb9015aad0 0x7feb900c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.805+0000 7feb9670f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9014f7d0 con 0x7feb9006d240 2023-10-08T05:56:53.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.805+0000 7feb9670f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9014fab0 con 0x7feb900b1750 2023-10-08T05:56:53.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.805+0000 7feb9670f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9014fd90 con 0x7feb9015aad0 2023-10-08T05:56:53.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.806+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9006d240 0x7feb900b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.806+0000 7feb8ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9006d240 0x7feb900b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33880/0 (socket says 172.21.15.70:33880) 2023-10-08T05:56:53.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.806+0000 7feb8ffff640 1 -- 172.21.15.70:0/1499679212 learned_addr learned my addr 172.21.15.70:0/1499679212 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:53.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.806+0000 7feb8f7fe640 1 --2- 172.21.15.70:0/1499679212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb900b1750 0x7feb9014bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.806+0000 7feb94cac640 1 --2- 172.21.15.70:0/1499679212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb9015aad0 0x7feb900c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.806+0000 7feb8ffff640 1 -- 172.21.15.70:0/1499679212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb9015aad0 msgr2=0x7feb900c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.806+0000 7feb8ffff640 1 --2- 172.21.15.70:0/1499679212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb9015aad0 0x7feb900c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.806+0000 7feb8ffff640 1 -- 172.21.15.70:0/1499679212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb900b1750 msgr2=0x7feb9014bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.806+0000 7feb8ffff640 1 --2- 172.21.15.70:0/1499679212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb900b1750 0x7feb9014bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.806+0000 7feb8ffff640 1 -- 172.21.15.70:0/1499679212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb78009400 con 0x7feb9006d240 2023-10-08T05:56:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.807+0000 7feb8ffff640 1 --2- 172.21.15.70:0/1499679212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb9006d240 0x7feb900b2b60 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7feb8000c400 tx=0x7feb8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.807+0000 7feb8cff9640 1 -- 172.21.15.70:0/1499679212 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80005ce0 con 0x7feb9006d240 2023-10-08T05:56:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.807+0000 7feb8cff9640 1 -- 172.21.15.70:0/1499679212 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb80005e40 con 0x7feb9006d240 2023-10-08T05:56:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.807+0000 7feb8cff9640 1 -- 172.21.15.70:0/1499679212 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb800072d0 con 0x7feb9006d240 2023-10-08T05:56:53.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.807+0000 7feb9670f640 1 -- 172.21.15.70:0/1499679212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb90150070 con 0x7feb9006d240 2023-10-08T05:56:53.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.807+0000 7feb9670f640 1 -- 172.21.15.70:0/1499679212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb900b30a0 con 0x7feb9006d240 2023-10-08T05:56:53.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.808+0000 7feb9670f640 1 -- 172.21.15.70:0/1499679212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb900b32f0 con 0x7feb9006d240 2023-10-08T05:56:53.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.809+0000 7feb8cff9640 1 -- 172.21.15.70:0/1499679212 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 72) v1 ==== 89405+0+0 (secure 0 0 0) 0x7feb80007430 con 0x7feb9006d240 2023-10-08T05:56:53.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.810+0000 7feb8cff9640 1 --2- 172.21.15.70:0/1499679212 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7feb7006b0a0 0x7feb7006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:53.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.810+0000 7feb8cff9640 1 -- 172.21.15.70:0/1499679212 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7feb80089250 con 0x7feb9006d240 2023-10-08T05:56:53.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.811+0000 7feb8f7fe640 1 --2- 172.21.15.70:0/1499679212 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7feb7006b0a0 0x7feb7006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:53.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.812+0000 7feb8f7fe640 1 --2- 172.21.15.70:0/1499679212 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7feb7006b0a0 0x7feb7006d560 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb78036000 tx=0x7feb78008030 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:53.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:53.816+0000 7feb8cff9640 1 -- 172.21.15.70:0/1499679212 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb80053d20 con 0x7feb9006d240 2023-10-08T05:56:54.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.011+0000 7feb9670f640 1 -- 172.21.15.70:0/1499679212 --> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1/bad_period_spec", "snap_schedule": "", "target": ["mon-mgr", ""]}) v1 -- 0x7feb9015b8e0 con 0x7feb7006b0a0 2023-10-08T05:56:54.015 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:54.015+0000 7f52f85fa640 -1 mgr.server reply reply (2) No such file or directory invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-08T05:56:54.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.015+0000 7feb8cff9640 1 -- 172.21.15.70:0/1499679212 <== mgr.6200 v2:172.21.15.181:6832/1964243521 1 ==== mgr_command_reply(tid 0: -2 invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc.) v1 ==== 138+0+0 (secure 0 0 0) 0x7feb78041040 con 0x7feb7006b0a0 2023-10-08T05:56:54.016 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENOENT: invalid schedule specified - period should be non-zero positive value and multiplier should be one of h,d,w,M,Y e.g. 1h or 4d etc. 2023-10-08T05:56:54.030 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T05:56:54.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:54.031 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/minutes) 2023-10-08T05:56:54.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:54.084 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/hourly) 2023-10-08T05:56:54.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:54.143 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/daily) 2023-10-08T05:56:54.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:54.201 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/weekly) 2023-10-08T05:56:54.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:54.252 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/monthly) 2023-10-08T05:56:54.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:54.305 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/yearly) 2023-10-08T05:56:54.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:54.359 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/bad_period_spec) 2023-10-08T05:56:54.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T05:56:54.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.623+0000 7f25c8d25640 1 Processor -- start 2023-10-08T05:56:54.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.623+0000 7f25c8d25640 1 -- start start 2023-10-08T05:56:54.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.625+0000 7f25c8d25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c415aad0 0x7f25c40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:54.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.625+0000 7f25c8d25640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f25c40c82b0 0x7f25c40c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:54.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.625+0000 7f25c8d25640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c415bf30 con 0x7f25c406cd00 2023-10-08T05:56:54.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.625+0000 7f25c8d25640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c415b320 con 0x7f25c40c82b0 2023-10-08T05:56:54.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.625+0000 7f25c8d25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c415b600 con 0x7f25c415aad0 2023-10-08T05:56:54.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.625+0000 7f25b9d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f25c40c82b0 0x7f25c40c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:54.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.625+0000 7f25c2d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f25c406cd00 0x7f25c4070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33246/0 (socket says 172.21.15.70:33246) 2023-10-08T05:56:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.625+0000 7f25c2d9d640 1 -- 172.21.15.70:0/128450283 learned_addr learned my addr 172.21.15.70:0/128450283 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.626+0000 7f25c259c640 1 --2- 172.21.15.70:0/128450283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c415aad0 0x7f25c40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.626+0000 7f25b9d9b640 1 -- 172.21.15.70:0/128450283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c415aad0 msgr2=0x7f25c40b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.626+0000 7f25b9d9b640 1 --2- 172.21.15.70:0/128450283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c415aad0 0x7f25c40b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.626+0000 7f25c1d9b640 1 -- 172.21.15.70:0/128450283 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697804780 0 0) 0x7f25c415bf30 con 0x7f25c406cd00 2023-10-08T05:56:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.626+0000 7f25b9d9b640 1 -- 172.21.15.70:0/128450283 >> v1:172.21.15.70:6789/0 conn(0x7f25c406cd00 legacy=0x7f25c4070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.626+0000 7f25b9d9b640 1 -- 172.21.15.70:0/128450283 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c415b8e0 con 0x7f25c40c82b0 2023-10-08T05:56:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.627+0000 7f25b9d9b640 1 --2- 172.21.15.70:0/128450283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 0x7f25c40c8710 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0097a0 tx=0x7f25ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956a32fb88cbd401 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.627+0000 7f25c1d9b640 1 -- 172.21.15.70:0/128450283 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac031830 con 0x7f25c40c82b0 2023-10-08T05:56:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.627+0000 7f25c1d9b640 1 -- 172.21.15.70:0/128450283 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac031990 con 0x7f25c40c82b0 2023-10-08T05:56:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.627+0000 7f25c1d9b640 1 -- 172.21.15.70:0/128450283 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac031d20 con 0x7f25c40c82b0 2023-10-08T05:56:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.628+0000 7f25c8d25640 1 -- 172.21.15.70:0/128450283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 msgr2=0x7f25c40c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.628+0000 7f25c8d25640 1 --2- 172.21.15.70:0/128450283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 0x7f25c40c8710 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0097a0 tx=0x7f25ac02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:56:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.628+0000 7f25c8d25640 1 -- 172.21.15.70:0/128450283 shutdown_connections 2023-10-08T05:56:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.628+0000 7f25c8d25640 1 --2- 172.21.15.70:0/128450283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 0x7f25c40c8710 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.628+0000 7f25c8d25640 1 --2- 172.21.15.70:0/128450283 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c415aad0 0x7f25c40b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.628+0000 7f25c8d25640 1 -- 172.21.15.70:0/128450283 >> 172.21.15.70:0/128450283 conn(0x7f25c4069d50 msgr2=0x7f25c406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.628+0000 7f25c8d25640 1 -- 172.21.15.70:0/128450283 shutdown_connections 2023-10-08T05:56:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.628+0000 7f25c8d25640 1 -- 172.21.15.70:0/128450283 wait complete. 2023-10-08T05:56:54.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.630+0000 7f25c8d25640 1 Processor -- start 2023-10-08T05:56:54.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.630+0000 7f25c8d25640 1 -- start start 2023-10-08T05:56:54.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.631+0000 7f25c8d25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c406cd00 0x7f25c41f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:54.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.631+0000 7f25c8d25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 0x7f25c41f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:54.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.631+0000 7f25c8d25640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c415aad0 0x7f25c40ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:54.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.631+0000 7f25c8d25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c40c3af0 con 0x7f25c40c82b0 2023-10-08T05:56:54.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.631+0000 7f25c8d25640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c4157810 con 0x7f25c415aad0 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.631+0000 7f25c8d25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c41f6540 con 0x7f25c406cd00 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.631+0000 7f25b9d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 0x7f25c41f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.631+0000 7f25b9d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 0x7f25c41f1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33906/0 (socket says 172.21.15.70:33906) 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.631+0000 7f25b9d9b640 1 -- 172.21.15.70:0/3553649767 learned_addr learned my addr 172.21.15.70:0/3553649767 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.632+0000 7f25c259c640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c406cd00 0x7f25c41f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.632+0000 7f25c2d9d640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c415aad0 0x7f25c40ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.632+0000 7f25b9d9b640 1 -- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c406cd00 msgr2=0x7f25c41f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.632+0000 7f25b9d9b640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c406cd00 0x7f25c41f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.632+0000 7f25b9d9b640 1 -- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c415aad0 msgr2=0x7f25c40ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.632+0000 7f25b9d9b640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c415aad0 0x7f25c40ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.632+0000 7f25b9d9b640 1 -- 172.21.15.70:0/3553649767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac009400 con 0x7f25c40c82b0 2023-10-08T05:56:54.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.632+0000 7f25c259c640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c406cd00 0x7f25c41f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:54.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.632+0000 7f25b9d9b640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 0x7f25c41f1990 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f25ac000c00 tx=0x7f25ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:54.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.633+0000 7f25bb7fe640 1 -- 172.21.15.70:0/3553649767 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac004230 con 0x7f25c40c82b0 2023-10-08T05:56:54.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.633+0000 7f25bb7fe640 1 -- 172.21.15.70:0/3553649767 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac004390 con 0x7f25c40c82b0 2023-10-08T05:56:54.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.633+0000 7f25bb7fe640 1 -- 172.21.15.70:0/3553649767 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac03a880 con 0x7f25c40c82b0 2023-10-08T05:56:54.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.633+0000 7f25c8d25640 1 -- 172.21.15.70:0/3553649767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c41f1ed0 con 0x7f25c40c82b0 2023-10-08T05:56:54.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.633+0000 7f25c8d25640 1 -- 172.21.15.70:0/3553649767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c4156ee0 con 0x7f25c40c82b0 2023-10-08T05:56:54.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.634+0000 7f25c8d25640 1 -- 172.21.15.70:0/3553649767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c404e160 con 0x7f25c40c82b0 2023-10-08T05:56:54.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.635+0000 7f25bb7fe640 1 -- 172.21.15.70:0/3553649767 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 72) v1 ==== 89405+0+0 (secure 0 0 0) 0x7f25ac03a9e0 con 0x7f25c40c82b0 2023-10-08T05:56:54.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.636+0000 7f25bb7fe640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f25a406b0a0 0x7f25a406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:54.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.636+0000 7f25bb7fe640 1 -- 172.21.15.70:0/3553649767 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f25ac0ac7e0 con 0x7f25c40c82b0 2023-10-08T05:56:54.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.637+0000 7f25c259c640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f25a406b0a0 0x7f25a406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:54.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.638+0000 7f25c259c640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f25a406b0a0 0x7f25a406d560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f25b4006fd0 tx=0x7f25b4008040 comp rx=0 tx=0).ready entity=mgr.6200 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:54.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.641+0000 7f25bb7fe640 1 -- 172.21.15.70:0/3553649767 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f25ac077320 con 0x7f25c40c82b0 2023-10-08T05:56:54.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:54.878+0000 7f25c8d25640 1 -- 172.21.15.70:0/3553649767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f25c40b4610 con 0x7f25c40c82b0 2023-10-08T05:56:55.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.440+0000 7f25bb7fe640 1 -- 172.21.15.70:0/3553649767 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v73) v1 ==== 93+0+0 (secure 0 0 0) 0x7f25ac07a940 con 0x7f25c40c82b0 2023-10-08T05:56:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.444+0000 7f25c8d25640 1 -- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f25a406b0a0 msgr2=0x7f25a406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.444+0000 7f25c8d25640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f25a406b0a0 0x7f25a406d560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f25b4006fd0 tx=0x7f25b4008040 comp rx=0 tx=0).stop 2023-10-08T05:56:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.444+0000 7f25c8d25640 1 -- 172.21.15.70:0/3553649767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 msgr2=0x7f25c41f1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.444+0000 7f25c8d25640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 0x7f25c41f1990 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f25ac000c00 tx=0x7f25ac0046c0 comp rx=0 tx=0).stop 2023-10-08T05:56:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.444+0000 7f25c8d25640 1 -- 172.21.15.70:0/3553649767 shutdown_connections 2023-10-08T05:56:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.444+0000 7f25c8d25640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f25a406b0a0 0x7f25a406d560 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.444+0000 7f25c8d25640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f25c415aad0 0x7f25c40ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.444+0000 7f25c8d25640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c40c82b0 0x7f25c41f1990 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.444+0000 7f25c8d25640 1 --2- 172.21.15.70:0/3553649767 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f25c406cd00 0x7f25c41f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:55.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.444+0000 7f25c8d25640 1 -- 172.21.15.70:0/3553649767 >> 172.21.15.70:0/3553649767 conn(0x7f25c4069d50 msgr2=0x7f25c415d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:55.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.445+0000 7f25c8d25640 1 -- 172.21.15.70:0/3553649767 shutdown_connections 2023-10-08T05:56:55.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:55.445+0000 7f25c8d25640 1 -- 172.21.15.70:0/3553649767 wait complete. 2023-10-08T05:56:55.468 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:56:55.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:56:55.575 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:56:55.673 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:56:55.674 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:56:55.674 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:55.728 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:56:55.728 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:55.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:55.973 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:55.972+0000 7f0968e56040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:56:55.980 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:55.979+0000 7fdcbf2b6040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:56:56.026 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:56.024+0000 7f0968e56040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:56:56.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.027+0000 7f413d705640 1 Processor -- start 2023-10-08T05:56:56.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.027+0000 7f413d705640 1 -- start start 2023-10-08T05:56:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.028+0000 7f413d705640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f413815ab70 0x7f41380cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.028+0000 7f413d705640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f41380d04e0 0x7f41380cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f413d705640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413815af50 con 0x7f41380cdca0 2023-10-08T05:56:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f413d705640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41380ce110 con 0x7f41380d04e0 2023-10-08T05:56:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f413d705640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41380ced20 con 0x7f413815ab70 2023-10-08T05:56:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f41377fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f41380cdca0 0x7f41380d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33252/0 (socket says 172.21.15.70:33252) 2023-10-08T05:56:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f41377fe640 1 -- 172.21.15.70:0/427517421 learned_addr learned my addr 172.21.15.70:0/427517421 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:56.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f412ffff640 1 --2- 172.21.15.70:0/427517421 >> v2:172.21.15.70:3300/0 conn(0x7f41380d04e0 0x7f41380cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f4136ffd640 1 --2- 172.21.15.70:0/427517421 >> v2:172.21.15.181:3300/0 conn(0x7f413815ab70 0x7f41380cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f412ffff640 1 -- 172.21.15.70:0/427517421 >> v2:172.21.15.181:3300/0 conn(0x7f413815ab70 msgr2=0x7f41380cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:56.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f412ffff640 1 --2- 172.21.15.70:0/427517421 >> v2:172.21.15.181:3300/0 conn(0x7f413815ab70 0x7f41380cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f412ffff640 1 -- 172.21.15.70:0/427517421 >> v1:172.21.15.70:6789/0 conn(0x7f41380cdca0 legacy=0x7f41380d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.030+0000 7f412ffff640 1 -- 172.21.15.70:0/427517421 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41380cf000 con 0x7f41380d04e0 2023-10-08T05:56:56.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.029+0000 7f41367fc640 1 -- 172.21.15.70:0/427517421 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843990166 0 0) 0x7f413815af50 con 0x7f41380cdca0 2023-10-08T05:56:56.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.031+0000 7f412ffff640 1 --2- 172.21.15.70:0/427517421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380d04e0 0x7f41380cd760 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f41380cd3d0 tx=0x7f412002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93693a9e756c7011 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:56.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.031+0000 7f41367fc640 1 -- 172.21.15.70:0/427517421 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120031830 con 0x7f41380d04e0 2023-10-08T05:56:56.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.031+0000 7f41367fc640 1 -- 172.21.15.70:0/427517421 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4120031990 con 0x7f41380d04e0 2023-10-08T05:56:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.031+0000 7f41367fc640 1 -- 172.21.15.70:0/427517421 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120031d20 con 0x7f41380d04e0 2023-10-08T05:56:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.032+0000 7f413d705640 1 -- 172.21.15.70:0/427517421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380d04e0 msgr2=0x7f41380cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.032+0000 7f413d705640 1 --2- 172.21.15.70:0/427517421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380d04e0 0x7f41380cd760 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f41380cd3d0 tx=0x7f412002b8d0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.032+0000 7f413d705640 1 -- 172.21.15.70:0/427517421 shutdown_connections 2023-10-08T05:56:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.032+0000 7f413d705640 1 --2- 172.21.15.70:0/427517421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380d04e0 0x7f41380cd760 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.032+0000 7f413d705640 1 --2- 172.21.15.70:0/427517421 >> v2:172.21.15.181:3300/0 conn(0x7f413815ab70 0x7f41380cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.032+0000 7f413d705640 1 -- 172.21.15.70:0/427517421 >> 172.21.15.70:0/427517421 conn(0x7f4138069f30 msgr2=0x7f413806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.032+0000 7f413d705640 1 -- 172.21.15.70:0/427517421 shutdown_connections 2023-10-08T05:56:56.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.032+0000 7f413d705640 1 -- 172.21.15.70:0/427517421 wait complete. 2023-10-08T05:56:56.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.033+0000 7f413d705640 1 Processor -- start 2023-10-08T05:56:56.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.034+0000 7f413d705640 1 -- start start 2023-10-08T05:56:56.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.034+0000 7f413d705640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380cdca0 0x7f413815c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f413d705640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41380d04e0 0x7f4138158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f413d705640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f413815ab70 0x7f413815dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f413d705640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4138158d40 con 0x7f41380cdca0 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f413d705640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41380c5d70 con 0x7f413815ab70 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f413d705640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41380c6050 con 0x7f41380d04e0 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f4136ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380cdca0 0x7f413815c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f4136ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380cdca0 0x7f413815c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33946/0 (socket says 172.21.15.70:33946) 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f4136ffd640 1 -- 172.21.15.70:0/2373104146 learned_addr learned my addr 172.21.15.70:0/2373104146 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f41377fe640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f413815ab70 0x7f413815dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f412ffff640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41380d04e0 0x7f4138158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f4136ffd640 1 -- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41380d04e0 msgr2=0x7f4138158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f4136ffd640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41380d04e0 0x7f4138158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f4136ffd640 1 -- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f413815ab70 msgr2=0x7f413815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f4136ffd640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f413815ab70 0x7f413815dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f4136ffd640 1 -- 172.21.15.70:0/2373104146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120009400 con 0x7f41380cdca0 2023-10-08T05:56:56.038 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:56.035+0000 7fdcbf2b6040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:56:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.035+0000 7f4136ffd640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380cdca0 0x7f413815c480 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f412800d5f0 tx=0x7f412800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.036+0000 7f412f7fe640 1 -- 172.21.15.70:0/2373104146 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280125f0 con 0x7f41380cdca0 2023-10-08T05:56:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.036+0000 7f412f7fe640 1 -- 172.21.15.70:0/2373104146 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4128012750 con 0x7f41380cdca0 2023-10-08T05:56:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.036+0000 7f412f7fe640 1 -- 172.21.15.70:0/2373104146 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128012b00 con 0x7f41380cdca0 2023-10-08T05:56:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.036+0000 7f413d705640 1 -- 172.21.15.70:0/2373104146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41380c6330 con 0x7f41380cdca0 2023-10-08T05:56:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.036+0000 7f413d705640 1 -- 172.21.15.70:0/2373104146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41380c68f0 con 0x7f41380cdca0 2023-10-08T05:56:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.037+0000 7f413d705640 1 -- 172.21.15.70:0/2373104146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41380c6a80 con 0x7f41380cdca0 2023-10-08T05:56:56.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.038+0000 7f412f7fe640 1 -- 172.21.15.70:0/2373104146 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 73) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f4128012c60 con 0x7f41380cdca0 2023-10-08T05:56:56.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.039+0000 7f412f7fe640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f411806b020 0x7f411806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.039+0000 7f412f7fe640 1 -- 172.21.15.70:0/2373104146 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f412808a180 con 0x7f41380cdca0 2023-10-08T05:56:56.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.039+0000 7f412ffff640 1 -- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f411806b020 msgr2=0x7f411806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:56.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.039+0000 7f412ffff640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f411806b020 0x7f411806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:56.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.047+0000 7f412f7fe640 1 -- 172.21.15.70:0/2373104146 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4128054c30 con 0x7f41380cdca0 2023-10-08T05:56:56.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.239+0000 7f413d705640 1 -- 172.21.15.70:0/2373104146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4138066d80 con 0x7f41380cdca0 2023-10-08T05:56:56.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.240+0000 7f412ffff640 1 -- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f411806b020 msgr2=0x7f411806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:56.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.240+0000 7f412ffff640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f411806b020 0x7f411806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:56:56.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.241+0000 7f412f7fe640 1 -- 172.21.15.70:0/2373104146 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+17395 (secure 0 0 0) 0x7f4128058250 con 0x7f41380cdca0 2023-10-08T05:56:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:56.243 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":150,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:46.566262+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:56:30.775239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-08T05:56:31.378327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1640710135":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/1739083188":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/202771123":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/3030439524":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/4192220981":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/3672243748":"2023-10-09T05:56:46.566167+0000","172.21.15.181:6833/3857845516":"2023-10-09T05:56:46.566167+0000","172.21.15.181:6832/3857845516":"2023-10-09T05:56:46.566167+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:56.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 -- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f411806b020 msgr2=0x7f411806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:56.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f411806b020 0x7f411806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 -- 172.21.15.70:0/2373104146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380cdca0 msgr2=0x7f413815c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380cdca0 0x7f413815c480 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f412800d5f0 tx=0x7f412800dab0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 -- 172.21.15.70:0/2373104146 shutdown_connections 2023-10-08T05:56:56.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f411806b020 0x7f411806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f413815ab70 0x7f413815dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41380d04e0 0x7f4138158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 --2- 172.21.15.70:0/2373104146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41380cdca0 0x7f413815c480 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 -- 172.21.15.70:0/2373104146 >> 172.21.15.70:0/2373104146 conn(0x7f4138069f30 msgr2=0x7f41380b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:56.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 -- 172.21.15.70:0/2373104146 shutdown_connections 2023-10-08T05:56:56.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.244+0000 7f413d705640 1 -- 172.21.15.70:0/2373104146 wait complete. 2023-10-08T05:56:56.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:56:30.775239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:56:31.378327+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:56:56.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:56:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.470+0000 7fa59359e640 1 Processor -- start 2023-10-08T05:56:56.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.470+0000 7fa59359e640 1 -- start start 2023-10-08T05:56:56.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa59359e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa59415aab0 0x7fa594070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa59359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5940caef0 0x7fa5940aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa59359e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59415ae90 con 0x7fa5940b04a0 2023-10-08T05:56:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa59359e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940b0930 con 0x7fa5940caef0 2023-10-08T05:56:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa59359e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5940cbdb0 con 0x7fa59415aab0 2023-10-08T05:56:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa58bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5940caef0 0x7fa5940aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa58bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5940caef0 0x7fa5940aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33954/0 (socket says 172.21.15.70:33954) 2023-10-08T05:56:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa58bfff640 1 -- 172.21.15.70:0/4098668856 learned_addr learned my addr 172.21.15.70:0/4098668856 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa59259c640 1 --2- 172.21.15.70:0/4098668856 >> v2:172.21.15.181:3300/0 conn(0x7fa59415aab0 0x7fa594070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa58bfff640 1 -- 172.21.15.70:0/4098668856 >> v2:172.21.15.181:3300/0 conn(0x7fa59415aab0 msgr2=0x7fa594070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa58bfff640 1 --2- 172.21.15.70:0/4098668856 >> v2:172.21.15.181:3300/0 conn(0x7fa59415aab0 0x7fa594070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa58bfff640 1 -- 172.21.15.70:0/4098668856 >> v1:172.21.15.181:6789/0 conn(0x7fa5940b04a0 legacy=0x7fa5940c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.472+0000 7fa58bfff640 1 -- 172.21.15.70:0/4098668856 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5940cb7f0 con 0x7fa5940caef0 2023-10-08T05:56:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.473+0000 7fa58bfff640 1 --2- 172.21.15.70:0/4098668856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5940caef0 0x7fa5940aff60 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009870 tx=0x7fa57c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ecf6901aac40df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.474+0000 7fa591d9b640 1 -- 172.21.15.70:0/4098668856 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c0317d0 con 0x7fa5940caef0 2023-10-08T05:56:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.474+0000 7fa591d9b640 1 -- 172.21.15.70:0/4098668856 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c031930 con 0x7fa5940caef0 2023-10-08T05:56:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.474+0000 7fa591d9b640 1 -- 172.21.15.70:0/4098668856 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c031cc0 con 0x7fa5940caef0 2023-10-08T05:56:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.474+0000 7fa59359e640 1 -- 172.21.15.70:0/4098668856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5940caef0 msgr2=0x7fa5940aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.474+0000 7fa59359e640 1 --2- 172.21.15.70:0/4098668856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5940caef0 0x7fa5940aff60 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009870 tx=0x7fa57c02b870 comp rx=0 tx=0).stop 2023-10-08T05:56:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.475+0000 7fa59359e640 1 -- 172.21.15.70:0/4098668856 shutdown_connections 2023-10-08T05:56:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.475+0000 7fa59359e640 1 --2- 172.21.15.70:0/4098668856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5940caef0 0x7fa5940aff60 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.475+0000 7fa59359e640 1 --2- 172.21.15.70:0/4098668856 >> v2:172.21.15.181:3300/0 conn(0x7fa59415aab0 0x7fa594070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.475+0000 7fa59359e640 1 -- 172.21.15.70:0/4098668856 >> 172.21.15.70:0/4098668856 conn(0x7fa59406c010 msgr2=0x7fa59406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.475+0000 7fa59359e640 1 -- 172.21.15.70:0/4098668856 shutdown_connections 2023-10-08T05:56:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.475+0000 7fa59359e640 1 -- 172.21.15.70:0/4098668856 wait complete. 2023-10-08T05:56:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.477+0000 7fa59359e640 1 Processor -- start 2023-10-08T05:56:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.477+0000 7fa59359e640 1 -- start start 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa59359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5940b04a0 0x7fa5941519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa59359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5940caef0 0x7fa5940c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa59359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59415aab0 0x7fa5940c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa59359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5940c5750 con 0x7fa59415aab0 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa59359e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59415c550 con 0x7fa5940b04a0 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa59359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa594151ef0 con 0x7fa5940caef0 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa592d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59415aab0 0x7fa5940c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa592d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59415aab0 0x7fa5940c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33966/0 (socket says 172.21.15.70:33966) 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa592d9d640 1 -- 172.21.15.70:0/995972150 learned_addr learned my addr 172.21.15.70:0/995972150 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa58bfff640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5940caef0 0x7fa5940c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa59259c640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5940b04a0 0x7fa5941519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa592d9d640 1 -- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5940caef0 msgr2=0x7fa5940c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa592d9d640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5940caef0 0x7fa5940c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa592d9d640 1 -- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5940b04a0 msgr2=0x7fa5941519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa592d9d640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5940b04a0 0x7fa5941519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa592d9d640 1 -- 172.21.15.70:0/995972150 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c0094d0 con 0x7fa59415aab0 2023-10-08T05:56:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa59259c640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5940b04a0 0x7fa5941519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.478+0000 7fa58bfff640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5940caef0 0x7fa5940c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.479+0000 7fa592d9d640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59415aab0 0x7fa5940c7f00 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00c780 tx=0x7fa58c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.479+0000 7fa58affd640 1 -- 172.21.15.70:0/995972150 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c003d80 con 0x7fa59415aab0 2023-10-08T05:56:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.479+0000 7fa58affd640 1 -- 172.21.15.70:0/995972150 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c00ce90 con 0x7fa59415aab0 2023-10-08T05:56:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.479+0000 7fa58affd640 1 -- 172.21.15.70:0/995972150 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0052b0 con 0x7fa59415aab0 2023-10-08T05:56:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.479+0000 7fa59359e640 1 -- 172.21.15.70:0/995972150 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59415bcb0 con 0x7fa59415aab0 2023-10-08T05:56:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.479+0000 7fa59359e640 1 -- 172.21.15.70:0/995972150 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59415c270 con 0x7fa59415aab0 2023-10-08T05:56:56.484 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:56.477+0000 7f0968e56040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:56:56.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.480+0000 7fa59359e640 1 -- 172.21.15.70:0/995972150 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5940c5890 con 0x7fa59415aab0 2023-10-08T05:56:56.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.481+0000 7fa58affd640 1 -- 172.21.15.70:0/995972150 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 73) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fa58c005410 con 0x7fa59415aab0 2023-10-08T05:56:56.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.482+0000 7fa58affd640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa57406b020 0x7fa57406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.482+0000 7fa58affd640 1 -- 172.21.15.70:0/995972150 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fa58c0890f0 con 0x7fa59415aab0 2023-10-08T05:56:56.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.482+0000 7fa59259c640 1 -- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa57406b020 msgr2=0x7fa57406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:56.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.482+0000 7fa59259c640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa57406b020 0x7fa57406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:56.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.490+0000 7fa58affd640 1 -- 172.21.15.70:0/995972150 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5940c5890 con 0x7fa59415aab0 2023-10-08T05:56:56.519 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:56.518+0000 7fdcbf2b6040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:56:56.531 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:56.530+0000 7f0968e56040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:56:56.573 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:56.572+0000 7fdcbf2b6040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:56:56.580 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:56.579+0000 7f0968e56040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:56:56.626 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:56.624+0000 7fdcbf2b6040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:56:56.642 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:56.641+0000 7f0968e56040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:56:56.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.682+0000 7fa59359e640 1 -- 172.21.15.70:0/995972150 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5940c5890 con 0x7fa59415aab0 2023-10-08T05:56:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.683+0000 7fa59259c640 1 -- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa57406b020 msgr2=0x7fa57406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.683+0000 7fa58affd640 1 -- 172.21.15.70:0/995972150 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5940c5890 con 0x7fa59415aab0 2023-10-08T05:56:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.683+0000 7fa59259c640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa57406b020 0x7fa57406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:56:56.685 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:56:56.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.686+0000 7fa59359e640 1 -- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa57406b020 msgr2=0x7fa57406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:56.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.686+0000 7fa59359e640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa57406b020 0x7fa57406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.686+0000 7fa59359e640 1 -- 172.21.15.70:0/995972150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59415aab0 msgr2=0x7fa5940c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.686+0000 7fa59359e640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59415aab0 0x7fa5940c7f00 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00c780 tx=0x7fa58c00cc40 comp rx=0 tx=0).stop 2023-10-08T05:56:56.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.687+0000 7fa59359e640 1 -- 172.21.15.70:0/995972150 shutdown_connections 2023-10-08T05:56:56.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.687+0000 7fa59359e640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fa57406b020 0x7fa57406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.687+0000 7fa59359e640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa59415aab0 0x7fa5940c7f00 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.687+0000 7fa59359e640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5940caef0 0x7fa5940c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.687+0000 7fa59359e640 1 --2- 172.21.15.70:0/995972150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5940b04a0 0x7fa5941519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.687+0000 7fa59359e640 1 -- 172.21.15.70:0/995972150 >> 172.21.15.70:0/995972150 conn(0x7fa59406c010 msgr2=0x7fa594070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.687+0000 7fa59359e640 1 -- 172.21.15.70:0/995972150 shutdown_connections 2023-10-08T05:56:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.687+0000 7fa59359e640 1 -- 172.21.15.70:0/995972150 wait complete. 2023-10-08T05:56:56.691 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:56.689+0000 7fdcbf2b6040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:56:56.691 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:56.690+0000 7f0968e56040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:56:56.707 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:56:56.743 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:56.742+0000 7fdcbf2b6040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:56:56.801 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:56.800+0000 7f0968e56040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:56:56.860 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:56.859+0000 7fdcbf2b6040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:56:56.862 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:56.861+0000 7f0968e56040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:56.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.903+0000 7fe426755640 1 Processor -- start 2023-10-08T05:56:56.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.903+0000 7fe426755640 1 -- start start 2023-10-08T05:56:56.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.904+0000 7fe426755640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe42015cd70 0x7fe4201612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.904+0000 7fe426755640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4200b4e10 0x7fe4200d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.904+0000 7fe426755640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4200b8c30 con 0x7fe4200cd4e0 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.904+0000 7fe426755640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4200cdd80 con 0x7fe42015cd70 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.904+0000 7fe426755640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4200ce060 con 0x7fe4200b4e10 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.905+0000 7fe41ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe42015cd70 0x7fe4201612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.905+0000 7fe41ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe42015cd70 0x7fe4201612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33974/0 (socket says 172.21.15.70:33974) 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.905+0000 7fe41ffff640 1 -- 172.21.15.70:0/817567311 learned_addr learned my addr 172.21.15.70:0/817567311 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.905+0000 7fe424cf2640 1 --2- 172.21.15.70:0/817567311 >> v2:172.21.15.181:3300/0 conn(0x7fe4200b4e10 0x7fe4200d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.905+0000 7fe41ffff640 1 -- 172.21.15.70:0/817567311 >> v2:172.21.15.181:3300/0 conn(0x7fe4200b4e10 msgr2=0x7fe4200d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.905+0000 7fe41ffff640 1 --2- 172.21.15.70:0/817567311 >> v2:172.21.15.181:3300/0 conn(0x7fe4200b4e10 0x7fe4200d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.905+0000 7fe41ffff640 1 -- 172.21.15.70:0/817567311 >> v1:172.21.15.181:6789/0 conn(0x7fe4200cd4e0 legacy=0x7fe4200b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.905+0000 7fe41ffff640 1 -- 172.21.15.70:0/817567311 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4200ce340 con 0x7fe42015cd70 2023-10-08T05:56:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.906+0000 7fe41ffff640 1 --2- 172.21.15.70:0/817567311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42015cd70 0x7fe4201612f0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe4200d07d0 tx=0x7fe41002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d51362c6833c445 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.907+0000 7fe41f7fe640 1 -- 172.21.15.70:0/817567311 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410031870 con 0x7fe42015cd70 2023-10-08T05:56:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.907+0000 7fe41f7fe640 1 -- 172.21.15.70:0/817567311 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4100319d0 con 0x7fe42015cd70 2023-10-08T05:56:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.907+0000 7fe41f7fe640 1 -- 172.21.15.70:0/817567311 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410031d60 con 0x7fe42015cd70 2023-10-08T05:56:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.907+0000 7fe426755640 1 -- 172.21.15.70:0/817567311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42015cd70 msgr2=0x7fe4201612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.907+0000 7fe426755640 1 --2- 172.21.15.70:0/817567311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42015cd70 0x7fe4201612f0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe4200d07d0 tx=0x7fe41002b910 comp rx=0 tx=0).stop 2023-10-08T05:56:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.907+0000 7fe426755640 1 -- 172.21.15.70:0/817567311 shutdown_connections 2023-10-08T05:56:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.907+0000 7fe426755640 1 --2- 172.21.15.70:0/817567311 >> v2:172.21.15.181:3300/0 conn(0x7fe4200b4e10 0x7fe4200d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.907+0000 7fe426755640 1 --2- 172.21.15.70:0/817567311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe42015cd70 0x7fe4201612f0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.907+0000 7fe426755640 1 -- 172.21.15.70:0/817567311 >> 172.21.15.70:0/817567311 conn(0x7fe4200cf700 msgr2=0x7fe4200cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.908+0000 7fe426755640 1 -- 172.21.15.70:0/817567311 shutdown_connections 2023-10-08T05:56:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.908+0000 7fe426755640 1 -- 172.21.15.70:0/817567311 wait complete. 2023-10-08T05:56:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.910+0000 7fe426755640 1 Processor -- start 2023-10-08T05:56:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.910+0000 7fe426755640 1 -- start start 2023-10-08T05:56:56.912 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:56.911+0000 7f0968e56040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.912+0000 7fe426755640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4200b4e10 0x7fe4200c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.912+0000 7fe426755640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4200cd4e0 0x7fe420151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.912+0000 7fe426755640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe42015cd70 0x7fe4200c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.912+0000 7fe426755640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200c95d0 con 0x7fe4200cd4e0 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.912+0000 7fe426755640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42015de40 con 0x7fe4200b4e10 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.912+0000 7fe426755640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42015e120 con 0x7fe42015cd70 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.912+0000 7fe417fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4200cd4e0 0x7fe420151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.912+0000 7fe417fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4200cd4e0 0x7fe420151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33988/0 (socket says 172.21.15.70:33988) 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.912+0000 7fe417fff640 1 -- 172.21.15.70:0/1430772420 learned_addr learned my addr 172.21.15.70:0/1430772420 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.912+0000 7fe41ffff640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4200b4e10 0x7fe4200c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe424cf2640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe42015cd70 0x7fe4200c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe417fff640 1 -- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe42015cd70 msgr2=0x7fe4200c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe417fff640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe42015cd70 0x7fe4200c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe417fff640 1 -- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4200b4e10 msgr2=0x7fe4200c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe417fff640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4200b4e10 0x7fe4200c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe417fff640 1 -- 172.21.15.70:0/1430772420 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4100094d0 con 0x7fe4200cd4e0 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe424cf2640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe42015cd70 0x7fe4200c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe417fff640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4200cd4e0 0x7fe420151100 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe40800c410 tx=0x7fe40800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe41d7fa640 1 -- 172.21.15.70:0/1430772420 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080035f0 con 0x7fe4200cd4e0 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe41d7fa640 1 -- 172.21.15.70:0/1430772420 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe408003750 con 0x7fe4200cd4e0 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe41d7fa640 1 -- 172.21.15.70:0/1430772420 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408003b40 con 0x7fe4200cd4e0 2023-10-08T05:56:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe426755640 1 -- 172.21.15.70:0/1430772420 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42015e400 con 0x7fe4200cd4e0 2023-10-08T05:56:56.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.913+0000 7fe426755640 1 -- 172.21.15.70:0/1430772420 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4200c8ce0 con 0x7fe4200cd4e0 2023-10-08T05:56:56.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.915+0000 7fe426755640 1 -- 172.21.15.70:0/1430772420 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4200c8f00 con 0x7fe4200cd4e0 2023-10-08T05:56:56.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.915+0000 7fe41d7fa640 1 -- 172.21.15.70:0/1430772420 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 73) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fe408003ca0 con 0x7fe4200cd4e0 2023-10-08T05:56:56.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.917+0000 7fe41d7fa640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe40006b020 0x7fe40006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:56.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.917+0000 7fe41d7fa640 1 -- 172.21.15.70:0/1430772420 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fe408088ec0 con 0x7fe4200cd4e0 2023-10-08T05:56:56.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.917+0000 7fe41ffff640 1 -- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe40006b020 msgr2=0x7fe40006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:56.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.917+0000 7fe41ffff640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe40006b020 0x7fe40006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:56.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:56.922+0000 7fe41d7fa640 1 -- 172.21.15.70:0/1430772420 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe408053970 con 0x7fe4200cd4e0 2023-10-08T05:56:56.929 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:56.927+0000 7fdcbf2b6040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:56.970 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:56.969+0000 7f0968e56040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:56:56.980 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:56.979+0000 7fdcbf2b6040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:56:57.020 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:57.019+0000 7f0968e56040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:56:57.039 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:57.038+0000 7fdcbf2b6040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:56:57.091 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:57.090+0000 7fdcbf2b6040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:56:57.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.111+0000 7fe426755640 1 -- 172.21.15.70:0/1430772420 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe42004e160 con 0x7fe4200cd4e0 2023-10-08T05:56:57.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.112+0000 7fe41d7fa640 1 -- 172.21.15.70:0/1430772420 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe408056f90 con 0x7fe4200cd4e0 2023-10-08T05:56:57.113 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:56:57.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 -- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe40006b020 msgr2=0x7fe40006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:57.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe40006b020 0x7fe40006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 -- 172.21.15.70:0/1430772420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4200cd4e0 msgr2=0x7fe420151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:57.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4200cd4e0 0x7fe420151100 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe40800c410 tx=0x7fe40800c8d0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 -- 172.21.15.70:0/1430772420 shutdown_connections 2023-10-08T05:56:57.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe40006b020 0x7fe40006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe42015cd70 0x7fe4200c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4200cd4e0 0x7fe420151100 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 --2- 172.21.15.70:0/1430772420 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4200b4e10 0x7fe4200c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 -- 172.21.15.70:0/1430772420 >> 172.21.15.70:0/1430772420 conn(0x7fe4200cf700 msgr2=0x7fe4200d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.114+0000 7fe426755640 1 -- 172.21.15.70:0/1430772420 shutdown_connections 2023-10-08T05:56:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.115+0000 7fe426755640 1 -- 172.21.15.70:0/1430772420 wait complete. 2023-10-08T05:56:57.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:56:57.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.386+0000 7f5fd9290640 1 Processor -- start 2023-10-08T05:56:57.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.387+0000 7f5fd9290640 1 -- start start 2023-10-08T05:56:57.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.388+0000 7f5fd9290640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5fd4069fc0 0x7f5fd4071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.388+0000 7f5fd9290640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5fd406ec00 0x7f5fd40c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.388+0000 7f5fd9290640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd4160d10 con 0x7f5fd40ca210 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.388+0000 7f5fd9290640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd4072070 con 0x7f5fd4069fc0 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.388+0000 7f5fd9290640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd4071580 con 0x7f5fd406ec00 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.389+0000 7f5fd3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5fd4069fc0 0x7f5fd4071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.389+0000 7f5fd3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5fd4069fc0 0x7f5fd4071860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33994/0 (socket says 172.21.15.70:33994) 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.389+0000 7f5fd3fff640 1 -- 172.21.15.70:0/3418766765 learned_addr learned my addr 172.21.15.70:0/3418766765 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.389+0000 7f5fd37fe640 1 --2- 172.21.15.70:0/3418766765 >> v2:172.21.15.181:3300/0 conn(0x7f5fd406ec00 0x7f5fd40c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.389+0000 7f5fd3fff640 1 -- 172.21.15.70:0/3418766765 >> v2:172.21.15.181:3300/0 conn(0x7f5fd406ec00 msgr2=0x7f5fd40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.389+0000 7f5fd3fff640 1 --2- 172.21.15.70:0/3418766765 >> v2:172.21.15.181:3300/0 conn(0x7f5fd406ec00 0x7f5fd40c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.389+0000 7f5fd3fff640 1 -- 172.21.15.70:0/3418766765 >> v1:172.21.15.181:6789/0 conn(0x7f5fd40ca210 legacy=0x7f5fd4160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:57.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.389+0000 7f5fd3fff640 1 -- 172.21.15.70:0/3418766765 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd4072330 con 0x7f5fd4069fc0 2023-10-08T05:56:57.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.390+0000 7f5fd3fff640 1 --2- 172.21.15.70:0/3418766765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 0x7f5fd4071860 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40cd250 tx=0x7f5fc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b671e3048419b522 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:57.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.391+0000 7f5fd2ffd640 1 -- 172.21.15.70:0/3418766765 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4031870 con 0x7f5fd4069fc0 2023-10-08T05:56:57.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.391+0000 7f5fd2ffd640 1 -- 172.21.15.70:0/3418766765 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc40319d0 con 0x7f5fd4069fc0 2023-10-08T05:56:57.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.391+0000 7f5fd2ffd640 1 -- 172.21.15.70:0/3418766765 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4031d60 con 0x7f5fd4069fc0 2023-10-08T05:56:57.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.391+0000 7f5fd9290640 1 -- 172.21.15.70:0/3418766765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 msgr2=0x7f5fd4071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:57.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.391+0000 7f5fd9290640 1 --2- 172.21.15.70:0/3418766765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 0x7f5fd4071860 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40cd250 tx=0x7f5fc402b910 comp rx=0 tx=0).stop 2023-10-08T05:56:57.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.391+0000 7f5fd9290640 1 -- 172.21.15.70:0/3418766765 shutdown_connections 2023-10-08T05:56:57.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.391+0000 7f5fd9290640 1 --2- 172.21.15.70:0/3418766765 >> v2:172.21.15.181:3300/0 conn(0x7f5fd406ec00 0x7f5fd40c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.391+0000 7f5fd9290640 1 --2- 172.21.15.70:0/3418766765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 0x7f5fd4071860 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.392+0000 7f5fd9290640 1 -- 172.21.15.70:0/3418766765 >> 172.21.15.70:0/3418766765 conn(0x7f5fd40cf5f0 msgr2=0x7f5fd40cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:57.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.392+0000 7f5fd9290640 1 -- 172.21.15.70:0/3418766765 shutdown_connections 2023-10-08T05:56:57.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.392+0000 7f5fd9290640 1 -- 172.21.15.70:0/3418766765 wait complete. 2023-10-08T05:56:57.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.393+0000 7f5fd9290640 1 Processor -- start 2023-10-08T05:56:57.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.393+0000 7f5fd9290640 1 -- start start 2023-10-08T05:56:57.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.394+0000 7f5fd9290640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 0x7f5fd41f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.394+0000 7f5fd9290640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fd406ec00 0x7f5fd41e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.394+0000 7f5fd9290640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fd40ca210 0x7f5fd40c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.394+0000 7f5fd9290640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41f3760 con 0x7f5fd4069fc0 2023-10-08T05:56:57.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.394+0000 7f5fd9290640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41e5eb0 con 0x7f5fd406ec00 2023-10-08T05:56:57.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.394+0000 7f5fd9290640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41e6150 con 0x7f5fd40ca210 2023-10-08T05:56:57.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 0x7f5fd41f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:57.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 0x7f5fd41f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33998/0 (socket says 172.21.15.70:33998) 2023-10-08T05:56:57.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd3fff640 1 -- 172.21.15.70:0/1429876005 learned_addr learned my addr 172.21.15.70:0/1429876005 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:57.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd8a8f640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fd40ca210 0x7f5fd40c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:57.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd37fe640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fd406ec00 0x7f5fd41e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:57.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd3fff640 1 -- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fd40ca210 msgr2=0x7f5fd40c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:57.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd3fff640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fd40ca210 0x7f5fd40c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd3fff640 1 -- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fd406ec00 msgr2=0x7f5fd41e71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:57.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd3fff640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fd406ec00 0x7f5fd41e71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd3fff640 1 -- 172.21.15.70:0/1429876005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40094d0 con 0x7f5fd4069fc0 2023-10-08T05:56:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.395+0000 7f5fd37fe640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fd406ec00 0x7f5fd41e71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.396+0000 7f5fd3fff640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 0x7f5fd41f3220 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4000c00 tx=0x7f5fc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.396+0000 7f5fd0ff9640 1 -- 172.21.15.70:0/1429876005 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4031840 con 0x7f5fd4069fc0 2023-10-08T05:56:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.396+0000 7f5fd0ff9640 1 -- 172.21.15.70:0/1429876005 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc40443f0 con 0x7f5fd4069fc0 2023-10-08T05:56:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.396+0000 7f5fd9290640 1 -- 172.21.15.70:0/1429876005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd41e6430 con 0x7f5fd4069fc0 2023-10-08T05:56:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.396+0000 7f5fd0ff9640 1 -- 172.21.15.70:0/1429876005 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40447a0 con 0x7f5fd4069fc0 2023-10-08T05:56:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.396+0000 7f5fd9290640 1 -- 172.21.15.70:0/1429876005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd41e69f0 con 0x7f5fd4069fc0 2023-10-08T05:56:57.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.397+0000 7f5fd9290640 1 -- 172.21.15.70:0/1429876005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd4074150 con 0x7f5fd4069fc0 2023-10-08T05:56:57.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.398+0000 7f5fd0ff9640 1 -- 172.21.15.70:0/1429876005 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 73) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f5fc404d450 con 0x7f5fd4069fc0 2023-10-08T05:56:57.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.399+0000 7f5fd0ff9640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f5fc006b050 0x7f5fc006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.399+0000 7f5fd0ff9640 1 -- 172.21.15.70:0/1429876005 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f5fc402d040 con 0x7f5fd4069fc0 2023-10-08T05:56:57.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.399+0000 7f5fd37fe640 1 -- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f5fc006b050 msgr2=0x7f5fc006d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:57.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.399+0000 7f5fd37fe640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f5fc006b050 0x7f5fc006d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:57.401 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:57.400+0000 7f0968e56040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:56:57.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.403+0000 7f5fd0ff9640 1 -- 172.21.15.70:0/1429876005 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fc4076f10 con 0x7f5fd4069fc0 2023-10-08T05:56:57.495 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:57.493+0000 7fdcbf2b6040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:56:57.560 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:57.559+0000 7f0968e56040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:56:57.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.593+0000 7f5fd9290640 1 -- 172.21.15.70:0/1429876005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5fd40b5570 con 0x7f5fd4069fc0 2023-10-08T05:56:57.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.593+0000 7f5fd0ff9640 1 -- 172.21.15.70:0/1429876005 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5fc407a530 con 0x7f5fd4069fc0 2023-10-08T05:56:57.595 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 -- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f5fc006b050 msgr2=0x7f5fc006d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f5fc006b050 0x7f5fc006d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 -- 172.21.15.70:0/1429876005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 msgr2=0x7f5fd41f3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:57.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 0x7f5fd41f3220 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4000c00 tx=0x7f5fc40046c0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 -- 172.21.15.70:0/1429876005 shutdown_connections 2023-10-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f5fc006b050 0x7f5fc006d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5fd40ca210 0x7f5fd40c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5fd406ec00 0x7f5fd41e71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 --2- 172.21.15.70:0/1429876005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fd4069fc0 0x7f5fd41f3220 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 -- 172.21.15.70:0/1429876005 >> 172.21.15.70:0/1429876005 conn(0x7f5fd40cf5f0 msgr2=0x7f5fd40acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 -- 172.21.15.70:0/1429876005 shutdown_connections 2023-10-08T05:56:57.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.596+0000 7f5fd9290640 1 -- 172.21.15.70:0/1429876005 wait complete. 2023-10-08T05:56:57.617 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T05:56:57.662 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:57.661+0000 7fdcbf2b6040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:56:57.669 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:57.668+0000 7f0968e56040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:56:57.719 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:57.718+0000 7f0968e56040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:56:57.775 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:57.773+0000 7fdcbf2b6040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:56:57.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.820+0000 7f8ccccfc640 1 Processor -- start 2023-10-08T05:56:57.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.820+0000 7f8ccccfc640 1 -- start start 2023-10-08T05:56:57.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.822+0000 7f8ccccfc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8cc80adeb0 0x7f8cc80ca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.822+0000 7f8ccccfc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80ae390 0x7f8cc80ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.822+0000 7f8ccccfc640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc806e940 con 0x7f8cc806e560 2023-10-08T05:56:57.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.822+0000 7f8ccccfc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc815d7c0 con 0x7f8cc80adeb0 2023-10-08T05:56:57.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.822+0000 7f8ccccfc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc806b710 con 0x7f8cc80ae390 2023-10-08T05:56:57.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.822+0000 7f8cc5d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8cc80adeb0 0x7f8cc80ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:57.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.823+0000 7f8cc6d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80ae390 0x7f8cc80ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:57.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.823+0000 7f8cc5d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8cc80adeb0 0x7f8cc80ca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:50902/0 (socket says 172.21.15.70:50902) 2023-10-08T05:56:57.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.823+0000 7f8cc5d9b640 1 -- 172.21.15.70:0/2371752988 learned_addr learned my addr 172.21.15.70:0/2371752988 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:57.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.823+0000 7f8cc5d9b640 1 -- 172.21.15.70:0/2371752988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80ae390 msgr2=0x7f8cc80ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:57.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.823+0000 7f8cc5d9b640 1 --2- 172.21.15.70:0/2371752988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80ae390 0x7f8cc80ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.823+0000 7f8cc5d9b640 1 -- 172.21.15.70:0/2371752988 >> v1:172.21.15.181:6789/0 conn(0x7f8cc806e560 legacy=0x7f8cc815dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:57.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.823+0000 7f8cc5d9b640 1 -- 172.21.15.70:0/2371752988 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc806b190 con 0x7f8cc80adeb0 2023-10-08T05:56:57.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.823+0000 7f8cc6d9d640 1 --2- 172.21.15.70:0/2371752988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80ae390 0x7f8cc80ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:56:57.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.825+0000 7f8cc5d9b640 1 --2- 172.21.15.70:0/2371752988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc80adeb0 0x7f8cc80ca400 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009870 tx=0x7f8cb002db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1115974d7caab814 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:57.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.825+0000 7f8cc559a640 1 -- 172.21.15.70:0/2371752988 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb002e940 con 0x7f8cc80adeb0 2023-10-08T05:56:57.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.825+0000 7f8cc559a640 1 -- 172.21.15.70:0/2371752988 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb002eaa0 con 0x7f8cc80adeb0 2023-10-08T05:56:57.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.825+0000 7f8cc559a640 1 -- 172.21.15.70:0/2371752988 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb002ee30 con 0x7f8cc80adeb0 2023-10-08T05:56:57.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.826+0000 7f8ccccfc640 1 -- 172.21.15.70:0/2371752988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc80adeb0 msgr2=0x7f8cc80ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:57.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.826+0000 7f8ccccfc640 1 --2- 172.21.15.70:0/2371752988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc80adeb0 0x7f8cc80ca400 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009870 tx=0x7f8cb002db90 comp rx=0 tx=0).stop 2023-10-08T05:56:57.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.826+0000 7f8ccccfc640 1 -- 172.21.15.70:0/2371752988 shutdown_connections 2023-10-08T05:56:57.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.826+0000 7f8ccccfc640 1 --2- 172.21.15.70:0/2371752988 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80ae390 0x7f8cc80ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.826+0000 7f8ccccfc640 1 --2- 172.21.15.70:0/2371752988 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc80adeb0 0x7f8cc80ca400 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.826+0000 7f8ccccfc640 1 -- 172.21.15.70:0/2371752988 >> 172.21.15.70:0/2371752988 conn(0x7f8cc80b0af0 msgr2=0x7f8cc80b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:57.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.826+0000 7f8ccccfc640 1 -- 172.21.15.70:0/2371752988 shutdown_connections 2023-10-08T05:56:57.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.826+0000 7f8ccccfc640 1 -- 172.21.15.70:0/2371752988 wait complete. 2023-10-08T05:56:57.828 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:57.826+0000 7f0968e56040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:57.829 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:57.828+0000 7fdcbf2b6040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:56:57.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.829+0000 7f8ccccfc640 1 Processor -- start 2023-10-08T05:56:57.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.829+0000 7f8ccccfc640 1 -- start start 2023-10-08T05:56:57.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8ccccfc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc806e560 0x7f8cc80c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8ccccfc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80adeb0 0x7f8cc8158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8ccccfc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc80ae390 0x7f8cc80b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8ccccfc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80c8f00 con 0x7f8cc80ae390 2023-10-08T05:56:57.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8ccccfc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80c58f0 con 0x7f8cc806e560 2023-10-08T05:56:57.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8ccccfc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80c2cd0 con 0x7f8cc80adeb0 2023-10-08T05:56:57.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8cc6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc80ae390 0x7f8cc80b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:57.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8cc6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc80ae390 0x7f8cc80b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34000/0 (socket says 172.21.15.70:34000) 2023-10-08T05:56:57.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8cc6d9d640 1 -- 172.21.15.70:0/710157687 learned_addr learned my addr 172.21.15.70:0/710157687 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:57.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8cc659c640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc806e560 0x7f8cc80c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:57.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8cc5d9b640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80adeb0 0x7f8cc8158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:57.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8cc6d9d640 1 -- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80adeb0 msgr2=0x7f8cc8158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:57.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8cc6d9d640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80adeb0 0x7f8cc8158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8cc6d9d640 1 -- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc806e560 msgr2=0x7f8cc80c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:57.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8cc6d9d640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc806e560 0x7f8cc80c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:57.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.831+0000 7f8cc6d9d640 1 -- 172.21.15.70:0/710157687 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0009a70 con 0x7f8cc80ae390 2023-10-08T05:56:57.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.832+0000 7f8cc6d9d640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc80ae390 0x7f8cc80b1c00 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000efc0 tx=0x7f8cc000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:57.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.832+0000 7f8cbeffd640 1 -- 172.21.15.70:0/710157687 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0009cd0 con 0x7f8cc80ae390 2023-10-08T05:56:57.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.832+0000 7f8cbeffd640 1 -- 172.21.15.70:0/710157687 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc000f040 con 0x7f8cc80ae390 2023-10-08T05:56:57.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.832+0000 7f8ccccfc640 1 -- 172.21.15.70:0/710157687 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb00094d0 con 0x7f8cc80ae390 2023-10-08T05:56:57.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.832+0000 7f8cbeffd640 1 -- 172.21.15.70:0/710157687 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0003990 con 0x7f8cc80ae390 2023-10-08T05:56:57.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.832+0000 7f8ccccfc640 1 -- 172.21.15.70:0/710157687 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc80c8940 con 0x7f8cc80ae390 2023-10-08T05:56:57.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.833+0000 7f8ccccfc640 1 -- 172.21.15.70:0/710157687 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc80c2e40 con 0x7f8cc80ae390 2023-10-08T05:56:57.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.834+0000 7f8cbeffd640 1 -- 172.21.15.70:0/710157687 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 73) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f8cc0014480 con 0x7f8cc80ae390 2023-10-08T05:56:57.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.836+0000 7f8cbeffd640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f8ca806b050 0x7f8ca806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:57.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.836+0000 7f8cbeffd640 1 -- 172.21.15.70:0/710157687 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f8cc008a960 con 0x7f8cc80ae390 2023-10-08T05:56:57.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.836+0000 7f8cc659c640 1 -- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f8ca806b050 msgr2=0x7f8ca806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:57.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.836+0000 7f8cc659c640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f8ca806b050 0x7f8ca806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:57.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:57.840+0000 7f8cbeffd640 1 -- 172.21.15.70:0/710157687 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8cc0055300 con 0x7f8cc80ae390 2023-10-08T05:56:57.934 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:57.933+0000 7f0968e56040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:56:57.945 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:57.944+0000 7fdcbf2b6040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:58.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.037+0000 7f8cc659c640 1 -- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f8ca806b050 msgr2=0x7f8ca806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:58.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.037+0000 7f8cc659c640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f8ca806b050 0x7f8ca806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:56:58.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.049+0000 7f8ccccfc640 1 -- 172.21.15.70:0/710157687 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8cc804e160 con 0x7f8cc80ae390 2023-10-08T05:56:58.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.050+0000 7f8cbeffd640 1 -- 172.21.15.70:0/710157687 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v150) v1 ==== 80+0+477 (secure 0 0 0) 0x7f8cc0058920 con 0x7f8cc80ae390 2023-10-08T05:56:58.051 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1640710135 2023-10-09T05:56:46.566167+0000 2023-10-08T05:56:58.051 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1739083188 2023-10-09T05:56:46.566167+0000 2023-10-08T05:56:58.051 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/202771123 2023-10-09T05:56:46.566167+0000 2023-10-08T05:56:58.052 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3030439524 2023-10-09T05:56:46.566167+0000 2023-10-08T05:56:58.052 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/4192220981 2023-10-09T05:56:46.566167+0000 2023-10-08T05:56:58.052 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3672243748 2023-10-09T05:56:46.566167+0000 2023-10-08T05:56:58.052 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/3857845516 2023-10-09T05:56:46.566167+0000 2023-10-08T05:56:58.052 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/3857845516 2023-10-09T05:56:46.566167+0000 2023-10-08T05:56:58.052 INFO:teuthology.orchestra.run.smithi070.stderr:listed 8 entries 2023-10-08T05:56:58.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.052+0000 7f8ccccfc640 1 -- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f8ca806b050 msgr2=0x7f8ca806d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:58.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.052+0000 7f8ccccfc640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f8ca806b050 0x7f8ca806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.052+0000 7f8ccccfc640 1 -- 172.21.15.70:0/710157687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc80ae390 msgr2=0x7f8cc80b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:58.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.052+0000 7f8ccccfc640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc80ae390 0x7f8cc80b1c00 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000efc0 tx=0x7f8cc000c490 comp rx=0 tx=0).stop 2023-10-08T05:56:58.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.052+0000 7f8ccccfc640 1 -- 172.21.15.70:0/710157687 shutdown_connections 2023-10-08T05:56:58.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.052+0000 7f8ccccfc640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7f8ca806b050 0x7f8ca806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.052+0000 7f8ccccfc640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc80ae390 0x7f8cc80b1c00 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.052+0000 7f8ccccfc640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cc80adeb0 0x7f8cc8158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.052+0000 7f8ccccfc640 1 --2- 172.21.15.70:0/710157687 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cc806e560 0x7f8cc80c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.052+0000 7f8ccccfc640 1 -- 172.21.15.70:0/710157687 >> 172.21.15.70:0/710157687 conn(0x7f8cc80b0af0 msgr2=0x7f8cc80ac430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:58.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.053+0000 7f8ccccfc640 1 -- 172.21.15.70:0/710157687 shutdown_connections 2023-10-08T05:56:58.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.053+0000 7f8ccccfc640 1 -- 172.21.15.70:0/710157687 wait complete. 2023-10-08T05:56:58.062 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:58.060+0000 7fdcbf2b6040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:56:58.072 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T05:56:58.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:58.072 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T05:56:58.096 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:58.094+0000 7f0968e56040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:58.145 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:58.144+0000 7f0968e56040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:58.228 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T05:56:58.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:56:58.228 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T05:56:58.229 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:58.228+0000 7fdcbf2b6040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:56:58.265 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:58.264+0000 7f0968e56040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:56:58.276 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:56:58.276 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:58.281 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:58.280+0000 7fdcbf2b6040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:56:58.332 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:56:58.332 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:56:58.332 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:58.384 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:56:58.385 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:56:58.385 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:56:58.406 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:58.405+0000 7fdcbf2b6040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:56:58.440 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:56:58.440 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:56:58.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.647+0000 7fc18874e640 1 Processor -- start 2023-10-08T05:56:58.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.648+0000 7fc18874e640 1 -- start start 2023-10-08T05:56:58.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.649+0000 7fc18874e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc18015cc20 0x7fc18006ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:58.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.649+0000 7fc18874e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18006b3a0 con 0x7fc1800cd770 2023-10-08T05:56:58.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.649+0000 7fc18874e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc180161750 con 0x7fc18015d000 2023-10-08T05:56:58.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.649+0000 7fc18874e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1800cb5c0 con 0x7fc18015cc20 2023-10-08T05:56:58.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.649+0000 7fc1864ea640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc18015cc20 0x7fc18006ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:58.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.650+0000 7fc1864ea640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc18015cc20 0x7fc18006ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34014/0 (socket says 172.21.15.70:34014) 2023-10-08T05:56:58.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.650+0000 7fc1864ea640 1 -- 172.21.15.70:0/1546808240 learned_addr learned my addr 172.21.15.70:0/1546808240 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:58.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.650+0000 7fc1864ea640 1 -- 172.21.15.70:0/1546808240 >> v1:172.21.15.181:6789/0 conn(0x7fc18015d000 legacy=0x7fc1800cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:58.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.650+0000 7fc1864ea640 1 -- 172.21.15.70:0/1546808240 >> v1:172.21.15.70:6789/0 conn(0x7fc1800cd770 legacy=0x7fc1800cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:58.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.650+0000 7fc1864ea640 1 -- 172.21.15.70:0/1546808240 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1800cb8a0 con 0x7fc18015cc20 2023-10-08T05:56:58.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.651+0000 7fc1864ea640 1 --2- 172.21.15.70:0/1546808240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc18015cc20 0x7fc18006ae60 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009870 tx=0x7fc17c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cbdcb6e0e1e46e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:58.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.651+0000 7fc1854e8640 1 -- 172.21.15.70:0/1546808240 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c030860 con 0x7fc18015cc20 2023-10-08T05:56:58.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.651+0000 7fc1854e8640 1 -- 172.21.15.70:0/1546808240 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c0309c0 con 0x7fc18015cc20 2023-10-08T05:56:58.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.651+0000 7fc1854e8640 1 -- 172.21.15.70:0/1546808240 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c030d50 con 0x7fc18015cc20 2023-10-08T05:56:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.652+0000 7fc18874e640 1 -- 172.21.15.70:0/1546808240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc18015cc20 msgr2=0x7fc18006ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.652+0000 7fc18874e640 1 --2- 172.21.15.70:0/1546808240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc18015cc20 0x7fc18006ae60 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009870 tx=0x7fc17c02b9a0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.652+0000 7fc18874e640 1 -- 172.21.15.70:0/1546808240 shutdown_connections 2023-10-08T05:56:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.652+0000 7fc18874e640 1 --2- 172.21.15.70:0/1546808240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc18015cc20 0x7fc18006ae60 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.652+0000 7fc18874e640 1 -- 172.21.15.70:0/1546808240 >> 172.21.15.70:0/1546808240 conn(0x7fc180069f30 msgr2=0x7fc18006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.652+0000 7fc18874e640 1 -- 172.21.15.70:0/1546808240 shutdown_connections 2023-10-08T05:56:58.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.652+0000 7fc18874e640 1 -- 172.21.15.70:0/1546808240 wait complete. 2023-10-08T05:56:58.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.654+0000 7fc18874e640 1 Processor -- start 2023-10-08T05:56:58.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.654+0000 7fc18874e640 1 -- start start 2023-10-08T05:56:58.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.655+0000 7fc18874e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1800cd770 0x7fc1801f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:58.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.655+0000 7fc18874e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc18015cc20 0x7fc1800c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:58.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.655+0000 7fc18874e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1800c8ec0 0x7fc1800c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.655+0000 7fc18874e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1801f25f0 con 0x7fc1800c8ec0 2023-10-08T05:56:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.655+0000 7fc18874e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800ca940 con 0x7fc18015cc20 2023-10-08T05:56:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.655+0000 7fc18874e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1800b3280 con 0x7fc1800cd770 2023-10-08T05:56:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.655+0000 7fc186ceb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1800c8ec0 0x7fc1800c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.656+0000 7fc186ceb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1800c8ec0 0x7fc1800c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34024/0 (socket says 172.21.15.70:34024) 2023-10-08T05:56:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.656+0000 7fc186ceb640 1 -- 172.21.15.70:0/54761597 learned_addr learned my addr 172.21.15.70:0/54761597 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.656+0000 7fc185ce9640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc18015cc20 0x7fc1800c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.656+0000 7fc1864ea640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1800cd770 0x7fc1801f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.656+0000 7fc186ceb640 1 -- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1800cd770 msgr2=0x7fc1801f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.656+0000 7fc186ceb640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1800cd770 0x7fc1801f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.656+0000 7fc186ceb640 1 -- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc18015cc20 msgr2=0x7fc1800c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:58.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.656+0000 7fc186ceb640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc18015cc20 0x7fc1800c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.656+0000 7fc186ceb640 1 -- 172.21.15.70:0/54761597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c0094d0 con 0x7fc1800c8ec0 2023-10-08T05:56:58.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.656+0000 7fc186ceb640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1800c8ec0 0x7fc1800c6390 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c7d0 tx=0x7fc17800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:58.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.657+0000 7fc176ffd640 1 -- 172.21.15.70:0/54761597 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178003bc0 con 0x7fc1800c8ec0 2023-10-08T05:56:58.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.657+0000 7fc176ffd640 1 -- 172.21.15.70:0/54761597 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc178003d40 con 0x7fc1800c8ec0 2023-10-08T05:56:58.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.657+0000 7fc176ffd640 1 -- 172.21.15.70:0/54761597 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17800aa30 con 0x7fc1800c8ec0 2023-10-08T05:56:58.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.657+0000 7fc18874e640 1 -- 172.21.15.70:0/54761597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1800ca0a0 con 0x7fc1800c8ec0 2023-10-08T05:56:58.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.657+0000 7fc18874e640 1 -- 172.21.15.70:0/54761597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1800ca660 con 0x7fc1800c8ec0 2023-10-08T05:56:58.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.658+0000 7fc18874e640 1 -- 172.21.15.70:0/54761597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc180066d80 con 0x7fc1800c8ec0 2023-10-08T05:56:58.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.659+0000 7fc176ffd640 1 -- 172.21.15.70:0/54761597 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 73) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fc17800ab90 con 0x7fc1800c8ec0 2023-10-08T05:56:58.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.660+0000 7fc176ffd640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc16406b020 0x7fc16406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:58.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.660+0000 7fc176ffd640 1 -- 172.21.15.70:0/54761597 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fc178089b30 con 0x7fc1800c8ec0 2023-10-08T05:56:58.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.660+0000 7fc1864ea640 1 -- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc16406b020 msgr2=0x7fc16406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:58.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.660+0000 7fc1864ea640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc16406b020 0x7fc16406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:58.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.664+0000 7fc176ffd640 1 -- 172.21.15.70:0/54761597 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1780545e0 con 0x7fc1800c8ec0 2023-10-08T05:56:58.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.861+0000 7fc1864ea640 1 -- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc16406b020 msgr2=0x7fc16406d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:58.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.861+0000 7fc1864ea640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc16406b020 0x7fc16406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:56:58.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.892+0000 7fc18874e640 1 -- 172.21.15.70:0/54761597 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1800b33c0 con 0x7fc1800c8ec0 2023-10-08T05:56:58.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.892+0000 7fc176ffd640 1 -- 172.21.15.70:0/54761597 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fc178057c00 con 0x7fc1800c8ec0 2023-10-08T05:56:58.894 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:58.894 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1820637663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1820637663},{"type":"v1","addr":"172.21.15.181:6835","nonce":1820637663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1666874980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1666874980},{"type":"v1","addr":"172.21.15.181:6837","nonce":1666874980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2918183907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2918183907},{"type":"v1","addr":"172.21.15.70:6834","nonce":2918183907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:56:32.378187+0000","modified":"2023-10-08T05:56:35.419118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6835/3869116018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3869116018},{"type":"v1","addr":"172.21.15.70:6835","nonce":3869116018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-08T05:56:58.894 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 68 2023-10-08T05:56:58.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 -- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc16406b020 msgr2=0x7fc16406d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:58.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc16406b020 0x7fc16406d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 -- 172.21.15.70:0/54761597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1800c8ec0 msgr2=0x7fc1800c6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:58.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1800c8ec0 0x7fc1800c6390 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c7d0 tx=0x7fc17800cc90 comp rx=0 tx=0).stop 2023-10-08T05:56:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 -- 172.21.15.70:0/54761597 shutdown_connections 2023-10-08T05:56:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fc16406b020 0x7fc16406d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1800c8ec0 0x7fc1800c6390 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc18015cc20 0x7fc1800c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 --2- 172.21.15.70:0/54761597 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1800cd770 0x7fc1801f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 -- 172.21.15.70:0/54761597 >> 172.21.15.70:0/54761597 conn(0x7fc180069f30 msgr2=0x7fc18006c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 -- 172.21.15.70:0/54761597 shutdown_connections 2023-10-08T05:56:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:58.895+0000 7fc18874e640 1 -- 172.21.15.70:0/54761597 wait complete. 2023-10-08T05:56:58.914 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:56:59.093 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.091+0000 7f0968e56040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:56:59.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.103+0000 7fcf02446640 1 Processor -- start 2023-10-08T05:56:59.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.103+0000 7fcf02446640 1 -- start start 2023-10-08T05:56:59.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.105+0000 7fcf02446640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc15ab70 0x7fcefc0cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.105+0000 7fcf02446640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcefc0d04e0 0x7fcefc0cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.105+0000 7fcf02446640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcefc15af50 con 0x7fcefc0cdca0 2023-10-08T05:56:59.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.105+0000 7fcf02446640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcefc0ce150 con 0x7fcefc0d04e0 2023-10-08T05:56:59.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.105+0000 7fcf02446640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0ced20 con 0x7fcefc15ab70 2023-10-08T05:56:59.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.105+0000 7fcf009e3640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcefc0cdca0 0x7fcefc0d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33266/0 (socket says 172.21.15.70:33266) 2023-10-08T05:56:59.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.105+0000 7fcf009e3640 1 -- 172.21.15.70:0/3025858721 learned_addr learned my addr 172.21.15.70:0/3025858721 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:59.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.105+0000 7fcefbfff640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc15ab70 0x7fcefc0cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcef3fff640 1 --2- 172.21.15.70:0/3025858721 >> v2:172.21.15.181:3300/0 conn(0x7fcefc0d04e0 0x7fcefc0cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106203735 0 0) 0x7fcefc15af50 con 0x7fcefc0cdca0 2023-10-08T05:56:59.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcee8003520 con 0x7fcefc0cdca0 2023-10-08T05:56:59.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2745518861 0 0) 0x7fcee8003520 con 0x7fcefc0cdca0 2023-10-08T05:56:59.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc15ab70 msgr2=0x7fcefc0cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:59.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc15ab70 0x7fcefc0cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 >> v2:172.21.15.181:3300/0 conn(0x7fcefc0d04e0 msgr2=0x7fcefc0cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:59.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 --2- 172.21.15.70:0/3025858721 >> v2:172.21.15.181:3300/0 conn(0x7fcefc0d04e0 0x7fcefc0cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc0cf000 con 0x7fcefc0cdca0 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fcef40019f0 con 0x7fcefc0cdca0 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 >> v1:172.21.15.70:6789/0 conn(0x7fcefc0cdca0 legacy=0x7fcefc0d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc15ab70 0x7fcee8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 0x7fcee80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8004d00 0x7fcee8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0ce150 con 0x7fcee8003f40 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0ced20 con 0x7fcee8004d00 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee800b980 con 0x7fcefc15ab70 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.106+0000 7fcef3fff640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 0x7fcee80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.107+0000 7fcefbfff640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc15ab70 0x7fcee8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.107+0000 7fcf009e3640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8004d00 0x7fcee8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.107+0000 7fcef3fff640 1 -- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc15ab70 msgr2=0x7fcee8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:59.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.107+0000 7fcef3fff640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc15ab70 0x7fcee8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.107+0000 7fcef3fff640 1 -- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8004d00 msgr2=0x7fcee8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:59.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.107+0000 7fcef3fff640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8004d00 0x7fcee8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.107+0000 7fcef3fff640 1 -- 172.21.15.70:0/3025858721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee8003520 con 0x7fcee8003f40 2023-10-08T05:56:59.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.107+0000 7fcefbfff640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc15ab70 0x7fcee8003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:59.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.107+0000 7fcf009e3640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcee8004d00 0x7fcee8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:59.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.108+0000 7fcef3fff640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 0x7fcee80047c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009af0 tx=0x7fcee80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b532eaa7eece939e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:59.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.108+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee402c710 con 0x7fcee8003f40 2023-10-08T05:56:59.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.108+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee402c870 con 0x7fcee8003f40 2023-10-08T05:56:59.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.108+0000 7fcefb7fe640 1 -- 172.21.15.70:0/3025858721 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee402cc00 con 0x7fcee8003f40 2023-10-08T05:56:59.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.109+0000 7fcf02446640 1 -- 172.21.15.70:0/3025858721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 msgr2=0x7fcee80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.109+0000 7fcf02446640 1 --2- 172.21.15.70:0/3025858721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 0x7fcee80047c0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009af0 tx=0x7fcee80057f0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.109+0000 7fcf009e3640 1 -- 172.21.15.70:0/3025858721 reap_dead start 2023-10-08T05:56:59.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.109+0000 7fcf02446640 1 -- 172.21.15.70:0/3025858721 shutdown_connections 2023-10-08T05:56:59.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.109+0000 7fcf02446640 1 -- 172.21.15.70:0/3025858721 >> 172.21.15.70:0/3025858721 conn(0x7fcefc069f30 msgr2=0x7fcefc06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:59.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.110+0000 7fcf02446640 1 -- 172.21.15.70:0/3025858721 shutdown_connections 2023-10-08T05:56:59.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.110+0000 7fcf02446640 1 -- 172.21.15.70:0/3025858721 wait complete. 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.111+0000 7fcf02446640 1 Processor -- start 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.111+0000 7fcf02446640 1 -- start start 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.112+0000 7fcf02446640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 0x7fcefc0c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.112+0000 7fcf02446640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc0a8100 0x7fcefc14ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.112+0000 7fcf02446640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcefc0c8f30 0x7fcefc0c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.112+0000 7fcf02446640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0c7460 con 0x7fcee8003f40 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.112+0000 7fcf02446640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0c98f0 con 0x7fcefc0c8f30 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.112+0000 7fcf02446640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0c7880 con 0x7fcefc0a8100 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcefbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 0x7fcefc0c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcefbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 0x7fcefc0c60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34040/0 (socket says 172.21.15.70:34040) 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcefbfff640 1 -- 172.21.15.70:0/2618602151 learned_addr learned my addr 172.21.15.70:0/2618602151 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:59.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcf009e3640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcefc0c8f30 0x7fcefc0c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcef3fff640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc0a8100 0x7fcefc14ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcefbfff640 1 -- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc0a8100 msgr2=0x7fcefc14ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcefbfff640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc0a8100 0x7fcefc14ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcefbfff640 1 -- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcefc0c8f30 msgr2=0x7fcefc0c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcefbfff640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcefc0c8f30 0x7fcefc0c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcefbfff640 1 -- 172.21.15.70:0/2618602151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc0a8580 con 0x7fcee8003f40 2023-10-08T05:56:59.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.113+0000 7fcefbfff640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 0x7fcefc0c60e0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fceec0108b0 tx=0x7fceec010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:59.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.114+0000 7fcef97fa640 1 -- 172.21.15.70:0/2618602151 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec00eb60 con 0x7fcee8003f40 2023-10-08T05:56:59.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.114+0000 7fcef97fa640 1 -- 172.21.15.70:0/2618602151 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec00ecc0 con 0x7fcee8003f40 2023-10-08T05:56:59.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.114+0000 7fcef97fa640 1 -- 172.21.15.70:0/2618602151 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec01c400 con 0x7fcee8003f40 2023-10-08T05:56:59.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.114+0000 7fcf02446640 1 -- 172.21.15.70:0/2618602151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc0c67e0 con 0x7fcee8003f40 2023-10-08T05:56:59.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.114+0000 7fcf02446640 1 -- 172.21.15.70:0/2618602151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc0c6da0 con 0x7fcee8003f40 2023-10-08T05:56:59.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.115+0000 7fcf02446640 1 -- 172.21.15.70:0/2618602151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcefc0c76d0 con 0x7fcee8003f40 2023-10-08T05:56:59.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.116+0000 7fcef97fa640 1 -- 172.21.15.70:0/2618602151 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 73) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fceec01c560 con 0x7fcee8003f40 2023-10-08T05:56:59.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.117+0000 7fcef97fa640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fcedc06b020 0x7fcedc06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.117+0000 7fcef97fa640 1 -- 172.21.15.70:0/2618602151 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fceec08a240 con 0x7fcee8003f40 2023-10-08T05:56:59.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.117+0000 7fcef3fff640 1 -- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fcedc06b020 msgr2=0x7fcedc06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:59.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.117+0000 7fcef3fff640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fcedc06b020 0x7fcedc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:59.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.121+0000 7fcef97fa640 1 -- 172.21.15.70:0/2618602151 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fceec054db0 con 0x7fcee8003f40 2023-10-08T05:56:59.251 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:59.249+0000 7fdcbf2b6040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:56:59.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.311+0000 7fcf02446640 1 -- 172.21.15.70:0/2618602151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcefc066d80 con 0x7fcee8003f40 2023-10-08T05:56:59.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.313+0000 7fcef97fa640 1 -- 172.21.15.70:0/2618602151 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+17395 (secure 0 0 0) 0x7fceec0583d0 con 0x7fcee8003f40 2023-10-08T05:56:59.315 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:56:59.315 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":150,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:46.566262+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:56:30.775239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-08T05:56:31.378327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1640710135":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/1739083188":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/202771123":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/3030439524":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/4192220981":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/3672243748":"2023-10-09T05:56:46.566167+0000","172.21.15.181:6833/3857845516":"2023-10-09T05:56:46.566167+0000","172.21.15.181:6832/3857845516":"2023-10-09T05:56:46.566167+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:56:59.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.315+0000 7fcf02446640 1 -- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fcedc06b020 msgr2=0x7fcedc06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:59.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.315+0000 7fcf02446640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fcedc06b020 0x7fcedc06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.315+0000 7fcf02446640 1 -- 172.21.15.70:0/2618602151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 msgr2=0x7fcefc0c60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.315+0000 7fcf02446640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 0x7fcefc0c60e0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fceec0108b0 tx=0x7fceec010d70 comp rx=0 tx=0).stop 2023-10-08T05:56:59.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.316+0000 7fcf02446640 1 -- 172.21.15.70:0/2618602151 shutdown_connections 2023-10-08T05:56:59.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.316+0000 7fcf02446640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fcedc06b020 0x7fcedc06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.316+0000 7fcf02446640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcefc0c8f30 0x7fcefc0c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.316+0000 7fcf02446640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcefc0a8100 0x7fcefc14ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.316+0000 7fcf02446640 1 --2- 172.21.15.70:0/2618602151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcee8003f40 0x7fcefc0c60e0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.316+0000 7fcf02446640 1 -- 172.21.15.70:0/2618602151 >> 172.21.15.70:0/2618602151 conn(0x7fcefc069f30 msgr2=0x7fcefc0b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:59.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.316+0000 7fcf02446640 1 -- 172.21.15.70:0/2618602151 shutdown_connections 2023-10-08T05:56:59.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.316+0000 7fcf02446640 1 -- 172.21.15.70:0/2618602151 wait complete. 2023-10-08T05:56:59.335 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:56:30.775239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:56:31.378327+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:56:59.336 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:56:59.391 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.390+0000 7f0968e56040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:56:59.447 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.446+0000 7f0968e56040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:56:59.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.540+0000 7fe27caf3640 1 Processor -- start 2023-10-08T05:56:59.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.540+0000 7fe27caf3640 1 -- start start 2023-10-08T05:56:59.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.542+0000 7fe27caf3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe2780d1120 0x7fe2780d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.542+0000 7fe27caf3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe2780d1500 0x7fe2781610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.542+0000 7fe27caf3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2780d0a80 con 0x7fe2780b5060 2023-10-08T05:56:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.542+0000 7fe27caf3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe278161620 con 0x7fe2780d1120 2023-10-08T05:56:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.542+0000 7fe27caf3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2780b1fe0 con 0x7fe2780d1500 2023-10-08T05:56:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.542+0000 7fe2777fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe2780d1120 0x7fe2780d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.542+0000 7fe2777fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe2780d1120 0x7fe2780d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34042/0 (socket says 172.21.15.70:34042) 2023-10-08T05:56:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.542+0000 7fe2777fe640 1 -- 172.21.15.70:0/1350191109 learned_addr learned my addr 172.21.15.70:0/1350191109 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.542+0000 7fe26ffff640 1 --2- 172.21.15.70:0/1350191109 >> v2:172.21.15.181:3300/0 conn(0x7fe2780d1500 0x7fe2781610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.543+0000 7fe2777fe640 1 -- 172.21.15.70:0/1350191109 >> v2:172.21.15.181:3300/0 conn(0x7fe2780d1500 msgr2=0x7fe2781610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.543+0000 7fe2777fe640 1 --2- 172.21.15.70:0/1350191109 >> v2:172.21.15.181:3300/0 conn(0x7fe2780d1500 0x7fe2781610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.543+0000 7fe2777fe640 1 -- 172.21.15.70:0/1350191109 >> v1:172.21.15.70:6789/0 conn(0x7fe2780b5060 legacy=0x7fe2780cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.543+0000 7fe2777fe640 1 -- 172.21.15.70:0/1350191109 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2780b4580 con 0x7fe2780d1120 2023-10-08T05:56:59.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.544+0000 7fe2777fe640 1 --2- 172.21.15.70:0/1350191109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 0x7fe2780d04a0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fe2780d28c0 tx=0x7fe26802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ca28c781519aef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:59.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.544+0000 7fe276ffd640 1 -- 172.21.15.70:0/1350191109 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268004230 con 0x7fe2780d1120 2023-10-08T05:56:59.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.544+0000 7fe276ffd640 1 -- 172.21.15.70:0/1350191109 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe268004390 con 0x7fe2780d1120 2023-10-08T05:56:59.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.544+0000 7fe276ffd640 1 -- 172.21.15.70:0/1350191109 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe268003810 con 0x7fe2780d1120 2023-10-08T05:56:59.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.545+0000 7fe27caf3640 1 -- 172.21.15.70:0/1350191109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 msgr2=0x7fe2780d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.545+0000 7fe27caf3640 1 --2- 172.21.15.70:0/1350191109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 0x7fe2780d04a0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fe2780d28c0 tx=0x7fe26802b860 comp rx=0 tx=0).stop 2023-10-08T05:56:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.545+0000 7fe27caf3640 1 -- 172.21.15.70:0/1350191109 shutdown_connections 2023-10-08T05:56:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.545+0000 7fe27caf3640 1 --2- 172.21.15.70:0/1350191109 >> v2:172.21.15.181:3300/0 conn(0x7fe2780d1500 0x7fe2781610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.545+0000 7fe27caf3640 1 --2- 172.21.15.70:0/1350191109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 0x7fe2780d04a0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.545+0000 7fe27caf3640 1 -- 172.21.15.70:0/1350191109 >> 172.21.15.70:0/1350191109 conn(0x7fe2780d2960 msgr2=0x7fe2780d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:59.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.545+0000 7fe27caf3640 1 -- 172.21.15.70:0/1350191109 shutdown_connections 2023-10-08T05:56:59.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.545+0000 7fe27caf3640 1 -- 172.21.15.70:0/1350191109 wait complete. 2023-10-08T05:56:59.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.547+0000 7fe27caf3640 1 Processor -- start 2023-10-08T05:56:59.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.547+0000 7fe27caf3640 1 -- start start 2023-10-08T05:56:59.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe27caf3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2780b5060 0x7fe27814ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe27caf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 0x7fe2780c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe27caf3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2780d1500 0x7fe278150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe27caf3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27814f6f0 con 0x7fe2780d1120 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe27caf3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27814f9d0 con 0x7fe2780b5060 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe27caf3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27814fcb0 con 0x7fe2780d1500 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe26ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 0x7fe2780c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe26ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 0x7fe2780c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34046/0 (socket says 172.21.15.70:34046) 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe26ffff640 1 -- 172.21.15.70:0/1447400249 learned_addr learned my addr 172.21.15.70:0/1447400249 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe2777fe640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2780b5060 0x7fe27814ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe277fff640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2780d1500 0x7fe278150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.548+0000 7fe26ffff640 1 -- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2780d1500 msgr2=0x7fe278150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe26ffff640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2780d1500 0x7fe278150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe26ffff640 1 -- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2780b5060 msgr2=0x7fe27814ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe26ffff640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2780b5060 0x7fe27814ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe26ffff640 1 -- 172.21.15.70:0/1447400249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe268009460 con 0x7fe2780d1120 2023-10-08T05:56:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe277fff640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2780d1500 0x7fe278150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:56:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe26ffff640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 0x7fe2780c63e0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe26000d5f0 tx=0x7fe26000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe274ff9640 1 -- 172.21.15.70:0/1447400249 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2600125f0 con 0x7fe2780d1120 2023-10-08T05:56:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe274ff9640 1 -- 172.21.15.70:0/1447400249 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe260012750 con 0x7fe2780d1120 2023-10-08T05:56:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe274ff9640 1 -- 172.21.15.70:0/1447400249 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe260012b00 con 0x7fe2780d1120 2023-10-08T05:56:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe27caf3640 1 -- 172.21.15.70:0/1447400249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27814ff90 con 0x7fe2780d1120 2023-10-08T05:56:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.549+0000 7fe27caf3640 1 -- 172.21.15.70:0/1447400249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27814b440 con 0x7fe2780d1120 2023-10-08T05:56:59.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.550+0000 7fe27caf3640 1 -- 172.21.15.70:0/1447400249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe278074150 con 0x7fe2780d1120 2023-10-08T05:56:59.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.551+0000 7fe274ff9640 1 -- 172.21.15.70:0/1447400249 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 73) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fe260012c60 con 0x7fe2780d1120 2023-10-08T05:56:59.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.552+0000 7fe274ff9640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe25806b020 0x7fe25806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.552+0000 7fe274ff9640 1 -- 172.21.15.70:0/1447400249 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6445+0+0 (secure 0 0 0) 0x7fe26008a180 con 0x7fe2780d1120 2023-10-08T05:56:59.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.552+0000 7fe2777fe640 1 -- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe25806b020 msgr2=0x7fe25806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1964243521 2023-10-08T05:56:59.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.552+0000 7fe2777fe640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe25806b020 0x7fe25806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:56:59.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.556+0000 7fe274ff9640 1 -- 172.21.15.70:0/1447400249 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe260054c30 con 0x7fe2780d1120 2023-10-08T05:56:59.563 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:59.562+0000 7fdcbf2b6040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:56:59.564 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.562+0000 7f0968e56040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:56:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.578+0000 7fe274ff9640 1 -- 172.21.15.70:0/1447400249 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 74) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe260055000 con 0x7fe2780d1120 2023-10-08T05:56:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.578+0000 7fe274ff9640 1 -- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe25806b020 msgr2=0x7fe25806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:56:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.578+0000 7fe274ff9640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe25806b020 0x7fe25806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.631 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:59.629+0000 7fdcbf2b6040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:56:59.647 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.646+0000 7f08eb22e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:59.648 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.646+0000 7f08eb22e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:59.648 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.646+0000 7f08eb22e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:59.648 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.646+0000 7f08eb22e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:59.648 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.646+0000 7f08eb22e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:59.648 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.647+0000 7f08ee234640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:59.648 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.647+0000 7f08ee234640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:59.648 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.647+0000 7f08ee234640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:59.648 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.647+0000 7f08ee234640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:59.648 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:56:59.647+0000 7f08ee234640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:56:59.749 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:56:59.747+0000 7fdcbf2b6040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:56:59.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.748+0000 7fe27caf3640 1 -- 172.21.15.70:0/1447400249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe27806a7a0 con 0x7fe2780d1120 2023-10-08T05:56:59.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.749+0000 7fe274ff9640 1 -- 172.21.15.70:0/1447400249 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe260086cf0 con 0x7fe2780d1120 2023-10-08T05:56:59.751 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:56:59.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.752+0000 7fe27caf3640 1 -- 172.21.15.70:0/1447400249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 msgr2=0x7fe2780c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.752+0000 7fe27caf3640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 0x7fe2780c63e0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fe26000d5f0 tx=0x7fe26000dab0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.752+0000 7fe27caf3640 1 -- 172.21.15.70:0/1447400249 shutdown_connections 2023-10-08T05:56:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.752+0000 7fe27caf3640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:6832/1964243521,v1:172.21.15.181:6833/1964243521] conn(0x7fe25806b020 0x7fe25806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.752+0000 7fe27caf3640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe2780d1500 0x7fe278150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.752+0000 7fe27caf3640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2780d1120 0x7fe2780c63e0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.752+0000 7fe27caf3640 1 --2- 172.21.15.70:0/1447400249 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe2780b5060 0x7fe27814ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.752+0000 7fe27caf3640 1 -- 172.21.15.70:0/1447400249 >> 172.21.15.70:0/1447400249 conn(0x7fe2780d2960 msgr2=0x7fe2780b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.752+0000 7fe27caf3640 1 -- 172.21.15.70:0/1447400249 shutdown_connections 2023-10-08T05:56:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.752+0000 7fe27caf3640 1 -- 172.21.15.70:0/1447400249 wait complete. 2023-10-08T05:56:59.770 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T05:56:59.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.962+0000 7fbda8f7c640 1 Processor -- start 2023-10-08T05:56:59.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.962+0000 7fbda8f7c640 1 -- start start 2023-10-08T05:56:59.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.964+0000 7fbda8f7c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbda4159b50 0x7fbda415e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.964+0000 7fbda8f7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ae740 0x7fbda40aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.964+0000 7fbda8f7c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda40cd430 con 0x7fbda40ceaa0 2023-10-08T05:56:59.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.964+0000 7fbda8f7c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda40af110 con 0x7fbda4159b50 2023-10-08T05:56:59.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.964+0000 7fbda8f7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40cf260 con 0x7fbda40ae740 2023-10-08T05:56:59.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.965+0000 7fbda259c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbda4159b50 0x7fbda415e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.965+0000 7fbda2d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ae740 0x7fbda40aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.965+0000 7fbda1d9b640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fbda40ceaa0 0x7fbda40cb190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:53466/0 (socket says 172.21.15.70:53466) 2023-10-08T05:56:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.965+0000 7fbda1d9b640 1 -- 172.21.15.70:0/4018587518 learned_addr learned my addr 172.21.15.70:0/4018587518 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:56:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.966+0000 7fbda2d9d640 1 -- 172.21.15.70:0/4018587518 >> v2:172.21.15.181:3300/0 conn(0x7fbda4159b50 msgr2=0x7fbda415e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:56:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.966+0000 7fbda2d9d640 1 --2- 172.21.15.70:0/4018587518 >> v2:172.21.15.181:3300/0 conn(0x7fbda4159b50 0x7fbda415e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.966+0000 7fbda2d9d640 1 -- 172.21.15.70:0/4018587518 >> v1:172.21.15.181:6789/0 conn(0x7fbda40ceaa0 legacy=0x7fbda40cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.966+0000 7fbda2d9d640 1 -- 172.21.15.70:0/4018587518 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda40cf540 con 0x7fbda40ae740 2023-10-08T05:56:59.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.966+0000 7fbda159a640 1 -- 172.21.15.70:0/4018587518 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479119744 0 0) 0x7fbda40cd430 con 0x7fbda40ceaa0 2023-10-08T05:56:59.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.966+0000 7fbda259c640 1 --2- 172.21.15.70:0/4018587518 >> v2:172.21.15.181:3300/0 conn(0x7fbda4159b50 0x7fbda415e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:56:59.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.967+0000 7fbda2d9d640 1 --2- 172.21.15.70:0/4018587518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ae740 0x7fbda40aebd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0097a0 tx=0x7fbd9c02dac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e13e88121bb84462 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:59.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.968+0000 7fbda159a640 1 -- 172.21.15.70:0/4018587518 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c02f880 con 0x7fbda40ae740 2023-10-08T05:56:59.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.968+0000 7fbda159a640 1 -- 172.21.15.70:0/4018587518 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c02f9e0 con 0x7fbda40ae740 2023-10-08T05:56:59.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.968+0000 7fbda159a640 1 -- 172.21.15.70:0/4018587518 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c02fd70 con 0x7fbda40ae740 2023-10-08T05:56:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.968+0000 7fbda8f7c640 1 -- 172.21.15.70:0/4018587518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ae740 msgr2=0x7fbda40aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.968+0000 7fbda8f7c640 1 --2- 172.21.15.70:0/4018587518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ae740 0x7fbda40aebd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0097a0 tx=0x7fbd9c02dac0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.968+0000 7fbda8f7c640 1 -- 172.21.15.70:0/4018587518 shutdown_connections 2023-10-08T05:56:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.968+0000 7fbda8f7c640 1 --2- 172.21.15.70:0/4018587518 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ae740 0x7fbda40aebd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.968+0000 7fbda8f7c640 1 --2- 172.21.15.70:0/4018587518 >> v2:172.21.15.181:3300/0 conn(0x7fbda4159b50 0x7fbda415e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.969+0000 7fbda8f7c640 1 -- 172.21.15.70:0/4018587518 >> 172.21.15.70:0/4018587518 conn(0x7fbda406d430 msgr2=0x7fbda406d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:56:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.969+0000 7fbda8f7c640 1 -- 172.21.15.70:0/4018587518 shutdown_connections 2023-10-08T05:56:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.969+0000 7fbda8f7c640 1 -- 172.21.15.70:0/4018587518 wait complete. 2023-10-08T05:56:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.970+0000 7fbda8f7c640 1 Processor -- start 2023-10-08T05:56:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.970+0000 7fbda8f7c640 1 -- start start 2023-10-08T05:56:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.971+0000 7fbda8f7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda40ae740 0x7fbda414d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.971+0000 7fbda8f7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ceaa0 0x7fbda40afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.971+0000 7fbda8f7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbda4159b50 0x7fbda40c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:56:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.971+0000 7fbda8f7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40b0790 con 0x7fbda40ae740 2023-10-08T05:56:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.971+0000 7fbda8f7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40b0a70 con 0x7fbda4159b50 2023-10-08T05:56:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.971+0000 7fbda8f7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40b0d50 con 0x7fbda40ceaa0 2023-10-08T05:56:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.972+0000 7fbda259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda40ae740 0x7fbda414d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.972+0000 7fbda259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda40ae740 0x7fbda414d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34052/0 (socket says 172.21.15.70:34052) 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.972+0000 7fbda259c640 1 -- 172.21.15.70:0/2138185796 learned_addr learned my addr 172.21.15.70:0/2138185796 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.972+0000 7fbda259c640 1 -- 172.21.15.70:0/2138185796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ceaa0 msgr2=0x7fbda40afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.972+0000 7fbda259c640 1 --2- 172.21.15.70:0/2138185796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ceaa0 0x7fbda40afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.972+0000 7fbda2d9d640 1 --2- 172.21.15.70:0/2138185796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbda4159b50 0x7fbda40c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.972+0000 7fbda259c640 1 -- 172.21.15.70:0/2138185796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbda4159b50 msgr2=0x7fbda40c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.972+0000 7fbda259c640 1 --2- 172.21.15.70:0/2138185796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbda4159b50 0x7fbda40c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.972+0000 7fbda259c640 1 -- 172.21.15.70:0/2138185796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd94009a70 con 0x7fbda40ae740 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.972+0000 7fbda259c640 1 --2- 172.21.15.70:0/2138185796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda40ae740 0x7fbda414d060 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400efc0 tx=0x7fbd9400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.973+0000 7fbd9affd640 1 -- 172.21.15.70:0/2138185796 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94014680 con 0x7fbda40ae740 2023-10-08T05:56:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.973+0000 7fbd9affd640 1 -- 172.21.15.70:0/2138185796 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9400f040 con 0x7fbda40ae740 2023-10-08T05:56:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.973+0000 7fbda8f7c640 1 -- 172.21.15.70:0/2138185796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c009400 con 0x7fbda40ae740 2023-10-08T05:56:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.973+0000 7fbd9affd640 1 -- 172.21.15.70:0/2138185796 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94014a00 con 0x7fbda40ae740 2023-10-08T05:56:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.973+0000 7fbda8f7c640 1 -- 172.21.15.70:0/2138185796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda40b1310 con 0x7fbda40ae740 2023-10-08T05:56:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.974+0000 7fbda8f7c640 1 -- 172.21.15.70:0/2138185796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda404e160 con 0x7fbda40ae740 2023-10-08T05:56:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.975+0000 7fbd9affd640 1 -- 172.21.15.70:0/2138185796 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 74) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fbd94014b60 con 0x7fbda40ae740 2023-10-08T05:56:59.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.976+0000 7fbd9affd640 1 -- 172.21.15.70:0/2138185796 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6832+0+0 (secure 0 0 0) 0x7fbd9408a700 con 0x7fbda40ae740 2023-10-08T05:56:59.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:56:59.980+0000 7fbd9affd640 1 -- 172.21.15.70:0/2138185796 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd94055640 con 0x7fbda40ae740 2023-10-08T05:57:00.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.169+0000 7fbda8f7c640 1 -- 172.21.15.70:0/2138185796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbda406a750 con 0x7fbda40ae740 2023-10-08T05:57:00.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.170+0000 7fbd9affd640 1 -- 172.21.15.70:0/2138185796 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v151) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd94058c60 con 0x7fbda40ae740 2023-10-08T05:57:00.171 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T05:57:00.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.172+0000 7fbda8f7c640 1 -- 172.21.15.70:0/2138185796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda40ae740 msgr2=0x7fbda414d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:00.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.172+0000 7fbda8f7c640 1 --2- 172.21.15.70:0/2138185796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda40ae740 0x7fbda414d060 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400efc0 tx=0x7fbd9400c490 comp rx=0 tx=0).stop 2023-10-08T05:57:00.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.173+0000 7fbda8f7c640 1 -- 172.21.15.70:0/2138185796 shutdown_connections 2023-10-08T05:57:00.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.173+0000 7fbda8f7c640 1 --2- 172.21.15.70:0/2138185796 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbda4159b50 0x7fbda40c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.173+0000 7fbda8f7c640 1 --2- 172.21.15.70:0/2138185796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ceaa0 0x7fbda40afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.173+0000 7fbda8f7c640 1 --2- 172.21.15.70:0/2138185796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda40ae740 0x7fbda414d060 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.173+0000 7fbda8f7c640 1 -- 172.21.15.70:0/2138185796 >> 172.21.15.70:0/2138185796 conn(0x7fbda406d430 msgr2=0x7fbda40cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:00.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.173+0000 7fbda8f7c640 1 -- 172.21.15.70:0/2138185796 shutdown_connections 2023-10-08T05:57:00.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.173+0000 7fbda8f7c640 1 -- 172.21.15.70:0/2138185796 wait complete. 2023-10-08T05:57:00.191 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T05:57:00.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.398+0000 7fe20458a640 1 Processor -- start 2023-10-08T05:57:00.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.398+0000 7fe20458a640 1 -- start start 2023-10-08T05:57:00.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.400+0000 7fe20458a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc160660 0x7fe1fc070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.400+0000 7fe20458a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1fc0cb620 0x7fe1fc0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.400+0000 7fe20458a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1fc06e150 con 0x7fe1fc0c9910 2023-10-08T05:57:00.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.400+0000 7fe20458a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1fc06de70 con 0x7fe1fc0cb620 2023-10-08T05:57:00.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.400+0000 7fe20458a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c8e30 con 0x7fe1fc160660 2023-10-08T05:57:00.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.400+0000 7fe201b25640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe1fc0c9910 0x7fe1fc0caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33274/0 (socket says 172.21.15.70:33274) 2023-10-08T05:57:00.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.400+0000 7fe201b25640 1 -- 172.21.15.70:0/2533417932 learned_addr learned my addr 172.21.15.70:0/2533417932 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.400+0000 7fe202b27640 1 --2- 172.21.15.70:0/2533417932 >> v2:172.21.15.181:3300/0 conn(0x7fe1fc0cb620 0x7fe1fc0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe202326640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc160660 0x7fe1fc070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422546886 0 0) 0x7fe1fc06e150 con 0x7fe1fc0c9910 2023-10-08T05:57:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e8003520 con 0x7fe1fc0c9910 2023-10-08T05:57:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 31425410 0 0) 0x7fe1e8003520 con 0x7fe1fc0c9910 2023-10-08T05:57:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc160660 msgr2=0x7fe1fc070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc160660 0x7fe1fc070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 >> v2:172.21.15.181:3300/0 conn(0x7fe1fc0cb620 msgr2=0x7fe1fc0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 --2- 172.21.15.70:0/2533417932 >> v2:172.21.15.181:3300/0 conn(0x7fe1fc0cb620 0x7fe1fc0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0c9110 con 0x7fe1fc0c9910 2023-10-08T05:57:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe1ec001670 con 0x7fe1fc0c9910 2023-10-08T05:57:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 >> v1:172.21.15.70:6789/0 conn(0x7fe1fc0c9910 legacy=0x7fe1fc0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8003970 0x7fe1e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc160660 0x7fe1e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8004d00 0x7fe1e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc06de70 con 0x7fe1e8004d00 2023-10-08T05:57:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c8e30 con 0x7fe1e8003970 2023-10-08T05:57:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.401+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e800b980 con 0x7fe1fc160660 2023-10-08T05:57:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.402+0000 7fe202b27640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8004d00 0x7fe1e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.402+0000 7fe202326640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8003970 0x7fe1e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.402+0000 7fe202b27640 1 -- 172.21.15.70:0/2533417932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc160660 msgr2=0x7fe1e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.402+0000 7fe202b27640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc160660 0x7fe1e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.402+0000 7fe202b27640 1 -- 172.21.15.70:0/2533417932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8003970 msgr2=0x7fe1e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.402+0000 7fe202b27640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8003970 0x7fe1e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.402+0000 7fe202b27640 1 -- 172.21.15.70:0/2533417932 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8003520 con 0x7fe1e8004d00 2023-10-08T05:57:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.403+0000 7fe202b27640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8004d00 0x7fe1e8005180 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009fc0 tx=0x7fe1e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb92471132a99f37 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.403+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f402cc60 con 0x7fe1e8004d00 2023-10-08T05:57:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.403+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f400f040 con 0x7fe1e8004d00 2023-10-08T05:57:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.403+0000 7fe201324640 1 -- 172.21.15.70:0/2533417932 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4034440 con 0x7fe1e8004d00 2023-10-08T05:57:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.404+0000 7fe20458a640 1 -- 172.21.15.70:0/2533417932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8004d00 msgr2=0x7fe1e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.404+0000 7fe20458a640 1 --2- 172.21.15.70:0/2533417932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8004d00 0x7fe1e8005180 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009fc0 tx=0x7fe1e80057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.404+0000 7fe202b27640 1 -- 172.21.15.70:0/2533417932 reap_dead start 2023-10-08T05:57:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.404+0000 7fe20458a640 1 -- 172.21.15.70:0/2533417932 shutdown_connections 2023-10-08T05:57:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.404+0000 7fe20458a640 1 -- 172.21.15.70:0/2533417932 >> 172.21.15.70:0/2533417932 conn(0x7fe1fc06a5d0 msgr2=0x7fe1fc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.404+0000 7fe20458a640 1 -- 172.21.15.70:0/2533417932 shutdown_connections 2023-10-08T05:57:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.404+0000 7fe20458a640 1 -- 172.21.15.70:0/2533417932 wait complete. 2023-10-08T05:57:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.406+0000 7fe20458a640 1 Processor -- start 2023-10-08T05:57:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.406+0000 7fe20458a640 1 -- start start 2023-10-08T05:57:00.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.407+0000 7fe20458a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8004d00 0x7fe1fc14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.407+0000 7fe20458a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc0ac7a0 0x7fe1fc0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.407+0000 7fe20458a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1fc0acc20 0x7fe1fc0abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.407+0000 7fe20458a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc14d200 con 0x7fe1fc0acc20 2023-10-08T05:57:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.407+0000 7fe20458a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c2250 con 0x7fe1e8004d00 2023-10-08T05:57:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.407+0000 7fe20458a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0c2af0 con 0x7fe1fc0ac7a0 2023-10-08T05:57:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.407+0000 7fe202b27640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1fc0acc20 0x7fe1fc0abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.407+0000 7fe202b27640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1fc0acc20 0x7fe1fc0abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34080/0 (socket says 172.21.15.70:34080) 2023-10-08T05:57:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.407+0000 7fe202b27640 1 -- 172.21.15.70:0/1018546183 learned_addr learned my addr 172.21.15.70:0/1018546183 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe202326640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8004d00 0x7fe1fc14ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe201b25640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc0ac7a0 0x7fe1fc0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe202b27640 1 -- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc0ac7a0 msgr2=0x7fe1fc0ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe202b27640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc0ac7a0 0x7fe1fc0ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe202b27640 1 -- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8004d00 msgr2=0x7fe1fc14ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe202b27640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8004d00 0x7fe1fc14ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe202b27640 1 -- 172.21.15.70:0/1018546183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0ad1f0 con 0x7fe1fc0acc20 2023-10-08T05:57:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe202326640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8004d00 0x7fe1fc14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe202b27640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1fc0acc20 0x7fe1fc0abe10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4037040 tx=0x7fe1f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe1f2ffd640 1 -- 172.21.15.70:0/1018546183 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4043070 con 0x7fe1fc0acc20 2023-10-08T05:57:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe1f2ffd640 1 -- 172.21.15.70:0/1018546183 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4034b50 con 0x7fe1fc0acc20 2023-10-08T05:57:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe1f2ffd640 1 -- 172.21.15.70:0/1018546183 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4003990 con 0x7fe1fc0acc20 2023-10-08T05:57:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.408+0000 7fe20458a640 1 -- 172.21.15.70:0/1018546183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1fc0c24a0 con 0x7fe1fc0acc20 2023-10-08T05:57:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.409+0000 7fe20458a640 1 -- 172.21.15.70:0/1018546183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc0af530 con 0x7fe1fc0acc20 2023-10-08T05:57:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.410+0000 7fe20458a640 1 -- 172.21.15.70:0/1018546183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1fc0af6f0 con 0x7fe1fc0acc20 2023-10-08T05:57:00.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.411+0000 7fe1f2ffd640 1 -- 172.21.15.70:0/1018546183 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 74) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe1f403e440 con 0x7fe1fc0acc20 2023-10-08T05:57:00.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.411+0000 7fe1f2ffd640 1 -- 172.21.15.70:0/1018546183 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6832+0+0 (secure 0 0 0) 0x7fe1f40aaff0 con 0x7fe1fc0acc20 2023-10-08T05:57:00.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.415+0000 7fe1f2ffd640 1 -- 172.21.15.70:0/1018546183 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1f4076090 con 0x7fe1fc0acc20 2023-10-08T05:57:00.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.592+0000 7fe1f2ffd640 1 -- 172.21.15.70:0/1018546183 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 75) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fe1f4075b40 con 0x7fe1fc0acc20 2023-10-08T05:57:00.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.593+0000 7fe1f2ffd640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe1e0070cd0 0x7fe1e00730c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.601+0000 7fe202326640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe1e0070cd0 0x7fe1e00730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.605+0000 7fe202326640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe1e0070cd0 0x7fe1e00730c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009a20 tx=0x7fe1f8008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:00.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.613+0000 7fe20458a640 1 -- 172.21.15.70:0/1018546183 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe1fc04e160 con 0x7fe1fc0acc20 2023-10-08T05:57:00.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.614+0000 7fe1f2ffd640 1 -- 172.21.15.70:0/1018546183 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v151) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe1f40390a0 con 0x7fe1fc0acc20 2023-10-08T05:57:00.615 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T05:57:00.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 -- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe1e0070cd0 msgr2=0x7fe1e00730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe1e0070cd0 0x7fe1e00730c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009a20 tx=0x7fe1f8008040 comp rx=0 tx=0).stop 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 -- 172.21.15.70:0/1018546183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1fc0acc20 msgr2=0x7fe1fc0abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1fc0acc20 0x7fe1fc0abe10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4037040 tx=0x7fe1f4009750 comp rx=0 tx=0).stop 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 -- 172.21.15.70:0/1018546183 shutdown_connections 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe1e0070cd0 0x7fe1e00730c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1fc0acc20 0x7fe1fc0abe10 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1fc0ac7a0 0x7fe1fc0ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 --2- 172.21.15.70:0/1018546183 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1e8004d00 0x7fe1fc14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 -- 172.21.15.70:0/1018546183 >> 172.21.15.70:0/1018546183 conn(0x7fe1fc06a5d0 msgr2=0x7fe1fc15f540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 -- 172.21.15.70:0/1018546183 shutdown_connections 2023-10-08T05:57:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.617+0000 7fe20458a640 1 -- 172.21.15.70:0/1018546183 wait complete. 2023-10-08T05:57:00.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:57:00.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.828+0000 7f97fa256640 1 Processor -- start 2023-10-08T05:57:00.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.829+0000 7f97fa256640 1 -- start start 2023-10-08T05:57:00.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97fa256640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f4070db0 0x7f97f415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97fa256640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f97f4071190 0x7f97f415e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97fa256640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f97f406ad00 0x7f97f406b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97fa256640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f40c9850 con 0x7f97f406ad00 2023-10-08T05:57:00.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97fa256640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f406a870 con 0x7f97f4071190 2023-10-08T05:57:00.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97fa256640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f415e280 con 0x7f97f4070db0 2023-10-08T05:57:00.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97f3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f97f406ad00 0x7f97f406b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97f3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f97f406ad00 0x7f97f406b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34090/0 (socket says 172.21.15.70:34090) 2023-10-08T05:57:00.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97f3fff640 1 -- 172.21.15.70:0/1608865060 learned_addr learned my addr 172.21.15.70:0/1608865060 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:00.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97f2ffd640 1 --2- 172.21.15.70:0/1608865060 >> v2:172.21.15.181:3300/0 conn(0x7f97f4071190 0x7f97f415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.830+0000 7f97f37fe640 1 --2- 172.21.15.70:0/1608865060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f4070db0 0x7f97f415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.831+0000 7f97f3fff640 1 -- 172.21.15.70:0/1608865060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f4070db0 msgr2=0x7f97f415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:00.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.831+0000 7f97f3fff640 1 --2- 172.21.15.70:0/1608865060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f4070db0 0x7f97f415dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.831+0000 7f97f3fff640 1 -- 172.21.15.70:0/1608865060 >> v2:172.21.15.181:3300/0 conn(0x7f97f4071190 msgr2=0x7f97f415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:00.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.831+0000 7f97f3fff640 1 --2- 172.21.15.70:0/1608865060 >> v2:172.21.15.181:3300/0 conn(0x7f97f4071190 0x7f97f415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.831+0000 7f97f3fff640 1 -- 172.21.15.70:0/1608865060 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f406b6b0 con 0x7f97f406ad00 2023-10-08T05:57:00.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.831+0000 7f97f2ffd640 1 --2- 172.21.15.70:0/1608865060 >> v2:172.21.15.181:3300/0 conn(0x7f97f4071190 0x7f97f415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:00.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.832+0000 7f97f3fff640 1 --2- 172.21.15.70:0/1608865060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 0x7f97f406b170 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f97f4071c00 tx=0x7f97ec02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aea4612e994ba39 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:00.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.832+0000 7f97f27fc640 1 -- 172.21.15.70:0/1608865060 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec030850 con 0x7f97f406ad00 2023-10-08T05:57:00.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.832+0000 7f97f27fc640 1 -- 172.21.15.70:0/1608865060 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec0309b0 con 0x7f97f406ad00 2023-10-08T05:57:00.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.832+0000 7f97f27fc640 1 -- 172.21.15.70:0/1608865060 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec030d40 con 0x7f97f406ad00 2023-10-08T05:57:00.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.833+0000 7f97fa256640 1 -- 172.21.15.70:0/1608865060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 msgr2=0x7f97f406b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:00.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.833+0000 7f97fa256640 1 --2- 172.21.15.70:0/1608865060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 0x7f97f406b170 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f97f4071c00 tx=0x7f97ec02b990 comp rx=0 tx=0).stop 2023-10-08T05:57:00.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.833+0000 7f97fa256640 1 -- 172.21.15.70:0/1608865060 shutdown_connections 2023-10-08T05:57:00.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.833+0000 7f97fa256640 1 --2- 172.21.15.70:0/1608865060 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 0x7f97f406b170 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.833+0000 7f97fa256640 1 --2- 172.21.15.70:0/1608865060 >> v2:172.21.15.181:3300/0 conn(0x7f97f4071190 0x7f97f415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.833+0000 7f97fa256640 1 --2- 172.21.15.70:0/1608865060 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f4070db0 0x7f97f415dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.833+0000 7f97fa256640 1 -- 172.21.15.70:0/1608865060 >> 172.21.15.70:0/1608865060 conn(0x7f97f40b1480 msgr2=0x7f97f406cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:00.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.833+0000 7f97fa256640 1 -- 172.21.15.70:0/1608865060 shutdown_connections 2023-10-08T05:57:00.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.833+0000 7f97fa256640 1 -- 172.21.15.70:0/1608865060 wait complete. 2023-10-08T05:57:00.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.834+0000 7f97fa256640 1 Processor -- start 2023-10-08T05:57:00.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.835+0000 7f97fa256640 1 -- start start 2023-10-08T05:57:00.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.835+0000 7f97fa256640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 0x7f97f40c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97fa256640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97f4070db0 0x7f97f41f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97fa256640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f4071190 0x7f97f4157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97fa256640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f41f6b90 con 0x7f97f406ad00 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97fa256640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f40c4a20 con 0x7f97f4070db0 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97fa256640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f40c7590 con 0x7f97f4071190 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97f37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 0x7f97f40c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97f37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 0x7f97f40c44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34102/0 (socket says 172.21.15.70:34102) 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97f37fe640 1 -- 172.21.15.70:0/3785617470 learned_addr learned my addr 172.21.15.70:0/3785617470 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97f37fe640 1 -- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f4071190 msgr2=0x7f97f4157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97f37fe640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f4071190 0x7f97f4157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97f37fe640 1 -- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97f4070db0 msgr2=0x7f97f41f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97f37fe640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97f4070db0 0x7f97f41f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:00.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97f37fe640 1 -- 172.21.15.70:0/3785617470 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0094d0 con 0x7f97f406ad00 2023-10-08T05:57:00.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.836+0000 7f97f37fe640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 0x7f97f40c44e0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f97e4007e90 tx=0x7f97e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:00.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.837+0000 7f97ebfff640 1 -- 172.21.15.70:0/3785617470 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4014620 con 0x7f97f406ad00 2023-10-08T05:57:00.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.837+0000 7f97ebfff640 1 -- 172.21.15.70:0/3785617470 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4014780 con 0x7f97f406ad00 2023-10-08T05:57:00.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.837+0000 7f97ebfff640 1 -- 172.21.15.70:0/3785617470 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4014b30 con 0x7f97f406ad00 2023-10-08T05:57:00.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.837+0000 7f97fa256640 1 -- 172.21.15.70:0/3785617470 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f4157d30 con 0x7f97f406ad00 2023-10-08T05:57:00.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.837+0000 7f97fa256640 1 -- 172.21.15.70:0/3785617470 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f40c3af0 con 0x7f97f406ad00 2023-10-08T05:57:00.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.838+0000 7f97fa256640 1 -- 172.21.15.70:0/3785617470 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f40cd5c0 con 0x7f97f406ad00 2023-10-08T05:57:00.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.839+0000 7f97ebfff640 1 -- 172.21.15.70:0/3785617470 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 75) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f97e4014c90 con 0x7f97f406ad00 2023-10-08T05:57:00.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.840+0000 7f97ebfff640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f97d406acc0 0x7f97d406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:00.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.840+0000 7f97ebfff640 1 -- 172.21.15.70:0/3785617470 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6832+0+0 (secure 0 0 0) 0x7f97e408a110 con 0x7f97f406ad00 2023-10-08T05:57:00.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.841+0000 7f97f2ffd640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f97d406acc0 0x7f97d406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:00.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.842+0000 7f97f2ffd640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f97d406acc0 0x7f97d406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f97dc006fd0 tx=0x7f97dc008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:00.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:00.845+0000 7f97ebfff640 1 -- 172.21.15.70:0/3785617470 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97e4054ff0 con 0x7f97f406ad00 2023-10-08T05:57:01.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.071+0000 7f97fa256640 1 -- 172.21.15.70:0/3785617470 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97f4067490 con 0x7f97f406ad00 2023-10-08T05:57:01.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.072+0000 7f97ebfff640 1 -- 172.21.15.70:0/3785617470 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f97e4058610 con 0x7f97f406ad00 2023-10-08T05:57:01.074 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:01.074 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1820637663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1820637663},{"type":"v1","addr":"172.21.15.181:6835","nonce":1820637663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1666874980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1666874980},{"type":"v1","addr":"172.21.15.181:6837","nonce":1666874980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2918183907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2918183907},{"type":"v1","addr":"172.21.15.70:6834","nonce":2918183907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:56:32.378187+0000","modified":"2023-10-08T05:56:35.419118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6835/3869116018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3869116018},{"type":"v1","addr":"172.21.15.70:6835","nonce":3869116018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-08T05:57:01.074 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 68 2023-10-08T05:57:01.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 -- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f97d406acc0 msgr2=0x7f97d406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f97d406acc0 0x7f97d406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f97dc006fd0 tx=0x7f97dc008040 comp rx=0 tx=0).stop 2023-10-08T05:57:01.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 -- 172.21.15.70:0/3785617470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 msgr2=0x7f97f40c44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 0x7f97f40c44e0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f97e4007e90 tx=0x7f97e400e4b0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 -- 172.21.15.70:0/3785617470 shutdown_connections 2023-10-08T05:57:01.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f97d406acc0 0x7f97d406d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f97f4071190 0x7f97f4157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f97f4070db0 0x7f97f41f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 --2- 172.21.15.70:0/3785617470 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97f406ad00 0x7f97f40c44e0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 -- 172.21.15.70:0/3785617470 >> 172.21.15.70:0/3785617470 conn(0x7f97f40b1480 msgr2=0x7f97f415aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:01.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 -- 172.21.15.70:0/3785617470 shutdown_connections 2023-10-08T05:57:01.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.075+0000 7f97fa256640 1 -- 172.21.15.70:0/3785617470 wait complete. 2023-10-08T05:57:01.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:57:01.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.305+0000 7f65503c5640 1 Processor -- start 2023-10-08T05:57:01.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.305+0000 7f65503c5640 1 -- start start 2023-10-08T05:57:01.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.307+0000 7f65503c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f654815ab40 0x7f65480ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.307+0000 7f65503c5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65480c90b0 0x7f65480adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.307+0000 7f65503c5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654815b2b0 con 0x7f65480cae70 2023-10-08T05:57:01.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.307+0000 7f65503c5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654815bf30 con 0x7f65480c90b0 2023-10-08T05:57:01.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.307+0000 7f65503c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654815f7a0 con 0x7f654815ab40 2023-10-08T05:57:01.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.308+0000 7f654d960640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65480c90b0 0x7f65480adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.308+0000 7f654e161640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f654815ab40 0x7f65480ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.308+0000 7f654d960640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65480c90b0 0x7f65480adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:51036/0 (socket says 172.21.15.70:51036) 2023-10-08T05:57:01.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.308+0000 7f654d960640 1 -- 172.21.15.70:0/1454999033 learned_addr learned my addr 172.21.15.70:0/1454999033 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:01.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.308+0000 7f654e161640 1 -- 172.21.15.70:0/1454999033 >> v2:172.21.15.181:3300/0 conn(0x7f65480c90b0 msgr2=0x7f65480adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:01.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.308+0000 7f654e161640 1 --2- 172.21.15.70:0/1454999033 >> v2:172.21.15.181:3300/0 conn(0x7f65480c90b0 0x7f65480adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.309+0000 7f654e161640 1 -- 172.21.15.70:0/1454999033 >> v1:172.21.15.181:6789/0 conn(0x7f65480cae70 legacy=0x7f65480702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.309+0000 7f654e161640 1 -- 172.21.15.70:0/1454999033 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654815b590 con 0x7f654815ab40 2023-10-08T05:57:01.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.309+0000 7f654d960640 1 --2- 172.21.15.70:0/1454999033 >> v2:172.21.15.181:3300/0 conn(0x7f65480c90b0 0x7f65480adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T05:57:01.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.310+0000 7f654e161640 1 --2- 172.21.15.70:0/1454999033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f654815ab40 0x7f65480ad720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f65440097a0 tx=0x7f654402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31fa4c4307610aee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:01.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.310+0000 7f654d15f640 1 -- 172.21.15.70:0/1454999033 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654402d040 con 0x7f654815ab40 2023-10-08T05:57:01.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.310+0000 7f654d15f640 1 -- 172.21.15.70:0/1454999033 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6544031810 con 0x7f654815ab40 2023-10-08T05:57:01.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.310+0000 7f654d15f640 1 -- 172.21.15.70:0/1454999033 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544031ba0 con 0x7f654815ab40 2023-10-08T05:57:01.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.311+0000 7f65503c5640 1 -- 172.21.15.70:0/1454999033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f654815ab40 msgr2=0x7f65480ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.311+0000 7f65503c5640 1 --2- 172.21.15.70:0/1454999033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f654815ab40 0x7f65480ad720 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f65440097a0 tx=0x7f654402b8b0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.311+0000 7f65503c5640 1 -- 172.21.15.70:0/1454999033 shutdown_connections 2023-10-08T05:57:01.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.311+0000 7f65503c5640 1 --2- 172.21.15.70:0/1454999033 >> v2:172.21.15.181:3300/0 conn(0x7f65480c90b0 0x7f65480adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.311+0000 7f65503c5640 1 --2- 172.21.15.70:0/1454999033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f654815ab40 0x7f65480ad720 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.311+0000 7f65503c5640 1 -- 172.21.15.70:0/1454999033 >> 172.21.15.70:0/1454999033 conn(0x7f654806a380 msgr2=0x7f65480b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:01.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.311+0000 7f65503c5640 1 -- 172.21.15.70:0/1454999033 shutdown_connections 2023-10-08T05:57:01.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.311+0000 7f65503c5640 1 -- 172.21.15.70:0/1454999033 wait complete. 2023-10-08T05:57:01.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.313+0000 7f65503c5640 1 Processor -- start 2023-10-08T05:57:01.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.314+0000 7f65503c5640 1 -- start start 2023-10-08T05:57:01.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.315+0000 7f65503c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65480c90b0 0x7f6548157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.315+0000 7f65503c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480cae70 0x7f6548150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.315+0000 7f65503c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654815ab40 0x7f65480c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.315+0000 7f65503c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6548151000 con 0x7f65480c90b0 2023-10-08T05:57:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.315+0000 7f65503c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65480c6d20 con 0x7f654815ab40 2023-10-08T05:57:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.315+0000 7f65503c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65480b1570 con 0x7f65480cae70 2023-10-08T05:57:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.315+0000 7f654e161640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65480c90b0 0x7f6548157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.315+0000 7f654e161640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65480c90b0 0x7f6548157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34108/0 (socket says 172.21.15.70:34108) 2023-10-08T05:57:01.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.315+0000 7f654e161640 1 -- 172.21.15.70:0/2889224883 learned_addr learned my addr 172.21.15.70:0/2889224883 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:01.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.315+0000 7f654d960640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480cae70 0x7f6548150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f654e161640 1 -- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480cae70 msgr2=0x7f6548150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f654e161640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480cae70 0x7f6548150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f654e962640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654815ab40 0x7f65480c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f654e161640 1 -- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654815ab40 msgr2=0x7f65480c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f654e161640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654815ab40 0x7f65480c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f654e161640 1 -- 172.21.15.70:0/2889224883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65380094d0 con 0x7f65480c90b0 2023-10-08T05:57:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f654e161640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65480c90b0 0x7f6548157090 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f654402b8f0 tx=0x7f65440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f653effd640 1 -- 172.21.15.70:0/2889224883 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544006e10 con 0x7f65480c90b0 2023-10-08T05:57:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f653effd640 1 -- 172.21.15.70:0/2889224883 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654403ad60 con 0x7f65480c90b0 2023-10-08T05:57:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f653effd640 1 -- 172.21.15.70:0/2889224883 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6544044610 con 0x7f65480c90b0 2023-10-08T05:57:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.316+0000 7f65503c5640 1 -- 172.21.15.70:0/2889224883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6544009400 con 0x7f65480c90b0 2023-10-08T05:57:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.317+0000 7f65503c5640 1 -- 172.21.15.70:0/2889224883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65480b1b30 con 0x7f65480c90b0 2023-10-08T05:57:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.318+0000 7f65503c5640 1 -- 172.21.15.70:0/2889224883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65480b13d0 con 0x7f65480c90b0 2023-10-08T05:57:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.318+0000 7f653effd640 1 -- 172.21.15.70:0/2889224883 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 75) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f654404d450 con 0x7f65480c90b0 2023-10-08T05:57:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.319+0000 7f653effd640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f652c06acf0 0x7f652c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.319+0000 7f653effd640 1 -- 172.21.15.70:0/2889224883 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6832+0+0 (secure 0 0 0) 0x7f65440ab900 con 0x7f65480c90b0 2023-10-08T05:57:01.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.320+0000 7f654d960640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f652c06acf0 0x7f652c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.321+0000 7f654d960640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f652c06acf0 0x7f652c06d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f65480ae060 tx=0x7f6538008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.324+0000 7f653effd640 1 -- 172.21.15.70:0/2889224883 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65480b13d0 con 0x7f65480c90b0 2023-10-08T05:57:01.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.516+0000 7f65503c5640 1 -- 172.21.15.70:0/2889224883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65480b13d0 con 0x7f65480c90b0 2023-10-08T05:57:01.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.518+0000 7f653effd640 1 -- 172.21.15.70:0/2889224883 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+17965 (secure 0 0 0) 0x7f65480b13d0 con 0x7f65480c90b0 2023-10-08T05:57:01.520 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:01.520 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":151,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:59.567035+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:56:30.775239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-08T05:56:31.378327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3419396085":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/3917000413":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/3217258198":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/145587121":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/402795356":"2023-10-09T05:56:59.566981+0000","172.21.15.181:6833/1964243521":"2023-10-09T05:56:59.566981+0000","172.21.15.181:6832/1964243521":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/413304125":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/1640710135":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/1739083188":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/202771123":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/3030439524":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/2088091464":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/4192220981":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/3672243748":"2023-10-09T05:56:46.566167+0000","172.21.15.181:6833/3857845516":"2023-10-09T05:56:46.566167+0000","172.21.15.181:6832/3857845516":"2023-10-09T05:56:46.566167+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:57:01.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.520+0000 7f65503c5640 1 -- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f652c06acf0 msgr2=0x7f652c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.520+0000 7f65503c5640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f652c06acf0 0x7f652c06d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f65480ae060 tx=0x7f6538008040 comp rx=0 tx=0).stop 2023-10-08T05:57:01.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.520+0000 7f65503c5640 1 -- 172.21.15.70:0/2889224883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65480c90b0 msgr2=0x7f6548157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.520+0000 7f65503c5640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65480c90b0 0x7f6548157090 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f654402b8f0 tx=0x7f65440046c0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.521+0000 7f65503c5640 1 -- 172.21.15.70:0/2889224883 shutdown_connections 2023-10-08T05:57:01.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.521+0000 7f65503c5640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f652c06acf0 0x7f652c06d1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.521+0000 7f65503c5640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f654815ab40 0x7f65480c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.521+0000 7f65503c5640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65480cae70 0x7f6548150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.521+0000 7f65503c5640 1 --2- 172.21.15.70:0/2889224883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65480c90b0 0x7f6548157090 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.521+0000 7f65503c5640 1 -- 172.21.15.70:0/2889224883 >> 172.21.15.70:0/2889224883 conn(0x7f654806a380 msgr2=0x7f654806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:01.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.521+0000 7f65503c5640 1 -- 172.21.15.70:0/2889224883 shutdown_connections 2023-10-08T05:57:01.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.521+0000 7f65503c5640 1 -- 172.21.15.70:0/2889224883 wait complete. 2023-10-08T05:57:01.540 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T05:57:01.540 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:57:01.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.762+0000 7fd4987ce640 1 Processor -- start 2023-10-08T05:57:01.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.762+0000 7fd4987ce640 1 -- start start 2023-10-08T05:57:01.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.764+0000 7fd4987ce640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd49015cd90 0x7fd4900702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.764+0000 7fd4987ce640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd49006c890 0x7fd49006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.764+0000 7fd4987ce640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4900c48f0 con 0x7fd490160940 2023-10-08T05:57:01.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.764+0000 7fd4987ce640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4900c4d80 con 0x7fd49015cd90 2023-10-08T05:57:01.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.764+0000 7fd4987ce640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900c3ce0 con 0x7fd49006c890 2023-10-08T05:57:01.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.764+0000 7fd495d69640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd490160940 0x7fd490070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33282/0 (socket says 172.21.15.70:33282) 2023-10-08T05:57:01.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.764+0000 7fd495d69640 1 -- 172.21.15.70:0/2515883066 learned_addr learned my addr 172.21.15.70:0/2515883066 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:01.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362770551 0 0) 0x7fd4900c48f0 con 0x7fd490160940 2023-10-08T05:57:01.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd47c003520 con 0x7fd490160940 2023-10-08T05:57:01.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd49656a640 1 --2- 172.21.15.70:0/2515883066 >> v2:172.21.15.181:3300/0 conn(0x7fd49015cd90 0x7fd4900702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd496d6b640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd49006c890 0x7fd49006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3568774565 0 0) 0x7fd47c003520 con 0x7fd490160940 2023-10-08T05:57:01.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd49006c890 msgr2=0x7fd49006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:01.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd49006c890 0x7fd49006cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 >> v2:172.21.15.181:3300/0 conn(0x7fd49015cd90 msgr2=0x7fd4900702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:01.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 --2- 172.21.15.70:0/2515883066 >> v2:172.21.15.181:3300/0 conn(0x7fd49015cd90 0x7fd4900702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4900c3fc0 con 0x7fd490160940 2023-10-08T05:57:01.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd480001670 con 0x7fd490160940 2023-10-08T05:57:01.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 >> v1:172.21.15.70:6789/0 conn(0x7fd490160940 legacy=0x7fd490070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd47c003970 0x7fd47c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd49006c890 0x7fd47c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd47c004d00 0x7fd47c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900c3ce0 con 0x7fd47c004d00 2023-10-08T05:57:01.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c00ba60 con 0x7fd47c003970 2023-10-08T05:57:01.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.765+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c00bc20 con 0x7fd49006c890 2023-10-08T05:57:01.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.766+0000 7fd496d6b640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd47c004d00 0x7fd47c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.766+0000 7fd495d69640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd49006c890 0x7fd47c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.766+0000 7fd496d6b640 1 -- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd49006c890 msgr2=0x7fd47c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:01.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.766+0000 7fd49656a640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd47c003970 0x7fd47c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.766+0000 7fd496d6b640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd49006c890 0x7fd47c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.766+0000 7fd496d6b640 1 -- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd47c003970 msgr2=0x7fd47c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:01.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.766+0000 7fd496d6b640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd47c003970 0x7fd47c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.766+0000 7fd496d6b640 1 -- 172.21.15.70:0/2515883066 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c003520 con 0x7fd47c004d00 2023-10-08T05:57:01.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.767+0000 7fd496d6b640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd47c004d00 0x7fd47c005180 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd488007360 tx=0x7fd47c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c995cc256cc7929b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:01.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.767+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48802ccd0 con 0x7fd47c004d00 2023-10-08T05:57:01.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.767+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48802ce30 con 0x7fd47c004d00 2023-10-08T05:57:01.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.767+0000 7fd495568640 1 -- 172.21.15.70:0/2515883066 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488034660 con 0x7fd47c004d00 2023-10-08T05:57:01.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.768+0000 7fd4987ce640 1 -- 172.21.15.70:0/2515883066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd47c004d00 msgr2=0x7fd47c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.768+0000 7fd4987ce640 1 --2- 172.21.15.70:0/2515883066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd47c004d00 0x7fd47c005180 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd488007360 tx=0x7fd47c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.768+0000 7fd496d6b640 1 -- 172.21.15.70:0/2515883066 reap_dead start 2023-10-08T05:57:01.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.768+0000 7fd4987ce640 1 -- 172.21.15.70:0/2515883066 shutdown_connections 2023-10-08T05:57:01.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.768+0000 7fd4987ce640 1 -- 172.21.15.70:0/2515883066 >> 172.21.15.70:0/2515883066 conn(0x7fd49006a380 msgr2=0x7fd4900b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:01.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.768+0000 7fd4987ce640 1 -- 172.21.15.70:0/2515883066 shutdown_connections 2023-10-08T05:57:01.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.768+0000 7fd4987ce640 1 -- 172.21.15.70:0/2515883066 wait complete. 2023-10-08T05:57:01.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.770+0000 7fd4987ce640 1 Processor -- start 2023-10-08T05:57:01.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.770+0000 7fd4987ce640 1 -- start start 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.771+0000 7fd4987ce640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd47c004d00 0x7fd490151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.771+0000 7fd4987ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd490153a10 0x7fd490151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.771+0000 7fd4987ce640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4900af370 0x7fd4900af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.771+0000 7fd4987ce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900afd30 con 0x7fd490153a10 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.771+0000 7fd4987ce640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd490151aa0 con 0x7fd47c004d00 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.771+0000 7fd4987ce640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd490152280 con 0x7fd4900af370 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.771+0000 7fd495d69640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd490153a10 0x7fd490151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.771+0000 7fd495d69640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd490153a10 0x7fd490151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34122/0 (socket says 172.21.15.70:34122) 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.771+0000 7fd495d69640 1 -- 172.21.15.70:0/2531340771 learned_addr learned my addr 172.21.15.70:0/2531340771 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd49656a640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd47c004d00 0x7fd490151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd496d6b640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4900af370 0x7fd4900af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd495d69640 1 -- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4900af370 msgr2=0x7fd4900af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd495d69640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4900af370 0x7fd4900af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd495d69640 1 -- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd47c004d00 msgr2=0x7fd490151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd495d69640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd47c004d00 0x7fd490151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd495d69640 1 -- 172.21.15.70:0/2531340771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd490152560 con 0x7fd490153a10 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd495d69640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd490153a10 0x7fd490151560 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd48000d810 tx=0x7fd48000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd486ffd640 1 -- 172.21.15.70:0/2531340771 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4800074e0 con 0x7fd490153a10 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd486ffd640 1 -- 172.21.15.70:0/2531340771 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd480007640 con 0x7fd490153a10 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd486ffd640 1 -- 172.21.15.70:0/2531340771 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480004a00 con 0x7fd490153a10 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.772+0000 7fd4987ce640 1 -- 172.21.15.70:0/2531340771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd490152840 con 0x7fd490153a10 2023-10-08T05:57:01.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.773+0000 7fd4987ce640 1 -- 172.21.15.70:0/2531340771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49015e500 con 0x7fd490153a10 2023-10-08T05:57:01.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.774+0000 7fd4987ce640 1 -- 172.21.15.70:0/2531340771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd490067490 con 0x7fd490153a10 2023-10-08T05:57:01.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.775+0000 7fd486ffd640 1 -- 172.21.15.70:0/2531340771 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 76) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fd480004b60 con 0x7fd490153a10 2023-10-08T05:57:01.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.776+0000 7fd486ffd640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd47406af30 0x7fd47406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:01.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.776+0000 7fd486ffd640 1 -- 172.21.15.70:0/2531340771 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6832+0+0 (secure 0 0 0) 0x7fd4800895a0 con 0x7fd490153a10 2023-10-08T05:57:01.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.776+0000 7fd49656a640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd47406af30 0x7fd47406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:01.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.777+0000 7fd49656a640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd47406af30 0x7fd47406d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd4900ca860 tx=0x7fd48c008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:01.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:01.780+0000 7fd486ffd640 1 -- 172.21.15.70:0/2531340771 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd480054010 con 0x7fd490153a10 2023-10-08T05:57:02.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.014+0000 7fd4987ce640 1 -- 172.21.15.70:0/2531340771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4900b0ef0 con 0x7fd490153a10 2023-10-08T05:57:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.015+0000 7fd486ffd640 1 -- 172.21.15.70:0/2531340771 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fd480057630 con 0x7fd490153a10 2023-10-08T05:57:02.017 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:02.017 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1820637663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1820637663},{"type":"v1","addr":"172.21.15.181:6835","nonce":1820637663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1666874980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1666874980},{"type":"v1","addr":"172.21.15.181:6837","nonce":1666874980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2918183907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2918183907},{"type":"v1","addr":"172.21.15.70:6834","nonce":2918183907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:56:32.378187+0000","modified":"2023-10-08T05:56:35.419118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6835/3869116018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3869116018},{"type":"v1","addr":"172.21.15.70:6835","nonce":3869116018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-08T05:57:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 68 2023-10-08T05:57:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 -- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd47406af30 msgr2=0x7fd47406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd47406af30 0x7fd47406d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd4900ca860 tx=0x7fd48c008040 comp rx=0 tx=0).stop 2023-10-08T05:57:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 -- 172.21.15.70:0/2531340771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd490153a10 msgr2=0x7fd490151560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd490153a10 0x7fd490151560 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fd48000d810 tx=0x7fd48000dcd0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 -- 172.21.15.70:0/2531340771 shutdown_connections 2023-10-08T05:57:02.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd47406af30 0x7fd47406d3f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4900af370 0x7fd4900af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd490153a10 0x7fd490151560 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 --2- 172.21.15.70:0/2531340771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd47c004d00 0x7fd490151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 -- 172.21.15.70:0/2531340771 >> 172.21.15.70:0/2531340771 conn(0x7fd49006a380 msgr2=0x7fd490157870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:02.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 -- 172.21.15.70:0/2531340771 shutdown_connections 2023-10-08T05:57:02.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.018+0000 7fd4987ce640 1 -- 172.21.15.70:0/2531340771 wait complete. 2023-10-08T05:57:02.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:57:02.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.248+0000 7f7982d7c640 1 Processor -- start 2023-10-08T05:57:02.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.248+0000 7f7982d7c640 1 -- start start 2023-10-08T05:57:02.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.250+0000 7f7982d7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159ac0 0x7f797c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.250+0000 7f7982d7c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f797c0ad790 0x7f797c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.250+0000 7f7982d7c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c06dee0 con 0x7f797c15e330 2023-10-08T05:57:02.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.250+0000 7f7982d7c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797c15d850 con 0x7f797c0ad790 2023-10-08T05:57:02.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.250+0000 7f7982d7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c15db30 con 0x7f797c159ac0 2023-10-08T05:57:02.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.250+0000 7f7981319640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f797c15e330 0x7f797c06c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33288/0 (socket says 172.21.15.70:33288) 2023-10-08T05:57:02.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.250+0000 7f7981319640 1 -- 172.21.15.70:0/2419704150 learned_addr learned my addr 172.21.15.70:0/2419704150 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:02.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f7973fff640 1 --2- 172.21.15.70:0/2419704150 >> v2:172.21.15.181:3300/0 conn(0x7f797c0ad790 0x7f797c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f7980b18640 1 --2- 172.21.15.70:0/2419704150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159ac0 0x7f797c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635348222 0 0) 0x7f797c06dee0 con 0x7f797c15e330 2023-10-08T05:57:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7964003520 con 0x7f797c15e330 2023-10-08T05:57:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 696583736 0 0) 0x7f7964003520 con 0x7f797c15e330 2023-10-08T05:57:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159ac0 msgr2=0x7f797c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 --2- 172.21.15.70:0/2419704150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159ac0 0x7f797c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 >> v2:172.21.15.181:3300/0 conn(0x7f797c0ad790 msgr2=0x7f797c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 --2- 172.21.15.70:0/2419704150 >> v2:172.21.15.181:3300/0 conn(0x7f797c0ad790 0x7f797c0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c15de10 con 0x7f797c15e330 2023-10-08T05:57:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f79740019f0 con 0x7f797c15e330 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 >> v1:172.21.15.70:6789/0 conn(0x7f797c15e330 legacy=0x7f797c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 --2- 172.21.15.70:0/2419704150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7964003970 0x7f7964003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 --2- 172.21.15.70:0/2419704150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79640047c0 0x7f7964004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 --2- 172.21.15.70:0/2419704150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159ac0 0x7f7964005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c15d850 con 0x7f7964003970 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c15db30 con 0x7f79640047c0 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.251+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796400b980 con 0x7f797c159ac0 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.252+0000 7f7980b18640 1 --2- 172.21.15.70:0/2419704150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7964003970 0x7f7964003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.252+0000 7f7980b18640 1 -- 172.21.15.70:0/2419704150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159ac0 msgr2=0x7f7964005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.252+0000 7f7980b18640 1 --2- 172.21.15.70:0/2419704150 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f797c159ac0 0x7f7964005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.252+0000 7f7980b18640 1 -- 172.21.15.70:0/2419704150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79640047c0 msgr2=0x7f7964004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:02.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.252+0000 7f7980b18640 1 --2- 172.21.15.70:0/2419704150 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79640047c0 0x7f7964004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.252+0000 7f7980b18640 1 -- 172.21.15.70:0/2419704150 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7964003520 con 0x7f7964003970 2023-10-08T05:57:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.253+0000 7f7980b18640 1 --2- 172.21.15.70:0/2419704150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7964003970 0x7f7964003d90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7978009ac0 tx=0x7f79640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4702c46a715095b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.253+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797802c710 con 0x7f7964003970 2023-10-08T05:57:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.253+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797802c870 con 0x7f7964003970 2023-10-08T05:57:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.253+0000 7f79737fe640 1 -- 172.21.15.70:0/2419704150 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797802cc00 con 0x7f7964003970 2023-10-08T05:57:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.254+0000 7f7982d7c640 1 -- 172.21.15.70:0/2419704150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7964003970 msgr2=0x7f7964003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.254+0000 7f7982d7c640 1 --2- 172.21.15.70:0/2419704150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7964003970 0x7f7964003d90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7978009ac0 tx=0x7f79640057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.254+0000 7f7981319640 1 -- 172.21.15.70:0/2419704150 reap_dead start 2023-10-08T05:57:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.254+0000 7f7982d7c640 1 -- 172.21.15.70:0/2419704150 shutdown_connections 2023-10-08T05:57:02.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.254+0000 7f7982d7c640 1 -- 172.21.15.70:0/2419704150 >> 172.21.15.70:0/2419704150 conn(0x7f797c06a380 msgr2=0x7f797c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:02.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.254+0000 7f7982d7c640 1 -- 172.21.15.70:0/2419704150 shutdown_connections 2023-10-08T05:57:02.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.254+0000 7f7982d7c640 1 -- 172.21.15.70:0/2419704150 wait complete. 2023-10-08T05:57:02.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.256+0000 7f7982d7c640 1 Processor -- start 2023-10-08T05:57:02.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.256+0000 7f7982d7c640 1 -- start start 2023-10-08T05:57:02.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.257+0000 7f7982d7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7964003970 0x7f797c15a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.257+0000 7f7982d7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c154480 0x7f797c14bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.257+0000 7f7982d7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f797c14c2b0 0x7f797c14a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.257+0000 7f7982d7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c14c730 con 0x7f797c14c2b0 2023-10-08T05:57:02.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.257+0000 7f7982d7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c15af50 con 0x7f797c154480 2023-10-08T05:57:02.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.257+0000 7f7982d7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c15a6b0 con 0x7f7964003970 2023-10-08T05:57:02.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.257+0000 7f7981319640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f797c14c2b0 0x7f797c14a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.257+0000 7f7981319640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f797c14c2b0 0x7f797c14a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34130/0 (socket says 172.21.15.70:34130) 2023-10-08T05:57:02.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.257+0000 7f7981319640 1 -- 172.21.15.70:0/1166790535 learned_addr learned my addr 172.21.15.70:0/1166790535 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7980b18640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7964003970 0x7f797c15a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7973fff640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c154480 0x7f797c14bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7981319640 1 -- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7964003970 msgr2=0x7f797c15a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7981319640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7964003970 0x7f797c15a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7981319640 1 -- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c154480 msgr2=0x7f797c14bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7981319640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c154480 0x7f797c14bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7981319640 1 -- 172.21.15.70:0/1166790535 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c15a990 con 0x7f797c14c2b0 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7980b18640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7964003970 0x7f797c15a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7973fff640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c154480 0x7f797c14bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7981319640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f797c14c2b0 0x7f797c14a560 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f79740012e0 tx=0x7f797400cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f79717fa640 1 -- 172.21.15.70:0/1166790535 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400a610 con 0x7f797c14c2b0 2023-10-08T05:57:02.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f79717fa640 1 -- 172.21.15.70:0/1166790535 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797400a770 con 0x7f797c14c2b0 2023-10-08T05:57:02.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f79717fa640 1 -- 172.21.15.70:0/1166790535 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400ab20 con 0x7f797c14c2b0 2023-10-08T05:57:02.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7982d7c640 1 -- 172.21.15.70:0/1166790535 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c15ac70 con 0x7f797c14c2b0 2023-10-08T05:57:02.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.258+0000 7f7982d7c640 1 -- 172.21.15.70:0/1166790535 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c154900 con 0x7f797c14c2b0 2023-10-08T05:57:02.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.260+0000 7f7982d7c640 1 -- 172.21.15.70:0/1166790535 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c067490 con 0x7f797c14c2b0 2023-10-08T05:57:02.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.261+0000 7f79717fa640 1 -- 172.21.15.70:0/1166790535 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 76) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f797400ac80 con 0x7f797c14c2b0 2023-10-08T05:57:02.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.261+0000 7f79717fa640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f796806af30 0x7f796806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.262+0000 7f79717fa640 1 -- 172.21.15.70:0/1166790535 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6832+0+0 (secure 0 0 0) 0x7f7974089460 con 0x7f797c14c2b0 2023-10-08T05:57:02.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.262+0000 7f7980b18640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f796806af30 0x7f796806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.263+0000 7f7980b18640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f796806af30 0x7f796806d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7978036000 tx=0x7f7978030000 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:02.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.266+0000 7f79717fa640 1 -- 172.21.15.70:0/1166790535 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7974054070 con 0x7f797c14c2b0 2023-10-08T05:57:02.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.492+0000 7f7982d7c640 1 -- 172.21.15.70:0/1166790535 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f797c0b0ff0 con 0x7f797c14c2b0 2023-10-08T05:57:02.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.493+0000 7f79717fa640 1 -- 172.21.15.70:0/1166790535 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f7974057690 con 0x7f797c14c2b0 2023-10-08T05:57:02.495 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:02.495 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":68,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1820637663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1820637663},{"type":"v1","addr":"172.21.15.181:6835","nonce":1820637663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1666874980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1666874980},{"type":"v1","addr":"172.21.15.181:6837","nonce":1666874980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2918183907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2918183907},{"type":"v1","addr":"172.21.15.70:6834","nonce":2918183907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:56:32.378187+0000","modified":"2023-10-08T05:56:35.419118+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8526":{"gid":8526,"name":"c","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6835/3869116018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":3869116018},{"type":"v1","addr":"172.21.15.70:6835","nonce":3869116018}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-10-08T05:57:02.495 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 68 2023-10-08T05:57:02.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 -- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f796806af30 msgr2=0x7f796806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f796806af30 0x7f796806d3f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7978036000 tx=0x7f7978030000 comp rx=0 tx=0).stop 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 -- 172.21.15.70:0/1166790535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f797c14c2b0 msgr2=0x7f797c14a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f797c14c2b0 0x7f797c14a560 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f79740012e0 tx=0x7f797400cec0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 -- 172.21.15.70:0/1166790535 shutdown_connections 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f796806af30 0x7f796806d3f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f797c14c2b0 0x7f797c14a560 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f797c154480 0x7f797c14bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 --2- 172.21.15.70:0/1166790535 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7964003970 0x7f797c15a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 -- 172.21.15.70:0/1166790535 >> 172.21.15.70:0/1166790535 conn(0x7f797c06a380 msgr2=0x7f797c0ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 -- 172.21.15.70:0/1166790535 shutdown_connections 2023-10-08T05:57:02.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.496+0000 7f7982d7c640 1 -- 172.21.15.70:0/1166790535 wait complete. 2023-10-08T05:57:02.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:57:02.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.728+0000 7fbf4b261640 1 Processor -- start 2023-10-08T05:57:02.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.728+0000 7fbf4b261640 1 -- start start 2023-10-08T05:57:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.730+0000 7fbf4b261640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf440add60 0x7fbf44162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.730+0000 7fbf4b261640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbf4406d4c0 0x7fbf440cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.730+0000 7fbf4b261640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 0x7fbf440cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.730+0000 7fbf4b261640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf44161f10 con 0x7fbf440add60 2023-10-08T05:57:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.730+0000 7fbf4b261640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf44161630 con 0x7fbf4406d4c0 2023-10-08T05:57:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.730+0000 7fbf4b261640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf440cbaf0 con 0x7fbf440cb150 2023-10-08T05:57:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.730+0000 7fbf48ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf440add60 0x7fbf44162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.731+0000 7fbf48ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf440add60 0x7fbf44162330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34134/0 (socket says 172.21.15.70:34134) 2023-10-08T05:57:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.731+0000 7fbf48ffd640 1 -- 172.21.15.70:0/1498776556 learned_addr learned my addr 172.21.15.70:0/1498776556 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:02.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.731+0000 7fbf43fff640 1 --2- 172.21.15.70:0/1498776556 >> v2:172.21.15.181:3300/0 conn(0x7fbf4406d4c0 0x7fbf440cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.731+0000 7fbf497fe640 1 --2- 172.21.15.70:0/1498776556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 0x7fbf440cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.731+0000 7fbf48ffd640 1 -- 172.21.15.70:0/1498776556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 msgr2=0x7fbf440cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:02.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.731+0000 7fbf48ffd640 1 --2- 172.21.15.70:0/1498776556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 0x7fbf440cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.731+0000 7fbf48ffd640 1 -- 172.21.15.70:0/1498776556 >> v2:172.21.15.181:3300/0 conn(0x7fbf4406d4c0 msgr2=0x7fbf440cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:02.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.731+0000 7fbf48ffd640 1 --2- 172.21.15.70:0/1498776556 >> v2:172.21.15.181:3300/0 conn(0x7fbf4406d4c0 0x7fbf440cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.731+0000 7fbf48ffd640 1 -- 172.21.15.70:0/1498776556 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf440ae140 con 0x7fbf440add60 2023-10-08T05:57:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.732+0000 7fbf48ffd640 1 --2- 172.21.15.70:0/1498776556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 0x7fbf44162330 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbf4406e560 tx=0x7fbf3802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d894d47cc3c8ec3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.733+0000 7fbf437fe640 1 -- 172.21.15.70:0/1498776556 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf380317c0 con 0x7fbf440add60 2023-10-08T05:57:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.733+0000 7fbf437fe640 1 -- 172.21.15.70:0/1498776556 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf38031920 con 0x7fbf440add60 2023-10-08T05:57:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.733+0000 7fbf437fe640 1 -- 172.21.15.70:0/1498776556 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38031cb0 con 0x7fbf440add60 2023-10-08T05:57:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.733+0000 7fbf4b261640 1 -- 172.21.15.70:0/1498776556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 msgr2=0x7fbf44162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.733+0000 7fbf4b261640 1 --2- 172.21.15.70:0/1498776556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 0x7fbf44162330 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbf4406e560 tx=0x7fbf3802b860 comp rx=0 tx=0).stop 2023-10-08T05:57:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.734+0000 7fbf4b261640 1 -- 172.21.15.70:0/1498776556 shutdown_connections 2023-10-08T05:57:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.734+0000 7fbf4b261640 1 --2- 172.21.15.70:0/1498776556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 0x7fbf440cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.734+0000 7fbf4b261640 1 --2- 172.21.15.70:0/1498776556 >> v2:172.21.15.181:3300/0 conn(0x7fbf4406d4c0 0x7fbf440cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.734+0000 7fbf4b261640 1 --2- 172.21.15.70:0/1498776556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 0x7fbf44162330 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.734+0000 7fbf4b261640 1 -- 172.21.15.70:0/1498776556 >> 172.21.15.70:0/1498776556 conn(0x7fbf4406a380 msgr2=0x7fbf440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.734+0000 7fbf4b261640 1 -- 172.21.15.70:0/1498776556 shutdown_connections 2023-10-08T05:57:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.734+0000 7fbf4b261640 1 -- 172.21.15.70:0/1498776556 wait complete. 2023-10-08T05:57:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.736+0000 7fbf4b261640 1 Processor -- start 2023-10-08T05:57:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.736+0000 7fbf4b261640 1 -- start start 2023-10-08T05:57:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.737+0000 7fbf4b261640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf4406d4c0 0x7fbf44156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.737+0000 7fbf4b261640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 0x7fbf441fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.737+0000 7fbf4b261640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 0x7fbf440c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.737+0000 7fbf4b261640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf440b16b0 con 0x7fbf440add60 2023-10-08T05:57:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.737+0000 7fbf4b261640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf440b2f90 con 0x7fbf4406d4c0 2023-10-08T05:57:02.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.737+0000 7fbf4b261640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf440b2c90 con 0x7fbf440cb150 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf43fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 0x7fbf441fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf43fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 0x7fbf441fa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34146/0 (socket says 172.21.15.70:34146) 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf43fff640 1 -- 172.21.15.70:0/4115153672 learned_addr learned my addr 172.21.15.70:0/4115153672 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf48ffd640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf4406d4c0 0x7fbf44156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf497fe640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 0x7fbf440c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf43fff640 1 -- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 msgr2=0x7fbf440c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf43fff640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 0x7fbf440c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf43fff640 1 -- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf4406d4c0 msgr2=0x7fbf44156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf43fff640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf4406d4c0 0x7fbf44156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf43fff640 1 -- 172.21.15.70:0/4115153672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf380094d0 con 0x7fbf440add60 2023-10-08T05:57:02.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf497fe640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 0x7fbf440c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.738+0000 7fbf43fff640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 0x7fbf441fa8b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d830 tx=0x7fbf3000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.739+0000 7fbf417fa640 1 -- 172.21.15.70:0/4115153672 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300096a0 con 0x7fbf440add60 2023-10-08T05:57:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.739+0000 7fbf417fa640 1 -- 172.21.15.70:0/4115153672 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf30009800 con 0x7fbf440add60 2023-10-08T05:57:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.739+0000 7fbf417fa640 1 -- 172.21.15.70:0/4115153672 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30009b90 con 0x7fbf440add60 2023-10-08T05:57:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.739+0000 7fbf4b261640 1 -- 172.21.15.70:0/4115153672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf44157040 con 0x7fbf440add60 2023-10-08T05:57:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.739+0000 7fbf4b261640 1 -- 172.21.15.70:0/4115153672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf44158e20 con 0x7fbf440add60 2023-10-08T05:57:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.740+0000 7fbf4b261640 1 -- 172.21.15.70:0/4115153672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf440b3140 con 0x7fbf440add60 2023-10-08T05:57:02.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.741+0000 7fbf417fa640 1 -- 172.21.15.70:0/4115153672 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 76) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fbf30014410 con 0x7fbf440add60 2023-10-08T05:57:02.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.742+0000 7fbf417fa640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbf2806af30 0x7fbf2806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:02.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.742+0000 7fbf417fa640 1 -- 172.21.15.70:0/4115153672 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6832+0+0 (secure 0 0 0) 0x7fbf3008a120 con 0x7fbf440add60 2023-10-08T05:57:02.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.743+0000 7fbf48ffd640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbf2806af30 0x7fbf2806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:02.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.744+0000 7fbf48ffd640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbf2806af30 0x7fbf2806d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbf38036000 tx=0x7fbf3803d040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:02.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.747+0000 7fbf417fa640 1 -- 172.21.15.70:0/4115153672 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf440b3140 con 0x7fbf440add60 2023-10-08T05:57:02.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.937+0000 7fbf4b261640 1 -- 172.21.15.70:0/4115153672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf440b3140 con 0x7fbf440add60 2023-10-08T05:57:02.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.939+0000 7fbf417fa640 1 -- 172.21.15.70:0/4115153672 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+17965 (secure 0 0 0) 0x7fbf440b3140 con 0x7fbf440add60 2023-10-08T05:57:02.940 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:02.941 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":151,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:56:59.567035+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:56:30.775239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-08T05:56:31.378327+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3419396085":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/3917000413":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/3217258198":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/145587121":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/402795356":"2023-10-09T05:56:59.566981+0000","172.21.15.181:6833/1964243521":"2023-10-09T05:56:59.566981+0000","172.21.15.181:6832/1964243521":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/413304125":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/1640710135":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/1739083188":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/202771123":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/3030439524":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/2088091464":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/4192220981":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/3672243748":"2023-10-09T05:56:46.566167+0000","172.21.15.181:6833/3857845516":"2023-10-09T05:56:46.566167+0000","172.21.15.181:6832/3857845516":"2023-10-09T05:56:46.566167+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:57:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 -- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbf2806af30 msgr2=0x7fbf2806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbf2806af30 0x7fbf2806d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbf38036000 tx=0x7fbf3803d040 comp rx=0 tx=0).stop 2023-10-08T05:57:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 -- 172.21.15.70:0/4115153672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 msgr2=0x7fbf441fa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 0x7fbf441fa8b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000d830 tx=0x7fbf3000dcf0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 -- 172.21.15.70:0/4115153672 shutdown_connections 2023-10-08T05:57:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbf2806af30 0x7fbf2806d3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbf440cb150 0x7fbf440c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf440add60 0x7fbf441fa8b0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 --2- 172.21.15.70:0/4115153672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbf4406d4c0 0x7fbf44156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 -- 172.21.15.70:0/4115153672 >> 172.21.15.70:0/4115153672 conn(0x7fbf4406a380 msgr2=0x7fbf440ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:02.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.941+0000 7fbf4b261640 1 -- 172.21.15.70:0/4115153672 shutdown_connections 2023-10-08T05:57:02.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:02.942+0000 7fbf4b261640 1 -- 172.21.15.70:0/4115153672 wait complete. 2023-10-08T05:57:02.961 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T05:57:03.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.172+0000 7fef2214e640 1 Processor -- start 2023-10-08T05:57:03.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.173+0000 7fef2214e640 1 -- start start 2023-10-08T05:57:03.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.174+0000 7fef2214e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c0c9cf0 0x7fef1c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.174+0000 7fef2214e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fef1c15aa00 0x7fef1c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.174+0000 7fef2214e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1c15b310 con 0x7fef1c06c3a0 2023-10-08T05:57:03.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.174+0000 7fef2214e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1c06e0a0 con 0x7fef1c15aa00 2023-10-08T05:57:03.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.174+0000 7fef2214e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c06d290 con 0x7fef1c0c9cf0 2023-10-08T05:57:03.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.174+0000 7fef1bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fef1c15aa00 0x7fef1c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.174+0000 7fef1bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fef1c15aa00 0x7fef1c0b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34150/0 (socket says 172.21.15.70:34150) 2023-10-08T05:57:03.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.174+0000 7fef1bfff640 1 -- 172.21.15.70:0/1582389070 learned_addr learned my addr 172.21.15.70:0/1582389070 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.175+0000 7fef1affd640 1 --2- 172.21.15.70:0/1582389070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c0c9cf0 0x7fef1c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.175+0000 7fef1bfff640 1 -- 172.21.15.70:0/1582389070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c0c9cf0 msgr2=0x7fef1c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.175+0000 7fef1bfff640 1 --2- 172.21.15.70:0/1582389070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c0c9cf0 0x7fef1c06db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.175+0000 7fef1bfff640 1 -- 172.21.15.70:0/1582389070 >> v1:172.21.15.181:6789/0 conn(0x7fef1c06c3a0 legacy=0x7fef1c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.175+0000 7fef1bfff640 1 -- 172.21.15.70:0/1582389070 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c0c9860 con 0x7fef1c15aa00 2023-10-08T05:57:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.175+0000 7fef1affd640 1 --2- 172.21.15.70:0/1582389070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c0c9cf0 0x7fef1c06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:03.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.176+0000 7fef1bfff640 1 --2- 172.21.15.70:0/1582389070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c15aa00 0x7fef1c0b2a90 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fef1400bb10 tx=0x7fef1402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9953f2c54781a669 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.176+0000 7fef1a7fc640 1 -- 172.21.15.70:0/1582389070 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1402ee70 con 0x7fef1c15aa00 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.176+0000 7fef1a7fc640 1 -- 172.21.15.70:0/1582389070 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef14036440 con 0x7fef1c15aa00 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.176+0000 7fef1a7fc640 1 -- 172.21.15.70:0/1582389070 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef140367d0 con 0x7fef1c15aa00 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.177+0000 7fef2214e640 1 -- 172.21.15.70:0/1582389070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c15aa00 msgr2=0x7fef1c0b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.177+0000 7fef2214e640 1 --2- 172.21.15.70:0/1582389070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c15aa00 0x7fef1c0b2a90 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fef1400bb10 tx=0x7fef1402e0c0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.177+0000 7fef2214e640 1 -- 172.21.15.70:0/1582389070 shutdown_connections 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.177+0000 7fef2214e640 1 --2- 172.21.15.70:0/1582389070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c15aa00 0x7fef1c0b2a90 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.177+0000 7fef2214e640 1 --2- 172.21.15.70:0/1582389070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c0c9cf0 0x7fef1c06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.177+0000 7fef2214e640 1 -- 172.21.15.70:0/1582389070 >> 172.21.15.70:0/1582389070 conn(0x7fef1c06a5d0 msgr2=0x7fef1c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.177+0000 7fef2214e640 1 -- 172.21.15.70:0/1582389070 shutdown_connections 2023-10-08T05:57:03.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.177+0000 7fef2214e640 1 -- 172.21.15.70:0/1582389070 wait complete. 2023-10-08T05:57:03.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.179+0000 7fef2214e640 1 Processor -- start 2023-10-08T05:57:03.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.179+0000 7fef2214e640 1 -- start start 2023-10-08T05:57:03.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.180+0000 7fef2214e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c06c3a0 0x7fef1c1f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.180+0000 7fef2214e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef1c0c9cf0 0x7fef1c0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.180+0000 7fef2214e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c15aa00 0x7fef1c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.180+0000 7fef2214e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c1f3520 con 0x7fef1c06c3a0 2023-10-08T05:57:03.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.180+0000 7fef2214e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c1f5ba0 con 0x7fef1c0c9cf0 2023-10-08T05:57:03.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.180+0000 7fef2214e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c1f2910 con 0x7fef1c15aa00 2023-10-08T05:57:03.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.180+0000 7fef1b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c06c3a0 0x7fef1c1f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.180+0000 7fef1b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c06c3a0 0x7fef1c1f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34156/0 (socket says 172.21.15.70:34156) 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.180+0000 7fef1b7fe640 1 -- 172.21.15.70:0/4029474615 learned_addr learned my addr 172.21.15.70:0/4029474615 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef1affd640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef1c0c9cf0 0x7fef1c0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef1bfff640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c15aa00 0x7fef1c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef1b7fe640 1 -- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c15aa00 msgr2=0x7fef1c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef1b7fe640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c15aa00 0x7fef1c0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef1b7fe640 1 -- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef1c0c9cf0 msgr2=0x7fef1c0ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef1b7fe640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef1c0c9cf0 0x7fef1c0ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef1b7fe640 1 -- 172.21.15.70:0/4029474615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1400b770 con 0x7fef1c06c3a0 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef1b7fe640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c06c3a0 0x7fef1c1f5660 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00c7c0 tx=0x7fef0c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef13fff640 1 -- 172.21.15.70:0/4029474615 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00aa50 con 0x7fef1c06c3a0 2023-10-08T05:57:03.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef13fff640 1 -- 172.21.15.70:0/4029474615 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0c00abb0 con 0x7fef1c06c3a0 2023-10-08T05:57:03.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef13fff640 1 -- 172.21.15.70:0/4029474615 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c012420 con 0x7fef1c06c3a0 2023-10-08T05:57:03.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.181+0000 7fef2214e640 1 -- 172.21.15.70:0/4029474615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c1f2bf0 con 0x7fef1c06c3a0 2023-10-08T05:57:03.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.182+0000 7fef2214e640 1 -- 172.21.15.70:0/4029474615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c1f31b0 con 0x7fef1c06c3a0 2023-10-08T05:57:03.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.183+0000 7fef2214e640 1 -- 172.21.15.70:0/4029474615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c1f3340 con 0x7fef1c06c3a0 2023-10-08T05:57:03.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.184+0000 7fef13fff640 1 -- 172.21.15.70:0/4029474615 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 76) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fef0c012580 con 0x7fef1c06c3a0 2023-10-08T05:57:03.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.185+0000 7fef13fff640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feefc06af00 0x7feefc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.185+0000 7fef13fff640 1 -- 172.21.15.70:0/4029474615 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6832+0+0 (secure 0 0 0) 0x7fef0c089e20 con 0x7fef1c06c3a0 2023-10-08T05:57:03.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.185+0000 7fef1affd640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feefc06af00 0x7feefc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.186+0000 7fef1affd640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feefc06af00 0x7feefc06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef1c159710 tx=0x7fef04008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:03.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.189+0000 7fef13fff640 1 -- 172.21.15.70:0/4029474615 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef0c0549a0 con 0x7fef1c06c3a0 2023-10-08T05:57:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.416+0000 7fef2214e640 1 -- 172.21.15.70:0/4029474615 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fef1c04e160 con 0x7fef1c06c3a0 2023-10-08T05:57:03.418 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T05:57:03.417+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T05:57:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.420+0000 7fef13fff640 1 -- 172.21.15.70:0/4029474615 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7fef0c057fc0 con 0x7fef1c06c3a0 2023-10-08T05:57:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T05:57:03.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 -- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feefc06af00 msgr2=0x7feefc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:03.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feefc06af00 0x7feefc06d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef1c159710 tx=0x7fef04008040 comp rx=0 tx=0).stop 2023-10-08T05:57:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 -- 172.21.15.70:0/4029474615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c06c3a0 msgr2=0x7fef1c1f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c06c3a0 0x7fef1c1f5660 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00c7c0 tx=0x7fef0c00cc80 comp rx=0 tx=0).stop 2023-10-08T05:57:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 -- 172.21.15.70:0/4029474615 shutdown_connections 2023-10-08T05:57:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feefc06af00 0x7feefc06d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef1c15aa00 0x7fef1c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef1c0c9cf0 0x7fef1c0ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 --2- 172.21.15.70:0/4029474615 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef1c06c3a0 0x7fef1c1f5660 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 -- 172.21.15.70:0/4029474615 >> 172.21.15.70:0/4029474615 conn(0x7fef1c06a5d0 msgr2=0x7fef1c06c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 -- 172.21.15.70:0/4029474615 shutdown_connections 2023-10-08T05:57:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.423+0000 7fef2214e640 1 -- 172.21.15.70:0/4029474615 wait complete. 2023-10-08T05:57:03.438 INFO:tasks.ceph.mds.c.smithi070.stderr: -8530> 2023-10-08T05:56:37.735+0000 7fc553112640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:57:03.445 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T05:57:03.495 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:57:03.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.635+0000 7f81c27ce640 1 Processor -- start 2023-10-08T05:57:03.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.635+0000 7f81c27ce640 1 -- start start 2023-10-08T05:57:03.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.636+0000 7f81c27ce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81bc06c3a0 0x7f81bc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.636+0000 7f81c27ce640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15aa00 0x7f81bc0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.636+0000 7f81c27ce640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81bc15b310 con 0x7f81bc06db60 2023-10-08T05:57:03.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.636+0000 7f81c27ce640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81bc0ca230 con 0x7f81bc06c3a0 2023-10-08T05:57:03.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.636+0000 7f81c27ce640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc06d290 con 0x7f81bc15aa00 2023-10-08T05:57:03.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.636+0000 7f81bbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81bc06c3a0 0x7f81bc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.636+0000 7f81b3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f81bc06db60 0x7f81bc06c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46294/0 (socket says 172.21.15.70:46294) 2023-10-08T05:57:03.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.636+0000 7f81b3fff640 1 -- 172.21.15.70:0/1009957390 learned_addr learned my addr 172.21.15.70:0/1009957390 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:03.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.637+0000 7f81c0d6b640 1 --2- 172.21.15.70:0/1009957390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15aa00 0x7f81bc0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.637+0000 7f81bbfff640 1 -- 172.21.15.70:0/1009957390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15aa00 msgr2=0x7f81bc0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:03.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.637+0000 7f81bbfff640 1 --2- 172.21.15.70:0/1009957390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15aa00 0x7f81bc0b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.637+0000 7f81bbfff640 1 -- 172.21.15.70:0/1009957390 >> v1:172.21.15.70:6789/0 conn(0x7f81bc06db60 legacy=0x7f81bc06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:03.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.637+0000 7f81bbfff640 1 -- 172.21.15.70:0/1009957390 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc0c9860 con 0x7f81bc06c3a0 2023-10-08T05:57:03.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.638+0000 7f81bbfff640 1 --2- 172.21.15.70:0/1009957390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06c3a0 0x7f81bc0c9cf0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0097a0 tx=0x7f81ac02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47008cbdce669c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:03.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.638+0000 7f81bb7fe640 1 -- 172.21.15.70:0/1009957390 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac02d040 con 0x7f81bc06c3a0 2023-10-08T05:57:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.638+0000 7f81bb7fe640 1 -- 172.21.15.70:0/1009957390 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac031810 con 0x7f81bc06c3a0 2023-10-08T05:57:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.638+0000 7f81bb7fe640 1 -- 172.21.15.70:0/1009957390 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac031ba0 con 0x7f81bc06c3a0 2023-10-08T05:57:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.639+0000 7f81c27ce640 1 -- 172.21.15.70:0/1009957390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06c3a0 msgr2=0x7f81bc0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.639+0000 7f81c27ce640 1 --2- 172.21.15.70:0/1009957390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06c3a0 0x7f81bc0c9cf0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f81ac0097a0 tx=0x7f81ac02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.639+0000 7f81c27ce640 1 -- 172.21.15.70:0/1009957390 shutdown_connections 2023-10-08T05:57:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.639+0000 7f81c27ce640 1 --2- 172.21.15.70:0/1009957390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15aa00 0x7f81bc0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.639+0000 7f81c27ce640 1 --2- 172.21.15.70:0/1009957390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06c3a0 0x7f81bc0c9cf0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.639+0000 7f81c27ce640 1 -- 172.21.15.70:0/1009957390 >> 172.21.15.70:0/1009957390 conn(0x7f81bc06a5d0 msgr2=0x7f81bc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.639+0000 7f81c27ce640 1 -- 172.21.15.70:0/1009957390 shutdown_connections 2023-10-08T05:57:03.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.639+0000 7f81c27ce640 1 -- 172.21.15.70:0/1009957390 wait complete. 2023-10-08T05:57:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.641+0000 7f81c27ce640 1 Processor -- start 2023-10-08T05:57:03.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.641+0000 7f81c27ce640 1 -- start start 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.642+0000 7f81c27ce640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc06c3a0 0x7f81bc0c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81c27ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06db60 0x7f81bc151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81c27ce640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15aa00 0x7f81bc0c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81c27ce640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc152c00 con 0x7f81bc06db60 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81c27ce640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc0c51c0 con 0x7f81bc06c3a0 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81c27ce640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc152640 con 0x7f81bc15aa00 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06db60 0x7f81bc151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81b3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06db60 0x7f81bc151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52398/0 (socket says 172.21.15.70:52398) 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81b3fff640 1 -- 172.21.15.70:0/1112722109 learned_addr learned my addr 172.21.15.70:0/1112722109 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81bbfff640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc06c3a0 0x7f81bc0c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81b3fff640 1 -- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15aa00 msgr2=0x7f81bc0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81b3fff640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15aa00 0x7f81bc0c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81b3fff640 1 -- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc06c3a0 msgr2=0x7f81bc0c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81b3fff640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc06c3a0 0x7f81bc0c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.643+0000 7f81b3fff640 1 -- 172.21.15.70:0/1112722109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac009400 con 0x7f81bc06db60 2023-10-08T05:57:03.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.644+0000 7f81b3fff640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06db60 0x7f81bc151020 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f81a400c4a0 tx=0x7f81a400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.644+0000 7f81b97fa640 1 -- 172.21.15.70:0/1112722109 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4005ce0 con 0x7f81bc06db60 2023-10-08T05:57:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.644+0000 7f81b97fa640 1 -- 172.21.15.70:0/1112722109 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4005e40 con 0x7f81bc06db60 2023-10-08T05:57:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.644+0000 7f81b97fa640 1 -- 172.21.15.70:0/1112722109 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4007270 con 0x7f81bc06db60 2023-10-08T05:57:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.644+0000 7f81c27ce640 1 -- 172.21.15.70:0/1112722109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc152920 con 0x7f81bc06db60 2023-10-08T05:57:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.644+0000 7f81c27ce640 1 -- 172.21.15.70:0/1112722109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc15b930 con 0x7f81bc06db60 2023-10-08T05:57:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.645+0000 7f81c27ce640 1 -- 172.21.15.70:0/1112722109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc15b4b0 con 0x7f81bc06db60 2023-10-08T05:57:03.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.646+0000 7f81b97fa640 1 -- 172.21.15.70:0/1112722109 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 76) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f81a4007490 con 0x7f81bc06db60 2023-10-08T05:57:03.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.647+0000 7f81b97fa640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f819c06af00 0x7f819c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:03.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.647+0000 7f81b97fa640 1 -- 172.21.15.70:0/1112722109 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6918+0+0 (secure 0 0 0) 0x7f81a4088ee0 con 0x7f81bc06db60 2023-10-08T05:57:03.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.648+0000 7f81bbfff640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f819c06af00 0x7f819c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:03.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.649+0000 7f81bbfff640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f819c06af00 0x7f819c06d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f81ac036000 tx=0x7f81ac03c040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:03.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.651+0000 7f81b97fa640 1 -- 172.21.15.70:0/1112722109 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81bc15b4b0 con 0x7f81bc06db60 2023-10-08T05:57:03.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:03.877+0000 7f81c27ce640 1 -- 172.21.15.70:0/1112722109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f81bc15bb20 con 0x7f81bc06db60 2023-10-08T05:57:04.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.423+0000 7f81b97fa640 1 -- 172.21.15.70:0/1112722109 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7f81a4017020 con 0x7f81bc06db60 2023-10-08T05:57:04.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.426+0000 7f81c27ce640 1 -- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f819c06af00 msgr2=0x7f819c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:04.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.426+0000 7f81c27ce640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f819c06af00 0x7f819c06d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f81ac036000 tx=0x7f81ac03c040 comp rx=0 tx=0).stop 2023-10-08T05:57:04.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.427+0000 7f81c27ce640 1 -- 172.21.15.70:0/1112722109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06db60 msgr2=0x7f81bc151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:04.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.427+0000 7f81c27ce640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06db60 0x7f81bc151020 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f81a400c4a0 tx=0x7f81a400c960 comp rx=0 tx=0).stop 2023-10-08T05:57:04.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.427+0000 7f81c27ce640 1 -- 172.21.15.70:0/1112722109 shutdown_connections 2023-10-08T05:57:04.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.427+0000 7f81c27ce640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f819c06af00 0x7f819c06d3c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.427+0000 7f81c27ce640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81bc15aa00 0x7f81bc0c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.427+0000 7f81c27ce640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81bc06db60 0x7f81bc151020 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.427+0000 7f81c27ce640 1 --2- 172.21.15.70:0/1112722109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81bc06c3a0 0x7f81bc0c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.427+0000 7f81c27ce640 1 -- 172.21.15.70:0/1112722109 >> 172.21.15.70:0/1112722109 conn(0x7f81bc06a5d0 msgr2=0x7f81bc0b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:04.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.427+0000 7f81c27ce640 1 -- 172.21.15.70:0/1112722109 shutdown_connections 2023-10-08T05:57:04.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.427+0000 7f81c27ce640 1 -- 172.21.15.70:0/1112722109 wait complete. 2023-10-08T05:57:04.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T05:57:04.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.654+0000 7fadffd0f640 1 Processor -- start 2023-10-08T05:57:04.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.654+0000 7fadffd0f640 1 -- start start 2023-10-08T05:57:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.655+0000 7fadffd0f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf806dfc0 0x7fadf8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.655+0000 7fadffd0f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fadf8071070 0x7fadf80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.655+0000 7fadffd0f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf80b1cc0 con 0x7fadf80afbe0 2023-10-08T05:57:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.655+0000 7fadffd0f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf80b2620 con 0x7fadf8071070 2023-10-08T05:57:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.655+0000 7fadffd0f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80b2060 con 0x7fadf806dfc0 2023-10-08T05:57:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.656+0000 7fadfdaab640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fadf80afbe0 0x7fadf80b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46302/0 (socket says 172.21.15.70:46302) 2023-10-08T05:57:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.656+0000 7fadfdaab640 1 -- 172.21.15.70:0/3726877671 learned_addr learned my addr 172.21.15.70:0/3726877671 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:04.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.656+0000 7fadfe2ac640 1 --2- 172.21.15.70:0/3726877671 >> v2:172.21.15.181:3300/0 conn(0x7fadf8071070 0x7fadf80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.656+0000 7fadfd2aa640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf806dfc0 0x7fadf8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.656+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914578251 0 0) 0x7fadf80b1cc0 con 0x7fadf80afbe0 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.656+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade4003520 con 0x7fadf80afbe0 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 483376544 0 0) 0x7fade4003520 con 0x7fadf80afbe0 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf806dfc0 msgr2=0x7fadf8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf806dfc0 0x7fadf8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 >> v2:172.21.15.181:3300/0 conn(0x7fadf8071070 msgr2=0x7fadf80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 --2- 172.21.15.70:0/3726877671 >> v2:172.21.15.181:3300/0 conn(0x7fadf8071070 0x7fadf80cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80b2340 con 0x7fadf80afbe0 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fadf4001670 con 0x7fadf80afbe0 2023-10-08T05:57:04.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfe2ac640 1 --2- 172.21.15.70:0/3726877671 >> v2:172.21.15.181:3300/0 conn(0x7fadf8071070 0x7fadf80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 >> v1:172.21.15.70:6789/0 conn(0x7fadf80afbe0 legacy=0x7fadf80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fade4003970 0x7fade4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf806dfc0 0x7fade40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade4004d00 0x7fade4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80b2620 con 0x7fade4003970 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80b2060 con 0x7fade4004d00 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade400b9d0 con 0x7fadf806dfc0 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfdaab640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fade4003970 0x7fade4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.657+0000 7fadfd2aa640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf806dfc0 0x7fade40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.658+0000 7fadfe2ac640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade4004d00 0x7fade4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.658+0000 7fadfdaab640 1 -- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf806dfc0 msgr2=0x7fade40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:04.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.658+0000 7fadfdaab640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf806dfc0 0x7fade40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.658+0000 7fadfdaab640 1 -- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade4004d00 msgr2=0x7fade4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:04.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.658+0000 7fadfdaab640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade4004d00 0x7fade4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.658+0000 7fadfdaab640 1 -- 172.21.15.70:0/3726877671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4003520 con 0x7fade4003970 2023-10-08T05:57:04.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.658+0000 7fadfe2ac640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fade4004d00 0x7fade4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:04.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.659+0000 7fadfdaab640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fade4003970 0x7fade4003d90 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c290 tx=0x7fade40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa072ee8d3517a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:04.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.659+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4005ad0 con 0x7fade4003970 2023-10-08T05:57:04.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.659+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf4005c30 con 0x7fade4003970 2023-10-08T05:57:04.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.659+0000 7fadfcaa9640 1 -- 172.21.15.70:0/3726877671 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf402c8d0 con 0x7fade4003970 2023-10-08T05:57:04.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.660+0000 7fadffd0f640 1 -- 172.21.15.70:0/3726877671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fade4003970 msgr2=0x7fade4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:04.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.660+0000 7fadffd0f640 1 --2- 172.21.15.70:0/3726877671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fade4003970 0x7fade4003d90 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c290 tx=0x7fade40057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.660+0000 7fadfe2ac640 1 -- 172.21.15.70:0/3726877671 reap_dead start 2023-10-08T05:57:04.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.660+0000 7fadffd0f640 1 -- 172.21.15.70:0/3726877671 shutdown_connections 2023-10-08T05:57:04.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.660+0000 7fadffd0f640 1 -- 172.21.15.70:0/3726877671 >> 172.21.15.70:0/3726877671 conn(0x7fadf806a5d0 msgr2=0x7fadf80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:04.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.660+0000 7fadffd0f640 1 -- 172.21.15.70:0/3726877671 shutdown_connections 2023-10-08T05:57:04.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.660+0000 7fadffd0f640 1 -- 172.21.15.70:0/3726877671 wait complete. 2023-10-08T05:57:04.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.662+0000 7fadffd0f640 1 Processor -- start 2023-10-08T05:57:04.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.662+0000 7fadffd0f640 1 -- start start 2023-10-08T05:57:04.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.663+0000 7fadffd0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf81f5590 0x7fadf815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:04.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.663+0000 7fadffd0f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf80c4150 0x7fadf80c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:04.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.663+0000 7fadffd0f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf80ca4f0 0x7fadf80c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:04.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.663+0000 7fadffd0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80ca9a0 con 0x7fadf81f5590 2023-10-08T05:57:04.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.663+0000 7fadffd0f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf81f24e0 con 0x7fadf80ca4f0 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.663+0000 7fadffd0f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80c51f0 con 0x7fadf80c4150 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.663+0000 7fadfdaab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf81f5590 0x7fadf815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.663+0000 7fadfdaab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf81f5590 0x7fadf815c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52422/0 (socket says 172.21.15.70:52422) 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.663+0000 7fadfdaab640 1 -- 172.21.15.70:0/3959526298 learned_addr learned my addr 172.21.15.70:0/3959526298 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadfe2ac640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf80ca4f0 0x7fadf80c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadfdaab640 1 -- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf80c4150 msgr2=0x7fadf80c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadfd2aa640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf80c4150 0x7fadf80c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadfdaab640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf80c4150 0x7fadf80c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadfdaab640 1 -- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf80ca4f0 msgr2=0x7fadf80c2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadfdaab640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf80ca4f0 0x7fadf80c2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadfdaab640 1 -- 172.21.15.70:0/3959526298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf80c45e0 con 0x7fadf81f5590 2023-10-08T05:57:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadfdaab640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf81f5590 0x7fadf815c8d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c380 tx=0x7fadf4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:04.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadee7fc640 1 -- 172.21.15.70:0/3959526298 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4039070 con 0x7fadf81f5590 2023-10-08T05:57:04.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadee7fc640 1 -- 172.21.15.70:0/3959526298 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf4004aa0 con 0x7fadf81f5590 2023-10-08T05:57:04.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadffd0f640 1 -- 172.21.15.70:0/3959526298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf80c48c0 con 0x7fadf81f5590 2023-10-08T05:57:04.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadee7fc640 1 -- 172.21.15.70:0/3959526298 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4004e50 con 0x7fadf81f5590 2023-10-08T05:57:04.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.664+0000 7fadffd0f640 1 -- 172.21.15.70:0/3959526298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf80c4e80 con 0x7fadf81f5590 2023-10-08T05:57:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.665+0000 7fadffd0f640 1 -- 172.21.15.70:0/3959526298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf804e160 con 0x7fadf81f5590 2023-10-08T05:57:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.666+0000 7fadee7fc640 1 -- 172.21.15.70:0/3959526298 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fadf4032070 con 0x7fadf81f5590 2023-10-08T05:57:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.667+0000 7fadee7fc640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7faddc06afc0 0x7faddc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.667+0000 7fadee7fc640 1 -- 172.21.15.70:0/3959526298 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6918+0+0 (secure 0 0 0) 0x7fadf403c040 con 0x7fadf81f5590 2023-10-08T05:57:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.668+0000 7fadfd2aa640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7faddc06afc0 0x7faddc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:04.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.669+0000 7fadfd2aa640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7faddc06afc0 0x7faddc06d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fadf80cd100 tx=0x7fade8008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:04.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.672+0000 7fadee7fc640 1 -- 172.21.15.70:0/3959526298 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fadf4075a40 con 0x7fadf81f5590 2023-10-08T05:57:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:04.860+0000 7fadffd0f640 1 -- 172.21.15.70:0/3959526298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fadf80b0ef0 con 0x7fadf81f5590 2023-10-08T05:57:05.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.622+0000 7fadee7fc640 1 -- 172.21.15.70:0/3959526298 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) v1 ==== 181+0+0 (secure 0 0 0) 0x7fadf4079060 con 0x7fadf81f5590 2023-10-08T05:57:05.623 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T05:57:05.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.626+0000 7fadffd0f640 1 -- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7faddc06afc0 msgr2=0x7faddc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:05.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.626+0000 7fadffd0f640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7faddc06afc0 0x7faddc06d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fadf80cd100 tx=0x7fade8008040 comp rx=0 tx=0).stop 2023-10-08T05:57:05.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.626+0000 7fadffd0f640 1 -- 172.21.15.70:0/3959526298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf81f5590 msgr2=0x7fadf815c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:05.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.626+0000 7fadffd0f640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf81f5590 0x7fadf815c8d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c380 tx=0x7fadf4000fe0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.627+0000 7fadffd0f640 1 -- 172.21.15.70:0/3959526298 shutdown_connections 2023-10-08T05:57:05.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.627+0000 7fadffd0f640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7faddc06afc0 0x7faddc06d480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.627+0000 7fadffd0f640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fadf80ca4f0 0x7fadf80c2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.627+0000 7fadffd0f640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fadf80c4150 0x7fadf80c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.627+0000 7fadffd0f640 1 --2- 172.21.15.70:0/3959526298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf81f5590 0x7fadf815c8d0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.627+0000 7fadffd0f640 1 -- 172.21.15.70:0/3959526298 >> 172.21.15.70:0/3959526298 conn(0x7fadf806a5d0 msgr2=0x7fadf80ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:05.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.628+0000 7fadffd0f640 1 -- 172.21.15.70:0/3959526298 shutdown_connections 2023-10-08T05:57:05.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.628+0000 7fadffd0f640 1 -- 172.21.15.70:0/3959526298 wait complete. 2023-10-08T05:57:05.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T05:57:05.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.860+0000 7f7a174d4640 1 Processor -- start 2023-10-08T05:57:05.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.861+0000 7f7a174d4640 1 -- start start 2023-10-08T05:57:05.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.862+0000 7f7a174d4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a100cea30 0x7f7a1015d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:05.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.862+0000 7f7a174d4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a100ccb40 0x7f7a100ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:05.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.862+0000 7f7a174d4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a100cd490 con 0x7f7a1015cdb0 2023-10-08T05:57:05.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.862+0000 7f7a174d4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a1006b6a0 con 0x7f7a100cea30 2023-10-08T05:57:05.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.862+0000 7f7a174d4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a100b19e0 con 0x7f7a100ccb40 2023-10-08T05:57:05.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.862+0000 7f7a15270640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7a1015cdb0 0x7f7a1006af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46316/0 (socket says 172.21.15.70:46316) 2023-10-08T05:57:05.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.862+0000 7f7a15270640 1 -- 172.21.15.70:0/972553011 learned_addr learned my addr 172.21.15.70:0/972553011 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:05.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.862+0000 7f7a14a6f640 1 --2- 172.21.15.70:0/972553011 >> v2:172.21.15.70:3300/0 conn(0x7f7a100cea30 0x7f7a1015d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.863+0000 7f7a15a71640 1 --2- 172.21.15.70:0/972553011 >> v2:172.21.15.181:3300/0 conn(0x7f7a100ccb40 0x7f7a100ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.863+0000 7f7a0ffff640 1 -- 172.21.15.70:0/972553011 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101091011 0 0) 0x7f7a100cd490 con 0x7f7a1015cdb0 2023-10-08T05:57:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.863+0000 7f7a0ffff640 1 -- 172.21.15.70:0/972553011 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f8003520 con 0x7f7a1015cdb0 2023-10-08T05:57:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.863+0000 7f7a14a6f640 1 -- 172.21.15.70:0/972553011 >> v2:172.21.15.181:3300/0 conn(0x7f7a100ccb40 msgr2=0x7f7a100ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.863+0000 7f7a14a6f640 1 --2- 172.21.15.70:0/972553011 >> v2:172.21.15.181:3300/0 conn(0x7f7a100ccb40 0x7f7a100ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.863+0000 7f7a14a6f640 1 -- 172.21.15.70:0/972553011 >> v1:172.21.15.70:6789/0 conn(0x7f7a1015cdb0 legacy=0x7f7a1006af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.863+0000 7f7a14a6f640 1 -- 172.21.15.70:0/972553011 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a10069ef0 con 0x7f7a100cea30 2023-10-08T05:57:05.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.865+0000 7f7a14a6f640 1 --2- 172.21.15.70:0/972553011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a100cea30 0x7f7a1015d190 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f7a100cb9c0 tx=0x7f7a0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1340b005aaf329a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:05.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.866+0000 7f7a0ffff640 1 -- 172.21.15.70:0/972553011 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04031830 con 0x7f7a100cea30 2023-10-08T05:57:05.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.866+0000 7f7a0ffff640 1 -- 172.21.15.70:0/972553011 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a04031990 con 0x7f7a100cea30 2023-10-08T05:57:05.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.866+0000 7f7a0ffff640 1 -- 172.21.15.70:0/972553011 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04031d00 con 0x7f7a100cea30 2023-10-08T05:57:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.866+0000 7f7a174d4640 1 -- 172.21.15.70:0/972553011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a100cea30 msgr2=0x7f7a1015d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.866+0000 7f7a174d4640 1 --2- 172.21.15.70:0/972553011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a100cea30 0x7f7a1015d190 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f7a100cb9c0 tx=0x7f7a0402b8d0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.867+0000 7f7a174d4640 1 -- 172.21.15.70:0/972553011 shutdown_connections 2023-10-08T05:57:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.867+0000 7f7a174d4640 1 --2- 172.21.15.70:0/972553011 >> v2:172.21.15.181:3300/0 conn(0x7f7a100ccb40 0x7f7a100ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.867+0000 7f7a174d4640 1 --2- 172.21.15.70:0/972553011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a100cea30 0x7f7a1015d190 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.867+0000 7f7a174d4640 1 -- 172.21.15.70:0/972553011 >> 172.21.15.70:0/972553011 conn(0x7f7a100b0af0 msgr2=0x7f7a100b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:05.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.867+0000 7f7a174d4640 1 -- 172.21.15.70:0/972553011 shutdown_connections 2023-10-08T05:57:05.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.867+0000 7f7a174d4640 1 -- 172.21.15.70:0/972553011 wait complete. 2023-10-08T05:57:05.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.868+0000 7f7a174d4640 1 Processor -- start 2023-10-08T05:57:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.869+0000 7f7a174d4640 1 -- start start 2023-10-08T05:57:05.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.869+0000 7f7a174d4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a100ccb40 0x7f7a1015b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:05.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.869+0000 7f7a174d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1015cdb0 0x7f7a100b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:05.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a174d4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a101542b0 0x7f7a10154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:05.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a174d4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a10154c80 con 0x7f7a1015cdb0 2023-10-08T05:57:05.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a174d4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a10155800 con 0x7f7a101542b0 2023-10-08T05:57:05.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a174d4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a10155aa0 con 0x7f7a100ccb40 2023-10-08T05:57:05.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a14a6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1015cdb0 0x7f7a100b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:05.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a14a6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1015cdb0 0x7f7a100b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52442/0 (socket says 172.21.15.70:52442) 2023-10-08T05:57:05.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a14a6f640 1 -- 172.21.15.70:0/1443863802 learned_addr learned my addr 172.21.15.70:0/1443863802 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:05.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a15270640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a100ccb40 0x7f7a1015b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:05.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a15a71640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a101542b0 0x7f7a10154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:05.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a14a6f640 1 -- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a100ccb40 msgr2=0x7f7a1015b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a14a6f640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a100ccb40 0x7f7a1015b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a14a6f640 1 -- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a101542b0 msgr2=0x7f7a10154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.870+0000 7f7a14a6f640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a101542b0 0x7f7a10154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.871+0000 7f7a14a6f640 1 -- 172.21.15.70:0/1443863802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a04009400 con 0x7f7a1015cdb0 2023-10-08T05:57:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.871+0000 7f7a14a6f640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1015cdb0 0x7f7a100b2220 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000c00 tx=0x7f7a040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.871+0000 7f7a0dffb640 1 -- 172.21.15.70:0/1443863802 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a040317b0 con 0x7f7a1015cdb0 2023-10-08T05:57:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.871+0000 7f7a0dffb640 1 -- 172.21.15.70:0/1443863802 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0403b9f0 con 0x7f7a1015cdb0 2023-10-08T05:57:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.871+0000 7f7a0dffb640 1 -- 172.21.15.70:0/1443863802 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0403bda0 con 0x7f7a1015cdb0 2023-10-08T05:57:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.871+0000 7f7a174d4640 1 -- 172.21.15.70:0/1443863802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a10155d80 con 0x7f7a1015cdb0 2023-10-08T05:57:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.871+0000 7f7a174d4640 1 -- 172.21.15.70:0/1443863802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a10156340 con 0x7f7a1015cdb0 2023-10-08T05:57:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.872+0000 7f7a174d4640 1 -- 172.21.15.70:0/1443863802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a10156530 con 0x7f7a1015cdb0 2023-10-08T05:57:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.873+0000 7f7a0dffb640 1 -- 172.21.15.70:0/1443863802 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7a040453f0 con 0x7f7a1015cdb0 2023-10-08T05:57:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.874+0000 7f7a0dffb640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f79fc06af90 0x7f79fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.874+0000 7f7a0dffb640 1 -- 172.21.15.70:0/1443863802 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6480+0+0 (secure 0 0 0) 0x7f7a040abe20 con 0x7f7a1015cdb0 2023-10-08T05:57:05.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.875+0000 7f7a15270640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f79fc06af90 0x7f79fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:05.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.876+0000 7f7a15270640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f79fc06af90 0x7f79fc06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005c00 tx=0x7f7a00009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:05.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:05.878+0000 7f7a0dffb640 1 -- 172.21.15.70:0/1443863802 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a04076a50 con 0x7f7a1015cdb0 2023-10-08T05:57:06.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.074+0000 7f7a174d4640 1 -- 172.21.15.70:0/1443863802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7a1004e160 con 0x7f7a1015cdb0 2023-10-08T05:57:06.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.625+0000 7f7a0dffb640 1 -- 172.21.15.70:0/1443863802 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7a04035020 con 0x7f7a1015cdb0 2023-10-08T05:57:06.627 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T05:57:06.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.629+0000 7f7a174d4640 1 -- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f79fc06af90 msgr2=0x7f79fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:06.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.629+0000 7f7a174d4640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f79fc06af90 0x7f79fc06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005c00 tx=0x7f7a00009040 comp rx=0 tx=0).stop 2023-10-08T05:57:06.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.629+0000 7f7a174d4640 1 -- 172.21.15.70:0/1443863802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1015cdb0 msgr2=0x7f7a100b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:06.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.629+0000 7f7a174d4640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1015cdb0 0x7f7a100b2220 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000c00 tx=0x7f7a040046c0 comp rx=0 tx=0).stop 2023-10-08T05:57:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.631+0000 7f7a174d4640 1 -- 172.21.15.70:0/1443863802 shutdown_connections 2023-10-08T05:57:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.631+0000 7f7a174d4640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f79fc06af90 0x7f79fc06d450 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7a00005c00 tx=0x7f7a00009040 comp rx=0 tx=0).stop 2023-10-08T05:57:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.631+0000 7f7a174d4640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a101542b0 0x7f7a10154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.631+0000 7f7a174d4640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1015cdb0 0x7f7a100b2220 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.631+0000 7f7a174d4640 1 --2- 172.21.15.70:0/1443863802 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a100ccb40 0x7f7a1015b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.631+0000 7f7a174d4640 1 -- 172.21.15.70:0/1443863802 >> 172.21.15.70:0/1443863802 conn(0x7f7a100b0af0 msgr2=0x7f7a1006e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.632+0000 7f7a174d4640 1 -- 172.21.15.70:0/1443863802 shutdown_connections 2023-10-08T05:57:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.632+0000 7f7a174d4640 1 -- 172.21.15.70:0/1443863802 wait complete. 2023-10-08T05:57:06.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T05:57:06.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.866+0000 7f65f4820640 1 Processor -- start 2023-10-08T05:57:06.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.867+0000 7f65f4820640 1 -- start start 2023-10-08T05:57:06.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.868+0000 7f65f4820640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65f00b0680 0x7f65f015e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:06.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.868+0000 7f65f4820640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 0x7f65f015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:06.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.868+0000 7f65f4820640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f00cab60 con 0x7f65f015a820 2023-10-08T05:57:06.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.868+0000 7f65f4820640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f00cb960 con 0x7f65f00b0680 2023-10-08T05:57:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.868+0000 7f65f4820640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f00cbdf0 con 0x7f65f00ca680 2023-10-08T05:57:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.868+0000 7f65e7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65f00b0680 0x7f65f015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.869+0000 7f65e7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65f00b0680 0x7f65f015e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52458/0 (socket says 172.21.15.70:52458) 2023-10-08T05:57:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.869+0000 7f65e7fff640 1 -- 172.21.15.70:0/2992258686 learned_addr learned my addr 172.21.15.70:0/2992258686 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.869+0000 7f65ef59e640 1 --2- 172.21.15.70:0/2992258686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 0x7f65f015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.869+0000 7f65e7fff640 1 -- 172.21.15.70:0/2992258686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 msgr2=0x7f65f015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.869+0000 7f65e7fff640 1 --2- 172.21.15.70:0/2992258686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 0x7f65f015ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.869+0000 7f65e7fff640 1 -- 172.21.15.70:0/2992258686 >> v1:172.21.15.181:6789/0 conn(0x7f65f015a820 legacy=0x7f65f00aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.869+0000 7f65e7fff640 1 -- 172.21.15.70:0/2992258686 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f00cb030 con 0x7f65f00b0680 2023-10-08T05:57:06.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.869+0000 7f65ef59e640 1 --2- 172.21.15.70:0/2992258686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 0x7f65f015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.870+0000 7f65e7fff640 1 --2- 172.21.15.70:0/2992258686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f00b0680 0x7f65f015e540 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009870 tx=0x7f65d802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3594dfdeabcff1d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.871+0000 7f65ee59c640 1 -- 172.21.15.70:0/2992258686 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80317c0 con 0x7f65f00b0680 2023-10-08T05:57:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.871+0000 7f65ee59c640 1 -- 172.21.15.70:0/2992258686 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d8031920 con 0x7f65f00b0680 2023-10-08T05:57:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.871+0000 7f65ee59c640 1 -- 172.21.15.70:0/2992258686 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8031cb0 con 0x7f65f00b0680 2023-10-08T05:57:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.871+0000 7f65f4820640 1 -- 172.21.15.70:0/2992258686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f00b0680 msgr2=0x7f65f015e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.871+0000 7f65f4820640 1 --2- 172.21.15.70:0/2992258686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f00b0680 0x7f65f015e540 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009870 tx=0x7f65d802b860 comp rx=0 tx=0).stop 2023-10-08T05:57:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.871+0000 7f65f4820640 1 -- 172.21.15.70:0/2992258686 shutdown_connections 2023-10-08T05:57:06.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.871+0000 7f65f4820640 1 --2- 172.21.15.70:0/2992258686 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 0x7f65f015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:06.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.871+0000 7f65f4820640 1 --2- 172.21.15.70:0/2992258686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f00b0680 0x7f65f015e540 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:06.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.871+0000 7f65f4820640 1 -- 172.21.15.70:0/2992258686 >> 172.21.15.70:0/2992258686 conn(0x7f65f006c050 msgr2=0x7f65f006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:06.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.872+0000 7f65f4820640 1 -- 172.21.15.70:0/2992258686 shutdown_connections 2023-10-08T05:57:06.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.872+0000 7f65f4820640 1 -- 172.21.15.70:0/2992258686 wait complete. 2023-10-08T05:57:06.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.873+0000 7f65f4820640 1 Processor -- start 2023-10-08T05:57:06.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.873+0000 7f65f4820640 1 -- start start 2023-10-08T05:57:06.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.874+0000 7f65f4820640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f00b0680 0x7f65f01553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:06.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.874+0000 7f65f4820640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 0x7f65f01545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:06.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.874+0000 7f65f4820640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f015a820 0x7f65f01f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:06.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.874+0000 7f65f4820640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f00c6ba0 con 0x7f65f015a820 2023-10-08T05:57:06.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.874+0000 7f65f4820640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f01f0150 con 0x7f65f00b0680 2023-10-08T05:57:06.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.874+0000 7f65f4820640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f0157240 con 0x7f65f00ca680 2023-10-08T05:57:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.875+0000 7f65ef59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f015a820 0x7f65f01f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.875+0000 7f65ef59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f015a820 0x7f65f01f05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52468/0 (socket says 172.21.15.70:52468) 2023-10-08T05:57:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.875+0000 7f65ef59e640 1 -- 172.21.15.70:0/2607599673 learned_addr learned my addr 172.21.15.70:0/2607599673 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.875+0000 7f65e7fff640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 0x7f65f01545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.875+0000 7f65ef59e640 1 -- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 msgr2=0x7f65f01545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.875+0000 7f65ef59e640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 0x7f65f01545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.875+0000 7f65ef59e640 1 -- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f00b0680 msgr2=0x7f65f01553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.875+0000 7f65eed9d640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f00b0680 0x7f65f01553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.875+0000 7f65ef59e640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f00b0680 0x7f65f01553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.876+0000 7f65ef59e640 1 -- 172.21.15.70:0/2607599673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d80094d0 con 0x7f65f015a820 2023-10-08T05:57:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.876+0000 7f65ef59e640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f015a820 0x7f65f01f05e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f65e800ce80 tx=0x7f65e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.876+0000 7f65e77fe640 1 -- 172.21.15.70:0/2607599673 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e800d840 con 0x7f65f015a820 2023-10-08T05:57:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.876+0000 7f65e77fe640 1 -- 172.21.15.70:0/2607599673 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e800d9e0 con 0x7f65f015a820 2023-10-08T05:57:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.876+0000 7f65e77fe640 1 -- 172.21.15.70:0/2607599673 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e800dd90 con 0x7f65f015a820 2023-10-08T05:57:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.876+0000 7f65f4820640 1 -- 172.21.15.70:0/2607599673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f0157520 con 0x7f65f015a820 2023-10-08T05:57:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.876+0000 7f65f4820640 1 -- 172.21.15.70:0/2607599673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f01efdb0 con 0x7f65f015a820 2023-10-08T05:57:06.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.877+0000 7f65f4820640 1 -- 172.21.15.70:0/2607599673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f01f02c0 con 0x7f65f015a820 2023-10-08T05:57:06.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.878+0000 7f65e77fe640 1 -- 172.21.15.70:0/2607599673 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f65e8015440 con 0x7f65f015a820 2023-10-08T05:57:06.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.880+0000 7f65e77fe640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f65d006af90 0x7f65d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:06.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.880+0000 7f65e77fe640 1 -- 172.21.15.70:0/2607599673 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f65e808a2a0 con 0x7f65f015a820 2023-10-08T05:57:06.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.880+0000 7f65eed9d640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f65d006af90 0x7f65d006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:06.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.881+0000 7f65eed9d640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f65d006af90 0x7f65d006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f65e000a750 tx=0x7f65e0008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:06.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:06.884+0000 7f65e77fe640 1 -- 172.21.15.70:0/2607599673 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65e80550e0 con 0x7f65f015a820 2023-10-08T05:57:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.072+0000 7f65f4820640 1 -- 172.21.15.70:0/2607599673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f65f0074150 con 0x7f65f015a820 2023-10-08T05:57:07.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.077+0000 7f65e77fe640 1 -- 172.21.15.70:0/2607599673 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f65e8058700 con 0x7f65f015a820 2023-10-08T05:57:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 -- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f65d006af90 msgr2=0x7f65d006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f65d006af90 0x7f65d006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f65e000a750 tx=0x7f65e0008040 comp rx=0 tx=0).stop 2023-10-08T05:57:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 -- 172.21.15.70:0/2607599673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f015a820 msgr2=0x7f65f01f05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f015a820 0x7f65f01f05e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f65e800ce80 tx=0x7f65e800a710 comp rx=0 tx=0).stop 2023-10-08T05:57:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 -- 172.21.15.70:0/2607599673 shutdown_connections 2023-10-08T05:57:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f65d006af90 0x7f65d006d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f015a820 0x7f65f01f05e0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65f00ca680 0x7f65f01545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 --2- 172.21.15.70:0/2607599673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65f00b0680 0x7f65f01553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 -- 172.21.15.70:0/2607599673 >> 172.21.15.70:0/2607599673 conn(0x7f65f006c050 msgr2=0x7f65f00b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:07.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.080+0000 7f65f4820640 1 -- 172.21.15.70:0/2607599673 shutdown_connections 2023-10-08T05:57:07.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.081+0000 7f65f4820640 1 -- 172.21.15.70:0/2607599673 wait complete. 2023-10-08T05:57:07.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T05:57:07.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.300+0000 7fe64da87640 1 Processor -- start 2023-10-08T05:57:07.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.300+0000 7fe64da87640 1 -- start start 2023-10-08T05:57:07.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.301+0000 7fe64da87640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64815aad0 0x7fe6480b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.301+0000 7fe64da87640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe64806d410 0x7fe6480c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.301+0000 7fe64da87640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64815bf30 con 0x7fe6480c82b0 2023-10-08T05:57:07.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.301+0000 7fe64da87640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64815b320 con 0x7fe64806d410 2023-10-08T05:57:07.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.301+0000 7fe64da87640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64815b600 con 0x7fe64815aad0 2023-10-08T05:57:07.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.302+0000 7fe647fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe64806d410 0x7fe6480c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.302+0000 7fe6477fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64815aad0 0x7fe6480b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.302+0000 7fe6477fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64815aad0 0x7fe6480b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:48246/0 (socket says 172.21.15.70:48246) 2023-10-08T05:57:07.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.302+0000 7fe6477fe640 1 -- 172.21.15.70:0/2433696645 learned_addr learned my addr 172.21.15.70:0/2433696645 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:07.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.303+0000 7fe6477fe640 1 -- 172.21.15.70:0/2433696645 >> v2:172.21.15.181:3300/0 conn(0x7fe64806d410 msgr2=0x7fe6480c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:07.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.303+0000 7fe6477fe640 1 --2- 172.21.15.70:0/2433696645 >> v2:172.21.15.181:3300/0 conn(0x7fe64806d410 0x7fe6480c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.303+0000 7fe6477fe640 1 -- 172.21.15.70:0/2433696645 >> v1:172.21.15.181:6789/0 conn(0x7fe6480c82b0 legacy=0x7fe64806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.303+0000 7fe6477fe640 1 -- 172.21.15.70:0/2433696645 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64815b8e0 con 0x7fe64815aad0 2023-10-08T05:57:07.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.304+0000 7fe6477fe640 1 --2- 172.21.15.70:0/2433696645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64815aad0 0x7fe6480b1750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe638009870 tx=0x7fe63802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af7340f8b192b27d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:07.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.305+0000 7fe646ffd640 1 -- 172.21.15.70:0/2433696645 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638030840 con 0x7fe64815aad0 2023-10-08T05:57:07.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.305+0000 7fe646ffd640 1 -- 172.21.15.70:0/2433696645 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6380309a0 con 0x7fe64815aad0 2023-10-08T05:57:07.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.305+0000 7fe646ffd640 1 -- 172.21.15.70:0/2433696645 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638030d30 con 0x7fe64815aad0 2023-10-08T05:57:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.305+0000 7fe64da87640 1 -- 172.21.15.70:0/2433696645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64815aad0 msgr2=0x7fe6480b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.305+0000 7fe64da87640 1 --2- 172.21.15.70:0/2433696645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64815aad0 0x7fe6480b1750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe638009870 tx=0x7fe63802b980 comp rx=0 tx=0).stop 2023-10-08T05:57:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.306+0000 7fe64da87640 1 -- 172.21.15.70:0/2433696645 shutdown_connections 2023-10-08T05:57:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.306+0000 7fe64da87640 1 --2- 172.21.15.70:0/2433696645 >> v2:172.21.15.181:3300/0 conn(0x7fe64806d410 0x7fe6480c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.306+0000 7fe64da87640 1 --2- 172.21.15.70:0/2433696645 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64815aad0 0x7fe6480b1750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.306+0000 7fe64da87640 1 -- 172.21.15.70:0/2433696645 >> 172.21.15.70:0/2433696645 conn(0x7fe648069d50 msgr2=0x7fe64806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:07.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.306+0000 7fe64da87640 1 -- 172.21.15.70:0/2433696645 shutdown_connections 2023-10-08T05:57:07.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.306+0000 7fe64da87640 1 -- 172.21.15.70:0/2433696645 wait complete. 2023-10-08T05:57:07.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.307+0000 7fe64da87640 1 Processor -- start 2023-10-08T05:57:07.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.307+0000 7fe64da87640 1 -- start start 2023-10-08T05:57:07.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.308+0000 7fe64da87640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64806d410 0x7fe6480b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.308+0000 7fe64da87640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480c82b0 0x7fe648157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.308+0000 7fe64da87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815aad0 0x7fe648153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.308+0000 7fe64da87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6481f2a70 con 0x7fe64815aad0 2023-10-08T05:57:07.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.308+0000 7fe64da87640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe648154ba0 con 0x7fe6480c82b0 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.308+0000 7fe64da87640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6481545e0 con 0x7fe64806d410 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe647fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815aad0 0x7fe648153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe647fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815aad0 0x7fe648153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52478/0 (socket says 172.21.15.70:52478) 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe647fff640 1 -- 172.21.15.70:0/507056471 learned_addr learned my addr 172.21.15.70:0/507056471 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe6477fe640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64806d410 0x7fe6480b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe63effd640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480c82b0 0x7fe648157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe647fff640 1 -- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64806d410 msgr2=0x7fe6480b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe647fff640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64806d410 0x7fe6480b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe647fff640 1 -- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480c82b0 msgr2=0x7fe648157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe647fff640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480c82b0 0x7fe648157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe647fff640 1 -- 172.21.15.70:0/507056471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6380094d0 con 0x7fe64815aad0 2023-10-08T05:57:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.309+0000 7fe647fff640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815aad0 0x7fe648153a70 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fe64000ef90 tx=0x7fe64000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.310+0000 7fe644ff9640 1 -- 172.21.15.70:0/507056471 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640009ca0 con 0x7fe64815aad0 2023-10-08T05:57:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.310+0000 7fe644ff9640 1 -- 172.21.15.70:0/507056471 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64000f040 con 0x7fe64815aad0 2023-10-08T05:57:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.310+0000 7fe644ff9640 1 -- 172.21.15.70:0/507056471 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6400146c0 con 0x7fe64815aad0 2023-10-08T05:57:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.310+0000 7fe64da87640 1 -- 172.21.15.70:0/507056471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6481548c0 con 0x7fe64815aad0 2023-10-08T05:57:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.310+0000 7fe64da87640 1 -- 172.21.15.70:0/507056471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe648156970 con 0x7fe64815aad0 2023-10-08T05:57:07.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.311+0000 7fe64da87640 1 -- 172.21.15.70:0/507056471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe648071400 con 0x7fe64815aad0 2023-10-08T05:57:07.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.312+0000 7fe644ff9640 1 -- 172.21.15.70:0/507056471 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe640014820 con 0x7fe64815aad0 2023-10-08T05:57:07.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.313+0000 7fe644ff9640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe62806af90 0x7fe62806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.313+0000 7fe644ff9640 1 -- 172.21.15.70:0/507056471 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe64008a9a0 con 0x7fe64815aad0 2023-10-08T05:57:07.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.314+0000 7fe6477fe640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe62806af90 0x7fe62806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.315+0000 7fe6477fe640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe62806af90 0x7fe62806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe638036000 tx=0x7fe63803b040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:07.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.317+0000 7fe644ff9640 1 -- 172.21.15.70:0/507056471 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6400556c0 con 0x7fe64815aad0 2023-10-08T05:57:07.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.505+0000 7fe64da87640 1 -- 172.21.15.70:0/507056471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fe648156b60 con 0x7fe64815aad0 2023-10-08T05:57:07.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.511+0000 7fe644ff9640 1 -- 172.21.15.70:0/507056471 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v50) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe640018020 con 0x7fe64815aad0 2023-10-08T05:57:07.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.513+0000 7fe64da87640 1 -- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe62806af90 msgr2=0x7fe62806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.513+0000 7fe64da87640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe62806af90 0x7fe62806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe638036000 tx=0x7fe63803b040 comp rx=0 tx=0).stop 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.514+0000 7fe64da87640 1 -- 172.21.15.70:0/507056471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815aad0 msgr2=0x7fe648153a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.514+0000 7fe64da87640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815aad0 0x7fe648153a70 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fe64000ef90 tx=0x7fe64000c490 comp rx=0 tx=0).stop 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.514+0000 7fe64da87640 1 -- 172.21.15.70:0/507056471 shutdown_connections 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.514+0000 7fe64da87640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe62806af90 0x7fe62806d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.514+0000 7fe64da87640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815aad0 0x7fe648153a70 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.514+0000 7fe64da87640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe6480c82b0 0x7fe648157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.514+0000 7fe64da87640 1 --2- 172.21.15.70:0/507056471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe64806d410 0x7fe6480b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.514+0000 7fe64da87640 1 -- 172.21.15.70:0/507056471 >> 172.21.15.70:0/507056471 conn(0x7fe648069d50 msgr2=0x7fe648155530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.514+0000 7fe64da87640 1 -- 172.21.15.70:0/507056471 shutdown_connections 2023-10-08T05:57:07.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.514+0000 7fe64da87640 1 -- 172.21.15.70:0/507056471 wait complete. 2023-10-08T05:57:07.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T05:57:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.748+0000 7f2560b33640 1 Processor -- start 2023-10-08T05:57:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.748+0000 7f2560b33640 1 -- start start 2023-10-08T05:57:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2560b33640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c160660 0x7f255c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2560b33640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f255c0c9910 0x7f255c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2560b33640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f255c06e150 con 0x7f255c0cb450 2023-10-08T05:57:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2560b33640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c0cb8c0 con 0x7f255c0c9910 2023-10-08T05:57:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2560b33640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c06de70 con 0x7f255c160660 2023-10-08T05:57:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2553fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f255c0c9910 0x7f255c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2553fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f255c0c9910 0x7f255c0caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52486/0 (socket says 172.21.15.70:52486) 2023-10-08T05:57:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2553fff640 1 -- 172.21.15.70:0/571538656 learned_addr learned my addr 172.21.15.70:0/571538656 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f255a59c640 1 --2- 172.21.15.70:0/571538656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c160660 0x7f255c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2553fff640 1 -- 172.21.15.70:0/571538656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c160660 msgr2=0x7f255c070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2553fff640 1 --2- 172.21.15.70:0/571538656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c160660 0x7f255c070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.750+0000 7f2553fff640 1 -- 172.21.15.70:0/571538656 >> v1:172.21.15.181:6789/0 conn(0x7f255c0cb450 legacy=0x7f255c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.751+0000 7f2553fff640 1 -- 172.21.15.70:0/571538656 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0c8e30 con 0x7f255c0c9910 2023-10-08T05:57:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.752+0000 7f2553fff640 1 --2- 172.21.15.70:0/571538656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c0c9910 0x7f255c0caf10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2544009870 tx=0x7f254402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a815386071fe66ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.752+0000 7f2559d9b640 1 -- 172.21.15.70:0/571538656 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544031640 con 0x7f255c0c9910 2023-10-08T05:57:07.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.752+0000 7f2559d9b640 1 -- 172.21.15.70:0/571538656 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25440317a0 con 0x7f255c0c9910 2023-10-08T05:57:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.752+0000 7f2559d9b640 1 -- 172.21.15.70:0/571538656 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544031b30 con 0x7f255c0c9910 2023-10-08T05:57:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.753+0000 7f2560b33640 1 -- 172.21.15.70:0/571538656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c0c9910 msgr2=0x7f255c0caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.753+0000 7f2560b33640 1 --2- 172.21.15.70:0/571538656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c0c9910 0x7f255c0caf10 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f2544009870 tx=0x7f254402b8f0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.753+0000 7f2560b33640 1 -- 172.21.15.70:0/571538656 shutdown_connections 2023-10-08T05:57:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.753+0000 7f2560b33640 1 --2- 172.21.15.70:0/571538656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c0c9910 0x7f255c0caf10 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.753+0000 7f2560b33640 1 --2- 172.21.15.70:0/571538656 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c160660 0x7f255c070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.753+0000 7f2560b33640 1 -- 172.21.15.70:0/571538656 >> 172.21.15.70:0/571538656 conn(0x7f255c06a5d0 msgr2=0x7f255c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.753+0000 7f2560b33640 1 -- 172.21.15.70:0/571538656 shutdown_connections 2023-10-08T05:57:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.753+0000 7f2560b33640 1 -- 172.21.15.70:0/571538656 wait complete. 2023-10-08T05:57:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.755+0000 7f2560b33640 1 Processor -- start 2023-10-08T05:57:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.755+0000 7f2560b33640 1 -- start start 2023-10-08T05:57:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f2560b33640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c0c9910 0x7f255c147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f2560b33640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f255c0cb450 0x7f255c157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f2560b33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c160660 0x7f255c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f2560b33640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c158730 con 0x7f255c160660 2023-10-08T05:57:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f2560b33640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c1481b0 con 0x7f255c0cb450 2023-10-08T05:57:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f2560b33640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0c5090 con 0x7f255c0c9910 2023-10-08T05:57:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f255ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c160660 0x7f255c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f255ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c160660 0x7f255c157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52494/0 (socket says 172.21.15.70:52494) 2023-10-08T05:57:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f255ad9d640 1 -- 172.21.15.70:0/3792133227 learned_addr learned my addr 172.21.15.70:0/3792133227 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f255a59c640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c0c9910 0x7f255c147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f255ad9d640 1 -- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c0c9910 msgr2=0x7f255c147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f2553fff640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f255c0cb450 0x7f255c157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f255ad9d640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c0c9910 0x7f255c147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f255ad9d640 1 -- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f255c0cb450 msgr2=0x7f255c157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f255ad9d640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f255c0cb450 0x7f255c157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.756+0000 7f255ad9d640 1 -- 172.21.15.70:0/3792133227 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25440094d0 con 0x7f255c160660 2023-10-08T05:57:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.757+0000 7f255a59c640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c0c9910 0x7f255c147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.757+0000 7f255ad9d640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c160660 0x7f255c157b40 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f255400c780 tx=0x7f255400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:07.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.757+0000 7f2552ffd640 1 -- 172.21.15.70:0/3792133227 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2554004b60 con 0x7f255c160660 2023-10-08T05:57:07.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.757+0000 7f2552ffd640 1 -- 172.21.15.70:0/3792133227 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255400ce90 con 0x7f255c160660 2023-10-08T05:57:07.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.757+0000 7f2552ffd640 1 -- 172.21.15.70:0/3792133227 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255400a850 con 0x7f255c160660 2023-10-08T05:57:07.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.757+0000 7f2560b33640 1 -- 172.21.15.70:0/3792133227 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c158170 con 0x7f255c160660 2023-10-08T05:57:07.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.757+0000 7f2560b33640 1 -- 172.21.15.70:0/3792133227 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c0c2510 con 0x7f255c160660 2023-10-08T05:57:07.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.758+0000 7f2560b33640 1 -- 172.21.15.70:0/3792133227 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255c0c2370 con 0x7f255c160660 2023-10-08T05:57:07.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.759+0000 7f2552ffd640 1 -- 172.21.15.70:0/3792133227 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f255400a9b0 con 0x7f255c160660 2023-10-08T05:57:07.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.761+0000 7f2552ffd640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f253c06af90 0x7f253c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:07.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.761+0000 7f2552ffd640 1 -- 172.21.15.70:0/3792133227 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2554089b80 con 0x7f255c160660 2023-10-08T05:57:07.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.762+0000 7f255a59c640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f253c06af90 0x7f253c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:07.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.763+0000 7f255a59c640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f253c06af90 0x7f253c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f254c00b820 tx=0x7f254c008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:07.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.765+0000 7f2552ffd640 1 -- 172.21.15.70:0/3792133227 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f255c0c2370 con 0x7f255c160660 2023-10-08T05:57:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.954+0000 7f2560b33640 1 -- 172.21.15.70:0/3792133227 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f255c0c90f0 con 0x7f255c160660 2023-10-08T05:57:07.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.960+0000 7f2552ffd640 1 -- 172.21.15.70:0/3792133227 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v51) v1 ==== 120+0+0 (secure 0 0 0) 0x7f255c0c90f0 con 0x7f255c160660 2023-10-08T05:57:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 -- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f253c06af90 msgr2=0x7f253c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f253c06af90 0x7f253c06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f254c00b820 tx=0x7f254c008040 comp rx=0 tx=0).stop 2023-10-08T05:57:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 -- 172.21.15.70:0/3792133227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c160660 msgr2=0x7f255c157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c160660 0x7f255c157b40 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f255400c780 tx=0x7f255400cc40 comp rx=0 tx=0).stop 2023-10-08T05:57:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 -- 172.21.15.70:0/3792133227 shutdown_connections 2023-10-08T05:57:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f253c06af90 0x7f253c06d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f255c160660 0x7f255c157b40 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f255c0cb450 0x7f255c157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 --2- 172.21.15.70:0/3792133227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f255c0c9910 0x7f255c147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:07.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 -- 172.21.15.70:0/3792133227 >> 172.21.15.70:0/3792133227 conn(0x7f255c06a5d0 msgr2=0x7f255c15c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:07.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 -- 172.21.15.70:0/3792133227 shutdown_connections 2023-10-08T05:57:07.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:07.963+0000 7f2560b33640 1 -- 172.21.15.70:0/3792133227 wait complete. 2023-10-08T05:57:07.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods' 2023-10-08T05:57:08.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.189+0000 7feb0f59e640 1 Processor -- start 2023-10-08T05:57:08.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.190+0000 7feb0f59e640 1 -- start start 2023-10-08T05:57:08.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.191+0000 7feb0f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb101599d0 0x7feb100ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.191+0000 7feb0f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feb100cc210 0x7feb100b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.191+0000 7feb0f59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feb10159db0 0x7feb100b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.191+0000 7feb0f59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb100b10f0 con 0x7feb100cc210 2023-10-08T05:57:08.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.191+0000 7feb0f59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb100cacf0 con 0x7feb10159db0 2023-10-08T05:57:08.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.191+0000 7feb0f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb100ca3c0 con 0x7feb101599d0 2023-10-08T05:57:08.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.191+0000 7feb05d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feb100cc210 0x7feb100b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:08.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.192+0000 7feb05d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feb100cc210 0x7feb100b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52504/0 (socket says 172.21.15.70:52504) 2023-10-08T05:57:08.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.192+0000 7feb05d9b640 1 -- 172.21.15.70:0/704410395 learned_addr learned my addr 172.21.15.70:0/704410395 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:08.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.192+0000 7feb0ed9d640 1 --2- 172.21.15.70:0/704410395 >> v2:172.21.15.181:3300/0 conn(0x7feb10159db0 0x7feb100b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.192+0000 7feb05d9b640 1 -- 172.21.15.70:0/704410395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb101599d0 msgr2=0x7feb100ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.192+0000 7feb05d9b640 1 --2- 172.21.15.70:0/704410395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb101599d0 0x7feb100ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.192+0000 7feb05d9b640 1 -- 172.21.15.70:0/704410395 >> v2:172.21.15.181:3300/0 conn(0x7feb10159db0 msgr2=0x7feb100b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.192+0000 7feb05d9b640 1 --2- 172.21.15.70:0/704410395 >> v2:172.21.15.181:3300/0 conn(0x7feb10159db0 0x7feb100b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.192+0000 7feb05d9b640 1 -- 172.21.15.70:0/704410395 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb100ca6a0 con 0x7feb100cc210 2023-10-08T05:57:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.193+0000 7feb05d9b640 1 --2- 172.21.15.70:0/704410395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 0x7feb100b00c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7feb100b4430 tx=0x7feaf802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174af76a1849ee6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.193+0000 7feb0dd9b640 1 -- 172.21.15.70:0/704410395 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf802ca80 con 0x7feb100cc210 2023-10-08T05:57:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.193+0000 7feb0dd9b640 1 -- 172.21.15.70:0/704410395 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf802cbe0 con 0x7feb100cc210 2023-10-08T05:57:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.193+0000 7feb0dd9b640 1 -- 172.21.15.70:0/704410395 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80343f0 con 0x7feb100cc210 2023-10-08T05:57:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.194+0000 7feb0f59e640 1 -- 172.21.15.70:0/704410395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 msgr2=0x7feb100b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.194+0000 7feb0f59e640 1 --2- 172.21.15.70:0/704410395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 0x7feb100b00c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7feb100b4430 tx=0x7feaf802bee0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.194+0000 7feb0f59e640 1 -- 172.21.15.70:0/704410395 shutdown_connections 2023-10-08T05:57:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.194+0000 7feb0f59e640 1 --2- 172.21.15.70:0/704410395 >> v2:172.21.15.181:3300/0 conn(0x7feb10159db0 0x7feb100b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.194+0000 7feb0f59e640 1 --2- 172.21.15.70:0/704410395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 0x7feb100b00c0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.194+0000 7feb0f59e640 1 --2- 172.21.15.70:0/704410395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb101599d0 0x7feb100ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.194+0000 7feb0f59e640 1 -- 172.21.15.70:0/704410395 >> 172.21.15.70:0/704410395 conn(0x7feb100cb0c0 msgr2=0x7feb100cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.194+0000 7feb0f59e640 1 -- 172.21.15.70:0/704410395 shutdown_connections 2023-10-08T05:57:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.194+0000 7feb0f59e640 1 -- 172.21.15.70:0/704410395 wait complete. 2023-10-08T05:57:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.196+0000 7feb0f59e640 1 Processor -- start 2023-10-08T05:57:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.196+0000 7feb0f59e640 1 -- start start 2023-10-08T05:57:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 0x7feb10151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb101599d0 0x7feb100c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10159db0 0x7feb100c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb100c5920 con 0x7feb100cc210 2023-10-08T05:57:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0f59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb100c5c00 con 0x7feb10159db0 2023-10-08T05:57:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb100c5ee0 con 0x7feb101599d0 2023-10-08T05:57:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 0x7feb10151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 0x7feb10151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52514/0 (socket says 172.21.15.70:52514) 2023-10-08T05:57:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0e59c640 1 -- 172.21.15.70:0/2014068414 learned_addr learned my addr 172.21.15.70:0/2014068414 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0e59c640 1 -- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb101599d0 msgr2=0x7feb100c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0e59c640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb101599d0 0x7feb100c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0e59c640 1 -- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10159db0 msgr2=0x7feb100c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0e59c640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10159db0 0x7feb100c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.197+0000 7feb0e59c640 1 -- 172.21.15.70:0/2014068414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf8009b30 con 0x7feb100cc210 2023-10-08T05:57:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.198+0000 7feb0e59c640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 0x7feb10151a80 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7feb0000b4a0 tx=0x7feb0000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.198+0000 7feb077fe640 1 -- 172.21.15.70:0/2014068414 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00010610 con 0x7feb100cc210 2023-10-08T05:57:08.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.198+0000 7feb077fe640 1 -- 172.21.15.70:0/2014068414 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb00010770 con 0x7feb100cc210 2023-10-08T05:57:08.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.198+0000 7feb0f59e640 1 -- 172.21.15.70:0/2014068414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb10152a60 con 0x7feb100cc210 2023-10-08T05:57:08.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.198+0000 7feb077fe640 1 -- 172.21.15.70:0/2014068414 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00010b20 con 0x7feb100cc210 2023-10-08T05:57:08.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.198+0000 7feb0f59e640 1 -- 172.21.15.70:0/2014068414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb10153020 con 0x7feb100cc210 2023-10-08T05:57:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.199+0000 7feb0f59e640 1 -- 172.21.15.70:0/2014068414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb1004e160 con 0x7feb100cc210 2023-10-08T05:57:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.201+0000 7feb077fe640 1 -- 172.21.15.70:0/2014068414 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feb00010c80 con 0x7feb100cc210 2023-10-08T05:57:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.202+0000 7feb077fe640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feaf006afc0 0x7feaf006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.202+0000 7feb077fe640 1 -- 172.21.15.70:0/2014068414 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6066+0+0 (secure 0 0 0) 0x7feb00089520 con 0x7feb100cc210 2023-10-08T05:57:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.202+0000 7feb05d9b640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feaf006afc0 0x7feaf006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:08.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.203+0000 7feb05d9b640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feaf006afc0 0x7feaf006d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feaf8035000 tx=0x7feaf802e000 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:08.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.206+0000 7feb077fe640 1 -- 172.21.15.70:0/2014068414 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb00054350 con 0x7feb100cc210 2023-10-08T05:57:08.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.461+0000 7feb0f59e640 1 -- 172.21.15.70:0/2014068414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]} v 0) v1 -- 0x7feb101641c0 con 0x7feb100cc210 2023-10-08T05:57:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.642+0000 7feb077fe640 1 -- 172.21.15.70:0/2014068414 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_all_periods"]}]=0 v826) v1 ==== 160+0+0 (secure 0 0 0) 0x7feb00057970 con 0x7feb100cc210 2023-10-08T05:57:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 -- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feaf006afc0 msgr2=0x7feaf006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feaf006afc0 0x7feaf006d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7feaf8035000 tx=0x7feaf802e000 comp rx=0 tx=0).stop 2023-10-08T05:57:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 -- 172.21.15.70:0/2014068414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 msgr2=0x7feb10151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 0x7feb10151a80 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7feb0000b4a0 tx=0x7feb0000b960 comp rx=0 tx=0).stop 2023-10-08T05:57:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 -- 172.21.15.70:0/2014068414 shutdown_connections 2023-10-08T05:57:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7feaf006afc0 0x7feaf006d480 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feb10159db0 0x7feb100c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feb101599d0 0x7feb100c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 --2- 172.21.15.70:0/2014068414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb100cc210 0x7feb10151a80 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 -- 172.21.15.70:0/2014068414 >> 172.21.15.70:0/2014068414 conn(0x7feb100cb0c0 msgr2=0x7feb100ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 -- 172.21.15.70:0/2014068414 shutdown_connections 2023-10-08T05:57:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.647+0000 7feb0f59e640 1 -- 172.21.15.70:0/2014068414 wait complete. 2023-10-08T05:57:08.669 INFO:tasks.cephfs_test_runner:test_snap_schedule_all_periods (tasks.cephfs.test_snap_schedules.TestSnapSchedules) ... ok 2023-10-08T05:57:08.669 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-10-08T05:57:08.669 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T05:57:08.670 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:57:08.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.886+0000 7f1af0e09640 1 Processor -- start 2023-10-08T05:57:08.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.886+0000 7f1af0e09640 1 -- start start 2023-10-08T05:57:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.888+0000 7f1af0e09640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 0x7f1aec0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.888+0000 7f1af0e09640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1aec0cb3b0 0x7f1aec15e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.888+0000 7f1af0e09640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec15b2b0 con 0x7f1aec15ab40 2023-10-08T05:57:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.888+0000 7f1af0e09640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aec15bf30 con 0x7f1aec0cb3b0 2023-10-08T05:57:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.888+0000 7f1af0e09640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0cb840 con 0x7f1aec0c90b0 2023-10-08T05:57:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.889+0000 7f1aead9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1aec0cb3b0 0x7f1aec15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.889+0000 7f1aead9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1aec0cb3b0 0x7f1aec15e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52526/0 (socket says 172.21.15.70:52526) 2023-10-08T05:57:08.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.889+0000 7f1aead9d640 1 -- 172.21.15.70:0/2354985993 learned_addr learned my addr 172.21.15.70:0/2354985993 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:08.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.889+0000 7f1ae9d9b640 1 --2- 172.21.15.70:0/2354985993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 0x7f1aec0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:08.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.889+0000 7f1aead9d640 1 -- 172.21.15.70:0/2354985993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 msgr2=0x7f1aec0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:08.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.889+0000 7f1aead9d640 1 --2- 172.21.15.70:0/2354985993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 0x7f1aec0cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.889+0000 7f1aead9d640 1 -- 172.21.15.70:0/2354985993 >> v1:172.21.15.181:6789/0 conn(0x7f1aec15ab40 legacy=0x7f1aec0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.889+0000 7f1aead9d640 1 -- 172.21.15.70:0/2354985993 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec15b590 con 0x7f1aec0cb3b0 2023-10-08T05:57:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.891+0000 7f1aead9d640 1 --2- 172.21.15.70:0/2354985993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec0cb3b0 0x7f1aec15e790 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009e50 tx=0x7f1ae402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cba156206f62b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.891+0000 7f1ae959a640 1 -- 172.21.15.70:0/2354985993 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae402cb00 con 0x7f1aec0cb3b0 2023-10-08T05:57:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.891+0000 7f1ae959a640 1 -- 172.21.15.70:0/2354985993 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae402cc60 con 0x7f1aec0cb3b0 2023-10-08T05:57:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.891+0000 7f1ae959a640 1 -- 172.21.15.70:0/2354985993 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40343f0 con 0x7f1aec0cb3b0 2023-10-08T05:57:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.892+0000 7f1af0e09640 1 -- 172.21.15.70:0/2354985993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec0cb3b0 msgr2=0x7f1aec15e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.892+0000 7f1af0e09640 1 --2- 172.21.15.70:0/2354985993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec0cb3b0 0x7f1aec15e790 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009e50 tx=0x7f1ae402bf60 comp rx=0 tx=0).stop 2023-10-08T05:57:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.892+0000 7f1af0e09640 1 -- 172.21.15.70:0/2354985993 shutdown_connections 2023-10-08T05:57:08.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.892+0000 7f1af0e09640 1 --2- 172.21.15.70:0/2354985993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec0cb3b0 0x7f1aec15e790 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.892+0000 7f1af0e09640 1 --2- 172.21.15.70:0/2354985993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 0x7f1aec0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.892+0000 7f1af0e09640 1 -- 172.21.15.70:0/2354985993 >> 172.21.15.70:0/2354985993 conn(0x7f1aec06a380 msgr2=0x7f1aec0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:08.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.892+0000 7f1af0e09640 1 -- 172.21.15.70:0/2354985993 shutdown_connections 2023-10-08T05:57:08.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.892+0000 7f1af0e09640 1 -- 172.21.15.70:0/2354985993 wait complete. 2023-10-08T05:57:08.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.894+0000 7f1af0e09640 1 Processor -- start 2023-10-08T05:57:08.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.894+0000 7f1af0e09640 1 -- start start 2023-10-08T05:57:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.895+0000 7f1af0e09640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 0x7f1aec151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.895+0000 7f1af0e09640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aec0cb3b0 0x7f1aec0b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.895+0000 7f1af0e09640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec15ab40 0x7f1aec0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.895+0000 7f1af0e09640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec1f02e0 con 0x7f1aec15ab40 2023-10-08T05:57:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.895+0000 7f1af0e09640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec156c70 con 0x7f1aec0cb3b0 2023-10-08T05:57:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.895+0000 7f1af0e09640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec156f10 con 0x7f1aec0c90b0 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.895+0000 7f1aead9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec15ab40 0x7f1aec0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.895+0000 7f1aead9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec15ab40 0x7f1aec0b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52542/0 (socket says 172.21.15.70:52542) 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.895+0000 7f1aead9d640 1 -- 172.21.15.70:0/941372354 learned_addr learned my addr 172.21.15.70:0/941372354 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1aea59c640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 0x7f1aec151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1aead9d640 1 -- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 msgr2=0x7f1aec151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1aead9d640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 0x7f1aec151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1aead9d640 1 -- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aec0cb3b0 msgr2=0x7f1aec0b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1aead9d640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aec0cb3b0 0x7f1aec0b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1aead9d640 1 -- 172.21.15.70:0/941372354 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae4009b30 con 0x7f1aec15ab40 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1aea59c640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 0x7f1aec151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1aead9d640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec15ab40 0x7f1aec0b3270 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009e50 tx=0x7f1ae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:08.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1adaffd640 1 -- 172.21.15.70:0/941372354 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4009510 con 0x7f1aec15ab40 2023-10-08T05:57:08.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1adaffd640 1 -- 172.21.15.70:0/941372354 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae403e450 con 0x7f1aec15ab40 2023-10-08T05:57:08.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1adaffd640 1 -- 172.21.15.70:0/941372354 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae403e800 con 0x7f1aec15ab40 2023-10-08T05:57:08.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.896+0000 7f1af0e09640 1 -- 172.21.15.70:0/941372354 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec1571f0 con 0x7f1aec15ab40 2023-10-08T05:57:08.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.897+0000 7f1af0e09640 1 -- 172.21.15.70:0/941372354 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec1577b0 con 0x7f1aec15ab40 2023-10-08T05:57:08.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.898+0000 7f1af0e09640 1 -- 172.21.15.70:0/941372354 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aec067490 con 0x7f1aec15ab40 2023-10-08T05:57:08.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.899+0000 7f1adaffd640 1 -- 172.21.15.70:0/941372354 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1ae403e960 con 0x7f1aec15ab40 2023-10-08T05:57:08.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.899+0000 7f1adaffd640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1acc06af90 0x7f1acc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:08.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.900+0000 7f1adaffd640 1 -- 172.21.15.70:0/941372354 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1ae40ac6d0 con 0x7f1aec15ab40 2023-10-08T05:57:08.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.900+0000 7f1aea59c640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1acc06af90 0x7f1acc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:08.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.902+0000 7f1aea59c640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1acc06af90 0x7f1acc06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1adc000f60 tx=0x7f1adc009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:08.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:08.904+0000 7f1adaffd640 1 -- 172.21.15.70:0/941372354 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ae40774f0 con 0x7f1aec15ab40 2023-10-08T05:57:09.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.098+0000 7f1af0e09640 1 -- 172.21.15.70:0/941372354 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aec0b0ef0 con 0x7f1aec15ab40 2023-10-08T05:57:09.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.101+0000 7f1adaffd640 1 -- 172.21.15.70:0/941372354 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+14383 (secure 0 0 0) 0x7f1ae407ab10 con 0x7f1aec15ab40 2023-10-08T05:57:09.102 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:09.102 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":154,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:57:06.623020+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3419396085":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/3917000413":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/3217258198":"2023-10-09T05:56:59.566981+0000","172.21.15.70:6833/3869116018":"2023-10-09T05:57:03.418313+0000","172.21.15.181:0/145587121":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/402795356":"2023-10-09T05:56:59.566981+0000","172.21.15.181:6833/1964243521":"2023-10-09T05:56:59.566981+0000","172.21.15.70:6835/3869116018":"2023-10-09T05:57:03.418313+0000","172.21.15.181:6832/1964243521":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/413304125":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/1640710135":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/1739083188":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/202771123":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/3030439524":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/2088091464":"2023-10-09T05:56:59.566981+0000","172.21.15.181:0/4192220981":"2023-10-09T05:56:46.566167+0000","172.21.15.181:0/3672243748":"2023-10-09T05:56:46.566167+0000","172.21.15.181:6833/3857845516":"2023-10-09T05:56:46.566167+0000","172.21.15.181:6832/3857845516":"2023-10-09T05:56:46.566167+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:57:09.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.103+0000 7f1af0e09640 1 -- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1acc06af90 msgr2=0x7f1acc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.103+0000 7f1af0e09640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1acc06af90 0x7f1acc06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1adc000f60 tx=0x7f1adc009040 comp rx=0 tx=0).stop 2023-10-08T05:57:09.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.103+0000 7f1af0e09640 1 -- 172.21.15.70:0/941372354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec15ab40 msgr2=0x7f1aec0b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.103+0000 7f1af0e09640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec15ab40 0x7f1aec0b3270 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009e50 tx=0x7f1ae4004a40 comp rx=0 tx=0).stop 2023-10-08T05:57:09.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.104+0000 7f1af0e09640 1 -- 172.21.15.70:0/941372354 shutdown_connections 2023-10-08T05:57:09.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.104+0000 7f1af0e09640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1acc06af90 0x7f1acc06d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.104+0000 7f1af0e09640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1aec15ab40 0x7f1aec0b3270 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.104+0000 7f1af0e09640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1aec0cb3b0 0x7f1aec0b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.104+0000 7f1af0e09640 1 --2- 172.21.15.70:0/941372354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1aec0c90b0 0x7f1aec151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.104+0000 7f1af0e09640 1 -- 172.21.15.70:0/941372354 >> 172.21.15.70:0/941372354 conn(0x7f1aec06a380 msgr2=0x7f1aec06d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:09.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.104+0000 7f1af0e09640 1 -- 172.21.15.70:0/941372354 shutdown_connections 2023-10-08T05:57:09.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.104+0000 7f1af0e09640 1 -- 172.21.15.70:0/941372354 wait complete. 2023-10-08T05:57:09.125 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:57:09.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:57:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.349+0000 7f7fd67fb640 1 Processor -- start 2023-10-08T05:57:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.349+0000 7f7fd67fb640 1 -- start start 2023-10-08T05:57:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fd67fb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7fd015c890 0x7f7fd00cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fd67fb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd00714a0 con 0x7f7fd00cfab0 2023-10-08T05:57:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fd67fb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd0071780 con 0x7f7fd015d380 2023-10-08T05:57:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fd67fb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd0160590 con 0x7f7fd015c890 2023-10-08T05:57:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fd4ff8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7fd00cfab0 0x7f7fd015cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46324/0 (socket says 172.21.15.70:46324) 2023-10-08T05:57:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fd4ff8640 1 -- 172.21.15.70:0/2235702648 learned_addr learned my addr 172.21.15.70:0/2235702648 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fd57f9640 1 --2- 172.21.15.70:0/2235702648 >> v2:172.21.15.181:3300/0 conn(0x7f7fd015c890 0x7f7fd00cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555143402 0 0) 0x7f7fd00714a0 con 0x7f7fd00cfab0 2023-10-08T05:57:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fbc003520 con 0x7f7fd00cfab0 2023-10-08T05:57:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 770959944 0 0) 0x7f7fbc003520 con 0x7f7fd00cfab0 2023-10-08T05:57:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 >> v2:172.21.15.181:3300/0 conn(0x7f7fd015c890 msgr2=0x7f7fd00cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fc7fff640 1 --2- 172.21.15.70:0/2235702648 >> v2:172.21.15.181:3300/0 conn(0x7f7fd015c890 0x7f7fd00cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 >> v1:172.21.15.181:6789/0 conn(0x7f7fd015d380 legacy=0x7f7fd01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd0071a60 con 0x7f7fd00cfab0 2023-10-08T05:57:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7fc0001670 con 0x7f7fd00cfab0 2023-10-08T05:57:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 >> v1:172.21.15.70:6789/0 conn(0x7f7fd00cfab0 legacy=0x7f7fd015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.351+0000 7f7fc7fff640 1 --2- 172.21.15.70:0/2235702648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fbc003970 0x7f7fbc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fc7fff640 1 --2- 172.21.15.70:0/2235702648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fbc0047c0 0x7f7fbc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fc7fff640 1 --2- 172.21.15.70:0/2235702648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fbc0052a0 0x7f7fbc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd0160590 con 0x7f7fbc0047c0 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc00c170 con 0x7f7fbc003970 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc00c330 con 0x7f7fbc0052a0 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fd4ff8640 1 --2- 172.21.15.70:0/2235702648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fbc0047c0 0x7f7fbc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fd4ff8640 1 -- 172.21.15.70:0/2235702648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fbc0052a0 msgr2=0x7f7fbc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fd4ff8640 1 --2- 172.21.15.70:0/2235702648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fbc0052a0 0x7f7fbc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fd4ff8640 1 -- 172.21.15.70:0/2235702648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fbc003970 msgr2=0x7f7fbc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:09.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fd57f9640 1 --2- 172.21.15.70:0/2235702648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fbc003970 0x7f7fbc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fd4ff8640 1 --2- 172.21.15.70:0/2235702648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fbc003970 0x7f7fbc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fd4ff8640 1 -- 172.21.15.70:0/2235702648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc003520 con 0x7f7fbc0047c0 2023-10-08T05:57:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.352+0000 7f7fd5ffa640 1 -- 172.21.15.70:0/2235702648 reap_dead start 2023-10-08T05:57:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.353+0000 7f7fd4ff8640 1 --2- 172.21.15.70:0/2235702648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fbc0047c0 0x7f7fbc004c40 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000c210 tx=0x7f7fbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1647abc51d9f16c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.353+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0005ad0 con 0x7f7fbc0047c0 2023-10-08T05:57:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.354+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc0005c30 con 0x7f7fbc0047c0 2023-10-08T05:57:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.354+0000 7f7fc7fff640 1 -- 172.21.15.70:0/2235702648 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc002cab0 con 0x7f7fbc0047c0 2023-10-08T05:57:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.354+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2235702648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fbc0047c0 msgr2=0x7f7fbc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.354+0000 7f7fd67fb640 1 --2- 172.21.15.70:0/2235702648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fbc0047c0 0x7f7fbc004c40 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000c210 tx=0x7f7fbc0057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.354+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2235702648 shutdown_connections 2023-10-08T05:57:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.354+0000 7f7fd67fb640 1 --2- 172.21.15.70:0/2235702648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fbc0047c0 0x7f7fbc004c40 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.354+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2235702648 >> 172.21.15.70:0/2235702648 conn(0x7f7fd006b250 msgr2=0x7f7fd006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.354+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2235702648 shutdown_connections 2023-10-08T05:57:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.354+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2235702648 wait complete. 2023-10-08T05:57:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.356+0000 7f7fd67fb640 1 Processor -- start 2023-10-08T05:57:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.356+0000 7f7fd67fb640 1 -- start start 2023-10-08T05:57:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.357+0000 7f7fd67fb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fbc0047c0 0x7f7fd00cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.357+0000 7f7fd67fb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd00c9240 0x7f7fd0159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.357+0000 7f7fd67fb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fd00cd190 0x7f7fd00cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.357+0000 7f7fd67fb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd00cdb50 con 0x7f7fd00c9240 2023-10-08T05:57:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.357+0000 7f7fd67fb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd00ad9d0 con 0x7f7fbc0047c0 2023-10-08T05:57:09.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.357+0000 7f7fd67fb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd00c97b0 con 0x7f7fd00cd190 2023-10-08T05:57:09.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.357+0000 7f7fd4ff8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd00c9240 0x7f7fd0159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.357+0000 7f7fd4ff8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd00c9240 0x7f7fd0159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52566/0 (socket says 172.21.15.70:52566) 2023-10-08T05:57:09.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.358+0000 7f7fd4ff8640 1 -- 172.21.15.70:0/2497777566 learned_addr learned my addr 172.21.15.70:0/2497777566 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:09.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.358+0000 7f7fd5ffa640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fd00cd190 0x7f7fd00cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.358+0000 7f7fd57f9640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fbc0047c0 0x7f7fd00cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.358+0000 7f7fd4ff8640 1 -- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fd00cd190 msgr2=0x7f7fd00cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.358+0000 7f7fd4ff8640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fd00cd190 0x7f7fd00cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.358+0000 7f7fd4ff8640 1 -- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fbc0047c0 msgr2=0x7f7fd00cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.358+0000 7f7fd4ff8640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fbc0047c0 0x7f7fd00cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.358+0000 7f7fd4ff8640 1 -- 172.21.15.70:0/2497777566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd00adca0 con 0x7f7fd00c9240 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.358+0000 7f7fd4ff8640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd00c9240 0x7f7fd0159920 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0001930 tx=0x7f7fc002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.359+0000 7f7fc5ffb640 1 -- 172.21.15.70:0/2497777566 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc003c040 con 0x7f7fd00c9240 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.359+0000 7f7fc5ffb640 1 -- 172.21.15.70:0/2497777566 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc002cd40 con 0x7f7fd00c9240 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.359+0000 7f7fc5ffb640 1 -- 172.21.15.70:0/2497777566 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00072b0 con 0x7f7fd00c9240 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.359+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2497777566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd00adf80 con 0x7f7fd00c9240 2023-10-08T05:57:09.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.359+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2497777566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd00cb7a0 con 0x7f7fd00c9240 2023-10-08T05:57:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.360+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2497777566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd00ae5d0 con 0x7f7fd00c9240 2023-10-08T05:57:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.361+0000 7f7fc5ffb640 1 -- 172.21.15.70:0/2497777566 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7fc0032070 con 0x7f7fd00c9240 2023-10-08T05:57:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.362+0000 7f7fc5ffb640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f7fb406afc0 0x7f7fb406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.362+0000 7f7fc5ffb640 1 -- 172.21.15.70:0/2497777566 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7fc00aae00 con 0x7f7fd00c9240 2023-10-08T05:57:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.362+0000 7f7fd57f9640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f7fb406afc0 0x7f7fb406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.364+0000 7f7fd57f9640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f7fb406afc0 0x7f7fb406d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7fd004fa60 tx=0x7f7fcc008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:09.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.366+0000 7f7fc5ffb640 1 -- 172.21.15.70:0/2497777566 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7fd00ae5d0 con 0x7f7fd00c9240 2023-10-08T05:57:09.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.556+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2497777566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fd00ae5d0 con 0x7f7fd00c9240 2023-10-08T05:57:09.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.557+0000 7f7fc5ffb640 1 -- 172.21.15.70:0/2497777566 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fd00ae5d0 con 0x7f7fd00c9240 2023-10-08T05:57:09.559 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:57:09.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f7fb406afc0 msgr2=0x7f7fb406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f7fb406afc0 0x7f7fb406d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7fd004fa60 tx=0x7f7fcc008040 comp rx=0 tx=0).stop 2023-10-08T05:57:09.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2497777566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd00c9240 msgr2=0x7f7fd0159920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd00c9240 0x7f7fd0159920 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0001930 tx=0x7f7fc002e070 comp rx=0 tx=0).stop 2023-10-08T05:57:09.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2497777566 shutdown_connections 2023-10-08T05:57:09.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f7fb406afc0 0x7f7fb406d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fd00cd190 0x7f7fd00cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd00c9240 0x7f7fd0159920 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 --2- 172.21.15.70:0/2497777566 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fbc0047c0 0x7f7fd00cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2497777566 >> 172.21.15.70:0/2497777566 conn(0x7f7fd006b250 msgr2=0x7f7fd00cc040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:09.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2497777566 shutdown_connections 2023-10-08T05:57:09.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.560+0000 7f7fd67fb640 1 -- 172.21.15.70:0/2497777566 wait complete. 2023-10-08T05:57:09.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-08T05:57:09.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.783+0000 7f3f78e46640 1 Processor -- start 2023-10-08T05:57:09.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.783+0000 7f3f78e46640 1 -- start start 2023-10-08T05:57:09.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f78e46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f7415cd90 0x7f3f740702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f78e46640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f74160940 0x7f3f74070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f78e46640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f740c48f0 con 0x7f3f74070d50 2023-10-08T05:57:09.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f78e46640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f740c4d80 con 0x7f3f7415cd90 2023-10-08T05:57:09.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f78e46640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f740c3ce0 con 0x7f3f74160940 2023-10-08T05:57:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f7259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f7415cd90 0x7f3f740702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f7259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f7415cd90 0x7f3f740702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52582/0 (socket says 172.21.15.70:52582) 2023-10-08T05:57:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f72d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3f74070d50 0x7f3f740cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46340/0 (socket says 172.21.15.70:46340) 2023-10-08T05:57:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f7259c640 1 -- 172.21.15.70:0/4001947734 learned_addr learned my addr 172.21.15.70:0/4001947734 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f6bfff640 1 --2- 172.21.15.70:0/4001947734 >> v2:172.21.15.181:3300/0 conn(0x7f3f74160940 0x7f3f74070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f7259c640 1 -- 172.21.15.70:0/4001947734 >> v2:172.21.15.181:3300/0 conn(0x7f3f74160940 msgr2=0x7f3f74070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f7259c640 1 --2- 172.21.15.70:0/4001947734 >> v2:172.21.15.181:3300/0 conn(0x7f3f74160940 0x7f3f74070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f7259c640 1 -- 172.21.15.70:0/4001947734 >> v1:172.21.15.70:6789/0 conn(0x7f3f74070d50 legacy=0x7f3f740cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f7259c640 1 -- 172.21.15.70:0/4001947734 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740c3fc0 con 0x7f3f7415cd90 2023-10-08T05:57:09.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.785+0000 7f3f6bfff640 1 --2- 172.21.15.70:0/4001947734 >> v2:172.21.15.181:3300/0 conn(0x7f3f74160940 0x7f3f74070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:09.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.786+0000 7f3f7259c640 1 --2- 172.21.15.70:0/4001947734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f7415cd90 0x7f3f740702d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f3f740cc090 tx=0x7f3f6402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99574a10fe2484bf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:09.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.786+0000 7f3f71d9b640 1 -- 172.21.15.70:0/4001947734 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64030870 con 0x7f3f7415cd90 2023-10-08T05:57:09.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.787+0000 7f3f71d9b640 1 -- 172.21.15.70:0/4001947734 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f640309d0 con 0x7f3f7415cd90 2023-10-08T05:57:09.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.787+0000 7f3f71d9b640 1 -- 172.21.15.70:0/4001947734 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64030d60 con 0x7f3f7415cd90 2023-10-08T05:57:09.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.787+0000 7f3f78e46640 1 -- 172.21.15.70:0/4001947734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f7415cd90 msgr2=0x7f3f740702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.787+0000 7f3f78e46640 1 --2- 172.21.15.70:0/4001947734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f7415cd90 0x7f3f740702d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f3f740cc090 tx=0x7f3f6402b9b0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.787+0000 7f3f78e46640 1 -- 172.21.15.70:0/4001947734 shutdown_connections 2023-10-08T05:57:09.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.787+0000 7f3f78e46640 1 --2- 172.21.15.70:0/4001947734 >> v2:172.21.15.181:3300/0 conn(0x7f3f74160940 0x7f3f74070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.787+0000 7f3f78e46640 1 --2- 172.21.15.70:0/4001947734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f7415cd90 0x7f3f740702d0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.787+0000 7f3f78e46640 1 -- 172.21.15.70:0/4001947734 >> 172.21.15.70:0/4001947734 conn(0x7f3f7406a380 msgr2=0x7f3f740b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:09.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.788+0000 7f3f78e46640 1 -- 172.21.15.70:0/4001947734 shutdown_connections 2023-10-08T05:57:09.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.788+0000 7f3f78e46640 1 -- 172.21.15.70:0/4001947734 wait complete. 2023-10-08T05:57:09.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.789+0000 7f3f78e46640 1 Processor -- start 2023-10-08T05:57:09.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.790+0000 7f3f78e46640 1 -- start start 2023-10-08T05:57:09.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.791+0000 7f3f78e46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74070d50 0x7f3f741f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.791+0000 7f3f78e46640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7415cd90 0x7f3f740cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.791+0000 7f3f78e46640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f74160940 0x7f3f74153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.791+0000 7f3f78e46640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f741f3320 con 0x7f3f74070d50 2023-10-08T05:57:09.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.791+0000 7f3f78e46640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f741f60b0 con 0x7f3f7415cd90 2023-10-08T05:57:09.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.791+0000 7f3f78e46640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740b3dc0 con 0x7f3f74160940 2023-10-08T05:57:09.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.791+0000 7f3f7259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74070d50 0x7f3f741f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.791+0000 7f3f7259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74070d50 0x7f3f741f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52588/0 (socket says 172.21.15.70:52588) 2023-10-08T05:57:09.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.791+0000 7f3f7259c640 1 -- 172.21.15.70:0/3484664377 learned_addr learned my addr 172.21.15.70:0/3484664377 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:09.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f6bfff640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7415cd90 0x7f3f740cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f7259c640 1 -- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f74160940 msgr2=0x7f3f74153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f72d9d640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f74160940 0x7f3f74153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f7259c640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f74160940 0x7f3f74153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f7259c640 1 -- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7415cd90 msgr2=0x7f3f740cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:09.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f7259c640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7415cd90 0x7f3f740cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:09.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f7259c640 1 -- 172.21.15.70:0/3484664377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f640094d0 con 0x7f3f74070d50 2023-10-08T05:57:09.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f7259c640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74070d50 0x7f3f741f2de0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f3f64000c00 tx=0x7f3f64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:09.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f6affd640 1 -- 172.21.15.70:0/3484664377 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64039d40 con 0x7f3f74070d50 2023-10-08T05:57:09.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f6affd640 1 -- 172.21.15.70:0/3484664377 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f64043400 con 0x7f3f74070d50 2023-10-08T05:57:09.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f6affd640 1 -- 172.21.15.70:0/3484664377 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64039ea0 con 0x7f3f74070d50 2023-10-08T05:57:09.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f78e46640 1 -- 172.21.15.70:0/3484664377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f740b3420 con 0x7f3f74070d50 2023-10-08T05:57:09.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.792+0000 7f3f78e46640 1 -- 172.21.15.70:0/3484664377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f741f5a60 con 0x7f3f74070d50 2023-10-08T05:57:09.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.793+0000 7f3f78e46640 1 -- 172.21.15.70:0/3484664377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f74067490 con 0x7f3f74070d50 2023-10-08T05:57:09.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.794+0000 7f3f6affd640 1 -- 172.21.15.70:0/3484664377 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3f640437b0 con 0x7f3f74070d50 2023-10-08T05:57:09.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.795+0000 7f3f6affd640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f3f5406afc0 0x7f3f5406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:09.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.795+0000 7f3f6affd640 1 -- 172.21.15.70:0/3484664377 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f3f6403e070 con 0x7f3f74070d50 2023-10-08T05:57:09.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.796+0000 7f3f6bfff640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f3f5406afc0 0x7f3f5406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:09.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.797+0000 7f3f6bfff640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f3f5406afc0 0x7f3f5406d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3f741603c0 tx=0x7f3f5c008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:09.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:09.801+0000 7f3f6affd640 1 -- 172.21.15.70:0/3484664377 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f64077a50 con 0x7f3f74070d50 2023-10-08T05:57:10.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.060+0000 7f3f78e46640 1 -- 172.21.15.70:0/3484664377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f3f740b0ef0 con 0x7f3f74070d50 2023-10-08T05:57:10.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.657+0000 7f3f6affd640 1 -- 172.21.15.70:0/3484664377 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v828) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3f6407b070 con 0x7f3f74070d50 2023-10-08T05:57:10.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 -- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f3f5406afc0 msgr2=0x7f3f5406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:10.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f3f5406afc0 0x7f3f5406d480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3f741603c0 tx=0x7f3f5c008040 comp rx=0 tx=0).stop 2023-10-08T05:57:10.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 -- 172.21.15.70:0/3484664377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74070d50 msgr2=0x7f3f741f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:10.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74070d50 0x7f3f741f2de0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f3f64000c00 tx=0x7f3f64004730 comp rx=0 tx=0).stop 2023-10-08T05:57:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 -- 172.21.15.70:0/3484664377 shutdown_connections 2023-10-08T05:57:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f3f5406afc0 0x7f3f5406d480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f74160940 0x7f3f74153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f7415cd90 0x7f3f740cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 --2- 172.21.15.70:0/3484664377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f74070d50 0x7f3f741f2de0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 -- 172.21.15.70:0/3484664377 >> 172.21.15.70:0/3484664377 conn(0x7f3f7406a380 msgr2=0x7f3f740cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 -- 172.21.15.70:0/3484664377 shutdown_connections 2023-10-08T05:57:10.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:10.661+0000 7f3f78e46640 1 -- 172.21.15.70:0/3484664377 wait complete. 2023-10-08T05:57:10.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T05:57:10.986 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:57:10.997 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:57:11.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.009+0000 7f96e00a9640 1 Processor -- start 2023-10-08T05:57:11.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.010+0000 7f96e00a9640 1 -- start start 2023-10-08T05:57:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.011+0000 7f96e00a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96d80afbe0 0x7f96d80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.011+0000 7f96e00a9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96d806dfc0 0x7f96d8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.011+0000 7f96e00a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d8071070 0x7f96d80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.011+0000 7f96e00a9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d80b2620 con 0x7f96d80afbe0 2023-10-08T05:57:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.011+0000 7f96e00a9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d80b2060 con 0x7f96d806dfc0 2023-10-08T05:57:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.011+0000 7f96e00a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80b2340 con 0x7f96d8071070 2023-10-08T05:57:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.011+0000 7f96dde45640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96d80afbe0 0x7f96d80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.011+0000 7f96dde45640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96d80afbe0 0x7f96d80b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52596/0 (socket says 172.21.15.70:52596) 2023-10-08T05:57:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.011+0000 7f96dde45640 1 -- 172.21.15.70:0/2765494736 learned_addr learned my addr 172.21.15.70:0/2765494736 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.012+0000 7f96dde45640 1 -- 172.21.15.70:0/2765494736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d8071070 msgr2=0x7f96d80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.012+0000 7f96dde45640 1 --2- 172.21.15.70:0/2765494736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d8071070 0x7f96d80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.012+0000 7f96dde45640 1 -- 172.21.15.70:0/2765494736 >> v2:172.21.15.181:3300/0 conn(0x7f96d806dfc0 msgr2=0x7f96d8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.012+0000 7f96dd644640 1 --2- 172.21.15.70:0/2765494736 >> v2:172.21.15.181:3300/0 conn(0x7f96d806dfc0 0x7f96d8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.012+0000 7f96dde45640 1 --2- 172.21.15.70:0/2765494736 >> v2:172.21.15.181:3300/0 conn(0x7f96d806dfc0 0x7f96d8070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.012+0000 7f96dde45640 1 -- 172.21.15.70:0/2765494736 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d8070350 con 0x7f96d80afbe0 2023-10-08T05:57:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.013+0000 7f96dde45640 1 --2- 172.21.15.70:0/2765494736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 0x7f96d80b15b0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f96d80b3980 tx=0x7f96d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0352bf224de4be3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.013+0000 7f96dce43640 1 -- 172.21.15.70:0/2765494736 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4031870 con 0x7f96d80afbe0 2023-10-08T05:57:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.013+0000 7f96dce43640 1 -- 172.21.15.70:0/2765494736 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d40319d0 con 0x7f96d80afbe0 2023-10-08T05:57:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.013+0000 7f96dce43640 1 -- 172.21.15.70:0/2765494736 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4031d60 con 0x7f96d80afbe0 2023-10-08T05:57:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.014+0000 7f96e00a9640 1 -- 172.21.15.70:0/2765494736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 msgr2=0x7f96d80b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.014+0000 7f96e00a9640 1 --2- 172.21.15.70:0/2765494736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 0x7f96d80b15b0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f96d80b3980 tx=0x7f96d402b910 comp rx=0 tx=0).stop 2023-10-08T05:57:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.014+0000 7f96e00a9640 1 -- 172.21.15.70:0/2765494736 shutdown_connections 2023-10-08T05:57:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.014+0000 7f96e00a9640 1 --2- 172.21.15.70:0/2765494736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d8071070 0x7f96d80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.014+0000 7f96e00a9640 1 --2- 172.21.15.70:0/2765494736 >> v2:172.21.15.181:3300/0 conn(0x7f96d806dfc0 0x7f96d8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.014+0000 7f96e00a9640 1 --2- 172.21.15.70:0/2765494736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 0x7f96d80b15b0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.014+0000 7f96e00a9640 1 -- 172.21.15.70:0/2765494736 >> 172.21.15.70:0/2765494736 conn(0x7f96d806a5d0 msgr2=0x7f96d80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.014+0000 7f96e00a9640 1 -- 172.21.15.70:0/2765494736 shutdown_connections 2023-10-08T05:57:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.014+0000 7f96e00a9640 1 -- 172.21.15.70:0/2765494736 wait complete. 2023-10-08T05:57:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.016+0000 7f96e00a9640 1 Processor -- start 2023-10-08T05:57:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.016+0000 7f96e00a9640 1 -- start start 2023-10-08T05:57:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.017+0000 7f96e00a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d806dfc0 0x7f96d8151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.017+0000 7f96e00a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d8071070 0x7f96d8151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.017+0000 7f96e00a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 0x7f96d80cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.017+0000 7f96e00a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d80cf550 con 0x7f96d80afbe0 2023-10-08T05:57:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.017+0000 7f96e00a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815c880 con 0x7f96d8071070 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.017+0000 7f96e00a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815cb60 con 0x7f96d806dfc0 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.017+0000 7f96de646640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 0x7f96d80cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.017+0000 7f96de646640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 0x7f96d80cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52606/0 (socket says 172.21.15.70:52606) 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.017+0000 7f96de646640 1 -- 172.21.15.70:0/1256509191 learned_addr learned my addr 172.21.15.70:0/1256509191 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.018+0000 7f96dde45640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d806dfc0 0x7f96d8151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.018+0000 7f96dd644640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d8071070 0x7f96d8151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.018+0000 7f96de646640 1 -- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d806dfc0 msgr2=0x7f96d8151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.018+0000 7f96de646640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d806dfc0 0x7f96d8151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.018+0000 7f96de646640 1 -- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d8071070 msgr2=0x7f96d8151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.018+0000 7f96de646640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d8071070 0x7f96d8151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.018+0000 7f96de646640 1 -- 172.21.15.70:0/1256509191 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d40094d0 con 0x7f96d80afbe0 2023-10-08T05:57:11.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.018+0000 7f96dd644640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d8071070 0x7f96d8151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:57:11.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.018+0000 7f96de646640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 0x7f96d80cf010 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f96d000b9e0 tx=0x7f96d000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:11.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.018+0000 7f96ce7fc640 1 -- 172.21.15.70:0/1256509191 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d000cb20 con 0x7f96d80afbe0 2023-10-08T05:57:11.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.019+0000 7f96ce7fc640 1 -- 172.21.15.70:0/1256509191 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d000cca0 con 0x7f96d80afbe0 2023-10-08T05:57:11.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.019+0000 7f96e00a9640 1 -- 172.21.15.70:0/1256509191 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d80ce810 con 0x7f96d80afbe0 2023-10-08T05:57:11.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.019+0000 7f96ce7fc640 1 -- 172.21.15.70:0/1256509191 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0014680 con 0x7f96d80afbe0 2023-10-08T05:57:11.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.019+0000 7f96e00a9640 1 -- 172.21.15.70:0/1256509191 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d80ce500 con 0x7f96d80afbe0 2023-10-08T05:57:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.020+0000 7f96e00a9640 1 -- 172.21.15.70:0/1256509191 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d80ced30 con 0x7f96d80afbe0 2023-10-08T05:57:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.021+0000 7f96ce7fc640 1 -- 172.21.15.70:0/1256509191 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f96d001d3b0 con 0x7f96d80afbe0 2023-10-08T05:57:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.022+0000 7f96ce7fc640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f96bc06af90 0x7f96bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.022+0000 7f96ce7fc640 1 -- 172.21.15.70:0/1256509191 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f96d00898f0 con 0x7f96d80afbe0 2023-10-08T05:57:11.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.023+0000 7f96dde45640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f96bc06af90 0x7f96bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.024+0000 7f96dde45640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f96bc06af90 0x7f96bc06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96d4036000 tx=0x7f96d403c040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.026+0000 7f96ce7fc640 1 -- 172.21.15.70:0/1256509191 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96d80ced30 con 0x7f96d80afbe0 2023-10-08T05:57:11.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.220+0000 7f96e00a9640 1 -- 172.21.15.70:0/1256509191 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f96d80ced30 con 0x7f96d80afbe0 2023-10-08T05:57:11.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.225+0000 7f96ce7fc640 1 -- 172.21.15.70:0/1256509191 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f96d80ced30 con 0x7f96d80afbe0 2023-10-08T05:57:11.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 -- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f96bc06af90 msgr2=0x7f96bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:11.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f96bc06af90 0x7f96bc06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96d4036000 tx=0x7f96d403c040 comp rx=0 tx=0).stop 2023-10-08T05:57:11.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 -- 172.21.15.70:0/1256509191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 msgr2=0x7f96d80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:11.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 0x7f96d80cf010 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f96d000b9e0 tx=0x7f96d000bea0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 -- 172.21.15.70:0/1256509191 shutdown_connections 2023-10-08T05:57:11.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f96bc06af90 0x7f96bc06d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d80afbe0 0x7f96d80cf010 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96d8071070 0x7f96d8151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 --2- 172.21.15.70:0/1256509191 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96d806dfc0 0x7f96d8151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 -- 172.21.15.70:0/1256509191 >> 172.21.15.70:0/1256509191 conn(0x7f96d806a5d0 msgr2=0x7f96d80c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:11.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 -- 172.21.15.70:0/1256509191 shutdown_connections 2023-10-08T05:57:11.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.228+0000 7f96e00a9640 1 -- 172.21.15.70:0/1256509191 wait complete. 2023-10-08T05:57:11.248 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T05:57:11.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:57:11.366 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T05:57:11.483 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T05:57:11.483 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:57:11.483 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:57:11.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:57:11.545 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:57:11.546 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:57:11.607 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:57:11.608 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:57:11.608 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:57:11.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T05:57:11.668 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T05:57:11.668 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:57:11.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T05:57:11.728 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T05:57:11.728 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:57:11.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.951+0000 7f4bb0409640 1 Processor -- start 2023-10-08T05:57:11.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.951+0000 7f4bb0409640 1 -- start start 2023-10-08T05:57:11.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.953+0000 7f4bb0409640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ba80c90b0 0x7f4ba80cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.953+0000 7f4bb0409640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba815b2b0 con 0x7f4ba80cb3b0 2023-10-08T05:57:11.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.953+0000 7f4bb0409640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba815b590 con 0x7f4ba815ab40 2023-10-08T05:57:11.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.953+0000 7f4bb0409640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba815bf30 con 0x7f4ba80c90b0 2023-10-08T05:57:11.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.953+0000 7f4bae9a6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4ba80cb3b0 0x7f4ba815e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46346/0 (socket says 172.21.15.70:46346) 2023-10-08T05:57:11.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.953+0000 7f4bae9a6640 1 -- 172.21.15.70:0/3849086 learned_addr learned my addr 172.21.15.70:0/3849086 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.953+0000 7f4bad9a4640 1 --2- 172.21.15.70:0/3849086 >> v2:172.21.15.181:3300/0 conn(0x7f4ba80c90b0 0x7f4ba80cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933370272 0 0) 0x7f4ba815b2b0 con 0x7f4ba80cb3b0 2023-10-08T05:57:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b94003520 con 0x7f4ba80cb3b0 2023-10-08T05:57:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 66070823 0 0) 0x7f4b94003520 con 0x7f4ba80cb3b0 2023-10-08T05:57:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 >> v2:172.21.15.181:3300/0 conn(0x7f4ba80c90b0 msgr2=0x7f4ba80cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 --2- 172.21.15.70:0/3849086 >> v2:172.21.15.181:3300/0 conn(0x7f4ba80c90b0 0x7f4ba80cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 >> v1:172.21.15.181:6789/0 conn(0x7f4ba815ab40 legacy=0x7f4ba80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba815b870 con 0x7f4ba80cb3b0 2023-10-08T05:57:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4ba00019f0 con 0x7f4ba80cb3b0 2023-10-08T05:57:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 >> v1:172.21.15.70:6789/0 conn(0x7f4ba80cb3b0 legacy=0x7f4ba815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b940039c0 0x7f4b94003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b940047c0 0x7f4b94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b94005180 0x7f4b9400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba815bf30 con 0x7f4b940047c0 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9400c080 con 0x7f4b940039c0 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9400c240 con 0x7f4b94005180 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.954+0000 7f4bad9a4640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b940047c0 0x7f4b94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.955+0000 7f4bae1a5640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b940039c0 0x7f4b94003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.955+0000 7f4bae9a6640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b94005180 0x7f4b9400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.955+0000 7f4bad9a4640 1 -- 172.21.15.70:0/3849086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b94005180 msgr2=0x7f4b9400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.955+0000 7f4bad9a4640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b94005180 0x7f4b9400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.955+0000 7f4bad9a4640 1 -- 172.21.15.70:0/3849086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b940039c0 msgr2=0x7f4b94003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:11.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.955+0000 7f4bad9a4640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4b940039c0 0x7f4b94003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.955+0000 7f4bad9a4640 1 -- 172.21.15.70:0/3849086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b94003520 con 0x7f4b940047c0 2023-10-08T05:57:11.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.955+0000 7f4bae9a6640 1 -- 172.21.15.70:0/3849086 reap_dead start 2023-10-08T05:57:11.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.956+0000 7f4bad9a4640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b940047c0 0x7f4b94004c40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009a90 tx=0x7f4b940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b20927162904ef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:11.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.956+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9802c680 con 0x7f4b940047c0 2023-10-08T05:57:11.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.956+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9802c7e0 con 0x7f4b940047c0 2023-10-08T05:57:11.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.956+0000 7f4bad1a3640 1 -- 172.21.15.70:0/3849086 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9802cb70 con 0x7f4b940047c0 2023-10-08T05:57:11.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.957+0000 7f4bb0409640 1 -- 172.21.15.70:0/3849086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b940047c0 msgr2=0x7f4b94004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:11.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.957+0000 7f4bb0409640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b940047c0 0x7f4b94004c40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009a90 tx=0x7f4b940057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.957+0000 7f4bb0409640 1 -- 172.21.15.70:0/3849086 shutdown_connections 2023-10-08T05:57:11.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.957+0000 7f4bb0409640 1 --2- 172.21.15.70:0/3849086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b940047c0 0x7f4b94004c40 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.957+0000 7f4bb0409640 1 -- 172.21.15.70:0/3849086 >> 172.21.15.70:0/3849086 conn(0x7f4ba806a380 msgr2=0x7f4ba80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:11.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.957+0000 7f4bb0409640 1 -- 172.21.15.70:0/3849086 shutdown_connections 2023-10-08T05:57:11.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.957+0000 7f4bb0409640 1 -- 172.21.15.70:0/3849086 wait complete. 2023-10-08T05:57:11.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.959+0000 7f4bb0409640 1 Processor -- start 2023-10-08T05:57:11.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.959+0000 7f4bb0409640 1 -- start start 2023-10-08T05:57:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.960+0000 7f4bb0409640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b940047c0 0x7f4ba814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.960+0000 7f4bb0409640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba81f01a0 0x7f4ba80b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.960+0000 7f4bb0409640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ba80b39d0 0x7f4ba81574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.960+0000 7f4bb0409640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80c7490 con 0x7f4ba80b39d0 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.960+0000 7f4bb0409640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80b3e30 con 0x7f4ba81f01a0 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.960+0000 7f4bb0409640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80c68a0 con 0x7f4b940047c0 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.960+0000 7f4bae9a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ba80b39d0 0x7f4ba81574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.960+0000 7f4bae9a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ba80b39d0 0x7f4ba81574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52620/0 (socket says 172.21.15.70:52620) 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.960+0000 7f4bae9a6640 1 -- 172.21.15.70:0/620004052 learned_addr learned my addr 172.21.15.70:0/620004052 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4bad9a4640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba81f01a0 0x7f4ba80b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4bae1a5640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b940047c0 0x7f4ba814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4bae9a6640 1 -- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b940047c0 msgr2=0x7f4ba814bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4bae9a6640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b940047c0 0x7f4ba814bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4bae9a6640 1 -- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba81f01a0 msgr2=0x7f4ba80b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4bae9a6640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba81f01a0 0x7f4ba80b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4bae9a6640 1 -- 172.21.15.70:0/620004052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80c6af0 con 0x7f4ba80b39d0 2023-10-08T05:57:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4bae9a6640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ba80b39d0 0x7f4ba81574e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80b0c20 tx=0x7f4ba000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4b9effd640 1 -- 172.21.15.70:0/620004052 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0004d60 con 0x7f4ba80b39d0 2023-10-08T05:57:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4b9effd640 1 -- 172.21.15.70:0/620004052 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba000bb80 con 0x7f4ba80b39d0 2023-10-08T05:57:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4b9effd640 1 -- 172.21.15.70:0/620004052 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0007070 con 0x7f4ba80b39d0 2023-10-08T05:57:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.961+0000 7f4bb0409640 1 -- 172.21.15.70:0/620004052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba80c6dd0 con 0x7f4ba80b39d0 2023-10-08T05:57:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.962+0000 7f4bb0409640 1 -- 172.21.15.70:0/620004052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba814c020 con 0x7f4ba80b39d0 2023-10-08T05:57:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.963+0000 7f4bb0409640 1 -- 172.21.15.70:0/620004052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba814c1e0 con 0x7f4ba80b39d0 2023-10-08T05:57:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.964+0000 7f4b9effd640 1 -- 172.21.15.70:0/620004052 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4ba00071d0 con 0x7f4ba80b39d0 2023-10-08T05:57:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.964+0000 7f4b9effd640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f4b8c06af90 0x7f4b8c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.965+0000 7f4b9effd640 1 -- 172.21.15.70:0/620004052 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f4ba0089e00 con 0x7f4ba80b39d0 2023-10-08T05:57:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.965+0000 7f4bae1a5640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f4b8c06af90 0x7f4b8c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.966+0000 7f4bae1a5640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f4b8c06af90 0x7f4b8c06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4000c00 tx=0x7f4ba4009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:11.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:11.969+0000 7f4b9effd640 1 -- 172.21.15.70:0/620004052 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ba0054b40 con 0x7f4ba80b39d0 2023-10-08T05:57:12.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.193+0000 7f4bb0409640 1 -- 172.21.15.70:0/620004052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ba80c75d0 con 0x7f4ba80b39d0 2023-10-08T05:57:12.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.194+0000 7f4b9effd640 1 -- 172.21.15.70:0/620004052 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f4ba80c75d0 con 0x7f4ba80b39d0 2023-10-08T05:57:12.195 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:12.195 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":70,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6128,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1820637663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1820637663},{"type":"v1","addr":"172.21.15.181:6835","nonce":1820637663}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62},{"gid":6134,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1666874980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":1666874980},{"type":"v1","addr":"172.21.15.181:6837","nonce":1666874980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2918183907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2918183907},{"type":"v1","addr":"172.21.15.70:6834","nonce":2918183907}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":8958,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/854559008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":854559008},{"type":"v1","addr":"172.21.15.70:6835","nonce":854559008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[]} 2023-10-08T05:57:12.195 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 70 2023-10-08T05:57:12.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 -- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f4b8c06af90 msgr2=0x7f4b8c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:12.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f4b8c06af90 0x7f4b8c06d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4000c00 tx=0x7f4ba4009040 comp rx=0 tx=0).stop 2023-10-08T05:57:12.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 -- 172.21.15.70:0/620004052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ba80b39d0 msgr2=0x7f4ba81574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:12.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ba80b39d0 0x7f4ba81574e0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80b0c20 tx=0x7f4ba000b4a0 comp rx=0 tx=0).stop 2023-10-08T05:57:12.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 -- 172.21.15.70:0/620004052 shutdown_connections 2023-10-08T05:57:12.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f4b8c06af90 0x7f4b8c06d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:12.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ba80b39d0 0x7f4ba81574e0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:12.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ba81f01a0 0x7f4ba80b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:12.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 --2- 172.21.15.70:0/620004052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4b940047c0 0x7f4ba814bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:12.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 -- 172.21.15.70:0/620004052 >> 172.21.15.70:0/620004052 conn(0x7f4ba806a380 msgr2=0x7f4ba80af460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:12.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 -- 172.21.15.70:0/620004052 shutdown_connections 2023-10-08T05:57:12.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:12.197+0000 7f4bb0409640 1 -- 172.21.15.70:0/620004052 wait complete. 2023-10-08T05:57:12.217 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T05:57:12.217 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T05:57:12.218 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T05:57:12.218 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:57:12.218 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T05:57:12.218 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T05:57:12.218 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T05:57:12.219 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:57:12.219 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T05:57:12.219 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T05:57:12.219 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T05:57:12.219 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:57:12.219 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T05:57:12.219 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T05:57:12.220 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T05:57:12.220 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T05:57:12.234 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T05:57:12.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T05:57:12.240 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T05:57:12.240 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T05:57:12.247 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T05:57:12.247 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T05:57:12.286 INFO:tasks.ceph.mds.a:Started 2023-10-08T05:57:12.286 DEBUG:teuthology.parallel:result is None 2023-10-08T05:57:12.293 INFO:tasks.ceph.mds.c:Started 2023-10-08T05:57:12.294 DEBUG:teuthology.parallel:result is None 2023-10-08T05:57:12.296 INFO:tasks.ceph.mds.b:Started 2023-10-08T05:57:12.296 DEBUG:teuthology.parallel:result is None 2023-10-08T05:57:12.452 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T05:57:12.459 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T05:57:12.490 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T05:57:17.395 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-08T05:57:18.247 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T05:57:18.247 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T05:57:18.257 INFO:tasks.ceph.mds.d:Started 2023-10-08T05:57:18.257 DEBUG:teuthology.parallel:result is None 2023-10-08T05:57:18.257 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T05:57:18.447 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T05:57:18.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.484+0000 7fd3f52c2640 1 Processor -- start 2023-10-08T05:57:18.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.484+0000 7fd3f52c2640 1 -- start start 2023-10-08T05:57:18.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.486+0000 7fd3f52c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3f00b21f0 0x7fd3f00c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.486+0000 7fd3f52c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00ca1a0 0x7fd3f006d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.486+0000 7fd3f52c2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f006c3a0 con 0x7fd3f00702b0 2023-10-08T05:57:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.486+0000 7fd3f52c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f006c640 con 0x7fd3f00b21f0 2023-10-08T05:57:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.486+0000 7fd3f52c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f006c920 con 0x7fd3f00ca1a0 2023-10-08T05:57:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.486+0000 7fd3e67fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3f00b21f0 0x7fd3f00c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.486+0000 7fd3e67fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd3f00b21f0 0x7fd3f00c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51914/0 (socket says 172.21.15.70:51914) 2023-10-08T05:57:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.486+0000 7fd3e67fc640 1 -- 172.21.15.70:0/1445160798 learned_addr learned my addr 172.21.15.70:0/1445160798 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:18.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.487+0000 7fd3ef7fe640 1 --2- 172.21.15.70:0/1445160798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00ca1a0 0x7fd3f006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.487+0000 7fd3e67fc640 1 -- 172.21.15.70:0/1445160798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00ca1a0 msgr2=0x7fd3f006d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.487+0000 7fd3e67fc640 1 --2- 172.21.15.70:0/1445160798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00ca1a0 0x7fd3f006d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.487+0000 7fd3e67fc640 1 -- 172.21.15.70:0/1445160798 >> v1:172.21.15.181:6789/0 conn(0x7fd3f00702b0 legacy=0x7fd3f0070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.487+0000 7fd3e67fc640 1 -- 172.21.15.70:0/1445160798 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f006cc00 con 0x7fd3f00b21f0 2023-10-08T05:57:18.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.488+0000 7fd3e67fc640 1 --2- 172.21.15.70:0/1445160798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 0x7fd3f00c9c60 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800bb00 tx=0x7fd3d802df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8165e3d68b4246b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:18.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.488+0000 7fd3ee7fc640 1 -- 172.21.15.70:0/1445160798 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d802ece0 con 0x7fd3f00b21f0 2023-10-08T05:57:18.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.488+0000 7fd3ee7fc640 1 -- 172.21.15.70:0/1445160798 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d8032070 con 0x7fd3f00b21f0 2023-10-08T05:57:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.488+0000 7fd3ee7fc640 1 -- 172.21.15.70:0/1445160798 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8038620 con 0x7fd3f00b21f0 2023-10-08T05:57:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.489+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1445160798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 msgr2=0x7fd3f00c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.489+0000 7fd3f52c2640 1 --2- 172.21.15.70:0/1445160798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 0x7fd3f00c9c60 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800bb00 tx=0x7fd3d802df30 comp rx=0 tx=0).stop 2023-10-08T05:57:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.489+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1445160798 shutdown_connections 2023-10-08T05:57:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.489+0000 7fd3f52c2640 1 --2- 172.21.15.70:0/1445160798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00ca1a0 0x7fd3f006d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.489+0000 7fd3f52c2640 1 --2- 172.21.15.70:0/1445160798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 0x7fd3f00c9c60 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.489+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1445160798 >> 172.21.15.70:0/1445160798 conn(0x7fd3f006a5d0 msgr2=0x7fd3f00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.489+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1445160798 shutdown_connections 2023-10-08T05:57:18.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.489+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1445160798 wait complete. 2023-10-08T05:57:18.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.491+0000 7fd3f52c2640 1 Processor -- start 2023-10-08T05:57:18.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.491+0000 7fd3f52c2640 1 -- start start 2023-10-08T05:57:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3f52c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00702b0 0x7fd3f014fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3f52c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 0x7fd3f00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3f52c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f00ca1a0 0x7fd3f01e2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3f52c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00cdb90 con 0x7fd3f00b21f0 2023-10-08T05:57:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3f52c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00cd9a0 con 0x7fd3f00ca1a0 2023-10-08T05:57:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3f52c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00cdda0 con 0x7fd3f00702b0 2023-10-08T05:57:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3e67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 0x7fd3f00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:18.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3e67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 0x7fd3f00cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51920/0 (socket says 172.21.15.70:51920) 2023-10-08T05:57:18.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3e67fc640 1 -- 172.21.15.70:0/1279843197 learned_addr learned my addr 172.21.15.70:0/1279843197 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:18.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3e67fc640 1 -- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00702b0 msgr2=0x7fd3f014fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:18.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3ef7fe640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f00ca1a0 0x7fd3f01e2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:18.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.492+0000 7fd3eeffd640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00702b0 0x7fd3f014fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:18.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3e67fc640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00702b0 0x7fd3f014fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:18.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3e67fc640 1 -- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f00ca1a0 msgr2=0x7fd3f01e2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:18.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3e67fc640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f00ca1a0 0x7fd3f01e2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:18.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3e67fc640 1 -- 172.21.15.70:0/1279843197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d800b760 con 0x7fd3f00b21f0 2023-10-08T05:57:18.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3ef7fe640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f00ca1a0 0x7fd3f01e2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:18.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3eeffd640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00702b0 0x7fd3f014fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:18.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3e67fc640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 0x7fd3f00cee40 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800bc20 tx=0x7fd3d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:18.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3e7fff640 1 -- 172.21.15.70:0/1279843197 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8038d40 con 0x7fd3f00b21f0 2023-10-08T05:57:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3e7fff640 1 -- 172.21.15.70:0/1279843197 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d80423f0 con 0x7fd3f00b21f0 2023-10-08T05:57:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1279843197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f00ce080 con 0x7fd3f00b21f0 2023-10-08T05:57:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3e7fff640 1 -- 172.21.15.70:0/1279843197 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d80427a0 con 0x7fd3f00b21f0 2023-10-08T05:57:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.493+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1279843197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f00ce640 con 0x7fd3f00b21f0 2023-10-08T05:57:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.494+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1279843197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f004e160 con 0x7fd3f00b21f0 2023-10-08T05:57:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.496+0000 7fd3e7fff640 1 -- 172.21.15.70:0/1279843197 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd3d8042900 con 0x7fd3f00b21f0 2023-10-08T05:57:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.496+0000 7fd3e7fff640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd3d006afc0 0x7fd3d006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.497+0000 7fd3e7fff640 1 -- 172.21.15.70:0/1279843197 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd3d80aba70 con 0x7fd3f00b21f0 2023-10-08T05:57:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.497+0000 7fd3eeffd640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd3d006afc0 0x7fd3d006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.498+0000 7fd3eeffd640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd3d006afc0 0x7fd3d006d480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0000f60 tx=0x7fd3e0009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:18.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.501+0000 7fd3e7fff640 1 -- 172.21.15.70:0/1279843197 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3d8076890 con 0x7fd3f00b21f0 2023-10-08T05:57:18.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:18.705+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1279843197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd3f0070730 con 0x7fd3f00b21f0 2023-10-08T05:57:20.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.455+0000 7fd3e7fff640 1 -- 172.21.15.70:0/1279843197 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v159) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd3d8046020 con 0x7fd3f00b21f0 2023-10-08T05:57:20.473 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T05:57:20.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.459+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd3d006afc0 msgr2=0x7fd3d006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:20.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.459+0000 7fd3f52c2640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd3d006afc0 0x7fd3d006d480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0000f60 tx=0x7fd3e0009040 comp rx=0 tx=0).stop 2023-10-08T05:57:20.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.459+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1279843197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 msgr2=0x7fd3f00cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:20.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.459+0000 7fd3f52c2640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 0x7fd3f00cee40 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800bc20 tx=0x7fd3d80046c0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.460+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1279843197 shutdown_connections 2023-10-08T05:57:20.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.460+0000 7fd3f52c2640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fd3d006afc0 0x7fd3d006d480 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.460+0000 7fd3f52c2640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3f00ca1a0 0x7fd3f01e2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.460+0000 7fd3f52c2640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3f00b21f0 0x7fd3f00cee40 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.460+0000 7fd3f52c2640 1 --2- 172.21.15.70:0/1279843197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3f00702b0 0x7fd3f014fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.460+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1279843197 >> 172.21.15.70:0/1279843197 conn(0x7fd3f006a5d0 msgr2=0x7fd3f00c6220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:20.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.460+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1279843197 shutdown_connections 2023-10-08T05:57:20.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.460+0000 7fd3f52c2640 1 -- 172.21.15.70:0/1279843197 wait complete. 2023-10-08T05:57:20.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T05:57:20.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.688+0000 7fe394226640 1 Processor -- start 2023-10-08T05:57:20.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.688+0000 7fe394226640 1 -- start start 2023-10-08T05:57:20.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.690+0000 7fe394226640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe38c15ab40 0x7fe38c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:20.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.690+0000 7fe394226640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe38c0cb580 0x7fe38c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:20.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.690+0000 7fe394226640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38c15b2b0 con 0x7fe38c0c90b0 2023-10-08T05:57:20.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.690+0000 7fe394226640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe38c15bf30 con 0x7fe38c0cb580 2023-10-08T05:57:20.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.690+0000 7fe394226640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c15f7a0 con 0x7fe38c15ab40 2023-10-08T05:57:20.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.690+0000 7fe3927c3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe38c0cb580 0x7fe38c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:20.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.690+0000 7fe3917c1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe38c0c90b0 0x7fe38c0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36030/0 (socket says 172.21.15.70:36030) 2023-10-08T05:57:20.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.690+0000 7fe3917c1640 1 -- 172.21.15.70:0/3789485408 learned_addr learned my addr 172.21.15.70:0/3789485408 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.691+0000 7fe391fc2640 1 --2- 172.21.15.70:0/3789485408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe38c15ab40 0x7fe38c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.691+0000 7fe3927c3640 1 -- 172.21.15.70:0/3789485408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe38c15ab40 msgr2=0x7fe38c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.691+0000 7fe3927c3640 1 --2- 172.21.15.70:0/3789485408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe38c15ab40 0x7fe38c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.691+0000 7fe3927c3640 1 -- 172.21.15.70:0/3789485408 >> v1:172.21.15.70:6789/0 conn(0x7fe38c0c90b0 legacy=0x7fe38c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.691+0000 7fe3927c3640 1 -- 172.21.15.70:0/3789485408 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c15b590 con 0x7fe38c0cb580 2023-10-08T05:57:20.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.692+0000 7fe3927c3640 1 --2- 172.21.15.70:0/3789485408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 0x7fe38c0adc60 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0097a0 tx=0x7fe37c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ed8bb13327b03e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.692+0000 7fe390fc0640 1 -- 172.21.15.70:0/3789485408 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c02d040 con 0x7fe38c0cb580 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.692+0000 7fe390fc0640 1 -- 172.21.15.70:0/3789485408 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37c031810 con 0x7fe38c0cb580 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.692+0000 7fe390fc0640 1 -- 172.21.15.70:0/3789485408 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c031ba0 con 0x7fe38c0cb580 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.693+0000 7fe394226640 1 -- 172.21.15.70:0/3789485408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 msgr2=0x7fe38c0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.693+0000 7fe394226640 1 --2- 172.21.15.70:0/3789485408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 0x7fe38c0adc60 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0097a0 tx=0x7fe37c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.693+0000 7fe394226640 1 -- 172.21.15.70:0/3789485408 shutdown_connections 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.693+0000 7fe394226640 1 --2- 172.21.15.70:0/3789485408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 0x7fe38c0adc60 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.693+0000 7fe394226640 1 --2- 172.21.15.70:0/3789485408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe38c15ab40 0x7fe38c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.693+0000 7fe394226640 1 -- 172.21.15.70:0/3789485408 >> 172.21.15.70:0/3789485408 conn(0x7fe38c06a380 msgr2=0x7fe38c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.693+0000 7fe394226640 1 -- 172.21.15.70:0/3789485408 shutdown_connections 2023-10-08T05:57:20.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.693+0000 7fe394226640 1 -- 172.21.15.70:0/3789485408 wait complete. 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.695+0000 7fe394226640 1 Processor -- start 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.695+0000 7fe394226640 1 -- start start 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe394226640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe38c0c90b0 0x7fe38c1449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe394226640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 0x7fe38c15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe394226640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe38c15ab40 0x7fe38c140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe394226640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c141f90 con 0x7fe38c0cb580 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe394226640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c13eca0 con 0x7fe38c15ab40 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe394226640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c141430 con 0x7fe38c0c90b0 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe3917c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 0x7fe38c15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe3917c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 0x7fe38c15d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51934/0 (socket says 172.21.15.70:51934) 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe3917c1640 1 -- 172.21.15.70:0/3383428757 learned_addr learned my addr 172.21.15.70:0/3383428757 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:20.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe3917c1640 1 -- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe38c0c90b0 msgr2=0x7fe38c1449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe3917c1640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe38c0c90b0 0x7fe38c1449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe3917c1640 1 -- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe38c15ab40 msgr2=0x7fe38c140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe3917c1640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe38c15ab40 0x7fe38c140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.696+0000 7fe3917c1640 1 -- 172.21.15.70:0/3383428757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c009400 con 0x7fe38c0cb580 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.697+0000 7fe3917c1640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 0x7fe38c15d940 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe38800cbc0 tx=0x7fe38800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.697+0000 7fe3827fc640 1 -- 172.21.15.70:0/3383428757 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38800aca0 con 0x7fe38c0cb580 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.697+0000 7fe3827fc640 1 -- 172.21.15.70:0/3383428757 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38800ae00 con 0x7fe38c0cb580 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.697+0000 7fe3827fc640 1 -- 172.21.15.70:0/3383428757 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388012600 con 0x7fe38c0cb580 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.697+0000 7fe394226640 1 -- 172.21.15.70:0/3383428757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c1416f0 con 0x7fe38c0cb580 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.697+0000 7fe394226640 1 -- 172.21.15.70:0/3383428757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c141cb0 con 0x7fe38c0cb580 2023-10-08T05:57:20.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.698+0000 7fe394226640 1 -- 172.21.15.70:0/3383428757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38c067490 con 0x7fe38c0cb580 2023-10-08T05:57:20.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.699+0000 7fe3827fc640 1 -- 172.21.15.70:0/3383428757 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe388007070 con 0x7fe38c0cb580 2023-10-08T05:57:20.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.700+0000 7fe3827fc640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe37806af90 0x7fe37806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:20.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.700+0000 7fe3827fc640 1 -- 172.21.15.70:0/3383428757 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe388089420 con 0x7fe38c0cb580 2023-10-08T05:57:20.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.701+0000 7fe391fc2640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe37806af90 0x7fe37806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:20.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.702+0000 7fe391fc2640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe37806af90 0x7fe37806d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe3840098f0 tx=0x7fe384008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:20.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.704+0000 7fe3827fc640 1 -- 172.21.15.70:0/3383428757 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe388054620 con 0x7fe38c0cb580 2023-10-08T05:57:20.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.961+0000 7fe394226640 1 -- 172.21.15.70:0/3383428757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe38c0b0ef0 con 0x7fe38c0cb580 2023-10-08T05:57:20.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.963+0000 7fe3827fc640 1 -- 172.21.15.70:0/3383428757 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe388057c40 con 0x7fe38c0cb580 2023-10-08T05:57:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:20.968 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:57:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T05:57:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T05:57:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T05:57:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:57:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:57:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:57:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T05:57:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T05:57:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:57:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:57:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:57:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:57:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T05:57:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T05:57:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T05:57:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T05:57:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T05:57:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:21.008 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:57:21.009 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:57:21.010 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T05:57:21.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:21.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:21.013 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T05:57:21.013 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T05:57:21.013 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:21.013 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:57:21.013 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:57:21.013 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:21.014 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:21.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T05:57:21.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:21.014 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T05:57:21.014 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T05:57:21.014 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T05:57:21.014 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T05:57:21.015 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T05:57:21.015 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T05:57:21.015 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:21.015 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:21.015 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T05:57:21.015 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:57:21.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 -- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe37806af90 msgr2=0x7fe37806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:21.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe37806af90 0x7fe37806d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe3840098f0 tx=0x7fe384008040 comp rx=0 tx=0).stop 2023-10-08T05:57:21.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 -- 172.21.15.70:0/3383428757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 msgr2=0x7fe38c15d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:21.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 0x7fe38c15d940 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fe38800cbc0 tx=0x7fe38800a060 comp rx=0 tx=0).stop 2023-10-08T05:57:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 -- 172.21.15.70:0/3383428757 shutdown_connections 2023-10-08T05:57:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fe37806af90 0x7fe37806d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe38c15ab40 0x7fe38c140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe38c0cb580 0x7fe38c15d940 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 --2- 172.21.15.70:0/3383428757 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe38c0c90b0 0x7fe38c1449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 -- 172.21.15.70:0/3383428757 >> 172.21.15.70:0/3383428757 conn(0x7fe38c06a380 msgr2=0x7fe38c0ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.965+0000 7fe394226640 1 -- 172.21.15.70:0/3383428757 shutdown_connections 2023-10-08T05:57:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:20.966+0000 7fe394226640 1 -- 172.21.15.70:0/3383428757 wait complete. 2023-10-08T05:57:21.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:57:21.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.273+0000 7fa410ae1640 1 Processor -- start 2023-10-08T05:57:21.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.273+0000 7fa410ae1640 1 -- start start 2023-10-08T05:57:21.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa410ae1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa40c1597d0 0x7fa40c06ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:21.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa410ae1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa40c0cb190 0x7fa40c0cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:21.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa410ae1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c0adfc0 con 0x7fa40c157000 2023-10-08T05:57:21.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa410ae1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0cbb60 con 0x7fa40c0cb190 2023-10-08T05:57:21.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa410ae1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c06d380 con 0x7fa40c1597d0 2023-10-08T05:57:21.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa40a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa40c157000 0x7fa40c0cc7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36040/0 (socket says 172.21.15.70:36040) 2023-10-08T05:57:21.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa40a59c640 1 -- 172.21.15.70:0/2248095191 learned_addr learned my addr 172.21.15.70:0/2248095191 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:21.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa40ad9d640 1 --2- 172.21.15.70:0/2248095191 >> v2:172.21.15.70:3300/0 conn(0x7fa40c0cb190 0x7fa40c0cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:21.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa403fff640 1 --2- 172.21.15.70:0/2248095191 >> v2:172.21.15.181:3300/0 conn(0x7fa40c1597d0 0x7fa40c06ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:21.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa409d9b640 1 -- 172.21.15.70:0/2248095191 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115393330 0 0) 0x7fa40c0adfc0 con 0x7fa40c157000 2023-10-08T05:57:21.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.275+0000 7fa409d9b640 1 -- 172.21.15.70:0/2248095191 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3f8003520 con 0x7fa40c157000 2023-10-08T05:57:21.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.276+0000 7fa40ad9d640 1 -- 172.21.15.70:0/2248095191 >> v2:172.21.15.181:3300/0 conn(0x7fa40c1597d0 msgr2=0x7fa40c06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:21.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.276+0000 7fa40ad9d640 1 --2- 172.21.15.70:0/2248095191 >> v2:172.21.15.181:3300/0 conn(0x7fa40c1597d0 0x7fa40c06ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.276+0000 7fa40ad9d640 1 -- 172.21.15.70:0/2248095191 >> v1:172.21.15.70:6789/0 conn(0x7fa40c157000 legacy=0x7fa40c0cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:21.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.276+0000 7fa40ad9d640 1 -- 172.21.15.70:0/2248095191 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c159cb0 con 0x7fa40c0cb190 2023-10-08T05:57:21.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.277+0000 7fa40ad9d640 1 --2- 172.21.15.70:0/2248095191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0cb190 0x7fa40c0cb620 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa40c071770 tx=0x7fa40402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71649e0743f5413a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:21.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.277+0000 7fa409d9b640 1 -- 172.21.15.70:0/2248095191 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40402cc80 con 0x7fa40c0cb190 2023-10-08T05:57:21.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.277+0000 7fa409d9b640 1 -- 172.21.15.70:0/2248095191 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40402cde0 con 0x7fa40c0cb190 2023-10-08T05:57:21.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.277+0000 7fa409d9b640 1 -- 172.21.15.70:0/2248095191 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa404034660 con 0x7fa40c0cb190 2023-10-08T05:57:21.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.278+0000 7fa410ae1640 1 -- 172.21.15.70:0/2248095191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0cb190 msgr2=0x7fa40c0cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:21.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.278+0000 7fa410ae1640 1 --2- 172.21.15.70:0/2248095191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0cb190 0x7fa40c0cb620 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa40c071770 tx=0x7fa40402bed0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.278+0000 7fa410ae1640 1 -- 172.21.15.70:0/2248095191 shutdown_connections 2023-10-08T05:57:21.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.278+0000 7fa410ae1640 1 --2- 172.21.15.70:0/2248095191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0cb190 0x7fa40c0cb620 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.278+0000 7fa410ae1640 1 --2- 172.21.15.70:0/2248095191 >> v2:172.21.15.181:3300/0 conn(0x7fa40c1597d0 0x7fa40c06ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.278+0000 7fa410ae1640 1 -- 172.21.15.70:0/2248095191 >> 172.21.15.70:0/2248095191 conn(0x7fa40c06a380 msgr2=0x7fa40c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:21.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.278+0000 7fa410ae1640 1 -- 172.21.15.70:0/2248095191 shutdown_connections 2023-10-08T05:57:21.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.278+0000 7fa410ae1640 1 -- 172.21.15.70:0/2248095191 wait complete. 2023-10-08T05:57:21.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.280+0000 7fa410ae1640 1 Processor -- start 2023-10-08T05:57:21.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.280+0000 7fa410ae1640 1 -- start start 2023-10-08T05:57:21.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.281+0000 7fa410ae1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa40c0cb190 0x7fa40c0b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:21.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.281+0000 7fa410ae1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa40c1597d0 0x7fa40c1e1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.281+0000 7fa410ae1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0c9ed0 0x7fa40c148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.281+0000 7fa410ae1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c149ff0 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.281+0000 7fa410ae1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1e2360 con 0x7fa40c1597d0 2023-10-08T05:57:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.281+0000 7fa410ae1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1e2e30 con 0x7fa40c0cb190 2023-10-08T05:57:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.281+0000 7fa40ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0c9ed0 0x7fa40c148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.281+0000 7fa40ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0c9ed0 0x7fa40c148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51942/0 (socket says 172.21.15.70:51942) 2023-10-08T05:57:21.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.281+0000 7fa40ad9d640 1 -- 172.21.15.70:0/1164333967 learned_addr learned my addr 172.21.15.70:0/1164333967 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa40a59c640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa40c0cb190 0x7fa40c0b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa40ad9d640 1 -- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa40c0cb190 msgr2=0x7fa40c0b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa40ad9d640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa40c0cb190 0x7fa40c0b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa40ad9d640 1 -- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa40c1597d0 msgr2=0x7fa40c1e1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa40ad9d640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa40c1597d0 0x7fa40c1e1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa40ad9d640 1 -- 172.21.15.70:0/1164333967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa404009a70 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa40a59c640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa40c0cb190 0x7fa40c0b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:21.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa40ad9d640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0c9ed0 0x7fa40c148f30 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa404009e50 tx=0x7fa404004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:21.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa402ffd640 1 -- 172.21.15.70:0/1164333967 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40403e600 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa402ffd640 1 -- 172.21.15.70:0/1164333967 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40403e760 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa402ffd640 1 -- 172.21.15.70:0/1164333967 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40403eb10 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.282+0000 7fa410ae1640 1 -- 172.21.15.70:0/1164333967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c1e2590 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.283+0000 7fa410ae1640 1 -- 172.21.15.70:0/1164333967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c1e2b50 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.284+0000 7fa410ae1640 1 -- 172.21.15.70:0/1164333967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c14a130 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.284+0000 7fa402ffd640 1 -- 172.21.15.70:0/1164333967 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa40403ec70 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.285+0000 7fa402ffd640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa3ec06af90 0x7fa3ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:21.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.286+0000 7fa402ffd640 1 -- 172.21.15.70:0/1164333967 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa4040ac160 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.286+0000 7fa40a59c640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa3ec06af90 0x7fa3ec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:21.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.287+0000 7fa40a59c640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa3ec06af90 0x7fa3ec06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0011e0 tx=0x7fa3fc009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:21.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.290+0000 7fa402ffd640 1 -- 172.21.15.70:0/1164333967 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa40c14a130 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.483+0000 7fa410ae1640 1 -- 172.21.15.70:0/1164333967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa40c14a130 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.485+0000 7fa402ffd640 1 -- 172.21.15.70:0/1164333967 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+13175 (secure 0 0 0) 0x7fa40c14a130 con 0x7fa40c0c9ed0 2023-10-08T05:57:21.549 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:21.549 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":159,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:57:20.451923+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":147,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:57:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 -- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa3ec06af90 msgr2=0x7fa3ec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa3ec06af90 0x7fa3ec06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0011e0 tx=0x7fa3fc009040 comp rx=0 tx=0).stop 2023-10-08T05:57:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 -- 172.21.15.70:0/1164333967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0c9ed0 msgr2=0x7fa40c148f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0c9ed0 0x7fa40c148f30 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa404009e50 tx=0x7fa404004a40 comp rx=0 tx=0).stop 2023-10-08T05:57:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 -- 172.21.15.70:0/1164333967 shutdown_connections 2023-10-08T05:57:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa3ec06af90 0x7fa3ec06d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa40c0c9ed0 0x7fa40c148f30 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa40c1597d0 0x7fa40c1e1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 --2- 172.21.15.70:0/1164333967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa40c0cb190 0x7fa40c0b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:21.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 -- 172.21.15.70:0/1164333967 >> 172.21.15.70:0/1164333967 conn(0x7fa40c06a380 msgr2=0x7fa40c0ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:21.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.488+0000 7fa410ae1640 1 -- 172.21.15.70:0/1164333967 shutdown_connections 2023-10-08T05:57:21.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.489+0000 7fa410ae1640 1 -- 172.21.15.70:0/1164333967 wait complete. 2023-10-08T05:57:21.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T05:57:21.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T05:57:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.785+0000 7f81a19f0640 1 Processor -- start 2023-10-08T05:57:22.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.785+0000 7f81a19f0640 1 -- start start 2023-10-08T05:57:22.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.787+0000 7f81a19f0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f819c0b1750 0x7f819c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:22.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.787+0000 7f81a19f0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f819c06d240 0x7f819c070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:22.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.787+0000 7f81a19f0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819c15bf30 con 0x7f819c15aad0 2023-10-08T05:57:22.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.787+0000 7f81a19f0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819c06d6d0 con 0x7f819c06d240 2023-10-08T05:57:22.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.787+0000 7f81a19f0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0b1bb0 con 0x7f819c0b1750 2023-10-08T05:57:22.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.788+0000 7f819affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f819c15aad0 0x7f819c0c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36042/0 (socket says 172.21.15.70:36042) 2023-10-08T05:57:22.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.788+0000 7f819affd640 1 -- 172.21.15.70:0/3942545949 learned_addr learned my addr 172.21.15.70:0/3942545949 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:22.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.788+0000 7f819b7fe640 1 --2- 172.21.15.70:0/3942545949 >> v2:172.21.15.181:3300/0 conn(0x7f819c06d240 0x7f819c070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:22.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.788+0000 7f8193fff640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f819c0b1750 0x7f819c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:22.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.788+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304769295 0 0) 0x7f819c15bf30 con 0x7f819c15aad0 2023-10-08T05:57:22.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.788+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8188003520 con 0x7f819c15aad0 2023-10-08T05:57:22.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3842307333 0 0) 0x7f8188003520 con 0x7f819c15aad0 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f819c0b1750 msgr2=0x7f819c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f819c0b1750 0x7f819c06cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 >> v2:172.21.15.181:3300/0 conn(0x7f819c06d240 msgr2=0x7f819c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 --2- 172.21.15.70:0/3942545949 >> v2:172.21.15.181:3300/0 conn(0x7f819c06d240 0x7f819c070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c15b320 con 0x7f819c15aad0 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f818c001670 con 0x7f819c15aad0 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 >> v1:172.21.15.70:6789/0 conn(0x7f819c15aad0 legacy=0x7f819c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8188003970 0x7f8188003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81880047c0 0x7f8188004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f819c0b1750 0x7f8188005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:22.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c06d6d0 con 0x7f8188003970 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0b1bb0 con 0x7f81880047c0 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818800b980 con 0x7f819c0b1750 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.789+0000 7f819affd640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8188003970 0x7f8188003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.790+0000 7f8193fff640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81880047c0 0x7f8188004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.790+0000 7f819b7fe640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f819c0b1750 0x7f8188005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.790+0000 7f819affd640 1 -- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f819c0b1750 msgr2=0x7f8188005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.790+0000 7f819affd640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f819c0b1750 0x7f8188005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.790+0000 7f819affd640 1 -- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81880047c0 msgr2=0x7f8188004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.790+0000 7f819affd640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81880047c0 0x7f8188004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.790+0000 7f819affd640 1 -- 172.21.15.70:0/3942545949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8188003520 con 0x7f8188003970 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.791+0000 7f819affd640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8188003970 0x7f8188003d90 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f818c00c110 tx=0x7f81880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4471f0da173079d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:22.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.791+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c005ad0 con 0x7f8188003970 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.791+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818c005c30 con 0x7f8188003970 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.791+0000 7f819a7fc640 1 -- 172.21.15.70:0/3942545949 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c02c810 con 0x7f8188003970 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.792+0000 7f81a19f0640 1 -- 172.21.15.70:0/3942545949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8188003970 msgr2=0x7f8188003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.792+0000 7f81a19f0640 1 --2- 172.21.15.70:0/3942545949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8188003970 0x7f8188003d90 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f818c00c110 tx=0x7f81880057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.792+0000 7f819b7fe640 1 -- 172.21.15.70:0/3942545949 reap_dead start 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.792+0000 7f81a19f0640 1 -- 172.21.15.70:0/3942545949 shutdown_connections 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.792+0000 7f81a19f0640 1 -- 172.21.15.70:0/3942545949 >> 172.21.15.70:0/3942545949 conn(0x7f819c069d50 msgr2=0x7f819c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.792+0000 7f81a19f0640 1 -- 172.21.15.70:0/3942545949 shutdown_connections 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.792+0000 7f81a19f0640 1 -- 172.21.15.70:0/3942545949 wait complete. 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.794+0000 7f81a19f0640 1 Processor -- start 2023-10-08T05:57:22.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.794+0000 7f81a19f0640 1 -- start start 2023-10-08T05:57:22.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f81a19f0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8188003970 0x7f819c156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:22.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f81a19f0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f819c15cdb0 0x7f819c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:22.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f81a19f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f819c14e480 0x7f819c14e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:22.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f81a19f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c14d160 con 0x7f819c14e480 2023-10-08T05:57:22.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f81a19f0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c15d770 con 0x7f819c15cdb0 2023-10-08T05:57:22.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f81a19f0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c14d3e0 con 0x7f8188003970 2023-10-08T05:57:22.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f819c14e480 0x7f819c14e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:22.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f819c14e480 0x7f819c14e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51962/0 (socket says 172.21.15.70:51962) 2023-10-08T05:57:22.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f819b7fe640 1 -- 172.21.15.70:0/4058525791 learned_addr learned my addr 172.21.15.70:0/4058525791 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:22.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f8193fff640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f819c15cdb0 0x7f819c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f819affd640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8188003970 0x7f819c156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.795+0000 7f819b7fe640 1 -- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8188003970 msgr2=0x7f819c156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f819b7fe640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8188003970 0x7f819c156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f819b7fe640 1 -- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f819c15cdb0 msgr2=0x7f819c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f819b7fe640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f819c15cdb0 0x7f819c15d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f819b7fe640 1 -- 172.21.15.70:0/4058525791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c14d6c0 con 0x7f819c14e480 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f8193fff640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f819c15cdb0 0x7f819c15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f819b7fe640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f819c14e480 0x7f819c14e930 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f819c0cb0a0 tx=0x7f819400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f8198ff9640 1 -- 172.21.15.70:0/4058525791 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194017400 con 0x7f819c14e480 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f8198ff9640 1 -- 172.21.15.70:0/4058525791 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8194011040 con 0x7f819c14e480 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f8198ff9640 1 -- 172.21.15.70:0/4058525791 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81940177c0 con 0x7f819c14e480 2023-10-08T05:57:22.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f81a19f0640 1 -- 172.21.15.70:0/4058525791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c14d9a0 con 0x7f819c14e480 2023-10-08T05:57:22.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.796+0000 7f81a19f0640 1 -- 172.21.15.70:0/4058525791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c14df60 con 0x7f819c14e480 2023-10-08T05:57:22.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.797+0000 7f81a19f0640 1 -- 172.21.15.70:0/4058525791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819c14e180 con 0x7f819c14e480 2023-10-08T05:57:22.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.798+0000 7f8198ff9640 1 -- 172.21.15.70:0/4058525791 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8194017920 con 0x7f819c14e480 2023-10-08T05:57:22.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.799+0000 7f8198ff9640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f817c06af90 0x7f817c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:22.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.799+0000 7f8198ff9640 1 -- 172.21.15.70:0/4058525791 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8194089bc0 con 0x7f819c14e480 2023-10-08T05:57:22.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.800+0000 7f819affd640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f817c06af90 0x7f817c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:22.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.801+0000 7f819affd640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f817c06af90 0x7f817c06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f818c034000 tx=0x7f818c007000 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:22.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.803+0000 7f8198ff9640 1 -- 172.21.15.70:0/4058525791 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8194054b70 con 0x7f819c14e480 2023-10-08T05:57:22.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.990+0000 7f81a19f0640 1 -- 172.21.15.70:0/4058525791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f819c04e160 con 0x7f819c14e480 2023-10-08T05:57:22.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.991+0000 7f8198ff9640 1 -- 172.21.15.70:0/4058525791 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7f819401c0a0 con 0x7f819c14e480 2023-10-08T05:57:22.693 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T05:57:22.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 -- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f817c06af90 msgr2=0x7f817c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:22.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f817c06af90 0x7f817c06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f818c034000 tx=0x7f818c007000 comp rx=0 tx=0).stop 2023-10-08T05:57:22.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 -- 172.21.15.70:0/4058525791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f819c14e480 msgr2=0x7f819c14e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:22.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f819c14e480 0x7f819c14e930 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f819c0cb0a0 tx=0x7f819400f4a0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 -- 172.21.15.70:0/4058525791 shutdown_connections 2023-10-08T05:57:22.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f817c06af90 0x7f817c06d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f819c14e480 0x7f819c14e930 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f819c15cdb0 0x7f819c15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 --2- 172.21.15.70:0/4058525791 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8188003970 0x7f819c156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:22.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 -- 172.21.15.70:0/4058525791 >> 172.21.15.70:0/4058525791 conn(0x7f819c069d50 msgr2=0x7f819c0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:22.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 -- 172.21.15.70:0/4058525791 shutdown_connections 2023-10-08T05:57:22.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:21.994+0000 7f81a19f0640 1 -- 172.21.15.70:0/4058525791 wait complete. 2023-10-08T05:57:22.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T05:57:22.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.919+0000 7f39996a9640 1 Processor -- start 2023-10-08T05:57:23.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.919+0000 7f39996a9640 1 -- start start 2023-10-08T05:57:23.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.921+0000 7f39996a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39940ad790 0x7f39940702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.921+0000 7f39996a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f399415e330 0x7f3994070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.921+0000 7f39996a9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399406dee0 con 0x7f3994159ac0 2023-10-08T05:57:23.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.921+0000 7f39996a9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399406dcf0 con 0x7f39940ad790 2023-10-08T05:57:23.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.921+0000 7f39996a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399415d850 con 0x7f399415e330 2023-10-08T05:57:23.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.922+0000 7f398bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39940ad790 0x7f39940702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.922+0000 7f3992ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3994159ac0 0x7f399406d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36054/0 (socket says 172.21.15.70:36054) 2023-10-08T05:57:23.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.922+0000 7f3992ffd640 1 -- 172.21.15.70:0/237644823 learned_addr learned my addr 172.21.15.70:0/237644823 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:23.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.922+0000 7f398bfff640 1 -- 172.21.15.70:0/237644823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f399415e330 msgr2=0x7f3994070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:23.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.922+0000 7f398bfff640 1 --2- 172.21.15.70:0/237644823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f399415e330 0x7f3994070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.922+0000 7f398bfff640 1 -- 172.21.15.70:0/237644823 >> v1:172.21.15.70:6789/0 conn(0x7f3994159ac0 legacy=0x7f399406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:23.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.922+0000 7f398bfff640 1 -- 172.21.15.70:0/237644823 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399415db30 con 0x7f39940ad790 2023-10-08T05:57:23.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.922+0000 7f39927fc640 1 -- 172.21.15.70:0/237644823 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821195445 0 0) 0x7f399406dee0 con 0x7f3994159ac0 2023-10-08T05:57:23.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.923+0000 7f398bfff640 1 --2- 172.21.15.70:0/237644823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39940ad790 0x7f39940702d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f397c0097a0 tx=0x7f397c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efd9dbb1d08a745 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:23.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39927fc640 1 -- 172.21.15.70:0/237644823 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c02d040 con 0x7f39940ad790 2023-10-08T05:57:23.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39927fc640 1 -- 172.21.15.70:0/237644823 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c031810 con 0x7f39940ad790 2023-10-08T05:57:23.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39927fc640 1 -- 172.21.15.70:0/237644823 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c031ba0 con 0x7f39940ad790 2023-10-08T05:57:23.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39996a9640 1 -- 172.21.15.70:0/237644823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39940ad790 msgr2=0x7f39940702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:23.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39996a9640 1 --2- 172.21.15.70:0/237644823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39940ad790 0x7f39940702d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f397c0097a0 tx=0x7f397c02b8b0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39996a9640 1 -- 172.21.15.70:0/237644823 shutdown_connections 2023-10-08T05:57:23.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39996a9640 1 --2- 172.21.15.70:0/237644823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f399415e330 0x7f3994070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39996a9640 1 --2- 172.21.15.70:0/237644823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39940ad790 0x7f39940702d0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39996a9640 1 -- 172.21.15.70:0/237644823 >> 172.21.15.70:0/237644823 conn(0x7f399406a380 msgr2=0x7f39940b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39996a9640 1 -- 172.21.15.70:0/237644823 shutdown_connections 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.924+0000 7f39996a9640 1 -- 172.21.15.70:0/237644823 wait complete. 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.926+0000 7f39996a9640 1 Processor -- start 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.926+0000 7f39996a9640 1 -- start start 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.927+0000 7f39996a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940ad790 0x7f3994158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.927+0000 7f39996a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3994159ac0 0x7f3994151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.927+0000 7f39996a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f399415e330 0x7f3994148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.927+0000 7f39996a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39941556f0 con 0x7f3994159ac0 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.927+0000 7f39996a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3994157c50 con 0x7f39940ad790 2023-10-08T05:57:23.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.927+0000 7f39996a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3994157f30 con 0x7f399415e330 2023-10-08T05:57:23.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.927+0000 7f398bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3994159ac0 0x7f3994151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f398bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3994159ac0 0x7f3994151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51984/0 (socket says 172.21.15.70:51984) 2023-10-08T05:57:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f398bfff640 1 -- 172.21.15.70:0/1062668163 learned_addr learned my addr 172.21.15.70:0/1062668163 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f3992ffd640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940ad790 0x7f3994158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f39937fe640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f399415e330 0x7f3994148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f398bfff640 1 -- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f399415e330 msgr2=0x7f3994148960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f398bfff640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f399415e330 0x7f3994148960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f398bfff640 1 -- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940ad790 msgr2=0x7f3994158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f398bfff640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940ad790 0x7f3994158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f398bfff640 1 -- 172.21.15.70:0/1062668163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c009400 con 0x7f3994159ac0 2023-10-08T05:57:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f3992ffd640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940ad790 0x7f3994158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f39937fe640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f399415e330 0x7f3994148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.928+0000 7f398bfff640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3994159ac0 0x7f3994151b90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f397c000c00 tx=0x7f397c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.929+0000 7f398b7fe640 1 -- 172.21.15.70:0/1062668163 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c03ad60 con 0x7f3994159ac0 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.929+0000 7f398b7fe640 1 -- 172.21.15.70:0/1062668163 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c0443f0 con 0x7f3994159ac0 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.929+0000 7f39996a9640 1 -- 172.21.15.70:0/1062668163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3994158210 con 0x7f3994159ac0 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.929+0000 7f398b7fe640 1 -- 172.21.15.70:0/1062668163 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c0447a0 con 0x7f3994159ac0 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.929+0000 7f39996a9640 1 -- 172.21.15.70:0/1062668163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39941587d0 con 0x7f3994159ac0 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.930+0000 7f39996a9640 1 -- 172.21.15.70:0/1062668163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39941589c0 con 0x7f3994159ac0 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.931+0000 7f398b7fe640 1 -- 172.21.15.70:0/1062668163 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f397c04d450 con 0x7f3994159ac0 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.932+0000 7f398b7fe640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f397406af90 0x7f397406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.932+0000 7f398b7fe640 1 -- 172.21.15.70:0/1062668163 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f397c0abbc0 con 0x7f3994159ac0 2023-10-08T05:57:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.933+0000 7f3992ffd640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f397406af90 0x7f397406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.934+0000 7f3992ffd640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f397406af90 0x7f397406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3984000f60 tx=0x7f3984009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:22.937+0000 7f398b7fe640 1 -- 172.21.15.70:0/1062668163 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f397c076c90 con 0x7f3994159ac0 2023-10-08T05:57:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.130+0000 7f39996a9640 1 -- 172.21.15.70:0/1062668163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3994067490 con 0x7f3994159ac0 2023-10-08T05:57:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.131+0000 7f398b7fe640 1 -- 172.21.15.70:0/1062668163 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v159) v1 ==== 84+0+65 (secure 0 0 0) 0x7f397c07a2b0 con 0x7f3994159ac0 2023-10-08T05:57:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:23.321 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:57:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T05:57:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T05:57:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:23.321 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:57:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 -- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f397406af90 msgr2=0x7f397406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f397406af90 0x7f397406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3984000f60 tx=0x7f3984009040 comp rx=0 tx=0).stop 2023-10-08T05:57:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 -- 172.21.15.70:0/1062668163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3994159ac0 msgr2=0x7f3994151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3994159ac0 0x7f3994151b90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f397c000c00 tx=0x7f397c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 -- 172.21.15.70:0/1062668163 shutdown_connections 2023-10-08T05:57:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f397406af90 0x7f397406d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f399415e330 0x7f3994148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3994159ac0 0x7f3994151b90 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 --2- 172.21.15.70:0/1062668163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940ad790 0x7f3994158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 -- 172.21.15.70:0/1062668163 >> 172.21.15.70:0/1062668163 conn(0x7f399406a380 msgr2=0x7f399406d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 -- 172.21.15.70:0/1062668163 shutdown_connections 2023-10-08T05:57:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.133+0000 7f39996a9640 1 -- 172.21.15.70:0/1062668163 wait complete. 2023-10-08T05:57:23.323 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T05:57:23.324 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T05:57:23.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.552+0000 7fda889c5640 1 Processor -- start 2023-10-08T05:57:23.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.552+0000 7fda889c5640 1 -- start start 2023-10-08T05:57:23.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.554+0000 7fda889c5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fda8415aad0 0x7fda840b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.554+0000 7fda889c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda8406d410 0x7fda840c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.554+0000 7fda889c5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda8415bf30 con 0x7fda840c82b0 2023-10-08T05:57:23.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.554+0000 7fda889c5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda8415b320 con 0x7fda8415aad0 2023-10-08T05:57:23.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.554+0000 7fda889c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8415b600 con 0x7fda8406d410 2023-10-08T05:57:23.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.554+0000 7fda8259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fda8415aad0 0x7fda840b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.554+0000 7fda8259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fda8415aad0 0x7fda840b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52120/0 (socket says 172.21.15.70:52120) 2023-10-08T05:57:23.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.554+0000 7fda8259c640 1 -- 172.21.15.70:0/3763703461 learned_addr learned my addr 172.21.15.70:0/3763703461 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:23.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.555+0000 7fda82d9d640 1 --2- 172.21.15.70:0/3763703461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda8406d410 0x7fda840c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.555+0000 7fda8259c640 1 -- 172.21.15.70:0/3763703461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda8406d410 msgr2=0x7fda840c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:23.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.555+0000 7fda8259c640 1 --2- 172.21.15.70:0/3763703461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda8406d410 0x7fda840c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.555+0000 7fda8259c640 1 -- 172.21.15.70:0/3763703461 >> v1:172.21.15.181:6789/0 conn(0x7fda840c82b0 legacy=0x7fda8406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:23.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.555+0000 7fda8259c640 1 -- 172.21.15.70:0/3763703461 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8415b8e0 con 0x7fda8415aad0 2023-10-08T05:57:23.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.556+0000 7fda8259c640 1 --2- 172.21.15.70:0/3763703461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda8415aad0 0x7fda840b1750 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fda74009870 tx=0x7fda7402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1621605679f274db server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:23.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.556+0000 7fda81d9b640 1 -- 172.21.15.70:0/3763703461 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74031640 con 0x7fda8415aad0 2023-10-08T05:57:23.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.556+0000 7fda81d9b640 1 -- 172.21.15.70:0/3763703461 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda740317a0 con 0x7fda8415aad0 2023-10-08T05:57:23.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.556+0000 7fda81d9b640 1 -- 172.21.15.70:0/3763703461 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74031b30 con 0x7fda8415aad0 2023-10-08T05:57:23.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.557+0000 7fda889c5640 1 -- 172.21.15.70:0/3763703461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda8415aad0 msgr2=0x7fda840b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:23.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.557+0000 7fda889c5640 1 --2- 172.21.15.70:0/3763703461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda8415aad0 0x7fda840b1750 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fda74009870 tx=0x7fda7402b8f0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.557+0000 7fda889c5640 1 -- 172.21.15.70:0/3763703461 shutdown_connections 2023-10-08T05:57:23.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.557+0000 7fda889c5640 1 --2- 172.21.15.70:0/3763703461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda8406d410 0x7fda840c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.557+0000 7fda889c5640 1 --2- 172.21.15.70:0/3763703461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda8415aad0 0x7fda840b1750 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.557+0000 7fda889c5640 1 -- 172.21.15.70:0/3763703461 >> 172.21.15.70:0/3763703461 conn(0x7fda84069d50 msgr2=0x7fda8406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:23.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.557+0000 7fda889c5640 1 -- 172.21.15.70:0/3763703461 shutdown_connections 2023-10-08T05:57:23.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.557+0000 7fda889c5640 1 -- 172.21.15.70:0/3763703461 wait complete. 2023-10-08T05:57:23.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.559+0000 7fda889c5640 1 Processor -- start 2023-10-08T05:57:23.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.560+0000 7fda889c5640 1 -- start start 2023-10-08T05:57:23.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.561+0000 7fda889c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda8406d410 0x7fda840b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.561+0000 7fda889c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda840c82b0 0x7fda841f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.561+0000 7fda889c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda8415aad0 0x7fda841f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.561+0000 7fda889c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8414dad0 con 0x7fda840c82b0 2023-10-08T05:57:23.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.561+0000 7fda889c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda84148b80 con 0x7fda8415aad0 2023-10-08T05:57:23.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.561+0000 7fda889c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda84154c10 con 0x7fda8406d410 2023-10-08T05:57:23.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.561+0000 7fda7bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda840c82b0 0x7fda841f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.561+0000 7fda7bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda840c82b0 0x7fda841f1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52130/0 (socket says 172.21.15.70:52130) 2023-10-08T05:57:23.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.561+0000 7fda7bfff640 1 -- 172.21.15.70:0/441912448 learned_addr learned my addr 172.21.15.70:0/441912448 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:23.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.561+0000 7fda8259c640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda8406d410 0x7fda840b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda82d9d640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda8415aad0 0x7fda841f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda7bfff640 1 -- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda8406d410 msgr2=0x7fda840b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda7bfff640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda8406d410 0x7fda840b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda7bfff640 1 -- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda8415aad0 msgr2=0x7fda841f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda7bfff640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda8415aad0 0x7fda841f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda7bfff640 1 -- 172.21.15.70:0/441912448 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda740094d0 con 0x7fda840c82b0 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda82d9d640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda8415aad0 0x7fda841f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda7bfff640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda840c82b0 0x7fda841f1790 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00c460 tx=0x7fda6c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda7affd640 1 -- 172.21.15.70:0/441912448 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c012830 con 0x7fda840c82b0 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda7affd640 1 -- 172.21.15.70:0/441912448 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c0129d0 con 0x7fda840c82b0 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda7affd640 1 -- 172.21.15.70:0/441912448 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c012d80 con 0x7fda840c82b0 2023-10-08T05:57:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.562+0000 7fda889c5640 1 -- 172.21.15.70:0/441912448 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda84154650 con 0x7fda840c82b0 2023-10-08T05:57:23.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.563+0000 7fda889c5640 1 -- 172.21.15.70:0/441912448 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda840b17b0 con 0x7fda840c82b0 2023-10-08T05:57:23.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.563+0000 7fda889c5640 1 -- 172.21.15.70:0/441912448 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8415c0c0 con 0x7fda840c82b0 2023-10-08T05:57:23.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.565+0000 7fda7affd640 1 -- 172.21.15.70:0/441912448 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fda6c01b400 con 0x7fda840c82b0 2023-10-08T05:57:23.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.565+0000 7fda7affd640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fda6406af90 0x7fda6406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:23.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.566+0000 7fda7affd640 1 -- 172.21.15.70:0/441912448 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fda6c089550 con 0x7fda840c82b0 2023-10-08T05:57:23.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.566+0000 7fda8259c640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fda6406af90 0x7fda6406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:23.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.567+0000 7fda8259c640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fda6406af90 0x7fda6406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fda74036000 tx=0x7fda7403c040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:23.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.570+0000 7fda7affd640 1 -- 172.21.15.70:0/441912448 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda6c0546a0 con 0x7fda840c82b0 2023-10-08T05:57:23.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:23.759+0000 7fda889c5640 1 -- 172.21.15.70:0/441912448 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fda8404e160 con 0x7fda840c82b0 2023-10-08T05:57:24.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.497+0000 7fda7affd640 1 -- 172.21.15.70:0/441912448 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v160) v1 ==== 138+0+0 (secure 0 0 0) 0x7fda6c057cc0 con 0x7fda840c82b0 2023-10-08T05:57:24.554 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T05:57:24.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.501+0000 7fda889c5640 1 -- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fda6406af90 msgr2=0x7fda6406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:24.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.501+0000 7fda889c5640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fda6406af90 0x7fda6406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fda74036000 tx=0x7fda7403c040 comp rx=0 tx=0).stop 2023-10-08T05:57:24.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.501+0000 7fda889c5640 1 -- 172.21.15.70:0/441912448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda840c82b0 msgr2=0x7fda841f1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:24.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.501+0000 7fda889c5640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda840c82b0 0x7fda841f1790 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fda6c00c460 tx=0x7fda6c00c920 comp rx=0 tx=0).stop 2023-10-08T05:57:24.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.501+0000 7fda889c5640 1 -- 172.21.15.70:0/441912448 shutdown_connections 2023-10-08T05:57:24.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.501+0000 7fda889c5640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fda6406af90 0x7fda6406d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:24.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.501+0000 7fda889c5640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda8415aad0 0x7fda841f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:24.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.501+0000 7fda889c5640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda840c82b0 0x7fda841f1790 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:24.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.501+0000 7fda889c5640 1 --2- 172.21.15.70:0/441912448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda8406d410 0x7fda840b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:24.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.501+0000 7fda889c5640 1 -- 172.21.15.70:0/441912448 >> 172.21.15.70:0/441912448 conn(0x7fda84069d50 msgr2=0x7fda84154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:24.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.502+0000 7fda889c5640 1 -- 172.21.15.70:0/441912448 shutdown_connections 2023-10-08T05:57:24.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.502+0000 7fda889c5640 1 -- 172.21.15.70:0/441912448 wait complete. 2023-10-08T05:57:24.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T05:57:24.656 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-08T05:57:24.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.784+0000 7fc8ad002640 1 Processor -- start 2023-10-08T05:57:24.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.784+0000 7fc8ad002640 1 -- start start 2023-10-08T05:57:24.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.786+0000 7fc8ad002640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a80d09d0 0x7fc8a80d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:24.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.786+0000 7fc8ad002640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a80d1370 con 0x7fc8a80b0c20 2023-10-08T05:57:24.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.786+0000 7fc8ad002640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a80b1990 con 0x7fc8a815af40 2023-10-08T05:57:24.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.786+0000 7fc8ad002640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80cbb10 con 0x7fc8a80d09d0 2023-10-08T05:57:24.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.786+0000 7fc8a759e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc8a80b0c20 0x7fc8a80b10b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50522/0 (socket says 172.21.15.70:50522) 2023-10-08T05:57:24.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.786+0000 7fc8a759e640 1 -- 172.21.15.70:0/904104576 learned_addr learned my addr 172.21.15.70:0/904104576 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:24.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.786+0000 7fc8a659c640 1 --2- 172.21.15.70:0/904104576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a80d09d0 0x7fc8a80d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:24.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.787+0000 7fc8a5d9b640 1 -- 172.21.15.70:0/904104576 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352515419 0 0) 0x7fc8a80d1370 con 0x7fc8a80b0c20 2023-10-08T05:57:24.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.787+0000 7fc8a5d9b640 1 -- 172.21.15.70:0/904104576 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc894003520 con 0x7fc8a80b0c20 2023-10-08T05:57:24.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.787+0000 7fc8a5d9b640 1 -- 172.21.15.70:0/904104576 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399185059 0 0) 0x7fc8a80b1990 con 0x7fc8a815af40 2023-10-08T05:57:24.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.787+0000 7fc8a5d9b640 1 -- 172.21.15.70:0/904104576 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8a80d1370 con 0x7fc8a815af40 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.787+0000 7fc8a5d9b640 1 -- 172.21.15.70:0/904104576 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1062917906 0 0) 0x7fc894003520 con 0x7fc8a80b0c20 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.787+0000 7fc8a659c640 1 -- 172.21.15.70:0/904104576 >> v1:172.21.15.181:6789/0 conn(0x7fc8a815af40 legacy=0x7fc8a80c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.787+0000 7fc8a659c640 1 -- 172.21.15.70:0/904104576 >> v1:172.21.15.70:6789/0 conn(0x7fc8a80b0c20 legacy=0x7fc8a80b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.787+0000 7fc8a659c640 1 -- 172.21.15.70:0/904104576 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80cbdf0 con 0x7fc8a80d09d0 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.789+0000 7fc8a659c640 1 --2- 172.21.15.70:0/904104576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a80d09d0 0x7fc8a80d0e30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc8900097d0 tx=0x7fc89002b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52c90963cdec3e6c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.789+0000 7fc8a5d9b640 1 -- 172.21.15.70:0/904104576 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a00016c0 con 0x7fc8a80d09d0 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.789+0000 7fc8a5d9b640 1 -- 172.21.15.70:0/904104576 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89002e050 con 0x7fc8a80d09d0 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.789+0000 7fc8a5d9b640 1 -- 172.21.15.70:0/904104576 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc890030710 con 0x7fc8a80d09d0 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.790+0000 7fc8ad002640 1 -- 172.21.15.70:0/904104576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a80d09d0 msgr2=0x7fc8a80d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.790+0000 7fc8ad002640 1 --2- 172.21.15.70:0/904104576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a80d09d0 0x7fc8a80d0e30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc8900097d0 tx=0x7fc89002b800 comp rx=0 tx=0).stop 2023-10-08T05:57:24.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.790+0000 7fc8ad002640 1 -- 172.21.15.70:0/904104576 shutdown_connections 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.790+0000 7fc8ad002640 1 --2- 172.21.15.70:0/904104576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a80d09d0 0x7fc8a80d0e30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.790+0000 7fc8ad002640 1 -- 172.21.15.70:0/904104576 >> 172.21.15.70:0/904104576 conn(0x7fc8a8089fe0 msgr2=0x7fc8a8099d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.790+0000 7fc8ad002640 1 -- 172.21.15.70:0/904104576 shutdown_connections 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.790+0000 7fc8ad002640 1 -- 172.21.15.70:0/904104576 wait complete. 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.792+0000 7fc8ad002640 1 Processor -- start 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.792+0000 7fc8ad002640 1 -- start start 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8ad002640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8a80b0c20 0x7fc8a814bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8ad002640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a80d09d0 0x7fc8a80ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8ad002640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a815af40 0x7fc8a815be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8ad002640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a806b700 con 0x7fc8a80b0c20 2023-10-08T05:57:24.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8ad002640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a806b9e0 con 0x7fc8a80d09d0 2023-10-08T05:57:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8ad002640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a80c95a0 con 0x7fc8a815af40 2023-10-08T05:57:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8a6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8a80b0c20 0x7fc8a814bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8a6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8a80b0c20 0x7fc8a814bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52140/0 (socket says 172.21.15.70:52140) 2023-10-08T05:57:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8a6d9d640 1 -- 172.21.15.70:0/3534835419 learned_addr learned my addr 172.21.15.70:0/3534835419 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8a759e640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a815af40 0x7fc8a815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8a659c640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a80d09d0 0x7fc8a80ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:24.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8a6d9d640 1 -- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a815af40 msgr2=0x7fc8a815be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8a6d9d640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a815af40 0x7fc8a815be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8a6d9d640 1 -- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a80d09d0 msgr2=0x7fc8a80ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.793+0000 7fc8a6d9d640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a80d09d0 0x7fc8a80ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.794+0000 7fc8a6d9d640 1 -- 172.21.15.70:0/3534835419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc890009400 con 0x7fc8a80b0c20 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.794+0000 7fc8a659c640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a80d09d0 0x7fc8a80ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.794+0000 7fc8a759e640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a815af40 0x7fc8a815be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.794+0000 7fc8a6d9d640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8a80b0c20 0x7fc8a814bfc0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fc89800c790 tx=0x7fc89800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.794+0000 7fc89f7fe640 1 -- 172.21.15.70:0/3534835419 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89800cea0 con 0x7fc8a80b0c20 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.794+0000 7fc89f7fe640 1 -- 172.21.15.70:0/3534835419 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc898003bc0 con 0x7fc8a80b0c20 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.794+0000 7fc8ad002640 1 -- 172.21.15.70:0/3534835419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a80c9880 con 0x7fc8a80b0c20 2023-10-08T05:57:24.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.794+0000 7fc89f7fe640 1 -- 172.21.15.70:0/3534835419 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89800a850 con 0x7fc8a80b0c20 2023-10-08T05:57:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.794+0000 7fc8ad002640 1 -- 172.21.15.70:0/3534835419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a815e2a0 con 0x7fc8a80b0c20 2023-10-08T05:57:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.795+0000 7fc8ad002640 1 -- 172.21.15.70:0/3534835419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a815e430 con 0x7fc8a80b0c20 2023-10-08T05:57:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.797+0000 7fc89f7fe640 1 -- 172.21.15.70:0/3534835419 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc89800a9b0 con 0x7fc8a80b0c20 2023-10-08T05:57:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.798+0000 7fc89f7fe640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fc88806af90 0x7fc88806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.798+0000 7fc89f7fe640 1 -- 172.21.15.70:0/3534835419 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fc89808a170 con 0x7fc8a80b0c20 2023-10-08T05:57:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.798+0000 7fc8a659c640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fc88806af90 0x7fc88806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.799+0000 7fc8a659c640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fc88806af90 0x7fc88806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc890034000 tx=0x7fc890008030 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.802+0000 7fc89f7fe640 1 -- 172.21.15.70:0/3534835419 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc898055140 con 0x7fc8a80b0c20 2023-10-08T05:57:24.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:24.990+0000 7fc8ad002640 1 -- 172.21.15.70:0/3534835419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc8a806b840 con 0x7fc8a80b0c20 2023-10-08T05:57:25.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.517+0000 7fc89f7fe640 1 -- 172.21.15.70:0/3534835419 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v161) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc8a806b840 con 0x7fc8a80b0c20 2023-10-08T05:57:25.524 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T05:57:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.521+0000 7fc8ad002640 1 -- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fc88806af90 msgr2=0x7fc88806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.521+0000 7fc8ad002640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fc88806af90 0x7fc88806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc890034000 tx=0x7fc890008030 comp rx=0 tx=0).stop 2023-10-08T05:57:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.521+0000 7fc8ad002640 1 -- 172.21.15.70:0/3534835419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8a80b0c20 msgr2=0x7fc8a814bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.521+0000 7fc8ad002640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8a80b0c20 0x7fc8a814bfc0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fc89800c790 tx=0x7fc89800cc50 comp rx=0 tx=0).stop 2023-10-08T05:57:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.522+0000 7fc8ad002640 1 -- 172.21.15.70:0/3534835419 shutdown_connections 2023-10-08T05:57:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.522+0000 7fc8ad002640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fc88806af90 0x7fc88806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.522+0000 7fc8ad002640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8a815af40 0x7fc8a815be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.522+0000 7fc8ad002640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8a80d09d0 0x7fc8a80ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.522+0000 7fc8ad002640 1 --2- 172.21.15.70:0/3534835419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8a80b0c20 0x7fc8a814bfc0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.522+0000 7fc8ad002640 1 -- 172.21.15.70:0/3534835419 >> 172.21.15.70:0/3534835419 conn(0x7fc8a8089fe0 msgr2=0x7fc8a80ad6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.522+0000 7fc8ad002640 1 -- 172.21.15.70:0/3534835419 shutdown_connections 2023-10-08T05:57:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.522+0000 7fc8ad002640 1 -- 172.21.15.70:0/3534835419 wait complete. 2023-10-08T05:57:25.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T05:57:25.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.825+0000 7f7077610640 1 Processor -- start 2023-10-08T05:57:25.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.825+0000 7f7077610640 1 -- start start 2023-10-08T05:57:25.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.826+0000 7f7077610640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f70700afbe0 0x7f70700b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:25.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.826+0000 7f7077610640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70700cb130 0x7f70700cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:25.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.826+0000 7f7077610640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70700cbad0 con 0x7f707006dfc0 2023-10-08T05:57:25.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.826+0000 7f7077610640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70700b2620 con 0x7f70700afbe0 2023-10-08T05:57:25.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.826+0000 7f7077610640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70700b2060 con 0x7f70700cb130 2023-10-08T05:57:25.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.827+0000 7f7074bab640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f707006dfc0 0x7f7070070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50534/0 (socket says 172.21.15.70:50534) 2023-10-08T05:57:25.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.827+0000 7f7074bab640 1 -- 172.21.15.70:0/970141626 learned_addr learned my addr 172.21.15.70:0/970141626 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:25.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.827+0000 7f70753ac640 1 --2- 172.21.15.70:0/970141626 >> v2:172.21.15.70:3300/0 conn(0x7f70700afbe0 0x7f70700b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:25.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.827+0000 7f7075bad640 1 --2- 172.21.15.70:0/970141626 >> v2:172.21.15.181:3300/0 conn(0x7f70700cb130 0x7f70700cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:25.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.827+0000 7f7067fff640 1 -- 172.21.15.70:0/970141626 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957931178 0 0) 0x7f70700cbad0 con 0x7f707006dfc0 2023-10-08T05:57:25.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.827+0000 7f7067fff640 1 -- 172.21.15.70:0/970141626 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f705c003520 con 0x7f707006dfc0 2023-10-08T05:57:25.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.827+0000 7f70753ac640 1 -- 172.21.15.70:0/970141626 >> v2:172.21.15.181:3300/0 conn(0x7f70700cb130 msgr2=0x7f70700cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.827+0000 7f70753ac640 1 --2- 172.21.15.70:0/970141626 >> v2:172.21.15.181:3300/0 conn(0x7f70700cb130 0x7f70700cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.827+0000 7f70753ac640 1 -- 172.21.15.70:0/970141626 >> v1:172.21.15.70:6789/0 conn(0x7f707006dfc0 legacy=0x7f7070070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.827+0000 7f70753ac640 1 -- 172.21.15.70:0/970141626 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70700b2340 con 0x7f70700afbe0 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.828+0000 7f70753ac640 1 --2- 172.21.15.70:0/970141626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700afbe0 0x7f70700b15b0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f70700c8d10 tx=0x7f706c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4dc5e47ac29404 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7067fff640 1 -- 172.21.15.70:0/970141626 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c031830 con 0x7f70700afbe0 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7067fff640 1 -- 172.21.15.70:0/970141626 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c031990 con 0x7f70700afbe0 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7067fff640 1 -- 172.21.15.70:0/970141626 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c031d00 con 0x7f70700afbe0 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7077610640 1 -- 172.21.15.70:0/970141626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700afbe0 msgr2=0x7f70700b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7077610640 1 --2- 172.21.15.70:0/970141626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700afbe0 0x7f70700b15b0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f70700c8d10 tx=0x7f706c02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7077610640 1 -- 172.21.15.70:0/970141626 shutdown_connections 2023-10-08T05:57:25.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7077610640 1 --2- 172.21.15.70:0/970141626 >> v2:172.21.15.181:3300/0 conn(0x7f70700cb130 0x7f70700cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:25.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7077610640 1 --2- 172.21.15.70:0/970141626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700afbe0 0x7f70700b15b0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:25.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7077610640 1 -- 172.21.15.70:0/970141626 >> 172.21.15.70:0/970141626 conn(0x7f707006a5d0 msgr2=0x7f70700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:25.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7077610640 1 -- 172.21.15.70:0/970141626 shutdown_connections 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.829+0000 7f7077610640 1 -- 172.21.15.70:0/970141626 wait complete. 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.831+0000 7f7077610640 1 Processor -- start 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.831+0000 7f7077610640 1 -- start start 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.832+0000 7f7077610640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f707006dfc0 0x7f70700c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.832+0000 7f7077610640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70700afbe0 0x7f70700cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.832+0000 7f7077610640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700cb130 0x7f70700ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.832+0000 7f7077610640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70700c4c60 con 0x7f70700cb130 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.832+0000 7f7077610640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70701f2bf0 con 0x7f707006dfc0 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.832+0000 7f7077610640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70700c4290 con 0x7f70700afbe0 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f7075bad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700cb130 0x7f70700ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f7075bad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700cb130 0x7f70700ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52154/0 (socket says 172.21.15.70:52154) 2023-10-08T05:57:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f7075bad640 1 -- 172.21.15.70:0/2384816698 learned_addr learned my addr 172.21.15.70:0/2384816698 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f70753ac640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f707006dfc0 0x7f70700c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f7074bab640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70700afbe0 0x7f70700cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f7075bad640 1 -- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70700afbe0 msgr2=0x7f70700cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f7075bad640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70700afbe0 0x7f70700cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f7075bad640 1 -- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f707006dfc0 msgr2=0x7f70700c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f7075bad640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f707006dfc0 0x7f70700c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f7075bad640 1 -- 172.21.15.70:0/2384816698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c009400 con 0x7f70700cb130 2023-10-08T05:57:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f70753ac640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f707006dfc0 0x7f70700c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.833+0000 7f7075bad640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700cb130 0x7f70700ccb80 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7068007e60 tx=0x7f706800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.834+0000 7f7065ffb640 1 -- 172.21.15.70:0/2384816698 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068014420 con 0x7f70700cb130 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.834+0000 7f7065ffb640 1 -- 172.21.15.70:0/2384816698 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70680145a0 con 0x7f70700cb130 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.834+0000 7f7065ffb640 1 -- 172.21.15.70:0/2384816698 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068014950 con 0x7f70700cb130 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.834+0000 7f7077610640 1 -- 172.21.15.70:0/2384816698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70700c3680 con 0x7f70700cb130 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.834+0000 7f7077610640 1 -- 172.21.15.70:0/2384816698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70700c3c40 con 0x7f70700cb130 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.835+0000 7f7077610640 1 -- 172.21.15.70:0/2384816698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70700c4440 con 0x7f70700cb130 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.836+0000 7f7065ffb640 1 -- 172.21.15.70:0/2384816698 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7068014ab0 con 0x7f70700cb130 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.837+0000 7f7065ffb640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f705406af90 0x7f705406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.837+0000 7f7065ffb640 1 -- 172.21.15.70:0/2384816698 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f706808a240 con 0x7f70700cb130 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.838+0000 7f70753ac640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f705406af90 0x7f705406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.839+0000 7f70753ac640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f705406af90 0x7f705406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f706c036000 tx=0x7f706c008030 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:25.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:25.842+0000 7f7065ffb640 1 -- 172.21.15.70:0/2384816698 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70700c4440 con 0x7f70700cb130 2023-10-08T05:57:26.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.071+0000 7f7077610640 1 -- 172.21.15.70:0/2384816698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f70700c4400 con 0x7f70700cb130 2023-10-08T05:57:26.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.545+0000 7f7065ffb640 1 -- 172.21.15.70:0/2384816698 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v73) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7068055010 con 0x7f70700cb130 2023-10-08T05:57:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 19 and data pool 20 2023-10-08T05:57:26.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.551+0000 7f7077610640 1 -- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f705406af90 msgr2=0x7f705406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:26.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.551+0000 7f7077610640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f705406af90 0x7f705406d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f706c036000 tx=0x7f706c008030 comp rx=0 tx=0).stop 2023-10-08T05:57:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.551+0000 7f7077610640 1 -- 172.21.15.70:0/2384816698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700cb130 msgr2=0x7f70700ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.551+0000 7f7077610640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700cb130 0x7f70700ccb80 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7068007e60 tx=0x7f706800e4b0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.551+0000 7f7077610640 1 -- 172.21.15.70:0/2384816698 shutdown_connections 2023-10-08T05:57:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.551+0000 7f7077610640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f705406af90 0x7f705406d450 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f706c036000 tx=0x7f706c008030 comp rx=0 tx=0).stop 2023-10-08T05:57:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.551+0000 7f7077610640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70700cb130 0x7f70700ccb80 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.551+0000 7f7077610640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70700afbe0 0x7f70700cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.551+0000 7f7077610640 1 --2- 172.21.15.70:0/2384816698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f707006dfc0 0x7f70700c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.551+0000 7f7077610640 1 -- 172.21.15.70:0/2384816698 >> 172.21.15.70:0/2384816698 conn(0x7f707006a5d0 msgr2=0x7f70700cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.560+0000 7f7077610640 1 -- 172.21.15.70:0/2384816698 shutdown_connections 2023-10-08T05:57:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.561+0000 7f7077610640 1 -- 172.21.15.70:0/2384816698 wait complete. 2023-10-08T05:57:26.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:57:26.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.790+0000 7f4356339640 1 Processor -- start 2023-10-08T05:57:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.790+0000 7f4356339640 1 -- start start 2023-10-08T05:57:26.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.792+0000 7f4356339640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4350165cc0 0x7f43501680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:26.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.792+0000 7f4356339640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43500c48f0 con 0x7f435015cd90 2023-10-08T05:57:26.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.792+0000 7f4356339640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43500710f0 con 0x7f4350160940 2023-10-08T05:57:26.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.792+0000 7f4356339640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43500c4d80 con 0x7f4350165cc0 2023-10-08T05:57:26.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.792+0000 7f434ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f435015cd90 0x7f43500702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50538/0 (socket says 172.21.15.70:50538) 2023-10-08T05:57:26.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.792+0000 7f434ffff640 1 -- 172.21.15.70:0/3749850361 learned_addr learned my addr 172.21.15.70:0/3749850361 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:26.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097831727 0 0) 0x7f43500c48f0 con 0x7f435015cd90 2023-10-08T05:57:26.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f433c003520 con 0x7f435015cd90 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f43548d6640 1 --2- 172.21.15.70:0/3749850361 >> v2:172.21.15.181:3300/0 conn(0x7f4350165cc0 0x7f43501680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 747591857 0 0) 0x7f433c003520 con 0x7f435015cd90 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 >> v2:172.21.15.181:3300/0 conn(0x7f4350165cc0 msgr2=0x7f43501680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 --2- 172.21.15.70:0/3749850361 >> v2:172.21.15.181:3300/0 conn(0x7f4350165cc0 0x7f43501680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 >> v1:172.21.15.181:6789/0 conn(0x7f4350160940 legacy=0x7f43500709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500c3ce0 con 0x7f435015cd90 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4340001670 con 0x7f435015cd90 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 >> v1:172.21.15.70:6789/0 conn(0x7f435015cd90 legacy=0x7f43500702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433c0039f0 0x7f433c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433c0047c0 0x7f433c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:26.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f433c0052a0 0x7f433c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c00c120 con 0x7f433c0052a0 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c00c290 con 0x7f433c0047c0 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c00c450 con 0x7f433c0039f0 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.793+0000 7f43548d6640 1 --2- 172.21.15.70:0/3749850361 >> v2:172.21.15.181:3300/0 conn(0x7f4350165cc0 0x7f43501680b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.794+0000 7f43548d6640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f433c0052a0 0x7f433c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.794+0000 7f434ffff640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433c0039f0 0x7f433c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.794+0000 7f4347fff640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433c0047c0 0x7f433c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.794+0000 7f43548d6640 1 -- 172.21.15.70:0/3749850361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433c0039f0 msgr2=0x7f433c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.794+0000 7f43548d6640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433c0039f0 0x7f433c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.794+0000 7f43548d6640 1 -- 172.21.15.70:0/3749850361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433c0047c0 msgr2=0x7f433c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:26.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.794+0000 7f43548d6640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433c0047c0 0x7f433c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.794+0000 7f43548d6640 1 -- 172.21.15.70:0/3749850361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c003520 con 0x7f433c0052a0 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.794+0000 7f4347fff640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f433c0047c0 0x7f433c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.795+0000 7f43548d6640 1 -- 172.21.15.70:0/3749850361 reap_dead start 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.795+0000 7f43548d6640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f433c0052a0 0x7f433c00b980 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f43500c36f0 tx=0x7f433c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d6aba6abe51cc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.795+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434802cd60 con 0x7f433c0052a0 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.796+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348034440 con 0x7f433c0052a0 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.796+0000 7f434f7fe640 1 -- 172.21.15.70:0/3749850361 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43480347d0 con 0x7f433c0052a0 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.796+0000 7f4356339640 1 -- 172.21.15.70:0/3749850361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f433c0052a0 msgr2=0x7f433c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.796+0000 7f4356339640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f433c0052a0 0x7f433c00b980 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f43500c36f0 tx=0x7f433c0057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.796+0000 7f4356339640 1 -- 172.21.15.70:0/3749850361 shutdown_connections 2023-10-08T05:57:26.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.796+0000 7f4356339640 1 --2- 172.21.15.70:0/3749850361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f433c0052a0 0x7f433c00b980 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.796+0000 7f4356339640 1 -- 172.21.15.70:0/3749850361 >> 172.21.15.70:0/3749850361 conn(0x7f435006a380 msgr2=0x7f43500b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:26.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.796+0000 7f4356339640 1 -- 172.21.15.70:0/3749850361 shutdown_connections 2023-10-08T05:57:26.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.796+0000 7f4356339640 1 -- 172.21.15.70:0/3749850361 wait complete. 2023-10-08T05:57:26.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.798+0000 7f4356339640 1 Processor -- start 2023-10-08T05:57:26.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.798+0000 7f4356339640 1 -- start start 2023-10-08T05:57:26.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.799+0000 7f4356339640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433c0052a0 0x7f435014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:26.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.799+0000 7f4356339640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f435014d180 0x7f43500ac120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:26.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.799+0000 7f4356339640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43500b1580 0x7f43500b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:26.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.799+0000 7f4356339640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43500b44a0 con 0x7f43500b1580 2023-10-08T05:57:26.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.799+0000 7f4356339640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43500b3cb0 con 0x7f435014d180 2023-10-08T05:57:26.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.799+0000 7f4356339640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435014ffd0 con 0x7f433c0052a0 2023-10-08T05:57:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.800+0000 7f43548d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43500b1580 0x7f43500b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.800+0000 7f43548d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43500b1580 0x7f43500b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52168/0 (socket says 172.21.15.70:52168) 2023-10-08T05:57:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.800+0000 7f43548d6640 1 -- 172.21.15.70:0/1969949901 learned_addr learned my addr 172.21.15.70:0/1969949901 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.800+0000 7f43548d6640 1 -- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433c0052a0 msgr2=0x7f435014fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.800+0000 7f434ffff640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433c0052a0 0x7f435014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.800+0000 7f43548d6640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433c0052a0 0x7f435014fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.800+0000 7f4347fff640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f435014d180 0x7f43500ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.800+0000 7f43548d6640 1 -- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f435014d180 msgr2=0x7f43500ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.800+0000 7f43548d6640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f435014d180 0x7f43500ac120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:26.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.800+0000 7f43548d6640 1 -- 172.21.15.70:0/1969949901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500adb40 con 0x7f43500b1580 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.801+0000 7f43548d6640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43500b1580 0x7f43500b1a40 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f434802c7d0 tx=0x7f4348004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.801+0000 7f434d7fa640 1 -- 172.21.15.70:0/1969949901 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348044070 con 0x7f43500b1580 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.801+0000 7f434d7fa640 1 -- 172.21.15.70:0/1969949901 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434803f650 con 0x7f43500b1580 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.801+0000 7f4356339640 1 -- 172.21.15.70:0/1969949901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43500b3ee0 con 0x7f43500b1580 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.801+0000 7f434d7fa640 1 -- 172.21.15.70:0/1969949901 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348009690 con 0x7f43500b1580 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.801+0000 7f4356339640 1 -- 172.21.15.70:0/1969949901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43500adde0 con 0x7f43500b1580 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.802+0000 7f4356339640 1 -- 172.21.15.70:0/1969949901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43500ae000 con 0x7f43500b1580 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.803+0000 7f434d7fa640 1 -- 172.21.15.70:0/1969949901 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f434803b040 con 0x7f43500b1580 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.804+0000 7f434d7fa640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f433006afc0 0x7f433006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.804+0000 7f434d7fa640 1 -- 172.21.15.70:0/1969949901 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f43480aaf70 con 0x7f43500b1580 2023-10-08T05:57:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.805+0000 7f434ffff640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f433006afc0 0x7f433006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:26.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.806+0000 7f434ffff640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f433006afc0 0x7f433006d480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4340001650 tx=0x7f4340000ed0 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:26.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:26.808+0000 7f434d7fa640 1 -- 172.21.15.70:0/1969949901 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4348075ce0 con 0x7f43500b1580 2023-10-08T05:57:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.001+0000 7f4356339640 1 -- 172.21.15.70:0/1969949901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43500c4270 con 0x7f43500b1580 2023-10-08T05:57:27.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.003+0000 7f434d7fa640 1 -- 172.21.15.70:0/1969949901 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16848 (secure 0 0 0) 0x7f4348039090 con 0x7f43500b1580 2023-10-08T05:57:27.046 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:27.047 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":162,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:57:26.073654+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:57:23.815675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-08T05:57:25.045292+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:57:27.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 -- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f433006afc0 msgr2=0x7f433006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f433006afc0 0x7f433006d480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4340001650 tx=0x7f4340000ed0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 -- 172.21.15.70:0/1969949901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43500b1580 msgr2=0x7f43500b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43500b1580 0x7f43500b1a40 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f434802c7d0 tx=0x7f4348004ab0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 -- 172.21.15.70:0/1969949901 shutdown_connections 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f433006afc0 0x7f433006d480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43500b1580 0x7f43500b1a40 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f435014d180 0x7f43500ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 --2- 172.21.15.70:0/1969949901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f433c0052a0 0x7f435014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 -- 172.21.15.70:0/1969949901 >> 172.21.15.70:0/1969949901 conn(0x7f435006a380 msgr2=0x7f43500af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 -- 172.21.15.70:0/1969949901 shutdown_connections 2023-10-08T05:57:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.006+0000 7f4356339640 1 -- 172.21.15.70:0/1969949901 wait complete. 2023-10-08T05:57:27.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:57:27.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.298+0000 7f9659e1a640 1 Processor -- start 2023-10-08T05:57:27.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.298+0000 7f9659e1a640 1 -- start start 2023-10-08T05:57:27.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.299+0000 7f9659e1a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540adc70 0x7f96540b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.299+0000 7f9659e1a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f965415af10 0x7f96540b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.299+0000 7f9659e1a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965415f7d0 con 0x7f965415ab30 2023-10-08T05:57:27.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.299+0000 7f9659e1a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965415efb0 con 0x7f965415af10 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.299+0000 7f9659e1a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965415f210 con 0x7f96540adc70 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.300+0000 7f9659619640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f965415af10 0x7f96540b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.300+0000 7f9659619640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f965415af10 0x7f96540b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52182/0 (socket says 172.21.15.70:52182) 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.300+0000 7f9659619640 1 -- 172.21.15.70:0/1945149579 learned_addr learned my addr 172.21.15.70:0/1945149579 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.300+0000 7f9653fff640 1 --2- 172.21.15.70:0/1945149579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540adc70 0x7f96540b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.300+0000 7f9659619640 1 -- 172.21.15.70:0/1945149579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540adc70 msgr2=0x7f96540b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.300+0000 7f9659619640 1 --2- 172.21.15.70:0/1945149579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540adc70 0x7f96540b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.300+0000 7f9659619640 1 -- 172.21.15.70:0/1945149579 >> v1:172.21.15.181:6789/0 conn(0x7f965415ab30 legacy=0x7f965415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.300+0000 7f9659619640 1 -- 172.21.15.70:0/1945149579 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965415f4f0 con 0x7f965415af10 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.302+0000 7f9659619640 1 --2- 172.21.15.70:0/1945149579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 0x7f96540b0600 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f96540cb000 tx=0x7f964c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4467a82ed107b5b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:27.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.302+0000 7f96537fe640 1 -- 172.21.15.70:0/1945149579 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c031830 con 0x7f965415af10 2023-10-08T05:57:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.302+0000 7f96537fe640 1 -- 172.21.15.70:0/1945149579 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c031990 con 0x7f965415af10 2023-10-08T05:57:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.302+0000 7f96537fe640 1 -- 172.21.15.70:0/1945149579 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c031d20 con 0x7f965415af10 2023-10-08T05:57:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.303+0000 7f9659e1a640 1 -- 172.21.15.70:0/1945149579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 msgr2=0x7f96540b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.303+0000 7f9659e1a640 1 --2- 172.21.15.70:0/1945149579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 0x7f96540b0600 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f96540cb000 tx=0x7f964c02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.303+0000 7f9659e1a640 1 -- 172.21.15.70:0/1945149579 shutdown_connections 2023-10-08T05:57:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.303+0000 7f9659e1a640 1 --2- 172.21.15.70:0/1945149579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 0x7f96540b0600 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.303+0000 7f9659e1a640 1 --2- 172.21.15.70:0/1945149579 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540adc70 0x7f96540b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.303+0000 7f9659e1a640 1 -- 172.21.15.70:0/1945149579 >> 172.21.15.70:0/1945149579 conn(0x7f96540cb0c0 msgr2=0x7f96540cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:27.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.303+0000 7f9659e1a640 1 -- 172.21.15.70:0/1945149579 shutdown_connections 2023-10-08T05:57:27.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.303+0000 7f9659e1a640 1 -- 172.21.15.70:0/1945149579 wait complete. 2023-10-08T05:57:27.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.304+0000 7f9659e1a640 1 Processor -- start 2023-10-08T05:57:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.305+0000 7f9659e1a640 1 -- start start 2023-10-08T05:57:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659e1a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540adc70 0x7f96541583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659e1a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f965415ab30 0x7f965414c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659e1a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 0x7f965415c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659e1a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9654155950 con 0x7f965415af10 2023-10-08T05:57:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659e1a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9654158910 con 0x7f965415ab30 2023-10-08T05:57:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659e1a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96540c5940 con 0x7f96540adc70 2023-10-08T05:57:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659619640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 0x7f965415c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659619640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 0x7f965415c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52184/0 (socket says 172.21.15.70:52184) 2023-10-08T05:57:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659619640 1 -- 172.21.15.70:0/2246632312 learned_addr learned my addr 172.21.15.70:0/2246632312 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659619640 1 -- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540adc70 msgr2=0x7f96541583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659619640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540adc70 0x7f96541583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659619640 1 -- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f965415ab30 msgr2=0x7f965414c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659619640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f965415ab30 0x7f965414c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.306+0000 7f9659619640 1 -- 172.21.15.70:0/2246632312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c0094d0 con 0x7f965415af10 2023-10-08T05:57:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.307+0000 7f9659619640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 0x7f965415c6c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f964c0090b0 tx=0x7f964c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.307+0000 7f96517fa640 1 -- 172.21.15.70:0/2246632312 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c03aa60 con 0x7f965415af10 2023-10-08T05:57:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.307+0000 7f96517fa640 1 -- 172.21.15.70:0/2246632312 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c03abe0 con 0x7f965415af10 2023-10-08T05:57:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.307+0000 7f96517fa640 1 -- 172.21.15.70:0/2246632312 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0443f0 con 0x7f965415af10 2023-10-08T05:57:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.307+0000 7f9659e1a640 1 -- 172.21.15.70:0/2246632312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96541550b0 con 0x7f965415af10 2023-10-08T05:57:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.307+0000 7f9659e1a640 1 -- 172.21.15.70:0/2246632312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9654155670 con 0x7f965415af10 2023-10-08T05:57:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.308+0000 7f9659e1a640 1 -- 172.21.15.70:0/2246632312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96540c5710 con 0x7f965415af10 2023-10-08T05:57:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.309+0000 7f96517fa640 1 -- 172.21.15.70:0/2246632312 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f964c04d450 con 0x7f965415af10 2023-10-08T05:57:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.310+0000 7f96517fa640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f964006af90 0x7f964006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.310+0000 7f96517fa640 1 -- 172.21.15.70:0/2246632312 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f964c0abcf0 con 0x7f965415af10 2023-10-08T05:57:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.311+0000 7f9658e18640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f964006af90 0x7f964006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.312+0000 7f9658e18640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f964006af90 0x7f964006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96440043c0 tx=0x7f9644007d30 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.314+0000 7f96517fa640 1 -- 172.21.15.70:0/2246632312 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f964c076950 con 0x7f965415af10 2023-10-08T05:57:27.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.498+0000 7f9659e1a640 1 -- 172.21.15.70:0/2246632312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f965404e160 con 0x7f965415af10 2023-10-08T05:57:27.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.500+0000 7f96517fa640 1 -- 172.21.15.70:0/2246632312 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+16848 (secure 0 0 0) 0x7f964c079f70 con 0x7f965415af10 2023-10-08T05:57:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:27.530 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":162,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:57:26.073654+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:57:23.815675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-08T05:57:25.045292+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:57:27.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.503+0000 7f9659e1a640 1 -- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f964006af90 msgr2=0x7f964006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:27.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.503+0000 7f9659e1a640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f964006af90 0x7f964006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96440043c0 tx=0x7f9644007d30 comp rx=0 tx=0).stop 2023-10-08T05:57:27.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.503+0000 7f9659e1a640 1 -- 172.21.15.70:0/2246632312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 msgr2=0x7f965415c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:27.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.503+0000 7f9659e1a640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 0x7f965415c6c0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f964c0090b0 tx=0x7f964c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.503+0000 7f9659e1a640 1 -- 172.21.15.70:0/2246632312 shutdown_connections 2023-10-08T05:57:27.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.503+0000 7f9659e1a640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f964006af90 0x7f964006d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.503+0000 7f9659e1a640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f965415af10 0x7f965415c6c0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.503+0000 7f9659e1a640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f965415ab30 0x7f965414c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.503+0000 7f9659e1a640 1 --2- 172.21.15.70:0/2246632312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540adc70 0x7f96541583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.503+0000 7f9659e1a640 1 -- 172.21.15.70:0/2246632312 >> 172.21.15.70:0/2246632312 conn(0x7f96540cb0c0 msgr2=0x7f9654070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:27.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.504+0000 7f9659e1a640 1 -- 172.21.15.70:0/2246632312 shutdown_connections 2023-10-08T05:57:27.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.504+0000 7f9659e1a640 1 -- 172.21.15.70:0/2246632312 wait complete. 2023-10-08T05:57:27.550 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T05:57:27.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.768+0000 7fad92797640 1 Processor -- start 2023-10-08T05:57:27.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.769+0000 7fad92797640 1 -- start start 2023-10-08T05:57:27.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.770+0000 7fad92797640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad8c160660 0x7fad8c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.770+0000 7fad92797640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c0c9910 0x7fad8c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.770+0000 7fad92797640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fad8c0cb450 0x7fad8c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.770+0000 7fad92797640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c06e150 con 0x7fad8c160660 2023-10-08T05:57:27.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.770+0000 7fad92797640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c0cb8c0 con 0x7fad8c0cb450 2023-10-08T05:57:27.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.770+0000 7fad92797640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c06de70 con 0x7fad8c0c9910 2023-10-08T05:57:27.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.771+0000 7fad8bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad8c160660 0x7fad8c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:27.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.771+0000 7fad8bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad8c160660 0x7fad8c070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52194/0 (socket says 172.21.15.70:52194) 2023-10-08T05:57:27.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.771+0000 7fad8bfff640 1 -- 172.21.15.70:0/879741336 learned_addr learned my addr 172.21.15.70:0/879741336 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:27.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.771+0000 7fad837fe640 1 --2- 172.21.15.70:0/879741336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c0c9910 0x7fad8c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:27.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.771+0000 7fad90d34640 1 --2- 172.21.15.70:0/879741336 >> v2:172.21.15.181:3300/0 conn(0x7fad8c0cb450 0x7fad8c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:27.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.771+0000 7fad8bfff640 1 -- 172.21.15.70:0/879741336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c0c9910 msgr2=0x7fad8c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:27.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.771+0000 7fad8bfff640 1 --2- 172.21.15.70:0/879741336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c0c9910 0x7fad8c0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.771+0000 7fad8bfff640 1 -- 172.21.15.70:0/879741336 >> v2:172.21.15.181:3300/0 conn(0x7fad8c0cb450 msgr2=0x7fad8c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:27.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.771+0000 7fad8bfff640 1 --2- 172.21.15.70:0/879741336 >> v2:172.21.15.181:3300/0 conn(0x7fad8c0cb450 0x7fad8c0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.771+0000 7fad8bfff640 1 -- 172.21.15.70:0/879741336 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0c8e30 con 0x7fad8c160660 2023-10-08T05:57:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.772+0000 7fad8bfff640 1 --2- 172.21.15.70:0/879741336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c160660 0x7fad8c070cc0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fad8c070100 tx=0x7fad7c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46d0cd4d51a2f5e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.773+0000 7fad8b7fe640 1 -- 172.21.15.70:0/879741336 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c031870 con 0x7fad8c160660 2023-10-08T05:57:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.773+0000 7fad8b7fe640 1 -- 172.21.15.70:0/879741336 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c0319d0 con 0x7fad8c160660 2023-10-08T05:57:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.773+0000 7fad8b7fe640 1 -- 172.21.15.70:0/879741336 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c031d60 con 0x7fad8c160660 2023-10-08T05:57:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.773+0000 7fad92797640 1 -- 172.21.15.70:0/879741336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c160660 msgr2=0x7fad8c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.773+0000 7fad92797640 1 --2- 172.21.15.70:0/879741336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c160660 0x7fad8c070cc0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fad8c070100 tx=0x7fad7c02b910 comp rx=0 tx=0).stop 2023-10-08T05:57:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.774+0000 7fad92797640 1 -- 172.21.15.70:0/879741336 shutdown_connections 2023-10-08T05:57:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.774+0000 7fad92797640 1 --2- 172.21.15.70:0/879741336 >> v2:172.21.15.181:3300/0 conn(0x7fad8c0cb450 0x7fad8c0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.774+0000 7fad92797640 1 --2- 172.21.15.70:0/879741336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c0c9910 0x7fad8c0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.774+0000 7fad92797640 1 --2- 172.21.15.70:0/879741336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c160660 0x7fad8c070cc0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.774+0000 7fad92797640 1 -- 172.21.15.70:0/879741336 >> 172.21.15.70:0/879741336 conn(0x7fad8c06a5d0 msgr2=0x7fad8c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.774+0000 7fad92797640 1 -- 172.21.15.70:0/879741336 shutdown_connections 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.774+0000 7fad92797640 1 -- 172.21.15.70:0/879741336 wait complete. 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.775+0000 7fad92797640 1 Processor -- start 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.775+0000 7fad92797640 1 -- start start 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.776+0000 7fad92797640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c0c9910 0x7fad8c0ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.776+0000 7fad92797640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c0cb450 0x7fad8c0c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.776+0000 7fad92797640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c160660 0x7fad8c154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.776+0000 7fad92797640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0c2c20 con 0x7fad8c0cb450 2023-10-08T05:57:27.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.776+0000 7fad92797640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0c4fa0 con 0x7fad8c160660 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.776+0000 7fad92797640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c155410 con 0x7fad8c0c9910 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.777+0000 7fad837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c0cb450 0x7fad8c0c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.777+0000 7fad837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c0cb450 0x7fad8c0c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52208/0 (socket says 172.21.15.70:52208) 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.777+0000 7fad837fe640 1 -- 172.21.15.70:0/2368539291 learned_addr learned my addr 172.21.15.70:0/2368539291 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.777+0000 7fad8bfff640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c0c9910 0x7fad8c0ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.777+0000 7fad837fe640 1 -- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c0c9910 msgr2=0x7fad8c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.777+0000 7fad837fe640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c0c9910 0x7fad8c0ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.777+0000 7fad837fe640 1 -- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c160660 msgr2=0x7fad8c154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.777+0000 7fad837fe640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c160660 0x7fad8c154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.777+0000 7fad837fe640 1 -- 172.21.15.70:0/2368539291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c0094d0 con 0x7fad8c0cb450 2023-10-08T05:57:27.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.777+0000 7fad837fe640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c0cb450 0x7fad8c0c4a60 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fad7400d800 tx=0x7fad7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.778+0000 7fad897fa640 1 -- 172.21.15.70:0/2368539291 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740096a0 con 0x7fad8c0cb450 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.778+0000 7fad897fa640 1 -- 172.21.15.70:0/2368539291 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad74009800 con 0x7fad8c0cb450 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.778+0000 7fad897fa640 1 -- 172.21.15.70:0/2368539291 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74009990 con 0x7fad8c0cb450 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.778+0000 7fad92797640 1 -- 172.21.15.70:0/2368539291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c0c22f0 con 0x7fad8c0cb450 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.778+0000 7fad92797640 1 -- 172.21.15.70:0/2368539291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c0c28b0 con 0x7fad8c0cb450 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.779+0000 7fad92797640 1 -- 172.21.15.70:0/2368539291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c0c2dd0 con 0x7fad8c0cb450 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.780+0000 7fad897fa640 1 -- 172.21.15.70:0/2368539291 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fad74009af0 con 0x7fad8c0cb450 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.781+0000 7fad897fa640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fad6c06af90 0x7fad6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.781+0000 7fad897fa640 1 -- 172.21.15.70:0/2368539291 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad74089d60 con 0x7fad8c0cb450 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.782+0000 7fad8bfff640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fad6c06af90 0x7fad6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.783+0000 7fad8bfff640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fad6c06af90 0x7fad6c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad7c036000 tx=0x7fad7c03c040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:27.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:27.786+0000 7fad897fa640 1 -- 172.21.15.70:0/2368539291 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad8c0c2dd0 con 0x7fad8c0cb450 2023-10-08T05:57:28.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.016+0000 7fad92797640 1 -- 172.21.15.70:0/2368539291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fad8c04e160 con 0x7fad8c0cb450 2023-10-08T05:57:28.449 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-10-08T05:57:28.448+0000 7f45d049a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T05:57:28.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.571+0000 7fad897fa640 1 -- 172.21.15.70:0/2368539291 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) v1 ==== 118+0+0 (secure 0 0 0) 0x7fad74054c00 con 0x7fad8c0cb450 2023-10-08T05:57:28.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 -- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fad6c06af90 msgr2=0x7fad6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fad6c06af90 0x7fad6c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fad7c036000 tx=0x7fad7c03c040 comp rx=0 tx=0).stop 2023-10-08T05:57:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 -- 172.21.15.70:0/2368539291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c0cb450 msgr2=0x7fad8c0c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c0cb450 0x7fad8c0c4a60 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fad7400d800 tx=0x7fad7400dcc0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 -- 172.21.15.70:0/2368539291 shutdown_connections 2023-10-08T05:57:28.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fad6c06af90 0x7fad6c06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad8c160660 0x7fad8c154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad8c0cb450 0x7fad8c0c4a60 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 --2- 172.21.15.70:0/2368539291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad8c0c9910 0x7fad8c0ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 -- 172.21.15.70:0/2368539291 >> 172.21.15.70:0/2368539291 conn(0x7fad8c06a5d0 msgr2=0x7fad8c15f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 -- 172.21.15.70:0/2368539291 shutdown_connections 2023-10-08T05:57:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.575+0000 7fad92797640 1 -- 172.21.15.70:0/2368539291 wait complete. 2023-10-08T05:57:28.610 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T05:57:28.610 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T05:57:28.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.828+0000 7f1e12887640 1 Processor -- start 2023-10-08T05:57:28.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.828+0000 7f1e12887640 1 -- start start 2023-10-08T05:57:28.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.830+0000 7f1e12887640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e0c0afbe0 0x7f1e0c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:28.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.830+0000 7f1e12887640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e0c0cb130 0x7f1e0c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:28.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.830+0000 7f1e12887640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0cbad0 con 0x7f1e0c06dfc0 2023-10-08T05:57:28.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.830+0000 7f1e12887640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e0c0b2620 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.830+0000 7f1e12887640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0b2060 con 0x7f1e0c0cb130 2023-10-08T05:57:28.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.830+0000 7f1e0bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e0c0afbe0 0x7f1e0c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:28.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.830+0000 7f1e037fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1e0c06dfc0 0x7f1e0c070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50544/0 (socket says 172.21.15.70:50544) 2023-10-08T05:57:28.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.830+0000 7f1e037fe640 1 -- 172.21.15.70:0/3143673896 learned_addr learned my addr 172.21.15.70:0/3143673896 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:28.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.831+0000 7f1e10e24640 1 --2- 172.21.15.70:0/3143673896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e0c0cb130 0x7f1e0c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:28.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.831+0000 7f1e0bfff640 1 -- 172.21.15.70:0/3143673896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e0c0cb130 msgr2=0x7f1e0c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:28.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.831+0000 7f1e0bfff640 1 --2- 172.21.15.70:0/3143673896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e0c0cb130 0x7f1e0c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.831+0000 7f1e0bfff640 1 -- 172.21.15.70:0/3143673896 >> v1:172.21.15.70:6789/0 conn(0x7f1e0c06dfc0 legacy=0x7f1e0c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:28.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.831+0000 7f1e0bfff640 1 -- 172.21.15.70:0/3143673896 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c0b2340 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.831+0000 7f1e0b7fe640 1 -- 172.21.15.70:0/3143673896 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632026985 0 0) 0x7f1e0c0cbad0 con 0x7f1e0c06dfc0 2023-10-08T05:57:28.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.832+0000 7f1e0bfff640 1 --2- 172.21.15.70:0/3143673896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 0x7f1e0c0b15b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009d90 tx=0x7f1dfc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5373a5564d0a6ab4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:28.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.832+0000 7f1e0b7fe640 1 -- 172.21.15.70:0/3143673896 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc02cc60 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.832+0000 7f1e0b7fe640 1 -- 172.21.15.70:0/3143673896 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc02cdc0 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.832+0000 7f1e0b7fe640 1 -- 172.21.15.70:0/3143673896 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc034660 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.833+0000 7f1e12887640 1 -- 172.21.15.70:0/3143673896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 msgr2=0x7f1e0c0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:28.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.833+0000 7f1e12887640 1 --2- 172.21.15.70:0/3143673896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 0x7f1e0c0b15b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc009d90 tx=0x7f1dfc02beb0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.833+0000 7f1e12887640 1 -- 172.21.15.70:0/3143673896 shutdown_connections 2023-10-08T05:57:28.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.833+0000 7f1e12887640 1 --2- 172.21.15.70:0/3143673896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e0c0cb130 0x7f1e0c0cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.833+0000 7f1e12887640 1 --2- 172.21.15.70:0/3143673896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 0x7f1e0c0b15b0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.833+0000 7f1e12887640 1 -- 172.21.15.70:0/3143673896 >> 172.21.15.70:0/3143673896 conn(0x7f1e0c06a5d0 msgr2=0x7f1e0c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:28.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.833+0000 7f1e12887640 1 -- 172.21.15.70:0/3143673896 shutdown_connections 2023-10-08T05:57:28.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.833+0000 7f1e12887640 1 -- 172.21.15.70:0/3143673896 wait complete. 2023-10-08T05:57:28.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.835+0000 7f1e12887640 1 Processor -- start 2023-10-08T05:57:28.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.835+0000 7f1e12887640 1 -- start start 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e12887640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e0c06dfc0 0x7f1e0c1f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e12887640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 0x7f1e0c0ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e12887640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e0c0cb130 0x7f1e0c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e12887640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c1f2520 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e12887640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0cf210 con 0x7f1e0c0cb130 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e12887640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0cf4f0 con 0x7f1e0c06dfc0 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 0x7f1e0c0ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 0x7f1e0c0ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52232/0 (socket says 172.21.15.70:52232) 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e037fe640 1 -- 172.21.15.70:0/2138746636 learned_addr learned my addr 172.21.15.70:0/2138746636 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e037fe640 1 -- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e0c06dfc0 msgr2=0x7f1e0c1f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e037fe640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e0c06dfc0 0x7f1e0c1f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e037fe640 1 -- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e0c0cb130 msgr2=0x7f1e0c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:28.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.836+0000 7f1e037fe640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e0c0cb130 0x7f1e0c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:28.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.837+0000 7f1e037fe640 1 -- 172.21.15.70:0/2138746636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc009a70 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.837+0000 7f1e037fe640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 0x7f1e0c0ce750 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c410 tx=0x7f1df400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:28.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.837+0000 7f1e097fa640 1 -- 172.21.15.70:0/2138746636 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40074e0 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.837+0000 7f1e097fa640 1 -- 172.21.15.70:0/2138746636 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4007660 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.837+0000 7f1e097fa640 1 -- 172.21.15.70:0/2138746636 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df4004a20 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.837+0000 7f1e12887640 1 -- 172.21.15.70:0/2138746636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c146520 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.837+0000 7f1e12887640 1 -- 172.21.15.70:0/2138746636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c146ae0 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.838+0000 7f1e12887640 1 -- 172.21.15.70:0/2138746636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c146cd0 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.839+0000 7f1e097fa640 1 -- 172.21.15.70:0/2138746636 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1df4004b80 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.841+0000 7f1e097fa640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1dec06af90 0x7f1dec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:28.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.841+0000 7f1e097fa640 1 -- 172.21.15.70:0/2138746636 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1df40897c0 con 0x7f1e0c0afbe0 2023-10-08T05:57:28.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.841+0000 7f1e0bfff640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1dec06af90 0x7f1dec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:28.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.843+0000 7f1e0bfff640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1dec06af90 0x7f1dec06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc036000 tx=0x7f1dfc02e000 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:28.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:28.845+0000 7f1e097fa640 1 -- 172.21.15.70:0/2138746636 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1df4057fc0 con 0x7f1e0c0afbe0 2023-10-08T05:57:29.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.074+0000 7f1e12887640 1 -- 172.21.15.70:0/2138746636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1e0c04e160 con 0x7f1e0c0afbe0 2023-10-08T05:57:29.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.575+0000 7f1e097fa640 1 -- 172.21.15.70:0/2138746636 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1df4057b50 con 0x7f1e0c0afbe0 2023-10-08T05:57:29.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.579+0000 7f1e12887640 1 -- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1dec06af90 msgr2=0x7f1dec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.579+0000 7f1e12887640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1dec06af90 0x7f1dec06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc036000 tx=0x7f1dfc02e000 comp rx=0 tx=0).stop 2023-10-08T05:57:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.579+0000 7f1e12887640 1 -- 172.21.15.70:0/2138746636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 msgr2=0x7f1e0c0ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.579+0000 7f1e12887640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 0x7f1e0c0ce750 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f1df400c410 tx=0x7f1df400c8d0 comp rx=0 tx=0).stop 2023-10-08T05:57:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.579+0000 7f1e12887640 1 -- 172.21.15.70:0/2138746636 shutdown_connections 2023-10-08T05:57:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.579+0000 7f1e12887640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f1dec06af90 0x7f1dec06d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.579+0000 7f1e12887640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1e0c0cb130 0x7f1e0c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.579+0000 7f1e12887640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e0c0afbe0 0x7f1e0c0ce750 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.579+0000 7f1e12887640 1 --2- 172.21.15.70:0/2138746636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1e0c06dfc0 0x7f1e0c1f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.579+0000 7f1e12887640 1 -- 172.21.15.70:0/2138746636 >> 172.21.15.70:0/2138746636 conn(0x7f1e0c06a5d0 msgr2=0x7f1e0c0c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:29.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.580+0000 7f1e12887640 1 -- 172.21.15.70:0/2138746636 shutdown_connections 2023-10-08T05:57:29.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.580+0000 7f1e12887640 1 -- 172.21.15.70:0/2138746636 wait complete. 2023-10-08T05:57:29.607 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T05:57:29.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:57:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.860+0000 7f2fdc64f640 1 Processor -- start 2023-10-08T05:57:29.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.860+0000 7f2fdc64f640 1 -- start start 2023-10-08T05:57:29.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.862+0000 7f2fdc64f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fd415ab40 0x7f2fd40ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:29.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.862+0000 7f2fdc64f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd415b2b0 con 0x7f2fd40cb580 2023-10-08T05:57:29.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.862+0000 7f2fdc64f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd415b590 con 0x7f2fd40c90b0 2023-10-08T05:57:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.862+0000 7f2fdc64f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd415bf30 con 0x7f2fd415ab40 2023-10-08T05:57:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.862+0000 7f2fdabec640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2fd40cb580 0x7f2fd415e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50560/0 (socket says 172.21.15.70:50560) 2023-10-08T05:57:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.862+0000 7f2fdabec640 1 -- 172.21.15.70:0/3132611646 learned_addr learned my addr 172.21.15.70:0/3132611646 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.862+0000 7f2fda3eb640 1 --2- 172.21.15.70:0/3132611646 >> v2:172.21.15.70:3300/0 conn(0x7f2fd415ab40 0x7f2fd40ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.863+0000 7f2fd93e9640 1 -- 172.21.15.70:0/3132611646 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030924465 0 0) 0x7f2fd415b2b0 con 0x7f2fd40cb580 2023-10-08T05:57:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.863+0000 7f2fda3eb640 1 -- 172.21.15.70:0/3132611646 >> v1:172.21.15.181:6789/0 conn(0x7f2fd40c90b0 legacy=0x7f2fd40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.863+0000 7f2fda3eb640 1 -- 172.21.15.70:0/3132611646 >> v1:172.21.15.70:6789/0 conn(0x7f2fd40cb580 legacy=0x7f2fd415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.863+0000 7f2fda3eb640 1 -- 172.21.15.70:0/3132611646 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd415b870 con 0x7f2fd415ab40 2023-10-08T05:57:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.864+0000 7f2fda3eb640 1 --2- 172.21.15.70:0/3132611646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd415ab40 0x7f2fd40ad720 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00097a0 tx=0x7f2fd002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16797df6aa9468d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.865+0000 7f2fd93e9640 1 -- 172.21.15.70:0/3132611646 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0030610 con 0x7f2fd415ab40 2023-10-08T05:57:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.865+0000 7f2fd93e9640 1 -- 172.21.15.70:0/3132611646 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd0030770 con 0x7f2fd415ab40 2023-10-08T05:57:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.865+0000 7f2fd93e9640 1 -- 172.21.15.70:0/3132611646 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0030b00 con 0x7f2fd415ab40 2023-10-08T05:57:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.865+0000 7f2fdc64f640 1 -- 172.21.15.70:0/3132611646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd415ab40 msgr2=0x7f2fd40ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.865+0000 7f2fdc64f640 1 --2- 172.21.15.70:0/3132611646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd415ab40 0x7f2fd40ad720 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00097a0 tx=0x7f2fd002b960 comp rx=0 tx=0).stop 2023-10-08T05:57:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.865+0000 7f2fdc64f640 1 -- 172.21.15.70:0/3132611646 shutdown_connections 2023-10-08T05:57:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.865+0000 7f2fdc64f640 1 --2- 172.21.15.70:0/3132611646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd415ab40 0x7f2fd40ad720 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.865+0000 7f2fdc64f640 1 -- 172.21.15.70:0/3132611646 >> 172.21.15.70:0/3132611646 conn(0x7f2fd406a380 msgr2=0x7f2fd40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.865+0000 7f2fdc64f640 1 -- 172.21.15.70:0/3132611646 shutdown_connections 2023-10-08T05:57:29.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.865+0000 7f2fdc64f640 1 -- 172.21.15.70:0/3132611646 wait complete. 2023-10-08T05:57:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.867+0000 7f2fdc64f640 1 Processor -- start 2023-10-08T05:57:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.867+0000 7f2fdc64f640 1 -- start start 2023-10-08T05:57:29.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.868+0000 7f2fdc64f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd40c90b0 0x7f2fd40ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:29.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.868+0000 7f2fdc64f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd40cb580 0x7f2fd414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:29.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.868+0000 7f2fdc64f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd415ab40 0x7f2fd41481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.868+0000 7f2fdc64f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd40b3400 con 0x7f2fd40cb580 2023-10-08T05:57:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.868+0000 7f2fdc64f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd41486f0 con 0x7f2fd40c90b0 2023-10-08T05:57:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.868+0000 7f2fdc64f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4155d30 con 0x7f2fd415ab40 2023-10-08T05:57:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fd9bea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd40cb580 0x7f2fd414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fd9bea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd40cb580 0x7f2fd414a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52260/0 (socket says 172.21.15.70:52260) 2023-10-08T05:57:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fd9bea640 1 -- 172.21.15.70:0/1141795903 learned_addr learned my addr 172.21.15.70:0/1141795903 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fda3eb640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd40c90b0 0x7f2fd40ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fdabec640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd415ab40 0x7f2fd41481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fd9bea640 1 -- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd415ab40 msgr2=0x7f2fd41481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:29.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fd9bea640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd415ab40 0x7f2fd41481b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fd9bea640 1 -- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd40c90b0 msgr2=0x7f2fd40ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fd9bea640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd40c90b0 0x7f2fd40ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fd9bea640 1 -- 172.21.15.70:0/1141795903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd0009400 con 0x7f2fd40cb580 2023-10-08T05:57:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fda3eb640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd40c90b0 0x7f2fd40ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:29.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.869+0000 7f2fd9bea640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd40cb580 0x7f2fd414a9c0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c400 tx=0x7f2fc400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:29.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.870+0000 7f2fcaffd640 1 -- 172.21.15.70:0/1141795903 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40052d0 con 0x7f2fd40cb580 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.870+0000 7f2fcaffd640 1 -- 172.21.15.70:0/1141795903 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc4005430 con 0x7f2fd40cb580 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.870+0000 7f2fdc64f640 1 -- 172.21.15.70:0/1141795903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd414b920 con 0x7f2fd40cb580 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.870+0000 7f2fcaffd640 1 -- 172.21.15.70:0/1141795903 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4003830 con 0x7f2fd40cb580 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.870+0000 7f2fdc64f640 1 -- 172.21.15.70:0/1141795903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd414b360 con 0x7f2fd40cb580 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.871+0000 7f2fdc64f640 1 -- 172.21.15.70:0/1141795903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd4067490 con 0x7f2fd40cb580 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.872+0000 7f2fcaffd640 1 -- 172.21.15.70:0/1141795903 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2fc4003990 con 0x7f2fd40cb580 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.873+0000 7f2fcaffd640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f2fb806af60 0x7f2fb806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.873+0000 7f2fcaffd640 1 -- 172.21.15.70:0/1141795903 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2fc4088f60 con 0x7f2fd40cb580 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.874+0000 7f2fda3eb640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f2fb806af60 0x7f2fb806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.875+0000 7f2fda3eb640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f2fb806af60 0x7f2fb806d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0036000 tx=0x7f2fd003b040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:29.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:29.877+0000 7f2fcaffd640 1 -- 172.21.15.70:0/1141795903 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fc4053d60 con 0x7f2fd40cb580 2023-10-08T05:57:30.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.104+0000 7f2fdc64f640 1 -- 172.21.15.70:0/1141795903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fd40b0ef0 con 0x7f2fd40cb580 2023-10-08T05:57:30.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.105+0000 7f2fcaffd640 1 -- 172.21.15.70:0/1141795903 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f2fc4017090 con 0x7f2fd40cb580 2023-10-08T05:57:30.107 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:30.107 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2889908260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2889908260},{"type":"v1","addr":"172.21.15.181:6837","nonce":2889908260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2651702867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2651702867},{"type":"v1","addr":"172.21.15.181:6835","nonce":2651702867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":9048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1574948577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1574948577},{"type":"v1","addr":"172.21.15.70:6833","nonce":1574948577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:57:26.074871+0000","modified":"2023-10-08T05:57:29.076265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9051":{"gid":9051,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6835/3537072579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3537072579},{"type":"v1","addr":"172.21.15.70:6835","nonce":3537072579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-08T05:57:30.107 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 76 2023-10-08T05:57:30.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.107+0000 7f2fdc64f640 1 -- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f2fb806af60 msgr2=0x7f2fb806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.107+0000 7f2fdc64f640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f2fb806af60 0x7f2fb806d420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0036000 tx=0x7f2fd003b040 comp rx=0 tx=0).stop 2023-10-08T05:57:30.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.108+0000 7f2fdc64f640 1 -- 172.21.15.70:0/1141795903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd40cb580 msgr2=0x7f2fd414a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.108+0000 7f2fdc64f640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd40cb580 0x7f2fd414a9c0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400c400 tx=0x7f2fc400c8c0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.108+0000 7f2fdc64f640 1 -- 172.21.15.70:0/1141795903 shutdown_connections 2023-10-08T05:57:30.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.108+0000 7f2fdc64f640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f2fb806af60 0x7f2fb806d420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.108+0000 7f2fdc64f640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd415ab40 0x7f2fd41481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.108+0000 7f2fdc64f640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd40cb580 0x7f2fd414a9c0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.108+0000 7f2fdc64f640 1 --2- 172.21.15.70:0/1141795903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd40c90b0 0x7f2fd40ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.108+0000 7f2fdc64f640 1 -- 172.21.15.70:0/1141795903 >> 172.21.15.70:0/1141795903 conn(0x7f2fd406a380 msgr2=0x7f2fd40c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:30.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.108+0000 7f2fdc64f640 1 -- 172.21.15.70:0/1141795903 shutdown_connections 2023-10-08T05:57:30.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.108+0000 7f2fdc64f640 1 -- 172.21.15.70:0/1141795903 wait complete. 2023-10-08T05:57:30.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T05:57:30.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.343+0000 7f30d71b8640 1 Processor -- start 2023-10-08T05:57:30.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.343+0000 7f30d71b8640 1 -- start start 2023-10-08T05:57:30.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.344+0000 7f30d71b8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d015ab30 0x7f30d015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.344+0000 7f30d71b8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d015f7d0 con 0x7f30d00adc70 2023-10-08T05:57:30.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.344+0000 7f30d71b8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d015f210 con 0x7f30d00b07d0 2023-10-08T05:57:30.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.344+0000 7f30d71b8640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d015f4f0 con 0x7f30d015ab30 2023-10-08T05:57:30.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.345+0000 7f30d59b5640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f30d00adc70 0x7f30d00b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50568/0 (socket says 172.21.15.70:50568) 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.345+0000 7f30d59b5640 1 -- 172.21.15.70:0/3979829907 learned_addr learned my addr 172.21.15.70:0/3979829907 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.345+0000 7f30d61b6640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d015ab30 0x7f30d015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.345+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014519838 0 0) 0x7f30d015f7d0 con 0x7f30d00adc70 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.345+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30bc003520 con 0x7f30d00adc70 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.345+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3915984484 0 0) 0x7f30bc003520 con 0x7f30d00adc70 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.345+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d015ab30 msgr2=0x7f30d015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.345+0000 7f30d51b4640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d015ab30 0x7f30d015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 >> v1:172.21.15.181:6789/0 conn(0x7f30d00b07d0 legacy=0x7f30d006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d006c550 con 0x7f30d00adc70 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f30c0001670 con 0x7f30d00adc70 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 >> v1:172.21.15.70:6789/0 conn(0x7f30d00adc70 legacy=0x7f30d00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d51b4640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30bc0039a0 0x7f30bc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d51b4640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d015ab30 0x7f30bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d51b4640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc004d00 0x7f30bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d015f4f0 con 0x7f30bc0039a0 2023-10-08T05:57:30.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc00bb60 con 0x7f30bc004d00 2023-10-08T05:57:30.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc00bd20 con 0x7f30d015ab30 2023-10-08T05:57:30.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d61b6640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30bc0039a0 0x7f30bc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d59b5640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d015ab30 0x7f30bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d61b6640 1 -- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d015ab30 msgr2=0x7f30bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:30.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d61b6640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d015ab30 0x7f30bc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d69b7640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc004d00 0x7f30bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.346+0000 7f30d61b6640 1 -- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc004d00 msgr2=0x7f30bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:30.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.347+0000 7f30d61b6640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc004d00 0x7f30bc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.347+0000 7f30d61b6640 1 -- 172.21.15.70:0/3979829907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc003520 con 0x7f30bc0039a0 2023-10-08T05:57:30.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.347+0000 7f30d69b7640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc004d00 0x7f30bc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:30.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.348+0000 7f30d61b6640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30bc0039a0 0x7f30bc003dc0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00ba30 tx=0x7f30bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6597b77e80c27685 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:30.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.348+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc032640 con 0x7f30bc0039a0 2023-10-08T05:57:30.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.348+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc0327a0 con 0x7f30bc0039a0 2023-10-08T05:57:30.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.348+0000 7f30d51b4640 1 -- 172.21.15.70:0/3979829907 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc032b30 con 0x7f30bc0039a0 2023-10-08T05:57:30.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.348+0000 7f30d71b8640 1 -- 172.21.15.70:0/3979829907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30bc0039a0 msgr2=0x7f30bc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.348+0000 7f30d71b8640 1 --2- 172.21.15.70:0/3979829907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30bc0039a0 0x7f30bc003dc0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00ba30 tx=0x7f30bc0057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.348+0000 7f30d69b7640 1 -- 172.21.15.70:0/3979829907 reap_dead start 2023-10-08T05:57:30.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.349+0000 7f30d71b8640 1 -- 172.21.15.70:0/3979829907 shutdown_connections 2023-10-08T05:57:30.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.349+0000 7f30d71b8640 1 -- 172.21.15.70:0/3979829907 >> 172.21.15.70:0/3979829907 conn(0x7f30d00cb0c0 msgr2=0x7f30d00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:30.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.349+0000 7f30d71b8640 1 -- 172.21.15.70:0/3979829907 shutdown_connections 2023-10-08T05:57:30.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.349+0000 7f30d71b8640 1 -- 172.21.15.70:0/3979829907 wait complete. 2023-10-08T05:57:30.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.350+0000 7f30d71b8640 1 Processor -- start 2023-10-08T05:57:30.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.351+0000 7f30d71b8640 1 -- start start 2023-10-08T05:57:30.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d71b8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc0039a0 0x7f30d00b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d71b8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d01442f0 0x7f30d01f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d71b8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d0143640 0x7f30d0143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d71b8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00b2c40 con 0x7f30d0143640 2023-10-08T05:57:30.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d71b8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00b2030 con 0x7f30bc0039a0 2023-10-08T05:57:30.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d71b8640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00b2310 con 0x7f30d01442f0 2023-10-08T05:57:30.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d69b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d0143640 0x7f30d0143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d69b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d0143640 0x7f30d0143ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52270/0 (socket says 172.21.15.70:52270) 2023-10-08T05:57:30.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d69b7640 1 -- 172.21.15.70:0/2395134790 learned_addr learned my addr 172.21.15.70:0/2395134790 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:30.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d59b5640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d01442f0 0x7f30d01f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d61b6640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc0039a0 0x7f30d00b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d69b7640 1 -- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d01442f0 msgr2=0x7f30d01f25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d69b7640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d01442f0 0x7f30d01f25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d69b7640 1 -- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc0039a0 msgr2=0x7f30d00b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d69b7640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc0039a0 0x7f30d00b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d69b7640 1 -- 172.21.15.70:0/2395134790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d00b25f0 con 0x7f30d0143640 2023-10-08T05:57:30.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.352+0000 7f30d61b6640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc0039a0 0x7f30d00b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:30.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.353+0000 7f30d69b7640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d0143640 0x7f30d0143ac0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f30c8000f80 tx=0x7f30c800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:30.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.353+0000 7f30c6ffd640 1 -- 172.21.15.70:0/2395134790 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8006d50 con 0x7f30d0143640 2023-10-08T05:57:30.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.353+0000 7f30c6ffd640 1 -- 172.21.15.70:0/2395134790 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c8003970 con 0x7f30d0143640 2023-10-08T05:57:30.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.353+0000 7f30c6ffd640 1 -- 172.21.15.70:0/2395134790 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8003d20 con 0x7f30d0143640 2023-10-08T05:57:30.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.353+0000 7f30d71b8640 1 -- 172.21.15.70:0/2395134790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d00b28d0 con 0x7f30d0143640 2023-10-08T05:57:30.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.353+0000 7f30d71b8640 1 -- 172.21.15.70:0/2395134790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d01f2b00 con 0x7f30d0143640 2023-10-08T05:57:30.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.354+0000 7f30d71b8640 1 -- 172.21.15.70:0/2395134790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d00b2df0 con 0x7f30d0143640 2023-10-08T05:57:30.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.355+0000 7f30c6ffd640 1 -- 172.21.15.70:0/2395134790 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f30c8005070 con 0x7f30d0143640 2023-10-08T05:57:30.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.357+0000 7f30c6ffd640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f30b406afc0 0x7f30b406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.357+0000 7f30c6ffd640 1 -- 172.21.15.70:0/2395134790 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30c8089790 con 0x7f30d0143640 2023-10-08T05:57:30.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.357+0000 7f30d61b6640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f30b406afc0 0x7f30b406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.359+0000 7f30d61b6640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f30b406afc0 0x7f30b406d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00ba00 tx=0x7f30cc03d040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:30.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.361+0000 7f30c6ffd640 1 -- 172.21.15.70:0/2395134790 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30d00b2df0 con 0x7f30d0143640 2023-10-08T05:57:30.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.551+0000 7f30d71b8640 1 -- 172.21.15.70:0/2395134790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30d004e160 con 0x7f30d0143640 2023-10-08T05:57:30.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.552+0000 7f30c6ffd640 1 -- 172.21.15.70:0/2395134790 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+16835 (secure 0 0 0) 0x7f30c8054480 con 0x7f30d0143640 2023-10-08T05:57:30.577 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:30.577 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":163,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T05:57:27.546039+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:57:23.815675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-08T05:57:25.045292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T05:57:30.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 -- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f30b406afc0 msgr2=0x7f30b406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f30b406afc0 0x7f30b406d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00ba00 tx=0x7f30cc03d040 comp rx=0 tx=0).stop 2023-10-08T05:57:30.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 -- 172.21.15.70:0/2395134790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d0143640 msgr2=0x7f30d0143ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d0143640 0x7f30d0143ac0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f30c8000f80 tx=0x7f30c800dc00 comp rx=0 tx=0).stop 2023-10-08T05:57:30.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 -- 172.21.15.70:0/2395134790 shutdown_connections 2023-10-08T05:57:30.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f30b406afc0 0x7f30b406d480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d0143640 0x7f30d0143ac0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30d01442f0 0x7f30d01f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 --2- 172.21.15.70:0/2395134790 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30bc0039a0 0x7f30d00b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 -- 172.21.15.70:0/2395134790 >> 172.21.15.70:0/2395134790 conn(0x7f30d00cb0c0 msgr2=0x7f30d006cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:30.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 -- 172.21.15.70:0/2395134790 shutdown_connections 2023-10-08T05:57:30.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.555+0000 7f30d71b8640 1 -- 172.21.15.70:0/2395134790 wait complete. 2023-10-08T05:57:30.580 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T05:57:30.624 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:57:30.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.842+0000 7fc019f05640 1 Processor -- start 2023-10-08T05:57:30.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.842+0000 7fc019f05640 1 -- start start 2023-10-08T05:57:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.844+0000 7fc019f05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 0x7fc0140702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.844+0000 7fc019f05640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc01406c890 0x7fc01406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.844+0000 7fc019f05640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0140c48f0 con 0x7fc014160940 2023-10-08T05:57:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.844+0000 7fc019f05640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0140c4d80 con 0x7fc01406c890 2023-10-08T05:57:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.844+0000 7fc019f05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140c3ce0 con 0x7fc01415cd90 2023-10-08T05:57:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.845+0000 7fc013fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc01406c890 0x7fc01406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.845+0000 7fc00affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc014160940 0x7fc014070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50572/0 (socket says 172.21.15.70:50572) 2023-10-08T05:57:30.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.845+0000 7fc00affd640 1 -- 172.21.15.70:0/2180626655 learned_addr learned my addr 172.21.15.70:0/2180626655 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:30.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.845+0000 7fc013fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc01406c890 0x7fc01406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52278/0 (socket says 172.21.15.70:52278) 2023-10-08T05:57:30.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.845+0000 7fc0137fe640 1 --2- 172.21.15.70:0/2180626655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 0x7fc0140702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.845+0000 7fc013fff640 1 -- 172.21.15.70:0/2180626655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 msgr2=0x7fc0140702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:30.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.845+0000 7fc013fff640 1 --2- 172.21.15.70:0/2180626655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 0x7fc0140702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.845+0000 7fc013fff640 1 -- 172.21.15.70:0/2180626655 >> v1:172.21.15.70:6789/0 conn(0x7fc014160940 legacy=0x7fc014070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.845+0000 7fc013fff640 1 -- 172.21.15.70:0/2180626655 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0140c3fc0 con 0x7fc01406c890 2023-10-08T05:57:30.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.845+0000 7fc012ffd640 1 -- 172.21.15.70:0/2180626655 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177845606 0 0) 0x7fc0140c48f0 con 0x7fc014160940 2023-10-08T05:57:30.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.846+0000 7fc013fff640 1 --2- 172.21.15.70:0/2180626655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 0x7fc01406cd20 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009d90 tx=0x7fc00c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51fa9040ea6ae22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:30.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.847+0000 7fc012ffd640 1 -- 172.21.15.70:0/2180626655 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c02cc50 con 0x7fc01406c890 2023-10-08T05:57:30.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.847+0000 7fc012ffd640 1 -- 172.21.15.70:0/2180626655 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c02cdb0 con 0x7fc01406c890 2023-10-08T05:57:30.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.847+0000 7fc012ffd640 1 -- 172.21.15.70:0/2180626655 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c034660 con 0x7fc01406c890 2023-10-08T05:57:30.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.847+0000 7fc019f05640 1 -- 172.21.15.70:0/2180626655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 msgr2=0x7fc01406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.847+0000 7fc019f05640 1 --2- 172.21.15.70:0/2180626655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 0x7fc01406cd20 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009d90 tx=0x7fc00c02bea0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.847+0000 7fc019f05640 1 -- 172.21.15.70:0/2180626655 shutdown_connections 2023-10-08T05:57:30.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.847+0000 7fc019f05640 1 --2- 172.21.15.70:0/2180626655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 0x7fc01406cd20 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.847+0000 7fc019f05640 1 --2- 172.21.15.70:0/2180626655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 0x7fc0140702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.847+0000 7fc019f05640 1 -- 172.21.15.70:0/2180626655 >> 172.21.15.70:0/2180626655 conn(0x7fc01406a380 msgr2=0x7fc0140b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.847+0000 7fc019f05640 1 -- 172.21.15.70:0/2180626655 shutdown_connections 2023-10-08T05:57:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.848+0000 7fc019f05640 1 -- 172.21.15.70:0/2180626655 wait complete. 2023-10-08T05:57:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.849+0000 7fc019f05640 1 Processor -- start 2023-10-08T05:57:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.849+0000 7fc019f05640 1 -- start start 2023-10-08T05:57:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.850+0000 7fc019f05640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 0x7fc0140b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.850+0000 7fc019f05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 0x7fc01415e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.850+0000 7fc019f05640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc014160940 0x7fc014155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.850+0000 7fc019f05640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01415fae0 con 0x7fc01406c890 2023-10-08T05:57:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.850+0000 7fc019f05640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140b4400 con 0x7fc014160940 2023-10-08T05:57:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.850+0000 7fc019f05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01415ebc0 con 0x7fc01415cd90 2023-10-08T05:57:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 0x7fc0140b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc0137fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 0x7fc0140b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52282/0 (socket says 172.21.15.70:52282) 2023-10-08T05:57:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc0137fe640 1 -- 172.21.15.70:0/1417077334 learned_addr learned my addr 172.21.15.70:0/1417077334 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc00affd640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 0x7fc01415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc013fff640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc014160940 0x7fc014155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc0137fe640 1 -- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 msgr2=0x7fc01415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc0137fe640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 0x7fc01415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc0137fe640 1 -- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc014160940 msgr2=0x7fc014155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:30.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc0137fe640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc014160940 0x7fc014155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:30.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc0137fe640 1 -- 172.21.15.70:0/1417077334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c009a70 con 0x7fc01406c890 2023-10-08T05:57:30.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc00affd640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 0x7fc01415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:30.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc013fff640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc014160940 0x7fc014155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:30.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.851+0000 7fc0137fe640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 0x7fc0140b26e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fc00400d800 tx=0x7fc00400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:30.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.852+0000 7fc010ff9640 1 -- 172.21.15.70:0/1417077334 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00400b660 con 0x7fc01406c890 2023-10-08T05:57:30.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.852+0000 7fc010ff9640 1 -- 172.21.15.70:0/1417077334 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00400b7c0 con 0x7fc01406c890 2023-10-08T05:57:30.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.852+0000 7fc019f05640 1 -- 172.21.15.70:0/1417077334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01415ee60 con 0x7fc01406c890 2023-10-08T05:57:30.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.852+0000 7fc010ff9640 1 -- 172.21.15.70:0/1417077334 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00400bb50 con 0x7fc01406c890 2023-10-08T05:57:30.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.852+0000 7fc019f05640 1 -- 172.21.15.70:0/1417077334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01415f420 con 0x7fc01406c890 2023-10-08T05:57:30.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.853+0000 7fc019f05640 1 -- 172.21.15.70:0/1417077334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc014067490 con 0x7fc01406c890 2023-10-08T05:57:30.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.854+0000 7fc010ff9640 1 -- 172.21.15.70:0/1417077334 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc00400bcb0 con 0x7fc01406c890 2023-10-08T05:57:30.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.855+0000 7fc010ff9640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbff806afc0 0x7fbff806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:30.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.855+0000 7fc010ff9640 1 -- 172.21.15.70:0/1417077334 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc004089e90 con 0x7fc01406c890 2023-10-08T05:57:30.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.856+0000 7fc00affd640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbff806afc0 0x7fbff806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:30.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.857+0000 7fc00affd640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbff806afc0 0x7fbff806d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0140c89c0 tx=0x7fbffc009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:30.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:30.859+0000 7fc010ff9640 1 -- 172.21.15.70:0/1417077334 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc004054c00 con 0x7fc01406c890 2023-10-08T05:57:31.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.042+0000 7fc019f05640 1 -- 172.21.15.70:0/1417077334 --> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0140c4580 con 0x7fbff806afc0 2023-10-08T05:57:31.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.071+0000 7fc010ff9640 1 -- 172.21.15.70:0/1417077334 <== mgr.6299 v2:172.21.15.181:6832/2068783299 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281797 (secure 0 0 0) 0x7fbffc001c70 con 0x7fbff806afc0 2023-10-08T05:57:31.089 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:31.090 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:57:31.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.078+0000 7fc019f05640 1 -- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbff806afc0 msgr2=0x7fbff806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:31.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.078+0000 7fc019f05640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbff806afc0 0x7fbff806d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0140c89c0 tx=0x7fbffc009040 comp rx=0 tx=0).stop 2023-10-08T05:57:31.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.078+0000 7fc019f05640 1 -- 172.21.15.70:0/1417077334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 msgr2=0x7fc0140b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:31.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.078+0000 7fc019f05640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 0x7fc0140b26e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fc00400d800 tx=0x7fc00400dcc0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.078+0000 7fc019f05640 1 -- 172.21.15.70:0/1417077334 shutdown_connections 2023-10-08T05:57:31.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.078+0000 7fc019f05640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fbff806afc0 0x7fbff806d480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.078+0000 7fc019f05640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc014160940 0x7fc014155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.078+0000 7fc019f05640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc01415cd90 0x7fc01415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.078+0000 7fc019f05640 1 --2- 172.21.15.70:0/1417077334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc01406c890 0x7fc0140b26e0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.078+0000 7fc019f05640 1 -- 172.21.15.70:0/1417077334 >> 172.21.15.70:0/1417077334 conn(0x7fc01406a380 msgr2=0x7fc014158670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:31.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.079+0000 7fc019f05640 1 -- 172.21.15.70:0/1417077334 shutdown_connections 2023-10-08T05:57:31.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.079+0000 7fc019f05640 1 -- 172.21.15.70:0/1417077334 wait complete. 2023-10-08T05:57:31.101 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-08T05:57:29.603683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1341,"num_read_kb":2764,"num_write":409,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2707,"ondisk_log_size":2707,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":361936,"kb_used_data":16744,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754612784,"statfs":{"total":773094113280,"available":772723490816,"internally_reserved":0,"allocated":17145856,"data_stored":9322570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":353357371},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.104736"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.531064+0000","last_change":"2023-10-08T05:57:25.531064+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.531064+0000","last_undegraded":"2023-10-08T05:57:25.531064+0000","last_fullsized":"2023-10-08T05:57:25.531064+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.529776+0000","last_change":"2023-10-08T05:57:25.529776+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.529776+0000","last_undegraded":"2023-10-08T05:57:25.529776+0000","last_fullsized":"2023-10-08T05:57:25.529776+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.530944+0000","last_change":"2023-10-08T05:57:25.530944+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.530944+0000","last_undegraded":"2023-10-08T05:57:25.530944+0000","last_fullsized":"2023-10-08T05:57:25.530944+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.530898+0000","last_change":"2023-10-08T05:57:25.530898+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.530898+0000","last_undegraded":"2023-10-08T05:57:25.530898+0000","last_fullsized":"2023-10-08T05:57:25.530898+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.532699+0000","last_change":"2023-10-08T05:57:25.532699+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.532699+0000","last_undegraded":"2023-10-08T05:57:25.532699+0000","last_fullsized":"2023-10-08T05:57:25.532699+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.539803+0000","last_change":"2023-10-08T05:57:25.539803+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.539803+0000","last_undegraded":"2023-10-08T05:57:25.539803+0000","last_fullsized":"2023-10-08T05:57:25.539803+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.538239+0000","last_change":"2023-10-08T05:57:25.538239+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.538239+0000","last_undegraded":"2023-10-08T05:57:25.538239+0000","last_fullsized":"2023-10-08T05:57:25.538239+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.537877+0000","last_change":"2023-10-08T05:57:25.537877+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.537877+0000","last_undegraded":"2023-10-08T05:57:25.537877+0000","last_fullsized":"2023-10-08T05:57:25.537877+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.539229+0000","last_change":"2023-10-08T05:57:25.539229+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.539229+0000","last_undegraded":"2023-10-08T05:57:25.539229+0000","last_fullsized":"2023-10-08T05:57:25.539229+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.535441+0000","last_change":"2023-10-08T05:57:25.535441+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.535441+0000","last_undegraded":"2023-10-08T05:57:25.535441+0000","last_fullsized":"2023-10-08T05:57:25.535441+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.528014+0000","last_change":"2023-10-08T05:57:25.528014+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.528014+0000","last_undegraded":"2023-10-08T05:57:25.528014+0000","last_fullsized":"2023-10-08T05:57:25.528014+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.530235+0000","last_change":"2023-10-08T05:57:25.530235+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.530235+0000","last_undegraded":"2023-10-08T05:57:25.530235+0000","last_fullsized":"2023-10-08T05:57:25.530235+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.537587+0000","last_change":"2023-10-08T05:57:25.537587+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.537587+0000","last_undegraded":"2023-10-08T05:57:25.537587+0000","last_fullsized":"2023-10-08T05:57:25.537587+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.530060+0000","last_change":"2023-10-08T05:57:25.530060+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.530060+0000","last_undegraded":"2023-10-08T05:57:25.530060+0000","last_fullsized":"2023-10-08T05:57:25.530060+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.530297+0000","last_change":"2023-10-08T05:57:25.530297+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.530297+0000","last_undegraded":"2023-10-08T05:57:25.530297+0000","last_fullsized":"2023-10-08T05:57:25.530297+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.542500+0000","last_change":"2023-10-08T05:57:25.542500+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.542500+0000","last_undegraded":"2023-10-08T05:57:25.542500+0000","last_fullsized":"2023-10-08T05:57:25.542500+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.538704+0000","last_change":"2023-10-08T05:57:25.538704+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.538704+0000","last_undegraded":"2023-10-08T05:57:25.538704+0000","last_fullsized":"2023-10-08T05:57:25.538704+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.532330+0000","last_change":"2023-10-08T05:57:25.532330+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.532330+0000","last_undegraded":"2023-10-08T05:57:25.532330+0000","last_fullsized":"2023-10-08T05:57:25.532330+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.543385+0000","last_change":"2023-10-08T05:57:25.543385+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.543385+0000","last_undegraded":"2023-10-08T05:57:25.543385+0000","last_fullsized":"2023-10-08T05:57:25.543385+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.531728+0000","last_change":"2023-10-08T05:57:25.531728+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.531728+0000","last_undegraded":"2023-10-08T05:57:25.531728+0000","last_fullsized":"2023-10-08T05:57:25.531728+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.528429+0000","last_change":"2023-10-08T05:57:25.528429+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.528429+0000","last_undegraded":"2023-10-08T05:57:25.528429+0000","last_fullsized":"2023-10-08T05:57:25.528429+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.528790+0000","last_change":"2023-10-08T05:57:25.528790+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.528790+0000","last_undegraded":"2023-10-08T05:57:25.528790+0000","last_fullsized":"2023-10-08T05:57:25.528790+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.526477+0000","last_change":"2023-10-08T05:57:25.526477+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.526477+0000","last_undegraded":"2023-10-08T05:57:25.526477+0000","last_fullsized":"2023-10-08T05:57:25.526477+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.542101+0000","last_change":"2023-10-08T05:57:25.542101+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.542101+0000","last_undegraded":"2023-10-08T05:57:25.542101+0000","last_fullsized":"2023-10-08T05:57:25.542101+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.533708+0000","last_change":"2023-10-08T05:57:25.533708+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.533708+0000","last_undegraded":"2023-10-08T05:57:25.533708+0000","last_fullsized":"2023-10-08T05:57:25.533708+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.532358+0000","last_change":"2023-10-08T05:57:25.532358+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.532358+0000","last_undegraded":"2023-10-08T05:57:25.532358+0000","last_fullsized":"2023-10-08T05:57:25.532358+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.540540+0000","last_change":"2023-10-08T05:57:25.540540+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.540540+0000","last_undegraded":"2023-10-08T05:57:25.540540+0000","last_fullsized":"2023-10-08T05:57:25.540540+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.536084+0000","last_change":"2023-10-08T05:57:25.536084+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.536084+0000","last_undegraded":"2023-10-08T05:57:25.536084+0000","last_fullsized":"2023-10-08T05:57:25.536084+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.532360+0000","last_change":"2023-10-08T05:57:25.532360+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.532360+0000","last_undegraded":"2023-10-08T05:57:25.532360+0000","last_fullsized":"2023-10-08T05:57:25.532360+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.531898+0000","last_change":"2023-10-08T05:57:25.531898+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.531898+0000","last_undegraded":"2023-10-08T05:57:25.531898+0000","last_fullsized":"2023-10-08T05:57:25.531898+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.534712+0000","last_change":"2023-10-08T05:57:25.534712+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.534712+0000","last_undegraded":"2023-10-08T05:57:25.534712+0000","last_fullsized":"2023-10-08T05:57:25.534712+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.534051+0000","last_change":"2023-10-08T05:57:25.534051+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.534051+0000","last_undegraded":"2023-10-08T05:57:25.534051+0000","last_fullsized":"2023-10-08T05:57:25.534051+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.535379+0000","last_change":"2023-10-08T05:57:25.535379+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.535379+0000","last_undegraded":"2023-10-08T05:57:25.535379+0000","last_fullsized":"2023-10-08T05:57:25.535379+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.554457+0000","last_change":"2023-10-08T05:57:25.552962+0000","last_active":"2023-10-08T05:57:25.554457+0000","last_peered":"2023-10-08T05:57:25.554457+0000","last_clean":"2023-10-08T05:57:25.554457+0000","last_became_active":"2023-10-08T05:57:25.552677+0000","last_became_peered":"2023-10-08T05:57:25.552677+0000","last_unstale":"2023-10-08T05:57:25.554457+0000","last_undegraded":"2023-10-08T05:57:25.554457+0000","last_fullsized":"2023-10-08T05:57:25.554457+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:25.773932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.528147+0000","last_change":"2023-10-08T05:57:25.528147+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.528147+0000","last_undegraded":"2023-10-08T05:57:25.528147+0000","last_fullsized":"2023-10-08T05:57:25.528147+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.545449+0000","last_change":"2023-10-08T05:57:25.540708+0000","last_active":"2023-10-08T05:57:25.545449+0000","last_peered":"2023-10-08T05:57:25.545449+0000","last_clean":"2023-10-08T05:57:25.545449+0000","last_became_active":"2023-10-08T05:57:25.540301+0000","last_became_peered":"2023-10-08T05:57:25.540301+0000","last_unstale":"2023-10-08T05:57:25.545449+0000","last_undegraded":"2023-10-08T05:57:25.545449+0000","last_fullsized":"2023-10-08T05:57:25.545449+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:20.958125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.527461+0000","last_change":"2023-10-08T05:57:25.527461+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.527461+0000","last_undegraded":"2023-10-08T05:57:25.527461+0000","last_fullsized":"2023-10-08T05:57:25.527461+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.557870+0000","last_change":"2023-10-08T05:57:25.557364+0000","last_active":"2023-10-08T05:57:25.557870+0000","last_peered":"2023-10-08T05:57:25.557870+0000","last_clean":"2023-10-08T05:57:25.557870+0000","last_became_active":"2023-10-08T05:57:25.555480+0000","last_became_peered":"2023-10-08T05:57:25.555480+0000","last_unstale":"2023-10-08T05:57:25.557870+0000","last_undegraded":"2023-10-08T05:57:25.557870+0000","last_fullsized":"2023-10-08T05:57:25.557870+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:46.551576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.529488+0000","last_change":"2023-10-08T05:57:25.529488+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.529488+0000","last_undegraded":"2023-10-08T05:57:25.529488+0000","last_fullsized":"2023-10-08T05:57:25.529488+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.550514+0000","last_change":"2023-10-08T05:57:25.549520+0000","last_active":"2023-10-08T05:57:25.550514+0000","last_peered":"2023-10-08T05:57:25.550514+0000","last_clean":"2023-10-08T05:57:25.550514+0000","last_became_active":"2023-10-08T05:57:25.548746+0000","last_became_peered":"2023-10-08T05:57:25.548746+0000","last_unstale":"2023-10-08T05:57:25.550514+0000","last_undegraded":"2023-10-08T05:57:25.550514+0000","last_fullsized":"2023-10-08T05:57:25.550514+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:06.248303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.535043+0000","last_change":"2023-10-08T05:57:25.535043+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.535043+0000","last_undegraded":"2023-10-08T05:57:25.535043+0000","last_fullsized":"2023-10-08T05:57:25.535043+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.551552+0000","last_change":"2023-10-08T05:57:25.547635+0000","last_active":"2023-10-08T05:57:25.551552+0000","last_peered":"2023-10-08T05:57:25.551552+0000","last_clean":"2023-10-08T05:57:25.551552+0000","last_became_active":"2023-10-08T05:57:25.547185+0000","last_became_peered":"2023-10-08T05:57:25.547185+0000","last_unstale":"2023-10-08T05:57:25.551552+0000","last_undegraded":"2023-10-08T05:57:25.551552+0000","last_fullsized":"2023-10-08T05:57:25.551552+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:11:49.496158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.533390+0000","last_change":"2023-10-08T05:57:25.533390+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.533390+0000","last_undegraded":"2023-10-08T05:57:25.533390+0000","last_fullsized":"2023-10-08T05:57:25.533390+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.552879+0000","last_change":"2023-10-08T05:57:25.547003+0000","last_active":"2023-10-08T05:57:25.552879+0000","last_peered":"2023-10-08T05:57:25.552879+0000","last_clean":"2023-10-08T05:57:25.552879+0000","last_became_active":"2023-10-08T05:57:25.546756+0000","last_became_peered":"2023-10-08T05:57:25.546756+0000","last_unstale":"2023-10-08T05:57:25.552879+0000","last_undegraded":"2023-10-08T05:57:25.552879+0000","last_fullsized":"2023-10-08T05:57:25.552879+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:16.163657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.531749+0000","last_change":"2023-10-08T05:57:25.531749+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.531749+0000","last_undegraded":"2023-10-08T05:57:25.531749+0000","last_fullsized":"2023-10-08T05:57:25.531749+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.556070+0000","last_change":"2023-10-08T05:57:25.554379+0000","last_active":"2023-10-08T05:57:25.556070+0000","last_peered":"2023-10-08T05:57:25.556070+0000","last_clean":"2023-10-08T05:57:25.556070+0000","last_became_active":"2023-10-08T05:57:25.554018+0000","last_became_peered":"2023-10-08T05:57:25.554018+0000","last_unstale":"2023-10-08T05:57:25.556070+0000","last_undegraded":"2023-10-08T05:57:25.556070+0000","last_fullsized":"2023-10-08T05:57:25.556070+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:46.136294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.542762+0000","last_change":"2023-10-08T05:57:25.542762+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.542762+0000","last_undegraded":"2023-10-08T05:57:25.542762+0000","last_fullsized":"2023-10-08T05:57:25.542762+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.555321+0000","last_change":"2023-10-08T05:57:25.553851+0000","last_active":"2023-10-08T05:57:25.555321+0000","last_peered":"2023-10-08T05:57:25.555321+0000","last_clean":"2023-10-08T05:57:25.555321+0000","last_became_active":"2023-10-08T05:57:25.553482+0000","last_became_peered":"2023-10-08T05:57:25.553482+0000","last_unstale":"2023-10-08T05:57:25.555321+0000","last_undegraded":"2023-10-08T05:57:25.555321+0000","last_fullsized":"2023-10-08T05:57:25.555321+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:54.670330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.528014+0000","last_change":"2023-10-08T05:57:25.528014+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.528014+0000","last_undegraded":"2023-10-08T05:57:25.528014+0000","last_fullsized":"2023-10-08T05:57:25.528014+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.551477+0000","last_change":"2023-10-08T05:57:25.549805+0000","last_active":"2023-10-08T05:57:25.551477+0000","last_peered":"2023-10-08T05:57:25.551477+0000","last_clean":"2023-10-08T05:57:25.551477+0000","last_became_active":"2023-10-08T05:57:25.549525+0000","last_became_peered":"2023-10-08T05:57:25.549525+0000","last_unstale":"2023-10-08T05:57:25.551477+0000","last_undegraded":"2023-10-08T05:57:25.551477+0000","last_fullsized":"2023-10-08T05:57:25.551477+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:20.958125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.541327+0000","last_change":"2023-10-08T05:57:25.541327+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.541327+0000","last_undegraded":"2023-10-08T05:57:25.541327+0000","last_fullsized":"2023-10-08T05:57:25.541327+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.548490+0000","last_change":"2023-10-08T05:57:25.544343+0000","last_active":"2023-10-08T05:57:25.548490+0000","last_peered":"2023-10-08T05:57:25.548490+0000","last_clean":"2023-10-08T05:57:25.548490+0000","last_became_active":"2023-10-08T05:57:25.544065+0000","last_became_peered":"2023-10-08T05:57:25.544065+0000","last_unstale":"2023-10-08T05:57:25.548490+0000","last_undegraded":"2023-10-08T05:57:25.548490+0000","last_fullsized":"2023-10-08T05:57:25.548490+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:22:05.696781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.530928+0000","last_change":"2023-10-08T05:57:25.530928+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.530928+0000","last_undegraded":"2023-10-08T05:57:25.530928+0000","last_fullsized":"2023-10-08T05:57:25.530928+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.557239+0000","last_change":"2023-10-08T05:57:25.554217+0000","last_active":"2023-10-08T05:57:25.557239+0000","last_peered":"2023-10-08T05:57:25.557239+0000","last_clean":"2023-10-08T05:57:25.557239+0000","last_became_active":"2023-10-08T05:57:25.553572+0000","last_became_peered":"2023-10-08T05:57:25.553572+0000","last_unstale":"2023-10-08T05:57:25.557239+0000","last_undegraded":"2023-10-08T05:57:25.557239+0000","last_fullsized":"2023-10-08T05:57:25.557239+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:16.163657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.531143+0000","last_change":"2023-10-08T05:57:25.531143+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.531143+0000","last_undegraded":"2023-10-08T05:57:25.531143+0000","last_fullsized":"2023-10-08T05:57:25.531143+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.550491+0000","last_change":"2023-10-08T05:57:25.549177+0000","last_active":"2023-10-08T05:57:25.550491+0000","last_peered":"2023-10-08T05:57:25.550491+0000","last_clean":"2023-10-08T05:57:25.550491+0000","last_became_active":"2023-10-08T05:57:25.548827+0000","last_became_peered":"2023-10-08T05:57:25.548827+0000","last_unstale":"2023-10-08T05:57:25.550491+0000","last_undegraded":"2023-10-08T05:57:25.550491+0000","last_fullsized":"2023-10-08T05:57:25.550491+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:43.468225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.533555+0000","last_change":"2023-10-08T05:57:25.533555+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.533555+0000","last_undegraded":"2023-10-08T05:57:25.533555+0000","last_fullsized":"2023-10-08T05:57:25.533555+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.547302+0000","last_change":"2023-10-08T05:57:25.539867+0000","last_active":"2023-10-08T05:57:25.547302+0000","last_peered":"2023-10-08T05:57:25.547302+0000","last_clean":"2023-10-08T05:57:25.547302+0000","last_became_active":"2023-10-08T05:57:25.539587+0000","last_became_peered":"2023-10-08T05:57:25.539587+0000","last_unstale":"2023-10-08T05:57:25.547302+0000","last_undegraded":"2023-10-08T05:57:25.547302+0000","last_fullsized":"2023-10-08T05:57:25.547302+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:53:57.764136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.540021+0000","last_change":"2023-10-08T05:57:25.540021+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.540021+0000","last_undegraded":"2023-10-08T05:57:25.540021+0000","last_fullsized":"2023-10-08T05:57:25.540021+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.553659+0000","last_change":"2023-10-08T05:57:25.550848+0000","last_active":"2023-10-08T05:57:25.553659+0000","last_peered":"2023-10-08T05:57:25.553659+0000","last_clean":"2023-10-08T05:57:25.553659+0000","last_became_active":"2023-10-08T05:57:25.550450+0000","last_became_peered":"2023-10-08T05:57:25.550450+0000","last_unstale":"2023-10-08T05:57:25.553659+0000","last_undegraded":"2023-10-08T05:57:25.553659+0000","last_fullsized":"2023-10-08T05:57:25.553659+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:14:28.052312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.529743+0000","last_change":"2023-10-08T05:57:25.529743+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.529743+0000","last_undegraded":"2023-10-08T05:57:25.529743+0000","last_fullsized":"2023-10-08T05:57:25.529743+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.556623+0000","last_change":"2023-10-08T05:57:25.555732+0000","last_active":"2023-10-08T05:57:25.556623+0000","last_peered":"2023-10-08T05:57:25.556623+0000","last_clean":"2023-10-08T05:57:25.556623+0000","last_became_active":"2023-10-08T05:57:25.555420+0000","last_became_peered":"2023-10-08T05:57:25.555420+0000","last_unstale":"2023-10-08T05:57:25.556623+0000","last_undegraded":"2023-10-08T05:57:25.556623+0000","last_fullsized":"2023-10-08T05:57:25.556623+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:04:23.250387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.530940+0000","last_change":"2023-10-08T05:57:25.530940+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.530940+0000","last_undegraded":"2023-10-08T05:57:25.530940+0000","last_fullsized":"2023-10-08T05:57:25.530940+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.551460+0000","last_change":"2023-10-08T05:57:25.548150+0000","last_active":"2023-10-08T05:57:25.551460+0000","last_peered":"2023-10-08T05:57:25.551460+0000","last_clean":"2023-10-08T05:57:25.551460+0000","last_became_active":"2023-10-08T05:57:25.547818+0000","last_became_peered":"2023-10-08T05:57:25.547818+0000","last_unstale":"2023-10-08T05:57:25.551460+0000","last_undegraded":"2023-10-08T05:57:25.551460+0000","last_fullsized":"2023-10-08T05:57:25.551460+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:41:16.163657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.537428+0000","last_change":"2023-10-08T05:57:25.537428+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.537428+0000","last_undegraded":"2023-10-08T05:57:25.537428+0000","last_fullsized":"2023-10-08T05:57:25.537428+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.552608+0000","last_change":"2023-10-08T05:57:25.550760+0000","last_active":"2023-10-08T05:57:25.552608+0000","last_peered":"2023-10-08T05:57:25.552608+0000","last_clean":"2023-10-08T05:57:25.552608+0000","last_became_active":"2023-10-08T05:57:25.550423+0000","last_became_peered":"2023-10-08T05:57:25.550423+0000","last_unstale":"2023-10-08T05:57:25.552608+0000","last_undegraded":"2023-10-08T05:57:25.552608+0000","last_fullsized":"2023-10-08T05:57:25.552608+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:02.466682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.538133+0000","last_change":"2023-10-08T05:57:25.538133+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.538133+0000","last_undegraded":"2023-10-08T05:57:25.538133+0000","last_fullsized":"2023-10-08T05:57:25.538133+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.555039+0000","last_change":"2023-10-08T05:57:25.549191+0000","last_active":"2023-10-08T05:57:25.555039+0000","last_peered":"2023-10-08T05:57:25.555039+0000","last_clean":"2023-10-08T05:57:25.555039+0000","last_became_active":"2023-10-08T05:57:25.548627+0000","last_became_peered":"2023-10-08T05:57:25.548627+0000","last_unstale":"2023-10-08T05:57:25.555039+0000","last_undegraded":"2023-10-08T05:57:25.555039+0000","last_fullsized":"2023-10-08T05:57:25.555039+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:31:57.375626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.536535+0000","last_change":"2023-10-08T05:57:25.536535+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.536535+0000","last_undegraded":"2023-10-08T05:57:25.536535+0000","last_fullsized":"2023-10-08T05:57:25.536535+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.544832+0000","last_change":"2023-10-08T05:57:25.533519+0000","last_active":"2023-10-08T05:57:25.544832+0000","last_peered":"2023-10-08T05:57:25.544832+0000","last_clean":"2023-10-08T05:57:25.544832+0000","last_became_active":"2023-10-08T05:57:25.533218+0000","last_became_peered":"2023-10-08T05:57:25.533218+0000","last_unstale":"2023-10-08T05:57:25.544832+0000","last_undegraded":"2023-10-08T05:57:25.544832+0000","last_fullsized":"2023-10-08T05:57:25.544832+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:46.551576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.530570+0000","last_change":"2023-10-08T05:57:25.530570+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.530570+0000","last_undegraded":"2023-10-08T05:57:25.530570+0000","last_fullsized":"2023-10-08T05:57:25.530570+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.554611+0000","last_change":"2023-10-08T05:57:25.551666+0000","last_active":"2023-10-08T05:57:25.554611+0000","last_peered":"2023-10-08T05:57:25.554611+0000","last_clean":"2023-10-08T05:57:25.554611+0000","last_became_active":"2023-10-08T05:57:25.551366+0000","last_became_peered":"2023-10-08T05:57:25.551366+0000","last_unstale":"2023-10-08T05:57:25.554611+0000","last_undegraded":"2023-10-08T05:57:25.554611+0000","last_fullsized":"2023-10-08T05:57:25.554611+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:15.895423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.528642+0000","last_change":"2023-10-08T05:57:25.528642+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.528642+0000","last_undegraded":"2023-10-08T05:57:25.528642+0000","last_fullsized":"2023-10-08T05:57:25.528642+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.551672+0000","last_change":"2023-10-08T05:57:25.547967+0000","last_active":"2023-10-08T05:57:25.551672+0000","last_peered":"2023-10-08T05:57:25.551672+0000","last_clean":"2023-10-08T05:57:25.551672+0000","last_became_active":"2023-10-08T05:57:25.547668+0000","last_became_peered":"2023-10-08T05:57:25.547668+0000","last_unstale":"2023-10-08T05:57:25.551672+0000","last_undegraded":"2023-10-08T05:57:25.551672+0000","last_fullsized":"2023-10-08T05:57:25.551672+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:16.523764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.529385+0000","last_change":"2023-10-08T05:57:25.529385+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.529385+0000","last_undegraded":"2023-10-08T05:57:25.529385+0000","last_fullsized":"2023-10-08T05:57:25.529385+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.554646+0000","last_change":"2023-10-08T05:57:25.553740+0000","last_active":"2023-10-08T05:57:25.554646+0000","last_peered":"2023-10-08T05:57:25.554646+0000","last_clean":"2023-10-08T05:57:25.554646+0000","last_became_active":"2023-10-08T05:57:25.553413+0000","last_became_peered":"2023-10-08T05:57:25.553413+0000","last_unstale":"2023-10-08T05:57:25.554646+0000","last_undegraded":"2023-10-08T05:57:25.554646+0000","last_fullsized":"2023-10-08T05:57:25.554646+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:58.858907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.531229+0000","last_change":"2023-10-08T05:57:25.531229+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.531229+0000","last_undegraded":"2023-10-08T05:57:25.531229+0000","last_fullsized":"2023-10-08T05:57:25.531229+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.556054+0000","last_change":"2023-10-08T05:57:25.551912+0000","last_active":"2023-10-08T05:57:25.556054+0000","last_peered":"2023-10-08T05:57:25.556054+0000","last_clean":"2023-10-08T05:57:25.556054+0000","last_became_active":"2023-10-08T05:57:25.551448+0000","last_became_peered":"2023-10-08T05:57:25.551448+0000","last_unstale":"2023-10-08T05:57:25.556054+0000","last_undegraded":"2023-10-08T05:57:25.556054+0000","last_fullsized":"2023-10-08T05:57:25.556054+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:45.166913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.528019+0000","last_change":"2023-10-08T05:57:25.528019+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.528019+0000","last_undegraded":"2023-10-08T05:57:25.528019+0000","last_fullsized":"2023-10-08T05:57:25.528019+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.548439+0000","last_change":"2023-10-08T05:57:25.544149+0000","last_active":"2023-10-08T05:57:25.548439+0000","last_peered":"2023-10-08T05:57:25.548439+0000","last_clean":"2023-10-08T05:57:25.548439+0000","last_became_active":"2023-10-08T05:57:25.543783+0000","last_became_peered":"2023-10-08T05:57:25.543783+0000","last_unstale":"2023-10-08T05:57:25.548439+0000","last_undegraded":"2023-10-08T05:57:25.548439+0000","last_fullsized":"2023-10-08T05:57:25.548439+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:36:25.097944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.529253+0000","last_change":"2023-10-08T05:57:25.529253+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.529253+0000","last_undegraded":"2023-10-08T05:57:25.529253+0000","last_fullsized":"2023-10-08T05:57:25.529253+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.548466+0000","last_change":"2023-10-08T05:57:25.546481+0000","last_active":"2023-10-08T05:57:25.548466+0000","last_peered":"2023-10-08T05:57:25.548466+0000","last_clean":"2023-10-08T05:57:25.548466+0000","last_became_active":"2023-10-08T05:57:25.545928+0000","last_became_peered":"2023-10-08T05:57:25.545928+0000","last_unstale":"2023-10-08T05:57:25.548466+0000","last_undegraded":"2023-10-08T05:57:25.548466+0000","last_fullsized":"2023-10-08T05:57:25.548466+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:03:58.989311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.538869+0000","last_change":"2023-10-08T05:57:25.538869+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.538869+0000","last_undegraded":"2023-10-08T05:57:25.538869+0000","last_fullsized":"2023-10-08T05:57:25.538869+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.555388+0000","last_change":"2023-10-08T05:57:25.553022+0000","last_active":"2023-10-08T05:57:25.555388+0000","last_peered":"2023-10-08T05:57:25.555388+0000","last_clean":"2023-10-08T05:57:25.555388+0000","last_became_active":"2023-10-08T05:57:25.552465+0000","last_became_peered":"2023-10-08T05:57:25.552465+0000","last_unstale":"2023-10-08T05:57:25.555388+0000","last_undegraded":"2023-10-08T05:57:25.555388+0000","last_fullsized":"2023-10-08T05:57:25.555388+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:59.374543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.535758+0000","last_change":"2023-10-08T05:57:25.535758+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.535758+0000","last_undegraded":"2023-10-08T05:57:25.535758+0000","last_fullsized":"2023-10-08T05:57:25.535758+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.555162+0000","last_change":"2023-10-08T05:57:25.551237+0000","last_active":"2023-10-08T05:57:25.555162+0000","last_peered":"2023-10-08T05:57:25.555162+0000","last_clean":"2023-10-08T05:57:25.555162+0000","last_became_active":"2023-10-08T05:57:25.550905+0000","last_became_peered":"2023-10-08T05:57:25.550905+0000","last_unstale":"2023-10-08T05:57:25.555162+0000","last_undegraded":"2023-10-08T05:57:25.555162+0000","last_fullsized":"2023-10-08T05:57:25.555162+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:53:00.279036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.536314+0000","last_change":"2023-10-08T05:57:25.536314+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.536314+0000","last_undegraded":"2023-10-08T05:57:25.536314+0000","last_fullsized":"2023-10-08T05:57:25.536314+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.551427+0000","last_change":"2023-10-08T05:57:25.549405+0000","last_active":"2023-10-08T05:57:25.551427+0000","last_peered":"2023-10-08T05:57:25.551427+0000","last_clean":"2023-10-08T05:57:25.551427+0000","last_became_active":"2023-10-08T05:57:25.549123+0000","last_became_peered":"2023-10-08T05:57:25.549123+0000","last_unstale":"2023-10-08T05:57:25.551427+0000","last_undegraded":"2023-10-08T05:57:25.551427+0000","last_fullsized":"2023-10-08T05:57:25.551427+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:46.551576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.537524+0000","last_change":"2023-10-08T05:57:25.537524+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.537524+0000","last_undegraded":"2023-10-08T05:57:25.537524+0000","last_fullsized":"2023-10-08T05:57:25.537524+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.550378+0000","last_change":"2023-10-08T05:57:25.548597+0000","last_active":"2023-10-08T05:57:25.550378+0000","last_peered":"2023-10-08T05:57:25.550378+0000","last_clean":"2023-10-08T05:57:25.550378+0000","last_became_active":"2023-10-08T05:57:25.548261+0000","last_became_peered":"2023-10-08T05:57:25.548261+0000","last_unstale":"2023-10-08T05:57:25.550378+0000","last_undegraded":"2023-10-08T05:57:25.550378+0000","last_fullsized":"2023-10-08T05:57:25.550378+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:26:49.752903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.538130+0000","last_change":"2023-10-08T05:57:25.538130+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.538130+0000","last_undegraded":"2023-10-08T05:57:25.538130+0000","last_fullsized":"2023-10-08T05:57:25.538130+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.555210+0000","last_change":"2023-10-08T05:57:25.550023+0000","last_active":"2023-10-08T05:57:25.555210+0000","last_peered":"2023-10-08T05:57:25.555210+0000","last_clean":"2023-10-08T05:57:25.555210+0000","last_became_active":"2023-10-08T05:57:25.549752+0000","last_became_peered":"2023-10-08T05:57:25.549752+0000","last_unstale":"2023-10-08T05:57:25.555210+0000","last_undegraded":"2023-10-08T05:57:25.555210+0000","last_fullsized":"2023-10-08T05:57:25.555210+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:05.669501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.539455+0000","last_change":"2023-10-08T05:57:25.539455+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.539455+0000","last_undegraded":"2023-10-08T05:57:25.539455+0000","last_fullsized":"2023-10-08T05:57:25.539455+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.553388+0000","last_change":"2023-10-08T05:57:25.550165+0000","last_active":"2023-10-08T05:57:25.553388+0000","last_peered":"2023-10-08T05:57:25.553388+0000","last_clean":"2023-10-08T05:57:25.553388+0000","last_became_active":"2023-10-08T05:57:25.549904+0000","last_became_peered":"2023-10-08T05:57:25.549904+0000","last_unstale":"2023-10-08T05:57:25.553388+0000","last_undegraded":"2023-10-08T05:57:25.553388+0000","last_fullsized":"2023-10-08T05:57:25.553388+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:27.802612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":1,"reported_epoch":161,"state":"creating+peering","last_fresh":"2023-10-08T05:57:25.529780+0000","last_change":"2023-10-08T05:57:25.529780+0000","last_active":"2023-10-08T05:57:25.498876+0000","last_peered":"2023-10-08T05:57:25.498876+0000","last_clean":"2023-10-08T05:57:25.498876+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T05:57:25.529780+0000","last_undegraded":"2023-10-08T05:57:25.529780+0000","last_fullsized":"2023-10-08T05:57:25.529780+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.557217+0000","last_change":"2023-10-08T05:57:25.554425+0000","last_active":"2023-10-08T05:57:25.557217+0000","last_peered":"2023-10-08T05:57:25.557217+0000","last_clean":"2023-10-08T05:57:25.557217+0000","last_became_active":"2023-10-08T05:57:25.554162+0000","last_became_peered":"2023-10-08T05:57:25.554162+0000","last_unstale":"2023-10-08T05:57:25.557217+0000","last_undegraded":"2023-10-08T05:57:25.557217+0000","last_fullsized":"2023-10-08T05:57:25.557217+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:49.822714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"151'2707","reported_seq":3269,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-10-08T05:57:25.074909+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:57:25.074909+0000","last_peered":"2023-10-08T05:57:25.074909+0000","last_clean":"2023-10-08T05:57:25.074909+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:57:25.074909+0000","last_undegraded":"2023-10-08T05:57:25.074909+0000","last_fullsized":"2023-10-08T05:57:25.074909+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2707,"log_dups_size":0,"ondisk_log_size":2707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1341,"num_read_kb":2764,"num_write":409,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1341,"num_read_kb":2764,"num_write":409,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2707,"ondisk_log_size":2707,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575100,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55908,"kb_used_data":2516,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315932,"statfs":{"total":96636764160,"available":96579514368,"internally_reserved":0,"allocated":2576384,"data_stored":1601832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.113,"15min":1.1890000000000001},"min":{"1min":0.69099999999999995,"5min":0.46899999999999997,"15min":0.36199999999999999},"max":{"1min":1.579,"5min":1.728,"15min":1.8859999999999999},"last":1.222},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1679999999999999,"15min":1.2010000000000001},"min":{"1min":0.65100000000000002,"5min":0.53100000000000003,"15min":0.44},"max":{"1min":1.585,"5min":1.794,"15min":1.794},"last":1.1479999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.208,"15min":1.234},"min":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.49199999999999999},"max":{"1min":1.613,"5min":1.8140000000000001,"15min":1.8360000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.28,"15min":1.276},"min":{"1min":0.78500000000000003,"5min":0.71599999999999997,"15min":0.47099999999999997},"max":{"1min":1.645,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.90700000000000003}]},{"osd":2,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.1970000000000001,"15min":1.236},"min":{"1min":0.89300000000000002,"5min":0.71599999999999997,"15min":0.626},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.8939999999999999},"last":1.099},{"interface":"front","average":{"1min":1.202,"5min":1.2809999999999999,"15min":1.3149999999999999},"min":{"1min":0.72599999999999998,"5min":0.58799999999999997,"15min":0.45400000000000001},"max":{"1min":1.472,"5min":1.97,"15min":1.97},"last":1.3280000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2609999999999999,"15min":1.282},"min":{"1min":0.99099999999999999,"5min":0.72399999999999998,"15min":0.69999999999999996},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.8799999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3580000000000001,"15min":1.363},"min":{"1min":0.82299999999999995,"5min":0.63,"15min":0.60199999999999998},"max":{"1min":1.5740000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.47}]},{"osd":4,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3480000000000001,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.76200000000000001,"15min":0.504},"max":{"1min":1.712,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3700000000000001,"15min":1.3380000000000001},"min":{"1min":0.98299999999999998,"5min":0.81599999999999995,"15min":0.63600000000000001},"max":{"1min":1.7390000000000001,"5min":2.0550000000000002,"15min":2.0699999999999998},"last":1.3680000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4419999999999999,"15min":1.411},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.53400000000000003},"max":{"1min":1.6819999999999999,"5min":1.9810000000000001,"15min":2.0339999999999998},"last":1.389},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.444,"15min":1.4179999999999999},"min":{"1min":1.052,"5min":0.93600000000000005,"15min":0.39000000000000001},"max":{"1min":1.698,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.2789999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.552,"15min":1.5229999999999999},"min":{"1min":1.1659999999999999,"5min":1.0940000000000001,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.498,"15min":1.47},"min":{"1min":1.087,"5min":0.70999999999999996,"15min":0.57799999999999996},"max":{"1min":1.9419999999999999,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.248}]}]},{"osd":6,"up_from":13,"seq":55834575099,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37288,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334552,"statfs":{"total":96636764160,"available":96598581248,"internally_reserved":0,"allocated":1994752,"data_stored":1017673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.1799999999999999,"15min":1.2589999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.8839999999999999,"5min":1.9119999999999999,"15min":2.1429999999999998},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.2769999999999999,"15min":1.2989999999999999},"min":{"1min":0.73599999999999999,"5min":0.63800000000000001,"15min":0.58399999999999996},"max":{"1min":1.599,"5min":1.8100000000000001,"15min":2.3130000000000002},"last":1.6020000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1439999999999999,"15min":1.159},"min":{"1min":0.67500000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5229999999999999,"5min":1.7430000000000001,"15min":1.7789999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.1739999999999999,"15min":1.177},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.466,"5min":1.8169999999999999,"15min":2.0510000000000002},"last":1.343}]},{"osd":2,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1739999999999999,"15min":1.1719999999999999},"min":{"1min":0.66000000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.448},{"interface":"front","average":{"1min":1.218,"5min":1.194,"15min":1.2390000000000001},"min":{"1min":0.77900000000000003,"5min":0.72899999999999998,"15min":0.58699999999999997},"max":{"1min":1.6299999999999999,"5min":1.661,"15min":1.8520000000000001},"last":1.387}]},{"osd":3,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.288,"15min":1.3500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6359999999999999,"5min":1.8420000000000001,"15min":2.137},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4039999999999999,"15min":1.4330000000000001},"min":{"1min":0.98299999999999998,"5min":0.81399999999999995,"15min":0.70499999999999996},"max":{"1min":2.02,"5min":2.024,"15min":2.1930000000000001},"last":1.7430000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.339,"15min":1.319},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.60099999999999998},"max":{"1min":1.778,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.427,"15min":1.3859999999999999},"min":{"1min":0.94699999999999995,"5min":0.72499999999999998,"15min":0.64300000000000002},"max":{"1min":1.929,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.796}]},{"osd":5,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.345,"15min":1.3260000000000001},"min":{"1min":1.0189999999999999,"5min":0.83599999999999997,"15min":0.69999999999999996},"max":{"1min":1.639,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.228},{"interface":"front","average":{"1min":1.431,"5min":1.4970000000000001,"15min":1.482},"min":{"1min":0.88,"5min":0.87,"15min":0.747},"max":{"1min":1.784,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.7090000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.556,"15min":1.506},"min":{"1min":1.1599999999999999,"5min":0.82399999999999995,"15min":0.77200000000000002},"max":{"1min":1.9590000000000001,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.4930000000000001,"15min":1.474},"min":{"1min":0.93899999999999995,"5min":0.85999999999999999,"15min":0.81899999999999995},"max":{"1min":1.9410000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.577}]}]},{"osd":5,"up_from":13,"seq":55834575099,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51176,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320664,"statfs":{"total":96636764160,"available":96584359936,"internally_reserved":0,"allocated":1994752,"data_stored":1017673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.161,"15min":1.1220000000000001},"min":{"1min":0.65000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.429},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.173,"15min":1.1950000000000001},"min":{"1min":0.72099999999999997,"5min":0.47899999999999998,"15min":0.442},"max":{"1min":1.5980000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.90700000000000003}]},{"osd":1,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0860000000000001,"15min":1.1819999999999999},"min":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.575,"5min":2.5009999999999999,"15min":3.2000000000000002},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.2150000000000001,"15min":1.2450000000000001},"min":{"1min":0.72299999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6779999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.288}]},{"osd":2,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.276,"15min":1.232},"min":{"1min":0.98099999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4590000000000001,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2230000000000001,"15min":1.27},"min":{"1min":0.90100000000000002,"5min":0.71399999999999997,"15min":0.65000000000000002},"max":{"1min":1.621,"5min":2.3809999999999998,"15min":3.282},"last":1.0649999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2989999999999999,"15min":1.3280000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.63500000000000001},"max":{"1min":1.8300000000000001,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.385,"15min":1.395},"min":{"1min":1.0369999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.7030000000000001,"5min":1.9370000000000001,"15min":2.2280000000000002},"last":1.3939999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3169999999999999,"15min":1.3069999999999999},"min":{"1min":0.85499999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.591,"5min":2.7480000000000002,"15min":4.0179999999999998},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.421,"15min":1.4350000000000001},"min":{"1min":1.014,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8700000000000001,"5min":2.927,"15min":3.456},"last":1.6000000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4590000000000001,"15min":1.4139999999999999},"min":{"1min":0.996,"5min":0.751,"15min":0.751},"max":{"1min":1.7290000000000001,"5min":2.149,"15min":2.359},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4770000000000001,"15min":1.4670000000000001},"min":{"1min":1.121,"5min":0.623,"15min":0.58099999999999996},"max":{"1min":1.673,"5min":2.6429999999999998,"15min":3.8620000000000001},"last":1.2010000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.54,"15min":1.544},"min":{"1min":1.1459999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9299999999999999,"5min":3.0270000000000001,"15min":3.4079999999999999},"last":1.98},{"interface":"front","average":{"1min":1.556,"5min":1.595,"15min":1.5640000000000001},"min":{"1min":1.1870000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8799999999999999,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.4710000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575099,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37304,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334536,"statfs":{"total":96636764160,"available":96598564864,"internally_reserved":0,"allocated":2007040,"data_stored":1024332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2250000000000001,"15min":1.2649999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.498},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.377,"5min":1.3160000000000001,"15min":1.357},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.756,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.46}]},{"osd":1,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.232,"15min":1.272},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.639,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.218,"15min":1.2290000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.893,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":1.377}]},{"osd":2,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.206,"15min":1.224},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.798,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.262,"15min":1.3260000000000001},"min":{"1min":0.71799999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.843,"5min":2.431,"15min":2.431},"last":1.1180000000000001}]},{"osd":3,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.258,"15min":1.272},"min":{"1min":0.747,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.704,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.80800000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.9379999999999999,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.0860000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.361,"15min":1.339},"min":{"1min":0.76700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.891,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.325},{"interface":"front","average":{"1min":1.355,"5min":1.3779999999999999,"15min":1.3660000000000001},"min":{"1min":0.83399999999999996,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7969999999999999,"5min":5.4189999999999996,"15min":5.4189999999999996},"last":1.6200000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.4419999999999999,"15min":1.3720000000000001},"min":{"1min":0.91600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.169,"5min":4.9059999999999997,"15min":4.9059999999999997},"last":1.417},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5449999999999999,"15min":1.5109999999999999},"min":{"1min":0.96299999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.9279999999999999,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":1.6679999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.52,"15min":1.4810000000000001},"min":{"1min":0.78900000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.0230000000000001,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.546,"5min":1.5680000000000001,"15min":1.5289999999999999},"min":{"1min":0.995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.1120000000000001,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":1.284}]}]},{"osd":3,"up_from":13,"seq":55834575101,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41448,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330392,"statfs":{"total":96636764160,"available":96594321408,"internally_reserved":0,"allocated":1994752,"data_stored":1017673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.88400000000000001,"15min":0.91500000000000004},"min":{"1min":0.625,"5min":0.314,"15min":0.314},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0469999999999999,"15min":1.087},"min":{"1min":0.57199999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.56899999999999995}]},{"osd":1,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90000000000000002,"15min":0.94499999999999995},"min":{"1min":0.66500000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.52,"5min":1.605,"15min":1.605},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.135,"5min":1.129,"15min":1.1859999999999999},"min":{"1min":0.58899999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.6439999999999999,"5min":1.6799999999999999,"15min":1.7210000000000001},"last":0.495}]},{"osd":2,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.98099999999999998,"15min":1.0489999999999999},"min":{"1min":0.56599999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.077,"15min":1.1339999999999999},"min":{"1min":0.64000000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":0.59699999999999998}]},{"osd":4,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1819999999999999,"15min":1.2130000000000001},"min":{"1min":0.73399999999999999,"5min":0.68999999999999995,"15min":0.54000000000000004},"max":{"1min":1.792,"5min":1.792,"15min":2.2010000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.373,"5min":1.341,"15min":1.375},"min":{"1min":0.95399999999999996,"5min":0.82699999999999996,"15min":0.72799999999999998},"max":{"1min":2.448,"5min":2.448,"15min":2.589},"last":1.008}]},{"osd":5,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.276,"15min":1.294},"min":{"1min":0.84899999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6919999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3,"15min":1.343},"min":{"1min":1.028,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.323,"15min":1.3220000000000001},"min":{"1min":0.84399999999999997,"5min":0.65700000000000003,"15min":0.61799999999999999},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.488,"5min":1.4279999999999999,"15min":1.4430000000000001},"min":{"1min":1.143,"5min":0.90300000000000002,"15min":0.86699999999999999},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.94599999999999995}]},{"osd":7,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.446,"15min":1.486},"min":{"1min":1.113,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.367,"15min":1.385},"min":{"1min":0.89900000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.1759999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575099,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63656,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308184,"statfs":{"total":96636764160,"available":96571580416,"internally_reserved":0,"allocated":1994752,"data_stored":1017673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78600000000000003,"15min":0.86299999999999999},"min":{"1min":0.34699999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.28,"5min":1.629,"15min":1.629},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.90700000000000003,"15min":1.0049999999999999},"min":{"1min":0.40400000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2470000000000001,"5min":1.5660000000000001,"15min":1.9470000000000001},"last":0.98199999999999998}]},{"osd":1,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.85899999999999999,"15min":0.92800000000000005},"min":{"1min":0.498,"5min":0.45900000000000002,"15min":0.42199999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":2.1070000000000002},"last":0.878},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.96699999999999997,"15min":1.0569999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.35099999999999998},"max":{"1min":1.7050000000000001,"5min":1.79,"15min":2.1469999999999998},"last":1.048}]},{"osd":3,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0209999999999999,"15min":1.099},"min":{"1min":0.58799999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.8120000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.085,"15min":1.1970000000000001},"min":{"1min":0.54100000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.5529999999999999,"5min":1.8859999999999999,"15min":7.5890000000000004},"last":1.161}]},{"osd":4,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.286,"15min":1.363},"min":{"1min":0.80200000000000005,"5min":0.66000000000000003,"15min":0.622},"max":{"1min":3.6379999999999999,"5min":3.6379999999999999,"15min":4.593},"last":1.27},{"interface":"front","average":{"1min":1.335,"5min":1.2350000000000001,"15min":1.2789999999999999},"min":{"1min":0.73599999999999999,"5min":0.624,"15min":0.624},"max":{"1min":4.7770000000000001,"5min":4.7770000000000001,"15min":6.6879999999999997},"last":1.3280000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2410000000000001,"15min":1.3480000000000001},"min":{"1min":0.75800000000000001,"5min":0.68799999999999994,"15min":0.63900000000000001},"max":{"1min":1.7,"5min":1.885,"15min":6.7140000000000004},"last":1.304},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.3979999999999999,"15min":1.4419999999999999},"min":{"1min":0.872,"5min":0.71999999999999997,"15min":0.56599999999999995},"max":{"1min":6.625,"5min":6.625,"15min":6.625},"last":1.3959999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2989999999999999,"15min":1.3380000000000001},"min":{"1min":0.73299999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.73,"5min":2.0089999999999999,"15min":2.7080000000000002},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.573,"5min":1.3640000000000001,"15min":1.4379999999999999},"min":{"1min":0.77000000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":5.9569999999999999,"5min":5.9569999999999999,"15min":5.9569999999999999},"last":1.373}]},{"osd":7,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3380000000000001,"15min":1.417},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8799999999999999,"5min":1.956,"15min":6.609},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.3660000000000001,"15min":1.47},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8819999999999999,"5min":2.0070000000000001,"15min":2.9950000000000001},"last":1.474}]}]},{"osd":1,"up_from":13,"seq":55834575099,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41448,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330392,"statfs":{"total":96636764160,"available":96594321408,"internally_reserved":0,"allocated":1994752,"data_stored":1017673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.94099999999999995,"15min":0.97199999999999998},"min":{"1min":0.58999999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.454,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":1.153},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.014,"15min":1.1080000000000001},"min":{"1min":0.52600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.5049999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.3009999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.84499999999999997,"15min":0.92500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45100000000000001,"15min":0.39500000000000002},"max":{"1min":0.998,"5min":1.377,"15min":1.8240000000000001},"last":1.111},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0609999999999999,"15min":1.093},"min":{"1min":0.55200000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.448,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":1.246}]},{"osd":3,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0700000000000001,"15min":1.1579999999999999},"min":{"1min":0.505,"5min":0.498,"15min":0.498},"max":{"1min":1.5269999999999999,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.98899999999999999,"15min":1.0609999999999999},"min":{"1min":0.47899999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.575,"5min":1.6120000000000001,"15min":1.788},"last":1.0609999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.216,"15min":1.2809999999999999},"min":{"1min":0.70599999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6539999999999999,"5min":1.891,"15min":1.966},"last":1.196},{"interface":"front","average":{"1min":1.099,"5min":1.1519999999999999,"15min":1.212},"min":{"1min":0.83499999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3640000000000001,"5min":1.99,"15min":1.99},"last":1.3540000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2390000000000001,"15min":1.3009999999999999},"min":{"1min":0.88700000000000001,"5min":0.73199999999999998,"15min":0.70399999999999996},"max":{"1min":1.518,"5min":1.7909999999999999,"15min":1.8680000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.4059999999999999,"15min":1.448},"min":{"1min":0.85499999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.73,"5min":2.226,"15min":2.226},"last":1.5549999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.359,"15min":1.409},"min":{"1min":0.84999999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.833,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.486},{"interface":"front","average":{"1min":1.325,"5min":1.339,"15min":1.4019999999999999},"min":{"1min":0.877,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6220000000000001,"5min":1.8580000000000001,"15min":2.8559999999999999},"last":1.621}]},{"osd":7,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.407,"15min":1.4550000000000001},"min":{"1min":0.94199999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.9490000000000001,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.3839999999999999,"15min":1.4410000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.9930000000000001,"5min":2.0819999999999999,"15min":2.306},"last":1.421}]}]},{"osd":0,"up_from":13,"seq":55834575100,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33708,"kb_used_data":2528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338132,"statfs":{"total":96636764160,"available":96602247168,"internally_reserved":0,"allocated":2588672,"data_stored":1608041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.86099999999999999,"15min":0.89800000000000002},"min":{"1min":0.54300000000000004,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":1.288,"5min":1.77,"15min":1.77},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94199999999999995,"15min":1.022},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.675},"last":1.6759999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.98499999999999999,"15min":1.0629999999999999},"min":{"1min":0.501,"5min":0.42599999999999999,"15min":0.39200000000000002},"max":{"1min":1.5700000000000001,"5min":1.5740000000000001,"15min":1.8700000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.93000000000000005,"15min":0.98199999999999998},"min":{"1min":0.51200000000000001,"5min":0.47399999999999998,"15min":0.40400000000000003},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":0.876}]},{"osd":3,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0960000000000001,"15min":1.1759999999999999},"min":{"1min":0.60199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4650000000000001,"5min":1.823,"15min":1.9730000000000001},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.036,"5min":1.016,"15min":1.0760000000000001},"min":{"1min":0.57999999999999996,"5min":0.46500000000000002,"15min":0.44700000000000001},"max":{"1min":1.891,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.7470000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.252},"min":{"1min":0.66900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.623,"5min":1.8129999999999999,"15min":1.9410000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.3100000000000001,"15min":1.333},"min":{"1min":0.83799999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.321}]},{"osd":5,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2669999999999999,"15min":1.3220000000000001},"min":{"1min":0.64100000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5980000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.371},{"interface":"front","average":{"1min":1.29,"5min":1.333,"15min":1.365},"min":{"1min":0.86099999999999999,"5min":0.71199999999999997,"15min":0.63},"max":{"1min":1.9710000000000001,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":1.5800000000000001}]},{"osd":6,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.72099999999999997,"5min":0.64700000000000002,"15min":0.58499999999999996},"max":{"1min":1.8089999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.3380000000000001,"15min":1.379},"min":{"1min":0.88100000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.4630000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.3839999999999999,"15min":1.419},"min":{"1min":0.92400000000000004,"5min":0.66300000000000003,"15min":0.63800000000000001},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.639},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4119999999999999,"15min":1.458},"min":{"1min":0.751,"5min":0.73699999999999999,"15min":0.54700000000000004},"max":{"1min":1.8839999999999999,"5min":2.0569999999999999,"15min":2.1059999999999999},"last":1.794}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:57:31.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T05:57:31.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.578+0000 7f939861a640 1 Processor -- start 2023-10-08T05:57:31.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.578+0000 7f939861a640 1 -- start start 2023-10-08T05:57:31.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.579+0000 7f939861a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9390159ac0 0x7f939006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:31.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.579+0000 7f939861a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93900ad790 0x7f93900702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:31.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.579+0000 7f939861a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f939015e330 0x7f9390070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:31.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.579+0000 7f939861a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939006dee0 con 0x7f939015e330 2023-10-08T05:57:31.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.580+0000 7f939861a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939015d850 con 0x7f9390159ac0 2023-10-08T05:57:31.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.580+0000 7f939861a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939015db30 con 0x7f93900ad790 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.580+0000 7f9396bb7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f939015e330 0x7f9390070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.580+0000 7f9396bb7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f939015e330 0x7f9390070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52286/0 (socket says 172.21.15.70:52286) 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.580+0000 7f9396bb7640 1 -- 172.21.15.70:0/4231851418 learned_addr learned my addr 172.21.15.70:0/4231851418 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.580+0000 7f9395bb5640 1 --2- 172.21.15.70:0/4231851418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93900ad790 0x7f93900702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.580+0000 7f93963b6640 1 --2- 172.21.15.70:0/4231851418 >> v2:172.21.15.181:3300/0 conn(0x7f9390159ac0 0x7f939006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.580+0000 7f9396bb7640 1 -- 172.21.15.70:0/4231851418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93900ad790 msgr2=0x7f93900702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.580+0000 7f9396bb7640 1 --2- 172.21.15.70:0/4231851418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93900ad790 0x7f93900702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.581+0000 7f9396bb7640 1 -- 172.21.15.70:0/4231851418 >> v2:172.21.15.181:3300/0 conn(0x7f9390159ac0 msgr2=0x7f939006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.581+0000 7f9396bb7640 1 --2- 172.21.15.70:0/4231851418 >> v2:172.21.15.181:3300/0 conn(0x7f9390159ac0 0x7f939006d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.581+0000 7f9396bb7640 1 -- 172.21.15.70:0/4231851418 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939015de10 con 0x7f939015e330 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.582+0000 7f9396bb7640 1 --2- 172.21.15.70:0/4231851418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 0x7f9390070810 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f939006e790 tx=0x7f938802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e08f8fea63a6c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:31.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.582+0000 7f93953b4640 1 -- 172.21.15.70:0/4231851418 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938802ca80 con 0x7f939015e330 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.582+0000 7f93953b4640 1 -- 172.21.15.70:0/4231851418 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938802cbe0 con 0x7f939015e330 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.582+0000 7f93953b4640 1 -- 172.21.15.70:0/4231851418 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93880343f0 con 0x7f939015e330 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.583+0000 7f939861a640 1 -- 172.21.15.70:0/4231851418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 msgr2=0x7f9390070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.583+0000 7f939861a640 1 --2- 172.21.15.70:0/4231851418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 0x7f9390070810 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f939006e790 tx=0x7f938802bee0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.583+0000 7f939861a640 1 -- 172.21.15.70:0/4231851418 shutdown_connections 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.583+0000 7f939861a640 1 --2- 172.21.15.70:0/4231851418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 0x7f9390070810 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.583+0000 7f939861a640 1 --2- 172.21.15.70:0/4231851418 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93900ad790 0x7f93900702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.583+0000 7f939861a640 1 --2- 172.21.15.70:0/4231851418 >> v2:172.21.15.181:3300/0 conn(0x7f9390159ac0 0x7f939006d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.583+0000 7f939861a640 1 -- 172.21.15.70:0/4231851418 >> 172.21.15.70:0/4231851418 conn(0x7f939006a380 msgr2=0x7f93900b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.583+0000 7f939861a640 1 -- 172.21.15.70:0/4231851418 shutdown_connections 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.583+0000 7f939861a640 1 -- 172.21.15.70:0/4231851418 wait complete. 2023-10-08T05:57:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.585+0000 7f939861a640 1 Processor -- start 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.585+0000 7f939861a640 1 -- start start 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f939861a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93900ad790 0x7f939014c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f939861a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9390159ac0 0x7f9390155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f939861a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 0x7f939014cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f939861a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93900b29d0 con 0x7f939015e330 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f939861a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9390155d10 con 0x7f93900ad790 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f939861a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93900b20b0 con 0x7f9390159ac0 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f9396bb7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 0x7f939014cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f9396bb7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 0x7f939014cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52290/0 (socket says 172.21.15.70:52290) 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f9396bb7640 1 -- 172.21.15.70:0/2340141026 learned_addr learned my addr 172.21.15.70:0/2340141026 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f9395bb5640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9390159ac0 0x7f9390155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:31.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f93963b6640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93900ad790 0x7f939014c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:31.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f9396bb7640 1 -- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9390159ac0 msgr2=0x7f9390155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:31.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f9396bb7640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9390159ac0 0x7f9390155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f9396bb7640 1 -- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93900ad790 msgr2=0x7f939014c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:31.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f9396bb7640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93900ad790 0x7f939014c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f9396bb7640 1 -- 172.21.15.70:0/2340141026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9388009b30 con 0x7f939015e330 2023-10-08T05:57:31.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.586+0000 7f9395bb5640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9390159ac0 0x7f9390155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:31.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.587+0000 7f9396bb7640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 0x7f939014cb30 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f9388009f40 tx=0x7f9388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:31.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.587+0000 7f9386ffd640 1 -- 172.21.15.70:0/2340141026 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388009510 con 0x7f939015e330 2023-10-08T05:57:31.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.587+0000 7f9386ffd640 1 -- 172.21.15.70:0/2340141026 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938803e450 con 0x7f939015e330 2023-10-08T05:57:31.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.587+0000 7f9386ffd640 1 -- 172.21.15.70:0/2340141026 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938803e800 con 0x7f939015e330 2023-10-08T05:57:31.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.587+0000 7f939861a640 1 -- 172.21.15.70:0/2340141026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93900b2350 con 0x7f939015e330 2023-10-08T05:57:31.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.587+0000 7f939861a640 1 -- 172.21.15.70:0/2340141026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93900b25f0 con 0x7f939015e330 2023-10-08T05:57:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.588+0000 7f939861a640 1 -- 172.21.15.70:0/2340141026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9390067490 con 0x7f939015e330 2023-10-08T05:57:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.589+0000 7f9386ffd640 1 -- 172.21.15.70:0/2340141026 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f938802cd50 con 0x7f939015e330 2023-10-08T05:57:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.591+0000 7f9386ffd640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f937406af90 0x7f937406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.591+0000 7f9386ffd640 1 -- 172.21.15.70:0/2340141026 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93880ac3f0 con 0x7f939015e330 2023-10-08T05:57:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.591+0000 7f93963b6640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f937406af90 0x7f937406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:31.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.592+0000 7f93963b6640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f937406af90 0x7f937406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f938c006fd0 tx=0x7f938c008040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:31.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.595+0000 7f9386ffd640 1 -- 172.21.15.70:0/2340141026 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93880771f0 con 0x7f939015e330 2023-10-08T05:57:31.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.782+0000 7f939861a640 1 -- 172.21.15.70:0/2340141026 --> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9390159ea0 con 0x7f937406af90 2023-10-08T05:57:31.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.805+0000 7f9386ffd640 1 -- 172.21.15.70:0/2340141026 <== mgr.6299 v2:172.21.15.181:6832/2068783299 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287347 (secure 0 0 0) 0x7f938c009d60 con 0x7f937406af90 2023-10-08T05:57:31.812 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:31.813 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T05:57:31.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 -- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f937406af90 msgr2=0x7f937406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:31.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f937406af90 0x7f937406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f938c006fd0 tx=0x7f938c008040 comp rx=0 tx=0).stop 2023-10-08T05:57:31.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 -- 172.21.15.70:0/2340141026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 msgr2=0x7f939014cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:31.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 0x7f939014cb30 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f9388009f40 tx=0x7f9388004a40 comp rx=0 tx=0).stop 2023-10-08T05:57:31.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 -- 172.21.15.70:0/2340141026 shutdown_connections 2023-10-08T05:57:31.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f937406af90 0x7f937406d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f939015e330 0x7f939014cb30 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9390159ac0 0x7f9390155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 --2- 172.21.15.70:0/2340141026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93900ad790 0x7f939014c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:31.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 -- 172.21.15.70:0/2340141026 >> 172.21.15.70:0/2340141026 conn(0x7f939006a380 msgr2=0x7f93900b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:31.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 -- 172.21.15.70:0/2340141026 shutdown_connections 2023-10-08T05:57:31.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:31.812+0000 7f939861a640 1 -- 172.21.15.70:0/2340141026 wait complete. 2023-10-08T05:57:31.858 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-10-08T05:57:31.604502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1341,"num_read_kb":2764,"num_write":454,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2746,"ondisk_log_size":2746,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":362300,"kb_used_data":17096,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754612420,"statfs":{"total":773094113280,"available":772723118080,"internally_reserved":0,"allocated":17506304,"data_stored":9491084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":353357371},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.085086"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.553754+0000","last_change":"2023-10-08T05:57:26.559692+0000","last_active":"2023-10-08T05:57:27.553754+0000","last_peered":"2023-10-08T05:57:27.553754+0000","last_clean":"2023-10-08T05:57:27.553754+0000","last_became_active":"2023-10-08T05:57:26.559407+0000","last_became_peered":"2023-10-08T05:57:26.559407+0000","last_unstale":"2023-10-08T05:57:27.553754+0000","last_undegraded":"2023-10-08T05:57:27.553754+0000","last_fullsized":"2023-10-08T05:57:27.553754+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:09.702913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.942041+0000","last_change":"2023-10-08T05:57:26.552169+0000","last_active":"2023-10-08T05:57:27.942041+0000","last_peered":"2023-10-08T05:57:27.942041+0000","last_clean":"2023-10-08T05:57:27.942041+0000","last_became_active":"2023-10-08T05:57:26.551856+0000","last_became_peered":"2023-10-08T05:57:26.551856+0000","last_unstale":"2023-10-08T05:57:27.942041+0000","last_undegraded":"2023-10-08T05:57:27.942041+0000","last_fullsized":"2023-10-08T05:57:27.942041+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:31:54.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.559850+0000","last_change":"2023-10-08T05:57:26.558821+0000","last_active":"2023-10-08T05:57:27.559850+0000","last_peered":"2023-10-08T05:57:27.559850+0000","last_clean":"2023-10-08T05:57:27.559850+0000","last_became_active":"2023-10-08T05:57:26.558197+0000","last_became_peered":"2023-10-08T05:57:26.558197+0000","last_unstale":"2023-10-08T05:57:27.559850+0000","last_undegraded":"2023-10-08T05:57:27.559850+0000","last_fullsized":"2023-10-08T05:57:27.559850+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:00.389213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940510+0000","last_change":"2023-10-08T05:57:26.559628+0000","last_active":"2023-10-08T05:57:27.940510+0000","last_peered":"2023-10-08T05:57:27.940510+0000","last_clean":"2023-10-08T05:57:27.940510+0000","last_became_active":"2023-10-08T05:57:26.558958+0000","last_became_peered":"2023-10-08T05:57:26.558958+0000","last_unstale":"2023-10-08T05:57:27.940510+0000","last_undegraded":"2023-10-08T05:57:27.940510+0000","last_fullsized":"2023-10-08T05:57:27.940510+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:01:04.467557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.938551+0000","last_change":"2023-10-08T05:57:26.555264+0000","last_active":"2023-10-08T05:57:27.938551+0000","last_peered":"2023-10-08T05:57:27.938551+0000","last_clean":"2023-10-08T05:57:27.938551+0000","last_became_active":"2023-10-08T05:57:26.554964+0000","last_became_peered":"2023-10-08T05:57:26.554964+0000","last_unstale":"2023-10-08T05:57:27.938551+0000","last_undegraded":"2023-10-08T05:57:27.938551+0000","last_fullsized":"2023-10-08T05:57:27.938551+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:31:54.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940238+0000","last_change":"2023-10-08T05:57:26.562073+0000","last_active":"2023-10-08T05:57:27.940238+0000","last_peered":"2023-10-08T05:57:27.940238+0000","last_clean":"2023-10-08T05:57:27.940238+0000","last_became_active":"2023-10-08T05:57:26.561685+0000","last_became_peered":"2023-10-08T05:57:26.561685+0000","last_unstale":"2023-10-08T05:57:27.940238+0000","last_undegraded":"2023-10-08T05:57:27.940238+0000","last_fullsized":"2023-10-08T05:57:27.940238+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:24.175250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.939928+0000","last_change":"2023-10-08T05:57:26.556662+0000","last_active":"2023-10-08T05:57:27.939928+0000","last_peered":"2023-10-08T05:57:27.939928+0000","last_clean":"2023-10-08T05:57:27.939928+0000","last_became_active":"2023-10-08T05:57:26.556356+0000","last_became_peered":"2023-10-08T05:57:26.556356+0000","last_unstale":"2023-10-08T05:57:27.939928+0000","last_undegraded":"2023-10-08T05:57:27.939928+0000","last_fullsized":"2023-10-08T05:57:27.939928+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:03.668214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.558104+0000","last_change":"2023-10-08T05:57:26.553393+0000","last_active":"2023-10-08T05:57:27.558104+0000","last_peered":"2023-10-08T05:57:27.558104+0000","last_clean":"2023-10-08T05:57:27.558104+0000","last_became_active":"2023-10-08T05:57:26.552810+0000","last_became_peered":"2023-10-08T05:57:26.552810+0000","last_unstale":"2023-10-08T05:57:27.558104+0000","last_undegraded":"2023-10-08T05:57:27.558104+0000","last_fullsized":"2023-10-08T05:57:27.558104+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:08.671470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.941105+0000","last_change":"2023-10-08T05:57:26.563039+0000","last_active":"2023-10-08T05:57:27.941105+0000","last_peered":"2023-10-08T05:57:27.941105+0000","last_clean":"2023-10-08T05:57:27.941105+0000","last_became_active":"2023-10-08T05:57:26.562451+0000","last_became_peered":"2023-10-08T05:57:26.562451+0000","last_unstale":"2023-10-08T05:57:27.941105+0000","last_undegraded":"2023-10-08T05:57:27.941105+0000","last_fullsized":"2023-10-08T05:57:27.941105+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:55:57.808523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940674+0000","last_change":"2023-10-08T05:57:26.554261+0000","last_active":"2023-10-08T05:57:27.940674+0000","last_peered":"2023-10-08T05:57:27.940674+0000","last_clean":"2023-10-08T05:57:27.940674+0000","last_became_active":"2023-10-08T05:57:26.553988+0000","last_became_peered":"2023-10-08T05:57:26.553988+0000","last_unstale":"2023-10-08T05:57:27.940674+0000","last_undegraded":"2023-10-08T05:57:27.940674+0000","last_fullsized":"2023-10-08T05:57:27.940674+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:09.024224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940870+0000","last_change":"2023-10-08T05:57:26.554484+0000","last_active":"2023-10-08T05:57:27.940870+0000","last_peered":"2023-10-08T05:57:27.940870+0000","last_clean":"2023-10-08T05:57:27.940870+0000","last_became_active":"2023-10-08T05:57:26.554148+0000","last_became_peered":"2023-10-08T05:57:26.554148+0000","last_unstale":"2023-10-08T05:57:27.940870+0000","last_undegraded":"2023-10-08T05:57:27.940870+0000","last_fullsized":"2023-10-08T05:57:27.940870+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:09.024224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.553650+0000","last_change":"2023-10-08T05:57:26.559032+0000","last_active":"2023-10-08T05:57:27.553650+0000","last_peered":"2023-10-08T05:57:27.553650+0000","last_clean":"2023-10-08T05:57:27.553650+0000","last_became_active":"2023-10-08T05:57:26.558557+0000","last_became_peered":"2023-10-08T05:57:26.558557+0000","last_unstale":"2023-10-08T05:57:27.553650+0000","last_undegraded":"2023-10-08T05:57:27.553650+0000","last_fullsized":"2023-10-08T05:57:27.553650+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:33:13.798978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940161+0000","last_change":"2023-10-08T05:57:26.562882+0000","last_active":"2023-10-08T05:57:27.940161+0000","last_peered":"2023-10-08T05:57:27.940161+0000","last_clean":"2023-10-08T05:57:27.940161+0000","last_became_active":"2023-10-08T05:57:26.561778+0000","last_became_peered":"2023-10-08T05:57:26.561778+0000","last_unstale":"2023-10-08T05:57:27.940161+0000","last_undegraded":"2023-10-08T05:57:27.940161+0000","last_fullsized":"2023-10-08T05:57:27.940161+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:48.355431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.553041+0000","last_change":"2023-10-08T05:57:26.567828+0000","last_active":"2023-10-08T05:57:27.553041+0000","last_peered":"2023-10-08T05:57:27.553041+0000","last_clean":"2023-10-08T05:57:27.553041+0000","last_became_active":"2023-10-08T05:57:26.567549+0000","last_became_peered":"2023-10-08T05:57:26.567549+0000","last_unstale":"2023-10-08T05:57:27.553041+0000","last_undegraded":"2023-10-08T05:57:27.553041+0000","last_fullsized":"2023-10-08T05:57:27.553041+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:38:30.460812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.552721+0000","last_change":"2023-10-08T05:57:26.566200+0000","last_active":"2023-10-08T05:57:27.552721+0000","last_peered":"2023-10-08T05:57:27.552721+0000","last_clean":"2023-10-08T05:57:27.552721+0000","last_became_active":"2023-10-08T05:57:26.565926+0000","last_became_peered":"2023-10-08T05:57:26.565926+0000","last_unstale":"2023-10-08T05:57:27.552721+0000","last_undegraded":"2023-10-08T05:57:27.552721+0000","last_fullsized":"2023-10-08T05:57:27.552721+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:20.391447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.939990+0000","last_change":"2023-10-08T05:57:26.563990+0000","last_active":"2023-10-08T05:57:27.939990+0000","last_peered":"2023-10-08T05:57:27.939990+0000","last_clean":"2023-10-08T05:57:27.939990+0000","last_became_active":"2023-10-08T05:57:26.563679+0000","last_became_peered":"2023-10-08T05:57:26.563679+0000","last_unstale":"2023-10-08T05:57:27.939990+0000","last_undegraded":"2023-10-08T05:57:27.939990+0000","last_fullsized":"2023-10-08T05:57:27.939990+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:47.508590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.941230+0000","last_change":"2023-10-08T05:57:26.565547+0000","last_active":"2023-10-08T05:57:27.941230+0000","last_peered":"2023-10-08T05:57:27.941230+0000","last_clean":"2023-10-08T05:57:27.941230+0000","last_became_active":"2023-10-08T05:57:26.565278+0000","last_became_peered":"2023-10-08T05:57:26.565278+0000","last_unstale":"2023-10-08T05:57:27.941230+0000","last_undegraded":"2023-10-08T05:57:27.941230+0000","last_fullsized":"2023-10-08T05:57:27.941230+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:16.155775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.939057+0000","last_change":"2023-10-08T05:57:26.554211+0000","last_active":"2023-10-08T05:57:27.939057+0000","last_peered":"2023-10-08T05:57:27.939057+0000","last_clean":"2023-10-08T05:57:27.939057+0000","last_became_active":"2023-10-08T05:57:26.553829+0000","last_became_peered":"2023-10-08T05:57:26.553829+0000","last_unstale":"2023-10-08T05:57:27.939057+0000","last_undegraded":"2023-10-08T05:57:27.939057+0000","last_fullsized":"2023-10-08T05:57:27.939057+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:03:40.726156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940143+0000","last_change":"2023-10-08T05:57:26.557521+0000","last_active":"2023-10-08T05:57:27.940143+0000","last_peered":"2023-10-08T05:57:27.940143+0000","last_clean":"2023-10-08T05:57:27.940143+0000","last_became_active":"2023-10-08T05:57:26.556986+0000","last_became_peered":"2023-10-08T05:57:26.556986+0000","last_unstale":"2023-10-08T05:57:27.940143+0000","last_undegraded":"2023-10-08T05:57:27.940143+0000","last_fullsized":"2023-10-08T05:57:27.940143+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:49:54.449762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.938777+0000","last_change":"2023-10-08T05:57:26.567366+0000","last_active":"2023-10-08T05:57:27.938777+0000","last_peered":"2023-10-08T05:57:27.938777+0000","last_clean":"2023-10-08T05:57:27.938777+0000","last_became_active":"2023-10-08T05:57:26.567112+0000","last_became_peered":"2023-10-08T05:57:26.567112+0000","last_unstale":"2023-10-08T05:57:27.938777+0000","last_undegraded":"2023-10-08T05:57:27.938777+0000","last_fullsized":"2023-10-08T05:57:27.938777+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:49:03.668214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.939870+0000","last_change":"2023-10-08T05:57:26.565890+0000","last_active":"2023-10-08T05:57:27.939870+0000","last_peered":"2023-10-08T05:57:27.939870+0000","last_clean":"2023-10-08T05:57:27.939870+0000","last_became_active":"2023-10-08T05:57:26.565609+0000","last_became_peered":"2023-10-08T05:57:26.565609+0000","last_unstale":"2023-10-08T05:57:27.939870+0000","last_undegraded":"2023-10-08T05:57:27.939870+0000","last_fullsized":"2023-10-08T05:57:27.939870+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:03:44.707339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.553047+0000","last_change":"2023-10-08T05:57:26.552650+0000","last_active":"2023-10-08T05:57:27.553047+0000","last_peered":"2023-10-08T05:57:27.553047+0000","last_clean":"2023-10-08T05:57:27.553047+0000","last_became_active":"2023-10-08T05:57:26.552346+0000","last_became_peered":"2023-10-08T05:57:26.552346+0000","last_unstale":"2023-10-08T05:57:27.553047+0000","last_undegraded":"2023-10-08T05:57:27.553047+0000","last_fullsized":"2023-10-08T05:57:27.553047+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:16:03.841121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.941469+0000","last_change":"2023-10-08T05:57:26.563609+0000","last_active":"2023-10-08T05:57:27.941469+0000","last_peered":"2023-10-08T05:57:27.941469+0000","last_clean":"2023-10-08T05:57:27.941469+0000","last_became_active":"2023-10-08T05:57:26.563310+0000","last_became_peered":"2023-10-08T05:57:26.563310+0000","last_unstale":"2023-10-08T05:57:27.941469+0000","last_undegraded":"2023-10-08T05:57:27.941469+0000","last_fullsized":"2023-10-08T05:57:27.941469+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:23.522332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.939882+0000","last_change":"2023-10-08T05:57:26.563828+0000","last_active":"2023-10-08T05:57:27.939882+0000","last_peered":"2023-10-08T05:57:27.939882+0000","last_clean":"2023-10-08T05:57:27.939882+0000","last_became_active":"2023-10-08T05:57:26.563092+0000","last_became_peered":"2023-10-08T05:57:26.563092+0000","last_unstale":"2023-10-08T05:57:27.939882+0000","last_undegraded":"2023-10-08T05:57:27.939882+0000","last_fullsized":"2023-10-08T05:57:27.939882+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:56:59.163319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.560143+0000","last_change":"2023-10-08T05:57:26.556474+0000","last_active":"2023-10-08T05:57:27.560143+0000","last_peered":"2023-10-08T05:57:27.560143+0000","last_clean":"2023-10-08T05:57:27.560143+0000","last_became_active":"2023-10-08T05:57:26.555683+0000","last_became_peered":"2023-10-08T05:57:26.555683+0000","last_unstale":"2023-10-08T05:57:27.560143+0000","last_undegraded":"2023-10-08T05:57:27.560143+0000","last_fullsized":"2023-10-08T05:57:27.560143+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:16.910093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.558517+0000","last_change":"2023-10-08T05:57:26.560999+0000","last_active":"2023-10-08T05:57:27.558517+0000","last_peered":"2023-10-08T05:57:27.558517+0000","last_clean":"2023-10-08T05:57:27.558517+0000","last_became_active":"2023-10-08T05:57:26.560723+0000","last_became_peered":"2023-10-08T05:57:26.560723+0000","last_unstale":"2023-10-08T05:57:27.558517+0000","last_undegraded":"2023-10-08T05:57:27.558517+0000","last_fullsized":"2023-10-08T05:57:27.558517+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:28.898067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940520+0000","last_change":"2023-10-08T05:57:26.569280+0000","last_active":"2023-10-08T05:57:27.940520+0000","last_peered":"2023-10-08T05:57:27.940520+0000","last_clean":"2023-10-08T05:57:27.940520+0000","last_became_active":"2023-10-08T05:57:26.567908+0000","last_became_peered":"2023-10-08T05:57:26.567908+0000","last_unstale":"2023-10-08T05:57:27.940520+0000","last_undegraded":"2023-10-08T05:57:27.940520+0000","last_fullsized":"2023-10-08T05:57:27.940520+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:47:23.939688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.941507+0000","last_change":"2023-10-08T05:57:26.557888+0000","last_active":"2023-10-08T05:57:27.941507+0000","last_peered":"2023-10-08T05:57:27.941507+0000","last_clean":"2023-10-08T05:57:27.941507+0000","last_became_active":"2023-10-08T05:57:26.556791+0000","last_became_peered":"2023-10-08T05:57:26.556791+0000","last_unstale":"2023-10-08T05:57:27.941507+0000","last_undegraded":"2023-10-08T05:57:27.941507+0000","last_fullsized":"2023-10-08T05:57:27.941507+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:37.571709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.558741+0000","last_change":"2023-10-08T05:57:26.565829+0000","last_active":"2023-10-08T05:57:27.558741+0000","last_peered":"2023-10-08T05:57:27.558741+0000","last_clean":"2023-10-08T05:57:27.558741+0000","last_became_active":"2023-10-08T05:57:26.565394+0000","last_became_peered":"2023-10-08T05:57:26.565394+0000","last_unstale":"2023-10-08T05:57:27.558741+0000","last_undegraded":"2023-10-08T05:57:27.558741+0000","last_fullsized":"2023-10-08T05:57:27.558741+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:48.672918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.558298+0000","last_change":"2023-10-08T05:57:26.554897+0000","last_active":"2023-10-08T05:57:27.558298+0000","last_peered":"2023-10-08T05:57:27.558298+0000","last_clean":"2023-10-08T05:57:27.558298+0000","last_became_active":"2023-10-08T05:57:26.554621+0000","last_became_peered":"2023-10-08T05:57:26.554621+0000","last_unstale":"2023-10-08T05:57:27.558298+0000","last_undegraded":"2023-10-08T05:57:27.558298+0000","last_fullsized":"2023-10-08T05:57:27.558298+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:48.441798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.557711+0000","last_change":"2023-10-08T05:57:26.565920+0000","last_active":"2023-10-08T05:57:27.557711+0000","last_peered":"2023-10-08T05:57:27.557711+0000","last_clean":"2023-10-08T05:57:27.557711+0000","last_became_active":"2023-10-08T05:57:26.565400+0000","last_became_peered":"2023-10-08T05:57:26.565400+0000","last_unstale":"2023-10-08T05:57:27.557711+0000","last_undegraded":"2023-10-08T05:57:27.557711+0000","last_fullsized":"2023-10-08T05:57:27.557711+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:25.805260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940654+0000","last_change":"2023-10-08T05:57:26.570093+0000","last_active":"2023-10-08T05:57:27.940654+0000","last_peered":"2023-10-08T05:57:27.940654+0000","last_clean":"2023-10-08T05:57:27.940654+0000","last_became_active":"2023-10-08T05:57:26.569801+0000","last_became_peered":"2023-10-08T05:57:26.569801+0000","last_unstale":"2023-10-08T05:57:27.940654+0000","last_undegraded":"2023-10-08T05:57:27.940654+0000","last_fullsized":"2023-10-08T05:57:27.940654+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:01:04.467557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.558477+0000","last_change":"2023-10-08T05:57:26.550512+0000","last_active":"2023-10-08T05:57:27.558477+0000","last_peered":"2023-10-08T05:57:27.558477+0000","last_clean":"2023-10-08T05:57:27.558477+0000","last_became_active":"2023-10-08T05:57:26.550198+0000","last_became_peered":"2023-10-08T05:57:26.550198+0000","last_unstale":"2023-10-08T05:57:27.558477+0000","last_undegraded":"2023-10-08T05:57:27.558477+0000","last_fullsized":"2023-10-08T05:57:27.558477+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:44.407007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"162'4","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.578880+0000","last_change":"2023-10-08T05:57:25.552962+0000","last_active":"2023-10-08T05:57:26.578880+0000","last_peered":"2023-10-08T05:57:26.578880+0000","last_clean":"2023-10-08T05:57:26.578880+0000","last_became_active":"2023-10-08T05:57:25.552677+0000","last_became_peered":"2023-10-08T05:57:25.552677+0000","last_unstale":"2023-10-08T05:57:26.578880+0000","last_undegraded":"2023-10-08T05:57:26.578880+0000","last_fullsized":"2023-10-08T05:57:26.578880+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:28.854446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.552977+0000","last_change":"2023-10-08T05:57:26.552013+0000","last_active":"2023-10-08T05:57:27.552977+0000","last_peered":"2023-10-08T05:57:27.552977+0000","last_clean":"2023-10-08T05:57:27.552977+0000","last_became_active":"2023-10-08T05:57:26.551718+0000","last_became_peered":"2023-10-08T05:57:26.551718+0000","last_unstale":"2023-10-08T05:57:27.552977+0000","last_undegraded":"2023-10-08T05:57:27.552977+0000","last_fullsized":"2023-10-08T05:57:27.552977+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:48.253474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.578954+0000","last_change":"2023-10-08T05:57:25.540708+0000","last_active":"2023-10-08T05:57:26.578954+0000","last_peered":"2023-10-08T05:57:26.578954+0000","last_clean":"2023-10-08T05:57:26.578954+0000","last_became_active":"2023-10-08T05:57:25.540301+0000","last_became_peered":"2023-10-08T05:57:25.540301+0000","last_unstale":"2023-10-08T05:57:26.578954+0000","last_undegraded":"2023-10-08T05:57:26.578954+0000","last_fullsized":"2023-10-08T05:57:26.578954+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:22.507663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.941618+0000","last_change":"2023-10-08T05:57:26.563772+0000","last_active":"2023-10-08T05:57:27.941618+0000","last_peered":"2023-10-08T05:57:27.941618+0000","last_clean":"2023-10-08T05:57:27.941618+0000","last_became_active":"2023-10-08T05:57:26.563475+0000","last_became_peered":"2023-10-08T05:57:26.563475+0000","last_unstale":"2023-10-08T05:57:27.941618+0000","last_undegraded":"2023-10-08T05:57:27.941618+0000","last_fullsized":"2023-10-08T05:57:27.941618+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:37.571709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.579404+0000","last_change":"2023-10-08T05:57:25.557364+0000","last_active":"2023-10-08T05:57:26.579404+0000","last_peered":"2023-10-08T05:57:26.579404+0000","last_clean":"2023-10-08T05:57:26.579404+0000","last_became_active":"2023-10-08T05:57:25.555480+0000","last_became_peered":"2023-10-08T05:57:25.555480+0000","last_unstale":"2023-10-08T05:57:26.579404+0000","last_undegraded":"2023-10-08T05:57:26.579404+0000","last_fullsized":"2023-10-08T05:57:26.579404+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:58.378676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.553169+0000","last_change":"2023-10-08T05:57:26.565523+0000","last_active":"2023-10-08T05:57:27.553169+0000","last_peered":"2023-10-08T05:57:27.553169+0000","last_clean":"2023-10-08T05:57:27.553169+0000","last_became_active":"2023-10-08T05:57:26.565238+0000","last_became_peered":"2023-10-08T05:57:26.565238+0000","last_unstale":"2023-10-08T05:57:27.553169+0000","last_undegraded":"2023-10-08T05:57:27.553169+0000","last_fullsized":"2023-10-08T05:57:27.553169+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:04.026754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.577212+0000","last_change":"2023-10-08T05:57:25.549520+0000","last_active":"2023-10-08T05:57:26.577212+0000","last_peered":"2023-10-08T05:57:26.577212+0000","last_clean":"2023-10-08T05:57:26.577212+0000","last_became_active":"2023-10-08T05:57:25.548746+0000","last_became_peered":"2023-10-08T05:57:25.548746+0000","last_unstale":"2023-10-08T05:57:26.577212+0000","last_undegraded":"2023-10-08T05:57:26.577212+0000","last_fullsized":"2023-10-08T05:57:26.577212+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:22.507663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.560351+0000","last_change":"2023-10-08T05:57:26.567855+0000","last_active":"2023-10-08T05:57:27.560351+0000","last_peered":"2023-10-08T05:57:27.560351+0000","last_clean":"2023-10-08T05:57:27.560351+0000","last_became_active":"2023-10-08T05:57:26.567546+0000","last_became_peered":"2023-10-08T05:57:26.567546+0000","last_unstale":"2023-10-08T05:57:27.560351+0000","last_undegraded":"2023-10-08T05:57:27.560351+0000","last_fullsized":"2023-10-08T05:57:27.560351+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:36:26.375694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.576998+0000","last_change":"2023-10-08T05:57:25.547635+0000","last_active":"2023-10-08T05:57:26.576998+0000","last_peered":"2023-10-08T05:57:26.576998+0000","last_clean":"2023-10-08T05:57:26.576998+0000","last_became_active":"2023-10-08T05:57:25.547185+0000","last_became_peered":"2023-10-08T05:57:25.547185+0000","last_unstale":"2023-10-08T05:57:26.576998+0000","last_undegraded":"2023-10-08T05:57:26.576998+0000","last_fullsized":"2023-10-08T05:57:26.576998+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:53:00.279036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.560456+0000","last_change":"2023-10-08T05:57:26.553360+0000","last_active":"2023-10-08T05:57:27.560456+0000","last_peered":"2023-10-08T05:57:27.560456+0000","last_clean":"2023-10-08T05:57:27.560456+0000","last_became_active":"2023-10-08T05:57:26.553002+0000","last_became_peered":"2023-10-08T05:57:26.553002+0000","last_unstale":"2023-10-08T05:57:27.560456+0000","last_undegraded":"2023-10-08T05:57:27.560456+0000","last_fullsized":"2023-10-08T05:57:27.560456+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:26.584229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.578791+0000","last_change":"2023-10-08T05:57:25.547003+0000","last_active":"2023-10-08T05:57:26.578791+0000","last_peered":"2023-10-08T05:57:26.578791+0000","last_clean":"2023-10-08T05:57:26.578791+0000","last_became_active":"2023-10-08T05:57:25.546756+0000","last_became_peered":"2023-10-08T05:57:25.546756+0000","last_unstale":"2023-10-08T05:57:26.578791+0000","last_undegraded":"2023-10-08T05:57:26.578791+0000","last_fullsized":"2023-10-08T05:57:26.578791+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:03:43.692669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.939191+0000","last_change":"2023-10-08T05:57:26.558909+0000","last_active":"2023-10-08T05:57:27.939191+0000","last_peered":"2023-10-08T05:57:27.939191+0000","last_clean":"2023-10-08T05:57:27.939191+0000","last_became_active":"2023-10-08T05:57:26.558646+0000","last_became_peered":"2023-10-08T05:57:26.558646+0000","last_unstale":"2023-10-08T05:57:27.939191+0000","last_undegraded":"2023-10-08T05:57:27.939191+0000","last_fullsized":"2023-10-08T05:57:27.939191+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:49:54.449762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.553250+0000","last_change":"2023-10-08T05:57:25.554379+0000","last_active":"2023-10-08T05:57:26.553250+0000","last_peered":"2023-10-08T05:57:26.553250+0000","last_clean":"2023-10-08T05:57:26.553250+0000","last_became_active":"2023-10-08T05:57:25.554018+0000","last_became_peered":"2023-10-08T05:57:25.554018+0000","last_unstale":"2023-10-08T05:57:26.553250+0000","last_undegraded":"2023-10-08T05:57:26.553250+0000","last_fullsized":"2023-10-08T05:57:26.553250+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:35:28.831220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.939851+0000","last_change":"2023-10-08T05:57:26.569383+0000","last_active":"2023-10-08T05:57:27.939851+0000","last_peered":"2023-10-08T05:57:27.939851+0000","last_clean":"2023-10-08T05:57:27.939851+0000","last_became_active":"2023-10-08T05:57:26.568887+0000","last_became_peered":"2023-10-08T05:57:26.568887+0000","last_unstale":"2023-10-08T05:57:27.939851+0000","last_undegraded":"2023-10-08T05:57:27.939851+0000","last_fullsized":"2023-10-08T05:57:27.939851+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:31:54.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.530933+0000","last_change":"2023-10-08T05:57:25.553851+0000","last_active":"2023-10-08T05:57:26.530933+0000","last_peered":"2023-10-08T05:57:26.530933+0000","last_clean":"2023-10-08T05:57:26.530933+0000","last_became_active":"2023-10-08T05:57:25.553482+0000","last_became_peered":"2023-10-08T05:57:25.553482+0000","last_unstale":"2023-10-08T05:57:26.530933+0000","last_undegraded":"2023-10-08T05:57:26.530933+0000","last_fullsized":"2023-10-08T05:57:26.530933+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:43.605316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.552749+0000","last_change":"2023-10-08T05:57:26.558198+0000","last_active":"2023-10-08T05:57:27.552749+0000","last_peered":"2023-10-08T05:57:27.552749+0000","last_clean":"2023-10-08T05:57:27.552749+0000","last_became_active":"2023-10-08T05:57:26.557901+0000","last_became_peered":"2023-10-08T05:57:26.557901+0000","last_unstale":"2023-10-08T05:57:27.552749+0000","last_undegraded":"2023-10-08T05:57:27.552749+0000","last_fullsized":"2023-10-08T05:57:27.552749+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:50.956457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.542885+0000","last_change":"2023-10-08T05:57:25.549805+0000","last_active":"2023-10-08T05:57:26.542885+0000","last_peered":"2023-10-08T05:57:26.542885+0000","last_clean":"2023-10-08T05:57:26.542885+0000","last_became_active":"2023-10-08T05:57:25.549525+0000","last_became_peered":"2023-10-08T05:57:25.549525+0000","last_unstale":"2023-10-08T05:57:26.542885+0000","last_undegraded":"2023-10-08T05:57:26.542885+0000","last_fullsized":"2023-10-08T05:57:26.542885+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:56:02.658363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.939838+0000","last_change":"2023-10-08T05:57:26.564985+0000","last_active":"2023-10-08T05:57:27.939838+0000","last_peered":"2023-10-08T05:57:27.939838+0000","last_clean":"2023-10-08T05:57:27.939838+0000","last_became_active":"2023-10-08T05:57:26.563185+0000","last_became_peered":"2023-10-08T05:57:26.563185+0000","last_unstale":"2023-10-08T05:57:27.939838+0000","last_undegraded":"2023-10-08T05:57:27.939838+0000","last_fullsized":"2023-10-08T05:57:27.939838+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:56.944215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.540857+0000","last_change":"2023-10-08T05:57:25.544343+0000","last_active":"2023-10-08T05:57:26.540857+0000","last_peered":"2023-10-08T05:57:26.540857+0000","last_clean":"2023-10-08T05:57:26.540857+0000","last_became_active":"2023-10-08T05:57:25.544065+0000","last_became_peered":"2023-10-08T05:57:25.544065+0000","last_unstale":"2023-10-08T05:57:26.540857+0000","last_undegraded":"2023-10-08T05:57:26.540857+0000","last_fullsized":"2023-10-08T05:57:26.540857+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:57.866209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.558709+0000","last_change":"2023-10-08T05:57:26.549901+0000","last_active":"2023-10-08T05:57:27.558709+0000","last_peered":"2023-10-08T05:57:27.558709+0000","last_clean":"2023-10-08T05:57:27.558709+0000","last_became_active":"2023-10-08T05:57:26.549019+0000","last_became_peered":"2023-10-08T05:57:26.549019+0000","last_unstale":"2023-10-08T05:57:27.558709+0000","last_undegraded":"2023-10-08T05:57:27.558709+0000","last_fullsized":"2023-10-08T05:57:27.558709+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:07:43.345140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"162'3","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.579688+0000","last_change":"2023-10-08T05:57:25.554217+0000","last_active":"2023-10-08T05:57:26.579688+0000","last_peered":"2023-10-08T05:57:26.579688+0000","last_clean":"2023-10-08T05:57:26.579688+0000","last_became_active":"2023-10-08T05:57:25.553572+0000","last_became_peered":"2023-10-08T05:57:25.553572+0000","last_unstale":"2023-10-08T05:57:26.579688+0000","last_undegraded":"2023-10-08T05:57:26.579688+0000","last_fullsized":"2023-10-08T05:57:26.579688+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:58.752104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.559371+0000","last_change":"2023-10-08T05:57:26.547725+0000","last_active":"2023-10-08T05:57:27.559371+0000","last_peered":"2023-10-08T05:57:27.559371+0000","last_clean":"2023-10-08T05:57:27.559371+0000","last_became_active":"2023-10-08T05:57:26.539727+0000","last_became_peered":"2023-10-08T05:57:26.539727+0000","last_unstale":"2023-10-08T05:57:27.559371+0000","last_undegraded":"2023-10-08T05:57:27.559371+0000","last_fullsized":"2023-10-08T05:57:27.559371+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:52.597221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"162'6","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.583208+0000","last_change":"2023-10-08T05:57:25.549177+0000","last_active":"2023-10-08T05:57:26.583208+0000","last_peered":"2023-10-08T05:57:26.583208+0000","last_clean":"2023-10-08T05:57:26.583208+0000","last_became_active":"2023-10-08T05:57:25.548827+0000","last_became_peered":"2023-10-08T05:57:25.548827+0000","last_unstale":"2023-10-08T05:57:26.583208+0000","last_undegraded":"2023-10-08T05:57:26.583208+0000","last_fullsized":"2023-10-08T05:57:26.583208+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:04:36.557039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.941803+0000","last_change":"2023-10-08T05:57:26.551774+0000","last_active":"2023-10-08T05:57:27.941803+0000","last_peered":"2023-10-08T05:57:27.941803+0000","last_clean":"2023-10-08T05:57:27.941803+0000","last_became_active":"2023-10-08T05:57:26.551502+0000","last_became_peered":"2023-10-08T05:57:26.551502+0000","last_unstale":"2023-10-08T05:57:27.941803+0000","last_undegraded":"2023-10-08T05:57:27.941803+0000","last_fullsized":"2023-10-08T05:57:27.941803+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:31:54.977916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.577415+0000","last_change":"2023-10-08T05:57:25.539867+0000","last_active":"2023-10-08T05:57:26.577415+0000","last_peered":"2023-10-08T05:57:26.577415+0000","last_clean":"2023-10-08T05:57:26.577415+0000","last_became_active":"2023-10-08T05:57:25.539587+0000","last_became_peered":"2023-10-08T05:57:25.539587+0000","last_unstale":"2023-10-08T05:57:26.577415+0000","last_undegraded":"2023-10-08T05:57:26.577415+0000","last_fullsized":"2023-10-08T05:57:26.577415+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:03:50.606311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.941780+0000","last_change":"2023-10-08T05:57:26.561888+0000","last_active":"2023-10-08T05:57:27.941780+0000","last_peered":"2023-10-08T05:57:27.941780+0000","last_clean":"2023-10-08T05:57:27.941780+0000","last_became_active":"2023-10-08T05:57:26.561578+0000","last_became_peered":"2023-10-08T05:57:26.561578+0000","last_unstale":"2023-10-08T05:57:27.941780+0000","last_undegraded":"2023-10-08T05:57:27.941780+0000","last_fullsized":"2023-10-08T05:57:27.941780+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:35.162739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.534511+0000","last_change":"2023-10-08T05:57:25.550848+0000","last_active":"2023-10-08T05:57:26.534511+0000","last_peered":"2023-10-08T05:57:26.534511+0000","last_clean":"2023-10-08T05:57:26.534511+0000","last_became_active":"2023-10-08T05:57:25.550450+0000","last_became_peered":"2023-10-08T05:57:25.550450+0000","last_unstale":"2023-10-08T05:57:26.534511+0000","last_undegraded":"2023-10-08T05:57:26.534511+0000","last_fullsized":"2023-10-08T05:57:26.534511+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:05.151523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.939439+0000","last_change":"2023-10-08T05:57:26.565768+0000","last_active":"2023-10-08T05:57:27.939439+0000","last_peered":"2023-10-08T05:57:27.939439+0000","last_clean":"2023-10-08T05:57:27.939439+0000","last_became_active":"2023-10-08T05:57:26.565443+0000","last_became_peered":"2023-10-08T05:57:26.565443+0000","last_unstale":"2023-10-08T05:57:27.939439+0000","last_undegraded":"2023-10-08T05:57:27.939439+0000","last_fullsized":"2023-10-08T05:57:27.939439+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:09.824252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"162'8","reported_seq":20,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.583937+0000","last_change":"2023-10-08T05:57:25.555732+0000","last_active":"2023-10-08T05:57:26.583937+0000","last_peered":"2023-10-08T05:57:26.583937+0000","last_clean":"2023-10-08T05:57:26.583937+0000","last_became_active":"2023-10-08T05:57:25.555420+0000","last_became_peered":"2023-10-08T05:57:25.555420+0000","last_unstale":"2023-10-08T05:57:26.583937+0000","last_undegraded":"2023-10-08T05:57:26.583937+0000","last_fullsized":"2023-10-08T05:57:26.583937+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:07.656801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.557814+0000","last_change":"2023-10-08T05:57:26.557212+0000","last_active":"2023-10-08T05:57:27.557814+0000","last_peered":"2023-10-08T05:57:27.557814+0000","last_clean":"2023-10-08T05:57:27.557814+0000","last_became_active":"2023-10-08T05:57:26.556864+0000","last_became_peered":"2023-10-08T05:57:26.556864+0000","last_unstale":"2023-10-08T05:57:27.557814+0000","last_undegraded":"2023-10-08T05:57:27.557814+0000","last_fullsized":"2023-10-08T05:57:27.557814+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:44.694050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.576398+0000","last_change":"2023-10-08T05:57:25.548150+0000","last_active":"2023-10-08T05:57:26.576398+0000","last_peered":"2023-10-08T05:57:26.576398+0000","last_clean":"2023-10-08T05:57:26.576398+0000","last_became_active":"2023-10-08T05:57:25.547818+0000","last_became_peered":"2023-10-08T05:57:25.547818+0000","last_unstale":"2023-10-08T05:57:26.576398+0000","last_undegraded":"2023-10-08T05:57:26.576398+0000","last_fullsized":"2023-10-08T05:57:26.576398+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:10.344093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940315+0000","last_change":"2023-10-08T05:57:26.566267+0000","last_active":"2023-10-08T05:57:27.940315+0000","last_peered":"2023-10-08T05:57:27.940315+0000","last_clean":"2023-10-08T05:57:27.940315+0000","last_became_active":"2023-10-08T05:57:26.565996+0000","last_became_peered":"2023-10-08T05:57:26.565996+0000","last_unstale":"2023-10-08T05:57:27.940315+0000","last_undegraded":"2023-10-08T05:57:27.940315+0000","last_fullsized":"2023-10-08T05:57:27.940315+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:09.824252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.578876+0000","last_change":"2023-10-08T05:57:25.550760+0000","last_active":"2023-10-08T05:57:26.578876+0000","last_peered":"2023-10-08T05:57:26.578876+0000","last_clean":"2023-10-08T05:57:26.578876+0000","last_became_active":"2023-10-08T05:57:25.550423+0000","last_became_peered":"2023-10-08T05:57:25.550423+0000","last_unstale":"2023-10-08T05:57:26.578876+0000","last_undegraded":"2023-10-08T05:57:26.578876+0000","last_fullsized":"2023-10-08T05:57:26.578876+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:58:50.512202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940880+0000","last_change":"2023-10-08T05:57:26.569109+0000","last_active":"2023-10-08T05:57:27.940880+0000","last_peered":"2023-10-08T05:57:27.940880+0000","last_clean":"2023-10-08T05:57:27.940880+0000","last_became_active":"2023-10-08T05:57:26.568772+0000","last_became_peered":"2023-10-08T05:57:26.568772+0000","last_unstale":"2023-10-08T05:57:27.940880+0000","last_undegraded":"2023-10-08T05:57:27.940880+0000","last_fullsized":"2023-10-08T05:57:27.940880+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:28.824442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.578599+0000","last_change":"2023-10-08T05:57:25.549191+0000","last_active":"2023-10-08T05:57:26.578599+0000","last_peered":"2023-10-08T05:57:26.578599+0000","last_clean":"2023-10-08T05:57:26.578599+0000","last_became_active":"2023-10-08T05:57:25.548627+0000","last_became_peered":"2023-10-08T05:57:25.548627+0000","last_unstale":"2023-10-08T05:57:26.578599+0000","last_undegraded":"2023-10-08T05:57:26.578599+0000","last_fullsized":"2023-10-08T05:57:26.578599+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:02:35.513243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.941670+0000","last_change":"2023-10-08T05:57:26.545244+0000","last_active":"2023-10-08T05:57:27.941670+0000","last_peered":"2023-10-08T05:57:27.941670+0000","last_clean":"2023-10-08T05:57:27.941670+0000","last_became_active":"2023-10-08T05:57:26.544629+0000","last_became_peered":"2023-10-08T05:57:26.544629+0000","last_unstale":"2023-10-08T05:57:27.941670+0000","last_undegraded":"2023-10-08T05:57:27.941670+0000","last_fullsized":"2023-10-08T05:57:27.941670+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:27:57.346585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.539571+0000","last_change":"2023-10-08T05:57:25.533519+0000","last_active":"2023-10-08T05:57:26.539571+0000","last_peered":"2023-10-08T05:57:26.539571+0000","last_clean":"2023-10-08T05:57:26.539571+0000","last_became_active":"2023-10-08T05:57:25.533218+0000","last_became_peered":"2023-10-08T05:57:25.533218+0000","last_unstale":"2023-10-08T05:57:26.539571+0000","last_undegraded":"2023-10-08T05:57:26.539571+0000","last_fullsized":"2023-10-08T05:57:26.539571+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:08.009555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.938915+0000","last_change":"2023-10-08T05:57:26.557742+0000","last_active":"2023-10-08T05:57:27.938915+0000","last_peered":"2023-10-08T05:57:27.938915+0000","last_clean":"2023-10-08T05:57:27.938915+0000","last_became_active":"2023-10-08T05:57:26.557456+0000","last_became_peered":"2023-10-08T05:57:26.557456+0000","last_unstale":"2023-10-08T05:57:27.938915+0000","last_undegraded":"2023-10-08T05:57:27.938915+0000","last_fullsized":"2023-10-08T05:57:27.938915+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:51:17.511120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.579107+0000","last_change":"2023-10-08T05:57:25.551666+0000","last_active":"2023-10-08T05:57:26.579107+0000","last_peered":"2023-10-08T05:57:26.579107+0000","last_clean":"2023-10-08T05:57:26.579107+0000","last_became_active":"2023-10-08T05:57:25.551366+0000","last_became_peered":"2023-10-08T05:57:25.551366+0000","last_unstale":"2023-10-08T05:57:26.579107+0000","last_undegraded":"2023-10-08T05:57:26.579107+0000","last_fullsized":"2023-10-08T05:57:26.579107+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:45.166913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.553646+0000","last_change":"2023-10-08T05:57:26.568091+0000","last_active":"2023-10-08T05:57:27.553646+0000","last_peered":"2023-10-08T05:57:27.553646+0000","last_clean":"2023-10-08T05:57:27.553646+0000","last_became_active":"2023-10-08T05:57:26.567720+0000","last_became_peered":"2023-10-08T05:57:26.567720+0000","last_unstale":"2023-10-08T05:57:27.553646+0000","last_undegraded":"2023-10-08T05:57:27.553646+0000","last_fullsized":"2023-10-08T05:57:27.553646+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:01.371848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.545155+0000","last_change":"2023-10-08T05:57:25.547967+0000","last_active":"2023-10-08T05:57:26.545155+0000","last_peered":"2023-10-08T05:57:26.545155+0000","last_clean":"2023-10-08T05:57:26.545155+0000","last_became_active":"2023-10-08T05:57:25.547668+0000","last_became_peered":"2023-10-08T05:57:25.547668+0000","last_unstale":"2023-10-08T05:57:26.545155+0000","last_undegraded":"2023-10-08T05:57:26.545155+0000","last_fullsized":"2023-10-08T05:57:26.545155+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:11:20.876418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.939534+0000","last_change":"2023-10-08T05:57:26.555735+0000","last_active":"2023-10-08T05:57:27.939534+0000","last_peered":"2023-10-08T05:57:27.939534+0000","last_clean":"2023-10-08T05:57:27.939534+0000","last_became_active":"2023-10-08T05:57:26.555463+0000","last_became_peered":"2023-10-08T05:57:26.555463+0000","last_unstale":"2023-10-08T05:57:27.939534+0000","last_undegraded":"2023-10-08T05:57:27.939534+0000","last_fullsized":"2023-10-08T05:57:27.939534+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:47:23.939688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.542816+0000","last_change":"2023-10-08T05:57:25.553740+0000","last_active":"2023-10-08T05:57:26.542816+0000","last_peered":"2023-10-08T05:57:26.542816+0000","last_clean":"2023-10-08T05:57:26.542816+0000","last_became_active":"2023-10-08T05:57:25.553413+0000","last_became_peered":"2023-10-08T05:57:25.553413+0000","last_unstale":"2023-10-08T05:57:26.542816+0000","last_undegraded":"2023-10-08T05:57:26.542816+0000","last_fullsized":"2023-10-08T05:57:26.542816+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:46:59.888125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.941881+0000","last_change":"2023-10-08T05:57:26.559521+0000","last_active":"2023-10-08T05:57:27.941881+0000","last_peered":"2023-10-08T05:57:27.941881+0000","last_clean":"2023-10-08T05:57:27.941881+0000","last_became_active":"2023-10-08T05:57:26.558635+0000","last_became_peered":"2023-10-08T05:57:26.558635+0000","last_unstale":"2023-10-08T05:57:27.941881+0000","last_undegraded":"2023-10-08T05:57:27.941881+0000","last_fullsized":"2023-10-08T05:57:27.941881+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:27:57.346585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.545789+0000","last_change":"2023-10-08T05:57:25.551912+0000","last_active":"2023-10-08T05:57:26.545789+0000","last_peered":"2023-10-08T05:57:26.545789+0000","last_clean":"2023-10-08T05:57:26.545789+0000","last_became_active":"2023-10-08T05:57:25.551448+0000","last_became_peered":"2023-10-08T05:57:25.551448+0000","last_unstale":"2023-10-08T05:57:26.545789+0000","last_undegraded":"2023-10-08T05:57:26.545789+0000","last_fullsized":"2023-10-08T05:57:26.545789+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:43.392338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.938356+0000","last_change":"2023-10-08T05:57:26.558263+0000","last_active":"2023-10-08T05:57:27.938356+0000","last_peered":"2023-10-08T05:57:27.938356+0000","last_clean":"2023-10-08T05:57:27.938356+0000","last_became_active":"2023-10-08T05:57:26.557980+0000","last_became_peered":"2023-10-08T05:57:26.557980+0000","last_unstale":"2023-10-08T05:57:27.938356+0000","last_undegraded":"2023-10-08T05:57:27.938356+0000","last_fullsized":"2023-10-08T05:57:27.938356+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:27:57.346585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"162'2","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.579662+0000","last_change":"2023-10-08T05:57:25.544149+0000","last_active":"2023-10-08T05:57:26.579662+0000","last_peered":"2023-10-08T05:57:26.579662+0000","last_clean":"2023-10-08T05:57:26.579662+0000","last_became_active":"2023-10-08T05:57:25.543783+0000","last_became_peered":"2023-10-08T05:57:25.543783+0000","last_unstale":"2023-10-08T05:57:26.579662+0000","last_undegraded":"2023-10-08T05:57:26.579662+0000","last_fullsized":"2023-10-08T05:57:26.579662+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:05.933532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.558481+0000","last_change":"2023-10-08T05:57:26.543978+0000","last_active":"2023-10-08T05:57:27.558481+0000","last_peered":"2023-10-08T05:57:27.558481+0000","last_clean":"2023-10-08T05:57:27.558481+0000","last_became_active":"2023-10-08T05:57:26.543615+0000","last_became_peered":"2023-10-08T05:57:26.543615+0000","last_unstale":"2023-10-08T05:57:27.558481+0000","last_undegraded":"2023-10-08T05:57:27.558481+0000","last_fullsized":"2023-10-08T05:57:27.558481+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:18.169218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"162'1","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.469360+0000","last_change":"2023-10-08T05:57:25.546481+0000","last_active":"2023-10-08T05:57:27.469360+0000","last_peered":"2023-10-08T05:57:27.469360+0000","last_clean":"2023-10-08T05:57:27.469360+0000","last_became_active":"2023-10-08T05:57:25.545928+0000","last_became_peered":"2023-10-08T05:57:25.545928+0000","last_unstale":"2023-10-08T05:57:27.469360+0000","last_undegraded":"2023-10-08T05:57:27.469360+0000","last_fullsized":"2023-10-08T05:57:27.469360+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:46.136294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940132+0000","last_change":"2023-10-08T05:57:26.561776+0000","last_active":"2023-10-08T05:57:27.940132+0000","last_peered":"2023-10-08T05:57:27.940132+0000","last_clean":"2023-10-08T05:57:27.940132+0000","last_became_active":"2023-10-08T05:57:26.561185+0000","last_became_peered":"2023-10-08T05:57:26.561185+0000","last_unstale":"2023-10-08T05:57:27.940132+0000","last_undegraded":"2023-10-08T05:57:27.940132+0000","last_fullsized":"2023-10-08T05:57:27.940132+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:03:40.726156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.536315+0000","last_change":"2023-10-08T05:57:25.553022+0000","last_active":"2023-10-08T05:57:26.536315+0000","last_peered":"2023-10-08T05:57:26.536315+0000","last_clean":"2023-10-08T05:57:26.536315+0000","last_became_active":"2023-10-08T05:57:25.552465+0000","last_became_peered":"2023-10-08T05:57:25.552465+0000","last_unstale":"2023-10-08T05:57:26.536315+0000","last_undegraded":"2023-10-08T05:57:26.536315+0000","last_fullsized":"2023-10-08T05:57:26.536315+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:32:56.176895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.941502+0000","last_change":"2023-10-08T05:57:26.557689+0000","last_active":"2023-10-08T05:57:27.941502+0000","last_peered":"2023-10-08T05:57:27.941502+0000","last_clean":"2023-10-08T05:57:27.941502+0000","last_became_active":"2023-10-08T05:57:26.557367+0000","last_became_peered":"2023-10-08T05:57:26.557367+0000","last_unstale":"2023-10-08T05:57:27.941502+0000","last_undegraded":"2023-10-08T05:57:27.941502+0000","last_fullsized":"2023-10-08T05:57:27.941502+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:23.522332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.552799+0000","last_change":"2023-10-08T05:57:25.551237+0000","last_active":"2023-10-08T05:57:26.552799+0000","last_peered":"2023-10-08T05:57:26.552799+0000","last_clean":"2023-10-08T05:57:26.552799+0000","last_became_active":"2023-10-08T05:57:25.550905+0000","last_became_peered":"2023-10-08T05:57:25.550905+0000","last_unstale":"2023-10-08T05:57:26.552799+0000","last_undegraded":"2023-10-08T05:57:26.552799+0000","last_fullsized":"2023-10-08T05:57:26.552799+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:53:48.429581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940516+0000","last_change":"2023-10-08T05:57:26.564621+0000","last_active":"2023-10-08T05:57:27.940516+0000","last_peered":"2023-10-08T05:57:27.940516+0000","last_clean":"2023-10-08T05:57:27.940516+0000","last_became_active":"2023-10-08T05:57:26.562989+0000","last_became_peered":"2023-10-08T05:57:26.562989+0000","last_unstale":"2023-10-08T05:57:27.940516+0000","last_undegraded":"2023-10-08T05:57:27.940516+0000","last_fullsized":"2023-10-08T05:57:27.940516+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:22.013714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.541652+0000","last_change":"2023-10-08T05:57:25.549405+0000","last_active":"2023-10-08T05:57:26.541652+0000","last_peered":"2023-10-08T05:57:26.541652+0000","last_clean":"2023-10-08T05:57:26.541652+0000","last_became_active":"2023-10-08T05:57:25.549123+0000","last_became_peered":"2023-10-08T05:57:25.549123+0000","last_unstale":"2023-10-08T05:57:26.541652+0000","last_undegraded":"2023-10-08T05:57:26.541652+0000","last_fullsized":"2023-10-08T05:57:26.541652+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:12.487894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940703+0000","last_change":"2023-10-08T05:57:26.566648+0000","last_active":"2023-10-08T05:57:27.940703+0000","last_peered":"2023-10-08T05:57:27.940703+0000","last_clean":"2023-10-08T05:57:27.940703+0000","last_became_active":"2023-10-08T05:57:26.566361+0000","last_became_peered":"2023-10-08T05:57:26.566361+0000","last_unstale":"2023-10-08T05:57:27.940703+0000","last_undegraded":"2023-10-08T05:57:27.940703+0000","last_fullsized":"2023-10-08T05:57:27.940703+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:56:43.264375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.548496+0000","last_change":"2023-10-08T05:57:25.548597+0000","last_active":"2023-10-08T05:57:26.548496+0000","last_peered":"2023-10-08T05:57:26.548496+0000","last_clean":"2023-10-08T05:57:26.548496+0000","last_became_active":"2023-10-08T05:57:25.548261+0000","last_became_peered":"2023-10-08T05:57:25.548261+0000","last_unstale":"2023-10-08T05:57:26.548496+0000","last_undegraded":"2023-10-08T05:57:26.548496+0000","last_fullsized":"2023-10-08T05:57:26.548496+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:08.009555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940789+0000","last_change":"2023-10-08T05:57:26.570259+0000","last_active":"2023-10-08T05:57:27.940789+0000","last_peered":"2023-10-08T05:57:27.940789+0000","last_clean":"2023-10-08T05:57:27.940789+0000","last_became_active":"2023-10-08T05:57:26.569932+0000","last_became_peered":"2023-10-08T05:57:26.569932+0000","last_unstale":"2023-10-08T05:57:27.940789+0000","last_undegraded":"2023-10-08T05:57:27.940789+0000","last_fullsized":"2023-10-08T05:57:27.940789+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:19.373305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.553547+0000","last_change":"2023-10-08T05:57:25.550023+0000","last_active":"2023-10-08T05:57:26.553547+0000","last_peered":"2023-10-08T05:57:26.553547+0000","last_clean":"2023-10-08T05:57:26.553547+0000","last_became_active":"2023-10-08T05:57:25.549752+0000","last_became_peered":"2023-10-08T05:57:25.549752+0000","last_unstale":"2023-10-08T05:57:26.553547+0000","last_undegraded":"2023-10-08T05:57:26.553547+0000","last_fullsized":"2023-10-08T05:57:26.553547+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:53.359506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.940039+0000","last_change":"2023-10-08T05:57:26.557477+0000","last_active":"2023-10-08T05:57:27.940039+0000","last_peered":"2023-10-08T05:57:27.940039+0000","last_clean":"2023-10-08T05:57:27.940039+0000","last_became_active":"2023-10-08T05:57:26.557196+0000","last_became_peered":"2023-10-08T05:57:26.557196+0000","last_unstale":"2023-10-08T05:57:27.940039+0000","last_undegraded":"2023-10-08T05:57:27.940039+0000","last_fullsized":"2023-10-08T05:57:27.940039+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:51:17.511120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.535597+0000","last_change":"2023-10-08T05:57:25.550165+0000","last_active":"2023-10-08T05:57:26.535597+0000","last_peered":"2023-10-08T05:57:26.535597+0000","last_clean":"2023-10-08T05:57:26.535597+0000","last_became_active":"2023-10-08T05:57:25.549904+0000","last_became_peered":"2023-10-08T05:57:25.549904+0000","last_unstale":"2023-10-08T05:57:26.535597+0000","last_undegraded":"2023-10-08T05:57:26.535597+0000","last_fullsized":"2023-10-08T05:57:26.535597+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:58.378676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":11,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-10-08T05:57:27.559428+0000","last_change":"2023-10-08T05:57:26.558710+0000","last_active":"2023-10-08T05:57:27.559428+0000","last_peered":"2023-10-08T05:57:27.559428+0000","last_clean":"2023-10-08T05:57:27.559428+0000","last_became_active":"2023-10-08T05:57:26.557735+0000","last_became_peered":"2023-10-08T05:57:26.557735+0000","last_unstale":"2023-10-08T05:57:27.559428+0000","last_undegraded":"2023-10-08T05:57:27.559428+0000","last_fullsized":"2023-10-08T05:57:27.559428+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:25.498876+0000","last_clean_scrub_stamp":"2023-10-08T05:57:25.498876+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:30.288019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.547905+0000","last_change":"2023-10-08T05:57:25.554425+0000","last_active":"2023-10-08T05:57:26.547905+0000","last_peered":"2023-10-08T05:57:26.547905+0000","last_clean":"2023-10-08T05:57:26.547905+0000","last_became_active":"2023-10-08T05:57:25.554162+0000","last_became_peered":"2023-10-08T05:57:25.554162+0000","last_unstale":"2023-10-08T05:57:26.547905+0000","last_undegraded":"2023-10-08T05:57:26.547905+0000","last_fullsized":"2023-10-08T05:57:26.547905+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:57:24.484206+0000","last_clean_scrub_stamp":"2023-10-08T05:57:24.484206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:03:43.692669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"151'2707","reported_seq":3271,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-10-08T05:57:26.545741+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T05:57:26.545741+0000","last_peered":"2023-10-08T05:57:26.545741+0000","last_clean":"2023-10-08T05:57:26.545741+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T05:57:26.545741+0000","last_undegraded":"2023-10-08T05:57:26.545741+0000","last_fullsized":"2023-10-08T05:57:26.545741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2707,"log_dups_size":0,"ondisk_log_size":2707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1341,"num_read_kb":2764,"num_write":409,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1341,"num_read_kb":2764,"num_write":409,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2707,"ondisk_log_size":2707,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575101,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55968,"kb_used_data":2580,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315872,"statfs":{"total":96636764160,"available":96579452928,"internally_reserved":0,"allocated":2641920,"data_stored":1630261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.113,"15min":1.1890000000000001},"min":{"1min":0.69099999999999995,"5min":0.46899999999999997,"15min":0.36199999999999999},"max":{"1min":1.579,"5min":1.728,"15min":1.8859999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1679999999999999,"15min":1.2010000000000001},"min":{"1min":0.65100000000000002,"5min":0.53100000000000003,"15min":0.44},"max":{"1min":1.585,"5min":1.794,"15min":1.794},"last":1.1830000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.208,"15min":1.234},"min":{"1min":0.52000000000000002,"5min":0.51100000000000001,"15min":0.49199999999999999},"max":{"1min":1.613,"5min":1.8140000000000001,"15min":1.8360000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.28,"15min":1.276},"min":{"1min":0.78500000000000003,"5min":0.71599999999999997,"15min":0.47099999999999997},"max":{"1min":1.645,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.0489999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.1970000000000001,"15min":1.236},"min":{"1min":0.89300000000000002,"5min":0.71599999999999997,"15min":0.626},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.8939999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.2809999999999999,"15min":1.3149999999999999},"min":{"1min":0.72599999999999998,"5min":0.58799999999999997,"15min":0.45400000000000001},"max":{"1min":1.472,"5min":1.97,"15min":1.97},"last":1.3169999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2609999999999999,"15min":1.282},"min":{"1min":0.99099999999999999,"5min":0.72399999999999998,"15min":0.69999999999999996},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.8799999999999999},"last":1.101},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3580000000000001,"15min":1.363},"min":{"1min":0.82299999999999995,"5min":0.63,"15min":0.60199999999999998},"max":{"1min":1.5740000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.4950000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3480000000000001,"15min":1.3},"min":{"1min":0.81899999999999995,"5min":0.76200000000000001,"15min":0.504},"max":{"1min":1.712,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3700000000000001,"15min":1.3380000000000001},"min":{"1min":0.98299999999999998,"5min":0.81599999999999995,"15min":0.63600000000000001},"max":{"1min":1.7390000000000001,"5min":2.0550000000000002,"15min":2.0699999999999998},"last":1.2270000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4419999999999999,"15min":1.411},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.53400000000000003},"max":{"1min":1.6819999999999999,"5min":1.9810000000000001,"15min":2.0339999999999998},"last":1.573},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.444,"15min":1.4179999999999999},"min":{"1min":1.052,"5min":0.93600000000000005,"15min":0.39000000000000001},"max":{"1min":1.698,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.3879999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.552,"15min":1.5229999999999999},"min":{"1min":1.1659999999999999,"5min":1.0940000000000001,"15min":0.54400000000000004},"max":{"1min":1.748,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.4870000000000001,"5min":1.498,"15min":1.47},"min":{"1min":1.087,"5min":0.70999999999999996,"15min":0.57799999999999996},"max":{"1min":1.9419999999999999,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.452}]}]},{"osd":6,"up_from":13,"seq":55834575100,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37340,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334500,"statfs":{"total":96636764160,"available":96598528000,"internally_reserved":0,"allocated":2039808,"data_stored":1038863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.1799999999999999,"15min":1.2589999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.8839999999999999,"5min":1.9119999999999999,"15min":2.1429999999999998},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.2769999999999999,"15min":1.2989999999999999},"min":{"1min":0.73599999999999999,"5min":0.63800000000000001,"15min":0.58399999999999996},"max":{"1min":1.599,"5min":1.8100000000000001,"15min":2.3130000000000002},"last":1.3140000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.1439999999999999,"15min":1.159},"min":{"1min":0.67500000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.5229999999999999,"5min":1.7430000000000001,"15min":1.7789999999999999},"last":1.423},{"interface":"front","average":{"1min":1.085,"5min":1.1739999999999999,"15min":1.177},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.466,"5min":1.8169999999999999,"15min":2.0510000000000002},"last":1.369}]},{"osd":2,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1739999999999999,"15min":1.1719999999999999},"min":{"1min":0.66000000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.054},{"interface":"front","average":{"1min":1.218,"5min":1.194,"15min":1.2390000000000001},"min":{"1min":0.77900000000000003,"5min":0.72899999999999998,"15min":0.58699999999999997},"max":{"1min":1.6299999999999999,"5min":1.661,"15min":1.8520000000000001},"last":0.82699999999999996}]},{"osd":3,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.288,"15min":1.3500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6359999999999999,"5min":1.8420000000000001,"15min":2.137},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4039999999999999,"15min":1.4330000000000001},"min":{"1min":0.98299999999999998,"5min":0.81399999999999995,"15min":0.70499999999999996},"max":{"1min":2.02,"5min":2.024,"15min":2.1930000000000001},"last":1.5329999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.339,"15min":1.319},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.60099999999999998},"max":{"1min":1.778,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.427,"15min":1.3859999999999999},"min":{"1min":0.94699999999999995,"5min":0.72499999999999998,"15min":0.64300000000000002},"max":{"1min":1.929,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.2070000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.345,"15min":1.3260000000000001},"min":{"1min":1.0189999999999999,"5min":0.83599999999999997,"15min":0.69999999999999996},"max":{"1min":1.639,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.268},{"interface":"front","average":{"1min":1.431,"5min":1.4970000000000001,"15min":1.482},"min":{"1min":0.88,"5min":0.87,"15min":0.747},"max":{"1min":1.784,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.776}]},{"osd":7,"last update":"Sun Oct 8 05:56:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5609999999999999,"5min":1.556,"15min":1.506},"min":{"1min":1.1599999999999999,"5min":0.82399999999999995,"15min":0.77200000000000002},"max":{"1min":1.9590000000000001,"5min":2.6070000000000002,"15min":2.6070000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.4930000000000001,"15min":1.474},"min":{"1min":0.93899999999999995,"5min":0.85999999999999999,"15min":0.81899999999999995},"max":{"1min":1.9410000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.5600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575100,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51216,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320624,"statfs":{"total":96636764160,"available":96584318976,"internally_reserved":0,"allocated":2035712,"data_stored":1038213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.161,"15min":1.1220000000000001},"min":{"1min":0.65000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.7310000000000001,"5min":2.6299999999999999,"15min":2.6299999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.173,"15min":1.1950000000000001},"min":{"1min":0.72099999999999997,"5min":0.47899999999999998,"15min":0.442},"max":{"1min":1.5980000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.86699999999999999}]},{"osd":1,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0860000000000001,"15min":1.1819999999999999},"min":{"1min":0.77600000000000002,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.575,"5min":2.5009999999999999,"15min":3.2000000000000002},"last":1.044},{"interface":"front","average":{"1min":1.123,"5min":1.2150000000000001,"15min":1.2450000000000001},"min":{"1min":0.72299999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6779999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.29}]},{"osd":2,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.276,"15min":1.232},"min":{"1min":0.98099999999999998,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4590000000000001,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2230000000000001,"15min":1.27},"min":{"1min":0.90100000000000002,"5min":0.71399999999999997,"15min":0.65000000000000002},"max":{"1min":1.621,"5min":2.3809999999999998,"15min":3.282},"last":1.238}]},{"osd":3,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.2989999999999999,"15min":1.3280000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.63500000000000001},"max":{"1min":1.8300000000000001,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.383,"5min":1.385,"15min":1.395},"min":{"1min":1.0369999999999999,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.7030000000000001,"5min":1.9370000000000001,"15min":2.2280000000000002},"last":1.363}]},{"osd":4,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.3169999999999999,"15min":1.3069999999999999},"min":{"1min":0.85499999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.591,"5min":2.7480000000000002,"15min":4.0179999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.421,"15min":1.4350000000000001},"min":{"1min":1.014,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8700000000000001,"5min":2.927,"15min":3.456},"last":1.1779999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4590000000000001,"15min":1.4139999999999999},"min":{"1min":0.996,"5min":0.751,"15min":0.751},"max":{"1min":1.7290000000000001,"5min":2.149,"15min":2.359},"last":1.478},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4770000000000001,"15min":1.4670000000000001},"min":{"1min":1.121,"5min":0.623,"15min":0.58099999999999996},"max":{"1min":1.673,"5min":2.6429999999999998,"15min":3.8620000000000001},"last":1.4259999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:56:52 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.54,"15min":1.544},"min":{"1min":1.1459999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9299999999999999,"5min":3.0270000000000001,"15min":3.4079999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.595,"15min":1.5640000000000001},"min":{"1min":1.1870000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.8799999999999999,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":1.544}]}]},{"osd":4,"up_from":13,"seq":55834575100,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37324,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334516,"statfs":{"total":96636764160,"available":96598544384,"internally_reserved":0,"allocated":2031616,"data_stored":1038191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2250000000000001,"15min":1.2649999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.498},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.99},{"interface":"front","average":{"1min":1.377,"5min":1.3160000000000001,"15min":1.357},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.756,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.7090000000000001}]},{"osd":1,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.232,"15min":1.272},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.639,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.218,"15min":1.2290000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.893,"5min":3.1499999999999999,"15min":3.1499999999999999},"last":1.1519999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.206,"15min":1.224},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.798,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.262,"15min":1.3260000000000001},"min":{"1min":0.71799999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.843,"5min":2.431,"15min":2.431},"last":1.4239999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.258,"15min":1.272},"min":{"1min":0.747,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.704,"5min":5.3070000000000004,"15min":5.3070000000000004},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3340000000000001,"15min":1.3380000000000001},"min":{"1min":0.80800000000000005,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.9379999999999999,"5min":2.1600000000000001,"15min":2.1600000000000001},"last":1.5740000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.361,"15min":1.339},"min":{"1min":0.76700000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.891,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.524},{"interface":"front","average":{"1min":1.355,"5min":1.3779999999999999,"15min":1.3660000000000001},"min":{"1min":0.83399999999999996,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7969999999999999,"5min":5.4189999999999996,"15min":5.4189999999999996},"last":1.6279999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.4419999999999999,"15min":1.3720000000000001},"min":{"1min":0.91600000000000004,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.169,"5min":4.9059999999999997,"15min":4.9059999999999997},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5449999999999999,"15min":1.5109999999999999},"min":{"1min":0.96299999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.9279999999999999,"5min":6.0970000000000004,"15min":6.0970000000000004},"last":1.911}]},{"osd":7,"last update":"Sun Oct 8 05:56:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.52,"15min":1.4810000000000001},"min":{"1min":0.78900000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.0230000000000001,"5min":6.6689999999999996,"15min":6.6689999999999996},"last":1.472},{"interface":"front","average":{"1min":1.546,"5min":1.5680000000000001,"15min":1.5289999999999999},"min":{"1min":0.995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.1120000000000001,"5min":4.1849999999999996,"15min":4.1849999999999996},"last":2.0830000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575102,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41500,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330340,"statfs":{"total":96636764160,"available":96594268160,"internally_reserved":0,"allocated":2048000,"data_stored":1039873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.88400000000000001,"15min":0.91500000000000004},"min":{"1min":0.625,"5min":0.314,"15min":0.314},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.0469999999999999,"15min":1.087},"min":{"1min":0.57199999999999995,"5min":0.439,"15min":0.439},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.83599999999999997}]},{"osd":1,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90000000000000002,"15min":0.94499999999999995},"min":{"1min":0.66500000000000004,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.52,"5min":1.605,"15min":1.605},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.135,"5min":1.129,"15min":1.1859999999999999},"min":{"1min":0.58899999999999997,"5min":0.433,"15min":0.433},"max":{"1min":1.6439999999999999,"5min":1.6799999999999999,"15min":1.7210000000000001},"last":1.3680000000000001}]},{"osd":2,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.98099999999999998,"15min":1.0489999999999999},"min":{"1min":0.56599999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.077,"15min":1.1339999999999999},"min":{"1min":0.64000000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.1799999999999999}]},{"osd":4,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1819999999999999,"15min":1.2130000000000001},"min":{"1min":0.73399999999999999,"5min":0.68999999999999995,"15min":0.54000000000000004},"max":{"1min":1.792,"5min":1.792,"15min":2.2010000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.341,"15min":1.375},"min":{"1min":0.95399999999999996,"5min":0.82699999999999996,"15min":0.72799999999999998},"max":{"1min":2.448,"5min":2.448,"15min":2.589},"last":1.583}]},{"osd":5,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.276,"15min":1.294},"min":{"1min":0.84899999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6919999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.512},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3,"15min":1.343},"min":{"1min":1.028,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":1.4219999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.323,"15min":1.3220000000000001},"min":{"1min":0.84399999999999997,"5min":0.65700000000000003,"15min":0.61799999999999999},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.4279999999999999,"15min":1.4430000000000001},"min":{"1min":1.143,"5min":0.90300000000000002,"15min":0.86699999999999999},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":1.6739999999999999}]},{"osd":7,"last update":"Sun Oct 8 05:56:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.446,"15min":1.486},"min":{"1min":1.113,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.411,"5min":2.411,"15min":2.411},"last":1.492},{"interface":"front","average":{"1min":1.377,"5min":1.367,"15min":1.385},"min":{"1min":0.89900000000000002,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.3140000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575100,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63708,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308132,"statfs":{"total":96636764160,"available":96571527168,"internally_reserved":0,"allocated":2039808,"data_stored":1038807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":63175110},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78600000000000003,"15min":0.86299999999999999},"min":{"1min":0.34699999999999998,"5min":0.255,"15min":0.255},"max":{"1min":1.28,"5min":1.629,"15min":1.629},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.90700000000000003,"15min":1.0049999999999999},"min":{"1min":0.40400000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.2470000000000001,"5min":1.5660000000000001,"15min":1.9470000000000001},"last":1.024}]},{"osd":1,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.85899999999999999,"15min":0.92800000000000005},"min":{"1min":0.498,"5min":0.45900000000000002,"15min":0.42199999999999999},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":2.1070000000000002},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.96699999999999997,"15min":1.0569999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.35099999999999998},"max":{"1min":1.7050000000000001,"5min":1.79,"15min":2.1469999999999998},"last":1.53}]},{"osd":3,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.0209999999999999,"15min":1.099},"min":{"1min":0.58799999999999997,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.8120000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.085,"15min":1.1970000000000001},"min":{"1min":0.54100000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.5529999999999999,"5min":1.8859999999999999,"15min":7.5890000000000004},"last":1.2170000000000001}]},{"osd":4,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.286,"15min":1.363},"min":{"1min":0.80200000000000005,"5min":0.66000000000000003,"15min":0.622},"max":{"1min":3.6379999999999999,"5min":3.6379999999999999,"15min":4.593},"last":1.365},{"interface":"front","average":{"1min":1.335,"5min":1.2350000000000001,"15min":1.2789999999999999},"min":{"1min":0.73599999999999999,"5min":0.624,"15min":0.624},"max":{"1min":4.7770000000000001,"5min":4.7770000000000001,"15min":6.6879999999999997},"last":1.1359999999999999}]},{"osd":5,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2410000000000001,"15min":1.3480000000000001},"min":{"1min":0.75800000000000001,"5min":0.68799999999999994,"15min":0.63900000000000001},"max":{"1min":1.7,"5min":1.885,"15min":6.7140000000000004},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.3979999999999999,"15min":1.4419999999999999},"min":{"1min":0.872,"5min":0.71999999999999997,"15min":0.56599999999999995},"max":{"1min":6.625,"5min":6.625,"15min":6.625},"last":1.6299999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2989999999999999,"15min":1.3380000000000001},"min":{"1min":0.73299999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.73,"5min":2.0089999999999999,"15min":2.7080000000000002},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.573,"5min":1.3640000000000001,"15min":1.4379999999999999},"min":{"1min":0.77000000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":5.9569999999999999,"5min":5.9569999999999999,"15min":5.9569999999999999},"last":1.577}]},{"osd":7,"last update":"Sun Oct 8 05:56:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.3380000000000001,"15min":1.417},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.8799999999999999,"5min":1.956,"15min":6.609},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.3660000000000001,"15min":1.47},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8819999999999999,"5min":2.0070000000000001,"15min":2.9950000000000001},"last":1.4790000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575100,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41488,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330352,"statfs":{"total":96636764160,"available":96594280448,"internally_reserved":0,"allocated":2035712,"data_stored":1038261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.94099999999999995,"15min":0.97199999999999998},"min":{"1min":0.58999999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.454,"5min":2.3010000000000002,"15min":2.3010000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.014,"15min":1.1080000000000001},"min":{"1min":0.52600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.5049999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.84499999999999997,"15min":0.92500000000000004},"min":{"1min":0.45200000000000001,"5min":0.45100000000000001,"15min":0.39500000000000002},"max":{"1min":0.998,"5min":1.377,"15min":1.8240000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0609999999999999,"15min":1.093},"min":{"1min":0.55200000000000005,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.448,"5min":3.0030000000000001,"15min":3.0030000000000001},"last":0.42499999999999999}]},{"osd":3,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0700000000000001,"15min":1.1579999999999999},"min":{"1min":0.505,"5min":0.498,"15min":0.498},"max":{"1min":1.5269999999999999,"5min":2.6629999999999998,"15min":2.6629999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.98899999999999999,"15min":1.0609999999999999},"min":{"1min":0.47899999999999998,"5min":0.432,"15min":0.432},"max":{"1min":1.575,"5min":1.6120000000000001,"15min":1.788},"last":0.56399999999999995}]},{"osd":4,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.216,"15min":1.2809999999999999},"min":{"1min":0.70599999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6539999999999999,"5min":1.891,"15min":1.966},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.099,"5min":1.1519999999999999,"15min":1.212},"min":{"1min":0.83499999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3640000000000001,"5min":1.99,"15min":1.99},"last":1.1060000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2390000000000001,"15min":1.3009999999999999},"min":{"1min":0.88700000000000001,"5min":0.73199999999999998,"15min":0.70399999999999996},"max":{"1min":1.518,"5min":1.7909999999999999,"15min":1.8680000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.4059999999999999,"15min":1.448},"min":{"1min":0.85499999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.73,"5min":2.226,"15min":2.226},"last":0.91400000000000003}]},{"osd":6,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.359,"15min":1.409},"min":{"1min":0.84999999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.833,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.151},{"interface":"front","average":{"1min":1.325,"5min":1.339,"15min":1.4019999999999999},"min":{"1min":0.877,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6220000000000001,"5min":1.8580000000000001,"15min":2.8559999999999999},"last":1.0580000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:56:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.407,"15min":1.4550000000000001},"min":{"1min":0.94199999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.9490000000000001,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.3839999999999999,"15min":1.4410000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.9930000000000001,"5min":2.0819999999999999,"15min":2.306},"last":0.81499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575101,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33756,"kb_used_data":2572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338084,"statfs":{"total":96636764160,"available":96602198016,"internally_reserved":0,"allocated":2633728,"data_stored":1628615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31914441},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.86099999999999999,"15min":0.89800000000000002},"min":{"1min":0.54300000000000004,"5min":0.30199999999999999,"15min":0.28999999999999998},"max":{"1min":1.288,"5min":1.77,"15min":1.77},"last":0.497},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94199999999999995,"15min":1.022},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.371},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.675},"last":0.91100000000000003}]},{"osd":2,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.98499999999999999,"15min":1.0629999999999999},"min":{"1min":0.501,"5min":0.42599999999999999,"15min":0.39200000000000002},"max":{"1min":1.5700000000000001,"5min":1.5740000000000001,"15min":1.8700000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":0.93000000000000005,"15min":0.98199999999999998},"min":{"1min":0.51200000000000001,"5min":0.47399999999999998,"15min":0.40400000000000003},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":0.82699999999999996}]},{"osd":3,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.0960000000000001,"15min":1.1759999999999999},"min":{"1min":0.60199999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4650000000000001,"5min":1.823,"15min":1.9730000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.036,"5min":1.016,"15min":1.0760000000000001},"min":{"1min":0.57999999999999996,"5min":0.46500000000000002,"15min":0.44700000000000001},"max":{"1min":1.891,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":1.036}]},{"osd":4,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.181,"15min":1.252},"min":{"1min":0.66900000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.623,"5min":1.8129999999999999,"15min":1.9410000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.3100000000000001,"15min":1.333},"min":{"1min":0.83799999999999997,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.1120000000000001}]},{"osd":5,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2669999999999999,"15min":1.3220000000000001},"min":{"1min":0.64100000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5980000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.377},{"interface":"front","average":{"1min":1.29,"5min":1.333,"15min":1.365},"min":{"1min":0.86099999999999999,"5min":0.71199999999999997,"15min":0.63},"max":{"1min":1.9710000000000001,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":1.1919999999999999}]},{"osd":6,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.72099999999999997,"5min":0.64700000000000002,"15min":0.58499999999999996},"max":{"1min":1.8089999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.3380000000000001,"15min":1.379},"min":{"1min":0.88100000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.0880000000000001}]},{"osd":7,"last update":"Sun Oct 8 05:56:53 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.3839999999999999,"15min":1.419},"min":{"1min":0.92400000000000004,"5min":0.66300000000000003,"15min":0.63800000000000001},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4119999999999999,"15min":1.458},"min":{"1min":0.751,"5min":0.73699999999999999,"15min":0.54700000000000004},"max":{"1min":1.8839999999999999,"5min":2.0569999999999999,"15min":2.1059999999999999},"last":1.1599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T05:57:32.040 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T05:57:32.107 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T05:57:32.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.340+0000 7f063192c640 1 Processor -- start 2023-10-08T05:57:32.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.341+0000 7f063192c640 1 -- start start 2023-10-08T05:57:32.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.342+0000 7f063192c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f062c0d07f0 0x7f062c0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.342+0000 7f063192c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062c0d12f0 con 0x7f062c0d2bb0 2023-10-08T05:57:32.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.342+0000 7f063192c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062c0b5cb0 con 0x7f062c0b17c0 2023-10-08T05:57:32.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.342+0000 7f063192c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062c0b8a60 con 0x7f062c0d07f0 2023-10-08T05:57:32.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.342+0000 7f062affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f062c0d2bb0 0x7f062c0d2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50580/0 (socket says 172.21.15.70:50580) 2023-10-08T05:57:32.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f062affd640 1 -- 172.21.15.70:0/149622654 learned_addr learned my addr 172.21.15.70:0/149622654 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:32.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610589498 0 0) 0x7f062c0d12f0 con 0x7f062c0d2bb0 2023-10-08T05:57:32.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0618003520 con 0x7f062c0d2bb0 2023-10-08T05:57:32.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f062a7fc640 1 --2- 172.21.15.70:0/149622654 >> v2:172.21.15.181:3300/0 conn(0x7f062c0d07f0 0x7f062c0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 554532597 0 0) 0x7f0618003520 con 0x7f062c0d2bb0 2023-10-08T05:57:32.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 >> v2:172.21.15.181:3300/0 conn(0x7f062c0d07f0 msgr2=0x7f062c0d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:32.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 --2- 172.21.15.70:0/149622654 >> v2:172.21.15.181:3300/0 conn(0x7f062c0d07f0 0x7f062c0d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 >> v1:172.21.15.181:6789/0 conn(0x7f062c0b17c0 legacy=0x7f062c0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:32.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c0b84a0 con 0x7f062c0d2bb0 2023-10-08T05:57:32.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f061c0019f0 con 0x7f062c0d2bb0 2023-10-08T05:57:32.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 >> v1:172.21.15.70:6789/0 conn(0x7f062c0d2bb0 legacy=0x7f062c0d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:32.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 --2- 172.21.15.70:0/149622654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06180039c0 0x7f0618003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 --2- 172.21.15.70:0/149622654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180047c0 0x7f0618004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 --2- 172.21.15.70:0/149622654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180052a0 0x7f061800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c0b8a60 con 0x7f06180052a0 2023-10-08T05:57:32.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061800c1f0 con 0x7f06180039c0 2023-10-08T05:57:32.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.343+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061800c3b0 con 0x7f06180047c0 2023-10-08T05:57:32.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.344+0000 7f062a7fc640 1 --2- 172.21.15.70:0/149622654 >> v2:172.21.15.181:3300/0 conn(0x7f062c0d07f0 0x7f062c0d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:32.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.344+0000 7f062b7fe640 1 --2- 172.21.15.70:0/149622654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180052a0 0x7f061800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.344+0000 7f062b7fe640 1 -- 172.21.15.70:0/149622654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180047c0 msgr2=0x7f0618004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:32.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.344+0000 7f062b7fe640 1 --2- 172.21.15.70:0/149622654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180047c0 0x7f0618004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.344+0000 7f062b7fe640 1 -- 172.21.15.70:0/149622654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06180039c0 msgr2=0x7f0618003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.344+0000 7f062b7fe640 1 --2- 172.21.15.70:0/149622654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06180039c0 0x7f0618003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.344+0000 7f062b7fe640 1 -- 172.21.15.70:0/149622654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0618003520 con 0x7f06180052a0 2023-10-08T05:57:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.345+0000 7f062b7fe640 1 -- 172.21.15.70:0/149622654 reap_dead start 2023-10-08T05:57:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.345+0000 7f062b7fe640 1 --2- 172.21.15.70:0/149622654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180052a0 0x7f061800b980 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0624003d80 tx=0x7f06180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853e664df265d0c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.346+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624007220 con 0x7f06180052a0 2023-10-08T05:57:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.346+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062402e050 con 0x7f06180052a0 2023-10-08T05:57:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.346+0000 7f0629ffb640 1 -- 172.21.15.70:0/149622654 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06240075e0 con 0x7f06180052a0 2023-10-08T05:57:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.346+0000 7f063192c640 1 -- 172.21.15.70:0/149622654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180052a0 msgr2=0x7f061800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.346+0000 7f063192c640 1 --2- 172.21.15.70:0/149622654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180052a0 0x7f061800b980 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0624003d80 tx=0x7f06180057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.346+0000 7f063192c640 1 -- 172.21.15.70:0/149622654 shutdown_connections 2023-10-08T05:57:32.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.346+0000 7f063192c640 1 --2- 172.21.15.70:0/149622654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180052a0 0x7f061800b980 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.346+0000 7f063192c640 1 -- 172.21.15.70:0/149622654 >> 172.21.15.70:0/149622654 conn(0x7f062c0718a0 msgr2=0x7f062c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:32.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.346+0000 7f063192c640 1 -- 172.21.15.70:0/149622654 shutdown_connections 2023-10-08T05:57:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.346+0000 7f063192c640 1 -- 172.21.15.70:0/149622654 wait complete. 2023-10-08T05:57:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.348+0000 7f063192c640 1 Processor -- start 2023-10-08T05:57:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.348+0000 7f063192c640 1 -- start start 2023-10-08T05:57:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.349+0000 7f063192c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06180052a0 0x7f062c152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.349+0000 7f063192c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062c1f22c0 0x7f062c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.349+0000 7f063192c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f062c0cafb0 0x7f062c0c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.349+0000 7f063192c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c0c85e0 con 0x7f062c1f22c0 2023-10-08T05:57:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.349+0000 7f063192c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c0b5530 con 0x7f06180052a0 2023-10-08T05:57:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.349+0000 7f063192c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062c1f2740 con 0x7f062c0cafb0 2023-10-08T05:57:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062c1f22c0 0x7f062c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062c1f22c0 0x7f062c0b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52314/0 (socket says 172.21.15.70:52314) 2023-10-08T05:57:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062a7fc640 1 -- 172.21.15.70:0/3928291822 learned_addr learned my addr 172.21.15.70:0/3928291822 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062b7fe640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f062c0cafb0 0x7f062c0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062affd640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06180052a0 0x7f062c152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062a7fc640 1 -- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f062c0cafb0 msgr2=0x7f062c0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062a7fc640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f062c0cafb0 0x7f062c0c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062a7fc640 1 -- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06180052a0 msgr2=0x7f062c152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062a7fc640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06180052a0 0x7f062c152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062a7fc640 1 -- 172.21.15.70:0/3928291822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c0c79d0 con 0x7f062c1f22c0 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062affd640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06180052a0 0x7f062c152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.350+0000 7f062a7fc640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062c1f22c0 0x7f062c0b4ff0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f061400c7c0 tx=0x7f061400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.351+0000 7f0623fff640 1 -- 172.21.15.70:0/3928291822 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614007820 con 0x7f062c1f22c0 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.351+0000 7f0623fff640 1 -- 172.21.15.70:0/3928291822 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0614007980 con 0x7f062c1f22c0 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.351+0000 7f0623fff640 1 -- 172.21.15.70:0/3928291822 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614007d30 con 0x7f062c1f22c0 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.351+0000 7f063192c640 1 -- 172.21.15.70:0/3928291822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c0c7cb0 con 0x7f062c1f22c0 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.351+0000 7f063192c640 1 -- 172.21.15.70:0/3928291822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c0c8270 con 0x7f062c1f22c0 2023-10-08T05:57:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.352+0000 7f063192c640 1 -- 172.21.15.70:0/3928291822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062c0c8790 con 0x7f062c1f22c0 2023-10-08T05:57:32.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.353+0000 7f0623fff640 1 -- 172.21.15.70:0/3928291822 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f06140113f0 con 0x7f062c1f22c0 2023-10-08T05:57:32.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.354+0000 7f0623fff640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f060c06afc0 0x7f060c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.355+0000 7f0623fff640 1 -- 172.21.15.70:0/3928291822 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0614089c10 con 0x7f062c1f22c0 2023-10-08T05:57:32.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.355+0000 7f062affd640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f060c06afc0 0x7f060c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.356+0000 7f062affd640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f060c06afc0 0x7f060c06d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f062c15c910 tx=0x7f061c009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:32.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.359+0000 7f0623fff640 1 -- 172.21.15.70:0/3928291822 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f062c0c8790 con 0x7f062c1f22c0 2023-10-08T05:57:32.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.622+0000 7f063192c640 1 -- 172.21.15.70:0/3928291822 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f062c0c8790 con 0x7f062c1f22c0 2023-10-08T05:57:32.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.626+0000 7f0623fff640 1 -- 172.21.15.70:0/3928291822 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7f062c0c8790 con 0x7f062c1f22c0 2023-10-08T05:57:32.630 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T05:57:32.631 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T05:57:32.631 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T05:57:32.631 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T05:57:32.631 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T05:57:32.632 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T05:57:32.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 -- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f060c06afc0 msgr2=0x7f060c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:32.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f060c06afc0 0x7f060c06d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f062c15c910 tx=0x7f061c009040 comp rx=0 tx=0).stop 2023-10-08T05:57:32.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 -- 172.21.15.70:0/3928291822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062c1f22c0 msgr2=0x7f062c0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:32.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062c1f22c0 0x7f062c0b4ff0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f061400c7c0 tx=0x7f061400cc80 comp rx=0 tx=0).stop 2023-10-08T05:57:32.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 -- 172.21.15.70:0/3928291822 shutdown_connections 2023-10-08T05:57:32.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f060c06afc0 0x7f060c06d480 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f062c0cafb0 0x7f062c0c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062c1f22c0 0x7f062c0b4ff0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 --2- 172.21.15.70:0/3928291822 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06180052a0 0x7f062c152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 -- 172.21.15.70:0/3928291822 >> 172.21.15.70:0/3928291822 conn(0x7f062c0718a0 msgr2=0x7f062c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:32.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 -- 172.21.15.70:0/3928291822 shutdown_connections 2023-10-08T05:57:32.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.629+0000 7f063192c640 1 -- 172.21.15.70:0/3928291822 wait complete. 2023-10-08T05:57:32.648 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T05:57:32.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.879+0000 7f039cdd0640 1 Processor -- start 2023-10-08T05:57:32.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.880+0000 7f039cdd0640 1 -- start start 2023-10-08T05:57:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.881+0000 7f039cdd0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03980add60 0x7f0398162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.881+0000 7f039cdd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb150 0x7f03980cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.881+0000 7f039cdd0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03980cbaf0 con 0x7f039806d4c0 2023-10-08T05:57:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.881+0000 7f039cdd0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0398161f10 con 0x7f03980add60 2023-10-08T05:57:32.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.881+0000 7f039cdd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0398161630 con 0x7f03980cb150 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.881+0000 7f038ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f039806d4c0 0x7f03980cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50584/0 (socket says 172.21.15.70:50584) 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.881+0000 7f038ffff640 1 -- 172.21.15.70:0/907705973 learned_addr learned my addr 172.21.15.70:0/907705973 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.881+0000 7f039659c640 1 --2- 172.21.15.70:0/907705973 >> v2:172.21.15.70:3300/0 conn(0x7f03980add60 0x7f0398162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.882+0000 7f0396d9d640 1 --2- 172.21.15.70:0/907705973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb150 0x7f03980cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.882+0000 7f0395d9b640 1 -- 172.21.15.70:0/907705973 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593680744 0 0) 0x7f03980cbaf0 con 0x7f039806d4c0 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.882+0000 7f0395d9b640 1 -- 172.21.15.70:0/907705973 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0384003520 con 0x7f039806d4c0 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.882+0000 7f039659c640 1 -- 172.21.15.70:0/907705973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb150 msgr2=0x7f03980cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.882+0000 7f039659c640 1 --2- 172.21.15.70:0/907705973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb150 0x7f03980cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.882+0000 7f039659c640 1 -- 172.21.15.70:0/907705973 >> v1:172.21.15.70:6789/0 conn(0x7f039806d4c0 legacy=0x7f03980cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.882+0000 7f039659c640 1 -- 172.21.15.70:0/907705973 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03980ae140 con 0x7f03980add60 2023-10-08T05:57:32.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.883+0000 7f039659c640 1 --2- 172.21.15.70:0/907705973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980add60 0x7f0398162330 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f03880097d0 tx=0x7f038802b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c830b840cc6a186 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.883+0000 7f0395d9b640 1 -- 172.21.15.70:0/907705973 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388031770 con 0x7f03980add60 2023-10-08T05:57:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.883+0000 7f0395d9b640 1 -- 172.21.15.70:0/907705973 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03880318d0 con 0x7f03980add60 2023-10-08T05:57:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.883+0000 7f0395d9b640 1 -- 172.21.15.70:0/907705973 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388031c60 con 0x7f03980add60 2023-10-08T05:57:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.884+0000 7f039cdd0640 1 -- 172.21.15.70:0/907705973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980add60 msgr2=0x7f0398162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.884+0000 7f039cdd0640 1 --2- 172.21.15.70:0/907705973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980add60 0x7f0398162330 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f03880097d0 tx=0x7f038802b810 comp rx=0 tx=0).stop 2023-10-08T05:57:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.884+0000 7f039cdd0640 1 -- 172.21.15.70:0/907705973 shutdown_connections 2023-10-08T05:57:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.884+0000 7f039cdd0640 1 --2- 172.21.15.70:0/907705973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980cb150 0x7f03980cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.884+0000 7f039cdd0640 1 --2- 172.21.15.70:0/907705973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980add60 0x7f0398162330 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.884+0000 7f039cdd0640 1 -- 172.21.15.70:0/907705973 >> 172.21.15.70:0/907705973 conn(0x7f039806a380 msgr2=0x7f03980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:32.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.884+0000 7f039cdd0640 1 -- 172.21.15.70:0/907705973 shutdown_connections 2023-10-08T05:57:32.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.884+0000 7f039cdd0640 1 -- 172.21.15.70:0/907705973 wait complete. 2023-10-08T05:57:32.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.886+0000 7f039cdd0640 1 Processor -- start 2023-10-08T05:57:32.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.886+0000 7f039cdd0640 1 -- start start 2023-10-08T05:57:32.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.887+0000 7f039cdd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039806d4c0 0x7f0398155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.887+0000 7f039cdd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980add60 0x7f0398158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.887+0000 7f039cdd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb150 0x7f03980ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.887+0000 7f039cdd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0398157f50 con 0x7f03980cb150 2023-10-08T05:57:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.887+0000 7f039cdd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03981581f0 con 0x7f039806d4c0 2023-10-08T05:57:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.887+0000 7f039cdd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03981584d0 con 0x7f03980add60 2023-10-08T05:57:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.888+0000 7f0396d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb150 0x7f03980ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.888+0000 7f0396d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb150 0x7f03980ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52332/0 (socket says 172.21.15.70:52332) 2023-10-08T05:57:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.888+0000 7f0396d9d640 1 -- 172.21.15.70:0/3979161152 learned_addr learned my addr 172.21.15.70:0/3979161152 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:32.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.888+0000 7f039659c640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039806d4c0 0x7f0398155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.888+0000 7f038ffff640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980add60 0x7f0398158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.888+0000 7f0396d9d640 1 -- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980add60 msgr2=0x7f0398158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.888+0000 7f0396d9d640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980add60 0x7f0398158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.888+0000 7f0396d9d640 1 -- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039806d4c0 msgr2=0x7f0398155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.888+0000 7f0396d9d640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039806d4c0 0x7f0398155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.888+0000 7f0396d9d640 1 -- 172.21.15.70:0/3979161152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0388009400 con 0x7f03980cb150 2023-10-08T05:57:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.889+0000 7f0396d9d640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb150 0x7f03980ae320 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f0390007e60 tx=0x7f039000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.889+0000 7f038effd640 1 -- 172.21.15.70:0/3979161152 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390014420 con 0x7f03980cb150 2023-10-08T05:57:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.889+0000 7f038effd640 1 -- 172.21.15.70:0/3979161152 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03900145a0 con 0x7f03980cb150 2023-10-08T05:57:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.889+0000 7f038effd640 1 -- 172.21.15.70:0/3979161152 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390014950 con 0x7f03980cb150 2023-10-08T05:57:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.889+0000 7f039cdd0640 1 -- 172.21.15.70:0/3979161152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03981587b0 con 0x7f03980cb150 2023-10-08T05:57:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.889+0000 7f039cdd0640 1 -- 172.21.15.70:0/3979161152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0398155f70 con 0x7f03980cb150 2023-10-08T05:57:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.890+0000 7f039cdd0640 1 -- 172.21.15.70:0/3979161152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0398158cd0 con 0x7f03980cb150 2023-10-08T05:57:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.891+0000 7f038effd640 1 -- 172.21.15.70:0/3979161152 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0390014ab0 con 0x7f03980cb150 2023-10-08T05:57:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.892+0000 7f038effd640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f037806af90 0x7f037806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.892+0000 7f038effd640 1 -- 172.21.15.70:0/3979161152 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f039008a260 con 0x7f03980cb150 2023-10-08T05:57:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.892+0000 7f039659c640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f037806af90 0x7f037806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.894+0000 7f039659c640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f037806af90 0x7f037806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0388036000 tx=0x7f038803d040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:32.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:32.896+0000 7f038effd640 1 -- 172.21.15.70:0/3979161152 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0398158cd0 con 0x7f03980cb150 2023-10-08T05:57:33.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.106+0000 7f039cdd0640 1 -- 172.21.15.70:0/3979161152 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0398158cd0 con 0x7f03980cb150 2023-10-08T05:57:33.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.108+0000 7f038effd640 1 -- 172.21.15.70:0/3979161152 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f0398158cd0 con 0x7f03980cb150 2023-10-08T05:57:33.133 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:33.133 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2889908260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2889908260},{"type":"v1","addr":"172.21.15.181:6837","nonce":2889908260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2651702867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2651702867},{"type":"v1","addr":"172.21.15.181:6835","nonce":2651702867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":9048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1574948577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1574948577},{"type":"v1","addr":"172.21.15.70:6833","nonce":1574948577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:57:26.074871+0000","modified":"2023-10-08T05:57:29.076265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9051":{"gid":9051,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6835/3537072579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3537072579},{"type":"v1","addr":"172.21.15.70:6835","nonce":3537072579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-08T05:57:33.135 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 76 2023-10-08T05:57:33.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 -- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f037806af90 msgr2=0x7f037806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:33.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f037806af90 0x7f037806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0388036000 tx=0x7f038803d040 comp rx=0 tx=0).stop 2023-10-08T05:57:33.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 -- 172.21.15.70:0/3979161152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb150 msgr2=0x7f03980ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:33.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb150 0x7f03980ae320 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f0390007e60 tx=0x7f039000e4b0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 -- 172.21.15.70:0/3979161152 shutdown_connections 2023-10-08T05:57:33.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f037806af90 0x7f037806d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03980cb150 0x7f03980ae320 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03980add60 0x7f0398158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 --2- 172.21.15.70:0/3979161152 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f039806d4c0 0x7f0398155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 -- 172.21.15.70:0/3979161152 >> 172.21.15.70:0/3979161152 conn(0x7f039806a380 msgr2=0x7f0398162860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:33.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 -- 172.21.15.70:0/3979161152 shutdown_connections 2023-10-08T05:57:33.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.110+0000 7f039cdd0640 1 -- 172.21.15.70:0/3979161152 wait complete. 2023-10-08T05:57:33.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T05:57:26.074871+0000', 'modified': '2023-10-08T05:57:29.076265+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9051}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9051': {'gid': 9051, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.70:6835/3537072579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 3537072579}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 3537072579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T05:57:33.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T05:57:33.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T05:57:33.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.777+0000 7f9e4fe32640 1 Processor -- start 2023-10-08T05:57:33.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.777+0000 7f9e4fe32640 1 -- start start 2023-10-08T05:57:33.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.779+0000 7f9e4fe32640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e480cb150 0x7f9e480cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.779+0000 7f9e4fe32640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4806e1e0 con 0x7f9e48161690 2023-10-08T05:57:33.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.779+0000 7f9e4fe32640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4806cc50 con 0x7f9e480b1f20 2023-10-08T05:57:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.779+0000 7f9e4fe32640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e480cbb40 con 0x7f9e480cb150 2023-10-08T05:57:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.779+0000 7f9e4dbce640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9e48161690 0x7f9e4806dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38480/0 (socket says 172.21.15.70:38480) 2023-10-08T05:57:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.779+0000 7f9e4dbce640 1 -- 172.21.15.70:0/1500198928 learned_addr learned my addr 172.21.15.70:0/1500198928 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.779+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/1500198928 >> v2:172.21.15.70:3300/0 conn(0x7f9e480cb150 0x7f9e480cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.780+0000 7f9e4cbcc640 1 -- 172.21.15.70:0/1500198928 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261356561 0 0) 0x7f9e4806e1e0 con 0x7f9e48161690 2023-10-08T05:57:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.780+0000 7f9e4cbcc640 1 -- 172.21.15.70:0/1500198928 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e30003520 con 0x7f9e48161690 2023-10-08T05:57:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.780+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/1500198928 >> v1:172.21.15.181:6789/0 conn(0x7f9e480b1f20 legacy=0x7f9e480b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.780+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/1500198928 >> v1:172.21.15.70:6789/0 conn(0x7f9e48161690 legacy=0x7f9e4806dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:33.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.780+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/1500198928 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4806cf30 con 0x7f9e480cb150 2023-10-08T05:57:33.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.781+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/1500198928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 0x7f9e480cb600 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0097a0 tx=0x7f9e3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90832a20647b2eac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:33.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.781+0000 7f9e4cbcc640 1 -- 172.21.15.70:0/1500198928 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c031830 con 0x7f9e480cb150 2023-10-08T05:57:33.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.781+0000 7f9e4cbcc640 1 -- 172.21.15.70:0/1500198928 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c031990 con 0x7f9e480cb150 2023-10-08T05:57:33.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.781+0000 7f9e4cbcc640 1 -- 172.21.15.70:0/1500198928 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c031d20 con 0x7f9e480cb150 2023-10-08T05:57:33.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.782+0000 7f9e4fe32640 1 -- 172.21.15.70:0/1500198928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 msgr2=0x7f9e480cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:33.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.782+0000 7f9e4fe32640 1 --2- 172.21.15.70:0/1500198928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 0x7f9e480cb600 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0097a0 tx=0x7f9e3c02b8d0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.782+0000 7f9e4fe32640 1 -- 172.21.15.70:0/1500198928 shutdown_connections 2023-10-08T05:57:33.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.782+0000 7f9e4fe32640 1 --2- 172.21.15.70:0/1500198928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 0x7f9e480cb600 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.782+0000 7f9e4fe32640 1 -- 172.21.15.70:0/1500198928 >> 172.21.15.70:0/1500198928 conn(0x7f9e4806a5d0 msgr2=0x7f9e480b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:33.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.782+0000 7f9e4fe32640 1 -- 172.21.15.70:0/1500198928 shutdown_connections 2023-10-08T05:57:33.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.782+0000 7f9e4fe32640 1 -- 172.21.15.70:0/1500198928 wait complete. 2023-10-08T05:57:33.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.784+0000 7f9e4fe32640 1 Processor -- start 2023-10-08T05:57:33.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.784+0000 7f9e4fe32640 1 -- start start 2023-10-08T05:57:33.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.785+0000 7f9e4fe32640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e480b1f20 0x7f9e480b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.785+0000 7f9e4fe32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 0x7f9e480b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.785+0000 7f9e4fe32640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e48161690 0x7f9e480c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.785+0000 7f9e4fe32640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480b4490 con 0x7f9e480cb150 2023-10-08T05:57:33.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.785+0000 7f9e4fe32640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e48163f90 con 0x7f9e480b1f20 2023-10-08T05:57:33.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.785+0000 7f9e4fe32640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e48164270 con 0x7f9e48161690 2023-10-08T05:57:33.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.785+0000 7f9e4d3cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 0x7f9e480b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.785+0000 7f9e4d3cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 0x7f9e480b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34626/0 (socket says 172.21.15.70:34626) 2023-10-08T05:57:33.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.785+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/2273854908 learned_addr learned my addr 172.21.15.70:0/2273854908 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:33.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e480b1f20 0x7f9e480b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e48161690 msgr2=0x7f9e480c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:33.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e4e3cf640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e48161690 0x7f9e480c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e48161690 0x7f9e480c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e480b1f20 msgr2=0x7f9e480b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:33.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e480b1f20 0x7f9e480b30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/2273854908 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c009400 con 0x7f9e480cb150 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e480b1f20 0x7f9e480b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 0x7f9e480b3600 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0098c0 tx=0x7f9e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e467fc640 1 -- 172.21.15.70:0/2273854908 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c03a6f0 con 0x7f9e480cb150 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e467fc640 1 -- 172.21.15.70:0/2273854908 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c03a870 con 0x7f9e480cb150 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e467fc640 1 -- 172.21.15.70:0/2273854908 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c03ac20 con 0x7f9e480cb150 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.786+0000 7f9e4fe32640 1 -- 172.21.15.70:0/2273854908 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e48164550 con 0x7f9e480cb150 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.787+0000 7f9e4fe32640 1 -- 172.21.15.70:0/2273854908 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e48163bb0 con 0x7f9e480cb150 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.789+0000 7f9e467fc640 1 -- 172.21.15.70:0/2273854908 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9e3c030380 con 0x7f9e480cb150 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.790+0000 7f9e467fc640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f9e3406af90 0x7f9e3406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.790+0000 7f9e467fc640 1 -- 172.21.15.70:0/2273854908 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9e3c0ab450 con 0x7f9e480cb150 2023-10-08T05:57:33.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.790+0000 7f9e529c0740 1 Processor -- start 2023-10-08T05:57:33.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.790+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f9e3406af90 0x7f9e3406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.791+0000 7f9e529c0740 1 -- start start 2023-10-08T05:57:33.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.791+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c63feba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.791+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018f40 0x556c6401b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.791+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c63febfa0 0x556c6401be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.791+0000 7f9e529c0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c63f5f700 con 0x556c64018b60 2023-10-08T05:57:33.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.791+0000 7f9e529c0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c63f5cdf0 con 0x556c63febfa0 2023-10-08T05:57:33.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.791+0000 7f9e529c0740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c63f586e0 con 0x556c64018f40 2023-10-08T05:57:33.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f9e3406af90 0x7f9e3406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e38005c00 tx=0x7f9e38009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:33.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4dbce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c63feba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4dbce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c63feba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34630/0 (socket says 172.21.15.70:34630) 2023-10-08T05:57:33.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4dbce640 1 -- 172.21.15.70:0/1675710772 learned_addr learned my addr 172.21.15.70:0/1675710772 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:33.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/1675710772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018f40 0x556c6401b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4e3cf640 1 --2- 172.21.15.70:0/1675710772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c63febfa0 0x556c6401be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4dbce640 1 -- 172.21.15.70:0/1675710772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018f40 msgr2=0x556c6401b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:33.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/1675710772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018f40 0x556c6401b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4dbce640 1 -- 172.21.15.70:0/1675710772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c63febfa0 msgr2=0x556c6401be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:33.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/1675710772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c63febfa0 0x556c6401be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4dbce640 1 -- 172.21.15.70:0/1675710772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c63f1c910 con 0x556c64018b60 2023-10-08T05:57:33.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4e3cf640 1 --2- 172.21.15.70:0/1675710772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c63febfa0 0x556c6401be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:33.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.792+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/1675710772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c63feba60 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800df20 tx=0x7f9e380021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d652b99e48f1a61 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:33.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e2f7fe640 1 -- 172.21.15.70:0/1675710772 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38012820 con 0x556c64018b60 2023-10-08T05:57:33.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e2f7fe640 1 -- 172.21.15.70:0/1675710772 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38012980 con 0x556c64018b60 2023-10-08T05:57:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e2f7fe640 1 -- 172.21.15.70:0/1675710772 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38012cf0 con 0x556c64018b60 2023-10-08T05:57:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e529c0740 1 -- 172.21.15.70:0/1675710772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 msgr2=0x556c63feba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e529c0740 1 --2- 172.21.15.70:0/1675710772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c63feba60 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800df20 tx=0x7f9e380021f0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e529c0740 1 -- 172.21.15.70:0/1675710772 shutdown_connections 2023-10-08T05:57:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e529c0740 1 --2- 172.21.15.70:0/1675710772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c63febfa0 0x556c6401be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e529c0740 1 --2- 172.21.15.70:0/1675710772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018f40 0x556c6401b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e529c0740 1 --2- 172.21.15.70:0/1675710772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c63feba60 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e529c0740 1 -- 172.21.15.70:0/1675710772 >> 172.21.15.70:0/1675710772 conn(0x556c63fedd70 msgr2=0x556c640368f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:33.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e529c0740 1 -- 172.21.15.70:0/1675710772 shutdown_connections 2023-10-08T05:57:33.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.793+0000 7f9e529c0740 1 -- 172.21.15.70:0/1675710772 wait complete. 2023-10-08T05:57:33.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.794+0000 7f9e529c0740 1 -- start start 2023-10-08T05:57:33.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.794+0000 7f9e529c0740 1 Processor -- start 2023-10-08T05:57:33.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.795+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018b60 0x556c640374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.795+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64018f40 0x556c640136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.795+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64013c00 0x556c640116b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.795+0000 7f9e529c0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c63ef6970 con 0x556c64013c00 2023-10-08T05:57:33.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.795+0000 7f9e529c0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c63ef6610 con 0x556c64018f40 2023-10-08T05:57:33.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.795+0000 7f9e529c0740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c63cbcf20 con 0x556c64018b60 2023-10-08T05:57:33.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.795+0000 7f9e529c0740 10 client.0 _ll_register_callbacks cb 0x556c63f34610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:57:33.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e4e3cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64013c00 0x556c640116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e4e3cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64013c00 0x556c640116b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34642/0 (socket says 172.21.15.70:34642) 2023-10-08T05:57:33.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e4e3cf640 1 -- 172.21.15.70:0/3229063664 learned_addr learned my addr 172.21.15.70:0/3229063664 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:33.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e4e3cf640 1 -- 172.21.15.70:0/3229063664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018b60 msgr2=0x556c640374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:33.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e4e3cf640 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018b60 0x556c640374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e4e3cf640 1 -- 172.21.15.70:0/3229063664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64018f40 msgr2=0x556c640136c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:33.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64018f40 0x556c640136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e4e3cf640 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64018f40 0x556c640136c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e4e3cf640 1 -- 172.21.15.70:0/3229063664 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e30003520 con 0x556c64013c00 2023-10-08T05:57:33.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e4e3cf640 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64013c00 0x556c640116b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000bef0 tx=0x7f9e4000d590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0086b0b313efed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:33.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:57:33.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.796+0000 7f9e2cff9640 1 -- 172.21.15.70:0/3229063664 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40012660 con 0x556c64013c00 2023-10-08T05:57:33.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e2cff9640 1 -- 172.21.15.70:0/3229063664 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9e400127c0 con 0x556c64013c00 2023-10-08T05:57:33.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e529c0740 1 -- 172.21.15.70:0/3229063664 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c63f516c0 con 0x556c64013c00 2023-10-08T05:57:33.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e2cff9640 1 -- 172.21.15.70:0/3229063664 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e400129d0 con 0x556c64013c00 2023-10-08T05:57:33.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e529c0740 10 client.9207 fetch_fsmap learned FSMap version 76 2023-10-08T05:57:33.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e529c0740 1 -- 172.21.15.70:0/3229063664 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x556c63f56d60 con 0x556c64013c00 2023-10-08T05:57:33.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e2cff9640 1 -- 172.21.15.70:0/3229063664 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c63f516c0 con 0x556c64013c00 2023-10-08T05:57:33.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e2cff9640 1 -- 172.21.15.70:0/3229063664 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e40012b30 con 0x556c64013c00 2023-10-08T05:57:33.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e529c0740 10 client.9207 fetch_fsmap finished waiting for FSMap version 76 2023-10-08T05:57:33.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e529c0740 10 client.9207 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T05:57:33.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e529c0740 20 client.9207 populate_metadata read hostname 'smithi070' 2023-10-08T05:57:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e529c0740 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.70:6834/3537072579,v1:172.21.15.70:6835/3537072579] conn(0x556c6401af20 0x556c6401a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e529c0740 4 client.9207 mds_command: new command op to 9051 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T05:57:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e529c0740 1 -- 172.21.15.70:0/3229063664 --> [v2:172.21.15.70:6834/3537072579,v1:172.21.15.70:6835/3537072579] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c63f513e0 con 0x556c6401af20 2023-10-08T05:57:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.797+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.70:6834/3537072579,v1:172.21.15.70:6835/3537072579] conn(0x556c6401af20 0x556c6401a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.798+0000 7f9e2cff9640 10 client.9207 ms_handle_connect on v2:172.21.15.70:6834/3537072579 2023-10-08T05:57:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.798+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.70:6834/3537072579,v1:172.21.15.70:6835/3537072579] conn(0x556c6401af20 0x556c6401a150 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62281f05e77a7504 server_cookie=9486bd2b61c254f8 in_seq=0 out_seq=0 2023-10-08T05:57:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e2cff9640 1 -- 172.21.15.70:0/3229063664 <== mds.0 v2:172.21.15.70:6834/3537072579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556c63f513e0 con 0x556c6401af20 2023-10-08T05:57:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e2cff9640 10 client.9207 handle_command_reply: tid=0 2023-10-08T05:57:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 client.9207 shutdown 2023-10-08T05:57:33.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 -- 172.21.15.70:0/3229063664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64013c00 msgr2=0x556c640116b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64013c00 0x556c640116b0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000bef0 tx=0x7f9e4000d590 comp rx=0 tx=0).stop 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 -- 172.21.15.70:0/3229063664 shutdown_connections 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.70:6834/3537072579,v1:172.21.15.70:6835/3537072579] conn(0x556c6401af20 0x556c6401a150 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64013c00 0x556c640116b0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64018f40 0x556c640136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e2cff9640 0 client.9207 ms_handle_reset on v2:172.21.15.70:6834/3537072579 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 --2- 172.21.15.70:0/3229063664 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018b60 0x556c640374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 -- 172.21.15.70:0/3229063664 >> 172.21.15.70:0/3229063664 conn(0x556c63febfa0 msgr2=0x556c64039e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 -- 172.21.15.70:0/3229063664 shutdown_connections 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.799+0000 7f9e529c0740 1 -- 172.21.15.70:0/3229063664 wait complete. 2023-10-08T05:57:33.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.800+0000 7f9e529c0740 20 client.9207 trim_cache size 0 max 16384 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.816+0000 7f9e529c0740 1 Processor -- start 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.816+0000 7f9e529c0740 1 -- start start 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.817+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 0x556c6401ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.817+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018b60 0x556c64039e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.817+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018f40 0x556c6403a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.817+0000 7f9e529c0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c63fe6770 con 0x556c64018f40 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.817+0000 7f9e529c0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c64036f30 con 0x556c64013c00 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.817+0000 7f9e529c0740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c63fefbf0 con 0x556c64018b60 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.817+0000 7f9e4e3cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018f40 0x556c6403a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.817+0000 7f9e4e3cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018f40 0x556c6403a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34658/0 (socket says 172.21.15.70:34658) 2023-10-08T05:57:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.817+0000 7f9e4e3cf640 1 -- 172.21.15.70:0/1526717310 learned_addr learned my addr 172.21.15.70:0/1526717310 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:33.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/1526717310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 0x556c6401ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e4e3cf640 1 -- 172.21.15.70:0/1526717310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018b60 msgr2=0x556c64039e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:33.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/1526717310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018b60 0x556c64039e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e4e3cf640 1 --2- 172.21.15.70:0/1526717310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018b60 0x556c64039e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e4e3cf640 1 -- 172.21.15.70:0/1526717310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 msgr2=0x556c6401ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e4e3cf640 1 --2- 172.21.15.70:0/1526717310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 0x556c6401ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e4e3cf640 1 -- 172.21.15.70:0/1526717310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c63ef5b80 con 0x556c64018f40 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/1526717310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 0x556c6401ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e4e3cf640 1 --2- 172.21.15.70:0/1526717310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018f40 0x556c6403a340 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f9e4001b540 tx=0x7f9e40012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3aae63bc360f10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e2ffff640 1 -- 172.21.15.70:0/1526717310 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40004240 con 0x556c64018f40 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e2ffff640 1 -- 172.21.15.70:0/1526717310 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4000c040 con 0x556c64018f40 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.818+0000 7f9e2ffff640 1 -- 172.21.15.70:0/1526717310 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40012460 con 0x556c64018f40 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.819+0000 7f9e529c0740 1 -- 172.21.15.70:0/1526717310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018f40 msgr2=0x556c6403a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.819+0000 7f9e529c0740 1 --2- 172.21.15.70:0/1526717310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018f40 0x556c6403a340 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f9e4001b540 tx=0x7f9e40012000 comp rx=0 tx=0).stop 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.819+0000 7f9e529c0740 1 -- 172.21.15.70:0/1526717310 shutdown_connections 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.819+0000 7f9e529c0740 1 --2- 172.21.15.70:0/1526717310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018f40 0x556c6403a340 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.819+0000 7f9e529c0740 1 --2- 172.21.15.70:0/1526717310 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018b60 0x556c64039e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.819+0000 7f9e529c0740 1 --2- 172.21.15.70:0/1526717310 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 0x556c6401ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.819+0000 7f9e529c0740 1 -- 172.21.15.70:0/1526717310 >> 172.21.15.70:0/1526717310 conn(0x556c63febfa0 msgr2=0x556c64010980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.819+0000 7f9e529c0740 1 -- 172.21.15.70:0/1526717310 shutdown_connections 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.819+0000 7f9e529c0740 1 -- 172.21.15.70:0/1526717310 wait complete. 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.820+0000 7f9e529c0740 1 -- start start 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.820+0000 7f9e529c0740 1 Processor -- start 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.821+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 0x556c64011670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.821+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c64039e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.821+0000 7f9e529c0740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018f40 0x556c6403a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.821+0000 7f9e529c0740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c64017140 con 0x556c64018b60 2023-10-08T05:57:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.821+0000 7f9e529c0740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6409fba0 con 0x556c64013c00 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.821+0000 7f9e529c0740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c64017a50 con 0x556c64018f40 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.821+0000 7f9e529c0740 10 client.0 _ll_register_callbacks cb 0x7f9e38001f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.821+0000 7f9e4d3cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c64039e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.821+0000 7f9e4d3cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c64039e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34660/0 (socket says 172.21.15.70:34660) 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.821+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/884921716 learned_addr learned my addr 172.21.15.70:0/884921716 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 0x556c64011670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e4e3cf640 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018f40 0x556c6403a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/884921716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018f40 msgr2=0x556c6403a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018f40 0x556c6403a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/884921716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 msgr2=0x556c64011670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 0x556c64011670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e4d3cd640 1 -- 172.21.15.70:0/884921716 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c63f56d60 con 0x556c64018b60 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e4d3cd640 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c64039e00 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c02be60 tx=0x7f9e3c0797c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e167521386e16d7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e2cff9640 1 -- 172.21.15.70:0/884921716 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c091070 con 0x556c64018b60 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e2cff9640 1 -- 172.21.15.70:0/884921716 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9e3c082070 con 0x556c64018b60 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e529c0740 1 -- 172.21.15.70:0/884921716 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e38012980 con 0x556c64018b60 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e2cff9640 1 -- 172.21.15.70:0/884921716 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e3c0a9170 con 0x556c64018b60 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e529c0740 10 client.9213 fetch_fsmap learned FSMap version 76 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.822+0000 7f9e529c0740 1 -- 172.21.15.70:0/884921716 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x556c63f1b170 con 0x556c64018b60 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.823+0000 7f9e2cff9640 1 -- 172.21.15.70:0/884921716 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38012980 con 0x556c64018b60 2023-10-08T05:57:33.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.823+0000 7f9e2cff9640 1 -- 172.21.15.70:0/884921716 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e3c076480 con 0x556c64018b60 2023-10-08T05:57:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.823+0000 7f9e529c0740 10 client.9213 fetch_fsmap finished waiting for FSMap version 76 2023-10-08T05:57:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.823+0000 7f9e529c0740 10 client.9213 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T05:57:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.823+0000 7f9e529c0740 20 client.9213 populate_metadata read hostname 'smithi070' 2023-10-08T05:57:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.823+0000 7f9e529c0740 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.70:6834/3537072579,v1:172.21.15.70:6835/3537072579] conn(0x556c640a29a0 0x556c6401b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.823+0000 7f9e529c0740 4 client.9213 mds_command: new command op to 9051 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T05:57:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.823+0000 7f9e529c0740 1 -- 172.21.15.70:0/884921716 --> [v2:172.21.15.70:6834/3537072579,v1:172.21.15.70:6835/3537072579] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556c63c5a6c0 con 0x556c640a29a0 2023-10-08T05:57:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.823+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.70:6834/3537072579,v1:172.21.15.70:6835/3537072579] conn(0x556c640a29a0 0x556c6401b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:33.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e4dbce640 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.70:6834/3537072579,v1:172.21.15.70:6835/3537072579] conn(0x556c640a29a0 0x556c6401b690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb61ea9ae55e82ea server_cookie=3a251fecce124073 in_seq=0 out_seq=0 2023-10-08T05:57:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e2cff9640 10 client.9213 ms_handle_connect on v2:172.21.15.70:6834/3537072579 2023-10-08T05:57:34.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e2cff9640 1 -- 172.21.15.70:0/884921716 <== mds.0 v2:172.21.15.70:6834/3537072579 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x556c63c5a6c0 con 0x556c640a29a0 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e2cff9640 10 client.9213 handle_command_reply: tid=0 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 client.9213 shutdown 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 -- 172.21.15.70:0/884921716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 msgr2=0x556c64039e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c64039e00 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c02be60 tx=0x7f9e3c0797c0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 -- 172.21.15.70:0/884921716 shutdown_connections 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.70:6834/3537072579,v1:172.21.15.70:6835/3537072579] conn(0x556c640a29a0 0x556c6401b690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x556c64018f40 0x556c6403a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x556c64018b60 0x556c64039e00 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 --2- 172.21.15.70:0/884921716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x556c64013c00 0x556c64011670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e2cff9640 0 client.9213 ms_handle_reset on v2:172.21.15.70:6834/3537072579 2023-10-08T05:57:34.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 -- 172.21.15.70:0/884921716 >> 172.21.15.70:0/884921716 conn(0x556c63febfa0 msgr2=0x556c64010980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:34.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 -- 172.21.15.70:0/884921716 shutdown_connections 2023-10-08T05:57:34.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 1 -- 172.21.15.70:0/884921716 wait complete. 2023-10-08T05:57:34.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.824+0000 7f9e529c0740 20 client.9213 trim_cache size 0 max 16384 2023-10-08T05:57:34.022 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T05:57:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T05:57:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T05:57:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 9051, 2023-10-08T05:57:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T05:57:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T05:57:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T05:57:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 7.2771902019999999, 2023-10-08T05:57:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 76, 2023-10-08T05:57:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 163, 2023-10-08T05:57:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 163, 2023-10-08T05:57:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 21.362210099999999 2023-10-08T05:57:34.023 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T05:57:34.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 -- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f9e3406af90 msgr2=0x7f9e3406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f9e3406af90 0x7f9e3406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e38005c00 tx=0x7f9e38009040 comp rx=0 tx=0).stop 2023-10-08T05:57:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 -- 172.21.15.70:0/2273854908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 msgr2=0x7f9e480b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 0x7f9e480b3600 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0098c0 tx=0x7f9e3c0046c0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 -- 172.21.15.70:0/2273854908 shutdown_connections 2023-10-08T05:57:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f9e3406af90 0x7f9e3406d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e48161690 0x7f9e480c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e480cb150 0x7f9e480b3600 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 --2- 172.21.15.70:0/2273854908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e480b1f20 0x7f9e480b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 -- 172.21.15.70:0/2273854908 >> 172.21.15.70:0/2273854908 conn(0x7f9e4806a5d0 msgr2=0x7f9e480cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 -- 172.21.15.70:0/2273854908 shutdown_connections 2023-10-08T05:57:34.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:33.825+0000 7f9e4fe32640 1 -- 172.21.15.70:0/2273854908 wait complete. 2023-10-08T05:57:34.026 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T05:57:34.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.247+0000 7f6d8f56e640 1 Processor -- start 2023-10-08T05:57:34.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.247+0000 7f6d8f56e640 1 -- start start 2023-10-08T05:57:34.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.249+0000 7f6d8f56e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d8815d1d0 0x7f6d880cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.249+0000 7f6d8f56e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d880cf670 con 0x7f6d8815cdf0 2023-10-08T05:57:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.249+0000 7f6d8f56e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d880caf80 con 0x7f6d880cd820 2023-10-08T05:57:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.249+0000 7f6d8f56e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880cf8c0 con 0x7f6d8815d1d0 2023-10-08T05:57:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.249+0000 7f6d8e56c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6d8815cdf0 0x7f6d880cd110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38492/0 (socket says 172.21.15.70:38492) 2023-10-08T05:57:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.249+0000 7f6d8e56c640 1 -- 172.21.15.70:0/4291038346 learned_addr learned my addr 172.21.15.70:0/4291038346 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177362500 0 0) 0x7f6d880cf670 con 0x7f6d8815cdf0 2023-10-08T05:57:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d74003520 con 0x7f6d8815cdf0 2023-10-08T05:57:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8ed6d640 1 --2- 172.21.15.70:0/4291038346 >> v2:172.21.15.181:3300/0 conn(0x7f6d8815d1d0 0x7f6d880cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1030811764 0 0) 0x7f6d74003520 con 0x7f6d8815cdf0 2023-10-08T05:57:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 >> v2:172.21.15.181:3300/0 conn(0x7f6d8815d1d0 msgr2=0x7f6d880cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 --2- 172.21.15.70:0/4291038346 >> v2:172.21.15.181:3300/0 conn(0x7f6d8815d1d0 0x7f6d880cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 >> v1:172.21.15.181:6789/0 conn(0x7f6d880cd820 legacy=0x7f6d880cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d880cfba0 con 0x7f6d8815cdf0 2023-10-08T05:57:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6d84001670 con 0x7f6d8815cdf0 2023-10-08T05:57:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 >> v1:172.21.15.70:6789/0 conn(0x7f6d8815cdf0 legacy=0x7f6d880cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:34.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d74003970 0x7f6d74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d740047c0 0x7f6d74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d7400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880cf8c0 con 0x7f6d74005180 2023-10-08T05:57:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7400c0d0 con 0x7f6d740047c0 2023-10-08T05:57:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.250+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7400c290 con 0x7f6d74003970 2023-10-08T05:57:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.251+0000 7f6d8ed6d640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d7400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.251+0000 7f6d8dd6b640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d740047c0 0x7f6d74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:34.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.251+0000 7f6d8e56c640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d74003970 0x7f6d74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:34.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.251+0000 7f6d8ed6d640 1 -- 172.21.15.70:0/4291038346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d74003970 msgr2=0x7f6d74003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:34.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.251+0000 7f6d8ed6d640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d74003970 0x7f6d74003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.251+0000 7f6d8ed6d640 1 -- 172.21.15.70:0/4291038346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d740047c0 msgr2=0x7f6d74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.251+0000 7f6d8ed6d640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d740047c0 0x7f6d74004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.251+0000 7f6d8ed6d640 1 -- 172.21.15.70:0/4291038346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d74003520 con 0x7f6d74005180 2023-10-08T05:57:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.251+0000 7f6d8e56c640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d74003970 0x7f6d74003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.252+0000 7f6d8ed6d640 1 -- 172.21.15.70:0/4291038346 reap_dead start 2023-10-08T05:57:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.252+0000 7f6d8ed6d640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d7400b980 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f6d80007330 tx=0x7f6d740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e06e2e074173951 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.252+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8002ccd0 con 0x7f6d74005180 2023-10-08T05:57:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.252+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8002ce30 con 0x7f6d74005180 2023-10-08T05:57:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.252+0000 7f6d8d56a640 1 -- 172.21.15.70:0/4291038346 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80003bd0 con 0x7f6d74005180 2023-10-08T05:57:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.253+0000 7f6d8f56e640 1 -- 172.21.15.70:0/4291038346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 msgr2=0x7f6d7400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.253+0000 7f6d8f56e640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d7400b980 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f6d80007330 tx=0x7f6d740057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.253+0000 7f6d8f56e640 1 -- 172.21.15.70:0/4291038346 shutdown_connections 2023-10-08T05:57:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.253+0000 7f6d8f56e640 1 --2- 172.21.15.70:0/4291038346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d7400b980 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.253+0000 7f6d8f56e640 1 -- 172.21.15.70:0/4291038346 >> 172.21.15.70:0/4291038346 conn(0x7f6d880d0890 msgr2=0x7f6d880d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.253+0000 7f6d8f56e640 1 -- 172.21.15.70:0/4291038346 shutdown_connections 2023-10-08T05:57:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.253+0000 7f6d8f56e640 1 -- 172.21.15.70:0/4291038346 wait complete. 2023-10-08T05:57:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.255+0000 7f6d8f56e640 1 Processor -- start 2023-10-08T05:57:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.255+0000 7f6d8f56e640 1 -- start start 2023-10-08T05:57:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.256+0000 7f6d8f56e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d881506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.256+0000 7f6d8f56e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d88150cd0 0x7f6d880c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.256+0000 7f6d8f56e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d880c43c0 0x7f6d880c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:34.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.256+0000 7f6d8f56e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880c6c70 con 0x7f6d74005180 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.256+0000 7f6d8f56e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880b61c0 con 0x7f6d88150cd0 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.256+0000 7f6d8f56e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880b35c0 con 0x7f6d880c43c0 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.256+0000 7f6d8e56c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d881506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.256+0000 7f6d8e56c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d881506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34668/0 (socket says 172.21.15.70:34668) 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.256+0000 7f6d8e56c640 1 -- 172.21.15.70:0/3658943070 learned_addr learned my addr 172.21.15.70:0/3658943070 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d8dd6b640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d88150cd0 0x7f6d880c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d8e56c640 1 -- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d880c43c0 msgr2=0x7f6d880c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d8ed6d640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d880c43c0 0x7f6d880c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d8e56c640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d880c43c0 0x7f6d880c4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d8e56c640 1 -- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d88150cd0 msgr2=0x7f6d880c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d8e56c640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d88150cd0 0x7f6d880c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d8e56c640 1 -- 172.21.15.70:0/3658943070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d880b38a0 con 0x7f6d74005180 2023-10-08T05:57:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d8dd6b640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d88150cd0 0x7f6d880c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d8e56c640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d881506b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000ff0 tx=0x7f6d8400dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d7effd640 1 -- 172.21.15.70:0/3658943070 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d840074e0 con 0x7f6d74005180 2023-10-08T05:57:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d7effd640 1 -- 172.21.15.70:0/3658943070 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d84007640 con 0x7f6d74005180 2023-10-08T05:57:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d8f56e640 1 -- 172.21.15.70:0/3658943070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d880b3b80 con 0x7f6d74005180 2023-10-08T05:57:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.257+0000 7f6d7effd640 1 -- 172.21.15.70:0/3658943070 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d840049e0 con 0x7f6d74005180 2023-10-08T05:57:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.258+0000 7f6d8f56e640 1 -- 172.21.15.70:0/3658943070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d880b40a0 con 0x7f6d74005180 2023-10-08T05:57:34.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.259+0000 7f6d8f56e640 1 -- 172.21.15.70:0/3658943070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d880b3420 con 0x7f6d74005180 2023-10-08T05:57:34.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.260+0000 7f6d7effd640 1 -- 172.21.15.70:0/3658943070 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6d84004b40 con 0x7f6d74005180 2023-10-08T05:57:34.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.261+0000 7f6d7effd640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f6d6c06afc0 0x7f6d6c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:34.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.261+0000 7f6d7effd640 1 -- 172.21.15.70:0/3658943070 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6d84089670 con 0x7f6d74005180 2023-10-08T05:57:34.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.261+0000 7f6d8dd6b640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f6d6c06afc0 0x7f6d6c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:34.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.262+0000 7f6d8dd6b640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f6d6c06afc0 0x7f6d6c06d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d78000c00 tx=0x7f6d78009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:34.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.265+0000 7f6d7effd640 1 -- 172.21.15.70:0/3658943070 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d880b3420 con 0x7f6d74005180 2023-10-08T05:57:34.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.494+0000 7f6d8f56e640 1 -- 172.21.15.70:0/3658943070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6d880b3420 con 0x7f6d74005180 2023-10-08T05:57:34.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.495+0000 7f6d7effd640 1 -- 172.21.15.70:0/3658943070 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6d880b3420 con 0x7f6d74005180 2023-10-08T05:57:34.508 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T05:57:34.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.497+0000 7f6d8f56e640 1 -- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f6d6c06afc0 msgr2=0x7f6d6c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:34.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.497+0000 7f6d8f56e640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f6d6c06afc0 0x7f6d6c06d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d78000c00 tx=0x7f6d78009040 comp rx=0 tx=0).stop 2023-10-08T05:57:34.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.497+0000 7f6d8f56e640 1 -- 172.21.15.70:0/3658943070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 msgr2=0x7f6d881506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:34.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.497+0000 7f6d8f56e640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d881506b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f6d84000ff0 tx=0x7f6d8400dc70 comp rx=0 tx=0).stop 2023-10-08T05:57:34.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.498+0000 7f6d8f56e640 1 -- 172.21.15.70:0/3658943070 shutdown_connections 2023-10-08T05:57:34.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.498+0000 7f6d8f56e640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f6d6c06afc0 0x7f6d6c06d480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.498+0000 7f6d8f56e640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d880c43c0 0x7f6d880c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.498+0000 7f6d8f56e640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d88150cd0 0x7f6d880c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.498+0000 7f6d8f56e640 1 --2- 172.21.15.70:0/3658943070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d74005180 0x7f6d881506b0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:34.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.498+0000 7f6d8f56e640 1 -- 172.21.15.70:0/3658943070 >> 172.21.15.70:0/3658943070 conn(0x7f6d880d0890 msgr2=0x7f6d8806e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:34.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.498+0000 7f6d8f56e640 1 -- 172.21.15.70:0/3658943070 shutdown_connections 2023-10-08T05:57:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:34.498+0000 7f6d8f56e640 1 -- 172.21.15.70:0/3658943070 wait complete. 2023-10-08T05:57:34.583 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T05:57:34.584 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T05:57:34.584 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T05:57:34.584 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T05:57:34.585 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T05:57:34.585 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T05:57:34.585 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T05:57:34.585 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T05:57:34.585 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:57:34.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:57:34.585 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T05:57:34.642 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T05:57:34.643 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T05:57:34.643 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T05:57:34.643 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:57:34.643 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T05:57:34.643 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:57:34.643 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:57:34.643 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:57:34.643 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T05:57:34.643 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:57:34.643 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41142sec preferred_lft 41142sec 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T05:57:34.644 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T05:57:34.645 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T05:57:34.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:57:34.646 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T05:57:34.736 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T05:57:34.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:57:34.740 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T05:57:34.796 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T05:57:34.796 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T05:57:34.796 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T05:57:34.796 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T05:57:34.796 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T05:57:34.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:57:34.797 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T05:57:34.797 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T05:57:34.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T05:57:34.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T05:57:34.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T05:57:34.797 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T05:57:34.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:57:35.119 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T05:57:35.178 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T05:57:35.198 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T05:57:35.199 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:57:35.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:57:35.200 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T05:57:35.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:57:35.354 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T05:57:35.414 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T05:57:35.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T05:57:35.424 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:57:35.477 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T05:57:35.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:57:35.511 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:57:35.659 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:57:35.672 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:57:35.693 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T05:57:35.694 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T05:57:35.695 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T05:57:35.695 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T05:57:35.695 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T05:57:35.696 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T05:57:35.696 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T05:57:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T05:57:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T05:57:35.697 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T05:57:35.700 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T05:57:35.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:57:35.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T05:57:35.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:57:35.820 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T05:57:35.893 INFO:teuthology.orchestra.run.smithi070.stdout:client9222 2023-10-08T05:57:35.893 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T05:57:35.893 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T05:57:35.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T05:57:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.190+0000 7f60b7f18640 1 Processor -- start 2023-10-08T05:57:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.191+0000 7f60b7f18640 1 -- start start 2023-10-08T05:57:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.192+0000 7f60b7f18640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60b00afbe0 0x7f60b00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.192+0000 7f60b7f18640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f60b006dfc0 0x7f60b0070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.192+0000 7f60b7f18640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f60b0071070 0x7f60b00cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.192+0000 7f60b7f18640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b00b2620 con 0x7f60b006dfc0 2023-10-08T05:57:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.192+0000 7f60b7f18640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b00b2060 con 0x7f60b0071070 2023-10-08T05:57:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.192+0000 7f60b7f18640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b00b2340 con 0x7f60b00afbe0 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b54b3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f60b006dfc0 0x7f60b0070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b54b3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f60b006dfc0 0x7f60b0070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34684/0 (socket says 172.21.15.70:34684) 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b54b3640 1 -- 172.21.15.70:0/353873736 learned_addr learned my addr 172.21.15.70:0/353873736 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b5cb4640 1 --2- 172.21.15.70:0/353873736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60b00afbe0 0x7f60b00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b64b5640 1 --2- 172.21.15.70:0/353873736 >> v2:172.21.15.181:3300/0 conn(0x7f60b0071070 0x7f60b00cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b54b3640 1 -- 172.21.15.70:0/353873736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60b00afbe0 msgr2=0x7f60b00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b54b3640 1 --2- 172.21.15.70:0/353873736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60b00afbe0 0x7f60b00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b54b3640 1 -- 172.21.15.70:0/353873736 >> v2:172.21.15.181:3300/0 conn(0x7f60b0071070 msgr2=0x7f60b00cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b54b3640 1 --2- 172.21.15.70:0/353873736 >> v2:172.21.15.181:3300/0 conn(0x7f60b0071070 0x7f60b00cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b54b3640 1 -- 172.21.15.70:0/353873736 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b0070350 con 0x7f60b006dfc0 2023-10-08T05:57:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b5cb4640 1 --2- 172.21.15.70:0/353873736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60b00afbe0 0x7f60b00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.193+0000 7f60b64b5640 1 --2- 172.21.15.70:0/353873736 >> v2:172.21.15.181:3300/0 conn(0x7f60b0071070 0x7f60b00cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.194+0000 7f60b54b3640 1 --2- 172.21.15.70:0/353873736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 0x7f60b0070b30 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f60b00b3980 tx=0x7f60a002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff9b907a7df072d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b4cb2640 1 -- 172.21.15.70:0/353873736 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0031880 con 0x7f60b006dfc0 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b4cb2640 1 -- 172.21.15.70:0/353873736 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a00319e0 con 0x7f60b006dfc0 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b4cb2640 1 -- 172.21.15.70:0/353873736 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a0031d70 con 0x7f60b006dfc0 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b7f18640 1 -- 172.21.15.70:0/353873736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 msgr2=0x7f60b0070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b7f18640 1 --2- 172.21.15.70:0/353873736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 0x7f60b0070b30 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f60b00b3980 tx=0x7f60a002b920 comp rx=0 tx=0).stop 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b7f18640 1 -- 172.21.15.70:0/353873736 shutdown_connections 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b7f18640 1 --2- 172.21.15.70:0/353873736 >> v2:172.21.15.181:3300/0 conn(0x7f60b0071070 0x7f60b00cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b7f18640 1 --2- 172.21.15.70:0/353873736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 0x7f60b0070b30 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b7f18640 1 --2- 172.21.15.70:0/353873736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60b00afbe0 0x7f60b00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b7f18640 1 -- 172.21.15.70:0/353873736 >> 172.21.15.70:0/353873736 conn(0x7f60b006a5d0 msgr2=0x7f60b00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:36.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b7f18640 1 -- 172.21.15.70:0/353873736 shutdown_connections 2023-10-08T05:57:36.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.195+0000 7f60b7f18640 1 -- 172.21.15.70:0/353873736 wait complete. 2023-10-08T05:57:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.197+0000 7f60b7f18640 1 Processor -- start 2023-10-08T05:57:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.197+0000 7f60b7f18640 1 -- start start 2023-10-08T05:57:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.198+0000 7f60b7f18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 0x7f60b00c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.198+0000 7f60b7f18640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b0071070 0x7f60b0152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.198+0000 7f60b7f18640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60b00afbe0 0x7f60b006ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.198+0000 7f60b7f18640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b00c2e10 con 0x7f60b006dfc0 2023-10-08T05:57:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.198+0000 7f60b7f18640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b00c9d60 con 0x7f60b0071070 2023-10-08T05:57:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.198+0000 7f60b7f18640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b00ca640 con 0x7f60b00afbe0 2023-10-08T05:57:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.199+0000 7f60b5cb4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 0x7f60b00c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.199+0000 7f60b5cb4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 0x7f60b00c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34692/0 (socket says 172.21.15.70:34692) 2023-10-08T05:57:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.199+0000 7f60b5cb4640 1 -- 172.21.15.70:0/1135188899 learned_addr learned my addr 172.21.15.70:0/1135188899 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.199+0000 7f60b5cb4640 1 -- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60b00afbe0 msgr2=0x7f60b006ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.199+0000 7f60b5cb4640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60b00afbe0 0x7f60b006ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.199+0000 7f60b5cb4640 1 -- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b0071070 msgr2=0x7f60b0152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:36.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.199+0000 7f60b5cb4640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b0071070 0x7f60b0152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.199+0000 7f60b5cb4640 1 -- 172.21.15.70:0/1135188899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a00094d0 con 0x7f60b006dfc0 2023-10-08T05:57:36.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.199+0000 7f60b5cb4640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 0x7f60b00c4cf0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00c7f0 tx=0x7f60ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:36.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.199+0000 7f60a67fc640 1 -- 172.21.15.70:0/1135188899 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac007a30 con 0x7f60b006dfc0 2023-10-08T05:57:36.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.200+0000 7f60a67fc640 1 -- 172.21.15.70:0/1135188899 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac007b90 con 0x7f60b006dfc0 2023-10-08T05:57:36.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.200+0000 7f60a67fc640 1 -- 172.21.15.70:0/1135188899 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00ce40 con 0x7f60b006dfc0 2023-10-08T05:57:36.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.200+0000 7f60b7f18640 1 -- 172.21.15.70:0/1135188899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b00c91e0 con 0x7f60b006dfc0 2023-10-08T05:57:36.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.200+0000 7f60b7f18640 1 -- 172.21.15.70:0/1135188899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b00c97a0 con 0x7f60b006dfc0 2023-10-08T05:57:36.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.201+0000 7f60b7f18640 1 -- 172.21.15.70:0/1135188899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b00c2f80 con 0x7f60b006dfc0 2023-10-08T05:57:36.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.202+0000 7f60a67fc640 1 -- 172.21.15.70:0/1135188899 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f60ac0113f0 con 0x7f60b006dfc0 2023-10-08T05:57:36.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.204+0000 7f60a67fc640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f609406af90 0x7f609406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.204+0000 7f60a67fc640 1 -- 172.21.15.70:0/1135188899 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f60ac089c70 con 0x7f60b006dfc0 2023-10-08T05:57:36.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.204+0000 7f60b54b3640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f609406af90 0x7f609406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:36.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.205+0000 7f60b54b3640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f609406af90 0x7f609406d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f60a0036000 tx=0x7f60a0030000 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:36.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.208+0000 7f60a67fc640 1 -- 172.21.15.70:0/1135188899 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60ac0548d0 con 0x7f60b006dfc0 2023-10-08T05:57:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.419+0000 7f60b7f18640 1 -- 172.21.15.70:0/1135188899 --> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f60b00affc0 con 0x7f609406af90 2023-10-08T05:57:36.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.424+0000 7f60a67fc640 1 -- 172.21.15.70:0/1135188899 <== mgr.6299 v2:172.21.15.181:6832/2068783299 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f60a003a420 con 0x7f609406af90 2023-10-08T05:57:36.438 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T05:57:36.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T05:57:36.438 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T05:57:36.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T05:57:36.438 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T05:57:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.425+0000 7f60b7f18640 1 -- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f609406af90 msgr2=0x7f609406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.425+0000 7f60b7f18640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f609406af90 0x7f609406d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f60a0036000 tx=0x7f60a0030000 comp rx=0 tx=0).stop 2023-10-08T05:57:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.426+0000 7f60b7f18640 1 -- 172.21.15.70:0/1135188899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 msgr2=0x7f60b00c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.426+0000 7f60b7f18640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 0x7f60b00c4cf0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00c7f0 tx=0x7f60ac00cbf0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.426+0000 7f60b7f18640 1 -- 172.21.15.70:0/1135188899 shutdown_connections 2023-10-08T05:57:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.426+0000 7f60b7f18640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f609406af90 0x7f609406d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.426+0000 7f60b7f18640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60b00afbe0 0x7f60b006ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.426+0000 7f60b7f18640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60b0071070 0x7f60b0152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.426+0000 7f60b7f18640 1 --2- 172.21.15.70:0/1135188899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b006dfc0 0x7f60b00c4cf0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.426+0000 7f60b7f18640 1 -- 172.21.15.70:0/1135188899 >> 172.21.15.70:0/1135188899 conn(0x7f60b006a5d0 msgr2=0x7f60b00c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.426+0000 7f60b7f18640 1 -- 172.21.15.70:0/1135188899 shutdown_connections 2023-10-08T05:57:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.426+0000 7f60b7f18640 1 -- 172.21.15.70:0/1135188899 wait complete. 2023-10-08T05:57:36.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T05:57:36.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.660+0000 7f8447421640 1 Processor -- start 2023-10-08T05:57:36.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.660+0000 7f8447421640 1 -- start start 2023-10-08T05:57:36.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.662+0000 7f8447421640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f84400b1750 0x7f844006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.662+0000 7f8447421640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f844006d240 0x7f8440070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.662+0000 7f8447421640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f844006d670 con 0x7f844015aad0 2023-10-08T05:57:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.662+0000 7f8447421640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844015bf30 con 0x7f844006d240 2023-10-08T05:57:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.662+0000 7f8447421640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84400b1bb0 con 0x7f84400b1750 2023-10-08T05:57:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.662+0000 7f84451bd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f844015aad0 0x7f84400c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38498/0 (socket says 172.21.15.70:38498) 2023-10-08T05:57:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.662+0000 7f84451bd640 1 -- 172.21.15.70:0/74665341 learned_addr learned my addr 172.21.15.70:0/74665341 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:36.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.662+0000 7f84459be640 1 --2- 172.21.15.70:0/74665341 >> v2:172.21.15.70:3300/0 conn(0x7f844006d240 0x7f8440070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.662+0000 7f84449bc640 1 --2- 172.21.15.70:0/74665341 >> v2:172.21.15.181:3300/0 conn(0x7f84400b1750 0x7f844006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.663+0000 7f8437fff640 1 -- 172.21.15.70:0/74665341 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2440582724 0 0) 0x7f844006d670 con 0x7f844015aad0 2023-10-08T05:57:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.663+0000 7f84459be640 1 -- 172.21.15.70:0/74665341 >> v2:172.21.15.181:3300/0 conn(0x7f84400b1750 msgr2=0x7f844006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.663+0000 7f84459be640 1 --2- 172.21.15.70:0/74665341 >> v2:172.21.15.181:3300/0 conn(0x7f84400b1750 0x7f844006cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.663+0000 7f84459be640 1 -- 172.21.15.70:0/74665341 >> v1:172.21.15.70:6789/0 conn(0x7f844015aad0 legacy=0x7f84400c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.663+0000 7f84459be640 1 -- 172.21.15.70:0/74665341 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844015b320 con 0x7f844006d240 2023-10-08T05:57:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.664+0000 7f84459be640 1 --2- 172.21.15.70:0/74665341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f844006d240 0x7f8440070570 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f844006e330 tx=0x7f843802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306b70caf9c730a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.664+0000 7f8437fff640 1 -- 172.21.15.70:0/74665341 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843802cc70 con 0x7f844006d240 2023-10-08T05:57:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.664+0000 7f8437fff640 1 -- 172.21.15.70:0/74665341 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843802cdd0 con 0x7f844006d240 2023-10-08T05:57:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.664+0000 7f8437fff640 1 -- 172.21.15.70:0/74665341 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438034660 con 0x7f844006d240 2023-10-08T05:57:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.665+0000 7f8447421640 1 -- 172.21.15.70:0/74665341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f844006d240 msgr2=0x7f8440070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.665+0000 7f8447421640 1 --2- 172.21.15.70:0/74665341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f844006d240 0x7f8440070570 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f844006e330 tx=0x7f843802bec0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.665+0000 7f8447421640 1 -- 172.21.15.70:0/74665341 shutdown_connections 2023-10-08T05:57:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.665+0000 7f8447421640 1 --2- 172.21.15.70:0/74665341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f844006d240 0x7f8440070570 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.665+0000 7f8447421640 1 --2- 172.21.15.70:0/74665341 >> v2:172.21.15.181:3300/0 conn(0x7f84400b1750 0x7f844006cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.665+0000 7f8447421640 1 -- 172.21.15.70:0/74665341 >> 172.21.15.70:0/74665341 conn(0x7f8440069d50 msgr2=0x7f844006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.665+0000 7f8447421640 1 -- 172.21.15.70:0/74665341 shutdown_connections 2023-10-08T05:57:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.665+0000 7f8447421640 1 -- 172.21.15.70:0/74665341 wait complete. 2023-10-08T05:57:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.666+0000 7f8447421640 1 Processor -- start 2023-10-08T05:57:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.667+0000 7f8447421640 1 -- start start 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.667+0000 7f8447421640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844006d240 0x7f84400c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.667+0000 7f8447421640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84400b1750 0x7f84401511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f8447421640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f844015aad0 0x7f844015d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f8447421640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84400c6550 con 0x7f84400b1750 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f8447421640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84400c5110 con 0x7f844006d240 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f8447421640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84400c4520 con 0x7f844015aad0 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f84449bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84400b1750 0x7f84401511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f84449bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84400b1750 0x7f84401511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34706/0 (socket says 172.21.15.70:34706) 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f84449bc640 1 -- 172.21.15.70:0/3193423726 learned_addr learned my addr 172.21.15.70:0/3193423726 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f84451bd640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844006d240 0x7f84400c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f84449bc640 1 -- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f844015aad0 msgr2=0x7f844015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f84449bc640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f844015aad0 0x7f844015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f84449bc640 1 -- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844006d240 msgr2=0x7f84400c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f84449bc640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844006d240 0x7f84400c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.668+0000 7f84449bc640 1 -- 172.21.15.70:0/3193423726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8438009a70 con 0x7f84400b1750 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.669+0000 7f84449bc640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84400b1750 0x7f84401511a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f843000d800 tx=0x7f843000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.669+0000 7f8435ffb640 1 -- 172.21.15.70:0/3193423726 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843000b6c0 con 0x7f84400b1750 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.669+0000 7f8435ffb640 1 -- 172.21.15.70:0/3193423726 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843000b820 con 0x7f84400b1750 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.669+0000 7f8435ffb640 1 -- 172.21.15.70:0/3193423726 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843000bbb0 con 0x7f84400b1750 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.669+0000 7f8447421640 1 -- 172.21.15.70:0/3193423726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84400c47e0 con 0x7f84400b1750 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.669+0000 7f8447421640 1 -- 172.21.15.70:0/3193423726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84400c4da0 con 0x7f84400b1750 2023-10-08T05:57:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.670+0000 7f8447421640 1 -- 172.21.15.70:0/3193423726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84400c4f60 con 0x7f84400b1750 2023-10-08T05:57:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.671+0000 7f8435ffb640 1 -- 172.21.15.70:0/3193423726 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 77) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f843000bd10 con 0x7f84400b1750 2023-10-08T05:57:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.673+0000 7f8435ffb640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f842406afc0 0x7f842406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.673+0000 7f8435ffb640 1 -- 172.21.15.70:0/3193423726 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8430089e50 con 0x7f84400b1750 2023-10-08T05:57:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.673+0000 7f84451bd640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f842406afc0 0x7f842406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.674+0000 7f84451bd640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f842406afc0 0x7f842406d480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f843c000f60 tx=0x7f843c009040 comp rx=0 tx=0).ready entity=mgr.6299 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.677+0000 7f8435ffb640 1 -- 172.21.15.70:0/3193423726 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8430054c60 con 0x7f84400b1750 2023-10-08T05:57:36.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:36.914+0000 7f8447421640 1 -- 172.21.15.70:0/3193423726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f84400c6690 con 0x7f84400b1750 2023-10-08T05:57:37.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.466+0000 7f8435ffb640 1 -- 172.21.15.70:0/3193423726 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v78) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8430058280 con 0x7f84400b1750 2023-10-08T05:57:37.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.469+0000 7f8447421640 1 -- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f842406afc0 msgr2=0x7f842406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:37.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.469+0000 7f8447421640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f842406afc0 0x7f842406d480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f843c000f60 tx=0x7f843c009040 comp rx=0 tx=0).stop 2023-10-08T05:57:37.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.469+0000 7f8447421640 1 -- 172.21.15.70:0/3193423726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84400b1750 msgr2=0x7f84401511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:37.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.469+0000 7f8447421640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84400b1750 0x7f84401511a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f843000d800 tx=0x7f843000dcc0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.470+0000 7f8447421640 1 -- 172.21.15.70:0/3193423726 shutdown_connections 2023-10-08T05:57:37.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.470+0000 7f8447421640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f842406afc0 0x7f842406d480 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.470+0000 7f8447421640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f844015aad0 0x7f844015d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.470+0000 7f8447421640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84400b1750 0x7f84401511a0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.470+0000 7f8447421640 1 --2- 172.21.15.70:0/3193423726 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f844006d240 0x7f84400c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.470+0000 7f8447421640 1 -- 172.21.15.70:0/3193423726 >> 172.21.15.70:0/3193423726 conn(0x7f8440069d50 msgr2=0x7f8440154cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:37.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.471+0000 7f8447421640 1 -- 172.21.15.70:0/3193423726 shutdown_connections 2023-10-08T05:57:37.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.471+0000 7f8447421640 1 -- 172.21.15.70:0/3193423726 wait complete. 2023-10-08T05:57:37.496 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T05:57:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.722+0000 7f0fb02fc640 1 Processor -- start 2023-10-08T05:57:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.722+0000 7f0fb02fc640 1 -- start start 2023-10-08T05:57:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.723+0000 7f0fb02fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fa815cba0 0x7f0fa80b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.723+0000 7f0fb02fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 0x7f0fa80d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.723+0000 7f0fb02fc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa80b54a0 con 0x7f0fa815cf80 2023-10-08T05:57:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.723+0000 7f0fb02fc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa80b7130 con 0x7f0fa815cba0 2023-10-08T05:57:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.723+0000 7f0fb02fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80b7410 con 0x7f0fa80b4d10 2023-10-08T05:57:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.724+0000 7f0fae098640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fa815cba0 0x7f0fa80b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.724+0000 7f0fae098640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fa815cba0 0x7f0fa80b84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34730/0 (socket says 172.21.15.70:34730) 2023-10-08T05:57:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.724+0000 7f0fae098640 1 -- 172.21.15.70:0/3464541902 learned_addr learned my addr 172.21.15.70:0/3464541902 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.724+0000 7f0fad897640 1 --1- 172.21.15.70:0/3464541902 >> v1:172.21.15.70:6789/0 conn(0x7f0fa815cf80 0x7f0fa80b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38514/0 (socket says 172.21.15.70:38514) 2023-10-08T05:57:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.724+0000 7f0fae899640 1 --2- 172.21.15.70:0/3464541902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 0x7f0fa80d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.724+0000 7f0fae098640 1 -- 172.21.15.70:0/3464541902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 msgr2=0x7f0fa80d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.724+0000 7f0fae098640 1 --2- 172.21.15.70:0/3464541902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 0x7f0fa80d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.724+0000 7f0fae098640 1 -- 172.21.15.70:0/3464541902 >> v1:172.21.15.70:6789/0 conn(0x7f0fa815cf80 legacy=0x7f0fa80b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.724+0000 7f0fae098640 1 -- 172.21.15.70:0/3464541902 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa80b76f0 con 0x7f0fa815cba0 2023-10-08T05:57:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.724+0000 7f0fad096640 1 -- 172.21.15.70:0/3464541902 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546377621 0 0) 0x7f0f98001110 con 0x7f0fa815cf80 2023-10-08T05:57:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.725+0000 7f0fae098640 1 --2- 172.21.15.70:0/3464541902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 0x7f0fa80b84a0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40098a0 tx=0x7f0fa402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862188f0eec73dcb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.726+0000 7f0fad096640 1 -- 172.21.15.70:0/3464541902 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4030840 con 0x7f0fa815cba0 2023-10-08T05:57:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.726+0000 7f0fad096640 1 -- 172.21.15.70:0/3464541902 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa40309a0 con 0x7f0fa815cba0 2023-10-08T05:57:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.726+0000 7f0fad096640 1 -- 172.21.15.70:0/3464541902 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4030d30 con 0x7f0fa815cba0 2023-10-08T05:57:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.726+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3464541902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 msgr2=0x7f0fa80b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.726+0000 7f0fb02fc640 1 --2- 172.21.15.70:0/3464541902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 0x7f0fa80b84a0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40098a0 tx=0x7f0fa402b980 comp rx=0 tx=0).stop 2023-10-08T05:57:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.727+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3464541902 shutdown_connections 2023-10-08T05:57:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.727+0000 7f0fb02fc640 1 --2- 172.21.15.70:0/3464541902 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 0x7f0fa80d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.727+0000 7f0fb02fc640 1 --2- 172.21.15.70:0/3464541902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 0x7f0fa80b84a0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.727+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3464541902 >> 172.21.15.70:0/3464541902 conn(0x7f0fa80718a0 msgr2=0x7f0fa8071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.727+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3464541902 shutdown_connections 2023-10-08T05:57:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.727+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3464541902 wait complete. 2023-10-08T05:57:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.728+0000 7f0fb02fc640 1 Processor -- start 2023-10-08T05:57:37.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.729+0000 7f0fb02fc640 1 -- start start 2023-10-08T05:57:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fb02fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 0x7f0fa81e2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fb02fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 0x7f0fa81e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fb02fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa815cf80 0x7f0fa81e74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fb02fc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa81e2b90 con 0x7f0fa815cba0 2023-10-08T05:57:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fb02fc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa81e2e70 con 0x7f0fa815cf80 2023-10-08T05:57:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fb02fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa81e3150 con 0x7f0fa80b4d10 2023-10-08T05:57:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fad897640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 0x7f0fa81e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fad897640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 0x7f0fa81e6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34740/0 (socket says 172.21.15.70:34740) 2023-10-08T05:57:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fad897640 1 -- 172.21.15.70:0/3885406714 learned_addr learned my addr 172.21.15.70:0/3885406714 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fae098640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 0x7f0fa81e2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:37.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.730+0000 7f0fae899640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa815cf80 0x7f0fa81e74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0fad897640 1 -- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 msgr2=0x7f0fa81e2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0fad897640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 0x7f0fa81e2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0fad897640 1 -- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa815cf80 msgr2=0x7f0fa81e74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0fad897640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa815cf80 0x7f0fa81e74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0fad897640 1 -- 172.21.15.70:0/3885406714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa40094d0 con 0x7f0fa815cba0 2023-10-08T05:57:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0fae098640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 0x7f0fa81e2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:57:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0fad897640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 0x7f0fa81e6f80 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800c780 tx=0x7f0f9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0f9effd640 1 -- 172.21.15.70:0/3885406714 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980072d0 con 0x7f0fa815cba0 2023-10-08T05:57:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0f9effd640 1 -- 172.21.15.70:0/3885406714 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f98007450 con 0x7f0fa815cba0 2023-10-08T05:57:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0f9effd640 1 -- 172.21.15.70:0/3885406714 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980047e0 con 0x7f0fa815cba0 2023-10-08T05:57:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3885406714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa81e3430 con 0x7f0fa815cba0 2023-10-08T05:57:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.731+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3885406714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa81e3c30 con 0x7f0fa815cba0 2023-10-08T05:57:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.733+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3885406714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa804e160 con 0x7f0fa815cba0 2023-10-08T05:57:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.734+0000 7f0f9effd640 1 -- 172.21.15.70:0/3885406714 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 78) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f0f98004960 con 0x7f0fa815cba0 2023-10-08T05:57:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.735+0000 7f0f9effd640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f0f8c06afe0 0x7f0f8c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.735+0000 7f0f9effd640 1 -- 172.21.15.70:0/3885406714 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f98088b20 con 0x7f0fa815cba0 2023-10-08T05:57:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.735+0000 7f0fae098640 1 -- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f0f8c06afe0 msgr2=0x7f0f8c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2068783299 2023-10-08T05:57:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.735+0000 7f0fae098640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f0f8c06afe0 0x7f0f8c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:57:37.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.740+0000 7f0f9effd640 1 -- 172.21.15.70:0/3885406714 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f98053920 con 0x7f0fa815cba0 2023-10-08T05:57:37.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.934+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3885406714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f0fa80b54a0 con 0x7f0fa815cba0 2023-10-08T05:57:37.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.936+0000 7f0fae098640 1 -- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f0f8c06afe0 msgr2=0x7f0f8c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2068783299 2023-10-08T05:57:37.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.936+0000 7f0fae098640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f0f8c06afe0 0x7f0f8c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:57:37.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.939+0000 7f0f9effd640 1 -- 172.21.15.70:0/3885406714 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0fa80b54a0 con 0x7f0fa815cba0 2023-10-08T05:57:37.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f0f8c06afe0 msgr2=0x7f0f8c06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:57:37.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f0f8c06afe0 0x7f0f8c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3885406714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 msgr2=0x7f0fa81e6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:37.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 0x7f0fa81e6f80 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800c780 tx=0x7f0f9800cc40 comp rx=0 tx=0).stop 2023-10-08T05:57:37.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3885406714 shutdown_connections 2023-10-08T05:57:37.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7f0f8c06afe0 0x7f0f8c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fa815cf80 0x7f0fa81e74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fa815cba0 0x7f0fa81e6f80 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 --2- 172.21.15.70:0/3885406714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fa80b4d10 0x7f0fa81e2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:37.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3885406714 >> 172.21.15.70:0/3885406714 conn(0x7f0fa80718a0 msgr2=0x7f0fa80d25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:37.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3885406714 shutdown_connections 2023-10-08T05:57:37.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:37.942+0000 7f0fb02fc640 1 -- 172.21.15.70:0/3885406714 wait complete. 2023-10-08T05:57:37.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T05:57:37.982 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:37.981+0000 7f9f76656040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:57:38.034 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:38.033+0000 7f9f76656040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:57:38.042 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:38.041+0000 7fbf11a1b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T05:57:38.097 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:38.096+0000 7fbf11a1b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T05:57:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.161+0000 7fa81bfff640 1 Processor -- start 2023-10-08T05:57:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.161+0000 7fa81bfff640 1 -- start start 2023-10-08T05:57:38.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.163+0000 7fa81bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa81c0ce290 0x7fa81c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.163+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa81c065ab0 0x7fa81c1608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.163+0000 7fa81bfff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81c069d50 con 0x7fa81c15dde0 2023-10-08T05:57:38.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.163+0000 7fa81bfff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81c06e6c0 con 0x7fa81c0ce290 2023-10-08T05:57:38.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.163+0000 7fa81bfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c06c750 con 0x7fa81c065ab0 2023-10-08T05:57:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.163+0000 7fa81affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa81c0ce290 0x7fa81c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.163+0000 7fa81affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa81c0ce290 0x7fa81c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34746/0 (socket says 172.21.15.70:34746) 2023-10-08T05:57:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.163+0000 7fa81affd640 1 -- 172.21.15.70:0/3657839964 learned_addr learned my addr 172.21.15.70:0/3657839964 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.163+0000 7fa81a7fc640 1 --2- 172.21.15.70:0/3657839964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa81c065ab0 0x7fa81c1608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.164+0000 7fa81affd640 1 -- 172.21.15.70:0/3657839964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa81c065ab0 msgr2=0x7fa81c1608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.164+0000 7fa81affd640 1 --2- 172.21.15.70:0/3657839964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa81c065ab0 0x7fa81c1608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.164+0000 7fa81affd640 1 -- 172.21.15.70:0/3657839964 >> v1:172.21.15.181:6789/0 conn(0x7fa81c15dde0 legacy=0x7fa81c160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:38.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.164+0000 7fa81affd640 1 -- 172.21.15.70:0/3657839964 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0cebb0 con 0x7fa81c0ce290 2023-10-08T05:57:38.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.165+0000 7fa81affd640 1 --2- 172.21.15.70:0/3657839964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 0x7fa81c0ce670 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009870 tx=0x7fa80c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0e3288a9f7160f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:38.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.165+0000 7fa819ffb640 1 -- 172.21.15.70:0/3657839964 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c031640 con 0x7fa81c0ce290 2023-10-08T05:57:38.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.165+0000 7fa819ffb640 1 -- 172.21.15.70:0/3657839964 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c0317a0 con 0x7fa81c0ce290 2023-10-08T05:57:38.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.165+0000 7fa819ffb640 1 -- 172.21.15.70:0/3657839964 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c031b30 con 0x7fa81c0ce290 2023-10-08T05:57:38.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.166+0000 7fa81bfff640 1 -- 172.21.15.70:0/3657839964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 msgr2=0x7fa81c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:38.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.166+0000 7fa81bfff640 1 --2- 172.21.15.70:0/3657839964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 0x7fa81c0ce670 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009870 tx=0x7fa80c02b8f0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.166+0000 7fa81bfff640 1 -- 172.21.15.70:0/3657839964 shutdown_connections 2023-10-08T05:57:38.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.166+0000 7fa81bfff640 1 --2- 172.21.15.70:0/3657839964 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa81c065ab0 0x7fa81c1608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.166+0000 7fa81bfff640 1 --2- 172.21.15.70:0/3657839964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 0x7fa81c0ce670 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.166+0000 7fa81bfff640 1 -- 172.21.15.70:0/3657839964 >> 172.21.15.70:0/3657839964 conn(0x7fa81c0b08d0 msgr2=0x7fa81c0b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:38.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.166+0000 7fa81bfff640 1 -- 172.21.15.70:0/3657839964 shutdown_connections 2023-10-08T05:57:38.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.166+0000 7fa81bfff640 1 -- 172.21.15.70:0/3657839964 wait complete. 2023-10-08T05:57:38.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.168+0000 7fa81bfff640 1 Processor -- start 2023-10-08T05:57:38.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.168+0000 7fa81bfff640 1 -- start start 2023-10-08T05:57:38.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa81c065ab0 0x7fa81c156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 0x7fa81c09ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa81c15dde0 0x7fa81c1f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81bfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0c4280 con 0x7fa81c0ce290 2023-10-08T05:57:38.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81bfff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c1572c0 con 0x7fa81c15dde0 2023-10-08T05:57:38.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81bfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c1f3260 con 0x7fa81c065ab0 2023-10-08T05:57:38.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 0x7fa81c09ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:38.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 0x7fa81c09ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34750/0 (socket says 172.21.15.70:34750) 2023-10-08T05:57:38.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81a7fc640 1 -- 172.21.15.70:0/578258880 learned_addr learned my addr 172.21.15.70:0/578258880 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:38.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81affd640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa81c065ab0 0x7fa81c156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:38.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.169+0000 7fa81b7fe640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa81c15dde0 0x7fa81c1f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:38.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.170+0000 7fa81a7fc640 1 -- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa81c065ab0 msgr2=0x7fa81c156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:38.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.170+0000 7fa81a7fc640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa81c065ab0 0x7fa81c156d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.170+0000 7fa81a7fc640 1 -- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa81c15dde0 msgr2=0x7fa81c1f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:38.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.170+0000 7fa81a7fc640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa81c15dde0 0x7fa81c1f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.170+0000 7fa81a7fc640 1 -- 172.21.15.70:0/578258880 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c0094d0 con 0x7fa81c0ce290 2023-10-08T05:57:38.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.170+0000 7fa81a7fc640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 0x7fa81c09ce60 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa81000d5d0 tx=0x7fa81000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:38.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.170+0000 7fa8037fe640 1 -- 172.21.15.70:0/578258880 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810004240 con 0x7fa81c0ce290 2023-10-08T05:57:38.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.170+0000 7fa8037fe640 1 -- 172.21.15.70:0/578258880 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8100043a0 con 0x7fa81c0ce290 2023-10-08T05:57:38.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.170+0000 7fa8037fe640 1 -- 172.21.15.70:0/578258880 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810003830 con 0x7fa81c0ce290 2023-10-08T05:57:38.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.170+0000 7fa81bfff640 1 -- 172.21.15.70:0/578258880 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c1f3540 con 0x7fa81c0ce290 2023-10-08T05:57:38.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.171+0000 7fa81bfff640 1 -- 172.21.15.70:0/578258880 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c0c4770 con 0x7fa81c0ce290 2023-10-08T05:57:38.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.172+0000 7fa81bfff640 1 -- 172.21.15.70:0/578258880 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81c1f36d0 con 0x7fa81c0ce290 2023-10-08T05:57:38.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.174+0000 7fa8037fe640 1 -- 172.21.15.70:0/578258880 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 78) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fa810003990 con 0x7fa81c0ce290 2023-10-08T05:57:38.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.174+0000 7fa8037fe640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa80806afe0 0x7fa80806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.174+0000 7fa8037fe640 1 -- 172.21.15.70:0/578258880 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa810089c60 con 0x7fa81c0ce290 2023-10-08T05:57:38.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.175+0000 7fa81affd640 1 -- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa80806afe0 msgr2=0x7fa80806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2068783299 2023-10-08T05:57:38.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.175+0000 7fa81affd640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa80806afe0 0x7fa80806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:57:38.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.179+0000 7fa8037fe640 1 -- 172.21.15.70:0/578258880 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8100549d0 con 0x7fa81c0ce290 2023-10-08T05:57:38.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.367+0000 7fa81bfff640 1 -- 172.21.15.70:0/578258880 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fa81c074150 con 0x7fa81c0ce290 2023-10-08T05:57:38.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.372+0000 7fa8037fe640 1 -- 172.21.15.70:0/578258880 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa810057ff0 con 0x7fa81c0ce290 2023-10-08T05:57:38.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.374+0000 7fa81bfff640 1 -- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa80806afe0 msgr2=0x7fa80806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:57:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.374+0000 7fa81bfff640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa80806afe0 0x7fa80806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.374+0000 7fa81bfff640 1 -- 172.21.15.70:0/578258880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 msgr2=0x7fa81c09ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.374+0000 7fa81bfff640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 0x7fa81c09ce60 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa81000d5d0 tx=0x7fa81000da90 comp rx=0 tx=0).stop 2023-10-08T05:57:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.374+0000 7fa81bfff640 1 -- 172.21.15.70:0/578258880 shutdown_connections 2023-10-08T05:57:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.374+0000 7fa81bfff640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fa80806afe0 0x7fa80806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.374+0000 7fa81bfff640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa81c15dde0 0x7fa81c1f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.374+0000 7fa81bfff640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa81c0ce290 0x7fa81c09ce60 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.374+0000 7fa81bfff640 1 --2- 172.21.15.70:0/578258880 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa81c065ab0 0x7fa81c156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.374+0000 7fa81bfff640 1 -- 172.21.15.70:0/578258880 >> 172.21.15.70:0/578258880 conn(0x7fa81c0b08d0 msgr2=0x7fa81c06a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:38.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.375+0000 7fa81bfff640 1 -- 172.21.15.70:0/578258880 shutdown_connections 2023-10-08T05:57:38.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.375+0000 7fa81bfff640 1 -- 172.21.15.70:0/578258880 wait complete. 2023-10-08T05:57:38.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T05:57:38.395 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T05:57:38.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1 1m 2023-10-08T05:57:38.479 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:38.478+0000 7f9f76656040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:57:38.530 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:38.529+0000 7f9f76656040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:57:38.573 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:38.572+0000 7fbf11a1b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T05:57:38.580 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:38.579+0000 7f9f76656040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:57:38.629 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:38.627+0000 7fbf11a1b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T05:57:38.641 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:38.640+0000 7f9f76656040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:57:38.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.656+0000 7fab8e0c2640 1 Processor -- start 2023-10-08T05:57:38.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.656+0000 7fab8e0c2640 1 -- start start 2023-10-08T05:57:38.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.657+0000 7fab8e0c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab880cb130 0x7fab880cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.657+0000 7fab8e0c2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab880b1cc0 con 0x7fab880afbe0 2023-10-08T05:57:38.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.657+0000 7fab8e0c2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab880cbad0 con 0x7fab8806dfc0 2023-10-08T05:57:38.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.657+0000 7fab8e0c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab880b2620 con 0x7fab880cb130 2023-10-08T05:57:38.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.658+0000 7fab877fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fab880afbe0 0x7fab880b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38516/0 (socket says 172.21.15.70:38516) 2023-10-08T05:57:38.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.658+0000 7fab877fe640 1 -- 172.21.15.70:0/2570408798 learned_addr learned my addr 172.21.15.70:0/2570408798 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:38.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.658+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777219076 0 0) 0x7fab880b1cc0 con 0x7fab880afbe0 2023-10-08T05:57:38.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.658+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab74003520 con 0x7fab880afbe0 2023-10-08T05:57:38.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.658+0000 7fab87fff640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab880cb130 0x7fab880cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:38.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.658+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3091037245 0 0) 0x7fab74003520 con 0x7fab880afbe0 2023-10-08T05:57:38.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.658+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab880cb130 msgr2=0x7fab880cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:38.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.658+0000 7fab86ffd640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab880cb130 0x7fab880cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.658+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 >> v1:172.21.15.181:6789/0 conn(0x7fab8806dfc0 legacy=0x7fab88070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.658+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab880b2060 con 0x7fab880afbe0 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fab78001670 con 0x7fab880afbe0 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 >> v1:172.21.15.70:6789/0 conn(0x7fab880afbe0 legacy=0x7fab880b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab86ffd640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab740039a0 0x7fab74003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab86ffd640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab880cb130 0x7fab740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab86ffd640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 0x7fab74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab880b2620 con 0x7fab74004d00 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7400bae0 con 0x7fab740039a0 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7400bca0 con 0x7fab880cb130 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab87fff640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 0x7fab74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:38.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab877fe640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab740039a0 0x7fab74003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:38.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab87fff640 1 -- 172.21.15.70:0/2570408798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab880cb130 msgr2=0x7fab740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:38.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab87fff640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab880cb130 0x7fab740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab87fff640 1 -- 172.21.15.70:0/2570408798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab740039a0 msgr2=0x7fab74003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:38.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab87fff640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab740039a0 0x7fab74003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.659+0000 7fab87fff640 1 -- 172.21.15.70:0/2570408798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab74003520 con 0x7fab74004d00 2023-10-08T05:57:38.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.660+0000 7fab87fff640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 0x7fab74005180 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fab80007360 tx=0x7fab740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ebcefbf91882b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:38.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.661+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8002ccf0 con 0x7fab74004d00 2023-10-08T05:57:38.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.661+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8002ce50 con 0x7fab74004d00 2023-10-08T05:57:38.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.661+0000 7fab86ffd640 1 -- 172.21.15.70:0/2570408798 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab80034660 con 0x7fab74004d00 2023-10-08T05:57:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.661+0000 7fab8e0c2640 1 -- 172.21.15.70:0/2570408798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 msgr2=0x7fab74005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.661+0000 7fab8e0c2640 1 --2- 172.21.15.70:0/2570408798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 0x7fab74005180 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fab80007360 tx=0x7fab740057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.661+0000 7fab87fff640 1 -- 172.21.15.70:0/2570408798 reap_dead start 2023-10-08T05:57:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.661+0000 7fab8e0c2640 1 -- 172.21.15.70:0/2570408798 shutdown_connections 2023-10-08T05:57:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.661+0000 7fab8e0c2640 1 -- 172.21.15.70:0/2570408798 >> 172.21.15.70:0/2570408798 conn(0x7fab8806a5d0 msgr2=0x7fab880b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.661+0000 7fab8e0c2640 1 -- 172.21.15.70:0/2570408798 shutdown_connections 2023-10-08T05:57:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.661+0000 7fab8e0c2640 1 -- 172.21.15.70:0/2570408798 wait complete. 2023-10-08T05:57:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.663+0000 7fab8e0c2640 1 Processor -- start 2023-10-08T05:57:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.663+0000 7fab8e0c2640 1 -- start start 2023-10-08T05:57:38.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.664+0000 7fab8e0c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 0x7fab8815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.664+0000 7fab8e0c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab880c2e70 0x7fab880c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.664+0000 7fab8e0c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8815b4f0 0x7fab8815b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.664+0000 7fab8e0c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab880c3830 con 0x7fab74004d00 2023-10-08T05:57:38.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.664+0000 7fab8e0c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8815cde0 con 0x7fab880c2e70 2023-10-08T05:57:38.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.664+0000 7fab8e0c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab880c5260 con 0x7fab8815b4f0 2023-10-08T05:57:38.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.664+0000 7fab877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 0x7fab8815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:38.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.664+0000 7fab877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 0x7fab8815c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34772/0 (socket says 172.21.15.70:34772) 2023-10-08T05:57:38.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.664+0000 7fab877fe640 1 -- 172.21.15.70:0/3595058431 learned_addr learned my addr 172.21.15.70:0/3595058431 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab7ffff640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab880c2e70 0x7fab880c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab877fe640 1 -- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8815b4f0 msgr2=0x7fab8815b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab877fe640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8815b4f0 0x7fab8815b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab877fe640 1 -- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab880c2e70 msgr2=0x7fab880c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab877fe640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab880c2e70 0x7fab880c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab877fe640 1 -- 172.21.15.70:0/3595058431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8815d510 con 0x7fab74004d00 2023-10-08T05:57:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab7ffff640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab880c2e70 0x7fab880c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T05:57:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab877fe640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 0x7fab8815c8a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fab7800e790 tx=0x7fab7800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7800eea0 con 0x7fab74004d00 2023-10-08T05:57:38.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab7800c8d0 con 0x7fab74004d00 2023-10-08T05:57:38.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab7800cc60 con 0x7fab74004d00 2023-10-08T05:57:38.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab8e0c2640 1 -- 172.21.15.70:0/3595058431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8815d7f0 con 0x7fab74004d00 2023-10-08T05:57:38.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.665+0000 7fab8e0c2640 1 -- 172.21.15.70:0/3595058431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8815ddb0 con 0x7fab74004d00 2023-10-08T05:57:38.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.666+0000 7fab8e0c2640 1 -- 172.21.15.70:0/3595058431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8804e160 con 0x7fab74004d00 2023-10-08T05:57:38.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.668+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 78) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fab7801d460 con 0x7fab74004d00 2023-10-08T05:57:38.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.669+0000 7fab84ff9640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 0x7fab6806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:38.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.669+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab780899b0 con 0x7fab74004d00 2023-10-08T05:57:38.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.669+0000 7fab7ffff640 1 -- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 msgr2=0x7fab6806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2068783299 2023-10-08T05:57:38.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.669+0000 7fab7ffff640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 0x7fab6806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T05:57:38.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.673+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab780547b0 con 0x7fab74004d00 2023-10-08T05:57:38.683 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:38.682+0000 7fbf11a1b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T05:57:38.690 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:38.689+0000 7f9f76656040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:57:38.747 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:38.746+0000 7fbf11a1b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T05:57:38.798 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:38.797+0000 7fbf11a1b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T05:57:38.798 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:38.797+0000 7f9f76656040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:57:38.860 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:38.858+0000 7f9f76656040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:57:38.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.870+0000 7fab7ffff640 1 -- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 msgr2=0x7fab6806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2068783299 2023-10-08T05:57:38.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.870+0000 7fab7ffff640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 0x7fab6806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T05:57:38.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:38.879+0000 7fab8e0c2640 1 -- 172.21.15.70:0/3595058431 --> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fab88070350 con 0x7fab6806afe0 2023-10-08T05:57:38.908 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:38.907+0000 7f9f76656040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:57:38.912 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:38.911+0000 7fbf11a1b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T05:57:38.964 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:38.963+0000 7f9f76656040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:57:38.977 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:38.976+0000 7fbf11a1b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T05:57:39.014 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:39.012+0000 7f9f76656040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:57:39.026 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:39.025+0000 7fbf11a1b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T05:57:39.084 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:39.082+0000 7fbf11a1b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T05:57:39.133 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:39.132+0000 7fbf11a1b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T05:57:39.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:39.271+0000 7fab7ffff640 1 -- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 msgr2=0x7fab6806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2068783299 2023-10-08T05:57:39.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:39.271+0000 7fab7ffff640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 0x7fab6806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T05:57:39.390 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:39.389+0000 7f9f76656040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:57:39.513 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:39.512+0000 7fbf11a1b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T05:57:39.545 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:39.544+0000 7f9f76656040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:57:39.652 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:39.651+0000 7f9f76656040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:57:39.672 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:39.670+0000 7fbf11a1b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T05:57:39.701 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:39.700+0000 7f9f76656040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:57:39.780 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:39.778+0000 7fbf11a1b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T05:57:39.807 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:39.806+0000 7f9f76656040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:57:39.830 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:39.829+0000 7fbf11a1b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T05:57:39.912 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:39.911+0000 7f9f76656040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:57:39.937 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:39.936+0000 7fbf11a1b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:57:40.043 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:40.042+0000 7fbf11a1b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T05:57:40.065 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:40.064+0000 7f9f76656040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:57:40.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:40.072+0000 7fab7ffff640 1 -- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 msgr2=0x7fab6806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2068783299 2023-10-08T05:57:40.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:40.072+0000 7fab7ffff640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 0x7fab6806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T05:57:40.113 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:40.112+0000 7f9f76656040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:57:40.200 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:40.198+0000 7fbf11a1b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T05:57:40.230 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:40.228+0000 7f9f76656040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:57:40.249 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:40.248+0000 7fbf11a1b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T05:57:40.368 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:40.367+0000 7fbf11a1b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T05:57:41.050 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.050+0000 7f9f76656040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:57:41.133 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:57:41.141 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:57:41.197 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:41.195+0000 7fbf11a1b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T05:57:41.348 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.348+0000 7f9f76656040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:57:41.403 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.403+0000 7f9f76656040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:57:41.495 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:41.494+0000 7fbf11a1b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T05:57:41.515 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.515+0000 7f9f76656040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:57:41.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:41.535+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 79) v1 ==== 88957+0+0 (secure 0 0 0) 0x7fab78054b80 con 0x7fab74004d00 2023-10-08T05:57:41.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:41.535+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 msgr2=0x7fab6806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T05:57:41.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:41.535+0000 7fab84ff9640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/2068783299,v1:172.21.15.181:6833/2068783299] conn(0x7fab6806afe0 0x7fab6806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:41.562 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:41.561+0000 7fbf11a1b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T05:57:41.689 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T05:57:41.687+0000 7fbf11a1b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T05:57:41.696 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.696+0000 7f9ef6229640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:57:41.696 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.696+0000 7f9ef6229640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:57:41.696 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.696+0000 7f9ef6229640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:57:41.696 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.696+0000 7f9ef6229640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:57:41.696 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.696+0000 7f9ef6229640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:57:41.697 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.697+0000 7f9ef31e3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:57:41.697 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.697+0000 7f9ef31e3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:57:41.697 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.697+0000 7f9ef31e3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:57:41.697 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.697+0000 7f9ef31e3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:57:41.697 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:57:41.697+0000 7f9ef31e3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:57:42.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.604+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 80) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fab78054b80 con 0x7fab74004d00 2023-10-08T05:57:42.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.605+0000 7fab84ff9640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fab68073600 0x7fab680759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:42.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.605+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 --> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fab68008150 con 0x7fab68073600 2023-10-08T05:57:42.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.619+0000 7fab7ffff640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fab68073600 0x7fab680759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:42.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.629+0000 7fab7ffff640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fab68073600 0x7fab680759f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab880c4c10 tx=0x7fab70007d30 comp rx=0 tx=0).ready entity=mgr.6467 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:42.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.785+0000 7fab84ff9640 1 -- 172.21.15.70:0/3595058431 <== mgr.6467 v2:172.21.15.181:6832/236882560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fab70005050 con 0x7fab68073600 2023-10-08T05:57:42.787 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1 2023-10-08T05:57:42.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.789+0000 7fab8e0c2640 1 -- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fab68073600 msgr2=0x7fab680759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:42.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.789+0000 7fab8e0c2640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fab68073600 0x7fab680759f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab880c4c10 tx=0x7fab70007d30 comp rx=0 tx=0).stop 2023-10-08T05:57:42.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.789+0000 7fab8e0c2640 1 -- 172.21.15.70:0/3595058431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 msgr2=0x7fab8815c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:42.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.789+0000 7fab8e0c2640 1 --2- 172.21.15.70:0/3595058431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab74004d00 0x7fab8815c8a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fab7800e790 tx=0x7fab7800ec50 comp rx=0 tx=0).stop 2023-10-08T05:57:42.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.789+0000 7fab87fff640 1 -- 172.21.15.70:0/3595058431 reap_dead start 2023-10-08T05:57:42.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.789+0000 7fab8e0c2640 1 -- 172.21.15.70:0/3595058431 shutdown_connections 2023-10-08T05:57:42.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.789+0000 7fab8e0c2640 1 -- 172.21.15.70:0/3595058431 >> 172.21.15.70:0/3595058431 conn(0x7fab8806a5d0 msgr2=0x7fab880cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:42.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.789+0000 7fab8e0c2640 1 -- 172.21.15.70:0/3595058431 shutdown_connections 2023-10-08T05:57:42.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:42.789+0000 7fab8e0c2640 1 -- 172.21.15.70:0/3595058431 wait complete. 2023-10-08T05:57:42.810 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-10-08T05:57:42.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add /snap_test_dir1 55n 2023-10-08T05:57:43.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.026+0000 7f9599d55640 1 Processor -- start 2023-10-08T05:57:43.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.027+0000 7f9599d55640 1 -- start start 2023-10-08T05:57:43.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f9599d55640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9594161690 0x7f959406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f9599d55640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95940cb150 0x7f95940cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f9599d55640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959406cc50 con 0x7f95940702b0 2023-10-08T05:57:43.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f9599d55640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95940cbb10 con 0x7f9594161690 2023-10-08T05:57:43.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f9599d55640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959406cf30 con 0x7f95940cb150 2023-10-08T05:57:43.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f9593fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f95940702b0 0x7f9594070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38522/0 (socket says 172.21.15.70:38522) 2023-10-08T05:57:43.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f9593fff640 1 -- 172.21.15.70:0/712646438 learned_addr learned my addr 172.21.15.70:0/712646438 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:43.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f95937fe640 1 --2- 172.21.15.70:0/712646438 >> v2:172.21.15.181:3300/0 conn(0x7f9594161690 0x7f959406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f958bfff640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95940cb150 0x7f95940cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817672356 0 0) 0x7f959406cc50 con 0x7f95940702b0 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.029+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9580003520 con 0x7f95940702b0 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 64594034 0 0) 0x7f9580003520 con 0x7f95940702b0 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95940cb150 msgr2=0x7f95940cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95940cb150 0x7f95940cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 >> v2:172.21.15.181:3300/0 conn(0x7f9594161690 msgr2=0x7f959406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 --2- 172.21.15.70:0/712646438 >> v2:172.21.15.181:3300/0 conn(0x7f9594161690 0x7f959406dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959406d210 con 0x7f95940702b0 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f958c0019f0 con 0x7f95940702b0 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 >> v1:172.21.15.70:6789/0 conn(0x7f95940702b0 legacy=0x7f9594070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9580003970 0x7f9580003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95800047c0 0x7f9580004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95940cb150 0x7f9580005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940cbb10 con 0x7f95800047c0 2023-10-08T05:57:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959406cf30 con 0x7f9580003970 2023-10-08T05:57:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958000b980 con 0x7f95940cb150 2023-10-08T05:57:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f958bfff640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95800047c0 0x7f9580004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.030+0000 7f9593fff640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95940cb150 0x7f9580005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.031+0000 7f95937fe640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9580003970 0x7f9580003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.031+0000 7f958bfff640 1 -- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95940cb150 msgr2=0x7f9580005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:43.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.031+0000 7f958bfff640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95940cb150 0x7f9580005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.031+0000 7f958bfff640 1 -- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9580003970 msgr2=0x7f9580003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:43.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.031+0000 7f958bfff640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9580003970 0x7f9580003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.031+0000 7f958bfff640 1 -- 172.21.15.70:0/712646438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9580003520 con 0x7f95800047c0 2023-10-08T05:57:43.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.033+0000 7f958bfff640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95800047c0 0x7f9580004c40 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f957c009ac0 tx=0x7f95800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f19c38c6c95e1f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:43.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.034+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c02c6b0 con 0x7f95800047c0 2023-10-08T05:57:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.034+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957c02c810 con 0x7f95800047c0 2023-10-08T05:57:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.034+0000 7f9592ffd640 1 -- 172.21.15.70:0/712646438 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c02cba0 con 0x7f95800047c0 2023-10-08T05:57:43.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.035+0000 7f9599d55640 1 -- 172.21.15.70:0/712646438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95800047c0 msgr2=0x7f9580004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.035+0000 7f9599d55640 1 --2- 172.21.15.70:0/712646438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95800047c0 0x7f9580004c40 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f957c009ac0 tx=0x7f95800057f0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.035+0000 7f9593fff640 1 -- 172.21.15.70:0/712646438 reap_dead start 2023-10-08T05:57:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.035+0000 7f9599d55640 1 -- 172.21.15.70:0/712646438 shutdown_connections 2023-10-08T05:57:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.035+0000 7f9599d55640 1 -- 172.21.15.70:0/712646438 >> 172.21.15.70:0/712646438 conn(0x7f959406a5d0 msgr2=0x7f95940b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.035+0000 7f9599d55640 1 -- 172.21.15.70:0/712646438 shutdown_connections 2023-10-08T05:57:43.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.035+0000 7f9599d55640 1 -- 172.21.15.70:0/712646438 wait complete. 2023-10-08T05:57:43.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.037+0000 7f9599d55640 1 Processor -- start 2023-10-08T05:57:43.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.037+0000 7f9599d55640 1 -- start start 2023-10-08T05:57:43.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.038+0000 7f9599d55640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95800047c0 0x7f9594156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.038+0000 7f9599d55640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594158010 0x7f9594158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.038+0000 7f9599d55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9594163c90 0x7f9594164110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.038+0000 7f9599d55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941572c0 con 0x7f9594163c90 2023-10-08T05:57:43.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.038+0000 7f9599d55640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9594158a90 con 0x7f9594158010 2023-10-08T05:57:43.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.038+0000 7f9599d55640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9594158d70 con 0x7f95800047c0 2023-10-08T05:57:43.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.038+0000 7f9593fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9594163c90 0x7f9594164110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.038+0000 7f9593fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9594163c90 0x7f9594164110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34794/0 (socket says 172.21.15.70:34794) 2023-10-08T05:57:43.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.038+0000 7f9593fff640 1 -- 172.21.15.70:0/1900355798 learned_addr learned my addr 172.21.15.70:0/1900355798 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:43.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9593fff640 1 -- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95800047c0 msgr2=0x7f9594156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9593fff640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95800047c0 0x7f9594156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9593fff640 1 -- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594158010 msgr2=0x7f9594158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f958bfff640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594158010 0x7f9594158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9593fff640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594158010 0x7f9594158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9593fff640 1 -- 172.21.15.70:0/1900355798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9594159050 con 0x7f9594163c90 2023-10-08T05:57:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9593fff640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9594163c90 0x7f9594164110 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f959406e5a0 tx=0x7f958c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9590ff9640 1 -- 172.21.15.70:0/1900355798 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c00ab60 con 0x7f9594163c90 2023-10-08T05:57:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9590ff9640 1 -- 172.21.15.70:0/1900355798 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c00ad00 con 0x7f9594163c90 2023-10-08T05:57:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9599d55640 1 -- 172.21.15.70:0/1900355798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9594159330 con 0x7f9594163c90 2023-10-08T05:57:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9590ff9640 1 -- 172.21.15.70:0/1900355798 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c013630 con 0x7f9594163c90 2023-10-08T05:57:43.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.039+0000 7f9599d55640 1 -- 172.21.15.70:0/1900355798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959415a490 con 0x7f9594163c90 2023-10-08T05:57:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.040+0000 7f9599d55640 1 -- 172.21.15.70:0/1900355798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959406c7f0 con 0x7f9594163c90 2023-10-08T05:57:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.041+0000 7f9590ff9640 1 -- 172.21.15.70:0/1900355798 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 80) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f958c013790 con 0x7f9594163c90 2023-10-08T05:57:43.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.042+0000 7f9590ff9640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f957406ad10 0x7f957406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.042+0000 7f9590ff9640 1 -- 172.21.15.70:0/1900355798 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f958c089cd0 con 0x7f9594163c90 2023-10-08T05:57:43.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.043+0000 7f95937fe640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f957406ad10 0x7f957406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.044+0000 7f95937fe640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f957406ad10 0x7f957406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9584006b10 tx=0x7f9584006aa0 comp rx=0 tx=0).ready entity=mgr.6467 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:43.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.048+0000 7f9590ff9640 1 -- 172.21.15.70:0/1900355798 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f958c054d10 con 0x7f9594163c90 2023-10-08T05:57:43.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.259+0000 7f9599d55640 1 -- 172.21.15.70:0/1900355798 --> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f95940b1990 con 0x7f957406ad10 2023-10-08T05:57:43.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.402+0000 7f9590ff9640 1 -- 172.21.15.70:0/1900355798 <== mgr.6467 v2:172.21.15.181:6832/236882560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f95840096f0 con 0x7f957406ad10 2023-10-08T05:57:43.403 INFO:teuthology.orchestra.run.smithi070.stdout:Retention added to path /snap_test_dir1 2023-10-08T05:57:43.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 -- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f957406ad10 msgr2=0x7f957406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f957406ad10 0x7f957406d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9584006b10 tx=0x7f9584006aa0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 -- 172.21.15.70:0/1900355798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9594163c90 msgr2=0x7f9594164110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9594163c90 0x7f9594164110 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f959406e5a0 tx=0x7f958c007b00 comp rx=0 tx=0).stop 2023-10-08T05:57:43.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 -- 172.21.15.70:0/1900355798 shutdown_connections 2023-10-08T05:57:43.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f957406ad10 0x7f957406d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9594163c90 0x7f9594164110 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9594158010 0x7f9594158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 --2- 172.21.15.70:0/1900355798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95800047c0 0x7f9594156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 -- 172.21.15.70:0/1900355798 >> 172.21.15.70:0/1900355798 conn(0x7f959406a5d0 msgr2=0x7f95940b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:43.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 -- 172.21.15.70:0/1900355798 shutdown_connections 2023-10-08T05:57:43.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.405+0000 7f9599d55640 1 -- 172.21.15.70:0/1900355798 wait complete. 2023-10-08T05:57:43.426 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-10-08T05:57:43.427 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [] 2023-10-08T05:57:43.427 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json /snap_test_dir1 2023-10-08T05:57:43.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.659+0000 7f73884bb640 1 Processor -- start 2023-10-08T05:57:43.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.659+0000 7f73884bb640 1 -- start start 2023-10-08T05:57:43.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.659+0000 7f73884bb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73800afbe0 0x7f73800b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.659+0000 7f73884bb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f73800cb130 0x7f73800cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.659+0000 7f73884bb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73800cbad0 con 0x7f738006dfc0 2023-10-08T05:57:43.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.659+0000 7f73884bb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73800b2620 con 0x7f73800afbe0 2023-10-08T05:57:43.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.659+0000 7f73884bb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73800b2060 con 0x7f73800cb130 2023-10-08T05:57:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.660+0000 7f7386257640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73800afbe0 0x7f73800b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.660+0000 7f7385a56640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f738006dfc0 0x7f7380070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49054/0 (socket says 172.21.15.70:49054) 2023-10-08T05:57:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.660+0000 7f7385a56640 1 -- 172.21.15.70:0/4008065402 learned_addr learned my addr 172.21.15.70:0/4008065402 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T05:57:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.660+0000 7f7386a58640 1 --2- 172.21.15.70:0/4008065402 >> v2:172.21.15.181:3300/0 conn(0x7f73800cb130 0x7f73800cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.660+0000 7f7386257640 1 -- 172.21.15.70:0/4008065402 >> v2:172.21.15.181:3300/0 conn(0x7f73800cb130 msgr2=0x7f73800cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T05:57:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.660+0000 7f7386257640 1 --2- 172.21.15.70:0/4008065402 >> v2:172.21.15.181:3300/0 conn(0x7f73800cb130 0x7f73800cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.660+0000 7f7386257640 1 -- 172.21.15.70:0/4008065402 >> v1:172.21.15.70:6789/0 conn(0x7f738006dfc0 legacy=0x7f7380070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.660+0000 7f7386257640 1 -- 172.21.15.70:0/4008065402 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73800b2340 con 0x7f73800afbe0 2023-10-08T05:57:43.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.660+0000 7f7385255640 1 -- 172.21.15.70:0/4008065402 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219187227 0 0) 0x7f73800cbad0 con 0x7f738006dfc0 2023-10-08T05:57:43.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.662+0000 7f7386257640 1 --2- 172.21.15.70:0/4008065402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73800afbe0 0x7f73800b15b0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f73800c8d10 tx=0x7f737002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6541d951fa73d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:43.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.662+0000 7f7385255640 1 -- 172.21.15.70:0/4008065402 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737002cc80 con 0x7f73800afbe0 2023-10-08T05:57:43.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.662+0000 7f7385255640 1 -- 172.21.15.70:0/4008065402 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737002cde0 con 0x7f73800afbe0 2023-10-08T05:57:43.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.662+0000 7f7385255640 1 -- 172.21.15.70:0/4008065402 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370034660 con 0x7f73800afbe0 2023-10-08T05:57:43.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.662+0000 7f73884bb640 1 -- 172.21.15.70:0/4008065402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73800afbe0 msgr2=0x7f73800b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.662+0000 7f73884bb640 1 --2- 172.21.15.70:0/4008065402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73800afbe0 0x7f73800b15b0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f73800c8d10 tx=0x7f737002bed0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.663+0000 7f73884bb640 1 -- 172.21.15.70:0/4008065402 shutdown_connections 2023-10-08T05:57:43.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.663+0000 7f73884bb640 1 --2- 172.21.15.70:0/4008065402 >> v2:172.21.15.181:3300/0 conn(0x7f73800cb130 0x7f73800cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.663+0000 7f73884bb640 1 --2- 172.21.15.70:0/4008065402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73800afbe0 0x7f73800b15b0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.663+0000 7f73884bb640 1 -- 172.21.15.70:0/4008065402 >> 172.21.15.70:0/4008065402 conn(0x7f738006a5d0 msgr2=0x7f73800b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:43.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.663+0000 7f73884bb640 1 -- 172.21.15.70:0/4008065402 shutdown_connections 2023-10-08T05:57:43.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.663+0000 7f73884bb640 1 -- 172.21.15.70:0/4008065402 wait complete. 2023-10-08T05:57:43.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.665+0000 7f73884bb640 1 Processor -- start 2023-10-08T05:57:43.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.665+0000 7f73884bb640 1 -- start start 2023-10-08T05:57:43.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f73884bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738006dfc0 0x7f7380155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f73884bb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73800afbe0 0x7f73800c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f73884bb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73800cb130 0x7f73800c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f73884bb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73800c2b50 con 0x7f738006dfc0 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f73884bb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73800c2e30 con 0x7f73800cb130 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f73884bb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73800c3110 con 0x7f73800afbe0 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f7386257640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738006dfc0 0x7f7380155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f7386257640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738006dfc0 0x7f7380155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36754/0 (socket says 172.21.15.70:36754) 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f7386257640 1 -- 172.21.15.70:0/2459549170 learned_addr learned my addr 172.21.15.70:0/2459549170 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f7386a58640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73800cb130 0x7f73800c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.666+0000 7f7385a56640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73800afbe0 0x7f73800c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f7386257640 1 -- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73800afbe0 msgr2=0x7f73800c2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f7386257640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73800afbe0 0x7f73800c2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f7386257640 1 -- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73800cb130 msgr2=0x7f73800c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f7386257640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73800cb130 0x7f73800c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f7386257640 1 -- 172.21.15.70:0/2459549170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7370009a70 con 0x7f738006dfc0 2023-10-08T05:57:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f7386a58640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73800cb130 0x7f73800c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T05:57:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f7386257640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738006dfc0 0x7f7380155c30 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f7370009e80 tx=0x7f7370004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f737effd640 1 -- 172.21.15.70:0/2459549170 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370003b90 con 0x7f738006dfc0 2023-10-08T05:57:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f737effd640 1 -- 172.21.15.70:0/2459549170 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7370003cf0 con 0x7f738006dfc0 2023-10-08T05:57:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f737effd640 1 -- 172.21.15.70:0/2459549170 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370004220 con 0x7f738006dfc0 2023-10-08T05:57:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f73884bb640 1 -- 172.21.15.70:0/2459549170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73800c33f0 con 0x7f738006dfc0 2023-10-08T05:57:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.667+0000 7f73884bb640 1 -- 172.21.15.70:0/2459549170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7380156170 con 0x7f738006dfc0 2023-10-08T05:57:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.669+0000 7f73884bb640 1 -- 172.21.15.70:0/2459549170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73800c3910 con 0x7f738006dfc0 2023-10-08T05:57:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.669+0000 7f737effd640 1 -- 172.21.15.70:0/2459549170 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 81) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f737003b050 con 0x7f738006dfc0 2023-10-08T05:57:43.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.671+0000 7f737effd640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f736c06afe0 0x7f736c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T05:57:43.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.671+0000 7f737effd640 1 -- 172.21.15.70:0/2459549170 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6445+0+0 (secure 0 0 0) 0x7f73700ac0b0 con 0x7f738006dfc0 2023-10-08T05:57:43.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.672+0000 7f7385a56640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f736c06afe0 0x7f736c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T05:57:43.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.672+0000 7f7385a56640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f736c06afe0 0x7f736c06d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f73800ce2a0 tx=0x7f7374007b50 comp rx=0 tx=0).ready entity=mgr.6467 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T05:57:43.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.676+0000 7f737effd640 1 -- 172.21.15.70:0/2459549170 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73800c3910 con 0x7f738006dfc0 2023-10-08T05:57:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.887+0000 7f73884bb640 1 -- 172.21.15.70:0/2459549170 --> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7380070630 con 0x7f736c06afe0 2023-10-08T05:57:43.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.903+0000 7f737effd640 1 -- 172.21.15.70:0/2459549170 <== mgr.6467 v2:172.21.15.181:6832/236882560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f73740076a0 con 0x7f736c06afe0 2023-10-08T05:57:43.904 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T05:57:43.904 INFO:teuthology.orchestra.run.smithi070.stdout:{"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2023-10-08T05:57:43.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 -- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f736c06afe0 msgr2=0x7f736c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f736c06afe0 0x7f736c06d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f73800ce2a0 tx=0x7f7374007b50 comp rx=0 tx=0).stop 2023-10-08T05:57:43.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 -- 172.21.15.70:0/2459549170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738006dfc0 msgr2=0x7f7380155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T05:57:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738006dfc0 0x7f7380155c30 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f7370009e80 tx=0x7f7370004a40 comp rx=0 tx=0).stop 2023-10-08T05:57:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 -- 172.21.15.70:0/2459549170 shutdown_connections 2023-10-08T05:57:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f736c06afe0 0x7f736c06d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f73800cb130 0x7f73800c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73800afbe0 0x7f73800c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 --2- 172.21.15.70:0/2459549170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f738006dfc0 0x7f7380155c30 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T05:57:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 -- 172.21.15.70:0/2459549170 >> 172.21.15.70:0/2459549170 conn(0x7f738006a5d0 msgr2=0x7f73800b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T05:57:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 -- 172.21.15.70:0/2459549170 shutdown_connections 2023-10-08T05:57:43.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T05:57:43.906+0000 7f73884bb640 1 -- 172.21.15.70:0/2459549170 wait complete. 2023-10-08T05:57:43.924 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1m"], "retention": [{"n": 55}]} 2023-10-08T05:57:43.925 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1m'], 'retention': [{'n': 55}]} 2023-10-08T05:58:00.425 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:58:00.424+0000 7f9ee51c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:58:00.425 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:58:00.424+0000 7f9ee51c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:58:00.426 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:58:00.424+0000 7f9ee51c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:58:00.426 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:58:00.424+0000 7f9ee51c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:58:00.426 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T05:58:00.424+0000 7f9ee51c7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T05:58:11.281 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:58:11.290 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:58:41.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:58:41.448 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:59:11.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:59:11.592 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:59:41.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T05:59:41.748 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:00:11.893 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:00:11.904 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:00:42.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:00:42.068 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:01:12.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:01:12.214 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:01:42.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:01:42.360 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:02:12.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:02:12.543 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:02:42.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:02:42.686 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:03:12.826 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:03:12.836 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:03:42.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:03:42.982 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:04:13.136 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:04:13.146 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:04:43.274 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:04:43.284 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:05:13.418 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:05:13.428 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:05:43.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:05:43.572 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:06:13.724 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:06:13.733 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:06:43.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:06:43.907 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:07:14.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:07:14.057 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:07:44.193 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:07:44.235 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:08:14.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:08:14.387 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:08:44.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:08:44.545 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:09:14.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:09:14.695 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:09:44.838 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:09:44.848 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:10:15.009 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:10:15.018 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:10:45.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:10:45.173 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:11:15.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:11:15.348 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:11:45.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:11:45.486 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:12:15.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:12:15.644 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:12:45.785 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:12:45.795 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:13:15.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:13:15.946 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:13:46.077 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:13:46.087 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:14:16.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:14:16.226 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:14:46.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:14:46.382 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:15:16.560 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:15:16.576 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:15:46.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:15:46.739 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:16:16.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:16:16.890 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:16:47.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:16:47.030 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:17:17.164 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:17:17.173 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:17:47.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:17:47.316 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:18:17.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:18:17.493 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:18:47.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:18:47.659 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:19:17.785 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:19:17.797 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:19:47.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:19:47.949 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:20:18.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:20:18.088 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:20:48.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:20:48.273 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:21:18.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:21:18.467 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:21:48.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:21:48.646 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:22:18.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:22:18.814 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:22:48.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:22:48.957 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:23:19.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:23:19.105 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:23:49.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:23:49.307 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:24:19.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:24:19.502 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:24:49.688 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:24:49.698 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:25:19.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:25:19.918 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:25:50.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:25:50.111 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:26:20.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:26:20.265 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:26:50.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:26:50.660 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:27:20.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:27:20.814 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:27:50.941 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:27:50.950 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:28:21.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:28:21.153 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:28:51.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:28:51.331 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:29:21.575 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:29:21.585 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:29:51.726 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:29:51.736 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:30:21.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:30:21.907 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:30:52.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:30:52.085 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:31:22.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:31:22.305 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:31:52.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:31:52.490 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:32:22.659 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:32:22.671 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:32:52.802 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:32:52.813 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:33:23.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:33:23.033 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:33:53.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:33:53.252 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:34:23.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:34:23.422 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:34:53.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:34:53.601 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:35:23.743 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:35:23.753 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:35:53.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:35:53.939 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:36:24.074 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:36:24.084 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:36:54.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:36:54.287 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:37:24.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:37:24.443 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:37:54.659 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:37:54.669 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:38:24.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:38:24.855 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:38:55.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:38:55.025 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:39:25.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:39:25.219 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:39:55.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:39:55.381 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:40:25.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:40:25.584 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:40:55.728 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:40:55.739 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:41:25.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:41:25.965 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:41:56.175 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:41:56.185 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:42:26.387 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:42:26.398 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:42:56.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:42:56.608 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:43:26.819 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:43:26.829 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:43:56.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:43:56.974 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:44:27.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:44:27.161 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:44:57.291 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:44:57.301 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:45:27.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:45:27.508 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:45:57.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:45:57.686 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:46:27.825 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:46:27.835 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:46:57.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:46:57.971 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:47:28.118 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:47:28.131 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:47:58.322 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:47:58.331 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:48:28.550 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:48:28.559 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:48:58.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:48:58.762 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:49:28.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:49:28.958 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:49:59.175 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:49:59.185 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:50:29.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:50:29.355 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:50:59.566 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:50:59.576 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:51:29.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:51:29.787 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:51:59.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:51:59.948 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:52:30.133 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:52:30.143 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:53:00.328 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:53:00.338 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:53:15.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:15.926 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T06:53:15.994 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_59_00_UTC 2023-10-08T06:53:15.994 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_00_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_01_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_02_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_03_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_04_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_05_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_06_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_07_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_08_01_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_09_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_10_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_11_00_UTC 2023-10-08T06:53:15.995 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_12_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_13_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_14_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_15_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_16_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_17_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_18_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_19_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_20_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_21_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_22_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_23_00_UTC 2023-10-08T06:53:15.996 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_24_00_UTC 2023-10-08T06:53:15.997 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_25_00_UTC 2023-10-08T06:53:15.997 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_26_00_UTC 2023-10-08T06:53:15.997 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_27_00_UTC 2023-10-08T06:53:15.997 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_28_00_UTC 2023-10-08T06:53:15.997 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_29_00_UTC 2023-10-08T06:53:15.997 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_30_00_UTC 2023-10-08T06:53:15.997 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_31_00_UTC 2023-10-08T06:53:15.997 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_32_00_UTC 2023-10-08T06:53:15.997 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_33_00_UTC 2023-10-08T06:53:15.997 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_34_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_35_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_36_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_37_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_38_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_39_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_40_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_41_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_42_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_43_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_44_00_UTC 2023-10-08T06:53:15.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_45_00_UTC 2023-10-08T06:53:15.999 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_46_00_UTC 2023-10-08T06:53:15.999 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_47_00_UTC 2023-10-08T06:53:15.999 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_48_00_UTC 2023-10-08T06:53:15.999 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_49_00_UTC 2023-10-08T06:53:15.999 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_50_00_UTC 2023-10-08T06:53:15.999 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_51_00_UTC 2023-10-08T06:53:15.999 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_52_00_UTC 2023-10-08T06:53:15.999 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_53_00_UTC 2023-10-08T06:53:16.000 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-05_59_00_UTC', 'scheduled-2023-10-08-06_00_00_UTC', 'scheduled-2023-10-08-06_01_00_UTC', 'scheduled-2023-10-08-06_02_00_UTC', 'scheduled-2023-10-08-06_03_00_UTC', 'scheduled-2023-10-08-06_04_00_UTC', 'scheduled-2023-10-08-06_05_00_UTC', 'scheduled-2023-10-08-06_06_00_UTC', 'scheduled-2023-10-08-06_07_00_UTC', 'scheduled-2023-10-08-06_08_01_UTC', 'scheduled-2023-10-08-06_09_00_UTC', 'scheduled-2023-10-08-06_10_00_UTC', 'scheduled-2023-10-08-06_11_00_UTC', 'scheduled-2023-10-08-06_12_00_UTC', 'scheduled-2023-10-08-06_13_00_UTC', 'scheduled-2023-10-08-06_14_00_UTC', 'scheduled-2023-10-08-06_15_00_UTC', 'scheduled-2023-10-08-06_16_00_UTC', 'scheduled-2023-10-08-06_17_00_UTC', 'scheduled-2023-10-08-06_18_00_UTC', 'scheduled-2023-10-08-06_19_00_UTC', 'scheduled-2023-10-08-06_20_00_UTC', 'scheduled-2023-10-08-06_21_00_UTC', 'scheduled-2023-10-08-06_22_00_UTC', 'scheduled-2023-10-08-06_23_00_UTC', 'scheduled-2023-10-08-06_24_00_UTC', 'scheduled-2023-10-08-06_25_00_UTC', 'scheduled-2023-10-08-06_26_00_UTC', 'scheduled-2023-10-08-06_27_00_UTC', 'scheduled-2023-10-08-06_28_00_UTC', 'scheduled-2023-10-08-06_29_00_UTC', 'scheduled-2023-10-08-06_30_00_UTC', 'scheduled-2023-10-08-06_31_00_UTC', 'scheduled-2023-10-08-06_32_00_UTC', 'scheduled-2023-10-08-06_33_00_UTC', 'scheduled-2023-10-08-06_34_00_UTC', 'scheduled-2023-10-08-06_35_00_UTC', 'scheduled-2023-10-08-06_36_00_UTC', 'scheduled-2023-10-08-06_37_00_UTC', 'scheduled-2023-10-08-06_38_00_UTC', 'scheduled-2023-10-08-06_39_00_UTC', 'scheduled-2023-10-08-06_40_00_UTC', 'scheduled-2023-10-08-06_41_00_UTC', 'scheduled-2023-10-08-06_42_00_UTC', 'scheduled-2023-10-08-06_43_00_UTC', 'scheduled-2023-10-08-06_44_00_UTC', 'scheduled-2023-10-08-06_45_00_UTC', 'scheduled-2023-10-08-06_46_00_UTC', 'scheduled-2023-10-08-06_47_00_UTC', 'scheduled-2023-10-08-06_48_00_UTC', 'scheduled-2023-10-08-06_49_00_UTC', 'scheduled-2023-10-08-06_50_00_UTC', 'scheduled-2023-10-08-06_51_00_UTC', 'scheduled-2023-10-08-06_52_00_UTC', 'scheduled-2023-10-08-06_53_00_UTC'] 2023-10-08T06:53:16.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1 2023-10-08T06:53:16.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.219+0000 7fbc7a13c640 1 Processor -- start 2023-10-08T06:53:16.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.219+0000 7fbc7a13c640 1 -- start start 2023-10-08T06:53:16.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7a13c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbc7415a820 0x7fbc740aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7a13c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc7415ac00 0x7fbc740b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7a13c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc740cab60 con 0x7fbc740ca680 2023-10-08T06:53:16.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7a13c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc740cb960 con 0x7fbc7415ac00 2023-10-08T06:53:16.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7a13c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc740cbdf0 con 0x7fbc7415a820 2023-10-08T06:53:16.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7993b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc7415ac00 0x7fbc740b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:16.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7993b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc7415ac00 0x7fbc740b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46642/0 (socket says 172.21.15.70:46642) 2023-10-08T06:53:16.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7993b640 1 -- 172.21.15.70:0/1400431374 learned_addr learned my addr 172.21.15.70:0/1400431374 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:16.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7993b640 1 -- 172.21.15.70:0/1400431374 >> v2:172.21.15.181:3300/0 conn(0x7fbc7415a820 msgr2=0x7fbc740aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:16.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7993b640 1 --2- 172.21.15.70:0/1400431374 >> v2:172.21.15.181:3300/0 conn(0x7fbc7415a820 0x7fbc740aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7993b640 1 -- 172.21.15.70:0/1400431374 >> v1:172.21.15.181:6789/0 conn(0x7fbc740ca680 legacy=0x7fbc740c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:16.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.221+0000 7fbc7993b640 1 -- 172.21.15.70:0/1400431374 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc740cb030 con 0x7fbc7415ac00 2023-10-08T06:53:16.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.223+0000 7fbc7993b640 1 --2- 172.21.15.70:0/1400431374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc7415ac00 0x7fbc740b04b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fbc7406da70 tx=0x7fbc6c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad989cff4689980 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:16.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.223+0000 7fbc6bfff640 1 -- 172.21.15.70:0/1400431374 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c02c9f0 con 0x7fbc7415ac00 2023-10-08T06:53:16.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.223+0000 7fbc6bfff640 1 -- 172.21.15.70:0/1400431374 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6c02cb50 con 0x7fbc7415ac00 2023-10-08T06:53:16.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.223+0000 7fbc6bfff640 1 -- 172.21.15.70:0/1400431374 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0343f0 con 0x7fbc7415ac00 2023-10-08T06:53:16.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.223+0000 7fbc7a13c640 1 -- 172.21.15.70:0/1400431374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc7415ac00 msgr2=0x7fbc740b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:16.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.223+0000 7fbc7a13c640 1 --2- 172.21.15.70:0/1400431374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc7415ac00 0x7fbc740b04b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fbc7406da70 tx=0x7fbc6c02be50 comp rx=0 tx=0).stop 2023-10-08T06:53:16.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.223+0000 7fbc7a13c640 1 -- 172.21.15.70:0/1400431374 shutdown_connections 2023-10-08T06:53:16.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.223+0000 7fbc7a13c640 1 --2- 172.21.15.70:0/1400431374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc7415ac00 0x7fbc740b04b0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.223+0000 7fbc7a13c640 1 --2- 172.21.15.70:0/1400431374 >> v2:172.21.15.181:3300/0 conn(0x7fbc7415a820 0x7fbc740aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.223+0000 7fbc7a13c640 1 -- 172.21.15.70:0/1400431374 >> 172.21.15.70:0/1400431374 conn(0x7fbc7406c050 msgr2=0x7fbc7406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:16.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.224+0000 7fbc7a13c640 1 -- 172.21.15.70:0/1400431374 shutdown_connections 2023-10-08T06:53:16.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.224+0000 7fbc7a13c640 1 -- 172.21.15.70:0/1400431374 wait complete. 2023-10-08T06:53:16.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.225+0000 7fbc7a13c640 1 Processor -- start 2023-10-08T06:53:16.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.226+0000 7fbc7a13c640 1 -- start start 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.226+0000 7fbc7a13c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc740ca680 0x7fbc740c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.226+0000 7fbc7a13c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7415a820 0x7fbc741f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7a13c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc7415ac00 0x7fbc74150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7a13c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc740c61f0 con 0x7fbc740ca680 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7a13c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc740c64d0 con 0x7fbc7415a820 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7a13c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc740c67b0 con 0x7fbc7415ac00 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7913a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc740ca680 0x7fbc740c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7913a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc740ca680 0x7fbc740c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46654/0 (socket says 172.21.15.70:46654) 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7913a640 1 -- 172.21.15.70:0/636701230 learned_addr learned my addr 172.21.15.70:0/636701230 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7913a640 1 -- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc7415ac00 msgr2=0x7fbc74150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7913a640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc7415ac00 0x7fbc74150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7913a640 1 -- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7415a820 msgr2=0x7fbc741f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:16.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7913a640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7415a820 0x7fbc741f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7913a640 1 -- 172.21.15.70:0/636701230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c009b30 con 0x7fbc740ca680 2023-10-08T06:53:16.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.227+0000 7fbc7913a640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc740ca680 0x7fbc740c7290 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000b4a0 tx=0x7fbc7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:16.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.228+0000 7fbc69ffb640 1 -- 172.21.15.70:0/636701230 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70010610 con 0x7fbc740ca680 2023-10-08T06:53:16.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.228+0000 7fbc69ffb640 1 -- 172.21.15.70:0/636701230 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc70010770 con 0x7fbc740ca680 2023-10-08T06:53:16.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.228+0000 7fbc7a13c640 1 -- 172.21.15.70:0/636701230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc740c6a90 con 0x7fbc740ca680 2023-10-08T06:53:16.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.228+0000 7fbc69ffb640 1 -- 172.21.15.70:0/636701230 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70010b20 con 0x7fbc740ca680 2023-10-08T06:53:16.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.228+0000 7fbc7a13c640 1 -- 172.21.15.70:0/636701230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc740c77d0 con 0x7fbc740ca680 2023-10-08T06:53:16.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.229+0000 7fbc7a13c640 1 -- 172.21.15.70:0/636701230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc740c6fb0 con 0x7fbc740ca680 2023-10-08T06:53:16.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.230+0000 7fbc69ffb640 1 -- 172.21.15.70:0/636701230 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 84) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fbc70010c80 con 0x7fbc740ca680 2023-10-08T06:53:16.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.231+0000 7fbc69ffb640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fbc5806afe0 0x7fbc5806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.232+0000 7fbc69ffb640 1 -- 172.21.15.70:0/636701230 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fbc70089750 con 0x7fbc740ca680 2023-10-08T06:53:16.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.232+0000 7fbc78939640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fbc5806afe0 0x7fbc5806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:16.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.233+0000 7fbc78939640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fbc5806afe0 0x7fbc5806d4a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc740b0380 tx=0x7fbc64007990 comp rx=0 tx=0).ready entity=mgr.6467 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:16.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.236+0000 7fbc69ffb640 1 -- 172.21.15.70:0/636701230 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc740c6fb0 con 0x7fbc740ca680 2023-10-08T06:53:16.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.446+0000 7fbc7a13c640 1 -- 172.21.15.70:0/636701230 --> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc740cb5f0 con 0x7fbc5806afe0 2023-10-08T06:53:16.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.465+0000 7fbc69ffb640 1 -- 172.21.15.70:0/636701230 <== mgr.6467 v2:172.21.15.181:6832/236882560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fbc640011f0 con 0x7fbc5806afe0 2023-10-08T06:53:16.466 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:16.466 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:57:42", "first": "2023-10-08T05:58:00", "last": "2023-10-08T06:53:00", "last_pruned": "2023-10-08T06:53:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-08T06:53:16.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 -- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fbc5806afe0 msgr2=0x7fbc5806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fbc5806afe0 0x7fbc5806d4a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbc740b0380 tx=0x7fbc64007990 comp rx=0 tx=0).stop 2023-10-08T06:53:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 -- 172.21.15.70:0/636701230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc740ca680 msgr2=0x7fbc740c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc740ca680 0x7fbc740c7290 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000b4a0 tx=0x7fbc7000b960 comp rx=0 tx=0).stop 2023-10-08T06:53:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 -- 172.21.15.70:0/636701230 shutdown_connections 2023-10-08T06:53:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fbc5806afe0 0x7fbc5806d4a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc7415ac00 0x7fbc74150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc7415a820 0x7fbc741f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 --2- 172.21.15.70:0/636701230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc740ca680 0x7fbc740c7290 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 -- 172.21.15.70:0/636701230 >> 172.21.15.70:0/636701230 conn(0x7fbc7406c050 msgr2=0x7fbc7415be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:16.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 -- 172.21.15.70:0/636701230 shutdown_connections 2023-10-08T06:53:16.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.468+0000 7fbc7a13c640 1 -- 172.21.15.70:0/636701230 wait complete. 2023-10-08T06:53:16.488 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1m", "retention": {"n": 55}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T05:57:42", "first": "2023-10-08T05:58:00", "last": "2023-10-08T06:53:00", "last_pruned": "2023-10-08T06:53:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-08T06:53:16.488 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1m', 'retention': {'n': 55}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T05:57:42', 'first': '2023-10-08T05:58:00', 'last': '2023-10-08T06:53:00', 'last_pruned': '2023-10-08T06:53:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-10-08T06:53:16.488 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-10-08T06:53:16.488 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-10-08T06:53:16.488 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1 2023-10-08T06:53:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.706+0000 7fdf3fa7c640 1 Processor -- start 2023-10-08T06:53:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.706+0000 7fdf3fa7c640 1 -- start start 2023-10-08T06:53:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.708+0000 7fdf3fa7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf380afbe0 0x7fdf380b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.708+0000 7fdf3fa7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf380cb130 0x7fdf380cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.708+0000 7fdf3fa7c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf380cbad0 con 0x7fdf3806dfc0 2023-10-08T06:53:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.708+0000 7fdf3fa7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf380b2620 con 0x7fdf380afbe0 2023-10-08T06:53:16.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.708+0000 7fdf3fa7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380b2060 con 0x7fdf380cb130 2023-10-08T06:53:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.708+0000 7fdf3d818640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf380afbe0 0x7fdf380b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.708+0000 7fdf3d818640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf380afbe0 0x7fdf380b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46662/0 (socket says 172.21.15.70:46662) 2023-10-08T06:53:16.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.708+0000 7fdf3d818640 1 -- 172.21.15.70:0/2085795552 learned_addr learned my addr 172.21.15.70:0/2085795552 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.709+0000 7fdf3d818640 1 -- 172.21.15.70:0/2085795552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf380cb130 msgr2=0x7fdf380cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.709+0000 7fdf3e019640 1 --2- 172.21.15.70:0/2085795552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf380cb130 0x7fdf380cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.709+0000 7fdf3d818640 1 --2- 172.21.15.70:0/2085795552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf380cb130 0x7fdf380cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.709+0000 7fdf3d818640 1 -- 172.21.15.70:0/2085795552 >> v1:172.21.15.181:6789/0 conn(0x7fdf3806dfc0 legacy=0x7fdf38070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.709+0000 7fdf3d818640 1 -- 172.21.15.70:0/2085795552 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380b2340 con 0x7fdf380afbe0 2023-10-08T06:53:16.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.710+0000 7fdf3d818640 1 --2- 172.21.15.70:0/2085795552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380afbe0 0x7fdf380b15b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009870 tx=0x7fdf3402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1863c9ae20aac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.710+0000 7fdf3c816640 1 -- 172.21.15.70:0/2085795552 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34031640 con 0x7fdf380afbe0 2023-10-08T06:53:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.710+0000 7fdf3c816640 1 -- 172.21.15.70:0/2085795552 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf340317a0 con 0x7fdf380afbe0 2023-10-08T06:53:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.710+0000 7fdf3c816640 1 -- 172.21.15.70:0/2085795552 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34031b30 con 0x7fdf380afbe0 2023-10-08T06:53:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.711+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/2085795552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380afbe0 msgr2=0x7fdf380b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.711+0000 7fdf3fa7c640 1 --2- 172.21.15.70:0/2085795552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380afbe0 0x7fdf380b15b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009870 tx=0x7fdf3402b8f0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.711+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/2085795552 shutdown_connections 2023-10-08T06:53:16.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.711+0000 7fdf3fa7c640 1 --2- 172.21.15.70:0/2085795552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf380cb130 0x7fdf380cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.711+0000 7fdf3fa7c640 1 --2- 172.21.15.70:0/2085795552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380afbe0 0x7fdf380b15b0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.711+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/2085795552 >> 172.21.15.70:0/2085795552 conn(0x7fdf3806a5d0 msgr2=0x7fdf380b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.711+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/2085795552 shutdown_connections 2023-10-08T06:53:16.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.711+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/2085795552 wait complete. 2023-10-08T06:53:16.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.713+0000 7fdf3fa7c640 1 Processor -- start 2023-10-08T06:53:16.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.713+0000 7fdf3fa7c640 1 -- start start 2023-10-08T06:53:16.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3fa7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf3806dfc0 0x7fdf380c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3fa7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf380afbe0 0x7fdf3814a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3fa7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380cb130 0x7fdf3815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3fa7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf381f50e0 con 0x7fdf380cb130 2023-10-08T06:53:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3fa7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380c5110 con 0x7fdf380afbe0 2023-10-08T06:53:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3fa7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf381f4e00 con 0x7fdf3806dfc0 2023-10-08T06:53:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3e019640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380cb130 0x7fdf3815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:16.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3e019640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380cb130 0x7fdf3815c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46666/0 (socket says 172.21.15.70:46666) 2023-10-08T06:53:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3e019640 1 -- 172.21.15.70:0/3940871340 learned_addr learned my addr 172.21.15.70:0/3940871340 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3d818640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf3806dfc0 0x7fdf380c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3e019640 1 -- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf3806dfc0 msgr2=0x7fdf380c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3d017640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf380afbe0 0x7fdf3814a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.714+0000 7fdf3e019640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf3806dfc0 0x7fdf380c4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.715+0000 7fdf3e019640 1 -- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf380afbe0 msgr2=0x7fdf3814a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.715+0000 7fdf3e019640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf380afbe0 0x7fdf3814a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.715+0000 7fdf3e019640 1 -- 172.21.15.70:0/3940871340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf340094d0 con 0x7fdf380cb130 2023-10-08T06:53:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.715+0000 7fdf3e019640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380cb130 0x7fdf3815c8d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000cd70 tx=0x7fdf3000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.715+0000 7fdf2e7fc640 1 -- 172.21.15.70:0/3940871340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30003b90 con 0x7fdf380cb130 2023-10-08T06:53:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.715+0000 7fdf2e7fc640 1 -- 172.21.15.70:0/3940871340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf30003cf0 con 0x7fdf380cb130 2023-10-08T06:53:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.715+0000 7fdf2e7fc640 1 -- 172.21.15.70:0/3940871340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30004240 con 0x7fdf380cb130 2023-10-08T06:53:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.715+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/3940871340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf381495e0 con 0x7fdf380cb130 2023-10-08T06:53:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.715+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/3940871340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf38149ba0 con 0x7fdf380cb130 2023-10-08T06:53:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.716+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/3940871340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf38149dc0 con 0x7fdf380cb130 2023-10-08T06:53:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.717+0000 7fdf2e7fc640 1 -- 172.21.15.70:0/3940871340 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 84) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fdf300043a0 con 0x7fdf380cb130 2023-10-08T06:53:16.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.719+0000 7fdf2e7fc640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fdf1c06afe0 0x7fdf1c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:16.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.719+0000 7fdf2e7fc640 1 -- 172.21.15.70:0/3940871340 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fdf30012070 con 0x7fdf380cb130 2023-10-08T06:53:16.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.720+0000 7fdf3d818640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fdf1c06afe0 0x7fdf1c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:16.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.721+0000 7fdf3d818640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fdf1c06afe0 0x7fdf1c06d4a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf34036000 tx=0x7fdf3403c040 comp rx=0 tx=0).ready entity=mgr.6467 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:16.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.723+0000 7fdf2e7fc640 1 -- 172.21.15.70:0/3940871340 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf30055130 con 0x7fdf380cb130 2023-10-08T06:53:16.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:16.935+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/3940871340 --> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf38070630 con 0x7fdf1c06afe0 2023-10-08T06:53:17.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.112+0000 7fdf2e7fc640 1 -- 172.21.15.70:0/3940871340 <== mgr.6467 v2:172.21.15.181:6832/236882560 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7fdf34031a90 con 0x7fdf1c06afe0 2023-10-08T06:53:17.113 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1 2023-10-08T06:53:17.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.115+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fdf1c06afe0 msgr2=0x7fdf1c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:17.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.115+0000 7fdf3fa7c640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fdf1c06afe0 0x7fdf1c06d4a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdf34036000 tx=0x7fdf3403c040 comp rx=0 tx=0).stop 2023-10-08T06:53:17.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.115+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/3940871340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380cb130 msgr2=0x7fdf3815c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:17.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.115+0000 7fdf3fa7c640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380cb130 0x7fdf3815c8d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000cd70 tx=0x7fdf3000a660 comp rx=0 tx=0).stop 2023-10-08T06:53:17.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.116+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/3940871340 shutdown_connections 2023-10-08T06:53:17.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.116+0000 7fdf3fa7c640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fdf1c06afe0 0x7fdf1c06d4a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:17.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.116+0000 7fdf3fa7c640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf380cb130 0x7fdf3815c8d0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:17.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.116+0000 7fdf3fa7c640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf380afbe0 0x7fdf3814a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:17.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.116+0000 7fdf3fa7c640 1 --2- 172.21.15.70:0/3940871340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf3806dfc0 0x7fdf380c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:17.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.116+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/3940871340 >> 172.21.15.70:0/3940871340 conn(0x7fdf3806a5d0 msgr2=0x7fdf380c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:17.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.116+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/3940871340 shutdown_connections 2023-10-08T06:53:17.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:17.116+0000 7fdf3fa7c640 1 -- 172.21.15.70:0/3940871340 wait complete. 2023-10-08T06:53:17.138 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-10-08T06:53:17.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.139 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T06:53:17.199 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-05_59_00_UTC 2023-10-08T06:53:17.199 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_00_00_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_01_00_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_02_00_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_03_00_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_04_00_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_05_00_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_06_00_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_07_00_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_08_01_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_09_00_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_10_00_UTC 2023-10-08T06:53:17.200 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_11_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_12_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_13_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_14_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_15_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_16_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_17_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_18_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_19_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_20_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_21_00_UTC 2023-10-08T06:53:17.201 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_22_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_23_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_24_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_25_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_26_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_27_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_28_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_29_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_30_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_31_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_32_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_33_00_UTC 2023-10-08T06:53:17.202 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_34_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_35_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_36_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_37_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_38_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_39_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_40_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_41_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_42_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_43_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_44_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_45_00_UTC 2023-10-08T06:53:17.203 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_46_00_UTC 2023-10-08T06:53:17.204 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_47_00_UTC 2023-10-08T06:53:17.204 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_48_00_UTC 2023-10-08T06:53:17.204 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_49_00_UTC 2023-10-08T06:53:17.204 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_50_00_UTC 2023-10-08T06:53:17.204 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_51_00_UTC 2023-10-08T06:53:17.204 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_52_00_UTC 2023-10-08T06:53:17.204 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_53_00_UTC 2023-10-08T06:53:17.205 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-05_59_00_UTC 2023-10-08T06:53:17.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.205 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-05_59_00_UTC) 2023-10-08T06:53:17.271 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_00_00_UTC 2023-10-08T06:53:17.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.271 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_00_00_UTC) 2023-10-08T06:53:17.333 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_01_00_UTC 2023-10-08T06:53:17.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.333 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_01_00_UTC) 2023-10-08T06:53:17.394 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_02_00_UTC 2023-10-08T06:53:17.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.395 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_02_00_UTC) 2023-10-08T06:53:17.460 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_03_00_UTC 2023-10-08T06:53:17.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.461 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_03_00_UTC) 2023-10-08T06:53:17.526 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_04_00_UTC 2023-10-08T06:53:17.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.527 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_04_00_UTC) 2023-10-08T06:53:17.588 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_05_00_UTC 2023-10-08T06:53:17.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.589 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_05_00_UTC) 2023-10-08T06:53:17.652 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_06_00_UTC 2023-10-08T06:53:17.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.653 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_06_00_UTC) 2023-10-08T06:53:17.717 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_07_00_UTC 2023-10-08T06:53:17.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.717 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_07_00_UTC) 2023-10-08T06:53:17.787 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_08_01_UTC 2023-10-08T06:53:17.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.788 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_08_01_UTC) 2023-10-08T06:53:17.850 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_09_00_UTC 2023-10-08T06:53:17.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.850 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_09_00_UTC) 2023-10-08T06:53:17.910 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_10_00_UTC 2023-10-08T06:53:17.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.910 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_10_00_UTC) 2023-10-08T06:53:17.975 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_11_00_UTC 2023-10-08T06:53:17.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:17.976 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_11_00_UTC) 2023-10-08T06:53:18.060 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_12_00_UTC 2023-10-08T06:53:18.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.060 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_12_00_UTC) 2023-10-08T06:53:18.114 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_13_00_UTC 2023-10-08T06:53:18.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.114 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_13_00_UTC) 2023-10-08T06:53:18.172 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_14_00_UTC 2023-10-08T06:53:18.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.172 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_14_00_UTC) 2023-10-08T06:53:18.240 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_15_00_UTC 2023-10-08T06:53:18.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.240 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_15_00_UTC) 2023-10-08T06:53:18.295 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_16_00_UTC 2023-10-08T06:53:18.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.296 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_16_00_UTC) 2023-10-08T06:53:18.356 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_17_00_UTC 2023-10-08T06:53:18.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.356 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_17_00_UTC) 2023-10-08T06:53:18.424 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_18_00_UTC 2023-10-08T06:53:18.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.425 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_18_00_UTC) 2023-10-08T06:53:18.478 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_19_00_UTC 2023-10-08T06:53:18.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.478 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_19_00_UTC) 2023-10-08T06:53:18.540 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_20_00_UTC 2023-10-08T06:53:18.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.540 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_20_00_UTC) 2023-10-08T06:53:18.602 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_21_00_UTC 2023-10-08T06:53:18.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.603 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_21_00_UTC) 2023-10-08T06:53:18.661 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_22_00_UTC 2023-10-08T06:53:18.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.661 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_22_00_UTC) 2023-10-08T06:53:18.723 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_23_00_UTC 2023-10-08T06:53:18.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.723 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_23_00_UTC) 2023-10-08T06:53:18.833 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_24_00_UTC 2023-10-08T06:53:18.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.833 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_24_00_UTC) 2023-10-08T06:53:18.883 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_25_00_UTC 2023-10-08T06:53:18.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.883 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_25_00_UTC) 2023-10-08T06:53:18.942 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_26_00_UTC 2023-10-08T06:53:18.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.943 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_26_00_UTC) 2023-10-08T06:53:18.994 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_27_00_UTC 2023-10-08T06:53:18.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:18.995 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_27_00_UTC) 2023-10-08T06:53:19.044 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_28_00_UTC 2023-10-08T06:53:19.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.044 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_28_00_UTC) 2023-10-08T06:53:19.096 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_29_00_UTC 2023-10-08T06:53:19.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.096 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_29_00_UTC) 2023-10-08T06:53:19.153 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_30_00_UTC 2023-10-08T06:53:19.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.153 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_30_00_UTC) 2023-10-08T06:53:19.217 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_31_00_UTC 2023-10-08T06:53:19.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.218 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_31_00_UTC) 2023-10-08T06:53:19.288 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_32_00_UTC 2023-10-08T06:53:19.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.289 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_32_00_UTC) 2023-10-08T06:53:19.353 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_33_00_UTC 2023-10-08T06:53:19.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.354 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_33_00_UTC) 2023-10-08T06:53:19.417 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_34_00_UTC 2023-10-08T06:53:19.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.418 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_34_00_UTC) 2023-10-08T06:53:19.482 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_35_00_UTC 2023-10-08T06:53:19.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.482 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_35_00_UTC) 2023-10-08T06:53:19.552 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_36_00_UTC 2023-10-08T06:53:19.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.553 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_36_00_UTC) 2023-10-08T06:53:19.618 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_37_00_UTC 2023-10-08T06:53:19.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.619 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_37_00_UTC) 2023-10-08T06:53:19.683 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_38_00_UTC 2023-10-08T06:53:19.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.684 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_38_00_UTC) 2023-10-08T06:53:19.753 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_39_00_UTC 2023-10-08T06:53:19.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.753 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_39_00_UTC) 2023-10-08T06:53:19.840 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_40_00_UTC 2023-10-08T06:53:19.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.841 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_40_00_UTC) 2023-10-08T06:53:19.893 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_41_00_UTC 2023-10-08T06:53:19.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.894 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_41_00_UTC) 2023-10-08T06:53:19.942 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_42_00_UTC 2023-10-08T06:53:19.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:19.942 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_42_00_UTC) 2023-10-08T06:53:20.010 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_43_00_UTC 2023-10-08T06:53:20.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.011 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_43_00_UTC) 2023-10-08T06:53:20.072 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_44_00_UTC 2023-10-08T06:53:20.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.072 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_44_00_UTC) 2023-10-08T06:53:20.121 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_45_00_UTC 2023-10-08T06:53:20.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.121 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_45_00_UTC) 2023-10-08T06:53:20.174 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_46_00_UTC 2023-10-08T06:53:20.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.174 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_46_00_UTC) 2023-10-08T06:53:20.227 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_47_00_UTC 2023-10-08T06:53:20.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.227 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_47_00_UTC) 2023-10-08T06:53:20.281 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_48_00_UTC 2023-10-08T06:53:20.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.281 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_48_00_UTC) 2023-10-08T06:53:20.348 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_49_00_UTC 2023-10-08T06:53:20.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.348 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_49_00_UTC) 2023-10-08T06:53:20.413 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_50_00_UTC 2023-10-08T06:53:20.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.414 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_50_00_UTC) 2023-10-08T06:53:20.478 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_51_00_UTC 2023-10-08T06:53:20.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.478 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_51_00_UTC) 2023-10-08T06:53:20.541 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_52_00_UTC 2023-10-08T06:53:20.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.541 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_52_00_UTC) 2023-10-08T06:53:20.604 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-06_53_00_UTC 2023-10-08T06:53:20.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.604 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-06_53_00_UTC) 2023-10-08T06:53:20.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:20.668 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T06:53:20.725 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T06:53:20.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.980+0000 7f81c72e1640 1 Processor -- start 2023-10-08T06:53:20.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.981+0000 7f81c72e1640 1 -- start start 2023-10-08T06:53:20.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.982+0000 7f81c72e1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f81c015cc80 0x7f81c006b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:20.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.982+0000 7f81c72e1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c006a130 con 0x7f81c00cd240 2023-10-08T06:53:20.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.982+0000 7f81c72e1640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c006a410 con 0x7f81c006dab0 2023-10-08T06:53:20.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.982+0000 7f81c72e1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c006b710 con 0x7f81c015cc80 2023-10-08T06:53:20.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.982+0000 7f81c587e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f81c00cd240 0x7f81c00c95b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53022/0 (socket says 172.21.15.70:53022) 2023-10-08T06:53:20.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.982+0000 7f81c587e640 1 -- 172.21.15.70:0/3891559181 learned_addr learned my addr 172.21.15.70:0/3891559181 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:20.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.982+0000 7f81c507d640 1 --2- 172.21.15.70:0/3891559181 >> v2:172.21.15.181:3300/0 conn(0x7f81c015cc80 0x7f81c006b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:20.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.982+0000 7f81b7fff640 1 -- 172.21.15.70:0/3891559181 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806631815 0 0) 0x7f81c006a130 con 0x7f81c00cd240 2023-10-08T06:53:20.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.983+0000 7f81b7fff640 1 -- 172.21.15.70:0/3891559181 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ac003520 con 0x7f81c00cd240 2023-10-08T06:53:20.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.983+0000 7f81b7fff640 1 -- 172.21.15.70:0/3891559181 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705174965 0 0) 0x7f81c006a410 con 0x7f81c006dab0 2023-10-08T06:53:20.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.983+0000 7f81b7fff640 1 -- 172.21.15.70:0/3891559181 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81c006a130 con 0x7f81c006dab0 2023-10-08T06:53:20.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.983+0000 7f81b7fff640 1 -- 172.21.15.70:0/3891559181 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1956245701 0 0) 0x7f81ac003520 con 0x7f81c00cd240 2023-10-08T06:53:20.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.983+0000 7f81c507d640 1 -- 172.21.15.70:0/3891559181 >> v1:172.21.15.181:6789/0 conn(0x7f81c006dab0 legacy=0x7f81c00ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:20.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.983+0000 7f81c507d640 1 -- 172.21.15.70:0/3891559181 >> v1:172.21.15.70:6789/0 conn(0x7f81c00cd240 legacy=0x7f81c00c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:20.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.983+0000 7f81c507d640 1 -- 172.21.15.70:0/3891559181 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c006a6f0 con 0x7f81c015cc80 2023-10-08T06:53:20.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.984+0000 7f81c507d640 1 --2- 172.21.15.70:0/3891559181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c015cc80 0x7f81c006b1d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0097a0 tx=0x7f81bc02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98bf7e5025001cff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:20.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.984+0000 7f81b7fff640 1 -- 172.21.15.70:0/3891559181 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc02f050 con 0x7f81c015cc80 2023-10-08T06:53:20.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.984+0000 7f81b7fff640 1 -- 172.21.15.70:0/3891559181 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81bc031500 con 0x7f81c015cc80 2023-10-08T06:53:20.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.984+0000 7f81b7fff640 1 -- 172.21.15.70:0/3891559181 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b80019f0 con 0x7f81c015cc80 2023-10-08T06:53:20.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.985+0000 7f81c72e1640 1 -- 172.21.15.70:0/3891559181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c015cc80 msgr2=0x7f81c006b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:20.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.985+0000 7f81c72e1640 1 --2- 172.21.15.70:0/3891559181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c015cc80 0x7f81c006b1d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0097a0 tx=0x7f81bc02b820 comp rx=0 tx=0).stop 2023-10-08T06:53:20.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.985+0000 7f81c72e1640 1 -- 172.21.15.70:0/3891559181 shutdown_connections 2023-10-08T06:53:20.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.985+0000 7f81c72e1640 1 --2- 172.21.15.70:0/3891559181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c015cc80 0x7f81c006b1d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:20.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.985+0000 7f81c72e1640 1 -- 172.21.15.70:0/3891559181 >> 172.21.15.70:0/3891559181 conn(0x7f81c00b0ae0 msgr2=0x7f81c00b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:20.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.985+0000 7f81c72e1640 1 -- 172.21.15.70:0/3891559181 shutdown_connections 2023-10-08T06:53:20.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.985+0000 7f81c72e1640 1 -- 172.21.15.70:0/3891559181 wait complete. 2023-10-08T06:53:20.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.987+0000 7f81c72e1640 1 Processor -- start 2023-10-08T06:53:20.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.987+0000 7f81c72e1640 1 -- start start 2023-10-08T06:53:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c72e1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c006dab0 0x7f81c01f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c72e1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81c00cd240 0x7f81c00c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c72e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c015cc80 0x7f81c00acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c72e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00b2d30 con 0x7f81c015cc80 2023-10-08T06:53:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c72e1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00b1e40 con 0x7f81c006dab0 2023-10-08T06:53:20.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c72e1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00b2120 con 0x7f81c00cd240 2023-10-08T06:53:20.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c587e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c015cc80 0x7f81c00acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:20.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c587e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c015cc80 0x7f81c00acaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46672/0 (socket says 172.21.15.70:46672) 2023-10-08T06:53:20.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c587e640 1 -- 172.21.15.70:0/3674042755 learned_addr learned my addr 172.21.15.70:0/3674042755 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:20.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c507d640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c006dab0 0x7f81c01f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:20.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.988+0000 7f81c487c640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81c00cd240 0x7f81c00c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:20.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.989+0000 7f81c587e640 1 -- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81c00cd240 msgr2=0x7f81c00c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:20.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.989+0000 7f81c587e640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81c00cd240 0x7f81c00c4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:20.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.989+0000 7f81c587e640 1 -- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c006dab0 msgr2=0x7f81c01f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:20.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.989+0000 7f81c587e640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c006dab0 0x7f81c01f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:20.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.989+0000 7f81c587e640 1 -- 172.21.15.70:0/3674042755 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc009400 con 0x7f81c015cc80 2023-10-08T06:53:20.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.989+0000 7f81c587e640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c015cc80 0x7f81c00acaa0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f81b8002000 tx=0x7f81b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.989+0000 7f81b5ffb640 1 -- 172.21.15.70:0/3674042755 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b8012400 con 0x7f81c015cc80 2023-10-08T06:53:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.989+0000 7f81b5ffb640 1 -- 172.21.15.70:0/3674042755 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b8012580 con 0x7f81c015cc80 2023-10-08T06:53:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.989+0000 7f81b5ffb640 1 -- 172.21.15.70:0/3674042755 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b8012970 con 0x7f81c015cc80 2023-10-08T06:53:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.989+0000 7f81c72e1640 1 -- 172.21.15.70:0/3674042755 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c00b2400 con 0x7f81c015cc80 2023-10-08T06:53:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.990+0000 7f81c72e1640 1 -- 172.21.15.70:0/3674042755 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c00b29c0 con 0x7f81c015cc80 2023-10-08T06:53:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.991+0000 7f81c72e1640 1 -- 172.21.15.70:0/3674042755 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c00b2ea0 con 0x7f81c015cc80 2023-10-08T06:53:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.992+0000 7f81b5ffb640 1 -- 172.21.15.70:0/3674042755 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 84) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f81b8012ad0 con 0x7f81c015cc80 2023-10-08T06:53:20.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.993+0000 7f81b5ffb640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f81a406afe0 0x7f81a406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:20.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.993+0000 7f81b5ffb640 1 -- 172.21.15.70:0/3674042755 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f81b808a250 con 0x7f81c015cc80 2023-10-08T06:53:20.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.993+0000 7f81c507d640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f81a406afe0 0x7f81a406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:20.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.994+0000 7f81c507d640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f81a406afe0 0x7f81a406d4a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f81bc035000 tx=0x7f81bc008030 comp rx=0 tx=0).ready entity=mgr.6467 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:20.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:20.997+0000 7f81b5ffb640 1 -- 172.21.15.70:0/3674042755 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81b8054f60 con 0x7f81c015cc80 2023-10-08T06:53:21.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.262+0000 7f81c72e1640 1 -- 172.21.15.70:0/3674042755 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f81c00c93b0 con 0x7f81c015cc80 2023-10-08T06:53:21.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.781+0000 7f81b5ffb640 1 -- 172.21.15.70:0/3674042755 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v85) v1 ==== 93+0+0 (secure 0 0 0) 0x7f81b8058580 con 0x7f81c015cc80 2023-10-08T06:53:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.785+0000 7f81c72e1640 1 -- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f81a406afe0 msgr2=0x7f81a406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.785+0000 7f81c72e1640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f81a406afe0 0x7f81a406d4a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f81bc035000 tx=0x7f81bc008030 comp rx=0 tx=0).stop 2023-10-08T06:53:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.785+0000 7f81c72e1640 1 -- 172.21.15.70:0/3674042755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c015cc80 msgr2=0x7f81c00acaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.785+0000 7f81c72e1640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c015cc80 0x7f81c00acaa0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f81b8002000 tx=0x7f81b800a430 comp rx=0 tx=0).stop 2023-10-08T06:53:21.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.785+0000 7f81c72e1640 1 -- 172.21.15.70:0/3674042755 shutdown_connections 2023-10-08T06:53:21.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.785+0000 7f81c72e1640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f81a406afe0 0x7f81a406d4a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:21.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.785+0000 7f81c72e1640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81c015cc80 0x7f81c00acaa0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:21.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.785+0000 7f81c72e1640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81c00cd240 0x7f81c00c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:21.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.785+0000 7f81c72e1640 1 --2- 172.21.15.70:0/3674042755 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81c006dab0 0x7f81c01f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:21.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.785+0000 7f81c72e1640 1 -- 172.21.15.70:0/3674042755 >> 172.21.15.70:0/3674042755 conn(0x7f81c00b0ae0 msgr2=0x7f81c00b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:21.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.786+0000 7f81c72e1640 1 -- 172.21.15.70:0/3674042755 shutdown_connections 2023-10-08T06:53:21.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:21.786+0000 7f81c72e1640 1 -- 172.21.15.70:0/3674042755 wait complete. 2023-10-08T06:53:21.813 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T06:53:21.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T06:53:21.975 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T06:53:22.068 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T06:53:22.068 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:53:22.068 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:53:22.117 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:53:22.118 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:53:22.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:53:22.396 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:22.396+0000 7fed3cc32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T06:53:22.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.416+0000 7f746b5a9640 1 Processor -- start 2023-10-08T06:53:22.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.416+0000 7f746b5a9640 1 -- start start 2023-10-08T06:53:22.417 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:22.416+0000 7f04e5612040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T06:53:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.418+0000 7f746b5a9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f746415ab40 0x7f74640ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.418+0000 7f746b5a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 0x7f74640adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.418+0000 7f746b5a9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746415b2b0 con 0x7f74640cae70 2023-10-08T06:53:22.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.418+0000 7f746b5a9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746415bf30 con 0x7f746415ab40 2023-10-08T06:53:22.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.418+0000 7f746b5a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746415f7a0 con 0x7f74640c90b0 2023-10-08T06:53:22.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.418+0000 7f7469345640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f746415ab40 0x7f74640ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.418+0000 7f7468b44640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 0x7f74640adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.418+0000 7f7469345640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f746415ab40 0x7f74640ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:56422/0 (socket says 172.21.15.70:56422) 2023-10-08T06:53:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.418+0000 7f7469345640 1 -- 172.21.15.70:0/1764687698 learned_addr learned my addr 172.21.15.70:0/1764687698 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.419+0000 7f7469345640 1 -- 172.21.15.70:0/1764687698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 msgr2=0x7f74640adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.419+0000 7f7469345640 1 --2- 172.21.15.70:0/1764687698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 0x7f74640adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.419+0000 7f7469345640 1 -- 172.21.15.70:0/1764687698 >> v1:172.21.15.181:6789/0 conn(0x7f74640cae70 legacy=0x7f74640702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.419+0000 7f7469345640 1 -- 172.21.15.70:0/1764687698 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746415b590 con 0x7f746415ab40 2023-10-08T06:53:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.420+0000 7f7469345640 1 --2- 172.21.15.70:0/1764687698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746415ab40 0x7f74640ad720 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7460009870 tx=0x7f746002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aa3452b5f98f6b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.420+0000 7f745bfff640 1 -- 172.21.15.70:0/1764687698 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460030840 con 0x7f746415ab40 2023-10-08T06:53:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.420+0000 7f745bfff640 1 -- 172.21.15.70:0/1764687698 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74600309a0 con 0x7f746415ab40 2023-10-08T06:53:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.420+0000 7f745bfff640 1 -- 172.21.15.70:0/1764687698 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460030d30 con 0x7f746415ab40 2023-10-08T06:53:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.421+0000 7f746b5a9640 1 -- 172.21.15.70:0/1764687698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746415ab40 msgr2=0x7f74640ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.421+0000 7f746b5a9640 1 --2- 172.21.15.70:0/1764687698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746415ab40 0x7f74640ad720 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7460009870 tx=0x7f746002b980 comp rx=0 tx=0).stop 2023-10-08T06:53:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.421+0000 7f746b5a9640 1 -- 172.21.15.70:0/1764687698 shutdown_connections 2023-10-08T06:53:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.421+0000 7f746b5a9640 1 --2- 172.21.15.70:0/1764687698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 0x7f74640adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.421+0000 7f746b5a9640 1 --2- 172.21.15.70:0/1764687698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746415ab40 0x7f74640ad720 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.421+0000 7f746b5a9640 1 -- 172.21.15.70:0/1764687698 >> 172.21.15.70:0/1764687698 conn(0x7f746406a380 msgr2=0x7f74640b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.421+0000 7f746b5a9640 1 -- 172.21.15.70:0/1764687698 shutdown_connections 2023-10-08T06:53:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.422+0000 7f746b5a9640 1 -- 172.21.15.70:0/1764687698 wait complete. 2023-10-08T06:53:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.424+0000 7f746b5a9640 1 Processor -- start 2023-10-08T06:53:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.424+0000 7f746b5a9640 1 -- start start 2023-10-08T06:53:22.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.425+0000 7f746b5a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 0x7f74640c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.425+0000 7f746b5a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640cae70 0x7f7464154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.425+0000 7f746b5a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746415ab40 0x7f7464155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.425+0000 7f746b5a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74640c3620 con 0x7f74640cae70 2023-10-08T06:53:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.425+0000 7f746b5a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74641f3040 con 0x7f746415ab40 2023-10-08T06:53:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.425+0000 7f746b5a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74641f3320 con 0x7f74640c90b0 2023-10-08T06:53:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.425+0000 7f7468b44640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640cae70 0x7f7464154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.425+0000 7f7468b44640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640cae70 0x7f7464154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46702/0 (socket says 172.21.15.70:46702) 2023-10-08T06:53:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.425+0000 7f7468b44640 1 -- 172.21.15.70:0/2286744251 learned_addr learned my addr 172.21.15.70:0/2286744251 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.425+0000 7f7469b46640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746415ab40 0x7f7464155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:22.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.426+0000 7f7469345640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 0x7f74640c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.426+0000 7f7468b44640 1 -- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 msgr2=0x7f74640c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.426+0000 7f7468b44640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 0x7f74640c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.426+0000 7f7468b44640 1 -- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746415ab40 msgr2=0x7f7464155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.426+0000 7f7468b44640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746415ab40 0x7f7464155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.426+0000 7f7468b44640 1 -- 172.21.15.70:0/2286744251 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74600094d0 con 0x7f74640cae70 2023-10-08T06:53:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.426+0000 7f7469345640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 0x7f74640c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T06:53:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.426+0000 7f7468b44640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640cae70 0x7f7464154cf0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f745400e870 tx=0x7f745400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.427+0000 7f7459ffb640 1 -- 172.21.15.70:0/2286744251 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74540094f0 con 0x7f74640cae70 2023-10-08T06:53:22.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.427+0000 7f7459ffb640 1 -- 172.21.15.70:0/2286744251 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745400c940 con 0x7f74640cae70 2023-10-08T06:53:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.427+0000 7f7459ffb640 1 -- 172.21.15.70:0/2286744251 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745400ccf0 con 0x7f74640cae70 2023-10-08T06:53:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.427+0000 7f746b5a9640 1 -- 172.21.15.70:0/2286744251 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74640c38c0 con 0x7f74640cae70 2023-10-08T06:53:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.427+0000 7f746b5a9640 1 -- 172.21.15.70:0/2286744251 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74640c4600 con 0x7f74640cae70 2023-10-08T06:53:22.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.428+0000 7f746b5a9640 1 -- 172.21.15.70:0/2286744251 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7464067490 con 0x7f74640cae70 2023-10-08T06:53:22.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.430+0000 7f7459ffb640 1 -- 172.21.15.70:0/2286744251 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 85) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f7454013470 con 0x7f74640cae70 2023-10-08T06:53:22.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.431+0000 7f7459ffb640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f744806af90 0x7f744806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.431+0000 7f7459ffb640 1 -- 172.21.15.70:0/2286744251 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f7454089be0 con 0x7f74640cae70 2023-10-08T06:53:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.431+0000 7f7469345640 1 -- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f744806af90 msgr2=0x7f744806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.431+0000 7f7469345640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f744806af90 0x7f744806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:53:22.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.435+0000 7f7459ffb640 1 -- 172.21.15.70:0/2286744251 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7454054850 con 0x7f74640cae70 2023-10-08T06:53:22.457 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:22.457+0000 7fed3cc32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T06:53:22.477 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:22.476+0000 7f04e5612040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T06:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.632+0000 7f7469345640 1 -- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f744806af90 msgr2=0x7f744806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:22.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.632+0000 7f7469345640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f744806af90 0x7f744806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:53:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.665+0000 7f746b5a9640 1 -- 172.21.15.70:0/2286744251 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74640acb80 con 0x7f74640cae70 2023-10-08T06:53:22.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.666+0000 7f7459ffb640 1 -- 172.21.15.70:0/2286744251 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+17511 (secure 0 0 0) 0x7f7454017090 con 0x7f74640cae70 2023-10-08T06:53:22.667 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:22.667 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":170,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:53:21.777623+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:57:23.815675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-08T05:57:25.045292+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":170,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3947544788":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3693343940":"2023-10-09T05:57:41.519220+0000","172.21.15.181:6833/2068783299":"2023-10-09T05:57:41.519220+0000","172.21.15.181:6832/2068783299":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3070110051":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3604402742":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/1336064042":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3791109415":"2023-10-09T05:57:41.519220+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":42,"length":15},{"begin":98,"length":15}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:53:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 -- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f744806af90 msgr2=0x7f744806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:53:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f744806af90 0x7f744806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 -- 172.21.15.70:0/2286744251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640cae70 msgr2=0x7f7464154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640cae70 0x7f7464154cf0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f745400e870 tx=0x7f745400ed30 comp rx=0 tx=0).stop 2023-10-08T06:53:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 -- 172.21.15.70:0/2286744251 shutdown_connections 2023-10-08T06:53:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f744806af90 0x7f744806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f746415ab40 0x7f7464155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640cae70 0x7f7464154cf0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 --2- 172.21.15.70:0/2286744251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f74640c90b0 0x7f74640c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 -- 172.21.15.70:0/2286744251 >> 172.21.15.70:0/2286744251 conn(0x7f746406a380 msgr2=0x7f74640ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 -- 172.21.15.70:0/2286744251 shutdown_connections 2023-10-08T06:53:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.669+0000 7f746b5a9640 1 -- 172.21.15.70:0/2286744251 wait complete. 2023-10-08T06:53:22.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:57:23.815675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:57:25.045292+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 170, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T06:53:22.688 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T06:53:22.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.885+0000 7f06bc10c640 1 Processor -- start 2023-10-08T06:53:22.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.885+0000 7f06bc10c640 1 -- start start 2023-10-08T06:53:22.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.886+0000 7f06bc10c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06b415aad0 0x7f06b40b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.886+0000 7f06bc10c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06b406d410 0x7f06b40c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.886+0000 7f06bc10c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b415bf30 con 0x7f06b40c82b0 2023-10-08T06:53:22.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.886+0000 7f06bc10c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b415b320 con 0x7f06b406d410 2023-10-08T06:53:22.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.886+0000 7f06bc10c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b415b600 con 0x7f06b415aad0 2023-10-08T06:53:22.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.887+0000 7f06b96a7640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f06b40c82b0 0x7f06b406cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53038/0 (socket says 172.21.15.70:53038) 2023-10-08T06:53:22.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.887+0000 7f06b96a7640 1 -- 172.21.15.70:0/2322883659 learned_addr learned my addr 172.21.15.70:0/2322883659 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:22.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.887+0000 7f06ba6a9640 1 --2- 172.21.15.70:0/2322883659 >> v2:172.21.15.181:3300/0 conn(0x7f06b406d410 0x7f06b40c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:22.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.887+0000 7f06b9ea8640 1 --2- 172.21.15.70:0/2322883659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06b415aad0 0x7f06b40b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:22.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.887+0000 7f06b8ea6640 1 -- 172.21.15.70:0/2322883659 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3348884940 0 0) 0x7f06b415bf30 con 0x7f06b40c82b0 2023-10-08T06:53:22.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.888+0000 7f06b8ea6640 1 -- 172.21.15.70:0/2322883659 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a0003520 con 0x7f06b40c82b0 2023-10-08T06:53:22.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.888+0000 7f06ba6a9640 1 -- 172.21.15.70:0/2322883659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06b415aad0 msgr2=0x7f06b40b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:22.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.888+0000 7f06ba6a9640 1 --2- 172.21.15.70:0/2322883659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06b415aad0 0x7f06b40b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.888+0000 7f06b8ea6640 1 -- 172.21.15.70:0/2322883659 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3669351206 0 0) 0x7f06a0003520 con 0x7f06b40c82b0 2023-10-08T06:53:22.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.888+0000 7f06ba6a9640 1 -- 172.21.15.70:0/2322883659 >> v1:172.21.15.70:6789/0 conn(0x7f06b40c82b0 legacy=0x7f06b406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:22.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.888+0000 7f06ba6a9640 1 -- 172.21.15.70:0/2322883659 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b415b8e0 con 0x7f06b406d410 2023-10-08T06:53:22.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.889+0000 7f06ba6a9640 1 --2- 172.21.15.70:0/2322883659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b406d410 0x7f06b40c8710 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009d90 tx=0x7f06ac02bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d38e806e12ecd34a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:22.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.889+0000 7f06b8ea6640 1 -- 172.21.15.70:0/2322883659 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a4001670 con 0x7f06b406d410 2023-10-08T06:53:22.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.890+0000 7f06b8ea6640 1 -- 172.21.15.70:0/2322883659 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ac02c8a0 con 0x7f06b406d410 2023-10-08T06:53:22.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.890+0000 7f06b8ea6640 1 -- 172.21.15.70:0/2322883659 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac02cc10 con 0x7f06b406d410 2023-10-08T06:53:22.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.890+0000 7f06bc10c640 1 -- 172.21.15.70:0/2322883659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b406d410 msgr2=0x7f06b40c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:22.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.890+0000 7f06bc10c640 1 --2- 172.21.15.70:0/2322883659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b406d410 0x7f06b40c8710 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009d90 tx=0x7f06ac02bd70 comp rx=0 tx=0).stop 2023-10-08T06:53:22.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.890+0000 7f06bc10c640 1 -- 172.21.15.70:0/2322883659 shutdown_connections 2023-10-08T06:53:22.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.890+0000 7f06bc10c640 1 --2- 172.21.15.70:0/2322883659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b406d410 0x7f06b40c8710 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.890+0000 7f06bc10c640 1 --2- 172.21.15.70:0/2322883659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06b415aad0 0x7f06b40b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.890+0000 7f06bc10c640 1 -- 172.21.15.70:0/2322883659 >> 172.21.15.70:0/2322883659 conn(0x7f06b4069d50 msgr2=0x7f06b406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:22.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.890+0000 7f06bc10c640 1 -- 172.21.15.70:0/2322883659 shutdown_connections 2023-10-08T06:53:22.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.890+0000 7f06bc10c640 1 -- 172.21.15.70:0/2322883659 wait complete. 2023-10-08T06:53:22.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.892+0000 7f06bc10c640 1 Processor -- start 2023-10-08T06:53:22.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.892+0000 7f06bc10c640 1 -- start start 2023-10-08T06:53:22.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.893+0000 7f06bc10c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b406d410 0x7f06b40c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.893+0000 7f06bc10c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06b40c82b0 0x7f06b41f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.893+0000 7f06bc10c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b415aad0 0x7f06b4157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.893+0000 7f06bc10c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41efd70 con 0x7f06b415aad0 2023-10-08T06:53:22.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.893+0000 7f06bc10c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40ca030 con 0x7f06b406d410 2023-10-08T06:53:22.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.893+0000 7f06bc10c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b41f3280 con 0x7f06b40c82b0 2023-10-08T06:53:22.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.893+0000 7f06ba6a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b415aad0 0x7f06b4157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:22.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.893+0000 7f06ba6a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b415aad0 0x7f06b4157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46704/0 (socket says 172.21.15.70:46704) 2023-10-08T06:53:22.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.893+0000 7f06ba6a9640 1 -- 172.21.15.70:0/808656465 learned_addr learned my addr 172.21.15.70:0/808656465 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:22.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06b9ea8640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b406d410 0x7f06b40c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:22.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06b96a7640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06b40c82b0 0x7f06b41f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:22.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06ba6a9640 1 -- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06b40c82b0 msgr2=0x7f06b41f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:22.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06ba6a9640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06b40c82b0 0x7f06b41f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06ba6a9640 1 -- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b406d410 msgr2=0x7f06b40c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:22.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06ba6a9640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b406d410 0x7f06b40c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:22.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06ba6a9640 1 -- 172.21.15.70:0/808656465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac009a70 con 0x7f06b415aad0 2023-10-08T06:53:22.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06ba6a9640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b415aad0 0x7f06b4157ea0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f06ac02c380 tx=0x7f06ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:22.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06aa7fc640 1 -- 172.21.15.70:0/808656465 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac035e60 con 0x7f06b415aad0 2023-10-08T06:53:22.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06aa7fc640 1 -- 172.21.15.70:0/808656465 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ac02ce90 con 0x7f06b415aad0 2023-10-08T06:53:22.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06bc10c640 1 -- 172.21.15.70:0/808656465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b40c9420 con 0x7f06b415aad0 2023-10-08T06:53:22.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.894+0000 7f06aa7fc640 1 -- 172.21.15.70:0/808656465 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac03e630 con 0x7f06b415aad0 2023-10-08T06:53:22.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.895+0000 7f06bc10c640 1 -- 172.21.15.70:0/808656465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b40c99e0 con 0x7f06b415aad0 2023-10-08T06:53:22.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.896+0000 7f06bc10c640 1 -- 172.21.15.70:0/808656465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b404e160 con 0x7f06b415aad0 2023-10-08T06:53:22.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.897+0000 7f06aa7fc640 1 -- 172.21.15.70:0/808656465 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 85) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f06ac03b050 con 0x7f06b415aad0 2023-10-08T06:53:22.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.898+0000 7f06aa7fc640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f069806af90 0x7f069806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:22.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.898+0000 7f06aa7fc640 1 -- 172.21.15.70:0/808656465 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f06ac0aba10 con 0x7f06b415aad0 2023-10-08T06:53:22.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.898+0000 7f06b9ea8640 1 -- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f069806af90 msgr2=0x7f069806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:22.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.898+0000 7f06b9ea8640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f069806af90 0x7f069806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:53:22.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:22.906+0000 7f06aa7fc640 1 -- 172.21.15.70:0/808656465 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f06ac076660 con 0x7f06b415aad0 2023-10-08T06:53:22.964 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:22.963+0000 7fed3cc32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T06:53:22.964 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:22.964+0000 7f04e5612040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T06:53:23.017 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:23.016+0000 7f04e5612040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T06:53:23.017 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:23.017+0000 7fed3cc32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T06:53:23.067 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:23.066+0000 7f04e5612040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T06:53:23.068 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:23.068+0000 7fed3cc32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T06:53:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.095+0000 7f06bc10c640 1 -- 172.21.15.70:0/808656465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06b40ca170 con 0x7f06b415aad0 2023-10-08T06:53:23.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.096+0000 7f06aa7fc640 1 -- 172.21.15.70:0/808656465 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v170) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06ac009450 con 0x7f06b415aad0 2023-10-08T06:53:23.096 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T06:53:23.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 -- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f069806af90 msgr2=0x7f069806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:53:23.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f069806af90 0x7f069806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 -- 172.21.15.70:0/808656465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b415aad0 msgr2=0x7f06b4157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:23.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b415aad0 0x7f06b4157ea0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f06ac02c380 tx=0x7f06ac004a40 comp rx=0 tx=0).stop 2023-10-08T06:53:23.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 -- 172.21.15.70:0/808656465 shutdown_connections 2023-10-08T06:53:23.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f069806af90 0x7f069806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b415aad0 0x7f06b4157ea0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06b40c82b0 0x7f06b41f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 --2- 172.21.15.70:0/808656465 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06b406d410 0x7f06b40c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 -- 172.21.15.70:0/808656465 >> 172.21.15.70:0/808656465 conn(0x7f06b4069d50 msgr2=0x7f06b4159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:23.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 -- 172.21.15.70:0/808656465 shutdown_connections 2023-10-08T06:53:23.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.098+0000 7f06bc10c640 1 -- 172.21.15.70:0/808656465 wait complete. 2023-10-08T06:53:23.115 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T06:53:23.127 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:23.126+0000 7f04e5612040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T06:53:23.132 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:23.132+0000 7fed3cc32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T06:53:23.176 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:23.175+0000 7f04e5612040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T06:53:23.184 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:23.183+0000 7fed3cc32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T06:53:23.286 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:23.285+0000 7f04e5612040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T06:53:23.300 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:23.299+0000 7fed3cc32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T06:53:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.318+0000 7f0c1ed90640 1 Processor -- start 2023-10-08T06:53:23.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.318+0000 7f0c1ed90640 1 -- start start 2023-10-08T06:53:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.319+0000 7f0c1ed90640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0c1806c3a0 0x7f0c180c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.319+0000 7f0c1ed90640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0c1806db60 0x7f0c1806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.319+0000 7f0c1ed90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c1815aa00 0x7f0c180b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.319+0000 7f0c1ed90640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c180ca230 con 0x7f0c1806c3a0 2023-10-08T06:53:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.319+0000 7f0c1ed90640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1815b310 con 0x7f0c1806db60 2023-10-08T06:53:23.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.319+0000 7f0c1ed90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1806d290 con 0x7f0c1815aa00 2023-10-08T06:53:23.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.320+0000 7f0c1cb2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0c1806c3a0 0x7f0c180c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:23.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.320+0000 7f0c1cb2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0c1806c3a0 0x7f0c180c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46706/0 (socket says 172.21.15.70:46706) 2023-10-08T06:53:23.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.320+0000 7f0c1cb2c640 1 -- 172.21.15.70:0/36006654 learned_addr learned my addr 172.21.15.70:0/36006654 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:23.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.320+0000 7f0c17fff640 1 --2- 172.21.15.70:0/36006654 >> v2:172.21.15.181:3300/0 conn(0x7f0c1806db60 0x7f0c1806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.320+0000 7f0c1d32d640 1 --2- 172.21.15.70:0/36006654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c1815aa00 0x7f0c180b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.320+0000 7f0c1cb2c640 1 -- 172.21.15.70:0/36006654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c1815aa00 msgr2=0x7f0c180b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.320+0000 7f0c1cb2c640 1 --2- 172.21.15.70:0/36006654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c1815aa00 0x7f0c180b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.320+0000 7f0c1cb2c640 1 -- 172.21.15.70:0/36006654 >> v2:172.21.15.181:3300/0 conn(0x7f0c1806db60 msgr2=0x7f0c1806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.320+0000 7f0c1cb2c640 1 --2- 172.21.15.70:0/36006654 >> v2:172.21.15.181:3300/0 conn(0x7f0c1806db60 0x7f0c1806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.320+0000 7f0c1cb2c640 1 -- 172.21.15.70:0/36006654 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c180c9860 con 0x7f0c1806c3a0 2023-10-08T06:53:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.322+0000 7f0c1cb2c640 1 --2- 172.21.15.70:0/36006654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806c3a0 0x7f0c180c9cf0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f0c180715c0 tx=0x7f0c0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb707fcb401ea0a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.322+0000 7f0c177fe640 1 -- 172.21.15.70:0/36006654 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c031870 con 0x7f0c1806c3a0 2023-10-08T06:53:23.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.322+0000 7f0c177fe640 1 -- 172.21.15.70:0/36006654 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c0319d0 con 0x7f0c1806c3a0 2023-10-08T06:53:23.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.322+0000 7f0c177fe640 1 -- 172.21.15.70:0/36006654 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c031d60 con 0x7f0c1806c3a0 2023-10-08T06:53:23.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.323+0000 7f0c1ed90640 1 -- 172.21.15.70:0/36006654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806c3a0 msgr2=0x7f0c180c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:23.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.323+0000 7f0c1ed90640 1 --2- 172.21.15.70:0/36006654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806c3a0 0x7f0c180c9cf0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f0c180715c0 tx=0x7f0c0c02b910 comp rx=0 tx=0).stop 2023-10-08T06:53:23.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.323+0000 7f0c1ed90640 1 -- 172.21.15.70:0/36006654 shutdown_connections 2023-10-08T06:53:23.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.323+0000 7f0c1ed90640 1 --2- 172.21.15.70:0/36006654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c1815aa00 0x7f0c180b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.323+0000 7f0c1ed90640 1 --2- 172.21.15.70:0/36006654 >> v2:172.21.15.181:3300/0 conn(0x7f0c1806db60 0x7f0c1806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.323+0000 7f0c1ed90640 1 --2- 172.21.15.70:0/36006654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806c3a0 0x7f0c180c9cf0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.323+0000 7f0c1ed90640 1 -- 172.21.15.70:0/36006654 >> 172.21.15.70:0/36006654 conn(0x7f0c1806a5d0 msgr2=0x7f0c180b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:23.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.323+0000 7f0c1ed90640 1 -- 172.21.15.70:0/36006654 shutdown_connections 2023-10-08T06:53:23.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.323+0000 7f0c1ed90640 1 -- 172.21.15.70:0/36006654 wait complete. 2023-10-08T06:53:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.325+0000 7f0c1ed90640 1 Processor -- start 2023-10-08T06:53:23.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.325+0000 7f0c1ed90640 1 -- start start 2023-10-08T06:53:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c1ed90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c1806c3a0 0x7f0c18159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c1ed90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806db60 0x7f0c18155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c1ed90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1815aa00 0x7f0c1815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c1ed90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1815e0b0 con 0x7f0c1806db60 2023-10-08T06:53:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c1ed90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1815a250 con 0x7f0c1815aa00 2023-10-08T06:53:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c1ed90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1815d4a0 con 0x7f0c1806c3a0 2023-10-08T06:53:23.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806db60 0x7f0c18155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806db60 0x7f0c18155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:46708/0 (socket says 172.21.15.70:46708) 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c17fff640 1 -- 172.21.15.70:0/1232582371 learned_addr learned my addr 172.21.15.70:0/1232582371 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c1d32d640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1815aa00 0x7f0c1815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.326+0000 7f0c1cb2c640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c1806c3a0 0x7f0c18159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c17fff640 1 -- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c1806c3a0 msgr2=0x7f0c18159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c17fff640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c1806c3a0 0x7f0c18159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c17fff640 1 -- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1815aa00 msgr2=0x7f0c1815b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c17fff640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1815aa00 0x7f0c1815b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c17fff640 1 -- 172.21.15.70:0/1232582371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c0094d0 con 0x7f0c1806db60 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c1d32d640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1815aa00 0x7f0c1815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T06:53:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c17fff640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806db60 0x7f0c18155f30 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400e8c0 tx=0x7f0c0400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c157fa640 1 -- 172.21.15.70:0/1232582371 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0400c940 con 0x7f0c1806db60 2023-10-08T06:53:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c157fa640 1 -- 172.21.15.70:0/1232582371 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0400caa0 con 0x7f0c1806db60 2023-10-08T06:53:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c157fa640 1 -- 172.21.15.70:0/1232582371 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0400ce30 con 0x7f0c1806db60 2023-10-08T06:53:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.327+0000 7f0c1ed90640 1 -- 172.21.15.70:0/1232582371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1815d780 con 0x7f0c1806db60 2023-10-08T06:53:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.328+0000 7f0c1ed90640 1 -- 172.21.15.70:0/1232582371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1815dd40 con 0x7f0c1806db60 2023-10-08T06:53:23.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.329+0000 7f0c1ed90640 1 -- 172.21.15.70:0/1232582371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1815e260 con 0x7f0c1806db60 2023-10-08T06:53:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.331+0000 7f0c157fa640 1 -- 172.21.15.70:0/1232582371 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 85) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f0c04013470 con 0x7f0c1806db60 2023-10-08T06:53:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.331+0000 7f0c157fa640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f0bfc06af90 0x7f0bfc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.332+0000 7f0c157fa640 1 -- 172.21.15.70:0/1232582371 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f0c04089c70 con 0x7f0c1806db60 2023-10-08T06:53:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.332+0000 7f0c1cb2c640 1 -- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f0bfc06af90 msgr2=0x7f0bfc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:23.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.332+0000 7f0c1cb2c640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f0bfc06af90 0x7f0bfc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:53:23.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.336+0000 7f0c157fa640 1 -- 172.21.15.70:0/1232582371 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c1815e260 con 0x7f0c1806db60 2023-10-08T06:53:23.350 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:23.349+0000 7f04e5612040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T06:53:23.368 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:23.367+0000 7fed3cc32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T06:53:23.400 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:23.398+0000 7f04e5612040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T06:53:23.418 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:23.417+0000 7fed3cc32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T06:53:23.455 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:23.454+0000 7f04e5612040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T06:53:23.474 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:23.474+0000 7fed3cc32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T06:53:23.505 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:23.504+0000 7f04e5612040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T06:53:23.524 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:23.524+0000 7fed3cc32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T06:53:23.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.526+0000 7f0c1ed90640 1 -- 172.21.15.70:0/1232582371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c1815e260 con 0x7f0c1806db60 2023-10-08T06:53:23.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.527+0000 7f0c157fa640 1 -- 172.21.15.70:0/1232582371 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v170) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c1815e260 con 0x7f0c1806db60 2023-10-08T06:53:23.528 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T06:53:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 -- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f0bfc06af90 msgr2=0x7f0bfc06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:53:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f0bfc06af90 0x7f0bfc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 -- 172.21.15.70:0/1232582371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806db60 msgr2=0x7f0c18155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806db60 0x7f0c18155f30 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400e8c0 tx=0x7f0c0400ed80 comp rx=0 tx=0).stop 2023-10-08T06:53:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 -- 172.21.15.70:0/1232582371 shutdown_connections 2023-10-08T06:53:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f0bfc06af90 0x7f0bfc06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0c1815aa00 0x7f0c1815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c1806db60 0x7f0c18155f30 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 --2- 172.21.15.70:0/1232582371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0c1806c3a0 0x7f0c18159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 -- 172.21.15.70:0/1232582371 >> 172.21.15.70:0/1232582371 conn(0x7f0c1806a5d0 msgr2=0x7f0c180ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:23.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 -- 172.21.15.70:0/1232582371 shutdown_connections 2023-10-08T06:53:23.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.530+0000 7f0c1ed90640 1 -- 172.21.15.70:0/1232582371 wait complete. 2023-10-08T06:53:23.547 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T06:53:23.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.799+0000 7f70a74b4640 1 Processor -- start 2023-10-08T06:53:23.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.799+0000 7f70a74b4640 1 -- start start 2023-10-08T06:53:23.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.800+0000 7f70a74b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f70a00c9cf0 0x7f70a006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.800+0000 7f70a74b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70a015aa00 0x7f70a00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.800+0000 7f70a74b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a015b310 con 0x7f70a006c3a0 2023-10-08T06:53:23.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.800+0000 7f70a74b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a006e0a0 con 0x7f70a00c9cf0 2023-10-08T06:53:23.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.800+0000 7f70a74b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a006d290 con 0x7f70a015aa00 2023-10-08T06:53:23.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.801+0000 7f70a4a4f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f70a00c9cf0 0x7f70a006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:23.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.801+0000 7f70a4a4f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f70a00c9cf0 0x7f70a006db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48238/0 (socket says 172.21.15.70:48238) 2023-10-08T06:53:23.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.801+0000 7f70a4a4f640 1 -- 172.21.15.70:0/262321212 learned_addr learned my addr 172.21.15.70:0/262321212 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:23.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.801+0000 7f70a5a51640 1 --2- 172.21.15.70:0/262321212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70a015aa00 0x7f70a00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:23.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.801+0000 7f70a4a4f640 1 -- 172.21.15.70:0/262321212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70a015aa00 msgr2=0x7f70a00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:23.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.801+0000 7f70a4a4f640 1 --2- 172.21.15.70:0/262321212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70a015aa00 0x7f70a00b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.801+0000 7f70a4a4f640 1 -- 172.21.15.70:0/262321212 >> v1:172.21.15.70:6789/0 conn(0x7f70a006c3a0 legacy=0x7f70a006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:23.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.801+0000 7f70a4a4f640 1 -- 172.21.15.70:0/262321212 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a00c9860 con 0x7f70a00c9cf0 2023-10-08T06:53:23.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.802+0000 7f70a4a4f640 1 --2- 172.21.15.70:0/262321212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a00c9cf0 0x7f70a006db60 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f7090009870 tx=0x7f709002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153daa8832177c66 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:23.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.803+0000 7f7097fff640 1 -- 172.21.15.70:0/262321212 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090030840 con 0x7f70a00c9cf0 2023-10-08T06:53:23.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.803+0000 7f7097fff640 1 -- 172.21.15.70:0/262321212 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70900309a0 con 0x7f70a00c9cf0 2023-10-08T06:53:23.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.803+0000 7f7097fff640 1 -- 172.21.15.70:0/262321212 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090030d30 con 0x7f70a00c9cf0 2023-10-08T06:53:23.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.804+0000 7f70a74b4640 1 -- 172.21.15.70:0/262321212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a00c9cf0 msgr2=0x7f70a006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:23.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.804+0000 7f70a74b4640 1 --2- 172.21.15.70:0/262321212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a00c9cf0 0x7f70a006db60 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f7090009870 tx=0x7f709002b980 comp rx=0 tx=0).stop 2023-10-08T06:53:23.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.804+0000 7f70a74b4640 1 -- 172.21.15.70:0/262321212 shutdown_connections 2023-10-08T06:53:23.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.804+0000 7f70a74b4640 1 --2- 172.21.15.70:0/262321212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70a015aa00 0x7f70a00b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.804+0000 7f70a74b4640 1 --2- 172.21.15.70:0/262321212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a00c9cf0 0x7f70a006db60 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.804+0000 7f70a74b4640 1 -- 172.21.15.70:0/262321212 >> 172.21.15.70:0/262321212 conn(0x7f70a006a5d0 msgr2=0x7f70a00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:23.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.804+0000 7f70a74b4640 1 -- 172.21.15.70:0/262321212 shutdown_connections 2023-10-08T06:53:23.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.804+0000 7f70a74b4640 1 -- 172.21.15.70:0/262321212 wait complete. 2023-10-08T06:53:23.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.806+0000 7f70a74b4640 1 Processor -- start 2023-10-08T06:53:23.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.806+0000 7f70a74b4640 1 -- start start 2023-10-08T06:53:23.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a74b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a006c3a0 0x7f70a0153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a74b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70a00c9cf0 0x7f70a0153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a74b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70a015aa00 0x7f70a014ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a74b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a00c70f0 con 0x7f70a006c3a0 2023-10-08T06:53:23.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a74b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0165fb0 con 0x7f70a00c9cf0 2023-10-08T06:53:23.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a74b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a00c6570 con 0x7f70a015aa00 2023-10-08T06:53:23.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a5250640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a006c3a0 0x7f70a0153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:23.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a5250640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a006c3a0 0x7f70a0153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48242/0 (socket says 172.21.15.70:48242) 2023-10-08T06:53:23.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a5250640 1 -- 172.21.15.70:0/764071236 learned_addr learned my addr 172.21.15.70:0/764071236 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:23.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a5250640 1 -- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70a015aa00 msgr2=0x7f70a014ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:23.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a5250640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70a015aa00 0x7f70a014ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a5250640 1 -- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70a00c9cf0 msgr2=0x7f70a0153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:23.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a5250640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70a00c9cf0 0x7f70a0153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:23.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.807+0000 7f70a5250640 1 -- 172.21.15.70:0/764071236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70900094d0 con 0x7f70a006c3a0 2023-10-08T06:53:23.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.808+0000 7f70a5250640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a006c3a0 0x7f70a0153920 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f709c00c450 tx=0x7f709c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:23.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.808+0000 7f7095ffb640 1 -- 172.21.15.70:0/764071236 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c0052d0 con 0x7f70a006c3a0 2023-10-08T06:53:23.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.808+0000 7f7095ffb640 1 -- 172.21.15.70:0/764071236 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c005430 con 0x7f70a006c3a0 2023-10-08T06:53:23.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.808+0000 7f70a74b4640 1 -- 172.21.15.70:0/764071236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a00c6850 con 0x7f70a006c3a0 2023-10-08T06:53:23.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.808+0000 7f7095ffb640 1 -- 172.21.15.70:0/764071236 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c012870 con 0x7f70a006c3a0 2023-10-08T06:53:23.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.808+0000 7f70a74b4640 1 -- 172.21.15.70:0/764071236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a00c6e10 con 0x7f70a006c3a0 2023-10-08T06:53:23.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.809+0000 7f70a74b4640 1 -- 172.21.15.70:0/764071236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a0166120 con 0x7f70a006c3a0 2023-10-08T06:53:23.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.811+0000 7f7095ffb640 1 -- 172.21.15.70:0/764071236 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 85) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f709c0129d0 con 0x7f70a006c3a0 2023-10-08T06:53:23.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.811+0000 7f7095ffb640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f708406af90 0x7f708406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:23.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.812+0000 7f7095ffb640 1 -- 172.21.15.70:0/764071236 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f709c0894b0 con 0x7f70a006c3a0 2023-10-08T06:53:23.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.812+0000 7f70a4a4f640 1 -- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f708406af90 msgr2=0x7f708406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:23.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.812+0000 7f70a4a4f640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f708406af90 0x7f708406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:53:23.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:23.820+0000 7f7095ffb640 1 -- 172.21.15.70:0/764071236 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f709c054100 con 0x7f70a006c3a0 2023-10-08T06:53:23.889 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:23.888+0000 7f04e5612040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T06:53:23.927 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:23.926+0000 7fed3cc32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T06:53:24.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.009+0000 7f70a74b4640 1 -- 172.21.15.70:0/764071236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70a015b310 con 0x7f70a006c3a0 2023-10-08T06:53:24.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.010+0000 7f7095ffb640 1 -- 172.21.15.70:0/764071236 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v170) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70a015b310 con 0x7f70a006c3a0 2023-10-08T06:53:24.010 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T06:53:24.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 -- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f708406af90 msgr2=0x7f708406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:53:24.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f708406af90 0x7f708406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 -- 172.21.15.70:0/764071236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a006c3a0 msgr2=0x7f70a0153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:24.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a006c3a0 0x7f70a0153920 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f709c00c450 tx=0x7f709c00c910 comp rx=0 tx=0).stop 2023-10-08T06:53:24.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 -- 172.21.15.70:0/764071236 shutdown_connections 2023-10-08T06:53:24.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f708406af90 0x7f708406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70a015aa00 0x7f70a014ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70a00c9cf0 0x7f70a0153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 --2- 172.21.15.70:0/764071236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70a006c3a0 0x7f70a0153920 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 -- 172.21.15.70:0/764071236 >> 172.21.15.70:0/764071236 conn(0x7f70a006a5d0 msgr2=0x7f70a0150af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:24.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 -- 172.21.15.70:0/764071236 shutdown_connections 2023-10-08T06:53:24.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.012+0000 7f70a74b4640 1 -- 172.21.15.70:0/764071236 wait complete. 2023-10-08T06:53:24.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T06:53:24.046 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:24.045+0000 7f04e5612040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T06:53:24.092 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:24.092+0000 7fed3cc32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T06:53:24.152 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:24.151+0000 7f04e5612040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T06:53:24.201 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:24.200+0000 7f04e5612040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T06:53:24.205 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:24.204+0000 7fed3cc32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T06:53:24.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.244+0000 7f4c151a9640 1 Processor -- start 2023-10-08T06:53:24.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.244+0000 7f4c151a9640 1 -- start start 2023-10-08T06:53:24.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.246+0000 7f4c151a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c1015cba0 0x7f4c100b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:24.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.246+0000 7f4c151a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4c100b4d10 0x7f4c100d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:24.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.246+0000 7f4c151a9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c100b54a0 con 0x7f4c1015cf80 2023-10-08T06:53:24.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.246+0000 7f4c151a9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c100b7130 con 0x7f4c100b4d10 2023-10-08T06:53:24.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.246+0000 7f4c151a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c100b7410 con 0x7f4c1015cba0 2023-10-08T06:53:24.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.246+0000 7f4c0f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4c100b4d10 0x7f4c100d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:24.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.246+0000 7f4c0f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4c100b4d10 0x7f4c100d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48252/0 (socket says 172.21.15.70:48252) 2023-10-08T06:53:24.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.246+0000 7f4c0f59e640 1 -- 172.21.15.70:0/1508898471 learned_addr learned my addr 172.21.15.70:0/1508898471 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:24.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.246+0000 7f4c0ed9d640 1 --2- 172.21.15.70:0/1508898471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c1015cba0 0x7f4c100b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:24.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.247+0000 7f4c0f59e640 1 -- 172.21.15.70:0/1508898471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c1015cba0 msgr2=0x7f4c100b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:24.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.247+0000 7f4c0f59e640 1 --2- 172.21.15.70:0/1508898471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c1015cba0 0x7f4c100b84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.247+0000 7f4c0f59e640 1 -- 172.21.15.70:0/1508898471 >> v1:172.21.15.181:6789/0 conn(0x7f4c1015cf80 legacy=0x7f4c100b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:24.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.247+0000 7f4c0f59e640 1 -- 172.21.15.70:0/1508898471 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c100b76f0 con 0x7f4c100b4d10 2023-10-08T06:53:24.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.247+0000 7f4c0ed9d640 1 --2- 172.21.15.70:0/1508898471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c1015cba0 0x7f4c100b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T06:53:24.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.249+0000 7f4c0f59e640 1 --2- 172.21.15.70:0/1508898471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c100b4d10 0x7f4c100d0ee0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009e50 tx=0x7f4c0802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a95da74fe593f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:24.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.249+0000 7f4c0dd9b640 1 -- 172.21.15.70:0/1508898471 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0802caf0 con 0x7f4c100b4d10 2023-10-08T06:53:24.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.249+0000 7f4c0dd9b640 1 -- 172.21.15.70:0/1508898471 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0802cc50 con 0x7f4c100b4d10 2023-10-08T06:53:24.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.249+0000 7f4c0dd9b640 1 -- 172.21.15.70:0/1508898471 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c080343f0 con 0x7f4c100b4d10 2023-10-08T06:53:24.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.250+0000 7f4c151a9640 1 -- 172.21.15.70:0/1508898471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c100b4d10 msgr2=0x7f4c100d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:24.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.250+0000 7f4c151a9640 1 --2- 172.21.15.70:0/1508898471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c100b4d10 0x7f4c100d0ee0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009e50 tx=0x7f4c0802bf50 comp rx=0 tx=0).stop 2023-10-08T06:53:24.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.250+0000 7f4c151a9640 1 -- 172.21.15.70:0/1508898471 shutdown_connections 2023-10-08T06:53:24.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.250+0000 7f4c151a9640 1 --2- 172.21.15.70:0/1508898471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c100b4d10 0x7f4c100d0ee0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.250+0000 7f4c151a9640 1 --2- 172.21.15.70:0/1508898471 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c1015cba0 0x7f4c100b84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.250+0000 7f4c151a9640 1 -- 172.21.15.70:0/1508898471 >> 172.21.15.70:0/1508898471 conn(0x7f4c100718a0 msgr2=0x7f4c10071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:24.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.250+0000 7f4c151a9640 1 -- 172.21.15.70:0/1508898471 shutdown_connections 2023-10-08T06:53:24.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.250+0000 7f4c151a9640 1 -- 172.21.15.70:0/1508898471 wait complete. 2023-10-08T06:53:24.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.251+0000 7f4c151a9640 1 Processor -- start 2023-10-08T06:53:24.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.252+0000 7f4c151a9640 1 -- start start 2023-10-08T06:53:24.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c151a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c100b4d10 0x7f4c101541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:24.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c151a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1015cba0 0x7f4c100c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:24.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c151a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1015cf80 0x7f4c100c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:24.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c151a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c101546e0 con 0x7f4c1015cba0 2023-10-08T06:53:24.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c151a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c10155ec0 con 0x7f4c1015cf80 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c151a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c100c7220 con 0x7f4c100b4d10 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1015cba0 0x7f4c100c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1015cba0 0x7f4c100c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48256/0 (socket says 172.21.15.70:48256) 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0e59c640 1 -- 172.21.15.70:0/1634392884 learned_addr learned my addr 172.21.15.70:0/1634392884 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0ed9d640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c100b4d10 0x7f4c101541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0f59e640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1015cf80 0x7f4c100c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0e59c640 1 -- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c100b4d10 msgr2=0x7f4c101541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0e59c640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c100b4d10 0x7f4c101541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0e59c640 1 -- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1015cf80 msgr2=0x7f4c100c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0e59c640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1015cf80 0x7f4c100c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0e59c640 1 -- 172.21.15.70:0/1634392884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c08009b30 con 0x7f4c1015cba0 2023-10-08T06:53:24.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.253+0000 7f4c0f59e640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1015cf80 0x7f4c100c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:24.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.254+0000 7f4c0e59c640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1015cba0 0x7f4c100c66b0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c400 tx=0x7f4c0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:24.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.254+0000 7f4bf77fe640 1 -- 172.21.15.70:0/1634392884 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c040052d0 con 0x7f4c1015cba0 2023-10-08T06:53:24.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.254+0000 7f4bf77fe640 1 -- 172.21.15.70:0/1634392884 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c04005430 con 0x7f4c1015cba0 2023-10-08T06:53:24.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.254+0000 7f4bf77fe640 1 -- 172.21.15.70:0/1634392884 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04003830 con 0x7f4c1015cba0 2023-10-08T06:53:24.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.254+0000 7f4c151a9640 1 -- 172.21.15.70:0/1634392884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c100c7500 con 0x7f4c1015cba0 2023-10-08T06:53:24.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.254+0000 7f4c151a9640 1 -- 172.21.15.70:0/1634392884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c100c7ac0 con 0x7f4c1015cba0 2023-10-08T06:53:24.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.255+0000 7f4bf57fa640 1 -- 172.21.15.70:0/1634392884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be8005250 con 0x7f4c1015cba0 2023-10-08T06:53:24.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.257+0000 7f4bf77fe640 1 -- 172.21.15.70:0/1634392884 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 85) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f4c04003990 con 0x7f4c1015cba0 2023-10-08T06:53:24.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.258+0000 7f4bf77fe640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f4bfc06af90 0x7f4bfc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:24.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.258+0000 7f4bf77fe640 1 -- 172.21.15.70:0/1634392884 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f4c04089380 con 0x7f4c1015cba0 2023-10-08T06:53:24.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.258+0000 7f4c0ed9d640 1 -- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f4bfc06af90 msgr2=0x7f4bfc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:24.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.258+0000 7f4c0ed9d640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f4bfc06af90 0x7f4bfc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:53:24.260 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:24.260+0000 7fed3cc32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T06:53:24.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.264+0000 7f4bf77fe640 1 -- 172.21.15.70:0/1634392884 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c04053fc0 con 0x7f4c1015cba0 2023-10-08T06:53:24.309 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:24.308+0000 7f04e5612040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:53:24.375 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:24.374+0000 7fed3cc32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:53:24.415 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:24.414+0000 7f04e5612040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T06:53:24.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.459+0000 7f4c0ed9d640 1 -- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f4bfc06af90 msgr2=0x7f4bfc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:24.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.459+0000 7f4c0ed9d640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f4bfc06af90 0x7f4bfc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:53:24.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.466+0000 7f4bf57fa640 1 -- 172.21.15.70:0/1634392884 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4be80057d0 con 0x7f4c1015cba0 2023-10-08T06:53:24.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.467+0000 7f4bf77fe640 1 -- 172.21.15.70:0/1634392884 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v170) v1 ==== 80+0+478 (secure 0 0 0) 0x7f4c040171f0 con 0x7f4c1015cba0 2023-10-08T06:53:24.468 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3947544788 2023-10-09T05:57:41.519220+0000 2023-10-08T06:53:24.468 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3693343940 2023-10-09T05:57:41.519220+0000 2023-10-08T06:53:24.468 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/2068783299 2023-10-09T05:57:41.519220+0000 2023-10-08T06:53:24.468 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/2068783299 2023-10-09T05:57:41.519220+0000 2023-10-08T06:53:24.468 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3070110051 2023-10-09T05:57:41.519220+0000 2023-10-08T06:53:24.468 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3604402742 2023-10-09T05:57:41.519220+0000 2023-10-08T06:53:24.468 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1336064042 2023-10-09T05:57:41.519220+0000 2023-10-08T06:53:24.469 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3791109415 2023-10-09T05:57:41.519220+0000 2023-10-08T06:53:24.469 INFO:teuthology.orchestra.run.smithi070.stderr:listed 8 entries 2023-10-08T06:53:24.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 -- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f4bfc06af90 msgr2=0x7f4bfc06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:53:24.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f4bfc06af90 0x7f4bfc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 -- 172.21.15.70:0/1634392884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1015cba0 msgr2=0x7f4c100c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:24.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1015cba0 0x7f4c100c66b0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c400 tx=0x7f4c0400c8c0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 -- 172.21.15.70:0/1634392884 shutdown_connections 2023-10-08T06:53:24.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f4bfc06af90 0x7f4bfc06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4c1015cf80 0x7f4c100c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c1015cba0 0x7f4c100c66b0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 --2- 172.21.15.70:0/1634392884 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4c100b4d10 0x7f4c101541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:24.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 -- 172.21.15.70:0/1634392884 >> 172.21.15.70:0/1634392884 conn(0x7f4c100718a0 msgr2=0x7f4c100cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:24.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 -- 172.21.15.70:0/1634392884 shutdown_connections 2023-10-08T06:53:24.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:24.470+0000 7f4bf57fa640 1 -- 172.21.15.70:0/1634392884 wait complete. 2023-10-08T06:53:24.486 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:24.486+0000 7fed3cc32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T06:53:24.488 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T06:53:24.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:24.488 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T06:53:24.570 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:24.569+0000 7f04e5612040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:53:24.595 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T06:53:24.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:53:24.596 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T06:53:24.620 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:24.619+0000 7f04e5612040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T06:53:24.639 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T06:53:24.639 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:53:24.653 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:24.652+0000 7fed3cc32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:53:24.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T06:53:24.693 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:53:24.694 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:53:24.703 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:24.703+0000 7fed3cc32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T06:53:24.744 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:24.743+0000 7f04e5612040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T06:53:24.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T06:53:24.747 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T06:53:24.747 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:53:24.802 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T06:53:24.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:53:24.827 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:24.827+0000 7fed3cc32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T06:53:25.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.016+0000 7fad2f106640 1 Processor -- start 2023-10-08T06:53:25.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.016+0000 7fad2f106640 1 -- start start 2023-10-08T06:53:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.017+0000 7fad2f106640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad280adc70 0x7fad280b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.017+0000 7fad2f106640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fad2815af10 0x7fad280b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.017+0000 7fad2f106640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2815f7d0 con 0x7fad2815ab30 2023-10-08T06:53:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2f106640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2815efb0 con 0x7fad280adc70 2023-10-08T06:53:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2f106640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2815f210 con 0x7fad2815af10 2023-10-08T06:53:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2d903640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad280adc70 0x7fad280b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2d903640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad280adc70 0x7fad280b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48266/0 (socket says 172.21.15.70:48266) 2023-10-08T06:53:25.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2d903640 1 -- 172.21.15.70:0/2289042577 learned_addr learned my addr 172.21.15.70:0/2289042577 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:25.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2e905640 1 --2- 172.21.15.70:0/2289042577 >> v2:172.21.15.181:3300/0 conn(0x7fad2815af10 0x7fad280b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2d903640 1 -- 172.21.15.70:0/2289042577 >> v2:172.21.15.181:3300/0 conn(0x7fad2815af10 msgr2=0x7fad280b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:25.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2d903640 1 --2- 172.21.15.70:0/2289042577 >> v2:172.21.15.181:3300/0 conn(0x7fad2815af10 0x7fad280b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2d903640 1 -- 172.21.15.70:0/2289042577 >> v1:172.21.15.181:6789/0 conn(0x7fad2815ab30 legacy=0x7fad2815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:25.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2d903640 1 -- 172.21.15.70:0/2289042577 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2815f4f0 con 0x7fad280adc70 2023-10-08T06:53:25.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.018+0000 7fad2e905640 1 --2- 172.21.15.70:0/2289042577 >> v2:172.21.15.181:3300/0 conn(0x7fad2815af10 0x7fad280b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:25.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.019+0000 7fad2d903640 1 --2- 172.21.15.70:0/2289042577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad280adc70 0x7fad280b00c0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fad280c56c0 tx=0x7fad1802b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b2daf5f9c096e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:25.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.020+0000 7fad2d102640 1 -- 172.21.15.70:0/2289042577 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18031640 con 0x7fad280adc70 2023-10-08T06:53:25.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.020+0000 7fad2d102640 1 -- 172.21.15.70:0/2289042577 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad180317a0 con 0x7fad280adc70 2023-10-08T06:53:25.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.020+0000 7fad2d102640 1 -- 172.21.15.70:0/2289042577 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18031b30 con 0x7fad280adc70 2023-10-08T06:53:25.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.020+0000 7fad2f106640 1 -- 172.21.15.70:0/2289042577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad280adc70 msgr2=0x7fad280b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.021+0000 7fad2f106640 1 --2- 172.21.15.70:0/2289042577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad280adc70 0x7fad280b00c0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fad280c56c0 tx=0x7fad1802b8e0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.021+0000 7fad2f106640 1 -- 172.21.15.70:0/2289042577 shutdown_connections 2023-10-08T06:53:25.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.021+0000 7fad2f106640 1 --2- 172.21.15.70:0/2289042577 >> v2:172.21.15.181:3300/0 conn(0x7fad2815af10 0x7fad280b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.021+0000 7fad2f106640 1 --2- 172.21.15.70:0/2289042577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad280adc70 0x7fad280b00c0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.021+0000 7fad2f106640 1 -- 172.21.15.70:0/2289042577 >> 172.21.15.70:0/2289042577 conn(0x7fad280cb0c0 msgr2=0x7fad280cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:25.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.021+0000 7fad2f106640 1 -- 172.21.15.70:0/2289042577 shutdown_connections 2023-10-08T06:53:25.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.021+0000 7fad2f106640 1 -- 172.21.15.70:0/2289042577 wait complete. 2023-10-08T06:53:25.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.023+0000 7fad2f106640 1 Processor -- start 2023-10-08T06:53:25.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.024+0000 7fad2f106640 1 -- start start 2023-10-08T06:53:25.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.025+0000 7fad2f106640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad280adc70 0x7fad2814ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.025+0000 7fad2f106640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad2815ab30 0x7fad280af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.025+0000 7fad2f106640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad2815af10 0x7fad280afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.025+0000 7fad2f106640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2814f440 con 0x7fad2815ab30 2023-10-08T06:53:25.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.025+0000 7fad2f106640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad281f0480 con 0x7fad280adc70 2023-10-08T06:53:25.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.025+0000 7fad2f106640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad280c6040 con 0x7fad2815af10 2023-10-08T06:53:25.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.025+0000 7fad2d903640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad2815ab30 0x7fad280af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.025+0000 7fad2d903640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad2815ab30 0x7fad280af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48280/0 (socket says 172.21.15.70:48280) 2023-10-08T06:53:25.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.025+0000 7fad2d903640 1 -- 172.21.15.70:0/3008675086 learned_addr learned my addr 172.21.15.70:0/3008675086 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:25.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad2e905640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad2815af10 0x7fad280afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad2e104640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad280adc70 0x7fad2814ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad2d903640 1 -- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad2815af10 msgr2=0x7fad280afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad2d903640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad2815af10 0x7fad280afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad2d903640 1 -- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad280adc70 msgr2=0x7fad2814ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad2d903640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad280adc70 0x7fad2814ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad2d903640 1 -- 172.21.15.70:0/3008675086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad180094d0 con 0x7fad2815ab30 2023-10-08T06:53:25.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad2d903640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad2815ab30 0x7fad280af8f0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fad1802b920 tx=0x7fad180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:25.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad1effd640 1 -- 172.21.15.70:0/3008675086 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18031e60 con 0x7fad2815ab30 2023-10-08T06:53:25.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad1effd640 1 -- 172.21.15.70:0/3008675086 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad180443f0 con 0x7fad2815ab30 2023-10-08T06:53:25.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.026+0000 7fad1effd640 1 -- 172.21.15.70:0/3008675086 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad180447a0 con 0x7fad2815ab30 2023-10-08T06:53:25.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.027+0000 7fad2f106640 1 -- 172.21.15.70:0/3008675086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2815d700 con 0x7fad2815ab30 2023-10-08T06:53:25.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.027+0000 7fad2f106640 1 -- 172.21.15.70:0/3008675086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad280c59f0 con 0x7fad2815ab30 2023-10-08T06:53:25.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.028+0000 7fad2f106640 1 -- 172.21.15.70:0/3008675086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad28156470 con 0x7fad2815ab30 2023-10-08T06:53:25.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.029+0000 7fad1effd640 1 -- 172.21.15.70:0/3008675086 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 85) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fad18044900 con 0x7fad2815ab30 2023-10-08T06:53:25.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.031+0000 7fad1effd640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fad0c06af90 0x7fad0c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.031+0000 7fad1effd640 1 -- 172.21.15.70:0/3008675086 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fad180ac780 con 0x7fad2815ab30 2023-10-08T06:53:25.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.031+0000 7fad2e104640 1 -- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fad0c06af90 msgr2=0x7fad0c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:25.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.031+0000 7fad2e104640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fad0c06af90 0x7fad0c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:53:25.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.034+0000 7fad1effd640 1 -- 172.21.15.70:0/3008675086 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad18077340 con 0x7fad2815ab30 2023-10-08T06:53:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.232+0000 7fad2e104640 1 -- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fad0c06af90 msgr2=0x7fad0c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.232+0000 7fad2e104640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fad0c06af90 0x7fad0c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:53:25.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.262+0000 7fad2f106640 1 -- 172.21.15.70:0/3008675086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad28156470 con 0x7fad2815ab30 2023-10-08T06:53:25.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.262+0000 7fad1effd640 1 -- 172.21.15.70:0/3008675086 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fad28156470 con 0x7fad2815ab30 2023-10-08T06:53:25.263 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:25.263 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2889908260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2889908260},{"type":"v1","addr":"172.21.15.181:6837","nonce":2889908260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2651702867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2651702867},{"type":"v1","addr":"172.21.15.181:6835","nonce":2651702867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":9048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1574948577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1574948577},{"type":"v1","addr":"172.21.15.70:6833","nonce":1574948577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:57:26.074871+0000","modified":"2023-10-08T05:57:29.076265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9051":{"gid":9051,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6835/3537072579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3537072579},{"type":"v1","addr":"172.21.15.70:6835","nonce":3537072579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-08T06:53:25.263 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 76 2023-10-08T06:53:25.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 -- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fad0c06af90 msgr2=0x7fad0c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:53:25.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fad0c06af90 0x7fad0c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 -- 172.21.15.70:0/3008675086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad2815ab30 msgr2=0x7fad280af8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad2815ab30 0x7fad280af8f0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fad1802b920 tx=0x7fad180046c0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 -- 172.21.15.70:0/3008675086 shutdown_connections 2023-10-08T06:53:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fad0c06af90 0x7fad0c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad2815af10 0x7fad280afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad2815ab30 0x7fad280af8f0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 --2- 172.21.15.70:0/3008675086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad280adc70 0x7fad2814ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 -- 172.21.15.70:0/3008675086 >> 172.21.15.70:0/3008675086 conn(0x7fad280cb0c0 msgr2=0x7fad2815e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 -- 172.21.15.70:0/3008675086 shutdown_connections 2023-10-08T06:53:25.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.265+0000 7fad2f106640 1 -- 172.21.15.70:0/3008675086 wait complete. 2023-10-08T06:53:25.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:53:25.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.518+0000 7fd500a32640 1 Processor -- start 2023-10-08T06:53:25.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.518+0000 7fd500a32640 1 -- start start 2023-10-08T06:53:25.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.520+0000 7fd500a32640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd4fc0d2010 0x7fd4fc0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.520+0000 7fd500a32640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc0d0510 con 0x7fd4fc15cd70 2023-10-08T06:53:25.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.520+0000 7fd500a32640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc0d2910 con 0x7fd4fc0d00c0 2023-10-08T06:53:25.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.520+0000 7fd500a32640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc15d150 con 0x7fd4fc0d2010 2023-10-08T06:53:25.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.520+0000 7fd4fa59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd4fc15cd70 0x7fd4fc0b8530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37246/0 (socket says 172.21.15.70:37246) 2023-10-08T06:53:25.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.520+0000 7fd4f3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd4fc0d2010 0x7fd4fc0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.520+0000 7fd4fa59c640 1 -- 172.21.15.70:0/278952197 learned_addr learned my addr 172.21.15.70:0/278952197 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:25.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.520+0000 7fd4f9d9b640 1 -- 172.21.15.70:0/278952197 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569659434 0 0) 0x7fd4ec001110 con 0x7fd4fc15cd70 2023-10-08T06:53:25.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.521+0000 7fd4f9d9b640 1 -- 172.21.15.70:0/278952197 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e8003520 con 0x7fd4fc15cd70 2023-10-08T06:53:25.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.521+0000 7fd4f3fff640 1 -- 172.21.15.70:0/278952197 >> v1:172.21.15.181:6789/0 conn(0x7fd4fc0d00c0 legacy=0x7fd4fc0b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.521+0000 7fd4f3fff640 1 -- 172.21.15.70:0/278952197 >> v1:172.21.15.70:6789/0 conn(0x7fd4fc15cd70 legacy=0x7fd4fc0b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.521+0000 7fd4f3fff640 1 -- 172.21.15.70:0/278952197 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc0d0c90 con 0x7fd4fc0d2010 2023-10-08T06:53:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.522+0000 7fd4f3fff640 1 --2- 172.21.15.70:0/278952197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc0d2010 0x7fd4fc0cfb80 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40097a0 tx=0x7fd4e402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fa1feb2ea637f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.523+0000 7fd4f9d9b640 1 -- 172.21.15.70:0/278952197 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e402e070 con 0x7fd4fc0d2010 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.523+0000 7fd4f9d9b640 1 -- 172.21.15.70:0/278952197 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e40315a0 con 0x7fd4fc0d2010 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.523+0000 7fd4f9d9b640 1 -- 172.21.15.70:0/278952197 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4031930 con 0x7fd4fc0d2010 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.524+0000 7fd500a32640 1 -- 172.21.15.70:0/278952197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc0d2010 msgr2=0x7fd4fc0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.524+0000 7fd500a32640 1 --2- 172.21.15.70:0/278952197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc0d2010 0x7fd4fc0cfb80 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fd4e40097a0 tx=0x7fd4e402b840 comp rx=0 tx=0).stop 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.524+0000 7fd500a32640 1 -- 172.21.15.70:0/278952197 shutdown_connections 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.524+0000 7fd500a32640 1 --2- 172.21.15.70:0/278952197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc0d2010 0x7fd4fc0cfb80 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.524+0000 7fd500a32640 1 -- 172.21.15.70:0/278952197 >> 172.21.15.70:0/278952197 conn(0x7fd4fc0718a0 msgr2=0x7fd4fc071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.524+0000 7fd500a32640 1 -- 172.21.15.70:0/278952197 shutdown_connections 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.524+0000 7fd500a32640 1 -- 172.21.15.70:0/278952197 wait complete. 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.525+0000 7fd500a32640 1 Processor -- start 2023-10-08T06:53:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.526+0000 7fd500a32640 1 -- start start 2023-10-08T06:53:25.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.526+0000 7fd500a32640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc0d00c0 0x7fd4fc15fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.526+0000 7fd500a32640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0d2010 0x7fd4fc15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.526+0000 7fd500a32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc15cd70 0x7fd4fc1f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.526+0000 7fd500a32640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc1602b0 con 0x7fd4fc15cd70 2023-10-08T06:53:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.526+0000 7fd500a32640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0cb2f0 con 0x7fd4fc0d00c0 2023-10-08T06:53:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.526+0000 7fd500a32640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0cb5d0 con 0x7fd4fc0d2010 2023-10-08T06:53:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.527+0000 7fd4fad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc15cd70 0x7fd4fc1f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.527+0000 7fd4fad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc15cd70 0x7fd4fc1f1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48300/0 (socket says 172.21.15.70:48300) 2023-10-08T06:53:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.527+0000 7fd4fad9d640 1 -- 172.21.15.70:0/863202803 learned_addr learned my addr 172.21.15.70:0/863202803 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.527+0000 7fd4f3fff640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0d2010 0x7fd4fc15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.527+0000 7fd4fad9d640 1 -- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0d2010 msgr2=0x7fd4fc15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.527+0000 7fd4fa59c640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc0d00c0 0x7fd4fc15fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.527+0000 7fd4fad9d640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0d2010 0x7fd4fc15d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.527+0000 7fd4fad9d640 1 -- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc0d00c0 msgr2=0x7fd4fc15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.527+0000 7fd4fad9d640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc0d00c0 0x7fd4fc15fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.527+0000 7fd4fad9d640 1 -- 172.21.15.70:0/863202803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e4009400 con 0x7fd4fc15cd70 2023-10-08T06:53:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.528+0000 7fd4fad9d640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc15cd70 0x7fd4fc1f1b00 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c7a0 tx=0x7fd4f400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.528+0000 7fd4f2ffd640 1 -- 172.21.15.70:0/863202803 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f400aa50 con 0x7fd4fc15cd70 2023-10-08T06:53:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.528+0000 7fd4f2ffd640 1 -- 172.21.15.70:0/863202803 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f400abb0 con 0x7fd4fc15cd70 2023-10-08T06:53:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.528+0000 7fd500a32640 1 -- 172.21.15.70:0/863202803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc0cb8b0 con 0x7fd4fc15cd70 2023-10-08T06:53:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.528+0000 7fd4f2ffd640 1 -- 172.21.15.70:0/863202803 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4012420 con 0x7fd4fc15cd70 2023-10-08T06:53:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.528+0000 7fd500a32640 1 -- 172.21.15.70:0/863202803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc0cbe70 con 0x7fd4fc15cd70 2023-10-08T06:53:25.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.529+0000 7fd500a32640 1 -- 172.21.15.70:0/863202803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc0cc060 con 0x7fd4fc15cd70 2023-10-08T06:53:25.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.530+0000 7fd4f2ffd640 1 -- 172.21.15.70:0/863202803 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 85) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fd4f40050d0 con 0x7fd4fc15cd70 2023-10-08T06:53:25.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.531+0000 7fd4f2ffd640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fd4dc06af90 0x7fd4dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.532+0000 7fd4f2ffd640 1 -- 172.21.15.70:0/863202803 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6533+0+0 (secure 0 0 0) 0x7fd4f4089750 con 0x7fd4fc15cd70 2023-10-08T06:53:25.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.532+0000 7fd4fa59c640 1 -- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fd4dc06af90 msgr2=0x7fd4dc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:25.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.532+0000 7fd4fa59c640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fd4dc06af90 0x7fd4dc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:53:25.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.535+0000 7fd4f2ffd640 1 -- 172.21.15.70:0/863202803 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4f4054310 con 0x7fd4fc15cd70 2023-10-08T06:53:25.562 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:25.561+0000 7f04e5612040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T06:53:25.671 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:25.670+0000 7fed3cc32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T06:53:25.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.729+0000 7fd500a32640 1 -- 172.21.15.70:0/863202803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4fc067490 con 0x7fd4fc15cd70 2023-10-08T06:53:25.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.731+0000 7fd4f2ffd640 1 -- 172.21.15.70:0/863202803 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+17511 (secure 0 0 0) 0x7fd4f405ce60 con 0x7fd4fc15cd70 2023-10-08T06:53:25.732 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:25.732 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":170,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:53:21.777623+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:57:23.815675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-08T05:57:25.045292+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":170,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3947544788":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3693343940":"2023-10-09T05:57:41.519220+0000","172.21.15.181:6833/2068783299":"2023-10-09T05:57:41.519220+0000","172.21.15.181:6832/2068783299":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3070110051":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3604402742":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/1336064042":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3791109415":"2023-10-09T05:57:41.519220+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":42,"length":15},{"begin":98,"length":15}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:53:25.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.733+0000 7fd4fa59c640 1 -- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fd4dc06af90 msgr2=0x7fd4dc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:25.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.733+0000 7fd4fa59c640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fd4dc06af90 0x7fd4dc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:53:25.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.733+0000 7fd500a32640 1 -- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fd4dc06af90 msgr2=0x7fd4dc06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:53:25.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.733+0000 7fd500a32640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fd4dc06af90 0x7fd4dc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.733+0000 7fd500a32640 1 -- 172.21.15.70:0/863202803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc15cd70 msgr2=0x7fd4fc1f1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.733+0000 7fd500a32640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc15cd70 0x7fd4fc1f1b00 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c7a0 tx=0x7fd4f400cc60 comp rx=0 tx=0).stop 2023-10-08T06:53:25.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.734+0000 7fd500a32640 1 -- 172.21.15.70:0/863202803 shutdown_connections 2023-10-08T06:53:25.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.734+0000 7fd500a32640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7fd4dc06af90 0x7fd4dc06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.734+0000 7fd500a32640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4fc15cd70 0x7fd4fc1f1b00 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.734+0000 7fd500a32640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4fc0d2010 0x7fd4fc15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.734+0000 7fd500a32640 1 --2- 172.21.15.70:0/863202803 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4fc0d00c0 0x7fd4fc15fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.734+0000 7fd500a32640 1 -- 172.21.15.70:0/863202803 >> 172.21.15.70:0/863202803 conn(0x7fd4fc0718a0 msgr2=0x7fd4fc0cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:25.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.734+0000 7fd500a32640 1 -- 172.21.15.70:0/863202803 shutdown_connections 2023-10-08T06:53:25.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.734+0000 7fd500a32640 1 -- 172.21.15.70:0/863202803 wait complete. 2023-10-08T06:53:25.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T05:57:23.815675+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T05:57:25.045292+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 170, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T06:53:25.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T06:53:25.862 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:25.860+0000 7f04e5612040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T06:53:25.919 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:25.918+0000 7f04e5612040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T06:53:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.962+0000 7f471d787640 1 Processor -- start 2023-10-08T06:53:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.962+0000 7f471d787640 1 -- start start 2023-10-08T06:53:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.963+0000 7f471d787640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f47180d3430 0x7f47180cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.963+0000 7f471d787640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f47180cf820 0x7f471806bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f471d787640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47180d1650 con 0x7f471815cd00 2023-10-08T06:53:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f471d787640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47180cfcb0 con 0x7f47180cf820 2023-10-08T06:53:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f471d787640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47180d3810 con 0x7f47180d3430 2023-10-08T06:53:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f471cf86640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f47180cf820 0x7f471806bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f471cf86640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f47180cf820 0x7f471806bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48314/0 (socket says 172.21.15.70:48314) 2023-10-08T06:53:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f471cf86640 1 -- 172.21.15.70:0/3749992124 learned_addr learned my addr 172.21.15.70:0/3749992124 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f470ffff640 1 --2- 172.21.15.70:0/3749992124 >> v2:172.21.15.181:3300/0 conn(0x7f47180d3430 0x7f47180cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f471cf86640 1 -- 172.21.15.70:0/3749992124 >> v2:172.21.15.181:3300/0 conn(0x7f47180d3430 msgr2=0x7f47180cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f471cf86640 1 --2- 172.21.15.70:0/3749992124 >> v2:172.21.15.181:3300/0 conn(0x7f47180d3430 0x7f47180cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f471cf86640 1 -- 172.21.15.70:0/3749992124 >> v1:172.21.15.181:6789/0 conn(0x7f471815cd00 legacy=0x7f47180d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.964+0000 7f471cf86640 1 -- 172.21.15.70:0/3749992124 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471806b030 con 0x7f47180cf820 2023-10-08T06:53:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.966+0000 7f471cf86640 1 --2- 172.21.15.70:0/3749992124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47180cf820 0x7f471806bb10 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f47180ccef0 tx=0x7f471002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c419e3bbf1eb71 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.966+0000 7f47177fe640 1 -- 172.21.15.70:0/3749992124 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471002cb20 con 0x7f47180cf820 2023-10-08T06:53:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.966+0000 7f47177fe640 1 -- 172.21.15.70:0/3749992124 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471002cc80 con 0x7f47180cf820 2023-10-08T06:53:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.966+0000 7f47177fe640 1 -- 172.21.15.70:0/3749992124 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100343f0 con 0x7f47180cf820 2023-10-08T06:53:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.967+0000 7f471d787640 1 -- 172.21.15.70:0/3749992124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47180cf820 msgr2=0x7f471806bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.967+0000 7f471d787640 1 --2- 172.21.15.70:0/3749992124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47180cf820 0x7f471806bb10 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f47180ccef0 tx=0x7f471002bf80 comp rx=0 tx=0).stop 2023-10-08T06:53:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.967+0000 7f471d787640 1 -- 172.21.15.70:0/3749992124 shutdown_connections 2023-10-08T06:53:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.967+0000 7f471d787640 1 --2- 172.21.15.70:0/3749992124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47180cf820 0x7f471806bb10 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.967+0000 7f471d787640 1 --2- 172.21.15.70:0/3749992124 >> v2:172.21.15.181:3300/0 conn(0x7f47180d3430 0x7f47180cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.967+0000 7f471d787640 1 -- 172.21.15.70:0/3749992124 >> 172.21.15.70:0/3749992124 conn(0x7f471809bc20 msgr2=0x7f47180d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.967+0000 7f471d787640 1 -- 172.21.15.70:0/3749992124 shutdown_connections 2023-10-08T06:53:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.967+0000 7f471d787640 1 -- 172.21.15.70:0/3749992124 wait complete. 2023-10-08T06:53:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.969+0000 7f471d787640 1 Processor -- start 2023-10-08T06:53:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.969+0000 7f471d787640 1 -- start start 2023-10-08T06:53:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.970+0000 7f471d787640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47180cf820 0x7f4718150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.970+0000 7f471d787640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47180d3430 0x7f47181503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.970+0000 7f471d787640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f471815cd00 0x7f471815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.970+0000 7f471d787640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471815ddd0 con 0x7f471815cd00 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.970+0000 7f471d787640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180b75d0 con 0x7f47180cf820 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.970+0000 7f471d787640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471815e020 con 0x7f47180d3430 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.970+0000 7f471cf86640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f471815cd00 0x7f471815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.971+0000 7f471cf86640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f471815cd00 0x7f471815f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48316/0 (socket says 172.21.15.70:48316) 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.971+0000 7f471cf86640 1 -- 172.21.15.70:0/3329082974 learned_addr learned my addr 172.21.15.70:0/3329082974 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.971+0000 7f4717fff640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47180cf820 0x7f4718150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.971+0000 7f470ffff640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47180d3430 0x7f47181503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.971+0000 7f471cf86640 1 -- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47180d3430 msgr2=0x7f47181503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.971+0000 7f471cf86640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47180d3430 0x7f47181503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.971+0000 7f471cf86640 1 -- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47180cf820 msgr2=0x7f4718150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:25.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.971+0000 7f471cf86640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47180cf820 0x7f4718150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:25.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.971+0000 7f471cf86640 1 -- 172.21.15.70:0/3329082974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4710009b30 con 0x7f471815cd00 2023-10-08T06:53:25.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.971+0000 7f471cf86640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f471815cd00 0x7f471815f0c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f4710009f40 tx=0x7f4710004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.972+0000 7f47157fa640 1 -- 172.21.15.70:0/3329082974 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710007e70 con 0x7f471815cd00 2023-10-08T06:53:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.972+0000 7f47157fa640 1 -- 172.21.15.70:0/3329082974 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4710009510 con 0x7f471815cd00 2023-10-08T06:53:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.972+0000 7f47157fa640 1 -- 172.21.15.70:0/3329082974 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471003e690 con 0x7f471815cd00 2023-10-08T06:53:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.972+0000 7f471d787640 1 -- 172.21.15.70:0/3329082974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471815e300 con 0x7f471815cd00 2023-10-08T06:53:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.972+0000 7f471d787640 1 -- 172.21.15.70:0/3329082974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471815e8c0 con 0x7f471815cd00 2023-10-08T06:53:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.973+0000 7f471d787640 1 -- 172.21.15.70:0/3329082974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47180d1f20 con 0x7f471815cd00 2023-10-08T06:53:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.974+0000 7f47157fa640 1 -- 172.21.15.70:0/3329082974 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 85) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f471003e7f0 con 0x7f471815cd00 2023-10-08T06:53:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.975+0000 7f47157fa640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f46f806af90 0x7f46f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.975+0000 7f47157fa640 1 -- 172.21.15.70:0/3329082974 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6533+0+0 (secure 0 0 0) 0x7f47100ac8c0 con 0x7f471815cd00 2023-10-08T06:53:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.975+0000 7f4717fff640 1 -- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f46f806af90 msgr2=0x7f46f806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/236882560 2023-10-08T06:53:25.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.975+0000 7f4717fff640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f46f806af90 0x7f46f806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:53:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:25.979+0000 7f47157fa640 1 -- 172.21.15.70:0/3329082974 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4710077500 con 0x7f471815cd00 2023-10-08T06:53:25.986 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:25.986+0000 7fed3cc32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T06:53:26.034 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.033+0000 7f04e5612040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T06:53:26.050 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:26.047+0000 7fed3cc32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T06:53:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.052+0000 7f47157fa640 1 -- 172.21.15.70:0/3329082974 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 86) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f47100778d0 con 0x7f471815cd00 2023-10-08T06:53:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.052+0000 7f47157fa640 1 -- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f46f806af90 msgr2=0x7f46f806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:53:26.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.052+0000 7f47157fa640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f46f806af90 0x7f46f806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.123+0000 7f046a4d4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:53:26.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.123+0000 7f046a4d4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:53:26.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.123+0000 7f046a4d4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:53:26.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.123+0000 7f046a4d4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:53:26.124 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.123+0000 7f046a4d4640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:53:26.125 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.123+0000 7f04674ce640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:53:26.125 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.123+0000 7f04674ce640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:53:26.126 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.123+0000 7f04674ce640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:53:26.126 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.123+0000 7f04674ce640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:53:26.126 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:53:26.123+0000 7f04674ce640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:53:26.171 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:53:26.170+0000 7fed3cc32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T06:53:26.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.173+0000 7f471d787640 1 -- 172.21.15.70:0/3329082974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4718074150 con 0x7f471815cd00 2023-10-08T06:53:26.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.174+0000 7f47157fa640 1 -- 172.21.15.70:0/3329082974 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7f471007eda0 con 0x7f471815cd00 2023-10-08T06:53:26.175 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T06:53:26.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.176+0000 7f471d787640 1 -- 172.21.15.70:0/3329082974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f471815cd00 msgr2=0x7f471815f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:26.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.176+0000 7f471d787640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f471815cd00 0x7f471815f0c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f4710009f40 tx=0x7f4710004a40 comp rx=0 tx=0).stop 2023-10-08T06:53:26.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.176+0000 7f471d787640 1 -- 172.21.15.70:0/3329082974 shutdown_connections 2023-10-08T06:53:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.176+0000 7f471d787640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:6832/236882560,v1:172.21.15.181:6833/236882560] conn(0x7f46f806af90 0x7f46f806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.176+0000 7f471d787640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f471815cd00 0x7f471815f0c0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.177+0000 7f471d787640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f47180d3430 0x7f47181503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.177+0000 7f471d787640 1 --2- 172.21.15.70:0/3329082974 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f47180cf820 0x7f4718150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.177+0000 7f471d787640 1 -- 172.21.15.70:0/3329082974 >> 172.21.15.70:0/3329082974 conn(0x7f471809bc20 msgr2=0x7f47180b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.177+0000 7f471d787640 1 -- 172.21.15.70:0/3329082974 shutdown_connections 2023-10-08T06:53:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.177+0000 7f471d787640 1 -- 172.21.15.70:0/3329082974 wait complete. 2023-10-08T06:53:26.194 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T06:53:26.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.401+0000 7fbbfd7ff640 1 Processor -- start 2023-10-08T06:53:26.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.401+0000 7fbbfd7ff640 1 -- start start 2023-10-08T06:53:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.402+0000 7fbbfd7ff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbbf815a820 0x7fbbf80aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.402+0000 7fbbfd7ff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbbf80ca680 0x7fbbf80b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.402+0000 7fbbfd7ff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf80cab60 con 0x7fbbf815ac00 2023-10-08T06:53:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.402+0000 7fbbfd7ff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80cb960 con 0x7fbbf80ca680 2023-10-08T06:53:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.402+0000 7fbbfd7ff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80cbdf0 con 0x7fbbf815a820 2023-10-08T06:53:26.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.403+0000 7fbbeffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbbf80ca680 0x7fbbf80b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:26.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.403+0000 7fbbeffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbbf80ca680 0x7fbbf80b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48318/0 (socket says 172.21.15.70:48318) 2023-10-08T06:53:26.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.403+0000 7fbbeffff640 1 -- 172.21.15.70:0/65041415 learned_addr learned my addr 172.21.15.70:0/65041415 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:26.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.403+0000 7fbbf7fff640 1 --2- 172.21.15.70:0/65041415 >> v2:172.21.15.181:3300/0 conn(0x7fbbf815a820 0x7fbbf80aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:26.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.403+0000 7fbbeffff640 1 -- 172.21.15.70:0/65041415 >> v2:172.21.15.181:3300/0 conn(0x7fbbf815a820 msgr2=0x7fbbf80aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:26.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.403+0000 7fbbeffff640 1 --2- 172.21.15.70:0/65041415 >> v2:172.21.15.181:3300/0 conn(0x7fbbf815a820 0x7fbbf80aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.403+0000 7fbbeffff640 1 -- 172.21.15.70:0/65041415 >> v1:172.21.15.70:6789/0 conn(0x7fbbf815ac00 legacy=0x7fbbf80c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:26.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.403+0000 7fbbeffff640 1 -- 172.21.15.70:0/65041415 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf80cb030 con 0x7fbbf80ca680 2023-10-08T06:53:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.404+0000 7fbbeffff640 1 --2- 172.21.15.70:0/65041415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf80ca680 0x7fbbf80b04b0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fbbf806da70 tx=0x7fbbe002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aaced0d334acb6b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbf77fe640 1 -- 172.21.15.70:0/65041415 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0031870 con 0x7fbbf80ca680 2023-10-08T06:53:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbf77fe640 1 -- 172.21.15.70:0/65041415 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe00319d0 con 0x7fbbf80ca680 2023-10-08T06:53:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbf77fe640 1 -- 172.21.15.70:0/65041415 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0031d60 con 0x7fbbf80ca680 2023-10-08T06:53:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/65041415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf80ca680 msgr2=0x7fbbf80b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbfd7ff640 1 --2- 172.21.15.70:0/65041415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf80ca680 0x7fbbf80b04b0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fbbf806da70 tx=0x7fbbe002b910 comp rx=0 tx=0).stop 2023-10-08T06:53:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/65041415 shutdown_connections 2023-10-08T06:53:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbfd7ff640 1 --2- 172.21.15.70:0/65041415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf80ca680 0x7fbbf80b04b0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbfd7ff640 1 --2- 172.21.15.70:0/65041415 >> v2:172.21.15.181:3300/0 conn(0x7fbbf815a820 0x7fbbf80aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/65041415 >> 172.21.15.70:0/65041415 conn(0x7fbbf806c050 msgr2=0x7fbbf806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:26.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/65041415 shutdown_connections 2023-10-08T06:53:26.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.405+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/65041415 wait complete. 2023-10-08T06:53:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.407+0000 7fbbfd7ff640 1 Processor -- start 2023-10-08T06:53:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.407+0000 7fbbfd7ff640 1 -- start start 2023-10-08T06:53:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.408+0000 7fbbfd7ff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbf80ca680 0x7fbbf8157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.408+0000 7fbbfd7ff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbf815a820 0x7fbbf80afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.408+0000 7fbbfd7ff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf815ac00 0x7fbbf8159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.408+0000 7fbbfd7ff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80c6930 con 0x7fbbf815ac00 2023-10-08T06:53:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.408+0000 7fbbfd7ff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80c5cb0 con 0x7fbbf80ca680 2023-10-08T06:53:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.408+0000 7fbbfd7ff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80c5f90 con 0x7fbbf815a820 2023-10-08T06:53:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.408+0000 7fbbfcffe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf815ac00 0x7fbbf8159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.408+0000 7fbbfcffe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf815ac00 0x7fbbf8159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48334/0 (socket says 172.21.15.70:48334) 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.408+0000 7fbbfcffe640 1 -- 172.21.15.70:0/2841318953 learned_addr learned my addr 172.21.15.70:0/2841318953 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbf7fff640 1 --2- 172.21.15.70:0/2841318953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbf80ca680 0x7fbbf8157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbeffff640 1 --2- 172.21.15.70:0/2841318953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbf815a820 0x7fbbf80afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbfcffe640 1 -- 172.21.15.70:0/2841318953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbf815a820 msgr2=0x7fbbf80afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbfcffe640 1 --2- 172.21.15.70:0/2841318953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbf815a820 0x7fbbf80afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbfcffe640 1 -- 172.21.15.70:0/2841318953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbf80ca680 msgr2=0x7fbbf8157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbfcffe640 1 --2- 172.21.15.70:0/2841318953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbf80ca680 0x7fbbf8157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbfcffe640 1 -- 172.21.15.70:0/2841318953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00094d0 con 0x7fbbf815ac00 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbfcffe640 1 --2- 172.21.15.70:0/2841318953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf815ac00 0x7fbbf8159030 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c680 tx=0x7fbbf000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbf57fa640 1 -- 172.21.15.70:0/2841318953 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0003b70 con 0x7fbbf815ac00 2023-10-08T06:53:26.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbf57fa640 1 -- 172.21.15.70:0/2841318953 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf0003cd0 con 0x7fbbf815ac00 2023-10-08T06:53:26.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbf57fa640 1 -- 172.21.15.70:0/2841318953 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0005310 con 0x7fbbf815ac00 2023-10-08T06:53:26.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.409+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/2841318953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf80c6270 con 0x7fbbf815ac00 2023-10-08T06:53:26.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.410+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/2841318953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf8157ff0 con 0x7fbbf815ac00 2023-10-08T06:53:26.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.410+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/2841318953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf80c6a70 con 0x7fbbf815ac00 2023-10-08T06:53:26.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.411+0000 7fbbf57fa640 1 -- 172.21.15.70:0/2841318953 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 86) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fbbf0005470 con 0x7fbbf815ac00 2023-10-08T06:53:26.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.412+0000 7fbbf57fa640 1 -- 172.21.15.70:0/2841318953 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fbbf00893b0 con 0x7fbbf815ac00 2023-10-08T06:53:26.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.417+0000 7fbbf57fa640 1 -- 172.21.15.70:0/2841318953 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbf0054350 con 0x7fbbf815ac00 2023-10-08T06:53:26.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.606+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/2841318953 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbbf80665e0 con 0x7fbbf815ac00 2023-10-08T06:53:26.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.607+0000 7fbbf57fa640 1 -- 172.21.15.70:0/2841318953 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v171) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbbf0057970 con 0x7fbbf815ac00 2023-10-08T06:53:26.610 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T06:53:26.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.610+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/2841318953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf815ac00 msgr2=0x7fbbf8159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:26.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.610+0000 7fbbfd7ff640 1 --2- 172.21.15.70:0/2841318953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf815ac00 0x7fbbf8159030 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fbbf000c680 tx=0x7fbbf000cb40 comp rx=0 tx=0).stop 2023-10-08T06:53:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.610+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/2841318953 shutdown_connections 2023-10-08T06:53:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.610+0000 7fbbfd7ff640 1 --2- 172.21.15.70:0/2841318953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf815ac00 0x7fbbf8159030 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.610+0000 7fbbfd7ff640 1 --2- 172.21.15.70:0/2841318953 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbbf815a820 0x7fbbf80afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.610+0000 7fbbfd7ff640 1 --2- 172.21.15.70:0/2841318953 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbbf80ca680 0x7fbbf8157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.610+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/2841318953 >> 172.21.15.70:0/2841318953 conn(0x7fbbf806c050 msgr2=0x7fbbf815bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:26.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.610+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/2841318953 shutdown_connections 2023-10-08T06:53:26.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.610+0000 7fbbfd7ff640 1 -- 172.21.15.70:0/2841318953 wait complete. 2023-10-08T06:53:26.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T06:53:26.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.843+0000 7f09a359e640 1 Processor -- start 2023-10-08T06:53:26.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.843+0000 7f09a359e640 1 -- start start 2023-10-08T06:53:26.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a359e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09a40b0680 0x7f09a415e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:26.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09a40ca680 0x7f09a415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:26.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a359e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40cab60 con 0x7f09a415a820 2023-10-08T06:53:26.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a359e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a40cb960 con 0x7f09a40ca680 2023-10-08T06:53:26.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a359e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a40cbdf0 con 0x7f09a40b0680 2023-10-08T06:53:26.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09a40ca680 0x7f09a415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:26.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09a40ca680 0x7f09a415ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48344/0 (socket says 172.21.15.70:48344) 2023-10-08T06:53:26.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a2d9d640 1 -- 172.21.15.70:0/162985301 learned_addr learned my addr 172.21.15.70:0/162985301 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:26.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a1d9b640 1 --2- 172.21.15.70:0/162985301 >> v2:172.21.15.181:3300/0 conn(0x7f09a40b0680 0x7f09a415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:26.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a2d9d640 1 -- 172.21.15.70:0/162985301 >> v2:172.21.15.181:3300/0 conn(0x7f09a40b0680 msgr2=0x7f09a415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:26.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.845+0000 7f09a2d9d640 1 --2- 172.21.15.70:0/162985301 >> v2:172.21.15.181:3300/0 conn(0x7f09a40b0680 0x7f09a415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.846+0000 7f09a2d9d640 1 -- 172.21.15.70:0/162985301 >> v1:172.21.15.181:6789/0 conn(0x7f09a415a820 legacy=0x7f09a40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:26.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.846+0000 7f09a2d9d640 1 -- 172.21.15.70:0/162985301 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40cb030 con 0x7f09a40ca680 2023-10-08T06:53:26.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.847+0000 7f09a2d9d640 1 --2- 172.21.15.70:0/162985301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 0x7f09a415ea80 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f09a40718e0 tx=0x7f099c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59446d36d1ac1910 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:26.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.847+0000 7f09a159a640 1 -- 172.21.15.70:0/162985301 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c02c9f0 con 0x7f09a40ca680 2023-10-08T06:53:26.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.847+0000 7f09a159a640 1 -- 172.21.15.70:0/162985301 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c02cb50 con 0x7f09a40ca680 2023-10-08T06:53:26.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.847+0000 7f09a159a640 1 -- 172.21.15.70:0/162985301 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c0343f0 con 0x7f09a40ca680 2023-10-08T06:53:26.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.848+0000 7f09a359e640 1 -- 172.21.15.70:0/162985301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 msgr2=0x7f09a415ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:26.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.848+0000 7f09a359e640 1 --2- 172.21.15.70:0/162985301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 0x7f09a415ea80 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f09a40718e0 tx=0x7f099c02be50 comp rx=0 tx=0).stop 2023-10-08T06:53:26.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.848+0000 7f09a359e640 1 -- 172.21.15.70:0/162985301 shutdown_connections 2023-10-08T06:53:26.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.848+0000 7f09a359e640 1 --2- 172.21.15.70:0/162985301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 0x7f09a415ea80 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.848+0000 7f09a359e640 1 --2- 172.21.15.70:0/162985301 >> v2:172.21.15.181:3300/0 conn(0x7f09a40b0680 0x7f09a415e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.848+0000 7f09a359e640 1 -- 172.21.15.70:0/162985301 >> 172.21.15.70:0/162985301 conn(0x7f09a406c050 msgr2=0x7f09a406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:26.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.848+0000 7f09a359e640 1 -- 172.21.15.70:0/162985301 shutdown_connections 2023-10-08T06:53:26.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.848+0000 7f09a359e640 1 -- 172.21.15.70:0/162985301 wait complete. 2023-10-08T06:53:26.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.849+0000 7f09a359e640 1 Processor -- start 2023-10-08T06:53:26.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.850+0000 7f09a359e640 1 -- start start 2023-10-08T06:53:26.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09a40b0680 0x7f09a41e5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:26.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 0x7f09a415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:26.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a415a820 0x7f09a40ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:26.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4158d70 con 0x7f09a40ca680 2023-10-08T06:53:26.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a359e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41e6f70 con 0x7f09a415a820 2023-10-08T06:53:26.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4158160 con 0x7f09a40b0680 2023-10-08T06:53:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a1d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 0x7f09a415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a1d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 0x7f09a415d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48354/0 (socket says 172.21.15.70:48354) 2023-10-08T06:53:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a1d9b640 1 -- 172.21.15.70:0/2832759102 learned_addr learned my addr 172.21.15.70:0/2832759102 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a1d9b640 1 -- 172.21.15.70:0/2832759102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09a40b0680 msgr2=0x7f09a41e5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a1d9b640 1 --2- 172.21.15.70:0/2832759102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09a40b0680 0x7f09a41e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a1d9b640 1 -- 172.21.15.70:0/2832759102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a415a820 msgr2=0x7f09a40ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a1d9b640 1 --2- 172.21.15.70:0/2832759102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a415a820 0x7f09a40ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a1d9b640 1 -- 172.21.15.70:0/2832759102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c009b30 con 0x7f09a40ca680 2023-10-08T06:53:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.851+0000 7f09a1d9b640 1 --2- 172.21.15.70:0/2832759102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 0x7f09a415d310 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f099800d830 tx=0x7f099800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.852+0000 7f0992ffd640 1 -- 172.21.15.70:0/2832759102 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800b6c0 con 0x7f09a40ca680 2023-10-08T06:53:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.852+0000 7f0992ffd640 1 -- 172.21.15.70:0/2832759102 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099800b820 con 0x7f09a40ca680 2023-10-08T06:53:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.852+0000 7f0992ffd640 1 -- 172.21.15.70:0/2832759102 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800bbb0 con 0x7f09a40ca680 2023-10-08T06:53:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.852+0000 7f09a359e640 1 -- 172.21.15.70:0/2832759102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a4158440 con 0x7f09a40ca680 2023-10-08T06:53:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.852+0000 7f09a359e640 1 -- 172.21.15.70:0/2832759102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a4158a00 con 0x7f09a40ca680 2023-10-08T06:53:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.853+0000 7f09a359e640 1 -- 172.21.15.70:0/2832759102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a40cf280 con 0x7f09a40ca680 2023-10-08T06:53:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.854+0000 7f0992ffd640 1 -- 172.21.15.70:0/2832759102 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 86) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f099801e460 con 0x7f09a40ca680 2023-10-08T06:53:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.855+0000 7f0992ffd640 1 -- 172.21.15.70:0/2832759102 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f0998089d80 con 0x7f09a40ca680 2023-10-08T06:53:26.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:26.859+0000 7f0992ffd640 1 -- 172.21.15.70:0/2832759102 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0998054d20 con 0x7f09a40ca680 2023-10-08T06:53:27.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.048+0000 7f09a359e640 1 -- 172.21.15.70:0/2832759102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09a40b1310 con 0x7f09a40ca680 2023-10-08T06:53:27.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.052+0000 7f0992ffd640 1 -- 172.21.15.70:0/2832759102 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0998058340 con 0x7f09a40ca680 2023-10-08T06:53:27.053 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T06:53:27.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.056+0000 7f09a359e640 1 -- 172.21.15.70:0/2832759102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 msgr2=0x7f09a415d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.056+0000 7f09a359e640 1 --2- 172.21.15.70:0/2832759102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 0x7f09a415d310 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f099800d830 tx=0x7f099800dcf0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.056+0000 7f09a359e640 1 -- 172.21.15.70:0/2832759102 shutdown_connections 2023-10-08T06:53:27.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.056+0000 7f09a359e640 1 --2- 172.21.15.70:0/2832759102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09a415a820 0x7f09a40ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.056+0000 7f09a359e640 1 --2- 172.21.15.70:0/2832759102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a40ca680 0x7f09a415d310 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.056+0000 7f09a359e640 1 --2- 172.21.15.70:0/2832759102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09a40b0680 0x7f09a41e5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.056+0000 7f09a359e640 1 -- 172.21.15.70:0/2832759102 >> 172.21.15.70:0/2832759102 conn(0x7f09a406c050 msgr2=0x7f09a4159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:27.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.056+0000 7f09a359e640 1 -- 172.21.15.70:0/2832759102 shutdown_connections 2023-10-08T06:53:27.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.056+0000 7f09a359e640 1 -- 172.21.15.70:0/2832759102 wait complete. 2023-10-08T06:53:27.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:53:27.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.289+0000 7f5bc7e56640 1 Processor -- start 2023-10-08T06:53:27.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.289+0000 7f5bc7e56640 1 -- start start 2023-10-08T06:53:27.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.290+0000 7f5bc7e56640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5bc015cd20 0x7f5bc00c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.290+0000 7f5bc7e56640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5bc006d310 0x7f5bc006ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.290+0000 7f5bc7e56640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc00ccec0 con 0x7f5bc006dad0 2023-10-08T06:53:27.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.290+0000 7f5bc7e56640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc0065d70 con 0x7f5bc015cd20 2023-10-08T06:53:27.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.290+0000 7f5bc7e56640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc00cd1a0 con 0x7f5bc006d310 2023-10-08T06:53:27.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.291+0000 7f5bc5bf2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5bc015cd20 0x7f5bc00c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.291+0000 7f5bc53f1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5bc006dad0 0x7f5bc006cc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37250/0 (socket says 172.21.15.70:37250) 2023-10-08T06:53:27.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.291+0000 7f5bc53f1640 1 -- 172.21.15.70:0/315668140 learned_addr learned my addr 172.21.15.70:0/315668140 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:27.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.291+0000 7f5bc5bf2640 1 -- 172.21.15.70:0/315668140 >> v2:172.21.15.181:3300/0 conn(0x7f5bc006d310 msgr2=0x7f5bc006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:27.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.291+0000 7f5bc5bf2640 1 --2- 172.21.15.70:0/315668140 >> v2:172.21.15.181:3300/0 conn(0x7f5bc006d310 0x7f5bc006ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.291+0000 7f5bc5bf2640 1 -- 172.21.15.70:0/315668140 >> v1:172.21.15.70:6789/0 conn(0x7f5bc006dad0 legacy=0x7f5bc006cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.291+0000 7f5bc5bf2640 1 -- 172.21.15.70:0/315668140 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00cd480 con 0x7f5bc015cd20 2023-10-08T06:53:27.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.291+0000 7f5bc4bf0640 1 -- 172.21.15.70:0/315668140 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797184269 0 0) 0x7f5bc00ccec0 con 0x7f5bc006dad0 2023-10-08T06:53:27.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.292+0000 7f5bc5bf2640 1 --2- 172.21.15.70:0/315668140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 0x7f5bc00c96c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f5bc006c8f0 tx=0x7f5bbc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0742d6554d38690 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:27.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc4bf0640 1 -- 172.21.15.70:0/315668140 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc02cd10 con 0x7f5bc015cd20 2023-10-08T06:53:27.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc4bf0640 1 -- 172.21.15.70:0/315668140 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bbc02ce70 con 0x7f5bc015cd20 2023-10-08T06:53:27.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc4bf0640 1 -- 172.21.15.70:0/315668140 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc034640 con 0x7f5bc015cd20 2023-10-08T06:53:27.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc7e56640 1 -- 172.21.15.70:0/315668140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 msgr2=0x7f5bc00c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc7e56640 1 --2- 172.21.15.70:0/315668140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 0x7f5bc00c96c0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f5bc006c8f0 tx=0x7f5bbc02bf60 comp rx=0 tx=0).stop 2023-10-08T06:53:27.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc7e56640 1 -- 172.21.15.70:0/315668140 shutdown_connections 2023-10-08T06:53:27.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc7e56640 1 --2- 172.21.15.70:0/315668140 >> v2:172.21.15.181:3300/0 conn(0x7f5bc006d310 0x7f5bc006ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc7e56640 1 --2- 172.21.15.70:0/315668140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 0x7f5bc00c96c0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc7e56640 1 -- 172.21.15.70:0/315668140 >> 172.21.15.70:0/315668140 conn(0x7f5bc00b0b40 msgr2=0x7f5bc00b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:27.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc7e56640 1 -- 172.21.15.70:0/315668140 shutdown_connections 2023-10-08T06:53:27.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.293+0000 7f5bc7e56640 1 -- 172.21.15.70:0/315668140 wait complete. 2023-10-08T06:53:27.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.295+0000 7f5bc7e56640 1 Processor -- start 2023-10-08T06:53:27.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.295+0000 7f5bc7e56640 1 -- start start 2023-10-08T06:53:27.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.296+0000 7f5bc7e56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bc006d310 0x7f5bc015a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.296+0000 7f5bc7e56640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc006dad0 0x7f5bc015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.296+0000 7f5bc7e56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 0x7f5bc01599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.296+0000 7f5bc7e56640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00c6090 con 0x7f5bc015cd20 2023-10-08T06:53:27.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.296+0000 7f5bc7e56640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00c62e0 con 0x7f5bc006dad0 2023-10-08T06:53:27.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.296+0000 7f5bc7e56640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00c65c0 con 0x7f5bc006d310 2023-10-08T06:53:27.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.296+0000 7f5bc63f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 0x7f5bc01599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc63f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 0x7f5bc01599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48372/0 (socket says 172.21.15.70:48372) 2023-10-08T06:53:27.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc63f3640 1 -- 172.21.15.70:0/3931954421 learned_addr learned my addr 172.21.15.70:0/3931954421 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:27.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc53f1640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc006dad0 0x7f5bc015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc5bf2640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bc006d310 0x7f5bc015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc63f3640 1 -- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bc006d310 msgr2=0x7f5bc015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc63f3640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bc006d310 0x7f5bc015a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc63f3640 1 -- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc006dad0 msgr2=0x7f5bc015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc63f3640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc006dad0 0x7f5bc015ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc63f3640 1 -- 172.21.15.70:0/3931954421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc009a70 con 0x7f5bc015cd20 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc53f1640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc006dad0 0x7f5bc015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc5bf2640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bc006d310 0x7f5bc015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.297+0000 7f5bc63f3640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 0x7f5bc01599e0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009fd0 tx=0x7f5bb800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.298+0000 7f5bb67fc640 1 -- 172.21.15.70:0/3931954421 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8011830 con 0x7f5bc015cd20 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.298+0000 7f5bb67fc640 1 -- 172.21.15.70:0/3931954421 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb8011990 con 0x7f5bc015cd20 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.298+0000 7f5bb67fc640 1 -- 172.21.15.70:0/3931954421 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8011d20 con 0x7f5bc015cd20 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.298+0000 7f5bc7e56640 1 -- 172.21.15.70:0/3931954421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc00c68a0 con 0x7f5bc015cd20 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.298+0000 7f5bc7e56640 1 -- 172.21.15.70:0/3931954421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc0159f20 con 0x7f5bc015cd20 2023-10-08T06:53:27.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.299+0000 7f5bc7e56640 1 -- 172.21.15.70:0/3931954421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc00c6dc0 con 0x7f5bc015cd20 2023-10-08T06:53:27.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.300+0000 7f5bb67fc640 1 -- 172.21.15.70:0/3931954421 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 87) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f5bb801b460 con 0x7f5bc015cd20 2023-10-08T06:53:27.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.301+0000 7f5bb67fc640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ba406acc0 0x7f5ba406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.301+0000 7f5bb67fc640 1 -- 172.21.15.70:0/3931954421 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f5bb8089510 con 0x7f5bc015cd20 2023-10-08T06:53:27.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.302+0000 7f5bc5bf2640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ba406acc0 0x7f5ba406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.303+0000 7f5bc5bf2640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ba406acc0 0x7f5ba406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc036000 tx=0x7f5bbc030000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:27.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.306+0000 7f5bb67fc640 1 -- 172.21.15.70:0/3931954421 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5bc00c6dc0 con 0x7f5bc015cd20 2023-10-08T06:53:27.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.533+0000 7f5bc7e56640 1 -- 172.21.15.70:0/3931954421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bc00c6dc0 con 0x7f5bc015cd20 2023-10-08T06:53:27.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.534+0000 7f5bb67fc640 1 -- 172.21.15.70:0/3931954421 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f5bc00c6dc0 con 0x7f5bc015cd20 2023-10-08T06:53:27.536 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:27.536 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2889908260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2889908260},{"type":"v1","addr":"172.21.15.181:6837","nonce":2889908260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2651702867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2651702867},{"type":"v1","addr":"172.21.15.181:6835","nonce":2651702867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":9048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1574948577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1574948577},{"type":"v1","addr":"172.21.15.70:6833","nonce":1574948577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:57:26.074871+0000","modified":"2023-10-08T05:57:29.076265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9051":{"gid":9051,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6835/3537072579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3537072579},{"type":"v1","addr":"172.21.15.70:6835","nonce":3537072579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-08T06:53:27.537 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 76 2023-10-08T06:53:27.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 -- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ba406acc0 msgr2=0x7f5ba406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ba406acc0 0x7f5ba406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc036000 tx=0x7f5bbc030000 comp rx=0 tx=0).stop 2023-10-08T06:53:27.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 -- 172.21.15.70:0/3931954421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 msgr2=0x7f5bc01599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 0x7f5bc01599e0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009fd0 tx=0x7f5bb800b8e0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 -- 172.21.15.70:0/3931954421 shutdown_connections 2023-10-08T06:53:27.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ba406acc0 0x7f5ba406d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc015cd20 0x7f5bc01599e0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bc006dad0 0x7f5bc015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 --2- 172.21.15.70:0/3931954421 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5bc006d310 0x7f5bc015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 -- 172.21.15.70:0/3931954421 >> 172.21.15.70:0/3931954421 conn(0x7f5bc00b0b40 msgr2=0x7f5bc0160670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:27.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.537+0000 7f5bc7e56640 1 -- 172.21.15.70:0/3931954421 shutdown_connections 2023-10-08T06:53:27.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.538+0000 7f5bc7e56640 1 -- 172.21.15.70:0/3931954421 wait complete. 2023-10-08T06:53:27.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:53:27.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.766+0000 7fbb2ffff640 1 Processor -- start 2023-10-08T06:53:27.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.766+0000 7fbb2ffff640 1 -- start start 2023-10-08T06:53:27.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.767+0000 7fbb2ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb301599d0 0x7fbb300ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.768+0000 7fbb2ffff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb300cacf0 con 0x7fbb300cc210 2023-10-08T06:53:27.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.768+0000 7fbb2ffff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb300ca3c0 con 0x7fbb300b07d0 2023-10-08T06:53:27.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.768+0000 7fbb2ffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300b10f0 con 0x7fbb301599d0 2023-10-08T06:53:27.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.768+0000 7fbb27fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbb300cc210 0x7fbb300b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37254/0 (socket says 172.21.15.70:37254) 2023-10-08T06:53:27.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.768+0000 7fbb27fff640 1 -- 172.21.15.70:0/953832212 learned_addr learned my addr 172.21.15.70:0/953832212 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:27.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.768+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658400586 0 0) 0x7fbb300cacf0 con 0x7fbb300cc210 2023-10-08T06:53:27.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.768+0000 7fbb2effd640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb301599d0 0x7fbb300ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.768+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb1c003520 con 0x7fbb300cc210 2023-10-08T06:53:27.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287062386 0 0) 0x7fbb1c003520 con 0x7fbb300cc210 2023-10-08T06:53:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb301599d0 msgr2=0x7fbb300ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb301599d0 0x7fbb300ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 >> v1:172.21.15.181:6789/0 conn(0x7fbb300b07d0 legacy=0x7fbb3006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb300ca6a0 con 0x7fbb300cc210 2023-10-08T06:53:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fbb18001670 con 0x7fbb300cc210 2023-10-08T06:53:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 >> v1:172.21.15.70:6789/0 conn(0x7fbb300cc210 legacy=0x7fbb300b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb301599d0 0x7fbb1c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb1c003f20 0x7fbb1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c004d00 0x7fbb1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300b10f0 con 0x7fbb1c003f20 2023-10-08T06:53:27.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c00ba60 con 0x7fbb1c004d00 2023-10-08T06:53:27.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c00bc20 con 0x7fbb301599d0 2023-10-08T06:53:27.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.769+0000 7fbb27fff640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb1c003f20 0x7fbb1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.770+0000 7fbb2effd640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb301599d0 0x7fbb1c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.770+0000 7fbb2f7fe640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c004d00 0x7fbb1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.770+0000 7fbb27fff640 1 -- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb301599d0 msgr2=0x7fbb1c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:27.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.770+0000 7fbb27fff640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb301599d0 0x7fbb1c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.770+0000 7fbb27fff640 1 -- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c004d00 msgr2=0x7fbb1c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:27.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.770+0000 7fbb27fff640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c004d00 0x7fbb1c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.770+0000 7fbb27fff640 1 -- 172.21.15.70:0/953832212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c003520 con 0x7fbb1c003f20 2023-10-08T06:53:27.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.770+0000 7fbb2f7fe640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c004d00 0x7fbb1c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:27.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.771+0000 7fbb27fff640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb1c003f20 0x7fbb1c0047c0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c210 tx=0x7fbb1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f744d0d8b0c73980 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:27.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.771+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18005ad0 con 0x7fbb1c003f20 2023-10-08T06:53:27.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.771+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb18005c30 con 0x7fbb1c003f20 2023-10-08T06:53:27.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.771+0000 7fbb2e7fc640 1 -- 172.21.15.70:0/953832212 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1802cad0 con 0x7fbb1c003f20 2023-10-08T06:53:27.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.772+0000 7fbb2ffff640 1 -- 172.21.15.70:0/953832212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb1c003f20 msgr2=0x7fbb1c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.772+0000 7fbb2ffff640 1 --2- 172.21.15.70:0/953832212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb1c003f20 0x7fbb1c0047c0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c210 tx=0x7fbb1c0057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.772+0000 7fbb2f7fe640 1 -- 172.21.15.70:0/953832212 reap_dead start 2023-10-08T06:53:27.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.772+0000 7fbb2ffff640 1 -- 172.21.15.70:0/953832212 shutdown_connections 2023-10-08T06:53:27.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.772+0000 7fbb2ffff640 1 -- 172.21.15.70:0/953832212 >> 172.21.15.70:0/953832212 conn(0x7fbb300cb0c0 msgr2=0x7fbb300cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.772+0000 7fbb2ffff640 1 -- 172.21.15.70:0/953832212 shutdown_connections 2023-10-08T06:53:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.772+0000 7fbb2ffff640 1 -- 172.21.15.70:0/953832212 wait complete. 2023-10-08T06:53:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.774+0000 7fbb2ffff640 1 Processor -- start 2023-10-08T06:53:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.774+0000 7fbb2ffff640 1 -- start start 2023-10-08T06:53:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb2ffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c003f20 0x7fbb30169010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb3014e550 0x7fbb3014e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb2ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb3014be60 0x7fbb3014c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb2ffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3014ef10 con 0x7fbb3014e550 2023-10-08T06:53:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb2ffff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3014cd50 con 0x7fbb1c003f20 2023-10-08T06:53:27.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb2ffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3014d030 con 0x7fbb3014be60 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb3014e550 0x7fbb3014e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb3014e550 0x7fbb3014e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48390/0 (socket says 172.21.15.70:48390) 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb27fff640 1 -- 172.21.15.70:0/2389410387 learned_addr learned my addr 172.21.15.70:0/2389410387 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb2effd640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c003f20 0x7fbb30169010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.775+0000 7fbb2f7fe640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb3014be60 0x7fbb3014c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb27fff640 1 -- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb3014be60 msgr2=0x7fbb3014c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb27fff640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb3014be60 0x7fbb3014c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb27fff640 1 -- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c003f20 msgr2=0x7fbb30169010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb27fff640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c003f20 0x7fbb30169010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb27fff640 1 -- 172.21.15.70:0/2389410387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3014d310 con 0x7fbb3014e550 2023-10-08T06:53:27.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb2effd640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c003f20 0x7fbb30169010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:27.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb27fff640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb3014e550 0x7fbb3014e9d0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c210 tx=0x7fbb18001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:27.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb277fe640 1 -- 172.21.15.70:0/2389410387 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1803c040 con 0x7fbb3014e550 2023-10-08T06:53:27.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb277fe640 1 -- 172.21.15.70:0/2389410387 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1802cca0 con 0x7fbb3014e550 2023-10-08T06:53:27.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb277fe640 1 -- 172.21.15.70:0/2389410387 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb18007270 con 0x7fbb3014e550 2023-10-08T06:53:27.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb2ffff640 1 -- 172.21.15.70:0/2389410387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3014d5f0 con 0x7fbb3014e550 2023-10-08T06:53:27.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.776+0000 7fbb2ffff640 1 -- 172.21.15.70:0/2389410387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb30168390 con 0x7fbb3014e550 2023-10-08T06:53:27.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.778+0000 7fbb2ffff640 1 -- 172.21.15.70:0/2389410387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb300ca030 con 0x7fbb3014e550 2023-10-08T06:53:27.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.779+0000 7fbb277fe640 1 -- 172.21.15.70:0/2389410387 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 87) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fbb18032070 con 0x7fbb3014e550 2023-10-08T06:53:27.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.779+0000 7fbb277fe640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fbb1006acf0 0x7fbb1006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:27.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.779+0000 7fbb277fe640 1 -- 172.21.15.70:0/2389410387 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fbb180aa8e0 con 0x7fbb3014e550 2023-10-08T06:53:27.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.780+0000 7fbb2effd640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fbb1006acf0 0x7fbb1006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:27.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.781+0000 7fbb2effd640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fbb1006acf0 0x7fbb1006d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbb3015d370 tx=0x7fbb20008040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:27.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.784+0000 7fbb277fe640 1 -- 172.21.15.70:0/2389410387 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb18075770 con 0x7fbb3014e550 2023-10-08T06:53:27.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.976+0000 7fbb2ffff640 1 -- 172.21.15.70:0/2389410387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb3004e160 con 0x7fbb3014e550 2023-10-08T06:53:27.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.979+0000 7fbb277fe640 1 -- 172.21.15.70:0/2389410387 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+18015 (secure 0 0 0) 0x7fbb18046020 con 0x7fbb3014e550 2023-10-08T06:53:27.981 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:27.981 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":172,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:53:27.041894+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:57:23.815675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-08T05:57:25.045292+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2061492414":"2023-10-09T06:53:26.036761+0000","172.21.15.181:6833/236882560":"2023-10-09T06:53:26.036761+0000","172.21.15.181:6832/236882560":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/23330666":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/599491381":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/2571920213":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/3947544788":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3693343940":"2023-10-09T05:57:41.519220+0000","172.21.15.181:6833/2068783299":"2023-10-09T05:57:41.519220+0000","172.21.15.181:6832/2068783299":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3070110051":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3604402742":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/1336064042":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3458973146":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/3875931700":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/3791109415":"2023-10-09T05:57:41.519220+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:53:27.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 -- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fbb1006acf0 msgr2=0x7fbb1006d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fbb1006acf0 0x7fbb1006d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbb3015d370 tx=0x7fbb20008040 comp rx=0 tx=0).stop 2023-10-08T06:53:27.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 -- 172.21.15.70:0/2389410387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb3014e550 msgr2=0x7fbb3014e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:27.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb3014e550 0x7fbb3014e9d0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fbb1800c210 tx=0x7fbb18001a00 comp rx=0 tx=0).stop 2023-10-08T06:53:27.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 -- 172.21.15.70:0/2389410387 shutdown_connections 2023-10-08T06:53:27.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb3014be60 0x7fbb3014c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fbb1006acf0 0x7fbb1006d1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb3014e550 0x7fbb3014e9d0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 --2- 172.21.15.70:0/2389410387 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb1c003f20 0x7fbb30169010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:27.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 -- 172.21.15.70:0/2389410387 >> 172.21.15.70:0/2389410387 conn(0x7fbb300cb0c0 msgr2=0x7fbb300ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:27.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 -- 172.21.15.70:0/2389410387 shutdown_connections 2023-10-08T06:53:27.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:27.982+0000 7fbb2ffff640 1 -- 172.21.15.70:0/2389410387 wait complete. 2023-10-08T06:53:28.003 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T06:53:28.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:53:28.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.225+0000 7f920b155640 1 Processor -- start 2023-10-08T06:53:28.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.225+0000 7f920b155640 1 -- start start 2023-10-08T06:53:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.226+0000 7f920b155640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92040c90b0 0x7f92040cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.226+0000 7f920b155640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920415b2b0 con 0x7f920415ab40 2023-10-08T06:53:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.226+0000 7f920b155640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920415b590 con 0x7f92040cb3b0 2023-10-08T06:53:28.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.226+0000 7f920b155640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920415bf30 con 0x7f92040c90b0 2023-10-08T06:53:28.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.226+0000 7f9208ef1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f920415ab40 0x7f92040ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37268/0 (socket says 172.21.15.70:37268) 2023-10-08T06:53:28.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.226+0000 7f9208ef1640 1 -- 172.21.15.70:0/3052536386 learned_addr learned my addr 172.21.15.70:0/3052536386 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:28.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658862263 0 0) 0x7f920415b2b0 con 0x7f920415ab40 2023-10-08T06:53:28.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f4003520 con 0x7f920415ab40 2023-10-08T06:53:28.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f9203fff640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92040c90b0 0x7f92040cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:28.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2453137481 0 0) 0x7f91f4003520 con 0x7f920415ab40 2023-10-08T06:53:28.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92040c90b0 msgr2=0x7f92040cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:28.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92040c90b0 0x7f92040cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 >> v1:172.21.15.181:6789/0 conn(0x7f92040cb3b0 legacy=0x7f920415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920415b870 con 0x7f920415ab40 2023-10-08T06:53:28.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f91f8001670 con 0x7f920415ab40 2023-10-08T06:53:28.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 >> v1:172.21.15.70:6789/0 conn(0x7f920415ab40 legacy=0x7f92040ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91f40039a0 0x7f91f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 0x7f91f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92040c90b0 0x7f91f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920415bf30 con 0x7f91f40047c0 2023-10-08T06:53:28.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f400bab0 con 0x7f91f40039a0 2023-10-08T06:53:28.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.227+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f400bc70 con 0x7f92040c90b0 2023-10-08T06:53:28.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.228+0000 7f9203fff640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 0x7f91f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:28.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.228+0000 7f92096f2640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92040c90b0 0x7f91f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:28.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.228+0000 7f9203fff640 1 -- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92040c90b0 msgr2=0x7f91f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:28.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.228+0000 7f9203fff640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92040c90b0 0x7f91f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.228+0000 7f9203fff640 1 -- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91f40039a0 msgr2=0x7f91f4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:28.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.228+0000 7f9203fff640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f91f40039a0 0x7f91f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.228+0000 7f9203fff640 1 -- 172.21.15.70:0/3052536386 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f4003520 con 0x7f91f40047c0 2023-10-08T06:53:28.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.228+0000 7f92096f2640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92040c90b0 0x7f91f40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:28.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.229+0000 7f9203fff640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 0x7f91f4004c40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f91f000ba30 tx=0x7f91f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69225e057a0db22d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:28.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.229+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0032640 con 0x7f91f40047c0 2023-10-08T06:53:28.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.229+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f00327a0 con 0x7f91f40047c0 2023-10-08T06:53:28.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.229+0000 7f92037fe640 1 -- 172.21.15.70:0/3052536386 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0032b30 con 0x7f91f40047c0 2023-10-08T06:53:28.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.230+0000 7f920b155640 1 -- 172.21.15.70:0/3052536386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 msgr2=0x7f91f4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.230+0000 7f920b155640 1 --2- 172.21.15.70:0/3052536386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 0x7f91f4004c40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f91f000ba30 tx=0x7f91f40057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.230+0000 7f92096f2640 1 -- 172.21.15.70:0/3052536386 reap_dead start 2023-10-08T06:53:28.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.230+0000 7f920b155640 1 -- 172.21.15.70:0/3052536386 shutdown_connections 2023-10-08T06:53:28.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.230+0000 7f920b155640 1 -- 172.21.15.70:0/3052536386 >> 172.21.15.70:0/3052536386 conn(0x7f920406a380 msgr2=0x7f92040b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:28.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.230+0000 7f920b155640 1 -- 172.21.15.70:0/3052536386 shutdown_connections 2023-10-08T06:53:28.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.230+0000 7f920b155640 1 -- 172.21.15.70:0/3052536386 wait complete. 2023-10-08T06:53:28.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.232+0000 7f920b155640 1 Processor -- start 2023-10-08T06:53:28.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.232+0000 7f920b155640 1 -- start start 2023-10-08T06:53:28.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.233+0000 7f920b155640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 0x7f92040adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.233+0000 7f920b155640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9204153f30 0x7f92040b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.233+0000 7f920b155640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9204144e70 0x7f9204145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.233+0000 7f920b155640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92041547b0 con 0x7f91f40047c0 2023-10-08T06:53:28.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.233+0000 7f920b155640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040b3010 con 0x7f9204153f30 2023-10-08T06:53:28.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.233+0000 7f920b155640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9204154ce0 con 0x7f9204144e70 2023-10-08T06:53:28.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.233+0000 7f9208ef1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 0x7f92040adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:28.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.233+0000 7f9208ef1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 0x7f92040adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48406/0 (socket says 172.21.15.70:48406) 2023-10-08T06:53:28.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.233+0000 7f9208ef1640 1 -- 172.21.15.70:0/2981412653 learned_addr learned my addr 172.21.15.70:0/2981412653 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:28.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f9208ef1640 1 -- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9204144e70 msgr2=0x7f9204145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f9208ef1640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9204144e70 0x7f9204145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f9208ef1640 1 -- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9204153f30 msgr2=0x7f92040b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f9208ef1640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9204153f30 0x7f92040b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f9208ef1640 1 -- 172.21.15.70:0/2981412653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9204154fc0 con 0x7f91f40047c0 2023-10-08T06:53:28.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f9208ef1640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 0x7f92040adab0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c730 tx=0x7f91f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:28.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f92017fa640 1 -- 172.21.15.70:0/2981412653 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80072d0 con 0x7f91f40047c0 2023-10-08T06:53:28.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f92017fa640 1 -- 172.21.15.70:0/2981412653 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f8007430 con 0x7f91f40047c0 2023-10-08T06:53:28.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f920b155640 1 -- 172.21.15.70:0/2981412653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92040b32b0 con 0x7f91f40047c0 2023-10-08T06:53:28.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f92017fa640 1 -- 172.21.15.70:0/2981412653 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8004a20 con 0x7f91f40047c0 2023-10-08T06:53:28.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.234+0000 7f920b155640 1 -- 172.21.15.70:0/2981412653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9204145860 con 0x7f91f40047c0 2023-10-08T06:53:28.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.235+0000 7f920b155640 1 -- 172.21.15.70:0/2981412653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92040b34f0 con 0x7f91f40047c0 2023-10-08T06:53:28.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.236+0000 7f92017fa640 1 -- 172.21.15.70:0/2981412653 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 88) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f91f800a5e0 con 0x7f91f40047c0 2023-10-08T06:53:28.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.237+0000 7f92017fa640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f91e806af30 0x7f91e806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.237+0000 7f92017fa640 1 -- 172.21.15.70:0/2981412653 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f91f8089090 con 0x7f91f40047c0 2023-10-08T06:53:28.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.238+0000 7f9203fff640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f91e806af30 0x7f91e806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:28.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.239+0000 7f9203fff640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f91e806af30 0x7f91e806d3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f91f000ba00 tx=0x7f91f0032000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:28.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.241+0000 7f92017fa640 1 -- 172.21.15.70:0/2981412653 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92040b34f0 con 0x7f91f40047c0 2023-10-08T06:53:28.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.467+0000 7f920b155640 1 -- 172.21.15.70:0/2981412653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92040b34f0 con 0x7f91f40047c0 2023-10-08T06:53:28.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.468+0000 7f92017fa640 1 -- 172.21.15.70:0/2981412653 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f92040b34f0 con 0x7f91f40047c0 2023-10-08T06:53:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:28.470 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2889908260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2889908260},{"type":"v1","addr":"172.21.15.181:6837","nonce":2889908260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2651702867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2651702867},{"type":"v1","addr":"172.21.15.181:6835","nonce":2651702867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":9048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1574948577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1574948577},{"type":"v1","addr":"172.21.15.70:6833","nonce":1574948577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:57:26.074871+0000","modified":"2023-10-08T05:57:29.076265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9051":{"gid":9051,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6835/3537072579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3537072579},{"type":"v1","addr":"172.21.15.70:6835","nonce":3537072579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-08T06:53:28.471 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 76 2023-10-08T06:53:28.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 -- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f91e806af30 msgr2=0x7f91e806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f91e806af30 0x7f91e806d3f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f91f000ba00 tx=0x7f91f0032000 comp rx=0 tx=0).stop 2023-10-08T06:53:28.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 -- 172.21.15.70:0/2981412653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 msgr2=0x7f92040adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 0x7f92040adab0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f91f800c730 tx=0x7f91f800cbf0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 -- 172.21.15.70:0/2981412653 shutdown_connections 2023-10-08T06:53:28.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9204144e70 0x7f9204145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9204153f30 0x7f92040b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f91e806af30 0x7f91e806d3f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 --2- 172.21.15.70:0/2981412653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91f40047c0 0x7f92040adab0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 -- 172.21.15.70:0/2981412653 >> 172.21.15.70:0/2981412653 conn(0x7f920406a380 msgr2=0x7f920406ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:28.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 -- 172.21.15.70:0/2981412653 shutdown_connections 2023-10-08T06:53:28.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.471+0000 7f920b155640 1 -- 172.21.15.70:0/2981412653 wait complete. 2023-10-08T06:53:28.490 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:53:28.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.691+0000 7fa568df0640 1 Processor -- start 2023-10-08T06:53:28.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.692+0000 7fa568df0640 1 -- start start 2023-10-08T06:53:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.693+0000 7fa568df0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa5641599d0 0x7fa5640ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.693+0000 7fa568df0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5640b07d0 0x7fa56406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.693+0000 7fa568df0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5640cacf0 con 0x7fa5640cc210 2023-10-08T06:53:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.693+0000 7fa568df0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5640b10f0 con 0x7fa5640b07d0 2023-10-08T06:53:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.693+0000 7fa568df0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5640ca3c0 con 0x7fa5641599d0 2023-10-08T06:53:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.693+0000 7fa563fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5640b07d0 0x7fa56406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.693+0000 7fa563fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa5640b07d0 0x7fa56406c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48416/0 (socket says 172.21.15.70:48416) 2023-10-08T06:53:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.693+0000 7fa563fff640 1 -- 172.21.15.70:0/1130867341 learned_addr learned my addr 172.21.15.70:0/1130867341 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.694+0000 7fa5637fe640 1 --2- 172.21.15.70:0/1130867341 >> v2:172.21.15.181:3300/0 conn(0x7fa5641599d0 0x7fa5640ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.694+0000 7fa563fff640 1 -- 172.21.15.70:0/1130867341 >> v2:172.21.15.181:3300/0 conn(0x7fa5641599d0 msgr2=0x7fa5640ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.694+0000 7fa563fff640 1 --2- 172.21.15.70:0/1130867341 >> v2:172.21.15.181:3300/0 conn(0x7fa5641599d0 0x7fa5640ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.694+0000 7fa563fff640 1 -- 172.21.15.70:0/1130867341 >> v1:172.21.15.181:6789/0 conn(0x7fa5640cc210 legacy=0x7fa5640b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.694+0000 7fa563fff640 1 -- 172.21.15.70:0/1130867341 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5640ca6a0 con 0x7fa5640b07d0 2023-10-08T06:53:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.694+0000 7fa5637fe640 1 --2- 172.21.15.70:0/1130867341 >> v2:172.21.15.181:3300/0 conn(0x7fa5641599d0 0x7fa5640ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:28.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.695+0000 7fa563fff640 1 --2- 172.21.15.70:0/1130867341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 0x7fa56406c310 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa564071350 tx=0x7fa55c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd825724ba489366 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:28.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.695+0000 7fa562ffd640 1 -- 172.21.15.70:0/1130867341 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c02ca80 con 0x7fa5640b07d0 2023-10-08T06:53:28.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.695+0000 7fa562ffd640 1 -- 172.21.15.70:0/1130867341 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c02cbe0 con 0x7fa5640b07d0 2023-10-08T06:53:28.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.695+0000 7fa562ffd640 1 -- 172.21.15.70:0/1130867341 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0343f0 con 0x7fa5640b07d0 2023-10-08T06:53:28.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.696+0000 7fa568df0640 1 -- 172.21.15.70:0/1130867341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 msgr2=0x7fa56406c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.696+0000 7fa568df0640 1 --2- 172.21.15.70:0/1130867341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 0x7fa56406c310 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa564071350 tx=0x7fa55c02bee0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.696+0000 7fa568df0640 1 -- 172.21.15.70:0/1130867341 shutdown_connections 2023-10-08T06:53:28.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.696+0000 7fa568df0640 1 --2- 172.21.15.70:0/1130867341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 0x7fa56406c310 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.696+0000 7fa568df0640 1 --2- 172.21.15.70:0/1130867341 >> v2:172.21.15.181:3300/0 conn(0x7fa5641599d0 0x7fa5640ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.696+0000 7fa568df0640 1 -- 172.21.15.70:0/1130867341 >> 172.21.15.70:0/1130867341 conn(0x7fa5640cb0c0 msgr2=0x7fa5640cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:28.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.696+0000 7fa568df0640 1 -- 172.21.15.70:0/1130867341 shutdown_connections 2023-10-08T06:53:28.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.696+0000 7fa568df0640 1 -- 172.21.15.70:0/1130867341 wait complete. 2023-10-08T06:53:28.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.698+0000 7fa568df0640 1 Processor -- start 2023-10-08T06:53:28.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.698+0000 7fa568df0640 1 -- start start 2023-10-08T06:53:28.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa568df0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 0x7fa56415b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa568df0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5640cc210 0x7fa5640c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa568df0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5641599d0 0x7fa5640c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa568df0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640aba70 con 0x7fa5640b07d0 2023-10-08T06:53:28.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa568df0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640abd50 con 0x7fa5640cc210 2023-10-08T06:53:28.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa568df0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5640ac030 con 0x7fa5641599d0 2023-10-08T06:53:28.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa5637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 0x7fa56415b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:28.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa5637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 0x7fa56415b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48418/0 (socket says 172.21.15.70:48418) 2023-10-08T06:53:28.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa5637fe640 1 -- 172.21.15.70:0/2243980537 learned_addr learned my addr 172.21.15.70:0/2243980537 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:28.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa5637fe640 1 -- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5641599d0 msgr2=0x7fa5640c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa5637fe640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5641599d0 0x7fa5640c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa5637fe640 1 -- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5640cc210 msgr2=0x7fa5640c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.699+0000 7fa55bfff640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5640cc210 0x7fa5640c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:28.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.700+0000 7fa5637fe640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5640cc210 0x7fa5640c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.700+0000 7fa5637fe640 1 -- 172.21.15.70:0/2243980537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c009b30 con 0x7fa5640b07d0 2023-10-08T06:53:28.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.700+0000 7fa5637fe640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 0x7fa56415b760 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa55400c7f0 tx=0x7fa55400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:28.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.700+0000 7fa560ff9640 1 -- 172.21.15.70:0/2243980537 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa554007820 con 0x7fa5640b07d0 2023-10-08T06:53:28.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.700+0000 7fa560ff9640 1 -- 172.21.15.70:0/2243980537 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa554007980 con 0x7fa5640b07d0 2023-10-08T06:53:28.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.700+0000 7fa560ff9640 1 -- 172.21.15.70:0/2243980537 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa554007cf0 con 0x7fa5640b07d0 2023-10-08T06:53:28.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.700+0000 7fa568df0640 1 -- 172.21.15.70:0/2243980537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5640ac310 con 0x7fa5640b07d0 2023-10-08T06:53:28.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.700+0000 7fa568df0640 1 -- 172.21.15.70:0/2243980537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa564156df0 con 0x7fa5640b07d0 2023-10-08T06:53:28.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.701+0000 7fa568df0640 1 -- 172.21.15.70:0/2243980537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56404e160 con 0x7fa5640b07d0 2023-10-08T06:53:28.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.702+0000 7fa560ff9640 1 -- 172.21.15.70:0/2243980537 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 88) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fa5540113f0 con 0x7fa5640b07d0 2023-10-08T06:53:28.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.703+0000 7fa560ff9640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa54406af30 0x7fa54406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:28.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.703+0000 7fa560ff9640 1 -- 172.21.15.70:0/2243980537 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6877+0+0 (secure 0 0 0) 0x7fa554089f60 con 0x7fa5640b07d0 2023-10-08T06:53:28.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.704+0000 7fa55bfff640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa54406af30 0x7fa54406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:28.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.705+0000 7fa55bfff640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa54406af30 0x7fa54406d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa564070ac0 tx=0x7fa54c009040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:28.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.708+0000 7fa560ff9640 1 -- 172.21.15.70:0/2243980537 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa554054a10 con 0x7fa5640b07d0 2023-10-08T06:53:28.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.940+0000 7fa568df0640 1 -- 172.21.15.70:0/2243980537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5640abbb0 con 0x7fa5640b07d0 2023-10-08T06:53:28.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.942+0000 7fa560ff9640 1 -- 172.21.15.70:0/2243980537 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fa5640abbb0 con 0x7fa5640b07d0 2023-10-08T06:53:28.943 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:28.944 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2889908260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2889908260},{"type":"v1","addr":"172.21.15.181:6837","nonce":2889908260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2651702867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2651702867},{"type":"v1","addr":"172.21.15.181:6835","nonce":2651702867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":9048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1574948577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1574948577},{"type":"v1","addr":"172.21.15.70:6833","nonce":1574948577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T05:57:26.074871+0000","modified":"2023-10-08T05:57:29.076265+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9051":{"gid":9051,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":5,"addr":"172.21.15.70:6835/3537072579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3537072579},{"type":"v1","addr":"172.21.15.70:6835","nonce":3537072579}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-10-08T06:53:28.944 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 76 2023-10-08T06:53:28.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.944+0000 7fa568df0640 1 -- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa54406af30 msgr2=0x7fa54406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.944+0000 7fa568df0640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa54406af30 0x7fa54406d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa564070ac0 tx=0x7fa54c009040 comp rx=0 tx=0).stop 2023-10-08T06:53:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.944+0000 7fa568df0640 1 -- 172.21.15.70:0/2243980537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 msgr2=0x7fa56415b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.944+0000 7fa568df0640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 0x7fa56415b760 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fa55400c7f0 tx=0x7fa55400ccb0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.944+0000 7fa568df0640 1 -- 172.21.15.70:0/2243980537 shutdown_connections 2023-10-08T06:53:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.944+0000 7fa568df0640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5641599d0 0x7fa5640c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.944+0000 7fa568df0640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5640cc210 0x7fa5640c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.944+0000 7fa568df0640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa54406af30 0x7fa54406d3f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.944+0000 7fa568df0640 1 --2- 172.21.15.70:0/2243980537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5640b07d0 0x7fa56415b760 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.944+0000 7fa568df0640 1 -- 172.21.15.70:0/2243980537 >> 172.21.15.70:0/2243980537 conn(0x7fa5640cb0c0 msgr2=0x7fa564158b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.945+0000 7fa568df0640 1 -- 172.21.15.70:0/2243980537 shutdown_connections 2023-10-08T06:53:28.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:28.945+0000 7fa568df0640 1 -- 172.21.15.70:0/2243980537 wait complete. 2023-10-08T06:53:28.965 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:53:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.180+0000 7f11f7fff640 1 Processor -- start 2023-10-08T06:53:29.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.180+0000 7f11f7fff640 1 -- start start 2023-10-08T06:53:29.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.182+0000 7f11f7fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11f80b07d0 0x7f11f8165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.182+0000 7f11f7fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f815f7d0 con 0x7f11f80adc70 2023-10-08T06:53:29.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.182+0000 7f11f7fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f815efb0 con 0x7f11f815ab30 2023-10-08T06:53:29.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.182+0000 7f11f7fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f815f210 con 0x7f11f80b07d0 2023-10-08T06:53:29.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.182+0000 7f11ee7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f11f80adc70 0x7f11f80b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37276/0 (socket says 172.21.15.70:37276) 2023-10-08T06:53:29.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.182+0000 7f11ee7fc640 1 -- 172.21.15.70:0/3913373760 learned_addr learned my addr 172.21.15.70:0/3913373760 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.182+0000 7f11f77fe640 1 --2- 172.21.15.70:0/3913373760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11f80b07d0 0x7f11f8165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:29.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589812208 0 0) 0x7f11f815f7d0 con 0x7f11f80adc70 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11e4003520 con 0x7f11f80adc70 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2596688215 0 0) 0x7f11e4003520 con 0x7f11f80adc70 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11f80b07d0 msgr2=0x7f11f8165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 --2- 172.21.15.70:0/3913373760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11f80b07d0 0x7f11f8165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 >> v1:172.21.15.181:6789/0 conn(0x7f11f815ab30 legacy=0x7f11f815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f815f4f0 con 0x7f11f80adc70 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f11e00019f0 con 0x7f11f80adc70 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 >> v1:172.21.15.70:6789/0 conn(0x7f11f80adc70 legacy=0x7f11f80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 --2- 172.21.15.70:0/3913373760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11e40039a0 0x7f11e4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 --2- 172.21.15.70:0/3913373760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11f80b07d0 0x7f11e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 --2- 172.21.15.70:0/3913373760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e4004d00 0x7f11e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f815f210 con 0x7f11e40039a0 2023-10-08T06:53:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e400ba60 con 0x7f11e4004d00 2023-10-08T06:53:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e400bc20 con 0x7f11f80b07d0 2023-10-08T06:53:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.183+0000 7f11f6ffd640 1 --2- 172.21.15.70:0/3913373760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11e40039a0 0x7f11e4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.184+0000 7f11f6ffd640 1 -- 172.21.15.70:0/3913373760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11f80b07d0 msgr2=0x7f11e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.184+0000 7f11f6ffd640 1 --2- 172.21.15.70:0/3913373760 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11f80b07d0 0x7f11e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.184+0000 7f11f6ffd640 1 -- 172.21.15.70:0/3913373760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e4004d00 msgr2=0x7f11e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.184+0000 7f11f6ffd640 1 --2- 172.21.15.70:0/3913373760 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11e4004d00 0x7f11e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.184+0000 7f11f6ffd640 1 -- 172.21.15.70:0/3913373760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e4003520 con 0x7f11e40039a0 2023-10-08T06:53:29.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.185+0000 7f11f6ffd640 1 --2- 172.21.15.70:0/3913373760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11e40039a0 0x7f11e4003dc0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f11f815f140 tx=0x7f11e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1fbc355c7d824 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:29.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.185+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80052d0 con 0x7f11e40039a0 2023-10-08T06:53:29.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.185+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e802e050 con 0x7f11e40039a0 2023-10-08T06:53:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.185+0000 7f11f67fc640 1 -- 172.21.15.70:0/3913373760 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80035f0 con 0x7f11e40039a0 2023-10-08T06:53:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.185+0000 7f11f7fff640 1 -- 172.21.15.70:0/3913373760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11e40039a0 msgr2=0x7f11e4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.185+0000 7f11f7fff640 1 --2- 172.21.15.70:0/3913373760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11e40039a0 0x7f11e4003dc0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f11f815f140 tx=0x7f11e40057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.186+0000 7f11f77fe640 1 -- 172.21.15.70:0/3913373760 reap_dead start 2023-10-08T06:53:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.186+0000 7f11f7fff640 1 -- 172.21.15.70:0/3913373760 shutdown_connections 2023-10-08T06:53:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.186+0000 7f11f7fff640 1 -- 172.21.15.70:0/3913373760 >> 172.21.15.70:0/3913373760 conn(0x7f11f80cb0c0 msgr2=0x7f11f80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.186+0000 7f11f7fff640 1 -- 172.21.15.70:0/3913373760 shutdown_connections 2023-10-08T06:53:29.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.186+0000 7f11f7fff640 1 -- 172.21.15.70:0/3913373760 wait complete. 2023-10-08T06:53:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.187+0000 7f11f7fff640 1 Processor -- start 2023-10-08T06:53:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.188+0000 7f11f7fff640 1 -- start start 2023-10-08T06:53:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f7fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11e40039a0 0x7f11f80c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f7fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f80c7230 0x7f11f80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11f80c5c50 0x7f11f806cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f7fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80c6da0 con 0x7f11f80c5c50 2023-10-08T06:53:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f7fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80c6190 con 0x7f11f80c7230 2023-10-08T06:53:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f7fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80c6470 con 0x7f11e40039a0 2023-10-08T06:53:29.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11f80c5c50 0x7f11f806cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11f80c5c50 0x7f11f806cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48440/0 (socket says 172.21.15.70:48440) 2023-10-08T06:53:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f77fe640 1 -- 172.21.15.70:0/1609637901 learned_addr learned my addr 172.21.15.70:0/1609637901 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f77fe640 1 -- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11e40039a0 msgr2=0x7f11f80c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f77fe640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11e40039a0 0x7f11f80c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f77fe640 1 -- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f80c7230 msgr2=0x7f11f80c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f77fe640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f80c7230 0x7f11f80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.189+0000 7f11f77fe640 1 -- 172.21.15.70:0/1609637901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80c6750 con 0x7f11f80c5c50 2023-10-08T06:53:29.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.190+0000 7f11f77fe640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11f80c5c50 0x7f11f806cc50 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f11f80c5690 tx=0x7f11f000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.190+0000 7f11effff640 1 -- 172.21.15.70:0/1609637901 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0019070 con 0x7f11f80c5c50 2023-10-08T06:53:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.190+0000 7f11effff640 1 -- 172.21.15.70:0/1609637901 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f0004020 con 0x7f11f80c5c50 2023-10-08T06:53:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.190+0000 7f11f7fff640 1 -- 172.21.15.70:0/1609637901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f80c6a30 con 0x7f11f80c5c50 2023-10-08T06:53:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.190+0000 7f11effff640 1 -- 172.21.15.70:0/1609637901 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0004210 con 0x7f11f80c5c50 2023-10-08T06:53:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.190+0000 7f11f7fff640 1 -- 172.21.15.70:0/1609637901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f80b2af0 con 0x7f11f80c5c50 2023-10-08T06:53:29.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.191+0000 7f11f7fff640 1 -- 172.21.15.70:0/1609637901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f80cb6d0 con 0x7f11f80c5c50 2023-10-08T06:53:29.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.192+0000 7f11effff640 1 -- 172.21.15.70:0/1609637901 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 88) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f11f0004370 con 0x7f11f80c5c50 2023-10-08T06:53:29.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.194+0000 7f11effff640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f11d806af00 0x7f11d806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.194+0000 7f11effff640 1 -- 172.21.15.70:0/1609637901 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f11f0089200 con 0x7f11f80c5c50 2023-10-08T06:53:29.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.194+0000 7f11f6ffd640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f11d806af00 0x7f11d806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:29.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.195+0000 7f11f6ffd640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f11d806af00 0x7f11d806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f11e8033000 tx=0x7f11e802f000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:29.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.198+0000 7f11effff640 1 -- 172.21.15.70:0/1609637901 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11f0053dc0 con 0x7f11f80c5c50 2023-10-08T06:53:29.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.395+0000 7f11f7fff640 1 -- 172.21.15.70:0/1609637901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11f804e160 con 0x7f11f80c5c50 2023-10-08T06:53:29.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.397+0000 7f11effff640 1 -- 172.21.15.70:0/1609637901 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+18015 (secure 0 0 0) 0x7f11f00573e0 con 0x7f11f80c5c50 2023-10-08T06:53:29.399 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:29.399 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":173,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:53:28.771982+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-08T05:57:23.815675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-08T05:57:25.045292+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2061492414":"2023-10-09T06:53:26.036761+0000","172.21.15.181:6833/236882560":"2023-10-09T06:53:26.036761+0000","172.21.15.181:6832/236882560":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/23330666":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/599491381":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/2571920213":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/3947544788":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3693343940":"2023-10-09T05:57:41.519220+0000","172.21.15.181:6833/2068783299":"2023-10-09T05:57:41.519220+0000","172.21.15.181:6832/2068783299":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3070110051":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3604402742":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/1336064042":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3458973146":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/3875931700":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/3791109415":"2023-10-09T05:57:41.519220+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":20,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:53:29.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.399+0000 7f11f7fff640 1 -- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f11d806af00 msgr2=0x7f11d806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.399+0000 7f11f7fff640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f11d806af00 0x7f11d806d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f11e8033000 tx=0x7f11e802f000 comp rx=0 tx=0).stop 2023-10-08T06:53:29.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.399+0000 7f11f7fff640 1 -- 172.21.15.70:0/1609637901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11f80c5c50 msgr2=0x7f11f806cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.399+0000 7f11f7fff640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11f80c5c50 0x7f11f806cc50 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f11f80c5690 tx=0x7f11f000ec00 comp rx=0 tx=0).stop 2023-10-08T06:53:29.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.400+0000 7f11f7fff640 1 -- 172.21.15.70:0/1609637901 shutdown_connections 2023-10-08T06:53:29.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.400+0000 7f11f7fff640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f11d806af00 0x7f11d806d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.400+0000 7f11f7fff640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11f80c5c50 0x7f11f806cc50 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.400+0000 7f11f7fff640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11f80c7230 0x7f11f80c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.400+0000 7f11f7fff640 1 --2- 172.21.15.70:0/1609637901 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11e40039a0 0x7f11f80c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.400+0000 7f11f7fff640 1 -- 172.21.15.70:0/1609637901 >> 172.21.15.70:0/1609637901 conn(0x7f11f80cb0c0 msgr2=0x7f11f80ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:29.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.400+0000 7f11f7fff640 1 -- 172.21.15.70:0/1609637901 shutdown_connections 2023-10-08T06:53:29.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.400+0000 7f11f7fff640 1 -- 172.21.15.70:0/1609637901 wait complete. 2023-10-08T06:53:29.420 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T06:53:29.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.639+0000 7f87adb39640 1 Processor -- start 2023-10-08T06:53:29.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.639+0000 7f87adb39640 1 -- start start 2023-10-08T06:53:29.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.640+0000 7f87adb39640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87a815cd80 0x7f87a80b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.640+0000 7f87adb39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87a80b6730 0x7f87a80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.640+0000 7f87adb39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a815d160 con 0x7f87a80cdda0 2023-10-08T06:53:29.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.640+0000 7f87adb39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a80b7b90 con 0x7f87a80b6730 2023-10-08T06:53:29.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.640+0000 7f87adb39640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a80ce1c0 con 0x7f87a815cd80 2023-10-08T06:53:29.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.641+0000 7f87a6ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87a80b6730 0x7f87a80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:29.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.641+0000 7f87a6ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87a80b6730 0x7f87a80cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48450/0 (socket says 172.21.15.70:48450) 2023-10-08T06:53:29.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.641+0000 7f87a6ffd640 1 -- 172.21.15.70:0/245671210 learned_addr learned my addr 172.21.15.70:0/245671210 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:29.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.641+0000 7f87a77fe640 1 --2- 172.21.15.70:0/245671210 >> v2:172.21.15.181:3300/0 conn(0x7f87a815cd80 0x7f87a80b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:29.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.641+0000 7f87a6ffd640 1 -- 172.21.15.70:0/245671210 >> v2:172.21.15.181:3300/0 conn(0x7f87a815cd80 msgr2=0x7f87a80b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:29.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.641+0000 7f87a6ffd640 1 --2- 172.21.15.70:0/245671210 >> v2:172.21.15.181:3300/0 conn(0x7f87a815cd80 0x7f87a80b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.641+0000 7f87a6ffd640 1 -- 172.21.15.70:0/245671210 >> v1:172.21.15.181:6789/0 conn(0x7f87a80cdda0 legacy=0x7f87a80b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:29.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.641+0000 7f87a6ffd640 1 -- 172.21.15.70:0/245671210 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80b8020 con 0x7f87a80b6730 2023-10-08T06:53:29.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.641+0000 7f87a77fe640 1 --2- 172.21.15.70:0/245671210 >> v2:172.21.15.181:3300/0 conn(0x7f87a815cd80 0x7f87a80b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:29.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.642+0000 7f87a6ffd640 1 --2- 172.21.15.70:0/245671210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 0x7f87a80cd860 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f87a80b5710 tx=0x7f879002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82090282b2e5d226 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:29.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.643+0000 7f87a67fc640 1 -- 172.21.15.70:0/245671210 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790031870 con 0x7f87a80b6730 2023-10-08T06:53:29.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.643+0000 7f87a67fc640 1 -- 172.21.15.70:0/245671210 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87900319d0 con 0x7f87a80b6730 2023-10-08T06:53:29.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.643+0000 7f87a67fc640 1 -- 172.21.15.70:0/245671210 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790031d60 con 0x7f87a80b6730 2023-10-08T06:53:29.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.643+0000 7f87adb39640 1 -- 172.21.15.70:0/245671210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 msgr2=0x7f87a80cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.643+0000 7f87adb39640 1 --2- 172.21.15.70:0/245671210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 0x7f87a80cd860 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f87a80b5710 tx=0x7f879002b910 comp rx=0 tx=0).stop 2023-10-08T06:53:29.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.644+0000 7f87adb39640 1 -- 172.21.15.70:0/245671210 shutdown_connections 2023-10-08T06:53:29.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.644+0000 7f87adb39640 1 --2- 172.21.15.70:0/245671210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 0x7f87a80cd860 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.644+0000 7f87adb39640 1 --2- 172.21.15.70:0/245671210 >> v2:172.21.15.181:3300/0 conn(0x7f87a815cd80 0x7f87a80b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.644+0000 7f87adb39640 1 -- 172.21.15.70:0/245671210 >> 172.21.15.70:0/245671210 conn(0x7f87a806b5d0 msgr2=0x7f87a806b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:29.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.644+0000 7f87adb39640 1 -- 172.21.15.70:0/245671210 shutdown_connections 2023-10-08T06:53:29.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.644+0000 7f87adb39640 1 -- 172.21.15.70:0/245671210 wait complete. 2023-10-08T06:53:29.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.645+0000 7f87adb39640 1 Processor -- start 2023-10-08T06:53:29.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.646+0000 7f87adb39640 1 -- start start 2023-10-08T06:53:29.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.646+0000 7f87adb39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 0x7f87a80cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87adb39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87a80cdda0 0x7f87a80cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87adb39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87a815cd80 0x7f87a80cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87adb39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a815fcc0 con 0x7f87a80b6730 2023-10-08T06:53:29.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87adb39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80cab60 con 0x7f87a80cdda0 2023-10-08T06:53:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87adb39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80cae40 con 0x7f87a815cd80 2023-10-08T06:53:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 0x7f87a80cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 0x7f87a80cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48452/0 (socket says 172.21.15.70:48452) 2023-10-08T06:53:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a77fe640 1 -- 172.21.15.70:0/2116658174 learned_addr learned my addr 172.21.15.70:0/2116658174 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a6ffd640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87a80cdda0 0x7f87a80cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:29.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a7fff640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87a815cd80 0x7f87a80cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a77fe640 1 -- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87a815cd80 msgr2=0x7f87a80cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a77fe640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87a815cd80 0x7f87a80cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a77fe640 1 -- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87a80cdda0 msgr2=0x7f87a80cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a77fe640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87a80cdda0 0x7f87a80cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a77fe640 1 -- 172.21.15.70:0/2116658174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87900094d0 con 0x7f87a80b6730 2023-10-08T06:53:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.647+0000 7f87a7fff640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87a815cd80 0x7f87a80cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.648+0000 7f87a77fe640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 0x7f87a80cb920 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f879800ce00 tx=0x7f879800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.648+0000 7f879ffff640 1 -- 172.21.15.70:0/2116658174 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800f890 con 0x7f87a80b6730 2023-10-08T06:53:29.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.648+0000 7f879ffff640 1 -- 172.21.15.70:0/2116658174 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879800f9f0 con 0x7f87a80b6730 2023-10-08T06:53:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.648+0000 7f87adb39640 1 -- 172.21.15.70:0/2116658174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a80cb120 con 0x7f87a80b6730 2023-10-08T06:53:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.648+0000 7f879ffff640 1 -- 172.21.15.70:0/2116658174 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800fdc0 con 0x7f87a80b6730 2023-10-08T06:53:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.648+0000 7f87adb39640 1 -- 172.21.15.70:0/2116658174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a80cc8e0 con 0x7f87a80b6730 2023-10-08T06:53:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.649+0000 7f87adb39640 1 -- 172.21.15.70:0/2116658174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a80ccb30 con 0x7f87a80b6730 2023-10-08T06:53:29.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.650+0000 7f879ffff640 1 -- 172.21.15.70:0/2116658174 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 88) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f879800b040 con 0x7f87a80b6730 2023-10-08T06:53:29.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.652+0000 7f879ffff640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f878806af00 0x7f878806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:29.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.652+0000 7f879ffff640 1 -- 172.21.15.70:0/2116658174 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6877+0+0 (secure 0 0 0) 0x7f879808a690 con 0x7f87a80b6730 2023-10-08T06:53:29.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.652+0000 7f87a6ffd640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f878806af00 0x7f878806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:29.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.653+0000 7f87a6ffd640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f878806af00 0x7f878806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8790036000 tx=0x7f879003c040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:29.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.656+0000 7f879ffff640 1 -- 172.21.15.70:0/2116658174 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87a80ccb30 con 0x7f87a80b6730 2023-10-08T06:53:29.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.882+0000 7f87adb39640 1 -- 172.21.15.70:0/2116658174 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f87a80b7520 con 0x7f87a80b6730 2023-10-08T06:53:29.885 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T06:53:29.883+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T06:53:29.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.887+0000 7f879ffff640 1 -- 172.21.15.70:0/2116658174 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) v1 ==== 161+0+0 (secure 0 0 0) 0x7f87a80b7520 con 0x7f87a80b6730 2023-10-08T06:53:29.888 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T06:53:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.890+0000 7f87adb39640 1 -- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f878806af00 msgr2=0x7f878806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.890+0000 7f87adb39640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f878806af00 0x7f878806d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8790036000 tx=0x7f879003c040 comp rx=0 tx=0).stop 2023-10-08T06:53:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.890+0000 7f87adb39640 1 -- 172.21.15.70:0/2116658174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 msgr2=0x7f87a80cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.890+0000 7f87adb39640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 0x7f87a80cb920 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f879800ce00 tx=0x7f879800a650 comp rx=0 tx=0).stop 2023-10-08T06:53:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.891+0000 7f87adb39640 1 -- 172.21.15.70:0/2116658174 shutdown_connections 2023-10-08T06:53:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.891+0000 7f87adb39640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87a815cd80 0x7f87a80cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.891+0000 7f87adb39640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87a80cdda0 0x7f87a80cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.891+0000 7f87adb39640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f878806af00 0x7f878806d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.891+0000 7f87adb39640 1 --2- 172.21.15.70:0/2116658174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87a80b6730 0x7f87a80cb920 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:29.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.891+0000 7f87adb39640 1 -- 172.21.15.70:0/2116658174 >> 172.21.15.70:0/2116658174 conn(0x7f87a806b5d0 msgr2=0x7f87a80d1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:29.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.892+0000 7f87adb39640 1 -- 172.21.15.70:0/2116658174 shutdown_connections 2023-10-08T06:53:29.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:29.892+0000 7f87adb39640 1 -- 172.21.15.70:0/2116658174 wait complete. 2023-10-08T06:53:29.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T06:53:29.971 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T06:53:30.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.109+0000 7f80b4ad7640 1 Processor -- start 2023-10-08T06:53:30.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.109+0000 7f80b4ad7640 1 -- start start 2023-10-08T06:53:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.110+0000 7f80b4ad7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b0160660 0x7f80b0070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:30.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.110+0000 7f80b4ad7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b006e150 con 0x7f80b00cb620 2023-10-08T06:53:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.110+0000 7f80b4ad7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b006de70 con 0x7f80b00c9910 2023-10-08T06:53:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.110+0000 7f80b4ad7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00c8e30 con 0x7f80b0160660 2023-10-08T06:53:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.111+0000 7f80aed9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f80b00cb620 0x7f80b00b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37284/0 (socket says 172.21.15.70:37284) 2023-10-08T06:53:30.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.111+0000 7f80aed9d640 1 -- 172.21.15.70:0/1164782007 learned_addr learned my addr 172.21.15.70:0/1164782007 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:30.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.111+0000 7f80ae59c640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b0160660 0x7f80b0070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:30.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.111+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608367294 0 0) 0x7f80b006e150 con 0x7f80b00cb620 2023-10-08T06:53:30.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.111+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8090003520 con 0x7f80b00cb620 2023-10-08T06:53:30.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.111+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 317036643 0 0) 0x7f8090003520 con 0x7f80b00cb620 2023-10-08T06:53:30.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b0160660 msgr2=0x7f80b0070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:30.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b0160660 0x7f80b0070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 >> v1:172.21.15.181:6789/0 conn(0x7f80b00c9910 legacy=0x7f80b00caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:30.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b00c9110 con 0x7f80b00cb620 2023-10-08T06:53:30.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f80a80019f0 con 0x7f80b00cb620 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 >> v1:172.21.15.70:6789/0 conn(0x7f80b00cb620 legacy=0x7f80b00b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80900039a0 0x7f8090003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 0x7f8090004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b0160660 0x7f8090005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b00c8e30 con 0x7f80900047c0 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809000ba30 con 0x7f80900039a0 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809000bbf0 con 0x7f80b0160660 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80a7fff640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 0x7f8090004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80ae59c640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80900039a0 0x7f8090003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80aed9d640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b0160660 0x7f8090005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:30.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80a7fff640 1 -- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b0160660 msgr2=0x7f8090005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:30.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80a7fff640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b0160660 0x7f8090005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80a7fff640 1 -- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80900039a0 msgr2=0x7f8090003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:30.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80a7fff640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80900039a0 0x7f8090003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80a7fff640 1 -- 172.21.15.70:0/1164782007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8090003520 con 0x7f80900047c0 2023-10-08T06:53:30.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.112+0000 7f80aed9d640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b0160660 0x7f8090005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:30.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.114+0000 7f80a7fff640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 0x7f8090004c40 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8098001720 tx=0x7f80900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3602c1b285eb0fcc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:30.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.114+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980052d0 con 0x7f80900047c0 2023-10-08T06:53:30.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.114+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809802e050 con 0x7f80900047c0 2023-10-08T06:53:30.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.114+0000 7f80add9b640 1 -- 172.21.15.70:0/1164782007 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80980035f0 con 0x7f80900047c0 2023-10-08T06:53:30.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.114+0000 7f80b4ad7640 1 -- 172.21.15.70:0/1164782007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 msgr2=0x7f8090004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:30.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.114+0000 7f80b4ad7640 1 --2- 172.21.15.70:0/1164782007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 0x7f8090004c40 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8098001720 tx=0x7f80900057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.114+0000 7f80aed9d640 1 -- 172.21.15.70:0/1164782007 reap_dead start 2023-10-08T06:53:30.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.114+0000 7f80b4ad7640 1 -- 172.21.15.70:0/1164782007 shutdown_connections 2023-10-08T06:53:30.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.114+0000 7f80b4ad7640 1 -- 172.21.15.70:0/1164782007 >> 172.21.15.70:0/1164782007 conn(0x7f80b006a5d0 msgr2=0x7f80b00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:30.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.115+0000 7f80b4ad7640 1 -- 172.21.15.70:0/1164782007 shutdown_connections 2023-10-08T06:53:30.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.115+0000 7f80b4ad7640 1 -- 172.21.15.70:0/1164782007 wait complete. 2023-10-08T06:53:30.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.116+0000 7f80b4ad7640 1 Processor -- start 2023-10-08T06:53:30.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.116+0000 7f80b4ad7640 1 -- start start 2023-10-08T06:53:30.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.117+0000 7f80b4ad7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 0x7f80b0158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:30.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.117+0000 7f80b4ad7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b00cd7b0 0x7f80b01593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:30.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.117+0000 7f80b4ad7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b00cdc30 0x7f80b01598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:30.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.117+0000 7f80b4ad7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b0159120 con 0x7f80900047c0 2023-10-08T06:53:30.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.117+0000 7f80b4ad7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01f27b0 con 0x7f80b00cdc30 2023-10-08T06:53:30.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.117+0000 7f80b4ad7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01f37e0 con 0x7f80b00cd7b0 2023-10-08T06:53:30.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80ae59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 0x7f80b0158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:30.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80ae59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 0x7f80b0158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48500/0 (socket says 172.21.15.70:48500) 2023-10-08T06:53:30.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80ae59c640 1 -- 172.21.15.70:0/140195014 learned_addr learned my addr 172.21.15.70:0/140195014 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:30.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80aed9d640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b00cdc30 0x7f80b01598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:30.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80ae59c640 1 -- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b00cd7b0 msgr2=0x7f80b01593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:30.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80a7fff640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b00cd7b0 0x7f80b01593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:30.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80ae59c640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b00cd7b0 0x7f80b01593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80ae59c640 1 -- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b00cdc30 msgr2=0x7f80b01598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:30.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80ae59c640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b00cdc30 0x7f80b01598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80ae59c640 1 -- 172.21.15.70:0/140195014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b01f2a20 con 0x7f80900047c0 2023-10-08T06:53:30.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.118+0000 7f80ae59c640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 0x7f80b0158be0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f80a0011600 tx=0x7f80a0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.119+0000 7f80a6ffd640 1 -- 172.21.15.70:0/140195014 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000f920 con 0x7f80900047c0 2023-10-08T06:53:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.119+0000 7f80a6ffd640 1 -- 172.21.15.70:0/140195014 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a000fa80 con 0x7f80900047c0 2023-10-08T06:53:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.119+0000 7f80a6ffd640 1 -- 172.21.15.70:0/140195014 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000fdf0 con 0x7f80900047c0 2023-10-08T06:53:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.119+0000 7f80b4ad7640 1 -- 172.21.15.70:0/140195014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b01f2d00 con 0x7f80900047c0 2023-10-08T06:53:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.119+0000 7f80b4ad7640 1 -- 172.21.15.70:0/140195014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b01f32c0 con 0x7f80900047c0 2023-10-08T06:53:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.120+0000 7f80b4ad7640 1 -- 172.21.15.70:0/140195014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b004e160 con 0x7f80900047c0 2023-10-08T06:53:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.121+0000 7f80a6ffd640 1 -- 172.21.15.70:0/140195014 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 88) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f80a0015460 con 0x7f80900047c0 2023-10-08T06:53:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.122+0000 7f80a6ffd640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f809406af30 0x7f809406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.122+0000 7f80a6ffd640 1 -- 172.21.15.70:0/140195014 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f80a008a0a0 con 0x7f80900047c0 2023-10-08T06:53:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.122+0000 7f80a7fff640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f809406af30 0x7f809406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.123+0000 7f80a7fff640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f809406af30 0x7f809406d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8098033000 tx=0x7f8098009030 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:30.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.126+0000 7f80a6ffd640 1 -- 172.21.15.70:0/140195014 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80a00bcca0 con 0x7f80900047c0 2023-10-08T06:53:30.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.354+0000 7f80b4ad7640 1 -- 172.21.15.70:0/140195014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f80b00b06c0 con 0x7f80900047c0 2023-10-08T06:53:30.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:53:30.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:53:30.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.890+0000 7f80a6ffd640 1 -- 172.21.15.70:0/140195014 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) v1 ==== 104+0+0 (secure 0 0 0) 0x7f80a0054ba0 con 0x7f80900047c0 2023-10-08T06:53:30.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 -- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f809406af30 msgr2=0x7f809406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:30.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f809406af30 0x7f809406d3f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8098033000 tx=0x7f8098009030 comp rx=0 tx=0).stop 2023-10-08T06:53:30.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 -- 172.21.15.70:0/140195014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 msgr2=0x7f80b0158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:30.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 0x7f80b0158be0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f80a0011600 tx=0x7f80a0011ac0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 -- 172.21.15.70:0/140195014 shutdown_connections 2023-10-08T06:53:30.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f80b00cdc30 0x7f80b01598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80b00cd7b0 0x7f80b01593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f809406af30 0x7f809406d3f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 --2- 172.21.15.70:0/140195014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80900047c0 0x7f80b0158be0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:30.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 -- 172.21.15.70:0/140195014 >> 172.21.15.70:0/140195014 conn(0x7f80b006a5d0 msgr2=0x7f80b00701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:30.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 -- 172.21.15.70:0/140195014 shutdown_connections 2023-10-08T06:53:30.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:30.895+0000 7f80b4ad7640 1 -- 172.21.15.70:0/140195014 wait complete. 2023-10-08T06:53:30.925 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T06:53:31.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.147+0000 7f17c2ab4640 1 Processor -- start 2023-10-08T06:53:31.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.147+0000 7f17c2ab4640 1 -- start start 2023-10-08T06:53:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.148+0000 7f17c2ab4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f17bc0cb130 0x7f17bc0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.148+0000 7f17c2ab4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17bc0b1cc0 con 0x7f17bc0afbe0 2023-10-08T06:53:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.148+0000 7f17c2ab4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17bc0cbad0 con 0x7f17bc06dfc0 2023-10-08T06:53:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.148+0000 7f17c2ab4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17bc0b2620 con 0x7f17bc0cb130 2023-10-08T06:53:31.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.149+0000 7f17c0850640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f17bc0afbe0 0x7f17bc0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37288/0 (socket says 172.21.15.70:37288) 2023-10-08T06:53:31.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.149+0000 7f17c0850640 1 -- 172.21.15.70:0/646822255 learned_addr learned my addr 172.21.15.70:0/646822255 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.149+0000 7f17c1051640 1 --2- 172.21.15.70:0/646822255 >> v2:172.21.15.181:3300/0 conn(0x7f17bc0cb130 0x7f17bc0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.149+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232206318 0 0) 0x7f17bc0b1cc0 con 0x7f17bc0afbe0 2023-10-08T06:53:31.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.149+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17a4003520 con 0x7f17bc0afbe0 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1110981351 0 0) 0x7f17a4003520 con 0x7f17bc0afbe0 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 >> v2:172.21.15.181:3300/0 conn(0x7f17bc0cb130 msgr2=0x7f17bc0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 --2- 172.21.15.70:0/646822255 >> v2:172.21.15.181:3300/0 conn(0x7f17bc0cb130 0x7f17bc0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 >> v1:172.21.15.181:6789/0 conn(0x7f17bc06dfc0 legacy=0x7f17bc070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc0b2060 con 0x7f17bc0afbe0 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f17b8001670 con 0x7f17bc0afbe0 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 >> v1:172.21.15.70:6789/0 conn(0x7f17bc0afbe0 legacy=0x7f17bc0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 --2- 172.21.15.70:0/646822255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17a4003970 0x7f17a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 --2- 172.21.15.70:0/646822255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17a40047c0 0x7f17a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 --2- 172.21.15.70:0/646822255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0b2620 con 0x7f17a4005180 2023-10-08T06:53:31.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a400c020 con 0x7f17a40047c0 2023-10-08T06:53:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a400c1e0 con 0x7f17a4003970 2023-10-08T06:53:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17c1051640 1 --2- 172.21.15.70:0/646822255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.150+0000 7f17b3fff640 1 --2- 172.21.15.70:0/646822255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17a40047c0 0x7f17a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.151+0000 7f17c1051640 1 -- 172.21.15.70:0/646822255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17a4003970 msgr2=0x7f17a4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.151+0000 7f17c1051640 1 --2- 172.21.15.70:0/646822255 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17a4003970 0x7f17a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.151+0000 7f17c1051640 1 -- 172.21.15.70:0/646822255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17a40047c0 msgr2=0x7f17a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.151+0000 7f17c1051640 1 --2- 172.21.15.70:0/646822255 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17a40047c0 0x7f17a4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:31.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.151+0000 7f17c1051640 1 -- 172.21.15.70:0/646822255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a4003520 con 0x7f17a4005180 2023-10-08T06:53:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.151+0000 7f17c1051640 1 -- 172.21.15.70:0/646822255 reap_dead start 2023-10-08T06:53:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.152+0000 7f17c1051640 1 --2- 172.21.15.70:0/646822255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17a400b980 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009fc0 tx=0x7f17a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85d7ae280eb2d75 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.152+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b402cc60 con 0x7f17a4005180 2023-10-08T06:53:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.152+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b400f040 con 0x7f17a4005180 2023-10-08T06:53:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.152+0000 7f17b37fe640 1 -- 172.21.15.70:0/646822255 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4003990 con 0x7f17a4005180 2023-10-08T06:53:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.152+0000 7f17c2ab4640 1 -- 172.21.15.70:0/646822255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 msgr2=0x7f17a400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.152+0000 7f17c2ab4640 1 --2- 172.21.15.70:0/646822255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17a400b980 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009fc0 tx=0x7f17a40057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.153+0000 7f17c2ab4640 1 -- 172.21.15.70:0/646822255 shutdown_connections 2023-10-08T06:53:31.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.153+0000 7f17c2ab4640 1 --2- 172.21.15.70:0/646822255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17a400b980 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.153+0000 7f17c2ab4640 1 -- 172.21.15.70:0/646822255 >> 172.21.15.70:0/646822255 conn(0x7f17bc06a5d0 msgr2=0x7f17bc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.153+0000 7f17c2ab4640 1 -- 172.21.15.70:0/646822255 shutdown_connections 2023-10-08T06:53:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.153+0000 7f17c2ab4640 1 -- 172.21.15.70:0/646822255 wait complete. 2023-10-08T06:53:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.154+0000 7f17c2ab4640 1 Processor -- start 2023-10-08T06:53:31.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.155+0000 7f17c2ab4640 1 -- start start 2023-10-08T06:53:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.155+0000 7f17c2ab4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17bc1f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.155+0000 7f17c2ab4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc0c2250 0x7f17bc156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.155+0000 7f17c2ab4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17bc156c80 0x7f17bc0c3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.155+0000 7f17c2ab4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc1f4af0 con 0x7f17a4005180 2023-10-08T06:53:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.155+0000 7f17c2ab4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc1f3c30 con 0x7f17bc0c2250 2023-10-08T06:53:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.155+0000 7f17c2ab4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0cadd0 con 0x7f17bc156c80 2023-10-08T06:53:31.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.156+0000 7f17c0850640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17bc1f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.156+0000 7f17c0850640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17bc1f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48524/0 (socket says 172.21.15.70:48524) 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.156+0000 7f17c0850640 1 -- 172.21.15.70:0/1747447026 learned_addr learned my addr 172.21.15.70:0/1747447026 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.156+0000 7f17c0850640 1 -- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17bc156c80 msgr2=0x7f17bc0c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.156+0000 7f17c0850640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17bc156c80 0x7f17bc0c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.156+0000 7f17c0850640 1 -- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc0c2250 msgr2=0x7f17bc156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.156+0000 7f17b3fff640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc0c2250 0x7f17bc156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.156+0000 7f17c0850640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc0c2250 0x7f17bc156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.156+0000 7f17c0850640 1 -- 172.21.15.70:0/1747447026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc1f3e70 con 0x7f17a4005180 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.156+0000 7f17c0850640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17bc1f36f0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f17b800c760 tx=0x7f17b800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.157+0000 7f17b17fa640 1 -- 172.21.15.70:0/1747447026 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b80072d0 con 0x7f17a4005180 2023-10-08T06:53:31.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.157+0000 7f17b17fa640 1 -- 172.21.15.70:0/1747447026 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b8007430 con 0x7f17a4005180 2023-10-08T06:53:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.157+0000 7f17b17fa640 1 -- 172.21.15.70:0/1747447026 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b80047e0 con 0x7f17a4005180 2023-10-08T06:53:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.157+0000 7f17c2ab4640 1 -- 172.21.15.70:0/1747447026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17bc1f4150 con 0x7f17a4005180 2023-10-08T06:53:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.157+0000 7f17c2ab4640 1 -- 172.21.15.70:0/1747447026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17bc1f4710 con 0x7f17a4005180 2023-10-08T06:53:31.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.158+0000 7f17c2ab4640 1 -- 172.21.15.70:0/1747447026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17bc1f4c30 con 0x7f17a4005180 2023-10-08T06:53:31.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.159+0000 7f17b17fa640 1 -- 172.21.15.70:0/1747447026 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f17b8004940 con 0x7f17a4005180 2023-10-08T06:53:31.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.160+0000 7f17b17fa640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f17a806af90 0x7f17a806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:31.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.160+0000 7f17b17fa640 1 -- 172.21.15.70:0/1747447026 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f17b8089040 con 0x7f17a4005180 2023-10-08T06:53:31.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.161+0000 7f17b3fff640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f17a806af90 0x7f17a806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:31.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.162+0000 7f17b3fff640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f17a806af90 0x7f17a806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f17ac007fd0 tx=0x7f17ac009040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:31.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.165+0000 7f17b17fa640 1 -- 172.21.15.70:0/1747447026 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17bc1f4c30 con 0x7f17a4005180 2023-10-08T06:53:31.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:31.357+0000 7f17c2ab4640 1 -- 172.21.15.70:0/1747447026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f17bc1f4c30 con 0x7f17a4005180 2023-10-08T06:53:32.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.099+0000 7f17b17fa640 1 -- 172.21.15.70:0/1747447026 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v175) v1 ==== 181+0+0 (secure 0 0 0) 0x7f17bc1f4c30 con 0x7f17a4005180 2023-10-08T06:53:32.101 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T06:53:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.103+0000 7f17c2ab4640 1 -- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f17a806af90 msgr2=0x7f17a806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.103+0000 7f17c2ab4640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f17a806af90 0x7f17a806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f17ac007fd0 tx=0x7f17ac009040 comp rx=0 tx=0).stop 2023-10-08T06:53:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.103+0000 7f17c2ab4640 1 -- 172.21.15.70:0/1747447026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 msgr2=0x7f17bc1f36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:32.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.103+0000 7f17c2ab4640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17bc1f36f0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f17b800c760 tx=0x7f17b800cc20 comp rx=0 tx=0).stop 2023-10-08T06:53:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.104+0000 7f17c2ab4640 1 -- 172.21.15.70:0/1747447026 shutdown_connections 2023-10-08T06:53:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.104+0000 7f17c2ab4640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17bc156c80 0x7f17bc0c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.104+0000 7f17c2ab4640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17bc0c2250 0x7f17bc156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.104+0000 7f17c2ab4640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f17a806af90 0x7f17a806d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.104+0000 7f17c2ab4640 1 --2- 172.21.15.70:0/1747447026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17a4005180 0x7f17bc1f36f0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.104+0000 7f17c2ab4640 1 -- 172.21.15.70:0/1747447026 >> 172.21.15.70:0/1747447026 conn(0x7f17bc06a5d0 msgr2=0x7f17bc0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:32.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.104+0000 7f17c2ab4640 1 -- 172.21.15.70:0/1747447026 shutdown_connections 2023-10-08T06:53:32.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.105+0000 7f17c2ab4640 1 -- 172.21.15.70:0/1747447026 wait complete. 2023-10-08T06:53:32.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T06:53:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.334+0000 7f813312c640 1 Processor -- start 2023-10-08T06:53:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.335+0000 7f813312c640 1 -- start start 2023-10-08T06:53:32.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.336+0000 7f813312c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f812c0cd190 0x7f812c0ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:32.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.336+0000 7f813312c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f812c0ccb40 0x7f812c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:32.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.336+0000 7f813312c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f812c15cc90 0x7f812c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:32.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.336+0000 7f813312c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812c0ae300 con 0x7f812c15cc90 2023-10-08T06:53:32.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.336+0000 7f813312c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812c0ccfa0 con 0x7f812c0ccb40 2023-10-08T06:53:32.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.336+0000 7f813312c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0cdab0 con 0x7f812c0cd190 2023-10-08T06:53:32.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.336+0000 7f81316c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f812c15cc90 0x7f812c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.337+0000 7f81316c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f812c15cc90 0x7f812c15d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48530/0 (socket says 172.21.15.70:48530) 2023-10-08T06:53:32.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.337+0000 7f81316c9640 1 -- 172.21.15.70:0/2558198492 learned_addr learned my addr 172.21.15.70:0/2558198492 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.337+0000 7f812bfff640 1 --2- 172.21.15.70:0/2558198492 >> v2:172.21.15.181:3300/0 conn(0x7f812c0ccb40 0x7f812c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.337+0000 7f8130ec8640 1 --2- 172.21.15.70:0/2558198492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f812c0cd190 0x7f812c0ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.337+0000 7f81316c9640 1 -- 172.21.15.70:0/2558198492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f812c0cd190 msgr2=0x7f812c0ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.337+0000 7f81316c9640 1 --2- 172.21.15.70:0/2558198492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f812c0cd190 0x7f812c0ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.337+0000 7f81316c9640 1 -- 172.21.15.70:0/2558198492 >> v2:172.21.15.181:3300/0 conn(0x7f812c0ccb40 msgr2=0x7f812c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:32.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.337+0000 7f81316c9640 1 --2- 172.21.15.70:0/2558198492 >> v2:172.21.15.181:3300/0 conn(0x7f812c0ccb40 0x7f812c0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.337+0000 7f81316c9640 1 -- 172.21.15.70:0/2558198492 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c0cb560 con 0x7f812c15cc90 2023-10-08T06:53:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.337+0000 7f812bfff640 1 --2- 172.21.15.70:0/2558198492 >> v2:172.21.15.181:3300/0 conn(0x7f812c0ccb40 0x7f812c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.338+0000 7f81316c9640 1 --2- 172.21.15.70:0/2558198492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c15cc90 0x7f812c15d120 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f812c0665b0 tx=0x7f812402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156f35801a864ea3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.338+0000 7f812b7fe640 1 -- 172.21.15.70:0/2558198492 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124031740 con 0x7f812c15cc90 2023-10-08T06:53:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.338+0000 7f812b7fe640 1 -- 172.21.15.70:0/2558198492 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81240318a0 con 0x7f812c15cc90 2023-10-08T06:53:32.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.338+0000 7f812b7fe640 1 -- 172.21.15.70:0/2558198492 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124031c30 con 0x7f812c15cc90 2023-10-08T06:53:32.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.339+0000 7f813312c640 1 -- 172.21.15.70:0/2558198492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c15cc90 msgr2=0x7f812c15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:32.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.339+0000 7f813312c640 1 --2- 172.21.15.70:0/2558198492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c15cc90 0x7f812c15d120 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f812c0665b0 tx=0x7f812402b7e0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.339+0000 7f813312c640 1 -- 172.21.15.70:0/2558198492 shutdown_connections 2023-10-08T06:53:32.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.339+0000 7f813312c640 1 --2- 172.21.15.70:0/2558198492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c15cc90 0x7f812c15d120 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.339+0000 7f813312c640 1 --2- 172.21.15.70:0/2558198492 >> v2:172.21.15.181:3300/0 conn(0x7f812c0ccb40 0x7f812c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.339+0000 7f813312c640 1 --2- 172.21.15.70:0/2558198492 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f812c0cd190 0x7f812c0ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.339+0000 7f813312c640 1 -- 172.21.15.70:0/2558198492 >> 172.21.15.70:0/2558198492 conn(0x7f812c0b0af0 msgr2=0x7f812c0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:32.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.339+0000 7f813312c640 1 -- 172.21.15.70:0/2558198492 shutdown_connections 2023-10-08T06:53:32.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.339+0000 7f813312c640 1 -- 172.21.15.70:0/2558198492 wait complete. 2023-10-08T06:53:32.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.341+0000 7f813312c640 1 Processor -- start 2023-10-08T06:53:32.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.341+0000 7f813312c640 1 -- start start 2023-10-08T06:53:32.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.342+0000 7f813312c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0ccb40 0x7f812c145e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:32.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.342+0000 7f813312c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f812c0cd190 0x7f812c146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:32.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.342+0000 7f813312c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f812c15cc90 0x7f812c157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:32.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.342+0000 7f813312c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c1555e0 con 0x7f812c0ccb40 2023-10-08T06:53:32.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.342+0000 7f813312c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c1549a0 con 0x7f812c15cc90 2023-10-08T06:53:32.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.342+0000 7f813312c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c154c40 con 0x7f812c0cd190 2023-10-08T06:53:32.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.342+0000 7f8130ec8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0ccb40 0x7f812c145e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:32.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.342+0000 7f8130ec8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0ccb40 0x7f812c145e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48540/0 (socket says 172.21.15.70:48540) 2023-10-08T06:53:32.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.342+0000 7f8130ec8640 1 -- 172.21.15.70:0/494898163 learned_addr learned my addr 172.21.15.70:0/494898163 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:32.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f8130ec8640 1 -- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f812c0cd190 msgr2=0x7f812c146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:32.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f8130ec8640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f812c0cd190 0x7f812c146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f8130ec8640 1 -- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f812c15cc90 msgr2=0x7f812c157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:32.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f8130ec8640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f812c15cc90 0x7f812c157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:32.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f8130ec8640 1 -- 172.21.15.70:0/494898163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81240094d0 con 0x7f812c0ccb40 2023-10-08T06:53:32.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f8130ec8640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0ccb40 0x7f812c145e30 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f811c00c7f0 tx=0x7f811c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:32.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f81297fa640 1 -- 172.21.15.70:0/494898163 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c004220 con 0x7f812c0ccb40 2023-10-08T06:53:32.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f81297fa640 1 -- 172.21.15.70:0/494898163 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c004380 con 0x7f812c0ccb40 2023-10-08T06:53:32.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f813312c640 1 -- 172.21.15.70:0/494898163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c154f20 con 0x7f812c0ccb40 2023-10-08T06:53:32.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f81297fa640 1 -- 172.21.15.70:0/494898163 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c007840 con 0x7f812c0ccb40 2023-10-08T06:53:32.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.343+0000 7f813312c640 1 -- 172.21.15.70:0/494898163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c155a00 con 0x7f812c0ccb40 2023-10-08T06:53:32.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.344+0000 7f813312c640 1 -- 172.21.15.70:0/494898163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c06dda0 con 0x7f812c0ccb40 2023-10-08T06:53:32.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.345+0000 7f81297fa640 1 -- 172.21.15.70:0/494898163 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f811c007bb0 con 0x7f812c0ccb40 2023-10-08T06:53:32.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.347+0000 7f81297fa640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f811806af90 0x7f811806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:32.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.347+0000 7f81297fa640 1 -- 172.21.15.70:0/494898163 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f811c089a40 con 0x7f812c0ccb40 2023-10-08T06:53:32.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.347+0000 7f812bfff640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f811806af90 0x7f811806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:32.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.349+0000 7f812bfff640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f811806af90 0x7f811806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f812c06b2c0 tx=0x7f8120008370 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:32.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.351+0000 7f81297fa640 1 -- 172.21.15.70:0/494898163 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f812c06dda0 con 0x7f812c0ccb40 2023-10-08T06:53:32.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:32.539+0000 7f813312c640 1 -- 172.21.15.70:0/494898163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f812c06dda0 con 0x7f812c0ccb40 2023-10-08T06:53:33.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.110+0000 7f81297fa640 1 -- 172.21.15.70:0/494898163 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v176) v1 ==== 169+0+0 (secure 0 0 0) 0x7f812c06dda0 con 0x7f812c0ccb40 2023-10-08T06:53:33.112 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T06:53:33.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.115+0000 7f813312c640 1 -- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f811806af90 msgr2=0x7f811806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.115+0000 7f813312c640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f811806af90 0x7f811806d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f812c06b2c0 tx=0x7f8120008370 comp rx=0 tx=0).stop 2023-10-08T06:53:33.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.115+0000 7f813312c640 1 -- 172.21.15.70:0/494898163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0ccb40 msgr2=0x7f812c145e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.115+0000 7f813312c640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0ccb40 0x7f812c145e30 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f811c00c7f0 tx=0x7f811c00ccb0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.115+0000 7f813312c640 1 -- 172.21.15.70:0/494898163 shutdown_connections 2023-10-08T06:53:33.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.115+0000 7f813312c640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f812c15cc90 0x7f812c157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.115+0000 7f813312c640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f812c0cd190 0x7f812c146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.115+0000 7f813312c640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f811806af90 0x7f811806d450 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f812c06b2c0 tx=0x7f8120008370 comp rx=0 tx=0).stop 2023-10-08T06:53:33.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.115+0000 7f813312c640 1 --2- 172.21.15.70:0/494898163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f812c0ccb40 0x7f812c145e30 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.115+0000 7f813312c640 1 -- 172.21.15.70:0/494898163 >> 172.21.15.70:0/494898163 conn(0x7f812c0b0af0 msgr2=0x7f812c0aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:33.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.119+0000 7f813312c640 1 -- 172.21.15.70:0/494898163 shutdown_connections 2023-10-08T06:53:33.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.121+0000 7f813312c640 1 -- 172.21.15.70:0/494898163 wait complete. 2023-10-08T06:53:33.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T06:53:33.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.340+0000 7fc2223d0640 1 Processor -- start 2023-10-08T06:53:33.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.341+0000 7fc2223d0640 1 -- start start 2023-10-08T06:53:33.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.342+0000 7fc2223d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c160660 0x7fc21c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.342+0000 7fc2223d0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc21c0cb620 0x7fc21c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.342+0000 7fc2223d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c06e150 con 0x7fc21c0c9910 2023-10-08T06:53:33.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.342+0000 7fc2223d0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21c06de70 con 0x7fc21c0cb620 2023-10-08T06:53:33.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.342+0000 7fc2223d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0c8e30 con 0x7fc21c160660 2023-10-08T06:53:33.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.343+0000 7fc21b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc21c0c9910 0x7fc21c0caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37302/0 (socket says 172.21.15.70:37302) 2023-10-08T06:53:33.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.343+0000 7fc21b7fe640 1 -- 172.21.15.70:0/78619954 learned_addr learned my addr 172.21.15.70:0/78619954 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:33.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.343+0000 7fc22096d640 1 --2- 172.21.15.70:0/78619954 >> v2:172.21.15.70:3300/0 conn(0x7fc21c0cb620 0x7fc21c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.343+0000 7fc21bfff640 1 --2- 172.21.15.70:0/78619954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c160660 0x7fc21c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.343+0000 7fc21affd640 1 -- 172.21.15.70:0/78619954 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240913762 0 0) 0x7fc21c06e150 con 0x7fc21c0c9910 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.343+0000 7fc21affd640 1 -- 172.21.15.70:0/78619954 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc208003520 con 0x7fc21c0c9910 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.343+0000 7fc22096d640 1 -- 172.21.15.70:0/78619954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c160660 msgr2=0x7fc21c070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.343+0000 7fc22096d640 1 --2- 172.21.15.70:0/78619954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c160660 0x7fc21c070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.343+0000 7fc22096d640 1 -- 172.21.15.70:0/78619954 >> v1:172.21.15.70:6789/0 conn(0x7fc21c0c9910 legacy=0x7fc21c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.343+0000 7fc22096d640 1 -- 172.21.15.70:0/78619954 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0c9110 con 0x7fc21c0cb620 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.345+0000 7fc22096d640 1 --2- 172.21.15.70:0/78619954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c0cb620 0x7fc21c0b1990 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fc214009d90 tx=0x7fc21402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c00f07cf367fe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.345+0000 7fc21affd640 1 -- 172.21.15.70:0/78619954 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21402cc60 con 0x7fc21c0cb620 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.345+0000 7fc21affd640 1 -- 172.21.15.70:0/78619954 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21402cdc0 con 0x7fc21c0cb620 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.345+0000 7fc21affd640 1 -- 172.21.15.70:0/78619954 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214034660 con 0x7fc21c0cb620 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.345+0000 7fc2223d0640 1 -- 172.21.15.70:0/78619954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c0cb620 msgr2=0x7fc21c0b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.345+0000 7fc2223d0640 1 --2- 172.21.15.70:0/78619954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c0cb620 0x7fc21c0b1990 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fc214009d90 tx=0x7fc21402beb0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.346+0000 7fc2223d0640 1 -- 172.21.15.70:0/78619954 shutdown_connections 2023-10-08T06:53:33.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.346+0000 7fc2223d0640 1 --2- 172.21.15.70:0/78619954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c0cb620 0x7fc21c0b1990 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.346+0000 7fc2223d0640 1 --2- 172.21.15.70:0/78619954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c160660 0x7fc21c070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.346+0000 7fc2223d0640 1 -- 172.21.15.70:0/78619954 >> 172.21.15.70:0/78619954 conn(0x7fc21c06a5d0 msgr2=0x7fc21c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:33.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.346+0000 7fc2223d0640 1 -- 172.21.15.70:0/78619954 shutdown_connections 2023-10-08T06:53:33.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.346+0000 7fc2223d0640 1 -- 172.21.15.70:0/78619954 wait complete. 2023-10-08T06:53:33.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.347+0000 7fc2223d0640 1 Processor -- start 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.348+0000 7fc2223d0640 1 -- start start 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.348+0000 7fc2223d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c0c9910 0x7fc21c14fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.348+0000 7fc2223d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c0cb620 0x7fc21c0aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.348+0000 7fc2223d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c160660 0x7fc21c0cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.348+0000 7fc2223d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0adc40 con 0x7fc21c160660 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.348+0000 7fc2223d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0ade70 con 0x7fc21c0cb620 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.348+0000 7fc2223d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c0ae150 con 0x7fc21c0c9910 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.349+0000 7fc22096d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c160660 0x7fc21c0cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.349+0000 7fc22096d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c160660 0x7fc21c0cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48554/0 (socket says 172.21.15.70:48554) 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.349+0000 7fc22096d640 1 -- 172.21.15.70:0/347716360 learned_addr learned my addr 172.21.15.70:0/347716360 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:33.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.349+0000 7fc22096d640 1 -- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c0c9910 msgr2=0x7fc21c14fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.349+0000 7fc22096d640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c0c9910 0x7fc21c14fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.349+0000 7fc22096d640 1 -- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c0cb620 msgr2=0x7fc21c0aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.349+0000 7fc22096d640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c0cb620 0x7fc21c0aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.349+0000 7fc22096d640 1 -- 172.21.15.70:0/347716360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214009a70 con 0x7fc21c160660 2023-10-08T06:53:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.349+0000 7fc22096d640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c160660 0x7fc21c0cd110 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fc214009d90 tx=0x7fc214004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.350+0000 7fc218ff9640 1 -- 172.21.15.70:0/347716360 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214040650 con 0x7fc21c160660 2023-10-08T06:53:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.350+0000 7fc218ff9640 1 -- 172.21.15.70:0/347716360 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2140407b0 con 0x7fc21c160660 2023-10-08T06:53:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.350+0000 7fc218ff9640 1 -- 172.21.15.70:0/347716360 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214040b60 con 0x7fc21c160660 2023-10-08T06:53:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.350+0000 7fc2223d0640 1 -- 172.21.15.70:0/347716360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21c0ae430 con 0x7fc21c160660 2023-10-08T06:53:33.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.350+0000 7fc2223d0640 1 -- 172.21.15.70:0/347716360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c0ae9f0 con 0x7fc21c160660 2023-10-08T06:53:33.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.351+0000 7fc2223d0640 1 -- 172.21.15.70:0/347716360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c0ca6d0 con 0x7fc21c160660 2023-10-08T06:53:33.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.352+0000 7fc218ff9640 1 -- 172.21.15.70:0/347716360 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc214040cc0 con 0x7fc21c160660 2023-10-08T06:53:33.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.353+0000 7fc218ff9640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc1fc06af90 0x7fc1fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.354+0000 7fc218ff9640 1 -- 172.21.15.70:0/347716360 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fc2140ac6d0 con 0x7fc21c160660 2023-10-08T06:53:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.354+0000 7fc21bfff640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc1fc06af90 0x7fc1fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.355+0000 7fc21bfff640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc1fc06af90 0x7fc1fc06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc20c006fd0 tx=0x7fc20c008040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:33.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.358+0000 7fc218ff9640 1 -- 172.21.15.70:0/347716360 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2140774f0 con 0x7fc21c160660 2023-10-08T06:53:33.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.552+0000 7fc2223d0640 1 -- 172.21.15.70:0/347716360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc21c04e160 con 0x7fc21c160660 2023-10-08T06:53:33.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.557+0000 7fc218ff9640 1 -- 172.21.15.70:0/347716360 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc214009450 con 0x7fc21c160660 2023-10-08T06:53:33.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 -- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc1fc06af90 msgr2=0x7fc1fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc1fc06af90 0x7fc1fc06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc20c006fd0 tx=0x7fc20c008040 comp rx=0 tx=0).stop 2023-10-08T06:53:33.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 -- 172.21.15.70:0/347716360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c160660 msgr2=0x7fc21c0cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c160660 0x7fc21c0cd110 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fc214009d90 tx=0x7fc214004a40 comp rx=0 tx=0).stop 2023-10-08T06:53:33.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 -- 172.21.15.70:0/347716360 shutdown_connections 2023-10-08T06:53:33.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc1fc06af90 0x7fc1fc06d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc21c160660 0x7fc21c0cd110 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc21c0cb620 0x7fc21c0aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 --2- 172.21.15.70:0/347716360 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc21c0c9910 0x7fc21c14fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 -- 172.21.15.70:0/347716360 >> 172.21.15.70:0/347716360 conn(0x7fc21c06a5d0 msgr2=0x7fc21c073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 -- 172.21.15.70:0/347716360 shutdown_connections 2023-10-08T06:53:33.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.560+0000 7fc2223d0640 1 -- 172.21.15.70:0/347716360 wait complete. 2023-10-08T06:53:33.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T06:53:33.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.790+0000 7fb01a13c640 1 Processor -- start 2023-10-08T06:53:33.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.790+0000 7fb01a13c640 1 -- start start 2023-10-08T06:53:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.791+0000 7fb01a13c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01415cd80 0x7fb01415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.791+0000 7fb01a13c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0140d3800 con 0x7fb0140d06c0 2023-10-08T06:53:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.791+0000 7fb01a13c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0140ce370 con 0x7fb0140d0ec0 2023-10-08T06:53:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.791+0000 7fb01a13c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01415d160 con 0x7fb01415cd80 2023-10-08T06:53:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.792+0000 7fb018939640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb0140d06c0 0x7fb01415e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33712/0 (socket says 172.21.15.70:33712) 2023-10-08T06:53:33.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.792+0000 7fb018939640 1 -- 172.21.15.70:0/795652836 learned_addr learned my addr 172.21.15.70:0/795652836 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:33.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.792+0000 7fb01913a640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01415cd80 0x7fb01415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.792+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108869732 0 0) 0x7fb0140d3800 con 0x7fb0140d06c0 2023-10-08T06:53:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.792+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb000003520 con 0x7fb0140d06c0 2023-10-08T06:53:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.792+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3963670080 0 0) 0x7fb000003520 con 0x7fb0140d06c0 2023-10-08T06:53:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.792+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01415cd80 msgr2=0x7fb01415dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.792+0000 7fb00bfff640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01415cd80 0x7fb01415dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.792+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 >> v1:172.21.15.181:6789/0 conn(0x7fb0140d0ec0 legacy=0x7fb0140ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01415ecf0 con 0x7fb0140d06c0 2023-10-08T06:53:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fb004001670 con 0x7fb0140d06c0 2023-10-08T06:53:33.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 >> v1:172.21.15.70:6789/0 conn(0x7fb0140d06c0 legacy=0x7fb01415e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb00bfff640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0000039a0 0x7fb000003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb00bfff640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000047c0 0x7fb000004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb00bfff640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01415cd80 0x7fb000005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01415d160 con 0x7fb0000039a0 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00000bb10 con 0x7fb0000047c0 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00000bcd0 con 0x7fb01415cd80 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb01913a640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0000039a0 0x7fb000003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb01993b640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01415cd80 0x7fb000005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb01913a640 1 -- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01415cd80 msgr2=0x7fb000005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb01913a640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01415cd80 0x7fb000005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb01913a640 1 -- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000047c0 msgr2=0x7fb000004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb018939640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000047c0 0x7fb000004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb01913a640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000047c0 0x7fb000004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb01913a640 1 -- 172.21.15.70:0/795652836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000003520 con 0x7fb0000039a0 2023-10-08T06:53:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.793+0000 7fb018939640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000047c0 0x7fb000004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.794+0000 7fb01913a640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0000039a0 0x7fb000003dc0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fb010009ab0 tx=0x7fb0000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc98f6a64a7254de server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.795+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb010005990 con 0x7fb0000039a0 2023-10-08T06:53:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.795+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb010005af0 con 0x7fb0000039a0 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.795+0000 7fb00bfff640 1 -- 172.21.15.70:0/795652836 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb010005ea0 con 0x7fb0000039a0 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.795+0000 7fb01a13c640 1 -- 172.21.15.70:0/795652836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0000039a0 msgr2=0x7fb000003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.795+0000 7fb01a13c640 1 --2- 172.21.15.70:0/795652836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0000039a0 0x7fb000003dc0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fb010009ab0 tx=0x7fb0000057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.795+0000 7fb01993b640 1 -- 172.21.15.70:0/795652836 reap_dead start 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.795+0000 7fb01a13c640 1 -- 172.21.15.70:0/795652836 shutdown_connections 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.795+0000 7fb01a13c640 1 -- 172.21.15.70:0/795652836 >> 172.21.15.70:0/795652836 conn(0x7fb01409bc40 msgr2=0x7fb0140d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.795+0000 7fb01a13c640 1 -- 172.21.15.70:0/795652836 shutdown_connections 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.795+0000 7fb01a13c640 1 -- 172.21.15.70:0/795652836 wait complete. 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.797+0000 7fb01a13c640 1 Processor -- start 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.797+0000 7fb01a13c640 1 -- start start 2023-10-08T06:53:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.798+0000 7fb01a13c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000039a0 0x7fb0141e5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.798+0000 7fb01a13c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01406c2e0 0x7fb0141e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.798+0000 7fb01a13c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01406c760 0x7fb0141e4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.798+0000 7fb01a13c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0141e7750 con 0x7fb01406c760 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.798+0000 7fb01a13c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0141e53f0 con 0x7fb0000039a0 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.798+0000 7fb01a13c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0141e47e0 con 0x7fb01406c2e0 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01993b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01406c760 0x7fb0141e4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01993b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01406c760 0x7fb0141e4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44224/0 (socket says 172.21.15.70:44224) 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01993b640 1 -- 172.21.15.70:0/4128961282 learned_addr learned my addr 172.21.15.70:0/4128961282 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01913a640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000039a0 0x7fb0141e5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01993b640 1 -- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01406c2e0 msgr2=0x7fb0141e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01993b640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01406c2e0 0x7fb0141e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01993b640 1 -- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000039a0 msgr2=0x7fb0141e5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:33.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01993b640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000039a0 0x7fb0141e5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:33.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01993b640 1 -- 172.21.15.70:0/4128961282 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0141e4ac0 con 0x7fb01406c760 2023-10-08T06:53:33.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01913a640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000039a0 0x7fb0141e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:33.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01993b640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01406c760 0x7fb0141e4110 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb01408cff0 tx=0x7fb00c00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:33.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb009ffb640 1 -- 172.21.15.70:0/4128961282 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c003b70 con 0x7fb01406c760 2023-10-08T06:53:33.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb009ffb640 1 -- 172.21.15.70:0/4128961282 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c003cd0 con 0x7fb01406c760 2023-10-08T06:53:33.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.799+0000 7fb01a13c640 1 -- 172.21.15.70:0/4128961282 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0141e4da0 con 0x7fb01406c760 2023-10-08T06:53:33.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.800+0000 7fb009ffb640 1 -- 172.21.15.70:0/4128961282 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0052f0 con 0x7fb01406c760 2023-10-08T06:53:33.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.800+0000 7fb01a13c640 1 -- 172.21.15.70:0/4128961282 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0141e5dc0 con 0x7fb01406c760 2023-10-08T06:53:33.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.801+0000 7fb01a13c640 1 -- 172.21.15.70:0/4128961282 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0141e5580 con 0x7fb01406c760 2023-10-08T06:53:33.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.802+0000 7fb009ffb640 1 -- 172.21.15.70:0/4128961282 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb00c005450 con 0x7fb01406c760 2023-10-08T06:53:33.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.803+0000 7fb009ffb640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7faff806afc0 0x7faff806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:33.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.803+0000 7fb009ffb640 1 -- 172.21.15.70:0/4128961282 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fb00c088f50 con 0x7fb01406c760 2023-10-08T06:53:33.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.803+0000 7fb01913a640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7faff806afc0 0x7faff806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:33.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.805+0000 7fb01913a640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7faff806afc0 0x7faff806d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb010009ab0 tx=0x7fb010030000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:33.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.807+0000 7fb009ffb640 1 -- 172.21.15.70:0/4128961282 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb00c053c80 con 0x7fb01406c760 2023-10-08T06:53:33.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:33.995+0000 7fb01a13c640 1 -- 172.21.15.70:0/4128961282 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fb014074150 con 0x7fb01406c760 2023-10-08T06:53:34.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.001+0000 7fb009ffb640 1 -- 172.21.15.70:0/4128961282 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v56) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb00c0572a0 con 0x7fb01406c760 2023-10-08T06:53:34.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.004+0000 7fb01a13c640 1 -- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7faff806afc0 msgr2=0x7faff806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.004+0000 7fb01a13c640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7faff806afc0 0x7faff806d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb010009ab0 tx=0x7fb010030000 comp rx=0 tx=0).stop 2023-10-08T06:53:34.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.004+0000 7fb01a13c640 1 -- 172.21.15.70:0/4128961282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01406c760 msgr2=0x7fb0141e4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.004+0000 7fb01a13c640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01406c760 0x7fb0141e4110 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb01408cff0 tx=0x7fb00c00d9f0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.005+0000 7fb01a13c640 1 -- 172.21.15.70:0/4128961282 shutdown_connections 2023-10-08T06:53:34.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.005+0000 7fb01a13c640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7faff806afc0 0x7faff806d480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.005+0000 7fb01a13c640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01406c760 0x7fb0141e4110 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.005+0000 7fb01a13c640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb01406c2e0 0x7fb0141e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.005+0000 7fb01a13c640 1 --2- 172.21.15.70:0/4128961282 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb0000039a0 0x7fb0141e5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.005+0000 7fb01a13c640 1 -- 172.21.15.70:0/4128961282 >> 172.21.15.70:0/4128961282 conn(0x7fb01409bc40 msgr2=0x7fb01415bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:34.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.005+0000 7fb01a13c640 1 -- 172.21.15.70:0/4128961282 shutdown_connections 2023-10-08T06:53:34.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.005+0000 7fb01a13c640 1 -- 172.21.15.70:0/4128961282 wait complete. 2023-10-08T06:53:34.025 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T06:53:34.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.238+0000 7f146e4bb640 1 Processor -- start 2023-10-08T06:53:34.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.239+0000 7f146e4bb640 1 -- start start 2023-10-08T06:53:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.240+0000 7f146e4bb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f146815a820 0x7f14680aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.240+0000 7f146e4bb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14680cab60 con 0x7f146815ac00 2023-10-08T06:53:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.240+0000 7f146e4bb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14680cb960 con 0x7f14680ca680 2023-10-08T06:53:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.240+0000 7f146e4bb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14680cbdf0 con 0x7f146815a820 2023-10-08T06:53:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.240+0000 7f146dcba640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f146815ac00 0x7f14680c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33718/0 (socket says 172.21.15.70:33718) 2023-10-08T06:53:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.240+0000 7f146dcba640 1 -- 172.21.15.70:0/3553907042 learned_addr learned my addr 172.21.15.70:0/3553907042 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f146d4b9640 1 --2- 172.21.15.70:0/3553907042 >> v2:172.21.15.181:3300/0 conn(0x7f146815a820 0x7f14680aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940953133 0 0) 0x7f14680cab60 con 0x7f146815ac00 2023-10-08T06:53:34.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1458003520 con 0x7f146815ac00 2023-10-08T06:53:34.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726989702 0 0) 0x7f1458003520 con 0x7f146815ac00 2023-10-08T06:53:34.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 >> v2:172.21.15.181:3300/0 conn(0x7f146815a820 msgr2=0x7f14680aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:34.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 --2- 172.21.15.70:0/3553907042 >> v2:172.21.15.181:3300/0 conn(0x7f146815a820 0x7f14680aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 >> v1:172.21.15.181:6789/0 conn(0x7f14680ca680 legacy=0x7f14680c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680c9f20 con 0x7f146815ac00 2023-10-08T06:53:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1460001670 con 0x7f146815ac00 2023-10-08T06:53:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 >> v1:172.21.15.70:6789/0 conn(0x7f146815ac00 legacy=0x7f14680c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 --2- 172.21.15.70:0/3553907042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f1458003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 --2- 172.21.15.70:0/3553907042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14580047c0 0x7f1458004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 --2- 172.21.15.70:0/3553907042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1458005180 0x7f145800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680cbdf0 con 0x7f1458003970 2023-10-08T06:53:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145800c120 con 0x7f14580047c0 2023-10-08T06:53:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.241+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145800c2e0 con 0x7f1458005180 2023-10-08T06:53:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.242+0000 7f146d4b9640 1 --2- 172.21.15.70:0/3553907042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f1458003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.242+0000 7f146ccb8640 1 --2- 172.21.15.70:0/3553907042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14580047c0 0x7f1458004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.242+0000 7f146d4b9640 1 -- 172.21.15.70:0/3553907042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1458005180 msgr2=0x7f145800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:34.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.242+0000 7f146d4b9640 1 --2- 172.21.15.70:0/3553907042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1458005180 0x7f145800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.242+0000 7f146d4b9640 1 -- 172.21.15.70:0/3553907042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14580047c0 msgr2=0x7f1458004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:34.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.242+0000 7f146d4b9640 1 --2- 172.21.15.70:0/3553907042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14580047c0 0x7f1458004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.242+0000 7f146d4b9640 1 -- 172.21.15.70:0/3553907042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1458003520 con 0x7f1458003970 2023-10-08T06:53:34.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.242+0000 7f146dcba640 1 -- 172.21.15.70:0/3553907042 reap_dead start 2023-10-08T06:53:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.243+0000 7f146d4b9640 1 --2- 172.21.15.70:0/3553907042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f1458003d90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f145c009fc0 tx=0x7f14580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea82dfa54249d761 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.244+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c02cc20 con 0x7f1458003970 2023-10-08T06:53:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.244+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c00f040 con 0x7f1458003970 2023-10-08T06:53:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.244+0000 7f1467fff640 1 -- 172.21.15.70:0/3553907042 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c034440 con 0x7f1458003970 2023-10-08T06:53:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.244+0000 7f146e4bb640 1 -- 172.21.15.70:0/3553907042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 msgr2=0x7f1458003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.244+0000 7f146e4bb640 1 --2- 172.21.15.70:0/3553907042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f1458003d90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f145c009fc0 tx=0x7f14580057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.244+0000 7f146e4bb640 1 -- 172.21.15.70:0/3553907042 shutdown_connections 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.244+0000 7f146e4bb640 1 --2- 172.21.15.70:0/3553907042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f1458003d90 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.244+0000 7f146e4bb640 1 -- 172.21.15.70:0/3553907042 >> 172.21.15.70:0/3553907042 conn(0x7f146806c050 msgr2=0x7f146806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.244+0000 7f146e4bb640 1 -- 172.21.15.70:0/3553907042 shutdown_connections 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.244+0000 7f146e4bb640 1 -- 172.21.15.70:0/3553907042 wait complete. 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.246+0000 7f146e4bb640 1 Processor -- start 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.246+0000 7f146e4bb640 1 -- start start 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.247+0000 7f146e4bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f14680c2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.247+0000 7f146e4bb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14681f0360 0x7f14680afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.247+0000 7f146e4bb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14680b0100 0x7f146814f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.247+0000 7f146e4bb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680c5fb0 con 0x7f1458003970 2023-10-08T06:53:34.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.247+0000 7f146e4bb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680c6290 con 0x7f14680b0100 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.247+0000 7f146e4bb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14680c6570 con 0x7f14681f0360 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146d4b9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f14680c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146d4b9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f14680c2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44246/0 (socket says 172.21.15.70:44246) 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146d4b9640 1 -- 172.21.15.70:0/2598345571 learned_addr learned my addr 172.21.15.70:0/2598345571 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146dcba640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14680b0100 0x7f146814f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146ccb8640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14681f0360 0x7f14680afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146d4b9640 1 -- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14681f0360 msgr2=0x7f14680afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146d4b9640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14681f0360 0x7f14680afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146d4b9640 1 -- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14680b0100 msgr2=0x7f146814f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146d4b9640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14680b0100 0x7f146814f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146d4b9640 1 -- 172.21.15.70:0/2598345571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680c6850 con 0x7f1458003970 2023-10-08T06:53:34.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146ccb8640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14681f0360 0x7f14680afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.248+0000 7f146d4b9640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f14680c2290 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f145c00e230 tx=0x7f145c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.249+0000 7f1465ffb640 1 -- 172.21.15.70:0/2598345571 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c034c50 con 0x7f1458003970 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.249+0000 7f1465ffb640 1 -- 172.21.15.70:0/2598345571 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c034dd0 con 0x7f1458003970 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.249+0000 7f1465ffb640 1 -- 172.21.15.70:0/2598345571 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c003bd0 con 0x7f1458003970 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.249+0000 7f146e4bb640 1 -- 172.21.15.70:0/2598345571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14680c6b30 con 0x7f1458003970 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.249+0000 7f146e4bb640 1 -- 172.21.15.70:0/2598345571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14680c27d0 con 0x7f1458003970 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.250+0000 7f146e4bb640 1 -- 172.21.15.70:0/2598345571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1468074150 con 0x7f1458003970 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.251+0000 7f1465ffb640 1 -- 172.21.15.70:0/2598345571 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f145c040450 con 0x7f1458003970 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.252+0000 7f1465ffb640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f144c06afc0 0x7f144c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.252+0000 7f1465ffb640 1 -- 172.21.15.70:0/2598345571 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f145c0ac0a0 con 0x7f1458003970 2023-10-08T06:53:34.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.253+0000 7f146ccb8640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f144c06afc0 0x7f144c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.254+0000 7f146ccb8640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f144c06afc0 0x7f144c06d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1454000c00 tx=0x7f1454009040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:34.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.256+0000 7f1465ffb640 1 -- 172.21.15.70:0/2598345571 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f145c076e30 con 0x7f1458003970 2023-10-08T06:53:34.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.455+0000 7f146e4bb640 1 -- 172.21.15.70:0/2598345571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f146815b8d0 con 0x7f1458003970 2023-10-08T06:53:34.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.462+0000 7f1465ffb640 1 -- 172.21.15.70:0/2598345571 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v57) v1 ==== 120+0+0 (secure 0 0 0) 0x7f145c07a450 con 0x7f1458003970 2023-10-08T06:53:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 -- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f144c06afc0 msgr2=0x7f144c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f144c06afc0 0x7f144c06d480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1454000c00 tx=0x7f1454009040 comp rx=0 tx=0).stop 2023-10-08T06:53:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 -- 172.21.15.70:0/2598345571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 msgr2=0x7f14680c2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f14680c2290 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f145c00e230 tx=0x7f145c009750 comp rx=0 tx=0).stop 2023-10-08T06:53:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 -- 172.21.15.70:0/2598345571 shutdown_connections 2023-10-08T06:53:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14680b0100 0x7f146814f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14681f0360 0x7f14680afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f144c06afc0 0x7f144c06d480 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 --2- 172.21.15.70:0/2598345571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458003970 0x7f14680c2290 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 -- 172.21.15.70:0/2598345571 >> 172.21.15.70:0/2598345571 conn(0x7f146806c050 msgr2=0x7f1468158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:34.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 -- 172.21.15.70:0/2598345571 shutdown_connections 2023-10-08T06:53:34.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.465+0000 7f146e4bb640 1 -- 172.21.15.70:0/2598345571 wait complete. 2023-10-08T06:53:34.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-08T06:53:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.694+0000 7fe4c6d90640 1 Processor -- start 2023-10-08T06:53:34.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.695+0000 7fe4c6d90640 1 -- start start 2023-10-08T06:53:34.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.696+0000 7fe4c6d90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4c00c90b0 0x7fe4c00cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.696+0000 7fe4c6d90640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4c00cb3b0 0x7fe4c015e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.696+0000 7fe4c6d90640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c015b2b0 con 0x7fe4c015ab40 2023-10-08T06:53:34.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.696+0000 7fe4c6d90640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c015bf30 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.696+0000 7fe4c6d90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00cb840 con 0x7fe4c00c90b0 2023-10-08T06:53:34.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.697+0000 7fe4c4b2c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe4c015ab40 0x7fe4c00ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33730/0 (socket says 172.21.15.70:33730) 2023-10-08T06:53:34.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.697+0000 7fe4c4b2c640 1 -- 172.21.15.70:0/1760141549 learned_addr learned my addr 172.21.15.70:0/1760141549 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:34.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.697+0000 7fe4c532d640 1 --2- 172.21.15.70:0/1760141549 >> v2:172.21.15.70:3300/0 conn(0x7fe4c00cb3b0 0x7fe4c015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.697+0000 7fe4bffff640 1 --2- 172.21.15.70:0/1760141549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4c00c90b0 0x7fe4c00cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.697+0000 7fe4bf7fe640 1 -- 172.21.15.70:0/1760141549 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469383622 0 0) 0x7fe4c015b2b0 con 0x7fe4c015ab40 2023-10-08T06:53:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.697+0000 7fe4bf7fe640 1 -- 172.21.15.70:0/1760141549 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4b0003520 con 0x7fe4c015ab40 2023-10-08T06:53:34.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.697+0000 7fe4c532d640 1 -- 172.21.15.70:0/1760141549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4c00c90b0 msgr2=0x7fe4c00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.697+0000 7fe4c532d640 1 --2- 172.21.15.70:0/1760141549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4c00c90b0 0x7fe4c00cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.697+0000 7fe4c532d640 1 -- 172.21.15.70:0/1760141549 >> v1:172.21.15.70:6789/0 conn(0x7fe4c015ab40 legacy=0x7fe4c00ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.697+0000 7fe4c532d640 1 -- 172.21.15.70:0/1760141549 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c015b590 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.698+0000 7fe4c532d640 1 --2- 172.21.15.70:0/1760141549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 0x7fe4c015e790 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009d90 tx=0x7fe4b802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f028bb3c4399dd78 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4bf7fe640 1 -- 172.21.15.70:0/1760141549 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b802cc50 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4bf7fe640 1 -- 172.21.15.70:0/1760141549 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b802cdb0 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4bf7fe640 1 -- 172.21.15.70:0/1760141549 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8034660 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1760141549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 msgr2=0x7fe4c015e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4c6d90640 1 --2- 172.21.15.70:0/1760141549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 0x7fe4c015e790 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009d90 tx=0x7fe4b802bea0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1760141549 shutdown_connections 2023-10-08T06:53:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4c6d90640 1 --2- 172.21.15.70:0/1760141549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 0x7fe4c015e790 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4c6d90640 1 --2- 172.21.15.70:0/1760141549 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4c00c90b0 0x7fe4c00cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1760141549 >> 172.21.15.70:0/1760141549 conn(0x7fe4c006a380 msgr2=0x7fe4c00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:34.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1760141549 shutdown_connections 2023-10-08T06:53:34.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.699+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1760141549 wait complete. 2023-10-08T06:53:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.701+0000 7fe4c6d90640 1 Processor -- start 2023-10-08T06:53:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.701+0000 7fe4c6d90640 1 -- start start 2023-10-08T06:53:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.702+0000 7fe4c6d90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4c00c90b0 0x7fe4c0154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.702+0000 7fe4c6d90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 0x7fe4c00b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.702+0000 7fe4c6d90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4c015ab40 0x7fe4c00ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.702+0000 7fe4c6d90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c0158310 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.702+0000 7fe4c6d90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00adf90 con 0x7fe4c00c90b0 2023-10-08T06:53:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.702+0000 7fe4c6d90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c0158560 con 0x7fe4c015ab40 2023-10-08T06:53:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.703+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 0x7fe4c00b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.703+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 0x7fe4c00b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44258/0 (socket says 172.21.15.70:44258) 2023-10-08T06:53:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.703+0000 7fe4bffff640 1 -- 172.21.15.70:0/1995766777 learned_addr learned my addr 172.21.15.70:0/1995766777 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.703+0000 7fe4c4b2c640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4c00c90b0 0x7fe4c0154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.703+0000 7fe4bffff640 1 -- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4c015ab40 msgr2=0x7fe4c00ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.703+0000 7fe4bffff640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4c015ab40 0x7fe4c00ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.703+0000 7fe4bffff640 1 -- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4c00c90b0 msgr2=0x7fe4c0154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.703+0000 7fe4bffff640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4c00c90b0 0x7fe4c0154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.703+0000 7fe4bffff640 1 -- 172.21.15.70:0/1995766777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8009a70 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.703+0000 7fe4bffff640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 0x7fe4c00b21f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00d800 tx=0x7fe4ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.704+0000 7fe4bd7fa640 1 -- 172.21.15.70:0/1995766777 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac00b6c0 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.704+0000 7fe4bd7fa640 1 -- 172.21.15.70:0/1995766777 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac00b820 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.704+0000 7fe4bd7fa640 1 -- 172.21.15.70:0/1995766777 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac00bbb0 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.704+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1995766777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c0158840 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.704+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1995766777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c00b2730 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.705+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1995766777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c0067490 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.706+0000 7fe4bd7fa640 1 -- 172.21.15.70:0/1995766777 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe4ac00bd10 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.707+0000 7fe4bd7fa640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe4a406af90 0x7fe4a406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:34.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.707+0000 7fe4bd7fa640 1 -- 172.21.15.70:0/1995766777 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fe4ac089f40 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.708+0000 7fe4c4b2c640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe4a406af90 0x7fe4a406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:34.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.709+0000 7fe4c4b2c640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe4a406af90 0x7fe4a406d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40011e0 tx=0x7fe4b4009040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:34.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.711+0000 7fe4bd7fa640 1 -- 172.21.15.70:0/1995766777 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4ac054cd0 con 0x7fe4c00cb3b0 2023-10-08T06:53:34.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:34.973+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1995766777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7fe4c00b0ef0 con 0x7fe4c00cb3b0 2023-10-08T06:53:35.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.116+0000 7fe4bd7fa640 1 -- 172.21.15.70:0/1995766777 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2544) v1 ==== 178+0+0 (secure 0 0 0) 0x7fe4ac0582f0 con 0x7fe4c00cb3b0 2023-10-08T06:53:35.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe4a406af90 msgr2=0x7fe4a406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe4a406af90 0x7fe4a406d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40011e0 tx=0x7fe4b4009040 comp rx=0 tx=0).stop 2023-10-08T06:53:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1995766777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 msgr2=0x7fe4c00b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 0x7fe4c00b21f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00d800 tx=0x7fe4ac00dcc0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1995766777 shutdown_connections 2023-10-08T06:53:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4c015ab40 0x7fe4c00ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe4a406af90 0x7fe4a406d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4c00cb3b0 0x7fe4c00b21f0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 --2- 172.21.15.70:0/1995766777 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4c00c90b0 0x7fe4c0154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1995766777 >> 172.21.15.70:0/1995766777 conn(0x7fe4c006a380 msgr2=0x7fe4c006ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1995766777 shutdown_connections 2023-10-08T06:53:35.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.121+0000 7fe4c6d90640 1 -- 172.21.15.70:0/1995766777 wait complete. 2023-10-08T06:53:35.142 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-10-08T06:53:35.143 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-10-08T06:53:35.143 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T06:53:35.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:53:35.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.362+0000 7f5f10ae5640 1 Processor -- start 2023-10-08T06:53:35.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.362+0000 7f5f10ae5640 1 -- start start 2023-10-08T06:53:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.364+0000 7f5f10ae5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f0c15ab40 0x7f5f0c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.364+0000 7f5f10ae5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0c15b2b0 con 0x7f5f0c0c90b0 2023-10-08T06:53:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.364+0000 7f5f10ae5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0c15b590 con 0x7f5f0c0cb580 2023-10-08T06:53:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.364+0000 7f5f10ae5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0c15bf30 con 0x7f5f0c15ab40 2023-10-08T06:53:35.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.364+0000 7f5f03fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5f0c0c90b0 0x7f5f0c0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33732/0 (socket says 172.21.15.70:33732) 2023-10-08T06:53:35.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.364+0000 7f5f03fff640 1 -- 172.21.15.70:0/3617703864 learned_addr learned my addr 172.21.15.70:0/3617703864 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:35.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.364+0000 7f5f0a59c640 1 --2- 172.21.15.70:0/3617703864 >> v2:172.21.15.181:3300/0 conn(0x7f5f0c15ab40 0x7f5f0c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.364+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428334984 0 0) 0x7f5f0c15b2b0 con 0x7f5f0c0c90b0 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ef8003520 con 0x7f5f0c0c90b0 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 166194625 0 0) 0x7f5ef8003520 con 0x7f5f0c0c90b0 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 >> v2:172.21.15.181:3300/0 conn(0x7f5f0c15ab40 msgr2=0x7f5f0c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 --2- 172.21.15.70:0/3617703864 >> v2:172.21.15.181:3300/0 conn(0x7f5f0c15ab40 0x7f5f0c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 >> v1:172.21.15.181:6789/0 conn(0x7f5f0c0cb580 legacy=0x7f5f0c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c15b870 con 0x7f5f0c0c90b0 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5ef4001670 con 0x7f5f0c0c90b0 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 >> v1:172.21.15.70:6789/0 conn(0x7f5f0c0c90b0 legacy=0x7f5f0c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039c0 0x7f5ef8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ef80047c0 0x7f5ef8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ef8005180 0x7f5ef800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c15bf30 con 0x7f5ef80039c0 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef800c050 con 0x7f5ef8005180 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef800c210 con 0x7f5ef80047c0 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.365+0000 7f5f0a59c640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039c0 0x7f5ef8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.366+0000 7f5f03fff640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ef80047c0 0x7f5ef8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.366+0000 7f5f0ad9d640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ef8005180 0x7f5ef800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.366+0000 7f5f0a59c640 1 -- 172.21.15.70:0/3617703864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ef80047c0 msgr2=0x7f5ef8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.366+0000 7f5f0a59c640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ef80047c0 0x7f5ef8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.366+0000 7f5f0a59c640 1 -- 172.21.15.70:0/3617703864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ef8005180 msgr2=0x7f5ef800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.366+0000 7f5f0a59c640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5ef8005180 0x7f5ef800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.366+0000 7f5f0a59c640 1 -- 172.21.15.70:0/3617703864 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef8003520 con 0x7f5ef80039c0 2023-10-08T06:53:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.366+0000 7f5f0ad9d640 1 -- 172.21.15.70:0/3617703864 reap_dead start 2023-10-08T06:53:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.367+0000 7f5f0a59c640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039c0 0x7f5ef8003dc0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009ac0 tx=0x7f5ef80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e940eb8ea551c416 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.367+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc02c6b0 con 0x7f5ef80039c0 2023-10-08T06:53:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.367+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5efc02c810 con 0x7f5ef80039c0 2023-10-08T06:53:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.367+0000 7f5f09d9b640 1 -- 172.21.15.70:0/3617703864 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc02cba0 con 0x7f5ef80039c0 2023-10-08T06:53:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.368+0000 7f5f10ae5640 1 -- 172.21.15.70:0/3617703864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039c0 msgr2=0x7f5ef8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.368+0000 7f5f10ae5640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039c0 0x7f5ef8003dc0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009ac0 tx=0x7f5ef80057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.368+0000 7f5f10ae5640 1 -- 172.21.15.70:0/3617703864 shutdown_connections 2023-10-08T06:53:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.368+0000 7f5f10ae5640 1 --2- 172.21.15.70:0/3617703864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039c0 0x7f5ef8003dc0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.368+0000 7f5f10ae5640 1 -- 172.21.15.70:0/3617703864 >> 172.21.15.70:0/3617703864 conn(0x7f5f0c06a380 msgr2=0x7f5f0c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.368+0000 7f5f10ae5640 1 -- 172.21.15.70:0/3617703864 shutdown_connections 2023-10-08T06:53:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.368+0000 7f5f10ae5640 1 -- 172.21.15.70:0/3617703864 wait complete. 2023-10-08T06:53:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.370+0000 7f5f10ae5640 1 Processor -- start 2023-10-08T06:53:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.371+0000 7f5f10ae5640 1 -- start start 2023-10-08T06:53:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f10ae5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ef80039c0 0x7f5f0c0c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f10ae5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f0c144bb0 0x7f5f0c0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f10ae5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c143e00 0x7f5f0c144280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f10ae5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c15d960 con 0x7f5f0c143e00 2023-10-08T06:53:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f10ae5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c15ddf0 con 0x7f5f0c144bb0 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f10ae5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1447c0 con 0x7f5ef80039c0 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f0ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c143e00 0x7f5f0c144280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f0ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c143e00 0x7f5f0c144280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44282/0 (socket says 172.21.15.70:44282) 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f0ad9d640 1 -- 172.21.15.70:0/923110533 learned_addr learned my addr 172.21.15.70:0/923110533 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f0a59c640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ef80039c0 0x7f5f0c0c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.372+0000 7f5f03fff640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f0c144bb0 0x7f5f0c0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f0ad9d640 1 -- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ef80039c0 msgr2=0x7f5f0c0c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f0ad9d640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ef80039c0 0x7f5f0c0c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f0ad9d640 1 -- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f0c144bb0 msgr2=0x7f5f0c0b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f0ad9d640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f0c144bb0 0x7f5f0c0b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f0ad9d640 1 -- 172.21.15.70:0/923110533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c15cd50 con 0x7f5f0c143e00 2023-10-08T06:53:35.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f0ad9d640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c143e00 0x7f5f0c144280 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0b0c20 tx=0x7f5f0400d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:35.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f02ffd640 1 -- 172.21.15.70:0/923110533 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04012800 con 0x7f5f0c143e00 2023-10-08T06:53:35.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f02ffd640 1 -- 172.21.15.70:0/923110533 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f04012960 con 0x7f5f0c143e00 2023-10-08T06:53:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f10ae5640 1 -- 172.21.15.70:0/923110533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c15d030 con 0x7f5f0c143e00 2023-10-08T06:53:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f02ffd640 1 -- 172.21.15.70:0/923110533 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04012d10 con 0x7f5f0c143e00 2023-10-08T06:53:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.373+0000 7f5f10ae5640 1 -- 172.21.15.70:0/923110533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c15d5f0 con 0x7f5f0c143e00 2023-10-08T06:53:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.374+0000 7f5f10ae5640 1 -- 172.21.15.70:0/923110533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0c15d7b0 con 0x7f5f0c143e00 2023-10-08T06:53:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.375+0000 7f5f02ffd640 1 -- 172.21.15.70:0/923110533 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5f0401e460 con 0x7f5f0c143e00 2023-10-08T06:53:35.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.377+0000 7f5f02ffd640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5eec06afc0 0x7f5eec06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.377+0000 7f5f02ffd640 1 -- 172.21.15.70:0/923110533 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f5f040898d0 con 0x7f5f0c143e00 2023-10-08T06:53:35.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.377+0000 7f5f0a59c640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5eec06afc0 0x7f5eec06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.379+0000 7f5f0a59c640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5eec06afc0 0x7f5eec06d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5efc036000 tx=0x7f5efc030000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:35.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.381+0000 7f5f02ffd640 1 -- 172.21.15.70:0/923110533 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f04054610 con 0x7f5f0c143e00 2023-10-08T06:53:35.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.570+0000 7f5f10ae5640 1 -- 172.21.15.70:0/923110533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f0c071500 con 0x7f5f0c143e00 2023-10-08T06:53:35.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.572+0000 7f5f02ffd640 1 -- 172.21.15.70:0/923110533 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+14390 (secure 0 0 0) 0x7f5f04057c30 con 0x7f5f0c143e00 2023-10-08T06:53:35.573 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:35.574 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":176,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:53:33.100860+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6835/3537072579":"2023-10-09T06:53:29.884461+0000","172.21.15.181:0/2061492414":"2023-10-09T06:53:26.036761+0000","172.21.15.181:6833/236882560":"2023-10-09T06:53:26.036761+0000","172.21.15.181:6832/236882560":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/23330666":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/599491381":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/2571920213":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/3947544788":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3693343940":"2023-10-09T05:57:41.519220+0000","172.21.15.181:6833/2068783299":"2023-10-09T05:57:41.519220+0000","172.21.15.181:6832/2068783299":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3070110051":"2023-10-09T05:57:41.519220+0000","172.21.15.70:6834/3537072579":"2023-10-09T06:53:29.884461+0000","172.21.15.181:0/3604402742":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/1336064042":"2023-10-09T05:57:41.519220+0000","172.21.15.181:0/3458973146":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/3875931700":"2023-10-09T06:53:26.036761+0000","172.21.15.181:0/3791109415":"2023-10-09T05:57:41.519220+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:53:35.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.574+0000 7f5f10ae5640 1 -- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5eec06afc0 msgr2=0x7f5eec06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.574+0000 7f5f10ae5640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5eec06afc0 0x7f5eec06d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5efc036000 tx=0x7f5efc030000 comp rx=0 tx=0).stop 2023-10-08T06:53:35.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.574+0000 7f5f10ae5640 1 -- 172.21.15.70:0/923110533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c143e00 msgr2=0x7f5f0c144280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.574+0000 7f5f10ae5640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c143e00 0x7f5f0c144280 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0b0c20 tx=0x7f5f0400d960 comp rx=0 tx=0).stop 2023-10-08T06:53:35.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.575+0000 7f5f10ae5640 1 -- 172.21.15.70:0/923110533 shutdown_connections 2023-10-08T06:53:35.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.575+0000 7f5f10ae5640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5eec06afc0 0x7f5eec06d480 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.575+0000 7f5f10ae5640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f0c143e00 0x7f5f0c144280 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.575+0000 7f5f10ae5640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f0c144bb0 0x7f5f0c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.575+0000 7f5f10ae5640 1 --2- 172.21.15.70:0/923110533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5ef80039c0 0x7f5f0c0c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.575+0000 7f5f10ae5640 1 -- 172.21.15.70:0/923110533 >> 172.21.15.70:0/923110533 conn(0x7f5f0c06a380 msgr2=0x7f5f0c0c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:35.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.575+0000 7f5f10ae5640 1 -- 172.21.15.70:0/923110533 shutdown_connections 2023-10-08T06:53:35.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.575+0000 7f5f10ae5640 1 -- 172.21.15.70:0/923110533 wait complete. 2023-10-08T06:53:35.596 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T06:53:35.596 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T06:53:35.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.816+0000 7f7caefff640 1 Processor -- start 2023-10-08T06:53:35.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.816+0000 7f7caefff640 1 -- start start 2023-10-08T06:53:35.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.818+0000 7f7caefff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ca80cdc30 0x7f7ca80cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.818+0000 7f7caefff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca815ac90 con 0x7f7ca80cce90 2023-10-08T06:53:35.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.818+0000 7f7caefff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca815a6d0 con 0x7f7ca80caef0 2023-10-08T06:53:35.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.818+0000 7f7caefff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca815a9b0 con 0x7f7ca80cdc30 2023-10-08T06:53:35.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.818+0000 7f7cad7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ca80cdc30 0x7f7ca80cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.818+0000 7f7cae7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7ca80cce90 0x7f7ca8159bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33734/0 (socket says 172.21.15.70:33734) 2023-10-08T06:53:35.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.818+0000 7f7cae7fe640 1 -- 172.21.15.70:0/3504476198 learned_addr learned my addr 172.21.15.70:0/3504476198 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:35.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.819+0000 7f7cad7fc640 1 -- 172.21.15.70:0/3504476198 >> v1:172.21.15.181:6789/0 conn(0x7f7ca80caef0 legacy=0x7f7ca80cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:35.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.819+0000 7f7cad7fc640 1 -- 172.21.15.70:0/3504476198 >> v1:172.21.15.70:6789/0 conn(0x7f7ca80cce90 legacy=0x7f7ca8159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.819+0000 7f7cad7fc640 1 -- 172.21.15.70:0/3504476198 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80cc150 con 0x7f7ca80cdc30 2023-10-08T06:53:35.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.819+0000 7f7cacffb640 1 -- 172.21.15.70:0/3504476198 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129276587 0 0) 0x7f7ca815ac90 con 0x7f7ca80cce90 2023-10-08T06:53:35.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.820+0000 7f7cad7fc640 1 --2- 172.21.15.70:0/3504476198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cdc30 0x7f7ca80cc950 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f7c980097a0 tx=0x7f7c9802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920dda2bcea7f614 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:35.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.820+0000 7f7cacffb640 1 -- 172.21.15.70:0/3504476198 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98031830 con 0x7f7ca80cdc30 2023-10-08T06:53:35.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.820+0000 7f7cacffb640 1 -- 172.21.15.70:0/3504476198 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c98031990 con 0x7f7ca80cdc30 2023-10-08T06:53:35.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.820+0000 7f7cacffb640 1 -- 172.21.15.70:0/3504476198 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98031d20 con 0x7f7ca80cdc30 2023-10-08T06:53:35.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.821+0000 7f7caefff640 1 -- 172.21.15.70:0/3504476198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cdc30 msgr2=0x7f7ca80cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.821+0000 7f7caefff640 1 --2- 172.21.15.70:0/3504476198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cdc30 0x7f7ca80cc950 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f7c980097a0 tx=0x7f7c9802b8d0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.821+0000 7f7caefff640 1 -- 172.21.15.70:0/3504476198 shutdown_connections 2023-10-08T06:53:35.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.821+0000 7f7caefff640 1 --2- 172.21.15.70:0/3504476198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cdc30 0x7f7ca80cc950 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.821+0000 7f7caefff640 1 -- 172.21.15.70:0/3504476198 >> 172.21.15.70:0/3504476198 conn(0x7f7ca806c010 msgr2=0x7f7ca806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:35.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.821+0000 7f7caefff640 1 -- 172.21.15.70:0/3504476198 shutdown_connections 2023-10-08T06:53:35.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.821+0000 7f7caefff640 1 -- 172.21.15.70:0/3504476198 wait complete. 2023-10-08T06:53:35.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.823+0000 7f7caefff640 1 Processor -- start 2023-10-08T06:53:35.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.823+0000 7f7caefff640 1 -- start start 2023-10-08T06:53:35.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7caefff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ca80caef0 0x7f7ca814ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7caefff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cce90 0x7f7ca8155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7caefff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca80cdc30 0x7f7ca8153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7caefff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8153c50 con 0x7f7ca80cce90 2023-10-08T06:53:35.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7caefff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80af140 con 0x7f7ca80cdc30 2023-10-08T06:53:35.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7caefff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8153e40 con 0x7f7ca80caef0 2023-10-08T06:53:35.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7cad7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cce90 0x7f7ca8155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7cad7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cce90 0x7f7ca8155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44304/0 (socket says 172.21.15.70:44304) 2023-10-08T06:53:35.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7cad7fc640 1 -- 172.21.15.70:0/1274861385 learned_addr learned my addr 172.21.15.70:0/1274861385 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:35.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7cae7fe640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca80cdc30 0x7f7ca8153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7cadffd640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ca80caef0 0x7f7ca814ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7cad7fc640 1 -- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ca80caef0 msgr2=0x7f7ca814ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7cad7fc640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ca80caef0 0x7f7ca814ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7cad7fc640 1 -- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca80cdc30 msgr2=0x7f7ca8153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.824+0000 7f7cad7fc640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca80cdc30 0x7f7ca8153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.825+0000 7f7cad7fc640 1 -- 172.21.15.70:0/1274861385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98009400 con 0x7f7ca80cce90 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.825+0000 7f7cadffd640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ca80caef0 0x7f7ca814ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.825+0000 7f7cad7fc640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cce90 0x7f7ca8155bf0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f7c98034040 tx=0x7f7c980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.825+0000 7f7c9e7fc640 1 -- 172.21.15.70:0/1274861385 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9803ba70 con 0x7f7ca80cce90 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.825+0000 7f7c9e7fc640 1 -- 172.21.15.70:0/1274861385 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9803bbf0 con 0x7f7ca80cce90 2023-10-08T06:53:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.825+0000 7f7c9e7fc640 1 -- 172.21.15.70:0/1274861385 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c980453f0 con 0x7f7ca80cce90 2023-10-08T06:53:35.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.825+0000 7f7caefff640 1 -- 172.21.15.70:0/1274861385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca8154120 con 0x7f7ca80cce90 2023-10-08T06:53:35.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.825+0000 7f7caefff640 1 -- 172.21.15.70:0/1274861385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca80af6e0 con 0x7f7ca80cce90 2023-10-08T06:53:35.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.827+0000 7f7caefff640 1 -- 172.21.15.70:0/1274861385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca81542e0 con 0x7f7ca80cce90 2023-10-08T06:53:35.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.827+0000 7f7c9e7fc640 1 -- 172.21.15.70:0/1274861385 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7c9804e450 con 0x7f7ca80cce90 2023-10-08T06:53:35.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.829+0000 7f7c9e7fc640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7c8c06af90 0x7f7c8c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:35.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.829+0000 7f7c9e7fc640 1 -- 172.21.15.70:0/1274861385 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f7c980abcc0 con 0x7f7ca80cce90 2023-10-08T06:53:35.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.830+0000 7f7cadffd640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7c8c06af90 0x7f7c8c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:35.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.831+0000 7f7cadffd640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7c8c06af90 0x7f7c8c06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4003eb0 tx=0x7f7ca4009040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:35.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:35.833+0000 7f7c9e7fc640 1 -- 172.21.15.70:0/1274861385 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c980b2080 con 0x7f7ca80cce90 2023-10-08T06:53:36.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.020+0000 7f7caefff640 1 -- 172.21.15.70:0/1274861385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ca8074150 con 0x7f7ca80cce90 2023-10-08T06:53:36.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.021+0000 7f7c9e7fc640 1 -- 172.21.15.70:0/1274861385 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c980769d0 con 0x7f7ca80cce90 2023-10-08T06:53:36.023 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T06:53:36.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 -- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7c8c06af90 msgr2=0x7f7c8c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:36.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7c8c06af90 0x7f7c8c06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4003eb0 tx=0x7f7ca4009040 comp rx=0 tx=0).stop 2023-10-08T06:53:36.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 -- 172.21.15.70:0/1274861385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cce90 msgr2=0x7f7ca8155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:36.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cce90 0x7f7ca8155bf0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f7c98034040 tx=0x7f7c980046c0 comp rx=0 tx=0).stop 2023-10-08T06:53:36.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 -- 172.21.15.70:0/1274861385 shutdown_connections 2023-10-08T06:53:36.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ca80cdc30 0x7f7ca8153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:36.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7c8c06af90 0x7f7c8c06d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:36.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ca80cce90 0x7f7ca8155bf0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:36.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 --2- 172.21.15.70:0/1274861385 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ca80caef0 0x7f7ca814ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:36.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 -- 172.21.15.70:0/1274861385 >> 172.21.15.70:0/1274861385 conn(0x7f7ca806c010 msgr2=0x7f7ca8158d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:36.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 -- 172.21.15.70:0/1274861385 shutdown_connections 2023-10-08T06:53:36.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.024+0000 7f7caefff640 1 -- 172.21.15.70:0/1274861385 wait complete. 2023-10-08T06:53:36.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-08T06:53:36.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.250+0000 7f6c4a1b0640 1 Processor -- start 2023-10-08T06:53:37.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.250+0000 7f6c4a1b0640 1 -- start start 2023-10-08T06:53:37.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.251+0000 7f6c4a1b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c4415ab40 0x7f6c440ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.252+0000 7f6c4a1b0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4415b2b0 con 0x7f6c440cb580 2023-10-08T06:53:37.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.252+0000 7f6c4a1b0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4415b590 con 0x7f6c440c90b0 2023-10-08T06:53:37.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.252+0000 7f6c4a1b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4415bf30 con 0x7f6c4415ab40 2023-10-08T06:53:37.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.252+0000 7f6c43fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6c440cb580 0x7f6c4415e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33740/0 (socket says 172.21.15.70:33740) 2023-10-08T06:53:37.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.252+0000 7f6c43fff640 1 -- 172.21.15.70:0/954893960 learned_addr learned my addr 172.21.15.70:0/954893960 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:37.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.252+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310307189 0 0) 0x7f6c4415b2b0 con 0x7f6c440cb580 2023-10-08T06:53:37.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.252+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c30003520 con 0x7f6c440cb580 2023-10-08T06:53:37.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.252+0000 7f6c437fe640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c4415ab40 0x7f6c440ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2860982663 0 0) 0x7f6c30003520 con 0x7f6c440cb580 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c4415ab40 msgr2=0x7f6c440ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c4415ab40 0x7f6c440ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 >> v1:172.21.15.181:6789/0 conn(0x7f6c440c90b0 legacy=0x7f6c440cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4415b870 con 0x7f6c440cb580 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6c3c0019f0 con 0x7f6c440cb580 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 >> v1:172.21.15.70:6789/0 conn(0x7f6c440cb580 legacy=0x7f6c4415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c4415ab40 0x7f6c300039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c30004020 0x7f6c300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c30004d00 0x7f6c30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4415bf30 con 0x7f6c30004d00 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3000bbc0 con 0x7f6c30004020 2023-10-08T06:53:37.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3000bd80 con 0x7f6c4415ab40 2023-10-08T06:53:37.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c43fff640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c30004d00 0x7f6c30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c3bfff640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c30004020 0x7f6c300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.253+0000 7f6c437fe640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c4415ab40 0x7f6c300039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.254+0000 7f6c43fff640 1 -- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c4415ab40 msgr2=0x7f6c300039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:37.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.254+0000 7f6c43fff640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c4415ab40 0x7f6c300039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.254+0000 7f6c43fff640 1 -- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c30004020 msgr2=0x7f6c300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:37.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.254+0000 7f6c43fff640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c30004020 0x7f6c300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.254+0000 7f6c43fff640 1 -- 172.21.15.70:0/954893960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30003520 con 0x7f6c30004d00 2023-10-08T06:53:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.254+0000 7f6c437fe640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c4415ab40 0x7f6c300039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.255+0000 7f6c43fff640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c30004d00 0x7f6c30005180 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6c440caa60 tx=0x7f6c300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f18459c48cc4c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.255+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c034630 con 0x7f6c30004d00 2023-10-08T06:53:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.255+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c02f070 con 0x7f6c30004d00 2023-10-08T06:53:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.255+0000 7f6c42ffd640 1 -- 172.21.15.70:0/954893960 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c034a50 con 0x7f6c30004d00 2023-10-08T06:53:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.255+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/954893960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c30004d00 msgr2=0x7f6c30005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.255+0000 7f6c4a1b0640 1 --2- 172.21.15.70:0/954893960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c30004d00 0x7f6c30005180 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f6c440caa60 tx=0x7f6c300057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.255+0000 7f6c43fff640 1 -- 172.21.15.70:0/954893960 reap_dead start 2023-10-08T06:53:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.256+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/954893960 shutdown_connections 2023-10-08T06:53:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.256+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/954893960 >> 172.21.15.70:0/954893960 conn(0x7f6c4406a380 msgr2=0x7f6c440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.256+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/954893960 shutdown_connections 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.256+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/954893960 wait complete. 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.258+0000 7f6c4a1b0640 1 Processor -- start 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.258+0000 7f6c4a1b0640 1 -- start start 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.259+0000 7f6c4a1b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c30004d00 0x7f6c4414f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.259+0000 7f6c4a1b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c441f2a10 0x7f6c441f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.259+0000 7f6c4a1b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c440c7c70 0x7f6c440c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.259+0000 7f6c4a1b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c441f4640 con 0x7f6c440c7c70 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.259+0000 7f6c4a1b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440c6470 con 0x7f6c441f2a10 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.259+0000 7f6c4a1b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c441f36b0 con 0x7f6c30004d00 2023-10-08T06:53:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c440c7c70 0x7f6c440c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c440c7c70 0x7f6c440c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44322/0 (socket says 172.21.15.70:44322) 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c43fff640 1 -- 172.21.15.70:0/3713894011 learned_addr learned my addr 172.21.15.70:0/3713894011 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c437fe640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c30004d00 0x7f6c4414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c3bfff640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c441f2a10 0x7f6c441f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c43fff640 1 -- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c30004d00 msgr2=0x7f6c4414f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c43fff640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c30004d00 0x7f6c4414f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c43fff640 1 -- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c441f2a10 msgr2=0x7f6c441f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c43fff640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c441f2a10 0x7f6c441f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c43fff640 1 -- 172.21.15.70:0/3713894011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c441f3990 con 0x7f6c440c7c70 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c3bfff640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c441f2a10 0x7f6c441f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:37.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.260+0000 7f6c43fff640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c440c7c70 0x7f6c440c80f0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c02cb40 tx=0x7f6c3c001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.261+0000 7f6c40ff9640 1 -- 172.21.15.70:0/3713894011 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c034640 con 0x7f6c440c7c70 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.261+0000 7f6c40ff9640 1 -- 172.21.15.70:0/3713894011 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c0347a0 con 0x7f6c440c7c70 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.261+0000 7f6c40ff9640 1 -- 172.21.15.70:0/3713894011 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c034b10 con 0x7f6c440c7c70 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.261+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/3713894011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c441f3ff0 con 0x7f6c440c7c70 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.261+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/3713894011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c441f33d0 con 0x7f6c440c7c70 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.262+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/3713894011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c441f47f0 con 0x7f6c440c7c70 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.263+0000 7f6c40ff9640 1 -- 172.21.15.70:0/3713894011 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6c3c031030 con 0x7f6c440c7c70 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.264+0000 7f6c40ff9640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f6c2406afc0 0x7f6c2406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.264+0000 7f6c40ff9640 1 -- 172.21.15.70:0/3713894011 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f6c3c0abe50 con 0x7f6c440c7c70 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.264+0000 7f6c437fe640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f6c2406afc0 0x7f6c2406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.266+0000 7f6c437fe640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f6c2406afc0 0x7f6c2406d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c44156190 tx=0x7f6c34008040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.267+0000 7f6c40ff9640 1 -- 172.21.15.70:0/3713894011 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c441f47f0 con 0x7f6c440c7c70 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:36.506+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/3713894011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f6c441f47f0 con 0x7f6c440c7c70 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.131+0000 7f6c40ff9640 1 -- 172.21.15.70:0/3713894011 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2546) v1 ==== 177+0+0 (secure 0 0 0) 0x7f6c441f47f0 con 0x7f6c440c7c70 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.134+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f6c2406afc0 msgr2=0x7f6c2406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f6c2406afc0 0x7f6c2406d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c44156190 tx=0x7f6c34008040 comp rx=0 tx=0).stop 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/3713894011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c440c7c70 msgr2=0x7f6c440c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c440c7c70 0x7f6c440c80f0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c02cb40 tx=0x7f6c3c001cb0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/3713894011 shutdown_connections 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f6c2406afc0 0x7f6c2406d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c440c7c70 0x7f6c440c80f0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6c441f2a10 0x7f6c441f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 --2- 172.21.15.70:0/3713894011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6c30004d00 0x7f6c4414f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/3713894011 >> 172.21.15.70:0/3713894011 conn(0x7f6c4406a380 msgr2=0x7f6c4406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:37.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/3713894011 shutdown_connections 2023-10-08T06:53:37.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.135+0000 7f6c4a1b0640 1 -- 172.21.15.70:0/3713894011 wait complete. 2023-10-08T06:53:37.181 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T06:53:37.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.402+0000 7f882e802640 1 Processor -- start 2023-10-08T06:53:37.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.402+0000 7f882e802640 1 -- start start 2023-10-08T06:53:37.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.403+0000 7f882e802640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f882806dfc0 0x7f8828070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f882e802640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8828071070 0x7f88280cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f882e802640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88280b1cc0 con 0x7f88280afbe0 2023-10-08T06:53:37.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f882e802640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88280b2620 con 0x7f882806dfc0 2023-10-08T06:53:37.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f882e802640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88280b2060 con 0x7f8828071070 2023-10-08T06:53:37.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f881ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f882806dfc0 0x7f8828070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f881ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f882806dfc0 0x7f8828070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44330/0 (socket says 172.21.15.70:44330) 2023-10-08T06:53:37.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f881ffff640 1 -- 172.21.15.70:0/1572741409 learned_addr learned my addr 172.21.15.70:0/1572741409 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:37.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f881ffff640 1 -- 172.21.15.70:0/1572741409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8828071070 msgr2=0x7f88280cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:37.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f882cd9f640 1 --2- 172.21.15.70:0/1572741409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8828071070 0x7f88280cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f881ffff640 1 --2- 172.21.15.70:0/1572741409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8828071070 0x7f88280cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f881ffff640 1 -- 172.21.15.70:0/1572741409 >> v1:172.21.15.181:6789/0 conn(0x7f88280afbe0 legacy=0x7f88280b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f881ffff640 1 -- 172.21.15.70:0/1572741409 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88280b2340 con 0x7f882806dfc0 2023-10-08T06:53:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.404+0000 7f882cd9f640 1 --2- 172.21.15.70:0/1572741409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8828071070 0x7f88280cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.406+0000 7f881ffff640 1 --2- 172.21.15.70:0/1572741409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 0x7f8828070b30 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f8810009870 tx=0x7f881002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8d69fa1c1f581f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.406+0000 7f88277fe640 1 -- 172.21.15.70:0/1572741409 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810031640 con 0x7f882806dfc0 2023-10-08T06:53:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.406+0000 7f88277fe640 1 -- 172.21.15.70:0/1572741409 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88100317a0 con 0x7f882806dfc0 2023-10-08T06:53:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.406+0000 7f88277fe640 1 -- 172.21.15.70:0/1572741409 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810031b30 con 0x7f882806dfc0 2023-10-08T06:53:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.407+0000 7f882e802640 1 -- 172.21.15.70:0/1572741409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 msgr2=0x7f8828070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.407+0000 7f882e802640 1 --2- 172.21.15.70:0/1572741409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 0x7f8828070b30 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f8810009870 tx=0x7f881002b8f0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.407+0000 7f882e802640 1 -- 172.21.15.70:0/1572741409 shutdown_connections 2023-10-08T06:53:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.407+0000 7f882e802640 1 --2- 172.21.15.70:0/1572741409 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8828071070 0x7f88280cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.407+0000 7f882e802640 1 --2- 172.21.15.70:0/1572741409 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 0x7f8828070b30 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.407+0000 7f882e802640 1 -- 172.21.15.70:0/1572741409 >> 172.21.15.70:0/1572741409 conn(0x7f882806a5d0 msgr2=0x7f88280b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.407+0000 7f882e802640 1 -- 172.21.15.70:0/1572741409 shutdown_connections 2023-10-08T06:53:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.407+0000 7f882e802640 1 -- 172.21.15.70:0/1572741409 wait complete. 2023-10-08T06:53:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.409+0000 7f882e802640 1 Processor -- start 2023-10-08T06:53:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.409+0000 7f882e802640 1 -- start start 2023-10-08T06:53:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f882e802640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 0x7f88280c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f882e802640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8828071070 0x7f882814fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f882e802640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88280afbe0 0x7f88281f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f882e802640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88280c3820 con 0x7f882806dfc0 2023-10-08T06:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f882e802640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88281502c0 con 0x7f88280afbe0 2023-10-08T06:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f882e802640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88281f1c70 con 0x7f8828071070 2023-10-08T06:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f8827fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 0x7f88280c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f8827fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 0x7f88280c32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44332/0 (socket says 172.21.15.70:44332) 2023-10-08T06:53:37.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f8827fff640 1 -- 172.21.15.70:0/3790860514 learned_addr learned my addr 172.21.15.70:0/3790860514 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f8827fff640 1 -- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8828071070 msgr2=0x7f882814fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f8827fff640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8828071070 0x7f882814fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f8827fff640 1 -- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88280afbe0 msgr2=0x7f88281f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f8827fff640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88280afbe0 0x7f88281f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.410+0000 7f8827fff640 1 -- 172.21.15.70:0/3790860514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88100094d0 con 0x7f882806dfc0 2023-10-08T06:53:37.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.411+0000 7f8827fff640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 0x7f88280c32e0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f881800b3f0 tx=0x7f881800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:37.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.411+0000 7f88257fa640 1 -- 172.21.15.70:0/3790860514 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818011830 con 0x7f882806dfc0 2023-10-08T06:53:37.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.411+0000 7f88257fa640 1 -- 172.21.15.70:0/3790860514 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8818011990 con 0x7f882806dfc0 2023-10-08T06:53:37.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.411+0000 7f88257fa640 1 -- 172.21.15.70:0/3790860514 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818011d40 con 0x7f882806dfc0 2023-10-08T06:53:37.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.411+0000 7f882e802640 1 -- 172.21.15.70:0/3790860514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88281f1f10 con 0x7f882806dfc0 2023-10-08T06:53:37.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.411+0000 7f882e802640 1 -- 172.21.15.70:0/3790860514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88281f24d0 con 0x7f882806dfc0 2023-10-08T06:53:37.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.412+0000 7f882e802640 1 -- 172.21.15.70:0/3790860514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882804e160 con 0x7f882806dfc0 2023-10-08T06:53:37.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.413+0000 7f88257fa640 1 -- 172.21.15.70:0/3790860514 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f881801a400 con 0x7f882806dfc0 2023-10-08T06:53:37.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.414+0000 7f88257fa640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f880806af90 0x7f880806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:37.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.414+0000 7f88257fa640 1 -- 172.21.15.70:0/3790860514 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f88180897b0 con 0x7f882806dfc0 2023-10-08T06:53:37.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.415+0000 7f881ffff640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f880806af90 0x7f880806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:37.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.416+0000 7f881ffff640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f880806af90 0x7f880806d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8810036000 tx=0x7f8810030000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:37.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.418+0000 7f88257fa640 1 -- 172.21.15.70:0/3790860514 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88180545e0 con 0x7f882806dfc0 2023-10-08T06:53:37.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.611+0000 7f882e802640 1 -- 172.21.15.70:0/3790860514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f88280b0ef0 con 0x7f882806dfc0 2023-10-08T06:53:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.618+0000 7f88257fa640 1 -- 172.21.15.70:0/3790860514 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8818057c00 con 0x7f882806dfc0 2023-10-08T06:53:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.621+0000 7f882e802640 1 -- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f880806af90 msgr2=0x7f880806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.621+0000 7f882e802640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f880806af90 0x7f880806d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8810036000 tx=0x7f8810030000 comp rx=0 tx=0).stop 2023-10-08T06:53:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.621+0000 7f882e802640 1 -- 172.21.15.70:0/3790860514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 msgr2=0x7f88280c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.621+0000 7f882e802640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 0x7f88280c32e0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f881800b3f0 tx=0x7f881800b8b0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.621+0000 7f882e802640 1 -- 172.21.15.70:0/3790860514 shutdown_connections 2023-10-08T06:53:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.621+0000 7f882e802640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f88280afbe0 0x7f88281f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.621+0000 7f882e802640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8828071070 0x7f882814fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.621+0000 7f882e802640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f880806af90 0x7f880806d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.621+0000 7f882e802640 1 --2- 172.21.15.70:0/3790860514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f882806dfc0 0x7f88280c32e0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:37.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.621+0000 7f882e802640 1 -- 172.21.15.70:0/3790860514 >> 172.21.15.70:0/3790860514 conn(0x7f882806a5d0 msgr2=0x7f882815de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:37.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.622+0000 7f882e802640 1 -- 172.21.15.70:0/3790860514 shutdown_connections 2023-10-08T06:53:37.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:37.622+0000 7f882e802640 1 -- 172.21.15.70:0/3790860514 wait complete. 2023-10-08T06:53:37.642 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T06:53:37.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T06:53:37.837 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T06:53:37.961 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T06:53:37.961 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:53:37.961 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:53:38.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T06:53:38.022 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T06:53:38.022 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:53:38.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T06:53:38.083 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:53:38.083 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:53:38.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T06:53:38.145 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T06:53:38.145 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:53:38.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T06:53:38.207 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T06:53:38.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:53:38.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.431+0000 7f94f756c640 1 Processor -- start 2023-10-08T06:53:38.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.431+0000 7f94f756c640 1 -- start start 2023-10-08T06:53:38.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.433+0000 7f94f756c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94f0071010 0x7f94f015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.433+0000 7f94f756c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 0x7f94f00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.433+0000 7f94f756c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f015d7d0 con 0x7f94f0150440 2023-10-08T06:53:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.433+0000 7f94f756c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f00714a0 con 0x7f94f0071010 2023-10-08T06:53:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.433+0000 7f94f756c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00c9f90 con 0x7f94f015d380 2023-10-08T06:53:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.433+0000 7f94f656a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f94f0150440 0x7f94f00c97d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33754/0 (socket says 172.21.15.70:33754) 2023-10-08T06:53:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.433+0000 7f94f656a640 1 -- 172.21.15.70:0/1061832994 learned_addr learned my addr 172.21.15.70:0/1061832994 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:38.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.433+0000 7f94f5d69640 1 --2- 172.21.15.70:0/1061832994 >> v2:172.21.15.70:3300/0 conn(0x7f94f0071010 0x7f94f015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.434+0000 7f94f6d6b640 1 --2- 172.21.15.70:0/1061832994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 0x7f94f00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.434+0000 7f94f5568640 1 -- 172.21.15.70:0/1061832994 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283042356 0 0) 0x7f94f015d7d0 con 0x7f94f0150440 2023-10-08T06:53:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.434+0000 7f94f5568640 1 -- 172.21.15.70:0/1061832994 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94dc003520 con 0x7f94f0150440 2023-10-08T06:53:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.434+0000 7f94f5d69640 1 -- 172.21.15.70:0/1061832994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 msgr2=0x7f94f00cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.434+0000 7f94f5d69640 1 --2- 172.21.15.70:0/1061832994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 0x7f94f00cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.434+0000 7f94f5d69640 1 -- 172.21.15.70:0/1061832994 >> v1:172.21.15.70:6789/0 conn(0x7f94f0150440 legacy=0x7f94f00c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:38.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.434+0000 7f94f5d69640 1 -- 172.21.15.70:0/1061832994 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00ca270 con 0x7f94f0071010 2023-10-08T06:53:38.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.435+0000 7f94f5d69640 1 --2- 172.21.15.70:0/1061832994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0071010 0x7f94f015ce40 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f94e00097d0 tx=0x7f94e002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca248f8b2419689 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:38.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.435+0000 7f94f5568640 1 -- 172.21.15.70:0/1061832994 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e002d040 con 0x7f94f0071010 2023-10-08T06:53:38.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.435+0000 7f94f5568640 1 -- 172.21.15.70:0/1061832994 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e0031810 con 0x7f94f0071010 2023-10-08T06:53:38.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.435+0000 7f94f5568640 1 -- 172.21.15.70:0/1061832994 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0031b80 con 0x7f94f0071010 2023-10-08T06:53:38.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.436+0000 7f94f756c640 1 -- 172.21.15.70:0/1061832994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0071010 msgr2=0x7f94f015ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:38.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.436+0000 7f94f756c640 1 --2- 172.21.15.70:0/1061832994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0071010 0x7f94f015ce40 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f94e00097d0 tx=0x7f94e002b8b0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.436+0000 7f94f756c640 1 -- 172.21.15.70:0/1061832994 shutdown_connections 2023-10-08T06:53:38.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.436+0000 7f94f756c640 1 --2- 172.21.15.70:0/1061832994 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 0x7f94f00cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.436+0000 7f94f756c640 1 --2- 172.21.15.70:0/1061832994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0071010 0x7f94f015ce40 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.436+0000 7f94f756c640 1 -- 172.21.15.70:0/1061832994 >> 172.21.15.70:0/1061832994 conn(0x7f94f00cb0c0 msgr2=0x7f94f00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:38.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.436+0000 7f94f756c640 1 -- 172.21.15.70:0/1061832994 shutdown_connections 2023-10-08T06:53:38.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.436+0000 7f94f756c640 1 -- 172.21.15.70:0/1061832994 wait complete. 2023-10-08T06:53:38.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.438+0000 7f94f756c640 1 Processor -- start 2023-10-08T06:53:38.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.438+0000 7f94f756c640 1 -- start start 2023-10-08T06:53:38.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.439+0000 7f94f756c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f0071010 0x7f94f00c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.439+0000 7f94f756c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0150440 0x7f94f00c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.439+0000 7f94f756c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 0x7f94f014c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.439+0000 7f94f756c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f014f030 con 0x7f94f0150440 2023-10-08T06:53:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.439+0000 7f94f756c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f014e420 con 0x7f94f0071010 2023-10-08T06:53:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.439+0000 7f94f756c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f014e700 con 0x7f94f015d380 2023-10-08T06:53:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.439+0000 7f94f5d69640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0150440 0x7f94f00c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.439+0000 7f94f5d69640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0150440 0x7f94f00c46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44352/0 (socket says 172.21.15.70:44352) 2023-10-08T06:53:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.439+0000 7f94f5d69640 1 -- 172.21.15.70:0/1558681431 learned_addr learned my addr 172.21.15.70:0/1558681431 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.439+0000 7f94f6d6b640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 0x7f94f014c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94f5d69640 1 -- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 msgr2=0x7f94f014c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94f656a640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f0071010 0x7f94f00c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94f5d69640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 0x7f94f014c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94f5d69640 1 -- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f0071010 msgr2=0x7f94f00c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:38.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94f5d69640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f0071010 0x7f94f00c4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94f5d69640 1 -- 172.21.15.70:0/1558681431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e0009400 con 0x7f94f0150440 2023-10-08T06:53:38.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94f6d6b640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 0x7f94f014c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:38.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94f5d69640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0150440 0x7f94f00c46c0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f94e0000c00 tx=0x7f94e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:38.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94e6ffd640 1 -- 172.21.15.70:0/1558681431 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0006e10 con 0x7f94f0150440 2023-10-08T06:53:38.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94e6ffd640 1 -- 172.21.15.70:0/1558681431 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e003ad60 con 0x7f94f0150440 2023-10-08T06:53:38.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94e6ffd640 1 -- 172.21.15.70:0/1558681431 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0044620 con 0x7f94f0150440 2023-10-08T06:53:38.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94f756c640 1 -- 172.21.15.70:0/1558681431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f014e9e0 con 0x7f94f0150440 2023-10-08T06:53:38.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.440+0000 7f94f756c640 1 -- 172.21.15.70:0/1558681431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f014df20 con 0x7f94f0150440 2023-10-08T06:53:38.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.441+0000 7f94f756c640 1 -- 172.21.15.70:0/1558681431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f014f1e0 con 0x7f94f0150440 2023-10-08T06:53:38.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.442+0000 7f94e6ffd640 1 -- 172.21.15.70:0/1558681431 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f94e004d440 con 0x7f94f0150440 2023-10-08T06:53:38.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.443+0000 7f94e6ffd640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f94d406af90 0x7f94d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.443+0000 7f94e6ffd640 1 -- 172.21.15.70:0/1558681431 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f94e00abcd0 con 0x7f94f0150440 2023-10-08T06:53:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.444+0000 7f94f656a640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f94d406af90 0x7f94d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.445+0000 7f94f656a640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f94d406af90 0x7f94d406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94ec000f60 tx=0x7f94ec009040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:38.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.448+0000 7f94e6ffd640 1 -- 172.21.15.70:0/1558681431 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f94f014f1e0 con 0x7f94f0150440 2023-10-08T06:53:38.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.673+0000 7f94f756c640 1 -- 172.21.15.70:0/1558681431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94f004e160 con 0x7f94f0150440 2023-10-08T06:53:38.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.674+0000 7f94e6ffd640 1 -- 172.21.15.70:0/1558681431 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f94e0035090 con 0x7f94f0150440 2023-10-08T06:53:38.676 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:38.676 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":78,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2889908260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2889908260},{"type":"v1","addr":"172.21.15.181:6837","nonce":2889908260}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7207,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/2651702867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2651702867},{"type":"v1","addr":"172.21.15.181:6835","nonce":2651702867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":9048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/1574948577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1574948577},{"type":"v1","addr":"172.21.15.70:6833","nonce":1574948577}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71},{"gid":9489,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2440060586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2440060586},{"type":"v1","addr":"172.21.15.70:6835","nonce":2440060586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[]} 2023-10-08T06:53:38.676 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 78 2023-10-08T06:53:38.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 -- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f94d406af90 msgr2=0x7f94d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f94d406af90 0x7f94d406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94ec000f60 tx=0x7f94ec009040 comp rx=0 tx=0).stop 2023-10-08T06:53:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 -- 172.21.15.70:0/1558681431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0150440 msgr2=0x7f94f00c46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0150440 0x7f94f00c46c0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f94e0000c00 tx=0x7f94e00046c0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 -- 172.21.15.70:0/1558681431 shutdown_connections 2023-10-08T06:53:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94f015d380 0x7f94f014c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f94d406af90 0x7f94d406d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94f0150440 0x7f94f00c46c0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 --2- 172.21.15.70:0/1558681431 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f94f0071010 0x7f94f00c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 -- 172.21.15.70:0/1558681431 >> 172.21.15.70:0/1558681431 conn(0x7f94f00cb0c0 msgr2=0x7f94f014b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:38.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 -- 172.21.15.70:0/1558681431 shutdown_connections 2023-10-08T06:53:38.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.677+0000 7f94f756c640 1 -- 172.21.15.70:0/1558681431 wait complete. 2023-10-08T06:53:38.698 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T06:53:38.698 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T06:53:38.699 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T06:53:38.699 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T06:53:38.699 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T06:53:38.699 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T06:53:38.699 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T06:53:38.699 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T06:53:38.700 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T06:53:38.700 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T06:53:38.700 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T06:53:38.700 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T06:53:38.700 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T06:53:38.700 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T06:53:38.701 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T06:53:38.701 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T06:53:38.715 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T06:53:38.715 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T06:53:38.722 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T06:53:38.722 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T06:53:38.729 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T06:53:38.729 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T06:53:38.730 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T06:53:38.730 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T06:53:38.770 INFO:tasks.ceph.mds.c:Started 2023-10-08T06:53:38.771 DEBUG:teuthology.parallel:result is None 2023-10-08T06:53:38.777 INFO:tasks.ceph.mds.b:Started 2023-10-08T06:53:38.778 DEBUG:teuthology.parallel:result is None 2023-10-08T06:53:38.778 INFO:tasks.ceph.mds.a:Started 2023-10-08T06:53:38.778 DEBUG:teuthology.parallel:result is None 2023-10-08T06:53:38.785 INFO:tasks.ceph.mds.d:Started 2023-10-08T06:53:38.785 DEBUG:teuthology.parallel:result is None 2023-10-08T06:53:38.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T06:53:38.934 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T06:53:38.934 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T06:53:38.940 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T06:53:38.941 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T06:53:38.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.972+0000 7f9e6c610640 1 Processor -- start 2023-10-08T06:53:38.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.972+0000 7f9e6c610640 1 -- start start 2023-10-08T06:53:38.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.973+0000 7f9e6c610640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e6406c3a0 0x7f9e640c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.973+0000 7f9e6c610640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e6415aa00 0x7f9e640b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.973+0000 7f9e6c610640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6415b310 con 0x7f9e6406db60 2023-10-08T06:53:38.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.973+0000 7f9e6c610640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e640ca230 con 0x7f9e6406c3a0 2023-10-08T06:53:38.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.973+0000 7f9e6c610640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6406d290 con 0x7f9e6415aa00 2023-10-08T06:53:38.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.974+0000 7f9e6a3ac640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e6406c3a0 0x7f9e640c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.974+0000 7f9e6abad640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e6415aa00 0x7f9e640b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.974+0000 7f9e6a3ac640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9e6406c3a0 0x7f9e640c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:47844/0 (socket says 172.21.15.70:47844) 2023-10-08T06:53:38.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.974+0000 7f9e6a3ac640 1 -- 172.21.15.70:0/328311868 learned_addr learned my addr 172.21.15.70:0/328311868 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:38.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.974+0000 7f9e69bab640 1 --1- 172.21.15.70:0/328311868 >> v1:172.21.15.181:6789/0 conn(0x7f9e6406db60 0x7f9e6406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:54920/0 (socket says 172.21.15.70:54920) 2023-10-08T06:53:38.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.974+0000 7f9e6a3ac640 1 -- 172.21.15.70:0/328311868 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e6415aa00 msgr2=0x7f9e640b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:38.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.974+0000 7f9e6a3ac640 1 --2- 172.21.15.70:0/328311868 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e6415aa00 0x7f9e640b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.974+0000 7f9e6a3ac640 1 -- 172.21.15.70:0/328311868 >> v1:172.21.15.181:6789/0 conn(0x7f9e6406db60 legacy=0x7f9e6406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:38.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.974+0000 7f9e6a3ac640 1 -- 172.21.15.70:0/328311868 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640c9860 con 0x7f9e6406c3a0 2023-10-08T06:53:38.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.975+0000 7f9e6a3ac640 1 --2- 172.21.15.70:0/328311868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e6406c3a0 0x7f9e640c9cf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009870 tx=0x7f9e6002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a52ac85da2e87a23 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:38.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.975+0000 7f9e693aa640 1 -- 172.21.15.70:0/328311868 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60030840 con 0x7f9e6406c3a0 2023-10-08T06:53:38.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.975+0000 7f9e693aa640 1 -- 172.21.15.70:0/328311868 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e600309a0 con 0x7f9e6406c3a0 2023-10-08T06:53:38.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.975+0000 7f9e693aa640 1 -- 172.21.15.70:0/328311868 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60030d30 con 0x7f9e6406c3a0 2023-10-08T06:53:38.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.976+0000 7f9e6c610640 1 -- 172.21.15.70:0/328311868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e6406c3a0 msgr2=0x7f9e640c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:38.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.976+0000 7f9e6c610640 1 --2- 172.21.15.70:0/328311868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e6406c3a0 0x7f9e640c9cf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9e60009870 tx=0x7f9e6002b980 comp rx=0 tx=0).stop 2023-10-08T06:53:38.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.976+0000 7f9e6c610640 1 -- 172.21.15.70:0/328311868 shutdown_connections 2023-10-08T06:53:38.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.976+0000 7f9e6c610640 1 --2- 172.21.15.70:0/328311868 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e6415aa00 0x7f9e640b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.976+0000 7f9e6c610640 1 --2- 172.21.15.70:0/328311868 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e6406c3a0 0x7f9e640c9cf0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.976+0000 7f9e6c610640 1 -- 172.21.15.70:0/328311868 >> 172.21.15.70:0/328311868 conn(0x7f9e6406a5d0 msgr2=0x7f9e640b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:38.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.976+0000 7f9e6c610640 1 -- 172.21.15.70:0/328311868 shutdown_connections 2023-10-08T06:53:38.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.976+0000 7f9e6c610640 1 -- 172.21.15.70:0/328311868 wait complete. 2023-10-08T06:53:38.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.978+0000 7f9e6c610640 1 Processor -- start 2023-10-08T06:53:38.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.978+0000 7f9e6c610640 1 -- start start 2023-10-08T06:53:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e6c610640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e6406c3a0 0x7f9e6414cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e6c610640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e6406db60 0x7f9e640c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e6c610640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e6415aa00 0x7f9e640ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e6c610640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64166370 con 0x7f9e6415aa00 2023-10-08T06:53:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e6c610640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640c5080 con 0x7f9e6406db60 2023-10-08T06:53:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e6c610640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6415a110 con 0x7f9e6406c3a0 2023-10-08T06:53:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e6abad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e6415aa00 0x7f9e640ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e6a3ac640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e6406c3a0 0x7f9e6414cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e6abad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e6415aa00 0x7f9e640ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44388/0 (socket says 172.21.15.70:44388) 2023-10-08T06:53:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e6abad640 1 -- 172.21.15.70:0/476996308 learned_addr learned my addr 172.21.15.70:0/476996308 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.979+0000 7f9e69bab640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e6406db60 0x7f9e640c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.980+0000 7f9e6abad640 1 -- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e6406c3a0 msgr2=0x7f9e6414cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.980+0000 7f9e6abad640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e6406c3a0 0x7f9e6414cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.980+0000 7f9e6abad640 1 -- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e6406db60 msgr2=0x7f9e640c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.980+0000 7f9e6abad640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e6406db60 0x7f9e640c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.980+0000 7f9e6abad640 1 -- 172.21.15.70:0/476996308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e600094d0 con 0x7f9e6415aa00 2023-10-08T06:53:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.980+0000 7f9e6abad640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e6415aa00 0x7f9e640ca500 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00efc0 tx=0x7f9e5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.980+0000 7f9e5affd640 1 -- 172.21.15.70:0/476996308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c009ba0 con 0x7f9e6415aa00 2023-10-08T06:53:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.980+0000 7f9e5affd640 1 -- 172.21.15.70:0/476996308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c00f040 con 0x7f9e6415aa00 2023-10-08T06:53:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.980+0000 7f9e5affd640 1 -- 172.21.15.70:0/476996308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c0146c0 con 0x7f9e6415aa00 2023-10-08T06:53:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.980+0000 7f9e6c610640 1 -- 172.21.15.70:0/476996308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e640c2270 con 0x7f9e6415aa00 2023-10-08T06:53:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.981+0000 7f9e6c610640 1 -- 172.21.15.70:0/476996308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e640c5af0 con 0x7f9e6415aa00 2023-10-08T06:53:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.982+0000 7f9e6c610640 1 -- 172.21.15.70:0/476996308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e641664b0 con 0x7f9e6415aa00 2023-10-08T06:53:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.983+0000 7f9e5affd640 1 -- 172.21.15.70:0/476996308 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9e5c014820 con 0x7f9e6415aa00 2023-10-08T06:53:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.984+0000 7f9e5affd640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f9e4806afc0 0x7f9e4806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.984+0000 7f9e5affd640 1 -- 172.21.15.70:0/476996308 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f9e5c08a2d0 con 0x7f9e6415aa00 2023-10-08T06:53:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.984+0000 7f9e6a3ac640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f9e4806afc0 0x7f9e4806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:38.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.985+0000 7f9e6a3ac640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f9e4806afc0 0x7f9e4806d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9e60036000 tx=0x7f9e60008030 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:38.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:38.988+0000 7f9e5affd640 1 -- 172.21.15.70:0/476996308 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e5c055130 con 0x7f9e6415aa00 2023-10-08T06:53:39.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:39.195+0000 7f9e6c610640 1 -- 172.21.15.70:0/476996308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9e6404e160 con 0x7f9e6415aa00 2023-10-08T06:53:40.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.657+0000 7f9e5affd640 1 -- 172.21.15.70:0/476996308 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v181) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9e5c058750 con 0x7f9e6415aa00 2023-10-08T06:53:42.532 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T06:53:42.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.661+0000 7f9e6c610640 1 -- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f9e4806afc0 msgr2=0x7f9e4806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:42.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.661+0000 7f9e6c610640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f9e4806afc0 0x7f9e4806d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9e60036000 tx=0x7f9e60008030 comp rx=0 tx=0).stop 2023-10-08T06:53:42.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.661+0000 7f9e6c610640 1 -- 172.21.15.70:0/476996308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e6415aa00 msgr2=0x7f9e640ca500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:42.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.661+0000 7f9e6c610640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e6415aa00 0x7f9e640ca500 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00efc0 tx=0x7f9e5c00c490 comp rx=0 tx=0).stop 2023-10-08T06:53:42.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.662+0000 7f9e6c610640 1 -- 172.21.15.70:0/476996308 shutdown_connections 2023-10-08T06:53:42.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.662+0000 7f9e6c610640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f9e4806afc0 0x7f9e4806d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:42.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.662+0000 7f9e6c610640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e6415aa00 0x7f9e640ca500 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:42.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.662+0000 7f9e6c610640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e6406db60 0x7f9e640c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:42.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.662+0000 7f9e6c610640 1 --2- 172.21.15.70:0/476996308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e6406c3a0 0x7f9e6414cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:42.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.662+0000 7f9e6c610640 1 -- 172.21.15.70:0/476996308 >> 172.21.15.70:0/476996308 conn(0x7f9e6406a5d0 msgr2=0x7f9e64158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:42.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.662+0000 7f9e6c610640 1 -- 172.21.15.70:0/476996308 shutdown_connections 2023-10-08T06:53:42.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:40.662+0000 7f9e6c610640 1 -- 172.21.15.70:0/476996308 wait complete. 2023-10-08T06:53:42.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T06:53:42.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.762+0000 7fda6a456640 1 Processor -- start 2023-10-08T06:53:42.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.763+0000 7fda6a456640 1 -- start start 2023-10-08T06:53:42.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.764+0000 7fda6a456640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fda64159ac0 0x7fda6406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:42.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.764+0000 7fda6a456640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640709e0 0x7fda640c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:42.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.764+0000 7fda6a456640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6406dee0 con 0x7fda640ad790 2023-10-08T06:53:42.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.764+0000 7fda6a456640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6415d850 con 0x7fda64159ac0 2023-10-08T06:53:42.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.764+0000 7fda6a456640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6415db30 con 0x7fda640709e0 2023-10-08T06:53:42.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.765+0000 7fda689f3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640709e0 0x7fda640c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:42.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.765+0000 7fda689f3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640709e0 0x7fda640c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:38076/0 (socket says 172.21.15.70:38076) 2023-10-08T06:53:42.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.765+0000 7fda689f3640 1 -- 172.21.15.70:0/1506569744 learned_addr learned my addr 172.21.15.70:0/1506569744 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:42.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.765+0000 7fda63fff640 1 --2- 172.21.15.70:0/1506569744 >> v2:172.21.15.181:3300/0 conn(0x7fda64159ac0 0x7fda6406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.766+0000 7fda62ffd640 1 -- 172.21.15.70:0/1506569744 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277851602 0 0) 0x7fda6406dee0 con 0x7fda640ad790 2023-10-08T06:53:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.766+0000 7fda689f3640 1 -- 172.21.15.70:0/1506569744 >> v2:172.21.15.181:3300/0 conn(0x7fda64159ac0 msgr2=0x7fda6406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.766+0000 7fda689f3640 1 --2- 172.21.15.70:0/1506569744 >> v2:172.21.15.181:3300/0 conn(0x7fda64159ac0 0x7fda6406d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.766+0000 7fda689f3640 1 -- 172.21.15.70:0/1506569744 >> v1:172.21.15.181:6789/0 conn(0x7fda640ad790 legacy=0x7fda640702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.766+0000 7fda689f3640 1 -- 172.21.15.70:0/1506569744 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6415de10 con 0x7fda640709e0 2023-10-08T06:53:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.766+0000 7fda63fff640 1 --2- 172.21.15.70:0/1506569744 >> v2:172.21.15.181:3300/0 conn(0x7fda64159ac0 0x7fda6406d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T06:53:42.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.767+0000 7fda689f3640 1 --2- 172.21.15.70:0/1506569744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640709e0 0x7fda640c8dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009870 tx=0x7fda5c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa96373a2dbdb01 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:42.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.768+0000 7fda62ffd640 1 -- 172.21.15.70:0/1506569744 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c030840 con 0x7fda640709e0 2023-10-08T06:53:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.768+0000 7fda62ffd640 1 -- 172.21.15.70:0/1506569744 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c0309a0 con 0x7fda640709e0 2023-10-08T06:53:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.768+0000 7fda62ffd640 1 -- 172.21.15.70:0/1506569744 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c030d30 con 0x7fda640709e0 2023-10-08T06:53:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.769+0000 7fda6a456640 1 -- 172.21.15.70:0/1506569744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640709e0 msgr2=0x7fda640c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.769+0000 7fda6a456640 1 --2- 172.21.15.70:0/1506569744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640709e0 0x7fda640c8dd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009870 tx=0x7fda5c02b980 comp rx=0 tx=0).stop 2023-10-08T06:53:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.769+0000 7fda6a456640 1 -- 172.21.15.70:0/1506569744 shutdown_connections 2023-10-08T06:53:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.769+0000 7fda6a456640 1 --2- 172.21.15.70:0/1506569744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640709e0 0x7fda640c8dd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.769+0000 7fda6a456640 1 --2- 172.21.15.70:0/1506569744 >> v2:172.21.15.181:3300/0 conn(0x7fda64159ac0 0x7fda6406d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.769+0000 7fda6a456640 1 -- 172.21.15.70:0/1506569744 >> 172.21.15.70:0/1506569744 conn(0x7fda6406a380 msgr2=0x7fda640b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.769+0000 7fda6a456640 1 -- 172.21.15.70:0/1506569744 shutdown_connections 2023-10-08T06:53:42.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.769+0000 7fda6a456640 1 -- 172.21.15.70:0/1506569744 wait complete. 2023-10-08T06:53:42.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.771+0000 7fda6a456640 1 Processor -- start 2023-10-08T06:53:42.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.771+0000 7fda6a456640 1 -- start start 2023-10-08T06:53:42.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.772+0000 7fda6a456640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda640709e0 0x7fda640b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:42.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.772+0000 7fda6a456640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640ad790 0x7fda641f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:42.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.772+0000 7fda6a456640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda64159ac0 0x7fda64152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.772+0000 7fda6a456640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda640b3f70 con 0x7fda64159ac0 2023-10-08T06:53:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.772+0000 7fda6a456640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda641e4000 con 0x7fda640709e0 2023-10-08T06:53:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.772+0000 7fda6a456640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda641e42e0 con 0x7fda640ad790 2023-10-08T06:53:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.772+0000 7fda689f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda64159ac0 0x7fda64152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.772+0000 7fda689f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda64159ac0 0x7fda64152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44398/0 (socket says 172.21.15.70:44398) 2023-10-08T06:53:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.772+0000 7fda689f3640 1 -- 172.21.15.70:0/528139714 learned_addr learned my addr 172.21.15.70:0/528139714 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.773+0000 7fda63fff640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda640709e0 0x7fda640b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.773+0000 7fda637fe640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640ad790 0x7fda641f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.773+0000 7fda689f3640 1 -- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640ad790 msgr2=0x7fda641f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.773+0000 7fda689f3640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640ad790 0x7fda641f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:42.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.773+0000 7fda689f3640 1 -- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda640709e0 msgr2=0x7fda640b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:42.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.773+0000 7fda689f3640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda640709e0 0x7fda640b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:42.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.773+0000 7fda689f3640 1 -- 172.21.15.70:0/528139714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54009a70 con 0x7fda64159ac0 2023-10-08T06:53:42.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.773+0000 7fda689f3640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda64159ac0 0x7fda64152d50 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009870 tx=0x7fda5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:42.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.773+0000 7fda60ff9640 1 -- 172.21.15.70:0/528139714 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c039c90 con 0x7fda64159ac0 2023-10-08T06:53:42.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.774+0000 7fda60ff9640 1 -- 172.21.15.70:0/528139714 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c039e10 con 0x7fda64159ac0 2023-10-08T06:53:42.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.774+0000 7fda60ff9640 1 -- 172.21.15.70:0/528139714 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c043640 con 0x7fda64159ac0 2023-10-08T06:53:42.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.774+0000 7fda6a456640 1 -- 172.21.15.70:0/528139714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c0094d0 con 0x7fda64159ac0 2023-10-08T06:53:42.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.774+0000 7fda6a456640 1 -- 172.21.15.70:0/528139714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda641e48a0 con 0x7fda64159ac0 2023-10-08T06:53:42.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.775+0000 7fda6a456640 1 -- 172.21.15.70:0/528139714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda641e4420 con 0x7fda64159ac0 2023-10-08T06:53:42.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.776+0000 7fda60ff9640 1 -- 172.21.15.70:0/528139714 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fda5c0437a0 con 0x7fda64159ac0 2023-10-08T06:53:42.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.777+0000 7fda60ff9640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fda4406af90 0x7fda4406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:42.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.777+0000 7fda60ff9640 1 -- 172.21.15.70:0/528139714 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fda5c03e070 con 0x7fda64159ac0 2023-10-08T06:53:42.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.777+0000 7fda63fff640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fda4406af90 0x7fda4406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:42.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.779+0000 7fda63fff640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fda4406af90 0x7fda4406d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fda640b19c0 tx=0x7fda5400b040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:42.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:42.781+0000 7fda60ff9640 1 -- 172.21.15.70:0/528139714 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda641e4420 con 0x7fda64159ac0 2023-10-08T06:53:43.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.047+0000 7fda6a456640 1 -- 172.21.15.70:0/528139714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fda641e4dc0 con 0x7fda64159ac0 2023-10-08T06:53:43.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.048+0000 7fda60ff9640 1 -- 172.21.15.70:0/528139714 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v43) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fda641e4dc0 con 0x7fda64159ac0 2023-10-08T06:53:43.049 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:43.049 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.051 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T06:53:43.052 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.053 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:53:43.053 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T06:53:43.053 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T06:53:43.053 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.053 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T06:53:43.054 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T06:53:43.055 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.055 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:53:43.055 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:53:43.055 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.055 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T06:53:43.055 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T06:53:43.055 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:53:43.056 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T06:53:43.058 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.059 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T06:53:43.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.062 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T06:53:43.062 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T06:53:43.062 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.062 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:53:43.063 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T06:53:43.063 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T06:53:43.063 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T06:53:43.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.063 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T06:53:43.063 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T06:53:43.063 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.063 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:53:43.064 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T06:53:43.064 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T06:53:43.064 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T06:53:43.064 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.064 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T06:53:43.064 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T06:53:43.065 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.065 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T06:53:43.065 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T06:53:43.065 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T06:53:43.065 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.065 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.065 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T06:53:43.066 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:53:43.067 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T06:53:43.067 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T06:53:43.067 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:53:43.067 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.067 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:43.067 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T06:53:43.067 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T06:53:43.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.050+0000 7fda6a456640 1 -- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fda4406af90 msgr2=0x7fda4406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.050+0000 7fda6a456640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fda4406af90 0x7fda4406d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fda640b19c0 tx=0x7fda5400b040 comp rx=0 tx=0).stop 2023-10-08T06:53:43.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.050+0000 7fda6a456640 1 -- 172.21.15.70:0/528139714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda64159ac0 msgr2=0x7fda64152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.050+0000 7fda6a456640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda64159ac0 0x7fda64152d50 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009870 tx=0x7fda5c004730 comp rx=0 tx=0).stop 2023-10-08T06:53:43.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.050+0000 7fda6a456640 1 -- 172.21.15.70:0/528139714 shutdown_connections 2023-10-08T06:53:43.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.050+0000 7fda6a456640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fda4406af90 0x7fda4406d450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.050+0000 7fda6a456640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda64159ac0 0x7fda64152d50 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.050+0000 7fda6a456640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda640ad790 0x7fda641f0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.050+0000 7fda6a456640 1 --2- 172.21.15.70:0/528139714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda640709e0 0x7fda640b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.050+0000 7fda6a456640 1 -- 172.21.15.70:0/528139714 >> 172.21.15.70:0/528139714 conn(0x7fda6406a380 msgr2=0x7fda640732d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:43.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.051+0000 7fda6a456640 1 -- 172.21.15.70:0/528139714 shutdown_connections 2023-10-08T06:53:43.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.051+0000 7fda6a456640 1 -- 172.21.15.70:0/528139714 wait complete. 2023-10-08T06:53:43.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:53:43.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.282+0000 7fee6ac39640 1 Processor -- start 2023-10-08T06:53:43.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.282+0000 7fee6ac39640 1 -- start start 2023-10-08T06:53:43.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.284+0000 7fee6ac39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee641600d0 0x7fee640d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.284+0000 7fee6ac39640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee640d1750 con 0x7fee640d02f0 2023-10-08T06:53:43.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.284+0000 7fee6ac39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee640d1a30 con 0x7fee640d11d0 2023-10-08T06:53:43.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.284+0000 7fee6ac39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640d1d10 con 0x7fee641600d0 2023-10-08T06:53:43.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.284+0000 7fee691d6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fee640d02f0 0x7fee640b5d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33770/0 (socket says 172.21.15.70:33770) 2023-10-08T06:53:43.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.284+0000 7fee691d6640 1 -- 172.21.15.70:0/306709483 learned_addr learned my addr 172.21.15.70:0/306709483 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:43.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.284+0000 7fee689d5640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee641600d0 0x7fee640d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:43.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619320006 0 0) 0x7fee640d1750 con 0x7fee640d02f0 2023-10-08T06:53:43.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee54003520 con 0x7fee640d02f0 2023-10-08T06:53:43.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3976343479 0 0) 0x7fee54003520 con 0x7fee640d02f0 2023-10-08T06:53:43.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee641600d0 msgr2=0x7fee640d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:43.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee641600d0 0x7fee640d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 >> v1:172.21.15.181:6789/0 conn(0x7fee640d11d0 legacy=0x7fee640cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640d1ff0 con 0x7fee640d02f0 2023-10-08T06:53:43.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fee5c0019f0 con 0x7fee640d02f0 2023-10-08T06:53:43.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 >> v1:172.21.15.70:6789/0 conn(0x7fee640d02f0 legacy=0x7fee640b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee540039a0 0x7fee54003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee641600d0 0x7fee540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee54004d00 0x7fee54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640d1d10 con 0x7fee540039a0 2023-10-08T06:53:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5400b980 con 0x7fee54004d00 2023-10-08T06:53:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5400bb40 con 0x7fee641600d0 2023-10-08T06:53:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.285+0000 7fee689d5640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee540039a0 0x7fee54003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.286+0000 7fee63fff640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee641600d0 0x7fee540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:43.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.286+0000 7fee689d5640 1 -- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee641600d0 msgr2=0x7fee540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.286+0000 7fee691d6640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee54004d00 0x7fee54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.286+0000 7fee689d5640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee641600d0 0x7fee540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.286+0000 7fee689d5640 1 -- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee54004d00 msgr2=0x7fee54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.286+0000 7fee689d5640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee54004d00 0x7fee54005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.286+0000 7fee689d5640 1 -- 172.21.15.70:0/306709483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54003520 con 0x7fee540039a0 2023-10-08T06:53:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.286+0000 7fee691d6640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee54004d00 0x7fee54005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:43.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.286+0000 7fee63fff640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee641600d0 0x7fee540047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:43.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.287+0000 7fee689d5640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee540039a0 0x7fee54003d80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fee58009ac0 tx=0x7fee540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79899b288c57e33 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.287+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5802c600 con 0x7fee540039a0 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.287+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5802c760 con 0x7fee540039a0 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.287+0000 7fee637fe640 1 -- 172.21.15.70:0/306709483 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5802caf0 con 0x7fee540039a0 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.288+0000 7fee6ac39640 1 -- 172.21.15.70:0/306709483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee540039a0 msgr2=0x7fee54003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.288+0000 7fee6ac39640 1 --2- 172.21.15.70:0/306709483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee540039a0 0x7fee54003d80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fee58009ac0 tx=0x7fee540057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.288+0000 7fee691d6640 1 -- 172.21.15.70:0/306709483 reap_dead start 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.288+0000 7fee6ac39640 1 -- 172.21.15.70:0/306709483 shutdown_connections 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.288+0000 7fee6ac39640 1 -- 172.21.15.70:0/306709483 >> 172.21.15.70:0/306709483 conn(0x7fee640718a0 msgr2=0x7fee64071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.288+0000 7fee6ac39640 1 -- 172.21.15.70:0/306709483 shutdown_connections 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.288+0000 7fee6ac39640 1 -- 172.21.15.70:0/306709483 wait complete. 2023-10-08T06:53:43.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.290+0000 7fee6ac39640 1 Processor -- start 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.290+0000 7fee6ac39640 1 -- start start 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.291+0000 7fee6ac39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee540039a0 0x7fee64151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.291+0000 7fee6ac39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640c7730 0x7fee640c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.291+0000 7fee6ac39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640ca220 0x7fee640cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.291+0000 7fee6ac39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641516d0 con 0x7fee640ca220 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.291+0000 7fee6ac39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640ca6c0 con 0x7fee640c7730 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.291+0000 7fee6ac39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640ca9a0 con 0x7fee540039a0 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.291+0000 7fee691d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640ca220 0x7fee640cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.291+0000 7fee691d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640ca220 0x7fee640cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44422/0 (socket says 172.21.15.70:44422) 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.291+0000 7fee691d6640 1 -- 172.21.15.70:0/4127547694 learned_addr learned my addr 172.21.15.70:0/4127547694 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:43.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee691d6640 1 -- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee540039a0 msgr2=0x7fee64151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee691d6640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee540039a0 0x7fee64151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee691d6640 1 -- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640c7730 msgr2=0x7fee640c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee691d6640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640c7730 0x7fee640c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee691d6640 1 -- 172.21.15.70:0/4127547694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640cac80 con 0x7fee640ca220 2023-10-08T06:53:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee691d6640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640ca220 0x7fee640cbf70 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fee5400bcd0 tx=0x7fee5c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee617fa640 1 -- 172.21.15.70:0/4127547694 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c004d60 con 0x7fee640ca220 2023-10-08T06:53:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee617fa640 1 -- 172.21.15.70:0/4127547694 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c00bb80 con 0x7fee640ca220 2023-10-08T06:53:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee617fa640 1 -- 172.21.15.70:0/4127547694 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c007070 con 0x7fee640ca220 2023-10-08T06:53:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee6ac39640 1 -- 172.21.15.70:0/4127547694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee640caf60 con 0x7fee640ca220 2023-10-08T06:53:43.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.292+0000 7fee6ac39640 1 -- 172.21.15.70:0/4127547694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee640cc4b0 con 0x7fee640ca220 2023-10-08T06:53:43.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.294+0000 7fee6ac39640 1 -- 172.21.15.70:0/4127547694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee640cb480 con 0x7fee640ca220 2023-10-08T06:53:43.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.294+0000 7fee617fa640 1 -- 172.21.15.70:0/4127547694 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fee5c0073e0 con 0x7fee640ca220 2023-10-08T06:53:43.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.295+0000 7fee617fa640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fee4806afc0 0x7fee4806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.295+0000 7fee617fa640 1 -- 172.21.15.70:0/4127547694 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fee5c089d60 con 0x7fee640ca220 2023-10-08T06:53:43.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.296+0000 7fee689d5640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fee4806afc0 0x7fee4806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:43.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.297+0000 7fee689d5640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fee4806afc0 0x7fee4806d480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee58036000 tx=0x7fee5802e000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:43.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.300+0000 7fee617fa640 1 -- 172.21.15.70:0/4127547694 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee640cb480 con 0x7fee640ca220 2023-10-08T06:53:43.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.491+0000 7fee6ac39640 1 -- 172.21.15.70:0/4127547694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee640cb480 con 0x7fee640ca220 2023-10-08T06:53:43.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.492+0000 7fee617fa640 1 -- 172.21.15.70:0/4127547694 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+13246 (secure 0 0 0) 0x7fee640cb480 con 0x7fee640ca220 2023-10-08T06:53:43.494 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:43.494 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":181,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:53:40.655531+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:53:43.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 -- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fee4806afc0 msgr2=0x7fee4806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fee4806afc0 0x7fee4806d480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fee58036000 tx=0x7fee5802e000 comp rx=0 tx=0).stop 2023-10-08T06:53:43.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 -- 172.21.15.70:0/4127547694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640ca220 msgr2=0x7fee640cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640ca220 0x7fee640cbf70 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fee5400bcd0 tx=0x7fee5c00b4a0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 -- 172.21.15.70:0/4127547694 shutdown_connections 2023-10-08T06:53:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fee4806afc0 0x7fee4806d480 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee640ca220 0x7fee640cbf70 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee640c7730 0x7fee640c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 --2- 172.21.15.70:0/4127547694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee540039a0 0x7fee64151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 -- 172.21.15.70:0/4127547694 >> 172.21.15.70:0/4127547694 conn(0x7fee640718a0 msgr2=0x7fee6415d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 -- 172.21.15.70:0/4127547694 shutdown_connections 2023-10-08T06:53:43.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.495+0000 7fee6ac39640 1 -- 172.21.15.70:0/4127547694 wait complete. 2023-10-08T06:53:43.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T06:53:43.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T06:53:43.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.727+0000 7fb3373a9640 1 Processor -- start 2023-10-08T06:53:43.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.727+0000 7fb3373a9640 1 -- start start 2023-10-08T06:53:43.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb3373a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb330146d10 0x7fb330162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb3373a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb330144020 0x7fb330164b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb3373a9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3301650b0 con 0x7fb330146930 2023-10-08T06:53:43.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb3373a9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb330165b80 con 0x7fb330146d10 2023-10-08T06:53:43.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb3373a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb330162540 con 0x7fb330144020 2023-10-08T06:53:43.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb334944640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb330146d10 0x7fb330162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:43.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb334944640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb330146d10 0x7fb330162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48114/0 (socket says 172.21.15.70:48114) 2023-10-08T06:53:43.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb334944640 1 -- 172.21.15.70:0/3052813926 learned_addr learned my addr 172.21.15.70:0/3052813926 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:43.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb335145640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb330146930 0x7fb330143910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43922/0 (socket says 172.21.15.70:43922) 2023-10-08T06:53:43.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb334944640 1 -- 172.21.15.70:0/3052813926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb330144020 msgr2=0x7fb330164b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:43.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb334944640 1 --2- 172.21.15.70:0/3052813926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb330144020 0x7fb330164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb334944640 1 -- 172.21.15.70:0/3052813926 >> v1:172.21.15.70:6789/0 conn(0x7fb330146930 legacy=0x7fb330143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.730+0000 7fb334944640 1 -- 172.21.15.70:0/3052813926 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33006c9d0 con 0x7fb330146d10 2023-10-08T06:53:43.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.729+0000 7fb327fff640 1 -- 172.21.15.70:0/3052813926 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841321063 0 0) 0x7fb3301650b0 con 0x7fb330146930 2023-10-08T06:53:43.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.731+0000 7fb334944640 1 --2- 172.21.15.70:0/3052813926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330146d10 0x7fb330162000 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fb320009870 tx=0x7fb32002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb06704edb7c5ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:43.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.731+0000 7fb327fff640 1 -- 172.21.15.70:0/3052813926 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320030840 con 0x7fb330146d10 2023-10-08T06:53:43.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.731+0000 7fb327fff640 1 -- 172.21.15.70:0/3052813926 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3200309a0 con 0x7fb330146d10 2023-10-08T06:53:43.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.731+0000 7fb327fff640 1 -- 172.21.15.70:0/3052813926 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320030d30 con 0x7fb330146d10 2023-10-08T06:53:43.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.731+0000 7fb3373a9640 1 -- 172.21.15.70:0/3052813926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330146d10 msgr2=0x7fb330162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.731+0000 7fb3373a9640 1 --2- 172.21.15.70:0/3052813926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330146d10 0x7fb330162000 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fb320009870 tx=0x7fb32002b980 comp rx=0 tx=0).stop 2023-10-08T06:53:43.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.732+0000 7fb3373a9640 1 -- 172.21.15.70:0/3052813926 shutdown_connections 2023-10-08T06:53:43.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.732+0000 7fb3373a9640 1 --2- 172.21.15.70:0/3052813926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb330144020 0x7fb330164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.732+0000 7fb3373a9640 1 --2- 172.21.15.70:0/3052813926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330146d10 0x7fb330162000 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.732+0000 7fb3373a9640 1 -- 172.21.15.70:0/3052813926 >> 172.21.15.70:0/3052813926 conn(0x7fb330069d50 msgr2=0x7fb33006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:43.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.732+0000 7fb3373a9640 1 -- 172.21.15.70:0/3052813926 shutdown_connections 2023-10-08T06:53:43.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.732+0000 7fb3373a9640 1 -- 172.21.15.70:0/3052813926 wait complete. 2023-10-08T06:53:43.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.733+0000 7fb3373a9640 1 Processor -- start 2023-10-08T06:53:43.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.733+0000 7fb3373a9640 1 -- start start 2023-10-08T06:53:43.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.734+0000 7fb3373a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330144020 0x7fb330140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.734+0000 7fb3373a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb330146930 0x7fb33013fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.734+0000 7fb3373a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb330146d10 0x7fb330145580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.734+0000 7fb3373a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330141b60 con 0x7fb330144020 2023-10-08T06:53:43.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.734+0000 7fb3373a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb330140f50 con 0x7fb330146d10 2023-10-08T06:53:43.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.734+0000 7fb3373a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb330141230 con 0x7fb330146930 2023-10-08T06:53:43.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.735+0000 7fb335145640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330144020 0x7fb330140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:43.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.735+0000 7fb335145640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330144020 0x7fb330140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48128/0 (socket says 172.21.15.70:48128) 2023-10-08T06:53:43.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.735+0000 7fb335145640 1 -- 172.21.15.70:0/3485542896 learned_addr learned my addr 172.21.15.70:0/3485542896 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.735+0000 7fb335145640 1 -- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb330146930 msgr2=0x7fb33013fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.735+0000 7fb335145640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb330146930 0x7fb33013fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.735+0000 7fb335145640 1 -- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb330146d10 msgr2=0x7fb330145580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.735+0000 7fb335145640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb330146d10 0x7fb330145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.735+0000 7fb335145640 1 -- 172.21.15.70:0/3485542896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3200094d0 con 0x7fb330144020 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.735+0000 7fb335145640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330144020 0x7fb330140970 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c4d0 tx=0x7fb32c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.736+0000 7fb325ffb640 1 -- 172.21.15.70:0/3485542896 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c0074e0 con 0x7fb330144020 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.736+0000 7fb325ffb640 1 -- 172.21.15.70:0/3485542896 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c007660 con 0x7fb330144020 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.736+0000 7fb3373a9640 1 -- 172.21.15.70:0/3485542896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb330141510 con 0x7fb330144020 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.736+0000 7fb325ffb640 1 -- 172.21.15.70:0/3485542896 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c004a20 con 0x7fb330144020 2023-10-08T06:53:43.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.736+0000 7fb3373a9640 1 -- 172.21.15.70:0/3485542896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb330140630 con 0x7fb330144020 2023-10-08T06:53:43.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.737+0000 7fb3373a9640 1 -- 172.21.15.70:0/3485542896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb330141cd0 con 0x7fb330144020 2023-10-08T06:53:43.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.738+0000 7fb325ffb640 1 -- 172.21.15.70:0/3485542896 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb32c004b80 con 0x7fb330144020 2023-10-08T06:53:43.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.739+0000 7fb325ffb640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fb31406afc0 0x7fb31406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:43.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.739+0000 7fb325ffb640 1 -- 172.21.15.70:0/3485542896 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fb32c088760 con 0x7fb330144020 2023-10-08T06:53:43.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.740+0000 7fb334944640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fb31406afc0 0x7fb31406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:43.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.741+0000 7fb334944640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fb31406afc0 0x7fb31406d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb320036000 tx=0x7fb320030000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:43.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.744+0000 7fb325ffb640 1 -- 172.21.15.70:0/3485542896 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb32c0538a0 con 0x7fb330144020 2023-10-08T06:53:43.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.933+0000 7fb3373a9640 1 -- 172.21.15.70:0/3485542896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb330159760 con 0x7fb330144020 2023-10-08T06:53:43.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.934+0000 7fb325ffb640 1 -- 172.21.15.70:0/3485542896 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v181) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb32c0180a0 con 0x7fb330144020 2023-10-08T06:53:43.935 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T06:53:43.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.936+0000 7fb3373a9640 1 -- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fb31406afc0 msgr2=0x7fb31406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.936+0000 7fb3373a9640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fb31406afc0 0x7fb31406d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb320036000 tx=0x7fb320030000 comp rx=0 tx=0).stop 2023-10-08T06:53:43.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.936+0000 7fb3373a9640 1 -- 172.21.15.70:0/3485542896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330144020 msgr2=0x7fb330140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:43.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.936+0000 7fb3373a9640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330144020 0x7fb330140970 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c4d0 tx=0x7fb32c00c990 comp rx=0 tx=0).stop 2023-10-08T06:53:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.937+0000 7fb3373a9640 1 -- 172.21.15.70:0/3485542896 shutdown_connections 2023-10-08T06:53:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.937+0000 7fb3373a9640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb330146d10 0x7fb330145580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.937+0000 7fb3373a9640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb330146930 0x7fb33013fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.937+0000 7fb3373a9640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fb31406afc0 0x7fb31406d480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.937+0000 7fb3373a9640 1 --2- 172.21.15.70:0/3485542896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb330144020 0x7fb330140970 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.937+0000 7fb3373a9640 1 -- 172.21.15.70:0/3485542896 >> 172.21.15.70:0/3485542896 conn(0x7fb330069d50 msgr2=0x7fb3300701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.937+0000 7fb3373a9640 1 -- 172.21.15.70:0/3485542896 shutdown_connections 2023-10-08T06:53:43.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:43.937+0000 7fb3373a9640 1 -- 172.21.15.70:0/3485542896 wait complete. 2023-10-08T06:53:43.958 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T06:53:44.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.167+0000 7f601f0c2640 1 Processor -- start 2023-10-08T06:53:44.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.167+0000 7f601f0c2640 1 -- start start 2023-10-08T06:53:44.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601f0c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f601815ab40 0x7f60180ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601f0c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60180c90b0 0x7f60180adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601f0c2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601815b2b0 con 0x7f60180cae70 2023-10-08T06:53:44.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601f0c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601815bf30 con 0x7f601815ab40 2023-10-08T06:53:44.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601f0c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601815f7a0 con 0x7f60180c90b0 2023-10-08T06:53:44.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601ce5e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f601815ab40 0x7f60180ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:44.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601ce5e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f601815ab40 0x7f60180ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48132/0 (socket says 172.21.15.70:48132) 2023-10-08T06:53:44.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601ce5e640 1 -- 172.21.15.70:0/3960131417 learned_addr learned my addr 172.21.15.70:0/3960131417 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:44.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f6017fff640 1 --2- 172.21.15.70:0/3960131417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60180c90b0 0x7f60180adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:44.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601ce5e640 1 -- 172.21.15.70:0/3960131417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60180c90b0 msgr2=0x7f60180adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:44.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601ce5e640 1 --2- 172.21.15.70:0/3960131417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60180c90b0 0x7f60180adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601ce5e640 1 -- 172.21.15.70:0/3960131417 >> v1:172.21.15.181:6789/0 conn(0x7f60180cae70 legacy=0x7f60180702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:44.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.170+0000 7f601ce5e640 1 -- 172.21.15.70:0/3960131417 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601815b590 con 0x7f601815ab40 2023-10-08T06:53:44.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.171+0000 7f601ce5e640 1 --2- 172.21.15.70:0/3960131417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 0x7f60180ad720 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f600c009870 tx=0x7f600c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2952c5654d28d69 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:44.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.172+0000 7f60177fe640 1 -- 172.21.15.70:0/3960131417 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c031640 con 0x7f601815ab40 2023-10-08T06:53:44.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.172+0000 7f60177fe640 1 -- 172.21.15.70:0/3960131417 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c0317a0 con 0x7f601815ab40 2023-10-08T06:53:44.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.172+0000 7f60177fe640 1 -- 172.21.15.70:0/3960131417 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c031b30 con 0x7f601815ab40 2023-10-08T06:53:44.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.172+0000 7f601f0c2640 1 -- 172.21.15.70:0/3960131417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 msgr2=0x7f60180ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:44.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.172+0000 7f601f0c2640 1 --2- 172.21.15.70:0/3960131417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 0x7f60180ad720 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f600c009870 tx=0x7f600c02b8f0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.173+0000 7f601f0c2640 1 -- 172.21.15.70:0/3960131417 shutdown_connections 2023-10-08T06:53:44.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.173+0000 7f601f0c2640 1 --2- 172.21.15.70:0/3960131417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60180c90b0 0x7f60180adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.173+0000 7f601f0c2640 1 --2- 172.21.15.70:0/3960131417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 0x7f60180ad720 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.173+0000 7f601f0c2640 1 -- 172.21.15.70:0/3960131417 >> 172.21.15.70:0/3960131417 conn(0x7f601806a380 msgr2=0x7f60180b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:44.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.173+0000 7f601f0c2640 1 -- 172.21.15.70:0/3960131417 shutdown_connections 2023-10-08T06:53:44.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.173+0000 7f601f0c2640 1 -- 172.21.15.70:0/3960131417 wait complete. 2023-10-08T06:53:44.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.174+0000 7f601f0c2640 1 Processor -- start 2023-10-08T06:53:44.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.174+0000 7f601f0c2640 1 -- start start 2023-10-08T06:53:44.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.175+0000 7f601f0c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60180c90b0 0x7f601814e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.175+0000 7f601f0c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60180cae70 0x7f60180c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.175+0000 7f601f0c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 0x7f6018158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.175+0000 7f601f0c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60181597e0 con 0x7f601815ab40 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.175+0000 7f601f0c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60180c6ab0 con 0x7f60180c90b0 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.175+0000 7f601f0c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60180c6d90 con 0x7f60180cae70 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.176+0000 7f601d65f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 0x7f6018158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.176+0000 7f601d65f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 0x7f6018158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48142/0 (socket says 172.21.15.70:48142) 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.176+0000 7f601d65f640 1 -- 172.21.15.70:0/450315764 learned_addr learned my addr 172.21.15.70:0/450315764 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.176+0000 7f601d65f640 1 -- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60180cae70 msgr2=0x7f60180c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.176+0000 7f601d65f640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60180cae70 0x7f60180c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.176+0000 7f601d65f640 1 -- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60180c90b0 msgr2=0x7f601814e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:44.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.176+0000 7f601d65f640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60180c90b0 0x7f601814e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.176+0000 7f601d65f640 1 -- 172.21.15.70:0/450315764 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0094d0 con 0x7f601815ab40 2023-10-08T06:53:44.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.176+0000 7f601d65f640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 0x7f6018158e70 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f601000b970 tx=0x7f601000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:44.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.177+0000 7f60157fa640 1 -- 172.21.15.70:0/450315764 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601000cb40 con 0x7f601815ab40 2023-10-08T06:53:44.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.177+0000 7f60157fa640 1 -- 172.21.15.70:0/450315764 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601000cca0 con 0x7f601815ab40 2023-10-08T06:53:44.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.177+0000 7f60157fa640 1 -- 172.21.15.70:0/450315764 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6010014680 con 0x7f601815ab40 2023-10-08T06:53:44.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.177+0000 7f601f0c2640 1 -- 172.21.15.70:0/450315764 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60180c7070 con 0x7f601815ab40 2023-10-08T06:53:44.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.177+0000 7f601f0c2640 1 -- 172.21.15.70:0/450315764 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60180c7ad0 con 0x7f601815ab40 2023-10-08T06:53:44.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.178+0000 7f601f0c2640 1 -- 172.21.15.70:0/450315764 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60180c72b0 con 0x7f601815ab40 2023-10-08T06:53:44.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.179+0000 7f60157fa640 1 -- 172.21.15.70:0/450315764 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f601001d3b0 con 0x7f601815ab40 2023-10-08T06:53:44.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.180+0000 7f60157fa640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ffc06af90 0x7f5ffc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.180+0000 7f60157fa640 1 -- 172.21.15.70:0/450315764 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f6010089390 con 0x7f601815ab40 2023-10-08T06:53:44.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.181+0000 7f601ce5e640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ffc06af90 0x7f5ffc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:44.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.182+0000 7f601ce5e640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ffc06af90 0x7f5ffc06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f600c036000 tx=0x7f600c030000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:44.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.185+0000 7f60157fa640 1 -- 172.21.15.70:0/450315764 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60180c72b0 con 0x7f601815ab40 2023-10-08T06:53:44.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.375+0000 7f601f0c2640 1 -- 172.21.15.70:0/450315764 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f60180c72b0 con 0x7f601815ab40 2023-10-08T06:53:44.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.376+0000 7f60157fa640 1 -- 172.21.15.70:0/450315764 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v181) v1 ==== 84+0+65 (secure 0 0 0) 0x7f60180c72b0 con 0x7f601815ab40 2023-10-08T06:53:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:44.378 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T06:53:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:53:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T06:53:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T06:53:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:53:44.378 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T06:53:44.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 -- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ffc06af90 msgr2=0x7f5ffc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:44.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ffc06af90 0x7f5ffc06d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f600c036000 tx=0x7f600c030000 comp rx=0 tx=0).stop 2023-10-08T06:53:44.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 -- 172.21.15.70:0/450315764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 msgr2=0x7f6018158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:44.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 0x7f6018158e70 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f601000b970 tx=0x7f601000be30 comp rx=0 tx=0).stop 2023-10-08T06:53:44.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 -- 172.21.15.70:0/450315764 shutdown_connections 2023-10-08T06:53:44.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f5ffc06af90 0x7f5ffc06d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f601815ab40 0x7f6018158e70 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60180cae70 0x7f60180c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 --2- 172.21.15.70:0/450315764 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60180c90b0 0x7f601814e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 -- 172.21.15.70:0/450315764 >> 172.21.15.70:0/450315764 conn(0x7f601806a380 msgr2=0x7f60180ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:44.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 -- 172.21.15.70:0/450315764 shutdown_connections 2023-10-08T06:53:44.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.379+0000 7f601f0c2640 1 -- 172.21.15.70:0/450315764 wait complete. 2023-10-08T06:53:44.398 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T06:53:44.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T06:53:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.612+0000 7fa99a8ad640 1 Processor -- start 2023-10-08T06:53:44.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.612+0000 7fa99a8ad640 1 -- start start 2023-10-08T06:53:44.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.614+0000 7fa99a8ad640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa99415aab0 0x7fa994070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.614+0000 7fa99a8ad640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940caef0 0x7fa9940aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.614+0000 7fa99a8ad640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99415ae90 con 0x7fa9940cb350 2023-10-08T06:53:44.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.614+0000 7fa99a8ad640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9940cbdb0 con 0x7fa99415aab0 2023-10-08T06:53:44.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.614+0000 7fa99a8ad640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940cb7f0 con 0x7fa9940caef0 2023-10-08T06:53:44.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.614+0000 7fa99a0ac640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa9940cb350 0x7fa9940c9040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43924/0 (socket says 172.21.15.70:43924) 2023-10-08T06:53:44.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.614+0000 7fa99a0ac640 1 -- 172.21.15.70:0/1647559286 learned_addr learned my addr 172.21.15.70:0/1647559286 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:44.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9998ab640 1 --2- 172.21.15.70:0/1647559286 >> v2:172.21.15.181:3300/0 conn(0x7fa99415aab0 0x7fa994070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:44.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9990aa640 1 --2- 172.21.15.70:0/1647559286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940caef0 0x7fa9940aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590551130 0 0) 0x7fa98c001490 con 0x7fa9940cb350 2023-10-08T06:53:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa980003520 con 0x7fa9940cb350 2023-10-08T06:53:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3679557151 0 0) 0x7fa980003520 con 0x7fa9940cb350 2023-10-08T06:53:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940caef0 msgr2=0x7fa9940aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 --2- 172.21.15.70:0/1647559286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940caef0 0x7fa9940aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 >> v2:172.21.15.181:3300/0 conn(0x7fa99415aab0 msgr2=0x7fa994070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 --2- 172.21.15.70:0/1647559286 >> v2:172.21.15.181:3300/0 conn(0x7fa99415aab0 0x7fa994070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940cbad0 con 0x7fa9940cb350 2023-10-08T06:53:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa98c001b80 con 0x7fa9940cb350 2023-10-08T06:53:44.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 >> v1:172.21.15.70:6789/0 conn(0x7fa9940cb350 legacy=0x7fa9940c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:44.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 --2- 172.21.15.70:0/1647559286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa980003970 0x7fa980003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 --2- 172.21.15.70:0/1647559286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 0x7fa980004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 --2- 172.21.15.70:0/1647559286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940caef0 0x7fa980005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940cbdb0 con 0x7fa9800047c0 2023-10-08T06:53:44.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940cb7f0 con 0x7fa980003970 2023-10-08T06:53:44.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.615+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98000b9b0 con 0x7fa9940caef0 2023-10-08T06:53:44.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.616+0000 7fa9990aa640 1 --2- 172.21.15.70:0/1647559286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 0x7fa980004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:44.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.616+0000 7fa9990aa640 1 -- 172.21.15.70:0/1647559286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940caef0 msgr2=0x7fa980005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:44.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.616+0000 7fa9990aa640 1 --2- 172.21.15.70:0/1647559286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940caef0 0x7fa980005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.616+0000 7fa9990aa640 1 -- 172.21.15.70:0/1647559286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa980003970 msgr2=0x7fa980003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:44.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.616+0000 7fa9990aa640 1 --2- 172.21.15.70:0/1647559286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa980003970 0x7fa980003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.616+0000 7fa9990aa640 1 -- 172.21.15.70:0/1647559286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa980003520 con 0x7fa9800047c0 2023-10-08T06:53:44.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.617+0000 7fa9990aa640 1 --2- 172.21.15.70:0/1647559286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 0x7fa980004c40 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fa984009ac0 tx=0x7fa9800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0fba1026c2ac32 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:44.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.617+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98402c6d0 con 0x7fa9800047c0 2023-10-08T06:53:44.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.618+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98402c830 con 0x7fa9800047c0 2023-10-08T06:53:44.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.618+0000 7fa9988a9640 1 -- 172.21.15.70:0/1647559286 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98402cbc0 con 0x7fa9800047c0 2023-10-08T06:53:44.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.618+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1647559286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 msgr2=0x7fa980004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:44.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.618+0000 7fa99a8ad640 1 --2- 172.21.15.70:0/1647559286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 0x7fa980004c40 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fa984009ac0 tx=0x7fa9800057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.618+0000 7fa99a0ac640 1 -- 172.21.15.70:0/1647559286 reap_dead start 2023-10-08T06:53:44.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.618+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1647559286 shutdown_connections 2023-10-08T06:53:44.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.618+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1647559286 >> 172.21.15.70:0/1647559286 conn(0x7fa99406c010 msgr2=0x7fa99406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:44.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.618+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1647559286 shutdown_connections 2023-10-08T06:53:44.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.618+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1647559286 wait complete. 2023-10-08T06:53:44.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.620+0000 7fa99a8ad640 1 Processor -- start 2023-10-08T06:53:44.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.620+0000 7fa99a8ad640 1 -- start start 2023-10-08T06:53:44.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.621+0000 7fa99a8ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 0x7fa99414d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.621+0000 7fa99a8ad640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa994150ce0 0x7fa9940ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.621+0000 7fa99a8ad640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940c75c0 0x7fa9940c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.621+0000 7fa99a8ad640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940c7f60 con 0x7fa9800047c0 2023-10-08T06:53:44.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.621+0000 7fa99a8ad640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940c3ad0 con 0x7fa994150ce0 2023-10-08T06:53:44.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.621+0000 7fa99a8ad640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940af1a0 con 0x7fa9940c75c0 2023-10-08T06:53:44.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.621+0000 7fa9998ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 0x7fa99414d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.621+0000 7fa9998ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 0x7fa99414d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48148/0 (socket says 172.21.15.70:48148) 2023-10-08T06:53:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.621+0000 7fa9998ab640 1 -- 172.21.15.70:0/1861080059 learned_addr learned my addr 172.21.15.70:0/1861080059 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.622+0000 7fa9998ab640 1 -- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940c75c0 msgr2=0x7fa9940c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.622+0000 7fa9998ab640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940c75c0 0x7fa9940c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.622+0000 7fa9998ab640 1 -- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa994150ce0 msgr2=0x7fa9940ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.622+0000 7fa9998ab640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa994150ce0 0x7fa9940ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.622+0000 7fa9998ab640 1 -- 172.21.15.70:0/1861080059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940c3480 con 0x7fa9800047c0 2023-10-08T06:53:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.622+0000 7fa9998ab640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 0x7fa99414d970 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fa99000e860 tx=0x7fa99000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.622+0000 7fa98a7fc640 1 -- 172.21.15.70:0/1861080059 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990009600 con 0x7fa9800047c0 2023-10-08T06:53:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.622+0000 7fa98a7fc640 1 -- 172.21.15.70:0/1861080059 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa99000c940 con 0x7fa9800047c0 2023-10-08T06:53:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.622+0000 7fa98a7fc640 1 -- 172.21.15.70:0/1861080059 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99000ccd0 con 0x7fa9800047c0 2023-10-08T06:53:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.622+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1861080059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9940c3760 con 0x7fa9800047c0 2023-10-08T06:53:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.623+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1861080059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9940af480 con 0x7fa9800047c0 2023-10-08T06:53:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.623+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1861080059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99415b7f0 con 0x7fa9800047c0 2023-10-08T06:53:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.624+0000 7fa98a7fc640 1 -- 172.21.15.70:0/1861080059 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa990013470 con 0x7fa9800047c0 2023-10-08T06:53:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.625+0000 7fa98a7fc640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa97806af90 0x7fa97806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.625+0000 7fa98a7fc640 1 -- 172.21.15.70:0/1861080059 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fa990089850 con 0x7fa9800047c0 2023-10-08T06:53:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.626+0000 7fa9990aa640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa97806af90 0x7fa97806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.627+0000 7fa9990aa640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa97806af90 0x7fa97806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa984036000 tx=0x7fa98402e000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.629+0000 7fa98a7fc640 1 -- 172.21.15.70:0/1861080059 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa990058410 con 0x7fa9800047c0 2023-10-08T06:53:44.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:44.817+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1861080059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa994074150 con 0x7fa9800047c0 2023-10-08T06:53:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.808+0000 7fa98a7fc640 1 -- 172.21.15.70:0/1861080059 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v182) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa990057fa0 con 0x7fa9800047c0 2023-10-08T06:53:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T06:53:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.811+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa97806af90 msgr2=0x7fa97806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.811+0000 7fa99a8ad640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa97806af90 0x7fa97806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa984036000 tx=0x7fa98402e000 comp rx=0 tx=0).stop 2023-10-08T06:53:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.811+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1861080059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 msgr2=0x7fa99414d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.811+0000 7fa99a8ad640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 0x7fa99414d970 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fa99000e860 tx=0x7fa99000ed20 comp rx=0 tx=0).stop 2023-10-08T06:53:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.812+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1861080059 shutdown_connections 2023-10-08T06:53:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.812+0000 7fa99a8ad640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940c75c0 0x7fa9940c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.812+0000 7fa99a8ad640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa994150ce0 0x7fa9940ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.812+0000 7fa99a8ad640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa97806af90 0x7fa97806d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.812+0000 7fa99a8ad640 1 --2- 172.21.15.70:0/1861080059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9800047c0 0x7fa99414d970 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:45.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.812+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1861080059 >> 172.21.15.70:0/1861080059 conn(0x7fa99406c010 msgr2=0x7fa9940b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:45.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.812+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1861080059 shutdown_connections 2023-10-08T06:53:45.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:45.812+0000 7fa99a8ad640 1 -- 172.21.15.70:0/1861080059 wait complete. 2023-10-08T06:53:45.838 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T06:53:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.049+0000 7f62953a9640 1 Processor -- start 2023-10-08T06:53:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.049+0000 7f62953a9640 1 -- start start 2023-10-08T06:53:46.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.052+0000 7f62953a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6290070a30 0x7f62900b0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:46.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.052+0000 7f62953a9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629015d140 con 0x7f6290159490 2023-10-08T06:53:46.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.052+0000 7f62953a9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629015d420 con 0x7f62900cac30 2023-10-08T06:53:46.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.052+0000 7f62953a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629015ddc0 con 0x7f6290070a30 2023-10-08T06:53:46.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.052+0000 7f628effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6290159490 0x7f6290070320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43936/0 (socket says 172.21.15.70:43936) 2023-10-08T06:53:46.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.052+0000 7f628effd640 1 -- 172.21.15.70:0/390685500 learned_addr learned my addr 172.21.15.70:0/390685500 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:46.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.052+0000 7f628e7fc640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6290070a30 0x7f62900b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:46.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.052+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803075020 0 0) 0x7f629015d140 con 0x7f6290159490 2023-10-08T06:53:46.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.052+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f627c003440 con 0x7f6290159490 2023-10-08T06:53:46.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 564446703 0 0) 0x7f627c003440 con 0x7f6290159490 2023-10-08T06:53:46.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6290070a30 msgr2=0x7f62900b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:46.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6290070a30 0x7f62900b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:46.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 >> v1:172.21.15.181:6789/0 conn(0x7f62900cac30 legacy=0x7f62900b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:46.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629015d700 con 0x7f6290159490 2023-10-08T06:53:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6280001670 con 0x7f6290159490 2023-10-08T06:53:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 >> v1:172.21.15.70:6789/0 conn(0x7f6290159490 legacy=0x7f6290070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627c003a90 0x7f627c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c0088a0 0x7f627c00ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6290070a30 0x7f627c00d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629015ddc0 con 0x7f627c003a90 2023-10-08T06:53:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c00dcc0 con 0x7f627c0088a0 2023-10-08T06:53:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c00de80 con 0x7f6290070a30 2023-10-08T06:53:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.053+0000 7f628effd640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627c003a90 0x7f627c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.054+0000 7f628e7fc640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c0088a0 0x7f627c00ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.054+0000 7f628effd640 1 -- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6290070a30 msgr2=0x7f627c00d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.054+0000 7f628effd640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6290070a30 0x7f627c00d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.054+0000 7f628effd640 1 -- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c0088a0 msgr2=0x7f627c00ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.054+0000 7f628effd640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c0088a0 0x7f627c00ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.054+0000 7f628effd640 1 -- 172.21.15.70:0/390685500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c003440 con 0x7f627c003a90 2023-10-08T06:53:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.054+0000 7f628e7fc640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c0088a0 0x7f627c00ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.055+0000 7f628effd640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627c003a90 0x7f627c003eb0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f628000c240 tx=0x7f627c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63695b3a2624bafb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.055+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6280005ad0 con 0x7f627c003a90 2023-10-08T06:53:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.055+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6280005c30 con 0x7f627c003a90 2023-10-08T06:53:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.055+0000 7f628dffb640 1 -- 172.21.15.70:0/390685500 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628002c8b0 con 0x7f627c003a90 2023-10-08T06:53:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.055+0000 7f62953a9640 1 -- 172.21.15.70:0/390685500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627c003a90 msgr2=0x7f627c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.055+0000 7f62953a9640 1 --2- 172.21.15.70:0/390685500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f627c003a90 0x7f627c003eb0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f628000c240 tx=0x7f627c0036f0 comp rx=0 tx=0).stop 2023-10-08T06:53:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.055+0000 7f628f7fe640 1 -- 172.21.15.70:0/390685500 reap_dead start 2023-10-08T06:53:46.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.056+0000 7f62953a9640 1 -- 172.21.15.70:0/390685500 shutdown_connections 2023-10-08T06:53:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.056+0000 7f62953a9640 1 -- 172.21.15.70:0/390685500 >> 172.21.15.70:0/390685500 conn(0x7f629006a140 msgr2=0x7f629006a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.056+0000 7f62953a9640 1 -- 172.21.15.70:0/390685500 shutdown_connections 2023-10-08T06:53:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.056+0000 7f62953a9640 1 -- 172.21.15.70:0/390685500 wait complete. 2023-10-08T06:53:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.057+0000 7f62953a9640 1 Processor -- start 2023-10-08T06:53:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.058+0000 7f62953a9640 1 -- start start 2023-10-08T06:53:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.058+0000 7f62953a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c003a90 0x7f62901e4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.058+0000 7f62953a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62901e2ed0 0x7f62901e3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f62953a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901470b0 0x7f6290147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:46.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f62953a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62901e3890 con 0x7f62901470b0 2023-10-08T06:53:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f62953a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62901e57b0 con 0x7f627c003a90 2023-10-08T06:53:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f62953a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62901e5a90 con 0x7f62901e2ed0 2023-10-08T06:53:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f628f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901470b0 0x7f6290147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f628f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901470b0 0x7f6290147530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48178/0 (socket says 172.21.15.70:48178) 2023-10-08T06:53:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f628f7fe640 1 -- 172.21.15.70:0/3801793010 learned_addr learned my addr 172.21.15.70:0/3801793010 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f628effd640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c003a90 0x7f62901e4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f628e7fc640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62901e2ed0 0x7f62901e3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f628f7fe640 1 -- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62901e2ed0 msgr2=0x7f62901e3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f628f7fe640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62901e2ed0 0x7f62901e3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:46.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f628f7fe640 1 -- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c003a90 msgr2=0x7f62901e4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:46.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f628f7fe640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c003a90 0x7f62901e4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:46.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.059+0000 7f628f7fe640 1 -- 172.21.15.70:0/3801793010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62901e5d70 con 0x7f62901470b0 2023-10-08T06:53:46.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.060+0000 7f628f7fe640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901470b0 0x7f6290147530 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f629006e430 tx=0x7f628800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:46.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.060+0000 7f62877fe640 1 -- 172.21.15.70:0/3801793010 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288005d40 con 0x7f62901470b0 2023-10-08T06:53:46.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.060+0000 7f62877fe640 1 -- 172.21.15.70:0/3801793010 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628800ced0 con 0x7f62901470b0 2023-10-08T06:53:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.060+0000 7f62877fe640 1 -- 172.21.15.70:0/3801793010 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800a850 con 0x7f62901470b0 2023-10-08T06:53:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.060+0000 7f62953a9640 1 -- 172.21.15.70:0/3801793010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62901e6050 con 0x7f62901470b0 2023-10-08T06:53:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.060+0000 7f62953a9640 1 -- 172.21.15.70:0/3801793010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6290147a70 con 0x7f62901470b0 2023-10-08T06:53:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.061+0000 7f62953a9640 1 -- 172.21.15.70:0/3801793010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6290071cb0 con 0x7f62901470b0 2023-10-08T06:53:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.062+0000 7f62877fe640 1 -- 172.21.15.70:0/3801793010 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f628800a9b0 con 0x7f62901470b0 2023-10-08T06:53:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.063+0000 7f62877fe640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.063+0000 7f62877fe640 1 -- 172.21.15.70:0/3801793010 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5663+0+0 (secure 0 0 0) 0x7f6288088730 con 0x7f62901470b0 2023-10-08T06:53:46.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.064+0000 7f628effd640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:46.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.065+0000 7f628effd640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f627006af90 0x7f627006d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6280034000 tx=0x7f6280009030 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:46.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.068+0000 7f62877fe640 1 -- 172.21.15.70:0/3801793010 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62880537b0 con 0x7f62901470b0 2023-10-08T06:53:46.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.256+0000 7f62953a9640 1 -- 172.21.15.70:0/3801793010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f629004e240 con 0x7f62901470b0 2023-10-08T06:53:46.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.815+0000 7f62877fe640 1 -- 172.21.15.70:0/3801793010 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v183) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6288056dd0 con 0x7f62901470b0 2023-10-08T06:53:46.816 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T06:53:46.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.819+0000 7f62953a9640 1 -- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f627006af90 msgr2=0x7f627006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:46.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.819+0000 7f62953a9640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f627006af90 0x7f627006d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6280034000 tx=0x7f6280009030 comp rx=0 tx=0).stop 2023-10-08T06:53:46.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.819+0000 7f62953a9640 1 -- 172.21.15.70:0/3801793010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901470b0 msgr2=0x7f6290147530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:46.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.819+0000 7f62953a9640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901470b0 0x7f6290147530 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f629006e430 tx=0x7f628800cb90 comp rx=0 tx=0).stop 2023-10-08T06:53:46.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.819+0000 7f62953a9640 1 -- 172.21.15.70:0/3801793010 shutdown_connections 2023-10-08T06:53:46.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.819+0000 7f62953a9640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f627006af90 0x7f627006d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:46.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.819+0000 7f62953a9640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62901470b0 0x7f6290147530 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:46.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.819+0000 7f62953a9640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62901e2ed0 0x7f62901e3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:46.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.819+0000 7f62953a9640 1 --2- 172.21.15.70:0/3801793010 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f627c003a90 0x7f62901e4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:46.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.819+0000 7f62953a9640 1 -- 172.21.15.70:0/3801793010 >> 172.21.15.70:0/3801793010 conn(0x7f629006a140 msgr2=0x7f62900cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:46.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.820+0000 7f62953a9640 1 -- 172.21.15.70:0/3801793010 shutdown_connections 2023-10-08T06:53:46.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:46.820+0000 7f62953a9640 1 -- 172.21.15.70:0/3801793010 wait complete. 2023-10-08T06:53:46.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T06:53:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.038+0000 7f153196e640 1 Processor -- start 2023-10-08T06:53:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.039+0000 7f153196e640 1 -- start start 2023-10-08T06:53:47.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.040+0000 7f153196e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f152c0cd190 0x7f152c0ae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:47.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.040+0000 7f153196e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f152c0c9d50 0x7f152c0ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:47.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.040+0000 7f153196e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152c0ccfa0 con 0x7f152c0ccb40 2023-10-08T06:53:47.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.040+0000 7f153196e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c0ae300 con 0x7f152c0c9d50 2023-10-08T06:53:47.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.040+0000 7f153196e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c0cb560 con 0x7f152c0cd190 2023-10-08T06:53:47.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.041+0000 7f152b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f152c0c9d50 0x7f152c0ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:47.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.041+0000 7f152b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f152c0c9d50 0x7f152c0ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48188/0 (socket says 172.21.15.70:48188) 2023-10-08T06:53:47.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.041+0000 7f152b7fe640 1 -- 172.21.15.70:0/3577924575 learned_addr learned my addr 172.21.15.70:0/3577924575 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:47.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.041+0000 7f152affd640 1 --2- 172.21.15.70:0/3577924575 >> v2:172.21.15.181:3300/0 conn(0x7f152c0cd190 0x7f152c0ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:47.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.041+0000 7f152b7fe640 1 -- 172.21.15.70:0/3577924575 >> v2:172.21.15.181:3300/0 conn(0x7f152c0cd190 msgr2=0x7f152c0ae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:47.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.041+0000 7f152b7fe640 1 --2- 172.21.15.70:0/3577924575 >> v2:172.21.15.181:3300/0 conn(0x7f152c0cd190 0x7f152c0ae490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:47.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.041+0000 7f152b7fe640 1 -- 172.21.15.70:0/3577924575 >> v1:172.21.15.70:6789/0 conn(0x7f152c0ccb40 legacy=0x7f152c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:47.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.041+0000 7f152b7fe640 1 -- 172.21.15.70:0/3577924575 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c0cb840 con 0x7f152c0c9d50 2023-10-08T06:53:47.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.041+0000 7f152a7fc640 1 -- 172.21.15.70:0/3577924575 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3727472604 0 0) 0x7f152c0ccfa0 con 0x7f152c0ccb40 2023-10-08T06:53:47.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.043+0000 7f152b7fe640 1 --2- 172.21.15.70:0/3577924575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0c9d50 0x7f152c0ca1c0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f152c06c930 tx=0x7f152402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8847c0ad9c58d1fc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:47.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.043+0000 7f152a7fc640 1 -- 172.21.15.70:0/3577924575 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152402c9f0 con 0x7f152c0c9d50 2023-10-08T06:53:47.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.043+0000 7f152a7fc640 1 -- 172.21.15.70:0/3577924575 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152402cb50 con 0x7f152c0c9d50 2023-10-08T06:53:47.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.043+0000 7f152a7fc640 1 -- 172.21.15.70:0/3577924575 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15240343f0 con 0x7f152c0c9d50 2023-10-08T06:53:47.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.043+0000 7f153196e640 1 -- 172.21.15.70:0/3577924575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0c9d50 msgr2=0x7f152c0ca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:47.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.043+0000 7f153196e640 1 --2- 172.21.15.70:0/3577924575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0c9d50 0x7f152c0ca1c0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f152c06c930 tx=0x7f152402be50 comp rx=0 tx=0).stop 2023-10-08T06:53:47.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.044+0000 7f153196e640 1 -- 172.21.15.70:0/3577924575 shutdown_connections 2023-10-08T06:53:47.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.044+0000 7f153196e640 1 --2- 172.21.15.70:0/3577924575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0c9d50 0x7f152c0ca1c0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:47.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.044+0000 7f153196e640 1 --2- 172.21.15.70:0/3577924575 >> v2:172.21.15.181:3300/0 conn(0x7f152c0cd190 0x7f152c0ae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:47.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.044+0000 7f153196e640 1 -- 172.21.15.70:0/3577924575 >> 172.21.15.70:0/3577924575 conn(0x7f152c0b0af0 msgr2=0x7f152c0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:47.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.044+0000 7f153196e640 1 -- 172.21.15.70:0/3577924575 shutdown_connections 2023-10-08T06:53:47.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.044+0000 7f153196e640 1 -- 172.21.15.70:0/3577924575 wait complete. 2023-10-08T06:53:47.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.045+0000 7f153196e640 1 Processor -- start 2023-10-08T06:53:47.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.045+0000 7f153196e640 1 -- start start 2023-10-08T06:53:47.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.046+0000 7f153196e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f152c0c9d50 0x7f152c14cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:47.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.046+0000 7f153196e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f152c0ccb40 0x7f152c1f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:47.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.046+0000 7f153196e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0cd190 0x7f152c1676b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:47.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.046+0000 7f153196e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c14d020 con 0x7f152c0cd190 2023-10-08T06:53:47.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.046+0000 7f153196e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c0ca130 con 0x7f152c0c9d50 2023-10-08T06:53:47.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.046+0000 7f153196e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c0ca3d0 con 0x7f152c0ccb40 2023-10-08T06:53:47.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.047+0000 7f152b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0cd190 0x7f152c1676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:47.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.047+0000 7f152b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0cd190 0x7f152c1676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48194/0 (socket says 172.21.15.70:48194) 2023-10-08T06:53:47.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.047+0000 7f152b7fe640 1 -- 172.21.15.70:0/558305041 learned_addr learned my addr 172.21.15.70:0/558305041 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:47.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.047+0000 7f152affd640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f152c0c9d50 0x7f152c14cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:47.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.047+0000 7f152b7fe640 1 -- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f152c0ccb40 msgr2=0x7f152c1f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.047+0000 7f15227fc640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f152c0ccb40 0x7f152c1f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.047+0000 7f152b7fe640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f152c0ccb40 0x7f152c1f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.047+0000 7f152b7fe640 1 -- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f152c0c9d50 msgr2=0x7f152c14cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.047+0000 7f152b7fe640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f152c0c9d50 0x7f152c14cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.047+0000 7f152b7fe640 1 -- 172.21.15.70:0/558305041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1524009b30 con 0x7f152c0cd190 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.048+0000 7f152b7fe640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0cd190 0x7f152c1676b0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1524036040 tx=0x7f1524004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.048+0000 7f1528ff9640 1 -- 172.21.15.70:0/558305041 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524009510 con 0x7f152c0cd190 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.048+0000 7f1528ff9640 1 -- 172.21.15.70:0/558305041 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152403e450 con 0x7f152c0cd190 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.048+0000 7f153196e640 1 -- 172.21.15.70:0/558305041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c0ca6b0 con 0x7f152c0cd190 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.048+0000 7f1528ff9640 1 -- 172.21.15.70:0/558305041 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152403e5e0 con 0x7f152c0cd190 2023-10-08T06:53:47.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.048+0000 7f153196e640 1 -- 172.21.15.70:0/558305041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c0cac70 con 0x7f152c0cd190 2023-10-08T06:53:47.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.049+0000 7f153196e640 1 -- 172.21.15.70:0/558305041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c0662c0 con 0x7f152c0cd190 2023-10-08T06:53:47.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.051+0000 7f1528ff9640 1 -- 172.21.15.70:0/558305041 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f152403e740 con 0x7f152c0cd190 2023-10-08T06:53:47.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.052+0000 7f1528ff9640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f150c06af90 0x7f150c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:47.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.052+0000 7f1528ff9640 1 -- 172.21.15.70:0/558305041 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f15240acab0 con 0x7f152c0cd190 2023-10-08T06:53:47.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.053+0000 7f152affd640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f150c06af90 0x7f150c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:47.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.054+0000 7f152affd640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f150c06af90 0x7f150c06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f151c006fd0 tx=0x7f151c008040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:47.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.056+0000 7f1528ff9640 1 -- 172.21.15.70:0/558305041 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15240778e0 con 0x7f152c0cd190 2023-10-08T06:53:47.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.284+0000 7f153196e640 1 -- 172.21.15.70:0/558305041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f152c15b080 con 0x7f152c0cd190 2023-10-08T06:53:47.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.844+0000 7f1528ff9640 1 -- 172.21.15.70:0/558305041 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v80) v1 ==== 174+0+0 (secure 0 0 0) 0x7f152c15b080 con 0x7f152c0cd190 2023-10-08T06:53:47.845 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 21 and data pool 22 2023-10-08T06:53:47.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.848+0000 7f153196e640 1 -- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f150c06af90 msgr2=0x7f150c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:47.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.848+0000 7f153196e640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f150c06af90 0x7f150c06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f151c006fd0 tx=0x7f151c008040 comp rx=0 tx=0).stop 2023-10-08T06:53:47.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.848+0000 7f153196e640 1 -- 172.21.15.70:0/558305041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0cd190 msgr2=0x7f152c1676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:47.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.848+0000 7f153196e640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0cd190 0x7f152c1676b0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1524036040 tx=0x7f1524004a40 comp rx=0 tx=0).stop 2023-10-08T06:53:47.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.849+0000 7f153196e640 1 -- 172.21.15.70:0/558305041 shutdown_connections 2023-10-08T06:53:47.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.849+0000 7f153196e640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f150c06af90 0x7f150c06d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:47.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.849+0000 7f153196e640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f152c0cd190 0x7f152c1676b0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:47.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.849+0000 7f153196e640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f152c0ccb40 0x7f152c1f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.849+0000 7f153196e640 1 --2- 172.21.15.70:0/558305041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f152c0c9d50 0x7f152c14cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.849+0000 7f153196e640 1 -- 172.21.15.70:0/558305041 >> 172.21.15.70:0/558305041 conn(0x7f152c0b0af0 msgr2=0x7f152c0aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.852+0000 7f153196e640 1 -- 172.21.15.70:0/558305041 shutdown_connections 2023-10-08T06:53:47.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:47.852+0000 7f153196e640 1 -- 172.21.15.70:0/558305041 wait complete. 2023-10-08T06:53:47.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:53:48.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.079+0000 7f04b54b4640 1 Processor -- start 2023-10-08T06:53:48.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.079+0000 7f04b54b4640 1 -- start start 2023-10-08T06:53:48.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.080+0000 7f04b54b4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04b0161320 0x7f04b00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.080+0000 7f04b54b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04b0164f70 0x7f04b00b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.080+0000 7f04b54b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b0165470 con 0x7f04b006c710 2023-10-08T06:53:48.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.080+0000 7f04b54b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b0165750 con 0x7f04b0164f70 2023-10-08T06:53:48.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.080+0000 7f04b54b4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b0165a30 con 0x7f04b0161320 2023-10-08T06:53:48.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.081+0000 7f04af7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04b0164f70 0x7f04b00b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.081+0000 7f04b4cb3640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f04b006c710 0x7f04b00c95b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43944/0 (socket says 172.21.15.70:43944) 2023-10-08T06:53:48.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.081+0000 7f04b4cb3640 1 -- 172.21.15.70:0/1927028002 learned_addr learned my addr 172.21.15.70:0/1927028002 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:48.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.081+0000 7f04af7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04b0164f70 0x7f04b00b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48196/0 (socket says 172.21.15.70:48196) 2023-10-08T06:53:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.081+0000 7f04af7fe640 1 -- 172.21.15.70:0/1927028002 >> v2:172.21.15.181:3300/0 conn(0x7f04b0161320 msgr2=0x7f04b00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.081+0000 7f04af7fe640 1 --2- 172.21.15.70:0/1927028002 >> v2:172.21.15.181:3300/0 conn(0x7f04b0161320 0x7f04b00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.081+0000 7f04af7fe640 1 -- 172.21.15.70:0/1927028002 >> v1:172.21.15.70:6789/0 conn(0x7f04b006c710 legacy=0x7f04b00c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.081+0000 7f04af7fe640 1 -- 172.21.15.70:0/1927028002 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b0165d10 con 0x7f04b0164f70 2023-10-08T06:53:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.081+0000 7f04aeffd640 1 -- 172.21.15.70:0/1927028002 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892596497 0 0) 0x7f04b0165470 con 0x7f04b006c710 2023-10-08T06:53:48.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.082+0000 7f04af7fe640 1 --2- 172.21.15.70:0/1927028002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0164f70 0x7f04b00b00c0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f04b00cf6e0 tx=0x7f04a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8ddcc6a4fcb7f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:48.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.083+0000 7f04aeffd640 1 -- 172.21.15.70:0/1927028002 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a4031830 con 0x7f04b0164f70 2023-10-08T06:53:48.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.083+0000 7f04aeffd640 1 -- 172.21.15.70:0/1927028002 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a4031990 con 0x7f04b0164f70 2023-10-08T06:53:48.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.083+0000 7f04aeffd640 1 -- 172.21.15.70:0/1927028002 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a4031d20 con 0x7f04b0164f70 2023-10-08T06:53:48.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.083+0000 7f04b54b4640 1 -- 172.21.15.70:0/1927028002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0164f70 msgr2=0x7f04b00b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.083+0000 7f04b54b4640 1 --2- 172.21.15.70:0/1927028002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0164f70 0x7f04b00b00c0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f04b00cf6e0 tx=0x7f04a402b8d0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.083+0000 7f04b54b4640 1 -- 172.21.15.70:0/1927028002 shutdown_connections 2023-10-08T06:53:48.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.083+0000 7f04b54b4640 1 --2- 172.21.15.70:0/1927028002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0164f70 0x7f04b00b00c0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.083+0000 7f04b54b4640 1 --2- 172.21.15.70:0/1927028002 >> v2:172.21.15.181:3300/0 conn(0x7f04b0161320 0x7f04b00cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.083+0000 7f04b54b4640 1 -- 172.21.15.70:0/1927028002 >> 172.21.15.70:0/1927028002 conn(0x7f04b00cb0c0 msgr2=0x7f04b00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:48.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.083+0000 7f04b54b4640 1 -- 172.21.15.70:0/1927028002 shutdown_connections 2023-10-08T06:53:48.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.084+0000 7f04b54b4640 1 -- 172.21.15.70:0/1927028002 wait complete. 2023-10-08T06:53:48.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.085+0000 7f04b54b4640 1 Processor -- start 2023-10-08T06:53:48.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.085+0000 7f04b54b4640 1 -- start start 2023-10-08T06:53:48.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.086+0000 7f04b54b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04b006c710 0x7f04b01f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.086+0000 7f04b54b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0161320 0x7f04b00b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.086+0000 7f04b54b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b0164f70 0x7f04b0163000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.086+0000 7f04b54b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b01f7d80 con 0x7f04b0161320 2023-10-08T06:53:48.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.086+0000 7f04b54b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00b1260 con 0x7f04b0164f70 2023-10-08T06:53:48.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.086+0000 7f04b54b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b00b1540 con 0x7f04b006c710 2023-10-08T06:53:48.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04af7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0161320 0x7f04b00b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04af7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0161320 0x7f04b00b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48200/0 (socket says 172.21.15.70:48200) 2023-10-08T06:53:48.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04af7fe640 1 -- 172.21.15.70:0/1662169340 learned_addr learned my addr 172.21.15.70:0/1662169340 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:48.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04b4cb3640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b0164f70 0x7f04b0163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04af7fe640 1 -- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04b006c710 msgr2=0x7f04b01f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04affff640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04b006c710 0x7f04b01f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04af7fe640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04b006c710 0x7f04b01f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04af7fe640 1 -- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b0164f70 msgr2=0x7f04b0163000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04af7fe640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b0164f70 0x7f04b0163000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04af7fe640 1 -- 172.21.15.70:0/1662169340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a4009400 con 0x7f04b0161320 2023-10-08T06:53:48.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04affff640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04b006c710 0x7f04b01f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:48.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.087+0000 7f04af7fe640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0161320 0x7f04b00b25e0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f04a4000c00 tx=0x7f04a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:48.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.088+0000 7f04acff9640 1 -- 172.21.15.70:0/1662169340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a403aa60 con 0x7f04b0161320 2023-10-08T06:53:48.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.088+0000 7f04acff9640 1 -- 172.21.15.70:0/1662169340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a403abe0 con 0x7f04b0161320 2023-10-08T06:53:48.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.088+0000 7f04acff9640 1 -- 172.21.15.70:0/1662169340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a40443f0 con 0x7f04b0161320 2023-10-08T06:53:48.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.088+0000 7f04b54b4640 1 -- 172.21.15.70:0/1662169340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b00b1820 con 0x7f04b0161320 2023-10-08T06:53:48.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.088+0000 7f04b54b4640 1 -- 172.21.15.70:0/1662169340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b00b1de0 con 0x7f04b0161320 2023-10-08T06:53:48.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.089+0000 7f04b54b4640 1 -- 172.21.15.70:0/1662169340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b004e160 con 0x7f04b0161320 2023-10-08T06:53:48.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.090+0000 7f04acff9640 1 -- 172.21.15.70:0/1662169340 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f04a404d450 con 0x7f04b0161320 2023-10-08T06:53:48.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.091+0000 7f04acff9640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f049c06af90 0x7f049c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.091+0000 7f04acff9640 1 -- 172.21.15.70:0/1662169340 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f04a40abcd0 con 0x7f04b0161320 2023-10-08T06:53:48.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.092+0000 7f04affff640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f049c06af90 0x7f049c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.093+0000 7f04affff640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f049c06af90 0x7f049c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f04a0006fd0 tx=0x7f04a0008040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:48.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.095+0000 7f04acff9640 1 -- 172.21.15.70:0/1662169340 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04a40769a0 con 0x7f04b0161320 2023-10-08T06:53:48.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.285+0000 7f04b54b4640 1 -- 172.21.15.70:0/1662169340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04b00b3900 con 0x7f04b0161320 2023-10-08T06:53:48.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.289+0000 7f04acff9640 1 -- 172.21.15.70:0/1662169340 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16911 (secure 0 0 0) 0x7f04a4079fc0 con 0x7f04b0161320 2023-10-08T06:53:48.289 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:48.290 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":184,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:53:47.285289+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:53:44.876671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-08T06:53:46.310621+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:53:48.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 -- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f049c06af90 msgr2=0x7f049c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f049c06af90 0x7f049c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f04a0006fd0 tx=0x7f04a0008040 comp rx=0 tx=0).stop 2023-10-08T06:53:48.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 -- 172.21.15.70:0/1662169340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0161320 msgr2=0x7f04b00b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0161320 0x7f04b00b25e0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f04a4000c00 tx=0x7f04a40046c0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 -- 172.21.15.70:0/1662169340 shutdown_connections 2023-10-08T06:53:48.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04b0164f70 0x7f04b0163000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f049c06af90 0x7f049c06d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b0161320 0x7f04b00b25e0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 --2- 172.21.15.70:0/1662169340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04b006c710 0x7f04b01f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 -- 172.21.15.70:0/1662169340 >> 172.21.15.70:0/1662169340 conn(0x7f04b00cb0c0 msgr2=0x7f04b0160b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:48.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 -- 172.21.15.70:0/1662169340 shutdown_connections 2023-10-08T06:53:48.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.291+0000 7f04b54b4640 1 -- 172.21.15.70:0/1662169340 wait complete. 2023-10-08T06:53:48.310 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:53:48.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.513+0000 7f21fe54c640 1 Processor -- start 2023-10-08T06:53:48.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.513+0000 7f21fe54c640 1 -- start start 2023-10-08T06:53:48.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.515+0000 7f21fe54c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21f815ab40 0x7f21f80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.515+0000 7f21fe54c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21f80c90b0 0x7f21f80adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.515+0000 7f21fe54c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f21f80cae70 0x7f21f80cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.515+0000 7f21fe54c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f815bf30 con 0x7f21f80cae70 2023-10-08T06:53:48.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.515+0000 7f21fe54c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f80cb840 con 0x7f21f815ab40 2023-10-08T06:53:48.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.515+0000 7f21fe54c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f815b2b0 con 0x7f21f80c90b0 2023-10-08T06:53:48.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.515+0000 7f21fcae9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f21f80cae70 0x7f21f80cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.515+0000 7f21fcae9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f21f80cae70 0x7f21f80cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48212/0 (socket says 172.21.15.70:48212) 2023-10-08T06:53:48.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.515+0000 7f21fcae9640 1 -- 172.21.15.70:0/3696004386 learned_addr learned my addr 172.21.15.70:0/3696004386 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:48.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.516+0000 7f21f7fff640 1 --2- 172.21.15.70:0/3696004386 >> v2:172.21.15.181:3300/0 conn(0x7f21f815ab40 0x7f21f80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.516+0000 7f21fcae9640 1 -- 172.21.15.70:0/3696004386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21f80c90b0 msgr2=0x7f21f80adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:48.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.516+0000 7f21fcae9640 1 --2- 172.21.15.70:0/3696004386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21f80c90b0 0x7f21f80adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.516+0000 7f21fcae9640 1 -- 172.21.15.70:0/3696004386 >> v2:172.21.15.181:3300/0 conn(0x7f21f815ab40 msgr2=0x7f21f80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:48.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.516+0000 7f21fcae9640 1 --2- 172.21.15.70:0/3696004386 >> v2:172.21.15.181:3300/0 conn(0x7f21f815ab40 0x7f21f80ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.516+0000 7f21fcae9640 1 -- 172.21.15.70:0/3696004386 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f815b590 con 0x7f21f80cae70 2023-10-08T06:53:48.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.516+0000 7f21f7fff640 1 --2- 172.21.15.70:0/3696004386 >> v2:172.21.15.181:3300/0 conn(0x7f21f815ab40 0x7f21f80ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:48.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.517+0000 7f21fcae9640 1 --2- 172.21.15.70:0/3696004386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f80cae70 0x7f21f80cb300 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f21f806e440 tx=0x7f21f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ba794f33ff738 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:48.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.517+0000 7f21f6ffd640 1 -- 172.21.15.70:0/3696004386 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0031870 con 0x7f21f80cae70 2023-10-08T06:53:48.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.517+0000 7f21f6ffd640 1 -- 172.21.15.70:0/3696004386 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f00319d0 con 0x7f21f80cae70 2023-10-08T06:53:48.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.517+0000 7f21f6ffd640 1 -- 172.21.15.70:0/3696004386 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0031d60 con 0x7f21f80cae70 2023-10-08T06:53:48.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.518+0000 7f21fe54c640 1 -- 172.21.15.70:0/3696004386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f80cae70 msgr2=0x7f21f80cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.518+0000 7f21fe54c640 1 --2- 172.21.15.70:0/3696004386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f80cae70 0x7f21f80cb300 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f21f806e440 tx=0x7f21f002b910 comp rx=0 tx=0).stop 2023-10-08T06:53:48.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.518+0000 7f21fe54c640 1 -- 172.21.15.70:0/3696004386 shutdown_connections 2023-10-08T06:53:48.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.518+0000 7f21fe54c640 1 --2- 172.21.15.70:0/3696004386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f80cae70 0x7f21f80cb300 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.518+0000 7f21fe54c640 1 --2- 172.21.15.70:0/3696004386 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21f80c90b0 0x7f21f80adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.518+0000 7f21fe54c640 1 --2- 172.21.15.70:0/3696004386 >> v2:172.21.15.181:3300/0 conn(0x7f21f815ab40 0x7f21f80ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.518+0000 7f21fe54c640 1 -- 172.21.15.70:0/3696004386 >> 172.21.15.70:0/3696004386 conn(0x7f21f806a380 msgr2=0x7f21f80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:48.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.518+0000 7f21fe54c640 1 -- 172.21.15.70:0/3696004386 shutdown_connections 2023-10-08T06:53:48.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.518+0000 7f21fe54c640 1 -- 172.21.15.70:0/3696004386 wait complete. 2023-10-08T06:53:48.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.520+0000 7f21fe54c640 1 Processor -- start 2023-10-08T06:53:48.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.520+0000 7f21fe54c640 1 -- start start 2023-10-08T06:53:48.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.520+0000 7f21fe54c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21f80c90b0 0x7f21f81efb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fe54c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21f80cae70 0x7f21f80c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fe54c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f815ab40 0x7f21f80b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fe54c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f8157820 con 0x7f21f815ab40 2023-10-08T06:53:48.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fe54c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80b13d0 con 0x7f21f80cae70 2023-10-08T06:53:48.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fe54c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80b16b0 con 0x7f21f80c90b0 2023-10-08T06:53:48.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fcae9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f815ab40 0x7f21f80b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fcae9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f815ab40 0x7f21f80b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48214/0 (socket says 172.21.15.70:48214) 2023-10-08T06:53:48.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fcae9640 1 -- 172.21.15.70:0/728665784 learned_addr learned my addr 172.21.15.70:0/728665784 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fcae9640 1 -- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21f80c90b0 msgr2=0x7f21f81efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21f7fff640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21f80c90b0 0x7f21f81efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21f77fe640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21f80cae70 0x7f21f80c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fcae9640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21f80c90b0 0x7f21f81efb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fcae9640 1 -- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21f80cae70 msgr2=0x7f21f80c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fcae9640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21f80cae70 0x7f21f80c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.521+0000 7f21fcae9640 1 -- 172.21.15.70:0/728665784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f00094d0 con 0x7f21f815ab40 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.522+0000 7f21fcae9640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f815ab40 0x7f21f80b2190 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f21f002b950 tx=0x7f21f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.522+0000 7f21f4ff9640 1 -- 172.21.15.70:0/728665784 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0031800 con 0x7f21f815ab40 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.522+0000 7f21f4ff9640 1 -- 172.21.15.70:0/728665784 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f00443f0 con 0x7f21f815ab40 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.522+0000 7f21f4ff9640 1 -- 172.21.15.70:0/728665784 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f00447a0 con 0x7f21f815ab40 2023-10-08T06:53:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.522+0000 7f21fe54c640 1 -- 172.21.15.70:0/728665784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f80b1990 con 0x7f21f815ab40 2023-10-08T06:53:48.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.522+0000 7f21fe54c640 1 -- 172.21.15.70:0/728665784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f81f0060 con 0x7f21f815ab40 2023-10-08T06:53:48.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.524+0000 7f21fe54c640 1 -- 172.21.15.70:0/728665784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f80b1eb0 con 0x7f21f815ab40 2023-10-08T06:53:48.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.524+0000 7f21f4ff9640 1 -- 172.21.15.70:0/728665784 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f21f0044900 con 0x7f21f815ab40 2023-10-08T06:53:48.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.525+0000 7f21f4ff9640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f21d806af90 0x7f21d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.525+0000 7f21f4ff9640 1 -- 172.21.15.70:0/728665784 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f21f002d040 con 0x7f21f815ab40 2023-10-08T06:53:48.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.526+0000 7f21f7fff640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f21d806af90 0x7f21d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.527+0000 7f21f7fff640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f21d806af90 0x7f21d806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21e8008850 tx=0x7f21e8008040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:48.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.530+0000 7f21f4ff9640 1 -- 172.21.15.70:0/728665784 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21f0076f40 con 0x7f21f815ab40 2023-10-08T06:53:48.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.719+0000 7f21fe54c640 1 -- 172.21.15.70:0/728665784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21f8067490 con 0x7f21f815ab40 2023-10-08T06:53:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.721+0000 7f21f4ff9640 1 -- 172.21.15.70:0/728665784 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16911 (secure 0 0 0) 0x7f21f007a560 con 0x7f21f815ab40 2023-10-08T06:53:48.722 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:48.722 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":184,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:53:47.285289+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:53:44.876671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-08T06:53:46.310621+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:53:48.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 -- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f21d806af90 msgr2=0x7f21d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f21d806af90 0x7f21d806d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f21e8008850 tx=0x7f21e8008040 comp rx=0 tx=0).stop 2023-10-08T06:53:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 -- 172.21.15.70:0/728665784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f815ab40 msgr2=0x7f21f80b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f815ab40 0x7f21f80b2190 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f21f002b950 tx=0x7f21f00046c0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 -- 172.21.15.70:0/728665784 shutdown_connections 2023-10-08T06:53:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f21d806af90 0x7f21d806d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21f815ab40 0x7f21f80b2190 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21f80cae70 0x7f21f80c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 --2- 172.21.15.70:0/728665784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21f80c90b0 0x7f21f81efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 -- 172.21.15.70:0/728665784 >> 172.21.15.70:0/728665784 conn(0x7f21f806a380 msgr2=0x7f21f80c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 -- 172.21.15.70:0/728665784 shutdown_connections 2023-10-08T06:53:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.724+0000 7f21fe54c640 1 -- 172.21.15.70:0/728665784 wait complete. 2023-10-08T06:53:48.743 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T06:53:48.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.950+0000 7f63d7efc640 1 Processor -- start 2023-10-08T06:53:48.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.950+0000 7f63d7efc640 1 -- start start 2023-10-08T06:53:48.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.952+0000 7f63d7efc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f63d0160210 0x7f63d00c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.952+0000 7f63d7efc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d006d1a0 con 0x7f63d01605f0 2023-10-08T06:53:48.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.952+0000 7f63d7efc640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d00cc7a0 con 0x7f63d00caa80 2023-10-08T06:53:48.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.952+0000 7f63d7efc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d00ccfb0 con 0x7f63d0160210 2023-10-08T06:53:48.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.952+0000 7f63d6499640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f63d01605f0 0x7f63d0070790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43958/0 (socket says 172.21.15.70:43958) 2023-10-08T06:53:48.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.952+0000 7f63d6499640 1 -- 172.21.15.70:0/2277639322 learned_addr learned my addr 172.21.15.70:0/2277639322 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:48.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.952+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218983269 0 0) 0x7f63d006d1a0 con 0x7f63d01605f0 2023-10-08T06:53:48.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.952+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63bc003520 con 0x7f63d01605f0 2023-10-08T06:53:48.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d5c98640 1 --2- 172.21.15.70:0/2277639322 >> v2:172.21.15.181:3300/0 conn(0x7f63d0160210 0x7f63d00c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1808909558 0 0) 0x7f63bc003520 con 0x7f63d01605f0 2023-10-08T06:53:48.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 >> v2:172.21.15.181:3300/0 conn(0x7f63d0160210 msgr2=0x7f63d00c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:48.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 --2- 172.21.15.70:0/2277639322 >> v2:172.21.15.181:3300/0 conn(0x7f63d0160210 0x7f63d00c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 >> v1:172.21.15.181:6789/0 conn(0x7f63d00caa80 legacy=0x7f63d00cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:48.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d00c9760 con 0x7f63d01605f0 2023-10-08T06:53:48.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f63c80019f0 con 0x7f63d01605f0 2023-10-08T06:53:48.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 >> v1:172.21.15.70:6789/0 conn(0x7f63d01605f0 legacy=0x7f63d0070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 --2- 172.21.15.70:0/2277639322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc003970 0x7f63bc003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 --2- 172.21.15.70:0/2277639322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63bc0047c0 0x7f63bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 --2- 172.21.15.70:0/2277639322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63bc005180 0x7f63bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00ccfb0 con 0x7f63bc005180 2023-10-08T06:53:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc00c0b0 con 0x7f63bc003970 2023-10-08T06:53:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc00c270 con 0x7f63bc0047c0 2023-10-08T06:53:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.953+0000 7f63d6499640 1 --2- 172.21.15.70:0/2277639322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63bc005180 0x7f63bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.954+0000 7f63d6499640 1 -- 172.21.15.70:0/2277639322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63bc0047c0 msgr2=0x7f63bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.954+0000 7f63d6499640 1 --2- 172.21.15.70:0/2277639322 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63bc0047c0 0x7f63bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.954+0000 7f63d6499640 1 -- 172.21.15.70:0/2277639322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc003970 msgr2=0x7f63bc003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.954+0000 7f63d6499640 1 --2- 172.21.15.70:0/2277639322 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc003970 0x7f63bc003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.954+0000 7f63d6499640 1 -- 172.21.15.70:0/2277639322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc003520 con 0x7f63bc005180 2023-10-08T06:53:48.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.954+0000 7f63d6499640 1 -- 172.21.15.70:0/2277639322 reap_dead start 2023-10-08T06:53:48.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.955+0000 7f63d6499640 1 --2- 172.21.15.70:0/2277639322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63bc005180 0x7f63bc00b980 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f63d0160a80 tx=0x7f63bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9259f89ed83d9ca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:48.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.955+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8034630 con 0x7f63bc005180 2023-10-08T06:53:48.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.955+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c802f070 con 0x7f63bc005180 2023-10-08T06:53:48.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.955+0000 7f63d4c96640 1 -- 172.21.15.70:0/2277639322 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c80349d0 con 0x7f63bc005180 2023-10-08T06:53:48.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.955+0000 7f63d7efc640 1 -- 172.21.15.70:0/2277639322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63bc005180 msgr2=0x7f63bc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.955+0000 7f63d7efc640 1 --2- 172.21.15.70:0/2277639322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63bc005180 0x7f63bc00b980 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f63d0160a80 tx=0x7f63bc0057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.956+0000 7f63d7efc640 1 -- 172.21.15.70:0/2277639322 shutdown_connections 2023-10-08T06:53:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.956+0000 7f63d7efc640 1 --2- 172.21.15.70:0/2277639322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63bc005180 0x7f63bc00b980 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.956+0000 7f63d7efc640 1 -- 172.21.15.70:0/2277639322 >> 172.21.15.70:0/2277639322 conn(0x7f63d00cebd0 msgr2=0x7f63d00cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.956+0000 7f63d7efc640 1 -- 172.21.15.70:0/2277639322 shutdown_connections 2023-10-08T06:53:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.956+0000 7f63d7efc640 1 -- 172.21.15.70:0/2277639322 wait complete. 2023-10-08T06:53:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.957+0000 7f63d7efc640 1 Processor -- start 2023-10-08T06:53:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.957+0000 7f63d7efc640 1 -- start start 2023-10-08T06:53:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.958+0000 7f63d7efc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc005180 0x7f63d00ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.958+0000 7f63d7efc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63d00aeb70 0x7f63d01f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.958+0000 7f63d7efc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63d01f5cf0 0x7f63d01f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.958+0000 7f63d7efc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d00aeff0 con 0x7f63d00aeb70 2023-10-08T06:53:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.958+0000 7f63d7efc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d015c970 con 0x7f63bc005180 2023-10-08T06:53:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.958+0000 7f63d7efc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d015c640 con 0x7f63d01f5cf0 2023-10-08T06:53:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d5497640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63d00aeb70 0x7f63d01f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d5497640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63d00aeb70 0x7f63d01f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48230/0 (socket says 172.21.15.70:48230) 2023-10-08T06:53:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d5497640 1 -- 172.21.15.70:0/2309820033 learned_addr learned my addr 172.21.15.70:0/2309820033 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d6499640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63d01f5cf0 0x7f63d01f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d5c98640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc005180 0x7f63d00ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d5497640 1 -- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63d01f5cf0 msgr2=0x7f63d01f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d5497640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63d01f5cf0 0x7f63d01f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d5497640 1 -- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc005180 msgr2=0x7f63d00ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:48.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d5497640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc005180 0x7f63d00ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:48.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d5497640 1 -- 172.21.15.70:0/2309820033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d00c7650 con 0x7f63d00aeb70 2023-10-08T06:53:48.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.959+0000 7f63d5497640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63d00aeb70 0x7f63d01f57b0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f63c000c450 tx=0x7f63c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:48.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.960+0000 7f63c67fc640 1 -- 172.21.15.70:0/2309820033 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0011790 con 0x7f63d00aeb70 2023-10-08T06:53:48.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.960+0000 7f63c67fc640 1 -- 172.21.15.70:0/2309820033 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c00118f0 con 0x7f63d00aeb70 2023-10-08T06:53:48.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.960+0000 7f63c67fc640 1 -- 172.21.15.70:0/2309820033 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0011ca0 con 0x7f63d00aeb70 2023-10-08T06:53:48.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.960+0000 7f63d7efc640 1 -- 172.21.15.70:0/2309820033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d00c7930 con 0x7f63d00aeb70 2023-10-08T06:53:48.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.960+0000 7f63d7efc640 1 -- 172.21.15.70:0/2309820033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d00c61d0 con 0x7f63d00aeb70 2023-10-08T06:53:48.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.961+0000 7f63d7efc640 1 -- 172.21.15.70:0/2309820033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d004e160 con 0x7f63d00aeb70 2023-10-08T06:53:48.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.962+0000 7f63c67fc640 1 -- 172.21.15.70:0/2309820033 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f63c001c460 con 0x7f63d00aeb70 2023-10-08T06:53:48.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.963+0000 7f63c67fc640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f63b406af90 0x7f63b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:48.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.963+0000 7f63c67fc640 1 -- 172.21.15.70:0/2309820033 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f63c00897d0 con 0x7f63d00aeb70 2023-10-08T06:53:48.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.963+0000 7f63d5c98640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f63b406af90 0x7f63b406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:48.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.965+0000 7f63d5c98640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f63b406af90 0x7f63b406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63cc006a90 tx=0x7f63cc008040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:48.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:48.967+0000 7f63c67fc640 1 -- 172.21.15.70:0/2309820033 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63c0054640 con 0x7f63d00aeb70 2023-10-08T06:53:49.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.177+0000 7f63d7efc640 1 -- 172.21.15.70:0/2309820033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f63d00c9130 con 0x7f63d00aeb70 2023-10-08T06:53:49.800 INFO:tasks.ceph.mds.a.smithi070.stderr:2023-10-08T06:53:49.799+0000 7f3445929640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T06:53:49.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.881+0000 7f63c67fc640 1 -- 172.21.15.70:0/2309820033 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) v1 ==== 118+0+0 (secure 0 0 0) 0x7f63c0057c60 con 0x7f63d00aeb70 2023-10-08T06:53:49.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.884+0000 7f63d7efc640 1 -- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f63b406af90 msgr2=0x7f63b406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:49.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.884+0000 7f63d7efc640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f63b406af90 0x7f63b406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f63cc006a90 tx=0x7f63cc008040 comp rx=0 tx=0).stop 2023-10-08T06:53:49.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.884+0000 7f63d7efc640 1 -- 172.21.15.70:0/2309820033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63d00aeb70 msgr2=0x7f63d01f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:49.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.884+0000 7f63d7efc640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63d00aeb70 0x7f63d01f57b0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f63c000c450 tx=0x7f63c000c910 comp rx=0 tx=0).stop 2023-10-08T06:53:49.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.884+0000 7f63d7efc640 1 -- 172.21.15.70:0/2309820033 shutdown_connections 2023-10-08T06:53:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.884+0000 7f63d7efc640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f63d01f5cf0 0x7f63d01f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.884+0000 7f63d7efc640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f63b406af90 0x7f63b406d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:49.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.884+0000 7f63d7efc640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63d00aeb70 0x7f63d01f57b0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:49.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.884+0000 7f63d7efc640 1 --2- 172.21.15.70:0/2309820033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f63bc005180 0x7f63d00ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:49.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.884+0000 7f63d7efc640 1 -- 172.21.15.70:0/2309820033 >> 172.21.15.70:0/2309820033 conn(0x7f63d00cebd0 msgr2=0x7f63d0155050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:49.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.885+0000 7f63d7efc640 1 -- 172.21.15.70:0/2309820033 shutdown_connections 2023-10-08T06:53:49.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:49.885+0000 7f63d7efc640 1 -- 172.21.15.70:0/2309820033 wait complete. 2023-10-08T06:53:49.907 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T06:53:49.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T06:53:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.129+0000 7f16c287c640 1 Processor -- start 2023-10-08T06:53:50.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.130+0000 7f16c287c640 1 -- start start 2023-10-08T06:53:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.131+0000 7f16c287c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16bc161690 0x7f16bc06dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.131+0000 7f16c287c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16bc0cb150 0x7f16bc0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.131+0000 7f16c287c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16bc06cc50 con 0x7f16bc0702b0 2023-10-08T06:53:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.131+0000 7f16c287c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16bc0cbb10 con 0x7f16bc0cb150 2023-10-08T06:53:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.131+0000 7f16c287c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc06cf30 con 0x7f16bc161690 2023-10-08T06:53:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.131+0000 7f16b3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16bc0cb150 0x7f16bc0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:50.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.132+0000 7f16c0e19640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f16bc0702b0 0x7f16bc070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43968/0 (socket says 172.21.15.70:43968) 2023-10-08T06:53:50.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.132+0000 7f16c0e19640 1 -- 172.21.15.70:0/2241080308 learned_addr learned my addr 172.21.15.70:0/2241080308 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:50.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.132+0000 7f16bbfff640 1 --2- 172.21.15.70:0/2241080308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16bc161690 0x7f16bc06dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:50.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.132+0000 7f16bb7fe640 1 -- 172.21.15.70:0/2241080308 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167756591 0 0) 0x7f16bc06cc50 con 0x7f16bc0702b0 2023-10-08T06:53:50.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.132+0000 7f16b3fff640 1 -- 172.21.15.70:0/2241080308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16bc161690 msgr2=0x7f16bc06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:50.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.132+0000 7f16b3fff640 1 --2- 172.21.15.70:0/2241080308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16bc161690 0x7f16bc06dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.132+0000 7f16b3fff640 1 -- 172.21.15.70:0/2241080308 >> v1:172.21.15.70:6789/0 conn(0x7f16bc0702b0 legacy=0x7f16bc070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:50.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.132+0000 7f16b3fff640 1 -- 172.21.15.70:0/2241080308 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc06d210 con 0x7f16bc0cb150 2023-10-08T06:53:50.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.133+0000 7f16b3fff640 1 --2- 172.21.15.70:0/2241080308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc0cb150 0x7f16bc0cb5d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f16a40097a0 tx=0x7f16a402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bc7a263b006e5c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:50.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.134+0000 7f16bb7fe640 1 -- 172.21.15.70:0/2241080308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a402d040 con 0x7f16bc0cb150 2023-10-08T06:53:50.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.134+0000 7f16bb7fe640 1 -- 172.21.15.70:0/2241080308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a4031810 con 0x7f16bc0cb150 2023-10-08T06:53:50.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.134+0000 7f16bb7fe640 1 -- 172.21.15.70:0/2241080308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a4031ba0 con 0x7f16bc0cb150 2023-10-08T06:53:50.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.134+0000 7f16c287c640 1 -- 172.21.15.70:0/2241080308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc0cb150 msgr2=0x7f16bc0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:50.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.134+0000 7f16c287c640 1 --2- 172.21.15.70:0/2241080308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc0cb150 0x7f16bc0cb5d0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f16a40097a0 tx=0x7f16a402b8b0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.134+0000 7f16c287c640 1 -- 172.21.15.70:0/2241080308 shutdown_connections 2023-10-08T06:53:50.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.134+0000 7f16c287c640 1 --2- 172.21.15.70:0/2241080308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc0cb150 0x7f16bc0cb5d0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.134+0000 7f16c287c640 1 --2- 172.21.15.70:0/2241080308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16bc161690 0x7f16bc06dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.134+0000 7f16c287c640 1 -- 172.21.15.70:0/2241080308 >> 172.21.15.70:0/2241080308 conn(0x7f16bc06a5d0 msgr2=0x7f16bc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:50.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.135+0000 7f16c287c640 1 -- 172.21.15.70:0/2241080308 shutdown_connections 2023-10-08T06:53:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.135+0000 7f16c287c640 1 -- 172.21.15.70:0/2241080308 wait complete. 2023-10-08T06:53:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.136+0000 7f16c287c640 1 Processor -- start 2023-10-08T06:53:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.136+0000 7f16c287c640 1 -- start start 2023-10-08T06:53:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.137+0000 7f16c287c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16bc0702b0 0x7f16bc0c1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.137+0000 7f16c287c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc0cb150 0x7f16bc1639e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.137+0000 7f16c287c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc161690 0x7f16bc163f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.137+0000 7f16c287c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc0c1e90 con 0x7f16bc161690 2023-10-08T06:53:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.137+0000 7f16c287c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc162930 con 0x7f16bc0cb150 2023-10-08T06:53:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.137+0000 7f16c287c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc162c10 con 0x7f16bc0702b0 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.138+0000 7f16c0e19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc161690 0x7f16bc163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.138+0000 7f16c0e19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc161690 0x7f16bc163f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48260/0 (socket says 172.21.15.70:48260) 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.138+0000 7f16c0e19640 1 -- 172.21.15.70:0/4132764293 learned_addr learned my addr 172.21.15.70:0/4132764293 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.138+0000 7f16bbfff640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16bc0702b0 0x7f16bc0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.138+0000 7f16c0e19640 1 -- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16bc0702b0 msgr2=0x7f16bc0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.138+0000 7f16c0e19640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16bc0702b0 0x7f16bc0c1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.138+0000 7f16c0e19640 1 -- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc0cb150 msgr2=0x7f16bc1639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.138+0000 7f16c0e19640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc0cb150 0x7f16bc1639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.138+0000 7f16c0e19640 1 -- 172.21.15.70:0/4132764293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a4009400 con 0x7f16bc161690 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.138+0000 7f16c0e19640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc161690 0x7f16bc163f20 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f16b400c7e0 tx=0x7f16b400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.139+0000 7f16b97fa640 1 -- 172.21.15.70:0/4132764293 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b400a640 con 0x7f16bc161690 2023-10-08T06:53:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.139+0000 7f16b97fa640 1 -- 172.21.15.70:0/4132764293 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b400a7c0 con 0x7f16bc161690 2023-10-08T06:53:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.139+0000 7f16c287c640 1 -- 172.21.15.70:0/4132764293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc0cc7e0 con 0x7f16bc161690 2023-10-08T06:53:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.139+0000 7f16b97fa640 1 -- 172.21.15.70:0/4132764293 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b400ab70 con 0x7f16bc161690 2023-10-08T06:53:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.139+0000 7f16c287c640 1 -- 172.21.15.70:0/4132764293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc161bc0 con 0x7f16bc161690 2023-10-08T06:53:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.140+0000 7f16c287c640 1 -- 172.21.15.70:0/4132764293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc0cc640 con 0x7f16bc161690 2023-10-08T06:53:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.141+0000 7f16b97fa640 1 -- 172.21.15.70:0/4132764293 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f16b400acd0 con 0x7f16bc161690 2023-10-08T06:53:50.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.142+0000 7f16b97fa640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f169c06af90 0x7f169c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:50.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.142+0000 7f16b97fa640 1 -- 172.21.15.70:0/4132764293 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f16b4089220 con 0x7f16bc161690 2023-10-08T06:53:50.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.143+0000 7f16bbfff640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f169c06af90 0x7f169c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:50.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.144+0000 7f16bbfff640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f169c06af90 0x7f169c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f16ac006fd0 tx=0x7f16ac008040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:50.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.147+0000 7f16b97fa640 1 -- 172.21.15.70:0/4132764293 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f16bc0cc640 con 0x7f16bc161690 2023-10-08T06:53:50.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.355+0000 7f16c287c640 1 -- 172.21.15.70:0/4132764293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f16bc0ccd00 con 0x7f16bc161690 2023-10-08T06:53:50.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.892+0000 7f16b97fa640 1 -- 172.21.15.70:0/4132764293 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) v1 ==== 122+0+0 (secure 0 0 0) 0x7f16bc0ccd00 con 0x7f16bc161690 2023-10-08T06:53:50.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.897+0000 7f16c287c640 1 -- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f169c06af90 msgr2=0x7f169c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:50.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.897+0000 7f16c287c640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f169c06af90 0x7f169c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f16ac006fd0 tx=0x7f16ac008040 comp rx=0 tx=0).stop 2023-10-08T06:53:50.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.897+0000 7f16c287c640 1 -- 172.21.15.70:0/4132764293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc161690 msgr2=0x7f16bc163f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:50.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.897+0000 7f16c287c640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc161690 0x7f16bc163f20 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f16b400c7e0 tx=0x7f16b400cca0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.898+0000 7f16c287c640 1 -- 172.21.15.70:0/4132764293 shutdown_connections 2023-10-08T06:53:50.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.898+0000 7f16c287c640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f169c06af90 0x7f169c06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.898+0000 7f16c287c640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc161690 0x7f16bc163f20 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.898+0000 7f16c287c640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16bc0cb150 0x7f16bc1639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.898+0000 7f16c287c640 1 --2- 172.21.15.70:0/4132764293 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16bc0702b0 0x7f16bc0c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:50.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.898+0000 7f16c287c640 1 -- 172.21.15.70:0/4132764293 >> 172.21.15.70:0/4132764293 conn(0x7f16bc06a5d0 msgr2=0x7f16bc160c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:50.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.898+0000 7f16c287c640 1 -- 172.21.15.70:0/4132764293 shutdown_connections 2023-10-08T06:53:50.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:50.898+0000 7f16c287c640 1 -- 172.21.15.70:0/4132764293 wait complete. 2023-10-08T06:53:50.920 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T06:53:50.920 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:53:51.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.134+0000 7fab8f390640 1 Processor -- start 2023-10-08T06:53:51.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.134+0000 7fab8f390640 1 -- start start 2023-10-08T06:53:51.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.136+0000 7fab8f390640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab8815cd90 0x7fab880702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.136+0000 7fab8f390640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8806c890 0x7fab8806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.136+0000 7fab8f390640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab880c48f0 con 0x7fab88160940 2023-10-08T06:53:51.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.136+0000 7fab8f390640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab880c4d80 con 0x7fab8815cd90 2023-10-08T06:53:51.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.136+0000 7fab8f390640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab880c3ce0 con 0x7fab8806c890 2023-10-08T06:53:51.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.136+0000 7fab8d12c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab8815cd90 0x7fab880702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.136+0000 7fab8d12c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab8815cd90 0x7fab880702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48264/0 (socket says 172.21.15.70:48264) 2023-10-08T06:53:51.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.136+0000 7fab8d12c640 1 -- 172.21.15.70:0/393114208 learned_addr learned my addr 172.21.15.70:0/393114208 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:51.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.136+0000 7fab8d92d640 1 --2- 172.21.15.70:0/393114208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8806c890 0x7fab8806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.137+0000 7fab8d12c640 1 -- 172.21.15.70:0/393114208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8806c890 msgr2=0x7fab8806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:51.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.137+0000 7fab8d12c640 1 --2- 172.21.15.70:0/393114208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8806c890 0x7fab8806cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.137+0000 7fab8d12c640 1 -- 172.21.15.70:0/393114208 >> v1:172.21.15.181:6789/0 conn(0x7fab88160940 legacy=0x7fab88070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:51.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.137+0000 7fab8d12c640 1 -- 172.21.15.70:0/393114208 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab880c3fc0 con 0x7fab8815cd90 2023-10-08T06:53:51.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.137+0000 7fab8d92d640 1 --2- 172.21.15.70:0/393114208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8806c890 0x7fab8806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:51.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.138+0000 7fab8d12c640 1 --2- 172.21.15.70:0/393114208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab8815cd90 0x7fab880702d0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fab84009870 tx=0x7fab8402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee286e3f88363e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:51.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.138+0000 7fab7ffff640 1 -- 172.21.15.70:0/393114208 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84031640 con 0x7fab8815cd90 2023-10-08T06:53:51.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.138+0000 7fab7ffff640 1 -- 172.21.15.70:0/393114208 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab840317a0 con 0x7fab8815cd90 2023-10-08T06:53:51.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.138+0000 7fab7ffff640 1 -- 172.21.15.70:0/393114208 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84031b30 con 0x7fab8815cd90 2023-10-08T06:53:51.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.138+0000 7fab8f390640 1 -- 172.21.15.70:0/393114208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab8815cd90 msgr2=0x7fab880702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:51.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.138+0000 7fab8f390640 1 --2- 172.21.15.70:0/393114208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab8815cd90 0x7fab880702d0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fab84009870 tx=0x7fab8402b8f0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.139+0000 7fab8f390640 1 -- 172.21.15.70:0/393114208 shutdown_connections 2023-10-08T06:53:51.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.139+0000 7fab8f390640 1 --2- 172.21.15.70:0/393114208 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8806c890 0x7fab8806cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.139+0000 7fab8f390640 1 --2- 172.21.15.70:0/393114208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab8815cd90 0x7fab880702d0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.139+0000 7fab8f390640 1 -- 172.21.15.70:0/393114208 >> 172.21.15.70:0/393114208 conn(0x7fab8806a380 msgr2=0x7fab880b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:51.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.139+0000 7fab8f390640 1 -- 172.21.15.70:0/393114208 shutdown_connections 2023-10-08T06:53:51.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.139+0000 7fab8f390640 1 -- 172.21.15.70:0/393114208 wait complete. 2023-10-08T06:53:51.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.140+0000 7fab8f390640 1 Processor -- start 2023-10-08T06:53:51.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.141+0000 7fab8f390640 1 -- start start 2023-10-08T06:53:51.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8f390640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab8806c890 0x7fab8814de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8f390640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8815cd90 0x7fab880c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8f390640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab88160940 0x7fab880ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8f390640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab880c5ca0 con 0x7fab88160940 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8f390640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8815e070 con 0x7fab8806c890 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8f390640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8815e350 con 0x7fab8815cd90 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8d92d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab88160940 0x7fab880ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8d92d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab88160940 0x7fab880ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48274/0 (socket says 172.21.15.70:48274) 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8d92d640 1 -- 172.21.15.70:0/1038580582 learned_addr learned my addr 172.21.15.70:0/1038580582 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8c92b640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8815cd90 0x7fab880c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8d12c640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab8806c890 0x7fab8814de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8d92d640 1 -- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8815cd90 msgr2=0x7fab880c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:51.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8d92d640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8815cd90 0x7fab880c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8d92d640 1 -- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab8806c890 msgr2=0x7fab8814de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8d92d640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab8806c890 0x7fab8814de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8d92d640 1 -- 172.21.15.70:0/1038580582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab840094d0 con 0x7fab88160940 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8c92b640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8815cd90 0x7fab880c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.142+0000 7fab8d12c640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab8806c890 0x7fab8814de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.143+0000 7fab8d92d640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab88160940 0x7fab880ae320 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fab8000cef0 tx=0x7fab8000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.143+0000 7fab7dffb640 1 -- 172.21.15.70:0/1038580582 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8000d840 con 0x7fab88160940 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.143+0000 7fab7dffb640 1 -- 172.21.15.70:0/1038580582 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8000d9c0 con 0x7fab88160940 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.143+0000 7fab7dffb640 1 -- 172.21.15.70:0/1038580582 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8000dd70 con 0x7fab88160940 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.143+0000 7fab8f390640 1 -- 172.21.15.70:0/1038580582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab8815e630 con 0x7fab88160940 2023-10-08T06:53:51.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.143+0000 7fab8f390640 1 -- 172.21.15.70:0/1038580582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab880c8d90 con 0x7fab88160940 2023-10-08T06:53:51.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.144+0000 7fab8f390640 1 -- 172.21.15.70:0/1038580582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8815e820 con 0x7fab88160940 2023-10-08T06:53:51.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.145+0000 7fab7dffb640 1 -- 172.21.15.70:0/1038580582 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fab8001e3b0 con 0x7fab88160940 2023-10-08T06:53:51.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.146+0000 7fab7dffb640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fab6c06af90 0x7fab6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.146+0000 7fab7dffb640 1 -- 172.21.15.70:0/1038580582 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fab8008a490 con 0x7fab88160940 2023-10-08T06:53:51.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.147+0000 7fab8d12c640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fab6c06af90 0x7fab6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.148+0000 7fab8d12c640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fab6c06af90 0x7fab6c06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fab84036000 tx=0x7fab8403c040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:51.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.150+0000 7fab7dffb640 1 -- 172.21.15.70:0/1038580582 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab80055290 con 0x7fab88160940 2023-10-08T06:53:51.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.373+0000 7fab8f390640 1 -- 172.21.15.70:0/1038580582 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab88067490 con 0x7fab88160940 2023-10-08T06:53:51.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.374+0000 7fab7dffb640 1 -- 172.21.15.70:0/1038580582 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4917 (secure 0 0 0) 0x7fab800588b0 con 0x7fab88160940 2023-10-08T06:53:51.375 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:51.375 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/542411062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":542411062},{"type":"v1","addr":"172.21.15.181:6836","nonce":542411062}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/646194010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":646194010},{"type":"v1","addr":"172.21.15.70:6835","nonce":646194010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1850202608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1850202608},{"type":"v1","addr":"172.21.15.181:6837","nonce":1850202608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:53:47.285813+0000","modified":"2023-10-08T06:53:50.356439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9591":{"gid":9591,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/3790014600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3790014600},{"type":"v1","addr":"172.21.15.70:6834","nonce":3790014600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-08T06:53:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 83 2023-10-08T06:53:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.376+0000 7fab8f390640 1 -- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fab6c06af90 msgr2=0x7fab6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.376+0000 7fab8f390640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fab6c06af90 0x7fab6c06d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fab84036000 tx=0x7fab8403c040 comp rx=0 tx=0).stop 2023-10-08T06:53:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.376+0000 7fab8f390640 1 -- 172.21.15.70:0/1038580582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab88160940 msgr2=0x7fab880ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.376+0000 7fab8f390640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab88160940 0x7fab880ae320 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fab8000cef0 tx=0x7fab8000a710 comp rx=0 tx=0).stop 2023-10-08T06:53:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.376+0000 7fab8f390640 1 -- 172.21.15.70:0/1038580582 shutdown_connections 2023-10-08T06:53:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.376+0000 7fab8f390640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fab6c06af90 0x7fab6c06d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.376+0000 7fab8f390640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab88160940 0x7fab880ae320 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.376+0000 7fab8f390640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fab8815cd90 0x7fab880c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.377+0000 7fab8f390640 1 --2- 172.21.15.70:0/1038580582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fab8806c890 0x7fab8814de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.377+0000 7fab8f390640 1 -- 172.21.15.70:0/1038580582 >> 172.21.15.70:0/1038580582 conn(0x7fab8806a380 msgr2=0x7fab880cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:51.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.377+0000 7fab8f390640 1 -- 172.21.15.70:0/1038580582 shutdown_connections 2023-10-08T06:53:51.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.377+0000 7fab8f390640 1 -- 172.21.15.70:0/1038580582 wait complete. 2023-10-08T06:53:51.395 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:53:51.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.598+0000 7f38fc859640 1 Processor -- start 2023-10-08T06:53:51.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.598+0000 7f38fc859640 1 -- start start 2023-10-08T06:53:51.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.600+0000 7f38fc859640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38f8150440 0x7f38f8071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.600+0000 7f38fc859640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38f80c97d0 0x7f38f80b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.600+0000 7f38fc859640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80c9f90 con 0x7f38f815ce40 2023-10-08T06:53:51.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.600+0000 7f38fc859640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f8071550 con 0x7f38f8150440 2023-10-08T06:53:51.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.600+0000 7f38fc859640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f80ca270 con 0x7f38f80c97d0 2023-10-08T06:53:51.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.600+0000 7f38f77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38f8150440 0x7f38f8071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.600+0000 7f38f77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38f8150440 0x7f38f8071010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48286/0 (socket says 172.21.15.70:48286) 2023-10-08T06:53:51.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.600+0000 7f38f77fe640 1 -- 172.21.15.70:0/3723388252 learned_addr learned my addr 172.21.15.70:0/3723388252 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:51.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.601+0000 7f38f77fe640 1 -- 172.21.15.70:0/3723388252 >> v2:172.21.15.181:3300/0 conn(0x7f38f80c97d0 msgr2=0x7f38f80b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:51.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.601+0000 7f38f77fe640 1 --2- 172.21.15.70:0/3723388252 >> v2:172.21.15.181:3300/0 conn(0x7f38f80c97d0 0x7f38f80b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.601+0000 7f38f77fe640 1 -- 172.21.15.70:0/3723388252 >> v1:172.21.15.181:6789/0 conn(0x7f38f815ce40 legacy=0x7f38f815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:51.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.601+0000 7f38f77fe640 1 -- 172.21.15.70:0/3723388252 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80ca550 con 0x7f38f8150440 2023-10-08T06:53:51.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.602+0000 7f38f77fe640 1 --2- 172.21.15.70:0/3723388252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f8150440 0x7f38f8071010 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f38f80b0fc0 tx=0x7f38e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1e37bec143ee44 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:51.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.602+0000 7f38f6ffd640 1 -- 172.21.15.70:0/3723388252 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8031870 con 0x7f38f8150440 2023-10-08T06:53:51.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.602+0000 7f38f6ffd640 1 -- 172.21.15.70:0/3723388252 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e80319d0 con 0x7f38f8150440 2023-10-08T06:53:51.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.602+0000 7f38f6ffd640 1 -- 172.21.15.70:0/3723388252 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8031d60 con 0x7f38f8150440 2023-10-08T06:53:51.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.603+0000 7f38fc859640 1 -- 172.21.15.70:0/3723388252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f8150440 msgr2=0x7f38f8071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:51.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.603+0000 7f38fc859640 1 --2- 172.21.15.70:0/3723388252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f8150440 0x7f38f8071010 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f38f80b0fc0 tx=0x7f38e802b910 comp rx=0 tx=0).stop 2023-10-08T06:53:51.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.603+0000 7f38fc859640 1 -- 172.21.15.70:0/3723388252 shutdown_connections 2023-10-08T06:53:51.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.603+0000 7f38fc859640 1 --2- 172.21.15.70:0/3723388252 >> v2:172.21.15.181:3300/0 conn(0x7f38f80c97d0 0x7f38f80b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.603+0000 7f38fc859640 1 --2- 172.21.15.70:0/3723388252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f8150440 0x7f38f8071010 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.603+0000 7f38fc859640 1 -- 172.21.15.70:0/3723388252 >> 172.21.15.70:0/3723388252 conn(0x7f38f80cb0c0 msgr2=0x7f38f80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:51.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.603+0000 7f38fc859640 1 -- 172.21.15.70:0/3723388252 shutdown_connections 2023-10-08T06:53:51.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.603+0000 7f38fc859640 1 -- 172.21.15.70:0/3723388252 wait complete. 2023-10-08T06:53:51.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.605+0000 7f38fc859640 1 Processor -- start 2023-10-08T06:53:51.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.605+0000 7f38fc859640 1 -- start start 2023-10-08T06:53:51.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.606+0000 7f38fc859640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f80c97d0 0x7f38f814eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.606+0000 7f38fc859640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f8150440 0x7f38f81589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.606+0000 7f38fc859640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f815ce40 0x7f38f81f06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.606+0000 7f38fc859640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8150d00 con 0x7f38f815ce40 2023-10-08T06:53:51.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.606+0000 7f38fc859640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8150fe0 con 0x7f38f80c97d0 2023-10-08T06:53:51.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.606+0000 7f38fc859640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f81512c0 con 0x7f38f8150440 2023-10-08T06:53:51.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.606+0000 7f38f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f815ce40 0x7f38f81f06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.606+0000 7f38f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f815ce40 0x7f38f81f06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48302/0 (socket says 172.21.15.70:48302) 2023-10-08T06:53:51.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.606+0000 7f38f7fff640 1 -- 172.21.15.70:0/3313235711 learned_addr learned my addr 172.21.15.70:0/3313235711 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:51.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38f77fe640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f80c97d0 0x7f38f814eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38effff640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f8150440 0x7f38f81589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38f7fff640 1 -- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f8150440 msgr2=0x7f38f81589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:51.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38f7fff640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f8150440 0x7f38f81589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38f7fff640 1 -- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f80c97d0 msgr2=0x7f38f814eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:51.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38f7fff640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f80c97d0 0x7f38f814eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38f7fff640 1 -- 172.21.15.70:0/3313235711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80094d0 con 0x7f38f815ce40 2023-10-08T06:53:51.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38f7fff640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f815ce40 0x7f38f81f06f0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f38f000c780 tx=0x7f38f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:51.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38f4ff9640 1 -- 172.21.15.70:0/3313235711 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f0003d80 con 0x7f38f815ce40 2023-10-08T06:53:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38f4ff9640 1 -- 172.21.15.70:0/3313235711 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f000ce90 con 0x7f38f815ce40 2023-10-08T06:53:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38f4ff9640 1 -- 172.21.15.70:0/3313235711 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f00052b0 con 0x7f38f815ce40 2023-10-08T06:53:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38fc859640 1 -- 172.21.15.70:0/3313235711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f81515a0 con 0x7f38f815ce40 2023-10-08T06:53:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.607+0000 7f38fc859640 1 -- 172.21.15.70:0/3313235711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f8158ee0 con 0x7f38f815ce40 2023-10-08T06:53:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.608+0000 7f38fc859640 1 -- 172.21.15.70:0/3313235711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f8151ac0 con 0x7f38f815ce40 2023-10-08T06:53:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.610+0000 7f38f4ff9640 1 -- 172.21.15.70:0/3313235711 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f38f0005410 con 0x7f38f815ce40 2023-10-08T06:53:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.610+0000 7f38f4ff9640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f38d806af90 0x7f38d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.610+0000 7f38f4ff9640 1 -- 172.21.15.70:0/3313235711 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f38f0088ec0 con 0x7f38f815ce40 2023-10-08T06:53:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.611+0000 7f38f77fe640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f38d806af90 0x7f38d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:51.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.612+0000 7f38f77fe640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f38d806af90 0x7f38d806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38e8036000 tx=0x7f38e803c040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:51.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.615+0000 7f38f4ff9640 1 -- 172.21.15.70:0/3313235711 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38f8151ac0 con 0x7f38f815ce40 2023-10-08T06:53:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.804+0000 7f38fc859640 1 -- 172.21.15.70:0/3313235711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38f8151ac0 con 0x7f38f815ce40 2023-10-08T06:53:51.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.806+0000 7f38f4ff9640 1 -- 172.21.15.70:0/3313235711 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+16898 (secure 0 0 0) 0x7f38f8151ac0 con 0x7f38f815ce40 2023-10-08T06:53:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:51.807 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":185,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:53:48.859604+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:53:44.876671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-08T06:53:46.310621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:53:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 -- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f38d806af90 msgr2=0x7f38d806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f38d806af90 0x7f38d806d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f38e8036000 tx=0x7f38e803c040 comp rx=0 tx=0).stop 2023-10-08T06:53:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 -- 172.21.15.70:0/3313235711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f815ce40 msgr2=0x7f38f81f06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f815ce40 0x7f38f81f06f0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f38f000c780 tx=0x7f38f000cc40 comp rx=0 tx=0).stop 2023-10-08T06:53:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 -- 172.21.15.70:0/3313235711 shutdown_connections 2023-10-08T06:53:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f38d806af90 0x7f38d806d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f815ce40 0x7f38f81f06f0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38f8150440 0x7f38f81589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 --2- 172.21.15.70:0/3313235711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f38f80c97d0 0x7f38f814eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:51.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 -- 172.21.15.70:0/3313235711 >> 172.21.15.70:0/3313235711 conn(0x7f38f80cb0c0 msgr2=0x7f38f815c540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:51.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 -- 172.21.15.70:0/3313235711 shutdown_connections 2023-10-08T06:53:51.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:51.809+0000 7f38fc859640 1 -- 172.21.15.70:0/3313235711 wait complete. 2023-10-08T06:53:51.828 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T06:53:51.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T06:53:52.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.043+0000 7f315097c640 1 Processor -- start 2023-10-08T06:53:52.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.044+0000 7f315097c640 1 -- start start 2023-10-08T06:53:52.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.045+0000 7f315097c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f314c157000 0x7f314c0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.045+0000 7f315097c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f314c0cb190 0x7f314c0cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.045+0000 7f315097c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314c0adfc0 con 0x7f314c1597d0 2023-10-08T06:53:52.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.045+0000 7f315097c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314c0cbb80 con 0x7f314c0cb190 2023-10-08T06:53:52.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.045+0000 7f315097c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0ccce0 con 0x7f314c157000 2023-10-08T06:53:52.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.045+0000 7f3149d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f314c1597d0 0x7f314c06ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43980/0 (socket says 172.21.15.70:43980) 2023-10-08T06:53:52.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.045+0000 7f3149d9b640 1 -- 172.21.15.70:0/4070357200 learned_addr learned my addr 172.21.15.70:0/4070357200 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:52.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314a59c640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f314c157000 0x7f314c0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314ad9d640 1 --2- 172.21.15.70:0/4070357200 >> v2:172.21.15.181:3300/0 conn(0x7f314c0cb190 0x7f314c0cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124159416 0 0) 0x7f314c0adfc0 con 0x7f314c1597d0 2023-10-08T06:53:52.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3134003520 con 0x7f314c1597d0 2023-10-08T06:53:52.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1541407399 0 0) 0x7f3134003520 con 0x7f314c1597d0 2023-10-08T06:53:52.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f314c157000 msgr2=0x7f314c0cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:52.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f314c157000 0x7f314c0cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 >> v2:172.21.15.181:3300/0 conn(0x7f314c0cb190 msgr2=0x7f314c0cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:52.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 --2- 172.21.15.70:0/4070357200 >> v2:172.21.15.181:3300/0 conn(0x7f314c0cb190 0x7f314c0cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c159cb0 con 0x7f314c1597d0 2023-10-08T06:53:52.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3140001670 con 0x7f314c1597d0 2023-10-08T06:53:52.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 >> v1:172.21.15.70:6789/0 conn(0x7f314c1597d0 legacy=0x7f314c06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3134003970 0x7f3134003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31340042d0 0x7f31340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.046+0000 7f314959a640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 0x7f3134005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0cbb80 con 0x7f3134004e10 2023-10-08T06:53:52.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0ccce0 con 0x7f31340042d0 2023-10-08T06:53:52.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313400bd00 con 0x7f3134003970 2023-10-08T06:53:52.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f314ad9d640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 0x7f3134005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f3149d9b640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31340042d0 0x7f31340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f314a59c640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3134003970 0x7f3134003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f314ad9d640 1 -- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3134003970 msgr2=0x7f3134003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:52.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f314ad9d640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3134003970 0x7f3134003d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f314ad9d640 1 -- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31340042d0 msgr2=0x7f31340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:52.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f314ad9d640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31340042d0 0x7f31340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.047+0000 7f314ad9d640 1 -- 172.21.15.70:0/4070357200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3134003520 con 0x7f3134004e10 2023-10-08T06:53:52.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.048+0000 7f314ad9d640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 0x7f3134005290 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3144009fc0 tx=0x7f31340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccb3c3bb2953e41 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:52.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.049+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314402ee70 con 0x7f3134004e10 2023-10-08T06:53:52.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.049+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314400f040 con 0x7f3134004e10 2023-10-08T06:53:52.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.049+0000 7f314959a640 1 -- 172.21.15.70:0/4070357200 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3144036680 con 0x7f3134004e10 2023-10-08T06:53:52.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.049+0000 7f315097c640 1 -- 172.21.15.70:0/4070357200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 msgr2=0x7f3134005290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.049+0000 7f315097c640 1 --2- 172.21.15.70:0/4070357200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 0x7f3134005290 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3144009fc0 tx=0x7f31340057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.049+0000 7f314ad9d640 1 -- 172.21.15.70:0/4070357200 reap_dead start 2023-10-08T06:53:52.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.049+0000 7f315097c640 1 -- 172.21.15.70:0/4070357200 shutdown_connections 2023-10-08T06:53:52.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.049+0000 7f315097c640 1 -- 172.21.15.70:0/4070357200 >> 172.21.15.70:0/4070357200 conn(0x7f314c06a380 msgr2=0x7f314c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:52.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.049+0000 7f315097c640 1 -- 172.21.15.70:0/4070357200 shutdown_connections 2023-10-08T06:53:52.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.049+0000 7f315097c640 1 -- 172.21.15.70:0/4070357200 wait complete. 2023-10-08T06:53:52.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.051+0000 7f315097c640 1 Processor -- start 2023-10-08T06:53:52.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.051+0000 7f315097c640 1 -- start start 2023-10-08T06:53:52.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.052+0000 7f315097c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 0x7f314c0b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.052+0000 7f315097c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f314c0b0390 0x7f314c1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.052+0000 7f315097c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f314c0b3000 0x7f314c0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.052+0000 7f315097c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0b39c0 con 0x7f3134004e10 2023-10-08T06:53:52.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.052+0000 7f315097c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c1f20d0 con 0x7f314c0b0390 2023-10-08T06:53:52.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.052+0000 7f315097c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c155d30 con 0x7f314c0b3000 2023-10-08T06:53:52.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.052+0000 7f314a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 0x7f314c0b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.052+0000 7f314a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 0x7f314c0b12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48320/0 (socket says 172.21.15.70:48320) 2023-10-08T06:53:52.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.052+0000 7f314a59c640 1 -- 172.21.15.70:0/474150367 learned_addr learned my addr 172.21.15.70:0/474150367 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:52.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f3149d9b640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f314c0b0390 0x7f314c1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f314ad9d640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f314c0b3000 0x7f314c0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f314a59c640 1 -- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f314c0b3000 msgr2=0x7f314c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f314a59c640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f314c0b3000 0x7f314c0b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f314a59c640 1 -- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f314c0b0390 msgr2=0x7f314c1541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f314a59c640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f314c0b0390 0x7f314c1541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f314a59c640 1 -- 172.21.15.70:0/474150367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c0b19b0 con 0x7f3134004e10 2023-10-08T06:53:52.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f314a59c640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 0x7f314c0b12b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f313c010890 tx=0x7f313c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:52.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f313affd640 1 -- 172.21.15.70:0/474150367 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c009610 con 0x7f3134004e10 2023-10-08T06:53:52.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f313affd640 1 -- 172.21.15.70:0/474150367 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c00eb60 con 0x7f3134004e10 2023-10-08T06:53:52.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f313affd640 1 -- 172.21.15.70:0/474150367 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c01c400 con 0x7f3134004e10 2023-10-08T06:53:52.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.053+0000 7f315097c640 1 -- 172.21.15.70:0/474150367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c0b1c90 con 0x7f3134004e10 2023-10-08T06:53:52.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.054+0000 7f315097c640 1 -- 172.21.15.70:0/474150367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c0b2250 con 0x7f3134004e10 2023-10-08T06:53:52.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.055+0000 7f315097c640 1 -- 172.21.15.70:0/474150367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314c1f2210 con 0x7f3134004e10 2023-10-08T06:53:52.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.056+0000 7f313affd640 1 -- 172.21.15.70:0/474150367 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f313c01c560 con 0x7f3134004e10 2023-10-08T06:53:52.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.057+0000 7f313affd640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f312c06afc0 0x7f312c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.057+0000 7f313affd640 1 -- 172.21.15.70:0/474150367 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f313c089e10 con 0x7f3134004e10 2023-10-08T06:53:52.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.058+0000 7f3149d9b640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f312c06afc0 0x7f312c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.059+0000 7f3149d9b640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f312c06afc0 0x7f312c06d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3140001050 tx=0x7f3140000fe0 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:52.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.061+0000 7f313affd640 1 -- 172.21.15.70:0/474150367 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f313c054d30 con 0x7f3134004e10 2023-10-08T06:53:52.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.235+0000 7f315097c640 1 -- 172.21.15.70:0/474150367 --> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f314c159f90 con 0x7f312c06afc0 2023-10-08T06:53:52.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.262+0000 7f313affd640 1 -- 172.21.15.70:0/474150367 <== mgr.9345 v2:172.21.15.181:6832/4245513204 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282075 (secure 0 0 0) 0x7f314000f760 con 0x7f312c06afc0 2023-10-08T06:53:52.268 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:52.270 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T06:53:52.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 -- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f312c06afc0 msgr2=0x7f312c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f312c06afc0 0x7f312c06d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3140001050 tx=0x7f3140000fe0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 -- 172.21.15.70:0/474150367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 msgr2=0x7f314c0b12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 0x7f314c0b12b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f313c010890 tx=0x7f313c010d50 comp rx=0 tx=0).stop 2023-10-08T06:53:52.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 -- 172.21.15.70:0/474150367 shutdown_connections 2023-10-08T06:53:52.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f314c0b3000 0x7f314c0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f314c0b0390 0x7f314c1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f312c06afc0 0x7f312c06d480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 --2- 172.21.15.70:0/474150367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134004e10 0x7f314c0b12b0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 -- 172.21.15.70:0/474150367 >> 172.21.15.70:0/474150367 conn(0x7f314c06a380 msgr2=0x7f314c0af300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:52.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 -- 172.21.15.70:0/474150367 shutdown_connections 2023-10-08T06:53:52.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.268+0000 7f315097c640 1 -- 172.21.15.70:0/474150367 wait complete. 2023-10-08T06:53:52.289 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-08T06:53:52.078361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365512,"kb_used_data":18920,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754609208,"statfs":{"total":773094113280,"available":772719828992,"internally_reserved":0,"allocated":19374080,"data_stored":10530744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":354799163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001853"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837391+0000","last_change":"2023-10-08T06:53:46.837391+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837391+0000","last_undegraded":"2023-10-08T06:53:46.837391+0000","last_fullsized":"2023-10-08T06:53:46.837391+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835340+0000","last_change":"2023-10-08T06:53:46.835340+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835340+0000","last_undegraded":"2023-10-08T06:53:46.835340+0000","last_fullsized":"2023-10-08T06:53:46.835340+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836968+0000","last_change":"2023-10-08T06:53:46.836968+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836968+0000","last_undegraded":"2023-10-08T06:53:46.836968+0000","last_fullsized":"2023-10-08T06:53:46.836968+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832209+0000","last_change":"2023-10-08T06:53:46.832209+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832209+0000","last_undegraded":"2023-10-08T06:53:46.832209+0000","last_fullsized":"2023-10-08T06:53:46.832209+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830068+0000","last_change":"2023-10-08T06:53:46.830068+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830068+0000","last_undegraded":"2023-10-08T06:53:46.830068+0000","last_fullsized":"2023-10-08T06:53:46.830068+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830196+0000","last_change":"2023-10-08T06:53:46.830196+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830196+0000","last_undegraded":"2023-10-08T06:53:46.830196+0000","last_fullsized":"2023-10-08T06:53:46.830196+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833262+0000","last_change":"2023-10-08T06:53:46.833262+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833262+0000","last_undegraded":"2023-10-08T06:53:46.833262+0000","last_fullsized":"2023-10-08T06:53:46.833262+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831994+0000","last_change":"2023-10-08T06:53:46.831994+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831994+0000","last_undegraded":"2023-10-08T06:53:46.831994+0000","last_fullsized":"2023-10-08T06:53:46.831994+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832152+0000","last_change":"2023-10-08T06:53:46.832152+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832152+0000","last_undegraded":"2023-10-08T06:53:46.832152+0000","last_fullsized":"2023-10-08T06:53:46.832152+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828877+0000","last_change":"2023-10-08T06:53:46.828877+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828877+0000","last_undegraded":"2023-10-08T06:53:46.828877+0000","last_fullsized":"2023-10-08T06:53:46.828877+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826488+0000","last_change":"2023-10-08T06:53:46.826488+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826488+0000","last_undegraded":"2023-10-08T06:53:46.826488+0000","last_fullsized":"2023-10-08T06:53:46.826488+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831044+0000","last_change":"2023-10-08T06:53:46.831044+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831044+0000","last_undegraded":"2023-10-08T06:53:46.831044+0000","last_fullsized":"2023-10-08T06:53:46.831044+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831261+0000","last_change":"2023-10-08T06:53:46.831261+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831261+0000","last_undegraded":"2023-10-08T06:53:46.831261+0000","last_fullsized":"2023-10-08T06:53:46.831261+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830002+0000","last_change":"2023-10-08T06:53:46.830002+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830002+0000","last_undegraded":"2023-10-08T06:53:46.830002+0000","last_fullsized":"2023-10-08T06:53:46.830002+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833676+0000","last_change":"2023-10-08T06:53:46.833676+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833676+0000","last_undegraded":"2023-10-08T06:53:46.833676+0000","last_fullsized":"2023-10-08T06:53:46.833676+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831113+0000","last_change":"2023-10-08T06:53:46.831113+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831113+0000","last_undegraded":"2023-10-08T06:53:46.831113+0000","last_fullsized":"2023-10-08T06:53:46.831113+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.834745+0000","last_change":"2023-10-08T06:53:46.834745+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.834745+0000","last_undegraded":"2023-10-08T06:53:46.834745+0000","last_fullsized":"2023-10-08T06:53:46.834745+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825966+0000","last_change":"2023-10-08T06:53:46.825966+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825966+0000","last_undegraded":"2023-10-08T06:53:46.825966+0000","last_fullsized":"2023-10-08T06:53:46.825966+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.841643+0000","last_change":"2023-10-08T06:53:46.841643+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.841643+0000","last_undegraded":"2023-10-08T06:53:46.841643+0000","last_fullsized":"2023-10-08T06:53:46.841643+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826979+0000","last_change":"2023-10-08T06:53:46.826979+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826979+0000","last_undegraded":"2023-10-08T06:53:46.826979+0000","last_fullsized":"2023-10-08T06:53:46.826979+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830751+0000","last_change":"2023-10-08T06:53:46.830751+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830751+0000","last_undegraded":"2023-10-08T06:53:46.830751+0000","last_fullsized":"2023-10-08T06:53:46.830751+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836034+0000","last_change":"2023-10-08T06:53:46.836034+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836034+0000","last_undegraded":"2023-10-08T06:53:46.836034+0000","last_fullsized":"2023-10-08T06:53:46.836034+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826700+0000","last_change":"2023-10-08T06:53:46.826700+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826700+0000","last_undegraded":"2023-10-08T06:53:46.826700+0000","last_fullsized":"2023-10-08T06:53:46.826700+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825977+0000","last_change":"2023-10-08T06:53:46.825977+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825977+0000","last_undegraded":"2023-10-08T06:53:46.825977+0000","last_fullsized":"2023-10-08T06:53:46.825977+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836905+0000","last_change":"2023-10-08T06:53:46.836905+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836905+0000","last_undegraded":"2023-10-08T06:53:46.836905+0000","last_fullsized":"2023-10-08T06:53:46.836905+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.838574+0000","last_change":"2023-10-08T06:53:46.838574+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.838574+0000","last_undegraded":"2023-10-08T06:53:46.838574+0000","last_fullsized":"2023-10-08T06:53:46.838574+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828887+0000","last_change":"2023-10-08T06:53:46.828887+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828887+0000","last_undegraded":"2023-10-08T06:53:46.828887+0000","last_fullsized":"2023-10-08T06:53:46.828887+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825937+0000","last_change":"2023-10-08T06:53:46.825937+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825937+0000","last_undegraded":"2023-10-08T06:53:46.825937+0000","last_fullsized":"2023-10-08T06:53:46.825937+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831793+0000","last_change":"2023-10-08T06:53:46.831793+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831793+0000","last_undegraded":"2023-10-08T06:53:46.831793+0000","last_fullsized":"2023-10-08T06:53:46.831793+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825313+0000","last_change":"2023-10-08T06:53:46.825313+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825313+0000","last_undegraded":"2023-10-08T06:53:46.825313+0000","last_fullsized":"2023-10-08T06:53:46.825313+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828751+0000","last_change":"2023-10-08T06:53:46.828751+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828751+0000","last_undegraded":"2023-10-08T06:53:46.828751+0000","last_fullsized":"2023-10-08T06:53:46.828751+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831002+0000","last_change":"2023-10-08T06:53:46.831002+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831002+0000","last_undegraded":"2023-10-08T06:53:46.831002+0000","last_fullsized":"2023-10-08T06:53:46.831002+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837790+0000","last_change":"2023-10-08T06:53:46.837790+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837790+0000","last_undegraded":"2023-10-08T06:53:46.837790+0000","last_fullsized":"2023-10-08T06:53:46.837790+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.841915+0000","last_change":"2023-10-08T06:53:46.838837+0000","last_active":"2023-10-08T06:53:46.841915+0000","last_peered":"2023-10-08T06:53:46.841915+0000","last_clean":"2023-10-08T06:53:46.841915+0000","last_became_active":"2023-10-08T06:53:46.838366+0000","last_became_peered":"2023-10-08T06:53:46.838366+0000","last_unstale":"2023-10-08T06:53:46.841915+0000","last_undegraded":"2023-10-08T06:53:46.841915+0000","last_fullsized":"2023-10-08T06:53:46.841915+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:26.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836525+0000","last_change":"2023-10-08T06:53:46.836525+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836525+0000","last_undegraded":"2023-10-08T06:53:46.836525+0000","last_fullsized":"2023-10-08T06:53:46.836525+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855058+0000","last_change":"2023-10-08T06:53:46.854455+0000","last_active":"2023-10-08T06:53:46.855058+0000","last_peered":"2023-10-08T06:53:46.855058+0000","last_clean":"2023-10-08T06:53:46.855058+0000","last_became_active":"2023-10-08T06:53:46.854195+0000","last_became_peered":"2023-10-08T06:53:46.854195+0000","last_unstale":"2023-10-08T06:53:46.855058+0000","last_undegraded":"2023-10-08T06:53:46.855058+0000","last_fullsized":"2023-10-08T06:53:46.855058+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:32.390630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832812+0000","last_change":"2023-10-08T06:53:46.832812+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832812+0000","last_undegraded":"2023-10-08T06:53:46.832812+0000","last_fullsized":"2023-10-08T06:53:46.832812+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.854602+0000","last_change":"2023-10-08T06:53:46.853301+0000","last_active":"2023-10-08T06:53:46.854602+0000","last_peered":"2023-10-08T06:53:46.854602+0000","last_clean":"2023-10-08T06:53:46.854602+0000","last_became_active":"2023-10-08T06:53:46.853014+0000","last_became_peered":"2023-10-08T06:53:46.853014+0000","last_unstale":"2023-10-08T06:53:46.854602+0000","last_undegraded":"2023-10-08T06:53:46.854602+0000","last_fullsized":"2023-10-08T06:53:46.854602+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:52.386749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.844177+0000","last_change":"2023-10-08T06:53:46.844177+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.844177+0000","last_undegraded":"2023-10-08T06:53:46.844177+0000","last_fullsized":"2023-10-08T06:53:46.844177+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855614+0000","last_change":"2023-10-08T06:53:46.854219+0000","last_active":"2023-10-08T06:53:46.855614+0000","last_peered":"2023-10-08T06:53:46.855614+0000","last_clean":"2023-10-08T06:53:46.855614+0000","last_became_active":"2023-10-08T06:53:46.853948+0000","last_became_peered":"2023-10-08T06:53:46.853948+0000","last_unstale":"2023-10-08T06:53:46.855614+0000","last_undegraded":"2023-10-08T06:53:46.855614+0000","last_fullsized":"2023-10-08T06:53:46.855614+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:36.674287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826941+0000","last_change":"2023-10-08T06:53:46.826941+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826941+0000","last_undegraded":"2023-10-08T06:53:46.826941+0000","last_fullsized":"2023-10-08T06:53:46.826941+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853355+0000","last_change":"2023-10-08T06:53:46.847384+0000","last_active":"2023-10-08T06:53:46.853355+0000","last_peered":"2023-10-08T06:53:46.853355+0000","last_clean":"2023-10-08T06:53:46.853355+0000","last_became_active":"2023-10-08T06:53:46.847114+0000","last_became_peered":"2023-10-08T06:53:46.847114+0000","last_unstale":"2023-10-08T06:53:46.853355+0000","last_undegraded":"2023-10-08T06:53:46.853355+0000","last_fullsized":"2023-10-08T06:53:46.853355+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:08.420118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831287+0000","last_change":"2023-10-08T06:53:46.831287+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831287+0000","last_undegraded":"2023-10-08T06:53:46.831287+0000","last_fullsized":"2023-10-08T06:53:46.831287+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.845097+0000","last_change":"2023-10-08T06:53:46.842021+0000","last_active":"2023-10-08T06:53:46.845097+0000","last_peered":"2023-10-08T06:53:46.845097+0000","last_clean":"2023-10-08T06:53:46.845097+0000","last_became_active":"2023-10-08T06:53:46.841767+0000","last_became_peered":"2023-10-08T06:53:46.841767+0000","last_unstale":"2023-10-08T06:53:46.845097+0000","last_undegraded":"2023-10-08T06:53:46.845097+0000","last_fullsized":"2023-10-08T06:53:46.845097+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:06.509561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835744+0000","last_change":"2023-10-08T06:53:46.835744+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835744+0000","last_undegraded":"2023-10-08T06:53:46.835744+0000","last_fullsized":"2023-10-08T06:53:46.835744+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.848938+0000","last_change":"2023-10-08T06:53:46.845053+0000","last_active":"2023-10-08T06:53:46.848938+0000","last_peered":"2023-10-08T06:53:46.848938+0000","last_clean":"2023-10-08T06:53:46.848938+0000","last_became_active":"2023-10-08T06:53:46.844706+0000","last_became_peered":"2023-10-08T06:53:46.844706+0000","last_unstale":"2023-10-08T06:53:46.848938+0000","last_undegraded":"2023-10-08T06:53:46.848938+0000","last_fullsized":"2023-10-08T06:53:46.848938+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:49.451557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.834908+0000","last_change":"2023-10-08T06:53:46.834908+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.834908+0000","last_undegraded":"2023-10-08T06:53:46.834908+0000","last_fullsized":"2023-10-08T06:53:46.834908+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.845369+0000","last_change":"2023-10-08T06:53:46.843744+0000","last_active":"2023-10-08T06:53:46.845369+0000","last_peered":"2023-10-08T06:53:46.845369+0000","last_clean":"2023-10-08T06:53:46.845369+0000","last_became_active":"2023-10-08T06:53:46.843504+0000","last_became_peered":"2023-10-08T06:53:46.843504+0000","last_unstale":"2023-10-08T06:53:46.845369+0000","last_undegraded":"2023-10-08T06:53:46.845369+0000","last_fullsized":"2023-10-08T06:53:46.845369+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:26.339899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828590+0000","last_change":"2023-10-08T06:53:46.828590+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828590+0000","last_undegraded":"2023-10-08T06:53:46.828590+0000","last_fullsized":"2023-10-08T06:53:46.828590+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.851054+0000","last_change":"2023-10-08T06:53:46.848518+0000","last_active":"2023-10-08T06:53:46.851054+0000","last_peered":"2023-10-08T06:53:46.851054+0000","last_clean":"2023-10-08T06:53:46.851054+0000","last_became_active":"2023-10-08T06:53:46.848237+0000","last_became_peered":"2023-10-08T06:53:46.848237+0000","last_unstale":"2023-10-08T06:53:46.851054+0000","last_undegraded":"2023-10-08T06:53:46.851054+0000","last_fullsized":"2023-10-08T06:53:46.851054+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:27:53.231805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825152+0000","last_change":"2023-10-08T06:53:46.825152+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825152+0000","last_undegraded":"2023-10-08T06:53:46.825152+0000","last_fullsized":"2023-10-08T06:53:46.825152+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.851503+0000","last_change":"2023-10-08T06:53:46.844822+0000","last_active":"2023-10-08T06:53:46.851503+0000","last_peered":"2023-10-08T06:53:46.851503+0000","last_clean":"2023-10-08T06:53:46.851503+0000","last_became_active":"2023-10-08T06:53:46.844540+0000","last_became_peered":"2023-10-08T06:53:46.844540+0000","last_unstale":"2023-10-08T06:53:46.851503+0000","last_undegraded":"2023-10-08T06:53:46.851503+0000","last_fullsized":"2023-10-08T06:53:46.851503+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:03.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825174+0000","last_change":"2023-10-08T06:53:46.825174+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825174+0000","last_undegraded":"2023-10-08T06:53:46.825174+0000","last_fullsized":"2023-10-08T06:53:46.825174+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.833708+0000","last_change":"2023-10-08T06:53:46.832905+0000","last_active":"2023-10-08T06:53:46.833708+0000","last_peered":"2023-10-08T06:53:46.833708+0000","last_clean":"2023-10-08T06:53:46.833708+0000","last_became_active":"2023-10-08T06:53:46.832624+0000","last_became_peered":"2023-10-08T06:53:46.832624+0000","last_unstale":"2023-10-08T06:53:46.833708+0000","last_undegraded":"2023-10-08T06:53:46.833708+0000","last_fullsized":"2023-10-08T06:53:46.833708+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:11.164730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830035+0000","last_change":"2023-10-08T06:53:46.830035+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830035+0000","last_undegraded":"2023-10-08T06:53:46.830035+0000","last_fullsized":"2023-10-08T06:53:46.830035+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.841506+0000","last_change":"2023-10-08T06:53:46.838423+0000","last_active":"2023-10-08T06:53:46.841506+0000","last_peered":"2023-10-08T06:53:46.841506+0000","last_clean":"2023-10-08T06:53:46.841506+0000","last_became_active":"2023-10-08T06:53:46.838071+0000","last_became_peered":"2023-10-08T06:53:46.838071+0000","last_unstale":"2023-10-08T06:53:46.841506+0000","last_undegraded":"2023-10-08T06:53:46.841506+0000","last_fullsized":"2023-10-08T06:53:46.841506+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:30.734197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833041+0000","last_change":"2023-10-08T06:53:46.833041+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833041+0000","last_undegraded":"2023-10-08T06:53:46.833041+0000","last_fullsized":"2023-10-08T06:53:46.833041+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.858068+0000","last_change":"2023-10-08T06:53:46.857001+0000","last_active":"2023-10-08T06:53:46.858068+0000","last_peered":"2023-10-08T06:53:46.858068+0000","last_clean":"2023-10-08T06:53:46.858068+0000","last_became_active":"2023-10-08T06:53:46.856707+0000","last_became_peered":"2023-10-08T06:53:46.856707+0000","last_unstale":"2023-10-08T06:53:46.858068+0000","last_undegraded":"2023-10-08T06:53:46.858068+0000","last_fullsized":"2023-10-08T06:53:46.858068+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:10.191719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.829716+0000","last_change":"2023-10-08T06:53:46.829716+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.829716+0000","last_undegraded":"2023-10-08T06:53:46.829716+0000","last_fullsized":"2023-10-08T06:53:46.829716+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.850126+0000","last_change":"2023-10-08T06:53:46.842178+0000","last_active":"2023-10-08T06:53:46.850126+0000","last_peered":"2023-10-08T06:53:46.850126+0000","last_clean":"2023-10-08T06:53:46.850126+0000","last_became_active":"2023-10-08T06:53:46.840731+0000","last_became_peered":"2023-10-08T06:53:46.840731+0000","last_unstale":"2023-10-08T06:53:46.850126+0000","last_undegraded":"2023-10-08T06:53:46.850126+0000","last_fullsized":"2023-10-08T06:53:46.850126+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:28.102115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835847+0000","last_change":"2023-10-08T06:53:46.835847+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835847+0000","last_undegraded":"2023-10-08T06:53:46.835847+0000","last_fullsized":"2023-10-08T06:53:46.835847+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.854676+0000","last_change":"2023-10-08T06:53:46.853950+0000","last_active":"2023-10-08T06:53:46.854676+0000","last_peered":"2023-10-08T06:53:46.854676+0000","last_clean":"2023-10-08T06:53:46.854676+0000","last_became_active":"2023-10-08T06:53:46.853668+0000","last_became_peered":"2023-10-08T06:53:46.853668+0000","last_unstale":"2023-10-08T06:53:46.854676+0000","last_undegraded":"2023-10-08T06:53:46.854676+0000","last_fullsized":"2023-10-08T06:53:46.854676+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:37.692117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.840640+0000","last_change":"2023-10-08T06:53:46.840640+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.840640+0000","last_undegraded":"2023-10-08T06:53:46.840640+0000","last_fullsized":"2023-10-08T06:53:46.840640+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.850594+0000","last_change":"2023-10-08T06:53:46.849784+0000","last_active":"2023-10-08T06:53:46.850594+0000","last_peered":"2023-10-08T06:53:46.850594+0000","last_clean":"2023-10-08T06:53:46.850594+0000","last_became_active":"2023-10-08T06:53:46.849527+0000","last_became_peered":"2023-10-08T06:53:46.849527+0000","last_unstale":"2023-10-08T06:53:46.850594+0000","last_undegraded":"2023-10-08T06:53:46.850594+0000","last_fullsized":"2023-10-08T06:53:46.850594+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:09.620134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.841115+0000","last_change":"2023-10-08T06:53:46.841115+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.841115+0000","last_undegraded":"2023-10-08T06:53:46.841115+0000","last_fullsized":"2023-10-08T06:53:46.841115+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853280+0000","last_change":"2023-10-08T06:53:46.849864+0000","last_active":"2023-10-08T06:53:46.853280+0000","last_peered":"2023-10-08T06:53:46.853280+0000","last_clean":"2023-10-08T06:53:46.853280+0000","last_became_active":"2023-10-08T06:53:46.849576+0000","last_became_peered":"2023-10-08T06:53:46.849576+0000","last_unstale":"2023-10-08T06:53:46.853280+0000","last_undegraded":"2023-10-08T06:53:46.853280+0000","last_fullsized":"2023-10-08T06:53:46.853280+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:36.369411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837276+0000","last_change":"2023-10-08T06:53:46.837276+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837276+0000","last_undegraded":"2023-10-08T06:53:46.837276+0000","last_fullsized":"2023-10-08T06:53:46.837276+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.847545+0000","last_change":"2023-10-08T06:53:46.846147+0000","last_active":"2023-10-08T06:53:46.847545+0000","last_peered":"2023-10-08T06:53:46.847545+0000","last_clean":"2023-10-08T06:53:46.847545+0000","last_became_active":"2023-10-08T06:53:46.845910+0000","last_became_peered":"2023-10-08T06:53:46.845910+0000","last_unstale":"2023-10-08T06:53:46.847545+0000","last_undegraded":"2023-10-08T06:53:46.847545+0000","last_fullsized":"2023-10-08T06:53:46.847545+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:11.739527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835581+0000","last_change":"2023-10-08T06:53:46.835581+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835581+0000","last_undegraded":"2023-10-08T06:53:46.835581+0000","last_fullsized":"2023-10-08T06:53:46.835581+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.858105+0000","last_change":"2023-10-08T06:53:46.857013+0000","last_active":"2023-10-08T06:53:46.858105+0000","last_peered":"2023-10-08T06:53:46.858105+0000","last_clean":"2023-10-08T06:53:46.858105+0000","last_became_active":"2023-10-08T06:53:46.856762+0000","last_became_peered":"2023-10-08T06:53:46.856762+0000","last_unstale":"2023-10-08T06:53:46.858105+0000","last_undegraded":"2023-10-08T06:53:46.858105+0000","last_fullsized":"2023-10-08T06:53:46.858105+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:12.313863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837818+0000","last_change":"2023-10-08T06:53:46.837818+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837818+0000","last_undegraded":"2023-10-08T06:53:46.837818+0000","last_fullsized":"2023-10-08T06:53:46.837818+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855358+0000","last_change":"2023-10-08T06:53:46.854554+0000","last_active":"2023-10-08T06:53:46.855358+0000","last_peered":"2023-10-08T06:53:46.855358+0000","last_clean":"2023-10-08T06:53:46.855358+0000","last_became_active":"2023-10-08T06:53:46.854235+0000","last_became_peered":"2023-10-08T06:53:46.854235+0000","last_unstale":"2023-10-08T06:53:46.855358+0000","last_undegraded":"2023-10-08T06:53:46.855358+0000","last_fullsized":"2023-10-08T06:53:46.855358+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:54:13.484888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832393+0000","last_change":"2023-10-08T06:53:46.832393+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832393+0000","last_undegraded":"2023-10-08T06:53:46.832393+0000","last_fullsized":"2023-10-08T06:53:46.832393+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.852782+0000","last_change":"2023-10-08T06:53:46.848822+0000","last_active":"2023-10-08T06:53:46.852782+0000","last_peered":"2023-10-08T06:53:46.852782+0000","last_clean":"2023-10-08T06:53:46.852782+0000","last_became_active":"2023-10-08T06:53:46.848489+0000","last_became_peered":"2023-10-08T06:53:46.848489+0000","last_unstale":"2023-10-08T06:53:46.852782+0000","last_undegraded":"2023-10-08T06:53:46.852782+0000","last_fullsized":"2023-10-08T06:53:46.852782+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:05:54.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.840127+0000","last_change":"2023-10-08T06:53:46.840127+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.840127+0000","last_undegraded":"2023-10-08T06:53:46.840127+0000","last_fullsized":"2023-10-08T06:53:46.840127+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855229+0000","last_change":"2023-10-08T06:53:46.853408+0000","last_active":"2023-10-08T06:53:46.855229+0000","last_peered":"2023-10-08T06:53:46.855229+0000","last_clean":"2023-10-08T06:53:46.855229+0000","last_became_active":"2023-10-08T06:53:46.853121+0000","last_became_peered":"2023-10-08T06:53:46.853121+0000","last_unstale":"2023-10-08T06:53:46.855229+0000","last_undegraded":"2023-10-08T06:53:46.855229+0000","last_fullsized":"2023-10-08T06:53:46.855229+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:01.990568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835282+0000","last_change":"2023-10-08T06:53:46.835282+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835282+0000","last_undegraded":"2023-10-08T06:53:46.835282+0000","last_fullsized":"2023-10-08T06:53:46.835282+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.852616+0000","last_change":"2023-10-08T06:53:46.841902+0000","last_active":"2023-10-08T06:53:46.852616+0000","last_peered":"2023-10-08T06:53:46.852616+0000","last_clean":"2023-10-08T06:53:46.852616+0000","last_became_active":"2023-10-08T06:53:46.841528+0000","last_became_peered":"2023-10-08T06:53:46.841528+0000","last_unstale":"2023-10-08T06:53:46.852616+0000","last_undegraded":"2023-10-08T06:53:46.852616+0000","last_fullsized":"2023-10-08T06:53:46.852616+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:16.903470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833006+0000","last_change":"2023-10-08T06:53:46.833006+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833006+0000","last_undegraded":"2023-10-08T06:53:46.833006+0000","last_fullsized":"2023-10-08T06:53:46.833006+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.857684+0000","last_change":"2023-10-08T06:53:46.856348+0000","last_active":"2023-10-08T06:53:46.857684+0000","last_peered":"2023-10-08T06:53:46.857684+0000","last_clean":"2023-10-08T06:53:46.857684+0000","last_became_active":"2023-10-08T06:53:46.856059+0000","last_became_peered":"2023-10-08T06:53:46.856059+0000","last_unstale":"2023-10-08T06:53:46.857684+0000","last_undegraded":"2023-10-08T06:53:46.857684+0000","last_fullsized":"2023-10-08T06:53:46.857684+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:28.598908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833355+0000","last_change":"2023-10-08T06:53:46.833355+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833355+0000","last_undegraded":"2023-10-08T06:53:46.833355+0000","last_fullsized":"2023-10-08T06:53:46.833355+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853288+0000","last_change":"2023-10-08T06:53:46.851826+0000","last_active":"2023-10-08T06:53:46.853288+0000","last_peered":"2023-10-08T06:53:46.853288+0000","last_clean":"2023-10-08T06:53:46.853288+0000","last_became_active":"2023-10-08T06:53:46.851307+0000","last_became_peered":"2023-10-08T06:53:46.851307+0000","last_unstale":"2023-10-08T06:53:46.853288+0000","last_undegraded":"2023-10-08T06:53:46.853288+0000","last_fullsized":"2023-10-08T06:53:46.853288+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:28.030370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836735+0000","last_change":"2023-10-08T06:53:46.836735+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836735+0000","last_undegraded":"2023-10-08T06:53:46.836735+0000","last_fullsized":"2023-10-08T06:53:46.836735+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855012+0000","last_change":"2023-10-08T06:53:46.854168+0000","last_active":"2023-10-08T06:53:46.855012+0000","last_peered":"2023-10-08T06:53:46.855012+0000","last_clean":"2023-10-08T06:53:46.855012+0000","last_became_active":"2023-10-08T06:53:46.853876+0000","last_became_peered":"2023-10-08T06:53:46.853876+0000","last_unstale":"2023-10-08T06:53:46.855012+0000","last_undegraded":"2023-10-08T06:53:46.855012+0000","last_fullsized":"2023-10-08T06:53:46.855012+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:46.027669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831727+0000","last_change":"2023-10-08T06:53:46.831727+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831727+0000","last_undegraded":"2023-10-08T06:53:46.831727+0000","last_fullsized":"2023-10-08T06:53:46.831727+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855809+0000","last_change":"2023-10-08T06:53:46.854126+0000","last_active":"2023-10-08T06:53:46.855809+0000","last_peered":"2023-10-08T06:53:46.855809+0000","last_clean":"2023-10-08T06:53:46.855809+0000","last_became_active":"2023-10-08T06:53:46.853833+0000","last_became_peered":"2023-10-08T06:53:46.853833+0000","last_unstale":"2023-10-08T06:53:46.855809+0000","last_undegraded":"2023-10-08T06:53:46.855809+0000","last_fullsized":"2023-10-08T06:53:46.855809+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:37.677941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825803+0000","last_change":"2023-10-08T06:53:46.825803+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825803+0000","last_undegraded":"2023-10-08T06:53:46.825803+0000","last_fullsized":"2023-10-08T06:53:46.825803+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853319+0000","last_change":"2023-10-08T06:53:46.851411+0000","last_active":"2023-10-08T06:53:46.853319+0000","last_peered":"2023-10-08T06:53:46.853319+0000","last_clean":"2023-10-08T06:53:46.853319+0000","last_became_active":"2023-10-08T06:53:46.850930+0000","last_became_peered":"2023-10-08T06:53:46.850930+0000","last_unstale":"2023-10-08T06:53:46.853319+0000","last_undegraded":"2023-10-08T06:53:46.853319+0000","last_fullsized":"2023-10-08T06:53:46.853319+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:27.974110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828614+0000","last_change":"2023-10-08T06:53:46.828614+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828614+0000","last_undegraded":"2023-10-08T06:53:46.828614+0000","last_fullsized":"2023-10-08T06:53:46.828614+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853067+0000","last_change":"2023-10-08T06:53:46.852038+0000","last_active":"2023-10-08T06:53:46.853067+0000","last_peered":"2023-10-08T06:53:46.853067+0000","last_clean":"2023-10-08T06:53:46.853067+0000","last_became_active":"2023-10-08T06:53:46.851840+0000","last_became_peered":"2023-10-08T06:53:46.851840+0000","last_unstale":"2023-10-08T06:53:46.853067+0000","last_undegraded":"2023-10-08T06:53:46.853067+0000","last_fullsized":"2023-10-08T06:53:46.853067+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:14.079295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.834744+0000","last_change":"2023-10-08T06:53:46.834744+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.834744+0000","last_undegraded":"2023-10-08T06:53:46.834744+0000","last_fullsized":"2023-10-08T06:53:46.834744+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.848916+0000","last_change":"2023-10-08T06:53:46.844152+0000","last_active":"2023-10-08T06:53:46.848916+0000","last_peered":"2023-10-08T06:53:46.848916+0000","last_clean":"2023-10-08T06:53:46.848916+0000","last_became_active":"2023-10-08T06:53:46.843835+0000","last_became_peered":"2023-10-08T06:53:46.843835+0000","last_unstale":"2023-10-08T06:53:46.848916+0000","last_undegraded":"2023-10-08T06:53:46.848916+0000","last_fullsized":"2023-10-08T06:53:46.848916+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:04:41.456787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832811+0000","last_change":"2023-10-08T06:53:46.832811+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832811+0000","last_undegraded":"2023-10-08T06:53:46.832811+0000","last_fullsized":"2023-10-08T06:53:46.832811+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.850421+0000","last_change":"2023-10-08T06:53:46.846292+0000","last_active":"2023-10-08T06:53:46.850421+0000","last_peered":"2023-10-08T06:53:46.850421+0000","last_clean":"2023-10-08T06:53:46.850421+0000","last_became_active":"2023-10-08T06:53:46.845869+0000","last_became_peered":"2023-10-08T06:53:46.845869+0000","last_unstale":"2023-10-08T06:53:46.850421+0000","last_undegraded":"2023-10-08T06:53:46.850421+0000","last_fullsized":"2023-10-08T06:53:46.850421+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:21:37.089110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.827498+0000","last_change":"2023-10-08T06:53:46.827498+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.827498+0000","last_undegraded":"2023-10-08T06:53:46.827498+0000","last_fullsized":"2023-10-08T06:53:46.827498+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855333+0000","last_change":"2023-10-08T06:53:46.853823+0000","last_active":"2023-10-08T06:53:46.855333+0000","last_peered":"2023-10-08T06:53:46.855333+0000","last_clean":"2023-10-08T06:53:46.855333+0000","last_became_active":"2023-10-08T06:53:46.853347+0000","last_became_peered":"2023-10-08T06:53:46.853347+0000","last_unstale":"2023-10-08T06:53:46.855333+0000","last_undegraded":"2023-10-08T06:53:46.855333+0000","last_fullsized":"2023-10-08T06:53:46.855333+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:02.956148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"172'2729","reported_seq":3394,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-08T06:53:45.826415+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T06:53:45.826415+0000","last_peered":"2023-10-08T06:53:45.826415+0000","last_clean":"2023-10-08T06:53:45.826415+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T06:53:45.826415+0000","last_undegraded":"2023-10-08T06:53:45.826415+0000","last_fullsized":"2023-10-08T06:53:45.826415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2729,"log_dups_size":0,"ondisk_log_size":2729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575775,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60348,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311492,"statfs":{"total":96636764160,"available":96574967808,"internally_reserved":0,"allocated":2867200,"data_stored":1759119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.21,"15min":1.2330000000000001},"min":{"1min":0.91900000000000004,"5min":0.78700000000000003,"15min":0.70399999999999996},"max":{"1min":1.6950000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.224,"15min":1.248},"min":{"1min":0.93799999999999994,"5min":0.747,"15min":0.72899999999999998},"max":{"1min":1.522,"5min":1.7150000000000001,"15min":1.849},"last":1.5009999999999999}]},{"osd":1,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2829999999999999,"15min":1.262},"min":{"1min":0.89200000000000002,"5min":0.84299999999999997,"15min":0.70399999999999996},"max":{"1min":1.6970000000000001,"5min":1.712,"15min":1.8480000000000001},"last":1.157},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.284,"15min":1.3220000000000001},"min":{"1min":0.95999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7450000000000001,"5min":1.9299999999999999,"15min":2.4809999999999999},"last":1.591}]},{"osd":2,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.0740000000000001,"5min":0.90900000000000003,"15min":0.81899999999999995},"max":{"1min":1.627,"5min":2.7040000000000002,"15min":3.4350000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.387,"5min":1.381,"15min":1.3740000000000001},"min":{"1min":0.95699999999999996,"5min":0.88900000000000001,"15min":0.83199999999999996},"max":{"1min":1.873,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":0.99099999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.339,"15min":1.335},"min":{"1min":1.0189999999999999,"5min":0.97799999999999998,"15min":0.85899999999999999},"max":{"1min":1.5609999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.212},{"interface":"front","average":{"1min":1.5,"5min":1.46,"15min":1.419},"min":{"1min":1.147,"5min":0.98899999999999999,"15min":0.83099999999999996},"max":{"1min":2.0379999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.3620000000000001}]},{"osd":4,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.333,"15min":1.2869999999999999},"min":{"1min":0.93999999999999995,"5min":0.83199999999999996,"15min":0.66200000000000003},"max":{"1min":1.952,"5min":1.952,"15min":3.2839999999999998},"last":1.792},{"interface":"front","average":{"1min":1.484,"5min":1.456,"15min":1.391},"min":{"1min":1.1579999999999999,"5min":0.90300000000000002,"15min":0.80800000000000005},"max":{"1min":1.708,"5min":2.004,"15min":2.1909999999999998},"last":1.4510000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.474,"15min":1.4399999999999999},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":0.86499999999999999},"max":{"1min":2.1499999999999999,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":1.272},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.504,"15min":1.4790000000000001},"min":{"1min":1.2909999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.6439999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.5680000000000001,"15min":1.556},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":0.94599999999999995},"max":{"1min":2.1970000000000001,"5min":2.4350000000000001,"15min":3.1360000000000001},"last":1.728},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5269999999999999,"15min":1.502},"min":{"1min":1.0049999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":2.0190000000000001,"5min":2.1309999999999998,"15min":2.3140000000000001},"last":1.6839999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41720,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330120,"statfs":{"total":96636764160,"available":96594042880,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3009999999999999,"15min":1.3480000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7310000000000001,"5min":1.792,"15min":2.508},"last":1.605},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3089999999999999,"15min":1.337},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.609,"5min":1.8280000000000001,"15min":2.1139999999999999},"last":1.3520000000000001}]},{"osd":1,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2170000000000001,"15min":1.248},"min":{"1min":0.876,"5min":0.876,"15min":0.59699999999999998},"max":{"1min":1.605,"5min":1.821,"15min":1.919},"last":1.413},{"interface":"front","average":{"1min":1.224,"5min":1.2769999999999999,"15min":1.2629999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.53800000000000003},"max":{"1min":1.7769999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3720000000000001}]},{"osd":2,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2170000000000001,"15min":1.2290000000000001},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.75600000000000001},"max":{"1min":1.6499999999999999,"5min":1.722,"15min":1.8919999999999999},"last":1.514},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.29,"15min":1.2989999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5229999999999999,"5min":1.6379999999999999,"15min":2.3839999999999999},"last":1.1739999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4259999999999999,"15min":1.4039999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.83399999999999996},"max":{"1min":1.8700000000000001,"5min":1.976,"15min":2.1499999999999999},"last":1.478},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.478,"15min":1.48},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.89400000000000002},"max":{"1min":1.8180000000000001,"5min":1.915,"15min":2.0289999999999999},"last":1.274}]},{"osd":4,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.3580000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.72399999999999998},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.226},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4379999999999999,"15min":1.4390000000000001},"min":{"1min":0.878,"5min":0.878,"15min":0.80200000000000005},"max":{"1min":1.9099999999999999,"5min":1.9910000000000001,"15min":2.0089999999999999},"last":1.006}]},{"osd":5,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.431,"15min":1.4139999999999999},"min":{"1min":0.997,"5min":0.94799999999999995,"15min":0.69499999999999995},"max":{"1min":1.798,"5min":1.893,"15min":2.7309999999999999},"last":1.319},{"interface":"front","average":{"1min":1.575,"5min":1.5589999999999999,"15min":1.548},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78300000000000003},"max":{"1min":2.0640000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.121}]},{"osd":7,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.591,"15min":1.579},"min":{"1min":1.0960000000000001,"5min":1.0680000000000001,"15min":0.92000000000000004},"max":{"1min":1.984,"5min":2.0710000000000002,"15min":2.2090000000000001},"last":1.454},{"interface":"front","average":{"1min":1.488,"5min":1.573,"15min":1.5660000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.82099999999999995},"max":{"1min":1.946,"5min":2.222,"15min":2.6469999999999998},"last":1.5549999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59768,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312072,"statfs":{"total":96636764160,"available":96575561728,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2310000000000001,"15min":1.218},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.66600000000000004},"max":{"1min":1.5740000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.296,"15min":1.2569999999999999},"min":{"1min":0.95399999999999996,"5min":0.82699999999999996,"15min":0.78800000000000003},"max":{"1min":1.629,"5min":1.897,"15min":2.4700000000000002},"last":1.206}]},{"osd":1,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.26,"15min":1.2230000000000001},"min":{"1min":0.95499999999999996,"5min":0.85799999999999998,"15min":0.72499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.349,"5min":1.375,"15min":1.323},"min":{"1min":0.90400000000000003,"5min":0.88700000000000001,"15min":0.78200000000000003},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.25}]},{"osd":2,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3480000000000001,"15min":1.3280000000000001},"min":{"1min":0.98399999999999999,"5min":0.85199999999999998,"15min":0.83199999999999996},"max":{"1min":1.6479999999999999,"5min":1.9359999999999999,"15min":2.351},"last":1.3},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.3160000000000001},"min":{"1min":0.91000000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.667,"5min":1.7809999999999999,"15min":1.964},"last":0.94499999999999995}]},{"osd":3,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3959999999999999,"15min":1.3540000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.528,"15min":1.464},"min":{"1min":1.139,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.9910000000000001,"5min":3.0059999999999998,"15min":5.1630000000000003},"last":1.042}]},{"osd":4,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3320000000000001,"15min":1.3240000000000001},"min":{"1min":0.96299999999999997,"5min":0.89300000000000002,"15min":0.81499999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4039999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4379999999999999,"15min":1.4239999999999999},"min":{"1min":0.96899999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.113}]},{"osd":6,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.4990000000000001,"15min":1.4750000000000001},"min":{"1min":1.248,"5min":0.96399999999999997,"15min":0.81200000000000006},"max":{"1min":2.1150000000000002,"5min":2.1659999999999999,"15min":5.5019999999999998},"last":1.155},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4930000000000001,"15min":1.48},"min":{"1min":1.0509999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.0139999999999998,"5min":2.1040000000000001,"15min":2.2890000000000001},"last":1.446}]},{"osd":7,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.5660000000000001,"15min":1.5629999999999999},"min":{"1min":1.3400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":5.8780000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.679,"5min":1.6839999999999999,"15min":1.6299999999999999},"min":{"1min":1.415,"5min":1.1719999999999999,"15min":0.97099999999999997},"max":{"1min":2.206,"5min":2.6789999999999998,"15min":5.399},"last":1.381}]}]},{"osd":4,"up_from":13,"seq":55834575774,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41720,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330120,"statfs":{"total":96636764160,"available":96594042880,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.389,"15min":1.3460000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.3999999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.69799999999999995},"max":{"1min":1.7789999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.518}]},{"osd":1,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2749999999999999,"15min":1.28},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.73,"5min":1.784,"15min":1.9339999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.64000000000000001},"max":{"1min":1.613,"5min":1.6870000000000001,"15min":1.7290000000000001},"last":1.232}]},{"osd":2,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.2889999999999999,"15min":1.248},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":1.046},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.3759999999999999},"min":{"1min":1,"5min":0.93100000000000005,"15min":0.85399999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.651}]},{"osd":3,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.339,"15min":1.3109999999999999},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.4670000000000001,"15min":1.4099999999999999},"min":{"1min":1.0629999999999999,"5min":0.97299999999999998,"15min":0.93400000000000005},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.5429999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.4139999999999999,"15min":1.351},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.66700000000000004},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.4450000000000001,"15min":1.375},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.749},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.6060000000000001}]},{"osd":6,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.5329999999999999,"15min":1.425},"min":{"1min":0.97899999999999998,"5min":0.91100000000000003,"15min":0.72699999999999998},"max":{"1min":5.3339999999999996,"5min":5.3339999999999996,"15min":5.3339999999999996},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6120000000000001,"15min":1.546},"min":{"1min":1.212,"5min":0.96699999999999997,"15min":0.91400000000000003},"max":{"1min":2.1059999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":1.7250000000000001}]},{"osd":7,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.548,"15min":1.49},"min":{"1min":1.202,"5min":1.135,"15min":0.99099999999999999},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.784,"5min":1.6859999999999999,"15min":1.6060000000000001},"min":{"1min":1.0049999999999999,"5min":0.98999999999999999,"15min":0.93600000000000005},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":1.6879999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575776,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47288,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324552,"statfs":{"total":96636764160,"available":96588341248,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.95499999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.44500000000000001},"max":{"1min":1.3200000000000001,"5min":1.71,"15min":1.71},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.196,"15min":1.175},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6719999999999999,"5min":1.851,"15min":2.137},"last":0.93500000000000005}]},{"osd":1,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.024,"15min":1.018},"min":{"1min":0.70999999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2370000000000001,"5min":1.71,"15min":1.823},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.321,"5min":1.2989999999999999,"15min":1.3049999999999999},"min":{"1min":0.91000000000000003,"5min":0.872,"15min":0.77000000000000002},"max":{"1min":1.51,"5min":1.75,"15min":1.893},"last":0.73599999999999999}]},{"osd":2,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1739999999999999,"15min":1.137},"min":{"1min":0.85299999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6279999999999999,"5min":1.726,"15min":1.7410000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1919999999999999,"15min":1.1930000000000001},"min":{"1min":0.90400000000000003,"5min":0.68600000000000005,"15min":0.61399999999999999},"max":{"1min":1.589,"5min":1.7809999999999999,"15min":1.8360000000000001},"last":1.171}]},{"osd":4,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.284,"15min":1.2649999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83299999999999996},"max":{"1min":1.8560000000000001,"5min":1.97,"15min":1.97},"last":1.349},{"interface":"front","average":{"1min":1.482,"5min":1.4650000000000001,"15min":1.452},"min":{"1min":1.1859999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.8100000000000001,"5min":1.905,"15min":2.1960000000000002},"last":1.3080000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.391,"15min":1.385},"min":{"1min":1.085,"5min":0.98399999999999999,"15min":0.88},"max":{"1min":1.6399999999999999,"5min":2.27,"15min":3.0009999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.415,"15min":1.421},"min":{"1min":1.1890000000000001,"5min":0.995,"15min":0.94099999999999995},"max":{"1min":1.6739999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.2809999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4059999999999999,"15min":1.3959999999999999},"min":{"1min":0.98399999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.016},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.534,"15min":1.518},"min":{"1min":1.2170000000000001,"5min":1.1639999999999999,"15min":0.99099999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.544},"last":1.371}]},{"osd":7,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.5649999999999999,"15min":1.5660000000000001},"min":{"1min":1.3069999999999999,"5min":1.1839999999999999,"15min":1.008},"max":{"1min":1.8939999999999999,"5min":2.008,"15min":2.347},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.5049999999999999,"15min":1.4790000000000001},"min":{"1min":1.0980000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1800000000000002},"last":1.2070000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47288,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324552,"statfs":{"total":96636764160,"available":96588341248,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.93700000000000006},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.4650000000000001,"15min":1.657},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.036,"5min":1.0549999999999999,"15min":1.0760000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5249999999999999,"5min":1.661,"15min":2.2679999999999998},"last":1.26}]},{"osd":1,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.004,"15min":1.0069999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.696,"5min":1.7170000000000001,"15min":1.996},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.137,"5min":1.1679999999999999,"15min":1.1819999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":2.1139999999999999},"last":0.92800000000000005}]},{"osd":3,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2270000000000001,"15min":1.2230000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.772,"5min":1.925,"15min":2.3929999999999998},"last":1.075},{"interface":"front","average":{"1min":1.177,"5min":1.256,"15min":1.258},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.46,"5min":1.804,"15min":1.891},"last":1.0389999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4330000000000001,"15min":1.444},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.859,"5min":1.9430000000000001,"15min":3.0939999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.3160000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7470000000000001,"5min":1.7549999999999999,"15min":3.7320000000000002},"last":1.4379999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.429,"15min":1.425},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":3.3519999999999999},"last":1.48},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.5029999999999999,"15min":1.528},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.9810000000000001,"5min":2.2719999999999998,"15min":2.5249999999999999},"last":1.321}]},{"osd":6,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.673,"5min":1.9990000000000001,"15min":2.206},"last":1.532},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.548,"15min":1.532},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9610000000000001,"5min":1.966,"15min":3.02},"last":1.6319999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.51,"15min":1.492},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.861,"5min":2.0790000000000002,"15min":4.2869999999999999},"last":1.591},{"interface":"front","average":{"1min":1.55,"5min":1.5940000000000001,"15min":1.579},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8540000000000001,"5min":2.0099999999999998,"15min":3.0760000000000001},"last":1.375}]}]},{"osd":1,"up_from":13,"seq":55834575774,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29240,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342600,"statfs":{"total":96636764160,"available":96606822400,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.0349999999999999,"15min":1.0029999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.601,"5min":4.601,"15min":4.601},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2190000000000001,"15min":1.22},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.72599999999999998}]},{"osd":2,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.0329999999999999,"15min":1.0229999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.3179999999999996,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.385,"5min":1.2250000000000001,"15min":1.1679999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":5.4059999999999997,"5min":5.4059999999999997,"15min":5.4059999999999997},"last":1.163}]},{"osd":3,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.3029999999999999,"15min":1.2609999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.5220000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1579999999999999,"15min":1.1160000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.0069999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.3919999999999999,"15min":1.3680000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":4.2409999999999997,"5min":4.2409999999999997,"15min":4.2409999999999997},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.337,"15min":1.298},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":1.1970000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.4279999999999999,"15min":1.3600000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":4.7530000000000001,"5min":4.7530000000000001,"15min":4.7530000000000001},"last":1.296},{"interface":"front","average":{"1min":1.742,"5min":1.5980000000000001,"15min":1.5329999999999999},"min":{"1min":1.03,"5min":1.03,"15min":0.75},"max":{"1min":5.7389999999999999,"5min":5.7389999999999999,"15min":5.7389999999999999},"last":1.429}]},{"osd":6,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.5700000000000001,"15min":1.53},"min":{"1min":1.05,"5min":1.05,"15min":1.02},"max":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.661,"5min":1.5329999999999999,"15min":1.4810000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":1.52}]},{"osd":7,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.605,"15min":1.5489999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":5.6289999999999996,"5min":5.6289999999999996,"15min":5.6289999999999996},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.607,"15min":1.5549999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.5270000000000001,"5min":4.5270000000000001,"15min":4.5270000000000001},"last":1.48}]}]},{"osd":0,"up_from":13,"seq":55834575775,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38140,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333700,"statfs":{"total":96636764160,"available":96597708800,"internally_reserved":0,"allocated":2867200,"data_stored":1759119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5669999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.173,"5min":1.123,"15min":1.091},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.222,"15min":1.163},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.226,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0580000000000001,"15min":1.0800000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.2669999999999999,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.294,"15min":1.27},"min":{"1min":0.625,"5min":0.625,"15min":0.621},"max":{"1min":2.8250000000000002,"5min":4.0220000000000002,"15min":4.0220000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.159,"5min":1.2110000000000001,"15min":1.1839999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.319}]},{"osd":4,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.349,"15min":1.3680000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4199999999999999,"15min":1.411},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.6799999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4330000000000001,"15min":1.4019999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.68700000000000006},"max":{"1min":2.0659999999999998,"5min":2.0920000000000001,"15min":2.2480000000000002},"last":1.339},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5249999999999999,"15min":1.478},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":3.2229999999999999,"5min":3.887,"15min":3.887},"last":1.262}]},{"osd":6,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.504,"15min":1.476},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.514,"15min":1.508},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.748},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.2070000000000001}]},{"osd":7,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.546,"15min":1.5369999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.86199999999999999},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.573,"15min":1.548},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.194,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.6459999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T06:53:52.301 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T06:53:52.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.548+0000 7f66ce143640 1 Processor -- start 2023-10-08T06:53:52.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.548+0000 7f66ce143640 1 -- start start 2023-10-08T06:53:52.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.549+0000 7f66ce143640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c81599d0 0x7f66c80ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.549+0000 7f66ce143640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66c80b07d0 0x7f66c806c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.549+0000 7f66ce143640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c80cacf0 con 0x7f66c80cc210 2023-10-08T06:53:52.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.549+0000 7f66ce143640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c80b10f0 con 0x7f66c80b07d0 2023-10-08T06:53:52.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.549+0000 7f66ce143640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80ca3c0 con 0x7f66c81599d0 2023-10-08T06:53:52.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.549+0000 7f66cd942640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66c80b07d0 0x7f66c806c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.549+0000 7f66cd942640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66c80b07d0 0x7f66c806c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48330/0 (socket says 172.21.15.70:48330) 2023-10-08T06:53:52.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.549+0000 7f66cd942640 1 -- 172.21.15.70:0/1889911174 learned_addr learned my addr 172.21.15.70:0/1889911174 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:52.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.550+0000 7f66cd141640 1 --2- 172.21.15.70:0/1889911174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c81599d0 0x7f66c80ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.550+0000 7f66cd942640 1 -- 172.21.15.70:0/1889911174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c81599d0 msgr2=0x7f66c80ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:52.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.550+0000 7f66cd942640 1 --2- 172.21.15.70:0/1889911174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c81599d0 0x7f66c80ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.550+0000 7f66cd942640 1 -- 172.21.15.70:0/1889911174 >> v1:172.21.15.181:6789/0 conn(0x7f66c80cc210 legacy=0x7f66c80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:52.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.550+0000 7f66cd942640 1 -- 172.21.15.70:0/1889911174 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c80ca6a0 con 0x7f66c80b07d0 2023-10-08T06:53:52.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.551+0000 7f66cd942640 1 --2- 172.21.15.70:0/1889911174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c80b07d0 0x7f66c806c310 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009e50 tx=0x7f66c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4200dc9d871c89 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:52.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.551+0000 7f66bffff640 1 -- 172.21.15.70:0/1889911174 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c002ca70 con 0x7f66c80b07d0 2023-10-08T06:53:52.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.551+0000 7f66bffff640 1 -- 172.21.15.70:0/1889911174 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c002cbd0 con 0x7f66c80b07d0 2023-10-08T06:53:52.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.551+0000 7f66bffff640 1 -- 172.21.15.70:0/1889911174 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c00343f0 con 0x7f66c80b07d0 2023-10-08T06:53:52.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.552+0000 7f66ce143640 1 -- 172.21.15.70:0/1889911174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c80b07d0 msgr2=0x7f66c806c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.552+0000 7f66ce143640 1 --2- 172.21.15.70:0/1889911174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c80b07d0 0x7f66c806c310 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009e50 tx=0x7f66c002bed0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.552+0000 7f66ce143640 1 -- 172.21.15.70:0/1889911174 shutdown_connections 2023-10-08T06:53:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.552+0000 7f66ce143640 1 --2- 172.21.15.70:0/1889911174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c80b07d0 0x7f66c806c310 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.552+0000 7f66ce143640 1 --2- 172.21.15.70:0/1889911174 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c81599d0 0x7f66c80ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.552+0000 7f66ce143640 1 -- 172.21.15.70:0/1889911174 >> 172.21.15.70:0/1889911174 conn(0x7f66c80cb0c0 msgr2=0x7f66c80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.552+0000 7f66ce143640 1 -- 172.21.15.70:0/1889911174 shutdown_connections 2023-10-08T06:53:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.552+0000 7f66ce143640 1 -- 172.21.15.70:0/1889911174 wait complete. 2023-10-08T06:53:52.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.554+0000 7f66ce143640 1 Processor -- start 2023-10-08T06:53:52.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.554+0000 7f66ce143640 1 -- start start 2023-10-08T06:53:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.555+0000 7f66ce143640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c80b07d0 0x7f66c80c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.555+0000 7f66ce143640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c80cc210 0x7f66c81648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.555+0000 7f66ce143640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c81599d0 0x7f66c8162cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.555+0000 7f66ce143640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81572e0 con 0x7f66c81599d0 2023-10-08T06:53:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.555+0000 7f66ce143640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c8164e10 con 0x7f66c80b07d0 2023-10-08T06:53:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.555+0000 7f66ce143640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81650b0 con 0x7f66c80cc210 2023-10-08T06:53:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.555+0000 7f66cd942640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c81599d0 0x7f66c8162cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.555+0000 7f66cd942640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c81599d0 0x7f66c8162cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48342/0 (socket says 172.21.15.70:48342) 2023-10-08T06:53:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.555+0000 7f66cd942640 1 -- 172.21.15.70:0/608698959 learned_addr learned my addr 172.21.15.70:0/608698959 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66cc940640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c80cc210 0x7f66c81648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66cd141640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c80b07d0 0x7f66c80c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66cd942640 1 -- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c80cc210 msgr2=0x7f66c81648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66cd942640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c80cc210 0x7f66c81648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66cd942640 1 -- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c80b07d0 msgr2=0x7f66c80c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66cd942640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c80b07d0 0x7f66c80c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66cd942640 1 -- 172.21.15.70:0/608698959 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c0009b30 con 0x7f66c81599d0 2023-10-08T06:53:52.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66cc940640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c80cc210 0x7f66c81648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:52.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66cd942640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c81599d0 0x7f66c8162cb0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009e50 tx=0x7f66c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:52.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66bdffb640 1 -- 172.21.15.70:0/608698959 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0009510 con 0x7f66c81599d0 2023-10-08T06:53:52.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66bdffb640 1 -- 172.21.15.70:0/608698959 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c003e450 con 0x7f66c81599d0 2023-10-08T06:53:52.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66bdffb640 1 -- 172.21.15.70:0/608698959 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c003e800 con 0x7f66c81599d0 2023-10-08T06:53:52.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66ce143640 1 -- 172.21.15.70:0/608698959 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c8165390 con 0x7f66c81599d0 2023-10-08T06:53:52.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.556+0000 7f66ce143640 1 -- 172.21.15.70:0/608698959 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c8165950 con 0x7f66c81599d0 2023-10-08T06:53:52.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.557+0000 7f66ce143640 1 -- 172.21.15.70:0/608698959 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c804e160 con 0x7f66c81599d0 2023-10-08T06:53:52.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.558+0000 7f66bdffb640 1 -- 172.21.15.70:0/608698959 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f66c003e960 con 0x7f66c81599d0 2023-10-08T06:53:52.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.559+0000 7f66bdffb640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f66ac06af90 0x7f66ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:52.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.559+0000 7f66bdffb640 1 -- 172.21.15.70:0/608698959 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f66c00ac790 con 0x7f66c81599d0 2023-10-08T06:53:52.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.560+0000 7f66cd141640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f66ac06af90 0x7f66ac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:52.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.561+0000 7f66cd141640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f66ac06af90 0x7f66ac06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66c4006fd0 tx=0x7f66c4008040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.564+0000 7f66bdffb640 1 -- 172.21.15.70:0/608698959 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66c00774d0 con 0x7f66c81599d0 2023-10-08T06:53:52.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.750+0000 7f66ce143640 1 -- 172.21.15.70:0/608698959 --> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66c80ca070 con 0x7f66ac06af90 2023-10-08T06:53:52.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.772+0000 7f66bdffb640 1 -- 172.21.15.70:0/608698959 <== mgr.9345 v2:172.21.15.181:6832/4245513204 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282075 (secure 0 0 0) 0x7f66c400e4c0 con 0x7f66ac06af90 2023-10-08T06:53:52.774 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:52.781 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T06:53:52.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.778+0000 7f66ce143640 1 -- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f66ac06af90 msgr2=0x7f66ac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.778+0000 7f66ce143640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f66ac06af90 0x7f66ac06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f66c4006fd0 tx=0x7f66c4008040 comp rx=0 tx=0).stop 2023-10-08T06:53:52.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.778+0000 7f66ce143640 1 -- 172.21.15.70:0/608698959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c81599d0 msgr2=0x7f66c8162cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:52.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.778+0000 7f66ce143640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c81599d0 0x7f66c8162cb0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009e50 tx=0x7f66c0004a40 comp rx=0 tx=0).stop 2023-10-08T06:53:52.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.778+0000 7f66ce143640 1 -- 172.21.15.70:0/608698959 shutdown_connections 2023-10-08T06:53:52.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.778+0000 7f66ce143640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f66ac06af90 0x7f66ac06d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.778+0000 7f66ce143640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66c81599d0 0x7f66c8162cb0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.778+0000 7f66ce143640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c80cc210 0x7f66c81648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.778+0000 7f66ce143640 1 --2- 172.21.15.70:0/608698959 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c80b07d0 0x7f66c80c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:52.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.779+0000 7f66ce143640 1 -- 172.21.15.70:0/608698959 >> 172.21.15.70:0/608698959 conn(0x7f66c80cb0c0 msgr2=0x7f66c80ae6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:52.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.779+0000 7f66ce143640 1 -- 172.21.15.70:0/608698959 shutdown_connections 2023-10-08T06:53:52.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:52.779+0000 7f66ce143640 1 -- 172.21.15.70:0/608698959 wait complete. 2023-10-08T06:53:52.798 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-08T06:53:52.078361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365512,"kb_used_data":18920,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754609208,"statfs":{"total":773094113280,"available":772719828992,"internally_reserved":0,"allocated":19374080,"data_stored":10530744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":354799163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001853"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837391+0000","last_change":"2023-10-08T06:53:46.837391+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837391+0000","last_undegraded":"2023-10-08T06:53:46.837391+0000","last_fullsized":"2023-10-08T06:53:46.837391+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835340+0000","last_change":"2023-10-08T06:53:46.835340+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835340+0000","last_undegraded":"2023-10-08T06:53:46.835340+0000","last_fullsized":"2023-10-08T06:53:46.835340+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836968+0000","last_change":"2023-10-08T06:53:46.836968+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836968+0000","last_undegraded":"2023-10-08T06:53:46.836968+0000","last_fullsized":"2023-10-08T06:53:46.836968+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832209+0000","last_change":"2023-10-08T06:53:46.832209+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832209+0000","last_undegraded":"2023-10-08T06:53:46.832209+0000","last_fullsized":"2023-10-08T06:53:46.832209+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830068+0000","last_change":"2023-10-08T06:53:46.830068+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830068+0000","last_undegraded":"2023-10-08T06:53:46.830068+0000","last_fullsized":"2023-10-08T06:53:46.830068+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830196+0000","last_change":"2023-10-08T06:53:46.830196+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830196+0000","last_undegraded":"2023-10-08T06:53:46.830196+0000","last_fullsized":"2023-10-08T06:53:46.830196+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833262+0000","last_change":"2023-10-08T06:53:46.833262+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833262+0000","last_undegraded":"2023-10-08T06:53:46.833262+0000","last_fullsized":"2023-10-08T06:53:46.833262+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831994+0000","last_change":"2023-10-08T06:53:46.831994+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831994+0000","last_undegraded":"2023-10-08T06:53:46.831994+0000","last_fullsized":"2023-10-08T06:53:46.831994+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832152+0000","last_change":"2023-10-08T06:53:46.832152+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832152+0000","last_undegraded":"2023-10-08T06:53:46.832152+0000","last_fullsized":"2023-10-08T06:53:46.832152+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828877+0000","last_change":"2023-10-08T06:53:46.828877+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828877+0000","last_undegraded":"2023-10-08T06:53:46.828877+0000","last_fullsized":"2023-10-08T06:53:46.828877+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826488+0000","last_change":"2023-10-08T06:53:46.826488+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826488+0000","last_undegraded":"2023-10-08T06:53:46.826488+0000","last_fullsized":"2023-10-08T06:53:46.826488+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831044+0000","last_change":"2023-10-08T06:53:46.831044+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831044+0000","last_undegraded":"2023-10-08T06:53:46.831044+0000","last_fullsized":"2023-10-08T06:53:46.831044+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831261+0000","last_change":"2023-10-08T06:53:46.831261+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831261+0000","last_undegraded":"2023-10-08T06:53:46.831261+0000","last_fullsized":"2023-10-08T06:53:46.831261+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830002+0000","last_change":"2023-10-08T06:53:46.830002+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830002+0000","last_undegraded":"2023-10-08T06:53:46.830002+0000","last_fullsized":"2023-10-08T06:53:46.830002+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833676+0000","last_change":"2023-10-08T06:53:46.833676+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833676+0000","last_undegraded":"2023-10-08T06:53:46.833676+0000","last_fullsized":"2023-10-08T06:53:46.833676+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831113+0000","last_change":"2023-10-08T06:53:46.831113+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831113+0000","last_undegraded":"2023-10-08T06:53:46.831113+0000","last_fullsized":"2023-10-08T06:53:46.831113+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.834745+0000","last_change":"2023-10-08T06:53:46.834745+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.834745+0000","last_undegraded":"2023-10-08T06:53:46.834745+0000","last_fullsized":"2023-10-08T06:53:46.834745+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825966+0000","last_change":"2023-10-08T06:53:46.825966+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825966+0000","last_undegraded":"2023-10-08T06:53:46.825966+0000","last_fullsized":"2023-10-08T06:53:46.825966+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.841643+0000","last_change":"2023-10-08T06:53:46.841643+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.841643+0000","last_undegraded":"2023-10-08T06:53:46.841643+0000","last_fullsized":"2023-10-08T06:53:46.841643+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826979+0000","last_change":"2023-10-08T06:53:46.826979+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826979+0000","last_undegraded":"2023-10-08T06:53:46.826979+0000","last_fullsized":"2023-10-08T06:53:46.826979+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830751+0000","last_change":"2023-10-08T06:53:46.830751+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830751+0000","last_undegraded":"2023-10-08T06:53:46.830751+0000","last_fullsized":"2023-10-08T06:53:46.830751+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836034+0000","last_change":"2023-10-08T06:53:46.836034+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836034+0000","last_undegraded":"2023-10-08T06:53:46.836034+0000","last_fullsized":"2023-10-08T06:53:46.836034+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826700+0000","last_change":"2023-10-08T06:53:46.826700+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826700+0000","last_undegraded":"2023-10-08T06:53:46.826700+0000","last_fullsized":"2023-10-08T06:53:46.826700+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825977+0000","last_change":"2023-10-08T06:53:46.825977+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825977+0000","last_undegraded":"2023-10-08T06:53:46.825977+0000","last_fullsized":"2023-10-08T06:53:46.825977+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836905+0000","last_change":"2023-10-08T06:53:46.836905+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836905+0000","last_undegraded":"2023-10-08T06:53:46.836905+0000","last_fullsized":"2023-10-08T06:53:46.836905+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.838574+0000","last_change":"2023-10-08T06:53:46.838574+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.838574+0000","last_undegraded":"2023-10-08T06:53:46.838574+0000","last_fullsized":"2023-10-08T06:53:46.838574+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828887+0000","last_change":"2023-10-08T06:53:46.828887+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828887+0000","last_undegraded":"2023-10-08T06:53:46.828887+0000","last_fullsized":"2023-10-08T06:53:46.828887+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825937+0000","last_change":"2023-10-08T06:53:46.825937+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825937+0000","last_undegraded":"2023-10-08T06:53:46.825937+0000","last_fullsized":"2023-10-08T06:53:46.825937+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831793+0000","last_change":"2023-10-08T06:53:46.831793+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831793+0000","last_undegraded":"2023-10-08T06:53:46.831793+0000","last_fullsized":"2023-10-08T06:53:46.831793+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825313+0000","last_change":"2023-10-08T06:53:46.825313+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825313+0000","last_undegraded":"2023-10-08T06:53:46.825313+0000","last_fullsized":"2023-10-08T06:53:46.825313+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828751+0000","last_change":"2023-10-08T06:53:46.828751+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828751+0000","last_undegraded":"2023-10-08T06:53:46.828751+0000","last_fullsized":"2023-10-08T06:53:46.828751+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831002+0000","last_change":"2023-10-08T06:53:46.831002+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831002+0000","last_undegraded":"2023-10-08T06:53:46.831002+0000","last_fullsized":"2023-10-08T06:53:46.831002+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837790+0000","last_change":"2023-10-08T06:53:46.837790+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837790+0000","last_undegraded":"2023-10-08T06:53:46.837790+0000","last_fullsized":"2023-10-08T06:53:46.837790+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.841915+0000","last_change":"2023-10-08T06:53:46.838837+0000","last_active":"2023-10-08T06:53:46.841915+0000","last_peered":"2023-10-08T06:53:46.841915+0000","last_clean":"2023-10-08T06:53:46.841915+0000","last_became_active":"2023-10-08T06:53:46.838366+0000","last_became_peered":"2023-10-08T06:53:46.838366+0000","last_unstale":"2023-10-08T06:53:46.841915+0000","last_undegraded":"2023-10-08T06:53:46.841915+0000","last_fullsized":"2023-10-08T06:53:46.841915+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:26.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836525+0000","last_change":"2023-10-08T06:53:46.836525+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836525+0000","last_undegraded":"2023-10-08T06:53:46.836525+0000","last_fullsized":"2023-10-08T06:53:46.836525+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855058+0000","last_change":"2023-10-08T06:53:46.854455+0000","last_active":"2023-10-08T06:53:46.855058+0000","last_peered":"2023-10-08T06:53:46.855058+0000","last_clean":"2023-10-08T06:53:46.855058+0000","last_became_active":"2023-10-08T06:53:46.854195+0000","last_became_peered":"2023-10-08T06:53:46.854195+0000","last_unstale":"2023-10-08T06:53:46.855058+0000","last_undegraded":"2023-10-08T06:53:46.855058+0000","last_fullsized":"2023-10-08T06:53:46.855058+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:32.390630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832812+0000","last_change":"2023-10-08T06:53:46.832812+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832812+0000","last_undegraded":"2023-10-08T06:53:46.832812+0000","last_fullsized":"2023-10-08T06:53:46.832812+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.854602+0000","last_change":"2023-10-08T06:53:46.853301+0000","last_active":"2023-10-08T06:53:46.854602+0000","last_peered":"2023-10-08T06:53:46.854602+0000","last_clean":"2023-10-08T06:53:46.854602+0000","last_became_active":"2023-10-08T06:53:46.853014+0000","last_became_peered":"2023-10-08T06:53:46.853014+0000","last_unstale":"2023-10-08T06:53:46.854602+0000","last_undegraded":"2023-10-08T06:53:46.854602+0000","last_fullsized":"2023-10-08T06:53:46.854602+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:52.386749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.844177+0000","last_change":"2023-10-08T06:53:46.844177+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.844177+0000","last_undegraded":"2023-10-08T06:53:46.844177+0000","last_fullsized":"2023-10-08T06:53:46.844177+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855614+0000","last_change":"2023-10-08T06:53:46.854219+0000","last_active":"2023-10-08T06:53:46.855614+0000","last_peered":"2023-10-08T06:53:46.855614+0000","last_clean":"2023-10-08T06:53:46.855614+0000","last_became_active":"2023-10-08T06:53:46.853948+0000","last_became_peered":"2023-10-08T06:53:46.853948+0000","last_unstale":"2023-10-08T06:53:46.855614+0000","last_undegraded":"2023-10-08T06:53:46.855614+0000","last_fullsized":"2023-10-08T06:53:46.855614+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:36.674287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826941+0000","last_change":"2023-10-08T06:53:46.826941+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826941+0000","last_undegraded":"2023-10-08T06:53:46.826941+0000","last_fullsized":"2023-10-08T06:53:46.826941+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853355+0000","last_change":"2023-10-08T06:53:46.847384+0000","last_active":"2023-10-08T06:53:46.853355+0000","last_peered":"2023-10-08T06:53:46.853355+0000","last_clean":"2023-10-08T06:53:46.853355+0000","last_became_active":"2023-10-08T06:53:46.847114+0000","last_became_peered":"2023-10-08T06:53:46.847114+0000","last_unstale":"2023-10-08T06:53:46.853355+0000","last_undegraded":"2023-10-08T06:53:46.853355+0000","last_fullsized":"2023-10-08T06:53:46.853355+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:08.420118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831287+0000","last_change":"2023-10-08T06:53:46.831287+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831287+0000","last_undegraded":"2023-10-08T06:53:46.831287+0000","last_fullsized":"2023-10-08T06:53:46.831287+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.845097+0000","last_change":"2023-10-08T06:53:46.842021+0000","last_active":"2023-10-08T06:53:46.845097+0000","last_peered":"2023-10-08T06:53:46.845097+0000","last_clean":"2023-10-08T06:53:46.845097+0000","last_became_active":"2023-10-08T06:53:46.841767+0000","last_became_peered":"2023-10-08T06:53:46.841767+0000","last_unstale":"2023-10-08T06:53:46.845097+0000","last_undegraded":"2023-10-08T06:53:46.845097+0000","last_fullsized":"2023-10-08T06:53:46.845097+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:06.509561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835744+0000","last_change":"2023-10-08T06:53:46.835744+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835744+0000","last_undegraded":"2023-10-08T06:53:46.835744+0000","last_fullsized":"2023-10-08T06:53:46.835744+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.848938+0000","last_change":"2023-10-08T06:53:46.845053+0000","last_active":"2023-10-08T06:53:46.848938+0000","last_peered":"2023-10-08T06:53:46.848938+0000","last_clean":"2023-10-08T06:53:46.848938+0000","last_became_active":"2023-10-08T06:53:46.844706+0000","last_became_peered":"2023-10-08T06:53:46.844706+0000","last_unstale":"2023-10-08T06:53:46.848938+0000","last_undegraded":"2023-10-08T06:53:46.848938+0000","last_fullsized":"2023-10-08T06:53:46.848938+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:49.451557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.834908+0000","last_change":"2023-10-08T06:53:46.834908+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.834908+0000","last_undegraded":"2023-10-08T06:53:46.834908+0000","last_fullsized":"2023-10-08T06:53:46.834908+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.845369+0000","last_change":"2023-10-08T06:53:46.843744+0000","last_active":"2023-10-08T06:53:46.845369+0000","last_peered":"2023-10-08T06:53:46.845369+0000","last_clean":"2023-10-08T06:53:46.845369+0000","last_became_active":"2023-10-08T06:53:46.843504+0000","last_became_peered":"2023-10-08T06:53:46.843504+0000","last_unstale":"2023-10-08T06:53:46.845369+0000","last_undegraded":"2023-10-08T06:53:46.845369+0000","last_fullsized":"2023-10-08T06:53:46.845369+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:26.339899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828590+0000","last_change":"2023-10-08T06:53:46.828590+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828590+0000","last_undegraded":"2023-10-08T06:53:46.828590+0000","last_fullsized":"2023-10-08T06:53:46.828590+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.851054+0000","last_change":"2023-10-08T06:53:46.848518+0000","last_active":"2023-10-08T06:53:46.851054+0000","last_peered":"2023-10-08T06:53:46.851054+0000","last_clean":"2023-10-08T06:53:46.851054+0000","last_became_active":"2023-10-08T06:53:46.848237+0000","last_became_peered":"2023-10-08T06:53:46.848237+0000","last_unstale":"2023-10-08T06:53:46.851054+0000","last_undegraded":"2023-10-08T06:53:46.851054+0000","last_fullsized":"2023-10-08T06:53:46.851054+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:27:53.231805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825152+0000","last_change":"2023-10-08T06:53:46.825152+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825152+0000","last_undegraded":"2023-10-08T06:53:46.825152+0000","last_fullsized":"2023-10-08T06:53:46.825152+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.851503+0000","last_change":"2023-10-08T06:53:46.844822+0000","last_active":"2023-10-08T06:53:46.851503+0000","last_peered":"2023-10-08T06:53:46.851503+0000","last_clean":"2023-10-08T06:53:46.851503+0000","last_became_active":"2023-10-08T06:53:46.844540+0000","last_became_peered":"2023-10-08T06:53:46.844540+0000","last_unstale":"2023-10-08T06:53:46.851503+0000","last_undegraded":"2023-10-08T06:53:46.851503+0000","last_fullsized":"2023-10-08T06:53:46.851503+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:03.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825174+0000","last_change":"2023-10-08T06:53:46.825174+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825174+0000","last_undegraded":"2023-10-08T06:53:46.825174+0000","last_fullsized":"2023-10-08T06:53:46.825174+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.833708+0000","last_change":"2023-10-08T06:53:46.832905+0000","last_active":"2023-10-08T06:53:46.833708+0000","last_peered":"2023-10-08T06:53:46.833708+0000","last_clean":"2023-10-08T06:53:46.833708+0000","last_became_active":"2023-10-08T06:53:46.832624+0000","last_became_peered":"2023-10-08T06:53:46.832624+0000","last_unstale":"2023-10-08T06:53:46.833708+0000","last_undegraded":"2023-10-08T06:53:46.833708+0000","last_fullsized":"2023-10-08T06:53:46.833708+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:11.164730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830035+0000","last_change":"2023-10-08T06:53:46.830035+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830035+0000","last_undegraded":"2023-10-08T06:53:46.830035+0000","last_fullsized":"2023-10-08T06:53:46.830035+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.841506+0000","last_change":"2023-10-08T06:53:46.838423+0000","last_active":"2023-10-08T06:53:46.841506+0000","last_peered":"2023-10-08T06:53:46.841506+0000","last_clean":"2023-10-08T06:53:46.841506+0000","last_became_active":"2023-10-08T06:53:46.838071+0000","last_became_peered":"2023-10-08T06:53:46.838071+0000","last_unstale":"2023-10-08T06:53:46.841506+0000","last_undegraded":"2023-10-08T06:53:46.841506+0000","last_fullsized":"2023-10-08T06:53:46.841506+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:30.734197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833041+0000","last_change":"2023-10-08T06:53:46.833041+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833041+0000","last_undegraded":"2023-10-08T06:53:46.833041+0000","last_fullsized":"2023-10-08T06:53:46.833041+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.858068+0000","last_change":"2023-10-08T06:53:46.857001+0000","last_active":"2023-10-08T06:53:46.858068+0000","last_peered":"2023-10-08T06:53:46.858068+0000","last_clean":"2023-10-08T06:53:46.858068+0000","last_became_active":"2023-10-08T06:53:46.856707+0000","last_became_peered":"2023-10-08T06:53:46.856707+0000","last_unstale":"2023-10-08T06:53:46.858068+0000","last_undegraded":"2023-10-08T06:53:46.858068+0000","last_fullsized":"2023-10-08T06:53:46.858068+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:10.191719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.829716+0000","last_change":"2023-10-08T06:53:46.829716+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.829716+0000","last_undegraded":"2023-10-08T06:53:46.829716+0000","last_fullsized":"2023-10-08T06:53:46.829716+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.850126+0000","last_change":"2023-10-08T06:53:46.842178+0000","last_active":"2023-10-08T06:53:46.850126+0000","last_peered":"2023-10-08T06:53:46.850126+0000","last_clean":"2023-10-08T06:53:46.850126+0000","last_became_active":"2023-10-08T06:53:46.840731+0000","last_became_peered":"2023-10-08T06:53:46.840731+0000","last_unstale":"2023-10-08T06:53:46.850126+0000","last_undegraded":"2023-10-08T06:53:46.850126+0000","last_fullsized":"2023-10-08T06:53:46.850126+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:28.102115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835847+0000","last_change":"2023-10-08T06:53:46.835847+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835847+0000","last_undegraded":"2023-10-08T06:53:46.835847+0000","last_fullsized":"2023-10-08T06:53:46.835847+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.854676+0000","last_change":"2023-10-08T06:53:46.853950+0000","last_active":"2023-10-08T06:53:46.854676+0000","last_peered":"2023-10-08T06:53:46.854676+0000","last_clean":"2023-10-08T06:53:46.854676+0000","last_became_active":"2023-10-08T06:53:46.853668+0000","last_became_peered":"2023-10-08T06:53:46.853668+0000","last_unstale":"2023-10-08T06:53:46.854676+0000","last_undegraded":"2023-10-08T06:53:46.854676+0000","last_fullsized":"2023-10-08T06:53:46.854676+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:37.692117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.840640+0000","last_change":"2023-10-08T06:53:46.840640+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.840640+0000","last_undegraded":"2023-10-08T06:53:46.840640+0000","last_fullsized":"2023-10-08T06:53:46.840640+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.850594+0000","last_change":"2023-10-08T06:53:46.849784+0000","last_active":"2023-10-08T06:53:46.850594+0000","last_peered":"2023-10-08T06:53:46.850594+0000","last_clean":"2023-10-08T06:53:46.850594+0000","last_became_active":"2023-10-08T06:53:46.849527+0000","last_became_peered":"2023-10-08T06:53:46.849527+0000","last_unstale":"2023-10-08T06:53:46.850594+0000","last_undegraded":"2023-10-08T06:53:46.850594+0000","last_fullsized":"2023-10-08T06:53:46.850594+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:09.620134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.841115+0000","last_change":"2023-10-08T06:53:46.841115+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.841115+0000","last_undegraded":"2023-10-08T06:53:46.841115+0000","last_fullsized":"2023-10-08T06:53:46.841115+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853280+0000","last_change":"2023-10-08T06:53:46.849864+0000","last_active":"2023-10-08T06:53:46.853280+0000","last_peered":"2023-10-08T06:53:46.853280+0000","last_clean":"2023-10-08T06:53:46.853280+0000","last_became_active":"2023-10-08T06:53:46.849576+0000","last_became_peered":"2023-10-08T06:53:46.849576+0000","last_unstale":"2023-10-08T06:53:46.853280+0000","last_undegraded":"2023-10-08T06:53:46.853280+0000","last_fullsized":"2023-10-08T06:53:46.853280+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:36.369411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837276+0000","last_change":"2023-10-08T06:53:46.837276+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837276+0000","last_undegraded":"2023-10-08T06:53:46.837276+0000","last_fullsized":"2023-10-08T06:53:46.837276+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.847545+0000","last_change":"2023-10-08T06:53:46.846147+0000","last_active":"2023-10-08T06:53:46.847545+0000","last_peered":"2023-10-08T06:53:46.847545+0000","last_clean":"2023-10-08T06:53:46.847545+0000","last_became_active":"2023-10-08T06:53:46.845910+0000","last_became_peered":"2023-10-08T06:53:46.845910+0000","last_unstale":"2023-10-08T06:53:46.847545+0000","last_undegraded":"2023-10-08T06:53:46.847545+0000","last_fullsized":"2023-10-08T06:53:46.847545+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:11.739527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835581+0000","last_change":"2023-10-08T06:53:46.835581+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835581+0000","last_undegraded":"2023-10-08T06:53:46.835581+0000","last_fullsized":"2023-10-08T06:53:46.835581+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.858105+0000","last_change":"2023-10-08T06:53:46.857013+0000","last_active":"2023-10-08T06:53:46.858105+0000","last_peered":"2023-10-08T06:53:46.858105+0000","last_clean":"2023-10-08T06:53:46.858105+0000","last_became_active":"2023-10-08T06:53:46.856762+0000","last_became_peered":"2023-10-08T06:53:46.856762+0000","last_unstale":"2023-10-08T06:53:46.858105+0000","last_undegraded":"2023-10-08T06:53:46.858105+0000","last_fullsized":"2023-10-08T06:53:46.858105+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:12.313863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837818+0000","last_change":"2023-10-08T06:53:46.837818+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837818+0000","last_undegraded":"2023-10-08T06:53:46.837818+0000","last_fullsized":"2023-10-08T06:53:46.837818+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855358+0000","last_change":"2023-10-08T06:53:46.854554+0000","last_active":"2023-10-08T06:53:46.855358+0000","last_peered":"2023-10-08T06:53:46.855358+0000","last_clean":"2023-10-08T06:53:46.855358+0000","last_became_active":"2023-10-08T06:53:46.854235+0000","last_became_peered":"2023-10-08T06:53:46.854235+0000","last_unstale":"2023-10-08T06:53:46.855358+0000","last_undegraded":"2023-10-08T06:53:46.855358+0000","last_fullsized":"2023-10-08T06:53:46.855358+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:54:13.484888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832393+0000","last_change":"2023-10-08T06:53:46.832393+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832393+0000","last_undegraded":"2023-10-08T06:53:46.832393+0000","last_fullsized":"2023-10-08T06:53:46.832393+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.852782+0000","last_change":"2023-10-08T06:53:46.848822+0000","last_active":"2023-10-08T06:53:46.852782+0000","last_peered":"2023-10-08T06:53:46.852782+0000","last_clean":"2023-10-08T06:53:46.852782+0000","last_became_active":"2023-10-08T06:53:46.848489+0000","last_became_peered":"2023-10-08T06:53:46.848489+0000","last_unstale":"2023-10-08T06:53:46.852782+0000","last_undegraded":"2023-10-08T06:53:46.852782+0000","last_fullsized":"2023-10-08T06:53:46.852782+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:05:54.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.840127+0000","last_change":"2023-10-08T06:53:46.840127+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.840127+0000","last_undegraded":"2023-10-08T06:53:46.840127+0000","last_fullsized":"2023-10-08T06:53:46.840127+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855229+0000","last_change":"2023-10-08T06:53:46.853408+0000","last_active":"2023-10-08T06:53:46.855229+0000","last_peered":"2023-10-08T06:53:46.855229+0000","last_clean":"2023-10-08T06:53:46.855229+0000","last_became_active":"2023-10-08T06:53:46.853121+0000","last_became_peered":"2023-10-08T06:53:46.853121+0000","last_unstale":"2023-10-08T06:53:46.855229+0000","last_undegraded":"2023-10-08T06:53:46.855229+0000","last_fullsized":"2023-10-08T06:53:46.855229+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:01.990568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835282+0000","last_change":"2023-10-08T06:53:46.835282+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835282+0000","last_undegraded":"2023-10-08T06:53:46.835282+0000","last_fullsized":"2023-10-08T06:53:46.835282+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.852616+0000","last_change":"2023-10-08T06:53:46.841902+0000","last_active":"2023-10-08T06:53:46.852616+0000","last_peered":"2023-10-08T06:53:46.852616+0000","last_clean":"2023-10-08T06:53:46.852616+0000","last_became_active":"2023-10-08T06:53:46.841528+0000","last_became_peered":"2023-10-08T06:53:46.841528+0000","last_unstale":"2023-10-08T06:53:46.852616+0000","last_undegraded":"2023-10-08T06:53:46.852616+0000","last_fullsized":"2023-10-08T06:53:46.852616+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:16.903470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833006+0000","last_change":"2023-10-08T06:53:46.833006+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833006+0000","last_undegraded":"2023-10-08T06:53:46.833006+0000","last_fullsized":"2023-10-08T06:53:46.833006+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.857684+0000","last_change":"2023-10-08T06:53:46.856348+0000","last_active":"2023-10-08T06:53:46.857684+0000","last_peered":"2023-10-08T06:53:46.857684+0000","last_clean":"2023-10-08T06:53:46.857684+0000","last_became_active":"2023-10-08T06:53:46.856059+0000","last_became_peered":"2023-10-08T06:53:46.856059+0000","last_unstale":"2023-10-08T06:53:46.857684+0000","last_undegraded":"2023-10-08T06:53:46.857684+0000","last_fullsized":"2023-10-08T06:53:46.857684+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:28.598908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833355+0000","last_change":"2023-10-08T06:53:46.833355+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833355+0000","last_undegraded":"2023-10-08T06:53:46.833355+0000","last_fullsized":"2023-10-08T06:53:46.833355+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853288+0000","last_change":"2023-10-08T06:53:46.851826+0000","last_active":"2023-10-08T06:53:46.853288+0000","last_peered":"2023-10-08T06:53:46.853288+0000","last_clean":"2023-10-08T06:53:46.853288+0000","last_became_active":"2023-10-08T06:53:46.851307+0000","last_became_peered":"2023-10-08T06:53:46.851307+0000","last_unstale":"2023-10-08T06:53:46.853288+0000","last_undegraded":"2023-10-08T06:53:46.853288+0000","last_fullsized":"2023-10-08T06:53:46.853288+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:28.030370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836735+0000","last_change":"2023-10-08T06:53:46.836735+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836735+0000","last_undegraded":"2023-10-08T06:53:46.836735+0000","last_fullsized":"2023-10-08T06:53:46.836735+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855012+0000","last_change":"2023-10-08T06:53:46.854168+0000","last_active":"2023-10-08T06:53:46.855012+0000","last_peered":"2023-10-08T06:53:46.855012+0000","last_clean":"2023-10-08T06:53:46.855012+0000","last_became_active":"2023-10-08T06:53:46.853876+0000","last_became_peered":"2023-10-08T06:53:46.853876+0000","last_unstale":"2023-10-08T06:53:46.855012+0000","last_undegraded":"2023-10-08T06:53:46.855012+0000","last_fullsized":"2023-10-08T06:53:46.855012+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:46.027669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831727+0000","last_change":"2023-10-08T06:53:46.831727+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831727+0000","last_undegraded":"2023-10-08T06:53:46.831727+0000","last_fullsized":"2023-10-08T06:53:46.831727+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855809+0000","last_change":"2023-10-08T06:53:46.854126+0000","last_active":"2023-10-08T06:53:46.855809+0000","last_peered":"2023-10-08T06:53:46.855809+0000","last_clean":"2023-10-08T06:53:46.855809+0000","last_became_active":"2023-10-08T06:53:46.853833+0000","last_became_peered":"2023-10-08T06:53:46.853833+0000","last_unstale":"2023-10-08T06:53:46.855809+0000","last_undegraded":"2023-10-08T06:53:46.855809+0000","last_fullsized":"2023-10-08T06:53:46.855809+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:37.677941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825803+0000","last_change":"2023-10-08T06:53:46.825803+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825803+0000","last_undegraded":"2023-10-08T06:53:46.825803+0000","last_fullsized":"2023-10-08T06:53:46.825803+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853319+0000","last_change":"2023-10-08T06:53:46.851411+0000","last_active":"2023-10-08T06:53:46.853319+0000","last_peered":"2023-10-08T06:53:46.853319+0000","last_clean":"2023-10-08T06:53:46.853319+0000","last_became_active":"2023-10-08T06:53:46.850930+0000","last_became_peered":"2023-10-08T06:53:46.850930+0000","last_unstale":"2023-10-08T06:53:46.853319+0000","last_undegraded":"2023-10-08T06:53:46.853319+0000","last_fullsized":"2023-10-08T06:53:46.853319+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:27.974110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828614+0000","last_change":"2023-10-08T06:53:46.828614+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828614+0000","last_undegraded":"2023-10-08T06:53:46.828614+0000","last_fullsized":"2023-10-08T06:53:46.828614+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853067+0000","last_change":"2023-10-08T06:53:46.852038+0000","last_active":"2023-10-08T06:53:46.853067+0000","last_peered":"2023-10-08T06:53:46.853067+0000","last_clean":"2023-10-08T06:53:46.853067+0000","last_became_active":"2023-10-08T06:53:46.851840+0000","last_became_peered":"2023-10-08T06:53:46.851840+0000","last_unstale":"2023-10-08T06:53:46.853067+0000","last_undegraded":"2023-10-08T06:53:46.853067+0000","last_fullsized":"2023-10-08T06:53:46.853067+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:14.079295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.834744+0000","last_change":"2023-10-08T06:53:46.834744+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.834744+0000","last_undegraded":"2023-10-08T06:53:46.834744+0000","last_fullsized":"2023-10-08T06:53:46.834744+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.848916+0000","last_change":"2023-10-08T06:53:46.844152+0000","last_active":"2023-10-08T06:53:46.848916+0000","last_peered":"2023-10-08T06:53:46.848916+0000","last_clean":"2023-10-08T06:53:46.848916+0000","last_became_active":"2023-10-08T06:53:46.843835+0000","last_became_peered":"2023-10-08T06:53:46.843835+0000","last_unstale":"2023-10-08T06:53:46.848916+0000","last_undegraded":"2023-10-08T06:53:46.848916+0000","last_fullsized":"2023-10-08T06:53:46.848916+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:04:41.456787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832811+0000","last_change":"2023-10-08T06:53:46.832811+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832811+0000","last_undegraded":"2023-10-08T06:53:46.832811+0000","last_fullsized":"2023-10-08T06:53:46.832811+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.850421+0000","last_change":"2023-10-08T06:53:46.846292+0000","last_active":"2023-10-08T06:53:46.850421+0000","last_peered":"2023-10-08T06:53:46.850421+0000","last_clean":"2023-10-08T06:53:46.850421+0000","last_became_active":"2023-10-08T06:53:46.845869+0000","last_became_peered":"2023-10-08T06:53:46.845869+0000","last_unstale":"2023-10-08T06:53:46.850421+0000","last_undegraded":"2023-10-08T06:53:46.850421+0000","last_fullsized":"2023-10-08T06:53:46.850421+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:21:37.089110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.827498+0000","last_change":"2023-10-08T06:53:46.827498+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.827498+0000","last_undegraded":"2023-10-08T06:53:46.827498+0000","last_fullsized":"2023-10-08T06:53:46.827498+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855333+0000","last_change":"2023-10-08T06:53:46.853823+0000","last_active":"2023-10-08T06:53:46.855333+0000","last_peered":"2023-10-08T06:53:46.855333+0000","last_clean":"2023-10-08T06:53:46.855333+0000","last_became_active":"2023-10-08T06:53:46.853347+0000","last_became_peered":"2023-10-08T06:53:46.853347+0000","last_unstale":"2023-10-08T06:53:46.855333+0000","last_undegraded":"2023-10-08T06:53:46.855333+0000","last_fullsized":"2023-10-08T06:53:46.855333+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:02.956148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"172'2729","reported_seq":3394,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-08T06:53:45.826415+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T06:53:45.826415+0000","last_peered":"2023-10-08T06:53:45.826415+0000","last_clean":"2023-10-08T06:53:45.826415+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T06:53:45.826415+0000","last_undegraded":"2023-10-08T06:53:45.826415+0000","last_fullsized":"2023-10-08T06:53:45.826415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2729,"log_dups_size":0,"ondisk_log_size":2729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575775,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60348,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311492,"statfs":{"total":96636764160,"available":96574967808,"internally_reserved":0,"allocated":2867200,"data_stored":1759119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.21,"15min":1.2330000000000001},"min":{"1min":0.91900000000000004,"5min":0.78700000000000003,"15min":0.70399999999999996},"max":{"1min":1.6950000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.224,"15min":1.248},"min":{"1min":0.93799999999999994,"5min":0.747,"15min":0.72899999999999998},"max":{"1min":1.522,"5min":1.7150000000000001,"15min":1.849},"last":1.5009999999999999}]},{"osd":1,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2829999999999999,"15min":1.262},"min":{"1min":0.89200000000000002,"5min":0.84299999999999997,"15min":0.70399999999999996},"max":{"1min":1.6970000000000001,"5min":1.712,"15min":1.8480000000000001},"last":1.157},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.284,"15min":1.3220000000000001},"min":{"1min":0.95999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7450000000000001,"5min":1.9299999999999999,"15min":2.4809999999999999},"last":1.591}]},{"osd":2,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.0740000000000001,"5min":0.90900000000000003,"15min":0.81899999999999995},"max":{"1min":1.627,"5min":2.7040000000000002,"15min":3.4350000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.387,"5min":1.381,"15min":1.3740000000000001},"min":{"1min":0.95699999999999996,"5min":0.88900000000000001,"15min":0.83199999999999996},"max":{"1min":1.873,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":0.99099999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.339,"15min":1.335},"min":{"1min":1.0189999999999999,"5min":0.97799999999999998,"15min":0.85899999999999999},"max":{"1min":1.5609999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.212},{"interface":"front","average":{"1min":1.5,"5min":1.46,"15min":1.419},"min":{"1min":1.147,"5min":0.98899999999999999,"15min":0.83099999999999996},"max":{"1min":2.0379999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.3620000000000001}]},{"osd":4,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.333,"15min":1.2869999999999999},"min":{"1min":0.93999999999999995,"5min":0.83199999999999996,"15min":0.66200000000000003},"max":{"1min":1.952,"5min":1.952,"15min":3.2839999999999998},"last":1.792},{"interface":"front","average":{"1min":1.484,"5min":1.456,"15min":1.391},"min":{"1min":1.1579999999999999,"5min":0.90300000000000002,"15min":0.80800000000000005},"max":{"1min":1.708,"5min":2.004,"15min":2.1909999999999998},"last":1.4510000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.474,"15min":1.4399999999999999},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":0.86499999999999999},"max":{"1min":2.1499999999999999,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":1.272},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.504,"15min":1.4790000000000001},"min":{"1min":1.2909999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.6439999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.5680000000000001,"15min":1.556},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":0.94599999999999995},"max":{"1min":2.1970000000000001,"5min":2.4350000000000001,"15min":3.1360000000000001},"last":1.728},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5269999999999999,"15min":1.502},"min":{"1min":1.0049999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":2.0190000000000001,"5min":2.1309999999999998,"15min":2.3140000000000001},"last":1.6839999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41720,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330120,"statfs":{"total":96636764160,"available":96594042880,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3009999999999999,"15min":1.3480000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7310000000000001,"5min":1.792,"15min":2.508},"last":1.605},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3089999999999999,"15min":1.337},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.609,"5min":1.8280000000000001,"15min":2.1139999999999999},"last":1.3520000000000001}]},{"osd":1,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2170000000000001,"15min":1.248},"min":{"1min":0.876,"5min":0.876,"15min":0.59699999999999998},"max":{"1min":1.605,"5min":1.821,"15min":1.919},"last":1.413},{"interface":"front","average":{"1min":1.224,"5min":1.2769999999999999,"15min":1.2629999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.53800000000000003},"max":{"1min":1.7769999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3720000000000001}]},{"osd":2,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2170000000000001,"15min":1.2290000000000001},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.75600000000000001},"max":{"1min":1.6499999999999999,"5min":1.722,"15min":1.8919999999999999},"last":1.514},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.29,"15min":1.2989999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5229999999999999,"5min":1.6379999999999999,"15min":2.3839999999999999},"last":1.1739999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4259999999999999,"15min":1.4039999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.83399999999999996},"max":{"1min":1.8700000000000001,"5min":1.976,"15min":2.1499999999999999},"last":1.478},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.478,"15min":1.48},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.89400000000000002},"max":{"1min":1.8180000000000001,"5min":1.915,"15min":2.0289999999999999},"last":1.274}]},{"osd":4,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.3580000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.72399999999999998},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.226},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4379999999999999,"15min":1.4390000000000001},"min":{"1min":0.878,"5min":0.878,"15min":0.80200000000000005},"max":{"1min":1.9099999999999999,"5min":1.9910000000000001,"15min":2.0089999999999999},"last":1.006}]},{"osd":5,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.431,"15min":1.4139999999999999},"min":{"1min":0.997,"5min":0.94799999999999995,"15min":0.69499999999999995},"max":{"1min":1.798,"5min":1.893,"15min":2.7309999999999999},"last":1.319},{"interface":"front","average":{"1min":1.575,"5min":1.5589999999999999,"15min":1.548},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78300000000000003},"max":{"1min":2.0640000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.121}]},{"osd":7,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.591,"15min":1.579},"min":{"1min":1.0960000000000001,"5min":1.0680000000000001,"15min":0.92000000000000004},"max":{"1min":1.984,"5min":2.0710000000000002,"15min":2.2090000000000001},"last":1.454},{"interface":"front","average":{"1min":1.488,"5min":1.573,"15min":1.5660000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.82099999999999995},"max":{"1min":1.946,"5min":2.222,"15min":2.6469999999999998},"last":1.5549999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59768,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312072,"statfs":{"total":96636764160,"available":96575561728,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2310000000000001,"15min":1.218},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.66600000000000004},"max":{"1min":1.5740000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.296,"15min":1.2569999999999999},"min":{"1min":0.95399999999999996,"5min":0.82699999999999996,"15min":0.78800000000000003},"max":{"1min":1.629,"5min":1.897,"15min":2.4700000000000002},"last":1.206}]},{"osd":1,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.26,"15min":1.2230000000000001},"min":{"1min":0.95499999999999996,"5min":0.85799999999999998,"15min":0.72499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.349,"5min":1.375,"15min":1.323},"min":{"1min":0.90400000000000003,"5min":0.88700000000000001,"15min":0.78200000000000003},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.25}]},{"osd":2,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3480000000000001,"15min":1.3280000000000001},"min":{"1min":0.98399999999999999,"5min":0.85199999999999998,"15min":0.83199999999999996},"max":{"1min":1.6479999999999999,"5min":1.9359999999999999,"15min":2.351},"last":1.3},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.3160000000000001},"min":{"1min":0.91000000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.667,"5min":1.7809999999999999,"15min":1.964},"last":0.94499999999999995}]},{"osd":3,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3959999999999999,"15min":1.3540000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.528,"15min":1.464},"min":{"1min":1.139,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.9910000000000001,"5min":3.0059999999999998,"15min":5.1630000000000003},"last":1.042}]},{"osd":4,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3320000000000001,"15min":1.3240000000000001},"min":{"1min":0.96299999999999997,"5min":0.89300000000000002,"15min":0.81499999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4039999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4379999999999999,"15min":1.4239999999999999},"min":{"1min":0.96899999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.113}]},{"osd":6,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.4990000000000001,"15min":1.4750000000000001},"min":{"1min":1.248,"5min":0.96399999999999997,"15min":0.81200000000000006},"max":{"1min":2.1150000000000002,"5min":2.1659999999999999,"15min":5.5019999999999998},"last":1.155},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4930000000000001,"15min":1.48},"min":{"1min":1.0509999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.0139999999999998,"5min":2.1040000000000001,"15min":2.2890000000000001},"last":1.446}]},{"osd":7,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.5660000000000001,"15min":1.5629999999999999},"min":{"1min":1.3400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":5.8780000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.679,"5min":1.6839999999999999,"15min":1.6299999999999999},"min":{"1min":1.415,"5min":1.1719999999999999,"15min":0.97099999999999997},"max":{"1min":2.206,"5min":2.6789999999999998,"15min":5.399},"last":1.381}]}]},{"osd":4,"up_from":13,"seq":55834575774,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41720,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330120,"statfs":{"total":96636764160,"available":96594042880,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.389,"15min":1.3460000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.3999999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.69799999999999995},"max":{"1min":1.7789999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.518}]},{"osd":1,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2749999999999999,"15min":1.28},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.73,"5min":1.784,"15min":1.9339999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.64000000000000001},"max":{"1min":1.613,"5min":1.6870000000000001,"15min":1.7290000000000001},"last":1.232}]},{"osd":2,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.2889999999999999,"15min":1.248},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":1.046},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.3759999999999999},"min":{"1min":1,"5min":0.93100000000000005,"15min":0.85399999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.651}]},{"osd":3,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.339,"15min":1.3109999999999999},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.4670000000000001,"15min":1.4099999999999999},"min":{"1min":1.0629999999999999,"5min":0.97299999999999998,"15min":0.93400000000000005},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.5429999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.4139999999999999,"15min":1.351},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.66700000000000004},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.4450000000000001,"15min":1.375},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.749},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.6060000000000001}]},{"osd":6,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.5329999999999999,"15min":1.425},"min":{"1min":0.97899999999999998,"5min":0.91100000000000003,"15min":0.72699999999999998},"max":{"1min":5.3339999999999996,"5min":5.3339999999999996,"15min":5.3339999999999996},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6120000000000001,"15min":1.546},"min":{"1min":1.212,"5min":0.96699999999999997,"15min":0.91400000000000003},"max":{"1min":2.1059999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":1.7250000000000001}]},{"osd":7,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.548,"15min":1.49},"min":{"1min":1.202,"5min":1.135,"15min":0.99099999999999999},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.784,"5min":1.6859999999999999,"15min":1.6060000000000001},"min":{"1min":1.0049999999999999,"5min":0.98999999999999999,"15min":0.93600000000000005},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":1.6879999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575776,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47288,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324552,"statfs":{"total":96636764160,"available":96588341248,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.95499999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.44500000000000001},"max":{"1min":1.3200000000000001,"5min":1.71,"15min":1.71},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.196,"15min":1.175},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6719999999999999,"5min":1.851,"15min":2.137},"last":0.93500000000000005}]},{"osd":1,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.024,"15min":1.018},"min":{"1min":0.70999999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2370000000000001,"5min":1.71,"15min":1.823},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.321,"5min":1.2989999999999999,"15min":1.3049999999999999},"min":{"1min":0.91000000000000003,"5min":0.872,"15min":0.77000000000000002},"max":{"1min":1.51,"5min":1.75,"15min":1.893},"last":0.73599999999999999}]},{"osd":2,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1739999999999999,"15min":1.137},"min":{"1min":0.85299999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6279999999999999,"5min":1.726,"15min":1.7410000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1919999999999999,"15min":1.1930000000000001},"min":{"1min":0.90400000000000003,"5min":0.68600000000000005,"15min":0.61399999999999999},"max":{"1min":1.589,"5min":1.7809999999999999,"15min":1.8360000000000001},"last":1.171}]},{"osd":4,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.284,"15min":1.2649999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83299999999999996},"max":{"1min":1.8560000000000001,"5min":1.97,"15min":1.97},"last":1.349},{"interface":"front","average":{"1min":1.482,"5min":1.4650000000000001,"15min":1.452},"min":{"1min":1.1859999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.8100000000000001,"5min":1.905,"15min":2.1960000000000002},"last":1.3080000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.391,"15min":1.385},"min":{"1min":1.085,"5min":0.98399999999999999,"15min":0.88},"max":{"1min":1.6399999999999999,"5min":2.27,"15min":3.0009999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.415,"15min":1.421},"min":{"1min":1.1890000000000001,"5min":0.995,"15min":0.94099999999999995},"max":{"1min":1.6739999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.2809999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4059999999999999,"15min":1.3959999999999999},"min":{"1min":0.98399999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.016},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.534,"15min":1.518},"min":{"1min":1.2170000000000001,"5min":1.1639999999999999,"15min":0.99099999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.544},"last":1.371}]},{"osd":7,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.5649999999999999,"15min":1.5660000000000001},"min":{"1min":1.3069999999999999,"5min":1.1839999999999999,"15min":1.008},"max":{"1min":1.8939999999999999,"5min":2.008,"15min":2.347},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.5049999999999999,"15min":1.4790000000000001},"min":{"1min":1.0980000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1800000000000002},"last":1.2070000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47288,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324552,"statfs":{"total":96636764160,"available":96588341248,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.93700000000000006},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.4650000000000001,"15min":1.657},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.036,"5min":1.0549999999999999,"15min":1.0760000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5249999999999999,"5min":1.661,"15min":2.2679999999999998},"last":1.26}]},{"osd":1,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.004,"15min":1.0069999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.696,"5min":1.7170000000000001,"15min":1.996},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.137,"5min":1.1679999999999999,"15min":1.1819999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":2.1139999999999999},"last":0.92800000000000005}]},{"osd":3,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2270000000000001,"15min":1.2230000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.772,"5min":1.925,"15min":2.3929999999999998},"last":1.075},{"interface":"front","average":{"1min":1.177,"5min":1.256,"15min":1.258},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.46,"5min":1.804,"15min":1.891},"last":1.0389999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4330000000000001,"15min":1.444},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.859,"5min":1.9430000000000001,"15min":3.0939999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.3160000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7470000000000001,"5min":1.7549999999999999,"15min":3.7320000000000002},"last":1.4379999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.429,"15min":1.425},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":3.3519999999999999},"last":1.48},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.5029999999999999,"15min":1.528},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.9810000000000001,"5min":2.2719999999999998,"15min":2.5249999999999999},"last":1.321}]},{"osd":6,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.673,"5min":1.9990000000000001,"15min":2.206},"last":1.532},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.548,"15min":1.532},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9610000000000001,"5min":1.966,"15min":3.02},"last":1.6319999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.51,"15min":1.492},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.861,"5min":2.0790000000000002,"15min":4.2869999999999999},"last":1.591},{"interface":"front","average":{"1min":1.55,"5min":1.5940000000000001,"15min":1.579},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8540000000000001,"5min":2.0099999999999998,"15min":3.0760000000000001},"last":1.375}]}]},{"osd":1,"up_from":13,"seq":55834575774,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29240,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342600,"statfs":{"total":96636764160,"available":96606822400,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.0349999999999999,"15min":1.0029999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.601,"5min":4.601,"15min":4.601},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2190000000000001,"15min":1.22},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.72599999999999998}]},{"osd":2,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.0329999999999999,"15min":1.0229999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.3179999999999996,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.385,"5min":1.2250000000000001,"15min":1.1679999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":5.4059999999999997,"5min":5.4059999999999997,"15min":5.4059999999999997},"last":1.163}]},{"osd":3,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.3029999999999999,"15min":1.2609999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.5220000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1579999999999999,"15min":1.1160000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.0069999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.3919999999999999,"15min":1.3680000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":4.2409999999999997,"5min":4.2409999999999997,"15min":4.2409999999999997},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.337,"15min":1.298},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":1.1970000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.4279999999999999,"15min":1.3600000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":4.7530000000000001,"5min":4.7530000000000001,"15min":4.7530000000000001},"last":1.296},{"interface":"front","average":{"1min":1.742,"5min":1.5980000000000001,"15min":1.5329999999999999},"min":{"1min":1.03,"5min":1.03,"15min":0.75},"max":{"1min":5.7389999999999999,"5min":5.7389999999999999,"15min":5.7389999999999999},"last":1.429}]},{"osd":6,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.5700000000000001,"15min":1.53},"min":{"1min":1.05,"5min":1.05,"15min":1.02},"max":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.661,"5min":1.5329999999999999,"15min":1.4810000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":1.52}]},{"osd":7,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.605,"15min":1.5489999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":5.6289999999999996,"5min":5.6289999999999996,"15min":5.6289999999999996},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.607,"15min":1.5549999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.5270000000000001,"5min":4.5270000000000001,"15min":4.5270000000000001},"last":1.48}]}]},{"osd":0,"up_from":13,"seq":55834575775,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38140,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333700,"statfs":{"total":96636764160,"available":96597708800,"internally_reserved":0,"allocated":2867200,"data_stored":1759119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5669999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.173,"5min":1.123,"15min":1.091},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.222,"15min":1.163},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.226,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0580000000000001,"15min":1.0800000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.2669999999999999,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.294,"15min":1.27},"min":{"1min":0.625,"5min":0.625,"15min":0.621},"max":{"1min":2.8250000000000002,"5min":4.0220000000000002,"15min":4.0220000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.159,"5min":1.2110000000000001,"15min":1.1839999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.319}]},{"osd":4,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.349,"15min":1.3680000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4199999999999999,"15min":1.411},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.6799999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4330000000000001,"15min":1.4019999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.68700000000000006},"max":{"1min":2.0659999999999998,"5min":2.0920000000000001,"15min":2.2480000000000002},"last":1.339},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5249999999999999,"15min":1.478},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":3.2229999999999999,"5min":3.887,"15min":3.887},"last":1.262}]},{"osd":6,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.504,"15min":1.476},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.514,"15min":1.508},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.748},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.2070000000000001}]},{"osd":7,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.546,"15min":1.5369999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.86199999999999999},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.573,"15min":1.548},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.194,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.6459999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T06:53:52.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T06:53:53.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.060+0000 7fd9549d7640 1 Processor -- start 2023-10-08T06:53:53.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.060+0000 7fd9549d7640 1 -- start start 2023-10-08T06:53:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.062+0000 7fd9549d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd950165cc0 0x7fd9501680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.062+0000 7fd9549d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9500c48f0 con 0x7fd950160940 2023-10-08T06:53:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.062+0000 7fd9549d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9500710f0 con 0x7fd95015cd90 2023-10-08T06:53:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.062+0000 7fd9549d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9500c4d80 con 0x7fd950165cc0 2023-10-08T06:53:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.062+0000 7fd945d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd950160940 0x7fd9500709e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43994/0 (socket says 172.21.15.70:43994) 2023-10-08T06:53:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.062+0000 7fd945d9b640 1 -- 172.21.15.70:0/3925458169 learned_addr learned my addr 172.21.15.70:0/3925458169 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.062+0000 7fd94ed9d640 1 --2- 172.21.15.70:0/3925458169 >> v2:172.21.15.70:3300/0 conn(0x7fd950165cc0 0x7fd9501680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:53.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.062+0000 7fd94ed9d640 1 -- 172.21.15.70:0/3925458169 >> v1:172.21.15.181:6789/0 conn(0x7fd95015cd90 legacy=0x7fd9500702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:53.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.062+0000 7fd94ed9d640 1 -- 172.21.15.70:0/3925458169 >> v1:172.21.15.70:6789/0 conn(0x7fd950160940 legacy=0x7fd9500709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.063+0000 7fd94ed9d640 1 -- 172.21.15.70:0/3925458169 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9500c3ce0 con 0x7fd950165cc0 2023-10-08T06:53:53.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.063+0000 7fd94dd9b640 1 -- 172.21.15.70:0/3925458169 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486480888 0 0) 0x7fd9500c48f0 con 0x7fd950160940 2023-10-08T06:53:53.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.064+0000 7fd94ed9d640 1 --2- 172.21.15.70:0/3925458169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd950165cc0 0x7fd9501680b0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd9480097d0 tx=0x7fd94802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15f6fa6fbbc7496 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:53.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.064+0000 7fd94dd9b640 1 -- 172.21.15.70:0/3925458169 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94802d040 con 0x7fd950165cc0 2023-10-08T06:53:53.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.064+0000 7fd94dd9b640 1 -- 172.21.15.70:0/3925458169 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd948031810 con 0x7fd950165cc0 2023-10-08T06:53:53.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.064+0000 7fd94dd9b640 1 -- 172.21.15.70:0/3925458169 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd948031ba0 con 0x7fd950165cc0 2023-10-08T06:53:53.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.064+0000 7fd9549d7640 1 -- 172.21.15.70:0/3925458169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd950165cc0 msgr2=0x7fd9501680b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.064+0000 7fd9549d7640 1 --2- 172.21.15.70:0/3925458169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd950165cc0 0x7fd9501680b0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd9480097d0 tx=0x7fd94802b8b0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.065+0000 7fd9549d7640 1 -- 172.21.15.70:0/3925458169 shutdown_connections 2023-10-08T06:53:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.065+0000 7fd9549d7640 1 --2- 172.21.15.70:0/3925458169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd950165cc0 0x7fd9501680b0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.065+0000 7fd9549d7640 1 -- 172.21.15.70:0/3925458169 >> 172.21.15.70:0/3925458169 conn(0x7fd95006a380 msgr2=0x7fd9500b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.065+0000 7fd9549d7640 1 -- 172.21.15.70:0/3925458169 shutdown_connections 2023-10-08T06:53:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.065+0000 7fd9549d7640 1 -- 172.21.15.70:0/3925458169 wait complete. 2023-10-08T06:53:53.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.066+0000 7fd9549d7640 1 Processor -- start 2023-10-08T06:53:53.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.067+0000 7fd9549d7640 1 -- start start 2023-10-08T06:53:53.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.067+0000 7fd9549d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd95015cd90 0x7fd9501489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.067+0000 7fd9549d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd950160940 0x7fd9500cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.067+0000 7fd9549d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd950165cc0 0x7fd950150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd9549d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd950151940 con 0x7fd95015cd90 2023-10-08T06:53:53.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd9549d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500ca950 con 0x7fd950160940 2023-10-08T06:53:53.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd9549d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd950148ef0 con 0x7fd950165cc0 2023-10-08T06:53:53.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd94e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd95015cd90 0x7fd9501489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:53.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd94e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd95015cd90 0x7fd9501489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48374/0 (socket says 172.21.15.70:48374) 2023-10-08T06:53:53.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd94e59c640 1 -- 172.21.15.70:0/989313595 learned_addr learned my addr 172.21.15.70:0/989313595 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:53.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd94e59c640 1 -- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd950165cc0 msgr2=0x7fd950150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd945d9b640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd950160940 0x7fd9500cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:53.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd94ed9d640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd950165cc0 0x7fd950150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:53.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd94e59c640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd950165cc0 0x7fd950150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd94e59c640 1 -- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd950160940 msgr2=0x7fd9500cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd94e59c640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd950160940 0x7fd9500cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.068+0000 7fd94e59c640 1 -- 172.21.15.70:0/989313595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948009400 con 0x7fd95015cd90 2023-10-08T06:53:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.069+0000 7fd94e59c640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd95015cd90 0x7fd9501489b0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fd94000c7f0 tx=0x7fd94000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.069+0000 7fd9477fe640 1 -- 172.21.15.70:0/989313595 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94000aa50 con 0x7fd95015cd90 2023-10-08T06:53:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.069+0000 7fd9477fe640 1 -- 172.21.15.70:0/989313595 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94000abb0 con 0x7fd95015cd90 2023-10-08T06:53:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.069+0000 7fd9477fe640 1 -- 172.21.15.70:0/989313595 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940012420 con 0x7fd95015cd90 2023-10-08T06:53:53.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.069+0000 7fd9549d7640 1 -- 172.21.15.70:0/989313595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd950151380 con 0x7fd95015cd90 2023-10-08T06:53:53.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.069+0000 7fd9549d7640 1 -- 172.21.15.70:0/989313595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9500ca5e0 con 0x7fd95015cd90 2023-10-08T06:53:53.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.070+0000 7fd9549d7640 1 -- 172.21.15.70:0/989313595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd950067490 con 0x7fd95015cd90 2023-10-08T06:53:53.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.071+0000 7fd9477fe640 1 -- 172.21.15.70:0/989313595 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd940012580 con 0x7fd95015cd90 2023-10-08T06:53:53.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.072+0000 7fd9477fe640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fd93006af90 0x7fd93006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.072+0000 7fd9477fe640 1 -- 172.21.15.70:0/989313595 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fd9400898c0 con 0x7fd95015cd90 2023-10-08T06:53:53.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.073+0000 7fd945d9b640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fd93006af90 0x7fd93006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:53.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.074+0000 7fd945d9b640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fd93006af90 0x7fd93006d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd938007fd0 tx=0x7fd938009040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:53.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.076+0000 7fd9477fe640 1 -- 172.21.15.70:0/989313595 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd940054690 con 0x7fd95015cd90 2023-10-08T06:53:53.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.250+0000 7fd9549d7640 1 -- 172.21.15.70:0/989313595 --> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9500c3fc0 con 0x7fd93006af90 2023-10-08T06:53:53.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.272+0000 7fd9477fe640 1 -- 172.21.15.70:0/989313595 <== mgr.9345 v2:172.21.15.181:6832/4245513204 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282075 (secure 0 0 0) 0x7fd938001c70 con 0x7fd93006af90 2023-10-08T06:53:53.282 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:53.291 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T06:53:53.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 -- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fd93006af90 msgr2=0x7fd93006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fd93006af90 0x7fd93006d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd938007fd0 tx=0x7fd938009040 comp rx=0 tx=0).stop 2023-10-08T06:53:53.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 -- 172.21.15.70:0/989313595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd95015cd90 msgr2=0x7fd9501489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd95015cd90 0x7fd9501489b0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fd94000c7f0 tx=0x7fd94000ccb0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 -- 172.21.15.70:0/989313595 shutdown_connections 2023-10-08T06:53:53.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd950165cc0 0x7fd950150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd950160940 0x7fd9500cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fd93006af90 0x7fd93006d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 --2- 172.21.15.70:0/989313595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd95015cd90 0x7fd9501489b0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 -- 172.21.15.70:0/989313595 >> 172.21.15.70:0/989313595 conn(0x7fd95006a380 msgr2=0x7fd9500ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:53.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.277+0000 7fd9549d7640 1 -- 172.21.15.70:0/989313595 shutdown_connections 2023-10-08T06:53:53.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.278+0000 7fd9549d7640 1 -- 172.21.15.70:0/989313595 wait complete. 2023-10-08T06:53:53.304 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-08T06:53:52.078361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365512,"kb_used_data":18920,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754609208,"statfs":{"total":773094113280,"available":772719828992,"internally_reserved":0,"allocated":19374080,"data_stored":10530744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":354799163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001853"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837391+0000","last_change":"2023-10-08T06:53:46.837391+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837391+0000","last_undegraded":"2023-10-08T06:53:46.837391+0000","last_fullsized":"2023-10-08T06:53:46.837391+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835340+0000","last_change":"2023-10-08T06:53:46.835340+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835340+0000","last_undegraded":"2023-10-08T06:53:46.835340+0000","last_fullsized":"2023-10-08T06:53:46.835340+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836968+0000","last_change":"2023-10-08T06:53:46.836968+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836968+0000","last_undegraded":"2023-10-08T06:53:46.836968+0000","last_fullsized":"2023-10-08T06:53:46.836968+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832209+0000","last_change":"2023-10-08T06:53:46.832209+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832209+0000","last_undegraded":"2023-10-08T06:53:46.832209+0000","last_fullsized":"2023-10-08T06:53:46.832209+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830068+0000","last_change":"2023-10-08T06:53:46.830068+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830068+0000","last_undegraded":"2023-10-08T06:53:46.830068+0000","last_fullsized":"2023-10-08T06:53:46.830068+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830196+0000","last_change":"2023-10-08T06:53:46.830196+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830196+0000","last_undegraded":"2023-10-08T06:53:46.830196+0000","last_fullsized":"2023-10-08T06:53:46.830196+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833262+0000","last_change":"2023-10-08T06:53:46.833262+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833262+0000","last_undegraded":"2023-10-08T06:53:46.833262+0000","last_fullsized":"2023-10-08T06:53:46.833262+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831994+0000","last_change":"2023-10-08T06:53:46.831994+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831994+0000","last_undegraded":"2023-10-08T06:53:46.831994+0000","last_fullsized":"2023-10-08T06:53:46.831994+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832152+0000","last_change":"2023-10-08T06:53:46.832152+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832152+0000","last_undegraded":"2023-10-08T06:53:46.832152+0000","last_fullsized":"2023-10-08T06:53:46.832152+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828877+0000","last_change":"2023-10-08T06:53:46.828877+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828877+0000","last_undegraded":"2023-10-08T06:53:46.828877+0000","last_fullsized":"2023-10-08T06:53:46.828877+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826488+0000","last_change":"2023-10-08T06:53:46.826488+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826488+0000","last_undegraded":"2023-10-08T06:53:46.826488+0000","last_fullsized":"2023-10-08T06:53:46.826488+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831044+0000","last_change":"2023-10-08T06:53:46.831044+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831044+0000","last_undegraded":"2023-10-08T06:53:46.831044+0000","last_fullsized":"2023-10-08T06:53:46.831044+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831261+0000","last_change":"2023-10-08T06:53:46.831261+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831261+0000","last_undegraded":"2023-10-08T06:53:46.831261+0000","last_fullsized":"2023-10-08T06:53:46.831261+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830002+0000","last_change":"2023-10-08T06:53:46.830002+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830002+0000","last_undegraded":"2023-10-08T06:53:46.830002+0000","last_fullsized":"2023-10-08T06:53:46.830002+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833676+0000","last_change":"2023-10-08T06:53:46.833676+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833676+0000","last_undegraded":"2023-10-08T06:53:46.833676+0000","last_fullsized":"2023-10-08T06:53:46.833676+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831113+0000","last_change":"2023-10-08T06:53:46.831113+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831113+0000","last_undegraded":"2023-10-08T06:53:46.831113+0000","last_fullsized":"2023-10-08T06:53:46.831113+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.834745+0000","last_change":"2023-10-08T06:53:46.834745+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.834745+0000","last_undegraded":"2023-10-08T06:53:46.834745+0000","last_fullsized":"2023-10-08T06:53:46.834745+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825966+0000","last_change":"2023-10-08T06:53:46.825966+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825966+0000","last_undegraded":"2023-10-08T06:53:46.825966+0000","last_fullsized":"2023-10-08T06:53:46.825966+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.841643+0000","last_change":"2023-10-08T06:53:46.841643+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.841643+0000","last_undegraded":"2023-10-08T06:53:46.841643+0000","last_fullsized":"2023-10-08T06:53:46.841643+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826979+0000","last_change":"2023-10-08T06:53:46.826979+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826979+0000","last_undegraded":"2023-10-08T06:53:46.826979+0000","last_fullsized":"2023-10-08T06:53:46.826979+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830751+0000","last_change":"2023-10-08T06:53:46.830751+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830751+0000","last_undegraded":"2023-10-08T06:53:46.830751+0000","last_fullsized":"2023-10-08T06:53:46.830751+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836034+0000","last_change":"2023-10-08T06:53:46.836034+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836034+0000","last_undegraded":"2023-10-08T06:53:46.836034+0000","last_fullsized":"2023-10-08T06:53:46.836034+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826700+0000","last_change":"2023-10-08T06:53:46.826700+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826700+0000","last_undegraded":"2023-10-08T06:53:46.826700+0000","last_fullsized":"2023-10-08T06:53:46.826700+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825977+0000","last_change":"2023-10-08T06:53:46.825977+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825977+0000","last_undegraded":"2023-10-08T06:53:46.825977+0000","last_fullsized":"2023-10-08T06:53:46.825977+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836905+0000","last_change":"2023-10-08T06:53:46.836905+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836905+0000","last_undegraded":"2023-10-08T06:53:46.836905+0000","last_fullsized":"2023-10-08T06:53:46.836905+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.838574+0000","last_change":"2023-10-08T06:53:46.838574+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.838574+0000","last_undegraded":"2023-10-08T06:53:46.838574+0000","last_fullsized":"2023-10-08T06:53:46.838574+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828887+0000","last_change":"2023-10-08T06:53:46.828887+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828887+0000","last_undegraded":"2023-10-08T06:53:46.828887+0000","last_fullsized":"2023-10-08T06:53:46.828887+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825937+0000","last_change":"2023-10-08T06:53:46.825937+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825937+0000","last_undegraded":"2023-10-08T06:53:46.825937+0000","last_fullsized":"2023-10-08T06:53:46.825937+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831793+0000","last_change":"2023-10-08T06:53:46.831793+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831793+0000","last_undegraded":"2023-10-08T06:53:46.831793+0000","last_fullsized":"2023-10-08T06:53:46.831793+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825313+0000","last_change":"2023-10-08T06:53:46.825313+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825313+0000","last_undegraded":"2023-10-08T06:53:46.825313+0000","last_fullsized":"2023-10-08T06:53:46.825313+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828751+0000","last_change":"2023-10-08T06:53:46.828751+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828751+0000","last_undegraded":"2023-10-08T06:53:46.828751+0000","last_fullsized":"2023-10-08T06:53:46.828751+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831002+0000","last_change":"2023-10-08T06:53:46.831002+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831002+0000","last_undegraded":"2023-10-08T06:53:46.831002+0000","last_fullsized":"2023-10-08T06:53:46.831002+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837790+0000","last_change":"2023-10-08T06:53:46.837790+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837790+0000","last_undegraded":"2023-10-08T06:53:46.837790+0000","last_fullsized":"2023-10-08T06:53:46.837790+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.841915+0000","last_change":"2023-10-08T06:53:46.838837+0000","last_active":"2023-10-08T06:53:46.841915+0000","last_peered":"2023-10-08T06:53:46.841915+0000","last_clean":"2023-10-08T06:53:46.841915+0000","last_became_active":"2023-10-08T06:53:46.838366+0000","last_became_peered":"2023-10-08T06:53:46.838366+0000","last_unstale":"2023-10-08T06:53:46.841915+0000","last_undegraded":"2023-10-08T06:53:46.841915+0000","last_fullsized":"2023-10-08T06:53:46.841915+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:26.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836525+0000","last_change":"2023-10-08T06:53:46.836525+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836525+0000","last_undegraded":"2023-10-08T06:53:46.836525+0000","last_fullsized":"2023-10-08T06:53:46.836525+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855058+0000","last_change":"2023-10-08T06:53:46.854455+0000","last_active":"2023-10-08T06:53:46.855058+0000","last_peered":"2023-10-08T06:53:46.855058+0000","last_clean":"2023-10-08T06:53:46.855058+0000","last_became_active":"2023-10-08T06:53:46.854195+0000","last_became_peered":"2023-10-08T06:53:46.854195+0000","last_unstale":"2023-10-08T06:53:46.855058+0000","last_undegraded":"2023-10-08T06:53:46.855058+0000","last_fullsized":"2023-10-08T06:53:46.855058+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:32.390630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832812+0000","last_change":"2023-10-08T06:53:46.832812+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832812+0000","last_undegraded":"2023-10-08T06:53:46.832812+0000","last_fullsized":"2023-10-08T06:53:46.832812+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.854602+0000","last_change":"2023-10-08T06:53:46.853301+0000","last_active":"2023-10-08T06:53:46.854602+0000","last_peered":"2023-10-08T06:53:46.854602+0000","last_clean":"2023-10-08T06:53:46.854602+0000","last_became_active":"2023-10-08T06:53:46.853014+0000","last_became_peered":"2023-10-08T06:53:46.853014+0000","last_unstale":"2023-10-08T06:53:46.854602+0000","last_undegraded":"2023-10-08T06:53:46.854602+0000","last_fullsized":"2023-10-08T06:53:46.854602+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:52.386749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.844177+0000","last_change":"2023-10-08T06:53:46.844177+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.844177+0000","last_undegraded":"2023-10-08T06:53:46.844177+0000","last_fullsized":"2023-10-08T06:53:46.844177+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855614+0000","last_change":"2023-10-08T06:53:46.854219+0000","last_active":"2023-10-08T06:53:46.855614+0000","last_peered":"2023-10-08T06:53:46.855614+0000","last_clean":"2023-10-08T06:53:46.855614+0000","last_became_active":"2023-10-08T06:53:46.853948+0000","last_became_peered":"2023-10-08T06:53:46.853948+0000","last_unstale":"2023-10-08T06:53:46.855614+0000","last_undegraded":"2023-10-08T06:53:46.855614+0000","last_fullsized":"2023-10-08T06:53:46.855614+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:36.674287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.826941+0000","last_change":"2023-10-08T06:53:46.826941+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.826941+0000","last_undegraded":"2023-10-08T06:53:46.826941+0000","last_fullsized":"2023-10-08T06:53:46.826941+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853355+0000","last_change":"2023-10-08T06:53:46.847384+0000","last_active":"2023-10-08T06:53:46.853355+0000","last_peered":"2023-10-08T06:53:46.853355+0000","last_clean":"2023-10-08T06:53:46.853355+0000","last_became_active":"2023-10-08T06:53:46.847114+0000","last_became_peered":"2023-10-08T06:53:46.847114+0000","last_unstale":"2023-10-08T06:53:46.853355+0000","last_undegraded":"2023-10-08T06:53:46.853355+0000","last_fullsized":"2023-10-08T06:53:46.853355+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:08.420118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831287+0000","last_change":"2023-10-08T06:53:46.831287+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831287+0000","last_undegraded":"2023-10-08T06:53:46.831287+0000","last_fullsized":"2023-10-08T06:53:46.831287+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.845097+0000","last_change":"2023-10-08T06:53:46.842021+0000","last_active":"2023-10-08T06:53:46.845097+0000","last_peered":"2023-10-08T06:53:46.845097+0000","last_clean":"2023-10-08T06:53:46.845097+0000","last_became_active":"2023-10-08T06:53:46.841767+0000","last_became_peered":"2023-10-08T06:53:46.841767+0000","last_unstale":"2023-10-08T06:53:46.845097+0000","last_undegraded":"2023-10-08T06:53:46.845097+0000","last_fullsized":"2023-10-08T06:53:46.845097+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:06.509561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835744+0000","last_change":"2023-10-08T06:53:46.835744+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835744+0000","last_undegraded":"2023-10-08T06:53:46.835744+0000","last_fullsized":"2023-10-08T06:53:46.835744+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.848938+0000","last_change":"2023-10-08T06:53:46.845053+0000","last_active":"2023-10-08T06:53:46.848938+0000","last_peered":"2023-10-08T06:53:46.848938+0000","last_clean":"2023-10-08T06:53:46.848938+0000","last_became_active":"2023-10-08T06:53:46.844706+0000","last_became_peered":"2023-10-08T06:53:46.844706+0000","last_unstale":"2023-10-08T06:53:46.848938+0000","last_undegraded":"2023-10-08T06:53:46.848938+0000","last_fullsized":"2023-10-08T06:53:46.848938+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:49.451557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.834908+0000","last_change":"2023-10-08T06:53:46.834908+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.834908+0000","last_undegraded":"2023-10-08T06:53:46.834908+0000","last_fullsized":"2023-10-08T06:53:46.834908+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.845369+0000","last_change":"2023-10-08T06:53:46.843744+0000","last_active":"2023-10-08T06:53:46.845369+0000","last_peered":"2023-10-08T06:53:46.845369+0000","last_clean":"2023-10-08T06:53:46.845369+0000","last_became_active":"2023-10-08T06:53:46.843504+0000","last_became_peered":"2023-10-08T06:53:46.843504+0000","last_unstale":"2023-10-08T06:53:46.845369+0000","last_undegraded":"2023-10-08T06:53:46.845369+0000","last_fullsized":"2023-10-08T06:53:46.845369+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:26.339899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828590+0000","last_change":"2023-10-08T06:53:46.828590+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828590+0000","last_undegraded":"2023-10-08T06:53:46.828590+0000","last_fullsized":"2023-10-08T06:53:46.828590+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.851054+0000","last_change":"2023-10-08T06:53:46.848518+0000","last_active":"2023-10-08T06:53:46.851054+0000","last_peered":"2023-10-08T06:53:46.851054+0000","last_clean":"2023-10-08T06:53:46.851054+0000","last_became_active":"2023-10-08T06:53:46.848237+0000","last_became_peered":"2023-10-08T06:53:46.848237+0000","last_unstale":"2023-10-08T06:53:46.851054+0000","last_undegraded":"2023-10-08T06:53:46.851054+0000","last_fullsized":"2023-10-08T06:53:46.851054+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:27:53.231805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825152+0000","last_change":"2023-10-08T06:53:46.825152+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825152+0000","last_undegraded":"2023-10-08T06:53:46.825152+0000","last_fullsized":"2023-10-08T06:53:46.825152+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.851503+0000","last_change":"2023-10-08T06:53:46.844822+0000","last_active":"2023-10-08T06:53:46.851503+0000","last_peered":"2023-10-08T06:53:46.851503+0000","last_clean":"2023-10-08T06:53:46.851503+0000","last_became_active":"2023-10-08T06:53:46.844540+0000","last_became_peered":"2023-10-08T06:53:46.844540+0000","last_unstale":"2023-10-08T06:53:46.851503+0000","last_undegraded":"2023-10-08T06:53:46.851503+0000","last_fullsized":"2023-10-08T06:53:46.851503+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:03.783647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825174+0000","last_change":"2023-10-08T06:53:46.825174+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825174+0000","last_undegraded":"2023-10-08T06:53:46.825174+0000","last_fullsized":"2023-10-08T06:53:46.825174+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.833708+0000","last_change":"2023-10-08T06:53:46.832905+0000","last_active":"2023-10-08T06:53:46.833708+0000","last_peered":"2023-10-08T06:53:46.833708+0000","last_clean":"2023-10-08T06:53:46.833708+0000","last_became_active":"2023-10-08T06:53:46.832624+0000","last_became_peered":"2023-10-08T06:53:46.832624+0000","last_unstale":"2023-10-08T06:53:46.833708+0000","last_undegraded":"2023-10-08T06:53:46.833708+0000","last_fullsized":"2023-10-08T06:53:46.833708+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:11.164730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.830035+0000","last_change":"2023-10-08T06:53:46.830035+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.830035+0000","last_undegraded":"2023-10-08T06:53:46.830035+0000","last_fullsized":"2023-10-08T06:53:46.830035+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.841506+0000","last_change":"2023-10-08T06:53:46.838423+0000","last_active":"2023-10-08T06:53:46.841506+0000","last_peered":"2023-10-08T06:53:46.841506+0000","last_clean":"2023-10-08T06:53:46.841506+0000","last_became_active":"2023-10-08T06:53:46.838071+0000","last_became_peered":"2023-10-08T06:53:46.838071+0000","last_unstale":"2023-10-08T06:53:46.841506+0000","last_undegraded":"2023-10-08T06:53:46.841506+0000","last_fullsized":"2023-10-08T06:53:46.841506+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:30.734197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833041+0000","last_change":"2023-10-08T06:53:46.833041+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833041+0000","last_undegraded":"2023-10-08T06:53:46.833041+0000","last_fullsized":"2023-10-08T06:53:46.833041+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.858068+0000","last_change":"2023-10-08T06:53:46.857001+0000","last_active":"2023-10-08T06:53:46.858068+0000","last_peered":"2023-10-08T06:53:46.858068+0000","last_clean":"2023-10-08T06:53:46.858068+0000","last_became_active":"2023-10-08T06:53:46.856707+0000","last_became_peered":"2023-10-08T06:53:46.856707+0000","last_unstale":"2023-10-08T06:53:46.858068+0000","last_undegraded":"2023-10-08T06:53:46.858068+0000","last_fullsized":"2023-10-08T06:53:46.858068+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:10.191719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.829716+0000","last_change":"2023-10-08T06:53:46.829716+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.829716+0000","last_undegraded":"2023-10-08T06:53:46.829716+0000","last_fullsized":"2023-10-08T06:53:46.829716+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.850126+0000","last_change":"2023-10-08T06:53:46.842178+0000","last_active":"2023-10-08T06:53:46.850126+0000","last_peered":"2023-10-08T06:53:46.850126+0000","last_clean":"2023-10-08T06:53:46.850126+0000","last_became_active":"2023-10-08T06:53:46.840731+0000","last_became_peered":"2023-10-08T06:53:46.840731+0000","last_unstale":"2023-10-08T06:53:46.850126+0000","last_undegraded":"2023-10-08T06:53:46.850126+0000","last_fullsized":"2023-10-08T06:53:46.850126+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:28.102115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835847+0000","last_change":"2023-10-08T06:53:46.835847+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835847+0000","last_undegraded":"2023-10-08T06:53:46.835847+0000","last_fullsized":"2023-10-08T06:53:46.835847+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.854676+0000","last_change":"2023-10-08T06:53:46.853950+0000","last_active":"2023-10-08T06:53:46.854676+0000","last_peered":"2023-10-08T06:53:46.854676+0000","last_clean":"2023-10-08T06:53:46.854676+0000","last_became_active":"2023-10-08T06:53:46.853668+0000","last_became_peered":"2023-10-08T06:53:46.853668+0000","last_unstale":"2023-10-08T06:53:46.854676+0000","last_undegraded":"2023-10-08T06:53:46.854676+0000","last_fullsized":"2023-10-08T06:53:46.854676+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:37.692117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.840640+0000","last_change":"2023-10-08T06:53:46.840640+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.840640+0000","last_undegraded":"2023-10-08T06:53:46.840640+0000","last_fullsized":"2023-10-08T06:53:46.840640+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.850594+0000","last_change":"2023-10-08T06:53:46.849784+0000","last_active":"2023-10-08T06:53:46.850594+0000","last_peered":"2023-10-08T06:53:46.850594+0000","last_clean":"2023-10-08T06:53:46.850594+0000","last_became_active":"2023-10-08T06:53:46.849527+0000","last_became_peered":"2023-10-08T06:53:46.849527+0000","last_unstale":"2023-10-08T06:53:46.850594+0000","last_undegraded":"2023-10-08T06:53:46.850594+0000","last_fullsized":"2023-10-08T06:53:46.850594+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:09.620134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.841115+0000","last_change":"2023-10-08T06:53:46.841115+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.841115+0000","last_undegraded":"2023-10-08T06:53:46.841115+0000","last_fullsized":"2023-10-08T06:53:46.841115+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853280+0000","last_change":"2023-10-08T06:53:46.849864+0000","last_active":"2023-10-08T06:53:46.853280+0000","last_peered":"2023-10-08T06:53:46.853280+0000","last_clean":"2023-10-08T06:53:46.853280+0000","last_became_active":"2023-10-08T06:53:46.849576+0000","last_became_peered":"2023-10-08T06:53:46.849576+0000","last_unstale":"2023-10-08T06:53:46.853280+0000","last_undegraded":"2023-10-08T06:53:46.853280+0000","last_fullsized":"2023-10-08T06:53:46.853280+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:36.369411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837276+0000","last_change":"2023-10-08T06:53:46.837276+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837276+0000","last_undegraded":"2023-10-08T06:53:46.837276+0000","last_fullsized":"2023-10-08T06:53:46.837276+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.847545+0000","last_change":"2023-10-08T06:53:46.846147+0000","last_active":"2023-10-08T06:53:46.847545+0000","last_peered":"2023-10-08T06:53:46.847545+0000","last_clean":"2023-10-08T06:53:46.847545+0000","last_became_active":"2023-10-08T06:53:46.845910+0000","last_became_peered":"2023-10-08T06:53:46.845910+0000","last_unstale":"2023-10-08T06:53:46.847545+0000","last_undegraded":"2023-10-08T06:53:46.847545+0000","last_fullsized":"2023-10-08T06:53:46.847545+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:11.739527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835581+0000","last_change":"2023-10-08T06:53:46.835581+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835581+0000","last_undegraded":"2023-10-08T06:53:46.835581+0000","last_fullsized":"2023-10-08T06:53:46.835581+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.858105+0000","last_change":"2023-10-08T06:53:46.857013+0000","last_active":"2023-10-08T06:53:46.858105+0000","last_peered":"2023-10-08T06:53:46.858105+0000","last_clean":"2023-10-08T06:53:46.858105+0000","last_became_active":"2023-10-08T06:53:46.856762+0000","last_became_peered":"2023-10-08T06:53:46.856762+0000","last_unstale":"2023-10-08T06:53:46.858105+0000","last_undegraded":"2023-10-08T06:53:46.858105+0000","last_fullsized":"2023-10-08T06:53:46.858105+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:12.313863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.837818+0000","last_change":"2023-10-08T06:53:46.837818+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.837818+0000","last_undegraded":"2023-10-08T06:53:46.837818+0000","last_fullsized":"2023-10-08T06:53:46.837818+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855358+0000","last_change":"2023-10-08T06:53:46.854554+0000","last_active":"2023-10-08T06:53:46.855358+0000","last_peered":"2023-10-08T06:53:46.855358+0000","last_clean":"2023-10-08T06:53:46.855358+0000","last_became_active":"2023-10-08T06:53:46.854235+0000","last_became_peered":"2023-10-08T06:53:46.854235+0000","last_unstale":"2023-10-08T06:53:46.855358+0000","last_undegraded":"2023-10-08T06:53:46.855358+0000","last_fullsized":"2023-10-08T06:53:46.855358+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:54:13.484888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832393+0000","last_change":"2023-10-08T06:53:46.832393+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832393+0000","last_undegraded":"2023-10-08T06:53:46.832393+0000","last_fullsized":"2023-10-08T06:53:46.832393+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.852782+0000","last_change":"2023-10-08T06:53:46.848822+0000","last_active":"2023-10-08T06:53:46.852782+0000","last_peered":"2023-10-08T06:53:46.852782+0000","last_clean":"2023-10-08T06:53:46.852782+0000","last_became_active":"2023-10-08T06:53:46.848489+0000","last_became_peered":"2023-10-08T06:53:46.848489+0000","last_unstale":"2023-10-08T06:53:46.852782+0000","last_undegraded":"2023-10-08T06:53:46.852782+0000","last_fullsized":"2023-10-08T06:53:46.852782+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:05:54.769070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.840127+0000","last_change":"2023-10-08T06:53:46.840127+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.840127+0000","last_undegraded":"2023-10-08T06:53:46.840127+0000","last_fullsized":"2023-10-08T06:53:46.840127+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855229+0000","last_change":"2023-10-08T06:53:46.853408+0000","last_active":"2023-10-08T06:53:46.855229+0000","last_peered":"2023-10-08T06:53:46.855229+0000","last_clean":"2023-10-08T06:53:46.855229+0000","last_became_active":"2023-10-08T06:53:46.853121+0000","last_became_peered":"2023-10-08T06:53:46.853121+0000","last_unstale":"2023-10-08T06:53:46.855229+0000","last_undegraded":"2023-10-08T06:53:46.855229+0000","last_fullsized":"2023-10-08T06:53:46.855229+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:52:01.990568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.835282+0000","last_change":"2023-10-08T06:53:46.835282+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.835282+0000","last_undegraded":"2023-10-08T06:53:46.835282+0000","last_fullsized":"2023-10-08T06:53:46.835282+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.852616+0000","last_change":"2023-10-08T06:53:46.841902+0000","last_active":"2023-10-08T06:53:46.852616+0000","last_peered":"2023-10-08T06:53:46.852616+0000","last_clean":"2023-10-08T06:53:46.852616+0000","last_became_active":"2023-10-08T06:53:46.841528+0000","last_became_peered":"2023-10-08T06:53:46.841528+0000","last_unstale":"2023-10-08T06:53:46.852616+0000","last_undegraded":"2023-10-08T06:53:46.852616+0000","last_fullsized":"2023-10-08T06:53:46.852616+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:16.903470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833006+0000","last_change":"2023-10-08T06:53:46.833006+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833006+0000","last_undegraded":"2023-10-08T06:53:46.833006+0000","last_fullsized":"2023-10-08T06:53:46.833006+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.857684+0000","last_change":"2023-10-08T06:53:46.856348+0000","last_active":"2023-10-08T06:53:46.857684+0000","last_peered":"2023-10-08T06:53:46.857684+0000","last_clean":"2023-10-08T06:53:46.857684+0000","last_became_active":"2023-10-08T06:53:46.856059+0000","last_became_peered":"2023-10-08T06:53:46.856059+0000","last_unstale":"2023-10-08T06:53:46.857684+0000","last_undegraded":"2023-10-08T06:53:46.857684+0000","last_fullsized":"2023-10-08T06:53:46.857684+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:28.598908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.833355+0000","last_change":"2023-10-08T06:53:46.833355+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.833355+0000","last_undegraded":"2023-10-08T06:53:46.833355+0000","last_fullsized":"2023-10-08T06:53:46.833355+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853288+0000","last_change":"2023-10-08T06:53:46.851826+0000","last_active":"2023-10-08T06:53:46.853288+0000","last_peered":"2023-10-08T06:53:46.853288+0000","last_clean":"2023-10-08T06:53:46.853288+0000","last_became_active":"2023-10-08T06:53:46.851307+0000","last_became_peered":"2023-10-08T06:53:46.851307+0000","last_unstale":"2023-10-08T06:53:46.853288+0000","last_undegraded":"2023-10-08T06:53:46.853288+0000","last_fullsized":"2023-10-08T06:53:46.853288+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:28.030370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.836735+0000","last_change":"2023-10-08T06:53:46.836735+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.836735+0000","last_undegraded":"2023-10-08T06:53:46.836735+0000","last_fullsized":"2023-10-08T06:53:46.836735+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855012+0000","last_change":"2023-10-08T06:53:46.854168+0000","last_active":"2023-10-08T06:53:46.855012+0000","last_peered":"2023-10-08T06:53:46.855012+0000","last_clean":"2023-10-08T06:53:46.855012+0000","last_became_active":"2023-10-08T06:53:46.853876+0000","last_became_peered":"2023-10-08T06:53:46.853876+0000","last_unstale":"2023-10-08T06:53:46.855012+0000","last_undegraded":"2023-10-08T06:53:46.855012+0000","last_fullsized":"2023-10-08T06:53:46.855012+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:46.027669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.831727+0000","last_change":"2023-10-08T06:53:46.831727+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.831727+0000","last_undegraded":"2023-10-08T06:53:46.831727+0000","last_fullsized":"2023-10-08T06:53:46.831727+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855809+0000","last_change":"2023-10-08T06:53:46.854126+0000","last_active":"2023-10-08T06:53:46.855809+0000","last_peered":"2023-10-08T06:53:46.855809+0000","last_clean":"2023-10-08T06:53:46.855809+0000","last_became_active":"2023-10-08T06:53:46.853833+0000","last_became_peered":"2023-10-08T06:53:46.853833+0000","last_unstale":"2023-10-08T06:53:46.855809+0000","last_undegraded":"2023-10-08T06:53:46.855809+0000","last_fullsized":"2023-10-08T06:53:46.855809+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:37.677941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.825803+0000","last_change":"2023-10-08T06:53:46.825803+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.825803+0000","last_undegraded":"2023-10-08T06:53:46.825803+0000","last_fullsized":"2023-10-08T06:53:46.825803+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853319+0000","last_change":"2023-10-08T06:53:46.851411+0000","last_active":"2023-10-08T06:53:46.853319+0000","last_peered":"2023-10-08T06:53:46.853319+0000","last_clean":"2023-10-08T06:53:46.853319+0000","last_became_active":"2023-10-08T06:53:46.850930+0000","last_became_peered":"2023-10-08T06:53:46.850930+0000","last_unstale":"2023-10-08T06:53:46.853319+0000","last_undegraded":"2023-10-08T06:53:46.853319+0000","last_fullsized":"2023-10-08T06:53:46.853319+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:27.974110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.828614+0000","last_change":"2023-10-08T06:53:46.828614+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.828614+0000","last_undegraded":"2023-10-08T06:53:46.828614+0000","last_fullsized":"2023-10-08T06:53:46.828614+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.853067+0000","last_change":"2023-10-08T06:53:46.852038+0000","last_active":"2023-10-08T06:53:46.853067+0000","last_peered":"2023-10-08T06:53:46.853067+0000","last_clean":"2023-10-08T06:53:46.853067+0000","last_became_active":"2023-10-08T06:53:46.851840+0000","last_became_peered":"2023-10-08T06:53:46.851840+0000","last_unstale":"2023-10-08T06:53:46.853067+0000","last_undegraded":"2023-10-08T06:53:46.853067+0000","last_fullsized":"2023-10-08T06:53:46.853067+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:14.079295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.834744+0000","last_change":"2023-10-08T06:53:46.834744+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.834744+0000","last_undegraded":"2023-10-08T06:53:46.834744+0000","last_fullsized":"2023-10-08T06:53:46.834744+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.848916+0000","last_change":"2023-10-08T06:53:46.844152+0000","last_active":"2023-10-08T06:53:46.848916+0000","last_peered":"2023-10-08T06:53:46.848916+0000","last_clean":"2023-10-08T06:53:46.848916+0000","last_became_active":"2023-10-08T06:53:46.843835+0000","last_became_peered":"2023-10-08T06:53:46.843835+0000","last_unstale":"2023-10-08T06:53:46.848916+0000","last_undegraded":"2023-10-08T06:53:46.848916+0000","last_fullsized":"2023-10-08T06:53:46.848916+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:04:41.456787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.832811+0000","last_change":"2023-10-08T06:53:46.832811+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.832811+0000","last_undegraded":"2023-10-08T06:53:46.832811+0000","last_fullsized":"2023-10-08T06:53:46.832811+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.850421+0000","last_change":"2023-10-08T06:53:46.846292+0000","last_active":"2023-10-08T06:53:46.850421+0000","last_peered":"2023-10-08T06:53:46.850421+0000","last_clean":"2023-10-08T06:53:46.850421+0000","last_became_active":"2023-10-08T06:53:46.845869+0000","last_became_peered":"2023-10-08T06:53:46.845869+0000","last_unstale":"2023-10-08T06:53:46.850421+0000","last_undegraded":"2023-10-08T06:53:46.850421+0000","last_fullsized":"2023-10-08T06:53:46.850421+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:21:37.089110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":1,"reported_epoch":183,"state":"creating+peering","last_fresh":"2023-10-08T06:53:46.827498+0000","last_change":"2023-10-08T06:53:46.827498+0000","last_active":"2023-10-08T06:53:46.808694+0000","last_peered":"2023-10-08T06:53:46.808694+0000","last_clean":"2023-10-08T06:53:46.808694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T06:53:46.827498+0000","last_undegraded":"2023-10-08T06:53:46.827498+0000","last_fullsized":"2023-10-08T06:53:46.827498+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T06:53:46.855333+0000","last_change":"2023-10-08T06:53:46.853823+0000","last_active":"2023-10-08T06:53:46.855333+0000","last_peered":"2023-10-08T06:53:46.855333+0000","last_clean":"2023-10-08T06:53:46.855333+0000","last_became_active":"2023-10-08T06:53:46.853347+0000","last_became_peered":"2023-10-08T06:53:46.853347+0000","last_unstale":"2023-10-08T06:53:46.855333+0000","last_undegraded":"2023-10-08T06:53:46.855333+0000","last_fullsized":"2023-10-08T06:53:46.855333+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:02.956148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"172'2729","reported_seq":3394,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-10-08T06:53:45.826415+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T06:53:45.826415+0000","last_peered":"2023-10-08T06:53:45.826415+0000","last_clean":"2023-10-08T06:53:45.826415+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T06:53:45.826415+0000","last_undegraded":"2023-10-08T06:53:45.826415+0000","last_fullsized":"2023-10-08T06:53:45.826415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2729,"log_dups_size":0,"ondisk_log_size":2729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575775,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60348,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311492,"statfs":{"total":96636764160,"available":96574967808,"internally_reserved":0,"allocated":2867200,"data_stored":1759119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.21,"15min":1.2330000000000001},"min":{"1min":0.91900000000000004,"5min":0.78700000000000003,"15min":0.70399999999999996},"max":{"1min":1.6950000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.224,"15min":1.248},"min":{"1min":0.93799999999999994,"5min":0.747,"15min":0.72899999999999998},"max":{"1min":1.522,"5min":1.7150000000000001,"15min":1.849},"last":1.5009999999999999}]},{"osd":1,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2829999999999999,"15min":1.262},"min":{"1min":0.89200000000000002,"5min":0.84299999999999997,"15min":0.70399999999999996},"max":{"1min":1.6970000000000001,"5min":1.712,"15min":1.8480000000000001},"last":1.157},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.284,"15min":1.3220000000000001},"min":{"1min":0.95999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7450000000000001,"5min":1.9299999999999999,"15min":2.4809999999999999},"last":1.591}]},{"osd":2,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.0740000000000001,"5min":0.90900000000000003,"15min":0.81899999999999995},"max":{"1min":1.627,"5min":2.7040000000000002,"15min":3.4350000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.387,"5min":1.381,"15min":1.3740000000000001},"min":{"1min":0.95699999999999996,"5min":0.88900000000000001,"15min":0.83199999999999996},"max":{"1min":1.873,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":0.99099999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.339,"15min":1.335},"min":{"1min":1.0189999999999999,"5min":0.97799999999999998,"15min":0.85899999999999999},"max":{"1min":1.5609999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.212},{"interface":"front","average":{"1min":1.5,"5min":1.46,"15min":1.419},"min":{"1min":1.147,"5min":0.98899999999999999,"15min":0.83099999999999996},"max":{"1min":2.0379999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.3620000000000001}]},{"osd":4,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.333,"15min":1.2869999999999999},"min":{"1min":0.93999999999999995,"5min":0.83199999999999996,"15min":0.66200000000000003},"max":{"1min":1.952,"5min":1.952,"15min":3.2839999999999998},"last":1.792},{"interface":"front","average":{"1min":1.484,"5min":1.456,"15min":1.391},"min":{"1min":1.1579999999999999,"5min":0.90300000000000002,"15min":0.80800000000000005},"max":{"1min":1.708,"5min":2.004,"15min":2.1909999999999998},"last":1.4510000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.474,"15min":1.4399999999999999},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":0.86499999999999999},"max":{"1min":2.1499999999999999,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":1.272},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.504,"15min":1.4790000000000001},"min":{"1min":1.2909999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.6439999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.5680000000000001,"15min":1.556},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":0.94599999999999995},"max":{"1min":2.1970000000000001,"5min":2.4350000000000001,"15min":3.1360000000000001},"last":1.728},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5269999999999999,"15min":1.502},"min":{"1min":1.0049999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":2.0190000000000001,"5min":2.1309999999999998,"15min":2.3140000000000001},"last":1.6839999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41720,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330120,"statfs":{"total":96636764160,"available":96594042880,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3009999999999999,"15min":1.3480000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7310000000000001,"5min":1.792,"15min":2.508},"last":1.605},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3089999999999999,"15min":1.337},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.609,"5min":1.8280000000000001,"15min":2.1139999999999999},"last":1.3520000000000001}]},{"osd":1,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2170000000000001,"15min":1.248},"min":{"1min":0.876,"5min":0.876,"15min":0.59699999999999998},"max":{"1min":1.605,"5min":1.821,"15min":1.919},"last":1.413},{"interface":"front","average":{"1min":1.224,"5min":1.2769999999999999,"15min":1.2629999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.53800000000000003},"max":{"1min":1.7769999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.3720000000000001}]},{"osd":2,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2170000000000001,"15min":1.2290000000000001},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.75600000000000001},"max":{"1min":1.6499999999999999,"5min":1.722,"15min":1.8919999999999999},"last":1.514},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.29,"15min":1.2989999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5229999999999999,"5min":1.6379999999999999,"15min":2.3839999999999999},"last":1.1739999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4259999999999999,"15min":1.4039999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.83399999999999996},"max":{"1min":1.8700000000000001,"5min":1.976,"15min":2.1499999999999999},"last":1.478},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.478,"15min":1.48},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.89400000000000002},"max":{"1min":1.8180000000000001,"5min":1.915,"15min":2.0289999999999999},"last":1.274}]},{"osd":4,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.3580000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.72399999999999998},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.226},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4379999999999999,"15min":1.4390000000000001},"min":{"1min":0.878,"5min":0.878,"15min":0.80200000000000005},"max":{"1min":1.9099999999999999,"5min":1.9910000000000001,"15min":2.0089999999999999},"last":1.006}]},{"osd":5,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.431,"15min":1.4139999999999999},"min":{"1min":0.997,"5min":0.94799999999999995,"15min":0.69499999999999995},"max":{"1min":1.798,"5min":1.893,"15min":2.7309999999999999},"last":1.319},{"interface":"front","average":{"1min":1.575,"5min":1.5589999999999999,"15min":1.548},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78300000000000003},"max":{"1min":2.0640000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.121}]},{"osd":7,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.591,"15min":1.579},"min":{"1min":1.0960000000000001,"5min":1.0680000000000001,"15min":0.92000000000000004},"max":{"1min":1.984,"5min":2.0710000000000002,"15min":2.2090000000000001},"last":1.454},{"interface":"front","average":{"1min":1.488,"5min":1.573,"15min":1.5660000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.82099999999999995},"max":{"1min":1.946,"5min":2.222,"15min":2.6469999999999998},"last":1.5549999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59768,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312072,"statfs":{"total":96636764160,"available":96575561728,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2310000000000001,"15min":1.218},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.66600000000000004},"max":{"1min":1.5740000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.296,"15min":1.2569999999999999},"min":{"1min":0.95399999999999996,"5min":0.82699999999999996,"15min":0.78800000000000003},"max":{"1min":1.629,"5min":1.897,"15min":2.4700000000000002},"last":1.206}]},{"osd":1,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.26,"15min":1.2230000000000001},"min":{"1min":0.95499999999999996,"5min":0.85799999999999998,"15min":0.72499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.349,"5min":1.375,"15min":1.323},"min":{"1min":0.90400000000000003,"5min":0.88700000000000001,"15min":0.78200000000000003},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.25}]},{"osd":2,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3480000000000001,"15min":1.3280000000000001},"min":{"1min":0.98399999999999999,"5min":0.85199999999999998,"15min":0.83199999999999996},"max":{"1min":1.6479999999999999,"5min":1.9359999999999999,"15min":2.351},"last":1.3},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.3160000000000001},"min":{"1min":0.91000000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.667,"5min":1.7809999999999999,"15min":1.964},"last":0.94499999999999995}]},{"osd":3,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3959999999999999,"15min":1.3540000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.528,"15min":1.464},"min":{"1min":1.139,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.9910000000000001,"5min":3.0059999999999998,"15min":5.1630000000000003},"last":1.042}]},{"osd":4,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3320000000000001,"15min":1.3240000000000001},"min":{"1min":0.96299999999999997,"5min":0.89300000000000002,"15min":0.81499999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4039999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4379999999999999,"15min":1.4239999999999999},"min":{"1min":0.96899999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.113}]},{"osd":6,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.4990000000000001,"15min":1.4750000000000001},"min":{"1min":1.248,"5min":0.96399999999999997,"15min":0.81200000000000006},"max":{"1min":2.1150000000000002,"5min":2.1659999999999999,"15min":5.5019999999999998},"last":1.155},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4930000000000001,"15min":1.48},"min":{"1min":1.0509999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.0139999999999998,"5min":2.1040000000000001,"15min":2.2890000000000001},"last":1.446}]},{"osd":7,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.5660000000000001,"15min":1.5629999999999999},"min":{"1min":1.3400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":5.8780000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.679,"5min":1.6839999999999999,"15min":1.6299999999999999},"min":{"1min":1.415,"5min":1.1719999999999999,"15min":0.97099999999999997},"max":{"1min":2.206,"5min":2.6789999999999998,"15min":5.399},"last":1.381}]}]},{"osd":4,"up_from":13,"seq":55834575774,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41720,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330120,"statfs":{"total":96636764160,"available":96594042880,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.389,"15min":1.3460000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.3999999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.69799999999999995},"max":{"1min":1.7789999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.518}]},{"osd":1,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2749999999999999,"15min":1.28},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.73,"5min":1.784,"15min":1.9339999999999999},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.64000000000000001},"max":{"1min":1.613,"5min":1.6870000000000001,"15min":1.7290000000000001},"last":1.232}]},{"osd":2,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.2889999999999999,"15min":1.248},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":1.046},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.3759999999999999},"min":{"1min":1,"5min":0.93100000000000005,"15min":0.85399999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.651}]},{"osd":3,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.339,"15min":1.3109999999999999},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.4670000000000001,"15min":1.4099999999999999},"min":{"1min":1.0629999999999999,"5min":0.97299999999999998,"15min":0.93400000000000005},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.5429999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.4139999999999999,"15min":1.351},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.66700000000000004},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.4450000000000001,"15min":1.375},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.749},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.6060000000000001}]},{"osd":6,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.5329999999999999,"15min":1.425},"min":{"1min":0.97899999999999998,"5min":0.91100000000000003,"15min":0.72699999999999998},"max":{"1min":5.3339999999999996,"5min":5.3339999999999996,"15min":5.3339999999999996},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6120000000000001,"15min":1.546},"min":{"1min":1.212,"5min":0.96699999999999997,"15min":0.91400000000000003},"max":{"1min":2.1059999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":1.7250000000000001}]},{"osd":7,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.548,"15min":1.49},"min":{"1min":1.202,"5min":1.135,"15min":0.99099999999999999},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.784,"5min":1.6859999999999999,"15min":1.6060000000000001},"min":{"1min":1.0049999999999999,"5min":0.98999999999999999,"15min":0.93600000000000005},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":1.6879999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575776,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47288,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324552,"statfs":{"total":96636764160,"available":96588341248,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.95499999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.44500000000000001},"max":{"1min":1.3200000000000001,"5min":1.71,"15min":1.71},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.196,"15min":1.175},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6719999999999999,"5min":1.851,"15min":2.137},"last":0.93500000000000005}]},{"osd":1,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.024,"15min":1.018},"min":{"1min":0.70999999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2370000000000001,"5min":1.71,"15min":1.823},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.321,"5min":1.2989999999999999,"15min":1.3049999999999999},"min":{"1min":0.91000000000000003,"5min":0.872,"15min":0.77000000000000002},"max":{"1min":1.51,"5min":1.75,"15min":1.893},"last":0.73599999999999999}]},{"osd":2,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1739999999999999,"15min":1.137},"min":{"1min":0.85299999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6279999999999999,"5min":1.726,"15min":1.7410000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1919999999999999,"15min":1.1930000000000001},"min":{"1min":0.90400000000000003,"5min":0.68600000000000005,"15min":0.61399999999999999},"max":{"1min":1.589,"5min":1.7809999999999999,"15min":1.8360000000000001},"last":1.171}]},{"osd":4,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.284,"15min":1.2649999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83299999999999996},"max":{"1min":1.8560000000000001,"5min":1.97,"15min":1.97},"last":1.349},{"interface":"front","average":{"1min":1.482,"5min":1.4650000000000001,"15min":1.452},"min":{"1min":1.1859999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.8100000000000001,"5min":1.905,"15min":2.1960000000000002},"last":1.3080000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.391,"15min":1.385},"min":{"1min":1.085,"5min":0.98399999999999999,"15min":0.88},"max":{"1min":1.6399999999999999,"5min":2.27,"15min":3.0009999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.415,"15min":1.421},"min":{"1min":1.1890000000000001,"5min":0.995,"15min":0.94099999999999995},"max":{"1min":1.6739999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.2809999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4059999999999999,"15min":1.3959999999999999},"min":{"1min":0.98399999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.016},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.534,"15min":1.518},"min":{"1min":1.2170000000000001,"5min":1.1639999999999999,"15min":0.99099999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.544},"last":1.371}]},{"osd":7,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.5649999999999999,"15min":1.5660000000000001},"min":{"1min":1.3069999999999999,"5min":1.1839999999999999,"15min":1.008},"max":{"1min":1.8939999999999999,"5min":2.008,"15min":2.347},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.5049999999999999,"15min":1.4790000000000001},"min":{"1min":1.0980000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1800000000000002},"last":1.2070000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575774,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47288,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324552,"statfs":{"total":96636764160,"available":96588341248,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.93700000000000006},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.4650000000000001,"15min":1.657},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.036,"5min":1.0549999999999999,"15min":1.0760000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5249999999999999,"5min":1.661,"15min":2.2679999999999998},"last":1.26}]},{"osd":1,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.004,"15min":1.0069999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.696,"5min":1.7170000000000001,"15min":1.996},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.137,"5min":1.1679999999999999,"15min":1.1819999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":2.1139999999999999},"last":0.92800000000000005}]},{"osd":3,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2270000000000001,"15min":1.2230000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.772,"5min":1.925,"15min":2.3929999999999998},"last":1.075},{"interface":"front","average":{"1min":1.177,"5min":1.256,"15min":1.258},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.46,"5min":1.804,"15min":1.891},"last":1.0389999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4330000000000001,"15min":1.444},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.859,"5min":1.9430000000000001,"15min":3.0939999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.3160000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7470000000000001,"5min":1.7549999999999999,"15min":3.7320000000000002},"last":1.4379999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.429,"15min":1.425},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":3.3519999999999999},"last":1.48},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.5029999999999999,"15min":1.528},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.9810000000000001,"5min":2.2719999999999998,"15min":2.5249999999999999},"last":1.321}]},{"osd":6,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.673,"5min":1.9990000000000001,"15min":2.206},"last":1.532},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.548,"15min":1.532},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9610000000000001,"5min":1.966,"15min":3.02},"last":1.6319999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.51,"15min":1.492},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.861,"5min":2.0790000000000002,"15min":4.2869999999999999},"last":1.591},{"interface":"front","average":{"1min":1.55,"5min":1.5940000000000001,"15min":1.579},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8540000000000001,"5min":2.0099999999999998,"15min":3.0760000000000001},"last":1.375}]}]},{"osd":1,"up_from":13,"seq":55834575774,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29240,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342600,"statfs":{"total":96636764160,"available":96606822400,"internally_reserved":0,"allocated":2273280,"data_stored":1168751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.0349999999999999,"15min":1.0029999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.601,"5min":4.601,"15min":4.601},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2190000000000001,"15min":1.22},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.72599999999999998}]},{"osd":2,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.0329999999999999,"15min":1.0229999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.3179999999999996,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.385,"5min":1.2250000000000001,"15min":1.1679999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":5.4059999999999997,"5min":5.4059999999999997,"15min":5.4059999999999997},"last":1.163}]},{"osd":3,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.3029999999999999,"15min":1.2609999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.5220000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1579999999999999,"15min":1.1160000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.0069999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.3919999999999999,"15min":1.3680000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":4.2409999999999997,"5min":4.2409999999999997,"15min":4.2409999999999997},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.337,"15min":1.298},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":1.1970000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.4279999999999999,"15min":1.3600000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":4.7530000000000001,"5min":4.7530000000000001,"15min":4.7530000000000001},"last":1.296},{"interface":"front","average":{"1min":1.742,"5min":1.5980000000000001,"15min":1.5329999999999999},"min":{"1min":1.03,"5min":1.03,"15min":0.75},"max":{"1min":5.7389999999999999,"5min":5.7389999999999999,"15min":5.7389999999999999},"last":1.429}]},{"osd":6,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.5700000000000001,"15min":1.53},"min":{"1min":1.05,"5min":1.05,"15min":1.02},"max":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.661,"5min":1.5329999999999999,"15min":1.4810000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":1.52}]},{"osd":7,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.605,"15min":1.5489999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":5.6289999999999996,"5min":5.6289999999999996,"15min":5.6289999999999996},"last":1.7110000000000001},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.607,"15min":1.5549999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.5270000000000001,"5min":4.5270000000000001,"15min":4.5270000000000001},"last":1.48}]}]},{"osd":0,"up_from":13,"seq":55834575775,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38140,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333700,"statfs":{"total":96636764160,"available":96597708800,"internally_reserved":0,"allocated":2867200,"data_stored":1759119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5669999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.173,"5min":1.123,"15min":1.091},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.222,"15min":1.163},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.226,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0580000000000001,"15min":1.0800000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.2669999999999999,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.294,"15min":1.27},"min":{"1min":0.625,"5min":0.625,"15min":0.621},"max":{"1min":2.8250000000000002,"5min":4.0220000000000002,"15min":4.0220000000000002},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.159,"5min":1.2110000000000001,"15min":1.1839999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.319}]},{"osd":4,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.349,"15min":1.3680000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4199999999999999,"15min":1.411},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.6799999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4330000000000001,"15min":1.4019999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.68700000000000006},"max":{"1min":2.0659999999999998,"5min":2.0920000000000001,"15min":2.2480000000000002},"last":1.339},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5249999999999999,"15min":1.478},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":3.2229999999999999,"5min":3.887,"15min":3.887},"last":1.262}]},{"osd":6,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.504,"15min":1.476},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.514,"15min":1.508},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.748},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.2070000000000001}]},{"osd":7,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.546,"15min":1.5369999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.86199999999999999},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.573,"15min":1.548},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.194,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.6459999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T06:53:53.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3f is not active+clean 2023-10-08T06:53:53.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.837391+0000', 'last_change': '2023-10-08T06:53:46.837391+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.837391+0000', 'last_undegraded': '2023-10-08T06:53:46.837391+0000', 'last_fullsized': '2023-10-08T06:53:46.837391+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T06:53:53.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3e is not active+clean 2023-10-08T06:53:53.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.835340+0000', 'last_change': '2023-10-08T06:53:46.835340+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.835340+0000', 'last_undegraded': '2023-10-08T06:53:46.835340+0000', 'last_fullsized': '2023-10-08T06:53:46.835340+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3d is not active+clean 2023-10-08T06:53:53.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.836968+0000', 'last_change': '2023-10-08T06:53:46.836968+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.836968+0000', 'last_undegraded': '2023-10-08T06:53:46.836968+0000', 'last_fullsized': '2023-10-08T06:53:46.836968+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3c is not active+clean 2023-10-08T06:53:53.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.832209+0000', 'last_change': '2023-10-08T06:53:46.832209+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.832209+0000', 'last_undegraded': '2023-10-08T06:53:46.832209+0000', 'last_fullsized': '2023-10-08T06:53:46.832209+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T06:53:53.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3b is not active+clean 2023-10-08T06:53:53.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.830068+0000', 'last_change': '2023-10-08T06:53:46.830068+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.830068+0000', 'last_undegraded': '2023-10-08T06:53:46.830068+0000', 'last_fullsized': '2023-10-08T06:53:46.830068+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3a is not active+clean 2023-10-08T06:53:53.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.830196+0000', 'last_change': '2023-10-08T06:53:46.830196+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.830196+0000', 'last_undegraded': '2023-10-08T06:53:46.830196+0000', 'last_fullsized': '2023-10-08T06:53:46.830196+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.39 is not active+clean 2023-10-08T06:53:53.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.833262+0000', 'last_change': '2023-10-08T06:53:46.833262+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.833262+0000', 'last_undegraded': '2023-10-08T06:53:46.833262+0000', 'last_fullsized': '2023-10-08T06:53:46.833262+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T06:53:53.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.38 is not active+clean 2023-10-08T06:53:53.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.831994+0000', 'last_change': '2023-10-08T06:53:46.831994+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.831994+0000', 'last_undegraded': '2023-10-08T06:53:46.831994+0000', 'last_fullsized': '2023-10-08T06:53:46.831994+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T06:53:53.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.37 is not active+clean 2023-10-08T06:53:53.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.832152+0000', 'last_change': '2023-10-08T06:53:46.832152+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.832152+0000', 'last_undegraded': '2023-10-08T06:53:46.832152+0000', 'last_fullsized': '2023-10-08T06:53:46.832152+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.36 is not active+clean 2023-10-08T06:53:53.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.828877+0000', 'last_change': '2023-10-08T06:53:46.828877+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.828877+0000', 'last_undegraded': '2023-10-08T06:53:46.828877+0000', 'last_fullsized': '2023-10-08T06:53:46.828877+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.35 is not active+clean 2023-10-08T06:53:53.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.826488+0000', 'last_change': '2023-10-08T06:53:46.826488+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.826488+0000', 'last_undegraded': '2023-10-08T06:53:46.826488+0000', 'last_fullsized': '2023-10-08T06:53:46.826488+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T06:53:53.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.34 is not active+clean 2023-10-08T06:53:53.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.831044+0000', 'last_change': '2023-10-08T06:53:46.831044+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.831044+0000', 'last_undegraded': '2023-10-08T06:53:46.831044+0000', 'last_fullsized': '2023-10-08T06:53:46.831044+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T06:53:53.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.33 is not active+clean 2023-10-08T06:53:53.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.831261+0000', 'last_change': '2023-10-08T06:53:46.831261+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.831261+0000', 'last_undegraded': '2023-10-08T06:53:46.831261+0000', 'last_fullsized': '2023-10-08T06:53:46.831261+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.32 is not active+clean 2023-10-08T06:53:53.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.830002+0000', 'last_change': '2023-10-08T06:53:46.830002+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.830002+0000', 'last_undegraded': '2023-10-08T06:53:46.830002+0000', 'last_fullsized': '2023-10-08T06:53:46.830002+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T06:53:53.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.31 is not active+clean 2023-10-08T06:53:53.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.833676+0000', 'last_change': '2023-10-08T06:53:46.833676+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.833676+0000', 'last_undegraded': '2023-10-08T06:53:46.833676+0000', 'last_fullsized': '2023-10-08T06:53:46.833676+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.30 is not active+clean 2023-10-08T06:53:53.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.831113+0000', 'last_change': '2023-10-08T06:53:46.831113+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.831113+0000', 'last_undegraded': '2023-10-08T06:53:46.831113+0000', 'last_fullsized': '2023-10-08T06:53:46.831113+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2f is not active+clean 2023-10-08T06:53:53.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.834745+0000', 'last_change': '2023-10-08T06:53:46.834745+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.834745+0000', 'last_undegraded': '2023-10-08T06:53:46.834745+0000', 'last_fullsized': '2023-10-08T06:53:46.834745+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T06:53:53.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2e is not active+clean 2023-10-08T06:53:53.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.825966+0000', 'last_change': '2023-10-08T06:53:46.825966+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.825966+0000', 'last_undegraded': '2023-10-08T06:53:46.825966+0000', 'last_fullsized': '2023-10-08T06:53:46.825966+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T06:53:53.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2d is not active+clean 2023-10-08T06:53:53.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.841643+0000', 'last_change': '2023-10-08T06:53:46.841643+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.841643+0000', 'last_undegraded': '2023-10-08T06:53:46.841643+0000', 'last_fullsized': '2023-10-08T06:53:46.841643+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T06:53:53.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2c is not active+clean 2023-10-08T06:53:53.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.826979+0000', 'last_change': '2023-10-08T06:53:46.826979+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.826979+0000', 'last_undegraded': '2023-10-08T06:53:46.826979+0000', 'last_fullsized': '2023-10-08T06:53:46.826979+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T06:53:53.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2b is not active+clean 2023-10-08T06:53:53.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.830751+0000', 'last_change': '2023-10-08T06:53:46.830751+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.830751+0000', 'last_undegraded': '2023-10-08T06:53:46.830751+0000', 'last_fullsized': '2023-10-08T06:53:46.830751+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2a is not active+clean 2023-10-08T06:53:53.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.836034+0000', 'last_change': '2023-10-08T06:53:46.836034+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.836034+0000', 'last_undegraded': '2023-10-08T06:53:46.836034+0000', 'last_fullsized': '2023-10-08T06:53:46.836034+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T06:53:53.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.29 is not active+clean 2023-10-08T06:53:53.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.826700+0000', 'last_change': '2023-10-08T06:53:46.826700+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.826700+0000', 'last_undegraded': '2023-10-08T06:53:46.826700+0000', 'last_fullsized': '2023-10-08T06:53:46.826700+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T06:53:53.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.28 is not active+clean 2023-10-08T06:53:53.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.825977+0000', 'last_change': '2023-10-08T06:53:46.825977+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.825977+0000', 'last_undegraded': '2023-10-08T06:53:46.825977+0000', 'last_fullsized': '2023-10-08T06:53:46.825977+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T06:53:53.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.27 is not active+clean 2023-10-08T06:53:53.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.836905+0000', 'last_change': '2023-10-08T06:53:46.836905+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.836905+0000', 'last_undegraded': '2023-10-08T06:53:46.836905+0000', 'last_fullsized': '2023-10-08T06:53:46.836905+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T06:53:53.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.26 is not active+clean 2023-10-08T06:53:53.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.838574+0000', 'last_change': '2023-10-08T06:53:46.838574+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.838574+0000', 'last_undegraded': '2023-10-08T06:53:46.838574+0000', 'last_fullsized': '2023-10-08T06:53:46.838574+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T06:53:53.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.25 is not active+clean 2023-10-08T06:53:53.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.828887+0000', 'last_change': '2023-10-08T06:53:46.828887+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.828887+0000', 'last_undegraded': '2023-10-08T06:53:46.828887+0000', 'last_fullsized': '2023-10-08T06:53:46.828887+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.24 is not active+clean 2023-10-08T06:53:53.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.825937+0000', 'last_change': '2023-10-08T06:53:46.825937+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.825937+0000', 'last_undegraded': '2023-10-08T06:53:46.825937+0000', 'last_fullsized': '2023-10-08T06:53:46.825937+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T06:53:53.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.23 is not active+clean 2023-10-08T06:53:53.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.831793+0000', 'last_change': '2023-10-08T06:53:46.831793+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.831793+0000', 'last_undegraded': '2023-10-08T06:53:46.831793+0000', 'last_fullsized': '2023-10-08T06:53:46.831793+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T06:53:53.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.22 is not active+clean 2023-10-08T06:53:53.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.825313+0000', 'last_change': '2023-10-08T06:53:46.825313+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.825313+0000', 'last_undegraded': '2023-10-08T06:53:46.825313+0000', 'last_fullsized': '2023-10-08T06:53:46.825313+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T06:53:53.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.21 is not active+clean 2023-10-08T06:53:53.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.828751+0000', 'last_change': '2023-10-08T06:53:46.828751+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.828751+0000', 'last_undegraded': '2023-10-08T06:53:46.828751+0000', 'last_fullsized': '2023-10-08T06:53:46.828751+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.20 is not active+clean 2023-10-08T06:53:53.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.831002+0000', 'last_change': '2023-10-08T06:53:46.831002+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.831002+0000', 'last_undegraded': '2023-10-08T06:53:46.831002+0000', 'last_fullsized': '2023-10-08T06:53:46.831002+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T06:53:53.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1c is not active+clean 2023-10-08T06:53:53.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.837790+0000', 'last_change': '2023-10-08T06:53:46.837790+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.837790+0000', 'last_undegraded': '2023-10-08T06:53:46.837790+0000', 'last_fullsized': '2023-10-08T06:53:46.837790+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1d is not active+clean 2023-10-08T06:53:53.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.836525+0000', 'last_change': '2023-10-08T06:53:46.836525+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.836525+0000', 'last_undegraded': '2023-10-08T06:53:46.836525+0000', 'last_fullsized': '2023-10-08T06:53:46.836525+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1e is not active+clean 2023-10-08T06:53:53.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.832812+0000', 'last_change': '2023-10-08T06:53:46.832812+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.832812+0000', 'last_undegraded': '2023-10-08T06:53:46.832812+0000', 'last_fullsized': '2023-10-08T06:53:46.832812+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T06:53:53.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1f is not active+clean 2023-10-08T06:53:53.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.844177+0000', 'last_change': '2023-10-08T06:53:46.844177+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.844177+0000', 'last_undegraded': '2023-10-08T06:53:46.844177+0000', 'last_fullsized': '2023-10-08T06:53:46.844177+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T06:53:53.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.18 is not active+clean 2023-10-08T06:53:53.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.826941+0000', 'last_change': '2023-10-08T06:53:46.826941+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.826941+0000', 'last_undegraded': '2023-10-08T06:53:46.826941+0000', 'last_fullsized': '2023-10-08T06:53:46.826941+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T06:53:53.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.19 is not active+clean 2023-10-08T06:53:53.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.831287+0000', 'last_change': '2023-10-08T06:53:46.831287+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.831287+0000', 'last_undegraded': '2023-10-08T06:53:46.831287+0000', 'last_fullsized': '2023-10-08T06:53:46.831287+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T06:53:53.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1a is not active+clean 2023-10-08T06:53:53.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.835744+0000', 'last_change': '2023-10-08T06:53:46.835744+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.835744+0000', 'last_undegraded': '2023-10-08T06:53:46.835744+0000', 'last_fullsized': '2023-10-08T06:53:46.835744+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1b is not active+clean 2023-10-08T06:53:53.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.834908+0000', 'last_change': '2023-10-08T06:53:46.834908+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.834908+0000', 'last_undegraded': '2023-10-08T06:53:46.834908+0000', 'last_fullsized': '2023-10-08T06:53:46.834908+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T06:53:53.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.14 is not active+clean 2023-10-08T06:53:53.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.828590+0000', 'last_change': '2023-10-08T06:53:46.828590+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.828590+0000', 'last_undegraded': '2023-10-08T06:53:46.828590+0000', 'last_fullsized': '2023-10-08T06:53:46.828590+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T06:53:53.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.15 is not active+clean 2023-10-08T06:53:53.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.825152+0000', 'last_change': '2023-10-08T06:53:46.825152+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.825152+0000', 'last_undegraded': '2023-10-08T06:53:46.825152+0000', 'last_fullsized': '2023-10-08T06:53:46.825152+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T06:53:53.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.16 is not active+clean 2023-10-08T06:53:53.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.825174+0000', 'last_change': '2023-10-08T06:53:46.825174+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.825174+0000', 'last_undegraded': '2023-10-08T06:53:46.825174+0000', 'last_fullsized': '2023-10-08T06:53:46.825174+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T06:53:53.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.10 is not active+clean 2023-10-08T06:53:53.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.830035+0000', 'last_change': '2023-10-08T06:53:46.830035+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.830035+0000', 'last_undegraded': '2023-10-08T06:53:46.830035+0000', 'last_fullsized': '2023-10-08T06:53:46.830035+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T06:53:53.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.11 is not active+clean 2023-10-08T06:53:53.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.833041+0000', 'last_change': '2023-10-08T06:53:46.833041+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.833041+0000', 'last_undegraded': '2023-10-08T06:53:46.833041+0000', 'last_fullsized': '2023-10-08T06:53:46.833041+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.12 is not active+clean 2023-10-08T06:53:53.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.829716+0000', 'last_change': '2023-10-08T06:53:46.829716+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.829716+0000', 'last_undegraded': '2023-10-08T06:53:46.829716+0000', 'last_fullsized': '2023-10-08T06:53:46.829716+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.13 is not active+clean 2023-10-08T06:53:53.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.835847+0000', 'last_change': '2023-10-08T06:53:46.835847+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.835847+0000', 'last_undegraded': '2023-10-08T06:53:46.835847+0000', 'last_fullsized': '2023-10-08T06:53:46.835847+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.c is not active+clean 2023-10-08T06:53:53.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.840640+0000', 'last_change': '2023-10-08T06:53:46.840640+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.840640+0000', 'last_undegraded': '2023-10-08T06:53:46.840640+0000', 'last_fullsized': '2023-10-08T06:53:46.840640+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T06:53:53.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.d is not active+clean 2023-10-08T06:53:53.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.841115+0000', 'last_change': '2023-10-08T06:53:46.841115+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.841115+0000', 'last_undegraded': '2023-10-08T06:53:46.841115+0000', 'last_fullsized': '2023-10-08T06:53:46.841115+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T06:53:53.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.e is not active+clean 2023-10-08T06:53:53.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.837276+0000', 'last_change': '2023-10-08T06:53:46.837276+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.837276+0000', 'last_undegraded': '2023-10-08T06:53:46.837276+0000', 'last_fullsized': '2023-10-08T06:53:46.837276+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T06:53:53.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.f is not active+clean 2023-10-08T06:53:53.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.835581+0000', 'last_change': '2023-10-08T06:53:46.835581+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.835581+0000', 'last_undegraded': '2023-10-08T06:53:46.835581+0000', 'last_fullsized': '2023-10-08T06:53:46.835581+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T06:53:53.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.8 is not active+clean 2023-10-08T06:53:53.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.837818+0000', 'last_change': '2023-10-08T06:53:46.837818+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.837818+0000', 'last_undegraded': '2023-10-08T06:53:46.837818+0000', 'last_fullsized': '2023-10-08T06:53:46.837818+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T06:53:53.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.9 is not active+clean 2023-10-08T06:53:53.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.832393+0000', 'last_change': '2023-10-08T06:53:46.832393+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.832393+0000', 'last_undegraded': '2023-10-08T06:53:46.832393+0000', 'last_fullsized': '2023-10-08T06:53:46.832393+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.a is not active+clean 2023-10-08T06:53:53.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.840127+0000', 'last_change': '2023-10-08T06:53:46.840127+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.840127+0000', 'last_undegraded': '2023-10-08T06:53:46.840127+0000', 'last_fullsized': '2023-10-08T06:53:46.840127+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T06:53:53.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.b is not active+clean 2023-10-08T06:53:53.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.835282+0000', 'last_change': '2023-10-08T06:53:46.835282+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.835282+0000', 'last_undegraded': '2023-10-08T06:53:46.835282+0000', 'last_fullsized': '2023-10-08T06:53:46.835282+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.4 is not active+clean 2023-10-08T06:53:53.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.833006+0000', 'last_change': '2023-10-08T06:53:46.833006+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.833006+0000', 'last_undegraded': '2023-10-08T06:53:46.833006+0000', 'last_fullsized': '2023-10-08T06:53:46.833006+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.5 is not active+clean 2023-10-08T06:53:53.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.833355+0000', 'last_change': '2023-10-08T06:53:46.833355+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.833355+0000', 'last_undegraded': '2023-10-08T06:53:46.833355+0000', 'last_fullsized': '2023-10-08T06:53:46.833355+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T06:53:53.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.6 is not active+clean 2023-10-08T06:53:53.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.836735+0000', 'last_change': '2023-10-08T06:53:46.836735+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.836735+0000', 'last_undegraded': '2023-10-08T06:53:46.836735+0000', 'last_fullsized': '2023-10-08T06:53:46.836735+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T06:53:53.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.7 is not active+clean 2023-10-08T06:53:53.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.831727+0000', 'last_change': '2023-10-08T06:53:46.831727+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.831727+0000', 'last_undegraded': '2023-10-08T06:53:46.831727+0000', 'last_fullsized': '2023-10-08T06:53:46.831727+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T06:53:53.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.0 is not active+clean 2023-10-08T06:53:53.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.825803+0000', 'last_change': '2023-10-08T06:53:46.825803+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.825803+0000', 'last_undegraded': '2023-10-08T06:53:46.825803+0000', 'last_fullsized': '2023-10-08T06:53:46.825803+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T06:53:53.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1 is not active+clean 2023-10-08T06:53:53.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.828614+0000', 'last_change': '2023-10-08T06:53:46.828614+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.828614+0000', 'last_undegraded': '2023-10-08T06:53:46.828614+0000', 'last_fullsized': '2023-10-08T06:53:46.828614+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T06:53:53.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2 is not active+clean 2023-10-08T06:53:53.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.834744+0000', 'last_change': '2023-10-08T06:53:46.834744+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.834744+0000', 'last_undegraded': '2023-10-08T06:53:46.834744+0000', 'last_fullsized': '2023-10-08T06:53:46.834744+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T06:53:53.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3 is not active+clean 2023-10-08T06:53:53.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.832811+0000', 'last_change': '2023-10-08T06:53:46.832811+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.832811+0000', 'last_undegraded': '2023-10-08T06:53:46.832811+0000', 'last_fullsized': '2023-10-08T06:53:46.832811+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T06:53:53.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.17 is not active+clean 2023-10-08T06:53:53.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 183, 'state': 'creating+peering', 'last_fresh': '2023-10-08T06:53:46.827498+0000', 'last_change': '2023-10-08T06:53:46.827498+0000', 'last_active': '2023-10-08T06:53:46.808694+0000', 'last_peered': '2023-10-08T06:53:46.808694+0000', 'last_clean': '2023-10-08T06:53:46.808694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T06:53:46.827498+0000', 'last_undegraded': '2023-10-08T06:53:46.827498+0000', 'last_fullsized': '2023-10-08T06:53:46.827498+0000', 'mapping_epoch': 183, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 183, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'last_clean_scrub_stamp': '2023-10-08T06:53:46.808694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T06:53:53.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T06:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.571+0000 7f1587c10640 1 Processor -- start 2023-10-08T06:53:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.572+0000 7f1587c10640 1 -- start start 2023-10-08T06:53:53.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.573+0000 7f1587c10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1580161310 0x7f15801616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.573+0000 7f1587c10640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f158015dd40 0x7f15800b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.573+0000 7f1587c10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158015e1f0 0x7f15800b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.573+0000 7f1587c10640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15800ce5f0 con 0x7f158015dd40 2023-10-08T06:53:53.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.573+0000 7f1587c10640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1580157a50 con 0x7f1580161310 2023-10-08T06:53:53.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.573+0000 7f1587c10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15800cd590 con 0x7f158015e1f0 2023-10-08T06:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.573+0000 7f15851ab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f158015dd40 0x7f15800b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.573+0000 7f15851ab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f158015dd40 0x7f15800b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39136/0 (socket says 172.21.15.70:39136) 2023-10-08T06:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.573+0000 7f15851ab640 1 -- 172.21.15.70:0/1731861482 learned_addr learned my addr 172.21.15.70:0/1731861482 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.574+0000 7f15861ad640 1 --2- 172.21.15.70:0/1731861482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158015e1f0 0x7f15800b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.574+0000 7f15859ac640 1 --2- 172.21.15.70:0/1731861482 >> v2:172.21.15.181:3300/0 conn(0x7f1580161310 0x7f15801616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.574+0000 7f15851ab640 1 -- 172.21.15.70:0/1731861482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158015e1f0 msgr2=0x7f15800b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.574+0000 7f15851ab640 1 --2- 172.21.15.70:0/1731861482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158015e1f0 0x7f15800b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.574+0000 7f15851ab640 1 -- 172.21.15.70:0/1731861482 >> v2:172.21.15.181:3300/0 conn(0x7f1580161310 msgr2=0x7f15801616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.574+0000 7f15851ab640 1 --2- 172.21.15.70:0/1731861482 >> v2:172.21.15.181:3300/0 conn(0x7f1580161310 0x7f15801616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.574+0000 7f15851ab640 1 -- 172.21.15.70:0/1731861482 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15800cd830 con 0x7f158015dd40 2023-10-08T06:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.574+0000 7f15859ac640 1 --2- 172.21.15.70:0/1731861482 >> v2:172.21.15.181:3300/0 conn(0x7f1580161310 0x7f15801616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.575+0000 7f15851ab640 1 --2- 172.21.15.70:0/1731861482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 0x7f15800b4720 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f158008a0f0 tx=0x7f157002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9603d0bf50eebb47 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f15849aa640 1 -- 172.21.15.70:0/1731861482 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570031860 con 0x7f158015dd40 2023-10-08T06:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f15849aa640 1 -- 172.21.15.70:0/1731861482 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15700319c0 con 0x7f158015dd40 2023-10-08T06:53:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f15849aa640 1 -- 172.21.15.70:0/1731861482 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570031d50 con 0x7f158015dd40 2023-10-08T06:53:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f1587c10640 1 -- 172.21.15.70:0/1731861482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 msgr2=0x7f15800b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f1587c10640 1 --2- 172.21.15.70:0/1731861482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 0x7f15800b4720 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f158008a0f0 tx=0x7f157002b900 comp rx=0 tx=0).stop 2023-10-08T06:53:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f1587c10640 1 -- 172.21.15.70:0/1731861482 shutdown_connections 2023-10-08T06:53:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f1587c10640 1 --2- 172.21.15.70:0/1731861482 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f158015e1f0 0x7f15800b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f1587c10640 1 --2- 172.21.15.70:0/1731861482 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 0x7f15800b4720 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f1587c10640 1 --2- 172.21.15.70:0/1731861482 >> v2:172.21.15.181:3300/0 conn(0x7f1580161310 0x7f15801616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f1587c10640 1 -- 172.21.15.70:0/1731861482 >> 172.21.15.70:0/1731861482 conn(0x7f15800d2ca0 msgr2=0x7f15800d3080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f1587c10640 1 -- 172.21.15.70:0/1731861482 shutdown_connections 2023-10-08T06:53:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.576+0000 7f1587c10640 1 -- 172.21.15.70:0/1731861482 wait complete. 2023-10-08T06:53:53.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.578+0000 7f1587c10640 1 Processor -- start 2023-10-08T06:53:53.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.578+0000 7f1587c10640 1 -- start start 2023-10-08T06:53:53.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.579+0000 7f1587c10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 0x7f1580154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.579+0000 7f1587c10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f158015e1f0 0x7f15801f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.579+0000 7f1587c10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1580161310 0x7f15801f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.579+0000 7f1587c10640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15800cb7e0 con 0x7f158015dd40 2023-10-08T06:53:53.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.579+0000 7f1587c10640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15800cbac0 con 0x7f158015e1f0 2023-10-08T06:53:53.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.579+0000 7f1587c10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15800cbda0 con 0x7f1580161310 2023-10-08T06:53:53.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.579+0000 7f15859ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 0x7f1580154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:53.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.579+0000 7f15859ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 0x7f1580154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39146/0 (socket says 172.21.15.70:39146) 2023-10-08T06:53:53.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.579+0000 7f15859ac640 1 -- 172.21.15.70:0/3800397323 learned_addr learned my addr 172.21.15.70:0/3800397323 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:53.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.580+0000 7f15859ac640 1 -- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1580161310 msgr2=0x7f15801f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.580+0000 7f15859ac640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1580161310 0x7f15801f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.580+0000 7f15859ac640 1 -- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f158015e1f0 msgr2=0x7f15801f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.580+0000 7f15859ac640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f158015e1f0 0x7f15801f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.580+0000 7f15859ac640 1 -- 172.21.15.70:0/3800397323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15700094d0 con 0x7f158015dd40 2023-10-08T06:53:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.580+0000 7f15859ac640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 0x7f1580154630 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f157c00cf60 tx=0x7f157c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.580+0000 7f15767fc640 1 -- 172.21.15.70:0/3800397323 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00d840 con 0x7f158015dd40 2023-10-08T06:53:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.580+0000 7f15767fc640 1 -- 172.21.15.70:0/3800397323 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c00d9a0 con 0x7f158015dd40 2023-10-08T06:53:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.580+0000 7f15767fc640 1 -- 172.21.15.70:0/3800397323 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c00dd30 con 0x7f158015dd40 2023-10-08T06:53:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.581+0000 7f1587c10640 1 -- 172.21.15.70:0/3800397323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15800cc080 con 0x7f158015dd40 2023-10-08T06:53:53.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.581+0000 7f1587c10640 1 -- 172.21.15.70:0/3800397323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1580160000 con 0x7f158015dd40 2023-10-08T06:53:53.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.582+0000 7f1587c10640 1 -- 172.21.15.70:0/3800397323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1580066d80 con 0x7f158015dd40 2023-10-08T06:53:53.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.583+0000 7f15767fc640 1 -- 172.21.15.70:0/3800397323 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f157c015440 con 0x7f158015dd40 2023-10-08T06:53:53.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.584+0000 7f15767fc640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f156406afc0 0x7f156406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:53.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.584+0000 7f15767fc640 1 -- 172.21.15.70:0/3800397323 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f157c08a310 con 0x7f158015dd40 2023-10-08T06:53:53.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.584+0000 7f15851ab640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f156406afc0 0x7f156406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:53.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.586+0000 7f15851ab640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f156406afc0 0x7f156406d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1570036000 tx=0x7f1570030000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:53.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.588+0000 7f15767fc640 1 -- 172.21.15.70:0/3800397323 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f157c0551a0 con 0x7f158015dd40 2023-10-08T06:53:53.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.845+0000 7f1587c10640 1 -- 172.21.15.70:0/3800397323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f15800cf500 con 0x7f158015dd40 2023-10-08T06:53:53.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.846+0000 7f15767fc640 1 -- 172.21.15.70:0/3800397323 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3233 (secure 0 0 0) 0x7f157c01acc0 con 0x7f158015dd40 2023-10-08T06:53:53.848 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:53.848 INFO:teuthology.orchestra.run.smithi070.stdout:{"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4681,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":185,"num_osds":8,"num_up_osds":8,"osd_up_since":1696743363,"num_in_osds":8,"osd_in_since":1696743355,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":374284288,"bytes_avail":772719828992,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":83,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":9591}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2023-10-08T06:53:30.071813+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-08T06:53:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 -- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f156406afc0 msgr2=0x7f156406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f156406afc0 0x7f156406d480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1570036000 tx=0x7f1570030000 comp rx=0 tx=0).stop 2023-10-08T06:53:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 -- 172.21.15.70:0/3800397323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 msgr2=0x7f1580154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 0x7f1580154630 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f157c00cf60 tx=0x7f157c00cf90 comp rx=0 tx=0).stop 2023-10-08T06:53:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 -- 172.21.15.70:0/3800397323 shutdown_connections 2023-10-08T06:53:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1580161310 0x7f15801f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f158015e1f0 0x7f15801f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f156406afc0 0x7f156406d480 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 --2- 172.21.15.70:0/3800397323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f158015dd40 0x7f1580154630 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 -- 172.21.15.70:0/3800397323 >> 172.21.15.70:0/3800397323 conn(0x7f15800d2ca0 msgr2=0x7f15800733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:53.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 -- 172.21.15.70:0/3800397323 shutdown_connections 2023-10-08T06:53:53.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:53.849+0000 7f1587c10640 1 -- 172.21.15.70:0/3800397323 wait complete. 2023-10-08T06:53:53.869 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T06:53:53.869 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T06:53:54.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.078+0000 7f2dc0b41640 1 Processor -- start 2023-10-08T06:53:54.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.078+0000 7f2dc0b41640 1 -- start start 2023-10-08T06:53:54.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.080+0000 7f2dc0b41640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dbc15cd90 0x7f2dbc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:54.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.080+0000 7f2dc0b41640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc160940 0x7f2dbc070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:54.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.080+0000 7f2dc0b41640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc0c48f0 con 0x7f2dbc070d50 2023-10-08T06:53:54.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.080+0000 7f2dc0b41640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc0c4d80 con 0x7f2dbc15cd90 2023-10-08T06:53:54.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.080+0000 7f2dc0b41640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0c3ce0 con 0x7f2dbc160940 2023-10-08T06:53:54.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.080+0000 7f2dba59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2dbc15cd90 0x7f2dbc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:54.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.080+0000 7f2dbad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2dbc070d50 0x7f2dbc0cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50676/0 (socket says 172.21.15.70:50676) 2023-10-08T06:53:54.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.080+0000 7f2dbad9d640 1 -- 172.21.15.70:0/2292860288 learned_addr learned my addr 172.21.15.70:0/2292860288 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:54.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.081+0000 7f2db1d9b640 1 --2- 172.21.15.70:0/2292860288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc160940 0x7f2dbc070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:54.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.081+0000 7f2db9d9b640 1 -- 172.21.15.70:0/2292860288 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669656740 0 0) 0x7f2dbc0c48f0 con 0x7f2dbc070d50 2023-10-08T06:53:54.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.081+0000 7f2db9d9b640 1 -- 172.21.15.70:0/2292860288 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d9c003520 con 0x7f2dbc070d50 2023-10-08T06:53:54.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.081+0000 7f2dba59c640 1 -- 172.21.15.70:0/2292860288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc160940 msgr2=0x7f2dbc070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:54.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.081+0000 7f2dba59c640 1 --2- 172.21.15.70:0/2292860288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc160940 0x7f2dbc070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.081+0000 7f2dba59c640 1 -- 172.21.15.70:0/2292860288 >> v1:172.21.15.70:6789/0 conn(0x7f2dbc070d50 legacy=0x7f2dbc0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:54.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.081+0000 7f2dba59c640 1 -- 172.21.15.70:0/2292860288 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc0c3fc0 con 0x7f2dbc15cd90 2023-10-08T06:53:54.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.082+0000 7f2dba59c640 1 --2- 172.21.15.70:0/2292860288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 0x7f2dbc0702d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0097a0 tx=0x7f2dac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ea9dda269ae351 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.083+0000 7f2db9d9b640 1 -- 172.21.15.70:0/2292860288 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac031830 con 0x7f2dbc15cd90 2023-10-08T06:53:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.083+0000 7f2db9d9b640 1 -- 172.21.15.70:0/2292860288 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac031990 con 0x7f2dbc15cd90 2023-10-08T06:53:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.083+0000 7f2db9d9b640 1 -- 172.21.15.70:0/2292860288 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac031d20 con 0x7f2dbc15cd90 2023-10-08T06:53:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.083+0000 7f2dc0b41640 1 -- 172.21.15.70:0/2292860288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 msgr2=0x7f2dbc0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.083+0000 7f2dc0b41640 1 --2- 172.21.15.70:0/2292860288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 0x7f2dbc0702d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0097a0 tx=0x7f2dac02b8d0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.083+0000 7f2dc0b41640 1 -- 172.21.15.70:0/2292860288 shutdown_connections 2023-10-08T06:53:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.083+0000 7f2dc0b41640 1 --2- 172.21.15.70:0/2292860288 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc160940 0x7f2dbc070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.083+0000 7f2dc0b41640 1 --2- 172.21.15.70:0/2292860288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 0x7f2dbc0702d0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.083+0000 7f2dc0b41640 1 -- 172.21.15.70:0/2292860288 >> 172.21.15.70:0/2292860288 conn(0x7f2dbc06a380 msgr2=0x7f2dbc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.084+0000 7f2dc0b41640 1 -- 172.21.15.70:0/2292860288 shutdown_connections 2023-10-08T06:53:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.084+0000 7f2dc0b41640 1 -- 172.21.15.70:0/2292860288 wait complete. 2023-10-08T06:53:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.085+0000 7f2dc0b41640 1 Processor -- start 2023-10-08T06:53:54.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.085+0000 7f2dc0b41640 1 -- start start 2023-10-08T06:53:54.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.086+0000 7f2dc0b41640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dbc070d50 0x7f2dbc15fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:54.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.086+0000 7f2dc0b41640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 0x7f2dbc0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.086+0000 7f2dc0b41640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc160940 0x7f2dbc0c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.086+0000 7f2dc0b41640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0b2890 con 0x7f2dbc15cd90 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2dc0b41640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0b2b70 con 0x7f2dbc070d50 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2dc0b41640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0b2e50 con 0x7f2dbc160940 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2db1d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 0x7f2dbc0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2db1d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 0x7f2dbc0cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39162/0 (socket says 172.21.15.70:39162) 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2db1d9b640 1 -- 172.21.15.70:0/1743649286 learned_addr learned my addr 172.21.15.70:0/1743649286 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2db1d9b640 1 -- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc160940 msgr2=0x7f2dbc0c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2db1d9b640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc160940 0x7f2dbc0c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2db1d9b640 1 -- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dbc070d50 msgr2=0x7f2dbc15fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:54.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2db1d9b640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dbc070d50 0x7f2dbc15fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2db1d9b640 1 -- 172.21.15.70:0/1743649286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac009400 con 0x7f2dbc15cd90 2023-10-08T06:53:54.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.087+0000 7f2db1d9b640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 0x7f2dbc0cb780 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f2da400d5f0 tx=0x7f2da400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:54.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.088+0000 7f2db37fe640 1 -- 172.21.15.70:0/1743649286 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da40125f0 con 0x7f2dbc15cd90 2023-10-08T06:53:54.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.088+0000 7f2db37fe640 1 -- 172.21.15.70:0/1743649286 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da4012750 con 0x7f2dbc15cd90 2023-10-08T06:53:54.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.088+0000 7f2db37fe640 1 -- 172.21.15.70:0/1743649286 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4012b00 con 0x7f2dbc15cd90 2023-10-08T06:53:54.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.088+0000 7f2dc0b41640 1 -- 172.21.15.70:0/1743649286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc0b3130 con 0x7f2dbc15cd90 2023-10-08T06:53:54.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.088+0000 7f2dc0b41640 1 -- 172.21.15.70:0/1743649286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc0c94d0 con 0x7f2dbc15cd90 2023-10-08T06:53:54.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.089+0000 7f2dc0b41640 1 -- 172.21.15.70:0/1743649286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc0c9690 con 0x7f2dbc15cd90 2023-10-08T06:53:54.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.090+0000 7f2db37fe640 1 -- 172.21.15.70:0/1743649286 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2da4012c60 con 0x7f2dbc15cd90 2023-10-08T06:53:54.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.091+0000 7f2db37fe640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2da006afc0 0x7f2da006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:54.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.091+0000 7f2db37fe640 1 -- 172.21.15.70:0/1743649286 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2da4089e30 con 0x7f2dbc15cd90 2023-10-08T06:53:54.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.092+0000 7f2dba59c640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2da006afc0 0x7f2da006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:54.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.093+0000 7f2dba59c640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2da006afc0 0x7f2da006d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2dac036000 tx=0x7f2dac030000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:54.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.096+0000 7f2db37fe640 1 -- 172.21.15.70:0/1743649286 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2da4054b70 con 0x7f2dbc15cd90 2023-10-08T06:53:54.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.281+0000 7f2dc0b41640 1 -- 172.21.15.70:0/1743649286 --> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2dbc0b2ce0 con 0x7f2da006afc0 2023-10-08T06:53:54.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.304+0000 7f2db37fe640 1 -- 172.21.15.70:0/1743649286 <== mgr.9345 v2:172.21.15.181:6832/4245513204 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287399 (secure 0 0 0) 0x7f2dbc0b2ce0 con 0x7f2da006afc0 2023-10-08T06:53:54.307 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:54.311 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T06:53:54.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.309+0000 7f2dc0b41640 1 -- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2da006afc0 msgr2=0x7f2da006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2da006afc0 0x7f2da006d480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2dac036000 tx=0x7f2dac030000 comp rx=0 tx=0).stop 2023-10-08T06:53:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 -- 172.21.15.70:0/1743649286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 msgr2=0x7f2dbc0cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 0x7f2dbc0cb780 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f2da400d5f0 tx=0x7f2da400dab0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 -- 172.21.15.70:0/1743649286 shutdown_connections 2023-10-08T06:53:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2dbc160940 0x7f2dbc0c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2da006afc0 0x7f2da006d480 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2dbc15cd90 0x7f2dbc0cb780 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 --2- 172.21.15.70:0/1743649286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2dbc070d50 0x7f2dbc15fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:54.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 -- 172.21.15.70:0/1743649286 >> 172.21.15.70:0/1743649286 conn(0x7f2dbc06a380 msgr2=0x7f2dbc158610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:54.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 -- 172.21.15.70:0/1743649286 shutdown_connections 2023-10-08T06:53:54.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:54.310+0000 7f2dc0b41640 1 -- 172.21.15.70:0/1743649286 wait complete. 2023-10-08T06:53:54.329 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-10-08T06:53:54.078946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":494,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2768,"ondisk_log_size":2768,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365888,"kb_used_data":19272,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754608832,"statfs":{"total":773094113280,"available":772719443968,"internally_reserved":0,"allocated":19734528,"data_stored":10700828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":354799163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.261343"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945912+0000","last_change":"2023-10-08T06:53:47.883681+0000","last_active":"2023-10-08T06:53:48.945912+0000","last_peered":"2023-10-08T06:53:48.945912+0000","last_clean":"2023-10-08T06:53:48.945912+0000","last_became_active":"2023-10-08T06:53:47.883436+0000","last_became_peered":"2023-10-08T06:53:47.883436+0000","last_unstale":"2023-10-08T06:53:48.945912+0000","last_undegraded":"2023-10-08T06:53:48.945912+0000","last_fullsized":"2023-10-08T06:53:48.945912+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:00.334530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.376169+0000","last_change":"2023-10-08T06:53:47.878367+0000","last_active":"2023-10-08T06:53:49.376169+0000","last_peered":"2023-10-08T06:53:49.376169+0000","last_clean":"2023-10-08T06:53:49.376169+0000","last_became_active":"2023-10-08T06:53:47.878109+0000","last_became_peered":"2023-10-08T06:53:47.878109+0000","last_unstale":"2023-10-08T06:53:49.376169+0000","last_undegraded":"2023-10-08T06:53:49.376169+0000","last_fullsized":"2023-10-08T06:53:49.376169+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:54.584120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.376327+0000","last_change":"2023-10-08T06:53:47.891864+0000","last_active":"2023-10-08T06:53:49.376327+0000","last_peered":"2023-10-08T06:53:49.376327+0000","last_clean":"2023-10-08T06:53:49.376327+0000","last_became_active":"2023-10-08T06:53:47.891571+0000","last_became_peered":"2023-10-08T06:53:47.891571+0000","last_unstale":"2023-10-08T06:53:49.376327+0000","last_undegraded":"2023-10-08T06:53:49.376327+0000","last_fullsized":"2023-10-08T06:53:49.376327+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:49:48.869017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.882355+0000","last_change":"2023-10-08T06:53:47.878270+0000","last_active":"2023-10-08T06:53:48.882355+0000","last_peered":"2023-10-08T06:53:48.882355+0000","last_clean":"2023-10-08T06:53:48.882355+0000","last_became_active":"2023-10-08T06:53:47.877994+0000","last_became_peered":"2023-10-08T06:53:47.877994+0000","last_unstale":"2023-10-08T06:53:48.882355+0000","last_undegraded":"2023-10-08T06:53:48.882355+0000","last_fullsized":"2023-10-08T06:53:48.882355+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:47:01.826263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374504+0000","last_change":"2023-10-08T06:53:47.866342+0000","last_active":"2023-10-08T06:53:49.374504+0000","last_peered":"2023-10-08T06:53:49.374504+0000","last_clean":"2023-10-08T06:53:49.374504+0000","last_became_active":"2023-10-08T06:53:47.866085+0000","last_became_peered":"2023-10-08T06:53:47.866085+0000","last_unstale":"2023-10-08T06:53:49.374504+0000","last_undegraded":"2023-10-08T06:53:49.374504+0000","last_fullsized":"2023-10-08T06:53:49.374504+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:58.859856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373127+0000","last_change":"2023-10-08T06:53:47.869713+0000","last_active":"2023-10-08T06:53:49.373127+0000","last_peered":"2023-10-08T06:53:49.373127+0000","last_clean":"2023-10-08T06:53:49.373127+0000","last_became_active":"2023-10-08T06:53:47.869401+0000","last_became_peered":"2023-10-08T06:53:47.869401+0000","last_unstale":"2023-10-08T06:53:49.373127+0000","last_undegraded":"2023-10-08T06:53:49.373127+0000","last_fullsized":"2023-10-08T06:53:49.373127+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:34.484753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945506+0000","last_change":"2023-10-08T06:53:47.873438+0000","last_active":"2023-10-08T06:53:48.945506+0000","last_peered":"2023-10-08T06:53:48.945506+0000","last_clean":"2023-10-08T06:53:48.945506+0000","last_became_active":"2023-10-08T06:53:47.873134+0000","last_became_peered":"2023-10-08T06:53:47.873134+0000","last_unstale":"2023-10-08T06:53:48.945506+0000","last_undegraded":"2023-10-08T06:53:48.945506+0000","last_fullsized":"2023-10-08T06:53:48.945506+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:59.922242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.878729+0000","last_change":"2023-10-08T06:53:47.881710+0000","last_active":"2023-10-08T06:53:48.878729+0000","last_peered":"2023-10-08T06:53:48.878729+0000","last_clean":"2023-10-08T06:53:48.878729+0000","last_became_active":"2023-10-08T06:53:47.881426+0000","last_became_peered":"2023-10-08T06:53:47.881426+0000","last_unstale":"2023-10-08T06:53:48.878729+0000","last_undegraded":"2023-10-08T06:53:48.878729+0000","last_fullsized":"2023-10-08T06:53:48.878729+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:58:08.231929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374374+0000","last_change":"2023-10-08T06:53:47.873331+0000","last_active":"2023-10-08T06:53:49.374374+0000","last_peered":"2023-10-08T06:53:49.374374+0000","last_clean":"2023-10-08T06:53:49.374374+0000","last_became_active":"2023-10-08T06:53:47.873070+0000","last_became_peered":"2023-10-08T06:53:47.873070+0000","last_unstale":"2023-10-08T06:53:49.374374+0000","last_undegraded":"2023-10-08T06:53:49.374374+0000","last_fullsized":"2023-10-08T06:53:49.374374+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:02.408761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375140+0000","last_change":"2023-10-08T06:53:47.864912+0000","last_active":"2023-10-08T06:53:49.375140+0000","last_peered":"2023-10-08T06:53:49.375140+0000","last_clean":"2023-10-08T06:53:49.375140+0000","last_became_active":"2023-10-08T06:53:47.864597+0000","last_became_peered":"2023-10-08T06:53:47.864597+0000","last_unstale":"2023-10-08T06:53:49.375140+0000","last_undegraded":"2023-10-08T06:53:49.375140+0000","last_fullsized":"2023-10-08T06:53:49.375140+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:23.125163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375231+0000","last_change":"2023-10-08T06:53:47.885735+0000","last_active":"2023-10-08T06:53:49.375231+0000","last_peered":"2023-10-08T06:53:49.375231+0000","last_clean":"2023-10-08T06:53:49.375231+0000","last_became_active":"2023-10-08T06:53:47.885476+0000","last_became_peered":"2023-10-08T06:53:47.885476+0000","last_unstale":"2023-10-08T06:53:49.375231+0000","last_undegraded":"2023-10-08T06:53:49.375231+0000","last_fullsized":"2023-10-08T06:53:49.375231+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:29.267997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.882686+0000","last_change":"2023-10-08T06:53:47.886577+0000","last_active":"2023-10-08T06:53:48.882686+0000","last_peered":"2023-10-08T06:53:48.882686+0000","last_clean":"2023-10-08T06:53:48.882686+0000","last_became_active":"2023-10-08T06:53:47.886312+0000","last_became_peered":"2023-10-08T06:53:47.886312+0000","last_unstale":"2023-10-08T06:53:48.882686+0000","last_undegraded":"2023-10-08T06:53:48.882686+0000","last_fullsized":"2023-10-08T06:53:48.882686+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:14:46.948507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373799+0000","last_change":"2023-10-08T06:53:47.864171+0000","last_active":"2023-10-08T06:53:49.373799+0000","last_peered":"2023-10-08T06:53:49.373799+0000","last_clean":"2023-10-08T06:53:49.373799+0000","last_became_active":"2023-10-08T06:53:47.863902+0000","last_became_peered":"2023-10-08T06:53:47.863902+0000","last_unstale":"2023-10-08T06:53:49.373799+0000","last_undegraded":"2023-10-08T06:53:49.373799+0000","last_fullsized":"2023-10-08T06:53:49.373799+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:56.220725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.882164+0000","last_change":"2023-10-08T06:53:47.885670+0000","last_active":"2023-10-08T06:53:48.882164+0000","last_peered":"2023-10-08T06:53:48.882164+0000","last_clean":"2023-10-08T06:53:48.882164+0000","last_became_active":"2023-10-08T06:53:47.885356+0000","last_became_peered":"2023-10-08T06:53:47.885356+0000","last_unstale":"2023-10-08T06:53:48.882164+0000","last_undegraded":"2023-10-08T06:53:48.882164+0000","last_fullsized":"2023-10-08T06:53:48.882164+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:35:08.460367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374955+0000","last_change":"2023-10-08T06:53:47.874415+0000","last_active":"2023-10-08T06:53:49.374955+0000","last_peered":"2023-10-08T06:53:49.374955+0000","last_clean":"2023-10-08T06:53:49.374955+0000","last_became_active":"2023-10-08T06:53:47.874165+0000","last_became_peered":"2023-10-08T06:53:47.874165+0000","last_unstale":"2023-10-08T06:53:49.374955+0000","last_undegraded":"2023-10-08T06:53:49.374955+0000","last_fullsized":"2023-10-08T06:53:49.374955+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:53.752457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.372528+0000","last_change":"2023-10-08T06:53:47.888368+0000","last_active":"2023-10-08T06:53:49.372528+0000","last_peered":"2023-10-08T06:53:49.372528+0000","last_clean":"2023-10-08T06:53:49.372528+0000","last_became_active":"2023-10-08T06:53:47.888109+0000","last_became_peered":"2023-10-08T06:53:47.888109+0000","last_unstale":"2023-10-08T06:53:49.372528+0000","last_undegraded":"2023-10-08T06:53:49.372528+0000","last_fullsized":"2023-10-08T06:53:49.372528+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:21.483421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.882720+0000","last_change":"2023-10-08T06:53:47.871573+0000","last_active":"2023-10-08T06:53:48.882720+0000","last_peered":"2023-10-08T06:53:48.882720+0000","last_clean":"2023-10-08T06:53:48.882720+0000","last_became_active":"2023-10-08T06:53:47.871281+0000","last_became_peered":"2023-10-08T06:53:47.871281+0000","last_unstale":"2023-10-08T06:53:48.882720+0000","last_undegraded":"2023-10-08T06:53:48.882720+0000","last_fullsized":"2023-10-08T06:53:48.882720+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:49:41.935880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375935+0000","last_change":"2023-10-08T06:53:47.871440+0000","last_active":"2023-10-08T06:53:49.375935+0000","last_peered":"2023-10-08T06:53:49.375935+0000","last_clean":"2023-10-08T06:53:49.375935+0000","last_became_active":"2023-10-08T06:53:47.871180+0000","last_became_peered":"2023-10-08T06:53:47.871180+0000","last_unstale":"2023-10-08T06:53:49.375935+0000","last_undegraded":"2023-10-08T06:53:49.375935+0000","last_fullsized":"2023-10-08T06:53:49.375935+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:44.026824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.951695+0000","last_change":"2023-10-08T06:53:47.897749+0000","last_active":"2023-10-08T06:53:48.951695+0000","last_peered":"2023-10-08T06:53:48.951695+0000","last_clean":"2023-10-08T06:53:48.951695+0000","last_became_active":"2023-10-08T06:53:47.896996+0000","last_became_peered":"2023-10-08T06:53:47.896996+0000","last_unstale":"2023-10-08T06:53:48.951695+0000","last_undegraded":"2023-10-08T06:53:48.951695+0000","last_fullsized":"2023-10-08T06:53:48.951695+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:02.408761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.875036+0000","last_change":"2023-10-08T06:53:47.870451+0000","last_active":"2023-10-08T06:53:48.875036+0000","last_peered":"2023-10-08T06:53:48.875036+0000","last_clean":"2023-10-08T06:53:48.875036+0000","last_became_active":"2023-10-08T06:53:47.870037+0000","last_became_peered":"2023-10-08T06:53:47.870037+0000","last_unstale":"2023-10-08T06:53:48.875036+0000","last_undegraded":"2023-10-08T06:53:48.875036+0000","last_fullsized":"2023-10-08T06:53:48.875036+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:23:40.923291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374137+0000","last_change":"2023-10-08T06:53:47.883382+0000","last_active":"2023-10-08T06:53:49.374137+0000","last_peered":"2023-10-08T06:53:49.374137+0000","last_clean":"2023-10-08T06:53:49.374137+0000","last_became_active":"2023-10-08T06:53:47.881355+0000","last_became_peered":"2023-10-08T06:53:47.881355+0000","last_unstale":"2023-10-08T06:53:49.374137+0000","last_undegraded":"2023-10-08T06:53:49.374137+0000","last_fullsized":"2023-10-08T06:53:49.374137+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:18.313136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945304+0000","last_change":"2023-10-08T06:53:47.881877+0000","last_active":"2023-10-08T06:53:48.945304+0000","last_peered":"2023-10-08T06:53:48.945304+0000","last_clean":"2023-10-08T06:53:48.945304+0000","last_became_active":"2023-10-08T06:53:47.881588+0000","last_became_peered":"2023-10-08T06:53:47.881588+0000","last_unstale":"2023-10-08T06:53:48.945304+0000","last_undegraded":"2023-10-08T06:53:48.945304+0000","last_fullsized":"2023-10-08T06:53:48.945304+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:53.402284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.879132+0000","last_change":"2023-10-08T06:53:47.884817+0000","last_active":"2023-10-08T06:53:48.879132+0000","last_peered":"2023-10-08T06:53:48.879132+0000","last_clean":"2023-10-08T06:53:48.879132+0000","last_became_active":"2023-10-08T06:53:47.884540+0000","last_became_peered":"2023-10-08T06:53:47.884540+0000","last_unstale":"2023-10-08T06:53:48.879132+0000","last_undegraded":"2023-10-08T06:53:48.879132+0000","last_fullsized":"2023-10-08T06:53:48.879132+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:29.117651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373895+0000","last_change":"2023-10-08T06:53:47.872966+0000","last_active":"2023-10-08T06:53:49.373895+0000","last_peered":"2023-10-08T06:53:49.373895+0000","last_clean":"2023-10-08T06:53:49.373895+0000","last_became_active":"2023-10-08T06:53:47.872549+0000","last_became_peered":"2023-10-08T06:53:47.872549+0000","last_unstale":"2023-10-08T06:53:49.373895+0000","last_undegraded":"2023-10-08T06:53:49.373895+0000","last_fullsized":"2023-10-08T06:53:49.373895+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:44.415095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.946128+0000","last_change":"2023-10-08T06:53:47.864951+0000","last_active":"2023-10-08T06:53:48.946128+0000","last_peered":"2023-10-08T06:53:48.946128+0000","last_clean":"2023-10-08T06:53:48.946128+0000","last_became_active":"2023-10-08T06:53:47.863680+0000","last_became_peered":"2023-10-08T06:53:47.863680+0000","last_unstale":"2023-10-08T06:53:48.946128+0000","last_undegraded":"2023-10-08T06:53:48.946128+0000","last_fullsized":"2023-10-08T06:53:48.946128+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:56.583934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945989+0000","last_change":"2023-10-08T06:53:47.886562+0000","last_active":"2023-10-08T06:53:48.945989+0000","last_peered":"2023-10-08T06:53:48.945989+0000","last_clean":"2023-10-08T06:53:48.945989+0000","last_became_active":"2023-10-08T06:53:47.876729+0000","last_became_peered":"2023-10-08T06:53:47.876729+0000","last_unstale":"2023-10-08T06:53:48.945989+0000","last_undegraded":"2023-10-08T06:53:48.945989+0000","last_fullsized":"2023-10-08T06:53:48.945989+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:17.085231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375318+0000","last_change":"2023-10-08T06:53:47.890896+0000","last_active":"2023-10-08T06:53:49.375318+0000","last_peered":"2023-10-08T06:53:49.375318+0000","last_clean":"2023-10-08T06:53:49.375318+0000","last_became_active":"2023-10-08T06:53:47.890597+0000","last_became_peered":"2023-10-08T06:53:47.890597+0000","last_unstale":"2023-10-08T06:53:49.375318+0000","last_undegraded":"2023-10-08T06:53:49.375318+0000","last_fullsized":"2023-10-08T06:53:49.375318+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:34:39.647914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374864+0000","last_change":"2023-10-08T06:53:47.865953+0000","last_active":"2023-10-08T06:53:49.374864+0000","last_peered":"2023-10-08T06:53:49.374864+0000","last_clean":"2023-10-08T06:53:49.374864+0000","last_became_active":"2023-10-08T06:53:47.865678+0000","last_became_peered":"2023-10-08T06:53:47.865678+0000","last_unstale":"2023-10-08T06:53:49.374864+0000","last_undegraded":"2023-10-08T06:53:49.374864+0000","last_fullsized":"2023-10-08T06:53:49.374864+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:37.391593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.879985+0000","last_change":"2023-10-08T06:53:47.896038+0000","last_active":"2023-10-08T06:53:48.879985+0000","last_peered":"2023-10-08T06:53:48.879985+0000","last_clean":"2023-10-08T06:53:48.879985+0000","last_became_active":"2023-10-08T06:53:47.895376+0000","last_became_peered":"2023-10-08T06:53:47.895376+0000","last_unstale":"2023-10-08T06:53:48.879985+0000","last_undegraded":"2023-10-08T06:53:48.879985+0000","last_fullsized":"2023-10-08T06:53:48.879985+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:13:08.813655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374091+0000","last_change":"2023-10-08T06:53:47.877806+0000","last_active":"2023-10-08T06:53:49.374091+0000","last_peered":"2023-10-08T06:53:49.374091+0000","last_clean":"2023-10-08T06:53:49.374091+0000","last_became_active":"2023-10-08T06:53:47.877381+0000","last_became_peered":"2023-10-08T06:53:47.877381+0000","last_unstale":"2023-10-08T06:53:49.374091+0000","last_undegraded":"2023-10-08T06:53:49.374091+0000","last_fullsized":"2023-10-08T06:53:49.374091+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:06.989627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.372632+0000","last_change":"2023-10-08T06:53:47.885965+0000","last_active":"2023-10-08T06:53:49.372632+0000","last_peered":"2023-10-08T06:53:49.372632+0000","last_clean":"2023-10-08T06:53:49.372632+0000","last_became_active":"2023-10-08T06:53:47.885448+0000","last_became_peered":"2023-10-08T06:53:47.885448+0000","last_unstale":"2023-10-08T06:53:49.372632+0000","last_undegraded":"2023-10-08T06:53:49.372632+0000","last_fullsized":"2023-10-08T06:53:49.372632+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:02.106148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.878775+0000","last_change":"2023-10-08T06:53:47.880561+0000","last_active":"2023-10-08T06:53:48.878775+0000","last_peered":"2023-10-08T06:53:48.878775+0000","last_clean":"2023-10-08T06:53:48.878775+0000","last_became_active":"2023-10-08T06:53:47.880267+0000","last_became_peered":"2023-10-08T06:53:47.880267+0000","last_unstale":"2023-10-08T06:53:48.878775+0000","last_undegraded":"2023-10-08T06:53:48.878775+0000","last_fullsized":"2023-10-08T06:53:48.878775+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:34.715994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373110+0000","last_change":"2023-10-08T06:53:47.891793+0000","last_active":"2023-10-08T06:53:49.373110+0000","last_peered":"2023-10-08T06:53:49.373110+0000","last_clean":"2023-10-08T06:53:49.373110+0000","last_became_active":"2023-10-08T06:53:47.891508+0000","last_became_peered":"2023-10-08T06:53:47.891508+0000","last_unstale":"2023-10-08T06:53:49.373110+0000","last_undegraded":"2023-10-08T06:53:49.373110+0000","last_fullsized":"2023-10-08T06:53:49.373110+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:26:43.527052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"184'4","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.892569+0000","last_change":"2023-10-08T06:53:46.838837+0000","last_active":"2023-10-08T06:53:47.892569+0000","last_peered":"2023-10-08T06:53:47.892569+0000","last_clean":"2023-10-08T06:53:47.892569+0000","last_became_active":"2023-10-08T06:53:46.838366+0000","last_became_peered":"2023-10-08T06:53:46.838366+0000","last_unstale":"2023-10-08T06:53:47.892569+0000","last_undegraded":"2023-10-08T06:53:47.892569+0000","last_fullsized":"2023-10-08T06:53:47.892569+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:16.809488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.376356+0000","last_change":"2023-10-08T06:53:47.879174+0000","last_active":"2023-10-08T06:53:49.376356+0000","last_peered":"2023-10-08T06:53:49.376356+0000","last_clean":"2023-10-08T06:53:49.376356+0000","last_became_active":"2023-10-08T06:53:47.878865+0000","last_became_peered":"2023-10-08T06:53:47.878865+0000","last_unstale":"2023-10-08T06:53:49.376356+0000","last_undegraded":"2023-10-08T06:53:49.376356+0000","last_fullsized":"2023-10-08T06:53:49.376356+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:11:06.570303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.892500+0000","last_change":"2023-10-08T06:53:46.854455+0000","last_active":"2023-10-08T06:53:47.892500+0000","last_peered":"2023-10-08T06:53:47.892500+0000","last_clean":"2023-10-08T06:53:47.892500+0000","last_became_active":"2023-10-08T06:53:46.854195+0000","last_became_peered":"2023-10-08T06:53:46.854195+0000","last_unstale":"2023-10-08T06:53:47.892500+0000","last_undegraded":"2023-10-08T06:53:47.892500+0000","last_fullsized":"2023-10-08T06:53:47.892500+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:27.010724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.880825+0000","last_change":"2023-10-08T06:53:47.888223+0000","last_active":"2023-10-08T06:53:48.880825+0000","last_peered":"2023-10-08T06:53:48.880825+0000","last_clean":"2023-10-08T06:53:48.880825+0000","last_became_active":"2023-10-08T06:53:47.887916+0000","last_became_peered":"2023-10-08T06:53:47.887916+0000","last_unstale":"2023-10-08T06:53:48.880825+0000","last_undegraded":"2023-10-08T06:53:48.880825+0000","last_fullsized":"2023-10-08T06:53:48.880825+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:51.272938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.891202+0000","last_change":"2023-10-08T06:53:46.853301+0000","last_active":"2023-10-08T06:53:47.891202+0000","last_peered":"2023-10-08T06:53:47.891202+0000","last_clean":"2023-10-08T06:53:47.891202+0000","last_became_active":"2023-10-08T06:53:46.853014+0000","last_became_peered":"2023-10-08T06:53:46.853014+0000","last_unstale":"2023-10-08T06:53:47.891202+0000","last_undegraded":"2023-10-08T06:53:47.891202+0000","last_fullsized":"2023-10-08T06:53:47.891202+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:01.090613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.951644+0000","last_change":"2023-10-08T06:53:47.887557+0000","last_active":"2023-10-08T06:53:48.951644+0000","last_peered":"2023-10-08T06:53:48.951644+0000","last_clean":"2023-10-08T06:53:48.951644+0000","last_became_active":"2023-10-08T06:53:47.887280+0000","last_became_peered":"2023-10-08T06:53:47.887280+0000","last_unstale":"2023-10-08T06:53:48.951644+0000","last_undegraded":"2023-10-08T06:53:48.951644+0000","last_fullsized":"2023-10-08T06:53:48.951644+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:07.878558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.882894+0000","last_change":"2023-10-08T06:53:46.854219+0000","last_active":"2023-10-08T06:53:47.882894+0000","last_peered":"2023-10-08T06:53:47.882894+0000","last_clean":"2023-10-08T06:53:47.882894+0000","last_became_active":"2023-10-08T06:53:46.853948+0000","last_became_peered":"2023-10-08T06:53:46.853948+0000","last_unstale":"2023-10-08T06:53:47.882894+0000","last_undegraded":"2023-10-08T06:53:47.882894+0000","last_fullsized":"2023-10-08T06:53:47.882894+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:56.207340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.874562+0000","last_change":"2023-10-08T06:53:47.873720+0000","last_active":"2023-10-08T06:53:48.874562+0000","last_peered":"2023-10-08T06:53:48.874562+0000","last_clean":"2023-10-08T06:53:48.874562+0000","last_became_active":"2023-10-08T06:53:47.873448+0000","last_became_peered":"2023-10-08T06:53:47.873448+0000","last_unstale":"2023-10-08T06:53:48.874562+0000","last_undegraded":"2023-10-08T06:53:48.874562+0000","last_fullsized":"2023-10-08T06:53:48.874562+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:04.799182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.882588+0000","last_change":"2023-10-08T06:53:46.847384+0000","last_active":"2023-10-08T06:53:47.882588+0000","last_peered":"2023-10-08T06:53:47.882588+0000","last_clean":"2023-10-08T06:53:47.882588+0000","last_became_active":"2023-10-08T06:53:46.847114+0000","last_became_peered":"2023-10-08T06:53:46.847114+0000","last_unstale":"2023-10-08T06:53:47.882588+0000","last_undegraded":"2023-10-08T06:53:47.882588+0000","last_fullsized":"2023-10-08T06:53:47.882588+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:54.188933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.881840+0000","last_change":"2023-10-08T06:53:47.875152+0000","last_active":"2023-10-08T06:53:48.881840+0000","last_peered":"2023-10-08T06:53:48.881840+0000","last_clean":"2023-10-08T06:53:48.881840+0000","last_became_active":"2023-10-08T06:53:47.874820+0000","last_became_peered":"2023-10-08T06:53:47.874820+0000","last_unstale":"2023-10-08T06:53:48.881840+0000","last_undegraded":"2023-10-08T06:53:48.881840+0000","last_fullsized":"2023-10-08T06:53:48.881840+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:58.222198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.891161+0000","last_change":"2023-10-08T06:53:46.842021+0000","last_active":"2023-10-08T06:53:47.891161+0000","last_peered":"2023-10-08T06:53:47.891161+0000","last_clean":"2023-10-08T06:53:47.891161+0000","last_became_active":"2023-10-08T06:53:46.841767+0000","last_became_peered":"2023-10-08T06:53:46.841767+0000","last_unstale":"2023-10-08T06:53:47.891161+0000","last_undegraded":"2023-10-08T06:53:47.891161+0000","last_fullsized":"2023-10-08T06:53:47.891161+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:27:53.231805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.372884+0000","last_change":"2023-10-08T06:53:47.868793+0000","last_active":"2023-10-08T06:53:49.372884+0000","last_peered":"2023-10-08T06:53:49.372884+0000","last_clean":"2023-10-08T06:53:49.372884+0000","last_became_active":"2023-10-08T06:53:47.868526+0000","last_became_peered":"2023-10-08T06:53:47.868526+0000","last_unstale":"2023-10-08T06:53:49.372884+0000","last_undegraded":"2023-10-08T06:53:49.372884+0000","last_fullsized":"2023-10-08T06:53:49.372884+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:58.882153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.853125+0000","last_change":"2023-10-08T06:53:46.845053+0000","last_active":"2023-10-08T06:53:47.853125+0000","last_peered":"2023-10-08T06:53:47.853125+0000","last_clean":"2023-10-08T06:53:47.853125+0000","last_became_active":"2023-10-08T06:53:46.844706+0000","last_became_peered":"2023-10-08T06:53:46.844706+0000","last_unstale":"2023-10-08T06:53:47.853125+0000","last_undegraded":"2023-10-08T06:53:47.853125+0000","last_fullsized":"2023-10-08T06:53:47.853125+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:51.981195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945333+0000","last_change":"2023-10-08T06:53:47.882250+0000","last_active":"2023-10-08T06:53:48.945333+0000","last_peered":"2023-10-08T06:53:48.945333+0000","last_clean":"2023-10-08T06:53:48.945333+0000","last_became_active":"2023-10-08T06:53:47.881919+0000","last_became_peered":"2023-10-08T06:53:47.881919+0000","last_unstale":"2023-10-08T06:53:48.945333+0000","last_undegraded":"2023-10-08T06:53:48.945333+0000","last_fullsized":"2023-10-08T06:53:48.945333+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:27:54.247340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.850586+0000","last_change":"2023-10-08T06:53:46.843744+0000","last_active":"2023-10-08T06:53:47.850586+0000","last_peered":"2023-10-08T06:53:47.850586+0000","last_clean":"2023-10-08T06:53:47.850586+0000","last_became_active":"2023-10-08T06:53:46.843504+0000","last_became_peered":"2023-10-08T06:53:46.843504+0000","last_unstale":"2023-10-08T06:53:47.850586+0000","last_undegraded":"2023-10-08T06:53:47.850586+0000","last_fullsized":"2023-10-08T06:53:47.850586+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:32.390630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.880259+0000","last_change":"2023-10-08T06:53:47.896027+0000","last_active":"2023-10-08T06:53:48.880259+0000","last_peered":"2023-10-08T06:53:48.880259+0000","last_clean":"2023-10-08T06:53:48.880259+0000","last_became_active":"2023-10-08T06:53:47.894974+0000","last_became_peered":"2023-10-08T06:53:47.894974+0000","last_unstale":"2023-10-08T06:53:48.880259+0000","last_undegraded":"2023-10-08T06:53:48.880259+0000","last_fullsized":"2023-10-08T06:53:48.880259+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:33.510351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.880131+0000","last_change":"2023-10-08T06:53:46.848518+0000","last_active":"2023-10-08T06:53:47.880131+0000","last_peered":"2023-10-08T06:53:47.880131+0000","last_clean":"2023-10-08T06:53:47.880131+0000","last_became_active":"2023-10-08T06:53:46.848237+0000","last_became_peered":"2023-10-08T06:53:46.848237+0000","last_unstale":"2023-10-08T06:53:47.880131+0000","last_undegraded":"2023-10-08T06:53:47.880131+0000","last_fullsized":"2023-10-08T06:53:47.880131+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:20.467886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375518+0000","last_change":"2023-10-08T06:53:47.881931+0000","last_active":"2023-10-08T06:53:49.375518+0000","last_peered":"2023-10-08T06:53:49.375518+0000","last_clean":"2023-10-08T06:53:49.375518+0000","last_became_active":"2023-10-08T06:53:47.881688+0000","last_became_peered":"2023-10-08T06:53:47.881688+0000","last_unstale":"2023-10-08T06:53:49.375518+0000","last_undegraded":"2023-10-08T06:53:49.375518+0000","last_fullsized":"2023-10-08T06:53:49.375518+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:33.930353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.856605+0000","last_change":"2023-10-08T06:53:46.844822+0000","last_active":"2023-10-08T06:53:47.856605+0000","last_peered":"2023-10-08T06:53:47.856605+0000","last_clean":"2023-10-08T06:53:47.856605+0000","last_became_active":"2023-10-08T06:53:46.844540+0000","last_became_peered":"2023-10-08T06:53:46.844540+0000","last_unstale":"2023-10-08T06:53:47.856605+0000","last_undegraded":"2023-10-08T06:53:47.856605+0000","last_fullsized":"2023-10-08T06:53:47.856605+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:32.390630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375998+0000","last_change":"2023-10-08T06:53:47.870838+0000","last_active":"2023-10-08T06:53:49.375998+0000","last_peered":"2023-10-08T06:53:49.375998+0000","last_clean":"2023-10-08T06:53:49.375998+0000","last_became_active":"2023-10-08T06:53:47.870553+0000","last_became_peered":"2023-10-08T06:53:47.870553+0000","last_unstale":"2023-10-08T06:53:49.375998+0000","last_undegraded":"2023-10-08T06:53:49.375998+0000","last_fullsized":"2023-10-08T06:53:49.375998+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:30.575901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"184'3","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.879422+0000","last_change":"2023-10-08T06:53:46.832905+0000","last_active":"2023-10-08T06:53:47.879422+0000","last_peered":"2023-10-08T06:53:47.879422+0000","last_clean":"2023-10-08T06:53:47.879422+0000","last_became_active":"2023-10-08T06:53:46.832624+0000","last_became_peered":"2023-10-08T06:53:46.832624+0000","last_unstale":"2023-10-08T06:53:47.879422+0000","last_undegraded":"2023-10-08T06:53:47.879422+0000","last_fullsized":"2023-10-08T06:53:47.879422+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:18:17.677806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.879155+0000","last_change":"2023-10-08T06:53:47.895198+0000","last_active":"2023-10-08T06:53:48.879155+0000","last_peered":"2023-10-08T06:53:48.879155+0000","last_clean":"2023-10-08T06:53:48.879155+0000","last_became_active":"2023-10-08T06:53:47.894749+0000","last_became_peered":"2023-10-08T06:53:47.894749+0000","last_unstale":"2023-10-08T06:53:48.879155+0000","last_undegraded":"2023-10-08T06:53:48.879155+0000","last_fullsized":"2023-10-08T06:53:48.879155+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:11.207254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.891718+0000","last_change":"2023-10-08T06:53:46.838423+0000","last_active":"2023-10-08T06:53:47.891718+0000","last_peered":"2023-10-08T06:53:47.891718+0000","last_clean":"2023-10-08T06:53:47.891718+0000","last_became_active":"2023-10-08T06:53:46.838071+0000","last_became_peered":"2023-10-08T06:53:46.838071+0000","last_unstale":"2023-10-08T06:53:47.891718+0000","last_undegraded":"2023-10-08T06:53:47.891718+0000","last_fullsized":"2023-10-08T06:53:47.891718+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:16.069696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375282+0000","last_change":"2023-10-08T06:53:47.873900+0000","last_active":"2023-10-08T06:53:49.375282+0000","last_peered":"2023-10-08T06:53:49.375282+0000","last_clean":"2023-10-08T06:53:49.375282+0000","last_became_active":"2023-10-08T06:53:47.873565+0000","last_became_peered":"2023-10-08T06:53:47.873565+0000","last_unstale":"2023-10-08T06:53:49.375282+0000","last_undegraded":"2023-10-08T06:53:49.375282+0000","last_fullsized":"2023-10-08T06:53:49.375282+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:44.391431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.859897+0000","last_change":"2023-10-08T06:53:46.857001+0000","last_active":"2023-10-08T06:53:47.859897+0000","last_peered":"2023-10-08T06:53:47.859897+0000","last_clean":"2023-10-08T06:53:47.859897+0000","last_became_active":"2023-10-08T06:53:46.856707+0000","last_became_peered":"2023-10-08T06:53:46.856707+0000","last_unstale":"2023-10-08T06:53:47.859897+0000","last_undegraded":"2023-10-08T06:53:47.859897+0000","last_fullsized":"2023-10-08T06:53:47.859897+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:05.134941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373221+0000","last_change":"2023-10-08T06:53:47.881369+0000","last_active":"2023-10-08T06:53:49.373221+0000","last_peered":"2023-10-08T06:53:49.373221+0000","last_clean":"2023-10-08T06:53:49.373221+0000","last_became_active":"2023-10-08T06:53:47.880888+0000","last_became_peered":"2023-10-08T06:53:47.880888+0000","last_unstale":"2023-10-08T06:53:49.373221+0000","last_undegraded":"2023-10-08T06:53:49.373221+0000","last_fullsized":"2023-10-08T06:53:49.373221+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:39.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.858439+0000","last_change":"2023-10-08T06:53:46.842178+0000","last_active":"2023-10-08T06:53:47.858439+0000","last_peered":"2023-10-08T06:53:47.858439+0000","last_clean":"2023-10-08T06:53:47.858439+0000","last_became_active":"2023-10-08T06:53:46.840731+0000","last_became_peered":"2023-10-08T06:53:46.840731+0000","last_unstale":"2023-10-08T06:53:47.858439+0000","last_undegraded":"2023-10-08T06:53:47.858439+0000","last_fullsized":"2023-10-08T06:53:47.858439+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:16.399937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373817+0000","last_change":"2023-10-08T06:53:47.887041+0000","last_active":"2023-10-08T06:53:49.373817+0000","last_peered":"2023-10-08T06:53:49.373817+0000","last_clean":"2023-10-08T06:53:49.373817+0000","last_became_active":"2023-10-08T06:53:47.886752+0000","last_became_peered":"2023-10-08T06:53:47.886752+0000","last_unstale":"2023-10-08T06:53:49.373817+0000","last_undegraded":"2023-10-08T06:53:49.373817+0000","last_fullsized":"2023-10-08T06:53:49.373817+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:01.337080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"184'8","reported_seq":20,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.890734+0000","last_change":"2023-10-08T06:53:46.853950+0000","last_active":"2023-10-08T06:53:47.890734+0000","last_peered":"2023-10-08T06:53:47.890734+0000","last_clean":"2023-10-08T06:53:47.890734+0000","last_became_active":"2023-10-08T06:53:46.853668+0000","last_became_peered":"2023-10-08T06:53:46.853668+0000","last_unstale":"2023-10-08T06:53:47.890734+0000","last_undegraded":"2023-10-08T06:53:47.890734+0000","last_fullsized":"2023-10-08T06:53:47.890734+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:26.339899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.951051+0000","last_change":"2023-10-08T06:53:47.895495+0000","last_active":"2023-10-08T06:53:48.951051+0000","last_peered":"2023-10-08T06:53:48.951051+0000","last_clean":"2023-10-08T06:53:48.951051+0000","last_became_active":"2023-10-08T06:53:47.895203+0000","last_became_peered":"2023-10-08T06:53:47.895203+0000","last_unstale":"2023-10-08T06:53:48.951051+0000","last_undegraded":"2023-10-08T06:53:48.951051+0000","last_fullsized":"2023-10-08T06:53:48.951051+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:18.313136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.889844+0000","last_change":"2023-10-08T06:53:46.849784+0000","last_active":"2023-10-08T06:53:47.889844+0000","last_peered":"2023-10-08T06:53:47.889844+0000","last_clean":"2023-10-08T06:53:47.889844+0000","last_became_active":"2023-10-08T06:53:46.849527+0000","last_became_peered":"2023-10-08T06:53:46.849527+0000","last_unstale":"2023-10-08T06:53:47.889844+0000","last_undegraded":"2023-10-08T06:53:47.889844+0000","last_fullsized":"2023-10-08T06:53:47.889844+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:24.077508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.951755+0000","last_change":"2023-10-08T06:53:47.897370+0000","last_active":"2023-10-08T06:53:48.951755+0000","last_peered":"2023-10-08T06:53:48.951755+0000","last_clean":"2023-10-08T06:53:48.951755+0000","last_became_active":"2023-10-08T06:53:47.897032+0000","last_became_peered":"2023-10-08T06:53:47.897032+0000","last_unstale":"2023-10-08T06:53:48.951755+0000","last_undegraded":"2023-10-08T06:53:48.951755+0000","last_fullsized":"2023-10-08T06:53:48.951755+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:58.859856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.894323+0000","last_change":"2023-10-08T06:53:46.849864+0000","last_active":"2023-10-08T06:53:47.894323+0000","last_peered":"2023-10-08T06:53:47.894323+0000","last_clean":"2023-10-08T06:53:47.894323+0000","last_became_active":"2023-10-08T06:53:46.849576+0000","last_became_peered":"2023-10-08T06:53:46.849576+0000","last_unstale":"2023-10-08T06:53:47.894323+0000","last_undegraded":"2023-10-08T06:53:47.894323+0000","last_fullsized":"2023-10-08T06:53:47.894323+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:16.809488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945947+0000","last_change":"2023-10-08T06:53:47.869652+0000","last_active":"2023-10-08T06:53:48.945947+0000","last_peered":"2023-10-08T06:53:48.945947+0000","last_clean":"2023-10-08T06:53:48.945947+0000","last_became_active":"2023-10-08T06:53:47.869012+0000","last_became_peered":"2023-10-08T06:53:47.869012+0000","last_unstale":"2023-10-08T06:53:48.945947+0000","last_undegraded":"2023-10-08T06:53:48.945947+0000","last_fullsized":"2023-10-08T06:53:48.945947+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:10.444749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.877913+0000","last_change":"2023-10-08T06:53:46.846147+0000","last_active":"2023-10-08T06:53:47.877913+0000","last_peered":"2023-10-08T06:53:47.877913+0000","last_clean":"2023-10-08T06:53:47.877913+0000","last_became_active":"2023-10-08T06:53:46.845910+0000","last_became_peered":"2023-10-08T06:53:46.845910+0000","last_unstale":"2023-10-08T06:53:47.877913+0000","last_undegraded":"2023-10-08T06:53:47.877913+0000","last_fullsized":"2023-10-08T06:53:47.877913+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:16:57.726805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.946161+0000","last_change":"2023-10-08T06:53:47.873795+0000","last_active":"2023-10-08T06:53:48.946161+0000","last_peered":"2023-10-08T06:53:48.946161+0000","last_clean":"2023-10-08T06:53:48.946161+0000","last_became_active":"2023-10-08T06:53:47.873455+0000","last_became_peered":"2023-10-08T06:53:47.873455+0000","last_unstale":"2023-10-08T06:53:48.946161+0000","last_undegraded":"2023-10-08T06:53:48.946161+0000","last_fullsized":"2023-10-08T06:53:48.946161+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:28.026259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.861962+0000","last_change":"2023-10-08T06:53:46.857013+0000","last_active":"2023-10-08T06:53:47.861962+0000","last_peered":"2023-10-08T06:53:47.861962+0000","last_clean":"2023-10-08T06:53:47.861962+0000","last_became_active":"2023-10-08T06:53:46.856762+0000","last_became_peered":"2023-10-08T06:53:46.856762+0000","last_unstale":"2023-10-08T06:53:47.861962+0000","last_undegraded":"2023-10-08T06:53:47.861962+0000","last_fullsized":"2023-10-08T06:53:47.861962+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:22.842337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.950711+0000","last_change":"2023-10-08T06:53:47.892327+0000","last_active":"2023-10-08T06:53:48.950711+0000","last_peered":"2023-10-08T06:53:48.950711+0000","last_clean":"2023-10-08T06:53:48.950711+0000","last_became_active":"2023-10-08T06:53:47.891986+0000","last_became_peered":"2023-10-08T06:53:47.891986+0000","last_unstale":"2023-10-08T06:53:48.950711+0000","last_undegraded":"2023-10-08T06:53:48.950711+0000","last_fullsized":"2023-10-08T06:53:48.950711+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:39.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.889254+0000","last_change":"2023-10-08T06:53:46.854554+0000","last_active":"2023-10-08T06:53:47.889254+0000","last_peered":"2023-10-08T06:53:47.889254+0000","last_clean":"2023-10-08T06:53:47.889254+0000","last_became_active":"2023-10-08T06:53:46.854235+0000","last_became_peered":"2023-10-08T06:53:46.854235+0000","last_unstale":"2023-10-08T06:53:47.889254+0000","last_undegraded":"2023-10-08T06:53:47.889254+0000","last_fullsized":"2023-10-08T06:53:47.889254+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:49.451557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375609+0000","last_change":"2023-10-08T06:53:47.870266+0000","last_active":"2023-10-08T06:53:49.375609+0000","last_peered":"2023-10-08T06:53:49.375609+0000","last_clean":"2023-10-08T06:53:49.375609+0000","last_became_active":"2023-10-08T06:53:47.869947+0000","last_became_peered":"2023-10-08T06:53:47.869947+0000","last_unstale":"2023-10-08T06:53:49.375609+0000","last_undegraded":"2023-10-08T06:53:49.375609+0000","last_fullsized":"2023-10-08T06:53:49.375609+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:21:32.856518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.871923+0000","last_change":"2023-10-08T06:53:46.848822+0000","last_active":"2023-10-08T06:53:47.871923+0000","last_peered":"2023-10-08T06:53:47.871923+0000","last_clean":"2023-10-08T06:53:47.871923+0000","last_became_active":"2023-10-08T06:53:46.848489+0000","last_became_peered":"2023-10-08T06:53:46.848489+0000","last_unstale":"2023-10-08T06:53:47.871923+0000","last_undegraded":"2023-10-08T06:53:47.871923+0000","last_fullsized":"2023-10-08T06:53:47.871923+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:25.902796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.950805+0000","last_change":"2023-10-08T06:53:47.887357+0000","last_active":"2023-10-08T06:53:48.950805+0000","last_peered":"2023-10-08T06:53:48.950805+0000","last_clean":"2023-10-08T06:53:48.950805+0000","last_became_active":"2023-10-08T06:53:47.887075+0000","last_became_peered":"2023-10-08T06:53:47.887075+0000","last_unstale":"2023-10-08T06:53:48.950805+0000","last_undegraded":"2023-10-08T06:53:48.950805+0000","last_fullsized":"2023-10-08T06:53:48.950805+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:56.220725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.855681+0000","last_change":"2023-10-08T06:53:46.853408+0000","last_active":"2023-10-08T06:53:47.855681+0000","last_peered":"2023-10-08T06:53:47.855681+0000","last_clean":"2023-10-08T06:53:47.855681+0000","last_became_active":"2023-10-08T06:53:46.853121+0000","last_became_peered":"2023-10-08T06:53:46.853121+0000","last_unstale":"2023-10-08T06:53:47.855681+0000","last_undegraded":"2023-10-08T06:53:47.855681+0000","last_fullsized":"2023-10-08T06:53:47.855681+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:37.692117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374152+0000","last_change":"2023-10-08T06:53:47.872247+0000","last_active":"2023-10-08T06:53:49.374152+0000","last_peered":"2023-10-08T06:53:49.374152+0000","last_clean":"2023-10-08T06:53:49.374152+0000","last_became_active":"2023-10-08T06:53:47.871935+0000","last_became_peered":"2023-10-08T06:53:47.871935+0000","last_unstale":"2023-10-08T06:53:49.374152+0000","last_undegraded":"2023-10-08T06:53:49.374152+0000","last_fullsized":"2023-10-08T06:53:49.374152+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:47.050032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.878143+0000","last_change":"2023-10-08T06:53:46.841902+0000","last_active":"2023-10-08T06:53:47.878143+0000","last_peered":"2023-10-08T06:53:47.878143+0000","last_clean":"2023-10-08T06:53:47.878143+0000","last_became_active":"2023-10-08T06:53:46.841528+0000","last_became_peered":"2023-10-08T06:53:46.841528+0000","last_unstale":"2023-10-08T06:53:47.878143+0000","last_undegraded":"2023-10-08T06:53:47.878143+0000","last_fullsized":"2023-10-08T06:53:47.878143+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:02:28.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374279+0000","last_change":"2023-10-08T06:53:47.867621+0000","last_active":"2023-10-08T06:53:49.374279+0000","last_peered":"2023-10-08T06:53:49.374279+0000","last_clean":"2023-10-08T06:53:49.374279+0000","last_became_active":"2023-10-08T06:53:47.867320+0000","last_became_peered":"2023-10-08T06:53:47.867320+0000","last_unstale":"2023-10-08T06:53:49.374279+0000","last_undegraded":"2023-10-08T06:53:49.374279+0000","last_fullsized":"2023-10-08T06:53:49.374279+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:07.878558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.888868+0000","last_change":"2023-10-08T06:53:46.856348+0000","last_active":"2023-10-08T06:53:47.888868+0000","last_peered":"2023-10-08T06:53:47.888868+0000","last_clean":"2023-10-08T06:53:47.888868+0000","last_became_active":"2023-10-08T06:53:46.856059+0000","last_became_peered":"2023-10-08T06:53:46.856059+0000","last_unstale":"2023-10-08T06:53:47.888868+0000","last_undegraded":"2023-10-08T06:53:47.888868+0000","last_fullsized":"2023-10-08T06:53:47.888868+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:25.375848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.883244+0000","last_change":"2023-10-08T06:53:47.885694+0000","last_active":"2023-10-08T06:53:48.883244+0000","last_peered":"2023-10-08T06:53:48.883244+0000","last_clean":"2023-10-08T06:53:48.883244+0000","last_became_active":"2023-10-08T06:53:47.885290+0000","last_became_peered":"2023-10-08T06:53:47.885290+0000","last_unstale":"2023-10-08T06:53:48.883244+0000","last_undegraded":"2023-10-08T06:53:48.883244+0000","last_fullsized":"2023-10-08T06:53:48.883244+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:52.930923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"185'1","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.946350+0000","last_change":"2023-10-08T06:53:46.851826+0000","last_active":"2023-10-08T06:53:48.946350+0000","last_peered":"2023-10-08T06:53:48.946350+0000","last_clean":"2023-10-08T06:53:48.946350+0000","last_became_active":"2023-10-08T06:53:46.851307+0000","last_became_peered":"2023-10-08T06:53:46.851307+0000","last_unstale":"2023-10-08T06:53:48.946350+0000","last_undegraded":"2023-10-08T06:53:48.946350+0000","last_fullsized":"2023-10-08T06:53:48.946350+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:36.369411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.946198+0000","last_change":"2023-10-08T06:53:47.886731+0000","last_active":"2023-10-08T06:53:48.946198+0000","last_peered":"2023-10-08T06:53:48.946198+0000","last_clean":"2023-10-08T06:53:48.946198+0000","last_became_active":"2023-10-08T06:53:47.886285+0000","last_became_peered":"2023-10-08T06:53:47.886285+0000","last_unstale":"2023-10-08T06:53:48.946198+0000","last_undegraded":"2023-10-08T06:53:48.946198+0000","last_fullsized":"2023-10-08T06:53:48.946198+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:17.825023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.865492+0000","last_change":"2023-10-08T06:53:46.854168+0000","last_active":"2023-10-08T06:53:47.865492+0000","last_peered":"2023-10-08T06:53:47.865492+0000","last_clean":"2023-10-08T06:53:47.865492+0000","last_became_active":"2023-10-08T06:53:46.853876+0000","last_became_peered":"2023-10-08T06:53:46.853876+0000","last_unstale":"2023-10-08T06:53:47.865492+0000","last_undegraded":"2023-10-08T06:53:47.865492+0000","last_fullsized":"2023-10-08T06:53:47.865492+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:47.384330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374760+0000","last_change":"2023-10-08T06:53:47.881235+0000","last_active":"2023-10-08T06:53:49.374760+0000","last_peered":"2023-10-08T06:53:49.374760+0000","last_clean":"2023-10-08T06:53:49.374760+0000","last_became_active":"2023-10-08T06:53:47.880974+0000","last_became_peered":"2023-10-08T06:53:47.880974+0000","last_unstale":"2023-10-08T06:53:49.374760+0000","last_undegraded":"2023-10-08T06:53:49.374760+0000","last_fullsized":"2023-10-08T06:53:49.374760+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:52.819577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.850661+0000","last_change":"2023-10-08T06:53:46.854126+0000","last_active":"2023-10-08T06:53:47.850661+0000","last_peered":"2023-10-08T06:53:47.850661+0000","last_clean":"2023-10-08T06:53:47.850661+0000","last_became_active":"2023-10-08T06:53:46.853833+0000","last_became_peered":"2023-10-08T06:53:46.853833+0000","last_unstale":"2023-10-08T06:53:47.850661+0000","last_undegraded":"2023-10-08T06:53:47.850661+0000","last_fullsized":"2023-10-08T06:53:47.850661+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:36.369411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.879159+0000","last_change":"2023-10-08T06:53:47.880003+0000","last_active":"2023-10-08T06:53:48.879159+0000","last_peered":"2023-10-08T06:53:48.879159+0000","last_clean":"2023-10-08T06:53:48.879159+0000","last_became_active":"2023-10-08T06:53:47.878993+0000","last_became_peered":"2023-10-08T06:53:47.878993+0000","last_unstale":"2023-10-08T06:53:48.879159+0000","last_undegraded":"2023-10-08T06:53:48.879159+0000","last_fullsized":"2023-10-08T06:53:48.879159+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:13.329398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.857483+0000","last_change":"2023-10-08T06:53:46.851411+0000","last_active":"2023-10-08T06:53:47.857483+0000","last_peered":"2023-10-08T06:53:47.857483+0000","last_clean":"2023-10-08T06:53:47.857483+0000","last_became_active":"2023-10-08T06:53:46.850930+0000","last_became_peered":"2023-10-08T06:53:46.850930+0000","last_unstale":"2023-10-08T06:53:47.857483+0000","last_undegraded":"2023-10-08T06:53:47.857483+0000","last_fullsized":"2023-10-08T06:53:47.857483+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:21:37.089110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.882479+0000","last_change":"2023-10-08T06:53:47.884876+0000","last_active":"2023-10-08T06:53:48.882479+0000","last_peered":"2023-10-08T06:53:48.882479+0000","last_clean":"2023-10-08T06:53:48.882479+0000","last_became_active":"2023-10-08T06:53:47.884593+0000","last_became_peered":"2023-10-08T06:53:47.884593+0000","last_unstale":"2023-10-08T06:53:48.882479+0000","last_undegraded":"2023-10-08T06:53:48.882479+0000","last_fullsized":"2023-10-08T06:53:48.882479+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:42.890785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.859410+0000","last_change":"2023-10-08T06:53:46.852038+0000","last_active":"2023-10-08T06:53:47.859410+0000","last_peered":"2023-10-08T06:53:47.859410+0000","last_clean":"2023-10-08T06:53:47.859410+0000","last_became_active":"2023-10-08T06:53:46.851840+0000","last_became_peered":"2023-10-08T06:53:46.851840+0000","last_unstale":"2023-10-08T06:53:47.859410+0000","last_undegraded":"2023-10-08T06:53:47.859410+0000","last_fullsized":"2023-10-08T06:53:47.859410+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:46.289951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373257+0000","last_change":"2023-10-08T06:53:47.882562+0000","last_active":"2023-10-08T06:53:49.373257+0000","last_peered":"2023-10-08T06:53:49.373257+0000","last_clean":"2023-10-08T06:53:49.373257+0000","last_became_active":"2023-10-08T06:53:47.882264+0000","last_became_peered":"2023-10-08T06:53:47.882264+0000","last_unstale":"2023-10-08T06:53:49.373257+0000","last_undegraded":"2023-10-08T06:53:49.373257+0000","last_fullsized":"2023-10-08T06:53:49.373257+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:49:03.345084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.853339+0000","last_change":"2023-10-08T06:53:46.844152+0000","last_active":"2023-10-08T06:53:47.853339+0000","last_peered":"2023-10-08T06:53:47.853339+0000","last_clean":"2023-10-08T06:53:47.853339+0000","last_became_active":"2023-10-08T06:53:46.843835+0000","last_became_peered":"2023-10-08T06:53:46.843835+0000","last_unstale":"2023-10-08T06:53:47.853339+0000","last_undegraded":"2023-10-08T06:53:47.853339+0000","last_fullsized":"2023-10-08T06:53:47.853339+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:30.734197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.879976+0000","last_change":"2023-10-08T06:53:47.896315+0000","last_active":"2023-10-08T06:53:48.879976+0000","last_peered":"2023-10-08T06:53:48.879976+0000","last_clean":"2023-10-08T06:53:48.879976+0000","last_became_active":"2023-10-08T06:53:47.895734+0000","last_became_peered":"2023-10-08T06:53:47.895734+0000","last_unstale":"2023-10-08T06:53:48.879976+0000","last_undegraded":"2023-10-08T06:53:48.879976+0000","last_fullsized":"2023-10-08T06:53:48.879976+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:29.614443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.866069+0000","last_change":"2023-10-08T06:53:46.846292+0000","last_active":"2023-10-08T06:53:47.866069+0000","last_peered":"2023-10-08T06:53:47.866069+0000","last_clean":"2023-10-08T06:53:47.866069+0000","last_became_active":"2023-10-08T06:53:46.845869+0000","last_became_peered":"2023-10-08T06:53:46.845869+0000","last_unstale":"2023-10-08T06:53:47.866069+0000","last_undegraded":"2023-10-08T06:53:47.866069+0000","last_fullsized":"2023-10-08T06:53:47.866069+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:59.318995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.880760+0000","last_change":"2023-10-08T06:53:47.893207+0000","last_active":"2023-10-08T06:53:48.880760+0000","last_peered":"2023-10-08T06:53:48.880760+0000","last_clean":"2023-10-08T06:53:48.880760+0000","last_became_active":"2023-10-08T06:53:47.892915+0000","last_became_peered":"2023-10-08T06:53:47.892915+0000","last_unstale":"2023-10-08T06:53:48.880760+0000","last_undegraded":"2023-10-08T06:53:48.880760+0000","last_fullsized":"2023-10-08T06:53:48.880760+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:20.197322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"184'6","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.894168+0000","last_change":"2023-10-08T06:53:46.853823+0000","last_active":"2023-10-08T06:53:47.894168+0000","last_peered":"2023-10-08T06:53:47.894168+0000","last_clean":"2023-10-08T06:53:47.894168+0000","last_became_active":"2023-10-08T06:53:46.853347+0000","last_became_peered":"2023-10-08T06:53:46.853347+0000","last_unstale":"2023-10-08T06:53:47.894168+0000","last_undegraded":"2023-10-08T06:53:47.894168+0000","last_fullsized":"2023-10-08T06:53:47.894168+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:28.030370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"172'2729","reported_seq":3396,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.866242+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T06:53:47.866242+0000","last_peered":"2023-10-08T06:53:47.866242+0000","last_clean":"2023-10-08T06:53:47.866242+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T06:53:47.866242+0000","last_undegraded":"2023-10-08T06:53:47.866242+0000","last_fullsized":"2023-10-08T06:53:47.866242+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2729,"log_dups_size":0,"ondisk_log_size":2729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575776,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60396,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311444,"statfs":{"total":96636764160,"available":96574918656,"internally_reserved":0,"allocated":2908160,"data_stored":1779799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.21,"15min":1.2330000000000001},"min":{"1min":0.91900000000000004,"5min":0.78700000000000003,"15min":0.70399999999999996},"max":{"1min":1.6950000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.365},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.224,"15min":1.248},"min":{"1min":0.93799999999999994,"5min":0.747,"15min":0.72899999999999998},"max":{"1min":1.522,"5min":1.7150000000000001,"15min":1.849},"last":0.91700000000000004}]},{"osd":1,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2829999999999999,"15min":1.262},"min":{"1min":0.89200000000000002,"5min":0.84299999999999997,"15min":0.70399999999999996},"max":{"1min":1.6970000000000001,"5min":1.712,"15min":1.8480000000000001},"last":1.163},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.284,"15min":1.3220000000000001},"min":{"1min":0.95999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7450000000000001,"5min":1.9299999999999999,"15min":2.4809999999999999},"last":1.2330000000000001}]},{"osd":2,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.0740000000000001,"5min":0.90900000000000003,"15min":0.81899999999999995},"max":{"1min":1.627,"5min":2.7040000000000002,"15min":3.4350000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.381,"15min":1.3740000000000001},"min":{"1min":0.95699999999999996,"5min":0.88900000000000001,"15min":0.83199999999999996},"max":{"1min":1.873,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":1.48}]},{"osd":3,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.339,"15min":1.335},"min":{"1min":1.0189999999999999,"5min":0.97799999999999998,"15min":0.85899999999999999},"max":{"1min":1.5609999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.46,"15min":1.419},"min":{"1min":1.147,"5min":0.98899999999999999,"15min":0.83099999999999996},"max":{"1min":2.0379999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.4339999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.333,"15min":1.2869999999999999},"min":{"1min":0.93999999999999995,"5min":0.83199999999999996,"15min":0.66200000000000003},"max":{"1min":1.952,"5min":1.952,"15min":3.2839999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.456,"15min":1.391},"min":{"1min":1.1579999999999999,"5min":0.90300000000000002,"15min":0.80800000000000005},"max":{"1min":1.708,"5min":2.004,"15min":2.1909999999999998},"last":1.3260000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.474,"15min":1.4399999999999999},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":0.86499999999999999},"max":{"1min":2.1499999999999999,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":1.534},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.504,"15min":1.4790000000000001},"min":{"1min":1.2909999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.139}]},{"osd":6,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.5680000000000001,"15min":1.556},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":0.94599999999999995},"max":{"1min":2.1970000000000001,"5min":2.4350000000000001,"15min":3.1360000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5269999999999999,"15min":1.502},"min":{"1min":1.0049999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":2.0190000000000001,"5min":2.1309999999999998,"15min":2.3140000000000001},"last":1.4510000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575775,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41768,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330072,"statfs":{"total":96636764160,"available":96593993728,"internally_reserved":0,"allocated":2314240,"data_stored":1189479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3009999999999999,"15min":1.3480000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7310000000000001,"5min":1.792,"15min":2.508},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3089999999999999,"15min":1.337},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.609,"5min":1.8280000000000001,"15min":2.1139999999999999},"last":0.93400000000000005}]},{"osd":1,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2170000000000001,"15min":1.248},"min":{"1min":0.876,"5min":0.876,"15min":0.59699999999999998},"max":{"1min":1.605,"5min":1.821,"15min":1.919},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.224,"5min":1.2769999999999999,"15min":1.2629999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.53800000000000003},"max":{"1min":1.7769999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.272}]},{"osd":2,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2170000000000001,"15min":1.2290000000000001},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.75600000000000001},"max":{"1min":1.6499999999999999,"5min":1.722,"15min":1.8919999999999999},"last":1.143},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.29,"15min":1.2989999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5229999999999999,"5min":1.6379999999999999,"15min":2.3839999999999999},"last":0.98799999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4259999999999999,"15min":1.4039999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.83399999999999996},"max":{"1min":1.8700000000000001,"5min":1.976,"15min":2.1499999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.478,"15min":1.48},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.89400000000000002},"max":{"1min":1.8180000000000001,"5min":1.915,"15min":2.0289999999999999},"last":1.2370000000000001}]},{"osd":4,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.3580000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.72399999999999998},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.792},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4379999999999999,"15min":1.4390000000000001},"min":{"1min":0.878,"5min":0.878,"15min":0.80200000000000005},"max":{"1min":1.9099999999999999,"5min":1.9910000000000001,"15min":2.0089999999999999},"last":1.3009999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.431,"15min":1.4139999999999999},"min":{"1min":0.997,"5min":0.94799999999999995,"15min":0.69499999999999995},"max":{"1min":1.798,"5min":1.893,"15min":2.7309999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.5589999999999999,"15min":1.548},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78300000000000003},"max":{"1min":2.0640000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.867}]},{"osd":7,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.591,"15min":1.579},"min":{"1min":1.0960000000000001,"5min":1.0680000000000001,"15min":0.92000000000000004},"max":{"1min":1.984,"5min":2.0710000000000002,"15min":2.2090000000000001},"last":1.351},{"interface":"front","average":{"1min":1.488,"5min":1.573,"15min":1.5660000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.82099999999999995},"max":{"1min":1.946,"5min":2.222,"15min":2.6469999999999998},"last":1.194}]}]},{"osd":5,"up_from":13,"seq":55834575775,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59804,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312036,"statfs":{"total":96636764160,"available":96575524864,"internally_reserved":0,"allocated":2310144,"data_stored":1189409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2310000000000001,"15min":1.218},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.66600000000000004},"max":{"1min":1.5740000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.296,"15min":1.2569999999999999},"min":{"1min":0.95399999999999996,"5min":0.82699999999999996,"15min":0.78800000000000003},"max":{"1min":1.629,"5min":1.897,"15min":2.4700000000000002},"last":1.1719999999999999}]},{"osd":1,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.26,"15min":1.2230000000000001},"min":{"1min":0.95499999999999996,"5min":0.85799999999999998,"15min":0.72499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.371},{"interface":"front","average":{"1min":1.349,"5min":1.375,"15min":1.323},"min":{"1min":0.90400000000000003,"5min":0.88700000000000001,"15min":0.78200000000000003},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.0800000000000001}]},{"osd":2,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3480000000000001,"15min":1.3280000000000001},"min":{"1min":0.98399999999999999,"5min":0.85199999999999998,"15min":0.83199999999999996},"max":{"1min":1.6479999999999999,"5min":1.9359999999999999,"15min":2.351},"last":1.329},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.3160000000000001},"min":{"1min":0.91000000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.667,"5min":1.7809999999999999,"15min":1.964},"last":1.242}]},{"osd":3,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3959999999999999,"15min":1.3540000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.528,"15min":1.464},"min":{"1min":1.139,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.9910000000000001,"5min":3.0059999999999998,"15min":5.1630000000000003},"last":1.2769999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3320000000000001,"15min":1.3240000000000001},"min":{"1min":0.96299999999999997,"5min":0.89300000000000002,"15min":0.81499999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4039999999999999},"last":1.006},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4379999999999999,"15min":1.4239999999999999},"min":{"1min":0.96899999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.403}]},{"osd":6,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.4990000000000001,"15min":1.4750000000000001},"min":{"1min":1.248,"5min":0.96399999999999997,"15min":0.81200000000000006},"max":{"1min":2.1150000000000002,"5min":2.1659999999999999,"15min":5.5019999999999998},"last":1.554},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4930000000000001,"15min":1.48},"min":{"1min":1.0509999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.0139999999999998,"5min":2.1040000000000001,"15min":2.2890000000000001},"last":1.1319999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.5660000000000001,"15min":1.5629999999999999},"min":{"1min":1.3400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":5.8780000000000001},"last":1.298},{"interface":"front","average":{"1min":1.679,"5min":1.6839999999999999,"15min":1.6299999999999999},"min":{"1min":1.415,"5min":1.1719999999999999,"15min":0.97099999999999997},"max":{"1min":2.206,"5min":2.6789999999999998,"15min":5.399},"last":1.4570000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575775,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41768,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330072,"statfs":{"total":96636764160,"available":96593993728,"internally_reserved":0,"allocated":2322432,"data_stored":1191021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.389,"15min":1.3460000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":1.198},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.3999999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.69799999999999995},"max":{"1min":1.7789999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.786}]},{"osd":1,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2749999999999999,"15min":1.28},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.73,"5min":1.784,"15min":1.9339999999999999},"last":1.133},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.64000000000000001},"max":{"1min":1.613,"5min":1.6870000000000001,"15min":1.7290000000000001},"last":1.2589999999999999}]},{"osd":2,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.2889999999999999,"15min":1.248},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.3759999999999999},"min":{"1min":1,"5min":0.93100000000000005,"15min":0.85399999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.7130000000000001}]},{"osd":3,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.339,"15min":1.3109999999999999},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.385},{"interface":"front","average":{"1min":1.55,"5min":1.4670000000000001,"15min":1.4099999999999999},"min":{"1min":1.0629999999999999,"5min":0.97299999999999998,"15min":0.93400000000000005},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.3260000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.4139999999999999,"15min":1.351},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.66700000000000004},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.593},{"interface":"front","average":{"1min":1.526,"5min":1.4450000000000001,"15min":1.375},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.749},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.5029999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.5329999999999999,"15min":1.425},"min":{"1min":0.97899999999999998,"5min":0.91100000000000003,"15min":0.72699999999999998},"max":{"1min":5.3339999999999996,"5min":5.3339999999999996,"15min":5.3339999999999996},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6120000000000001,"15min":1.546},"min":{"1min":1.212,"5min":0.96699999999999997,"15min":0.91400000000000003},"max":{"1min":2.1059999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":1.8500000000000001}]},{"osd":7,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.548,"15min":1.49},"min":{"1min":1.202,"5min":1.135,"15min":0.99099999999999999},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.784,"5min":1.6859999999999999,"15min":1.6060000000000001},"min":{"1min":1.0049999999999999,"5min":0.98999999999999999,"15min":0.93600000000000005},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":1.546}]}]},{"osd":3,"up_from":13,"seq":55834575777,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47336,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324504,"statfs":{"total":96636764160,"available":96588292096,"internally_reserved":0,"allocated":2318336,"data_stored":1190025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.95499999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.44500000000000001},"max":{"1min":1.3200000000000001,"5min":1.71,"15min":1.71},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.196,"15min":1.175},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6719999999999999,"5min":1.851,"15min":2.137},"last":0.79400000000000004}]},{"osd":1,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.024,"15min":1.018},"min":{"1min":0.70999999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2370000000000001,"5min":1.71,"15min":1.823},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.2989999999999999,"15min":1.3049999999999999},"min":{"1min":0.91000000000000003,"5min":0.872,"15min":0.77000000000000002},"max":{"1min":1.51,"5min":1.75,"15min":1.893},"last":1.248}]},{"osd":2,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1739999999999999,"15min":1.137},"min":{"1min":0.85299999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6279999999999999,"5min":1.726,"15min":1.7410000000000001},"last":0.746},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1919999999999999,"15min":1.1930000000000001},"min":{"1min":0.90400000000000003,"5min":0.68600000000000005,"15min":0.61399999999999999},"max":{"1min":1.589,"5min":1.7809999999999999,"15min":1.8360000000000001},"last":0.85299999999999998}]},{"osd":4,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.284,"15min":1.2649999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83299999999999996},"max":{"1min":1.8560000000000001,"5min":1.97,"15min":1.97},"last":1.141},{"interface":"front","average":{"1min":1.482,"5min":1.4650000000000001,"15min":1.452},"min":{"1min":1.1859999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.8100000000000001,"5min":1.905,"15min":2.1960000000000002},"last":1.327}]},{"osd":5,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.391,"15min":1.385},"min":{"1min":1.085,"5min":0.98399999999999999,"15min":0.88},"max":{"1min":1.6399999999999999,"5min":2.27,"15min":3.0009999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.415,"15min":1.421},"min":{"1min":1.1890000000000001,"5min":0.995,"15min":0.94099999999999995},"max":{"1min":1.6739999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.091}]},{"osd":6,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4059999999999999,"15min":1.3959999999999999},"min":{"1min":0.98399999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.016},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.534,"15min":1.518},"min":{"1min":1.2170000000000001,"5min":1.1639999999999999,"15min":0.99099999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.544},"last":1.45}]},{"osd":7,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.5649999999999999,"15min":1.5660000000000001},"min":{"1min":1.3069999999999999,"5min":1.1839999999999999,"15min":1.008},"max":{"1min":1.8939999999999999,"5min":2.008,"15min":2.347},"last":1.3},{"interface":"front","average":{"1min":1.55,"5min":1.5049999999999999,"15min":1.4790000000000001},"min":{"1min":1.0980000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1800000000000002},"last":1.5}]}]},{"osd":2,"up_from":13,"seq":55834575775,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2318336,"data_stored":1189521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.93700000000000006},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.4650000000000001,"15min":1.657},"last":1.167},{"interface":"front","average":{"1min":1.036,"5min":1.0549999999999999,"15min":1.0760000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5249999999999999,"5min":1.661,"15min":2.2679999999999998},"last":1.1379999999999999}]},{"osd":1,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.004,"15min":1.0069999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.696,"5min":1.7170000000000001,"15min":1.996},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.137,"5min":1.1679999999999999,"15min":1.1819999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":2.1139999999999999},"last":1.077}]},{"osd":3,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2270000000000001,"15min":1.2230000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.772,"5min":1.925,"15min":2.3929999999999998},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.256,"15min":1.258},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.46,"5min":1.804,"15min":1.891},"last":1.2290000000000001}]},{"osd":4,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4330000000000001,"15min":1.444},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.859,"5min":1.9430000000000001,"15min":3.0939999999999999},"last":1.29},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.3160000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7470000000000001,"5min":1.7549999999999999,"15min":3.7320000000000002},"last":1.3089999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.429,"15min":1.425},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":3.3519999999999999},"last":1.827},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.5029999999999999,"15min":1.528},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.9810000000000001,"5min":2.2719999999999998,"15min":2.5249999999999999},"last":1.3580000000000001}]},{"osd":6,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.673,"5min":1.9990000000000001,"15min":2.206},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.548,"15min":1.532},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9610000000000001,"5min":1.966,"15min":3.02},"last":1.964}]},{"osd":7,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.51,"15min":1.492},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.861,"5min":2.0790000000000002,"15min":4.2869999999999999},"last":1.782},{"interface":"front","average":{"1min":1.55,"5min":1.5940000000000001,"15min":1.579},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8540000000000001,"5min":2.0099999999999998,"15min":3.0760000000000001},"last":1.915}]}]},{"osd":1,"up_from":13,"seq":55834575775,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29284,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342556,"statfs":{"total":96636764160,"available":96606777344,"internally_reserved":0,"allocated":2314240,"data_stored":1189479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.0349999999999999,"15min":1.0029999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.601,"5min":4.601,"15min":4.601},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2190000000000001,"15min":1.22},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.1639999999999999}]},{"osd":2,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.0329999999999999,"15min":1.0229999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.3179999999999996,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.385,"5min":1.2250000000000001,"15min":1.1679999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":5.4059999999999997,"5min":5.4059999999999997,"15min":5.4059999999999997},"last":1.2230000000000001}]},{"osd":3,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.3029999999999999,"15min":1.2609999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.5220000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":1.325},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1579999999999999,"15min":1.1160000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.052}]},{"osd":4,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.3919999999999999,"15min":1.3680000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":4.2409999999999997,"5min":4.2409999999999997,"15min":4.2409999999999997},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.337,"15min":1.298},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":1.637}]},{"osd":5,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.4279999999999999,"15min":1.3600000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":4.7530000000000001,"5min":4.7530000000000001,"15min":4.7530000000000001},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.742,"5min":1.5980000000000001,"15min":1.5329999999999999},"min":{"1min":1.03,"5min":1.03,"15min":0.75},"max":{"1min":5.7389999999999999,"5min":5.7389999999999999,"15min":5.7389999999999999},"last":1.48}]},{"osd":6,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.5700000000000001,"15min":1.53},"min":{"1min":1.05,"5min":1.05,"15min":1.02},"max":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":1.429},{"interface":"front","average":{"1min":1.661,"5min":1.5329999999999999,"15min":1.4810000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":1.2529999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.605,"15min":1.5489999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":5.6289999999999996,"5min":5.6289999999999996,"15min":5.6289999999999996},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.607,"15min":1.5549999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.5270000000000001,"5min":4.5270000000000001,"15min":4.5270000000000001},"last":1.5780000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575776,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38200,"kb_used_data":2860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333640,"statfs":{"total":96636764160,"available":96597647360,"internally_reserved":0,"allocated":2928640,"data_stored":1782095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5669999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":1.272},{"interface":"front","average":{"1min":1.173,"5min":1.123,"15min":1.091},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.222,"15min":1.163},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.226,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0580000000000001,"15min":1.0800000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.2669999999999999,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.224}]},{"osd":3,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.294,"15min":1.27},"min":{"1min":0.625,"5min":0.625,"15min":0.621},"max":{"1min":2.8250000000000002,"5min":4.0220000000000002,"15min":4.0220000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.2110000000000001,"15min":1.1839999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.94499999999999995}]},{"osd":4,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.349,"15min":1.3680000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.353},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4199999999999999,"15min":1.411},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.319}]},{"osd":5,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4330000000000001,"15min":1.4019999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.68700000000000006},"max":{"1min":2.0659999999999998,"5min":2.0920000000000001,"15min":2.2480000000000002},"last":1.534},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5249999999999999,"15min":1.478},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":3.2229999999999999,"5min":3.887,"15min":3.887},"last":1.0629999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.504,"15min":1.476},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.115},{"interface":"front","average":{"1min":1.488,"5min":1.514,"15min":1.508},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.748},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.482}]},{"osd":7,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.546,"15min":1.5369999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.86199999999999999},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.573,"15min":1.548},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.194,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.1970000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T06:53:57.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T06:53:57.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.562+0000 7ff93da25640 1 Processor -- start 2023-10-08T06:53:57.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.562+0000 7ff93da25640 1 -- start start 2023-10-08T06:53:57.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.564+0000 7ff93da25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff93815cca0 0x7ff9381614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:57.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.564+0000 7ff93da25640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff9380b07d0 0x7ff9380c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:57.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.564+0000 7ff93da25640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9380c9020 con 0x7ff9380cab40 2023-10-08T06:53:57.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.564+0000 7ff93da25640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380ccc20 con 0x7ff9380b07d0 2023-10-08T06:53:57.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.564+0000 7ff93da25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380c92e0 con 0x7ff93815cca0 2023-10-08T06:53:57.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.565+0000 7ff93ca23640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff93815cca0 0x7ff9381614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:57.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.565+0000 7ff93ca23640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff93815cca0 0x7ff9381614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:34554/0 (socket says 172.21.15.70:34554) 2023-10-08T06:53:57.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.565+0000 7ff93ca23640 1 -- 172.21.15.70:0/4124793484 learned_addr learned my addr 172.21.15.70:0/4124793484 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:57.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.565+0000 7ff93d224640 1 --2- 172.21.15.70:0/4124793484 >> v2:172.21.15.181:3300/0 conn(0x7ff9380b07d0 0x7ff9380c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:57.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.565+0000 7ff937fff640 1 --1- 172.21.15.70:0/4124793484 >> v1:172.21.15.181:6789/0 conn(0x7ff9380cab40 0x7ff9380b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:42204/0 (socket says 172.21.15.70:42204) 2023-10-08T06:53:57.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.566+0000 7ff93d224640 1 -- 172.21.15.70:0/4124793484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff93815cca0 msgr2=0x7ff9381614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:57.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.566+0000 7ff93d224640 1 --2- 172.21.15.70:0/4124793484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff93815cca0 0x7ff9381614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:57.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.566+0000 7ff93d224640 1 -- 172.21.15.70:0/4124793484 >> v1:172.21.15.181:6789/0 conn(0x7ff9380cab40 legacy=0x7ff9380b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:57.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.566+0000 7ff93d224640 1 -- 172.21.15.70:0/4124793484 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380cc7b0 con 0x7ff9380b07d0 2023-10-08T06:53:57.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.566+0000 7ff93ca23640 1 --2- 172.21.15.70:0/4124793484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff93815cca0 0x7ff9381614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T06:53:57.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.567+0000 7ff93d224640 1 --2- 172.21.15.70:0/4124793484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380b07d0 0x7ff9380c9770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff9300097a0 tx=0x7ff93002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36b7fae2f6916a0b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:57.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.568+0000 7ff9377fe640 1 -- 172.21.15.70:0/4124793484 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9300316f0 con 0x7ff9380b07d0 2023-10-08T06:53:57.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.568+0000 7ff9377fe640 1 -- 172.21.15.70:0/4124793484 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff930031850 con 0x7ff9380b07d0 2023-10-08T06:53:57.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.568+0000 7ff9377fe640 1 -- 172.21.15.70:0/4124793484 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930031be0 con 0x7ff9380b07d0 2023-10-08T06:53:57.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.569+0000 7ff93da25640 1 -- 172.21.15.70:0/4124793484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380b07d0 msgr2=0x7ff9380c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:57.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.569+0000 7ff93da25640 1 --2- 172.21.15.70:0/4124793484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380b07d0 0x7ff9380c9770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff9300097a0 tx=0x7ff93002b790 comp rx=0 tx=0).stop 2023-10-08T06:53:57.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.569+0000 7ff93da25640 1 -- 172.21.15.70:0/4124793484 shutdown_connections 2023-10-08T06:53:57.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.569+0000 7ff93da25640 1 --2- 172.21.15.70:0/4124793484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380b07d0 0x7ff9380c9770 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:57.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.569+0000 7ff93da25640 1 --2- 172.21.15.70:0/4124793484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff93815cca0 0x7ff9381614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:57.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.569+0000 7ff93da25640 1 -- 172.21.15.70:0/4124793484 >> 172.21.15.70:0/4124793484 conn(0x7ff9380cb0c0 msgr2=0x7ff9380cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:57.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.569+0000 7ff93da25640 1 -- 172.21.15.70:0/4124793484 shutdown_connections 2023-10-08T06:53:57.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.569+0000 7ff93da25640 1 -- 172.21.15.70:0/4124793484 wait complete. 2023-10-08T06:53:57.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.571+0000 7ff93da25640 1 Processor -- start 2023-10-08T06:53:57.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.572+0000 7ff93da25640 1 -- start start 2023-10-08T06:53:57.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.573+0000 7ff93da25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9380b07d0 0x7ff9381f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.573+0000 7ff93da25640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380cab40 0x7ff93815d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.573+0000 7ff93da25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff93815cca0 0x7ff93815f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.573+0000 7ff93da25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93815da10 con 0x7ff93815cca0 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.573+0000 7ff93da25640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93815e220 con 0x7ff9380cab40 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.573+0000 7ff93da25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93815dc60 con 0x7ff9380b07d0 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.573+0000 7ff93d224640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff93815cca0 0x7ff93815f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.573+0000 7ff93d224640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff93815cca0 0x7ff93815f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39178/0 (socket says 172.21.15.70:39178) 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.573+0000 7ff93d224640 1 -- 172.21.15.70:0/2574784943 learned_addr learned my addr 172.21.15.70:0/2574784943 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.573+0000 7ff93ca23640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9380b07d0 0x7ff9381f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff937fff640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380cab40 0x7ff93815d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff93d224640 1 -- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9380b07d0 msgr2=0x7ff9381f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff93d224640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9380b07d0 0x7ff9381f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff93d224640 1 -- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380cab40 msgr2=0x7ff93815d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff93d224640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380cab40 0x7ff93815d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff93d224640 1 -- 172.21.15.70:0/2574784943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9280094d0 con 0x7ff93815cca0 2023-10-08T06:53:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff937fff640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380cab40 0x7ff93815d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff93d224640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff93815cca0 0x7ff93815f9e0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff9300097a0 tx=0x7ff9300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff9357fa640 1 -- 172.21.15.70:0/2574784943 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930034680 con 0x7ff93815cca0 2023-10-08T06:53:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff9357fa640 1 -- 172.21.15.70:0/2574784943 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9300347e0 con 0x7ff93815cca0 2023-10-08T06:53:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.574+0000 7ff9357fa640 1 -- 172.21.15.70:0/2574784943 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930034b70 con 0x7ff93815cca0 2023-10-08T06:53:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.575+0000 7ff93da25640 1 -- 172.21.15.70:0/2574784943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff930009400 con 0x7ff93815cca0 2023-10-08T06:53:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.575+0000 7ff93da25640 1 -- 172.21.15.70:0/2574784943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9380ac2c0 con 0x7ff93815cca0 2023-10-08T06:53:57.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.576+0000 7ff93da25640 1 -- 172.21.15.70:0/2574784943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9380ac120 con 0x7ff93815cca0 2023-10-08T06:53:57.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.577+0000 7ff9357fa640 1 -- 172.21.15.70:0/2574784943 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff93003b040 con 0x7ff93815cca0 2023-10-08T06:53:57.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.578+0000 7ff9357fa640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff91806af90 0x7ff91806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:57.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.578+0000 7ff9357fa640 1 -- 172.21.15.70:0/2574784943 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7ff9300aae70 con 0x7ff93815cca0 2023-10-08T06:53:57.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.578+0000 7ff93ca23640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff91806af90 0x7ff91806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:57.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.580+0000 7ff93ca23640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff91806af90 0x7ff91806d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9380c7f70 tx=0x7ff928008040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:57.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.582+0000 7ff9357fa640 1 -- 172.21.15.70:0/2574784943 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9380ac120 con 0x7ff93815cca0 2023-10-08T06:53:57.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.767+0000 7ff93da25640 1 -- 172.21.15.70:0/2574784943 --> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9380cd150 con 0x7ff91806af90 2023-10-08T06:53:57.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.789+0000 7ff9357fa640 1 -- 172.21.15.70:0/2574784943 <== mgr.9345 v2:172.21.15.181:6832/4245513204 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287399 (secure 0 0 0) 0x7ff928011e90 con 0x7ff91806af90 2023-10-08T06:53:57.793 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:57.795 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T06:53:57.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.795+0000 7ff93da25640 1 -- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff91806af90 msgr2=0x7ff91806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:57.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.795+0000 7ff93da25640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff91806af90 0x7ff91806d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff9380c7f70 tx=0x7ff928008040 comp rx=0 tx=0).stop 2023-10-08T06:53:57.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.795+0000 7ff93da25640 1 -- 172.21.15.70:0/2574784943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff93815cca0 msgr2=0x7ff93815f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.795+0000 7ff93da25640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff93815cca0 0x7ff93815f9e0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff9300097a0 tx=0x7ff9300046c0 comp rx=0 tx=0).stop 2023-10-08T06:53:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.796+0000 7ff93da25640 1 -- 172.21.15.70:0/2574784943 shutdown_connections 2023-10-08T06:53:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.796+0000 7ff93da25640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff91806af90 0x7ff91806d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.796+0000 7ff93da25640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff93815cca0 0x7ff93815f9e0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.796+0000 7ff93da25640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9380cab40 0x7ff93815d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.796+0000 7ff93da25640 1 --2- 172.21.15.70:0/2574784943 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9380b07d0 0x7ff9381f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.796+0000 7ff93da25640 1 -- 172.21.15.70:0/2574784943 >> 172.21.15.70:0/2574784943 conn(0x7ff9380cb0c0 msgr2=0x7ff9380700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.796+0000 7ff93da25640 1 -- 172.21.15.70:0/2574784943 shutdown_connections 2023-10-08T06:53:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:57.796+0000 7ff93da25640 1 -- 172.21.15.70:0/2574784943 wait complete. 2023-10-08T06:53:57.815 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-10-08T06:53:56.080018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":494,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2768,"ondisk_log_size":2768,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365888,"kb_used_data":19272,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754608832,"statfs":{"total":773094113280,"available":772719443968,"internally_reserved":0,"allocated":19734528,"data_stored":10700828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":354799163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.232776"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945912+0000","last_change":"2023-10-08T06:53:47.883681+0000","last_active":"2023-10-08T06:53:48.945912+0000","last_peered":"2023-10-08T06:53:48.945912+0000","last_clean":"2023-10-08T06:53:48.945912+0000","last_became_active":"2023-10-08T06:53:47.883436+0000","last_became_peered":"2023-10-08T06:53:47.883436+0000","last_unstale":"2023-10-08T06:53:48.945912+0000","last_undegraded":"2023-10-08T06:53:48.945912+0000","last_fullsized":"2023-10-08T06:53:48.945912+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:00.334530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.376169+0000","last_change":"2023-10-08T06:53:47.878367+0000","last_active":"2023-10-08T06:53:49.376169+0000","last_peered":"2023-10-08T06:53:49.376169+0000","last_clean":"2023-10-08T06:53:49.376169+0000","last_became_active":"2023-10-08T06:53:47.878109+0000","last_became_peered":"2023-10-08T06:53:47.878109+0000","last_unstale":"2023-10-08T06:53:49.376169+0000","last_undegraded":"2023-10-08T06:53:49.376169+0000","last_fullsized":"2023-10-08T06:53:49.376169+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:54.584120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.376327+0000","last_change":"2023-10-08T06:53:47.891864+0000","last_active":"2023-10-08T06:53:49.376327+0000","last_peered":"2023-10-08T06:53:49.376327+0000","last_clean":"2023-10-08T06:53:49.376327+0000","last_became_active":"2023-10-08T06:53:47.891571+0000","last_became_peered":"2023-10-08T06:53:47.891571+0000","last_unstale":"2023-10-08T06:53:49.376327+0000","last_undegraded":"2023-10-08T06:53:49.376327+0000","last_fullsized":"2023-10-08T06:53:49.376327+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:49:48.869017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.882355+0000","last_change":"2023-10-08T06:53:47.878270+0000","last_active":"2023-10-08T06:53:48.882355+0000","last_peered":"2023-10-08T06:53:48.882355+0000","last_clean":"2023-10-08T06:53:48.882355+0000","last_became_active":"2023-10-08T06:53:47.877994+0000","last_became_peered":"2023-10-08T06:53:47.877994+0000","last_unstale":"2023-10-08T06:53:48.882355+0000","last_undegraded":"2023-10-08T06:53:48.882355+0000","last_fullsized":"2023-10-08T06:53:48.882355+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:47:01.826263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374504+0000","last_change":"2023-10-08T06:53:47.866342+0000","last_active":"2023-10-08T06:53:49.374504+0000","last_peered":"2023-10-08T06:53:49.374504+0000","last_clean":"2023-10-08T06:53:49.374504+0000","last_became_active":"2023-10-08T06:53:47.866085+0000","last_became_peered":"2023-10-08T06:53:47.866085+0000","last_unstale":"2023-10-08T06:53:49.374504+0000","last_undegraded":"2023-10-08T06:53:49.374504+0000","last_fullsized":"2023-10-08T06:53:49.374504+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:58.859856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373127+0000","last_change":"2023-10-08T06:53:47.869713+0000","last_active":"2023-10-08T06:53:49.373127+0000","last_peered":"2023-10-08T06:53:49.373127+0000","last_clean":"2023-10-08T06:53:49.373127+0000","last_became_active":"2023-10-08T06:53:47.869401+0000","last_became_peered":"2023-10-08T06:53:47.869401+0000","last_unstale":"2023-10-08T06:53:49.373127+0000","last_undegraded":"2023-10-08T06:53:49.373127+0000","last_fullsized":"2023-10-08T06:53:49.373127+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:34.484753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945506+0000","last_change":"2023-10-08T06:53:47.873438+0000","last_active":"2023-10-08T06:53:48.945506+0000","last_peered":"2023-10-08T06:53:48.945506+0000","last_clean":"2023-10-08T06:53:48.945506+0000","last_became_active":"2023-10-08T06:53:47.873134+0000","last_became_peered":"2023-10-08T06:53:47.873134+0000","last_unstale":"2023-10-08T06:53:48.945506+0000","last_undegraded":"2023-10-08T06:53:48.945506+0000","last_fullsized":"2023-10-08T06:53:48.945506+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:59.922242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.878729+0000","last_change":"2023-10-08T06:53:47.881710+0000","last_active":"2023-10-08T06:53:48.878729+0000","last_peered":"2023-10-08T06:53:48.878729+0000","last_clean":"2023-10-08T06:53:48.878729+0000","last_became_active":"2023-10-08T06:53:47.881426+0000","last_became_peered":"2023-10-08T06:53:47.881426+0000","last_unstale":"2023-10-08T06:53:48.878729+0000","last_undegraded":"2023-10-08T06:53:48.878729+0000","last_fullsized":"2023-10-08T06:53:48.878729+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:58:08.231929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374374+0000","last_change":"2023-10-08T06:53:47.873331+0000","last_active":"2023-10-08T06:53:49.374374+0000","last_peered":"2023-10-08T06:53:49.374374+0000","last_clean":"2023-10-08T06:53:49.374374+0000","last_became_active":"2023-10-08T06:53:47.873070+0000","last_became_peered":"2023-10-08T06:53:47.873070+0000","last_unstale":"2023-10-08T06:53:49.374374+0000","last_undegraded":"2023-10-08T06:53:49.374374+0000","last_fullsized":"2023-10-08T06:53:49.374374+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:02.408761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375140+0000","last_change":"2023-10-08T06:53:47.864912+0000","last_active":"2023-10-08T06:53:49.375140+0000","last_peered":"2023-10-08T06:53:49.375140+0000","last_clean":"2023-10-08T06:53:49.375140+0000","last_became_active":"2023-10-08T06:53:47.864597+0000","last_became_peered":"2023-10-08T06:53:47.864597+0000","last_unstale":"2023-10-08T06:53:49.375140+0000","last_undegraded":"2023-10-08T06:53:49.375140+0000","last_fullsized":"2023-10-08T06:53:49.375140+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:23.125163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375231+0000","last_change":"2023-10-08T06:53:47.885735+0000","last_active":"2023-10-08T06:53:49.375231+0000","last_peered":"2023-10-08T06:53:49.375231+0000","last_clean":"2023-10-08T06:53:49.375231+0000","last_became_active":"2023-10-08T06:53:47.885476+0000","last_became_peered":"2023-10-08T06:53:47.885476+0000","last_unstale":"2023-10-08T06:53:49.375231+0000","last_undegraded":"2023-10-08T06:53:49.375231+0000","last_fullsized":"2023-10-08T06:53:49.375231+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:29.267997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.882686+0000","last_change":"2023-10-08T06:53:47.886577+0000","last_active":"2023-10-08T06:53:48.882686+0000","last_peered":"2023-10-08T06:53:48.882686+0000","last_clean":"2023-10-08T06:53:48.882686+0000","last_became_active":"2023-10-08T06:53:47.886312+0000","last_became_peered":"2023-10-08T06:53:47.886312+0000","last_unstale":"2023-10-08T06:53:48.882686+0000","last_undegraded":"2023-10-08T06:53:48.882686+0000","last_fullsized":"2023-10-08T06:53:48.882686+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:14:46.948507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373799+0000","last_change":"2023-10-08T06:53:47.864171+0000","last_active":"2023-10-08T06:53:49.373799+0000","last_peered":"2023-10-08T06:53:49.373799+0000","last_clean":"2023-10-08T06:53:49.373799+0000","last_became_active":"2023-10-08T06:53:47.863902+0000","last_became_peered":"2023-10-08T06:53:47.863902+0000","last_unstale":"2023-10-08T06:53:49.373799+0000","last_undegraded":"2023-10-08T06:53:49.373799+0000","last_fullsized":"2023-10-08T06:53:49.373799+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:56.220725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.882164+0000","last_change":"2023-10-08T06:53:47.885670+0000","last_active":"2023-10-08T06:53:48.882164+0000","last_peered":"2023-10-08T06:53:48.882164+0000","last_clean":"2023-10-08T06:53:48.882164+0000","last_became_active":"2023-10-08T06:53:47.885356+0000","last_became_peered":"2023-10-08T06:53:47.885356+0000","last_unstale":"2023-10-08T06:53:48.882164+0000","last_undegraded":"2023-10-08T06:53:48.882164+0000","last_fullsized":"2023-10-08T06:53:48.882164+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:35:08.460367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374955+0000","last_change":"2023-10-08T06:53:47.874415+0000","last_active":"2023-10-08T06:53:49.374955+0000","last_peered":"2023-10-08T06:53:49.374955+0000","last_clean":"2023-10-08T06:53:49.374955+0000","last_became_active":"2023-10-08T06:53:47.874165+0000","last_became_peered":"2023-10-08T06:53:47.874165+0000","last_unstale":"2023-10-08T06:53:49.374955+0000","last_undegraded":"2023-10-08T06:53:49.374955+0000","last_fullsized":"2023-10-08T06:53:49.374955+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:53.752457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.372528+0000","last_change":"2023-10-08T06:53:47.888368+0000","last_active":"2023-10-08T06:53:49.372528+0000","last_peered":"2023-10-08T06:53:49.372528+0000","last_clean":"2023-10-08T06:53:49.372528+0000","last_became_active":"2023-10-08T06:53:47.888109+0000","last_became_peered":"2023-10-08T06:53:47.888109+0000","last_unstale":"2023-10-08T06:53:49.372528+0000","last_undegraded":"2023-10-08T06:53:49.372528+0000","last_fullsized":"2023-10-08T06:53:49.372528+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:21.483421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.882720+0000","last_change":"2023-10-08T06:53:47.871573+0000","last_active":"2023-10-08T06:53:48.882720+0000","last_peered":"2023-10-08T06:53:48.882720+0000","last_clean":"2023-10-08T06:53:48.882720+0000","last_became_active":"2023-10-08T06:53:47.871281+0000","last_became_peered":"2023-10-08T06:53:47.871281+0000","last_unstale":"2023-10-08T06:53:48.882720+0000","last_undegraded":"2023-10-08T06:53:48.882720+0000","last_fullsized":"2023-10-08T06:53:48.882720+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:49:41.935880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375935+0000","last_change":"2023-10-08T06:53:47.871440+0000","last_active":"2023-10-08T06:53:49.375935+0000","last_peered":"2023-10-08T06:53:49.375935+0000","last_clean":"2023-10-08T06:53:49.375935+0000","last_became_active":"2023-10-08T06:53:47.871180+0000","last_became_peered":"2023-10-08T06:53:47.871180+0000","last_unstale":"2023-10-08T06:53:49.375935+0000","last_undegraded":"2023-10-08T06:53:49.375935+0000","last_fullsized":"2023-10-08T06:53:49.375935+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:44.026824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.951695+0000","last_change":"2023-10-08T06:53:47.897749+0000","last_active":"2023-10-08T06:53:48.951695+0000","last_peered":"2023-10-08T06:53:48.951695+0000","last_clean":"2023-10-08T06:53:48.951695+0000","last_became_active":"2023-10-08T06:53:47.896996+0000","last_became_peered":"2023-10-08T06:53:47.896996+0000","last_unstale":"2023-10-08T06:53:48.951695+0000","last_undegraded":"2023-10-08T06:53:48.951695+0000","last_fullsized":"2023-10-08T06:53:48.951695+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:02.408761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.875036+0000","last_change":"2023-10-08T06:53:47.870451+0000","last_active":"2023-10-08T06:53:48.875036+0000","last_peered":"2023-10-08T06:53:48.875036+0000","last_clean":"2023-10-08T06:53:48.875036+0000","last_became_active":"2023-10-08T06:53:47.870037+0000","last_became_peered":"2023-10-08T06:53:47.870037+0000","last_unstale":"2023-10-08T06:53:48.875036+0000","last_undegraded":"2023-10-08T06:53:48.875036+0000","last_fullsized":"2023-10-08T06:53:48.875036+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:23:40.923291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374137+0000","last_change":"2023-10-08T06:53:47.883382+0000","last_active":"2023-10-08T06:53:49.374137+0000","last_peered":"2023-10-08T06:53:49.374137+0000","last_clean":"2023-10-08T06:53:49.374137+0000","last_became_active":"2023-10-08T06:53:47.881355+0000","last_became_peered":"2023-10-08T06:53:47.881355+0000","last_unstale":"2023-10-08T06:53:49.374137+0000","last_undegraded":"2023-10-08T06:53:49.374137+0000","last_fullsized":"2023-10-08T06:53:49.374137+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:18.313136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945304+0000","last_change":"2023-10-08T06:53:47.881877+0000","last_active":"2023-10-08T06:53:48.945304+0000","last_peered":"2023-10-08T06:53:48.945304+0000","last_clean":"2023-10-08T06:53:48.945304+0000","last_became_active":"2023-10-08T06:53:47.881588+0000","last_became_peered":"2023-10-08T06:53:47.881588+0000","last_unstale":"2023-10-08T06:53:48.945304+0000","last_undegraded":"2023-10-08T06:53:48.945304+0000","last_fullsized":"2023-10-08T06:53:48.945304+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:53.402284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.879132+0000","last_change":"2023-10-08T06:53:47.884817+0000","last_active":"2023-10-08T06:53:48.879132+0000","last_peered":"2023-10-08T06:53:48.879132+0000","last_clean":"2023-10-08T06:53:48.879132+0000","last_became_active":"2023-10-08T06:53:47.884540+0000","last_became_peered":"2023-10-08T06:53:47.884540+0000","last_unstale":"2023-10-08T06:53:48.879132+0000","last_undegraded":"2023-10-08T06:53:48.879132+0000","last_fullsized":"2023-10-08T06:53:48.879132+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:29.117651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373895+0000","last_change":"2023-10-08T06:53:47.872966+0000","last_active":"2023-10-08T06:53:49.373895+0000","last_peered":"2023-10-08T06:53:49.373895+0000","last_clean":"2023-10-08T06:53:49.373895+0000","last_became_active":"2023-10-08T06:53:47.872549+0000","last_became_peered":"2023-10-08T06:53:47.872549+0000","last_unstale":"2023-10-08T06:53:49.373895+0000","last_undegraded":"2023-10-08T06:53:49.373895+0000","last_fullsized":"2023-10-08T06:53:49.373895+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:44.415095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.946128+0000","last_change":"2023-10-08T06:53:47.864951+0000","last_active":"2023-10-08T06:53:48.946128+0000","last_peered":"2023-10-08T06:53:48.946128+0000","last_clean":"2023-10-08T06:53:48.946128+0000","last_became_active":"2023-10-08T06:53:47.863680+0000","last_became_peered":"2023-10-08T06:53:47.863680+0000","last_unstale":"2023-10-08T06:53:48.946128+0000","last_undegraded":"2023-10-08T06:53:48.946128+0000","last_fullsized":"2023-10-08T06:53:48.946128+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:21:56.583934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945989+0000","last_change":"2023-10-08T06:53:47.886562+0000","last_active":"2023-10-08T06:53:48.945989+0000","last_peered":"2023-10-08T06:53:48.945989+0000","last_clean":"2023-10-08T06:53:48.945989+0000","last_became_active":"2023-10-08T06:53:47.876729+0000","last_became_peered":"2023-10-08T06:53:47.876729+0000","last_unstale":"2023-10-08T06:53:48.945989+0000","last_undegraded":"2023-10-08T06:53:48.945989+0000","last_fullsized":"2023-10-08T06:53:48.945989+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:17.085231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375318+0000","last_change":"2023-10-08T06:53:47.890896+0000","last_active":"2023-10-08T06:53:49.375318+0000","last_peered":"2023-10-08T06:53:49.375318+0000","last_clean":"2023-10-08T06:53:49.375318+0000","last_became_active":"2023-10-08T06:53:47.890597+0000","last_became_peered":"2023-10-08T06:53:47.890597+0000","last_unstale":"2023-10-08T06:53:49.375318+0000","last_undegraded":"2023-10-08T06:53:49.375318+0000","last_fullsized":"2023-10-08T06:53:49.375318+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:34:39.647914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374864+0000","last_change":"2023-10-08T06:53:47.865953+0000","last_active":"2023-10-08T06:53:49.374864+0000","last_peered":"2023-10-08T06:53:49.374864+0000","last_clean":"2023-10-08T06:53:49.374864+0000","last_became_active":"2023-10-08T06:53:47.865678+0000","last_became_peered":"2023-10-08T06:53:47.865678+0000","last_unstale":"2023-10-08T06:53:49.374864+0000","last_undegraded":"2023-10-08T06:53:49.374864+0000","last_fullsized":"2023-10-08T06:53:49.374864+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:37.391593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.879985+0000","last_change":"2023-10-08T06:53:47.896038+0000","last_active":"2023-10-08T06:53:48.879985+0000","last_peered":"2023-10-08T06:53:48.879985+0000","last_clean":"2023-10-08T06:53:48.879985+0000","last_became_active":"2023-10-08T06:53:47.895376+0000","last_became_peered":"2023-10-08T06:53:47.895376+0000","last_unstale":"2023-10-08T06:53:48.879985+0000","last_undegraded":"2023-10-08T06:53:48.879985+0000","last_fullsized":"2023-10-08T06:53:48.879985+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:13:08.813655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374091+0000","last_change":"2023-10-08T06:53:47.877806+0000","last_active":"2023-10-08T06:53:49.374091+0000","last_peered":"2023-10-08T06:53:49.374091+0000","last_clean":"2023-10-08T06:53:49.374091+0000","last_became_active":"2023-10-08T06:53:47.877381+0000","last_became_peered":"2023-10-08T06:53:47.877381+0000","last_unstale":"2023-10-08T06:53:49.374091+0000","last_undegraded":"2023-10-08T06:53:49.374091+0000","last_fullsized":"2023-10-08T06:53:49.374091+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:06.989627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.372632+0000","last_change":"2023-10-08T06:53:47.885965+0000","last_active":"2023-10-08T06:53:49.372632+0000","last_peered":"2023-10-08T06:53:49.372632+0000","last_clean":"2023-10-08T06:53:49.372632+0000","last_became_active":"2023-10-08T06:53:47.885448+0000","last_became_peered":"2023-10-08T06:53:47.885448+0000","last_unstale":"2023-10-08T06:53:49.372632+0000","last_undegraded":"2023-10-08T06:53:49.372632+0000","last_fullsized":"2023-10-08T06:53:49.372632+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:02.106148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.878775+0000","last_change":"2023-10-08T06:53:47.880561+0000","last_active":"2023-10-08T06:53:48.878775+0000","last_peered":"2023-10-08T06:53:48.878775+0000","last_clean":"2023-10-08T06:53:48.878775+0000","last_became_active":"2023-10-08T06:53:47.880267+0000","last_became_peered":"2023-10-08T06:53:47.880267+0000","last_unstale":"2023-10-08T06:53:48.878775+0000","last_undegraded":"2023-10-08T06:53:48.878775+0000","last_fullsized":"2023-10-08T06:53:48.878775+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:34.715994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373110+0000","last_change":"2023-10-08T06:53:47.891793+0000","last_active":"2023-10-08T06:53:49.373110+0000","last_peered":"2023-10-08T06:53:49.373110+0000","last_clean":"2023-10-08T06:53:49.373110+0000","last_became_active":"2023-10-08T06:53:47.891508+0000","last_became_peered":"2023-10-08T06:53:47.891508+0000","last_unstale":"2023-10-08T06:53:49.373110+0000","last_undegraded":"2023-10-08T06:53:49.373110+0000","last_fullsized":"2023-10-08T06:53:49.373110+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:26:43.527052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"184'4","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.892569+0000","last_change":"2023-10-08T06:53:46.838837+0000","last_active":"2023-10-08T06:53:47.892569+0000","last_peered":"2023-10-08T06:53:47.892569+0000","last_clean":"2023-10-08T06:53:47.892569+0000","last_became_active":"2023-10-08T06:53:46.838366+0000","last_became_peered":"2023-10-08T06:53:46.838366+0000","last_unstale":"2023-10-08T06:53:47.892569+0000","last_undegraded":"2023-10-08T06:53:47.892569+0000","last_fullsized":"2023-10-08T06:53:47.892569+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:16.809488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.376356+0000","last_change":"2023-10-08T06:53:47.879174+0000","last_active":"2023-10-08T06:53:49.376356+0000","last_peered":"2023-10-08T06:53:49.376356+0000","last_clean":"2023-10-08T06:53:49.376356+0000","last_became_active":"2023-10-08T06:53:47.878865+0000","last_became_peered":"2023-10-08T06:53:47.878865+0000","last_unstale":"2023-10-08T06:53:49.376356+0000","last_undegraded":"2023-10-08T06:53:49.376356+0000","last_fullsized":"2023-10-08T06:53:49.376356+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:11:06.570303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.892500+0000","last_change":"2023-10-08T06:53:46.854455+0000","last_active":"2023-10-08T06:53:47.892500+0000","last_peered":"2023-10-08T06:53:47.892500+0000","last_clean":"2023-10-08T06:53:47.892500+0000","last_became_active":"2023-10-08T06:53:46.854195+0000","last_became_peered":"2023-10-08T06:53:46.854195+0000","last_unstale":"2023-10-08T06:53:47.892500+0000","last_undegraded":"2023-10-08T06:53:47.892500+0000","last_fullsized":"2023-10-08T06:53:47.892500+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:27.010724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.880825+0000","last_change":"2023-10-08T06:53:47.888223+0000","last_active":"2023-10-08T06:53:48.880825+0000","last_peered":"2023-10-08T06:53:48.880825+0000","last_clean":"2023-10-08T06:53:48.880825+0000","last_became_active":"2023-10-08T06:53:47.887916+0000","last_became_peered":"2023-10-08T06:53:47.887916+0000","last_unstale":"2023-10-08T06:53:48.880825+0000","last_undegraded":"2023-10-08T06:53:48.880825+0000","last_fullsized":"2023-10-08T06:53:48.880825+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:51.272938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.891202+0000","last_change":"2023-10-08T06:53:46.853301+0000","last_active":"2023-10-08T06:53:47.891202+0000","last_peered":"2023-10-08T06:53:47.891202+0000","last_clean":"2023-10-08T06:53:47.891202+0000","last_became_active":"2023-10-08T06:53:46.853014+0000","last_became_peered":"2023-10-08T06:53:46.853014+0000","last_unstale":"2023-10-08T06:53:47.891202+0000","last_undegraded":"2023-10-08T06:53:47.891202+0000","last_fullsized":"2023-10-08T06:53:47.891202+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:01.090613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.951644+0000","last_change":"2023-10-08T06:53:47.887557+0000","last_active":"2023-10-08T06:53:48.951644+0000","last_peered":"2023-10-08T06:53:48.951644+0000","last_clean":"2023-10-08T06:53:48.951644+0000","last_became_active":"2023-10-08T06:53:47.887280+0000","last_became_peered":"2023-10-08T06:53:47.887280+0000","last_unstale":"2023-10-08T06:53:48.951644+0000","last_undegraded":"2023-10-08T06:53:48.951644+0000","last_fullsized":"2023-10-08T06:53:48.951644+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:07.878558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.882894+0000","last_change":"2023-10-08T06:53:46.854219+0000","last_active":"2023-10-08T06:53:47.882894+0000","last_peered":"2023-10-08T06:53:47.882894+0000","last_clean":"2023-10-08T06:53:47.882894+0000","last_became_active":"2023-10-08T06:53:46.853948+0000","last_became_peered":"2023-10-08T06:53:46.853948+0000","last_unstale":"2023-10-08T06:53:47.882894+0000","last_undegraded":"2023-10-08T06:53:47.882894+0000","last_fullsized":"2023-10-08T06:53:47.882894+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:19:56.207340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.874562+0000","last_change":"2023-10-08T06:53:47.873720+0000","last_active":"2023-10-08T06:53:48.874562+0000","last_peered":"2023-10-08T06:53:48.874562+0000","last_clean":"2023-10-08T06:53:48.874562+0000","last_became_active":"2023-10-08T06:53:47.873448+0000","last_became_peered":"2023-10-08T06:53:47.873448+0000","last_unstale":"2023-10-08T06:53:48.874562+0000","last_undegraded":"2023-10-08T06:53:48.874562+0000","last_fullsized":"2023-10-08T06:53:48.874562+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:04.799182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.882588+0000","last_change":"2023-10-08T06:53:46.847384+0000","last_active":"2023-10-08T06:53:47.882588+0000","last_peered":"2023-10-08T06:53:47.882588+0000","last_clean":"2023-10-08T06:53:47.882588+0000","last_became_active":"2023-10-08T06:53:46.847114+0000","last_became_peered":"2023-10-08T06:53:46.847114+0000","last_unstale":"2023-10-08T06:53:47.882588+0000","last_undegraded":"2023-10-08T06:53:47.882588+0000","last_fullsized":"2023-10-08T06:53:47.882588+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:54.188933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.881840+0000","last_change":"2023-10-08T06:53:47.875152+0000","last_active":"2023-10-08T06:53:48.881840+0000","last_peered":"2023-10-08T06:53:48.881840+0000","last_clean":"2023-10-08T06:53:48.881840+0000","last_became_active":"2023-10-08T06:53:47.874820+0000","last_became_peered":"2023-10-08T06:53:47.874820+0000","last_unstale":"2023-10-08T06:53:48.881840+0000","last_undegraded":"2023-10-08T06:53:48.881840+0000","last_fullsized":"2023-10-08T06:53:48.881840+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:58.222198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.891161+0000","last_change":"2023-10-08T06:53:46.842021+0000","last_active":"2023-10-08T06:53:47.891161+0000","last_peered":"2023-10-08T06:53:47.891161+0000","last_clean":"2023-10-08T06:53:47.891161+0000","last_became_active":"2023-10-08T06:53:46.841767+0000","last_became_peered":"2023-10-08T06:53:46.841767+0000","last_unstale":"2023-10-08T06:53:47.891161+0000","last_undegraded":"2023-10-08T06:53:47.891161+0000","last_fullsized":"2023-10-08T06:53:47.891161+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:27:53.231805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.372884+0000","last_change":"2023-10-08T06:53:47.868793+0000","last_active":"2023-10-08T06:53:49.372884+0000","last_peered":"2023-10-08T06:53:49.372884+0000","last_clean":"2023-10-08T06:53:49.372884+0000","last_became_active":"2023-10-08T06:53:47.868526+0000","last_became_peered":"2023-10-08T06:53:47.868526+0000","last_unstale":"2023-10-08T06:53:49.372884+0000","last_undegraded":"2023-10-08T06:53:49.372884+0000","last_fullsized":"2023-10-08T06:53:49.372884+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:58.882153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.853125+0000","last_change":"2023-10-08T06:53:46.845053+0000","last_active":"2023-10-08T06:53:47.853125+0000","last_peered":"2023-10-08T06:53:47.853125+0000","last_clean":"2023-10-08T06:53:47.853125+0000","last_became_active":"2023-10-08T06:53:46.844706+0000","last_became_peered":"2023-10-08T06:53:46.844706+0000","last_unstale":"2023-10-08T06:53:47.853125+0000","last_undegraded":"2023-10-08T06:53:47.853125+0000","last_fullsized":"2023-10-08T06:53:47.853125+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:51.981195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945333+0000","last_change":"2023-10-08T06:53:47.882250+0000","last_active":"2023-10-08T06:53:48.945333+0000","last_peered":"2023-10-08T06:53:48.945333+0000","last_clean":"2023-10-08T06:53:48.945333+0000","last_became_active":"2023-10-08T06:53:47.881919+0000","last_became_peered":"2023-10-08T06:53:47.881919+0000","last_unstale":"2023-10-08T06:53:48.945333+0000","last_undegraded":"2023-10-08T06:53:48.945333+0000","last_fullsized":"2023-10-08T06:53:48.945333+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:27:54.247340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.850586+0000","last_change":"2023-10-08T06:53:46.843744+0000","last_active":"2023-10-08T06:53:47.850586+0000","last_peered":"2023-10-08T06:53:47.850586+0000","last_clean":"2023-10-08T06:53:47.850586+0000","last_became_active":"2023-10-08T06:53:46.843504+0000","last_became_peered":"2023-10-08T06:53:46.843504+0000","last_unstale":"2023-10-08T06:53:47.850586+0000","last_undegraded":"2023-10-08T06:53:47.850586+0000","last_fullsized":"2023-10-08T06:53:47.850586+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:32.390630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.880259+0000","last_change":"2023-10-08T06:53:47.896027+0000","last_active":"2023-10-08T06:53:48.880259+0000","last_peered":"2023-10-08T06:53:48.880259+0000","last_clean":"2023-10-08T06:53:48.880259+0000","last_became_active":"2023-10-08T06:53:47.894974+0000","last_became_peered":"2023-10-08T06:53:47.894974+0000","last_unstale":"2023-10-08T06:53:48.880259+0000","last_undegraded":"2023-10-08T06:53:48.880259+0000","last_fullsized":"2023-10-08T06:53:48.880259+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:56:33.510351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.880131+0000","last_change":"2023-10-08T06:53:46.848518+0000","last_active":"2023-10-08T06:53:47.880131+0000","last_peered":"2023-10-08T06:53:47.880131+0000","last_clean":"2023-10-08T06:53:47.880131+0000","last_became_active":"2023-10-08T06:53:46.848237+0000","last_became_peered":"2023-10-08T06:53:46.848237+0000","last_unstale":"2023-10-08T06:53:47.880131+0000","last_undegraded":"2023-10-08T06:53:47.880131+0000","last_fullsized":"2023-10-08T06:53:47.880131+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:20.467886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375518+0000","last_change":"2023-10-08T06:53:47.881931+0000","last_active":"2023-10-08T06:53:49.375518+0000","last_peered":"2023-10-08T06:53:49.375518+0000","last_clean":"2023-10-08T06:53:49.375518+0000","last_became_active":"2023-10-08T06:53:47.881688+0000","last_became_peered":"2023-10-08T06:53:47.881688+0000","last_unstale":"2023-10-08T06:53:49.375518+0000","last_undegraded":"2023-10-08T06:53:49.375518+0000","last_fullsized":"2023-10-08T06:53:49.375518+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:33.930353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.856605+0000","last_change":"2023-10-08T06:53:46.844822+0000","last_active":"2023-10-08T06:53:47.856605+0000","last_peered":"2023-10-08T06:53:47.856605+0000","last_clean":"2023-10-08T06:53:47.856605+0000","last_became_active":"2023-10-08T06:53:46.844540+0000","last_became_peered":"2023-10-08T06:53:46.844540+0000","last_unstale":"2023-10-08T06:53:47.856605+0000","last_undegraded":"2023-10-08T06:53:47.856605+0000","last_fullsized":"2023-10-08T06:53:47.856605+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:32.390630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375998+0000","last_change":"2023-10-08T06:53:47.870838+0000","last_active":"2023-10-08T06:53:49.375998+0000","last_peered":"2023-10-08T06:53:49.375998+0000","last_clean":"2023-10-08T06:53:49.375998+0000","last_became_active":"2023-10-08T06:53:47.870553+0000","last_became_peered":"2023-10-08T06:53:47.870553+0000","last_unstale":"2023-10-08T06:53:49.375998+0000","last_undegraded":"2023-10-08T06:53:49.375998+0000","last_fullsized":"2023-10-08T06:53:49.375998+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:30.575901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"184'3","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.879422+0000","last_change":"2023-10-08T06:53:46.832905+0000","last_active":"2023-10-08T06:53:47.879422+0000","last_peered":"2023-10-08T06:53:47.879422+0000","last_clean":"2023-10-08T06:53:47.879422+0000","last_became_active":"2023-10-08T06:53:46.832624+0000","last_became_peered":"2023-10-08T06:53:46.832624+0000","last_unstale":"2023-10-08T06:53:47.879422+0000","last_undegraded":"2023-10-08T06:53:47.879422+0000","last_fullsized":"2023-10-08T06:53:47.879422+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:18:17.677806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.879155+0000","last_change":"2023-10-08T06:53:47.895198+0000","last_active":"2023-10-08T06:53:48.879155+0000","last_peered":"2023-10-08T06:53:48.879155+0000","last_clean":"2023-10-08T06:53:48.879155+0000","last_became_active":"2023-10-08T06:53:47.894749+0000","last_became_peered":"2023-10-08T06:53:47.894749+0000","last_unstale":"2023-10-08T06:53:48.879155+0000","last_undegraded":"2023-10-08T06:53:48.879155+0000","last_fullsized":"2023-10-08T06:53:48.879155+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:11.207254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.891718+0000","last_change":"2023-10-08T06:53:46.838423+0000","last_active":"2023-10-08T06:53:47.891718+0000","last_peered":"2023-10-08T06:53:47.891718+0000","last_clean":"2023-10-08T06:53:47.891718+0000","last_became_active":"2023-10-08T06:53:46.838071+0000","last_became_peered":"2023-10-08T06:53:46.838071+0000","last_unstale":"2023-10-08T06:53:47.891718+0000","last_undegraded":"2023-10-08T06:53:47.891718+0000","last_fullsized":"2023-10-08T06:53:47.891718+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:16.069696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375282+0000","last_change":"2023-10-08T06:53:47.873900+0000","last_active":"2023-10-08T06:53:49.375282+0000","last_peered":"2023-10-08T06:53:49.375282+0000","last_clean":"2023-10-08T06:53:49.375282+0000","last_became_active":"2023-10-08T06:53:47.873565+0000","last_became_peered":"2023-10-08T06:53:47.873565+0000","last_unstale":"2023-10-08T06:53:49.375282+0000","last_undegraded":"2023-10-08T06:53:49.375282+0000","last_fullsized":"2023-10-08T06:53:49.375282+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:44.391431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.859897+0000","last_change":"2023-10-08T06:53:46.857001+0000","last_active":"2023-10-08T06:53:47.859897+0000","last_peered":"2023-10-08T06:53:47.859897+0000","last_clean":"2023-10-08T06:53:47.859897+0000","last_became_active":"2023-10-08T06:53:46.856707+0000","last_became_peered":"2023-10-08T06:53:46.856707+0000","last_unstale":"2023-10-08T06:53:47.859897+0000","last_undegraded":"2023-10-08T06:53:47.859897+0000","last_fullsized":"2023-10-08T06:53:47.859897+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:05.134941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373221+0000","last_change":"2023-10-08T06:53:47.881369+0000","last_active":"2023-10-08T06:53:49.373221+0000","last_peered":"2023-10-08T06:53:49.373221+0000","last_clean":"2023-10-08T06:53:49.373221+0000","last_became_active":"2023-10-08T06:53:47.880888+0000","last_became_peered":"2023-10-08T06:53:47.880888+0000","last_unstale":"2023-10-08T06:53:49.373221+0000","last_undegraded":"2023-10-08T06:53:49.373221+0000","last_fullsized":"2023-10-08T06:53:49.373221+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:39.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.858439+0000","last_change":"2023-10-08T06:53:46.842178+0000","last_active":"2023-10-08T06:53:47.858439+0000","last_peered":"2023-10-08T06:53:47.858439+0000","last_clean":"2023-10-08T06:53:47.858439+0000","last_became_active":"2023-10-08T06:53:46.840731+0000","last_became_peered":"2023-10-08T06:53:46.840731+0000","last_unstale":"2023-10-08T06:53:47.858439+0000","last_undegraded":"2023-10-08T06:53:47.858439+0000","last_fullsized":"2023-10-08T06:53:47.858439+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:16.399937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373817+0000","last_change":"2023-10-08T06:53:47.887041+0000","last_active":"2023-10-08T06:53:49.373817+0000","last_peered":"2023-10-08T06:53:49.373817+0000","last_clean":"2023-10-08T06:53:49.373817+0000","last_became_active":"2023-10-08T06:53:47.886752+0000","last_became_peered":"2023-10-08T06:53:47.886752+0000","last_unstale":"2023-10-08T06:53:49.373817+0000","last_undegraded":"2023-10-08T06:53:49.373817+0000","last_fullsized":"2023-10-08T06:53:49.373817+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:01.337080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"184'8","reported_seq":20,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.890734+0000","last_change":"2023-10-08T06:53:46.853950+0000","last_active":"2023-10-08T06:53:47.890734+0000","last_peered":"2023-10-08T06:53:47.890734+0000","last_clean":"2023-10-08T06:53:47.890734+0000","last_became_active":"2023-10-08T06:53:46.853668+0000","last_became_peered":"2023-10-08T06:53:46.853668+0000","last_unstale":"2023-10-08T06:53:47.890734+0000","last_undegraded":"2023-10-08T06:53:47.890734+0000","last_fullsized":"2023-10-08T06:53:47.890734+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:58:26.339899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.951051+0000","last_change":"2023-10-08T06:53:47.895495+0000","last_active":"2023-10-08T06:53:48.951051+0000","last_peered":"2023-10-08T06:53:48.951051+0000","last_clean":"2023-10-08T06:53:48.951051+0000","last_became_active":"2023-10-08T06:53:47.895203+0000","last_became_peered":"2023-10-08T06:53:47.895203+0000","last_unstale":"2023-10-08T06:53:48.951051+0000","last_undegraded":"2023-10-08T06:53:48.951051+0000","last_fullsized":"2023-10-08T06:53:48.951051+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:18.313136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.889844+0000","last_change":"2023-10-08T06:53:46.849784+0000","last_active":"2023-10-08T06:53:47.889844+0000","last_peered":"2023-10-08T06:53:47.889844+0000","last_clean":"2023-10-08T06:53:47.889844+0000","last_became_active":"2023-10-08T06:53:46.849527+0000","last_became_peered":"2023-10-08T06:53:46.849527+0000","last_unstale":"2023-10-08T06:53:47.889844+0000","last_undegraded":"2023-10-08T06:53:47.889844+0000","last_fullsized":"2023-10-08T06:53:47.889844+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:24.077508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.951755+0000","last_change":"2023-10-08T06:53:47.897370+0000","last_active":"2023-10-08T06:53:48.951755+0000","last_peered":"2023-10-08T06:53:48.951755+0000","last_clean":"2023-10-08T06:53:48.951755+0000","last_became_active":"2023-10-08T06:53:47.897032+0000","last_became_peered":"2023-10-08T06:53:47.897032+0000","last_unstale":"2023-10-08T06:53:48.951755+0000","last_undegraded":"2023-10-08T06:53:48.951755+0000","last_fullsized":"2023-10-08T06:53:48.951755+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:58.859856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.894323+0000","last_change":"2023-10-08T06:53:46.849864+0000","last_active":"2023-10-08T06:53:47.894323+0000","last_peered":"2023-10-08T06:53:47.894323+0000","last_clean":"2023-10-08T06:53:47.894323+0000","last_became_active":"2023-10-08T06:53:46.849576+0000","last_became_peered":"2023-10-08T06:53:46.849576+0000","last_unstale":"2023-10-08T06:53:47.894323+0000","last_undegraded":"2023-10-08T06:53:47.894323+0000","last_fullsized":"2023-10-08T06:53:47.894323+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:16.809488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.945947+0000","last_change":"2023-10-08T06:53:47.869652+0000","last_active":"2023-10-08T06:53:48.945947+0000","last_peered":"2023-10-08T06:53:48.945947+0000","last_clean":"2023-10-08T06:53:48.945947+0000","last_became_active":"2023-10-08T06:53:47.869012+0000","last_became_peered":"2023-10-08T06:53:47.869012+0000","last_unstale":"2023-10-08T06:53:48.945947+0000","last_undegraded":"2023-10-08T06:53:48.945947+0000","last_fullsized":"2023-10-08T06:53:48.945947+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:10.444749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"184'1","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.877913+0000","last_change":"2023-10-08T06:53:46.846147+0000","last_active":"2023-10-08T06:53:47.877913+0000","last_peered":"2023-10-08T06:53:47.877913+0000","last_clean":"2023-10-08T06:53:47.877913+0000","last_became_active":"2023-10-08T06:53:46.845910+0000","last_became_peered":"2023-10-08T06:53:46.845910+0000","last_unstale":"2023-10-08T06:53:47.877913+0000","last_undegraded":"2023-10-08T06:53:47.877913+0000","last_fullsized":"2023-10-08T06:53:47.877913+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:16:57.726805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.946161+0000","last_change":"2023-10-08T06:53:47.873795+0000","last_active":"2023-10-08T06:53:48.946161+0000","last_peered":"2023-10-08T06:53:48.946161+0000","last_clean":"2023-10-08T06:53:48.946161+0000","last_became_active":"2023-10-08T06:53:47.873455+0000","last_became_peered":"2023-10-08T06:53:47.873455+0000","last_unstale":"2023-10-08T06:53:48.946161+0000","last_undegraded":"2023-10-08T06:53:48.946161+0000","last_fullsized":"2023-10-08T06:53:48.946161+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:08:28.026259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.861962+0000","last_change":"2023-10-08T06:53:46.857013+0000","last_active":"2023-10-08T06:53:47.861962+0000","last_peered":"2023-10-08T06:53:47.861962+0000","last_clean":"2023-10-08T06:53:47.861962+0000","last_became_active":"2023-10-08T06:53:46.856762+0000","last_became_peered":"2023-10-08T06:53:46.856762+0000","last_unstale":"2023-10-08T06:53:47.861962+0000","last_undegraded":"2023-10-08T06:53:47.861962+0000","last_fullsized":"2023-10-08T06:53:47.861962+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:22.842337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.950711+0000","last_change":"2023-10-08T06:53:47.892327+0000","last_active":"2023-10-08T06:53:48.950711+0000","last_peered":"2023-10-08T06:53:48.950711+0000","last_clean":"2023-10-08T06:53:48.950711+0000","last_became_active":"2023-10-08T06:53:47.891986+0000","last_became_peered":"2023-10-08T06:53:47.891986+0000","last_unstale":"2023-10-08T06:53:48.950711+0000","last_undegraded":"2023-10-08T06:53:48.950711+0000","last_fullsized":"2023-10-08T06:53:48.950711+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:07:39.473909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.889254+0000","last_change":"2023-10-08T06:53:46.854554+0000","last_active":"2023-10-08T06:53:47.889254+0000","last_peered":"2023-10-08T06:53:47.889254+0000","last_clean":"2023-10-08T06:53:47.889254+0000","last_became_active":"2023-10-08T06:53:46.854235+0000","last_became_peered":"2023-10-08T06:53:46.854235+0000","last_unstale":"2023-10-08T06:53:47.889254+0000","last_undegraded":"2023-10-08T06:53:47.889254+0000","last_fullsized":"2023-10-08T06:53:47.889254+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:49.451557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.375609+0000","last_change":"2023-10-08T06:53:47.870266+0000","last_active":"2023-10-08T06:53:49.375609+0000","last_peered":"2023-10-08T06:53:49.375609+0000","last_clean":"2023-10-08T06:53:49.375609+0000","last_became_active":"2023-10-08T06:53:47.869947+0000","last_became_peered":"2023-10-08T06:53:47.869947+0000","last_unstale":"2023-10-08T06:53:49.375609+0000","last_undegraded":"2023-10-08T06:53:49.375609+0000","last_fullsized":"2023-10-08T06:53:49.375609+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:21:32.856518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.871923+0000","last_change":"2023-10-08T06:53:46.848822+0000","last_active":"2023-10-08T06:53:47.871923+0000","last_peered":"2023-10-08T06:53:47.871923+0000","last_clean":"2023-10-08T06:53:47.871923+0000","last_became_active":"2023-10-08T06:53:46.848489+0000","last_became_peered":"2023-10-08T06:53:46.848489+0000","last_unstale":"2023-10-08T06:53:47.871923+0000","last_undegraded":"2023-10-08T06:53:47.871923+0000","last_fullsized":"2023-10-08T06:53:47.871923+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:25.902796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.950805+0000","last_change":"2023-10-08T06:53:47.887357+0000","last_active":"2023-10-08T06:53:48.950805+0000","last_peered":"2023-10-08T06:53:48.950805+0000","last_clean":"2023-10-08T06:53:48.950805+0000","last_became_active":"2023-10-08T06:53:47.887075+0000","last_became_peered":"2023-10-08T06:53:47.887075+0000","last_unstale":"2023-10-08T06:53:48.950805+0000","last_undegraded":"2023-10-08T06:53:48.950805+0000","last_fullsized":"2023-10-08T06:53:48.950805+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:56.220725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.855681+0000","last_change":"2023-10-08T06:53:46.853408+0000","last_active":"2023-10-08T06:53:47.855681+0000","last_peered":"2023-10-08T06:53:47.855681+0000","last_clean":"2023-10-08T06:53:47.855681+0000","last_became_active":"2023-10-08T06:53:46.853121+0000","last_became_peered":"2023-10-08T06:53:46.853121+0000","last_unstale":"2023-10-08T06:53:47.855681+0000","last_undegraded":"2023-10-08T06:53:47.855681+0000","last_fullsized":"2023-10-08T06:53:47.855681+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:37.692117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374152+0000","last_change":"2023-10-08T06:53:47.872247+0000","last_active":"2023-10-08T06:53:49.374152+0000","last_peered":"2023-10-08T06:53:49.374152+0000","last_clean":"2023-10-08T06:53:49.374152+0000","last_became_active":"2023-10-08T06:53:47.871935+0000","last_became_peered":"2023-10-08T06:53:47.871935+0000","last_unstale":"2023-10-08T06:53:49.374152+0000","last_undegraded":"2023-10-08T06:53:49.374152+0000","last_fullsized":"2023-10-08T06:53:49.374152+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:19:47.050032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.878143+0000","last_change":"2023-10-08T06:53:46.841902+0000","last_active":"2023-10-08T06:53:47.878143+0000","last_peered":"2023-10-08T06:53:47.878143+0000","last_clean":"2023-10-08T06:53:47.878143+0000","last_became_active":"2023-10-08T06:53:46.841528+0000","last_became_peered":"2023-10-08T06:53:46.841528+0000","last_unstale":"2023-10-08T06:53:47.878143+0000","last_undegraded":"2023-10-08T06:53:47.878143+0000","last_fullsized":"2023-10-08T06:53:47.878143+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:02:28.076684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374279+0000","last_change":"2023-10-08T06:53:47.867621+0000","last_active":"2023-10-08T06:53:49.374279+0000","last_peered":"2023-10-08T06:53:49.374279+0000","last_clean":"2023-10-08T06:53:49.374279+0000","last_became_active":"2023-10-08T06:53:47.867320+0000","last_became_peered":"2023-10-08T06:53:47.867320+0000","last_unstale":"2023-10-08T06:53:49.374279+0000","last_undegraded":"2023-10-08T06:53:49.374279+0000","last_fullsized":"2023-10-08T06:53:49.374279+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:07.878558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"184'2","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.888868+0000","last_change":"2023-10-08T06:53:46.856348+0000","last_active":"2023-10-08T06:53:47.888868+0000","last_peered":"2023-10-08T06:53:47.888868+0000","last_clean":"2023-10-08T06:53:47.888868+0000","last_became_active":"2023-10-08T06:53:46.856059+0000","last_became_peered":"2023-10-08T06:53:46.856059+0000","last_unstale":"2023-10-08T06:53:47.888868+0000","last_undegraded":"2023-10-08T06:53:47.888868+0000","last_fullsized":"2023-10-08T06:53:47.888868+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:25.375848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.883244+0000","last_change":"2023-10-08T06:53:47.885694+0000","last_active":"2023-10-08T06:53:48.883244+0000","last_peered":"2023-10-08T06:53:48.883244+0000","last_clean":"2023-10-08T06:53:48.883244+0000","last_became_active":"2023-10-08T06:53:47.885290+0000","last_became_peered":"2023-10-08T06:53:47.885290+0000","last_unstale":"2023-10-08T06:53:48.883244+0000","last_undegraded":"2023-10-08T06:53:48.883244+0000","last_fullsized":"2023-10-08T06:53:48.883244+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:52.930923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"185'1","reported_seq":13,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.946350+0000","last_change":"2023-10-08T06:53:46.851826+0000","last_active":"2023-10-08T06:53:48.946350+0000","last_peered":"2023-10-08T06:53:48.946350+0000","last_clean":"2023-10-08T06:53:48.946350+0000","last_became_active":"2023-10-08T06:53:46.851307+0000","last_became_peered":"2023-10-08T06:53:46.851307+0000","last_unstale":"2023-10-08T06:53:48.946350+0000","last_undegraded":"2023-10-08T06:53:48.946350+0000","last_fullsized":"2023-10-08T06:53:48.946350+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:36.369411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.946198+0000","last_change":"2023-10-08T06:53:47.886731+0000","last_active":"2023-10-08T06:53:48.946198+0000","last_peered":"2023-10-08T06:53:48.946198+0000","last_clean":"2023-10-08T06:53:48.946198+0000","last_became_active":"2023-10-08T06:53:47.886285+0000","last_became_peered":"2023-10-08T06:53:47.886285+0000","last_unstale":"2023-10-08T06:53:48.946198+0000","last_undegraded":"2023-10-08T06:53:48.946198+0000","last_fullsized":"2023-10-08T06:53:48.946198+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:17.825023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.865492+0000","last_change":"2023-10-08T06:53:46.854168+0000","last_active":"2023-10-08T06:53:47.865492+0000","last_peered":"2023-10-08T06:53:47.865492+0000","last_clean":"2023-10-08T06:53:47.865492+0000","last_became_active":"2023-10-08T06:53:46.853876+0000","last_became_peered":"2023-10-08T06:53:46.853876+0000","last_unstale":"2023-10-08T06:53:47.865492+0000","last_undegraded":"2023-10-08T06:53:47.865492+0000","last_fullsized":"2023-10-08T06:53:47.865492+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:47.384330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.374760+0000","last_change":"2023-10-08T06:53:47.881235+0000","last_active":"2023-10-08T06:53:49.374760+0000","last_peered":"2023-10-08T06:53:49.374760+0000","last_clean":"2023-10-08T06:53:49.374760+0000","last_became_active":"2023-10-08T06:53:47.880974+0000","last_became_peered":"2023-10-08T06:53:47.880974+0000","last_unstale":"2023-10-08T06:53:49.374760+0000","last_undegraded":"2023-10-08T06:53:49.374760+0000","last_fullsized":"2023-10-08T06:53:49.374760+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:43:52.819577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.850661+0000","last_change":"2023-10-08T06:53:46.854126+0000","last_active":"2023-10-08T06:53:47.850661+0000","last_peered":"2023-10-08T06:53:47.850661+0000","last_clean":"2023-10-08T06:53:47.850661+0000","last_became_active":"2023-10-08T06:53:46.853833+0000","last_became_peered":"2023-10-08T06:53:46.853833+0000","last_unstale":"2023-10-08T06:53:47.850661+0000","last_undegraded":"2023-10-08T06:53:47.850661+0000","last_fullsized":"2023-10-08T06:53:47.850661+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:36.369411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.879159+0000","last_change":"2023-10-08T06:53:47.880003+0000","last_active":"2023-10-08T06:53:48.879159+0000","last_peered":"2023-10-08T06:53:48.879159+0000","last_clean":"2023-10-08T06:53:48.879159+0000","last_became_active":"2023-10-08T06:53:47.878993+0000","last_became_peered":"2023-10-08T06:53:47.878993+0000","last_unstale":"2023-10-08T06:53:48.879159+0000","last_undegraded":"2023-10-08T06:53:48.879159+0000","last_fullsized":"2023-10-08T06:53:48.879159+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:51:13.329398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.857483+0000","last_change":"2023-10-08T06:53:46.851411+0000","last_active":"2023-10-08T06:53:47.857483+0000","last_peered":"2023-10-08T06:53:47.857483+0000","last_clean":"2023-10-08T06:53:47.857483+0000","last_became_active":"2023-10-08T06:53:46.850930+0000","last_became_peered":"2023-10-08T06:53:46.850930+0000","last_unstale":"2023-10-08T06:53:47.857483+0000","last_undegraded":"2023-10-08T06:53:47.857483+0000","last_fullsized":"2023-10-08T06:53:47.857483+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:21:37.089110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.882479+0000","last_change":"2023-10-08T06:53:47.884876+0000","last_active":"2023-10-08T06:53:48.882479+0000","last_peered":"2023-10-08T06:53:48.882479+0000","last_clean":"2023-10-08T06:53:48.882479+0000","last_became_active":"2023-10-08T06:53:47.884593+0000","last_became_peered":"2023-10-08T06:53:47.884593+0000","last_unstale":"2023-10-08T06:53:48.882479+0000","last_undegraded":"2023-10-08T06:53:48.882479+0000","last_fullsized":"2023-10-08T06:53:48.882479+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:42.890785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.859410+0000","last_change":"2023-10-08T06:53:46.852038+0000","last_active":"2023-10-08T06:53:47.859410+0000","last_peered":"2023-10-08T06:53:47.859410+0000","last_clean":"2023-10-08T06:53:47.859410+0000","last_became_active":"2023-10-08T06:53:46.851840+0000","last_became_peered":"2023-10-08T06:53:46.851840+0000","last_unstale":"2023-10-08T06:53:47.859410+0000","last_undegraded":"2023-10-08T06:53:47.859410+0000","last_fullsized":"2023-10-08T06:53:47.859410+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:42:46.289951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:49.373257+0000","last_change":"2023-10-08T06:53:47.882562+0000","last_active":"2023-10-08T06:53:49.373257+0000","last_peered":"2023-10-08T06:53:49.373257+0000","last_clean":"2023-10-08T06:53:49.373257+0000","last_became_active":"2023-10-08T06:53:47.882264+0000","last_became_peered":"2023-10-08T06:53:47.882264+0000","last_unstale":"2023-10-08T06:53:49.373257+0000","last_undegraded":"2023-10-08T06:53:49.373257+0000","last_fullsized":"2023-10-08T06:53:49.373257+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:49:03.345084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.853339+0000","last_change":"2023-10-08T06:53:46.844152+0000","last_active":"2023-10-08T06:53:47.853339+0000","last_peered":"2023-10-08T06:53:47.853339+0000","last_clean":"2023-10-08T06:53:47.853339+0000","last_became_active":"2023-10-08T06:53:46.843835+0000","last_became_peered":"2023-10-08T06:53:46.843835+0000","last_unstale":"2023-10-08T06:53:47.853339+0000","last_undegraded":"2023-10-08T06:53:47.853339+0000","last_fullsized":"2023-10-08T06:53:47.853339+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:30.734197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.879976+0000","last_change":"2023-10-08T06:53:47.896315+0000","last_active":"2023-10-08T06:53:48.879976+0000","last_peered":"2023-10-08T06:53:48.879976+0000","last_clean":"2023-10-08T06:53:48.879976+0000","last_became_active":"2023-10-08T06:53:47.895734+0000","last_became_peered":"2023-10-08T06:53:47.895734+0000","last_unstale":"2023-10-08T06:53:48.879976+0000","last_undegraded":"2023-10-08T06:53:48.879976+0000","last_fullsized":"2023-10-08T06:53:48.879976+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:03:29.614443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.866069+0000","last_change":"2023-10-08T06:53:46.846292+0000","last_active":"2023-10-08T06:53:47.866069+0000","last_peered":"2023-10-08T06:53:47.866069+0000","last_clean":"2023-10-08T06:53:47.866069+0000","last_became_active":"2023-10-08T06:53:46.845869+0000","last_became_peered":"2023-10-08T06:53:46.845869+0000","last_unstale":"2023-10-08T06:53:47.866069+0000","last_undegraded":"2023-10-08T06:53:47.866069+0000","last_fullsized":"2023-10-08T06:53:47.866069+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:59.318995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-10-08T06:53:48.880760+0000","last_change":"2023-10-08T06:53:47.893207+0000","last_active":"2023-10-08T06:53:48.880760+0000","last_peered":"2023-10-08T06:53:48.880760+0000","last_clean":"2023-10-08T06:53:48.880760+0000","last_became_active":"2023-10-08T06:53:47.892915+0000","last_became_peered":"2023-10-08T06:53:47.892915+0000","last_unstale":"2023-10-08T06:53:48.880760+0000","last_undegraded":"2023-10-08T06:53:48.880760+0000","last_fullsized":"2023-10-08T06:53:48.880760+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:46.808694+0000","last_clean_scrub_stamp":"2023-10-08T06:53:46.808694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:20.197322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"184'6","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.894168+0000","last_change":"2023-10-08T06:53:46.853823+0000","last_active":"2023-10-08T06:53:47.894168+0000","last_peered":"2023-10-08T06:53:47.894168+0000","last_clean":"2023-10-08T06:53:47.894168+0000","last_became_active":"2023-10-08T06:53:46.853347+0000","last_became_peered":"2023-10-08T06:53:46.853347+0000","last_unstale":"2023-10-08T06:53:47.894168+0000","last_undegraded":"2023-10-08T06:53:47.894168+0000","last_fullsized":"2023-10-08T06:53:47.894168+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:53:45.793159+0000","last_clean_scrub_stamp":"2023-10-08T06:53:45.793159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:28.030370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"172'2729","reported_seq":3396,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-10-08T06:53:47.866242+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T06:53:47.866242+0000","last_peered":"2023-10-08T06:53:47.866242+0000","last_clean":"2023-10-08T06:53:47.866242+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T06:53:47.866242+0000","last_undegraded":"2023-10-08T06:53:47.866242+0000","last_fullsized":"2023-10-08T06:53:47.866242+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2729,"log_dups_size":0,"ondisk_log_size":2729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1533,"num_read_kb":3176,"num_write":449,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2729,"ondisk_log_size":2729,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575776,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60396,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311444,"statfs":{"total":96636764160,"available":96574918656,"internally_reserved":0,"allocated":2908160,"data_stored":1779799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.21,"15min":1.2330000000000001},"min":{"1min":0.91900000000000004,"5min":0.78700000000000003,"15min":0.70399999999999996},"max":{"1min":1.6950000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.365},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.224,"15min":1.248},"min":{"1min":0.93799999999999994,"5min":0.747,"15min":0.72899999999999998},"max":{"1min":1.522,"5min":1.7150000000000001,"15min":1.849},"last":0.91700000000000004}]},{"osd":1,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2829999999999999,"15min":1.262},"min":{"1min":0.89200000000000002,"5min":0.84299999999999997,"15min":0.70399999999999996},"max":{"1min":1.6970000000000001,"5min":1.712,"15min":1.8480000000000001},"last":1.163},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.284,"15min":1.3220000000000001},"min":{"1min":0.95999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7450000000000001,"5min":1.9299999999999999,"15min":2.4809999999999999},"last":1.2330000000000001}]},{"osd":2,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3140000000000001,"15min":1.3109999999999999},"min":{"1min":1.0740000000000001,"5min":0.90900000000000003,"15min":0.81899999999999995},"max":{"1min":1.627,"5min":2.7040000000000002,"15min":3.4350000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.381,"15min":1.3740000000000001},"min":{"1min":0.95699999999999996,"5min":0.88900000000000001,"15min":0.83199999999999996},"max":{"1min":1.873,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":1.48}]},{"osd":3,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.339,"15min":1.335},"min":{"1min":1.0189999999999999,"5min":0.97799999999999998,"15min":0.85899999999999999},"max":{"1min":1.5609999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.46,"15min":1.419},"min":{"1min":1.147,"5min":0.98899999999999999,"15min":0.83099999999999996},"max":{"1min":2.0379999999999998,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.4339999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.333,"15min":1.2869999999999999},"min":{"1min":0.93999999999999995,"5min":0.83199999999999996,"15min":0.66200000000000003},"max":{"1min":1.952,"5min":1.952,"15min":3.2839999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.456,"15min":1.391},"min":{"1min":1.1579999999999999,"5min":0.90300000000000002,"15min":0.80800000000000005},"max":{"1min":1.708,"5min":2.004,"15min":2.1909999999999998},"last":1.3260000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.474,"15min":1.4399999999999999},"min":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":0.86499999999999999},"max":{"1min":2.1499999999999999,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":1.534},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.504,"15min":1.4790000000000001},"min":{"1min":1.2909999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.139}]},{"osd":6,"last update":"Sun Oct 8 06:52:59 2023","interfaces":[{"interface":"back","average":{"1min":1.593,"5min":1.5680000000000001,"15min":1.556},"min":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":0.94599999999999995},"max":{"1min":2.1970000000000001,"5min":2.4350000000000001,"15min":3.1360000000000001},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.5269999999999999,"15min":1.502},"min":{"1min":1.0049999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"max":{"1min":2.0190000000000001,"5min":2.1309999999999998,"15min":2.3140000000000001},"last":1.4510000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575775,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41768,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330072,"statfs":{"total":96636764160,"available":96593993728,"internally_reserved":0,"allocated":2314240,"data_stored":1189479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3009999999999999,"15min":1.3480000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7310000000000001,"5min":1.792,"15min":2.508},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3089999999999999,"15min":1.337},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.609,"5min":1.8280000000000001,"15min":2.1139999999999999},"last":0.93400000000000005}]},{"osd":1,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.2170000000000001,"15min":1.248},"min":{"1min":0.876,"5min":0.876,"15min":0.59699999999999998},"max":{"1min":1.605,"5min":1.821,"15min":1.919},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.224,"5min":1.2769999999999999,"15min":1.2629999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.53800000000000003},"max":{"1min":1.7769999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":1.272}]},{"osd":2,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.2170000000000001,"15min":1.2290000000000001},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.75600000000000001},"max":{"1min":1.6499999999999999,"5min":1.722,"15min":1.8919999999999999},"last":1.143},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.29,"15min":1.2989999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.5229999999999999,"5min":1.6379999999999999,"15min":2.3839999999999999},"last":0.98799999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.4259999999999999,"15min":1.4039999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.83399999999999996},"max":{"1min":1.8700000000000001,"5min":1.976,"15min":2.1499999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.478,"15min":1.48},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.89400000000000002},"max":{"1min":1.8180000000000001,"5min":1.915,"15min":2.0289999999999999},"last":1.2370000000000001}]},{"osd":4,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.3580000000000001},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.72399999999999998},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.792},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4379999999999999,"15min":1.4390000000000001},"min":{"1min":0.878,"5min":0.878,"15min":0.80200000000000005},"max":{"1min":1.9099999999999999,"5min":1.9910000000000001,"15min":2.0089999999999999},"last":1.3009999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.431,"15min":1.4139999999999999},"min":{"1min":0.997,"5min":0.94799999999999995,"15min":0.69499999999999995},"max":{"1min":1.798,"5min":1.893,"15min":2.7309999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.5589999999999999,"15min":1.548},"min":{"1min":1.095,"5min":1.0249999999999999,"15min":0.78300000000000003},"max":{"1min":2.0640000000000001,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.867}]},{"osd":7,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.591,"15min":1.579},"min":{"1min":1.0960000000000001,"5min":1.0680000000000001,"15min":0.92000000000000004},"max":{"1min":1.984,"5min":2.0710000000000002,"15min":2.2090000000000001},"last":1.351},{"interface":"front","average":{"1min":1.488,"5min":1.573,"15min":1.5660000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.82099999999999995},"max":{"1min":1.946,"5min":2.222,"15min":2.6469999999999998},"last":1.194}]}]},{"osd":5,"up_from":13,"seq":55834575775,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59804,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312036,"statfs":{"total":96636764160,"available":96575524864,"internally_reserved":0,"allocated":2310144,"data_stored":1189409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2310000000000001,"15min":1.218},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.66600000000000004},"max":{"1min":1.5740000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.296,"15min":1.2569999999999999},"min":{"1min":0.95399999999999996,"5min":0.82699999999999996,"15min":0.78800000000000003},"max":{"1min":1.629,"5min":1.897,"15min":2.4700000000000002},"last":1.1719999999999999}]},{"osd":1,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.26,"15min":1.2230000000000001},"min":{"1min":0.95499999999999996,"5min":0.85799999999999998,"15min":0.72499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.371},{"interface":"front","average":{"1min":1.349,"5min":1.375,"15min":1.323},"min":{"1min":0.90400000000000003,"5min":0.88700000000000001,"15min":0.78200000000000003},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.0800000000000001}]},{"osd":2,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3480000000000001,"15min":1.3280000000000001},"min":{"1min":0.98399999999999999,"5min":0.85199999999999998,"15min":0.83199999999999996},"max":{"1min":1.6479999999999999,"5min":1.9359999999999999,"15min":2.351},"last":1.329},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.3160000000000001},"min":{"1min":0.91000000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.667,"5min":1.7809999999999999,"15min":1.964},"last":1.242}]},{"osd":3,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3959999999999999,"15min":1.3540000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.528,"15min":1.464},"min":{"1min":1.139,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.9910000000000001,"5min":3.0059999999999998,"15min":5.1630000000000003},"last":1.2769999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3320000000000001,"15min":1.3240000000000001},"min":{"1min":0.96299999999999997,"5min":0.89300000000000002,"15min":0.81499999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4039999999999999},"last":1.006},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4379999999999999,"15min":1.4239999999999999},"min":{"1min":0.96899999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.403}]},{"osd":6,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.4990000000000001,"15min":1.4750000000000001},"min":{"1min":1.248,"5min":0.96399999999999997,"15min":0.81200000000000006},"max":{"1min":2.1150000000000002,"5min":2.1659999999999999,"15min":5.5019999999999998},"last":1.554},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4930000000000001,"15min":1.48},"min":{"1min":1.0509999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.0139999999999998,"5min":2.1040000000000001,"15min":2.2890000000000001},"last":1.1319999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.5660000000000001,"15min":1.5629999999999999},"min":{"1min":1.3400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":5.8780000000000001},"last":1.298},{"interface":"front","average":{"1min":1.679,"5min":1.6839999999999999,"15min":1.6299999999999999},"min":{"1min":1.415,"5min":1.1719999999999999,"15min":0.97099999999999997},"max":{"1min":2.206,"5min":2.6789999999999998,"15min":5.399},"last":1.4570000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575775,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41768,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330072,"statfs":{"total":96636764160,"available":96593993728,"internally_reserved":0,"allocated":2322432,"data_stored":1191021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.389,"15min":1.3460000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":1.198},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.3999999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.69799999999999995},"max":{"1min":1.7789999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.786}]},{"osd":1,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2749999999999999,"15min":1.28},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.73,"5min":1.784,"15min":1.9339999999999999},"last":1.133},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.64000000000000001},"max":{"1min":1.613,"5min":1.6870000000000001,"15min":1.7290000000000001},"last":1.2589999999999999}]},{"osd":2,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.2889999999999999,"15min":1.248},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.3759999999999999},"min":{"1min":1,"5min":0.93100000000000005,"15min":0.85399999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.7130000000000001}]},{"osd":3,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.339,"15min":1.3109999999999999},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.385},{"interface":"front","average":{"1min":1.55,"5min":1.4670000000000001,"15min":1.4099999999999999},"min":{"1min":1.0629999999999999,"5min":0.97299999999999998,"15min":0.93400000000000005},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.3260000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.4139999999999999,"15min":1.351},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.66700000000000004},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.593},{"interface":"front","average":{"1min":1.526,"5min":1.4450000000000001,"15min":1.375},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.749},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.5029999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.5329999999999999,"15min":1.425},"min":{"1min":0.97899999999999998,"5min":0.91100000000000003,"15min":0.72699999999999998},"max":{"1min":5.3339999999999996,"5min":5.3339999999999996,"15min":5.3339999999999996},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6120000000000001,"15min":1.546},"min":{"1min":1.212,"5min":0.96699999999999997,"15min":0.91400000000000003},"max":{"1min":2.1059999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":1.8500000000000001}]},{"osd":7,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.548,"15min":1.49},"min":{"1min":1.202,"5min":1.135,"15min":0.99099999999999999},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.784,"5min":1.6859999999999999,"15min":1.6060000000000001},"min":{"1min":1.0049999999999999,"5min":0.98999999999999999,"15min":0.93600000000000005},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":1.546}]}]},{"osd":3,"up_from":13,"seq":55834575777,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47336,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324504,"statfs":{"total":96636764160,"available":96588292096,"internally_reserved":0,"allocated":2318336,"data_stored":1190025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.93000000000000005,"15min":0.95499999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.44500000000000001},"max":{"1min":1.3200000000000001,"5min":1.71,"15min":1.71},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.196,"15min":1.175},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6719999999999999,"5min":1.851,"15min":2.137},"last":0.79400000000000004}]},{"osd":1,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.024,"15min":1.018},"min":{"1min":0.70999999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.2370000000000001,"5min":1.71,"15min":1.823},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.2989999999999999,"15min":1.3049999999999999},"min":{"1min":0.91000000000000003,"5min":0.872,"15min":0.77000000000000002},"max":{"1min":1.51,"5min":1.75,"15min":1.893},"last":1.248}]},{"osd":2,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.1739999999999999,"15min":1.137},"min":{"1min":0.85299999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6279999999999999,"5min":1.726,"15min":1.7410000000000001},"last":0.746},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.1919999999999999,"15min":1.1930000000000001},"min":{"1min":0.90400000000000003,"5min":0.68600000000000005,"15min":0.61399999999999999},"max":{"1min":1.589,"5min":1.7809999999999999,"15min":1.8360000000000001},"last":0.85299999999999998}]},{"osd":4,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.284,"15min":1.2649999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83299999999999996},"max":{"1min":1.8560000000000001,"5min":1.97,"15min":1.97},"last":1.141},{"interface":"front","average":{"1min":1.482,"5min":1.4650000000000001,"15min":1.452},"min":{"1min":1.1859999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.8100000000000001,"5min":1.905,"15min":2.1960000000000002},"last":1.327}]},{"osd":5,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.391,"15min":1.385},"min":{"1min":1.085,"5min":0.98399999999999999,"15min":0.88},"max":{"1min":1.6399999999999999,"5min":2.27,"15min":3.0009999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.415,"15min":1.421},"min":{"1min":1.1890000000000001,"5min":0.995,"15min":0.94099999999999995},"max":{"1min":1.6739999999999999,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.091}]},{"osd":6,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4059999999999999,"15min":1.3959999999999999},"min":{"1min":0.98399999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.016},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.534,"15min":1.518},"min":{"1min":1.2170000000000001,"5min":1.1639999999999999,"15min":0.99099999999999999},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.544},"last":1.45}]},{"osd":7,"last update":"Sun Oct 8 06:53:02 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.5649999999999999,"15min":1.5660000000000001},"min":{"1min":1.3069999999999999,"5min":1.1839999999999999,"15min":1.008},"max":{"1min":1.8939999999999999,"5min":2.008,"15min":2.347},"last":1.3},{"interface":"front","average":{"1min":1.55,"5min":1.5049999999999999,"15min":1.4790000000000001},"min":{"1min":1.0980000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"max":{"1min":2.1099999999999999,"5min":2.1099999999999999,"15min":2.1800000000000002},"last":1.5}]}]},{"osd":2,"up_from":13,"seq":55834575775,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47332,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324508,"statfs":{"total":96636764160,"available":96588296192,"internally_reserved":0,"allocated":2318336,"data_stored":1189521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91400000000000003,"15min":0.93700000000000006},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.345,"5min":1.4650000000000001,"15min":1.657},"last":1.167},{"interface":"front","average":{"1min":1.036,"5min":1.0549999999999999,"15min":1.0760000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.5249999999999999,"5min":1.661,"15min":2.2679999999999998},"last":1.1379999999999999}]},{"osd":1,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.004,"15min":1.0069999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.696,"5min":1.7170000000000001,"15min":1.996},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.137,"5min":1.1679999999999999,"15min":1.1819999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":2.1139999999999999},"last":1.077}]},{"osd":3,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2270000000000001,"15min":1.2230000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.772,"5min":1.925,"15min":2.3929999999999998},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.256,"15min":1.258},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.46,"5min":1.804,"15min":1.891},"last":1.2290000000000001}]},{"osd":4,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.4330000000000001,"15min":1.444},"min":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.859,"5min":1.9430000000000001,"15min":3.0939999999999999},"last":1.29},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.3160000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7470000000000001,"5min":1.7549999999999999,"15min":3.7320000000000002},"last":1.3089999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.429,"15min":1.425},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":3.3519999999999999},"last":1.827},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.5029999999999999,"15min":1.528},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.9810000000000001,"5min":2.2719999999999998,"15min":2.5249999999999999},"last":1.3580000000000001}]},{"osd":6,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.673,"5min":1.9990000000000001,"15min":2.206},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.548,"15min":1.532},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9610000000000001,"5min":1.966,"15min":3.02},"last":1.964}]},{"osd":7,"last update":"Sun Oct 8 06:53:29 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.51,"15min":1.492},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.861,"5min":2.0790000000000002,"15min":4.2869999999999999},"last":1.782},{"interface":"front","average":{"1min":1.55,"5min":1.5940000000000001,"15min":1.579},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.8540000000000001,"5min":2.0099999999999998,"15min":3.0760000000000001},"last":1.915}]}]},{"osd":1,"up_from":13,"seq":55834575775,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29284,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342556,"statfs":{"total":96636764160,"available":96606777344,"internally_reserved":0,"allocated":2314240,"data_stored":1189479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.0349999999999999,"15min":1.0029999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.601,"5min":4.601,"15min":4.601},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2190000000000001,"15min":1.22},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.1639999999999999}]},{"osd":2,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.0329999999999999,"15min":1.0229999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":4.3179999999999996,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.385,"5min":1.2250000000000001,"15min":1.1679999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":5.4059999999999997,"5min":5.4059999999999997,"15min":5.4059999999999997},"last":1.2230000000000001}]},{"osd":3,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.3029999999999999,"15min":1.2609999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":5.5220000000000002,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":1.325},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1579999999999999,"15min":1.1160000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.052}]},{"osd":4,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.4890000000000001,"5min":1.3919999999999999,"15min":1.3680000000000001},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":4.2409999999999997,"5min":4.2409999999999997,"15min":4.2409999999999997},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.472,"5min":1.337,"15min":1.298},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":1.637}]},{"osd":5,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.577,"5min":1.4279999999999999,"15min":1.3600000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":4.7530000000000001,"5min":4.7530000000000001,"15min":4.7530000000000001},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.742,"5min":1.5980000000000001,"15min":1.5329999999999999},"min":{"1min":1.03,"5min":1.03,"15min":0.75},"max":{"1min":5.7389999999999999,"5min":5.7389999999999999,"15min":5.7389999999999999},"last":1.48}]},{"osd":6,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.5700000000000001,"15min":1.53},"min":{"1min":1.05,"5min":1.05,"15min":1.02},"max":{"1min":4.4180000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":1.429},{"interface":"front","average":{"1min":1.661,"5min":1.5329999999999999,"15min":1.4810000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":4.6660000000000004,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":1.2529999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:53:24 2023","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.605,"15min":1.5489999999999999},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":5.6289999999999996,"5min":5.6289999999999996,"15min":5.6289999999999996},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.607,"15min":1.5549999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":4.5270000000000001,"5min":4.5270000000000001,"15min":4.5270000000000001},"last":1.5780000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575776,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38200,"kb_used_data":2860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333640,"statfs":{"total":96636764160,"available":96597647360,"internally_reserved":0,"allocated":2928640,"data_stored":1782095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.97999999999999998,"15min":0.97099999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5669999999999999,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":1.272},{"interface":"front","average":{"1min":1.173,"5min":1.123,"15min":1.091},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.222,"15min":1.163},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":2.226,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0580000000000001,"15min":1.0800000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.2669999999999999,"5min":2.7330000000000001,"15min":2.7330000000000001},"last":1.224}]},{"osd":3,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.294,"15min":1.27},"min":{"1min":0.625,"5min":0.625,"15min":0.621},"max":{"1min":2.8250000000000002,"5min":4.0220000000000002,"15min":4.0220000000000002},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.2110000000000001,"15min":1.1839999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.94499999999999995}]},{"osd":4,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.349,"15min":1.3680000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.353},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4199999999999999,"15min":1.411},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.319}]},{"osd":5,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4330000000000001,"15min":1.4019999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.68700000000000006},"max":{"1min":2.0659999999999998,"5min":2.0920000000000001,"15min":2.2480000000000002},"last":1.534},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5249999999999999,"15min":1.478},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":3.2229999999999999,"5min":3.887,"15min":3.887},"last":1.0629999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.504,"15min":1.476},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.115},{"interface":"front","average":{"1min":1.488,"5min":1.514,"15min":1.508},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.748},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.482}]},{"osd":7,"last update":"Sun Oct 8 06:53:33 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.546,"15min":1.5369999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.86199999999999999},"max":{"1min":2.3679999999999999,"5min":2.3679999999999999,"15min":2.3679999999999999},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.573,"15min":1.548},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":2.194,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.1970000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T06:53:57.825 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T06:53:57.826 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T06:53:58.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.084+0000 7f2c16621640 1 Processor -- start 2023-10-08T06:53:58.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.085+0000 7f2c16621640 1 -- start start 2023-10-08T06:53:58.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.086+0000 7f2c16621640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2c1006cfa0 0x7f2c1015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.086+0000 7f2c16621640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c1015b630 0x7f2c10070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.086+0000 7f2c16621640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c100c8c60 con 0x7f2c100cb500 2023-10-08T06:53:58.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.086+0000 7f2c16621640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c100b1ac0 con 0x7f2c1015b630 2023-10-08T06:53:58.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.086+0000 7f2c16621640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c100b1da0 con 0x7f2c1006cfa0 2023-10-08T06:53:58.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.086+0000 7f2c14bbe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c1015b630 0x7f2c10070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:58.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.087+0000 7f2c14bbe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c1015b630 0x7f2c10070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39192/0 (socket says 172.21.15.70:39192) 2023-10-08T06:53:58.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.087+0000 7f2c14bbe640 1 -- 172.21.15.70:0/3241788017 learned_addr learned my addr 172.21.15.70:0/3241788017 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:58.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.087+0000 7f2c0ffff640 1 --1- 172.21.15.70:0/3241788017 >> v1:172.21.15.70:6789/0 conn(0x7f2c100cb500 0x7f2c100cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50688/0 (socket says 172.21.15.70:50688) 2023-10-08T06:53:58.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.087+0000 7f2c14bbe640 1 -- 172.21.15.70:0/3241788017 >> v2:172.21.15.181:3300/0 conn(0x7f2c1006cfa0 msgr2=0x7f2c1015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:58.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.087+0000 7f2c07fff640 1 --2- 172.21.15.70:0/3241788017 >> v2:172.21.15.181:3300/0 conn(0x7f2c1006cfa0 0x7f2c1015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:58.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.087+0000 7f2c14bbe640 1 --2- 172.21.15.70:0/3241788017 >> v2:172.21.15.181:3300/0 conn(0x7f2c1006cfa0 0x7f2c1015b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.087+0000 7f2c14bbe640 1 -- 172.21.15.70:0/3241788017 >> v1:172.21.15.70:6789/0 conn(0x7f2c100cb500 legacy=0x7f2c100cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.087+0000 7f2c14bbe640 1 -- 172.21.15.70:0/3241788017 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c100b2080 con 0x7f2c1015b630 2023-10-08T06:53:58.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.088+0000 7f2c14bbe640 1 --2- 172.21.15.70:0/3241788017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 0x7f2c10070690 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f2c100b0690 tx=0x7f2c0802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0963c448c67dd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:58.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.088+0000 7f2c0f7fe640 1 -- 172.21.15.70:0/3241788017 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0802ca80 con 0x7f2c1015b630 2023-10-08T06:53:58.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.088+0000 7f2c0f7fe640 1 -- 172.21.15.70:0/3241788017 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0802cbe0 con 0x7f2c1015b630 2023-10-08T06:53:58.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.088+0000 7f2c0f7fe640 1 -- 172.21.15.70:0/3241788017 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c080343f0 con 0x7f2c1015b630 2023-10-08T06:53:58.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.089+0000 7f2c16621640 1 -- 172.21.15.70:0/3241788017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 msgr2=0x7f2c10070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.089+0000 7f2c16621640 1 --2- 172.21.15.70:0/3241788017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 0x7f2c10070690 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f2c100b0690 tx=0x7f2c0802bee0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.089+0000 7f2c16621640 1 -- 172.21.15.70:0/3241788017 shutdown_connections 2023-10-08T06:53:58.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.089+0000 7f2c16621640 1 --2- 172.21.15.70:0/3241788017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 0x7f2c10070690 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.089+0000 7f2c16621640 1 --2- 172.21.15.70:0/3241788017 >> v2:172.21.15.181:3300/0 conn(0x7f2c1006cfa0 0x7f2c1015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.089+0000 7f2c16621640 1 -- 172.21.15.70:0/3241788017 >> 172.21.15.70:0/3241788017 conn(0x7f2c1006a140 msgr2=0x7f2c1006a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:58.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.089+0000 7f2c16621640 1 -- 172.21.15.70:0/3241788017 shutdown_connections 2023-10-08T06:53:58.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.089+0000 7f2c16621640 1 -- 172.21.15.70:0/3241788017 wait complete. 2023-10-08T06:53:58.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.091+0000 7f2c16621640 1 Processor -- start 2023-10-08T06:53:58.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.091+0000 7f2c16621640 1 -- start start 2023-10-08T06:53:58.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.092+0000 7f2c16621640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c1006cfa0 0x7f2c1014d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.092+0000 7f2c16621640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c100cb500 0x7f2c100ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.092+0000 7f2c16621640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 0x7f2c100cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.092+0000 7f2c16621640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1014d550 con 0x7f2c1015b630 2023-10-08T06:53:58.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.092+0000 7f2c16621640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1014d930 con 0x7f2c100cb500 2023-10-08T06:53:58.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.092+0000 7f2c16621640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c100ad350 con 0x7f2c1006cfa0 2023-10-08T06:53:58.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.092+0000 7f2c14bbe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 0x7f2c100cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:58.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.092+0000 7f2c14bbe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 0x7f2c100cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39200/0 (socket says 172.21.15.70:39200) 2023-10-08T06:53:58.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.092+0000 7f2c14bbe640 1 -- 172.21.15.70:0/61075040 learned_addr learned my addr 172.21.15.70:0/61075040 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:58.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.093+0000 7f2c14bbe640 1 -- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c1006cfa0 msgr2=0x7f2c1014d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.093+0000 7f2c14bbe640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c1006cfa0 0x7f2c1014d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.093+0000 7f2c14bbe640 1 -- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c100cb500 msgr2=0x7f2c100ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.093+0000 7f2c14bbe640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c100cb500 0x7f2c100ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.093+0000 7f2c14bbe640 1 -- 172.21.15.70:0/61075040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c08009b30 con 0x7f2c1015b630 2023-10-08T06:53:58.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.093+0000 7f2c14bbe640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 0x7f2c100cd2b0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f2c08036040 tx=0x7f2c08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:58.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.093+0000 7f2c0d7fa640 1 -- 172.21.15.70:0/61075040 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0803e450 con 0x7f2c1015b630 2023-10-08T06:53:58.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.093+0000 7f2c0d7fa640 1 -- 172.21.15.70:0/61075040 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0803e5d0 con 0x7f2c1015b630 2023-10-08T06:53:58.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.093+0000 7f2c0d7fa640 1 -- 172.21.15.70:0/61075040 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0803e980 con 0x7f2c1015b630 2023-10-08T06:53:58.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.093+0000 7f2c16621640 1 -- 172.21.15.70:0/61075040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c100ad630 con 0x7f2c1015b630 2023-10-08T06:53:58.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.094+0000 7f2c16621640 1 -- 172.21.15.70:0/61075040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c100b4ba0 con 0x7f2c1015b630 2023-10-08T06:53:58.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.095+0000 7f2c16621640 1 -- 172.21.15.70:0/61075040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c100adb50 con 0x7f2c1015b630 2023-10-08T06:53:58.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.095+0000 7f2c0d7fa640 1 -- 172.21.15.70:0/61075040 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2c0802cd50 con 0x7f2c1015b630 2023-10-08T06:53:58.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.096+0000 7f2c0d7fa640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2bf006af90 0x7f2bf006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.096+0000 7f2c0d7fa640 1 -- 172.21.15.70:0/61075040 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2c080ac660 con 0x7f2c1015b630 2023-10-08T06:53:58.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.097+0000 7f2c0ffff640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2bf006af90 0x7f2bf006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:58.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.098+0000 7f2c0ffff640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2bf006af90 0x7f2bf006d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c100ce170 tx=0x7f2c000079c0 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:58.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.100+0000 7f2c0d7fa640 1 -- 172.21.15.70:0/61075040 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c080773a0 con 0x7f2c1015b630 2023-10-08T06:53:58.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.358+0000 7f2c16621640 1 -- 172.21.15.70:0/61075040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2c1014daf0 con 0x7f2c1015b630 2023-10-08T06:53:58.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.362+0000 7f2c0d7fa640 1 -- 172.21.15.70:0/61075040 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v44) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2c0807a9c0 con 0x7f2c1015b630 2023-10-08T06:53:58.363 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T06:53:58.363 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T06:53:58.363 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T06:53:58.363 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T06:53:58.364 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T06:53:58.364 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T06:53:58.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.364+0000 7f2c16621640 1 -- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2bf006af90 msgr2=0x7f2bf006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.364+0000 7f2c16621640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2bf006af90 0x7f2bf006d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c100ce170 tx=0x7f2c000079c0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.364+0000 7f2c16621640 1 -- 172.21.15.70:0/61075040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 msgr2=0x7f2c100cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.364+0000 7f2c16621640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 0x7f2c100cd2b0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f2c08036040 tx=0x7f2c08004a40 comp rx=0 tx=0).stop 2023-10-08T06:53:58.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.365+0000 7f2c16621640 1 -- 172.21.15.70:0/61075040 shutdown_connections 2023-10-08T06:53:58.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.365+0000 7f2c16621640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f2bf006af90 0x7f2bf006d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.365+0000 7f2c16621640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c1015b630 0x7f2c100cd2b0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.365+0000 7f2c16621640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c100cb500 0x7f2c100ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.365+0000 7f2c16621640 1 --2- 172.21.15.70:0/61075040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c1006cfa0 0x7f2c1014d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.365+0000 7f2c16621640 1 -- 172.21.15.70:0/61075040 >> 172.21.15.70:0/61075040 conn(0x7f2c1006a140 msgr2=0x7f2c10070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:58.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.365+0000 7f2c16621640 1 -- 172.21.15.70:0/61075040 shutdown_connections 2023-10-08T06:53:58.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.365+0000 7f2c16621640 1 -- 172.21.15.70:0/61075040 wait complete. 2023-10-08T06:53:58.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:53:58.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.596+0000 7faa09111640 1 Processor -- start 2023-10-08T06:53:58.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.597+0000 7faa09111640 1 -- start start 2023-10-08T06:53:58.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.598+0000 7faa09111640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faa041599d0 0x7faa040ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.598+0000 7faa09111640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faa040cc210 0x7faa040b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.598+0000 7faa09111640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa040cacf0 con 0x7faa04159db0 2023-10-08T06:53:58.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.598+0000 7faa09111640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa040b10f0 con 0x7faa041599d0 2023-10-08T06:53:58.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.598+0000 7faa09111640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa040ca3c0 con 0x7faa040cc210 2023-10-08T06:53:58.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.598+0000 7faa03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faa041599d0 0x7faa040ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:58.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.598+0000 7faa03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faa041599d0 0x7faa040ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39216/0 (socket says 172.21.15.70:39216) 2023-10-08T06:53:58.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.598+0000 7faa03fff640 1 -- 172.21.15.70:0/2956199052 learned_addr learned my addr 172.21.15.70:0/2956199052 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:58.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.599+0000 7faa03fff640 1 -- 172.21.15.70:0/2956199052 >> v2:172.21.15.181:3300/0 conn(0x7faa040cc210 msgr2=0x7faa040b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:58.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.599+0000 7faa03fff640 1 --2- 172.21.15.70:0/2956199052 >> v2:172.21.15.181:3300/0 conn(0x7faa040cc210 0x7faa040b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.599+0000 7faa03fff640 1 -- 172.21.15.70:0/2956199052 >> v1:172.21.15.181:6789/0 conn(0x7faa04159db0 legacy=0x7faa0415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:58.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.599+0000 7faa03fff640 1 -- 172.21.15.70:0/2956199052 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa040ca6a0 con 0x7faa041599d0 2023-10-08T06:53:58.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.600+0000 7faa03fff640 1 --2- 172.21.15.70:0/2956199052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 0x7faa040ad730 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009870 tx=0x7fa9f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a576292f4bc9b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:58.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.600+0000 7faa02ffd640 1 -- 172.21.15.70:0/2956199052 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4031870 con 0x7faa041599d0 2023-10-08T06:53:58.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.600+0000 7faa02ffd640 1 -- 172.21.15.70:0/2956199052 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f40319d0 con 0x7faa041599d0 2023-10-08T06:53:58.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.600+0000 7faa02ffd640 1 -- 172.21.15.70:0/2956199052 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4031d60 con 0x7faa041599d0 2023-10-08T06:53:58.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.601+0000 7faa09111640 1 -- 172.21.15.70:0/2956199052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 msgr2=0x7faa040ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.601+0000 7faa09111640 1 --2- 172.21.15.70:0/2956199052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 0x7faa040ad730 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009870 tx=0x7fa9f402b910 comp rx=0 tx=0).stop 2023-10-08T06:53:58.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.601+0000 7faa09111640 1 -- 172.21.15.70:0/2956199052 shutdown_connections 2023-10-08T06:53:58.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.601+0000 7faa09111640 1 --2- 172.21.15.70:0/2956199052 >> v2:172.21.15.181:3300/0 conn(0x7faa040cc210 0x7faa040b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.601+0000 7faa09111640 1 --2- 172.21.15.70:0/2956199052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 0x7faa040ad730 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.601+0000 7faa09111640 1 -- 172.21.15.70:0/2956199052 >> 172.21.15.70:0/2956199052 conn(0x7faa040cb0c0 msgr2=0x7faa040cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:58.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.601+0000 7faa09111640 1 -- 172.21.15.70:0/2956199052 shutdown_connections 2023-10-08T06:53:58.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.601+0000 7faa09111640 1 -- 172.21.15.70:0/2956199052 wait complete. 2023-10-08T06:53:58.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.602+0000 7faa09111640 1 Processor -- start 2023-10-08T06:53:58.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.603+0000 7faa09111640 1 -- start start 2023-10-08T06:53:58.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.603+0000 7faa09111640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa040cc210 0x7faa040b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa09111640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 0x7faa0415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa09111640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa04159db0 0x7faa040c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa09111640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0415a730 con 0x7faa041599d0 2023-10-08T06:53:58.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa09111640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0415a920 con 0x7faa040cc210 2023-10-08T06:53:58.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa09111640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0415ac00 con 0x7faa04159db0 2023-10-08T06:53:58.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 0x7faa0415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:58.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 0x7faa0415a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39224/0 (socket says 172.21.15.70:39224) 2023-10-08T06:53:58.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa037fe640 1 -- 172.21.15.70:0/3490691413 learned_addr learned my addr 172.21.15.70:0/3490691413 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:58.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa037fe640 1 -- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa04159db0 msgr2=0x7faa040c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa037fe640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa04159db0 0x7faa040c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa037fe640 1 -- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa040cc210 msgr2=0x7faa040b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa037fe640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa040cc210 0x7faa040b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.604+0000 7faa037fe640 1 -- 172.21.15.70:0/3490691413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f40094d0 con 0x7faa041599d0 2023-10-08T06:53:58.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.605+0000 7faa037fe640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 0x7faa0415a1f0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800b4a0 tx=0x7fa9f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:58.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.605+0000 7faa00ff9640 1 -- 172.21.15.70:0/3490691413 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8010610 con 0x7faa041599d0 2023-10-08T06:53:58.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.605+0000 7faa00ff9640 1 -- 172.21.15.70:0/3490691413 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f8010770 con 0x7faa041599d0 2023-10-08T06:53:58.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.605+0000 7faa09111640 1 -- 172.21.15.70:0/3490691413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0415aee0 con 0x7faa041599d0 2023-10-08T06:53:58.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.605+0000 7faa00ff9640 1 -- 172.21.15.70:0/3490691413 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8010b20 con 0x7faa041599d0 2023-10-08T06:53:58.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.605+0000 7faa09111640 1 -- 172.21.15.70:0/3490691413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0415b4a0 con 0x7faa041599d0 2023-10-08T06:53:58.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.606+0000 7faa09111640 1 -- 172.21.15.70:0/3490691413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0415b6c0 con 0x7faa041599d0 2023-10-08T06:53:58.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.607+0000 7faa00ff9640 1 -- 172.21.15.70:0/3490691413 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa9f8010c80 con 0x7faa041599d0 2023-10-08T06:53:58.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.608+0000 7faa00ff9640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa9f006af90 0x7fa9f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:58.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.608+0000 7faa00ff9640 1 -- 172.21.15.70:0/3490691413 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa9f80895d0 con 0x7faa041599d0 2023-10-08T06:53:58.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.609+0000 7faa03fff640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa9f006af90 0x7fa9f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:58.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.610+0000 7faa03fff640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa9f006af90 0x7fa9f006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4036000 tx=0x7fa9f4030000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:58.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.612+0000 7faa00ff9640 1 -- 172.21.15.70:0/3490691413 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa9f80543a0 con 0x7faa041599d0 2023-10-08T06:53:58.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.837+0000 7faa09111640 1 -- 172.21.15.70:0/3490691413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa0404e160 con 0x7faa041599d0 2023-10-08T06:53:58.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.838+0000 7faa00ff9640 1 -- 172.21.15.70:0/3490691413 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4917 (secure 0 0 0) 0x7fa9f80579c0 con 0x7faa041599d0 2023-10-08T06:53:58.839 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:53:58.839 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/542411062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":542411062},{"type":"v1","addr":"172.21.15.181:6836","nonce":542411062}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/646194010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":646194010},{"type":"v1","addr":"172.21.15.70:6835","nonce":646194010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1850202608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1850202608},{"type":"v1","addr":"172.21.15.181:6837","nonce":1850202608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:53:47.285813+0000","modified":"2023-10-08T06:53:50.356439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9591":{"gid":9591,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/3790014600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3790014600},{"type":"v1","addr":"172.21.15.70:6834","nonce":3790014600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-08T06:53:58.839 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 83 2023-10-08T06:53:58.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.840+0000 7faa09111640 1 -- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa9f006af90 msgr2=0x7fa9f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.840+0000 7faa09111640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa9f006af90 0x7fa9f006d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4036000 tx=0x7fa9f4030000 comp rx=0 tx=0).stop 2023-10-08T06:53:58.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.840+0000 7faa09111640 1 -- 172.21.15.70:0/3490691413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 msgr2=0x7faa0415a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:58.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.840+0000 7faa09111640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 0x7faa0415a1f0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800b4a0 tx=0x7fa9f800b960 comp rx=0 tx=0).stop 2023-10-08T06:53:58.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.841+0000 7faa09111640 1 -- 172.21.15.70:0/3490691413 shutdown_connections 2023-10-08T06:53:58.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.841+0000 7faa09111640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faa04159db0 0x7faa040c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.841+0000 7faa09111640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fa9f006af90 0x7fa9f006d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.841+0000 7faa09111640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa041599d0 0x7faa0415a1f0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.841+0000 7faa09111640 1 --2- 172.21.15.70:0/3490691413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faa040cc210 0x7faa040b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:58.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.841+0000 7faa09111640 1 -- 172.21.15.70:0/3490691413 >> 172.21.15.70:0/3490691413 conn(0x7faa040cb0c0 msgr2=0x7faa0415d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:58.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.841+0000 7faa09111640 1 -- 172.21.15.70:0/3490691413 shutdown_connections 2023-10-08T06:53:58.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:58.841+0000 7faa09111640 1 -- 172.21.15.70:0/3490691413 wait complete. 2023-10-08T06:53:59.265 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T06:53:47.285813+0000', 'modified': '2023-10-08T06:53:50.356439+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9591}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9591': {'gid': 9591, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.70:6834/3790014600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 3790014600}, {'type': 'v1', 'addr': '172.21.15.70:6834', 'nonce': 3790014600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T06:53:59.265 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T06:53:59.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T06:53:59.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.491+0000 7f263ebc5640 1 Processor -- start 2023-10-08T06:53:59.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.491+0000 7f263ebc5640 1 -- start start 2023-10-08T06:53:59.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.492+0000 7f263ebc5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380cd250 0x7f263815d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.492+0000 7f263ebc5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263806b6a0 con 0x7f26380cea30 2023-10-08T06:53:59.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.493+0000 7f263ebc5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380b19e0 con 0x7f263815cdb0 2023-10-08T06:53:59.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.493+0000 7f263ebc5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2638069ef0 con 0x7f26380cd250 2023-10-08T06:53:59.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.493+0000 7f2637fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f26380cea30 0x7f26380ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50692/0 (socket says 172.21.15.70:50692) 2023-10-08T06:53:59.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.493+0000 7f2637fff640 1 -- 172.21.15.70:0/2183925711 learned_addr learned my addr 172.21.15.70:0/2183925711 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:59.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.493+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661387464 0 0) 0x7f263806b6a0 con 0x7f26380cea30 2023-10-08T06:53:59.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.493+0000 7f263d162640 1 --2- 172.21.15.70:0/2183925711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380cd250 0x7f263815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.493+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2624003520 con 0x7f26380cea30 2023-10-08T06:53:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2231821016 0 0) 0x7f2624003520 con 0x7f26380cea30 2023-10-08T06:53:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380cd250 msgr2=0x7f263815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 --2- 172.21.15.70:0/2183925711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380cd250 0x7f263815d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 >> v1:172.21.15.181:6789/0 conn(0x7f263815cdb0 legacy=0x7f263806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263806a1d0 con 0x7f26380cea30 2023-10-08T06:53:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2620001670 con 0x7f26380cea30 2023-10-08T06:53:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 >> v1:172.21.15.70:6789/0 conn(0x7f26380cea30 legacy=0x7f26380ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 --2- 172.21.15.70:0/2183925711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26240039a0 0x7f2624003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 --2- 172.21.15.70:0/2183925711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380cd250 0x7f26240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 --2- 172.21.15.70:0/2183925711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2624004d00 0x7f2624005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2638069ef0 con 0x7f26240039a0 2023-10-08T06:53:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262400bb90 con 0x7f2624004d00 2023-10-08T06:53:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262400bd50 con 0x7f26380cd250 2023-10-08T06:53:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.494+0000 7f263c961640 1 --2- 172.21.15.70:0/2183925711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26240039a0 0x7f2624003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.495+0000 7f263c961640 1 -- 172.21.15.70:0/2183925711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380cd250 msgr2=0x7f26240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.495+0000 7f263c961640 1 --2- 172.21.15.70:0/2183925711 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380cd250 0x7f26240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.495+0000 7f263c961640 1 -- 172.21.15.70:0/2183925711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2624004d00 msgr2=0x7f2624005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.495+0000 7f263c961640 1 --2- 172.21.15.70:0/2183925711 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2624004d00 0x7f2624005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.495+0000 7f263c961640 1 -- 172.21.15.70:0/2183925711 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2624003520 con 0x7f26240039a0 2023-10-08T06:53:59.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.496+0000 7f263c961640 1 --2- 172.21.15.70:0/2183925711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26240039a0 0x7f2624003dc0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f2628001e50 tx=0x7f26240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efee386a41c9df1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:59.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.496+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628003b70 con 0x7f26240039a0 2023-10-08T06:53:59.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.496+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628003cd0 con 0x7f26240039a0 2023-10-08T06:53:59.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.496+0000 7f26377fe640 1 -- 172.21.15.70:0/2183925711 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628005310 con 0x7f26240039a0 2023-10-08T06:53:59.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.497+0000 7f263ebc5640 1 -- 172.21.15.70:0/2183925711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26240039a0 msgr2=0x7f2624003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.497+0000 7f263ebc5640 1 --2- 172.21.15.70:0/2183925711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26240039a0 0x7f2624003dc0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f2628001e50 tx=0x7f26240057f0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.497+0000 7f263d162640 1 -- 172.21.15.70:0/2183925711 reap_dead start 2023-10-08T06:53:59.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.497+0000 7f263ebc5640 1 -- 172.21.15.70:0/2183925711 shutdown_connections 2023-10-08T06:53:59.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.497+0000 7f263ebc5640 1 -- 172.21.15.70:0/2183925711 >> 172.21.15.70:0/2183925711 conn(0x7f26380b0af0 msgr2=0x7f26380b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:59.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.497+0000 7f263ebc5640 1 -- 172.21.15.70:0/2183925711 shutdown_connections 2023-10-08T06:53:59.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.497+0000 7f263ebc5640 1 -- 172.21.15.70:0/2183925711 wait complete. 2023-10-08T06:53:59.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.499+0000 7f263ebc5640 1 Processor -- start 2023-10-08T06:53:59.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.499+0000 7f263ebc5640 1 -- start start 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f263ebc5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26240039a0 0x7f26381435f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f263ebc5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380b46b0 0x7f263815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f263ebc5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380b4b30 0x7f263815b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f263ebc5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380b27b0 con 0x7f26380b46b0 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f263ebc5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380b2a90 con 0x7f26240039a0 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f263ebc5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380b2d70 con 0x7f26380b4b30 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f2637fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380b46b0 0x7f263815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f2637fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380b46b0 0x7f263815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39230/0 (socket says 172.21.15.70:39230) 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f2637fff640 1 -- 172.21.15.70:0/2048954529 learned_addr learned my addr 172.21.15.70:0/2048954529 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f2637fff640 1 -- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380b4b30 msgr2=0x7f263815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f2637fff640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380b4b30 0x7f263815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f2637fff640 1 -- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26240039a0 msgr2=0x7f26381435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f2637fff640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26240039a0 0x7f26381435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.500+0000 7f2637fff640 1 -- 172.21.15.70:0/2048954529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26380b3050 con 0x7f26380b46b0 2023-10-08T06:53:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.501+0000 7f2637fff640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380b46b0 0x7f263815b0d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f262000c730 tx=0x7f262000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.501+0000 7f26357fa640 1 -- 172.21.15.70:0/2048954529 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26200072d0 con 0x7f26380b46b0 2023-10-08T06:53:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.501+0000 7f26357fa640 1 -- 172.21.15.70:0/2048954529 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2620007470 con 0x7f26380b46b0 2023-10-08T06:53:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.501+0000 7f263ebc5640 1 -- 172.21.15.70:0/2048954529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26380b3330 con 0x7f26380b46b0 2023-10-08T06:53:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.501+0000 7f26357fa640 1 -- 172.21.15.70:0/2048954529 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620004a20 con 0x7f26380b46b0 2023-10-08T06:53:59.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.501+0000 7f263ebc5640 1 -- 172.21.15.70:0/2048954529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2638158eb0 con 0x7f26380b46b0 2023-10-08T06:53:59.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.503+0000 7f26357fa640 1 -- 172.21.15.70:0/2048954529 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f262000a5e0 con 0x7f26380b46b0 2023-10-08T06:53:59.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.504+0000 7f26357fa640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f261806af90 0x7f261806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.504+0000 7f26357fa640 1 -- 172.21.15.70:0/2048954529 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2620089420 con 0x7f26380b46b0 2023-10-08T06:53:59.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.505+0000 7f263c961640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f261806af90 0x7f261806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.505+0000 7f264182b740 1 Processor -- start 2023-10-08T06:53:59.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.505+0000 7f264182b740 1 -- start start 2023-10-08T06:53:59.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.506+0000 7f264182b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc53a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.506+0000 7f264182b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80f40 0x5556afc838f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.506+0000 7f264182b740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc53fa0 0x5556afc83e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.506+0000 7f264182b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556afbc7700 con 0x5556afc80b60 2023-10-08T06:53:59.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.506+0000 7f264182b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556afbc4df0 con 0x5556afc80f40 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.506+0000 7f264182b740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556afbc06e0 con 0x5556afc53fa0 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.506+0000 7f263c961640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f261806af90 0x7f261806d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2628034000 tx=0x7f2628030000 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.506+0000 7f263c961640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc53a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.506+0000 7f263c961640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc53a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39244/0 (socket says 172.21.15.70:39244) 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.506+0000 7f263c961640 1 -- 172.21.15.70:0/13682140 learned_addr learned my addr 172.21.15.70:0/13682140 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.507+0000 7f2637fff640 1 --2- 172.21.15.70:0/13682140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80f40 0x5556afc838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.507+0000 7f263c961640 1 -- 172.21.15.70:0/13682140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc53fa0 msgr2=0x5556afc83e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.507+0000 7f263c961640 1 --2- 172.21.15.70:0/13682140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc53fa0 0x5556afc83e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.507+0000 7f263c961640 1 -- 172.21.15.70:0/13682140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80f40 msgr2=0x5556afc838f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.507+0000 7f263c961640 1 --2- 172.21.15.70:0/13682140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80f40 0x5556afc838f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.507+0000 7f263c961640 1 -- 172.21.15.70:0/13682140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556afb84910 con 0x5556afc80b60 2023-10-08T06:53:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.507+0000 7f263c961640 1 --2- 172.21.15.70:0/13682140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc53a60 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f2628001f40 tx=0x7f2628031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7deb3a4bc4d64c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.507+0000 7f262e7fc640 1 -- 172.21.15.70:0/13682140 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262804b070 con 0x5556afc80b60 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.507+0000 7f262e7fc640 1 -- 172.21.15.70:0/13682140 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628006b50 con 0x5556afc80b60 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.507+0000 7f262e7fc640 1 -- 172.21.15.70:0/13682140 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628007030 con 0x5556afc80b60 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.508+0000 7f264182b740 1 -- 172.21.15.70:0/13682140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 msgr2=0x5556afc53a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.508+0000 7f264182b740 1 --2- 172.21.15.70:0/13682140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc53a60 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f2628001f40 tx=0x7f2628031000 comp rx=0 tx=0).stop 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.508+0000 7f264182b740 1 -- 172.21.15.70:0/13682140 shutdown_connections 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.508+0000 7f264182b740 1 --2- 172.21.15.70:0/13682140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc53fa0 0x5556afc83e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.508+0000 7f264182b740 1 --2- 172.21.15.70:0/13682140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80f40 0x5556afc838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.508+0000 7f264182b740 1 --2- 172.21.15.70:0/13682140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc53a60 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.508+0000 7f264182b740 1 -- 172.21.15.70:0/13682140 >> 172.21.15.70:0/13682140 conn(0x5556afc55d70 msgr2=0x5556afc9e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.508+0000 7f264182b740 1 -- 172.21.15.70:0/13682140 shutdown_connections 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.508+0000 7f264182b740 1 -- 172.21.15.70:0/13682140 wait complete. 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.509+0000 7f264182b740 1 -- start start 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.509+0000 7f264182b740 1 Processor -- start 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f264182b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc71df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f264182b740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 0x5556afc9c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f264182b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc9cc00 0x5556afc9e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f264182b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556afb5e970 con 0x5556afc80b60 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f264182b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556afb5e610 con 0x5556afc9cc00 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f264182b740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556af924f20 con 0x5556afc80f40 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f264182b740 10 client.0 _ll_register_callbacks cb 0x5556afb9c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T06:53:59.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f263c961640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc71df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f263c961640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc71df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39254/0 (socket says 172.21.15.70:39254) 2023-10-08T06:53:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f263c961640 1 -- 172.21.15.70:0/626650571 learned_addr learned my addr 172.21.15.70:0/626650571 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.510+0000 7f263c961640 1 -- 172.21.15.70:0/626650571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 msgr2=0x5556afc9c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f263c961640 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 0x5556afc9c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f263c961640 1 -- 172.21.15.70:0/626650571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc9cc00 msgr2=0x5556afc9e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f263c961640 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc9cc00 0x5556afc9e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f263c961640 1 -- 172.21.15.70:0/626650571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2624003520 con 0x5556afc80b60 2023-10-08T06:53:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f263c961640 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc71df0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2628001f70 tx=0x7f2628001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8dfe115e11c9ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f26177fe640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T06:53:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f26177fe640 1 -- 172.21.15.70:0/626650571 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262804b040 con 0x5556afc80b60 2023-10-08T06:53:59.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f26177fe640 1 -- 172.21.15.70:0/626650571 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f262802f460 con 0x5556afc80b60 2023-10-08T06:53:59.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f264182b740 1 -- 172.21.15.70:0/626650571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556afbb96c0 con 0x5556afc80b60 2023-10-08T06:53:59.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f26177fe640 1 -- 172.21.15.70:0/626650571 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f262802f620 con 0x5556afc80b60 2023-10-08T06:53:59.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f264182b740 10 client.9756 fetch_fsmap learned FSMap version 83 2023-10-08T06:53:59.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.511+0000 7f264182b740 1 -- 172.21.15.70:0/626650571 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5556afbbed60 con 0x5556afc80b60 2023-10-08T06:53:59.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.512+0000 7f26177fe640 1 -- 172.21.15.70:0/626650571 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5556afbb96c0 con 0x5556afc80b60 2023-10-08T06:53:59.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.512+0000 7f26177fe640 1 -- 172.21.15.70:0/626650571 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f262802f780 con 0x5556afc80b60 2023-10-08T06:53:59.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.512+0000 7f264182b740 10 client.9756 fetch_fsmap finished waiting for FSMap version 83 2023-10-08T06:53:59.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.512+0000 7f264182b740 10 client.9756 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T06:53:59.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.512+0000 7f264182b740 20 client.9756 populate_metadata read hostname 'smithi070' 2023-10-08T06:53:59.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.512+0000 7f264182b740 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.70:6832/3790014600,v1:172.21.15.70:6834/3790014600] conn(0x5556afc861d0 0x5556afcbbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.512+0000 7f264182b740 4 client.9756 mds_command: new command op to 9591 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T06:53:59.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.512+0000 7f264182b740 1 -- 172.21.15.70:0/626650571 --> [v2:172.21.15.70:6832/3790014600,v1:172.21.15.70:6834/3790014600] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556afbb93e0 con 0x5556afc861d0 2023-10-08T06:53:59.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.512+0000 7f2637fff640 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.70:6832/3790014600,v1:172.21.15.70:6834/3790014600] conn(0x5556afc861d0 0x5556afcbbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.513+0000 7f26177fe640 10 client.9756 ms_handle_connect on v2:172.21.15.70:6832/3790014600 2023-10-08T06:53:59.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.513+0000 7f2637fff640 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.70:6832/3790014600,v1:172.21.15.70:6834/3790014600] conn(0x5556afc861d0 0x5556afcbbe90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f4b2d66a3f84d88 server_cookie=9559254549d9bb54 in_seq=0 out_seq=0 2023-10-08T06:53:59.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.513+0000 7f26177fe640 1 -- 172.21.15.70:0/626650571 <== mds.0 v2:172.21.15.70:6832/3790014600 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5556afbb93e0 con 0x5556afc861d0 2023-10-08T06:53:59.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.513+0000 7f26177fe640 10 client.9756 handle_command_reply: tid=0 2023-10-08T06:53:59.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.513+0000 7f264182b740 1 client.9756 shutdown 2023-10-08T06:53:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 1 -- 172.21.15.70:0/626650571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 msgr2=0x5556afc71df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc71df0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2628001f70 tx=0x7f2628001510 comp rx=0 tx=0).stop 2023-10-08T06:53:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 1 -- 172.21.15.70:0/626650571 shutdown_connections 2023-10-08T06:53:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc9cc00 0x5556afc9e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.70:6832/3790014600,v1:172.21.15.70:6834/3790014600] conn(0x5556afc861d0 0x5556afcbbe90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 0x5556afc9c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 1 --2- 172.21.15.70:0/626650571 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc80b60 0x5556afc71df0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f26177fe640 0 client.9756 ms_handle_reset on v2:172.21.15.70:6832/3790014600 2023-10-08T06:53:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 1 -- 172.21.15.70:0/626650571 >> 172.21.15.70:0/626650571 conn(0x5556afc53fa0 msgr2=0x5556afc57b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 1 -- 172.21.15.70:0/626650571 shutdown_connections 2023-10-08T06:53:59.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 1 -- 172.21.15.70:0/626650571 wait complete. 2023-10-08T06:53:59.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.514+0000 7f264182b740 20 client.9756 trim_cache size 0 max 16384 2023-10-08T06:53:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.530+0000 7f264182b740 1 Processor -- start 2023-10-08T06:53:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.530+0000 7f264182b740 1 -- start start 2023-10-08T06:53:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.531+0000 7f264182b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80b60 0x5556afc83a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.531+0000 7f264182b740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 0x5556afca1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.531+0000 7f264182b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afca2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.531+0000 7f264182b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556afc7fc70 con 0x5556afc9cc00 2023-10-08T06:53:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.531+0000 7f264182b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556afc57300 con 0x5556afc80b60 2023-10-08T06:53:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.531+0000 7f264182b740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556afc9aba0 con 0x5556afc80f40 2023-10-08T06:53:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.531+0000 7f263d162640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afca2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.531+0000 7f263d162640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afca2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39266/0 (socket says 172.21.15.70:39266) 2023-10-08T06:53:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.531+0000 7f263d162640 1 -- 172.21.15.70:0/1725951184 learned_addr learned my addr 172.21.15.70:0/1725951184 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.532+0000 7f263d162640 1 -- 172.21.15.70:0/1725951184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 msgr2=0x5556afca1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.532+0000 7f263d162640 1 --2- 172.21.15.70:0/1725951184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 0x5556afca1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.532+0000 7f263d162640 1 -- 172.21.15.70:0/1725951184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80b60 msgr2=0x5556afc83a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.532+0000 7f263d162640 1 --2- 172.21.15.70:0/1725951184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80b60 0x5556afc83a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.532+0000 7f263d162640 1 -- 172.21.15.70:0/1725951184 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556afb5db80 con 0x5556afc9cc00 2023-10-08T06:53:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.532+0000 7f263d162640 1 --2- 172.21.15.70:0/1725951184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afca2340 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f26380cd6e0 tx=0x7f263000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492442da32b232a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.532+0000 7f262effd640 1 -- 172.21.15.70:0/1725951184 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26300125f0 con 0x5556afc9cc00 2023-10-08T06:53:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.532+0000 7f262effd640 1 -- 172.21.15.70:0/1725951184 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2630012750 con 0x5556afc9cc00 2023-10-08T06:53:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.532+0000 7f262effd640 1 -- 172.21.15.70:0/1725951184 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630012b00 con 0x5556afc9cc00 2023-10-08T06:53:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.533+0000 7f264182b740 1 -- 172.21.15.70:0/1725951184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 msgr2=0x5556afca2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.533+0000 7f264182b740 1 --2- 172.21.15.70:0/1725951184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afca2340 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f26380cd6e0 tx=0x7f263000b900 comp rx=0 tx=0).stop 2023-10-08T06:53:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.533+0000 7f264182b740 1 -- 172.21.15.70:0/1725951184 shutdown_connections 2023-10-08T06:53:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.533+0000 7f264182b740 1 --2- 172.21.15.70:0/1725951184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afca2340 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.533+0000 7f264182b740 1 --2- 172.21.15.70:0/1725951184 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 0x5556afca1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.533+0000 7f264182b740 1 --2- 172.21.15.70:0/1725951184 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80b60 0x5556afc83a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.533+0000 7f264182b740 1 -- 172.21.15.70:0/1725951184 >> 172.21.15.70:0/1725951184 conn(0x5556afc53fa0 msgr2=0x5556afc868d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:59.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.533+0000 7f264182b740 1 -- 172.21.15.70:0/1725951184 shutdown_connections 2023-10-08T06:53:59.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.533+0000 7f264182b740 1 -- 172.21.15.70:0/1725951184 wait complete. 2023-10-08T06:53:59.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.534+0000 7f264182b740 1 -- start start 2023-10-08T06:53:59.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.534+0000 7f264182b740 1 Processor -- start 2023-10-08T06:53:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.535+0000 7f264182b740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80b60 0x5556afd079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.535+0000 7f264182b740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 0x5556afcaca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.535+0000 7f264182b740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afcacf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.535+0000 7f264182b740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556afca00b0 con 0x5556afc9cc00 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.535+0000 7f264182b740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556afcadd40 con 0x5556afc80b60 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.535+0000 7f264182b740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5556afc4f380 con 0x5556afc80f40 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.535+0000 7f264182b740 10 client.0 _ll_register_callbacks cb 0x7f26240036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.535+0000 7f263d162640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afcacf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.535+0000 7f263d162640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afcacf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39274/0 (socket says 172.21.15.70:39274) 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.535+0000 7f263d162640 1 -- 172.21.15.70:0/2507317163 learned_addr learned my addr 172.21.15.70:0/2507317163 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f263d162640 1 -- 172.21.15.70:0/2507317163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 msgr2=0x5556afcaca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f263d162640 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 0x5556afcaca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f263d162640 1 -- 172.21.15.70:0/2507317163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80b60 msgr2=0x5556afd079a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f263d162640 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80b60 0x5556afd079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f263d162640 1 -- 172.21.15.70:0/2507317163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263000e040 con 0x5556afc9cc00 2023-10-08T06:53:59.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f263d162640 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afcacf60 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f2630008010 tx=0x7f2630006a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c74fc737712d91 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:59.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f26177fe640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T06:53:59.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f26177fe640 1 -- 172.21.15.70:0/2507317163 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630006dd0 con 0x5556afc9cc00 2023-10-08T06:53:59.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f26177fe640 1 -- 172.21.15.70:0/2507317163 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f2630004370 con 0x5556afc9cc00 2023-10-08T06:53:59.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f264182b740 1 -- 172.21.15.70:0/2507317163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2628006b50 con 0x5556afc9cc00 2023-10-08T06:53:59.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f26177fe640 1 -- 172.21.15.70:0/2507317163 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26300128c0 con 0x5556afc9cc00 2023-10-08T06:53:59.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f264182b740 10 client.9762 fetch_fsmap learned FSMap version 83 2023-10-08T06:53:59.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.536+0000 7f264182b740 1 -- 172.21.15.70:0/2507317163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5556afb83170 con 0x5556afc9cc00 2023-10-08T06:53:59.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f26177fe640 1 -- 172.21.15.70:0/2507317163 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628006b50 con 0x5556afc9cc00 2023-10-08T06:53:59.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f26177fe640 1 -- 172.21.15.70:0/2507317163 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2630012a20 con 0x5556afc9cc00 2023-10-08T06:53:59.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f264182b740 10 client.9762 fetch_fsmap finished waiting for FSMap version 83 2023-10-08T06:53:59.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f264182b740 10 client.9762 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T06:53:59.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f264182b740 20 client.9762 populate_metadata read hostname 'smithi070' 2023-10-08T06:53:59.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f264182b740 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.70:6832/3790014600,v1:172.21.15.70:6834/3790014600] conn(0x5556afc9bef0 0x5556afd0a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f264182b740 4 client.9762 mds_command: new command op to 9591 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T06:53:59.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f264182b740 1 -- 172.21.15.70:0/2507317163 --> [v2:172.21.15.70:6832/3790014600,v1:172.21.15.70:6834/3790014600] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5556af8c26c0 con 0x5556afc9bef0 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f263c961640 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.70:6832/3790014600,v1:172.21.15.70:6834/3790014600] conn(0x5556afc9bef0 0x5556afd0a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f26177fe640 10 client.9762 ms_handle_connect on v2:172.21.15.70:6832/3790014600 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.537+0000 7f263c961640 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.70:6832/3790014600,v1:172.21.15.70:6834/3790014600] conn(0x5556afc9bef0 0x5556afd0a550 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3be391a4c3ec34 server_cookie=b5a5d1f35c611267 in_seq=0 out_seq=0 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f26177fe640 1 -- 172.21.15.70:0/2507317163 <== mds.0 v2:172.21.15.70:6832/3790014600 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5556af8c26c0 con 0x5556afc9bef0 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f26177fe640 10 client.9762 handle_command_reply: tid=0 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 client.9762 shutdown 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 -- 172.21.15.70:0/2507317163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 msgr2=0x5556afcacf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afcacf60 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f2630008010 tx=0x7f2630006a20 comp rx=0 tx=0).stop 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 -- 172.21.15.70:0/2507317163 shutdown_connections 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5556afc9cc00 0x5556afcacf60 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.70:6832/3790014600,v1:172.21.15.70:6834/3790014600] conn(0x5556afc9bef0 0x5556afd0a550 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5556afc80f40 0x5556afcaca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 --2- 172.21.15.70:0/2507317163 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5556afc80b60 0x5556afd079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f26177fe640 0 client.9762 ms_handle_reset on v2:172.21.15.70:6832/3790014600 2023-10-08T06:53:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 -- 172.21.15.70:0/2507317163 >> 172.21.15.70:0/2507317163 conn(0x5556afc53fa0 msgr2=0x5556afc57b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 -- 172.21.15.70:0/2507317163 shutdown_connections 2023-10-08T06:53:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 1 -- 172.21.15.70:0/2507317163 wait complete. 2023-10-08T06:53:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.538+0000 7f264182b740 20 client.9762 trim_cache size 0 max 16384 2023-10-08T06:53:59.545 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T06:53:59.545 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 9591, 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 11.691374100999999, 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 83, 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 185, 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 185, 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 20.600252060999999 2023-10-08T06:53:59.546 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T06:53:59.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 -- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f261806af90 msgr2=0x7f261806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f261806af90 0x7f261806d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2628034000 tx=0x7f2628030000 comp rx=0 tx=0).stop 2023-10-08T06:53:59.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 -- 172.21.15.70:0/2048954529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380b46b0 msgr2=0x7f263815b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380b46b0 0x7f263815b0d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f262000c730 tx=0x7f262000cbf0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 -- 172.21.15.70:0/2048954529 shutdown_connections 2023-10-08T06:53:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26380b4b30 0x7f263815b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f261806af90 0x7f261806d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26380b46b0 0x7f263815b0d0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 --2- 172.21.15.70:0/2048954529 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26240039a0 0x7f26381435f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 -- 172.21.15.70:0/2048954529 >> 172.21.15.70:0/2048954529 conn(0x7f26380b0af0 msgr2=0x7f26380c54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 -- 172.21.15.70:0/2048954529 shutdown_connections 2023-10-08T06:53:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.539+0000 7f263ebc5640 1 -- 172.21.15.70:0/2048954529 wait complete. 2023-10-08T06:53:59.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T06:53:59.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.759+0000 7fdf3b59e640 1 Processor -- start 2023-10-08T06:53:59.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.759+0000 7fdf3b59e640 1 -- start start 2023-10-08T06:53:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.760+0000 7fdf3b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf3c15caa0 0x7fdf3c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.760+0000 7fdf3b59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c1614b0 con 0x7fdf3c0cc460 2023-10-08T06:53:59.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.760+0000 7fdf3b59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c0cc8f0 con 0x7fdf3c0d0230 2023-10-08T06:53:59.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.760+0000 7fdf3b59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c0ccbd0 con 0x7fdf3c15caa0 2023-10-08T06:53:59.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.760+0000 7fdf3a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf3c15caa0 0x7fdf3c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.760+0000 7fdf39d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdf3c0cc460 0x7fdf3c160690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50700/0 (socket says 172.21.15.70:50700) 2023-10-08T06:53:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.760+0000 7fdf39d9b640 1 -- 172.21.15.70:0/3456927799 learned_addr learned my addr 172.21.15.70:0/3456927799 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:53:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.761+0000 7fdf3959a640 1 -- 172.21.15.70:0/3456927799 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636996009 0 0) 0x7fdf3c1614b0 con 0x7fdf3c0cc460 2023-10-08T06:53:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.761+0000 7fdf3959a640 1 -- 172.21.15.70:0/3456927799 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf28003520 con 0x7fdf3c0cc460 2023-10-08T06:53:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.761+0000 7fdf3a59c640 1 -- 172.21.15.70:0/3456927799 >> v1:172.21.15.181:6789/0 conn(0x7fdf3c0d0230 legacy=0x7fdf3c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:53:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.761+0000 7fdf3a59c640 1 -- 172.21.15.70:0/3456927799 >> v1:172.21.15.70:6789/0 conn(0x7fdf3c0cc460 legacy=0x7fdf3c160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.761+0000 7fdf3a59c640 1 -- 172.21.15.70:0/3456927799 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0cf750 con 0x7fdf3c15caa0 2023-10-08T06:53:59.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.762+0000 7fdf3a59c640 1 --2- 172.21.15.70:0/3456927799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c15caa0 0x7fdf3c069d50 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0097a0 tx=0x7fdf2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2f7c48265584ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:59.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.763+0000 7fdf3959a640 1 -- 172.21.15.70:0/3456927799 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c031830 con 0x7fdf3c15caa0 2023-10-08T06:53:59.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.763+0000 7fdf3959a640 1 -- 172.21.15.70:0/3456927799 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c031990 con 0x7fdf3c15caa0 2023-10-08T06:53:59.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.763+0000 7fdf3959a640 1 -- 172.21.15.70:0/3456927799 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c031d00 con 0x7fdf3c15caa0 2023-10-08T06:53:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.763+0000 7fdf3b59e640 1 -- 172.21.15.70:0/3456927799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c15caa0 msgr2=0x7fdf3c069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.763+0000 7fdf3b59e640 1 --2- 172.21.15.70:0/3456927799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c15caa0 0x7fdf3c069d50 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0097a0 tx=0x7fdf2c02b8d0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.763+0000 7fdf3b59e640 1 -- 172.21.15.70:0/3456927799 shutdown_connections 2023-10-08T06:53:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.763+0000 7fdf3b59e640 1 --2- 172.21.15.70:0/3456927799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c15caa0 0x7fdf3c069d50 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.763+0000 7fdf3b59e640 1 -- 172.21.15.70:0/3456927799 >> 172.21.15.70:0/3456927799 conn(0x7fdf3c06ae60 msgr2=0x7fdf3c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:53:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.763+0000 7fdf3b59e640 1 -- 172.21.15.70:0/3456927799 shutdown_connections 2023-10-08T06:53:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.763+0000 7fdf3b59e640 1 -- 172.21.15.70:0/3456927799 wait complete. 2023-10-08T06:53:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.765+0000 7fdf3b59e640 1 Processor -- start 2023-10-08T06:53:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.765+0000 7fdf3b59e640 1 -- start start 2023-10-08T06:53:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.766+0000 7fdf3b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf3c0d0230 0x7fdf3c154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.766+0000 7fdf3b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf3c15caa0 0x7fdf3c14acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.766+0000 7fdf3b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c0c7da0 0x7fdf3c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.766+0000 7fdf3b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c155620 con 0x7fdf3c0c7da0 2023-10-08T06:53:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.766+0000 7fdf3b59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c14b210 con 0x7fdf3c0d0230 2023-10-08T06:53:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.766+0000 7fdf3b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0c9200 con 0x7fdf3c15caa0 2023-10-08T06:53:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf3ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c0c7da0 0x7fdf3c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf3ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c0c7da0 0x7fdf3c1596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39296/0 (socket says 172.21.15.70:39296) 2023-10-08T06:53:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf3ad9d640 1 -- 172.21.15.70:0/2221104190 learned_addr learned my addr 172.21.15.70:0/2221104190 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:53:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf39d9b640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf3c15caa0 0x7fdf3c14acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf3a59c640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf3c0d0230 0x7fdf3c154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf3ad9d640 1 -- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf3c15caa0 msgr2=0x7fdf3c14acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf3ad9d640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf3c15caa0 0x7fdf3c14acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf3ad9d640 1 -- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf3c0d0230 msgr2=0x7fdf3c154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:53:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf3ad9d640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf3c0d0230 0x7fdf3c154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:53:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf3ad9d640 1 -- 172.21.15.70:0/2221104190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c009400 con 0x7fdf3c0c7da0 2023-10-08T06:53:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.767+0000 7fdf3a59c640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf3c0d0230 0x7fdf3c154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:53:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.768+0000 7fdf3ad9d640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c0c7da0 0x7fdf3c1596f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400c780 tx=0x7fdf3400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.768+0000 7fdf32ffd640 1 -- 172.21.15.70:0/2221104190 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34004b60 con 0x7fdf3c0c7da0 2023-10-08T06:53:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.768+0000 7fdf32ffd640 1 -- 172.21.15.70:0/2221104190 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3400ce90 con 0x7fdf3c0c7da0 2023-10-08T06:53:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.768+0000 7fdf32ffd640 1 -- 172.21.15.70:0/2221104190 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400a850 con 0x7fdf3c0c7da0 2023-10-08T06:53:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.768+0000 7fdf3b59e640 1 -- 172.21.15.70:0/2221104190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c0c85f0 con 0x7fdf3c0c7da0 2023-10-08T06:53:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.768+0000 7fdf3b59e640 1 -- 172.21.15.70:0/2221104190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c0c8bb0 con 0x7fdf3c0c7da0 2023-10-08T06:53:59.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.769+0000 7fdf3b59e640 1 -- 172.21.15.70:0/2221104190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c04e160 con 0x7fdf3c0c7da0 2023-10-08T06:53:59.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.770+0000 7fdf32ffd640 1 -- 172.21.15.70:0/2221104190 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdf3400a9b0 con 0x7fdf3c0c7da0 2023-10-08T06:53:59.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.771+0000 7fdf32ffd640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf1c06afc0 0x7fdf1c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:53:59.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.771+0000 7fdf32ffd640 1 -- 172.21.15.70:0/2221104190 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fdf34089b60 con 0x7fdf3c0c7da0 2023-10-08T06:53:59.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.772+0000 7fdf3a59c640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf1c06afc0 0x7fdf1c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:53:59.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.773+0000 7fdf3a59c640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf1c06afc0 0x7fdf1c06d480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c036000 tx=0x7fdf2c008030 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:53:59.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:53:59.776+0000 7fdf32ffd640 1 -- 172.21.15.70:0/2221104190 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf340548a0 con 0x7fdf3c0c7da0 2023-10-08T06:54:00.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.005+0000 7fdf3b59e640 1 -- 172.21.15.70:0/2221104190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdf3c0c9340 con 0x7fdf3c0c7da0 2023-10-08T06:54:00.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.006+0000 7fdf32ffd640 1 -- 172.21.15.70:0/2221104190 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdf34057ec0 con 0x7fdf3c0c7da0 2023-10-08T06:54:00.006 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T06:54:00.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.008+0000 7fdf3b59e640 1 -- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf1c06afc0 msgr2=0x7fdf1c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:00.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.008+0000 7fdf3b59e640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf1c06afc0 0x7fdf1c06d480 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c036000 tx=0x7fdf2c008030 comp rx=0 tx=0).stop 2023-10-08T06:54:00.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.008+0000 7fdf3b59e640 1 -- 172.21.15.70:0/2221104190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c0c7da0 msgr2=0x7fdf3c1596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:00.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.008+0000 7fdf3b59e640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c0c7da0 0x7fdf3c1596f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400c780 tx=0x7fdf3400cc40 comp rx=0 tx=0).stop 2023-10-08T06:54:00.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.009+0000 7fdf3b59e640 1 -- 172.21.15.70:0/2221104190 shutdown_connections 2023-10-08T06:54:00.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.009+0000 7fdf3b59e640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf1c06afc0 0x7fdf1c06d480 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:00.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.009+0000 7fdf3b59e640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf3c0c7da0 0x7fdf3c1596f0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:00.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.009+0000 7fdf3b59e640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf3c15caa0 0x7fdf3c14acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:00.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.009+0000 7fdf3b59e640 1 --2- 172.21.15.70:0/2221104190 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf3c0d0230 0x7fdf3c154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:00.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.009+0000 7fdf3b59e640 1 -- 172.21.15.70:0/2221104190 >> 172.21.15.70:0/2221104190 conn(0x7fdf3c06ae60 msgr2=0x7fdf3c0ab8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:00.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.009+0000 7fdf3b59e640 1 -- 172.21.15.70:0/2221104190 shutdown_connections 2023-10-08T06:54:00.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:00.009+0000 7fdf3b59e640 1 -- 172.21.15.70:0/2221104190 wait complete. 2023-10-08T06:54:00.029 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T06:54:00.029 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T06:54:00.029 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T06:54:00.029 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T06:54:00.029 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T06:54:00.029 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T06:54:00.029 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T06:54:00.029 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T06:54:00.029 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T06:54:00.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:00.030 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T06:54:00.088 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T06:54:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T06:54:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T06:54:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T06:54:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:00.089 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T06:54:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T06:54:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T06:54:00.089 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T06:54:00.089 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 37756sec preferred_lft 37756sec 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T06:54:00.090 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T06:54:00.091 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:00.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:00.092 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T06:54:00.182 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T06:54:00.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:00.187 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T06:54:00.240 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T06:54:00.241 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T06:54:00.241 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T06:54:00.241 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T06:54:00.241 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T06:54:00.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:00.241 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T06:54:00.241 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T06:54:00.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T06:54:00.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T06:54:00.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T06:54:00.242 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T06:54:00.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:00.429 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T06:54:00.479 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T06:54:00.479 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T06:54:00.480 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T06:54:00.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:00.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T06:54:00.575 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T06:54:00.575 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T06:54:00.628 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T06:54:00.629 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T06:54:00.629 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T06:54:00.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:54:00.681 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:54:00.682 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T06:54:00.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:00.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T06:54:00.849 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T06:54:00.849 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T06:54:00.881 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T06:54:00.881 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T06:54:00.881 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T06:54:00.882 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T06:54:00.882 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T06:54:00.883 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T06:54:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T06:54:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T06:54:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T06:54:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T06:54:00.887 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T06:54:00.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:00.887 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T06:54:00.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:00.968 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T06:54:01.037 INFO:teuthology.orchestra.run.smithi070.stdout:client9774 2023-10-08T06:54:01.037 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:54:01.037 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:01.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T06:54:01.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.348+0000 7fc279de7640 1 Processor -- start 2023-10-08T06:54:01.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.349+0000 7fc279de7640 1 -- start start 2023-10-08T06:54:01.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.350+0000 7fc279de7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc27415cdb0 0x7fc2740cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.350+0000 7fc279de7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc27415d190 0x7fc27406af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.350+0000 7fc279de7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27415d5f0 con 0x7fc2740ccb40 2023-10-08T06:54:01.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.350+0000 7fc279de7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2740b19e0 con 0x7fc27415d190 2023-10-08T06:54:01.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.350+0000 7fc279de7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc274069ef0 con 0x7fc27415cdb0 2023-10-08T06:54:01.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.351+0000 7fc272ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc27415d190 0x7fc27406af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.351+0000 7fc273fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc2740ccb40 0x7fc27406cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50704/0 (socket says 172.21.15.70:50704) 2023-10-08T06:54:01.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.351+0000 7fc273fff640 1 -- 172.21.15.70:0/3228289387 learned_addr learned my addr 172.21.15.70:0/3228289387 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:01.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.351+0000 7fc2737fe640 1 --2- 172.21.15.70:0/3228289387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc27415cdb0 0x7fc2740cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.351+0000 7fc2727fc640 1 -- 172.21.15.70:0/3228289387 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844190039 0 0) 0x7fc27415d5f0 con 0x7fc2740ccb40 2023-10-08T06:54:01.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.351+0000 7fc272ffd640 1 -- 172.21.15.70:0/3228289387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc27415cdb0 msgr2=0x7fc2740cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:01.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.351+0000 7fc272ffd640 1 --2- 172.21.15.70:0/3228289387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc27415cdb0 0x7fc2740cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.351+0000 7fc272ffd640 1 -- 172.21.15.70:0/3228289387 >> v1:172.21.15.70:6789/0 conn(0x7fc2740ccb40 legacy=0x7fc27406cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:01.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.351+0000 7fc272ffd640 1 -- 172.21.15.70:0/3228289387 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27406a1d0 con 0x7fc27415d190 2023-10-08T06:54:01.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.352+0000 7fc272ffd640 1 --2- 172.21.15.70:0/3228289387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415d190 0x7fc27406af90 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0097a0 tx=0x7fc25c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beba3e4f6fbb824f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:01.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc2727fc640 1 -- 172.21.15.70:0/3228289387 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c02d040 con 0x7fc27415d190 2023-10-08T06:54:01.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc2727fc640 1 -- 172.21.15.70:0/3228289387 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c031810 con 0x7fc27415d190 2023-10-08T06:54:01.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc2727fc640 1 -- 172.21.15.70:0/3228289387 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c031ba0 con 0x7fc27415d190 2023-10-08T06:54:01.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc279de7640 1 -- 172.21.15.70:0/3228289387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415d190 msgr2=0x7fc27406af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:01.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc279de7640 1 --2- 172.21.15.70:0/3228289387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415d190 0x7fc27406af90 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0097a0 tx=0x7fc25c02b8b0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc279de7640 1 -- 172.21.15.70:0/3228289387 shutdown_connections 2023-10-08T06:54:01.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc279de7640 1 --2- 172.21.15.70:0/3228289387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415d190 0x7fc27406af90 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc279de7640 1 --2- 172.21.15.70:0/3228289387 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc27415cdb0 0x7fc2740cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc279de7640 1 -- 172.21.15.70:0/3228289387 >> 172.21.15.70:0/3228289387 conn(0x7fc2740b0af0 msgr2=0x7fc2740b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:01.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc279de7640 1 -- 172.21.15.70:0/3228289387 shutdown_connections 2023-10-08T06:54:01.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.353+0000 7fc279de7640 1 -- 172.21.15.70:0/3228289387 wait complete. 2023-10-08T06:54:01.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.355+0000 7fc279de7640 1 Processor -- start 2023-10-08T06:54:01.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.355+0000 7fc279de7640 1 -- start start 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.356+0000 7fc279de7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2740ccb40 0x7fc2741488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.356+0000 7fc279de7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415cdb0 0x7fc2740c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.356+0000 7fc279de7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc27415d190 0x7fc2741454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.356+0000 7fc279de7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740c49a0 con 0x7fc27415cdb0 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.356+0000 7fc279de7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc274148e10 con 0x7fc27415d190 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.356+0000 7fc279de7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740c3af0 con 0x7fc2740ccb40 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.356+0000 7fc272ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415cdb0 0x7fc2740c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc272ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415cdb0 0x7fc2740c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39326/0 (socket says 172.21.15.70:39326) 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc272ffd640 1 -- 172.21.15.70:0/2820196128 learned_addr learned my addr 172.21.15.70:0/2820196128 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc2737fe640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2740ccb40 0x7fc2741488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc273fff640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc27415d190 0x7fc2741454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc272ffd640 1 -- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2740ccb40 msgr2=0x7fc2741488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc272ffd640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2740ccb40 0x7fc2741488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc272ffd640 1 -- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc27415d190 msgr2=0x7fc2741454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc272ffd640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc27415d190 0x7fc2741454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc272ffd640 1 -- 172.21.15.70:0/2820196128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c009400 con 0x7fc27415cdb0 2023-10-08T06:54:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc272ffd640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415cdb0 0x7fc2740c4e30 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fc25c02bec0 tx=0x7fc25c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:01.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.357+0000 7fc26bfff640 1 -- 172.21.15.70:0/2820196128 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c03ad60 con 0x7fc27415cdb0 2023-10-08T06:54:01.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.358+0000 7fc26bfff640 1 -- 172.21.15.70:0/2820196128 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c0423f0 con 0x7fc27415cdb0 2023-10-08T06:54:01.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.358+0000 7fc279de7640 1 -- 172.21.15.70:0/2820196128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2740c3d90 con 0x7fc27415cdb0 2023-10-08T06:54:01.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.358+0000 7fc26bfff640 1 -- 172.21.15.70:0/2820196128 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0427a0 con 0x7fc27415cdb0 2023-10-08T06:54:01.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.358+0000 7fc279de7640 1 -- 172.21.15.70:0/2820196128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2740c4350 con 0x7fc27415cdb0 2023-10-08T06:54:01.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.359+0000 7fc279de7640 1 -- 172.21.15.70:0/2820196128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27404e160 con 0x7fc27415cdb0 2023-10-08T06:54:01.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.360+0000 7fc26bfff640 1 -- 172.21.15.70:0/2820196128 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc25c042900 con 0x7fc27415cdb0 2023-10-08T06:54:01.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.361+0000 7fc26bfff640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc25406afc0 0x7fc25406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.362+0000 7fc26bfff640 1 -- 172.21.15.70:0/2820196128 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc25c0aba80 con 0x7fc27415cdb0 2023-10-08T06:54:01.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.362+0000 7fc2737fe640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc25406afc0 0x7fc25406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.363+0000 7fc2737fe640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc25406afc0 0x7fc25406d480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc264009f30 tx=0x7fc26400b040 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:01.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.366+0000 7fc26bfff640 1 -- 172.21.15.70:0/2820196128 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc25c0766b0 con 0x7fc27415cdb0 2023-10-08T06:54:01.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.583+0000 7fc279de7640 1 -- 172.21.15.70:0/2820196128 --> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc27406a790 con 0x7fc25406afc0 2023-10-08T06:54:01.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.588+0000 7fc26bfff640 1 -- 172.21.15.70:0/2820196128 <== mgr.9345 v2:172.21.15.181:6832/4245513204 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc264009b20 con 0x7fc25406afc0 2023-10-08T06:54:01.589 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T06:54:01.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:01.589 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T06:54:01.589 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:01.589 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T06:54:01.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.589+0000 7fc279de7640 1 -- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc25406afc0 msgr2=0x7fc25406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:01.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.589+0000 7fc279de7640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc25406afc0 0x7fc25406d480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc264009f30 tx=0x7fc26400b040 comp rx=0 tx=0).stop 2023-10-08T06:54:01.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.589+0000 7fc279de7640 1 -- 172.21.15.70:0/2820196128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415cdb0 msgr2=0x7fc2740c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:01.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.589+0000 7fc279de7640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415cdb0 0x7fc2740c4e30 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fc25c02bec0 tx=0x7fc25c0046c0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.590+0000 7fc279de7640 1 -- 172.21.15.70:0/2820196128 shutdown_connections 2023-10-08T06:54:01.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.590+0000 7fc279de7640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc27415d190 0x7fc2741454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.590+0000 7fc279de7640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fc25406afc0 0x7fc25406d480 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.590+0000 7fc279de7640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc27415cdb0 0x7fc2740c4e30 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.590+0000 7fc279de7640 1 --2- 172.21.15.70:0/2820196128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc2740ccb40 0x7fc2741488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.590+0000 7fc279de7640 1 -- 172.21.15.70:0/2820196128 >> 172.21.15.70:0/2820196128 conn(0x7fc2740b0af0 msgr2=0x7fc2740abd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:01.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.590+0000 7fc279de7640 1 -- 172.21.15.70:0/2820196128 shutdown_connections 2023-10-08T06:54:01.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.590+0000 7fc279de7640 1 -- 172.21.15.70:0/2820196128 wait complete. 2023-10-08T06:54:01.609 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T06:54:01.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.824+0000 7fe7765ed640 1 Processor -- start 2023-10-08T06:54:01.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.824+0000 7fe7765ed640 1 -- start start 2023-10-08T06:54:01.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.826+0000 7fe7765ed640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe77015aab0 0x7fe770070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.826+0000 7fe7765ed640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7700cb350 0x7fe7700b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.826+0000 7fe7765ed640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77015ae90 con 0x7fe7700caef0 2023-10-08T06:54:01.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.826+0000 7fe7765ed640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7700cbdb0 con 0x7fe7700cb350 2023-10-08T06:54:01.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.826+0000 7fe7765ed640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700cb7f0 con 0x7fe77015aab0 2023-10-08T06:54:01.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.826+0000 7fe774dea640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe7700caef0 0x7fe7700aff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50706/0 (socket says 172.21.15.70:50706) 2023-10-08T06:54:01.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.827+0000 7fe774dea640 1 -- 172.21.15.70:0/2806364267 learned_addr learned my addr 172.21.15.70:0/2806364267 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:01.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.827+0000 7fe775dec640 1 --2- 172.21.15.70:0/2806364267 >> v2:172.21.15.70:3300/0 conn(0x7fe7700cb350 0x7fe7700b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.827+0000 7fe7755eb640 1 --2- 172.21.15.70:0/2806364267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe77015aab0 0x7fe770070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.827+0000 7fe767fff640 1 -- 172.21.15.70:0/2806364267 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239365962 0 0) 0x7fe760001110 con 0x7fe7700caef0 2023-10-08T06:54:01.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.827+0000 7fe767fff640 1 -- 172.21.15.70:0/2806364267 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe75c003520 con 0x7fe7700caef0 2023-10-08T06:54:01.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.827+0000 7fe775dec640 1 -- 172.21.15.70:0/2806364267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe77015aab0 msgr2=0x7fe770070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:01.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.827+0000 7fe775dec640 1 --2- 172.21.15.70:0/2806364267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe77015aab0 0x7fe770070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.827+0000 7fe775dec640 1 -- 172.21.15.70:0/2806364267 >> v1:172.21.15.70:6789/0 conn(0x7fe7700caef0 legacy=0x7fe7700aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:01.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.827+0000 7fe775dec640 1 -- 172.21.15.70:0/2806364267 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7700cbad0 con 0x7fe7700cb350 2023-10-08T06:54:01.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.828+0000 7fe775dec640 1 --2- 172.21.15.70:0/2806364267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 0x7fe7700b1c60 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe768009d90 tx=0x7fe76802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272b27fce7f0b19c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:01.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.829+0000 7fe767fff640 1 -- 172.21.15.70:0/2806364267 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76802cbd0 con 0x7fe7700cb350 2023-10-08T06:54:01.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.829+0000 7fe767fff640 1 -- 172.21.15.70:0/2806364267 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76802cd30 con 0x7fe7700cb350 2023-10-08T06:54:01.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.829+0000 7fe767fff640 1 -- 172.21.15.70:0/2806364267 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76802cea0 con 0x7fe7700cb350 2023-10-08T06:54:01.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.829+0000 7fe7765ed640 1 -- 172.21.15.70:0/2806364267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 msgr2=0x7fe7700b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:01.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.829+0000 7fe7765ed640 1 --2- 172.21.15.70:0/2806364267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 0x7fe7700b1c60 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe768009d90 tx=0x7fe76802be20 comp rx=0 tx=0).stop 2023-10-08T06:54:01.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.829+0000 7fe7765ed640 1 -- 172.21.15.70:0/2806364267 shutdown_connections 2023-10-08T06:54:01.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.829+0000 7fe7765ed640 1 --2- 172.21.15.70:0/2806364267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 0x7fe7700b1c60 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.829+0000 7fe7765ed640 1 --2- 172.21.15.70:0/2806364267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe77015aab0 0x7fe770070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.829+0000 7fe7765ed640 1 -- 172.21.15.70:0/2806364267 >> 172.21.15.70:0/2806364267 conn(0x7fe77006c010 msgr2=0x7fe77006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.829+0000 7fe7765ed640 1 -- 172.21.15.70:0/2806364267 shutdown_connections 2023-10-08T06:54:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.830+0000 7fe7765ed640 1 -- 172.21.15.70:0/2806364267 wait complete. 2023-10-08T06:54:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.831+0000 7fe7765ed640 1 Processor -- start 2023-10-08T06:54:01.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.831+0000 7fe7765ed640 1 -- start start 2023-10-08T06:54:01.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.832+0000 7fe7765ed640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7700caef0 0x7fe770150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.832+0000 7fe7765ed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 0x7fe7700c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.832+0000 7fe7765ed640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe77015aab0 0x7fe77015c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.832+0000 7fe7765ed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c6fb0 con 0x7fe7700cb350 2023-10-08T06:54:01.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.832+0000 7fe7765ed640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c63a0 con 0x7fe77015aab0 2023-10-08T06:54:01.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.832+0000 7fe7765ed640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c6680 con 0x7fe7700caef0 2023-10-08T06:54:01.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.833+0000 7fe774dea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 0x7fe7700c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.833+0000 7fe774dea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 0x7fe7700c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39346/0 (socket says 172.21.15.70:39346) 2023-10-08T06:54:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.833+0000 7fe774dea640 1 -- 172.21.15.70:0/3547541217 learned_addr learned my addr 172.21.15.70:0/3547541217 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.833+0000 7fe7755eb640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7700caef0 0x7fe770150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.833+0000 7fe775dec640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe77015aab0 0x7fe77015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.833+0000 7fe774dea640 1 -- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7700caef0 msgr2=0x7fe770150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.833+0000 7fe774dea640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7700caef0 0x7fe770150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.833+0000 7fe774dea640 1 -- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe77015aab0 msgr2=0x7fe77015c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.833+0000 7fe774dea640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe77015aab0 0x7fe77015c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.833+0000 7fe774dea640 1 -- 172.21.15.70:0/3547541217 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768009a70 con 0x7fe7700cb350 2023-10-08T06:54:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.834+0000 7fe774dea640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 0x7fe7700c7990 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c420 tx=0x7fe76000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:01.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.834+0000 7fe765ffb640 1 -- 172.21.15.70:0/3547541217 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760005ce0 con 0x7fe7700cb350 2023-10-08T06:54:01.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.834+0000 7fe765ffb640 1 -- 172.21.15.70:0/3547541217 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760005e40 con 0x7fe7700cb350 2023-10-08T06:54:01.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.834+0000 7fe765ffb640 1 -- 172.21.15.70:0/3547541217 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600072f0 con 0x7fe7700cb350 2023-10-08T06:54:01.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.834+0000 7fe7765ed640 1 -- 172.21.15.70:0/3547541217 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7700c6960 con 0x7fe7700cb350 2023-10-08T06:54:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.834+0000 7fe7765ed640 1 -- 172.21.15.70:0/3547541217 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe770151420 con 0x7fe7700cb350 2023-10-08T06:54:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.835+0000 7fe7765ed640 1 -- 172.21.15.70:0/3547541217 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7700c7120 con 0x7fe7700cb350 2023-10-08T06:54:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.836+0000 7fe765ffb640 1 -- 172.21.15.70:0/3547541217 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 89) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe760007510 con 0x7fe7700cb350 2023-10-08T06:54:01.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.837+0000 7fe765ffb640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe75406af90 0x7fe75406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:01.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.837+0000 7fe765ffb640 1 -- 172.21.15.70:0/3547541217 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe760088d40 con 0x7fe7700cb350 2023-10-08T06:54:01.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.838+0000 7fe7755eb640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe75406af90 0x7fe75406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:01.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.839+0000 7fe7755eb640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe75406af90 0x7fe75406d450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009b40 tx=0x7fe76c009190 comp rx=0 tx=0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:01.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:01.842+0000 7fe765ffb640 1 -- 172.21.15.70:0/3547541217 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe760053bc0 con 0x7fe7700cb350 2023-10-08T06:54:02.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.081+0000 7fe7765ed640 1 -- 172.21.15.70:0/3547541217 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe770074150 con 0x7fe7700cb350 2023-10-08T06:54:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.953+0000 7fe765ffb640 1 -- 172.21.15.70:0/3547541217 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v90) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe760017020 con 0x7fe7700cb350 2023-10-08T06:54:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.956+0000 7fe7765ed640 1 -- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe75406af90 msgr2=0x7fe75406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.956+0000 7fe7765ed640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe75406af90 0x7fe75406d450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009b40 tx=0x7fe76c009190 comp rx=0 tx=0).stop 2023-10-08T06:54:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.956+0000 7fe7765ed640 1 -- 172.21.15.70:0/3547541217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 msgr2=0x7fe7700c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.956+0000 7fe7765ed640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 0x7fe7700c7990 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c420 tx=0x7fe76000c8e0 comp rx=0 tx=0).stop 2023-10-08T06:54:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.956+0000 7fe7765ed640 1 -- 172.21.15.70:0/3547541217 shutdown_connections 2023-10-08T06:54:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.956+0000 7fe7765ed640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe77015aab0 0x7fe77015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.956+0000 7fe7765ed640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fe75406af90 0x7fe75406d450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.956+0000 7fe7765ed640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7700cb350 0x7fe7700c7990 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.956+0000 7fe7765ed640 1 --2- 172.21.15.70:0/3547541217 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe7700caef0 0x7fe770150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.956+0000 7fe7765ed640 1 -- 172.21.15.70:0/3547541217 >> 172.21.15.70:0/3547541217 conn(0x7fe77006c010 msgr2=0x7fe7700afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.957+0000 7fe7765ed640 1 -- 172.21.15.70:0/3547541217 shutdown_connections 2023-10-08T06:54:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:02.957+0000 7fe7765ed640 1 -- 172.21.15.70:0/3547541217 wait complete. 2023-10-08T06:54:02.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T06:54:03.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.184+0000 7f7f59d5a640 1 Processor -- start 2023-10-08T06:54:03.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.184+0000 7f7f59d5a640 1 -- start start 2023-10-08T06:54:03.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.185+0000 7f7f59d5a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f541601b0 0x7f7f540cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.185+0000 7f7f59d5a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7f54071c30 0x7f7f54069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.185+0000 7f7f59d5a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f540ce210 con 0x7f7f54072090 2023-10-08T06:54:03.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.185+0000 7f7f59d5a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f540cdee0 con 0x7f7f54071c30 2023-10-08T06:54:03.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.185+0000 7f7f59d5a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54161530 con 0x7f7f541601b0 2023-10-08T06:54:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.186+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f541601b0 0x7f7f540cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.186+0000 7f7f537fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f541601b0 0x7f7f540cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:34710/0 (socket says 172.21.15.70:34710) 2023-10-08T06:54:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.186+0000 7f7f537fe640 1 -- 172.21.15.70:0/1936661066 learned_addr learned my addr 172.21.15.70:0/1936661066 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.186+0000 7f7f52ffd640 1 --2- 172.21.15.70:0/1936661066 >> v2:172.21.15.181:3300/0 conn(0x7f7f54071c30 0x7f7f54069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.187+0000 7f7f527fc640 1 -- 172.21.15.70:0/1936661066 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499418680 0 0) 0x7f7f540ce210 con 0x7f7f54072090 2023-10-08T06:54:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.187+0000 7f7f527fc640 1 -- 172.21.15.70:0/1936661066 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f3c003520 con 0x7f7f54072090 2023-10-08T06:54:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.187+0000 7f7f537fe640 1 -- 172.21.15.70:0/1936661066 >> v2:172.21.15.181:3300/0 conn(0x7f7f54071c30 msgr2=0x7f7f54069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.187+0000 7f7f537fe640 1 --2- 172.21.15.70:0/1936661066 >> v2:172.21.15.181:3300/0 conn(0x7f7f54071c30 0x7f7f54069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.187+0000 7f7f537fe640 1 -- 172.21.15.70:0/1936661066 >> v1:172.21.15.181:6789/0 conn(0x7f7f54072090 legacy=0x7f7f540ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:03.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.187+0000 7f7f537fe640 1 -- 172.21.15.70:0/1936661066 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54161810 con 0x7f7f541601b0 2023-10-08T06:54:03.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.188+0000 7f7f537fe640 1 --2- 172.21.15.70:0/1936661066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f541601b0 0x7f7f540cd430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009870 tx=0x7f7f4402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644cc01cc47fccdd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:03.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.189+0000 7f7f527fc640 1 -- 172.21.15.70:0/1936661066 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44030840 con 0x7f7f541601b0 2023-10-08T06:54:03.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.189+0000 7f7f527fc640 1 -- 172.21.15.70:0/1936661066 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f440309a0 con 0x7f7f541601b0 2023-10-08T06:54:03.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.189+0000 7f7f527fc640 1 -- 172.21.15.70:0/1936661066 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44030d10 con 0x7f7f541601b0 2023-10-08T06:54:03.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.189+0000 7f7f59d5a640 1 -- 172.21.15.70:0/1936661066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f541601b0 msgr2=0x7f7f540cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:03.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.189+0000 7f7f59d5a640 1 --2- 172.21.15.70:0/1936661066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f541601b0 0x7f7f540cd430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009870 tx=0x7f7f4402b980 comp rx=0 tx=0).stop 2023-10-08T06:54:03.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.189+0000 7f7f59d5a640 1 -- 172.21.15.70:0/1936661066 shutdown_connections 2023-10-08T06:54:03.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.189+0000 7f7f59d5a640 1 --2- 172.21.15.70:0/1936661066 >> v2:172.21.15.181:3300/0 conn(0x7f7f54071c30 0x7f7f54069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.189+0000 7f7f59d5a640 1 --2- 172.21.15.70:0/1936661066 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f541601b0 0x7f7f540cd430 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.189+0000 7f7f59d5a640 1 -- 172.21.15.70:0/1936661066 >> 172.21.15.70:0/1936661066 conn(0x7f7f5406d430 msgr2=0x7f7f5406d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:03.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.190+0000 7f7f59d5a640 1 -- 172.21.15.70:0/1936661066 shutdown_connections 2023-10-08T06:54:03.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.190+0000 7f7f59d5a640 1 -- 172.21.15.70:0/1936661066 wait complete. 2023-10-08T06:54:03.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.191+0000 7f7f59d5a640 1 Processor -- start 2023-10-08T06:54:03.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.191+0000 7f7f59d5a640 1 -- start start 2023-10-08T06:54:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.192+0000 7f7f59d5a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f54071c30 0x7f7f5414a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.192+0000 7f7f59d5a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f54072090 0x7f7f5415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.192+0000 7f7f59d5a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f541601b0 0x7f7f54150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.192+0000 7f7f59d5a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54150f80 con 0x7f7f541601b0 2023-10-08T06:54:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.192+0000 7f7f59d5a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540c7980 con 0x7f7f54071c30 2023-10-08T06:54:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.192+0000 7f7f59d5a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5415df90 con 0x7f7f54072090 2023-10-08T06:54:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f541601b0 0x7f7f54150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f541601b0 0x7f7f54150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39370/0 (socket says 172.21.15.70:39370) 2023-10-08T06:54:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f53fff640 1 -- 172.21.15.70:0/384468291 learned_addr learned my addr 172.21.15.70:0/384468291 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f537fe640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f54071c30 0x7f7f5414a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f52ffd640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f54072090 0x7f7f5415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f53fff640 1 -- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f54072090 msgr2=0x7f7f5415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f53fff640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f54072090 0x7f7f5415bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f53fff640 1 -- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f54071c30 msgr2=0x7f7f5414a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f53fff640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f54071c30 0x7f7f5414a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f53fff640 1 -- 172.21.15.70:0/384468291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440094d0 con 0x7f7f541601b0 2023-10-08T06:54:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.193+0000 7f7f537fe640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f54071c30 0x7f7f5414a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T06:54:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.194+0000 7f7f53fff640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f541601b0 0x7f7f54150a40 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00c710 tx=0x7f7f4c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.194+0000 7f7f3bfff640 1 -- 172.21.15.70:0/384468291 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c003b70 con 0x7f7f541601b0 2023-10-08T06:54:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.194+0000 7f7f3bfff640 1 -- 172.21.15.70:0/384468291 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c003cd0 con 0x7f7f541601b0 2023-10-08T06:54:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.194+0000 7f7f3bfff640 1 -- 172.21.15.70:0/384468291 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00aa50 con 0x7f7f541601b0 2023-10-08T06:54:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.194+0000 7f7f59d5a640 1 -- 172.21.15.70:0/384468291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5415e250 con 0x7f7f541601b0 2023-10-08T06:54:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.194+0000 7f7f59d5a640 1 -- 172.21.15.70:0/384468291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5415e810 con 0x7f7f541601b0 2023-10-08T06:54:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.195+0000 7f7f59d5a640 1 -- 172.21.15.70:0/384468291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5404e160 con 0x7f7f541601b0 2023-10-08T06:54:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.196+0000 7f7f3bfff640 1 -- 172.21.15.70:0/384468291 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 90) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f7f4c00abb0 con 0x7f7f541601b0 2023-10-08T06:54:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.197+0000 7f7f3bfff640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7f4006afe0 0x7f7f4006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.197+0000 7f7f3bfff640 1 -- 172.21.15.70:0/384468291 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7f4c0897d0 con 0x7f7f541601b0 2023-10-08T06:54:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.197+0000 7f7f537fe640 1 -- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7f4006afe0 msgr2=0x7f7f4006d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245513204 2023-10-08T06:54:03.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.197+0000 7f7f537fe640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7f4006afe0 0x7f7f4006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:03.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.201+0000 7f7f3bfff640 1 -- 172.21.15.70:0/384468291 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f4c0545a0 con 0x7f7f541601b0 2023-10-08T06:54:03.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.389+0000 7f7f59d5a640 1 -- 172.21.15.70:0/384468291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7f540c7ac0 con 0x7f7f541601b0 2023-10-08T06:54:03.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.394+0000 7f7f3bfff640 1 -- 172.21.15.70:0/384468291 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7f4c00ae60 con 0x7f7f541601b0 2023-10-08T06:54:03.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.396+0000 7f7f59d5a640 1 -- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7f4006afe0 msgr2=0x7f7f4006d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:03.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.396+0000 7f7f59d5a640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7f4006afe0 0x7f7f4006d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.396+0000 7f7f59d5a640 1 -- 172.21.15.70:0/384468291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f541601b0 msgr2=0x7f7f54150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.396+0000 7f7f59d5a640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f541601b0 0x7f7f54150a40 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00c710 tx=0x7f7f4c00cbd0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.396+0000 7f7f59d5a640 1 -- 172.21.15.70:0/384468291 shutdown_connections 2023-10-08T06:54:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.396+0000 7f7f59d5a640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7f7f4006afe0 0x7f7f4006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.396+0000 7f7f59d5a640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f541601b0 0x7f7f54150a40 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.396+0000 7f7f59d5a640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7f54072090 0x7f7f5415bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.396+0000 7f7f59d5a640 1 --2- 172.21.15.70:0/384468291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7f54071c30 0x7f7f5414a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.396+0000 7f7f59d5a640 1 -- 172.21.15.70:0/384468291 >> 172.21.15.70:0/384468291 conn(0x7f7f5406d430 msgr2=0x7f7f54069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.397+0000 7f7f59d5a640 1 -- 172.21.15.70:0/384468291 shutdown_connections 2023-10-08T06:54:03.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.397+0000 7f7f59d5a640 1 -- 172.21.15.70:0/384468291 wait complete. 2023-10-08T06:54:03.416 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T06:54:03.479 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:03.478+0000 7fabc4c0f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T06:54:03.480 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:03.479+0000 7f8852c13040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T06:54:03.533 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:03.532+0000 7f8852c13040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T06:54:03.535 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:03.533+0000 7fabc4c0f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T06:54:03.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.618+0000 7fdf47fff640 1 Processor -- start 2023-10-08T06:54:03.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.618+0000 7fdf47fff640 1 -- start start 2023-10-08T06:54:03.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.620+0000 7fdf47fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf480cf2b0 0x7fdf480d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.620+0000 7fdf47fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf480cf690 0x7fdf480ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.620+0000 7fdf47fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf480d1100 0x7fdf4815c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.620+0000 7fdf47fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf480cd520 con 0x7fdf480cf690 2023-10-08T06:54:03.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.620+0000 7fdf47fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf480d1910 con 0x7fdf480d1100 2023-10-08T06:54:03.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.620+0000 7fdf47fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480ce080 con 0x7fdf480cf2b0 2023-10-08T06:54:03.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.620+0000 7fdf467fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf480cf690 0x7fdf480ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.620+0000 7fdf467fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf480cf690 0x7fdf480ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42532/0 (socket says 172.21.15.70:42532) 2023-10-08T06:54:03.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.620+0000 7fdf467fc640 1 -- 172.21.15.70:0/3539154676 learned_addr learned my addr 172.21.15.70:0/3539154676 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:03.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.620+0000 7fdf46ffd640 1 --2- 172.21.15.70:0/3539154676 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf480cf2b0 0x7fdf480d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.621+0000 7fdf477fe640 1 --2- 172.21.15.70:0/3539154676 >> v2:172.21.15.181:3300/0 conn(0x7fdf480d1100 0x7fdf4815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.621+0000 7fdf467fc640 1 -- 172.21.15.70:0/3539154676 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf480cf2b0 msgr2=0x7fdf480d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:03.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.621+0000 7fdf467fc640 1 --2- 172.21.15.70:0/3539154676 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf480cf2b0 0x7fdf480d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.621+0000 7fdf467fc640 1 -- 172.21.15.70:0/3539154676 >> v2:172.21.15.181:3300/0 conn(0x7fdf480d1100 msgr2=0x7fdf4815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:03.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.621+0000 7fdf467fc640 1 --2- 172.21.15.70:0/3539154676 >> v2:172.21.15.181:3300/0 conn(0x7fdf480d1100 0x7fdf4815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.621+0000 7fdf467fc640 1 -- 172.21.15.70:0/3539154676 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480cd750 con 0x7fdf480cf690 2023-10-08T06:54:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.621+0000 7fdf477fe640 1 --2- 172.21.15.70:0/3539154676 >> v2:172.21.15.181:3300/0 conn(0x7fdf480d1100 0x7fdf4815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.622+0000 7fdf467fc640 1 --2- 172.21.15.70:0/3539154676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480cf690 0x7fdf480ce510 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fdf480d0020 tx=0x7fdf3c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396677e606fa18a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.622+0000 7fdf45ffb640 1 -- 172.21.15.70:0/3539154676 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0317e0 con 0x7fdf480cf690 2023-10-08T06:54:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.622+0000 7fdf45ffb640 1 -- 172.21.15.70:0/3539154676 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c031940 con 0x7fdf480cf690 2023-10-08T06:54:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.622+0000 7fdf45ffb640 1 -- 172.21.15.70:0/3539154676 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c031cd0 con 0x7fdf480cf690 2023-10-08T06:54:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.623+0000 7fdf47fff640 1 -- 172.21.15.70:0/3539154676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480cf690 msgr2=0x7fdf480ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.623+0000 7fdf47fff640 1 --2- 172.21.15.70:0/3539154676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480cf690 0x7fdf480ce510 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fdf480d0020 tx=0x7fdf3c02b880 comp rx=0 tx=0).stop 2023-10-08T06:54:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.623+0000 7fdf47fff640 1 -- 172.21.15.70:0/3539154676 shutdown_connections 2023-10-08T06:54:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.623+0000 7fdf47fff640 1 --2- 172.21.15.70:0/3539154676 >> v2:172.21.15.181:3300/0 conn(0x7fdf480d1100 0x7fdf4815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.623+0000 7fdf47fff640 1 --2- 172.21.15.70:0/3539154676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480cf690 0x7fdf480ce510 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.623+0000 7fdf47fff640 1 --2- 172.21.15.70:0/3539154676 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf480cf2b0 0x7fdf480d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.623+0000 7fdf47fff640 1 -- 172.21.15.70:0/3539154676 >> 172.21.15.70:0/3539154676 conn(0x7fdf4809bc40 msgr2=0x7fdf480d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:03.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.623+0000 7fdf47fff640 1 -- 172.21.15.70:0/3539154676 shutdown_connections 2023-10-08T06:54:03.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.623+0000 7fdf47fff640 1 -- 172.21.15.70:0/3539154676 wait complete. 2023-10-08T06:54:03.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.625+0000 7fdf47fff640 1 Processor -- start 2023-10-08T06:54:03.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.625+0000 7fdf47fff640 1 -- start start 2023-10-08T06:54:03.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf47fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf480cf2b0 0x7fdf481f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf47fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf480cf690 0x7fdf4815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf47fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480d1100 0x7fdf4809d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf47fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4809d610 con 0x7fdf480d1100 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf47fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481f39e0 con 0x7fdf480cf2b0 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf47fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf481f3cc0 con 0x7fdf480cf690 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf477fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480d1100 0x7fdf4809d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf477fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480d1100 0x7fdf4809d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42536/0 (socket says 172.21.15.70:42536) 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf477fe640 1 -- 172.21.15.70:0/525581469 learned_addr learned my addr 172.21.15.70:0/525581469 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf46ffd640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf480cf2b0 0x7fdf481f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.626+0000 7fdf467fc640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf480cf690 0x7fdf4815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf477fe640 1 -- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf480cf690 msgr2=0x7fdf4815a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:03.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf477fe640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf480cf690 0x7fdf4815a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf477fe640 1 -- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf480cf2b0 msgr2=0x7fdf481f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf477fe640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf480cf2b0 0x7fdf481f3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf477fe640 1 -- 172.21.15.70:0/525581469 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0094d0 con 0x7fdf480d1100 2023-10-08T06:54:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf46ffd640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf480cf2b0 0x7fdf481f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T06:54:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf477fe640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480d1100 0x7fdf4809d0d0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000cd50 tx=0x7fdf4000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf2f7fe640 1 -- 172.21.15.70:0/525581469 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000f680 con 0x7fdf480d1100 2023-10-08T06:54:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf2f7fe640 1 -- 172.21.15.70:0/525581469 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4000f800 con 0x7fdf480d1100 2023-10-08T06:54:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf2f7fe640 1 -- 172.21.15.70:0/525581469 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000fbb0 con 0x7fdf480d1100 2023-10-08T06:54:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf47fff640 1 -- 172.21.15.70:0/525581469 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf481f3fa0 con 0x7fdf480d1100 2023-10-08T06:54:03.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.627+0000 7fdf47fff640 1 -- 172.21.15.70:0/525581469 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf481f4560 con 0x7fdf480d1100 2023-10-08T06:54:03.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.628+0000 7fdf47fff640 1 -- 172.21.15.70:0/525581469 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf48074150 con 0x7fdf480d1100 2023-10-08T06:54:03.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.629+0000 7fdf2f7fe640 1 -- 172.21.15.70:0/525581469 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 90) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fdf4000b040 con 0x7fdf480d1100 2023-10-08T06:54:03.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.630+0000 7fdf2f7fe640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf3406b010 0x7fdf3406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:03.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.630+0000 7fdf2f7fe640 1 -- 172.21.15.70:0/525581469 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fdf4008a4a0 con 0x7fdf480d1100 2023-10-08T06:54:03.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.631+0000 7fdf46ffd640 1 -- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf3406b010 msgr2=0x7fdf3406d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245513204 2023-10-08T06:54:03.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.631+0000 7fdf46ffd640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf3406b010 0x7fdf3406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:03.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.635+0000 7fdf2f7fe640 1 -- 172.21.15.70:0/525581469 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf40055150 con 0x7fdf480d1100 2023-10-08T06:54:03.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.817+0000 7fdf47fff640 1 -- 172.21.15.70:0/525581469 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fdf481f4750 con 0x7fdf480d1100 2023-10-08T06:54:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.822+0000 7fdf2f7fe640 1 -- 172.21.15.70:0/525581469 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7fdf40058770 con 0x7fdf480d1100 2023-10-08T06:54:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.824+0000 7fdf47fff640 1 -- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf3406b010 msgr2=0x7fdf3406d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.824+0000 7fdf47fff640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf3406b010 0x7fdf3406d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.824+0000 7fdf47fff640 1 -- 172.21.15.70:0/525581469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480d1100 msgr2=0x7fdf4809d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.824+0000 7fdf47fff640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480d1100 0x7fdf4809d0d0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000cd50 tx=0x7fdf4000a650 comp rx=0 tx=0).stop 2023-10-08T06:54:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.825+0000 7fdf47fff640 1 -- 172.21.15.70:0/525581469 shutdown_connections 2023-10-08T06:54:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.825+0000 7fdf47fff640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7fdf3406b010 0x7fdf3406d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.825+0000 7fdf47fff640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf480d1100 0x7fdf4809d0d0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.825+0000 7fdf47fff640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf480cf690 0x7fdf4815a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.825+0000 7fdf47fff640 1 --2- 172.21.15.70:0/525581469 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf480cf2b0 0x7fdf481f3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.825+0000 7fdf47fff640 1 -- 172.21.15.70:0/525581469 >> 172.21.15.70:0/525581469 conn(0x7fdf4809bc40 msgr2=0x7fdf480b7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.825+0000 7fdf47fff640 1 -- 172.21.15.70:0/525581469 shutdown_connections 2023-10-08T06:54:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:03.825+0000 7fdf47fff640 1 -- 172.21.15.70:0/525581469 wait complete. 2023-10-08T06:54:03.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:03.843 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T06:54:03.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1h 2023-10-08T06:54:03.995 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:03.993+0000 7f8852c13040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T06:54:04.011 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.010+0000 7fabc4c0f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T06:54:04.047 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:04.045+0000 7f8852c13040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T06:54:04.065 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.064+0000 7fabc4c0f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T06:54:04.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.094+0000 7ff5265f4640 1 Processor -- start 2023-10-08T06:54:04.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.094+0000 7ff5265f4640 1 -- start start 2023-10-08T06:54:04.096 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:04.095+0000 7f8852c13040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T06:54:04.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.096+0000 7ff5265f4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5200cf2b0 0x7ff5200d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:04.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.096+0000 7ff5265f4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff5200cf690 0x7ff5200ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:04.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.096+0000 7ff5265f4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5200cd520 con 0x7ff5200d1100 2023-10-08T06:54:04.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.096+0000 7ff5265f4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200d1910 con 0x7ff5200cf2b0 2023-10-08T06:54:04.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.096+0000 7ff5265f4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200ce080 con 0x7ff5200cf690 2023-10-08T06:54:04.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.096+0000 7ff5255f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5200cf2b0 0x7ff5200d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:04.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.096+0000 7ff5255f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5200cf2b0 0x7ff5200d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42540/0 (socket says 172.21.15.70:42540) 2023-10-08T06:54:04.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.096+0000 7ff5255f2640 1 -- 172.21.15.70:0/4288305563 learned_addr learned my addr 172.21.15.70:0/4288305563 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:04.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.096+0000 7ff5255f2640 1 -- 172.21.15.70:0/4288305563 >> v2:172.21.15.181:3300/0 conn(0x7ff5200cf690 msgr2=0x7ff5200ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:04.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.096+0000 7ff524df1640 1 --2- 172.21.15.70:0/4288305563 >> v2:172.21.15.181:3300/0 conn(0x7ff5200cf690 0x7ff5200ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:04.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.097+0000 7ff5255f2640 1 --2- 172.21.15.70:0/4288305563 >> v2:172.21.15.181:3300/0 conn(0x7ff5200cf690 0x7ff5200ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:04.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.097+0000 7ff5255f2640 1 -- 172.21.15.70:0/4288305563 >> v1:172.21.15.181:6789/0 conn(0x7ff5200d1100 legacy=0x7ff52015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:04.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.097+0000 7ff5255f2640 1 -- 172.21.15.70:0/4288305563 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5200cd750 con 0x7ff5200cf2b0 2023-10-08T06:54:04.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.097+0000 7ff524df1640 1 --2- 172.21.15.70:0/4288305563 >> v2:172.21.15.181:3300/0 conn(0x7ff5200cf690 0x7ff5200ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:04.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.098+0000 7ff5255f2640 1 --2- 172.21.15.70:0/4288305563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200cf2b0 0x7ff5200d34d0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff52008baa0 tx=0x7ff51c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0d8a4607ad8464 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:04.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.098+0000 7ff517fff640 1 -- 172.21.15.70:0/4288305563 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0317e0 con 0x7ff5200cf2b0 2023-10-08T06:54:04.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.098+0000 7ff517fff640 1 -- 172.21.15.70:0/4288305563 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c031940 con 0x7ff5200cf2b0 2023-10-08T06:54:04.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.098+0000 7ff517fff640 1 -- 172.21.15.70:0/4288305563 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c031cd0 con 0x7ff5200cf2b0 2023-10-08T06:54:04.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.099+0000 7ff5265f4640 1 -- 172.21.15.70:0/4288305563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200cf2b0 msgr2=0x7ff5200d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:04.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.099+0000 7ff5265f4640 1 --2- 172.21.15.70:0/4288305563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200cf2b0 0x7ff5200d34d0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff52008baa0 tx=0x7ff51c02b880 comp rx=0 tx=0).stop 2023-10-08T06:54:04.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.099+0000 7ff5265f4640 1 -- 172.21.15.70:0/4288305563 shutdown_connections 2023-10-08T06:54:04.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.099+0000 7ff5265f4640 1 --2- 172.21.15.70:0/4288305563 >> v2:172.21.15.181:3300/0 conn(0x7ff5200cf690 0x7ff5200ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:04.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.099+0000 7ff5265f4640 1 --2- 172.21.15.70:0/4288305563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200cf2b0 0x7ff5200d34d0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:04.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.099+0000 7ff5265f4640 1 -- 172.21.15.70:0/4288305563 >> 172.21.15.70:0/4288305563 conn(0x7ff52009bc40 msgr2=0x7ff5200d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:04.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.099+0000 7ff5265f4640 1 -- 172.21.15.70:0/4288305563 shutdown_connections 2023-10-08T06:54:04.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.099+0000 7ff5265f4640 1 -- 172.21.15.70:0/4288305563 wait complete. 2023-10-08T06:54:04.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.100+0000 7ff5265f4640 1 Processor -- start 2023-10-08T06:54:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.101+0000 7ff5265f4640 1 -- start start 2023-10-08T06:54:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.101+0000 7ff5265f4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5200cf2b0 0x7ff5200cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff5265f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200cf690 0x7ff5200c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff5265f4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5200d1100 0x7ff52015dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff5265f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200ccd20 con 0x7ff5200cf690 2023-10-08T06:54:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff5265f4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200cc7f0 con 0x7ff5200d1100 2023-10-08T06:54:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff5265f4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200c94b0 con 0x7ff5200cf2b0 2023-10-08T06:54:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff524df1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200cf690 0x7ff5200c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff524df1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200cf690 0x7ff5200c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42556/0 (socket says 172.21.15.70:42556) 2023-10-08T06:54:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff524df1640 1 -- 172.21.15.70:0/718445794 learned_addr learned my addr 172.21.15.70:0/718445794 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff5255f2640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5200cf2b0 0x7ff5200cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff524df1640 1 -- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5200cf2b0 msgr2=0x7ff5200cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff524df1640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5200cf2b0 0x7ff5200cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff524df1640 1 -- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5200d1100 msgr2=0x7ff52015dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff524df1640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5200d1100 0x7ff52015dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.102+0000 7ff524df1640 1 -- 172.21.15.70:0/718445794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c0094d0 con 0x7ff5200cf690 2023-10-08T06:54:04.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.103+0000 7ff524df1640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200cf690 0x7ff5200c8f70 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c820 tx=0x7ff51000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:04.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.103+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510007a30 con 0x7ff5200cf690 2023-10-08T06:54:04.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.103+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff510007b90 con 0x7ff5200cf690 2023-10-08T06:54:04.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.103+0000 7ff5265f4640 1 -- 172.21.15.70:0/718445794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52015cb40 con 0x7ff5200cf690 2023-10-08T06:54:04.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.103+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51000ce70 con 0x7ff5200cf690 2023-10-08T06:54:04.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.103+0000 7ff5265f4640 1 -- 172.21.15.70:0/718445794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52015d100 con 0x7ff5200cf690 2023-10-08T06:54:04.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.104+0000 7ff5265f4640 1 -- 172.21.15.70:0/718445794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520074150 con 0x7ff5200cf690 2023-10-08T06:54:04.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.105+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 90) v1 ==== 89348+0+0 (secure 0 0 0) 0x7ff5100113f0 con 0x7ff5200cf690 2023-10-08T06:54:04.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.107+0000 7ff515ffb640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 0x7ff50406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:04.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.107+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6145+0+0 (secure 0 0 0) 0x7ff510089cf0 con 0x7ff5200cf690 2023-10-08T06:54:04.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.107+0000 7ff5255f2640 1 -- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 msgr2=0x7ff50406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245513204 2023-10-08T06:54:04.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.107+0000 7ff5255f2640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 0x7ff50406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:04.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.110+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff510054920 con 0x7ff5200cf690 2023-10-08T06:54:04.120 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.119+0000 7fabc4c0f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T06:54:04.158 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:04.157+0000 7f8852c13040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T06:54:04.184 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.183+0000 7fabc4c0f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T06:54:04.207 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:04.206+0000 7f8852c13040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T06:54:04.235 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.233+0000 7fabc4c0f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T06:54:04.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.308+0000 7ff5255f2640 1 -- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 msgr2=0x7ff50406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245513204 2023-10-08T06:54:04.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.308+0000 7ff5255f2640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 0x7ff50406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:54:04.317 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:04.316+0000 7f8852c13040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T06:54:04.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.322+0000 7ff5265f4640 1 -- 172.21.15.70:0/718445794 --> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5200cdd10 con 0x7ff50406afe0 2023-10-08T06:54:04.349 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.347+0000 7fabc4c0f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T06:54:04.381 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:04.379+0000 7f8852c13040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:04.410 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.409+0000 7fabc4c0f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:04.429 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:04.428+0000 7f8852c13040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T06:54:04.460 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.459+0000 7fabc4c0f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T06:54:04.487 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:04.485+0000 7f8852c13040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T06:54:04.516 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.515+0000 7fabc4c0f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T06:54:04.536 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:04.535+0000 7f8852c13040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T06:54:04.566 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.565+0000 7fabc4c0f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T06:54:04.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.709+0000 7ff5255f2640 1 -- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 msgr2=0x7ff50406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245513204 2023-10-08T06:54:04.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:04.709+0000 7ff5255f2640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 0x7ff50406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T06:54:04.919 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:04.918+0000 7f8852c13040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T06:54:04.947 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:04.946+0000 7fabc4c0f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T06:54:05.078 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:05.076+0000 7f8852c13040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T06:54:05.105 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:05.103+0000 7fabc4c0f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T06:54:05.186 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:05.184+0000 7f8852c13040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T06:54:05.212 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:05.210+0000 7fabc4c0f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T06:54:05.236 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:05.235+0000 7f8852c13040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T06:54:05.261 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:05.260+0000 7fabc4c0f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T06:54:05.343 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:05.342+0000 7f8852c13040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:05.369 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:05.367+0000 7fabc4c0f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:05.450 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:05.449+0000 7f8852c13040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T06:54:05.476 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:05.474+0000 7fabc4c0f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T06:54:05.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:05.510+0000 7ff5255f2640 1 -- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 msgr2=0x7ff50406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245513204 2023-10-08T06:54:05.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:05.510+0000 7ff5255f2640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 0x7ff50406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T06:54:05.609 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:05.607+0000 7f8852c13040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:05.632 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:05.631+0000 7fabc4c0f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:05.658 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:05.657+0000 7f8852c13040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:05.682 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:05.680+0000 7fabc4c0f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:05.777 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:05.775+0000 7f8852c13040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T06:54:05.801 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:05.799+0000 7fabc4c0f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T06:54:06.601 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:06.600+0000 7f8852c13040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T06:54:06.622 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:06.621+0000 7fabc4c0f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T06:54:06.900 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:06.899+0000 7f8852c13040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T06:54:06.917 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:06.915+0000 7fabc4c0f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T06:54:06.958 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:06.957+0000 7f8852c13040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T06:54:06.974 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:06.973+0000 7fabc4c0f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T06:54:07.072 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.071+0000 7f8852c13040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T06:54:07.091 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:07.088+0000 7fabc4c0f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T06:54:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:07.092+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 91) v1 ==== 88957+0+0 (secure 0 0 0) 0x7ff510054cf0 con 0x7ff5200cf690 2023-10-08T06:54:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:07.093+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 msgr2=0x7ff50406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:07.093+0000 7ff515ffb640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/4245513204,v1:172.21.15.181:6833/4245513204] conn(0x7ff50406afe0 0x7ff50406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:07.241 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.239+0000 7f87cf2c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:07.241 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.239+0000 7f87cf2c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:07.241 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.239+0000 7f87cf2c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:07.241 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.239+0000 7f87cf2c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:07.241 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.239+0000 7f87cf2c1640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:07.241 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.239+0000 7f87d2307640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:07.241 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.239+0000 7f87d2307640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:07.241 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.239+0000 7f87d2307640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:07.241 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.239+0000 7f87d2307640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:07.242 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:07.239+0000 7f87d2307640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:08.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.103+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 92) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ff510056400 con 0x7ff5200cf690 2023-10-08T06:54:08.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.104+0000 7ff515ffb640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7ff5040358e0 0x7ff504075530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:08.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.104+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 --> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7ff504035cc0 con 0x7ff5040358e0 2023-10-08T06:54:08.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.106+0000 7ff5255f2640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7ff5040358e0 0x7ff504075530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:08.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.118+0000 7ff5255f2640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7ff5040358e0 0x7ff504075530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff51c036000 tx=0x7ff51c03d040 comp rx=0 tx=0).ready entity=mgr.9804 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:08.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.263+0000 7ff515ffb640 1 -- 172.21.15.70:0/718445794 <== mgr.9804 v2:172.21.15.181:6832/1385757024 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff504035cc0 con 0x7ff5040358e0 2023-10-08T06:54:08.266 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1 2023-10-08T06:54:08.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.266+0000 7ff5265f4640 1 -- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7ff5040358e0 msgr2=0x7ff504075530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:08.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.266+0000 7ff5265f4640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7ff5040358e0 0x7ff504075530 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff51c036000 tx=0x7ff51c03d040 comp rx=0 tx=0).stop 2023-10-08T06:54:08.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.266+0000 7ff5265f4640 1 -- 172.21.15.70:0/718445794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200cf690 msgr2=0x7ff5200c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:08.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.266+0000 7ff5265f4640 1 --2- 172.21.15.70:0/718445794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5200cf690 0x7ff5200c8f70 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c820 tx=0x7ff51000cc20 comp rx=0 tx=0).stop 2023-10-08T06:54:08.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.266+0000 7ff525df3640 1 -- 172.21.15.70:0/718445794 reap_dead start 2023-10-08T06:54:08.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.266+0000 7ff5265f4640 1 -- 172.21.15.70:0/718445794 shutdown_connections 2023-10-08T06:54:08.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.266+0000 7ff5265f4640 1 -- 172.21.15.70:0/718445794 >> 172.21.15.70:0/718445794 conn(0x7ff52009bc40 msgr2=0x7ff5201551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:08.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.266+0000 7ff5265f4640 1 -- 172.21.15.70:0/718445794 shutdown_connections 2023-10-08T06:54:08.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.266+0000 7ff5265f4640 1 -- 172.21.15.70:0/718445794 wait complete. 2023-10-08T06:54:08.287 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T06:54:08.288 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-08T06:54:08.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.503+0000 7fae53e7c640 1 Processor -- start 2023-10-08T06:54:08.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.503+0000 7fae53e7c640 1 -- start start 2023-10-08T06:54:08.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.504+0000 7fae53e7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae4c0afbe0 0x7fae4c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:08.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.504+0000 7fae53e7c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fae4c06dfc0 0x7fae4c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:08.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.504+0000 7fae53e7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae4c071070 0x7fae4c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:08.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.504+0000 7fae53e7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4c0b2620 con 0x7fae4c071070 2023-10-08T06:54:08.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.504+0000 7fae53e7c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4c0b2060 con 0x7fae4c06dfc0 2023-10-08T06:54:08.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.504+0000 7fae53e7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c0b2340 con 0x7fae4c0afbe0 2023-10-08T06:54:08.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae52419640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae4c071070 0x7fae4c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:08.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae52419640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae4c071070 0x7fae4c0cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42564/0 (socket says 172.21.15.70:42564) 2023-10-08T06:54:08.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae52419640 1 -- 172.21.15.70:0/3072615657 learned_addr learned my addr 172.21.15.70:0/3072615657 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:08.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae51c18640 1 --2- 172.21.15.70:0/3072615657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae4c0afbe0 0x7fae4c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:08.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae51417640 1 --2- 172.21.15.70:0/3072615657 >> v2:172.21.15.181:3300/0 conn(0x7fae4c06dfc0 0x7fae4c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:08.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae52419640 1 -- 172.21.15.70:0/3072615657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae4c0afbe0 msgr2=0x7fae4c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:08.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae52419640 1 --2- 172.21.15.70:0/3072615657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae4c0afbe0 0x7fae4c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae52419640 1 -- 172.21.15.70:0/3072615657 >> v2:172.21.15.181:3300/0 conn(0x7fae4c06dfc0 msgr2=0x7fae4c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae52419640 1 --2- 172.21.15.70:0/3072615657 >> v2:172.21.15.181:3300/0 conn(0x7fae4c06dfc0 0x7fae4c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae52419640 1 -- 172.21.15.70:0/3072615657 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c070350 con 0x7fae4c071070 2023-10-08T06:54:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.505+0000 7fae51417640 1 --2- 172.21.15.70:0/3072615657 >> v2:172.21.15.181:3300/0 conn(0x7fae4c06dfc0 0x7fae4c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.506+0000 7fae52419640 1 --2- 172.21.15.70:0/3072615657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c071070 0x7fae4c0cb130 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0cb0b0 tx=0x7fae4402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5618999f7d0af5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.507+0000 7fae50c16640 1 -- 172.21.15.70:0/3072615657 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4402ca80 con 0x7fae4c071070 2023-10-08T06:54:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.507+0000 7fae50c16640 1 -- 172.21.15.70:0/3072615657 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4402cbe0 con 0x7fae4c071070 2023-10-08T06:54:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.507+0000 7fae50c16640 1 -- 172.21.15.70:0/3072615657 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae440343f0 con 0x7fae4c071070 2023-10-08T06:54:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.507+0000 7fae53e7c640 1 -- 172.21.15.70:0/3072615657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c071070 msgr2=0x7fae4c0cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.507+0000 7fae53e7c640 1 --2- 172.21.15.70:0/3072615657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c071070 0x7fae4c0cb130 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0cb0b0 tx=0x7fae4402bee0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.508+0000 7fae53e7c640 1 -- 172.21.15.70:0/3072615657 shutdown_connections 2023-10-08T06:54:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.508+0000 7fae53e7c640 1 --2- 172.21.15.70:0/3072615657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c071070 0x7fae4c0cb130 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.508+0000 7fae53e7c640 1 --2- 172.21.15.70:0/3072615657 >> v2:172.21.15.181:3300/0 conn(0x7fae4c06dfc0 0x7fae4c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.508+0000 7fae53e7c640 1 --2- 172.21.15.70:0/3072615657 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae4c0afbe0 0x7fae4c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.508+0000 7fae53e7c640 1 -- 172.21.15.70:0/3072615657 >> 172.21.15.70:0/3072615657 conn(0x7fae4c06a5d0 msgr2=0x7fae4c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.508+0000 7fae53e7c640 1 -- 172.21.15.70:0/3072615657 shutdown_connections 2023-10-08T06:54:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.508+0000 7fae53e7c640 1 -- 172.21.15.70:0/3072615657 wait complete. 2023-10-08T06:54:08.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.509+0000 7fae53e7c640 1 Processor -- start 2023-10-08T06:54:08.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.510+0000 7fae53e7c640 1 -- start start 2023-10-08T06:54:08.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae53e7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae4c06dfc0 0x7fae4c0c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:08.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae53e7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae4c071070 0x7fae4c145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:08.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae53e7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c0afbe0 0x7fae4c0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:08.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae53e7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c146680 con 0x7fae4c0afbe0 2023-10-08T06:54:08.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae53e7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c145a00 con 0x7fae4c071070 2023-10-08T06:54:08.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae53e7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4c145ce0 con 0x7fae4c06dfc0 2023-10-08T06:54:08.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae52419640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c0afbe0 0x7fae4c0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:08.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae52419640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c0afbe0 0x7fae4c0cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42578/0 (socket says 172.21.15.70:42578) 2023-10-08T06:54:08.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae52419640 1 -- 172.21.15.70:0/4040694366 learned_addr learned my addr 172.21.15.70:0/4040694366 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:08.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae51c18640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae4c06dfc0 0x7fae4c0c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:08.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae52419640 1 -- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae4c06dfc0 msgr2=0x7fae4c0c2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:08.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae52419640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae4c06dfc0 0x7fae4c0c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae52419640 1 -- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae4c071070 msgr2=0x7fae4c145070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:08.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae51417640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae4c071070 0x7fae4c145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:08.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae52419640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae4c071070 0x7fae4c145070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae52419640 1 -- 172.21.15.70:0/4040694366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae44009b30 con 0x7fae4c0afbe0 2023-10-08T06:54:08.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.511+0000 7fae51417640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae4c071070 0x7fae4c145070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:08.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.512+0000 7fae52419640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c0afbe0 0x7fae4c0cdd30 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fae44036040 tx=0x7fae44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:08.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.512+0000 7fae427fc640 1 -- 172.21.15.70:0/4040694366 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44009510 con 0x7fae4c0afbe0 2023-10-08T06:54:08.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.512+0000 7fae427fc640 1 -- 172.21.15.70:0/4040694366 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4403e450 con 0x7fae4c0afbe0 2023-10-08T06:54:08.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.512+0000 7fae53e7c640 1 -- 172.21.15.70:0/4040694366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4c145fc0 con 0x7fae4c0afbe0 2023-10-08T06:54:08.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.512+0000 7fae427fc640 1 -- 172.21.15.70:0/4040694366 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4403e800 con 0x7fae4c0afbe0 2023-10-08T06:54:08.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.512+0000 7fae53e7c640 1 -- 172.21.15.70:0/4040694366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4c1455b0 con 0x7fae4c0afbe0 2023-10-08T06:54:08.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.514+0000 7fae53e7c640 1 -- 172.21.15.70:0/4040694366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4c1467c0 con 0x7fae4c0afbe0 2023-10-08T06:54:08.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.514+0000 7fae427fc640 1 -- 172.21.15.70:0/4040694366 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 92) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fae4402cd50 con 0x7fae4c0afbe0 2023-10-08T06:54:08.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.515+0000 7fae427fc640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7fae3006ad10 0x7fae3006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:08.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.516+0000 7fae427fc640 1 -- 172.21.15.70:0/4040694366 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fae440ac0e0 con 0x7fae4c0afbe0 2023-10-08T06:54:08.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.516+0000 7fae51c18640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7fae3006ad10 0x7fae3006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:08.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.517+0000 7fae51c18640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7fae3006ad10 0x7fae3006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae48006fd0 tx=0x7fae48008040 comp rx=0 tx=0).ready entity=mgr.9804 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:08.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.520+0000 7fae427fc640 1 -- 172.21.15.70:0/4040694366 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae4c1467c0 con 0x7fae4c0afbe0 2023-10-08T06:54:08.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.731+0000 7fae53e7c640 1 -- 172.21.15.70:0/4040694366 --> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fae4c0affc0 con 0x7fae3006ad10 2023-10-08T06:54:08.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.887+0000 7fae427fc640 1 -- 172.21.15.70:0/4040694366 <== mgr.9804 v2:172.21.15.181:6832/1385757024 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fae48009d60 con 0x7fae3006ad10 2023-10-08T06:54:08.889 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path snap_test_dir1 2023-10-08T06:54:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.890+0000 7fae53e7c640 1 -- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7fae3006ad10 msgr2=0x7fae3006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.890+0000 7fae53e7c640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7fae3006ad10 0x7fae3006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae48006fd0 tx=0x7fae48008040 comp rx=0 tx=0).stop 2023-10-08T06:54:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.890+0000 7fae53e7c640 1 -- 172.21.15.70:0/4040694366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c0afbe0 msgr2=0x7fae4c0cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.890+0000 7fae53e7c640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c0afbe0 0x7fae4c0cdd30 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fae44036040 tx=0x7fae44004a40 comp rx=0 tx=0).stop 2023-10-08T06:54:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.891+0000 7fae53e7c640 1 -- 172.21.15.70:0/4040694366 shutdown_connections 2023-10-08T06:54:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.891+0000 7fae53e7c640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7fae3006ad10 0x7fae3006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.891+0000 7fae53e7c640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae4c0afbe0 0x7fae4c0cdd30 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.891+0000 7fae53e7c640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae4c071070 0x7fae4c145070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.891+0000 7fae53e7c640 1 --2- 172.21.15.70:0/4040694366 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae4c06dfc0 0x7fae4c0c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.891+0000 7fae53e7c640 1 -- 172.21.15.70:0/4040694366 >> 172.21.15.70:0/4040694366 conn(0x7fae4c06a5d0 msgr2=0x7fae4c15e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.891+0000 7fae53e7c640 1 -- 172.21.15.70:0/4040694366 shutdown_connections 2023-10-08T06:54:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:08.891+0000 7fae53e7c640 1 -- 172.21.15.70:0/4040694366 wait complete. 2023-10-08T06:54:08.913 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-08T06:54:08.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-10-08T06:54:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.134+0000 7ff3ef221640 1 Processor -- start 2023-10-08T06:54:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.134+0000 7ff3ef221640 1 -- start start 2023-10-08T06:54:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.136+0000 7ff3ef221640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3e80c9cf0 0x7ff3e806db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.136+0000 7ff3ef221640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3e815aa00 0x7ff3e80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.136+0000 7ff3ef221640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e815b310 con 0x7ff3e806c3a0 2023-10-08T06:54:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.136+0000 7ff3ef221640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e806e0a0 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.136+0000 7ff3ef221640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e806d290 con 0x7ff3e815aa00 2023-10-08T06:54:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.136+0000 7ff3e7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3e80c9cf0 0x7ff3e806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.136+0000 7ff3e7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3e80c9cf0 0x7ff3e806db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42592/0 (socket says 172.21.15.70:42592) 2023-10-08T06:54:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.136+0000 7ff3e7fff640 1 -- 172.21.15.70:0/2964423715 learned_addr learned my addr 172.21.15.70:0/2964423715 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.137+0000 7ff3ed7be640 1 --2- 172.21.15.70:0/2964423715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3e815aa00 0x7ff3e80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.137+0000 7ff3e7fff640 1 -- 172.21.15.70:0/2964423715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3e815aa00 msgr2=0x7ff3e80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.137+0000 7ff3e7fff640 1 --2- 172.21.15.70:0/2964423715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3e815aa00 0x7ff3e80b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.137+0000 7ff3e7fff640 1 -- 172.21.15.70:0/2964423715 >> v1:172.21.15.181:6789/0 conn(0x7ff3e806c3a0 legacy=0x7ff3e806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.137+0000 7ff3e7fff640 1 -- 172.21.15.70:0/2964423715 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e80c9860 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.138+0000 7ff3e7fff640 1 --2- 172.21.15.70:0/2964423715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80c9cf0 0x7ff3e806db60 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009870 tx=0x7ff3d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165fc3797b080 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.138+0000 7ff3e77fe640 1 -- 172.21.15.70:0/2964423715 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0031640 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.138+0000 7ff3e77fe640 1 -- 172.21.15.70:0/2964423715 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d00317a0 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.138+0000 7ff3e77fe640 1 -- 172.21.15.70:0/2964423715 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0031b30 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.139+0000 7ff3ef221640 1 -- 172.21.15.70:0/2964423715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80c9cf0 msgr2=0x7ff3e806db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.139+0000 7ff3ef221640 1 --2- 172.21.15.70:0/2964423715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80c9cf0 0x7ff3e806db60 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009870 tx=0x7ff3d002b8f0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.139+0000 7ff3ef221640 1 -- 172.21.15.70:0/2964423715 shutdown_connections 2023-10-08T06:54:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.139+0000 7ff3ef221640 1 --2- 172.21.15.70:0/2964423715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3e815aa00 0x7ff3e80b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.139+0000 7ff3ef221640 1 --2- 172.21.15.70:0/2964423715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80c9cf0 0x7ff3e806db60 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.139+0000 7ff3ef221640 1 -- 172.21.15.70:0/2964423715 >> 172.21.15.70:0/2964423715 conn(0x7ff3e806a5d0 msgr2=0x7ff3e80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.139+0000 7ff3ef221640 1 -- 172.21.15.70:0/2964423715 shutdown_connections 2023-10-08T06:54:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.139+0000 7ff3ef221640 1 -- 172.21.15.70:0/2964423715 wait complete. 2023-10-08T06:54:09.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.141+0000 7ff3ef221640 1 Processor -- start 2023-10-08T06:54:09.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.141+0000 7ff3ef221640 1 -- start start 2023-10-08T06:54:09.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.142+0000 7ff3ef221640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3e806c3a0 0x7ff3e8167610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.142+0000 7ff3ef221640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80c9cf0 0x7ff3e80c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.142+0000 7ff3ef221640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e815aa00 0x7ff3e8158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.142+0000 7ff3ef221640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8166b70 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.142+0000 7ff3ef221640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80c82e0 con 0x7ff3e815aa00 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.142+0000 7ff3ef221640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e8166e10 con 0x7ff3e806c3a0 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.142+0000 7ff3e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80c9cf0 0x7ff3e80c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.142+0000 7ff3e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80c9cf0 0x7ff3e80c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42608/0 (socket says 172.21.15.70:42608) 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.142+0000 7ff3e7fff640 1 -- 172.21.15.70:0/2160229177 learned_addr learned my addr 172.21.15.70:0/2160229177 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3ecfbd640 1 --2- 172.21.15.70:0/2160229177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3e806c3a0 0x7ff3e8167610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3e7fff640 1 -- 172.21.15.70:0/2160229177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3e806c3a0 msgr2=0x7ff3e8167610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3e7fff640 1 --2- 172.21.15.70:0/2160229177 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3e806c3a0 0x7ff3e8167610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3e7fff640 1 -- 172.21.15.70:0/2160229177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e815aa00 msgr2=0x7ff3e8158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:09.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3e7fff640 1 --2- 172.21.15.70:0/2160229177 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e815aa00 0x7ff3e8158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3e7fff640 1 -- 172.21.15.70:0/2160229177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d00094d0 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3e7fff640 1 --2- 172.21.15.70:0/2160229177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80c9cf0 0x7ff3e80c7820 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009870 tx=0x7ff3d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:09.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3e57fa640 1 -- 172.21.15.70:0/2160229177 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d003ac60 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3e57fa640 1 -- 172.21.15.70:0/2160229177 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d003ade0 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3ef221640 1 -- 172.21.15.70:0/2160229177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e81670f0 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.143+0000 7ff3e57fa640 1 -- 172.21.15.70:0/2160229177 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d0044630 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.144+0000 7ff3ef221640 1 -- 172.21.15.70:0/2160229177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e8152c50 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.145+0000 7ff3ef221640 1 -- 172.21.15.70:0/2160229177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e81669d0 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.146+0000 7ff3e57fa640 1 -- 172.21.15.70:0/2160229177 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 92) v1 ==== 89084+0+0 (secure 0 0 0) 0x7ff3d002f050 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.147+0000 7ff3e57fa640 1 --2- 172.21.15.70:0/2160229177 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7ff3c806ad40 0x7ff3c806d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.147+0000 7ff3e57fa640 1 -- 172.21.15.70:0/2160229177 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7ff3d00ab640 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.148+0000 7ff3ecfbd640 1 --2- 172.21.15.70:0/2160229177 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7ff3c806ad40 0x7ff3c806d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.149+0000 7ff3ecfbd640 1 --2- 172.21.15.70:0/2160229177 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7ff3c806ad40 0x7ff3c806d200 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8007fd0 tx=0x7ff3d8009040 comp rx=0 tx=0).ready entity=mgr.9804 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:09.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.152+0000 7ff3e57fa640 1 -- 172.21.15.70:0/2160229177 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3e81669d0 con 0x7ff3e80c9cf0 2023-10-08T06:54:09.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.364+0000 7ff3ef221640 1 -- 172.21.15.70:0/2160229177 --> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3e80c8f30 con 0x7ff3c806ad40 2023-10-08T06:54:09.382 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:09.380+0000 7f87d9315640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-08T06:54:09.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.381+0000 7ff3e57fa640 1 -- 172.21.15.70:0/2160229177 <== mgr.9804 v2:172.21.15.181:6832/1385757024 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7ff3d8004910 con 0x7ff3c806ad40 2023-10-08T06:54:09.383 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-08T06:54:09.392 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T06:54:09.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:09.393 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T06:54:09.450 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T06:54:09.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.667+0000 7feeff59e640 1 Processor -- start 2023-10-08T06:54:09.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.668+0000 7feeff59e640 1 -- start start 2023-10-08T06:54:09.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.669+0000 7feeff59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef0015aab0 0x7fef00070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.669+0000 7feeff59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0015ae90 con 0x7fef000cb350 2023-10-08T06:54:09.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.669+0000 7feeff59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef000cbdb0 con 0x7fef000caef0 2023-10-08T06:54:09.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.669+0000 7feeff59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef000cb7f0 con 0x7fef0015aab0 2023-10-08T06:54:09.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.670+0000 7feefed9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fef000cb350 0x7fef000b1c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50350/0 (socket says 172.21.15.70:50350) 2023-10-08T06:54:09.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.670+0000 7feefed9d640 1 -- 172.21.15.70:0/3928602681 learned_addr learned my addr 172.21.15.70:0/3928602681 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:09.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.670+0000 7feefe59c640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef0015aab0 0x7fef00070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.670+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683808439 0 0) 0x7feef8001490 con 0x7fef000cb350 2023-10-08T06:54:09.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.670+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feee8003520 con 0x7fef000cb350 2023-10-08T06:54:09.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 171970708 0 0) 0x7feee8003520 con 0x7fef000cb350 2023-10-08T06:54:09.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef0015aab0 msgr2=0x7fef00070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:09.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef0015aab0 0x7fef00070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 >> v1:172.21.15.181:6789/0 conn(0x7fef000caef0 legacy=0x7fef000aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:09.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000cbad0 con 0x7fef000cb350 2023-10-08T06:54:09.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7feef8001b80 con 0x7fef000cb350 2023-10-08T06:54:09.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 >> v1:172.21.15.70:6789/0 conn(0x7fef000cb350 legacy=0x7fef000b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:09.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee80039a0 0x7feee8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 0x7feee8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef0015aab0 0x7feee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef000cb7f0 con 0x7feee80047c0 2023-10-08T06:54:09.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee800bab0 con 0x7feee80039a0 2023-10-08T06:54:09.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee800bc70 con 0x7fef0015aab0 2023-10-08T06:54:09.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefdd9b640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 0x7feee8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.672+0000 7feefe59c640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee80039a0 0x7feee8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.671+0000 7feefed9d640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef0015aab0 0x7feee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.672+0000 7feefdd9b640 1 -- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef0015aab0 msgr2=0x7feee80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:09.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.672+0000 7feefdd9b640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef0015aab0 0x7feee80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.672+0000 7feefdd9b640 1 -- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee80039a0 msgr2=0x7feee8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:09.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.672+0000 7feefdd9b640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee80039a0 0x7feee8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.672+0000 7feefdd9b640 1 -- 172.21.15.70:0/3928602681 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8003520 con 0x7feee80047c0 2023-10-08T06:54:09.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.672+0000 7feefe59c640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feee80039a0 0x7feee8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:09.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.672+0000 7feefed9d640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef0015aab0 0x7feee80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:09.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.673+0000 7feefdd9b640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 0x7feee8004c40 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7feef4001750 tx=0x7feee80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fdd892ef353d94 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:09.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.673+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef40052d0 con 0x7feee80047c0 2023-10-08T06:54:09.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.673+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef402e050 con 0x7feee80047c0 2023-10-08T06:54:09.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.673+0000 7feefd59a640 1 -- 172.21.15.70:0/3928602681 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef40035f0 con 0x7feee80047c0 2023-10-08T06:54:09.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.673+0000 7feeff59e640 1 -- 172.21.15.70:0/3928602681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 msgr2=0x7feee8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:09.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.673+0000 7feeff59e640 1 --2- 172.21.15.70:0/3928602681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 0x7feee8004c40 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7feef4001750 tx=0x7feee80057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.673+0000 7feefed9d640 1 -- 172.21.15.70:0/3928602681 reap_dead start 2023-10-08T06:54:09.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.674+0000 7feeff59e640 1 -- 172.21.15.70:0/3928602681 shutdown_connections 2023-10-08T06:54:09.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.674+0000 7feeff59e640 1 -- 172.21.15.70:0/3928602681 >> 172.21.15.70:0/3928602681 conn(0x7fef0006c010 msgr2=0x7fef0006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:09.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.674+0000 7feeff59e640 1 -- 172.21.15.70:0/3928602681 shutdown_connections 2023-10-08T06:54:09.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.674+0000 7feeff59e640 1 -- 172.21.15.70:0/3928602681 wait complete. 2023-10-08T06:54:09.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.675+0000 7feeff59e640 1 Processor -- start 2023-10-08T06:54:09.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.676+0000 7feeff59e640 1 -- start start 2023-10-08T06:54:09.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.676+0000 7feeff59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 0x7fef000c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feeff59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef00155f20 0x7fef000c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feeff59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef00157040 0x7fef001574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feeff59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef00156bb0 con 0x7feee80047c0 2023-10-08T06:54:09.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feeff59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef000c7150 con 0x7fef00157040 2023-10-08T06:54:09.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feeff59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef000c6e70 con 0x7fef00155f20 2023-10-08T06:54:09.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feefe59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 0x7fef000c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feefe59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 0x7fef000c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42624/0 (socket says 172.21.15.70:42624) 2023-10-08T06:54:09.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feefe59c640 1 -- 172.21.15.70:0/470720924 learned_addr learned my addr 172.21.15.70:0/470720924 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:09.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feefe59c640 1 -- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef00155f20 msgr2=0x7fef000c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:09.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feefe59c640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef00155f20 0x7fef000c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feefe59c640 1 -- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef00157040 msgr2=0x7fef001574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:09.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feefe59c640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef00157040 0x7fef001574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:09.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feefe59c640 1 -- 172.21.15.70:0/470720924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef00156560 con 0x7feee80047c0 2023-10-08T06:54:09.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.677+0000 7feefe59c640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 0x7fef000c6890 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7feef00108b0 tx=0x7feef0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:09.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.678+0000 7feeeeffd640 1 -- 172.21.15.70:0/470720924 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef000eb60 con 0x7feee80047c0 2023-10-08T06:54:09.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.678+0000 7feeeeffd640 1 -- 172.21.15.70:0/470720924 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef000ecc0 con 0x7feee80047c0 2023-10-08T06:54:09.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.678+0000 7feeeeffd640 1 -- 172.21.15.70:0/470720924 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef001c400 con 0x7feee80047c0 2023-10-08T06:54:09.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.678+0000 7feeff59e640 1 -- 172.21.15.70:0/470720924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef00156840 con 0x7feee80047c0 2023-10-08T06:54:09.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.678+0000 7feeff59e640 1 -- 172.21.15.70:0/470720924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef000ac920 con 0x7feee80047c0 2023-10-08T06:54:09.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.679+0000 7feeff59e640 1 -- 172.21.15.70:0/470720924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef001563c0 con 0x7feee80047c0 2023-10-08T06:54:09.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.680+0000 7feeeeffd640 1 -- 172.21.15.70:0/470720924 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 92) v1 ==== 89084+0+0 (secure 0 0 0) 0x7feef001c560 con 0x7feee80047c0 2023-10-08T06:54:09.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.681+0000 7feeeeffd640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7feee006ad10 0x7feee006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:09.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.681+0000 7feeeeffd640 1 -- 172.21.15.70:0/470720924 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7feef0089c50 con 0x7feee80047c0 2023-10-08T06:54:09.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.682+0000 7feefdd9b640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7feee006ad10 0x7feee006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:09.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.683+0000 7feefdd9b640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7feee006ad10 0x7feee006d1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feef4033000 tx=0x7feef402f000 comp rx=0 tx=0).ready entity=mgr.9804 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:09.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.685+0000 7feeeeffd640 1 -- 172.21.15.70:0/470720924 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef001563c0 con 0x7feee80047c0 2023-10-08T06:54:09.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.814+0000 7feeeeffd640 1 -- 172.21.15.70:0/470720924 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 93) v1 ==== 89349+0+0 (secure 0 0 0) 0x7feef00bcca0 con 0x7feee80047c0 2023-10-08T06:54:09.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:09.930+0000 7feeff59e640 1 -- 172.21.15.70:0/470720924 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fef001563c0 con 0x7feee80047c0 2023-10-08T06:54:10.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.813+0000 7feeeeffd640 1 -- 172.21.15.70:0/470720924 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v94) v1 ==== 93+0+0 (secure 0 0 0) 0x7fef001563c0 con 0x7feee80047c0 2023-10-08T06:54:10.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.817+0000 7feeff59e640 1 -- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7feee006ad10 msgr2=0x7feee006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:10.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.817+0000 7feeff59e640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7feee006ad10 0x7feee006d1d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feef4033000 tx=0x7feef402f000 comp rx=0 tx=0).stop 2023-10-08T06:54:10.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.817+0000 7feeff59e640 1 -- 172.21.15.70:0/470720924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 msgr2=0x7fef000c6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:10.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.817+0000 7feeff59e640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 0x7fef000c6890 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7feef00108b0 tx=0x7feef0010d70 comp rx=0 tx=0).stop 2023-10-08T06:54:10.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.818+0000 7feeff59e640 1 -- 172.21.15.70:0/470720924 shutdown_connections 2023-10-08T06:54:10.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.818+0000 7feeff59e640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef00157040 0x7fef001574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:10.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.818+0000 7feeff59e640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef00155f20 0x7fef000c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:10.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.818+0000 7feeff59e640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7feee006ad10 0x7feee006d1d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:10.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.818+0000 7feeff59e640 1 --2- 172.21.15.70:0/470720924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feee80047c0 0x7fef000c6890 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:10.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.818+0000 7feeff59e640 1 -- 172.21.15.70:0/470720924 >> 172.21.15.70:0/470720924 conn(0x7fef0006c010 msgr2=0x7fef000c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:10.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.819+0000 7feeff59e640 1 -- 172.21.15.70:0/470720924 shutdown_connections 2023-10-08T06:54:10.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:10.819+0000 7feeff59e640 1 -- 172.21.15.70:0/470720924 wait complete. 2023-10-08T06:54:10.842 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T06:54:10.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T06:54:10.945 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T06:54:11.037 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T06:54:11.038 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:54:11.038 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:11.091 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:54:11.091 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:11.187 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:54:11.343 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:11.341+0000 7f82cae14040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T06:54:11.344 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:11.344+0000 7f82c6c14040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T06:54:11.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.384+0000 7f5cf4646640 1 Processor -- start 2023-10-08T06:54:11.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.384+0000 7f5cf4646640 1 -- start start 2023-10-08T06:54:11.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.386+0000 7f5cf4646640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec070db0 0x7f5cec15dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.386+0000 7f5cf4646640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec0c9850 con 0x7f5cec071190 2023-10-08T06:54:11.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.386+0000 7f5cf4646640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cec06a870 con 0x7f5cec06ad00 2023-10-08T06:54:11.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.386+0000 7f5cf4646640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec15e280 con 0x7f5cec070db0 2023-10-08T06:54:11.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.386+0000 7f5cf1be1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5cec071190 0x7f5cec15e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50366/0 (socket says 172.21.15.70:50366) 2023-10-08T06:54:11.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.386+0000 7f5cf1be1640 1 -- 172.21.15.70:0/4054753247 learned_addr learned my addr 172.21.15.70:0/4054753247 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:11.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.386+0000 7f5cf23e2640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec070db0 0x7f5cec15dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025691633 0 0) 0x7f5cec0c9850 con 0x7f5cec071190 2023-10-08T06:54:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cd8003520 con 0x7f5cec071190 2023-10-08T06:54:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1487436918 0 0) 0x7f5cd8003520 con 0x7f5cec071190 2023-10-08T06:54:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec070db0 msgr2=0x7f5cec15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec070db0 0x7f5cec15dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 >> v1:172.21.15.181:6789/0 conn(0x7f5cec06ad00 legacy=0x7f5cec06b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec15eb90 con 0x7f5cec071190 2023-10-08T06:54:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5cdc001670 con 0x7f5cec071190 2023-10-08T06:54:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 >> v1:172.21.15.70:6789/0 conn(0x7f5cec071190 legacy=0x7f5cec15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec070db0 0x7f5cd80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 0x7f5cd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8004d00 0x7f5cd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec15e280 con 0x7f5cd8003f70 2023-10-08T06:54:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd800bb50 con 0x7f5cd8004d00 2023-10-08T06:54:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd800bd10 con 0x7f5cec070db0 2023-10-08T06:54:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.387+0000 7f5cf1be1640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 0x7f5cd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.388+0000 7f5cf23e2640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec070db0 0x7f5cd80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.388+0000 7f5cf1be1640 1 -- 172.21.15.70:0/4054753247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec070db0 msgr2=0x7f5cd80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.388+0000 7f5cf1be1640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec070db0 0x7f5cd80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.388+0000 7f5cf1be1640 1 -- 172.21.15.70:0/4054753247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8004d00 msgr2=0x7f5cd8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.388+0000 7f5cf1be1640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cd8004d00 0x7f5cd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.388+0000 7f5cf1be1640 1 -- 172.21.15.70:0/4054753247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd8003520 con 0x7f5cd8003f70 2023-10-08T06:54:11.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.389+0000 7f5cf1be1640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 0x7f5cd80047c0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c0e0 tx=0x7f5cd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25953b883a9a7729 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:11.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.390+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc005ad0 con 0x7f5cd8003f70 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.390+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc005c30 con 0x7f5cd8003f70 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.390+0000 7f5cf13e0640 1 -- 172.21.15.70:0/4054753247 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0072b0 con 0x7f5cd8003f70 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.390+0000 7f5cf4646640 1 -- 172.21.15.70:0/4054753247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 msgr2=0x7f5cd80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.390+0000 7f5cf4646640 1 --2- 172.21.15.70:0/4054753247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 0x7f5cd80047c0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c0e0 tx=0x7f5cd80057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.390+0000 7f5cf2be3640 1 -- 172.21.15.70:0/4054753247 reap_dead start 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.390+0000 7f5cf4646640 1 -- 172.21.15.70:0/4054753247 shutdown_connections 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.390+0000 7f5cf4646640 1 -- 172.21.15.70:0/4054753247 >> 172.21.15.70:0/4054753247 conn(0x7f5cec0b1480 msgr2=0x7f5cec06cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.391+0000 7f5cf4646640 1 -- 172.21.15.70:0/4054753247 shutdown_connections 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.391+0000 7f5cf4646640 1 -- 172.21.15.70:0/4054753247 wait complete. 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.392+0000 7f5cf4646640 1 Processor -- start 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.392+0000 7f5cf4646640 1 -- start start 2023-10-08T06:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.393+0000 7f5cf4646640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 0x7f5cec15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.393+0000 7f5cf4646640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec15b6a0 0x7f5cec15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.393+0000 7f5cf4646640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0c2b90 0x7f5cec0c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.393+0000 7f5cf4646640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec15c060 con 0x7f5cd8003f70 2023-10-08T06:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.393+0000 7f5cf4646640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec15aaf0 con 0x7f5cec15b6a0 2023-10-08T06:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.393+0000 7f5cf4646640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0c38a0 con 0x7f5cec0c2b90 2023-10-08T06:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf23e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 0x7f5cec15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf23e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 0x7f5cec15a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42636/0 (socket says 172.21.15.70:42636) 2023-10-08T06:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf23e2640 1 -- 172.21.15.70:0/615710016 learned_addr learned my addr 172.21.15.70:0/615710016 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf2be3640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0c2b90 0x7f5cec0c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf23e2640 1 -- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0c2b90 msgr2=0x7f5cec0c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf23e2640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0c2b90 0x7f5cec0c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf23e2640 1 -- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec15b6a0 msgr2=0x7f5cec15bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf23e2640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec15b6a0 0x7f5cec15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf23e2640 1 -- 172.21.15.70:0/615710016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec0c3b80 con 0x7f5cd8003f70 2023-10-08T06:54:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf2be3640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0c2b90 0x7f5cec0c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5cf23e2640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 0x7f5cec15a5b0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8010860 tx=0x7f5ce8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.394+0000 7f5ce2ffd640 1 -- 172.21.15.70:0/615710016 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8009610 con 0x7f5cd8003f70 2023-10-08T06:54:11.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.395+0000 7f5ce2ffd640 1 -- 172.21.15.70:0/615710016 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce800eb60 con 0x7f5cd8003f70 2023-10-08T06:54:11.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.395+0000 7f5ce2ffd640 1 -- 172.21.15.70:0/615710016 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce801c400 con 0x7f5cd8003f70 2023-10-08T06:54:11.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.395+0000 7f5cf4646640 1 -- 172.21.15.70:0/615710016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cec0af080 con 0x7f5cd8003f70 2023-10-08T06:54:11.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.395+0000 7f5cf4646640 1 -- 172.21.15.70:0/615710016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec0aed30 con 0x7f5cd8003f70 2023-10-08T06:54:11.400 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:11.397+0000 7f82c6c14040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T06:54:11.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.396+0000 7f5cf4646640 1 -- 172.21.15.70:0/615710016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec0cd5c0 con 0x7f5cd8003f70 2023-10-08T06:54:11.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.397+0000 7f5ce2ffd640 1 -- 172.21.15.70:0/615710016 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 94) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f5ce801c560 con 0x7f5cd8003f70 2023-10-08T06:54:11.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.398+0000 7f5ce2ffd640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f5cd006afc0 0x7f5cd006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.398+0000 7f5ce2ffd640 1 -- 172.21.15.70:0/615710016 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f5ce8089f60 con 0x7f5cd8003f70 2023-10-08T06:54:11.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.398+0000 7f5cf1be1640 1 -- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f5cd006afc0 msgr2=0x7f5cd006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:11.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.398+0000 7f5cf1be1640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f5cd006afc0 0x7f5cd006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:11.402 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:11.398+0000 7f82cae14040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T06:54:11.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.402+0000 7f5ce2ffd640 1 -- 172.21.15.70:0/615710016 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ce8054d30 con 0x7f5cd8003f70 2023-10-08T06:54:11.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.593+0000 7f5cf4646640 1 -- 172.21.15.70:0/615710016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cec067490 con 0x7f5cd8003f70 2023-10-08T06:54:11.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.595+0000 7f5ce2ffd640 1 -- 172.21.15.70:0/615710016 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+17407 (secure 0 0 0) 0x7f5ce8058350 con 0x7f5cd8003f70 2023-10-08T06:54:11.597 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:11.597 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":186,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:54:07.074946+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:53:44.876671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-08T06:53:46.310621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/4245513204":"2023-10-09T06:54:07.074878+0000","172.21.15.181:6832/4245513204":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/1356319692":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/4081249518":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2711042436":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2536889452":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2861165849":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/4176429568":"2023-10-09T06:54:07.074878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:54:11.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 -- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f5cd006afc0 msgr2=0x7f5cd006d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:11.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f5cd006afc0 0x7f5cd006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 -- 172.21.15.70:0/615710016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 msgr2=0x7f5cec15a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:11.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 0x7f5cec15a5b0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8010860 tx=0x7f5ce8010d20 comp rx=0 tx=0).stop 2023-10-08T06:54:11.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 -- 172.21.15.70:0/615710016 shutdown_connections 2023-10-08T06:54:11.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cec0c2b90 0x7f5cec0c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cec15b6a0 0x7f5cec15bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f5cd006afc0 0x7f5cd006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 --2- 172.21.15.70:0/615710016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cd8003f70 0x7f5cec15a5b0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 -- 172.21.15.70:0/615710016 >> 172.21.15.70:0/615710016 conn(0x7f5cec0b1480 msgr2=0x7f5cec0cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:11.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 -- 172.21.15.70:0/615710016 shutdown_connections 2023-10-08T06:54:11.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.598+0000 7f5cf4646640 1 -- 172.21.15.70:0/615710016 wait complete. 2023-10-08T06:54:11.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T06:53:44.876671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T06:53:46.310621+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T06:54:11.619 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T06:54:11.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.821+0000 7f8569a4b640 1 Processor -- start 2023-10-08T06:54:11.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.821+0000 7f8569a4b640 1 -- start start 2023-10-08T06:54:11.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.823+0000 7f8569a4b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f856415cd80 0x7f856406cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.823+0000 7f8569a4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85640cb140 0x7f856406d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.823+0000 7f8569a4b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85640caf30 con 0x7f8564065ab0 2023-10-08T06:54:11.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.823+0000 7f8569a4b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f856406dbd0 con 0x7f856415cd80 2023-10-08T06:54:11.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.823+0000 7f8569a4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856406deb0 con 0x7f85640cb140 2023-10-08T06:54:11.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.823+0000 7f855a7fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85640cb140 0x7f856406d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:11.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.823+0000 7f855a7fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85640cb140 0x7f856406d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:48982/0 (socket says 172.21.15.70:48982) 2023-10-08T06:54:11.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.823+0000 7f855a7fc640 1 -- 172.21.15.70:0/345713836 learned_addr learned my addr 172.21.15.70:0/345713836 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:11.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.824+0000 7f8562ffd640 1 --2- 172.21.15.70:0/345713836 >> v2:172.21.15.181:3300/0 conn(0x7f856415cd80 0x7f856406cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:11.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.824+0000 7f8562ffd640 1 -- 172.21.15.70:0/345713836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85640cb140 msgr2=0x7f856406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:11.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.824+0000 7f8562ffd640 1 --2- 172.21.15.70:0/345713836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85640cb140 0x7f856406d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.824+0000 7f8562ffd640 1 -- 172.21.15.70:0/345713836 >> v1:172.21.15.181:6789/0 conn(0x7f8564065ab0 legacy=0x7f8564160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:11.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.824+0000 7f8562ffd640 1 -- 172.21.15.70:0/345713836 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85640c9a00 con 0x7f856415cd80 2023-10-08T06:54:11.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.824+0000 7f855a7fc640 1 --2- 172.21.15.70:0/345713836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85640cb140 0x7f856406d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T06:54:11.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.826+0000 7f8562ffd640 1 --2- 172.21.15.70:0/345713836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f856415cd80 0x7f856406cc60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f85540097a0 tx=0x7f855402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3afbf77dfcb8cc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:11.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.826+0000 7f85627fc640 1 -- 172.21.15.70:0/345713836 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855402d040 con 0x7f856415cd80 2023-10-08T06:54:11.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.826+0000 7f85627fc640 1 -- 172.21.15.70:0/345713836 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8554031810 con 0x7f856415cd80 2023-10-08T06:54:11.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.826+0000 7f85627fc640 1 -- 172.21.15.70:0/345713836 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554031ba0 con 0x7f856415cd80 2023-10-08T06:54:11.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.827+0000 7f8569a4b640 1 -- 172.21.15.70:0/345713836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f856415cd80 msgr2=0x7f856406cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:11.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.827+0000 7f8569a4b640 1 --2- 172.21.15.70:0/345713836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f856415cd80 0x7f856406cc60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f85540097a0 tx=0x7f855402b8b0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.827+0000 7f8569a4b640 1 -- 172.21.15.70:0/345713836 shutdown_connections 2023-10-08T06:54:11.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.827+0000 7f8569a4b640 1 --2- 172.21.15.70:0/345713836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85640cb140 0x7f856406d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.827+0000 7f8569a4b640 1 --2- 172.21.15.70:0/345713836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f856415cd80 0x7f856406cc60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.827+0000 7f8569a4b640 1 -- 172.21.15.70:0/345713836 >> 172.21.15.70:0/345713836 conn(0x7f85640b0aa0 msgr2=0x7f85640b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:11.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.827+0000 7f8569a4b640 1 -- 172.21.15.70:0/345713836 shutdown_connections 2023-10-08T06:54:11.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.827+0000 7f8569a4b640 1 -- 172.21.15.70:0/345713836 wait complete. 2023-10-08T06:54:11.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.829+0000 7f8569a4b640 1 Processor -- start 2023-10-08T06:54:11.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.830+0000 7f8569a4b640 1 -- start start 2023-10-08T06:54:11.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.831+0000 7f8569a4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8564065ab0 0x7f85640c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.831+0000 7f8569a4b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85640cb140 0x7f85640c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.831+0000 7f8569a4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f856415cd80 0x7f85641f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.831+0000 7f8569a4b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85640c7190 con 0x7f856415cd80 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.831+0000 7f8569a4b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85640c7e30 con 0x7f85640cb140 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.831+0000 7f8569a4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85640c8110 con 0x7f8564065ab0 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.831+0000 7f85637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f856415cd80 0x7f85641f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.832+0000 7f85637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f856415cd80 0x7f85641f1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42648/0 (socket says 172.21.15.70:42648) 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.832+0000 7f85637fe640 1 -- 172.21.15.70:0/1313214330 learned_addr learned my addr 172.21.15.70:0/1313214330 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.832+0000 7f8562ffd640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8564065ab0 0x7f85640c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.832+0000 7f855a7fc640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85640cb140 0x7f85640c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.832+0000 7f85637fe640 1 -- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8564065ab0 msgr2=0x7f85640c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.832+0000 7f85637fe640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8564065ab0 0x7f85640c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.832+0000 7f85637fe640 1 -- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85640cb140 msgr2=0x7f85640c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:11.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.832+0000 7f85637fe640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85640cb140 0x7f85640c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:11.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.832+0000 7f85637fe640 1 -- 172.21.15.70:0/1313214330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c0094d0 con 0x7f856415cd80 2023-10-08T06:54:11.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.832+0000 7f85637fe640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f856415cd80 0x7f85641f1f80 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f855c00c6b0 tx=0x7f855c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:11.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.833+0000 7f8560ff9640 1 -- 172.21.15.70:0/1313214330 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c003b70 con 0x7f856415cd80 2023-10-08T06:54:11.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.833+0000 7f8560ff9640 1 -- 172.21.15.70:0/1313214330 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855c003cf0 con 0x7f856415cd80 2023-10-08T06:54:11.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.833+0000 7f8560ff9640 1 -- 172.21.15.70:0/1313214330 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c003ea0 con 0x7f856415cd80 2023-10-08T06:54:11.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.833+0000 7f8569a4b640 1 -- 172.21.15.70:0/1313214330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8554009400 con 0x7f856415cd80 2023-10-08T06:54:11.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.833+0000 7f8569a4b640 1 -- 172.21.15.70:0/1313214330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856415d380 con 0x7f856415cd80 2023-10-08T06:54:11.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.834+0000 7f8569a4b640 1 -- 172.21.15.70:0/1313214330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856404e160 con 0x7f856415cd80 2023-10-08T06:54:11.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.835+0000 7f8560ff9640 1 -- 172.21.15.70:0/1313214330 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 94) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f855c005310 con 0x7f856415cd80 2023-10-08T06:54:11.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.836+0000 7f8560ff9640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f854406af90 0x7f854406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:11.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.837+0000 7f8560ff9640 1 -- 172.21.15.70:0/1313214330 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f855c0898e0 con 0x7f856415cd80 2023-10-08T06:54:11.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.837+0000 7f8562ffd640 1 -- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f854406af90 msgr2=0x7f854406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:11.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.837+0000 7f8562ffd640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f854406af90 0x7f854406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:11.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:11.841+0000 7f8560ff9640 1 -- 172.21.15.70:0/1313214330 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f855c0544e0 con 0x7f856415cd80 2023-10-08T06:54:11.844 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:11.843+0000 7f82c6c14040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T06:54:11.865 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:11.864+0000 7f82cae14040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T06:54:11.900 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:11.899+0000 7f82c6c14040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T06:54:11.924 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:11.922+0000 7f82cae14040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T06:54:11.950 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:11.949+0000 7f82c6c14040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T06:54:11.977 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:11.975+0000 7f82cae14040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T06:54:12.011 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:12.010+0000 7f82c6c14040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T06:54:12.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.034+0000 7f8569a4b640 1 -- 172.21.15.70:0/1313214330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f856415d160 con 0x7f856415cd80 2023-10-08T06:54:12.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.035+0000 7f8560ff9640 1 -- 172.21.15.70:0/1313214330 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7f855c057b00 con 0x7f856415cd80 2023-10-08T06:54:12.036 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T06:54:12.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 -- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f854406af90 msgr2=0x7f854406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:12.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f854406af90 0x7f854406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 -- 172.21.15.70:0/1313214330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f856415cd80 msgr2=0x7f85641f1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f856415cd80 0x7f85641f1f80 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f855c00c6b0 tx=0x7f855c00cb70 comp rx=0 tx=0).stop 2023-10-08T06:54:12.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 -- 172.21.15.70:0/1313214330 shutdown_connections 2023-10-08T06:54:12.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f854406af90 0x7f854406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f856415cd80 0x7f85641f1f80 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f85640cb140 0x7f85640c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 --2- 172.21.15.70:0/1313214330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8564065ab0 0x7f85640c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 -- 172.21.15.70:0/1313214330 >> 172.21.15.70:0/1313214330 conn(0x7f85640b0aa0 msgr2=0x7f8564160890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:12.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 -- 172.21.15.70:0/1313214330 shutdown_connections 2023-10-08T06:54:12.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.037+0000 7f8569a4b640 1 -- 172.21.15.70:0/1313214330 wait complete. 2023-10-08T06:54:12.040 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:12.038+0000 7f82cae14040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T06:54:12.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T06:54:12.059 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:12.058+0000 7f82c6c14040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T06:54:12.092 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:12.091+0000 7f82cae14040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T06:54:12.168 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:12.166+0000 7f82c6c14040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T06:54:12.209 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:12.207+0000 7f82cae14040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T06:54:12.229 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:12.228+0000 7f82c6c14040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:12.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.255+0000 7f2215209640 1 Processor -- start 2023-10-08T06:54:12.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.256+0000 7f2215209640 1 -- start start 2023-10-08T06:54:12.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.257+0000 7f2215209640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2210071c30 0x7f2210069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.257+0000 7f2215209640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22100720c0 0x7f221006a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.257+0000 7f2215209640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22100ce210 con 0x7f22101601b0 2023-10-08T06:54:12.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.257+0000 7f2215209640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22100cdb40 con 0x7f2210071c30 2023-10-08T06:54:12.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.257+0000 7f2215209640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22100cdee0 con 0x7f22100720c0 2023-10-08T06:54:12.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.257+0000 7f2207fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2210071c30 0x7f2210069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.257+0000 7f2207fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2210071c30 0x7f2210069d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42656/0 (socket says 172.21.15.70:42656) 2023-10-08T06:54:12.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.257+0000 7f2207fff640 1 -- 172.21.15.70:0/2255726390 learned_addr learned my addr 172.21.15.70:0/2255726390 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:12.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.258+0000 7f220f59e640 1 --2- 172.21.15.70:0/2255726390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22100720c0 0x7f221006a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.258+0000 7f2207fff640 1 -- 172.21.15.70:0/2255726390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22100720c0 msgr2=0x7f221006a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:12.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.258+0000 7f2207fff640 1 --2- 172.21.15.70:0/2255726390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22100720c0 0x7f221006a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.258+0000 7f2207fff640 1 -- 172.21.15.70:0/2255726390 >> v1:172.21.15.181:6789/0 conn(0x7f22101601b0 legacy=0x7f22100cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:12.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.258+0000 7f2207fff640 1 -- 172.21.15.70:0/2255726390 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2210161530 con 0x7f2210071c30 2023-10-08T06:54:12.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.260+0000 7f2207fff640 1 --2- 172.21.15.70:0/2255726390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2210071c30 0x7f2210069d50 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009870 tx=0x7f21f802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d44b1c27d074e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:12.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.260+0000 7f220e59c640 1 -- 172.21.15.70:0/2255726390 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8030840 con 0x7f2210071c30 2023-10-08T06:54:12.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.260+0000 7f220e59c640 1 -- 172.21.15.70:0/2255726390 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f80309a0 con 0x7f2210071c30 2023-10-08T06:54:12.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.260+0000 7f220e59c640 1 -- 172.21.15.70:0/2255726390 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8030d30 con 0x7f2210071c30 2023-10-08T06:54:12.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.261+0000 7f2215209640 1 -- 172.21.15.70:0/2255726390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2210071c30 msgr2=0x7f2210069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.261+0000 7f2215209640 1 --2- 172.21.15.70:0/2255726390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2210071c30 0x7f2210069d50 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009870 tx=0x7f21f802b980 comp rx=0 tx=0).stop 2023-10-08T06:54:12.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.261+0000 7f2215209640 1 -- 172.21.15.70:0/2255726390 shutdown_connections 2023-10-08T06:54:12.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.261+0000 7f2215209640 1 --2- 172.21.15.70:0/2255726390 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22100720c0 0x7f221006a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.261+0000 7f2215209640 1 --2- 172.21.15.70:0/2255726390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2210071c30 0x7f2210069d50 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.261+0000 7f2215209640 1 -- 172.21.15.70:0/2255726390 >> 172.21.15.70:0/2255726390 conn(0x7f221006d430 msgr2=0x7f221006d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:12.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.261+0000 7f2215209640 1 -- 172.21.15.70:0/2255726390 shutdown_connections 2023-10-08T06:54:12.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.261+0000 7f2215209640 1 -- 172.21.15.70:0/2255726390 wait complete. 2023-10-08T06:54:12.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.263+0000 7f2215209640 1 Processor -- start 2023-10-08T06:54:12.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.263+0000 7f2215209640 1 -- start start 2023-10-08T06:54:12.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2215209640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2210071c30 0x7f22100c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2215209640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22100720c0 0x7f2210154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2215209640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22101601b0 0x7f22101f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2215209640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22101554a0 con 0x7f22100720c0 2023-10-08T06:54:12.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2215209640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22100ab9e0 con 0x7f22101601b0 2023-10-08T06:54:12.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2215209640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22100abcc0 con 0x7f2210071c30 2023-10-08T06:54:12.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2207fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22100720c0 0x7f2210154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2207fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22100720c0 0x7f2210154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42664/0 (socket says 172.21.15.70:42664) 2023-10-08T06:54:12.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2207fff640 1 -- 172.21.15.70:0/833504194 learned_addr learned my addr 172.21.15.70:0/833504194 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:12.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2207fff640 1 -- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2210071c30 msgr2=0x7f22100c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f220ed9d640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2210071c30 0x7f22100c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2207fff640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2210071c30 0x7f22100c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2207fff640 1 -- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22101601b0 msgr2=0x7f22101f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2207fff640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22101601b0 0x7f22101f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.264+0000 7f2207fff640 1 -- 172.21.15.70:0/833504194 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80094d0 con 0x7f22100720c0 2023-10-08T06:54:12.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.265+0000 7f2207fff640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22100720c0 0x7f2210154f60 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f21f8000c00 tx=0x7f21f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:12.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.265+0000 7f22077fe640 1 -- 172.21.15.70:0/833504194 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8039a90 con 0x7f22100720c0 2023-10-08T06:54:12.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.265+0000 7f22077fe640 1 -- 172.21.15.70:0/833504194 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f8039bf0 con 0x7f22100720c0 2023-10-08T06:54:12.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.265+0000 7f22077fe640 1 -- 172.21.15.70:0/833504194 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8043410 con 0x7f22100720c0 2023-10-08T06:54:12.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.265+0000 7f2215209640 1 -- 172.21.15.70:0/833504194 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22100abfa0 con 0x7f22100720c0 2023-10-08T06:54:12.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.265+0000 7f2215209640 1 -- 172.21.15.70:0/833504194 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22100ac560 con 0x7f22100720c0 2023-10-08T06:54:12.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.266+0000 7f2215209640 1 -- 172.21.15.70:0/833504194 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22100ac780 con 0x7f22100720c0 2023-10-08T06:54:12.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.267+0000 7f22077fe640 1 -- 172.21.15.70:0/833504194 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 94) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f21f802f390 con 0x7f22100720c0 2023-10-08T06:54:12.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.268+0000 7f22077fe640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f21f006af90 0x7f21f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.269+0000 7f22077fe640 1 -- 172.21.15.70:0/833504194 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f21f80abd60 con 0x7f22100720c0 2023-10-08T06:54:12.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.269+0000 7f220ed9d640 1 -- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f21f006af90 msgr2=0x7f21f006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:12.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.269+0000 7f220ed9d640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f21f006af90 0x7f21f006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:12.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.273+0000 7f22077fe640 1 -- 172.21.15.70:0/833504194 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21f80768e0 con 0x7f22100720c0 2023-10-08T06:54:12.276 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:12.275+0000 7f82cae14040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:12.278 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:12.277+0000 7f82c6c14040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T06:54:12.328 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:12.327+0000 7f82cae14040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T06:54:12.335 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:12.334+0000 7f82c6c14040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T06:54:12.385 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:12.384+0000 7f82c6c14040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T06:54:12.387 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:12.386+0000 7f82cae14040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T06:54:12.439 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:12.438+0000 7f82cae14040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T06:54:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.470+0000 7f220ed9d640 1 -- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f21f006af90 msgr2=0x7f21f006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.470+0000 7f220ed9d640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f21f006af90 0x7f21f006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:54:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.502+0000 7f2215209640 1 -- 172.21.15.70:0/833504194 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f221004e160 con 0x7f22100720c0 2023-10-08T06:54:12.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.503+0000 7f22077fe640 1 -- 172.21.15.70:0/833504194 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v186) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21f8034020 con 0x7f22100720c0 2023-10-08T06:54:12.504 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T06:54:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 -- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f21f006af90 msgr2=0x7f21f006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f21f006af90 0x7f21f006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 -- 172.21.15.70:0/833504194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22100720c0 msgr2=0x7f2210154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22100720c0 0x7f2210154f60 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f21f8000c00 tx=0x7f21f8004730 comp rx=0 tx=0).stop 2023-10-08T06:54:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 -- 172.21.15.70:0/833504194 shutdown_connections 2023-10-08T06:54:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22101601b0 0x7f22101f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f21f006af90 0x7f21f006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22100720c0 0x7f2210154f60 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 --2- 172.21.15.70:0/833504194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2210071c30 0x7f22100c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 -- 172.21.15.70:0/833504194 >> 172.21.15.70:0/833504194 conn(0x7f221006d430 msgr2=0x7f22100cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 -- 172.21.15.70:0/833504194 shutdown_connections 2023-10-08T06:54:12.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.505+0000 7f2215209640 1 -- 172.21.15.70:0/833504194 wait complete. 2023-10-08T06:54:12.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T06:54:12.766 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:12.762+0000 7f82c6c14040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T06:54:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.769+0000 7f1d4ce83640 1 Processor -- start 2023-10-08T06:54:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.770+0000 7f1d4ce83640 1 -- start start 2023-10-08T06:54:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.771+0000 7f1d4ce83640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d4815ccb0 0x7f1d480d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.771+0000 7f1d4ce83640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d480d2870 con 0x7f1d480d0b90 2023-10-08T06:54:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.771+0000 7f1d4ce83640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d480d1150 con 0x7f1d480d00e0 2023-10-08T06:54:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.771+0000 7f1d4ce83640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480d1430 con 0x7f1d4815ccb0 2023-10-08T06:54:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.772+0000 7f1d3ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1d480d0b90 0x7f1d480cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50380/0 (socket says 172.21.15.70:50380) 2023-10-08T06:54:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.772+0000 7f1d3ffff640 1 -- 172.21.15.70:0/2141331391 learned_addr learned my addr 172.21.15.70:0/2141331391 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.772+0000 7f1d477fe640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d4815ccb0 0x7f1d480d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.772+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3711084788 0 0) 0x7f1d480d2870 con 0x7f1d480d0b90 2023-10-08T06:54:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.772+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d34003520 con 0x7f1d480d0b90 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.772+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 185758364 0 0) 0x7f1d34003520 con 0x7f1d480d0b90 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.772+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d4815ccb0 msgr2=0x7f1d480d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d4815ccb0 0x7f1d480d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 >> v1:172.21.15.181:6789/0 conn(0x7f1d480d00e0 legacy=0x7f1d480b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d480d1710 con 0x7f1d480d0b90 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1d30001670 con 0x7f1d480d0b90 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 >> v1:172.21.15.70:6789/0 conn(0x7f1d480d0b90 legacy=0x7f1d480cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d340039a0 0x7f1d34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d4815ccb0 0x7f1d340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d34004d00 0x7f1d34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480d1430 con 0x7f1d34004d00 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3400ba30 con 0x7f1d340039a0 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3400bbf0 con 0x7f1d4815ccb0 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d47fff640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d34004d00 0x7f1d34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d3ffff640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d4815ccb0 0x7f1d340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d477fe640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d340039a0 0x7f1d34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d47fff640 1 -- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d4815ccb0 msgr2=0x7f1d340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d47fff640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d4815ccb0 0x7f1d340047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d47fff640 1 -- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d340039a0 msgr2=0x7f1d34003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d47fff640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d340039a0 0x7f1d34003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d47fff640 1 -- 172.21.15.70:0/2141331391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34003520 con 0x7f1d34004d00 2023-10-08T06:54:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.773+0000 7f1d477fe640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d340039a0 0x7f1d34003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.774+0000 7f1d47fff640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d34004d00 0x7f1d34005180 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f1d480b85f0 tx=0x7f1d340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28673d84827b856 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.775+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d40003b70 con 0x7f1d34004d00 2023-10-08T06:54:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.775+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d40003cd0 con 0x7f1d34004d00 2023-10-08T06:54:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.775+0000 7f1d46ffd640 1 -- 172.21.15.70:0/2141331391 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d40005310 con 0x7f1d34004d00 2023-10-08T06:54:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.775+0000 7f1d4ce83640 1 -- 172.21.15.70:0/2141331391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d34004d00 msgr2=0x7f1d34005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.775+0000 7f1d4ce83640 1 --2- 172.21.15.70:0/2141331391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d34004d00 0x7f1d34005180 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f1d480b85f0 tx=0x7f1d340057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.775+0000 7f1d47fff640 1 -- 172.21.15.70:0/2141331391 reap_dead start 2023-10-08T06:54:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.775+0000 7f1d4ce83640 1 -- 172.21.15.70:0/2141331391 shutdown_connections 2023-10-08T06:54:12.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.775+0000 7f1d4ce83640 1 -- 172.21.15.70:0/2141331391 >> 172.21.15.70:0/2141331391 conn(0x7f1d48071660 msgr2=0x7f1d48071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.776+0000 7f1d4ce83640 1 -- 172.21.15.70:0/2141331391 shutdown_connections 2023-10-08T06:54:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.776+0000 7f1d4ce83640 1 -- 172.21.15.70:0/2141331391 wait complete. 2023-10-08T06:54:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.777+0000 7f1d4ce83640 1 Processor -- start 2023-10-08T06:54:12.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.777+0000 7f1d4ce83640 1 -- start start 2023-10-08T06:54:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.778+0000 7f1d4ce83640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d48156000 0x7f1d4815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.778+0000 7f1d4ce83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d481563e0 0x7f1d4815d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.778+0000 7f1d4ce83640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d480c8110 0x7f1d4815de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.778+0000 7f1d4ce83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480c9350 con 0x7f1d481563e0 2023-10-08T06:54:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.778+0000 7f1d4ce83640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480c86d0 con 0x7f1d48156000 2023-10-08T06:54:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.778+0000 7f1d4ce83640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d480c89b0 con 0x7f1d480c8110 2023-10-08T06:54:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d3ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d481563e0 0x7f1d4815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d3ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d481563e0 0x7f1d4815d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42682/0 (socket says 172.21.15.70:42682) 2023-10-08T06:54:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d3ffff640 1 -- 172.21.15.70:0/716521715 learned_addr learned my addr 172.21.15.70:0/716521715 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d477fe640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d48156000 0x7f1d4815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d47fff640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d480c8110 0x7f1d4815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d3ffff640 1 -- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d480c8110 msgr2=0x7f1d4815de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d3ffff640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d480c8110 0x7f1d4815de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d3ffff640 1 -- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d48156000 msgr2=0x7f1d4815e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d3ffff640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d48156000 0x7f1d4815e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d3ffff640 1 -- 172.21.15.70:0/716521715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d480c8c90 con 0x7f1d481563e0 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.779+0000 7f1d477fe640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d48156000 0x7f1d4815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.780+0000 7f1d3ffff640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d481563e0 0x7f1d4815d910 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000dae0 tx=0x7f1d30001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.780+0000 7f1d44ff9640 1 -- 172.21.15.70:0/716521715 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30004cb0 con 0x7f1d481563e0 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.780+0000 7f1d44ff9640 1 -- 172.21.15.70:0/716521715 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d30004e10 con 0x7f1d481563e0 2023-10-08T06:54:12.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.780+0000 7f1d44ff9640 1 -- 172.21.15.70:0/716521715 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30007310 con 0x7f1d481563e0 2023-10-08T06:54:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.780+0000 7f1d4ce83640 1 -- 172.21.15.70:0/716521715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d480c8f70 con 0x7f1d481563e0 2023-10-08T06:54:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.780+0000 7f1d4ce83640 1 -- 172.21.15.70:0/716521715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d480c66f0 con 0x7f1d481563e0 2023-10-08T06:54:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.781+0000 7f1d4ce83640 1 -- 172.21.15.70:0/716521715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d480b5ad0 con 0x7f1d481563e0 2023-10-08T06:54:12.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.783+0000 7f1d44ff9640 1 -- 172.21.15.70:0/716521715 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 94) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f1d30007470 con 0x7f1d481563e0 2023-10-08T06:54:12.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.784+0000 7f1d44ff9640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1d2806afc0 0x7f1d2806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.784+0000 7f1d44ff9640 1 -- 172.21.15.70:0/716521715 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f1d30089590 con 0x7f1d481563e0 2023-10-08T06:54:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.784+0000 7f1d477fe640 1 -- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1d2806afc0 msgr2=0x7f1d2806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:12.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.784+0000 7f1d477fe640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1d2806afc0 0x7f1d2806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.788+0000 7f1d44ff9640 1 -- 172.21.15.70:0/716521715 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d30054100 con 0x7f1d481563e0 2023-10-08T06:54:12.842 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:12.840+0000 7f82cae14040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T06:54:12.925 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:12.924+0000 7f82c6c14040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T06:54:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.979+0000 7f1d4ce83640 1 -- 172.21.15.70:0/716521715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d481f1ea0 con 0x7f1d481563e0 2023-10-08T06:54:12.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.980+0000 7f1d44ff9640 1 -- 172.21.15.70:0/716521715 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v186) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d30057720 con 0x7f1d481563e0 2023-10-08T06:54:12.981 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T06:54:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 -- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1d2806afc0 msgr2=0x7f1d2806d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1d2806afc0 0x7f1d2806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 -- 172.21.15.70:0/716521715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d481563e0 msgr2=0x7f1d4815d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d481563e0 0x7f1d4815d910 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000dae0 tx=0x7f1d30001d80 comp rx=0 tx=0).stop 2023-10-08T06:54:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 -- 172.21.15.70:0/716521715 shutdown_connections 2023-10-08T06:54:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d480c8110 0x7f1d4815de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1d2806afc0 0x7f1d2806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d481563e0 0x7f1d4815d910 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 --2- 172.21.15.70:0/716521715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d48156000 0x7f1d4815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 -- 172.21.15.70:0/716521715 >> 172.21.15.70:0/716521715 conn(0x7f1d48071660 msgr2=0x7f1d480b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:12.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 -- 172.21.15.70:0/716521715 shutdown_connections 2023-10-08T06:54:12.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:12.983+0000 7f1d4ce83640 1 -- 172.21.15.70:0/716521715 wait complete. 2023-10-08T06:54:13.002 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T06:54:13.007 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:13.006+0000 7f82cae14040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T06:54:13.033 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:13.031+0000 7f82c6c14040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T06:54:13.082 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:13.080+0000 7f82c6c14040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T06:54:13.122 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:13.120+0000 7f82cae14040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T06:54:13.173 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:13.172+0000 7f82cae14040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T06:54:13.188 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:13.187+0000 7f82c6c14040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:13.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.202+0000 7f1f47247640 1 Processor -- start 2023-10-08T06:54:13.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.202+0000 7f1f47247640 1 -- start start 2023-10-08T06:54:13.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f47247640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f400cf2b0 0x7f1f400d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f47247640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1f400ce510 0x7f1f4015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f47247640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f400cd520 con 0x7f1f400cf690 2023-10-08T06:54:13.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f47247640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f400d1910 con 0x7f1f400cf2b0 2023-10-08T06:54:13.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f47247640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f400ce080 con 0x7f1f400ce510 2023-10-08T06:54:13.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f46245640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f400cf2b0 0x7f1f400d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f46245640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1f400cf2b0 0x7f1f400d34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42694/0 (socket says 172.21.15.70:42694) 2023-10-08T06:54:13.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f46245640 1 -- 172.21.15.70:0/4133326128 learned_addr learned my addr 172.21.15.70:0/4133326128 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:13.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f46245640 1 -- 172.21.15.70:0/4133326128 >> v2:172.21.15.181:3300/0 conn(0x7f1f400ce510 msgr2=0x7f1f4015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:13.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f46245640 1 --2- 172.21.15.70:0/4133326128 >> v2:172.21.15.181:3300/0 conn(0x7f1f400ce510 0x7f1f4015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f46245640 1 -- 172.21.15.70:0/4133326128 >> v1:172.21.15.181:6789/0 conn(0x7f1f400cf690 legacy=0x7f1f400d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:13.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.204+0000 7f1f46245640 1 -- 172.21.15.70:0/4133326128 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f400cd750 con 0x7f1f400cf2b0 2023-10-08T06:54:13.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.205+0000 7f1f46245640 1 --2- 172.21.15.70:0/4133326128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400cf2b0 0x7f1f400d34d0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f1f400cfb20 tx=0x7f1f3c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfc768a5ba697ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:13.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.206+0000 7f1f45243640 1 -- 172.21.15.70:0/4133326128 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0317e0 con 0x7f1f400cf2b0 2023-10-08T06:54:13.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.206+0000 7f1f45243640 1 -- 172.21.15.70:0/4133326128 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c031940 con 0x7f1f400cf2b0 2023-10-08T06:54:13.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.206+0000 7f1f45243640 1 -- 172.21.15.70:0/4133326128 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c031cd0 con 0x7f1f400cf2b0 2023-10-08T06:54:13.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.206+0000 7f1f47247640 1 -- 172.21.15.70:0/4133326128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400cf2b0 msgr2=0x7f1f400d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:13.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.206+0000 7f1f47247640 1 --2- 172.21.15.70:0/4133326128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400cf2b0 0x7f1f400d34d0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f1f400cfb20 tx=0x7f1f3c02b880 comp rx=0 tx=0).stop 2023-10-08T06:54:13.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.206+0000 7f1f47247640 1 -- 172.21.15.70:0/4133326128 shutdown_connections 2023-10-08T06:54:13.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.206+0000 7f1f47247640 1 --2- 172.21.15.70:0/4133326128 >> v2:172.21.15.181:3300/0 conn(0x7f1f400ce510 0x7f1f4015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.206+0000 7f1f47247640 1 --2- 172.21.15.70:0/4133326128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400cf2b0 0x7f1f400d34d0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.207+0000 7f1f47247640 1 -- 172.21.15.70:0/4133326128 >> 172.21.15.70:0/4133326128 conn(0x7f1f4009bc40 msgr2=0x7f1f400d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:13.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.207+0000 7f1f47247640 1 -- 172.21.15.70:0/4133326128 shutdown_connections 2023-10-08T06:54:13.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.207+0000 7f1f47247640 1 -- 172.21.15.70:0/4133326128 wait complete. 2023-10-08T06:54:13.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.208+0000 7f1f47247640 1 Processor -- start 2023-10-08T06:54:13.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.208+0000 7f1f47247640 1 -- start start 2023-10-08T06:54:13.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.209+0000 7f1f47247640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400ce510 0x7f1f400c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.209+0000 7f1f47247640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f400cf2b0 0x7f1f4014b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.209+0000 7f1f47247640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f400cf690 0x7f1f4009c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.209+0000 7f1f47247640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4014d070 con 0x7f1f400ce510 2023-10-08T06:54:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.209+0000 7f1f47247640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4014c3f0 con 0x7f1f400cf690 2023-10-08T06:54:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.209+0000 7f1f47247640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4014c6d0 con 0x7f1f400cf2b0 2023-10-08T06:54:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f46245640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400ce510 0x7f1f400c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f46245640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400ce510 0x7f1f400c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42704/0 (socket says 172.21.15.70:42704) 2023-10-08T06:54:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f46245640 1 -- 172.21.15.70:0/897161093 learned_addr learned my addr 172.21.15.70:0/897161093 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f45a44640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f400cf2b0 0x7f1f4014b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f46245640 1 -- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f400cf2b0 msgr2=0x7f1f4014b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f46245640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f400cf2b0 0x7f1f4014b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f46245640 1 -- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f400cf690 msgr2=0x7f1f4009c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:13.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f46a46640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f400cf690 0x7f1f4009c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f46245640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f400cf690 0x7f1f4009c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f46245640 1 -- 172.21.15.70:0/897161093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3c0094d0 con 0x7f1f400ce510 2023-10-08T06:54:13.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f45a44640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f400cf2b0 0x7f1f4014b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:13.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.210+0000 7f1f46245640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400ce510 0x7f1f400c86b0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c02b8c0 tx=0x7f1f3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:13.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.211+0000 7f1f36ffd640 1 -- 172.21.15.70:0/897161093 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c032940 con 0x7f1f400ce510 2023-10-08T06:54:13.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.211+0000 7f1f36ffd640 1 -- 172.21.15.70:0/897161093 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c032aa0 con 0x7f1f400ce510 2023-10-08T06:54:13.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.211+0000 7f1f36ffd640 1 -- 172.21.15.70:0/897161093 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c032e50 con 0x7f1f400ce510 2023-10-08T06:54:13.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.211+0000 7f1f47247640 1 -- 172.21.15.70:0/897161093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4014c9b0 con 0x7f1f400ce510 2023-10-08T06:54:13.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.211+0000 7f1f47247640 1 -- 172.21.15.70:0/897161093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f400c8bf0 con 0x7f1f400ce510 2023-10-08T06:54:13.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.212+0000 7f1f47247640 1 -- 172.21.15.70:0/897161093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f40074150 con 0x7f1f400ce510 2023-10-08T06:54:13.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.213+0000 7f1f36ffd640 1 -- 172.21.15.70:0/897161093 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 94) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f1f3c04e450 con 0x7f1f400ce510 2023-10-08T06:54:13.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.214+0000 7f1f36ffd640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1f2406af90 0x7f1f2406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.214+0000 7f1f36ffd640 1 -- 172.21.15.70:0/897161093 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f1f3c0abe20 con 0x7f1f400ce510 2023-10-08T06:54:13.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.214+0000 7f1f45a44640 1 -- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1f2406af90 msgr2=0x7f1f2406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:13.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.214+0000 7f1f45a44640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1f2406af90 0x7f1f2406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:13.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.218+0000 7f1f36ffd640 1 -- 172.21.15.70:0/897161093 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f3c076990 con 0x7f1f400ce510 2023-10-08T06:54:13.287 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:13.285+0000 7f82cae14040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:13.295 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:13.294+0000 7f82c6c14040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T06:54:13.399 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:13.396+0000 7f82cae14040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T06:54:13.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.415+0000 7f1f45a44640 1 -- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1f2406af90 msgr2=0x7f1f2406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:13.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.415+0000 7f1f45a44640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1f2406af90 0x7f1f2406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:54:13.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.420+0000 7f1f47247640 1 -- 172.21.15.70:0/897161093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1f400d0070 con 0x7f1f400ce510 2023-10-08T06:54:13.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.421+0000 7f1f36ffd640 1 -- 172.21.15.70:0/897161093 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v186) v1 ==== 80+0+478 (secure 0 0 0) 0x7f1f3c079fb0 con 0x7f1f400ce510 2023-10-08T06:54:13.422 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/4245513204 2023-10-09T06:54:07.074878+0000 2023-10-08T06:54:13.422 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/4245513204 2023-10-09T06:54:07.074878+0000 2023-10-08T06:54:13.423 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1356319692 2023-10-09T06:54:07.074878+0000 2023-10-08T06:54:13.423 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/4081249518 2023-10-09T06:54:07.074878+0000 2023-10-08T06:54:13.423 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2711042436 2023-10-09T06:54:07.074878+0000 2023-10-08T06:54:13.423 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2536889452 2023-10-09T06:54:07.074878+0000 2023-10-08T06:54:13.423 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2861165849 2023-10-09T06:54:07.074878+0000 2023-10-08T06:54:13.423 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/4176429568 2023-10-09T06:54:07.074878+0000 2023-10-08T06:54:13.423 INFO:teuthology.orchestra.run.smithi070.stderr:listed 8 entries 2023-10-08T06:54:13.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 -- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1f2406af90 msgr2=0x7f1f2406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:13.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1f2406af90 0x7f1f2406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 -- 172.21.15.70:0/897161093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400ce510 msgr2=0x7f1f400c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:13.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400ce510 0x7f1f400c86b0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c02b8c0 tx=0x7f1f3c0046c0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 -- 172.21.15.70:0/897161093 shutdown_connections 2023-10-08T06:54:13.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1f400cf690 0x7f1f4009c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1f400cf2b0 0x7f1f4014b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f1f2406af90 0x7f1f2406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 --2- 172.21.15.70:0/897161093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f400ce510 0x7f1f400c86b0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 -- 172.21.15.70:0/897161093 >> 172.21.15.70:0/897161093 conn(0x7f1f4009bc40 msgr2=0x7f1f4015bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:13.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 -- 172.21.15.70:0/897161093 shutdown_connections 2023-10-08T06:54:13.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.423+0000 7f1f47247640 1 -- 172.21.15.70:0/897161093 wait complete. 2023-10-08T06:54:13.443 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T06:54:13.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:13.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T06:54:13.450 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:13.449+0000 7f82c6c14040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:13.499 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:13.498+0000 7f82c6c14040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:13.559 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T06:54:13.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:13.560 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T06:54:13.565 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:13.563+0000 7f82cae14040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:13.606 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T06:54:13.606 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:13.617 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:13.616+0000 7f82cae14040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:13.620 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:13.619+0000 7f82c6c14040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T06:54:13.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T06:54:13.661 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:54:13.661 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:13.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T06:54:13.715 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T06:54:13.715 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:13.741 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:13.739+0000 7f82cae14040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T06:54:13.770 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T06:54:13.770 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:54:13.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.970+0000 7f094c939640 1 Processor -- start 2023-10-08T06:54:13.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.970+0000 7f094c939640 1 -- start start 2023-10-08T06:54:13.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.972+0000 7f094c939640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0948070620 0x7f094806d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.972+0000 7f094c939640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f094815ead0 0x7f094815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.972+0000 7f094c939640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f094806da40 con 0x7f09480b11f0 2023-10-08T06:54:13.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.972+0000 7f094c939640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09480cd3e0 con 0x7f094815ead0 2023-10-08T06:54:13.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.972+0000 7f094c939640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094806cca0 con 0x7f0948070620 2023-10-08T06:54:13.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.972+0000 7f093ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f09480b11f0 0x7f0948070a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51298/0 (socket says 172.21.15.70:51298) 2023-10-08T06:54:13.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.972+0000 7f093ffff640 1 -- 172.21.15.70:0/2891483608 learned_addr learned my addr 172.21.15.70:0/2891483608 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:13.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f094659c640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0948070620 0x7f094806d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0946d9d640 1 --2- 172.21.15.70:0/2891483608 >> v2:172.21.15.181:3300/0 conn(0x7f094815ead0 0x7f094815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198296382 0 0) 0x7f094806da40 con 0x7f09480b11f0 2023-10-08T06:54:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0934003520 con 0x7f09480b11f0 2023-10-08T06:54:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3062056257 0 0) 0x7f0934003520 con 0x7f09480b11f0 2023-10-08T06:54:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0948070620 msgr2=0x7f094806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0948070620 0x7f094806d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 >> v2:172.21.15.181:3300/0 conn(0x7f094815ead0 msgr2=0x7f094815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:13.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 --2- 172.21.15.70:0/2891483608 >> v2:172.21.15.181:3300/0 conn(0x7f094815ead0 0x7f094815ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094806ddc0 con 0x7f09480b11f0 2023-10-08T06:54:13.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0930001670 con 0x7f09480b11f0 2023-10-08T06:54:13.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 >> v1:172.21.15.70:6789/0 conn(0x7f09480b11f0 legacy=0x7f0948070a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:13.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0934003970 0x7f0934003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 0x7f0934004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0948070620 0x7f0934005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09480cd3e0 con 0x7f09340047c0 2023-10-08T06:54:13.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094806cca0 con 0x7f0934003970 2023-10-08T06:54:13.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093400b980 con 0x7f0948070620 2023-10-08T06:54:13.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.973+0000 7f093ffff640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 0x7f0934004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.974+0000 7f0946d9d640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0948070620 0x7f0934005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.974+0000 7f093ffff640 1 -- 172.21.15.70:0/2891483608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0948070620 msgr2=0x7f0934005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:13.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.974+0000 7f093ffff640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0948070620 0x7f0934005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.974+0000 7f093ffff640 1 -- 172.21.15.70:0/2891483608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0934003970 msgr2=0x7f0934003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:13.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.974+0000 7f093ffff640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0934003970 0x7f0934003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.974+0000 7f093ffff640 1 -- 172.21.15.70:0/2891483608 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0934003520 con 0x7f09340047c0 2023-10-08T06:54:13.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.975+0000 7f093ffff640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 0x7f0934004c40 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f093000c2c0 tx=0x7f09340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab50c70d5b382f52 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:13.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.975+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930005ad0 con 0x7f09340047c0 2023-10-08T06:54:13.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.975+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0930005c30 con 0x7f09340047c0 2023-10-08T06:54:13.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.975+0000 7f0945d9b640 1 -- 172.21.15.70:0/2891483608 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093002c8b0 con 0x7f09340047c0 2023-10-08T06:54:13.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.976+0000 7f094c939640 1 -- 172.21.15.70:0/2891483608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 msgr2=0x7f0934004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:13.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.976+0000 7f094c939640 1 --2- 172.21.15.70:0/2891483608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 0x7f0934004c40 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f093000c2c0 tx=0x7f09340057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.976+0000 7f0946d9d640 1 -- 172.21.15.70:0/2891483608 reap_dead start 2023-10-08T06:54:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.976+0000 7f094c939640 1 -- 172.21.15.70:0/2891483608 shutdown_connections 2023-10-08T06:54:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.976+0000 7f094c939640 1 -- 172.21.15.70:0/2891483608 >> 172.21.15.70:0/2891483608 conn(0x7f09480b0700 msgr2=0x7f09480b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.976+0000 7f094c939640 1 -- 172.21.15.70:0/2891483608 shutdown_connections 2023-10-08T06:54:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.976+0000 7f094c939640 1 -- 172.21.15.70:0/2891483608 wait complete. 2023-10-08T06:54:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.978+0000 7f094c939640 1 Processor -- start 2023-10-08T06:54:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.978+0000 7f094c939640 1 -- start start 2023-10-08T06:54:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.979+0000 7f094c939640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 0x7f09480ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.979+0000 7f094c939640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f094815bea0 0x7f09480af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.979+0000 7f094c939640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09480ca970 0x7f09480cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.979+0000 7f094c939640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09480acb30 con 0x7f09340047c0 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.979+0000 7f094c939640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09480cb360 con 0x7f09480ca970 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.979+0000 7f094c939640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094815c320 con 0x7f094815bea0 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.979+0000 7f094659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 0x7f09480ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.979+0000 7f094659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 0x7f09480ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57596/0 (socket says 172.21.15.70:57596) 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.979+0000 7f094659c640 1 -- 172.21.15.70:0/2099554379 learned_addr learned my addr 172.21.15.70:0/2099554379 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f093ffff640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f094815bea0 0x7f09480af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f094659c640 1 -- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f094815bea0 msgr2=0x7f09480af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f094659c640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f094815bea0 0x7f09480af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f094659c640 1 -- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09480ca970 msgr2=0x7f09480cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:13.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f0946d9d640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09480ca970 0x7f09480cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f094659c640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09480ca970 0x7f09480cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f094659c640 1 -- 172.21.15.70:0/2099554379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480abeb0 con 0x7f09340047c0 2023-10-08T06:54:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f094659c640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 0x7f09480ab8d0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f093800e7d0 tx=0x7f093800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f093effd640 1 -- 172.21.15.70:0/2099554379 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093800eee0 con 0x7f09340047c0 2023-10-08T06:54:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f093effd640 1 -- 172.21.15.70:0/2099554379 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093800c940 con 0x7f09340047c0 2023-10-08T06:54:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f093effd640 1 -- 172.21.15.70:0/2099554379 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093800ccd0 con 0x7f09340047c0 2023-10-08T06:54:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f094c939640 1 -- 172.21.15.70:0/2099554379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09480ac190 con 0x7f09340047c0 2023-10-08T06:54:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.980+0000 7f094c939640 1 -- 172.21.15.70:0/2099554379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09480ac750 con 0x7f09340047c0 2023-10-08T06:54:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.982+0000 7f094c939640 1 -- 172.21.15.70:0/2099554379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0948067490 con 0x7f09340047c0 2023-10-08T06:54:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.983+0000 7f093effd640 1 -- 172.21.15.70:0/2099554379 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 94) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f0938013470 con 0x7f09340047c0 2023-10-08T06:54:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.983+0000 7f093effd640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.984+0000 7f093effd640 1 -- 172.21.15.70:0/2099554379 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f0938089aa0 con 0x7f09340047c0 2023-10-08T06:54:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.984+0000 7f093ffff640 1 -- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f092806af90 msgr2=0x7f092806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:13.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.984+0000 7f093ffff640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:13.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:13.992+0000 7f093effd640 1 -- 172.21.15.70:0/2099554379 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0938054730 con 0x7f09340047c0 2023-10-08T06:54:14.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.185+0000 7f093ffff640 1 -- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f092806af90 msgr2=0x7f092806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:14.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.185+0000 7f093ffff640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:54:14.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.222+0000 7f094c939640 1 -- 172.21.15.70:0/2099554379 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09480ac910 con 0x7f09340047c0 2023-10-08T06:54:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.223+0000 7f093effd640 1 -- 172.21.15.70:0/2099554379 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4917 (secure 0 0 0) 0x7f0938017090 con 0x7f09340047c0 2023-10-08T06:54:14.225 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:14.225 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/542411062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":542411062},{"type":"v1","addr":"172.21.15.181:6836","nonce":542411062}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/646194010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":646194010},{"type":"v1","addr":"172.21.15.70:6835","nonce":646194010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1850202608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1850202608},{"type":"v1","addr":"172.21.15.181:6837","nonce":1850202608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:53:47.285813+0000","modified":"2023-10-08T06:53:50.356439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9591":{"gid":9591,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/3790014600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3790014600},{"type":"v1","addr":"172.21.15.70:6834","nonce":3790014600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-08T06:54:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 83 2023-10-08T06:54:14.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 -- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f092806af90 msgr2=0x7f092806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:14.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 -- 172.21.15.70:0/2099554379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 msgr2=0x7f09480ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:14.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 0x7f09480ab8d0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f093800e7d0 tx=0x7f093800ec90 comp rx=0 tx=0).stop 2023-10-08T06:54:14.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 -- 172.21.15.70:0/2099554379 shutdown_connections 2023-10-08T06:54:14.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09480ca970 0x7f09480cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f094815bea0 0x7f09480af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f092806af90 0x7f092806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 --2- 172.21.15.70:0/2099554379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09340047c0 0x7f09480ab8d0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 -- 172.21.15.70:0/2099554379 >> 172.21.15.70:0/2099554379 conn(0x7f09480b0700 msgr2=0x7f094815ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 -- 172.21.15.70:0/2099554379 shutdown_connections 2023-10-08T06:54:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.226+0000 7f094c939640 1 -- 172.21.15.70:0/2099554379 wait complete. 2023-10-08T06:54:14.244 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:54:14.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.431+0000 7f71470c2640 1 Processor -- start 2023-10-08T06:54:14.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.431+0000 7f71470c2640 1 -- start start 2023-10-08T06:54:14.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f71470c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f714015ab40 0x7f71400ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f71470c2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f71400c90b0 0x7f71400adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f71470c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f71400cae70 0x7f71400cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f71470c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714015bf30 con 0x7f71400cae70 2023-10-08T06:54:14.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f71470c2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71400cb840 con 0x7f71400c90b0 2023-10-08T06:54:14.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f71470c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714015b2b0 con 0x7f714015ab40 2023-10-08T06:54:14.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f714565f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f71400cae70 0x7f71400cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f714565f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f71400cae70 0x7f71400cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57610/0 (socket says 172.21.15.70:57610) 2023-10-08T06:54:14.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f714565f640 1 -- 172.21.15.70:0/607787399 learned_addr learned my addr 172.21.15.70:0/607787399 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:14.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f7144e5e640 1 --2- 172.21.15.70:0/607787399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f714015ab40 0x7f71400ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.433+0000 7f713ffff640 1 --2- 172.21.15.70:0/607787399 >> v2:172.21.15.181:3300/0 conn(0x7f71400c90b0 0x7f71400adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.434+0000 7f714565f640 1 -- 172.21.15.70:0/607787399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f714015ab40 msgr2=0x7f71400ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:14.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.434+0000 7f714565f640 1 --2- 172.21.15.70:0/607787399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f714015ab40 0x7f71400ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.434+0000 7f714565f640 1 -- 172.21.15.70:0/607787399 >> v2:172.21.15.181:3300/0 conn(0x7f71400c90b0 msgr2=0x7f71400adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:14.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.434+0000 7f714565f640 1 --2- 172.21.15.70:0/607787399 >> v2:172.21.15.181:3300/0 conn(0x7f71400c90b0 0x7f71400adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.434+0000 7f714565f640 1 -- 172.21.15.70:0/607787399 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714015b590 con 0x7f71400cae70 2023-10-08T06:54:14.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.434+0000 7f713ffff640 1 --2- 172.21.15.70:0/607787399 >> v2:172.21.15.181:3300/0 conn(0x7f71400c90b0 0x7f71400adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:14.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.434+0000 7f7144e5e640 1 --2- 172.21.15.70:0/607787399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f714015ab40 0x7f71400ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:14.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.435+0000 7f714565f640 1 --2- 172.21.15.70:0/607787399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71400cae70 0x7f71400cb300 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f714006e470 tx=0x7f713802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cee5df516fd91dc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:14.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.435+0000 7f713f7fe640 1 -- 172.21.15.70:0/607787399 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713802ca80 con 0x7f71400cae70 2023-10-08T06:54:14.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.435+0000 7f713f7fe640 1 -- 172.21.15.70:0/607787399 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713802cbe0 con 0x7f71400cae70 2023-10-08T06:54:14.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.435+0000 7f713f7fe640 1 -- 172.21.15.70:0/607787399 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380343f0 con 0x7f71400cae70 2023-10-08T06:54:14.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.436+0000 7f71470c2640 1 -- 172.21.15.70:0/607787399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71400cae70 msgr2=0x7f71400cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:14.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.436+0000 7f71470c2640 1 --2- 172.21.15.70:0/607787399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71400cae70 0x7f71400cb300 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f714006e470 tx=0x7f713802bee0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.436+0000 7f71470c2640 1 -- 172.21.15.70:0/607787399 shutdown_connections 2023-10-08T06:54:14.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.436+0000 7f71470c2640 1 --2- 172.21.15.70:0/607787399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71400cae70 0x7f71400cb300 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.436+0000 7f71470c2640 1 --2- 172.21.15.70:0/607787399 >> v2:172.21.15.181:3300/0 conn(0x7f71400c90b0 0x7f71400adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.436+0000 7f71470c2640 1 --2- 172.21.15.70:0/607787399 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f714015ab40 0x7f71400ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.436+0000 7f71470c2640 1 -- 172.21.15.70:0/607787399 >> 172.21.15.70:0/607787399 conn(0x7f714006a380 msgr2=0x7f71400b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:14.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.436+0000 7f71470c2640 1 -- 172.21.15.70:0/607787399 shutdown_connections 2023-10-08T06:54:14.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.436+0000 7f71470c2640 1 -- 172.21.15.70:0/607787399 wait complete. 2023-10-08T06:54:14.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.438+0000 7f71470c2640 1 Processor -- start 2023-10-08T06:54:14.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.438+0000 7f71470c2640 1 -- start start 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f71470c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71400c90b0 0x7f71400acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f71470c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71400cae70 0x7f7140153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f71470c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714015ab40 0x7f7140159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f71470c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400ad2d0 con 0x7f714015ab40 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f71470c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c6640 con 0x7f71400c90b0 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f71470c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c6920 con 0x7f71400cae70 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f714565f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714015ab40 0x7f7140159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f714565f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714015ab40 0x7f7140159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57612/0 (socket says 172.21.15.70:57612) 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f714565f640 1 -- 172.21.15.70:0/649730626 learned_addr learned my addr 172.21.15.70:0/649730626 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f7144e5e640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71400c90b0 0x7f71400acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f713ffff640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71400cae70 0x7f7140153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f714565f640 1 -- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71400cae70 msgr2=0x7f7140153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f714565f640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71400cae70 0x7f7140153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f714565f640 1 -- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71400c90b0 msgr2=0x7f71400acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f714565f640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71400c90b0 0x7f71400acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.439+0000 7f714565f640 1 -- 172.21.15.70:0/649730626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7138009b30 con 0x7f714015ab40 2023-10-08T06:54:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.440+0000 7f714565f640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714015ab40 0x7f7140159180 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f7138036040 tx=0x7f7138004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.440+0000 7f713d7fa640 1 -- 172.21.15.70:0/649730626 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138009510 con 0x7f714015ab40 2023-10-08T06:54:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.440+0000 7f713d7fa640 1 -- 172.21.15.70:0/649730626 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713803e450 con 0x7f714015ab40 2023-10-08T06:54:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.440+0000 7f713d7fa640 1 -- 172.21.15.70:0/649730626 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713803e800 con 0x7f714015ab40 2023-10-08T06:54:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.440+0000 7f71470c2640 1 -- 172.21.15.70:0/649730626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71400c6c00 con 0x7f714015ab40 2023-10-08T06:54:14.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.440+0000 7f71470c2640 1 -- 172.21.15.70:0/649730626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71400c71c0 con 0x7f714015ab40 2023-10-08T06:54:14.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.441+0000 7f71470c2640 1 -- 172.21.15.70:0/649730626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71400c73b0 con 0x7f714015ab40 2023-10-08T06:54:14.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.442+0000 7f713d7fa640 1 -- 172.21.15.70:0/649730626 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 94) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f713802cd50 con 0x7f714015ab40 2023-10-08T06:54:14.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.443+0000 7f713d7fa640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f712c06af90 0x7f712c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.443+0000 7f713d7fa640 1 -- 172.21.15.70:0/649730626 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f71380ac6c0 con 0x7f714015ab40 2023-10-08T06:54:14.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.443+0000 7f7144e5e640 1 -- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f712c06af90 msgr2=0x7f712c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:14.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.443+0000 7f7144e5e640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f712c06af90 0x7f712c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:14.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.448+0000 7f713d7fa640 1 -- 172.21.15.70:0/649730626 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7138077400 con 0x7f714015ab40 2023-10-08T06:54:14.464 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:14.463+0000 7f82c6c14040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T06:54:14.574 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:14.572+0000 7f82cae14040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T06:54:14.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.639+0000 7f71470c2640 1 -- 172.21.15.70:0/649730626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7140067490 con 0x7f714015ab40 2023-10-08T06:54:14.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.641+0000 7f713d7fa640 1 -- 172.21.15.70:0/649730626 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+17407 (secure 0 0 0) 0x7f713807aa20 con 0x7f714015ab40 2023-10-08T06:54:14.643 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:14.643 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":186,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:54:07.074946+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:53:44.876671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-08T06:53:46.310621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/4245513204":"2023-10-09T06:54:07.074878+0000","172.21.15.181:6832/4245513204":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/1356319692":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/4081249518":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2711042436":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2536889452":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2861165849":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/4176429568":"2023-10-09T06:54:07.074878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:54:14.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 -- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f712c06af90 msgr2=0x7f712c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T06:54:14.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f712c06af90 0x7f712c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 -- 172.21.15.70:0/649730626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714015ab40 msgr2=0x7f7140159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:14.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714015ab40 0x7f7140159180 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f7138036040 tx=0x7f7138004a40 comp rx=0 tx=0).stop 2023-10-08T06:54:14.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 -- 172.21.15.70:0/649730626 shutdown_connections 2023-10-08T06:54:14.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f712c06af90 0x7f712c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714015ab40 0x7f7140159180 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71400cae70 0x7f7140153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 --2- 172.21.15.70:0/649730626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71400c90b0 0x7f71400acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 -- 172.21.15.70:0/649730626 >> 172.21.15.70:0/649730626 conn(0x7f714006a380 msgr2=0x7f71400ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:14.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 -- 172.21.15.70:0/649730626 shutdown_connections 2023-10-08T06:54:14.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.645+0000 7f71470c2640 1 -- 172.21.15.70:0/649730626 wait complete. 2023-10-08T06:54:14.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T06:53:44.876671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T06:53:46.310621+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '185', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T06:54:14.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T06:54:14.758 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:14.756+0000 7f82c6c14040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T06:54:14.814 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:14.812+0000 7f82c6c14040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T06:54:14.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.869+0000 7f571a90f640 1 Processor -- start 2023-10-08T06:54:14.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.870+0000 7f571a90f640 1 -- start start 2023-10-08T06:54:14.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.871+0000 7f571a90f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5714069d50 0x7f57140ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.871+0000 7f571a90f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 0x7f571406b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.871+0000 7f571a90f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571415d340 con 0x7f57140cdda0 2023-10-08T06:54:14.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.871+0000 7f571a90f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571406a1f0 con 0x7f5714069d50 2023-10-08T06:54:14.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.871+0000 7f571a90f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571406a4d0 con 0x7f57140ce940 2023-10-08T06:54:14.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.872+0000 7f5718eac640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 0x7f571406b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.872+0000 7f5718eac640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 0x7f571406b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:37796/0 (socket says 172.21.15.70:37796) 2023-10-08T06:54:14.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.872+0000 7f5718eac640 1 -- 172.21.15.70:0/2212059170 learned_addr learned my addr 172.21.15.70:0/2212059170 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:14.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.872+0000 7f57137fe640 1 --2- 172.21.15.70:0/2212059170 >> v2:172.21.15.181:3300/0 conn(0x7f5714069d50 0x7f57140ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.873+0000 7f5712ffd640 1 -- 172.21.15.70:0/2212059170 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516885502 0 0) 0x7f571415d340 con 0x7f57140cdda0 2023-10-08T06:54:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.873+0000 7f5712ffd640 1 -- 172.21.15.70:0/2212059170 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56fc003520 con 0x7f57140cdda0 2023-10-08T06:54:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.873+0000 7f5718eac640 1 -- 172.21.15.70:0/2212059170 >> v2:172.21.15.181:3300/0 conn(0x7f5714069d50 msgr2=0x7f57140ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.873+0000 7f5718eac640 1 --2- 172.21.15.70:0/2212059170 >> v2:172.21.15.181:3300/0 conn(0x7f5714069d50 0x7f57140ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.873+0000 7f5718eac640 1 -- 172.21.15.70:0/2212059170 >> v1:172.21.15.181:6789/0 conn(0x7f57140cdda0 legacy=0x7f57140ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.873+0000 7f5718eac640 1 -- 172.21.15.70:0/2212059170 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571406a7b0 con 0x7f57140ce940 2023-10-08T06:54:14.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.874+0000 7f5718eac640 1 --2- 172.21.15.70:0/2212059170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 0x7f571406b290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f570c009e50 tx=0x7f570c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afe3111c096707b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:14.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.874+0000 7f5712ffd640 1 -- 172.21.15.70:0/2212059170 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c02cb00 con 0x7f57140ce940 2023-10-08T06:54:14.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.875+0000 7f5712ffd640 1 -- 172.21.15.70:0/2212059170 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c02cc60 con 0x7f57140ce940 2023-10-08T06:54:14.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.875+0000 7f5712ffd640 1 -- 172.21.15.70:0/2212059170 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c0343f0 con 0x7f57140ce940 2023-10-08T06:54:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.875+0000 7f571a90f640 1 -- 172.21.15.70:0/2212059170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 msgr2=0x7f571406b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.875+0000 7f571a90f640 1 --2- 172.21.15.70:0/2212059170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 0x7f571406b290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f570c009e50 tx=0x7f570c02bf60 comp rx=0 tx=0).stop 2023-10-08T06:54:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.875+0000 7f571a90f640 1 -- 172.21.15.70:0/2212059170 shutdown_connections 2023-10-08T06:54:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.875+0000 7f571a90f640 1 --2- 172.21.15.70:0/2212059170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 0x7f571406b290 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.875+0000 7f571a90f640 1 --2- 172.21.15.70:0/2212059170 >> v2:172.21.15.181:3300/0 conn(0x7f5714069d50 0x7f57140ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.875+0000 7f571a90f640 1 -- 172.21.15.70:0/2212059170 >> 172.21.15.70:0/2212059170 conn(0x7f57140b0aa0 msgr2=0x7f57140b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.875+0000 7f571a90f640 1 -- 172.21.15.70:0/2212059170 shutdown_connections 2023-10-08T06:54:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.875+0000 7f571a90f640 1 -- 172.21.15.70:0/2212059170 wait complete. 2023-10-08T06:54:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.877+0000 7f571a90f640 1 Processor -- start 2023-10-08T06:54:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.877+0000 7f571a90f640 1 -- start start 2023-10-08T06:54:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.878+0000 7f571a90f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714069d50 0x7f57141547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.878+0000 7f571a90f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57140cdda0 0x7f57140afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.878+0000 7f571a90f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 0x7f57140c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.878+0000 7f571a90f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57140b0390 con 0x7f57140cdda0 2023-10-08T06:54:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.878+0000 7f571a90f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57140c7d80 con 0x7f5714069d50 2023-10-08T06:54:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.878+0000 7f571a90f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57140c7170 con 0x7f57140ce940 2023-10-08T06:54:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.878+0000 7f57137fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57140cdda0 0x7f57140afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.878+0000 7f57137fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57140cdda0 0x7f57140afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57626/0 (socket says 172.21.15.70:57626) 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.878+0000 7f57137fe640 1 -- 172.21.15.70:0/705868963 learned_addr learned my addr 172.21.15.70:0/705868963 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f5718eac640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 0x7f57140c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f5713fff640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714069d50 0x7f57141547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f57137fe640 1 -- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 msgr2=0x7f57140c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f57137fe640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 0x7f57140c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f57137fe640 1 -- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714069d50 msgr2=0x7f57141547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f57137fe640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714069d50 0x7f57141547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f57137fe640 1 -- 172.21.15.70:0/705868963 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c009b30 con 0x7f57140cdda0 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f57137fe640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57140cdda0 0x7f57140afe50 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f570800e9b0 tx=0x7f570800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f5710ff9640 1 -- 172.21.15.70:0/705868963 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5708009670 con 0x7f57140cdda0 2023-10-08T06:54:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f5710ff9640 1 -- 172.21.15.70:0/705868963 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570800c9a0 con 0x7f57140cdda0 2023-10-08T06:54:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f5710ff9640 1 -- 172.21.15.70:0/705868963 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570800cd50 con 0x7f57140cdda0 2023-10-08T06:54:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.879+0000 7f571a90f640 1 -- 172.21.15.70:0/705868963 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57140c7450 con 0x7f57140cdda0 2023-10-08T06:54:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.880+0000 7f571a90f640 1 -- 172.21.15.70:0/705868963 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57140c7a10 con 0x7f57140cdda0 2023-10-08T06:54:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.881+0000 7f571a90f640 1 -- 172.21.15.70:0/705868963 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57140ca0b0 con 0x7f57140cdda0 2023-10-08T06:54:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.882+0000 7f5710ff9640 1 -- 172.21.15.70:0/705868963 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 94) v1 ==== 89332+0+0 (secure 0 0 0) 0x7f5708013470 con 0x7f57140cdda0 2023-10-08T06:54:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.882+0000 7f5710ff9640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f570006af90 0x7f570006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.883+0000 7f5710ff9640 1 -- 172.21.15.70:0/705868963 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f5708089cb0 con 0x7f57140cdda0 2023-10-08T06:54:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.883+0000 7f5713fff640 1 -- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f570006af90 msgr2=0x7f570006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1385757024 2023-10-08T06:54:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.883+0000 7f5713fff640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f570006af90 0x7f570006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:14.886 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:14.885+0000 7f82cae14040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T06:54:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.887+0000 7f5710ff9640 1 -- 172.21.15.70:0/705868963 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5708054940 con 0x7f57140cdda0 2023-10-08T06:54:14.930 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:14.929+0000 7f82c6c14040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T06:54:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.947+0000 7f5710ff9640 1 -- 172.21.15.70:0/705868963 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 95) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f5708087ec0 con 0x7f57140cdda0 2023-10-08T06:54:14.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.947+0000 7f5710ff9640 1 -- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f570006af90 msgr2=0x7f570006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:14.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:14.947+0000 7f5710ff9640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f570006af90 0x7f570006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:14.952 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:14.951+0000 7f82cae14040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T06:54:15.017 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:15.016+0000 7f824bc6f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:15.017 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:15.016+0000 7f824bc6f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:15.017 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:15.016+0000 7f824bc6f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:15.017 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:15.016+0000 7f824bc6f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:15.018 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:15.016+0000 7f824bc6f640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:15.018 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:15.016+0000 7f8248c69640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:15.018 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:15.016+0000 7f8248c69640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:15.018 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:15.016+0000 7f8248c69640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:15.018 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:15.016+0000 7f8248c69640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:15.018 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:15.016+0000 7f8248c69640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:15.075 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:15.074+0000 7f82cae14040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T06:54:15.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.088+0000 7f571a90f640 1 -- 172.21.15.70:0/705868963 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57140b1db0 con 0x7f57140cdda0 2023-10-08T06:54:15.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.088+0000 7f5710ff9640 1 -- 172.21.15.70:0/705868963 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v187) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5708087a10 con 0x7f57140cdda0 2023-10-08T06:54:15.090 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T06:54:15.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.091+0000 7f571a90f640 1 -- 172.21.15.70:0/705868963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57140cdda0 msgr2=0x7f57140afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.091+0000 7f571a90f640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57140cdda0 0x7f57140afe50 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f570800e9b0 tx=0x7f570800ee70 comp rx=0 tx=0).stop 2023-10-08T06:54:15.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.091+0000 7f571a90f640 1 -- 172.21.15.70:0/705868963 shutdown_connections 2023-10-08T06:54:15.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.091+0000 7f571a90f640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57140ce940 0x7f57140c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.091+0000 7f571a90f640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:6832/1385757024,v1:172.21.15.181:6833/1385757024] conn(0x7f570006af90 0x7f570006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.091+0000 7f571a90f640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57140cdda0 0x7f57140afe50 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.091+0000 7f571a90f640 1 --2- 172.21.15.70:0/705868963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5714069d50 0x7f57141547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.091+0000 7f571a90f640 1 -- 172.21.15.70:0/705868963 >> 172.21.15.70:0/705868963 conn(0x7f57140b0aa0 msgr2=0x7f57140c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:15.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.091+0000 7f571a90f640 1 -- 172.21.15.70:0/705868963 shutdown_connections 2023-10-08T06:54:15.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.091+0000 7f571a90f640 1 -- 172.21.15.70:0/705868963 wait complete. 2023-10-08T06:54:15.110 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T06:54:15.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.301+0000 7f4487511640 1 Processor -- start 2023-10-08T06:54:15.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.301+0000 7f4487511640 1 -- start start 2023-10-08T06:54:15.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.303+0000 7f4487511640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44801599d0 0x7f44800cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.303+0000 7f4487511640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b3c50 0x7f44800caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.303+0000 7f4487511640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44800cb8a0 con 0x7f44800cb410 2023-10-08T06:54:15.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.303+0000 7f4487511640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4480159db0 con 0x7f44801599d0 2023-10-08T06:54:15.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.303+0000 7f4487511640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4480071fe0 con 0x7f44800b3c50 2023-10-08T06:54:15.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.303+0000 7f448650f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44801599d0 0x7f44800cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:15.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.303+0000 7f448650f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44801599d0 0x7f44800cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57636/0 (socket says 172.21.15.70:57636) 2023-10-08T06:54:15.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.303+0000 7f4486d10640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f44800cb410 0x7f44800c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51308/0 (socket says 172.21.15.70:51308) 2023-10-08T06:54:15.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.303+0000 7f448650f640 1 -- 172.21.15.70:0/3023531306 learned_addr learned my addr 172.21.15.70:0/3023531306 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:15.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.303+0000 7f4485d0e640 1 --2- 172.21.15.70:0/3023531306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b3c50 0x7f44800caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:15.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.304+0000 7f448650f640 1 -- 172.21.15.70:0/3023531306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b3c50 msgr2=0x7f44800caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:15.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.304+0000 7f448650f640 1 --2- 172.21.15.70:0/3023531306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b3c50 0x7f44800caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.304+0000 7f448650f640 1 -- 172.21.15.70:0/3023531306 >> v1:172.21.15.70:6789/0 conn(0x7f44800cb410 legacy=0x7f44800c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.304+0000 7f448650f640 1 -- 172.21.15.70:0/3023531306 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44800b44c0 con 0x7f44801599d0 2023-10-08T06:54:15.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.304+0000 7f448550d640 1 -- 172.21.15.70:0/3023531306 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510137410 0 0) 0x7f44800cb8a0 con 0x7f44800cb410 2023-10-08T06:54:15.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.304+0000 7f4485d0e640 1 --2- 172.21.15.70:0/3023531306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b3c50 0x7f44800caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:15.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.305+0000 7f448650f640 1 --2- 172.21.15.70:0/3023531306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 0x7f44800cdcd0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f447c009870 tx=0x7f447c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d832dc828593562 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.305+0000 7f448550d640 1 -- 172.21.15.70:0/3023531306 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c030840 con 0x7f44801599d0 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.305+0000 7f448550d640 1 -- 172.21.15.70:0/3023531306 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c0309a0 con 0x7f44801599d0 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.305+0000 7f448550d640 1 -- 172.21.15.70:0/3023531306 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c030d30 con 0x7f44801599d0 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.305+0000 7f4487511640 1 -- 172.21.15.70:0/3023531306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 msgr2=0x7f44800cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.305+0000 7f4487511640 1 --2- 172.21.15.70:0/3023531306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 0x7f44800cdcd0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f447c009870 tx=0x7f447c02b980 comp rx=0 tx=0).stop 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.306+0000 7f4487511640 1 -- 172.21.15.70:0/3023531306 shutdown_connections 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.306+0000 7f4487511640 1 --2- 172.21.15.70:0/3023531306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800b3c50 0x7f44800caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.306+0000 7f4487511640 1 --2- 172.21.15.70:0/3023531306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 0x7f44800cdcd0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.306+0000 7f4487511640 1 -- 172.21.15.70:0/3023531306 >> 172.21.15.70:0/3023531306 conn(0x7f448006c050 msgr2=0x7f448006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.306+0000 7f4487511640 1 -- 172.21.15.70:0/3023531306 shutdown_connections 2023-10-08T06:54:15.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.306+0000 7f4487511640 1 -- 172.21.15.70:0/3023531306 wait complete. 2023-10-08T06:54:15.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.307+0000 7f4487511640 1 Processor -- start 2023-10-08T06:54:15.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.308+0000 7f4487511640 1 -- start start 2023-10-08T06:54:15.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.308+0000 7f4487511640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44800b3c50 0x7f44801e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.308+0000 7f4487511640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800cb410 0x7f44801e4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.308+0000 7f4487511640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 0x7f44801e6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.308+0000 7f4487511640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44801e6b80 con 0x7f44801599d0 2023-10-08T06:54:15.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.308+0000 7f4487511640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44801e35d0 con 0x7f44800b3c50 2023-10-08T06:54:15.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.309+0000 7f4487511640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44801e38b0 con 0x7f44800cb410 2023-10-08T06:54:15.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.309+0000 7f4486d10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 0x7f44801e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:15.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.309+0000 7f4486d10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 0x7f44801e6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57644/0 (socket says 172.21.15.70:57644) 2023-10-08T06:54:15.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.309+0000 7f4486d10640 1 -- 172.21.15.70:0/2845933498 learned_addr learned my addr 172.21.15.70:0/2845933498 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:15.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.309+0000 7f4486d10640 1 -- 172.21.15.70:0/2845933498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800cb410 msgr2=0x7f44801e4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.309+0000 7f4486d10640 1 --2- 172.21.15.70:0/2845933498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800cb410 0x7f44801e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.309+0000 7f4486d10640 1 -- 172.21.15.70:0/2845933498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44800b3c50 msgr2=0x7f44801e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.309+0000 7f4486d10640 1 --2- 172.21.15.70:0/2845933498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44800b3c50 0x7f44801e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.309+0000 7f4486d10640 1 -- 172.21.15.70:0/2845933498 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0094d0 con 0x7f44801599d0 2023-10-08T06:54:15.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.309+0000 7f4486d10640 1 --2- 172.21.15.70:0/2845933498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 0x7f44801e6640 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f447800c7a0 tx=0x7f447800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:15.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.310+0000 7f4476ffd640 1 -- 172.21.15.70:0/2845933498 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800a640 con 0x7f44801599d0 2023-10-08T06:54:15.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.310+0000 7f4476ffd640 1 -- 172.21.15.70:0/2845933498 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447800a7a0 con 0x7f44801599d0 2023-10-08T06:54:15.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.310+0000 7f4487511640 1 -- 172.21.15.70:0/2845933498 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44801e3b90 con 0x7f44801599d0 2023-10-08T06:54:15.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.310+0000 7f4476ffd640 1 -- 172.21.15.70:0/2845933498 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800abd0 con 0x7f44801599d0 2023-10-08T06:54:15.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.310+0000 7f4487511640 1 -- 172.21.15.70:0/2845933498 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44801e4150 con 0x7f44801599d0 2023-10-08T06:54:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.311+0000 7f4487511640 1 -- 172.21.15.70:0/2845933498 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44801e4370 con 0x7f44801599d0 2023-10-08T06:54:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.312+0000 7f4476ffd640 1 -- 172.21.15.70:0/2845933498 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 95) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f447800ad30 con 0x7f44801599d0 2023-10-08T06:54:15.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.313+0000 7f4476ffd640 1 -- 172.21.15.70:0/2845933498 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f4478088e00 con 0x7f44801599d0 2023-10-08T06:54:15.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.317+0000 7f4476ffd640 1 -- 172.21.15.70:0/2845933498 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4478053da0 con 0x7f44801599d0 2023-10-08T06:54:15.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.508+0000 7f4487511640 1 -- 172.21.15.70:0/2845933498 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4480074150 con 0x7f44801599d0 2023-10-08T06:54:15.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.508+0000 7f4476ffd640 1 -- 172.21.15.70:0/2845933498 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v187) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44780180a0 con 0x7f44801599d0 2023-10-08T06:54:15.510 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T06:54:15.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.511+0000 7f4487511640 1 -- 172.21.15.70:0/2845933498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 msgr2=0x7f44801e6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.511+0000 7f4487511640 1 --2- 172.21.15.70:0/2845933498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 0x7f44801e6640 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f447800c7a0 tx=0x7f447800cc60 comp rx=0 tx=0).stop 2023-10-08T06:54:15.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.511+0000 7f4487511640 1 -- 172.21.15.70:0/2845933498 shutdown_connections 2023-10-08T06:54:15.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.511+0000 7f4487511640 1 --2- 172.21.15.70:0/2845933498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44801599d0 0x7f44801e6640 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.511+0000 7f4487511640 1 --2- 172.21.15.70:0/2845933498 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44800cb410 0x7f44801e4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.511+0000 7f4487511640 1 --2- 172.21.15.70:0/2845933498 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f44800b3c50 0x7f44801e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.511+0000 7f4487511640 1 -- 172.21.15.70:0/2845933498 >> 172.21.15.70:0/2845933498 conn(0x7f448006c050 msgr2=0x7f4480159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:15.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.511+0000 7f4487511640 1 -- 172.21.15.70:0/2845933498 shutdown_connections 2023-10-08T06:54:15.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.511+0000 7f4487511640 1 -- 172.21.15.70:0/2845933498 wait complete. 2023-10-08T06:54:15.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T06:54:15.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.750+0000 7ff23e97c640 1 Processor -- start 2023-10-08T06:54:15.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.750+0000 7ff23e97c640 1 -- start start 2023-10-08T06:54:15.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.751+0000 7ff23e97c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff238070cc0 0x7ff2380caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.751+0000 7ff23e97c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff238071170 con 0x7ff2380cb450 2023-10-08T06:54:15.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.751+0000 7ff23e97c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23806e150 con 0x7ff238160660 2023-10-08T06:54:15.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.751+0000 7ff23e97c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23806de70 con 0x7ff238070cc0 2023-10-08T06:54:15.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.752+0000 7ff23cf19640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff2380cb450 0x7ff2380b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51318/0 (socket says 172.21.15.70:51318) 2023-10-08T06:54:15.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.752+0000 7ff23cf19640 1 -- 172.21.15.70:0/1548888004 learned_addr learned my addr 172.21.15.70:0/1548888004 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:15.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.752+0000 7ff22ffff640 1 --2- 172.21.15.70:0/1548888004 >> v2:172.21.15.181:3300/0 conn(0x7ff238070cc0 0x7ff2380caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:15.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.752+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115591471 0 0) 0x7ff230001490 con 0x7ff2380cb450 2023-10-08T06:54:15.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.752+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff224003520 con 0x7ff2380cb450 2023-10-08T06:54:15.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2421484725 0 0) 0x7ff224003520 con 0x7ff2380cb450 2023-10-08T06:54:15.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 >> v2:172.21.15.181:3300/0 conn(0x7ff238070cc0 msgr2=0x7ff2380caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:15.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 --2- 172.21.15.70:0/1548888004 >> v2:172.21.15.181:3300/0 conn(0x7ff238070cc0 0x7ff2380caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 >> v1:172.21.15.181:6789/0 conn(0x7ff238160660 legacy=0x7ff2380c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2380c8e30 con 0x7ff2380cb450 2023-10-08T06:54:15.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff230001b80 con 0x7ff2380cb450 2023-10-08T06:54:15.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 >> v1:172.21.15.70:6789/0 conn(0x7ff2380cb450 legacy=0x7ff2380b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 --2- 172.21.15.70:0/1548888004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2240039c0 0x7ff224003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 --2- 172.21.15.70:0/1548888004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2240047c0 0x7ff224004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 --2- 172.21.15.70:0/1548888004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff224005180 0x7ff22400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23806de70 con 0x7ff224005180 2023-10-08T06:54:15.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22400c0b0 con 0x7ff2240039c0 2023-10-08T06:54:15.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22400c270 con 0x7ff2240047c0 2023-10-08T06:54:15.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.753+0000 7ff23cf19640 1 --2- 172.21.15.70:0/1548888004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff224005180 0x7ff22400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:15.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.754+0000 7ff237fff640 1 --2- 172.21.15.70:0/1548888004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2240039c0 0x7ff224003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:15.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.754+0000 7ff23cf19640 1 -- 172.21.15.70:0/1548888004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2240047c0 msgr2=0x7ff224004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:15.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.754+0000 7ff23cf19640 1 --2- 172.21.15.70:0/1548888004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff2240047c0 0x7ff224004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.754+0000 7ff23cf19640 1 -- 172.21.15.70:0/1548888004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2240039c0 msgr2=0x7ff224003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:15.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.754+0000 7ff23cf19640 1 --2- 172.21.15.70:0/1548888004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff2240039c0 0x7ff224003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.754+0000 7ff23cf19640 1 -- 172.21.15.70:0/1548888004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224003520 con 0x7ff224005180 2023-10-08T06:54:15.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.755+0000 7ff23cf19640 1 -- 172.21.15.70:0/1548888004 reap_dead start 2023-10-08T06:54:15.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.755+0000 7ff23cf19640 1 --2- 172.21.15.70:0/1548888004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff224005180 0x7ff22400b980 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7ff23000a730 tx=0x7ff2240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ef7a52aed3e639 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:15.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.755+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300072d0 con 0x7ff224005180 2023-10-08T06:54:15.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.755+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23002f070 con 0x7ff224005180 2023-10-08T06:54:15.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.755+0000 7ff2377fe640 1 -- 172.21.15.70:0/1548888004 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300076b0 con 0x7ff224005180 2023-10-08T06:54:15.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.756+0000 7ff23e97c640 1 -- 172.21.15.70:0/1548888004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff224005180 msgr2=0x7ff22400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.756+0000 7ff23e97c640 1 --2- 172.21.15.70:0/1548888004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff224005180 0x7ff22400b980 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7ff23000a730 tx=0x7ff2240057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.756+0000 7ff23e97c640 1 -- 172.21.15.70:0/1548888004 shutdown_connections 2023-10-08T06:54:15.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.756+0000 7ff23e97c640 1 --2- 172.21.15.70:0/1548888004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff224005180 0x7ff22400b980 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.756+0000 7ff23e97c640 1 -- 172.21.15.70:0/1548888004 >> 172.21.15.70:0/1548888004 conn(0x7ff23806a5d0 msgr2=0x7ff2380b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:15.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.756+0000 7ff23e97c640 1 -- 172.21.15.70:0/1548888004 shutdown_connections 2023-10-08T06:54:15.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.756+0000 7ff23e97c640 1 -- 172.21.15.70:0/1548888004 wait complete. 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.758+0000 7ff23e97c640 1 Processor -- start 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.758+0000 7ff23e97c640 1 -- start start 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff23e97c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff224005180 0x7ff2380cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff23e97c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2380cc640 0x7ff238149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff23e97c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff238157570 0x7ff2381579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff23e97c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff238157f30 con 0x7ff2380cc640 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff23e97c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2380cd8c0 con 0x7ff224005180 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff23e97c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2380ccac0 con 0x7ff238157570 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff22ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2380cc640 0x7ff238149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff22ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2380cc640 0x7ff238149950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57656/0 (socket says 172.21.15.70:57656) 2023-10-08T06:54:15.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff22ffff640 1 -- 172.21.15.70:0/3206595926 learned_addr learned my addr 172.21.15.70:0/3206595926 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff22ffff640 1 -- 172.21.15.70:0/3206595926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff238157570 msgr2=0x7ff2381579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff22ffff640 1 --2- 172.21.15.70:0/3206595926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff238157570 0x7ff2381579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff22ffff640 1 -- 172.21.15.70:0/3206595926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff224005180 msgr2=0x7ff2380cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff237fff640 1 --2- 172.21.15.70:0/3206595926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff224005180 0x7ff2380cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff22ffff640 1 --2- 172.21.15.70:0/3206595926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff224005180 0x7ff2380cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.759+0000 7ff22ffff640 1 -- 172.21.15.70:0/3206595926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23815ca20 con 0x7ff2380cc640 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.760+0000 7ff22ffff640 1 --2- 172.21.15.70:0/3206595926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2380cc640 0x7ff238149950 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7ff22000e7d0 tx=0x7ff22000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.760+0000 7ff2357fa640 1 -- 172.21.15.70:0/3206595926 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22000eee0 con 0x7ff2380cc640 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.760+0000 7ff2357fa640 1 -- 172.21.15.70:0/3206595926 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22000c940 con 0x7ff2380cc640 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.760+0000 7ff2357fa640 1 -- 172.21.15.70:0/3206595926 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22000ccd0 con 0x7ff2380cc640 2023-10-08T06:54:15.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.760+0000 7ff23e97c640 1 -- 172.21.15.70:0/3206595926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff238149e90 con 0x7ff2380cc640 2023-10-08T06:54:15.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.760+0000 7ff23e97c640 1 -- 172.21.15.70:0/3206595926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2380cd5e0 con 0x7ff2380cc640 2023-10-08T06:54:15.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.761+0000 7ff23e97c640 1 -- 172.21.15.70:0/3206595926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23804e160 con 0x7ff2380cc640 2023-10-08T06:54:15.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.762+0000 7ff2357fa640 1 -- 172.21.15.70:0/3206595926 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 95) v1 ==== 88940+0+0 (secure 0 0 0) 0x7ff220013470 con 0x7ff2380cc640 2023-10-08T06:54:15.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.764+0000 7ff2357fa640 1 -- 172.21.15.70:0/3206595926 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6833+0+0 (secure 0 0 0) 0x7ff2200896b0 con 0x7ff2380cc640 2023-10-08T06:54:15.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.768+0000 7ff2357fa640 1 -- 172.21.15.70:0/3206595926 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff220054650 con 0x7ff2380cc640 2023-10-08T06:54:15.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.947+0000 7ff23e97c640 1 -- 172.21.15.70:0/3206595926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff23815b2c0 con 0x7ff2380cc640 2023-10-08T06:54:15.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.948+0000 7ff2357fa640 1 -- 172.21.15.70:0/3206595926 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v187) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff220057c70 con 0x7ff2380cc640 2023-10-08T06:54:15.949 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T06:54:15.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.950+0000 7ff23e97c640 1 -- 172.21.15.70:0/3206595926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2380cc640 msgr2=0x7ff238149950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:15.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.950+0000 7ff23e97c640 1 --2- 172.21.15.70:0/3206595926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2380cc640 0x7ff238149950 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7ff22000e7d0 tx=0x7ff22000ec90 comp rx=0 tx=0).stop 2023-10-08T06:54:15.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.950+0000 7ff23e97c640 1 -- 172.21.15.70:0/3206595926 shutdown_connections 2023-10-08T06:54:15.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.950+0000 7ff23e97c640 1 --2- 172.21.15.70:0/3206595926 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff238157570 0x7ff2381579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.950+0000 7ff23e97c640 1 --2- 172.21.15.70:0/3206595926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2380cc640 0x7ff238149950 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.950+0000 7ff23e97c640 1 --2- 172.21.15.70:0/3206595926 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff224005180 0x7ff2380cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:15.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.950+0000 7ff23e97c640 1 -- 172.21.15.70:0/3206595926 >> 172.21.15.70:0/3206595926 conn(0x7ff23806a5d0 msgr2=0x7ff2380c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:15.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.951+0000 7ff23e97c640 1 -- 172.21.15.70:0/3206595926 shutdown_connections 2023-10-08T06:54:15.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:15.951+0000 7ff23e97c640 1 -- 172.21.15.70:0/3206595926 wait complete. 2023-10-08T06:54:15.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:54:16.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.170+0000 7ff146875640 1 Processor -- start 2023-10-08T06:54:16.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.170+0000 7ff146875640 1 -- start start 2023-10-08T06:54:16.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.172+0000 7ff146875640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff140159ac0 0x7ff14006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.172+0000 7ff146875640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1400ad790 0x7ff1400702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.172+0000 7ff146875640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff14006dee0 con 0x7ff14015e330 2023-10-08T06:54:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.172+0000 7ff146875640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff14015d850 con 0x7ff1400ad790 2023-10-08T06:54:16.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.172+0000 7ff146875640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14015db30 con 0x7ff140159ac0 2023-10-08T06:54:16.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.172+0000 7ff13f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1400ad790 0x7ff1400702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.172+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff140159ac0 0x7ff14006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.173+0000 7ff13ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff140159ac0 0x7ff14006d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:37854/0 (socket says 172.21.15.70:37854) 2023-10-08T06:54:16.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.173+0000 7ff13ffff640 1 -- 172.21.15.70:0/3855709268 learned_addr learned my addr 172.21.15.70:0/3855709268 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:16.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.173+0000 7ff13f7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1400ad790 0x7ff1400702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:34584/0 (socket says 172.21.15.70:34584) 2023-10-08T06:54:16.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.173+0000 7ff144e12640 1 --1- 172.21.15.70:0/3855709268 >> v1:172.21.15.181:6789/0 conn(0x7ff14015e330 0x7ff14006c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:33748/0 (socket says 172.21.15.70:33748) 2023-10-08T06:54:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.173+0000 7ff13ffff640 1 -- 172.21.15.70:0/3855709268 >> v2:172.21.15.181:3300/0 conn(0x7ff1400ad790 msgr2=0x7ff1400702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.173+0000 7ff13ffff640 1 --2- 172.21.15.70:0/3855709268 >> v2:172.21.15.181:3300/0 conn(0x7ff1400ad790 0x7ff1400702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.173+0000 7ff13ffff640 1 -- 172.21.15.70:0/3855709268 >> v1:172.21.15.181:6789/0 conn(0x7ff14015e330 legacy=0x7ff14006c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.173+0000 7ff13ffff640 1 -- 172.21.15.70:0/3855709268 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14015de10 con 0x7ff140159ac0 2023-10-08T06:54:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.175+0000 7ff13ffff640 1 --2- 172.21.15.70:0/3855709268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff140159ac0 0x7ff14006d5e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff130009870 tx=0x7ff13002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f99ee8b6791bf7c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.175+0000 7ff13effd640 1 -- 172.21.15.70:0/3855709268 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130030840 con 0x7ff140159ac0 2023-10-08T06:54:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.175+0000 7ff13effd640 1 -- 172.21.15.70:0/3855709268 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1300309a0 con 0x7ff140159ac0 2023-10-08T06:54:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.175+0000 7ff13effd640 1 -- 172.21.15.70:0/3855709268 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130030d30 con 0x7ff140159ac0 2023-10-08T06:54:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.176+0000 7ff146875640 1 -- 172.21.15.70:0/3855709268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff140159ac0 msgr2=0x7ff14006d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.176+0000 7ff146875640 1 --2- 172.21.15.70:0/3855709268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff140159ac0 0x7ff14006d5e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff130009870 tx=0x7ff13002b980 comp rx=0 tx=0).stop 2023-10-08T06:54:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.176+0000 7ff146875640 1 -- 172.21.15.70:0/3855709268 shutdown_connections 2023-10-08T06:54:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.176+0000 7ff146875640 1 --2- 172.21.15.70:0/3855709268 >> v2:172.21.15.181:3300/0 conn(0x7ff1400ad790 0x7ff1400702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.176+0000 7ff146875640 1 --2- 172.21.15.70:0/3855709268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff140159ac0 0x7ff14006d5e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.176+0000 7ff146875640 1 -- 172.21.15.70:0/3855709268 >> 172.21.15.70:0/3855709268 conn(0x7ff14006a380 msgr2=0x7ff1400b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.176+0000 7ff146875640 1 -- 172.21.15.70:0/3855709268 shutdown_connections 2023-10-08T06:54:16.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.176+0000 7ff146875640 1 -- 172.21.15.70:0/3855709268 wait complete. 2023-10-08T06:54:16.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.178+0000 7ff146875640 1 Processor -- start 2023-10-08T06:54:16.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.178+0000 7ff146875640 1 -- start start 2023-10-08T06:54:16.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.179+0000 7ff146875640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1400ad790 0x7ff140158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.179+0000 7ff146875640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff140159ac0 0x7ff1400b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.179+0000 7ff146875640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff14015e330 0x7ff14013e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.179+0000 7ff146875640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff140157510 con 0x7ff140159ac0 2023-10-08T06:54:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.179+0000 7ff146875640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1400b45e0 con 0x7ff1400ad790 2023-10-08T06:54:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.179+0000 7ff146875640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1400b3a80 con 0x7ff14015e330 2023-10-08T06:54:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.179+0000 7ff13f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff140159ac0 0x7ff1400b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.179+0000 7ff13f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff140159ac0 0x7ff1400b3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57672/0 (socket says 172.21.15.70:57672) 2023-10-08T06:54:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.179+0000 7ff13f7fe640 1 -- 172.21.15.70:0/3634987903 learned_addr learned my addr 172.21.15.70:0/3634987903 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff13ffff640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1400ad790 0x7ff140158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff13f7fe640 1 -- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff14015e330 msgr2=0x7ff14013e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff13f7fe640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff14015e330 0x7ff14013e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff13f7fe640 1 -- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1400ad790 msgr2=0x7ff140158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff13f7fe640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1400ad790 0x7ff140158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff13f7fe640 1 -- 172.21.15.70:0/3634987903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1300094d0 con 0x7ff140159ac0 2023-10-08T06:54:16.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff13f7fe640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff140159ac0 0x7ff1400b3540 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7ff12800efc0 tx=0x7ff12800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:16.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff13cff9640 1 -- 172.21.15.70:0/3634987903 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128009be0 con 0x7ff140159ac0 2023-10-08T06:54:16.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff13cff9640 1 -- 172.21.15.70:0/3634987903 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12800f040 con 0x7ff140159ac0 2023-10-08T06:54:16.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff13cff9640 1 -- 172.21.15.70:0/3634987903 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128014680 con 0x7ff140159ac0 2023-10-08T06:54:16.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff146875640 1 -- 172.21.15.70:0/3634987903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1400b3d40 con 0x7ff140159ac0 2023-10-08T06:54:16.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.180+0000 7ff146875640 1 -- 172.21.15.70:0/3634987903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1400b4300 con 0x7ff140159ac0 2023-10-08T06:54:16.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.181+0000 7ff146875640 1 -- 172.21.15.70:0/3634987903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff140157680 con 0x7ff140159ac0 2023-10-08T06:54:16.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.183+0000 7ff13cff9640 1 -- 172.21.15.70:0/3634987903 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 96) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff1280147e0 con 0x7ff140159ac0 2023-10-08T06:54:16.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.183+0000 7ff13cff9640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff12406acc0 0x7ff12406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.184+0000 7ff13cff9640 1 -- 172.21.15.70:0/3634987903 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6833+0+0 (secure 0 0 0) 0x7ff12808a660 con 0x7ff140159ac0 2023-10-08T06:54:16.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.184+0000 7ff13ffff640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff12406acc0 0x7ff12406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.185+0000 7ff13ffff640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff12406acc0 0x7ff12406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff130036000 tx=0x7ff13003b040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:16.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.188+0000 7ff13cff9640 1 -- 172.21.15.70:0/3634987903 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff128055510 con 0x7ff140159ac0 2023-10-08T06:54:16.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.412+0000 7ff146875640 1 -- 172.21.15.70:0/3634987903 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff140067490 con 0x7ff140159ac0 2023-10-08T06:54:16.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.413+0000 7ff13cff9640 1 -- 172.21.15.70:0/3634987903 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4917 (secure 0 0 0) 0x7ff128058b30 con 0x7ff140159ac0 2023-10-08T06:54:16.415 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:16.415 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/542411062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":542411062},{"type":"v1","addr":"172.21.15.181:6836","nonce":542411062}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/646194010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":646194010},{"type":"v1","addr":"172.21.15.70:6835","nonce":646194010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1850202608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1850202608},{"type":"v1","addr":"172.21.15.181:6837","nonce":1850202608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:53:47.285813+0000","modified":"2023-10-08T06:53:50.356439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9591":{"gid":9591,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/3790014600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3790014600},{"type":"v1","addr":"172.21.15.70:6834","nonce":3790014600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-08T06:54:16.416 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 83 2023-10-08T06:54:16.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.415+0000 7ff146875640 1 -- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff12406acc0 msgr2=0x7ff12406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.415+0000 7ff146875640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff12406acc0 0x7ff12406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff130036000 tx=0x7ff13003b040 comp rx=0 tx=0).stop 2023-10-08T06:54:16.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.415+0000 7ff146875640 1 -- 172.21.15.70:0/3634987903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff140159ac0 msgr2=0x7ff1400b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.415+0000 7ff146875640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff140159ac0 0x7ff1400b3540 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7ff12800efc0 tx=0x7ff12800c490 comp rx=0 tx=0).stop 2023-10-08T06:54:16.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.416+0000 7ff146875640 1 -- 172.21.15.70:0/3634987903 shutdown_connections 2023-10-08T06:54:16.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.416+0000 7ff146875640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff12406acc0 0x7ff12406d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.416+0000 7ff146875640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff14015e330 0x7ff14013e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.416+0000 7ff146875640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff140159ac0 0x7ff1400b3540 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.416+0000 7ff146875640 1 --2- 172.21.15.70:0/3634987903 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1400ad790 0x7ff140158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.416+0000 7ff146875640 1 -- 172.21.15.70:0/3634987903 >> 172.21.15.70:0/3634987903 conn(0x7ff14006a380 msgr2=0x7ff14015c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.416+0000 7ff146875640 1 -- 172.21.15.70:0/3634987903 shutdown_connections 2023-10-08T06:54:16.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.416+0000 7ff146875640 1 -- 172.21.15.70:0/3634987903 wait complete. 2023-10-08T06:54:16.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:54:16.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.654+0000 7f6efd4d7640 1 Processor -- start 2023-10-08T06:54:16.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.654+0000 7f6efd4d7640 1 -- start start 2023-10-08T06:54:16.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.655+0000 7f6efd4d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ef815ab40 0x7f6ef80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.655+0000 7f6efd4d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6ef80c90b0 0x7f6ef80adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.655+0000 7f6efd4d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef815b2b0 con 0x7f6ef80cae70 2023-10-08T06:54:16.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.655+0000 7f6efd4d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef815bf30 con 0x7f6ef80c90b0 2023-10-08T06:54:16.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.655+0000 7f6efd4d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef815f7a0 con 0x7f6ef815ab40 2023-10-08T06:54:16.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.656+0000 7f6eeffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6ef80c90b0 0x7f6ef80adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.656+0000 7f6eeffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6ef80c90b0 0x7f6ef80adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57678/0 (socket says 172.21.15.70:57678) 2023-10-08T06:54:16.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.656+0000 7f6eeffff640 1 -- 172.21.15.70:0/1866587631 learned_addr learned my addr 172.21.15.70:0/1866587631 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:16.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.656+0000 7f6ef6ffd640 1 --2- 172.21.15.70:0/1866587631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ef815ab40 0x7f6ef80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.656+0000 7f6eeffff640 1 -- 172.21.15.70:0/1866587631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ef815ab40 msgr2=0x7f6ef80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:16.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.656+0000 7f6eeffff640 1 --2- 172.21.15.70:0/1866587631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ef815ab40 0x7f6ef80ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.656+0000 7f6eeffff640 1 -- 172.21.15.70:0/1866587631 >> v1:172.21.15.181:6789/0 conn(0x7f6ef80cae70 legacy=0x7f6ef80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:16.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.656+0000 7f6eeffff640 1 -- 172.21.15.70:0/1866587631 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef815b590 con 0x7f6ef80c90b0 2023-10-08T06:54:16.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.657+0000 7f6eeffff640 1 --2- 172.21.15.70:0/1866587631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 0x7f6ef80adc60 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009870 tx=0x7f6ee002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af73bbecf0efd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:16.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6ef67fc640 1 -- 172.21.15.70:0/1866587631 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0031640 con 0x7f6ef80c90b0 2023-10-08T06:54:16.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6ef67fc640 1 -- 172.21.15.70:0/1866587631 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee00317a0 con 0x7f6ef80c90b0 2023-10-08T06:54:16.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6ef67fc640 1 -- 172.21.15.70:0/1866587631 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0031b30 con 0x7f6ef80c90b0 2023-10-08T06:54:16.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6efd4d7640 1 -- 172.21.15.70:0/1866587631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 msgr2=0x7f6ef80adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6efd4d7640 1 --2- 172.21.15.70:0/1866587631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 0x7f6ef80adc60 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009870 tx=0x7f6ee002b8f0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6efd4d7640 1 -- 172.21.15.70:0/1866587631 shutdown_connections 2023-10-08T06:54:16.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6efd4d7640 1 --2- 172.21.15.70:0/1866587631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 0x7f6ef80adc60 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6efd4d7640 1 --2- 172.21.15.70:0/1866587631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ef815ab40 0x7f6ef80ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6efd4d7640 1 -- 172.21.15.70:0/1866587631 >> 172.21.15.70:0/1866587631 conn(0x7f6ef806a380 msgr2=0x7f6ef80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:16.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6efd4d7640 1 -- 172.21.15.70:0/1866587631 shutdown_connections 2023-10-08T06:54:16.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.658+0000 7f6efd4d7640 1 -- 172.21.15.70:0/1866587631 wait complete. 2023-10-08T06:54:16.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.660+0000 7f6efd4d7640 1 Processor -- start 2023-10-08T06:54:16.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.660+0000 7f6efd4d7640 1 -- start start 2023-10-08T06:54:16.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.661+0000 7f6efd4d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 0x7f6ef80c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.661+0000 7f6efd4d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ef80cae70 0x7f6ef80c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.661+0000 7f6efd4d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ef815ab40 0x7f6ef81599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.661+0000 7f6efd4d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef81595a0 con 0x7f6ef80c90b0 2023-10-08T06:54:16.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.661+0000 7f6efd4d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef8158640 con 0x7f6ef815ab40 2023-10-08T06:54:16.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.661+0000 7f6efd4d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef8158920 con 0x7f6ef80cae70 2023-10-08T06:54:16.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.661+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 0x7f6ef80c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.662+0000 7f6ef6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 0x7f6ef80c4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57694/0 (socket says 172.21.15.70:57694) 2023-10-08T06:54:16.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.662+0000 7f6ef6ffd640 1 -- 172.21.15.70:0/3374584739 learned_addr learned my addr 172.21.15.70:0/3374584739 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:16.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.662+0000 7f6ef77fe640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ef815ab40 0x7f6ef81599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.662+0000 7f6ef6ffd640 1 -- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ef80cae70 msgr2=0x7f6ef80c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.662+0000 7f6eeffff640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ef80cae70 0x7f6ef80c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.662+0000 7f6ef6ffd640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ef80cae70 0x7f6ef80c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.662+0000 7f6ef6ffd640 1 -- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ef815ab40 msgr2=0x7f6ef81599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.662+0000 7f6ef6ffd640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ef815ab40 0x7f6ef81599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.662+0000 7f6ef6ffd640 1 -- 172.21.15.70:0/3374584739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee00094d0 con 0x7f6ef80c90b0 2023-10-08T06:54:16.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.662+0000 7f6ef6ffd640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 0x7f6ef80c4860 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c7f0 tx=0x7f6ee800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:16.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.663+0000 7f6eef7fe640 1 -- 172.21.15.70:0/3374584739 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8004220 con 0x7f6ef80c90b0 2023-10-08T06:54:16.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.663+0000 7f6eef7fe640 1 -- 172.21.15.70:0/3374584739 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee80043a0 con 0x7f6ef80c90b0 2023-10-08T06:54:16.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.663+0000 7f6efd4d7640 1 -- 172.21.15.70:0/3374584739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef8158c00 con 0x7f6ef80c90b0 2023-10-08T06:54:16.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.663+0000 7f6eef7fe640 1 -- 172.21.15.70:0/3374584739 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8007860 con 0x7f6ef80c90b0 2023-10-08T06:54:16.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.663+0000 7f6efd4d7640 1 -- 172.21.15.70:0/3374584739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef81591c0 con 0x7f6ef80c90b0 2023-10-08T06:54:16.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.664+0000 7f6efd4d7640 1 -- 172.21.15.70:0/3374584739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef81596e0 con 0x7f6ef80c90b0 2023-10-08T06:54:16.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.665+0000 7f6eef7fe640 1 -- 172.21.15.70:0/3374584739 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 96) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f6ee8007bd0 con 0x7f6ef80c90b0 2023-10-08T06:54:16.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.666+0000 7f6eef7fe640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f6ed806acc0 0x7f6ed806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:16.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.666+0000 7f6eef7fe640 1 -- 172.21.15.70:0/3374584739 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f6ee80897a0 con 0x7f6ef80c90b0 2023-10-08T06:54:16.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.667+0000 7f6eeffff640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f6ed806acc0 0x7f6ed806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:16.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.668+0000 7f6eeffff640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f6ed806acc0 0x7f6ed806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0036000 tx=0x7f6ee003c040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:16.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.671+0000 7f6eef7fe640 1 -- 172.21.15.70:0/3374584739 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ee8054650 con 0x7f6ef80c90b0 2023-10-08T06:54:16.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.861+0000 7f6efd4d7640 1 -- 172.21.15.70:0/3374584739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ef8067490 con 0x7f6ef80c90b0 2023-10-08T06:54:16.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.865+0000 7f6eef7fe640 1 -- 172.21.15.70:0/3374584739 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+17916 (secure 0 0 0) 0x7f6ee801e090 con 0x7f6ef80c90b0 2023-10-08T06:54:16.867 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:16.867 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":187,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:54:14.934496+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:53:44.876671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-08T06:53:46.310621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2596125885":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/1985247114":"2023-10-09T06:54:14.934383+0000","172.21.15.181:6833/1385757024":"2023-10-09T06:54:14.934383+0000","172.21.15.181:6832/1385757024":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/2383107395":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/1278239547":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/3845072443":"2023-10-09T06:54:14.934383+0000","172.21.15.181:6833/4245513204":"2023-10-09T06:54:07.074878+0000","172.21.15.181:6832/4245513204":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/1356319692":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/4081249518":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2711042436":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2536889452":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2861165849":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/998820505":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/4176429568":"2023-10-09T06:54:07.074878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:54:16.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.867+0000 7f6efd4d7640 1 -- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f6ed806acc0 msgr2=0x7f6ed806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.867+0000 7f6efd4d7640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f6ed806acc0 0x7f6ed806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0036000 tx=0x7f6ee003c040 comp rx=0 tx=0).stop 2023-10-08T06:54:16.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.867+0000 7f6efd4d7640 1 -- 172.21.15.70:0/3374584739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 msgr2=0x7f6ef80c4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:16.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.867+0000 7f6efd4d7640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 0x7f6ef80c4860 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c7f0 tx=0x7f6ee800ccb0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.867+0000 7f6efd4d7640 1 -- 172.21.15.70:0/3374584739 shutdown_connections 2023-10-08T06:54:16.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.867+0000 7f6efd4d7640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f6ed806acc0 0x7f6ed806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.867+0000 7f6efd4d7640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ef815ab40 0x7f6ef81599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.867+0000 7f6efd4d7640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ef80cae70 0x7f6ef80c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.867+0000 7f6efd4d7640 1 --2- 172.21.15.70:0/3374584739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ef80c90b0 0x7f6ef80c4860 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:16.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.867+0000 7f6efd4d7640 1 -- 172.21.15.70:0/3374584739 >> 172.21.15.70:0/3374584739 conn(0x7f6ef806a380 msgr2=0x7f6ef80701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:16.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.868+0000 7f6efd4d7640 1 -- 172.21.15.70:0/3374584739 shutdown_connections 2023-10-08T06:54:16.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:16.868+0000 7f6efd4d7640 1 -- 172.21.15.70:0/3374584739 wait complete. 2023-10-08T06:54:16.888 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T06:54:16.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:54:17.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.094+0000 7f5b35be2640 1 Processor -- start 2023-10-08T06:54:17.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.095+0000 7f5b35be2640 1 -- start start 2023-10-08T06:54:17.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.096+0000 7f5b35be2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5b3006a3a0 0x7f5b30071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.096+0000 7f5b35be2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3015fdb0 0x7f5b30160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.096+0000 7f5b35be2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b30072480 con 0x7f5b30069fc0 2023-10-08T06:54:17.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.096+0000 7f5b35be2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b300ce280 con 0x7f5b3006a3a0 2023-10-08T06:54:17.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.096+0000 7f5b35be2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300ce560 con 0x7f5b3015fdb0 2023-10-08T06:54:17.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.097+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3015fdb0 0x7f5b30160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.097+0000 7f5b2f7fe640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f5b30069fc0 0x7f5b30161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:33760/0 (socket says 172.21.15.70:33760) 2023-10-08T06:54:17.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.097+0000 7f5b2f7fe640 1 -- 172.21.15.70:0/79042413 learned_addr learned my addr 172.21.15.70:0/79042413 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:17.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.097+0000 7f5b27fff640 1 --2- 172.21.15.70:0/79042413 >> v2:172.21.15.181:3300/0 conn(0x7f5b3006a3a0 0x7f5b30071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.098+0000 7f5b2effd640 1 -- 172.21.15.70:0/79042413 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8356061 0 0) 0x7f5b30072480 con 0x7f5b30069fc0 2023-10-08T06:54:17.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.098+0000 7f5b2effd640 1 -- 172.21.15.70:0/79042413 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b1c003520 con 0x7f5b30069fc0 2023-10-08T06:54:17.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.098+0000 7f5b27fff640 1 -- 172.21.15.70:0/79042413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3015fdb0 msgr2=0x7f5b30160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:17.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.098+0000 7f5b27fff640 1 --2- 172.21.15.70:0/79042413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3015fdb0 0x7f5b30160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.098+0000 7f5b27fff640 1 -- 172.21.15.70:0/79042413 >> v1:172.21.15.181:6789/0 conn(0x7f5b30069fc0 legacy=0x7f5b30161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.098+0000 7f5b27fff640 1 -- 172.21.15.70:0/79042413 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b30071480 con 0x7f5b3006a3a0 2023-10-08T06:54:17.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.098+0000 7f5b2ffff640 1 --2- 172.21.15.70:0/79042413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3015fdb0 0x7f5b30160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T06:54:17.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.099+0000 7f5b27fff640 1 --2- 172.21.15.70:0/79042413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3006a3a0 0x7f5b30071f40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5b180097a0 tx=0x7f5b1802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be218d3720f908c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:17.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.100+0000 7f5b2effd640 1 -- 172.21.15.70:0/79042413 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18030800 con 0x7f5b3006a3a0 2023-10-08T06:54:17.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.100+0000 7f5b2effd640 1 -- 172.21.15.70:0/79042413 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b18030960 con 0x7f5b3006a3a0 2023-10-08T06:54:17.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.100+0000 7f5b2effd640 1 -- 172.21.15.70:0/79042413 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18030cd0 con 0x7f5b3006a3a0 2023-10-08T06:54:17.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.100+0000 7f5b35be2640 1 -- 172.21.15.70:0/79042413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3006a3a0 msgr2=0x7f5b30071f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.100+0000 7f5b35be2640 1 --2- 172.21.15.70:0/79042413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3006a3a0 0x7f5b30071f40 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5b180097a0 tx=0x7f5b1802b940 comp rx=0 tx=0).stop 2023-10-08T06:54:17.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.100+0000 7f5b35be2640 1 -- 172.21.15.70:0/79042413 shutdown_connections 2023-10-08T06:54:17.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.100+0000 7f5b35be2640 1 --2- 172.21.15.70:0/79042413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b3015fdb0 0x7f5b30160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.100+0000 7f5b35be2640 1 --2- 172.21.15.70:0/79042413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3006a3a0 0x7f5b30071f40 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.101+0000 7f5b35be2640 1 -- 172.21.15.70:0/79042413 >> 172.21.15.70:0/79042413 conn(0x7f5b3006d450 msgr2=0x7f5b3006d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:17.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.101+0000 7f5b35be2640 1 -- 172.21.15.70:0/79042413 shutdown_connections 2023-10-08T06:54:17.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.101+0000 7f5b35be2640 1 -- 172.21.15.70:0/79042413 wait complete. 2023-10-08T06:54:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.102+0000 7f5b35be2640 1 Processor -- start 2023-10-08T06:54:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.103+0000 7f5b35be2640 1 -- start start 2023-10-08T06:54:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.103+0000 7f5b35be2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b30069fc0 0x7f5b30154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.103+0000 7f5b35be2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3006a3a0 0x7f5b300cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b35be2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b3015fdb0 0x7f5b300b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b35be2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300b2950 con 0x7f5b3015fdb0 2023-10-08T06:54:17.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b35be2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300cb4f0 con 0x7f5b3006a3a0 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b35be2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300cb7d0 con 0x7f5b30069fc0 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b3015fdb0 0x7f5b300b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b3015fdb0 0x7f5b300b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57706/0 (socket says 172.21.15.70:57706) 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b2ffff640 1 -- 172.21.15.70:0/303847439 learned_addr learned my addr 172.21.15.70:0/303847439 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b2f7fe640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b30069fc0 0x7f5b30154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b27fff640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3006a3a0 0x7f5b300cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b2ffff640 1 -- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b30069fc0 msgr2=0x7f5b30154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b2ffff640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b30069fc0 0x7f5b30154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b2ffff640 1 -- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3006a3a0 msgr2=0x7f5b300cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b2ffff640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3006a3a0 0x7f5b300cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b2ffff640 1 -- 172.21.15.70:0/303847439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28009a70 con 0x7f5b3015fdb0 2023-10-08T06:54:17.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b2f7fe640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b30069fc0 0x7f5b30154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.104+0000 7f5b27fff640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3006a3a0 0x7f5b300cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.105+0000 7f5b2ffff640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b3015fdb0 0x7f5b300b2410 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800efc0 tx=0x7f5b2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.105+0000 7f5b2cff9640 1 -- 172.21.15.70:0/303847439 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b28009cd0 con 0x7f5b3015fdb0 2023-10-08T06:54:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.105+0000 7f5b2cff9640 1 -- 172.21.15.70:0/303847439 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2800f040 con 0x7f5b3015fdb0 2023-10-08T06:54:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.105+0000 7f5b2cff9640 1 -- 172.21.15.70:0/303847439 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b28003990 con 0x7f5b3015fdb0 2023-10-08T06:54:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.105+0000 7f5b35be2640 1 -- 172.21.15.70:0/303847439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b18009400 con 0x7f5b3015fdb0 2023-10-08T06:54:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.105+0000 7f5b35be2640 1 -- 172.21.15.70:0/303847439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b300c6eb0 con 0x7f5b3015fdb0 2023-10-08T06:54:17.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.106+0000 7f5b35be2640 1 -- 172.21.15.70:0/303847439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b30066d80 con 0x7f5b3015fdb0 2023-10-08T06:54:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.107+0000 7f5b2cff9640 1 -- 172.21.15.70:0/303847439 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 96) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f5b28014480 con 0x7f5b3015fdb0 2023-10-08T06:54:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.108+0000 7f5b2cff9640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f5b1006acc0 0x7f5b1006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.108+0000 7f5b2cff9640 1 -- 172.21.15.70:0/303847439 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f5b2808a3e0 con 0x7f5b3015fdb0 2023-10-08T06:54:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.109+0000 7f5b2f7fe640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f5b1006acc0 0x7f5b1006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.109+0000 7f5b2f7fe640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f5b1006acc0 0x7f5b1006d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5b200058d0 tx=0x7f5b20009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:17.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.112+0000 7f5b2cff9640 1 -- 172.21.15.70:0/303847439 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5b28055210 con 0x7f5b3015fdb0 2023-10-08T06:54:17.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.338+0000 7f5b35be2640 1 -- 172.21.15.70:0/303847439 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b300ccb70 con 0x7f5b3015fdb0 2023-10-08T06:54:17.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.339+0000 7f5b2cff9640 1 -- 172.21.15.70:0/303847439 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4917 (secure 0 0 0) 0x7f5b28058830 con 0x7f5b3015fdb0 2023-10-08T06:54:17.341 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:17.341 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/542411062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":542411062},{"type":"v1","addr":"172.21.15.181:6836","nonce":542411062}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/646194010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":646194010},{"type":"v1","addr":"172.21.15.70:6835","nonce":646194010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1850202608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1850202608},{"type":"v1","addr":"172.21.15.181:6837","nonce":1850202608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:53:47.285813+0000","modified":"2023-10-08T06:53:50.356439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9591":{"gid":9591,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/3790014600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3790014600},{"type":"v1","addr":"172.21.15.70:6834","nonce":3790014600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-08T06:54:17.341 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 83 2023-10-08T06:54:17.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.341+0000 7f5b35be2640 1 -- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f5b1006acc0 msgr2=0x7f5b1006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.341+0000 7f5b35be2640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f5b1006acc0 0x7f5b1006d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5b200058d0 tx=0x7f5b20009040 comp rx=0 tx=0).stop 2023-10-08T06:54:17.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.341+0000 7f5b35be2640 1 -- 172.21.15.70:0/303847439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b3015fdb0 msgr2=0x7f5b300b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.341+0000 7f5b35be2640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b3015fdb0 0x7f5b300b2410 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800efc0 tx=0x7f5b2800c490 comp rx=0 tx=0).stop 2023-10-08T06:54:17.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.342+0000 7f5b35be2640 1 -- 172.21.15.70:0/303847439 shutdown_connections 2023-10-08T06:54:17.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.342+0000 7f5b35be2640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f5b1006acc0 0x7f5b1006d180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.342+0000 7f5b35be2640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b3015fdb0 0x7f5b300b2410 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.342+0000 7f5b35be2640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5b3006a3a0 0x7f5b300cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.342+0000 7f5b35be2640 1 --2- 172.21.15.70:0/303847439 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5b30069fc0 0x7f5b30154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.342+0000 7f5b35be2640 1 -- 172.21.15.70:0/303847439 >> 172.21.15.70:0/303847439 conn(0x7f5b3006d450 msgr2=0x7f5b30071e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:17.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.342+0000 7f5b35be2640 1 -- 172.21.15.70:0/303847439 shutdown_connections 2023-10-08T06:54:17.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.342+0000 7f5b35be2640 1 -- 172.21.15.70:0/303847439 wait complete. 2023-10-08T06:54:17.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:54:17.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.566+0000 7fa3983b0640 1 Processor -- start 2023-10-08T06:54:17.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.566+0000 7fa3983b0640 1 -- start start 2023-10-08T06:54:17.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.568+0000 7fa3983b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3900c90b0 0x7fa3900cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.568+0000 7fa3983b0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39015b2b0 con 0x7fa39015ab40 2023-10-08T06:54:17.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.568+0000 7fa3983b0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39015b590 con 0x7fa3900cb3b0 2023-10-08T06:54:17.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.568+0000 7fa3983b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39015bf30 con 0x7fa3900c90b0 2023-10-08T06:54:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.568+0000 7fa39614c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa39015ab40 0x7fa3900ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51332/0 (socket says 172.21.15.70:51332) 2023-10-08T06:54:17.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.568+0000 7fa39614c640 1 -- 172.21.15.70:0/251283125 learned_addr learned my addr 172.21.15.70:0/251283125 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:17.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.568+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678703642 0 0) 0x7fa39015b2b0 con 0x7fa39015ab40 2023-10-08T06:54:17.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.568+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa37c003520 con 0x7fa39015ab40 2023-10-08T06:54:17.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.568+0000 7fa39594b640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3900c90b0 0x7fa3900cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2464839095 0 0) 0x7fa37c003520 con 0x7fa39015ab40 2023-10-08T06:54:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3900c90b0 msgr2=0x7fa3900cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3900c90b0 0x7fa3900cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 >> v1:172.21.15.181:6789/0 conn(0x7fa3900cb3b0 legacy=0x7fa39015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39015b870 con 0x7fa39015ab40 2023-10-08T06:54:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa38c001670 con 0x7fa39015ab40 2023-10-08T06:54:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 >> v1:172.21.15.70:6789/0 conn(0x7fa39015ab40 legacy=0x7fa3900ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa37c0039a0 0x7fa37c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa37c0047c0 0x7fa37c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3900c90b0 0x7fa37c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39015bf30 con 0x7fa37c0039a0 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c00bab0 con 0x7fa37c0047c0 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37c00bc70 con 0x7fa3900c90b0 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.569+0000 7fa39614c640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa37c0039a0 0x7fa37c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.570+0000 7fa39694d640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3900c90b0 0x7fa37c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.570+0000 7fa39594b640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa37c0047c0 0x7fa37c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.570+0000 7fa39614c640 1 -- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3900c90b0 msgr2=0x7fa37c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.570+0000 7fa39614c640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3900c90b0 0x7fa37c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.570+0000 7fa39614c640 1 -- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa37c0047c0 msgr2=0x7fa37c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:17.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.570+0000 7fa39614c640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa37c0047c0 0x7fa37c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.570+0000 7fa39614c640 1 -- 172.21.15.70:0/251283125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c003520 con 0x7fa37c0039a0 2023-10-08T06:54:17.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.570+0000 7fa39694d640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3900c90b0 0x7fa37c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:17.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.571+0000 7fa39614c640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa37c0039a0 0x7fa37c003dc0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00c190 tx=0x7fa37c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ded816ccfe2e2eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:17.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.571+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c005ad0 con 0x7fa37c0039a0 2023-10-08T06:54:17.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.571+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c005c50 con 0x7fa37c0039a0 2023-10-08T06:54:17.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.571+0000 7fa39514a640 1 -- 172.21.15.70:0/251283125 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c02c8a0 con 0x7fa37c0039a0 2023-10-08T06:54:17.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.572+0000 7fa3983b0640 1 -- 172.21.15.70:0/251283125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa37c0039a0 msgr2=0x7fa37c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.572+0000 7fa3983b0640 1 --2- 172.21.15.70:0/251283125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa37c0039a0 0x7fa37c003dc0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00c190 tx=0x7fa37c0057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.572+0000 7fa39694d640 1 -- 172.21.15.70:0/251283125 reap_dead start 2023-10-08T06:54:17.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.572+0000 7fa3983b0640 1 -- 172.21.15.70:0/251283125 shutdown_connections 2023-10-08T06:54:17.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.572+0000 7fa3983b0640 1 -- 172.21.15.70:0/251283125 >> 172.21.15.70:0/251283125 conn(0x7fa39006a380 msgr2=0x7fa3900b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:17.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.572+0000 7fa3983b0640 1 -- 172.21.15.70:0/251283125 shutdown_connections 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.572+0000 7fa3983b0640 1 -- 172.21.15.70:0/251283125 wait complete. 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.573+0000 7fa3983b0640 1 Processor -- start 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.574+0000 7fa3983b0640 1 -- start start 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa3983b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa37c0039a0 0x7fa39013f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa3983b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa390140ed0 0x7fa39013ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa3983b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa390140450 0x7fa39013e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa3983b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900b3ce0 con 0x7fa390140ed0 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa3983b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900b3060 con 0x7fa37c0039a0 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa3983b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900b3340 con 0x7fa390140450 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa39594b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa390140ed0 0x7fa39013ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa39594b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa390140ed0 0x7fa39013ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57724/0 (socket says 172.21.15.70:57724) 2023-10-08T06:54:17.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa39594b640 1 -- 172.21.15.70:0/3876058998 learned_addr learned my addr 172.21.15.70:0/3876058998 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa39594b640 1 -- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa390140450 msgr2=0x7fa39013e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa39594b640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa390140450 0x7fa39013e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa39594b640 1 -- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa37c0039a0 msgr2=0x7fa39013f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa39594b640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa37c0039a0 0x7fa39013f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.575+0000 7fa39594b640 1 -- 172.21.15.70:0/3876058998 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900b3620 con 0x7fa390140ed0 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.576+0000 7fa39594b640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa390140ed0 0x7fa39013ff10 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fa38000e7e0 tx=0x7fa38000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.576+0000 7fa386ffd640 1 -- 172.21.15.70:0/3876058998 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000c6d0 con 0x7fa390140ed0 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.576+0000 7fa386ffd640 1 -- 172.21.15.70:0/3876058998 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38000c830 con 0x7fa390140ed0 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.576+0000 7fa3983b0640 1 -- 172.21.15.70:0/3876058998 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3900b3900 con 0x7fa390140ed0 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.576+0000 7fa386ffd640 1 -- 172.21.15.70:0/3876058998 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000cba0 con 0x7fa390140ed0 2023-10-08T06:54:17.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.576+0000 7fa3983b0640 1 -- 172.21.15.70:0/3876058998 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa390140ae0 con 0x7fa390140ed0 2023-10-08T06:54:17.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.577+0000 7fa3983b0640 1 -- 172.21.15.70:0/3876058998 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa390067490 con 0x7fa390140ed0 2023-10-08T06:54:17.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.578+0000 7fa386ffd640 1 -- 172.21.15.70:0/3876058998 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 96) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fa38000cd00 con 0x7fa390140ed0 2023-10-08T06:54:17.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.579+0000 7fa386ffd640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa37406acc0 0x7fa37406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:17.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.580+0000 7fa386ffd640 1 -- 172.21.15.70:0/3876058998 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fa380089470 con 0x7fa390140ed0 2023-10-08T06:54:17.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.580+0000 7fa39614c640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa37406acc0 0x7fa37406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:17.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.581+0000 7fa39614c640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa37406acc0 0x7fa37406d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa38c034000 tx=0x7fa38c007000 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:17.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.583+0000 7fa386ffd640 1 -- 172.21.15.70:0/3876058998 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3800543f0 con 0x7fa390140ed0 2023-10-08T06:54:17.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.810+0000 7fa3983b0640 1 -- 172.21.15.70:0/3876058998 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3900b3e20 con 0x7fa390140ed0 2023-10-08T06:54:17.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.812+0000 7fa386ffd640 1 -- 172.21.15.70:0/3876058998 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4917 (secure 0 0 0) 0x7fa3900b3e20 con 0x7fa390140ed0 2023-10-08T06:54:17.813 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:17.813 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":83,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/542411062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":542411062},{"type":"v1","addr":"172.21.15.181:6836","nonce":542411062}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/646194010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":646194010},{"type":"v1","addr":"172.21.15.70:6835","nonce":646194010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1850202608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1850202608},{"type":"v1","addr":"172.21.15.181:6837","nonce":1850202608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:53:47.285813+0000","modified":"2023-10-08T06:53:50.356439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9591},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9591":{"gid":9591,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6834/3790014600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3790014600},{"type":"v1","addr":"172.21.15.70:6834","nonce":3790014600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-10-08T06:54:17.814 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 83 2023-10-08T06:54:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 -- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa37406acc0 msgr2=0x7fa37406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa37406acc0 0x7fa37406d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa38c034000 tx=0x7fa38c007000 comp rx=0 tx=0).stop 2023-10-08T06:54:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 -- 172.21.15.70:0/3876058998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa390140ed0 msgr2=0x7fa39013ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa390140ed0 0x7fa39013ff10 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fa38000e7e0 tx=0x7fa38000eca0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 -- 172.21.15.70:0/3876058998 shutdown_connections 2023-10-08T06:54:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa37406acc0 0x7fa37406d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa390140450 0x7fa39013e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa390140ed0 0x7fa39013ff10 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 --2- 172.21.15.70:0/3876058998 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa37c0039a0 0x7fa39013f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:17.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 -- 172.21.15.70:0/3876058998 >> 172.21.15.70:0/3876058998 conn(0x7fa39006a380 msgr2=0x7fa39006c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:17.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.814+0000 7fa3983b0640 1 -- 172.21.15.70:0/3876058998 shutdown_connections 2023-10-08T06:54:17.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:17.815+0000 7fa3983b0640 1 -- 172.21.15.70:0/3876058998 wait complete. 2023-10-08T06:54:17.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:54:18.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.043+0000 7f08aa452640 1 Processor -- start 2023-10-08T06:54:18.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.043+0000 7f08aa452640 1 -- start start 2023-10-08T06:54:18.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.044+0000 7f08aa452640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f08a40ad790 0x7f08a40702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.044+0000 7f08aa452640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f08a415e330 0x7f08a4070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.044+0000 7f08aa452640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a406dee0 con 0x7f08a4159ac0 2023-10-08T06:54:18.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.044+0000 7f08aa452640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a406dcf0 con 0x7f08a40ad790 2023-10-08T06:54:18.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.044+0000 7f08aa452640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a415d850 con 0x7f08a415e330 2023-10-08T06:54:18.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.044+0000 7f089bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f08a40ad790 0x7f08a40702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.044+0000 7f089bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f08a40ad790 0x7f08a40702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57728/0 (socket says 172.21.15.70:57728) 2023-10-08T06:54:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.044+0000 7f089bfff640 1 -- 172.21.15.70:0/4270848637 learned_addr learned my addr 172.21.15.70:0/4270848637 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.045+0000 7f08a89ef640 1 --2- 172.21.15.70:0/4270848637 >> v2:172.21.15.181:3300/0 conn(0x7f08a415e330 0x7f08a4070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.045+0000 7f089bfff640 1 -- 172.21.15.70:0/4270848637 >> v2:172.21.15.181:3300/0 conn(0x7f08a415e330 msgr2=0x7f08a4070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.045+0000 7f089bfff640 1 --2- 172.21.15.70:0/4270848637 >> v2:172.21.15.181:3300/0 conn(0x7f08a415e330 0x7f08a4070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.045+0000 7f089bfff640 1 -- 172.21.15.70:0/4270848637 >> v1:172.21.15.181:6789/0 conn(0x7f08a4159ac0 legacy=0x7f08a406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:18.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.045+0000 7f089bfff640 1 -- 172.21.15.70:0/4270848637 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a415db30 con 0x7f08a40ad790 2023-10-08T06:54:18.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.045+0000 7f08a89ef640 1 --2- 172.21.15.70:0/4270848637 >> v2:172.21.15.181:3300/0 conn(0x7f08a415e330 0x7f08a4070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:18.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.046+0000 7f089bfff640 1 --2- 172.21.15.70:0/4270848637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a40ad790 0x7f08a40702d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f08a40caab0 tx=0x7f088c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1542b4afc7bea256 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:18.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.046+0000 7f08a37fe640 1 -- 172.21.15.70:0/4270848637 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c031870 con 0x7f08a40ad790 2023-10-08T06:54:18.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.047+0000 7f08a37fe640 1 -- 172.21.15.70:0/4270848637 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088c0319d0 con 0x7f08a40ad790 2023-10-08T06:54:18.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.047+0000 7f08a37fe640 1 -- 172.21.15.70:0/4270848637 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c031d60 con 0x7f08a40ad790 2023-10-08T06:54:18.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.047+0000 7f08aa452640 1 -- 172.21.15.70:0/4270848637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a40ad790 msgr2=0x7f08a40702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.047+0000 7f08aa452640 1 --2- 172.21.15.70:0/4270848637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a40ad790 0x7f08a40702d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f08a40caab0 tx=0x7f088c02b910 comp rx=0 tx=0).stop 2023-10-08T06:54:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.047+0000 7f08aa452640 1 -- 172.21.15.70:0/4270848637 shutdown_connections 2023-10-08T06:54:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.047+0000 7f08aa452640 1 --2- 172.21.15.70:0/4270848637 >> v2:172.21.15.181:3300/0 conn(0x7f08a415e330 0x7f08a4070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.047+0000 7f08aa452640 1 --2- 172.21.15.70:0/4270848637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a40ad790 0x7f08a40702d0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.047+0000 7f08aa452640 1 -- 172.21.15.70:0/4270848637 >> 172.21.15.70:0/4270848637 conn(0x7f08a406a380 msgr2=0x7f08a40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.047+0000 7f08aa452640 1 -- 172.21.15.70:0/4270848637 shutdown_connections 2023-10-08T06:54:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.047+0000 7f08aa452640 1 -- 172.21.15.70:0/4270848637 wait complete. 2023-10-08T06:54:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.049+0000 7f08aa452640 1 Processor -- start 2023-10-08T06:54:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.049+0000 7f08aa452640 1 -- start start 2023-10-08T06:54:18.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.050+0000 7f08aa452640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40ad790 0x7f08a414c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.050+0000 7f08aa452640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a4159ac0 0x7f08a40cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.050+0000 7f08aa452640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a415e330 0x7f08a40cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.050+0000 7f08aa452640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40c6d40 con 0x7f08a4159ac0 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.050+0000 7f08aa452640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a415b210 con 0x7f08a415e330 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.050+0000 7f08aa452640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a40c64a0 con 0x7f08a40ad790 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.050+0000 7f089bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a4159ac0 0x7f08a40cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f089bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a4159ac0 0x7f08a40cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57740/0 (socket says 172.21.15.70:57740) 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f089bfff640 1 -- 172.21.15.70:0/2876209979 learned_addr learned my addr 172.21.15.70:0/2876209979 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f08a89ef640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a415e330 0x7f08a40cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f08a3fff640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40ad790 0x7f08a414c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f089bfff640 1 -- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40ad790 msgr2=0x7f08a414c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f089bfff640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40ad790 0x7f08a414c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f089bfff640 1 -- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a415e330 msgr2=0x7f08a40cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f089bfff640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a415e330 0x7f08a40cdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f089bfff640 1 -- 172.21.15.70:0/2876209979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0094d0 con 0x7f08a4159ac0 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f08a89ef640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a415e330 0x7f08a40cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f08a3fff640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40ad790 0x7f08a414c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f089bfff640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a4159ac0 0x7f08a40cd6d0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f088c000c00 tx=0x7f088c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.051+0000 7f08a17fa640 1 -- 172.21.15.70:0/2876209979 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c031840 con 0x7f08a4159ac0 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.052+0000 7f08a17fa640 1 -- 172.21.15.70:0/2876209979 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088c0443f0 con 0x7f08a4159ac0 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.052+0000 7f08a17fa640 1 -- 172.21.15.70:0/2876209979 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c0447a0 con 0x7f08a4159ac0 2023-10-08T06:54:18.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.052+0000 7f08aa452640 1 -- 172.21.15.70:0/2876209979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a40c6780 con 0x7f08a4159ac0 2023-10-08T06:54:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.052+0000 7f08aa452640 1 -- 172.21.15.70:0/2876209979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a415b4b0 con 0x7f08a4159ac0 2023-10-08T06:54:18.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.053+0000 7f08aa452640 1 -- 172.21.15.70:0/2876209979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a40cc130 con 0x7f08a4159ac0 2023-10-08T06:54:18.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.054+0000 7f08a17fa640 1 -- 172.21.15.70:0/2876209979 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 97) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f088c04d450 con 0x7f08a4159ac0 2023-10-08T06:54:18.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.055+0000 7f08a17fa640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f088406af00 0x7f088406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.055+0000 7f08a17fa640 1 -- 172.21.15.70:0/2876209979 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f088c02d040 con 0x7f08a4159ac0 2023-10-08T06:54:18.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.055+0000 7f08a3fff640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f088406af00 0x7f088406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.056+0000 7f08a3fff640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f088406af00 0x7f088406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0894003eb0 tx=0x7f0894009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.059+0000 7f08a17fa640 1 -- 172.21.15.70:0/2876209979 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f088c076ef0 con 0x7f08a4159ac0 2023-10-08T06:54:18.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.250+0000 7f08aa452640 1 -- 172.21.15.70:0/2876209979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08a4067490 con 0x7f08a4159ac0 2023-10-08T06:54:18.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.252+0000 7f08a17fa640 1 -- 172.21.15.70:0/2876209979 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+17916 (secure 0 0 0) 0x7f088c07a510 con 0x7f08a4159ac0 2023-10-08T06:54:18.254 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:18.254 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":187,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:54:14.934496+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:53:44.876671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-08T06:53:46.310621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2596125885":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/1985247114":"2023-10-09T06:54:14.934383+0000","172.21.15.181:6833/1385757024":"2023-10-09T06:54:14.934383+0000","172.21.15.181:6832/1385757024":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/2383107395":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/1278239547":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/3845072443":"2023-10-09T06:54:14.934383+0000","172.21.15.181:6833/4245513204":"2023-10-09T06:54:07.074878+0000","172.21.15.181:6832/4245513204":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/1356319692":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/4081249518":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2711042436":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2536889452":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2861165849":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/998820505":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/4176429568":"2023-10-09T06:54:07.074878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:54:18.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.254+0000 7f08aa452640 1 -- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f088406af00 msgr2=0x7f088406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.254+0000 7f08aa452640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f088406af00 0x7f088406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0894003eb0 tx=0x7f0894009040 comp rx=0 tx=0).stop 2023-10-08T06:54:18.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.254+0000 7f08aa452640 1 -- 172.21.15.70:0/2876209979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a4159ac0 msgr2=0x7f08a40cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.254+0000 7f08aa452640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a4159ac0 0x7f08a40cd6d0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f088c000c00 tx=0x7f088c0046c0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.255+0000 7f08aa452640 1 -- 172.21.15.70:0/2876209979 shutdown_connections 2023-10-08T06:54:18.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.255+0000 7f08aa452640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f088406af00 0x7f088406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.255+0000 7f08aa452640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08a415e330 0x7f08a40cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.255+0000 7f08aa452640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08a4159ac0 0x7f08a40cd6d0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.255+0000 7f08aa452640 1 --2- 172.21.15.70:0/2876209979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08a40ad790 0x7f08a414c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.255+0000 7f08aa452640 1 -- 172.21.15.70:0/2876209979 >> 172.21.15.70:0/2876209979 conn(0x7f08a406a380 msgr2=0x7f08a40c91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:18.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.255+0000 7f08aa452640 1 -- 172.21.15.70:0/2876209979 shutdown_connections 2023-10-08T06:54:18.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.255+0000 7f08aa452640 1 -- 172.21.15.70:0/2876209979 wait complete. 2023-10-08T06:54:18.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T06:54:18.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.490+0000 7f51ca856640 1 Processor -- start 2023-10-08T06:54:18.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.490+0000 7f51ca856640 1 -- start start 2023-10-08T06:54:18.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.491+0000 7f51ca856640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c406dfc0 0x7f51c4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.491+0000 7f51ca856640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c40b1cc0 con 0x7f51c40afbe0 2023-10-08T06:54:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.491+0000 7f51ca856640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c40b2620 con 0x7f51c4071070 2023-10-08T06:54:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.491+0000 7f51ca856640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40b2060 con 0x7f51c406dfc0 2023-10-08T06:54:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.491+0000 7f51c3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f51c40afbe0 0x7f51c40b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51334/0 (socket says 172.21.15.70:51334) 2023-10-08T06:54:18.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.491+0000 7f51c3fff640 1 -- 172.21.15.70:0/1073049638 learned_addr learned my addr 172.21.15.70:0/1073049638 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:18.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51bb7fe640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c406dfc0 0x7f51c4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410554073 0 0) 0x7f51c40b1cc0 con 0x7f51c40afbe0 2023-10-08T06:54:18.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b0003520 con 0x7f51c40afbe0 2023-10-08T06:54:18.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3118081040 0 0) 0x7f51b0003520 con 0x7f51c40afbe0 2023-10-08T06:54:18.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c406dfc0 msgr2=0x7f51c4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c406dfc0 0x7f51c4070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 >> v1:172.21.15.181:6789/0 conn(0x7f51c4071070 legacy=0x7f51c40cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c40b2340 con 0x7f51c40afbe0 2023-10-08T06:54:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f51b40019f0 con 0x7f51c40afbe0 2023-10-08T06:54:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 >> v1:172.21.15.70:6789/0 conn(0x7f51c40afbe0 legacy=0x7f51c40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b00039a0 0x7f51b0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c406dfc0 0x7f51b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51b0004d00 0x7f51b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40b2060 con 0x7f51b0004d00 2023-10-08T06:54:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b000ba60 con 0x7f51b00039a0 2023-10-08T06:54:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.492+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b000bc20 con 0x7f51c406dfc0 2023-10-08T06:54:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.493+0000 7f51c8df3640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51b0004d00 0x7f51b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.493+0000 7f51c3fff640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b00039a0 0x7f51b0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.493+0000 7f51bb7fe640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c406dfc0 0x7f51b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.493+0000 7f51c8df3640 1 -- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c406dfc0 msgr2=0x7f51b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.493+0000 7f51c8df3640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c406dfc0 0x7f51b00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.493+0000 7f51c8df3640 1 -- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b00039a0 msgr2=0x7f51b0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.493+0000 7f51c8df3640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b00039a0 0x7f51b0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.493+0000 7f51c8df3640 1 -- 172.21.15.70:0/1073049638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0003520 con 0x7f51b0004d00 2023-10-08T06:54:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.493+0000 7f51bb7fe640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c406dfc0 0x7f51b00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:18.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.494+0000 7f51c8df3640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51b0004d00 0x7f51b0005180 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f51bc001750 tx=0x7f51b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45394ba85448242a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:18.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.494+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0052d0 con 0x7f51b0004d00 2023-10-08T06:54:18.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.494+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc02e050 con 0x7f51b0004d00 2023-10-08T06:54:18.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.494+0000 7f51c37fe640 1 -- 172.21.15.70:0/1073049638 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc0035f0 con 0x7f51b0004d00 2023-10-08T06:54:18.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.495+0000 7f51ca856640 1 -- 172.21.15.70:0/1073049638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51b0004d00 msgr2=0x7f51b0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.495+0000 7f51ca856640 1 --2- 172.21.15.70:0/1073049638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51b0004d00 0x7f51b0005180 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f51bc001750 tx=0x7f51b00057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.495+0000 7f51c8df3640 1 -- 172.21.15.70:0/1073049638 reap_dead start 2023-10-08T06:54:18.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.495+0000 7f51ca856640 1 -- 172.21.15.70:0/1073049638 shutdown_connections 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.495+0000 7f51ca856640 1 -- 172.21.15.70:0/1073049638 >> 172.21.15.70:0/1073049638 conn(0x7f51c406a5d0 msgr2=0x7f51c40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.495+0000 7f51ca856640 1 -- 172.21.15.70:0/1073049638 shutdown_connections 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.495+0000 7f51ca856640 1 -- 172.21.15.70:0/1073049638 wait complete. 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.497+0000 7f51ca856640 1 Processor -- start 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.497+0000 7f51ca856640 1 -- start start 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.498+0000 7f51ca856640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b0004d00 0x7f51c414c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.498+0000 7f51ca856640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51c40c5d90 0x7f51c4154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.498+0000 7f51ca856640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c40c6210 0x7f51c41553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.498+0000 7f51ca856640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4155900 con 0x7f51c40c5d90 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.498+0000 7f51ca856640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c41543e0 con 0x7f51b0004d00 2023-10-08T06:54:18.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.498+0000 7f51ca856640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40c2fd0 con 0x7f51c40c6210 2023-10-08T06:54:18.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.498+0000 7f51bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51c40c5d90 0x7f51c4154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.498+0000 7f51bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51c40c5d90 0x7f51c4154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57754/0 (socket says 172.21.15.70:57754) 2023-10-08T06:54:18.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.498+0000 7f51bb7fe640 1 -- 172.21.15.70:0/4120209022 learned_addr learned my addr 172.21.15.70:0/4120209022 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:18.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51c3fff640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b0004d00 0x7f51c414c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51c8df3640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c40c6210 0x7f51c41553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51bb7fe640 1 -- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c40c6210 msgr2=0x7f51c41553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51bb7fe640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c40c6210 0x7f51c41553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51bb7fe640 1 -- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b0004d00 msgr2=0x7f51c414c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51bb7fe640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b0004d00 0x7f51c414c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51bb7fe640 1 -- 172.21.15.70:0/4120209022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c4154680 con 0x7f51c40c5d90 2023-10-08T06:54:18.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51c8df3640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c40c6210 0x7f51c41553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:18.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51bb7fe640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51c40c5d90 0x7f51c4154e80 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f51ac010880 tx=0x7f51ac010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:18.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51c17fa640 1 -- 172.21.15.70:0/4120209022 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00eb60 con 0x7f51c40c5d90 2023-10-08T06:54:18.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51c17fa640 1 -- 172.21.15.70:0/4120209022 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac00ecc0 con 0x7f51c40c5d90 2023-10-08T06:54:18.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.499+0000 7f51c17fa640 1 -- 172.21.15.70:0/4120209022 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac01c400 con 0x7f51c40c5d90 2023-10-08T06:54:18.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.500+0000 7f51ca856640 1 -- 172.21.15.70:0/4120209022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c4154960 con 0x7f51c40c5d90 2023-10-08T06:54:18.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.500+0000 7f51ca856640 1 -- 172.21.15.70:0/4120209022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c40c22a0 con 0x7f51c40c5d90 2023-10-08T06:54:18.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.501+0000 7f51ca856640 1 -- 172.21.15.70:0/4120209022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c404e160 con 0x7f51c40c5d90 2023-10-08T06:54:18.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.503+0000 7f51c17fa640 1 -- 172.21.15.70:0/4120209022 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 97) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f51ac01c560 con 0x7f51c40c5d90 2023-10-08T06:54:18.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.503+0000 7f51c17fa640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f51a406af00 0x7f51a406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.503+0000 7f51c17fa640 1 -- 172.21.15.70:0/4120209022 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f51ac08a1c0 con 0x7f51c40c5d90 2023-10-08T06:54:18.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.504+0000 7f51c3fff640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f51a406af00 0x7f51a406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.505+0000 7f51c3fff640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f51a406af00 0x7f51a406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f51b4002010 tx=0x7f51b4009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:18.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.509+0000 7f51c17fa640 1 -- 172.21.15.70:0/4120209022 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51ac054e50 con 0x7f51c40c5d90 2023-10-08T06:54:18.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.731+0000 7f51ca856640 1 -- 172.21.15.70:0/4120209022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f51c40b0770 con 0x7f51c40c5d90 2023-10-08T06:54:18.732 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T06:54:18.731+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T06:54:18.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.734+0000 7f51c17fa640 1 -- 172.21.15.70:0/4120209022 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f51ac058470 con 0x7f51c40c5d90 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.736+0000 7f51ca856640 1 -- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f51a406af00 msgr2=0x7f51a406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.736+0000 7f51ca856640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f51a406af00 0x7f51a406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f51b4002010 tx=0x7f51b4009040 comp rx=0 tx=0).stop 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.737+0000 7f51ca856640 1 -- 172.21.15.70:0/4120209022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51c40c5d90 msgr2=0x7f51c4154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.737+0000 7f51ca856640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51c40c5d90 0x7f51c4154e80 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f51ac010880 tx=0x7f51ac010d40 comp rx=0 tx=0).stop 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.737+0000 7f51ca856640 1 -- 172.21.15.70:0/4120209022 shutdown_connections 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.737+0000 7f51ca856640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f51a406af00 0x7f51a406d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.737+0000 7f51ca856640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51c40c6210 0x7f51c41553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.737+0000 7f51ca856640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51c40c5d90 0x7f51c4154e80 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.737+0000 7f51ca856640 1 --2- 172.21.15.70:0/4120209022 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51b0004d00 0x7f51c414c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.737+0000 7f51ca856640 1 -- 172.21.15.70:0/4120209022 >> 172.21.15.70:0/4120209022 conn(0x7f51c406a5d0 msgr2=0x7f51c40ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:18.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.737+0000 7f51ca856640 1 -- 172.21.15.70:0/4120209022 shutdown_connections 2023-10-08T06:54:18.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.737+0000 7f51ca856640 1 -- 172.21.15.70:0/4120209022 wait complete. 2023-10-08T06:54:18.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T06:54:18.758 INFO:tasks.ceph.mds.a.smithi070.stderr: -4169> 2023-10-08T06:53:49.799+0000 7f3445929640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T06:54:18.824 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T06:54:18.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.947+0000 7f945e890640 1 Processor -- start 2023-10-08T06:54:18.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.947+0000 7f945e890640 1 -- start start 2023-10-08T06:54:18.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.948+0000 7f945e890640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f94581603a0 0x7f94580cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.948+0000 7f945e890640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9458071d50 0x7f9458160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.948+0000 7f945e890640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94580c9ff0 con 0x7f94580cdff0 2023-10-08T06:54:18.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.948+0000 7f945e890640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94580c3100 con 0x7f9458071d50 2023-10-08T06:54:18.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.948+0000 7f945e890640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94580ce480 con 0x7f94581603a0 2023-10-08T06:54:18.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.949+0000 7f945ce2d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9458071d50 0x7f9458160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.949+0000 7f945ce2d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9458071d50 0x7f9458160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57768/0 (socket says 172.21.15.70:57768) 2023-10-08T06:54:18.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.949+0000 7f945ce2d640 1 -- 172.21.15.70:0/2269428602 learned_addr learned my addr 172.21.15.70:0/2269428602 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:18.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.949+0000 7f945ce2d640 1 -- 172.21.15.70:0/2269428602 >> v2:172.21.15.181:3300/0 conn(0x7f94581603a0 msgr2=0x7f94580cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:18.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.949+0000 7f945ce2d640 1 --2- 172.21.15.70:0/2269428602 >> v2:172.21.15.181:3300/0 conn(0x7f94581603a0 0x7f94580cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.949+0000 7f945ce2d640 1 -- 172.21.15.70:0/2269428602 >> v1:172.21.15.181:6789/0 conn(0x7f94580cdff0 legacy=0x7f9458071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:18.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.949+0000 7f945ce2d640 1 -- 172.21.15.70:0/2269428602 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458072290 con 0x7f9458071d50 2023-10-08T06:54:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.950+0000 7f945ce2d640 1 --2- 172.21.15.70:0/2269428602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458071d50 0x7f9458160910 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f94580ccef0 tx=0x7f945002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de631b917fc4e129 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.950+0000 7f94577fe640 1 -- 172.21.15.70:0/2269428602 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945002ca80 con 0x7f9458071d50 2023-10-08T06:54:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.951+0000 7f94577fe640 1 -- 172.21.15.70:0/2269428602 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945002cbe0 con 0x7f9458071d50 2023-10-08T06:54:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.951+0000 7f94577fe640 1 -- 172.21.15.70:0/2269428602 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94500343f0 con 0x7f9458071d50 2023-10-08T06:54:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.951+0000 7f945e890640 1 -- 172.21.15.70:0/2269428602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458071d50 msgr2=0x7f9458160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.951+0000 7f945e890640 1 --2- 172.21.15.70:0/2269428602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458071d50 0x7f9458160910 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f94580ccef0 tx=0x7f945002bee0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.951+0000 7f945e890640 1 -- 172.21.15.70:0/2269428602 shutdown_connections 2023-10-08T06:54:18.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.951+0000 7f945e890640 1 --2- 172.21.15.70:0/2269428602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9458071d50 0x7f9458160910 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.951+0000 7f945e890640 1 --2- 172.21.15.70:0/2269428602 >> v2:172.21.15.181:3300/0 conn(0x7f94581603a0 0x7f94580cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.951+0000 7f945e890640 1 -- 172.21.15.70:0/2269428602 >> 172.21.15.70:0/2269428602 conn(0x7f94580cc2e0 msgr2=0x7f94580cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:18.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.951+0000 7f945e890640 1 -- 172.21.15.70:0/2269428602 shutdown_connections 2023-10-08T06:54:18.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.951+0000 7f945e890640 1 -- 172.21.15.70:0/2269428602 wait complete. 2023-10-08T06:54:18.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.953+0000 7f945e890640 1 Processor -- start 2023-10-08T06:54:18.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.953+0000 7f945e890640 1 -- start start 2023-10-08T06:54:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.954+0000 7f945e890640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9458071d50 0x7f94580ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.954+0000 7f945e890640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94580cdff0 0x7f9458159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.954+0000 7f945e890640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94581603a0 0x7f94581595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.954+0000 7f945e890640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94580acd20 con 0x7f94580cdff0 2023-10-08T06:54:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.954+0000 7f945e890640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94581548c0 con 0x7f9458071d50 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.954+0000 7f945e890640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9458153cb0 con 0x7f94581603a0 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.954+0000 7f944ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94580cdff0 0x7f9458159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.954+0000 7f944ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94580cdff0 0x7f9458159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57774/0 (socket says 172.21.15.70:57774) 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.954+0000 7f944ffff640 1 -- 172.21.15.70:0/1077227589 learned_addr learned my addr 172.21.15.70:0/1077227589 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f9457fff640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9458071d50 0x7f94580ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f944ffff640 1 -- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94581603a0 msgr2=0x7f94581595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f945ce2d640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94581603a0 0x7f94581595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f944ffff640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94581603a0 0x7f94581595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f944ffff640 1 -- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9458071d50 msgr2=0x7f94580ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f944ffff640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9458071d50 0x7f94580ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:18.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f944ffff640 1 -- 172.21.15.70:0/1077227589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9450009b30 con 0x7f94580cdff0 2023-10-08T06:54:18.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f9457fff640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9458071d50 0x7f94580ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:18.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f944ffff640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94580cdff0 0x7f9458159060 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f944000c440 tx=0x7f944000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:18.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f94557fa640 1 -- 172.21.15.70:0/1077227589 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440005480 con 0x7f94580cdff0 2023-10-08T06:54:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f94557fa640 1 -- 172.21.15.70:0/1077227589 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94400035f0 con 0x7f94580cdff0 2023-10-08T06:54:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f94557fa640 1 -- 172.21.15.70:0/1077227589 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400039c0 con 0x7f94580cdff0 2023-10-08T06:54:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f945e890640 1 -- 172.21.15.70:0/1077227589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9458153f90 con 0x7f94580cdff0 2023-10-08T06:54:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.955+0000 7f945e890640 1 -- 172.21.15.70:0/1077227589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9458154550 con 0x7f94580cdff0 2023-10-08T06:54:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.956+0000 7f945e890640 1 -- 172.21.15.70:0/1077227589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945804e160 con 0x7f94580cdff0 2023-10-08T06:54:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.957+0000 7f94557fa640 1 -- 172.21.15.70:0/1077227589 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 97) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9440003b20 con 0x7f94580cdff0 2023-10-08T06:54:18.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.958+0000 7f94557fa640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f943806af00 0x7f943806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:18.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.958+0000 7f94557fa640 1 -- 172.21.15.70:0/1077227589 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6919+0+0 (secure 0 0 0) 0x7f9440088c70 con 0x7f94580cdff0 2023-10-08T06:54:18.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.959+0000 7f9457fff640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f943806af00 0x7f943806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:18.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.960+0000 7f9457fff640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f943806af00 0x7f943806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9448006fd0 tx=0x7f9448008040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:18.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:18.963+0000 7f94557fa640 1 -- 172.21.15.70:0/1077227589 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9440053900 con 0x7f94580cdff0 2023-10-08T06:54:19.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.190+0000 7f945e890640 1 -- 172.21.15.70:0/1077227589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f94580b57e0 con 0x7f94580cdff0 2023-10-08T06:54:19.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.796+0000 7f94557fa640 1 -- 172.21.15.70:0/1077227589 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9440056f20 con 0x7f94580cdff0 2023-10-08T06:54:19.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 -- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f943806af00 msgr2=0x7f943806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:19.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f943806af00 0x7f943806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9448006fd0 tx=0x7f9448008040 comp rx=0 tx=0).stop 2023-10-08T06:54:19.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 -- 172.21.15.70:0/1077227589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94580cdff0 msgr2=0x7f9458159060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:19.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94580cdff0 0x7f9458159060 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f944000c440 tx=0x7f944000c900 comp rx=0 tx=0).stop 2023-10-08T06:54:19.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 -- 172.21.15.70:0/1077227589 shutdown_connections 2023-10-08T06:54:19.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f943806af00 0x7f943806d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:19.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94581603a0 0x7f94581595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:19.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94580cdff0 0x7f9458159060 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:19.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 --2- 172.21.15.70:0/1077227589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9458071d50 0x7f94580ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:19.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 -- 172.21.15.70:0/1077227589 >> 172.21.15.70:0/1077227589 conn(0x7f94580cc2e0 msgr2=0x7f945806a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:19.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 -- 172.21.15.70:0/1077227589 shutdown_connections 2023-10-08T06:54:19.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:19.800+0000 7f945e890640 1 -- 172.21.15.70:0/1077227589 wait complete. 2023-10-08T06:54:19.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T06:54:20.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.036+0000 7fd03e675640 1 Processor -- start 2023-10-08T06:54:20.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.036+0000 7fd03e675640 1 -- start start 2023-10-08T06:54:20.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd03e675640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd0380702b0 0x7fd038070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:20.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd03e675640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd0380ca370 0x7fd03806d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:20.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd03e675640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03806c3a0 con 0x7fd0380b21f0 2023-10-08T06:54:20.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd03e675640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd038070bd0 con 0x7fd0380702b0 2023-10-08T06:54:20.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd03e675640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03806c640 con 0x7fd0380ca370 2023-10-08T06:54:20.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd037fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd0380702b0 0x7fd038070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:20.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd037fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd0380702b0 0x7fd038070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57780/0 (socket says 172.21.15.70:57780) 2023-10-08T06:54:20.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd037fff640 1 -- 172.21.15.70:0/3636099324 learned_addr learned my addr 172.21.15.70:0/3636099324 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:20.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd037fff640 1 -- 172.21.15.70:0/3636099324 >> v2:172.21.15.181:3300/0 conn(0x7fd0380ca370 msgr2=0x7fd03806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:20.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd037fff640 1 --2- 172.21.15.70:0/3636099324 >> v2:172.21.15.181:3300/0 conn(0x7fd0380ca370 0x7fd03806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:20.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd037fff640 1 -- 172.21.15.70:0/3636099324 >> v1:172.21.15.181:6789/0 conn(0x7fd0380b21f0 legacy=0x7fd0380c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:20.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.038+0000 7fd037fff640 1 -- 172.21.15.70:0/3636099324 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03806c920 con 0x7fd0380702b0 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.040+0000 7fd037fff640 1 --2- 172.21.15.70:0/3636099324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380702b0 0x7fd038070690 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fd0380c8d10 tx=0x7fd02802cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bfa06f827fc47e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.040+0000 7fd0377fe640 1 -- 172.21.15.70:0/3636099324 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02802db00 con 0x7fd0380702b0 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.040+0000 7fd0377fe640 1 -- 172.21.15.70:0/3636099324 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02802dc60 con 0x7fd0380702b0 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.040+0000 7fd0377fe640 1 -- 172.21.15.70:0/3636099324 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028035440 con 0x7fd0380702b0 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.041+0000 7fd03e675640 1 -- 172.21.15.70:0/3636099324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380702b0 msgr2=0x7fd038070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.041+0000 7fd03e675640 1 --2- 172.21.15.70:0/3636099324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380702b0 0x7fd038070690 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fd0380c8d10 tx=0x7fd02802cd50 comp rx=0 tx=0).stop 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.041+0000 7fd03e675640 1 -- 172.21.15.70:0/3636099324 shutdown_connections 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.041+0000 7fd03e675640 1 --2- 172.21.15.70:0/3636099324 >> v2:172.21.15.181:3300/0 conn(0x7fd0380ca370 0x7fd03806d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.041+0000 7fd03e675640 1 --2- 172.21.15.70:0/3636099324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380702b0 0x7fd038070690 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.041+0000 7fd03e675640 1 -- 172.21.15.70:0/3636099324 >> 172.21.15.70:0/3636099324 conn(0x7fd03806a5d0 msgr2=0x7fd0380b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.041+0000 7fd03e675640 1 -- 172.21.15.70:0/3636099324 shutdown_connections 2023-10-08T06:54:20.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.041+0000 7fd03e675640 1 -- 172.21.15.70:0/3636099324 wait complete. 2023-10-08T06:54:20.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.043+0000 7fd03e675640 1 Processor -- start 2023-10-08T06:54:20.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.043+0000 7fd03e675640 1 -- start start 2023-10-08T06:54:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03e675640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0380702b0 0x7fd0380ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03e675640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0380b21f0 0x7fd0381485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03e675640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380ca370 0x7fd038145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03e675640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038158f70 con 0x7fd0380ca370 2023-10-08T06:54:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03e675640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd038148b10 con 0x7fd0380b21f0 2023-10-08T06:54:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03e675640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0380acd70 con 0x7fd0380702b0 2023-10-08T06:54:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03cc12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380ca370 0x7fd038145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03cc12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380ca370 0x7fd038145260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57792/0 (socket says 172.21.15.70:57792) 2023-10-08T06:54:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03cc12640 1 -- 172.21.15.70:0/2098835737 learned_addr learned my addr 172.21.15.70:0/2098835737 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd037fff640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0380702b0 0x7fd0380ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03cc12640 1 -- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0380702b0 msgr2=0x7fd0380ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03cc12640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0380702b0 0x7fd0380ade10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03cc12640 1 -- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0380b21f0 msgr2=0x7fd0381485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03cc12640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0380b21f0 0x7fd0381485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.044+0000 7fd03cc12640 1 -- 172.21.15.70:0/2098835737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02800a670 con 0x7fd0380ca370 2023-10-08T06:54:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.045+0000 7fd03cc12640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380ca370 0x7fd038145260 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fd03000b970 tx=0x7fd03000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.045+0000 7fd0357fa640 1 -- 172.21.15.70:0/2098835737 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03000cb40 con 0x7fd0380ca370 2023-10-08T06:54:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.045+0000 7fd0357fa640 1 -- 172.21.15.70:0/2098835737 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03000ccc0 con 0x7fd0380ca370 2023-10-08T06:54:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.045+0000 7fd0357fa640 1 -- 172.21.15.70:0/2098835737 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd030014680 con 0x7fd0380ca370 2023-10-08T06:54:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.045+0000 7fd03e675640 1 -- 172.21.15.70:0/2098835737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0380ad050 con 0x7fd0380ca370 2023-10-08T06:54:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.045+0000 7fd03e675640 1 -- 172.21.15.70:0/2098835737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0380ad610 con 0x7fd0380ca370 2023-10-08T06:54:20.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.046+0000 7fd03e675640 1 -- 172.21.15.70:0/2098835737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03804e160 con 0x7fd0380ca370 2023-10-08T06:54:20.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.047+0000 7fd0357fa640 1 -- 172.21.15.70:0/2098835737 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd03001d3b0 con 0x7fd0380ca370 2023-10-08T06:54:20.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.048+0000 7fd0357fa640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd01c06af90 0x7fd01c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:20.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.048+0000 7fd0357fa640 1 -- 172.21.15.70:0/2098835737 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6919+0+0 (secure 0 0 0) 0x7fd030089b40 con 0x7fd0380ca370 2023-10-08T06:54:20.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.049+0000 7fd037fff640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd01c06af90 0x7fd01c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:20.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.050+0000 7fd037fff640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd01c06af90 0x7fd01c06d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd028037000 tx=0x7fd028008040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:20.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.053+0000 7fd0357fa640 1 -- 172.21.15.70:0/2098835737 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0300546b0 con 0x7fd0380ca370 2023-10-08T06:54:20.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.241+0000 7fd03e675640 1 -- 172.21.15.70:0/2098835737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd0380b1dd0 con 0x7fd0380ca370 2023-10-08T06:54:20.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.889+0000 7fd0357fa640 1 -- 172.21.15.70:0/2098835737 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v189) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd0300190d0 con 0x7fd0380ca370 2023-10-08T06:54:20.891 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T06:54:20.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.893+0000 7fd03e675640 1 -- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd01c06af90 msgr2=0x7fd01c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.893+0000 7fd03e675640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd01c06af90 0x7fd01c06d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd028037000 tx=0x7fd028008040 comp rx=0 tx=0).stop 2023-10-08T06:54:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.893+0000 7fd03e675640 1 -- 172.21.15.70:0/2098835737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380ca370 msgr2=0x7fd038145260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.893+0000 7fd03e675640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380ca370 0x7fd038145260 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fd03000b970 tx=0x7fd03000be30 comp rx=0 tx=0).stop 2023-10-08T06:54:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.893+0000 7fd03e675640 1 -- 172.21.15.70:0/2098835737 shutdown_connections 2023-10-08T06:54:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.893+0000 7fd03e675640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd01c06af90 0x7fd01c06d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.893+0000 7fd03e675640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0380ca370 0x7fd038145260 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.893+0000 7fd03e675640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0380b21f0 0x7fd0381485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:20.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.893+0000 7fd03e675640 1 --2- 172.21.15.70:0/2098835737 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0380702b0 0x7fd0380ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:20.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.893+0000 7fd03e675640 1 -- 172.21.15.70:0/2098835737 >> 172.21.15.70:0/2098835737 conn(0x7fd03806a5d0 msgr2=0x7fd03815bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:20.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.894+0000 7fd03e675640 1 -- 172.21.15.70:0/2098835737 shutdown_connections 2023-10-08T06:54:20.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:20.894+0000 7fd03e675640 1 -- 172.21.15.70:0/2098835737 wait complete. 2023-10-08T06:54:20.920 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T06:54:21.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.125+0000 7f96c38bb640 1 Processor -- start 2023-10-08T06:54:21.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.125+0000 7f96c38bb640 1 -- start start 2023-10-08T06:54:21.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.127+0000 7f96c38bb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc070cc0 0x7f96bc0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:21.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.127+0000 7f96c38bb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f96bc0cb450 0x7f96bc0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:21.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.127+0000 7f96c38bb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc071140 con 0x7f96bc160660 2023-10-08T06:54:21.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.127+0000 7f96c38bb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96bc06e150 con 0x7f96bc0cb450 2023-10-08T06:54:21.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.127+0000 7f96c38bb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0cb8c0 con 0x7f96bc070cc0 2023-10-08T06:54:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.127+0000 7f96c0e56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc070cc0 0x7f96bc0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.128+0000 7f96c0e56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc070cc0 0x7f96bc0caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:38010/0 (socket says 172.21.15.70:38010) 2023-10-08T06:54:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.128+0000 7f96c0e56640 1 -- 172.21.15.70:0/4101214999 learned_addr learned my addr 172.21.15.70:0/4101214999 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.128+0000 7f96c1e58640 1 --2- 172.21.15.70:0/4101214999 >> v2:172.21.15.181:3300/0 conn(0x7f96bc0cb450 0x7f96bc0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.128+0000 7f96c0e56640 1 -- 172.21.15.70:0/4101214999 >> v2:172.21.15.181:3300/0 conn(0x7f96bc0cb450 msgr2=0x7f96bc0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.128+0000 7f96c0e56640 1 --2- 172.21.15.70:0/4101214999 >> v2:172.21.15.181:3300/0 conn(0x7f96bc0cb450 0x7f96bc0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.128+0000 7f96c0e56640 1 -- 172.21.15.70:0/4101214999 >> v1:172.21.15.181:6789/0 conn(0x7f96bc160660 legacy=0x7f96bc0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.128+0000 7f96c0e56640 1 -- 172.21.15.70:0/4101214999 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc06de70 con 0x7f96bc070cc0 2023-10-08T06:54:21.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.130+0000 7f96c0e56640 1 --2- 172.21.15.70:0/4101214999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc070cc0 0x7f96bc0caf10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009870 tx=0x7f96ac02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6faacca8ae50f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:21.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.130+0000 7f96b3fff640 1 -- 172.21.15.70:0/4101214999 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac030840 con 0x7f96bc070cc0 2023-10-08T06:54:21.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.130+0000 7f96b3fff640 1 -- 172.21.15.70:0/4101214999 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac0309a0 con 0x7f96bc070cc0 2023-10-08T06:54:21.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.130+0000 7f96b3fff640 1 -- 172.21.15.70:0/4101214999 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac030d30 con 0x7f96bc070cc0 2023-10-08T06:54:21.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.131+0000 7f96c38bb640 1 -- 172.21.15.70:0/4101214999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc070cc0 msgr2=0x7f96bc0caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:21.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.131+0000 7f96c38bb640 1 --2- 172.21.15.70:0/4101214999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc070cc0 0x7f96bc0caf10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009870 tx=0x7f96ac02b980 comp rx=0 tx=0).stop 2023-10-08T06:54:21.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.131+0000 7f96c38bb640 1 -- 172.21.15.70:0/4101214999 shutdown_connections 2023-10-08T06:54:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.131+0000 7f96c38bb640 1 --2- 172.21.15.70:0/4101214999 >> v2:172.21.15.181:3300/0 conn(0x7f96bc0cb450 0x7f96bc0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.131+0000 7f96c38bb640 1 --2- 172.21.15.70:0/4101214999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc070cc0 0x7f96bc0caf10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.131+0000 7f96c38bb640 1 -- 172.21.15.70:0/4101214999 >> 172.21.15.70:0/4101214999 conn(0x7f96bc06a5d0 msgr2=0x7f96bc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.131+0000 7f96c38bb640 1 -- 172.21.15.70:0/4101214999 shutdown_connections 2023-10-08T06:54:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.131+0000 7f96c38bb640 1 -- 172.21.15.70:0/4101214999 wait complete. 2023-10-08T06:54:21.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.133+0000 7f96c38bb640 1 Processor -- start 2023-10-08T06:54:21.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.133+0000 7f96c38bb640 1 -- start start 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.134+0000 7f96c38bb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96bc070cc0 0x7f96bc1f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.134+0000 7f96c38bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96bc0cb450 0x7f96bc0c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.134+0000 7f96c38bb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc160660 0x7f96bc1f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.134+0000 7f96c38bb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc1f6090 con 0x7f96bc0cb450 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.134+0000 7f96c38bb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0c5080 con 0x7f96bc070cc0 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.134+0000 7f96c38bb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0c4ac0 con 0x7f96bc160660 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.135+0000 7f96c0e56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96bc0cb450 0x7f96bc0c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.135+0000 7f96c0e56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96bc0cb450 0x7f96bc0c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57802/0 (socket says 172.21.15.70:57802) 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.135+0000 7f96c0e56640 1 -- 172.21.15.70:0/1130234719 learned_addr learned my addr 172.21.15.70:0/1130234719 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.135+0000 7f96c1657640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96bc070cc0 0x7f96bc1f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:21.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.135+0000 7f96c1e58640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc160660 0x7f96bc1f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:21.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.135+0000 7f96c0e56640 1 -- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc160660 msgr2=0x7f96bc1f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:21.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.135+0000 7f96c0e56640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc160660 0x7f96bc1f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:21.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.135+0000 7f96c0e56640 1 -- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96bc070cc0 msgr2=0x7f96bc1f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:21.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.135+0000 7f96c0e56640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96bc070cc0 0x7f96bc1f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:21.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.135+0000 7f96c0e56640 1 -- 172.21.15.70:0/1130234719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ac0094d0 con 0x7f96bc0cb450 2023-10-08T06:54:21.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.136+0000 7f96c0e56640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96bc0cb450 0x7f96bc0c4190 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f96ac000c00 tx=0x7f96ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:21.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.136+0000 7f96b1ffb640 1 -- 172.21.15.70:0/1130234719 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac039c90 con 0x7f96bc0cb450 2023-10-08T06:54:21.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.136+0000 7f96b1ffb640 1 -- 172.21.15.70:0/1130234719 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac039df0 con 0x7f96bc0cb450 2023-10-08T06:54:21.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.136+0000 7f96c38bb640 1 -- 172.21.15.70:0/1130234719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc0c4da0 con 0x7f96bc0cb450 2023-10-08T06:54:21.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.136+0000 7f96b1ffb640 1 -- 172.21.15.70:0/1130234719 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac043640 con 0x7f96bc0cb450 2023-10-08T06:54:21.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.136+0000 7f96c38bb640 1 -- 172.21.15.70:0/1130234719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc1f69f0 con 0x7f96bc0cb450 2023-10-08T06:54:21.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.137+0000 7f96c38bb640 1 -- 172.21.15.70:0/1130234719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc04e160 con 0x7f96bc0cb450 2023-10-08T06:54:21.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.138+0000 7f96b1ffb640 1 -- 172.21.15.70:0/1130234719 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f96ac0437a0 con 0x7f96bc0cb450 2023-10-08T06:54:21.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.139+0000 7f96b1ffb640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f96a006af90 0x7f96a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:21.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.139+0000 7f96b1ffb640 1 -- 172.21.15.70:0/1130234719 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f96ac03e070 con 0x7f96bc0cb450 2023-10-08T06:54:21.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.140+0000 7f96c1657640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f96a006af90 0x7f96a006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:21.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.141+0000 7f96c1657640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f96a006af90 0x7f96a006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f96b8000f60 tx=0x7f96b8009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:21.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.144+0000 7f96b1ffb640 1 -- 172.21.15.70:0/1130234719 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96ac077800 con 0x7f96bc0cb450 2023-10-08T06:54:21.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.332+0000 7f96c38bb640 1 -- 172.21.15.70:0/1130234719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f96bc0c1ca0 con 0x7f96bc0cb450 2023-10-08T06:54:21.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.914+0000 7f96b1ffb640 1 -- 172.21.15.70:0/1130234719 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v190) v1 ==== 169+0+0 (secure 0 0 0) 0x7f96ac07ae20 con 0x7f96bc0cb450 2023-10-08T06:54:21.915 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T06:54:21.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 -- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f96a006af90 msgr2=0x7f96a006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:21.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f96a006af90 0x7f96a006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f96b8000f60 tx=0x7f96b8009040 comp rx=0 tx=0).stop 2023-10-08T06:54:21.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 -- 172.21.15.70:0/1130234719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96bc0cb450 msgr2=0x7f96bc0c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:21.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96bc0cb450 0x7f96bc0c4190 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f96ac000c00 tx=0x7f96ac004730 comp rx=0 tx=0).stop 2023-10-08T06:54:21.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 -- 172.21.15.70:0/1130234719 shutdown_connections 2023-10-08T06:54:21.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f96a006af90 0x7f96a006d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:21.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96bc160660 0x7f96bc1f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:21.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96bc0cb450 0x7f96bc0c4190 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:21.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 --2- 172.21.15.70:0/1130234719 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96bc070cc0 0x7f96bc1f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:21.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 -- 172.21.15.70:0/1130234719 >> 172.21.15.70:0/1130234719 conn(0x7f96bc06a5d0 msgr2=0x7f96bc15f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:21.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 -- 172.21.15.70:0/1130234719 shutdown_connections 2023-10-08T06:54:21.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:21.921+0000 7f96c38bb640 1 -- 172.21.15.70:0/1130234719 wait complete. 2023-10-08T06:54:21.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T06:54:22.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.148+0000 7f0b7637c640 1 Processor -- start 2023-10-08T06:54:22.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.149+0000 7f0b7637c640 1 -- start start 2023-10-08T06:54:22.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.150+0000 7f0b7637c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0b700afbe0 0x7f0b700b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.150+0000 7f0b7637c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0b7006dfc0 0x7f0b70070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.150+0000 7f0b7637c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b700b2620 con 0x7f0b70071070 2023-10-08T06:54:22.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.150+0000 7f0b7637c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b700b2060 con 0x7f0b700afbe0 2023-10-08T06:54:22.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.150+0000 7f0b7637c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b700b2340 con 0x7f0b7006dfc0 2023-10-08T06:54:22.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.150+0000 7f0b6ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0b700afbe0 0x7f0b700b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.150+0000 7f0b6ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0b700afbe0 0x7f0b700b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57806/0 (socket says 172.21.15.70:57806) 2023-10-08T06:54:22.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.151+0000 7f0b6ffff640 1 -- 172.21.15.70:0/4223869393 learned_addr learned my addr 172.21.15.70:0/4223869393 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:22.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.151+0000 7f0b6ffff640 1 -- 172.21.15.70:0/4223869393 >> v2:172.21.15.181:3300/0 conn(0x7f0b7006dfc0 msgr2=0x7f0b70070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:22.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.151+0000 7f0b6ffff640 1 --2- 172.21.15.70:0/4223869393 >> v2:172.21.15.181:3300/0 conn(0x7f0b7006dfc0 0x7f0b70070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.151+0000 7f0b6ffff640 1 -- 172.21.15.70:0/4223869393 >> v1:172.21.15.181:6789/0 conn(0x7f0b70071070 legacy=0x7f0b700cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:22.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.151+0000 7f0b6ffff640 1 -- 172.21.15.70:0/4223869393 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b70070350 con 0x7f0b700afbe0 2023-10-08T06:54:22.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.152+0000 7f0b6ffff640 1 --2- 172.21.15.70:0/4223869393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b700afbe0 0x7f0b700b15b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f0b700b3980 tx=0x7f0b6002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c40125a342411b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:22.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.153+0000 7f0b6effd640 1 -- 172.21.15.70:0/4223869393 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6002ca80 con 0x7f0b700afbe0 2023-10-08T06:54:22.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.153+0000 7f0b6effd640 1 -- 172.21.15.70:0/4223869393 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6002cbe0 con 0x7f0b700afbe0 2023-10-08T06:54:22.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.153+0000 7f0b6effd640 1 -- 172.21.15.70:0/4223869393 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b600343f0 con 0x7f0b700afbe0 2023-10-08T06:54:22.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.153+0000 7f0b7637c640 1 -- 172.21.15.70:0/4223869393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b700afbe0 msgr2=0x7f0b700b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.153+0000 7f0b7637c640 1 --2- 172.21.15.70:0/4223869393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b700afbe0 0x7f0b700b15b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f0b700b3980 tx=0x7f0b6002bee0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.154+0000 7f0b7637c640 1 -- 172.21.15.70:0/4223869393 shutdown_connections 2023-10-08T06:54:22.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.154+0000 7f0b7637c640 1 --2- 172.21.15.70:0/4223869393 >> v2:172.21.15.181:3300/0 conn(0x7f0b7006dfc0 0x7f0b70070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.154+0000 7f0b7637c640 1 --2- 172.21.15.70:0/4223869393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b700afbe0 0x7f0b700b15b0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.154+0000 7f0b7637c640 1 -- 172.21.15.70:0/4223869393 >> 172.21.15.70:0/4223869393 conn(0x7f0b7006a5d0 msgr2=0x7f0b700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:22.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.154+0000 7f0b7637c640 1 -- 172.21.15.70:0/4223869393 shutdown_connections 2023-10-08T06:54:22.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.154+0000 7f0b7637c640 1 -- 172.21.15.70:0/4223869393 wait complete. 2023-10-08T06:54:22.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.155+0000 7f0b7637c640 1 Processor -- start 2023-10-08T06:54:22.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.156+0000 7f0b7637c640 1 -- start start 2023-10-08T06:54:22.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.156+0000 7f0b7637c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b7006dfc0 0x7f0b7014d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b7637c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b70071070 0x7f0b7015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b7637c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b700afbe0 0x7f0b7015db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b7637c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7015e4b0 con 0x7f0b70071070 2023-10-08T06:54:22.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b7637c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b70156d10 con 0x7f0b7006dfc0 2023-10-08T06:54:22.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b7637c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7015e1d0 con 0x7f0b700afbe0 2023-10-08T06:54:22.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b6f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b70071070 0x7f0b7015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b6f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b70071070 0x7f0b7015d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57812/0 (socket says 172.21.15.70:57812) 2023-10-08T06:54:22.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b6f7fe640 1 -- 172.21.15.70:0/3355790766 learned_addr learned my addr 172.21.15.70:0/3355790766 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:22.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b6f7fe640 1 -- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b700afbe0 msgr2=0x7f0b7015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b74919640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b700afbe0 0x7f0b7015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b6f7fe640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b700afbe0 0x7f0b7015db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b6ffff640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b7006dfc0 0x7f0b7014d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b6f7fe640 1 -- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b7006dfc0 msgr2=0x7f0b7014d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b6f7fe640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b7006dfc0 0x7f0b7014d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.157+0000 7f0b6f7fe640 1 -- 172.21.15.70:0/3355790766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b60009b30 con 0x7f0b70071070 2023-10-08T06:54:22.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.158+0000 7f0b6f7fe640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b70071070 0x7f0b7015d650 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400b410 tx=0x7f0b6400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:22.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.158+0000 7f0b6cff9640 1 -- 172.21.15.70:0/3355790766 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b64011830 con 0x7f0b70071070 2023-10-08T06:54:22.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.158+0000 7f0b6cff9640 1 -- 172.21.15.70:0/3355790766 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b64011990 con 0x7f0b70071070 2023-10-08T06:54:22.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.158+0000 7f0b6cff9640 1 -- 172.21.15.70:0/3355790766 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b64011d40 con 0x7f0b70071070 2023-10-08T06:54:22.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.158+0000 7f0b7637c640 1 -- 172.21.15.70:0/3355790766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b700cc7e0 con 0x7f0b70071070 2023-10-08T06:54:22.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.158+0000 7f0b7637c640 1 -- 172.21.15.70:0/3355790766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b700ccda0 con 0x7f0b70071070 2023-10-08T06:54:22.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.159+0000 7f0b7637c640 1 -- 172.21.15.70:0/3355790766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b700cc640 con 0x7f0b70071070 2023-10-08T06:54:22.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.160+0000 7f0b6cff9640 1 -- 172.21.15.70:0/3355790766 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0b6401a400 con 0x7f0b70071070 2023-10-08T06:54:22.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.161+0000 7f0b6cff9640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0b5c06af90 0x7f0b5c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.161+0000 7f0b6cff9640 1 -- 172.21.15.70:0/3355790766 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f0b64089b20 con 0x7f0b70071070 2023-10-08T06:54:22.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.162+0000 7f0b6ffff640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0b5c06af90 0x7f0b5c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.163+0000 7f0b6ffff640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0b5c06af90 0x7f0b5c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0b60035000 tx=0x7f0b60030000 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:22.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.166+0000 7f0b6cff9640 1 -- 172.21.15.70:0/3355790766 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b700cc640 con 0x7f0b70071070 2023-10-08T06:54:22.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.354+0000 7f0b7637c640 1 -- 172.21.15.70:0/3355790766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0b700cc640 con 0x7f0b70071070 2023-10-08T06:54:22.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.361+0000 7f0b6cff9640 1 -- 172.21.15.70:0/3355790766 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0b700cc640 con 0x7f0b70071070 2023-10-08T06:54:22.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 -- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0b5c06af90 msgr2=0x7f0b5c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0b5c06af90 0x7f0b5c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0b60035000 tx=0x7f0b60030000 comp rx=0 tx=0).stop 2023-10-08T06:54:22.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 -- 172.21.15.70:0/3355790766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b70071070 msgr2=0x7f0b7015d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b70071070 0x7f0b7015d650 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400b410 tx=0x7f0b6400b8d0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 -- 172.21.15.70:0/3355790766 shutdown_connections 2023-10-08T06:54:22.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0b5c06af90 0x7f0b5c06d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0b700afbe0 0x7f0b7015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b70071070 0x7f0b7015d650 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 --2- 172.21.15.70:0/3355790766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0b7006dfc0 0x7f0b7014d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 -- 172.21.15.70:0/3355790766 >> 172.21.15.70:0/3355790766 conn(0x7f0b7006a5d0 msgr2=0x7f0b7015e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:22.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 -- 172.21.15.70:0/3355790766 shutdown_connections 2023-10-08T06:54:22.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.364+0000 7f0b7637c640 1 -- 172.21.15.70:0/3355790766 wait complete. 2023-10-08T06:54:22.384 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T06:54:22.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.597+0000 7f2ebc92c640 1 Processor -- start 2023-10-08T06:54:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.597+0000 7f2ebc92c640 1 -- start start 2023-10-08T06:54:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.599+0000 7f2ebc92c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2eb80702d0 0x7f2eb815d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.599+0000 7f2ebc92c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb80b40e0 0x7f2eb80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.599+0000 7f2ebc92c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb80b30a0 con 0x7f2eb80b1750 2023-10-08T06:54:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.599+0000 7f2ebc92c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb815dcf0 con 0x7f2eb80702d0 2023-10-08T06:54:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.599+0000 7f2ebc92c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80b3320 con 0x7f2eb80b40e0 2023-10-08T06:54:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.599+0000 7f2eaffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2eb80b1750 0x7f2eb80706b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51356/0 (socket says 172.21.15.70:51356) 2023-10-08T06:54:22.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.599+0000 7f2eaffff640 1 -- 172.21.15.70:0/488436015 learned_addr learned my addr 172.21.15.70:0/488436015 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:22.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.599+0000 7f2eb659c640 1 --2- 172.21.15.70:0/488436015 >> v2:172.21.15.181:3300/0 conn(0x7f2eb80702d0 0x7f2eb815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb6d9d640 1 --2- 172.21.15.70:0/488436015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb80b40e0 0x7f2eb80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460524612 0 0) 0x7f2eb80b30a0 con 0x7f2eb80b1750 2023-10-08T06:54:22.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea4003520 con 0x7f2eb80b1750 2023-10-08T06:54:22.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1910458517 0 0) 0x7f2ea4003520 con 0x7f2eb80b1750 2023-10-08T06:54:22.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb80b40e0 msgr2=0x7f2eb80c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:22.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 --2- 172.21.15.70:0/488436015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb80b40e0 0x7f2eb80c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 >> v2:172.21.15.181:3300/0 conn(0x7f2eb80702d0 msgr2=0x7f2eb815d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:22.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 --2- 172.21.15.70:0/488436015 >> v2:172.21.15.181:3300/0 conn(0x7f2eb80702d0 0x7f2eb815d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80b3600 con 0x7f2eb80b1750 2023-10-08T06:54:22.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2ea0001670 con 0x7f2eb80b1750 2023-10-08T06:54:22.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 >> v1:172.21.15.70:6789/0 conn(0x7f2eb80b1750 legacy=0x7f2eb80706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 --2- 172.21.15.70:0/488436015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4003970 0x7f2ea4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 --2- 172.21.15.70:0/488436015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea40047c0 0x7f2ea4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 --2- 172.21.15.70:0/488436015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb80b40e0 0x7f2ea4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb815dcf0 con 0x7f2ea4003970 2023-10-08T06:54:22.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80b3320 con 0x7f2ea40047c0 2023-10-08T06:54:22.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.600+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea400dbc0 con 0x7f2eb80b40e0 2023-10-08T06:54:22.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.601+0000 7f2eb659c640 1 --2- 172.21.15.70:0/488436015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4003970 0x7f2ea4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.601+0000 7f2eb659c640 1 -- 172.21.15.70:0/488436015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb80b40e0 msgr2=0x7f2ea4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:22.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.601+0000 7f2eb659c640 1 --2- 172.21.15.70:0/488436015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb80b40e0 0x7f2ea4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.601+0000 7f2eb659c640 1 -- 172.21.15.70:0/488436015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea40047c0 msgr2=0x7f2ea4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.601+0000 7f2eb659c640 1 --2- 172.21.15.70:0/488436015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea40047c0 0x7f2ea4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.601+0000 7f2eb659c640 1 -- 172.21.15.70:0/488436015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea4003520 con 0x7f2ea4003970 2023-10-08T06:54:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.602+0000 7f2eb659c640 1 --2- 172.21.15.70:0/488436015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4003970 0x7f2ea4003d90 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009ac0 tx=0x7f2ea40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4610524a1ad4f5db server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:22.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.602+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea802c6d0 con 0x7f2ea4003970 2023-10-08T06:54:22.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.602+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea802c830 con 0x7f2ea4003970 2023-10-08T06:54:22.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.602+0000 7f2eb5d9b640 1 -- 172.21.15.70:0/488436015 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea802cbc0 con 0x7f2ea4003970 2023-10-08T06:54:22.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.603+0000 7f2ebc92c640 1 -- 172.21.15.70:0/488436015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4003970 msgr2=0x7f2ea4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.603+0000 7f2ebc92c640 1 --2- 172.21.15.70:0/488436015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4003970 0x7f2ea4003d90 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009ac0 tx=0x7f2ea40057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.603+0000 7f2eb6d9d640 1 -- 172.21.15.70:0/488436015 reap_dead start 2023-10-08T06:54:22.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.603+0000 7f2ebc92c640 1 -- 172.21.15.70:0/488436015 shutdown_connections 2023-10-08T06:54:22.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.603+0000 7f2ebc92c640 1 -- 172.21.15.70:0/488436015 >> 172.21.15.70:0/488436015 conn(0x7f2eb8069d50 msgr2=0x7f2eb806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:22.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.603+0000 7f2ebc92c640 1 -- 172.21.15.70:0/488436015 shutdown_connections 2023-10-08T06:54:22.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.603+0000 7f2ebc92c640 1 -- 172.21.15.70:0/488436015 wait complete. 2023-10-08T06:54:22.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.605+0000 7f2ebc92c640 1 Processor -- start 2023-10-08T06:54:22.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.605+0000 7f2ebc92c640 1 -- start start 2023-10-08T06:54:22.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.606+0000 7f2ebc92c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea4003970 0x7f2eb8150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.606+0000 7f2ebc92c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb80c9980 0x7f2eb81f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.606+0000 7f2ebc92c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb806c4f0 0x7f2eb806c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.606+0000 7f2ebc92c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb806cec0 con 0x7f2eb80c9980 2023-10-08T06:54:22.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.606+0000 7f2ebc92c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb8150bb0 con 0x7f2ea4003970 2023-10-08T06:54:22.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.606+0000 7f2ebc92c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81f3150 con 0x7f2eb806c4f0 2023-10-08T06:54:22.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.606+0000 7f2eaffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb80c9980 0x7f2eb81f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.606+0000 7f2eaffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb80c9980 0x7f2eb81f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57828/0 (socket says 172.21.15.70:57828) 2023-10-08T06:54:22.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.606+0000 7f2eaffff640 1 -- 172.21.15.70:0/2521305874 learned_addr learned my addr 172.21.15.70:0/2521305874 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:22.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2eaffff640 1 -- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb806c4f0 msgr2=0x7f2eb806c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2eaffff640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb806c4f0 0x7f2eb806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2eb659c640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea4003970 0x7f2eb8150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2eaffff640 1 -- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea4003970 msgr2=0x7f2eb8150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2eaffff640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea4003970 0x7f2eb8150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2eaffff640 1 -- 172.21.15.70:0/2521305874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb81f3430 con 0x7f2eb80c9980 2023-10-08T06:54:22.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2eaffff640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb80c9980 0x7f2eb81f27e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000c680 tx=0x7f2ea000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:22.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2eaeffd640 1 -- 172.21.15.70:0/2521305874 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea00072d0 con 0x7f2eb80c9980 2023-10-08T06:54:22.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2eaeffd640 1 -- 172.21.15.70:0/2521305874 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea0007430 con 0x7f2eb80c9980 2023-10-08T06:54:22.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2ebc92c640 1 -- 172.21.15.70:0/2521305874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb81f3710 con 0x7f2eb80c9980 2023-10-08T06:54:22.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2eaeffd640 1 -- 172.21.15.70:0/2521305874 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea00047e0 con 0x7f2eb80c9980 2023-10-08T06:54:22.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.607+0000 7f2ebc92c640 1 -- 172.21.15.70:0/2521305874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb81f3d20 con 0x7f2eb80c9980 2023-10-08T06:54:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.608+0000 7f2ebc92c640 1 -- 172.21.15.70:0/2521305874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb804e160 con 0x7f2eb80c9980 2023-10-08T06:54:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.609+0000 7f2eaeffd640 1 -- 172.21.15.70:0/2521305874 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2ea0004940 con 0x7f2eb80c9980 2023-10-08T06:54:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.610+0000 7f2eaeffd640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f2e9806af90 0x7f2e9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.611+0000 7f2eaeffd640 1 -- 172.21.15.70:0/2521305874 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f2ea0088d60 con 0x7f2eb80c9980 2023-10-08T06:54:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.611+0000 7f2eb659c640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f2e9806af90 0x7f2e9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.612+0000 7f2eb659c640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f2e9806af90 0x7f2e9806d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8036000 tx=0x7f2ea8030000 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.615+0000 7f2eaeffd640 1 -- 172.21.15.70:0/2521305874 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ea0053b90 con 0x7f2eb80c9980 2023-10-08T06:54:22.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.804+0000 7f2ebc92c640 1 -- 172.21.15.70:0/2521305874 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f2eb80710b0 con 0x7f2eb80c9980 2023-10-08T06:54:22.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.809+0000 7f2eaeffd640 1 -- 172.21.15.70:0/2521305874 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v62) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2ea0018070 con 0x7f2eb80c9980 2023-10-08T06:54:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 -- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f2e9806af90 msgr2=0x7f2e9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f2e9806af90 0x7f2e9806d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8036000 tx=0x7f2ea8030000 comp rx=0 tx=0).stop 2023-10-08T06:54:22.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 -- 172.21.15.70:0/2521305874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb80c9980 msgr2=0x7f2eb81f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:22.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb80c9980 0x7f2eb81f27e0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000c680 tx=0x7f2ea000cb40 comp rx=0 tx=0).stop 2023-10-08T06:54:22.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 -- 172.21.15.70:0/2521305874 shutdown_connections 2023-10-08T06:54:22.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f2e9806af90 0x7f2e9806d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2eb806c4f0 0x7f2eb806c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb80c9980 0x7f2eb81f27e0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 --2- 172.21.15.70:0/2521305874 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2ea4003970 0x7f2eb8150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:22.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 -- 172.21.15.70:0/2521305874 >> 172.21.15.70:0/2521305874 conn(0x7f2eb8069d50 msgr2=0x7f2eb80ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:22.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 -- 172.21.15.70:0/2521305874 shutdown_connections 2023-10-08T06:54:22.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:22.812+0000 7f2ebc92c640 1 -- 172.21.15.70:0/2521305874 wait complete. 2023-10-08T06:54:22.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T06:54:23.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.032+0000 7fd0eaa7c640 1 Processor -- start 2023-10-08T06:54:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.033+0000 7fd0eaa7c640 1 -- start start 2023-10-08T06:54:23.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.034+0000 7fd0eaa7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd0e4070cc0 0x7fd0e40caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.034+0000 7fd0eaa7c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e4071170 con 0x7fd0e4160660 2023-10-08T06:54:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.034+0000 7fd0eaa7c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e406e150 con 0x7fd0e40cb450 2023-10-08T06:54:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.034+0000 7fd0eaa7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e406de70 con 0x7fd0e4070cc0 2023-10-08T06:54:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.034+0000 7fd0e3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd0e4070cc0 0x7fd0e40caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.034+0000 7fd0e8818640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd0e4160660 0x7fd0e40c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51368/0 (socket says 172.21.15.70:51368) 2023-10-08T06:54:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.034+0000 7fd0e8818640 1 -- 172.21.15.70:0/4069792083 learned_addr learned my addr 172.21.15.70:0/4069792083 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:23.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.035+0000 7fd0e3fff640 1 -- 172.21.15.70:0/4069792083 >> v1:172.21.15.181:6789/0 conn(0x7fd0e40cb450 legacy=0x7fd0e40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:23.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.035+0000 7fd0e3fff640 1 -- 172.21.15.70:0/4069792083 >> v1:172.21.15.70:6789/0 conn(0x7fd0e4160660 legacy=0x7fd0e40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.035+0000 7fd0e3fff640 1 -- 172.21.15.70:0/4069792083 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e40c8e30 con 0x7fd0e4070cc0 2023-10-08T06:54:23.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.035+0000 7fd0e37fe640 1 -- 172.21.15.70:0/4069792083 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062442532 0 0) 0x7fd0d8001110 con 0x7fd0e4160660 2023-10-08T06:54:23.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.036+0000 7fd0e3fff640 1 --2- 172.21.15.70:0/4069792083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 0x7fd0e40caf10 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd0e406e340 tx=0x7fd0d002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd91f1cb1af9719 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:23.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.036+0000 7fd0e37fe640 1 -- 172.21.15.70:0/4069792083 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d002d040 con 0x7fd0e4070cc0 2023-10-08T06:54:23.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.036+0000 7fd0e37fe640 1 -- 172.21.15.70:0/4069792083 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d0031810 con 0x7fd0e4070cc0 2023-10-08T06:54:23.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.036+0000 7fd0e37fe640 1 -- 172.21.15.70:0/4069792083 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0031ba0 con 0x7fd0e4070cc0 2023-10-08T06:54:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.037+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4069792083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 msgr2=0x7fd0e40caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.037+0000 7fd0eaa7c640 1 --2- 172.21.15.70:0/4069792083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 0x7fd0e40caf10 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd0e406e340 tx=0x7fd0d002b8b0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.037+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4069792083 shutdown_connections 2023-10-08T06:54:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.037+0000 7fd0eaa7c640 1 --2- 172.21.15.70:0/4069792083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 0x7fd0e40caf10 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.037+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4069792083 >> 172.21.15.70:0/4069792083 conn(0x7fd0e406a5d0 msgr2=0x7fd0e40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.037+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4069792083 shutdown_connections 2023-10-08T06:54:23.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.037+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4069792083 wait complete. 2023-10-08T06:54:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.039+0000 7fd0eaa7c640 1 Processor -- start 2023-10-08T06:54:23.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.039+0000 7fd0eaa7c640 1 -- start start 2023-10-08T06:54:23.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.040+0000 7fd0eaa7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 0x7fd0e40c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.040+0000 7fd0eaa7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e40cb450 0x7fd0e41f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.040+0000 7fd0eaa7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0e4160660 0x7fd0e4155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.040+0000 7fd0eaa7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e415b6e0 con 0x7fd0e4070cc0 2023-10-08T06:54:23.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.040+0000 7fd0eaa7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e41f2f30 con 0x7fd0e40cb450 2023-10-08T06:54:23.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.040+0000 7fd0eaa7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e41f3210 con 0x7fd0e4160660 2023-10-08T06:54:23.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.040+0000 7fd0e8818640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 0x7fd0e40c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.040+0000 7fd0e8818640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 0x7fd0e40c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57834/0 (socket says 172.21.15.70:57834) 2023-10-08T06:54:23.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.040+0000 7fd0e8818640 1 -- 172.21.15.70:0/4026949413 learned_addr learned my addr 172.21.15.70:0/4026949413 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:23.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e3fff640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e40cb450 0x7fd0e41f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e8818640 1 -- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0e4160660 msgr2=0x7fd0e4155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e9019640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0e4160660 0x7fd0e4155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e8818640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0e4160660 0x7fd0e4155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e8818640 1 -- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e40cb450 msgr2=0x7fd0e41f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e8818640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e40cb450 0x7fd0e41f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e8818640 1 -- 172.21.15.70:0/4026949413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d0009400 con 0x7fd0e4070cc0 2023-10-08T06:54:23.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e3fff640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e40cb450 0x7fd0e41f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:23.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e8818640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 0x7fd0e40c5260 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800c780 tx=0x7fd0d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:23.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e17fa640 1 -- 172.21.15.70:0/4026949413 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80072d0 con 0x7fd0e4070cc0 2023-10-08T06:54:23.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e17fa640 1 -- 172.21.15.70:0/4026949413 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d8007450 con 0x7fd0e4070cc0 2023-10-08T06:54:23.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0e17fa640 1 -- 172.21.15.70:0/4026949413 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d80047e0 con 0x7fd0e4070cc0 2023-10-08T06:54:23.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.041+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4026949413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e41f34f0 con 0x7fd0e4070cc0 2023-10-08T06:54:23.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.042+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4026949413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e41f2b60 con 0x7fd0e4070cc0 2023-10-08T06:54:23.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.043+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4026949413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e404e160 con 0x7fd0e4070cc0 2023-10-08T06:54:23.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.044+0000 7fd0e17fa640 1 -- 172.21.15.70:0/4026949413 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd0d8004960 con 0x7fd0e4070cc0 2023-10-08T06:54:23.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.044+0000 7fd0e17fa640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd0c806af90 0x7fd0c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.045+0000 7fd0e17fa640 1 -- 172.21.15.70:0/4026949413 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fd0d8088a90 con 0x7fd0e4070cc0 2023-10-08T06:54:23.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.045+0000 7fd0e3fff640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd0c806af90 0x7fd0c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.046+0000 7fd0e3fff640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd0c806af90 0x7fd0c806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0036000 tx=0x7fd0d003c040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:23.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.049+0000 7fd0e17fa640 1 -- 172.21.15.70:0/4026949413 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0d80538c0 con 0x7fd0e4070cc0 2023-10-08T06:54:23.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.238+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4026949413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd0e4071170 con 0x7fd0e4070cc0 2023-10-08T06:54:23.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.246+0000 7fd0e17fa640 1 -- 172.21.15.70:0/4026949413 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v63) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd0e4071170 con 0x7fd0e4070cc0 2023-10-08T06:54:23.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd0c806af90 msgr2=0x7fd0c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd0c806af90 0x7fd0c806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0036000 tx=0x7fd0d003c040 comp rx=0 tx=0).stop 2023-10-08T06:54:23.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4026949413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 msgr2=0x7fd0e40c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 0x7fd0e40c5260 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800c780 tx=0x7fd0d800cc40 comp rx=0 tx=0).stop 2023-10-08T06:54:23.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4026949413 shutdown_connections 2023-10-08T06:54:23.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd0c806af90 0x7fd0c806d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0e4160660 0x7fd0e4155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0e40cb450 0x7fd0e41f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 --2- 172.21.15.70:0/4026949413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4070cc0 0x7fd0e40c5260 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4026949413 >> 172.21.15.70:0/4026949413 conn(0x7fd0e406a5d0 msgr2=0x7fd0e415f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:23.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.249+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4026949413 shutdown_connections 2023-10-08T06:54:23.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.250+0000 7fd0eaa7c640 1 -- 172.21.15.70:0/4026949413 wait complete. 2023-10-08T06:54:23.268 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-08T06:54:23.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.481+0000 7fd421c87640 1 Processor -- start 2023-10-08T06:54:23.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.481+0000 7fd421c87640 1 -- start start 2023-10-08T06:54:23.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd421c87640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0c90b0 0x7fd41c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd421c87640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd41c0cb3b0 0x7fd41c15e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd421c87640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41c15b2b0 con 0x7fd41c15ab40 2023-10-08T06:54:23.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd421c87640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41c15bf30 con 0x7fd41c0cb3b0 2023-10-08T06:54:23.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd421c87640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c0cb840 con 0x7fd41c0c90b0 2023-10-08T06:54:23.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd41bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd41c0cb3b0 0x7fd41c15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd41bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd41c0cb3b0 0x7fd41c15e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57838/0 (socket says 172.21.15.70:57838) 2023-10-08T06:54:23.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd41bfff640 1 -- 172.21.15.70:0/367987725 learned_addr learned my addr 172.21.15.70:0/367987725 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:23.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd41bfff640 1 -- 172.21.15.70:0/367987725 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0c90b0 msgr2=0x7fd41c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:23.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd413fff640 1 --2- 172.21.15.70:0/367987725 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0c90b0 0x7fd41c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd41bfff640 1 --2- 172.21.15.70:0/367987725 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0c90b0 0x7fd41c0cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd41bfff640 1 -- 172.21.15.70:0/367987725 >> v1:172.21.15.181:6789/0 conn(0x7fd41c15ab40 legacy=0x7fd41c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:23.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.483+0000 7fd41bfff640 1 -- 172.21.15.70:0/367987725 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c15b590 con 0x7fd41c0cb3b0 2023-10-08T06:54:23.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.484+0000 7fd413fff640 1 --2- 172.21.15.70:0/367987725 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0c90b0 0x7fd41c0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:23.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.485+0000 7fd41bfff640 1 --2- 172.21.15.70:0/367987725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0cb3b0 0x7fd41c15e790 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fd414009e50 tx=0x7fd41402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887d23747d5334b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:23.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.485+0000 7fd41affd640 1 -- 172.21.15.70:0/367987725 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41402ca70 con 0x7fd41c0cb3b0 2023-10-08T06:54:23.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.485+0000 7fd41affd640 1 -- 172.21.15.70:0/367987725 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41402cbd0 con 0x7fd41c0cb3b0 2023-10-08T06:54:23.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.485+0000 7fd41affd640 1 -- 172.21.15.70:0/367987725 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4140343f0 con 0x7fd41c0cb3b0 2023-10-08T06:54:23.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.486+0000 7fd421c87640 1 -- 172.21.15.70:0/367987725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0cb3b0 msgr2=0x7fd41c15e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.486+0000 7fd421c87640 1 --2- 172.21.15.70:0/367987725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0cb3b0 0x7fd41c15e790 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fd414009e50 tx=0x7fd41402bed0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.486+0000 7fd421c87640 1 -- 172.21.15.70:0/367987725 shutdown_connections 2023-10-08T06:54:23.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.486+0000 7fd421c87640 1 --2- 172.21.15.70:0/367987725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0cb3b0 0x7fd41c15e790 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.486+0000 7fd421c87640 1 --2- 172.21.15.70:0/367987725 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0c90b0 0x7fd41c0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.486+0000 7fd421c87640 1 -- 172.21.15.70:0/367987725 >> 172.21.15.70:0/367987725 conn(0x7fd41c06a380 msgr2=0x7fd41c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:23.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.486+0000 7fd421c87640 1 -- 172.21.15.70:0/367987725 shutdown_connections 2023-10-08T06:54:23.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.486+0000 7fd421c87640 1 -- 172.21.15.70:0/367987725 wait complete. 2023-10-08T06:54:23.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.488+0000 7fd421c87640 1 Processor -- start 2023-10-08T06:54:23.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.488+0000 7fd421c87640 1 -- start start 2023-10-08T06:54:23.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd421c87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0c90b0 0x7fd41c151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd421c87640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0cb3b0 0x7fd41c152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd421c87640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41c15ab40 0x7fd41c151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd421c87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c152330 con 0x7fd41c0c90b0 2023-10-08T06:54:23.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd421c87640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c159420 con 0x7fd41c15ab40 2023-10-08T06:54:23.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd421c87640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41c159700 con 0x7fd41c0cb3b0 2023-10-08T06:54:23.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd41b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0c90b0 0x7fd41c151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd41b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0c90b0 0x7fd41c151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57846/0 (socket says 172.21.15.70:57846) 2023-10-08T06:54:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd41b7fe640 1 -- 172.21.15.70:0/2150128587 learned_addr learned my addr 172.21.15.70:0/2150128587 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd413fff640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0cb3b0 0x7fd41c152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd41bfff640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41c15ab40 0x7fd41c151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd41b7fe640 1 -- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0cb3b0 msgr2=0x7fd41c152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.489+0000 7fd41b7fe640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0cb3b0 0x7fd41c152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.490+0000 7fd41b7fe640 1 -- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41c15ab40 msgr2=0x7fd41c151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.490+0000 7fd41b7fe640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41c15ab40 0x7fd41c151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.490+0000 7fd41b7fe640 1 -- 172.21.15.70:0/2150128587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd414009b30 con 0x7fd41c0c90b0 2023-10-08T06:54:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.490+0000 7fd41bfff640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41c15ab40 0x7fd41c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.490+0000 7fd41b7fe640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0c90b0 0x7fd41c151ca0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00c430 tx=0x7fd40c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.490+0000 7fd418ff9640 1 -- 172.21.15.70:0/2150128587 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c011790 con 0x7fd41c0c90b0 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.490+0000 7fd418ff9640 1 -- 172.21.15.70:0/2150128587 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c0118f0 con 0x7fd41c0c90b0 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.490+0000 7fd418ff9640 1 -- 172.21.15.70:0/2150128587 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c011ce0 con 0x7fd41c0c90b0 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.490+0000 7fd421c87640 1 -- 172.21.15.70:0/2150128587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c1599e0 con 0x7fd41c0c90b0 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.490+0000 7fd421c87640 1 -- 172.21.15.70:0/2150128587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41c152cd0 con 0x7fd41c0c90b0 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.491+0000 7fd421c87640 1 -- 172.21.15.70:0/2150128587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41c159f00 con 0x7fd41c0c90b0 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.492+0000 7fd418ff9640 1 -- 172.21.15.70:0/2150128587 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd40c01c460 con 0x7fd41c0c90b0 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.493+0000 7fd418ff9640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd3fc06af90 0x7fd3fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.493+0000 7fd418ff9640 1 -- 172.21.15.70:0/2150128587 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fd40c089880 con 0x7fd41c0c90b0 2023-10-08T06:54:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.494+0000 7fd413fff640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd3fc06af90 0x7fd3fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:23.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.495+0000 7fd413fff640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd3fc06af90 0x7fd3fc06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0ad330 tx=0x7fd404008040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:23.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.498+0000 7fd418ff9640 1 -- 172.21.15.70:0/2150128587 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd40c0546a0 con 0x7fd41c0c90b0 2023-10-08T06:54:23.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.756+0000 7fd421c87640 1 -- 172.21.15.70:0/2150128587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7fd41c067490 con 0x7fd41c0c90b0 2023-10-08T06:54:23.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.930+0000 7fd418ff9640 1 -- 172.21.15.70:0/2150128587 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2586) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd40c057cc0 con 0x7fd41c0c90b0 2023-10-08T06:54:23.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 -- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd3fc06af90 msgr2=0x7fd3fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd3fc06af90 0x7fd3fc06d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0ad330 tx=0x7fd404008040 comp rx=0 tx=0).stop 2023-10-08T06:54:23.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 -- 172.21.15.70:0/2150128587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0c90b0 msgr2=0x7fd41c151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:23.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0c90b0 0x7fd41c151ca0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00c430 tx=0x7fd40c00c8f0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 -- 172.21.15.70:0/2150128587 shutdown_connections 2023-10-08T06:54:23.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fd3fc06af90 0x7fd3fc06d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd41c15ab40 0x7fd41c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd41c0cb3b0 0x7fd41c152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 --2- 172.21.15.70:0/2150128587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd41c0c90b0 0x7fd41c151ca0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:23.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 -- 172.21.15.70:0/2150128587 >> 172.21.15.70:0/2150128587 conn(0x7fd41c06a380 msgr2=0x7fd41c0afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:23.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 -- 172.21.15.70:0/2150128587 shutdown_connections 2023-10-08T06:54:23.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:23.933+0000 7fd421c87640 1 -- 172.21.15.70:0/2150128587 wait complete. 2023-10-08T06:54:23.955 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-10-08T06:54:23.956 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-10-08T06:54:23.956 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T06:54:23.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:54:24.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.170+0000 7fc6565b8640 1 Processor -- start 2023-10-08T06:54:24.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.170+0000 7fc6565b8640 1 -- start start 2023-10-08T06:54:24.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.171+0000 7fc6565b8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc6500adc70 0x7fc6500b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.171+0000 7fc6565b8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65015f7d0 con 0x7fc65015af10 2023-10-08T06:54:24.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.171+0000 7fc6565b8640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65015efb0 con 0x7fc65015ab30 2023-10-08T06:54:24.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.171+0000 7fc6565b8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65015f210 con 0x7fc6500adc70 2023-10-08T06:54:24.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.172+0000 7fc654db5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc6500adc70 0x7fc6500b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.172+0000 7fc654db5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc6500adc70 0x7fc6500b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55938/0 (socket says 172.21.15.70:55938) 2023-10-08T06:54:24.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.172+0000 7fc654db5640 1 -- 172.21.15.70:0/4277006037 learned_addr learned my addr 172.21.15.70:0/4277006037 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:24.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.172+0000 7fc655db7640 1 --1- 172.21.15.70:0/4277006037 >> v1:172.21.15.70:6789/0 conn(0x7fc65015af10 0x7fc65006cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55476/0 (socket says 172.21.15.70:55476) 2023-10-08T06:54:24.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.172+0000 7fc654db5640 1 -- 172.21.15.70:0/4277006037 >> v1:172.21.15.181:6789/0 conn(0x7fc65015ab30 legacy=0x7fc65015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:24.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.172+0000 7fc654db5640 1 -- 172.21.15.70:0/4277006037 >> v1:172.21.15.70:6789/0 conn(0x7fc65015af10 legacy=0x7fc65006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.172+0000 7fc654db5640 1 -- 172.21.15.70:0/4277006037 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65015f4f0 con 0x7fc6500adc70 2023-10-08T06:54:24.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.173+0000 7fc654db5640 1 --2- 172.21.15.70:0/4277006037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6500adc70 0x7fc6500b00c0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc64002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c3232d9b6a33aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:24.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.173+0000 7fc647fff640 1 -- 172.21.15.70:0/4277006037 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640030860 con 0x7fc6500adc70 2023-10-08T06:54:24.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.174+0000 7fc647fff640 1 -- 172.21.15.70:0/4277006037 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6400309c0 con 0x7fc6500adc70 2023-10-08T06:54:24.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.174+0000 7fc647fff640 1 -- 172.21.15.70:0/4277006037 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640030d50 con 0x7fc6500adc70 2023-10-08T06:54:24.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.174+0000 7fc6565b8640 1 -- 172.21.15.70:0/4277006037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6500adc70 msgr2=0x7fc6500b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.174+0000 7fc6565b8640 1 --2- 172.21.15.70:0/4277006037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6500adc70 0x7fc6500b00c0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc640009870 tx=0x7fc64002b9a0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.174+0000 7fc6565b8640 1 -- 172.21.15.70:0/4277006037 shutdown_connections 2023-10-08T06:54:24.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.174+0000 7fc6565b8640 1 --2- 172.21.15.70:0/4277006037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6500adc70 0x7fc6500b00c0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.174+0000 7fc6565b8640 1 -- 172.21.15.70:0/4277006037 >> 172.21.15.70:0/4277006037 conn(0x7fc6500cb0c0 msgr2=0x7fc6500cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:24.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.174+0000 7fc6565b8640 1 -- 172.21.15.70:0/4277006037 shutdown_connections 2023-10-08T06:54:24.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.174+0000 7fc6565b8640 1 -- 172.21.15.70:0/4277006037 wait complete. 2023-10-08T06:54:24.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.176+0000 7fc6565b8640 1 Processor -- start 2023-10-08T06:54:24.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.176+0000 7fc6565b8640 1 -- start start 2023-10-08T06:54:24.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.177+0000 7fc6565b8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6500adc70 0x7fc65015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.177+0000 7fc6565b8640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65015ab30 0x7fc6500c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.177+0000 7fc6565b8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65015af10 0x7fc6500c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.177+0000 7fc6565b8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500cd0e0 con 0x7fc65015af10 2023-10-08T06:54:24.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.177+0000 7fc6565b8640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500cc4d0 con 0x7fc65015ab30 2023-10-08T06:54:24.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.177+0000 7fc6565b8640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500cc7b0 con 0x7fc6500adc70 2023-10-08T06:54:24.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.177+0000 7fc655db7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65015af10 0x7fc6500c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.177+0000 7fc655db7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65015af10 0x7fc6500c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55940/0 (socket says 172.21.15.70:55940) 2023-10-08T06:54:24.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.177+0000 7fc655db7640 1 -- 172.21.15.70:0/1181595627 learned_addr learned my addr 172.21.15.70:0/1181595627 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:24.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc654db5640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65015ab30 0x7fc6500c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc655db7640 1 -- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6500adc70 msgr2=0x7fc65015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc6555b6640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6500adc70 0x7fc65015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc655db7640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6500adc70 0x7fc65015b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc655db7640 1 -- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65015ab30 msgr2=0x7fc6500c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc655db7640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65015ab30 0x7fc6500c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc655db7640 1 -- 172.21.15.70:0/1181595627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6400094d0 con 0x7fc65015af10 2023-10-08T06:54:24.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc654db5640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65015ab30 0x7fc6500c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:24.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc655db7640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65015af10 0x7fc6500c6270 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fc64800c6b0 tx=0x7fc64800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:24.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc645ffb640 1 -- 172.21.15.70:0/1181595627 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc648004d60 con 0x7fc65015af10 2023-10-08T06:54:24.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc645ffb640 1 -- 172.21.15.70:0/1181595627 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64800a640 con 0x7fc65015af10 2023-10-08T06:54:24.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc6565b8640 1 -- 172.21.15.70:0/1181595627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6500cca90 con 0x7fc65015af10 2023-10-08T06:54:24.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc645ffb640 1 -- 172.21.15.70:0/1181595627 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64800a9f0 con 0x7fc65015af10 2023-10-08T06:54:24.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.178+0000 7fc6565b8640 1 -- 172.21.15.70:0/1181595627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6500c8460 con 0x7fc65015af10 2023-10-08T06:54:24.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.180+0000 7fc6565b8640 1 -- 172.21.15.70:0/1181595627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6500cd250 con 0x7fc65015af10 2023-10-08T06:54:24.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.180+0000 7fc645ffb640 1 -- 172.21.15.70:0/1181595627 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc64800ab50 con 0x7fc65015af10 2023-10-08T06:54:24.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.181+0000 7fc645ffb640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fc63406af90 0x7fc63406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.181+0000 7fc645ffb640 1 -- 172.21.15.70:0/1181595627 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fc648089080 con 0x7fc65015af10 2023-10-08T06:54:24.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.182+0000 7fc6555b6640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fc63406af90 0x7fc63406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.183+0000 7fc6555b6640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fc63406af90 0x7fc63406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc64c003d90 tx=0x7fc64c009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:24.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.186+0000 7fc645ffb640 1 -- 172.21.15.70:0/1181595627 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc648053eb0 con 0x7fc65015af10 2023-10-08T06:54:24.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.377+0000 7fc6565b8640 1 -- 172.21.15.70:0/1181595627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc65004e160 con 0x7fc65015af10 2023-10-08T06:54:24.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.379+0000 7fc645ffb640 1 -- 172.21.15.70:0/1181595627 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+14394 (secure 0 0 0) 0x7fc6480574d0 con 0x7fc65015af10 2023-10-08T06:54:24.380 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:24.380 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":190,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:54:21.904369+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6834/3790014600":"2023-10-09T06:54:18.731985+0000","172.21.15.70:6832/3790014600":"2023-10-09T06:54:18.731985+0000","172.21.15.181:0/2596125885":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/1985247114":"2023-10-09T06:54:14.934383+0000","172.21.15.181:6833/1385757024":"2023-10-09T06:54:14.934383+0000","172.21.15.181:6832/1385757024":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/2383107395":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/1278239547":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/3845072443":"2023-10-09T06:54:14.934383+0000","172.21.15.181:6833/4245513204":"2023-10-09T06:54:07.074878+0000","172.21.15.181:6832/4245513204":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/1356319692":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/4081249518":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2711042436":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2536889452":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/2861165849":"2023-10-09T06:54:07.074878+0000","172.21.15.181:0/998820505":"2023-10-09T06:54:14.934383+0000","172.21.15.181:0/4176429568":"2023-10-09T06:54:07.074878+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:54:24.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.381+0000 7fc6565b8640 1 -- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fc63406af90 msgr2=0x7fc63406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.381+0000 7fc6565b8640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fc63406af90 0x7fc63406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc64c003d90 tx=0x7fc64c009040 comp rx=0 tx=0).stop 2023-10-08T06:54:24.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.382+0000 7fc6565b8640 1 -- 172.21.15.70:0/1181595627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65015af10 msgr2=0x7fc6500c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.382+0000 7fc6565b8640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65015af10 0x7fc6500c6270 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fc64800c6b0 tx=0x7fc64800cb70 comp rx=0 tx=0).stop 2023-10-08T06:54:24.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.382+0000 7fc6565b8640 1 -- 172.21.15.70:0/1181595627 shutdown_connections 2023-10-08T06:54:24.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.382+0000 7fc6565b8640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fc63406af90 0x7fc63406d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.382+0000 7fc6565b8640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc65015af10 0x7fc6500c6270 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.382+0000 7fc6565b8640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc65015ab30 0x7fc6500c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.382+0000 7fc6565b8640 1 --2- 172.21.15.70:0/1181595627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc6500adc70 0x7fc65015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.382+0000 7fc6565b8640 1 -- 172.21.15.70:0/1181595627 >> 172.21.15.70:0/1181595627 conn(0x7fc6500cb0c0 msgr2=0x7fc650070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:24.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.382+0000 7fc6565b8640 1 -- 172.21.15.70:0/1181595627 shutdown_connections 2023-10-08T06:54:24.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.382+0000 7fc6565b8640 1 -- 172.21.15.70:0/1181595627 wait complete. 2023-10-08T06:54:24.402 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T06:54:24.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T06:54:24.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.610+0000 7f15acd87640 1 Processor -- start 2023-10-08T06:54:24.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.610+0000 7f15acd87640 1 -- start start 2023-10-08T06:54:24.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.611+0000 7f15acd87640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a815aab0 0x7f15a8070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.611+0000 7f15acd87640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15a80caef0 0x7f15a80aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.611+0000 7f15acd87640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f15a80cb350 0x7f15a80b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.611+0000 7f15acd87640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a815ae90 con 0x7f15a80cb350 2023-10-08T06:54:24.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.611+0000 7f15acd87640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80cbdb0 con 0x7f15a80caef0 2023-10-08T06:54:24.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.611+0000 7f15acd87640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80cb7f0 con 0x7f15a815aab0 2023-10-08T06:54:24.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f15a7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f15a80cb350 0x7f15a80b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f15a7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f15a80cb350 0x7f15a80b04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55942/0 (socket says 172.21.15.70:55942) 2023-10-08T06:54:24.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f15a7fff640 1 -- 172.21.15.70:0/673564359 learned_addr learned my addr 172.21.15.70:0/673564359 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:24.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f159ffff640 1 --2- 172.21.15.70:0/673564359 >> v2:172.21.15.181:3300/0 conn(0x7f15a80caef0 0x7f15a80aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f15a7fff640 1 -- 172.21.15.70:0/673564359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a815aab0 msgr2=0x7f15a8070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:24.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f15a77fe640 1 --2- 172.21.15.70:0/673564359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a815aab0 0x7f15a8070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f15a7fff640 1 --2- 172.21.15.70:0/673564359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a815aab0 0x7f15a8070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f15a7fff640 1 -- 172.21.15.70:0/673564359 >> v2:172.21.15.181:3300/0 conn(0x7f15a80caef0 msgr2=0x7f15a80aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f15a7fff640 1 --2- 172.21.15.70:0/673564359 >> v2:172.21.15.181:3300/0 conn(0x7f15a80caef0 0x7f15a80aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f15a7fff640 1 -- 172.21.15.70:0/673564359 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80cbad0 con 0x7f15a80cb350 2023-10-08T06:54:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f159ffff640 1 --2- 172.21.15.70:0/673564359 >> v2:172.21.15.181:3300/0 conn(0x7f15a80caef0 0x7f15a80aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.612+0000 7f15a77fe640 1 --2- 172.21.15.70:0/673564359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a815aab0 0x7f15a8070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.613+0000 7f15a7fff640 1 --2- 172.21.15.70:0/673564359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80cb350 0x7f15a80b04a0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f15a806e530 tx=0x7f15a002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60aad48edc280817 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15a6ffd640 1 -- 172.21.15.70:0/673564359 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a002c9f0 con 0x7f15a80cb350 2023-10-08T06:54:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15a6ffd640 1 -- 172.21.15.70:0/673564359 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a002cb50 con 0x7f15a80cb350 2023-10-08T06:54:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15a6ffd640 1 -- 172.21.15.70:0/673564359 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a00343f0 con 0x7f15a80cb350 2023-10-08T06:54:24.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15acd87640 1 -- 172.21.15.70:0/673564359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80cb350 msgr2=0x7f15a80b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15acd87640 1 --2- 172.21.15.70:0/673564359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80cb350 0x7f15a80b04a0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f15a806e530 tx=0x7f15a002be50 comp rx=0 tx=0).stop 2023-10-08T06:54:24.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15acd87640 1 -- 172.21.15.70:0/673564359 shutdown_connections 2023-10-08T06:54:24.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15acd87640 1 --2- 172.21.15.70:0/673564359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80cb350 0x7f15a80b04a0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15acd87640 1 --2- 172.21.15.70:0/673564359 >> v2:172.21.15.181:3300/0 conn(0x7f15a80caef0 0x7f15a80aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15acd87640 1 --2- 172.21.15.70:0/673564359 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a815aab0 0x7f15a8070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15acd87640 1 -- 172.21.15.70:0/673564359 >> 172.21.15.70:0/673564359 conn(0x7f15a806c010 msgr2=0x7f15a806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:24.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.614+0000 7f15acd87640 1 -- 172.21.15.70:0/673564359 shutdown_connections 2023-10-08T06:54:24.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.615+0000 7f15acd87640 1 -- 172.21.15.70:0/673564359 wait complete. 2023-10-08T06:54:24.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.616+0000 7f15acd87640 1 Processor -- start 2023-10-08T06:54:24.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.616+0000 7f15acd87640 1 -- start start 2023-10-08T06:54:24.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.617+0000 7f15acd87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80caef0 0x7f15a80add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.617+0000 7f15acd87640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a80cb350 0x7f15a814f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.617+0000 7f15acd87640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a815aab0 0x7f15a8157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.617+0000 7f15acd87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a81536e0 con 0x7f15a80caef0 2023-10-08T06:54:24.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.617+0000 7f15acd87640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8153120 con 0x7f15a80cb350 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.617+0000 7f15acd87640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8153400 con 0x7f15a815aab0 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.617+0000 7f15a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80caef0 0x7f15a80add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.617+0000 7f15a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80caef0 0x7f15a80add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55946/0 (socket says 172.21.15.70:55946) 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.617+0000 7f15a77fe640 1 -- 172.21.15.70:0/359072584 learned_addr learned my addr 172.21.15.70:0/359072584 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f159ffff640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a80cb350 0x7f15a814f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f15a77fe640 1 -- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a815aab0 msgr2=0x7f15a8157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f15a77fe640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a815aab0 0x7f15a8157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f15a77fe640 1 -- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a80cb350 msgr2=0x7f15a814f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f15a77fe640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a80cb350 0x7f15a814f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f15a77fe640 1 -- 172.21.15.70:0/359072584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0009b30 con 0x7f15a80caef0 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f159ffff640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a80cb350 0x7f15a814f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:24.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f15a77fe640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80caef0 0x7f15a80add20 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f159800c7f0 tx=0x7f159800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:24.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f15a4ff9640 1 -- 172.21.15.70:0/359072584 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598007820 con 0x7f15a80caef0 2023-10-08T06:54:24.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f15a4ff9640 1 -- 172.21.15.70:0/359072584 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1598007980 con 0x7f15a80caef0 2023-10-08T06:54:24.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f15acd87640 1 -- 172.21.15.70:0/359072584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a81514f0 con 0x7f15a80caef0 2023-10-08T06:54:24.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.618+0000 7f15a4ff9640 1 -- 172.21.15.70:0/359072584 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598007cf0 con 0x7f15a80caef0 2023-10-08T06:54:24.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.619+0000 7f15acd87640 1 -- 172.21.15.70:0/359072584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a814ffb0 con 0x7f15a80caef0 2023-10-08T06:54:24.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.620+0000 7f15acd87640 1 -- 172.21.15.70:0/359072584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a80c8fc0 con 0x7f15a80caef0 2023-10-08T06:54:24.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.621+0000 7f15a4ff9640 1 -- 172.21.15.70:0/359072584 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f15980113f0 con 0x7f15a80caef0 2023-10-08T06:54:24.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.622+0000 7f15a4ff9640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f158806af90 0x7f158806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:24.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.622+0000 7f15a4ff9640 1 -- 172.21.15.70:0/359072584 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f1598089de0 con 0x7f15a80caef0 2023-10-08T06:54:24.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.623+0000 7f159ffff640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f158806af90 0x7f158806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:24.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.624+0000 7f159ffff640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f158806af90 0x7f158806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15a81f3480 tx=0x7f1590008040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:24.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.626+0000 7f15a4ff9640 1 -- 172.21.15.70:0/359072584 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1598054a70 con 0x7f15a80caef0 2023-10-08T06:54:24.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.814+0000 7f15acd87640 1 -- 172.21.15.70:0/359072584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15a8074150 con 0x7f15a80caef0 2023-10-08T06:54:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.816+0000 7f15a4ff9640 1 -- 172.21.15.70:0/359072584 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) v1 ==== 93+0+10 (secure 0 0 0) 0x7f159801f020 con 0x7f15a80caef0 2023-10-08T06:54:24.816 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T06:54:24.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.818+0000 7f15acd87640 1 -- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f158806af90 msgr2=0x7f158806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.818+0000 7f15acd87640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f158806af90 0x7f158806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15a81f3480 tx=0x7f1590008040 comp rx=0 tx=0).stop 2023-10-08T06:54:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.818+0000 7f15acd87640 1 -- 172.21.15.70:0/359072584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80caef0 msgr2=0x7f15a80add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.818+0000 7f15acd87640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80caef0 0x7f15a80add20 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f159800c7f0 tx=0x7f159800ccb0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.819+0000 7f15acd87640 1 -- 172.21.15.70:0/359072584 shutdown_connections 2023-10-08T06:54:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.819+0000 7f15acd87640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f158806af90 0x7f158806d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.819+0000 7f15acd87640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15a815aab0 0x7f15a8157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.819+0000 7f15acd87640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f15a80cb350 0x7f15a814f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.819+0000 7f15acd87640 1 --2- 172.21.15.70:0/359072584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15a80caef0 0x7f15a80add20 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.819+0000 7f15acd87640 1 -- 172.21.15.70:0/359072584 >> 172.21.15.70:0/359072584 conn(0x7f15a806c010 msgr2=0x7f15a815d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.819+0000 7f15acd87640 1 -- 172.21.15.70:0/359072584 shutdown_connections 2023-10-08T06:54:24.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:24.819+0000 7f15acd87640 1 -- 172.21.15.70:0/359072584 wait complete. 2023-10-08T06:54:24.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-08T06:54:25.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.044+0000 7fb5ee3cc640 1 Processor -- start 2023-10-08T06:54:25.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.044+0000 7fb5ee3cc640 1 -- start start 2023-10-08T06:54:25.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.046+0000 7fb5ee3cc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb5e8157000 0x7fb5e80cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:25.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.046+0000 7fb5ee3cc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb5e81597d0 0x7fb5e806ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:25.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.046+0000 7fb5ee3cc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e80adfc0 con 0x7fb5e80cb190 2023-10-08T06:54:25.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.046+0000 7fb5ee3cc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e80ccce0 con 0x7fb5e8157000 2023-10-08T06:54:25.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.046+0000 7fb5ee3cc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e806d380 con 0x7fb5e81597d0 2023-10-08T06:54:25.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.046+0000 7fb5e7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb5e8157000 0x7fb5e80cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:25.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.046+0000 7fb5ec969640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb5e80cb190 0x7fb5e815d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55492/0 (socket says 172.21.15.70:55492) 2023-10-08T06:54:25.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.046+0000 7fb5ec969640 1 -- 172.21.15.70:0/1267644061 learned_addr learned my addr 172.21.15.70:0/1267644061 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:25.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.046+0000 7fb5e7fff640 1 -- 172.21.15.70:0/1267644061 >> v2:172.21.15.181:3300/0 conn(0x7fb5e81597d0 msgr2=0x7fb5e806ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:25.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.047+0000 7fb5e7fff640 1 --2- 172.21.15.70:0/1267644061 >> v2:172.21.15.181:3300/0 conn(0x7fb5e81597d0 0x7fb5e806ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.047+0000 7fb5e7fff640 1 -- 172.21.15.70:0/1267644061 >> v1:172.21.15.70:6789/0 conn(0x7fb5e80cb190 legacy=0x7fb5e815d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:25.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.047+0000 7fb5e7fff640 1 -- 172.21.15.70:0/1267644061 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8159cb0 con 0x7fb5e8157000 2023-10-08T06:54:25.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.047+0000 7fb5e77fe640 1 -- 172.21.15.70:0/1267644061 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348050788 0 0) 0x7fb5e80adfc0 con 0x7fb5e80cb190 2023-10-08T06:54:25.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.048+0000 7fb5e7fff640 1 --2- 172.21.15.70:0/1267644061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e8157000 0x7fb5e80cc7a0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8071770 tx=0x7fb5d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93f4ca7d089a04e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:25.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.048+0000 7fb5e77fe640 1 -- 172.21.15.70:0/1267644061 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8031830 con 0x7fb5e8157000 2023-10-08T06:54:25.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.048+0000 7fb5e77fe640 1 -- 172.21.15.70:0/1267644061 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d8031990 con 0x7fb5e8157000 2023-10-08T06:54:25.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.048+0000 7fb5e77fe640 1 -- 172.21.15.70:0/1267644061 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8031d20 con 0x7fb5e8157000 2023-10-08T06:54:25.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.048+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/1267644061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e8157000 msgr2=0x7fb5e80cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:25.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.048+0000 7fb5ee3cc640 1 --2- 172.21.15.70:0/1267644061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e8157000 0x7fb5e80cc7a0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8071770 tx=0x7fb5d802b8d0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.049+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/1267644061 shutdown_connections 2023-10-08T06:54:25.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.049+0000 7fb5ee3cc640 1 --2- 172.21.15.70:0/1267644061 >> v2:172.21.15.181:3300/0 conn(0x7fb5e81597d0 0x7fb5e806ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.049+0000 7fb5ee3cc640 1 --2- 172.21.15.70:0/1267644061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e8157000 0x7fb5e80cc7a0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.049+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/1267644061 >> 172.21.15.70:0/1267644061 conn(0x7fb5e806a380 msgr2=0x7fb5e80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:25.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.049+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/1267644061 shutdown_connections 2023-10-08T06:54:25.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.049+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/1267644061 wait complete. 2023-10-08T06:54:25.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.050+0000 7fb5ee3cc640 1 Processor -- start 2023-10-08T06:54:25.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.051+0000 7fb5ee3cc640 1 -- start start 2023-10-08T06:54:25.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.051+0000 7fb5ee3cc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e80cb190 0x7fb5e8143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:25.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.051+0000 7fb5ee3cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e81597d0 0x7fb5e8144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:25.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5ee3cc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5e8144650 0x7fb5e80c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:25.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5ee3cc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80b19d0 con 0x7fb5e81597d0 2023-10-08T06:54:25.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5ee3cc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8157fa0 con 0x7fb5e80cb190 2023-10-08T06:54:25.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5ee3cc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8158240 con 0x7fb5e8144650 2023-10-08T06:54:25.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5df7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e81597d0 0x7fb5e8144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:25.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5df7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e81597d0 0x7fb5e8144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55960/0 (socket says 172.21.15.70:55960) 2023-10-08T06:54:25.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5df7fe640 1 -- 172.21.15.70:0/2867704243 learned_addr learned my addr 172.21.15.70:0/2867704243 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:25.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5df7fe640 1 -- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5e8144650 msgr2=0x7fb5e80c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:25.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5ec969640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5e8144650 0x7fb5e80c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:25.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5e7fff640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e80cb190 0x7fb5e8143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:25.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5df7fe640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5e8144650 0x7fb5e80c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5df7fe640 1 -- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e80cb190 msgr2=0x7fb5e8143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:25.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5df7fe640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e80cb190 0x7fb5e8143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5df7fe640 1 -- 172.21.15.70:0/2867704243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8009400 con 0x7fb5e81597d0 2023-10-08T06:54:25.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.052+0000 7fb5ec969640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5e8144650 0x7fb5e80c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:25.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.053+0000 7fb5df7fe640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e81597d0 0x7fb5e8144110 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000b410 tx=0x7fb5d000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:25.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.053+0000 7fb5e57fa640 1 -- 172.21.15.70:0/2867704243 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0011830 con 0x7fb5e81597d0 2023-10-08T06:54:25.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.053+0000 7fb5e57fa640 1 -- 172.21.15.70:0/2867704243 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d00119b0 con 0x7fb5e81597d0 2023-10-08T06:54:25.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.053+0000 7fb5e57fa640 1 -- 172.21.15.70:0/2867704243 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0011d60 con 0x7fb5e81597d0 2023-10-08T06:54:25.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.053+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/2867704243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e8158520 con 0x7fb5e81597d0 2023-10-08T06:54:25.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.053+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/2867704243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e815ac50 con 0x7fb5e81597d0 2023-10-08T06:54:25.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.054+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/2867704243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e815ae70 con 0x7fb5e81597d0 2023-10-08T06:54:25.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.055+0000 7fb5e57fa640 1 -- 172.21.15.70:0/2867704243 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb5d001a400 con 0x7fb5e81597d0 2023-10-08T06:54:25.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.056+0000 7fb5e57fa640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb5c806af90 0x7fb5c806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:25.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.056+0000 7fb5e57fa640 1 -- 172.21.15.70:0/2867704243 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fb5d0089a20 con 0x7fb5e81597d0 2023-10-08T06:54:25.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.057+0000 7fb5e7fff640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb5c806af90 0x7fb5c806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:25.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.058+0000 7fb5e7fff640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb5c806af90 0x7fb5c806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8036000 tx=0x7fb5d8008030 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:25.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.060+0000 7fb5e57fa640 1 -- 172.21.15.70:0/2867704243 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5d00548e0 con 0x7fb5e81597d0 2023-10-08T06:54:25.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.317+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/2867704243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fb5e815a240 con 0x7fb5e81597d0 2023-10-08T06:54:25.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.942+0000 7fb5e57fa640 1 -- 172.21.15.70:0/2867704243 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2588) v1 ==== 168+0+0 (secure 0 0 0) 0x7fb5d00150e0 con 0x7fb5e81597d0 2023-10-08T06:54:25.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.945+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb5c806af90 msgr2=0x7fb5c806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:25.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.945+0000 7fb5ee3cc640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb5c806af90 0x7fb5c806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8036000 tx=0x7fb5d8008030 comp rx=0 tx=0).stop 2023-10-08T06:54:25.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.945+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/2867704243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e81597d0 msgr2=0x7fb5e8144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:25.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.945+0000 7fb5ee3cc640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e81597d0 0x7fb5e8144110 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000b410 tx=0x7fb5d000b8d0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.946+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/2867704243 shutdown_connections 2023-10-08T06:54:25.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.946+0000 7fb5ee3cc640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb5c806af90 0x7fb5c806d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.946+0000 7fb5ee3cc640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5e8144650 0x7fb5e80c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.946+0000 7fb5ee3cc640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5e81597d0 0x7fb5e8144110 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.946+0000 7fb5ee3cc640 1 --2- 172.21.15.70:0/2867704243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5e80cb190 0x7fb5e8143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.946+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/2867704243 >> 172.21.15.70:0/2867704243 conn(0x7fb5e806a380 msgr2=0x7fb5e80cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.946+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/2867704243 shutdown_connections 2023-10-08T06:54:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:25.946+0000 7fb5ee3cc640 1 -- 172.21.15.70:0/2867704243 wait complete. 2023-10-08T06:54:25.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T06:54:26.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.175+0000 7f1baddf0640 1 Processor -- start 2023-10-08T06:54:26.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.175+0000 7f1baddf0640 1 -- start start 2023-10-08T06:54:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.176+0000 7f1baddf0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ba80afbe0 0x7f1ba80b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.176+0000 7f1baddf0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1ba80cb130 0x7f1ba80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.176+0000 7f1baddf0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba80b2620 con 0x7f1ba806dfc0 2023-10-08T06:54:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.176+0000 7f1baddf0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba80cbae0 con 0x7f1ba80cb130 2023-10-08T06:54:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.176+0000 7f1baddf0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba80b2060 con 0x7f1ba80afbe0 2023-10-08T06:54:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1ba7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1ba80cb130 0x7f1ba80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1b9ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1ba806dfc0 0x7f1ba8070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55494/0 (socket says 172.21.15.70:55494) 2023-10-08T06:54:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1b9ffff640 1 -- 172.21.15.70:0/1609882252 learned_addr learned my addr 172.21.15.70:0/1609882252 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1ba77fe640 1 --2- 172.21.15.70:0/1609882252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ba80afbe0 0x7f1ba80b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1ba6ffd640 1 -- 172.21.15.70:0/1609882252 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927673395 0 0) 0x7f1ba80b2620 con 0x7f1ba806dfc0 2023-10-08T06:54:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1ba6ffd640 1 -- 172.21.15.70:0/1609882252 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b94003520 con 0x7f1ba806dfc0 2023-10-08T06:54:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1ba7fff640 1 -- 172.21.15.70:0/1609882252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ba80afbe0 msgr2=0x7f1ba80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1ba7fff640 1 --2- 172.21.15.70:0/1609882252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ba80afbe0 0x7f1ba80b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1ba7fff640 1 -- 172.21.15.70:0/1609882252 >> v1:172.21.15.70:6789/0 conn(0x7f1ba806dfc0 legacy=0x7f1ba8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1ba7fff640 1 -- 172.21.15.70:0/1609882252 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba80b2340 con 0x7f1ba80cb130 2023-10-08T06:54:26.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.177+0000 7f1ba77fe640 1 --2- 172.21.15.70:0/1609882252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ba80afbe0 0x7f1ba80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:26.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.178+0000 7f1ba7fff640 1 --2- 172.21.15.70:0/1609882252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80cb130 0x7f1ba80cb5a0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009d90 tx=0x7f1ba002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa5cffd5f5f36ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:26.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.179+0000 7f1ba6ffd640 1 -- 172.21.15.70:0/1609882252 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba002cc50 con 0x7f1ba80cb130 2023-10-08T06:54:26.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.179+0000 7f1ba6ffd640 1 -- 172.21.15.70:0/1609882252 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba002cdb0 con 0x7f1ba80cb130 2023-10-08T06:54:26.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.179+0000 7f1ba6ffd640 1 -- 172.21.15.70:0/1609882252 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0034640 con 0x7f1ba80cb130 2023-10-08T06:54:26.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.179+0000 7f1baddf0640 1 -- 172.21.15.70:0/1609882252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80cb130 msgr2=0x7f1ba80cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:26.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.179+0000 7f1baddf0640 1 --2- 172.21.15.70:0/1609882252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80cb130 0x7f1ba80cb5a0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0009d90 tx=0x7f1ba002bea0 comp rx=0 tx=0).stop 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.179+0000 7f1baddf0640 1 -- 172.21.15.70:0/1609882252 shutdown_connections 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.179+0000 7f1baddf0640 1 --2- 172.21.15.70:0/1609882252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80cb130 0x7f1ba80cb5a0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.179+0000 7f1baddf0640 1 --2- 172.21.15.70:0/1609882252 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ba80afbe0 0x7f1ba80b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.179+0000 7f1baddf0640 1 -- 172.21.15.70:0/1609882252 >> 172.21.15.70:0/1609882252 conn(0x7f1ba806a5d0 msgr2=0x7f1ba80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.180+0000 7f1baddf0640 1 -- 172.21.15.70:0/1609882252 shutdown_connections 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.180+0000 7f1baddf0640 1 -- 172.21.15.70:0/1609882252 wait complete. 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.181+0000 7f1baddf0640 1 Processor -- start 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.181+0000 7f1baddf0640 1 -- start start 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.182+0000 7f1baddf0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba806dfc0 0x7f1ba814da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.182+0000 7f1baddf0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80afbe0 0x7f1ba80ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:26.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.182+0000 7f1baddf0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ba80cb130 0x7f1ba80ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.182+0000 7f1baddf0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba81f36e0 con 0x7f1ba80afbe0 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.182+0000 7f1baddf0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba80ce4d0 con 0x7f1ba806dfc0 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.182+0000 7f1baddf0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba80ce7b0 con 0x7f1ba80cb130 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80afbe0 0x7f1ba80ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80afbe0 0x7f1ba80ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55970/0 (socket says 172.21.15.70:55970) 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1b9ffff640 1 -- 172.21.15.70:0/3069373512 learned_addr learned my addr 172.21.15.70:0/3069373512 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1ba77fe640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba806dfc0 0x7f1ba814da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1b9ffff640 1 -- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ba80cb130 msgr2=0x7f1ba80ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1b9ffff640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ba80cb130 0x7f1ba80ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1b9ffff640 1 -- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba806dfc0 msgr2=0x7f1ba814da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:26.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1b9ffff640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba806dfc0 0x7f1ba814da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:26.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1b9ffff640 1 -- 172.21.15.70:0/3069373512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba0009a70 con 0x7f1ba80afbe0 2023-10-08T06:54:26.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1ba77fe640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba806dfc0 0x7f1ba814da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:26.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1b9ffff640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80afbe0 0x7f1ba80ca0b0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c390 tx=0x7f1b9000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:26.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1ba4ff9640 1 -- 172.21.15.70:0/3069373512 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b90005ce0 con 0x7f1ba80afbe0 2023-10-08T06:54:26.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1ba4ff9640 1 -- 172.21.15.70:0/3069373512 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b90005e40 con 0x7f1ba80afbe0 2023-10-08T06:54:26.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.183+0000 7f1ba4ff9640 1 -- 172.21.15.70:0/3069373512 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b900072f0 con 0x7f1ba80afbe0 2023-10-08T06:54:26.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.184+0000 7f1baddf0640 1 -- 172.21.15.70:0/3069373512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba80cea90 con 0x7f1ba80afbe0 2023-10-08T06:54:26.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.184+0000 7f1baddf0640 1 -- 172.21.15.70:0/3069373512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba80cf290 con 0x7f1ba80afbe0 2023-10-08T06:54:26.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.185+0000 7f1baddf0640 1 -- 172.21.15.70:0/3069373512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba80cefb0 con 0x7f1ba80afbe0 2023-10-08T06:54:26.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.186+0000 7f1ba4ff9640 1 -- 172.21.15.70:0/3069373512 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1b90007450 con 0x7f1ba80afbe0 2023-10-08T06:54:26.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.186+0000 7f1ba4ff9640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f1b8806af90 0x7f1b8806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:26.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.187+0000 7f1ba4ff9640 1 -- 172.21.15.70:0/3069373512 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f1b90088f30 con 0x7f1ba80afbe0 2023-10-08T06:54:26.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.187+0000 7f1ba77fe640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f1b8806af90 0x7f1b8806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:26.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.188+0000 7f1ba77fe640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f1b8806af90 0x7f1b8806d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b98006fd0 tx=0x7f1b98008040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:26.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.191+0000 7f1ba4ff9640 1 -- 172.21.15.70:0/3069373512 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ba80cefb0 con 0x7f1ba80afbe0 2023-10-08T06:54:26.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.377+0000 7f1baddf0640 1 -- 172.21.15.70:0/3069373512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1ba80cefb0 con 0x7f1ba80afbe0 2023-10-08T06:54:26.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.382+0000 7f1ba4ff9640 1 -- 172.21.15.70:0/3069373512 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1ba80cefb0 con 0x7f1ba80afbe0 2023-10-08T06:54:26.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.384+0000 7f1baddf0640 1 -- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f1b8806af90 msgr2=0x7f1b8806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f1b8806af90 0x7f1b8806d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b98006fd0 tx=0x7f1b98008040 comp rx=0 tx=0).stop 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 -- 172.21.15.70:0/3069373512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80afbe0 msgr2=0x7f1ba80ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80afbe0 0x7f1ba80ca0b0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000c390 tx=0x7f1b9000c850 comp rx=0 tx=0).stop 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 -- 172.21.15.70:0/3069373512 shutdown_connections 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f1b8806af90 0x7f1b8806d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1ba80cb130 0x7f1ba80ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ba80afbe0 0x7f1ba80ca0b0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 --2- 172.21.15.70:0/3069373512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1ba806dfc0 0x7f1ba814da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 -- 172.21.15.70:0/3069373512 >> 172.21.15.70:0/3069373512 conn(0x7f1ba806a5d0 msgr2=0x7f1ba80c8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 -- 172.21.15.70:0/3069373512 shutdown_connections 2023-10-08T06:54:26.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:26.385+0000 7f1baddf0640 1 -- 172.21.15.70:0/3069373512 wait complete. 2023-10-08T06:54:26.404 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T06:54:26.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T06:54:26.516 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T06:54:26.648 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T06:54:26.648 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:54:26.648 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:26.707 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T06:54:26.708 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T06:54:26.708 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:26.773 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T06:54:26.774 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:54:26.774 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:26.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T06:54:26.837 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T06:54:26.837 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:26.897 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T06:54:26.897 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T06:54:26.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:54:27.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.123+0000 7f9cced55640 1 Processor -- start 2023-10-08T06:54:27.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.123+0000 7f9cced55640 1 -- start start 2023-10-08T06:54:27.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.125+0000 7f9cced55640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc80cc210 0x7f9cc80b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.125+0000 7f9cced55640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc80cacf0 con 0x7f9cc81599d0 2023-10-08T06:54:27.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.125+0000 7f9cced55640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc80ca3c0 con 0x7f9cc8159db0 2023-10-08T06:54:27.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.125+0000 7f9cced55640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80b10f0 con 0x7f9cc80cc210 2023-10-08T06:54:27.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.125+0000 7f9ccdd53640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9cc81599d0 0x7f9cc80ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55510/0 (socket says 172.21.15.70:55510) 2023-10-08T06:54:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.125+0000 7f9ccdd53640 1 -- 172.21.15.70:0/3211787137 learned_addr learned my addr 172.21.15.70:0/3211787137 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9ccd552640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc80cc210 0x7f9cc80b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171349396 0 0) 0x7f9cc80cacf0 con 0x7f9cc81599d0 2023-10-08T06:54:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cb4003520 con 0x7f9cc81599d0 2023-10-08T06:54:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1576654743 0 0) 0x7f9cb4003520 con 0x7f9cc81599d0 2023-10-08T06:54:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc80cc210 msgr2=0x7f9cc80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc80cc210 0x7f9cc80b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 >> v1:172.21.15.181:6789/0 conn(0x7f9cc8159db0 legacy=0x7f9cc806c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc80ca6a0 con 0x7f9cc81599d0 2023-10-08T06:54:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9cc4001670 con 0x7f9cc81599d0 2023-10-08T06:54:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 >> v1:172.21.15.70:6789/0 conn(0x7f9cc81599d0 legacy=0x7f9cc80ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc80cc210 0x7f9cb40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb4003f20 0x7f9cb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80b10f0 con 0x7f9cb4004d00 2023-10-08T06:54:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb400ba60 con 0x7f9cb4003f20 2023-10-08T06:54:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.126+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb400bc20 con 0x7f9cc80cc210 2023-10-08T06:54:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.127+0000 7f9cce554640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.127+0000 7f9ccd552640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb4003f20 0x7f9cb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.127+0000 7f9ccdd53640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc80cc210 0x7f9cb40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.127+0000 7f9cce554640 1 -- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc80cc210 msgr2=0x7f9cb40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.127+0000 7f9cce554640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc80cc210 0x7f9cb40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.127+0000 7f9cce554640 1 -- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb4003f20 msgr2=0x7f9cb40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.127+0000 7f9cce554640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb4003f20 0x7f9cb40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.127+0000 7f9cce554640 1 -- 172.21.15.70:0/3211787137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb4003520 con 0x7f9cb4004d00 2023-10-08T06:54:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.128+0000 7f9cce554640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0001a30 tx=0x7f9cb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a397460fb2aaa2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:27.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.129+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0003b70 con 0x7f9cb4004d00 2023-10-08T06:54:27.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.129+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc0003cd0 con 0x7f9cb4004d00 2023-10-08T06:54:27.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.129+0000 7f9cccd51640 1 -- 172.21.15.70:0/3211787137 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0005310 con 0x7f9cb4004d00 2023-10-08T06:54:27.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.129+0000 7f9cced55640 1 -- 172.21.15.70:0/3211787137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4004d00 msgr2=0x7f9cb4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.129+0000 7f9cced55640 1 --2- 172.21.15.70:0/3211787137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0001a30 tx=0x7f9cb40057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.129+0000 7f9cce554640 1 -- 172.21.15.70:0/3211787137 reap_dead start 2023-10-08T06:54:27.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.129+0000 7f9cced55640 1 -- 172.21.15.70:0/3211787137 shutdown_connections 2023-10-08T06:54:27.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.129+0000 7f9cced55640 1 -- 172.21.15.70:0/3211787137 >> 172.21.15.70:0/3211787137 conn(0x7f9cc80cb0c0 msgr2=0x7f9cc80cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:27.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.129+0000 7f9cced55640 1 -- 172.21.15.70:0/3211787137 shutdown_connections 2023-10-08T06:54:27.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.129+0000 7f9cced55640 1 -- 172.21.15.70:0/3211787137 wait complete. 2023-10-08T06:54:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.131+0000 7f9cced55640 1 Processor -- start 2023-10-08T06:54:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.131+0000 7f9cced55640 1 -- start start 2023-10-08T06:54:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.132+0000 7f9cced55640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb4004d00 0x7f9cc815b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.132+0000 7f9cced55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc8155800 0x7f9cc8156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.132+0000 7f9cced55640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc8155d30 0x7f9cc8154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.132+0000 7f9cced55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc815a190 con 0x7f9cc8155800 2023-10-08T06:54:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.132+0000 7f9cced55640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc815a470 con 0x7f9cb4004d00 2023-10-08T06:54:27.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.132+0000 7f9cced55640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc815a750 con 0x7f9cc8155d30 2023-10-08T06:54:27.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9ccd552640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc8155800 0x7f9cc8156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9ccd552640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc8155800 0x7f9cc8156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55998/0 (socket says 172.21.15.70:55998) 2023-10-08T06:54:27.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9ccd552640 1 -- 172.21.15.70:0/875066220 learned_addr learned my addr 172.21.15.70:0/875066220 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:27.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9ccdd53640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb4004d00 0x7f9cc815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9cce554640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc8155d30 0x7f9cc8154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9ccd552640 1 -- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc8155d30 msgr2=0x7f9cc8154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9ccd552640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc8155d30 0x7f9cc8154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9ccd552640 1 -- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb4004d00 msgr2=0x7f9cc815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9ccd552640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb4004d00 0x7f9cc815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9ccd552640 1 -- 172.21.15.70:0/875066220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc815aa30 con 0x7f9cc8155800 2023-10-08T06:54:27.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.133+0000 7f9cce554640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc8155d30 0x7f9cc8154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:27.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.134+0000 7f9ccd552640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc8155800 0x7f9cc8156510 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8011560 tx=0x7f9cb8011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:27.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.134+0000 7f9cbe7fc640 1 -- 172.21.15.70:0/875066220 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8004220 con 0x7f9cc8155800 2023-10-08T06:54:27.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.134+0000 7f9cbe7fc640 1 -- 172.21.15.70:0/875066220 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb8004380 con 0x7f9cc8155800 2023-10-08T06:54:27.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.134+0000 7f9cbe7fc640 1 -- 172.21.15.70:0/875066220 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8015870 con 0x7f9cc8155800 2023-10-08T06:54:27.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.134+0000 7f9cced55640 1 -- 172.21.15.70:0/875066220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc815ad10 con 0x7f9cc8155800 2023-10-08T06:54:27.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.134+0000 7f9cced55640 1 -- 172.21.15.70:0/875066220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc815ba50 con 0x7f9cc8155800 2023-10-08T06:54:27.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.135+0000 7f9cced55640 1 -- 172.21.15.70:0/875066220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc804e160 con 0x7f9cc8155800 2023-10-08T06:54:27.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.136+0000 7f9cbe7fc640 1 -- 172.21.15.70:0/875066220 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9cb80159d0 con 0x7f9cc8155800 2023-10-08T06:54:27.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.137+0000 7f9cbe7fc640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9cac06af90 0x7f9cac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.137+0000 7f9cbe7fc640 1 -- 172.21.15.70:0/875066220 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f9cb8089bd0 con 0x7f9cc8155800 2023-10-08T06:54:27.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.138+0000 7f9ccdd53640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9cac06af90 0x7f9cac06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.139+0000 7f9ccdd53640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9cac06af90 0x7f9cac06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4001c90 tx=0x7f9cc4009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:27.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.141+0000 7f9cbe7fc640 1 -- 172.21.15.70:0/875066220 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9cb8054960 con 0x7f9cc8155800 2023-10-08T06:54:27.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.368+0000 7f9cced55640 1 -- 172.21.15.70:0/875066220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cc80adc60 con 0x7f9cc8155800 2023-10-08T06:54:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.369+0000 7f9cbe7fc640 1 -- 172.21.15.70:0/875066220 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f9cb8022090 con 0x7f9cc8155800 2023-10-08T06:54:27.370 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:27.370 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":85,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6680,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/542411062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":542411062},{"type":"v1","addr":"172.21.15.181:6836","nonce":542411062}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/646194010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":646194010},{"type":"v1","addr":"172.21.15.70:6835","nonce":646194010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":7492,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/1850202608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":1850202608},{"type":"v1","addr":"172.21.15.181:6837","nonce":1850202608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":9984,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1213234455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1213234455},{"type":"v1","addr":"172.21.15.70:6834","nonce":1213234455}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[]} 2023-10-08T06:54:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 85 2023-10-08T06:54:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 -- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9cac06af90 msgr2=0x7f9cac06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9cac06af90 0x7f9cac06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4001c90 tx=0x7f9cc4009040 comp rx=0 tx=0).stop 2023-10-08T06:54:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 -- 172.21.15.70:0/875066220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc8155800 msgr2=0x7f9cc8156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc8155800 0x7f9cc8156510 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8011560 tx=0x7f9cb8011a20 comp rx=0 tx=0).stop 2023-10-08T06:54:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 -- 172.21.15.70:0/875066220 shutdown_connections 2023-10-08T06:54:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9cac06af90 0x7f9cac06d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9cc8155d30 0x7f9cc8154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cc8155800 0x7f9cc8156510 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 --2- 172.21.15.70:0/875066220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9cb4004d00 0x7f9cc815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 -- 172.21.15.70:0/875066220 >> 172.21.15.70:0/875066220 conn(0x7f9cc80cb0c0 msgr2=0x7f9cc80b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:27.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 -- 172.21.15.70:0/875066220 shutdown_connections 2023-10-08T06:54:27.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.372+0000 7f9cced55640 1 -- 172.21.15.70:0/875066220 wait complete. 2023-10-08T06:54:27.392 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T06:54:27.392 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T06:54:27.393 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T06:54:27.393 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T06:54:27.393 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T06:54:27.393 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T06:54:27.393 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T06:54:27.393 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T06:54:27.394 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T06:54:27.394 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T06:54:27.394 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T06:54:27.394 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T06:54:27.394 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T06:54:27.394 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T06:54:27.395 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T06:54:27.395 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T06:54:27.414 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T06:54:27.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T06:54:27.415 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T06:54:27.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T06:54:27.416 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T06:54:27.417 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T06:54:27.423 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T06:54:27.424 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T06:54:27.465 INFO:tasks.ceph.mds.a:Started 2023-10-08T06:54:27.466 DEBUG:teuthology.parallel:result is None 2023-10-08T06:54:27.475 INFO:tasks.ceph.mds.c:Started 2023-10-08T06:54:27.475 INFO:tasks.ceph.mds.b:Started 2023-10-08T06:54:27.475 DEBUG:teuthology.parallel:result is None 2023-10-08T06:54:27.475 DEBUG:teuthology.parallel:result is None 2023-10-08T06:54:27.482 INFO:tasks.ceph.mds.d:Started 2023-10-08T06:54:27.482 DEBUG:teuthology.parallel:result is None 2023-10-08T06:54:27.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T06:54:27.624 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T06:54:27.637 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T06:54:27.637 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T06:54:27.641 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T06:54:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.673+0000 7fdf641c5640 1 Processor -- start 2023-10-08T06:54:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.673+0000 7fdf641c5640 1 -- start start 2023-10-08T06:54:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.674+0000 7fdf641c5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf5c0afbe0 0x7fdf5c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.674+0000 7fdf641c5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdf5c0cb130 0x7fdf5c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.674+0000 7fdf641c5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5c0cbad0 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.674+0000 7fdf641c5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c0b2620 con 0x7fdf5c0afbe0 2023-10-08T06:54:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.674+0000 7fdf641c5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c0b2060 con 0x7fdf5c0cb130 2023-10-08T06:54:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.675+0000 7fdf61f61640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf5c0afbe0 0x7fdf5c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.675+0000 7fdf61f61640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdf5c0afbe0 0x7fdf5c0b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56020/0 (socket says 172.21.15.70:56020) 2023-10-08T06:54:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.675+0000 7fdf61f61640 1 -- 172.21.15.70:0/676622219 learned_addr learned my addr 172.21.15.70:0/676622219 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.675+0000 7fdf62762640 1 --2- 172.21.15.70:0/676622219 >> v2:172.21.15.181:3300/0 conn(0x7fdf5c0cb130 0x7fdf5c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.675+0000 7fdf61f61640 1 -- 172.21.15.70:0/676622219 >> v2:172.21.15.181:3300/0 conn(0x7fdf5c0cb130 msgr2=0x7fdf5c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.675+0000 7fdf61f61640 1 --2- 172.21.15.70:0/676622219 >> v2:172.21.15.181:3300/0 conn(0x7fdf5c0cb130 0x7fdf5c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.675+0000 7fdf61f61640 1 -- 172.21.15.70:0/676622219 >> v1:172.21.15.181:6789/0 conn(0x7fdf5c06dfc0 legacy=0x7fdf5c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.675+0000 7fdf61f61640 1 -- 172.21.15.70:0/676622219 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0b2340 con 0x7fdf5c0afbe0 2023-10-08T06:54:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.676+0000 7fdf61f61640 1 --2- 172.21.15.70:0/676622219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5c0afbe0 0x7fdf5c0b15b0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0c8d10 tx=0x7fdf5802b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deec51d9de35ac2b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf60f5f640 1 -- 172.21.15.70:0/676622219 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58030880 con 0x7fdf5c0afbe0 2023-10-08T06:54:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf60f5f640 1 -- 172.21.15.70:0/676622219 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf580309e0 con 0x7fdf5c0afbe0 2023-10-08T06:54:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf60f5f640 1 -- 172.21.15.70:0/676622219 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58030db0 con 0x7fdf5c0afbe0 2023-10-08T06:54:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf641c5640 1 -- 172.21.15.70:0/676622219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5c0afbe0 msgr2=0x7fdf5c0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf641c5640 1 --2- 172.21.15.70:0/676622219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5c0afbe0 0x7fdf5c0b15b0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0c8d10 tx=0x7fdf5802b9c0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf641c5640 1 -- 172.21.15.70:0/676622219 shutdown_connections 2023-10-08T06:54:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf641c5640 1 --2- 172.21.15.70:0/676622219 >> v2:172.21.15.181:3300/0 conn(0x7fdf5c0cb130 0x7fdf5c0cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf641c5640 1 --2- 172.21.15.70:0/676622219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5c0afbe0 0x7fdf5c0b15b0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf641c5640 1 -- 172.21.15.70:0/676622219 >> 172.21.15.70:0/676622219 conn(0x7fdf5c06a5d0 msgr2=0x7fdf5c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf641c5640 1 -- 172.21.15.70:0/676622219 shutdown_connections 2023-10-08T06:54:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.677+0000 7fdf641c5640 1 -- 172.21.15.70:0/676622219 wait complete. 2023-10-08T06:54:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.679+0000 7fdf641c5640 1 Processor -- start 2023-10-08T06:54:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.679+0000 7fdf641c5640 1 -- start start 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.680+0000 7fdf641c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf5c06dfc0 0x7fdf5c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.680+0000 7fdf641c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5c0afbe0 0x7fdf5c0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.680+0000 7fdf641c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c0cb130 0x7fdf5c0c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.680+0000 7fdf641c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0ce6b0 con 0x7fdf5c0afbe0 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.680+0000 7fdf641c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c156200 con 0x7fdf5c0cb130 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.680+0000 7fdf641c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0c6bf0 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf61f61640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf5c06dfc0 0x7fdf5c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf61f61640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf5c06dfc0 0x7fdf5c155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:33156/0 (socket says 172.21.15.70:33156) 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf61f61640 1 -- 172.21.15.70:0/602589690 learned_addr learned my addr 172.21.15.70:0/602589690 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf61760640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5c0afbe0 0x7fdf5c0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf62762640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c0cb130 0x7fdf5c0c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf61f61640 1 -- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c0cb130 msgr2=0x7fdf5c0c2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf61f61640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c0cb130 0x7fdf5c0c2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf61f61640 1 -- 172.21.15.70:0/602589690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5c0afbe0 msgr2=0x7fdf5c0c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf61f61640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5c0afbe0 0x7fdf5c0c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf61f61640 1 -- 172.21.15.70:0/602589690 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580094d0 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.681+0000 7fdf61f61640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf5c06dfc0 0x7fdf5c155cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009960 tx=0x7fdf58004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.682+0000 7fdf527fc640 1 -- 172.21.15.70:0/602589690 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58043400 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.682+0000 7fdf527fc640 1 -- 172.21.15.70:0/602589690 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf58043560 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.682+0000 7fdf641c5640 1 -- 172.21.15.70:0/602589690 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c0cd7c0 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.682+0000 7fdf527fc640 1 -- 172.21.15.70:0/602589690 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58043910 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.682+0000 7fdf641c5640 1 -- 172.21.15.70:0/602589690 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c0cdd80 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.683+0000 7fdf641c5640 1 -- 172.21.15.70:0/602589690 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c04e160 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.684+0000 7fdf527fc640 1 -- 172.21.15.70:0/602589690 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdf58043a70 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.685+0000 7fdf527fc640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fdf4006af90 0x7fdf4006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:27.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.685+0000 7fdf527fc640 1 -- 172.21.15.70:0/602589690 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6067+0+0 (secure 0 0 0) 0x7fdf5803e070 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.685+0000 7fdf61760640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fdf4006af90 0x7fdf4006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:27.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.686+0000 7fdf61760640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fdf4006af90 0x7fdf4006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0cbce0 tx=0x7fdf4c009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:27.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.690+0000 7fdf527fc640 1 -- 172.21.15.70:0/602589690 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf580779b0 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.889+0000 7fdf641c5640 1 -- 172.21.15.70:0/602589690 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fdf5c0b0ef0 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.975+0000 7fdf527fc640 1 -- 172.21.15.70:0/602589690 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v192) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdf5807afd0 con 0x7fdf5c06dfc0 2023-10-08T06:54:27.977 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T06:54:27.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 -- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fdf4006af90 msgr2=0x7fdf4006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fdf4006af90 0x7fdf4006d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c0cbce0 tx=0x7fdf4c009040 comp rx=0 tx=0).stop 2023-10-08T06:54:27.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 -- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf5c06dfc0 msgr2=0x7fdf5c155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:27.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf5c06dfc0 0x7fdf5c155cc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009960 tx=0x7fdf58004730 comp rx=0 tx=0).stop 2023-10-08T06:54:27.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 -- 172.21.15.70:0/602589690 shutdown_connections 2023-10-08T06:54:27.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fdf4006af90 0x7fdf4006d450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdf5c0cb130 0x7fdf5c0c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf5c0afbe0 0x7fdf5c0c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 --2- 172.21.15.70:0/602589690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdf5c06dfc0 0x7fdf5c155cc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:27.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 -- 172.21.15.70:0/602589690 >> 172.21.15.70:0/602589690 conn(0x7fdf5c06a5d0 msgr2=0x7fdf5c0c9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:27.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 -- 172.21.15.70:0/602589690 shutdown_connections 2023-10-08T06:54:27.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:27.978+0000 7fdf641c5640 1 -- 172.21.15.70:0/602589690 wait complete. 2023-10-08T06:54:28.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T06:54:28.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.210+0000 7ff7e359e640 1 Processor -- start 2023-10-08T06:54:28.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.211+0000 7ff7e359e640 1 -- start start 2023-10-08T06:54:28.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.212+0000 7ff7e359e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7e415ab30 0x7ff7e415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.212+0000 7ff7e359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7e40b07d0 0x7ff7e406cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.212+0000 7ff7e359e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e415f7d0 con 0x7ff7e40adc70 2023-10-08T06:54:28.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.212+0000 7ff7e359e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e415f210 con 0x7ff7e415ab30 2023-10-08T06:54:28.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.212+0000 7ff7e359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415f4f0 con 0x7ff7e40b07d0 2023-10-08T06:54:28.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.213+0000 7ff7dbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff7e40adc70 0x7ff7e40b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55550/0 (socket says 172.21.15.70:55550) 2023-10-08T06:54:28.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.213+0000 7ff7dbfff640 1 -- 172.21.15.70:0/3725686981 learned_addr learned my addr 172.21.15.70:0/3725686981 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:28.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.213+0000 7ff7e259c640 1 --2- 172.21.15.70:0/3725686981 >> v2:172.21.15.181:3300/0 conn(0x7ff7e415ab30 0x7ff7e415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.213+0000 7ff7e2d9d640 1 --2- 172.21.15.70:0/3725686981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7e40b07d0 0x7ff7e406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.213+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696963951 0 0) 0x7ff7e415f7d0 con 0x7ff7e40adc70 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.213+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d0003520 con 0x7ff7e40adc70 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.213+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2620515818 0 0) 0x7ff7d0003520 con 0x7ff7e40adc70 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.213+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7e40b07d0 msgr2=0x7ff7e406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.213+0000 7ff7e1d9b640 1 --2- 172.21.15.70:0/3725686981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7e40b07d0 0x7ff7e406cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 >> v2:172.21.15.181:3300/0 conn(0x7ff7e415ab30 msgr2=0x7ff7e415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 --2- 172.21.15.70:0/3725686981 >> v2:172.21.15.181:3300/0 conn(0x7ff7e415ab30 0x7ff7e415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e406c550 con 0x7ff7e40adc70 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff7cc001670 con 0x7ff7e40adc70 2023-10-08T06:54:28.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 >> v1:172.21.15.70:6789/0 conn(0x7ff7e40adc70 legacy=0x7ff7e40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 --2- 172.21.15.70:0/3725686981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7e40b07d0 0x7ff7d0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 --2- 172.21.15.70:0/3725686981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d0003f40 0x7ff7d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 --2- 172.21.15.70:0/3725686981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d0004d00 0x7ff7d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415f210 con 0x7ff7d0003f40 2023-10-08T06:54:28.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415f4f0 con 0x7ff7d0004d00 2023-10-08T06:54:28.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d000b980 con 0x7ff7e40b07d0 2023-10-08T06:54:28.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7dbfff640 1 --2- 172.21.15.70:0/3725686981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d0003f40 0x7ff7d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7dbfff640 1 -- 172.21.15.70:0/3725686981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7e40b07d0 msgr2=0x7ff7d0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:28.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7dbfff640 1 --2- 172.21.15.70:0/3725686981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7e40b07d0 0x7ff7d0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7dbfff640 1 -- 172.21.15.70:0/3725686981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d0004d00 msgr2=0x7ff7d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:28.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7dbfff640 1 --2- 172.21.15.70:0/3725686981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7d0004d00 0x7ff7d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.214+0000 7ff7dbfff640 1 -- 172.21.15.70:0/3725686981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d0003520 con 0x7ff7d0003f40 2023-10-08T06:54:28.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.215+0000 7ff7dbfff640 1 --2- 172.21.15.70:0/3725686981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d0003f40 0x7ff7d00047c0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c140 tx=0x7ff7d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184f741eaf63c88d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:28.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.216+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc005ad0 con 0x7ff7d0003f40 2023-10-08T06:54:28.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.216+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc005c50 con 0x7ff7d0003f40 2023-10-08T06:54:28.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.216+0000 7ff7e1d9b640 1 -- 172.21.15.70:0/3725686981 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc02c810 con 0x7ff7d0003f40 2023-10-08T06:54:28.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.216+0000 7ff7e359e640 1 -- 172.21.15.70:0/3725686981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d0003f40 msgr2=0x7ff7d00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.216+0000 7ff7e359e640 1 --2- 172.21.15.70:0/3725686981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7d0003f40 0x7ff7d00047c0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c140 tx=0x7ff7d00057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.216+0000 7ff7e2d9d640 1 -- 172.21.15.70:0/3725686981 reap_dead start 2023-10-08T06:54:28.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.216+0000 7ff7e359e640 1 -- 172.21.15.70:0/3725686981 shutdown_connections 2023-10-08T06:54:28.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.216+0000 7ff7e359e640 1 -- 172.21.15.70:0/3725686981 >> 172.21.15.70:0/3725686981 conn(0x7ff7e40cb0c0 msgr2=0x7ff7e40cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:28.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.216+0000 7ff7e359e640 1 -- 172.21.15.70:0/3725686981 shutdown_connections 2023-10-08T06:54:28.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.216+0000 7ff7e359e640 1 -- 172.21.15.70:0/3725686981 wait complete. 2023-10-08T06:54:28.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.218+0000 7ff7e359e640 1 Processor -- start 2023-10-08T06:54:28.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.218+0000 7ff7e359e640 1 -- start start 2023-10-08T06:54:28.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.219+0000 7ff7e359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7d0003f40 0x7ff7e414d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.219+0000 7ff7e359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7e4155e80 0x7ff7e40b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.219+0000 7ff7e359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7e40b39b0 0x7ff7e40c6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.219+0000 7ff7e359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40c6f10 con 0x7ff7e4155e80 2023-10-08T06:54:28.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.219+0000 7ff7e359e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e414de50 con 0x7ff7e40b39b0 2023-10-08T06:54:28.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.219+0000 7ff7e359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40b23d0 con 0x7ff7d0003f40 2023-10-08T06:54:28.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.219+0000 7ff7dbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7e4155e80 0x7ff7e40b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.219+0000 7ff7dbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7e4155e80 0x7ff7e40b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56042/0 (socket says 172.21.15.70:56042) 2023-10-08T06:54:28.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.219+0000 7ff7dbfff640 1 -- 172.21.15.70:0/145421368 learned_addr learned my addr 172.21.15.70:0/145421368 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:28.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7e259c640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7d0003f40 0x7ff7e414d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7e2d9d640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7e40b39b0 0x7ff7e40c6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7dbfff640 1 -- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7d0003f40 msgr2=0x7ff7e414d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7dbfff640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7d0003f40 0x7ff7e414d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7dbfff640 1 -- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7e40b39b0 msgr2=0x7ff7e40c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7dbfff640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7e40b39b0 0x7ff7e40c6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7dbfff640 1 -- 172.21.15.70:0/145421368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e40b26b0 con 0x7ff7e4155e80 2023-10-08T06:54:28.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7dbfff640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7e4155e80 0x7ff7e40b3470 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc02c3b0 tx=0x7ff7cc001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:28.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7daffd640 1 -- 172.21.15.70:0/145421368 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc03c040 con 0x7ff7e4155e80 2023-10-08T06:54:28.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7daffd640 1 -- 172.21.15.70:0/145421368 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc02dcb0 con 0x7ff7e4155e80 2023-10-08T06:54:28.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7e359e640 1 -- 172.21.15.70:0/145421368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e40b2990 con 0x7ff7e4155e80 2023-10-08T06:54:28.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.220+0000 7ff7daffd640 1 -- 172.21.15.70:0/145421368 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc0072b0 con 0x7ff7e4155e80 2023-10-08T06:54:28.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.221+0000 7ff7e359e640 1 -- 172.21.15.70:0/145421368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e40b2f50 con 0x7ff7e4155e80 2023-10-08T06:54:28.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.222+0000 7ff7e359e640 1 -- 172.21.15.70:0/145421368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e404e160 con 0x7ff7e4155e80 2023-10-08T06:54:28.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.223+0000 7ff7daffd640 1 -- 172.21.15.70:0/145421368 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff7cc032070 con 0x7ff7e4155e80 2023-10-08T06:54:28.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.223+0000 7ff7daffd640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff7c406af90 0x7ff7c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.224+0000 7ff7daffd640 1 -- 172.21.15.70:0/145421368 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5293+0+0 (secure 0 0 0) 0x7ff7cc0aac80 con 0x7ff7e4155e80 2023-10-08T06:54:28.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.224+0000 7ff7e259c640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff7c406af90 0x7ff7c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.225+0000 7ff7e259c640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff7c406af90 0x7ff7c406d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40099f0 tx=0x7ff7d4008040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:28.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.228+0000 7ff7daffd640 1 -- 172.21.15.70:0/145421368 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7cc075dc0 con 0x7ff7e4155e80 2023-10-08T06:54:28.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.466+0000 7ff7e359e640 1 -- 172.21.15.70:0/145421368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff7e40ada50 con 0x7ff7e4155e80 2023-10-08T06:54:28.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.467+0000 7ff7daffd640 1 -- 172.21.15.70:0/145421368 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v44) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff7cc0793e0 con 0x7ff7e4155e80 2023-10-08T06:54:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:28.468 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T06:54:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T06:54:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T06:54:28.468 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.469 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:54:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T06:54:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T06:54:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T06:54:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T06:54:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.470 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T06:54:28.471 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.473 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T06:54:28.474 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:54:28.475 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.476 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.477 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T06:54:28.478 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:54:28.479 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T06:54:28.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.482 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T06:54:28.483 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T06:54:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T06:54:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T06:54:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:28.484 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T06:54:28.484 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T06:54:28.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 -- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff7c406af90 msgr2=0x7ff7c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff7c406af90 0x7ff7c406d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40099f0 tx=0x7ff7d4008040 comp rx=0 tx=0).stop 2023-10-08T06:54:28.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 -- 172.21.15.70:0/145421368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7e4155e80 msgr2=0x7ff7e40b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7e4155e80 0x7ff7e40b3470 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc02c3b0 tx=0x7ff7cc001930 comp rx=0 tx=0).stop 2023-10-08T06:54:28.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 -- 172.21.15.70:0/145421368 shutdown_connections 2023-10-08T06:54:28.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7ff7c406af90 0x7ff7c406d450 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff7e40b39b0 0x7ff7e40c6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7e4155e80 0x7ff7e40b3470 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 --2- 172.21.15.70:0/145421368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7d0003f40 0x7ff7e414d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 -- 172.21.15.70:0/145421368 >> 172.21.15.70:0/145421368 conn(0x7ff7e40cb0c0 msgr2=0x7ff7e40af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:28.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 -- 172.21.15.70:0/145421368 shutdown_connections 2023-10-08T06:54:28.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.470+0000 7ff7e359e640 1 -- 172.21.15.70:0/145421368 wait complete. 2023-10-08T06:54:28.490 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:54:28.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.685+0000 7f27e8e10640 1 Processor -- start 2023-10-08T06:54:28.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.685+0000 7f27e8e10640 1 -- start start 2023-10-08T06:54:28.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.687+0000 7f27e8e10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f27e40cb580 0x7f27e4165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.687+0000 7f27e8e10640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e415b2b0 con 0x7f27e415ab40 2023-10-08T06:54:28.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.687+0000 7f27e8e10640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e415b590 con 0x7f27e40c90b0 2023-10-08T06:54:28.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.687+0000 7f27e8e10640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e415bf30 con 0x7f27e40cb580 2023-10-08T06:54:28.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.687+0000 7f27e259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f27e415ab40 0x7f27e40ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55564/0 (socket says 172.21.15.70:55564) 2023-10-08T06:54:28.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.687+0000 7f27e259c640 1 -- 172.21.15.70:0/1861303588 learned_addr learned my addr 172.21.15.70:0/1861303588 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:28.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.687+0000 7f27e2d9d640 1 --2- 172.21.15.70:0/1861303588 >> v2:172.21.15.181:3300/0 conn(0x7f27e40cb580 0x7f27e4165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.687+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528422323 0 0) 0x7f27e415b2b0 con 0x7f27e415ab40 2023-10-08T06:54:28.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.687+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27d0003520 con 0x7f27e415ab40 2023-10-08T06:54:28.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2732865727 0 0) 0x7f27d0003520 con 0x7f27e415ab40 2023-10-08T06:54:28.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 >> v2:172.21.15.181:3300/0 conn(0x7f27e40cb580 msgr2=0x7f27e4165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:28.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 --2- 172.21.15.70:0/1861303588 >> v2:172.21.15.181:3300/0 conn(0x7f27e40cb580 0x7f27e4165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 >> v1:172.21.15.181:6789/0 conn(0x7f27e40c90b0 legacy=0x7f27e40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e415b870 con 0x7f27e415ab40 2023-10-08T06:54:28.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f27d4001670 con 0x7f27e415ab40 2023-10-08T06:54:28.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 >> v1:172.21.15.70:6789/0 conn(0x7f27e415ab40 legacy=0x7f27e40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00039c0 0x7f27d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d0005180 0x7f27d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e415bf30 con 0x7f27d00047c0 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d000c080 con 0x7f27d00039c0 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d000c240 con 0x7f27d0005180 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.688+0000 7f27dbfff640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.689+0000 7f27dbfff640 1 -- 172.21.15.70:0/1861303588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d0005180 msgr2=0x7f27d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.689+0000 7f27dbfff640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d0005180 0x7f27d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.689+0000 7f27dbfff640 1 -- 172.21.15.70:0/1861303588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00039c0 msgr2=0x7f27d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.689+0000 7f27e259c640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00039c0 0x7f27d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.689+0000 7f27dbfff640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00039c0 0x7f27d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.689+0000 7f27dbfff640 1 -- 172.21.15.70:0/1861303588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d0003520 con 0x7f27d00047c0 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.689+0000 7f27e2d9d640 1 -- 172.21.15.70:0/1861303588 reap_dead start 2023-10-08T06:54:28.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.689+0000 7f27e259c640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00039c0 0x7f27d0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:28.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.690+0000 7f27dbfff640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27d0004c40 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f27cc001720 tx=0x7f27d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda995c41aacaea2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:28.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.690+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc0052d0 con 0x7f27d00047c0 2023-10-08T06:54:28.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.690+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27cc02e050 con 0x7f27d00047c0 2023-10-08T06:54:28.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.690+0000 7f27e1d9b640 1 -- 172.21.15.70:0/1861303588 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc003830 con 0x7f27d00047c0 2023-10-08T06:54:28.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.690+0000 7f27e8e10640 1 -- 172.21.15.70:0/1861303588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 msgr2=0x7f27d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.690+0000 7f27e8e10640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27d0004c40 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f27cc001720 tx=0x7f27d00057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.691+0000 7f27e8e10640 1 -- 172.21.15.70:0/1861303588 shutdown_connections 2023-10-08T06:54:28.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.691+0000 7f27e8e10640 1 --2- 172.21.15.70:0/1861303588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27d0004c40 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.691+0000 7f27e8e10640 1 -- 172.21.15.70:0/1861303588 >> 172.21.15.70:0/1861303588 conn(0x7f27e406a380 msgr2=0x7f27e40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:28.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.691+0000 7f27e8e10640 1 -- 172.21.15.70:0/1861303588 shutdown_connections 2023-10-08T06:54:28.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.691+0000 7f27e8e10640 1 -- 172.21.15.70:0/1861303588 wait complete. 2023-10-08T06:54:28.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.692+0000 7f27e8e10640 1 Processor -- start 2023-10-08T06:54:28.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.693+0000 7f27e8e10640 1 -- start start 2023-10-08T06:54:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.693+0000 7f27e8e10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27e40c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.693+0000 7f27e8e10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27e40ab8c0 0x7f27e40c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.693+0000 7f27e8e10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e4155300 0x7f27e41f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.693+0000 7f27e8e10640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e415f7a0 con 0x7f27d00047c0 2023-10-08T06:54:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.693+0000 7f27e8e10640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40abd40 con 0x7f27e4155300 2023-10-08T06:54:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.693+0000 7f27e8e10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40c3620 con 0x7f27e40ab8c0 2023-10-08T06:54:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.694+0000 7f27e259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27e40c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.694+0000 7f27e259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27e40c4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56062/0 (socket says 172.21.15.70:56062) 2023-10-08T06:54:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.694+0000 7f27e259c640 1 -- 172.21.15.70:0/1626629021 learned_addr learned my addr 172.21.15.70:0/1626629021 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.694+0000 7f27e259c640 1 -- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27e40ab8c0 msgr2=0x7f27e40c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.694+0000 7f27e259c640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27e40ab8c0 0x7f27e40c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.694+0000 7f27e259c640 1 -- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e4155300 msgr2=0x7f27e41f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.694+0000 7f27e259c640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e4155300 0x7f27e41f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.694+0000 7f27e259c640 1 -- 172.21.15.70:0/1626629021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40c3ec0 con 0x7f27d00047c0 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.694+0000 7f27e259c640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27e40c4350 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f27d400d7a0 tx=0x7f27d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.694+0000 7f27daffd640 1 -- 172.21.15.70:0/1626629021 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d40074e0 con 0x7f27d00047c0 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.695+0000 7f27daffd640 1 -- 172.21.15.70:0/1626629021 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d4007640 con 0x7f27d00047c0 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.695+0000 7f27e8e10640 1 -- 172.21.15.70:0/1626629021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e40c5620 con 0x7f27d00047c0 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.695+0000 7f27daffd640 1 -- 172.21.15.70:0/1626629021 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d40049e0 con 0x7f27d00047c0 2023-10-08T06:54:28.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.695+0000 7f27e8e10640 1 -- 172.21.15.70:0/1626629021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e40c3870 con 0x7f27d00047c0 2023-10-08T06:54:28.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.696+0000 7f27e8e10640 1 -- 172.21.15.70:0/1626629021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e4067490 con 0x7f27d00047c0 2023-10-08T06:54:28.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.697+0000 7f27daffd640 1 -- 172.21.15.70:0/1626629021 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f27d4004b40 con 0x7f27d00047c0 2023-10-08T06:54:28.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.698+0000 7f27daffd640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f27c406afc0 0x7f27c406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:28.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.698+0000 7f27daffd640 1 -- 172.21.15.70:0/1626629021 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f27d4088f50 con 0x7f27d00047c0 2023-10-08T06:54:28.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.698+0000 7f27dbfff640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f27c406afc0 0x7f27c406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:28.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.700+0000 7f27dbfff640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f27c406afc0 0x7f27c406d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27cc033000 tx=0x7f27cc02f000 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:28.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.702+0000 7f27daffd640 1 -- 172.21.15.70:0/1626629021 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27d4053ee0 con 0x7f27d00047c0 2023-10-08T06:54:28.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.895+0000 7f27e8e10640 1 -- 172.21.15.70:0/1626629021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27e40b0ef0 con 0x7f27d00047c0 2023-10-08T06:54:28.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.896+0000 7f27daffd640 1 -- 172.21.15.70:0/1626629021 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+13246 (secure 0 0 0) 0x7f27d4019090 con 0x7f27d00047c0 2023-10-08T06:54:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:28.897 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":194,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:54:27.981773+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:54:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 -- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f27c406afc0 msgr2=0x7f27c406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f27c406afc0 0x7f27c406d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f27cc033000 tx=0x7f27cc02f000 comp rx=0 tx=0).stop 2023-10-08T06:54:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 -- 172.21.15.70:0/1626629021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 msgr2=0x7f27e40c4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27e40c4350 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f27d400d7a0 tx=0x7f27d400dc60 comp rx=0 tx=0).stop 2023-10-08T06:54:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 -- 172.21.15.70:0/1626629021 shutdown_connections 2023-10-08T06:54:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f27c406afc0 0x7f27c406d480 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27e4155300 0x7f27e41f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27e40ab8c0 0x7f27e40c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 --2- 172.21.15.70:0/1626629021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00047c0 0x7f27e40c4350 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 -- 172.21.15.70:0/1626629021 >> 172.21.15.70:0/1626629021 conn(0x7f27e406a380 msgr2=0x7f27e406d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 -- 172.21.15.70:0/1626629021 shutdown_connections 2023-10-08T06:54:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:28.899+0000 7f27e8e10640 1 -- 172.21.15.70:0/1626629021 wait complete. 2023-10-08T06:54:28.919 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T06:54:28.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T06:54:29.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.134+0000 7fe090852640 1 Processor -- start 2023-10-08T06:54:29.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.134+0000 7fe090852640 1 -- start start 2023-10-08T06:54:29.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.135+0000 7fe090852640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe08c1432f0 0x7fe08c0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.135+0000 7fe090852640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe08c0b4240 0x7fe08c159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.135+0000 7fe090852640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08c0cbb30 con 0x7fe08c0cb6b0 2023-10-08T06:54:29.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.135+0000 7fe090852640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c1436d0 con 0x7fe08c1432f0 2023-10-08T06:54:29.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.135+0000 7fe090852640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c15d440 con 0x7fe08c0b4240 2023-10-08T06:54:29.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.136+0000 7fe08a59c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe08c1432f0 0x7fe08c0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.136+0000 7fe08ad9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe08c0b4240 0x7fe08c159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.136+0000 7fe08a59c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe08c1432f0 0x7fe08c0cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39248/0 (socket says 172.21.15.70:39248) 2023-10-08T06:54:29.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.136+0000 7fe08a59c640 1 -- 172.21.15.70:0/2868382458 learned_addr learned my addr 172.21.15.70:0/2868382458 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:29.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.137+0000 7fe08a59c640 1 -- 172.21.15.70:0/2868382458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe08c0b4240 msgr2=0x7fe08c159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:29.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.137+0000 7fe08a59c640 1 --2- 172.21.15.70:0/2868382458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe08c0b4240 0x7fe08c159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.137+0000 7fe08a59c640 1 -- 172.21.15.70:0/2868382458 >> v1:172.21.15.181:6789/0 conn(0x7fe08c0cb6b0 legacy=0x7fe08c0b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.137+0000 7fe08a59c640 1 -- 172.21.15.70:0/2868382458 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c15d690 con 0x7fe08c1432f0 2023-10-08T06:54:29.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.137+0000 7fe089d9b640 1 -- 172.21.15.70:0/2868382458 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276695542 0 0) 0x7fe08c0cbb30 con 0x7fe08c0cb6b0 2023-10-08T06:54:29.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.138+0000 7fe08a59c640 1 --2- 172.21.15.70:0/2868382458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c1432f0 0x7fe08c0cb170 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00bb10 tx=0x7fe07c02e0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6796626465d3910d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:29.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.139+0000 7fe089d9b640 1 -- 172.21.15.70:0/2868382458 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c02ee70 con 0x7fe08c1432f0 2023-10-08T06:54:29.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.139+0000 7fe089d9b640 1 -- 172.21.15.70:0/2868382458 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c036440 con 0x7fe08c1432f0 2023-10-08T06:54:29.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.139+0000 7fe089d9b640 1 -- 172.21.15.70:0/2868382458 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0367d0 con 0x7fe08c1432f0 2023-10-08T06:54:29.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.139+0000 7fe090852640 1 -- 172.21.15.70:0/2868382458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c1432f0 msgr2=0x7fe08c0cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.139+0000 7fe090852640 1 --2- 172.21.15.70:0/2868382458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c1432f0 0x7fe08c0cb170 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00bb10 tx=0x7fe07c02e0c0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.140+0000 7fe090852640 1 -- 172.21.15.70:0/2868382458 shutdown_connections 2023-10-08T06:54:29.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.140+0000 7fe090852640 1 --2- 172.21.15.70:0/2868382458 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe08c0b4240 0x7fe08c159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.140+0000 7fe090852640 1 --2- 172.21.15.70:0/2868382458 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c1432f0 0x7fe08c0cb170 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.140+0000 7fe090852640 1 -- 172.21.15.70:0/2868382458 >> 172.21.15.70:0/2868382458 conn(0x7fe08c069d50 msgr2=0x7fe08c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:29.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.140+0000 7fe090852640 1 -- 172.21.15.70:0/2868382458 shutdown_connections 2023-10-08T06:54:29.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.140+0000 7fe090852640 1 -- 172.21.15.70:0/2868382458 wait complete. 2023-10-08T06:54:29.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.141+0000 7fe090852640 1 Processor -- start 2023-10-08T06:54:29.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.141+0000 7fe090852640 1 -- start start 2023-10-08T06:54:29.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.142+0000 7fe090852640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe08c0b4240 0x7fe08c139a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.142+0000 7fe090852640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c0cb6b0 0x7fe08c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.142+0000 7fe090852640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08c1432f0 0x7fe08c0c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.142+0000 7fe090852640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c153d60 con 0x7fe08c1432f0 2023-10-08T06:54:29.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe090852640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c139f50 con 0x7fe08c0cb6b0 2023-10-08T06:54:29.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe090852640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c15a7a0 con 0x7fe08c0b4240 2023-10-08T06:54:29.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe08ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08c1432f0 0x7fe08c0c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe08ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08c1432f0 0x7fe08c0c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56074/0 (socket says 172.21.15.70:56074) 2023-10-08T06:54:29.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe08ad9d640 1 -- 172.21.15.70:0/2242000460 learned_addr learned my addr 172.21.15.70:0/2242000460 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:29.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe08a59c640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe08c0b4240 0x7fe08c139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe083fff640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c0cb6b0 0x7fe08c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe08ad9d640 1 -- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe08c0b4240 msgr2=0x7fe08c139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe08ad9d640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe08c0b4240 0x7fe08c139a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe08ad9d640 1 -- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c0cb6b0 msgr2=0x7fe08c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe08ad9d640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c0cb6b0 0x7fe08c159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.143+0000 7fe08ad9d640 1 -- 172.21.15.70:0/2242000460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c00b770 con 0x7fe08c1432f0 2023-10-08T06:54:29.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.144+0000 7fe08ad9d640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08c1432f0 0x7fe08c0c7ad0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fe08400efc0 tx=0x7fe08400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:29.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.144+0000 7fe082ffd640 1 -- 172.21.15.70:0/2242000460 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084014480 con 0x7fe08c1432f0 2023-10-08T06:54:29.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.144+0000 7fe082ffd640 1 -- 172.21.15.70:0/2242000460 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08400f040 con 0x7fe08c1432f0 2023-10-08T06:54:29.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.144+0000 7fe082ffd640 1 -- 172.21.15.70:0/2242000460 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084014820 con 0x7fe08c1432f0 2023-10-08T06:54:29.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.144+0000 7fe090852640 1 -- 172.21.15.70:0/2242000460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c15aa80 con 0x7fe08c1432f0 2023-10-08T06:54:29.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.144+0000 7fe090852640 1 -- 172.21.15.70:0/2242000460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c1379f0 con 0x7fe08c1432f0 2023-10-08T06:54:29.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.145+0000 7fe090852640 1 -- 172.21.15.70:0/2242000460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08c15d0d0 con 0x7fe08c1432f0 2023-10-08T06:54:29.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.146+0000 7fe082ffd640 1 -- 172.21.15.70:0/2242000460 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe084014980 con 0x7fe08c1432f0 2023-10-08T06:54:29.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.147+0000 7fe082ffd640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fe06c06af90 0x7fe06c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.148+0000 7fe082ffd640 1 -- 172.21.15.70:0/2242000460 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fe08408a2e0 con 0x7fe08c1432f0 2023-10-08T06:54:29.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.148+0000 7fe08a59c640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fe06c06af90 0x7fe06c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.149+0000 7fe08a59c640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fe06c06af90 0x7fe06c06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe07c038000 tx=0x7fe07c008040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:29.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.152+0000 7fe082ffd640 1 -- 172.21.15.70:0/2242000460 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe084055300 con 0x7fe08c1432f0 2023-10-08T06:54:29.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.326+0000 7fe090852640 1 -- 172.21.15.70:0/2242000460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe08c137be0 con 0x7fe08c1432f0 2023-10-08T06:54:29.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.327+0000 7fe082ffd640 1 -- 172.21.15.70:0/2242000460 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v194) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe084058920 con 0x7fe08c1432f0 2023-10-08T06:54:29.327 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T06:54:29.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.329+0000 7fe090852640 1 -- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fe06c06af90 msgr2=0x7fe06c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.329+0000 7fe090852640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fe06c06af90 0x7fe06c06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe07c038000 tx=0x7fe07c008040 comp rx=0 tx=0).stop 2023-10-08T06:54:29.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.329+0000 7fe090852640 1 -- 172.21.15.70:0/2242000460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08c1432f0 msgr2=0x7fe08c0c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.329+0000 7fe090852640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08c1432f0 0x7fe08c0c7ad0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fe08400efc0 tx=0x7fe08400c490 comp rx=0 tx=0).stop 2023-10-08T06:54:29.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.329+0000 7fe090852640 1 -- 172.21.15.70:0/2242000460 shutdown_connections 2023-10-08T06:54:29.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.329+0000 7fe090852640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fe06c06af90 0x7fe06c06d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.329+0000 7fe090852640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe08c1432f0 0x7fe08c0c7ad0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.329+0000 7fe090852640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe08c0cb6b0 0x7fe08c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.329+0000 7fe090852640 1 --2- 172.21.15.70:0/2242000460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe08c0b4240 0x7fe08c139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.329+0000 7fe090852640 1 -- 172.21.15.70:0/2242000460 >> 172.21.15.70:0/2242000460 conn(0x7fe08c069d50 msgr2=0x7fe08c13fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:29.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.330+0000 7fe090852640 1 -- 172.21.15.70:0/2242000460 shutdown_connections 2023-10-08T06:54:29.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.330+0000 7fe090852640 1 -- 172.21.15.70:0/2242000460 wait complete. 2023-10-08T06:54:29.349 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T06:54:29.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.540+0000 7f233dd7c640 1 Processor -- start 2023-10-08T06:54:29.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.540+0000 7f233dd7c640 1 -- start start 2023-10-08T06:54:29.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.541+0000 7f233dd7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2338070b40 0x7f233815ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.541+0000 7f233dd7c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233815e8b0 con 0x7f2338065cd0 2023-10-08T06:54:29.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.541+0000 7f233dd7c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2338071040 con 0x7f233815ab00 2023-10-08T06:54:29.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.541+0000 7f233dd7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2338071320 con 0x7f2338070b40 2023-10-08T06:54:29.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.542+0000 7f2337fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2338065cd0 0x7f23380cba20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55570/0 (socket says 172.21.15.70:55570) 2023-10-08T06:54:29.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.542+0000 7f2337fff640 1 -- 172.21.15.70:0/3356066172 learned_addr learned my addr 172.21.15.70:0/3356066172 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:29.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.542+0000 7f2336ffd640 1 --2- 172.21.15.70:0/3356066172 >> v2:172.21.15.70:3300/0 conn(0x7f2338070b40 0x7f233815ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.542+0000 7f23367fc640 1 -- 172.21.15.70:0/3356066172 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260633582 0 0) 0x7f233815e8b0 con 0x7f2338065cd0 2023-10-08T06:54:29.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.542+0000 7f23367fc640 1 -- 172.21.15.70:0/3356066172 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2324003520 con 0x7f2338065cd0 2023-10-08T06:54:29.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.542+0000 7f2336ffd640 1 -- 172.21.15.70:0/3356066172 >> v1:172.21.15.181:6789/0 conn(0x7f233815ab00 legacy=0x7f23380b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.542+0000 7f2336ffd640 1 -- 172.21.15.70:0/3356066172 >> v1:172.21.15.70:6789/0 conn(0x7f2338065cd0 legacy=0x7f23380cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.542+0000 7f2336ffd640 1 -- 172.21.15.70:0/3356066172 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2338071600 con 0x7f2338070b40 2023-10-08T06:54:29.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.543+0000 7f2336ffd640 1 --2- 172.21.15.70:0/3356066172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338070b40 0x7f233815ed40 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f2320009d90 tx=0x7f232002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bd73058ea1d9c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:29.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.544+0000 7f23367fc640 1 -- 172.21.15.70:0/3356066172 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232002ccf0 con 0x7f2338070b40 2023-10-08T06:54:29.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.544+0000 7f23367fc640 1 -- 172.21.15.70:0/3356066172 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232002ce50 con 0x7f2338070b40 2023-10-08T06:54:29.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.544+0000 7f23367fc640 1 -- 172.21.15.70:0/3356066172 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2320034640 con 0x7f2338070b40 2023-10-08T06:54:29.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.544+0000 7f233dd7c640 1 -- 172.21.15.70:0/3356066172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338070b40 msgr2=0x7f233815ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.544+0000 7f233dd7c640 1 --2- 172.21.15.70:0/3356066172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338070b40 0x7f233815ed40 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f2320009d90 tx=0x7f232002bf40 comp rx=0 tx=0).stop 2023-10-08T06:54:29.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.544+0000 7f233dd7c640 1 -- 172.21.15.70:0/3356066172 shutdown_connections 2023-10-08T06:54:29.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.544+0000 7f233dd7c640 1 --2- 172.21.15.70:0/3356066172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338070b40 0x7f233815ed40 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.544+0000 7f233dd7c640 1 -- 172.21.15.70:0/3356066172 >> 172.21.15.70:0/3356066172 conn(0x7f23380b1480 msgr2=0x7f233806cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:29.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.544+0000 7f233dd7c640 1 -- 172.21.15.70:0/3356066172 shutdown_connections 2023-10-08T06:54:29.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.544+0000 7f233dd7c640 1 -- 172.21.15.70:0/3356066172 wait complete. 2023-10-08T06:54:29.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.546+0000 7f233dd7c640 1 Processor -- start 2023-10-08T06:54:29.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.546+0000 7f233dd7c640 1 -- start start 2023-10-08T06:54:29.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.547+0000 7f233dd7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338065cd0 0x7f23380ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.547+0000 7f233dd7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2338070b40 0x7f23380ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.547+0000 7f233dd7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233815ab00 0x7f23380c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.547+0000 7f233dd7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23380c7e70 con 0x7f2338065cd0 2023-10-08T06:54:29.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.547+0000 7f233dd7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23380c8150 con 0x7f233815ab00 2023-10-08T06:54:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.547+0000 7f233dd7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23380c8430 con 0x7f2338070b40 2023-10-08T06:54:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.547+0000 7f23377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338065cd0 0x7f23380ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f23377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338065cd0 0x7f23380ac2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56094/0 (socket says 172.21.15.70:56094) 2023-10-08T06:54:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f23377fe640 1 -- 172.21.15.70:0/1863654973 learned_addr learned my addr 172.21.15.70:0/1863654973 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f23377fe640 1 -- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2338070b40 msgr2=0x7f23380ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f2336ffd640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2338070b40 0x7f23380ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f23377fe640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2338070b40 0x7f23380ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f2337fff640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233815ab00 0x7f23380c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f23377fe640 1 -- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233815ab00 msgr2=0x7f23380c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f23377fe640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233815ab00 0x7f23380c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f23377fe640 1 -- 172.21.15.70:0/1863654973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2320009a70 con 0x7f2338065cd0 2023-10-08T06:54:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f23377fe640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338065cd0 0x7f23380ac2c0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f232800c400 tx=0x7f232800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f232ffff640 1 -- 172.21.15.70:0/1863654973 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23280052d0 con 0x7f2338065cd0 2023-10-08T06:54:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f232ffff640 1 -- 172.21.15.70:0/1863654973 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2328005430 con 0x7f2338065cd0 2023-10-08T06:54:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f232ffff640 1 -- 172.21.15.70:0/1863654973 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328003830 con 0x7f2338065cd0 2023-10-08T06:54:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.548+0000 7f233dd7c640 1 -- 172.21.15.70:0/1863654973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23380c8710 con 0x7f2338065cd0 2023-10-08T06:54:29.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.549+0000 7f233dd7c640 1 -- 172.21.15.70:0/1863654973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23380c76c0 con 0x7f2338065cd0 2023-10-08T06:54:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.550+0000 7f233dd7c640 1 -- 172.21.15.70:0/1863654973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23380cb1c0 con 0x7f2338065cd0 2023-10-08T06:54:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.551+0000 7f232ffff640 1 -- 172.21.15.70:0/1863654973 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2328003990 con 0x7f2338065cd0 2023-10-08T06:54:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.551+0000 7f232ffff640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f231806af90 0x7f231806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.552+0000 7f232ffff640 1 -- 172.21.15.70:0/1863654973 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f2328088ae0 con 0x7f2338065cd0 2023-10-08T06:54:29.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.552+0000 7f2336ffd640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f231806af90 0x7f231806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.553+0000 7f2336ffd640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f231806af90 0x7f231806d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2320036000 tx=0x7f2320030000 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:29.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.556+0000 7f232ffff640 1 -- 172.21.15.70:0/1863654973 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2328053cb0 con 0x7f2338065cd0 2023-10-08T06:54:29.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.745+0000 7f233dd7c640 1 -- 172.21.15.70:0/1863654973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f23380ccdc0 con 0x7f2338065cd0 2023-10-08T06:54:29.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.746+0000 7f232ffff640 1 -- 172.21.15.70:0/1863654973 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v194) v1 ==== 84+0+65 (secure 0 0 0) 0x7f23380ccdc0 con 0x7f2338065cd0 2023-10-08T06:54:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:29.747 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T06:54:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T06:54:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T06:54:29.747 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:29.747 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T06:54:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 -- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f231806af90 msgr2=0x7f231806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f231806af90 0x7f231806d450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2320036000 tx=0x7f2320030000 comp rx=0 tx=0).stop 2023-10-08T06:54:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 -- 172.21.15.70:0/1863654973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338065cd0 msgr2=0x7f23380ac2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338065cd0 0x7f23380ac2c0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f232800c400 tx=0x7f232800c8c0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 -- 172.21.15.70:0/1863654973 shutdown_connections 2023-10-08T06:54:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f231806af90 0x7f231806d450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f233815ab00 0x7f23380c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2338070b40 0x7f23380ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 --2- 172.21.15.70:0/1863654973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2338065cd0 0x7f23380ac2c0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 -- 172.21.15.70:0/1863654973 >> 172.21.15.70:0/1863654973 conn(0x7f23380b1480 msgr2=0x7f23380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:29.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 -- 172.21.15.70:0/1863654973 shutdown_connections 2023-10-08T06:54:29.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.749+0000 7f233dd7c640 1 -- 172.21.15.70:0/1863654973 wait complete. 2023-10-08T06:54:29.767 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T06:54:29.767 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T06:54:29.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.984+0000 7f0e189d7640 1 Processor -- start 2023-10-08T06:54:29.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.984+0000 7f0e189d7640 1 -- start start 2023-10-08T06:54:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.986+0000 7f0e189d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0e141432f0 0x7f0e140cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.986+0000 7f0e189d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e140b4240 0x7f0e14159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.986+0000 7f0e189d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e140cbb30 con 0x7f0e140cb6b0 2023-10-08T06:54:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.986+0000 7f0e189d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e141436d0 con 0x7f0e140b4240 2023-10-08T06:54:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.986+0000 7f0e189d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1415d440 con 0x7f0e141432f0 2023-10-08T06:54:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.986+0000 7f0e12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e140b4240 0x7f0e14159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.986+0000 7f0e12d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e140b4240 0x7f0e14159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56102/0 (socket says 172.21.15.70:56102) 2023-10-08T06:54:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.986+0000 7f0e12d9d640 1 -- 172.21.15.70:0/683498408 learned_addr learned my addr 172.21.15.70:0/683498408 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.986+0000 7f0e1259c640 1 --2- 172.21.15.70:0/683498408 >> v2:172.21.15.181:3300/0 conn(0x7f0e141432f0 0x7f0e140cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.987+0000 7f0e12d9d640 1 -- 172.21.15.70:0/683498408 >> v2:172.21.15.181:3300/0 conn(0x7f0e141432f0 msgr2=0x7f0e140cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.987+0000 7f0e12d9d640 1 --2- 172.21.15.70:0/683498408 >> v2:172.21.15.181:3300/0 conn(0x7f0e141432f0 0x7f0e140cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.987+0000 7f0e12d9d640 1 -- 172.21.15.70:0/683498408 >> v1:172.21.15.181:6789/0 conn(0x7f0e140cb6b0 legacy=0x7f0e140b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.987+0000 7f0e12d9d640 1 -- 172.21.15.70:0/683498408 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1415d690 con 0x7f0e140b4240 2023-10-08T06:54:29.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.988+0000 7f0e12d9d640 1 --2- 172.21.15.70:0/683498408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 0x7f0e14159e50 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f0e140cc1a0 tx=0x7f0e0c02da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb0c608a070ed38 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.988+0000 7f0e11d9b640 1 -- 172.21.15.70:0/683498408 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c02f850 con 0x7f0e140b4240 2023-10-08T06:54:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.988+0000 7f0e11d9b640 1 -- 172.21.15.70:0/683498408 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c02f9b0 con 0x7f0e140b4240 2023-10-08T06:54:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.988+0000 7f0e11d9b640 1 -- 172.21.15.70:0/683498408 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c02fd40 con 0x7f0e140b4240 2023-10-08T06:54:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.989+0000 7f0e189d7640 1 -- 172.21.15.70:0/683498408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 msgr2=0x7f0e14159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.989+0000 7f0e189d7640 1 --2- 172.21.15.70:0/683498408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 0x7f0e14159e50 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f0e140cc1a0 tx=0x7f0e0c02da90 comp rx=0 tx=0).stop 2023-10-08T06:54:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.989+0000 7f0e189d7640 1 -- 172.21.15.70:0/683498408 shutdown_connections 2023-10-08T06:54:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.989+0000 7f0e189d7640 1 --2- 172.21.15.70:0/683498408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 0x7f0e14159e50 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.989+0000 7f0e189d7640 1 --2- 172.21.15.70:0/683498408 >> v2:172.21.15.181:3300/0 conn(0x7f0e141432f0 0x7f0e140cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.989+0000 7f0e189d7640 1 -- 172.21.15.70:0/683498408 >> 172.21.15.70:0/683498408 conn(0x7f0e14069d50 msgr2=0x7f0e1406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.989+0000 7f0e189d7640 1 -- 172.21.15.70:0/683498408 shutdown_connections 2023-10-08T06:54:29.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.989+0000 7f0e189d7640 1 -- 172.21.15.70:0/683498408 wait complete. 2023-10-08T06:54:29.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.991+0000 7f0e189d7640 1 Processor -- start 2023-10-08T06:54:29.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.991+0000 7f0e189d7640 1 -- start start 2023-10-08T06:54:29.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.992+0000 7f0e189d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 0x7f0e14156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.992+0000 7f0e189d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e140cb6b0 0x7f0e140c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.992+0000 7f0e189d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e141432f0 0x7f0e140c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.992+0000 7f0e189d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14139c90 con 0x7f0e140b4240 2023-10-08T06:54:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.992+0000 7f0e189d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e141391d0 con 0x7f0e140cb6b0 2023-10-08T06:54:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.992+0000 7f0e189d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14156980 con 0x7f0e141432f0 2023-10-08T06:54:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.993+0000 7f0e1259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 0x7f0e14156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.993+0000 7f0e1259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 0x7f0e14156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56106/0 (socket says 172.21.15.70:56106) 2023-10-08T06:54:29.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.993+0000 7f0e1259c640 1 -- 172.21.15.70:0/2664123235 learned_addr learned my addr 172.21.15.70:0/2664123235 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:29.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.993+0000 7f0e09d9b640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e140cb6b0 0x7f0e140c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:29.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.993+0000 7f0e1259c640 1 -- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e141432f0 msgr2=0x7f0e140c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.993+0000 7f0e1259c640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e141432f0 0x7f0e140c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.993+0000 7f0e1259c640 1 -- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e140cb6b0 msgr2=0x7f0e140c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:29.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.993+0000 7f0e1259c640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e140cb6b0 0x7f0e140c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:29.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.993+0000 7f0e1259c640 1 -- 172.21.15.70:0/2664123235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c0094d0 con 0x7f0e140b4240 2023-10-08T06:54:29.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.993+0000 7f0e1259c640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 0x7f0e14156440 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f0e04007e40 tx=0x7f0e0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:29.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.994+0000 7f0e0b7fe640 1 -- 172.21.15.70:0/2664123235 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e04014620 con 0x7f0e140b4240 2023-10-08T06:54:29.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.994+0000 7f0e0b7fe640 1 -- 172.21.15.70:0/2664123235 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0400ee90 con 0x7f0e140b4240 2023-10-08T06:54:29.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.994+0000 7f0e0b7fe640 1 -- 172.21.15.70:0/2664123235 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040149c0 con 0x7f0e140b4240 2023-10-08T06:54:29.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.994+0000 7f0e189d7640 1 -- 172.21.15.70:0/2664123235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1415c5e0 con 0x7f0e140b4240 2023-10-08T06:54:29.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.994+0000 7f0e189d7640 1 -- 172.21.15.70:0/2664123235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1415cba0 con 0x7f0e140b4240 2023-10-08T06:54:29.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.995+0000 7f0e189d7640 1 -- 172.21.15.70:0/2664123235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e140ca7e0 con 0x7f0e140b4240 2023-10-08T06:54:30.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.996+0000 7f0e0b7fe640 1 -- 172.21.15.70:0/2664123235 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0e04014b20 con 0x7f0e140b4240 2023-10-08T06:54:30.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.997+0000 7f0e0b7fe640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0df406af90 0x7f0df406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:30.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.997+0000 7f0e0b7fe640 1 -- 172.21.15.70:0/2664123235 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f0e04089f30 con 0x7f0e140b4240 2023-10-08T06:54:30.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.997+0000 7f0e09d9b640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0df406af90 0x7f0df406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:30.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:29.999+0000 7f0e09d9b640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0df406af90 0x7f0df406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e140cda80 tx=0x7f0dfc009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:30.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.001+0000 7f0e0b7fe640 1 -- 172.21.15.70:0/2664123235 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e04055070 con 0x7f0e140b4240 2023-10-08T06:54:30.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.192+0000 7f0e189d7640 1 -- 172.21.15.70:0/2664123235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0e1404e160 con 0x7f0e140b4240 2023-10-08T06:54:30.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:54:30.820 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:54:30.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.993+0000 7f0e0b7fe640 1 -- 172.21.15.70:0/2664123235 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v195) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0e04058690 con 0x7f0e140b4240 2023-10-08T06:54:30.995 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T06:54:30.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 -- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0df406af90 msgr2=0x7f0df406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:30.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0df406af90 0x7f0df406d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0e140cda80 tx=0x7f0dfc009040 comp rx=0 tx=0).stop 2023-10-08T06:54:30.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 -- 172.21.15.70:0/2664123235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 msgr2=0x7f0e14156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:30.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 0x7f0e14156440 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f0e04007e40 tx=0x7f0e0400e4b0 comp rx=0 tx=0).stop 2023-10-08T06:54:30.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 -- 172.21.15.70:0/2664123235 shutdown_connections 2023-10-08T06:54:30.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f0df406af90 0x7f0df406d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:30.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e141432f0 0x7f0e140c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:30.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e140cb6b0 0x7f0e140c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:30.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 --2- 172.21.15.70:0/2664123235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e140b4240 0x7f0e14156440 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:30.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 -- 172.21.15.70:0/2664123235 >> 172.21.15.70:0/2664123235 conn(0x7f0e14069d50 msgr2=0x7f0e14141c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:30.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 -- 172.21.15.70:0/2664123235 shutdown_connections 2023-10-08T06:54:30.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:30.997+0000 7f0e189d7640 1 -- 172.21.15.70:0/2664123235 wait complete. 2023-10-08T06:54:31.024 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T06:54:31.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.226+0000 7f9c09db0640 1 Processor -- start 2023-10-08T06:54:31.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.226+0000 7f9c09db0640 1 -- start start 2023-10-08T06:54:31.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.228+0000 7f9c09db0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9c040b0860 0x7f9c040cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:31.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.228+0000 7f9c09db0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0415d140 con 0x7f9c040b4560 2023-10-08T06:54:31.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.228+0000 7f9c09db0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0415d420 con 0x7f9c04159490 2023-10-08T06:54:31.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.228+0000 7f9c09db0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0415ddc0 con 0x7f9c040b0860 2023-10-08T06:54:31.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.228+0000 7f9c037fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9c040b4560 0x7f9c040cb500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55574/0 (socket says 172.21.15.70:55574) 2023-10-08T06:54:31.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.228+0000 7f9c037fe640 1 -- 172.21.15.70:0/1077577041 learned_addr learned my addr 172.21.15.70:0/1077577041 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:31.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188184394 0 0) 0x7f9c0415d140 con 0x7f9c040b4560 2023-10-08T06:54:31.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf0005730 con 0x7f9c040b4560 2023-10-08T06:54:31.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9bfbfff640 1 --2- 172.21.15.70:0/1077577041 >> v2:172.21.15.181:3300/0 conn(0x7f9c040b0860 0x7f9c040cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:31.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3994807216 0 0) 0x7f9bf0005730 con 0x7f9c040b4560 2023-10-08T06:54:31.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 >> v2:172.21.15.181:3300/0 conn(0x7f9c040b0860 msgr2=0x7f9c040cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:31.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 --2- 172.21.15.70:0/1077577041 >> v2:172.21.15.181:3300/0 conn(0x7f9c040b0860 0x7f9c040cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:31.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 >> v1:172.21.15.181:6789/0 conn(0x7f9c04159490 legacy=0x7f9c04070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:31.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0415d700 con 0x7f9c040b4560 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9bf4001670 con 0x7f9c040b4560 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 >> v1:172.21.15.70:6789/0 conn(0x7f9c040b4560 legacy=0x7f9c040cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 --2- 172.21.15.70:0/1077577041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0005bd0 0x7f9bf0005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 --2- 172.21.15.70:0/1077577041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00069d0 0x7f9bf0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 --2- 172.21.15.70:0/1077577041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0007390 0x7f9bf000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0415ddc0 con 0x7f9bf0005bd0 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf000e290 con 0x7f9bf0007390 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.229+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf000e450 con 0x7f9bf00069d0 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.230+0000 7f9c037fe640 1 --2- 172.21.15.70:0/1077577041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0005bd0 0x7f9bf0005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.230+0000 7f9c03fff640 1 --2- 172.21.15.70:0/1077577041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0007390 0x7f9bf000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:31.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.230+0000 7f9c037fe640 1 -- 172.21.15.70:0/1077577041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00069d0 msgr2=0x7f9bf0006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:31.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.230+0000 7f9c037fe640 1 --2- 172.21.15.70:0/1077577041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00069d0 0x7f9bf0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:31.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.230+0000 7f9c037fe640 1 -- 172.21.15.70:0/1077577041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0007390 msgr2=0x7f9bf000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:31.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.230+0000 7f9c037fe640 1 --2- 172.21.15.70:0/1077577041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0007390 0x7f9bf000db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:31.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.230+0000 7f9c037fe640 1 -- 172.21.15.70:0/1077577041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0005730 con 0x7f9bf0005bd0 2023-10-08T06:54:31.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.230+0000 7f9c03fff640 1 -- 172.21.15.70:0/1077577041 reap_dead start 2023-10-08T06:54:31.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.231+0000 7f9c037fe640 1 --2- 172.21.15.70:0/1077577041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0005bd0 0x7f9bf0005ff0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c0e0 tx=0x7f9bf0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992552dfde34fd62 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:31.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.232+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4005ad0 con 0x7f9bf0005bd0 2023-10-08T06:54:31.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.232+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf4005c30 con 0x7f9bf0005bd0 2023-10-08T06:54:31.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.232+0000 7f9c02ffd640 1 -- 172.21.15.70:0/1077577041 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf402c7d0 con 0x7f9bf0005bd0 2023-10-08T06:54:31.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.232+0000 7f9c09db0640 1 -- 172.21.15.70:0/1077577041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0005bd0 msgr2=0x7f9bf0005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:31.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.232+0000 7f9c09db0640 1 --2- 172.21.15.70:0/1077577041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0005bd0 0x7f9bf0005ff0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c0e0 tx=0x7f9bf0007a00 comp rx=0 tx=0).stop 2023-10-08T06:54:31.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.232+0000 7f9c09db0640 1 -- 172.21.15.70:0/1077577041 shutdown_connections 2023-10-08T06:54:31.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.232+0000 7f9c09db0640 1 --2- 172.21.15.70:0/1077577041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0005bd0 0x7f9bf0005ff0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:31.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.232+0000 7f9c09db0640 1 -- 172.21.15.70:0/1077577041 >> 172.21.15.70:0/1077577041 conn(0x7f9c0406a140 msgr2=0x7f9c0406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:31.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.232+0000 7f9c09db0640 1 -- 172.21.15.70:0/1077577041 shutdown_connections 2023-10-08T06:54:31.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.232+0000 7f9c09db0640 1 -- 172.21.15.70:0/1077577041 wait complete. 2023-10-08T06:54:31.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.234+0000 7f9c09db0640 1 Processor -- start 2023-10-08T06:54:31.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.234+0000 7f9c09db0640 1 -- start start 2023-10-08T06:54:31.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.235+0000 7f9c09db0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0005bd0 0x7f9c040b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:31.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.235+0000 7f9c09db0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414b220 0x7f9c041f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:31.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.235+0000 7f9c09db0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c0414b6a0 0x7f9c041f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:31.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.235+0000 7f9c09db0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041f7320 con 0x7f9c0414b6a0 2023-10-08T06:54:31.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.235+0000 7f9c09db0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041f6710 con 0x7f9bf0005bd0 2023-10-08T06:54:31.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.235+0000 7f9c09db0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041f69f0 con 0x7f9c0414b220 2023-10-08T06:54:31.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c03fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c0414b6a0 0x7f9c041f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:31.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c03fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c0414b6a0 0x7f9c041f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56134/0 (socket says 172.21.15.70:56134) 2023-10-08T06:54:31.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c03fff640 1 -- 172.21.15.70:0/3171294156 learned_addr learned my addr 172.21.15.70:0/3171294156 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:31.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9bfbfff640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414b220 0x7f9c041f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:31.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c03fff640 1 -- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414b220 msgr2=0x7f9c041f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:31.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c037fe640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0005bd0 0x7f9c040b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c03fff640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414b220 0x7f9c041f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c03fff640 1 -- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0005bd0 msgr2=0x7f9c040b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c03fff640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0005bd0 0x7f9c040b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c03fff640 1 -- 172.21.15.70:0/3171294156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c041f6cd0 con 0x7f9c0414b6a0 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c03fff640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c0414b6a0 0x7f9c041f3b80 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f9c0406daf0 tx=0x7f9bfc00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.236+0000 7f9c00ff9640 1 -- 172.21.15.70:0/3171294156 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc003d80 con 0x7f9c0414b6a0 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.237+0000 7f9c00ff9640 1 -- 172.21.15.70:0/3171294156 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc005070 con 0x7f9c0414b6a0 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.237+0000 7f9c09db0640 1 -- 172.21.15.70:0/3171294156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c041f6fb0 con 0x7f9c0414b6a0 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.237+0000 7f9c00ff9640 1 -- 172.21.15.70:0/3171294156 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc005420 con 0x7f9c0414b6a0 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.237+0000 7f9c09db0640 1 -- 172.21.15.70:0/3171294156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c040b3fc0 con 0x7f9c0414b6a0 2023-10-08T06:54:31.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.238+0000 7f9c09db0640 1 -- 172.21.15.70:0/3171294156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c040c23a0 con 0x7f9c0414b6a0 2023-10-08T06:54:31.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.239+0000 7f9c00ff9640 1 -- 172.21.15.70:0/3171294156 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9bfc005580 con 0x7f9c0414b6a0 2023-10-08T06:54:31.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.240+0000 7f9c00ff9640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9be406afc0 0x7f9be406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:31.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.240+0000 7f9c00ff9640 1 -- 172.21.15.70:0/3171294156 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5663+0+0 (secure 0 0 0) 0x7f9bfc0891d0 con 0x7f9c0414b6a0 2023-10-08T06:54:31.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.241+0000 7f9c037fe640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9be406afc0 0x7f9be406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:31.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.242+0000 7f9c037fe640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9be406afc0 0x7f9be406d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4034000 tx=0x7f9bf4009030 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:31.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.245+0000 7f9c00ff9640 1 -- 172.21.15.70:0/3171294156 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c040c23a0 con 0x7f9c0414b6a0 2023-10-08T06:54:31.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:31.433+0000 7f9c09db0640 1 -- 172.21.15.70:0/3171294156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9c040c5600 con 0x7f9c0414b6a0 2023-10-08T06:54:32.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.013+0000 7f9c00ff9640 1 -- 172.21.15.70:0/3171294156 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v196) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9bfc053ff0 con 0x7f9c0414b6a0 2023-10-08T06:54:32.014 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T06:54:32.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.017+0000 7f9c09db0640 1 -- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9be406afc0 msgr2=0x7f9be406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:32.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.018+0000 7f9c09db0640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9be406afc0 0x7f9be406d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4034000 tx=0x7f9bf4009030 comp rx=0 tx=0).stop 2023-10-08T06:54:32.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.018+0000 7f9c09db0640 1 -- 172.21.15.70:0/3171294156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c0414b6a0 msgr2=0x7f9c041f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:32.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.018+0000 7f9c09db0640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c0414b6a0 0x7f9c041f3b80 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f9c0406daf0 tx=0x7f9bfc00d9d0 comp rx=0 tx=0).stop 2023-10-08T06:54:32.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.019+0000 7f9c09db0640 1 -- 172.21.15.70:0/3171294156 shutdown_connections 2023-10-08T06:54:32.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.019+0000 7f9c09db0640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f9be406afc0 0x7f9be406d480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:32.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.019+0000 7f9c09db0640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c0414b6a0 0x7f9c041f3b80 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:32.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.019+0000 7f9c09db0640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9c0414b220 0x7f9c041f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:32.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.019+0000 7f9c09db0640 1 --2- 172.21.15.70:0/3171294156 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf0005bd0 0x7f9c040b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:32.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.019+0000 7f9c09db0640 1 -- 172.21.15.70:0/3171294156 >> 172.21.15.70:0/3171294156 conn(0x7f9c0406a140 msgr2=0x7f9c0414cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:32.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.020+0000 7f9c09db0640 1 -- 172.21.15.70:0/3171294156 shutdown_connections 2023-10-08T06:54:32.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.031+0000 7f9c09db0640 1 -- 172.21.15.70:0/3171294156 wait complete. 2023-10-08T06:54:32.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T06:54:32.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.246+0000 7fb7b1bd0640 1 Processor -- start 2023-10-08T06:54:32.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.247+0000 7fb7b1bd0640 1 -- start start 2023-10-08T06:54:32.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.248+0000 7fb7b1bd0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7ac0cb150 0x7fb7ac0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:32.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.248+0000 7fb7b1bd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b1f20 0x7fb7ac0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:32.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.248+0000 7fb7b1bd0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ac06e1e0 con 0x7fb7ac161690 2023-10-08T06:54:32.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.248+0000 7fb7b1bd0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ac0cbb40 con 0x7fb7ac0cb150 2023-10-08T06:54:32.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.248+0000 7fb7b1bd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac06cc50 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.249+0000 7fb7aaffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7ac0cb150 0x7fb7ac0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:32.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.249+0000 7fb7aaffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7ac0cb150 0x7fb7ac0cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39330/0 (socket says 172.21.15.70:39330) 2023-10-08T06:54:32.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.249+0000 7fb7aaffd640 1 -- 172.21.15.70:0/181283866 learned_addr learned my addr 172.21.15.70:0/181283866 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:32.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.249+0000 7fb7abfff640 1 --2- 172.21.15.70:0/181283866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b1f20 0x7fb7ac0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:32.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.250+0000 7fb7aaffd640 1 -- 172.21.15.70:0/181283866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b1f20 msgr2=0x7fb7ac0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:32.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.250+0000 7fb7aaffd640 1 --2- 172.21.15.70:0/181283866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b1f20 0x7fb7ac0b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:32.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.250+0000 7fb7aaffd640 1 -- 172.21.15.70:0/181283866 >> v1:172.21.15.181:6789/0 conn(0x7fb7ac161690 legacy=0x7fb7ac06dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:32.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.250+0000 7fb7aaffd640 1 -- 172.21.15.70:0/181283866 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ac06cf30 con 0x7fb7ac0cb150 2023-10-08T06:54:32.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.251+0000 7fb7aaffd640 1 --2- 172.21.15.70:0/181283866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac0cb150 0x7fb7ac0cb600 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb794009870 tx=0x7fb79402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=488146551c5d248a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:32.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.252+0000 7fb7aa7fc640 1 -- 172.21.15.70:0/181283866 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794030840 con 0x7fb7ac0cb150 2023-10-08T06:54:32.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.252+0000 7fb7aa7fc640 1 -- 172.21.15.70:0/181283866 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7940309a0 con 0x7fb7ac0cb150 2023-10-08T06:54:32.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.252+0000 7fb7aa7fc640 1 -- 172.21.15.70:0/181283866 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794030d30 con 0x7fb7ac0cb150 2023-10-08T06:54:32.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.252+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/181283866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac0cb150 msgr2=0x7fb7ac0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:32.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.252+0000 7fb7b1bd0640 1 --2- 172.21.15.70:0/181283866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac0cb150 0x7fb7ac0cb600 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb794009870 tx=0x7fb79402b980 comp rx=0 tx=0).stop 2023-10-08T06:54:32.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.252+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/181283866 shutdown_connections 2023-10-08T06:54:32.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.252+0000 7fb7b1bd0640 1 --2- 172.21.15.70:0/181283866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac0b1f20 0x7fb7ac0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:32.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.252+0000 7fb7b1bd0640 1 --2- 172.21.15.70:0/181283866 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac0cb150 0x7fb7ac0cb600 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:32.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.252+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/181283866 >> 172.21.15.70:0/181283866 conn(0x7fb7ac06a5d0 msgr2=0x7fb7ac0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.252+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/181283866 shutdown_connections 2023-10-08T06:54:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.253+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/181283866 wait complete. 2023-10-08T06:54:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.254+0000 7fb7b1bd0640 1 Processor -- start 2023-10-08T06:54:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.254+0000 7fb7b1bd0640 1 -- start start 2023-10-08T06:54:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.255+0000 7fb7b1bd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ac0b1f20 0x7fb7ac070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.255+0000 7fb7b1bd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac0cb150 0x7fb7ac0be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.255+0000 7fb7b1bd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac161690 0x7fb7ac163cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.255+0000 7fb7b1bd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac0c42b0 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.255+0000 7fb7b1bd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac161a70 con 0x7fb7ac0cb150 2023-10-08T06:54:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.255+0000 7fb7b1bd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ac1648d0 con 0x7fb7ac161690 2023-10-08T06:54:32.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7ab7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ac0b1f20 0x7fb7ac070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:32.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7ab7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ac0b1f20 0x7fb7ac070420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56146/0 (socket says 172.21.15.70:56146) 2023-10-08T06:54:32.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7ab7fe640 1 -- 172.21.15.70:0/3052944376 learned_addr learned my addr 172.21.15.70:0/3052944376 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:32.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7aaffd640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac0cb150 0x7fb7ac0be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:32.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7abfff640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac161690 0x7fb7ac163cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:32.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7ab7fe640 1 -- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac161690 msgr2=0x7fb7ac163cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:32.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7ab7fe640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac161690 0x7fb7ac163cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:32.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7ab7fe640 1 -- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac0cb150 msgr2=0x7fb7ac0be9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:32.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7ab7fe640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac0cb150 0x7fb7ac0be9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:32.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7ab7fe640 1 -- 172.21.15.70:0/3052944376 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7940094d0 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.256+0000 7fb7ab7fe640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ac0b1f20 0x7fb7ac070420 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00c710 tx=0x7fb79c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.257+0000 7fb7a3fff640 1 -- 172.21.15.70:0/3052944376 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c003b90 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.257+0000 7fb7a3fff640 1 -- 172.21.15.70:0/3052944376 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c003cf0 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.257+0000 7fb7a3fff640 1 -- 172.21.15.70:0/3052944376 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00aa90 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.257+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/3052944376 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7ac161d30 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.257+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/3052944376 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ac1622f0 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.258+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/3052944376 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ac04e160 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.260+0000 7fb7a3fff640 1 -- 172.21.15.70:0/3052944376 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb79c00abf0 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.261+0000 7fb7a3fff640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb78c06af90 0x7fb78c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:32.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.261+0000 7fb7a3fff640 1 -- 172.21.15.70:0/3052944376 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6045+0+0 (secure 0 0 0) 0x7fb79c08a1d0 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.262+0000 7fb7aaffd640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb78c06af90 0x7fb78c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:32.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.263+0000 7fb7aaffd640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb78c06af90 0x7fb78c06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb794036000 tx=0x7fb794008030 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:32.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.265+0000 7fb7a3fff640 1 -- 172.21.15.70:0/3052944376 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb79c055080 con 0x7fb7ac0b1f20 2023-10-08T06:54:32.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:32.489+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/3052944376 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb7ac06dce0 con 0x7fb7ac0b1f20 2023-10-08T06:54:33.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.045+0000 7fb7a3fff640 1 -- 172.21.15.70:0/3052944376 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v88) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb79c0586a0 con 0x7fb7ac0b1f20 2023-10-08T06:54:33.046 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 23 and data pool 24 2023-10-08T06:54:33.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.050+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb78c06af90 msgr2=0x7fb78c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.050+0000 7fb7b1bd0640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb78c06af90 0x7fb78c06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb794036000 tx=0x7fb794008030 comp rx=0 tx=0).stop 2023-10-08T06:54:33.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.050+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/3052944376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ac0b1f20 msgr2=0x7fb7ac070420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.050+0000 7fb7b1bd0640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ac0b1f20 0x7fb7ac070420 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00c710 tx=0x7fb79c00cbd0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.051+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/3052944376 shutdown_connections 2023-10-08T06:54:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.051+0000 7fb7b1bd0640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fb78c06af90 0x7fb78c06d450 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb794036000 tx=0x7fb794008030 comp rx=0 tx=0).stop 2023-10-08T06:54:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.051+0000 7fb7b1bd0640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7ac161690 0x7fb7ac163cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.051+0000 7fb7b1bd0640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7ac0cb150 0x7fb7ac0be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.051+0000 7fb7b1bd0640 1 --2- 172.21.15.70:0/3052944376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7ac0b1f20 0x7fb7ac070420 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.051+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/3052944376 >> 172.21.15.70:0/3052944376 conn(0x7fb7ac06a5d0 msgr2=0x7fb7ac160d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:33.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.054+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/3052944376 shutdown_connections 2023-10-08T06:54:33.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.054+0000 7fb7b1bd0640 1 -- 172.21.15.70:0/3052944376 wait complete. 2023-10-08T06:54:33.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:54:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.269+0000 7f673f4af640 1 Processor -- start 2023-10-08T06:54:33.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.269+0000 7f673f4af640 1 -- start start 2023-10-08T06:54:33.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.271+0000 7f673f4af640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f673815ab40 0x7f67380ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.271+0000 7f673f4af640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380c90b0 0x7f67380adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.271+0000 7f673f4af640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f673815b2b0 con 0x7f67380cae70 2023-10-08T06:54:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.271+0000 7f673f4af640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673815bf30 con 0x7f673815ab40 2023-10-08T06:54:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.271+0000 7f673f4af640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673815f7a0 con 0x7f67380c90b0 2023-10-08T06:54:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.272+0000 7f673ca4a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380c90b0 0x7f67380adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.272+0000 7f673d24b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f673815ab40 0x7f67380ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.272+0000 7f673ca4a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380c90b0 0x7f67380adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:33274/0 (socket says 172.21.15.70:33274) 2023-10-08T06:54:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.272+0000 7f673ca4a640 1 -- 172.21.15.70:0/2000626710 learned_addr learned my addr 172.21.15.70:0/2000626710 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.273+0000 7f673ca4a640 1 -- 172.21.15.70:0/2000626710 >> v2:172.21.15.181:3300/0 conn(0x7f673815ab40 msgr2=0x7f67380ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.273+0000 7f673ca4a640 1 --2- 172.21.15.70:0/2000626710 >> v2:172.21.15.181:3300/0 conn(0x7f673815ab40 0x7f67380ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.273+0000 7f673ca4a640 1 -- 172.21.15.70:0/2000626710 >> v1:172.21.15.181:6789/0 conn(0x7f67380cae70 legacy=0x7f67380702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.273+0000 7f673ca4a640 1 -- 172.21.15.70:0/2000626710 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673815b590 con 0x7f67380c90b0 2023-10-08T06:54:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.274+0000 7f673ca4a640 1 --2- 172.21.15.70:0/2000626710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380c90b0 0x7f67380adc60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6728009870 tx=0x7f672802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7420804a333578e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:33.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.274+0000 7f672ffff640 1 -- 172.21.15.70:0/2000626710 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728031640 con 0x7f67380c90b0 2023-10-08T06:54:33.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.274+0000 7f672ffff640 1 -- 172.21.15.70:0/2000626710 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67280317a0 con 0x7f67380c90b0 2023-10-08T06:54:33.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.274+0000 7f672ffff640 1 -- 172.21.15.70:0/2000626710 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728031b30 con 0x7f67380c90b0 2023-10-08T06:54:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.275+0000 7f673f4af640 1 -- 172.21.15.70:0/2000626710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380c90b0 msgr2=0x7f67380adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.275+0000 7f673f4af640 1 --2- 172.21.15.70:0/2000626710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380c90b0 0x7f67380adc60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6728009870 tx=0x7f672802b8f0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.275+0000 7f673f4af640 1 -- 172.21.15.70:0/2000626710 shutdown_connections 2023-10-08T06:54:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.275+0000 7f673f4af640 1 --2- 172.21.15.70:0/2000626710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380c90b0 0x7f67380adc60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.275+0000 7f673f4af640 1 --2- 172.21.15.70:0/2000626710 >> v2:172.21.15.181:3300/0 conn(0x7f673815ab40 0x7f67380ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.275+0000 7f673f4af640 1 -- 172.21.15.70:0/2000626710 >> 172.21.15.70:0/2000626710 conn(0x7f673806a380 msgr2=0x7f67380b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.275+0000 7f673f4af640 1 -- 172.21.15.70:0/2000626710 shutdown_connections 2023-10-08T06:54:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.275+0000 7f673f4af640 1 -- 172.21.15.70:0/2000626710 wait complete. 2023-10-08T06:54:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.277+0000 7f673f4af640 1 Processor -- start 2023-10-08T06:54:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.277+0000 7f673f4af640 1 -- start start 2023-10-08T06:54:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.278+0000 7f673f4af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380c90b0 0x7f6738150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.278+0000 7f673f4af640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380cae70 0x7f673814f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.278+0000 7f673f4af640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f673815ab40 0x7f67381516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.278+0000 7f673f4af640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6738159c20 con 0x7f67380c90b0 2023-10-08T06:54:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.278+0000 7f673f4af640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67380ac470 con 0x7f673815ab40 2023-10-08T06:54:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.278+0000 7f673f4af640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67380ac750 con 0x7f67380cae70 2023-10-08T06:54:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.278+0000 7f673d24b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380c90b0 0x7f6738150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.278+0000 7f673d24b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380c90b0 0x7f6738150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56152/0 (socket says 172.21.15.70:56152) 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.278+0000 7f673d24b640 1 -- 172.21.15.70:0/623476057 learned_addr learned my addr 172.21.15.70:0/623476057 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f673ca4a640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380cae70 0x7f673814f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f673d24b640 1 -- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380cae70 msgr2=0x7f673814f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f673d24b640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380cae70 0x7f673814f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f673da4c640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f673815ab40 0x7f67381516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f673d24b640 1 -- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f673815ab40 msgr2=0x7f67381516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f673d24b640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f673815ab40 0x7f67381516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f673d24b640 1 -- 172.21.15.70:0/623476057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67280094d0 con 0x7f67380c90b0 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f673d24b640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380c90b0 0x7f6738150090 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f673400e9b0 tx=0x7f673400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f672dffb640 1 -- 172.21.15.70:0/623476057 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734009670 con 0x7f67380c90b0 2023-10-08T06:54:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f672dffb640 1 -- 172.21.15.70:0/623476057 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673400c9a0 con 0x7f67380c90b0 2023-10-08T06:54:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f673f4af640 1 -- 172.21.15.70:0/623476057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67380aca30 con 0x7f67380c90b0 2023-10-08T06:54:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.279+0000 7f672dffb640 1 -- 172.21.15.70:0/623476057 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400cd50 con 0x7f67380c90b0 2023-10-08T06:54:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.280+0000 7f673f4af640 1 -- 172.21.15.70:0/623476057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67380acf50 con 0x7f67380c90b0 2023-10-08T06:54:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.281+0000 7f673f4af640 1 -- 172.21.15.70:0/623476057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67380ac2a0 con 0x7f67380c90b0 2023-10-08T06:54:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.282+0000 7f672dffb640 1 -- 172.21.15.70:0/623476057 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6734013470 con 0x7f67380c90b0 2023-10-08T06:54:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.283+0000 7f672dffb640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f671c06af90 0x7f671c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.283+0000 7f672dffb640 1 -- 172.21.15.70:0/623476057 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f6734089c20 con 0x7f67380c90b0 2023-10-08T06:54:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.283+0000 7f673ca4a640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f671c06af90 0x7f671c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.285+0000 7f673ca4a640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f671c06af90 0x7f671c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6728036000 tx=0x7f672803c040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:33.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.287+0000 7f672dffb640 1 -- 172.21.15.70:0/623476057 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6734054980 con 0x7f67380c90b0 2023-10-08T06:54:33.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.474+0000 7f673f4af640 1 -- 172.21.15.70:0/623476057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6738067490 con 0x7f67380c90b0 2023-10-08T06:54:33.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.476+0000 7f672dffb640 1 -- 172.21.15.70:0/623476057 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16903 (secure 0 0 0) 0x7f6734057fa0 con 0x7f67380c90b0 2023-10-08T06:54:33.477 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:33.477 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":197,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:54:32.490548+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:54:30.246375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-08T06:54:31.487900+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:54:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 -- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f671c06af90 msgr2=0x7f671c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f671c06af90 0x7f671c06d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6728036000 tx=0x7f672803c040 comp rx=0 tx=0).stop 2023-10-08T06:54:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 -- 172.21.15.70:0/623476057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380c90b0 msgr2=0x7f6738150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380c90b0 0x7f6738150090 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f673400e9b0 tx=0x7f673400ee70 comp rx=0 tx=0).stop 2023-10-08T06:54:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 -- 172.21.15.70:0/623476057 shutdown_connections 2023-10-08T06:54:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f671c06af90 0x7f671c06d450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f673815ab40 0x7f67381516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f67380cae70 0x7f673814f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 --2- 172.21.15.70:0/623476057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67380c90b0 0x7f6738150090 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 -- 172.21.15.70:0/623476057 >> 172.21.15.70:0/623476057 conn(0x7f673806a380 msgr2=0x7f67380ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 -- 172.21.15.70:0/623476057 shutdown_connections 2023-10-08T06:54:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.479+0000 7f673f4af640 1 -- 172.21.15.70:0/623476057 wait complete. 2023-10-08T06:54:33.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:54:33.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.698+0000 7fcc38a72640 1 Processor -- start 2023-10-08T06:54:33.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.699+0000 7fcc38a72640 1 -- start start 2023-10-08T06:54:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.700+0000 7fcc38a72640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc340b07d0 0x7fcc341614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.700+0000 7fcc38a72640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc340c9020 con 0x7fcc340cab40 2023-10-08T06:54:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.700+0000 7fcc38a72640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc340ccc20 con 0x7fcc3415cca0 2023-10-08T06:54:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.700+0000 7fcc38a72640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc340c92e0 con 0x7fcc340b07d0 2023-10-08T06:54:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.701+0000 7fcc2bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc340b07d0 0x7fcc341614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.701+0000 7fcc2bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc340b07d0 0x7fcc341614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60582/0 (socket says 172.21.15.70:60582) 2023-10-08T06:54:33.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.701+0000 7fcc2bfff640 1 -- 172.21.15.70:0/1188516325 learned_addr learned my addr 172.21.15.70:0/1188516325 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:33.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.701+0000 7fcc33fff640 1 --1- 172.21.15.70:0/1188516325 >> v1:172.21.15.70:6789/0 conn(0x7fcc340cab40 0x7fcc340c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37728/0 (socket says 172.21.15.70:37728) 2023-10-08T06:54:33.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.701+0000 7fcc2bfff640 1 -- 172.21.15.70:0/1188516325 >> v1:172.21.15.181:6789/0 conn(0x7fcc3415cca0 legacy=0x7fcc340b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:33.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.701+0000 7fcc2bfff640 1 -- 172.21.15.70:0/1188516325 >> v1:172.21.15.70:6789/0 conn(0x7fcc340cab40 legacy=0x7fcc340c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.701+0000 7fcc2bfff640 1 -- 172.21.15.70:0/1188516325 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc340cc7b0 con 0x7fcc340b07d0 2023-10-08T06:54:33.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.702+0000 7fcc2bfff640 1 --2- 172.21.15.70:0/1188516325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc340b07d0 0x7fcc341614b0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009870 tx=0x7fcc1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d5e1c98e5fb385 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:33.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.703+0000 7fcc32ffd640 1 -- 172.21.15.70:0/1188516325 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0317e0 con 0x7fcc340b07d0 2023-10-08T06:54:33.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.703+0000 7fcc32ffd640 1 -- 172.21.15.70:0/1188516325 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c031940 con 0x7fcc340b07d0 2023-10-08T06:54:33.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.703+0000 7fcc32ffd640 1 -- 172.21.15.70:0/1188516325 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c031cd0 con 0x7fcc340b07d0 2023-10-08T06:54:33.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.703+0000 7fcc38a72640 1 -- 172.21.15.70:0/1188516325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc340b07d0 msgr2=0x7fcc341614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.703+0000 7fcc38a72640 1 --2- 172.21.15.70:0/1188516325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc340b07d0 0x7fcc341614b0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009870 tx=0x7fcc1c02b880 comp rx=0 tx=0).stop 2023-10-08T06:54:33.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.703+0000 7fcc38a72640 1 -- 172.21.15.70:0/1188516325 shutdown_connections 2023-10-08T06:54:33.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.703+0000 7fcc38a72640 1 --2- 172.21.15.70:0/1188516325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc340b07d0 0x7fcc341614b0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.703+0000 7fcc38a72640 1 -- 172.21.15.70:0/1188516325 >> 172.21.15.70:0/1188516325 conn(0x7fcc340cb0c0 msgr2=0x7fcc340cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.703+0000 7fcc38a72640 1 -- 172.21.15.70:0/1188516325 shutdown_connections 2023-10-08T06:54:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.703+0000 7fcc38a72640 1 -- 172.21.15.70:0/1188516325 wait complete. 2023-10-08T06:54:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.705+0000 7fcc38a72640 1 Processor -- start 2023-10-08T06:54:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.705+0000 7fcc38a72640 1 -- start start 2023-10-08T06:54:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.706+0000 7fcc38a72640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc340cab40 0x7fcc34149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.706+0000 7fcc38a72640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3415cca0 0x7fcc340c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.706+0000 7fcc38a72640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc340c6660 0x7fcc340ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.706+0000 7fcc38a72640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3414a2f0 con 0x7fcc3415cca0 2023-10-08T06:54:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.706+0000 7fcc38a72640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340b1060 con 0x7fcc340c6660 2023-10-08T06:54:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.706+0000 7fcc38a72640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc340c6ec0 con 0x7fcc340cab40 2023-10-08T06:54:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.706+0000 7fcc2bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3415cca0 0x7fcc340c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc2bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3415cca0 0x7fcc340c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60598/0 (socket says 172.21.15.70:60598) 2023-10-08T06:54:33.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc2bfff640 1 -- 172.21.15.70:0/230225966 learned_addr learned my addr 172.21.15.70:0/230225966 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc337fe640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc340cab40 0x7fcc34149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc33fff640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc340c6660 0x7fcc340ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc2bfff640 1 -- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc340cab40 msgr2=0x7fcc34149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc2bfff640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc340cab40 0x7fcc34149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc2bfff640 1 -- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc340c6660 msgr2=0x7fcc340ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc2bfff640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc340c6660 0x7fcc340ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc2bfff640 1 -- 172.21.15.70:0/230225966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c0094d0 con 0x7fcc3415cca0 2023-10-08T06:54:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc337fe640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc340cab40 0x7fcc34149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc33fff640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc340c6660 0x7fcc340ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.707+0000 7fcc2bfff640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3415cca0 0x7fcc340c6120 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0090b0 tx=0x7fcc1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.708+0000 7fcc30ff9640 1 -- 172.21.15.70:0/230225966 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c004230 con 0x7fcc3415cca0 2023-10-08T06:54:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.708+0000 7fcc30ff9640 1 -- 172.21.15.70:0/230225966 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c004390 con 0x7fcc3415cca0 2023-10-08T06:54:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.708+0000 7fcc30ff9640 1 -- 172.21.15.70:0/230225966 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c003830 con 0x7fcc3415cca0 2023-10-08T06:54:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.708+0000 7fcc38a72640 1 -- 172.21.15.70:0/230225966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc340b12b0 con 0x7fcc3415cca0 2023-10-08T06:54:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.708+0000 7fcc38a72640 1 -- 172.21.15.70:0/230225966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc340c6b50 con 0x7fcc3415cca0 2023-10-08T06:54:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.709+0000 7fcc38a72640 1 -- 172.21.15.70:0/230225966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc340c7050 con 0x7fcc3415cca0 2023-10-08T06:54:33.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.710+0000 7fcc30ff9640 1 -- 172.21.15.70:0/230225966 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcc1c043070 con 0x7fcc3415cca0 2023-10-08T06:54:33.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.711+0000 7fcc30ff9640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fcc1406afc0 0x7fcc1406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:33.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.711+0000 7fcc30ff9640 1 -- 172.21.15.70:0/230225966 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fcc1c0ab8c0 con 0x7fcc3415cca0 2023-10-08T06:54:33.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.711+0000 7fcc337fe640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fcc1406afc0 0x7fcc1406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:33.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.713+0000 7fcc337fe640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fcc1406afc0 0x7fcc1406d480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcc24003eb0 tx=0x7fcc24009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:33.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.715+0000 7fcc30ff9640 1 -- 172.21.15.70:0/230225966 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc1c0766b0 con 0x7fcc3415cca0 2023-10-08T06:54:33.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.911+0000 7fcc38a72640 1 -- 172.21.15.70:0/230225966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc3404e160 con 0x7fcc3415cca0 2023-10-08T06:54:33.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.914+0000 7fcc30ff9640 1 -- 172.21.15.70:0/230225966 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16903 (secure 0 0 0) 0x7fcc1c079cd0 con 0x7fcc3415cca0 2023-10-08T06:54:33.915 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:33.915 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":197,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:54:32.490548+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:54:30.246375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-08T06:54:31.487900+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:54:33.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.916+0000 7fcc38a72640 1 -- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fcc1406afc0 msgr2=0x7fcc1406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.916+0000 7fcc38a72640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fcc1406afc0 0x7fcc1406d480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcc24003eb0 tx=0x7fcc24009040 comp rx=0 tx=0).stop 2023-10-08T06:54:33.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.917+0000 7fcc38a72640 1 -- 172.21.15.70:0/230225966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3415cca0 msgr2=0x7fcc340c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:33.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.917+0000 7fcc38a72640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3415cca0 0x7fcc340c6120 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0090b0 tx=0x7fcc1c0046c0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.917+0000 7fcc38a72640 1 -- 172.21.15.70:0/230225966 shutdown_connections 2023-10-08T06:54:33.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.917+0000 7fcc38a72640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fcc1406afc0 0x7fcc1406d480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.917+0000 7fcc38a72640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc340c6660 0x7fcc340ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.917+0000 7fcc38a72640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3415cca0 0x7fcc340c6120 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.917+0000 7fcc38a72640 1 --2- 172.21.15.70:0/230225966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc340cab40 0x7fcc34149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:33.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.917+0000 7fcc38a72640 1 -- 172.21.15.70:0/230225966 >> 172.21.15.70:0/230225966 conn(0x7fcc340cb0c0 msgr2=0x7fcc34070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:33.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.917+0000 7fcc38a72640 1 -- 172.21.15.70:0/230225966 shutdown_connections 2023-10-08T06:54:33.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:33.917+0000 7fcc38a72640 1 -- 172.21.15.70:0/230225966 wait complete. 2023-10-08T06:54:33.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T06:54:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.162+0000 7f26d42e1640 1 Processor -- start 2023-10-08T06:54:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.162+0000 7f26d42e1640 1 -- start start 2023-10-08T06:54:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.163+0000 7f26d42e1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26cc0cb130 0x7f26cc0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.163+0000 7f26d42e1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26cc0b1cc0 con 0x7f26cc06dfc0 2023-10-08T06:54:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.163+0000 7f26d42e1640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26cc0cbad0 con 0x7f26cc0afbe0 2023-10-08T06:54:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.163+0000 7f26d42e1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26cc0b2620 con 0x7f26cc0cb130 2023-10-08T06:54:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.164+0000 7f26d287e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26cc0cb130 0x7f26cc0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.164+0000 7f26d287e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f26cc0cb130 0x7f26cc0cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60614/0 (socket says 172.21.15.70:60614) 2023-10-08T06:54:34.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.164+0000 7f26d187c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f26cc06dfc0 0x7f26cc070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37740/0 (socket says 172.21.15.70:37740) 2023-10-08T06:54:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.164+0000 7f26d287e640 1 -- 172.21.15.70:0/1447997329 learned_addr learned my addr 172.21.15.70:0/1447997329 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.164+0000 7f26d287e640 1 -- 172.21.15.70:0/1447997329 >> v1:172.21.15.181:6789/0 conn(0x7f26cc0afbe0 legacy=0x7f26cc0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.164+0000 7f26d287e640 1 -- 172.21.15.70:0/1447997329 >> v1:172.21.15.70:6789/0 conn(0x7f26cc06dfc0 legacy=0x7f26cc070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.164+0000 7f26d287e640 1 -- 172.21.15.70:0/1447997329 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc0b2060 con 0x7f26cc0cb130 2023-10-08T06:54:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.164+0000 7f26d107b640 1 -- 172.21.15.70:0/1447997329 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951624298 0 0) 0x7f26cc0b1cc0 con 0x7f26cc06dfc0 2023-10-08T06:54:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.165+0000 7f26d287e640 1 --2- 172.21.15.70:0/1447997329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 0x7f26cc0cb590 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009e50 tx=0x7f26c402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d8b29e9e0c6f0e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.166+0000 7f26d107b640 1 -- 172.21.15.70:0/1447997329 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c402cb20 con 0x7f26cc0cb130 2023-10-08T06:54:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.166+0000 7f26d107b640 1 -- 172.21.15.70:0/1447997329 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c402cc80 con 0x7f26cc0cb130 2023-10-08T06:54:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.166+0000 7f26d107b640 1 -- 172.21.15.70:0/1447997329 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c40343f0 con 0x7f26cc0cb130 2023-10-08T06:54:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.166+0000 7f26d42e1640 1 -- 172.21.15.70:0/1447997329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 msgr2=0x7f26cc0cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.166+0000 7f26d42e1640 1 --2- 172.21.15.70:0/1447997329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 0x7f26cc0cb590 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009e50 tx=0x7f26c402bf80 comp rx=0 tx=0).stop 2023-10-08T06:54:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.166+0000 7f26d42e1640 1 -- 172.21.15.70:0/1447997329 shutdown_connections 2023-10-08T06:54:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.166+0000 7f26d42e1640 1 --2- 172.21.15.70:0/1447997329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 0x7f26cc0cb590 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.166+0000 7f26d42e1640 1 -- 172.21.15.70:0/1447997329 >> 172.21.15.70:0/1447997329 conn(0x7f26cc06a5d0 msgr2=0x7f26cc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.166+0000 7f26d42e1640 1 -- 172.21.15.70:0/1447997329 shutdown_connections 2023-10-08T06:54:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.166+0000 7f26d42e1640 1 -- 172.21.15.70:0/1447997329 wait complete. 2023-10-08T06:54:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.168+0000 7f26d42e1640 1 Processor -- start 2023-10-08T06:54:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.168+0000 7f26d42e1640 1 -- start start 2023-10-08T06:54:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.169+0000 7f26d42e1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26cc06dfc0 0x7f26cc0c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.169+0000 7f26d42e1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26cc0afbe0 0x7f26cc0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.169+0000 7f26d42e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 0x7f26cc0c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.169+0000 7f26d42e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0c2250 con 0x7f26cc0cb130 2023-10-08T06:54:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.169+0000 7f26d42e1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0cf4b0 con 0x7f26cc0afbe0 2023-10-08T06:54:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.169+0000 7f26d42e1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc0c24a0 con 0x7f26cc06dfc0 2023-10-08T06:54:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d287e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 0x7f26cc0c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:34.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d287e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 0x7f26cc0c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60628/0 (socket says 172.21.15.70:60628) 2023-10-08T06:54:34.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d287e640 1 -- 172.21.15.70:0/3304045161 learned_addr learned my addr 172.21.15.70:0/3304045161 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:34.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d187c640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26cc0afbe0 0x7f26cc0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:34.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d287e640 1 -- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26cc06dfc0 msgr2=0x7f26cc0c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d207d640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26cc06dfc0 0x7f26cc0c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d287e640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26cc06dfc0 0x7f26cc0c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d287e640 1 -- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26cc0afbe0 msgr2=0x7f26cc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d287e640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26cc0afbe0 0x7f26cc0cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d287e640 1 -- 172.21.15.70:0/3304045161 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c4009b30 con 0x7f26cc0cb130 2023-10-08T06:54:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d187c640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26cc0afbe0 0x7f26cc0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.170+0000 7f26d287e640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 0x7f26cc0c9f20 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f26c4036040 tx=0x7f26c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.171+0000 7f26beffd640 1 -- 172.21.15.70:0/3304045161 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4009510 con 0x7f26cc0cb130 2023-10-08T06:54:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.171+0000 7f26beffd640 1 -- 172.21.15.70:0/3304045161 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c4007e50 con 0x7f26cc0cb130 2023-10-08T06:54:34.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.171+0000 7f26beffd640 1 -- 172.21.15.70:0/3304045161 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c403e690 con 0x7f26cc0cb130 2023-10-08T06:54:34.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.171+0000 7f26d42e1640 1 -- 172.21.15.70:0/3304045161 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc0c2780 con 0x7f26cc0cb130 2023-10-08T06:54:34.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.171+0000 7f26d42e1640 1 -- 172.21.15.70:0/3304045161 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc0c2d40 con 0x7f26cc0cb130 2023-10-08T06:54:34.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.172+0000 7f26d42e1640 1 -- 172.21.15.70:0/3304045161 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc0c2f60 con 0x7f26cc0cb130 2023-10-08T06:54:34.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.173+0000 7f26beffd640 1 -- 172.21.15.70:0/3304045161 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f26c403e7f0 con 0x7f26cc0cb130 2023-10-08T06:54:34.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.174+0000 7f26beffd640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f26b006af90 0x7f26b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:34.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.174+0000 7f26beffd640 1 -- 172.21.15.70:0/3304045161 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f26c40ac660 con 0x7f26cc0cb130 2023-10-08T06:54:34.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.174+0000 7f26d207d640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f26b006af90 0x7f26b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:34.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.176+0000 7f26d207d640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f26b006af90 0x7f26b006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f26c8003d90 tx=0x7f26c8009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.179+0000 7f26beffd640 1 -- 172.21.15.70:0/3304045161 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26c40773a0 con 0x7f26cc0cb130 2023-10-08T06:54:34.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:34.404+0000 7f26d42e1640 1 -- 172.21.15.70:0/3304045161 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f26cc070320 con 0x7f26cc0cb130 2023-10-08T06:54:35.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.052+0000 7f26beffd640 1 -- 172.21.15.70:0/3304045161 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) v1 ==== 118+0+0 (secure 0 0 0) 0x7f26c407a9c0 con 0x7f26cc0cb130 2023-10-08T06:54:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 -- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f26b006af90 msgr2=0x7f26b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f26b006af90 0x7f26b006d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f26c8003d90 tx=0x7f26c8009040 comp rx=0 tx=0).stop 2023-10-08T06:54:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 -- 172.21.15.70:0/3304045161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 msgr2=0x7f26cc0c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 0x7f26cc0c9f20 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f26c4036040 tx=0x7f26c4004a40 comp rx=0 tx=0).stop 2023-10-08T06:54:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 -- 172.21.15.70:0/3304045161 shutdown_connections 2023-10-08T06:54:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f26b006af90 0x7f26b006d450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc0cb130 0x7f26cc0c9f20 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26cc0afbe0 0x7f26cc0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 --2- 172.21.15.70:0/3304045161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26cc06dfc0 0x7f26cc0c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 -- 172.21.15.70:0/3304045161 >> 172.21.15.70:0/3304045161 conn(0x7f26cc06a5d0 msgr2=0x7f26cc06c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.055+0000 7f26d42e1640 1 -- 172.21.15.70:0/3304045161 shutdown_connections 2023-10-08T06:54:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.056+0000 7f26d42e1640 1 -- 172.21.15.70:0/3304045161 wait complete. 2023-10-08T06:54:35.076 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T06:54:35.076 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T06:54:35.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.289+0000 7fa83170c640 1 Processor -- start 2023-10-08T06:54:35.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.289+0000 7fa83170c640 1 -- start start 2023-10-08T06:54:35.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.291+0000 7fa83170c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa82c0ce290 0x7fa82c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:35.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.291+0000 7fa83170c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa82c065ab0 0x7fa82c1608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:35.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.291+0000 7fa83170c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c069d50 con 0x7fa82c15dde0 2023-10-08T06:54:35.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.291+0000 7fa83170c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c06e6c0 con 0x7fa82c065ab0 2023-10-08T06:54:35.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.291+0000 7fa83170c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c06c750 con 0x7fa82c0ce290 2023-10-08T06:54:35.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.291+0000 7fa8237fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa82c065ab0 0x7fa82c1608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:35.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.291+0000 7fa8237fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa82c065ab0 0x7fa82c1608d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60642/0 (socket says 172.21.15.70:60642) 2023-10-08T06:54:35.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.291+0000 7fa8237fe640 1 -- 172.21.15.70:0/91846842 learned_addr learned my addr 172.21.15.70:0/91846842 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:35.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.292+0000 7fa82bfff640 1 --2- 172.21.15.70:0/91846842 >> v2:172.21.15.181:3300/0 conn(0x7fa82c0ce290 0x7fa82c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:35.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.292+0000 7fa8237fe640 1 -- 172.21.15.70:0/91846842 >> v2:172.21.15.181:3300/0 conn(0x7fa82c0ce290 msgr2=0x7fa82c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:35.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.292+0000 7fa8237fe640 1 --2- 172.21.15.70:0/91846842 >> v2:172.21.15.181:3300/0 conn(0x7fa82c0ce290 0x7fa82c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:35.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.292+0000 7fa8237fe640 1 -- 172.21.15.70:0/91846842 >> v1:172.21.15.181:6789/0 conn(0x7fa82c15dde0 legacy=0x7fa82c160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:35.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.292+0000 7fa8237fe640 1 -- 172.21.15.70:0/91846842 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0cebb0 con 0x7fa82c065ab0 2023-10-08T06:54:35.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.293+0000 7fa8237fe640 1 --2- 172.21.15.70:0/91846842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c065ab0 0x7fa82c1608d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fa82c067520 tx=0x7fa81402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaf2754b7c3f5b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:35.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.293+0000 7fa82b7fe640 1 -- 172.21.15.70:0/91846842 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814031890 con 0x7fa82c065ab0 2023-10-08T06:54:35.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.293+0000 7fa82b7fe640 1 -- 172.21.15.70:0/91846842 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8140319f0 con 0x7fa82c065ab0 2023-10-08T06:54:35.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.293+0000 7fa82b7fe640 1 -- 172.21.15.70:0/91846842 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814031d80 con 0x7fa82c065ab0 2023-10-08T06:54:35.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.294+0000 7fa83170c640 1 -- 172.21.15.70:0/91846842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c065ab0 msgr2=0x7fa82c1608d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:35.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.294+0000 7fa83170c640 1 --2- 172.21.15.70:0/91846842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c065ab0 0x7fa82c1608d0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fa82c067520 tx=0x7fa81402b930 comp rx=0 tx=0).stop 2023-10-08T06:54:35.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.294+0000 7fa83170c640 1 -- 172.21.15.70:0/91846842 shutdown_connections 2023-10-08T06:54:35.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.294+0000 7fa83170c640 1 --2- 172.21.15.70:0/91846842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c065ab0 0x7fa82c1608d0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:35.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.294+0000 7fa83170c640 1 --2- 172.21.15.70:0/91846842 >> v2:172.21.15.181:3300/0 conn(0x7fa82c0ce290 0x7fa82c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:35.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.294+0000 7fa83170c640 1 -- 172.21.15.70:0/91846842 >> 172.21.15.70:0/91846842 conn(0x7fa82c0b08d0 msgr2=0x7fa82c0b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:35.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.294+0000 7fa83170c640 1 -- 172.21.15.70:0/91846842 shutdown_connections 2023-10-08T06:54:35.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.294+0000 7fa83170c640 1 -- 172.21.15.70:0/91846842 wait complete. 2023-10-08T06:54:35.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.296+0000 7fa83170c640 1 Processor -- start 2023-10-08T06:54:35.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.296+0000 7fa83170c640 1 -- start start 2023-10-08T06:54:35.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.297+0000 7fa83170c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c065ab0 0x7fa82c0b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.297+0000 7fa83170c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c0ce290 0x7fa82c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.297+0000 7fa83170c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa82c15dde0 0x7fa82c1f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.297+0000 7fa83170c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c0c5fc0 con 0x7fa82c0ce290 2023-10-08T06:54:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.297+0000 7fa83170c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1f3200 con 0x7fa82c065ab0 2023-10-08T06:54:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.297+0000 7fa83170c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c09ec60 con 0x7fa82c15dde0 2023-10-08T06:54:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.297+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c0ce290 0x7fa82c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.297+0000 7fa8237fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c0ce290 0x7fa82c0c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60654/0 (socket says 172.21.15.70:60654) 2023-10-08T06:54:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.297+0000 7fa8237fe640 1 -- 172.21.15.70:0/280881562 learned_addr learned my addr 172.21.15.70:0/280881562 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa82bfff640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c065ab0 0x7fa82c0b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa8237fe640 1 -- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa82c15dde0 msgr2=0x7fa82c1f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:35.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa830f0b640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa82c15dde0 0x7fa82c1f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:35.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa8237fe640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa82c15dde0 0x7fa82c1f22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:35.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa8237fe640 1 -- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c065ab0 msgr2=0x7fa82c0b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:35.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa8237fe640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c065ab0 0x7fa82c0b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:35.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa8237fe640 1 -- 172.21.15.70:0/280881562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140094d0 con 0x7fa82c0ce290 2023-10-08T06:54:35.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa82bfff640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c065ab0 0x7fa82c0b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:35.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa8237fe640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c0ce290 0x7fa82c0c8840 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fa814000c00 tx=0x7fa8140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:35.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa8297fa640 1 -- 172.21.15.70:0/280881562 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814031840 con 0x7fa82c0ce290 2023-10-08T06:54:35.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa8297fa640 1 -- 172.21.15.70:0/280881562 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8140443f0 con 0x7fa82c0ce290 2023-10-08T06:54:35.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa8297fa640 1 -- 172.21.15.70:0/280881562 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8140447a0 con 0x7fa82c0ce290 2023-10-08T06:54:35.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.298+0000 7fa83170c640 1 -- 172.21.15.70:0/280881562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c09dbc0 con 0x7fa82c0ce290 2023-10-08T06:54:35.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.299+0000 7fa83170c640 1 -- 172.21.15.70:0/280881562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c09e180 con 0x7fa82c0ce290 2023-10-08T06:54:35.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.300+0000 7fa83170c640 1 -- 172.21.15.70:0/280881562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c074150 con 0x7fa82c0ce290 2023-10-08T06:54:35.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.301+0000 7fa8297fa640 1 -- 172.21.15.70:0/280881562 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa81404d450 con 0x7fa82c0ce290 2023-10-08T06:54:35.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.301+0000 7fa8297fa640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa80c06af90 0x7fa80c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:35.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.302+0000 7fa8297fa640 1 -- 172.21.15.70:0/280881562 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa8140ac3e0 con 0x7fa82c0ce290 2023-10-08T06:54:35.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.302+0000 7fa82bfff640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa80c06af90 0x7fa80c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:35.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.303+0000 7fa82bfff640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa80c06af90 0x7fa80c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa81c007eb0 tx=0x7fa81c007400 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:35.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.306+0000 7fa8297fa640 1 -- 172.21.15.70:0/280881562 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa814076f80 con 0x7fa82c0ce290 2023-10-08T06:54:35.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:35.532+0000 7fa83170c640 1 -- 172.21.15.70:0/280881562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa82c06e3b0 con 0x7fa82c0ce290 2023-10-08T06:54:36.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.056+0000 7fa8297fa640 1 -- 172.21.15.70:0/280881562 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa81407a5a0 con 0x7fa82c0ce290 2023-10-08T06:54:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 -- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa80c06af90 msgr2=0x7fa80c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa80c06af90 0x7fa80c06d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa81c007eb0 tx=0x7fa81c007400 comp rx=0 tx=0).stop 2023-10-08T06:54:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 -- 172.21.15.70:0/280881562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c0ce290 msgr2=0x7fa82c0c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c0ce290 0x7fa82c0c8840 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fa814000c00 tx=0x7fa8140046c0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 -- 172.21.15.70:0/280881562 shutdown_connections 2023-10-08T06:54:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fa80c06af90 0x7fa80c06d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa82c15dde0 0x7fa82c1f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa82c0ce290 0x7fa82c0c8840 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 --2- 172.21.15.70:0/280881562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa82c065ab0 0x7fa82c0b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 -- 172.21.15.70:0/280881562 >> 172.21.15.70:0/280881562 conn(0x7fa82c0b08d0 msgr2=0x7fa82c15f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:36.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.060+0000 7fa83170c640 1 -- 172.21.15.70:0/280881562 shutdown_connections 2023-10-08T06:54:36.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.061+0000 7fa83170c640 1 -- 172.21.15.70:0/280881562 wait complete. 2023-10-08T06:54:36.084 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T06:54:36.084 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:54:36.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.299+0000 7f3bfa8c2640 1 Processor -- start 2023-10-08T06:54:36.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.299+0000 7f3bfa8c2640 1 -- start start 2023-10-08T06:54:36.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.301+0000 7f3bfa8c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf4159ac0 0x7f3bf406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:36.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.301+0000 7f3bfa8c2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3bf40ad790 0x7f3bf40702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.301+0000 7f3bfa8c2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf406dee0 con 0x7f3bf415e330 2023-10-08T06:54:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.301+0000 7f3bfa8c2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf415d850 con 0x7f3bf40ad790 2023-10-08T06:54:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.301+0000 7f3bfa8c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf415db30 con 0x7f3bf4159ac0 2023-10-08T06:54:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.301+0000 7f3bf8e5f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3bf415e330 0x7f3bf406c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37742/0 (socket says 172.21.15.70:37742) 2023-10-08T06:54:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.301+0000 7f3bf8e5f640 1 -- 172.21.15.70:0/3591906621 learned_addr learned my addr 172.21.15.70:0/3591906621 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf37fe640 1 --2- 172.21.15.70:0/3591906621 >> v2:172.21.15.181:3300/0 conn(0x7f3bf40ad790 0x7f3bf40702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf3fff640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf4159ac0 0x7f3bf406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150832936 0 0) 0x7f3bf406dee0 con 0x7f3bf415e330 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bdc003520 con 0x7f3bf415e330 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2189679480 0 0) 0x7f3bdc003520 con 0x7f3bf415e330 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf4159ac0 msgr2=0x7f3bf406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf4159ac0 0x7f3bf406d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 >> v2:172.21.15.181:3300/0 conn(0x7f3bf40ad790 msgr2=0x7f3bf40702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 --2- 172.21.15.70:0/3591906621 >> v2:172.21.15.181:3300/0 conn(0x7f3bf40ad790 0x7f3bf40702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf415de10 con 0x7f3bf415e330 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3bec0019f0 con 0x7f3bf415e330 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf37fe640 1 --2- 172.21.15.70:0/3591906621 >> v2:172.21.15.181:3300/0 conn(0x7f3bf40ad790 0x7f3bf40702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T06:54:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 >> v1:172.21.15.70:6789/0 conn(0x7f3bf415e330 legacy=0x7f3bf406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bdc003970 0x7f3bdc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf4159ac0 0x7f3bdc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 0x7f3bdc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf415d850 con 0x7f3bdc004d00 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf415db30 con 0x7f3bdc003970 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.302+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc00b980 con 0x7f3bf4159ac0 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.303+0000 7f3bf8e5f640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 0x7f3bdc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.303+0000 7f3bf3fff640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bdc003970 0x7f3bdc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.303+0000 7f3bf37fe640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf4159ac0 0x7f3bdc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.303+0000 7f3bf8e5f640 1 -- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf4159ac0 msgr2=0x7f3bdc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.303+0000 7f3bf8e5f640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf4159ac0 0x7f3bdc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.303+0000 7f3bf8e5f640 1 -- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bdc003970 msgr2=0x7f3bdc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:36.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.303+0000 7f3bf8e5f640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bdc003970 0x7f3bdc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.303+0000 7f3bf8e5f640 1 -- 172.21.15.70:0/3591906621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc003520 con 0x7f3bdc004d00 2023-10-08T06:54:36.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.303+0000 7f3bf3fff640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bdc003970 0x7f3bdc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:36.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.304+0000 7f3bf8e5f640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 0x7f3bdc005180 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f3bec00a890 tx=0x7f3bdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f0e51ac8e1d092 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:36.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.304+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec034630 con 0x7f3bdc004d00 2023-10-08T06:54:36.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.304+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec02f070 con 0x7f3bdc004d00 2023-10-08T06:54:36.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.304+0000 7f3bf2ffd640 1 -- 172.21.15.70:0/3591906621 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0349f0 con 0x7f3bdc004d00 2023-10-08T06:54:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.305+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/3591906621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 msgr2=0x7f3bdc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.305+0000 7f3bfa8c2640 1 --2- 172.21.15.70:0/3591906621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 0x7f3bdc005180 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f3bec00a890 tx=0x7f3bdc0057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.305+0000 7f3bf8e5f640 1 -- 172.21.15.70:0/3591906621 reap_dead start 2023-10-08T06:54:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.305+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/3591906621 shutdown_connections 2023-10-08T06:54:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.305+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/3591906621 >> 172.21.15.70:0/3591906621 conn(0x7f3bf406a380 msgr2=0x7f3bf40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.305+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/3591906621 shutdown_connections 2023-10-08T06:54:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.305+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/3591906621 wait complete. 2023-10-08T06:54:36.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.307+0000 7f3bfa8c2640 1 Processor -- start 2023-10-08T06:54:36.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.307+0000 7f3bfa8c2640 1 -- start start 2023-10-08T06:54:36.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.308+0000 7f3bfa8c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 0x7f3bf4157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:36.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.308+0000 7f3bfa8c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf40cda10 0x7f3bf41482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:36.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.308+0000 7f3bfa8c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf414c310 0x7f3bf414c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:36.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.308+0000 7f3bfa8c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf414ccd0 con 0x7f3bdc004d00 2023-10-08T06:54:36.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.308+0000 7f3bfa8c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4157c80 con 0x7f3bf40cda10 2023-10-08T06:54:36.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.308+0000 7f3bfa8c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf414d430 con 0x7f3bf414c310 2023-10-08T06:54:36.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.308+0000 7f3bf3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 0x7f3bf4157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:36.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.308+0000 7f3bf3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 0x7f3bf4157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60658/0 (socket says 172.21.15.70:60658) 2023-10-08T06:54:36.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.308+0000 7f3bf3fff640 1 -- 172.21.15.70:0/256501209 learned_addr learned my addr 172.21.15.70:0/256501209 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:36.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bf3fff640 1 -- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf414c310 msgr2=0x7f3bf414c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:36.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bf3fff640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf414c310 0x7f3bf414c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bf3fff640 1 -- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf40cda10 msgr2=0x7f3bf41482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:36.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bf3fff640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf40cda10 0x7f3bf41482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:36.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bf3fff640 1 -- 172.21.15.70:0/256501209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf40cd580 con 0x7f3bdc004d00 2023-10-08T06:54:36.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bf3fff640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 0x7f3bf4157740 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f3be4010890 tx=0x7f3be4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:36.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bf0ff9640 1 -- 172.21.15.70:0/256501209 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400ea80 con 0x7f3bdc004d00 2023-10-08T06:54:36.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bf0ff9640 1 -- 172.21.15.70:0/256501209 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be400ebe0 con 0x7f3bdc004d00 2023-10-08T06:54:36.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bf0ff9640 1 -- 172.21.15.70:0/256501209 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be401c400 con 0x7f3bdc004d00 2023-10-08T06:54:36.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/256501209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf414cee0 con 0x7f3bdc004d00 2023-10-08T06:54:36.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.309+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/256501209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf40cd210 con 0x7f3bdc004d00 2023-10-08T06:54:36.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.311+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/256501209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf40cd040 con 0x7f3bdc004d00 2023-10-08T06:54:36.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.311+0000 7f3bf0ff9640 1 -- 172.21.15.70:0/256501209 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3be401c560 con 0x7f3bdc004d00 2023-10-08T06:54:36.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.312+0000 7f3bf0ff9640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f3be006afc0 0x7f3be006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:36.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.312+0000 7f3bf0ff9640 1 -- 172.21.15.70:0/256501209 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f3be4089d60 con 0x7f3bdc004d00 2023-10-08T06:54:37.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.313+0000 7f3bf37fe640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f3be006afc0 0x7f3be006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.314+0000 7f3bf37fe640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f3be006afc0 0x7f3be006d480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40ce330 tx=0x7f3be800c350 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:37.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.317+0000 7f3bf0ff9640 1 -- 172.21.15.70:0/256501209 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3be4054bf0 con 0x7f3bdc004d00 2023-10-08T06:54:37.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.542+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/256501209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bf4067490 con 0x7f3bdc004d00 2023-10-08T06:54:37.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.543+0000 7f3bf0ff9640 1 -- 172.21.15.70:0/256501209 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f3be4058210 con 0x7f3bdc004d00 2023-10-08T06:54:37.055 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:37.055 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/381523318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":381523318},{"type":"v1","addr":"172.21.15.181:6835","nonce":381523318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2153071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2153071509},{"type":"v1","addr":"172.21.15.181:6837","nonce":2153071509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2673497542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2673497542},{"type":"v1","addr":"172.21.15.70:6835","nonce":2673497542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:54:32.491331+0000","modified":"2023-10-08T06:54:35.534957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10074":{"gid":10074,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2138719113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2138719113},{"type":"v1","addr":"172.21.15.70:6833","nonce":2138719113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-08T06:54:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-10-08T06:54:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.545+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f3be006afc0 msgr2=0x7f3be006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.545+0000 7f3bfa8c2640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f3be006afc0 0x7f3be006d480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40ce330 tx=0x7f3be800c350 comp rx=0 tx=0).stop 2023-10-08T06:54:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.545+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/256501209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 msgr2=0x7f3bf4157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.545+0000 7f3bfa8c2640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 0x7f3bf4157740 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f3be4010890 tx=0x7f3be4010d50 comp rx=0 tx=0).stop 2023-10-08T06:54:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.546+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/256501209 shutdown_connections 2023-10-08T06:54:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.546+0000 7f3bfa8c2640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f3be006afc0 0x7f3be006d480 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.546+0000 7f3bfa8c2640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3bf414c310 0x7f3bf414c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.546+0000 7f3bfa8c2640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3bf40cda10 0x7f3bf41482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.546+0000 7f3bfa8c2640 1 --2- 172.21.15.70:0/256501209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bdc004d00 0x7f3bf4157740 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.546+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/256501209 >> 172.21.15.70:0/256501209 conn(0x7f3bf406a380 msgr2=0x7f3bf4156060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.546+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/256501209 shutdown_connections 2023-10-08T06:54:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:36.546+0000 7f3bfa8c2640 1 -- 172.21.15.70:0/256501209 wait complete. 2023-10-08T06:54:37.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T06:54:37.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.276+0000 7f9078b93640 1 Processor -- start 2023-10-08T06:54:37.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.277+0000 7f9078b93640 1 -- start start 2023-10-08T06:54:37.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.278+0000 7f9078b93640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f907406bf60 0x7f9074167c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.278+0000 7f9078b93640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90740cb590 con 0x7f90740cd8a0 2023-10-08T06:54:37.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.278+0000 7f9078b93640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90740cfc90 con 0x7f90740cf720 2023-10-08T06:54:37.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.278+0000 7f9078b93640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90740cff70 con 0x7f907406bf60 2023-10-08T06:54:37.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.278+0000 7f90737fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f90740cd8a0 0x7f907406b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37756/0 (socket says 172.21.15.70:37756) 2023-10-08T06:54:37.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.278+0000 7f90737fe640 1 -- 172.21.15.70:0/1716200130 learned_addr learned my addr 172.21.15.70:0/1716200130 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:37.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789232054 0 0) 0x7f90740cb590 con 0x7f90740cd8a0 2023-10-08T06:54:37.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9060003520 con 0x7f90740cd8a0 2023-10-08T06:54:37.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9073fff640 1 --2- 172.21.15.70:0/1716200130 >> v2:172.21.15.181:3300/0 conn(0x7f907406bf60 0x7f9074167c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 428124847 0 0) 0x7f9060003520 con 0x7f90740cd8a0 2023-10-08T06:54:37.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 >> v2:172.21.15.181:3300/0 conn(0x7f907406bf60 msgr2=0x7f9074167c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:37.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 --2- 172.21.15.70:0/1716200130 >> v2:172.21.15.181:3300/0 conn(0x7f907406bf60 0x7f9074167c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 >> v1:172.21.15.181:6789/0 conn(0x7f90740cf720 legacy=0x7f907406b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:37.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90740caf40 con 0x7f90740cd8a0 2023-10-08T06:54:37.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9064001670 con 0x7f90740cd8a0 2023-10-08T06:54:37.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 >> v1:172.21.15.70:6789/0 conn(0x7f90740cd8a0 legacy=0x7f907406b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 --2- 172.21.15.70:0/1716200130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060003970 0x7f9060003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 --2- 172.21.15.70:0/1716200130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90600047c0 0x7f9060004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 --2- 172.21.15.70:0/1716200130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9060005180 0x7f906000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90740cff70 con 0x7f9060003970 2023-10-08T06:54:37.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906000c0b0 con 0x7f9060005180 2023-10-08T06:54:37.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906000c270 con 0x7f90600047c0 2023-10-08T06:54:37.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.279+0000 7f90737fe640 1 --2- 172.21.15.70:0/1716200130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060003970 0x7f9060003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.280+0000 7f9073fff640 1 --2- 172.21.15.70:0/1716200130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9060005180 0x7f906000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.280+0000 7f90737fe640 1 -- 172.21.15.70:0/1716200130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90600047c0 msgr2=0x7f9060004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.280+0000 7f90737fe640 1 --2- 172.21.15.70:0/1716200130 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f90600047c0 0x7f9060004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.280+0000 7f90737fe640 1 -- 172.21.15.70:0/1716200130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9060005180 msgr2=0x7f906000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.280+0000 7f90737fe640 1 --2- 172.21.15.70:0/1716200130 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9060005180 0x7f906000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.280+0000 7f90737fe640 1 -- 172.21.15.70:0/1716200130 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060003520 con 0x7f9060003970 2023-10-08T06:54:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.280+0000 7f9073fff640 1 -- 172.21.15.70:0/1716200130 reap_dead start 2023-10-08T06:54:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.281+0000 7f90737fe640 1 --2- 172.21.15.70:0/1716200130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060003970 0x7f9060003d70 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f9064001eb0 tx=0x7f90600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f02628a8f30a6a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.281+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064005ad0 con 0x7f9060003970 2023-10-08T06:54:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.281+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9064005c50 con 0x7f9060003970 2023-10-08T06:54:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.281+0000 7f9072ffd640 1 -- 172.21.15.70:0/1716200130 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906402c8b0 con 0x7f9060003970 2023-10-08T06:54:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.282+0000 7f9078b93640 1 -- 172.21.15.70:0/1716200130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060003970 msgr2=0x7f9060003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.282+0000 7f9078b93640 1 --2- 172.21.15.70:0/1716200130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060003970 0x7f9060003d70 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f9064001eb0 tx=0x7f90600057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.282+0000 7f9078b93640 1 -- 172.21.15.70:0/1716200130 shutdown_connections 2023-10-08T06:54:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.282+0000 7f9078b93640 1 --2- 172.21.15.70:0/1716200130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9060003970 0x7f9060003d70 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.282+0000 7f9078b93640 1 -- 172.21.15.70:0/1716200130 >> 172.21.15.70:0/1716200130 conn(0x7f9074099b60 msgr2=0x7f90740d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.282+0000 7f9078b93640 1 -- 172.21.15.70:0/1716200130 shutdown_connections 2023-10-08T06:54:37.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.282+0000 7f9078b93640 1 -- 172.21.15.70:0/1716200130 wait complete. 2023-10-08T06:54:37.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.284+0000 7f9078b93640 1 Processor -- start 2023-10-08T06:54:37.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.284+0000 7f9078b93640 1 -- start start 2023-10-08T06:54:37.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.285+0000 7f9078b93640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9060003970 0x7f907409ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.285+0000 7f9078b93640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907406f7e0 0x7f907406fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.285+0000 7f9078b93640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f907414e780 0x7f907414ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.285+0000 7f9078b93640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90740701a0 con 0x7f907406f7e0 2023-10-08T06:54:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.285+0000 7f9078b93640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9074150590 con 0x7f907414e780 2023-10-08T06:54:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.285+0000 7f9078b93640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907414f4f0 con 0x7f9060003970 2023-10-08T06:54:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.285+0000 7f906bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907406f7e0 0x7f907406fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.285+0000 7f906bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907406f7e0 0x7f907406fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60676/0 (socket says 172.21.15.70:60676) 2023-10-08T06:54:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.285+0000 7f906bfff640 1 -- 172.21.15.70:0/3535136001 learned_addr learned my addr 172.21.15.70:0/3535136001 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f90737fe640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9060003970 0x7f907409ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f906bfff640 1 -- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9060003970 msgr2=0x7f907409ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f906bfff640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9060003970 0x7f907409ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f906bfff640 1 -- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f907414e780 msgr2=0x7f907414ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f906bfff640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f907414e780 0x7f907414ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f906bfff640 1 -- 172.21.15.70:0/3535136001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907414f7d0 con 0x7f907406f7e0 2023-10-08T06:54:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f906bfff640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907406f7e0 0x7f907406fc60 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c3c0 tx=0x7f905c00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f9070ff9640 1 -- 172.21.15.70:0/3535136001 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c011790 con 0x7f907406f7e0 2023-10-08T06:54:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f9070ff9640 1 -- 172.21.15.70:0/3535136001 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c0118f0 con 0x7f907406f7e0 2023-10-08T06:54:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f9070ff9640 1 -- 172.21.15.70:0/3535136001 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c011ce0 con 0x7f907406f7e0 2023-10-08T06:54:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f9078b93640 1 -- 172.21.15.70:0/3535136001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907414fab0 con 0x7f907406f7e0 2023-10-08T06:54:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.286+0000 7f9078b93640 1 -- 172.21.15.70:0/3535136001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9074150070 con 0x7f907406f7e0 2023-10-08T06:54:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.287+0000 7f9078b93640 1 -- 172.21.15.70:0/3535136001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9074150290 con 0x7f907406f7e0 2023-10-08T06:54:37.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.288+0000 7f9070ff9640 1 -- 172.21.15.70:0/3535136001 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f905c01c460 con 0x7f907406f7e0 2023-10-08T06:54:37.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.289+0000 7f9070ff9640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f905406afc0 0x7f905406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.289+0000 7f9070ff9640 1 -- 172.21.15.70:0/3535136001 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f905c0897b0 con 0x7f907406f7e0 2023-10-08T06:54:37.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.290+0000 7f90737fe640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f905406afc0 0x7f905406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.292+0000 7f90737fe640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f905406afc0 0x7f905406d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9064034000 tx=0x7f9064009030 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:37.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.294+0000 7f9070ff9640 1 -- 172.21.15.70:0/3535136001 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f905c0545b0 con 0x7f907406f7e0 2023-10-08T06:54:37.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.482+0000 7f9078b93640 1 -- 172.21.15.70:0/3535136001 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f907404e160 con 0x7f907406f7e0 2023-10-08T06:54:37.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.486+0000 7f9070ff9640 1 -- 172.21.15.70:0/3535136001 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+16890 (secure 0 0 0) 0x7f905c057bd0 con 0x7f907406f7e0 2023-10-08T06:54:37.488 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:37.488 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":198,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T06:54:34.017656+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:54:30.246375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-08T06:54:31.487900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T06:54:37.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.488+0000 7f9078b93640 1 -- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f905406afc0 msgr2=0x7f905406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.488+0000 7f9078b93640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f905406afc0 0x7f905406d480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9064034000 tx=0x7f9064009030 comp rx=0 tx=0).stop 2023-10-08T06:54:37.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.488+0000 7f9078b93640 1 -- 172.21.15.70:0/3535136001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907406f7e0 msgr2=0x7f907406fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.488+0000 7f9078b93640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907406f7e0 0x7f907406fc60 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c3c0 tx=0x7f905c00c7c0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.489+0000 7f9078b93640 1 -- 172.21.15.70:0/3535136001 shutdown_connections 2023-10-08T06:54:37.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.489+0000 7f9078b93640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f905406afc0 0x7f905406d480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.489+0000 7f9078b93640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f907414e780 0x7f907414ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.489+0000 7f9078b93640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f907406f7e0 0x7f907406fc60 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.489+0000 7f9078b93640 1 --2- 172.21.15.70:0/3535136001 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9060003970 0x7f907409ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.489+0000 7f9078b93640 1 -- 172.21.15.70:0/3535136001 >> 172.21.15.70:0/3535136001 conn(0x7f9074099b60 msgr2=0x7f90740b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:37.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.489+0000 7f9078b93640 1 -- 172.21.15.70:0/3535136001 shutdown_connections 2023-10-08T06:54:37.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.489+0000 7f9078b93640 1 -- 172.21.15.70:0/3535136001 wait complete. 2023-10-08T06:54:37.510 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T06:54:37.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T06:54:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.721+0000 7fbfa8ee7640 1 Processor -- start 2023-10-08T06:54:37.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.721+0000 7fbfa8ee7640 1 -- start start 2023-10-08T06:54:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.723+0000 7fbfa8ee7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 0x7fbfa4162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.723+0000 7fbfa8ee7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfa40cb150 0x7fbfa40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.723+0000 7fbfa8ee7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa40ae140 con 0x7fbfa40add60 2023-10-08T06:54:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.723+0000 7fbfa8ee7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa4161f10 con 0x7fbfa40cb150 2023-10-08T06:54:37.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.723+0000 7fbfa8ee7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40cbb20 con 0x7fbfa406d4c0 2023-10-08T06:54:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.724+0000 7fbfa2d9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfa40cb150 0x7fbfa40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.724+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 0x7fbfa4162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.724+0000 7fbfa2d9d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfa40cb150 0x7fbfa40cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:35832/0 (socket says 172.21.15.70:35832) 2023-10-08T06:54:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.724+0000 7fbfa2d9d640 1 -- 172.21.15.70:0/3756329183 learned_addr learned my addr 172.21.15.70:0/3756329183 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:37.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.725+0000 7fbf9bfff640 1 -- 172.21.15.70:0/3756329183 >> v2:172.21.15.181:3300/0 conn(0x7fbfa40cb150 msgr2=0x7fbfa40cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.725+0000 7fbf9bfff640 1 --2- 172.21.15.70:0/3756329183 >> v2:172.21.15.181:3300/0 conn(0x7fbfa40cb150 0x7fbfa40cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.725+0000 7fbf9bfff640 1 -- 172.21.15.70:0/3756329183 >> v1:172.21.15.181:6789/0 conn(0x7fbfa40add60 legacy=0x7fbfa40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.725+0000 7fbf9bfff640 1 -- 172.21.15.70:0/3756329183 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa40ccf20 con 0x7fbfa406d4c0 2023-10-08T06:54:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.726+0000 7fbf9bfff640 1 --2- 172.21.15.70:0/3756329183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 0x7fbfa4162330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0097a0 tx=0x7fbf8c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8629d5d0b083d4ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.727+0000 7fbfa1d9b640 1 -- 172.21.15.70:0/3756329183 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c031780 con 0x7fbfa406d4c0 2023-10-08T06:54:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.727+0000 7fbfa1d9b640 1 -- 172.21.15.70:0/3756329183 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c0318e0 con 0x7fbfa406d4c0 2023-10-08T06:54:37.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.727+0000 7fbfa1d9b640 1 -- 172.21.15.70:0/3756329183 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c031c70 con 0x7fbfa406d4c0 2023-10-08T06:54:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.727+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3756329183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 msgr2=0x7fbfa4162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.727+0000 7fbfa8ee7640 1 --2- 172.21.15.70:0/3756329183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 0x7fbfa4162330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0097a0 tx=0x7fbf8c02b820 comp rx=0 tx=0).stop 2023-10-08T06:54:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.728+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3756329183 shutdown_connections 2023-10-08T06:54:37.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.728+0000 7fbfa8ee7640 1 --2- 172.21.15.70:0/3756329183 >> v2:172.21.15.181:3300/0 conn(0x7fbfa40cb150 0x7fbfa40cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.728+0000 7fbfa8ee7640 1 --2- 172.21.15.70:0/3756329183 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 0x7fbfa4162330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.728+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3756329183 >> 172.21.15.70:0/3756329183 conn(0x7fbfa406a380 msgr2=0x7fbfa40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:37.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.728+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3756329183 shutdown_connections 2023-10-08T06:54:37.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.728+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3756329183 wait complete. 2023-10-08T06:54:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.730+0000 7fbfa8ee7640 1 Processor -- start 2023-10-08T06:54:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.730+0000 7fbfa8ee7640 1 -- start start 2023-10-08T06:54:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.731+0000 7fbfa8ee7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 0x7fbfa4156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.731+0000 7fbfa8ee7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40add60 0x7fbfa40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.731+0000 7fbfa8ee7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa40cb150 0x7fbfa41f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.731+0000 7fbfa8ee7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4156b30 con 0x7fbfa40add60 2023-10-08T06:54:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.731+0000 7fbfa8ee7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4158f20 con 0x7fbfa40cb150 2023-10-08T06:54:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.731+0000 7fbfa8ee7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4159200 con 0x7fbfa406d4c0 2023-10-08T06:54:37.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40add60 0x7fbfa40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbf9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40add60 0x7fbfa40b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60692/0 (socket says 172.21.15.70:60692) 2023-10-08T06:54:37.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbf9bfff640 1 -- 172.21.15.70:0/3321036978 learned_addr learned my addr 172.21.15.70:0/3321036978 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbfa2d9d640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa40cb150 0x7fbfa41f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbfa259c640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 0x7fbfa4156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbf9bfff640 1 -- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 msgr2=0x7fbfa4156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbf9bfff640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 0x7fbfa4156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbf9bfff640 1 -- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa40cb150 msgr2=0x7fbfa41f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbf9bfff640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa40cb150 0x7fbfa41f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbf9bfff640 1 -- 172.21.15.70:0/3321036978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c009400 con 0x7fbfa40add60 2023-10-08T06:54:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbfa2d9d640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa40cb150 0x7fbfa41f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.732+0000 7fbf9bfff640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40add60 0x7fbfa40b3fd0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c034040 tx=0x7fbf8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.733+0000 7fbf9affd640 1 -- 172.21.15.70:0/3321036978 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c032820 con 0x7fbfa40add60 2023-10-08T06:54:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.733+0000 7fbf9affd640 1 -- 172.21.15.70:0/3321036978 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8c032980 con 0x7fbfa40add60 2023-10-08T06:54:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.733+0000 7fbf9affd640 1 -- 172.21.15.70:0/3321036978 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c032d30 con 0x7fbfa40add60 2023-10-08T06:54:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.733+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3321036978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa41594e0 con 0x7fbfa40add60 2023-10-08T06:54:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.733+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3321036978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa40b4510 con 0x7fbfa40add60 2023-10-08T06:54:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.734+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3321036978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa4067490 con 0x7fbfa40add60 2023-10-08T06:54:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.735+0000 7fbf9affd640 1 -- 172.21.15.70:0/3321036978 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbf8c04e440 con 0x7fbfa40add60 2023-10-08T06:54:37.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.736+0000 7fbf9affd640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fbf8406af90 0x7fbf8406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:37.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.736+0000 7fbf9affd640 1 -- 172.21.15.70:0/3321036978 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fbf8c0abc90 con 0x7fbfa40add60 2023-10-08T06:54:37.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.737+0000 7fbfa259c640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fbf8406af90 0x7fbf8406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.738+0000 7fbfa259c640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fbf8406af90 0x7fbf8406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf940015a0 tx=0x7fbf94009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.740+0000 7fbf9affd640 1 -- 172.21.15.70:0/3321036978 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf8c0768c0 con 0x7fbfa40add60 2023-10-08T06:54:37.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.924+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3321036978 --> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfa40cd1c0 con 0x7fbf8406af90 2023-10-08T06:54:37.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.954+0000 7fbf9affd640 1 -- 172.21.15.70:0/3321036978 <== mgr.6821 v2:172.21.15.181:6832/3709692999 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287938 (secure 0 0 0) 0x7fbf9400d4a0 con 0x7fbf8406af90 2023-10-08T06:54:37.959 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:37.960 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T06:54:37.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.959+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fbf8406af90 msgr2=0x7fbf8406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.959+0000 7fbfa8ee7640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fbf8406af90 0x7fbf8406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf940015a0 tx=0x7fbf94009040 comp rx=0 tx=0).stop 2023-10-08T06:54:37.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.959+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3321036978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40add60 msgr2=0x7fbfa40b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:37.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.959+0000 7fbfa8ee7640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40add60 0x7fbfa40b3fd0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c034040 tx=0x7fbf8c0046c0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.960+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3321036978 shutdown_connections 2023-10-08T06:54:37.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.960+0000 7fbfa8ee7640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fbf8406af90 0x7fbf8406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.960+0000 7fbfa8ee7640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa40cb150 0x7fbfa41f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.960+0000 7fbfa8ee7640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40add60 0x7fbfa40b3fd0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.960+0000 7fbfa8ee7640 1 --2- 172.21.15.70:0/3321036978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa406d4c0 0x7fbfa4156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:37.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.960+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3321036978 >> 172.21.15.70:0/3321036978 conn(0x7fbfa406a380 msgr2=0x7fbfa415fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:37.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.960+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3321036978 shutdown_connections 2023-10-08T06:54:37.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:37.960+0000 7fbfa8ee7640 1 -- 172.21.15.70:0/3321036978 wait complete. 2023-10-08T06:54:37.963 INFO:tasks.ceph.mds.a.smithi070.stderr:2023-10-08T06:54:37.960+0000 7f59ab9c1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T06:54:37.980 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-08T06:54:36.974195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1725,"num_read_kb":3588,"num_write":533,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2789,"ondisk_log_size":2789,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":367184,"kb_used_data":20560,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754607536,"statfs":{"total":773094113280,"available":772718116864,"internally_reserved":0,"allocated":21053440,"data_stored":11373036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":354799163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.976331"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752828+0000","last_change":"2023-10-08T06:54:33.060573+0000","last_active":"2023-10-08T06:54:35.752828+0000","last_peered":"2023-10-08T06:54:35.752828+0000","last_clean":"2023-10-08T06:54:35.752828+0000","last_became_active":"2023-10-08T06:54:33.060193+0000","last_became_peered":"2023-10-08T06:54:33.060193+0000","last_unstale":"2023-10-08T06:54:35.752828+0000","last_undegraded":"2023-10-08T06:54:35.752828+0000","last_fullsized":"2023-10-08T06:54:35.752828+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:18.930716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753733+0000","last_change":"2023-10-08T06:54:33.040076+0000","last_active":"2023-10-08T06:54:35.753733+0000","last_peered":"2023-10-08T06:54:35.753733+0000","last_clean":"2023-10-08T06:54:35.753733+0000","last_became_active":"2023-10-08T06:54:33.039751+0000","last_became_peered":"2023-10-08T06:54:33.039751+0000","last_unstale":"2023-10-08T06:54:35.753733+0000","last_undegraded":"2023-10-08T06:54:35.753733+0000","last_fullsized":"2023-10-08T06:54:35.753733+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:30.676194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751306+0000","last_change":"2023-10-08T06:54:33.074559+0000","last_active":"2023-10-08T06:54:35.751306+0000","last_peered":"2023-10-08T06:54:35.751306+0000","last_clean":"2023-10-08T06:54:35.751306+0000","last_became_active":"2023-10-08T06:54:33.074270+0000","last_became_peered":"2023-10-08T06:54:33.074270+0000","last_unstale":"2023-10-08T06:54:35.751306+0000","last_undegraded":"2023-10-08T06:54:35.751306+0000","last_fullsized":"2023-10-08T06:54:35.751306+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:32.229059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752245+0000","last_change":"2023-10-08T06:54:33.066200+0000","last_active":"2023-10-08T06:54:35.752245+0000","last_peered":"2023-10-08T06:54:35.752245+0000","last_clean":"2023-10-08T06:54:35.752245+0000","last_became_active":"2023-10-08T06:54:33.065790+0000","last_became_peered":"2023-10-08T06:54:33.065790+0000","last_unstale":"2023-10-08T06:54:35.752245+0000","last_undegraded":"2023-10-08T06:54:35.752245+0000","last_fullsized":"2023-10-08T06:54:35.752245+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:57.940917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153750+0000","last_change":"2023-10-08T06:54:33.056727+0000","last_active":"2023-10-08T06:54:34.153750+0000","last_peered":"2023-10-08T06:54:34.153750+0000","last_clean":"2023-10-08T06:54:34.153750+0000","last_became_active":"2023-10-08T06:54:33.056472+0000","last_became_peered":"2023-10-08T06:54:33.056472+0000","last_unstale":"2023-10-08T06:54:34.153750+0000","last_undegraded":"2023-10-08T06:54:34.153750+0000","last_fullsized":"2023-10-08T06:54:34.153750+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:03:29.890214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752009+0000","last_change":"2023-10-08T06:54:33.058338+0000","last_active":"2023-10-08T06:54:35.752009+0000","last_peered":"2023-10-08T06:54:35.752009+0000","last_clean":"2023-10-08T06:54:35.752009+0000","last_became_active":"2023-10-08T06:54:33.057427+0000","last_became_peered":"2023-10-08T06:54:33.057427+0000","last_unstale":"2023-10-08T06:54:35.752009+0000","last_undegraded":"2023-10-08T06:54:35.752009+0000","last_fullsized":"2023-10-08T06:54:35.752009+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:47.092594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026384+0000","last_change":"2023-10-08T06:54:33.063218+0000","last_active":"2023-10-08T06:54:34.026384+0000","last_peered":"2023-10-08T06:54:34.026384+0000","last_clean":"2023-10-08T06:54:34.026384+0000","last_became_active":"2023-10-08T06:54:33.062736+0000","last_became_peered":"2023-10-08T06:54:33.062736+0000","last_unstale":"2023-10-08T06:54:34.026384+0000","last_undegraded":"2023-10-08T06:54:34.026384+0000","last_fullsized":"2023-10-08T06:54:34.026384+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:45.520385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026317+0000","last_change":"2023-10-08T06:54:33.057671+0000","last_active":"2023-10-08T06:54:34.026317+0000","last_peered":"2023-10-08T06:54:34.026317+0000","last_clean":"2023-10-08T06:54:34.026317+0000","last_became_active":"2023-10-08T06:54:33.057360+0000","last_became_peered":"2023-10-08T06:54:33.057360+0000","last_unstale":"2023-10-08T06:54:34.026317+0000","last_undegraded":"2023-10-08T06:54:34.026317+0000","last_fullsized":"2023-10-08T06:54:34.026317+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:45.108097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027340+0000","last_change":"2023-10-08T06:54:33.058685+0000","last_active":"2023-10-08T06:54:34.027340+0000","last_peered":"2023-10-08T06:54:34.027340+0000","last_clean":"2023-10-08T06:54:34.027340+0000","last_became_active":"2023-10-08T06:54:33.058390+0000","last_became_peered":"2023-10-08T06:54:33.058390+0000","last_unstale":"2023-10-08T06:54:34.027340+0000","last_undegraded":"2023-10-08T06:54:34.027340+0000","last_fullsized":"2023-10-08T06:54:34.027340+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:37.433398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027328+0000","last_change":"2023-10-08T06:54:33.061146+0000","last_active":"2023-10-08T06:54:34.027328+0000","last_peered":"2023-10-08T06:54:34.027328+0000","last_clean":"2023-10-08T06:54:34.027328+0000","last_became_active":"2023-10-08T06:54:33.060860+0000","last_became_peered":"2023-10-08T06:54:33.060860+0000","last_unstale":"2023-10-08T06:54:34.027328+0000","last_undegraded":"2023-10-08T06:54:34.027328+0000","last_fullsized":"2023-10-08T06:54:34.027328+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:05.111072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754508+0000","last_change":"2023-10-08T06:54:33.047535+0000","last_active":"2023-10-08T06:54:35.754508+0000","last_peered":"2023-10-08T06:54:35.754508+0000","last_clean":"2023-10-08T06:54:35.754508+0000","last_became_active":"2023-10-08T06:54:33.047244+0000","last_became_peered":"2023-10-08T06:54:33.047244+0000","last_unstale":"2023-10-08T06:54:35.754508+0000","last_undegraded":"2023-10-08T06:54:35.754508+0000","last_fullsized":"2023-10-08T06:54:35.754508+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:41.081711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752599+0000","last_change":"2023-10-08T06:54:33.061136+0000","last_active":"2023-10-08T06:54:35.752599+0000","last_peered":"2023-10-08T06:54:35.752599+0000","last_clean":"2023-10-08T06:54:35.752599+0000","last_became_active":"2023-10-08T06:54:33.060845+0000","last_became_peered":"2023-10-08T06:54:33.060845+0000","last_unstale":"2023-10-08T06:54:35.752599+0000","last_undegraded":"2023-10-08T06:54:35.752599+0000","last_fullsized":"2023-10-08T06:54:35.752599+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:31:23.962246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071150+0000","last_change":"2023-10-08T06:54:33.038054+0000","last_active":"2023-10-08T06:54:34.071150+0000","last_peered":"2023-10-08T06:54:34.071150+0000","last_clean":"2023-10-08T06:54:34.071150+0000","last_became_active":"2023-10-08T06:54:33.037251+0000","last_became_peered":"2023-10-08T06:54:33.037251+0000","last_unstale":"2023-10-08T06:54:34.071150+0000","last_undegraded":"2023-10-08T06:54:34.071150+0000","last_fullsized":"2023-10-08T06:54:34.071150+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:59:04.592809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.152935+0000","last_change":"2023-10-08T06:54:33.068411+0000","last_active":"2023-10-08T06:54:34.152935+0000","last_peered":"2023-10-08T06:54:34.152935+0000","last_clean":"2023-10-08T06:54:34.152935+0000","last_became_active":"2023-10-08T06:54:33.068144+0000","last_became_peered":"2023-10-08T06:54:33.068144+0000","last_unstale":"2023-10-08T06:54:34.152935+0000","last_undegraded":"2023-10-08T06:54:34.152935+0000","last_fullsized":"2023-10-08T06:54:34.152935+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:47.594616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751468+0000","last_change":"2023-10-08T06:54:33.049383+0000","last_active":"2023-10-08T06:54:35.751468+0000","last_peered":"2023-10-08T06:54:35.751468+0000","last_clean":"2023-10-08T06:54:35.751468+0000","last_became_active":"2023-10-08T06:54:33.049095+0000","last_became_peered":"2023-10-08T06:54:33.049095+0000","last_unstale":"2023-10-08T06:54:35.751468+0000","last_undegraded":"2023-10-08T06:54:35.751468+0000","last_fullsized":"2023-10-08T06:54:35.751468+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:52:27.346652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753627+0000","last_change":"2023-10-08T06:54:33.064938+0000","last_active":"2023-10-08T06:54:35.753627+0000","last_peered":"2023-10-08T06:54:35.753627+0000","last_clean":"2023-10-08T06:54:35.753627+0000","last_became_active":"2023-10-08T06:54:33.064580+0000","last_became_peered":"2023-10-08T06:54:33.064580+0000","last_unstale":"2023-10-08T06:54:35.753627+0000","last_undegraded":"2023-10-08T06:54:35.753627+0000","last_fullsized":"2023-10-08T06:54:35.753627+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:46:31.329546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027518+0000","last_change":"2023-10-08T06:54:33.050305+0000","last_active":"2023-10-08T06:54:34.027518+0000","last_peered":"2023-10-08T06:54:34.027518+0000","last_clean":"2023-10-08T06:54:34.027518+0000","last_became_active":"2023-10-08T06:54:33.049829+0000","last_became_peered":"2023-10-08T06:54:33.049829+0000","last_unstale":"2023-10-08T06:54:34.027518+0000","last_undegraded":"2023-10-08T06:54:34.027518+0000","last_fullsized":"2023-10-08T06:54:34.027518+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:06.669276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026431+0000","last_change":"2023-10-08T06:54:33.049856+0000","last_active":"2023-10-08T06:54:34.026431+0000","last_peered":"2023-10-08T06:54:34.026431+0000","last_clean":"2023-10-08T06:54:34.026431+0000","last_became_active":"2023-10-08T06:54:33.049550+0000","last_became_peered":"2023-10-08T06:54:33.049550+0000","last_unstale":"2023-10-08T06:54:34.026431+0000","last_undegraded":"2023-10-08T06:54:34.026431+0000","last_fullsized":"2023-10-08T06:54:34.026431+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:56.201798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754501+0000","last_change":"2023-10-08T06:54:33.050078+0000","last_active":"2023-10-08T06:54:35.754501+0000","last_peered":"2023-10-08T06:54:35.754501+0000","last_clean":"2023-10-08T06:54:35.754501+0000","last_became_active":"2023-10-08T06:54:33.049798+0000","last_became_peered":"2023-10-08T06:54:33.049798+0000","last_unstale":"2023-10-08T06:54:35.754501+0000","last_undegraded":"2023-10-08T06:54:35.754501+0000","last_fullsized":"2023-10-08T06:54:35.754501+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:25.595884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024143+0000","last_change":"2023-10-08T06:54:33.065142+0000","last_active":"2023-10-08T06:54:34.024143+0000","last_peered":"2023-10-08T06:54:34.024143+0000","last_clean":"2023-10-08T06:54:34.024143+0000","last_became_active":"2023-10-08T06:54:33.064871+0000","last_became_peered":"2023-10-08T06:54:33.064871+0000","last_unstale":"2023-10-08T06:54:34.024143+0000","last_undegraded":"2023-10-08T06:54:34.024143+0000","last_fullsized":"2023-10-08T06:54:34.024143+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:58.069314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071296+0000","last_change":"2023-10-08T06:54:33.059200+0000","last_active":"2023-10-08T06:54:34.071296+0000","last_peered":"2023-10-08T06:54:34.071296+0000","last_clean":"2023-10-08T06:54:34.071296+0000","last_became_active":"2023-10-08T06:54:33.058922+0000","last_became_peered":"2023-10-08T06:54:33.058922+0000","last_unstale":"2023-10-08T06:54:34.071296+0000","last_undegraded":"2023-10-08T06:54:34.071296+0000","last_fullsized":"2023-10-08T06:54:34.071296+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:54.253876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752961+0000","last_change":"2023-10-08T06:54:33.054103+0000","last_active":"2023-10-08T06:54:35.752961+0000","last_peered":"2023-10-08T06:54:35.752961+0000","last_clean":"2023-10-08T06:54:35.752961+0000","last_became_active":"2023-10-08T06:54:33.053376+0000","last_became_peered":"2023-10-08T06:54:33.053376+0000","last_unstale":"2023-10-08T06:54:35.752961+0000","last_undegraded":"2023-10-08T06:54:35.752961+0000","last_fullsized":"2023-10-08T06:54:35.752961+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:44:42.544090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153251+0000","last_change":"2023-10-08T06:54:33.049115+0000","last_active":"2023-10-08T06:54:34.153251+0000","last_peered":"2023-10-08T06:54:34.153251+0000","last_clean":"2023-10-08T06:54:34.153251+0000","last_became_active":"2023-10-08T06:54:33.048807+0000","last_became_peered":"2023-10-08T06:54:33.048807+0000","last_unstale":"2023-10-08T06:54:34.153251+0000","last_undegraded":"2023-10-08T06:54:34.153251+0000","last_fullsized":"2023-10-08T06:54:34.153251+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:44:38.005432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.750982+0000","last_change":"2023-10-08T06:54:33.064964+0000","last_active":"2023-10-08T06:54:35.750982+0000","last_peered":"2023-10-08T06:54:35.750982+0000","last_clean":"2023-10-08T06:54:35.750982+0000","last_became_active":"2023-10-08T06:54:33.064576+0000","last_became_peered":"2023-10-08T06:54:33.064576+0000","last_unstale":"2023-10-08T06:54:35.750982+0000","last_undegraded":"2023-10-08T06:54:35.750982+0000","last_fullsized":"2023-10-08T06:54:35.750982+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:00.870715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071763+0000","last_change":"2023-10-08T06:54:33.042671+0000","last_active":"2023-10-08T06:54:34.071763+0000","last_peered":"2023-10-08T06:54:34.071763+0000","last_clean":"2023-10-08T06:54:34.071763+0000","last_became_active":"2023-10-08T06:54:33.042097+0000","last_became_peered":"2023-10-08T06:54:33.042097+0000","last_unstale":"2023-10-08T06:54:34.071763+0000","last_undegraded":"2023-10-08T06:54:34.071763+0000","last_fullsized":"2023-10-08T06:54:34.071763+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:43.301465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024383+0000","last_change":"2023-10-08T06:54:33.063267+0000","last_active":"2023-10-08T06:54:34.024383+0000","last_peered":"2023-10-08T06:54:34.024383+0000","last_clean":"2023-10-08T06:54:34.024383+0000","last_became_active":"2023-10-08T06:54:33.062984+0000","last_became_peered":"2023-10-08T06:54:33.062984+0000","last_unstale":"2023-10-08T06:54:34.024383+0000","last_undegraded":"2023-10-08T06:54:34.024383+0000","last_fullsized":"2023-10-08T06:54:34.024383+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:43.408053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752271+0000","last_change":"2023-10-08T06:54:33.057947+0000","last_active":"2023-10-08T06:54:35.752271+0000","last_peered":"2023-10-08T06:54:35.752271+0000","last_clean":"2023-10-08T06:54:35.752271+0000","last_became_active":"2023-10-08T06:54:33.057585+0000","last_became_peered":"2023-10-08T06:54:33.057585+0000","last_unstale":"2023-10-08T06:54:35.752271+0000","last_undegraded":"2023-10-08T06:54:35.752271+0000","last_fullsized":"2023-10-08T06:54:35.752271+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:29.346638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.070932+0000","last_change":"2023-10-08T06:54:33.064936+0000","last_active":"2023-10-08T06:54:34.070932+0000","last_peered":"2023-10-08T06:54:34.070932+0000","last_clean":"2023-10-08T06:54:34.070932+0000","last_became_active":"2023-10-08T06:54:33.064634+0000","last_became_peered":"2023-10-08T06:54:33.064634+0000","last_unstale":"2023-10-08T06:54:34.070932+0000","last_undegraded":"2023-10-08T06:54:34.070932+0000","last_fullsized":"2023-10-08T06:54:34.070932+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:24:26.246925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752489+0000","last_change":"2023-10-08T06:54:33.049254+0000","last_active":"2023-10-08T06:54:35.752489+0000","last_peered":"2023-10-08T06:54:35.752489+0000","last_clean":"2023-10-08T06:54:35.752489+0000","last_became_active":"2023-10-08T06:54:33.048872+0000","last_became_peered":"2023-10-08T06:54:33.048872+0000","last_unstale":"2023-10-08T06:54:35.752489+0000","last_undegraded":"2023-10-08T06:54:35.752489+0000","last_fullsized":"2023-10-08T06:54:35.752489+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:44.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026737+0000","last_change":"2023-10-08T06:54:33.055510+0000","last_active":"2023-10-08T06:54:34.026737+0000","last_peered":"2023-10-08T06:54:34.026737+0000","last_clean":"2023-10-08T06:54:34.026737+0000","last_became_active":"2023-10-08T06:54:33.055175+0000","last_became_peered":"2023-10-08T06:54:33.055175+0000","last_unstale":"2023-10-08T06:54:34.026737+0000","last_undegraded":"2023-10-08T06:54:34.026737+0000","last_fullsized":"2023-10-08T06:54:34.026737+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:09:41.518993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754194+0000","last_change":"2023-10-08T06:54:33.044545+0000","last_active":"2023-10-08T06:54:35.754194+0000","last_peered":"2023-10-08T06:54:35.754194+0000","last_clean":"2023-10-08T06:54:35.754194+0000","last_became_active":"2023-10-08T06:54:33.044191+0000","last_became_peered":"2023-10-08T06:54:33.044191+0000","last_unstale":"2023-10-08T06:54:35.754194+0000","last_undegraded":"2023-10-08T06:54:35.754194+0000","last_fullsized":"2023-10-08T06:54:35.754194+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:20:37.430101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026187+0000","last_change":"2023-10-08T06:54:33.046114+0000","last_active":"2023-10-08T06:54:34.026187+0000","last_peered":"2023-10-08T06:54:34.026187+0000","last_clean":"2023-10-08T06:54:34.026187+0000","last_became_active":"2023-10-08T06:54:33.045771+0000","last_became_peered":"2023-10-08T06:54:33.045771+0000","last_unstale":"2023-10-08T06:54:34.026187+0000","last_undegraded":"2023-10-08T06:54:34.026187+0000","last_fullsized":"2023-10-08T06:54:34.026187+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:15:15.571393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753366+0000","last_change":"2023-10-08T06:54:33.043116+0000","last_active":"2023-10-08T06:54:35.753366+0000","last_peered":"2023-10-08T06:54:35.753366+0000","last_clean":"2023-10-08T06:54:35.753366+0000","last_became_active":"2023-10-08T06:54:33.042813+0000","last_became_peered":"2023-10-08T06:54:33.042813+0000","last_unstale":"2023-10-08T06:54:35.753366+0000","last_undegraded":"2023-10-08T06:54:35.753366+0000","last_fullsized":"2023-10-08T06:54:35.753366+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:38.937297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"197'4","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754127+0000","last_change":"2023-10-08T06:54:32.043648+0000","last_active":"2023-10-08T06:54:35.754127+0000","last_peered":"2023-10-08T06:54:35.754127+0000","last_clean":"2023-10-08T06:54:35.754127+0000","last_became_active":"2023-10-08T06:54:32.043312+0000","last_became_peered":"2023-10-08T06:54:32.043312+0000","last_unstale":"2023-10-08T06:54:35.754127+0000","last_undegraded":"2023-10-08T06:54:35.754127+0000","last_fullsized":"2023-10-08T06:54:35.754127+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:14.119264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.070557+0000","last_change":"2023-10-08T06:54:33.050590+0000","last_active":"2023-10-08T06:54:34.070557+0000","last_peered":"2023-10-08T06:54:34.070557+0000","last_clean":"2023-10-08T06:54:34.070557+0000","last_became_active":"2023-10-08T06:54:33.050291+0000","last_became_peered":"2023-10-08T06:54:33.050291+0000","last_unstale":"2023-10-08T06:54:34.070557+0000","last_undegraded":"2023-10-08T06:54:34.070557+0000","last_fullsized":"2023-10-08T06:54:34.070557+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:45.208279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"197'1","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153117+0000","last_change":"2023-10-08T06:54:32.050135+0000","last_active":"2023-10-08T06:54:34.153117+0000","last_peered":"2023-10-08T06:54:34.153117+0000","last_clean":"2023-10-08T06:54:34.153117+0000","last_became_active":"2023-10-08T06:54:32.049865+0000","last_became_peered":"2023-10-08T06:54:32.049865+0000","last_unstale":"2023-10-08T06:54:34.153117+0000","last_undegraded":"2023-10-08T06:54:34.153117+0000","last_fullsized":"2023-10-08T06:54:34.153117+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:18.654684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153278+0000","last_change":"2023-10-08T06:54:33.060636+0000","last_active":"2023-10-08T06:54:34.153278+0000","last_peered":"2023-10-08T06:54:34.153278+0000","last_clean":"2023-10-08T06:54:34.153278+0000","last_became_active":"2023-10-08T06:54:33.060354+0000","last_became_peered":"2023-10-08T06:54:33.060354+0000","last_unstale":"2023-10-08T06:54:34.153278+0000","last_undegraded":"2023-10-08T06:54:34.153278+0000","last_fullsized":"2023-10-08T06:54:34.153278+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:08.311019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754776+0000","last_change":"2023-10-08T06:54:32.038290+0000","last_active":"2023-10-08T06:54:35.754776+0000","last_peered":"2023-10-08T06:54:35.754776+0000","last_clean":"2023-10-08T06:54:35.754776+0000","last_became_active":"2023-10-08T06:54:32.037726+0000","last_became_peered":"2023-10-08T06:54:32.037726+0000","last_unstale":"2023-10-08T06:54:35.754776+0000","last_undegraded":"2023-10-08T06:54:35.754776+0000","last_fullsized":"2023-10-08T06:54:35.754776+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:51.656021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752829+0000","last_change":"2023-10-08T06:54:33.047365+0000","last_active":"2023-10-08T06:54:35.752829+0000","last_peered":"2023-10-08T06:54:35.752829+0000","last_clean":"2023-10-08T06:54:35.752829+0000","last_became_active":"2023-10-08T06:54:33.046439+0000","last_became_peered":"2023-10-08T06:54:33.046439+0000","last_unstale":"2023-10-08T06:54:35.752829+0000","last_undegraded":"2023-10-08T06:54:35.752829+0000","last_fullsized":"2023-10-08T06:54:35.752829+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:50.161307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024205+0000","last_change":"2023-10-08T06:54:32.045832+0000","last_active":"2023-10-08T06:54:34.024205+0000","last_peered":"2023-10-08T06:54:34.024205+0000","last_clean":"2023-10-08T06:54:34.024205+0000","last_became_active":"2023-10-08T06:54:32.045554+0000","last_became_peered":"2023-10-08T06:54:32.045554+0000","last_unstale":"2023-10-08T06:54:34.024205+0000","last_undegraded":"2023-10-08T06:54:34.024205+0000","last_fullsized":"2023-10-08T06:54:34.024205+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:04.367253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026500+0000","last_change":"2023-10-08T06:54:33.061146+0000","last_active":"2023-10-08T06:54:34.026500+0000","last_peered":"2023-10-08T06:54:34.026500+0000","last_clean":"2023-10-08T06:54:34.026500+0000","last_became_active":"2023-10-08T06:54:33.060872+0000","last_became_peered":"2023-10-08T06:54:33.060872+0000","last_unstale":"2023-10-08T06:54:34.026500+0000","last_undegraded":"2023-10-08T06:54:34.026500+0000","last_fullsized":"2023-10-08T06:54:34.026500+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:55.630605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"197'1","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026820+0000","last_change":"2023-10-08T06:54:32.045657+0000","last_active":"2023-10-08T06:54:34.026820+0000","last_peered":"2023-10-08T06:54:34.026820+0000","last_clean":"2023-10-08T06:54:34.026820+0000","last_became_active":"2023-10-08T06:54:32.045357+0000","last_became_peered":"2023-10-08T06:54:32.045357+0000","last_unstale":"2023-10-08T06:54:34.026820+0000","last_undegraded":"2023-10-08T06:54:34.026820+0000","last_fullsized":"2023-10-08T06:54:34.026820+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:22.863407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.023942+0000","last_change":"2023-10-08T06:54:33.050456+0000","last_active":"2023-10-08T06:54:34.023942+0000","last_peered":"2023-10-08T06:54:34.023942+0000","last_clean":"2023-10-08T06:54:34.023942+0000","last_became_active":"2023-10-08T06:54:33.050181+0000","last_became_peered":"2023-10-08T06:54:33.050181+0000","last_unstale":"2023-10-08T06:54:34.023942+0000","last_undegraded":"2023-10-08T06:54:34.023942+0000","last_fullsized":"2023-10-08T06:54:34.023942+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:51.336332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"197'1","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153337+0000","last_change":"2023-10-08T06:54:32.037541+0000","last_active":"2023-10-08T06:54:34.153337+0000","last_peered":"2023-10-08T06:54:34.153337+0000","last_clean":"2023-10-08T06:54:34.153337+0000","last_became_active":"2023-10-08T06:54:32.037095+0000","last_became_peered":"2023-10-08T06:54:32.037095+0000","last_unstale":"2023-10-08T06:54:34.153337+0000","last_undegraded":"2023-10-08T06:54:34.153337+0000","last_fullsized":"2023-10-08T06:54:34.153337+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:46.276079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024406+0000","last_change":"2023-10-08T06:54:33.046496+0000","last_active":"2023-10-08T06:54:34.024406+0000","last_peered":"2023-10-08T06:54:34.024406+0000","last_clean":"2023-10-08T06:54:34.024406+0000","last_became_active":"2023-10-08T06:54:33.046207+0000","last_became_peered":"2023-10-08T06:54:33.046207+0000","last_unstale":"2023-10-08T06:54:34.024406+0000","last_undegraded":"2023-10-08T06:54:34.024406+0000","last_fullsized":"2023-10-08T06:54:34.024406+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:35:53.646222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.023588+0000","last_change":"2023-10-08T06:54:32.041108+0000","last_active":"2023-10-08T06:54:34.023588+0000","last_peered":"2023-10-08T06:54:34.023588+0000","last_clean":"2023-10-08T06:54:34.023588+0000","last_became_active":"2023-10-08T06:54:32.040801+0000","last_became_peered":"2023-10-08T06:54:32.040801+0000","last_unstale":"2023-10-08T06:54:34.023588+0000","last_undegraded":"2023-10-08T06:54:34.023588+0000","last_fullsized":"2023-10-08T06:54:34.023588+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:17.680282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753961+0000","last_change":"2023-10-08T06:54:33.059025+0000","last_active":"2023-10-08T06:54:35.753961+0000","last_peered":"2023-10-08T06:54:35.753961+0000","last_clean":"2023-10-08T06:54:35.753961+0000","last_became_active":"2023-10-08T06:54:33.058737+0000","last_became_peered":"2023-10-08T06:54:33.058737+0000","last_unstale":"2023-10-08T06:54:35.753961+0000","last_undegraded":"2023-10-08T06:54:35.753961+0000","last_fullsized":"2023-10-08T06:54:35.753961+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:27.339921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026580+0000","last_change":"2023-10-08T06:54:32.048819+0000","last_active":"2023-10-08T06:54:34.026580+0000","last_peered":"2023-10-08T06:54:34.026580+0000","last_clean":"2023-10-08T06:54:34.026580+0000","last_became_active":"2023-10-08T06:54:32.048555+0000","last_became_peered":"2023-10-08T06:54:32.048555+0000","last_unstale":"2023-10-08T06:54:34.026580+0000","last_undegraded":"2023-10-08T06:54:34.026580+0000","last_fullsized":"2023-10-08T06:54:34.026580+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:30.638813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751338+0000","last_change":"2023-10-08T06:54:33.048206+0000","last_active":"2023-10-08T06:54:35.751338+0000","last_peered":"2023-10-08T06:54:35.751338+0000","last_clean":"2023-10-08T06:54:35.751338+0000","last_became_active":"2023-10-08T06:54:33.047919+0000","last_became_peered":"2023-10-08T06:54:33.047919+0000","last_unstale":"2023-10-08T06:54:35.751338+0000","last_undegraded":"2023-10-08T06:54:35.751338+0000","last_fullsized":"2023-10-08T06:54:35.751338+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:50:53.030168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071451+0000","last_change":"2023-10-08T06:54:32.051894+0000","last_active":"2023-10-08T06:54:34.071451+0000","last_peered":"2023-10-08T06:54:34.071451+0000","last_clean":"2023-10-08T06:54:34.071451+0000","last_became_active":"2023-10-08T06:54:32.051638+0000","last_became_peered":"2023-10-08T06:54:32.051638+0000","last_unstale":"2023-10-08T06:54:34.071451+0000","last_undegraded":"2023-10-08T06:54:34.071451+0000","last_fullsized":"2023-10-08T06:54:34.071451+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:50:52.014244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.070879+0000","last_change":"2023-10-08T06:54:33.057016+0000","last_active":"2023-10-08T06:54:34.070879+0000","last_peered":"2023-10-08T06:54:34.070879+0000","last_clean":"2023-10-08T06:54:34.070879+0000","last_became_active":"2023-10-08T06:54:33.056741+0000","last_became_peered":"2023-10-08T06:54:33.056741+0000","last_unstale":"2023-10-08T06:54:34.070879+0000","last_undegraded":"2023-10-08T06:54:34.070879+0000","last_fullsized":"2023-10-08T06:54:34.070879+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:09.299982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"197'3","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751822+0000","last_change":"2023-10-08T06:54:32.046984+0000","last_active":"2023-10-08T06:54:35.751822+0000","last_peered":"2023-10-08T06:54:35.751822+0000","last_clean":"2023-10-08T06:54:35.751822+0000","last_became_active":"2023-10-08T06:54:32.046651+0000","last_became_peered":"2023-10-08T06:54:32.046651+0000","last_unstale":"2023-10-08T06:54:35.751822+0000","last_undegraded":"2023-10-08T06:54:35.751822+0000","last_fullsized":"2023-10-08T06:54:35.751822+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:07:14.032996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024165+0000","last_change":"2023-10-08T06:54:33.064396+0000","last_active":"2023-10-08T06:54:34.024165+0000","last_peered":"2023-10-08T06:54:34.024165+0000","last_clean":"2023-10-08T06:54:34.024165+0000","last_became_active":"2023-10-08T06:54:33.064103+0000","last_became_peered":"2023-10-08T06:54:33.064103+0000","last_unstale":"2023-10-08T06:54:34.024165+0000","last_undegraded":"2023-10-08T06:54:34.024165+0000","last_fullsized":"2023-10-08T06:54:34.024165+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:09.043727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"197'6","reported_seq":18,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071186+0000","last_change":"2023-10-08T06:54:32.047958+0000","last_active":"2023-10-08T06:54:34.071186+0000","last_peered":"2023-10-08T06:54:34.071186+0000","last_clean":"2023-10-08T06:54:34.071186+0000","last_became_active":"2023-10-08T06:54:32.047693+0000","last_became_peered":"2023-10-08T06:54:32.047693+0000","last_unstale":"2023-10-08T06:54:34.071186+0000","last_undegraded":"2023-10-08T06:54:34.071186+0000","last_fullsized":"2023-10-08T06:54:34.071186+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:44:41.528166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026185+0000","last_change":"2023-10-08T06:54:33.044331+0000","last_active":"2023-10-08T06:54:34.026185+0000","last_peered":"2023-10-08T06:54:34.026185+0000","last_clean":"2023-10-08T06:54:34.026185+0000","last_became_active":"2023-10-08T06:54:33.044023+0000","last_became_peered":"2023-10-08T06:54:33.044023+0000","last_unstale":"2023-10-08T06:54:34.026185+0000","last_undegraded":"2023-10-08T06:54:34.026185+0000","last_fullsized":"2023-10-08T06:54:34.026185+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:52:01.936324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153019+0000","last_change":"2023-10-08T06:54:32.034716+0000","last_active":"2023-10-08T06:54:34.153019+0000","last_peered":"2023-10-08T06:54:34.153019+0000","last_clean":"2023-10-08T06:54:34.153019+0000","last_became_active":"2023-10-08T06:54:32.034376+0000","last_became_peered":"2023-10-08T06:54:32.034376+0000","last_unstale":"2023-10-08T06:54:34.153019+0000","last_undegraded":"2023-10-08T06:54:34.153019+0000","last_fullsized":"2023-10-08T06:54:34.153019+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:36.417473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027075+0000","last_change":"2023-10-08T06:54:33.056311+0000","last_active":"2023-10-08T06:54:34.027075+0000","last_peered":"2023-10-08T06:54:34.027075+0000","last_clean":"2023-10-08T06:54:34.027075+0000","last_became_active":"2023-10-08T06:54:33.056039+0000","last_became_peered":"2023-10-08T06:54:33.056039+0000","last_unstale":"2023-10-08T06:54:34.027075+0000","last_undegraded":"2023-10-08T06:54:34.027075+0000","last_fullsized":"2023-10-08T06:54:34.027075+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:41.769789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.070781+0000","last_change":"2023-10-08T06:54:32.035824+0000","last_active":"2023-10-08T06:54:34.070781+0000","last_peered":"2023-10-08T06:54:34.070781+0000","last_clean":"2023-10-08T06:54:34.070781+0000","last_became_active":"2023-10-08T06:54:32.035561+0000","last_became_peered":"2023-10-08T06:54:32.035561+0000","last_unstale":"2023-10-08T06:54:34.070781+0000","last_undegraded":"2023-10-08T06:54:34.070781+0000","last_fullsized":"2023-10-08T06:54:34.070781+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:44:53.075538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754624+0000","last_change":"2023-10-08T06:54:33.037252+0000","last_active":"2023-10-08T06:54:35.754624+0000","last_peered":"2023-10-08T06:54:35.754624+0000","last_clean":"2023-10-08T06:54:35.754624+0000","last_became_active":"2023-10-08T06:54:33.036860+0000","last_became_peered":"2023-10-08T06:54:33.036860+0000","last_unstale":"2023-10-08T06:54:35.754624+0000","last_undegraded":"2023-10-08T06:54:35.754624+0000","last_fullsized":"2023-10-08T06:54:35.754624+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:50:53.030168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754036+0000","last_change":"2023-10-08T06:54:32.032192+0000","last_active":"2023-10-08T06:54:35.754036+0000","last_peered":"2023-10-08T06:54:35.754036+0000","last_clean":"2023-10-08T06:54:35.754036+0000","last_became_active":"2023-10-08T06:54:32.030540+0000","last_became_peered":"2023-10-08T06:54:32.030540+0000","last_unstale":"2023-10-08T06:54:35.754036+0000","last_undegraded":"2023-10-08T06:54:35.754036+0000","last_fullsized":"2023-10-08T06:54:35.754036+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:20.639692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024796+0000","last_change":"2023-10-08T06:54:33.063188+0000","last_active":"2023-10-08T06:54:34.024796+0000","last_peered":"2023-10-08T06:54:34.024796+0000","last_clean":"2023-10-08T06:54:34.024796+0000","last_became_active":"2023-10-08T06:54:33.062900+0000","last_became_peered":"2023-10-08T06:54:33.062900+0000","last_unstale":"2023-10-08T06:54:34.024796+0000","last_undegraded":"2023-10-08T06:54:34.024796+0000","last_fullsized":"2023-10-08T06:54:34.024796+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:32.134362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"197'8","reported_seq":20,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026824+0000","last_change":"2023-10-08T06:54:32.048261+0000","last_active":"2023-10-08T06:54:34.026824+0000","last_peered":"2023-10-08T06:54:34.026824+0000","last_clean":"2023-10-08T06:54:34.026824+0000","last_became_active":"2023-10-08T06:54:32.047999+0000","last_became_peered":"2023-10-08T06:54:32.047999+0000","last_unstale":"2023-10-08T06:54:34.026824+0000","last_undegraded":"2023-10-08T06:54:34.026824+0000","last_fullsized":"2023-10-08T06:54:34.026824+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:51.338067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027103+0000","last_change":"2023-10-08T06:54:33.068754+0000","last_active":"2023-10-08T06:54:34.027103+0000","last_peered":"2023-10-08T06:54:34.027103+0000","last_clean":"2023-10-08T06:54:34.027103+0000","last_became_active":"2023-10-08T06:54:33.068392+0000","last_became_peered":"2023-10-08T06:54:33.068392+0000","last_unstale":"2023-10-08T06:54:34.027103+0000","last_undegraded":"2023-10-08T06:54:34.027103+0000","last_fullsized":"2023-10-08T06:54:34.027103+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:03.010878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"197'1","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071695+0000","last_change":"2023-10-08T06:54:32.043283+0000","last_active":"2023-10-08T06:54:34.071695+0000","last_peered":"2023-10-08T06:54:34.071695+0000","last_clean":"2023-10-08T06:54:34.071695+0000","last_became_active":"2023-10-08T06:54:32.042956+0000","last_became_peered":"2023-10-08T06:54:32.042956+0000","last_unstale":"2023-10-08T06:54:34.071695+0000","last_undegraded":"2023-10-08T06:54:34.071695+0000","last_fullsized":"2023-10-08T06:54:34.071695+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:37.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026831+0000","last_change":"2023-10-08T06:54:33.042983+0000","last_active":"2023-10-08T06:54:34.026831+0000","last_peered":"2023-10-08T06:54:34.026831+0000","last_clean":"2023-10-08T06:54:34.026831+0000","last_became_active":"2023-10-08T06:54:33.042684+0000","last_became_peered":"2023-10-08T06:54:33.042684+0000","last_unstale":"2023-10-08T06:54:34.026831+0000","last_undegraded":"2023-10-08T06:54:34.026831+0000","last_fullsized":"2023-10-08T06:54:34.026831+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:34.913155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027517+0000","last_change":"2023-10-08T06:54:32.050029+0000","last_active":"2023-10-08T06:54:34.027517+0000","last_peered":"2023-10-08T06:54:34.027517+0000","last_clean":"2023-10-08T06:54:34.027517+0000","last_became_active":"2023-10-08T06:54:32.049367+0000","last_became_peered":"2023-10-08T06:54:32.049367+0000","last_unstale":"2023-10-08T06:54:34.027517+0000","last_undegraded":"2023-10-08T06:54:34.027517+0000","last_fullsized":"2023-10-08T06:54:34.027517+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:37.572215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753578+0000","last_change":"2023-10-08T06:54:33.058492+0000","last_active":"2023-10-08T06:54:35.753578+0000","last_peered":"2023-10-08T06:54:35.753578+0000","last_clean":"2023-10-08T06:54:35.753578+0000","last_became_active":"2023-10-08T06:54:33.058132+0000","last_became_peered":"2023-10-08T06:54:33.058132+0000","last_unstale":"2023-10-08T06:54:35.753578+0000","last_undegraded":"2023-10-08T06:54:35.753578+0000","last_fullsized":"2023-10-08T06:54:35.753578+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:44:54.091462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"197'1","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153435+0000","last_change":"2023-10-08T06:54:32.043662+0000","last_active":"2023-10-08T06:54:34.153435+0000","last_peered":"2023-10-08T06:54:34.153435+0000","last_clean":"2023-10-08T06:54:34.153435+0000","last_became_active":"2023-10-08T06:54:32.043387+0000","last_became_peered":"2023-10-08T06:54:32.043387+0000","last_unstale":"2023-10-08T06:54:34.153435+0000","last_undegraded":"2023-10-08T06:54:34.153435+0000","last_fullsized":"2023-10-08T06:54:34.153435+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:04.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753688+0000","last_change":"2023-10-08T06:54:33.048206+0000","last_active":"2023-10-08T06:54:35.753688+0000","last_peered":"2023-10-08T06:54:35.753688+0000","last_clean":"2023-10-08T06:54:35.753688+0000","last_became_active":"2023-10-08T06:54:33.047923+0000","last_became_peered":"2023-10-08T06:54:33.047923+0000","last_unstale":"2023-10-08T06:54:35.753688+0000","last_undegraded":"2023-10-08T06:54:35.753688+0000","last_fullsized":"2023-10-08T06:54:35.753688+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:20.011867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026757+0000","last_change":"2023-10-08T06:54:32.047616+0000","last_active":"2023-10-08T06:54:34.026757+0000","last_peered":"2023-10-08T06:54:34.026757+0000","last_clean":"2023-10-08T06:54:34.026757+0000","last_became_active":"2023-10-08T06:54:32.046275+0000","last_became_peered":"2023-10-08T06:54:32.046275+0000","last_unstale":"2023-10-08T06:54:34.026757+0000","last_undegraded":"2023-10-08T06:54:34.026757+0000","last_fullsized":"2023-10-08T06:54:34.026757+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:38.417271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071169+0000","last_change":"2023-10-08T06:54:33.064640+0000","last_active":"2023-10-08T06:54:34.071169+0000","last_peered":"2023-10-08T06:54:34.071169+0000","last_clean":"2023-10-08T06:54:34.071169+0000","last_became_active":"2023-10-08T06:54:33.064354+0000","last_became_peered":"2023-10-08T06:54:33.064354+0000","last_unstale":"2023-10-08T06:54:34.071169+0000","last_undegraded":"2023-10-08T06:54:34.071169+0000","last_fullsized":"2023-10-08T06:54:34.071169+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:35.691941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153103+0000","last_change":"2023-10-08T06:54:32.042997+0000","last_active":"2023-10-08T06:54:34.153103+0000","last_peered":"2023-10-08T06:54:34.153103+0000","last_clean":"2023-10-08T06:54:34.153103+0000","last_became_active":"2023-10-08T06:54:32.042708+0000","last_became_peered":"2023-10-08T06:54:32.042708+0000","last_unstale":"2023-10-08T06:54:34.153103+0000","last_undegraded":"2023-10-08T06:54:34.153103+0000","last_fullsized":"2023-10-08T06:54:34.153103+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:08:23.643840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153383+0000","last_change":"2023-10-08T06:54:33.051442+0000","last_active":"2023-10-08T06:54:34.153383+0000","last_peered":"2023-10-08T06:54:34.153383+0000","last_clean":"2023-10-08T06:54:34.153383+0000","last_became_active":"2023-10-08T06:54:33.051192+0000","last_became_peered":"2023-10-08T06:54:33.051192+0000","last_unstale":"2023-10-08T06:54:34.153383+0000","last_undegraded":"2023-10-08T06:54:34.153383+0000","last_fullsized":"2023-10-08T06:54:34.153383+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:35:24.833769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753801+0000","last_change":"2023-10-08T06:54:32.032750+0000","last_active":"2023-10-08T06:54:35.753801+0000","last_peered":"2023-10-08T06:54:35.753801+0000","last_clean":"2023-10-08T06:54:35.753801+0000","last_became_active":"2023-10-08T06:54:32.032019+0000","last_became_peered":"2023-10-08T06:54:32.032019+0000","last_unstale":"2023-10-08T06:54:35.753801+0000","last_undegraded":"2023-10-08T06:54:35.753801+0000","last_fullsized":"2023-10-08T06:54:35.753801+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:31:59.960631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153012+0000","last_change":"2023-10-08T06:54:33.050673+0000","last_active":"2023-10-08T06:54:34.153012+0000","last_peered":"2023-10-08T06:54:34.153012+0000","last_clean":"2023-10-08T06:54:34.153012+0000","last_became_active":"2023-10-08T06:54:33.050385+0000","last_became_peered":"2023-10-08T06:54:33.050385+0000","last_unstale":"2023-10-08T06:54:34.153012+0000","last_undegraded":"2023-10-08T06:54:34.153012+0000","last_fullsized":"2023-10-08T06:54:34.153012+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:44.045711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.023701+0000","last_change":"2023-10-08T06:54:32.046229+0000","last_active":"2023-10-08T06:54:34.023701+0000","last_peered":"2023-10-08T06:54:34.023701+0000","last_clean":"2023-10-08T06:54:34.023701+0000","last_became_active":"2023-10-08T06:54:32.045966+0000","last_became_peered":"2023-10-08T06:54:32.045966+0000","last_unstale":"2023-10-08T06:54:34.023701+0000","last_undegraded":"2023-10-08T06:54:34.023701+0000","last_fullsized":"2023-10-08T06:54:34.023701+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:35.442869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753389+0000","last_change":"2023-10-08T06:54:33.056671+0000","last_active":"2023-10-08T06:54:35.753389+0000","last_peered":"2023-10-08T06:54:35.753389+0000","last_clean":"2023-10-08T06:54:35.753389+0000","last_became_active":"2023-10-08T06:54:33.054122+0000","last_became_peered":"2023-10-08T06:54:33.054122+0000","last_unstale":"2023-10-08T06:54:35.753389+0000","last_undegraded":"2023-10-08T06:54:35.753389+0000","last_fullsized":"2023-10-08T06:54:35.753389+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:56.433839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751857+0000","last_change":"2023-10-08T06:54:32.046908+0000","last_active":"2023-10-08T06:54:35.751857+0000","last_peered":"2023-10-08T06:54:35.751857+0000","last_clean":"2023-10-08T06:54:35.751857+0000","last_became_active":"2023-10-08T06:54:32.046497+0000","last_became_peered":"2023-10-08T06:54:32.046497+0000","last_unstale":"2023-10-08T06:54:35.751857+0000","last_undegraded":"2023-10-08T06:54:35.751857+0000","last_fullsized":"2023-10-08T06:54:35.751857+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:05.399479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752910+0000","last_change":"2023-10-08T06:54:33.039976+0000","last_active":"2023-10-08T06:54:35.752910+0000","last_peered":"2023-10-08T06:54:35.752910+0000","last_clean":"2023-10-08T06:54:35.752910+0000","last_became_active":"2023-10-08T06:54:33.039667+0000","last_became_peered":"2023-10-08T06:54:33.039667+0000","last_unstale":"2023-10-08T06:54:35.752910+0000","last_undegraded":"2023-10-08T06:54:35.752910+0000","last_fullsized":"2023-10-08T06:54:35.752910+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:28.018050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153380+0000","last_change":"2023-10-08T06:54:32.037232+0000","last_active":"2023-10-08T06:54:34.153380+0000","last_peered":"2023-10-08T06:54:34.153380+0000","last_clean":"2023-10-08T06:54:34.153380+0000","last_became_active":"2023-10-08T06:54:32.036959+0000","last_became_peered":"2023-10-08T06:54:32.036959+0000","last_unstale":"2023-10-08T06:54:34.153380+0000","last_undegraded":"2023-10-08T06:54:34.153380+0000","last_fullsized":"2023-10-08T06:54:34.153380+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:05.653352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026687+0000","last_change":"2023-10-08T06:54:33.063379+0000","last_active":"2023-10-08T06:54:34.026687+0000","last_peered":"2023-10-08T06:54:34.026687+0000","last_clean":"2023-10-08T06:54:34.026687+0000","last_became_active":"2023-10-08T06:54:33.063100+0000","last_became_peered":"2023-10-08T06:54:33.063100+0000","last_unstale":"2023-10-08T06:54:34.026687+0000","last_undegraded":"2023-10-08T06:54:34.026687+0000","last_fullsized":"2023-10-08T06:54:34.026687+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:02.271086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071446+0000","last_change":"2023-10-08T06:54:32.052930+0000","last_active":"2023-10-08T06:54:34.071446+0000","last_peered":"2023-10-08T06:54:34.071446+0000","last_clean":"2023-10-08T06:54:34.071446+0000","last_became_active":"2023-10-08T06:54:32.052685+0000","last_became_peered":"2023-10-08T06:54:32.052685+0000","last_unstale":"2023-10-08T06:54:34.071446+0000","last_undegraded":"2023-10-08T06:54:34.071446+0000","last_fullsized":"2023-10-08T06:54:34.071446+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:43.972314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024563+0000","last_change":"2023-10-08T06:54:33.050388+0000","last_active":"2023-10-08T06:54:34.024563+0000","last_peered":"2023-10-08T06:54:34.024563+0000","last_clean":"2023-10-08T06:54:34.024563+0000","last_became_active":"2023-10-08T06:54:33.050119+0000","last_became_peered":"2023-10-08T06:54:33.050119+0000","last_unstale":"2023-10-08T06:54:34.024563+0000","last_undegraded":"2023-10-08T06:54:34.024563+0000","last_fullsized":"2023-10-08T06:54:34.024563+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:47:47.012118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071222+0000","last_change":"2023-10-08T06:54:32.045837+0000","last_active":"2023-10-08T06:54:34.071222+0000","last_peered":"2023-10-08T06:54:34.071222+0000","last_clean":"2023-10-08T06:54:34.071222+0000","last_became_active":"2023-10-08T06:54:32.045576+0000","last_became_peered":"2023-10-08T06:54:32.045576+0000","last_unstale":"2023-10-08T06:54:34.071222+0000","last_undegraded":"2023-10-08T06:54:34.071222+0000","last_fullsized":"2023-10-08T06:54:34.071222+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:18.995943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026778+0000","last_change":"2023-10-08T06:54:33.063026+0000","last_active":"2023-10-08T06:54:34.026778+0000","last_peered":"2023-10-08T06:54:34.026778+0000","last_clean":"2023-10-08T06:54:34.026778+0000","last_became_active":"2023-10-08T06:54:33.062676+0000","last_became_peered":"2023-10-08T06:54:33.062676+0000","last_unstale":"2023-10-08T06:54:34.026778+0000","last_undegraded":"2023-10-08T06:54:34.026778+0000","last_fullsized":"2023-10-08T06:54:34.026778+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:13.212114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024364+0000","last_change":"2023-10-08T06:54:32.049177+0000","last_active":"2023-10-08T06:54:34.024364+0000","last_peered":"2023-10-08T06:54:34.024364+0000","last_clean":"2023-10-08T06:54:34.024364+0000","last_became_active":"2023-10-08T06:54:32.048907+0000","last_became_peered":"2023-10-08T06:54:32.048907+0000","last_unstale":"2023-10-08T06:54:34.024364+0000","last_undegraded":"2023-10-08T06:54:34.024364+0000","last_fullsized":"2023-10-08T06:54:34.024364+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:00:01.585403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752641+0000","last_change":"2023-10-08T06:54:33.059600+0000","last_active":"2023-10-08T06:54:35.752641+0000","last_peered":"2023-10-08T06:54:35.752641+0000","last_clean":"2023-10-08T06:54:35.752641+0000","last_became_active":"2023-10-08T06:54:33.059250+0000","last_became_peered":"2023-10-08T06:54:33.059250+0000","last_unstale":"2023-10-08T06:54:35.752641+0000","last_undegraded":"2023-10-08T06:54:35.752641+0000","last_fullsized":"2023-10-08T06:54:35.752641+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:30:22.875677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751560+0000","last_change":"2023-10-08T06:54:32.046714+0000","last_active":"2023-10-08T06:54:35.751560+0000","last_peered":"2023-10-08T06:54:35.751560+0000","last_clean":"2023-10-08T06:54:35.751560+0000","last_became_active":"2023-10-08T06:54:32.046376+0000","last_became_peered":"2023-10-08T06:54:32.046376+0000","last_unstale":"2023-10-08T06:54:35.751560+0000","last_undegraded":"2023-10-08T06:54:35.751560+0000","last_fullsized":"2023-10-08T06:54:35.751560+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:32:21.385999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752473+0000","last_change":"2023-10-08T06:54:33.057577+0000","last_active":"2023-10-08T06:54:35.752473+0000","last_peered":"2023-10-08T06:54:35.752473+0000","last_clean":"2023-10-08T06:54:35.752473+0000","last_became_active":"2023-10-08T06:54:33.057222+0000","last_became_peered":"2023-10-08T06:54:33.057222+0000","last_unstale":"2023-10-08T06:54:35.752473+0000","last_undegraded":"2023-10-08T06:54:35.752473+0000","last_fullsized":"2023-10-08T06:54:35.752473+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:30.486245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071118+0000","last_change":"2023-10-08T06:54:32.034798+0000","last_active":"2023-10-08T06:54:34.071118+0000","last_peered":"2023-10-08T06:54:34.071118+0000","last_clean":"2023-10-08T06:54:34.071118+0000","last_became_active":"2023-10-08T06:54:32.034515+0000","last_became_peered":"2023-10-08T06:54:32.034515+0000","last_unstale":"2023-10-08T06:54:34.071118+0000","last_undegraded":"2023-10-08T06:54:34.071118+0000","last_fullsized":"2023-10-08T06:54:34.071118+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:40.065787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071189+0000","last_change":"2023-10-08T06:54:33.057531+0000","last_active":"2023-10-08T06:54:34.071189+0000","last_peered":"2023-10-08T06:54:34.071189+0000","last_clean":"2023-10-08T06:54:34.071189+0000","last_became_active":"2023-10-08T06:54:33.057247+0000","last_became_peered":"2023-10-08T06:54:33.057247+0000","last_unstale":"2023-10-08T06:54:34.071189+0000","last_undegraded":"2023-10-08T06:54:34.071189+0000","last_fullsized":"2023-10-08T06:54:34.071189+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:45.453690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754288+0000","last_change":"2023-10-08T06:54:32.037190+0000","last_active":"2023-10-08T06:54:35.754288+0000","last_peered":"2023-10-08T06:54:35.754288+0000","last_clean":"2023-10-08T06:54:35.754288+0000","last_became_active":"2023-10-08T06:54:32.036775+0000","last_became_peered":"2023-10-08T06:54:32.036775+0000","last_unstale":"2023-10-08T06:54:35.754288+0000","last_undegraded":"2023-10-08T06:54:35.754288+0000","last_fullsized":"2023-10-08T06:54:35.754288+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:38:38.588149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024670+0000","last_change":"2023-10-08T06:54:33.070533+0000","last_active":"2023-10-08T06:54:34.024670+0000","last_peered":"2023-10-08T06:54:34.024670+0000","last_clean":"2023-10-08T06:54:34.024670+0000","last_became_active":"2023-10-08T06:54:33.070235+0000","last_became_peered":"2023-10-08T06:54:33.070235+0000","last_unstale":"2023-10-08T06:54:34.024670+0000","last_undegraded":"2023-10-08T06:54:34.024670+0000","last_fullsized":"2023-10-08T06:54:34.024670+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:28.076640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.152729+0000","last_change":"2023-10-08T06:54:32.039639+0000","last_active":"2023-10-08T06:54:34.152729+0000","last_peered":"2023-10-08T06:54:34.152729+0000","last_clean":"2023-10-08T06:54:34.152729+0000","last_became_active":"2023-10-08T06:54:32.039367+0000","last_became_peered":"2023-10-08T06:54:32.039367+0000","last_unstale":"2023-10-08T06:54:34.152729+0000","last_undegraded":"2023-10-08T06:54:34.152729+0000","last_fullsized":"2023-10-08T06:54:34.152729+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:40.390656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751966+0000","last_change":"2023-10-08T06:54:33.047603+0000","last_active":"2023-10-08T06:54:35.751966+0000","last_peered":"2023-10-08T06:54:35.751966+0000","last_clean":"2023-10-08T06:54:35.751966+0000","last_became_active":"2023-10-08T06:54:33.046439+0000","last_became_peered":"2023-10-08T06:54:33.046439+0000","last_unstale":"2023-10-08T06:54:35.751966+0000","last_undegraded":"2023-10-08T06:54:35.751966+0000","last_fullsized":"2023-10-08T06:54:35.751966+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.647045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071422+0000","last_change":"2023-10-08T06:54:32.052930+0000","last_active":"2023-10-08T06:54:34.071422+0000","last_peered":"2023-10-08T06:54:34.071422+0000","last_clean":"2023-10-08T06:54:34.071422+0000","last_became_active":"2023-10-08T06:54:32.052513+0000","last_became_peered":"2023-10-08T06:54:32.052513+0000","last_unstale":"2023-10-08T06:54:34.071422+0000","last_undegraded":"2023-10-08T06:54:34.071422+0000","last_fullsized":"2023-10-08T06:54:34.071422+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:31:22.946322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"187'2751","reported_seq":3486,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-08T06:54:33.029296+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T06:54:33.029296+0000","last_peered":"2023-10-08T06:54:33.029296+0000","last_clean":"2023-10-08T06:54:33.029296+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T06:54:33.029296+0000","last_undegraded":"2023-10-08T06:54:33.029296+0000","last_fullsized":"2023-10-08T06:54:33.029296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2751,"log_dups_size":0,"ondisk_log_size":2751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1725,"num_read_kb":3588,"num_write":489,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1725,"num_read_kb":3588,"num_write":489,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2751,"ondisk_log_size":2751,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575784,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60548,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311292,"statfs":{"total":96636764160,"available":96574763008,"internally_reserved":0,"allocated":3072000,"data_stored":1865487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.2010000000000001,"15min":1.218},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.462,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.206,"15min":1.2470000000000001},"min":{"1min":0.75800000000000001,"5min":0.747,"15min":0.72899999999999998},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.849},"last":0.85299999999999998}]},{"osd":1,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2589999999999999,"15min":1.2609999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.387},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.284,"15min":1.3149999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.82999999999999996},"max":{"1min":1.7290000000000001,"5min":1.9299999999999999,"15min":2.4809999999999999},"last":1.107}]},{"osd":2,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.2769999999999999,"15min":1.2909999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4970000000000001,"5min":2.7040000000000002,"15min":3.4350000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3520000000000001,"15min":1.3640000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.0019999999999998,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":1.014}]},{"osd":3,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3180000000000001,"15min":1.323},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.637,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.379,"5min":1.4550000000000001,"15min":1.419},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.2240000000000002,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.286}]},{"osd":4,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.373,"15min":1.292},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.66200000000000003},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":3.2839999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.444,"15min":1.3879999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.9930000000000001,"5min":2.004,"15min":2.1909999999999998},"last":1.4690000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4810000000000001,"15min":1.4390000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.3319999999999999,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":1.45},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.528,"15min":1.4850000000000001},"min":{"1min":0.97799999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.4219999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5740000000000001,"15min":1.5589999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":2.4209999999999998,"5min":2.4350000000000001,"15min":3.1360000000000001},"last":1.329},{"interface":"front","average":{"1min":1.514,"5min":1.5469999999999999,"15min":1.5029999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.3140000000000001},"last":0.96099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575783,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41920,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329920,"statfs":{"total":96636764160,"available":96593838080,"internally_reserved":0,"allocated":2478080,"data_stored":1275119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.264,"15min":1.337},"min":{"1min":0.68899999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.508},"last":1.573},{"interface":"front","average":{"1min":1.325,"5min":1.302,"15min":1.339},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":3.7519999999999998,"5min":3.7519999999999998,"15min":3.7519999999999998},"last":1.302}]},{"osd":1,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.222,"15min":1.244},"min":{"1min":0.623,"5min":0.623,"15min":0.59699999999999998},"max":{"1min":1.5800000000000001,"5min":1.821,"15min":1.821},"last":1.383},{"interface":"front","average":{"1min":1.256,"5min":1.2689999999999999,"15min":1.2629999999999999},"min":{"1min":0.83099999999999996,"5min":0.65200000000000002,"15min":0.53800000000000003},"max":{"1min":1.7809999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.93799999999999994}]},{"osd":2,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.2470000000000001,"15min":1.2390000000000001},"min":{"1min":0.88100000000000001,"5min":0.85099999999999998,"15min":0.75600000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":1.653},{"interface":"front","average":{"1min":1.137,"5min":1.26,"15min":1.288},"min":{"1min":0.872,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4950000000000001,"5min":1.6379999999999999,"15min":2.3839999999999999},"last":1.2430000000000001}]},{"osd":3,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4199999999999999,"15min":1.3999999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.83399999999999996},"max":{"1min":2.048,"5min":2.048,"15min":2.1499999999999999},"last":1.7969999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4730000000000001,"15min":1.4790000000000001},"min":{"1min":1.01,"5min":0.96699999999999997,"15min":0.89400000000000002},"max":{"1min":1.829,"5min":1.915,"15min":2.0289999999999999},"last":1.125}]},{"osd":4,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3859999999999999,"15min":1.357},"min":{"1min":0.89100000000000001,"5min":0.80100000000000005,"15min":0.72399999999999998},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.4670000000000001,"15min":1.454},"min":{"1min":1.006,"5min":0.878,"15min":0.80200000000000005},"max":{"1min":2.9199999999999999,"5min":2.9199999999999999,"15min":2.9199999999999999},"last":1.4199999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4299999999999999,"15min":1.4159999999999999},"min":{"1min":0.97499999999999998,"5min":0.94799999999999995,"15min":0.69499999999999995},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.7309999999999999},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.5589999999999999,"15min":1.5489999999999999},"min":{"1min":1.099,"5min":1.0249999999999999,"15min":0.78300000000000003},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.9419999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.776,"5min":1.629,"15min":1.5940000000000001},"min":{"1min":1.351,"5min":1.0800000000000001,"15min":0.92000000000000004},"max":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":1.889},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5600000000000001,"15min":1.5640000000000001},"min":{"1min":1.1319999999999999,"5min":0.90800000000000003,"15min":0.82099999999999995},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.6070000000000002},"last":1.48}]}]},{"osd":5,"up_from":13,"seq":55834575783,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59972,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311868,"statfs":{"total":96636764160,"available":96575352832,"internally_reserved":0,"allocated":2473984,"data_stored":1268507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2310000000000001,"15min":1.218},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.66600000000000004},"max":{"1min":1.5740000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.296,"15min":1.2569999999999999},"min":{"1min":0.95399999999999996,"5min":0.82699999999999996,"15min":0.78800000000000003},"max":{"1min":1.629,"5min":1.897,"15min":2.4700000000000002},"last":0.91700000000000004}]},{"osd":1,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.26,"15min":1.2230000000000001},"min":{"1min":0.95499999999999996,"5min":0.85799999999999998,"15min":0.72499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.375,"15min":1.323},"min":{"1min":0.90400000000000003,"5min":0.88700000000000001,"15min":0.78200000000000003},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.7150000000000001}]},{"osd":2,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3480000000000001,"15min":1.3280000000000001},"min":{"1min":0.98399999999999999,"5min":0.85199999999999998,"15min":0.83199999999999996},"max":{"1min":1.6479999999999999,"5min":1.9359999999999999,"15min":2.351},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.3160000000000001},"min":{"1min":0.91000000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.667,"5min":1.7809999999999999,"15min":1.964},"last":1.0820000000000001}]},{"osd":3,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3959999999999999,"15min":1.3540000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.528,"15min":1.464},"min":{"1min":1.139,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.9910000000000001,"5min":3.0059999999999998,"15min":5.1630000000000003},"last":1.125}]},{"osd":4,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3320000000000001,"15min":1.3240000000000001},"min":{"1min":0.96299999999999997,"5min":0.89300000000000002,"15min":0.81499999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4039999999999999},"last":1.383},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4379999999999999,"15min":1.4239999999999999},"min":{"1min":0.96899999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.7989999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.4990000000000001,"15min":1.4750000000000001},"min":{"1min":1.248,"5min":0.96399999999999997,"15min":0.81200000000000006},"max":{"1min":2.1150000000000002,"5min":2.1659999999999999,"15min":5.5019999999999998},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4930000000000001,"15min":1.48},"min":{"1min":1.0509999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.0139999999999998,"5min":2.1040000000000001,"15min":2.2890000000000001},"last":1.466}]},{"osd":7,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.5660000000000001,"15min":1.5629999999999999},"min":{"1min":1.3400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":5.8780000000000001},"last":1.869},{"interface":"front","average":{"1min":1.679,"5min":1.6839999999999999,"15min":1.6299999999999999},"min":{"1min":1.415,"5min":1.1719999999999999,"15min":0.97099999999999997},"max":{"1min":2.206,"5min":2.6789999999999998,"15min":5.399},"last":1.671}]}]},{"osd":4,"up_from":13,"seq":55834575783,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41932,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329908,"statfs":{"total":96636764160,"available":96593825792,"internally_reserved":0,"allocated":2482176,"data_stored":1269147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.389,"15min":1.3460000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":2.0880000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.3999999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.69799999999999995},"max":{"1min":1.7789999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.5249999999999999}]},{"osd":1,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2749999999999999,"15min":1.28},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.73,"5min":1.784,"15min":1.9339999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.64000000000000001},"max":{"1min":1.613,"5min":1.6870000000000001,"15min":1.7290000000000001},"last":1.329}]},{"osd":2,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.2889999999999999,"15min":1.248},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.3759999999999999},"min":{"1min":1,"5min":0.93100000000000005,"15min":0.85399999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.393}]},{"osd":3,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.339,"15min":1.3109999999999999},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.206},{"interface":"front","average":{"1min":1.55,"5min":1.4670000000000001,"15min":1.4099999999999999},"min":{"1min":1.0629999999999999,"5min":0.97299999999999998,"15min":0.93400000000000005},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.867}]},{"osd":5,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.4139999999999999,"15min":1.351},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.66700000000000004},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.4450000000000001,"15min":1.375},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.749},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.7150000000000001}]},{"osd":6,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.5329999999999999,"15min":1.425},"min":{"1min":0.97899999999999998,"5min":0.91100000000000003,"15min":0.72699999999999998},"max":{"1min":5.3339999999999996,"5min":5.3339999999999996,"15min":5.3339999999999996},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6120000000000001,"15min":1.546},"min":{"1min":1.212,"5min":0.96699999999999997,"15min":0.91400000000000003},"max":{"1min":2.1059999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":1.669}]},{"osd":7,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.548,"15min":1.49},"min":{"1min":1.202,"5min":1.135,"15min":0.99099999999999999},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.784,"5min":1.6859999999999999,"15min":1.6060000000000001},"min":{"1min":1.0049999999999999,"5min":0.98999999999999999,"15min":0.93600000000000005},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":2.355}]}]},{"osd":3,"up_from":13,"seq":55834575785,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47512,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324328,"statfs":{"total":96636764160,"available":96588111872,"internally_reserved":0,"allocated":2498560,"data_stored":1277335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.92100000000000004,"15min":0.95099999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.44500000000000001},"max":{"1min":1.1910000000000001,"5min":1.71,"15min":1.71},"last":1.4339999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.1499999999999999,"15min":1.165},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.2330000000000001,"5min":1.851,"15min":2.137},"last":1.0600000000000001}]},{"osd":1,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.98199999999999998,"15min":1.0049999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.341,"5min":1.659,"15min":1.823},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.2430000000000001,"15min":1.2869999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.613,"5min":1.75,"15min":1.893},"last":1.5349999999999999}]},{"osd":2,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.1299999999999999,"15min":1.117},"min":{"1min":0.57299999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.423,"5min":1.6899999999999999,"15min":1.7410000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.1479999999999999,"15min":1.179},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.454,"5min":1.7809999999999999,"15min":1.8360000000000001},"last":0.84199999999999997}]},{"osd":4,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.268,"15min":1.2629999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7370000000000001,"5min":1.97,"15min":1.97},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.4399999999999999,"15min":1.446},"min":{"1min":1.0149999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.7070000000000001,"5min":1.905,"15min":2.1960000000000002},"last":1.278}]},{"osd":5,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.341,"15min":1.369},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.73,"5min":2.27,"15min":3.0009999999999999},"last":1.107},{"interface":"front","average":{"1min":1.25,"5min":1.389,"15min":1.4119999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.724,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.806}]},{"osd":6,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3720000000000001,"15min":1.3879999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.748,"5min":1.9950000000000001,"15min":2.016},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4990000000000001,"15min":1.508},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":0.99099999999999999},"max":{"1min":1.8029999999999999,"5min":2.0489999999999999,"15min":2.544},"last":1.353}]},{"osd":7,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.5329999999999999,"15min":1.5589999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.948,"5min":2.008,"15min":2.347},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4790000000000001,"15min":1.474},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.873,"5min":2.1099999999999999,"15min":2.1800000000000002},"last":1.2}]}]},{"osd":2,"up_from":13,"seq":55834575783,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47496,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324344,"statfs":{"total":96636764160,"available":96588128256,"internally_reserved":0,"allocated":2478080,"data_stored":1275119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.88400000000000001,"15min":0.91900000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.167,"5min":1.345,"15min":1.657},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.046,"15min":1.073},"min":{"1min":0.46800000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3360000000000001,"5min":1.661,"15min":2.2679999999999998},"last":0.53300000000000003}]},{"osd":1,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.98999999999999999,"15min":0.996},"min":{"1min":0.52900000000000003,"5min":0.373,"15min":0.373},"max":{"1min":1.637,"5min":1.7170000000000001,"15min":1.996},"last":0.495},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.1220000000000001,"15min":1.1659999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5269999999999999,"5min":1.8500000000000001,"15min":2.1139999999999999},"last":0.624}]},{"osd":3,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2190000000000001,"15min":1.2230000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.956,"5min":1.956,"15min":2.3929999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.228,"15min":1.2490000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7050000000000001,"5min":1.804,"15min":1.891},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4079999999999999,"15min":1.4279999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.776,"5min":1.9430000000000001,"15min":3.0939999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.321,"5min":1.3180000000000001,"15min":1.3260000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":3.7320000000000002},"last":0.59599999999999997}]},{"osd":5,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.417,"15min":1.4279999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8540000000000001,"5min":1.9770000000000001,"15min":3.3519999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.478,"15min":1.512},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.911,"5min":2.2719999999999998,"15min":2.5249999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.446,"15min":1.411},"min":{"1min":1,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.798,"5min":1.9990000000000001,"15min":2.206},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.605,"5min":1.575,"15min":1.54},"min":{"1min":1.177,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":3.02},"last":0.78500000000000003}]},{"osd":7,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.5169999999999999,"15min":1.4990000000000001},"min":{"1min":0.81000000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.9259999999999999,"5min":2.0790000000000002,"15min":4.2869999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.528,"5min":1.5860000000000001,"15min":1.579},"min":{"1min":0.89100000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":3.0760000000000001},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575783,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29456,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342384,"statfs":{"total":96636764160,"available":96606601216,"internally_reserved":0,"allocated":2490368,"data_stored":1276731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.333,"5min":4.601,"15min":4.601},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.155,"15min":1.2},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.613,"5min":1.917,"15min":1.917},"last":0.92100000000000004}]},{"osd":2,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.98899999999999999,"15min":1.0149999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1519999999999999,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":0.874},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.1759999999999999,"15min":1.1579999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5049999999999999,"5min":5.4059999999999997,"15min":5.4059999999999997},"last":1.1659999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.256,"15min":1.248},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5289999999999999,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.115,"15min":1.107},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.484,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.0449999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.702,"5min":4.2409999999999997,"15min":4.2409999999999997},"last":0.995},{"interface":"front","average":{"1min":1.157,"5min":1.302,"15min":1.2929999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6879999999999999,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":1.4279999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.391,"15min":1.3520000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5189999999999999,"5min":4.7530000000000001,"15min":4.7530000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.5529999999999999,"15min":1.5249999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.6699999999999999,"5min":5.7389999999999999,"15min":5.7389999999999999},"last":1.266}]},{"osd":6,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.5249999999999999,"15min":1.516},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8140000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":1.083},{"interface":"front","average":{"1min":1.349,"5min":1.506,"15min":1.4770000000000001},"min":{"1min":0.86899999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8169999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":1.5700000000000001}]},{"osd":7,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.5529999999999999,"15min":1.542},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7450000000000001,"5min":5.6289999999999996,"15min":5.6289999999999996},"last":1.385},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.5529999999999999,"15min":1.5429999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9039999999999999,"5min":4.5270000000000001,"15min":4.5270000000000001},"last":1.1240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575784,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38348,"kb_used_data":3008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333492,"statfs":{"total":96636764160,"available":96597495808,"internally_reserved":0,"allocated":3080192,"data_stored":1865591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.113,"15min":1.014},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":14.417999999999999,"5min":14.417999999999999,"15min":14.417999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1419999999999999,"15min":1.091},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":8.5289999999999999,"5min":8.5289999999999999,"15min":8.5289999999999999},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.605,"5min":1.3029999999999999,"15min":1.1899999999999999},"min":{"1min":0.54100000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":14.459,"5min":14.459,"15min":14.459},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.995,"5min":1.042,"15min":1.071},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.502}]},{"osd":3,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.7669999999999999,"5min":1.3839999999999999,"15min":1.3009999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":14.364000000000001,"5min":14.364000000000001,"15min":14.364000000000001},"last":1.099},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2170000000000001,"15min":1.1910000000000001},"min":{"1min":0.47399999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":8.5609999999999999,"5min":8.5609999999999999,"15min":8.5609999999999999},"last":1.0349999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3029999999999999,"15min":1.3520000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.859,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.143},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.466,"15min":1.4259999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":8.4450000000000003,"5min":8.4450000000000003,"15min":8.4450000000000003},"last":1.3979999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":2.0390000000000001,"5min":1.631,"15min":1.5149999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":14.515000000000001,"5min":14.515000000000001,"15min":14.515000000000001},"last":1.2130000000000001}]},{"osd":6,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.528,"15min":1.4910000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":8.4939999999999998,"5min":8.4939999999999998,"15min":8.4939999999999998},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.492,"15min":1.504},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.7779999999999996,"5min":4.7779999999999996,"15min":4.7779999999999996},"last":1.3759999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.5289999999999999,"15min":1.532},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.8260000000000001,"5min":1.613,"15min":1.5669999999999999},"min":{"1min":0.92100000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":10.195,"5min":10.195,"15min":10.195},"last":1.484}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T06:54:37.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T06:54:38.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.240+0000 7f0760bed640 1 Processor -- start 2023-10-08T06:54:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.240+0000 7f0760bed640 1 -- start start 2023-10-08T06:54:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.241+0000 7f0760bed640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f075c0b07d0 0x7f075c165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.241+0000 7f0760bed640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f075c15f7d0 con 0x7f075c0adc70 2023-10-08T06:54:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.241+0000 7f0760bed640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f075c15efb0 con 0x7f075c15ab30 2023-10-08T06:54:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.241+0000 7f0760bed640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f075c15f210 con 0x7f075c0b07d0 2023-10-08T06:54:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.242+0000 7f0753fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f075c0adc70 0x7f075c0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37760/0 (socket says 172.21.15.70:37760) 2023-10-08T06:54:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.242+0000 7f0753fff640 1 -- 172.21.15.70:0/3407048413 learned_addr learned my addr 172.21.15.70:0/3407048413 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.242+0000 7f075bfff640 1 --2- 172.21.15.70:0/3407048413 >> v2:172.21.15.70:3300/0 conn(0x7f075c0b07d0 0x7f075c165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.242+0000 7f075affd640 1 -- 172.21.15.70:0/3407048413 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537242391 0 0) 0x7f075c15f7d0 con 0x7f075c0adc70 2023-10-08T06:54:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.242+0000 7f075affd640 1 -- 172.21.15.70:0/3407048413 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0748003520 con 0x7f075c0adc70 2023-10-08T06:54:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.242+0000 7f075bfff640 1 -- 172.21.15.70:0/3407048413 >> v1:172.21.15.181:6789/0 conn(0x7f075c15ab30 legacy=0x7f075c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.242+0000 7f075bfff640 1 -- 172.21.15.70:0/3407048413 >> v1:172.21.15.70:6789/0 conn(0x7f075c0adc70 legacy=0x7f075c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.242+0000 7f075bfff640 1 -- 172.21.15.70:0/3407048413 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c15f4f0 con 0x7f075c0b07d0 2023-10-08T06:54:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.243+0000 7f075bfff640 1 --2- 172.21.15.70:0/3407048413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 0x7f075c165bd0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f075c0ca4a0 tx=0x7f075402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd278d7e8ada04fe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.244+0000 7f075affd640 1 -- 172.21.15.70:0/3407048413 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075402cc40 con 0x7f075c0b07d0 2023-10-08T06:54:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.244+0000 7f075affd640 1 -- 172.21.15.70:0/3407048413 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075402cda0 con 0x7f075c0b07d0 2023-10-08T06:54:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.244+0000 7f075affd640 1 -- 172.21.15.70:0/3407048413 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754034660 con 0x7f075c0b07d0 2023-10-08T06:54:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.244+0000 7f0760bed640 1 -- 172.21.15.70:0/3407048413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 msgr2=0x7f075c165bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.244+0000 7f0760bed640 1 --2- 172.21.15.70:0/3407048413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 0x7f075c165bd0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f075c0ca4a0 tx=0x7f075402be90 comp rx=0 tx=0).stop 2023-10-08T06:54:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.244+0000 7f0760bed640 1 -- 172.21.15.70:0/3407048413 shutdown_connections 2023-10-08T06:54:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.244+0000 7f0760bed640 1 --2- 172.21.15.70:0/3407048413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 0x7f075c165bd0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.244+0000 7f0760bed640 1 -- 172.21.15.70:0/3407048413 >> 172.21.15.70:0/3407048413 conn(0x7f075c0cb0c0 msgr2=0x7f075c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.244+0000 7f0760bed640 1 -- 172.21.15.70:0/3407048413 shutdown_connections 2023-10-08T06:54:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.244+0000 7f0760bed640 1 -- 172.21.15.70:0/3407048413 wait complete. 2023-10-08T06:54:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.246+0000 7f0760bed640 1 Processor -- start 2023-10-08T06:54:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.246+0000 7f0760bed640 1 -- start start 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.247+0000 7f0760bed640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f075c0adc70 0x7f075c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.247+0000 7f0760bed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 0x7f075c15da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.247+0000 7f0760bed640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f075c15ab30 0x7f075c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.247+0000 7f0760bed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c0c67e0 con 0x7f075c0b07d0 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.247+0000 7f0760bed640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c0c6ac0 con 0x7f075c0adc70 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.247+0000 7f0760bed640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075c0c6da0 con 0x7f075c15ab30 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.247+0000 7f0753fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 0x7f075c15da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.247+0000 7f0753fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 0x7f075c15da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60706/0 (socket says 172.21.15.70:60706) 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.247+0000 7f0753fff640 1 -- 172.21.15.70:0/661617778 learned_addr learned my addr 172.21.15.70:0/661617778 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f075b7fe640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f075c0adc70 0x7f075c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f0753fff640 1 -- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f075c15ab30 msgr2=0x7f075c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f0753fff640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f075c15ab30 0x7f075c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f0753fff640 1 -- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f075c0adc70 msgr2=0x7f075c15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f0753fff640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f075c0adc70 0x7f075c15b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f0753fff640 1 -- 172.21.15.70:0/661617778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0754009a70 con 0x7f075c0b07d0 2023-10-08T06:54:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f075b7fe640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f075c0adc70 0x7f075c15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f0753fff640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 0x7f075c15da00 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f074400c500 tx=0x7f074400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f0758ff9640 1 -- 172.21.15.70:0/661617778 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744005ce0 con 0x7f075c0b07d0 2023-10-08T06:54:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f0758ff9640 1 -- 172.21.15.70:0/661617778 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0744005e40 con 0x7f075c0b07d0 2023-10-08T06:54:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f0758ff9640 1 -- 172.21.15.70:0/661617778 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744007330 con 0x7f075c0b07d0 2023-10-08T06:54:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.248+0000 7f0760bed640 1 -- 172.21.15.70:0/661617778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075c0c7080 con 0x7f075c0b07d0 2023-10-08T06:54:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.249+0000 7f0760bed640 1 -- 172.21.15.70:0/661617778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075c1493b0 con 0x7f075c0b07d0 2023-10-08T06:54:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.250+0000 7f0760bed640 1 -- 172.21.15.70:0/661617778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075c04e160 con 0x7f075c0b07d0 2023-10-08T06:54:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.251+0000 7f0758ff9640 1 -- 172.21.15.70:0/661617778 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0744007550 con 0x7f075c0b07d0 2023-10-08T06:54:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.252+0000 7f0758ff9640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f073c06af90 0x7f073c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.252+0000 7f0758ff9640 1 -- 172.21.15.70:0/661617778 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f0744088fc0 con 0x7f075c0b07d0 2023-10-08T06:54:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.253+0000 7f075b7fe640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f073c06af90 0x7f073c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.254+0000 7f075b7fe640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f073c06af90 0x7f073c06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f074c00a830 tx=0x7f074c008040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.257+0000 7f0758ff9640 1 -- 172.21.15.70:0/661617778 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0744053e50 con 0x7f075c0b07d0 2023-10-08T06:54:38.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.442+0000 7f0760bed640 1 -- 172.21.15.70:0/661617778 --> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f075c0c6c30 con 0x7f073c06af90 2023-10-08T06:54:38.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.464+0000 7f0758ff9640 1 -- 172.21.15.70:0/661617778 <== mgr.6821 v2:172.21.15.181:6832/3709692999 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287938 (secure 0 0 0) 0x7f075c0c6c30 con 0x7f073c06af90 2023-10-08T06:54:38.469 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:38.471 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T06:54:38.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 -- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f073c06af90 msgr2=0x7f073c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:38.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f073c06af90 0x7f073c06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f074c00a830 tx=0x7f074c008040 comp rx=0 tx=0).stop 2023-10-08T06:54:38.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 -- 172.21.15.70:0/661617778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 msgr2=0x7f075c15da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:38.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 0x7f075c15da00 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f074400c500 tx=0x7f074400c9c0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 -- 172.21.15.70:0/661617778 shutdown_connections 2023-10-08T06:54:38.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f073c06af90 0x7f073c06d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f075c15ab30 0x7f075c0af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f075c0b07d0 0x7f075c15da00 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 --2- 172.21.15.70:0/661617778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f075c0adc70 0x7f075c15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 -- 172.21.15.70:0/661617778 >> 172.21.15.70:0/661617778 conn(0x7f075c0cb0c0 msgr2=0x7f075c0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:38.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 -- 172.21.15.70:0/661617778 shutdown_connections 2023-10-08T06:54:38.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.470+0000 7f0760bed640 1 -- 172.21.15.70:0/661617778 wait complete. 2023-10-08T06:54:38.491 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-08T06:54:36.974195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1725,"num_read_kb":3588,"num_write":533,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2789,"ondisk_log_size":2789,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":367184,"kb_used_data":20560,"kb_used_omap":12,"kb_used_meta":346483,"kb_avail":754607536,"statfs":{"total":773094113280,"available":772718116864,"internally_reserved":0,"allocated":21053440,"data_stored":11373036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":354799163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.976331"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752828+0000","last_change":"2023-10-08T06:54:33.060573+0000","last_active":"2023-10-08T06:54:35.752828+0000","last_peered":"2023-10-08T06:54:35.752828+0000","last_clean":"2023-10-08T06:54:35.752828+0000","last_became_active":"2023-10-08T06:54:33.060193+0000","last_became_peered":"2023-10-08T06:54:33.060193+0000","last_unstale":"2023-10-08T06:54:35.752828+0000","last_undegraded":"2023-10-08T06:54:35.752828+0000","last_fullsized":"2023-10-08T06:54:35.752828+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:18.930716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753733+0000","last_change":"2023-10-08T06:54:33.040076+0000","last_active":"2023-10-08T06:54:35.753733+0000","last_peered":"2023-10-08T06:54:35.753733+0000","last_clean":"2023-10-08T06:54:35.753733+0000","last_became_active":"2023-10-08T06:54:33.039751+0000","last_became_peered":"2023-10-08T06:54:33.039751+0000","last_unstale":"2023-10-08T06:54:35.753733+0000","last_undegraded":"2023-10-08T06:54:35.753733+0000","last_fullsized":"2023-10-08T06:54:35.753733+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:30.676194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751306+0000","last_change":"2023-10-08T06:54:33.074559+0000","last_active":"2023-10-08T06:54:35.751306+0000","last_peered":"2023-10-08T06:54:35.751306+0000","last_clean":"2023-10-08T06:54:35.751306+0000","last_became_active":"2023-10-08T06:54:33.074270+0000","last_became_peered":"2023-10-08T06:54:33.074270+0000","last_unstale":"2023-10-08T06:54:35.751306+0000","last_undegraded":"2023-10-08T06:54:35.751306+0000","last_fullsized":"2023-10-08T06:54:35.751306+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:32.229059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752245+0000","last_change":"2023-10-08T06:54:33.066200+0000","last_active":"2023-10-08T06:54:35.752245+0000","last_peered":"2023-10-08T06:54:35.752245+0000","last_clean":"2023-10-08T06:54:35.752245+0000","last_became_active":"2023-10-08T06:54:33.065790+0000","last_became_peered":"2023-10-08T06:54:33.065790+0000","last_unstale":"2023-10-08T06:54:35.752245+0000","last_undegraded":"2023-10-08T06:54:35.752245+0000","last_fullsized":"2023-10-08T06:54:35.752245+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:57.940917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153750+0000","last_change":"2023-10-08T06:54:33.056727+0000","last_active":"2023-10-08T06:54:34.153750+0000","last_peered":"2023-10-08T06:54:34.153750+0000","last_clean":"2023-10-08T06:54:34.153750+0000","last_became_active":"2023-10-08T06:54:33.056472+0000","last_became_peered":"2023-10-08T06:54:33.056472+0000","last_unstale":"2023-10-08T06:54:34.153750+0000","last_undegraded":"2023-10-08T06:54:34.153750+0000","last_fullsized":"2023-10-08T06:54:34.153750+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:03:29.890214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752009+0000","last_change":"2023-10-08T06:54:33.058338+0000","last_active":"2023-10-08T06:54:35.752009+0000","last_peered":"2023-10-08T06:54:35.752009+0000","last_clean":"2023-10-08T06:54:35.752009+0000","last_became_active":"2023-10-08T06:54:33.057427+0000","last_became_peered":"2023-10-08T06:54:33.057427+0000","last_unstale":"2023-10-08T06:54:35.752009+0000","last_undegraded":"2023-10-08T06:54:35.752009+0000","last_fullsized":"2023-10-08T06:54:35.752009+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:47.092594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026384+0000","last_change":"2023-10-08T06:54:33.063218+0000","last_active":"2023-10-08T06:54:34.026384+0000","last_peered":"2023-10-08T06:54:34.026384+0000","last_clean":"2023-10-08T06:54:34.026384+0000","last_became_active":"2023-10-08T06:54:33.062736+0000","last_became_peered":"2023-10-08T06:54:33.062736+0000","last_unstale":"2023-10-08T06:54:34.026384+0000","last_undegraded":"2023-10-08T06:54:34.026384+0000","last_fullsized":"2023-10-08T06:54:34.026384+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:45.520385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026317+0000","last_change":"2023-10-08T06:54:33.057671+0000","last_active":"2023-10-08T06:54:34.026317+0000","last_peered":"2023-10-08T06:54:34.026317+0000","last_clean":"2023-10-08T06:54:34.026317+0000","last_became_active":"2023-10-08T06:54:33.057360+0000","last_became_peered":"2023-10-08T06:54:33.057360+0000","last_unstale":"2023-10-08T06:54:34.026317+0000","last_undegraded":"2023-10-08T06:54:34.026317+0000","last_fullsized":"2023-10-08T06:54:34.026317+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:45.108097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027340+0000","last_change":"2023-10-08T06:54:33.058685+0000","last_active":"2023-10-08T06:54:34.027340+0000","last_peered":"2023-10-08T06:54:34.027340+0000","last_clean":"2023-10-08T06:54:34.027340+0000","last_became_active":"2023-10-08T06:54:33.058390+0000","last_became_peered":"2023-10-08T06:54:33.058390+0000","last_unstale":"2023-10-08T06:54:34.027340+0000","last_undegraded":"2023-10-08T06:54:34.027340+0000","last_fullsized":"2023-10-08T06:54:34.027340+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:37.433398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027328+0000","last_change":"2023-10-08T06:54:33.061146+0000","last_active":"2023-10-08T06:54:34.027328+0000","last_peered":"2023-10-08T06:54:34.027328+0000","last_clean":"2023-10-08T06:54:34.027328+0000","last_became_active":"2023-10-08T06:54:33.060860+0000","last_became_peered":"2023-10-08T06:54:33.060860+0000","last_unstale":"2023-10-08T06:54:34.027328+0000","last_undegraded":"2023-10-08T06:54:34.027328+0000","last_fullsized":"2023-10-08T06:54:34.027328+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:05.111072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754508+0000","last_change":"2023-10-08T06:54:33.047535+0000","last_active":"2023-10-08T06:54:35.754508+0000","last_peered":"2023-10-08T06:54:35.754508+0000","last_clean":"2023-10-08T06:54:35.754508+0000","last_became_active":"2023-10-08T06:54:33.047244+0000","last_became_peered":"2023-10-08T06:54:33.047244+0000","last_unstale":"2023-10-08T06:54:35.754508+0000","last_undegraded":"2023-10-08T06:54:35.754508+0000","last_fullsized":"2023-10-08T06:54:35.754508+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:41.081711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752599+0000","last_change":"2023-10-08T06:54:33.061136+0000","last_active":"2023-10-08T06:54:35.752599+0000","last_peered":"2023-10-08T06:54:35.752599+0000","last_clean":"2023-10-08T06:54:35.752599+0000","last_became_active":"2023-10-08T06:54:33.060845+0000","last_became_peered":"2023-10-08T06:54:33.060845+0000","last_unstale":"2023-10-08T06:54:35.752599+0000","last_undegraded":"2023-10-08T06:54:35.752599+0000","last_fullsized":"2023-10-08T06:54:35.752599+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:31:23.962246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071150+0000","last_change":"2023-10-08T06:54:33.038054+0000","last_active":"2023-10-08T06:54:34.071150+0000","last_peered":"2023-10-08T06:54:34.071150+0000","last_clean":"2023-10-08T06:54:34.071150+0000","last_became_active":"2023-10-08T06:54:33.037251+0000","last_became_peered":"2023-10-08T06:54:33.037251+0000","last_unstale":"2023-10-08T06:54:34.071150+0000","last_undegraded":"2023-10-08T06:54:34.071150+0000","last_fullsized":"2023-10-08T06:54:34.071150+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:59:04.592809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.152935+0000","last_change":"2023-10-08T06:54:33.068411+0000","last_active":"2023-10-08T06:54:34.152935+0000","last_peered":"2023-10-08T06:54:34.152935+0000","last_clean":"2023-10-08T06:54:34.152935+0000","last_became_active":"2023-10-08T06:54:33.068144+0000","last_became_peered":"2023-10-08T06:54:33.068144+0000","last_unstale":"2023-10-08T06:54:34.152935+0000","last_undegraded":"2023-10-08T06:54:34.152935+0000","last_fullsized":"2023-10-08T06:54:34.152935+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:24:47.594616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751468+0000","last_change":"2023-10-08T06:54:33.049383+0000","last_active":"2023-10-08T06:54:35.751468+0000","last_peered":"2023-10-08T06:54:35.751468+0000","last_clean":"2023-10-08T06:54:35.751468+0000","last_became_active":"2023-10-08T06:54:33.049095+0000","last_became_peered":"2023-10-08T06:54:33.049095+0000","last_unstale":"2023-10-08T06:54:35.751468+0000","last_undegraded":"2023-10-08T06:54:35.751468+0000","last_fullsized":"2023-10-08T06:54:35.751468+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:52:27.346652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753627+0000","last_change":"2023-10-08T06:54:33.064938+0000","last_active":"2023-10-08T06:54:35.753627+0000","last_peered":"2023-10-08T06:54:35.753627+0000","last_clean":"2023-10-08T06:54:35.753627+0000","last_became_active":"2023-10-08T06:54:33.064580+0000","last_became_peered":"2023-10-08T06:54:33.064580+0000","last_unstale":"2023-10-08T06:54:35.753627+0000","last_undegraded":"2023-10-08T06:54:35.753627+0000","last_fullsized":"2023-10-08T06:54:35.753627+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:46:31.329546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027518+0000","last_change":"2023-10-08T06:54:33.050305+0000","last_active":"2023-10-08T06:54:34.027518+0000","last_peered":"2023-10-08T06:54:34.027518+0000","last_clean":"2023-10-08T06:54:34.027518+0000","last_became_active":"2023-10-08T06:54:33.049829+0000","last_became_peered":"2023-10-08T06:54:33.049829+0000","last_unstale":"2023-10-08T06:54:34.027518+0000","last_undegraded":"2023-10-08T06:54:34.027518+0000","last_fullsized":"2023-10-08T06:54:34.027518+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:06.669276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026431+0000","last_change":"2023-10-08T06:54:33.049856+0000","last_active":"2023-10-08T06:54:34.026431+0000","last_peered":"2023-10-08T06:54:34.026431+0000","last_clean":"2023-10-08T06:54:34.026431+0000","last_became_active":"2023-10-08T06:54:33.049550+0000","last_became_peered":"2023-10-08T06:54:33.049550+0000","last_unstale":"2023-10-08T06:54:34.026431+0000","last_undegraded":"2023-10-08T06:54:34.026431+0000","last_fullsized":"2023-10-08T06:54:34.026431+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:56.201798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754501+0000","last_change":"2023-10-08T06:54:33.050078+0000","last_active":"2023-10-08T06:54:35.754501+0000","last_peered":"2023-10-08T06:54:35.754501+0000","last_clean":"2023-10-08T06:54:35.754501+0000","last_became_active":"2023-10-08T06:54:33.049798+0000","last_became_peered":"2023-10-08T06:54:33.049798+0000","last_unstale":"2023-10-08T06:54:35.754501+0000","last_undegraded":"2023-10-08T06:54:35.754501+0000","last_fullsized":"2023-10-08T06:54:35.754501+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:50:25.595884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024143+0000","last_change":"2023-10-08T06:54:33.065142+0000","last_active":"2023-10-08T06:54:34.024143+0000","last_peered":"2023-10-08T06:54:34.024143+0000","last_clean":"2023-10-08T06:54:34.024143+0000","last_became_active":"2023-10-08T06:54:33.064871+0000","last_became_peered":"2023-10-08T06:54:33.064871+0000","last_unstale":"2023-10-08T06:54:34.024143+0000","last_undegraded":"2023-10-08T06:54:34.024143+0000","last_fullsized":"2023-10-08T06:54:34.024143+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:58.069314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071296+0000","last_change":"2023-10-08T06:54:33.059200+0000","last_active":"2023-10-08T06:54:34.071296+0000","last_peered":"2023-10-08T06:54:34.071296+0000","last_clean":"2023-10-08T06:54:34.071296+0000","last_became_active":"2023-10-08T06:54:33.058922+0000","last_became_peered":"2023-10-08T06:54:33.058922+0000","last_unstale":"2023-10-08T06:54:34.071296+0000","last_undegraded":"2023-10-08T06:54:34.071296+0000","last_fullsized":"2023-10-08T06:54:34.071296+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:54.253876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752961+0000","last_change":"2023-10-08T06:54:33.054103+0000","last_active":"2023-10-08T06:54:35.752961+0000","last_peered":"2023-10-08T06:54:35.752961+0000","last_clean":"2023-10-08T06:54:35.752961+0000","last_became_active":"2023-10-08T06:54:33.053376+0000","last_became_peered":"2023-10-08T06:54:33.053376+0000","last_unstale":"2023-10-08T06:54:35.752961+0000","last_undegraded":"2023-10-08T06:54:35.752961+0000","last_fullsized":"2023-10-08T06:54:35.752961+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:44:42.544090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153251+0000","last_change":"2023-10-08T06:54:33.049115+0000","last_active":"2023-10-08T06:54:34.153251+0000","last_peered":"2023-10-08T06:54:34.153251+0000","last_clean":"2023-10-08T06:54:34.153251+0000","last_became_active":"2023-10-08T06:54:33.048807+0000","last_became_peered":"2023-10-08T06:54:33.048807+0000","last_unstale":"2023-10-08T06:54:34.153251+0000","last_undegraded":"2023-10-08T06:54:34.153251+0000","last_fullsized":"2023-10-08T06:54:34.153251+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:44:38.005432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.750982+0000","last_change":"2023-10-08T06:54:33.064964+0000","last_active":"2023-10-08T06:54:35.750982+0000","last_peered":"2023-10-08T06:54:35.750982+0000","last_clean":"2023-10-08T06:54:35.750982+0000","last_became_active":"2023-10-08T06:54:33.064576+0000","last_became_peered":"2023-10-08T06:54:33.064576+0000","last_unstale":"2023-10-08T06:54:35.750982+0000","last_undegraded":"2023-10-08T06:54:35.750982+0000","last_fullsized":"2023-10-08T06:54:35.750982+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:00.870715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071763+0000","last_change":"2023-10-08T06:54:33.042671+0000","last_active":"2023-10-08T06:54:34.071763+0000","last_peered":"2023-10-08T06:54:34.071763+0000","last_clean":"2023-10-08T06:54:34.071763+0000","last_became_active":"2023-10-08T06:54:33.042097+0000","last_became_peered":"2023-10-08T06:54:33.042097+0000","last_unstale":"2023-10-08T06:54:34.071763+0000","last_undegraded":"2023-10-08T06:54:34.071763+0000","last_fullsized":"2023-10-08T06:54:34.071763+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:43.301465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024383+0000","last_change":"2023-10-08T06:54:33.063267+0000","last_active":"2023-10-08T06:54:34.024383+0000","last_peered":"2023-10-08T06:54:34.024383+0000","last_clean":"2023-10-08T06:54:34.024383+0000","last_became_active":"2023-10-08T06:54:33.062984+0000","last_became_peered":"2023-10-08T06:54:33.062984+0000","last_unstale":"2023-10-08T06:54:34.024383+0000","last_undegraded":"2023-10-08T06:54:34.024383+0000","last_fullsized":"2023-10-08T06:54:34.024383+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:43.408053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752271+0000","last_change":"2023-10-08T06:54:33.057947+0000","last_active":"2023-10-08T06:54:35.752271+0000","last_peered":"2023-10-08T06:54:35.752271+0000","last_clean":"2023-10-08T06:54:35.752271+0000","last_became_active":"2023-10-08T06:54:33.057585+0000","last_became_peered":"2023-10-08T06:54:33.057585+0000","last_unstale":"2023-10-08T06:54:35.752271+0000","last_undegraded":"2023-10-08T06:54:35.752271+0000","last_fullsized":"2023-10-08T06:54:35.752271+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:29.346638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.070932+0000","last_change":"2023-10-08T06:54:33.064936+0000","last_active":"2023-10-08T06:54:34.070932+0000","last_peered":"2023-10-08T06:54:34.070932+0000","last_clean":"2023-10-08T06:54:34.070932+0000","last_became_active":"2023-10-08T06:54:33.064634+0000","last_became_peered":"2023-10-08T06:54:33.064634+0000","last_unstale":"2023-10-08T06:54:34.070932+0000","last_undegraded":"2023-10-08T06:54:34.070932+0000","last_fullsized":"2023-10-08T06:54:34.070932+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:24:26.246925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752489+0000","last_change":"2023-10-08T06:54:33.049254+0000","last_active":"2023-10-08T06:54:35.752489+0000","last_peered":"2023-10-08T06:54:35.752489+0000","last_clean":"2023-10-08T06:54:35.752489+0000","last_became_active":"2023-10-08T06:54:33.048872+0000","last_became_peered":"2023-10-08T06:54:33.048872+0000","last_unstale":"2023-10-08T06:54:35.752489+0000","last_undegraded":"2023-10-08T06:54:35.752489+0000","last_fullsized":"2023-10-08T06:54:35.752489+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:44.988238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026737+0000","last_change":"2023-10-08T06:54:33.055510+0000","last_active":"2023-10-08T06:54:34.026737+0000","last_peered":"2023-10-08T06:54:34.026737+0000","last_clean":"2023-10-08T06:54:34.026737+0000","last_became_active":"2023-10-08T06:54:33.055175+0000","last_became_peered":"2023-10-08T06:54:33.055175+0000","last_unstale":"2023-10-08T06:54:34.026737+0000","last_undegraded":"2023-10-08T06:54:34.026737+0000","last_fullsized":"2023-10-08T06:54:34.026737+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:09:41.518993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754194+0000","last_change":"2023-10-08T06:54:33.044545+0000","last_active":"2023-10-08T06:54:35.754194+0000","last_peered":"2023-10-08T06:54:35.754194+0000","last_clean":"2023-10-08T06:54:35.754194+0000","last_became_active":"2023-10-08T06:54:33.044191+0000","last_became_peered":"2023-10-08T06:54:33.044191+0000","last_unstale":"2023-10-08T06:54:35.754194+0000","last_undegraded":"2023-10-08T06:54:35.754194+0000","last_fullsized":"2023-10-08T06:54:35.754194+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:20:37.430101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026187+0000","last_change":"2023-10-08T06:54:33.046114+0000","last_active":"2023-10-08T06:54:34.026187+0000","last_peered":"2023-10-08T06:54:34.026187+0000","last_clean":"2023-10-08T06:54:34.026187+0000","last_became_active":"2023-10-08T06:54:33.045771+0000","last_became_peered":"2023-10-08T06:54:33.045771+0000","last_unstale":"2023-10-08T06:54:34.026187+0000","last_undegraded":"2023-10-08T06:54:34.026187+0000","last_fullsized":"2023-10-08T06:54:34.026187+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:15:15.571393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753366+0000","last_change":"2023-10-08T06:54:33.043116+0000","last_active":"2023-10-08T06:54:35.753366+0000","last_peered":"2023-10-08T06:54:35.753366+0000","last_clean":"2023-10-08T06:54:35.753366+0000","last_became_active":"2023-10-08T06:54:33.042813+0000","last_became_peered":"2023-10-08T06:54:33.042813+0000","last_unstale":"2023-10-08T06:54:35.753366+0000","last_undegraded":"2023-10-08T06:54:35.753366+0000","last_fullsized":"2023-10-08T06:54:35.753366+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:38.937297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"197'4","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754127+0000","last_change":"2023-10-08T06:54:32.043648+0000","last_active":"2023-10-08T06:54:35.754127+0000","last_peered":"2023-10-08T06:54:35.754127+0000","last_clean":"2023-10-08T06:54:35.754127+0000","last_became_active":"2023-10-08T06:54:32.043312+0000","last_became_peered":"2023-10-08T06:54:32.043312+0000","last_unstale":"2023-10-08T06:54:35.754127+0000","last_undegraded":"2023-10-08T06:54:35.754127+0000","last_fullsized":"2023-10-08T06:54:35.754127+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:14.119264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.070557+0000","last_change":"2023-10-08T06:54:33.050590+0000","last_active":"2023-10-08T06:54:34.070557+0000","last_peered":"2023-10-08T06:54:34.070557+0000","last_clean":"2023-10-08T06:54:34.070557+0000","last_became_active":"2023-10-08T06:54:33.050291+0000","last_became_peered":"2023-10-08T06:54:33.050291+0000","last_unstale":"2023-10-08T06:54:34.070557+0000","last_undegraded":"2023-10-08T06:54:34.070557+0000","last_fullsized":"2023-10-08T06:54:34.070557+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:45.208279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"197'1","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153117+0000","last_change":"2023-10-08T06:54:32.050135+0000","last_active":"2023-10-08T06:54:34.153117+0000","last_peered":"2023-10-08T06:54:34.153117+0000","last_clean":"2023-10-08T06:54:34.153117+0000","last_became_active":"2023-10-08T06:54:32.049865+0000","last_became_peered":"2023-10-08T06:54:32.049865+0000","last_unstale":"2023-10-08T06:54:34.153117+0000","last_undegraded":"2023-10-08T06:54:34.153117+0000","last_fullsized":"2023-10-08T06:54:34.153117+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:18.654684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153278+0000","last_change":"2023-10-08T06:54:33.060636+0000","last_active":"2023-10-08T06:54:34.153278+0000","last_peered":"2023-10-08T06:54:34.153278+0000","last_clean":"2023-10-08T06:54:34.153278+0000","last_became_active":"2023-10-08T06:54:33.060354+0000","last_became_peered":"2023-10-08T06:54:33.060354+0000","last_unstale":"2023-10-08T06:54:34.153278+0000","last_undegraded":"2023-10-08T06:54:34.153278+0000","last_fullsized":"2023-10-08T06:54:34.153278+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:08.311019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754776+0000","last_change":"2023-10-08T06:54:32.038290+0000","last_active":"2023-10-08T06:54:35.754776+0000","last_peered":"2023-10-08T06:54:35.754776+0000","last_clean":"2023-10-08T06:54:35.754776+0000","last_became_active":"2023-10-08T06:54:32.037726+0000","last_became_peered":"2023-10-08T06:54:32.037726+0000","last_unstale":"2023-10-08T06:54:35.754776+0000","last_undegraded":"2023-10-08T06:54:35.754776+0000","last_fullsized":"2023-10-08T06:54:35.754776+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:51.656021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752829+0000","last_change":"2023-10-08T06:54:33.047365+0000","last_active":"2023-10-08T06:54:35.752829+0000","last_peered":"2023-10-08T06:54:35.752829+0000","last_clean":"2023-10-08T06:54:35.752829+0000","last_became_active":"2023-10-08T06:54:33.046439+0000","last_became_peered":"2023-10-08T06:54:33.046439+0000","last_unstale":"2023-10-08T06:54:35.752829+0000","last_undegraded":"2023-10-08T06:54:35.752829+0000","last_fullsized":"2023-10-08T06:54:35.752829+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:50.161307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024205+0000","last_change":"2023-10-08T06:54:32.045832+0000","last_active":"2023-10-08T06:54:34.024205+0000","last_peered":"2023-10-08T06:54:34.024205+0000","last_clean":"2023-10-08T06:54:34.024205+0000","last_became_active":"2023-10-08T06:54:32.045554+0000","last_became_peered":"2023-10-08T06:54:32.045554+0000","last_unstale":"2023-10-08T06:54:34.024205+0000","last_undegraded":"2023-10-08T06:54:34.024205+0000","last_fullsized":"2023-10-08T06:54:34.024205+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:04.367253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026500+0000","last_change":"2023-10-08T06:54:33.061146+0000","last_active":"2023-10-08T06:54:34.026500+0000","last_peered":"2023-10-08T06:54:34.026500+0000","last_clean":"2023-10-08T06:54:34.026500+0000","last_became_active":"2023-10-08T06:54:33.060872+0000","last_became_peered":"2023-10-08T06:54:33.060872+0000","last_unstale":"2023-10-08T06:54:34.026500+0000","last_undegraded":"2023-10-08T06:54:34.026500+0000","last_fullsized":"2023-10-08T06:54:34.026500+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:55.630605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"197'1","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026820+0000","last_change":"2023-10-08T06:54:32.045657+0000","last_active":"2023-10-08T06:54:34.026820+0000","last_peered":"2023-10-08T06:54:34.026820+0000","last_clean":"2023-10-08T06:54:34.026820+0000","last_became_active":"2023-10-08T06:54:32.045357+0000","last_became_peered":"2023-10-08T06:54:32.045357+0000","last_unstale":"2023-10-08T06:54:34.026820+0000","last_undegraded":"2023-10-08T06:54:34.026820+0000","last_fullsized":"2023-10-08T06:54:34.026820+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:22.863407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.023942+0000","last_change":"2023-10-08T06:54:33.050456+0000","last_active":"2023-10-08T06:54:34.023942+0000","last_peered":"2023-10-08T06:54:34.023942+0000","last_clean":"2023-10-08T06:54:34.023942+0000","last_became_active":"2023-10-08T06:54:33.050181+0000","last_became_peered":"2023-10-08T06:54:33.050181+0000","last_unstale":"2023-10-08T06:54:34.023942+0000","last_undegraded":"2023-10-08T06:54:34.023942+0000","last_fullsized":"2023-10-08T06:54:34.023942+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:51.336332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"197'1","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153337+0000","last_change":"2023-10-08T06:54:32.037541+0000","last_active":"2023-10-08T06:54:34.153337+0000","last_peered":"2023-10-08T06:54:34.153337+0000","last_clean":"2023-10-08T06:54:34.153337+0000","last_became_active":"2023-10-08T06:54:32.037095+0000","last_became_peered":"2023-10-08T06:54:32.037095+0000","last_unstale":"2023-10-08T06:54:34.153337+0000","last_undegraded":"2023-10-08T06:54:34.153337+0000","last_fullsized":"2023-10-08T06:54:34.153337+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:46.276079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024406+0000","last_change":"2023-10-08T06:54:33.046496+0000","last_active":"2023-10-08T06:54:34.024406+0000","last_peered":"2023-10-08T06:54:34.024406+0000","last_clean":"2023-10-08T06:54:34.024406+0000","last_became_active":"2023-10-08T06:54:33.046207+0000","last_became_peered":"2023-10-08T06:54:33.046207+0000","last_unstale":"2023-10-08T06:54:34.024406+0000","last_undegraded":"2023-10-08T06:54:34.024406+0000","last_fullsized":"2023-10-08T06:54:34.024406+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:35:53.646222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.023588+0000","last_change":"2023-10-08T06:54:32.041108+0000","last_active":"2023-10-08T06:54:34.023588+0000","last_peered":"2023-10-08T06:54:34.023588+0000","last_clean":"2023-10-08T06:54:34.023588+0000","last_became_active":"2023-10-08T06:54:32.040801+0000","last_became_peered":"2023-10-08T06:54:32.040801+0000","last_unstale":"2023-10-08T06:54:34.023588+0000","last_undegraded":"2023-10-08T06:54:34.023588+0000","last_fullsized":"2023-10-08T06:54:34.023588+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:57:17.680282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753961+0000","last_change":"2023-10-08T06:54:33.059025+0000","last_active":"2023-10-08T06:54:35.753961+0000","last_peered":"2023-10-08T06:54:35.753961+0000","last_clean":"2023-10-08T06:54:35.753961+0000","last_became_active":"2023-10-08T06:54:33.058737+0000","last_became_peered":"2023-10-08T06:54:33.058737+0000","last_unstale":"2023-10-08T06:54:35.753961+0000","last_undegraded":"2023-10-08T06:54:35.753961+0000","last_fullsized":"2023-10-08T06:54:35.753961+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:27.339921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026580+0000","last_change":"2023-10-08T06:54:32.048819+0000","last_active":"2023-10-08T06:54:34.026580+0000","last_peered":"2023-10-08T06:54:34.026580+0000","last_clean":"2023-10-08T06:54:34.026580+0000","last_became_active":"2023-10-08T06:54:32.048555+0000","last_became_peered":"2023-10-08T06:54:32.048555+0000","last_unstale":"2023-10-08T06:54:34.026580+0000","last_undegraded":"2023-10-08T06:54:34.026580+0000","last_fullsized":"2023-10-08T06:54:34.026580+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:20:30.638813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751338+0000","last_change":"2023-10-08T06:54:33.048206+0000","last_active":"2023-10-08T06:54:35.751338+0000","last_peered":"2023-10-08T06:54:35.751338+0000","last_clean":"2023-10-08T06:54:35.751338+0000","last_became_active":"2023-10-08T06:54:33.047919+0000","last_became_peered":"2023-10-08T06:54:33.047919+0000","last_unstale":"2023-10-08T06:54:35.751338+0000","last_undegraded":"2023-10-08T06:54:35.751338+0000","last_fullsized":"2023-10-08T06:54:35.751338+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:50:53.030168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071451+0000","last_change":"2023-10-08T06:54:32.051894+0000","last_active":"2023-10-08T06:54:34.071451+0000","last_peered":"2023-10-08T06:54:34.071451+0000","last_clean":"2023-10-08T06:54:34.071451+0000","last_became_active":"2023-10-08T06:54:32.051638+0000","last_became_peered":"2023-10-08T06:54:32.051638+0000","last_unstale":"2023-10-08T06:54:34.071451+0000","last_undegraded":"2023-10-08T06:54:34.071451+0000","last_fullsized":"2023-10-08T06:54:34.071451+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:50:52.014244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.070879+0000","last_change":"2023-10-08T06:54:33.057016+0000","last_active":"2023-10-08T06:54:34.070879+0000","last_peered":"2023-10-08T06:54:34.070879+0000","last_clean":"2023-10-08T06:54:34.070879+0000","last_became_active":"2023-10-08T06:54:33.056741+0000","last_became_peered":"2023-10-08T06:54:33.056741+0000","last_unstale":"2023-10-08T06:54:34.070879+0000","last_undegraded":"2023-10-08T06:54:34.070879+0000","last_fullsized":"2023-10-08T06:54:34.070879+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:09.299982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"197'3","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751822+0000","last_change":"2023-10-08T06:54:32.046984+0000","last_active":"2023-10-08T06:54:35.751822+0000","last_peered":"2023-10-08T06:54:35.751822+0000","last_clean":"2023-10-08T06:54:35.751822+0000","last_became_active":"2023-10-08T06:54:32.046651+0000","last_became_peered":"2023-10-08T06:54:32.046651+0000","last_unstale":"2023-10-08T06:54:35.751822+0000","last_undegraded":"2023-10-08T06:54:35.751822+0000","last_fullsized":"2023-10-08T06:54:35.751822+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:07:14.032996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024165+0000","last_change":"2023-10-08T06:54:33.064396+0000","last_active":"2023-10-08T06:54:34.024165+0000","last_peered":"2023-10-08T06:54:34.024165+0000","last_clean":"2023-10-08T06:54:34.024165+0000","last_became_active":"2023-10-08T06:54:33.064103+0000","last_became_peered":"2023-10-08T06:54:33.064103+0000","last_unstale":"2023-10-08T06:54:34.024165+0000","last_undegraded":"2023-10-08T06:54:34.024165+0000","last_fullsized":"2023-10-08T06:54:34.024165+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:09.043727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"197'6","reported_seq":18,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071186+0000","last_change":"2023-10-08T06:54:32.047958+0000","last_active":"2023-10-08T06:54:34.071186+0000","last_peered":"2023-10-08T06:54:34.071186+0000","last_clean":"2023-10-08T06:54:34.071186+0000","last_became_active":"2023-10-08T06:54:32.047693+0000","last_became_peered":"2023-10-08T06:54:32.047693+0000","last_unstale":"2023-10-08T06:54:34.071186+0000","last_undegraded":"2023-10-08T06:54:34.071186+0000","last_fullsized":"2023-10-08T06:54:34.071186+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:44:41.528166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026185+0000","last_change":"2023-10-08T06:54:33.044331+0000","last_active":"2023-10-08T06:54:34.026185+0000","last_peered":"2023-10-08T06:54:34.026185+0000","last_clean":"2023-10-08T06:54:34.026185+0000","last_became_active":"2023-10-08T06:54:33.044023+0000","last_became_peered":"2023-10-08T06:54:33.044023+0000","last_unstale":"2023-10-08T06:54:34.026185+0000","last_undegraded":"2023-10-08T06:54:34.026185+0000","last_fullsized":"2023-10-08T06:54:34.026185+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:52:01.936324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153019+0000","last_change":"2023-10-08T06:54:32.034716+0000","last_active":"2023-10-08T06:54:34.153019+0000","last_peered":"2023-10-08T06:54:34.153019+0000","last_clean":"2023-10-08T06:54:34.153019+0000","last_became_active":"2023-10-08T06:54:32.034376+0000","last_became_peered":"2023-10-08T06:54:32.034376+0000","last_unstale":"2023-10-08T06:54:34.153019+0000","last_undegraded":"2023-10-08T06:54:34.153019+0000","last_fullsized":"2023-10-08T06:54:34.153019+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:36.417473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027075+0000","last_change":"2023-10-08T06:54:33.056311+0000","last_active":"2023-10-08T06:54:34.027075+0000","last_peered":"2023-10-08T06:54:34.027075+0000","last_clean":"2023-10-08T06:54:34.027075+0000","last_became_active":"2023-10-08T06:54:33.056039+0000","last_became_peered":"2023-10-08T06:54:33.056039+0000","last_unstale":"2023-10-08T06:54:34.027075+0000","last_undegraded":"2023-10-08T06:54:34.027075+0000","last_fullsized":"2023-10-08T06:54:34.027075+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:41.769789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.070781+0000","last_change":"2023-10-08T06:54:32.035824+0000","last_active":"2023-10-08T06:54:34.070781+0000","last_peered":"2023-10-08T06:54:34.070781+0000","last_clean":"2023-10-08T06:54:34.070781+0000","last_became_active":"2023-10-08T06:54:32.035561+0000","last_became_peered":"2023-10-08T06:54:32.035561+0000","last_unstale":"2023-10-08T06:54:34.070781+0000","last_undegraded":"2023-10-08T06:54:34.070781+0000","last_fullsized":"2023-10-08T06:54:34.070781+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:44:53.075538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754624+0000","last_change":"2023-10-08T06:54:33.037252+0000","last_active":"2023-10-08T06:54:35.754624+0000","last_peered":"2023-10-08T06:54:35.754624+0000","last_clean":"2023-10-08T06:54:35.754624+0000","last_became_active":"2023-10-08T06:54:33.036860+0000","last_became_peered":"2023-10-08T06:54:33.036860+0000","last_unstale":"2023-10-08T06:54:35.754624+0000","last_undegraded":"2023-10-08T06:54:35.754624+0000","last_fullsized":"2023-10-08T06:54:35.754624+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:50:53.030168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754036+0000","last_change":"2023-10-08T06:54:32.032192+0000","last_active":"2023-10-08T06:54:35.754036+0000","last_peered":"2023-10-08T06:54:35.754036+0000","last_clean":"2023-10-08T06:54:35.754036+0000","last_became_active":"2023-10-08T06:54:32.030540+0000","last_became_peered":"2023-10-08T06:54:32.030540+0000","last_unstale":"2023-10-08T06:54:35.754036+0000","last_undegraded":"2023-10-08T06:54:35.754036+0000","last_fullsized":"2023-10-08T06:54:35.754036+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:35:20.639692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024796+0000","last_change":"2023-10-08T06:54:33.063188+0000","last_active":"2023-10-08T06:54:34.024796+0000","last_peered":"2023-10-08T06:54:34.024796+0000","last_clean":"2023-10-08T06:54:34.024796+0000","last_became_active":"2023-10-08T06:54:33.062900+0000","last_became_peered":"2023-10-08T06:54:33.062900+0000","last_unstale":"2023-10-08T06:54:34.024796+0000","last_undegraded":"2023-10-08T06:54:34.024796+0000","last_fullsized":"2023-10-08T06:54:34.024796+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:32.134362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"197'8","reported_seq":20,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026824+0000","last_change":"2023-10-08T06:54:32.048261+0000","last_active":"2023-10-08T06:54:34.026824+0000","last_peered":"2023-10-08T06:54:34.026824+0000","last_clean":"2023-10-08T06:54:34.026824+0000","last_became_active":"2023-10-08T06:54:32.047999+0000","last_became_peered":"2023-10-08T06:54:32.047999+0000","last_unstale":"2023-10-08T06:54:34.026824+0000","last_undegraded":"2023-10-08T06:54:34.026824+0000","last_fullsized":"2023-10-08T06:54:34.026824+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:09:51.338067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027103+0000","last_change":"2023-10-08T06:54:33.068754+0000","last_active":"2023-10-08T06:54:34.027103+0000","last_peered":"2023-10-08T06:54:34.027103+0000","last_clean":"2023-10-08T06:54:34.027103+0000","last_became_active":"2023-10-08T06:54:33.068392+0000","last_became_peered":"2023-10-08T06:54:33.068392+0000","last_unstale":"2023-10-08T06:54:34.027103+0000","last_undegraded":"2023-10-08T06:54:34.027103+0000","last_fullsized":"2023-10-08T06:54:34.027103+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:03.010878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"197'1","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071695+0000","last_change":"2023-10-08T06:54:32.043283+0000","last_active":"2023-10-08T06:54:34.071695+0000","last_peered":"2023-10-08T06:54:34.071695+0000","last_clean":"2023-10-08T06:54:34.071695+0000","last_became_active":"2023-10-08T06:54:32.042956+0000","last_became_peered":"2023-10-08T06:54:32.042956+0000","last_unstale":"2023-10-08T06:54:34.071695+0000","last_undegraded":"2023-10-08T06:54:34.071695+0000","last_fullsized":"2023-10-08T06:54:34.071695+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:37.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026831+0000","last_change":"2023-10-08T06:54:33.042983+0000","last_active":"2023-10-08T06:54:34.026831+0000","last_peered":"2023-10-08T06:54:34.026831+0000","last_clean":"2023-10-08T06:54:34.026831+0000","last_became_active":"2023-10-08T06:54:33.042684+0000","last_became_peered":"2023-10-08T06:54:33.042684+0000","last_unstale":"2023-10-08T06:54:34.026831+0000","last_undegraded":"2023-10-08T06:54:34.026831+0000","last_fullsized":"2023-10-08T06:54:34.026831+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:00:34.913155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.027517+0000","last_change":"2023-10-08T06:54:32.050029+0000","last_active":"2023-10-08T06:54:34.027517+0000","last_peered":"2023-10-08T06:54:34.027517+0000","last_clean":"2023-10-08T06:54:34.027517+0000","last_became_active":"2023-10-08T06:54:32.049367+0000","last_became_peered":"2023-10-08T06:54:32.049367+0000","last_unstale":"2023-10-08T06:54:34.027517+0000","last_undegraded":"2023-10-08T06:54:34.027517+0000","last_fullsized":"2023-10-08T06:54:34.027517+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:37.572215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753578+0000","last_change":"2023-10-08T06:54:33.058492+0000","last_active":"2023-10-08T06:54:35.753578+0000","last_peered":"2023-10-08T06:54:35.753578+0000","last_clean":"2023-10-08T06:54:35.753578+0000","last_became_active":"2023-10-08T06:54:33.058132+0000","last_became_peered":"2023-10-08T06:54:33.058132+0000","last_unstale":"2023-10-08T06:54:35.753578+0000","last_undegraded":"2023-10-08T06:54:35.753578+0000","last_fullsized":"2023-10-08T06:54:35.753578+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:44:54.091462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"197'1","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153435+0000","last_change":"2023-10-08T06:54:32.043662+0000","last_active":"2023-10-08T06:54:34.153435+0000","last_peered":"2023-10-08T06:54:34.153435+0000","last_clean":"2023-10-08T06:54:34.153435+0000","last_became_active":"2023-10-08T06:54:32.043387+0000","last_became_peered":"2023-10-08T06:54:32.043387+0000","last_unstale":"2023-10-08T06:54:34.153435+0000","last_undegraded":"2023-10-08T06:54:34.153435+0000","last_fullsized":"2023-10-08T06:54:34.153435+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:34:04.095148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753688+0000","last_change":"2023-10-08T06:54:33.048206+0000","last_active":"2023-10-08T06:54:35.753688+0000","last_peered":"2023-10-08T06:54:35.753688+0000","last_clean":"2023-10-08T06:54:35.753688+0000","last_became_active":"2023-10-08T06:54:33.047923+0000","last_became_peered":"2023-10-08T06:54:33.047923+0000","last_unstale":"2023-10-08T06:54:35.753688+0000","last_undegraded":"2023-10-08T06:54:35.753688+0000","last_fullsized":"2023-10-08T06:54:35.753688+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:20.011867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026757+0000","last_change":"2023-10-08T06:54:32.047616+0000","last_active":"2023-10-08T06:54:34.026757+0000","last_peered":"2023-10-08T06:54:34.026757+0000","last_clean":"2023-10-08T06:54:34.026757+0000","last_became_active":"2023-10-08T06:54:32.046275+0000","last_became_peered":"2023-10-08T06:54:32.046275+0000","last_unstale":"2023-10-08T06:54:34.026757+0000","last_undegraded":"2023-10-08T06:54:34.026757+0000","last_fullsized":"2023-10-08T06:54:34.026757+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:38.417271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071169+0000","last_change":"2023-10-08T06:54:33.064640+0000","last_active":"2023-10-08T06:54:34.071169+0000","last_peered":"2023-10-08T06:54:34.071169+0000","last_clean":"2023-10-08T06:54:34.071169+0000","last_became_active":"2023-10-08T06:54:33.064354+0000","last_became_peered":"2023-10-08T06:54:33.064354+0000","last_unstale":"2023-10-08T06:54:34.071169+0000","last_undegraded":"2023-10-08T06:54:34.071169+0000","last_fullsized":"2023-10-08T06:54:34.071169+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:35.691941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153103+0000","last_change":"2023-10-08T06:54:32.042997+0000","last_active":"2023-10-08T06:54:34.153103+0000","last_peered":"2023-10-08T06:54:34.153103+0000","last_clean":"2023-10-08T06:54:34.153103+0000","last_became_active":"2023-10-08T06:54:32.042708+0000","last_became_peered":"2023-10-08T06:54:32.042708+0000","last_unstale":"2023-10-08T06:54:34.153103+0000","last_undegraded":"2023-10-08T06:54:34.153103+0000","last_fullsized":"2023-10-08T06:54:34.153103+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:08:23.643840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153383+0000","last_change":"2023-10-08T06:54:33.051442+0000","last_active":"2023-10-08T06:54:34.153383+0000","last_peered":"2023-10-08T06:54:34.153383+0000","last_clean":"2023-10-08T06:54:34.153383+0000","last_became_active":"2023-10-08T06:54:33.051192+0000","last_became_peered":"2023-10-08T06:54:33.051192+0000","last_unstale":"2023-10-08T06:54:34.153383+0000","last_undegraded":"2023-10-08T06:54:34.153383+0000","last_fullsized":"2023-10-08T06:54:34.153383+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:35:24.833769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753801+0000","last_change":"2023-10-08T06:54:32.032750+0000","last_active":"2023-10-08T06:54:35.753801+0000","last_peered":"2023-10-08T06:54:35.753801+0000","last_clean":"2023-10-08T06:54:35.753801+0000","last_became_active":"2023-10-08T06:54:32.032019+0000","last_became_peered":"2023-10-08T06:54:32.032019+0000","last_unstale":"2023-10-08T06:54:35.753801+0000","last_undegraded":"2023-10-08T06:54:35.753801+0000","last_fullsized":"2023-10-08T06:54:35.753801+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:31:59.960631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153012+0000","last_change":"2023-10-08T06:54:33.050673+0000","last_active":"2023-10-08T06:54:34.153012+0000","last_peered":"2023-10-08T06:54:34.153012+0000","last_clean":"2023-10-08T06:54:34.153012+0000","last_became_active":"2023-10-08T06:54:33.050385+0000","last_became_peered":"2023-10-08T06:54:33.050385+0000","last_unstale":"2023-10-08T06:54:34.153012+0000","last_undegraded":"2023-10-08T06:54:34.153012+0000","last_fullsized":"2023-10-08T06:54:34.153012+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:04:44.045711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.023701+0000","last_change":"2023-10-08T06:54:32.046229+0000","last_active":"2023-10-08T06:54:34.023701+0000","last_peered":"2023-10-08T06:54:34.023701+0000","last_clean":"2023-10-08T06:54:34.023701+0000","last_became_active":"2023-10-08T06:54:32.045966+0000","last_became_peered":"2023-10-08T06:54:32.045966+0000","last_unstale":"2023-10-08T06:54:34.023701+0000","last_undegraded":"2023-10-08T06:54:34.023701+0000","last_fullsized":"2023-10-08T06:54:34.023701+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:35.442869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.753389+0000","last_change":"2023-10-08T06:54:33.056671+0000","last_active":"2023-10-08T06:54:35.753389+0000","last_peered":"2023-10-08T06:54:35.753389+0000","last_clean":"2023-10-08T06:54:35.753389+0000","last_became_active":"2023-10-08T06:54:33.054122+0000","last_became_peered":"2023-10-08T06:54:33.054122+0000","last_unstale":"2023-10-08T06:54:35.753389+0000","last_undegraded":"2023-10-08T06:54:35.753389+0000","last_fullsized":"2023-10-08T06:54:35.753389+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:56.433839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751857+0000","last_change":"2023-10-08T06:54:32.046908+0000","last_active":"2023-10-08T06:54:35.751857+0000","last_peered":"2023-10-08T06:54:35.751857+0000","last_clean":"2023-10-08T06:54:35.751857+0000","last_became_active":"2023-10-08T06:54:32.046497+0000","last_became_peered":"2023-10-08T06:54:32.046497+0000","last_unstale":"2023-10-08T06:54:35.751857+0000","last_undegraded":"2023-10-08T06:54:35.751857+0000","last_fullsized":"2023-10-08T06:54:35.751857+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:05.399479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752910+0000","last_change":"2023-10-08T06:54:33.039976+0000","last_active":"2023-10-08T06:54:35.752910+0000","last_peered":"2023-10-08T06:54:35.752910+0000","last_clean":"2023-10-08T06:54:35.752910+0000","last_became_active":"2023-10-08T06:54:33.039667+0000","last_became_peered":"2023-10-08T06:54:33.039667+0000","last_unstale":"2023-10-08T06:54:35.752910+0000","last_undegraded":"2023-10-08T06:54:35.752910+0000","last_fullsized":"2023-10-08T06:54:35.752910+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:04:28.018050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"197'2","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.153380+0000","last_change":"2023-10-08T06:54:32.037232+0000","last_active":"2023-10-08T06:54:34.153380+0000","last_peered":"2023-10-08T06:54:34.153380+0000","last_clean":"2023-10-08T06:54:34.153380+0000","last_became_active":"2023-10-08T06:54:32.036959+0000","last_became_peered":"2023-10-08T06:54:32.036959+0000","last_unstale":"2023-10-08T06:54:34.153380+0000","last_undegraded":"2023-10-08T06:54:34.153380+0000","last_fullsized":"2023-10-08T06:54:34.153380+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:44:05.653352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026687+0000","last_change":"2023-10-08T06:54:33.063379+0000","last_active":"2023-10-08T06:54:34.026687+0000","last_peered":"2023-10-08T06:54:34.026687+0000","last_clean":"2023-10-08T06:54:34.026687+0000","last_became_active":"2023-10-08T06:54:33.063100+0000","last_became_peered":"2023-10-08T06:54:33.063100+0000","last_unstale":"2023-10-08T06:54:34.026687+0000","last_undegraded":"2023-10-08T06:54:34.026687+0000","last_fullsized":"2023-10-08T06:54:34.026687+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:02.271086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071446+0000","last_change":"2023-10-08T06:54:32.052930+0000","last_active":"2023-10-08T06:54:34.071446+0000","last_peered":"2023-10-08T06:54:34.071446+0000","last_clean":"2023-10-08T06:54:34.071446+0000","last_became_active":"2023-10-08T06:54:32.052685+0000","last_became_peered":"2023-10-08T06:54:32.052685+0000","last_unstale":"2023-10-08T06:54:34.071446+0000","last_undegraded":"2023-10-08T06:54:34.071446+0000","last_fullsized":"2023-10-08T06:54:34.071446+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:09:43.972314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024563+0000","last_change":"2023-10-08T06:54:33.050388+0000","last_active":"2023-10-08T06:54:34.024563+0000","last_peered":"2023-10-08T06:54:34.024563+0000","last_clean":"2023-10-08T06:54:34.024563+0000","last_became_active":"2023-10-08T06:54:33.050119+0000","last_became_peered":"2023-10-08T06:54:33.050119+0000","last_unstale":"2023-10-08T06:54:34.024563+0000","last_undegraded":"2023-10-08T06:54:34.024563+0000","last_fullsized":"2023-10-08T06:54:34.024563+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:47:47.012118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071222+0000","last_change":"2023-10-08T06:54:32.045837+0000","last_active":"2023-10-08T06:54:34.071222+0000","last_peered":"2023-10-08T06:54:34.071222+0000","last_clean":"2023-10-08T06:54:34.071222+0000","last_became_active":"2023-10-08T06:54:32.045576+0000","last_became_peered":"2023-10-08T06:54:32.045576+0000","last_unstale":"2023-10-08T06:54:34.071222+0000","last_undegraded":"2023-10-08T06:54:34.071222+0000","last_fullsized":"2023-10-08T06:54:34.071222+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:18.995943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.026778+0000","last_change":"2023-10-08T06:54:33.063026+0000","last_active":"2023-10-08T06:54:34.026778+0000","last_peered":"2023-10-08T06:54:34.026778+0000","last_clean":"2023-10-08T06:54:34.026778+0000","last_became_active":"2023-10-08T06:54:33.062676+0000","last_became_peered":"2023-10-08T06:54:33.062676+0000","last_unstale":"2023-10-08T06:54:34.026778+0000","last_undegraded":"2023-10-08T06:54:34.026778+0000","last_fullsized":"2023-10-08T06:54:34.026778+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:13.212114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024364+0000","last_change":"2023-10-08T06:54:32.049177+0000","last_active":"2023-10-08T06:54:34.024364+0000","last_peered":"2023-10-08T06:54:34.024364+0000","last_clean":"2023-10-08T06:54:34.024364+0000","last_became_active":"2023-10-08T06:54:32.048907+0000","last_became_peered":"2023-10-08T06:54:32.048907+0000","last_unstale":"2023-10-08T06:54:34.024364+0000","last_undegraded":"2023-10-08T06:54:34.024364+0000","last_fullsized":"2023-10-08T06:54:34.024364+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:00:01.585403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752641+0000","last_change":"2023-10-08T06:54:33.059600+0000","last_active":"2023-10-08T06:54:35.752641+0000","last_peered":"2023-10-08T06:54:35.752641+0000","last_clean":"2023-10-08T06:54:35.752641+0000","last_became_active":"2023-10-08T06:54:33.059250+0000","last_became_peered":"2023-10-08T06:54:33.059250+0000","last_unstale":"2023-10-08T06:54:35.752641+0000","last_undegraded":"2023-10-08T06:54:35.752641+0000","last_fullsized":"2023-10-08T06:54:35.752641+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:30:22.875677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751560+0000","last_change":"2023-10-08T06:54:32.046714+0000","last_active":"2023-10-08T06:54:35.751560+0000","last_peered":"2023-10-08T06:54:35.751560+0000","last_clean":"2023-10-08T06:54:35.751560+0000","last_became_active":"2023-10-08T06:54:32.046376+0000","last_became_peered":"2023-10-08T06:54:32.046376+0000","last_unstale":"2023-10-08T06:54:35.751560+0000","last_undegraded":"2023-10-08T06:54:35.751560+0000","last_fullsized":"2023-10-08T06:54:35.751560+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:32:21.385999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.752473+0000","last_change":"2023-10-08T06:54:33.057577+0000","last_active":"2023-10-08T06:54:35.752473+0000","last_peered":"2023-10-08T06:54:35.752473+0000","last_clean":"2023-10-08T06:54:35.752473+0000","last_became_active":"2023-10-08T06:54:33.057222+0000","last_became_peered":"2023-10-08T06:54:33.057222+0000","last_unstale":"2023-10-08T06:54:35.752473+0000","last_undegraded":"2023-10-08T06:54:35.752473+0000","last_fullsized":"2023-10-08T06:54:35.752473+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:30.486245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071118+0000","last_change":"2023-10-08T06:54:32.034798+0000","last_active":"2023-10-08T06:54:34.071118+0000","last_peered":"2023-10-08T06:54:34.071118+0000","last_clean":"2023-10-08T06:54:34.071118+0000","last_became_active":"2023-10-08T06:54:32.034515+0000","last_became_peered":"2023-10-08T06:54:32.034515+0000","last_unstale":"2023-10-08T06:54:34.071118+0000","last_undegraded":"2023-10-08T06:54:34.071118+0000","last_fullsized":"2023-10-08T06:54:34.071118+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:40.065787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071189+0000","last_change":"2023-10-08T06:54:33.057531+0000","last_active":"2023-10-08T06:54:34.071189+0000","last_peered":"2023-10-08T06:54:34.071189+0000","last_clean":"2023-10-08T06:54:34.071189+0000","last_became_active":"2023-10-08T06:54:33.057247+0000","last_became_peered":"2023-10-08T06:54:33.057247+0000","last_unstale":"2023-10-08T06:54:34.071189+0000","last_undegraded":"2023-10-08T06:54:34.071189+0000","last_fullsized":"2023-10-08T06:54:34.071189+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:45.453690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.754288+0000","last_change":"2023-10-08T06:54:32.037190+0000","last_active":"2023-10-08T06:54:35.754288+0000","last_peered":"2023-10-08T06:54:35.754288+0000","last_clean":"2023-10-08T06:54:35.754288+0000","last_became_active":"2023-10-08T06:54:32.036775+0000","last_became_peered":"2023-10-08T06:54:32.036775+0000","last_unstale":"2023-10-08T06:54:35.754288+0000","last_undegraded":"2023-10-08T06:54:35.754288+0000","last_fullsized":"2023-10-08T06:54:35.754288+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:38:38.588149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.024670+0000","last_change":"2023-10-08T06:54:33.070533+0000","last_active":"2023-10-08T06:54:34.024670+0000","last_peered":"2023-10-08T06:54:34.024670+0000","last_clean":"2023-10-08T06:54:34.024670+0000","last_became_active":"2023-10-08T06:54:33.070235+0000","last_became_peered":"2023-10-08T06:54:33.070235+0000","last_unstale":"2023-10-08T06:54:34.024670+0000","last_undegraded":"2023-10-08T06:54:34.024670+0000","last_fullsized":"2023-10-08T06:54:34.024670+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:18:28.076640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.152729+0000","last_change":"2023-10-08T06:54:32.039639+0000","last_active":"2023-10-08T06:54:34.152729+0000","last_peered":"2023-10-08T06:54:34.152729+0000","last_clean":"2023-10-08T06:54:34.152729+0000","last_became_active":"2023-10-08T06:54:32.039367+0000","last_became_peered":"2023-10-08T06:54:32.039367+0000","last_unstale":"2023-10-08T06:54:34.152729+0000","last_undegraded":"2023-10-08T06:54:34.152729+0000","last_fullsized":"2023-10-08T06:54:34.152729+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:40.390656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:35.751966+0000","last_change":"2023-10-08T06:54:33.047603+0000","last_active":"2023-10-08T06:54:35.751966+0000","last_peered":"2023-10-08T06:54:35.751966+0000","last_clean":"2023-10-08T06:54:35.751966+0000","last_became_active":"2023-10-08T06:54:33.046439+0000","last_became_peered":"2023-10-08T06:54:33.046439+0000","last_unstale":"2023-10-08T06:54:35.751966+0000","last_undegraded":"2023-10-08T06:54:35.751966+0000","last_fullsized":"2023-10-08T06:54:35.751966+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:31.994549+0000","last_clean_scrub_stamp":"2023-10-08T06:54:31.994549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.647045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-10-08T06:54:34.071422+0000","last_change":"2023-10-08T06:54:32.052930+0000","last_active":"2023-10-08T06:54:34.071422+0000","last_peered":"2023-10-08T06:54:34.071422+0000","last_clean":"2023-10-08T06:54:34.071422+0000","last_became_active":"2023-10-08T06:54:32.052513+0000","last_became_peered":"2023-10-08T06:54:32.052513+0000","last_unstale":"2023-10-08T06:54:34.071422+0000","last_undegraded":"2023-10-08T06:54:34.071422+0000","last_fullsized":"2023-10-08T06:54:34.071422+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T06:54:30.978625+0000","last_clean_scrub_stamp":"2023-10-08T06:54:30.978625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:31:22.946322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"187'2751","reported_seq":3486,"reported_epoch":197,"state":"active+clean","last_fresh":"2023-10-08T06:54:33.029296+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T06:54:33.029296+0000","last_peered":"2023-10-08T06:54:33.029296+0000","last_clean":"2023-10-08T06:54:33.029296+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T06:54:33.029296+0000","last_undegraded":"2023-10-08T06:54:33.029296+0000","last_fullsized":"2023-10-08T06:54:33.029296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2751,"log_dups_size":0,"ondisk_log_size":2751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1725,"num_read_kb":3588,"num_write":489,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1725,"num_read_kb":3588,"num_write":489,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2751,"ondisk_log_size":2751,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575784,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60548,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311292,"statfs":{"total":96636764160,"available":96574763008,"internally_reserved":0,"allocated":3072000,"data_stored":1865487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.2010000000000001,"15min":1.218},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.462,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.206,"15min":1.2470000000000001},"min":{"1min":0.75800000000000001,"5min":0.747,"15min":0.72899999999999998},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.849},"last":0.85299999999999998}]},{"osd":1,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2589999999999999,"15min":1.2609999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.387},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.284,"15min":1.3149999999999999},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.82999999999999996},"max":{"1min":1.7290000000000001,"5min":1.9299999999999999,"15min":2.4809999999999999},"last":1.107}]},{"osd":2,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.2769999999999999,"15min":1.2909999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4970000000000001,"5min":2.7040000000000002,"15min":3.4350000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3520000000000001,"15min":1.3640000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.0019999999999998,"5min":2.7120000000000002,"15min":2.7120000000000002},"last":1.014}]},{"osd":3,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3180000000000001,"15min":1.323},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.637,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.379,"5min":1.4550000000000001,"15min":1.419},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.2240000000000002,"5min":2.8239999999999998,"15min":2.8239999999999998},"last":1.286}]},{"osd":4,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.373,"15min":1.292},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.66200000000000003},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":3.2839999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.444,"15min":1.3879999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.9930000000000001,"5min":2.004,"15min":2.1909999999999998},"last":1.4690000000000001}]},{"osd":5,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4810000000000001,"15min":1.4390000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.3319999999999999,"5min":2.9329999999999998,"15min":2.9329999999999998},"last":1.45},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.528,"15min":1.4850000000000001},"min":{"1min":0.97799999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.4219999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:54:04 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.5740000000000001,"15min":1.5589999999999999},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":2.4209999999999998,"5min":2.4350000000000001,"15min":3.1360000000000001},"last":1.329},{"interface":"front","average":{"1min":1.514,"5min":1.5469999999999999,"15min":1.5029999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.3140000000000001},"last":0.96099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575783,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41920,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329920,"statfs":{"total":96636764160,"available":96593838080,"internally_reserved":0,"allocated":2478080,"data_stored":1275119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.264,"15min":1.337},"min":{"1min":0.68899999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.508},"last":1.573},{"interface":"front","average":{"1min":1.325,"5min":1.302,"15min":1.339},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":3.7519999999999998,"5min":3.7519999999999998,"15min":3.7519999999999998},"last":1.302}]},{"osd":1,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.222,"15min":1.244},"min":{"1min":0.623,"5min":0.623,"15min":0.59699999999999998},"max":{"1min":1.5800000000000001,"5min":1.821,"15min":1.821},"last":1.383},{"interface":"front","average":{"1min":1.256,"5min":1.2689999999999999,"15min":1.2629999999999999},"min":{"1min":0.83099999999999996,"5min":0.65200000000000002,"15min":0.53800000000000003},"max":{"1min":1.7809999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.93799999999999994}]},{"osd":2,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.2470000000000001,"15min":1.2390000000000001},"min":{"1min":0.88100000000000001,"5min":0.85099999999999998,"15min":0.75600000000000001},"max":{"1min":3.46,"5min":3.46,"15min":3.46},"last":1.653},{"interface":"front","average":{"1min":1.137,"5min":1.26,"15min":1.288},"min":{"1min":0.872,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4950000000000001,"5min":1.6379999999999999,"15min":2.3839999999999999},"last":1.2430000000000001}]},{"osd":3,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4199999999999999,"15min":1.3999999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.83399999999999996},"max":{"1min":2.048,"5min":2.048,"15min":2.1499999999999999},"last":1.7969999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4730000000000001,"15min":1.4790000000000001},"min":{"1min":1.01,"5min":0.96699999999999997,"15min":0.89400000000000002},"max":{"1min":1.829,"5min":1.915,"15min":2.0289999999999999},"last":1.125}]},{"osd":4,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3859999999999999,"15min":1.357},"min":{"1min":0.89100000000000001,"5min":0.80100000000000005,"15min":0.72399999999999998},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.4670000000000001,"15min":1.454},"min":{"1min":1.006,"5min":0.878,"15min":0.80200000000000005},"max":{"1min":2.9199999999999999,"5min":2.9199999999999999,"15min":2.9199999999999999},"last":1.4199999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4299999999999999,"15min":1.4159999999999999},"min":{"1min":0.97499999999999998,"5min":0.94799999999999995,"15min":0.69499999999999995},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.7309999999999999},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.5589999999999999,"15min":1.5489999999999999},"min":{"1min":1.099,"5min":1.0249999999999999,"15min":0.78300000000000003},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":1.9419999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.776,"5min":1.629,"15min":1.5940000000000001},"min":{"1min":1.351,"5min":1.0800000000000001,"15min":0.92000000000000004},"max":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":1.889},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5600000000000001,"15min":1.5640000000000001},"min":{"1min":1.1319999999999999,"5min":0.90800000000000003,"15min":0.82099999999999995},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.6070000000000002},"last":1.48}]}]},{"osd":5,"up_from":13,"seq":55834575783,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59972,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311868,"statfs":{"total":96636764160,"available":96575352832,"internally_reserved":0,"allocated":2473984,"data_stored":1268507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.2310000000000001,"15min":1.218},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.66600000000000004},"max":{"1min":1.5740000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.296,"15min":1.2569999999999999},"min":{"1min":0.95399999999999996,"5min":0.82699999999999996,"15min":0.78800000000000003},"max":{"1min":1.629,"5min":1.897,"15min":2.4700000000000002},"last":0.91700000000000004}]},{"osd":1,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.26,"15min":1.2230000000000001},"min":{"1min":0.95499999999999996,"5min":0.85799999999999998,"15min":0.72499999999999998},"max":{"1min":1.8839999999999999,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.375,"15min":1.323},"min":{"1min":0.90400000000000003,"5min":0.88700000000000001,"15min":0.78200000000000003},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.7150000000000001}]},{"osd":2,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3480000000000001,"15min":1.3280000000000001},"min":{"1min":0.98399999999999999,"5min":0.85199999999999998,"15min":0.83199999999999996},"max":{"1min":1.6479999999999999,"5min":1.9359999999999999,"15min":2.351},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.333,"15min":1.3160000000000001},"min":{"1min":0.91000000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.667,"5min":1.7809999999999999,"15min":1.964},"last":1.0820000000000001}]},{"osd":3,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.3959999999999999,"15min":1.3540000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.528,"15min":1.464},"min":{"1min":1.139,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":1.9910000000000001,"5min":3.0059999999999998,"15min":5.1630000000000003},"last":1.125}]},{"osd":4,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3320000000000001,"15min":1.3240000000000001},"min":{"1min":0.96299999999999997,"5min":0.89300000000000002,"15min":0.81499999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":2.4039999999999999},"last":1.383},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4379999999999999,"15min":1.4239999999999999},"min":{"1min":0.96899999999999997,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.7989999999999999}]},{"osd":6,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.4990000000000001,"15min":1.4750000000000001},"min":{"1min":1.248,"5min":0.96399999999999997,"15min":0.81200000000000006},"max":{"1min":2.1150000000000002,"5min":2.1659999999999999,"15min":5.5019999999999998},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4930000000000001,"15min":1.48},"min":{"1min":1.0509999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.0139999999999998,"5min":2.1040000000000001,"15min":2.2890000000000001},"last":1.466}]},{"osd":7,"last update":"Sun Oct 8 06:53:35 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.5660000000000001,"15min":1.5629999999999999},"min":{"1min":1.3400000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":5.8780000000000001},"last":1.869},{"interface":"front","average":{"1min":1.679,"5min":1.6839999999999999,"15min":1.6299999999999999},"min":{"1min":1.415,"5min":1.1719999999999999,"15min":0.97099999999999997},"max":{"1min":2.206,"5min":2.6789999999999998,"15min":5.399},"last":1.671}]}]},{"osd":4,"up_from":13,"seq":55834575783,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41932,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329908,"statfs":{"total":96636764160,"available":96593825792,"internally_reserved":0,"allocated":2482176,"data_stored":1269147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.389,"15min":1.3460000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"last":2.0880000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.3999999999999999},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.69799999999999995},"max":{"1min":1.7789999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.5249999999999999}]},{"osd":1,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2749999999999999,"15min":1.28},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.73,"5min":1.784,"15min":1.9339999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2689999999999999,"15min":1.2529999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.64000000000000001},"max":{"1min":1.613,"5min":1.6870000000000001,"15min":1.7290000000000001},"last":1.329}]},{"osd":2,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.2889999999999999,"15min":1.248},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":4.3719999999999999,"5min":4.3719999999999999,"15min":4.3719999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.383,"15min":1.3759999999999999},"min":{"1min":1,"5min":0.93100000000000005,"15min":0.85399999999999998},"max":{"1min":1.9399999999999999,"5min":1.9399999999999999,"15min":1.9399999999999999},"last":1.393}]},{"osd":3,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.339,"15min":1.3109999999999999},"min":{"1min":0.92800000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":1.206},{"interface":"front","average":{"1min":1.55,"5min":1.4670000000000001,"15min":1.4099999999999999},"min":{"1min":1.0629999999999999,"5min":0.97299999999999998,"15min":0.93400000000000005},"max":{"1min":4.4889999999999999,"5min":4.4889999999999999,"15min":4.4889999999999999},"last":1.867}]},{"osd":5,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.4139999999999999,"15min":1.351},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.66700000000000004},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.4450000000000001,"15min":1.375},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.749},"max":{"1min":2.4540000000000002,"5min":2.4540000000000002,"15min":2.4540000000000002},"last":1.7150000000000001}]},{"osd":6,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":1.5329999999999999,"15min":1.425},"min":{"1min":0.97899999999999998,"5min":0.91100000000000003,"15min":0.72699999999999998},"max":{"1min":5.3339999999999996,"5min":5.3339999999999996,"15min":5.3339999999999996},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.6120000000000001,"15min":1.546},"min":{"1min":1.212,"5min":0.96699999999999997,"15min":0.91400000000000003},"max":{"1min":2.1059999999999999,"5min":2.6240000000000001,"15min":2.6240000000000001},"last":1.669}]},{"osd":7,"last update":"Sun Oct 8 06:53:38 2023","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.548,"15min":1.49},"min":{"1min":1.202,"5min":1.135,"15min":0.99099999999999999},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.784,"5min":1.6859999999999999,"15min":1.6060000000000001},"min":{"1min":1.0049999999999999,"5min":0.98999999999999999,"15min":0.93600000000000005},"max":{"1min":5.4450000000000003,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":2.355}]}]},{"osd":3,"up_from":13,"seq":55834575785,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47512,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324328,"statfs":{"total":96636764160,"available":96588111872,"internally_reserved":0,"allocated":2498560,"data_stored":1277335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.92100000000000004,"15min":0.95099999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.44500000000000001},"max":{"1min":1.1910000000000001,"5min":1.71,"15min":1.71},"last":1.4339999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.1499999999999999,"15min":1.165},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.2330000000000001,"5min":1.851,"15min":2.137},"last":1.0600000000000001}]},{"osd":1,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.98199999999999998,"15min":1.0049999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.341,"5min":1.659,"15min":1.823},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.2430000000000001,"15min":1.2869999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.613,"5min":1.75,"15min":1.893},"last":1.5349999999999999}]},{"osd":2,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.1299999999999999,"15min":1.117},"min":{"1min":0.57299999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.423,"5min":1.6899999999999999,"15min":1.7410000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.1479999999999999,"15min":1.179},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.454,"5min":1.7809999999999999,"15min":1.8360000000000001},"last":0.84199999999999997}]},{"osd":4,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.268,"15min":1.2629999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.7370000000000001,"5min":1.97,"15min":1.97},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.4399999999999999,"15min":1.446},"min":{"1min":1.0149999999999999,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.7070000000000001,"5min":1.905,"15min":2.1960000000000002},"last":1.278}]},{"osd":5,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.341,"15min":1.369},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.73,"5min":2.27,"15min":3.0009999999999999},"last":1.107},{"interface":"front","average":{"1min":1.25,"5min":1.389,"15min":1.4119999999999999},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.724,"5min":2.2429999999999999,"15min":2.2429999999999999},"last":1.806}]},{"osd":6,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.3720000000000001,"15min":1.3879999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.748,"5min":1.9950000000000001,"15min":2.016},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4990000000000001,"15min":1.508},"min":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":0.99099999999999999},"max":{"1min":1.8029999999999999,"5min":2.0489999999999999,"15min":2.544},"last":1.353}]},{"osd":7,"last update":"Sun Oct 8 06:54:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.5329999999999999,"15min":1.5589999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.948,"5min":2.008,"15min":2.347},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4790000000000001,"15min":1.474},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.873,"5min":2.1099999999999999,"15min":2.1800000000000002},"last":1.2}]}]},{"osd":2,"up_from":13,"seq":55834575783,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47496,"kb_used_data":2420,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324344,"statfs":{"total":96636764160,"available":96588128256,"internally_reserved":0,"allocated":2478080,"data_stored":1275119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.88400000000000001,"15min":0.91900000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.167,"5min":1.345,"15min":1.657},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.96699999999999997,"5min":1.046,"15min":1.073},"min":{"1min":0.46800000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3360000000000001,"5min":1.661,"15min":2.2679999999999998},"last":0.53300000000000003}]},{"osd":1,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.98999999999999999,"15min":0.996},"min":{"1min":0.52900000000000003,"5min":0.373,"15min":0.373},"max":{"1min":1.637,"5min":1.7170000000000001,"15min":1.996},"last":0.495},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.1220000000000001,"15min":1.1659999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.5269999999999999,"5min":1.8500000000000001,"15min":2.1139999999999999},"last":0.624}]},{"osd":3,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2190000000000001,"15min":1.2230000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.956,"5min":1.956,"15min":2.3929999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.228,"15min":1.2490000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7050000000000001,"5min":1.804,"15min":1.891},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.4079999999999999,"15min":1.4279999999999999},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.776,"5min":1.9430000000000001,"15min":3.0939999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.321,"5min":1.3180000000000001,"15min":1.3260000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.8260000000000001,"5min":1.8260000000000001,"15min":3.7320000000000002},"last":0.59599999999999997}]},{"osd":5,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.417,"15min":1.4279999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8540000000000001,"5min":1.9770000000000001,"15min":3.3519999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.478,"15min":1.512},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.911,"5min":2.2719999999999998,"15min":2.5249999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.446,"15min":1.411},"min":{"1min":1,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.798,"5min":1.9990000000000001,"15min":2.206},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.605,"5min":1.575,"15min":1.54},"min":{"1min":1.177,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":3.02},"last":0.78500000000000003}]},{"osd":7,"last update":"Sun Oct 8 06:54:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.5169999999999999,"15min":1.4990000000000001},"min":{"1min":0.81000000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.9259999999999999,"5min":2.0790000000000002,"15min":4.2869999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.528,"5min":1.5860000000000001,"15min":1.579},"min":{"1min":0.89100000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":3.0760000000000001},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575783,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29456,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342384,"statfs":{"total":96636764160,"available":96606601216,"internally_reserved":0,"allocated":2490368,"data_stored":1276731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.333,"5min":4.601,"15min":4.601},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.90700000000000003,"5min":1.155,"15min":1.2},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.613,"5min":1.917,"15min":1.917},"last":0.92100000000000004}]},{"osd":2,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.98899999999999999,"15min":1.0149999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1519999999999999,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":0.874},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.1759999999999999,"15min":1.1579999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.5049999999999999,"5min":5.4059999999999997,"15min":5.4059999999999997},"last":1.1659999999999999}]},{"osd":3,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":1.256,"15min":1.248},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5289999999999999,"5min":5.5220000000000002,"15min":5.5220000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.95399999999999996,"5min":1.115,"15min":1.107},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.484,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.0449999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.3520000000000001,"15min":1.353},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.702,"5min":4.2409999999999997,"15min":4.2409999999999997},"last":0.995},{"interface":"front","average":{"1min":1.157,"5min":1.302,"15min":1.2929999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6879999999999999,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":1.4279999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.391,"15min":1.3520000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5189999999999999,"5min":4.7530000000000001,"15min":4.7530000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.5529999999999999,"15min":1.5249999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.6699999999999999,"5min":5.7389999999999999,"15min":5.7389999999999999},"last":1.266}]},{"osd":6,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.5249999999999999,"15min":1.516},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8140000000000001,"5min":4.4180000000000001,"15min":4.4180000000000001},"last":1.083},{"interface":"front","average":{"1min":1.349,"5min":1.506,"15min":1.4770000000000001},"min":{"1min":0.86899999999999999,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8169999999999999,"5min":4.6660000000000004,"15min":4.6660000000000004},"last":1.5700000000000001}]},{"osd":7,"last update":"Sun Oct 8 06:54:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.5529999999999999,"15min":1.542},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7450000000000001,"5min":5.6289999999999996,"15min":5.6289999999999996},"last":1.385},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.5529999999999999,"15min":1.5429999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.9039999999999999,"5min":4.5270000000000001,"15min":4.5270000000000001},"last":1.1240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575784,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38348,"kb_used_data":3008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333492,"statfs":{"total":96636764160,"available":96597495808,"internally_reserved":0,"allocated":3080192,"data_stored":1865591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":36174281},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.113,"15min":1.014},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":14.417999999999999,"5min":14.417999999999999,"15min":14.417999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1419999999999999,"15min":1.091},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":8.5289999999999999,"5min":8.5289999999999999,"15min":8.5289999999999999},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.605,"5min":1.3029999999999999,"15min":1.1899999999999999},"min":{"1min":0.54100000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":14.459,"5min":14.459,"15min":14.459},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.995,"5min":1.042,"15min":1.071},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":4.7270000000000003,"5min":4.7270000000000003,"15min":4.7270000000000003},"last":0.502}]},{"osd":3,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.7669999999999999,"5min":1.3839999999999999,"15min":1.3009999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":14.364000000000001,"5min":14.364000000000001,"15min":14.364000000000001},"last":1.099},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2170000000000001,"15min":1.1910000000000001},"min":{"1min":0.47399999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":8.5609999999999999,"5min":8.5609999999999999,"15min":8.5609999999999999},"last":1.0349999999999999}]},{"osd":4,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.3029999999999999,"15min":1.3520000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.859,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.143},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.466,"15min":1.4259999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":8.4450000000000003,"5min":8.4450000000000003,"15min":8.4450000000000003},"last":1.3979999999999999}]},{"osd":5,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.4199999999999999,"15min":1.399},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.6859999999999999,"5min":4.6859999999999999,"15min":4.6859999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":2.0390000000000001,"5min":1.631,"15min":1.5149999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":14.515000000000001,"5min":14.515000000000001,"15min":14.515000000000001},"last":1.2130000000000001}]},{"osd":6,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.528,"15min":1.4910000000000001},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":8.4939999999999998,"5min":8.4939999999999998,"15min":8.4939999999999998},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.444,"5min":1.492,"15min":1.504},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.7779999999999996,"5min":4.7779999999999996,"15min":4.7779999999999996},"last":1.3759999999999999}]},{"osd":7,"last update":"Sun Oct 8 06:54:34 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.5289999999999999,"15min":1.532},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":4.641,"5min":4.641,"15min":4.641},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.8260000000000001,"5min":1.613,"15min":1.5669999999999999},"min":{"1min":0.92100000000000004,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":10.195,"5min":10.195,"15min":10.195},"last":1.484}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T06:54:38.751 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T06:54:38.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T06:54:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.983+0000 7f597ba32640 1 Processor -- start 2023-10-08T06:54:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.983+0000 7f597ba32640 1 -- start start 2023-10-08T06:54:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.985+0000 7f597ba32640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f597415e300 0x7f59740cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.985+0000 7f597ba32640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59740cba40 0x7f59740c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.985+0000 7f597ba32640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59740b2890 con 0x7f5974159d50 2023-10-08T06:54:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.985+0000 7f597ba32640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597415d8b0 con 0x7f59740cba40 2023-10-08T06:54:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.985+0000 7f597ba32640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59740cbe70 con 0x7f597415e300 2023-10-08T06:54:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.985+0000 7f5979fcf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59740cba40 0x7f59740c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.985+0000 7f5979fcf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59740cba40 0x7f59740c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60720/0 (socket says 172.21.15.70:60720) 2023-10-08T06:54:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.985+0000 7f5979fcf640 1 -- 172.21.15.70:0/909877741 learned_addr learned my addr 172.21.15.70:0/909877741 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:38.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.986+0000 7f5978fcd640 1 --2- 172.21.15.70:0/909877741 >> v2:172.21.15.181:3300/0 conn(0x7f597415e300 0x7f59740cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:38.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.986+0000 7f5979fcf640 1 -- 172.21.15.70:0/909877741 >> v2:172.21.15.181:3300/0 conn(0x7f597415e300 msgr2=0x7f59740cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:38.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.986+0000 7f5979fcf640 1 --2- 172.21.15.70:0/909877741 >> v2:172.21.15.181:3300/0 conn(0x7f597415e300 0x7f59740cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.986+0000 7f5979fcf640 1 -- 172.21.15.70:0/909877741 >> v1:172.21.15.181:6789/0 conn(0x7f5974159d50 legacy=0x7f59740b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:38.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.986+0000 7f5979fcf640 1 -- 172.21.15.70:0/909877741 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597415e790 con 0x7f59740cba40 2023-10-08T06:54:38.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.987+0000 7f5979fcf640 1 --2- 172.21.15.70:0/909877741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59740cba40 0x7f59740c9310 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f597406dac0 tx=0x7f596c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8469c75ee79b502e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:38.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.987+0000 7f596bfff640 1 -- 172.21.15.70:0/909877741 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c031870 con 0x7f59740cba40 2023-10-08T06:54:38.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.987+0000 7f596bfff640 1 -- 172.21.15.70:0/909877741 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c0319d0 con 0x7f59740cba40 2023-10-08T06:54:38.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.987+0000 7f596bfff640 1 -- 172.21.15.70:0/909877741 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c031d60 con 0x7f59740cba40 2023-10-08T06:54:38.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.988+0000 7f597ba32640 1 -- 172.21.15.70:0/909877741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59740cba40 msgr2=0x7f59740c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:38.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.988+0000 7f597ba32640 1 --2- 172.21.15.70:0/909877741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59740cba40 0x7f59740c9310 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f597406dac0 tx=0x7f596c02b910 comp rx=0 tx=0).stop 2023-10-08T06:54:38.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.988+0000 7f597ba32640 1 -- 172.21.15.70:0/909877741 shutdown_connections 2023-10-08T06:54:38.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.988+0000 7f597ba32640 1 --2- 172.21.15.70:0/909877741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59740cba40 0x7f59740c9310 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.988+0000 7f597ba32640 1 --2- 172.21.15.70:0/909877741 >> v2:172.21.15.181:3300/0 conn(0x7f597415e300 0x7f59740cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.988+0000 7f597ba32640 1 -- 172.21.15.70:0/909877741 >> 172.21.15.70:0/909877741 conn(0x7f597406a140 msgr2=0x7f597406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:38.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.988+0000 7f597ba32640 1 -- 172.21.15.70:0/909877741 shutdown_connections 2023-10-08T06:54:38.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.988+0000 7f597ba32640 1 -- 172.21.15.70:0/909877741 wait complete. 2023-10-08T06:54:38.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.990+0000 7f597ba32640 1 Processor -- start 2023-10-08T06:54:38.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.990+0000 7f597ba32640 1 -- start start 2023-10-08T06:54:38.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.991+0000 7f597ba32640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59740cba40 0x7f597415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.991+0000 7f597ba32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5974159d50 0x7f59741f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.991+0000 7f597ba32640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f597415e300 0x7f5974155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.991+0000 7f597ba32640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597415a130 con 0x7f5974159d50 2023-10-08T06:54:38.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.991+0000 7f597ba32640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59741f6d10 con 0x7f597415e300 2023-10-08T06:54:38.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.991+0000 7f597ba32640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597415a3d0 con 0x7f59740cba40 2023-10-08T06:54:38.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.991+0000 7f5978fcd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5974159d50 0x7f59741f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:38.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.991+0000 7f5978fcd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5974159d50 0x7f59741f67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60728/0 (socket says 172.21.15.70:60728) 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.991+0000 7f5978fcd640 1 -- 172.21.15.70:0/242154692 learned_addr learned my addr 172.21.15.70:0/242154692 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5979fcf640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f597415e300 0x7f5974155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f59797ce640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59740cba40 0x7f597415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5978fcd640 1 -- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59740cba40 msgr2=0x7f597415cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5978fcd640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59740cba40 0x7f597415cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5978fcd640 1 -- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f597415e300 msgr2=0x7f5974155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5978fcd640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f597415e300 0x7f5974155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5978fcd640 1 -- 172.21.15.70:0/242154692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0094d0 con 0x7f5974159d50 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5979fcf640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f597415e300 0x7f5974155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5978fcd640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5974159d50 0x7f59741f67d0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f596400c7f0 tx=0x7f596400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5969ffb640 1 -- 172.21.15.70:0/242154692 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964004220 con 0x7f5974159d50 2023-10-08T06:54:38.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5969ffb640 1 -- 172.21.15.70:0/242154692 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5964004380 con 0x7f5974159d50 2023-10-08T06:54:38.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f5969ffb640 1 -- 172.21.15.70:0/242154692 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964003830 con 0x7f5974159d50 2023-10-08T06:54:38.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.992+0000 7f597ba32640 1 -- 172.21.15.70:0/242154692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5974156090 con 0x7f5974159d50 2023-10-08T06:54:38.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.993+0000 7f597ba32640 1 -- 172.21.15.70:0/242154692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59741f7200 con 0x7f5974159d50 2023-10-08T06:54:38.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.994+0000 7f597ba32640 1 -- 172.21.15.70:0/242154692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597406aa50 con 0x7f5974159d50 2023-10-08T06:54:38.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.995+0000 7f5969ffb640 1 -- 172.21.15.70:0/242154692 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5964003ba0 con 0x7f5974159d50 2023-10-08T06:54:38.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.996+0000 7f5969ffb640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f595806afc0 0x7f595806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:38.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.996+0000 7f5969ffb640 1 -- 172.21.15.70:0/242154692 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5964089800 con 0x7f5974159d50 2023-10-08T06:54:38.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.996+0000 7f59797ce640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f595806afc0 0x7f595806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:38.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:38.997+0000 7f59797ce640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f595806afc0 0x7f595806d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5970007fd0 tx=0x7f5970009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:39.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.000+0000 7f5969ffb640 1 -- 172.21.15.70:0/242154692 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5964054560 con 0x7f5974159d50 2023-10-08T06:54:39.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.259+0000 7f597ba32640 1 -- 172.21.15.70:0/242154692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5974159560 con 0x7f5974159d50 2023-10-08T06:54:39.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.263+0000 7f5969ffb640 1 -- 172.21.15.70:0/242154692 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v45) v1 ==== 190+0+147 (secure 0 0 0) 0x7f596401e0e0 con 0x7f5974159d50 2023-10-08T06:54:39.265 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T06:54:39.265 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T06:54:39.265 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T06:54:39.265 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T06:54:39.266 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T06:54:39.266 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T06:54:39.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 -- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f595806afc0 msgr2=0x7f595806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:39.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f595806afc0 0x7f595806d480 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5970007fd0 tx=0x7f5970009040 comp rx=0 tx=0).stop 2023-10-08T06:54:39.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 -- 172.21.15.70:0/242154692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5974159d50 msgr2=0x7f59741f67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:39.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5974159d50 0x7f59741f67d0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f596400c7f0 tx=0x7f596400ccb0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 -- 172.21.15.70:0/242154692 shutdown_connections 2023-10-08T06:54:39.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f595806afc0 0x7f595806d480 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f597415e300 0x7f5974155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5974159d50 0x7f59741f67d0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 --2- 172.21.15.70:0/242154692 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f59740cba40 0x7f597415cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 -- 172.21.15.70:0/242154692 >> 172.21.15.70:0/242154692 conn(0x7f597406a140 msgr2=0x7f59740c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:39.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 -- 172.21.15.70:0/242154692 shutdown_connections 2023-10-08T06:54:39.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.266+0000 7f597ba32640 1 -- 172.21.15.70:0/242154692 wait complete. 2023-10-08T06:54:39.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T06:54:39.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.490+0000 7fecf353c640 1 Processor -- start 2023-10-08T06:54:39.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.491+0000 7fecf353c640 1 -- start start 2023-10-08T06:54:39.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.492+0000 7fecf353c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fecec06d4c0 0x7fecec162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:39.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.492+0000 7fecf353c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec0cb150 0x7fecec0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:39.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.492+0000 7fecf353c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec0cbaf0 con 0x7fecec0add60 2023-10-08T06:54:39.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.492+0000 7fecf353c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec161f10 con 0x7fecec06d4c0 2023-10-08T06:54:39.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.492+0000 7fecf353c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec161630 con 0x7fecec0cb150 2023-10-08T06:54:39.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.492+0000 7fecf12d8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fecec0add60 0x7fecec0cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37764/0 (socket says 172.21.15.70:37764) 2023-10-08T06:54:39.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.492+0000 7fecf12d8640 1 -- 172.21.15.70:0/3877425112 learned_addr learned my addr 172.21.15.70:0/3877425112 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:39.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.493+0000 7fecf0ad7640 1 --2- 172.21.15.70:0/3877425112 >> v2:172.21.15.70:3300/0 conn(0x7fecec06d4c0 0x7fecec162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:39.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.493+0000 7fecf1ad9640 1 --2- 172.21.15.70:0/3877425112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec0cb150 0x7fecec0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:39.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.493+0000 7fecf0ad7640 1 -- 172.21.15.70:0/3877425112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec0cb150 msgr2=0x7fecec0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:39.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.493+0000 7fecf0ad7640 1 --2- 172.21.15.70:0/3877425112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec0cb150 0x7fecec0cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.493+0000 7fecf0ad7640 1 -- 172.21.15.70:0/3877425112 >> v1:172.21.15.70:6789/0 conn(0x7fecec0add60 legacy=0x7fecec0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:39.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.493+0000 7fecf0ad7640 1 -- 172.21.15.70:0/3877425112 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0ae140 con 0x7fecec06d4c0 2023-10-08T06:54:39.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.493+0000 7fece3fff640 1 -- 172.21.15.70:0/3877425112 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279227071 0 0) 0x7fecec0cbaf0 con 0x7fecec0add60 2023-10-08T06:54:39.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.494+0000 7fecf0ad7640 1 --2- 172.21.15.70:0/3877425112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec06d4c0 0x7fecec162330 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0097d0 tx=0x7fecdc02b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e458183c15e171b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:39.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.494+0000 7fece3fff640 1 -- 172.21.15.70:0/3877425112 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc031770 con 0x7fecec06d4c0 2023-10-08T06:54:39.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.494+0000 7fece3fff640 1 -- 172.21.15.70:0/3877425112 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc0318d0 con 0x7fecec06d4c0 2023-10-08T06:54:39.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.494+0000 7fece3fff640 1 -- 172.21.15.70:0/3877425112 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc031c60 con 0x7fecec06d4c0 2023-10-08T06:54:39.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.495+0000 7fecf353c640 1 -- 172.21.15.70:0/3877425112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec06d4c0 msgr2=0x7fecec162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:39.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.495+0000 7fecf353c640 1 --2- 172.21.15.70:0/3877425112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec06d4c0 0x7fecec162330 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0097d0 tx=0x7fecdc02b810 comp rx=0 tx=0).stop 2023-10-08T06:54:39.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.495+0000 7fecf353c640 1 -- 172.21.15.70:0/3877425112 shutdown_connections 2023-10-08T06:54:39.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.495+0000 7fecf353c640 1 --2- 172.21.15.70:0/3877425112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec0cb150 0x7fecec0cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.495+0000 7fecf353c640 1 --2- 172.21.15.70:0/3877425112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec06d4c0 0x7fecec162330 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.495+0000 7fecf353c640 1 -- 172.21.15.70:0/3877425112 >> 172.21.15.70:0/3877425112 conn(0x7fecec06a380 msgr2=0x7fecec0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:39.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.495+0000 7fecf353c640 1 -- 172.21.15.70:0/3877425112 shutdown_connections 2023-10-08T06:54:39.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.495+0000 7fecf353c640 1 -- 172.21.15.70:0/3877425112 wait complete. 2023-10-08T06:54:39.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.497+0000 7fecf353c640 1 Processor -- start 2023-10-08T06:54:39.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.497+0000 7fecf353c640 1 -- start start 2023-10-08T06:54:39.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.498+0000 7fecf353c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec06d4c0 0x7fecec0b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:39.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.498+0000 7fecf353c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecec0add60 0x7fecec1fa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:39.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.498+0000 7fecf353c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec0cb150 0x7fecec0af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:39.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.498+0000 7fecf353c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec1572a0 con 0x7fecec0cb150 2023-10-08T06:54:39.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.498+0000 7fecf353c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b1480 con 0x7fecec0add60 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.498+0000 7fecf353c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b16d0 con 0x7fecec06d4c0 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.498+0000 7fecf1ad9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec0cb150 0x7fecec0af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.498+0000 7fecf1ad9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec0cb150 0x7fecec0af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60760/0 (socket says 172.21.15.70:60760) 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.498+0000 7fecf1ad9640 1 -- 172.21.15.70:0/990845275 learned_addr learned my addr 172.21.15.70:0/990845275 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fecf12d8640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec06d4c0 0x7fecec0b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fecf1ad9640 1 -- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec06d4c0 msgr2=0x7fecec0b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fecf0ad7640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecec0add60 0x7fecec1fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fecf1ad9640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec06d4c0 0x7fecec0b3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fecf1ad9640 1 -- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecec0add60 msgr2=0x7fecec1fa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fecf1ad9640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecec0add60 0x7fecec1fa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fecf1ad9640 1 -- 172.21.15.70:0/990845275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc009400 con 0x7fecec0cb150 2023-10-08T06:54:39.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fecf1ad9640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec0cb150 0x7fecec0af880 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fece4007e60 tx=0x7fece400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:39.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fece1ffb640 1 -- 172.21.15.70:0/990845275 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4014420 con 0x7fecec0cb150 2023-10-08T06:54:39.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fece1ffb640 1 -- 172.21.15.70:0/990845275 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece40145a0 con 0x7fecec0cb150 2023-10-08T06:54:39.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fece1ffb640 1 -- 172.21.15.70:0/990845275 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4014950 con 0x7fecec0cb150 2023-10-08T06:54:39.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fecf353c640 1 -- 172.21.15.70:0/990845275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec0b19b0 con 0x7fecec0cb150 2023-10-08T06:54:39.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.499+0000 7fecf353c640 1 -- 172.21.15.70:0/990845275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec0b1f70 con 0x7fecec0cb150 2023-10-08T06:54:39.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.501+0000 7fecf353c640 1 -- 172.21.15.70:0/990845275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec066880 con 0x7fecec0cb150 2023-10-08T06:54:39.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.501+0000 7fece1ffb640 1 -- 172.21.15.70:0/990845275 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fece4014ab0 con 0x7fecec0cb150 2023-10-08T06:54:39.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.502+0000 7fece1ffb640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fecd006af90 0x7fecd006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:39.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.502+0000 7fece1ffb640 1 -- 172.21.15.70:0/990845275 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fece408a340 con 0x7fecec0cb150 2023-10-08T06:54:39.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.503+0000 7fecf12d8640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fecd006af90 0x7fecd006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:39.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.504+0000 7fecf12d8640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fecd006af90 0x7fecd006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fece8007fd0 tx=0x7fece8009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:39.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.507+0000 7fece1ffb640 1 -- 172.21.15.70:0/990845275 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fece4055080 con 0x7fecec0cb150 2023-10-08T06:54:39.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.733+0000 7fecf353c640 1 -- 172.21.15.70:0/990845275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecec067490 con 0x7fecec0cb150 2023-10-08T06:54:39.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.734+0000 7fece1ffb640 1 -- 172.21.15.70:0/990845275 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fece40586a0 con 0x7fecec0cb150 2023-10-08T06:54:39.736 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T06:54:39.736 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/381523318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":381523318},{"type":"v1","addr":"172.21.15.181:6835","nonce":381523318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2153071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2153071509},{"type":"v1","addr":"172.21.15.181:6837","nonce":2153071509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2673497542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2673497542},{"type":"v1","addr":"172.21.15.70:6835","nonce":2673497542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:54:32.491331+0000","modified":"2023-10-08T06:54:35.534957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10074":{"gid":10074,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2138719113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2138719113},{"type":"v1","addr":"172.21.15.70:6833","nonce":2138719113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-08T06:54:39.737 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-10-08T06:54:39.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.737+0000 7fecf353c640 1 -- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fecd006af90 msgr2=0x7fecd006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:39.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.737+0000 7fecf353c640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fecd006af90 0x7fecd006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fece8007fd0 tx=0x7fece8009040 comp rx=0 tx=0).stop 2023-10-08T06:54:39.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.737+0000 7fecf353c640 1 -- 172.21.15.70:0/990845275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec0cb150 msgr2=0x7fecec0af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:39.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.737+0000 7fecf353c640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec0cb150 0x7fecec0af880 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fece4007e60 tx=0x7fece400e4b0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.737+0000 7fecf353c640 1 -- 172.21.15.70:0/990845275 shutdown_connections 2023-10-08T06:54:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.737+0000 7fecf353c640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fecd006af90 0x7fecd006d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.737+0000 7fecf353c640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fecec0cb150 0x7fecec0af880 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.737+0000 7fecf353c640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fecec0add60 0x7fecec1fa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.737+0000 7fecf353c640 1 --2- 172.21.15.70:0/990845275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fecec06d4c0 0x7fecec0b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.737+0000 7fecf353c640 1 -- 172.21.15.70:0/990845275 >> 172.21.15.70:0/990845275 conn(0x7fecec06a380 msgr2=0x7fecec165560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.738+0000 7fecf353c640 1 -- 172.21.15.70:0/990845275 shutdown_connections 2023-10-08T06:54:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:39.738+0000 7fecf353c640 1 -- 172.21.15.70:0/990845275 wait complete. 2023-10-08T06:54:40.160 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T06:54:32.491331+0000', 'modified': '2023-10-08T06:54:35.534957+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10074}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10074': {'gid': 10074, 'name': 'a', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6833/2138719113', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2138719113}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2138719113}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T06:54:40.283 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T06:54:40.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T06:54:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.509+0000 7f6498bb6640 1 Processor -- start 2023-10-08T06:54:40.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.510+0000 7f6498bb6640 1 -- start start 2023-10-08T06:54:40.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.511+0000 7f6498bb6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f649415ce10 0x7f64940ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.511+0000 7f6498bb6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f64940cf3e0 0x7f649406b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.511+0000 7f6498bb6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f649406bab0 0x7f64940b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.511+0000 7f6498bb6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64940cf840 con 0x7f649415ce10 2023-10-08T06:54:40.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.511+0000 7f6498bb6640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f649406bf40 con 0x7f64940cf3e0 2023-10-08T06:54:40.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.511+0000 7f6498bb6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649406cd90 con 0x7f649406bab0 2023-10-08T06:54:40.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f649259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f649415ce10 0x7f64940ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f649259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f649415ce10 0x7f64940ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60774/0 (socket says 172.21.15.70:60774) 2023-10-08T06:54:40.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f649259c640 1 -- 172.21.15.70:0/3591655392 learned_addr learned my addr 172.21.15.70:0/3591655392 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f648bfff640 1 --2- 172.21.15.70:0/3591655392 >> v2:172.21.15.181:3300/0 conn(0x7f64940cf3e0 0x7f649406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f649259c640 1 -- 172.21.15.70:0/3591655392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f649406bab0 msgr2=0x7f64940b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f6492d9d640 1 --2- 172.21.15.70:0/3591655392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f649406bab0 0x7f64940b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f649259c640 1 --2- 172.21.15.70:0/3591655392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f649406bab0 0x7f64940b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f649259c640 1 -- 172.21.15.70:0/3591655392 >> v2:172.21.15.181:3300/0 conn(0x7f64940cf3e0 msgr2=0x7f649406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f649259c640 1 --2- 172.21.15.70:0/3591655392 >> v2:172.21.15.181:3300/0 conn(0x7f64940cf3e0 0x7f649406b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f649259c640 1 -- 172.21.15.70:0/3591655392 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6494160ca0 con 0x7f649415ce10 2023-10-08T06:54:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f648bfff640 1 --2- 172.21.15.70:0/3591655392 >> v2:172.21.15.181:3300/0 conn(0x7f64940cf3e0 0x7f649406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.512+0000 7f6492d9d640 1 --2- 172.21.15.70:0/3591655392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f649406bab0 0x7f64940b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:40.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.513+0000 7f649259c640 1 --2- 172.21.15.70:0/3591655392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 0x7f64940ceea0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f64940c4380 tx=0x7f648402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdc70878551404b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:40.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.514+0000 7f6491d9b640 1 -- 172.21.15.70:0/3591655392 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840317e0 con 0x7f649415ce10 2023-10-08T06:54:40.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.514+0000 7f6491d9b640 1 -- 172.21.15.70:0/3591655392 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6484031940 con 0x7f649415ce10 2023-10-08T06:54:40.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.514+0000 7f6491d9b640 1 -- 172.21.15.70:0/3591655392 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484031cd0 con 0x7f649415ce10 2023-10-08T06:54:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.515+0000 7f6498bb6640 1 -- 172.21.15.70:0/3591655392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 msgr2=0x7f64940ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.515+0000 7f6498bb6640 1 --2- 172.21.15.70:0/3591655392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 0x7f64940ceea0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f64940c4380 tx=0x7f648402b880 comp rx=0 tx=0).stop 2023-10-08T06:54:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.515+0000 7f6498bb6640 1 -- 172.21.15.70:0/3591655392 shutdown_connections 2023-10-08T06:54:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.515+0000 7f6498bb6640 1 --2- 172.21.15.70:0/3591655392 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f649406bab0 0x7f64940b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.515+0000 7f6498bb6640 1 --2- 172.21.15.70:0/3591655392 >> v2:172.21.15.181:3300/0 conn(0x7f64940cf3e0 0x7f649406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.515+0000 7f6498bb6640 1 --2- 172.21.15.70:0/3591655392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 0x7f64940ceea0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.515+0000 7f6498bb6640 1 -- 172.21.15.70:0/3591655392 >> 172.21.15.70:0/3591655392 conn(0x7f649406e630 msgr2=0x7f649406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.515+0000 7f6498bb6640 1 -- 172.21.15.70:0/3591655392 shutdown_connections 2023-10-08T06:54:40.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.515+0000 7f6498bb6640 1 -- 172.21.15.70:0/3591655392 wait complete. 2023-10-08T06:54:40.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.517+0000 7f6498bb6640 1 Processor -- start 2023-10-08T06:54:40.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.517+0000 7f6498bb6640 1 -- start start 2023-10-08T06:54:40.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6498bb6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649406bab0 0x7f64940c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6498bb6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64940cf3e0 0x7f649414bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6498bb6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 0x7f64940b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6498bb6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649414c4e0 con 0x7f649415ce10 2023-10-08T06:54:40.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6498bb6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64940b5a50 con 0x7f649406bab0 2023-10-08T06:54:40.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6498bb6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64940b6420 con 0x7f64940cf3e0 2023-10-08T06:54:40.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6492d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 0x7f64940b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6492d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 0x7f64940b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60786/0 (socket says 172.21.15.70:60786) 2023-10-08T06:54:40.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6492d9d640 1 -- 172.21.15.70:0/3047005286 learned_addr learned my addr 172.21.15.70:0/3047005286 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:40.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f648bfff640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64940cf3e0 0x7f649414bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6492d9d640 1 -- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64940cf3e0 msgr2=0x7f649414bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f649259c640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649406bab0 0x7f64940c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6492d9d640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64940cf3e0 0x7f649414bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6492d9d640 1 -- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649406bab0 msgr2=0x7f64940c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6492d9d640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649406bab0 0x7f64940c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.518+0000 7f6492d9d640 1 -- 172.21.15.70:0/3047005286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64840094d0 con 0x7f649415ce10 2023-10-08T06:54:40.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.519+0000 7f6492d9d640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 0x7f64940b5ee0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f648c00cd10 tx=0x7f648c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:40.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.519+0000 7f648affd640 1 -- 172.21.15.70:0/3047005286 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00f680 con 0x7f649415ce10 2023-10-08T06:54:40.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.519+0000 7f648affd640 1 -- 172.21.15.70:0/3047005286 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c00f800 con 0x7f649415ce10 2023-10-08T06:54:40.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.519+0000 7f648affd640 1 -- 172.21.15.70:0/3047005286 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00fbb0 con 0x7f649415ce10 2023-10-08T06:54:40.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.519+0000 7f6498bb6640 1 -- 172.21.15.70:0/3047005286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64940b4e40 con 0x7f649415ce10 2023-10-08T06:54:40.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.519+0000 7f6498bb6640 1 -- 172.21.15.70:0/3047005286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64940b5400 con 0x7f649415ce10 2023-10-08T06:54:40.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.522+0000 7f648affd640 1 -- 172.21.15.70:0/3047005286 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f648c00b040 con 0x7f649415ce10 2023-10-08T06:54:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.522+0000 7f648affd640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f647406af90 0x7f647406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.523+0000 7f648affd640 1 -- 172.21.15.70:0/3047005286 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f648c08a450 con 0x7f649415ce10 2023-10-08T06:54:40.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.523+0000 7f649259c640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f647406af90 0x7f647406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.523+0000 7f649b822740 1 Processor -- start 2023-10-08T06:54:40.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.524+0000 7f649b822740 1 -- start start 2023-10-08T06:54:40.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.524+0000 7f649259c640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f647406af90 0x7f647406d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6484036000 tx=0x7f648403d040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:40.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.524+0000 7f649b822740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdb60 0x55f8844a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.524+0000 7f649b822740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdf40 0x55f8844d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.524+0000 7f649b822740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844a0fa0 0x55f8844d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.524+0000 7f649b822740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884414700 con 0x55f8844a0fa0 2023-10-08T06:54:40.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.524+0000 7f649b822740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884411df0 con 0x55f8844cdf40 2023-10-08T06:54:40.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.524+0000 7f649b822740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f88440d6e0 con 0x55f8844cdb60 2023-10-08T06:54:40.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f6492d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844a0fa0 0x55f8844d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f6492d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844a0fa0 0x55f8844d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60790/0 (socket says 172.21.15.70:60790) 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f6492d9d640 1 -- 172.21.15.70:0/3232838752 learned_addr learned my addr 172.21.15.70:0/3232838752 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f649259c640 1 --2- 172.21.15.70:0/3232838752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdb60 0x55f8844a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f6492d9d640 1 -- 172.21.15.70:0/3232838752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdb60 msgr2=0x55f8844a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f648bfff640 1 --2- 172.21.15.70:0/3232838752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdf40 0x55f8844d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f6492d9d640 1 --2- 172.21.15.70:0/3232838752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdb60 0x55f8844a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f6492d9d640 1 -- 172.21.15.70:0/3232838752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdf40 msgr2=0x55f8844d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f6492d9d640 1 --2- 172.21.15.70:0/3232838752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdf40 0x55f8844d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f6492d9d640 1 -- 172.21.15.70:0/3232838752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8843d1910 con 0x55f8844a0fa0 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f649259c640 1 --2- 172.21.15.70:0/3232838752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdb60 0x55f8844a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.525+0000 7f6492d9d640 1 --2- 172.21.15.70:0/3232838752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844a0fa0 0x55f8844d0e30 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f648c060bb0 tx=0x7f648c058de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0379957e3a703e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:40.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.526+0000 7f6473fff640 1 -- 172.21.15.70:0/3232838752 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c05f070 con 0x55f8844a0fa0 2023-10-08T06:54:40.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.526+0000 7f6473fff640 1 -- 172.21.15.70:0/3232838752 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c057180 con 0x55f8844a0fa0 2023-10-08T06:54:40.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.526+0000 7f6473fff640 1 -- 172.21.15.70:0/3232838752 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c0887b0 con 0x55f8844a0fa0 2023-10-08T06:54:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.526+0000 7f649b822740 1 -- 172.21.15.70:0/3232838752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844a0fa0 msgr2=0x55f8844d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.526+0000 7f649b822740 1 --2- 172.21.15.70:0/3232838752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844a0fa0 0x55f8844d0e30 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f648c060bb0 tx=0x7f648c058de0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.526+0000 7f649b822740 1 -- 172.21.15.70:0/3232838752 shutdown_connections 2023-10-08T06:54:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.526+0000 7f649b822740 1 --2- 172.21.15.70:0/3232838752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844a0fa0 0x55f8844d0e30 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.526+0000 7f649b822740 1 --2- 172.21.15.70:0/3232838752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdf40 0x55f8844d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.526+0000 7f649b822740 1 --2- 172.21.15.70:0/3232838752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdb60 0x55f8844a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.526+0000 7f649b822740 1 -- 172.21.15.70:0/3232838752 >> 172.21.15.70:0/3232838752 conn(0x55f8844a2d70 msgr2=0x55f8844eb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:40.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.527+0000 7f649b822740 1 -- 172.21.15.70:0/3232838752 shutdown_connections 2023-10-08T06:54:40.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.527+0000 7f649b822740 1 -- 172.21.15.70:0/3232838752 wait complete. 2023-10-08T06:54:40.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.527+0000 7f649b822740 1 -- start start 2023-10-08T06:54:40.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.528+0000 7f649b822740 1 Processor -- start 2023-10-08T06:54:40.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.528+0000 7f649b822740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 0x55f8844bdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f649b822740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f8844eee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f649b822740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844fe960 0x55f8844ef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f649b822740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8843ab970 con 0x55f8844cdf40 2023-10-08T06:54:40.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f649b822740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8843ab610 con 0x55f8844cdb60 2023-10-08T06:54:40.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f649b822740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f884171f20 con 0x55f8844fe960 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f649b822740 10 client.0 _ll_register_callbacks cb 0x55f8843e9610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f648bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f8844eee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f648bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f8844eee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60798/0 (socket says 172.21.15.70:60798) 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f648bfff640 1 -- 172.21.15.70:0/1069211869 learned_addr learned my addr 172.21.15.70:0/1069211869 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f648bfff640 1 -- 172.21.15.70:0/1069211869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844fe960 msgr2=0x55f8844ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f648bfff640 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844fe960 0x55f8844ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f648bfff640 1 -- 172.21.15.70:0/1069211869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 msgr2=0x55f8844bdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f649259c640 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 0x55f8844bdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f648bfff640 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 0x55f8844bdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f648bfff640 1 -- 172.21.15.70:0/1069211869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64940b5400 con 0x55f8844cdf40 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.529+0000 7f649259c640 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 0x55f8844bdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:40.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.530+0000 7f648bfff640 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f8844eee00 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f647c00e310 tx=0x7f647c00e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d73e754931fed3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.530+0000 7f64717fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T06:54:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.530+0000 7f64717fa640 1 -- 172.21.15.70:0/1069211869 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c012840 con 0x55f8844cdf40 2023-10-08T06:54:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.530+0000 7f64717fa640 1 -- 172.21.15.70:0/1069211869 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f647c00b030 con 0x55f8844cdf40 2023-10-08T06:54:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.530+0000 7f649b822740 1 -- 172.21.15.70:0/1069211869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8844066c0 con 0x55f8844cdf40 2023-10-08T06:54:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.530+0000 7f64717fa640 1 -- 172.21.15.70:0/1069211869 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f647c0129a0 con 0x55f8844cdf40 2023-10-08T06:54:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.530+0000 7f649b822740 10 client.10215 fetch_fsmap learned FSMap version 91 2023-10-08T06:54:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.530+0000 7f649b822740 1 -- 172.21.15.70:0/1069211869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55f88440bd60 con 0x55f8844cdf40 2023-10-08T06:54:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.530+0000 7f64717fa640 1 -- 172.21.15.70:0/1069211869 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f8844066c0 con 0x55f8844cdf40 2023-10-08T06:54:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.531+0000 7f64717fa640 1 -- 172.21.15.70:0/1069211869 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f647c018020 con 0x55f8844cdf40 2023-10-08T06:54:40.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.531+0000 7f649b822740 10 client.10215 fetch_fsmap finished waiting for FSMap version 91 2023-10-08T06:54:40.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.531+0000 7f649b822740 10 client.10215 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T06:54:40.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.531+0000 7f649b822740 20 client.10215 populate_metadata read hostname 'smithi070' 2023-10-08T06:54:40.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.531+0000 7f649b822740 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.70:6832/2138719113,v1:172.21.15.70:6833/2138719113] conn(0x55f8844d34e0 0x55f8844ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.531+0000 7f649b822740 4 client.10215 mds_command: new command op to 10074 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T06:54:40.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.531+0000 7f649b822740 1 -- 172.21.15.70:0/1069211869 --> [v2:172.21.15.70:6832/2138719113,v1:172.21.15.70:6833/2138719113] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8844063e0 con 0x55f8844d34e0 2023-10-08T06:54:40.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.531+0000 7f649259c640 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.70:6832/2138719113,v1:172.21.15.70:6833/2138719113] conn(0x55f8844d34e0 0x55f8844ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.531+0000 7f649259c640 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.70:6832/2138719113,v1:172.21.15.70:6833/2138719113] conn(0x55f8844d34e0 0x55f8844ca1c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b4994f68a8013c server_cookie=75a08cc5287340d8 in_seq=0 out_seq=0 2023-10-08T06:54:40.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.531+0000 7f64717fa640 10 client.10215 ms_handle_connect on v2:172.21.15.70:6832/2138719113 2023-10-08T06:54:40.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.532+0000 7f64717fa640 1 -- 172.21.15.70:0/1069211869 <== mds.0 v2:172.21.15.70:6832/2138719113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f8844063e0 con 0x55f8844d34e0 2023-10-08T06:54:40.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.532+0000 7f64717fa640 10 client.10215 handle_command_reply: tid=0 2023-10-08T06:54:40.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.532+0000 7f649b822740 1 client.10215 shutdown 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.532+0000 7f649b822740 1 -- 172.21.15.70:0/1069211869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 msgr2=0x55f8844eee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f649b822740 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f8844eee00 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f647c00e310 tx=0x7f647c00e7d0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f649b822740 1 -- 172.21.15.70:0/1069211869 shutdown_connections 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f649b822740 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.70:6832/2138719113,v1:172.21.15.70:6833/2138719113] conn(0x55f8844d34e0 0x55f8844ca1c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f649b822740 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844fe960 0x55f8844ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f649b822740 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f8844eee00 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f649b822740 1 --2- 172.21.15.70:0/1069211869 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 0x55f8844bdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f64717fa640 0 client.10215 ms_handle_reset on v2:172.21.15.70:6832/2138719113 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f649b822740 1 -- 172.21.15.70:0/1069211869 >> 172.21.15.70:0/1069211869 conn(0x55f8844a0fa0 msgr2=0x55f8844a49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f649b822740 1 -- 172.21.15.70:0/1069211869 shutdown_connections 2023-10-08T06:54:40.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f649b822740 1 -- 172.21.15.70:0/1069211869 wait complete. 2023-10-08T06:54:40.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.533+0000 7f649b822740 20 client.10215 trim_cache size 0 max 16384 2023-10-08T06:54:40.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.550+0000 7f649b822740 1 Processor -- start 2023-10-08T06:54:40.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.550+0000 7f649b822740 1 -- start start 2023-10-08T06:54:40.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f649b822740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdb60 0x55f8844d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f649b822740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f88449fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f649b822740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844fe960 0x55f88449ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f649b822740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8844a5660 con 0x55f8844cdf40 2023-10-08T06:54:40.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f649b822740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8844c9790 con 0x55f8844fe960 2023-10-08T06:54:40.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f649b822740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8844ebf40 con 0x55f8844cdb60 2023-10-08T06:54:40.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f648bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f88449fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f648bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f88449fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60808/0 (socket says 172.21.15.70:60808) 2023-10-08T06:54:40.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f648bfff640 1 -- 172.21.15.70:0/1266795672 learned_addr learned my addr 172.21.15.70:0/1266795672 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:40.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f648bfff640 1 -- 172.21.15.70:0/1266795672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdb60 msgr2=0x55f8844d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f648bfff640 1 --2- 172.21.15.70:0/1266795672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdb60 0x55f8844d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f648bfff640 1 -- 172.21.15.70:0/1266795672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844fe960 msgr2=0x55f88449ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f648bfff640 1 --2- 172.21.15.70:0/1266795672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844fe960 0x55f88449ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.551+0000 7f648bfff640 1 -- 172.21.15.70:0/1266795672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8843aab80 con 0x55f8844cdf40 2023-10-08T06:54:40.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f648bfff640 1 --2- 172.21.15.70:0/1266795672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f88449fa40 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c920 tx=0x7f647c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c83e9739e99f190 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:40.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f6488ff9640 1 -- 172.21.15.70:0/1266795672 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c00ecd0 con 0x55f8844cdf40 2023-10-08T06:54:40.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f6488ff9640 1 -- 172.21.15.70:0/1266795672 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c012640 con 0x55f8844cdf40 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f6488ff9640 1 -- 172.21.15.70:0/1266795672 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c01a400 con 0x55f8844cdf40 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f649b822740 1 -- 172.21.15.70:0/1266795672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 msgr2=0x55f88449fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f649b822740 1 --2- 172.21.15.70:0/1266795672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f88449fa40 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c920 tx=0x7f647c013000 comp rx=0 tx=0).stop 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f649b822740 1 -- 172.21.15.70:0/1266795672 shutdown_connections 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f649b822740 1 --2- 172.21.15.70:0/1266795672 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844fe960 0x55f88449ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f649b822740 1 --2- 172.21.15.70:0/1266795672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844cdf40 0x55f88449fa40 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f649b822740 1 --2- 172.21.15.70:0/1266795672 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdb60 0x55f8844d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.552+0000 7f649b822740 1 -- 172.21.15.70:0/1266795672 >> 172.21.15.70:0/1266795672 conn(0x55f8844a0fa0 msgr2=0x55f8844cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.553+0000 7f649b822740 1 -- 172.21.15.70:0/1266795672 shutdown_connections 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.553+0000 7f649b822740 1 -- 172.21.15.70:0/1266795672 wait complete. 2023-10-08T06:54:40.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.553+0000 7f649b822740 1 -- start start 2023-10-08T06:54:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.553+0000 7f649b822740 1 Processor -- start 2023-10-08T06:54:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.554+0000 7f649b822740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 0x55f88449fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.554+0000 7f649b822740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdf40 0x55f8844e1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.554+0000 7f649b822740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844fe960 0x55f8844bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.554+0000 7f649b822740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f884555720 con 0x55f8844fe960 2023-10-08T06:54:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.554+0000 7f649b822740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8844e1ab0 con 0x55f8844cdb60 2023-10-08T06:54:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.554+0000 7f649b822740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8844a0380 con 0x55f8844cdf40 2023-10-08T06:54:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f649b822740 10 client.0 _ll_register_callbacks cb 0x7f648c0551a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T06:54:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f6492d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844fe960 0x55f8844bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f6492d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844fe960 0x55f8844bcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60810/0 (socket says 172.21.15.70:60810) 2023-10-08T06:54:40.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f6492d9d640 1 -- 172.21.15.70:0/2927453368 learned_addr learned my addr 172.21.15.70:0/2927453368 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:40.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f649259c640 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 0x55f88449fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f648bfff640 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdf40 0x55f8844e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f6492d9d640 1 -- 172.21.15.70:0/2927453368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdf40 msgr2=0x55f8844e1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f6492d9d640 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdf40 0x55f8844e1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f6492d9d640 1 -- 172.21.15.70:0/2927453368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 msgr2=0x55f88449fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f6492d9d640 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 0x55f88449fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f6492d9d640 1 -- 172.21.15.70:0/2927453368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f88440bd60 con 0x55f8844fe960 2023-10-08T06:54:40.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f6492d9d640 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844fe960 0x55f8844bcf20 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f648c058ae0 tx=0x7f648c05cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a1fe97061d3144 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:40.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.555+0000 7f64717fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T06:54:40.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f64717fa640 1 -- 172.21.15.70:0/2927453368 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c0078e0 con 0x55f8844fe960 2023-10-08T06:54:40.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f64717fa640 1 -- 172.21.15.70:0/2927453368 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f648c088de0 con 0x55f8844fe960 2023-10-08T06:54:40.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f649b822740 1 -- 172.21.15.70:0/2927453368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f648c057180 con 0x55f8844fe960 2023-10-08T06:54:40.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f64717fa640 1 -- 172.21.15.70:0/2927453368 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f648c05f070 con 0x55f8844fe960 2023-10-08T06:54:40.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f649b822740 10 client.10221 fetch_fsmap learned FSMap version 91 2023-10-08T06:54:40.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f649b822740 1 -- 172.21.15.70:0/2927453368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x55f8843d0170 con 0x55f8844fe960 2023-10-08T06:54:40.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f64717fa640 1 -- 172.21.15.70:0/2927453368 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c07b080 con 0x55f8844fe960 2023-10-08T06:54:40.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f64717fa640 1 -- 172.21.15.70:0/2927453368 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f648c020070 con 0x55f8844fe960 2023-10-08T06:54:40.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f649b822740 10 client.10221 fetch_fsmap finished waiting for FSMap version 91 2023-10-08T06:54:40.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f649b822740 10 client.10221 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T06:54:40.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f649b822740 20 client.10221 populate_metadata read hostname 'smithi070' 2023-10-08T06:54:40.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f649b822740 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.70:6832/2138719113,v1:172.21.15.70:6833/2138719113] conn(0x55f88454d8d0 0x55f88454dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f649b822740 4 client.10221 mds_command: new command op to 10074 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T06:54:40.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.556+0000 7f649b822740 1 -- 172.21.15.70:0/2927453368 --> [v2:172.21.15.70:6832/2138719113,v1:172.21.15.70:6833/2138719113] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f88410f6c0 con 0x55f88454d8d0 2023-10-08T06:54:40.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.557+0000 7f649259c640 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.70:6832/2138719113,v1:172.21.15.70:6833/2138719113] conn(0x55f88454d8d0 0x55f88454dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.557+0000 7f649259c640 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.70:6832/2138719113,v1:172.21.15.70:6833/2138719113] conn(0x55f88454d8d0 0x55f88454dcb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f235c3d886c630aa server_cookie=5f2b130f0d35d933 in_seq=0 out_seq=0 2023-10-08T06:54:40.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.557+0000 7f64717fa640 10 client.10221 ms_handle_connect on v2:172.21.15.70:6832/2138719113 2023-10-08T06:54:40.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.557+0000 7f64717fa640 1 -- 172.21.15.70:0/2927453368 <== mds.0 v2:172.21.15.70:6832/2138719113 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55f88410f6c0 con 0x55f88454d8d0 2023-10-08T06:54:40.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.557+0000 7f64717fa640 10 client.10221 handle_command_reply: tid=0 2023-10-08T06:54:40.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.557+0000 7f649b822740 1 client.10221 shutdown 2023-10-08T06:54:40.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.557+0000 7f649b822740 1 -- 172.21.15.70:0/2927453368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844fe960 msgr2=0x55f8844bcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.557+0000 7f649b822740 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844fe960 0x55f8844bcf20 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f648c058ae0 tx=0x7f648c05cfb0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f649b822740 1 -- 172.21.15.70:0/2927453368 shutdown_connections 2023-10-08T06:54:40.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f649b822740 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.70:6832/2138719113,v1:172.21.15.70:6833/2138719113] conn(0x55f88454d8d0 0x55f88454dcb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f649b822740 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f8844fe960 0x55f8844bcf20 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f649b822740 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f8844cdf40 0x55f8844e1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f649b822740 1 --2- 172.21.15.70:0/2927453368 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f8844cdb60 0x55f88449fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f64717fa640 0 client.10221 ms_handle_reset on v2:172.21.15.70:6832/2138719113 2023-10-08T06:54:40.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f649b822740 1 -- 172.21.15.70:0/2927453368 >> 172.21.15.70:0/2927453368 conn(0x55f8844a0fa0 msgr2=0x55f8844a6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:40.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f649b822740 1 -- 172.21.15.70:0/2927453368 shutdown_connections 2023-10-08T06:54:40.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f649b822740 1 -- 172.21.15.70:0/2927453368 wait complete. 2023-10-08T06:54:40.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f649b822740 20 client.10221 trim_cache size 0 max 16384 2023-10-08T06:54:40.627 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T06:54:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T06:54:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T06:54:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 10074, 2023-10-08T06:54:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T06:54:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T06:54:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T06:54:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 7.5175844469999999, 2023-10-08T06:54:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 91, 2023-10-08T06:54:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 198, 2023-10-08T06:54:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 198, 2023-10-08T06:54:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 12.930512308999999 2023-10-08T06:54:40.629 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T06:54:40.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f6498bb6640 1 -- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f647406af90 msgr2=0x7f647406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f6498bb6640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f647406af90 0x7f647406d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6484036000 tx=0x7f648403d040 comp rx=0 tx=0).stop 2023-10-08T06:54:40.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f6498bb6640 1 -- 172.21.15.70:0/3047005286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 msgr2=0x7f64940b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.558+0000 7f6498bb6640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 0x7f64940b5ee0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f648c00cd10 tx=0x7f648c00a650 comp rx=0 tx=0).stop 2023-10-08T06:54:40.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.559+0000 7f6498bb6640 1 -- 172.21.15.70:0/3047005286 shutdown_connections 2023-10-08T06:54:40.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.559+0000 7f6498bb6640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f647406af90 0x7f647406d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.559+0000 7f6498bb6640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f649415ce10 0x7f64940b5ee0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.559+0000 7f6498bb6640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64940cf3e0 0x7f649414bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.559+0000 7f6498bb6640 1 --2- 172.21.15.70:0/3047005286 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f649406bab0 0x7f64940c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.559+0000 7f6498bb6640 1 -- 172.21.15.70:0/3047005286 >> 172.21.15.70:0/3047005286 conn(0x7f649406e630 msgr2=0x7f649415f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:40.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.559+0000 7f6498bb6640 1 -- 172.21.15.70:0/3047005286 shutdown_connections 2023-10-08T06:54:40.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.559+0000 7f6498bb6640 1 -- 172.21.15.70:0/3047005286 wait complete. 2023-10-08T06:54:40.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T06:54:40.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.856+0000 7f13397de640 1 Processor -- start 2023-10-08T06:54:40.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.857+0000 7f13397de640 1 -- start start 2023-10-08T06:54:40.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.858+0000 7f13397de640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f133415a970 0x7f13340cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.858+0000 7f13397de640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f133406dd00 0x7f13340cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.858+0000 7f13397de640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13340ca9e0 con 0x7f13340c9e60 2023-10-08T06:54:40.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.858+0000 7f13397de640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133406cfd0 con 0x7f133406dd00 2023-10-08T06:54:40.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.858+0000 7f13397de640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133406d870 con 0x7f133415a970 2023-10-08T06:54:40.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.858+0000 7f13337fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f133406dd00 0x7f13340cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.858+0000 7f13337fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f133406dd00 0x7f13340cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60826/0 (socket says 172.21.15.70:60826) 2023-10-08T06:54:40.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.858+0000 7f13337fe640 1 -- 172.21.15.70:0/1358619604 learned_addr learned my addr 172.21.15.70:0/1358619604 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:40.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.859+0000 7f1333fff640 1 --2- 172.21.15.70:0/1358619604 >> v2:172.21.15.181:3300/0 conn(0x7f133415a970 0x7f13340cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.859+0000 7f13337fe640 1 -- 172.21.15.70:0/1358619604 >> v2:172.21.15.181:3300/0 conn(0x7f133415a970 msgr2=0x7f13340cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.859+0000 7f13337fe640 1 --2- 172.21.15.70:0/1358619604 >> v2:172.21.15.181:3300/0 conn(0x7f133415a970 0x7f13340cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.859+0000 7f13337fe640 1 -- 172.21.15.70:0/1358619604 >> v1:172.21.15.181:6789/0 conn(0x7f13340c9e60 legacy=0x7f13340ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.859+0000 7f13337fe640 1 -- 172.21.15.70:0/1358619604 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133415ad50 con 0x7f133406dd00 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.860+0000 7f13337fe640 1 --2- 172.21.15.70:0/1358619604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 0x7f13340cb990 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f13340cc130 tx=0x7f131c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54350e6d7de5ee5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.860+0000 7f1332ffd640 1 -- 172.21.15.70:0/1358619604 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c031750 con 0x7f133406dd00 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.860+0000 7f1332ffd640 1 -- 172.21.15.70:0/1358619604 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131c0318b0 con 0x7f133406dd00 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.860+0000 7f1332ffd640 1 -- 172.21.15.70:0/1358619604 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c031c40 con 0x7f133406dd00 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.861+0000 7f13397de640 1 -- 172.21.15.70:0/1358619604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 msgr2=0x7f13340cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.861+0000 7f13397de640 1 --2- 172.21.15.70:0/1358619604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 0x7f13340cb990 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f13340cc130 tx=0x7f131c02b7f0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.861+0000 7f13397de640 1 -- 172.21.15.70:0/1358619604 shutdown_connections 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.861+0000 7f13397de640 1 --2- 172.21.15.70:0/1358619604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 0x7f13340cb990 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.861+0000 7f13397de640 1 --2- 172.21.15.70:0/1358619604 >> v2:172.21.15.181:3300/0 conn(0x7f133415a970 0x7f13340cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.861+0000 7f13397de640 1 -- 172.21.15.70:0/1358619604 >> 172.21.15.70:0/1358619604 conn(0x7f13340ce9d0 msgr2=0x7f13340cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:40.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.861+0000 7f13397de640 1 -- 172.21.15.70:0/1358619604 shutdown_connections 2023-10-08T06:54:40.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.861+0000 7f13397de640 1 -- 172.21.15.70:0/1358619604 wait complete. 2023-10-08T06:54:40.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.863+0000 7f13397de640 1 Processor -- start 2023-10-08T06:54:40.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.863+0000 7f13397de640 1 -- start start 2023-10-08T06:54:40.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f13397de640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 0x7f13340ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f13397de640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13340c9e60 0x7f133415c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f13397de640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f133415a970 0x7f13340c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f13397de640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13340afa10 con 0x7f133406dd00 2023-10-08T06:54:40.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f13397de640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13341f1130 con 0x7f13340c9e60 2023-10-08T06:54:40.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f13397de640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13340ae910 con 0x7f133415a970 2023-10-08T06:54:40.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f1333fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 0x7f13340ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f1333fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 0x7f13340ae010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60842/0 (socket says 172.21.15.70:60842) 2023-10-08T06:54:40.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f1333fff640 1 -- 172.21.15.70:0/2334908559 learned_addr learned my addr 172.21.15.70:0/2334908559 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:40.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f1333fff640 1 -- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f133415a970 msgr2=0x7f13340c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f1333fff640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f133415a970 0x7f13340c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f1333fff640 1 -- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13340c9e60 msgr2=0x7f133415c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:40.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f1333fff640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13340c9e60 0x7f133415c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:40.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.864+0000 7f1333fff640 1 -- 172.21.15.70:0/2334908559 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c0094d0 con 0x7f133406dd00 2023-10-08T06:54:40.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.865+0000 7f1333fff640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 0x7f13340ae010 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f132400ce10 tx=0x7f132400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.865+0000 7f1330ff9640 1 -- 172.21.15.70:0/2334908559 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324003b90 con 0x7f133406dd00 2023-10-08T06:54:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.865+0000 7f1330ff9640 1 -- 172.21.15.70:0/2334908559 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1324003cf0 con 0x7f133406dd00 2023-10-08T06:54:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.865+0000 7f1330ff9640 1 -- 172.21.15.70:0/2334908559 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324003ea0 con 0x7f133406dd00 2023-10-08T06:54:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.865+0000 7f13397de640 1 -- 172.21.15.70:0/2334908559 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13340aebf0 con 0x7f133406dd00 2023-10-08T06:54:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.865+0000 7f13397de640 1 -- 172.21.15.70:0/2334908559 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13340af5a0 con 0x7f133406dd00 2023-10-08T06:54:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.866+0000 7f13397de640 1 -- 172.21.15.70:0/2334908559 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133404e160 con 0x7f133406dd00 2023-10-08T06:54:40.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.867+0000 7f1330ff9640 1 -- 172.21.15.70:0/2334908559 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1324004240 con 0x7f133406dd00 2023-10-08T06:54:40.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.868+0000 7f1330ff9640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f131406af90 0x7f131406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:40.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.869+0000 7f1330ff9640 1 -- 172.21.15.70:0/2334908559 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f1324012070 con 0x7f133406dd00 2023-10-08T06:54:40.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.869+0000 7f13337fe640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f131406af90 0x7f131406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:40.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.870+0000 7f13337fe640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f131406af90 0x7f131406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f131c035000 tx=0x7f131c030000 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:40.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:40.872+0000 7f1330ff9640 1 -- 172.21.15.70:0/2334908559 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1324055150 con 0x7f133406dd00 2023-10-08T06:54:41.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.107+0000 7f13397de640 1 -- 172.21.15.70:0/2334908559 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f13340ab540 con 0x7f133406dd00 2023-10-08T06:54:41.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.108+0000 7f1330ff9640 1 -- 172.21.15.70:0/2334908559 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1324058770 con 0x7f133406dd00 2023-10-08T06:54:41.110 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T06:54:41.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 -- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f131406af90 msgr2=0x7f131406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:41.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f131406af90 0x7f131406d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f131c035000 tx=0x7f131c030000 comp rx=0 tx=0).stop 2023-10-08T06:54:41.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 -- 172.21.15.70:0/2334908559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 msgr2=0x7f13340ae010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:41.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 0x7f13340ae010 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f132400ce10 tx=0x7f132400a660 comp rx=0 tx=0).stop 2023-10-08T06:54:41.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 -- 172.21.15.70:0/2334908559 shutdown_connections 2023-10-08T06:54:41.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f131406af90 0x7f131406d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:41.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f133415a970 0x7f13340c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:41.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13340c9e60 0x7f133415c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:41.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 --2- 172.21.15.70:0/2334908559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f133406dd00 0x7f13340ae010 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:41.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 -- 172.21.15.70:0/2334908559 >> 172.21.15.70:0/2334908559 conn(0x7f13340ce9d0 msgr2=0x7f1334070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:41.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 -- 172.21.15.70:0/2334908559 shutdown_connections 2023-10-08T06:54:41.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:41.111+0000 7f13397de640 1 -- 172.21.15.70:0/2334908559 wait complete. 2023-10-08T06:54:41.131 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T06:54:41.131 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T06:54:41.131 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T06:54:41.131 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T06:54:41.132 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T06:54:41.132 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T06:54:41.132 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T06:54:41.132 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T06:54:41.132 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T06:54:41.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:41.133 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T06:54:41.184 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T06:54:41.184 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T06:54:41.184 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T06:54:41.184 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:41.184 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T06:54:41.185 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 37715sec preferred_lft 37715sec 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:41.186 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T06:54:41.187 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T06:54:41.187 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T06:54:41.187 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:41.187 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T06:54:41.187 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T06:54:41.187 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T06:54:41.187 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T06:54:41.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:41.188 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T06:54:41.279 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T06:54:41.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:41.283 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T06:54:41.336 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T06:54:41.337 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T06:54:41.337 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T06:54:41.337 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T06:54:41.337 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T06:54:41.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:41.337 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T06:54:41.337 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T06:54:41.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T06:54:41.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T06:54:41.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T06:54:41.338 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T06:54:41.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:41.526 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T06:54:41.577 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T06:54:41.577 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T06:54:41.578 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T06:54:41.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:41.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T06:54:41.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T06:54:41.665 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T06:54:41.716 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T06:54:41.716 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T06:54:41.716 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T06:54:41.768 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T06:54:41.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:41.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T06:54:41.905 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T06:54:41.905 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T06:54:41.936 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T06:54:41.936 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T06:54:41.936 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T06:54:41.937 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T06:54:41.937 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T06:54:41.938 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T06:54:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T06:54:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T06:54:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T06:54:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T06:54:41.942 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T06:54:41.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:41.942 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T06:54:42.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:42.021 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T06:54:42.085 INFO:teuthology.orchestra.run.smithi070.stdout:client6962 2023-10-08T06:54:42.085 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T06:54:42.085 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T06:54:42.138 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T06:54:42.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.346+0000 7f32e9010640 1 Processor -- start 2023-10-08T06:54:42.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.347+0000 7f32e9010640 1 -- start start 2023-10-08T06:54:42.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.348+0000 7f32e9010640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 0x7f32e4070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.348+0000 7f32e9010640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32e40cb620 0x7f32e40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.348+0000 7f32e9010640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e406e150 con 0x7f32e40c9910 2023-10-08T06:54:42.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.348+0000 7f32e9010640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e406de70 con 0x7f32e40cb620 2023-10-08T06:54:42.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.348+0000 7f32e9010640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40c8e30 con 0x7f32e4160660 2023-10-08T06:54:42.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.349+0000 7f32e2d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 0x7f32e4070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.349+0000 7f32e359e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f32e40cb620 0x7f32e40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.349+0000 7f32e2d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 0x7f32e4070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53200/0 (socket says 172.21.15.70:53200) 2023-10-08T06:54:42.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.349+0000 7f32e2d9d640 1 -- 172.21.15.70:0/1481877635 learned_addr learned my addr 172.21.15.70:0/1481877635 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:42.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.350+0000 7f32e2d9d640 1 -- 172.21.15.70:0/1481877635 >> v2:172.21.15.181:3300/0 conn(0x7f32e40cb620 msgr2=0x7f32e40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:42.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.350+0000 7f32e2d9d640 1 --2- 172.21.15.70:0/1481877635 >> v2:172.21.15.181:3300/0 conn(0x7f32e40cb620 0x7f32e40b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.350+0000 7f32e2d9d640 1 -- 172.21.15.70:0/1481877635 >> v1:172.21.15.181:6789/0 conn(0x7f32e40c9910 legacy=0x7f32e40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.350+0000 7f32e2d9d640 1 -- 172.21.15.70:0/1481877635 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e40c9110 con 0x7f32e4160660 2023-10-08T06:54:42.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.351+0000 7f32e2d9d640 1 --2- 172.21.15.70:0/1481877635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 0x7f32e4070cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009870 tx=0x7f32d402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3310dd686461c36 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:42.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.351+0000 7f32e259c640 1 -- 172.21.15.70:0/1481877635 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4030840 con 0x7f32e4160660 2023-10-08T06:54:42.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.351+0000 7f32e259c640 1 -- 172.21.15.70:0/1481877635 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d40309a0 con 0x7f32e4160660 2023-10-08T06:54:42.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.351+0000 7f32e259c640 1 -- 172.21.15.70:0/1481877635 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4030d30 con 0x7f32e4160660 2023-10-08T06:54:42.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.352+0000 7f32e9010640 1 -- 172.21.15.70:0/1481877635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 msgr2=0x7f32e4070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.352+0000 7f32e9010640 1 --2- 172.21.15.70:0/1481877635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 0x7f32e4070cc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009870 tx=0x7f32d402b980 comp rx=0 tx=0).stop 2023-10-08T06:54:42.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.352+0000 7f32e9010640 1 -- 172.21.15.70:0/1481877635 shutdown_connections 2023-10-08T06:54:42.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.352+0000 7f32e9010640 1 --2- 172.21.15.70:0/1481877635 >> v2:172.21.15.181:3300/0 conn(0x7f32e40cb620 0x7f32e40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.352+0000 7f32e9010640 1 --2- 172.21.15.70:0/1481877635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 0x7f32e4070cc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.352+0000 7f32e9010640 1 -- 172.21.15.70:0/1481877635 >> 172.21.15.70:0/1481877635 conn(0x7f32e406a5d0 msgr2=0x7f32e40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:42.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.352+0000 7f32e9010640 1 -- 172.21.15.70:0/1481877635 shutdown_connections 2023-10-08T06:54:42.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.352+0000 7f32e9010640 1 -- 172.21.15.70:0/1481877635 wait complete. 2023-10-08T06:54:42.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.355+0000 7f32e9010640 1 Processor -- start 2023-10-08T06:54:42.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.355+0000 7f32e9010640 1 -- start start 2023-10-08T06:54:42.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.356+0000 7f32e9010640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e40c9910 0x7f32e414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.356+0000 7f32e9010640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e40cb620 0x7f32e41f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.356+0000 7f32e9010640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 0x7f32e40c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.356+0000 7f32e9010640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e41f2620 con 0x7f32e40c9910 2023-10-08T06:54:42.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.356+0000 7f32e9010640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e414c3f0 con 0x7f32e40cb620 2023-10-08T06:54:42.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.356+0000 7f32e9010640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40c41f0 con 0x7f32e4160660 2023-10-08T06:54:42.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.356+0000 7f32e2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e40c9910 0x7f32e414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.356+0000 7f32e2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e40c9910 0x7f32e414f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60852/0 (socket says 172.21.15.70:60852) 2023-10-08T06:54:42.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.356+0000 7f32e2d9d640 1 -- 172.21.15.70:0/626497487 learned_addr learned my addr 172.21.15.70:0/626497487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:42.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32dbfff640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e40cb620 0x7f32e41f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32e359e640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 0x7f32e40c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32e2d9d640 1 -- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 msgr2=0x7f32e40c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32e2d9d640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 0x7f32e40c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32e2d9d640 1 -- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e40cb620 msgr2=0x7f32e41f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32e2d9d640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e40cb620 0x7f32e41f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32e2d9d640 1 -- 172.21.15.70:0/626497487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d40094d0 con 0x7f32e40c9910 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32e2d9d640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e40c9910 0x7f32e414f020 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f32d402b9c0 tx=0x7f32d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32db7fe640 1 -- 172.21.15.70:0/626497487 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d403e070 con 0x7f32e40c9910 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32db7fe640 1 -- 172.21.15.70:0/626497487 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d4039a10 con 0x7f32e40c9910 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.357+0000 7f32db7fe640 1 -- 172.21.15.70:0/626497487 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4039de0 con 0x7f32e40c9910 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.358+0000 7f32e9010640 1 -- 172.21.15.70:0/626497487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e40c4450 con 0x7f32e40c9910 2023-10-08T06:54:42.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.358+0000 7f32e9010640 1 -- 172.21.15.70:0/626497487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e40c4a10 con 0x7f32e40c9910 2023-10-08T06:54:42.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.359+0000 7f32e9010640 1 -- 172.21.15.70:0/626497487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e404e160 con 0x7f32e40c9910 2023-10-08T06:54:42.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.360+0000 7f32db7fe640 1 -- 172.21.15.70:0/626497487 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f32d404b400 con 0x7f32e40c9910 2023-10-08T06:54:42.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.361+0000 7f32db7fe640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f32c406af90 0x7f32c406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.361+0000 7f32db7fe640 1 -- 172.21.15.70:0/626497487 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f32d40abfc0 con 0x7f32e40c9910 2023-10-08T06:54:42.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.361+0000 7f32dbfff640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f32c406af90 0x7f32c406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.363+0000 7f32dbfff640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f32c406af90 0x7f32c406d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f32e40afc80 tx=0x7f32cc009040 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:42.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.365+0000 7f32db7fe640 1 -- 172.21.15.70:0/626497487 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32d4076c80 con 0x7f32e40c9910 2023-10-08T06:54:42.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.582+0000 7f32e9010640 1 -- 172.21.15.70:0/626497487 --> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f32e40c93f0 con 0x7f32c406af90 2023-10-08T06:54:42.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.586+0000 7f32db7fe640 1 -- 172.21.15.70:0/626497487 <== mgr.6821 v2:172.21.15.181:6832/3709692999 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f32cc001c70 con 0x7f32c406af90 2023-10-08T06:54:42.587 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T06:54:42.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T06:54:42.588 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T06:54:42.588 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T06:54:42.588 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T06:54:42.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 -- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f32c406af90 msgr2=0x7f32c406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f32c406af90 0x7f32c406d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f32e40afc80 tx=0x7f32cc009040 comp rx=0 tx=0).stop 2023-10-08T06:54:42.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 -- 172.21.15.70:0/626497487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e40c9910 msgr2=0x7f32e414f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e40c9910 0x7f32e414f020 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f32d402b9c0 tx=0x7f32d40046c0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 -- 172.21.15.70:0/626497487 shutdown_connections 2023-10-08T06:54:42.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f32c406af90 0x7f32c406d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f32e4160660 0x7f32e40c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f32e40cb620 0x7f32e41f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 --2- 172.21.15.70:0/626497487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e40c9910 0x7f32e414f020 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 -- 172.21.15.70:0/626497487 >> 172.21.15.70:0/626497487 conn(0x7f32e406a5d0 msgr2=0x7f32e40c60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:42.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 -- 172.21.15.70:0/626497487 shutdown_connections 2023-10-08T06:54:42.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.588+0000 7f32e9010640 1 -- 172.21.15.70:0/626497487 wait complete. 2023-10-08T06:54:42.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T06:54:42.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.818+0000 7feaafeb4640 1 Processor -- start 2023-10-08T06:54:42.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.818+0000 7feaafeb4640 1 -- start start 2023-10-08T06:54:42.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.820+0000 7feaafeb4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feaa815aad0 0x7feaa80b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.820+0000 7feaafeb4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa815bf30 con 0x7feaa806d410 2023-10-08T06:54:42.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.820+0000 7feaafeb4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa815b320 con 0x7feaa80c82b0 2023-10-08T06:54:42.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.820+0000 7feaafeb4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa815b600 con 0x7feaa815aad0 2023-10-08T06:54:42.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.820+0000 7feaae451640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feaa806d410 0x7feaa8070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37768/0 (socket says 172.21.15.70:37768) 2023-10-08T06:54:42.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.820+0000 7feaae451640 1 -- 172.21.15.70:0/3691452863 learned_addr learned my addr 172.21.15.70:0/3691452863 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:42.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaadc50640 1 --2- 172.21.15.70:0/3691452863 >> v2:172.21.15.181:3300/0 conn(0x7feaa815aad0 0x7feaa80b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288584736 0 0) 0x7feaa815bf30 con 0x7feaa806d410 2023-10-08T06:54:42.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea94003520 con 0x7feaa806d410 2023-10-08T06:54:42.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3562904355 0 0) 0x7fea94003520 con 0x7feaa806d410 2023-10-08T06:54:42.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 >> v2:172.21.15.181:3300/0 conn(0x7feaa815aad0 msgr2=0x7feaa80b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 --2- 172.21.15.70:0/3691452863 >> v2:172.21.15.181:3300/0 conn(0x7feaa815aad0 0x7feaa80b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 >> v1:172.21.15.181:6789/0 conn(0x7feaa80c82b0 legacy=0x7feaa806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa815b8e0 con 0x7feaa806d410 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7feaa0001a10 con 0x7feaa806d410 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 >> v1:172.21.15.70:6789/0 conn(0x7feaa806d410 legacy=0x7feaa8070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 --2- 172.21.15.70:0/3691452863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea940039c0 0x7fea94003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 --2- 172.21.15.70:0/3691452863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7fea94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.821+0000 7feaacc4e640 1 --2- 172.21.15.70:0/3691452863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea94005180 0x7fea9400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa815b600 con 0x7fea940047c0 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9400c080 con 0x7fea94005180 2023-10-08T06:54:42.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9400c240 con 0x7fea940039c0 2023-10-08T06:54:42.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaad44f640 1 --2- 172.21.15.70:0/3691452863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7fea94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaadc50640 1 --2- 172.21.15.70:0/3691452863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea940039c0 0x7fea94003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaad44f640 1 -- 172.21.15.70:0/3691452863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea940039c0 msgr2=0x7fea94003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:42.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaad44f640 1 --2- 172.21.15.70:0/3691452863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea940039c0 0x7fea94003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaad44f640 1 -- 172.21.15.70:0/3691452863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea94005180 msgr2=0x7fea9400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:42.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaad44f640 1 --2- 172.21.15.70:0/3691452863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea94005180 0x7fea9400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaad44f640 1 -- 172.21.15.70:0/3691452863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94003520 con 0x7fea940047c0 2023-10-08T06:54:42.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.822+0000 7feaae451640 1 -- 172.21.15.70:0/3691452863 reap_dead start 2023-10-08T06:54:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.823+0000 7feaad44f640 1 --2- 172.21.15.70:0/3691452863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7fea94004c40 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fea98001750 tx=0x7fea940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70683f6ad697d17a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.823+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea980052d0 con 0x7fea940047c0 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.823+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9802e050 con 0x7fea940047c0 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.823+0000 7feaacc4e640 1 -- 172.21.15.70:0/3691452863 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98003830 con 0x7fea940047c0 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.824+0000 7feaafeb4640 1 -- 172.21.15.70:0/3691452863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 msgr2=0x7fea94004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.824+0000 7feaafeb4640 1 --2- 172.21.15.70:0/3691452863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7fea94004c40 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7fea98001750 tx=0x7fea940057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.824+0000 7feaafeb4640 1 -- 172.21.15.70:0/3691452863 shutdown_connections 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.824+0000 7feaafeb4640 1 --2- 172.21.15.70:0/3691452863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7fea94004c40 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.824+0000 7feaafeb4640 1 -- 172.21.15.70:0/3691452863 >> 172.21.15.70:0/3691452863 conn(0x7feaa8069d50 msgr2=0x7feaa806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.824+0000 7feaafeb4640 1 -- 172.21.15.70:0/3691452863 shutdown_connections 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.824+0000 7feaafeb4640 1 -- 172.21.15.70:0/3691452863 wait complete. 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.826+0000 7feaafeb4640 1 Processor -- start 2023-10-08T06:54:42.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.826+0000 7feaafeb4640 1 -- start start 2023-10-08T06:54:42.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.827+0000 7feaafeb4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7feaa8149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.827+0000 7feaafeb4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa80c7030 0x7feaa815c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.827+0000 7feaafeb4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa80c74b0 0x7feaa815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.827+0000 7feaafeb4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa8157850 con 0x7fea940047c0 2023-10-08T06:54:42.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.827+0000 7feaafeb4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa80c6550 con 0x7feaa80c7030 2023-10-08T06:54:42.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.827+0000 7feaafeb4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa80c6830 con 0x7feaa80c74b0 2023-10-08T06:54:42.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.827+0000 7feaadc50640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7feaa8149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.827+0000 7feaadc50640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7feaa8149430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60866/0 (socket says 172.21.15.70:60866) 2023-10-08T06:54:42.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.827+0000 7feaadc50640 1 -- 172.21.15.70:0/2966235244 learned_addr learned my addr 172.21.15.70:0/2966235244 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:42.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.828+0000 7feaad44f640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa80c7030 0x7feaa815c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.828+0000 7feaadc50640 1 -- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa80c74b0 msgr2=0x7feaa815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.828+0000 7feaadc50640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa80c74b0 0x7feaa815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.828+0000 7feaadc50640 1 -- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa80c7030 msgr2=0x7feaa815c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:42.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.828+0000 7feaadc50640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa80c7030 0x7feaa815c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:42.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.828+0000 7feaadc50640 1 -- 172.21.15.70:0/2966235244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa80c6b10 con 0x7fea940047c0 2023-10-08T06:54:42.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.828+0000 7feaadc50640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7feaa8149430 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7feaa40108e0 tx=0x7feaa4010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:42.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.831+0000 7fea9e7fc640 1 -- 172.21.15.70:0/2966235244 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa400eb60 con 0x7fea940047c0 2023-10-08T06:54:42.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.831+0000 7fea9e7fc640 1 -- 172.21.15.70:0/2966235244 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa400ecc0 con 0x7fea940047c0 2023-10-08T06:54:42.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.831+0000 7fea9e7fc640 1 -- 172.21.15.70:0/2966235244 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa401c400 con 0x7fea940047c0 2023-10-08T06:54:42.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.831+0000 7feaafeb4640 1 -- 172.21.15.70:0/2966235244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa8149970 con 0x7fea940047c0 2023-10-08T06:54:42.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.831+0000 7feaafeb4640 1 -- 172.21.15.70:0/2966235244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa81587a0 con 0x7fea940047c0 2023-10-08T06:54:42.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.833+0000 7feaafeb4640 1 -- 172.21.15.70:0/2966235244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa80c6390 con 0x7fea940047c0 2023-10-08T06:54:42.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.833+0000 7fea9e7fc640 1 -- 172.21.15.70:0/2966235244 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 98) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feaa401c560 con 0x7fea940047c0 2023-10-08T06:54:42.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.834+0000 7fea9e7fc640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fea8c06af90 0x7fea8c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:42.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.834+0000 7fea9e7fc640 1 -- 172.21.15.70:0/2966235244 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7feaa4089fc0 con 0x7fea940047c0 2023-10-08T06:54:42.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.835+0000 7feaad44f640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fea8c06af90 0x7fea8c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:42.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.836+0000 7feaad44f640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fea8c06af90 0x7fea8c06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fea98033000 tx=0x7fea98009030 comp rx=0 tx=0).ready entity=mgr.6821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:42.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:42.839+0000 7fea9e7fc640 1 -- 172.21.15.70:0/2966235244 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feaa40bcca0 con 0x7fea940047c0 2023-10-08T06:54:43.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.079+0000 7feaafeb4640 1 -- 172.21.15.70:0/2966235244 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7feaa80c6ca0 con 0x7fea940047c0 2023-10-08T06:54:43.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.837+0000 7fea9e7fc640 1 -- 172.21.15.70:0/2966235244 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v99) v1 ==== 92+0+0 (secure 0 0 0) 0x7feaa4054ec0 con 0x7fea940047c0 2023-10-08T06:54:43.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.840+0000 7feaafeb4640 1 -- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fea8c06af90 msgr2=0x7fea8c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:43.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.840+0000 7feaafeb4640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fea8c06af90 0x7fea8c06d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fea98033000 tx=0x7fea98009030 comp rx=0 tx=0).stop 2023-10-08T06:54:43.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.840+0000 7feaafeb4640 1 -- 172.21.15.70:0/2966235244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 msgr2=0x7feaa8149430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:43.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.840+0000 7feaafeb4640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7feaa8149430 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7feaa40108e0 tx=0x7feaa4010da0 comp rx=0 tx=0).stop 2023-10-08T06:54:43.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.840+0000 7feaafeb4640 1 -- 172.21.15.70:0/2966235244 shutdown_connections 2023-10-08T06:54:43.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.840+0000 7feaafeb4640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7fea8c06af90 0x7fea8c06d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:43.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.840+0000 7feaafeb4640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaa80c74b0 0x7feaa815cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:43.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.840+0000 7feaafeb4640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaa80c7030 0x7feaa815c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:43.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.840+0000 7feaafeb4640 1 --2- 172.21.15.70:0/2966235244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea940047c0 0x7feaa8149430 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:43.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.840+0000 7feaafeb4640 1 -- 172.21.15.70:0/2966235244 >> 172.21.15.70:0/2966235244 conn(0x7feaa8069d50 msgr2=0x7feaa80cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:43.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.841+0000 7feaafeb4640 1 -- 172.21.15.70:0/2966235244 shutdown_connections 2023-10-08T06:54:43.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:43.841+0000 7feaafeb4640 1 -- 172.21.15.70:0/2966235244 wait complete. 2023-10-08T06:54:43.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T06:54:44.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.071+0000 7f17ebbd7640 1 Processor -- start 2023-10-08T06:54:44.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.071+0000 7f17ebbd7640 1 -- start start 2023-10-08T06:54:44.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.073+0000 7f17ebbd7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f17e406b1b0 0x7f17e406f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.073+0000 7f17ebbd7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e40c9840 con 0x7f17e415af60 2023-10-08T06:54:44.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.073+0000 7f17ebbd7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e40ca430 con 0x7f17e415ab80 2023-10-08T06:54:44.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.073+0000 7f17ebbd7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e406b5c0 con 0x7f17e406b1b0 2023-10-08T06:54:44.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.073+0000 7f17e9172640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f17e415af60 0x7f17e406e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49040/0 (socket says 172.21.15.70:49040) 2023-10-08T06:54:44.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.073+0000 7f17e9172640 1 -- 172.21.15.70:0/1985812716 learned_addr learned my addr 172.21.15.70:0/1985812716 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:44.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.073+0000 7f17ea174640 1 --2- 172.21.15.70:0/1985812716 >> v2:172.21.15.181:3300/0 conn(0x7f17e406b1b0 0x7f17e406f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.074+0000 7f17e8971640 1 -- 172.21.15.70:0/1985812716 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950564377 0 0) 0x7f17e40c9840 con 0x7f17e415af60 2023-10-08T06:54:44.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.074+0000 7f17e8971640 1 -- 172.21.15.70:0/1985812716 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17d0003520 con 0x7f17e415af60 2023-10-08T06:54:44.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.074+0000 7f17ea174640 1 -- 172.21.15.70:0/1985812716 >> v1:172.21.15.181:6789/0 conn(0x7f17e415ab80 legacy=0x7f17e40b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.074+0000 7f17ea174640 1 -- 172.21.15.70:0/1985812716 >> v1:172.21.15.70:6789/0 conn(0x7f17e415af60 legacy=0x7f17e406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.074+0000 7f17e8971640 1 -- 172.21.15.70:0/1985812716 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2148328878 0 0) 0x7f17d0003520 con 0x7f17e415af60 2023-10-08T06:54:44.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.074+0000 7f17ea174640 1 -- 172.21.15.70:0/1985812716 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e406f5b0 con 0x7f17e406b1b0 2023-10-08T06:54:44.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.076+0000 7f17ea174640 1 --2- 172.21.15.70:0/1985812716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e406b1b0 0x7f17e406f070 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009d90 tx=0x7f17dc02bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c86fb7d81b620a86 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:44.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.076+0000 7f17e8971640 1 -- 172.21.15.70:0/1985812716 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4001670 con 0x7f17e406b1b0 2023-10-08T06:54:44.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.076+0000 7f17e8971640 1 -- 172.21.15.70:0/1985812716 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc02c8b0 con 0x7f17e406b1b0 2023-10-08T06:54:44.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.076+0000 7f17e8971640 1 -- 172.21.15.70:0/1985812716 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc02cc40 con 0x7f17e406b1b0 2023-10-08T06:54:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.077+0000 7f17ebbd7640 1 -- 172.21.15.70:0/1985812716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e406b1b0 msgr2=0x7f17e406f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.077+0000 7f17ebbd7640 1 --2- 172.21.15.70:0/1985812716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e406b1b0 0x7f17e406f070 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009d90 tx=0x7f17dc02bd80 comp rx=0 tx=0).stop 2023-10-08T06:54:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.077+0000 7f17ebbd7640 1 -- 172.21.15.70:0/1985812716 shutdown_connections 2023-10-08T06:54:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.077+0000 7f17ebbd7640 1 --2- 172.21.15.70:0/1985812716 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e406b1b0 0x7f17e406f070 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.077+0000 7f17ebbd7640 1 -- 172.21.15.70:0/1985812716 >> 172.21.15.70:0/1985812716 conn(0x7f17e40cba60 msgr2=0x7f17e40cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.077+0000 7f17ebbd7640 1 -- 172.21.15.70:0/1985812716 shutdown_connections 2023-10-08T06:54:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.077+0000 7f17ebbd7640 1 -- 172.21.15.70:0/1985812716 wait complete. 2023-10-08T06:54:44.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.079+0000 7f17ebbd7640 1 Processor -- start 2023-10-08T06:54:44.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.079+0000 7f17ebbd7640 1 -- start start 2023-10-08T06:54:44.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17ebbd7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e406b1b0 0x7f17e414b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17ebbd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e415ab80 0x7f17e40afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17ebbd7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17e415af60 0x7f17e40b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17ebbd7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e414b9f0 con 0x7f17e415ab80 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17ebbd7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40c7100 con 0x7f17e406b1b0 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17ebbd7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40ceac0 con 0x7f17e415af60 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17e9172640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e415ab80 0x7f17e40afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17e9172640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e415ab80 0x7f17e40afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56128/0 (socket says 172.21.15.70:56128) 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17e9172640 1 -- 172.21.15.70:0/3823627858 learned_addr learned my addr 172.21.15.70:0/3823627858 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17e9973640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e406b1b0 0x7f17e414b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17ea174640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17e415af60 0x7f17e40b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17e9172640 1 -- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17e415af60 msgr2=0x7f17e40b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17e9172640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17e415af60 0x7f17e40b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17e9172640 1 -- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e406b1b0 msgr2=0x7f17e414b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17e9172640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e406b1b0 0x7f17e414b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17e9172640 1 -- 172.21.15.70:0/3823627858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc009a70 con 0x7f17e415ab80 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.080+0000 7f17ea174640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17e415af60 0x7f17e40b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.081+0000 7f17e9172640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e415ab80 0x7f17e40afc60 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f17d400c750 tx=0x7f17d400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.081+0000 7f17da7fc640 1 -- 172.21.15.70:0/3823627858 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4005ad0 con 0x7f17e415ab80 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.081+0000 7f17da7fc640 1 -- 172.21.15.70:0/3823627858 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d4005c30 con 0x7f17e415ab80 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.081+0000 7f17ebbd7640 1 -- 172.21.15.70:0/3823627858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e40c73c0 con 0x7f17e415ab80 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.081+0000 7f17da7fc640 1 -- 172.21.15.70:0/3823627858 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d40072b0 con 0x7f17e415ab80 2023-10-08T06:54:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.081+0000 7f17ebbd7640 1 -- 172.21.15.70:0/3823627858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e40c7980 con 0x7f17e415ab80 2023-10-08T06:54:44.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.082+0000 7f17ebbd7640 1 -- 172.21.15.70:0/3823627858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e40c7ba0 con 0x7f17e415ab80 2023-10-08T06:54:44.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.084+0000 7f17da7fc640 1 -- 172.21.15.70:0/3823627858 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 99) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f17d4007410 con 0x7f17e415ab80 2023-10-08T06:54:44.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.085+0000 7f17da7fc640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f17c806b010 0x7f17c806d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.085+0000 7f17da7fc640 1 -- 172.21.15.70:0/3823627858 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f17d40890f0 con 0x7f17e415ab80 2023-10-08T06:54:44.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.085+0000 7f17e9973640 1 -- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f17c806b010 msgr2=0x7f17c806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3709692999 2023-10-08T06:54:44.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.085+0000 7f17e9973640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f17c806b010 0x7f17c806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:44.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.090+0000 7f17da7fc640 1 -- 172.21.15.70:0/3823627858 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17d4053e30 con 0x7f17e415ab80 2023-10-08T06:54:44.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.279+0000 7f17ebbd7640 1 -- 172.21.15.70:0/3823627858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f17e404e160 con 0x7f17e415ab80 2023-10-08T06:54:44.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.286+0000 7f17e9973640 1 -- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f17c806b010 msgr2=0x7f17c806d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3709692999 2023-10-08T06:54:44.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.286+0000 7f17e9973640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f17c806b010 0x7f17c806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:54:44.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.286+0000 7f17da7fc640 1 -- 172.21.15.70:0/3823627858 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f17d40170d0 con 0x7f17e415ab80 2023-10-08T06:54:44.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 -- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f17c806b010 msgr2=0x7f17c806d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:44.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f17c806b010 0x7f17c806d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 -- 172.21.15.70:0/3823627858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e415ab80 msgr2=0x7f17e40afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e415ab80 0x7f17e40afc60 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f17d400c750 tx=0x7f17d400cc10 comp rx=0 tx=0).stop 2023-10-08T06:54:44.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 -- 172.21.15.70:0/3823627858 shutdown_connections 2023-10-08T06:54:44.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f17c806b010 0x7f17c806d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17e415af60 0x7f17e40b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e415ab80 0x7f17e40afc60 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 --2- 172.21.15.70:0/3823627858 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f17e406b1b0 0x7f17e414b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 -- 172.21.15.70:0/3823627858 >> 172.21.15.70:0/3823627858 conn(0x7f17e40cba60 msgr2=0x7f17e40b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:44.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 -- 172.21.15.70:0/3823627858 shutdown_connections 2023-10-08T06:54:44.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.289+0000 7f17ebbd7640 1 -- 172.21.15.70:0/3823627858 wait complete. 2023-10-08T06:54:44.310 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T06:54:44.390 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:44.389+0000 7f82f8c3e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T06:54:44.399 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:44.397+0000 7f1bfb012040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T06:54:44.444 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:44.443+0000 7f82f8c3e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T06:54:44.454 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:44.452+0000 7f1bfb012040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T06:54:44.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.508+0000 7f66c70fc640 1 Processor -- start 2023-10-08T06:54:44.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.508+0000 7f66c70fc640 1 -- start start 2023-10-08T06:54:44.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.510+0000 7f66c70fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f66c00cf2b0 0x7f66c00d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.510+0000 7f66c70fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00ce510 0x7f66c015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.510+0000 7f66c70fc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c00cd520 con 0x7f66c00cf690 2023-10-08T06:54:44.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.510+0000 7f66c70fc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c00d1910 con 0x7f66c00cf2b0 2023-10-08T06:54:44.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.510+0000 7f66c70fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00ce080 con 0x7f66c00ce510 2023-10-08T06:54:44.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.510+0000 7f66c58f9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f66c00cf690 0x7f66c00d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49044/0 (socket says 172.21.15.70:49044) 2023-10-08T06:54:44.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.510+0000 7f66c58f9640 1 -- 172.21.15.70:0/690074966 learned_addr learned my addr 172.21.15.70:0/690074966 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:44.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c60fa640 1 --2- 172.21.15.70:0/690074966 >> v2:172.21.15.181:3300/0 conn(0x7f66c00cf2b0 0x7f66c00d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c68fb640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00ce510 0x7f66c015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764872554 0 0) 0x7f66c00cd520 con 0x7f66c00cf690 2023-10-08T06:54:44.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66ac003520 con 0x7f66c00cf690 2023-10-08T06:54:44.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1123477206 0 0) 0x7f66ac003520 con 0x7f66c00cf690 2023-10-08T06:54:44.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00ce510 msgr2=0x7f66c015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:44.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00ce510 0x7f66c015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 >> v2:172.21.15.181:3300/0 conn(0x7f66c00cf2b0 msgr2=0x7f66c00d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:44.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 --2- 172.21.15.70:0/690074966 >> v2:172.21.15.181:3300/0 conn(0x7f66c00cf2b0 0x7f66c00d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c00cd750 con 0x7f66c00cf690 2023-10-08T06:54:44.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f66b0001670 con 0x7f66c00cf690 2023-10-08T06:54:44.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 >> v1:172.21.15.70:6789/0 conn(0x7f66c00cf690 legacy=0x7f66c00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00ce510 0x7f66ac003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66ac003f40 0x7f66ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 0x7f66ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00d1910 con 0x7f66ac004d00 2023-10-08T06:54:44.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00ce080 con 0x7f66ac003f40 2023-10-08T06:54:44.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.511+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac00ba30 con 0x7f66c00ce510 2023-10-08T06:54:44.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.512+0000 7f66c68fb640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 0x7f66ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.512+0000 7f66c58f9640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66ac003f40 0x7f66ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.512+0000 7f66c60fa640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00ce510 0x7f66ac003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.512+0000 7f66c68fb640 1 -- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00ce510 msgr2=0x7f66ac003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:44.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.512+0000 7f66c68fb640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00ce510 0x7f66ac003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.512+0000 7f66c68fb640 1 -- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66ac003f40 msgr2=0x7f66ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:44.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.512+0000 7f66c68fb640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66ac003f40 0x7f66ac0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.512+0000 7f66c68fb640 1 -- 172.21.15.70:0/690074966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac003520 con 0x7f66ac004d00 2023-10-08T06:54:44.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.514+0000 7f66c68fb640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 0x7f66ac005180 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009fc0 tx=0x7f66ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f958e35492e109 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:44.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.514+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b802cc20 con 0x7f66ac004d00 2023-10-08T06:54:44.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.514+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b800f040 con 0x7f66ac004d00 2023-10-08T06:54:44.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.514+0000 7f66c50f8640 1 -- 172.21.15.70:0/690074966 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8034440 con 0x7f66ac004d00 2023-10-08T06:54:44.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.515+0000 7f66c70fc640 1 -- 172.21.15.70:0/690074966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 msgr2=0x7f66ac005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.515+0000 7f66c70fc640 1 --2- 172.21.15.70:0/690074966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 0x7f66ac005180 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f66b8009fc0 tx=0x7f66ac0057f0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.515+0000 7f66c68fb640 1 -- 172.21.15.70:0/690074966 reap_dead start 2023-10-08T06:54:44.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.515+0000 7f66c70fc640 1 -- 172.21.15.70:0/690074966 shutdown_connections 2023-10-08T06:54:44.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.515+0000 7f66c70fc640 1 -- 172.21.15.70:0/690074966 >> 172.21.15.70:0/690074966 conn(0x7f66c009bc40 msgr2=0x7f66c00d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:44.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.515+0000 7f66c70fc640 1 -- 172.21.15.70:0/690074966 shutdown_connections 2023-10-08T06:54:44.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.515+0000 7f66c70fc640 1 -- 172.21.15.70:0/690074966 wait complete. 2023-10-08T06:54:44.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.518+0000 7f66c70fc640 1 Processor -- start 2023-10-08T06:54:44.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.518+0000 7f66c70fc640 1 -- start start 2023-10-08T06:54:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.520+0000 7f66c70fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 0x7f66c01f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.520+0000 7f66c70fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00c99e0 0x7f66c00b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.520+0000 7f66c70fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c006b840 0x7f66c006bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.520+0000 7f66c70fc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c006c210 con 0x7f66ac004d00 2023-10-08T06:54:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.520+0000 7f66c70fc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c01f32b0 con 0x7f66c006b840 2023-10-08T06:54:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.520+0000 7f66c70fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c00cc7d0 con 0x7f66c00c99e0 2023-10-08T06:54:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.520+0000 7f66c60fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 0x7f66c01f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.520+0000 7f66c60fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 0x7f66c01f2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56150/0 (socket says 172.21.15.70:56150) 2023-10-08T06:54:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.520+0000 7f66c60fa640 1 -- 172.21.15.70:0/3102431175 learned_addr learned my addr 172.21.15.70:0/3102431175 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66c58f9640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00c99e0 0x7f66c00b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66c68fb640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c006b840 0x7f66c006bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66c60fa640 1 -- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00c99e0 msgr2=0x7f66c00b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66c60fa640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00c99e0 0x7f66c00b7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66c60fa640 1 -- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c006b840 msgr2=0x7f66c006bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66c60fa640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c006b840 0x7f66c006bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66c60fa640 1 -- 172.21.15.70:0/3102431175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c00c9ea0 con 0x7f66ac004d00 2023-10-08T06:54:44.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66c60fa640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 0x7f66c01f2d70 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f66bc010910 tx=0x7f66bc010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66b6ffd640 1 -- 172.21.15.70:0/3102431175 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc009690 con 0x7f66ac004d00 2023-10-08T06:54:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66b6ffd640 1 -- 172.21.15.70:0/3102431175 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc00ec20 con 0x7f66ac004d00 2023-10-08T06:54:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66c70fc640 1 -- 172.21.15.70:0/3102431175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c00ca180 con 0x7f66ac004d00 2023-10-08T06:54:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.521+0000 7f66b6ffd640 1 -- 172.21.15.70:0/3102431175 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc01c400 con 0x7f66ac004d00 2023-10-08T06:54:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.522+0000 7f66c70fc640 1 -- 172.21.15.70:0/3102431175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c00ca740 con 0x7f66ac004d00 2023-10-08T06:54:44.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.523+0000 7f66c70fc640 1 -- 172.21.15.70:0/3102431175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c0074150 con 0x7f66ac004d00 2023-10-08T06:54:44.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.524+0000 7f66b6ffd640 1 -- 172.21.15.70:0/3102431175 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 99) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f66bc01c560 con 0x7f66ac004d00 2023-10-08T06:54:44.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.524+0000 7f66b6ffd640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f66a406afe0 0x7f66a406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:44.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.525+0000 7f66b6ffd640 1 -- 172.21.15.70:0/3102431175 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f66bc089ff0 con 0x7f66ac004d00 2023-10-08T06:54:44.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.525+0000 7f66c58f9640 1 -- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f66a406afe0 msgr2=0x7f66a406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3709692999 2023-10-08T06:54:44.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.525+0000 7f66c58f9640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f66a406afe0 0x7f66a406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:44.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.529+0000 7f66b6ffd640 1 -- 172.21.15.70:0/3102431175 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66bc0bcca0 con 0x7f66ac004d00 2023-10-08T06:54:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.726+0000 7f66c58f9640 1 -- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f66a406afe0 msgr2=0x7f66a406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3709692999 2023-10-08T06:54:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.726+0000 7f66c58f9640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f66a406afe0 0x7f66a406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:54:44.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.735+0000 7f66c70fc640 1 -- 172.21.15.70:0/3102431175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f66c00d0070 con 0x7f66ac004d00 2023-10-08T06:54:44.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.741+0000 7f66b6ffd640 1 -- 172.21.15.70:0/3102431175 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7f66bc054d70 con 0x7f66ac004d00 2023-10-08T06:54:44.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 -- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f66a406afe0 msgr2=0x7f66a406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:44.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f66a406afe0 0x7f66a406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 -- 172.21.15.70:0/3102431175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 msgr2=0x7f66c01f2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:44.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 0x7f66c01f2d70 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f66bc010910 tx=0x7f66bc010dd0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 -- 172.21.15.70:0/3102431175 shutdown_connections 2023-10-08T06:54:44.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f66a406afe0 0x7f66a406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f66c006b840 0x7f66c006bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66c00c99e0 0x7f66c00b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 --2- 172.21.15.70:0/3102431175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66ac004d00 0x7f66c01f2d70 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:44.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 -- 172.21.15.70:0/3102431175 >> 172.21.15.70:0/3102431175 conn(0x7f66c009bc40 msgr2=0x7f66c00b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:44.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 -- 172.21.15.70:0/3102431175 shutdown_connections 2023-10-08T06:54:44.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:44.743+0000 7f66c70fc640 1 -- 172.21.15.70:0/3102431175 wait complete. 2023-10-08T06:54:44.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:44.770 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T06:54:44.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T06:54:44.828 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-10-08T06:54:44.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/test_restart 1m 2023-10-08T06:54:44.894 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:44.893+0000 7f82f8c3e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T06:54:44.946 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:44.945+0000 7f82f8c3e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T06:54:44.956 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:44.955+0000 7f1bfb012040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T06:54:44.996 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:44.995+0000 7f82f8c3e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T06:54:45.011 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:45.009+0000 7f1bfb012040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T06:54:45.056 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:45.055+0000 7f82f8c3e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T06:54:45.063 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:45.062+0000 7f1bfb012040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T06:54:45.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.072+0000 7f1616905640 1 Processor -- start 2023-10-08T06:54:45.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.072+0000 7f1616905640 1 -- start start 2023-10-08T06:54:45.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.073+0000 7f1616905640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f161006c3a0 0x7f16100c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:45.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.074+0000 7f1616905640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161015b310 con 0x7f161006db60 2023-10-08T06:54:45.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.074+0000 7f1616905640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161006d290 con 0x7f161015aa00 2023-10-08T06:54:45.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.074+0000 7f1616905640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16100ca230 con 0x7f161006c3a0 2023-10-08T06:54:45.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.074+0000 7f160ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f161006c3a0 0x7f16100c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:45.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.074+0000 7f160f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f161006db60 0x7f161006c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49054/0 (socket says 172.21.15.70:49054) 2023-10-08T06:54:45.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.074+0000 7f160f7fe640 1 -- 172.21.15.70:0/2227862431 learned_addr learned my addr 172.21.15.70:0/2227862431 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T06:54:45.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.074+0000 7f160ffff640 1 -- 172.21.15.70:0/2227862431 >> v1:172.21.15.181:6789/0 conn(0x7f161015aa00 legacy=0x7f16100b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:45.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.074+0000 7f160ffff640 1 -- 172.21.15.70:0/2227862431 >> v1:172.21.15.70:6789/0 conn(0x7f161006db60 legacy=0x7f161006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:45.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.074+0000 7f160effd640 1 -- 172.21.15.70:0/2227862431 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852313112 0 0) 0x7f1604001110 con 0x7f161006db60 2023-10-08T06:54:45.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.074+0000 7f160ffff640 1 -- 172.21.15.70:0/2227862431 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16100c9860 con 0x7f161006c3a0 2023-10-08T06:54:45.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.076+0000 7f160ffff640 1 --2- 172.21.15.70:0/2227862431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161006c3a0 0x7f16100c9cf0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f16000097a0 tx=0x7f160002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b5f3bc0255661b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:45.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.076+0000 7f160effd640 1 -- 172.21.15.70:0/2227862431 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600031830 con 0x7f161006c3a0 2023-10-08T06:54:45.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.076+0000 7f160effd640 1 -- 172.21.15.70:0/2227862431 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1600031990 con 0x7f161006c3a0 2023-10-08T06:54:45.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.076+0000 7f160effd640 1 -- 172.21.15.70:0/2227862431 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600031d20 con 0x7f161006c3a0 2023-10-08T06:54:45.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.077+0000 7f1616905640 1 -- 172.21.15.70:0/2227862431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161006c3a0 msgr2=0x7f16100c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:45.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.077+0000 7f1616905640 1 --2- 172.21.15.70:0/2227862431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161006c3a0 0x7f16100c9cf0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f16000097a0 tx=0x7f160002b8d0 comp rx=0 tx=0).stop 2023-10-08T06:54:45.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.077+0000 7f1616905640 1 -- 172.21.15.70:0/2227862431 shutdown_connections 2023-10-08T06:54:45.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.077+0000 7f1616905640 1 --2- 172.21.15.70:0/2227862431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161006c3a0 0x7f16100c9cf0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:45.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.077+0000 7f1616905640 1 -- 172.21.15.70:0/2227862431 >> 172.21.15.70:0/2227862431 conn(0x7f161006a5d0 msgr2=0x7f16100b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:45.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.077+0000 7f1616905640 1 -- 172.21.15.70:0/2227862431 shutdown_connections 2023-10-08T06:54:45.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.077+0000 7f1616905640 1 -- 172.21.15.70:0/2227862431 wait complete. 2023-10-08T06:54:45.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.079+0000 7f1616905640 1 Processor -- start 2023-10-08T06:54:45.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.079+0000 7f1616905640 1 -- start start 2023-10-08T06:54:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.080+0000 7f1616905640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f161006c3a0 0x7f16101492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.080+0000 7f1616905640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161006db60 0x7f16100c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:45.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.080+0000 7f1616905640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161015aa00 0x7f16100c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1616905640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100c2f10 con 0x7f161015aa00 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1616905640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100c5140 con 0x7f161006db60 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1616905640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161014c450 con 0x7f161006c3a0 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1614ea2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161015aa00 0x7f16100c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1614ea2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161015aa00 0x7f16100c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56172/0 (socket says 172.21.15.70:56172) 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1614ea2640 1 -- 172.21.15.70:0/1517671593 learned_addr learned my addr 172.21.15.70:0/1517671593 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f160ffff640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f161006c3a0 0x7f16101492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f160f7fe640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161006db60 0x7f16100c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1614ea2640 1 -- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f161006c3a0 msgr2=0x7f16101492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1614ea2640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f161006c3a0 0x7f16101492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1614ea2640 1 -- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161006db60 msgr2=0x7f16100c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:45.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1614ea2640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161006db60 0x7f16100c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:45.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f1614ea2640 1 -- 172.21.15.70:0/1517671593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600009400 con 0x7f161015aa00 2023-10-08T06:54:45.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.081+0000 7f160f7fe640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f161006db60 0x7f16100c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:45.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.082+0000 7f1614ea2640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161015aa00 0x7f16100c4850 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f160800b970 tx=0x7f160800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:45.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.082+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160800cb40 con 0x7f161015aa00 2023-10-08T06:54:45.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.082+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160800cca0 con 0x7f161015aa00 2023-10-08T06:54:45.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.082+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1608014680 con 0x7f161015aa00 2023-10-08T06:54:45.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.082+0000 7f1616905640 1 -- 172.21.15.70:0/1517671593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16100c4e60 con 0x7f161015aa00 2023-10-08T06:54:45.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.082+0000 7f1616905640 1 -- 172.21.15.70:0/1517671593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161014c0e0 con 0x7f161015aa00 2023-10-08T06:54:45.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.083+0000 7f1616905640 1 -- 172.21.15.70:0/1517671593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161014c600 con 0x7f161015aa00 2023-10-08T06:54:45.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.084+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 99) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f16080147e0 con 0x7f161015aa00 2023-10-08T06:54:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.085+0000 7f160cff9640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 0x7f15fc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.085+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f16080897c0 con 0x7f161015aa00 2023-10-08T06:54:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.085+0000 7f160ffff640 1 -- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 msgr2=0x7f15fc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3709692999 2023-10-08T06:54:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.085+0000 7f160ffff640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 0x7f15fc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T06:54:45.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.089+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f161014c600 con 0x7f161015aa00 2023-10-08T06:54:45.106 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:45.105+0000 7f82f8c3e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T06:54:45.128 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:45.127+0000 7f1bfb012040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T06:54:45.179 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:45.177+0000 7f1bfb012040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T06:54:45.216 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:45.215+0000 7f82f8c3e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T06:54:45.277 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:45.276+0000 7f82f8c3e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:45.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.286+0000 7f160ffff640 1 -- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 msgr2=0x7f15fc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3709692999 2023-10-08T06:54:45.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.286+0000 7f160ffff640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 0x7f15fc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T06:54:45.294 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:45.292+0000 7f1bfb012040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T06:54:45.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.302+0000 7f1616905640 1 -- 172.21.15.70:0/1517671593 --> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f16100c9210 con 0x7f15fc06afe0 2023-10-08T06:54:45.326 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:45.325+0000 7f82f8c3e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T06:54:45.357 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:45.356+0000 7f1bfb012040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:45.383 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:45.382+0000 7f82f8c3e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T06:54:45.406 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:45.404+0000 7f1bfb012040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T06:54:45.433 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:45.432+0000 7f82f8c3e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T06:54:45.463 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:45.461+0000 7f1bfb012040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T06:54:45.513 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:45.512+0000 7f1bfb012040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T06:54:45.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.687+0000 7f160ffff640 1 -- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 msgr2=0x7f15fc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3709692999 2023-10-08T06:54:45.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:45.687+0000 7f160ffff640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 0x7f15fc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T06:54:45.813 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:45.811+0000 7f82f8c3e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T06:54:45.896 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:45.894+0000 7f1bfb012040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T06:54:45.971 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:45.970+0000 7f82f8c3e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T06:54:46.054 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:46.053+0000 7f1bfb012040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T06:54:46.079 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:46.078+0000 7f82f8c3e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T06:54:46.129 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:46.127+0000 7f82f8c3e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T06:54:46.162 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:46.161+0000 7f1bfb012040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T06:54:46.212 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:46.210+0000 7f1bfb012040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T06:54:46.234 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:46.233+0000 7f82f8c3e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:46.319 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:46.318+0000 7f1bfb012040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:46.340 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:46.339+0000 7f82f8c3e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T06:54:46.425 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:46.423+0000 7f1bfb012040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T06:54:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:46.488+0000 7f160ffff640 1 -- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 msgr2=0x7f15fc06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3709692999 2023-10-08T06:54:46.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:46.488+0000 7f160ffff640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 0x7f15fc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T06:54:46.494 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:46.492+0000 7f82f8c3e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:46.543 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:46.542+0000 7f82f8c3e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:46.580 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:46.578+0000 7f1bfb012040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T06:54:46.629 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:46.627+0000 7f1bfb012040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T06:54:46.661 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:46.660+0000 7f82f8c3e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T06:54:46.747 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:46.745+0000 7f1bfb012040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T06:54:47.480 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:47.478+0000 7f82f8c3e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T06:54:47.571 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:47.569+0000 7f1bfb012040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T06:54:47.774 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:47.773+0000 7f82f8c3e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T06:54:47.831 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:47.830+0000 7f82f8c3e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T06:54:47.865 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:47.863+0000 7f1bfb012040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T06:54:47.922 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:47.921+0000 7f1bfb012040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T06:54:47.944 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:47.943+0000 7f82f8c3e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T06:54:47.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:47.964+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 100) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f1608054960 con 0x7f161015aa00 2023-10-08T06:54:47.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:47.965+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 msgr2=0x7f15fc06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T06:54:47.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:47.965+0000 7f160cff9640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3709692999,v1:172.21.15.181:6833/3709692999] conn(0x7f15fc06afe0 0x7f15fc06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:48.055 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T06:54:48.053+0000 7f1bfb012040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T06:54:48.143 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:48.142+0000 7f8275720640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:48.144 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:48.142+0000 7f8275720640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:48.144 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:48.142+0000 7f8275720640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:48.144 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:48.142+0000 7f8275720640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:48.144 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:48.142+0000 7f8275720640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:48.144 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:48.143+0000 7f8278766640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:48.145 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:48.143+0000 7f8278766640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:48.145 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:48.143+0000 7f8278766640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:48.145 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:48.143+0000 7f8278766640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:48.145 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:54:48.143+0000 7f8278766640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:54:48.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:48.974+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 101) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f16080600c0 con 0x7f161015aa00 2023-10-08T06:54:48.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:48.975+0000 7f160cff9640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f15fc035880 0x7f15fc075530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:48.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:48.975+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 --> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f15fc035c60 con 0x7f15fc035880 2023-10-08T06:54:48.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:48.977+0000 7f160ffff640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f15fc035880 0x7f15fc075530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:48.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:48.991+0000 7f160ffff640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f15fc035880 0x7f15fc075530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1600036000 tx=0x7f1600008030 comp rx=0 tx=0).ready entity=mgr.7849 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:49.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.162+0000 7f160cff9640 1 -- 172.21.15.70:0/1517671593 <== mgr.7849 v2:172.21.15.181:6832/3979056493 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f15fc035c60 con 0x7f15fc035880 2023-10-08T06:54:49.164 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-10-08T06:54:49.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.165+0000 7f1616905640 1 -- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f15fc035880 msgr2=0x7f15fc075530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:49.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.165+0000 7f1616905640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f15fc035880 0x7f15fc075530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1600036000 tx=0x7f1600008030 comp rx=0 tx=0).stop 2023-10-08T06:54:49.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.165+0000 7f1616905640 1 -- 172.21.15.70:0/1517671593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161015aa00 msgr2=0x7f16100c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:49.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.165+0000 7f1616905640 1 --2- 172.21.15.70:0/1517671593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f161015aa00 0x7f16100c4850 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f160800b970 tx=0x7f160800be30 comp rx=0 tx=0).stop 2023-10-08T06:54:49.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.165+0000 7f1614ea2640 1 -- 172.21.15.70:0/1517671593 reap_dead start 2023-10-08T06:54:49.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.166+0000 7f1616905640 1 -- 172.21.15.70:0/1517671593 shutdown_connections 2023-10-08T06:54:49.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.166+0000 7f1616905640 1 -- 172.21.15.70:0/1517671593 >> 172.21.15.70:0/1517671593 conn(0x7f161006a5d0 msgr2=0x7f1610159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:49.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.166+0000 7f1616905640 1 -- 172.21.15.70:0/1517671593 shutdown_connections 2023-10-08T06:54:49.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.166+0000 7f1616905640 1 -- 172.21.15.70:0/1517671593 wait complete. 2023-10-08T06:54:49.187 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-10-08T06:54:49.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/test_restart 1m 2023-10-08T06:54:49.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.400+0000 7fad52f3c640 1 Processor -- start 2023-10-08T06:54:49.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.400+0000 7fad52f3c640 1 -- start start 2023-10-08T06:54:49.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.402+0000 7fad52f3c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad4c156a10 0x7fad4c1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.402+0000 7fad52f3c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fad4c0b0680 0x7fad4c0c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.402+0000 7fad52f3c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4c071d90 con 0x7fad4c070ac0 2023-10-08T06:54:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.402+0000 7fad52f3c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4c070f20 con 0x7fad4c156a10 2023-10-08T06:54:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.402+0000 7fad52f3c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4c071180 con 0x7fad4c0b0680 2023-10-08T06:54:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.402+0000 7fad51f3a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad4c156a10 0x7fad4c1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.402+0000 7fad51f3a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad4c156a10 0x7fad4c1594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56186/0 (socket says 172.21.15.70:56186) 2023-10-08T06:54:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.402+0000 7fad51f3a640 1 -- 172.21.15.70:0/1442570122 learned_addr learned my addr 172.21.15.70:0/1442570122 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:49.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.403+0000 7fad5273b640 1 --2- 172.21.15.70:0/1442570122 >> v2:172.21.15.181:3300/0 conn(0x7fad4c0b0680 0x7fad4c0c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:49.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.403+0000 7fad51f3a640 1 -- 172.21.15.70:0/1442570122 >> v2:172.21.15.181:3300/0 conn(0x7fad4c0b0680 msgr2=0x7fad4c0c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T06:54:49.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.403+0000 7fad51f3a640 1 --2- 172.21.15.70:0/1442570122 >> v2:172.21.15.181:3300/0 conn(0x7fad4c0b0680 0x7fad4c0c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:49.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.403+0000 7fad51f3a640 1 -- 172.21.15.70:0/1442570122 >> v1:172.21.15.70:6789/0 conn(0x7fad4c070ac0 legacy=0x7fad4c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:49.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.403+0000 7fad51f3a640 1 -- 172.21.15.70:0/1442570122 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c071460 con 0x7fad4c156a10 2023-10-08T06:54:49.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.403+0000 7fad5273b640 1 --2- 172.21.15.70:0/1442570122 >> v2:172.21.15.181:3300/0 conn(0x7fad4c0b0680 0x7fad4c0c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T06:54:49.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.404+0000 7fad51f3a640 1 --2- 172.21.15.70:0/1442570122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 0x7fad4c1594e0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0adee0 tx=0x7fad4802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa41bad8d99fc98f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.404+0000 7fad50f38640 1 -- 172.21.15.70:0/1442570122 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48030870 con 0x7fad4c156a10 2023-10-08T06:54:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.404+0000 7fad50f38640 1 -- 172.21.15.70:0/1442570122 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad480309d0 con 0x7fad4c156a10 2023-10-08T06:54:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.404+0000 7fad50f38640 1 -- 172.21.15.70:0/1442570122 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48030d60 con 0x7fad4c156a10 2023-10-08T06:54:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.405+0000 7fad52f3c640 1 -- 172.21.15.70:0/1442570122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 msgr2=0x7fad4c1594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.405+0000 7fad52f3c640 1 --2- 172.21.15.70:0/1442570122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 0x7fad4c1594e0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0adee0 tx=0x7fad4802b9b0 comp rx=0 tx=0).stop 2023-10-08T06:54:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.405+0000 7fad52f3c640 1 -- 172.21.15.70:0/1442570122 shutdown_connections 2023-10-08T06:54:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.405+0000 7fad52f3c640 1 --2- 172.21.15.70:0/1442570122 >> v2:172.21.15.181:3300/0 conn(0x7fad4c0b0680 0x7fad4c0c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.405+0000 7fad52f3c640 1 --2- 172.21.15.70:0/1442570122 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 0x7fad4c1594e0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.405+0000 7fad52f3c640 1 -- 172.21.15.70:0/1442570122 >> 172.21.15.70:0/1442570122 conn(0x7fad4c06c050 msgr2=0x7fad4c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.405+0000 7fad52f3c640 1 -- 172.21.15.70:0/1442570122 shutdown_connections 2023-10-08T06:54:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.405+0000 7fad52f3c640 1 -- 172.21.15.70:0/1442570122 wait complete. 2023-10-08T06:54:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.407+0000 7fad52f3c640 1 Processor -- start 2023-10-08T06:54:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.407+0000 7fad52f3c640 1 -- start start 2023-10-08T06:54:49.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad52f3c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad4c070ac0 0x7fad4c0b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad52f3c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad4c0b0680 0x7fad4c14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad52f3c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 0x7fad4c0ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad52f3c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0b3e30 con 0x7fad4c156a10 2023-10-08T06:54:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad52f3c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0c6da0 con 0x7fad4c070ac0 2023-10-08T06:54:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad52f3c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4c0b3220 con 0x7fad4c0b0680 2023-10-08T06:54:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad5273b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 0x7fad4c0ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad5273b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 0x7fad4c0ce480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56188/0 (socket says 172.21.15.70:56188) 2023-10-08T06:54:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad5273b640 1 -- 172.21.15.70:0/47868466 learned_addr learned my addr 172.21.15.70:0/47868466 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T06:54:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad5273b640 1 -- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad4c0b0680 msgr2=0x7fad4c14ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad5273b640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad4c0b0680 0x7fad4c14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad5273b640 1 -- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad4c070ac0 msgr2=0x7fad4c0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad5273b640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad4c070ac0 0x7fad4c0b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.408+0000 7fad5273b640 1 -- 172.21.15.70:0/47868466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad480094d0 con 0x7fad4c156a10 2023-10-08T06:54:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.409+0000 7fad5273b640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 0x7fad4c0ce480 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fad4400cd80 tx=0x7fad4400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.409+0000 7fad427fc640 1 -- 172.21.15.70:0/47868466 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400f680 con 0x7fad4c156a10 2023-10-08T06:54:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.409+0000 7fad427fc640 1 -- 172.21.15.70:0/47868466 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4400f800 con 0x7fad4c156a10 2023-10-08T06:54:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.409+0000 7fad52f3c640 1 -- 172.21.15.70:0/47868466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4c0b3500 con 0x7fad4c156a10 2023-10-08T06:54:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.409+0000 7fad427fc640 1 -- 172.21.15.70:0/47868466 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400fbb0 con 0x7fad4c156a10 2023-10-08T06:54:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.409+0000 7fad52f3c640 1 -- 172.21.15.70:0/47868466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4c0b3ac0 con 0x7fad4c156a10 2023-10-08T06:54:49.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.410+0000 7fad52f3c640 1 -- 172.21.15.70:0/47868466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4c0c2570 con 0x7fad4c156a10 2023-10-08T06:54:49.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.411+0000 7fad427fc640 1 -- 172.21.15.70:0/47868466 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 101) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fad4400b040 con 0x7fad4c156a10 2023-10-08T06:54:49.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.413+0000 7fad427fc640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7fad3006ad10 0x7fad3006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T06:54:49.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.413+0000 7fad427fc640 1 -- 172.21.15.70:0/47868466 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6489+0+0 (secure 0 0 0) 0x7fad4408a0b0 con 0x7fad4c156a10 2023-10-08T06:54:49.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.413+0000 7fad51f3a640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7fad3006ad10 0x7fad3006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T06:54:49.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.414+0000 7fad51f3a640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7fad3006ad10 0x7fad3006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fad48036000 tx=0x7fad4803b040 comp rx=0 tx=0).ready entity=mgr.7849 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T06:54:49.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.417+0000 7fad427fc640 1 -- 172.21.15.70:0/47868466 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad440550c0 con 0x7fad4c156a10 2023-10-08T06:54:49.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.630+0000 7fad52f3c640 1 -- 172.21.15.70:0/47868466 --> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fad4c071a20 con 0x7fad3006ad10 2023-10-08T06:54:49.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.749+0000 7fad427fc640 1 -- 172.21.15.70:0/47868466 <== mgr.7849 v2:172.21.15.181:6832/3979056493 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7fad48030d30 con 0x7fad3006ad10 2023-10-08T06:54:49.751 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-10-08T06:54:49.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.752+0000 7fad52f3c640 1 -- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7fad3006ad10 msgr2=0x7fad3006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:49.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.752+0000 7fad52f3c640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7fad3006ad10 0x7fad3006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fad48036000 tx=0x7fad4803b040 comp rx=0 tx=0).stop 2023-10-08T06:54:49.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.752+0000 7fad52f3c640 1 -- 172.21.15.70:0/47868466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 msgr2=0x7fad4c0ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T06:54:49.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.752+0000 7fad52f3c640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 0x7fad4c0ce480 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fad4400cd80 tx=0x7fad4400a650 comp rx=0 tx=0).stop 2023-10-08T06:54:49.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.752+0000 7fad52f3c640 1 -- 172.21.15.70:0/47868466 shutdown_connections 2023-10-08T06:54:49.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.752+0000 7fad52f3c640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad4c156a10 0x7fad4c0ce480 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:49.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.752+0000 7fad52f3c640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7fad3006ad10 0x7fad3006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:49.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.752+0000 7fad52f3c640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fad4c0b0680 0x7fad4c14ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:49.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.752+0000 7fad52f3c640 1 --2- 172.21.15.70:0/47868466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fad4c070ac0 0x7fad4c0b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T06:54:49.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.752+0000 7fad52f3c640 1 -- 172.21.15.70:0/47868466 >> 172.21.15.70:0/47868466 conn(0x7fad4c06c050 msgr2=0x7fad4c0cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T06:54:49.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.753+0000 7fad52f3c640 1 -- 172.21.15.70:0/47868466 shutdown_connections 2023-10-08T06:54:49.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T06:54:49.753+0000 7fad52f3c640 1 -- 172.21.15.70:0/47868466 wait complete. 2023-10-08T06:54:49.774 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-10-08T06:55:00.775 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:55:00.774+0000 7f8267704640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:55:00.775 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:55:00.774+0000 7f8267704640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:55:00.775 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:55:00.774+0000 7f8267704640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:55:00.775 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:55:00.774+0000 7f8267704640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:55:00.775 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T06:55:00.774+0000 7f8267704640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T06:55:00.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:55:00.976 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:55:31.183 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:55:31.192 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:56:01.329 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:56:01.340 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:56:31.522 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:56:31.532 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:57:01.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:57:01.718 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:57:31.920 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:57:31.930 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:58:02.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:58:02.135 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:58:32.318 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:58:32.329 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:59:02.544 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:59:02.554 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:59:32.715 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T06:59:32.724 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:00:02.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:00:02.900 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:00:33.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:00:33.098 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:01:03.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:01:03.242 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:01:33.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:01:33.579 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:02:03.705 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:02:03.715 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:02:33.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:02:33.869 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:03:04.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:03:04.022 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:03:34.148 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:03:34.158 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:04:04.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:04:04.353 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:04:34.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:04:34.573 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:05:04.715 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:05:04.725 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:05:15.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:05:15.776 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-08T07:05:15.842 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_55_00_UTC 2023-10-08T07:05:15.842 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_56_00_UTC 2023-10-08T07:05:15.842 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_57_00_UTC 2023-10-08T07:05:15.842 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_58_00_UTC 2023-10-08T07:05:15.842 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_59_00_UTC 2023-10-08T07:05:15.842 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_00_00_UTC 2023-10-08T07:05:15.842 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_01_00_UTC 2023-10-08T07:05:15.842 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_02_00_UTC 2023-10-08T07:05:15.843 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_03_00_UTC 2023-10-08T07:05:15.843 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_04_00_UTC 2023-10-08T07:05:15.843 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_05_00_UTC 2023-10-08T07:05:15.844 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-06_55_00_UTC', 'scheduled-2023-10-08-06_56_00_UTC', 'scheduled-2023-10-08-06_57_00_UTC', 'scheduled-2023-10-08-06_58_00_UTC', 'scheduled-2023-10-08-06_59_00_UTC', 'scheduled-2023-10-08-07_00_00_UTC', 'scheduled-2023-10-08-07_01_00_UTC', 'scheduled-2023-10-08-07_02_00_UTC', 'scheduled-2023-10-08-07_03_00_UTC', 'scheduled-2023-10-08-07_04_00_UTC', 'scheduled-2023-10-08-07_05_00_UTC'] 2023-10-08T07:05:15.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-10-08T07:05:16.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.063+0000 7efed8ec2640 1 Processor -- start 2023-10-08T07:05:16.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.064+0000 7efed8ec2640 1 -- start start 2023-10-08T07:05:16.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.065+0000 7efed8ec2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed4160660 0x7efed4070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.065+0000 7efed8ec2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efed40c9910 0x7efed40caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.065+0000 7efed8ec2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed406e150 con 0x7efed40cb450 2023-10-08T07:05:16.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.065+0000 7efed8ec2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed40cb8c0 con 0x7efed40c9910 2023-10-08T07:05:16.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.065+0000 7efed8ec2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed406de70 con 0x7efed4160660 2023-10-08T07:05:16.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.065+0000 7efed2d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efed40cb450 0x7efed40b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59516/0 (socket says 172.21.15.70:59516) 2023-10-08T07:05:16.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.066+0000 7efed2d9d640 1 -- 172.21.15.70:0/2980540397 learned_addr learned my addr 172.21.15.70:0/2980540397 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:05:16.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.066+0000 7efed1d9b640 1 --2- 172.21.15.70:0/2980540397 >> v2:172.21.15.181:3300/0 conn(0x7efed40c9910 0x7efed40caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.066+0000 7efed259c640 1 --2- 172.21.15.70:0/2980540397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed4160660 0x7efed4070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.066+0000 7efed159a640 1 -- 172.21.15.70:0/2980540397 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430627045 0 0) 0x7efed406e150 con 0x7efed40cb450 2023-10-08T07:05:16.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.066+0000 7efed159a640 1 -- 172.21.15.70:0/2980540397 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec0003520 con 0x7efed40cb450 2023-10-08T07:05:16.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.067+0000 7efed259c640 1 -- 172.21.15.70:0/2980540397 >> v2:172.21.15.181:3300/0 conn(0x7efed40c9910 msgr2=0x7efed40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:05:16.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.067+0000 7efed259c640 1 --2- 172.21.15.70:0/2980540397 >> v2:172.21.15.181:3300/0 conn(0x7efed40c9910 0x7efed40caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.067+0000 7efed259c640 1 -- 172.21.15.70:0/2980540397 >> v1:172.21.15.70:6789/0 conn(0x7efed40cb450 legacy=0x7efed40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.067+0000 7efed259c640 1 -- 172.21.15.70:0/2980540397 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40c8e30 con 0x7efed4160660 2023-10-08T07:05:16.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.067+0000 7efed1d9b640 1 --2- 172.21.15.70:0/2980540397 >> v2:172.21.15.181:3300/0 conn(0x7efed40c9910 0x7efed40caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:05:16.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.068+0000 7efed259c640 1 --2- 172.21.15.70:0/2980540397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed4160660 0x7efed4070cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efec40097a0 tx=0x7efec402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48651dbd2b73a07d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:05:16.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.077+0000 7efed159a640 1 -- 172.21.15.70:0/2980540397 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec402d040 con 0x7efed4160660 2023-10-08T07:05:16.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.077+0000 7efed159a640 1 -- 172.21.15.70:0/2980540397 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec4031810 con 0x7efed4160660 2023-10-08T07:05:16.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.077+0000 7efed159a640 1 -- 172.21.15.70:0/2980540397 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec4031ba0 con 0x7efed4160660 2023-10-08T07:05:16.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.078+0000 7efed8ec2640 1 -- 172.21.15.70:0/2980540397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed4160660 msgr2=0x7efed4070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.078+0000 7efed8ec2640 1 --2- 172.21.15.70:0/2980540397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed4160660 0x7efed4070cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efec40097a0 tx=0x7efec402b8b0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.078+0000 7efed8ec2640 1 -- 172.21.15.70:0/2980540397 shutdown_connections 2023-10-08T07:05:16.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.078+0000 7efed8ec2640 1 --2- 172.21.15.70:0/2980540397 >> v2:172.21.15.181:3300/0 conn(0x7efed40c9910 0x7efed40caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.078+0000 7efed8ec2640 1 --2- 172.21.15.70:0/2980540397 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed4160660 0x7efed4070cc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.078+0000 7efed8ec2640 1 -- 172.21.15.70:0/2980540397 >> 172.21.15.70:0/2980540397 conn(0x7efed406a5d0 msgr2=0x7efed40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:05:16.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.078+0000 7efed8ec2640 1 -- 172.21.15.70:0/2980540397 shutdown_connections 2023-10-08T07:05:16.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.078+0000 7efed8ec2640 1 -- 172.21.15.70:0/2980540397 wait complete. 2023-10-08T07:05:16.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.080+0000 7efed8ec2640 1 Processor -- start 2023-10-08T07:05:16.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.080+0000 7efed8ec2640 1 -- start start 2023-10-08T07:05:16.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.081+0000 7efed8ec2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed40c9910 0x7efed40c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.081+0000 7efed8ec2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed40cb450 0x7efed4149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.081+0000 7efed8ec2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4160660 0x7efed40cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.081+0000 7efed8ec2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed40abde0 con 0x7efed4160660 2023-10-08T07:05:16.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.081+0000 7efed8ec2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed40afac0 con 0x7efed40c9910 2023-10-08T07:05:16.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.081+0000 7efed8ec2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed40af500 con 0x7efed40cb450 2023-10-08T07:05:16.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.081+0000 7efed1d9b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed40cb450 0x7efed4149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.081+0000 7efed2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4160660 0x7efed40cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.082+0000 7efed2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4160660 0x7efed40cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34834/0 (socket says 172.21.15.70:34834) 2023-10-08T07:05:16.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.082+0000 7efed2d9d640 1 -- 172.21.15.70:0/1505944378 learned_addr learned my addr 172.21.15.70:0/1505944378 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:05:16.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.082+0000 7efed259c640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed40c9910 0x7efed40c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.082+0000 7efed2d9d640 1 -- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed40cb450 msgr2=0x7efed4149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.082+0000 7efed2d9d640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed40cb450 0x7efed4149a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.082+0000 7efed2d9d640 1 -- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed40c9910 msgr2=0x7efed40c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.082+0000 7efed2d9d640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed40c9910 0x7efed40c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.082+0000 7efed2d9d640 1 -- 172.21.15.70:0/1505944378 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc009400 con 0x7efed4160660 2023-10-08T07:05:16.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.082+0000 7efed2d9d640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4160660 0x7efed40cd690 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7efecc00cb40 tx=0x7efecc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:05:16.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.083+0000 7efecaffd640 1 -- 172.21.15.70:0/1505944378 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc00aca0 con 0x7efed4160660 2023-10-08T07:05:16.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.083+0000 7efecaffd640 1 -- 172.21.15.70:0/1505944378 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc00ae00 con 0x7efed4160660 2023-10-08T07:05:16.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.083+0000 7efed8ec2640 1 -- 172.21.15.70:0/1505944378 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec4009400 con 0x7efed4160660 2023-10-08T07:05:16.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.083+0000 7efecaffd640 1 -- 172.21.15.70:0/1505944378 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc012620 con 0x7efed4160660 2023-10-08T07:05:16.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.083+0000 7efed8ec2640 1 -- 172.21.15.70:0/1505944378 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed40aba00 con 0x7efed4160660 2023-10-08T07:05:16.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.084+0000 7efed8ec2640 1 -- 172.21.15.70:0/1505944378 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed40abf20 con 0x7efed4160660 2023-10-08T07:05:16.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.085+0000 7efecaffd640 1 -- 172.21.15.70:0/1505944378 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 105) v1 ==== 89360+0+0 (secure 0 0 0) 0x7efecc007070 con 0x7efed4160660 2023-10-08T07:05:16.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.086+0000 7efecaffd640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7efeb406afe0 0x7efeb406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.086+0000 7efecaffd640 1 -- 172.21.15.70:0/1505944378 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6489+0+0 (secure 0 0 0) 0x7efecc0899e0 con 0x7efed4160660 2023-10-08T07:05:16.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.086+0000 7efed259c640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7efeb406afe0 0x7efeb406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.087+0000 7efed259c640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7efeb406afe0 0x7efeb406d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efec4036000 tx=0x7efec403c040 comp rx=0 tx=0).ready entity=mgr.7849 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:05:16.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.091+0000 7efecaffd640 1 -- 172.21.15.70:0/1505944378 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efecc0547b0 con 0x7efed4160660 2023-10-08T07:05:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.300+0000 7efed8ec2640 1 -- 172.21.15.70:0/1505944378 --> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efed40c93f0 con 0x7efeb406afe0 2023-10-08T07:05:16.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.318+0000 7efecaffd640 1 -- 172.21.15.70:0/1505944378 <== mgr.7849 v2:172.21.15.181:6832/3979056493 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7efec4031ad0 con 0x7efeb406afe0 2023-10-08T07:05:16.320 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:05:16.320 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T06:54:49", "first": "2023-10-08T06:55:00", "last": "2023-10-08T07:05:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-08T07:05:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 -- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7efeb406afe0 msgr2=0x7efeb406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7efeb406afe0 0x7efeb406d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efec4036000 tx=0x7efec403c040 comp rx=0 tx=0).stop 2023-10-08T07:05:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 -- 172.21.15.70:0/1505944378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4160660 msgr2=0x7efed40cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4160660 0x7efed40cd690 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7efecc00cb40 tx=0x7efecc00a060 comp rx=0 tx=0).stop 2023-10-08T07:05:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 -- 172.21.15.70:0/1505944378 shutdown_connections 2023-10-08T07:05:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4160660 0x7efed40cd690 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7efeb406afe0 0x7efeb406d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efed40cb450 0x7efed4149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 --2- 172.21.15.70:0/1505944378 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efed40c9910 0x7efed40c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 -- 172.21.15.70:0/1505944378 >> 172.21.15.70:0/1505944378 conn(0x7efed406a5d0 msgr2=0x7efed4073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:05:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 -- 172.21.15.70:0/1505944378 shutdown_connections 2023-10-08T07:05:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.321+0000 7efed8ec2640 1 -- 172.21.15.70:0/1505944378 wait complete. 2023-10-08T07:05:16.341 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T06:54:49", "first": "2023-10-08T06:55:00", "last": "2023-10-08T07:05:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-08T07:05:16.341 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T06:54:49', 'first': '2023-10-08T06:55:00', 'last': '2023-10-08T07:05:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-10-08T07:05:16.341 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T07:05:16.341 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T07:05:16.341 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-08T07:05:16.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.550+0000 7f7a59c1a640 1 Processor -- start 2023-10-08T07:05:16.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.550+0000 7f7a59c1a640 1 -- start start 2023-10-08T07:05:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.552+0000 7f7a59c1a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a540cb580 0x7f7a54165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.552+0000 7f7a59c1a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5415b2b0 con 0x7f7a540c90b0 2023-10-08T07:05:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.552+0000 7f7a59c1a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5415b590 con 0x7f7a5415ab40 2023-10-08T07:05:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.552+0000 7f7a59c1a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5415bf30 con 0x7f7a540cb580 2023-10-08T07:05:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.552+0000 7f7a53fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a540cb580 0x7f7a54165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.552+0000 7f7a4bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7a540c90b0 0x7f7a540cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59520/0 (socket says 172.21.15.70:59520) 2023-10-08T07:05:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.552+0000 7f7a4bfff640 1 -- 172.21.15.70:0/1547902799 learned_addr learned my addr 172.21.15.70:0/1547902799 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:05:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.552+0000 7f7a53fff640 1 -- 172.21.15.70:0/1547902799 >> v1:172.21.15.181:6789/0 conn(0x7f7a5415ab40 legacy=0x7f7a540ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:05:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.552+0000 7f7a53fff640 1 -- 172.21.15.70:0/1547902799 >> v1:172.21.15.70:6789/0 conn(0x7f7a540c90b0 legacy=0x7f7a540cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.553+0000 7f7a53fff640 1 -- 172.21.15.70:0/1547902799 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5415b870 con 0x7f7a540cb580 2023-10-08T07:05:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.552+0000 7f7a52ffd640 1 -- 172.21.15.70:0/1547902799 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129688184 0 0) 0x7f7a5415b2b0 con 0x7f7a540c90b0 2023-10-08T07:05:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.554+0000 7f7a53fff640 1 --2- 172.21.15.70:0/1547902799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540cb580 0x7f7a54165b20 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009d90 tx=0x7f7a4c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f8479546684508 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:05:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.554+0000 7f7a52ffd640 1 -- 172.21.15.70:0/1547902799 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c02cc60 con 0x7f7a540cb580 2023-10-08T07:05:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.554+0000 7f7a52ffd640 1 -- 172.21.15.70:0/1547902799 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c02cdc0 con 0x7f7a540cb580 2023-10-08T07:05:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.554+0000 7f7a52ffd640 1 -- 172.21.15.70:0/1547902799 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c034660 con 0x7f7a540cb580 2023-10-08T07:05:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.554+0000 7f7a59c1a640 1 -- 172.21.15.70:0/1547902799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540cb580 msgr2=0x7f7a54165b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.554+0000 7f7a59c1a640 1 --2- 172.21.15.70:0/1547902799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540cb580 0x7f7a54165b20 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009d90 tx=0x7f7a4c02beb0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.555+0000 7f7a59c1a640 1 -- 172.21.15.70:0/1547902799 shutdown_connections 2023-10-08T07:05:16.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.555+0000 7f7a59c1a640 1 --2- 172.21.15.70:0/1547902799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540cb580 0x7f7a54165b20 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.555+0000 7f7a59c1a640 1 -- 172.21.15.70:0/1547902799 >> 172.21.15.70:0/1547902799 conn(0x7f7a5406a380 msgr2=0x7f7a540b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:05:16.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.555+0000 7f7a59c1a640 1 -- 172.21.15.70:0/1547902799 shutdown_connections 2023-10-08T07:05:16.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.555+0000 7f7a59c1a640 1 -- 172.21.15.70:0/1547902799 wait complete. 2023-10-08T07:05:16.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.556+0000 7f7a59c1a640 1 Processor -- start 2023-10-08T07:05:16.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.556+0000 7f7a59c1a640 1 -- start start 2023-10-08T07:05:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.557+0000 7f7a59c1a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540c90b0 0x7f7a54157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.557+0000 7f7a59c1a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a540cb580 0x7f7a540c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.557+0000 7f7a59c1a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a5415ab40 0x7f7a540c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.557+0000 7f7a59c1a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5415f7a0 con 0x7f7a540c90b0 2023-10-08T07:05:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.557+0000 7f7a59c1a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540c5c00 con 0x7f7a5415ab40 2023-10-08T07:05:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.557+0000 7f7a59c1a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5415dbb0 con 0x7f7a540cb580 2023-10-08T07:05:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540c90b0 0x7f7a54157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540c90b0 0x7f7a54157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34840/0 (socket says 172.21.15.70:34840) 2023-10-08T07:05:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a537fe640 1 -- 172.21.15.70:0/4028145689 learned_addr learned my addr 172.21.15.70:0/4028145689 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:05:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a4bfff640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a540cb580 0x7f7a540c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a537fe640 1 -- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a540cb580 msgr2=0x7f7a540c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a53fff640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a5415ab40 0x7f7a540c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a537fe640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a540cb580 0x7f7a540c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a537fe640 1 -- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a5415ab40 msgr2=0x7f7a540c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a537fe640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a5415ab40 0x7f7a540c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a537fe640 1 -- 172.21.15.70:0/4028145689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c009a70 con 0x7f7a540c90b0 2023-10-08T07:05:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.558+0000 7f7a537fe640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540c90b0 0x7f7a54157160 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400b420 tx=0x7f7a4400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:05:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.559+0000 7f7a50ff9640 1 -- 172.21.15.70:0/4028145689 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44011830 con 0x7f7a540c90b0 2023-10-08T07:05:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.559+0000 7f7a50ff9640 1 -- 172.21.15.70:0/4028145689 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a44011990 con 0x7f7a540c90b0 2023-10-08T07:05:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.559+0000 7f7a50ff9640 1 -- 172.21.15.70:0/4028145689 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44011d40 con 0x7f7a540c90b0 2023-10-08T07:05:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.559+0000 7f7a59c1a640 1 -- 172.21.15.70:0/4028145689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a54152b90 con 0x7f7a540c90b0 2023-10-08T07:05:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.559+0000 7f7a59c1a640 1 -- 172.21.15.70:0/4028145689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a540add90 con 0x7f7a540c90b0 2023-10-08T07:05:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.560+0000 7f7a59c1a640 1 -- 172.21.15.70:0/4028145689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a54067490 con 0x7f7a540c90b0 2023-10-08T07:05:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.561+0000 7f7a50ff9640 1 -- 172.21.15.70:0/4028145689 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 105) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f7a4401b460 con 0x7f7a540c90b0 2023-10-08T07:05:16.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.562+0000 7f7a50ff9640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f7a3406afe0 0x7f7a3406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:05:16.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.562+0000 7f7a50ff9640 1 -- 172.21.15.70:0/4028145689 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f7a44089450 con 0x7f7a540c90b0 2023-10-08T07:05:16.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.562+0000 7f7a4bfff640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f7a3406afe0 0x7f7a3406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:05:16.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.564+0000 7f7a4bfff640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f7a3406afe0 0x7f7a3406d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c007fd0 tx=0x7f7a3c009040 comp rx=0 tx=0).ready entity=mgr.7849 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:05:16.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.566+0000 7f7a50ff9640 1 -- 172.21.15.70:0/4028145689 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a44054170 con 0x7f7a540c90b0 2023-10-08T07:05:16.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.807+0000 7f7a59c1a640 1 -- 172.21.15.70:0/4028145689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7a54152b90 con 0x7f7a540c90b0 2023-10-08T07:05:16.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.815+0000 7f7a50ff9640 1 -- 172.21.15.70:0/4028145689 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v105) v1 ==== 74+0+171246 (secure 0 0 0) 0x7f7a54152b90 con 0x7f7a540c90b0 2023-10-08T07:05:16.819 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:05:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 -- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f7a3406afe0 msgr2=0x7f7a3406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f7a3406afe0 0x7f7a3406d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c007fd0 tx=0x7f7a3c009040 comp rx=0 tx=0).stop 2023-10-08T07:05:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 -- 172.21.15.70:0/4028145689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540c90b0 msgr2=0x7f7a54157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:05:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540c90b0 0x7f7a54157160 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400b420 tx=0x7f7a4400b8e0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 -- 172.21.15.70:0/4028145689 shutdown_connections 2023-10-08T07:05:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a5415ab40 0x7f7a540c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:6832/3979056493,v1:172.21.15.181:6833/3979056493] conn(0x7f7a3406afe0 0x7f7a3406d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a540cb580 0x7f7a540c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 --2- 172.21.15.70:0/4028145689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a540c90b0 0x7f7a54157160 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:05:16.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 -- 172.21.15.70:0/4028145689 >> 172.21.15.70:0/4028145689 conn(0x7f7a5406a380 msgr2=0x7f7a540c9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:05:16.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 -- 172.21.15.70:0/4028145689 shutdown_connections 2023-10-08T07:05:16.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:05:16.821+0000 7f7a59c1a640 1 -- 172.21.15.70:0/4028145689 wait complete. 2023-10-08T07:05:16.843 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":105,"active_gid":7849,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6832","nonce":3979056493},{"type":"v1","addr":"172.21.15.181:6833","nonce":3979056493}]},"active_addr":"172.21.15.181:6833/3979056493","active_change":"2023-10-08T06:54:47.949518+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":10245,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":199,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":4017559811}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":1286852011}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":340227645}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":2946222544}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":1250722759}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.181:0","nonce":849801254}]}]} 2023-10-08T07:05:16.848 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2023-10-08T07:05:16.848 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-08T07:05:16.848 INFO:tasks.ceph.mgr.x:Stopping old one... 2023-10-08T07:05:16.848 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-08T07:05:16.848 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:05:16.922 INFO:tasks.ceph.mgr.x:Stopped 2023-10-08T07:05:16.922 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-08T07:05:16.927 INFO:tasks.ceph.mgr.x:Started 2023-10-08T07:05:17.534 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:17.533+0000 7fe131211040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:05:17.586 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:17.585+0000 7fe131211040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:05:18.032 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.031+0000 7fe131211040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:05:18.084 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.082+0000 7fe131211040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:05:18.134 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.132+0000 7fe131211040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:05:18.194 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.193+0000 7fe131211040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:05:18.243 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.241+0000 7fe131211040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:05:18.354 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.352+0000 7fe131211040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:05:18.416 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.414+0000 7fe131211040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:05:18.464 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.463+0000 7fe131211040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:05:18.521 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.519+0000 7fe131211040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:05:18.570 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.569+0000 7fe131211040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:05:18.950 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:18.948+0000 7fe131211040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:05:19.108 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:19.107+0000 7fe131211040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:05:19.217 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:19.216+0000 7fe131211040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:05:19.267 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:19.266+0000 7fe131211040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:05:19.374 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:19.373+0000 7fe131211040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:05:19.481 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:19.480+0000 7fe131211040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:05:19.638 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:19.636+0000 7fe131211040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:05:19.687 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:19.686+0000 7fe131211040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:05:19.805 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:19.804+0000 7fe131211040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:05:20.639 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:20.637+0000 7fe131211040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:05:20.934 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:20.933+0000 7fe131211040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:05:20.990 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:20.989+0000 7fe131211040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:05:21.108 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.107+0000 7fe131211040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:05:21.436 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.435+0000 7fe0afc51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:05:21.437 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.435+0000 7fe0afc51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:05:21.437 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.435+0000 7fe0afc51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:05:21.437 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.435+0000 7fe0afc51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:05:21.437 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.435+0000 7fe0afc51640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:05:21.437 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.436+0000 7fe0adc4d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:05:21.438 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.436+0000 7fe0adc4d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:05:21.438 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.436+0000 7fe0adc4d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:05:21.438 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.436+0000 7fe0adc4d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:05:21.438 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:05:21.436+0000 7fe0adc4d640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:05:34.860 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:05:34.870 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:06:00.435 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:06:00.434+0000 7fe0b8ca3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:06:00.435 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:06:00.434+0000 7fe0b8ca3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:06:00.436 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:06:00.434+0000 7fe0b8ca3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:06:00.436 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:06:00.434+0000 7fe0b8ca3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:06:00.436 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:06:00.434+0000 7fe0b8ca3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:06:05.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:06:05.050 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:06:35.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:06:35.227 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:07:05.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:07:05.387 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:07:35.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:07:35.590 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:08:05.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:08:05.810 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:08:36.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:08:36.012 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:09:06.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:09:06.225 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:09:36.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:09:36.385 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:10:06.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:10:06.559 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:10:16.928 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/test_restart 1m 2023-10-08T07:10:17.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.162+0000 7fa38c925640 1 Processor -- start 2023-10-08T07:10:17.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.162+0000 7fa38c925640 1 -- start start 2023-10-08T07:10:17.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.163+0000 7fa38c925640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa38815de50 0x7fa3880b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.164+0000 7fa38c925640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3880cb850 0x7fa3880b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.164+0000 7fa38c925640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3880b11a0 con 0x7fa38815e230 2023-10-08T07:10:17.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.164+0000 7fa38c925640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa38815f440 con 0x7fa38815de50 2023-10-08T07:10:17.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.164+0000 7fa38c925640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880b17b0 con 0x7fa3880cb850 2023-10-08T07:10:17.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.164+0000 7fa37ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa38815e230 0x7fa3880cb140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41838/0 (socket says 172.21.15.70:41838) 2023-10-08T07:10:17.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.164+0000 7fa37ffff640 1 -- 172.21.15.70:0/2674801033 learned_addr learned my addr 172.21.15.70:0/2674801033 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:17.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.164+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890791050 0 0) 0x7fa3880b11a0 con 0x7fa38815e230 2023-10-08T07:10:17.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.164+0000 7fa386d9d640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3880cb850 0x7fa3880b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.164+0000 7fa38659c640 1 --2- 172.21.15.70:0/2674801033 >> v2:172.21.15.181:3300/0 conn(0x7fa38815de50 0x7fa3880b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.164+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa374003520 con 0x7fa38815e230 2023-10-08T07:10:17.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4283369624 0 0) 0x7fa374003520 con 0x7fa38815e230 2023-10-08T07:10:17.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3880cb850 msgr2=0x7fa3880b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:17.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3880cb850 0x7fa3880b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 >> v2:172.21.15.181:3300/0 conn(0x7fa38815de50 msgr2=0x7fa3880b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 --2- 172.21.15.70:0/2674801033 >> v2:172.21.15.181:3300/0 conn(0x7fa38815de50 0x7fa3880b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38815e7c0 con 0x7fa38815e230 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa38659c640 1 --2- 172.21.15.70:0/2674801033 >> v2:172.21.15.181:3300/0 conn(0x7fa38815de50 0x7fa3880b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa370001670 con 0x7fa38815e230 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 >> v1:172.21.15.70:6789/0 conn(0x7fa38815e230 legacy=0x7fa3880cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3880cb850 0x7fa374003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa374003ef0 0x7fa3740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa374004d00 0x7fa374005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38815f440 con 0x7fa374004d00 2023-10-08T07:10:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880b17b0 con 0x7fa374003ef0 2023-10-08T07:10:17.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37400ba80 con 0x7fa3880cb850 2023-10-08T07:10:17.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.165+0000 7fa386d9d640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa374004d00 0x7fa374005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.166+0000 7fa37ffff640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa374003ef0 0x7fa3740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.166+0000 7fa386d9d640 1 -- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3880cb850 msgr2=0x7fa374003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:17.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.166+0000 7fa386d9d640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3880cb850 0x7fa374003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.166+0000 7fa386d9d640 1 -- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa374003ef0 msgr2=0x7fa3740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:17.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.166+0000 7fa386d9d640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa374003ef0 0x7fa3740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.166+0000 7fa386d9d640 1 -- 172.21.15.70:0/2674801033 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374003520 con 0x7fa374004d00 2023-10-08T07:10:17.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.166+0000 7fa37ffff640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa374003ef0 0x7fa3740047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:17.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.167+0000 7fa386d9d640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa374004d00 0x7fa374005180 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fa380009fc0 tx=0x7fa3740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8fa65fe1ac096f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:17.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.167+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38002cbf0 con 0x7fa374004d00 2023-10-08T07:10:17.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.167+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38000f040 con 0x7fa374004d00 2023-10-08T07:10:17.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.167+0000 7fa385d9b640 1 -- 172.21.15.70:0/2674801033 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380034440 con 0x7fa374004d00 2023-10-08T07:10:17.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.168+0000 7fa38c925640 1 -- 172.21.15.70:0/2674801033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa374004d00 msgr2=0x7fa374005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:17.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.168+0000 7fa38c925640 1 --2- 172.21.15.70:0/2674801033 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa374004d00 0x7fa374005180 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fa380009fc0 tx=0x7fa3740057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.168+0000 7fa386d9d640 1 -- 172.21.15.70:0/2674801033 reap_dead start 2023-10-08T07:10:17.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.168+0000 7fa38c925640 1 -- 172.21.15.70:0/2674801033 shutdown_connections 2023-10-08T07:10:17.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.168+0000 7fa38c925640 1 -- 172.21.15.70:0/2674801033 >> 172.21.15.70:0/2674801033 conn(0x7fa388069d50 msgr2=0x7fa38806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:17.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.168+0000 7fa38c925640 1 -- 172.21.15.70:0/2674801033 shutdown_connections 2023-10-08T07:10:17.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.168+0000 7fa38c925640 1 -- 172.21.15.70:0/2674801033 wait complete. 2023-10-08T07:10:17.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.170+0000 7fa38c925640 1 Processor -- start 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.170+0000 7fa38c925640 1 -- start start 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.171+0000 7fa38c925640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa374004d00 0x7fa3881525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.171+0000 7fa38c925640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3880c8100 0x7fa3880c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.171+0000 7fa38c925640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa388151c20 0x7fa38815b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.171+0000 7fa38c925640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880c6c80 con 0x7fa388151c20 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.171+0000 7fa38c925640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880c5d60 con 0x7fa3880c8100 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.171+0000 7fa38c925640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880c6000 con 0x7fa374004d00 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.171+0000 7fa386d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa388151c20 0x7fa38815b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.171+0000 7fa386d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa388151c20 0x7fa38815b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60866/0 (socket says 172.21.15.70:60866) 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.171+0000 7fa386d9d640 1 -- 172.21.15.70:0/1960850151 learned_addr learned my addr 172.21.15.70:0/1960850151 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa37ffff640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3880c8100 0x7fa3880c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa38659c640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa374004d00 0x7fa3881525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa386d9d640 1 -- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa374004d00 msgr2=0x7fa3881525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:17.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa386d9d640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa374004d00 0x7fa3881525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa386d9d640 1 -- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3880c8100 msgr2=0x7fa3880c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:17.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa386d9d640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3880c8100 0x7fa3880c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa386d9d640 1 -- 172.21.15.70:0/1960850151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3880c62e0 con 0x7fa388151c20 2023-10-08T07:10:17.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa386d9d640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa388151c20 0x7fa38815b5f0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fa380037040 tx=0x7fa380009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:17.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa37effd640 1 -- 172.21.15.70:0/1960850151 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380034cd0 con 0x7fa388151c20 2023-10-08T07:10:17.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa37effd640 1 -- 172.21.15.70:0/1960850151 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38002ce90 con 0x7fa388151c20 2023-10-08T07:10:17.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.172+0000 7fa38c925640 1 -- 172.21.15.70:0/1960850151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3880c65c0 con 0x7fa388151c20 2023-10-08T07:10:17.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.173+0000 7fa37effd640 1 -- 172.21.15.70:0/1960850151 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38003e680 con 0x7fa388151c20 2023-10-08T07:10:17.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.173+0000 7fa38c925640 1 -- 172.21.15.70:0/1960850151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3881520a0 con 0x7fa388151c20 2023-10-08T07:10:17.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.174+0000 7fa38c925640 1 -- 172.21.15.70:0/1960850151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3880ce350 con 0x7fa388151c20 2023-10-08T07:10:17.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.175+0000 7fa37effd640 1 -- 172.21.15.70:0/1960850151 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 111) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fa38003e7e0 con 0x7fa388151c20 2023-10-08T07:10:17.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.176+0000 7fa37effd640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa36806afe0 0x7fa36806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.176+0000 7fa37effd640 1 -- 172.21.15.70:0/1960850151 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fa3800abb10 con 0x7fa388151c20 2023-10-08T07:10:17.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.176+0000 7fa38659c640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa36806afe0 0x7fa36806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.177+0000 7fa38659c640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa36806afe0 0x7fa36806d4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa37800d8d0 tx=0x7fa378008040 comp rx=0 tx=0).ready entity=mgr.7040 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:17.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.180+0000 7fa37effd640 1 -- 172.21.15.70:0/1960850151 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3800766f0 con 0x7fa388151c20 2023-10-08T07:10:17.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.392+0000 7fa38c925640 1 -- 172.21.15.70:0/1960850151 --> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fa388152260 con 0x7fa36806afe0 2023-10-08T07:10:17.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.543+0000 7fa37effd640 1 -- 172.21.15.70:0/1960850151 <== mgr.7040 v2:172.21.15.181:6832/2291259069 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fa388152260 con 0x7fa36806afe0 2023-10-08T07:10:17.545 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-08T07:10:17.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.546+0000 7fa38c925640 1 -- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa36806afe0 msgr2=0x7fa36806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:17.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.546+0000 7fa38c925640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa36806afe0 0x7fa36806d4a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa37800d8d0 tx=0x7fa378008040 comp rx=0 tx=0).stop 2023-10-08T07:10:17.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.546+0000 7fa38c925640 1 -- 172.21.15.70:0/1960850151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa388151c20 msgr2=0x7fa38815b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:17.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.546+0000 7fa38c925640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa388151c20 0x7fa38815b5f0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fa380037040 tx=0x7fa380009750 comp rx=0 tx=0).stop 2023-10-08T07:10:17.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.546+0000 7fa38c925640 1 -- 172.21.15.70:0/1960850151 shutdown_connections 2023-10-08T07:10:17.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.546+0000 7fa38c925640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa36806afe0 0x7fa36806d4a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.546+0000 7fa38c925640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa388151c20 0x7fa38815b5f0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.546+0000 7fa38c925640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa3880c8100 0x7fa3880c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.546+0000 7fa38c925640 1 --2- 172.21.15.70:0/1960850151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa374004d00 0x7fa3881525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.546+0000 7fa38c925640 1 -- 172.21.15.70:0/1960850151 >> 172.21.15.70:0/1960850151 conn(0x7fa388069d50 msgr2=0x7fa38815d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:17.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.547+0000 7fa38c925640 1 -- 172.21.15.70:0/1960850151 shutdown_connections 2023-10-08T07:10:17.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.547+0000 7fa38c925640 1 -- 172.21.15.70:0/1960850151 wait complete. 2023-10-08T07:10:17.567 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-08T07:10:17.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:17.568 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-08T07:10:17.625 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_55_00_UTC 2023-10-08T07:10:17.625 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_56_00_UTC 2023-10-08T07:10:17.625 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_57_00_UTC 2023-10-08T07:10:17.625 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_58_00_UTC 2023-10-08T07:10:17.626 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_59_00_UTC 2023-10-08T07:10:17.626 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_00_00_UTC 2023-10-08T07:10:17.626 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_01_00_UTC 2023-10-08T07:10:17.626 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_02_00_UTC 2023-10-08T07:10:17.626 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_03_00_UTC 2023-10-08T07:10:17.626 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_04_00_UTC 2023-10-08T07:10:17.626 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_05_00_UTC 2023-10-08T07:10:17.626 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_06_00_UTC 2023-10-08T07:10:17.627 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_07_00_UTC 2023-10-08T07:10:17.627 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_08_00_UTC 2023-10-08T07:10:17.627 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_09_00_UTC 2023-10-08T07:10:17.627 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_10_00_UTC 2023-10-08T07:10:17.630 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-06_55_00_UTC', 'scheduled-2023-10-08-06_56_00_UTC', 'scheduled-2023-10-08-06_57_00_UTC', 'scheduled-2023-10-08-06_58_00_UTC', 'scheduled-2023-10-08-06_59_00_UTC', 'scheduled-2023-10-08-07_00_00_UTC', 'scheduled-2023-10-08-07_01_00_UTC', 'scheduled-2023-10-08-07_02_00_UTC', 'scheduled-2023-10-08-07_03_00_UTC', 'scheduled-2023-10-08-07_04_00_UTC', 'scheduled-2023-10-08-07_05_00_UTC', 'scheduled-2023-10-08-07_06_00_UTC', 'scheduled-2023-10-08-07_07_00_UTC', 'scheduled-2023-10-08-07_08_00_UTC', 'scheduled-2023-10-08-07_09_00_UTC', 'scheduled-2023-10-08-07_10_00_UTC'] 2023-10-08T07:10:17.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-10-08T07:10:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.843+0000 7fac5e013640 1 Processor -- start 2023-10-08T07:10:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.844+0000 7fac5e013640 1 -- start start 2023-10-08T07:10:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.845+0000 7fac5e013640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac5815a820 0x7fac580aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.845+0000 7fac5e013640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac580ca680 0x7fac580b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.845+0000 7fac5e013640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fac5815ac00 0x7fac580c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.845+0000 7fac5e013640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac580cab60 con 0x7fac580ca680 2023-10-08T07:10:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.845+0000 7fac5e013640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac580cb960 con 0x7fac5815ac00 2023-10-08T07:10:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.845+0000 7fac5e013640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac580cbdf0 con 0x7fac5815a820 2023-10-08T07:10:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.845+0000 7fac5c810640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac580ca680 0x7fac580b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.845+0000 7fac5c810640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac580ca680 0x7fac580b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60878/0 (socket says 172.21.15.70:60878) 2023-10-08T07:10:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.845+0000 7fac5c810640 1 -- 172.21.15.70:0/2041634961 learned_addr learned my addr 172.21.15.70:0/2041634961 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.846+0000 7fac5c810640 1 -- 172.21.15.70:0/2041634961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac5815a820 msgr2=0x7fac580aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.846+0000 7fac5c810640 1 --2- 172.21.15.70:0/2041634961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac5815a820 0x7fac580aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.846+0000 7fac5c810640 1 -- 172.21.15.70:0/2041634961 >> v2:172.21.15.181:3300/0 conn(0x7fac5815ac00 msgr2=0x7fac580c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.846+0000 7fac5d812640 1 --2- 172.21.15.70:0/2041634961 >> v2:172.21.15.181:3300/0 conn(0x7fac5815ac00 0x7fac580c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.846+0000 7fac5c810640 1 --2- 172.21.15.70:0/2041634961 >> v2:172.21.15.181:3300/0 conn(0x7fac5815ac00 0x7fac580c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.846+0000 7fac5c810640 1 -- 172.21.15.70:0/2041634961 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac580cb030 con 0x7fac580ca680 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.847+0000 7fac5c810640 1 --2- 172.21.15.70:0/2041634961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac580ca680 0x7fac580b04b0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fac580ce340 tx=0x7fac4802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377e2c999b4c2d17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.847+0000 7fac4ffff640 1 -- 172.21.15.70:0/2041634961 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac480317e0 con 0x7fac580ca680 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.847+0000 7fac4ffff640 1 -- 172.21.15.70:0/2041634961 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac48031940 con 0x7fac580ca680 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.847+0000 7fac4ffff640 1 -- 172.21.15.70:0/2041634961 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48031cd0 con 0x7fac580ca680 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.848+0000 7fac5e013640 1 -- 172.21.15.70:0/2041634961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac580ca680 msgr2=0x7fac580b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.848+0000 7fac5e013640 1 --2- 172.21.15.70:0/2041634961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac580ca680 0x7fac580b04b0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fac580ce340 tx=0x7fac4802b880 comp rx=0 tx=0).stop 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.848+0000 7fac5e013640 1 -- 172.21.15.70:0/2041634961 shutdown_connections 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.848+0000 7fac5e013640 1 --2- 172.21.15.70:0/2041634961 >> v2:172.21.15.181:3300/0 conn(0x7fac5815ac00 0x7fac580c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.848+0000 7fac5e013640 1 --2- 172.21.15.70:0/2041634961 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac580ca680 0x7fac580b04b0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.848+0000 7fac5e013640 1 --2- 172.21.15.70:0/2041634961 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac5815a820 0x7fac580aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.848+0000 7fac5e013640 1 -- 172.21.15.70:0/2041634961 >> 172.21.15.70:0/2041634961 conn(0x7fac5806c050 msgr2=0x7fac5806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.848+0000 7fac5e013640 1 -- 172.21.15.70:0/2041634961 shutdown_connections 2023-10-08T07:10:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.848+0000 7fac5e013640 1 -- 172.21.15.70:0/2041634961 wait complete. 2023-10-08T07:10:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.850+0000 7fac5e013640 1 Processor -- start 2023-10-08T07:10:17.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.850+0000 7fac5e013640 1 -- start start 2023-10-08T07:10:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.851+0000 7fac5e013640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac580ca680 0x7fac58152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.851+0000 7fac5e013640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac5815a820 0x7fac581f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.851+0000 7fac5e013640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5815ac00 0x7fac5815d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.851+0000 7fac5e013640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac581f2030 con 0x7fac5815ac00 2023-10-08T07:10:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.851+0000 7fac5e013640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac58153080 con 0x7fac5815a820 2023-10-08T07:10:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.851+0000 7fac5e013640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac581f3720 con 0x7fac580ca680 2023-10-08T07:10:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.851+0000 7fac5d812640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5815ac00 0x7fac5815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.851+0000 7fac5d812640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5815ac00 0x7fac5815d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60880/0 (socket says 172.21.15.70:60880) 2023-10-08T07:10:17.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.851+0000 7fac5d812640 1 -- 172.21.15.70:0/4198321679 learned_addr learned my addr 172.21.15.70:0/4198321679 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:17.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac5c810640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac5815a820 0x7fac581f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac5d812640 1 -- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac580ca680 msgr2=0x7fac58152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac5d812640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac580ca680 0x7fac58152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac5d812640 1 -- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac5815a820 msgr2=0x7fac581f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac5d812640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac5815a820 0x7fac581f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac5d812640 1 -- 172.21.15.70:0/4198321679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac480094d0 con 0x7fac5815ac00 2023-10-08T07:10:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac5d812640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5815ac00 0x7fac5815d660 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fac5000cbd0 tx=0x7fac5000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac4dffb640 1 -- 172.21.15.70:0/4198321679 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5000ccc0 con 0x7fac5815ac00 2023-10-08T07:10:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac4dffb640 1 -- 172.21.15.70:0/4198321679 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5000ce20 con 0x7fac5815ac00 2023-10-08T07:10:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac5e013640 1 -- 172.21.15.70:0/4198321679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac580af470 con 0x7fac5815ac00 2023-10-08T07:10:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac4dffb640 1 -- 172.21.15.70:0/4198321679 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50003bb0 con 0x7fac5815ac00 2023-10-08T07:10:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.852+0000 7fac5e013640 1 -- 172.21.15.70:0/4198321679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac580afa30 con 0x7fac5815ac00 2023-10-08T07:10:17.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.853+0000 7fac5e013640 1 -- 172.21.15.70:0/4198321679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac581f2170 con 0x7fac5815ac00 2023-10-08T07:10:17.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.854+0000 7fac4dffb640 1 -- 172.21.15.70:0/4198321679 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 111) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fac5000f3f0 con 0x7fac5815ac00 2023-10-08T07:10:17.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.855+0000 7fac4dffb640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fac3c06afe0 0x7fac3c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:17.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.855+0000 7fac4dffb640 1 -- 172.21.15.70:0/4198321679 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fac5008a0b0 con 0x7fac5815ac00 2023-10-08T07:10:17.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.856+0000 7fac5d011640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fac3c06afe0 0x7fac3c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:17.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.857+0000 7fac5d011640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fac3c06afe0 0x7fac3c06d4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fac54009710 tx=0x7fac54006c90 comp rx=0 tx=0).ready entity=mgr.7040 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:17.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:17.860+0000 7fac4dffb640 1 -- 172.21.15.70:0/4198321679 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac581f2170 con 0x7fac5815ac00 2023-10-08T07:10:18.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.072+0000 7fac5e013640 1 -- 172.21.15.70:0/4198321679 --> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac580cb5f0 con 0x7fac3c06afe0 2023-10-08T07:10:18.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.089+0000 7fac4dffb640 1 -- 172.21.15.70:0/4198321679 <== mgr.7040 v2:172.21.15.181:6832/2291259069 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7fac5400b480 con 0x7fac3c06afe0 2023-10-08T07:10:18.091 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:18.091 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T06:54:49", "first": "2023-10-08T06:55:00", "last": "2023-10-08T07:10:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-08T07:10:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 -- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fac3c06afe0 msgr2=0x7fac3c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fac3c06afe0 0x7fac3c06d4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fac54009710 tx=0x7fac54006c90 comp rx=0 tx=0).stop 2023-10-08T07:10:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 -- 172.21.15.70:0/4198321679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5815ac00 msgr2=0x7fac5815d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5815ac00 0x7fac5815d660 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fac5000cbd0 tx=0x7fac5000a5c0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 -- 172.21.15.70:0/4198321679 shutdown_connections 2023-10-08T07:10:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fac3c06afe0 0x7fac3c06d4a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5815ac00 0x7fac5815d660 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac5815a820 0x7fac581f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 --2- 172.21.15.70:0/4198321679 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac580ca680 0x7fac58152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 -- 172.21.15.70:0/4198321679 >> 172.21.15.70:0/4198321679 conn(0x7fac5806c050 msgr2=0x7fac580cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 -- 172.21.15.70:0/4198321679 shutdown_connections 2023-10-08T07:10:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.092+0000 7fac5e013640 1 -- 172.21.15.70:0/4198321679 wait complete. 2023-10-08T07:10:18.112 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1m", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T06:54:49", "first": "2023-10-08T06:55:00", "last": "2023-10-08T07:10:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-08T07:10:18.113 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1m', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T06:54:49', 'first': '2023-10-08T06:55:00', 'last': '2023-10-08T07:10:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-10-08T07:10:18.113 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-10-08T07:10:18.113 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-10-08T07:10:18.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/test_restart 1m 2023-10-08T07:10:18.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.330+0000 7f3128002640 1 Processor -- start 2023-10-08T07:10:18.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.330+0000 7f3128002640 1 -- start start 2023-10-08T07:10:18.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.331+0000 7f3128002640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3120070cc0 0x7f31200caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:18.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.331+0000 7f3128002640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31200cb450 0x7f31200b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:18.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.331+0000 7f3128002640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3120071140 con 0x7f3120160660 2023-10-08T07:10:18.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.331+0000 7f3128002640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312006e150 con 0x7f31200cb450 2023-10-08T07:10:18.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.331+0000 7f3128002640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31200cb8c0 con 0x7f3120070cc0 2023-10-08T07:10:18.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.332+0000 7f312659f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31200cb450 0x7f31200b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:18.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.332+0000 7f312659f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31200cb450 0x7f31200b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60894/0 (socket says 172.21.15.70:60894) 2023-10-08T07:10:18.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.332+0000 7f312659f640 1 -- 172.21.15.70:0/3520755192 learned_addr learned my addr 172.21.15.70:0/3520755192 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:18.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.332+0000 7f312559d640 1 --2- 172.21.15.70:0/3520755192 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3120070cc0 0x7f31200caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:18.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.332+0000 7f312659f640 1 -- 172.21.15.70:0/3520755192 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3120070cc0 msgr2=0x7f31200caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:18.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.332+0000 7f312659f640 1 --2- 172.21.15.70:0/3520755192 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3120070cc0 0x7f31200caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.332+0000 7f312659f640 1 -- 172.21.15.70:0/3520755192 >> v1:172.21.15.181:6789/0 conn(0x7f3120160660 legacy=0x7f31200c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:18.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.332+0000 7f312659f640 1 -- 172.21.15.70:0/3520755192 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312006de70 con 0x7f31200cb450 2023-10-08T07:10:18.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.334+0000 7f312659f640 1 --2- 172.21.15.70:0/3520755192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31200cb450 0x7f31200b1990 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f3118009e50 tx=0x7f311802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef89d1cb44358f2b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.334+0000 7f3124d9c640 1 -- 172.21.15.70:0/3520755192 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311802ca70 con 0x7f31200cb450 2023-10-08T07:10:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.334+0000 7f3124d9c640 1 -- 172.21.15.70:0/3520755192 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311802cbd0 con 0x7f31200cb450 2023-10-08T07:10:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.334+0000 7f3124d9c640 1 -- 172.21.15.70:0/3520755192 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180343f0 con 0x7f31200cb450 2023-10-08T07:10:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.335+0000 7f3128002640 1 -- 172.21.15.70:0/3520755192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31200cb450 msgr2=0x7f31200b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.335+0000 7f3128002640 1 --2- 172.21.15.70:0/3520755192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31200cb450 0x7f31200b1990 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f3118009e50 tx=0x7f311802bed0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.335+0000 7f3128002640 1 -- 172.21.15.70:0/3520755192 shutdown_connections 2023-10-08T07:10:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.335+0000 7f3128002640 1 --2- 172.21.15.70:0/3520755192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31200cb450 0x7f31200b1990 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.335+0000 7f3128002640 1 --2- 172.21.15.70:0/3520755192 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3120070cc0 0x7f31200caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.335+0000 7f3128002640 1 -- 172.21.15.70:0/3520755192 >> 172.21.15.70:0/3520755192 conn(0x7f312006a5d0 msgr2=0x7f31200b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.335+0000 7f3128002640 1 -- 172.21.15.70:0/3520755192 shutdown_connections 2023-10-08T07:10:18.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.335+0000 7f3128002640 1 -- 172.21.15.70:0/3520755192 wait complete. 2023-10-08T07:10:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.337+0000 7f3128002640 1 Processor -- start 2023-10-08T07:10:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.337+0000 7f3128002640 1 -- start start 2023-10-08T07:10:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.338+0000 7f3128002640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3120070cc0 0x7f31200c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.338+0000 7f3128002640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31200cb450 0x7f31200c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.338+0000 7f3128002640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3120160660 0x7f31201f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:18.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.338+0000 7f3128002640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31201f6470 con 0x7f3120070cc0 2023-10-08T07:10:18.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.338+0000 7f3128002640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31201555b0 con 0x7f31200cb450 2023-10-08T07:10:18.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.338+0000 7f3128002640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3120155850 con 0x7f3120160660 2023-10-08T07:10:18.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f3125d9e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3120070cc0 0x7f31200c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:18.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f3125d9e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3120070cc0 0x7f31200c1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60896/0 (socket says 172.21.15.70:60896) 2023-10-08T07:10:18.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f3125d9e640 1 -- 172.21.15.70:0/2525770400 learned_addr learned my addr 172.21.15.70:0/2525770400 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:18.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f312659f640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3120160660 0x7f31201f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:18.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f312559d640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31200cb450 0x7f31200c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:18.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f3125d9e640 1 -- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3120160660 msgr2=0x7f31201f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:18.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f3125d9e640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3120160660 0x7f31201f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f3125d9e640 1 -- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31200cb450 msgr2=0x7f31200c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:18.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f3125d9e640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31200cb450 0x7f31200c2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f3125d9e640 1 -- 172.21.15.70:0/2525770400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118009b30 con 0x7f3120070cc0 2023-10-08T07:10:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.339+0000 7f3125d9e640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3120070cc0 0x7f31200c1a90 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f311c00b3f0 tx=0x7f311c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.340+0000 7f31167fc640 1 -- 172.21.15.70:0/2525770400 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c011830 con 0x7f3120070cc0 2023-10-08T07:10:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.340+0000 7f31167fc640 1 -- 172.21.15.70:0/2525770400 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c011990 con 0x7f3120070cc0 2023-10-08T07:10:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.340+0000 7f31167fc640 1 -- 172.21.15.70:0/2525770400 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c011d40 con 0x7f3120070cc0 2023-10-08T07:10:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.340+0000 7f3128002640 1 -- 172.21.15.70:0/2525770400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31201f6100 con 0x7f3120070cc0 2023-10-08T07:10:18.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.340+0000 7f3128002640 1 -- 172.21.15.70:0/2525770400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31200c2eb0 con 0x7f3120070cc0 2023-10-08T07:10:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.341+0000 7f3128002640 1 -- 172.21.15.70:0/2525770400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312004e160 con 0x7f3120070cc0 2023-10-08T07:10:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.342+0000 7f31167fc640 1 -- 172.21.15.70:0/2525770400 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 111) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f311c01a400 con 0x7f3120070cc0 2023-10-08T07:10:18.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.343+0000 7f31167fc640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f310406afe0 0x7f310406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:18.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.343+0000 7f31167fc640 1 -- 172.21.15.70:0/2525770400 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f311c089ab0 con 0x7f3120070cc0 2023-10-08T07:10:18.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.344+0000 7f312559d640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f310406afe0 0x7f310406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:18.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.345+0000 7f312559d640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f310406afe0 0x7f310406d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f31200c5780 tx=0x7f3110008040 comp rx=0 tx=0).ready entity=mgr.7040 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:18.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.347+0000 7f31167fc640 1 -- 172.21.15.70:0/2525770400 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f311c0545e0 con 0x7f3120070cc0 2023-10-08T07:10:18.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.557+0000 7f3128002640 1 -- 172.21.15.70:0/2525770400 --> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/test_restart", "repeat": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f31200c9110 con 0x7f310406afe0 2023-10-08T07:10:18.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.720+0000 7f31167fc640 1 -- 172.21.15.70:0/2525770400 <== mgr.7040 v2:172.21.15.181:6832/2291259069 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f3110009520 con 0x7f310406afe0 2023-10-08T07:10:18.722 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-10-08T07:10:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.724+0000 7f3128002640 1 -- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f310406afe0 msgr2=0x7f310406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.724+0000 7f3128002640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f310406afe0 0x7f310406d4a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f31200c5780 tx=0x7f3110008040 comp rx=0 tx=0).stop 2023-10-08T07:10:18.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.724+0000 7f3128002640 1 -- 172.21.15.70:0/2525770400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3120070cc0 msgr2=0x7f31200c1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.724+0000 7f3128002640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3120070cc0 0x7f31200c1a90 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f311c00b3f0 tx=0x7f311c00b8b0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.724+0000 7f3128002640 1 -- 172.21.15.70:0/2525770400 shutdown_connections 2023-10-08T07:10:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.724+0000 7f3128002640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f310406afe0 0x7f310406d4a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.724+0000 7f3128002640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3120160660 0x7f31201f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.724+0000 7f3128002640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f31200cb450 0x7f31200c2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.724+0000 7f3128002640 1 --2- 172.21.15.70:0/2525770400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3120070cc0 0x7f31200c1a90 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.724+0000 7f3128002640 1 -- 172.21.15.70:0/2525770400 >> 172.21.15.70:0/2525770400 conn(0x7f312006a5d0 msgr2=0x7f312015f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.725+0000 7f3128002640 1 -- 172.21.15.70:0/2525770400 shutdown_connections 2023-10-08T07:10:18.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:18.725+0000 7f3128002640 1 -- 172.21.15.70:0/2525770400 wait complete. 2023-10-08T07:10:18.746 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-10-08T07:10:18.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:18.747 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-08T07:10:18.797 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_55_00_UTC 2023-10-08T07:10:18.797 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_56_00_UTC 2023-10-08T07:10:18.798 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_57_00_UTC 2023-10-08T07:10:18.798 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_58_00_UTC 2023-10-08T07:10:18.798 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-06_59_00_UTC 2023-10-08T07:10:18.798 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_00_00_UTC 2023-10-08T07:10:18.798 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_01_00_UTC 2023-10-08T07:10:18.798 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_02_00_UTC 2023-10-08T07:10:18.798 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_03_00_UTC 2023-10-08T07:10:18.798 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_04_00_UTC 2023-10-08T07:10:18.799 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_05_00_UTC 2023-10-08T07:10:18.799 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_06_00_UTC 2023-10-08T07:10:18.799 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_07_00_UTC 2023-10-08T07:10:18.799 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_08_00_UTC 2023-10-08T07:10:18.799 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_09_00_UTC 2023-10-08T07:10:18.799 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_10_00_UTC 2023-10-08T07:10:18.800 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-06_55_00_UTC 2023-10-08T07:10:18.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:18.800 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-06_55_00_UTC) 2023-10-08T07:10:18.855 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-06_56_00_UTC 2023-10-08T07:10:18.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:18.856 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-06_56_00_UTC) 2023-10-08T07:10:18.915 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-06_57_00_UTC 2023-10-08T07:10:18.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:18.915 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-06_57_00_UTC) 2023-10-08T07:10:18.973 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-06_58_00_UTC 2023-10-08T07:10:18.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:18.974 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-06_58_00_UTC) 2023-10-08T07:10:19.027 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-06_59_00_UTC 2023-10-08T07:10:19.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.028 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-06_59_00_UTC) 2023-10-08T07:10:19.081 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_00_00_UTC 2023-10-08T07:10:19.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.081 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_00_00_UTC) 2023-10-08T07:10:19.135 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_01_00_UTC 2023-10-08T07:10:19.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.136 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_01_00_UTC) 2023-10-08T07:10:19.197 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_02_00_UTC 2023-10-08T07:10:19.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.197 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_02_00_UTC) 2023-10-08T07:10:19.261 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_03_00_UTC 2023-10-08T07:10:19.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.262 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_03_00_UTC) 2023-10-08T07:10:19.321 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_04_00_UTC 2023-10-08T07:10:19.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.321 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_04_00_UTC) 2023-10-08T07:10:19.383 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_05_00_UTC 2023-10-08T07:10:19.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.384 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_05_00_UTC) 2023-10-08T07:10:19.442 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_06_00_UTC 2023-10-08T07:10:19.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.443 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_06_00_UTC) 2023-10-08T07:10:19.503 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_07_00_UTC 2023-10-08T07:10:19.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.503 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_07_00_UTC) 2023-10-08T07:10:19.563 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_08_00_UTC 2023-10-08T07:10:19.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.563 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_08_00_UTC) 2023-10-08T07:10:19.619 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_09_00_UTC 2023-10-08T07:10:19.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.619 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_09_00_UTC) 2023-10-08T07:10:19.674 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_10_00_UTC 2023-10-08T07:10:19.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.674 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-07_10_00_UTC) 2023-10-08T07:10:19.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:19.730 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-10-08T07:10:19.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T07:10:19.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.986+0000 7f1b07aa2640 1 Processor -- start 2023-10-08T07:10:19.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.987+0000 7f1b07aa2640 1 -- start start 2023-10-08T07:10:19.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.988+0000 7f1b07aa2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0015aad0 0x7f1b000b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:19.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.988+0000 7f1b07aa2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1b000c82b0 0x7f1b000c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:19.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.988+0000 7f1b07aa2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1b0006cd00 0x7f1b0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:19.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.988+0000 7f1b07aa2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0006d6d0 con 0x7f1b0006cd00 2023-10-08T07:10:19.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.988+0000 7f1b07aa2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0015bf30 con 0x7f1b000c82b0 2023-10-08T07:10:19.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.988+0000 7f1b07aa2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0015b320 con 0x7f1b0015aad0 2023-10-08T07:10:19.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.989+0000 7f1b0603f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1b0006cd00 0x7f1b0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:19.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.989+0000 7f1b0603f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1b0006cd00 0x7f1b0006d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60900/0 (socket says 172.21.15.70:60900) 2023-10-08T07:10:19.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.989+0000 7f1b0603f640 1 -- 172.21.15.70:0/2245137991 learned_addr learned my addr 172.21.15.70:0/2245137991 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:19.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.989+0000 7f1b0503d640 1 --2- 172.21.15.70:0/2245137991 >> v2:172.21.15.181:3300/0 conn(0x7f1b000c82b0 0x7f1b000c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:19.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.989+0000 7f1b0583e640 1 --2- 172.21.15.70:0/2245137991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0015aad0 0x7f1b000b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:19.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.989+0000 7f1b0603f640 1 -- 172.21.15.70:0/2245137991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0015aad0 msgr2=0x7f1b000b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:19.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.989+0000 7f1b0603f640 1 --2- 172.21.15.70:0/2245137991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0015aad0 0x7f1b000b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:19.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.989+0000 7f1b0603f640 1 -- 172.21.15.70:0/2245137991 >> v2:172.21.15.181:3300/0 conn(0x7f1b000c82b0 msgr2=0x7f1b000c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:19.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.989+0000 7f1b0603f640 1 --2- 172.21.15.70:0/2245137991 >> v2:172.21.15.181:3300/0 conn(0x7f1b000c82b0 0x7f1b000c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:19.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.989+0000 7f1b0603f640 1 -- 172.21.15.70:0/2245137991 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0015b600 con 0x7f1b0006cd00 2023-10-08T07:10:19.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.990+0000 7f1b0603f640 1 --2- 172.21.15.70:0/2245137991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b0006cd00 0x7f1b0006d190 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f1b000cc300 tx=0x7f1af802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d31d104ce79859 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:19.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b0483c640 1 -- 172.21.15.70:0/2245137991 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af802ca70 con 0x7f1b0006cd00 2023-10-08T07:10:19.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b0483c640 1 -- 172.21.15.70:0/2245137991 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af802cbd0 con 0x7f1b0006cd00 2023-10-08T07:10:19.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b0483c640 1 -- 172.21.15.70:0/2245137991 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af80343f0 con 0x7f1b0006cd00 2023-10-08T07:10:19.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b07aa2640 1 -- 172.21.15.70:0/2245137991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b0006cd00 msgr2=0x7f1b0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:19.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b07aa2640 1 --2- 172.21.15.70:0/2245137991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b0006cd00 0x7f1b0006d190 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f1b000cc300 tx=0x7f1af802bed0 comp rx=0 tx=0).stop 2023-10-08T07:10:19.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b07aa2640 1 -- 172.21.15.70:0/2245137991 shutdown_connections 2023-10-08T07:10:19.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b07aa2640 1 --2- 172.21.15.70:0/2245137991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b0006cd00 0x7f1b0006d190 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:19.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b07aa2640 1 --2- 172.21.15.70:0/2245137991 >> v2:172.21.15.181:3300/0 conn(0x7f1b000c82b0 0x7f1b000c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:19.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b07aa2640 1 --2- 172.21.15.70:0/2245137991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0015aad0 0x7f1b000b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:19.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b07aa2640 1 -- 172.21.15.70:0/2245137991 >> 172.21.15.70:0/2245137991 conn(0x7f1b00069d50 msgr2=0x7f1b0006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:19.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b07aa2640 1 -- 172.21.15.70:0/2245137991 shutdown_connections 2023-10-08T07:10:19.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.991+0000 7f1b07aa2640 1 -- 172.21.15.70:0/2245137991 wait complete. 2023-10-08T07:10:19.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.993+0000 7f1b07aa2640 1 Processor -- start 2023-10-08T07:10:19.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.993+0000 7f1b07aa2640 1 -- start start 2023-10-08T07:10:19.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.994+0000 7f1b07aa2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0006cd00 0x7f1b00156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:19.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.994+0000 7f1b07aa2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b000c82b0 0x7f1b00159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:19.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.994+0000 7f1b07aa2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0015aad0 0x7f1b001597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:19.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.994+0000 7f1b07aa2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b00158bc0 con 0x7f1b000c82b0 2023-10-08T07:10:19.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.994+0000 7f1b07aa2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b00156e70 con 0x7f1b0015aad0 2023-10-08T07:10:19.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.994+0000 7f1b07aa2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b00157cd0 con 0x7f1b0006cd00 2023-10-08T07:10:19.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0503d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b000c82b0 0x7f1b00159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:19.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0503d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b000c82b0 0x7f1b00159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60912/0 (socket says 172.21.15.70:60912) 2023-10-08T07:10:19.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0503d640 1 -- 172.21.15.70:0/1021935784 learned_addr learned my addr 172.21.15.70:0/1021935784 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:19.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0583e640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0006cd00 0x7f1b00156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:20.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0603f640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0015aad0 0x7f1b001597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:20.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0503d640 1 -- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0006cd00 msgr2=0x7f1b00156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:20.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0503d640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0006cd00 0x7f1b00156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:20.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0503d640 1 -- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0015aad0 msgr2=0x7f1b001597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:20.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0503d640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0015aad0 0x7f1b001597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:20.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0503d640 1 -- 172.21.15.70:0/1021935784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af8009b30 con 0x7f1b000c82b0 2023-10-08T07:10:20.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.995+0000 7f1b0503d640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b000c82b0 0x7f1b00159270 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f1af000d800 tx=0x7f1af000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:20.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.996+0000 7f1af67fc640 1 -- 172.21.15.70:0/1021935784 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af00096a0 con 0x7f1b000c82b0 2023-10-08T07:10:20.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.996+0000 7f1af67fc640 1 -- 172.21.15.70:0/1021935784 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af0009800 con 0x7f1b000c82b0 2023-10-08T07:10:20.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.996+0000 7f1af67fc640 1 -- 172.21.15.70:0/1021935784 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0009b90 con 0x7f1b000c82b0 2023-10-08T07:10:20.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.996+0000 7f1b07aa2640 1 -- 172.21.15.70:0/1021935784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b00157fb0 con 0x7f1b000c82b0 2023-10-08T07:10:20.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.996+0000 7f1b07aa2640 1 -- 172.21.15.70:0/1021935784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b00158570 con 0x7f1b000c82b0 2023-10-08T07:10:20.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.997+0000 7f1b07aa2640 1 -- 172.21.15.70:0/1021935784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0004e160 con 0x7f1b000c82b0 2023-10-08T07:10:20.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.998+0000 7f1af67fc640 1 -- 172.21.15.70:0/1021935784 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 111) v1 ==== 89360+0+0 (secure 0 0 0) 0x7f1af0009cf0 con 0x7f1b000c82b0 2023-10-08T07:10:20.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.999+0000 7f1af67fc640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f1ae406afe0 0x7f1ae406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:20.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.999+0000 7f1af67fc640 1 -- 172.21.15.70:0/1021935784 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f1af008a510 con 0x7f1b000c82b0 2023-10-08T07:10:20.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:19.999+0000 7f1b0583e640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f1ae406afe0 0x7f1ae406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:20.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:20.000+0000 7f1b0583e640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f1ae406afe0 0x7f1ae406d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1afc006fd0 tx=0x7f1afc008040 comp rx=0 tx=0).ready entity=mgr.7040 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:20.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:20.003+0000 7f1af67fc640 1 -- 172.21.15.70:0/1021935784 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1af0054fa0 con 0x7f1b000c82b0 2023-10-08T07:10:20.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:20.241+0000 7f1b07aa2640 1 -- 172.21.15.70:0/1021935784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f1b0015ed10 con 0x7f1b000c82b0 2023-10-08T07:10:21.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.195+0000 7f1af67fc640 1 -- 172.21.15.70:0/1021935784 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v112) v1 ==== 93+0+0 (secure 0 0 0) 0x7f1af000b440 con 0x7f1b000c82b0 2023-10-08T07:10:21.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.198+0000 7f1b07aa2640 1 -- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f1ae406afe0 msgr2=0x7f1ae406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:21.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.199+0000 7f1b07aa2640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f1ae406afe0 0x7f1ae406d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1afc006fd0 tx=0x7f1afc008040 comp rx=0 tx=0).stop 2023-10-08T07:10:21.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.199+0000 7f1b07aa2640 1 -- 172.21.15.70:0/1021935784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b000c82b0 msgr2=0x7f1b00159270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:21.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.199+0000 7f1b07aa2640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b000c82b0 0x7f1b00159270 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f1af000d800 tx=0x7f1af000dcc0 comp rx=0 tx=0).stop 2023-10-08T07:10:21.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.199+0000 7f1b07aa2640 1 -- 172.21.15.70:0/1021935784 shutdown_connections 2023-10-08T07:10:21.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.199+0000 7f1b07aa2640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f1ae406afe0 0x7f1ae406d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:21.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.199+0000 7f1b07aa2640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b0015aad0 0x7f1b001597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:21.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.199+0000 7f1b07aa2640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b000c82b0 0x7f1b00159270 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:21.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.199+0000 7f1b07aa2640 1 --2- 172.21.15.70:0/1021935784 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b0006cd00 0x7f1b00156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:21.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.199+0000 7f1b07aa2640 1 -- 172.21.15.70:0/1021935784 >> 172.21.15.70:0/1021935784 conn(0x7f1b00069d50 msgr2=0x7f1b0015d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:21.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.200+0000 7f1b07aa2640 1 -- 172.21.15.70:0/1021935784 shutdown_connections 2023-10-08T07:10:21.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.200+0000 7f1b07aa2640 1 -- 172.21.15.70:0/1021935784 wait complete. 2023-10-08T07:10:21.230 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T07:10:21.230 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:10:21.339 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:10:21.439 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T07:10:21.439 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:10:21.439 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:10:21.491 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:10:21.492 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:10:21.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:10:21.727 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:21.725+0000 7f0ec4c10040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:10:21.739 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:21.737+0000 7fc7b9777040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:10:21.779 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:21.778+0000 7f0ec4c10040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:10:21.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.786+0000 7f84d1c61640 1 Processor -- start 2023-10-08T07:10:21.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.786+0000 7f84d1c61640 1 -- start start 2023-10-08T07:10:21.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.787+0000 7f84d1c61640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84cc15a990 0x7f84cc071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:21.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.787+0000 7f84d1c61640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84cc06a460 0x7f84cc0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:21.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.787+0000 7f84d1c61640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84cc0724a0 con 0x7f84cc15ad70 2023-10-08T07:10:21.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.787+0000 7f84d1c61640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84cc071ad0 con 0x7f84cc06a460 2023-10-08T07:10:21.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.787+0000 7f84d1c61640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc0b5500 con 0x7f84cc15a990 2023-10-08T07:10:21.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.788+0000 7f84cbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84cc06a460 0x7f84cc0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:21.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.788+0000 7f84cbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f84cc06a460 0x7f84cc0b27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60936/0 (socket says 172.21.15.70:60936) 2023-10-08T07:10:21.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.788+0000 7f84cbfff640 1 -- 172.21.15.70:0/3702770456 learned_addr learned my addr 172.21.15.70:0/3702770456 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:21.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.788+0000 7f84cb7fe640 1 --2- 172.21.15.70:0/3702770456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84cc15a990 0x7f84cc071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:21.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.788+0000 7f84cbfff640 1 -- 172.21.15.70:0/3702770456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84cc15a990 msgr2=0x7f84cc071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:21.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.788+0000 7f84cbfff640 1 --2- 172.21.15.70:0/3702770456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84cc15a990 0x7f84cc071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:21.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.788+0000 7f84cbfff640 1 -- 172.21.15.70:0/3702770456 >> v1:172.21.15.181:6789/0 conn(0x7f84cc15ad70 legacy=0x7f84cc069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:21.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.788+0000 7f84cbfff640 1 -- 172.21.15.70:0/3702770456 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc071480 con 0x7f84cc06a460 2023-10-08T07:10:21.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.790+0000 7f84cbfff640 1 --2- 172.21.15.70:0/3702770456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 0x7f84cc0b27e0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009e50 tx=0x7f84c402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d36eeb4ee147f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:21.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.790+0000 7f84ca7fc640 1 -- 172.21.15.70:0/3702770456 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c402cb00 con 0x7f84cc06a460 2023-10-08T07:10:21.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.791+0000 7f84ca7fc640 1 -- 172.21.15.70:0/3702770456 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c402cc60 con 0x7f84cc06a460 2023-10-08T07:10:21.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.791+0000 7f84ca7fc640 1 -- 172.21.15.70:0/3702770456 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c40343f0 con 0x7f84cc06a460 2023-10-08T07:10:21.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.791+0000 7f84d1c61640 1 -- 172.21.15.70:0/3702770456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 msgr2=0x7f84cc0b27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:21.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.791+0000 7f84d1c61640 1 --2- 172.21.15.70:0/3702770456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 0x7f84cc0b27e0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009e50 tx=0x7f84c402bf60 comp rx=0 tx=0).stop 2023-10-08T07:10:21.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.791+0000 7f84d1c61640 1 -- 172.21.15.70:0/3702770456 shutdown_connections 2023-10-08T07:10:21.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.791+0000 7f84d1c61640 1 --2- 172.21.15.70:0/3702770456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 0x7f84cc0b27e0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:21.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.791+0000 7f84d1c61640 1 --2- 172.21.15.70:0/3702770456 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84cc15a990 0x7f84cc071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:21.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.791+0000 7f84d1c61640 1 -- 172.21.15.70:0/3702770456 >> 172.21.15.70:0/3702770456 conn(0x7f84cc06d450 msgr2=0x7f84cc06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:21.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.791+0000 7f84d1c61640 1 -- 172.21.15.70:0/3702770456 shutdown_connections 2023-10-08T07:10:21.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.791+0000 7f84d1c61640 1 -- 172.21.15.70:0/3702770456 wait complete. 2023-10-08T07:10:21.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.794+0000 7f84d1c61640 1 Processor -- start 2023-10-08T07:10:21.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.795+0000 7f84d1c61640 1 -- start start 2023-10-08T07:10:21.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.796+0000 7f84d1c61640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 0x7f84cc0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:21.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.796+0000 7f84d1c61640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84cc15a990 0x7f84cc0c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:21.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.796+0000 7f84d1c61640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84cc15ad70 0x7f84cc0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:21.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.796+0000 7f84d1c61640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc0c6ee0 con 0x7f84cc06a460 2023-10-08T07:10:21.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.796+0000 7f84d1c61640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc15d220 con 0x7f84cc15a990 2023-10-08T07:10:21.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.796+0000 7f84d1c61640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc15d500 con 0x7f84cc15ad70 2023-10-08T07:10:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.796+0000 7f84cb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 0x7f84cc0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.796+0000 7f84cb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 0x7f84cc0c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60946/0 (socket says 172.21.15.70:60946) 2023-10-08T07:10:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.796+0000 7f84cb7fe640 1 -- 172.21.15.70:0/615996487 learned_addr learned my addr 172.21.15.70:0/615996487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.797+0000 7f84cbfff640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84cc15ad70 0x7f84cc0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.797+0000 7f84caffd640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84cc15a990 0x7f84cc0c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.797+0000 7f84cb7fe640 1 -- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84cc15ad70 msgr2=0x7f84cc0c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.797+0000 7f84cb7fe640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84cc15ad70 0x7f84cc0c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.797+0000 7f84cb7fe640 1 -- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84cc15a990 msgr2=0x7f84cc0c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.797+0000 7f84cb7fe640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84cc15a990 0x7f84cc0c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.797+0000 7f84cb7fe640 1 -- 172.21.15.70:0/615996487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4009b30 con 0x7f84cc06a460 2023-10-08T07:10:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.797+0000 7f84cb7fe640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 0x7f84cc0c9510 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00e9b0 tx=0x7f84bc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.797+0000 7f84b3fff640 1 -- 172.21.15.70:0/615996487 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc009670 con 0x7f84cc06a460 2023-10-08T07:10:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.798+0000 7f84b3fff640 1 -- 172.21.15.70:0/615996487 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc00c9a0 con 0x7f84cc06a460 2023-10-08T07:10:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.798+0000 7f84b3fff640 1 -- 172.21.15.70:0/615996487 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc00cd50 con 0x7f84cc06a460 2023-10-08T07:10:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.798+0000 7f84d1c61640 1 -- 172.21.15.70:0/615996487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84cc0c6b00 con 0x7f84cc06a460 2023-10-08T07:10:21.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.798+0000 7f84d1c61640 1 -- 172.21.15.70:0/615996487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc15cfe0 con 0x7f84cc06a460 2023-10-08T07:10:21.802 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:21.798+0000 7fc7b9777040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:10:21.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.799+0000 7f84d1c61640 1 -- 172.21.15.70:0/615996487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84cc0c7150 con 0x7f84cc06a460 2023-10-08T07:10:21.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.800+0000 7f84b3fff640 1 -- 172.21.15.70:0/615996487 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 112) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f84bc013470 con 0x7f84cc06a460 2023-10-08T07:10:21.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.801+0000 7f84b3fff640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f84b806af90 0x7f84b806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:21.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.801+0000 7f84b3fff640 1 -- 172.21.15.70:0/615996487 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f84bc089ba0 con 0x7f84cc06a460 2023-10-08T07:10:21.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.801+0000 7f84caffd640 1 -- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f84b806af90 msgr2=0x7f84b806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:21.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.801+0000 7f84caffd640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f84b806af90 0x7f84b806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:21.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.805+0000 7f84b3fff640 1 -- 172.21.15.70:0/615996487 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84bc054810 con 0x7f84cc06a460 2023-10-08T07:10:21.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.996+0000 7f84d1c61640 1 -- 172.21.15.70:0/615996487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84cc15cfe0 con 0x7f84cc06a460 2023-10-08T07:10:22.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:21.998+0000 7f84b3fff640 1 -- 172.21.15.70:0/615996487 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+17902 (secure 0 0 0) 0x7f84cc15cfe0 con 0x7f84cc06a460 2023-10-08T07:10:22.000 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:22.000 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":200,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:05:21.112543+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:54:30.246375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-08T06:54:31.487900+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2946222544":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/1250722759":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6833/3979056493":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6832/3979056493":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/340227645":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/849801254":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/4017559811":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/64410277":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/811429527":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/772181101":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/1286852011":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6832/3709692999":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/4147168289":"2023-10-09T06:54:47.949157+0000","172.21.15.181:6833/3709692999":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/602716182":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/1777884673":"2023-10-09T06:54:47.949157+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:10:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 -- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f84b806af90 msgr2=0x7f84b806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f84b806af90 0x7f84b806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 -- 172.21.15.70:0/615996487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 msgr2=0x7f84cc0c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 0x7f84cc0c9510 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00e9b0 tx=0x7f84bc00ee70 comp rx=0 tx=0).stop 2023-10-08T07:10:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 -- 172.21.15.70:0/615996487 shutdown_connections 2023-10-08T07:10:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f84b806af90 0x7f84b806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f84cc15ad70 0x7f84cc0c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f84cc15a990 0x7f84cc0c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 --2- 172.21.15.70:0/615996487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84cc06a460 0x7f84cc0c9510 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 -- 172.21.15.70:0/615996487 >> 172.21.15.70:0/615996487 conn(0x7f84cc06d450 msgr2=0x7f84cc0b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 -- 172.21.15.70:0/615996487 shutdown_connections 2023-10-08T07:10:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.001+0000 7f84d1c61640 1 -- 172.21.15.70:0/615996487 wait complete. 2023-10-08T07:10:22.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T06:54:30.246375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T06:54:31.487900+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:10:22.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:10:22.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.217+0000 7fa5ce6fc640 1 Processor -- start 2023-10-08T07:10:22.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.217+0000 7fa5ce6fc640 1 -- start start 2023-10-08T07:10:22.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.221+0000 7fa5ce6fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa5c80aeea0 0x7fa5c80cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.221+0000 7fa5c7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa5c815cca0 0x7fa5c8071d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41846/0 (socket says 172.21.15.70:41846) 2023-10-08T07:10:22.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.221+0000 7fa5c7fff640 1 -- 172.21.15.70:0/2099726991 learned_addr learned my addr 172.21.15.70:0/2099726991 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:22.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.221+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2099726991 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c8072460 con 0x7fa5c815cca0 2023-10-08T07:10:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.221+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2099726991 --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c8071590 con 0x7fa5c80cb800 2023-10-08T07:10:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2099726991 --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c80b1900 con 0x7fa5c80aeea0 2023-10-08T07:10:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899671625 0 0) 0x7fa5c8072460 con 0x7fa5c815cca0 2023-10-08T07:10:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c77fe640 1 --2- 172.21.15.70:0/2099726991 >> v2:172.21.15.181:3300/0 conn(0x7fa5c80aeea0 0x7fa5c80cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b0003520 con 0x7fa5c815cca0 2023-10-08T07:10:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 391800290 0 0) 0x7fa5b0003520 con 0x7fa5c815cca0 2023-10-08T07:10:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 >> v2:172.21.15.181:3300/0 conn(0x7fa5c80aeea0 msgr2=0x7fa5c80cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 --2- 172.21.15.70:0/2099726991 >> v2:172.21.15.181:3300/0 conn(0x7fa5c80aeea0 0x7fa5c80cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 >> v1:172.21.15.181:6789/0 conn(0x7fa5c80cb800 legacy=0x7fa5c80ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c815d080 con 0x7fa5c815cca0 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa5b8001670 con 0x7fa5c815cca0 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 >> v1:172.21.15.70:6789/0 conn(0x7fa5c815cca0 legacy=0x7fa5c8071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 0x7fa5b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5b00047c0 0x7fa5b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5b00051d0 0x7fa5b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80b1900 con 0x7fa5b00051d0 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b000c120 con 0x7fa5b00039c0 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.222+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b000c2e0 con 0x7fa5b00047c0 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.223+0000 7fa5c7fff640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 0x7fa5b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.223+0000 7fa5ccc99640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5b00051d0 0x7fa5b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.223+0000 7fa5c77fe640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5b00047c0 0x7fa5b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.223+0000 7fa5c7fff640 1 -- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5b00047c0 msgr2=0x7fa5b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.224+0000 7fa5c7fff640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5b00047c0 0x7fa5b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.224+0000 7fa5c7fff640 1 -- 172.21.15.70:0/2099726991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5b00051d0 msgr2=0x7fa5b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.224+0000 7fa5c7fff640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5b00051d0 0x7fa5b000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.224+0000 7fa5c7fff640 1 -- 172.21.15.70:0/2099726991 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0003520 con 0x7fa5b00039c0 2023-10-08T07:10:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.224+0000 7fa5c77fe640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5b00047c0 0x7fa5b0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:10:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.224+0000 7fa5ccc99640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5b00051d0 0x7fa5b000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:10:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.224+0000 7fa5ccc99640 1 -- 172.21.15.70:0/2099726991 reap_dead start 2023-10-08T07:10:22.228 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:22.225+0000 7f0ec4c10040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:10:22.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.225+0000 7fa5c7fff640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 0x7fa5b0003de0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8001eb0 tx=0x7fa5b00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e87d208fbbd89897 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:22.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.228+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8005ad0 con 0x7fa5b00039c0 2023-10-08T07:10:22.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.228+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b8005c50 con 0x7fa5b00039c0 2023-10-08T07:10:22.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.228+0000 7fa5c6ffd640 1 -- 172.21.15.70:0/2099726991 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b802c7a0 con 0x7fa5b00039c0 2023-10-08T07:10:22.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.229+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 msgr2=0x7fa5b0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.229+0000 7fa5ce6fc640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 0x7fa5b0003de0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8001eb0 tx=0x7fa5b00057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.229+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2099726991 shutdown_connections 2023-10-08T07:10:22.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.229+0000 7fa5ce6fc640 1 --2- 172.21.15.70:0/2099726991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 0x7fa5b0003de0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.229+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2099726991 >> 172.21.15.70:0/2099726991 conn(0x7fa5c806b250 msgr2=0x7fa5c806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:22.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.229+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2099726991 shutdown_connections 2023-10-08T07:10:22.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.229+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2099726991 wait complete. 2023-10-08T07:10:22.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.232+0000 7fa5ce6fc640 1 Processor -- start 2023-10-08T07:10:22.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.232+0000 7fa5ce6fc640 1 -- start start 2023-10-08T07:10:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ce6fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 0x7fa5c815d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ce6fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5c80c9320 0x7fa5c815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ce6fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c8147b50 0x7fa5c8149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ce6fc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80acaa0 con 0x7fa5c8147b50 2023-10-08T07:10:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ce6fc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80acd80 con 0x7fa5b00039c0 2023-10-08T07:10:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ce6fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80ad060 con 0x7fa5c80c9320 2023-10-08T07:10:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ccc99640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c8147b50 0x7fa5c8149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ccc99640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c8147b50 0x7fa5c8149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60968/0 (socket says 172.21.15.70:60968) 2023-10-08T07:10:22.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ccc99640 1 -- 172.21.15.70:0/2768850894 learned_addr learned my addr 172.21.15.70:0/2768850894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:22.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5c77fe640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5c80c9320 0x7fa5c815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5c7fff640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 0x7fa5c815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ccc99640 1 -- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5c80c9320 msgr2=0x7fa5c815fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ccc99640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5c80c9320 0x7fa5c815fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ccc99640 1 -- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 msgr2=0x7fa5c815d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ccc99640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 0x7fa5c815d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.234+0000 7fa5ccc99640 1 -- 172.21.15.70:0/2768850894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c80ad340 con 0x7fa5c8147b50 2023-10-08T07:10:22.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.235+0000 7fa5ccc99640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c8147b50 0x7fa5c8149be0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80cd3e0 tx=0x7fa5c00019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:22.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.235+0000 7fa5c4ff9640 1 -- 172.21.15.70:0/2768850894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0003b80 con 0x7fa5c8147b50 2023-10-08T07:10:22.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.235+0000 7fa5c4ff9640 1 -- 172.21.15.70:0/2768850894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c0003d00 con 0x7fa5c8147b50 2023-10-08T07:10:22.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.235+0000 7fa5c4ff9640 1 -- 172.21.15.70:0/2768850894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0005310 con 0x7fa5c8147b50 2023-10-08T07:10:22.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.235+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2768850894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c815ffa0 con 0x7fa5c8147b50 2023-10-08T07:10:22.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.235+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2768850894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c8147fd0 con 0x7fa5c8147b50 2023-10-08T07:10:22.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.236+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2768850894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c81481f0 con 0x7fa5c8147b50 2023-10-08T07:10:22.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.237+0000 7fa5c4ff9640 1 -- 172.21.15.70:0/2768850894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 112) v1 ==== 89343+0+0 (secure 0 0 0) 0x7fa5c0005470 con 0x7fa5c8147b50 2023-10-08T07:10:22.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.239+0000 7fa5c4ff9640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa5b406af90 0x7fa5b406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.239+0000 7fa5c4ff9640 1 -- 172.21.15.70:0/2768850894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fa5c0089a40 con 0x7fa5c8147b50 2023-10-08T07:10:22.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.239+0000 7fa5c7fff640 1 -- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa5b406af90 msgr2=0x7fa5b406d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:22.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.239+0000 7fa5c7fff640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa5b406af90 0x7fa5b406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:22.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.244+0000 7fa5c4ff9640 1 -- 172.21.15.70:0/2768850894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5c00544d0 con 0x7fa5c8147b50 2023-10-08T07:10:22.269 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:22.268+0000 7fc7b9777040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:10:22.280 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:22.279+0000 7f0ec4c10040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:10:22.323 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:22.321+0000 7fc7b9777040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:10:22.331 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:22.330+0000 7f0ec4c10040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:10:22.373 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:22.371+0000 7fc7b9777040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:10:22.392 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:22.390+0000 7f0ec4c10040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:10:22.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.432+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2768850894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5c80acbe0 con 0x7fa5c8147b50 2023-10-08T07:10:22.434 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:22.432+0000 7fc7b9777040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:10:22.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.433+0000 7fa5c4ff9640 1 -- 172.21.15.70:0/2768850894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v200) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5c80acbe0 con 0x7fa5c8147b50 2023-10-08T07:10:22.435 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:10:22.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa5b406af90 msgr2=0x7fa5b406d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:22.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa5b406af90 0x7fa5b406d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2768850894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c8147b50 msgr2=0x7fa5c8149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c8147b50 0x7fa5c8149be0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80cd3e0 tx=0x7fa5c00019f0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2768850894 shutdown_connections 2023-10-08T07:10:22.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7fa5b406af90 0x7fa5b406d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5c8147b50 0x7fa5c8149be0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa5c80c9320 0x7fa5c815fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 --2- 172.21.15.70:0/2768850894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa5b00039c0 0x7fa5c815d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2768850894 >> 172.21.15.70:0/2768850894 conn(0x7fa5c806b250 msgr2=0x7fa5c80cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:22.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2768850894 shutdown_connections 2023-10-08T07:10:22.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.436+0000 7fa5ce6fc640 1 -- 172.21.15.70:0/2768850894 wait complete. 2023-10-08T07:10:22.440 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:22.439+0000 7f0ec4c10040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:10:22.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:10:22.486 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:22.484+0000 7fc7b9777040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:10:22.550 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:22.549+0000 7f0ec4c10040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:10:22.603 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:22.601+0000 7fc7b9777040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:10:22.612 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:22.611+0000 7f0ec4c10040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:10:22.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.655+0000 7f78baed7640 1 Processor -- start 2023-10-08T07:10:22.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.656+0000 7f78baed7640 1 -- start start 2023-10-08T07:10:22.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.658+0000 7f78baed7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b406dfc0 0x7f78b4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.658+0000 7f78baed7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f78b4071070 0x7f78b40cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.658+0000 7f78baed7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78b40b1cc0 con 0x7f78b40afbe0 2023-10-08T07:10:22.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.658+0000 7f78baed7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b40b2620 con 0x7f78b4071070 2023-10-08T07:10:22.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.658+0000 7f78baed7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40b2060 con 0x7f78b406dfc0 2023-10-08T07:10:22.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.659+0000 7f78b8c73640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f78b40afbe0 0x7f78b40b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41860/0 (socket says 172.21.15.70:41860) 2023-10-08T07:10:22.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.659+0000 7f78b8c73640 1 -- 172.21.15.70:0/1753557122 learned_addr learned my addr 172.21.15.70:0/1753557122 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:22.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.659+0000 7f78b3fff640 1 --2- 172.21.15.70:0/1753557122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b406dfc0 0x7f78b4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.659+0000 7f78b9474640 1 --2- 172.21.15.70:0/1753557122 >> v2:172.21.15.181:3300/0 conn(0x7f78b4071070 0x7f78b40cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.659+0000 7f78b37fe640 1 -- 172.21.15.70:0/1753557122 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876624638 0 0) 0x7f78b40b1cc0 con 0x7f78b40afbe0 2023-10-08T07:10:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.659+0000 7f78b37fe640 1 -- 172.21.15.70:0/1753557122 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78a4003520 con 0x7f78b40afbe0 2023-10-08T07:10:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.660+0000 7f78b3fff640 1 -- 172.21.15.70:0/1753557122 >> v2:172.21.15.181:3300/0 conn(0x7f78b4071070 msgr2=0x7f78b40cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.660+0000 7f78b3fff640 1 --2- 172.21.15.70:0/1753557122 >> v2:172.21.15.181:3300/0 conn(0x7f78b4071070 0x7f78b40cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.660+0000 7f78b3fff640 1 -- 172.21.15.70:0/1753557122 >> v1:172.21.15.70:6789/0 conn(0x7f78b40afbe0 legacy=0x7f78b40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.660+0000 7f78b3fff640 1 -- 172.21.15.70:0/1753557122 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40b2340 con 0x7f78b406dfc0 2023-10-08T07:10:22.663 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:22.660+0000 7f0ec4c10040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:10:22.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.661+0000 7f78b3fff640 1 --2- 172.21.15.70:0/1753557122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b406dfc0 0x7f78b4070b30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f78a00097a0 tx=0x7f78a002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddbbac2c215dfe4c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:22.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.662+0000 7f78b37fe640 1 -- 172.21.15.70:0/1753557122 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a002d040 con 0x7f78b406dfc0 2023-10-08T07:10:22.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.662+0000 7f78b37fe640 1 -- 172.21.15.70:0/1753557122 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a0031810 con 0x7f78b406dfc0 2023-10-08T07:10:22.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.662+0000 7f78b37fe640 1 -- 172.21.15.70:0/1753557122 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0031b80 con 0x7f78b406dfc0 2023-10-08T07:10:22.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.663+0000 7f78baed7640 1 -- 172.21.15.70:0/1753557122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b406dfc0 msgr2=0x7f78b4070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.663+0000 7f78baed7640 1 --2- 172.21.15.70:0/1753557122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b406dfc0 0x7f78b4070b30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f78a00097a0 tx=0x7f78a002b8b0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.663+0000 7f78baed7640 1 -- 172.21.15.70:0/1753557122 shutdown_connections 2023-10-08T07:10:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.663+0000 7f78baed7640 1 --2- 172.21.15.70:0/1753557122 >> v2:172.21.15.181:3300/0 conn(0x7f78b4071070 0x7f78b40cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.663+0000 7f78baed7640 1 --2- 172.21.15.70:0/1753557122 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b406dfc0 0x7f78b4070b30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.663+0000 7f78baed7640 1 -- 172.21.15.70:0/1753557122 >> 172.21.15.70:0/1753557122 conn(0x7f78b406a5d0 msgr2=0x7f78b40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.663+0000 7f78baed7640 1 -- 172.21.15.70:0/1753557122 shutdown_connections 2023-10-08T07:10:22.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.663+0000 7f78baed7640 1 -- 172.21.15.70:0/1753557122 wait complete. 2023-10-08T07:10:22.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.664+0000 7f78baed7640 1 Processor -- start 2023-10-08T07:10:22.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.665+0000 7f78baed7640 1 -- start start 2023-10-08T07:10:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.665+0000 7f78baed7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78b406dfc0 0x7f78b415db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.665+0000 7f78baed7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78b4071070 0x7f78b4158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78baed7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b40afbe0 0x7f78b4158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78baed7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40cd740 con 0x7f78b406dfc0 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78baed7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40cd2b0 con 0x7f78b4071070 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78baed7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b40cc6e0 con 0x7f78b40afbe0 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b8c73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78b406dfc0 0x7f78b415db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b8c73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78b406dfc0 0x7f78b415db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60980/0 (socket says 172.21.15.70:60980) 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b8c73640 1 -- 172.21.15.70:0/1909948813 learned_addr learned my addr 172.21.15.70:0/1909948813 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b9474640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b40afbe0 0x7f78b4158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b8c73640 1 -- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b40afbe0 msgr2=0x7f78b4158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b3fff640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78b4071070 0x7f78b4158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b8c73640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b40afbe0 0x7f78b4158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b8c73640 1 -- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78b4071070 msgr2=0x7f78b4158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b8c73640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78b4071070 0x7f78b4158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b8c73640 1 -- 172.21.15.70:0/1909948813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0009400 con 0x7f78b406dfc0 2023-10-08T07:10:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.666+0000 7f78b9474640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b40afbe0 0x7f78b4158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.667+0000 7f78b8c73640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78b406dfc0 0x7f78b415db90 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f78a800c390 tx=0x7f78a800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.667+0000 7f78b17fa640 1 -- 172.21.15.70:0/1909948813 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a8005ce0 con 0x7f78b406dfc0 2023-10-08T07:10:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.667+0000 7f78baed7640 1 -- 172.21.15.70:0/1909948813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b40cc980 con 0x7f78b406dfc0 2023-10-08T07:10:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.667+0000 7f78b17fa640 1 -- 172.21.15.70:0/1909948813 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a8005e40 con 0x7f78b406dfc0 2023-10-08T07:10:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.667+0000 7f78b17fa640 1 -- 172.21.15.70:0/1909948813 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a80072f0 con 0x7f78b406dfc0 2023-10-08T07:10:22.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.667+0000 7f78baed7640 1 -- 172.21.15.70:0/1909948813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b40ccf40 con 0x7f78b406dfc0 2023-10-08T07:10:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.668+0000 7f78baed7640 1 -- 172.21.15.70:0/1909948813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b404e160 con 0x7f78b406dfc0 2023-10-08T07:10:22.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.669+0000 7f78b17fa640 1 -- 172.21.15.70:0/1909948813 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 112) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f78a8007450 con 0x7f78b406dfc0 2023-10-08T07:10:22.671 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:22.670+0000 7fc7b9777040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:10:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.671+0000 7f78b17fa640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f789806afc0 0x7f789806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:22.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.671+0000 7f78b17fa640 1 -- 172.21.15.70:0/1909948813 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f78a8089230 con 0x7f78b406dfc0 2023-10-08T07:10:22.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.671+0000 7f78b3fff640 1 -- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f789806afc0 msgr2=0x7f789806d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:22.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.671+0000 7f78b3fff640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f789806afc0 0x7f789806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:22.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.675+0000 7f78b17fa640 1 -- 172.21.15.70:0/1909948813 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78a8053cd0 con 0x7f78b406dfc0 2023-10-08T07:10:22.720 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:22.719+0000 7f0ec4c10040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:10:22.723 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:22.721+0000 7fc7b9777040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:10:22.771 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:22.770+0000 7f0ec4c10040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:10:22.782 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:22.780+0000 7fc7b9777040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:10:22.833 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:22.832+0000 7fc7b9777040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:10:22.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.866+0000 7f78baed7640 1 -- 172.21.15.70:0/1909948813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78b40b0ef0 con 0x7f78b406dfc0 2023-10-08T07:10:22.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.867+0000 7f78b17fa640 1 -- 172.21.15.70:0/1909948813 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v200) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78a80572f0 con 0x7f78b406dfc0 2023-10-08T07:10:22.869 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:10:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 -- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f789806afc0 msgr2=0x7f789806d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f789806afc0 0x7f789806d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 -- 172.21.15.70:0/1909948813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78b406dfc0 msgr2=0x7f78b415db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78b406dfc0 0x7f78b415db90 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f78a800c390 tx=0x7f78a800c850 comp rx=0 tx=0).stop 2023-10-08T07:10:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 -- 172.21.15.70:0/1909948813 shutdown_connections 2023-10-08T07:10:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f789806afc0 0x7f789806d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78b40afbe0 0x7f78b4158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78b4071070 0x7f78b4158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 --2- 172.21.15.70:0/1909948813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78b406dfc0 0x7f78b415db90 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 -- 172.21.15.70:0/1909948813 >> 172.21.15.70:0/1909948813 conn(0x7f78b406a5d0 msgr2=0x7f78b40c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:22.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 -- 172.21.15.70:0/1909948813 shutdown_connections 2023-10-08T07:10:22.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:22.870+0000 7f78baed7640 1 -- 172.21.15.70:0/1909948813 wait complete. 2023-10-08T07:10:22.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:10:23.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.141+0000 7f8f737ee640 1 Processor -- start 2023-10-08T07:10:23.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.141+0000 7f8f737ee640 1 -- start start 2023-10-08T07:10:23.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.142+0000 7f8f737ee640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8f6c15cf80 0x7f8f6c0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.142+0000 7f8f737ee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f6c0b4b40 0x7f8f6c0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.142+0000 7f8f737ee640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c0b54a0 con 0x7f8f6c15cba0 2023-10-08T07:10:23.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.142+0000 7f8f737ee640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6c0b4ff0 con 0x7f8f6c0b4b40 2023-10-08T07:10:23.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.142+0000 7f8f737ee640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6c0b8bb0 con 0x7f8f6c15cf80 2023-10-08T07:10:23.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.143+0000 7f8f71d8b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f6c0b4b40 0x7f8f6c0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.143+0000 7f8f71d8b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f6c0b4b40 0x7f8f6c0d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60992/0 (socket says 172.21.15.70:60992) 2023-10-08T07:10:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.143+0000 7f8f71d8b640 1 -- 172.21.15.70:0/3254032578 learned_addr learned my addr 172.21.15.70:0/3254032578 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.143+0000 7f8f71d8b640 1 -- 172.21.15.70:0/3254032578 >> v2:172.21.15.181:3300/0 conn(0x7f8f6c15cf80 msgr2=0x7f8f6c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.143+0000 7f8f71d8b640 1 --2- 172.21.15.70:0/3254032578 >> v2:172.21.15.181:3300/0 conn(0x7f8f6c15cf80 0x7f8f6c0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.143+0000 7f8f71d8b640 1 -- 172.21.15.70:0/3254032578 >> v1:172.21.15.181:6789/0 conn(0x7f8f6c15cba0 legacy=0x7f8f6c0b84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.143+0000 7f8f71d8b640 1 -- 172.21.15.70:0/3254032578 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0b7130 con 0x7f8f6c0b4b40 2023-10-08T07:10:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.144+0000 7f8f71d8b640 1 --2- 172.21.15.70:0/3254032578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c0b4b40 0x7f8f6c0d0ee0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009e50 tx=0x7f8f6402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc450c46616a4b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.145+0000 7f8f63fff640 1 -- 172.21.15.70:0/3254032578 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6402c9f0 con 0x7f8f6c0b4b40 2023-10-08T07:10:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.145+0000 7f8f63fff640 1 -- 172.21.15.70:0/3254032578 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6402cb50 con 0x7f8f6c0b4b40 2023-10-08T07:10:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.145+0000 7f8f63fff640 1 -- 172.21.15.70:0/3254032578 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f640343f0 con 0x7f8f6c0b4b40 2023-10-08T07:10:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.145+0000 7f8f737ee640 1 -- 172.21.15.70:0/3254032578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c0b4b40 msgr2=0x7f8f6c0d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.145+0000 7f8f737ee640 1 --2- 172.21.15.70:0/3254032578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c0b4b40 0x7f8f6c0d0ee0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009e50 tx=0x7f8f6402be50 comp rx=0 tx=0).stop 2023-10-08T07:10:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.145+0000 7f8f737ee640 1 -- 172.21.15.70:0/3254032578 shutdown_connections 2023-10-08T07:10:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.145+0000 7f8f737ee640 1 --2- 172.21.15.70:0/3254032578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c0b4b40 0x7f8f6c0d0ee0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.145+0000 7f8f737ee640 1 --2- 172.21.15.70:0/3254032578 >> v2:172.21.15.181:3300/0 conn(0x7f8f6c15cf80 0x7f8f6c0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.145+0000 7f8f737ee640 1 -- 172.21.15.70:0/3254032578 >> 172.21.15.70:0/3254032578 conn(0x7f8f6c0718a0 msgr2=0x7f8f6c071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:23.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.146+0000 7f8f737ee640 1 -- 172.21.15.70:0/3254032578 shutdown_connections 2023-10-08T07:10:23.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.146+0000 7f8f737ee640 1 -- 172.21.15.70:0/3254032578 wait complete. 2023-10-08T07:10:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.147+0000 7f8f737ee640 1 Processor -- start 2023-10-08T07:10:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.147+0000 7f8f737ee640 1 -- start start 2023-10-08T07:10:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.148+0000 7f8f737ee640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f6c0b4b40 0x7f8f6c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.148+0000 7f8f737ee640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f6c15cba0 0x7f8f6c1514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.148+0000 7f8f737ee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15cf80 0x7f8f6c1f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.148+0000 7f8f737ee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0cc6a0 con 0x7f8f6c15cf80 2023-10-08T07:10:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.148+0000 7f8f737ee640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0caa60 con 0x7f8f6c0b4b40 2023-10-08T07:10:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.148+0000 7f8f737ee640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c151a00 con 0x7f8f6c15cba0 2023-10-08T07:10:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f71d8b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15cf80 0x7f8f6c1f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f71d8b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15cf80 0x7f8f6c1f5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60996/0 (socket says 172.21.15.70:60996) 2023-10-08T07:10:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f71d8b640 1 -- 172.21.15.70:0/523099830 learned_addr learned my addr 172.21.15.70:0/523099830 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f7158a640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f6c0b4b40 0x7f8f6c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f70d89640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f6c15cba0 0x7f8f6c1514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f71d8b640 1 -- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f6c15cba0 msgr2=0x7f8f6c1514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f71d8b640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f6c15cba0 0x7f8f6c1514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f71d8b640 1 -- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f6c0b4b40 msgr2=0x7f8f6c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f71d8b640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f6c0b4b40 0x7f8f6c0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f71d8b640 1 -- 172.21.15.70:0/523099830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f64009b30 con 0x7f8f6c15cf80 2023-10-08T07:10:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f7158a640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f6c0b4b40 0x7f8f6c0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:10:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.149+0000 7f8f71d8b640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15cf80 0x7f8f6c1f5e90 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009e50 tx=0x7f8f64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.150+0000 7f8f61ffb640 1 -- 172.21.15.70:0/523099830 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64009510 con 0x7f8f6c15cf80 2023-10-08T07:10:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.150+0000 7f8f61ffb640 1 -- 172.21.15.70:0/523099830 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6403e450 con 0x7f8f6c15cf80 2023-10-08T07:10:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.150+0000 7f8f61ffb640 1 -- 172.21.15.70:0/523099830 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6403e7e0 con 0x7f8f6c15cf80 2023-10-08T07:10:23.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.150+0000 7f8f737ee640 1 -- 172.21.15.70:0/523099830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c0cbe10 con 0x7f8f6c15cf80 2023-10-08T07:10:23.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.150+0000 7f8f737ee640 1 -- 172.21.15.70:0/523099830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c0ca4a0 con 0x7f8f6c15cf80 2023-10-08T07:10:23.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.151+0000 7f8f737ee640 1 -- 172.21.15.70:0/523099830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c0d1510 con 0x7f8f6c15cf80 2023-10-08T07:10:23.154 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:23.153+0000 7f0ec4c10040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:10:23.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.152+0000 7f8f61ffb640 1 -- 172.21.15.70:0/523099830 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 112) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f8f6403e940 con 0x7f8f6c15cf80 2023-10-08T07:10:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.153+0000 7f8f61ffb640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f8f5006af90 0x7f8f5006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.153+0000 7f8f61ffb640 1 -- 172.21.15.70:0/523099830 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6833+0+0 (secure 0 0 0) 0x7f8f640ac920 con 0x7f8f6c15cf80 2023-10-08T07:10:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.153+0000 7f8f7158a640 1 -- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f8f5006af90 msgr2=0x7f8f5006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.154+0000 7f8f7158a640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f8f5006af90 0x7f8f5006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:23.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.157+0000 7f8f61ffb640 1 -- 172.21.15.70:0/523099830 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f64077440 con 0x7f8f6c15cf80 2023-10-08T07:10:23.239 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:23.235+0000 7fc7b9777040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:10:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.355+0000 7f8f7158a640 1 -- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f8f5006af90 msgr2=0x7f8f5006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.355+0000 7f8f7158a640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f8f5006af90 0x7f8f5006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:10:23.362 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:23.361+0000 7f0ec4c10040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:10:23.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.416+0000 7f8f737ee640 1 -- 172.21.15.70:0/523099830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f6c0b54a0 con 0x7f8f6c15cf80 2023-10-08T07:10:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.418+0000 7f8f61ffb640 1 -- 172.21.15.70:0/523099830 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v201) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f6c0b54a0 con 0x7f8f6c15cf80 2023-10-08T07:10:23.419 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:10:23.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.421+0000 7f8f737ee640 1 -- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f8f5006af90 msgr2=0x7f8f5006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:23.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.421+0000 7f8f737ee640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f8f5006af90 0x7f8f5006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.421+0000 7f8f737ee640 1 -- 172.21.15.70:0/523099830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15cf80 msgr2=0x7f8f6c1f5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:23.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.421+0000 7f8f737ee640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15cf80 0x7f8f6c1f5e90 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009e50 tx=0x7f8f64004a40 comp rx=0 tx=0).stop 2023-10-08T07:10:23.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.421+0000 7f8f737ee640 1 -- 172.21.15.70:0/523099830 shutdown_connections 2023-10-08T07:10:23.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.421+0000 7f8f737ee640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f8f5006af90 0x7f8f5006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.421+0000 7f8f737ee640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6c15cf80 0x7f8f6c1f5e90 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.421+0000 7f8f737ee640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f6c15cba0 0x7f8f6c1514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.421+0000 7f8f737ee640 1 --2- 172.21.15.70:0/523099830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f6c0b4b40 0x7f8f6c0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.421+0000 7f8f737ee640 1 -- 172.21.15.70:0/523099830 >> 172.21.15.70:0/523099830 conn(0x7f8f6c0718a0 msgr2=0x7f8f6c0b6500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.422+0000 7f8f737ee640 1 -- 172.21.15.70:0/523099830 shutdown_connections 2023-10-08T07:10:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.422+0000 7f8f737ee640 1 -- 172.21.15.70:0/523099830 wait complete. 2023-10-08T07:10:23.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T07:10:23.480 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:23.478+0000 7fc7b9777040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:10:23.528 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:23.526+0000 7f0ec4c10040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:10:23.579 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:23.578+0000 7f0ec4c10040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:10:23.618 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:23.617+0000 7fc7b9777040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:10:23.670 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:23.668+0000 7fc7b9777040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:10:23.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.669+0000 7f6246802640 1 Processor -- start 2023-10-08T07:10:23.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.669+0000 7f6246802640 1 -- start start 2023-10-08T07:10:23.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.670+0000 7f6246802640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62400afbe0 0x7f62400b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.670+0000 7f6246802640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f624006dfc0 0x7f6240070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.670+0000 7f6246802640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62400b2620 con 0x7f6240071070 2023-10-08T07:10:23.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.670+0000 7f6246802640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62400b2060 con 0x7f624006dfc0 2023-10-08T07:10:23.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.670+0000 7f6246802640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62400b2340 con 0x7f62400afbe0 2023-10-08T07:10:23.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f6244d9f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6240071070 0x7f62400cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53432/0 (socket says 172.21.15.70:53432) 2023-10-08T07:10:23.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f6244d9f640 1 -- 172.21.15.70:0/892749914 learned_addr learned my addr 172.21.15.70:0/892749914 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:23.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623ffff640 1 --2- 172.21.15.70:0/892749914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62400afbe0 0x7f62400b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:23.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623f7fe640 1 --2- 172.21.15.70:0/892749914 >> v2:172.21.15.181:3300/0 conn(0x7f624006dfc0 0x7f6240070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990146828 0 0) 0x7f62400b2620 con 0x7f6240071070 2023-10-08T07:10:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6228003520 con 0x7f6240071070 2023-10-08T07:10:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 492788213 0 0) 0x7f6228003520 con 0x7f6240071070 2023-10-08T07:10:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62400afbe0 msgr2=0x7f62400b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623effd640 1 --2- 172.21.15.70:0/892749914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62400afbe0 0x7f62400b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 >> v2:172.21.15.181:3300/0 conn(0x7f624006dfc0 msgr2=0x7f6240070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:23.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623effd640 1 --2- 172.21.15.70:0/892749914 >> v2:172.21.15.181:3300/0 conn(0x7f624006dfc0 0x7f6240070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6240070350 con 0x7f6240071070 2023-10-08T07:10:23.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.671+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f62380019f0 con 0x7f6240071070 2023-10-08T07:10:23.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 >> v1:172.21.15.70:6789/0 conn(0x7f6240071070 legacy=0x7f62400cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:23.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623effd640 1 --2- 172.21.15.70:0/892749914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6228003970 0x7f6228003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623effd640 1 --2- 172.21.15.70:0/892749914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62280047c0 0x7f6228004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623effd640 1 --2- 172.21.15.70:0/892749914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62400afbe0 0x7f6228005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62400b2060 con 0x7f62280047c0 2023-10-08T07:10:23.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62400b2340 con 0x7f6228003970 2023-10-08T07:10:23.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622800b980 con 0x7f62400afbe0 2023-10-08T07:10:23.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623f7fe640 1 --2- 172.21.15.70:0/892749914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62280047c0 0x7f6228004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:23.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623f7fe640 1 -- 172.21.15.70:0/892749914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62400afbe0 msgr2=0x7f6228005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:23.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623f7fe640 1 --2- 172.21.15.70:0/892749914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62400afbe0 0x7f6228005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623f7fe640 1 -- 172.21.15.70:0/892749914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6228003970 msgr2=0x7f6228003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:23.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623f7fe640 1 --2- 172.21.15.70:0/892749914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6228003970 0x7f6228003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.672+0000 7f623f7fe640 1 -- 172.21.15.70:0/892749914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228003520 con 0x7f62280047c0 2023-10-08T07:10:23.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.673+0000 7f623f7fe640 1 --2- 172.21.15.70:0/892749914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62280047c0 0x7f6228004c40 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f6234009ac0 tx=0x7f62280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e3921bff8fe777 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:23.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.674+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623402c710 con 0x7f62280047c0 2023-10-08T07:10:23.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.674+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623402c870 con 0x7f62280047c0 2023-10-08T07:10:23.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.674+0000 7f623effd640 1 -- 172.21.15.70:0/892749914 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623402cc00 con 0x7f62280047c0 2023-10-08T07:10:23.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.674+0000 7f6246802640 1 -- 172.21.15.70:0/892749914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62280047c0 msgr2=0x7f6228004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:23.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.674+0000 7f6246802640 1 --2- 172.21.15.70:0/892749914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62280047c0 0x7f6228004c40 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f6234009ac0 tx=0x7f62280057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.674+0000 7f6244d9f640 1 -- 172.21.15.70:0/892749914 reap_dead start 2023-10-08T07:10:23.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.675+0000 7f6246802640 1 -- 172.21.15.70:0/892749914 shutdown_connections 2023-10-08T07:10:23.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.675+0000 7f6246802640 1 -- 172.21.15.70:0/892749914 >> 172.21.15.70:0/892749914 conn(0x7f624006a5d0 msgr2=0x7f62400b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:23.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.675+0000 7f6246802640 1 -- 172.21.15.70:0/892749914 shutdown_connections 2023-10-08T07:10:23.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.675+0000 7f6246802640 1 -- 172.21.15.70:0/892749914 wait complete. 2023-10-08T07:10:23.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.676+0000 7f6246802640 1 Processor -- start 2023-10-08T07:10:23.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.677+0000 7f6246802640 1 -- start start 2023-10-08T07:10:23.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.677+0000 7f6246802640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62280047c0 0x7f624014fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.677+0000 7f6246802640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6240150d00 0x7f62400cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.677+0000 7f6246802640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62400cdb60 0x7f62400c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.677+0000 7f6246802640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62400cdfe0 con 0x7f62400cdb60 2023-10-08T07:10:23.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.677+0000 7f6246802640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62400c65c0 con 0x7f6240150d00 2023-10-08T07:10:23.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.677+0000 7f6246802640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62400c59b0 con 0x7f62280047c0 2023-10-08T07:10:23.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.678+0000 7f6244d9f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62400cdb60 0x7f62400c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.678+0000 7f6244d9f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62400cdb60 0x7f62400c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54074/0 (socket says 172.21.15.70:54074) 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.678+0000 7f6244d9f640 1 -- 172.21.15.70:0/1400058160 learned_addr learned my addr 172.21.15.70:0/1400058160 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.678+0000 7f6244d9f640 1 -- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62280047c0 msgr2=0x7f624014fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.678+0000 7f6244d9f640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62280047c0 0x7f624014fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.678+0000 7f6244d9f640 1 -- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6240150d00 msgr2=0x7f62400cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.678+0000 7f6244d9f640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6240150d00 0x7f62400cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.678+0000 7f6244d9f640 1 -- 172.21.15.70:0/1400058160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62400c5c90 con 0x7f62400cdb60 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.678+0000 7f6244d9f640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62400cdb60 0x7f62400c9b60 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f6238001370 tx=0x7f623800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.679+0000 7f623cff9640 1 -- 172.21.15.70:0/1400058160 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623800a610 con 0x7f62400cdb60 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.679+0000 7f623cff9640 1 -- 172.21.15.70:0/1400058160 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623800a770 con 0x7f62400cdb60 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.679+0000 7f623cff9640 1 -- 172.21.15.70:0/1400058160 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623800ab00 con 0x7f62400cdb60 2023-10-08T07:10:23.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.679+0000 7f6246802640 1 -- 172.21.15.70:0/1400058160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62400c5f70 con 0x7f62400cdb60 2023-10-08T07:10:23.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.679+0000 7f6246802640 1 -- 172.21.15.70:0/1400058160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62401500c0 con 0x7f62400cdb60 2023-10-08T07:10:23.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.680+0000 7f6246802640 1 -- 172.21.15.70:0/1400058160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624004e160 con 0x7f62400cdb60 2023-10-08T07:10:23.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.681+0000 7f623cff9640 1 -- 172.21.15.70:0/1400058160 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 112) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f623800ac60 con 0x7f62400cdb60 2023-10-08T07:10:23.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.682+0000 7f623cff9640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f622c06afc0 0x7f622c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:23.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.682+0000 7f623cff9640 1 -- 172.21.15.70:0/1400058160 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6921+0+0 (secure 0 0 0) 0x7f62380894e0 con 0x7f62400cdb60 2023-10-08T07:10:23.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.682+0000 7f623ffff640 1 -- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f622c06afc0 msgr2=0x7f622c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:23.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.682+0000 7f623ffff640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f622c06afc0 0x7f622c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:23.691 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:23.686+0000 7f0ec4c10040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:10:23.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.686+0000 7f623cff9640 1 -- 172.21.15.70:0/1400058160 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6238053f20 con 0x7f62400cdb60 2023-10-08T07:10:23.785 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:23.782+0000 7fc7b9777040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:10:23.796 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:23.795+0000 7f0ec4c10040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:10:23.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.883+0000 7f623ffff640 1 -- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f622c06afc0 msgr2=0x7f622c06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:23.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.883+0000 7f623ffff640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f622c06afc0 0x7f622c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:10:23.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.889+0000 7f6246802640 1 -- 172.21.15.70:0/1400058160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f62400cbd10 con 0x7f62400cdb60 2023-10-08T07:10:23.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.890+0000 7f623cff9640 1 -- 172.21.15.70:0/1400058160 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v201) v1 ==== 81+0+949 (secure 0 0 0) 0x7f623801a080 con 0x7f62400cdb60 2023-10-08T07:10:23.892 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2946222544 2023-10-09T07:05:21.112418+0000 2023-10-08T07:10:23.892 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1250722759 2023-10-09T07:05:21.112418+0000 2023-10-08T07:10:23.892 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/3979056493 2023-10-09T07:05:21.112418+0000 2023-10-08T07:10:23.892 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/3979056493 2023-10-09T07:05:21.112418+0000 2023-10-08T07:10:23.892 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/340227645 2023-10-09T07:05:21.112418+0000 2023-10-08T07:10:23.892 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/849801254 2023-10-09T07:05:21.112418+0000 2023-10-08T07:10:23.893 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/4017559811 2023-10-09T07:05:21.112418+0000 2023-10-08T07:10:23.893 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/64410277 2023-10-09T06:54:47.949157+0000 2023-10-08T07:10:23.893 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/811429527 2023-10-09T06:54:47.949157+0000 2023-10-08T07:10:23.893 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/772181101 2023-10-09T06:54:47.949157+0000 2023-10-08T07:10:23.893 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1286852011 2023-10-09T07:05:21.112418+0000 2023-10-08T07:10:23.893 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/3709692999 2023-10-09T06:54:47.949157+0000 2023-10-08T07:10:23.893 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/4147168289 2023-10-09T06:54:47.949157+0000 2023-10-08T07:10:23.893 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/3709692999 2023-10-09T06:54:47.949157+0000 2023-10-08T07:10:23.893 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/602716182 2023-10-09T06:54:47.949157+0000 2023-10-08T07:10:23.893 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1777884673 2023-10-09T06:54:47.949157+0000 2023-10-08T07:10:23.894 INFO:teuthology.orchestra.run.smithi070.stderr:listed 16 entries 2023-10-08T07:10:23.894 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:23.893+0000 7fc7b9777040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:10:23.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 -- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f622c06afc0 msgr2=0x7f622c06d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:23.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f622c06afc0 0x7f622c06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 -- 172.21.15.70:0/1400058160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62400cdb60 msgr2=0x7f62400c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:23.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62400cdb60 0x7f62400c9b60 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f6238001370 tx=0x7f623800ccd0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 -- 172.21.15.70:0/1400058160 shutdown_connections 2023-10-08T07:10:23.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f622c06afc0 0x7f622c06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62400cdb60 0x7f62400c9b60 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6240150d00 0x7f62400cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 --2- 172.21.15.70:0/1400058160 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62280047c0 0x7f624014fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 -- 172.21.15.70:0/1400058160 >> 172.21.15.70:0/1400058160 conn(0x7f624006a5d0 msgr2=0x7f62400b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 -- 172.21.15.70:0/1400058160 shutdown_connections 2023-10-08T07:10:23.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:23.893+0000 7f6246802640 1 -- 172.21.15.70:0/1400058160 wait complete. 2023-10-08T07:10:23.912 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T07:10:23.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:23.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T07:10:23.952 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:23.951+0000 7f0ec4c10040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:10:24.003 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:24.001+0000 7f0ec4c10040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:10:24.062 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:24.060+0000 7fc7b9777040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:10:24.063 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T07:10:24.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:24.063 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T07:10:24.109 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:10:24.109 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:10:24.116 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:24.114+0000 7fc7b9777040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:10:24.128 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:24.127+0000 7f0ec4c10040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:10:24.163 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:10:24.164 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:10:24.164 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:10:24.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:10:24.216 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:10:24.216 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:10:24.243 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:24.241+0000 7fc7b9777040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:10:24.271 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:10:24.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:10:24.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.481+0000 7f947afe9640 1 Processor -- start 2023-10-08T07:10:24.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.481+0000 7f947afe9640 1 -- start start 2023-10-08T07:10:24.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.483+0000 7f947afe9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f947406aa80 0x7f94740711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.483+0000 7f947afe9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9474069d50 con 0x7f9474160240 2023-10-08T07:10:24.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.483+0000 7f947afe9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9474071820 con 0x7f94740b1280 2023-10-08T07:10:24.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.483+0000 7f947afe9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9474069fa0 con 0x7f947406aa80 2023-10-08T07:10:24.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.483+0000 7f9478d85640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9474160240 0x7f94740b0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53448/0 (socket says 172.21.15.70:53448) 2023-10-08T07:10:24.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.483+0000 7f9478d85640 1 -- 172.21.15.70:0/1045830932 learned_addr learned my addr 172.21.15.70:0/1045830932 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:24.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.483+0000 7f9479586640 1 --2- 172.21.15.70:0/1045830932 >> v2:172.21.15.70:3300/0 conn(0x7f947406aa80 0x7f94740711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:24.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.483+0000 7f9479586640 1 -- 172.21.15.70:0/1045830932 >> v1:172.21.15.181:6789/0 conn(0x7f94740b1280 legacy=0x7f9474070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:24.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.483+0000 7f9479586640 1 -- 172.21.15.70:0/1045830932 >> v1:172.21.15.70:6789/0 conn(0x7f9474160240 legacy=0x7f94740b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:24.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.483+0000 7f9479586640 1 -- 172.21.15.70:0/1045830932 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947406a280 con 0x7f947406aa80 2023-10-08T07:10:24.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.485+0000 7f9479586640 1 --2- 172.21.15.70:0/1045830932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 0x7f94740711e0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f946c009d90 tx=0x7f946c02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90825dcc7bb1227 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:24.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.485+0000 7f94737fe640 1 -- 172.21.15.70:0/1045830932 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c02ccf0 con 0x7f947406aa80 2023-10-08T07:10:24.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.485+0000 7f94737fe640 1 -- 172.21.15.70:0/1045830932 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c02ce50 con 0x7f947406aa80 2023-10-08T07:10:24.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.485+0000 7f94737fe640 1 -- 172.21.15.70:0/1045830932 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c034660 con 0x7f947406aa80 2023-10-08T07:10:24.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.485+0000 7f947afe9640 1 -- 172.21.15.70:0/1045830932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 msgr2=0x7f94740711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:24.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.485+0000 7f947afe9640 1 --2- 172.21.15.70:0/1045830932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 0x7f94740711e0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f946c009d90 tx=0x7f946c02bf40 comp rx=0 tx=0).stop 2023-10-08T07:10:24.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.486+0000 7f947afe9640 1 -- 172.21.15.70:0/1045830932 shutdown_connections 2023-10-08T07:10:24.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.486+0000 7f947afe9640 1 --2- 172.21.15.70:0/1045830932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 0x7f94740711e0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.486+0000 7f947afe9640 1 -- 172.21.15.70:0/1045830932 >> 172.21.15.70:0/1045830932 conn(0x7f94740cf890 msgr2=0x7f94740cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:24.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.486+0000 7f947afe9640 1 -- 172.21.15.70:0/1045830932 shutdown_connections 2023-10-08T07:10:24.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.486+0000 7f947afe9640 1 -- 172.21.15.70:0/1045830932 wait complete. 2023-10-08T07:10:24.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.487+0000 7f947afe9640 1 Processor -- start 2023-10-08T07:10:24.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.487+0000 7f947afe9640 1 -- start start 2023-10-08T07:10:24.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.488+0000 7f947afe9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 0x7f94740afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.488+0000 7f947afe9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94740b1280 0x7f94740b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.488+0000 7f947afe9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9474160240 0x7f947415c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.488+0000 7f947afe9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947415d200 con 0x7f947406aa80 2023-10-08T07:10:24.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.488+0000 7f947afe9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94740c6f60 con 0x7f9474160240 2023-10-08T07:10:24.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.488+0000 7f947afe9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94740c7240 con 0x7f94740b1280 2023-10-08T07:10:24.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9478d85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 0x7f94740afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:24.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9478d85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 0x7f94740afce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54082/0 (socket says 172.21.15.70:54082) 2023-10-08T07:10:24.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9478d85640 1 -- 172.21.15.70:0/3869250041 learned_addr learned my addr 172.21.15.70:0/3869250041 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:24.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9473fff640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94740b1280 0x7f94740b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:24.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9478d85640 1 -- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94740b1280 msgr2=0x7f94740b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:24.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9479586640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9474160240 0x7f947415c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9478d85640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94740b1280 0x7f94740b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9478d85640 1 -- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9474160240 msgr2=0x7f947415c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9478d85640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9474160240 0x7f947415c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9478d85640 1 -- 172.21.15.70:0/3869250041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c009a70 con 0x7f947406aa80 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9473fff640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94740b1280 0x7f94740b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.489+0000 7f9478d85640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 0x7f94740afce0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f946800c480 tx=0x7f946800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.490+0000 7f94717fa640 1 -- 172.21.15.70:0/3869250041 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468005ce0 con 0x7f947406aa80 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.490+0000 7f94717fa640 1 -- 172.21.15.70:0/3869250041 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468005e40 con 0x7f947406aa80 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.490+0000 7f947afe9640 1 -- 172.21.15.70:0/3869250041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94740c7520 con 0x7f947406aa80 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.490+0000 7f94717fa640 1 -- 172.21.15.70:0/3869250041 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680072f0 con 0x7f947406aa80 2023-10-08T07:10:24.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.490+0000 7f947afe9640 1 -- 172.21.15.70:0/3869250041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947415ac30 con 0x7f947406aa80 2023-10-08T07:10:24.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.491+0000 7f947afe9640 1 -- 172.21.15.70:0/3869250041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94740662c0 con 0x7f947406aa80 2023-10-08T07:10:24.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.492+0000 7f94717fa640 1 -- 172.21.15.70:0/3869250041 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 112) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f9468007510 con 0x7f947406aa80 2023-10-08T07:10:24.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.493+0000 7f94717fa640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f945806af90 0x7f945806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.493+0000 7f94717fa640 1 -- 172.21.15.70:0/3869250041 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f9468089140 con 0x7f947406aa80 2023-10-08T07:10:24.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.493+0000 7f9473fff640 1 -- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f945806af90 msgr2=0x7f945806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:24.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.493+0000 7f9473fff640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f945806af90 0x7f945806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:24.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.497+0000 7f94717fa640 1 -- 172.21.15.70:0/3869250041 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9468053cd0 con 0x7f947406aa80 2023-10-08T07:10:24.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.694+0000 7f9473fff640 1 -- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f945806af90 msgr2=0x7f945806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:24.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.694+0000 7f9473fff640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f945806af90 0x7f945806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:10:24.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.729+0000 7f947afe9640 1 -- 172.21.15.70:0/3869250041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9474066d80 con 0x7f947406aa80 2023-10-08T07:10:24.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.730+0000 7f94717fa640 1 -- 172.21.15.70:0/3869250041 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f94680572f0 con 0x7f947406aa80 2023-10-08T07:10:24.732 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:24.732 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/381523318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":381523318},{"type":"v1","addr":"172.21.15.181:6835","nonce":381523318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2153071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2153071509},{"type":"v1","addr":"172.21.15.181:6837","nonce":2153071509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2673497542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2673497542},{"type":"v1","addr":"172.21.15.70:6835","nonce":2673497542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:54:32.491331+0000","modified":"2023-10-08T06:54:35.534957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10074":{"gid":10074,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2138719113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2138719113},{"type":"v1","addr":"172.21.15.70:6833","nonce":2138719113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-08T07:10:24.732 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-10-08T07:10:24.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.732+0000 7f947afe9640 1 -- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f945806af90 msgr2=0x7f945806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:24.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.732+0000 7f947afe9640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f945806af90 0x7f945806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.732+0000 7f947afe9640 1 -- 172.21.15.70:0/3869250041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 msgr2=0x7f94740afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:24.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.732+0000 7f947afe9640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 0x7f94740afce0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f946800c480 tx=0x7f946800c940 comp rx=0 tx=0).stop 2023-10-08T07:10:24.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.732+0000 7f947afe9640 1 -- 172.21.15.70:0/3869250041 shutdown_connections 2023-10-08T07:10:24.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.732+0000 7f947afe9640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f945806af90 0x7f945806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.732+0000 7f947afe9640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9474160240 0x7f947415c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.732+0000 7f947afe9640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f94740b1280 0x7f94740b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.732+0000 7f947afe9640 1 --2- 172.21.15.70:0/3869250041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f947406aa80 0x7f94740afce0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.732+0000 7f947afe9640 1 -- 172.21.15.70:0/3869250041 >> 172.21.15.70:0/3869250041 conn(0x7f94740cf890 msgr2=0x7f9474160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:24.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.733+0000 7f947afe9640 1 -- 172.21.15.70:0/3869250041 shutdown_connections 2023-10-08T07:10:24.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.733+0000 7f947afe9640 1 -- 172.21.15.70:0/3869250041 wait complete. 2023-10-08T07:10:24.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:10:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.959+0000 7f9fa7de2640 1 Processor -- start 2023-10-08T07:10:24.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.959+0000 7f9fa7de2640 1 -- start start 2023-10-08T07:10:24.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa7de2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa015cc20 0x7f9fa006ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa7de2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9fa015d000 0x7f9fa00cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa7de2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9fa00cf480 0x7f9fa00cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa7de2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa006b3a0 con 0x7f9fa00cf480 2023-10-08T07:10:24.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa7de2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa0161750 con 0x7f9fa015d000 2023-10-08T07:10:24.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa7de2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00cb5c0 con 0x7f9fa015cc20 2023-10-08T07:10:24.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa637f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9fa00cf480 0x7f9fa00cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:24.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa637f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9fa00cf480 0x7f9fa00cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54092/0 (socket says 172.21.15.70:54092) 2023-10-08T07:10:24.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa637f640 1 -- 172.21.15.70:0/1990472112 learned_addr learned my addr 172.21.15.70:0/1990472112 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:24.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa5b7e640 1 --2- 172.21.15.70:0/1990472112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa015cc20 0x7f9fa006ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:24.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa637f640 1 -- 172.21.15.70:0/1990472112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa015cc20 msgr2=0x7f9fa006ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:24.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa637f640 1 --2- 172.21.15.70:0/1990472112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa015cc20 0x7f9fa006ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa637f640 1 -- 172.21.15.70:0/1990472112 >> v2:172.21.15.181:3300/0 conn(0x7f9fa015d000 msgr2=0x7f9fa00cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:24.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa637f640 1 --2- 172.21.15.70:0/1990472112 >> v2:172.21.15.181:3300/0 conn(0x7f9fa015d000 0x7f9fa00cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.961+0000 7f9fa637f640 1 -- 172.21.15.70:0/1990472112 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa00cb8a0 con 0x7f9fa00cf480 2023-10-08T07:10:24.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.962+0000 7f9fa637f640 1 --2- 172.21.15.70:0/1990472112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 0x7f9fa00cd770 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00ce320 tx=0x7f9f9802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d7e59f92e9acd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:24.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.963+0000 7f9fa4b7c640 1 -- 172.21.15.70:0/1990472112 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9802ca70 con 0x7f9fa00cf480 2023-10-08T07:10:24.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.963+0000 7f9fa4b7c640 1 -- 172.21.15.70:0/1990472112 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9802cbd0 con 0x7f9fa00cf480 2023-10-08T07:10:24.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.963+0000 7f9fa4b7c640 1 -- 172.21.15.70:0/1990472112 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980343f0 con 0x7f9fa00cf480 2023-10-08T07:10:24.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.963+0000 7f9fa7de2640 1 -- 172.21.15.70:0/1990472112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 msgr2=0x7f9fa00cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:24.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.963+0000 7f9fa7de2640 1 --2- 172.21.15.70:0/1990472112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 0x7f9fa00cd770 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00ce320 tx=0x7f9f9802bed0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.963+0000 7f9fa7de2640 1 -- 172.21.15.70:0/1990472112 shutdown_connections 2023-10-08T07:10:24.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.963+0000 7f9fa7de2640 1 --2- 172.21.15.70:0/1990472112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 0x7f9fa00cd770 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.963+0000 7f9fa7de2640 1 --2- 172.21.15.70:0/1990472112 >> v2:172.21.15.181:3300/0 conn(0x7f9fa015d000 0x7f9fa00cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.963+0000 7f9fa7de2640 1 --2- 172.21.15.70:0/1990472112 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa015cc20 0x7f9fa006ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.963+0000 7f9fa7de2640 1 -- 172.21.15.70:0/1990472112 >> 172.21.15.70:0/1990472112 conn(0x7f9fa0069f30 msgr2=0x7f9fa006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:24.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.964+0000 7f9fa7de2640 1 -- 172.21.15.70:0/1990472112 shutdown_connections 2023-10-08T07:10:24.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.964+0000 7f9fa7de2640 1 -- 172.21.15.70:0/1990472112 wait complete. 2023-10-08T07:10:24.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.965+0000 7f9fa7de2640 1 Processor -- start 2023-10-08T07:10:24.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.965+0000 7f9fa7de2640 1 -- start start 2023-10-08T07:10:24.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.966+0000 7f9fa7de2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 0x7f9fa00b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.966+0000 7f9fa7de2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa015cc20 0x7f9fa00c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.966+0000 7f9fa7de2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa00ca2d0 0x7f9fa006c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.966+0000 7f9fa7de2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00c6e60 con 0x7f9fa00cf480 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.966+0000 7f9fa7de2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00b5710 con 0x7f9fa015cc20 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.966+0000 7f9fa7de2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00b5150 con 0x7f9fa00ca2d0 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.966+0000 7f9fa5b7e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 0x7f9fa00b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa5b7e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 0x7f9fa00b4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54094/0 (socket says 172.21.15.70:54094) 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa5b7e640 1 -- 172.21.15.70:0/2140563648 learned_addr learned my addr 172.21.15.70:0/2140563648 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa637f640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa00ca2d0 0x7f9fa006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa5b7e640 1 -- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa00ca2d0 msgr2=0x7f9fa006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa537d640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa015cc20 0x7f9fa00c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa5b7e640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa00ca2d0 0x7f9fa006c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa5b7e640 1 -- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa015cc20 msgr2=0x7f9fa00c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa5b7e640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa015cc20 0x7f9fa00c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:24.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa5b7e640 1 -- 172.21.15.70:0/2140563648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98009b30 con 0x7f9fa00cf480 2023-10-08T07:10:24.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa537d640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa015cc20 0x7f9fa00c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:24.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9fa5b7e640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 0x7f9fa00b4b20 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00d800 tx=0x7f9f9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:24.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.967+0000 7f9f967fc640 1 -- 172.21.15.70:0/2140563648 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c00b6c0 con 0x7f9fa00cf480 2023-10-08T07:10:24.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.968+0000 7f9f967fc640 1 -- 172.21.15.70:0/2140563648 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9c00b820 con 0x7f9fa00cf480 2023-10-08T07:10:24.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.968+0000 7f9f967fc640 1 -- 172.21.15.70:0/2140563648 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c00bbb0 con 0x7f9fa00cf480 2023-10-08T07:10:24.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.968+0000 7f9fa7de2640 1 -- 172.21.15.70:0/2140563648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa00b5430 con 0x7f9fa00cf480 2023-10-08T07:10:24.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.968+0000 7f9fa7de2640 1 -- 172.21.15.70:0/2140563648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa00c45d0 con 0x7f9fa00cf480 2023-10-08T07:10:24.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.969+0000 7f9fa7de2640 1 -- 172.21.15.70:0/2140563648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa00c4790 con 0x7f9fa00cf480 2023-10-08T07:10:24.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.970+0000 7f9f967fc640 1 -- 172.21.15.70:0/2140563648 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 112) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f9f9c00bd10 con 0x7f9fa00cf480 2023-10-08T07:10:24.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.970+0000 7f9f967fc640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f9f8406afc0 0x7f9f8406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:24.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.971+0000 7f9f967fc640 1 -- 172.21.15.70:0/2140563648 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f9f9c08a140 con 0x7f9fa00cf480 2023-10-08T07:10:24.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.971+0000 7f9fa537d640 1 -- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f9f8406afc0 msgr2=0x7f9f8406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:24.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.971+0000 7f9fa537d640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f9f8406afc0 0x7f9f8406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:24.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:24.975+0000 7f9f967fc640 1 -- 172.21.15.70:0/2140563648 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f9c054c20 con 0x7f9fa00cf480 2023-10-08T07:10:24.989 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:24.987+0000 7f0ec4c10040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:10:25.096 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:25.094+0000 7fc7b9777040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:10:25.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.170+0000 7f9fa7de2640 1 -- 172.21.15.70:0/2140563648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fa0066d80 con 0x7f9fa00cf480 2023-10-08T07:10:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.172+0000 7f9fa537d640 1 -- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f9f8406afc0 msgr2=0x7f9f8406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:25.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.172+0000 7f9fa537d640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f9f8406afc0 0x7f9f8406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:10:25.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.172+0000 7f9f967fc640 1 -- 172.21.15.70:0/2140563648 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+18041 (secure 0 0 0) 0x7f9f9c058240 con 0x7f9fa00cf480 2023-10-08T07:10:25.174 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:25.174 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":202,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:10:24.225192+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:54:30.246375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-08T06:54:31.487900+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"202","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":202,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2946222544":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/1250722759":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6833/3979056493":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6832/3979056493":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/340227645":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/849801254":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/4017559811":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/64410277":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/811429527":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/772181101":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/1286852011":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6832/3709692999":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/4147168289":"2023-10-09T06:54:47.949157+0000","172.21.15.181:6833/3709692999":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/602716182":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/1777884673":"2023-10-09T06:54:47.949157+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:10:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.175+0000 7f9fa7de2640 1 -- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f9f8406afc0 msgr2=0x7f9f8406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:25.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.175+0000 7f9fa7de2640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f9f8406afc0 0x7f9f8406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.175+0000 7f9fa7de2640 1 -- 172.21.15.70:0/2140563648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 msgr2=0x7f9fa00b4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.175+0000 7f9fa7de2640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 0x7f9fa00b4b20 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00d800 tx=0x7f9f9c00dcc0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.175+0000 7f9fa7de2640 1 -- 172.21.15.70:0/2140563648 shutdown_connections 2023-10-08T07:10:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.175+0000 7f9fa7de2640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f9f8406afc0 0x7f9f8406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.175+0000 7f9fa7de2640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9fa00ca2d0 0x7f9fa006c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.175+0000 7f9fa7de2640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9fa015cc20 0x7f9fa00c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.175+0000 7f9fa7de2640 1 --2- 172.21.15.70:0/2140563648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fa00cf480 0x7f9fa00b4b20 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.175+0000 7f9fa7de2640 1 -- 172.21.15.70:0/2140563648 >> 172.21.15.70:0/2140563648 conn(0x7f9fa0069f30 msgr2=0x7f9fa00afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.176+0000 7f9fa7de2640 1 -- 172.21.15.70:0/2140563648 shutdown_connections 2023-10-08T07:10:25.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.176+0000 7f9fa7de2640 1 -- 172.21.15.70:0/2140563648 wait complete. 2023-10-08T07:10:25.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T06:54:30.246375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T06:54:31.487900+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '202', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 202, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:10:25.196 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:10:25.291 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.290+0000 7f0ec4c10040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:10:25.348 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.347+0000 7f0ec4c10040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:10:25.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.397+0000 7f69effff640 1 Processor -- start 2023-10-08T07:10:25.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.397+0000 7f69effff640 1 -- start start 2023-10-08T07:10:25.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.399+0000 7f69effff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f69f00caef0 0x7f69f00cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.399+0000 7f69effff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cdc30 0x7f69f00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.400+0000 7f69effff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f00cb810 con 0x7f69f00cce90 2023-10-08T07:10:25.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.400+0000 7f69effff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f015ac90 con 0x7f69f00caef0 2023-10-08T07:10:25.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.400+0000 7f69effff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f015a6d0 con 0x7f69f00cdc30 2023-10-08T07:10:25.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.400+0000 7f69e7fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cdc30 0x7f69f00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.400+0000 7f69e7fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cdc30 0x7f69f00cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:50024/0 (socket says 172.21.15.70:50024) 2023-10-08T07:10:25.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.400+0000 7f69e7fff640 1 -- 172.21.15.70:0/4075663733 learned_addr learned my addr 172.21.15.70:0/4075663733 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:25.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.400+0000 7f69eeffd640 1 --2- 172.21.15.70:0/4075663733 >> v2:172.21.15.181:3300/0 conn(0x7f69f00caef0 0x7f69f00cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.401+0000 7f69e7fff640 1 -- 172.21.15.70:0/4075663733 >> v2:172.21.15.181:3300/0 conn(0x7f69f00caef0 msgr2=0x7f69f00cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:25.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.401+0000 7f69e7fff640 1 --2- 172.21.15.70:0/4075663733 >> v2:172.21.15.181:3300/0 conn(0x7f69f00caef0 0x7f69f00cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.401+0000 7f69e7fff640 1 -- 172.21.15.70:0/4075663733 >> v1:172.21.15.181:6789/0 conn(0x7f69f00cce90 legacy=0x7f69f0159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.401+0000 7f69e7fff640 1 -- 172.21.15.70:0/4075663733 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f015a9b0 con 0x7f69f00cdc30 2023-10-08T07:10:25.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.402+0000 7f69e7fff640 1 --2- 172.21.15.70:0/4075663733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cdc30 0x7f69f00cc950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009870 tx=0x7f69d802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95bf1ae3140253c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:25.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.403+0000 7f69ee7fc640 1 -- 172.21.15.70:0/4075663733 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8030840 con 0x7f69f00cdc30 2023-10-08T07:10:25.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.403+0000 7f69ee7fc640 1 -- 172.21.15.70:0/4075663733 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d80309a0 con 0x7f69f00cdc30 2023-10-08T07:10:25.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.403+0000 7f69ee7fc640 1 -- 172.21.15.70:0/4075663733 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8030d30 con 0x7f69f00cdc30 2023-10-08T07:10:25.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.403+0000 7f69effff640 1 -- 172.21.15.70:0/4075663733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cdc30 msgr2=0x7f69f00cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.403+0000 7f69effff640 1 --2- 172.21.15.70:0/4075663733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cdc30 0x7f69f00cc950 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009870 tx=0x7f69d802b980 comp rx=0 tx=0).stop 2023-10-08T07:10:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.404+0000 7f69effff640 1 -- 172.21.15.70:0/4075663733 shutdown_connections 2023-10-08T07:10:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.404+0000 7f69effff640 1 --2- 172.21.15.70:0/4075663733 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cdc30 0x7f69f00cc950 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.404+0000 7f69effff640 1 --2- 172.21.15.70:0/4075663733 >> v2:172.21.15.181:3300/0 conn(0x7f69f00caef0 0x7f69f00cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.404+0000 7f69effff640 1 -- 172.21.15.70:0/4075663733 >> 172.21.15.70:0/4075663733 conn(0x7f69f006c010 msgr2=0x7f69f006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.404+0000 7f69effff640 1 -- 172.21.15.70:0/4075663733 shutdown_connections 2023-10-08T07:10:25.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.404+0000 7f69effff640 1 -- 172.21.15.70:0/4075663733 wait complete. 2023-10-08T07:10:25.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.406+0000 7f69effff640 1 Processor -- start 2023-10-08T07:10:25.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.407+0000 7f69effff640 1 -- start start 2023-10-08T07:10:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.408+0000 7f69effff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f00caef0 0x7f69f00c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.408+0000 7f69effff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cce90 0x7f69f01f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.408+0000 7f69effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69f00cdc30 0x7f69f015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.408+0000 7f69effff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f01f2000 con 0x7f69f00cdc30 2023-10-08T07:10:25.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.408+0000 7f69effff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f01f1a40 con 0x7f69f00caef0 2023-10-08T07:10:25.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.408+0000 7f69effff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f01f1d20 con 0x7f69f00cce90 2023-10-08T07:10:25.411 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:25.408+0000 7fc7b9777040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:10:25.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.408+0000 7f69ef7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69f00cdc30 0x7f69f015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.409+0000 7f69ef7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69f00cdc30 0x7f69f015add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54100/0 (socket says 172.21.15.70:54100) 2023-10-08T07:10:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.409+0000 7f69ef7fe640 1 -- 172.21.15.70:0/3957467967 learned_addr learned my addr 172.21.15.70:0/3957467967 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.409+0000 7f69e7fff640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cce90 0x7f69f01f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.409+0000 7f69eeffd640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f00caef0 0x7f69f00c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.409+0000 7f69ef7fe640 1 -- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cce90 msgr2=0x7f69f01f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.409+0000 7f69ef7fe640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cce90 0x7f69f01f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.409+0000 7f69ef7fe640 1 -- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f00caef0 msgr2=0x7f69f00c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.409+0000 7f69ef7fe640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f00caef0 0x7f69f00c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.409+0000 7f69ef7fe640 1 -- 172.21.15.70:0/3957467967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80094d0 con 0x7f69f00cdc30 2023-10-08T07:10:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.409+0000 7f69ef7fe640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69f00cdc30 0x7f69f015add0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f69e800c730 tx=0x7f69e800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.410+0000 7f69e77fe640 1 -- 172.21.15.70:0/3957467967 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8003b70 con 0x7f69f00cdc30 2023-10-08T07:10:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.410+0000 7f69e77fe640 1 -- 172.21.15.70:0/3957467967 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e8003cd0 con 0x7f69f00cdc30 2023-10-08T07:10:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.410+0000 7f69e77fe640 1 -- 172.21.15.70:0/3957467967 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e800aa90 con 0x7f69f00cdc30 2023-10-08T07:10:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.410+0000 7f69effff640 1 -- 172.21.15.70:0/3957467967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f00c57e0 con 0x7f69f00cdc30 2023-10-08T07:10:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.410+0000 7f69effff640 1 -- 172.21.15.70:0/3957467967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f00c88c0 con 0x7f69f00cdc30 2023-10-08T07:10:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.411+0000 7f69effff640 1 -- 172.21.15.70:0/3957467967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f00c5d00 con 0x7f69f00cdc30 2023-10-08T07:10:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.413+0000 7f69e77fe640 1 -- 172.21.15.70:0/3957467967 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 112) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f69e800abf0 con 0x7f69f00cdc30 2023-10-08T07:10:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.414+0000 7f69e77fe640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f69d006af90 0x7f69d006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.414+0000 7f69e77fe640 1 -- 172.21.15.70:0/3957467967 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6905+0+0 (secure 0 0 0) 0x7f69e80899a0 con 0x7f69f00cdc30 2023-10-08T07:10:25.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.414+0000 7f69eeffd640 1 -- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f69d006af90 msgr2=0x7f69d006d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2291259069 2023-10-08T07:10:25.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.414+0000 7f69eeffd640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f69d006af90 0x7f69d006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:25.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.418+0000 7f69e77fe640 1 -- 172.21.15.70:0/3957467967 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f69f00c5d00 con 0x7f69f00cdc30 2023-10-08T07:10:25.464 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.463+0000 7f0ec4c10040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:10:25.471 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:25.469+0000 7fc7b9777040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:10:25.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.483+0000 7f69e77fe640 1 -- 172.21.15.70:0/3957467967 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 113) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f69e8054850 con 0x7f69f00cdc30 2023-10-08T07:10:25.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.483+0000 7f69e77fe640 1 -- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f69d006af90 msgr2=0x7f69d006d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:25.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.483+0000 7f69e77fe640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f69d006af90 0x7f69d006d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.558 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.557+0000 7f0e46ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:25.558 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.557+0000 7f0e492d7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:25.558 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.557+0000 7f0e492d7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:25.558 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.557+0000 7f0e492d7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:25.559 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.557+0000 7f0e46ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:25.559 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.557+0000 7f0e46ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:25.559 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.557+0000 7f0e492d7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:25.559 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.557+0000 7f0e492d7640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:25.559 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.557+0000 7f0e46ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:25.559 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:25.557+0000 7f0e46ad2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:25.599 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:25.598+0000 7fc7b9777040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:10:25.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.623+0000 7f69effff640 1 -- 172.21.15.70:0/3957467967 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69f00c5d00 con 0x7f69f00cdc30 2023-10-08T07:10:25.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.624+0000 7f69e77fe640 1 -- 172.21.15.70:0/3957467967 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v203) v1 ==== 93+0+10 (secure 0 0 0) 0x7f69f00c5d00 con 0x7f69f00cdc30 2023-10-08T07:10:25.626 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:10:25.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.628+0000 7f69effff640 1 -- 172.21.15.70:0/3957467967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69f00cdc30 msgr2=0x7f69f015add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.628+0000 7f69effff640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69f00cdc30 0x7f69f015add0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f69e800c730 tx=0x7f69e800cbf0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.628+0000 7f69effff640 1 -- 172.21.15.70:0/3957467967 shutdown_connections 2023-10-08T07:10:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.628+0000 7f69effff640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:6832/2291259069,v1:172.21.15.181:6833/2291259069] conn(0x7f69d006af90 0x7f69d006d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.628+0000 7f69effff640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69f00cdc30 0x7f69f015add0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.628+0000 7f69effff640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f69f00cce90 0x7f69f01f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.628+0000 7f69effff640 1 --2- 172.21.15.70:0/3957467967 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f69f00caef0 0x7f69f00c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.628+0000 7f69effff640 1 -- 172.21.15.70:0/3957467967 >> 172.21.15.70:0/3957467967 conn(0x7f69f006c010 msgr2=0x7f69f00ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.628+0000 7f69effff640 1 -- 172.21.15.70:0/3957467967 shutdown_connections 2023-10-08T07:10:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.628+0000 7f69effff640 1 -- 172.21.15.70:0/3957467967 wait complete. 2023-10-08T07:10:25.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:10:25.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.855+0000 7f707764b640 1 Processor -- start 2023-10-08T07:10:25.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.856+0000 7f707764b640 1 -- start start 2023-10-08T07:10:25.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.857+0000 7f707764b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f707015cd80 0x7f70700b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.857+0000 7f707764b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f707015d160 con 0x7f70700cdf70 2023-10-08T07:10:25.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.857+0000 7f707764b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70700b7b90 con 0x7f70700b6730 2023-10-08T07:10:25.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.857+0000 7f707764b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70700b8020 con 0x7f707015cd80 2023-10-08T07:10:25.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.857+0000 7f7075be8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f70700cdf70 0x7f70700d0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53454/0 (socket says 172.21.15.70:53454) 2023-10-08T07:10:25.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.857+0000 7f7075be8640 1 -- 172.21.15.70:0/848070224 learned_addr learned my addr 172.21.15.70:0/848070224 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f70753e7640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f707015cd80 0x7f70700b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296669493 0 0) 0x7f707015d160 con 0x7f70700cdf70 2023-10-08T07:10:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7058003520 con 0x7f70700cdf70 2023-10-08T07:10:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2816051527 0 0) 0x7f7058003520 con 0x7f70700cdf70 2023-10-08T07:10:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f707015cd80 msgr2=0x7f70700b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f707015cd80 0x7f70700b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 >> v1:172.21.15.181:6789/0 conn(0x7f70700b6730 legacy=0x7f70700cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70700b6f80 con 0x7f70700cdf70 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f70680019f0 con 0x7f70700cdf70 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 >> v1:172.21.15.70:6789/0 conn(0x7f70700cdf70 legacy=0x7f70700d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70580039a0 0x7f7058003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f707015cd80 0x7f70580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 0x7f7058005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70700b8020 con 0x7f7058004d00 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705800ba60 con 0x7f70580039a0 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.858+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705800bc20 con 0x7f707015cd80 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.859+0000 7f7075be8640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 0x7f7058005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.859+0000 7f70753e7640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70580039a0 0x7f7058003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.859+0000 7f7075be8640 1 -- 172.21.15.70:0/848070224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f707015cd80 msgr2=0x7f70580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.859+0000 7f7075be8640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f707015cd80 0x7f70580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.859+0000 7f7075be8640 1 -- 172.21.15.70:0/848070224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70580039a0 msgr2=0x7f7058003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.859+0000 7f7075be8640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70580039a0 0x7f7058003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.859+0000 7f7075be8640 1 -- 172.21.15.70:0/848070224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7058003520 con 0x7f7058004d00 2023-10-08T07:10:25.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.860+0000 7f7075be8640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 0x7f7058005180 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f7068002230 tx=0x7f70580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425becf8b8d6f3ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:25.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.860+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068034630 con 0x7f7058004d00 2023-10-08T07:10:25.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.860+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706802f070 con 0x7f7058004d00 2023-10-08T07:10:25.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.860+0000 7f706ffff640 1 -- 172.21.15.70:0/848070224 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7068034a10 con 0x7f7058004d00 2023-10-08T07:10:25.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.861+0000 7f707764b640 1 -- 172.21.15.70:0/848070224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 msgr2=0x7f7058005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.861+0000 7f707764b640 1 --2- 172.21.15.70:0/848070224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 0x7f7058005180 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f7068002230 tx=0x7f70580057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.861+0000 7f7075be8640 1 -- 172.21.15.70:0/848070224 reap_dead start 2023-10-08T07:10:25.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.861+0000 7f707764b640 1 -- 172.21.15.70:0/848070224 shutdown_connections 2023-10-08T07:10:25.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.861+0000 7f707764b640 1 -- 172.21.15.70:0/848070224 >> 172.21.15.70:0/848070224 conn(0x7f707006b5d0 msgr2=0x7f707006b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:25.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.861+0000 7f707764b640 1 -- 172.21.15.70:0/848070224 shutdown_connections 2023-10-08T07:10:25.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.861+0000 7f707764b640 1 -- 172.21.15.70:0/848070224 wait complete. 2023-10-08T07:10:25.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.863+0000 7f707764b640 1 Processor -- start 2023-10-08T07:10:25.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.863+0000 7f707764b640 1 -- start start 2023-10-08T07:10:25.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.864+0000 7f707764b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 0x7f70700cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.864+0000 7f707764b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7070158e40 0x7f70701e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.864+0000 7f707764b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70701e5810 0x7f70701e6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:25.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.864+0000 7f707764b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70701e5c90 con 0x7f7058004d00 2023-10-08T07:10:25.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.864+0000 7f707764b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70700cb5a0 con 0x7f70701e5810 2023-10-08T07:10:25.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.864+0000 7f707764b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70700cb880 con 0x7f7070158e40 2023-10-08T07:10:25.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.864+0000 7f70753e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 0x7f70700cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.864+0000 7f70753e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 0x7f70700cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54110/0 (socket says 172.21.15.70:54110) 2023-10-08T07:10:25.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.864+0000 7f70753e7640 1 -- 172.21.15.70:0/2825878175 learned_addr learned my addr 172.21.15.70:0/2825878175 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:25.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f70753e7640 1 -- 172.21.15.70:0/2825878175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7070158e40 msgr2=0x7f70701e52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f7074be6640 1 --2- 172.21.15.70:0/2825878175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7070158e40 0x7f70701e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f7075be8640 1 --2- 172.21.15.70:0/2825878175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70701e5810 0x7f70701e6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f70753e7640 1 --2- 172.21.15.70:0/2825878175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7070158e40 0x7f70701e52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f70753e7640 1 -- 172.21.15.70:0/2825878175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70701e5810 msgr2=0x7f70701e6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f70753e7640 1 --2- 172.21.15.70:0/2825878175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70701e5810 0x7f70701e6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f70753e7640 1 -- 172.21.15.70:0/2825878175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70700cbb60 con 0x7f7058004d00 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f7074be6640 1 --2- 172.21.15.70:0/2825878175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7070158e40 0x7f70701e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f70753e7640 1 --2- 172.21.15.70:0/2825878175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 0x7f70700cc640 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f70600116a0 tx=0x7f7060011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f706dffb640 1 -- 172.21.15.70:0/2825878175 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706000f9c0 con 0x7f7058004d00 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f706dffb640 1 -- 172.21.15.70:0/2825878175 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706000fb20 con 0x7f7058004d00 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f706dffb640 1 -- 172.21.15.70:0/2825878175 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706000fed0 con 0x7f7058004d00 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f707764b640 1 -- 172.21.15.70:0/2825878175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70700cbe40 con 0x7f7058004d00 2023-10-08T07:10:25.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.865+0000 7f707764b640 1 -- 172.21.15.70:0/2825878175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70701592c0 con 0x7f7058004d00 2023-10-08T07:10:25.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.867+0000 7f707764b640 1 -- 172.21.15.70:0/2825878175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70700ced60 con 0x7f7058004d00 2023-10-08T07:10:25.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.867+0000 7f706dffb640 1 -- 172.21.15.70:0/2825878175 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 113) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f7060015460 con 0x7f7058004d00 2023-10-08T07:10:25.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.869+0000 7f706dffb640 1 -- 172.21.15.70:0/2825878175 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f7060089b60 con 0x7f7058004d00 2023-10-08T07:10:25.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:25.873+0000 7f706dffb640 1 -- 172.21.15.70:0/2825878175 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7060054a80 con 0x7f7058004d00 2023-10-08T07:10:26.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.064+0000 7f707764b640 1 -- 172.21.15.70:0/2825878175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f707004e160 con 0x7f7058004d00 2023-10-08T07:10:26.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.065+0000 7f706dffb640 1 -- 172.21.15.70:0/2825878175 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v203) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70600580a0 con 0x7f7058004d00 2023-10-08T07:10:26.066 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:10:26.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.067+0000 7f707764b640 1 -- 172.21.15.70:0/2825878175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 msgr2=0x7f70700cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.067+0000 7f707764b640 1 --2- 172.21.15.70:0/2825878175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 0x7f70700cc640 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f70600116a0 tx=0x7f7060011b60 comp rx=0 tx=0).stop 2023-10-08T07:10:26.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.068+0000 7f707764b640 1 -- 172.21.15.70:0/2825878175 shutdown_connections 2023-10-08T07:10:26.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.068+0000 7f707764b640 1 --2- 172.21.15.70:0/2825878175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70701e5810 0x7f70701e6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.068+0000 7f707764b640 1 --2- 172.21.15.70:0/2825878175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7070158e40 0x7f70701e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.068+0000 7f707764b640 1 --2- 172.21.15.70:0/2825878175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7058004d00 0x7f70700cc640 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.068+0000 7f707764b640 1 -- 172.21.15.70:0/2825878175 >> 172.21.15.70:0/2825878175 conn(0x7f707006b5d0 msgr2=0x7f70700b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:26.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.068+0000 7f707764b640 1 -- 172.21.15.70:0/2825878175 shutdown_connections 2023-10-08T07:10:26.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.068+0000 7f707764b640 1 -- 172.21.15.70:0/2825878175 wait complete. 2023-10-08T07:10:26.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:10:26.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.300+0000 7f16c6256640 1 Processor -- start 2023-10-08T07:10:26.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.300+0000 7f16c6256640 1 -- start start 2023-10-08T07:10:26.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.302+0000 7f16c6256640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c00c9cf0 0x7f16c006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.302+0000 7f16c6256640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16c015aa00 0x7f16c00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.302+0000 7f16c6256640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c015b310 con 0x7f16c006c3a0 2023-10-08T07:10:26.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.302+0000 7f16c6256640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c006e0a0 con 0x7f16c015aa00 2023-10-08T07:10:26.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.302+0000 7f16c6256640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c006d290 con 0x7f16c00c9cf0 2023-10-08T07:10:26.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.302+0000 7f16bffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16c015aa00 0x7f16c00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:26.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.302+0000 7f16bffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16c015aa00 0x7f16c00b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54114/0 (socket says 172.21.15.70:54114) 2023-10-08T07:10:26.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.302+0000 7f16bffff640 1 -- 172.21.15.70:0/1931397627 learned_addr learned my addr 172.21.15.70:0/1931397627 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:26.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.303+0000 7f16beffd640 1 --2- 172.21.15.70:0/1931397627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c00c9cf0 0x7f16c006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:26.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.303+0000 7f16bffff640 1 -- 172.21.15.70:0/1931397627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c00c9cf0 msgr2=0x7f16c006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:26.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.303+0000 7f16be7fc640 1 -- 172.21.15.70:0/1931397627 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069057517 0 0) 0x7f16b0001110 con 0x7f16c006c3a0 2023-10-08T07:10:26.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.303+0000 7f16bffff640 1 --2- 172.21.15.70:0/1931397627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c00c9cf0 0x7f16c006db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.303+0000 7f16bffff640 1 -- 172.21.15.70:0/1931397627 >> v1:172.21.15.70:6789/0 conn(0x7f16c006c3a0 legacy=0x7f16c006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.303+0000 7f16bffff640 1 -- 172.21.15.70:0/1931397627 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00c9860 con 0x7f16c015aa00 2023-10-08T07:10:26.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.303+0000 7f16beffd640 1 --2- 172.21.15.70:0/1931397627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c00c9cf0 0x7f16c006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:26.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.304+0000 7f16bffff640 1 --2- 172.21.15.70:0/1931397627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015aa00 0x7f16c00b2a90 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009e50 tx=0x7f16b802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d31a778d931dc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:26.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.304+0000 7f16be7fc640 1 -- 172.21.15.70:0/1931397627 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b802cb00 con 0x7f16c015aa00 2023-10-08T07:10:26.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.304+0000 7f16be7fc640 1 -- 172.21.15.70:0/1931397627 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b802cc60 con 0x7f16c015aa00 2023-10-08T07:10:26.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.304+0000 7f16be7fc640 1 -- 172.21.15.70:0/1931397627 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b80343f0 con 0x7f16c015aa00 2023-10-08T07:10:26.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.304+0000 7f16c6256640 1 -- 172.21.15.70:0/1931397627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015aa00 msgr2=0x7f16c00b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.304+0000 7f16c6256640 1 --2- 172.21.15.70:0/1931397627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015aa00 0x7f16c00b2a90 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f16b8009e50 tx=0x7f16b802bf60 comp rx=0 tx=0).stop 2023-10-08T07:10:26.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.305+0000 7f16c6256640 1 -- 172.21.15.70:0/1931397627 shutdown_connections 2023-10-08T07:10:26.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.305+0000 7f16c6256640 1 --2- 172.21.15.70:0/1931397627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c015aa00 0x7f16c00b2a90 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.305+0000 7f16c6256640 1 --2- 172.21.15.70:0/1931397627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c00c9cf0 0x7f16c006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.305+0000 7f16c6256640 1 -- 172.21.15.70:0/1931397627 >> 172.21.15.70:0/1931397627 conn(0x7f16c006a5d0 msgr2=0x7f16c00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:26.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.305+0000 7f16c6256640 1 -- 172.21.15.70:0/1931397627 shutdown_connections 2023-10-08T07:10:26.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.305+0000 7f16c6256640 1 -- 172.21.15.70:0/1931397627 wait complete. 2023-10-08T07:10:26.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.306+0000 7f16c6256640 1 Processor -- start 2023-10-08T07:10:26.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.306+0000 7f16c6256640 1 -- start start 2023-10-08T07:10:26.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.307+0000 7f16c6256640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c006c3a0 0x7f16c014fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.307+0000 7f16c6256640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00c9cf0 0x7f16c00ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.307+0000 7f16c6256640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c015aa00 0x7f16c00ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.307+0000 7f16c6256640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00c7500 con 0x7f16c00c9cf0 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.307+0000 7f16c6256640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00c77e0 con 0x7f16c006c3a0 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.307+0000 7f16c6256640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00c7ac0 con 0x7f16c015aa00 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.308+0000 7f16beffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00c9cf0 0x7f16c00ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.308+0000 7f16beffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00c9cf0 0x7f16c00ca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54128/0 (socket says 172.21.15.70:54128) 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.308+0000 7f16beffd640 1 -- 172.21.15.70:0/1460298308 learned_addr learned my addr 172.21.15.70:0/1460298308 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.308+0000 7f16beffd640 1 -- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c015aa00 msgr2=0x7f16c00ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.308+0000 7f16beffd640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c015aa00 0x7f16c00ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.308+0000 7f16beffd640 1 -- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c006c3a0 msgr2=0x7f16c014fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.308+0000 7f16beffd640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c006c3a0 0x7f16c014fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.308+0000 7f16beffd640 1 -- 172.21.15.70:0/1460298308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8009b30 con 0x7f16c00c9cf0 2023-10-08T07:10:26.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.308+0000 7f16beffd640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00c9cf0 0x7f16c00ca400 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f16b400c7f0 tx=0x7f16b400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:26.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.309+0000 7f16a7fff640 1 -- 172.21.15.70:0/1460298308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4007a30 con 0x7f16c00c9cf0 2023-10-08T07:10:26.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.309+0000 7f16a7fff640 1 -- 172.21.15.70:0/1460298308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b4007b90 con 0x7f16c00c9cf0 2023-10-08T07:10:26.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.309+0000 7f16a7fff640 1 -- 172.21.15.70:0/1460298308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b400ce40 con 0x7f16c00c9cf0 2023-10-08T07:10:26.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.309+0000 7f16c6256640 1 -- 172.21.15.70:0/1460298308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c00c7da0 con 0x7f16c00c9cf0 2023-10-08T07:10:26.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.309+0000 7f16c6256640 1 -- 172.21.15.70:0/1460298308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c00c6860 con 0x7f16c00c9cf0 2023-10-08T07:10:26.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.310+0000 7f16c6256640 1 -- 172.21.15.70:0/1460298308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c00667d0 con 0x7f16c00c9cf0 2023-10-08T07:10:26.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.311+0000 7f16a7fff640 1 -- 172.21.15.70:0/1460298308 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 113) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f16b40113f0 con 0x7f16c00c9cf0 2023-10-08T07:10:26.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.312+0000 7f16a7fff640 1 -- 172.21.15.70:0/1460298308 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f16b4089890 con 0x7f16c00c9cf0 2023-10-08T07:10:26.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.316+0000 7f16a7fff640 1 -- 172.21.15.70:0/1460298308 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f16b40547b0 con 0x7f16c00c9cf0 2023-10-08T07:10:26.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.492+0000 7f16a7fff640 1 -- 172.21.15.70:0/1460298308 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 114) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f16b4054260 con 0x7f16c00c9cf0 2023-10-08T07:10:26.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.492+0000 7f16a7fff640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f16ac070fa0 0x7f16ac073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.494+0000 7f16bf7fe640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f16ac070fa0 0x7f16ac073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:26.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.510+0000 7f16bf7fe640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f16ac070fa0 0x7f16ac073390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f16b0007fd0 tx=0x7f16b00077c0 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:26.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.512+0000 7f16c6256640 1 -- 172.21.15.70:0/1460298308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16c00c7640 con 0x7f16c00c9cf0 2023-10-08T07:10:26.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.513+0000 7f16a7fff640 1 -- 172.21.15.70:0/1460298308 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v203) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16c00c7640 con 0x7f16c00c9cf0 2023-10-08T07:10:26.515 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:10:26.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 -- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f16ac070fa0 msgr2=0x7f16ac073390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f16ac070fa0 0x7f16ac073390 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f16b0007fd0 tx=0x7f16b00077c0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 -- 172.21.15.70:0/1460298308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00c9cf0 msgr2=0x7f16c00ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00c9cf0 0x7f16c00ca400 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f16b400c7f0 tx=0x7f16b400cbf0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 -- 172.21.15.70:0/1460298308 shutdown_connections 2023-10-08T07:10:26.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f16ac070fa0 0x7f16ac073390 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f16c015aa00 0x7f16c00ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16c00c9cf0 0x7f16c00ca400 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 --2- 172.21.15.70:0/1460298308 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f16c006c3a0 0x7f16c014fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 -- 172.21.15.70:0/1460298308 >> 172.21.15.70:0/1460298308 conn(0x7f16c006a5d0 msgr2=0x7f16c015e010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:26.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 -- 172.21.15.70:0/1460298308 shutdown_connections 2023-10-08T07:10:26.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.516+0000 7f16c6256640 1 -- 172.21.15.70:0/1460298308 wait complete. 2023-10-08T07:10:26.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:10:26.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.743+0000 7f8f61856640 1 Processor -- start 2023-10-08T07:10:26.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.744+0000 7f8f61856640 1 -- start start 2023-10-08T07:10:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.745+0000 7f8f61856640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15ab40 0x7f8f5c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.746+0000 7f8f61856640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5c15b2b0 con 0x7f8f5c0cb580 2023-10-08T07:10:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.746+0000 7f8f61856640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5c15b590 con 0x7f8f5c0c90b0 2023-10-08T07:10:26.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.746+0000 7f8f61856640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c15bf30 con 0x7f8f5c15ab40 2023-10-08T07:10:26.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.746+0000 7f8f5b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8f5c0cb580 0x7f8f5c15e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53472/0 (socket says 172.21.15.70:53472) 2023-10-08T07:10:26.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.746+0000 7f8f5b7fe640 1 -- 172.21.15.70:0/4145224264 learned_addr learned my addr 172.21.15.70:0/4145224264 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:26.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.746+0000 7f8f5affd640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15ab40 0x7f8f5c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.746+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16371446 0 0) 0x7f8f5c15b2b0 con 0x7f8f5c0cb580 2023-10-08T07:10:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.746+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f48003520 con 0x7f8f5c0cb580 2023-10-08T07:10:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2561801710 0 0) 0x7f8f48003520 con 0x7f8f5c0cb580 2023-10-08T07:10:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15ab40 msgr2=0x7f8f5c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15ab40 0x7f8f5c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 >> v1:172.21.15.181:6789/0 conn(0x7f8f5c0c90b0 legacy=0x7f8f5c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c15b870 con 0x7f8f5c0cb580 2023-10-08T07:10:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f8f540019f0 con 0x7f8f5c0cb580 2023-10-08T07:10:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 >> v1:172.21.15.70:6789/0 conn(0x7f8f5c0cb580 legacy=0x7f8f5c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f480039a0 0x7f8f48003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15ab40 0x7f8f480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f48004d00 0x7f8f48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c15bf30 con 0x7f8f480039a0 2023-10-08T07:10:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4800ba30 con 0x7f8f48004d00 2023-10-08T07:10:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4800bbf0 con 0x7f8f5c15ab40 2023-10-08T07:10:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.747+0000 7f8f5affd640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f480039a0 0x7f8f48003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.748+0000 7f8f53fff640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15ab40 0x7f8f480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.748+0000 7f8f5affd640 1 -- 172.21.15.70:0/4145224264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15ab40 msgr2=0x7f8f480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.748+0000 7f8f5affd640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f5c15ab40 0x7f8f480047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.748+0000 7f8f5affd640 1 -- 172.21.15.70:0/4145224264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f48004d00 msgr2=0x7f8f48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:26.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.748+0000 7f8f5affd640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f48004d00 0x7f8f48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.748+0000 7f8f5affd640 1 -- 172.21.15.70:0/4145224264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f48003520 con 0x7f8f480039a0 2023-10-08T07:10:26.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.749+0000 7f8f5affd640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f480039a0 0x7f8f48003dc0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009a90 tx=0x7f8f480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b68e9061b920d26 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:26.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.749+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c02c680 con 0x7f8f480039a0 2023-10-08T07:10:26.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.749+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4c02c7e0 con 0x7f8f480039a0 2023-10-08T07:10:26.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.749+0000 7f8f5a7fc640 1 -- 172.21.15.70:0/4145224264 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c02cb70 con 0x7f8f480039a0 2023-10-08T07:10:26.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.749+0000 7f8f61856640 1 -- 172.21.15.70:0/4145224264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f480039a0 msgr2=0x7f8f48003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.749+0000 7f8f61856640 1 --2- 172.21.15.70:0/4145224264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f480039a0 0x7f8f48003dc0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009a90 tx=0x7f8f480057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.749+0000 7f8f5b7fe640 1 -- 172.21.15.70:0/4145224264 reap_dead start 2023-10-08T07:10:26.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.750+0000 7f8f61856640 1 -- 172.21.15.70:0/4145224264 shutdown_connections 2023-10-08T07:10:26.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.750+0000 7f8f61856640 1 -- 172.21.15.70:0/4145224264 >> 172.21.15.70:0/4145224264 conn(0x7f8f5c06a380 msgr2=0x7f8f5c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:26.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.750+0000 7f8f61856640 1 -- 172.21.15.70:0/4145224264 shutdown_connections 2023-10-08T07:10:26.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.750+0000 7f8f61856640 1 -- 172.21.15.70:0/4145224264 wait complete. 2023-10-08T07:10:26.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.751+0000 7f8f61856640 1 Processor -- start 2023-10-08T07:10:26.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.751+0000 7f8f61856640 1 -- start start 2023-10-08T07:10:26.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.752+0000 7f8f61856640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f480039a0 0x7f8f5c0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.752+0000 7f8f61856640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f5c0c84e0 0x7f8f5c1f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f61856640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5c0aca40 0x7f8f5c0aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f61856640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0ad410 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f61856640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0b1920 con 0x7f8f5c0aca40 2023-10-08T07:10:26.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f61856640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0b1c00 con 0x7f8f480039a0 2023-10-08T07:10:26.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f5c0c84e0 0x7f8f5c1f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:26.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f5c0c84e0 0x7f8f5c1f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54156/0 (socket says 172.21.15.70:54156) 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f53fff640 1 -- 172.21.15.70:0/1176752721 learned_addr learned my addr 172.21.15.70:0/1176752721 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f53fff640 1 -- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f480039a0 msgr2=0x7f8f5c0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f53fff640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f480039a0 0x7f8f5c0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f53fff640 1 -- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5c0aca40 msgr2=0x7f8f5c0aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f53fff640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5c0aca40 0x7f8f5c0aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f53fff640 1 -- 172.21.15.70:0/1176752721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c0b1ee0 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.753+0000 7f8f53fff640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f5c0c84e0 0x7f8f5c1f6560 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400d570 tx=0x7f8f4400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.754+0000 7f8f58ff9640 1 -- 172.21.15.70:0/1176752721 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44005480 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.754+0000 7f8f58ff9640 1 -- 172.21.15.70:0/1176752721 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f44003610 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.754+0000 7f8f61856640 1 -- 172.21.15.70:0/1176752721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5c0b21c0 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.754+0000 7f8f58ff9640 1 -- 172.21.15.70:0/1176752721 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44003980 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.754+0000 7f8f61856640 1 -- 172.21.15.70:0/1176752721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5c0c5fc0 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.755+0000 7f8f61856640 1 -- 172.21.15.70:0/1176752721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5c067490 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.756+0000 7f8f58ff9640 1 -- 172.21.15.70:0/1176752721 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 114) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f8f44003ae0 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.757+0000 7f8f58ff9640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8f3c06acf0 0x7f8f3c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:26.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.757+0000 7f8f58ff9640 1 -- 172.21.15.70:0/1176752721 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f8f44089810 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.758+0000 7f8f5affd640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8f3c06acf0 0x7f8f3c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:26.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.759+0000 7f8f5affd640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8f3c06acf0 0x7f8f3c06d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c036000 tx=0x7f8f4c02e000 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:26.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.762+0000 7f8f58ff9640 1 -- 172.21.15.70:0/1176752721 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f44054550 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.987+0000 7f8f61856640 1 -- 172.21.15.70:0/1176752721 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f5c0c6150 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.988+0000 7f8f58ff9640 1 -- 172.21.15.70:0/1176752721 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f8f44001230 con 0x7f8f5c0c84e0 2023-10-08T07:10:26.990 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:26.990 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/381523318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":381523318},{"type":"v1","addr":"172.21.15.181:6835","nonce":381523318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2153071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2153071509},{"type":"v1","addr":"172.21.15.181:6837","nonce":2153071509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2673497542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2673497542},{"type":"v1","addr":"172.21.15.70:6835","nonce":2673497542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:54:32.491331+0000","modified":"2023-10-08T06:54:35.534957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10074":{"gid":10074,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2138719113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2138719113},{"type":"v1","addr":"172.21.15.70:6833","nonce":2138719113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-08T07:10:26.990 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-10-08T07:10:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 -- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8f3c06acf0 msgr2=0x7f8f3c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8f3c06acf0 0x7f8f3c06d1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c036000 tx=0x7f8f4c02e000 comp rx=0 tx=0).stop 2023-10-08T07:10:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 -- 172.21.15.70:0/1176752721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f5c0c84e0 msgr2=0x7f8f5c1f6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f5c0c84e0 0x7f8f5c1f6560 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400d570 tx=0x7f8f4400da30 comp rx=0 tx=0).stop 2023-10-08T07:10:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 -- 172.21.15.70:0/1176752721 shutdown_connections 2023-10-08T07:10:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8f3c06acf0 0x7f8f3c06d1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8f5c0aca40 0x7f8f5c0aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f5c0c84e0 0x7f8f5c1f6560 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 --2- 172.21.15.70:0/1176752721 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8f480039a0 0x7f8f5c0c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:26.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 -- 172.21.15.70:0/1176752721 >> 172.21.15.70:0/1176752721 conn(0x7f8f5c06a380 msgr2=0x7f8f5c0ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:26.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 -- 172.21.15.70:0/1176752721 shutdown_connections 2023-10-08T07:10:26.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:26.991+0000 7f8f61856640 1 -- 172.21.15.70:0/1176752721 wait complete. 2023-10-08T07:10:27.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:10:27.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.216+0000 7fd4c073c640 1 Processor -- start 2023-10-08T07:10:27.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.216+0000 7fd4c073c640 1 -- start start 2023-10-08T07:10:27.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.218+0000 7fd4c073c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd4b8157000 0x7fd4b80cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.218+0000 7fd4c073c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b80adfc0 con 0x7fd4b80cb190 2023-10-08T07:10:27.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.218+0000 7fd4c073c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b80ccce0 con 0x7fd4b81597d0 2023-10-08T07:10:27.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.218+0000 7fd4c073c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b8159cb0 con 0x7fd4b8157000 2023-10-08T07:10:27.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.218+0000 7fd4be4d8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd4b8157000 0x7fd4b80cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.218+0000 7fd4be4d8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd4b8157000 0x7fd4b80cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54160/0 (socket says 172.21.15.70:54160) 2023-10-08T07:10:27.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.218+0000 7fd4be4d8640 1 -- 172.21.15.70:0/1305591405 learned_addr learned my addr 172.21.15.70:0/1305591405 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:27.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.219+0000 7fd4be4d8640 1 -- 172.21.15.70:0/1305591405 >> v1:172.21.15.181:6789/0 conn(0x7fd4b81597d0 legacy=0x7fd4b806ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:27.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.219+0000 7fd4be4d8640 1 -- 172.21.15.70:0/1305591405 >> v1:172.21.15.70:6789/0 conn(0x7fd4b80cb190 legacy=0x7fd4b815d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.219+0000 7fd4be4d8640 1 -- 172.21.15.70:0/1305591405 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b8159f90 con 0x7fd4b8157000 2023-10-08T07:10:27.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.220+0000 7fd4be4d8640 1 --2- 172.21.15.70:0/1305591405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b8157000 0x7fd4b80cc7a0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400aa10 tx=0x7fd4b402ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65495af950c3671d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:27.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.220+0000 7fd4bd4d6640 1 -- 172.21.15.70:0/1305591405 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b402dc10 con 0x7fd4b8157000 2023-10-08T07:10:27.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.220+0000 7fd4bd4d6640 1 -- 172.21.15.70:0/1305591405 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b402dd70 con 0x7fd4b8157000 2023-10-08T07:10:27.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.220+0000 7fd4bd4d6640 1 -- 172.21.15.70:0/1305591405 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4035660 con 0x7fd4b8157000 2023-10-08T07:10:27.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.221+0000 7fd4c073c640 1 -- 172.21.15.70:0/1305591405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b8157000 msgr2=0x7fd4b80cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.221+0000 7fd4c073c640 1 --2- 172.21.15.70:0/1305591405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b8157000 0x7fd4b80cc7a0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400aa10 tx=0x7fd4b402ce60 comp rx=0 tx=0).stop 2023-10-08T07:10:27.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.221+0000 7fd4c073c640 1 -- 172.21.15.70:0/1305591405 shutdown_connections 2023-10-08T07:10:27.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.221+0000 7fd4c073c640 1 --2- 172.21.15.70:0/1305591405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b8157000 0x7fd4b80cc7a0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.221+0000 7fd4c073c640 1 -- 172.21.15.70:0/1305591405 >> 172.21.15.70:0/1305591405 conn(0x7fd4b806a380 msgr2=0x7fd4b80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:27.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.221+0000 7fd4c073c640 1 -- 172.21.15.70:0/1305591405 shutdown_connections 2023-10-08T07:10:27.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.221+0000 7fd4c073c640 1 -- 172.21.15.70:0/1305591405 wait complete. 2023-10-08T07:10:27.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.223+0000 7fd4c073c640 1 Processor -- start 2023-10-08T07:10:27.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.223+0000 7fd4c073c640 1 -- start start 2023-10-08T07:10:27.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.224+0000 7fd4c073c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b80cb190 0x7fd4b81e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.224+0000 7fd4c073c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b81597d0 0x7fd4b814aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.224+0000 7fd4c073c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b80af4a0 0x7fd4b80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.224+0000 7fd4c073c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b80afe90 con 0x7fd4b81597d0 2023-10-08T07:10:27.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.224+0000 7fd4c073c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b815a8c0 con 0x7fd4b80cb190 2023-10-08T07:10:27.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.224+0000 7fd4c073c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b8152ae0 con 0x7fd4b80af4a0 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.224+0000 7fd4bdcd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b81597d0 0x7fd4b814aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.224+0000 7fd4bdcd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b81597d0 0x7fd4b814aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54164/0 (socket says 172.21.15.70:54164) 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.224+0000 7fd4bdcd7640 1 -- 172.21.15.70:0/3691267710 learned_addr learned my addr 172.21.15.70:0/3691267710 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4bdcd7640 1 -- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b80af4a0 msgr2=0x7fd4b80af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4bdcd7640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b80af4a0 0x7fd4b80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4be4d8640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b80cb190 0x7fd4b81e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4bdcd7640 1 -- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b80cb190 msgr2=0x7fd4b81e4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4bdcd7640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b80cb190 0x7fd4b81e4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4bdcd7640 1 -- 172.21.15.70:0/3691267710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b400a670 con 0x7fd4b81597d0 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4bdcd7640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b81597d0 0x7fd4b814aeb0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b420 tx=0x7fd4a800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:27.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4aeffd640 1 -- 172.21.15.70:0/3691267710 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8011830 con 0x7fd4b81597d0 2023-10-08T07:10:27.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4aeffd640 1 -- 172.21.15.70:0/3691267710 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a8011990 con 0x7fd4b81597d0 2023-10-08T07:10:27.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4aeffd640 1 -- 172.21.15.70:0/3691267710 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8011d40 con 0x7fd4b81597d0 2023-10-08T07:10:27.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.225+0000 7fd4c073c640 1 -- 172.21.15.70:0/3691267710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b815ab10 con 0x7fd4b81597d0 2023-10-08T07:10:27.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.226+0000 7fd4c073c640 1 -- 172.21.15.70:0/3691267710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b815b0d0 con 0x7fd4b81597d0 2023-10-08T07:10:27.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.227+0000 7fd4c073c640 1 -- 172.21.15.70:0/3691267710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b8067490 con 0x7fd4b81597d0 2023-10-08T07:10:27.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.228+0000 7fd4aeffd640 1 -- 172.21.15.70:0/3691267710 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 114) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fd4a801b460 con 0x7fd4b81597d0 2023-10-08T07:10:27.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.228+0000 7fd4aeffd640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fd49c06acc0 0x7fd49c06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.229+0000 7fd4aeffd640 1 -- 172.21.15.70:0/3691267710 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7fd4a80893e0 con 0x7fd4b81597d0 2023-10-08T07:10:27.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.229+0000 7fd4be4d8640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fd49c06acc0 0x7fd49c06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.230+0000 7fd4be4d8640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fd49c06acc0 0x7fd49c06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4037000 tx=0x7fd4b4008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:27.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.233+0000 7fd4aeffd640 1 -- 172.21.15.70:0/3691267710 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4a80541b0 con 0x7fd4b81597d0 2023-10-08T07:10:27.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.421+0000 7fd4c073c640 1 -- 172.21.15.70:0/3691267710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4b80c4b00 con 0x7fd4b81597d0 2023-10-08T07:10:27.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.425+0000 7fd4aeffd640 1 -- 172.21.15.70:0/3691267710 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+18478 (secure 0 0 0) 0x7fd4a80577d0 con 0x7fd4b81597d0 2023-10-08T07:10:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:27.427 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":203,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:10:25.468736+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:54:30.246375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-08T06:54:31.487900+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"202","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":202,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6832/2291259069":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/2956479135":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/4233293606":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/2946222544":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/1250722759":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/1792732560":"2023-10-09T07:10:25.468616+0000","172.21.15.181:6833/3979056493":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6832/3979056493":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/340227645":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/849801254":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/4017559811":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/217631810":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/64410277":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/811429527":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/93223744":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/772181101":"2023-10-09T06:54:47.949157+0000","172.21.15.181:6833/2291259069":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/1286852011":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6832/3709692999":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/4147168289":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/437133264":"2023-10-09T07:10:25.468616+0000","172.21.15.181:6833/3709692999":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/602716182":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/1777884673":"2023-10-09T06:54:47.949157+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:10:27.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 -- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fd49c06acc0 msgr2=0x7fd49c06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fd49c06acc0 0x7fd49c06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4b4037000 tx=0x7fd4b4008040 comp rx=0 tx=0).stop 2023-10-08T07:10:27.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 -- 172.21.15.70:0/3691267710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b81597d0 msgr2=0x7fd4b814aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b81597d0 0x7fd4b814aeb0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b420 tx=0x7fd4a800b8e0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 -- 172.21.15.70:0/3691267710 shutdown_connections 2023-10-08T07:10:27.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fd49c06acc0 0x7fd49c06d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd4b80af4a0 0x7fd4b80af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b81597d0 0x7fd4b814aeb0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 --2- 172.21.15.70:0/3691267710 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd4b80cb190 0x7fd4b81e4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 -- 172.21.15.70:0/3691267710 >> 172.21.15.70:0/3691267710 conn(0x7fd4b806a380 msgr2=0x7fd4b80c5020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:27.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 -- 172.21.15.70:0/3691267710 shutdown_connections 2023-10-08T07:10:27.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.427+0000 7fd4c073c640 1 -- 172.21.15.70:0/3691267710 wait complete. 2023-10-08T07:10:27.447 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T07:10:27.448 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:10:27.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.668+0000 7f0debb6c640 1 Processor -- start 2023-10-08T07:10:27.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.668+0000 7f0debb6c640 1 -- start start 2023-10-08T07:10:27.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.669+0000 7f0debb6c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0de415cd90 0x7f0de40702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.670+0000 7f0debb6c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de40c48f0 con 0x7f0de4160940 2023-10-08T07:10:27.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.670+0000 7f0debb6c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de40c3ce0 con 0x7f0de406c890 2023-10-08T07:10:27.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.670+0000 7f0debb6c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de40c4d80 con 0x7f0de415cd90 2023-10-08T07:10:27.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.670+0000 7f0de9107640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0de4160940 0x7f0de4070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53486/0 (socket says 172.21.15.70:53486) 2023-10-08T07:10:27.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.670+0000 7f0de9107640 1 -- 172.21.15.70:0/1256277914 learned_addr learned my addr 172.21.15.70:0/1256277914 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:27.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.670+0000 7f0de9908640 1 --2- 172.21.15.70:0/1256277914 >> v2:172.21.15.181:3300/0 conn(0x7f0de415cd90 0x7f0de40702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.670+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437075063 0 0) 0x7f0de40c48f0 con 0x7f0de4160940 2023-10-08T07:10:27.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd0003520 con 0x7f0de4160940 2023-10-08T07:10:27.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4171914483 0 0) 0x7f0dd0003520 con 0x7f0de4160940 2023-10-08T07:10:27.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 >> v2:172.21.15.181:3300/0 conn(0x7f0de415cd90 msgr2=0x7f0de40702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 --2- 172.21.15.70:0/1256277914 >> v2:172.21.15.181:3300/0 conn(0x7f0de415cd90 0x7f0de40702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 >> v1:172.21.15.181:6789/0 conn(0x7f0de406c890 legacy=0x7f0de406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de40c3fc0 con 0x7f0de4160940 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0dd8001670 con 0x7f0de4160940 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 >> v1:172.21.15.70:6789/0 conn(0x7f0de4160940 legacy=0x7f0de4070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd00039c0 0x7f0dd0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dd00047c0 0x7f0dd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dd0005180 0x7f0dd000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd000c050 con 0x7f0dd0005180 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd000c1c0 con 0x7f0dd00039c0 2023-10-08T07:10:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd000c380 con 0x7f0dd00047c0 2023-10-08T07:10:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.671+0000 7f0dea109640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dd0005180 0x7f0dd000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.672+0000 7f0de9908640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd00039c0 0x7f0dd0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.672+0000 7f0de9107640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dd00047c0 0x7f0dd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.672+0000 7f0dea109640 1 -- 172.21.15.70:0/1256277914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dd00047c0 msgr2=0x7f0dd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.672+0000 7f0dea109640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dd00047c0 0x7f0dd0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.672+0000 7f0dea109640 1 -- 172.21.15.70:0/1256277914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd00039c0 msgr2=0x7f0dd0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.672+0000 7f0dea109640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd00039c0 0x7f0dd0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.672+0000 7f0dea109640 1 -- 172.21.15.70:0/1256277914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd0003520 con 0x7f0dd0005180 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.673+0000 7f0dea109640 1 -- 172.21.15.70:0/1256277914 reap_dead start 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.673+0000 7f0dea109640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dd0005180 0x7f0dd000b980 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc001e50 tx=0x7f0dd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d2cf15ecea7640 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.673+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc003b70 con 0x7f0dd0005180 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.673+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc003d10 con 0x7f0dd0005180 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.673+0000 7f0de8906640 1 -- 172.21.15.70:0/1256277914 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc005310 con 0x7f0dd0005180 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.674+0000 7f0debb6c640 1 -- 172.21.15.70:0/1256277914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dd0005180 msgr2=0x7f0dd000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.674+0000 7f0debb6c640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dd0005180 0x7f0dd000b980 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc001e50 tx=0x7f0dd00057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.674+0000 7f0debb6c640 1 -- 172.21.15.70:0/1256277914 shutdown_connections 2023-10-08T07:10:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.674+0000 7f0debb6c640 1 --2- 172.21.15.70:0/1256277914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dd0005180 0x7f0dd000b980 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.674+0000 7f0debb6c640 1 -- 172.21.15.70:0/1256277914 >> 172.21.15.70:0/1256277914 conn(0x7f0de406a380 msgr2=0x7f0de40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.674+0000 7f0debb6c640 1 -- 172.21.15.70:0/1256277914 shutdown_connections 2023-10-08T07:10:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.674+0000 7f0debb6c640 1 -- 172.21.15.70:0/1256277914 wait complete. 2023-10-08T07:10:27.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.676+0000 7f0debb6c640 1 Processor -- start 2023-10-08T07:10:27.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.676+0000 7f0debb6c640 1 -- start start 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.677+0000 7f0debb6c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd0005180 0x7f0de40c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.677+0000 7f0debb6c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0de4156130 0x7f0de40b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.677+0000 7f0debb6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0de40b2ee0 0x7f0de415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.677+0000 7f0debb6c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de40c8270 con 0x7f0de40b2ee0 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.677+0000 7f0debb6c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de40b4320 con 0x7f0dd0005180 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.677+0000 7f0debb6c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de40b3f60 con 0x7f0de4156130 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.677+0000 7f0dea109640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0de40b2ee0 0x7f0de415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.677+0000 7f0dea109640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0de40b2ee0 0x7f0de415d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54176/0 (socket says 172.21.15.70:54176) 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.677+0000 7f0dea109640 1 -- 172.21.15.70:0/440151223 learned_addr learned my addr 172.21.15.70:0/440151223 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0de9107640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0de4156130 0x7f0de40b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0de9908640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd0005180 0x7f0de40c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0dea109640 1 -- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0de4156130 msgr2=0x7f0de40b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0dea109640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0de4156130 0x7f0de40b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0dea109640 1 -- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd0005180 msgr2=0x7f0de40c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0dea109640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd0005180 0x7f0de40c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0dea109640 1 -- 172.21.15.70:0/440151223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de40b2a50 con 0x7f0de40b2ee0 2023-10-08T07:10:27.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0dea109640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0de40b2ee0 0x7f0de415d5f0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc02c060 tx=0x7f0ddc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0dd67fc640 1 -- 172.21.15.70:0/440151223 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc006d50 con 0x7f0de40b2ee0 2023-10-08T07:10:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0dd67fc640 1 -- 172.21.15.70:0/440151223 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc039480 con 0x7f0de40b2ee0 2023-10-08T07:10:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0debb6c640 1 -- 172.21.15.70:0/440151223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de40b1e40 con 0x7f0de40b2ee0 2023-10-08T07:10:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.678+0000 7f0dd67fc640 1 -- 172.21.15.70:0/440151223 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc039810 con 0x7f0de40b2ee0 2023-10-08T07:10:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.679+0000 7f0debb6c640 1 -- 172.21.15.70:0/440151223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de40b2400 con 0x7f0de40b2ee0 2023-10-08T07:10:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.680+0000 7f0debb6c640 1 -- 172.21.15.70:0/440151223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de4067490 con 0x7f0de40b2ee0 2023-10-08T07:10:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.681+0000 7f0dd67fc640 1 -- 172.21.15.70:0/440151223 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 115) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f0ddc039970 con 0x7f0de40b2ee0 2023-10-08T07:10:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.682+0000 7f0dd67fc640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0dc806af30 0x7f0dc806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.682+0000 7f0dd67fc640 1 -- 172.21.15.70:0/440151223 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f0ddc0aba90 con 0x7f0de40b2ee0 2023-10-08T07:10:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.682+0000 7f0de9908640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0dc806af30 0x7f0dc806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.684+0000 7f0de9908640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0dc806af30 0x7f0dc806d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0de41f5a40 tx=0x7f0de0008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:27.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.686+0000 7f0dd67fc640 1 -- 172.21.15.70:0/440151223 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ddc076360 con 0x7f0de40b2ee0 2023-10-08T07:10:27.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.914+0000 7f0debb6c640 1 -- 172.21.15.70:0/440151223 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0de40b0ef0 con 0x7f0de40b2ee0 2023-10-08T07:10:27.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.916+0000 7f0dd67fc640 1 -- 172.21.15.70:0/440151223 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f0ddc047070 con 0x7f0de40b2ee0 2023-10-08T07:10:27.917 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:27.918 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/381523318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":381523318},{"type":"v1","addr":"172.21.15.181:6835","nonce":381523318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2153071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2153071509},{"type":"v1","addr":"172.21.15.181:6837","nonce":2153071509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2673497542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2673497542},{"type":"v1","addr":"172.21.15.70:6835","nonce":2673497542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:54:32.491331+0000","modified":"2023-10-08T06:54:35.534957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10074":{"gid":10074,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2138719113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2138719113},{"type":"v1","addr":"172.21.15.70:6833","nonce":2138719113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-08T07:10:27.918 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-10-08T07:10:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.918+0000 7f0debb6c640 1 -- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0dc806af30 msgr2=0x7f0dc806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.918+0000 7f0debb6c640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0dc806af30 0x7f0dc806d3f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0de41f5a40 tx=0x7f0de0008040 comp rx=0 tx=0).stop 2023-10-08T07:10:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.918+0000 7f0debb6c640 1 -- 172.21.15.70:0/440151223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0de40b2ee0 msgr2=0x7f0de415d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.918+0000 7f0debb6c640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0de40b2ee0 0x7f0de415d5f0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc02c060 tx=0x7f0ddc0014a0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.918+0000 7f0debb6c640 1 -- 172.21.15.70:0/440151223 shutdown_connections 2023-10-08T07:10:27.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.918+0000 7f0debb6c640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0dc806af30 0x7f0dc806d3f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.918+0000 7f0debb6c640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0de40b2ee0 0x7f0de415d5f0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.918+0000 7f0debb6c640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0de4156130 0x7f0de40b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.918+0000 7f0debb6c640 1 --2- 172.21.15.70:0/440151223 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dd0005180 0x7f0de40c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.918+0000 7f0debb6c640 1 -- 172.21.15.70:0/440151223 >> 172.21.15.70:0/440151223 conn(0x7f0de406a380 msgr2=0x7f0de41571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.919+0000 7f0debb6c640 1 -- 172.21.15.70:0/440151223 shutdown_connections 2023-10-08T07:10:27.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:27.919+0000 7f0debb6c640 1 -- 172.21.15.70:0/440151223 wait complete. 2023-10-08T07:10:27.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:10:28.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.135+0000 7f5d6e7e9640 1 Processor -- start 2023-10-08T07:10:28.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.136+0000 7f5d6e7e9640 1 -- start start 2023-10-08T07:10:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.137+0000 7f5d6e7e9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d6815ab30 0x7f5d6815e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.137+0000 7f5d6e7e9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6815f7d0 con 0x7f5d680b07d0 2023-10-08T07:10:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.137+0000 7f5d6e7e9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6815f210 con 0x7f5d680adc70 2023-10-08T07:10:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.137+0000 7f5d6e7e9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6815f4f0 con 0x7f5d6815ab30 2023-10-08T07:10:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.138+0000 7f5d6d7e7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d6815ab30 0x7f5d6815e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.138+0000 7f5d6d7e7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d6815ab30 0x7f5d6815e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54180/0 (socket says 172.21.15.70:54180) 2023-10-08T07:10:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.138+0000 7f5d6d7e7640 1 -- 172.21.15.70:0/1561879974 learned_addr learned my addr 172.21.15.70:0/1561879974 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.138+0000 7f5d6d7e7640 1 -- 172.21.15.70:0/1561879974 >> v1:172.21.15.181:6789/0 conn(0x7f5d680adc70 legacy=0x7f5d680b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.138+0000 7f5d6d7e7640 1 -- 172.21.15.70:0/1561879974 >> v1:172.21.15.70:6789/0 conn(0x7f5d680b07d0 legacy=0x7f5d6806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.138+0000 7f5d6d7e7640 1 -- 172.21.15.70:0/1561879974 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6806c550 con 0x7f5d6815ab30 2023-10-08T07:10:28.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.139+0000 7f5d6d7e7640 1 --2- 172.21.15.70:0/1561879974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 0x7f5d6815e8a0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009870 tx=0x7f5d6402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3002c324caa90c3e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:28.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.139+0000 7f5d5ffff640 1 -- 172.21.15.70:0/1561879974 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d64030850 con 0x7f5d6815ab30 2023-10-08T07:10:28.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.140+0000 7f5d5ffff640 1 -- 172.21.15.70:0/1561879974 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d640309b0 con 0x7f5d6815ab30 2023-10-08T07:10:28.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.140+0000 7f5d5ffff640 1 -- 172.21.15.70:0/1561879974 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d64030d40 con 0x7f5d6815ab30 2023-10-08T07:10:28.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.140+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/1561879974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 msgr2=0x7f5d6815e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.140+0000 7f5d6e7e9640 1 --2- 172.21.15.70:0/1561879974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 0x7f5d6815e8a0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f5d64009870 tx=0x7f5d6402b990 comp rx=0 tx=0).stop 2023-10-08T07:10:28.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.140+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/1561879974 shutdown_connections 2023-10-08T07:10:28.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.140+0000 7f5d6e7e9640 1 --2- 172.21.15.70:0/1561879974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 0x7f5d6815e8a0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.140+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/1561879974 >> 172.21.15.70:0/1561879974 conn(0x7f5d680cb0c0 msgr2=0x7f5d680cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:28.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.140+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/1561879974 shutdown_connections 2023-10-08T07:10:28.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.140+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/1561879974 wait complete. 2023-10-08T07:10:28.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.142+0000 7f5d6e7e9640 1 Processor -- start 2023-10-08T07:10:28.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.142+0000 7f5d6e7e9640 1 -- start start 2023-10-08T07:10:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.143+0000 7f5d6e7e9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d680adc70 0x7f5d68157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.143+0000 7f5d6e7e9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d680b07d0 0x7f5d680c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.143+0000 7f5d6e7e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 0x7f5d680c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.143+0000 7f5d6e7e9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680afec0 con 0x7f5d6815ab30 2023-10-08T07:10:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.143+0000 7f5d6e7e9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681583d0 con 0x7f5d680adc70 2023-10-08T07:10:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.143+0000 7f5d6e7e9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d680c83f0 con 0x7f5d680b07d0 2023-10-08T07:10:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.143+0000 7f5d6dfe8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 0x7f5d680c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.143+0000 7f5d6dfe8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 0x7f5d680c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54188/0 (socket says 172.21.15.70:54188) 2023-10-08T07:10:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.143+0000 7f5d6dfe8640 1 -- 172.21.15.70:0/919524367 learned_addr learned my addr 172.21.15.70:0/919524367 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d6dfe8640 1 -- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d680b07d0 msgr2=0x7f5d680c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d6dfe8640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d680b07d0 0x7f5d680c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d6d7e7640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d680adc70 0x7f5d68157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d6dfe8640 1 -- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d680adc70 msgr2=0x7f5d68157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d6dfe8640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d680adc70 0x7f5d68157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d6dfe8640 1 -- 172.21.15.70:0/919524367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d640094d0 con 0x7f5d6815ab30 2023-10-08T07:10:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d6dfe8640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 0x7f5d680c64d0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c7a0 tx=0x7f5d6000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d5dffb640 1 -- 172.21.15.70:0/919524367 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60003b70 con 0x7f5d6815ab30 2023-10-08T07:10:28.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d5dffb640 1 -- 172.21.15.70:0/919524367 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d60003cf0 con 0x7f5d6815ab30 2023-10-08T07:10:28.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d5dffb640 1 -- 172.21.15.70:0/919524367 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6000aa90 con 0x7f5d6815ab30 2023-10-08T07:10:28.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.144+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/919524367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d680af240 con 0x7f5d6815ab30 2023-10-08T07:10:28.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.145+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/919524367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d680af800 con 0x7f5d6815ab30 2023-10-08T07:10:28.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.146+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/919524367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d680b0000 con 0x7f5d6815ab30 2023-10-08T07:10:28.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.146+0000 7f5d5dffb640 1 -- 172.21.15.70:0/919524367 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 115) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5d6000abf0 con 0x7f5d6815ab30 2023-10-08T07:10:28.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.147+0000 7f5d5dffb640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5d4c06af00 0x7f5d4c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.147+0000 7f5d5dffb640 1 -- 172.21.15.70:0/919524367 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f5d60089c10 con 0x7f5d6815ab30 2023-10-08T07:10:28.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.148+0000 7f5d6d7e7640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5d4c06af00 0x7f5d4c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:28.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.149+0000 7f5d6d7e7640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5d4c06af00 0x7f5d4c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d64036000 tx=0x7f5d6403b040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:28.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.152+0000 7f5d5dffb640 1 -- 172.21.15.70:0/919524367 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d680b0000 con 0x7f5d6815ab30 2023-10-08T07:10:28.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.369+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/919524367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d680b0000 con 0x7f5d6815ab30 2023-10-08T07:10:28.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.370+0000 7f5d5dffb640 1 -- 172.21.15.70:0/919524367 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4923 (secure 0 0 0) 0x7f5d680b0000 con 0x7f5d6815ab30 2023-10-08T07:10:28.372 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:28.372 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/381523318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":381523318},{"type":"v1","addr":"172.21.15.181:6835","nonce":381523318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2153071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2153071509},{"type":"v1","addr":"172.21.15.181:6837","nonce":2153071509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2673497542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2673497542},{"type":"v1","addr":"172.21.15.70:6835","nonce":2673497542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T06:54:32.491331+0000","modified":"2023-10-08T06:54:35.534957+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10074},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10074":{"gid":10074,"name":"a","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6833/2138719113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2138719113},{"type":"v1","addr":"172.21.15.70:6833","nonce":2138719113}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-10-08T07:10:28.372 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-10-08T07:10:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5d4c06af00 msgr2=0x7f5d4c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5d4c06af00 0x7f5d4c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5d64036000 tx=0x7f5d6403b040 comp rx=0 tx=0).stop 2023-10-08T07:10:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/919524367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 msgr2=0x7f5d680c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 0x7f5d680c64d0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c7a0 tx=0x7f5d6000cc60 comp rx=0 tx=0).stop 2023-10-08T07:10:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/919524367 shutdown_connections 2023-10-08T07:10:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5d4c06af00 0x7f5d4c06d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d6815ab30 0x7f5d680c64d0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d680b07d0 0x7f5d680c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 --2- 172.21.15.70:0/919524367 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d680adc70 0x7f5d68157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/919524367 >> 172.21.15.70:0/919524367 conn(0x7f5d680cb0c0 msgr2=0x7f5d6806cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:28.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/919524367 shutdown_connections 2023-10-08T07:10:28.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.373+0000 7f5d6e7e9640 1 -- 172.21.15.70:0/919524367 wait complete. 2023-10-08T07:10:28.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:10:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.588+0000 7f7049a4f640 1 Processor -- start 2023-10-08T07:10:28.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.589+0000 7f7049a4f640 1 -- start start 2023-10-08T07:10:28.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.590+0000 7f7049a4f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7044159ac0 0x7f704406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7049a4f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70440ad790 0x7f70440702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7049a4f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704415e330 0x7f7044070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7049a4f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704406dee0 con 0x7f7044159ac0 2023-10-08T07:10:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7049a4f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704415d850 con 0x7f70440ad790 2023-10-08T07:10:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7049a4f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704415db30 con 0x7f704415e330 2023-10-08T07:10:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7042ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7044159ac0 0x7f704406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7042ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7044159ac0 0x7f704406d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54192/0 (socket says 172.21.15.70:54192) 2023-10-08T07:10:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7042ffd640 1 -- 172.21.15.70:0/1484543119 learned_addr learned my addr 172.21.15.70:0/1484543119 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7042ffd640 1 -- 172.21.15.70:0/1484543119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704415e330 msgr2=0x7f7044070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7042ffd640 1 --2- 172.21.15.70:0/1484543119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704415e330 0x7f7044070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7042ffd640 1 -- 172.21.15.70:0/1484543119 >> v2:172.21.15.181:3300/0 conn(0x7f70440ad790 msgr2=0x7f70440702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7042ffd640 1 --2- 172.21.15.70:0/1484543119 >> v2:172.21.15.181:3300/0 conn(0x7f70440ad790 0x7f70440702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.591+0000 7f7042ffd640 1 -- 172.21.15.70:0/1484543119 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704415de10 con 0x7f7044159ac0 2023-10-08T07:10:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.592+0000 7f7042ffd640 1 --2- 172.21.15.70:0/1484543119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 0x7f704406d5e0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f70440b4970 tx=0x7f703402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a0dfedd1199ad7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7041ffb640 1 -- 172.21.15.70:0/1484543119 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703402ea80 con 0x7f7044159ac0 2023-10-08T07:10:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7041ffb640 1 -- 172.21.15.70:0/1484543119 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703402ebe0 con 0x7f7044159ac0 2023-10-08T07:10:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7041ffb640 1 -- 172.21.15.70:0/1484543119 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7034009600 con 0x7f7044159ac0 2023-10-08T07:10:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7049a4f640 1 -- 172.21.15.70:0/1484543119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 msgr2=0x7f704406d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7049a4f640 1 --2- 172.21.15.70:0/1484543119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 0x7f704406d5e0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f70440b4970 tx=0x7f703402e0e0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7049a4f640 1 -- 172.21.15.70:0/1484543119 shutdown_connections 2023-10-08T07:10:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7049a4f640 1 --2- 172.21.15.70:0/1484543119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704415e330 0x7f7044070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7049a4f640 1 --2- 172.21.15.70:0/1484543119 >> v2:172.21.15.181:3300/0 conn(0x7f70440ad790 0x7f70440702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7049a4f640 1 --2- 172.21.15.70:0/1484543119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 0x7f704406d5e0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7049a4f640 1 -- 172.21.15.70:0/1484543119 >> 172.21.15.70:0/1484543119 conn(0x7f704406a380 msgr2=0x7f70440b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7049a4f640 1 -- 172.21.15.70:0/1484543119 shutdown_connections 2023-10-08T07:10:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.593+0000 7f7049a4f640 1 -- 172.21.15.70:0/1484543119 wait complete. 2023-10-08T07:10:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.595+0000 7f7049a4f640 1 Processor -- start 2023-10-08T07:10:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.595+0000 7f7049a4f640 1 -- start start 2023-10-08T07:10:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.596+0000 7f7049a4f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70440ad790 0x7f704415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.596+0000 7f7049a4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 0x7f704414f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.596+0000 7f7049a4f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704415e330 0x7f7044156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.596+0000 7f7049a4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70441538e0 con 0x7f7044159ac0 2023-10-08T07:10:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.596+0000 7f7049a4f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7044153b80 con 0x7f70440ad790 2023-10-08T07:10:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.596+0000 7f7049a4f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7044153e60 con 0x7f704415e330 2023-10-08T07:10:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.596+0000 7f70427fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 0x7f704414f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.596+0000 7f70427fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 0x7f704414f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54202/0 (socket says 172.21.15.70:54202) 2023-10-08T07:10:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.596+0000 7f70427fc640 1 -- 172.21.15.70:0/2503837181 learned_addr learned my addr 172.21.15.70:0/2503837181 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f70427fc640 1 -- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704415e330 msgr2=0x7f7044156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f70427fc640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704415e330 0x7f7044156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f70427fc640 1 -- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70440ad790 msgr2=0x7f704415ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f70427fc640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70440ad790 0x7f704415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f70427fc640 1 -- 172.21.15.70:0/2503837181 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703400bd40 con 0x7f7044159ac0 2023-10-08T07:10:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f70427fc640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 0x7f704414f290 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f702c00b410 tx=0x7f702c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f703bfff640 1 -- 172.21.15.70:0/2503837181 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c011830 con 0x7f7044159ac0 2023-10-08T07:10:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f703bfff640 1 -- 172.21.15.70:0/2503837181 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c0119b0 con 0x7f7044159ac0 2023-10-08T07:10:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f7049a4f640 1 -- 172.21.15.70:0/2503837181 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7044154140 con 0x7f7044159ac0 2023-10-08T07:10:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f703bfff640 1 -- 172.21.15.70:0/2503837181 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c011d60 con 0x7f7044159ac0 2023-10-08T07:10:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.597+0000 7f7049a4f640 1 -- 172.21.15.70:0/2503837181 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7044154940 con 0x7f7044159ac0 2023-10-08T07:10:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.599+0000 7f7049a4f640 1 -- 172.21.15.70:0/2503837181 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7044154660 con 0x7f7044159ac0 2023-10-08T07:10:28.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.599+0000 7f703bfff640 1 -- 172.21.15.70:0/2503837181 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 115) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f702c01a400 con 0x7f7044159ac0 2023-10-08T07:10:28.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.601+0000 7f703bfff640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f702406af00 0x7f702406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.601+0000 7f703bfff640 1 -- 172.21.15.70:0/2503837181 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f702c089e20 con 0x7f7044159ac0 2023-10-08T07:10:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.601+0000 7f7042ffd640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f702406af00 0x7f702406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.603+0000 7f7042ffd640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f702406af00 0x7f702406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7034037000 tx=0x7f7034030000 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.605+0000 7f703bfff640 1 -- 172.21.15.70:0/2503837181 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7044154660 con 0x7f7044159ac0 2023-10-08T07:10:28.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.780+0000 7f7049a4f640 1 -- 172.21.15.70:0/2503837181 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7044154660 con 0x7f7044159ac0 2023-10-08T07:10:28.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.783+0000 7f703bfff640 1 -- 172.21.15.70:0/2503837181 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f7044154660 con 0x7f7044159ac0 2023-10-08T07:10:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:28.785 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":203,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:10:25.468736+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-08T06:54:30.246375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-08T06:54:31.487900+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"202","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":202,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6832/2291259069":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/2956479135":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/4233293606":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/2946222544":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/1250722759":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/1792732560":"2023-10-09T07:10:25.468616+0000","172.21.15.181:6833/3979056493":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6832/3979056493":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/340227645":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/849801254":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/4017559811":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/217631810":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/64410277":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/811429527":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/93223744":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/772181101":"2023-10-09T06:54:47.949157+0000","172.21.15.181:6833/2291259069":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/1286852011":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6832/3709692999":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/4147168289":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/437133264":"2023-10-09T07:10:25.468616+0000","172.21.15.181:6833/3709692999":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/602716182":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/1777884673":"2023-10-09T06:54:47.949157+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":24,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:10:28.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.785+0000 7f7049a4f640 1 -- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f702406af00 msgr2=0x7f702406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.785+0000 7f7049a4f640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f702406af00 0x7f702406d3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7034037000 tx=0x7f7034030000 comp rx=0 tx=0).stop 2023-10-08T07:10:28.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.785+0000 7f7049a4f640 1 -- 172.21.15.70:0/2503837181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 msgr2=0x7f704414f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:28.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.785+0000 7f7049a4f640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 0x7f704414f290 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f702c00b410 tx=0x7f702c00b8d0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.786+0000 7f7049a4f640 1 -- 172.21.15.70:0/2503837181 shutdown_connections 2023-10-08T07:10:28.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.786+0000 7f7049a4f640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f702406af00 0x7f702406d3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.786+0000 7f7049a4f640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f704415e330 0x7f7044156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.786+0000 7f7049a4f640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7044159ac0 0x7f704414f290 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.786+0000 7f7049a4f640 1 --2- 172.21.15.70:0/2503837181 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70440ad790 0x7f704415ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:28.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.786+0000 7f7049a4f640 1 -- 172.21.15.70:0/2503837181 >> 172.21.15.70:0/2503837181 conn(0x7f704406a380 msgr2=0x7f70440701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:28.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.786+0000 7f7049a4f640 1 -- 172.21.15.70:0/2503837181 shutdown_connections 2023-10-08T07:10:28.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:28.786+0000 7f7049a4f640 1 -- 172.21.15.70:0/2503837181 wait complete. 2023-10-08T07:10:28.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T07:10:29.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.013+0000 7f2399955640 1 Processor -- start 2023-10-08T07:10:29.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.014+0000 7f2399955640 1 -- start start 2023-10-08T07:10:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.015+0000 7f2399955640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f239406dfc0 0x7f2394070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.015+0000 7f2399955640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2394071070 0x7f23940cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.015+0000 7f2399955640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23940b1cc0 con 0x7f23940afbe0 2023-10-08T07:10:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.015+0000 7f2399955640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23940b2620 con 0x7f239406dfc0 2023-10-08T07:10:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.015+0000 7f2399955640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23940b2060 con 0x7f2394071070 2023-10-08T07:10:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.016+0000 7f23927fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f239406dfc0 0x7f2394070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.016+0000 7f23927fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f239406dfc0 0x7f2394070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54216/0 (socket says 172.21.15.70:54216) 2023-10-08T07:10:29.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.016+0000 7f23927fc640 1 -- 172.21.15.70:0/50358017 learned_addr learned my addr 172.21.15.70:0/50358017 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:29.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.016+0000 7f23937fe640 1 --2- 172.21.15.70:0/50358017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2394071070 0x7f23940cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:29.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.016+0000 7f23927fc640 1 -- 172.21.15.70:0/50358017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2394071070 msgr2=0x7f23940cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:29.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.016+0000 7f23927fc640 1 --2- 172.21.15.70:0/50358017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2394071070 0x7f23940cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.016+0000 7f23927fc640 1 -- 172.21.15.70:0/50358017 >> v1:172.21.15.181:6789/0 conn(0x7f23940afbe0 legacy=0x7f23940b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:29.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.016+0000 7f23927fc640 1 -- 172.21.15.70:0/50358017 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23940b2340 con 0x7f239406dfc0 2023-10-08T07:10:29.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.016+0000 7f23937fe640 1 --2- 172.21.15.70:0/50358017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2394071070 0x7f23940cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:29.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.017+0000 7f23927fc640 1 --2- 172.21.15.70:0/50358017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f239406dfc0 0x7f2394070b30 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f237c009870 tx=0x7f237c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf460ec9a3d475c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.017+0000 7f2391ffb640 1 -- 172.21.15.70:0/50358017 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c031640 con 0x7f239406dfc0 2023-10-08T07:10:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.017+0000 7f2391ffb640 1 -- 172.21.15.70:0/50358017 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c0317a0 con 0x7f239406dfc0 2023-10-08T07:10:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.017+0000 7f2391ffb640 1 -- 172.21.15.70:0/50358017 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c031b30 con 0x7f239406dfc0 2023-10-08T07:10:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.018+0000 7f2399955640 1 -- 172.21.15.70:0/50358017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f239406dfc0 msgr2=0x7f2394070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.018+0000 7f2399955640 1 --2- 172.21.15.70:0/50358017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f239406dfc0 0x7f2394070b30 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f237c009870 tx=0x7f237c02b8f0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.018+0000 7f2399955640 1 -- 172.21.15.70:0/50358017 shutdown_connections 2023-10-08T07:10:29.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.018+0000 7f2399955640 1 --2- 172.21.15.70:0/50358017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2394071070 0x7f23940cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.018+0000 7f2399955640 1 --2- 172.21.15.70:0/50358017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f239406dfc0 0x7f2394070b30 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.018+0000 7f2399955640 1 -- 172.21.15.70:0/50358017 >> 172.21.15.70:0/50358017 conn(0x7f239406a5d0 msgr2=0x7f23940b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:29.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.018+0000 7f2399955640 1 -- 172.21.15.70:0/50358017 shutdown_connections 2023-10-08T07:10:29.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.018+0000 7f2399955640 1 -- 172.21.15.70:0/50358017 wait complete. 2023-10-08T07:10:29.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.020+0000 7f2399955640 1 Processor -- start 2023-10-08T07:10:29.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.020+0000 7f2399955640 1 -- start start 2023-10-08T07:10:29.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f2399955640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f239406dfc0 0x7f23940cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f2399955640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2394071070 0x7f23940cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f2399955640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23940afbe0 0x7f23940c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f2399955640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23941f1130 con 0x7f23940afbe0 2023-10-08T07:10:29.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f2399955640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23941f0520 con 0x7f239406dfc0 2023-10-08T07:10:29.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f2399955640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23941f0800 con 0x7f2394071070 2023-10-08T07:10:29.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f23937fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23940afbe0 0x7f23940c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:29.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f23937fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23940afbe0 0x7f23940c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54226/0 (socket says 172.21.15.70:54226) 2023-10-08T07:10:29.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f23937fe640 1 -- 172.21.15.70:0/2323391444 learned_addr learned my addr 172.21.15.70:0/2323391444 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:29.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f23937fe640 1 -- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2394071070 msgr2=0x7f23940cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:29.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f23937fe640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2394071070 0x7f23940cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f23937fe640 1 -- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f239406dfc0 msgr2=0x7f23940cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:29.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f2392ffd640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f239406dfc0 0x7f23940cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:29.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f23937fe640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f239406dfc0 0x7f23940cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.021+0000 7f23937fe640 1 -- 172.21.15.70:0/2323391444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c0094d0 con 0x7f23940afbe0 2023-10-08T07:10:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.022+0000 7f23937fe640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23940afbe0 0x7f23940c5d20 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c7f0 tx=0x7f238c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.022+0000 7f238bfff640 1 -- 172.21.15.70:0/2323391444 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c007820 con 0x7f23940afbe0 2023-10-08T07:10:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.022+0000 7f238bfff640 1 -- 172.21.15.70:0/2323391444 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c007980 con 0x7f23940afbe0 2023-10-08T07:10:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.022+0000 7f238bfff640 1 -- 172.21.15.70:0/2323391444 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c007d30 con 0x7f23940afbe0 2023-10-08T07:10:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.022+0000 7f2399955640 1 -- 172.21.15.70:0/2323391444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23941f0ae0 con 0x7f23940afbe0 2023-10-08T07:10:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.022+0000 7f2399955640 1 -- 172.21.15.70:0/2323391444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23940c6260 con 0x7f23940afbe0 2023-10-08T07:10:29.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.023+0000 7f2399955640 1 -- 172.21.15.70:0/2323391444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239404e160 con 0x7f23940afbe0 2023-10-08T07:10:29.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.024+0000 7f238bfff640 1 -- 172.21.15.70:0/2323391444 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 115) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f238c0113f0 con 0x7f23940afbe0 2023-10-08T07:10:29.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.025+0000 7f238bfff640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f237406af00 0x7f237406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.025+0000 7f238bfff640 1 -- 172.21.15.70:0/2323391444 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 7205+0+0 (secure 0 0 0) 0x7f238c08a070 con 0x7f23940afbe0 2023-10-08T07:10:29.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.026+0000 7f2392ffd640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f237406af00 0x7f237406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:29.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.027+0000 7f2392ffd640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f237406af00 0x7f237406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2384007bf0 tx=0x7f2384007240 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:29.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.029+0000 7f238bfff640 1 -- 172.21.15.70:0/2323391444 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f238c054ad0 con 0x7f23940afbe0 2023-10-08T07:10:29.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.252+0000 7f2399955640 1 -- 172.21.15.70:0/2323391444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f23940b0ef0 con 0x7f23940afbe0 2023-10-08T07:10:29.255 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T07:10:29.253+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T07:10:29.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.257+0000 7f238bfff640 1 -- 172.21.15.70:0/2323391444 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7f238c01f090 con 0x7f23940afbe0 2023-10-08T07:10:29.258 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T07:10:29.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.259+0000 7f2399955640 1 -- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f237406af00 msgr2=0x7f237406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:29.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.259+0000 7f2399955640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f237406af00 0x7f237406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2384007bf0 tx=0x7f2384007240 comp rx=0 tx=0).stop 2023-10-08T07:10:29.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.259+0000 7f2399955640 1 -- 172.21.15.70:0/2323391444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23940afbe0 msgr2=0x7f23940c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:29.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.259+0000 7f2399955640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23940afbe0 0x7f23940c5d20 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c7f0 tx=0x7f238c00ccb0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.260+0000 7f2399955640 1 -- 172.21.15.70:0/2323391444 shutdown_connections 2023-10-08T07:10:29.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.260+0000 7f2399955640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f237406af00 0x7f237406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.260+0000 7f2399955640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23940afbe0 0x7f23940c5d20 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.260+0000 7f2399955640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2394071070 0x7f23940cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.260+0000 7f2399955640 1 --2- 172.21.15.70:0/2323391444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f239406dfc0 0x7f23940cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.260+0000 7f2399955640 1 -- 172.21.15.70:0/2323391444 >> 172.21.15.70:0/2323391444 conn(0x7f239406a5d0 msgr2=0x7f239415de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:29.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.260+0000 7f2399955640 1 -- 172.21.15.70:0/2323391444 shutdown_connections 2023-10-08T07:10:29.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.260+0000 7f2399955640 1 -- 172.21.15.70:0/2323391444 wait complete. 2023-10-08T07:10:29.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T07:10:29.352 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T07:10:29.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.463+0000 7f5cc264f640 1 Processor -- start 2023-10-08T07:10:29.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.463+0000 7f5cc264f640 1 -- start start 2023-10-08T07:10:29.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.464+0000 7f5cc264f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5cbc06c3a0 0x7f5cbc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.464+0000 7f5cc264f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cbc06db60 0x7f5cbc06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.465+0000 7f5cc264f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cbc15b310 con 0x7f5cbc15aa00 2023-10-08T07:10:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.465+0000 7f5cc264f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cbc0ca230 con 0x7f5cbc06c3a0 2023-10-08T07:10:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.465+0000 7f5cc264f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc06d290 con 0x7f5cbc06db60 2023-10-08T07:10:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.465+0000 7f5cbbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5cbc06c3a0 0x7f5cbc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.465+0000 7f5cbbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5cbc06c3a0 0x7f5cbc0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54248/0 (socket says 172.21.15.70:54248) 2023-10-08T07:10:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.465+0000 7f5cbbfff640 1 -- 172.21.15.70:0/413979607 learned_addr learned my addr 172.21.15.70:0/413979607 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.465+0000 7f5cbbfff640 1 -- 172.21.15.70:0/413979607 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cbc06db60 msgr2=0x7f5cbc06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.465+0000 7f5cbbfff640 1 --2- 172.21.15.70:0/413979607 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cbc06db60 0x7f5cbc06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.465+0000 7f5cbbfff640 1 -- 172.21.15.70:0/413979607 >> v1:172.21.15.181:6789/0 conn(0x7f5cbc15aa00 legacy=0x7f5cbc0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.465+0000 7f5cbbfff640 1 -- 172.21.15.70:0/413979607 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc0c9860 con 0x7f5cbc06c3a0 2023-10-08T07:10:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.466+0000 7f5cbbfff640 1 --2- 172.21.15.70:0/413979607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06c3a0 0x7f5cbc0c9cf0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009870 tx=0x7f5cac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e3ad558185a0ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:29.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cbb7fe640 1 -- 172.21.15.70:0/413979607 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac031640 con 0x7f5cbc06c3a0 2023-10-08T07:10:29.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cbb7fe640 1 -- 172.21.15.70:0/413979607 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac0317a0 con 0x7f5cbc06c3a0 2023-10-08T07:10:29.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cbb7fe640 1 -- 172.21.15.70:0/413979607 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac031b30 con 0x7f5cbc06c3a0 2023-10-08T07:10:29.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cc264f640 1 -- 172.21.15.70:0/413979607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06c3a0 msgr2=0x7f5cbc0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:29.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cc264f640 1 --2- 172.21.15.70:0/413979607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06c3a0 0x7f5cbc0c9cf0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009870 tx=0x7f5cac02b8f0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cc264f640 1 -- 172.21.15.70:0/413979607 shutdown_connections 2023-10-08T07:10:29.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cc264f640 1 --2- 172.21.15.70:0/413979607 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cbc06db60 0x7f5cbc06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cc264f640 1 --2- 172.21.15.70:0/413979607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06c3a0 0x7f5cbc0c9cf0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cc264f640 1 -- 172.21.15.70:0/413979607 >> 172.21.15.70:0/413979607 conn(0x7f5cbc06a5d0 msgr2=0x7f5cbc0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:29.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cc264f640 1 -- 172.21.15.70:0/413979607 shutdown_connections 2023-10-08T07:10:29.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.467+0000 7f5cc264f640 1 -- 172.21.15.70:0/413979607 wait complete. 2023-10-08T07:10:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.469+0000 7f5cc264f640 1 Processor -- start 2023-10-08T07:10:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.469+0000 7f5cc264f640 1 -- start start 2023-10-08T07:10:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.470+0000 7f5cc264f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cbc06c3a0 0x7f5cbc145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.470+0000 7f5cc264f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06db60 0x7f5cbc146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.470+0000 7f5cc264f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cbc15aa00 0x7f5cbc0c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.470+0000 7f5cc264f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc0c2310 con 0x7f5cbc06db60 2023-10-08T07:10:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.470+0000 7f5cc264f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc0c25f0 con 0x7f5cbc06c3a0 2023-10-08T07:10:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.470+0000 7f5cc264f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc0c28d0 con 0x7f5cbc15aa00 2023-10-08T07:10:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06db60 0x7f5cbc146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06db60 0x7f5cbc146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54262/0 (socket says 172.21.15.70:54262) 2023-10-08T07:10:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cb3fff640 1 -- 172.21.15.70:0/3379716016 learned_addr learned my addr 172.21.15.70:0/3379716016 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cbbfff640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cbc06c3a0 0x7f5cbc145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cc0bec640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cbc15aa00 0x7f5cbc0c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cb3fff640 1 -- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cbc15aa00 msgr2=0x7f5cbc0c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cb3fff640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cbc15aa00 0x7f5cbc0c33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cb3fff640 1 -- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cbc06c3a0 msgr2=0x7f5cbc145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cb3fff640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cbc06c3a0 0x7f5cbc145ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cb3fff640 1 -- 172.21.15.70:0/3379716016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0094d0 con 0x7f5cbc06db60 2023-10-08T07:10:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.471+0000 7f5cc0bec640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cbc15aa00 0x7f5cbc0c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.472+0000 7f5cb3fff640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06db60 0x7f5cbc146630 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b4a0 tx=0x7f5ca400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.472+0000 7f5cb97fa640 1 -- 172.21.15.70:0/3379716016 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4010610 con 0x7f5cbc06db60 2023-10-08T07:10:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.472+0000 7f5cb97fa640 1 -- 172.21.15.70:0/3379716016 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca4010770 con 0x7f5cbc06db60 2023-10-08T07:10:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.472+0000 7f5cb97fa640 1 -- 172.21.15.70:0/3379716016 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4010b20 con 0x7f5cbc06db60 2023-10-08T07:10:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.472+0000 7f5cc264f640 1 -- 172.21.15.70:0/3379716016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc0c2bb0 con 0x7f5cbc06db60 2023-10-08T07:10:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.472+0000 7f5cc264f640 1 -- 172.21.15.70:0/3379716016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc146000 con 0x7f5cbc06db60 2023-10-08T07:10:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.473+0000 7f5cc264f640 1 -- 172.21.15.70:0/3379716016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc0c30d0 con 0x7f5cbc06db60 2023-10-08T07:10:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.474+0000 7f5cb97fa640 1 -- 172.21.15.70:0/3379716016 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 115) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f5ca4010c80 con 0x7f5cbc06db60 2023-10-08T07:10:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.475+0000 7f5cb97fa640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5ca006af00 0x7f5ca006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.475+0000 7f5cb97fa640 1 -- 172.21.15.70:0/3379716016 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 7263+0+0 (secure 0 0 0) 0x7f5ca4089960 con 0x7f5cbc06db60 2023-10-08T07:10:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.476+0000 7f5cbbfff640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5ca006af00 0x7f5ca006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.477+0000 7f5cbbfff640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5ca006af00 0x7f5ca006d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5cac036000 tx=0x7f5cac03c040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.480+0000 7f5cb97fa640 1 -- 172.21.15.70:0/3379716016 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5cbc0c30d0 con 0x7f5cbc06db60 2023-10-08T07:10:29.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:29.690+0000 7f5cc264f640 1 -- 172.21.15.70:0/3379716016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5cbc0c30d0 con 0x7f5cbc06db60 2023-10-08T07:10:30.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.260+0000 7f5cb97fa640 1 -- 172.21.15.70:0/3379716016 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5cbc0c30d0 con 0x7f5cbc06db60 2023-10-08T07:10:30.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.263+0000 7f5cc264f640 1 -- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5ca006af00 msgr2=0x7f5ca006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:30.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.263+0000 7f5cc264f640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5ca006af00 0x7f5ca006d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5cac036000 tx=0x7f5cac03c040 comp rx=0 tx=0).stop 2023-10-08T07:10:30.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.263+0000 7f5cc264f640 1 -- 172.21.15.70:0/3379716016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06db60 msgr2=0x7f5cbc146630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:30.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.263+0000 7f5cc264f640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06db60 0x7f5cbc146630 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b4a0 tx=0x7f5ca400b960 comp rx=0 tx=0).stop 2023-10-08T07:10:30.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.263+0000 7f5cc264f640 1 -- 172.21.15.70:0/3379716016 shutdown_connections 2023-10-08T07:10:30.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.263+0000 7f5cc264f640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f5ca006af00 0x7f5ca006d3c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:30.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.263+0000 7f5cc264f640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5cbc15aa00 0x7f5cbc0c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:30.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.263+0000 7f5cc264f640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5cbc06db60 0x7f5cbc146630 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:30.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.263+0000 7f5cc264f640 1 --2- 172.21.15.70:0/3379716016 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5cbc06c3a0 0x7f5cbc145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:30.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.264+0000 7f5cc264f640 1 -- 172.21.15.70:0/3379716016 >> 172.21.15.70:0/3379716016 conn(0x7f5cbc06a5d0 msgr2=0x7f5cbc0ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:30.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.264+0000 7f5cc264f640 1 -- 172.21.15.70:0/3379716016 shutdown_connections 2023-10-08T07:10:30.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.264+0000 7f5cc264f640 1 -- 172.21.15.70:0/3379716016 wait complete. 2023-10-08T07:10:30.288 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T07:10:30.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.495+0000 7fce1d539640 1 Processor -- start 2023-10-08T07:10:30.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.495+0000 7fce1d539640 1 -- start start 2023-10-08T07:10:30.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.497+0000 7fce1d539640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce18161090 0x7fce180cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:30.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.497+0000 7fce1d539640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce180cde10 con 0x7fce1815ce00 2023-10-08T07:10:30.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.497+0000 7fce1d539640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce18161920 con 0x7fce180cd470 2023-10-08T07:10:30.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.497+0000 7fce1d539640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce18161640 con 0x7fce18161090 2023-10-08T07:10:30.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.497+0000 7fce16ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fce1815ce00 0x7fce180cb370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53520/0 (socket says 172.21.15.70:53520) 2023-10-08T07:10:30.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.497+0000 7fce16ffd640 1 -- 172.21.15.70:0/849640938 learned_addr learned my addr 172.21.15.70:0/849640938 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:30.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce177fe640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce18161090 0x7fce180cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:30.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112686427 0 0) 0x7fce180cde10 con 0x7fce1815ce00 2023-10-08T07:10:30.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce04003520 con 0x7fce1815ce00 2023-10-08T07:10:30.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 543283206 0 0) 0x7fce04003520 con 0x7fce1815ce00 2023-10-08T07:10:30.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce18161090 msgr2=0x7fce180cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce18161090 0x7fce180cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 >> v1:172.21.15.181:6789/0 conn(0x7fce180cd470 legacy=0x7fce18160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1806b690 con 0x7fce1815ce00 2023-10-08T07:10:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fce08001b70 con 0x7fce1815ce00 2023-10-08T07:10:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 >> v1:172.21.15.70:6789/0 conn(0x7fce1815ce00 legacy=0x7fce180cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce040039a0 0x7fce04003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce18161090 0x7fce040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce04004d00 0x7fce04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce18161640 con 0x7fce04004d00 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0400bb10 con 0x7fce040039a0 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.498+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0400bcd0 con 0x7fce18161090 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.499+0000 7fce177fe640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce04004d00 0x7fce04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.499+0000 7fce177fe640 1 -- 172.21.15.70:0/849640938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce18161090 msgr2=0x7fce040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.499+0000 7fce167fc640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce18161090 0x7fce040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.499+0000 7fce177fe640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce18161090 0x7fce040047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.499+0000 7fce177fe640 1 -- 172.21.15.70:0/849640938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce040039a0 msgr2=0x7fce04003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.499+0000 7fce177fe640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce040039a0 0x7fce04003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.499+0000 7fce177fe640 1 -- 172.21.15.70:0/849640938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce04003520 con 0x7fce04004d00 2023-10-08T07:10:30.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.500+0000 7fce177fe640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce04004d00 0x7fce04005180 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fce10009ac0 tx=0x7fce040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb18e09a503a60f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:30.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.500+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1002c6b0 con 0x7fce04004d00 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.500+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1002c810 con 0x7fce04004d00 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.500+0000 7fce15ffb640 1 -- 172.21.15.70:0/849640938 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1002cba0 con 0x7fce04004d00 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.501+0000 7fce1d539640 1 -- 172.21.15.70:0/849640938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce04004d00 msgr2=0x7fce04005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.501+0000 7fce1d539640 1 --2- 172.21.15.70:0/849640938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce04004d00 0x7fce04005180 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fce10009ac0 tx=0x7fce040057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.501+0000 7fce177fe640 1 -- 172.21.15.70:0/849640938 reap_dead start 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.501+0000 7fce1d539640 1 -- 172.21.15.70:0/849640938 shutdown_connections 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.501+0000 7fce1d539640 1 -- 172.21.15.70:0/849640938 >> 172.21.15.70:0/849640938 conn(0x7fce1806e650 msgr2=0x7fce1806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.501+0000 7fce1d539640 1 -- 172.21.15.70:0/849640938 shutdown_connections 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.501+0000 7fce1d539640 1 -- 172.21.15.70:0/849640938 wait complete. 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.503+0000 7fce1d539640 1 Processor -- start 2023-10-08T07:10:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.503+0000 7fce1d539640 1 -- start start 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce1d539640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce04004d00 0x7fce1815ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce1d539640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce180b3020 0x7fce181594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce1d539640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce180b1770 0x7fce180cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce1d539640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce180b2b90 con 0x7fce180b1770 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce1d539640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce18159a10 con 0x7fce180b3020 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce1d539640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce180b1f80 con 0x7fce04004d00 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce180b1770 0x7fce180cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce180b1770 0x7fce180cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54282/0 (socket says 172.21.15.70:54282) 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce177fe640 1 -- 172.21.15.70:0/325401824 learned_addr learned my addr 172.21.15.70:0/325401824 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce177fe640 1 -- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce04004d00 msgr2=0x7fce1815ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce177fe640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce04004d00 0x7fce1815ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:30.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce177fe640 1 -- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce180b3020 msgr2=0x7fce181594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:30.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce177fe640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce180b3020 0x7fce181594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:30.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.504+0000 7fce177fe640 1 -- 172.21.15.70:0/325401824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce180b2260 con 0x7fce180b1770 2023-10-08T07:10:30.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.505+0000 7fce177fe640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce180b1770 0x7fce180cbab0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fce10037040 tx=0x7fce10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:30.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.505+0000 7fce0f7fe640 1 -- 172.21.15.70:0/325401824 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10003810 con 0x7fce180b1770 2023-10-08T07:10:30.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.505+0000 7fce0f7fe640 1 -- 172.21.15.70:0/325401824 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce10003970 con 0x7fce180b1770 2023-10-08T07:10:30.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.505+0000 7fce0f7fe640 1 -- 172.21.15.70:0/325401824 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce10003d20 con 0x7fce180b1770 2023-10-08T07:10:30.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.505+0000 7fce1d539640 1 -- 172.21.15.70:0/325401824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce180b2540 con 0x7fce180b1770 2023-10-08T07:10:30.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.505+0000 7fce1d539640 1 -- 172.21.15.70:0/325401824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce180b1c20 con 0x7fce180b1770 2023-10-08T07:10:30.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.506+0000 7fce1d539640 1 -- 172.21.15.70:0/325401824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1806cff0 con 0x7fce180b1770 2023-10-08T07:10:30.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.507+0000 7fce0f7fe640 1 -- 172.21.15.70:0/325401824 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fce1003e440 con 0x7fce180b1770 2023-10-08T07:10:30.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.508+0000 7fce0f7fe640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcdf806afc0 0x7fcdf806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:30.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.508+0000 7fce0f7fe640 1 -- 172.21.15.70:0/325401824 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 7263+0+0 (secure 0 0 0) 0x7fce10000e40 con 0x7fce180b1770 2023-10-08T07:10:30.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.509+0000 7fce16ffd640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcdf806afc0 0x7fcdf806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:30.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.510+0000 7fce16ffd640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcdf806afc0 0x7fcdf806d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fce08002230 tx=0x7fce080019b0 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:30.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.513+0000 7fce0f7fe640 1 -- 172.21.15.70:0/325401824 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce10076460 con 0x7fce180b1770 2023-10-08T07:10:30.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:30.701+0000 7fce1d539640 1 -- 172.21.15.70:0/325401824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce1804e160 con 0x7fce180b1770 2023-10-08T07:10:31.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.528+0000 7fce0f7fe640 1 -- 172.21.15.70:0/325401824 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v206) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce10079a80 con 0x7fce180b1770 2023-10-08T07:10:31.530 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T07:10:31.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.532+0000 7fce1d539640 1 -- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcdf806afc0 msgr2=0x7fcdf806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:31.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.532+0000 7fce1d539640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcdf806afc0 0x7fcdf806d480 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fce08002230 tx=0x7fce080019b0 comp rx=0 tx=0).stop 2023-10-08T07:10:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.532+0000 7fce1d539640 1 -- 172.21.15.70:0/325401824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce180b1770 msgr2=0x7fce180cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.532+0000 7fce1d539640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce180b1770 0x7fce180cbab0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fce10037040 tx=0x7fce10004730 comp rx=0 tx=0).stop 2023-10-08T07:10:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.533+0000 7fce1d539640 1 -- 172.21.15.70:0/325401824 shutdown_connections 2023-10-08T07:10:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.533+0000 7fce1d539640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcdf806afc0 0x7fcdf806d480 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.533+0000 7fce1d539640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce180b1770 0x7fce180cbab0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:31.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.533+0000 7fce1d539640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fce180b3020 0x7fce181594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:31.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.533+0000 7fce1d539640 1 --2- 172.21.15.70:0/325401824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fce04004d00 0x7fce1815ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:31.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.533+0000 7fce1d539640 1 -- 172.21.15.70:0/325401824 >> 172.21.15.70:0/325401824 conn(0x7fce1806e650 msgr2=0x7fce1806ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:31.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.534+0000 7fce1d539640 1 -- 172.21.15.70:0/325401824 shutdown_connections 2023-10-08T07:10:31.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.534+0000 7fce1d539640 1 -- 172.21.15.70:0/325401824 wait complete. 2023-10-08T07:10:31.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T07:10:31.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.767+0000 7f5091bb0640 1 Processor -- start 2023-10-08T07:10:31.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.768+0000 7f5091bb0640 1 -- start start 2023-10-08T07:10:31.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.769+0000 7f5091bb0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f508c06dfc0 0x7f508c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:31.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.769+0000 7f5091bb0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f508c071070 0x7f508c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:31.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.769+0000 7f5091bb0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c0b1cc0 con 0x7f508c0afbe0 2023-10-08T07:10:31.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.769+0000 7f5091bb0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c0b2620 con 0x7f508c071070 2023-10-08T07:10:31.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.769+0000 7f5091bb0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0b2060 con 0x7f508c06dfc0 2023-10-08T07:10:31.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.770+0000 7f508bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f508c071070 0x7f508c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:31.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.770+0000 7f508affd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f508c06dfc0 0x7f508c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:31.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.770+0000 7f508bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f508c071070 0x7f508c0cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39904/0 (socket says 172.21.15.70:39904) 2023-10-08T07:10:31.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.770+0000 7f508bfff640 1 -- 172.21.15.70:0/300229648 learned_addr learned my addr 172.21.15.70:0/300229648 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:31.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.770+0000 7f508bfff640 1 -- 172.21.15.70:0/300229648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f508c06dfc0 msgr2=0x7f508c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:31.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.770+0000 7f508bfff640 1 --2- 172.21.15.70:0/300229648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f508c06dfc0 0x7f508c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:31.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.770+0000 7f508bfff640 1 -- 172.21.15.70:0/300229648 >> v1:172.21.15.181:6789/0 conn(0x7f508c0afbe0 legacy=0x7f508c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:31.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.770+0000 7f508bfff640 1 -- 172.21.15.70:0/300229648 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0b2340 con 0x7f508c071070 2023-10-08T07:10:31.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.772+0000 7f508bfff640 1 --2- 172.21.15.70:0/300229648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c071070 0x7f508c0cb130 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5084009870 tx=0x7f508402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae704148ddddfa39 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:31.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.772+0000 7f508a7fc640 1 -- 172.21.15.70:0/300229648 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084030840 con 0x7f508c071070 2023-10-08T07:10:31.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.772+0000 7f508a7fc640 1 -- 172.21.15.70:0/300229648 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50840309a0 con 0x7f508c071070 2023-10-08T07:10:31.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.772+0000 7f508a7fc640 1 -- 172.21.15.70:0/300229648 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084030d30 con 0x7f508c071070 2023-10-08T07:10:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.773+0000 7f5091bb0640 1 -- 172.21.15.70:0/300229648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c071070 msgr2=0x7f508c0cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.773+0000 7f5091bb0640 1 --2- 172.21.15.70:0/300229648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c071070 0x7f508c0cb130 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5084009870 tx=0x7f508402b980 comp rx=0 tx=0).stop 2023-10-08T07:10:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.773+0000 7f5091bb0640 1 -- 172.21.15.70:0/300229648 shutdown_connections 2023-10-08T07:10:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.773+0000 7f5091bb0640 1 --2- 172.21.15.70:0/300229648 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c071070 0x7f508c0cb130 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.773+0000 7f5091bb0640 1 --2- 172.21.15.70:0/300229648 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f508c06dfc0 0x7f508c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.773+0000 7f5091bb0640 1 -- 172.21.15.70:0/300229648 >> 172.21.15.70:0/300229648 conn(0x7f508c06a5d0 msgr2=0x7f508c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.773+0000 7f5091bb0640 1 -- 172.21.15.70:0/300229648 shutdown_connections 2023-10-08T07:10:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.773+0000 7f5091bb0640 1 -- 172.21.15.70:0/300229648 wait complete. 2023-10-08T07:10:31.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.775+0000 7f5091bb0640 1 Processor -- start 2023-10-08T07:10:31.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.775+0000 7f5091bb0640 1 -- start start 2023-10-08T07:10:31.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.776+0000 7f5091bb0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f508c06dfc0 0x7f508c1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.776+0000 7f5091bb0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c071070 0x7f508c15cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.776+0000 7f5091bb0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f508c0afbe0 0x7f508c0c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.776+0000 7f5091bb0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0cd7d0 con 0x7f508c0afbe0 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.776+0000 7f5091bb0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0cd480 con 0x7f508c071070 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.776+0000 7f5091bb0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c15d9d0 con 0x7f508c06dfc0 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.776+0000 7f508bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f508c0afbe0 0x7f508c0c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.776+0000 7f508bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f508c0afbe0 0x7f508c0c45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54288/0 (socket says 172.21.15.70:54288) 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.776+0000 7f508bfff640 1 -- 172.21.15.70:0/330575243 learned_addr learned my addr 172.21.15.70:0/330575243 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.777+0000 7f508affd640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c071070 0x7f508c15cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.777+0000 7f508b7fe640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f508c06dfc0 0x7f508c1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.777+0000 7f508bfff640 1 -- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f508c06dfc0 msgr2=0x7f508c1519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:31.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.777+0000 7f508bfff640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f508c06dfc0 0x7f508c1519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:31.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.777+0000 7f508bfff640 1 -- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c071070 msgr2=0x7f508c15cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:31.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.777+0000 7f508bfff640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c071070 0x7f508c15cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:31.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.777+0000 7f508bfff640 1 -- 172.21.15.70:0/330575243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50840094d0 con 0x7f508c0afbe0 2023-10-08T07:10:31.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.777+0000 7f508affd640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c071070 0x7f508c15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:31.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.777+0000 7f508bfff640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f508c0afbe0 0x7f508c0c45a0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f5084009870 tx=0x7f5084004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.777+0000 7f5083fff640 1 -- 172.21.15.70:0/330575243 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084039c90 con 0x7f508c0afbe0 2023-10-08T07:10:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.778+0000 7f5083fff640 1 -- 172.21.15.70:0/330575243 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5084039e10 con 0x7f508c0afbe0 2023-10-08T07:10:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.778+0000 7f5083fff640 1 -- 172.21.15.70:0/330575243 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084043640 con 0x7f508c0afbe0 2023-10-08T07:10:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.778+0000 7f5091bb0640 1 -- 172.21.15.70:0/330575243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c15dcb0 con 0x7f508c0afbe0 2023-10-08T07:10:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.778+0000 7f5091bb0640 1 -- 172.21.15.70:0/330575243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c151f20 con 0x7f508c0afbe0 2023-10-08T07:10:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.779+0000 7f5091bb0640 1 -- 172.21.15.70:0/330575243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c0cd940 con 0x7f508c0afbe0 2023-10-08T07:10:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.780+0000 7f5083fff640 1 -- 172.21.15.70:0/330575243 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f50840437a0 con 0x7f508c0afbe0 2023-10-08T07:10:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.781+0000 7f5083fff640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f506c06af90 0x7f506c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.781+0000 7f5083fff640 1 -- 172.21.15.70:0/330575243 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f508403e070 con 0x7f508c0afbe0 2023-10-08T07:10:31.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.781+0000 7f508b7fe640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f506c06af90 0x7f506c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.782+0000 7f508b7fe640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f506c06af90 0x7f506c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f507c000f60 tx=0x7f507c009040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:31.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.785+0000 7f5083fff640 1 -- 172.21.15.70:0/330575243 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5084077760 con 0x7f508c0afbe0 2023-10-08T07:10:31.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:31.983+0000 7f5091bb0640 1 -- 172.21.15.70:0/330575243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f508c04e160 con 0x7f508c0afbe0 2023-10-08T07:10:32.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.542+0000 7f5083fff640 1 -- 172.21.15.70:0/330575243 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v207) v1 ==== 169+0+0 (secure 0 0 0) 0x7f50840340d0 con 0x7f508c0afbe0 2023-10-08T07:10:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T07:10:32.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.545+0000 7f5091bb0640 1 -- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f506c06af90 msgr2=0x7f506c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:32.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.546+0000 7f5091bb0640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f506c06af90 0x7f506c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f507c000f60 tx=0x7f507c009040 comp rx=0 tx=0).stop 2023-10-08T07:10:32.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.546+0000 7f5091bb0640 1 -- 172.21.15.70:0/330575243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f508c0afbe0 msgr2=0x7f508c0c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:32.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.546+0000 7f5091bb0640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f508c0afbe0 0x7f508c0c45a0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f5084009870 tx=0x7f5084004730 comp rx=0 tx=0).stop 2023-10-08T07:10:32.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.548+0000 7f5091bb0640 1 -- 172.21.15.70:0/330575243 shutdown_connections 2023-10-08T07:10:32.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.548+0000 7f5091bb0640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f506c06af90 0x7f506c06d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:32.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.548+0000 7f5091bb0640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f508c0afbe0 0x7f508c0c45a0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.548+0000 7f5091bb0640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f508c071070 0x7f508c15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.548+0000 7f5091bb0640 1 --2- 172.21.15.70:0/330575243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f508c06dfc0 0x7f508c1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.548+0000 7f5091bb0640 1 -- 172.21.15.70:0/330575243 >> 172.21.15.70:0/330575243 conn(0x7f508c06a5d0 msgr2=0x7f508c15e870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.548+0000 7f5091bb0640 1 -- 172.21.15.70:0/330575243 shutdown_connections 2023-10-08T07:10:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.549+0000 7f5091bb0640 1 -- 172.21.15.70:0/330575243 wait complete. 2023-10-08T07:10:32.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T07:10:32.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.782+0000 7fc002cfc640 1 Processor -- start 2023-10-08T07:10:32.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.783+0000 7fc002cfc640 1 -- start start 2023-10-08T07:10:32.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.784+0000 7fc002cfc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbffc1601b0 0x7fbffc0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:32.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.784+0000 7fc002cfc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 0x7fbffc069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:32.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.785+0000 7fc002cfc640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc0ce210 con 0x7fbffc072090 2023-10-08T07:10:32.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.785+0000 7fc002cfc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc0cdee0 con 0x7fbffc1601b0 2023-10-08T07:10:32.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.785+0000 7fc002cfc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc161530 con 0x7fbffc071c30 2023-10-08T07:10:32.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.785+0000 7fc000a98640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbffc1601b0 0x7fbffc0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:32.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.785+0000 7fbffbfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 0x7fbffc069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:32.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.785+0000 7fc000a98640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbffc1601b0 0x7fbffc0cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:39918/0 (socket says 172.21.15.70:39918) 2023-10-08T07:10:32.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.785+0000 7fc000a98640 1 -- 172.21.15.70:0/4144728029 learned_addr learned my addr 172.21.15.70:0/4144728029 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:32.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.786+0000 7fbffbfff640 1 -- 172.21.15.70:0/4144728029 >> v2:172.21.15.181:3300/0 conn(0x7fbffc1601b0 msgr2=0x7fbffc0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:32.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.786+0000 7fbffbfff640 1 --2- 172.21.15.70:0/4144728029 >> v2:172.21.15.181:3300/0 conn(0x7fbffc1601b0 0x7fbffc0cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:32.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.786+0000 7fbffbfff640 1 -- 172.21.15.70:0/4144728029 >> v1:172.21.15.181:6789/0 conn(0x7fbffc072090 legacy=0x7fbffc0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:32.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.786+0000 7fbffbfff640 1 -- 172.21.15.70:0/4144728029 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc161810 con 0x7fbffc071c30 2023-10-08T07:10:32.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.786+0000 7fc000a98640 1 --2- 172.21.15.70:0/4144728029 >> v2:172.21.15.181:3300/0 conn(0x7fbffc1601b0 0x7fbffc0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:10:32.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.788+0000 7fbffbfff640 1 --2- 172.21.15.70:0/4144728029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 0x7fbffc069d50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80097a0 tx=0x7fbfe802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58a75867c29e2799 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:32.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.788+0000 7fbffb7fe640 1 -- 172.21.15.70:0/4144728029 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8031830 con 0x7fbffc071c30 2023-10-08T07:10:32.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.788+0000 7fbffb7fe640 1 -- 172.21.15.70:0/4144728029 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe8031990 con 0x7fbffc071c30 2023-10-08T07:10:32.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.788+0000 7fbffb7fe640 1 -- 172.21.15.70:0/4144728029 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8031d20 con 0x7fbffc071c30 2023-10-08T07:10:32.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.789+0000 7fc002cfc640 1 -- 172.21.15.70:0/4144728029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 msgr2=0x7fbffc069d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:32.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.789+0000 7fc002cfc640 1 --2- 172.21.15.70:0/4144728029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 0x7fbffc069d50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbfe80097a0 tx=0x7fbfe802b8d0 comp rx=0 tx=0).stop 2023-10-08T07:10:32.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.789+0000 7fc002cfc640 1 -- 172.21.15.70:0/4144728029 shutdown_connections 2023-10-08T07:10:32.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.789+0000 7fc002cfc640 1 --2- 172.21.15.70:0/4144728029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 0x7fbffc069d50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:32.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.789+0000 7fc002cfc640 1 --2- 172.21.15.70:0/4144728029 >> v2:172.21.15.181:3300/0 conn(0x7fbffc1601b0 0x7fbffc0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:32.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.789+0000 7fc002cfc640 1 -- 172.21.15.70:0/4144728029 >> 172.21.15.70:0/4144728029 conn(0x7fbffc06d430 msgr2=0x7fbffc06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:32.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.789+0000 7fc002cfc640 1 -- 172.21.15.70:0/4144728029 shutdown_connections 2023-10-08T07:10:32.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.789+0000 7fc002cfc640 1 -- 172.21.15.70:0/4144728029 wait complete. 2023-10-08T07:10:32.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.791+0000 7fc002cfc640 1 Processor -- start 2023-10-08T07:10:32.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.791+0000 7fc002cfc640 1 -- start start 2023-10-08T07:10:32.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fc002cfc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 0x7fbffc0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:32.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fc002cfc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc072090 0x7fbffc0c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:32.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fc002cfc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbffc1601b0 0x7fbffc15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:32.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fc002cfc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc1f4420 con 0x7fbffc1601b0 2023-10-08T07:10:32.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fc002cfc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0c48b0 con 0x7fbffc072090 2023-10-08T07:10:32.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fc002cfc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc1f3a80 con 0x7fbffc071c30 2023-10-08T07:10:32.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fc001299640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbffc1601b0 0x7fbffc15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:32.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fc001299640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbffc1601b0 0x7fbffc15a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54292/0 (socket says 172.21.15.70:54292) 2023-10-08T07:10:32.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fc001299640 1 -- 172.21.15.70:0/2374568316 learned_addr learned my addr 172.21.15.70:0/2374568316 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:32.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fc000a98640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 0x7fbffc0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:32.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.792+0000 7fbffbfff640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc072090 0x7fbffc0c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:32.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fc001299640 1 -- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 msgr2=0x7fbffc0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:32.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fc001299640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 0x7fbffc0c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:32.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fc001299640 1 -- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc072090 msgr2=0x7fbffc0c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:32.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fc001299640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc072090 0x7fbffc0c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:32.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fc001299640 1 -- 172.21.15.70:0/2374568316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00094d0 con 0x7fbffc1601b0 2023-10-08T07:10:32.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fbffbfff640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc072090 0x7fbffc0c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:32.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fc001299640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbffc1601b0 0x7fbffc15a8c0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fbff400c6b0 tx=0x7fbff400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:32.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fbff97fa640 1 -- 172.21.15.70:0/2374568316 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4003b70 con 0x7fbffc1601b0 2023-10-08T07:10:32.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fbff97fa640 1 -- 172.21.15.70:0/2374568316 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff4003cf0 con 0x7fbffc1601b0 2023-10-08T07:10:32.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fbff97fa640 1 -- 172.21.15.70:0/2374568316 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4003ea0 con 0x7fbffc1601b0 2023-10-08T07:10:32.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fc002cfc640 1 -- 172.21.15.70:0/2374568316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe8009400 con 0x7fbffc1601b0 2023-10-08T07:10:32.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.793+0000 7fc002cfc640 1 -- 172.21.15.70:0/2374568316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc1f4040 con 0x7fbffc1601b0 2023-10-08T07:10:32.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.794+0000 7fc002cfc640 1 -- 172.21.15.70:0/2374568316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc1f4560 con 0x7fbffc1601b0 2023-10-08T07:10:32.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.795+0000 7fbff97fa640 1 -- 172.21.15.70:0/2374568316 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbff4005310 con 0x7fbffc1601b0 2023-10-08T07:10:32.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.796+0000 7fbff97fa640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fbfe006af90 0x7fbfe006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:32.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.796+0000 7fbff97fa640 1 -- 172.21.15.70:0/2374568316 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6411+0+0 (secure 0 0 0) 0x7fbff40898a0 con 0x7fbffc1601b0 2023-10-08T07:10:32.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.797+0000 7fc000a98640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fbfe006af90 0x7fbfe006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:32.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.798+0000 7fc000a98640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fbfe006af90 0x7fbfe006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009c50 tx=0x7fbff0008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:32.803+0000 7fbff97fa640 1 -- 172.21.15.70:0/2374568316 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbff40544e0 con 0x7fbffc1601b0 2023-10-08T07:10:33.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.003+0000 7fc002cfc640 1 -- 172.21.15.70:0/2374568316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbffc04e160 con 0x7fbffc1601b0 2023-10-08T07:10:33.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.010+0000 7fbff97fa640 1 -- 172.21.15.70:0/2374568316 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbff4057b00 con 0x7fbffc1601b0 2023-10-08T07:10:33.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 -- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fbfe006af90 msgr2=0x7fbfe006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fbfe006af90 0x7fbfe006d450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009c50 tx=0x7fbff0008040 comp rx=0 tx=0).stop 2023-10-08T07:10:33.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 -- 172.21.15.70:0/2374568316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbffc1601b0 msgr2=0x7fbffc15a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbffc1601b0 0x7fbffc15a8c0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fbff400c6b0 tx=0x7fbff400cb70 comp rx=0 tx=0).stop 2023-10-08T07:10:33.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 -- 172.21.15.70:0/2374568316 shutdown_connections 2023-10-08T07:10:33.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fbfe006af90 0x7fbfe006d450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbffc1601b0 0x7fbffc15a8c0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbffc072090 0x7fbffc0c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 --2- 172.21.15.70:0/2374568316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbffc071c30 0x7fbffc0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 -- 172.21.15.70:0/2374568316 >> 172.21.15.70:0/2374568316 conn(0x7fbffc06d430 msgr2=0x7fbffc069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:33.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 -- 172.21.15.70:0/2374568316 shutdown_connections 2023-10-08T07:10:33.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.013+0000 7fc002cfc640 1 -- 172.21.15.70:0/2374568316 wait complete. 2023-10-08T07:10:33.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T07:10:33.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.249+0000 7fcc3cdd0640 1 Processor -- start 2023-10-08T07:10:33.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.249+0000 7fcc3cdd0640 1 -- start start 2023-10-08T07:10:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.250+0000 7fcc3cdd0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcc3815aad0 0x7fcc380b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.250+0000 7fcc3cdd0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc380c82b0 0x7fcc380c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.251+0000 7fcc3cdd0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3815bf30 con 0x7fcc3806cd00 2023-10-08T07:10:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.251+0000 7fcc3cdd0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3815b320 con 0x7fcc380c82b0 2023-10-08T07:10:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.251+0000 7fcc3cdd0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3815b600 con 0x7fcc3815aad0 2023-10-08T07:10:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.251+0000 7fcc2ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc380c82b0 0x7fcc380c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.251+0000 7fcc2ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc380c82b0 0x7fcc380c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54298/0 (socket says 172.21.15.70:54298) 2023-10-08T07:10:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.251+0000 7fcc2ffff640 1 -- 172.21.15.70:0/788414630 learned_addr learned my addr 172.21.15.70:0/788414630 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.251+0000 7fcc2ffff640 1 -- 172.21.15.70:0/788414630 >> v2:172.21.15.181:3300/0 conn(0x7fcc3815aad0 msgr2=0x7fcc380b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.251+0000 7fcc2ffff640 1 --2- 172.21.15.70:0/788414630 >> v2:172.21.15.181:3300/0 conn(0x7fcc3815aad0 0x7fcc380b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.251+0000 7fcc2ffff640 1 -- 172.21.15.70:0/788414630 >> v1:172.21.15.181:6789/0 conn(0x7fcc3806cd00 legacy=0x7fcc38070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:33.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.251+0000 7fcc2ffff640 1 -- 172.21.15.70:0/788414630 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3815b8e0 con 0x7fcc380c82b0 2023-10-08T07:10:33.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.252+0000 7fcc2ffff640 1 --2- 172.21.15.70:0/788414630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc380c82b0 0x7fcc380c8710 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fcc3806e6a0 tx=0x7fcc2002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3024dbe0c72c66b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:33.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.253+0000 7fcc35d9b640 1 -- 172.21.15.70:0/788414630 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20031890 con 0x7fcc380c82b0 2023-10-08T07:10:33.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.253+0000 7fcc35d9b640 1 -- 172.21.15.70:0/788414630 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc200319f0 con 0x7fcc380c82b0 2023-10-08T07:10:33.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.253+0000 7fcc35d9b640 1 -- 172.21.15.70:0/788414630 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20031d80 con 0x7fcc380c82b0 2023-10-08T07:10:33.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.253+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/788414630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc380c82b0 msgr2=0x7fcc380c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.253+0000 7fcc3cdd0640 1 --2- 172.21.15.70:0/788414630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc380c82b0 0x7fcc380c8710 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fcc3806e6a0 tx=0x7fcc2002b930 comp rx=0 tx=0).stop 2023-10-08T07:10:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.254+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/788414630 shutdown_connections 2023-10-08T07:10:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.254+0000 7fcc3cdd0640 1 --2- 172.21.15.70:0/788414630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc380c82b0 0x7fcc380c8710 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.254+0000 7fcc3cdd0640 1 --2- 172.21.15.70:0/788414630 >> v2:172.21.15.181:3300/0 conn(0x7fcc3815aad0 0x7fcc380b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.254+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/788414630 >> 172.21.15.70:0/788414630 conn(0x7fcc38069d50 msgr2=0x7fcc3806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.254+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/788414630 shutdown_connections 2023-10-08T07:10:33.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.254+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/788414630 wait complete. 2023-10-08T07:10:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.255+0000 7fcc3cdd0640 1 Processor -- start 2023-10-08T07:10:33.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.256+0000 7fcc3cdd0640 1 -- start start 2023-10-08T07:10:33.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc3cdd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc3806cd00 0x7fcc38141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc3cdd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc380c82b0 0x7fcc381441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc3cdd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3815aad0 0x7fcc38153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc3cdd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc380ccf20 con 0x7fcc3815aad0 2023-10-08T07:10:33.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc3cdd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc380cc310 con 0x7fcc3806cd00 2023-10-08T07:10:33.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc3cdd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc380cc5f0 con 0x7fcc380c82b0 2023-10-08T07:10:33.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc36d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3815aad0 0x7fcc38153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc36d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3815aad0 0x7fcc38153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54310/0 (socket says 172.21.15.70:54310) 2023-10-08T07:10:33.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc36d9d640 1 -- 172.21.15.70:0/2780783193 learned_addr learned my addr 172.21.15.70:0/2780783193 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:33.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc2ffff640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc380c82b0 0x7fcc381441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc36d9d640 1 -- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc380c82b0 msgr2=0x7fcc381441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc36d9d640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc380c82b0 0x7fcc381441d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc36d9d640 1 -- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc3806cd00 msgr2=0x7fcc38141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc3659c640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc3806cd00 0x7fcc38141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc36d9d640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc3806cd00 0x7fcc38141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.257+0000 7fcc36d9d640 1 -- 172.21.15.70:0/2780783193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc200094d0 con 0x7fcc3815aad0 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.258+0000 7fcc3659c640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc3806cd00 0x7fcc38141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.258+0000 7fcc36d9d640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3815aad0 0x7fcc38153dd0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000c780 tx=0x7fcc3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.258+0000 7fcc2effd640 1 -- 172.21.15.70:0/2780783193 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30003d80 con 0x7fcc3815aad0 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.258+0000 7fcc2effd640 1 -- 172.21.15.70:0/2780783193 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3000ce90 con 0x7fcc3815aad0 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.258+0000 7fcc2effd640 1 -- 172.21.15.70:0/2780783193 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc300052b0 con 0x7fcc3815aad0 2023-10-08T07:10:33.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.258+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/2780783193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc380cc8d0 con 0x7fcc3815aad0 2023-10-08T07:10:33.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.258+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/2780783193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc38142380 con 0x7fcc3815aad0 2023-10-08T07:10:33.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.259+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/2780783193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc380cd090 con 0x7fcc3815aad0 2023-10-08T07:10:33.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.260+0000 7fcc2effd640 1 -- 172.21.15.70:0/2780783193 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcc30005410 con 0x7fcc3815aad0 2023-10-08T07:10:33.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.261+0000 7fcc2effd640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcc1806af90 0x7fcc1806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.261+0000 7fcc2effd640 1 -- 172.21.15.70:0/2780783193 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6411+0+0 (secure 0 0 0) 0x7fcc30088f30 con 0x7fcc3815aad0 2023-10-08T07:10:33.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.262+0000 7fcc3659c640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcc1806af90 0x7fcc1806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.263+0000 7fcc3659c640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcc1806af90 0x7fcc1806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800a870 tx=0x7fcc28008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.266+0000 7fcc2effd640 1 -- 172.21.15.70:0/2780783193 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc30053c00 con 0x7fcc3815aad0 2023-10-08T07:10:33.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.454+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/2780783193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fcc3804e160 con 0x7fcc3815aad0 2023-10-08T07:10:33.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.460+0000 7fcc2effd640 1 -- 172.21.15.70:0/2780783193 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v68) v1 ==== 115+0+0 (secure 0 0 0) 0x7fcc30017020 con 0x7fcc3815aad0 2023-10-08T07:10:33.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.462+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcc1806af90 msgr2=0x7fcc1806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.462+0000 7fcc3cdd0640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcc1806af90 0x7fcc1806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcc2800a870 tx=0x7fcc28008040 comp rx=0 tx=0).stop 2023-10-08T07:10:33.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.462+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/2780783193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3815aad0 msgr2=0x7fcc38153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.462+0000 7fcc3cdd0640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3815aad0 0x7fcc38153dd0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000c780 tx=0x7fcc3000cc40 comp rx=0 tx=0).stop 2023-10-08T07:10:33.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.462+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/2780783193 shutdown_connections 2023-10-08T07:10:33.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.462+0000 7fcc3cdd0640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fcc1806af90 0x7fcc1806d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.462+0000 7fcc3cdd0640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc3815aad0 0x7fcc38153dd0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.462+0000 7fcc3cdd0640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc380c82b0 0x7fcc381441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.462+0000 7fcc3cdd0640 1 --2- 172.21.15.70:0/2780783193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc3806cd00 0x7fcc38141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.462+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/2780783193 >> 172.21.15.70:0/2780783193 conn(0x7fcc38069d50 msgr2=0x7fcc38070570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:33.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.463+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/2780783193 shutdown_connections 2023-10-08T07:10:33.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.463+0000 7fcc3cdd0640 1 -- 172.21.15.70:0/2780783193 wait complete. 2023-10-08T07:10:33.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T07:10:33.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.694+0000 7fb8232d7640 1 Processor -- start 2023-10-08T07:10:33.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.695+0000 7fb8232d7640 1 -- start start 2023-10-08T07:10:33.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.696+0000 7fb8232d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb81c06dfc0 0x7fb81c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.696+0000 7fb8232d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c071070 0x7fb81c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.696+0000 7fb8232d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c0b1cc0 con 0x7fb81c0afbe0 2023-10-08T07:10:33.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.696+0000 7fb8232d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c0b2620 con 0x7fb81c06dfc0 2023-10-08T07:10:33.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.696+0000 7fb8232d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0b2060 con 0x7fb81c071070 2023-10-08T07:10:33.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.697+0000 7fb820872640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb81c06dfc0 0x7fb81c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.697+0000 7fb820872640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb81c06dfc0 0x7fb81c070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48398/0 (socket says 172.21.15.70:48398) 2023-10-08T07:10:33.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.697+0000 7fb820872640 1 -- 172.21.15.70:0/43249346 learned_addr learned my addr 172.21.15.70:0/43249346 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:33.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.697+0000 7fb821874640 1 --2- 172.21.15.70:0/43249346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c071070 0x7fb81c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.697+0000 7fb820872640 1 -- 172.21.15.70:0/43249346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c071070 msgr2=0x7fb81c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:33.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.697+0000 7fb820872640 1 --2- 172.21.15.70:0/43249346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c071070 0x7fb81c0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.697+0000 7fb820872640 1 -- 172.21.15.70:0/43249346 >> v1:172.21.15.181:6789/0 conn(0x7fb81c0afbe0 legacy=0x7fb81c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.697+0000 7fb820872640 1 -- 172.21.15.70:0/43249346 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c0b2340 con 0x7fb81c06dfc0 2023-10-08T07:10:33.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.698+0000 7fb820872640 1 --2- 172.21.15.70:0/43249346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 0x7fb81c070b30 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fb810009870 tx=0x7fb81002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c68e1a3a1e6644 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:33.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb81bfff640 1 -- 172.21.15.70:0/43249346 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810030840 con 0x7fb81c06dfc0 2023-10-08T07:10:33.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb81bfff640 1 -- 172.21.15.70:0/43249346 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8100309a0 con 0x7fb81c06dfc0 2023-10-08T07:10:33.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb81bfff640 1 -- 172.21.15.70:0/43249346 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810030d30 con 0x7fb81c06dfc0 2023-10-08T07:10:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb8232d7640 1 -- 172.21.15.70:0/43249346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 msgr2=0x7fb81c070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb8232d7640 1 --2- 172.21.15.70:0/43249346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 0x7fb81c070b30 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fb810009870 tx=0x7fb81002b980 comp rx=0 tx=0).stop 2023-10-08T07:10:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb8232d7640 1 -- 172.21.15.70:0/43249346 shutdown_connections 2023-10-08T07:10:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb8232d7640 1 --2- 172.21.15.70:0/43249346 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c071070 0x7fb81c0cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb8232d7640 1 --2- 172.21.15.70:0/43249346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 0x7fb81c070b30 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb8232d7640 1 -- 172.21.15.70:0/43249346 >> 172.21.15.70:0/43249346 conn(0x7fb81c06a5d0 msgr2=0x7fb81c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb8232d7640 1 -- 172.21.15.70:0/43249346 shutdown_connections 2023-10-08T07:10:33.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.699+0000 7fb8232d7640 1 -- 172.21.15.70:0/43249346 wait complete. 2023-10-08T07:10:33.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.701+0000 7fb8232d7640 1 Processor -- start 2023-10-08T07:10:33.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.701+0000 7fb8232d7640 1 -- start start 2023-10-08T07:10:33.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.702+0000 7fb8232d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 0x7fb81c0c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.702+0000 7fb8232d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c071070 0x7fb81c15cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.702+0000 7fb8232d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c0afbe0 0x7fb81c1479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.702+0000 7fb8232d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c147f10 con 0x7fb81c06dfc0 2023-10-08T07:10:33.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.702+0000 7fb8232d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0cd080 con 0x7fb81c071070 2023-10-08T07:10:33.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.702+0000 7fb8232d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c0c9370 con 0x7fb81c0afbe0 2023-10-08T07:10:33.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821073640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 0x7fb81c0c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821073640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 0x7fb81c0c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48400/0 (socket says 172.21.15.70:48400) 2023-10-08T07:10:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821073640 1 -- 172.21.15.70:0/2665087052 learned_addr learned my addr 172.21.15.70:0/2665087052 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb820872640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c071070 0x7fb81c15cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821874640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c0afbe0 0x7fb81c1479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821073640 1 -- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c0afbe0 msgr2=0x7fb81c1479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821073640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c0afbe0 0x7fb81c1479d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821073640 1 -- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c071070 msgr2=0x7fb81c15cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821073640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c071070 0x7fb81c15cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821073640 1 -- 172.21.15.70:0/2665087052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8100094d0 con 0x7fb81c06dfc0 2023-10-08T07:10:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821874640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c0afbe0 0x7fb81c1479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:10:33.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb820872640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c071070 0x7fb81c15cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:10:33.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.703+0000 7fb821073640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 0x7fb81c0c8d90 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00c730 tx=0x7fb80c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:33.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.704+0000 7fb819ffb640 1 -- 172.21.15.70:0/2665087052 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c006d50 con 0x7fb81c06dfc0 2023-10-08T07:10:33.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.704+0000 7fb819ffb640 1 -- 172.21.15.70:0/2665087052 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c00ce40 con 0x7fb81c06dfc0 2023-10-08T07:10:33.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.704+0000 7fb819ffb640 1 -- 172.21.15.70:0/2665087052 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c003bb0 con 0x7fb81c06dfc0 2023-10-08T07:10:33.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.704+0000 7fb8232d7640 1 -- 172.21.15.70:0/2665087052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c0c9650 con 0x7fb81c06dfc0 2023-10-08T07:10:33.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.704+0000 7fb8232d7640 1 -- 172.21.15.70:0/2665087052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c0c9c10 con 0x7fb81c06dfc0 2023-10-08T07:10:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.705+0000 7fb8232d7640 1 -- 172.21.15.70:0/2665087052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c04e160 con 0x7fb81c06dfc0 2023-10-08T07:10:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.706+0000 7fb819ffb640 1 -- 172.21.15.70:0/2665087052 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb80c003d10 con 0x7fb81c06dfc0 2023-10-08T07:10:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.707+0000 7fb819ffb640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb80806af90 0x7fb80806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.707+0000 7fb819ffb640 1 -- 172.21.15.70:0/2665087052 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6411+0+0 (secure 0 0 0) 0x7fb80c089640 con 0x7fb81c06dfc0 2023-10-08T07:10:33.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.707+0000 7fb820872640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb80806af90 0x7fb80806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.709+0000 7fb820872640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb80806af90 0x7fb80806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb810036000 tx=0x7fb810008030 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:33.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.711+0000 7fb819ffb640 1 -- 172.21.15.70:0/2665087052 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb80c057da0 con 0x7fb81c06dfc0 2023-10-08T07:10:33.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.900+0000 7fb8232d7640 1 -- 172.21.15.70:0/2665087052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fb81c0b0ef0 con 0x7fb81c06dfc0 2023-10-08T07:10:33.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.906+0000 7fb819ffb640 1 -- 172.21.15.70:0/2665087052 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v69) v1 ==== 120+0+0 (secure 0 0 0) 0x7fb80c05ce60 con 0x7fb81c06dfc0 2023-10-08T07:10:33.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.908+0000 7fb8232d7640 1 -- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb80806af90 msgr2=0x7fb80806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.908+0000 7fb8232d7640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb80806af90 0x7fb80806d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb810036000 tx=0x7fb810008030 comp rx=0 tx=0).stop 2023-10-08T07:10:33.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.908+0000 7fb8232d7640 1 -- 172.21.15.70:0/2665087052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 msgr2=0x7fb81c0c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:33.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.908+0000 7fb8232d7640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 0x7fb81c0c8d90 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00c730 tx=0x7fb80c00cbf0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.909+0000 7fb8232d7640 1 -- 172.21.15.70:0/2665087052 shutdown_connections 2023-10-08T07:10:33.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.909+0000 7fb8232d7640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb80806af90 0x7fb80806d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.909+0000 7fb8232d7640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb81c0afbe0 0x7fb81c1479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.909+0000 7fb8232d7640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb81c071070 0x7fb81c15cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.909+0000 7fb8232d7640 1 --2- 172.21.15.70:0/2665087052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb81c06dfc0 0x7fb81c0c8d90 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:33.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.909+0000 7fb8232d7640 1 -- 172.21.15.70:0/2665087052 >> 172.21.15.70:0/2665087052 conn(0x7fb81c06a5d0 msgr2=0x7fb81c15de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:33.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.909+0000 7fb8232d7640 1 -- 172.21.15.70:0/2665087052 shutdown_connections 2023-10-08T07:10:33.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:33.909+0000 7fb8232d7640 1 -- 172.21.15.70:0/2665087052 wait complete. 2023-10-08T07:10:33.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-08T07:10:34.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.136+0000 7f62af7a5640 1 Processor -- start 2023-10-08T07:10:34.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.137+0000 7f62af7a5640 1 -- start start 2023-10-08T07:10:34.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.138+0000 7f62af7a5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f62a8069d50 0x7f62a806a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.138+0000 7f62af7a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a80cb880 0x7f62a80b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.138+0000 7f62af7a5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a80ce550 con 0x7f62a80cbce0 2023-10-08T07:10:34.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.138+0000 7f62af7a5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a81609d0 con 0x7f62a8069d50 2023-10-08T07:10:34.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.138+0000 7f62af7a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80cdcf0 con 0x7f62a80cb880 2023-10-08T07:10:34.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.139+0000 7f62add42640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f62a80cbce0 0x7f62a8160e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41606/0 (socket says 172.21.15.70:41606) 2023-10-08T07:10:34.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.139+0000 7f62add42640 1 -- 172.21.15.70:0/3126014339 learned_addr learned my addr 172.21.15.70:0/3126014339 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:34.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.139+0000 7f62acd40640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a80cb880 0x7f62a80b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.139+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632234927 0 0) 0x7f62a80ce550 con 0x7f62a80cbce0 2023-10-08T07:10:34.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.139+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6294003520 con 0x7f62a80cbce0 2023-10-08T07:10:34.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.139+0000 7f62ad541640 1 --2- 172.21.15.70:0/3126014339 >> v2:172.21.15.181:3300/0 conn(0x7f62a8069d50 0x7f62a806a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.139+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518121880 0 0) 0x7f6294003520 con 0x7f62a80cbce0 2023-10-08T07:10:34.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.139+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a80cb880 msgr2=0x7f62a80b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:34.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.139+0000 7f629ffff640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a80cb880 0x7f62a80b1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 >> v2:172.21.15.181:3300/0 conn(0x7f62a8069d50 msgr2=0x7f62a806a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 --2- 172.21.15.70:0/3126014339 >> v2:172.21.15.181:3300/0 conn(0x7f62a8069d50 0x7f62a806a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a806a670 con 0x7f62a80cbce0 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f62a0001670 con 0x7f62a80cbce0 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 >> v1:172.21.15.70:6789/0 conn(0x7f62a80cbce0 legacy=0x7f62a8160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294003970 0x7f6294003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62940047c0 0x7f6294004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a80cb880 0x7f6294005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a81609d0 con 0x7f6294003970 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80cdcf0 con 0x7f62940047c0 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629400b9d0 con 0x7f62a80cb880 2023-10-08T07:10:34.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f62ad541640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294003970 0x7f6294003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f62add42640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a80cb880 0x7f6294005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f62ad541640 1 -- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a80cb880 msgr2=0x7f6294005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:34.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f62ad541640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a80cb880 0x7f6294005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f62ad541640 1 -- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62940047c0 msgr2=0x7f6294004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:34.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f62ad541640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62940047c0 0x7f6294004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f62ad541640 1 -- 172.21.15.70:0/3126014339 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6294003520 con 0x7f6294003970 2023-10-08T07:10:34.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.140+0000 7f62add42640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a80cb880 0x7f6294005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:34.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.142+0000 7f62ad541640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294003970 0x7f6294003d90 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f62a4007360 tx=0x7f62940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40db66830514ba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:34.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.142+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a402ccf0 con 0x7f6294003970 2023-10-08T07:10:34.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.142+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a402ce50 con 0x7f6294003970 2023-10-08T07:10:34.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.142+0000 7f629ffff640 1 -- 172.21.15.70:0/3126014339 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a4034660 con 0x7f6294003970 2023-10-08T07:10:34.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.142+0000 7f62af7a5640 1 -- 172.21.15.70:0/3126014339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294003970 msgr2=0x7f6294003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.142+0000 7f62af7a5640 1 --2- 172.21.15.70:0/3126014339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6294003970 0x7f6294003d90 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f62a4007360 tx=0x7f62940057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.142+0000 7f62add42640 1 -- 172.21.15.70:0/3126014339 reap_dead start 2023-10-08T07:10:34.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.142+0000 7f62af7a5640 1 -- 172.21.15.70:0/3126014339 shutdown_connections 2023-10-08T07:10:34.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.142+0000 7f62af7a5640 1 -- 172.21.15.70:0/3126014339 >> 172.21.15.70:0/3126014339 conn(0x7f62a806d450 msgr2=0x7f62a806d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:34.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.143+0000 7f62af7a5640 1 -- 172.21.15.70:0/3126014339 shutdown_connections 2023-10-08T07:10:34.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.143+0000 7f62af7a5640 1 -- 172.21.15.70:0/3126014339 wait complete. 2023-10-08T07:10:34.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.144+0000 7f62af7a5640 1 Processor -- start 2023-10-08T07:10:34.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.144+0000 7f62af7a5640 1 -- start start 2023-10-08T07:10:34.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.145+0000 7f62af7a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6294003970 0x7f62a80b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.145+0000 7f62af7a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a814a5f0 0x7f62a8147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.145+0000 7f62af7a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a8149240 0x7f62a80cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.145+0000 7f62af7a5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8148e20 con 0x7f62a814a5f0 2023-10-08T07:10:34.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.145+0000 7f62af7a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a814aa70 con 0x7f6294003970 2023-10-08T07:10:34.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.145+0000 7f62af7a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8147f70 con 0x7f62a8149240 2023-10-08T07:10:34.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.146+0000 7f62acd40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a814a5f0 0x7f62a8147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.146+0000 7f62acd40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a814a5f0 0x7f62a8147a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48424/0 (socket says 172.21.15.70:48424) 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.146+0000 7f62acd40640 1 -- 172.21.15.70:0/3686593057 learned_addr learned my addr 172.21.15.70:0/3686593057 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.146+0000 7f62ad541640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6294003970 0x7f62a80b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.146+0000 7f62add42640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a8149240 0x7f62a80cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.146+0000 7f62acd40640 1 -- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a8149240 msgr2=0x7f62a80cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.146+0000 7f62acd40640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a8149240 0x7f62a80cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.146+0000 7f62acd40640 1 -- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6294003970 msgr2=0x7f62a80b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.146+0000 7f62acd40640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6294003970 0x7f62a80b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.146+0000 7f62acd40640 1 -- 172.21.15.70:0/3686593057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a81481a0 con 0x7f62a814a5f0 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.147+0000 7f62acd40640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a814a5f0 0x7f62a8147a30 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f629800e7d0 tx=0x7f629800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.147+0000 7f629dffb640 1 -- 172.21.15.70:0/3686593057 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800eee0 con 0x7f62a814a5f0 2023-10-08T07:10:34.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.147+0000 7f629dffb640 1 -- 172.21.15.70:0/3686593057 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629800c940 con 0x7f62a814a5f0 2023-10-08T07:10:34.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.147+0000 7f629dffb640 1 -- 172.21.15.70:0/3686593057 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629800ccd0 con 0x7f62a814a5f0 2023-10-08T07:10:34.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.147+0000 7f62af7a5640 1 -- 172.21.15.70:0/3686593057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a8148480 con 0x7f62a814a5f0 2023-10-08T07:10:34.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.147+0000 7f62af7a5640 1 -- 172.21.15.70:0/3686593057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a8148a40 con 0x7f62a814a5f0 2023-10-08T07:10:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.148+0000 7f62af7a5640 1 -- 172.21.15.70:0/3686593057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a8148f60 con 0x7f62a814a5f0 2023-10-08T07:10:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.149+0000 7f629dffb640 1 -- 172.21.15.70:0/3686593057 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6298013470 con 0x7f62a814a5f0 2023-10-08T07:10:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.150+0000 7f629dffb640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f628c06afc0 0x7f628c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.150+0000 7f629dffb640 1 -- 172.21.15.70:0/3686593057 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f6298089bc0 con 0x7f62a814a5f0 2023-10-08T07:10:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.151+0000 7f62ad541640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f628c06afc0 0x7f628c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.152+0000 7f62ad541640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f628c06afc0 0x7f628c06d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f62a4036000 tx=0x7f62a4030000 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:34.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.154+0000 7f629dffb640 1 -- 172.21.15.70:0/3686593057 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62a8148f60 con 0x7f62a814a5f0 2023-10-08T07:10:34.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.413+0000 7f62af7a5640 1 -- 172.21.15.70:0/3686593057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f62a8148f60 con 0x7f62a814a5f0 2023-10-08T07:10:34.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.548+0000 7f629dffb640 1 -- 172.21.15.70:0/3686593057 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v3090) v1 ==== 165+0+0 (secure 0 0 0) 0x7f62a8148f60 con 0x7f62a814a5f0 2023-10-08T07:10:34.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 -- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f628c06afc0 msgr2=0x7f628c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f628c06afc0 0x7f628c06d480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f62a4036000 tx=0x7f62a4030000 comp rx=0 tx=0).stop 2023-10-08T07:10:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 -- 172.21.15.70:0/3686593057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a814a5f0 msgr2=0x7f62a8147a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a814a5f0 0x7f62a8147a30 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f629800e7d0 tx=0x7f629800ec90 comp rx=0 tx=0).stop 2023-10-08T07:10:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 -- 172.21.15.70:0/3686593057 shutdown_connections 2023-10-08T07:10:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f628c06afc0 0x7f628c06d480 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62a8149240 0x7f62a80cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62a814a5f0 0x7f62a8147a30 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 --2- 172.21.15.70:0/3686593057 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6294003970 0x7f62a80b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 -- 172.21.15.70:0/3686593057 >> 172.21.15.70:0/3686593057 conn(0x7f62a806d450 msgr2=0x7f62a815abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 -- 172.21.15.70:0/3686593057 shutdown_connections 2023-10-08T07:10:34.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.552+0000 7f62af7a5640 1 -- 172.21.15.70:0/3686593057 wait complete. 2023-10-08T07:10:34.572 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-10-08T07:10:34.573 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-10-08T07:10:34.573 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T07:10:34.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:10:34.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.784+0000 7f0fb2ad0640 1 Processor -- start 2023-10-08T07:10:34.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.784+0000 7f0fb2ad0640 1 -- start start 2023-10-08T07:10:34.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.786+0000 7f0fb2ad0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0fac15ab40 0x7f0fac0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.786+0000 7f0fb2ad0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fac0c90b0 0x7f0fac0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.786+0000 7f0fb2ad0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fac0cae70 0x7f0fac0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.786+0000 7f0fb2ad0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac15bf30 con 0x7f0fac0c90b0 2023-10-08T07:10:34.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.786+0000 7f0fb2ad0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac0cb840 con 0x7f0fac15ab40 2023-10-08T07:10:34.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.786+0000 7f0fb2ad0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac15b2b0 con 0x7f0fac0cae70 2023-10-08T07:10:34.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.786+0000 7f0fabfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fac0c90b0 0x7f0fac0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.786+0000 7f0fabfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fac0c90b0 0x7f0fac0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48440/0 (socket says 172.21.15.70:48440) 2023-10-08T07:10:34.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.786+0000 7f0fabfff640 1 -- 172.21.15.70:0/3096975769 learned_addr learned my addr 172.21.15.70:0/3096975769 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:34.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.786+0000 7f0fb106d640 1 --2- 172.21.15.70:0/3096975769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fac0cae70 0x7f0fac0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.787+0000 7f0fb086c640 1 --2- 172.21.15.70:0/3096975769 >> v2:172.21.15.181:3300/0 conn(0x7f0fac15ab40 0x7f0fac0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.787+0000 7f0fabfff640 1 -- 172.21.15.70:0/3096975769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fac0cae70 msgr2=0x7f0fac0cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:34.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.787+0000 7f0fabfff640 1 --2- 172.21.15.70:0/3096975769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fac0cae70 0x7f0fac0cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.787+0000 7f0fabfff640 1 -- 172.21.15.70:0/3096975769 >> v2:172.21.15.181:3300/0 conn(0x7f0fac15ab40 msgr2=0x7f0fac0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:34.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.787+0000 7f0fabfff640 1 --2- 172.21.15.70:0/3096975769 >> v2:172.21.15.181:3300/0 conn(0x7f0fac15ab40 0x7f0fac0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.787+0000 7f0fabfff640 1 -- 172.21.15.70:0/3096975769 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac15b590 con 0x7f0fac0c90b0 2023-10-08T07:10:34.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.788+0000 7f0fabfff640 1 --2- 172.21.15.70:0/3096975769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac0c90b0 0x7f0fac0adc60 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f0fac06e440 tx=0x7f0f9802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf1a1f2d9bae6ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:34.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.788+0000 7f0fab7fe640 1 -- 172.21.15.70:0/3096975769 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98031870 con 0x7f0fac0c90b0 2023-10-08T07:10:34.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.788+0000 7f0fab7fe640 1 -- 172.21.15.70:0/3096975769 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f980319d0 con 0x7f0fac0c90b0 2023-10-08T07:10:34.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.788+0000 7f0fab7fe640 1 -- 172.21.15.70:0/3096975769 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98031d60 con 0x7f0fac0c90b0 2023-10-08T07:10:34.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.789+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/3096975769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac0c90b0 msgr2=0x7f0fac0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.789+0000 7f0fb2ad0640 1 --2- 172.21.15.70:0/3096975769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac0c90b0 0x7f0fac0adc60 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f0fac06e440 tx=0x7f0f9802b910 comp rx=0 tx=0).stop 2023-10-08T07:10:34.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.789+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/3096975769 shutdown_connections 2023-10-08T07:10:34.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.789+0000 7f0fb2ad0640 1 --2- 172.21.15.70:0/3096975769 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fac0cae70 0x7f0fac0cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.789+0000 7f0fb2ad0640 1 --2- 172.21.15.70:0/3096975769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac0c90b0 0x7f0fac0adc60 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.789+0000 7f0fb2ad0640 1 --2- 172.21.15.70:0/3096975769 >> v2:172.21.15.181:3300/0 conn(0x7f0fac15ab40 0x7f0fac0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.789+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/3096975769 >> 172.21.15.70:0/3096975769 conn(0x7f0fac06a380 msgr2=0x7f0fac0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:34.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.789+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/3096975769 shutdown_connections 2023-10-08T07:10:34.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.789+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/3096975769 wait complete. 2023-10-08T07:10:34.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.791+0000 7f0fb2ad0640 1 Processor -- start 2023-10-08T07:10:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.791+0000 7f0fb2ad0640 1 -- start start 2023-10-08T07:10:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.792+0000 7f0fb2ad0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fac0c90b0 0x7f0fac0ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.792+0000 7f0fb2ad0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fac0cae70 0x7f0fac0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.792+0000 7f0fb2ad0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac15ab40 0x7f0fac13f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.792+0000 7f0fb2ad0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac151680 con 0x7f0fac15ab40 2023-10-08T07:10:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.792+0000 7f0fb2ad0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac151350 con 0x7f0fac0cae70 2023-10-08T07:10:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.792+0000 7f0fb2ad0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0b3590 con 0x7f0fac0c90b0 2023-10-08T07:10:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.792+0000 7f0fb106d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac15ab40 0x7f0fac13f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.792+0000 7f0fb106d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac15ab40 0x7f0fac13f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48446/0 (socket says 172.21.15.70:48446) 2023-10-08T07:10:34.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.792+0000 7f0fb106d640 1 -- 172.21.15.70:0/974522193 learned_addr learned my addr 172.21.15.70:0/974522193 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:34.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fabfff640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fac0cae70 0x7f0fac0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fb086c640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fac0c90b0 0x7f0fac0ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fb106d640 1 -- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fac0c90b0 msgr2=0x7f0fac0ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fb106d640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fac0c90b0 0x7f0fac0ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fb106d640 1 -- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fac0cae70 msgr2=0x7f0fac0c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fb106d640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fac0cae70 0x7f0fac0c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fb106d640 1 -- 172.21.15.70:0/974522193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f980094d0 con 0x7f0fac15ab40 2023-10-08T07:10:34.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fabfff640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fac0cae70 0x7f0fac0c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:34.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fb106d640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac15ab40 0x7f0fac13f420 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4007d20 tx=0x7f0fa400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:34.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fa97fa640 1 -- 172.21.15.70:0/974522193 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4014420 con 0x7f0fac15ab40 2023-10-08T07:10:34.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fa97fa640 1 -- 172.21.15.70:0/974522193 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa40145a0 con 0x7f0fac15ab40 2023-10-08T07:10:34.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/974522193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fac0b2fd0 con 0x7f0fac15ab40 2023-10-08T07:10:34.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fa97fa640 1 -- 172.21.15.70:0/974522193 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4014950 con 0x7f0fac15ab40 2023-10-08T07:10:34.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.793+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/974522193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac14f370 con 0x7f0fac15ab40 2023-10-08T07:10:34.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.794+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/974522193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fac0cf6c0 con 0x7f0fac15ab40 2023-10-08T07:10:34.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.795+0000 7f0fa97fa640 1 -- 172.21.15.70:0/974522193 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0fa4014ab0 con 0x7f0fac15ab40 2023-10-08T07:10:34.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.796+0000 7f0fa97fa640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0f9006af90 0x7f0f9006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:34.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.796+0000 7f0fa97fa640 1 -- 172.21.15.70:0/974522193 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f0fa408a3d0 con 0x7f0fac15ab40 2023-10-08T07:10:34.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.797+0000 7f0fb086c640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0f9006af90 0x7f0f9006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:34.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.798+0000 7f0fb086c640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0f9006af90 0x7f0f9006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0006fd0 tx=0x7f0fa0008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:34.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.801+0000 7f0fa97fa640 1 -- 172.21.15.70:0/974522193 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0fa4055010 con 0x7f0fac15ab40 2023-10-08T07:10:34.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.990+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/974522193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fac1511b0 con 0x7f0fac15ab40 2023-10-08T07:10:34.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.992+0000 7f0fa97fa640 1 -- 172.21.15.70:0/974522193 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+14894 (secure 0 0 0) 0x7f0fac1511b0 con 0x7f0fac15ab40 2023-10-08T07:10:34.993 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:34.993 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":207,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:10:32.529971+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6833/2138719113":"2023-10-09T07:10:29.254843+0000","172.21.15.70:6832/2138719113":"2023-10-09T07:10:29.254843+0000","172.21.15.181:6832/2291259069":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/2956479135":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/4233293606":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/2946222544":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/1250722759":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/1792732560":"2023-10-09T07:10:25.468616+0000","172.21.15.181:6833/3979056493":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6832/3979056493":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/340227645":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/849801254":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/4017559811":"2023-10-09T07:05:21.112418+0000","172.21.15.181:0/217631810":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/64410277":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/811429527":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/93223744":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/772181101":"2023-10-09T06:54:47.949157+0000","172.21.15.181:6833/2291259069":"2023-10-09T07:10:25.468616+0000","172.21.15.181:0/1286852011":"2023-10-09T07:05:21.112418+0000","172.21.15.181:6832/3709692999":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/4147168289":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/437133264":"2023-10-09T07:10:25.468616+0000","172.21.15.181:6833/3709692999":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/602716182":"2023-10-09T06:54:47.949157+0000","172.21.15.181:0/1777884673":"2023-10-09T06:54:47.949157+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:10:34.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0f9006af90 msgr2=0x7f0f9006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0f9006af90 0x7f0f9006d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0006fd0 tx=0x7f0fa0008040 comp rx=0 tx=0).stop 2023-10-08T07:10:34.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/974522193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac15ab40 msgr2=0x7f0fac13f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:34.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac15ab40 0x7f0fac13f420 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4007d20 tx=0x7f0fa400e4b0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/974522193 shutdown_connections 2023-10-08T07:10:34.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f0f9006af90 0x7f0f9006d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fac15ab40 0x7f0fac13f420 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0fac0cae70 0x7f0fac0c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 --2- 172.21.15.70:0/974522193 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0fac0c90b0 0x7f0fac0ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:34.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/974522193 >> 172.21.15.70:0/974522193 conn(0x7f0fac06a380 msgr2=0x7f0fac159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:34.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/974522193 shutdown_connections 2023-10-08T07:10:34.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:34.995+0000 7f0fb2ad0640 1 -- 172.21.15.70:0/974522193 wait complete. 2023-10-08T07:10:35.014 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:10:35.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:10:35.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.221+0000 7fb9d2fa7640 1 Processor -- start 2023-10-08T07:10:35.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.221+0000 7fb9d2fa7640 1 -- start start 2023-10-08T07:10:35.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.222+0000 7fb9d2fa7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9cc0cdc30 0x7fb9cc0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.222+0000 7fb9d2fa7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb9cc0cce90 0x7fb9cc159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.222+0000 7fb9d2fa7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc15ac90 con 0x7fb9cc0caef0 2023-10-08T07:10:35.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.222+0000 7fb9d2fa7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc15a6d0 con 0x7fb9cc0cdc30 2023-10-08T07:10:35.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.222+0000 7fb9d2fa7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc15a9b0 con 0x7fb9cc0cce90 2023-10-08T07:10:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.223+0000 7fb9d17a4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9cc0cdc30 0x7fb9cc0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.223+0000 7fb9d17a4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9cc0cdc30 0x7fb9cc0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48456/0 (socket says 172.21.15.70:48456) 2023-10-08T07:10:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.223+0000 7fb9d17a4640 1 -- 172.21.15.70:0/2821442077 learned_addr learned my addr 172.21.15.70:0/2821442077 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.223+0000 7fb9d27a6640 1 --2- 172.21.15.70:0/2821442077 >> v2:172.21.15.181:3300/0 conn(0x7fb9cc0cce90 0x7fb9cc159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.223+0000 7fb9d17a4640 1 -- 172.21.15.70:0/2821442077 >> v2:172.21.15.181:3300/0 conn(0x7fb9cc0cce90 msgr2=0x7fb9cc159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.223+0000 7fb9d17a4640 1 --2- 172.21.15.70:0/2821442077 >> v2:172.21.15.181:3300/0 conn(0x7fb9cc0cce90 0x7fb9cc159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.223+0000 7fb9d17a4640 1 -- 172.21.15.70:0/2821442077 >> v1:172.21.15.181:6789/0 conn(0x7fb9cc0caef0 legacy=0x7fb9cc0cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.223+0000 7fb9d17a4640 1 -- 172.21.15.70:0/2821442077 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc0cc150 con 0x7fb9cc0cdc30 2023-10-08T07:10:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.223+0000 7fb9d27a6640 1 --2- 172.21.15.70:0/2821442077 >> v2:172.21.15.181:3300/0 conn(0x7fb9cc0cce90 0x7fb9cc159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.225+0000 7fb9d17a4640 1 --2- 172.21.15.70:0/2821442077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cdc30 0x7fb9cc0cc950 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc15d250 tx=0x7fb9bc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92480465a313976e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:35.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.225+0000 7fb9d0fa3640 1 -- 172.21.15.70:0/2821442077 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc031870 con 0x7fb9cc0cdc30 2023-10-08T07:10:35.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.225+0000 7fb9d0fa3640 1 -- 172.21.15.70:0/2821442077 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc0319d0 con 0x7fb9cc0cdc30 2023-10-08T07:10:35.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.225+0000 7fb9d0fa3640 1 -- 172.21.15.70:0/2821442077 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc031d60 con 0x7fb9cc0cdc30 2023-10-08T07:10:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.226+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2821442077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cdc30 msgr2=0x7fb9cc0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.226+0000 7fb9d2fa7640 1 --2- 172.21.15.70:0/2821442077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cdc30 0x7fb9cc0cc950 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc15d250 tx=0x7fb9bc02b910 comp rx=0 tx=0).stop 2023-10-08T07:10:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.226+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2821442077 shutdown_connections 2023-10-08T07:10:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.226+0000 7fb9d2fa7640 1 --2- 172.21.15.70:0/2821442077 >> v2:172.21.15.181:3300/0 conn(0x7fb9cc0cce90 0x7fb9cc159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.226+0000 7fb9d2fa7640 1 --2- 172.21.15.70:0/2821442077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cdc30 0x7fb9cc0cc950 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.226+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2821442077 >> 172.21.15.70:0/2821442077 conn(0x7fb9cc06c010 msgr2=0x7fb9cc06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.226+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2821442077 shutdown_connections 2023-10-08T07:10:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.226+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2821442077 wait complete. 2023-10-08T07:10:35.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.228+0000 7fb9d2fa7640 1 Processor -- start 2023-10-08T07:10:35.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.228+0000 7fb9d2fa7640 1 -- start start 2023-10-08T07:10:35.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d2fa7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9cc0caef0 0x7fb9cc1f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d2fa7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cce90 0x7fb9cc1ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d2fa7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9cc0cdc30 0x7fb9cc0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d2fa7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0afef0 con 0x7fb9cc0cce90 2023-10-08T07:10:35.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d2fa7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc1f3280 con 0x7fb9cc0cdc30 2023-10-08T07:10:35.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d2fa7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc1efcf0 con 0x7fb9cc0caef0 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d17a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cce90 0x7fb9cc1ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d17a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cce90 0x7fb9cc1ef7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48472/0 (socket says 172.21.15.70:48472) 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d17a4640 1 -- 172.21.15.70:0/2292941823 learned_addr learned my addr 172.21.15.70:0/2292941823 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d17a4640 1 -- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9cc0caef0 msgr2=0x7fb9cc1f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d1fa5640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9cc0caef0 0x7fb9cc1f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d17a4640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9cc0caef0 0x7fb9cc1f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d17a4640 1 -- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9cc0cdc30 msgr2=0x7fb9cc0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d27a6640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9cc0cdc30 0x7fb9cc0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.229+0000 7fb9d17a4640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9cc0cdc30 0x7fb9cc0b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.230+0000 7fb9d17a4640 1 -- 172.21.15.70:0/2292941823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc0094d0 con 0x7fb9cc0cce90 2023-10-08T07:10:35.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.230+0000 7fb9d17a4640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cce90 0x7fb9cc1ef7b0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc02b950 tx=0x7fb9bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:35.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.230+0000 7fb9c27fc640 1 -- 172.21.15.70:0/2292941823 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc031820 con 0x7fb9cc0cce90 2023-10-08T07:10:35.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.230+0000 7fb9c27fc640 1 -- 172.21.15.70:0/2292941823 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc0443f0 con 0x7fb9cc0cce90 2023-10-08T07:10:35.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.230+0000 7fb9c27fc640 1 -- 172.21.15.70:0/2292941823 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc0447a0 con 0x7fb9cc0cce90 2023-10-08T07:10:35.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.230+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2292941823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc0b11e0 con 0x7fb9cc0cce90 2023-10-08T07:10:35.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.230+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2292941823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc0b0420 con 0x7fb9cc0cce90 2023-10-08T07:10:35.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.231+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2292941823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc074150 con 0x7fb9cc0cce90 2023-10-08T07:10:35.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.232+0000 7fb9c27fc640 1 -- 172.21.15.70:0/2292941823 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb9bc04d450 con 0x7fb9cc0cce90 2023-10-08T07:10:35.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.233+0000 7fb9c27fc640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9b006af90 0x7fb9b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.233+0000 7fb9c27fc640 1 -- 172.21.15.70:0/2292941823 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6411+0+0 (secure 0 0 0) 0x7fb9bc02d040 con 0x7fb9cc0cce90 2023-10-08T07:10:35.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.234+0000 7fb9d1fa5640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9b006af90 0x7fb9b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:35.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.235+0000 7fb9d1fa5640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9b006af90 0x7fb9b006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc15c1e0 tx=0x7fb9c8009040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:35.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.238+0000 7fb9c27fc640 1 -- 172.21.15.70:0/2292941823 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9bc076e90 con 0x7fb9cc0cce90 2023-10-08T07:10:35.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.415+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2292941823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9cc15cb20 con 0x7fb9cc0cce90 2023-10-08T07:10:35.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.416+0000 7fb9c27fc640 1 -- 172.21.15.70:0/2292941823 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v207) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9bc07a4b0 con 0x7fb9cc0cce90 2023-10-08T07:10:35.417 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:10:35.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.418+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9b006af90 msgr2=0x7fb9b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:35.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.418+0000 7fb9d2fa7640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9b006af90 0x7fb9b006d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc15c1e0 tx=0x7fb9c8009040 comp rx=0 tx=0).stop 2023-10-08T07:10:35.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.418+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2292941823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cce90 msgr2=0x7fb9cc1ef7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.418+0000 7fb9d2fa7640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cce90 0x7fb9cc1ef7b0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc02b950 tx=0x7fb9bc0046c0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.418+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2292941823 shutdown_connections 2023-10-08T07:10:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.418+0000 7fb9d2fa7640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9b006af90 0x7fb9b006d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.418+0000 7fb9d2fa7640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9cc0cdc30 0x7fb9cc0b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.418+0000 7fb9d2fa7640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9cc0cce90 0x7fb9cc1ef7b0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.418+0000 7fb9d2fa7640 1 --2- 172.21.15.70:0/2292941823 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9cc0caef0 0x7fb9cc1f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.418+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2292941823 >> 172.21.15.70:0/2292941823 conn(0x7fb9cc06c010 msgr2=0x7fb9cc0707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.419+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2292941823 shutdown_connections 2023-10-08T07:10:35.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.419+0000 7fb9d2fa7640 1 -- 172.21.15.70:0/2292941823 wait complete. 2023-10-08T07:10:35.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-08T07:10:35.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.643+0000 7ff8a2cec640 1 Processor -- start 2023-10-08T07:10:35.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.643+0000 7ff8a2cec640 1 -- start start 2023-10-08T07:10:35.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.644+0000 7ff8a2cec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff89c15ab40 0x7ff89c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.644+0000 7ff8a2cec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89c15b2b0 con 0x7ff89c0cb580 2023-10-08T07:10:35.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.644+0000 7ff8a2cec640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89c15b590 con 0x7ff89c0c90b0 2023-10-08T07:10:35.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.644+0000 7ff8a2cec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89c15bf30 con 0x7ff89c15ab40 2023-10-08T07:10:35.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.645+0000 7ff8a0a88640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff89c15ab40 0x7ff89c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:35.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.645+0000 7ff8a1289640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff89c0cb580 0x7ff89c15e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41618/0 (socket says 172.21.15.70:41618) 2023-10-08T07:10:35.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.645+0000 7ff8a1289640 1 -- 172.21.15.70:0/2141062478 learned_addr learned my addr 172.21.15.70:0/2141062478 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:35.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.645+0000 7ff8a0a88640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff89c15ab40 0x7ff89c0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48482/0 (socket says 172.21.15.70:48482) 2023-10-08T07:10:35.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.645+0000 7ff8a0a88640 1 -- 172.21.15.70:0/2141062478 >> v1:172.21.15.181:6789/0 conn(0x7ff89c0c90b0 legacy=0x7ff89c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:35.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.645+0000 7ff8a0a88640 1 -- 172.21.15.70:0/2141062478 >> v1:172.21.15.70:6789/0 conn(0x7ff89c0cb580 legacy=0x7ff89c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:35.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.645+0000 7ff8a0a88640 1 -- 172.21.15.70:0/2141062478 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c15b870 con 0x7ff89c15ab40 2023-10-08T07:10:35.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.646+0000 7ff8a0a88640 1 --2- 172.21.15.70:0/2141062478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c15ab40 0x7ff89c0ad720 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7ff8900097a0 tx=0x7ff89002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5db19eaefc8465 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:35.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.647+0000 7ff89b7fe640 1 -- 172.21.15.70:0/2141062478 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890031830 con 0x7ff89c15ab40 2023-10-08T07:10:35.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.647+0000 7ff89b7fe640 1 -- 172.21.15.70:0/2141062478 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff890031990 con 0x7ff89c15ab40 2023-10-08T07:10:35.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.647+0000 7ff89b7fe640 1 -- 172.21.15.70:0/2141062478 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890031d20 con 0x7ff89c15ab40 2023-10-08T07:10:35.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.647+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2141062478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c15ab40 msgr2=0x7ff89c0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:35.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.647+0000 7ff8a2cec640 1 --2- 172.21.15.70:0/2141062478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c15ab40 0x7ff89c0ad720 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7ff8900097a0 tx=0x7ff89002b8d0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.647+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2141062478 shutdown_connections 2023-10-08T07:10:35.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.647+0000 7ff8a2cec640 1 --2- 172.21.15.70:0/2141062478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c15ab40 0x7ff89c0ad720 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.647+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2141062478 >> 172.21.15.70:0/2141062478 conn(0x7ff89c06a380 msgr2=0x7ff89c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:35.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.647+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2141062478 shutdown_connections 2023-10-08T07:10:35.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.648+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2141062478 wait complete. 2023-10-08T07:10:35.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.649+0000 7ff8a2cec640 1 Processor -- start 2023-10-08T07:10:35.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.649+0000 7ff8a2cec640 1 -- start start 2023-10-08T07:10:35.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.650+0000 7ff8a2cec640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89c0c90b0 0x7ff89c0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.650+0000 7ff8a2cec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c0cb580 0x7ff89c155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.650+0000 7ff8a2cec640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff89c15ab40 0x7ff89c155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.650+0000 7ff8a2cec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0c5e80 con 0x7ff89c0cb580 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.650+0000 7ff8a2cec640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0c6ed0 con 0x7ff89c0c90b0 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.650+0000 7ff8a2cec640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c15a290 con 0x7ff89c15ab40 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.651+0000 7ff89bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c0cb580 0x7ff89c155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.651+0000 7ff89bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c0cb580 0x7ff89c155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48494/0 (socket says 172.21.15.70:48494) 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.651+0000 7ff89bfff640 1 -- 172.21.15.70:0/2710275285 learned_addr learned my addr 172.21.15.70:0/2710275285 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.651+0000 7ff89bfff640 1 -- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff89c15ab40 msgr2=0x7ff89c155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.651+0000 7ff89bfff640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff89c15ab40 0x7ff89c155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.651+0000 7ff89bfff640 1 -- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89c0c90b0 msgr2=0x7ff89c0accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.651+0000 7ff8a0a88640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89c0c90b0 0x7ff89c0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.651+0000 7ff89bfff640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89c0c90b0 0x7ff89c0accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:35.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.651+0000 7ff89bfff640 1 -- 172.21.15.70:0/2710275285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff890009400 con 0x7ff89c0cb580 2023-10-08T07:10:35.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.651+0000 7ff89bfff640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c0cb580 0x7ff89c155d30 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ff88800c400 tx=0x7ff88800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:35.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.652+0000 7ff8997fa640 1 -- 172.21.15.70:0/2710275285 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880035f0 con 0x7ff89c0cb580 2023-10-08T07:10:35.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.652+0000 7ff8997fa640 1 -- 172.21.15.70:0/2710275285 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff888003750 con 0x7ff89c0cb580 2023-10-08T07:10:35.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.652+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2710275285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89c0c5850 con 0x7ff89c0cb580 2023-10-08T07:10:35.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.652+0000 7ff8997fa640 1 -- 172.21.15.70:0/2710275285 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888003b40 con 0x7ff89c0cb580 2023-10-08T07:10:35.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.652+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2710275285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89c0adee0 con 0x7ff89c0cb580 2023-10-08T07:10:35.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.653+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2710275285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89c0c5fc0 con 0x7ff89c0cb580 2023-10-08T07:10:35.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.654+0000 7ff8997fa640 1 -- 172.21.15.70:0/2710275285 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff888003ca0 con 0x7ff89c0cb580 2023-10-08T07:10:35.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.655+0000 7ff8997fa640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7ff88006af90 0x7ff88006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:35.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.656+0000 7ff8997fa640 1 -- 172.21.15.70:0/2710275285 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6411+0+0 (secure 0 0 0) 0x7ff888088ea0 con 0x7ff89c0cb580 2023-10-08T07:10:35.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.656+0000 7ff8a0a88640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7ff88006af90 0x7ff88006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:35.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.657+0000 7ff8a0a88640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7ff88006af90 0x7ff88006d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff890036000 tx=0x7ff89003c040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:35.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.660+0000 7ff8997fa640 1 -- 172.21.15.70:0/2710275285 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff89c0c5fc0 con 0x7ff89c0cb580 2023-10-08T07:10:35.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:35.918+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2710275285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7ff89c0c5fc0 con 0x7ff89c0cb580 2023-10-08T07:10:36.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.563+0000 7ff8997fa640 1 -- 172.21.15.70:0/2710275285 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3092) v1 ==== 166+0+0 (secure 0 0 0) 0x7ff89c0c5fc0 con 0x7ff89c0cb580 2023-10-08T07:10:36.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.566+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7ff88006af90 msgr2=0x7ff88006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.566+0000 7ff8a2cec640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7ff88006af90 0x7ff88006d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff890036000 tx=0x7ff89003c040 comp rx=0 tx=0).stop 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.567+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2710275285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c0cb580 msgr2=0x7ff89c155d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.567+0000 7ff8a2cec640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c0cb580 0x7ff89c155d30 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7ff88800c400 tx=0x7ff88800c8c0 comp rx=0 tx=0).stop 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.567+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2710275285 shutdown_connections 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.567+0000 7ff8a2cec640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7ff88006af90 0x7ff88006d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.567+0000 7ff8a2cec640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff89c15ab40 0x7ff89c155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.567+0000 7ff8a2cec640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff89c0cb580 0x7ff89c155d30 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.567+0000 7ff8a2cec640 1 --2- 172.21.15.70:0/2710275285 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff89c0c90b0 0x7ff89c0accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.567+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2710275285 >> 172.21.15.70:0/2710275285 conn(0x7ff89c06a380 msgr2=0x7ff89c0aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.567+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2710275285 shutdown_connections 2023-10-08T07:10:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.567+0000 7ff8a2cec640 1 -- 172.21.15.70:0/2710275285 wait complete. 2023-10-08T07:10:36.588 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T07:10:36.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:10:36.726 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:10:36.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.791+0000 7f320b59e640 1 Processor -- start 2023-10-08T07:10:36.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.791+0000 7f320b59e640 1 -- start start 2023-10-08T07:10:36.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.792+0000 7f320b59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f320c0cf2b0 0x7f320c0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:36.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.792+0000 7f320b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f320c0cf690 0x7f320c0ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:36.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.792+0000 7f320b59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f320c0cd520 con 0x7f320c0d1100 2023-10-08T07:10:36.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.792+0000 7f320b59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320c0d1910 con 0x7f320c0cf690 2023-10-08T07:10:36.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.792+0000 7f320b59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320c0ce080 con 0x7f320c0cf2b0 2023-10-08T07:10:36.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.793+0000 7f3203fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f320c0cf690 0x7f320c0ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:36.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.793+0000 7f3203fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f320c0cf690 0x7f320c0ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48502/0 (socket says 172.21.15.70:48502) 2023-10-08T07:10:36.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.793+0000 7f3203fff640 1 -- 172.21.15.70:0/2718313837 learned_addr learned my addr 172.21.15.70:0/2718313837 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:36.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.793+0000 7f320a59c640 1 --2- 172.21.15.70:0/2718313837 >> v2:172.21.15.181:3300/0 conn(0x7f320c0cf2b0 0x7f320c0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.793+0000 7f3203fff640 1 -- 172.21.15.70:0/2718313837 >> v2:172.21.15.181:3300/0 conn(0x7f320c0cf2b0 msgr2=0x7f320c0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.793+0000 7f3203fff640 1 --2- 172.21.15.70:0/2718313837 >> v2:172.21.15.181:3300/0 conn(0x7f320c0cf2b0 0x7f320c0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.793+0000 7f3203fff640 1 -- 172.21.15.70:0/2718313837 >> v1:172.21.15.181:6789/0 conn(0x7f320c0d1100 legacy=0x7f320c15c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.793+0000 7f3203fff640 1 -- 172.21.15.70:0/2718313837 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c0cd750 con 0x7f320c0cf690 2023-10-08T07:10:36.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.795+0000 7f3203fff640 1 --2- 172.21.15.70:0/2718313837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0cf690 0x7f320c0ce510 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f320c0b54b0 tx=0x7f31f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b13603e3ddc382e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.795+0000 7f3209d9b640 1 -- 172.21.15.70:0/2718313837 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4031870 con 0x7f320c0cf690 2023-10-08T07:10:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.795+0000 7f3209d9b640 1 -- 172.21.15.70:0/2718313837 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f40319d0 con 0x7f320c0cf690 2023-10-08T07:10:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.795+0000 7f3209d9b640 1 -- 172.21.15.70:0/2718313837 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4031d60 con 0x7f320c0cf690 2023-10-08T07:10:36.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.796+0000 7f320b59e640 1 -- 172.21.15.70:0/2718313837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0cf690 msgr2=0x7f320c0ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.796+0000 7f320b59e640 1 --2- 172.21.15.70:0/2718313837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0cf690 0x7f320c0ce510 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f320c0b54b0 tx=0x7f31f402b910 comp rx=0 tx=0).stop 2023-10-08T07:10:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.796+0000 7f320b59e640 1 -- 172.21.15.70:0/2718313837 shutdown_connections 2023-10-08T07:10:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.796+0000 7f320b59e640 1 --2- 172.21.15.70:0/2718313837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0cf690 0x7f320c0ce510 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.796+0000 7f320b59e640 1 --2- 172.21.15.70:0/2718313837 >> v2:172.21.15.181:3300/0 conn(0x7f320c0cf2b0 0x7f320c0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.796+0000 7f320b59e640 1 -- 172.21.15.70:0/2718313837 >> 172.21.15.70:0/2718313837 conn(0x7f320c09bc40 msgr2=0x7f320c0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.796+0000 7f320b59e640 1 -- 172.21.15.70:0/2718313837 shutdown_connections 2023-10-08T07:10:36.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.796+0000 7f320b59e640 1 -- 172.21.15.70:0/2718313837 wait complete. 2023-10-08T07:10:36.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.797+0000 7f320b59e640 1 Processor -- start 2023-10-08T07:10:36.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.798+0000 7f320b59e640 1 -- start start 2023-10-08T07:10:36.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.798+0000 7f320b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f320c0cf2b0 0x7f320c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.798+0000 7f320b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f320c0cf690 0x7f320c0b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0d1100 0x7f320c0b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c151a50 con 0x7f320c0d1100 2023-10-08T07:10:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320b59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1f22a0 con 0x7f320c0cf2b0 2023-10-08T07:10:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320c150dd0 con 0x7f320c0cf690 2023-10-08T07:10:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0d1100 0x7f320c0b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0d1100 0x7f320c0b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48506/0 (socket says 172.21.15.70:48506) 2023-10-08T07:10:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320a59c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f320c0cf2b0 0x7f320c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:36.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320ad9d640 1 -- 172.21.15.70:0/643219039 learned_addr learned my addr 172.21.15.70:0/643219039 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:36.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f3203fff640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f320c0cf690 0x7f320c0b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:36.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320ad9d640 1 -- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f320c0cf690 msgr2=0x7f320c0b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:36.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320ad9d640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f320c0cf690 0x7f320c0b7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:36.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320ad9d640 1 -- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f320c0cf2b0 msgr2=0x7f320c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:36.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320ad9d640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f320c0cf2b0 0x7f320c0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:36.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320ad9d640 1 -- 172.21.15.70:0/643219039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f40094d0 con 0x7f320c0d1100 2023-10-08T07:10:36.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.799+0000 7f320a59c640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f320c0cf2b0 0x7f320c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:10:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.800+0000 7f320ad9d640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0d1100 0x7f320c0b6da0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f320400c7a0 tx=0x7f320400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.800+0000 7f3202ffd640 1 -- 172.21.15.70:0/643219039 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204003b70 con 0x7f320c0d1100 2023-10-08T07:10:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.800+0000 7f3202ffd640 1 -- 172.21.15.70:0/643219039 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3204003cf0 con 0x7f320c0d1100 2023-10-08T07:10:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.800+0000 7f3202ffd640 1 -- 172.21.15.70:0/643219039 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204003ea0 con 0x7f320c0d1100 2023-10-08T07:10:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.800+0000 7f320b59e640 1 -- 172.21.15.70:0/643219039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c1510b0 con 0x7f320c0d1100 2023-10-08T07:10:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.800+0000 7f320b59e640 1 -- 172.21.15.70:0/643219039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c151670 con 0x7f320c0d1100 2023-10-08T07:10:36.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.801+0000 7f320b59e640 1 -- 172.21.15.70:0/643219039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c074150 con 0x7f320c0d1100 2023-10-08T07:10:36.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.802+0000 7f3202ffd640 1 -- 172.21.15.70:0/643219039 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f320400aa90 con 0x7f320c0d1100 2023-10-08T07:10:36.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.803+0000 7f3202ffd640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f31ec06af90 0x7f31ec06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:36.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.803+0000 7f3202ffd640 1 -- 172.21.15.70:0/643219039 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f3204089690 con 0x7f320c0d1100 2023-10-08T07:10:36.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.803+0000 7f320a59c640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f31ec06af90 0x7f31ec06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:36.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.804+0000 7f320a59c640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f31ec06af90 0x7f31ec06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009e90 tx=0x7f31fc009190 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:36.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.807+0000 7f3202ffd640 1 -- 172.21.15.70:0/643219039 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32040544b0 con 0x7f320c0d1100 2023-10-08T07:10:36.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:36.995+0000 7f320b59e640 1 -- 172.21.15.70:0/643219039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f320c0d0070 con 0x7f320c0d1100 2023-10-08T07:10:37.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.002+0000 7f3202ffd640 1 -- 172.21.15.70:0/643219039 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3204057ad0 con 0x7f320c0d1100 2023-10-08T07:10:37.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 -- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f31ec06af90 msgr2=0x7f31ec06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:37.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f31ec06af90 0x7f31ec06d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f31fc009e90 tx=0x7f31fc009190 comp rx=0 tx=0).stop 2023-10-08T07:10:37.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 -- 172.21.15.70:0/643219039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0d1100 msgr2=0x7f320c0b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:37.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0d1100 0x7f320c0b6da0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f320400c7a0 tx=0x7f320400cc60 comp rx=0 tx=0).stop 2023-10-08T07:10:37.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 -- 172.21.15.70:0/643219039 shutdown_connections 2023-10-08T07:10:37.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f31ec06af90 0x7f31ec06d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f320c0d1100 0x7f320c0b6da0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f320c0cf690 0x7f320c0b7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 --2- 172.21.15.70:0/643219039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f320c0cf2b0 0x7f320c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 -- 172.21.15.70:0/643219039 >> 172.21.15.70:0/643219039 conn(0x7f320c09bc40 msgr2=0x7f320c0b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:37.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 -- 172.21.15.70:0/643219039 shutdown_connections 2023-10-08T07:10:37.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.005+0000 7f320b59e640 1 -- 172.21.15.70:0/643219039 wait complete. 2023-10-08T07:10:37.025 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T07:10:37.025 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:10:37.150 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:10:37.271 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T07:10:37.272 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:10:37.272 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:10:37.334 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:10:37.335 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:10:37.336 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:10:37.396 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:10:37.396 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:10:37.396 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:10:37.454 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:10:37.455 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:10:37.455 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:10:37.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:10:37.515 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T07:10:37.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:10:37.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.736+0000 7f7fcbef2640 1 Processor -- start 2023-10-08T07:10:37.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.736+0000 7f7fcbef2640 1 -- start start 2023-10-08T07:10:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.738+0000 7f7fcbef2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fc40add60 0x7f7fc4162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.738+0000 7f7fcbef2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc40ae140 con 0x7f7fc406d4c0 2023-10-08T07:10:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.738+0000 7f7fcbef2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc40ccf20 con 0x7f7fc40cb150 2023-10-08T07:10:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.738+0000 7f7fcbef2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc4161f10 con 0x7f7fc40add60 2023-10-08T07:10:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.738+0000 7f7fc948d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7fc406d4c0 0x7f7fc40cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41620/0 (socket says 172.21.15.70:41620) 2023-10-08T07:10:37.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc948d640 1 -- 172.21.15.70:0/4180089985 learned_addr learned my addr 172.21.15.70:0/4180089985 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:37.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc9c8e640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fc40add60 0x7f7fc4162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:37.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136527938 0 0) 0x7f7fc40ae140 con 0x7f7fc406d4c0 2023-10-08T07:10:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fb4003520 con 0x7f7fc406d4c0 2023-10-08T07:10:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3112992775 0 0) 0x7f7fb4003520 con 0x7f7fc406d4c0 2023-10-08T07:10:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fc40add60 msgr2=0x7f7fc4162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fc40add60 0x7f7fc4162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 >> v1:172.21.15.181:6789/0 conn(0x7f7fc40cb150 legacy=0x7f7fc4164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40cd1c0 con 0x7f7fc406d4c0 2023-10-08T07:10:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7fb0001670 con 0x7f7fc406d4c0 2023-10-08T07:10:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 >> v1:172.21.15.70:6789/0 conn(0x7f7fc406d4c0 legacy=0x7f7fc40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:37.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb40039a0 0x7f7fb4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fc40add60 0x7f7fb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb4004d00 0x7f7fb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc4161f10 con 0x7f7fb40039a0 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb400bc00 con 0x7f7fb4004d00 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.739+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb400bdc0 con 0x7f7fc40add60 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.740+0000 7f7fc9c8e640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb40039a0 0x7f7fb4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.740+0000 7f7fca48f640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb4004d00 0x7f7fb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.740+0000 7f7fc948d640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fc40add60 0x7f7fb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.740+0000 7f7fc9c8e640 1 -- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fc40add60 msgr2=0x7f7fb40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.740+0000 7f7fc9c8e640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fc40add60 0x7f7fb40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.740+0000 7f7fc9c8e640 1 -- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb4004d00 msgr2=0x7f7fb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:37.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.740+0000 7f7fc9c8e640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fb4004d00 0x7f7fb4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.740+0000 7f7fc9c8e640 1 -- 172.21.15.70:0/4180089985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4003520 con 0x7f7fb40039a0 2023-10-08T07:10:37.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.740+0000 7f7fc948d640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fc40add60 0x7f7fb40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:37.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.741+0000 7f7fc9c8e640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb40039a0 0x7f7fb4003dc0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8007360 tx=0x7f7fb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec054310eb07baa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:37.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.742+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb802ccb0 con 0x7f7fb40039a0 2023-10-08T07:10:37.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.742+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb802ce10 con 0x7f7fb40039a0 2023-10-08T07:10:37.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.742+0000 7f7fc8c8c640 1 -- 172.21.15.70:0/4180089985 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8034660 con 0x7f7fb40039a0 2023-10-08T07:10:37.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.742+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4180089985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb40039a0 msgr2=0x7f7fb4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.742+0000 7f7fcbef2640 1 --2- 172.21.15.70:0/4180089985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fb40039a0 0x7f7fb4003dc0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8007360 tx=0x7f7fb40057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.742+0000 7f7fca48f640 1 -- 172.21.15.70:0/4180089985 reap_dead start 2023-10-08T07:10:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.742+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4180089985 shutdown_connections 2023-10-08T07:10:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.742+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4180089985 >> 172.21.15.70:0/4180089985 conn(0x7f7fc406a380 msgr2=0x7f7fc40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.742+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4180089985 shutdown_connections 2023-10-08T07:10:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.742+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4180089985 wait complete. 2023-10-08T07:10:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.744+0000 7f7fcbef2640 1 Processor -- start 2023-10-08T07:10:37.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.744+0000 7f7fcbef2640 1 -- start start 2023-10-08T07:10:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.745+0000 7f7fcbef2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fb40039a0 0x7f7fc40af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.745+0000 7f7fcbef2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4148340 0x7f7fc4156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.745+0000 7f7fcbef2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc40b4060 0x7f7fc4147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.745+0000 7f7fcbef2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc41597d0 con 0x7f7fc4148340 2023-10-08T07:10:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.745+0000 7f7fcbef2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc4159210 con 0x7f7fc40b4060 2023-10-08T07:10:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.745+0000 7f7fcbef2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc41594f0 con 0x7f7fb40039a0 2023-10-08T07:10:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.745+0000 7f7fc948d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4148340 0x7f7fc4156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.745+0000 7f7fc948d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4148340 0x7f7fc4156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48520/0 (socket says 172.21.15.70:48520) 2023-10-08T07:10:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.745+0000 7f7fc948d640 1 -- 172.21.15.70:0/4027206224 learned_addr learned my addr 172.21.15.70:0/4027206224 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.746+0000 7f7fc948d640 1 -- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fb40039a0 msgr2=0x7f7fc40af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.746+0000 7f7fc948d640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fb40039a0 0x7f7fc40af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.746+0000 7f7fc948d640 1 -- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc40b4060 msgr2=0x7f7fc4147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.746+0000 7f7fc948d640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc40b4060 0x7f7fc4147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.746+0000 7f7fc948d640 1 -- 172.21.15.70:0/4027206224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc40aee30 con 0x7f7fc4148340 2023-10-08T07:10:37.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.746+0000 7f7fc948d640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4148340 0x7f7fc4156470 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000d860 tx=0x7f7fb000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:37.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.746+0000 7f7fc27fc640 1 -- 172.21.15.70:0/4027206224 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb00074e0 con 0x7f7fc4148340 2023-10-08T07:10:37.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.746+0000 7f7fc27fc640 1 -- 172.21.15.70:0/4027206224 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb0007640 con 0x7f7fc4148340 2023-10-08T07:10:37.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.746+0000 7f7fc27fc640 1 -- 172.21.15.70:0/4027206224 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb00049e0 con 0x7f7fc4148340 2023-10-08T07:10:37.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.746+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4027206224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc40af110 con 0x7f7fc4148340 2023-10-08T07:10:37.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.747+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4027206224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc41569b0 con 0x7f7fc4148340 2023-10-08T07:10:37.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.748+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4027206224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc40af2a0 con 0x7f7fc4148340 2023-10-08T07:10:37.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.749+0000 7f7fc27fc640 1 -- 172.21.15.70:0/4027206224 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7fb0004b40 con 0x7f7fc4148340 2023-10-08T07:10:37.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.750+0000 7f7fc27fc640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f7fa806afc0 0x7f7fa806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:37.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.750+0000 7f7fc27fc640 1 -- 172.21.15.70:0/4027206224 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f7fb00895e0 con 0x7f7fc4148340 2023-10-08T07:10:37.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.750+0000 7f7fc9c8e640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f7fa806afc0 0x7f7fa806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:37.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.751+0000 7f7fc9c8e640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f7fa806afc0 0x7f7fa806d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8036000 tx=0x7f7fb802e000 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:37.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.754+0000 7f7fc27fc640 1 -- 172.21.15.70:0/4027206224 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7fb0054110 con 0x7f7fc4148340 2023-10-08T07:10:37.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.983+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4027206224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fc4067490 con 0x7f7fc4148340 2023-10-08T07:10:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.984+0000 7f7fc27fc640 1 -- 172.21.15.70:0/4027206224 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3700 (secure 0 0 0) 0x7f7fb0019e90 con 0x7f7fc4148340 2023-10-08T07:10:37.984 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:37.985 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6926,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/381523318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":381523318},{"type":"v1","addr":"172.21.15.181:6835","nonce":381523318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6929,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2153071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":2153071509},{"type":"v1","addr":"172.21.15.181:6837","nonce":2153071509}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":7765,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2673497542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2673497542},{"type":"v1","addr":"172.21.15.70:6835","nonce":2673497542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":10527,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/152299924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":152299924},{"type":"v1","addr":"172.21.15.70:6833","nonce":152299924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93}],"filesystems":[]} 2023-10-08T07:10:37.985 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 93 2023-10-08T07:10:37.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f7fa806afc0 msgr2=0x7f7fa806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f7fa806afc0 0x7f7fa806d480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8036000 tx=0x7f7fb802e000 comp rx=0 tx=0).stop 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4027206224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4148340 msgr2=0x7f7fc4156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4148340 0x7f7fc4156470 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000d860 tx=0x7f7fb000dd20 comp rx=0 tx=0).stop 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4027206224 shutdown_connections 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f7fa806afc0 0x7f7fa806d480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7fc40b4060 0x7f7fc4147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fc4148340 0x7f7fc4156470 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 --2- 172.21.15.70:0/4027206224 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7fb40039a0 0x7f7fc40af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4027206224 >> 172.21.15.70:0/4027206224 conn(0x7f7fc406a380 msgr2=0x7f7fc41626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4027206224 shutdown_connections 2023-10-08T07:10:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:37.987+0000 7f7fcbef2640 1 -- 172.21.15.70:0/4027206224 wait complete. 2023-10-08T07:10:38.008 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T07:10:38.008 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T07:10:38.009 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T07:10:38.009 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:10:38.009 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T07:10:38.009 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T07:10:38.009 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T07:10:38.009 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:10:38.010 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T07:10:38.010 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T07:10:38.010 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T07:10:38.010 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:10:38.010 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T07:10:38.010 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T07:10:38.011 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T07:10:38.011 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:10:38.030 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T07:10:38.030 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T07:10:38.031 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T07:10:38.031 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T07:10:38.053 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T07:10:38.054 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T07:10:38.055 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T07:10:38.055 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T07:10:38.079 INFO:tasks.ceph.mds.a:Started 2023-10-08T07:10:38.079 DEBUG:teuthology.parallel:result is None 2023-10-08T07:10:38.086 INFO:tasks.ceph.mds.c:Started 2023-10-08T07:10:38.086 DEBUG:teuthology.parallel:result is None 2023-10-08T07:10:38.103 INFO:tasks.ceph.mds.d:Started 2023-10-08T07:10:38.104 DEBUG:teuthology.parallel:result is None 2023-10-08T07:10:38.111 INFO:tasks.ceph.mds.b:Started 2023-10-08T07:10:38.111 DEBUG:teuthology.parallel:result is None 2023-10-08T07:10:38.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T07:10:38.241 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T07:10:38.245 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T07:10:38.269 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T07:10:38.278 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T07:10:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.300+0000 7f8296c25640 1 Processor -- start 2023-10-08T07:10:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.300+0000 7f8296c25640 1 -- start start 2023-10-08T07:10:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.301+0000 7f8296c25640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82900d00e0 0x7f82900d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.301+0000 7f8296c25640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f82900d0b90 0x7f82900b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.301+0000 7f8296c25640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82900d2870 con 0x7f829015ccb0 2023-10-08T07:10:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.301+0000 7f8296c25640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82900d1150 con 0x7f82900d00e0 2023-10-08T07:10:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.301+0000 7f8296c25640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82900d1430 con 0x7f82900d0b90 2023-10-08T07:10:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.301+0000 7f8295422640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82900d00e0 0x7f82900d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.301+0000 7f8295c23640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f829015ccb0 0x7f82900cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41646/0 (socket says 172.21.15.70:41646) 2023-10-08T07:10:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.301+0000 7f8295c23640 1 -- 172.21.15.70:0/921302174 learned_addr learned my addr 172.21.15.70:0/921302174 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.302+0000 7f8296424640 1 --2- 172.21.15.70:0/921302174 >> v2:172.21.15.181:3300/0 conn(0x7f82900d0b90 0x7f82900b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.302+0000 7f8295422640 1 -- 172.21.15.70:0/921302174 >> v2:172.21.15.181:3300/0 conn(0x7f82900d0b90 msgr2=0x7f82900b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.302+0000 7f8295422640 1 --2- 172.21.15.70:0/921302174 >> v2:172.21.15.181:3300/0 conn(0x7f82900d0b90 0x7f82900b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.302+0000 7f8295422640 1 -- 172.21.15.70:0/921302174 >> v1:172.21.15.70:6789/0 conn(0x7f829015ccb0 legacy=0x7f82900cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.302+0000 7f8295422640 1 -- 172.21.15.70:0/921302174 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82900d1710 con 0x7f82900d00e0 2023-10-08T07:10:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.302+0000 7f8294c21640 1 -- 172.21.15.70:0/921302174 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20630621 0 0) 0x7f82900d2870 con 0x7f829015ccb0 2023-10-08T07:10:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.303+0000 7f8295422640 1 --2- 172.21.15.70:0/921302174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 0x7f82900d1fb0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f82900d2520 tx=0x7f828002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d11c161dd215bb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.303+0000 7f8294c21640 1 -- 172.21.15.70:0/921302174 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280031830 con 0x7f82900d00e0 2023-10-08T07:10:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.303+0000 7f8294c21640 1 -- 172.21.15.70:0/921302174 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8280031990 con 0x7f82900d00e0 2023-10-08T07:10:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.303+0000 7f8294c21640 1 -- 172.21.15.70:0/921302174 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280031d20 con 0x7f82900d00e0 2023-10-08T07:10:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.304+0000 7f8296c25640 1 -- 172.21.15.70:0/921302174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 msgr2=0x7f82900d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.304+0000 7f8296c25640 1 --2- 172.21.15.70:0/921302174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 0x7f82900d1fb0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f82900d2520 tx=0x7f828002b8d0 comp rx=0 tx=0).stop 2023-10-08T07:10:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.304+0000 7f8296c25640 1 -- 172.21.15.70:0/921302174 shutdown_connections 2023-10-08T07:10:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.304+0000 7f8296c25640 1 --2- 172.21.15.70:0/921302174 >> v2:172.21.15.181:3300/0 conn(0x7f82900d0b90 0x7f82900b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.304+0000 7f8296c25640 1 --2- 172.21.15.70:0/921302174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 0x7f82900d1fb0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.304+0000 7f8296c25640 1 -- 172.21.15.70:0/921302174 >> 172.21.15.70:0/921302174 conn(0x7f8290071660 msgr2=0x7f8290071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.304+0000 7f8296c25640 1 -- 172.21.15.70:0/921302174 shutdown_connections 2023-10-08T07:10:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.304+0000 7f8296c25640 1 -- 172.21.15.70:0/921302174 wait complete. 2023-10-08T07:10:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.306+0000 7f8296c25640 1 Processor -- start 2023-10-08T07:10:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.306+0000 7f8296c25640 1 -- start start 2023-10-08T07:10:38.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8296c25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 0x7f82900ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:38.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8296c25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82900d0b90 0x7f829015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:38.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8296c25640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829015ccb0 0x7f82901587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8296c25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82901f5670 con 0x7f82900d00e0 2023-10-08T07:10:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8296c25640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82900cb670 con 0x7f829015ccb0 2023-10-08T07:10:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8296c25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829015b9c0 con 0x7f82900d0b90 2023-10-08T07:10:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8295c23640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 0x7f82900ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8295c23640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 0x7f82900ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48578/0 (socket says 172.21.15.70:48578) 2023-10-08T07:10:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8295c23640 1 -- 172.21.15.70:0/2724403515 learned_addr learned my addr 172.21.15.70:0/2724403515 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8295422640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82900d0b90 0x7f829015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.307+0000 7f8296424640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829015ccb0 0x7f82901587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:38.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f8295c23640 1 -- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82900d0b90 msgr2=0x7f829015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f8295c23640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82900d0b90 0x7f829015ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f8295c23640 1 -- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829015ccb0 msgr2=0x7f82901587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f8295c23640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829015ccb0 0x7f82901587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f8295c23640 1 -- 172.21.15.70:0/2724403515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8280009400 con 0x7f82900d00e0 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f8296424640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829015ccb0 0x7f82901587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f8295c23640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 0x7f82900ca360 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c460 tx=0x7f828c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f82867fc640 1 -- 172.21.15.70:0/2724403515 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c0074e0 con 0x7f82900d00e0 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f82867fc640 1 -- 172.21.15.70:0/2724403515 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c007660 con 0x7f82900d00e0 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f82867fc640 1 -- 172.21.15.70:0/2724403515 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c004a20 con 0x7f82900d00e0 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f8296c25640 1 -- 172.21.15.70:0/2724403515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82900ca9f0 con 0x7f82900d00e0 2023-10-08T07:10:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.308+0000 7f8296c25640 1 -- 172.21.15.70:0/2724403515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82900cafb0 con 0x7f82900d00e0 2023-10-08T07:10:38.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.310+0000 7f8296c25640 1 -- 172.21.15.70:0/2724403515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82901f34d0 con 0x7f82900d00e0 2023-10-08T07:10:38.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.310+0000 7f82867fc640 1 -- 172.21.15.70:0/2724403515 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f828c004b80 con 0x7f82900d00e0 2023-10-08T07:10:38.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.311+0000 7f82867fc640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f827406af90 0x7f827406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:38.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.312+0000 7f82867fc640 1 -- 172.21.15.70:0/2724403515 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f828c088bd0 con 0x7f82900d00e0 2023-10-08T07:10:38.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.312+0000 7f8295422640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f827406af90 0x7f827406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:38.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.313+0000 7f8295422640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f827406af90 0x7f827406d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8280036000 tx=0x7f8280008030 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:38.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.316+0000 7f82867fc640 1 -- 172.21.15.70:0/2724403515 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f828c053940 con 0x7f82900d00e0 2023-10-08T07:10:38.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:38.513+0000 7f8296c25640 1 -- 172.21.15.70:0/2724403515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8290074150 con 0x7f82900d00e0 2023-10-08T07:10:39.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.602+0000 7f82867fc640 1 -- 172.21.15.70:0/2724403515 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v212) v1 ==== 97+0+0 (secure 0 0 0) 0x7f828c018070 con 0x7f82900d00e0 2023-10-08T07:10:39.604 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T07:10:39.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.607+0000 7f8296c25640 1 -- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f827406af90 msgr2=0x7f827406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:39.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.607+0000 7f8296c25640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f827406af90 0x7f827406d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8280036000 tx=0x7f8280008030 comp rx=0 tx=0).stop 2023-10-08T07:10:39.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.607+0000 7f8296c25640 1 -- 172.21.15.70:0/2724403515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 msgr2=0x7f82900ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:39.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.607+0000 7f8296c25640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 0x7f82900ca360 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c460 tx=0x7f828c00c920 comp rx=0 tx=0).stop 2023-10-08T07:10:39.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.607+0000 7f8296c25640 1 -- 172.21.15.70:0/2724403515 shutdown_connections 2023-10-08T07:10:39.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.607+0000 7f8296c25640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f827406af90 0x7f827406d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:39.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.607+0000 7f8296c25640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f829015ccb0 0x7f82901587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:39.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.607+0000 7f8296c25640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82900d0b90 0x7f829015ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:39.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.607+0000 7f8296c25640 1 --2- 172.21.15.70:0/2724403515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82900d00e0 0x7f82900ca360 secure :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c460 tx=0x7f828c00c920 comp rx=0 tx=0).stop 2023-10-08T07:10:39.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.607+0000 7f8296c25640 1 -- 172.21.15.70:0/2724403515 >> 172.21.15.70:0/2724403515 conn(0x7f8290071660 msgr2=0x7f8290073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:39.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.608+0000 7f8296c25640 1 -- 172.21.15.70:0/2724403515 shutdown_connections 2023-10-08T07:10:39.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.608+0000 7f8296c25640 1 -- 172.21.15.70:0/2724403515 wait complete. 2023-10-08T07:10:39.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T07:10:39.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.851+0000 7fa9ae390640 1 Processor -- start 2023-10-08T07:10:39.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.851+0000 7fa9ae390640 1 -- start start 2023-10-08T07:10:39.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.853+0000 7fa9ae390640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa9a8160940 0x7fa9a80709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:39.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.853+0000 7fa9ae390640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a80c48f0 con 0x7fa9a806c890 2023-10-08T07:10:39.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.853+0000 7fa9ae390640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a80c3ce0 con 0x7fa9a815cd90 2023-10-08T07:10:39.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.853+0000 7fa9ae390640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a80c4d80 con 0x7fa9a8160940 2023-10-08T07:10:39.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.853+0000 7fa9ac92d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa9a806c890 0x7fa9a806cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41654/0 (socket says 172.21.15.70:41654) 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.853+0000 7fa9ac92d640 1 -- 172.21.15.70:0/1340762744 learned_addr learned my addr 172.21.15.70:0/1340762744 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/1340762744 >> v2:172.21.15.181:3300/0 conn(0x7fa9a8160940 0x7fa9a80709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720457748 0 0) 0x7fa9a80c48f0 con 0x7fa9a806c890 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa994003520 con 0x7fa9a806c890 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1921639393 0 0) 0x7fa994003520 con 0x7fa9a806c890 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 >> v2:172.21.15.181:3300/0 conn(0x7fa9a8160940 msgr2=0x7fa9a80709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 --2- 172.21.15.70:0/1340762744 >> v2:172.21.15.181:3300/0 conn(0x7fa9a8160940 0x7fa9a80709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 >> v1:172.21.15.181:6789/0 conn(0x7fa9a815cd90 legacy=0x7fa9a80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a80c3fc0 con 0x7fa9a806c890 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa9a0001670 con 0x7fa9a806c890 2023-10-08T07:10:39.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 >> v1:172.21.15.70:6789/0 conn(0x7fa9a806c890 legacy=0x7fa9a806cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 --2- 172.21.15.70:0/1340762744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa994003970 0x7fa994003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 --2- 172.21.15.70:0/1340762744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940047c0 0x7fa994004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 --2- 172.21.15.70:0/1340762744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa994005180 0x7fa99400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99400c080 con 0x7fa9940047c0 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99400c1f0 con 0x7fa994005180 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.854+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99400c3b0 con 0x7fa994003970 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.855+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/1340762744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940047c0 0x7fa994004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.855+0000 7fa99f7fe640 1 -- 172.21.15.70:0/1340762744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa994003970 msgr2=0x7fa994003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.855+0000 7fa9a7fff640 1 --2- 172.21.15.70:0/1340762744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa994003970 0x7fa994003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.855+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/1340762744 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa994003970 0x7fa994003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.855+0000 7fa99f7fe640 1 -- 172.21.15.70:0/1340762744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa994005180 msgr2=0x7fa99400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:39.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.855+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/1340762744 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa994005180 0x7fa99400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:39.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.855+0000 7fa99f7fe640 1 -- 172.21.15.70:0/1340762744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa994003520 con 0x7fa9940047c0 2023-10-08T07:10:39.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.855+0000 7fa9ac92d640 1 -- 172.21.15.70:0/1340762744 reap_dead start 2023-10-08T07:10:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.856+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/1340762744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940047c0 0x7fa994004c40 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fa99000ba00 tx=0x7fa9940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc50c92c064f1d32 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.856+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990032640 con 0x7fa9940047c0 2023-10-08T07:10:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.856+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9900327a0 con 0x7fa9940047c0 2023-10-08T07:10:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.856+0000 7fa9a77fe640 1 -- 172.21.15.70:0/1340762744 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990032b30 con 0x7fa9940047c0 2023-10-08T07:10:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.857+0000 7fa9ae390640 1 -- 172.21.15.70:0/1340762744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940047c0 msgr2=0x7fa994004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.857+0000 7fa9ae390640 1 --2- 172.21.15.70:0/1340762744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940047c0 0x7fa994004c40 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fa99000ba00 tx=0x7fa9940057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.857+0000 7fa9ae390640 1 -- 172.21.15.70:0/1340762744 shutdown_connections 2023-10-08T07:10:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.857+0000 7fa9ae390640 1 --2- 172.21.15.70:0/1340762744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9940047c0 0x7fa994004c40 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:39.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.857+0000 7fa9ae390640 1 -- 172.21.15.70:0/1340762744 >> 172.21.15.70:0/1340762744 conn(0x7fa9a806a380 msgr2=0x7fa9a80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:39.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.857+0000 7fa9ae390640 1 -- 172.21.15.70:0/1340762744 shutdown_connections 2023-10-08T07:10:39.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.857+0000 7fa9ae390640 1 -- 172.21.15.70:0/1340762744 wait complete. 2023-10-08T07:10:39.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.859+0000 7fa9ae390640 1 Processor -- start 2023-10-08T07:10:39.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.859+0000 7fa9ae390640 1 -- start start 2023-10-08T07:10:39.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.860+0000 7fa9ae390640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940047c0 0x7fa9a8140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:39.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.860+0000 7fa9ae390640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a80cb4d0 0x7fa9a815de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:39.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.860+0000 7fa9ae390640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a815e3a0 0x7fa9a80cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:39.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.860+0000 7fa9ae390640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80cb2c0 con 0x7fa9a80cb4d0 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.860+0000 7fa9ae390640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80c9ca0 con 0x7fa9a815e3a0 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.860+0000 7fa9ae390640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80c9120 con 0x7fa9940047c0 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.860+0000 7fa99f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a80cb4d0 0x7fa9a815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.860+0000 7fa99f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a80cb4d0 0x7fa9a815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48596/0 (socket says 172.21.15.70:48596) 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.860+0000 7fa99f7fe640 1 -- 172.21.15.70:0/739480992 learned_addr learned my addr 172.21.15.70:0/739480992 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa99f7fe640 1 -- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940047c0 msgr2=0x7fa9a8140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940047c0 0x7fa9a8140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa99f7fe640 1 -- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a815e3a0 msgr2=0x7fa9a80cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a815e3a0 0x7fa9a80cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa99f7fe640 1 -- 172.21.15.70:0/739480992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a80c9400 con 0x7fa9a80cb4d0 2023-10-08T07:10:39.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa99f7fe640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a80cb4d0 0x7fa9a815de60 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fa99002bec0 tx=0x7fa990004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa9a57fa640 1 -- 172.21.15.70:0/739480992 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990004220 con 0x7fa9a80cb4d0 2023-10-08T07:10:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa9a57fa640 1 -- 172.21.15.70:0/739480992 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9900043a0 con 0x7fa9a80cb4d0 2023-10-08T07:10:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa9a57fa640 1 -- 172.21.15.70:0/739480992 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99003b9f0 con 0x7fa9a80cb4d0 2023-10-08T07:10:39.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa9ae390640 1 -- 172.21.15.70:0/739480992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a80c96e0 con 0x7fa9a80cb4d0 2023-10-08T07:10:39.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.861+0000 7fa9ae390640 1 -- 172.21.15.70:0/739480992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a80c8db0 con 0x7fa9a80cb4d0 2023-10-08T07:10:39.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.862+0000 7fa9ae390640 1 -- 172.21.15.70:0/739480992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a8067490 con 0x7fa9a80cb4d0 2023-10-08T07:10:39.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.863+0000 7fa9a57fa640 1 -- 172.21.15.70:0/739480992 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa99003bb50 con 0x7fa9a80cb4d0 2023-10-08T07:10:39.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.865+0000 7fa9a57fa640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa98806af90 0x7fa98806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:39.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.865+0000 7fa9a57fa640 1 -- 172.21.15.70:0/739480992 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fa9900ac220 con 0x7fa9a80cb4d0 2023-10-08T07:10:39.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.865+0000 7fa9a7fff640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa98806af90 0x7fa98806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:39.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.866+0000 7fa9a7fff640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa98806af90 0x7fa98806d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9a804fa60 tx=0x7fa998009040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:39.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:39.870+0000 7fa9a57fa640 1 -- 172.21.15.70:0/739480992 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa9900772d0 con 0x7fa9a80cb4d0 2023-10-08T07:10:40.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.126+0000 7fa9ae390640 1 -- 172.21.15.70:0/739480992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa9a8067490 con 0x7fa9a80cb4d0 2023-10-08T07:10:40.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.127+0000 7fa9a57fa640 1 -- 172.21.15.70:0/739480992 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v49) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa99007a8f0 con 0x7fa9a80cb4d0 2023-10-08T07:10:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:40.128 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:10:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T07:10:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T07:10:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T07:10:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:10:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:10:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T07:10:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:10:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T07:10:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:10:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:10:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T07:10:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T07:10:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:10:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:10:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:10:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T07:10:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T07:10:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:10:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T07:10:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T07:10:40.144 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:10:40.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 -- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa98806af90 msgr2=0x7fa98806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa98806af90 0x7fa98806d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa9a804fa60 tx=0x7fa998009040 comp rx=0 tx=0).stop 2023-10-08T07:10:40.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 -- 172.21.15.70:0/739480992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a80cb4d0 msgr2=0x7fa9a815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a80cb4d0 0x7fa9a815de60 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fa99002bec0 tx=0x7fa990004730 comp rx=0 tx=0).stop 2023-10-08T07:10:40.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 -- 172.21.15.70:0/739480992 shutdown_connections 2023-10-08T07:10:40.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa98806af90 0x7fa98806d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa9a815e3a0 0x7fa9a80cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9a80cb4d0 0x7fa9a815de60 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 --2- 172.21.15.70:0/739480992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa9940047c0 0x7fa9a8140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 -- 172.21.15.70:0/739480992 >> 172.21.15.70:0/739480992 conn(0x7fa9a806a380 msgr2=0x7fa9a80af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:40.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 -- 172.21.15.70:0/739480992 shutdown_connections 2023-10-08T07:10:40.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.130+0000 7fa9ae390640 1 -- 172.21.15.70:0/739480992 wait complete. 2023-10-08T07:10:40.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:10:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.355+0000 7f1d90400640 1 Processor -- start 2023-10-08T07:10:40.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.356+0000 7f1d90400640 1 -- start start 2023-10-08T07:10:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.357+0000 7f1d90400640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1d8815ab40 0x7f1d880ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.357+0000 7f1d90400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d880c90b0 0x7f1d880adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.357+0000 7f1d90400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1d880cae70 0x7f1d880cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.357+0000 7f1d90400640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8815bf30 con 0x7f1d880cae70 2023-10-08T07:10:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.357+0000 7f1d90400640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d880cb840 con 0x7f1d8815ab40 2023-10-08T07:10:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.357+0000 7f1d90400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8815b2b0 con 0x7f1d880c90b0 2023-10-08T07:10:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8e99d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1d880cae70 0x7f1d880cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8e99d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1d880cae70 0x7f1d880cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48598/0 (socket says 172.21.15.70:48598) 2023-10-08T07:10:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8e99d640 1 -- 172.21.15.70:0/428762949 learned_addr learned my addr 172.21.15.70:0/428762949 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:40.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8e99d640 1 -- 172.21.15.70:0/428762949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d880c90b0 msgr2=0x7f1d880adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:40.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8d99b640 1 --2- 172.21.15.70:0/428762949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d880c90b0 0x7f1d880adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8e19c640 1 --2- 172.21.15.70:0/428762949 >> v2:172.21.15.181:3300/0 conn(0x7f1d8815ab40 0x7f1d880ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8e99d640 1 --2- 172.21.15.70:0/428762949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d880c90b0 0x7f1d880adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8e99d640 1 -- 172.21.15.70:0/428762949 >> v2:172.21.15.181:3300/0 conn(0x7f1d8815ab40 msgr2=0x7f1d880ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:40.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8e99d640 1 --2- 172.21.15.70:0/428762949 >> v2:172.21.15.181:3300/0 conn(0x7f1d8815ab40 0x7f1d880ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8e99d640 1 -- 172.21.15.70:0/428762949 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8815b590 con 0x7f1d880cae70 2023-10-08T07:10:40.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.358+0000 7f1d8e19c640 1 --2- 172.21.15.70:0/428762949 >> v2:172.21.15.181:3300/0 conn(0x7f1d8815ab40 0x7f1d880ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:40.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.359+0000 7f1d8e99d640 1 --2- 172.21.15.70:0/428762949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d880cae70 0x7f1d880cb300 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f1d8806e440 tx=0x7f1d8002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883189c337d2f29c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:40.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d8d19a640 1 -- 172.21.15.70:0/428762949 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8002ca80 con 0x7f1d880cae70 2023-10-08T07:10:40.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d8d19a640 1 -- 172.21.15.70:0/428762949 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8002cbe0 con 0x7f1d880cae70 2023-10-08T07:10:40.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d8d19a640 1 -- 172.21.15.70:0/428762949 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800343f0 con 0x7f1d880cae70 2023-10-08T07:10:40.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d90400640 1 -- 172.21.15.70:0/428762949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d880cae70 msgr2=0x7f1d880cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d90400640 1 --2- 172.21.15.70:0/428762949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d880cae70 0x7f1d880cb300 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f1d8806e440 tx=0x7f1d8002bee0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d90400640 1 -- 172.21.15.70:0/428762949 shutdown_connections 2023-10-08T07:10:40.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d90400640 1 --2- 172.21.15.70:0/428762949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d880cae70 0x7f1d880cb300 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d90400640 1 --2- 172.21.15.70:0/428762949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d880c90b0 0x7f1d880adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d90400640 1 --2- 172.21.15.70:0/428762949 >> v2:172.21.15.181:3300/0 conn(0x7f1d8815ab40 0x7f1d880ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d90400640 1 -- 172.21.15.70:0/428762949 >> 172.21.15.70:0/428762949 conn(0x7f1d8806a380 msgr2=0x7f1d880b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:40.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.360+0000 7f1d90400640 1 -- 172.21.15.70:0/428762949 shutdown_connections 2023-10-08T07:10:40.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.361+0000 7f1d90400640 1 -- 172.21.15.70:0/428762949 wait complete. 2023-10-08T07:10:40.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.362+0000 7f1d90400640 1 Processor -- start 2023-10-08T07:10:40.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.362+0000 7f1d90400640 1 -- start start 2023-10-08T07:10:40.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.363+0000 7f1d90400640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d880c90b0 0x7f1d881490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.363+0000 7f1d90400640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880cae70 0x7f1d8815d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.363+0000 7f1d90400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d8815ab40 0x7f1d8815d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.363+0000 7f1d90400640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880b2c60 con 0x7f1d8815ab40 2023-10-08T07:10:40.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.363+0000 7f1d90400640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880c84f0 con 0x7f1d880cae70 2023-10-08T07:10:40.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.363+0000 7f1d90400640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d880b1fe0 con 0x7f1d880c90b0 2023-10-08T07:10:40.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.364+0000 7f1d8e99d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d8815ab40 0x7f1d8815d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.364+0000 7f1d8e99d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d8815ab40 0x7f1d8815d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48612/0 (socket says 172.21.15.70:48612) 2023-10-08T07:10:40.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.364+0000 7f1d8e99d640 1 -- 172.21.15.70:0/1435344527 learned_addr learned my addr 172.21.15.70:0/1435344527 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:40.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.364+0000 7f1d8e99d640 1 -- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d880c90b0 msgr2=0x7f1d881490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.364+0000 7f1d8e99d640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d880c90b0 0x7f1d881490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.364+0000 7f1d8e99d640 1 -- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880cae70 msgr2=0x7f1d8815d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.364+0000 7f1d8d99b640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880cae70 0x7f1d8815d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.364+0000 7f1d8e99d640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880cae70 0x7f1d8815d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.364+0000 7f1d8e99d640 1 -- 172.21.15.70:0/1435344527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d80009b30 con 0x7f1d8815ab40 2023-10-08T07:10:40.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.364+0000 7f1d8e99d640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d8815ab40 0x7f1d8815d8d0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f1d80036040 tx=0x7f1d80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:40.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.365+0000 7f1d7effd640 1 -- 172.21.15.70:0/1435344527 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80009510 con 0x7f1d8815ab40 2023-10-08T07:10:40.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.365+0000 7f1d7effd640 1 -- 172.21.15.70:0/1435344527 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8003e450 con 0x7f1d8815ab40 2023-10-08T07:10:40.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.365+0000 7f1d7effd640 1 -- 172.21.15.70:0/1435344527 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8003e800 con 0x7f1d8815ab40 2023-10-08T07:10:40.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.365+0000 7f1d90400640 1 -- 172.21.15.70:0/1435344527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d880b22c0 con 0x7f1d8815ab40 2023-10-08T07:10:40.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.365+0000 7f1d90400640 1 -- 172.21.15.70:0/1435344527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d880b2880 con 0x7f1d8815ab40 2023-10-08T07:10:40.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.366+0000 7f1d90400640 1 -- 172.21.15.70:0/1435344527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d88067490 con 0x7f1d8815ab40 2023-10-08T07:10:40.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.367+0000 7f1d7effd640 1 -- 172.21.15.70:0/1435344527 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1d8002cd50 con 0x7f1d8815ab40 2023-10-08T07:10:40.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.368+0000 7f1d7effd640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f1d6c06af90 0x7f1d6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.368+0000 7f1d7effd640 1 -- 172.21.15.70:0/1435344527 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f1d800ac1f0 con 0x7f1d8815ab40 2023-10-08T07:10:40.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.369+0000 7f1d8e19c640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f1d6c06af90 0x7f1d6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.370+0000 7f1d8e19c640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f1d6c06af90 0x7f1d6c06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d84006fd0 tx=0x7f1d84008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:40.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.372+0000 7f1d7effd640 1 -- 172.21.15.70:0/1435344527 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d80077500 con 0x7f1d8815ab40 2023-10-08T07:10:40.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.559+0000 7f1d90400640 1 -- 172.21.15.70:0/1435344527 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d880b0ef0 con 0x7f1d8815ab40 2023-10-08T07:10:40.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.561+0000 7f1d7effd640 1 -- 172.21.15.70:0/1435344527 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+13246 (secure 0 0 0) 0x7f1d8007ab20 con 0x7f1d8815ab40 2023-10-08T07:10:40.562 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:40.562 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":212,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:10:39.599458+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:10:40.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.563+0000 7f1d90400640 1 -- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f1d6c06af90 msgr2=0x7f1d6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.563+0000 7f1d90400640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f1d6c06af90 0x7f1d6c06d450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1d84006fd0 tx=0x7f1d84008040 comp rx=0 tx=0).stop 2023-10-08T07:10:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.563+0000 7f1d90400640 1 -- 172.21.15.70:0/1435344527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d8815ab40 msgr2=0x7f1d8815d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.563+0000 7f1d90400640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d8815ab40 0x7f1d8815d8d0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f1d80036040 tx=0x7f1d80004a40 comp rx=0 tx=0).stop 2023-10-08T07:10:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.564+0000 7f1d90400640 1 -- 172.21.15.70:0/1435344527 shutdown_connections 2023-10-08T07:10:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.564+0000 7f1d90400640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f1d6c06af90 0x7f1d6c06d450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.564+0000 7f1d90400640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d8815ab40 0x7f1d8815d8d0 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.564+0000 7f1d90400640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1d880cae70 0x7f1d8815d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.564+0000 7f1d90400640 1 --2- 172.21.15.70:0/1435344527 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1d880c90b0 0x7f1d881490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.564+0000 7f1d90400640 1 -- 172.21.15.70:0/1435344527 >> 172.21.15.70:0/1435344527 conn(0x7f1d8806a380 msgr2=0x7f1d880af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.564+0000 7f1d90400640 1 -- 172.21.15.70:0/1435344527 shutdown_connections 2023-10-08T07:10:40.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.564+0000 7f1d90400640 1 -- 172.21.15.70:0/1435344527 wait complete. 2023-10-08T07:10:40.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:10:40.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:10:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.784+0000 7f60a83d7640 1 Processor -- start 2023-10-08T07:10:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.784+0000 7f60a83d7640 1 -- start start 2023-10-08T07:10:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.786+0000 7f60a83d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f60a00b1750 0x7f60a006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.786+0000 7f60a83d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a006d670 con 0x7f60a015aad0 2023-10-08T07:10:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.786+0000 7f60a83d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a015bf30 con 0x7f60a006d240 2023-10-08T07:10:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.786+0000 7f60a83d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a00b1bb0 con 0x7f60a00b1750 2023-10-08T07:10:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.786+0000 7f60a6173640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f60a015aad0 0x7f60a00c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41670/0 (socket says 172.21.15.70:41670) 2023-10-08T07:10:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.786+0000 7f60a6173640 1 -- 172.21.15.70:0/3269171904 learned_addr learned my addr 172.21.15.70:0/3269171904 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:40.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775438242 0 0) 0x7f60a006d670 con 0x7f60a015aad0 2023-10-08T07:10:40.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f608c003520 con 0x7f60a015aad0 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5972640 1 --2- 172.21.15.70:0/3269171904 >> v2:172.21.15.181:3300/0 conn(0x7f60a00b1750 0x7f60a006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1731623436 0 0) 0x7f608c003520 con 0x7f60a015aad0 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 >> v2:172.21.15.181:3300/0 conn(0x7f60a00b1750 msgr2=0x7f60a006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 --2- 172.21.15.70:0/3269171904 >> v2:172.21.15.181:3300/0 conn(0x7f60a00b1750 0x7f60a006cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 >> v1:172.21.15.181:6789/0 conn(0x7f60a006d240 legacy=0x7f60a0070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a015b320 con 0x7f60a015aad0 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f609c001690 con 0x7f60a015aad0 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 >> v1:172.21.15.70:6789/0 conn(0x7f60a015aad0 legacy=0x7f60a00c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 --2- 172.21.15.70:0/3269171904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f608c0039c0 0x7f608c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 --2- 172.21.15.70:0/3269171904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f608c0047c0 0x7f608c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 --2- 172.21.15.70:0/3269171904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f608c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00b1bb0 con 0x7f608c0051d0 2023-10-08T07:10:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c00c100 con 0x7f608c0039c0 2023-10-08T07:10:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c00c2c0 con 0x7f608c0047c0 2023-10-08T07:10:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.787+0000 7f60a6974640 1 --2- 172.21.15.70:0/3269171904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f608c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.788+0000 7f60a6173640 1 --2- 172.21.15.70:0/3269171904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f608c0039c0 0x7f608c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.788+0000 7f60a6974640 1 -- 172.21.15.70:0/3269171904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f608c0047c0 msgr2=0x7f608c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.788+0000 7f60a6974640 1 --2- 172.21.15.70:0/3269171904 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f608c0047c0 0x7f608c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.788+0000 7f60a6974640 1 -- 172.21.15.70:0/3269171904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f608c0039c0 msgr2=0x7f608c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.788+0000 7f60a6974640 1 --2- 172.21.15.70:0/3269171904 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f608c0039c0 0x7f608c003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.788+0000 7f60a6974640 1 -- 172.21.15.70:0/3269171904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c003520 con 0x7f608c0051d0 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.789+0000 7f60a6974640 1 -- 172.21.15.70:0/3269171904 reap_dead start 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.789+0000 7f60a6974640 1 --2- 172.21.15.70:0/3269171904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f608c00b980 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f6098004480 tx=0x7f608c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74b5bb7ebbb6a93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.789+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098006b50 con 0x7f608c0051d0 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.789+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6098006cd0 con 0x7f608c0051d0 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.789+0000 7f60a5171640 1 -- 172.21.15.70:0/3269171904 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098007260 con 0x7f608c0051d0 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.790+0000 7f60a83d7640 1 -- 172.21.15.70:0/3269171904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 msgr2=0x7f608c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.790+0000 7f60a83d7640 1 --2- 172.21.15.70:0/3269171904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f608c00b980 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f6098004480 tx=0x7f608c0057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.790+0000 7f60a83d7640 1 -- 172.21.15.70:0/3269171904 shutdown_connections 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.790+0000 7f60a83d7640 1 --2- 172.21.15.70:0/3269171904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f608c00b980 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.790+0000 7f60a83d7640 1 -- 172.21.15.70:0/3269171904 >> 172.21.15.70:0/3269171904 conn(0x7f60a0069d50 msgr2=0x7f60a006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.790+0000 7f60a83d7640 1 -- 172.21.15.70:0/3269171904 shutdown_connections 2023-10-08T07:10:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.790+0000 7f60a83d7640 1 -- 172.21.15.70:0/3269171904 wait complete. 2023-10-08T07:10:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.792+0000 7f60a83d7640 1 Processor -- start 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.792+0000 7f60a83d7640 1 -- start start 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a83d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f60a00ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a83d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a014e9b0 0x7f60a01f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a83d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60a00b26e0 0x7f60a00b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a83d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00b30a0 con 0x7f608c0051d0 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a83d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a0158620 con 0x7f60a014e9b0 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a83d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00cae00 con 0x7f60a00b26e0 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a6173640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f60a00ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a6173640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f60a00ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48616/0 (socket says 172.21.15.70:48616) 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a6173640 1 -- 172.21.15.70:0/546310671 learned_addr learned my addr 172.21.15.70:0/546310671 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a6173640 1 -- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60a00b26e0 msgr2=0x7f60a00b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a6173640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60a00b26e0 0x7f60a00b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a6173640 1 -- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a014e9b0 msgr2=0x7f60a01f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a6173640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a014e9b0 0x7f60a01f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.793+0000 7f60a6173640 1 -- 172.21.15.70:0/546310671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a0159180 con 0x7f608c0051d0 2023-10-08T07:10:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.794+0000 7f60a6173640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f60a00ca820 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f609c00d7a0 tx=0x7f609c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.794+0000 7f6096ffd640 1 -- 172.21.15.70:0/546310671 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c0074e0 con 0x7f608c0051d0 2023-10-08T07:10:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.794+0000 7f6096ffd640 1 -- 172.21.15.70:0/546310671 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609c007640 con 0x7f608c0051d0 2023-10-08T07:10:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.794+0000 7f6096ffd640 1 -- 172.21.15.70:0/546310671 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609c004a20 con 0x7f608c0051d0 2023-10-08T07:10:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.794+0000 7f60a83d7640 1 -- 172.21.15.70:0/546310671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a0158850 con 0x7f608c0051d0 2023-10-08T07:10:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.794+0000 7f60a83d7640 1 -- 172.21.15.70:0/546310671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a0158e10 con 0x7f608c0051d0 2023-10-08T07:10:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.795+0000 7f60a83d7640 1 -- 172.21.15.70:0/546310671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a004e160 con 0x7f608c0051d0 2023-10-08T07:10:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.796+0000 7f6096ffd640 1 -- 172.21.15.70:0/546310671 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f609c004b80 con 0x7f608c0051d0 2023-10-08T07:10:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.797+0000 7f6096ffd640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f608406afc0 0x7f608406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.797+0000 7f6096ffd640 1 -- 172.21.15.70:0/546310671 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f609c088f60 con 0x7f608c0051d0 2023-10-08T07:10:40.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.798+0000 7f60a5972640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f608406afc0 0x7f608406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:40.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.799+0000 7f60a5972640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f608406afc0 0x7f608406d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6090006f00 tx=0x7f6090008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:40.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.801+0000 7f6096ffd640 1 -- 172.21.15.70:0/546310671 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f609c053f90 con 0x7f608c0051d0 2023-10-08T07:10:40.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.994+0000 7f60a83d7640 1 -- 172.21.15.70:0/546310671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60a0155370 con 0x7f608c0051d0 2023-10-08T07:10:40.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.994+0000 7f6096ffd640 1 -- 172.21.15.70:0/546310671 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v212) v1 ==== 93+0+10 (secure 0 0 0) 0x7f609c019020 con 0x7f608c0051d0 2023-10-08T07:10:40.995 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:10:40.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.997+0000 7f60a83d7640 1 -- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f608406afc0 msgr2=0x7f608406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:40.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.997+0000 7f60a83d7640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f608406afc0 0x7f608406d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6090006f00 tx=0x7f6090008040 comp rx=0 tx=0).stop 2023-10-08T07:10:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.997+0000 7f60a83d7640 1 -- 172.21.15.70:0/546310671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 msgr2=0x7f60a00ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.997+0000 7f60a83d7640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f60a00ca820 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f609c00d7a0 tx=0x7f609c00dc60 comp rx=0 tx=0).stop 2023-10-08T07:10:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.997+0000 7f60a83d7640 1 -- 172.21.15.70:0/546310671 shutdown_connections 2023-10-08T07:10:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.997+0000 7f60a83d7640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f608406afc0 0x7f608406d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.997+0000 7f60a83d7640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60a00b26e0 0x7f60a00b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.997+0000 7f60a83d7640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a014e9b0 0x7f60a01f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.997+0000 7f60a83d7640 1 --2- 172.21.15.70:0/546310671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f608c0051d0 0x7f60a00ca820 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.997+0000 7f60a83d7640 1 -- 172.21.15.70:0/546310671 >> 172.21.15.70:0/546310671 conn(0x7f60a0069d50 msgr2=0x7f60a015d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.998+0000 7f60a83d7640 1 -- 172.21.15.70:0/546310671 shutdown_connections 2023-10-08T07:10:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:40.998+0000 7f60a83d7640 1 -- 172.21.15.70:0/546310671 wait complete. 2023-10-08T07:10:41.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T07:10:41.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.226+0000 7efe03353640 1 Processor -- start 2023-10-08T07:10:41.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.226+0000 7efe03353640 1 -- start start 2023-10-08T07:10:41.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.228+0000 7efe03353640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efdfc0ad790 0x7efdfc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.228+0000 7efe03353640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efdfc15e330 0x7efdfc070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.228+0000 7efe03353640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc06dee0 con 0x7efdfc159ac0 2023-10-08T07:10:41.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.228+0000 7efe03353640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc06dcf0 con 0x7efdfc15e330 2023-10-08T07:10:41.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.228+0000 7efe03353640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc15d850 con 0x7efdfc0ad790 2023-10-08T07:10:41.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.228+0000 7efe018f0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efdfc15e330 0x7efdfc070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.228+0000 7efe018f0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efdfc15e330 0x7efdfc070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48632/0 (socket says 172.21.15.70:48632) 2023-10-08T07:10:41.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.228+0000 7efe018f0640 1 -- 172.21.15.70:0/3049304075 learned_addr learned my addr 172.21.15.70:0/3049304075 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:41.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.229+0000 7efe008ee640 1 --2- 172.21.15.70:0/3049304075 >> v2:172.21.15.181:3300/0 conn(0x7efdfc0ad790 0x7efdfc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.229+0000 7efe018f0640 1 -- 172.21.15.70:0/3049304075 >> v2:172.21.15.181:3300/0 conn(0x7efdfc0ad790 msgr2=0x7efdfc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:41.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.229+0000 7efe018f0640 1 --2- 172.21.15.70:0/3049304075 >> v2:172.21.15.181:3300/0 conn(0x7efdfc0ad790 0x7efdfc0702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.229+0000 7efe018f0640 1 -- 172.21.15.70:0/3049304075 >> v1:172.21.15.181:6789/0 conn(0x7efdfc159ac0 legacy=0x7efdfc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:41.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.229+0000 7efe018f0640 1 -- 172.21.15.70:0/3049304075 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc15db30 con 0x7efdfc15e330 2023-10-08T07:10:41.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.230+0000 7efe018f0640 1 --2- 172.21.15.70:0/3049304075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc15e330 0x7efdfc070810 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0b4970 tx=0x7efdf402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f68491e719c6f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:41.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.230+0000 7efdf3fff640 1 -- 172.21.15.70:0/3049304075 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf402ca80 con 0x7efdfc15e330 2023-10-08T07:10:41.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.230+0000 7efdf3fff640 1 -- 172.21.15.70:0/3049304075 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf402cbe0 con 0x7efdfc15e330 2023-10-08T07:10:41.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.230+0000 7efdf3fff640 1 -- 172.21.15.70:0/3049304075 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf40343f0 con 0x7efdfc15e330 2023-10-08T07:10:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.231+0000 7efe03353640 1 -- 172.21.15.70:0/3049304075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc15e330 msgr2=0x7efdfc070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.231+0000 7efe03353640 1 --2- 172.21.15.70:0/3049304075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc15e330 0x7efdfc070810 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7efdfc0b4970 tx=0x7efdf402bee0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.231+0000 7efe03353640 1 -- 172.21.15.70:0/3049304075 shutdown_connections 2023-10-08T07:10:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.231+0000 7efe03353640 1 --2- 172.21.15.70:0/3049304075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc15e330 0x7efdfc070810 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.231+0000 7efe03353640 1 --2- 172.21.15.70:0/3049304075 >> v2:172.21.15.181:3300/0 conn(0x7efdfc0ad790 0x7efdfc0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.231+0000 7efe03353640 1 -- 172.21.15.70:0/3049304075 >> 172.21.15.70:0/3049304075 conn(0x7efdfc06a380 msgr2=0x7efdfc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.231+0000 7efe03353640 1 -- 172.21.15.70:0/3049304075 shutdown_connections 2023-10-08T07:10:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.231+0000 7efe03353640 1 -- 172.21.15.70:0/3049304075 wait complete. 2023-10-08T07:10:41.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.233+0000 7efe03353640 1 Processor -- start 2023-10-08T07:10:41.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.233+0000 7efe03353640 1 -- start start 2023-10-08T07:10:41.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.234+0000 7efe03353640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0ad790 0x7efdfc155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.234+0000 7efe03353640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc159ac0 0x7efdfc15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.234+0000 7efe03353640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc15e330 0x7efdfc152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.234+0000 7efe03353640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc155ba0 con 0x7efdfc159ac0 2023-10-08T07:10:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.234+0000 7efe03353640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1586c0 con 0x7efdfc15e330 2023-10-08T07:10:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.234+0000 7efe03353640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc15bf30 con 0x7efdfc0ad790 2023-10-08T07:10:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.234+0000 7efe008ee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc159ac0 0x7efdfc15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe008ee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc159ac0 0x7efdfc15b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48638/0 (socket says 172.21.15.70:48638) 2023-10-08T07:10:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe008ee640 1 -- 172.21.15.70:0/3804837461 learned_addr learned my addr 172.21.15.70:0/3804837461 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe018f0640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc15e330 0x7efdfc152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe010ef640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0ad790 0x7efdfc155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe008ee640 1 -- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0ad790 msgr2=0x7efdfc155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:41.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe008ee640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0ad790 0x7efdfc155660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe008ee640 1 -- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc15e330 msgr2=0x7efdfc152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:41.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe008ee640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc15e330 0x7efdfc152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe008ee640 1 -- 172.21.15.70:0/3804837461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009b30 con 0x7efdfc159ac0 2023-10-08T07:10:41.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe018f0640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc15e330 0x7efdfc152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:41.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.235+0000 7efe008ee640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc159ac0 0x7efdfc15b9f0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c7f0 tx=0x7efdec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.236+0000 7efdf1ffb640 1 -- 172.21.15.70:0/3804837461 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec004220 con 0x7efdfc159ac0 2023-10-08T07:10:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.236+0000 7efdf1ffb640 1 -- 172.21.15.70:0/3804837461 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdec0043a0 con 0x7efdfc159ac0 2023-10-08T07:10:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.236+0000 7efdf1ffb640 1 -- 172.21.15.70:0/3804837461 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdec007860 con 0x7efdfc159ac0 2023-10-08T07:10:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.236+0000 7efe03353640 1 -- 172.21.15.70:0/3804837461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc1f29f0 con 0x7efdfc159ac0 2023-10-08T07:10:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.236+0000 7efe03353640 1 -- 172.21.15.70:0/3804837461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc158910 con 0x7efdfc159ac0 2023-10-08T07:10:41.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.237+0000 7efe03353640 1 -- 172.21.15.70:0/3804837461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc067490 con 0x7efdfc159ac0 2023-10-08T07:10:41.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.238+0000 7efdf1ffb640 1 -- 172.21.15.70:0/3804837461 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efdec007bd0 con 0x7efdfc159ac0 2023-10-08T07:10:41.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.239+0000 7efdf1ffb640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7efde006af90 0x7efde006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.239+0000 7efdf1ffb640 1 -- 172.21.15.70:0/3804837461 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5293+0+0 (secure 0 0 0) 0x7efdec089690 con 0x7efdfc159ac0 2023-10-08T07:10:41.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.239+0000 7efe010ef640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7efde006af90 0x7efde006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.241+0000 7efe010ef640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7efde006af90 0x7efde006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efdf80015d0 tx=0x7efdf8009040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:41.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.243+0000 7efdf1ffb640 1 -- 172.21.15.70:0/3804837461 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efdec0546b0 con 0x7efdfc159ac0 2023-10-08T07:10:41.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.424+0000 7efe03353640 1 -- 172.21.15.70:0/3804837461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efdfc0c8da0 con 0x7efdfc159ac0 2023-10-08T07:10:41.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.425+0000 7efdf1ffb640 1 -- 172.21.15.70:0/3804837461 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v212) v1 ==== 84+0+65 (secure 0 0 0) 0x7efdec01e090 con 0x7efdfc159ac0 2023-10-08T07:10:41.426 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:41.426 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T07:10:41.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:41.426 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T07:10:41.426 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T07:10:41.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:41.426 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T07:10:41.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.427+0000 7efe03353640 1 -- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7efde006af90 msgr2=0x7efde006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:41.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.427+0000 7efe03353640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7efde006af90 0x7efde006d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efdf80015d0 tx=0x7efdf8009040 comp rx=0 tx=0).stop 2023-10-08T07:10:41.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.427+0000 7efe03353640 1 -- 172.21.15.70:0/3804837461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc159ac0 msgr2=0x7efdfc15b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:41.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.427+0000 7efe03353640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc159ac0 0x7efdfc15b9f0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7efdec00c7f0 tx=0x7efdec00ccb0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.428+0000 7efe03353640 1 -- 172.21.15.70:0/3804837461 shutdown_connections 2023-10-08T07:10:41.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.428+0000 7efe03353640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7efde006af90 0x7efde006d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.428+0000 7efe03353640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc15e330 0x7efdfc152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.428+0000 7efe03353640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc159ac0 0x7efdfc15b9f0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.428+0000 7efe03353640 1 --2- 172.21.15.70:0/3804837461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0ad790 0x7efdfc155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.428+0000 7efe03353640 1 -- 172.21.15.70:0/3804837461 >> 172.21.15.70:0/3804837461 conn(0x7efdfc06a380 msgr2=0x7efdfc0c9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:41.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.428+0000 7efe03353640 1 -- 172.21.15.70:0/3804837461 shutdown_connections 2023-10-08T07:10:41.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.428+0000 7efe03353640 1 -- 172.21.15.70:0/3804837461 wait complete. 2023-10-08T07:10:41.446 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T07:10:41.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T07:10:41.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.657+0000 7fba21597640 1 Processor -- start 2023-10-08T07:10:41.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.657+0000 7fba21597640 1 -- start start 2023-10-08T07:10:41.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.659+0000 7fba21597640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c0b1750 0x7fba1c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.659+0000 7fba21597640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fba1c06d240 0x7fba1c070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.659+0000 7fba21597640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c15bf30 con 0x7fba1c15aad0 2023-10-08T07:10:41.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.659+0000 7fba21597640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c06d6d0 con 0x7fba1c06d240 2023-10-08T07:10:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.659+0000 7fba21597640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0b1bb0 con 0x7fba1c0b1750 2023-10-08T07:10:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.659+0000 7fba1b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fba1c06d240 0x7fba1c070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.659+0000 7fba1b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fba1c06d240 0x7fba1c070570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48640/0 (socket says 172.21.15.70:48640) 2023-10-08T07:10:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.659+0000 7fba1b7fe640 1 -- 172.21.15.70:0/267347812 learned_addr learned my addr 172.21.15.70:0/267347812 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.660+0000 7fba1b7fe640 1 -- 172.21.15.70:0/267347812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c0b1750 msgr2=0x7fba1c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.660+0000 7fba13fff640 1 --2- 172.21.15.70:0/267347812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c0b1750 0x7fba1c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.660+0000 7fba1b7fe640 1 --2- 172.21.15.70:0/267347812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c0b1750 0x7fba1c06cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.660+0000 7fba1b7fe640 1 -- 172.21.15.70:0/267347812 >> v1:172.21.15.181:6789/0 conn(0x7fba1c15aad0 legacy=0x7fba1c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.660+0000 7fba1b7fe640 1 -- 172.21.15.70:0/267347812 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c15b320 con 0x7fba1c06d240 2023-10-08T07:10:41.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.660+0000 7fba13fff640 1 --2- 172.21.15.70:0/267347812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c0b1750 0x7fba1c06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.661+0000 7fba1b7fe640 1 --2- 172.21.15.70:0/267347812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c06d240 0x7fba1c070570 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fba14009e50 tx=0x7fba1402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b4ed02f19658f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.661+0000 7fba1a7fc640 1 -- 172.21.15.70:0/267347812 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1402ca80 con 0x7fba1c06d240 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.661+0000 7fba1a7fc640 1 -- 172.21.15.70:0/267347812 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1402cbe0 con 0x7fba1c06d240 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.661+0000 7fba1a7fc640 1 -- 172.21.15.70:0/267347812 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba140343f0 con 0x7fba1c06d240 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.661+0000 7fba21597640 1 -- 172.21.15.70:0/267347812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c06d240 msgr2=0x7fba1c070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.661+0000 7fba21597640 1 --2- 172.21.15.70:0/267347812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c06d240 0x7fba1c070570 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fba14009e50 tx=0x7fba1402bee0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.662+0000 7fba21597640 1 -- 172.21.15.70:0/267347812 shutdown_connections 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.662+0000 7fba21597640 1 --2- 172.21.15.70:0/267347812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c06d240 0x7fba1c070570 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.662+0000 7fba21597640 1 --2- 172.21.15.70:0/267347812 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c0b1750 0x7fba1c06cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.662+0000 7fba21597640 1 -- 172.21.15.70:0/267347812 >> 172.21.15.70:0/267347812 conn(0x7fba1c069d50 msgr2=0x7fba1c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:41.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.662+0000 7fba21597640 1 -- 172.21.15.70:0/267347812 shutdown_connections 2023-10-08T07:10:41.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.662+0000 7fba21597640 1 -- 172.21.15.70:0/267347812 wait complete. 2023-10-08T07:10:41.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.663+0000 7fba21597640 1 Processor -- start 2023-10-08T07:10:41.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.663+0000 7fba21597640 1 -- start start 2023-10-08T07:10:41.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.664+0000 7fba21597640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c06d240 0x7fba1c157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.664+0000 7fba21597640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c0b1750 0x7fba1c1f3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.664+0000 7fba21597640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c15aad0 0x7fba1c0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.664+0000 7fba21597640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c1f4330 con 0x7fba1c0b1750 2023-10-08T07:10:41.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.664+0000 7fba21597640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c1f3d70 con 0x7fba1c06d240 2023-10-08T07:10:41.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.664+0000 7fba21597640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c1f4050 con 0x7fba1c15aad0 2023-10-08T07:10:41.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.665+0000 7fba13fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c0b1750 0x7fba1c1f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.665+0000 7fba13fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c0b1750 0x7fba1c1f3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48650/0 (socket says 172.21.15.70:48650) 2023-10-08T07:10:41.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.665+0000 7fba13fff640 1 -- 172.21.15.70:0/2399240369 learned_addr learned my addr 172.21.15.70:0/2399240369 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:41.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.665+0000 7fba13fff640 1 -- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c15aad0 msgr2=0x7fba1c0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:41.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.665+0000 7fba13fff640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c15aad0 0x7fba1c0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.665+0000 7fba13fff640 1 -- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c06d240 msgr2=0x7fba1c157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:41.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.665+0000 7fba1affd640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c06d240 0x7fba1c157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.665+0000 7fba13fff640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c06d240 0x7fba1c157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:41.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.665+0000 7fba13fff640 1 -- 172.21.15.70:0/2399240369 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba14009b30 con 0x7fba1c0b1750 2023-10-08T07:10:41.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.665+0000 7fba13fff640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c0b1750 0x7fba1c1f3400 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c7f0 tx=0x7fba0400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:41.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.666+0000 7fba137fe640 1 -- 172.21.15.70:0/2399240369 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04007a30 con 0x7fba1c0b1750 2023-10-08T07:10:41.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.666+0000 7fba137fe640 1 -- 172.21.15.70:0/2399240369 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04007b90 con 0x7fba1c0b1750 2023-10-08T07:10:41.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.666+0000 7fba137fe640 1 -- 172.21.15.70:0/2399240369 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0400ce40 con 0x7fba1c0b1750 2023-10-08T07:10:41.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.666+0000 7fba21597640 1 -- 172.21.15.70:0/2399240369 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c1575e0 con 0x7fba1c0b1750 2023-10-08T07:10:41.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.666+0000 7fba21597640 1 -- 172.21.15.70:0/2399240369 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c158320 con 0x7fba1c0b1750 2023-10-08T07:10:41.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.667+0000 7fba21597640 1 -- 172.21.15.70:0/2399240369 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c157b00 con 0x7fba1c0b1750 2023-10-08T07:10:41.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.668+0000 7fba137fe640 1 -- 172.21.15.70:0/2399240369 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fba040113f0 con 0x7fba1c0b1750 2023-10-08T07:10:41.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.669+0000 7fba137fe640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9fc06af90 0x7fb9fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:41.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.669+0000 7fba137fe640 1 -- 172.21.15.70:0/2399240369 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fba04089840 con 0x7fba1c0b1750 2023-10-08T07:10:41.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.670+0000 7fba1affd640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9fc06af90 0x7fb9fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:41.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.671+0000 7fba1affd640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9fc06af90 0x7fb9fc06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00a750 tx=0x7fba0c008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:41.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.674+0000 7fba137fe640 1 -- 172.21.15.70:0/2399240369 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fba04054900 con 0x7fba1c0b1750 2023-10-08T07:10:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:41.863+0000 7fba21597640 1 -- 172.21.15.70:0/2399240369 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fba1c04e160 con 0x7fba1c0b1750 2023-10-08T07:10:42.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.634+0000 7fba137fe640 1 -- 172.21.15.70:0/2399240369 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v213) v1 ==== 138+0+0 (secure 0 0 0) 0x7fba0401f0b0 con 0x7fba1c0b1750 2023-10-08T07:10:42.635 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T07:10:42.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.637+0000 7fba21597640 1 -- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9fc06af90 msgr2=0x7fb9fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:42.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.637+0000 7fba21597640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9fc06af90 0x7fb9fc06d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00a750 tx=0x7fba0c008040 comp rx=0 tx=0).stop 2023-10-08T07:10:42.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.637+0000 7fba21597640 1 -- 172.21.15.70:0/2399240369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c0b1750 msgr2=0x7fba1c1f3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:42.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.637+0000 7fba21597640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c0b1750 0x7fba1c1f3400 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c7f0 tx=0x7fba0400cbf0 comp rx=0 tx=0).stop 2023-10-08T07:10:42.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.638+0000 7fba21597640 1 -- 172.21.15.70:0/2399240369 shutdown_connections 2023-10-08T07:10:42.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.638+0000 7fba21597640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fb9fc06af90 0x7fb9fc06d450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:42.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.638+0000 7fba21597640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c15aad0 0x7fba1c0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:42.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.638+0000 7fba21597640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba1c0b1750 0x7fba1c1f3400 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:42.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.638+0000 7fba21597640 1 --2- 172.21.15.70:0/2399240369 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c06d240 0x7fba1c157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:42.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.638+0000 7fba21597640 1 -- 172.21.15.70:0/2399240369 >> 172.21.15.70:0/2399240369 conn(0x7fba1c069d50 msgr2=0x7fba1c0cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:42.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.638+0000 7fba21597640 1 -- 172.21.15.70:0/2399240369 shutdown_connections 2023-10-08T07:10:42.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.638+0000 7fba21597640 1 -- 172.21.15.70:0/2399240369 wait complete. 2023-10-08T07:10:42.663 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T07:10:42.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.868+0000 7f6fce602640 1 Processor -- start 2023-10-08T07:10:42.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.869+0000 7f6fce602640 1 -- start start 2023-10-08T07:10:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.870+0000 7f6fce602640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6fc815d580 0x7f6fc80b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.870+0000 7f6fce602640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc80cb0d0 con 0x7f6fc80cb500 2023-10-08T07:10:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.870+0000 7f6fce602640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc815da60 con 0x7f6fc80b3a00 2023-10-08T07:10:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.870+0000 7f6fce602640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc815dd40 con 0x7f6fc815d580 2023-10-08T07:10:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.871+0000 7f6fbffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6fc815d580 0x7f6fc80b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.871+0000 7f6fccb9f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6fc80cb500 0x7f6fc8070700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41678/0 (socket says 172.21.15.70:41678) 2023-10-08T07:10:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.871+0000 7f6fccb9f640 1 -- 172.21.15.70:0/3028538104 learned_addr learned my addr 172.21.15.70:0/3028538104 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.871+0000 7f6fbffff640 1 -- 172.21.15.70:0/3028538104 >> v1:172.21.15.181:6789/0 conn(0x7f6fc80b3a00 legacy=0x7f6fc806d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.871+0000 7f6fbffff640 1 -- 172.21.15.70:0/3028538104 >> v1:172.21.15.70:6789/0 conn(0x7f6fc80cb500 legacy=0x7f6fc8070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.871+0000 7f6fbffff640 1 -- 172.21.15.70:0/3028538104 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc815e020 con 0x7f6fc815d580 2023-10-08T07:10:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.871+0000 7f6fc77fe640 1 -- 172.21.15.70:0/3028538104 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031803407 0 0) 0x7f6fc80cb0d0 con 0x7f6fc80cb500 2023-10-08T07:10:42.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.872+0000 7f6fbffff640 1 --2- 172.21.15.70:0/3028538104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc815d580 0x7f6fc80b2bc0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00097a0 tx=0x7f6fb002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b1d491f130edb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:42.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.873+0000 7f6fc77fe640 1 -- 172.21.15.70:0/3028538104 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0031830 con 0x7f6fc815d580 2023-10-08T07:10:42.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.873+0000 7f6fc77fe640 1 -- 172.21.15.70:0/3028538104 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb0031990 con 0x7f6fc815d580 2023-10-08T07:10:42.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.873+0000 7f6fc77fe640 1 -- 172.21.15.70:0/3028538104 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0031d20 con 0x7f6fc815d580 2023-10-08T07:10:42.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.873+0000 7f6fce602640 1 -- 172.21.15.70:0/3028538104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc815d580 msgr2=0x7f6fc80b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:42.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.873+0000 7f6fce602640 1 --2- 172.21.15.70:0/3028538104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc815d580 0x7f6fc80b2bc0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00097a0 tx=0x7f6fb002b8d0 comp rx=0 tx=0).stop 2023-10-08T07:10:42.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.873+0000 7f6fce602640 1 -- 172.21.15.70:0/3028538104 shutdown_connections 2023-10-08T07:10:42.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.873+0000 7f6fce602640 1 --2- 172.21.15.70:0/3028538104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc815d580 0x7f6fc80b2bc0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:42.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.873+0000 7f6fce602640 1 -- 172.21.15.70:0/3028538104 >> 172.21.15.70:0/3028538104 conn(0x7f6fc806a140 msgr2=0x7f6fc806a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:42.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.873+0000 7f6fce602640 1 -- 172.21.15.70:0/3028538104 shutdown_connections 2023-10-08T07:10:42.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.873+0000 7f6fce602640 1 -- 172.21.15.70:0/3028538104 wait complete. 2023-10-08T07:10:42.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.875+0000 7f6fce602640 1 Processor -- start 2023-10-08T07:10:42.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.875+0000 7f6fce602640 1 -- start start 2023-10-08T07:10:42.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.876+0000 7f6fce602640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fc80b3a00 0x7f6fc80ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:42.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.876+0000 7f6fce602640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc80cb500 0x7f6fc815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:42.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.876+0000 7f6fce602640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fc815d580 0x7f6fc81f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:42.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.876+0000 7f6fce602640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc815f620 con 0x7f6fc80cb500 2023-10-08T07:10:42.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.876+0000 7f6fce602640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc81f30e0 con 0x7f6fc80b3a00 2023-10-08T07:10:42.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.876+0000 7f6fce602640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc81f3570 con 0x7f6fc815d580 2023-10-08T07:10:42.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.876+0000 7f6fbffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc80cb500 0x7f6fc815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:42.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.876+0000 7f6fbffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc80cb500 0x7f6fc815eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48668/0 (socket says 172.21.15.70:48668) 2023-10-08T07:10:42.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.876+0000 7f6fbffff640 1 -- 172.21.15.70:0/2881159424 learned_addr learned my addr 172.21.15.70:0/2881159424 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:42.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.877+0000 7f6fc7fff640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fc80b3a00 0x7f6fc80ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:42.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.877+0000 7f6fccb9f640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fc815d580 0x7f6fc81f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:42.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.877+0000 7f6fbffff640 1 -- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fc815d580 msgr2=0x7f6fc81f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:42.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.877+0000 7f6fbffff640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fc815d580 0x7f6fc81f5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:42.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.877+0000 7f6fbffff640 1 -- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fc80b3a00 msgr2=0x7f6fc80ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:42.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.877+0000 7f6fbffff640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fc80b3a00 0x7f6fc80ac4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:42.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.877+0000 7f6fbffff640 1 -- 172.21.15.70:0/2881159424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb0009400 con 0x7f6fc80cb500 2023-10-08T07:10:42.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.877+0000 7f6fbffff640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc80cb500 0x7f6fc815eb60 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00096f0 tx=0x7f6fb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:42.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.878+0000 7f6fc57fa640 1 -- 172.21.15.70:0/2881159424 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb003aa60 con 0x7f6fc80cb500 2023-10-08T07:10:42.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.878+0000 7f6fc57fa640 1 -- 172.21.15.70:0/2881159424 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb003abc0 con 0x7f6fc80cb500 2023-10-08T07:10:42.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.878+0000 7f6fc57fa640 1 -- 172.21.15.70:0/2881159424 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00443f0 con 0x7f6fc80cb500 2023-10-08T07:10:42.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.878+0000 7f6fce602640 1 -- 172.21.15.70:0/2881159424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc81f27b0 con 0x7f6fc80cb500 2023-10-08T07:10:42.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.878+0000 7f6fce602640 1 -- 172.21.15.70:0/2881159424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc81f2d70 con 0x7f6fc80cb500 2023-10-08T07:10:42.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.879+0000 7f6fce602640 1 -- 172.21.15.70:0/2881159424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc81f3270 con 0x7f6fc80cb500 2023-10-08T07:10:42.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.881+0000 7f6fc57fa640 1 -- 172.21.15.70:0/2881159424 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f6fb004d450 con 0x7f6fc80cb500 2023-10-08T07:10:42.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.882+0000 7f6fc57fa640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f6fa806af90 0x7f6fa806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:42.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.882+0000 7f6fc57fa640 1 -- 172.21.15.70:0/2881159424 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 5663+0+0 (secure 0 0 0) 0x7f6fb00abe30 con 0x7f6fc80cb500 2023-10-08T07:10:42.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.883+0000 7f6fc7fff640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f6fa806af90 0x7f6fa806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:42.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.884+0000 7f6fc7fff640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f6fa806af90 0x7f6fa806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8004ea0 tx=0x7f6fb8009040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:42.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:42.887+0000 7f6fc57fa640 1 -- 172.21.15.70:0/2881159424 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6fb0076d60 con 0x7f6fc80cb500 2023-10-08T07:10:43.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.069+0000 7f6fce602640 1 -- 172.21.15.70:0/2881159424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6fc8071cb0 con 0x7f6fc80cb500 2023-10-08T07:10:43.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.651+0000 7f6fc57fa640 1 -- 172.21.15.70:0/2881159424 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v214) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6fb007a380 con 0x7f6fc80cb500 2023-10-08T07:10:43.652 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T07:10:43.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.655+0000 7f6fce602640 1 -- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f6fa806af90 msgr2=0x7f6fa806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.655+0000 7f6fce602640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f6fa806af90 0x7f6fa806d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8004ea0 tx=0x7f6fb8009040 comp rx=0 tx=0).stop 2023-10-08T07:10:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.655+0000 7f6fce602640 1 -- 172.21.15.70:0/2881159424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc80cb500 msgr2=0x7f6fc815eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.655+0000 7f6fce602640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc80cb500 0x7f6fc815eb60 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00096f0 tx=0x7f6fb00046c0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.657+0000 7f6fce602640 1 -- 172.21.15.70:0/2881159424 shutdown_connections 2023-10-08T07:10:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.657+0000 7f6fce602640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f6fa806af90 0x7f6fa806d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.657+0000 7f6fce602640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fc815d580 0x7f6fc81f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.657+0000 7f6fce602640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fc80cb500 0x7f6fc815eb60 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.657+0000 7f6fce602640 1 --2- 172.21.15.70:0/2881159424 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fc80b3a00 0x7f6fc80ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.657+0000 7f6fce602640 1 -- 172.21.15.70:0/2881159424 >> 172.21.15.70:0/2881159424 conn(0x7f6fc806a140 msgr2=0x7f6fc815fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:43.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.657+0000 7f6fce602640 1 -- 172.21.15.70:0/2881159424 shutdown_connections 2023-10-08T07:10:43.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.659+0000 7f6fce602640 1 -- 172.21.15.70:0/2881159424 wait complete. 2023-10-08T07:10:43.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T07:10:43.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.880+0000 7f493433c640 1 Processor -- start 2023-10-08T07:10:43.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.880+0000 7f493433c640 1 -- start start 2023-10-08T07:10:43.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.882+0000 7f493433c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0afbe0 0x7f492c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:43.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.882+0000 7f493433c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c0b2620 con 0x7f492c0cb130 2023-10-08T07:10:43.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.882+0000 7f493433c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c0b2060 con 0x7f492c06dfc0 2023-10-08T07:10:43.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.882+0000 7f493433c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0b2340 con 0x7f492c0afbe0 2023-10-08T07:10:43.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.882+0000 7f49328d9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f492c0cb130 0x7f492c06c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57662/0 (socket says 172.21.15.70:57662) 2023-10-08T07:10:43.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.882+0000 7f49328d9640 1 -- 172.21.15.70:0/2139461113 learned_addr learned my addr 172.21.15.70:0/2139461113 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:43.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49320d8640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0afbe0 0x7f492c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:43.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003980883 0 0) 0x7f492c0b2620 con 0x7f492c0cb130 2023-10-08T07:10:43.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4918003520 con 0x7f492c0cb130 2023-10-08T07:10:43.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2926359491 0 0) 0x7f4918003520 con 0x7f492c0cb130 2023-10-08T07:10:43.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0afbe0 msgr2=0x7f492c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0afbe0 0x7f492c0b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 >> v1:172.21.15.181:6789/0 conn(0x7f492c06dfc0 legacy=0x7f492c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c070350 con 0x7f492c0cb130 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f49240019f0 con 0x7f492c0cb130 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49320d8640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0afbe0 0x7f492c0b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 >> v1:172.21.15.70:6789/0 conn(0x7f492c0cb130 legacy=0x7f492c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180039a0 0x7f4918003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0afbe0 0x7f49180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4918004d00 0x7f4918005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0b2340 con 0x7f49180039a0 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491800bb10 con 0x7f4918004d00 2023-10-08T07:10:43.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.883+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491800bca0 con 0x7f492c0afbe0 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.884+0000 7f49320d8640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180039a0 0x7f4918003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.884+0000 7f49318d7640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0afbe0 0x7f49180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.884+0000 7f49320d8640 1 -- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0afbe0 msgr2=0x7f49180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.884+0000 7f49320d8640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f492c0afbe0 0x7f49180047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.884+0000 7f49320d8640 1 -- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4918004d00 msgr2=0x7f4918005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.884+0000 7f49320d8640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4918004d00 0x7f4918005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.884+0000 7f49320d8640 1 -- 172.21.15.70:0/2139461113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4918003520 con 0x7f49180039a0 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.885+0000 7f49320d8640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180039a0 0x7f4918003dc0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f49280098a0 tx=0x7f49180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994fe212b1c97b4c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.885+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928004230 con 0x7f49180039a0 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.886+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928004390 con 0x7f49180039a0 2023-10-08T07:10:43.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.886+0000 7f49310d6640 1 -- 172.21.15.70:0/2139461113 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928003830 con 0x7f49180039a0 2023-10-08T07:10:43.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.886+0000 7f493433c640 1 -- 172.21.15.70:0/2139461113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180039a0 msgr2=0x7f4918003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:43.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.886+0000 7f493433c640 1 --2- 172.21.15.70:0/2139461113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49180039a0 0x7f4918003dc0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f49280098a0 tx=0x7f49180057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.886+0000 7f49328d9640 1 -- 172.21.15.70:0/2139461113 reap_dead start 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.886+0000 7f493433c640 1 -- 172.21.15.70:0/2139461113 shutdown_connections 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.886+0000 7f493433c640 1 -- 172.21.15.70:0/2139461113 >> 172.21.15.70:0/2139461113 conn(0x7f492c06a5d0 msgr2=0x7f492c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.886+0000 7f493433c640 1 -- 172.21.15.70:0/2139461113 shutdown_connections 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.886+0000 7f493433c640 1 -- 172.21.15.70:0/2139461113 wait complete. 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.888+0000 7f493433c640 1 Processor -- start 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.888+0000 7f493433c640 1 -- start start 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.889+0000 7f493433c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49180039a0 0x7f492c1f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.889+0000 7f493433c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0c9de0 0x7f492c0ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.889+0000 7f493433c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c158850 0x7f492c158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.889+0000 7f493433c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0ca7a0 con 0x7f492c0c9de0 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.889+0000 7f493433c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c158300 con 0x7f492c158850 2023-10-08T07:10:43.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.889+0000 7f493433c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c157260 con 0x7f49180039a0 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.889+0000 7f49318d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0c9de0 0x7f492c0ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.889+0000 7f49318d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0c9de0 0x7f492c0ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60650/0 (socket says 172.21.15.70:60650) 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f49318d7640 1 -- 172.21.15.70:0/741511992 learned_addr learned my addr 172.21.15.70:0/741511992 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f49320d8640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49180039a0 0x7f492c1f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f49318d7640 1 -- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49180039a0 msgr2=0x7f492c1f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f49318d7640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49180039a0 0x7f492c1f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f49318d7640 1 -- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c158850 msgr2=0x7f492c158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f49328d9640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c158850 0x7f492c158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f49318d7640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c158850 0x7f492c158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f49318d7640 1 -- 172.21.15.70:0/741511992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c157540 con 0x7f492c0c9de0 2023-10-08T07:10:43.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f49318d7640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0c9de0 0x7f492c0ca260 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f491c00d4c0 tx=0x7f491c00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:43.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f4922ffd640 1 -- 172.21.15.70:0/741511992 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c005480 con 0x7f492c0c9de0 2023-10-08T07:10:43.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f4922ffd640 1 -- 172.21.15.70:0/741511992 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c003610 con 0x7f492c0c9de0 2023-10-08T07:10:43.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.890+0000 7f493433c640 1 -- 172.21.15.70:0/741511992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c157820 con 0x7f492c0c9de0 2023-10-08T07:10:43.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.891+0000 7f4922ffd640 1 -- 172.21.15.70:0/741511992 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0039e0 con 0x7f492c0c9de0 2023-10-08T07:10:43.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.891+0000 7f493433c640 1 -- 172.21.15.70:0/741511992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c157de0 con 0x7f492c0c9de0 2023-10-08T07:10:43.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.892+0000 7f493433c640 1 -- 172.21.15.70:0/741511992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c157f70 con 0x7f492c0c9de0 2023-10-08T07:10:43.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.893+0000 7f4922ffd640 1 -- 172.21.15.70:0/741511992 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f491c003b40 con 0x7f492c0c9de0 2023-10-08T07:10:43.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.894+0000 7f4922ffd640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f491006afc0 0x7f491006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:43.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.894+0000 7f4922ffd640 1 -- 172.21.15.70:0/741511992 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f491c089920 con 0x7f492c0c9de0 2023-10-08T07:10:43.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.894+0000 7f49320d8640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f491006afc0 0x7f491006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:43.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.896+0000 7f49320d8640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f491006afc0 0x7f491006d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4928035000 tx=0x7f4928031000 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:43.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:43.899+0000 7f4922ffd640 1 -- 172.21.15.70:0/741511992 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f491c0545d0 con 0x7f492c0c9de0 2023-10-08T07:10:44.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.125+0000 7f493433c640 1 -- 172.21.15.70:0/741511992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f492c04e160 con 0x7f492c0c9de0 2023-10-08T07:10:44.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.654+0000 7f4922ffd640 1 -- 172.21.15.70:0/741511992 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v95) v1 ==== 174+0+0 (secure 0 0 0) 0x7f491c057bf0 con 0x7f492c0c9de0 2023-10-08T07:10:44.656 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 25 and data pool 26 2023-10-08T07:10:44.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.658+0000 7f493433c640 1 -- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f491006afc0 msgr2=0x7f491006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:44.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.658+0000 7f493433c640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f491006afc0 0x7f491006d480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4928035000 tx=0x7f4928031000 comp rx=0 tx=0).stop 2023-10-08T07:10:44.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.658+0000 7f493433c640 1 -- 172.21.15.70:0/741511992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0c9de0 msgr2=0x7f492c0ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:44.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.658+0000 7f493433c640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0c9de0 0x7f492c0ca260 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f491c00d4c0 tx=0x7f491c00d980 comp rx=0 tx=0).stop 2023-10-08T07:10:44.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.658+0000 7f493433c640 1 -- 172.21.15.70:0/741511992 shutdown_connections 2023-10-08T07:10:44.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.658+0000 7f493433c640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f491006afc0 0x7f491006d480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:44.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.658+0000 7f493433c640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f492c158850 0x7f492c158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:44.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.658+0000 7f493433c640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c0c9de0 0x7f492c0ca260 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:44.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.658+0000 7f493433c640 1 --2- 172.21.15.70:0/741511992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f49180039a0 0x7f492c1f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:44.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.658+0000 7f493433c640 1 -- 172.21.15.70:0/741511992 >> 172.21.15.70:0/741511992 conn(0x7f492c06a5d0 msgr2=0x7f492c0b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:44.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.659+0000 7f493433c640 1 -- 172.21.15.70:0/741511992 shutdown_connections 2023-10-08T07:10:44.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.659+0000 7f493433c640 1 -- 172.21.15.70:0/741511992 wait complete. 2023-10-08T07:10:44.684 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:10:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.870+0000 7fa033fff640 1 Processor -- start 2023-10-08T07:10:44.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.870+0000 7fa033fff640 1 -- start start 2023-10-08T07:10:44.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.872+0000 7fa033fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03415ab30 0x7fa03415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:44.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.872+0000 7fa033fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa0340b07d0 0x7fa03406cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:44.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.872+0000 7fa033fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03415f7d0 con 0x7fa0340adc70 2023-10-08T07:10:44.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.872+0000 7fa033fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa03415f210 con 0x7fa0340b07d0 2023-10-08T07:10:44.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.872+0000 7fa033fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03415f4f0 con 0x7fa03415ab30 2023-10-08T07:10:44.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.872+0000 7fa0337fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa0340b07d0 0x7fa03406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:44.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.872+0000 7fa032ffd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03415ab30 0x7fa03415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:44.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.873+0000 7fa032ffd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03415ab30 0x7fa03415e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:52848/0 (socket says 172.21.15.70:52848) 2023-10-08T07:10:44.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.873+0000 7fa0337fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa0340b07d0 0x7fa03406cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:35920/0 (socket says 172.21.15.70:35920) 2023-10-08T07:10:44.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.873+0000 7fa032ffd640 1 -- 172.21.15.70:0/3706147582 learned_addr learned my addr 172.21.15.70:0/3706147582 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:44.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.873+0000 7fa0337fe640 1 -- 172.21.15.70:0/3706147582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03415ab30 msgr2=0x7fa03415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:44.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.873+0000 7fa0337fe640 1 --2- 172.21.15.70:0/3706147582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03415ab30 0x7fa03415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:44.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.873+0000 7fa0337fe640 1 -- 172.21.15.70:0/3706147582 >> v1:172.21.15.181:6789/0 conn(0x7fa0340adc70 legacy=0x7fa0340b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:44.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.873+0000 7fa0337fe640 1 -- 172.21.15.70:0/3706147582 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03406c550 con 0x7fa0340b07d0 2023-10-08T07:10:44.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.875+0000 7fa0337fe640 1 --2- 172.21.15.70:0/3706147582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0340b07d0 0x7fa03406cd50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009d90 tx=0x7fa02c02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c49e91cdfa75e9b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:44.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.875+0000 7fa031ffb640 1 -- 172.21.15.70:0/3706147582 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c02cc60 con 0x7fa0340b07d0 2023-10-08T07:10:44.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.875+0000 7fa031ffb640 1 -- 172.21.15.70:0/3706147582 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02c02cdc0 con 0x7fa0340b07d0 2023-10-08T07:10:44.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.875+0000 7fa031ffb640 1 -- 172.21.15.70:0/3706147582 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02c034660 con 0x7fa0340b07d0 2023-10-08T07:10:44.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.876+0000 7fa033fff640 1 -- 172.21.15.70:0/3706147582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0340b07d0 msgr2=0x7fa03406cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:44.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.876+0000 7fa033fff640 1 --2- 172.21.15.70:0/3706147582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0340b07d0 0x7fa03406cd50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009d90 tx=0x7fa02c02beb0 comp rx=0 tx=0).stop 2023-10-08T07:10:44.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.876+0000 7fa033fff640 1 -- 172.21.15.70:0/3706147582 shutdown_connections 2023-10-08T07:10:44.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.876+0000 7fa033fff640 1 --2- 172.21.15.70:0/3706147582 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0340b07d0 0x7fa03406cd50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:44.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.876+0000 7fa033fff640 1 --2- 172.21.15.70:0/3706147582 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa03415ab30 0x7fa03415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:44.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.876+0000 7fa033fff640 1 -- 172.21.15.70:0/3706147582 >> 172.21.15.70:0/3706147582 conn(0x7fa0340cb0c0 msgr2=0x7fa0340cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:44.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.876+0000 7fa033fff640 1 -- 172.21.15.70:0/3706147582 shutdown_connections 2023-10-08T07:10:44.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.876+0000 7fa033fff640 1 -- 172.21.15.70:0/3706147582 wait complete. 2023-10-08T07:10:44.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.878+0000 7fa033fff640 1 Processor -- start 2023-10-08T07:10:44.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.878+0000 7fa033fff640 1 -- start start 2023-10-08T07:10:44.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.879+0000 7fa033fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0340adc70 0x7fa0340c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:44.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.879+0000 7fa033fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0340b07d0 0x7fa0340c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:44.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.879+0000 7fa033fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03415ab30 0x7fa0340c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:44.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.879+0000 7fa033fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0340c6a10 con 0x7fa0340adc70 2023-10-08T07:10:44.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.879+0000 7fa033fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0340c7820 con 0x7fa03415ab30 2023-10-08T07:10:44.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.879+0000 7fa033fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0340c5710 con 0x7fa0340b07d0 2023-10-08T07:10:44.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.879+0000 7fa032ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0340adc70 0x7fa0340c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:44.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.879+0000 7fa032ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0340adc70 0x7fa0340c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60658/0 (socket says 172.21.15.70:60658) 2023-10-08T07:10:44.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.879+0000 7fa032ffd640 1 -- 172.21.15.70:0/3146865627 learned_addr learned my addr 172.21.15.70:0/3146865627 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:44.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.880+0000 7fa032ffd640 1 -- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0340b07d0 msgr2=0x7fa0340c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:44.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.880+0000 7fa032ffd640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0340b07d0 0x7fa0340c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:44.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.880+0000 7fa032ffd640 1 -- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03415ab30 msgr2=0x7fa0340c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:44.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.880+0000 7fa032ffd640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03415ab30 0x7fa0340c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:44.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.880+0000 7fa032ffd640 1 -- 172.21.15.70:0/3146865627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02c009a70 con 0x7fa0340adc70 2023-10-08T07:10:44.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.880+0000 7fa032ffd640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0340adc70 0x7fa0340c64d0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fa02400e840 tx=0x7fa02400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:44.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.880+0000 7fa01bfff640 1 -- 172.21.15.70:0/3146865627 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02400c8e0 con 0x7fa0340adc70 2023-10-08T07:10:44.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.880+0000 7fa01bfff640 1 -- 172.21.15.70:0/3146865627 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02400ca40 con 0x7fa0340adc70 2023-10-08T07:10:44.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.880+0000 7fa01bfff640 1 -- 172.21.15.70:0/3146865627 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02400cdf0 con 0x7fa0340adc70 2023-10-08T07:10:44.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.880+0000 7fa033fff640 1 -- 172.21.15.70:0/3146865627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0340c59f0 con 0x7fa0340adc70 2023-10-08T07:10:44.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.881+0000 7fa033fff640 1 -- 172.21.15.70:0/3146865627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0340c5fb0 con 0x7fa0340adc70 2023-10-08T07:10:44.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.882+0000 7fa033fff640 1 -- 172.21.15.70:0/3146865627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03404e160 con 0x7fa0340adc70 2023-10-08T07:10:44.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.883+0000 7fa01bfff640 1 -- 172.21.15.70:0/3146865627 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa024013470 con 0x7fa0340adc70 2023-10-08T07:10:44.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.884+0000 7fa01bfff640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa02006af90 0x7fa02006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:44.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.884+0000 7fa01bfff640 1 -- 172.21.15.70:0/3146865627 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa024089a00 con 0x7fa0340adc70 2023-10-08T07:10:44.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.884+0000 7fa0327fc640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa02006af90 0x7fa02006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:44.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.885+0000 7fa0327fc640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa02006af90 0x7fa02006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa028000f60 tx=0x7fa0280079b0 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:44.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:44.888+0000 7fa01bfff640 1 -- 172.21.15.70:0/3146865627 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0240547e0 con 0x7fa0340adc70 2023-10-08T07:10:45.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.080+0000 7fa033fff640 1 -- 172.21.15.70:0/3146865627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0340ae100 con 0x7fa0340adc70 2023-10-08T07:10:45.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.083+0000 7fa01bfff640 1 -- 172.21.15.70:0/3146865627 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+16907 (secure 0 0 0) 0x7fa024017090 con 0x7fa0340adc70 2023-10-08T07:10:45.084 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:45.085 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":215,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:10:44.127293+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:10:41.917258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-08T07:10:43.125413+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:10:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 -- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa02006af90 msgr2=0x7fa02006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa02006af90 0x7fa02006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa028000f60 tx=0x7fa0280079b0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 -- 172.21.15.70:0/3146865627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0340adc70 msgr2=0x7fa0340c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0340adc70 0x7fa0340c64d0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fa02400e840 tx=0x7fa02400ed00 comp rx=0 tx=0).stop 2023-10-08T07:10:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 -- 172.21.15.70:0/3146865627 shutdown_connections 2023-10-08T07:10:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa02006af90 0x7fa02006d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa03415ab30 0x7fa0340c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0340b07d0 0x7fa0340c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 --2- 172.21.15.70:0/3146865627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0340adc70 0x7fa0340c64d0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 -- 172.21.15.70:0/3146865627 >> 172.21.15.70:0/3146865627 conn(0x7fa0340cb0c0 msgr2=0x7fa0340cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:45.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 -- 172.21.15.70:0/3146865627 shutdown_connections 2023-10-08T07:10:45.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.086+0000 7fa033fff640 1 -- 172.21.15.70:0/3146865627 wait complete. 2023-10-08T07:10:45.105 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:10:45.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.320+0000 7f467b59e640 1 Processor -- start 2023-10-08T07:10:45.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.320+0000 7f467b59e640 1 -- start start 2023-10-08T07:10:45.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.321+0000 7f467b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467c0adc70 0x7f467c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.321+0000 7f467b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f467c15af10 0x7f467c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.321+0000 7f467b59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c15f7d0 con 0x7f467c15ab30 2023-10-08T07:10:45.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.321+0000 7f467b59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c15efb0 con 0x7f467c15af10 2023-10-08T07:10:45.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.321+0000 7f467b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c15f210 con 0x7f467c0adc70 2023-10-08T07:10:45.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.322+0000 7f467ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f467c15af10 0x7f467c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:45.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.322+0000 7f467ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f467c15af10 0x7f467c0b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60674/0 (socket says 172.21.15.70:60674) 2023-10-08T07:10:45.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.322+0000 7f467ad9d640 1 -- 172.21.15.70:0/3405249495 learned_addr learned my addr 172.21.15.70:0/3405249495 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:45.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.322+0000 7f4673fff640 1 --2- 172.21.15.70:0/3405249495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467c0adc70 0x7f467c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:45.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.322+0000 7f467ad9d640 1 -- 172.21.15.70:0/3405249495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467c0adc70 msgr2=0x7f467c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:45.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.322+0000 7f467ad9d640 1 --2- 172.21.15.70:0/3405249495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467c0adc70 0x7f467c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.322+0000 7f467ad9d640 1 -- 172.21.15.70:0/3405249495 >> v1:172.21.15.181:6789/0 conn(0x7f467c15ab30 legacy=0x7f467c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:45.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.322+0000 7f467ad9d640 1 -- 172.21.15.70:0/3405249495 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c15f4f0 con 0x7f467c15af10 2023-10-08T07:10:45.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.322+0000 7f4673fff640 1 --2- 172.21.15.70:0/3405249495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467c0adc70 0x7f467c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:45.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.323+0000 7f467ad9d640 1 --2- 172.21.15.70:0/3405249495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15af10 0x7f467c0b0600 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f467c0cb000 tx=0x7f467402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255732b66a287435 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:45.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.324+0000 7f4679d9b640 1 -- 172.21.15.70:0/3405249495 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467402ca30 con 0x7f467c15af10 2023-10-08T07:10:45.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.324+0000 7f4679d9b640 1 -- 172.21.15.70:0/3405249495 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467402cb90 con 0x7f467c15af10 2023-10-08T07:10:45.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.324+0000 7f4679d9b640 1 -- 172.21.15.70:0/3405249495 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46740343f0 con 0x7f467c15af10 2023-10-08T07:10:45.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.324+0000 7f467b59e640 1 -- 172.21.15.70:0/3405249495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15af10 msgr2=0x7f467c0b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.324+0000 7f467b59e640 1 --2- 172.21.15.70:0/3405249495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15af10 0x7f467c0b0600 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f467c0cb000 tx=0x7f467402be90 comp rx=0 tx=0).stop 2023-10-08T07:10:45.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.324+0000 7f467b59e640 1 -- 172.21.15.70:0/3405249495 shutdown_connections 2023-10-08T07:10:45.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.324+0000 7f467b59e640 1 --2- 172.21.15.70:0/3405249495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15af10 0x7f467c0b0600 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.324+0000 7f467b59e640 1 --2- 172.21.15.70:0/3405249495 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467c0adc70 0x7f467c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.324+0000 7f467b59e640 1 -- 172.21.15.70:0/3405249495 >> 172.21.15.70:0/3405249495 conn(0x7f467c0cb0c0 msgr2=0x7f467c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:45.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.325+0000 7f467b59e640 1 -- 172.21.15.70:0/3405249495 shutdown_connections 2023-10-08T07:10:45.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.325+0000 7f467b59e640 1 -- 172.21.15.70:0/3405249495 wait complete. 2023-10-08T07:10:45.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.326+0000 7f467b59e640 1 Processor -- start 2023-10-08T07:10:45.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.327+0000 7f467b59e640 1 -- start start 2023-10-08T07:10:45.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c0adc70 0x7f467c0c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467c15ab30 0x7f467c0af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c15af10 0x7f467c155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c15c570 con 0x7f467c0adc70 2023-10-08T07:10:45.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467b59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0c7d90 con 0x7f467c15af10 2023-10-08T07:10:45.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c15b960 con 0x7f467c15ab30 2023-10-08T07:10:45.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c0adc70 0x7f467c0c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:45.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c0adc70 0x7f467c0c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60678/0 (socket says 172.21.15.70:60678) 2023-10-08T07:10:45.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467a59c640 1 -- 172.21.15.70:0/3772453017 learned_addr learned my addr 172.21.15.70:0/3772453017 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:45.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467a59c640 1 -- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467c15ab30 msgr2=0x7f467c0af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467a59c640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467c15ab30 0x7f467c0af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467a59c640 1 -- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c15af10 msgr2=0x7f467c155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467a59c640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c15af10 0x7f467c155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.328+0000 7f467a59c640 1 -- 172.21.15.70:0/3772453017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4674009b30 con 0x7f467c0adc70 2023-10-08T07:10:45.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.329+0000 7f467a59c640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c0adc70 0x7f467c0c7850 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c460 tx=0x7f466c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:45.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.329+0000 7f4672ffd640 1 -- 172.21.15.70:0/3772453017 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c012830 con 0x7f467c0adc70 2023-10-08T07:10:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.329+0000 7f4672ffd640 1 -- 172.21.15.70:0/3772453017 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466c0129d0 con 0x7f467c0adc70 2023-10-08T07:10:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.329+0000 7f467b59e640 1 -- 172.21.15.70:0/3772453017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c15bc40 con 0x7f467c0adc70 2023-10-08T07:10:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.329+0000 7f4672ffd640 1 -- 172.21.15.70:0/3772453017 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466c012d80 con 0x7f467c0adc70 2023-10-08T07:10:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.329+0000 7f467b59e640 1 -- 172.21.15.70:0/3772453017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c15c200 con 0x7f467c0adc70 2023-10-08T07:10:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.330+0000 7f467b59e640 1 -- 172.21.15.70:0/3772453017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c0cab60 con 0x7f467c0adc70 2023-10-08T07:10:45.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.331+0000 7f4672ffd640 1 -- 172.21.15.70:0/3772453017 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f466c01b400 con 0x7f467c0adc70 2023-10-08T07:10:45.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.332+0000 7f4672ffd640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f465c06afc0 0x7f465c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.332+0000 7f4672ffd640 1 -- 172.21.15.70:0/3772453017 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f466c0898c0 con 0x7f467c0adc70 2023-10-08T07:10:45.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.333+0000 7f4673fff640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f465c06afc0 0x7f465c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:45.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.334+0000 7f4673fff640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f465c06afc0 0x7f465c06d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4664006fd0 tx=0x7f4664008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:45.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.336+0000 7f4672ffd640 1 -- 172.21.15.70:0/3772453017 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f466c0546a0 con 0x7f467c0adc70 2023-10-08T07:10:45.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.525+0000 7f467b59e640 1 -- 172.21.15.70:0/3772453017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f467c06c830 con 0x7f467c0adc70 2023-10-08T07:10:45.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.527+0000 7f4672ffd640 1 -- 172.21.15.70:0/3772453017 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f466c057cc0 con 0x7f467c0adc70 2023-10-08T07:10:45.528 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:45.529 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":216,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:10:45.193352+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:10:41.917258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-08T07:10:43.125413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:10:45.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 -- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f465c06afc0 msgr2=0x7f465c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f465c06afc0 0x7f465c06d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4664006fd0 tx=0x7f4664008040 comp rx=0 tx=0).stop 2023-10-08T07:10:45.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 -- 172.21.15.70:0/3772453017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c0adc70 msgr2=0x7f467c0c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c0adc70 0x7f467c0c7850 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c460 tx=0x7f466c00c920 comp rx=0 tx=0).stop 2023-10-08T07:10:45.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 -- 172.21.15.70:0/3772453017 shutdown_connections 2023-10-08T07:10:45.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f465c06afc0 0x7f465c06d480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f467c15af10 0x7f467c155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467c15ab30 0x7f467c0af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 --2- 172.21.15.70:0/3772453017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c0adc70 0x7f467c0c7850 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 -- 172.21.15.70:0/3772453017 >> 172.21.15.70:0/3772453017 conn(0x7f467c0cb0c0 msgr2=0x7f467c0ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:45.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 -- 172.21.15.70:0/3772453017 shutdown_connections 2023-10-08T07:10:45.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.530+0000 7f467b59e640 1 -- 172.21.15.70:0/3772453017 wait complete. 2023-10-08T07:10:45.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T07:10:45.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.771+0000 7f72c350a640 1 Processor -- start 2023-10-08T07:10:45.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.771+0000 7f72c350a640 1 -- start start 2023-10-08T07:10:45.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.773+0000 7f72c350a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72bc1599d0 0x7f72bc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.773+0000 7f72c350a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 0x7f72bc0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.773+0000 7f72c350a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72bc159db0 con 0x7f72bc0b3c50 2023-10-08T07:10:45.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.773+0000 7f72c350a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc071fe0 con 0x7f72bc1599d0 2023-10-08T07:10:45.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.773+0000 7f72c350a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc0b44c0 con 0x7f72bc0cb5e0 2023-10-08T07:10:45.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.773+0000 7f72c1d07640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f72bc0b3c50 0x7f72bc0caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57676/0 (socket says 172.21.15.70:57676) 2023-10-08T07:10:45.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.773+0000 7f72c1d07640 1 -- 172.21.15.70:0/92919992 learned_addr learned my addr 172.21.15.70:0/92919992 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:45.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.773+0000 7f72c2508640 1 --2- 172.21.15.70:0/92919992 >> v2:172.21.15.70:3300/0 conn(0x7f72bc1599d0 0x7f72bc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:45.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.774+0000 7f72c2d09640 1 --2- 172.21.15.70:0/92919992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 0x7f72bc0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:45.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.774+0000 7f72c1506640 1 -- 172.21.15.70:0/92919992 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216742214 0 0) 0x7f72ac001110 con 0x7f72bc0b3c50 2023-10-08T07:10:45.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.774+0000 7f72c1506640 1 -- 172.21.15.70:0/92919992 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a8003520 con 0x7f72bc0b3c50 2023-10-08T07:10:45.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.774+0000 7f72c2508640 1 -- 172.21.15.70:0/92919992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 msgr2=0x7f72bc0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:45.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.774+0000 7f72c2508640 1 --2- 172.21.15.70:0/92919992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 0x7f72bc0cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.774+0000 7f72c2508640 1 -- 172.21.15.70:0/92919992 >> v1:172.21.15.70:6789/0 conn(0x7f72bc0b3c50 legacy=0x7f72bc0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.774+0000 7f72c2508640 1 -- 172.21.15.70:0/92919992 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc0713d0 con 0x7f72bc1599d0 2023-10-08T07:10:45.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.775+0000 7f72c2508640 1 --2- 172.21.15.70:0/92919992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc1599d0 0x7f72bc0cdcd0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009d90 tx=0x7f72b802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f5218ad107ea96 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:45.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c1506640 1 -- 172.21.15.70:0/92919992 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b802cc50 con 0x7f72bc1599d0 2023-10-08T07:10:45.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c1506640 1 -- 172.21.15.70:0/92919992 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b802cdb0 con 0x7f72bc1599d0 2023-10-08T07:10:45.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c1506640 1 -- 172.21.15.70:0/92919992 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8034640 con 0x7f72bc1599d0 2023-10-08T07:10:45.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c350a640 1 -- 172.21.15.70:0/92919992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc1599d0 msgr2=0x7f72bc0cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c350a640 1 --2- 172.21.15.70:0/92919992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc1599d0 0x7f72bc0cdcd0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009d90 tx=0x7f72b802bea0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c350a640 1 -- 172.21.15.70:0/92919992 shutdown_connections 2023-10-08T07:10:45.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c350a640 1 --2- 172.21.15.70:0/92919992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 0x7f72bc0cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c350a640 1 --2- 172.21.15.70:0/92919992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc1599d0 0x7f72bc0cdcd0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c350a640 1 -- 172.21.15.70:0/92919992 >> 172.21.15.70:0/92919992 conn(0x7f72bc06c050 msgr2=0x7f72bc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:45.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c350a640 1 -- 172.21.15.70:0/92919992 shutdown_connections 2023-10-08T07:10:45.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.776+0000 7f72c350a640 1 -- 172.21.15.70:0/92919992 wait complete. 2023-10-08T07:10:45.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.778+0000 7f72c350a640 1 Processor -- start 2023-10-08T07:10:45.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.778+0000 7f72c350a640 1 -- start start 2023-10-08T07:10:45.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.779+0000 7f72c350a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc0b3c50 0x7f72bc1f0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.779+0000 7f72c350a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 0x7f72bc157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.779+0000 7f72c350a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc1599d0 0x7f72bc1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.779+0000 7f72c350a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc145000 con 0x7f72bc0b3c50 2023-10-08T07:10:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.779+0000 7f72c350a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc1452e0 con 0x7f72bc1599d0 2023-10-08T07:10:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.779+0000 7f72c350a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc1455c0 con 0x7f72bc0cb5e0 2023-10-08T07:10:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.779+0000 7f72c2508640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc0b3c50 0x7f72bc1f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c2508640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc0b3c50 0x7f72bc1f0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60688/0 (socket says 172.21.15.70:60688) 2023-10-08T07:10:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c2508640 1 -- 172.21.15.70:0/1536510164 learned_addr learned my addr 172.21.15.70:0/1536510164 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c1d07640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 0x7f72bc157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c2d09640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc1599d0 0x7f72bc1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:45.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c2508640 1 -- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 msgr2=0x7f72bc157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c2508640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 0x7f72bc157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c2508640 1 -- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc1599d0 msgr2=0x7f72bc1533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:45.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c2508640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc1599d0 0x7f72bc1533c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:45.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c2508640 1 -- 172.21.15.70:0/1536510164 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8009a70 con 0x7f72bc0b3c50 2023-10-08T07:10:45.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c1d07640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 0x7f72bc157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:45.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.780+0000 7f72c2508640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc0b3c50 0x7f72bc1f0800 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009eb0 tx=0x7f72b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:45.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.781+0000 7f72b2ffd640 1 -- 172.21.15.70:0/1536510164 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b803e610 con 0x7f72bc0b3c50 2023-10-08T07:10:45.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.781+0000 7f72b2ffd640 1 -- 172.21.15.70:0/1536510164 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b803e770 con 0x7f72bc0b3c50 2023-10-08T07:10:45.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.781+0000 7f72c350a640 1 -- 172.21.15.70:0/1536510164 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc1458a0 con 0x7f72bc0b3c50 2023-10-08T07:10:45.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.781+0000 7f72b2ffd640 1 -- 172.21.15.70:0/1536510164 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b803eb20 con 0x7f72bc0b3c50 2023-10-08T07:10:45.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.781+0000 7f72c350a640 1 -- 172.21.15.70:0/1536510164 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc1f0d40 con 0x7f72bc0b3c50 2023-10-08T07:10:45.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.782+0000 7f72c350a640 1 -- 172.21.15.70:0/1536510164 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72bc145dc0 con 0x7f72bc0b3c50 2023-10-08T07:10:45.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.783+0000 7f72b2ffd640 1 -- 172.21.15.70:0/1536510164 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f72b803b050 con 0x7f72bc0b3c50 2023-10-08T07:10:45.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.784+0000 7f72b2ffd640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f72a006afc0 0x7f72a006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:45.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.784+0000 7f72b2ffd640 1 -- 172.21.15.70:0/1536510164 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f72b80ac1e0 con 0x7f72bc0b3c50 2023-10-08T07:10:45.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.785+0000 7f72c1d07640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f72a006afc0 0x7f72a006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:45.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.786+0000 7f72c1d07640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f72a006afc0 0x7f72a006d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f72ac001270 tx=0x7f72ac009040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:45.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:45.788+0000 7f72b2ffd640 1 -- 172.21.15.70:0/1536510164 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72b8076fb0 con 0x7f72bc0b3c50 2023-10-08T07:10:46.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.014+0000 7f72c350a640 1 -- 172.21.15.70:0/1536510164 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f72bc074150 con 0x7f72bc0b3c50 2023-10-08T07:10:46.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.676+0000 7f72b2ffd640 1 -- 172.21.15.70:0/1536510164 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7f72b8009450 con 0x7f72bc0b3c50 2023-10-08T07:10:46.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.680+0000 7f72c350a640 1 -- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f72a006afc0 msgr2=0x7f72a006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:46.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.680+0000 7f72c350a640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f72a006afc0 0x7f72a006d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f72ac001270 tx=0x7f72ac009040 comp rx=0 tx=0).stop 2023-10-08T07:10:46.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.680+0000 7f72c350a640 1 -- 172.21.15.70:0/1536510164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc0b3c50 msgr2=0x7f72bc1f0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:46.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.680+0000 7f72c350a640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc0b3c50 0x7f72bc1f0800 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009eb0 tx=0x7f72b8004a40 comp rx=0 tx=0).stop 2023-10-08T07:10:46.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.681+0000 7f72c350a640 1 -- 172.21.15.70:0/1536510164 shutdown_connections 2023-10-08T07:10:46.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.681+0000 7f72c350a640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f72a006afc0 0x7f72a006d480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:46.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.681+0000 7f72c350a640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f72bc1599d0 0x7f72bc1533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:46.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.681+0000 7f72c350a640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f72bc0cb5e0 0x7f72bc157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:46.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.681+0000 7f72c350a640 1 --2- 172.21.15.70:0/1536510164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72bc0b3c50 0x7f72bc1f0800 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:46.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.681+0000 7f72c350a640 1 -- 172.21.15.70:0/1536510164 >> 172.21.15.70:0/1536510164 conn(0x7f72bc06c050 msgr2=0x7f72bc0cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:46.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.681+0000 7f72c350a640 1 -- 172.21.15.70:0/1536510164 shutdown_connections 2023-10-08T07:10:46.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.681+0000 7f72c350a640 1 -- 172.21.15.70:0/1536510164 wait complete. 2023-10-08T07:10:46.704 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T07:10:46.704 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T07:10:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.922+0000 7fed365dc640 1 Processor -- start 2023-10-08T07:10:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.922+0000 7fed365dc640 1 -- start start 2023-10-08T07:10:46.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.924+0000 7fed365dc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed30160660 0x7fed30070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:46.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.924+0000 7fed365dc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3006e150 con 0x7fed300cb620 2023-10-08T07:10:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.924+0000 7fed365dc640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3006de70 con 0x7fed300c9910 2023-10-08T07:10:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.924+0000 7fed365dc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed300c8e30 con 0x7fed30160660 2023-10-08T07:10:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.924+0000 7fed34b79640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fed300cb620 0x7fed300b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57684/0 (socket says 172.21.15.70:57684) 2023-10-08T07:10:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.924+0000 7fed2ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed30160660 0x7fed30070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.924+0000 7fed34b79640 1 -- 172.21.15.70:0/921456972 learned_addr learned my addr 172.21.15.70:0/921456972 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.925+0000 7fed2ffff640 1 -- 172.21.15.70:0/921456972 >> v1:172.21.15.181:6789/0 conn(0x7fed300c9910 legacy=0x7fed300caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.925+0000 7fed2ffff640 1 -- 172.21.15.70:0/921456972 >> v1:172.21.15.70:6789/0 conn(0x7fed300cb620 legacy=0x7fed300b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.925+0000 7fed2ffff640 1 -- 172.21.15.70:0/921456972 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed300c9110 con 0x7fed30160660 2023-10-08T07:10:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.926+0000 7fed2ffff640 1 --2- 172.21.15.70:0/921456972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed30160660 0x7fed30070cc0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fed3006e340 tx=0x7fed2002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6657395ab79194e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.926+0000 7fed2f7fe640 1 -- 172.21.15.70:0/921456972 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20031780 con 0x7fed30160660 2023-10-08T07:10:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.926+0000 7fed2f7fe640 1 -- 172.21.15.70:0/921456972 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed200318e0 con 0x7fed30160660 2023-10-08T07:10:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.926+0000 7fed2f7fe640 1 -- 172.21.15.70:0/921456972 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20031c70 con 0x7fed30160660 2023-10-08T07:10:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.927+0000 7fed365dc640 1 -- 172.21.15.70:0/921456972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed30160660 msgr2=0x7fed30070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.927+0000 7fed365dc640 1 --2- 172.21.15.70:0/921456972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed30160660 0x7fed30070cc0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fed3006e340 tx=0x7fed2002b820 comp rx=0 tx=0).stop 2023-10-08T07:10:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.927+0000 7fed365dc640 1 -- 172.21.15.70:0/921456972 shutdown_connections 2023-10-08T07:10:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.927+0000 7fed365dc640 1 --2- 172.21.15.70:0/921456972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed30160660 0x7fed30070cc0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.927+0000 7fed365dc640 1 -- 172.21.15.70:0/921456972 >> 172.21.15.70:0/921456972 conn(0x7fed3006a5d0 msgr2=0x7fed300b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.927+0000 7fed365dc640 1 -- 172.21.15.70:0/921456972 shutdown_connections 2023-10-08T07:10:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.927+0000 7fed365dc640 1 -- 172.21.15.70:0/921456972 wait complete. 2023-10-08T07:10:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.929+0000 7fed365dc640 1 Processor -- start 2023-10-08T07:10:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.929+0000 7fed365dc640 1 -- start start 2023-10-08T07:10:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed365dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed300c9910 0x7fed3015e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed365dc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed300cb620 0x7fed30158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed365dc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed30160660 0x7fed301589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed365dc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed30158fc0 con 0x7fed300c9910 2023-10-08T07:10:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed365dc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed301592a0 con 0x7fed300cb620 2023-10-08T07:10:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed365dc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed30159580 con 0x7fed30160660 2023-10-08T07:10:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed300c9910 0x7fed3015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed300c9910 0x7fed3015e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60706/0 (socket says 172.21.15.70:60706) 2023-10-08T07:10:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed2ffff640 1 -- 172.21.15.70:0/3614832979 learned_addr learned my addr 172.21.15.70:0/3614832979 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:46.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed2ffff640 1 -- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed30160660 msgr2=0x7fed301589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:46.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed2ffff640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed30160660 0x7fed301589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:46.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed27fff640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed300cb620 0x7fed30158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:46.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed2ffff640 1 -- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed300cb620 msgr2=0x7fed30158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:46.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed2ffff640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed300cb620 0x7fed30158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:46.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.931+0000 7fed2ffff640 1 -- 172.21.15.70:0/3614832979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed20009400 con 0x7fed300c9910 2023-10-08T07:10:46.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.932+0000 7fed2ffff640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed300c9910 0x7fed3015e340 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fed20009890 tx=0x7fed200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:46.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.932+0000 7fed2d7fa640 1 -- 172.21.15.70:0/3614832979 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed200328e0 con 0x7fed300c9910 2023-10-08T07:10:46.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.932+0000 7fed2d7fa640 1 -- 172.21.15.70:0/3614832979 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed20032a80 con 0x7fed300c9910 2023-10-08T07:10:46.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.932+0000 7fed2d7fa640 1 -- 172.21.15.70:0/3614832979 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed20032e30 con 0x7fed300c9910 2023-10-08T07:10:46.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.932+0000 7fed365dc640 1 -- 172.21.15.70:0/3614832979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed30159860 con 0x7fed300c9910 2023-10-08T07:10:46.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.932+0000 7fed365dc640 1 -- 172.21.15.70:0/3614832979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3015e880 con 0x7fed300c9910 2023-10-08T07:10:46.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.933+0000 7fed365dc640 1 -- 172.21.15.70:0/3614832979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed300c32a0 con 0x7fed300c9910 2023-10-08T07:10:46.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.934+0000 7fed2d7fa640 1 -- 172.21.15.70:0/3614832979 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fed2004e450 con 0x7fed300c9910 2023-10-08T07:10:46.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.935+0000 7fed2d7fa640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fed1006af90 0x7fed1006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:46.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.935+0000 7fed2d7fa640 1 -- 172.21.15.70:0/3614832979 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fed200abb80 con 0x7fed300c9910 2023-10-08T07:10:46.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.936+0000 7fed27fff640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fed1006af90 0x7fed1006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:46.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.937+0000 7fed27fff640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fed1006af90 0x7fed1006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fed30144450 tx=0x7fed18009040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:46.939+0000 7fed2d7fa640 1 -- 172.21.15.70:0/3614832979 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed200768e0 con 0x7fed300c9910 2023-10-08T07:10:47.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.164+0000 7fed365dc640 1 -- 172.21.15.70:0/3614832979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fed3004e160 con 0x7fed300c9910 2023-10-08T07:10:47.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.686+0000 7fed2d7fa640 1 -- 172.21.15.70:0/3614832979 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7fed20079f00 con 0x7fed300c9910 2023-10-08T07:10:47.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.691+0000 7fed365dc640 1 -- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fed1006af90 msgr2=0x7fed1006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:47.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.691+0000 7fed365dc640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fed1006af90 0x7fed1006d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fed30144450 tx=0x7fed18009040 comp rx=0 tx=0).stop 2023-10-08T07:10:47.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.691+0000 7fed365dc640 1 -- 172.21.15.70:0/3614832979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed300c9910 msgr2=0x7fed3015e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.691+0000 7fed365dc640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed300c9910 0x7fed3015e340 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fed20009890 tx=0x7fed200046c0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.692+0000 7fed365dc640 1 -- 172.21.15.70:0/3614832979 shutdown_connections 2023-10-08T07:10:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.692+0000 7fed365dc640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fed1006af90 0x7fed1006d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.692+0000 7fed365dc640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed30160660 0x7fed301589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.692+0000 7fed365dc640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed300cb620 0x7fed30158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.692+0000 7fed365dc640 1 --2- 172.21.15.70:0/3614832979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed300c9910 0x7fed3015e340 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.692+0000 7fed365dc640 1 -- 172.21.15.70:0/3614832979 >> 172.21.15.70:0/3614832979 conn(0x7fed3006a5d0 msgr2=0x7fed3015d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.692+0000 7fed365dc640 1 -- 172.21.15.70:0/3614832979 shutdown_connections 2023-10-08T07:10:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.692+0000 7fed365dc640 1 -- 172.21.15.70:0/3614832979 wait complete. 2023-10-08T07:10:47.712 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T07:10:47.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:10:47.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.921+0000 7fa106a4b640 1 Processor -- start 2023-10-08T07:10:47.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.921+0000 7fa106a4b640 1 -- start start 2023-10-08T07:10:47.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.922+0000 7fa106a4b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa100159ac0 0x7fa10006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:47.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.922+0000 7fa106a4b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10006dee0 con 0x7fa1000709e0 2023-10-08T07:10:47.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.922+0000 7fa106a4b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10015d850 con 0x7fa1000ad790 2023-10-08T07:10:47.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.922+0000 7fa106a4b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10015db30 con 0x7fa100159ac0 2023-10-08T07:10:47.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.923+0000 7fa104fe8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa1000709e0 0x7fa1000c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57694/0 (socket says 172.21.15.70:57694) 2023-10-08T07:10:47.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.923+0000 7fa104fe8640 1 -- 172.21.15.70:0/3771862514 learned_addr learned my addr 172.21.15.70:0/3771862514 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.923+0000 7fa0fffff640 1 --2- 172.21.15.70:0/3771862514 >> v2:172.21.15.181:3300/0 conn(0x7fa100159ac0 0x7fa10006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.923+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637777286 0 0) 0x7fa10006dee0 con 0x7fa1000709e0 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.923+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0ec003520 con 0x7fa1000709e0 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.923+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2601000971 0 0) 0x7fa0ec003520 con 0x7fa1000709e0 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 >> v2:172.21.15.181:3300/0 conn(0x7fa100159ac0 msgr2=0x7fa10006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 --2- 172.21.15.70:0/3771862514 >> v2:172.21.15.181:3300/0 conn(0x7fa100159ac0 0x7fa10006d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 >> v1:172.21.15.181:6789/0 conn(0x7fa1000ad790 legacy=0x7fa1000702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10015de10 con 0x7fa1000709e0 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa0f80019f0 con 0x7fa1000709e0 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 >> v1:172.21.15.70:6789/0 conn(0x7fa1000709e0 legacy=0x7fa1000c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:47.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 --2- 172.21.15.70:0/3771862514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0ec003970 0x7fa0ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:47.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 --2- 172.21.15.70:0/3771862514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa0ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:47.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 --2- 172.21.15.70:0/3771862514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec005180 0x7fa0ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:47.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10015db30 con 0x7fa0ec0047c0 2023-10-08T07:10:47.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec00c020 con 0x7fa0ec005180 2023-10-08T07:10:47.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec00c1e0 con 0x7fa0ec003970 2023-10-08T07:10:47.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0f7fff640 1 --2- 172.21.15.70:0/3771862514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa0ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:47.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0f7fff640 1 -- 172.21.15.70:0/3771862514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0ec003970 msgr2=0x7fa0ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa104fe8640 1 --2- 172.21.15.70:0/3771862514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec005180 0x7fa0ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0f7fff640 1 --2- 172.21.15.70:0/3771862514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0ec003970 0x7fa0ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0f7fff640 1 -- 172.21.15.70:0/3771862514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec005180 msgr2=0x7fa0ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0f7fff640 1 --2- 172.21.15.70:0/3771862514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0ec005180 0x7fa0ec00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa0f7fff640 1 -- 172.21.15.70:0/3771862514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec003520 con 0x7fa0ec0047c0 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.924+0000 7fa104fe8640 1 -- 172.21.15.70:0/3771862514 reap_dead start 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.926+0000 7fa0f7fff640 1 --2- 172.21.15.70:0/3771862514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa0ec004c40 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8001750 tx=0x7fa0ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae350d440122618a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.926+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80052d0 con 0x7fa0ec0047c0 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.926+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e802e050 con 0x7fa0ec0047c0 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.926+0000 7fa0ff7fe640 1 -- 172.21.15.70:0/3771862514 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8003830 con 0x7fa0ec0047c0 2023-10-08T07:10:47.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.926+0000 7fa106a4b640 1 -- 172.21.15.70:0/3771862514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 msgr2=0x7fa0ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.926+0000 7fa106a4b640 1 --2- 172.21.15.70:0/3771862514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa0ec004c40 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8001750 tx=0x7fa0ec0057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.926+0000 7fa106a4b640 1 -- 172.21.15.70:0/3771862514 shutdown_connections 2023-10-08T07:10:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.926+0000 7fa106a4b640 1 --2- 172.21.15.70:0/3771862514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa0ec004c40 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.926+0000 7fa106a4b640 1 -- 172.21.15.70:0/3771862514 >> 172.21.15.70:0/3771862514 conn(0x7fa10006a380 msgr2=0x7fa1000b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.927+0000 7fa106a4b640 1 -- 172.21.15.70:0/3771862514 shutdown_connections 2023-10-08T07:10:47.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.927+0000 7fa106a4b640 1 -- 172.21.15.70:0/3771862514 wait complete. 2023-10-08T07:10:47.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.928+0000 7fa106a4b640 1 Processor -- start 2023-10-08T07:10:47.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.928+0000 7fa106a4b640 1 -- start start 2023-10-08T07:10:47.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.929+0000 7fa106a4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa100155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:47.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.929+0000 7fa106a4b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa10015aa30 0x7fa10015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:47.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.929+0000 7fa106a4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1000b30d0 0x7fa1000b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:47.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.929+0000 7fa106a4b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1000b3a90 con 0x7fa0ec0047c0 2023-10-08T07:10:47.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.929+0000 7fa106a4b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10015afd0 con 0x7fa10015aa30 2023-10-08T07:10:47.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.929+0000 7fa106a4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10015b2b0 con 0x7fa1000b30d0 2023-10-08T07:10:47.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.930+0000 7fa0fffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa100155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:47.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.930+0000 7fa0fffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa100155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60718/0 (socket says 172.21.15.70:60718) 2023-10-08T07:10:47.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.930+0000 7fa0fffff640 1 -- 172.21.15.70:0/4079192565 learned_addr learned my addr 172.21.15.70:0/4079192565 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:47.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.930+0000 7fa0fffff640 1 -- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1000b30d0 msgr2=0x7fa1000b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:47.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.930+0000 7fa0fffff640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1000b30d0 0x7fa1000b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.930+0000 7fa0fffff640 1 -- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa10015aa30 msgr2=0x7fa10015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:47.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.930+0000 7fa0fffff640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa10015aa30 0x7fa10015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:47.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.930+0000 7fa0fffff640 1 -- 172.21.15.70:0/4079192565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10015b590 con 0x7fa0ec0047c0 2023-10-08T07:10:47.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.930+0000 7fa0fffff640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa100155480 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000e860 tx=0x7fa0f000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:47.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.931+0000 7fa0fd7fa640 1 -- 172.21.15.70:0/4079192565 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0009600 con 0x7fa0ec0047c0 2023-10-08T07:10:47.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.931+0000 7fa0fd7fa640 1 -- 172.21.15.70:0/4079192565 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f000c940 con 0x7fa0ec0047c0 2023-10-08T07:10:47.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.931+0000 7fa0fd7fa640 1 -- 172.21.15.70:0/4079192565 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000ccd0 con 0x7fa0ec0047c0 2023-10-08T07:10:47.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.931+0000 7fa106a4b640 1 -- 172.21.15.70:0/4079192565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10015b870 con 0x7fa0ec0047c0 2023-10-08T07:10:47.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.931+0000 7fa106a4b640 1 -- 172.21.15.70:0/4079192565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1001f2950 con 0x7fa0ec0047c0 2023-10-08T07:10:47.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.932+0000 7fa106a4b640 1 -- 172.21.15.70:0/4079192565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa100067490 con 0x7fa0ec0047c0 2023-10-08T07:10:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.933+0000 7fa0fd7fa640 1 -- 172.21.15.70:0/4079192565 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa0f0013470 con 0x7fa0ec0047c0 2023-10-08T07:10:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.934+0000 7fa0fd7fa640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa0e006af90 0x7fa0e006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:47.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.934+0000 7fa0fd7fa640 1 -- 172.21.15.70:0/4079192565 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fa0f0089ba0 con 0x7fa0ec0047c0 2023-10-08T07:10:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.934+0000 7fa0f7fff640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa0e006af90 0x7fa0e006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:47.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.936+0000 7fa0f7fff640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa0e006af90 0x7fa0e006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8033000 tx=0x7fa0e802f000 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:47.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:47.938+0000 7fa0fd7fa640 1 -- 172.21.15.70:0/4079192565 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0f0058410 con 0x7fa0ec0047c0 2023-10-08T07:10:48.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.164+0000 7fa106a4b640 1 -- 172.21.15.70:0/4079192565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa10015ba00 con 0x7fa0ec0047c0 2023-10-08T07:10:48.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.165+0000 7fa0fd7fa640 1 -- 172.21.15.70:0/4079192565 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7fa0f0057fa0 con 0x7fa0ec0047c0 2023-10-08T07:10:48.166 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:48.166 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/4185609199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4185609199},{"type":"v1","addr":"172.21.15.70:6833","nonce":4185609199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3102206290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3102206290},{"type":"v1","addr":"172.21.15.181:6837","nonce":3102206290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1824427895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1824427895},{"type":"v1","addr":"172.21.15.181:6835","nonce":1824427895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:10:44.128443+0000","modified":"2023-10-08T07:10:47.166174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/2355440757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2355440757},{"type":"v1","addr":"172.21.15.70:6835","nonce":2355440757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-08T07:10:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 99 2023-10-08T07:10:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 -- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa0e006af90 msgr2=0x7fa0e006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa0e006af90 0x7fa0e006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8033000 tx=0x7fa0e802f000 comp rx=0 tx=0).stop 2023-10-08T07:10:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 -- 172.21.15.70:0/4079192565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 msgr2=0x7fa100155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa100155480 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000e860 tx=0x7fa0f000ed20 comp rx=0 tx=0).stop 2023-10-08T07:10:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 -- 172.21.15.70:0/4079192565 shutdown_connections 2023-10-08T07:10:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fa0e006af90 0x7fa0e006d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1000b30d0 0x7fa1000b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa10015aa30 0x7fa10015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 --2- 172.21.15.70:0/4079192565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0ec0047c0 0x7fa100155480 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 -- 172.21.15.70:0/4079192565 >> 172.21.15.70:0/4079192565 conn(0x7fa10006a380 msgr2=0x7fa1000c4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 -- 172.21.15.70:0/4079192565 shutdown_connections 2023-10-08T07:10:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.168+0000 7fa106a4b640 1 -- 172.21.15.70:0/4079192565 wait complete. 2023-10-08T07:10:48.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:10:48.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.392+0000 7f009a52c640 1 Processor -- start 2023-10-08T07:10:48.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.393+0000 7f009a52c640 1 -- start start 2023-10-08T07:10:48.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.394+0000 7f009a52c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f009415ab40 0x7f00940ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.394+0000 7f009a52c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00940c90b0 0x7f00940adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.394+0000 7f009a52c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009415b2b0 con 0x7f00940cae70 2023-10-08T07:10:48.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.394+0000 7f009a52c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009415bf30 con 0x7f00940c90b0 2023-10-08T07:10:48.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.394+0000 7f009a52c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009415f7a0 con 0x7f009415ab40 2023-10-08T07:10:48.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.395+0000 7f008bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00940c90b0 0x7f00940adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.395+0000 7f0093fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f009415ab40 0x7f00940ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.395+0000 7f0093fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f009415ab40 0x7f00940ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:52936/0 (socket says 172.21.15.70:52936) 2023-10-08T07:10:48.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.395+0000 7f0093fff640 1 -- 172.21.15.70:0/296269238 learned_addr learned my addr 172.21.15.70:0/296269238 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:48.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.395+0000 7f008bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f00940c90b0 0x7f00940adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:35998/0 (socket says 172.21.15.70:35998) 2023-10-08T07:10:48.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.396+0000 7f0093fff640 1 -- 172.21.15.70:0/296269238 >> v2:172.21.15.181:3300/0 conn(0x7f00940c90b0 msgr2=0x7f00940adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:48.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.396+0000 7f0093fff640 1 --2- 172.21.15.70:0/296269238 >> v2:172.21.15.181:3300/0 conn(0x7f00940c90b0 0x7f00940adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.396+0000 7f0093fff640 1 -- 172.21.15.70:0/296269238 >> v1:172.21.15.181:6789/0 conn(0x7f00940cae70 legacy=0x7f00940702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.396+0000 7f0093fff640 1 -- 172.21.15.70:0/296269238 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009415b590 con 0x7f009415ab40 2023-10-08T07:10:48.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.397+0000 7f0093fff640 1 --2- 172.21.15.70:0/296269238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f009415ab40 0x7f00940ad720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0084009870 tx=0x7f008402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=534b13747ea99936 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:48.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.398+0000 7f00937fe640 1 -- 172.21.15.70:0/296269238 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084030840 con 0x7f009415ab40 2023-10-08T07:10:48.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.398+0000 7f00937fe640 1 -- 172.21.15.70:0/296269238 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00840309a0 con 0x7f009415ab40 2023-10-08T07:10:48.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.398+0000 7f00937fe640 1 -- 172.21.15.70:0/296269238 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084030d30 con 0x7f009415ab40 2023-10-08T07:10:48.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.399+0000 7f009a52c640 1 -- 172.21.15.70:0/296269238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f009415ab40 msgr2=0x7f00940ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.399+0000 7f009a52c640 1 --2- 172.21.15.70:0/296269238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f009415ab40 0x7f00940ad720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0084009870 tx=0x7f008402b980 comp rx=0 tx=0).stop 2023-10-08T07:10:48.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.399+0000 7f009a52c640 1 -- 172.21.15.70:0/296269238 shutdown_connections 2023-10-08T07:10:48.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.399+0000 7f009a52c640 1 --2- 172.21.15.70:0/296269238 >> v2:172.21.15.181:3300/0 conn(0x7f00940c90b0 0x7f00940adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.399+0000 7f009a52c640 1 --2- 172.21.15.70:0/296269238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f009415ab40 0x7f00940ad720 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.399+0000 7f009a52c640 1 -- 172.21.15.70:0/296269238 >> 172.21.15.70:0/296269238 conn(0x7f009406a380 msgr2=0x7f00940b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:48.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.399+0000 7f009a52c640 1 -- 172.21.15.70:0/296269238 shutdown_connections 2023-10-08T07:10:48.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.399+0000 7f009a52c640 1 -- 172.21.15.70:0/296269238 wait complete. 2023-10-08T07:10:48.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.401+0000 7f009a52c640 1 Processor -- start 2023-10-08T07:10:48.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.401+0000 7f009a52c640 1 -- start start 2023-10-08T07:10:48.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.402+0000 7f009a52c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00940c90b0 0x7f0094158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.402+0000 7f009a52c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00940cae70 0x7f00940c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.402+0000 7f009a52c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f009415ab40 0x7f0094155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.402+0000 7f009a52c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00940c4930 con 0x7f009415ab40 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.402+0000 7f009a52c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0094156410 con 0x7f00940cae70 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.402+0000 7f009a52c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0094158f40 con 0x7f00940c90b0 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.402+0000 7f0098ac9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f009415ab40 0x7f0094155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f0098ac9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f009415ab40 0x7f0094155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60722/0 (socket says 172.21.15.70:60722) 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f0098ac9640 1 -- 172.21.15.70:0/2652183297 learned_addr learned my addr 172.21.15.70:0/2652183297 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f008bfff640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00940cae70 0x7f00940c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f0093fff640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00940c90b0 0x7f0094158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f0098ac9640 1 -- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00940c90b0 msgr2=0x7f0094158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f0098ac9640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00940c90b0 0x7f0094158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f0098ac9640 1 -- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00940cae70 msgr2=0x7f00940c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f0098ac9640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00940cae70 0x7f00940c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f0098ac9640 1 -- 172.21.15.70:0/2652183297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00840094d0 con 0x7f009415ab40 2023-10-08T07:10:48.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f0093fff640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00940c90b0 0x7f0094158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:48.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.403+0000 7f0098ac9640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f009415ab40 0x7f0094155ed0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f008c00c6b0 tx=0x7f008c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:48.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.404+0000 7f00917fa640 1 -- 172.21.15.70:0/2652183297 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c003b70 con 0x7f009415ab40 2023-10-08T07:10:48.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.404+0000 7f00917fa640 1 -- 172.21.15.70:0/2652183297 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c003cf0 con 0x7f009415ab40 2023-10-08T07:10:48.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.404+0000 7f00917fa640 1 -- 172.21.15.70:0/2652183297 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c003ea0 con 0x7f009415ab40 2023-10-08T07:10:48.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.404+0000 7f009a52c640 1 -- 172.21.15.70:0/2652183297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0094159220 con 0x7f009415ab40 2023-10-08T07:10:48.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.404+0000 7f009a52c640 1 -- 172.21.15.70:0/2652183297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00941597e0 con 0x7f009415ab40 2023-10-08T07:10:48.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.405+0000 7f009a52c640 1 -- 172.21.15.70:0/2652183297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0094067490 con 0x7f009415ab40 2023-10-08T07:10:48.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.406+0000 7f00917fa640 1 -- 172.21.15.70:0/2652183297 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f008c005310 con 0x7f009415ab40 2023-10-08T07:10:48.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.407+0000 7f00917fa640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f007406af90 0x7f007406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.407+0000 7f00917fa640 1 -- 172.21.15.70:0/2652183297 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f008c089670 con 0x7f009415ab40 2023-10-08T07:10:48.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.407+0000 7f0093fff640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f007406af90 0x7f007406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.409+0000 7f0093fff640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f007406af90 0x7f007406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0084036000 tx=0x7f008403b040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:48.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.411+0000 7f00917fa640 1 -- 172.21.15.70:0/2652183297 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f008c0544e0 con 0x7f009415ab40 2023-10-08T07:10:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.599+0000 7f009a52c640 1 -- 172.21.15.70:0/2652183297 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f009406cf00 con 0x7f009415ab40 2023-10-08T07:10:48.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.601+0000 7f00917fa640 1 -- 172.21.15.70:0/2652183297 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f008c057b00 con 0x7f009415ab40 2023-10-08T07:10:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:48.603 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":216,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:10:45.193352+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:10:41.917258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-08T07:10:43.125413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:10:48.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.603+0000 7f009a52c640 1 -- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f007406af90 msgr2=0x7f007406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.603+0000 7f009a52c640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f007406af90 0x7f007406d450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0084036000 tx=0x7f008403b040 comp rx=0 tx=0).stop 2023-10-08T07:10:48.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.603+0000 7f009a52c640 1 -- 172.21.15.70:0/2652183297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f009415ab40 msgr2=0x7f0094155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.603+0000 7f009a52c640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f009415ab40 0x7f0094155ed0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f008c00c6b0 tx=0x7f008c00cb70 comp rx=0 tx=0).stop 2023-10-08T07:10:48.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.603+0000 7f009a52c640 1 -- 172.21.15.70:0/2652183297 shutdown_connections 2023-10-08T07:10:48.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.603+0000 7f009a52c640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f007406af90 0x7f007406d450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.603+0000 7f009a52c640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f009415ab40 0x7f0094155ed0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.603+0000 7f009a52c640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f00940cae70 0x7f00940c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.603+0000 7f009a52c640 1 --2- 172.21.15.70:0/2652183297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f00940c90b0 0x7f0094158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.603+0000 7f009a52c640 1 -- 172.21.15.70:0/2652183297 >> 172.21.15.70:0/2652183297 conn(0x7f009406a380 msgr2=0x7f009406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:48.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.604+0000 7f009a52c640 1 -- 172.21.15.70:0/2652183297 shutdown_connections 2023-10-08T07:10:48.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.604+0000 7f009a52c640 1 -- 172.21.15.70:0/2652183297 wait complete. 2023-10-08T07:10:48.626 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T07:10:48.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:10:48.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.823+0000 7f051849b640 1 Processor -- start 2023-10-08T07:10:48.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.824+0000 7f051849b640 1 -- start start 2023-10-08T07:10:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.825+0000 7f051849b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f051015cd90 0x7f05100702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.825+0000 7f051849b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05100c48f0 con 0x7f051006c890 2023-10-08T07:10:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.825+0000 7f051849b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05100c3ce0 con 0x7f0510160940 2023-10-08T07:10:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.825+0000 7f051849b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05100c4d80 con 0x7f051015cd90 2023-10-08T07:10:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.826+0000 7f0516a38640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f051006c890 0x7f051006cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57706/0 (socket says 172.21.15.70:57706) 2023-10-08T07:10:48.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.826+0000 7f0516237640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f051015cd90 0x7f05100702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.826+0000 7f0516a38640 1 -- 172.21.15.70:0/582232534 learned_addr learned my addr 172.21.15.70:0/582232534 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:48.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.826+0000 7f0516237640 1 -- 172.21.15.70:0/582232534 >> v1:172.21.15.181:6789/0 conn(0x7f0510160940 legacy=0x7f0510070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:48.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.826+0000 7f0516237640 1 -- 172.21.15.70:0/582232534 >> v1:172.21.15.70:6789/0 conn(0x7f051006c890 legacy=0x7f051006cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.826+0000 7f0516237640 1 -- 172.21.15.70:0/582232534 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05100c3fc0 con 0x7f051015cd90 2023-10-08T07:10:48.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.826+0000 7f0515235640 1 -- 172.21.15.70:0/582232534 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165984748 0 0) 0x7f05100c48f0 con 0x7f051006c890 2023-10-08T07:10:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.827+0000 7f0516237640 1 --2- 172.21.15.70:0/582232534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051015cd90 0x7f05100702d0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f050c0097a0 tx=0x7f050c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa15b6fa471dd11 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.827+0000 7f0515235640 1 -- 172.21.15.70:0/582232534 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c031830 con 0x7f051015cd90 2023-10-08T07:10:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.827+0000 7f0515235640 1 -- 172.21.15.70:0/582232534 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c031990 con 0x7f051015cd90 2023-10-08T07:10:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.828+0000 7f0515235640 1 -- 172.21.15.70:0/582232534 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c031d20 con 0x7f051015cd90 2023-10-08T07:10:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.828+0000 7f051849b640 1 -- 172.21.15.70:0/582232534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051015cd90 msgr2=0x7f05100702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.828+0000 7f051849b640 1 --2- 172.21.15.70:0/582232534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051015cd90 0x7f05100702d0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f050c0097a0 tx=0x7f050c02b8d0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.828+0000 7f051849b640 1 -- 172.21.15.70:0/582232534 shutdown_connections 2023-10-08T07:10:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.828+0000 7f051849b640 1 --2- 172.21.15.70:0/582232534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051015cd90 0x7f05100702d0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.828+0000 7f051849b640 1 -- 172.21.15.70:0/582232534 >> 172.21.15.70:0/582232534 conn(0x7f051006a380 msgr2=0x7f05100b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.828+0000 7f051849b640 1 -- 172.21.15.70:0/582232534 shutdown_connections 2023-10-08T07:10:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.828+0000 7f051849b640 1 -- 172.21.15.70:0/582232534 wait complete. 2023-10-08T07:10:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.830+0000 7f051849b640 1 Processor -- start 2023-10-08T07:10:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.830+0000 7f051849b640 1 -- start start 2023-10-08T07:10:48.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.831+0000 7f051849b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051006c890 0x7f05101513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.831+0000 7f051849b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f051015cd90 0x7f05100c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.831+0000 7f051849b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510160940 0x7f05100ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.831+0000 7f051849b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05100b1500 con 0x7f051006c890 2023-10-08T07:10:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.831+0000 7f051849b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05100ca110 con 0x7f0510160940 2023-10-08T07:10:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.831+0000 7f051849b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05100c9870 con 0x7f051015cd90 2023-10-08T07:10:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.831+0000 7f0516237640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051006c890 0x7f05101513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.831+0000 7f0516237640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051006c890 0x7f05101513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60738/0 (socket says 172.21.15.70:60738) 2023-10-08T07:10:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.831+0000 7f0516237640 1 -- 172.21.15.70:0/455106689 learned_addr learned my addr 172.21.15.70:0/455106689 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.832+0000 7f0515a36640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f051015cd90 0x7f05100c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.832+0000 7f0516a38640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510160940 0x7f05100ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.832+0000 7f0516237640 1 -- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f051015cd90 msgr2=0x7f05100c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.832+0000 7f0516237640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f051015cd90 0x7f05100c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.832+0000 7f0516237640 1 -- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510160940 msgr2=0x7f05100ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:48.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.832+0000 7f0516237640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510160940 0x7f05100ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:48.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.832+0000 7f0516237640 1 -- 172.21.15.70:0/455106689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c009400 con 0x7f051006c890 2023-10-08T07:10:48.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.832+0000 7f0515a36640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f051015cd90 0x7f05100c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:48.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.832+0000 7f0516237640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051006c890 0x7f05101513f0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f050c000c00 tx=0x7f050c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.832+0000 7f0506ffd640 1 -- 172.21.15.70:0/455106689 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c004230 con 0x7f051006c890 2023-10-08T07:10:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.833+0000 7f0506ffd640 1 -- 172.21.15.70:0/455106689 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c004390 con 0x7f051006c890 2023-10-08T07:10:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.833+0000 7f051849b640 1 -- 172.21.15.70:0/455106689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05100c9b50 con 0x7f051006c890 2023-10-08T07:10:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.833+0000 7f0506ffd640 1 -- 172.21.15.70:0/455106689 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c03a940 con 0x7f051006c890 2023-10-08T07:10:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.833+0000 7f051849b640 1 -- 172.21.15.70:0/455106689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051015c0c0 con 0x7f051006c890 2023-10-08T07:10:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.834+0000 7f051849b640 1 -- 172.21.15.70:0/455106689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051015c250 con 0x7f051006c890 2023-10-08T07:10:48.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.835+0000 7f0506ffd640 1 -- 172.21.15.70:0/455106689 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f050c03aaa0 con 0x7f051006c890 2023-10-08T07:10:48.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.836+0000 7f0506ffd640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f04f406afc0 0x7f04f406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.836+0000 7f0506ffd640 1 -- 172.21.15.70:0/455106689 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f050c0ac0f0 con 0x7f051006c890 2023-10-08T07:10:48.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.837+0000 7f0515a36640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f04f406afc0 0x7f04f406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:48.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.838+0000 7f0515a36640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f04f406afc0 0x7f04f406d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0510158400 tx=0x7f0500008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:48.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:48.841+0000 7f0506ffd640 1 -- 172.21.15.70:0/455106689 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f050c076e50 con 0x7f051006c890 2023-10-08T07:10:49.017 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-10-08T07:10:49.016+0000 7f2d1d34c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:10:49.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.023+0000 7f051849b640 1 -- 172.21.15.70:0/455106689 --> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05100c4580 con 0x7f04f406afc0 2023-10-08T07:10:49.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.053+0000 7f0506ffd640 1 -- 172.21.15.70:0/455106689 <== mgr.7112 v2:172.21.15.181:6832/2638163552 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286827 (secure 0 0 0) 0x7f050000ad60 con 0x7f04f406afc0 2023-10-08T07:10:49.060 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:49.061 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:10:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.058+0000 7f051849b640 1 -- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f04f406afc0 msgr2=0x7f04f406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f04f406afc0 0x7f04f406d480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0510158400 tx=0x7f0500008040 comp rx=0 tx=0).stop 2023-10-08T07:10:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 -- 172.21.15.70:0/455106689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051006c890 msgr2=0x7f05101513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051006c890 0x7f05101513f0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f050c000c00 tx=0x7f050c0046c0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 -- 172.21.15.70:0/455106689 shutdown_connections 2023-10-08T07:10:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f04f406afc0 0x7f04f406d480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0510160940 0x7f05100ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f051015cd90 0x7f05100c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 --2- 172.21.15.70:0/455106689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f051006c890 0x7f05101513f0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 -- 172.21.15.70:0/455106689 >> 172.21.15.70:0/455106689 conn(0x7f051006a380 msgr2=0x7f05100ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 -- 172.21.15.70:0/455106689 shutdown_connections 2023-10-08T07:10:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.059+0000 7f051849b640 1 -- 172.21.15.70:0/455106689 wait complete. 2023-10-08T07:10:49.079 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-08T07:10:47.507245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2013,"num_read_kb":4206,"num_write":593,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2822,"ondisk_log_size":2822,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423148,"kb_used_data":22376,"kb_used_omap":12,"kb_used_meta":400627,"kb_avail":754551572,"statfs":{"total":773094113280,"available":772660809728,"internally_reserved":0,"allocated":22913024,"data_stored":12398500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":410242619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.870271"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241265+0000","last_change":"2023-10-08T07:10:44.226703+0000","last_active":"2023-10-08T07:10:45.241265+0000","last_peered":"2023-10-08T07:10:45.241265+0000","last_clean":"2023-10-08T07:10:45.241265+0000","last_became_active":"2023-10-08T07:10:44.226429+0000","last_became_peered":"2023-10-08T07:10:44.226429+0000","last_unstale":"2023-10-08T07:10:45.241265+0000","last_undegraded":"2023-10-08T07:10:45.241265+0000","last_fullsized":"2023-10-08T07:10:45.241265+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:46:52.000759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241407+0000","last_change":"2023-10-08T07:10:44.234875+0000","last_active":"2023-10-08T07:10:45.241407+0000","last_peered":"2023-10-08T07:10:45.241407+0000","last_clean":"2023-10-08T07:10:45.241407+0000","last_became_active":"2023-10-08T07:10:44.234520+0000","last_became_peered":"2023-10-08T07:10:44.234520+0000","last_unstale":"2023-10-08T07:10:45.241407+0000","last_undegraded":"2023-10-08T07:10:45.241407+0000","last_fullsized":"2023-10-08T07:10:45.241407+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:59.672226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206463+0000","last_change":"2023-10-08T07:10:44.220366+0000","last_active":"2023-10-08T07:10:45.206463+0000","last_peered":"2023-10-08T07:10:45.206463+0000","last_clean":"2023-10-08T07:10:45.206463+0000","last_became_active":"2023-10-08T07:10:44.219848+0000","last_became_peered":"2023-10-08T07:10:44.219848+0000","last_unstale":"2023-10-08T07:10:45.206463+0000","last_undegraded":"2023-10-08T07:10:45.206463+0000","last_fullsized":"2023-10-08T07:10:45.206463+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:41.310469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238444+0000","last_change":"2023-10-08T07:10:44.227862+0000","last_active":"2023-10-08T07:10:45.238444+0000","last_peered":"2023-10-08T07:10:45.238444+0000","last_clean":"2023-10-08T07:10:45.238444+0000","last_became_active":"2023-10-08T07:10:44.227562+0000","last_became_peered":"2023-10-08T07:10:44.227562+0000","last_unstale":"2023-10-08T07:10:45.238444+0000","last_undegraded":"2023-10-08T07:10:45.238444+0000","last_fullsized":"2023-10-08T07:10:45.238444+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:43.902521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241311+0000","last_change":"2023-10-08T07:10:44.236102+0000","last_active":"2023-10-08T07:10:45.241311+0000","last_peered":"2023-10-08T07:10:45.241311+0000","last_clean":"2023-10-08T07:10:45.241311+0000","last_became_active":"2023-10-08T07:10:44.235838+0000","last_became_peered":"2023-10-08T07:10:44.235838+0000","last_unstale":"2023-10-08T07:10:45.241311+0000","last_undegraded":"2023-10-08T07:10:45.241311+0000","last_fullsized":"2023-10-08T07:10:45.241311+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:40.764386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239007+0000","last_change":"2023-10-08T07:10:44.221854+0000","last_active":"2023-10-08T07:10:45.239007+0000","last_peered":"2023-10-08T07:10:45.239007+0000","last_clean":"2023-10-08T07:10:45.239007+0000","last_became_active":"2023-10-08T07:10:44.221582+0000","last_became_peered":"2023-10-08T07:10:44.221582+0000","last_unstale":"2023-10-08T07:10:45.239007+0000","last_undegraded":"2023-10-08T07:10:45.239007+0000","last_fullsized":"2023-10-08T07:10:45.239007+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:48.276610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198620+0000","last_change":"2023-10-08T07:10:44.231349+0000","last_active":"2023-10-08T07:10:45.198620+0000","last_peered":"2023-10-08T07:10:45.198620+0000","last_clean":"2023-10-08T07:10:45.198620+0000","last_became_active":"2023-10-08T07:10:44.230967+0000","last_became_peered":"2023-10-08T07:10:44.230967+0000","last_unstale":"2023-10-08T07:10:45.198620+0000","last_undegraded":"2023-10-08T07:10:45.198620+0000","last_fullsized":"2023-10-08T07:10:45.198620+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:29.380143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238357+0000","last_change":"2023-10-08T07:10:44.227636+0000","last_active":"2023-10-08T07:10:45.238357+0000","last_peered":"2023-10-08T07:10:45.238357+0000","last_clean":"2023-10-08T07:10:45.238357+0000","last_became_active":"2023-10-08T07:10:44.227332+0000","last_became_peered":"2023-10-08T07:10:44.227332+0000","last_unstale":"2023-10-08T07:10:45.238357+0000","last_undegraded":"2023-10-08T07:10:45.238357+0000","last_fullsized":"2023-10-08T07:10:45.238357+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:51.725953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241319+0000","last_change":"2023-10-08T07:10:44.231079+0000","last_active":"2023-10-08T07:10:45.241319+0000","last_peered":"2023-10-08T07:10:45.241319+0000","last_clean":"2023-10-08T07:10:45.241319+0000","last_became_active":"2023-10-08T07:10:44.230756+0000","last_became_peered":"2023-10-08T07:10:44.230756+0000","last_unstale":"2023-10-08T07:10:45.241319+0000","last_undegraded":"2023-10-08T07:10:45.241319+0000","last_fullsized":"2023-10-08T07:10:45.241319+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:01.919740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238499+0000","last_change":"2023-10-08T07:10:44.232986+0000","last_active":"2023-10-08T07:10:45.238499+0000","last_peered":"2023-10-08T07:10:45.238499+0000","last_clean":"2023-10-08T07:10:45.238499+0000","last_became_active":"2023-10-08T07:10:44.232705+0000","last_became_peered":"2023-10-08T07:10:44.232705+0000","last_unstale":"2023-10-08T07:10:45.238499+0000","last_undegraded":"2023-10-08T07:10:45.238499+0000","last_fullsized":"2023-10-08T07:10:45.238499+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:23.294834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.204015+0000","last_change":"2023-10-08T07:10:44.238805+0000","last_active":"2023-10-08T07:10:45.204015+0000","last_peered":"2023-10-08T07:10:45.204015+0000","last_clean":"2023-10-08T07:10:45.204015+0000","last_became_active":"2023-10-08T07:10:44.238455+0000","last_became_peered":"2023-10-08T07:10:44.238455+0000","last_unstale":"2023-10-08T07:10:45.204015+0000","last_undegraded":"2023-10-08T07:10:45.204015+0000","last_fullsized":"2023-10-08T07:10:45.204015+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:03:52.074707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238750+0000","last_change":"2023-10-08T07:10:44.224088+0000","last_active":"2023-10-08T07:10:45.238750+0000","last_peered":"2023-10-08T07:10:45.238750+0000","last_clean":"2023-10-08T07:10:45.238750+0000","last_became_active":"2023-10-08T07:10:44.223831+0000","last_became_peered":"2023-10-08T07:10:44.223831+0000","last_unstale":"2023-10-08T07:10:45.238750+0000","last_undegraded":"2023-10-08T07:10:45.238750+0000","last_fullsized":"2023-10-08T07:10:45.238750+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:51.683157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237714+0000","last_change":"2023-10-08T07:10:44.237662+0000","last_active":"2023-10-08T07:10:45.237714+0000","last_peered":"2023-10-08T07:10:45.237714+0000","last_clean":"2023-10-08T07:10:45.237714+0000","last_became_active":"2023-10-08T07:10:44.236939+0000","last_became_peered":"2023-10-08T07:10:44.236939+0000","last_unstale":"2023-10-08T07:10:45.237714+0000","last_undegraded":"2023-10-08T07:10:45.237714+0000","last_fullsized":"2023-10-08T07:10:45.237714+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:57.868685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198788+0000","last_change":"2023-10-08T07:10:44.232856+0000","last_active":"2023-10-08T07:10:45.198788+0000","last_peered":"2023-10-08T07:10:45.198788+0000","last_clean":"2023-10-08T07:10:45.198788+0000","last_became_active":"2023-10-08T07:10:44.232512+0000","last_became_peered":"2023-10-08T07:10:44.232512+0000","last_unstale":"2023-10-08T07:10:45.198788+0000","last_undegraded":"2023-10-08T07:10:45.198788+0000","last_fullsized":"2023-10-08T07:10:45.198788+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:14.876986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239919+0000","last_change":"2023-10-08T07:10:44.232537+0000","last_active":"2023-10-08T07:10:45.239919+0000","last_peered":"2023-10-08T07:10:45.239919+0000","last_clean":"2023-10-08T07:10:45.239919+0000","last_became_active":"2023-10-08T07:10:44.231905+0000","last_became_peered":"2023-10-08T07:10:44.231905+0000","last_unstale":"2023-10-08T07:10:45.239919+0000","last_undegraded":"2023-10-08T07:10:45.239919+0000","last_fullsized":"2023-10-08T07:10:45.239919+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:10.784865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238247+0000","last_change":"2023-10-08T07:10:44.224894+0000","last_active":"2023-10-08T07:10:45.238247+0000","last_peered":"2023-10-08T07:10:45.238247+0000","last_clean":"2023-10-08T07:10:45.238247+0000","last_became_active":"2023-10-08T07:10:44.224646+0000","last_became_peered":"2023-10-08T07:10:44.224646+0000","last_unstale":"2023-10-08T07:10:45.238247+0000","last_undegraded":"2023-10-08T07:10:45.238247+0000","last_fullsized":"2023-10-08T07:10:45.238247+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:46.835534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239232+0000","last_change":"2023-10-08T07:10:44.223684+0000","last_active":"2023-10-08T07:10:45.239232+0000","last_peered":"2023-10-08T07:10:45.239232+0000","last_clean":"2023-10-08T07:10:45.239232+0000","last_became_active":"2023-10-08T07:10:44.223417+0000","last_became_peered":"2023-10-08T07:10:44.223417+0000","last_unstale":"2023-10-08T07:10:45.239232+0000","last_undegraded":"2023-10-08T07:10:45.239232+0000","last_fullsized":"2023-10-08T07:10:45.239232+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:56:36.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238569+0000","last_change":"2023-10-08T07:10:44.232582+0000","last_active":"2023-10-08T07:10:45.238569+0000","last_peered":"2023-10-08T07:10:45.238569+0000","last_clean":"2023-10-08T07:10:45.238569+0000","last_became_active":"2023-10-08T07:10:44.232264+0000","last_became_peered":"2023-10-08T07:10:44.232264+0000","last_unstale":"2023-10-08T07:10:45.238569+0000","last_undegraded":"2023-10-08T07:10:45.238569+0000","last_fullsized":"2023-10-08T07:10:45.238569+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:06.376230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239571+0000","last_change":"2023-10-08T07:10:44.229054+0000","last_active":"2023-10-08T07:10:45.239571+0000","last_peered":"2023-10-08T07:10:45.239571+0000","last_clean":"2023-10-08T07:10:45.239571+0000","last_became_active":"2023-10-08T07:10:44.228712+0000","last_became_peered":"2023-10-08T07:10:44.228712+0000","last_unstale":"2023-10-08T07:10:45.239571+0000","last_undegraded":"2023-10-08T07:10:45.239571+0000","last_fullsized":"2023-10-08T07:10:45.239571+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:26:57.282217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239280+0000","last_change":"2023-10-08T07:10:44.223195+0000","last_active":"2023-10-08T07:10:45.239280+0000","last_peered":"2023-10-08T07:10:45.239280+0000","last_clean":"2023-10-08T07:10:45.239280+0000","last_became_active":"2023-10-08T07:10:44.222907+0000","last_became_peered":"2023-10-08T07:10:44.222907+0000","last_unstale":"2023-10-08T07:10:45.239280+0000","last_undegraded":"2023-10-08T07:10:45.239280+0000","last_fullsized":"2023-10-08T07:10:45.239280+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:39.781083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240982+0000","last_change":"2023-10-08T07:10:44.231122+0000","last_active":"2023-10-08T07:10:45.240982+0000","last_peered":"2023-10-08T07:10:45.240982+0000","last_clean":"2023-10-08T07:10:45.240982+0000","last_became_active":"2023-10-08T07:10:44.230844+0000","last_became_peered":"2023-10-08T07:10:44.230844+0000","last_unstale":"2023-10-08T07:10:45.240982+0000","last_undegraded":"2023-10-08T07:10:45.240982+0000","last_fullsized":"2023-10-08T07:10:45.240982+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:39:30.420420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240596+0000","last_change":"2023-10-08T07:10:44.231127+0000","last_active":"2023-10-08T07:10:45.240596+0000","last_peered":"2023-10-08T07:10:45.240596+0000","last_clean":"2023-10-08T07:10:45.240596+0000","last_became_active":"2023-10-08T07:10:44.230808+0000","last_became_peered":"2023-10-08T07:10:44.230808+0000","last_unstale":"2023-10-08T07:10:45.240596+0000","last_undegraded":"2023-10-08T07:10:45.240596+0000","last_fullsized":"2023-10-08T07:10:45.240596+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:59:42.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239727+0000","last_change":"2023-10-08T07:10:44.228180+0000","last_active":"2023-10-08T07:10:45.239727+0000","last_peered":"2023-10-08T07:10:45.239727+0000","last_clean":"2023-10-08T07:10:45.239727+0000","last_became_active":"2023-10-08T07:10:44.227845+0000","last_became_peered":"2023-10-08T07:10:44.227845+0000","last_unstale":"2023-10-08T07:10:45.239727+0000","last_undegraded":"2023-10-08T07:10:45.239727+0000","last_fullsized":"2023-10-08T07:10:45.239727+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:17.503602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241686+0000","last_change":"2023-10-08T07:10:44.220960+0000","last_active":"2023-10-08T07:10:45.241686+0000","last_peered":"2023-10-08T07:10:45.241686+0000","last_clean":"2023-10-08T07:10:45.241686+0000","last_became_active":"2023-10-08T07:10:44.219821+0000","last_became_peered":"2023-10-08T07:10:44.219821+0000","last_unstale":"2023-10-08T07:10:45.241686+0000","last_undegraded":"2023-10-08T07:10:45.241686+0000","last_fullsized":"2023-10-08T07:10:45.241686+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:36.260436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.199267+0000","last_change":"2023-10-08T07:10:44.217654+0000","last_active":"2023-10-08T07:10:45.199267+0000","last_peered":"2023-10-08T07:10:45.199267+0000","last_clean":"2023-10-08T07:10:45.199267+0000","last_became_active":"2023-10-08T07:10:44.217378+0000","last_became_peered":"2023-10-08T07:10:44.217378+0000","last_unstale":"2023-10-08T07:10:45.199267+0000","last_undegraded":"2023-10-08T07:10:45.199267+0000","last_fullsized":"2023-10-08T07:10:45.199267+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:44:45.377128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206297+0000","last_change":"2023-10-08T07:10:44.238798+0000","last_active":"2023-10-08T07:10:45.206297+0000","last_peered":"2023-10-08T07:10:45.206297+0000","last_clean":"2023-10-08T07:10:45.206297+0000","last_became_active":"2023-10-08T07:10:44.238387+0000","last_became_peered":"2023-10-08T07:10:44.238387+0000","last_unstale":"2023-10-08T07:10:45.206297+0000","last_undegraded":"2023-10-08T07:10:45.206297+0000","last_fullsized":"2023-10-08T07:10:45.206297+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:45.076883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239417+0000","last_change":"2023-10-08T07:10:44.229217+0000","last_active":"2023-10-08T07:10:45.239417+0000","last_peered":"2023-10-08T07:10:45.239417+0000","last_clean":"2023-10-08T07:10:45.239417+0000","last_became_active":"2023-10-08T07:10:44.228943+0000","last_became_peered":"2023-10-08T07:10:44.228943+0000","last_unstale":"2023-10-08T07:10:45.239417+0000","last_undegraded":"2023-10-08T07:10:45.239417+0000","last_fullsized":"2023-10-08T07:10:45.239417+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:22.640492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239545+0000","last_change":"2023-10-08T07:10:44.225988+0000","last_active":"2023-10-08T07:10:45.239545+0000","last_peered":"2023-10-08T07:10:45.239545+0000","last_clean":"2023-10-08T07:10:45.239545+0000","last_became_active":"2023-10-08T07:10:44.225718+0000","last_became_peered":"2023-10-08T07:10:44.225718+0000","last_unstale":"2023-10-08T07:10:45.239545+0000","last_undegraded":"2023-10-08T07:10:45.239545+0000","last_fullsized":"2023-10-08T07:10:45.239545+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:36.502279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.203936+0000","last_change":"2023-10-08T07:10:44.239219+0000","last_active":"2023-10-08T07:10:45.203936+0000","last_peered":"2023-10-08T07:10:45.203936+0000","last_clean":"2023-10-08T07:10:45.203936+0000","last_became_active":"2023-10-08T07:10:44.238599+0000","last_became_peered":"2023-10-08T07:10:44.238599+0000","last_unstale":"2023-10-08T07:10:45.203936+0000","last_undegraded":"2023-10-08T07:10:45.203936+0000","last_fullsized":"2023-10-08T07:10:45.203936+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:24:19.793863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.203543+0000","last_change":"2023-10-08T07:10:44.224589+0000","last_active":"2023-10-08T07:10:45.203543+0000","last_peered":"2023-10-08T07:10:45.203543+0000","last_clean":"2023-10-08T07:10:45.203543+0000","last_became_active":"2023-10-08T07:10:44.224259+0000","last_became_peered":"2023-10-08T07:10:44.224259+0000","last_unstale":"2023-10-08T07:10:45.203543+0000","last_undegraded":"2023-10-08T07:10:45.203543+0000","last_fullsized":"2023-10-08T07:10:45.203543+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:46.580370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198810+0000","last_change":"2023-10-08T07:10:44.222116+0000","last_active":"2023-10-08T07:10:45.198810+0000","last_peered":"2023-10-08T07:10:45.198810+0000","last_clean":"2023-10-08T07:10:45.198810+0000","last_became_active":"2023-10-08T07:10:44.221853+0000","last_became_peered":"2023-10-08T07:10:44.221853+0000","last_unstale":"2023-10-08T07:10:45.198810+0000","last_undegraded":"2023-10-08T07:10:45.198810+0000","last_fullsized":"2023-10-08T07:10:45.198810+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:40.761989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238402+0000","last_change":"2023-10-08T07:10:44.228359+0000","last_active":"2023-10-08T07:10:45.238402+0000","last_peered":"2023-10-08T07:10:45.238402+0000","last_clean":"2023-10-08T07:10:45.238402+0000","last_became_active":"2023-10-08T07:10:44.228049+0000","last_became_peered":"2023-10-08T07:10:44.228049+0000","last_unstale":"2023-10-08T07:10:45.238402+0000","last_undegraded":"2023-10-08T07:10:45.238402+0000","last_fullsized":"2023-10-08T07:10:45.238402+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:00.141299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239162+0000","last_change":"2023-10-08T07:10:44.228337+0000","last_active":"2023-10-08T07:10:45.239162+0000","last_peered":"2023-10-08T07:10:45.239162+0000","last_clean":"2023-10-08T07:10:45.239162+0000","last_became_active":"2023-10-08T07:10:44.227910+0000","last_became_peered":"2023-10-08T07:10:44.227910+0000","last_unstale":"2023-10-08T07:10:45.239162+0000","last_undegraded":"2023-10-08T07:10:45.239162+0000","last_fullsized":"2023-10-08T07:10:45.239162+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:17.731669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"215'4","reported_seq":16,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239314+0000","last_change":"2023-10-08T07:10:43.678287+0000","last_active":"2023-10-08T07:10:45.239314+0000","last_peered":"2023-10-08T07:10:45.239314+0000","last_clean":"2023-10-08T07:10:45.239314+0000","last_became_active":"2023-10-08T07:10:43.678046+0000","last_became_peered":"2023-10-08T07:10:43.678046+0000","last_unstale":"2023-10-08T07:10:45.239314+0000","last_undegraded":"2023-10-08T07:10:45.239314+0000","last_fullsized":"2023-10-08T07:10:45.239314+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:09.768687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.199308+0000","last_change":"2023-10-08T07:10:44.224347+0000","last_active":"2023-10-08T07:10:45.199308+0000","last_peered":"2023-10-08T07:10:45.199308+0000","last_clean":"2023-10-08T07:10:45.199308+0000","last_became_active":"2023-10-08T07:10:44.224082+0000","last_became_peered":"2023-10-08T07:10:44.224082+0000","last_unstale":"2023-10-08T07:10:45.199308+0000","last_undegraded":"2023-10-08T07:10:45.199308+0000","last_fullsized":"2023-10-08T07:10:45.199308+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:58.980437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239093+0000","last_change":"2023-10-08T07:10:43.683141+0000","last_active":"2023-10-08T07:10:45.239093+0000","last_peered":"2023-10-08T07:10:45.239093+0000","last_clean":"2023-10-08T07:10:45.239093+0000","last_became_active":"2023-10-08T07:10:43.682845+0000","last_became_peered":"2023-10-08T07:10:43.682845+0000","last_unstale":"2023-10-08T07:10:45.239093+0000","last_undegraded":"2023-10-08T07:10:45.239093+0000","last_fullsized":"2023-10-08T07:10:45.239093+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:37:56.842435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206606+0000","last_change":"2023-10-08T07:10:44.233237+0000","last_active":"2023-10-08T07:10:45.206606+0000","last_peered":"2023-10-08T07:10:45.206606+0000","last_clean":"2023-10-08T07:10:45.206606+0000","last_became_active":"2023-10-08T07:10:44.232959+0000","last_became_peered":"2023-10-08T07:10:44.232959+0000","last_unstale":"2023-10-08T07:10:45.206606+0000","last_undegraded":"2023-10-08T07:10:45.206606+0000","last_fullsized":"2023-10-08T07:10:45.206606+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:00.122036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.205248+0000","last_change":"2023-10-08T07:10:43.687400+0000","last_active":"2023-10-08T07:10:45.205248+0000","last_peered":"2023-10-08T07:10:45.205248+0000","last_clean":"2023-10-08T07:10:45.205248+0000","last_became_active":"2023-10-08T07:10:43.686675+0000","last_became_peered":"2023-10-08T07:10:43.686675+0000","last_unstale":"2023-10-08T07:10:45.205248+0000","last_undegraded":"2023-10-08T07:10:45.205248+0000","last_fullsized":"2023-10-08T07:10:45.205248+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:42:28.651968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206007+0000","last_change":"2023-10-08T07:10:44.224561+0000","last_active":"2023-10-08T07:10:45.206007+0000","last_peered":"2023-10-08T07:10:45.206007+0000","last_clean":"2023-10-08T07:10:45.206007+0000","last_became_active":"2023-10-08T07:10:44.224219+0000","last_became_peered":"2023-10-08T07:10:44.224219+0000","last_unstale":"2023-10-08T07:10:45.206007+0000","last_undegraded":"2023-10-08T07:10:45.206007+0000","last_fullsized":"2023-10-08T07:10:45.206007+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:19.841213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238252+0000","last_change":"2023-10-08T07:10:43.685955+0000","last_active":"2023-10-08T07:10:45.238252+0000","last_peered":"2023-10-08T07:10:45.238252+0000","last_clean":"2023-10-08T07:10:45.238252+0000","last_became_active":"2023-10-08T07:10:43.685678+0000","last_became_peered":"2023-10-08T07:10:43.685678+0000","last_unstale":"2023-10-08T07:10:45.238252+0000","last_undegraded":"2023-10-08T07:10:45.238252+0000","last_fullsized":"2023-10-08T07:10:45.238252+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:32.700444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237268+0000","last_change":"2023-10-08T07:10:44.229103+0000","last_active":"2023-10-08T07:10:45.237268+0000","last_peered":"2023-10-08T07:10:45.237268+0000","last_clean":"2023-10-08T07:10:45.237268+0000","last_became_active":"2023-10-08T07:10:44.228795+0000","last_became_peered":"2023-10-08T07:10:44.228795+0000","last_unstale":"2023-10-08T07:10:45.237268+0000","last_undegraded":"2023-10-08T07:10:45.237268+0000","last_fullsized":"2023-10-08T07:10:45.237268+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:23.596413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241576+0000","last_change":"2023-10-08T07:10:43.682622+0000","last_active":"2023-10-08T07:10:45.241576+0000","last_peered":"2023-10-08T07:10:45.241576+0000","last_clean":"2023-10-08T07:10:45.241576+0000","last_became_active":"2023-10-08T07:10:43.682356+0000","last_became_peered":"2023-10-08T07:10:43.682356+0000","last_unstale":"2023-10-08T07:10:45.241576+0000","last_undegraded":"2023-10-08T07:10:45.241576+0000","last_fullsized":"2023-10-08T07:10:45.241576+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:16.487425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240356+0000","last_change":"2023-10-08T07:10:44.220256+0000","last_active":"2023-10-08T07:10:45.240356+0000","last_peered":"2023-10-08T07:10:45.240356+0000","last_clean":"2023-10-08T07:10:45.240356+0000","last_became_active":"2023-10-08T07:10:44.219413+0000","last_became_peered":"2023-10-08T07:10:44.219413+0000","last_unstale":"2023-10-08T07:10:45.240356+0000","last_undegraded":"2023-10-08T07:10:45.240356+0000","last_fullsized":"2023-10-08T07:10:45.240356+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.355955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237790+0000","last_change":"2023-10-08T07:10:43.682977+0000","last_active":"2023-10-08T07:10:45.237790+0000","last_peered":"2023-10-08T07:10:45.237790+0000","last_clean":"2023-10-08T07:10:45.237790+0000","last_became_active":"2023-10-08T07:10:43.682711+0000","last_became_peered":"2023-10-08T07:10:43.682711+0000","last_unstale":"2023-10-08T07:10:45.237790+0000","last_undegraded":"2023-10-08T07:10:45.237790+0000","last_fullsized":"2023-10-08T07:10:45.237790+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:55:43.040815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240617+0000","last_change":"2023-10-08T07:10:44.234287+0000","last_active":"2023-10-08T07:10:45.240617+0000","last_peered":"2023-10-08T07:10:45.240617+0000","last_clean":"2023-10-08T07:10:45.240617+0000","last_became_active":"2023-10-08T07:10:44.234001+0000","last_became_peered":"2023-10-08T07:10:44.234001+0000","last_unstale":"2023-10-08T07:10:45.240617+0000","last_undegraded":"2023-10-08T07:10:45.240617+0000","last_fullsized":"2023-10-08T07:10:45.240617+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:43.902521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237817+0000","last_change":"2023-10-08T07:10:43.683020+0000","last_active":"2023-10-08T07:10:45.237817+0000","last_peered":"2023-10-08T07:10:45.237817+0000","last_clean":"2023-10-08T07:10:45.237817+0000","last_became_active":"2023-10-08T07:10:43.682737+0000","last_became_peered":"2023-10-08T07:10:43.682737+0000","last_unstale":"2023-10-08T07:10:45.237817+0000","last_undegraded":"2023-10-08T07:10:45.237817+0000","last_fullsized":"2023-10-08T07:10:45.237817+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:58:42.063439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239081+0000","last_change":"2023-10-08T07:10:44.231423+0000","last_active":"2023-10-08T07:10:45.239081+0000","last_peered":"2023-10-08T07:10:45.239081+0000","last_clean":"2023-10-08T07:10:45.239081+0000","last_became_active":"2023-10-08T07:10:44.231151+0000","last_became_peered":"2023-10-08T07:10:44.231151+0000","last_unstale":"2023-10-08T07:10:45.239081+0000","last_undegraded":"2023-10-08T07:10:45.239081+0000","last_fullsized":"2023-10-08T07:10:45.239081+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:06.269410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239484+0000","last_change":"2023-10-08T07:10:43.675546+0000","last_active":"2023-10-08T07:10:45.239484+0000","last_peered":"2023-10-08T07:10:45.239484+0000","last_clean":"2023-10-08T07:10:45.239484+0000","last_became_active":"2023-10-08T07:10:43.675280+0000","last_became_peered":"2023-10-08T07:10:43.675280+0000","last_unstale":"2023-10-08T07:10:45.239484+0000","last_undegraded":"2023-10-08T07:10:45.239484+0000","last_fullsized":"2023-10-08T07:10:45.239484+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:22.967066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240457+0000","last_change":"2023-10-08T07:10:44.218798+0000","last_active":"2023-10-08T07:10:45.240457+0000","last_peered":"2023-10-08T07:10:45.240457+0000","last_clean":"2023-10-08T07:10:45.240457+0000","last_became_active":"2023-10-08T07:10:44.218481+0000","last_became_peered":"2023-10-08T07:10:44.218481+0000","last_unstale":"2023-10-08T07:10:45.240457+0000","last_undegraded":"2023-10-08T07:10:45.240457+0000","last_fullsized":"2023-10-08T07:10:45.240457+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:14.551225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241619+0000","last_change":"2023-10-08T07:10:43.670774+0000","last_active":"2023-10-08T07:10:45.241619+0000","last_peered":"2023-10-08T07:10:45.241619+0000","last_clean":"2023-10-08T07:10:45.241619+0000","last_became_active":"2023-10-08T07:10:43.670506+0000","last_became_peered":"2023-10-08T07:10:43.670506+0000","last_unstale":"2023-10-08T07:10:45.241619+0000","last_undegraded":"2023-10-08T07:10:45.241619+0000","last_fullsized":"2023-10-08T07:10:45.241619+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:05.911601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238861+0000","last_change":"2023-10-08T07:10:44.240324+0000","last_active":"2023-10-08T07:10:45.238861+0000","last_peered":"2023-10-08T07:10:45.238861+0000","last_clean":"2023-10-08T07:10:45.238861+0000","last_became_active":"2023-10-08T07:10:44.239850+0000","last_became_peered":"2023-10-08T07:10:44.239850+0000","last_unstale":"2023-10-08T07:10:45.238861+0000","last_undegraded":"2023-10-08T07:10:45.238861+0000","last_fullsized":"2023-10-08T07:10:45.238861+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:18.399874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"215'3","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238032+0000","last_change":"2023-10-08T07:10:43.683401+0000","last_active":"2023-10-08T07:10:45.238032+0000","last_peered":"2023-10-08T07:10:45.238032+0000","last_clean":"2023-10-08T07:10:45.238032+0000","last_became_active":"2023-10-08T07:10:43.683027+0000","last_became_peered":"2023-10-08T07:10:43.683027+0000","last_unstale":"2023-10-08T07:10:45.238032+0000","last_undegraded":"2023-10-08T07:10:45.238032+0000","last_fullsized":"2023-10-08T07:10:45.238032+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:45:43.396156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241534+0000","last_change":"2023-10-08T07:10:44.234632+0000","last_active":"2023-10-08T07:10:45.241534+0000","last_peered":"2023-10-08T07:10:45.241534+0000","last_clean":"2023-10-08T07:10:45.241534+0000","last_became_active":"2023-10-08T07:10:44.234330+0000","last_became_peered":"2023-10-08T07:10:44.234330+0000","last_unstale":"2023-10-08T07:10:45.241534+0000","last_undegraded":"2023-10-08T07:10:45.241534+0000","last_fullsized":"2023-10-08T07:10:45.241534+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:59:48.167356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"215'6","reported_seq":18,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238318+0000","last_change":"2023-10-08T07:10:43.690014+0000","last_active":"2023-10-08T07:10:45.238318+0000","last_peered":"2023-10-08T07:10:45.238318+0000","last_clean":"2023-10-08T07:10:45.238318+0000","last_became_active":"2023-10-08T07:10:43.689722+0000","last_became_peered":"2023-10-08T07:10:43.689722+0000","last_unstale":"2023-10-08T07:10:45.238318+0000","last_undegraded":"2023-10-08T07:10:45.238318+0000","last_fullsized":"2023-10-08T07:10:45.238318+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:38.447879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198620+0000","last_change":"2023-10-08T07:10:44.224343+0000","last_active":"2023-10-08T07:10:45.198620+0000","last_peered":"2023-10-08T07:10:45.198620+0000","last_clean":"2023-10-08T07:10:45.198620+0000","last_became_active":"2023-10-08T07:10:44.224056+0000","last_became_peered":"2023-10-08T07:10:44.224056+0000","last_unstale":"2023-10-08T07:10:45.198620+0000","last_undegraded":"2023-10-08T07:10:45.198620+0000","last_fullsized":"2023-10-08T07:10:45.198620+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:56:51.457269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238753+0000","last_change":"2023-10-08T07:10:43.687611+0000","last_active":"2023-10-08T07:10:45.238753+0000","last_peered":"2023-10-08T07:10:45.238753+0000","last_clean":"2023-10-08T07:10:45.238753+0000","last_became_active":"2023-10-08T07:10:43.687016+0000","last_became_peered":"2023-10-08T07:10:43.687016+0000","last_unstale":"2023-10-08T07:10:45.238753+0000","last_undegraded":"2023-10-08T07:10:45.238753+0000","last_fullsized":"2023-10-08T07:10:45.238753+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:40.294292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238250+0000","last_change":"2023-10-08T07:10:44.225167+0000","last_active":"2023-10-08T07:10:45.238250+0000","last_peered":"2023-10-08T07:10:45.238250+0000","last_clean":"2023-10-08T07:10:45.238250+0000","last_became_active":"2023-10-08T07:10:44.224817+0000","last_became_peered":"2023-10-08T07:10:44.224817+0000","last_unstale":"2023-10-08T07:10:45.238250+0000","last_undegraded":"2023-10-08T07:10:45.238250+0000","last_fullsized":"2023-10-08T07:10:45.238250+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:01:14.855954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240821+0000","last_change":"2023-10-08T07:10:43.676142+0000","last_active":"2023-10-08T07:10:45.240821+0000","last_peered":"2023-10-08T07:10:45.240821+0000","last_clean":"2023-10-08T07:10:45.240821+0000","last_became_active":"2023-10-08T07:10:43.675868+0000","last_became_peered":"2023-10-08T07:10:43.675868+0000","last_unstale":"2023-10-08T07:10:45.240821+0000","last_undegraded":"2023-10-08T07:10:45.240821+0000","last_fullsized":"2023-10-08T07:10:45.240821+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:45:43.396156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237757+0000","last_change":"2023-10-08T07:10:44.234251+0000","last_active":"2023-10-08T07:10:45.237757+0000","last_peered":"2023-10-08T07:10:45.237757+0000","last_clean":"2023-10-08T07:10:45.237757+0000","last_became_active":"2023-10-08T07:10:44.233968+0000","last_became_peered":"2023-10-08T07:10:44.233968+0000","last_unstale":"2023-10-08T07:10:45.237757+0000","last_undegraded":"2023-10-08T07:10:45.237757+0000","last_fullsized":"2023-10-08T07:10:45.237757+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:38.036478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237207+0000","last_change":"2023-10-08T07:10:43.682849+0000","last_active":"2023-10-08T07:10:45.237207+0000","last_peered":"2023-10-08T07:10:45.237207+0000","last_clean":"2023-10-08T07:10:45.237207+0000","last_became_active":"2023-10-08T07:10:43.681842+0000","last_became_peered":"2023-10-08T07:10:43.681842+0000","last_unstale":"2023-10-08T07:10:45.237207+0000","last_undegraded":"2023-10-08T07:10:45.237207+0000","last_fullsized":"2023-10-08T07:10:45.237207+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:23.125441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237803+0000","last_change":"2023-10-08T07:10:44.229889+0000","last_active":"2023-10-08T07:10:45.237803+0000","last_peered":"2023-10-08T07:10:45.237803+0000","last_clean":"2023-10-08T07:10:45.237803+0000","last_became_active":"2023-10-08T07:10:44.229605+0000","last_became_peered":"2023-10-08T07:10:44.229605+0000","last_unstale":"2023-10-08T07:10:45.237803+0000","last_undegraded":"2023-10-08T07:10:45.237803+0000","last_fullsized":"2023-10-08T07:10:45.237803+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:04.514916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"215'8","reported_seq":20,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.203562+0000","last_change":"2023-10-08T07:10:43.687967+0000","last_active":"2023-10-08T07:10:45.203562+0000","last_peered":"2023-10-08T07:10:45.203562+0000","last_clean":"2023-10-08T07:10:45.203562+0000","last_became_active":"2023-10-08T07:10:43.687409+0000","last_became_peered":"2023-10-08T07:10:43.687409+0000","last_unstale":"2023-10-08T07:10:45.203562+0000","last_undegraded":"2023-10-08T07:10:45.203562+0000","last_fullsized":"2023-10-08T07:10:45.203562+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:34:13.969749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198898+0000","last_change":"2023-10-08T07:10:44.222398+0000","last_active":"2023-10-08T07:10:45.198898+0000","last_peered":"2023-10-08T07:10:45.198898+0000","last_clean":"2023-10-08T07:10:45.198898+0000","last_became_active":"2023-10-08T07:10:44.222088+0000","last_became_peered":"2023-10-08T07:10:44.222088+0000","last_unstale":"2023-10-08T07:10:45.198898+0000","last_undegraded":"2023-10-08T07:10:45.198898+0000","last_fullsized":"2023-10-08T07:10:45.198898+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:07.926407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206313+0000","last_change":"2023-10-08T07:10:43.678801+0000","last_active":"2023-10-08T07:10:45.206313+0000","last_peered":"2023-10-08T07:10:45.206313+0000","last_clean":"2023-10-08T07:10:45.206313+0000","last_became_active":"2023-10-08T07:10:43.678489+0000","last_became_peered":"2023-10-08T07:10:43.678489+0000","last_unstale":"2023-10-08T07:10:45.206313+0000","last_undegraded":"2023-10-08T07:10:45.206313+0000","last_fullsized":"2023-10-08T07:10:45.206313+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:14.746255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240224+0000","last_change":"2023-10-08T07:10:44.232130+0000","last_active":"2023-10-08T07:10:45.240224+0000","last_peered":"2023-10-08T07:10:45.240224+0000","last_clean":"2023-10-08T07:10:45.240224+0000","last_became_active":"2023-10-08T07:10:44.231832+0000","last_became_peered":"2023-10-08T07:10:44.231832+0000","last_unstale":"2023-10-08T07:10:45.240224+0000","last_undegraded":"2023-10-08T07:10:45.240224+0000","last_fullsized":"2023-10-08T07:10:45.240224+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:21.355790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238315+0000","last_change":"2023-10-08T07:10:43.688805+0000","last_active":"2023-10-08T07:10:45.238315+0000","last_peered":"2023-10-08T07:10:45.238315+0000","last_clean":"2023-10-08T07:10:45.238315+0000","last_became_active":"2023-10-08T07:10:43.688392+0000","last_became_peered":"2023-10-08T07:10:43.688392+0000","last_unstale":"2023-10-08T07:10:45.238315+0000","last_undegraded":"2023-10-08T07:10:45.238315+0000","last_fullsized":"2023-10-08T07:10:45.238315+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:12:14.290252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206578+0000","last_change":"2023-10-08T07:10:44.234357+0000","last_active":"2023-10-08T07:10:45.206578+0000","last_peered":"2023-10-08T07:10:45.206578+0000","last_clean":"2023-10-08T07:10:45.206578+0000","last_became_active":"2023-10-08T07:10:44.234051+0000","last_became_peered":"2023-10-08T07:10:44.234051+0000","last_unstale":"2023-10-08T07:10:45.206578+0000","last_undegraded":"2023-10-08T07:10:45.206578+0000","last_fullsized":"2023-10-08T07:10:45.206578+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:32:20.060577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206512+0000","last_change":"2023-10-08T07:10:43.686002+0000","last_active":"2023-10-08T07:10:45.206512+0000","last_peered":"2023-10-08T07:10:45.206512+0000","last_clean":"2023-10-08T07:10:45.206512+0000","last_became_active":"2023-10-08T07:10:43.685708+0000","last_became_peered":"2023-10-08T07:10:43.685708+0000","last_unstale":"2023-10-08T07:10:45.206512+0000","last_undegraded":"2023-10-08T07:10:45.206512+0000","last_fullsized":"2023-10-08T07:10:45.206512+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:33:01.502206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241659+0000","last_change":"2023-10-08T07:10:44.217941+0000","last_active":"2023-10-08T07:10:45.241659+0000","last_peered":"2023-10-08T07:10:45.241659+0000","last_clean":"2023-10-08T07:10:45.241659+0000","last_became_active":"2023-10-08T07:10:44.217615+0000","last_became_peered":"2023-10-08T07:10:44.217615+0000","last_unstale":"2023-10-08T07:10:45.241659+0000","last_undegraded":"2023-10-08T07:10:45.241659+0000","last_fullsized":"2023-10-08T07:10:45.241659+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:49.740346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.205205+0000","last_change":"2023-10-08T07:10:43.683428+0000","last_active":"2023-10-08T07:10:45.205205+0000","last_peered":"2023-10-08T07:10:45.205205+0000","last_clean":"2023-10-08T07:10:45.205205+0000","last_became_active":"2023-10-08T07:10:43.679200+0000","last_became_peered":"2023-10-08T07:10:43.679200+0000","last_unstale":"2023-10-08T07:10:45.205205+0000","last_undegraded":"2023-10-08T07:10:45.205205+0000","last_fullsized":"2023-10-08T07:10:45.205205+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:35:46.189345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240180+0000","last_change":"2023-10-08T07:10:44.216903+0000","last_active":"2023-10-08T07:10:45.240180+0000","last_peered":"2023-10-08T07:10:45.240180+0000","last_clean":"2023-10-08T07:10:45.240180+0000","last_became_active":"2023-10-08T07:10:44.216541+0000","last_became_peered":"2023-10-08T07:10:44.216541+0000","last_unstale":"2023-10-08T07:10:45.240180+0000","last_undegraded":"2023-10-08T07:10:45.240180+0000","last_fullsized":"2023-10-08T07:10:45.240180+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:25:47.667238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238035+0000","last_change":"2023-10-08T07:10:43.688323+0000","last_active":"2023-10-08T07:10:45.238035+0000","last_peered":"2023-10-08T07:10:45.238035+0000","last_clean":"2023-10-08T07:10:45.238035+0000","last_became_active":"2023-10-08T07:10:43.687375+0000","last_became_peered":"2023-10-08T07:10:43.687375+0000","last_unstale":"2023-10-08T07:10:45.238035+0000","last_undegraded":"2023-10-08T07:10:45.238035+0000","last_fullsized":"2023-10-08T07:10:45.238035+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:00:08.992350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237054+0000","last_change":"2023-10-08T07:10:44.227627+0000","last_active":"2023-10-08T07:10:45.237054+0000","last_peered":"2023-10-08T07:10:45.237054+0000","last_clean":"2023-10-08T07:10:45.237054+0000","last_became_active":"2023-10-08T07:10:44.227344+0000","last_became_peered":"2023-10-08T07:10:44.227344+0000","last_unstale":"2023-10-08T07:10:45.237054+0000","last_undegraded":"2023-10-08T07:10:45.237054+0000","last_fullsized":"2023-10-08T07:10:45.237054+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.355955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237839+0000","last_change":"2023-10-08T07:10:43.678230+0000","last_active":"2023-10-08T07:10:45.237839+0000","last_peered":"2023-10-08T07:10:45.237839+0000","last_clean":"2023-10-08T07:10:45.237839+0000","last_became_active":"2023-10-08T07:10:43.677955+0000","last_became_peered":"2023-10-08T07:10:43.677955+0000","last_unstale":"2023-10-08T07:10:45.237839+0000","last_undegraded":"2023-10-08T07:10:45.237839+0000","last_fullsized":"2023-10-08T07:10:45.237839+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:25.207607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237626+0000","last_change":"2023-10-08T07:10:44.222001+0000","last_active":"2023-10-08T07:10:45.237626+0000","last_peered":"2023-10-08T07:10:45.237626+0000","last_clean":"2023-10-08T07:10:45.237626+0000","last_became_active":"2023-10-08T07:10:44.221728+0000","last_became_peered":"2023-10-08T07:10:44.221728+0000","last_unstale":"2023-10-08T07:10:45.237626+0000","last_undegraded":"2023-10-08T07:10:45.237626+0000","last_fullsized":"2023-10-08T07:10:45.237626+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:59:42.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198949+0000","last_change":"2023-10-08T07:10:43.676861+0000","last_active":"2023-10-08T07:10:45.198949+0000","last_peered":"2023-10-08T07:10:45.198949+0000","last_clean":"2023-10-08T07:10:45.198949+0000","last_became_active":"2023-10-08T07:10:43.676603+0000","last_became_peered":"2023-10-08T07:10:43.676603+0000","last_unstale":"2023-10-08T07:10:45.198949+0000","last_undegraded":"2023-10-08T07:10:45.198949+0000","last_fullsized":"2023-10-08T07:10:45.198949+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:57.059695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241392+0000","last_change":"2023-10-08T07:10:44.218601+0000","last_active":"2023-10-08T07:10:45.241392+0000","last_peered":"2023-10-08T07:10:45.241392+0000","last_clean":"2023-10-08T07:10:45.241392+0000","last_became_active":"2023-10-08T07:10:44.218303+0000","last_became_peered":"2023-10-08T07:10:44.218303+0000","last_unstale":"2023-10-08T07:10:45.241392+0000","last_undegraded":"2023-10-08T07:10:45.241392+0000","last_fullsized":"2023-10-08T07:10:45.241392+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:10.677040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.204831+0000","last_change":"2023-10-08T07:10:43.678826+0000","last_active":"2023-10-08T07:10:45.204831+0000","last_peered":"2023-10-08T07:10:45.204831+0000","last_clean":"2023-10-08T07:10:45.204831+0000","last_became_active":"2023-10-08T07:10:43.678537+0000","last_became_peered":"2023-10-08T07:10:43.678537+0000","last_unstale":"2023-10-08T07:10:45.204831+0000","last_undegraded":"2023-10-08T07:10:45.204831+0000","last_fullsized":"2023-10-08T07:10:45.204831+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:38.447879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198948+0000","last_change":"2023-10-08T07:10:44.234170+0000","last_active":"2023-10-08T07:10:45.198948+0000","last_peered":"2023-10-08T07:10:45.198948+0000","last_clean":"2023-10-08T07:10:45.198948+0000","last_became_active":"2023-10-08T07:10:44.233836+0000","last_became_peered":"2023-10-08T07:10:44.233836+0000","last_unstale":"2023-10-08T07:10:45.198948+0000","last_undegraded":"2023-10-08T07:10:45.198948+0000","last_fullsized":"2023-10-08T07:10:45.198948+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:46.303194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239585+0000","last_change":"2023-10-08T07:10:43.680830+0000","last_active":"2023-10-08T07:10:45.239585+0000","last_peered":"2023-10-08T07:10:45.239585+0000","last_clean":"2023-10-08T07:10:45.239585+0000","last_became_active":"2023-10-08T07:10:43.679042+0000","last_became_peered":"2023-10-08T07:10:43.679042+0000","last_unstale":"2023-10-08T07:10:45.239585+0000","last_undegraded":"2023-10-08T07:10:45.239585+0000","last_fullsized":"2023-10-08T07:10:45.239585+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:29:59.531874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239963+0000","last_change":"2023-10-08T07:10:44.218588+0000","last_active":"2023-10-08T07:10:45.239963+0000","last_peered":"2023-10-08T07:10:45.239963+0000","last_clean":"2023-10-08T07:10:45.239963+0000","last_became_active":"2023-10-08T07:10:44.218267+0000","last_became_peered":"2023-10-08T07:10:44.218267+0000","last_unstale":"2023-10-08T07:10:45.239963+0000","last_undegraded":"2023-10-08T07:10:45.239963+0000","last_fullsized":"2023-10-08T07:10:45.239963+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:00.548052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198799+0000","last_change":"2023-10-08T07:10:43.673755+0000","last_active":"2023-10-08T07:10:45.198799+0000","last_peered":"2023-10-08T07:10:45.198799+0000","last_clean":"2023-10-08T07:10:45.198799+0000","last_became_active":"2023-10-08T07:10:43.673236+0000","last_became_peered":"2023-10-08T07:10:43.673236+0000","last_unstale":"2023-10-08T07:10:45.198799+0000","last_undegraded":"2023-10-08T07:10:45.198799+0000","last_fullsized":"2023-10-08T07:10:45.198799+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:35.386951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239607+0000","last_change":"2023-10-08T07:10:44.230899+0000","last_active":"2023-10-08T07:10:45.239607+0000","last_peered":"2023-10-08T07:10:45.239607+0000","last_clean":"2023-10-08T07:10:45.239607+0000","last_became_active":"2023-10-08T07:10:44.230519+0000","last_became_peered":"2023-10-08T07:10:44.230519+0000","last_unstale":"2023-10-08T07:10:45.239607+0000","last_undegraded":"2023-10-08T07:10:45.239607+0000","last_fullsized":"2023-10-08T07:10:45.239607+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:49.577566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240635+0000","last_change":"2023-10-08T07:10:43.685043+0000","last_active":"2023-10-08T07:10:45.240635+0000","last_peered":"2023-10-08T07:10:45.240635+0000","last_clean":"2023-10-08T07:10:45.240635+0000","last_became_active":"2023-10-08T07:10:43.684594+0000","last_became_peered":"2023-10-08T07:10:43.684594+0000","last_unstale":"2023-10-08T07:10:45.240635+0000","last_undegraded":"2023-10-08T07:10:45.240635+0000","last_fullsized":"2023-10-08T07:10:45.240635+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:59:19.490322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.199298+0000","last_change":"2023-10-08T07:10:44.233672+0000","last_active":"2023-10-08T07:10:45.199298+0000","last_peered":"2023-10-08T07:10:45.199298+0000","last_clean":"2023-10-08T07:10:45.199298+0000","last_became_active":"2023-10-08T07:10:44.233185+0000","last_became_peered":"2023-10-08T07:10:44.233185+0000","last_unstale":"2023-10-08T07:10:45.199298+0000","last_undegraded":"2023-10-08T07:10:45.199298+0000","last_fullsized":"2023-10-08T07:10:45.199298+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:22.243718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239699+0000","last_change":"2023-10-08T07:10:43.668769+0000","last_active":"2023-10-08T07:10:45.239699+0000","last_peered":"2023-10-08T07:10:45.239699+0000","last_clean":"2023-10-08T07:10:45.239699+0000","last_became_active":"2023-10-08T07:10:43.667704+0000","last_became_peered":"2023-10-08T07:10:43.667704+0000","last_unstale":"2023-10-08T07:10:45.239699+0000","last_undegraded":"2023-10-08T07:10:45.239699+0000","last_fullsized":"2023-10-08T07:10:45.239699+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:42.874794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.204869+0000","last_change":"2023-10-08T07:10:44.218609+0000","last_active":"2023-10-08T07:10:45.204869+0000","last_peered":"2023-10-08T07:10:45.204869+0000","last_clean":"2023-10-08T07:10:45.204869+0000","last_became_active":"2023-10-08T07:10:44.218129+0000","last_became_peered":"2023-10-08T07:10:44.218129+0000","last_unstale":"2023-10-08T07:10:45.204869+0000","last_undegraded":"2023-10-08T07:10:45.204869+0000","last_fullsized":"2023-10-08T07:10:45.204869+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:03.789076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238800+0000","last_change":"2023-10-08T07:10:43.684266+0000","last_active":"2023-10-08T07:10:45.238800+0000","last_peered":"2023-10-08T07:10:45.238800+0000","last_clean":"2023-10-08T07:10:45.238800+0000","last_became_active":"2023-10-08T07:10:43.683925+0000","last_became_peered":"2023-10-08T07:10:43.683925+0000","last_unstale":"2023-10-08T07:10:45.238800+0000","last_undegraded":"2023-10-08T07:10:45.238800+0000","last_fullsized":"2023-10-08T07:10:45.238800+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:14:29.123634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237607+0000","last_change":"2023-10-08T07:10:44.237247+0000","last_active":"2023-10-08T07:10:45.237607+0000","last_peered":"2023-10-08T07:10:45.237607+0000","last_clean":"2023-10-08T07:10:45.237607+0000","last_became_active":"2023-10-08T07:10:44.236916+0000","last_became_peered":"2023-10-08T07:10:44.236916+0000","last_unstale":"2023-10-08T07:10:45.237607+0000","last_undegraded":"2023-10-08T07:10:45.237607+0000","last_fullsized":"2023-10-08T07:10:45.237607+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:58:12.789588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238147+0000","last_change":"2023-10-08T07:10:43.681762+0000","last_active":"2023-10-08T07:10:45.238147+0000","last_peered":"2023-10-08T07:10:45.238147+0000","last_clean":"2023-10-08T07:10:45.238147+0000","last_became_active":"2023-10-08T07:10:43.681403+0000","last_became_peered":"2023-10-08T07:10:43.681403+0000","last_unstale":"2023-10-08T07:10:45.238147+0000","last_undegraded":"2023-10-08T07:10:45.238147+0000","last_fullsized":"2023-10-08T07:10:45.238147+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:17.389772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240975+0000","last_change":"2023-10-08T07:10:44.220396+0000","last_active":"2023-10-08T07:10:45.240975+0000","last_peered":"2023-10-08T07:10:45.240975+0000","last_clean":"2023-10-08T07:10:45.240975+0000","last_became_active":"2023-10-08T07:10:44.220038+0000","last_became_peered":"2023-10-08T07:10:44.220038+0000","last_unstale":"2023-10-08T07:10:45.240975+0000","last_undegraded":"2023-10-08T07:10:45.240975+0000","last_fullsized":"2023-10-08T07:10:45.240975+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:51.683157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.199051+0000","last_change":"2023-10-08T07:10:43.685876+0000","last_active":"2023-10-08T07:10:45.199051+0000","last_peered":"2023-10-08T07:10:45.199051+0000","last_clean":"2023-10-08T07:10:45.199051+0000","last_became_active":"2023-10-08T07:10:43.685578+0000","last_became_peered":"2023-10-08T07:10:43.685578+0000","last_unstale":"2023-10-08T07:10:45.199051+0000","last_undegraded":"2023-10-08T07:10:45.199051+0000","last_fullsized":"2023-10-08T07:10:45.199051+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:34.344958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238576+0000","last_change":"2023-10-08T07:10:44.227526+0000","last_active":"2023-10-08T07:10:45.238576+0000","last_peered":"2023-10-08T07:10:45.238576+0000","last_clean":"2023-10-08T07:10:45.238576+0000","last_became_active":"2023-10-08T07:10:44.227224+0000","last_became_peered":"2023-10-08T07:10:44.227224+0000","last_unstale":"2023-10-08T07:10:45.238576+0000","last_undegraded":"2023-10-08T07:10:45.238576+0000","last_fullsized":"2023-10-08T07:10:45.238576+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:56.419001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.205983+0000","last_change":"2023-10-08T07:10:43.682574+0000","last_active":"2023-10-08T07:10:45.205983+0000","last_peered":"2023-10-08T07:10:45.205983+0000","last_clean":"2023-10-08T07:10:45.205983+0000","last_became_active":"2023-10-08T07:10:43.682194+0000","last_became_peered":"2023-10-08T07:10:43.682194+0000","last_unstale":"2023-10-08T07:10:45.205983+0000","last_undegraded":"2023-10-08T07:10:45.205983+0000","last_fullsized":"2023-10-08T07:10:45.205983+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:58:42.063439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.205826+0000","last_change":"2023-10-08T07:10:44.216912+0000","last_active":"2023-10-08T07:10:45.205826+0000","last_peered":"2023-10-08T07:10:45.205826+0000","last_clean":"2023-10-08T07:10:45.205826+0000","last_became_active":"2023-10-08T07:10:44.216268+0000","last_became_peered":"2023-10-08T07:10:44.216268+0000","last_unstale":"2023-10-08T07:10:45.205826+0000","last_undegraded":"2023-10-08T07:10:45.205826+0000","last_fullsized":"2023-10-08T07:10:45.205826+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:17.895682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.236990+0000","last_change":"2023-10-08T07:10:43.685985+0000","last_active":"2023-10-08T07:10:45.236990+0000","last_peered":"2023-10-08T07:10:45.236990+0000","last_clean":"2023-10-08T07:10:45.236990+0000","last_became_active":"2023-10-08T07:10:43.685720+0000","last_became_peered":"2023-10-08T07:10:43.685720+0000","last_unstale":"2023-10-08T07:10:45.236990+0000","last_undegraded":"2023-10-08T07:10:45.236990+0000","last_fullsized":"2023-10-08T07:10:45.236990+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:25:46.651061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"203'2784","reported_seq":3628,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-08T07:10:44.214136+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:10:44.214136+0000","last_peered":"2023-10-08T07:10:44.214136+0000","last_clean":"2023-10-08T07:10:44.214136+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:10:44.214136+0000","last_undegraded":"2023-10-08T07:10:44.214136+0000","last_fullsized":"2023-10-08T07:10:44.214136+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2784,"log_dups_size":0,"ondisk_log_size":2784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2013,"num_read_kb":4206,"num_write":549,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2013,"num_read_kb":4206,"num_write":549,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2784,"ondisk_log_size":2784,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575976,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60772,"kb_used_data":3224,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311068,"statfs":{"total":96636764160,"available":96574533632,"internally_reserved":0,"allocated":3301376,"data_stored":1991986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.218},"min":{"1min":0.78400000000000003,"5min":0.70799999999999996,"15min":0.64400000000000002},"max":{"1min":1.7170000000000001,"5min":1.804,"15min":2.008},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.2170000000000001,"15min":1.2210000000000001},"min":{"1min":0.92800000000000005,"5min":0.82199999999999995,"15min":0.78100000000000003},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.7989999999999999},"last":1.03}]},{"osd":1,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2609999999999999,"15min":1.266},"min":{"1min":1.0009999999999999,"5min":0.78600000000000003,"15min":0.67300000000000004},"max":{"1min":1.601,"5min":1.742,"15min":1.998},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.325},"min":{"1min":0.97599999999999998,"5min":0.873,"15min":0.84199999999999997},"max":{"1min":1.7490000000000001,"5min":1.9159999999999999,"15min":2.1019999999999999},"last":1.5940000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3169999999999999,"15min":1.2969999999999999},"min":{"1min":0.90400000000000003,"5min":0.88200000000000001,"15min":0.60799999999999998},"max":{"1min":1.5569999999999999,"5min":1.7789999999999999,"15min":2.1640000000000001},"last":1.464},{"interface":"front","average":{"1min":1.397,"5min":1.335,"15min":1.345},"min":{"1min":0.96599999999999997,"5min":0.91800000000000004,"15min":0.86799999999999999},"max":{"1min":1.76,"5min":1.76,"15min":2.3170000000000002},"last":1.4350000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3460000000000001,"15min":1.3400000000000001},"min":{"1min":0.86899999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8160000000000001,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.409,"15min":1.427},"min":{"1min":1.0469999999999999,"5min":1.022,"15min":0.98499999999999999},"max":{"1min":1.8480000000000001,"5min":1.8700000000000001,"15min":1.931},"last":0.98299999999999998}]},{"osd":4,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3120000000000001,"15min":1.3069999999999999},"min":{"1min":0.86699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.8,"5min":1.8,"15min":2.0249999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3939999999999999,"15min":1.4059999999999999},"min":{"1min":0.94799999999999995,"5min":0.93600000000000005,"15min":0.85199999999999998},"max":{"1min":1.8140000000000001,"5min":2.0379999999999998,"15min":2.1600000000000001},"last":1.5329999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.431,"15min":1.456},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.9630000000000001,"5min":2.1139999999999999,"15min":2.7360000000000002},"last":1.625},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4870000000000001,"15min":1.5109999999999999},"min":{"1min":1.006,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":1.964,"5min":1.972,"15min":2.569},"last":1.3149999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5940000000000001,"15min":1.589},"min":{"1min":1.276,"5min":1.155,"15min":1.1299999999999999},"max":{"1min":2.0289999999999999,"5min":2.157,"15min":2.2189999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.5269999999999999,"15min":1.534},"min":{"1min":1.1910000000000001,"5min":1.022,"15min":0.90100000000000002},"max":{"1min":2.21,"5min":2.21,"15min":2.262},"last":1.0960000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575975,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42156,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329684,"statfs":{"total":96636764160,"available":96593596416,"internally_reserved":0,"allocated":2711552,"data_stored":1401688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.282,"15min":1.3220000000000001},"min":{"1min":0.81100000000000005,"5min":0.68400000000000005,"15min":0.52200000000000002},"max":{"1min":1.794,"5min":1.905,"15min":1.962},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.371,"15min":1.3740000000000001},"min":{"1min":1.1479999999999999,"5min":1.042,"15min":0.40799999999999997},"max":{"1min":1.6100000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.2390000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2529999999999999,"15min":1.2410000000000001},"min":{"1min":0.89100000000000001,"5min":0.86299999999999999,"15min":0.65300000000000002},"max":{"1min":1.722,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.109},{"interface":"front","average":{"1min":1.153,"5min":1.216,"15min":1.238},"min":{"1min":0.86899999999999999,"5min":0.76300000000000001,"15min":0.63100000000000001},"max":{"1min":1.5549999999999999,"5min":1.97,"15min":1.97},"last":1.2969999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.21,"15min":1.256},"min":{"1min":0.84899999999999998,"5min":0.78800000000000003,"15min":0.67400000000000004},"max":{"1min":1.4450000000000001,"5min":2.339,"15min":2.339},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3200000000000001,"15min":1.3160000000000001},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.59499999999999997},"max":{"1min":1.6970000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.3580000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4039999999999999,"15min":1.409},"min":{"1min":0.97699999999999998,"5min":0.89900000000000002,"15min":0.69499999999999995},"max":{"1min":1.7749999999999999,"5min":1.95,"15min":1.95},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.5329999999999999,"15min":1.5109999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.71099999999999997},"max":{"1min":1.8380000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.90000000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.4159999999999999,"15min":1.3839999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.63900000000000001},"max":{"1min":1.78,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.413},{"interface":"front","average":{"1min":1.405,"5min":1.4850000000000001,"15min":1.4419999999999999},"min":{"1min":1.091,"5min":0.57099999999999995,"15min":0.55300000000000005},"max":{"1min":1.75,"5min":2.484,"15min":2.484},"last":1.8640000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4750000000000001,"15min":1.421},"min":{"1min":1.0489999999999999,"5min":0.89100000000000001,"15min":0.42899999999999999},"max":{"1min":1.6950000000000001,"5min":2.403,"15min":2.403},"last":1.581},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5629999999999999,"15min":1.548},"min":{"1min":1.105,"5min":1.071,"15min":0.73799999999999999},"max":{"1min":1.875,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":1.7689999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5980000000000001,"15min":1.5880000000000001},"min":{"1min":1.012,"5min":1.012,"15min":0.62},"max":{"1min":1.798,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.641},{"interface":"front","average":{"1min":1.54,"5min":1.585,"15min":1.5509999999999999},"min":{"1min":1.228,"5min":1.024,"15min":0.79900000000000004},"max":{"1min":1.8220000000000001,"5min":2.665,"15min":2.665},"last":1.7090000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60216,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311624,"statfs":{"total":96636764160,"available":96575102976,"internally_reserved":0,"allocated":2723840,"data_stored":1403252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1970000000000001,"15min":1.2250000000000001},"min":{"1min":0.85899999999999999,"5min":0.75700000000000001,"15min":0.74099999999999999},"max":{"1min":1.589,"5min":1.8320000000000001,"15min":1.841},"last":1.387},{"interface":"front","average":{"1min":1.204,"5min":1.234,"15min":1.236},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6699999999999999,"5min":1.7609999999999999,"15min":1.9610000000000001},"last":1.0469999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.22,"15min":1.2490000000000001},"min":{"1min":0.92900000000000005,"5min":0.82199999999999995,"15min":0.60799999999999998},"max":{"1min":1.524,"5min":1.712,"15min":1.9550000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.429,"5min":1.3220000000000001,"15min":1.3200000000000001},"min":{"1min":0.999,"5min":0.83999999999999997,"15min":0.56899999999999995},"max":{"1min":1.825,"5min":1.825,"15min":1.978},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.296,"15min":1.3049999999999999},"min":{"1min":0.82599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6020000000000001,"5min":2.056,"15min":2.0870000000000002},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.2949999999999999,"15min":1.306},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.6479999999999999,"5min":1.7090000000000001,"15min":2.0870000000000002},"last":1.5009999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.387,"15min":1.395},"min":{"1min":1.2030000000000001,"5min":0.91900000000000004,"15min":0.69399999999999995},"max":{"1min":1.794,"5min":1.794,"15min":2.1499999999999999},"last":1.48},{"interface":"front","average":{"1min":1.371,"5min":1.3979999999999999,"15min":1.446},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.681,"5min":2.173,"15min":2.173},"last":1.631}]},{"osd":4,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.363,"15min":1.337},"min":{"1min":0.92600000000000005,"5min":0.85299999999999998,"15min":0.82599999999999996},"max":{"1min":1.6799999999999999,"5min":1.8220000000000001,"15min":2.0150000000000001},"last":1.113},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.427},"min":{"1min":0.88,"5min":0.748,"15min":0.54000000000000004},"max":{"1min":1.8759999999999999,"5min":2.117,"15min":2.2240000000000002},"last":1.5409999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.415,"15min":1.4430000000000001},"min":{"1min":0.66500000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.736,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.454,"15min":1.4690000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.7490000000000001,"5min":1.905,"15min":2.145},"last":1.2370000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.546,"15min":1.5640000000000001},"min":{"1min":1.113,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8779999999999999,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.595},{"interface":"front","average":{"1min":1.474,"5min":1.577,"15min":1.6080000000000001},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.53500000000000003},"max":{"1min":1.8420000000000001,"5min":2.282,"15min":2.3199999999999998},"last":1.6930000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":2723840,"data_stored":1403264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3620000000000001,"15min":1.361},"min":{"1min":1.0409999999999999,"5min":0.85599999999999998,"15min":0.76800000000000002},"max":{"1min":1.698,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.4510000000000001,"15min":1.419},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.749},"max":{"1min":1.778,"5min":1.925,"15min":2.903},"last":1.0660000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.339,"15min":1.3109999999999999},"min":{"1min":0.89600000000000002,"5min":0.88300000000000001,"15min":0.76200000000000001},"max":{"1min":1.806,"5min":1.976,"15min":2.1600000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.2150000000000001,"15min":1.2450000000000001},"min":{"1min":0.93899999999999995,"5min":0.82599999999999996,"15min":0.65200000000000002},"max":{"1min":1.756,"5min":1.7929999999999999,"15min":1.875},"last":1.274}]},{"osd":2,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.256,"15min":1.2649999999999999},"min":{"1min":0.90600000000000003,"5min":0.90000000000000002,"15min":0.59399999999999997},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.2589999999999999},"last":1.329},{"interface":"front","average":{"1min":1.343,"5min":1.403,"15min":1.3680000000000001},"min":{"1min":1.0409999999999999,"5min":0.89900000000000002,"15min":0.81699999999999995},"max":{"1min":1.7450000000000001,"5min":1.853,"15min":2.3149999999999999},"last":1.454}]},{"osd":3,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3340000000000001,"15min":1.319},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.872},"max":{"1min":1.6850000000000001,"5min":1.7190000000000001,"15min":2.028},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.4279999999999999,"15min":1.425},"min":{"1min":1.341,"5min":0.83799999999999997,"15min":0.73399999999999999},"max":{"1min":1.677,"5min":1.8979999999999999,"15min":2.093},"last":1.4199999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.3600000000000001,"15min":1.371},"min":{"1min":1.0409999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.7969999999999999,"5min":2.0289999999999999,"15min":2.0830000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.3959999999999999,"15min":1.3839999999999999},"min":{"1min":0.97999999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.853,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.8700000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4570000000000001,"15min":1.45},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7430000000000001,"5min":1.992,"15min":2.6829999999999998},"last":1.54},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5620000000000001,"15min":1.5429999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.70499999999999996},"max":{"1min":1.794,"5min":2.1429999999999998,"15min":2.7429999999999999},"last":1.913}]},{"osd":7,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.524,"15min":1.5049999999999999},"min":{"1min":1.3500000000000001,"5min":0.95599999999999996,"15min":0.95499999999999996},"max":{"1min":1.8879999999999999,"5min":2.1230000000000002,"15min":2.1240000000000001},"last":1.9470000000000001},{"interface":"front","average":{"1min":1.722,"5min":1.6559999999999999,"15min":1.629},"min":{"1min":1.278,"5min":1.194,"15min":1.0309999999999999},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.52},"last":1.639}]}]},{"osd":3,"up_from":13,"seq":55834575977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51880,"kb_used_data":2652,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319960,"statfs":{"total":96636764160,"available":96583639040,"internally_reserved":0,"allocated":2715648,"data_stored":1401778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.012,"15min":0.995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.7599999999999998,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.325,"5min":1.2090000000000001,"15min":1.1950000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.5659999999999998,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.1459999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3169999999999999,"5min":1.79,"15min":2.4390000000000001},"last":0.996},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2989999999999999,"15min":1.3009999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.587,"5min":1.8260000000000001,"15min":2.4889999999999999},"last":1.27}]},{"osd":2,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.161,"15min":1.1599999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.728,"5min":1.728,"15min":2.2250000000000001},"last":1.204},{"interface":"front","average":{"1min":1.175,"5min":1.2070000000000001,"15min":1.2110000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.63},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":2.2930000000000001},"last":1.0900000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2889999999999999,"15min":1.292},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.67500000000000004},"max":{"1min":1.857,"5min":1.857,"15min":2.5379999999999998},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4650000000000001,"15min":1.4730000000000001},"min":{"1min":1.032,"5min":1.032,"15min":0.78100000000000003},"max":{"1min":1.8109999999999999,"5min":1.867,"15min":2.8340000000000001},"last":1.502}]},{"osd":5,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3540000000000001,"15min":1.391},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5369999999999999,"5min":1.7529999999999999,"15min":2.9740000000000002},"last":1.397},{"interface":"front","average":{"1min":1.407,"5min":1.464,"15min":1.4350000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.012,"5min":2.1869999999999998,"15min":2.7330000000000001},"last":1.5429999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.4330000000000001,"15min":1.4179999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.976,"5min":1.976,"15min":2.653},"last":1.458},{"interface":"front","average":{"1min":1.425,"5min":1.524,"15min":1.542},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.792,"5min":2.1579999999999999,"15min":3.0499999999999998},"last":1.6379999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.593,"15min":1.5760000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.353},{"interface":"front","average":{"1min":1.456,"5min":1.5289999999999999,"15min":1.5389999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.9410000000000001,"5min":2.0670000000000002,"15min":2.7749999999999999},"last":1.3140000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575975,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51880,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319960,"statfs":{"total":96636764160,"available":96583639040,"internally_reserved":0,"allocated":2711552,"data_stored":1402200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.99299999999999999,"15min":0.97099999999999997},"min":{"1min":0.66000000000000003,"5min":0.63700000000000001,"15min":0.41399999999999998},"max":{"1min":1.3180000000000001,"5min":1.74,"15min":1.948},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.998,"5min":1.1020000000000001,"15min":1.0780000000000001},"min":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.38400000000000001},"max":{"1min":1.3320000000000001,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.58899999999999997}]},{"osd":1,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.999,"15min":1.0229999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.495},"max":{"1min":1.522,"5min":2.044,"15min":2.044},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.256,"5min":1.226,"15min":1.1850000000000001},"min":{"1min":0.97499999999999998,"5min":0.83099999999999996,"15min":0.52500000000000002},"max":{"1min":1.579,"5min":1.724,"15min":2.1309999999999998},"last":1.5249999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.194,"15min":1.1930000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.56000000000000005},"max":{"1min":1.649,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.296,"15min":1.2769999999999999},"min":{"1min":0.98699999999999999,"5min":0.92000000000000004,"15min":0.57399999999999995},"max":{"1min":1.6299999999999999,"5min":1.8,"15min":2.0670000000000002},"last":1.3340000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4510000000000001,"15min":1.4430000000000001},"min":{"1min":1.0840000000000001,"5min":1.0580000000000001,"15min":0.66800000000000004},"max":{"1min":1.631,"5min":1.9570000000000001,"15min":2.1789999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.3300000000000001,"15min":1.294},"min":{"1min":1.036,"5min":0.96599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6830000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.95799999999999996}]},{"osd":5,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.401,"15min":1.4159999999999999},"min":{"1min":1.085,"5min":1.022,"15min":0.61399999999999999},"max":{"1min":1.7270000000000001,"5min":2.1429999999999998,"15min":2.3420000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.524,"15min":1.53},"min":{"1min":1.1930000000000001,"5min":1.093,"15min":0.70699999999999996},"max":{"1min":1.9630000000000001,"5min":2.081,"15min":2.9390000000000001},"last":1.6140000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.403,"15min":1.4019999999999999},"min":{"1min":0.96299999999999997,"5min":0.92600000000000005,"15min":0.85399999999999998},"max":{"1min":1.6919999999999999,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.5349999999999999,"15min":1.554},"min":{"1min":1.3129999999999999,"5min":1.1499999999999999,"15min":0.76700000000000002},"max":{"1min":1.7529999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.387}]},{"osd":7,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4970000000000001,"15min":1.4810000000000001},"min":{"1min":1.1379999999999999,"5min":1.0680000000000001,"15min":0.64200000000000002},"max":{"1min":1.8859999999999999,"5min":2.3069999999999999,"15min":2.4689999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5649999999999999,"15min":1.605},"min":{"1min":1.252,"5min":1.23,"15min":0.72799999999999998},"max":{"1min":1.847,"5min":2.2829999999999999,"15min":2.3879999999999999},"last":1.5569999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51872,"kb_used_data":2644,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319968,"statfs":{"total":96636764160,"available":96583647232,"internally_reserved":0,"allocated":2707456,"data_stored":1401618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0129999999999999,"15min":1.0169999999999999},"min":{"1min":0.79900000000000004,"5min":0.60699999999999998,"15min":0.55400000000000005},"max":{"1min":1.5660000000000001,"5min":1.657,"15min":1.7769999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.98299999999999998,"5min":0.91100000000000003,"15min":0.46400000000000002},"max":{"1min":1.6950000000000001,"5min":1.7609999999999999,"15min":1.79},"last":0.63800000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.002,"15min":0.98799999999999999},"min":{"1min":0.83499999999999996,"5min":0.70499999999999996,"15min":0.47999999999999998},"max":{"1min":1.476,"5min":1.542,"15min":2.1890000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.208,"15min":1.1839999999999999},"min":{"1min":0.97899999999999998,"5min":0.86499999999999999,"15min":0.54600000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.7929999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2929999999999999,"15min":1.2729999999999999},"min":{"1min":0.82399999999999995,"5min":0.78300000000000003,"15min":0.59399999999999997},"max":{"1min":1.609,"5min":1.651,"15min":1.9139999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.204,"5min":1.143,"15min":1.1719999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.62},"max":{"1min":1.883,"5min":1.883,"15min":2.7429999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3819999999999999,"15min":1.3919999999999999},"min":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.86699999999999999},"max":{"1min":1.6080000000000001,"5min":2.0129999999999999,"15min":2.8460000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.3460000000000001,"15min":1.3160000000000001},"min":{"1min":1.0960000000000001,"5min":0.99099999999999999,"15min":0.66500000000000004},"max":{"1min":1.613,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.86099999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3899999999999999,"15min":1.387},"min":{"1min":1.179,"5min":1.01,"15min":0.76700000000000002},"max":{"1min":1.827,"5min":1.978,"15min":2.3500000000000001},"last":1.103},{"interface":"front","average":{"1min":1.635,"5min":1.589,"15min":1.5529999999999999},"min":{"1min":1.3759999999999999,"5min":1.1499999999999999,"15min":0.72899999999999998},"max":{"1min":2.0659999999999998,"5min":2.2610000000000001,"15min":2.528},"last":0.84599999999999997}]},{"osd":6,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.536,"15min":1.5409999999999999},"min":{"1min":1.4470000000000001,"5min":1.123,"15min":0.82699999999999996},"max":{"1min":1.8620000000000001,"5min":1.9730000000000001,"15min":2.9079999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.585,"5min":1.5389999999999999,"15min":1.5069999999999999},"min":{"1min":1.234,"5min":1.1080000000000001,"15min":0.71699999999999997},"max":{"1min":1.966,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.1419999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.619,"15min":1.5740000000000001},"min":{"1min":1.236,"5min":1.071,"15min":0.69399999999999995},"max":{"1min":2.1499999999999999,"5min":2.2610000000000001,"15min":2.46},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6000000000000001,"15min":1.6020000000000001},"min":{"1min":1.2250000000000001,"5min":1.2050000000000001,"15min":0.81200000000000006},"max":{"1min":1.9299999999999999,"5min":2.0139999999999998,"15min":3.2160000000000002},"last":1.1240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56636,"kb_used_data":3240,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315204,"statfs":{"total":96636764160,"available":96578768896,"internally_reserved":0,"allocated":3317760,"data_stored":1992714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":54655433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.95099999999999996,"15min":0.97799999999999998},"min":{"1min":0.68300000000000005,"5min":0.60399999999999998,"15min":0.46000000000000002},"max":{"1min":1.2709999999999999,"5min":1.5369999999999999,"15min":1.571},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1259999999999999,"15min":1.113},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.51500000000000001},"max":{"1min":1.466,"5min":1.6619999999999999,"15min":1.726},"last":1.073}]},{"osd":2,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.1559999999999999,"15min":1.1439999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.497},"max":{"1min":1.4299999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.234},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.085,"15min":1.087},"min":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.42299999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":1.1259999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2669999999999999,"15min":1.2649999999999999},"min":{"1min":0.94499999999999995,"5min":0.91800000000000004,"15min":0.52800000000000002},"max":{"1min":1.6299999999999999,"5min":1.77,"15min":1.8420000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.1659999999999999,"15min":1.1779999999999999},"min":{"1min":0.872,"5min":0.80000000000000004,"15min":0.47999999999999998},"max":{"1min":1.752,"5min":1.752,"15min":2.0979999999999999},"last":1.367}]},{"osd":4,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.329,"15min":1.3600000000000001},"min":{"1min":1.044,"5min":1.0069999999999999,"15min":0.79100000000000004},"max":{"1min":1.589,"5min":1.73,"15min":2.1040000000000001},"last":1.609},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4379999999999999,"15min":1.4390000000000001},"min":{"1min":1.079,"5min":1.079,"15min":0.96099999999999997},"max":{"1min":1.956,"5min":1.956,"15min":2.181},"last":1.1930000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4379999999999999,"15min":1.4199999999999999},"min":{"1min":1.107,"5min":1.087,"15min":0.873},"max":{"1min":1.919,"5min":1.919,"15min":2.1309999999999998},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.466,"15min":1.4870000000000001},"min":{"1min":1.2410000000000001,"5min":0.998,"15min":0.81000000000000005},"max":{"1min":1.7749999999999999,"5min":1.9490000000000001,"15min":2.4900000000000002},"last":1.5820000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4730000000000001,"15min":1.49},"min":{"1min":1.1930000000000001,"5min":1.036,"15min":1.036},"max":{"1min":2.0369999999999999,"5min":2.109,"15min":2.2149999999999999},"last":1.282},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.514,"15min":1.514},"min":{"1min":1.1719999999999999,"5min":1.024,"15min":0.89500000000000002},"max":{"1min":1.849,"5min":2.0099999999999998,"15min":2.355},"last":1.4490000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.552,"15min":1.552},"min":{"1min":1.2609999999999999,"5min":1.099,"15min":0.82099999999999995},"max":{"1min":2.0859999999999999,"5min":2.1200000000000001,"15min":2.3100000000000001},"last":1.321},{"interface":"front","average":{"1min":1.605,"5min":1.544,"15min":1.556},"min":{"1min":1.3160000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"max":{"1min":2.0950000000000002,"5min":2.0990000000000002,"15min":2.1349999999999998},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:10:49.090 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:10:49.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.341+0000 7f862b59e640 1 Processor -- start 2023-10-08T07:10:49.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.341+0000 7f862b59e640 1 -- start start 2023-10-08T07:10:49.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.342+0000 7f862b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f862c15ab30 0x7f862c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.342+0000 7f862b59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c15f7d0 con 0x7f862c0adc70 2023-10-08T07:10:49.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.342+0000 7f862b59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c15f210 con 0x7f862c0b07d0 2023-10-08T07:10:49.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.342+0000 7f862b59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862c15f4f0 con 0x7f862c15ab30 2023-10-08T07:10:49.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.342+0000 7f862a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f862c15ab30 0x7f862c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:49.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.342+0000 7f8621d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f862c0adc70 0x7f862c0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57712/0 (socket says 172.21.15.70:57712) 2023-10-08T07:10:49.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.342+0000 7f8621d9b640 1 -- 172.21.15.70:0/3434120463 learned_addr learned my addr 172.21.15.70:0/3434120463 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:49.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.343+0000 7f862a59c640 1 -- 172.21.15.70:0/3434120463 >> v1:172.21.15.181:6789/0 conn(0x7f862c0b07d0 legacy=0x7f862c06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:49.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.343+0000 7f862a59c640 1 -- 172.21.15.70:0/3434120463 >> v1:172.21.15.70:6789/0 conn(0x7f862c0adc70 legacy=0x7f862c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.343+0000 7f862a59c640 1 -- 172.21.15.70:0/3434120463 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c06c550 con 0x7f862c15ab30 2023-10-08T07:10:49.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.343+0000 7f8629d9b640 1 -- 172.21.15.70:0/3434120463 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988858427 0 0) 0x7f862c15f7d0 con 0x7f862c0adc70 2023-10-08T07:10:49.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.344+0000 7f862a59c640 1 --2- 172.21.15.70:0/3434120463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c15ab30 0x7f862c15e8a0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f861c0097a0 tx=0x7f861c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be18c3cda18ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:49.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.344+0000 7f8629d9b640 1 -- 172.21.15.70:0/3434120463 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c031830 con 0x7f862c15ab30 2023-10-08T07:10:49.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.344+0000 7f8629d9b640 1 -- 172.21.15.70:0/3434120463 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c031990 con 0x7f862c15ab30 2023-10-08T07:10:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.344+0000 7f8629d9b640 1 -- 172.21.15.70:0/3434120463 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c031d20 con 0x7f862c15ab30 2023-10-08T07:10:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.345+0000 7f862b59e640 1 -- 172.21.15.70:0/3434120463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c15ab30 msgr2=0x7f862c15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.345+0000 7f862b59e640 1 --2- 172.21.15.70:0/3434120463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c15ab30 0x7f862c15e8a0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f861c0097a0 tx=0x7f861c02b8d0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.345+0000 7f862b59e640 1 -- 172.21.15.70:0/3434120463 shutdown_connections 2023-10-08T07:10:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.345+0000 7f862b59e640 1 --2- 172.21.15.70:0/3434120463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c15ab30 0x7f862c15e8a0 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.345+0000 7f862b59e640 1 -- 172.21.15.70:0/3434120463 >> 172.21.15.70:0/3434120463 conn(0x7f862c0cb0c0 msgr2=0x7f862c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.345+0000 7f862b59e640 1 -- 172.21.15.70:0/3434120463 shutdown_connections 2023-10-08T07:10:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.345+0000 7f862b59e640 1 -- 172.21.15.70:0/3434120463 wait complete. 2023-10-08T07:10:49.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.346+0000 7f862b59e640 1 Processor -- start 2023-10-08T07:10:49.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.347+0000 7f862b59e640 1 -- start start 2023-10-08T07:10:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f862b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f862c0adc70 0x7f862c152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f862b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c0b07d0 0x7f862c1f0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f862b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c15ab30 0x7f862c15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f862b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c15b790 con 0x7f862c0b07d0 2023-10-08T07:10:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f862b59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1f0580 con 0x7f862c15ab30 2023-10-08T07:10:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f862b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1503f0 con 0x7f862c0adc70 2023-10-08T07:10:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f8621d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c0b07d0 0x7f862c1f0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f8621d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c0b07d0 0x7f862c1f0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60764/0 (socket says 172.21.15.70:60764) 2023-10-08T07:10:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f8621d9b640 1 -- 172.21.15.70:0/1363342278 learned_addr learned my addr 172.21.15.70:0/1363342278 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:49.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f8621d9b640 1 -- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f862c0adc70 msgr2=0x7f862c152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f862a59c640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f862c0adc70 0x7f862c152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:49.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f862ad9d640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c15ab30 0x7f862c15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f8621d9b640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f862c0adc70 0x7f862c152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f8621d9b640 1 -- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c15ab30 msgr2=0x7f862c15bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f8621d9b640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c15ab30 0x7f862c15bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.348+0000 7f8621d9b640 1 -- 172.21.15.70:0/1363342278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c009400 con 0x7f862c0b07d0 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.349+0000 7f8621d9b640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c0b07d0 0x7f862c1f0040 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f861400c4d0 tx=0x7f861400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.349+0000 7f86237fe640 1 -- 172.21.15.70:0/1363342278 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86140074e0 con 0x7f862c0b07d0 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.349+0000 7f86237fe640 1 -- 172.21.15.70:0/1363342278 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8614007660 con 0x7f862c0b07d0 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.349+0000 7f86237fe640 1 -- 172.21.15.70:0/1363342278 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8614004a20 con 0x7f862c0b07d0 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.349+0000 7f862b59e640 1 -- 172.21.15.70:0/1363342278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c1506d0 con 0x7f862c0b07d0 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.349+0000 7f862b59e640 1 -- 172.21.15.70:0/1363342278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c15b420 con 0x7f862c0b07d0 2023-10-08T07:10:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.350+0000 7f862b59e640 1 -- 172.21.15.70:0/1363342278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862c15afa0 con 0x7f862c0b07d0 2023-10-08T07:10:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.351+0000 7f86237fe640 1 -- 172.21.15.70:0/1363342278 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8614004b80 con 0x7f862c0b07d0 2023-10-08T07:10:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.352+0000 7f86237fe640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f860c06af90 0x7f860c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.352+0000 7f86237fe640 1 -- 172.21.15.70:0/1363342278 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f8614088af0 con 0x7f862c0b07d0 2023-10-08T07:10:49.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.353+0000 7f862a59c640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f860c06af90 0x7f860c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:49.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.354+0000 7f862a59c640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f860c06af90 0x7f860c06d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f861c036000 tx=0x7f861c008030 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:49.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.356+0000 7f86237fe640 1 -- 172.21.15.70:0/1363342278 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f862c15afa0 con 0x7f862c0b07d0 2023-10-08T07:10:49.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.540+0000 7f862b59e640 1 -- 172.21.15.70:0/1363342278 --> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f862c06c830 con 0x7f860c06af90 2023-10-08T07:10:49.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.565+0000 7f86237fe640 1 -- 172.21.15.70:0/1363342278 <== mgr.7112 v2:172.21.15.181:6832/2638163552 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286827 (secure 0 0 0) 0x7f861c031b00 con 0x7f860c06af90 2023-10-08T07:10:49.571 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:49.573 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:10:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 -- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f860c06af90 msgr2=0x7f860c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f860c06af90 0x7f860c06d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f861c036000 tx=0x7f861c008030 comp rx=0 tx=0).stop 2023-10-08T07:10:49.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 -- 172.21.15.70:0/1363342278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c0b07d0 msgr2=0x7f862c1f0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c0b07d0 0x7f862c1f0040 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f861400c4d0 tx=0x7f861400c990 comp rx=0 tx=0).stop 2023-10-08T07:10:49.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 -- 172.21.15.70:0/1363342278 shutdown_connections 2023-10-08T07:10:49.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f860c06af90 0x7f860c06d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f862c15ab30 0x7f862c15bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f862c0b07d0 0x7f862c1f0040 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 --2- 172.21.15.70:0/1363342278 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f862c0adc70 0x7f862c152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 -- 172.21.15.70:0/1363342278 >> 172.21.15.70:0/1363342278 conn(0x7f862c0cb0c0 msgr2=0x7f862c0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:49.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 -- 172.21.15.70:0/1363342278 shutdown_connections 2023-10-08T07:10:49.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.570+0000 7f862b59e640 1 -- 172.21.15.70:0/1363342278 wait complete. 2023-10-08T07:10:49.590 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-08T07:10:49.508164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2013,"num_read_kb":4206,"num_write":593,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2822,"ondisk_log_size":2822,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423148,"kb_used_data":22376,"kb_used_omap":12,"kb_used_meta":400627,"kb_avail":754551572,"statfs":{"total":773094113280,"available":772660809728,"internally_reserved":0,"allocated":22913024,"data_stored":12398500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":410242619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002116"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241265+0000","last_change":"2023-10-08T07:10:44.226703+0000","last_active":"2023-10-08T07:10:45.241265+0000","last_peered":"2023-10-08T07:10:45.241265+0000","last_clean":"2023-10-08T07:10:45.241265+0000","last_became_active":"2023-10-08T07:10:44.226429+0000","last_became_peered":"2023-10-08T07:10:44.226429+0000","last_unstale":"2023-10-08T07:10:45.241265+0000","last_undegraded":"2023-10-08T07:10:45.241265+0000","last_fullsized":"2023-10-08T07:10:45.241265+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:46:52.000759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241407+0000","last_change":"2023-10-08T07:10:44.234875+0000","last_active":"2023-10-08T07:10:45.241407+0000","last_peered":"2023-10-08T07:10:45.241407+0000","last_clean":"2023-10-08T07:10:45.241407+0000","last_became_active":"2023-10-08T07:10:44.234520+0000","last_became_peered":"2023-10-08T07:10:44.234520+0000","last_unstale":"2023-10-08T07:10:45.241407+0000","last_undegraded":"2023-10-08T07:10:45.241407+0000","last_fullsized":"2023-10-08T07:10:45.241407+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:59.672226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206463+0000","last_change":"2023-10-08T07:10:44.220366+0000","last_active":"2023-10-08T07:10:45.206463+0000","last_peered":"2023-10-08T07:10:45.206463+0000","last_clean":"2023-10-08T07:10:45.206463+0000","last_became_active":"2023-10-08T07:10:44.219848+0000","last_became_peered":"2023-10-08T07:10:44.219848+0000","last_unstale":"2023-10-08T07:10:45.206463+0000","last_undegraded":"2023-10-08T07:10:45.206463+0000","last_fullsized":"2023-10-08T07:10:45.206463+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:41.310469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238444+0000","last_change":"2023-10-08T07:10:44.227862+0000","last_active":"2023-10-08T07:10:45.238444+0000","last_peered":"2023-10-08T07:10:45.238444+0000","last_clean":"2023-10-08T07:10:45.238444+0000","last_became_active":"2023-10-08T07:10:44.227562+0000","last_became_peered":"2023-10-08T07:10:44.227562+0000","last_unstale":"2023-10-08T07:10:45.238444+0000","last_undegraded":"2023-10-08T07:10:45.238444+0000","last_fullsized":"2023-10-08T07:10:45.238444+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:43.902521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241311+0000","last_change":"2023-10-08T07:10:44.236102+0000","last_active":"2023-10-08T07:10:45.241311+0000","last_peered":"2023-10-08T07:10:45.241311+0000","last_clean":"2023-10-08T07:10:45.241311+0000","last_became_active":"2023-10-08T07:10:44.235838+0000","last_became_peered":"2023-10-08T07:10:44.235838+0000","last_unstale":"2023-10-08T07:10:45.241311+0000","last_undegraded":"2023-10-08T07:10:45.241311+0000","last_fullsized":"2023-10-08T07:10:45.241311+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:40.764386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239007+0000","last_change":"2023-10-08T07:10:44.221854+0000","last_active":"2023-10-08T07:10:45.239007+0000","last_peered":"2023-10-08T07:10:45.239007+0000","last_clean":"2023-10-08T07:10:45.239007+0000","last_became_active":"2023-10-08T07:10:44.221582+0000","last_became_peered":"2023-10-08T07:10:44.221582+0000","last_unstale":"2023-10-08T07:10:45.239007+0000","last_undegraded":"2023-10-08T07:10:45.239007+0000","last_fullsized":"2023-10-08T07:10:45.239007+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:40:48.276610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198620+0000","last_change":"2023-10-08T07:10:44.231349+0000","last_active":"2023-10-08T07:10:45.198620+0000","last_peered":"2023-10-08T07:10:45.198620+0000","last_clean":"2023-10-08T07:10:45.198620+0000","last_became_active":"2023-10-08T07:10:44.230967+0000","last_became_peered":"2023-10-08T07:10:44.230967+0000","last_unstale":"2023-10-08T07:10:45.198620+0000","last_undegraded":"2023-10-08T07:10:45.198620+0000","last_fullsized":"2023-10-08T07:10:45.198620+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:29.380143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238357+0000","last_change":"2023-10-08T07:10:44.227636+0000","last_active":"2023-10-08T07:10:45.238357+0000","last_peered":"2023-10-08T07:10:45.238357+0000","last_clean":"2023-10-08T07:10:45.238357+0000","last_became_active":"2023-10-08T07:10:44.227332+0000","last_became_peered":"2023-10-08T07:10:44.227332+0000","last_unstale":"2023-10-08T07:10:45.238357+0000","last_undegraded":"2023-10-08T07:10:45.238357+0000","last_fullsized":"2023-10-08T07:10:45.238357+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:51.725953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241319+0000","last_change":"2023-10-08T07:10:44.231079+0000","last_active":"2023-10-08T07:10:45.241319+0000","last_peered":"2023-10-08T07:10:45.241319+0000","last_clean":"2023-10-08T07:10:45.241319+0000","last_became_active":"2023-10-08T07:10:44.230756+0000","last_became_peered":"2023-10-08T07:10:44.230756+0000","last_unstale":"2023-10-08T07:10:45.241319+0000","last_undegraded":"2023-10-08T07:10:45.241319+0000","last_fullsized":"2023-10-08T07:10:45.241319+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:01.919740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238499+0000","last_change":"2023-10-08T07:10:44.232986+0000","last_active":"2023-10-08T07:10:45.238499+0000","last_peered":"2023-10-08T07:10:45.238499+0000","last_clean":"2023-10-08T07:10:45.238499+0000","last_became_active":"2023-10-08T07:10:44.232705+0000","last_became_peered":"2023-10-08T07:10:44.232705+0000","last_unstale":"2023-10-08T07:10:45.238499+0000","last_undegraded":"2023-10-08T07:10:45.238499+0000","last_fullsized":"2023-10-08T07:10:45.238499+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:54:23.294834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.204015+0000","last_change":"2023-10-08T07:10:44.238805+0000","last_active":"2023-10-08T07:10:45.204015+0000","last_peered":"2023-10-08T07:10:45.204015+0000","last_clean":"2023-10-08T07:10:45.204015+0000","last_became_active":"2023-10-08T07:10:44.238455+0000","last_became_peered":"2023-10-08T07:10:44.238455+0000","last_unstale":"2023-10-08T07:10:45.204015+0000","last_undegraded":"2023-10-08T07:10:45.204015+0000","last_fullsized":"2023-10-08T07:10:45.204015+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:03:52.074707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238750+0000","last_change":"2023-10-08T07:10:44.224088+0000","last_active":"2023-10-08T07:10:45.238750+0000","last_peered":"2023-10-08T07:10:45.238750+0000","last_clean":"2023-10-08T07:10:45.238750+0000","last_became_active":"2023-10-08T07:10:44.223831+0000","last_became_peered":"2023-10-08T07:10:44.223831+0000","last_unstale":"2023-10-08T07:10:45.238750+0000","last_undegraded":"2023-10-08T07:10:45.238750+0000","last_fullsized":"2023-10-08T07:10:45.238750+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:51.683157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237714+0000","last_change":"2023-10-08T07:10:44.237662+0000","last_active":"2023-10-08T07:10:45.237714+0000","last_peered":"2023-10-08T07:10:45.237714+0000","last_clean":"2023-10-08T07:10:45.237714+0000","last_became_active":"2023-10-08T07:10:44.236939+0000","last_became_peered":"2023-10-08T07:10:44.236939+0000","last_unstale":"2023-10-08T07:10:45.237714+0000","last_undegraded":"2023-10-08T07:10:45.237714+0000","last_fullsized":"2023-10-08T07:10:45.237714+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:57.868685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198788+0000","last_change":"2023-10-08T07:10:44.232856+0000","last_active":"2023-10-08T07:10:45.198788+0000","last_peered":"2023-10-08T07:10:45.198788+0000","last_clean":"2023-10-08T07:10:45.198788+0000","last_became_active":"2023-10-08T07:10:44.232512+0000","last_became_peered":"2023-10-08T07:10:44.232512+0000","last_unstale":"2023-10-08T07:10:45.198788+0000","last_undegraded":"2023-10-08T07:10:45.198788+0000","last_fullsized":"2023-10-08T07:10:45.198788+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:14.876986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239919+0000","last_change":"2023-10-08T07:10:44.232537+0000","last_active":"2023-10-08T07:10:45.239919+0000","last_peered":"2023-10-08T07:10:45.239919+0000","last_clean":"2023-10-08T07:10:45.239919+0000","last_became_active":"2023-10-08T07:10:44.231905+0000","last_became_peered":"2023-10-08T07:10:44.231905+0000","last_unstale":"2023-10-08T07:10:45.239919+0000","last_undegraded":"2023-10-08T07:10:45.239919+0000","last_fullsized":"2023-10-08T07:10:45.239919+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:10.784865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238247+0000","last_change":"2023-10-08T07:10:44.224894+0000","last_active":"2023-10-08T07:10:45.238247+0000","last_peered":"2023-10-08T07:10:45.238247+0000","last_clean":"2023-10-08T07:10:45.238247+0000","last_became_active":"2023-10-08T07:10:44.224646+0000","last_became_peered":"2023-10-08T07:10:44.224646+0000","last_unstale":"2023-10-08T07:10:45.238247+0000","last_undegraded":"2023-10-08T07:10:45.238247+0000","last_fullsized":"2023-10-08T07:10:45.238247+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:46.835534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239232+0000","last_change":"2023-10-08T07:10:44.223684+0000","last_active":"2023-10-08T07:10:45.239232+0000","last_peered":"2023-10-08T07:10:45.239232+0000","last_clean":"2023-10-08T07:10:45.239232+0000","last_became_active":"2023-10-08T07:10:44.223417+0000","last_became_peered":"2023-10-08T07:10:44.223417+0000","last_unstale":"2023-10-08T07:10:45.239232+0000","last_undegraded":"2023-10-08T07:10:45.239232+0000","last_fullsized":"2023-10-08T07:10:45.239232+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:56:36.566069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238569+0000","last_change":"2023-10-08T07:10:44.232582+0000","last_active":"2023-10-08T07:10:45.238569+0000","last_peered":"2023-10-08T07:10:45.238569+0000","last_clean":"2023-10-08T07:10:45.238569+0000","last_became_active":"2023-10-08T07:10:44.232264+0000","last_became_peered":"2023-10-08T07:10:44.232264+0000","last_unstale":"2023-10-08T07:10:45.238569+0000","last_undegraded":"2023-10-08T07:10:45.238569+0000","last_fullsized":"2023-10-08T07:10:45.238569+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:06.376230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239571+0000","last_change":"2023-10-08T07:10:44.229054+0000","last_active":"2023-10-08T07:10:45.239571+0000","last_peered":"2023-10-08T07:10:45.239571+0000","last_clean":"2023-10-08T07:10:45.239571+0000","last_became_active":"2023-10-08T07:10:44.228712+0000","last_became_peered":"2023-10-08T07:10:44.228712+0000","last_unstale":"2023-10-08T07:10:45.239571+0000","last_undegraded":"2023-10-08T07:10:45.239571+0000","last_fullsized":"2023-10-08T07:10:45.239571+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:26:57.282217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239280+0000","last_change":"2023-10-08T07:10:44.223195+0000","last_active":"2023-10-08T07:10:45.239280+0000","last_peered":"2023-10-08T07:10:45.239280+0000","last_clean":"2023-10-08T07:10:45.239280+0000","last_became_active":"2023-10-08T07:10:44.222907+0000","last_became_peered":"2023-10-08T07:10:44.222907+0000","last_unstale":"2023-10-08T07:10:45.239280+0000","last_undegraded":"2023-10-08T07:10:45.239280+0000","last_fullsized":"2023-10-08T07:10:45.239280+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:05:39.781083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240982+0000","last_change":"2023-10-08T07:10:44.231122+0000","last_active":"2023-10-08T07:10:45.240982+0000","last_peered":"2023-10-08T07:10:45.240982+0000","last_clean":"2023-10-08T07:10:45.240982+0000","last_became_active":"2023-10-08T07:10:44.230844+0000","last_became_peered":"2023-10-08T07:10:44.230844+0000","last_unstale":"2023-10-08T07:10:45.240982+0000","last_undegraded":"2023-10-08T07:10:45.240982+0000","last_fullsized":"2023-10-08T07:10:45.240982+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:39:30.420420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240596+0000","last_change":"2023-10-08T07:10:44.231127+0000","last_active":"2023-10-08T07:10:45.240596+0000","last_peered":"2023-10-08T07:10:45.240596+0000","last_clean":"2023-10-08T07:10:45.240596+0000","last_became_active":"2023-10-08T07:10:44.230808+0000","last_became_peered":"2023-10-08T07:10:44.230808+0000","last_unstale":"2023-10-08T07:10:45.240596+0000","last_undegraded":"2023-10-08T07:10:45.240596+0000","last_fullsized":"2023-10-08T07:10:45.240596+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:59:42.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239727+0000","last_change":"2023-10-08T07:10:44.228180+0000","last_active":"2023-10-08T07:10:45.239727+0000","last_peered":"2023-10-08T07:10:45.239727+0000","last_clean":"2023-10-08T07:10:45.239727+0000","last_became_active":"2023-10-08T07:10:44.227845+0000","last_became_peered":"2023-10-08T07:10:44.227845+0000","last_unstale":"2023-10-08T07:10:45.239727+0000","last_undegraded":"2023-10-08T07:10:45.239727+0000","last_fullsized":"2023-10-08T07:10:45.239727+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:17.503602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241686+0000","last_change":"2023-10-08T07:10:44.220960+0000","last_active":"2023-10-08T07:10:45.241686+0000","last_peered":"2023-10-08T07:10:45.241686+0000","last_clean":"2023-10-08T07:10:45.241686+0000","last_became_active":"2023-10-08T07:10:44.219821+0000","last_became_peered":"2023-10-08T07:10:44.219821+0000","last_unstale":"2023-10-08T07:10:45.241686+0000","last_undegraded":"2023-10-08T07:10:45.241686+0000","last_fullsized":"2023-10-08T07:10:45.241686+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:36.260436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.199267+0000","last_change":"2023-10-08T07:10:44.217654+0000","last_active":"2023-10-08T07:10:45.199267+0000","last_peered":"2023-10-08T07:10:45.199267+0000","last_clean":"2023-10-08T07:10:45.199267+0000","last_became_active":"2023-10-08T07:10:44.217378+0000","last_became_peered":"2023-10-08T07:10:44.217378+0000","last_unstale":"2023-10-08T07:10:45.199267+0000","last_undegraded":"2023-10-08T07:10:45.199267+0000","last_fullsized":"2023-10-08T07:10:45.199267+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:44:45.377128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206297+0000","last_change":"2023-10-08T07:10:44.238798+0000","last_active":"2023-10-08T07:10:45.206297+0000","last_peered":"2023-10-08T07:10:45.206297+0000","last_clean":"2023-10-08T07:10:45.206297+0000","last_became_active":"2023-10-08T07:10:44.238387+0000","last_became_peered":"2023-10-08T07:10:44.238387+0000","last_unstale":"2023-10-08T07:10:45.206297+0000","last_undegraded":"2023-10-08T07:10:45.206297+0000","last_fullsized":"2023-10-08T07:10:45.206297+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:50:45.076883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239417+0000","last_change":"2023-10-08T07:10:44.229217+0000","last_active":"2023-10-08T07:10:45.239417+0000","last_peered":"2023-10-08T07:10:45.239417+0000","last_clean":"2023-10-08T07:10:45.239417+0000","last_became_active":"2023-10-08T07:10:44.228943+0000","last_became_peered":"2023-10-08T07:10:44.228943+0000","last_unstale":"2023-10-08T07:10:45.239417+0000","last_undegraded":"2023-10-08T07:10:45.239417+0000","last_fullsized":"2023-10-08T07:10:45.239417+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:22.640492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239545+0000","last_change":"2023-10-08T07:10:44.225988+0000","last_active":"2023-10-08T07:10:45.239545+0000","last_peered":"2023-10-08T07:10:45.239545+0000","last_clean":"2023-10-08T07:10:45.239545+0000","last_became_active":"2023-10-08T07:10:44.225718+0000","last_became_peered":"2023-10-08T07:10:44.225718+0000","last_unstale":"2023-10-08T07:10:45.239545+0000","last_undegraded":"2023-10-08T07:10:45.239545+0000","last_fullsized":"2023-10-08T07:10:45.239545+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:49:36.502279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.203936+0000","last_change":"2023-10-08T07:10:44.239219+0000","last_active":"2023-10-08T07:10:45.203936+0000","last_peered":"2023-10-08T07:10:45.203936+0000","last_clean":"2023-10-08T07:10:45.203936+0000","last_became_active":"2023-10-08T07:10:44.238599+0000","last_became_peered":"2023-10-08T07:10:44.238599+0000","last_unstale":"2023-10-08T07:10:45.203936+0000","last_undegraded":"2023-10-08T07:10:45.203936+0000","last_fullsized":"2023-10-08T07:10:45.203936+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:24:19.793863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.203543+0000","last_change":"2023-10-08T07:10:44.224589+0000","last_active":"2023-10-08T07:10:45.203543+0000","last_peered":"2023-10-08T07:10:45.203543+0000","last_clean":"2023-10-08T07:10:45.203543+0000","last_became_active":"2023-10-08T07:10:44.224259+0000","last_became_peered":"2023-10-08T07:10:44.224259+0000","last_unstale":"2023-10-08T07:10:45.203543+0000","last_undegraded":"2023-10-08T07:10:45.203543+0000","last_fullsized":"2023-10-08T07:10:45.203543+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:46.580370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198810+0000","last_change":"2023-10-08T07:10:44.222116+0000","last_active":"2023-10-08T07:10:45.198810+0000","last_peered":"2023-10-08T07:10:45.198810+0000","last_clean":"2023-10-08T07:10:45.198810+0000","last_became_active":"2023-10-08T07:10:44.221853+0000","last_became_peered":"2023-10-08T07:10:44.221853+0000","last_unstale":"2023-10-08T07:10:45.198810+0000","last_undegraded":"2023-10-08T07:10:45.198810+0000","last_fullsized":"2023-10-08T07:10:45.198810+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:40.761989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238402+0000","last_change":"2023-10-08T07:10:44.228359+0000","last_active":"2023-10-08T07:10:45.238402+0000","last_peered":"2023-10-08T07:10:45.238402+0000","last_clean":"2023-10-08T07:10:45.238402+0000","last_became_active":"2023-10-08T07:10:44.228049+0000","last_became_peered":"2023-10-08T07:10:44.228049+0000","last_unstale":"2023-10-08T07:10:45.238402+0000","last_undegraded":"2023-10-08T07:10:45.238402+0000","last_fullsized":"2023-10-08T07:10:45.238402+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:00.141299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239162+0000","last_change":"2023-10-08T07:10:44.228337+0000","last_active":"2023-10-08T07:10:45.239162+0000","last_peered":"2023-10-08T07:10:45.239162+0000","last_clean":"2023-10-08T07:10:45.239162+0000","last_became_active":"2023-10-08T07:10:44.227910+0000","last_became_peered":"2023-10-08T07:10:44.227910+0000","last_unstale":"2023-10-08T07:10:45.239162+0000","last_undegraded":"2023-10-08T07:10:45.239162+0000","last_fullsized":"2023-10-08T07:10:45.239162+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:17.731669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"215'4","reported_seq":16,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239314+0000","last_change":"2023-10-08T07:10:43.678287+0000","last_active":"2023-10-08T07:10:45.239314+0000","last_peered":"2023-10-08T07:10:45.239314+0000","last_clean":"2023-10-08T07:10:45.239314+0000","last_became_active":"2023-10-08T07:10:43.678046+0000","last_became_peered":"2023-10-08T07:10:43.678046+0000","last_unstale":"2023-10-08T07:10:45.239314+0000","last_undegraded":"2023-10-08T07:10:45.239314+0000","last_fullsized":"2023-10-08T07:10:45.239314+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:09.768687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.199308+0000","last_change":"2023-10-08T07:10:44.224347+0000","last_active":"2023-10-08T07:10:45.199308+0000","last_peered":"2023-10-08T07:10:45.199308+0000","last_clean":"2023-10-08T07:10:45.199308+0000","last_became_active":"2023-10-08T07:10:44.224082+0000","last_became_peered":"2023-10-08T07:10:44.224082+0000","last_unstale":"2023-10-08T07:10:45.199308+0000","last_undegraded":"2023-10-08T07:10:45.199308+0000","last_fullsized":"2023-10-08T07:10:45.199308+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:58.980437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239093+0000","last_change":"2023-10-08T07:10:43.683141+0000","last_active":"2023-10-08T07:10:45.239093+0000","last_peered":"2023-10-08T07:10:45.239093+0000","last_clean":"2023-10-08T07:10:45.239093+0000","last_became_active":"2023-10-08T07:10:43.682845+0000","last_became_peered":"2023-10-08T07:10:43.682845+0000","last_unstale":"2023-10-08T07:10:45.239093+0000","last_undegraded":"2023-10-08T07:10:45.239093+0000","last_fullsized":"2023-10-08T07:10:45.239093+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:37:56.842435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206606+0000","last_change":"2023-10-08T07:10:44.233237+0000","last_active":"2023-10-08T07:10:45.206606+0000","last_peered":"2023-10-08T07:10:45.206606+0000","last_clean":"2023-10-08T07:10:45.206606+0000","last_became_active":"2023-10-08T07:10:44.232959+0000","last_became_peered":"2023-10-08T07:10:44.232959+0000","last_unstale":"2023-10-08T07:10:45.206606+0000","last_undegraded":"2023-10-08T07:10:45.206606+0000","last_fullsized":"2023-10-08T07:10:45.206606+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:00.122036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.205248+0000","last_change":"2023-10-08T07:10:43.687400+0000","last_active":"2023-10-08T07:10:45.205248+0000","last_peered":"2023-10-08T07:10:45.205248+0000","last_clean":"2023-10-08T07:10:45.205248+0000","last_became_active":"2023-10-08T07:10:43.686675+0000","last_became_peered":"2023-10-08T07:10:43.686675+0000","last_unstale":"2023-10-08T07:10:45.205248+0000","last_undegraded":"2023-10-08T07:10:45.205248+0000","last_fullsized":"2023-10-08T07:10:45.205248+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:42:28.651968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206007+0000","last_change":"2023-10-08T07:10:44.224561+0000","last_active":"2023-10-08T07:10:45.206007+0000","last_peered":"2023-10-08T07:10:45.206007+0000","last_clean":"2023-10-08T07:10:45.206007+0000","last_became_active":"2023-10-08T07:10:44.224219+0000","last_became_peered":"2023-10-08T07:10:44.224219+0000","last_unstale":"2023-10-08T07:10:45.206007+0000","last_undegraded":"2023-10-08T07:10:45.206007+0000","last_fullsized":"2023-10-08T07:10:45.206007+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:19.841213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238252+0000","last_change":"2023-10-08T07:10:43.685955+0000","last_active":"2023-10-08T07:10:45.238252+0000","last_peered":"2023-10-08T07:10:45.238252+0000","last_clean":"2023-10-08T07:10:45.238252+0000","last_became_active":"2023-10-08T07:10:43.685678+0000","last_became_peered":"2023-10-08T07:10:43.685678+0000","last_unstale":"2023-10-08T07:10:45.238252+0000","last_undegraded":"2023-10-08T07:10:45.238252+0000","last_fullsized":"2023-10-08T07:10:45.238252+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:32.700444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237268+0000","last_change":"2023-10-08T07:10:44.229103+0000","last_active":"2023-10-08T07:10:45.237268+0000","last_peered":"2023-10-08T07:10:45.237268+0000","last_clean":"2023-10-08T07:10:45.237268+0000","last_became_active":"2023-10-08T07:10:44.228795+0000","last_became_peered":"2023-10-08T07:10:44.228795+0000","last_unstale":"2023-10-08T07:10:45.237268+0000","last_undegraded":"2023-10-08T07:10:45.237268+0000","last_fullsized":"2023-10-08T07:10:45.237268+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:23.596413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241576+0000","last_change":"2023-10-08T07:10:43.682622+0000","last_active":"2023-10-08T07:10:45.241576+0000","last_peered":"2023-10-08T07:10:45.241576+0000","last_clean":"2023-10-08T07:10:45.241576+0000","last_became_active":"2023-10-08T07:10:43.682356+0000","last_became_peered":"2023-10-08T07:10:43.682356+0000","last_unstale":"2023-10-08T07:10:45.241576+0000","last_undegraded":"2023-10-08T07:10:45.241576+0000","last_fullsized":"2023-10-08T07:10:45.241576+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:16.487425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240356+0000","last_change":"2023-10-08T07:10:44.220256+0000","last_active":"2023-10-08T07:10:45.240356+0000","last_peered":"2023-10-08T07:10:45.240356+0000","last_clean":"2023-10-08T07:10:45.240356+0000","last_became_active":"2023-10-08T07:10:44.219413+0000","last_became_peered":"2023-10-08T07:10:44.219413+0000","last_unstale":"2023-10-08T07:10:45.240356+0000","last_undegraded":"2023-10-08T07:10:45.240356+0000","last_fullsized":"2023-10-08T07:10:45.240356+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.355955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237790+0000","last_change":"2023-10-08T07:10:43.682977+0000","last_active":"2023-10-08T07:10:45.237790+0000","last_peered":"2023-10-08T07:10:45.237790+0000","last_clean":"2023-10-08T07:10:45.237790+0000","last_became_active":"2023-10-08T07:10:43.682711+0000","last_became_peered":"2023-10-08T07:10:43.682711+0000","last_unstale":"2023-10-08T07:10:45.237790+0000","last_undegraded":"2023-10-08T07:10:45.237790+0000","last_fullsized":"2023-10-08T07:10:45.237790+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:55:43.040815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240617+0000","last_change":"2023-10-08T07:10:44.234287+0000","last_active":"2023-10-08T07:10:45.240617+0000","last_peered":"2023-10-08T07:10:45.240617+0000","last_clean":"2023-10-08T07:10:45.240617+0000","last_became_active":"2023-10-08T07:10:44.234001+0000","last_became_peered":"2023-10-08T07:10:44.234001+0000","last_unstale":"2023-10-08T07:10:45.240617+0000","last_undegraded":"2023-10-08T07:10:45.240617+0000","last_fullsized":"2023-10-08T07:10:45.240617+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:43.902521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237817+0000","last_change":"2023-10-08T07:10:43.683020+0000","last_active":"2023-10-08T07:10:45.237817+0000","last_peered":"2023-10-08T07:10:45.237817+0000","last_clean":"2023-10-08T07:10:45.237817+0000","last_became_active":"2023-10-08T07:10:43.682737+0000","last_became_peered":"2023-10-08T07:10:43.682737+0000","last_unstale":"2023-10-08T07:10:45.237817+0000","last_undegraded":"2023-10-08T07:10:45.237817+0000","last_fullsized":"2023-10-08T07:10:45.237817+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:58:42.063439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239081+0000","last_change":"2023-10-08T07:10:44.231423+0000","last_active":"2023-10-08T07:10:45.239081+0000","last_peered":"2023-10-08T07:10:45.239081+0000","last_clean":"2023-10-08T07:10:45.239081+0000","last_became_active":"2023-10-08T07:10:44.231151+0000","last_became_peered":"2023-10-08T07:10:44.231151+0000","last_unstale":"2023-10-08T07:10:45.239081+0000","last_undegraded":"2023-10-08T07:10:45.239081+0000","last_fullsized":"2023-10-08T07:10:45.239081+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:06.269410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239484+0000","last_change":"2023-10-08T07:10:43.675546+0000","last_active":"2023-10-08T07:10:45.239484+0000","last_peered":"2023-10-08T07:10:45.239484+0000","last_clean":"2023-10-08T07:10:45.239484+0000","last_became_active":"2023-10-08T07:10:43.675280+0000","last_became_peered":"2023-10-08T07:10:43.675280+0000","last_unstale":"2023-10-08T07:10:45.239484+0000","last_undegraded":"2023-10-08T07:10:45.239484+0000","last_fullsized":"2023-10-08T07:10:45.239484+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:22.967066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240457+0000","last_change":"2023-10-08T07:10:44.218798+0000","last_active":"2023-10-08T07:10:45.240457+0000","last_peered":"2023-10-08T07:10:45.240457+0000","last_clean":"2023-10-08T07:10:45.240457+0000","last_became_active":"2023-10-08T07:10:44.218481+0000","last_became_peered":"2023-10-08T07:10:44.218481+0000","last_unstale":"2023-10-08T07:10:45.240457+0000","last_undegraded":"2023-10-08T07:10:45.240457+0000","last_fullsized":"2023-10-08T07:10:45.240457+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:14.551225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241619+0000","last_change":"2023-10-08T07:10:43.670774+0000","last_active":"2023-10-08T07:10:45.241619+0000","last_peered":"2023-10-08T07:10:45.241619+0000","last_clean":"2023-10-08T07:10:45.241619+0000","last_became_active":"2023-10-08T07:10:43.670506+0000","last_became_peered":"2023-10-08T07:10:43.670506+0000","last_unstale":"2023-10-08T07:10:45.241619+0000","last_undegraded":"2023-10-08T07:10:45.241619+0000","last_fullsized":"2023-10-08T07:10:45.241619+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:05.911601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238861+0000","last_change":"2023-10-08T07:10:44.240324+0000","last_active":"2023-10-08T07:10:45.238861+0000","last_peered":"2023-10-08T07:10:45.238861+0000","last_clean":"2023-10-08T07:10:45.238861+0000","last_became_active":"2023-10-08T07:10:44.239850+0000","last_became_peered":"2023-10-08T07:10:44.239850+0000","last_unstale":"2023-10-08T07:10:45.238861+0000","last_undegraded":"2023-10-08T07:10:45.238861+0000","last_fullsized":"2023-10-08T07:10:45.238861+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:18.399874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"215'3","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238032+0000","last_change":"2023-10-08T07:10:43.683401+0000","last_active":"2023-10-08T07:10:45.238032+0000","last_peered":"2023-10-08T07:10:45.238032+0000","last_clean":"2023-10-08T07:10:45.238032+0000","last_became_active":"2023-10-08T07:10:43.683027+0000","last_became_peered":"2023-10-08T07:10:43.683027+0000","last_unstale":"2023-10-08T07:10:45.238032+0000","last_undegraded":"2023-10-08T07:10:45.238032+0000","last_fullsized":"2023-10-08T07:10:45.238032+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:45:43.396156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241534+0000","last_change":"2023-10-08T07:10:44.234632+0000","last_active":"2023-10-08T07:10:45.241534+0000","last_peered":"2023-10-08T07:10:45.241534+0000","last_clean":"2023-10-08T07:10:45.241534+0000","last_became_active":"2023-10-08T07:10:44.234330+0000","last_became_peered":"2023-10-08T07:10:44.234330+0000","last_unstale":"2023-10-08T07:10:45.241534+0000","last_undegraded":"2023-10-08T07:10:45.241534+0000","last_fullsized":"2023-10-08T07:10:45.241534+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:59:48.167356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"215'6","reported_seq":18,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238318+0000","last_change":"2023-10-08T07:10:43.690014+0000","last_active":"2023-10-08T07:10:45.238318+0000","last_peered":"2023-10-08T07:10:45.238318+0000","last_clean":"2023-10-08T07:10:45.238318+0000","last_became_active":"2023-10-08T07:10:43.689722+0000","last_became_peered":"2023-10-08T07:10:43.689722+0000","last_unstale":"2023-10-08T07:10:45.238318+0000","last_undegraded":"2023-10-08T07:10:45.238318+0000","last_fullsized":"2023-10-08T07:10:45.238318+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:38.447879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198620+0000","last_change":"2023-10-08T07:10:44.224343+0000","last_active":"2023-10-08T07:10:45.198620+0000","last_peered":"2023-10-08T07:10:45.198620+0000","last_clean":"2023-10-08T07:10:45.198620+0000","last_became_active":"2023-10-08T07:10:44.224056+0000","last_became_peered":"2023-10-08T07:10:44.224056+0000","last_unstale":"2023-10-08T07:10:45.198620+0000","last_undegraded":"2023-10-08T07:10:45.198620+0000","last_fullsized":"2023-10-08T07:10:45.198620+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:56:51.457269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238753+0000","last_change":"2023-10-08T07:10:43.687611+0000","last_active":"2023-10-08T07:10:45.238753+0000","last_peered":"2023-10-08T07:10:45.238753+0000","last_clean":"2023-10-08T07:10:45.238753+0000","last_became_active":"2023-10-08T07:10:43.687016+0000","last_became_peered":"2023-10-08T07:10:43.687016+0000","last_unstale":"2023-10-08T07:10:45.238753+0000","last_undegraded":"2023-10-08T07:10:45.238753+0000","last_fullsized":"2023-10-08T07:10:45.238753+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:40.294292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238250+0000","last_change":"2023-10-08T07:10:44.225167+0000","last_active":"2023-10-08T07:10:45.238250+0000","last_peered":"2023-10-08T07:10:45.238250+0000","last_clean":"2023-10-08T07:10:45.238250+0000","last_became_active":"2023-10-08T07:10:44.224817+0000","last_became_peered":"2023-10-08T07:10:44.224817+0000","last_unstale":"2023-10-08T07:10:45.238250+0000","last_undegraded":"2023-10-08T07:10:45.238250+0000","last_fullsized":"2023-10-08T07:10:45.238250+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:01:14.855954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240821+0000","last_change":"2023-10-08T07:10:43.676142+0000","last_active":"2023-10-08T07:10:45.240821+0000","last_peered":"2023-10-08T07:10:45.240821+0000","last_clean":"2023-10-08T07:10:45.240821+0000","last_became_active":"2023-10-08T07:10:43.675868+0000","last_became_peered":"2023-10-08T07:10:43.675868+0000","last_unstale":"2023-10-08T07:10:45.240821+0000","last_undegraded":"2023-10-08T07:10:45.240821+0000","last_fullsized":"2023-10-08T07:10:45.240821+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:45:43.396156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237757+0000","last_change":"2023-10-08T07:10:44.234251+0000","last_active":"2023-10-08T07:10:45.237757+0000","last_peered":"2023-10-08T07:10:45.237757+0000","last_clean":"2023-10-08T07:10:45.237757+0000","last_became_active":"2023-10-08T07:10:44.233968+0000","last_became_peered":"2023-10-08T07:10:44.233968+0000","last_unstale":"2023-10-08T07:10:45.237757+0000","last_undegraded":"2023-10-08T07:10:45.237757+0000","last_fullsized":"2023-10-08T07:10:45.237757+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:38.036478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237207+0000","last_change":"2023-10-08T07:10:43.682849+0000","last_active":"2023-10-08T07:10:45.237207+0000","last_peered":"2023-10-08T07:10:45.237207+0000","last_clean":"2023-10-08T07:10:45.237207+0000","last_became_active":"2023-10-08T07:10:43.681842+0000","last_became_peered":"2023-10-08T07:10:43.681842+0000","last_unstale":"2023-10-08T07:10:45.237207+0000","last_undegraded":"2023-10-08T07:10:45.237207+0000","last_fullsized":"2023-10-08T07:10:45.237207+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:23.125441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237803+0000","last_change":"2023-10-08T07:10:44.229889+0000","last_active":"2023-10-08T07:10:45.237803+0000","last_peered":"2023-10-08T07:10:45.237803+0000","last_clean":"2023-10-08T07:10:45.237803+0000","last_became_active":"2023-10-08T07:10:44.229605+0000","last_became_peered":"2023-10-08T07:10:44.229605+0000","last_unstale":"2023-10-08T07:10:45.237803+0000","last_undegraded":"2023-10-08T07:10:45.237803+0000","last_fullsized":"2023-10-08T07:10:45.237803+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:04.514916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"215'8","reported_seq":20,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.203562+0000","last_change":"2023-10-08T07:10:43.687967+0000","last_active":"2023-10-08T07:10:45.203562+0000","last_peered":"2023-10-08T07:10:45.203562+0000","last_clean":"2023-10-08T07:10:45.203562+0000","last_became_active":"2023-10-08T07:10:43.687409+0000","last_became_peered":"2023-10-08T07:10:43.687409+0000","last_unstale":"2023-10-08T07:10:45.203562+0000","last_undegraded":"2023-10-08T07:10:45.203562+0000","last_fullsized":"2023-10-08T07:10:45.203562+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:34:13.969749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198898+0000","last_change":"2023-10-08T07:10:44.222398+0000","last_active":"2023-10-08T07:10:45.198898+0000","last_peered":"2023-10-08T07:10:45.198898+0000","last_clean":"2023-10-08T07:10:45.198898+0000","last_became_active":"2023-10-08T07:10:44.222088+0000","last_became_peered":"2023-10-08T07:10:44.222088+0000","last_unstale":"2023-10-08T07:10:45.198898+0000","last_undegraded":"2023-10-08T07:10:45.198898+0000","last_fullsized":"2023-10-08T07:10:45.198898+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:07.926407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206313+0000","last_change":"2023-10-08T07:10:43.678801+0000","last_active":"2023-10-08T07:10:45.206313+0000","last_peered":"2023-10-08T07:10:45.206313+0000","last_clean":"2023-10-08T07:10:45.206313+0000","last_became_active":"2023-10-08T07:10:43.678489+0000","last_became_peered":"2023-10-08T07:10:43.678489+0000","last_unstale":"2023-10-08T07:10:45.206313+0000","last_undegraded":"2023-10-08T07:10:45.206313+0000","last_fullsized":"2023-10-08T07:10:45.206313+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:14.746255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240224+0000","last_change":"2023-10-08T07:10:44.232130+0000","last_active":"2023-10-08T07:10:45.240224+0000","last_peered":"2023-10-08T07:10:45.240224+0000","last_clean":"2023-10-08T07:10:45.240224+0000","last_became_active":"2023-10-08T07:10:44.231832+0000","last_became_peered":"2023-10-08T07:10:44.231832+0000","last_unstale":"2023-10-08T07:10:45.240224+0000","last_undegraded":"2023-10-08T07:10:45.240224+0000","last_fullsized":"2023-10-08T07:10:45.240224+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:21.355790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238315+0000","last_change":"2023-10-08T07:10:43.688805+0000","last_active":"2023-10-08T07:10:45.238315+0000","last_peered":"2023-10-08T07:10:45.238315+0000","last_clean":"2023-10-08T07:10:45.238315+0000","last_became_active":"2023-10-08T07:10:43.688392+0000","last_became_peered":"2023-10-08T07:10:43.688392+0000","last_unstale":"2023-10-08T07:10:45.238315+0000","last_undegraded":"2023-10-08T07:10:45.238315+0000","last_fullsized":"2023-10-08T07:10:45.238315+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:12:14.290252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206578+0000","last_change":"2023-10-08T07:10:44.234357+0000","last_active":"2023-10-08T07:10:45.206578+0000","last_peered":"2023-10-08T07:10:45.206578+0000","last_clean":"2023-10-08T07:10:45.206578+0000","last_became_active":"2023-10-08T07:10:44.234051+0000","last_became_peered":"2023-10-08T07:10:44.234051+0000","last_unstale":"2023-10-08T07:10:45.206578+0000","last_undegraded":"2023-10-08T07:10:45.206578+0000","last_fullsized":"2023-10-08T07:10:45.206578+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:32:20.060577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"215'1","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.206512+0000","last_change":"2023-10-08T07:10:43.686002+0000","last_active":"2023-10-08T07:10:45.206512+0000","last_peered":"2023-10-08T07:10:45.206512+0000","last_clean":"2023-10-08T07:10:45.206512+0000","last_became_active":"2023-10-08T07:10:43.685708+0000","last_became_peered":"2023-10-08T07:10:43.685708+0000","last_unstale":"2023-10-08T07:10:45.206512+0000","last_undegraded":"2023-10-08T07:10:45.206512+0000","last_fullsized":"2023-10-08T07:10:45.206512+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:33:01.502206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241659+0000","last_change":"2023-10-08T07:10:44.217941+0000","last_active":"2023-10-08T07:10:45.241659+0000","last_peered":"2023-10-08T07:10:45.241659+0000","last_clean":"2023-10-08T07:10:45.241659+0000","last_became_active":"2023-10-08T07:10:44.217615+0000","last_became_peered":"2023-10-08T07:10:44.217615+0000","last_unstale":"2023-10-08T07:10:45.241659+0000","last_undegraded":"2023-10-08T07:10:45.241659+0000","last_fullsized":"2023-10-08T07:10:45.241659+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:49.740346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.205205+0000","last_change":"2023-10-08T07:10:43.683428+0000","last_active":"2023-10-08T07:10:45.205205+0000","last_peered":"2023-10-08T07:10:45.205205+0000","last_clean":"2023-10-08T07:10:45.205205+0000","last_became_active":"2023-10-08T07:10:43.679200+0000","last_became_peered":"2023-10-08T07:10:43.679200+0000","last_unstale":"2023-10-08T07:10:45.205205+0000","last_undegraded":"2023-10-08T07:10:45.205205+0000","last_fullsized":"2023-10-08T07:10:45.205205+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:35:46.189345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240180+0000","last_change":"2023-10-08T07:10:44.216903+0000","last_active":"2023-10-08T07:10:45.240180+0000","last_peered":"2023-10-08T07:10:45.240180+0000","last_clean":"2023-10-08T07:10:45.240180+0000","last_became_active":"2023-10-08T07:10:44.216541+0000","last_became_peered":"2023-10-08T07:10:44.216541+0000","last_unstale":"2023-10-08T07:10:45.240180+0000","last_undegraded":"2023-10-08T07:10:45.240180+0000","last_fullsized":"2023-10-08T07:10:45.240180+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:25:47.667238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238035+0000","last_change":"2023-10-08T07:10:43.688323+0000","last_active":"2023-10-08T07:10:45.238035+0000","last_peered":"2023-10-08T07:10:45.238035+0000","last_clean":"2023-10-08T07:10:45.238035+0000","last_became_active":"2023-10-08T07:10:43.687375+0000","last_became_peered":"2023-10-08T07:10:43.687375+0000","last_unstale":"2023-10-08T07:10:45.238035+0000","last_undegraded":"2023-10-08T07:10:45.238035+0000","last_fullsized":"2023-10-08T07:10:45.238035+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:00:08.992350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237054+0000","last_change":"2023-10-08T07:10:44.227627+0000","last_active":"2023-10-08T07:10:45.237054+0000","last_peered":"2023-10-08T07:10:45.237054+0000","last_clean":"2023-10-08T07:10:45.237054+0000","last_became_active":"2023-10-08T07:10:44.227344+0000","last_became_peered":"2023-10-08T07:10:44.227344+0000","last_unstale":"2023-10-08T07:10:45.237054+0000","last_undegraded":"2023-10-08T07:10:45.237054+0000","last_fullsized":"2023-10-08T07:10:45.237054+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:26:21.355955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237839+0000","last_change":"2023-10-08T07:10:43.678230+0000","last_active":"2023-10-08T07:10:45.237839+0000","last_peered":"2023-10-08T07:10:45.237839+0000","last_clean":"2023-10-08T07:10:45.237839+0000","last_became_active":"2023-10-08T07:10:43.677955+0000","last_became_peered":"2023-10-08T07:10:43.677955+0000","last_unstale":"2023-10-08T07:10:45.237839+0000","last_undegraded":"2023-10-08T07:10:45.237839+0000","last_fullsized":"2023-10-08T07:10:45.237839+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:25.207607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237626+0000","last_change":"2023-10-08T07:10:44.222001+0000","last_active":"2023-10-08T07:10:45.237626+0000","last_peered":"2023-10-08T07:10:45.237626+0000","last_clean":"2023-10-08T07:10:45.237626+0000","last_became_active":"2023-10-08T07:10:44.221728+0000","last_became_peered":"2023-10-08T07:10:44.221728+0000","last_unstale":"2023-10-08T07:10:45.237626+0000","last_undegraded":"2023-10-08T07:10:45.237626+0000","last_fullsized":"2023-10-08T07:10:45.237626+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:59:42.132321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198949+0000","last_change":"2023-10-08T07:10:43.676861+0000","last_active":"2023-10-08T07:10:45.198949+0000","last_peered":"2023-10-08T07:10:45.198949+0000","last_clean":"2023-10-08T07:10:45.198949+0000","last_became_active":"2023-10-08T07:10:43.676603+0000","last_became_peered":"2023-10-08T07:10:43.676603+0000","last_unstale":"2023-10-08T07:10:45.198949+0000","last_undegraded":"2023-10-08T07:10:45.198949+0000","last_fullsized":"2023-10-08T07:10:45.198949+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:57.059695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.241392+0000","last_change":"2023-10-08T07:10:44.218601+0000","last_active":"2023-10-08T07:10:45.241392+0000","last_peered":"2023-10-08T07:10:45.241392+0000","last_clean":"2023-10-08T07:10:45.241392+0000","last_became_active":"2023-10-08T07:10:44.218303+0000","last_became_peered":"2023-10-08T07:10:44.218303+0000","last_unstale":"2023-10-08T07:10:45.241392+0000","last_undegraded":"2023-10-08T07:10:45.241392+0000","last_fullsized":"2023-10-08T07:10:45.241392+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:10.677040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.204831+0000","last_change":"2023-10-08T07:10:43.678826+0000","last_active":"2023-10-08T07:10:45.204831+0000","last_peered":"2023-10-08T07:10:45.204831+0000","last_clean":"2023-10-08T07:10:45.204831+0000","last_became_active":"2023-10-08T07:10:43.678537+0000","last_became_peered":"2023-10-08T07:10:43.678537+0000","last_unstale":"2023-10-08T07:10:45.204831+0000","last_undegraded":"2023-10-08T07:10:45.204831+0000","last_fullsized":"2023-10-08T07:10:45.204831+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:38.447879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198948+0000","last_change":"2023-10-08T07:10:44.234170+0000","last_active":"2023-10-08T07:10:45.198948+0000","last_peered":"2023-10-08T07:10:45.198948+0000","last_clean":"2023-10-08T07:10:45.198948+0000","last_became_active":"2023-10-08T07:10:44.233836+0000","last_became_peered":"2023-10-08T07:10:44.233836+0000","last_unstale":"2023-10-08T07:10:45.198948+0000","last_undegraded":"2023-10-08T07:10:45.198948+0000","last_fullsized":"2023-10-08T07:10:45.198948+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:46.303194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"215'2","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239585+0000","last_change":"2023-10-08T07:10:43.680830+0000","last_active":"2023-10-08T07:10:45.239585+0000","last_peered":"2023-10-08T07:10:45.239585+0000","last_clean":"2023-10-08T07:10:45.239585+0000","last_became_active":"2023-10-08T07:10:43.679042+0000","last_became_peered":"2023-10-08T07:10:43.679042+0000","last_unstale":"2023-10-08T07:10:45.239585+0000","last_undegraded":"2023-10-08T07:10:45.239585+0000","last_fullsized":"2023-10-08T07:10:45.239585+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:29:59.531874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239963+0000","last_change":"2023-10-08T07:10:44.218588+0000","last_active":"2023-10-08T07:10:45.239963+0000","last_peered":"2023-10-08T07:10:45.239963+0000","last_clean":"2023-10-08T07:10:45.239963+0000","last_became_active":"2023-10-08T07:10:44.218267+0000","last_became_peered":"2023-10-08T07:10:44.218267+0000","last_unstale":"2023-10-08T07:10:45.239963+0000","last_undegraded":"2023-10-08T07:10:45.239963+0000","last_fullsized":"2023-10-08T07:10:45.239963+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:00.548052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.198799+0000","last_change":"2023-10-08T07:10:43.673755+0000","last_active":"2023-10-08T07:10:45.198799+0000","last_peered":"2023-10-08T07:10:45.198799+0000","last_clean":"2023-10-08T07:10:45.198799+0000","last_became_active":"2023-10-08T07:10:43.673236+0000","last_became_peered":"2023-10-08T07:10:43.673236+0000","last_unstale":"2023-10-08T07:10:45.198799+0000","last_undegraded":"2023-10-08T07:10:45.198799+0000","last_fullsized":"2023-10-08T07:10:45.198799+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:35.386951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239607+0000","last_change":"2023-10-08T07:10:44.230899+0000","last_active":"2023-10-08T07:10:45.239607+0000","last_peered":"2023-10-08T07:10:45.239607+0000","last_clean":"2023-10-08T07:10:45.239607+0000","last_became_active":"2023-10-08T07:10:44.230519+0000","last_became_peered":"2023-10-08T07:10:44.230519+0000","last_unstale":"2023-10-08T07:10:45.239607+0000","last_undegraded":"2023-10-08T07:10:45.239607+0000","last_fullsized":"2023-10-08T07:10:45.239607+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:49.577566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240635+0000","last_change":"2023-10-08T07:10:43.685043+0000","last_active":"2023-10-08T07:10:45.240635+0000","last_peered":"2023-10-08T07:10:45.240635+0000","last_clean":"2023-10-08T07:10:45.240635+0000","last_became_active":"2023-10-08T07:10:43.684594+0000","last_became_peered":"2023-10-08T07:10:43.684594+0000","last_unstale":"2023-10-08T07:10:45.240635+0000","last_undegraded":"2023-10-08T07:10:45.240635+0000","last_fullsized":"2023-10-08T07:10:45.240635+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:59:19.490322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.199298+0000","last_change":"2023-10-08T07:10:44.233672+0000","last_active":"2023-10-08T07:10:45.199298+0000","last_peered":"2023-10-08T07:10:45.199298+0000","last_clean":"2023-10-08T07:10:45.199298+0000","last_became_active":"2023-10-08T07:10:44.233185+0000","last_became_peered":"2023-10-08T07:10:44.233185+0000","last_unstale":"2023-10-08T07:10:45.199298+0000","last_undegraded":"2023-10-08T07:10:45.199298+0000","last_fullsized":"2023-10-08T07:10:45.199298+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:22.243718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.239699+0000","last_change":"2023-10-08T07:10:43.668769+0000","last_active":"2023-10-08T07:10:45.239699+0000","last_peered":"2023-10-08T07:10:45.239699+0000","last_clean":"2023-10-08T07:10:45.239699+0000","last_became_active":"2023-10-08T07:10:43.667704+0000","last_became_peered":"2023-10-08T07:10:43.667704+0000","last_unstale":"2023-10-08T07:10:45.239699+0000","last_undegraded":"2023-10-08T07:10:45.239699+0000","last_fullsized":"2023-10-08T07:10:45.239699+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:42.874794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.204869+0000","last_change":"2023-10-08T07:10:44.218609+0000","last_active":"2023-10-08T07:10:45.204869+0000","last_peered":"2023-10-08T07:10:45.204869+0000","last_clean":"2023-10-08T07:10:45.204869+0000","last_became_active":"2023-10-08T07:10:44.218129+0000","last_became_peered":"2023-10-08T07:10:44.218129+0000","last_unstale":"2023-10-08T07:10:45.204869+0000","last_undegraded":"2023-10-08T07:10:45.204869+0000","last_fullsized":"2023-10-08T07:10:45.204869+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:19:03.789076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238800+0000","last_change":"2023-10-08T07:10:43.684266+0000","last_active":"2023-10-08T07:10:45.238800+0000","last_peered":"2023-10-08T07:10:45.238800+0000","last_clean":"2023-10-08T07:10:45.238800+0000","last_became_active":"2023-10-08T07:10:43.683925+0000","last_became_peered":"2023-10-08T07:10:43.683925+0000","last_unstale":"2023-10-08T07:10:45.238800+0000","last_undegraded":"2023-10-08T07:10:45.238800+0000","last_fullsized":"2023-10-08T07:10:45.238800+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:14:29.123634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.237607+0000","last_change":"2023-10-08T07:10:44.237247+0000","last_active":"2023-10-08T07:10:45.237607+0000","last_peered":"2023-10-08T07:10:45.237607+0000","last_clean":"2023-10-08T07:10:45.237607+0000","last_became_active":"2023-10-08T07:10:44.236916+0000","last_became_peered":"2023-10-08T07:10:44.236916+0000","last_unstale":"2023-10-08T07:10:45.237607+0000","last_undegraded":"2023-10-08T07:10:45.237607+0000","last_fullsized":"2023-10-08T07:10:45.237607+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:58:12.789588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238147+0000","last_change":"2023-10-08T07:10:43.681762+0000","last_active":"2023-10-08T07:10:45.238147+0000","last_peered":"2023-10-08T07:10:45.238147+0000","last_clean":"2023-10-08T07:10:45.238147+0000","last_became_active":"2023-10-08T07:10:43.681403+0000","last_became_peered":"2023-10-08T07:10:43.681403+0000","last_unstale":"2023-10-08T07:10:45.238147+0000","last_undegraded":"2023-10-08T07:10:45.238147+0000","last_fullsized":"2023-10-08T07:10:45.238147+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:50:17.389772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.240975+0000","last_change":"2023-10-08T07:10:44.220396+0000","last_active":"2023-10-08T07:10:45.240975+0000","last_peered":"2023-10-08T07:10:45.240975+0000","last_clean":"2023-10-08T07:10:45.240975+0000","last_became_active":"2023-10-08T07:10:44.220038+0000","last_became_peered":"2023-10-08T07:10:44.220038+0000","last_unstale":"2023-10-08T07:10:45.240975+0000","last_undegraded":"2023-10-08T07:10:45.240975+0000","last_fullsized":"2023-10-08T07:10:45.240975+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:28:51.683157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.199051+0000","last_change":"2023-10-08T07:10:43.685876+0000","last_active":"2023-10-08T07:10:45.199051+0000","last_peered":"2023-10-08T07:10:45.199051+0000","last_clean":"2023-10-08T07:10:45.199051+0000","last_became_active":"2023-10-08T07:10:43.685578+0000","last_became_peered":"2023-10-08T07:10:43.685578+0000","last_unstale":"2023-10-08T07:10:45.199051+0000","last_undegraded":"2023-10-08T07:10:45.199051+0000","last_fullsized":"2023-10-08T07:10:45.199051+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:34.344958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.238576+0000","last_change":"2023-10-08T07:10:44.227526+0000","last_active":"2023-10-08T07:10:45.238576+0000","last_peered":"2023-10-08T07:10:45.238576+0000","last_clean":"2023-10-08T07:10:45.238576+0000","last_became_active":"2023-10-08T07:10:44.227224+0000","last_became_peered":"2023-10-08T07:10:44.227224+0000","last_unstale":"2023-10-08T07:10:45.238576+0000","last_undegraded":"2023-10-08T07:10:45.238576+0000","last_fullsized":"2023-10-08T07:10:45.238576+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:56.419001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.205983+0000","last_change":"2023-10-08T07:10:43.682574+0000","last_active":"2023-10-08T07:10:45.205983+0000","last_peered":"2023-10-08T07:10:45.205983+0000","last_clean":"2023-10-08T07:10:45.205983+0000","last_became_active":"2023-10-08T07:10:43.682194+0000","last_became_peered":"2023-10-08T07:10:43.682194+0000","last_unstale":"2023-10-08T07:10:45.205983+0000","last_undegraded":"2023-10-08T07:10:45.205983+0000","last_fullsized":"2023-10-08T07:10:45.205983+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:58:42.063439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.205826+0000","last_change":"2023-10-08T07:10:44.216912+0000","last_active":"2023-10-08T07:10:45.205826+0000","last_peered":"2023-10-08T07:10:45.205826+0000","last_clean":"2023-10-08T07:10:45.205826+0000","last_became_active":"2023-10-08T07:10:44.216268+0000","last_became_peered":"2023-10-08T07:10:44.216268+0000","last_unstale":"2023-10-08T07:10:45.205826+0000","last_undegraded":"2023-10-08T07:10:45.205826+0000","last_fullsized":"2023-10-08T07:10:45.205826+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:43.635807+0000","last_clean_scrub_stamp":"2023-10-08T07:10:43.635807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:17.895682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-10-08T07:10:45.236990+0000","last_change":"2023-10-08T07:10:43.685985+0000","last_active":"2023-10-08T07:10:45.236990+0000","last_peered":"2023-10-08T07:10:45.236990+0000","last_clean":"2023-10-08T07:10:45.236990+0000","last_became_active":"2023-10-08T07:10:43.685720+0000","last_became_peered":"2023-10-08T07:10:43.685720+0000","last_unstale":"2023-10-08T07:10:45.236990+0000","last_undegraded":"2023-10-08T07:10:45.236990+0000","last_fullsized":"2023-10-08T07:10:45.236990+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:10:42.619630+0000","last_clean_scrub_stamp":"2023-10-08T07:10:42.619630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:25:46.651061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"203'2784","reported_seq":3628,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-10-08T07:10:44.214136+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:10:44.214136+0000","last_peered":"2023-10-08T07:10:44.214136+0000","last_clean":"2023-10-08T07:10:44.214136+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:10:44.214136+0000","last_undegraded":"2023-10-08T07:10:44.214136+0000","last_fullsized":"2023-10-08T07:10:44.214136+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2784,"log_dups_size":0,"ondisk_log_size":2784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2013,"num_read_kb":4206,"num_write":549,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2013,"num_read_kb":4206,"num_write":549,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2784,"ondisk_log_size":2784,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575976,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60772,"kb_used_data":3224,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311068,"statfs":{"total":96636764160,"available":96574533632,"internally_reserved":0,"allocated":3301376,"data_stored":1991986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.218},"min":{"1min":0.78400000000000003,"5min":0.70799999999999996,"15min":0.64400000000000002},"max":{"1min":1.7170000000000001,"5min":1.804,"15min":2.008},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.2170000000000001,"15min":1.2210000000000001},"min":{"1min":0.92800000000000005,"5min":0.82199999999999995,"15min":0.78100000000000003},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.7989999999999999},"last":1.03}]},{"osd":1,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2609999999999999,"15min":1.266},"min":{"1min":1.0009999999999999,"5min":0.78600000000000003,"15min":0.67300000000000004},"max":{"1min":1.601,"5min":1.742,"15min":1.998},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.325},"min":{"1min":0.97599999999999998,"5min":0.873,"15min":0.84199999999999997},"max":{"1min":1.7490000000000001,"5min":1.9159999999999999,"15min":2.1019999999999999},"last":1.5940000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3169999999999999,"15min":1.2969999999999999},"min":{"1min":0.90400000000000003,"5min":0.88200000000000001,"15min":0.60799999999999998},"max":{"1min":1.5569999999999999,"5min":1.7789999999999999,"15min":2.1640000000000001},"last":1.464},{"interface":"front","average":{"1min":1.397,"5min":1.335,"15min":1.345},"min":{"1min":0.96599999999999997,"5min":0.91800000000000004,"15min":0.86799999999999999},"max":{"1min":1.76,"5min":1.76,"15min":2.3170000000000002},"last":1.4350000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3460000000000001,"15min":1.3400000000000001},"min":{"1min":0.86899999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8160000000000001,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.409,"15min":1.427},"min":{"1min":1.0469999999999999,"5min":1.022,"15min":0.98499999999999999},"max":{"1min":1.8480000000000001,"5min":1.8700000000000001,"15min":1.931},"last":0.98299999999999998}]},{"osd":4,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3120000000000001,"15min":1.3069999999999999},"min":{"1min":0.86699999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.8,"5min":1.8,"15min":2.0249999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3939999999999999,"15min":1.4059999999999999},"min":{"1min":0.94799999999999995,"5min":0.93600000000000005,"15min":0.85199999999999998},"max":{"1min":1.8140000000000001,"5min":2.0379999999999998,"15min":2.1600000000000001},"last":1.5329999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.431,"15min":1.456},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.9630000000000001,"5min":2.1139999999999999,"15min":2.7360000000000002},"last":1.625},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4870000000000001,"15min":1.5109999999999999},"min":{"1min":1.006,"5min":0.96899999999999997,"15min":0.96899999999999997},"max":{"1min":1.964,"5min":1.972,"15min":2.569},"last":1.3149999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.5940000000000001,"15min":1.589},"min":{"1min":1.276,"5min":1.155,"15min":1.1299999999999999},"max":{"1min":2.0289999999999999,"5min":2.157,"15min":2.2189999999999999},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.5269999999999999,"15min":1.534},"min":{"1min":1.1910000000000001,"5min":1.022,"15min":0.90100000000000002},"max":{"1min":2.21,"5min":2.21,"15min":2.262},"last":1.0960000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575975,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42156,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329684,"statfs":{"total":96636764160,"available":96593596416,"internally_reserved":0,"allocated":2711552,"data_stored":1401688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.282,"15min":1.3220000000000001},"min":{"1min":0.81100000000000005,"5min":0.68400000000000005,"15min":0.52200000000000002},"max":{"1min":1.794,"5min":1.905,"15min":1.962},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.371,"15min":1.3740000000000001},"min":{"1min":1.1479999999999999,"5min":1.042,"15min":0.40799999999999997},"max":{"1min":1.6100000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.2390000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2529999999999999,"15min":1.2410000000000001},"min":{"1min":0.89100000000000001,"5min":0.86299999999999999,"15min":0.65300000000000002},"max":{"1min":1.722,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.109},{"interface":"front","average":{"1min":1.153,"5min":1.216,"15min":1.238},"min":{"1min":0.86899999999999999,"5min":0.76300000000000001,"15min":0.63100000000000001},"max":{"1min":1.5549999999999999,"5min":1.97,"15min":1.97},"last":1.2969999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.21,"15min":1.256},"min":{"1min":0.84899999999999998,"5min":0.78800000000000003,"15min":0.67400000000000004},"max":{"1min":1.4450000000000001,"5min":2.339,"15min":2.339},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3200000000000001,"15min":1.3160000000000001},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.59499999999999997},"max":{"1min":1.6970000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.3580000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4039999999999999,"15min":1.409},"min":{"1min":0.97699999999999998,"5min":0.89900000000000002,"15min":0.69499999999999995},"max":{"1min":1.7749999999999999,"5min":1.95,"15min":1.95},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.5329999999999999,"15min":1.5109999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.71099999999999997},"max":{"1min":1.8380000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.90000000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.4159999999999999,"15min":1.3839999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.63900000000000001},"max":{"1min":1.78,"5min":2.0859999999999999,"15min":2.0859999999999999},"last":1.413},{"interface":"front","average":{"1min":1.405,"5min":1.4850000000000001,"15min":1.4419999999999999},"min":{"1min":1.091,"5min":0.57099999999999995,"15min":0.55300000000000005},"max":{"1min":1.75,"5min":2.484,"15min":2.484},"last":1.8640000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4750000000000001,"15min":1.421},"min":{"1min":1.0489999999999999,"5min":0.89100000000000001,"15min":0.42899999999999999},"max":{"1min":1.6950000000000001,"5min":2.403,"15min":2.403},"last":1.581},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5629999999999999,"15min":1.548},"min":{"1min":1.105,"5min":1.071,"15min":0.73799999999999999},"max":{"1min":1.875,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":1.7689999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:10:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.5980000000000001,"15min":1.5880000000000001},"min":{"1min":1.012,"5min":1.012,"15min":0.62},"max":{"1min":1.798,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.641},{"interface":"front","average":{"1min":1.54,"5min":1.585,"15min":1.5509999999999999},"min":{"1min":1.228,"5min":1.024,"15min":0.79900000000000004},"max":{"1min":1.8220000000000001,"5min":2.665,"15min":2.665},"last":1.7090000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60216,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311624,"statfs":{"total":96636764160,"available":96575102976,"internally_reserved":0,"allocated":2723840,"data_stored":1403252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.1970000000000001,"15min":1.2250000000000001},"min":{"1min":0.85899999999999999,"5min":0.75700000000000001,"15min":0.74099999999999999},"max":{"1min":1.589,"5min":1.8320000000000001,"15min":1.841},"last":1.387},{"interface":"front","average":{"1min":1.204,"5min":1.234,"15min":1.236},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6699999999999999,"5min":1.7609999999999999,"15min":1.9610000000000001},"last":1.0469999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.22,"15min":1.2490000000000001},"min":{"1min":0.92900000000000005,"5min":0.82199999999999995,"15min":0.60799999999999998},"max":{"1min":1.524,"5min":1.712,"15min":1.9550000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.429,"5min":1.3220000000000001,"15min":1.3200000000000001},"min":{"1min":0.999,"5min":0.83999999999999997,"15min":0.56899999999999995},"max":{"1min":1.825,"5min":1.825,"15min":1.978},"last":0.73399999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.296,"15min":1.3049999999999999},"min":{"1min":0.82599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6020000000000001,"5min":2.056,"15min":2.0870000000000002},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.2949999999999999,"15min":1.306},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.6479999999999999,"5min":1.7090000000000001,"15min":2.0870000000000002},"last":1.5009999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.387,"15min":1.395},"min":{"1min":1.2030000000000001,"5min":0.91900000000000004,"15min":0.69399999999999995},"max":{"1min":1.794,"5min":1.794,"15min":2.1499999999999999},"last":1.48},{"interface":"front","average":{"1min":1.371,"5min":1.3979999999999999,"15min":1.446},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.681,"5min":2.173,"15min":2.173},"last":1.631}]},{"osd":4,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.363,"15min":1.337},"min":{"1min":0.92600000000000005,"5min":0.85299999999999998,"15min":0.82599999999999996},"max":{"1min":1.6799999999999999,"5min":1.8220000000000001,"15min":2.0150000000000001},"last":1.113},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.427},"min":{"1min":0.88,"5min":0.748,"15min":0.54000000000000004},"max":{"1min":1.8759999999999999,"5min":2.117,"15min":2.2240000000000002},"last":1.5409999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.415,"15min":1.4430000000000001},"min":{"1min":0.66500000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.736,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.454,"15min":1.4690000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.7490000000000001,"5min":1.905,"15min":2.145},"last":1.2370000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:10:16 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.546,"15min":1.5640000000000001},"min":{"1min":1.113,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.8779999999999999,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":1.595},{"interface":"front","average":{"1min":1.474,"5min":1.577,"15min":1.6080000000000001},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.53500000000000003},"max":{"1min":1.8420000000000001,"5min":2.282,"15min":2.3199999999999998},"last":1.6930000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47736,"kb_used_data":2660,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324104,"statfs":{"total":96636764160,"available":96587882496,"internally_reserved":0,"allocated":2723840,"data_stored":1403264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3620000000000001,"15min":1.361},"min":{"1min":1.0409999999999999,"5min":0.85599999999999998,"15min":0.76800000000000002},"max":{"1min":1.698,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.4510000000000001,"15min":1.419},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.749},"max":{"1min":1.778,"5min":1.925,"15min":2.903},"last":1.0660000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.339,"15min":1.3109999999999999},"min":{"1min":0.89600000000000002,"5min":0.88300000000000001,"15min":0.76200000000000001},"max":{"1min":1.806,"5min":1.976,"15min":2.1600000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.2150000000000001,"15min":1.2450000000000001},"min":{"1min":0.93899999999999995,"5min":0.82599999999999996,"15min":0.65200000000000002},"max":{"1min":1.756,"5min":1.7929999999999999,"15min":1.875},"last":1.274}]},{"osd":2,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.256,"15min":1.2649999999999999},"min":{"1min":0.90600000000000003,"5min":0.90000000000000002,"15min":0.59399999999999997},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":2.2589999999999999},"last":1.329},{"interface":"front","average":{"1min":1.343,"5min":1.403,"15min":1.3680000000000001},"min":{"1min":1.0409999999999999,"5min":0.89900000000000002,"15min":0.81699999999999995},"max":{"1min":1.7450000000000001,"5min":1.853,"15min":2.3149999999999999},"last":1.454}]},{"osd":3,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.3340000000000001,"15min":1.319},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.872},"max":{"1min":1.6850000000000001,"5min":1.7190000000000001,"15min":2.028},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.4279999999999999,"15min":1.425},"min":{"1min":1.341,"5min":0.83799999999999997,"15min":0.73399999999999999},"max":{"1min":1.677,"5min":1.8979999999999999,"15min":2.093},"last":1.4199999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.3600000000000001,"15min":1.371},"min":{"1min":1.0409999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.7969999999999999,"5min":2.0289999999999999,"15min":2.0830000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.3959999999999999,"15min":1.3839999999999999},"min":{"1min":0.97999999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.853,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.8700000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.4570000000000001,"15min":1.45},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.7430000000000001,"5min":1.992,"15min":2.6829999999999998},"last":1.54},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5620000000000001,"15min":1.5429999999999999},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.70499999999999996},"max":{"1min":1.794,"5min":2.1429999999999998,"15min":2.7429999999999999},"last":1.913}]},{"osd":7,"last update":"Sun Oct 8 07:10:15 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.524,"15min":1.5049999999999999},"min":{"1min":1.3500000000000001,"5min":0.95599999999999996,"15min":0.95499999999999996},"max":{"1min":1.8879999999999999,"5min":2.1230000000000002,"15min":2.1240000000000001},"last":1.9470000000000001},{"interface":"front","average":{"1min":1.722,"5min":1.6559999999999999,"15min":1.629},"min":{"1min":1.278,"5min":1.194,"15min":1.0309999999999999},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.52},"last":1.639}]}]},{"osd":3,"up_from":13,"seq":55834575977,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51880,"kb_used_data":2652,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319960,"statfs":{"total":96636764160,"available":96583639040,"internally_reserved":0,"allocated":2715648,"data_stored":1401778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.012,"15min":0.995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.7599999999999998,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.325,"5min":1.2090000000000001,"15min":1.1950000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.5659999999999998,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.1459999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.0189999999999999,"15min":1.0429999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3169999999999999,"5min":1.79,"15min":2.4390000000000001},"last":0.996},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2989999999999999,"15min":1.3009999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.587,"5min":1.8260000000000001,"15min":2.4889999999999999},"last":1.27}]},{"osd":2,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.161,"15min":1.1599999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.728,"5min":1.728,"15min":2.2250000000000001},"last":1.204},{"interface":"front","average":{"1min":1.175,"5min":1.2070000000000001,"15min":1.2110000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.63},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":2.2930000000000001},"last":1.0900000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2889999999999999,"15min":1.292},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.67500000000000004},"max":{"1min":1.857,"5min":1.857,"15min":2.5379999999999998},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4650000000000001,"15min":1.4730000000000001},"min":{"1min":1.032,"5min":1.032,"15min":0.78100000000000003},"max":{"1min":1.8109999999999999,"5min":1.867,"15min":2.8340000000000001},"last":1.502}]},{"osd":5,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3540000000000001,"15min":1.391},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5369999999999999,"5min":1.7529999999999999,"15min":2.9740000000000002},"last":1.397},{"interface":"front","average":{"1min":1.407,"5min":1.464,"15min":1.4350000000000001},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":2.012,"5min":2.1869999999999998,"15min":2.7330000000000001},"last":1.5429999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.4330000000000001,"15min":1.4179999999999999},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.976,"5min":1.976,"15min":2.653},"last":1.458},{"interface":"front","average":{"1min":1.425,"5min":1.524,"15min":1.542},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.792,"5min":2.1579999999999999,"15min":3.0499999999999998},"last":1.6379999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:10:34 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.593,"15min":1.5760000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.353},{"interface":"front","average":{"1min":1.456,"5min":1.5289999999999999,"15min":1.5389999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.9410000000000001,"5min":2.0670000000000002,"15min":2.7749999999999999},"last":1.3140000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575975,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51880,"kb_used_data":2648,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319960,"statfs":{"total":96636764160,"available":96583639040,"internally_reserved":0,"allocated":2711552,"data_stored":1402200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.99299999999999999,"15min":0.97099999999999997},"min":{"1min":0.66000000000000003,"5min":0.63700000000000001,"15min":0.41399999999999998},"max":{"1min":1.3180000000000001,"5min":1.74,"15min":1.948},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.998,"5min":1.1020000000000001,"15min":1.0780000000000001},"min":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.38400000000000001},"max":{"1min":1.3320000000000001,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.58899999999999997}]},{"osd":1,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.999,"15min":1.0229999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.495},"max":{"1min":1.522,"5min":2.044,"15min":2.044},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.256,"5min":1.226,"15min":1.1850000000000001},"min":{"1min":0.97499999999999998,"5min":0.83099999999999996,"15min":0.52500000000000002},"max":{"1min":1.579,"5min":1.724,"15min":2.1309999999999998},"last":1.5249999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.194,"15min":1.1930000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.56000000000000005},"max":{"1min":1.649,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.296,"15min":1.2769999999999999},"min":{"1min":0.98699999999999999,"5min":0.92000000000000004,"15min":0.57399999999999995},"max":{"1min":1.6299999999999999,"5min":1.8,"15min":2.0670000000000002},"last":1.3340000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4510000000000001,"15min":1.4430000000000001},"min":{"1min":1.0840000000000001,"5min":1.0580000000000001,"15min":0.66800000000000004},"max":{"1min":1.631,"5min":1.9570000000000001,"15min":2.1789999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.3300000000000001,"15min":1.294},"min":{"1min":1.036,"5min":0.96599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6830000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.95799999999999996}]},{"osd":5,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.401,"15min":1.4159999999999999},"min":{"1min":1.085,"5min":1.022,"15min":0.61399999999999999},"max":{"1min":1.7270000000000001,"5min":2.1429999999999998,"15min":2.3420000000000001},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.524,"15min":1.53},"min":{"1min":1.1930000000000001,"5min":1.093,"15min":0.70699999999999996},"max":{"1min":1.9630000000000001,"5min":2.081,"15min":2.9390000000000001},"last":1.6140000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.403,"15min":1.4019999999999999},"min":{"1min":0.96299999999999997,"5min":0.92600000000000005,"15min":0.85399999999999998},"max":{"1min":1.6919999999999999,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.5269999999999999,"5min":1.5349999999999999,"15min":1.554},"min":{"1min":1.3129999999999999,"5min":1.1499999999999999,"15min":0.76700000000000002},"max":{"1min":1.7529999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.387}]},{"osd":7,"last update":"Sun Oct 8 07:09:58 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4970000000000001,"15min":1.4810000000000001},"min":{"1min":1.1379999999999999,"5min":1.0680000000000001,"15min":0.64200000000000002},"max":{"1min":1.8859999999999999,"5min":2.3069999999999999,"15min":2.4689999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.5649999999999999,"15min":1.605},"min":{"1min":1.252,"5min":1.23,"15min":0.72799999999999998},"max":{"1min":1.847,"5min":2.2829999999999999,"15min":2.3879999999999999},"last":1.5569999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51872,"kb_used_data":2644,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319968,"statfs":{"total":96636764160,"available":96583647232,"internally_reserved":0,"allocated":2707456,"data_stored":1401618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0129999999999999,"15min":1.0169999999999999},"min":{"1min":0.79900000000000004,"5min":0.60699999999999998,"15min":0.55400000000000005},"max":{"1min":1.5660000000000001,"5min":1.657,"15min":1.7769999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.98299999999999998,"5min":0.91100000000000003,"15min":0.46400000000000002},"max":{"1min":1.6950000000000001,"5min":1.7609999999999999,"15min":1.79},"last":0.63800000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.002,"15min":0.98799999999999999},"min":{"1min":0.83499999999999996,"5min":0.70499999999999996,"15min":0.47999999999999998},"max":{"1min":1.476,"5min":1.542,"15min":2.1890000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.208,"15min":1.1839999999999999},"min":{"1min":0.97899999999999998,"5min":0.86499999999999999,"15min":0.54600000000000004},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.7929999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2929999999999999,"15min":1.2729999999999999},"min":{"1min":0.82399999999999995,"5min":0.78300000000000003,"15min":0.59399999999999997},"max":{"1min":1.609,"5min":1.651,"15min":1.9139999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.204,"5min":1.143,"15min":1.1719999999999999},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.62},"max":{"1min":1.883,"5min":1.883,"15min":2.7429999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3819999999999999,"15min":1.3919999999999999},"min":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.86699999999999999},"max":{"1min":1.6080000000000001,"5min":2.0129999999999999,"15min":2.8460000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.3460000000000001,"15min":1.3160000000000001},"min":{"1min":1.0960000000000001,"5min":0.99099999999999999,"15min":0.66500000000000004},"max":{"1min":1.613,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.86099999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3899999999999999,"15min":1.387},"min":{"1min":1.179,"5min":1.01,"15min":0.76700000000000002},"max":{"1min":1.827,"5min":1.978,"15min":2.3500000000000001},"last":1.103},{"interface":"front","average":{"1min":1.635,"5min":1.589,"15min":1.5529999999999999},"min":{"1min":1.3759999999999999,"5min":1.1499999999999999,"15min":0.72899999999999998},"max":{"1min":2.0659999999999998,"5min":2.2610000000000001,"15min":2.528},"last":0.84599999999999997}]},{"osd":6,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.536,"15min":1.5409999999999999},"min":{"1min":1.4470000000000001,"5min":1.123,"15min":0.82699999999999996},"max":{"1min":1.8620000000000001,"5min":1.9730000000000001,"15min":2.9079999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.585,"5min":1.5389999999999999,"15min":1.5069999999999999},"min":{"1min":1.234,"5min":1.1080000000000001,"15min":0.71699999999999997},"max":{"1min":1.966,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.1419999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:09:44 2023","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.619,"15min":1.5740000000000001},"min":{"1min":1.236,"5min":1.071,"15min":0.69399999999999995},"max":{"1min":2.1499999999999999,"5min":2.2610000000000001,"15min":2.46},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6000000000000001,"15min":1.6020000000000001},"min":{"1min":1.2250000000000001,"5min":1.2050000000000001,"15min":0.81200000000000006},"max":{"1min":1.9299999999999999,"5min":2.0139999999999998,"15min":3.2160000000000002},"last":1.1240000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575976,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56636,"kb_used_data":3240,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315204,"statfs":{"total":96636764160,"available":96578768896,"internally_reserved":0,"allocated":3317760,"data_stored":1992714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":54655433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.95099999999999996,"15min":0.97799999999999998},"min":{"1min":0.68300000000000005,"5min":0.60399999999999998,"15min":0.46000000000000002},"max":{"1min":1.2709999999999999,"5min":1.5369999999999999,"15min":1.571},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1259999999999999,"15min":1.113},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.51500000000000001},"max":{"1min":1.466,"5min":1.6619999999999999,"15min":1.726},"last":1.073}]},{"osd":2,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.1559999999999999,"15min":1.1439999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.497},"max":{"1min":1.4299999999999999,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.234},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.085,"15min":1.087},"min":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.42299999999999999},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":1.1259999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2669999999999999,"15min":1.2649999999999999},"min":{"1min":0.94499999999999995,"5min":0.91800000000000004,"15min":0.52800000000000002},"max":{"1min":1.6299999999999999,"5min":1.77,"15min":1.8420000000000001},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.1659999999999999,"15min":1.1779999999999999},"min":{"1min":0.872,"5min":0.80000000000000004,"15min":0.47999999999999998},"max":{"1min":1.752,"5min":1.752,"15min":2.0979999999999999},"last":1.367}]},{"osd":4,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.329,"15min":1.3600000000000001},"min":{"1min":1.044,"5min":1.0069999999999999,"15min":0.79100000000000004},"max":{"1min":1.589,"5min":1.73,"15min":2.1040000000000001},"last":1.609},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4379999999999999,"15min":1.4390000000000001},"min":{"1min":1.079,"5min":1.079,"15min":0.96099999999999997},"max":{"1min":1.956,"5min":1.956,"15min":2.181},"last":1.1930000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.4379999999999999,"15min":1.4199999999999999},"min":{"1min":1.107,"5min":1.087,"15min":0.873},"max":{"1min":1.919,"5min":1.919,"15min":2.1309999999999998},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.466,"15min":1.4870000000000001},"min":{"1min":1.2410000000000001,"5min":0.998,"15min":0.81000000000000005},"max":{"1min":1.7749999999999999,"5min":1.9490000000000001,"15min":2.4900000000000002},"last":1.5820000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4730000000000001,"15min":1.49},"min":{"1min":1.1930000000000001,"5min":1.036,"15min":1.036},"max":{"1min":2.0369999999999999,"5min":2.109,"15min":2.2149999999999999},"last":1.282},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.514,"15min":1.514},"min":{"1min":1.1719999999999999,"5min":1.024,"15min":0.89500000000000002},"max":{"1min":1.849,"5min":2.0099999999999998,"15min":2.355},"last":1.4490000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:10:02 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.552,"15min":1.552},"min":{"1min":1.2609999999999999,"5min":1.099,"15min":0.82099999999999995},"max":{"1min":2.0859999999999999,"5min":2.1200000000000001,"15min":2.3100000000000001},"last":1.321},{"interface":"front","average":{"1min":1.605,"5min":1.544,"15min":1.556},"min":{"1min":1.3160000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"max":{"1min":2.0950000000000002,"5min":2.0990000000000002,"15min":2.1349999999999998},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:10:49.596 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T07:10:49.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T07:10:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.854+0000 7f061f4d7640 1 Processor -- start 2023-10-08T07:10:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.854+0000 7f061f4d7640 1 -- start start 2023-10-08T07:10:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.856+0000 7f061f4d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0618159d50 0x7f061815e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.856+0000 7f061f4d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180b16e0 0x7f06180cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.856+0000 7f061f4d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06180cba40 0x7f06180c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.856+0000 7f061f4d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06180cbeb0 con 0x7f06180cba40 2023-10-08T07:10:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.856+0000 7f061f4d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06180b2890 con 0x7f0618159d50 2023-10-08T07:10:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.856+0000 7f061f4d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061815d8b0 con 0x7f06180b16e0 2023-10-08T07:10:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.856+0000 7f061da74640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06180cba40 0x7f06180c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.857+0000 7f061da74640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06180cba40 0x7f06180c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60778/0 (socket says 172.21.15.70:60778) 2023-10-08T07:10:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.857+0000 7f061da74640 1 -- 172.21.15.70:0/3362878280 learned_addr learned my addr 172.21.15.70:0/3362878280 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.857+0000 7f061d273640 1 --2- 172.21.15.70:0/3362878280 >> v2:172.21.15.181:3300/0 conn(0x7f0618159d50 0x7f061815e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.857+0000 7f061ca72640 1 --2- 172.21.15.70:0/3362878280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180b16e0 0x7f06180cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.857+0000 7f061da74640 1 -- 172.21.15.70:0/3362878280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180b16e0 msgr2=0x7f06180cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.857+0000 7f061da74640 1 --2- 172.21.15.70:0/3362878280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180b16e0 0x7f06180cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.857+0000 7f061da74640 1 -- 172.21.15.70:0/3362878280 >> v2:172.21.15.181:3300/0 conn(0x7f0618159d50 msgr2=0x7f061815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.857+0000 7f061da74640 1 --2- 172.21.15.70:0/3362878280 >> v2:172.21.15.181:3300/0 conn(0x7f0618159d50 0x7f061815e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.857+0000 7f061da74640 1 -- 172.21.15.70:0/3362878280 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06180b1ff0 con 0x7f06180cba40 2023-10-08T07:10:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.857+0000 7f061d273640 1 --2- 172.21.15.70:0/3362878280 >> v2:172.21.15.181:3300/0 conn(0x7f0618159d50 0x7f061815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:10:49.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.858+0000 7f061da74640 1 --2- 172.21.15.70:0/3362878280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180cba40 0x7f06180c9310 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f06180ce630 tx=0x7f061002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac64ba696085e55 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:49.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.859+0000 7f060ffff640 1 -- 172.21.15.70:0/3362878280 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061002ca70 con 0x7f06180cba40 2023-10-08T07:10:49.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.859+0000 7f060ffff640 1 -- 172.21.15.70:0/3362878280 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061002cbd0 con 0x7f06180cba40 2023-10-08T07:10:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.859+0000 7f060ffff640 1 -- 172.21.15.70:0/3362878280 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06100343f0 con 0x7f06180cba40 2023-10-08T07:10:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.860+0000 7f061f4d7640 1 -- 172.21.15.70:0/3362878280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180cba40 msgr2=0x7f06180c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.860+0000 7f061f4d7640 1 --2- 172.21.15.70:0/3362878280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180cba40 0x7f06180c9310 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f06180ce630 tx=0x7f061002bed0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.860+0000 7f061f4d7640 1 -- 172.21.15.70:0/3362878280 shutdown_connections 2023-10-08T07:10:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.860+0000 7f061f4d7640 1 --2- 172.21.15.70:0/3362878280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180cba40 0x7f06180c9310 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.860+0000 7f061f4d7640 1 --2- 172.21.15.70:0/3362878280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180b16e0 0x7f06180cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.860+0000 7f061f4d7640 1 --2- 172.21.15.70:0/3362878280 >> v2:172.21.15.181:3300/0 conn(0x7f0618159d50 0x7f061815e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.860+0000 7f061f4d7640 1 -- 172.21.15.70:0/3362878280 >> 172.21.15.70:0/3362878280 conn(0x7f061806a140 msgr2=0x7f061806a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.860+0000 7f061f4d7640 1 -- 172.21.15.70:0/3362878280 shutdown_connections 2023-10-08T07:10:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.860+0000 7f061f4d7640 1 -- 172.21.15.70:0/3362878280 wait complete. 2023-10-08T07:10:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.862+0000 7f061f4d7640 1 Processor -- start 2023-10-08T07:10:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.862+0000 7f061f4d7640 1 -- start start 2023-10-08T07:10:49.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061f4d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180b16e0 0x7f06181577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061f4d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180cba40 0x7f061815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061f4d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0618159d50 0x7f061815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061f4d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06180c5940 con 0x7f06180b16e0 2023-10-08T07:10:49.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061f4d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0618157d00 con 0x7f0618159d50 2023-10-08T07:10:49.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061f4d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061815b100 con 0x7f06180cba40 2023-10-08T07:10:49.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061d273640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180b16e0 0x7f06181577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:49.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061d273640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180b16e0 0x7f06181577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60790/0 (socket says 172.21.15.70:60790) 2023-10-08T07:10:49.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061d273640 1 -- 172.21.15.70:0/2565846315 learned_addr learned my addr 172.21.15.70:0/2565846315 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:49.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061d273640 1 -- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180cba40 msgr2=0x7f061815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061d273640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180cba40 0x7f061815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061d273640 1 -- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0618159d50 msgr2=0x7f061815c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061d273640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0618159d50 0x7f061815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.863+0000 7f061d273640 1 -- 172.21.15.70:0/2565846315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610009b30 con 0x7f06180b16e0 2023-10-08T07:10:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.864+0000 7f061d273640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180b16e0 0x7f06181577c0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f061400d7d0 tx=0x7f061400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.864+0000 7f060dffb640 1 -- 172.21.15.70:0/2565846315 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614004030 con 0x7f06180b16e0 2023-10-08T07:10:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.864+0000 7f060dffb640 1 -- 172.21.15.70:0/2565846315 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0614004190 con 0x7f06180b16e0 2023-10-08T07:10:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.864+0000 7f060dffb640 1 -- 172.21.15.70:0/2565846315 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614004520 con 0x7f06180b16e0 2023-10-08T07:10:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.864+0000 7f061f4d7640 1 -- 172.21.15.70:0/2565846315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061815b3e0 con 0x7f06180b16e0 2023-10-08T07:10:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.864+0000 7f061f4d7640 1 -- 172.21.15.70:0/2565846315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06180c5b50 con 0x7f06180b16e0 2023-10-08T07:10:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.865+0000 7f061f4d7640 1 -- 172.21.15.70:0/2565846315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0618071cb0 con 0x7f06180b16e0 2023-10-08T07:10:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.866+0000 7f060dffb640 1 -- 172.21.15.70:0/2565846315 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f06140096a0 con 0x7f06180b16e0 2023-10-08T07:10:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.867+0000 7f060dffb640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f05fc06af90 0x7f05fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.867+0000 7f060dffb640 1 -- 172.21.15.70:0/2565846315 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f0614089bf0 con 0x7f06180b16e0 2023-10-08T07:10:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.868+0000 7f061ca72640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f05fc06af90 0x7f05fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:49.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.869+0000 7f061ca72640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f05fc06af90 0x7f05fc06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f06180c2130 tx=0x7f0608008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:49.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:49.872+0000 7f060dffb640 1 -- 172.21.15.70:0/2565846315 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0614054930 con 0x7f06180b16e0 2023-10-08T07:10:50.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.129+0000 7f061f4d7640 1 -- 172.21.15.70:0/2565846315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f06180b42f0 con 0x7f06180b16e0 2023-10-08T07:10:50.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.133+0000 7f060dffb640 1 -- 172.21.15.70:0/2565846315 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v50) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0614057f50 con 0x7f06180b16e0 2023-10-08T07:10:50.134 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T07:10:50.135 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T07:10:50.135 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T07:10:50.135 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T07:10:50.135 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T07:10:50.135 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T07:10:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 -- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f05fc06af90 msgr2=0x7f05fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f05fc06af90 0x7f05fc06d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f06180c2130 tx=0x7f0608008040 comp rx=0 tx=0).stop 2023-10-08T07:10:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 -- 172.21.15.70:0/2565846315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180b16e0 msgr2=0x7f06181577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180b16e0 0x7f06181577c0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f061400d7d0 tx=0x7f061400dc90 comp rx=0 tx=0).stop 2023-10-08T07:10:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 -- 172.21.15.70:0/2565846315 shutdown_connections 2023-10-08T07:10:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f05fc06af90 0x7f05fc06d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0618159d50 0x7f061815c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06180cba40 0x7f061815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 --2- 172.21.15.70:0/2565846315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06180b16e0 0x7f06181577c0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 -- 172.21.15.70:0/2565846315 >> 172.21.15.70:0/2565846315 conn(0x7f061806a140 msgr2=0x7f061815e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 -- 172.21.15.70:0/2565846315 shutdown_connections 2023-10-08T07:10:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.136+0000 7f061f4d7640 1 -- 172.21.15.70:0/2565846315 wait complete. 2023-10-08T07:10:50.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:10:50.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.359+0000 7fe8ae10f640 1 Processor -- start 2023-10-08T07:10:50.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.360+0000 7fe8ae10f640 1 -- start start 2023-10-08T07:10:50.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.361+0000 7fe8ae10f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a815ab40 0x7fe8a80ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:50.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.361+0000 7fe8ae10f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe8a80c90b0 0x7fe8a80adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:50.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.361+0000 7fe8ae10f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a815b2b0 con 0x7fe8a80cae70 2023-10-08T07:10:50.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.361+0000 7fe8ae10f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a815bf30 con 0x7fe8a80c90b0 2023-10-08T07:10:50.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.361+0000 7fe8ae10f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a815f7a0 con 0x7fe8a815ab40 2023-10-08T07:10:50.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.362+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a815ab40 0x7fe8a80ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:50.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.362+0000 7fe89ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe8a80c90b0 0x7fe8a80adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:50.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.362+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a815ab40 0x7fe8a80ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:52996/0 (socket says 172.21.15.70:52996) 2023-10-08T07:10:50.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.362+0000 7fe8a77fe640 1 -- 172.21.15.70:0/369149985 learned_addr learned my addr 172.21.15.70:0/369149985 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:50.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.363+0000 7fe8a77fe640 1 -- 172.21.15.70:0/369149985 >> v2:172.21.15.181:3300/0 conn(0x7fe8a80c90b0 msgr2=0x7fe8a80adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:50.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.363+0000 7fe8a77fe640 1 --2- 172.21.15.70:0/369149985 >> v2:172.21.15.181:3300/0 conn(0x7fe8a80c90b0 0x7fe8a80adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.363+0000 7fe8a77fe640 1 -- 172.21.15.70:0/369149985 >> v1:172.21.15.181:6789/0 conn(0x7fe8a80cae70 legacy=0x7fe8a80702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:50.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.363+0000 7fe8a77fe640 1 -- 172.21.15.70:0/369149985 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a815b590 con 0x7fe8a815ab40 2023-10-08T07:10:50.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.364+0000 7fe8a77fe640 1 --2- 172.21.15.70:0/369149985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a815ab40 0x7fe8a80ad720 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe898009870 tx=0x7fe89802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7caff3aa149000e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:50.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.365+0000 7fe8a6ffd640 1 -- 172.21.15.70:0/369149985 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898031640 con 0x7fe8a815ab40 2023-10-08T07:10:50.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.365+0000 7fe8a6ffd640 1 -- 172.21.15.70:0/369149985 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8980317a0 con 0x7fe8a815ab40 2023-10-08T07:10:50.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.365+0000 7fe8a6ffd640 1 -- 172.21.15.70:0/369149985 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe898031b30 con 0x7fe8a815ab40 2023-10-08T07:10:50.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.366+0000 7fe8ae10f640 1 -- 172.21.15.70:0/369149985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a815ab40 msgr2=0x7fe8a80ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:50.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.366+0000 7fe8ae10f640 1 --2- 172.21.15.70:0/369149985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a815ab40 0x7fe8a80ad720 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe898009870 tx=0x7fe89802b8f0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.366+0000 7fe8ae10f640 1 -- 172.21.15.70:0/369149985 shutdown_connections 2023-10-08T07:10:50.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.366+0000 7fe8ae10f640 1 --2- 172.21.15.70:0/369149985 >> v2:172.21.15.181:3300/0 conn(0x7fe8a80c90b0 0x7fe8a80adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.366+0000 7fe8ae10f640 1 --2- 172.21.15.70:0/369149985 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a815ab40 0x7fe8a80ad720 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.366+0000 7fe8ae10f640 1 -- 172.21.15.70:0/369149985 >> 172.21.15.70:0/369149985 conn(0x7fe8a806a380 msgr2=0x7fe8a80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:50.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.366+0000 7fe8ae10f640 1 -- 172.21.15.70:0/369149985 shutdown_connections 2023-10-08T07:10:50.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.366+0000 7fe8ae10f640 1 -- 172.21.15.70:0/369149985 wait complete. 2023-10-08T07:10:50.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.368+0000 7fe8ae10f640 1 Processor -- start 2023-10-08T07:10:50.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.368+0000 7fe8ae10f640 1 -- start start 2023-10-08T07:10:50.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8ae10f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8a80c90b0 0x7fe8a81f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:50.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8ae10f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a80cae70 0x7fe8a80c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:50.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8ae10f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a815ab40 0x7fe8a80c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8ae10f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a80c6de0 con 0x7fe8a80c90b0 2023-10-08T07:10:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8ae10f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a81f56d0 con 0x7fe8a815ab40 2023-10-08T07:10:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8ae10f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a80c4200 con 0x7fe8a80cae70 2023-10-08T07:10:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8a80c90b0 0x7fe8a81f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8a80c90b0 0x7fe8a81f5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60802/0 (socket says 172.21.15.70:60802) 2023-10-08T07:10:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8a77fe640 1 -- 172.21.15.70:0/3432991759 learned_addr learned my addr 172.21.15.70:0/3432991759 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:50.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe89ffff640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a80cae70 0x7fe8a80c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:50.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8a7fff640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a815ab40 0x7fe8a80c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:50.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8a77fe640 1 -- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a80cae70 msgr2=0x7fe8a80c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8a77fe640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a80cae70 0x7fe8a80c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8a77fe640 1 -- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a815ab40 msgr2=0x7fe8a80c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8a77fe640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a815ab40 0x7fe8a80c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe8a77fe640 1 -- 172.21.15.70:0/3432991759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980094d0 con 0x7fe8a80c90b0 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.370+0000 7fe89ffff640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a80cae70 0x7fe8a80c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.371+0000 7fe8a77fe640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8a80c90b0 0x7fe8a81f5190 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fe8980090b0 tx=0x7fe8980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.371+0000 7fe8a4ff9640 1 -- 172.21.15.70:0/3432991759 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89803b940 con 0x7fe8a80c90b0 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.371+0000 7fe8a4ff9640 1 -- 172.21.15.70:0/3432991759 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89803bac0 con 0x7fe8a80c90b0 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.371+0000 7fe8a4ff9640 1 -- 172.21.15.70:0/3432991759 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89803be70 con 0x7fe8a80c90b0 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.371+0000 7fe8ae10f640 1 -- 172.21.15.70:0/3432991759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a80c44e0 con 0x7fe8a80c90b0 2023-10-08T07:10:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.371+0000 7fe8ae10f640 1 -- 172.21.15.70:0/3432991759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a80c4aa0 con 0x7fe8a80c90b0 2023-10-08T07:10:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.372+0000 7fe8ae10f640 1 -- 172.21.15.70:0/3432991759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a8067490 con 0x7fe8a80c90b0 2023-10-08T07:10:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.373+0000 7fe8a4ff9640 1 -- 172.21.15.70:0/3432991759 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe89802f050 con 0x7fe8a80c90b0 2023-10-08T07:10:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.375+0000 7fe8a4ff9640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fe88806af90 0x7fe88806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.375+0000 7fe8a4ff9640 1 -- 172.21.15.70:0/3432991759 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe8980ab9a0 con 0x7fe8a80c90b0 2023-10-08T07:10:50.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.375+0000 7fe89ffff640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fe88806af90 0x7fe88806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.377+0000 7fe89ffff640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fe88806af90 0x7fe88806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80c51d0 tx=0x7fe890008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.379+0000 7fe8a4ff9640 1 -- 172.21.15.70:0/3432991759 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe898076700 con 0x7fe8a80c90b0 2023-10-08T07:10:50.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.606+0000 7fe8ae10f640 1 -- 172.21.15.70:0/3432991759 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8a80b0ef0 con 0x7fe8a80c90b0 2023-10-08T07:10:50.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.608+0000 7fe8a4ff9640 1 -- 172.21.15.70:0/3432991759 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7fe898035090 con 0x7fe8a80c90b0 2023-10-08T07:10:50.610 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:10:50.610 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/4185609199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4185609199},{"type":"v1","addr":"172.21.15.70:6833","nonce":4185609199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3102206290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3102206290},{"type":"v1","addr":"172.21.15.181:6837","nonce":3102206290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1824427895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1824427895},{"type":"v1","addr":"172.21.15.181:6835","nonce":1824427895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:10:44.128443+0000","modified":"2023-10-08T07:10:47.166174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/2355440757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2355440757},{"type":"v1","addr":"172.21.15.70:6835","nonce":2355440757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-08T07:10:50.610 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 99 2023-10-08T07:10:50.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 -- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fe88806af90 msgr2=0x7fe88806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:50.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fe88806af90 0x7fe88806d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80c51d0 tx=0x7fe890008040 comp rx=0 tx=0).stop 2023-10-08T07:10:50.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 -- 172.21.15.70:0/3432991759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8a80c90b0 msgr2=0x7fe8a81f5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:50.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8a80c90b0 0x7fe8a81f5190 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7fe8980090b0 tx=0x7fe8980046c0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 -- 172.21.15.70:0/3432991759 shutdown_connections 2023-10-08T07:10:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fe88806af90 0x7fe88806d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8a815ab40 0x7fe8a80c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8a80cae70 0x7fe8a80c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 --2- 172.21.15.70:0/3432991759 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8a80c90b0 0x7fe8a81f5190 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 -- 172.21.15.70:0/3432991759 >> 172.21.15.70:0/3432991759 conn(0x7fe8a806a380 msgr2=0x7fe8a80cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.610+0000 7fe8ae10f640 1 -- 172.21.15.70:0/3432991759 shutdown_connections 2023-10-08T07:10:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:50.611+0000 7fe8ae10f640 1 -- 172.21.15.70:0/3432991759 wait complete. 2023-10-08T07:10:51.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T07:10:44.128443+0000', 'modified': '2023-10-08T07:10:47.166174+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10611}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10611': {'gid': 10611, 'name': 'c', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6835/2355440757', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 2355440757}, {'type': 'v1', 'addr': '172.21.15.70:6835', 'nonce': 2355440757}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T07:10:51.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T07:10:51.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T07:10:51.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.257+0000 7fc14c810640 1 Processor -- start 2023-10-08T07:10:51.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.257+0000 7fc14c810640 1 -- start start 2023-10-08T07:10:51.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.258+0000 7fc14c810640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1480c9cf0 0x7fc14806db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.258+0000 7fc14c810640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc14815b310 con 0x7fc14806c3a0 2023-10-08T07:10:51.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.258+0000 7fc14c810640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc14806d290 con 0x7fc14815aa00 2023-10-08T07:10:51.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.258+0000 7fc14c810640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14806e0a0 con 0x7fc1480c9cf0 2023-10-08T07:10:51.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.258+0000 7fc14659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc14806c3a0 0x7fc14806c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57724/0 (socket says 172.21.15.70:57724) 2023-10-08T07:10:51.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.258+0000 7fc14659c640 1 -- 172.21.15.70:0/989366104 learned_addr learned my addr 172.21.15.70:0/989366104 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592196544 0 0) 0x7fc138001110 con 0x7fc14806c3a0 2023-10-08T07:10:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc134003520 con 0x7fc14806c3a0 2023-10-08T07:10:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc13ffff640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1480c9cf0 0x7fc14806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3418339408 0 0) 0x7fc134003520 con 0x7fc14806c3a0 2023-10-08T07:10:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1480c9cf0 msgr2=0x7fc14806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1480c9cf0 0x7fc14806db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 >> v1:172.21.15.181:6789/0 conn(0x7fc14815aa00 legacy=0x7fc1480b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1480c9860 con 0x7fc14806c3a0 2023-10-08T07:10:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc138001800 con 0x7fc14806c3a0 2023-10-08T07:10:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 >> v1:172.21.15.70:6789/0 conn(0x7fc14806c3a0 legacy=0x7fc14806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1480c9cf0 0x7fc1340039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 0x7fc1340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc134004d00 0x7fc134005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14806e0a0 con 0x7fc134004000 2023-10-08T07:10:51.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13400bb70 con 0x7fc134004d00 2023-10-08T07:10:51.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13400df40 con 0x7fc1480c9cf0 2023-10-08T07:10:51.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.259+0000 7fc13ffff640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 0x7fc1340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.260+0000 7fc146d9d640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc134004d00 0x7fc134005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.260+0000 7fc14659c640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1480c9cf0 0x7fc1340039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.260+0000 7fc13ffff640 1 -- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1480c9cf0 msgr2=0x7fc1340039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.260+0000 7fc13ffff640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1480c9cf0 0x7fc1340039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.260+0000 7fc13ffff640 1 -- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc134004d00 msgr2=0x7fc134005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.260+0000 7fc13ffff640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc134004d00 0x7fc134005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.260+0000 7fc13ffff640 1 -- 172.21.15.70:0/989366104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc134003520 con 0x7fc134004000 2023-10-08T07:10:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.260+0000 7fc146d9d640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc134004d00 0x7fc134005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.261+0000 7fc13ffff640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 0x7fc1340047c0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fc13000ba30 tx=0x7fc1340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688eaba5bb1501d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:51.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.261+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130032640 con 0x7fc134004000 2023-10-08T07:10:51.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.261+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1300327a0 con 0x7fc134004000 2023-10-08T07:10:51.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.261+0000 7fc145d9b640 1 -- 172.21.15.70:0/989366104 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130032b30 con 0x7fc134004000 2023-10-08T07:10:51.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.262+0000 7fc14c810640 1 -- 172.21.15.70:0/989366104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 msgr2=0x7fc1340047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.262+0000 7fc14c810640 1 --2- 172.21.15.70:0/989366104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 0x7fc1340047c0 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fc13000ba30 tx=0x7fc1340057f0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.262+0000 7fc146d9d640 1 -- 172.21.15.70:0/989366104 reap_dead start 2023-10-08T07:10:51.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.262+0000 7fc14c810640 1 -- 172.21.15.70:0/989366104 shutdown_connections 2023-10-08T07:10:51.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.262+0000 7fc14c810640 1 -- 172.21.15.70:0/989366104 >> 172.21.15.70:0/989366104 conn(0x7fc14806a5d0 msgr2=0x7fc1480b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.262+0000 7fc14c810640 1 -- 172.21.15.70:0/989366104 shutdown_connections 2023-10-08T07:10:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.262+0000 7fc14c810640 1 -- 172.21.15.70:0/989366104 wait complete. 2023-10-08T07:10:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.264+0000 7fc14c810640 1 Processor -- start 2023-10-08T07:10:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.264+0000 7fc14c810640 1 -- start start 2023-10-08T07:10:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14c810640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 0x7fc14815de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14c810640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc14814d570 0x7fc14814cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14c810640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1480c2270 0x7fc14815bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14c810640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14815e340 con 0x7fc134004000 2023-10-08T07:10:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14c810640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480c6880 con 0x7fc1480c2270 2023-10-08T07:10:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14c810640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480c6af0 con 0x7fc14814d570 2023-10-08T07:10:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 0x7fc14815de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 0x7fc14815de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60814/0 (socket says 172.21.15.70:60814) 2023-10-08T07:10:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14659c640 1 -- 172.21.15.70:0/532861180 learned_addr learned my addr 172.21.15.70:0/532861180 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14659c640 1 -- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc14814d570 msgr2=0x7fc14814cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14659c640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc14814d570 0x7fc14814cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14659c640 1 -- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1480c2270 msgr2=0x7fc14815bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14659c640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1480c2270 0x7fc14815bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.265+0000 7fc14659c640 1 -- 172.21.15.70:0/532861180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1480c6dd0 con 0x7fc134004000 2023-10-08T07:10:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.266+0000 7fc14659c640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 0x7fc14815de00 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fc138001eb0 tx=0x7fc13800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:51.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.266+0000 7fc13effd640 1 -- 172.21.15.70:0/532861180 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380074e0 con 0x7fc134004000 2023-10-08T07:10:51.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.266+0000 7fc13effd640 1 -- 172.21.15.70:0/532861180 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc138007640 con 0x7fc134004000 2023-10-08T07:10:51.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.266+0000 7fc14c810640 1 -- 172.21.15.70:0/532861180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1480c70b0 con 0x7fc134004000 2023-10-08T07:10:51.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.266+0000 7fc13effd640 1 -- 172.21.15.70:0/532861180 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380049e0 con 0x7fc134004000 2023-10-08T07:10:51.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.266+0000 7fc14c810640 1 -- 172.21.15.70:0/532861180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14814d300 con 0x7fc134004000 2023-10-08T07:10:51.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.269+0000 7fc13effd640 1 -- 172.21.15.70:0/532861180 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc138004b40 con 0x7fc134004000 2023-10-08T07:10:51.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.269+0000 7fc13effd640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fc12806afc0 0x7fc12806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.270+0000 7fc13effd640 1 -- 172.21.15.70:0/532861180 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc138089550 con 0x7fc134004000 2023-10-08T07:10:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.270+0000 7fc14f39f740 1 Processor -- start 2023-10-08T07:10:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.270+0000 7fc13ffff640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fc12806afc0 0x7fc12806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.270+0000 7fc14f39f740 1 -- start start 2023-10-08T07:10:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.271+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c868b60 0x55931c83ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.271+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c86b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.271+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c83bfa0 0x55931c86be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.271+0000 7fc14f39f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931c7af700 con 0x55931c868f40 2023-10-08T07:10:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.271+0000 7fc14f39f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931c7acdf0 con 0x55931c868b60 2023-10-08T07:10:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.271+0000 7fc14f39f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55931c7a86e0 con 0x55931c83bfa0 2023-10-08T07:10:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.271+0000 7fc13ffff640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fc12806afc0 0x7fc12806d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc13000ba00 tx=0x7fc130032000 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.271+0000 7fc13ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c86b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.271+0000 7fc13ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c86b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60826/0 (socket says 172.21.15.70:60826) 2023-10-08T07:10:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.271+0000 7fc13ffff640 1 -- 172.21.15.70:0/696788887 learned_addr learned my addr 172.21.15.70:0/696788887 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.272+0000 7fc14659c640 1 --2- 172.21.15.70:0/696788887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c868b60 0x55931c83ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.272+0000 7fc146d9d640 1 --2- 172.21.15.70:0/696788887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c83bfa0 0x55931c86be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.272+0000 7fc13ffff640 1 -- 172.21.15.70:0/696788887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c83bfa0 msgr2=0x55931c86be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.272+0000 7fc13ffff640 1 --2- 172.21.15.70:0/696788887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c83bfa0 0x55931c86be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.272+0000 7fc13ffff640 1 -- 172.21.15.70:0/696788887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c868b60 msgr2=0x55931c83ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.272+0000 7fc13ffff640 1 --2- 172.21.15.70:0/696788887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c868b60 0x55931c83ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.272+0000 7fc13ffff640 1 -- 172.21.15.70:0/696788887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55931c76c910 con 0x55931c868f40 2023-10-08T07:10:51.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.272+0000 7fc146d9d640 1 --2- 172.21.15.70:0/696788887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c83bfa0 0x55931c86be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.272+0000 7fc13ffff640 1 --2- 172.21.15.70:0/696788887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c86b8f0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fc13002bf90 tx=0x7fc130033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a77fa322bb69750 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc127fff640 1 -- 172.21.15.70:0/696788887 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130050430 con 0x55931c868f40 2023-10-08T07:10:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc127fff640 1 -- 172.21.15.70:0/696788887 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13004b070 con 0x55931c868f40 2023-10-08T07:10:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc127fff640 1 -- 172.21.15.70:0/696788887 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1300507d0 con 0x55931c868f40 2023-10-08T07:10:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc14f39f740 1 -- 172.21.15.70:0/696788887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 msgr2=0x55931c86b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc14f39f740 1 --2- 172.21.15.70:0/696788887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c86b8f0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fc13002bf90 tx=0x7fc130033000 comp rx=0 tx=0).stop 2023-10-08T07:10:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc14f39f740 1 -- 172.21.15.70:0/696788887 shutdown_connections 2023-10-08T07:10:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc14f39f740 1 --2- 172.21.15.70:0/696788887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c83bfa0 0x55931c86be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc14f39f740 1 --2- 172.21.15.70:0/696788887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c86b8f0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc14f39f740 1 --2- 172.21.15.70:0/696788887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c868b60 0x55931c83ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc14f39f740 1 -- 172.21.15.70:0/696788887 >> 172.21.15.70:0/696788887 conn(0x55931c83dd70 msgr2=0x55931c8868f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc14f39f740 1 -- 172.21.15.70:0/696788887 shutdown_connections 2023-10-08T07:10:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.273+0000 7fc14f39f740 1 -- 172.21.15.70:0/696788887 wait complete. 2023-10-08T07:10:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.274+0000 7fc14f39f740 1 -- start start 2023-10-08T07:10:51.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.274+0000 7fc14f39f740 1 Processor -- start 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.275+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868b60 0x55931c867e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.275+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c87ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.275+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.275+0000 7fc14f39f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931c746970 con 0x55931c868f40 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.275+0000 7fc14f39f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931c746610 con 0x55931c87f270 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.275+0000 7fc14f39f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55931c50cf20 con 0x55931c868b60 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.275+0000 7fc14f39f740 10 client.0 _ll_register_callbacks cb 0x55931c784610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc13ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c87ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc13ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c87ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60830/0 (socket says 172.21.15.70:60830) 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc13ffff640 1 -- 172.21.15.70:0/1424008652 learned_addr learned my addr 172.21.15.70:0/1424008652 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:51.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc14659c640 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868b60 0x55931c867e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc13ffff640 1 -- 172.21.15.70:0/1424008652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868b60 msgr2=0x55931c867e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc13ffff640 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868b60 0x55931c867e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc13ffff640 1 -- 172.21.15.70:0/1424008652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 msgr2=0x55931c86de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc13ffff640 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc13ffff640 1 -- 172.21.15.70:0/1424008652 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc130041070 con 0x55931c868f40 2023-10-08T07:10:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc13ffff640 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c87ed30 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fc13004d000 tx=0x7fc1300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93496ac524de0e91 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.276+0000 7fc1257fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:10:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc1257fa640 1 -- 172.21.15.70:0/1424008652 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130050b20 con 0x55931c868f40 2023-10-08T07:10:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc14f39f740 1 -- 172.21.15.70:0/1424008652 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55931c7a16c0 con 0x55931c868f40 2023-10-08T07:10:51.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc1257fa640 1 -- 172.21.15.70:0/1424008652 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc130050c80 con 0x55931c868f40 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc1257fa640 1 -- 172.21.15.70:0/1424008652 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc130050e40 con 0x55931c868f40 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc14f39f740 10 client.10746 fetch_fsmap learned FSMap version 99 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc14f39f740 1 -- 172.21.15.70:0/1424008652 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55931c7a6d60 con 0x55931c868f40 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc1257fa640 1 -- 172.21.15.70:0/1424008652 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55931c7a16c0 con 0x55931c868f40 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc1257fa640 1 -- 172.21.15.70:0/1424008652 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc130058440 con 0x55931c868f40 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc14f39f740 10 client.10746 fetch_fsmap finished waiting for FSMap version 99 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc14f39f740 10 client.10746 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc14f39f740 20 client.10746 populate_metadata read hostname 'smithi070' 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc14f39f740 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.70:6834/2355440757,v1:172.21.15.70:6835/2355440757] conn(0x55931c87e100 0x55931c896d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc14f39f740 4 client.10746 mds_command: new command op to 10611 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.277+0000 7fc14f39f740 1 -- 172.21.15.70:0/1424008652 --> [v2:172.21.15.70:6834/2355440757,v1:172.21.15.70:6835/2355440757] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55931c7a13e0 con 0x55931c87e100 2023-10-08T07:10:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.278+0000 7fc14659c640 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.70:6834/2355440757,v1:172.21.15.70:6835/2355440757] conn(0x55931c87e100 0x55931c896d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.278+0000 7fc1257fa640 10 client.10746 ms_handle_connect on v2:172.21.15.70:6834/2355440757 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.278+0000 7fc14659c640 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.70:6834/2355440757,v1:172.21.15.70:6835/2355440757] conn(0x55931c87e100 0x55931c896d40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270edaf7e458df0d server_cookie=c66208dc02e27f93 in_seq=0 out_seq=0 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc1257fa640 1 -- 172.21.15.70:0/1424008652 <== mds.0 v2:172.21.15.70:6834/2355440757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55931c7a13e0 con 0x55931c87e100 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc1257fa640 10 client.10746 handle_command_reply: tid=0 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc14f39f740 1 client.10746 shutdown 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc14f39f740 1 -- 172.21.15.70:0/1424008652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 msgr2=0x55931c87ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc14f39f740 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c87ed30 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fc13004d000 tx=0x7fc1300046c0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc14f39f740 1 -- 172.21.15.70:0/1424008652 shutdown_connections 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc14f39f740 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.70:6834/2355440757,v1:172.21.15.70:6835/2355440757] conn(0x55931c87e100 0x55931c896d40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc14f39f740 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc14f39f740 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868f40 0x55931c87ed30 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc14f39f740 1 --2- 172.21.15.70:0/1424008652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868b60 0x55931c867e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc1257fa640 0 client.10746 ms_handle_reset on v2:172.21.15.70:6834/2355440757 2023-10-08T07:10:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc14f39f740 1 -- 172.21.15.70:0/1424008652 >> 172.21.15.70:0/1424008652 conn(0x55931c83bfa0 msgr2=0x55931c86b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.279+0000 7fc14f39f740 1 -- 172.21.15.70:0/1424008652 shutdown_connections 2023-10-08T07:10:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.280+0000 7fc14f39f740 1 -- 172.21.15.70:0/1424008652 wait complete. 2023-10-08T07:10:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.280+0000 7fc14f39f740 20 client.10746 trim_cache size 0 max 16384 2023-10-08T07:10:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.296+0000 7fc14f39f740 1 Processor -- start 2023-10-08T07:10:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.296+0000 7fc14f39f740 1 -- start start 2023-10-08T07:10:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.297+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868b60 0x55931c881b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.297+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868f40 0x55931c86c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.297+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.297+0000 7fc14f39f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931c7aa660 con 0x55931c868b60 2023-10-08T07:10:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.297+0000 7fc14f39f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931c86f3c0 con 0x55931c87f270 2023-10-08T07:10:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.297+0000 7fc14f39f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55931c868700 con 0x55931c868f40 2023-10-08T07:10:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.297+0000 7fc14659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868b60 0x55931c881b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc14659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868b60 0x55931c881b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60846/0 (socket says 172.21.15.70:60846) 2023-10-08T07:10:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc14659c640 1 -- 172.21.15.70:0/1652151554 learned_addr learned my addr 172.21.15.70:0/1652151554 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc13ffff640 1 --2- 172.21.15.70:0/1652151554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868f40 0x55931c86c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc146d9d640 1 --2- 172.21.15.70:0/1652151554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc14659c640 1 -- 172.21.15.70:0/1652151554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868f40 msgr2=0x55931c86c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc14659c640 1 --2- 172.21.15.70:0/1652151554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868f40 0x55931c86c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc14659c640 1 -- 172.21.15.70:0/1652151554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 msgr2=0x55931c86ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc14659c640 1 --2- 172.21.15.70:0/1652151554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc14659c640 1 -- 172.21.15.70:0/1652151554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55931c745b80 con 0x55931c868b60 2023-10-08T07:10:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc146d9d640 1 --2- 172.21.15.70:0/1652151554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc14659c640 1 --2- 172.21.15.70:0/1652151554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868b60 0x55931c881b20 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fc138057e70 tx=0x7fc138089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507673e05f8ab987 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc13cff9640 1 -- 172.21.15.70:0/1652151554 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138056020 con 0x55931c868b60 2023-10-08T07:10:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc13cff9640 1 -- 172.21.15.70:0/1652151554 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc138056280 con 0x55931c868b60 2023-10-08T07:10:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.298+0000 7fc13cff9640 1 -- 172.21.15.70:0/1652151554 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380545f0 con 0x55931c868b60 2023-10-08T07:10:51.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.299+0000 7fc14f39f740 1 -- 172.21.15.70:0/1652151554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868b60 msgr2=0x55931c881b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.299+0000 7fc14f39f740 1 --2- 172.21.15.70:0/1652151554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868b60 0x55931c881b20 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fc138057e70 tx=0x7fc138089040 comp rx=0 tx=0).stop 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.299+0000 7fc14f39f740 1 -- 172.21.15.70:0/1652151554 shutdown_connections 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.299+0000 7fc14f39f740 1 --2- 172.21.15.70:0/1652151554 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.299+0000 7fc14f39f740 1 --2- 172.21.15.70:0/1652151554 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868f40 0x55931c86c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.299+0000 7fc14f39f740 1 --2- 172.21.15.70:0/1652151554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c868b60 0x55931c881b20 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.299+0000 7fc14f39f740 1 -- 172.21.15.70:0/1652151554 >> 172.21.15.70:0/1652151554 conn(0x55931c83bfa0 msgr2=0x55931c86b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.299+0000 7fc14f39f740 1 -- 172.21.15.70:0/1652151554 shutdown_connections 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.299+0000 7fc14f39f740 1 -- 172.21.15.70:0/1652151554 wait complete. 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.300+0000 7fc14f39f740 1 -- start start 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.300+0000 7fc14f39f740 1 Processor -- start 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.301+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868f40 0x55931c8d5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.301+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.301+0000 7fc14f39f740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c8a80e0 0x55931c86bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.301+0000 7fc14f39f740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931c86d7e0 con 0x55931c8a80e0 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.301+0000 7fc14f39f740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931c8a9640 con 0x55931c87f270 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.301+0000 7fc14f39f740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55931c8a85a0 con 0x55931c868f40 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.301+0000 7fc14f39f740 10 client.0 _ll_register_callbacks cb 0x7fc1340036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.301+0000 7fc146d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c8a80e0 0x55931c86bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.301+0000 7fc146d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c8a80e0 0x55931c86bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60848/0 (socket says 172.21.15.70:60848) 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.301+0000 7fc146d9d640 1 -- 172.21.15.70:0/3118845589 learned_addr learned my addr 172.21.15.70:0/3118845589 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc14659c640 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868f40 0x55931c8d5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc13ffff640 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc146d9d640 1 -- 172.21.15.70:0/3118845589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868f40 msgr2=0x55931c8d5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc146d9d640 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868f40 0x55931c8d5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc146d9d640 1 -- 172.21.15.70:0/3118845589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 msgr2=0x55931c86b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc146d9d640 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc146d9d640 1 -- 172.21.15.70:0/3118845589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14814d300 con 0x55931c8a80e0 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc13ffff640 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc146d9d640 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c8a80e0 0x55931c86bc50 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fc14815ade0 tx=0x7fc14000f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d257ae0a2b6ad523 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc1257fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc1257fa640 1 -- 172.21.15.70:0/3118845589 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140006d50 con 0x55931c8a80e0 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc1257fa640 1 -- 172.21.15.70:0/3118845589 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc14000c030 con 0x55931c8a80e0 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.302+0000 7fc14f39f740 1 -- 172.21.15.70:0/3118845589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc13004b070 con 0x55931c8a80e0 2023-10-08T07:10:51.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.303+0000 7fc1257fa640 1 -- 172.21.15.70:0/3118845589 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc140013640 con 0x55931c8a80e0 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.303+0000 7fc14f39f740 10 client.10752 fetch_fsmap learned FSMap version 99 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.303+0000 7fc14f39f740 1 -- 172.21.15.70:0/3118845589 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55931c76b170 con 0x55931c8a80e0 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.303+0000 7fc1257fa640 1 -- 172.21.15.70:0/3118845589 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13004b070 con 0x55931c8a80e0 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.303+0000 7fc1257fa640 1 -- 172.21.15.70:0/3118845589 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc14001c430 con 0x55931c8a80e0 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.303+0000 7fc14f39f740 10 client.10752 fetch_fsmap finished waiting for FSMap version 99 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.303+0000 7fc14f39f740 10 client.10752 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.304+0000 7fc14f39f740 20 client.10752 populate_metadata read hostname 'smithi070' 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.304+0000 7fc14f39f740 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.70:6834/2355440757,v1:172.21.15.70:6835/2355440757] conn(0x55931c86d040 0x55931c896d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.304+0000 7fc14f39f740 4 client.10752 mds_command: new command op to 10611 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.304+0000 7fc14f39f740 1 -- 172.21.15.70:0/3118845589 --> [v2:172.21.15.70:6834/2355440757,v1:172.21.15.70:6835/2355440757] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55931c4aa6c0 con 0x55931c86d040 2023-10-08T07:10:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.304+0000 7fc14659c640 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.70:6834/2355440757,v1:172.21.15.70:6835/2355440757] conn(0x55931c86d040 0x55931c896d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.304+0000 7fc1257fa640 10 client.10752 ms_handle_connect on v2:172.21.15.70:6834/2355440757 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.304+0000 7fc14659c640 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.70:6834/2355440757,v1:172.21.15.70:6835/2355440757] conn(0x55931c86d040 0x55931c896d70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdfa35e393ea6208 server_cookie=55cde56ca9ec771f in_seq=0 out_seq=0 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc1257fa640 1 -- 172.21.15.70:0/3118845589 <== mds.0 v2:172.21.15.70:6834/2355440757 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55931c4aa6c0 con 0x55931c86d040 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc1257fa640 10 client.10752 handle_command_reply: tid=0 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 client.10752 shutdown 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 -- 172.21.15.70:0/3118845589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c8a80e0 msgr2=0x55931c86bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c8a80e0 0x55931c86bc50 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fc14815ade0 tx=0x7fc14000f8f0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 -- 172.21.15.70:0/3118845589 shutdown_connections 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.70:6834/2355440757,v1:172.21.15.70:6835/2355440757] conn(0x55931c86d040 0x55931c896d70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55931c8a80e0 0x55931c86bc50 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55931c87f270 0x55931c86b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 --2- 172.21.15.70:0/3118845589 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55931c868f40 0x55931c8d5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 -- 172.21.15.70:0/3118845589 >> 172.21.15.70:0/3118845589 conn(0x55931c83bfa0 msgr2=0x55931c881b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc1257fa640 0 client.10752 ms_handle_reset on v2:172.21.15.70:6834/2355440757 2023-10-08T07:10:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 -- 172.21.15.70:0/3118845589 shutdown_connections 2023-10-08T07:10:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 1 -- 172.21.15.70:0/3118845589 wait complete. 2023-10-08T07:10:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.305+0000 7fc14f39f740 20 client.10752 trim_cache size 0 max 16384 2023-10-08T07:10:51.314 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:10:51.314 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T07:10:51.314 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T07:10:51.315 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 10611, 2023-10-08T07:10:51.315 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T07:10:51.315 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T07:10:51.315 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T07:10:51.315 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.6501000230000002, 2023-10-08T07:10:51.315 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 99, 2023-10-08T07:10:51.315 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 216, 2023-10-08T07:10:51.315 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 216, 2023-10-08T07:10:51.315 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 13.054800859 2023-10-08T07:10:51.315 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:10:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 -- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fc12806afc0 msgr2=0x7fc12806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fc12806afc0 0x7fc12806d480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc13000ba00 tx=0x7fc130032000 comp rx=0 tx=0).stop 2023-10-08T07:10:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 -- 172.21.15.70:0/532861180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 msgr2=0x7fc14815de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 0x7fc14815de00 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fc138001eb0 tx=0x7fc13800dc00 comp rx=0 tx=0).stop 2023-10-08T07:10:51.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 -- 172.21.15.70:0/532861180 shutdown_connections 2023-10-08T07:10:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7fc12806afc0 0x7fc12806d480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1480c2270 0x7fc14815bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc14814d570 0x7fc14814cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 --2- 172.21.15.70:0/532861180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc134004000 0x7fc14815de00 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 -- 172.21.15.70:0/532861180 >> 172.21.15.70:0/532861180 conn(0x7fc14806a5d0 msgr2=0x7fc1480cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 -- 172.21.15.70:0/532861180 shutdown_connections 2023-10-08T07:10:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.306+0000 7fc14c810640 1 -- 172.21.15.70:0/532861180 wait complete. 2023-10-08T07:10:51.326 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T07:10:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.534+0000 7f611e16e640 1 Processor -- start 2023-10-08T07:10:51.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.534+0000 7f611e16e640 1 -- start start 2023-10-08T07:10:51.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.535+0000 7f611e16e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61180cbb60 0x7f61180c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.535+0000 7f611e16e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f611806dd00 0x7f61180ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.535+0000 7f611e16e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611806cfd0 con 0x7f611815a970 2023-10-08T07:10:51.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.535+0000 7f611e16e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611806d870 con 0x7f611806dd00 2023-10-08T07:10:51.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.535+0000 7f611e16e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611815ad50 con 0x7f61180cbb60 2023-10-08T07:10:51.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.536+0000 7f611d96d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f611806dd00 0x7f61180ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.536+0000 7f611d96d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f611806dd00 0x7f61180ca3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60864/0 (socket says 172.21.15.70:60864) 2023-10-08T07:10:51.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.536+0000 7f611d96d640 1 -- 172.21.15.70:0/147448432 learned_addr learned my addr 172.21.15.70:0/147448432 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:51.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.536+0000 7f611d96d640 1 -- 172.21.15.70:0/147448432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61180cbb60 msgr2=0x7f61180c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:51.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.536+0000 7f611c96b640 1 --2- 172.21.15.70:0/147448432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61180cbb60 0x7f61180c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.536+0000 7f611d96d640 1 --2- 172.21.15.70:0/147448432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61180cbb60 0x7f61180c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.536+0000 7f611d96d640 1 -- 172.21.15.70:0/147448432 >> v1:172.21.15.181:6789/0 conn(0x7f611815a970 legacy=0x7f61180cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.536+0000 7f611d96d640 1 -- 172.21.15.70:0/147448432 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61180cac20 con 0x7f611806dd00 2023-10-08T07:10:51.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.537+0000 7f611c96b640 1 --2- 172.21.15.70:0/147448432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61180cbb60 0x7f61180c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:51.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.538+0000 7f611d96d640 1 --2- 172.21.15.70:0/147448432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611806dd00 0x7f61180ca3a0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f6110009e50 tx=0x7f611002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dbc0f503457dbf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:51.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.538+0000 7f610ffff640 1 -- 172.21.15.70:0/147448432 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611002c9f0 con 0x7f611806dd00 2023-10-08T07:10:51.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.538+0000 7f610ffff640 1 -- 172.21.15.70:0/147448432 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611002cb50 con 0x7f611806dd00 2023-10-08T07:10:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.538+0000 7f610ffff640 1 -- 172.21.15.70:0/147448432 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61100343f0 con 0x7f611806dd00 2023-10-08T07:10:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.539+0000 7f611e16e640 1 -- 172.21.15.70:0/147448432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611806dd00 msgr2=0x7f61180ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.539+0000 7f611e16e640 1 --2- 172.21.15.70:0/147448432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611806dd00 0x7f61180ca3a0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f6110009e50 tx=0x7f611002be50 comp rx=0 tx=0).stop 2023-10-08T07:10:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.539+0000 7f611e16e640 1 -- 172.21.15.70:0/147448432 shutdown_connections 2023-10-08T07:10:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.539+0000 7f611e16e640 1 --2- 172.21.15.70:0/147448432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611806dd00 0x7f61180ca3a0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.539+0000 7f611e16e640 1 --2- 172.21.15.70:0/147448432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f61180cbb60 0x7f61180c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.539+0000 7f611e16e640 1 -- 172.21.15.70:0/147448432 >> 172.21.15.70:0/147448432 conn(0x7f61180ce9d0 msgr2=0x7f61180cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.539+0000 7f611e16e640 1 -- 172.21.15.70:0/147448432 shutdown_connections 2023-10-08T07:10:51.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.539+0000 7f611e16e640 1 -- 172.21.15.70:0/147448432 wait complete. 2023-10-08T07:10:51.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.541+0000 7f611e16e640 1 Processor -- start 2023-10-08T07:10:51.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.541+0000 7f611e16e640 1 -- start start 2023-10-08T07:10:51.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611e16e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f611806dd00 0x7f61180b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611e16e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61180cbb60 0x7f6118159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611e16e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f611815a970 0x7f61180c3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611e16e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611815a260 con 0x7f61180cbb60 2023-10-08T07:10:51.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611e16e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6118159890 con 0x7f611806dd00 2023-10-08T07:10:51.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611e16e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6118156e90 con 0x7f611815a970 2023-10-08T07:10:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611c96b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61180cbb60 0x7f6118159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611c96b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61180cbb60 0x7f6118159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60868/0 (socket says 172.21.15.70:60868) 2023-10-08T07:10:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611c96b640 1 -- 172.21.15.70:0/3081382455 learned_addr learned my addr 172.21.15.70:0/3081382455 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611d96d640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f611815a970 0x7f61180c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611d16c640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f611806dd00 0x7f61180b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611c96b640 1 -- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f611815a970 msgr2=0x7f61180c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.542+0000 7f611c96b640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f611815a970 0x7f61180c3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.543+0000 7f611c96b640 1 -- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f611806dd00 msgr2=0x7f61180b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.543+0000 7f611c96b640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f611806dd00 0x7f61180b0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.543+0000 7f611c96b640 1 -- 172.21.15.70:0/3081382455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6110009b30 con 0x7f61180cbb60 2023-10-08T07:10:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.543+0000 7f611c96b640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61180cbb60 0x7f6118159d20 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f610800c820 tx=0x7f610800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.543+0000 7f610dffb640 1 -- 172.21.15.70:0/3081382455 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108007a30 con 0x7f61180cbb60 2023-10-08T07:10:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.543+0000 7f610dffb640 1 -- 172.21.15.70:0/3081382455 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6108007b90 con 0x7f61180cbb60 2023-10-08T07:10:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.543+0000 7f610dffb640 1 -- 172.21.15.70:0/3081382455 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610800ce10 con 0x7f61180cbb60 2023-10-08T07:10:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.543+0000 7f611e16e640 1 -- 172.21.15.70:0/3081382455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6118159240 con 0x7f61180cbb60 2023-10-08T07:10:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.543+0000 7f611e16e640 1 -- 172.21.15.70:0/3081382455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61180c5200 con 0x7f61180cbb60 2023-10-08T07:10:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.544+0000 7f611e16e640 1 -- 172.21.15.70:0/3081382455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61181590a0 con 0x7f61180cbb60 2023-10-08T07:10:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.545+0000 7f610dffb640 1 -- 172.21.15.70:0/3081382455 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f61080113f0 con 0x7f61180cbb60 2023-10-08T07:10:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.546+0000 7f610dffb640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f60fc06af90 0x7f60fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.546+0000 7f610dffb640 1 -- 172.21.15.70:0/3081382455 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f6108089b50 con 0x7f61180cbb60 2023-10-08T07:10:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.547+0000 7f611d16c640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f60fc06af90 0x7f60fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.548+0000 7f611d16c640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f60fc06af90 0x7f60fc06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6114000f60 tx=0x7f6114009040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.551+0000 7f610dffb640 1 -- 172.21.15.70:0/3081382455 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61181590a0 con 0x7f61180cbb60 2023-10-08T07:10:51.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.779+0000 7f611e16e640 1 -- 172.21.15.70:0/3081382455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f61181590a0 con 0x7f61180cbb60 2023-10-08T07:10:51.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.780+0000 7f610dffb640 1 -- 172.21.15.70:0/3081382455 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7f61181590a0 con 0x7f61180cbb60 2023-10-08T07:10:51.781 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T07:10:51.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.782+0000 7f611e16e640 1 -- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f60fc06af90 msgr2=0x7f60fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.782+0000 7f611e16e640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f60fc06af90 0x7f60fc06d450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6114000f60 tx=0x7f6114009040 comp rx=0 tx=0).stop 2023-10-08T07:10:51.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.782+0000 7f611e16e640 1 -- 172.21.15.70:0/3081382455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61180cbb60 msgr2=0x7f6118159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:51.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.782+0000 7f611e16e640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61180cbb60 0x7f6118159d20 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f610800c820 tx=0x7f610800cce0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.782+0000 7f611e16e640 1 -- 172.21.15.70:0/3081382455 shutdown_connections 2023-10-08T07:10:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.782+0000 7f611e16e640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f60fc06af90 0x7f60fc06d450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.783+0000 7f611e16e640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f611815a970 0x7f61180c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.783+0000 7f611e16e640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61180cbb60 0x7f6118159d20 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.783+0000 7f611e16e640 1 --2- 172.21.15.70:0/3081382455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f611806dd00 0x7f61180b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.783+0000 7f611e16e640 1 -- 172.21.15.70:0/3081382455 >> 172.21.15.70:0/3081382455 conn(0x7f61180ce9d0 msgr2=0x7f61180b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.783+0000 7f611e16e640 1 -- 172.21.15.70:0/3081382455 shutdown_connections 2023-10-08T07:10:51.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:51.783+0000 7f611e16e640 1 -- 172.21.15.70:0/3081382455 wait complete. 2023-10-08T07:10:51.801 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T07:10:51.801 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T07:10:51.802 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T07:10:51.802 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T07:10:51.802 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T07:10:51.802 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T07:10:51.802 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T07:10:51.802 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T07:10:51.802 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:10:51.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:51.802 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T07:10:51.852 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T07:10:51.852 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T07:10:51.853 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:10:51.854 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:10:51.854 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T07:10:51.854 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:10:51.854 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T07:10:51.854 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 36745sec preferred_lft 36745sec 2023-10-08T07:10:51.854 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T07:10:51.854 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:10:51.854 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T07:10:51.854 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:10:51.855 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T07:10:51.855 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:10:51.855 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T07:10:51.855 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:10:51.855 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T07:10:51.855 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T07:10:51.855 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T07:10:51.855 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:10:51.855 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T07:10:51.855 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T07:10:51.856 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T07:10:51.856 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:10:51.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:51.857 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T07:10:51.950 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T07:10:51.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:51.954 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T07:10:52.009 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T07:10:52.009 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T07:10:52.009 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T07:10:52.010 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T07:10:52.010 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T07:10:52.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:52.010 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T07:10:52.010 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T07:10:52.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T07:10:52.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T07:10:52.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T07:10:52.011 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T07:10:52.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:52.198 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T07:10:52.249 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T07:10:52.249 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T07:10:52.250 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:10:52.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:52.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T07:10:52.326 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T07:10:52.326 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T07:10:52.374 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:10:52.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T07:10:52.375 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T07:10:52.422 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T07:10:52.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:52.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:10:52.613 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:10:52.614 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T07:10:52.648 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:10:52.649 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T07:10:52.649 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T07:10:52.649 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T07:10:52.649 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T07:10:52.651 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T07:10:52.651 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T07:10:52.651 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T07:10:52.651 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T07:10:52.651 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T07:10:52.655 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T07:10:52.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:52.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T07:10:52.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:52.749 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T07:10:52.821 INFO:teuthology.orchestra.run.smithi070.stdout:client7262 2023-10-08T07:10:52.821 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:10:52.821 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:10:52.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T07:10:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.087+0000 7f9a0d4d7640 1 Processor -- start 2023-10-08T07:10:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.087+0000 7f9a0d4d7640 1 -- start start 2023-10-08T07:10:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.089+0000 7f9a0d4d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a080afbe0 0x7f9a080b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.089+0000 7f9a0d4d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9a080cb130 0x7f9a080cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.089+0000 7f9a0d4d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a080cbad0 con 0x7f9a0806dfc0 2023-10-08T07:10:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.089+0000 7f9a0d4d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a080b2620 con 0x7f9a080afbe0 2023-10-08T07:10:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.089+0000 7f9a0d4d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a080b2060 con 0x7f9a080cb130 2023-10-08T07:10:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.089+0000 7f9a06ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a080afbe0 0x7f9a080b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.089+0000 7f9a06ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a080afbe0 0x7f9a080b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60870/0 (socket says 172.21.15.70:60870) 2023-10-08T07:10:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.089+0000 7f9a06ffd640 1 -- 172.21.15.70:0/1738400678 learned_addr learned my addr 172.21.15.70:0/1738400678 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.090+0000 7f9a077fe640 1 --2- 172.21.15.70:0/1738400678 >> v2:172.21.15.181:3300/0 conn(0x7f9a080cb130 0x7f9a080cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.090+0000 7f9a06ffd640 1 -- 172.21.15.70:0/1738400678 >> v2:172.21.15.181:3300/0 conn(0x7f9a080cb130 msgr2=0x7f9a080cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.090+0000 7f9a06ffd640 1 --2- 172.21.15.70:0/1738400678 >> v2:172.21.15.181:3300/0 conn(0x7f9a080cb130 0x7f9a080cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.090+0000 7f9a06ffd640 1 -- 172.21.15.70:0/1738400678 >> v1:172.21.15.181:6789/0 conn(0x7f9a0806dfc0 legacy=0x7f9a08070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.090+0000 7f9a06ffd640 1 -- 172.21.15.70:0/1738400678 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a080b2340 con 0x7f9a080afbe0 2023-10-08T07:10:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.090+0000 7f9a077fe640 1 --2- 172.21.15.70:0/1738400678 >> v2:172.21.15.181:3300/0 conn(0x7f9a080cb130 0x7f9a080cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.091+0000 7f9a06ffd640 1 --2- 172.21.15.70:0/1738400678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a080afbe0 0x7f9a080b15b0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f9a080c8d10 tx=0x7f99f802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2daef885bdf2dac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.091+0000 7f9a05ffb640 1 -- 172.21.15.70:0/1738400678 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f802ca80 con 0x7f9a080afbe0 2023-10-08T07:10:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.091+0000 7f9a05ffb640 1 -- 172.21.15.70:0/1738400678 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f802cbe0 con 0x7f9a080afbe0 2023-10-08T07:10:53.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.091+0000 7f9a05ffb640 1 -- 172.21.15.70:0/1738400678 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f80343f0 con 0x7f9a080afbe0 2023-10-08T07:10:53.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.092+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/1738400678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a080afbe0 msgr2=0x7f9a080b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:53.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.092+0000 7f9a0d4d7640 1 --2- 172.21.15.70:0/1738400678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a080afbe0 0x7f9a080b15b0 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f9a080c8d10 tx=0x7f99f802bee0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.092+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/1738400678 shutdown_connections 2023-10-08T07:10:53.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.092+0000 7f9a0d4d7640 1 --2- 172.21.15.70:0/1738400678 >> v2:172.21.15.181:3300/0 conn(0x7f9a080cb130 0x7f9a080cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.092+0000 7f9a0d4d7640 1 --2- 172.21.15.70:0/1738400678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a080afbe0 0x7f9a080b15b0 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.092+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/1738400678 >> 172.21.15.70:0/1738400678 conn(0x7f9a0806a5d0 msgr2=0x7f9a080b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:53.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.092+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/1738400678 shutdown_connections 2023-10-08T07:10:53.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.092+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/1738400678 wait complete. 2023-10-08T07:10:53.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.094+0000 7f9a0d4d7640 1 Processor -- start 2023-10-08T07:10:53.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.094+0000 7f9a0d4d7640 1 -- start start 2023-10-08T07:10:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.095+0000 7f9a0d4d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a0806dfc0 0x7f9a080c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.095+0000 7f9a0d4d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a080afbe0 0x7f9a081492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.095+0000 7f9a0d4d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a080cb130 0x7f9a081497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.095+0000 7f9a0d4d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081f6bd0 con 0x7f9a0806dfc0 2023-10-08T07:10:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.095+0000 7f9a0d4d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081f5f50 con 0x7f9a080afbe0 2023-10-08T07:10:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.095+0000 7f9a0d4d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081f6230 con 0x7f9a080cb130 2023-10-08T07:10:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.095+0000 7f9a06ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a0806dfc0 0x7f9a080c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.095+0000 7f9a06ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a0806dfc0 0x7f9a080c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60882/0 (socket says 172.21.15.70:60882) 2023-10-08T07:10:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.095+0000 7f9a06ffd640 1 -- 172.21.15.70:0/3968981403 learned_addr learned my addr 172.21.15.70:0/3968981403 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.096+0000 7f9a067fc640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a080afbe0 0x7f9a081492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.096+0000 7f9a06ffd640 1 -- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a080cb130 msgr2=0x7f9a081497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.096+0000 7f9a06ffd640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a080cb130 0x7f9a081497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.096+0000 7f9a06ffd640 1 -- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a080afbe0 msgr2=0x7f9a081492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.096+0000 7f9a06ffd640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a080afbe0 0x7f9a081492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.096+0000 7f9a06ffd640 1 -- 172.21.15.70:0/3968981403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8009b30 con 0x7f9a0806dfc0 2023-10-08T07:10:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.096+0000 7f9a06ffd640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a0806dfc0 0x7f9a080c6c00 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f99f8000f80 tx=0x7f99f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.096+0000 7f99ff7fe640 1 -- 172.21.15.70:0/3968981403 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f803e450 con 0x7f9a0806dfc0 2023-10-08T07:10:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.096+0000 7f99ff7fe640 1 -- 172.21.15.70:0/3968981403 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f803e5f0 con 0x7f9a0806dfc0 2023-10-08T07:10:53.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.096+0000 7f99ff7fe640 1 -- 172.21.15.70:0/3968981403 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f803e9a0 con 0x7f9a0806dfc0 2023-10-08T07:10:53.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.097+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/3968981403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a081f6510 con 0x7f9a0806dfc0 2023-10-08T07:10:53.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.097+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/3968981403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a080c7140 con 0x7f9a0806dfc0 2023-10-08T07:10:53.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.098+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/3968981403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0804e160 con 0x7f9a0806dfc0 2023-10-08T07:10:53.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.099+0000 7f99ff7fe640 1 -- 172.21.15.70:0/3968981403 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f99f802cd50 con 0x7f9a0806dfc0 2023-10-08T07:10:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.100+0000 7f99ff7fe640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f99e806af90 0x7f99e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.100+0000 7f99ff7fe640 1 -- 172.21.15.70:0/3968981403 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f99f80ac5f0 con 0x7f9a0806dfc0 2023-10-08T07:10:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.100+0000 7f9a067fc640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f99e806af90 0x7f99e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.101+0000 7f9a067fc640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f99e806af90 0x7f99e806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a0815de90 tx=0x7f99f0008040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:53.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.104+0000 7f99ff7fe640 1 -- 172.21.15.70:0/3968981403 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99f80774f0 con 0x7f9a0806dfc0 2023-10-08T07:10:53.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.327+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/3968981403 --> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a08070630 con 0x7f99e806af90 2023-10-08T07:10:53.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.331+0000 7f99ff7fe640 1 -- 172.21.15.70:0/3968981403 <== mgr.7112 v2:172.21.15.181:6832/2638163552 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f99f000a380 con 0x7f99e806af90 2023-10-08T07:10:53.332 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T07:10:53.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:10:53.333 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T07:10:53.333 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:10:53.333 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T07:10:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f99e806af90 msgr2=0x7f99e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:53.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f99e806af90 0x7f99e806d450 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a0815de90 tx=0x7f99f0008040 comp rx=0 tx=0).stop 2023-10-08T07:10:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/3968981403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a0806dfc0 msgr2=0x7f9a080c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a0806dfc0 0x7f9a080c6c00 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f99f8000f80 tx=0x7f99f8004a40 comp rx=0 tx=0).stop 2023-10-08T07:10:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/3968981403 shutdown_connections 2023-10-08T07:10:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f99e806af90 0x7f99e806d450 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a080cb130 0x7f9a081497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a080afbe0 0x7f9a081492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 --2- 172.21.15.70:0/3968981403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a0806dfc0 0x7f9a080c6c00 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/3968981403 >> 172.21.15.70:0/3968981403 conn(0x7f9a0806a5d0 msgr2=0x7f9a080b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.333+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/3968981403 shutdown_connections 2023-10-08T07:10:53.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.334+0000 7f9a0d4d7640 1 -- 172.21.15.70:0/3968981403 wait complete. 2023-10-08T07:10:53.354 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T07:10:53.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.565+0000 7f9afe683640 1 Processor -- start 2023-10-08T07:10:53.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.565+0000 7f9afe683640 1 -- start start 2023-10-08T07:10:53.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.567+0000 7f9afe683640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9af815a820 0x7f9af8160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.567+0000 7f9afe683640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9af80c9080 0x7f9af80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.567+0000 7f9afe683640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9af8070b50 0x7f9af8070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.567+0000 7f9afe683640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af80c9a40 con 0x7f9af815a820 2023-10-08T07:10:53.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.567+0000 7f9afe683640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af8071520 con 0x7f9af80c9080 2023-10-08T07:10:53.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.567+0000 7f9afe683640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af815ac00 con 0x7f9af8070b50 2023-10-08T07:10:53.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.567+0000 7f9af7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9af815a820 0x7f9af8160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.567+0000 7f9af7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9af815a820 0x7f9af8160750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60052/0 (socket says 172.21.15.70:60052) 2023-10-08T07:10:53.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.568+0000 7f9af7fff640 1 -- 172.21.15.70:0/902369419 learned_addr learned my addr 172.21.15.70:0/902369419 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:53.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.568+0000 7f9aeffff640 1 --2- 172.21.15.70:0/902369419 >> v2:172.21.15.181:3300/0 conn(0x7f9af80c9080 0x7f9af80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.568+0000 7f9afcc20640 1 --2- 172.21.15.70:0/902369419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9af8070b50 0x7f9af8070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.568+0000 7f9af7fff640 1 -- 172.21.15.70:0/902369419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9af8070b50 msgr2=0x7f9af8070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.568+0000 7f9af7fff640 1 --2- 172.21.15.70:0/902369419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9af8070b50 0x7f9af8070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.568+0000 7f9af7fff640 1 -- 172.21.15.70:0/902369419 >> v2:172.21.15.181:3300/0 conn(0x7f9af80c9080 msgr2=0x7f9af80c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.568+0000 7f9af7fff640 1 --2- 172.21.15.70:0/902369419 >> v2:172.21.15.181:3300/0 conn(0x7f9af80c9080 0x7f9af80c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.568+0000 7f9af7fff640 1 -- 172.21.15.70:0/902369419 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80c7ad0 con 0x7f9af815a820 2023-10-08T07:10:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.569+0000 7f9af7fff640 1 --2- 172.21.15.70:0/902369419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 0x7f9af8160750 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f9af806ab90 tx=0x7f9ae802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70fc4c9fc45095b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.570+0000 7f9af77fe640 1 -- 172.21.15.70:0/902369419 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8031870 con 0x7f9af815a820 2023-10-08T07:10:53.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.570+0000 7f9af77fe640 1 -- 172.21.15.70:0/902369419 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae80319d0 con 0x7f9af815a820 2023-10-08T07:10:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.570+0000 7f9af77fe640 1 -- 172.21.15.70:0/902369419 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8031d60 con 0x7f9af815a820 2023-10-08T07:10:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.570+0000 7f9afe683640 1 -- 172.21.15.70:0/902369419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 msgr2=0x7f9af8160750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.570+0000 7f9afe683640 1 --2- 172.21.15.70:0/902369419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 0x7f9af8160750 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f9af806ab90 tx=0x7f9ae802b910 comp rx=0 tx=0).stop 2023-10-08T07:10:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.571+0000 7f9afe683640 1 -- 172.21.15.70:0/902369419 shutdown_connections 2023-10-08T07:10:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.571+0000 7f9afe683640 1 --2- 172.21.15.70:0/902369419 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9af8070b50 0x7f9af8070fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.571+0000 7f9afe683640 1 --2- 172.21.15.70:0/902369419 >> v2:172.21.15.181:3300/0 conn(0x7f9af80c9080 0x7f9af80c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.571+0000 7f9afe683640 1 --2- 172.21.15.70:0/902369419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 0x7f9af8160750 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.571+0000 7f9afe683640 1 -- 172.21.15.70:0/902369419 >> 172.21.15.70:0/902369419 conn(0x7f9af8069d50 msgr2=0x7f9af806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.571+0000 7f9afe683640 1 -- 172.21.15.70:0/902369419 shutdown_connections 2023-10-08T07:10:53.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.571+0000 7f9afe683640 1 -- 172.21.15.70:0/902369419 wait complete. 2023-10-08T07:10:53.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.572+0000 7f9afe683640 1 Processor -- start 2023-10-08T07:10:53.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.573+0000 7f9afe683640 1 -- start start 2023-10-08T07:10:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afe683640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9af8070b50 0x7f9af81498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afe683640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9af80c9080 0x7f9af81f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afe683640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 0x7f9af80b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afe683640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af815d890 con 0x7f9af815a820 2023-10-08T07:10:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afe683640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af81f6140 con 0x7f9af8070b50 2023-10-08T07:10:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afe683640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af81f6420 con 0x7f9af80c9080 2023-10-08T07:10:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afcc20640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 0x7f9af80b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afcc20640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 0x7f9af80b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60062/0 (socket says 172.21.15.70:60062) 2023-10-08T07:10:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afcc20640 1 -- 172.21.15.70:0/152963749 learned_addr learned my addr 172.21.15.70:0/152963749 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afcc20640 1 -- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9af80c9080 msgr2=0x7f9af81f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afcc20640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9af80c9080 0x7f9af81f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9af7fff640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9af8070b50 0x7f9af81498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afcc20640 1 -- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9af8070b50 msgr2=0x7f9af81498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afcc20640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9af8070b50 0x7f9af81498a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.574+0000 7f9afcc20640 1 -- 172.21.15.70:0/152963749 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80094d0 con 0x7f9af815a820 2023-10-08T07:10:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.575+0000 7f9afcc20640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 0x7f9af80b2e80 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f9af0007e40 tx=0x7f9af000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.575+0000 7f9af57fa640 1 -- 172.21.15.70:0/152963749 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0014620 con 0x7f9af815a820 2023-10-08T07:10:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.575+0000 7f9af57fa640 1 -- 172.21.15.70:0/152963749 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af000ee90 con 0x7f9af815a820 2023-10-08T07:10:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.575+0000 7f9afe683640 1 -- 172.21.15.70:0/152963749 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af815d1d0 con 0x7f9af815a820 2023-10-08T07:10:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.575+0000 7f9af57fa640 1 -- 172.21.15.70:0/152963749 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00149e0 con 0x7f9af815a820 2023-10-08T07:10:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.575+0000 7f9afe683640 1 -- 172.21.15.70:0/152963749 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af8158e80 con 0x7f9af815a820 2023-10-08T07:10:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.576+0000 7f9afe683640 1 -- 172.21.15.70:0/152963749 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af815d9d0 con 0x7f9af815a820 2023-10-08T07:10:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.577+0000 7f9af57fa640 1 -- 172.21.15.70:0/152963749 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 116) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9af0014b40 con 0x7f9af815a820 2023-10-08T07:10:53.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.578+0000 7f9af57fa640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f9ad806af90 0x7f9ad806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:53.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.578+0000 7f9af57fa640 1 -- 172.21.15.70:0/152963749 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f9af008a330 con 0x7f9af815a820 2023-10-08T07:10:53.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.579+0000 7f9af7fff640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f9ad806af90 0x7f9ad806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:53.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.580+0000 7f9af7fff640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f9ad806af90 0x7f9ad806d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8036000 tx=0x7f9ae803c040 comp rx=0 tx=0).ready entity=mgr.7112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:53.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.583+0000 7f9af57fa640 1 -- 172.21.15.70:0/152963749 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9af815d9d0 con 0x7f9af815a820 2023-10-08T07:10:53.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:53.819+0000 7f9afe683640 1 -- 172.21.15.70:0/152963749 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f9af815d9d0 con 0x7f9af815a820 2023-10-08T07:10:54.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.234+0000 7f9af57fa640 1 -- 172.21.15.70:0/152963749 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v117) v1 ==== 92+0+0 (secure 0 0 0) 0x7f9af815d9d0 con 0x7f9af815a820 2023-10-08T07:10:54.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.239+0000 7f9afe683640 1 -- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f9ad806af90 msgr2=0x7f9ad806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:54.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.240+0000 7f9afe683640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f9ad806af90 0x7f9ad806d450 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8036000 tx=0x7f9ae803c040 comp rx=0 tx=0).stop 2023-10-08T07:10:54.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.240+0000 7f9afe683640 1 -- 172.21.15.70:0/152963749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 msgr2=0x7f9af80b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:54.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.240+0000 7f9afe683640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 0x7f9af80b2e80 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f9af0007e40 tx=0x7f9af000e4b0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.241+0000 7f9afe683640 1 -- 172.21.15.70:0/152963749 shutdown_connections 2023-10-08T07:10:54.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.241+0000 7f9afe683640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f9ad806af90 0x7f9ad806d450 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.241+0000 7f9afe683640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9af815a820 0x7f9af80b2e80 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.241+0000 7f9afe683640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9af80c9080 0x7f9af81f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.241+0000 7f9afe683640 1 --2- 172.21.15.70:0/152963749 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9af8070b50 0x7f9af81498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.241+0000 7f9afe683640 1 -- 172.21.15.70:0/152963749 >> 172.21.15.70:0/152963749 conn(0x7f9af8069d50 msgr2=0x7f9af80cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:54.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.241+0000 7f9afe683640 1 -- 172.21.15.70:0/152963749 shutdown_connections 2023-10-08T07:10:54.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.242+0000 7f9afe683640 1 -- 172.21.15.70:0/152963749 wait complete. 2023-10-08T07:10:54.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T07:10:54.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.459+0000 7f586a464640 1 Processor -- start 2023-10-08T07:10:54.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.459+0000 7f586a464640 1 -- start start 2023-10-08T07:10:54.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.460+0000 7f586a464640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f586406dfc0 0x7f5864070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.460+0000 7f586a464640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5864071070 0x7f58640cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.460+0000 7f586a464640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58640b1cc0 con 0x7f58640afbe0 2023-10-08T07:10:54.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.460+0000 7f586a464640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58640b2620 con 0x7f586406dfc0 2023-10-08T07:10:54.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.460+0000 7f586a464640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58640b2060 con 0x7f5864071070 2023-10-08T07:10:54.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.460+0000 7f58637fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f586406dfc0 0x7f5864070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.461+0000 7f58637fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f586406dfc0 0x7f5864070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60076/0 (socket says 172.21.15.70:60076) 2023-10-08T07:10:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.461+0000 7f58637fe640 1 -- 172.21.15.70:0/651926093 learned_addr learned my addr 172.21.15.70:0/651926093 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.461+0000 7f5868a01640 1 --2- 172.21.15.70:0/651926093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5864071070 0x7f58640cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.461+0000 7f58637fe640 1 -- 172.21.15.70:0/651926093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5864071070 msgr2=0x7f58640cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.461+0000 7f58637fe640 1 --2- 172.21.15.70:0/651926093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5864071070 0x7f58640cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.461+0000 7f58637fe640 1 -- 172.21.15.70:0/651926093 >> v1:172.21.15.181:6789/0 conn(0x7f58640afbe0 legacy=0x7f58640b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.461+0000 7f58637fe640 1 -- 172.21.15.70:0/651926093 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58640b2340 con 0x7f586406dfc0 2023-10-08T07:10:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.462+0000 7f58637fe640 1 --2- 172.21.15.70:0/651926093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f586406dfc0 0x7f5864070b30 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f584c009870 tx=0x7f584c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e010e1a4685bdd60 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.462+0000 7f5862ffd640 1 -- 172.21.15.70:0/651926093 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c030840 con 0x7f586406dfc0 2023-10-08T07:10:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.463+0000 7f5862ffd640 1 -- 172.21.15.70:0/651926093 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c0309a0 con 0x7f586406dfc0 2023-10-08T07:10:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.463+0000 7f5862ffd640 1 -- 172.21.15.70:0/651926093 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c030d30 con 0x7f586406dfc0 2023-10-08T07:10:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.463+0000 7f586a464640 1 -- 172.21.15.70:0/651926093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f586406dfc0 msgr2=0x7f5864070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:54.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.463+0000 7f586a464640 1 --2- 172.21.15.70:0/651926093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f586406dfc0 0x7f5864070b30 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f584c009870 tx=0x7f584c02b980 comp rx=0 tx=0).stop 2023-10-08T07:10:54.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.463+0000 7f586a464640 1 -- 172.21.15.70:0/651926093 shutdown_connections 2023-10-08T07:10:54.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.463+0000 7f586a464640 1 --2- 172.21.15.70:0/651926093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5864071070 0x7f58640cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.463+0000 7f586a464640 1 --2- 172.21.15.70:0/651926093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f586406dfc0 0x7f5864070b30 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.463+0000 7f586a464640 1 -- 172.21.15.70:0/651926093 >> 172.21.15.70:0/651926093 conn(0x7f586406a5d0 msgr2=0x7f58640b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:54.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.463+0000 7f586a464640 1 -- 172.21.15.70:0/651926093 shutdown_connections 2023-10-08T07:10:54.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.463+0000 7f586a464640 1 -- 172.21.15.70:0/651926093 wait complete. 2023-10-08T07:10:54.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.465+0000 7f586a464640 1 Processor -- start 2023-10-08T07:10:54.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.465+0000 7f586a464640 1 -- start start 2023-10-08T07:10:54.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.466+0000 7f586a464640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f586406dfc0 0x7f58640cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.466+0000 7f586a464640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5864071070 0x7f58640c4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.466+0000 7f586a464640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58640afbe0 0x7f5864155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.466+0000 7f586a464640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58640c4ec0 con 0x7f5864071070 2023-10-08T07:10:54.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.466+0000 7f586a464640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864155b80 con 0x7f58640afbe0 2023-10-08T07:10:54.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.466+0000 7f586a464640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58640c50f0 con 0x7f586406dfc0 2023-10-08T07:10:54.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.466+0000 7f58637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5864071070 0x7f58640c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:54.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f58637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5864071070 0x7f58640c4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60090/0 (socket says 172.21.15.70:60090) 2023-10-08T07:10:54.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f58637fe640 1 -- 172.21.15.70:0/2656342041 learned_addr learned my addr 172.21.15.70:0/2656342041 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:54.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f58637fe640 1 -- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f586406dfc0 msgr2=0x7f58640cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:54.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f5863fff640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f586406dfc0 0x7f58640cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:54.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f5868a01640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58640afbe0 0x7f5864155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:54.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f58637fe640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f586406dfc0 0x7f58640cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f58637fe640 1 -- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58640afbe0 msgr2=0x7f5864155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:54.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f58637fe640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58640afbe0 0x7f5864155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f58637fe640 1 -- 172.21.15.70:0/2656342041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c0094d0 con 0x7f5864071070 2023-10-08T07:10:54.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f5863fff640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f586406dfc0 0x7f58640cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:54.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f5868a01640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58640afbe0 0x7f5864155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:54.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.467+0000 7f58637fe640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5864071070 0x7f58640c4980 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f584c000c00 tx=0x7f584c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:54.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.468+0000 7f5860ff9640 1 -- 172.21.15.70:0/2656342041 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c030810 con 0x7f5864071070 2023-10-08T07:10:54.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.468+0000 7f5860ff9640 1 -- 172.21.15.70:0/2656342041 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c043400 con 0x7f5864071070 2023-10-08T07:10:54.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.468+0000 7f5860ff9640 1 -- 172.21.15.70:0/2656342041 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c0437d0 con 0x7f5864071070 2023-10-08T07:10:54.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.468+0000 7f586a464640 1 -- 172.21.15.70:0/2656342041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5864153a70 con 0x7f5864071070 2023-10-08T07:10:54.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.468+0000 7f586a464640 1 -- 172.21.15.70:0/2656342041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5864154030 con 0x7f5864071070 2023-10-08T07:10:54.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.469+0000 7f586a464640 1 -- 172.21.15.70:0/2656342041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586404e160 con 0x7f5864071070 2023-10-08T07:10:54.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.470+0000 7f5860ff9640 1 -- 172.21.15.70:0/2656342041 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 117) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f584c043930 con 0x7f5864071070 2023-10-08T07:10:54.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.471+0000 7f5860ff9640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f584406afe0 0x7f584406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.471+0000 7f5860ff9640 1 -- 172.21.15.70:0/2656342041 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f584c03e070 con 0x7f5864071070 2023-10-08T07:10:54.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.471+0000 7f5863fff640 1 -- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f584406afe0 msgr2=0x7f584406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2638163552 2023-10-08T07:10:54.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.471+0000 7f5863fff640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f584406afe0 0x7f584406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:54.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.476+0000 7f5860ff9640 1 -- 172.21.15.70:0/2656342041 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f584c0779f0 con 0x7f5864071070 2023-10-08T07:10:54.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.670+0000 7f586a464640 1 -- 172.21.15.70:0/2656342041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f58640b0ef0 con 0x7f5864071070 2023-10-08T07:10:54.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.672+0000 7f5863fff640 1 -- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f584406afe0 msgr2=0x7f584406d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2638163552 2023-10-08T07:10:54.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.672+0000 7f5863fff640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f584406afe0 0x7f584406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:10:54.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.675+0000 7f5860ff9640 1 -- 172.21.15.70:0/2656342041 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f584c07b010 con 0x7f5864071070 2023-10-08T07:10:54.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.677+0000 7f586a464640 1 -- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f584406afe0 msgr2=0x7f584406d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:54.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.677+0000 7f586a464640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f584406afe0 0x7f584406d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.677+0000 7f586a464640 1 -- 172.21.15.70:0/2656342041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5864071070 msgr2=0x7f58640c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:54.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.677+0000 7f586a464640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5864071070 0x7f58640c4980 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f584c000c00 tx=0x7f584c004730 comp rx=0 tx=0).stop 2023-10-08T07:10:54.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.678+0000 7f586a464640 1 -- 172.21.15.70:0/2656342041 shutdown_connections 2023-10-08T07:10:54.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.678+0000 7f586a464640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f584406afe0 0x7f584406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.678+0000 7f586a464640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f58640afbe0 0x7f5864155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.678+0000 7f586a464640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5864071070 0x7f58640c4980 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.678+0000 7f586a464640 1 --2- 172.21.15.70:0/2656342041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f586406dfc0 0x7f58640cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.678+0000 7f586a464640 1 -- 172.21.15.70:0/2656342041 >> 172.21.15.70:0/2656342041 conn(0x7f586406a5d0 msgr2=0x7f5864159310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:54.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.678+0000 7f586a464640 1 -- 172.21.15.70:0/2656342041 shutdown_connections 2023-10-08T07:10:54.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.678+0000 7f586a464640 1 -- 172.21.15.70:0/2656342041 wait complete. 2023-10-08T07:10:54.698 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T07:10:54.777 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:54.776+0000 7fe0a862b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:10:54.798 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:54.796+0000 7f2446030040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:10:54.830 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:54.828+0000 7fe0a862b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:10:54.858 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:54.856+0000 7f2446030040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:10:54.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.905+0000 7f8bd3905640 1 Processor -- start 2023-10-08T07:10:54.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.906+0000 7f8bd3905640 1 -- start start 2023-10-08T07:10:54.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.907+0000 7f8bd3905640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bcc15cc70 0x7f8bcc0cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.907+0000 7f8bd3905640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8bcc0b4600 0x7f8bcc0b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.907+0000 7f8bd3905640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc0b8b00 con 0x7f8bcc0d31a0 2023-10-08T07:10:54.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.907+0000 7f8bd3905640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bcc0b4fd0 con 0x7f8bcc0b4600 2023-10-08T07:10:54.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.907+0000 7f8bd3905640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0b8540 con 0x7f8bcc15cc70 2023-10-08T07:10:54.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.908+0000 7f8bd1ea2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8bcc0b4600 0x7f8bcc0b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:54.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.908+0000 7f8bd1ea2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8bcc0b4600 0x7f8bcc0b4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60092/0 (socket says 172.21.15.70:60092) 2023-10-08T07:10:54.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.908+0000 7f8bd1ea2640 1 -- 172.21.15.70:0/3711694572 learned_addr learned my addr 172.21.15.70:0/3711694572 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:54.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.908+0000 7f8bd16a1640 1 --2- 172.21.15.70:0/3711694572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bcc15cc70 0x7f8bcc0cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:54.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.908+0000 7f8bd1ea2640 1 -- 172.21.15.70:0/3711694572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bcc15cc70 msgr2=0x7f8bcc0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:54.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.908+0000 7f8bd1ea2640 1 --2- 172.21.15.70:0/3711694572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bcc15cc70 0x7f8bcc0cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.908+0000 7f8bd1ea2640 1 -- 172.21.15.70:0/3711694572 >> v1:172.21.15.181:6789/0 conn(0x7f8bcc0d31a0 legacy=0x7f8bcc0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:54.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.908+0000 7f8bd1ea2640 1 -- 172.21.15.70:0/3711694572 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc0b8820 con 0x7f8bcc0b4600 2023-10-08T07:10:54.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.909+0000 7f8bd1ea2640 1 --2- 172.21.15.70:0/3711694572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc0b4600 0x7f8bcc0b4a90 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009e50 tx=0x7f8bc402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fe89bbb4107fdf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:54.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.909+0000 7f8bc3fff640 1 -- 172.21.15.70:0/3711694572 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc402c9e0 con 0x7f8bcc0b4600 2023-10-08T07:10:54.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.909+0000 7f8bc3fff640 1 -- 172.21.15.70:0/3711694572 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc402cb40 con 0x7f8bcc0b4600 2023-10-08T07:10:54.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.909+0000 7f8bc3fff640 1 -- 172.21.15.70:0/3711694572 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40343f0 con 0x7f8bcc0b4600 2023-10-08T07:10:54.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.910+0000 7f8bd3905640 1 -- 172.21.15.70:0/3711694572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc0b4600 msgr2=0x7f8bcc0b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:54.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.910+0000 7f8bd3905640 1 --2- 172.21.15.70:0/3711694572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc0b4600 0x7f8bcc0b4a90 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009e50 tx=0x7f8bc402be40 comp rx=0 tx=0).stop 2023-10-08T07:10:54.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.910+0000 7f8bd3905640 1 -- 172.21.15.70:0/3711694572 shutdown_connections 2023-10-08T07:10:54.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.910+0000 7f8bd3905640 1 --2- 172.21.15.70:0/3711694572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc0b4600 0x7f8bcc0b4a90 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.910+0000 7f8bd3905640 1 --2- 172.21.15.70:0/3711694572 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bcc15cc70 0x7f8bcc0cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.910+0000 7f8bd3905640 1 -- 172.21.15.70:0/3711694572 >> 172.21.15.70:0/3711694572 conn(0x7f8bcc0718a0 msgr2=0x7f8bcc071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:54.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.910+0000 7f8bd3905640 1 -- 172.21.15.70:0/3711694572 shutdown_connections 2023-10-08T07:10:54.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.910+0000 7f8bd3905640 1 -- 172.21.15.70:0/3711694572 wait complete. 2023-10-08T07:10:54.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.912+0000 7f8bd3905640 1 Processor -- start 2023-10-08T07:10:54.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.912+0000 7f8bd3905640 1 -- start start 2023-10-08T07:10:54.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.913+0000 7f8bd3905640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bcc0b4600 0x7f8bcc14cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.913+0000 7f8bd3905640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bcc0d31a0 0x7f8bcc15f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.913+0000 7f8bd3905640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc15cc70 0x7f8bcc0cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.913+0000 7f8bd3905640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc15d050 con 0x7f8bcc15cc70 2023-10-08T07:10:54.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.913+0000 7f8bd3905640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc15d2a0 con 0x7f8bcc0b4600 2023-10-08T07:10:54.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.913+0000 7f8bd3905640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc15d580 con 0x7f8bcc0d31a0 2023-10-08T07:10:54.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.913+0000 7f8bd1ea2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc15cc70 0x7f8bcc0cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:54.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.913+0000 7f8bd1ea2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc15cc70 0x7f8bcc0cbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60096/0 (socket says 172.21.15.70:60096) 2023-10-08T07:10:54.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.913+0000 7f8bd1ea2640 1 -- 172.21.15.70:0/1452512211 learned_addr learned my addr 172.21.15.70:0/1452512211 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:54.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bd0ea0640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bcc0d31a0 0x7f8bcc15f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:54.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bd1ea2640 1 -- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bcc0d31a0 msgr2=0x7f8bcc15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:54.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bd1ea2640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bcc0d31a0 0x7f8bcc15f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bd1ea2640 1 -- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bcc0b4600 msgr2=0x7f8bcc14cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:54.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bd1ea2640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bcc0b4600 0x7f8bcc14cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:54.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bd1ea2640 1 -- 172.21.15.70:0/1452512211 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4009b30 con 0x7f8bcc15cc70 2023-10-08T07:10:54.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bd1ea2640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc15cc70 0x7f8bcc0cbe20 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4036040 tx=0x7f8bc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:54.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bc1ffb640 1 -- 172.21.15.70:0/1452512211 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4009510 con 0x7f8bcc15cc70 2023-10-08T07:10:54.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bc1ffb640 1 -- 172.21.15.70:0/1452512211 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc403e450 con 0x7f8bcc15cc70 2023-10-08T07:10:54.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bd3905640 1 -- 172.21.15.70:0/1452512211 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc15d860 con 0x7f8bcc15cc70 2023-10-08T07:10:54.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bc1ffb640 1 -- 172.21.15.70:0/1452512211 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc403e7e0 con 0x7f8bcc15cc70 2023-10-08T07:10:54.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.914+0000 7f8bd3905640 1 -- 172.21.15.70:0/1452512211 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc15de20 con 0x7f8bcc15cc70 2023-10-08T07:10:54.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.916+0000 7f8bd3905640 1 -- 172.21.15.70:0/1452512211 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc04e160 con 0x7f8bcc15cc70 2023-10-08T07:10:54.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.916+0000 7f8bc1ffb640 1 -- 172.21.15.70:0/1452512211 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 117) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f8bc403e940 con 0x7f8bcc15cc70 2023-10-08T07:10:54.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.917+0000 7f8bc1ffb640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8bb006b010 0x7f8bb006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:54.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.917+0000 7f8bc1ffb640 1 -- 172.21.15.70:0/1452512211 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f8bc40ac4e0 con 0x7f8bcc15cc70 2023-10-08T07:10:54.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.918+0000 7f8bd16a1640 1 -- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8bb006b010 msgr2=0x7f8bb006d4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2638163552 2023-10-08T07:10:54.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.918+0000 7f8bd16a1640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8bb006b010 0x7f8bb006d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:54.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:54.922+0000 7f8bc1ffb640 1 -- 172.21.15.70:0/1452512211 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bc4077220 con 0x7f8bcc15cc70 2023-10-08T07:10:55.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.111+0000 7f8bd3905640 1 -- 172.21.15.70:0/1452512211 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f8bcc0d28e0 con 0x7f8bcc15cc70 2023-10-08T07:10:55.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.116+0000 7f8bc1ffb640 1 -- 172.21.15.70:0/1452512211 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f8bc4033e80 con 0x7f8bcc15cc70 2023-10-08T07:10:55.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.118+0000 7f8bd3905640 1 -- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8bb006b010 msgr2=0x7f8bb006d4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:55.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8bb006b010 0x7f8bb006d4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:55.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 -- 172.21.15.70:0/1452512211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc15cc70 msgr2=0x7f8bcc0cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:55.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc15cc70 0x7f8bcc0cbe20 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4036040 tx=0x7f8bc4004a40 comp rx=0 tx=0).stop 2023-10-08T07:10:55.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 -- 172.21.15.70:0/1452512211 shutdown_connections 2023-10-08T07:10:55.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f8bb006b010 0x7f8bb006d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:55.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8bcc15cc70 0x7f8bcc0cbe20 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:55.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8bcc0d31a0 0x7f8bcc15f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:55.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 --2- 172.21.15.70:0/1452512211 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8bcc0b4600 0x7f8bcc14cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:55.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 -- 172.21.15.70:0/1452512211 >> 172.21.15.70:0/1452512211 conn(0x7f8bcc0718a0 msgr2=0x7f8bcc0d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:55.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 -- 172.21.15.70:0/1452512211 shutdown_connections 2023-10-08T07:10:55.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.119+0000 7f8bd3905640 1 -- 172.21.15.70:0/1452512211 wait complete. 2023-10-08T07:10:55.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:10:55.139 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T07:10:55.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-08T07:10:55.283 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:55.281+0000 7fe0a862b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:10:55.332 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:55.330+0000 7f2446030040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:10:55.335 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:55.333+0000 7fe0a862b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:10:55.384 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:55.383+0000 7fe0a862b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:10:55.385 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:55.384+0000 7f2446030040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:10:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.394+0000 7f784cfb7640 1 Processor -- start 2023-10-08T07:10:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.394+0000 7f784cfb7640 1 -- start start 2023-10-08T07:10:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.396+0000 7f784cfb7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f784815af60 0x7f784806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.396+0000 7f784cfb7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f784806e960 0x7f784806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.396+0000 7f784cfb7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78480c9840 con 0x7f784815ab80 2023-10-08T07:10:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.396+0000 7f784cfb7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78480ca430 con 0x7f784815af60 2023-10-08T07:10:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.396+0000 7f784cfb7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78480c9a90 con 0x7f784806e960 2023-10-08T07:10:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.396+0000 7f783ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f784815af60 0x7f784806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.396+0000 7f783ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f784815af60 0x7f784806b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60108/0 (socket says 172.21.15.70:60108) 2023-10-08T07:10:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.396+0000 7f783ffff640 1 -- 172.21.15.70:0/977779934 learned_addr learned my addr 172.21.15.70:0/977779934 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.396+0000 7f783ffff640 1 -- 172.21.15.70:0/977779934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f784806e960 msgr2=0x7f784806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.396+0000 7f7846d9d640 1 --2- 172.21.15.70:0/977779934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f784806e960 0x7f784806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.397+0000 7f783ffff640 1 --2- 172.21.15.70:0/977779934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f784806e960 0x7f784806edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.397+0000 7f783ffff640 1 -- 172.21.15.70:0/977779934 >> v1:172.21.15.181:6789/0 conn(0x7f784815ab80 legacy=0x7f78480b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.397+0000 7f783ffff640 1 -- 172.21.15.70:0/977779934 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78480c9d70 con 0x7f784815af60 2023-10-08T07:10:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.397+0000 7f7846d9d640 1 --2- 172.21.15.70:0/977779934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f784806e960 0x7f784806edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.398+0000 7f783ffff640 1 --2- 172.21.15.70:0/977779934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784815af60 0x7f784806b1b0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f7830009870 tx=0x7f783002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3bcff250b3a092 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.398+0000 7f7845d9b640 1 -- 172.21.15.70:0/977779934 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78300317c0 con 0x7f784815af60 2023-10-08T07:10:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.398+0000 7f7845d9b640 1 -- 172.21.15.70:0/977779934 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7830031920 con 0x7f784815af60 2023-10-08T07:10:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.398+0000 7f7845d9b640 1 -- 172.21.15.70:0/977779934 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830031cb0 con 0x7f784815af60 2023-10-08T07:10:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.399+0000 7f784cfb7640 1 -- 172.21.15.70:0/977779934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784815af60 msgr2=0x7f784806b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.399+0000 7f784cfb7640 1 --2- 172.21.15.70:0/977779934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784815af60 0x7f784806b1b0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f7830009870 tx=0x7f783002b860 comp rx=0 tx=0).stop 2023-10-08T07:10:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.399+0000 7f784cfb7640 1 -- 172.21.15.70:0/977779934 shutdown_connections 2023-10-08T07:10:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.399+0000 7f784cfb7640 1 --2- 172.21.15.70:0/977779934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f784806e960 0x7f784806edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.399+0000 7f784cfb7640 1 --2- 172.21.15.70:0/977779934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784815af60 0x7f784806b1b0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.399+0000 7f784cfb7640 1 -- 172.21.15.70:0/977779934 >> 172.21.15.70:0/977779934 conn(0x7f78480cba60 msgr2=0x7f78480cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.399+0000 7f784cfb7640 1 -- 172.21.15.70:0/977779934 shutdown_connections 2023-10-08T07:10:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.399+0000 7f784cfb7640 1 -- 172.21.15.70:0/977779934 wait complete. 2023-10-08T07:10:55.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.401+0000 7f784cfb7640 1 Processor -- start 2023-10-08T07:10:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.401+0000 7f784cfb7640 1 -- start start 2023-10-08T07:10:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.401+0000 7f784cfb7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f784806e960 0x7f78480addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f784cfb7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f784815ab80 0x7f78480afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f784cfb7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784815af60 0x7f78480b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f784cfb7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784815e220 con 0x7f784815af60 2023-10-08T07:10:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f784cfb7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78480ae2f0 con 0x7f784815ab80 2023-10-08T07:10:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f784cfb7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784815d610 con 0x7f784806e960 2023-10-08T07:10:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f7846d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784815af60 0x7f78480b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f7846d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784815af60 0x7f78480b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60116/0 (socket says 172.21.15.70:60116) 2023-10-08T07:10:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f7846d9d640 1 -- 172.21.15.70:0/977079038 learned_addr learned my addr 172.21.15.70:0/977079038 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:55.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f784659c640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f784806e960 0x7f78480addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:55.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f7846d9d640 1 -- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f784806e960 msgr2=0x7f78480addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:55.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f7846d9d640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f784806e960 0x7f78480addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:55.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f7846d9d640 1 -- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f784815ab80 msgr2=0x7f78480afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:55.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f7846d9d640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f784815ab80 0x7f78480afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:55.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.402+0000 7f7846d9d640 1 -- 172.21.15.70:0/977079038 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78300094d0 con 0x7f784815af60 2023-10-08T07:10:55.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.403+0000 7f7846d9d640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784815af60 0x7f78480b0130 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f784000cef0 tx=0x7f784000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:55.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.403+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784000d840 con 0x7f784815af60 2023-10-08T07:10:55.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.403+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784000d9c0 con 0x7f784815af60 2023-10-08T07:10:55.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.403+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784000dd70 con 0x7f784815af60 2023-10-08T07:10:55.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.403+0000 7f784cfb7640 1 -- 172.21.15.70:0/977079038 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784815d8f0 con 0x7f784815af60 2023-10-08T07:10:55.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.403+0000 7f784cfb7640 1 -- 172.21.15.70:0/977079038 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784815deb0 con 0x7f784815af60 2023-10-08T07:10:55.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.404+0000 7f784cfb7640 1 -- 172.21.15.70:0/977079038 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784804e160 con 0x7f784815af60 2023-10-08T07:10:55.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.405+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 117) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f784001e3b0 con 0x7f784815af60 2023-10-08T07:10:55.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.406+0000 7f783effd640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 0x7f782806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:55.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.406+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f784008a550 con 0x7f784815af60 2023-10-08T07:10:55.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.406+0000 7f784659c640 1 -- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 msgr2=0x7f782806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2638163552 2023-10-08T07:10:55.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.406+0000 7f784659c640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 0x7f782806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:10:55.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.410+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7840055290 con 0x7f784815af60 2023-10-08T07:10:55.440 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:55.439+0000 7f2446030040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:10:55.447 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:55.446+0000 7fe0a862b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:10:55.496 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:55.495+0000 7fe0a862b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:10:55.503 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:55.501+0000 7f2446030040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:10:55.553 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:55.551+0000 7f2446030040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:10:55.606 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:55.605+0000 7fe0a862b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:10:55.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.607+0000 7f784659c640 1 -- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 msgr2=0x7f782806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2638163552 2023-10-08T07:10:55.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.607+0000 7f784659c640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 0x7f782806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:10:55.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:55.620+0000 7f784cfb7640 1 -- 172.21.15.70:0/977079038 --> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f78480ca0c0 con 0x7f782806afe0 2023-10-08T07:10:55.666 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:55.664+0000 7f2446030040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:10:55.668 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:55.667+0000 7fe0a862b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:10:55.718 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:55.717+0000 7fe0a862b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:10:55.729 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:55.727+0000 7f2446030040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:10:55.775 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:55.774+0000 7fe0a862b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:10:55.778 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:55.776+0000 7f2446030040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:10:55.824 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:55.823+0000 7fe0a862b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:10:55.834 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:55.832+0000 7f2446030040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:10:55.884 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:55.882+0000 7f2446030040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:10:56.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:56.008+0000 7f784659c640 1 -- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 msgr2=0x7f782806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2638163552 2023-10-08T07:10:56.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:56.008+0000 7f784659c640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 0x7f782806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T07:10:56.201 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:56.199+0000 7fe0a862b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:10:56.262 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:56.260+0000 7f2446030040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:10:56.359 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:56.358+0000 7fe0a862b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:10:56.421 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:56.419+0000 7f2446030040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:10:56.466 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:56.465+0000 7fe0a862b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:10:56.516 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:56.515+0000 7fe0a862b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:10:56.528 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:56.526+0000 7f2446030040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:10:56.578 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:56.577+0000 7f2446030040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:10:56.623 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:56.622+0000 7fe0a862b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:10:56.687 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:56.685+0000 7f2446030040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:10:56.730 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:56.729+0000 7fe0a862b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:10:56.794 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:56.793+0000 7f2446030040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:10:56.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:56.809+0000 7f784659c640 1 -- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 msgr2=0x7f782806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2638163552 2023-10-08T07:10:56.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:56.809+0000 7f784659c640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 0x7f782806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T07:10:56.885 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:56.883+0000 7fe0a862b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:10:56.934 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:56.932+0000 7fe0a862b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:10:56.952 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:56.950+0000 7f2446030040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:10:57.001 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:56.999+0000 7f2446030040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:10:57.053 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:57.051+0000 7fe0a862b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:10:57.119 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:57.116+0000 7f2446030040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:10:57.862 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:57.861+0000 7fe0a862b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:10:57.942 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:57.940+0000 7f2446030040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:10:58.157 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.156+0000 7fe0a862b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:10:58.214 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.212+0000 7fe0a862b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:10:58.239 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:58.237+0000 7f2446030040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:10:58.297 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:58.295+0000 7f2446030040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:10:58.329 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.327+0000 7fe0a862b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:10:58.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:58.347+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 118) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f7840055660 con 0x7f784815af60 2023-10-08T07:10:58.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:58.348+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 msgr2=0x7f782806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:10:58.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:58.348+0000 7f783effd640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2638163552,v1:172.21.15.181:6833/2638163552] conn(0x7f782806afe0 0x7f782806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:58.426 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:10:58.425+0000 7f2446030040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:10:58.528 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.527+0000 7fe02490e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:58.528 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.527+0000 7fe02490e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:58.528 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.527+0000 7fe02490e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:58.529 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.527+0000 7fe02490e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:58.529 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.527+0000 7fe02490e640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:58.529 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.527+0000 7fe026952640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:58.529 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.527+0000 7fe026952640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:58.529 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.527+0000 7fe026952640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:58.529 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.527+0000 7fe026952640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:58.529 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:10:58.527+0000 7fe026952640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:10:59.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.357+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 119) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f7840088810 con 0x7f784815af60 2023-10-08T07:10:59.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.358+0000 7f783effd640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f78280355a0 0x7f7828035980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:59.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.358+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 --> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7828035ec0 con 0x7f78280355a0 2023-10-08T07:10:59.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.359+0000 7f784659c640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f78280355a0 0x7f7828035980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:59.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.361+0000 7f784659c640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f78280355a0 0x7f7828035980 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7838007fd0 tx=0x7f7838009040 comp rx=0 tx=0).ready entity=mgr.8188 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:59.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.521+0000 7f783effd640 1 -- 172.21.15.70:0/977079038 <== mgr.8188 v2:172.21.15.181:6832/2613103103 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7828035ec0 con 0x7f78280355a0 2023-10-08T07:10:59.523 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1 2023-10-08T07:10:59.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.524+0000 7f784cfb7640 1 -- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f78280355a0 msgr2=0x7f7828035980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:59.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.524+0000 7f784cfb7640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f78280355a0 0x7f7828035980 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7838007fd0 tx=0x7f7838009040 comp rx=0 tx=0).stop 2023-10-08T07:10:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.524+0000 7f784cfb7640 1 -- 172.21.15.70:0/977079038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784815af60 msgr2=0x7f78480b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.525+0000 7f784cfb7640 1 --2- 172.21.15.70:0/977079038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f784815af60 0x7f78480b0130 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f784000cef0 tx=0x7f784000a710 comp rx=0 tx=0).stop 2023-10-08T07:10:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.525+0000 7f7846d9d640 1 -- 172.21.15.70:0/977079038 reap_dead start 2023-10-08T07:10:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.525+0000 7f784cfb7640 1 -- 172.21.15.70:0/977079038 shutdown_connections 2023-10-08T07:10:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.525+0000 7f784cfb7640 1 -- 172.21.15.70:0/977079038 >> 172.21.15.70:0/977079038 conn(0x7f78480cba60 msgr2=0x7f78480cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.525+0000 7f784cfb7640 1 -- 172.21.15.70:0/977079038 shutdown_connections 2023-10-08T07:10:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.525+0000 7f784cfb7640 1 -- 172.21.15.70:0/977079038 wait complete. 2023-10-08T07:10:59.547 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T07:10:59.547 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2023-10-08T07:10:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.758+0000 7f06abc75640 1 Processor -- start 2023-10-08T07:10:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.758+0000 7f06abc75640 1 -- start start 2023-10-08T07:10:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.760+0000 7f06abc75640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f06a406c3a0 0x7f06a40c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.760+0000 7f06abc75640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06a406db60 0x7f06a406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.760+0000 7f06abc75640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a415b310 con 0x7f06a415aa00 2023-10-08T07:10:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.760+0000 7f06abc75640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a40ca230 con 0x7f06a406db60 2023-10-08T07:10:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.760+0000 7f06abc75640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a406d290 con 0x7f06a406c3a0 2023-10-08T07:10:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.760+0000 7f06a9210640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06a406db60 0x7f06a406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.760+0000 7f06aa212640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f06a415aa00 0x7f06a40b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38736/0 (socket says 172.21.15.70:38736) 2023-10-08T07:10:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.760+0000 7f06a9210640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06a406db60 0x7f06a406c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60120/0 (socket says 172.21.15.70:60120) 2023-10-08T07:10:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.760+0000 7f06aa212640 1 -- 172.21.15.70:0/1759728272 learned_addr learned my addr 172.21.15.70:0/1759728272 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:10:59.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.761+0000 7f06a9a11640 1 --2- 172.21.15.70:0/1759728272 >> v2:172.21.15.181:3300/0 conn(0x7f06a406c3a0 0x7f06a40c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:59.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.761+0000 7f06a9210640 1 -- 172.21.15.70:0/1759728272 >> v2:172.21.15.181:3300/0 conn(0x7f06a406c3a0 msgr2=0x7f06a40c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:10:59.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.761+0000 7f06a9210640 1 --2- 172.21.15.70:0/1759728272 >> v2:172.21.15.181:3300/0 conn(0x7f06a406c3a0 0x7f06a40c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:59.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.761+0000 7f06a9210640 1 -- 172.21.15.70:0/1759728272 >> v1:172.21.15.70:6789/0 conn(0x7f06a415aa00 legacy=0x7f06a40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.761+0000 7f06a9210640 1 -- 172.21.15.70:0/1759728272 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a40c9860 con 0x7f06a406db60 2023-10-08T07:10:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.761+0000 7f06a9a11640 1 --2- 172.21.15.70:0/1759728272 >> v2:172.21.15.181:3300/0 conn(0x7f06a406c3a0 0x7f06a40c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:10:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.761+0000 7f06a8a0f640 1 -- 172.21.15.70:0/1759728272 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556269040 0 0) 0x7f069c001490 con 0x7f06a415aa00 2023-10-08T07:10:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.762+0000 7f06a9210640 1 --2- 172.21.15.70:0/1759728272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 0x7f06a406c780 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f06a40716b0 tx=0x7f069402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f67a14c1b8134ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.762+0000 7f06a8a0f640 1 -- 172.21.15.70:0/1759728272 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694031830 con 0x7f06a406db60 2023-10-08T07:10:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.762+0000 7f06a8a0f640 1 -- 172.21.15.70:0/1759728272 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694031990 con 0x7f06a406db60 2023-10-08T07:10:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.762+0000 7f06a8a0f640 1 -- 172.21.15.70:0/1759728272 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694031d20 con 0x7f06a406db60 2023-10-08T07:10:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.763+0000 7f06abc75640 1 -- 172.21.15.70:0/1759728272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 msgr2=0x7f06a406c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.763+0000 7f06abc75640 1 --2- 172.21.15.70:0/1759728272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 0x7f06a406c780 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f06a40716b0 tx=0x7f069402b8d0 comp rx=0 tx=0).stop 2023-10-08T07:10:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.763+0000 7f06abc75640 1 -- 172.21.15.70:0/1759728272 shutdown_connections 2023-10-08T07:10:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.763+0000 7f06abc75640 1 --2- 172.21.15.70:0/1759728272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 0x7f06a406c780 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.763+0000 7f06abc75640 1 --2- 172.21.15.70:0/1759728272 >> v2:172.21.15.181:3300/0 conn(0x7f06a406c3a0 0x7f06a40c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.763+0000 7f06abc75640 1 -- 172.21.15.70:0/1759728272 >> 172.21.15.70:0/1759728272 conn(0x7f06a406a5d0 msgr2=0x7f06a40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:10:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.763+0000 7f06abc75640 1 -- 172.21.15.70:0/1759728272 shutdown_connections 2023-10-08T07:10:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.763+0000 7f06abc75640 1 -- 172.21.15.70:0/1759728272 wait complete. 2023-10-08T07:10:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.765+0000 7f06abc75640 1 Processor -- start 2023-10-08T07:10:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.765+0000 7f06abc75640 1 -- start start 2023-10-08T07:10:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.766+0000 7f06abc75640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a406c3a0 0x7f06a40c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.766+0000 7f06abc75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 0x7f06a4156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.766+0000 7f06abc75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06a415aa00 0x7f06a40c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.766+0000 7f06abc75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40cde00 con 0x7f06a406db60 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.766+0000 7f06abc75640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40aef60 con 0x7f06a406c3a0 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.766+0000 7f06abc75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a4156cc0 con 0x7f06a415aa00 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.766+0000 7f06a9210640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 0x7f06a4156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.766+0000 7f06a9210640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 0x7f06a4156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60130/0 (socket says 172.21.15.70:60130) 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.766+0000 7f06a9210640 1 -- 172.21.15.70:0/1301781475 learned_addr learned my addr 172.21.15.70:0/1301781475 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.767+0000 7f06a9210640 1 -- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06a415aa00 msgr2=0x7f06a40c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.767+0000 7f06a9210640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06a415aa00 0x7f06a40c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.767+0000 7f06a9210640 1 -- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a406c3a0 msgr2=0x7f06a40c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.767+0000 7f06a9210640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a406c3a0 0x7f06a40c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:10:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.767+0000 7f06a9210640 1 -- 172.21.15.70:0/1301781475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0694009400 con 0x7f06a406db60 2023-10-08T07:10:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.767+0000 7f06a9210640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 0x7f06a4156780 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f0694009890 tx=0x7f06940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.767+0000 7f069a7fc640 1 -- 172.21.15.70:0/1301781475 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069403ba70 con 0x7f06a406db60 2023-10-08T07:10:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.767+0000 7f069a7fc640 1 -- 172.21.15.70:0/1301781475 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069403bbf0 con 0x7f06a406db60 2023-10-08T07:10:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.767+0000 7f069a7fc640 1 -- 172.21.15.70:0/1301781475 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694045400 con 0x7f06a406db60 2023-10-08T07:10:59.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.767+0000 7f06abc75640 1 -- 172.21.15.70:0/1301781475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a40af200 con 0x7f06a406db60 2023-10-08T07:10:59.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.768+0000 7f06abc75640 1 -- 172.21.15.70:0/1301781475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a40cdb20 con 0x7f06a406db60 2023-10-08T07:10:59.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.769+0000 7f06abc75640 1 -- 172.21.15.70:0/1301781475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a404e160 con 0x7f06a406db60 2023-10-08T07:10:59.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.769+0000 7f069a7fc640 1 -- 172.21.15.70:0/1301781475 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 119) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f069404e440 con 0x7f06a406db60 2023-10-08T07:10:59.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.770+0000 7f069a7fc640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f068806ad10 0x7f068806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:10:59.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.770+0000 7f069a7fc640 1 -- 172.21.15.70:0/1301781475 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f06940ab8a0 con 0x7f06a406db60 2023-10-08T07:10:59.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.771+0000 7f06a9a11640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f068806ad10 0x7f068806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:10:59.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.772+0000 7f06a9a11640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f068806ad10 0x7f068806d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f06a0006fd0 tx=0x7f06a0008040 comp rx=0 tx=0).ready entity=mgr.8188 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:10:59.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.775+0000 7f069a7fc640 1 -- 172.21.15.70:0/1301781475 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0694076aa0 con 0x7f06a406db60 2023-10-08T07:10:59.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:10:59.987+0000 7f06abc75640 1 -- 172.21.15.70:0/1301781475 --> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f06a40c8f30 con 0x7f068806ad10 2023-10-08T07:11:00.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.125+0000 7f069a7fc640 1 -- 172.21.15.70:0/1301781475 <== mgr.8188 v2:172.21.15.181:6832/2613103103 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f06a000ae40 con 0x7f068806ad10 2023-10-08T07:11:00.126 INFO:teuthology.orchestra.run.smithi070.stdout:Retention added to path snap_test_dir1 2023-10-08T07:11:00.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.127+0000 7f06abc75640 1 -- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f068806ad10 msgr2=0x7f068806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:11:00.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.127+0000 7f06abc75640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f068806ad10 0x7f068806d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f06a0006fd0 tx=0x7f06a0008040 comp rx=0 tx=0).stop 2023-10-08T07:11:00.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.127+0000 7f06abc75640 1 -- 172.21.15.70:0/1301781475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 msgr2=0x7f06a4156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:11:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.127+0000 7f06abc75640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 0x7f06a4156780 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f0694009890 tx=0x7f06940046c0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.128+0000 7f06abc75640 1 -- 172.21.15.70:0/1301781475 shutdown_connections 2023-10-08T07:11:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.128+0000 7f06abc75640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f068806ad10 0x7f068806d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.128+0000 7f06abc75640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06a415aa00 0x7f06a40c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.128+0000 7f06abc75640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a406db60 0x7f06a4156780 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.128+0000 7f06abc75640 1 --2- 172.21.15.70:0/1301781475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f06a406c3a0 0x7f06a40c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.128+0000 7f06abc75640 1 -- 172.21.15.70:0/1301781475 >> 172.21.15.70:0/1301781475 conn(0x7f06a406a5d0 msgr2=0x7f06a40b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:11:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.128+0000 7f06abc75640 1 -- 172.21.15.70:0/1301781475 shutdown_connections 2023-10-08T07:11:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.128+0000 7f06abc75640 1 -- 172.21.15.70:0/1301781475 wait complete. 2023-10-08T07:11:00.150 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-08T07:11:00.150 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-08-07_12_00 in ~60s... 2023-10-08T07:11:00.150 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2023-10-08T07:11:00.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-08T07:11:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.365+0000 7f52c656e640 1 Processor -- start 2023-10-08T07:11:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.365+0000 7f52c656e640 1 -- start start 2023-10-08T07:11:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.367+0000 7f52c656e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f52c015a820 0x7f52c00aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:11:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.367+0000 7f52c656e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f52c00ca680 0x7f52c00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:11:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.367+0000 7f52c656e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00cab60 con 0x7f52c015ac00 2023-10-08T07:11:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.367+0000 7f52c656e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c00cb960 con 0x7f52c015a820 2023-10-08T07:11:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.367+0000 7f52c656e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c00cbdf0 con 0x7f52c00ca680 2023-10-08T07:11:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.367+0000 7f52c556c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f52c015a820 0x7f52c00aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:11:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.367+0000 7f52c556c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f52c015a820 0x7f52c00aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60132/0 (socket says 172.21.15.70:60132) 2023-10-08T07:11:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.367+0000 7f52c556c640 1 -- 172.21.15.70:0/2271759846 learned_addr learned my addr 172.21.15.70:0/2271759846 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:11:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.367+0000 7f52c5d6d640 1 --1- 172.21.15.70:0/2271759846 >> v1:172.21.15.70:6789/0 conn(0x7f52c015ac00 0x7f52c00c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38752/0 (socket says 172.21.15.70:38752) 2023-10-08T07:11:00.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.368+0000 7f52c556c640 1 -- 172.21.15.70:0/2271759846 >> v2:172.21.15.181:3300/0 conn(0x7f52c00ca680 msgr2=0x7f52c00b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:11:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.368+0000 7f52c4d6b640 1 --2- 172.21.15.70:0/2271759846 >> v2:172.21.15.181:3300/0 conn(0x7f52c00ca680 0x7f52c00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:11:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.368+0000 7f52c556c640 1 --2- 172.21.15.70:0/2271759846 >> v2:172.21.15.181:3300/0 conn(0x7f52c00ca680 0x7f52c00b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.368+0000 7f52c556c640 1 -- 172.21.15.70:0/2271759846 >> v1:172.21.15.70:6789/0 conn(0x7f52c015ac00 legacy=0x7f52c00c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:11:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.368+0000 7f52c556c640 1 -- 172.21.15.70:0/2271759846 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00cb030 con 0x7f52c015a820 2023-10-08T07:11:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.368+0000 7f52bffff640 1 -- 172.21.15.70:0/2271759846 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118227913 0 0) 0x7f52c00cab60 con 0x7f52c015ac00 2023-10-08T07:11:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.369+0000 7f52c556c640 1 --2- 172.21.15.70:0/2271759846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 0x7f52c00aff70 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f52c00718e0 tx=0x7f52b002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820a31074e0b11f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:11:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.369+0000 7f52bffff640 1 -- 172.21.15.70:0/2271759846 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b002ca80 con 0x7f52c015a820 2023-10-08T07:11:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.369+0000 7f52bffff640 1 -- 172.21.15.70:0/2271759846 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b002cbe0 con 0x7f52c015a820 2023-10-08T07:11:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.369+0000 7f52bffff640 1 -- 172.21.15.70:0/2271759846 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b00343f0 con 0x7f52c015a820 2023-10-08T07:11:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.370+0000 7f52c656e640 1 -- 172.21.15.70:0/2271759846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 msgr2=0x7f52c00aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:11:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.370+0000 7f52c656e640 1 --2- 172.21.15.70:0/2271759846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 0x7f52c00aff70 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f52c00718e0 tx=0x7f52b002bee0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.370+0000 7f52c656e640 1 -- 172.21.15.70:0/2271759846 shutdown_connections 2023-10-08T07:11:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.370+0000 7f52c656e640 1 --2- 172.21.15.70:0/2271759846 >> v2:172.21.15.181:3300/0 conn(0x7f52c00ca680 0x7f52c00b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.370+0000 7f52c656e640 1 --2- 172.21.15.70:0/2271759846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 0x7f52c00aff70 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.370+0000 7f52c656e640 1 -- 172.21.15.70:0/2271759846 >> 172.21.15.70:0/2271759846 conn(0x7f52c006c050 msgr2=0x7f52c006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:11:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.370+0000 7f52c656e640 1 -- 172.21.15.70:0/2271759846 shutdown_connections 2023-10-08T07:11:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.370+0000 7f52c656e640 1 -- 172.21.15.70:0/2271759846 wait complete. 2023-10-08T07:11:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.372+0000 7f52c656e640 1 Processor -- start 2023-10-08T07:11:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.372+0000 7f52c656e640 1 -- start start 2023-10-08T07:11:00.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.373+0000 7f52c656e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c00ca680 0x7f52c00ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:11:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.373+0000 7f52c656e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 0x7f52c014f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:11:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.373+0000 7f52c656e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c015ac00 0x7f52c00cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:11:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.373+0000 7f52c656e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c014f860 con 0x7f52c015a820 2023-10-08T07:11:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.373+0000 7f52c656e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00abed0 con 0x7f52c00ca680 2023-10-08T07:11:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.373+0000 7f52c656e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00ac1b0 con 0x7f52c015ac00 2023-10-08T07:11:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.373+0000 7f52c4d6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 0x7f52c014f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:11:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.373+0000 7f52c4d6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 0x7f52c014f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60140/0 (socket says 172.21.15.70:60140) 2023-10-08T07:11:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.373+0000 7f52c4d6b640 1 -- 172.21.15.70:0/3979152203 learned_addr learned my addr 172.21.15.70:0/3979152203 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:11:00.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52c556c640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c00ca680 0x7f52c00ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:11:00.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52c5d6d640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c015ac00 0x7f52c00cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52c4d6b640 1 -- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c015ac00 msgr2=0x7f52c00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52c4d6b640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c015ac00 0x7f52c00cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52c4d6b640 1 -- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c00ca680 msgr2=0x7f52c00ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52c4d6b640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c00ca680 0x7f52c00ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52c4d6b640 1 -- 172.21.15.70:0/3979152203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b0009b30 con 0x7f52c015a820 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52c4d6b640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 0x7f52c014f320 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f52b400c7f0 tx=0x7f52b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52bdffb640 1 -- 172.21.15.70:0/3979152203 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4004220 con 0x7f52c015a820 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52bdffb640 1 -- 172.21.15.70:0/3979152203 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4004380 con 0x7f52c015a820 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52bdffb640 1 -- 172.21.15.70:0/3979152203 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4007840 con 0x7f52c015a820 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.374+0000 7f52c656e640 1 -- 172.21.15.70:0/3979152203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c00ac490 con 0x7f52c015a820 2023-10-08T07:11:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.375+0000 7f52c656e640 1 -- 172.21.15.70:0/3979152203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c00aca50 con 0x7f52c015a820 2023-10-08T07:11:00.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.376+0000 7f52c656e640 1 -- 172.21.15.70:0/3979152203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c0074150 con 0x7f52c015a820 2023-10-08T07:11:00.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.377+0000 7f52bdffb640 1 -- 172.21.15.70:0/3979152203 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 119) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f52b4007bb0 con 0x7f52c015a820 2023-10-08T07:11:00.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.378+0000 7f52bdffb640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f52ac06ad10 0x7f52ac06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:11:00.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.378+0000 7f52bdffb640 1 -- 172.21.15.70:0/3979152203 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 6489+0+0 (secure 0 0 0) 0x7f52b40895b0 con 0x7f52c015a820 2023-10-08T07:11:00.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.378+0000 7f52c556c640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f52ac06ad10 0x7f52ac06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:11:00.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.379+0000 7f52c556c640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f52ac06ad10 0x7f52ac06d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f52b0035000 tx=0x7f52b0030000 comp rx=0 tx=0).ready entity=mgr.8188 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:11:00.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.382+0000 7f52bdffb640 1 -- 172.21.15.70:0/3979152203 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52b40545c0 con 0x7f52c015a820 2023-10-08T07:11:00.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.401+0000 7f52bdffb640 1 -- 172.21.15.70:0/3979152203 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 120) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f52b4054990 con 0x7f52c015a820 2023-10-08T07:11:00.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.600+0000 7f52c656e640 1 -- 172.21.15.70:0/3979152203 --> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52c00cb5f0 con 0x7f52ac06ad10 2023-10-08T07:11:00.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.615+0000 7f52bdffb640 1 -- 172.21.15.70:0/3979152203 <== mgr.8188 v2:172.21.15.181:6832/2613103103 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f52b0034470 con 0x7f52ac06ad10 2023-10-08T07:11:00.617 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:11:00.618 INFO:teuthology.orchestra.run.smithi070.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-08T07:11:00.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 -- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f52ac06ad10 msgr2=0x7f52ac06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:11:00.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f52ac06ad10 0x7f52ac06d1d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f52b0035000 tx=0x7f52b0030000 comp rx=0 tx=0).stop 2023-10-08T07:11:00.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 -- 172.21.15.70:0/3979152203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 msgr2=0x7f52c014f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:11:00.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 0x7f52c014f320 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f52b400c7f0 tx=0x7f52b400ccb0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 -- 172.21.15.70:0/3979152203 shutdown_connections 2023-10-08T07:11:00.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f52ac06ad10 0x7f52ac06d1d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f52c015ac00 0x7f52c00cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52c015a820 0x7f52c014f320 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 --2- 172.21.15.70:0/3979152203 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f52c00ca680 0x7f52c00ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:11:00.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 -- 172.21.15.70:0/3979152203 >> 172.21.15.70:0/3979152203 conn(0x7f52c006c050 msgr2=0x7f52c015b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:11:00.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 -- 172.21.15.70:0/3979152203 shutdown_connections 2023-10-08T07:11:00.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:11:00.618+0000 7f52c656e640 1 -- 172.21.15.70:0/3979152203 wait complete. 2023-10-08T07:11:00.639 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-08T07:11:00.639 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2023-10-08T07:11:00.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:00.640 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:00.691 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:00.691 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:00.691 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:01.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:01.693 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:01.755 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:01.756 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:01.756 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:02.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:02.757 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:02.821 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:02.821 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:02.821 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:03.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:03.823 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:03.888 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:03.889 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:03.889 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:04.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:04.890 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:04.954 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:04.954 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:04.954 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:05.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:05.956 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:06.019 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:06.019 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:06.019 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:06.928 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:11:06.938 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:11:07.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:07.020 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:07.077 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:07.077 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:07.077 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:08.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:08.078 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:08.142 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:08.163 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:08.163 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:09.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:09.165 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:09.229 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:09.229 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:09.230 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:10.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:10.231 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:10.293 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:10.294 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:10.294 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:11.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:11.295 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:11.359 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:11.359 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:11.360 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:12.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:12.360 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:12.423 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:12.424 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:12.424 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:13.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:13.425 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:13.489 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:13.490 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:13.490 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:14.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:14.491 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:14.554 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:14.554 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:14.554 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:15.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:15.556 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:15.619 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:15.620 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:15.620 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:16.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:16.621 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:16.684 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:16.684 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:16.684 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:17.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:17.685 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:17.746 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:17.746 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:17.746 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:18.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:18.748 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:18.811 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:18.811 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:18.811 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:19.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:19.812 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:19.876 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:19.876 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:19.876 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:20.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:20.878 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:20.941 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:20.942 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:20.942 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:21.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:21.943 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:22.005 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:22.006 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:22.006 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:23.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:23.007 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:23.070 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:23.070 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:23.070 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:24.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:24.072 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:24.132 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:24.133 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:24.133 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:25.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:25.134 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:25.199 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:25.199 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:25.199 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:26.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:26.200 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:26.263 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:26.263 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:26.264 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:27.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:27.265 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:27.328 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:27.328 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:27.329 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:28.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:28.330 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:28.392 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:28.393 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:28.393 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:29.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:29.393 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:29.457 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:29.458 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:29.458 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:30.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:30.459 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:30.524 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:30.525 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:30.525 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:31.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:31.526 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:31.587 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:31.587 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:31.587 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:32.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:32.589 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:32.653 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:32.653 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:32.653 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:33.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:33.654 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:33.717 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:33.717 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:33.717 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:34.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:34.718 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:34.781 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:34.781 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:34.781 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:35.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:35.782 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:35.846 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:35.846 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:35.846 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:36.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:36.847 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:36.908 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:36.908 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:36.908 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:37.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:11:37.153 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:11:37.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:37.909 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:37.973 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:37.973 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:37.973 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:38.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:38.974 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:39.037 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:39.037 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:39.037 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:40.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:40.038 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:40.103 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:40.103 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:40.103 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:41.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:41.105 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:41.168 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:41.168 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:41.168 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:42.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:42.169 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:42.233 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:42.233 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:42.233 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:43.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:43.234 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:43.297 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:43.297 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:43.298 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:44.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:44.298 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:44.362 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:44.363 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:44.363 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:45.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:45.364 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:45.428 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:45.429 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:45.429 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:46.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:46.430 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:46.492 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:46.492 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:46.492 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:47.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:47.493 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:47.555 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:47.555 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:47.555 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:48.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:48.556 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:48.620 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:48.620 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:48.620 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:49.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:49.621 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:49.682 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:49.682 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:49.682 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:50.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:50.683 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:50.745 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:50.745 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:50.745 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:51.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:51.746 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:51.810 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:51.810 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:51.810 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:52.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:53.164 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:53.228 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:53.232 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:53.232 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:54.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:54.242 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:54.307 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:54.345 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:54.345 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:55.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:55.394 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:55.457 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:55.474 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:55.474 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:56.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:56.476 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:56.539 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:56.539 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:56.539 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:57.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:57.550 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:57.616 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:57.636 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:57.636 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:58.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:58.642 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:58.706 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:58.711 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:58.711 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:11:59.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:11:59.728 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:11:59.791 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T07:11:59.796 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:11:59.796 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:00.149 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:12:00.148+0000 7fe0168f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:12:00.179 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:12:00.148+0000 7fe0168f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:12:00.179 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:12:00.148+0000 7fe0168f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:12:00.180 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:12:00.148+0000 7fe0168f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:12:00.180 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:12:00.148+0000 7fe0168f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:12:00.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:00.806 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:00.869 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:00.870 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:00.870 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:00.870 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:00.870 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-08-07_12_00_UTC'] 2023-10-08T07:12:00.870 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696749060.1504402, now = 1696749120.8709757, timo = 60 2023-10-08T07:12:01.871 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-10-08-07_12_00 in ~120s... 2023-10-08T07:12:01.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:01.871 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:01.933 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:01.934 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:01.934 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:01.934 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:02.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:02.935 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:02.998 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:02.998 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:02.999 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:02.999 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:04.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:04.000 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:04.064 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:04.065 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:04.065 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:04.065 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:05.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:05.066 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:05.128 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:05.129 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:05.130 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:05.130 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:06.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:06.131 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:06.192 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:06.194 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:06.194 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:06.194 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:07.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:07.195 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:07.256 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:07.257 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:07.257 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:07.257 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:07.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:12:07.374 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:12:08.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:08.258 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:08.322 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:08.323 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:08.323 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:08.323 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:09.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:09.324 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:09.383 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:09.385 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:09.385 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:09.385 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:10.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:10.386 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:10.450 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:10.451 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:10.451 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:10.451 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:11.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:11.452 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:11.514 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:11.515 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:11.515 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:11.515 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:12.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:12.517 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:12.576 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:12.577 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:12.577 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:12.577 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:13.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:13.578 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:13.641 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:13.642 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:13.642 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:13.642 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:14.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:14.643 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:14.707 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:14.708 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:14.708 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:14.708 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:15.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:15.709 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:15.773 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:15.774 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:15.774 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:15.775 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:16.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:16.776 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:16.839 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:16.841 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:16.841 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:16.841 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:17.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:17.842 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:17.904 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:17.906 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:17.906 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:17.906 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:18.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:18.907 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:18.970 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:18.972 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:18.972 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:18.972 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:19.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:19.973 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:20.035 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:20.036 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:20.036 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:20.036 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:21.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:21.055 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:21.119 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:21.130 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:21.130 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:21.130 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:22.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:22.131 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:22.194 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:22.195 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:22.196 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:22.196 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:23.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:23.197 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:23.259 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:23.260 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:23.261 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:23.261 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:24.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:24.261 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:24.322 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:24.323 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:24.323 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:24.323 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:25.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:25.324 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:25.386 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:25.387 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:25.387 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:25.388 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:26.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:26.389 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:26.448 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:26.449 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:26.449 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:26.450 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:27.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:27.450 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:27.512 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:27.514 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:27.514 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:27.514 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:28.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:28.516 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:28.581 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:28.582 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:28.582 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:28.582 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:29.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:29.583 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:29.645 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:29.647 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:29.647 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:29.647 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:30.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:30.648 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:30.708 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:30.709 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:30.709 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:30.709 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:31.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:31.711 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:31.773 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:31.774 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:31.774 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:31.775 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:32.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:32.775 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:32.837 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:32.838 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:32.838 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:32.838 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:33.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:33.840 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:33.900 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:33.901 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:33.902 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:33.902 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:34.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:34.903 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:34.964 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:34.965 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:34.965 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:34.965 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:35.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:35.966 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:36.028 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:36.029 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:36.029 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:36.029 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:37.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:37.030 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:37.092 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:37.093 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:37.093 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:37.093 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:37.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:12:37.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:12:38.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:38.094 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:38.156 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:38.157 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:38.157 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:38.157 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:39.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:39.159 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:39.216 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:39.217 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:39.217 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:39.217 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:40.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:40.218 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:40.276 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:40.277 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:40.277 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:40.277 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:41.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:41.278 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:41.342 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:41.343 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:41.343 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:41.343 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:42.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:42.344 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:42.407 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:42.408 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:42.408 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:42.408 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:43.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:43.410 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:43.472 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:43.473 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:43.474 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:43.474 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:44.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:44.475 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:44.537 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:44.539 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:44.539 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:44.539 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:45.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:45.540 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:45.602 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:45.603 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:45.603 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:45.603 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:46.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:46.604 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:46.667 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:46.668 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:46.669 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:46.669 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:47.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:47.670 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:47.739 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:47.743 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:47.744 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:47.744 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:48.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:48.745 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:48.807 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:48.808 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:48.808 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:48.808 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:49.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:49.809 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:49.872 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:49.873 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:49.873 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:49.873 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:50.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:50.875 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:50.936 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:50.937 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:50.937 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:50.938 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:51.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:51.938 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:52.003 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:52.005 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:52.005 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:52.005 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:53.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:53.006 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:53.069 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:53.071 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:53.071 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:53.071 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:54.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:54.072 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:54.134 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:54.135 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:54.135 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:54.135 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:55.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:55.136 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:55.199 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:55.200 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:55.200 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:55.200 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:56.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:56.201 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:56.263 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:56.265 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:56.265 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:56.265 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:57.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:57.266 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:57.330 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:57.331 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:57.331 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:57.331 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:58.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:58.332 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:58.394 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:58.395 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:58.395 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:58.395 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:12:59.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:12:59.396 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:12:59.457 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_12_00_UTC 2023-10-08T07:12:59.458 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:12:59.458 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T07:12:59.458 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T07:13:00.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:00.459 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:13:00.519 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_13_00_UTC 2023-10-08T07:13:00.520 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-07_13_00_UTC'} 2023-10-08T07:13:00.520 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-08-07_13_00_UTC'} 2023-10-08T07:13:00.521 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-10-08-07_12_00_UTC'} 2023-10-08T07:13:00.521 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-10-08-07_12_00_UTC'] 2023-10-08T07:13:00.521 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696749060.1504402, now = 1696749180.5212073, timo = 120 2023-10-08T07:13:01.522 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-08T07:13:01.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.749+0000 7fb62b59e640 1 Processor -- start 2023-10-08T07:13:01.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.749+0000 7fb62b59e640 1 -- start start 2023-10-08T07:13:01.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.751+0000 7fb62b59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb62c1599d0 0x7fb62c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:01.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.751+0000 7fb62b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb62c0cb5e0 0x7fb62c0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:01.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.751+0000 7fb62b59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c159db0 con 0x7fb62c0b3c50 2023-10-08T07:13:01.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.751+0000 7fb62b59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c071fe0 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.751+0000 7fb62b59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c0b44c0 con 0x7fb62c1599d0 2023-10-08T07:13:01.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.751+0000 7fb62ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb62c0cb5e0 0x7fb62c0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:01.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.751+0000 7fb62ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb62c0cb5e0 0x7fb62c0cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56450/0 (socket says 172.21.15.70:56450) 2023-10-08T07:13:01.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.751+0000 7fb62ad9d640 1 -- 172.21.15.70:0/2009856141 learned_addr learned my addr 172.21.15.70:0/2009856141 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:01.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.752+0000 7fb62ad9d640 1 -- 172.21.15.70:0/2009856141 >> v2:172.21.15.181:3300/0 conn(0x7fb62c1599d0 msgr2=0x7fb62c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:01.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.752+0000 7fb62a59c640 1 --2- 172.21.15.70:0/2009856141 >> v2:172.21.15.181:3300/0 conn(0x7fb62c1599d0 0x7fb62c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:01.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.752+0000 7fb62ad9d640 1 --2- 172.21.15.70:0/2009856141 >> v2:172.21.15.181:3300/0 conn(0x7fb62c1599d0 0x7fb62c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:01.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.752+0000 7fb62ad9d640 1 -- 172.21.15.70:0/2009856141 >> v1:172.21.15.181:6789/0 conn(0x7fb62c0b3c50 legacy=0x7fb62c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:01.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.752+0000 7fb62ad9d640 1 -- 172.21.15.70:0/2009856141 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0713d0 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.752+0000 7fb62a59c640 1 --2- 172.21.15.70:0/2009856141 >> v2:172.21.15.181:3300/0 conn(0x7fb62c1599d0 0x7fb62c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:01.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.753+0000 7fb62ad9d640 1 --2- 172.21.15.70:0/2009856141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 0x7fb62c0cc0c0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fb62c06e150 tx=0x7fb62402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45b1cfe223017cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:01.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.753+0000 7fb629d9b640 1 -- 172.21.15.70:0/2009856141 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62402ca80 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.753+0000 7fb629d9b640 1 -- 172.21.15.70:0/2009856141 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62402cbe0 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.753+0000 7fb629d9b640 1 -- 172.21.15.70:0/2009856141 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6240343f0 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.754+0000 7fb62b59e640 1 -- 172.21.15.70:0/2009856141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 msgr2=0x7fb62c0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:01.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.754+0000 7fb62b59e640 1 --2- 172.21.15.70:0/2009856141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 0x7fb62c0cc0c0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fb62c06e150 tx=0x7fb62402bee0 comp rx=0 tx=0).stop 2023-10-08T07:13:01.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.754+0000 7fb62b59e640 1 -- 172.21.15.70:0/2009856141 shutdown_connections 2023-10-08T07:13:01.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.754+0000 7fb62b59e640 1 --2- 172.21.15.70:0/2009856141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 0x7fb62c0cc0c0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:01.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.754+0000 7fb62b59e640 1 --2- 172.21.15.70:0/2009856141 >> v2:172.21.15.181:3300/0 conn(0x7fb62c1599d0 0x7fb62c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:01.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.754+0000 7fb62b59e640 1 -- 172.21.15.70:0/2009856141 >> 172.21.15.70:0/2009856141 conn(0x7fb62c06c050 msgr2=0x7fb62c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:01.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.754+0000 7fb62b59e640 1 -- 172.21.15.70:0/2009856141 shutdown_connections 2023-10-08T07:13:01.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.754+0000 7fb62b59e640 1 -- 172.21.15.70:0/2009856141 wait complete. 2023-10-08T07:13:01.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.756+0000 7fb62b59e640 1 Processor -- start 2023-10-08T07:13:01.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.756+0000 7fb62b59e640 1 -- start start 2023-10-08T07:13:01.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.757+0000 7fb62b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62c0b3c50 0x7fb62c1562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:01.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.757+0000 7fb62b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 0x7fb62c158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:01.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.757+0000 7fb62b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62c1599d0 0x7fb62c14dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:01.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.757+0000 7fb62b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b1010 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.757+0000 7fb62b59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b12f0 con 0x7fb62c0b3c50 2023-10-08T07:13:01.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.757+0000 7fb62b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c0b15d0 con 0x7fb62c1599d0 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.757+0000 7fb623fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 0x7fb62c158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb623fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 0x7fb62c158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56462/0 (socket says 172.21.15.70:56462) 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb623fff640 1 -- 172.21.15.70:0/302899324 learned_addr learned my addr 172.21.15.70:0/302899324 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb62a59c640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62c0b3c50 0x7fb62c1562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb62ad9d640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62c1599d0 0x7fb62c14dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb623fff640 1 -- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62c1599d0 msgr2=0x7fb62c14dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb623fff640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62c1599d0 0x7fb62c14dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb623fff640 1 -- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62c0b3c50 msgr2=0x7fb62c1562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb623fff640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62c0b3c50 0x7fb62c1562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb623fff640 1 -- 172.21.15.70:0/302899324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624009b30 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb62ad9d640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62c1599d0 0x7fb62c14dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:01.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb62a59c640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62c0b3c50 0x7fb62c1562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:13:01.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb623fff640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 0x7fb62c158cb0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fb61400c430 tx=0x7fb61400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:01.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.758+0000 7fb622ffd640 1 -- 172.21.15.70:0/302899324 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614011790 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.759+0000 7fb622ffd640 1 -- 172.21.15.70:0/302899324 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6140118f0 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.759+0000 7fb622ffd640 1 -- 172.21.15.70:0/302899324 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614011ce0 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.759+0000 7fb62b59e640 1 -- 172.21.15.70:0/302899324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c0b18b0 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.759+0000 7fb62b59e640 1 -- 172.21.15.70:0/302899324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c0b1e70 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.760+0000 7fb62b59e640 1 -- 172.21.15.70:0/302899324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c074150 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.761+0000 7fb622ffd640 1 -- 172.21.15.70:0/302899324 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 123) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fb61401c460 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.762+0000 7fb622ffd640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fb60c06afe0 0x7fb60c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:01.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.762+0000 7fb622ffd640 1 -- 172.21.15.70:0/302899324 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fb614089b70 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.762+0000 7fb62a59c640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fb60c06afe0 0x7fb60c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:01.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.764+0000 7fb62a59c640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fb60c06afe0 0x7fb60c06d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00a990 tx=0x7fb61c008040 comp rx=0 tx=0).ready entity=mgr.8188 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:01.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.766+0000 7fb622ffd640 1 -- 172.21.15.70:0/302899324 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb614054730 con 0x7fb62c0cb5e0 2023-10-08T07:13:01.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:01.977+0000 7fb62b59e640 1 -- 172.21.15.70:0/302899324 --> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb62c071990 con 0x7fb60c06afe0 2023-10-08T07:13:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.134+0000 7fb622ffd640 1 -- 172.21.15.70:0/302899324 <== mgr.8188 v2:172.21.15.181:6832/2613103103 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb61c00f860 con 0x7fb60c06afe0 2023-10-08T07:13:02.135 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path snap_test_dir1 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.137+0000 7fb62b59e640 1 -- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fb60c06afe0 msgr2=0x7fb60c06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.137+0000 7fb62b59e640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fb60c06afe0 0x7fb60c06d4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00a990 tx=0x7fb61c008040 comp rx=0 tx=0).stop 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.137+0000 7fb62b59e640 1 -- 172.21.15.70:0/302899324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 msgr2=0x7fb62c158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.137+0000 7fb62b59e640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 0x7fb62c158cb0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fb61400c430 tx=0x7fb61400c8f0 comp rx=0 tx=0).stop 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.137+0000 7fb62b59e640 1 -- 172.21.15.70:0/302899324 shutdown_connections 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.137+0000 7fb62b59e640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fb60c06afe0 0x7fb60c06d4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.137+0000 7fb62b59e640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb62c1599d0 0x7fb62c14dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.137+0000 7fb62b59e640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c0cb5e0 0x7fb62c158cb0 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.137+0000 7fb62b59e640 1 --2- 172.21.15.70:0/302899324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb62c0b3c50 0x7fb62c1562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.137+0000 7fb62b59e640 1 -- 172.21.15.70:0/302899324 >> 172.21.15.70:0/302899324 conn(0x7fb62c06c050 msgr2=0x7fb62c15ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.138+0000 7fb62b59e640 1 -- 172.21.15.70:0/302899324 shutdown_connections 2023-10-08T07:13:02.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.138+0000 7fb62b59e640 1 -- 172.21.15.70:0/302899324 wait complete. 2023-10-08T07:13:02.159 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-08T07:13:02.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:02.159 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:13:02.212 INFO:teuthology.orchestra.run.smithi070.stdout:scheduled-2023-10-08-07_13_00_UTC 2023-10-08T07:13:02.213 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-07_13_00_UTC 2023-10-08T07:13:02.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:02.213 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-07_13_00_UTC) 2023-10-08T07:13:02.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:02.276 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T07:13:02.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T07:13:02.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.542+0000 7fd875852640 1 Processor -- start 2023-10-08T07:13:02.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.542+0000 7fd875852640 1 -- start start 2023-10-08T07:13:02.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.543+0000 7fd875852640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd87015aab0 0x7fd870070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.543+0000 7fd875852640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8700caef0 0x7fd8700aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.543+0000 7fd875852640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87015ae90 con 0x7fd8700cb350 2023-10-08T07:13:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.544+0000 7fd875852640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8700cbdb0 con 0x7fd8700caef0 2023-10-08T07:13:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.544+0000 7fd875852640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8700cb7f0 con 0x7fd87015aab0 2023-10-08T07:13:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.544+0000 7fd86ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8700caef0 0x7fd8700aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.544+0000 7fd86ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8700caef0 0x7fd8700aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56476/0 (socket says 172.21.15.70:56476) 2023-10-08T07:13:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.544+0000 7fd86ffff640 1 -- 172.21.15.70:0/2502823522 learned_addr learned my addr 172.21.15.70:0/2502823522 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.544+0000 7fd86ffff640 1 -- 172.21.15.70:0/2502823522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd87015aab0 msgr2=0x7fd870070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:02.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.544+0000 7fd874850640 1 --2- 172.21.15.70:0/2502823522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd87015aab0 0x7fd870070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:02.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.544+0000 7fd86ffff640 1 --2- 172.21.15.70:0/2502823522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd87015aab0 0x7fd870070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:02.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.544+0000 7fd86ffff640 1 -- 172.21.15.70:0/2502823522 >> v1:172.21.15.181:6789/0 conn(0x7fd8700cb350 legacy=0x7fd8700c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:02.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.544+0000 7fd86ffff640 1 -- 172.21.15.70:0/2502823522 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8700cbad0 con 0x7fd8700caef0 2023-10-08T07:13:02.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.546+0000 7fd86ffff640 1 --2- 172.21.15.70:0/2502823522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8700caef0 0x7fd8700aff60 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009870 tx=0x7fd85c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ab39f3ea28f54a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:02.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.546+0000 7fd86f7fe640 1 -- 172.21.15.70:0/2502823522 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0317c0 con 0x7fd8700caef0 2023-10-08T07:13:02.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.546+0000 7fd86f7fe640 1 -- 172.21.15.70:0/2502823522 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c031920 con 0x7fd8700caef0 2023-10-08T07:13:02.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.546+0000 7fd86f7fe640 1 -- 172.21.15.70:0/2502823522 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c031cb0 con 0x7fd8700caef0 2023-10-08T07:13:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.547+0000 7fd875852640 1 -- 172.21.15.70:0/2502823522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8700caef0 msgr2=0x7fd8700aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.547+0000 7fd875852640 1 --2- 172.21.15.70:0/2502823522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8700caef0 0x7fd8700aff60 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fd85c009870 tx=0x7fd85c02b860 comp rx=0 tx=0).stop 2023-10-08T07:13:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.547+0000 7fd875852640 1 -- 172.21.15.70:0/2502823522 shutdown_connections 2023-10-08T07:13:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.547+0000 7fd875852640 1 --2- 172.21.15.70:0/2502823522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8700caef0 0x7fd8700aff60 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.547+0000 7fd875852640 1 --2- 172.21.15.70:0/2502823522 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd87015aab0 0x7fd870070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.547+0000 7fd875852640 1 -- 172.21.15.70:0/2502823522 >> 172.21.15.70:0/2502823522 conn(0x7fd87006c010 msgr2=0x7fd87006c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.547+0000 7fd875852640 1 -- 172.21.15.70:0/2502823522 shutdown_connections 2023-10-08T07:13:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.547+0000 7fd875852640 1 -- 172.21.15.70:0/2502823522 wait complete. 2023-10-08T07:13:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.549+0000 7fd875852640 1 Processor -- start 2023-10-08T07:13:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.549+0000 7fd875852640 1 -- start start 2023-10-08T07:13:02.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875852640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8700caef0 0x7fd8701f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:02.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875852640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8700cb350 0x7fd8700af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:02.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875852640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd87015aab0 0x7fd870155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:02.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875852640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8700afb60 con 0x7fd87015aab0 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875852640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8700ce930 con 0x7fd8700caef0 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875852640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8700ce000 con 0x7fd8700cb350 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875051640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd87015aab0 0x7fd870155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875051640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd87015aab0 0x7fd870155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56492/0 (socket says 172.21.15.70:56492) 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875051640 1 -- 172.21.15.70:0/1536946630 learned_addr learned my addr 172.21.15.70:0/1536946630 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875051640 1 -- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8700cb350 msgr2=0x7fd8700af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875051640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8700cb350 0x7fd8700af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875051640 1 -- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8700caef0 msgr2=0x7fd8701f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875051640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8700caef0 0x7fd8701f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.550+0000 7fd875051640 1 -- 172.21.15.70:0/1536946630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0094d0 con 0x7fd87015aab0 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.551+0000 7fd875051640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd87015aab0 0x7fd870155a60 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fd86800c440 tx=0x7fd86800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.551+0000 7fd86d7fa640 1 -- 172.21.15.70:0/1536946630 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868011790 con 0x7fd87015aab0 2023-10-08T07:13:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.551+0000 7fd86d7fa640 1 -- 172.21.15.70:0/1536946630 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8680118f0 con 0x7fd87015aab0 2023-10-08T07:13:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.551+0000 7fd86d7fa640 1 -- 172.21.15.70:0/1536946630 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868011ca0 con 0x7fd87015aab0 2023-10-08T07:13:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.551+0000 7fd875852640 1 -- 172.21.15.70:0/1536946630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8700ce2e0 con 0x7fd87015aab0 2023-10-08T07:13:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.551+0000 7fd875852640 1 -- 172.21.15.70:0/1536946630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8700af850 con 0x7fd87015aab0 2023-10-08T07:13:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.553+0000 7fd875852640 1 -- 172.21.15.70:0/1536946630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd870074150 con 0x7fd87015aab0 2023-10-08T07:13:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.553+0000 7fd86d7fa640 1 -- 172.21.15.70:0/1536946630 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 123) v1 ==== 89360+0+0 (secure 0 0 0) 0x7fd86801c460 con 0x7fd87015aab0 2023-10-08T07:13:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.554+0000 7fd86d7fa640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fd85406b010 0x7fd85406d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.554+0000 7fd86d7fa640 1 -- 172.21.15.70:0/1536946630 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6576+0+0 (secure 0 0 0) 0x7fd868089930 con 0x7fd87015aab0 2023-10-08T07:13:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.555+0000 7fd874850640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fd85406b010 0x7fd85406d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:02.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.556+0000 7fd874850640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fd85406b010 0x7fd85406d4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd864007dc0 tx=0x7fd8640072f0 comp rx=0 tx=0).ready entity=mgr.8188 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:02.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.558+0000 7fd86d7fa640 1 -- 172.21.15.70:0/1536946630 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd868054600 con 0x7fd87015aab0 2023-10-08T07:13:02.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:02.796+0000 7fd875852640 1 -- 172.21.15.70:0/1536946630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fd8700cea70 con 0x7fd87015aab0 2023-10-08T07:13:03.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.286+0000 7fd86d7fa640 1 -- 172.21.15.70:0/1536946630 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v124) v1 ==== 93+0+0 (secure 0 0 0) 0x7fd868057c20 con 0x7fd87015aab0 2023-10-08T07:13:03.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.291+0000 7fd875852640 1 -- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fd85406b010 msgr2=0x7fd85406d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:03.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.291+0000 7fd875852640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fd85406b010 0x7fd85406d4d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd864007dc0 tx=0x7fd8640072f0 comp rx=0 tx=0).stop 2023-10-08T07:13:03.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.291+0000 7fd875852640 1 -- 172.21.15.70:0/1536946630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd87015aab0 msgr2=0x7fd870155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:03.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.291+0000 7fd875852640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd87015aab0 0x7fd870155a60 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fd86800c440 tx=0x7fd86800c900 comp rx=0 tx=0).stop 2023-10-08T07:13:03.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.292+0000 7fd875852640 1 -- 172.21.15.70:0/1536946630 shutdown_connections 2023-10-08T07:13:03.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.292+0000 7fd875852640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7fd85406b010 0x7fd85406d4d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:03.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.292+0000 7fd875852640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd87015aab0 0x7fd870155a60 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:03.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.292+0000 7fd875852640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd8700cb350 0x7fd8700af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:03.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.292+0000 7fd875852640 1 --2- 172.21.15.70:0/1536946630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd8700caef0 0x7fd8701f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:03.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.292+0000 7fd875852640 1 -- 172.21.15.70:0/1536946630 >> 172.21.15.70:0/1536946630 conn(0x7fd87006c010 msgr2=0x7fd8700ca5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:03.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.295+0000 7fd875852640 1 -- 172.21.15.70:0/1536946630 shutdown_connections 2023-10-08T07:13:03.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.296+0000 7fd875852640 1 -- 172.21.15.70:0/1536946630 wait complete. 2023-10-08T07:13:03.318 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T07:13:03.318 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:13:03.410 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:13:03.503 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T07:13:03.504 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:13:03.504 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:13:03.553 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:13:03.553 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:13:03.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:03.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.836+0000 7f21cf4d7640 1 Processor -- start 2023-10-08T07:13:03.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.836+0000 7f21cf4d7640 1 -- start start 2023-10-08T07:13:03.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.838+0000 7f21cf4d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c81597d0 0x7f21c806ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:03.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.838+0000 7f21cf4d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c80adfc0 con 0x7f21c80cb190 2023-10-08T07:13:03.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.838+0000 7f21cf4d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c806d380 con 0x7f21c8157000 2023-10-08T07:13:03.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.838+0000 7f21cf4d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8159cb0 con 0x7f21c81597d0 2023-10-08T07:13:03.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21cda74640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f21c80cb190 0x7f21c815d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51500/0 (socket says 172.21.15.70:51500) 2023-10-08T07:13:03.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21cda74640 1 -- 172.21.15.70:0/2419893514 learned_addr learned my addr 172.21.15.70:0/2419893514 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:03.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21cca72640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c81597d0 0x7f21c806ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:03.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360825936 0 0) 0x7f21c80adfc0 con 0x7f21c80cb190 2023-10-08T07:13:03.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21b4004630 con 0x7f21c80cb190 2023-10-08T07:13:03.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2129322535 0 0) 0x7f21b4004630 con 0x7f21c80cb190 2023-10-08T07:13:03.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c81597d0 msgr2=0x7f21c806ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:03.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21bffff640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c81597d0 0x7f21c806ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:03.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 >> v1:172.21.15.181:6789/0 conn(0x7f21c8157000 legacy=0x7f21c80cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:03.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c8159f90 con 0x7f21c80cb190 2023-10-08T07:13:03.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.839+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f21c00019f0 con 0x7f21c80cb190 2023-10-08T07:13:03.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 >> v1:172.21.15.70:6789/0 conn(0x7f21c80cb190 legacy=0x7f21c815d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:03.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21bffff640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b4004ab0 0x7f21b4004e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:03.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21bffff640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21b40058d0 0x7f21b4005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:03.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21bffff640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c81597d0 0x7f21b4006290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:03.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8159cb0 con 0x7f21b40058d0 2023-10-08T07:13:03.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b400cb40 con 0x7f21b4004ab0 2023-10-08T07:13:03.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b4003470 con 0x7f21c81597d0 2023-10-08T07:13:03.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21cca72640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21b40058d0 0x7f21b4005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:03.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21cd273640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b4004ab0 0x7f21b4004e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:03.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21cda74640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c81597d0 0x7f21b4006290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:03.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21cca72640 1 -- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c81597d0 msgr2=0x7f21b4006290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:03.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21cca72640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c81597d0 0x7f21b4006290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:03.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21cca72640 1 -- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b4004ab0 msgr2=0x7f21b4004e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:03.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21cca72640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b4004ab0 0x7f21b4004e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:03.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21cca72640 1 -- 172.21.15.70:0/2419893514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b4004630 con 0x7f21b40058d0 2023-10-08T07:13:03.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.840+0000 7f21cda74640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c81597d0 0x7f21b4006290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:03.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.841+0000 7f21cca72640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21b40058d0 0x7f21b4005d50 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009a90 tx=0x7f21b4006900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98972809f53118f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:03.847 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:03.842+0000 7f1cea779040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:13:03.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.842+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b802c650 con 0x7f21b40058d0 2023-10-08T07:13:03.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.842+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b802c7b0 con 0x7f21b40058d0 2023-10-08T07:13:03.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.842+0000 7f21bffff640 1 -- 172.21.15.70:0/2419893514 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b802cb40 con 0x7f21b40058d0 2023-10-08T07:13:03.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.842+0000 7f21cf4d7640 1 -- 172.21.15.70:0/2419893514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21b40058d0 msgr2=0x7f21b4005d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.842+0000 7f21cf4d7640 1 --2- 172.21.15.70:0/2419893514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21b40058d0 0x7f21b4005d50 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009a90 tx=0x7f21b4006900 comp rx=0 tx=0).stop 2023-10-08T07:13:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.842+0000 7f21cda74640 1 -- 172.21.15.70:0/2419893514 reap_dead start 2023-10-08T07:13:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.843+0000 7f21cf4d7640 1 -- 172.21.15.70:0/2419893514 shutdown_connections 2023-10-08T07:13:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.843+0000 7f21cf4d7640 1 -- 172.21.15.70:0/2419893514 >> 172.21.15.70:0/2419893514 conn(0x7f21c806a380 msgr2=0x7f21c80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.843+0000 7f21cf4d7640 1 -- 172.21.15.70:0/2419893514 shutdown_connections 2023-10-08T07:13:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.843+0000 7f21cf4d7640 1 -- 172.21.15.70:0/2419893514 wait complete. 2023-10-08T07:13:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.844+0000 7f21cf4d7640 1 Processor -- start 2023-10-08T07:13:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.844+0000 7f21cf4d7640 1 -- start start 2023-10-08T07:13:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.845+0000 7f21cf4d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b40058d0 0x7f21c80b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.845+0000 7f21cf4d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c8144d30 0x7f21c81451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.845+0000 7f21cf4d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c81e3f70 0x7f21c81e4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.845+0000 7f21cf4d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c80b14a0 con 0x7f21c81e3f70 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.845+0000 7f21cf4d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c80b16f0 con 0x7f21b40058d0 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.845+0000 7f21cf4d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c80b19d0 con 0x7f21c8144d30 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cda74640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c81e3f70 0x7f21c81e4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cda74640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c81e3f70 0x7f21c81e4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42752/0 (socket says 172.21.15.70:42752) 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cda74640 1 -- 172.21.15.70:0/631140894 learned_addr learned my addr 172.21.15.70:0/631140894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cd273640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b40058d0 0x7f21c80b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cca72640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c8144d30 0x7f21c81451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cda74640 1 -- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c8144d30 msgr2=0x7f21c81451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cda74640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c8144d30 0x7f21c81451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cda74640 1 -- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b40058d0 msgr2=0x7f21c80b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cda74640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b40058d0 0x7f21c80b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cda74640 1 -- 172.21.15.70:0/631140894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c80b1cb0 con 0x7f21c81e3f70 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.846+0000 7f21cd273640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b40058d0 0x7f21c80b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.847+0000 7f21cda74640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c81e3f70 0x7f21c81e4420 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f21c80c3ef0 tx=0x7f21c000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.847+0000 7f21bdffb640 1 -- 172.21.15.70:0/631140894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0004d60 con 0x7f21c81e3f70 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.847+0000 7f21bdffb640 1 -- 172.21.15.70:0/631140894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c000bb80 con 0x7f21c81e3f70 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.847+0000 7f21bdffb640 1 -- 172.21.15.70:0/631140894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0007110 con 0x7f21c81e3f70 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.847+0000 7f21cf4d7640 1 -- 172.21.15.70:0/631140894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c80b1f90 con 0x7f21c81e3f70 2023-10-08T07:13:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.847+0000 7f21cf4d7640 1 -- 172.21.15.70:0/631140894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c81e4960 con 0x7f21c81e3f70 2023-10-08T07:13:03.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.848+0000 7f21cf4d7640 1 -- 172.21.15.70:0/631140894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c815a530 con 0x7f21c81e3f70 2023-10-08T07:13:03.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.849+0000 7f21bdffb640 1 -- 172.21.15.70:0/631140894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 124) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f21c0007270 con 0x7f21c81e3f70 2023-10-08T07:13:03.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.851+0000 7f21bdffb640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f21ac06af90 0x7f21ac06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:03.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.851+0000 7f21bdffb640 1 -- 172.21.15.70:0/631140894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6664+0+0 (secure 0 0 0) 0x7f21c008a0d0 con 0x7f21c81e3f70 2023-10-08T07:13:03.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.851+0000 7f21cd273640 1 -- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f21ac06af90 msgr2=0x7f21ac06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:03.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.851+0000 7f21cd273640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f21ac06af90 0x7f21ac06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:13:03.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:03.856+0000 7f21bdffb640 1 -- 172.21.15.70:0/631140894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21c815a530 con 0x7f21c81e3f70 2023-10-08T07:13:03.873 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:03.871+0000 7f429de32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:13:03.896 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:03.896+0000 7f1cea779040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:13:03.929 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:03.927+0000 7f429de32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:13:04.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.047+0000 7f21cf4d7640 1 -- 172.21.15.70:0/631140894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21c815a530 con 0x7f21c81e3f70 2023-10-08T07:13:04.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.049+0000 7f21bdffb640 1 -- 172.21.15.70:0/631140894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v220) v1 ==== 74+0+17698 (secure 0 0 0) 0x7f21c815a530 con 0x7f21c81e3f70 2023-10-08T07:13:04.051 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:04.051 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":220,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:03.271506+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:10:41.917258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-08T07:10:43.125413+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"220","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":4,"snap_epoch":220,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.28","mappings":[{"from":1,"to":0}]},{"pgid":"26.36","mappings":[{"from":3,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/294214378":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/3667065295":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/3162325425":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6833/2638163552":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6832/2638163552":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/785931869":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/265797938":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/39996703":"2023-10-09T07:10:58.332667+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":26,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[{"pool":26,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:04.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.051+0000 7f21cf4d7640 1 -- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f21ac06af90 msgr2=0x7f21ac06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:13:04.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.051+0000 7f21cf4d7640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f21ac06af90 0x7f21ac06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.051+0000 7f21cf4d7640 1 -- 172.21.15.70:0/631140894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c81e3f70 msgr2=0x7f21c81e4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.051+0000 7f21cf4d7640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c81e3f70 0x7f21c81e4420 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f21c80c3ef0 tx=0x7f21c000b4a0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.051+0000 7f21cf4d7640 1 -- 172.21.15.70:0/631140894 shutdown_connections 2023-10-08T07:13:04.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.051+0000 7f21cf4d7640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f21ac06af90 0x7f21ac06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.051+0000 7f21cf4d7640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21c81e3f70 0x7f21c81e4420 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.051+0000 7f21cf4d7640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21c8144d30 0x7f21c81451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.051+0000 7f21cf4d7640 1 --2- 172.21.15.70:0/631140894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21b40058d0 0x7f21c80b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.051+0000 7f21cf4d7640 1 -- 172.21.15.70:0/631140894 >> 172.21.15.70:0/631140894 conn(0x7f21c806a380 msgr2=0x7f21c80c4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:04.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.052+0000 7f21cf4d7640 1 -- 172.21.15.70:0/631140894 shutdown_connections 2023-10-08T07:13:04.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.052+0000 7f21cf4d7640 1 -- 172.21.15.70:0/631140894 wait complete. 2023-10-08T07:13:04.071 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T07:10:41.917258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '215', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T07:10:43.125413+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '220', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 220, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:13:04.072 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:13:04.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.281+0000 7f655eccf640 1 Processor -- start 2023-10-08T07:13:04.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.281+0000 7f655eccf640 1 -- start start 2023-10-08T07:13:04.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.282+0000 7f655eccf640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580cd930 0x7f65580cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.282+0000 7f655eccf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65580d0d30 0x7f65580d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.282+0000 7f655eccf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65580b7e70 con 0x7f655815cca0 2023-10-08T07:13:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.282+0000 7f655eccf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655815d080 con 0x7f65580d0d30 2023-10-08T07:13:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.282+0000 7f655eccf640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65580b7260 con 0x7f65580cd930 2023-10-08T07:13:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.283+0000 7f655ca6b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f655815cca0 0x7f65580b6790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51504/0 (socket says 172.21.15.70:51504) 2023-10-08T07:13:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.283+0000 7f655ca6b640 1 -- 172.21.15.70:0/2295828093 learned_addr learned my addr 172.21.15.70:0/2295828093 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:04.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.283+0000 7f655d26c640 1 --2- 172.21.15.70:0/2295828093 >> v2:172.21.15.70:3300/0 conn(0x7f65580d0d30 0x7f65580d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:04.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.283+0000 7f6557fff640 1 --2- 172.21.15.70:0/2295828093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580cd930 0x7f65580cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:04.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.283+0000 7f65577fe640 1 -- 172.21.15.70:0/2295828093 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056001976 0 0) 0x7f65580b7e70 con 0x7f655815cca0 2023-10-08T07:13:04.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.283+0000 7f65577fe640 1 -- 172.21.15.70:0/2295828093 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6548003520 con 0x7f655815cca0 2023-10-08T07:13:04.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.283+0000 7f655d26c640 1 -- 172.21.15.70:0/2295828093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580cd930 msgr2=0x7f65580cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:04.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.283+0000 7f655d26c640 1 --2- 172.21.15.70:0/2295828093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580cd930 0x7f65580cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.283+0000 7f655d26c640 1 -- 172.21.15.70:0/2295828093 >> v1:172.21.15.70:6789/0 conn(0x7f655815cca0 legacy=0x7f65580b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.283+0000 7f655d26c640 1 -- 172.21.15.70:0/2295828093 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65580b7540 con 0x7f65580d0d30 2023-10-08T07:13:04.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.284+0000 7f655d26c640 1 --2- 172.21.15.70:0/2295828093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580d0d30 0x7f65580d11c0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f6550009d90 tx=0x7f655002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7343989ce8d5b11 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:04.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.285+0000 7f65577fe640 1 -- 172.21.15.70:0/2295828093 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655002cce0 con 0x7f65580d0d30 2023-10-08T07:13:04.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.285+0000 7f65577fe640 1 -- 172.21.15.70:0/2295828093 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655002ce40 con 0x7f65580d0d30 2023-10-08T07:13:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.285+0000 7f65577fe640 1 -- 172.21.15.70:0/2295828093 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550034660 con 0x7f65580d0d30 2023-10-08T07:13:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.286+0000 7f655eccf640 1 -- 172.21.15.70:0/2295828093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580d0d30 msgr2=0x7f65580d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.286+0000 7f655eccf640 1 --2- 172.21.15.70:0/2295828093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580d0d30 0x7f65580d11c0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f6550009d90 tx=0x7f655002bf30 comp rx=0 tx=0).stop 2023-10-08T07:13:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.286+0000 7f655eccf640 1 -- 172.21.15.70:0/2295828093 shutdown_connections 2023-10-08T07:13:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.286+0000 7f655eccf640 1 --2- 172.21.15.70:0/2295828093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580d0d30 0x7f65580d11c0 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.286+0000 7f655eccf640 1 --2- 172.21.15.70:0/2295828093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65580cd930 0x7f65580cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.286+0000 7f655eccf640 1 -- 172.21.15.70:0/2295828093 >> 172.21.15.70:0/2295828093 conn(0x7f65580718a0 msgr2=0x7f6558071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.286+0000 7f655eccf640 1 -- 172.21.15.70:0/2295828093 shutdown_connections 2023-10-08T07:13:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.286+0000 7f655eccf640 1 -- 172.21.15.70:0/2295828093 wait complete. 2023-10-08T07:13:04.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.288+0000 7f655eccf640 1 Processor -- start 2023-10-08T07:13:04.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.288+0000 7f655eccf640 1 -- start start 2023-10-08T07:13:04.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.291+0000 7f655eccf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580cd930 0x7f6558149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655eccf640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65580d0d30 0x7f655815f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655eccf640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f655815cca0 0x7f65580ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655eccf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655815e920 con 0x7f65580cd930 2023-10-08T07:13:04.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655eccf640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655815ec00 con 0x7f65580d0d30 2023-10-08T07:13:04.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655eccf640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655815eee0 con 0x7f655815cca0 2023-10-08T07:13:04.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655ca6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580cd930 0x7f6558149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:04.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655ca6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580cd930 0x7f6558149140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42776/0 (socket says 172.21.15.70:42776) 2023-10-08T07:13:04.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655ca6b640 1 -- 172.21.15.70:0/4190788689 learned_addr learned my addr 172.21.15.70:0/4190788689 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:04.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655ca6b640 1 -- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f655815cca0 msgr2=0x7f65580ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655ca6b640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f655815cca0 0x7f65580ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655ca6b640 1 -- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65580d0d30 msgr2=0x7f655815f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655ca6b640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65580d0d30 0x7f655815f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.292+0000 7f655ca6b640 1 -- 172.21.15.70:0/4190788689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6550009a70 con 0x7f65580cd930 2023-10-08T07:13:04.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.293+0000 7f655ca6b640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580cd930 0x7f6558149140 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f654c00c500 tx=0x7f654c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:04.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.293+0000 7f65557fa640 1 -- 172.21.15.70:0/4190788689 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c005ce0 con 0x7f65580cd930 2023-10-08T07:13:04.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.293+0000 7f65557fa640 1 -- 172.21.15.70:0/4190788689 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c005e40 con 0x7f65580cd930 2023-10-08T07:13:04.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.293+0000 7f65557fa640 1 -- 172.21.15.70:0/4190788689 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c007330 con 0x7f65580cd930 2023-10-08T07:13:04.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.293+0000 7f655eccf640 1 -- 172.21.15.70:0/4190788689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655815f1c0 con 0x7f65580cd930 2023-10-08T07:13:04.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.293+0000 7f655eccf640 1 -- 172.21.15.70:0/4190788689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65580c9fc0 con 0x7f65580cd930 2023-10-08T07:13:04.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.295+0000 7f655eccf640 1 -- 172.21.15.70:0/4190788689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655804e160 con 0x7f65580cd930 2023-10-08T07:13:04.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.296+0000 7f65557fa640 1 -- 172.21.15.70:0/4190788689 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 124) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f654c007550 con 0x7f65580cd930 2023-10-08T07:13:04.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.297+0000 7f65557fa640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f653c06af90 0x7f653c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.297+0000 7f65557fa640 1 -- 172.21.15.70:0/4190788689 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6648+0+0 (secure 0 0 0) 0x7f654c0891f0 con 0x7f65580cd930 2023-10-08T07:13:04.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.297+0000 7f6557fff640 1 -- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f653c06af90 msgr2=0x7f653c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:04.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.297+0000 7f6557fff640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f653c06af90 0x7f653c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:13:04.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.301+0000 7f65557fa640 1 -- 172.21.15.70:0/4190788689 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f654c053e90 con 0x7f65580cd930 2023-10-08T07:13:04.354 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:04.353+0000 7f1cea779040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:13:04.406 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:04.405+0000 7f1cea779040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:13:04.416 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:04.414+0000 7f429de32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:13:04.456 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:04.455+0000 7f1cea779040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:13:04.467 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:04.466+0000 7f429de32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:13:04.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.498+0000 7f6557fff640 1 -- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f653c06af90 msgr2=0x7f653c06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:04.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.498+0000 7f6557fff640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f653c06af90 0x7f653c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:13:04.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.509+0000 7f655eccf640 1 -- 172.21.15.70:0/4190788689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f655815ea60 con 0x7f65580cd930 2023-10-08T07:13:04.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.510+0000 7f65557fa640 1 -- 172.21.15.70:0/4190788689 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v221) v1 ==== 93+0+10 (secure 0 0 0) 0x7f655815ea60 con 0x7f65580cd930 2023-10-08T07:13:04.511 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:13:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 -- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f653c06af90 msgr2=0x7f653c06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:13:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f653c06af90 0x7f653c06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 -- 172.21.15.70:0/4190788689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580cd930 msgr2=0x7f6558149140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580cd930 0x7f6558149140 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f654c00c500 tx=0x7f654c00c9c0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 -- 172.21.15.70:0/4190788689 shutdown_connections 2023-10-08T07:13:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f653c06af90 0x7f653c06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f655815cca0 0x7f65580ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65580d0d30 0x7f655815f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 --2- 172.21.15.70:0/4190788689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65580cd930 0x7f6558149140 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 -- 172.21.15.70:0/4190788689 >> 172.21.15.70:0/4190788689 conn(0x7f65580718a0 msgr2=0x7f655815bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:04.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 -- 172.21.15.70:0/4190788689 shutdown_connections 2023-10-08T07:13:04.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.513+0000 7f655eccf640 1 -- 172.21.15.70:0/4190788689 wait complete. 2023-10-08T07:13:04.516 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:04.516+0000 7f1cea779040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:13:04.519 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:04.517+0000 7f429de32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:13:04.532 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:13:04.568 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:04.567+0000 7f1cea779040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:13:04.583 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:04.582+0000 7f429de32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:13:04.638 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:04.636+0000 7f429de32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:13:04.680 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:04.679+0000 7f1cea779040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:13:04.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.740+0000 7faccbfff640 1 Processor -- start 2023-10-08T07:13:04.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.740+0000 7faccbfff640 1 -- start start 2023-10-08T07:13:04.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.741+0000 7faccbfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7faccc15a820 0x7faccc0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.742+0000 7faccbfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 0x7faccc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.742+0000 7faccbfff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc0cab60 con 0x7faccc0ca680 2023-10-08T07:13:04.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.742+0000 7faccbfff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc0cb960 con 0x7faccc15a820 2023-10-08T07:13:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.742+0000 7faccbfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0cbdf0 con 0x7faccc15ac00 2023-10-08T07:13:04.744 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:04.743+0000 7f1cea779040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:13:04.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.742+0000 7faccb7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 0x7faccc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:04.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.742+0000 7facca7fc640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7faccc0ca680 0x7faccc0c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:47328/0 (socket says 172.21.15.70:47328) 2023-10-08T07:13:04.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.742+0000 7facca7fc640 1 -- 172.21.15.70:0/2609086332 learned_addr learned my addr 172.21.15.70:0/2609086332 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:04.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.743+0000 7faccaffd640 1 --2- 172.21.15.70:0/2609086332 >> v2:172.21.15.181:3300/0 conn(0x7faccc15a820 0x7faccc0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:04.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.743+0000 7facc9ffb640 1 -- 172.21.15.70:0/2609086332 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35998289 0 0) 0x7faccc0cab60 con 0x7faccc0ca680 2023-10-08T07:13:04.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.743+0000 7facc9ffb640 1 -- 172.21.15.70:0/2609086332 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb4003520 con 0x7faccc0ca680 2023-10-08T07:13:04.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.743+0000 7faccaffd640 1 -- 172.21.15.70:0/2609086332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 msgr2=0x7faccc0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:04.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.743+0000 7faccaffd640 1 --2- 172.21.15.70:0/2609086332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 0x7faccc0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.743+0000 7faccaffd640 1 -- 172.21.15.70:0/2609086332 >> v1:172.21.15.181:6789/0 conn(0x7faccc0ca680 legacy=0x7faccc0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.743+0000 7faccaffd640 1 -- 172.21.15.70:0/2609086332 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc0cb030 con 0x7faccc15a820 2023-10-08T07:13:04.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.743+0000 7faccb7fe640 1 --2- 172.21.15.70:0/2609086332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 0x7faccc0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:13:04.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.745+0000 7faccaffd640 1 --2- 172.21.15.70:0/2609086332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc15a820 0x7faccc0aff70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7facbc0097a0 tx=0x7facbc02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b42c59e5c6d5747d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:04.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.745+0000 7facc9ffb640 1 -- 172.21.15.70:0/2609086332 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc031780 con 0x7faccc15a820 2023-10-08T07:13:04.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.745+0000 7facc9ffb640 1 -- 172.21.15.70:0/2609086332 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc0318e0 con 0x7faccc15a820 2023-10-08T07:13:04.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.745+0000 7facc9ffb640 1 -- 172.21.15.70:0/2609086332 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc031c70 con 0x7faccc15a820 2023-10-08T07:13:04.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.745+0000 7faccbfff640 1 -- 172.21.15.70:0/2609086332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc15a820 msgr2=0x7faccc0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.745+0000 7faccbfff640 1 --2- 172.21.15.70:0/2609086332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc15a820 0x7faccc0aff70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7facbc0097a0 tx=0x7facbc02b820 comp rx=0 tx=0).stop 2023-10-08T07:13:04.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.746+0000 7faccbfff640 1 -- 172.21.15.70:0/2609086332 shutdown_connections 2023-10-08T07:13:04.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.746+0000 7faccbfff640 1 --2- 172.21.15.70:0/2609086332 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 0x7faccc0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.746+0000 7faccbfff640 1 --2- 172.21.15.70:0/2609086332 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc15a820 0x7faccc0aff70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.746+0000 7faccbfff640 1 -- 172.21.15.70:0/2609086332 >> 172.21.15.70:0/2609086332 conn(0x7faccc06c050 msgr2=0x7faccc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.746+0000 7faccbfff640 1 -- 172.21.15.70:0/2609086332 shutdown_connections 2023-10-08T07:13:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.746+0000 7faccbfff640 1 -- 172.21.15.70:0/2609086332 wait complete. 2023-10-08T07:13:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.747+0000 7faccbfff640 1 Processor -- start 2023-10-08T07:13:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.747+0000 7faccbfff640 1 -- start start 2023-10-08T07:13:04.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.748+0000 7faccbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc0ca680 0x7faccc0c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.748+0000 7faccbfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc15a820 0x7faccc0afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.748+0000 7faccbfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 0x7faccc0c3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.748+0000 7faccbfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c2aa0 con 0x7faccc0ca680 2023-10-08T07:13:04.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.748+0000 7faccbfff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c5b00 con 0x7faccc15a820 2023-10-08T07:13:04.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.748+0000 7faccbfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c8350 con 0x7faccc15ac00 2023-10-08T07:13:04.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.749+0000 7faccaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc0ca680 0x7faccc0c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:04.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.749+0000 7faccaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc0ca680 0x7faccc0c55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42784/0 (socket says 172.21.15.70:42784) 2023-10-08T07:13:04.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.749+0000 7faccaffd640 1 -- 172.21.15.70:0/2558592774 learned_addr learned my addr 172.21.15.70:0/2558592774 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:04.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.749+0000 7faccb7fe640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 0x7faccc0c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:04.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.749+0000 7facca7fc640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc15a820 0x7faccc0afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:04.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.749+0000 7faccaffd640 1 -- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 msgr2=0x7faccc0c3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.749+0000 7faccaffd640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 0x7faccc0c3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.749+0000 7faccaffd640 1 -- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc15a820 msgr2=0x7faccc0afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.749+0000 7faccaffd640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc15a820 0x7faccc0afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.749+0000 7faccaffd640 1 -- 172.21.15.70:0/2558592774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc4009a70 con 0x7faccc0ca680 2023-10-08T07:13:04.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.750+0000 7faccaffd640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc0ca680 0x7faccc0c55c0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7facbc0098c0 tx=0x7facbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:04.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.750+0000 7facb37fe640 1 -- 172.21.15.70:0/2558592774 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc032820 con 0x7faccc0ca680 2023-10-08T07:13:04.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.750+0000 7facb37fe640 1 -- 172.21.15.70:0/2558592774 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc0329c0 con 0x7faccc0ca680 2023-10-08T07:13:04.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.750+0000 7facb37fe640 1 -- 172.21.15.70:0/2558592774 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc032d70 con 0x7faccc0ca680 2023-10-08T07:13:04.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.750+0000 7faccbfff640 1 -- 172.21.15.70:0/2558592774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facbc009400 con 0x7faccc0ca680 2023-10-08T07:13:04.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.750+0000 7faccbfff640 1 -- 172.21.15.70:0/2558592774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc0c7d90 con 0x7faccc0ca680 2023-10-08T07:13:04.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.751+0000 7faccbfff640 1 -- 172.21.15.70:0/2558592774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc074150 con 0x7faccc0ca680 2023-10-08T07:13:04.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.753+0000 7facb37fe640 1 -- 172.21.15.70:0/2558592774 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 124) v1 ==== 89343+0+0 (secure 0 0 0) 0x7facbc0453f0 con 0x7faccc0ca680 2023-10-08T07:13:04.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.754+0000 7facb37fe640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7facb806af90 0x7facb806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:04.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.754+0000 7facb37fe640 1 -- 172.21.15.70:0/2558592774 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6648+0+0 (secure 0 0 0) 0x7facbc0ac3e0 con 0x7faccc0ca680 2023-10-08T07:13:04.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.754+0000 7facca7fc640 1 -- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7facb806af90 msgr2=0x7facb806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:04.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.754+0000 7facca7fc640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7facb806af90 0x7facb806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:13:04.758 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:04.756+0000 7f429de32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:13:04.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.757+0000 7facb37fe640 1 -- 172.21.15.70:0/2558592774 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7facbc076fb0 con 0x7faccc0ca680 2023-10-08T07:13:04.795 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:04.795+0000 7f1cea779040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:13:04.823 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:04.821+0000 7f429de32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:13:04.852 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:04.851+0000 7f1cea779040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:13:04.874 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:04.872+0000 7f429de32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:13:04.907 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:04.906+0000 7f1cea779040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:13:04.933 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:04.931+0000 7f429de32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:13:04.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.948+0000 7faccbfff640 1 -- 172.21.15.70:0/2558592774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faccc070aa0 con 0x7faccc0ca680 2023-10-08T07:13:04.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.949+0000 7facb37fe640 1 -- 172.21.15.70:0/2558592774 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v221) v1 ==== 104+0+11 (secure 0 0 0) 0x7facbc07a5d0 con 0x7faccc0ca680 2023-10-08T07:13:04.951 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:13:04.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 -- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7facb806af90 msgr2=0x7facb806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:13:04.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7facb806af90 0x7facb806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 -- 172.21.15.70:0/2558592774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc0ca680 msgr2=0x7faccc0c55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:04.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc0ca680 0x7faccc0c55c0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7facbc0098c0 tx=0x7facbc0046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 -- 172.21.15.70:0/2558592774 shutdown_connections 2023-10-08T07:13:04.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7facb806af90 0x7facb806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faccc15ac00 0x7faccc0c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faccc15a820 0x7faccc0afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 --2- 172.21.15.70:0/2558592774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faccc0ca680 0x7faccc0c55c0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 -- 172.21.15.70:0/2558592774 >> 172.21.15.70:0/2558592774 conn(0x7faccc06c050 msgr2=0x7faccc159ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 -- 172.21.15.70:0/2558592774 shutdown_connections 2023-10-08T07:13:04.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:04.952+0000 7faccbfff640 1 -- 172.21.15.70:0/2558592774 wait complete. 2023-10-08T07:13:04.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:13:04.991 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:04.990+0000 7f429de32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:13:05.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.169+0000 7f93eeac2640 1 Processor -- start 2023-10-08T07:13:05.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.169+0000 7f93eeac2640 1 -- start start 2023-10-08T07:13:05.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.171+0000 7f93eeac2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93e80b0680 0x7f93e815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.171+0000 7f93eeac2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93e80ca680 0x7f93e815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.171+0000 7f93eeac2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e80cab60 con 0x7f93e815a820 2023-10-08T07:13:05.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.171+0000 7f93eeac2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e80cb960 con 0x7f93e80ca680 2023-10-08T07:13:05.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.171+0000 7f93eeac2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80cbdf0 con 0x7f93e80b0680 2023-10-08T07:13:05.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.171+0000 7f93ee2c1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93e80ca680 0x7f93e815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:05.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.171+0000 7f93edac0640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f93e815a820 0x7f93e80aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51506/0 (socket says 172.21.15.70:51506) 2023-10-08T07:13:05.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.171+0000 7f93edac0640 1 -- 172.21.15.70:0/2701202870 learned_addr learned my addr 172.21.15.70:0/2701202870 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:05.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.171+0000 7f93ee2c1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93e80ca680 0x7f93e815ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42794/0 (socket says 172.21.15.70:42794) 2023-10-08T07:13:05.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.172+0000 7f93ed2bf640 1 --2- 172.21.15.70:0/2701202870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93e80b0680 0x7f93e815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:05.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.172+0000 7f93ee2c1640 1 -- 172.21.15.70:0/2701202870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93e80b0680 msgr2=0x7f93e815e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:05.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.172+0000 7f93ee2c1640 1 --2- 172.21.15.70:0/2701202870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93e80b0680 0x7f93e815e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.172+0000 7f93ee2c1640 1 -- 172.21.15.70:0/2701202870 >> v1:172.21.15.70:6789/0 conn(0x7f93e815a820 legacy=0x7f93e80aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:05.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.172+0000 7f93ee2c1640 1 -- 172.21.15.70:0/2701202870 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e80cb030 con 0x7f93e80ca680 2023-10-08T07:13:05.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.173+0000 7f93ee2c1640 1 --2- 172.21.15.70:0/2701202870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e80ca680 0x7f93e815ea80 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009d90 tx=0x7f93e002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29acaa236733367 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:05.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.173+0000 7f93ecabe640 1 -- 172.21.15.70:0/2701202870 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e002cbd0 con 0x7f93e80ca680 2023-10-08T07:13:05.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.173+0000 7f93ecabe640 1 -- 172.21.15.70:0/2701202870 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e002cd30 con 0x7f93e80ca680 2023-10-08T07:13:05.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.173+0000 7f93ecabe640 1 -- 172.21.15.70:0/2701202870 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e002cea0 con 0x7f93e80ca680 2023-10-08T07:13:05.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.174+0000 7f93eeac2640 1 -- 172.21.15.70:0/2701202870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e80ca680 msgr2=0x7f93e815ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:05.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.174+0000 7f93eeac2640 1 --2- 172.21.15.70:0/2701202870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e80ca680 0x7f93e815ea80 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009d90 tx=0x7f93e002be20 comp rx=0 tx=0).stop 2023-10-08T07:13:05.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.174+0000 7f93eeac2640 1 -- 172.21.15.70:0/2701202870 shutdown_connections 2023-10-08T07:13:05.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.174+0000 7f93eeac2640 1 --2- 172.21.15.70:0/2701202870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e80ca680 0x7f93e815ea80 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.174+0000 7f93eeac2640 1 --2- 172.21.15.70:0/2701202870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93e80b0680 0x7f93e815e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.174+0000 7f93eeac2640 1 -- 172.21.15.70:0/2701202870 >> 172.21.15.70:0/2701202870 conn(0x7f93e806c050 msgr2=0x7f93e806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.174+0000 7f93eeac2640 1 -- 172.21.15.70:0/2701202870 shutdown_connections 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.174+0000 7f93eeac2640 1 -- 172.21.15.70:0/2701202870 wait complete. 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.175+0000 7f93eeac2640 1 Processor -- start 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.176+0000 7f93eeac2640 1 -- start start 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.176+0000 7f93eeac2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93e80b0680 0x7f93e81f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93eeac2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93e80ca680 0x7f93e8155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93eeac2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e815a820 0x7f93e815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93eeac2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e81f3ba0 con 0x7f93e815a820 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93eeac2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8156ac0 con 0x7f93e80ca680 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93eeac2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8156da0 con 0x7f93e80b0680 2023-10-08T07:13:05.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93ee2c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e815a820 0x7f93e815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93ee2c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e815a820 0x7f93e815c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42800/0 (socket says 172.21.15.70:42800) 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93ee2c1640 1 -- 172.21.15.70:0/3460157087 learned_addr learned my addr 172.21.15.70:0/3460157087 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93ee2c1640 1 -- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93e80b0680 msgr2=0x7f93e81f3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93ee2c1640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93e80b0680 0x7f93e81f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93ee2c1640 1 -- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93e80ca680 msgr2=0x7f93e8155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93ee2c1640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93e80ca680 0x7f93e8155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.177+0000 7f93ee2c1640 1 -- 172.21.15.70:0/3460157087 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e0009a70 con 0x7f93e815a820 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.178+0000 7f93ee2c1640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e815a820 0x7f93e815c4b0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009d90 tx=0x7f93e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.178+0000 7f93de7fc640 1 -- 172.21.15.70:0/3460157087 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e002cea0 con 0x7f93e815a820 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.178+0000 7f93de7fc640 1 -- 172.21.15.70:0/3460157087 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e0040670 con 0x7f93e815a820 2023-10-08T07:13:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.178+0000 7f93de7fc640 1 -- 172.21.15.70:0/3460157087 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0040a20 con 0x7f93e815a820 2023-10-08T07:13:05.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.178+0000 7f93eeac2640 1 -- 172.21.15.70:0/3460157087 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e8157080 con 0x7f93e815a820 2023-10-08T07:13:05.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.178+0000 7f93eeac2640 1 -- 172.21.15.70:0/3460157087 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e81583e0 con 0x7f93e815a820 2023-10-08T07:13:05.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.179+0000 7f93eeac2640 1 -- 172.21.15.70:0/3460157087 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e8074150 con 0x7f93e815a820 2023-10-08T07:13:05.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.180+0000 7f93de7fc640 1 -- 172.21.15.70:0/3460157087 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 124) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f93e0040b80 con 0x7f93e815a820 2023-10-08T07:13:05.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.181+0000 7f93de7fc640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f93cc06af90 0x7f93cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.182+0000 7f93de7fc640 1 -- 172.21.15.70:0/3460157087 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6648+0+0 (secure 0 0 0) 0x7f93e00ac920 con 0x7f93e815a820 2023-10-08T07:13:05.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.182+0000 7f93edac0640 1 -- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f93cc06af90 msgr2=0x7f93cc06d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:05.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.182+0000 7f93edac0640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f93cc06af90 0x7f93cc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:13:05.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.185+0000 7f93de7fc640 1 -- 172.21.15.70:0/3460157087 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93e0077500 con 0x7f93e815a820 2023-10-08T07:13:05.292 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:05.292+0000 7f1cea779040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:13:05.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.373+0000 7f93eeac2640 1 -- 172.21.15.70:0/3460157087 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93e80aff50 con 0x7f93e815a820 2023-10-08T07:13:05.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.373+0000 7f93de7fc640 1 -- 172.21.15.70:0/3460157087 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v221) v1 ==== 100+0+11 (secure 0 0 0) 0x7f93e007ab20 con 0x7f93e815a820 2023-10-08T07:13:05.375 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:13:05.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 -- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f93cc06af90 msgr2=0x7f93cc06d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:13:05.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f93cc06af90 0x7f93cc06d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 -- 172.21.15.70:0/3460157087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e815a820 msgr2=0x7f93e815c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:05.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e815a820 0x7f93e815c4b0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009d90 tx=0x7f93e0004a40 comp rx=0 tx=0).stop 2023-10-08T07:13:05.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 -- 172.21.15.70:0/3460157087 shutdown_connections 2023-10-08T07:13:05.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f93cc06af90 0x7f93cc06d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93e815a820 0x7f93e815c4b0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f93e80ca680 0x7f93e8155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 --2- 172.21.15.70:0/3460157087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f93e80b0680 0x7f93e81f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 -- 172.21.15.70:0/3460157087 >> 172.21.15.70:0/3460157087 conn(0x7f93e806c050 msgr2=0x7f93e80ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:05.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 -- 172.21.15.70:0/3460157087 shutdown_connections 2023-10-08T07:13:05.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.377+0000 7f93eeac2640 1 -- 172.21.15.70:0/3460157087 wait complete. 2023-10-08T07:13:05.391 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:05.389+0000 7f429de32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:13:05.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T07:13:05.453 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:05.453+0000 7f1cea779040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:13:05.556 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:05.555+0000 7f429de32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:13:05.561 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:05.561+0000 7f1cea779040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:13:05.611 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:05.609+0000 7f1cea779040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:13:05.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.645+0000 7ff83d44d640 1 Processor -- start 2023-10-08T07:13:05.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.646+0000 7ff83d44d640 1 -- start start 2023-10-08T07:13:05.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.648+0000 7ff83d44d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff8381599d0 0x7ff8380cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.648+0000 7ff83d44d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8380b3c50 0x7ff8380caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.648+0000 7ff83d44d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8380cb8a0 con 0x7ff8380cb410 2023-10-08T07:13:05.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.648+0000 7ff83d44d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff838159db0 con 0x7ff8381599d0 2023-10-08T07:13:05.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.648+0000 7ff83d44d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff838071fe0 con 0x7ff8380b3c50 2023-10-08T07:13:05.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.648+0000 7ff83cc4c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff8380cb410 0x7ff8380c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51520/0 (socket says 172.21.15.70:51520) 2023-10-08T07:13:05.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.648+0000 7ff83cc4c640 1 -- 172.21.15.70:0/3295544270 learned_addr learned my addr 172.21.15.70:0/3295544270 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:05.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.648+0000 7ff837fff640 1 --2- 172.21.15.70:0/3295544270 >> v2:172.21.15.181:3300/0 conn(0x7ff8381599d0 0x7ff8380cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:05.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.648+0000 7ff82ffff640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8380b3c50 0x7ff8380caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:05.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.648+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901286690 0 0) 0x7ff8380cb8a0 con 0x7ff8380cb410 2023-10-08T07:13:05.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff824003520 con 0x7ff8380cb410 2023-10-08T07:13:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843221387 0 0) 0x7ff824003520 con 0x7ff8380cb410 2023-10-08T07:13:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8380b3c50 msgr2=0x7ff8380caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8380b3c50 0x7ff8380caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 >> v2:172.21.15.181:3300/0 conn(0x7ff8381599d0 msgr2=0x7ff8380cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 --2- 172.21.15.70:0/3295544270 >> v2:172.21.15.181:3300/0 conn(0x7ff8381599d0 0x7ff8380cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8380b44c0 con 0x7ff8380cb410 2023-10-08T07:13:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff8300019f0 con 0x7ff8380cb410 2023-10-08T07:13:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 >> v1:172.21.15.70:6789/0 conn(0x7ff8380cb410 legacy=0x7ff8380c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff824003970 0x7ff824003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8380b3c50 0x7ff8240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 0x7ff824005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff838159db0 con 0x7ff824004d00 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff838071fe0 con 0x7ff824003970 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82400b980 con 0x7ff8380b3c50 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.649+0000 7ff83cc4c640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 0x7ff824005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.650+0000 7ff82ffff640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8380b3c50 0x7ff8240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.650+0000 7ff837fff640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff824003970 0x7ff824003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.650+0000 7ff83cc4c640 1 -- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8380b3c50 msgr2=0x7ff8240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.650+0000 7ff83cc4c640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8380b3c50 0x7ff8240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.650+0000 7ff83cc4c640 1 -- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff824003970 msgr2=0x7ff824003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.650+0000 7ff83cc4c640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff824003970 0x7ff824003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.650+0000 7ff83cc4c640 1 -- 172.21.15.70:0/3295544270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff824003520 con 0x7ff824004d00 2023-10-08T07:13:05.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.651+0000 7ff83cc4c640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 0x7ff824005180 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7ff83000a8d0 tx=0x7ff8240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2f3f76f127e0e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:05.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.651+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff830034630 con 0x7ff824004d00 2023-10-08T07:13:05.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.651+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83002f070 con 0x7ff824004d00 2023-10-08T07:13:05.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.651+0000 7ff8377fe640 1 -- 172.21.15.70:0/3295544270 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8300349d0 con 0x7ff824004d00 2023-10-08T07:13:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.652+0000 7ff83d44d640 1 -- 172.21.15.70:0/3295544270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 msgr2=0x7ff824005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.652+0000 7ff83d44d640 1 --2- 172.21.15.70:0/3295544270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 0x7ff824005180 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7ff83000a8d0 tx=0x7ff8240057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.652+0000 7ff83cc4c640 1 -- 172.21.15.70:0/3295544270 reap_dead start 2023-10-08T07:13:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.652+0000 7ff83d44d640 1 -- 172.21.15.70:0/3295544270 shutdown_connections 2023-10-08T07:13:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.652+0000 7ff83d44d640 1 -- 172.21.15.70:0/3295544270 >> 172.21.15.70:0/3295544270 conn(0x7ff83806c050 msgr2=0x7ff83806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.652+0000 7ff83d44d640 1 -- 172.21.15.70:0/3295544270 shutdown_connections 2023-10-08T07:13:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.652+0000 7ff83d44d640 1 -- 172.21.15.70:0/3295544270 wait complete. 2023-10-08T07:13:05.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.655+0000 7ff83d44d640 1 Processor -- start 2023-10-08T07:13:05.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.655+0000 7ff83d44d640 1 -- start start 2023-10-08T07:13:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.656+0000 7ff83d44d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 0x7ff83814db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.656+0000 7ff83d44d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8381f2860 0x7ff8381f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.656+0000 7ff83d44d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8381f49f0 0x7ff83815a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff83d44d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8381f3680 con 0x7ff824004d00 2023-10-08T07:13:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff83d44d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8381f3950 con 0x7ff8381f2860 2023-10-08T07:13:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff83d44d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8381f3c30 con 0x7ff8381f49f0 2023-10-08T07:13:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff837fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 0x7ff83814db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff837fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 0x7ff83814db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42830/0 (socket says 172.21.15.70:42830) 2023-10-08T07:13:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff837fff640 1 -- 172.21.15.70:0/2926743337 learned_addr learned my addr 172.21.15.70:0/2926743337 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff83cc4c640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8381f49f0 0x7ff83815a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff837fff640 1 -- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8381f49f0 msgr2=0x7ff83815a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff837fff640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8381f49f0 0x7ff83815a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff837fff640 1 -- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8381f2860 msgr2=0x7ff8381f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff837fff640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8381f2860 0x7ff8381f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff837fff640 1 -- 172.21.15.70:0/2926743337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8381f3f10 con 0x7ff824004d00 2023-10-08T07:13:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.657+0000 7ff837fff640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 0x7ff83814db80 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7ff8280116d0 tx=0x7ff828011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.658+0000 7ff8357fa640 1 -- 172.21.15.70:0/2926743337 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800f9c0 con 0x7ff824004d00 2023-10-08T07:13:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.658+0000 7ff8357fa640 1 -- 172.21.15.70:0/2926743337 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82800fb20 con 0x7ff824004d00 2023-10-08T07:13:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.658+0000 7ff83d44d640 1 -- 172.21.15.70:0/2926743337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8381f41f0 con 0x7ff824004d00 2023-10-08T07:13:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.658+0000 7ff8357fa640 1 -- 172.21.15.70:0/2926743337 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800fe90 con 0x7ff824004d00 2023-10-08T07:13:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.658+0000 7ff83d44d640 1 -- 172.21.15.70:0/2926743337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8381f4e50 con 0x7ff824004d00 2023-10-08T07:13:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.659+0000 7ff83d44d640 1 -- 172.21.15.70:0/2926743337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff838074150 con 0x7ff824004d00 2023-10-08T07:13:05.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.660+0000 7ff8357fa640 1 -- 172.21.15.70:0/2926743337 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 124) v1 ==== 89343+0+0 (secure 0 0 0) 0x7ff828015460 con 0x7ff824004d00 2023-10-08T07:13:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.661+0000 7ff8357fa640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7ff81806af90 0x7ff81806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.661+0000 7ff8357fa640 1 -- 172.21.15.70:0/2926743337 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6648+0+0 (secure 0 0 0) 0x7ff82808a4f0 con 0x7ff824004d00 2023-10-08T07:13:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.661+0000 7ff82ffff640 1 -- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7ff81806af90 msgr2=0x7ff81806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:05.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.661+0000 7ff82ffff640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7ff81806af90 0x7ff81806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:13:05.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.666+0000 7ff8357fa640 1 -- 172.21.15.70:0/2926743337 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff828055040 con 0x7ff824004d00 2023-10-08T07:13:05.672 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:05.671+0000 7f429de32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:13:05.721 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:05.719+0000 7f1cea779040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:13:05.724 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:05.722+0000 7f429de32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:13:05.827 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:05.826+0000 7f1cea779040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:13:05.833 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:05.832+0000 7f429de32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:13:05.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.862+0000 7ff82ffff640 1 -- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7ff81806af90 msgr2=0x7ff81806d450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:05.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.862+0000 7ff82ffff640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7ff81806af90 0x7ff81806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:13:05.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.864+0000 7ff83d44d640 1 -- 172.21.15.70:0/2926743337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff8380cda20 con 0x7ff824004d00 2023-10-08T07:13:05.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.865+0000 7ff8357fa640 1 -- 172.21.15.70:0/2926743337 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v221) v1 ==== 80+0+473 (secure 0 0 0) 0x7ff828058660 con 0x7ff824004d00 2023-10-08T07:13:05.867 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/294214378 2023-10-09T07:10:58.332667+0000 2023-10-08T07:13:05.867 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3667065295 2023-10-09T07:10:58.332667+0000 2023-10-08T07:13:05.867 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3162325425 2023-10-09T07:10:58.332667+0000 2023-10-08T07:13:05.868 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/2638163552 2023-10-09T07:10:58.332667+0000 2023-10-08T07:13:05.868 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/2638163552 2023-10-09T07:10:58.332667+0000 2023-10-08T07:13:05.868 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/785931869 2023-10-09T07:10:58.332667+0000 2023-10-08T07:13:05.868 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/265797938 2023-10-09T07:10:58.332667+0000 2023-10-08T07:13:05.868 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/39996703 2023-10-09T07:10:58.332667+0000 2023-10-08T07:13:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:listed 8 entries 2023-10-08T07:13:05.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 -- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7ff81806af90 msgr2=0x7ff81806d450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:13:05.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7ff81806af90 0x7ff81806d450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 -- 172.21.15.70:0/2926743337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 msgr2=0x7ff83814db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 0x7ff83814db80 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7ff8280116d0 tx=0x7ff828011b90 comp rx=0 tx=0).stop 2023-10-08T07:13:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 -- 172.21.15.70:0/2926743337 shutdown_connections 2023-10-08T07:13:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7ff81806af90 0x7ff81806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff8381f49f0 0x7ff83815a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff8381f2860 0x7ff8381f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 --2- 172.21.15.70:0/2926743337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff824004d00 0x7ff83814db80 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 -- 172.21.15.70:0/2926743337 >> 172.21.15.70:0/2926743337 conn(0x7ff83806c050 msgr2=0x7ff83815b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 -- 172.21.15.70:0/2926743337 shutdown_connections 2023-10-08T07:13:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:05.868+0000 7ff83d44d640 1 -- 172.21.15.70:0/2926743337 wait complete. 2023-10-08T07:13:05.888 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T07:13:05.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:05.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T07:13:05.948 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:05.947+0000 7f429de32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:13:05.985 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:05.983+0000 7f1cea779040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:13:06.005 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T07:13:06.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:06.005 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T07:13:06.034 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:06.033+0000 7f1cea779040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:13:06.049 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:13:06.050 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:13:06.104 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:13:06.105 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:13:06.105 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:13:06.116 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:06.114+0000 7f429de32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:13:06.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:13:06.158 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:13:06.158 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:13:06.159 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:06.157+0000 7f1cea779040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:13:06.168 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:06.167+0000 7f429de32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:13:06.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:13:06.213 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:13:06.296 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:06.294+0000 7f429de32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:13:06.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.427+0000 7f10c6725640 1 Processor -- start 2023-10-08T07:13:06.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.427+0000 7f10c6725640 1 -- start start 2023-10-08T07:13:06.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.429+0000 7f10c6725640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10c0069d50 0x7f10c006a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.429+0000 7f10c6725640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c00ce550 con 0x7f10c00cb880 2023-10-08T07:13:06.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.429+0000 7f10c6725640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c01609d0 con 0x7f10c00cbce0 2023-10-08T07:13:06.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.429+0000 7f10c6725640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c00cdcf0 con 0x7f10c0069d50 2023-10-08T07:13:06.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.429+0000 7f10bffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10c0069d50 0x7f10c006a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.429+0000 7f10bf7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f10c00cb880 0x7f10c00b1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51530/0 (socket says 172.21.15.70:51530) 2023-10-08T07:13:06.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.429+0000 7f10bf7fe640 1 -- 172.21.15.70:0/592236522 learned_addr learned my addr 172.21.15.70:0/592236522 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:06.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.429+0000 7f10bffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10c0069d50 0x7f10c006a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42836/0 (socket says 172.21.15.70:42836) 2023-10-08T07:13:06.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.430+0000 7f10bffff640 1 -- 172.21.15.70:0/592236522 >> v1:172.21.15.181:6789/0 conn(0x7f10c00cbce0 legacy=0x7f10c0160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:06.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.430+0000 7f10bffff640 1 -- 172.21.15.70:0/592236522 >> v1:172.21.15.70:6789/0 conn(0x7f10c00cb880 legacy=0x7f10c00b1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:06.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.430+0000 7f10bffff640 1 -- 172.21.15.70:0/592236522 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c006a670 con 0x7f10c0069d50 2023-10-08T07:13:06.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.430+0000 7f10beffd640 1 -- 172.21.15.70:0/592236522 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765817229 0 0) 0x7f10c00ce550 con 0x7f10c00cb880 2023-10-08T07:13:06.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.431+0000 7f10bffff640 1 --2- 172.21.15.70:0/592236522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c0069d50 0x7f10c006a130 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f10b00097a0 tx=0x7f10b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1639afc871a0c1e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:06.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.431+0000 7f10beffd640 1 -- 172.21.15.70:0/592236522 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0031830 con 0x7f10c0069d50 2023-10-08T07:13:06.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.431+0000 7f10beffd640 1 -- 172.21.15.70:0/592236522 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b0031990 con 0x7f10c0069d50 2023-10-08T07:13:06.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.431+0000 7f10beffd640 1 -- 172.21.15.70:0/592236522 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0031d20 con 0x7f10c0069d50 2023-10-08T07:13:06.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.432+0000 7f10c6725640 1 -- 172.21.15.70:0/592236522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c0069d50 msgr2=0x7f10c006a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:06.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.432+0000 7f10c6725640 1 --2- 172.21.15.70:0/592236522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c0069d50 0x7f10c006a130 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f10b00097a0 tx=0x7f10b002b8d0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.432+0000 7f10c6725640 1 -- 172.21.15.70:0/592236522 shutdown_connections 2023-10-08T07:13:06.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.432+0000 7f10c6725640 1 --2- 172.21.15.70:0/592236522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c0069d50 0x7f10c006a130 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.432+0000 7f10c6725640 1 -- 172.21.15.70:0/592236522 >> 172.21.15.70:0/592236522 conn(0x7f10c006d450 msgr2=0x7f10c006d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:06.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.432+0000 7f10c6725640 1 -- 172.21.15.70:0/592236522 shutdown_connections 2023-10-08T07:13:06.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.432+0000 7f10c6725640 1 -- 172.21.15.70:0/592236522 wait complete. 2023-10-08T07:13:06.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.434+0000 7f10c6725640 1 Processor -- start 2023-10-08T07:13:06.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.434+0000 7f10c6725640 1 -- start start 2023-10-08T07:13:06.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.435+0000 7f10c6725640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c0069d50 0x7f10c015fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.435+0000 7f10c6725640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00cb880 0x7f10c00afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.435+0000 7f10c6725640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c00cbce0 0x7f10c01f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.435+0000 7f10c6725640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00af2f0 con 0x7f10c00cb880 2023-10-08T07:13:06.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.435+0000 7f10c6725640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00af870 con 0x7f10c00cbce0 2023-10-08T07:13:06.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.435+0000 7f10c6725640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c01f6c70 con 0x7f10c0069d50 2023-10-08T07:13:06.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.435+0000 7f10bf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00cb880 0x7f10c00afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.435+0000 7f10bf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00cb880 0x7f10c00afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42850/0 (socket says 172.21.15.70:42850) 2023-10-08T07:13:06.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.435+0000 7f10bf7fe640 1 -- 172.21.15.70:0/2550704086 learned_addr learned my addr 172.21.15.70:0/2550704086 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:06.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.435+0000 7f10bffff640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c0069d50 0x7f10c015fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10c4cc2640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c00cbce0 0x7f10c01f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10bf7fe640 1 -- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c0069d50 msgr2=0x7f10c015fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:06.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10bf7fe640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c0069d50 0x7f10c015fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10bf7fe640 1 -- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c00cbce0 msgr2=0x7f10c01f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:06.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10bf7fe640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c00cbce0 0x7f10c01f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10bf7fe640 1 -- 172.21.15.70:0/2550704086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b0009400 con 0x7f10c00cb880 2023-10-08T07:13:06.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10c4cc2640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c00cbce0 0x7f10c01f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:06.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10bf7fe640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00cb880 0x7f10c00afd00 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f10b400c4d0 tx=0x7f10b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:06.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10bcff9640 1 -- 172.21.15.70:0/2550704086 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40074e0 con 0x7f10c00cb880 2023-10-08T07:13:06.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10bcff9640 1 -- 172.21.15.70:0/2550704086 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b4007660 con 0x7f10c00cb880 2023-10-08T07:13:06.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10c6725640 1 -- 172.21.15.70:0/2550704086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c01f6f50 con 0x7f10c00cb880 2023-10-08T07:13:06.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10bcff9640 1 -- 172.21.15.70:0/2550704086 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4004a20 con 0x7f10c00cb880 2023-10-08T07:13:06.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.436+0000 7f10c6725640 1 -- 172.21.15.70:0/2550704086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c014d6d0 con 0x7f10c00cb880 2023-10-08T07:13:06.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.437+0000 7f10c6725640 1 -- 172.21.15.70:0/2550704086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c00af9e0 con 0x7f10c00cb880 2023-10-08T07:13:06.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.438+0000 7f10bcff9640 1 -- 172.21.15.70:0/2550704086 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 124) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f10b4004b80 con 0x7f10c00cb880 2023-10-08T07:13:06.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.439+0000 7f10bcff9640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f10ac06afc0 0x7f10ac06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.439+0000 7f10bcff9640 1 -- 172.21.15.70:0/2550704086 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6648+0+0 (secure 0 0 0) 0x7f10b4088d00 con 0x7f10c00cb880 2023-10-08T07:13:06.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.439+0000 7f10bffff640 1 -- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f10ac06afc0 msgr2=0x7f10ac06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:06.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.439+0000 7f10bffff640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f10ac06afc0 0x7f10ac06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:13:06.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.443+0000 7f10bcff9640 1 -- 172.21.15.70:0/2550704086 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10b40538e0 con 0x7f10c00cb880 2023-10-08T07:13:06.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.640+0000 7f10bffff640 1 -- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f10ac06afc0 msgr2=0x7f10ac06d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:06.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.640+0000 7f10bffff640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f10ac06afc0 0x7f10ac06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:13:06.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.670+0000 7f10c6725640 1 -- 172.21.15.70:0/2550704086 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10c0067490 con 0x7f10c00cb880 2023-10-08T07:13:06.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.671+0000 7f10bcff9640 1 -- 172.21.15.70:0/2550704086 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f10b4018020 con 0x7f10c00cb880 2023-10-08T07:13:06.673 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:06.673 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/4185609199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4185609199},{"type":"v1","addr":"172.21.15.70:6833","nonce":4185609199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3102206290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3102206290},{"type":"v1","addr":"172.21.15.181:6837","nonce":3102206290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1824427895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1824427895},{"type":"v1","addr":"172.21.15.181:6835","nonce":1824427895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:10:44.128443+0000","modified":"2023-10-08T07:10:47.166174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/2355440757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2355440757},{"type":"v1","addr":"172.21.15.70:6835","nonce":2355440757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-08T07:13:06.674 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 99 2023-10-08T07:13:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.673+0000 7f10c6725640 1 -- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f10ac06afc0 msgr2=0x7f10ac06d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:13:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.673+0000 7f10c6725640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f10ac06afc0 0x7f10ac06d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.674+0000 7f10c6725640 1 -- 172.21.15.70:0/2550704086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00cb880 msgr2=0x7f10c00afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.674+0000 7f10c6725640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00cb880 0x7f10c00afd00 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f10b400c4d0 tx=0x7f10b400c990 comp rx=0 tx=0).stop 2023-10-08T07:13:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.674+0000 7f10c6725640 1 -- 172.21.15.70:0/2550704086 shutdown_connections 2023-10-08T07:13:06.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.674+0000 7f10c6725640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f10ac06afc0 0x7f10ac06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.674+0000 7f10c6725640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10c00cbce0 0x7f10c01f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.674+0000 7f10c6725640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10c00cb880 0x7f10c00afd00 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.674+0000 7f10c6725640 1 --2- 172.21.15.70:0/2550704086 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10c0069d50 0x7f10c015fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.674+0000 7f10c6725640 1 -- 172.21.15.70:0/2550704086 >> 172.21.15.70:0/2550704086 conn(0x7f10c006d450 msgr2=0x7f10c015ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:06.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.674+0000 7f10c6725640 1 -- 172.21.15.70:0/2550704086 shutdown_connections 2023-10-08T07:13:06.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.674+0000 7f10c6725640 1 -- 172.21.15.70:0/2550704086 wait complete. 2023-10-08T07:13:06.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:06.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.882+0000 7f3339283640 1 Processor -- start 2023-10-08T07:13:06.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.882+0000 7f3339283640 1 -- start start 2023-10-08T07:13:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.884+0000 7f3339283640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f333406ae60 0x7f33340b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.884+0000 7f3339283640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33340b0340 con 0x7f333406b520 2023-10-08T07:13:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.884+0000 7f3339283640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333406b2f0 con 0x7f33340af8b0 2023-10-08T07:13:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.884+0000 7f3339283640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33340b0150 con 0x7f333406ae60 2023-10-08T07:13:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.884+0000 7f3332ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f333406b520 0x7f333406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51546/0 (socket says 172.21.15.70:51546) 2023-10-08T07:13:06.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.884+0000 7f3332ffd640 1 -- 172.21.15.70:0/1543069736 learned_addr learned my addr 172.21.15.70:0/1543069736 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:06.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756169402 0 0) 0x7f33340b0340 con 0x7f333406b520 2023-10-08T07:13:06.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3320003520 con 0x7f333406b520 2023-10-08T07:13:06.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33337fe640 1 --2- 172.21.15.70:0/1543069736 >> v2:172.21.15.181:3300/0 conn(0x7f333406ae60 0x7f33340b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1266426640 0 0) 0x7f3320003520 con 0x7f333406b520 2023-10-08T07:13:06.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 >> v2:172.21.15.181:3300/0 conn(0x7f333406ae60 msgr2=0x7f33340b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:06.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 --2- 172.21.15.70:0/1543069736 >> v2:172.21.15.181:3300/0 conn(0x7f333406ae60 0x7f33340b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 >> v1:172.21.15.181:6789/0 conn(0x7f33340af8b0 legacy=0x7f33340b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:06.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33340cf8f0 con 0x7f333406b520 2023-10-08T07:13:06.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f33240017f0 con 0x7f333406b520 2023-10-08T07:13:06.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 >> v1:172.21.15.70:6789/0 conn(0x7f333406b520 legacy=0x7f333406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:06.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33200039c0 0x7f3320003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33200047c0 0x7f3320004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33200052a0 0x7f332000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340b0150 con 0x7f33200047c0 2023-10-08T07:13:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332000c1c0 con 0x7f33200052a0 2023-10-08T07:13:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.885+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332000c380 con 0x7f33200039c0 2023-10-08T07:13:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.886+0000 7f332bfff640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33200047c0 0x7f3320004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.886+0000 7f3332ffd640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33200039c0 0x7f3320003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.886+0000 7f33337fe640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33200052a0 0x7f332000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.886+0000 7f332bfff640 1 -- 172.21.15.70:0/1543069736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33200039c0 msgr2=0x7f3320003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.886+0000 7f332bfff640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33200039c0 0x7f3320003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.886+0000 7f332bfff640 1 -- 172.21.15.70:0/1543069736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33200052a0 msgr2=0x7f332000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.886+0000 7f332bfff640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33200052a0 0x7f332000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.886+0000 7f332bfff640 1 -- 172.21.15.70:0/1543069736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3320003520 con 0x7f33200047c0 2023-10-08T07:13:06.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.886+0000 7f33337fe640 1 -- 172.21.15.70:0/1543069736 reap_dead start 2023-10-08T07:13:06.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.887+0000 7f332bfff640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33200047c0 0x7f3320004c40 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f331c001750 tx=0x7f33200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7666a68ec124d42 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:06.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.888+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c0052d0 con 0x7f33200047c0 2023-10-08T07:13:06.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.888+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331c02e050 con 0x7f33200047c0 2023-10-08T07:13:06.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.888+0000 7f33327fc640 1 -- 172.21.15.70:0/1543069736 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c003830 con 0x7f33200047c0 2023-10-08T07:13:06.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.888+0000 7f3339283640 1 -- 172.21.15.70:0/1543069736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33200047c0 msgr2=0x7f3320004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:06.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.888+0000 7f3339283640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33200047c0 0x7f3320004c40 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f331c001750 tx=0x7f33200057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.888+0000 7f3339283640 1 -- 172.21.15.70:0/1543069736 shutdown_connections 2023-10-08T07:13:06.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.888+0000 7f3339283640 1 --2- 172.21.15.70:0/1543069736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33200047c0 0x7f3320004c40 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.888+0000 7f3339283640 1 -- 172.21.15.70:0/1543069736 >> 172.21.15.70:0/1543069736 conn(0x7f33340d09e0 msgr2=0x7f33340d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:06.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.889+0000 7f3339283640 1 -- 172.21.15.70:0/1543069736 shutdown_connections 2023-10-08T07:13:06.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.889+0000 7f3339283640 1 -- 172.21.15.70:0/1543069736 wait complete. 2023-10-08T07:13:06.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.890+0000 7f3339283640 1 Processor -- start 2023-10-08T07:13:06.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.890+0000 7f3339283640 1 -- start start 2023-10-08T07:13:06.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.891+0000 7f3339283640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33200047c0 0x7f33340c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.891+0000 7f3339283640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333415e040 0x7f333415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.891+0000 7f3339283640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3334158a40 0x7f3334158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.891+0000 7f3339283640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334159400 con 0x7f333415e040 2023-10-08T07:13:06.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.891+0000 7f3339283640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334160170 con 0x7f33200047c0 2023-10-08T07:13:06.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.891+0000 7f3339283640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3334160450 con 0x7f3334158a40 2023-10-08T07:13:06.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f332bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333415e040 0x7f333415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f332bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333415e040 0x7f333415e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42866/0 (socket says 172.21.15.70:42866) 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f332bfff640 1 -- 172.21.15.70:0/1659723326 learned_addr learned my addr 172.21.15.70:0/1659723326 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f3332ffd640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33200047c0 0x7f33340c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f33337fe640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3334158a40 0x7f3334158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f332bfff640 1 -- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3334158a40 msgr2=0x7f3334158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f332bfff640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3334158a40 0x7f3334158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f332bfff640 1 -- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33200047c0 msgr2=0x7f33340c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f332bfff640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33200047c0 0x7f33340c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f332bfff640 1 -- 172.21.15.70:0/1659723326 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33340b4e80 con 0x7f333415e040 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f3332ffd640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33200047c0 0x7f33340c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.892+0000 7f332bfff640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333415e040 0x7f333415e4c0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f331c005040 tx=0x7f331c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:06.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.893+0000 7f332b7fe640 1 -- 172.21.15.70:0/1659723326 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c007a30 con 0x7f333415e040 2023-10-08T07:13:06.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.893+0000 7f332b7fe640 1 -- 172.21.15.70:0/1659723326 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331c0069e0 con 0x7f333415e040 2023-10-08T07:13:06.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.893+0000 7f332b7fe640 1 -- 172.21.15.70:0/1659723326 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c006d70 con 0x7f333415e040 2023-10-08T07:13:06.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.893+0000 7f3339283640 1 -- 172.21.15.70:0/1659723326 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33340b5160 con 0x7f333415e040 2023-10-08T07:13:06.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.893+0000 7f3339283640 1 -- 172.21.15.70:0/1659723326 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33340c65e0 con 0x7f333415e040 2023-10-08T07:13:06.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.894+0000 7f3339283640 1 -- 172.21.15.70:0/1659723326 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33340cfe90 con 0x7f333415e040 2023-10-08T07:13:06.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.895+0000 7f332b7fe640 1 -- 172.21.15.70:0/1659723326 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 124) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f331c0035f0 con 0x7f333415e040 2023-10-08T07:13:06.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.896+0000 7f332b7fe640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f331406afc0 0x7f331406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:06.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.896+0000 7f332b7fe640 1 -- 172.21.15.70:0/1659723326 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6648+0+0 (secure 0 0 0) 0x7f331c0abb40 con 0x7f333415e040 2023-10-08T07:13:06.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.896+0000 7f3332ffd640 1 -- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f331406afc0 msgr2=0x7f331406d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:06.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.896+0000 7f3332ffd640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f331406afc0 0x7f331406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:13:06.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:06.900+0000 7f332b7fe640 1 -- 172.21.15.70:0/1659723326 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33340cfe90 con 0x7f333415e040 2023-10-08T07:13:06.988 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:06.987+0000 7f1cea779040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:13:07.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.091+0000 7f3339283640 1 -- 172.21.15.70:0/1659723326 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33340c65e0 con 0x7f333415e040 2023-10-08T07:13:07.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.093+0000 7f332b7fe640 1 -- 172.21.15.70:0/1659723326 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+17675 (secure 0 0 0) 0x7f33340c65e0 con 0x7f333415e040 2023-10-08T07:13:07.095 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:07.095 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":221,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:04.286078+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:10:41.917258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-08T07:10:43.125413+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":221,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.28","mappings":[{"from":1,"to":0}]},{"pgid":"26.36","mappings":[{"from":3,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/294214378":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/3667065295":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/3162325425":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6833/2638163552":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6832/2638163552":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/785931869":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/265797938":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/39996703":"2023-10-09T07:10:58.332667+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":26,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.095+0000 7f3339283640 1 -- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f331406afc0 msgr2=0x7f331406d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:13:07.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.095+0000 7f3339283640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f331406afc0 0x7f331406d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.095+0000 7f3339283640 1 -- 172.21.15.70:0/1659723326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333415e040 msgr2=0x7f333415e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.095+0000 7f3339283640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333415e040 0x7f333415e4c0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f331c005040 tx=0x7f331c001120 comp rx=0 tx=0).stop 2023-10-08T07:13:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.095+0000 7f3339283640 1 -- 172.21.15.70:0/1659723326 shutdown_connections 2023-10-08T07:13:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.095+0000 7f3339283640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f331406afc0 0x7f331406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.095+0000 7f3339283640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3334158a40 0x7f3334158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.095+0000 7f3339283640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f333415e040 0x7f333415e4c0 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.095+0000 7f3339283640 1 --2- 172.21.15.70:0/1659723326 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33200047c0 0x7f33340c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.096+0000 7f3339283640 1 -- 172.21.15.70:0/1659723326 >> 172.21.15.70:0/1659723326 conn(0x7f33340d09e0 msgr2=0x7f3334160870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.096+0000 7f3339283640 1 -- 172.21.15.70:0/1659723326 shutdown_connections 2023-10-08T07:13:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.096+0000 7f3339283640 1 -- 172.21.15.70:0/1659723326 wait complete. 2023-10-08T07:13:07.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T07:10:41.917258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '215', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T07:10:43.125413+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '221', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 221, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:13:07.116 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:13:07.128 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:07.126+0000 7f429de32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:13:07.284 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.283+0000 7f1cea779040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:13:07.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.320+0000 7f34e3fff640 1 Processor -- start 2023-10-08T07:13:07.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.320+0000 7f34e3fff640 1 -- start start 2023-10-08T07:13:07.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.322+0000 7f34e3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e415dcd0 0x7f34e40d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.322+0000 7f34e3fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e40ce290 con 0x7f34e40d1500 2023-10-08T07:13:07.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.322+0000 7f34e3fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e40d0a80 con 0x7f34e40d1120 2023-10-08T07:13:07.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.322+0000 7f34e3fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40b1fe0 con 0x7f34e415dcd0 2023-10-08T07:13:07.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.322+0000 7f34e27fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f34e40d1500 0x7f34e40cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51550/0 (socket says 172.21.15.70:51550) 2023-10-08T07:13:07.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.322+0000 7f34e27fc640 1 -- 172.21.15.70:0/3358399636 learned_addr learned my addr 172.21.15.70:0/3358399636 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:07.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.322+0000 7f34e37fe640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e415dcd0 0x7f34e40d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961374772 0 0) 0x7f34e40ce290 con 0x7f34e40d1500 2023-10-08T07:13:07.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34cc003520 con 0x7f34e40d1500 2023-10-08T07:13:07.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1869219335 0 0) 0x7f34cc003520 con 0x7f34e40d1500 2023-10-08T07:13:07.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e415dcd0 msgr2=0x7f34e40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:07.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e415dcd0 0x7f34e40d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 >> v1:172.21.15.181:6789/0 conn(0x7f34e40d1120 legacy=0x7f34e40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e40b4580 con 0x7f34e40d1500 2023-10-08T07:13:07.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f34d8001670 con 0x7f34e40d1500 2023-10-08T07:13:07.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 >> v1:172.21.15.70:6789/0 conn(0x7f34e40d1500 legacy=0x7f34e40cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34cc0039a0 0x7f34cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34cc0047c0 0x7f34cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e415dcd0 0x7f34cc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40b1fe0 con 0x7f34cc0047c0 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc00bb40 con 0x7f34cc0039a0 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.323+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc00bd00 con 0x7f34e415dcd0 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.324+0000 7f34e27fc640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34cc0047c0 0x7f34cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.324+0000 7f34e2ffd640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34cc0039a0 0x7f34cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.324+0000 7f34e37fe640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e415dcd0 0x7f34cc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.324+0000 7f34e27fc640 1 -- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e415dcd0 msgr2=0x7f34cc005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.324+0000 7f34e27fc640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e415dcd0 0x7f34cc005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.324+0000 7f34e27fc640 1 -- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34cc0039a0 msgr2=0x7f34cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.324+0000 7f34e27fc640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34cc0039a0 0x7f34cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.324+0000 7f34e27fc640 1 -- 172.21.15.70:0/3358399636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc003520 con 0x7f34cc0047c0 2023-10-08T07:13:07.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.325+0000 7f34e27fc640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34cc0047c0 0x7f34cc004c40 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c0e0 tx=0x7f34cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2e57560619a3d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:07.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.325+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8005ad0 con 0x7f34cc0047c0 2023-10-08T07:13:07.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.325+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d8005c30 con 0x7f34cc0047c0 2023-10-08T07:13:07.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.325+0000 7f34e1ffb640 1 -- 172.21.15.70:0/3358399636 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d802c810 con 0x7f34cc0047c0 2023-10-08T07:13:07.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.326+0000 7f34e3fff640 1 -- 172.21.15.70:0/3358399636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34cc0047c0 msgr2=0x7f34cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.326+0000 7f34e3fff640 1 --2- 172.21.15.70:0/3358399636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34cc0047c0 0x7f34cc004c40 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c0e0 tx=0x7f34cc0057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.326+0000 7f34e37fe640 1 -- 172.21.15.70:0/3358399636 reap_dead start 2023-10-08T07:13:07.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.326+0000 7f34e3fff640 1 -- 172.21.15.70:0/3358399636 shutdown_connections 2023-10-08T07:13:07.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.326+0000 7f34e3fff640 1 -- 172.21.15.70:0/3358399636 >> 172.21.15.70:0/3358399636 conn(0x7f34e40d2960 msgr2=0x7f34e40d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:07.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.326+0000 7f34e3fff640 1 -- 172.21.15.70:0/3358399636 shutdown_connections 2023-10-08T07:13:07.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.326+0000 7f34e3fff640 1 -- 172.21.15.70:0/3358399636 wait complete. 2023-10-08T07:13:07.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.328+0000 7f34e3fff640 1 Processor -- start 2023-10-08T07:13:07.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.328+0000 7f34e3fff640 1 -- start start 2023-10-08T07:13:07.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.329+0000 7f34e3fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34cc0047c0 0x7f34e4157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.329+0000 7f34e3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e40b5410 0x7f34e40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.329+0000 7f34e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34e40cc070 0x7f34e40cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.329+0000 7f34e3fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e4158260 con 0x7f34e40cc070 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.329+0000 7f34e3fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40cad30 con 0x7f34cc0047c0 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.329+0000 7f34e3fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40cafd0 con 0x7f34e40b5410 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.329+0000 7f34e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34e40cc070 0x7f34e40cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.329+0000 7f34e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34e40cc070 0x7f34e40cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42882/0 (socket says 172.21.15.70:42882) 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.329+0000 7f34e37fe640 1 -- 172.21.15.70:0/1043312799 learned_addr learned my addr 172.21.15.70:0/1043312799 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34e2ffd640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34cc0047c0 0x7f34e4157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34e27fc640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e40b5410 0x7f34e40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34e37fe640 1 -- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e40b5410 msgr2=0x7f34e40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34e37fe640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e40b5410 0x7f34e40b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34e37fe640 1 -- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34cc0047c0 msgr2=0x7f34e4157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34e37fe640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34cc0047c0 0x7f34e4157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34e37fe640 1 -- 172.21.15.70:0/1043312799 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e40cb2b0 con 0x7f34e40cc070 2023-10-08T07:13:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34e2ffd640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34cc0047c0 0x7f34e4157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:13:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34e37fe640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34e40cc070 0x7f34e40cc4f0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f34e415e290 tx=0x7f34dc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34cb7fe640 1 -- 172.21.15.70:0/1043312799 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00e070 con 0x7f34e40cc070 2023-10-08T07:13:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34cb7fe640 1 -- 172.21.15.70:0/1043312799 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34dc011ed0 con 0x7f34e40cc070 2023-10-08T07:13:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34cb7fe640 1 -- 172.21.15.70:0/1043312799 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc003990 con 0x7f34e40cc070 2023-10-08T07:13:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.330+0000 7f34e3fff640 1 -- 172.21.15.70:0/1043312799 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e40cb590 con 0x7f34e40cc070 2023-10-08T07:13:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.331+0000 7f34e3fff640 1 -- 172.21.15.70:0/1043312799 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e40cbb50 con 0x7f34e40cc070 2023-10-08T07:13:07.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.332+0000 7f34e3fff640 1 -- 172.21.15.70:0/1043312799 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e4074150 con 0x7f34e40cc070 2023-10-08T07:13:07.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.333+0000 7f34cb7fe640 1 -- 172.21.15.70:0/1043312799 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 124) v1 ==== 89343+0+0 (secure 0 0 0) 0x7f34dc003af0 con 0x7f34e40cc070 2023-10-08T07:13:07.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.333+0000 7f34cb7fe640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f34d006afc0 0x7f34d006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.334+0000 7f34cb7fe640 1 -- 172.21.15.70:0/1043312799 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 6648+0+0 (secure 0 0 0) 0x7f34dc08a310 con 0x7f34e40cc070 2023-10-08T07:13:07.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.334+0000 7f34e2ffd640 1 -- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f34d006afc0 msgr2=0x7f34d006d480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2613103103 2023-10-08T07:13:07.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.334+0000 7f34e2ffd640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f34d006afc0 0x7f34d006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:13:07.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.338+0000 7f34cb7fe640 1 -- 172.21.15.70:0/1043312799 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34dc054de0 con 0x7f34e40cc070 2023-10-08T07:13:07.347 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.345+0000 7f1cea779040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:13:07.447 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:07.445+0000 7f429de32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:13:07.460 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.459+0000 7f1cea779040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:13:07.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.483+0000 7f34cb7fe640 1 -- 172.21.15.70:0/1043312799 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 125) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f34dc0551b0 con 0x7f34e40cc070 2023-10-08T07:13:07.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.483+0000 7f34cb7fe640 1 -- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f34d006afc0 msgr2=0x7f34d006d480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:13:07.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.483+0000 7f34cb7fe640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f34d006afc0 0x7f34d006d480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.513 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:07.511+0000 7f429de32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:13:07.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.529+0000 7f34e3fff640 1 -- 172.21.15.70:0/1043312799 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34e406a7a0 con 0x7f34e40cc070 2023-10-08T07:13:07.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.530+0000 7f34cb7fe640 1 -- 172.21.15.70:0/1043312799 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v222) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34dc086070 con 0x7f34e40cc070 2023-10-08T07:13:07.532 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:13:07.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.533+0000 7f34e3fff640 1 -- 172.21.15.70:0/1043312799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34e40cc070 msgr2=0x7f34e40cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.533+0000 7f34e3fff640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34e40cc070 0x7f34e40cc4f0 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f34e415e290 tx=0x7f34dc0115a0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.533+0000 7f34e3fff640 1 -- 172.21.15.70:0/1043312799 shutdown_connections 2023-10-08T07:13:07.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.533+0000 7f34e3fff640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:6832/2613103103,v1:172.21.15.181:6833/2613103103] conn(0x7f34d006afc0 0x7f34d006d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.533+0000 7f34e3fff640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34e40cc070 0x7f34e40cc4f0 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.533+0000 7f34e3fff640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f34e40b5410 0x7f34e40b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.533+0000 7f34e3fff640 1 --2- 172.21.15.70:0/1043312799 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f34cc0047c0 0x7f34e4157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.533+0000 7f34e3fff640 1 -- 172.21.15.70:0/1043312799 >> 172.21.15.70:0/1043312799 conn(0x7f34e40d2960 msgr2=0x7f34e406c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:07.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.533+0000 7f34e3fff640 1 -- 172.21.15.70:0/1043312799 shutdown_connections 2023-10-08T07:13:07.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.533+0000 7f34e3fff640 1 -- 172.21.15.70:0/1043312799 wait complete. 2023-10-08T07:13:07.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:13:07.552 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.551+0000 7f1c6b6c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:13:07.552 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.551+0000 7f1c6b6c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:13:07.552 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.551+0000 7f1c6b6c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:13:07.553 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.551+0000 7f1c6b6c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:13:07.553 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.551+0000 7f1c6b6c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:13:07.553 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.552+0000 7f1c6e6c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:13:07.553 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.552+0000 7f1c6e6c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:13:07.554 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.552+0000 7f1c6e6c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:13:07.554 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.552+0000 7f1c6e6c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:13:07.554 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:13:07.552+0000 7f1c6e6c9640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:13:07.634 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:13:07.633+0000 7f429de32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:13:07.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:13:07.733 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:13:07.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.794+0000 7f554bac2640 1 Processor -- start 2023-10-08T07:13:07.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.794+0000 7f554bac2640 1 -- start start 2023-10-08T07:13:07.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.795+0000 7f554bac2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f554415d0c0 0x7f55440b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.795+0000 7f554bac2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f55440b30d0 0x7f55440b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.795+0000 7f554bac2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554406ce20 con 0x7f554415cce0 2023-10-08T07:13:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.795+0000 7f554bac2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55440cf850 con 0x7f554415d0c0 2023-10-08T07:13:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.795+0000 7f554bac2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554406c060 con 0x7f55440b30d0 2023-10-08T07:13:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.795+0000 7f554905d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f554415d0c0 0x7f55440b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.795+0000 7f554985e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f554415cce0 0x7f55440cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51560/0 (socket says 172.21.15.70:51560) 2023-10-08T07:13:07.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.795+0000 7f554985e640 1 -- 172.21.15.70:0/1527965884 learned_addr learned my addr 172.21.15.70:0/1527965884 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:07.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.796+0000 7f554a05f640 1 --2- 172.21.15.70:0/1527965884 >> v2:172.21.15.181:3300/0 conn(0x7f55440b30d0 0x7f55440b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.796+0000 7f554905d640 1 -- 172.21.15.70:0/1527965884 >> v2:172.21.15.181:3300/0 conn(0x7f55440b30d0 msgr2=0x7f55440b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.796+0000 7f554905d640 1 --2- 172.21.15.70:0/1527965884 >> v2:172.21.15.181:3300/0 conn(0x7f55440b30d0 0x7f55440b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.796+0000 7f554905d640 1 -- 172.21.15.70:0/1527965884 >> v1:172.21.15.70:6789/0 conn(0x7f554415cce0 legacy=0x7f55440cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.796+0000 7f554905d640 1 -- 172.21.15.70:0/1527965884 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554406c340 con 0x7f554415d0c0 2023-10-08T07:13:07.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.796+0000 7f554885c640 1 -- 172.21.15.70:0/1527965884 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593179450 0 0) 0x7f554406ce20 con 0x7f554415cce0 2023-10-08T07:13:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.797+0000 7f554905d640 1 --2- 172.21.15.70:0/1527965884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415d0c0 0x7f55440b2530 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f55440b04b0 tx=0x7f553402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ea08e338346b66 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.798+0000 7f554885c640 1 -- 172.21.15.70:0/1527965884 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534031830 con 0x7f554415d0c0 2023-10-08T07:13:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.798+0000 7f554885c640 1 -- 172.21.15.70:0/1527965884 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5534031990 con 0x7f554415d0c0 2023-10-08T07:13:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.798+0000 7f554885c640 1 -- 172.21.15.70:0/1527965884 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534031d20 con 0x7f554415d0c0 2023-10-08T07:13:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.798+0000 7f554bac2640 1 -- 172.21.15.70:0/1527965884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415d0c0 msgr2=0x7f55440b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.798+0000 7f554bac2640 1 --2- 172.21.15.70:0/1527965884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415d0c0 0x7f55440b2530 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f55440b04b0 tx=0x7f553402b8d0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.798+0000 7f554bac2640 1 -- 172.21.15.70:0/1527965884 shutdown_connections 2023-10-08T07:13:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.798+0000 7f554bac2640 1 --2- 172.21.15.70:0/1527965884 >> v2:172.21.15.181:3300/0 conn(0x7f55440b30d0 0x7f55440b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.798+0000 7f554bac2640 1 --2- 172.21.15.70:0/1527965884 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415d0c0 0x7f55440b2530 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.798+0000 7f554bac2640 1 -- 172.21.15.70:0/1527965884 >> 172.21.15.70:0/1527965884 conn(0x7f554406e650 msgr2=0x7f554406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.799+0000 7f554bac2640 1 -- 172.21.15.70:0/1527965884 shutdown_connections 2023-10-08T07:13:07.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.799+0000 7f554bac2640 1 -- 172.21.15.70:0/1527965884 wait complete. 2023-10-08T07:13:07.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.800+0000 7f554bac2640 1 Processor -- start 2023-10-08T07:13:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.800+0000 7f554bac2640 1 -- start start 2023-10-08T07:13:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.801+0000 7f554bac2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55440b30d0 0x7f55441501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.801+0000 7f554bac2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415cce0 0x7f55440c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.801+0000 7f554bac2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55440c9c10 0x7f55440b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:07.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.801+0000 7f554bac2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55440c9070 con 0x7f554415cce0 2023-10-08T07:13:07.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.801+0000 7f554bac2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55440c7fd0 con 0x7f55440b30d0 2023-10-08T07:13:07.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.801+0000 7f554bac2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55440c82b0 con 0x7f55440c9c10 2023-10-08T07:13:07.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.802+0000 7f554905d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415cce0 0x7f55440c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.802+0000 7f554905d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415cce0 0x7f55440c96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42888/0 (socket says 172.21.15.70:42888) 2023-10-08T07:13:07.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.802+0000 7f554905d640 1 -- 172.21.15.70:0/3755431222 learned_addr learned my addr 172.21.15.70:0/3755431222 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:07.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.802+0000 7f554a05f640 1 --2- 172.21.15.70:0/3755431222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55440c9c10 0x7f55440b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.802+0000 7f554985e640 1 --2- 172.21.15.70:0/3755431222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55440b30d0 0x7f55441501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.802+0000 7f554905d640 1 -- 172.21.15.70:0/3755431222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55440c9c10 msgr2=0x7f55440b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.802+0000 7f554905d640 1 --2- 172.21.15.70:0/3755431222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55440c9c10 0x7f55440b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.802+0000 7f554905d640 1 -- 172.21.15.70:0/3755431222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55440b30d0 msgr2=0x7f55441501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.802+0000 7f554905d640 1 --2- 172.21.15.70:0/3755431222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55440b30d0 0x7f55441501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.802+0000 7f554905d640 1 -- 172.21.15.70:0/3755431222 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5534009400 con 0x7f554415cce0 2023-10-08T07:13:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.803+0000 7f554905d640 1 --2- 172.21.15.70:0/3755431222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415cce0 0x7f55440c96d0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f5534000c00 tx=0x7f55340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:07.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.803+0000 7f553a7fc640 1 -- 172.21.15.70:0/3755431222 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55340317d0 con 0x7f554415cce0 2023-10-08T07:13:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.803+0000 7f553a7fc640 1 -- 172.21.15.70:0/3755431222 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553403aa60 con 0x7f554415cce0 2023-10-08T07:13:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.803+0000 7f553a7fc640 1 -- 172.21.15.70:0/3755431222 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553403ae10 con 0x7f554415cce0 2023-10-08T07:13:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.803+0000 7f554bac2640 1 -- 172.21.15.70:0/3755431222 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55440c8590 con 0x7f554415cce0 2023-10-08T07:13:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.803+0000 7f554bac2640 1 -- 172.21.15.70:0/3755431222 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55440c8b50 con 0x7f554415cce0 2023-10-08T07:13:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.804+0000 7f554bac2640 1 -- 172.21.15.70:0/3755431222 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55440c8d40 con 0x7f554415cce0 2023-10-08T07:13:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.805+0000 7f553a7fc640 1 -- 172.21.15.70:0/3755431222 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 125) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f553404d450 con 0x7f554415cce0 2023-10-08T07:13:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.806+0000 7f553a7fc640 1 -- 172.21.15.70:0/3755431222 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6948+0+0 (secure 0 0 0) 0x7f55340ab640 con 0x7f554415cce0 2023-10-08T07:13:07.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.810+0000 7f553a7fc640 1 -- 172.21.15.70:0/3755431222 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5534076660 con 0x7f554415cce0 2023-10-08T07:13:07.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.986+0000 7f554bac2640 1 -- 172.21.15.70:0/3755431222 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f554404e160 con 0x7f554415cce0 2023-10-08T07:13:07.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.987+0000 7f553a7fc640 1 -- 172.21.15.70:0/3755431222 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v222) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5534035020 con 0x7f554415cce0 2023-10-08T07:13:07.988 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:13:07.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.989+0000 7f554bac2640 1 -- 172.21.15.70:0/3755431222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415cce0 msgr2=0x7f55440c96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:07.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.989+0000 7f554bac2640 1 --2- 172.21.15.70:0/3755431222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415cce0 0x7f55440c96d0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f5534000c00 tx=0x7f55340046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.990+0000 7f554bac2640 1 -- 172.21.15.70:0/3755431222 shutdown_connections 2023-10-08T07:13:07.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.990+0000 7f554bac2640 1 --2- 172.21.15.70:0/3755431222 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f55440c9c10 0x7f55440b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.990+0000 7f554bac2640 1 --2- 172.21.15.70:0/3755431222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f554415cce0 0x7f55440c96d0 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.990+0000 7f554bac2640 1 --2- 172.21.15.70:0/3755431222 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f55440b30d0 0x7f55441501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:07.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.990+0000 7f554bac2640 1 -- 172.21.15.70:0/3755431222 >> 172.21.15.70:0/3755431222 conn(0x7f554406e650 msgr2=0x7f55440c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:07.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.990+0000 7f554bac2640 1 -- 172.21.15.70:0/3755431222 shutdown_connections 2023-10-08T07:13:07.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:07.990+0000 7f554bac2640 1 -- 172.21.15.70:0/3755431222 wait complete. 2023-10-08T07:13:08.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:13:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.210+0000 7f03c82d7640 1 Processor -- start 2023-10-08T07:13:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.210+0000 7f03c82d7640 1 -- start start 2023-10-08T07:13:08.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.212+0000 7f03c82d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03c00b21f0 0x7f03c00c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.212+0000 7f03c82d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f03c00ca1a0 0x7f03c006d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.212+0000 7f03c82d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c006c3a0 con 0x7f03c00702b0 2023-10-08T07:13:08.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.212+0000 7f03c82d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c006c640 con 0x7f03c00b21f0 2023-10-08T07:13:08.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.212+0000 7f03c82d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03c006c920 con 0x7f03c00ca1a0 2023-10-08T07:13:08.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.212+0000 7f03c6073640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f03c00702b0 0x7f03c0070690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51564/0 (socket says 172.21.15.70:51564) 2023-10-08T07:13:08.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.212+0000 7f03c6073640 1 -- 172.21.15.70:0/3143639370 learned_addr learned my addr 172.21.15.70:0/3143639370 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:08.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.212+0000 7f03c5872640 1 --2- 172.21.15.70:0/3143639370 >> v2:172.21.15.70:3300/0 conn(0x7f03c00b21f0 0x7f03c00c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:08.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.212+0000 7f03c6874640 1 --2- 172.21.15.70:0/3143639370 >> v2:172.21.15.181:3300/0 conn(0x7f03c00ca1a0 0x7f03c006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:08.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.213+0000 7f03c5872640 1 -- 172.21.15.70:0/3143639370 >> v2:172.21.15.181:3300/0 conn(0x7f03c00ca1a0 msgr2=0x7f03c006d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:08.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.213+0000 7f03c5872640 1 --2- 172.21.15.70:0/3143639370 >> v2:172.21.15.181:3300/0 conn(0x7f03c00ca1a0 0x7f03c006d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.213+0000 7f03c5872640 1 -- 172.21.15.70:0/3143639370 >> v1:172.21.15.70:6789/0 conn(0x7f03c00702b0 legacy=0x7f03c0070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.213+0000 7f03c5071640 1 -- 172.21.15.70:0/3143639370 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491134271 0 0) 0x7f03c006c3a0 con 0x7f03c00702b0 2023-10-08T07:13:08.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.213+0000 7f03c5872640 1 -- 172.21.15.70:0/3143639370 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c006cc00 con 0x7f03c00b21f0 2023-10-08T07:13:08.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.213+0000 7f03c6874640 1 --2- 172.21.15.70:0/3143639370 >> v2:172.21.15.181:3300/0 conn(0x7f03c00ca1a0 0x7f03c006d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:08.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.214+0000 7f03c5872640 1 --2- 172.21.15.70:0/3143639370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00b21f0 0x7f03c00c9c60 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f03c00b1350 tx=0x7f03b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafda8c31ab24422 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:08.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.214+0000 7f03c5071640 1 -- 172.21.15.70:0/3143639370 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0031830 con 0x7f03c00b21f0 2023-10-08T07:13:08.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.214+0000 7f03c5071640 1 -- 172.21.15.70:0/3143639370 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b0031990 con 0x7f03c00b21f0 2023-10-08T07:13:08.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.214+0000 7f03c5071640 1 -- 172.21.15.70:0/3143639370 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0031d20 con 0x7f03c00b21f0 2023-10-08T07:13:08.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.214+0000 7f03c82d7640 1 -- 172.21.15.70:0/3143639370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00b21f0 msgr2=0x7f03c00c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.214+0000 7f03c82d7640 1 --2- 172.21.15.70:0/3143639370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00b21f0 0x7f03c00c9c60 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f03c00b1350 tx=0x7f03b002b8d0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.215+0000 7f03c82d7640 1 -- 172.21.15.70:0/3143639370 shutdown_connections 2023-10-08T07:13:08.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.215+0000 7f03c82d7640 1 --2- 172.21.15.70:0/3143639370 >> v2:172.21.15.181:3300/0 conn(0x7f03c00ca1a0 0x7f03c006d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.215+0000 7f03c82d7640 1 --2- 172.21.15.70:0/3143639370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00b21f0 0x7f03c00c9c60 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.215+0000 7f03c82d7640 1 -- 172.21.15.70:0/3143639370 >> 172.21.15.70:0/3143639370 conn(0x7f03c006a5d0 msgr2=0x7f03c00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:08.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.215+0000 7f03c82d7640 1 -- 172.21.15.70:0/3143639370 shutdown_connections 2023-10-08T07:13:08.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.215+0000 7f03c82d7640 1 -- 172.21.15.70:0/3143639370 wait complete. 2023-10-08T07:13:08.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.217+0000 7f03c82d7640 1 Processor -- start 2023-10-08T07:13:08.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.217+0000 7f03c82d7640 1 -- start start 2023-10-08T07:13:08.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c82d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00702b0 0x7f03c0154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c82d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03c00b21f0 0x7f03c00ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c82d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03c00ca1a0 0x7f03c00ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c82d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00cec00 con 0x7f03c00702b0 2023-10-08T07:13:08.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c82d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00ceee0 con 0x7f03c00ca1a0 2023-10-08T07:13:08.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c82d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00aca10 con 0x7f03c00b21f0 2023-10-08T07:13:08.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c6073640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00702b0 0x7f03c0154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:08.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c6073640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00702b0 0x7f03c0154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42906/0 (socket says 172.21.15.70:42906) 2023-10-08T07:13:08.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c6073640 1 -- 172.21.15.70:0/2027559830 learned_addr learned my addr 172.21.15.70:0/2027559830 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:08.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c6073640 1 -- 172.21.15.70:0/2027559830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03c00b21f0 msgr2=0x7f03c00ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c6073640 1 --2- 172.21.15.70:0/2027559830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03c00b21f0 0x7f03c00ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c6073640 1 -- 172.21.15.70:0/2027559830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03c00ca1a0 msgr2=0x7f03c00ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c6073640 1 --2- 172.21.15.70:0/2027559830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03c00ca1a0 0x7f03c00ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.218+0000 7f03c6073640 1 -- 172.21.15.70:0/2027559830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b0009400 con 0x7f03c00702b0 2023-10-08T07:13:08.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.219+0000 7f03c6073640 1 --2- 172.21.15.70:0/2027559830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00702b0 0x7f03c0154aa0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00c4d0 tx=0x7f03bc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:08.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.219+0000 7f03b6ffd640 1 -- 172.21.15.70:0/2027559830 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0074e0 con 0x7f03c00702b0 2023-10-08T07:13:08.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.219+0000 7f03b6ffd640 1 -- 172.21.15.70:0/2027559830 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc007660 con 0x7f03c00702b0 2023-10-08T07:13:08.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.219+0000 7f03b6ffd640 1 -- 172.21.15.70:0/2027559830 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc004a20 con 0x7f03c00702b0 2023-10-08T07:13:08.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.219+0000 7f03c82d7640 1 -- 172.21.15.70:0/2027559830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c00accf0 con 0x7f03c00702b0 2023-10-08T07:13:08.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.219+0000 7f03c82d7640 1 -- 172.21.15.70:0/2027559830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c00ce840 con 0x7f03c00702b0 2023-10-08T07:13:08.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.220+0000 7f03c82d7640 1 -- 172.21.15.70:0/2027559830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c00cf100 con 0x7f03c00702b0 2023-10-08T07:13:08.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.221+0000 7f03b6ffd640 1 -- 172.21.15.70:0/2027559830 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 125) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f03bc004b80 con 0x7f03c00702b0 2023-10-08T07:13:08.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.223+0000 7f03b6ffd640 1 -- 172.21.15.70:0/2027559830 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6948+0+0 (secure 0 0 0) 0x7f03bc088810 con 0x7f03c00702b0 2023-10-08T07:13:08.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.227+0000 7f03b6ffd640 1 -- 172.21.15.70:0/2027559830 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03bc053740 con 0x7f03c00702b0 2023-10-08T07:13:08.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.409+0000 7f03c82d7640 1 -- 172.21.15.70:0/2027559830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03c004e160 con 0x7f03c00702b0 2023-10-08T07:13:08.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.410+0000 7f03b6ffd640 1 -- 172.21.15.70:0/2027559830 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v222) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03bc0180a0 con 0x7f03c00702b0 2023-10-08T07:13:08.412 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:13:08.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.412+0000 7f03c82d7640 1 -- 172.21.15.70:0/2027559830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00702b0 msgr2=0x7f03c0154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.412+0000 7f03c82d7640 1 --2- 172.21.15.70:0/2027559830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00702b0 0x7f03c0154aa0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00c4d0 tx=0x7f03bc00c990 comp rx=0 tx=0).stop 2023-10-08T07:13:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.412+0000 7f03c82d7640 1 -- 172.21.15.70:0/2027559830 shutdown_connections 2023-10-08T07:13:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.412+0000 7f03c82d7640 1 --2- 172.21.15.70:0/2027559830 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f03c00ca1a0 0x7f03c00ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.412+0000 7f03c82d7640 1 --2- 172.21.15.70:0/2027559830 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f03c00b21f0 0x7f03c00ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.412+0000 7f03c82d7640 1 --2- 172.21.15.70:0/2027559830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03c00702b0 0x7f03c0154aa0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.412+0000 7f03c82d7640 1 -- 172.21.15.70:0/2027559830 >> 172.21.15.70:0/2027559830 conn(0x7f03c006a5d0 msgr2=0x7f03c015af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.412+0000 7f03c82d7640 1 -- 172.21.15.70:0/2027559830 shutdown_connections 2023-10-08T07:13:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.412+0000 7f03c82d7640 1 -- 172.21.15.70:0/2027559830 wait complete. 2023-10-08T07:13:08.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:13:08.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.640+0000 7fe92a76e640 1 Processor -- start 2023-10-08T07:13:08.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.640+0000 7fe92a76e640 1 -- start start 2023-10-08T07:13:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.642+0000 7fe92a76e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe92406ad00 0x7fe92406b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.642+0000 7fe92a76e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9240c9850 con 0x7fe924070db0 2023-10-08T07:13:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.642+0000 7fe92a76e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92406a870 con 0x7fe924071190 2023-10-08T07:13:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.642+0000 7fe92a76e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92415eb90 con 0x7fe92406ad00 2023-10-08T07:13:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.642+0000 7fe923fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe924070db0 0x7fe92415dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51568/0 (socket says 172.21.15.70:51568) 2023-10-08T07:13:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.642+0000 7fe923fff640 1 -- 172.21.15.70:0/1791716824 learned_addr learned my addr 172.21.15.70:0/1791716824 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:08.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.642+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434935504 0 0) 0x7fe9240c9850 con 0x7fe924070db0 2023-10-08T07:13:08.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe910003520 con 0x7fe924070db0 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe928d0b640 1 --2- 172.21.15.70:0/1791716824 >> v2:172.21.15.181:3300/0 conn(0x7fe92406ad00 0x7fe92406b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2593215047 0 0) 0x7fe910003520 con 0x7fe924070db0 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 >> v2:172.21.15.181:3300/0 conn(0x7fe92406ad00 msgr2=0x7fe92406b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 --2- 172.21.15.70:0/1791716824 >> v2:172.21.15.181:3300/0 conn(0x7fe92406ad00 0x7fe92406b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 >> v1:172.21.15.181:6789/0 conn(0x7fe924071190 legacy=0x7fe92415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9240cd640 con 0x7fe924070db0 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe9140019f0 con 0x7fe924070db0 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 >> v1:172.21.15.70:6789/0 conn(0x7fe924070db0 legacy=0x7fe92415dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe910003970 0x7fe910003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9100047c0 0x7fe910004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe910005180 0x7fe91000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92415eb90 con 0x7fe910003970 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91000c030 con 0x7fe910005180 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91000c1f0 con 0x7fe9100047c0 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.643+0000 7fe923fff640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe910003970 0x7fe910003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.644+0000 7fe928d0b640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe910005180 0x7fe91000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.644+0000 7fe9237fe640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9100047c0 0x7fe910004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.644+0000 7fe923fff640 1 -- 172.21.15.70:0/1791716824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9100047c0 msgr2=0x7fe910004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.644+0000 7fe923fff640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9100047c0 0x7fe910004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.644+0000 7fe923fff640 1 -- 172.21.15.70:0/1791716824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe910005180 msgr2=0x7fe91000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.644+0000 7fe923fff640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe910005180 0x7fe91000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.644+0000 7fe923fff640 1 -- 172.21.15.70:0/1791716824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe910003520 con 0x7fe910003970 2023-10-08T07:13:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.644+0000 7fe928d0b640 1 -- 172.21.15.70:0/1791716824 reap_dead start 2023-10-08T07:13:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.644+0000 7fe9237fe640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9100047c0 0x7fe910004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.645+0000 7fe923fff640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe910003970 0x7fe910003d90 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fe91400a830 tx=0x7fe9100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580743a829ba291a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.645+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140072d0 con 0x7fe910003970 2023-10-08T07:13:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.645+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91402f070 con 0x7fe910003970 2023-10-08T07:13:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.645+0000 7fe922ffd640 1 -- 172.21.15.70:0/1791716824 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9140076f0 con 0x7fe910003970 2023-10-08T07:13:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.646+0000 7fe92a76e640 1 -- 172.21.15.70:0/1791716824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe910003970 msgr2=0x7fe910003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.646+0000 7fe92a76e640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe910003970 0x7fe910003d90 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fe91400a830 tx=0x7fe9100057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.646+0000 7fe92a76e640 1 -- 172.21.15.70:0/1791716824 shutdown_connections 2023-10-08T07:13:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.646+0000 7fe92a76e640 1 --2- 172.21.15.70:0/1791716824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe910003970 0x7fe910003d90 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.646+0000 7fe92a76e640 1 -- 172.21.15.70:0/1791716824 >> 172.21.15.70:0/1791716824 conn(0x7fe9240b1480 msgr2=0x7fe92406cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.646+0000 7fe92a76e640 1 -- 172.21.15.70:0/1791716824 shutdown_connections 2023-10-08T07:13:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.646+0000 7fe92a76e640 1 -- 172.21.15.70:0/1791716824 wait complete. 2023-10-08T07:13:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.648+0000 7fe92a76e640 1 Processor -- start 2023-10-08T07:13:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.648+0000 7fe92a76e640 1 -- start start 2023-10-08T07:13:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.649+0000 7fe92a76e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe910003970 0x7fe924152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.649+0000 7fe92a76e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9240c6d50 0x7fe9240b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.649+0000 7fe92a76e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9240b4580 0x7fe9240c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.649+0000 7fe92a76e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9240cb1d0 con 0x7fe9240b4580 2023-10-08T07:13:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.649+0000 7fe92a76e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9240c71d0 con 0x7fe9240c6d50 2023-10-08T07:13:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.649+0000 7fe92a76e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9241531b0 con 0x7fe910003970 2023-10-08T07:13:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.649+0000 7fe928d0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9240b4580 0x7fe9240c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.649+0000 7fe928d0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9240b4580 0x7fe9240c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42926/0 (socket says 172.21.15.70:42926) 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.649+0000 7fe928d0b640 1 -- 172.21.15.70:0/2701346434 learned_addr learned my addr 172.21.15.70:0/2701346434 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe928d0b640 1 -- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe910003970 msgr2=0x7fe924152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe928d0b640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe910003970 0x7fe924152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe928d0b640 1 -- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9240c6d50 msgr2=0x7fe9240b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe928d0b640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9240c6d50 0x7fe9240b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe928d0b640 1 -- 172.21.15.70:0/2701346434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9240ca830 con 0x7fe9240b4580 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe928d0b640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9240b4580 0x7fe9240c8630 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fe924160810 tx=0x7fe91c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe920ff9640 1 -- 172.21.15.70:0/2701346434 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c015930 con 0x7fe9240b4580 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe920ff9640 1 -- 172.21.15.70:0/2701346434 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c015a90 con 0x7fe9240b4580 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe920ff9640 1 -- 172.21.15.70:0/2701346434 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c015e40 con 0x7fe9240b4580 2023-10-08T07:13:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe92a76e640 1 -- 172.21.15.70:0/2701346434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9240cab10 con 0x7fe9240b4580 2023-10-08T07:13:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.650+0000 7fe92a76e640 1 -- 172.21.15.70:0/2701346434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9240b2e00 con 0x7fe9240b4580 2023-10-08T07:13:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.651+0000 7fe92a76e640 1 -- 172.21.15.70:0/2701346434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92406a1a0 con 0x7fe9240b4580 2023-10-08T07:13:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.652+0000 7fe920ff9640 1 -- 172.21.15.70:0/2701346434 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 126) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fe91c021460 con 0x7fe9240b4580 2023-10-08T07:13:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.653+0000 7fe920ff9640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe90406acc0 0x7fe90406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.653+0000 7fe920ff9640 1 -- 172.21.15.70:0/2701346434 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6948+0+0 (secure 0 0 0) 0x7fe91c089b40 con 0x7fe9240b4580 2023-10-08T07:13:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.654+0000 7fe923fff640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe90406acc0 0x7fe90406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.655+0000 7fe923fff640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe90406acc0 0x7fe90406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe914034000 tx=0x7fe91402f000 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.658+0000 7fe920ff9640 1 -- 172.21.15.70:0/2701346434 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe91c054a50 con 0x7fe9240b4580 2023-10-08T07:13:08.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.886+0000 7fe92a76e640 1 -- 172.21.15.70:0/2701346434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe924067490 con 0x7fe9240b4580 2023-10-08T07:13:08.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.887+0000 7fe920ff9640 1 -- 172.21.15.70:0/2701346434 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7fe91c006c30 con 0x7fe9240b4580 2023-10-08T07:13:08.889 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:08.889 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/4185609199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4185609199},{"type":"v1","addr":"172.21.15.70:6833","nonce":4185609199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3102206290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3102206290},{"type":"v1","addr":"172.21.15.181:6837","nonce":3102206290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1824427895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1824427895},{"type":"v1","addr":"172.21.15.181:6835","nonce":1824427895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:10:44.128443+0000","modified":"2023-10-08T07:10:47.166174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/2355440757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2355440757},{"type":"v1","addr":"172.21.15.70:6835","nonce":2355440757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-08T07:13:08.889 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 99 2023-10-08T07:13:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 -- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe90406acc0 msgr2=0x7fe90406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe90406acc0 0x7fe90406d180 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe914034000 tx=0x7fe91402f000 comp rx=0 tx=0).stop 2023-10-08T07:13:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 -- 172.21.15.70:0/2701346434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9240b4580 msgr2=0x7fe9240c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9240b4580 0x7fe9240c8630 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fe924160810 tx=0x7fe91c011920 comp rx=0 tx=0).stop 2023-10-08T07:13:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 -- 172.21.15.70:0/2701346434 shutdown_connections 2023-10-08T07:13:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe90406acc0 0x7fe90406d180 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9240b4580 0x7fe9240c8630 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9240c6d50 0x7fe9240b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 --2- 172.21.15.70:0/2701346434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe910003970 0x7fe924152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 -- 172.21.15.70:0/2701346434 >> 172.21.15.70:0/2701346434 conn(0x7fe9240b1480 msgr2=0x7fe92415ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 -- 172.21.15.70:0/2701346434 shutdown_connections 2023-10-08T07:13:08.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:08.890+0000 7fe92a76e640 1 -- 172.21.15.70:0/2701346434 wait complete. 2023-10-08T07:13:08.911 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:09.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.123+0000 7f09f0090640 1 Processor -- start 2023-10-08T07:13:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.124+0000 7f09f0090640 1 -- start start 2023-10-08T07:13:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.125+0000 7f09f0090640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09e80c90b0 0x7f09e80cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.125+0000 7f09f0090640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80cb3b0 0x7f09e815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.125+0000 7f09f0090640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e815b2b0 con 0x7f09e815ab40 2023-10-08T07:13:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.125+0000 7f09f0090640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e815bf30 con 0x7f09e80c90b0 2023-10-08T07:13:09.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.125+0000 7f09f0090640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e80cb820 con 0x7f09e80cb3b0 2023-10-08T07:13:09.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.126+0000 7f09ee62d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80cb3b0 0x7f09e815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.126+0000 7f09ede2c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f09e815ab40 0x7f09e80ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:47372/0 (socket says 172.21.15.70:47372) 2023-10-08T07:13:09.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.126+0000 7f09ed62b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f09e80c90b0 0x7f09e80cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.126+0000 7f09ede2c640 1 -- 172.21.15.70:0/875683336 learned_addr learned my addr 172.21.15.70:0/875683336 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.127+0000 7f09ee62d640 1 -- 172.21.15.70:0/875683336 >> v2:172.21.15.181:3300/0 conn(0x7f09e80c90b0 msgr2=0x7f09e80cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.127+0000 7f09ee62d640 1 --2- 172.21.15.70:0/875683336 >> v2:172.21.15.181:3300/0 conn(0x7f09e80c90b0 0x7f09e80cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.127+0000 7f09ee62d640 1 -- 172.21.15.70:0/875683336 >> v1:172.21.15.181:6789/0 conn(0x7f09e815ab40 legacy=0x7f09e80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.127+0000 7f09ee62d640 1 -- 172.21.15.70:0/875683336 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e815b590 con 0x7f09e80cb3b0 2023-10-08T07:13:09.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.127+0000 7f09ece2a640 1 -- 172.21.15.70:0/875683336 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180084558 0 0) 0x7f09e815b2b0 con 0x7f09e815ab40 2023-10-08T07:13:09.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.128+0000 7f09ee62d640 1 --2- 172.21.15.70:0/875683336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80cb3b0 0x7f09e815e790 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f09e00097a0 tx=0x7f09e002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3197313b33e62238 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:09.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.129+0000 7f09ece2a640 1 -- 172.21.15.70:0/875683336 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e002d040 con 0x7f09e80cb3b0 2023-10-08T07:13:09.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.129+0000 7f09ece2a640 1 -- 172.21.15.70:0/875683336 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e0031810 con 0x7f09e80cb3b0 2023-10-08T07:13:09.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.129+0000 7f09ece2a640 1 -- 172.21.15.70:0/875683336 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e0031ba0 con 0x7f09e80cb3b0 2023-10-08T07:13:09.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.130+0000 7f09f0090640 1 -- 172.21.15.70:0/875683336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80cb3b0 msgr2=0x7f09e815e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.130+0000 7f09f0090640 1 --2- 172.21.15.70:0/875683336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80cb3b0 0x7f09e815e790 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f09e00097a0 tx=0x7f09e002b8b0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.130+0000 7f09f0090640 1 -- 172.21.15.70:0/875683336 shutdown_connections 2023-10-08T07:13:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.130+0000 7f09f0090640 1 --2- 172.21.15.70:0/875683336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e80cb3b0 0x7f09e815e790 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.130+0000 7f09f0090640 1 --2- 172.21.15.70:0/875683336 >> v2:172.21.15.181:3300/0 conn(0x7f09e80c90b0 0x7f09e80cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.130+0000 7f09f0090640 1 -- 172.21.15.70:0/875683336 >> 172.21.15.70:0/875683336 conn(0x7f09e806a380 msgr2=0x7f09e80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.130+0000 7f09f0090640 1 -- 172.21.15.70:0/875683336 shutdown_connections 2023-10-08T07:13:09.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.130+0000 7f09f0090640 1 -- 172.21.15.70:0/875683336 wait complete. 2023-10-08T07:13:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.132+0000 7f09f0090640 1 Processor -- start 2023-10-08T07:13:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.132+0000 7f09f0090640 1 -- start start 2023-10-08T07:13:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09f0090640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80c90b0 0x7f09e8149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09f0090640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09e80cb3b0 0x7f09e80b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09f0090640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e815ab40 0x7f09e81534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09f0090640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e8158130 con 0x7f09e80cb3b0 2023-10-08T07:13:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09f0090640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e81583d0 con 0x7f09e80c90b0 2023-10-08T07:13:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09f0090640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e81586b0 con 0x7f09e815ab40 2023-10-08T07:13:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09ed62b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09e80cb3b0 0x7f09e80b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09ed62b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09e80cb3b0 0x7f09e80b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42940/0 (socket says 172.21.15.70:42940) 2023-10-08T07:13:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09ed62b640 1 -- 172.21.15.70:0/3662468093 learned_addr learned my addr 172.21.15.70:0/3662468093 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09ede2c640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80c90b0 0x7f09e8149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.133+0000 7f09ee62d640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e815ab40 0x7f09e81534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09ed62b640 1 -- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e815ab40 msgr2=0x7f09e81534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09ed62b640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e815ab40 0x7f09e81534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09ed62b640 1 -- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80c90b0 msgr2=0x7f09e8149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09ed62b640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80c90b0 0x7f09e8149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09ed62b640 1 -- 172.21.15.70:0/3662468093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e0009400 con 0x7f09e80cb3b0 2023-10-08T07:13:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09ede2c640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80c90b0 0x7f09e8149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09ee62d640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e815ab40 0x7f09e81534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09ed62b640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09e80cb3b0 0x7f09e80b1480 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f09d800e840 tx=0x7f09d800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:09.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09de7fc640 1 -- 172.21.15.70:0/3662468093 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d80094f0 con 0x7f09e80cb3b0 2023-10-08T07:13:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09de7fc640 1 -- 172.21.15.70:0/3662468093 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d800c940 con 0x7f09e80cb3b0 2023-10-08T07:13:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09f0090640 1 -- 172.21.15.70:0/3662468093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e8158990 con 0x7f09e80cb3b0 2023-10-08T07:13:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09de7fc640 1 -- 172.21.15.70:0/3662468093 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d800ccd0 con 0x7f09e80cb3b0 2023-10-08T07:13:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.134+0000 7f09f0090640 1 -- 172.21.15.70:0/3662468093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e8158f50 con 0x7f09e80cb3b0 2023-10-08T07:13:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.135+0000 7f09f0090640 1 -- 172.21.15.70:0/3662468093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e8067490 con 0x7f09e80cb3b0 2023-10-08T07:13:09.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.136+0000 7f09de7fc640 1 -- 172.21.15.70:0/3662468093 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 126) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f09d8013470 con 0x7f09e80cb3b0 2023-10-08T07:13:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.138+0000 7f09de7fc640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f09cc06acc0 0x7f09cc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.138+0000 7f09de7fc640 1 -- 172.21.15.70:0/3662468093 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6948+0+0 (secure 0 0 0) 0x7f09d80898b0 con 0x7f09e80cb3b0 2023-10-08T07:13:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.138+0000 7f09ede2c640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f09cc06acc0 0x7f09cc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.139+0000 7f09ede2c640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f09cc06acc0 0x7f09cc06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f09e40012e0 tx=0x7f09e4009040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:09.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.142+0000 7f09de7fc640 1 -- 172.21.15.70:0/3662468093 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09d80546f0 con 0x7f09e80cb3b0 2023-10-08T07:13:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.334+0000 7f09f0090640 1 -- 172.21.15.70:0/3662468093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09e80b0ef0 con 0x7f09e80cb3b0 2023-10-08T07:13:09.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.336+0000 7f09de7fc640 1 -- 172.21.15.70:0/3662468093 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) v1 ==== 74+0+18118 (secure 0 0 0) 0x7f09d8057d10 con 0x7f09e80cb3b0 2023-10-08T07:13:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:09.339 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":222,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:07.464415+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:10:41.917258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-08T07:10:43.125413+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":221,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.28","mappings":[{"from":1,"to":0}]},{"pgid":"26.36","mappings":[{"from":3,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1545365181":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/4119540053":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/2433436977":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/2511790348":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/294214378":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/3667065295":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/1205099199":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/3162325425":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6833/2638163552":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6832/2638163552":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/785931869":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6833/2613103103":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/265797938":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6832/2613103103":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/2110654177":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/39996703":"2023-10-09T07:10:58.332667+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:09.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 -- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f09cc06acc0 msgr2=0x7f09cc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f09cc06acc0 0x7f09cc06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f09e40012e0 tx=0x7f09e4009040 comp rx=0 tx=0).stop 2023-10-08T07:13:09.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 -- 172.21.15.70:0/3662468093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09e80cb3b0 msgr2=0x7f09e80b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09e80cb3b0 0x7f09e80b1480 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f09d800e840 tx=0x7f09d800ed00 comp rx=0 tx=0).stop 2023-10-08T07:13:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 -- 172.21.15.70:0/3662468093 shutdown_connections 2023-10-08T07:13:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f09e815ab40 0x7f09e81534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f09cc06acc0 0x7f09cc06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09e80cb3b0 0x7f09e80b1480 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 --2- 172.21.15.70:0/3662468093 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f09e80c90b0 0x7f09e8149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 -- 172.21.15.70:0/3662468093 >> 172.21.15.70:0/3662468093 conn(0x7f09e806a380 msgr2=0x7f09e80701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 -- 172.21.15.70:0/3662468093 shutdown_connections 2023-10-08T07:13:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.339+0000 7f09f0090640 1 -- 172.21.15.70:0/3662468093 wait complete. 2023-10-08T07:13:09.361 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T07:13:09.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:13:09.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.571+0000 7f1576311640 1 Processor -- start 2023-10-08T07:13:09.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.572+0000 7f1576311640 1 -- start start 2023-10-08T07:13:09.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.573+0000 7f1576311640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f157015ab30 0x7f157015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.573+0000 7f1576311640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f15700adc70 0x7f15700b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.573+0000 7f1576311640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157015f7d0 con 0x7f157015af10 2023-10-08T07:13:09.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.573+0000 7f1576311640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157015f210 con 0x7f157015ab30 2023-10-08T07:13:09.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.573+0000 7f1576311640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157015f4f0 con 0x7f15700adc70 2023-10-08T07:13:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.574+0000 7f157530f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f157015ab30 0x7f157015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.574+0000 7f157530f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f157015ab30 0x7f157015e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42946/0 (socket says 172.21.15.70:42946) 2023-10-08T07:13:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.574+0000 7f157530f640 1 -- 172.21.15.70:0/3522481527 learned_addr learned my addr 172.21.15.70:0/3522481527 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.574+0000 7f1574b0e640 1 --2- 172.21.15.70:0/3522481527 >> v2:172.21.15.181:3300/0 conn(0x7f15700adc70 0x7f15700b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.574+0000 7f157530f640 1 -- 172.21.15.70:0/3522481527 >> v2:172.21.15.181:3300/0 conn(0x7f15700adc70 msgr2=0x7f15700b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.574+0000 7f157530f640 1 --2- 172.21.15.70:0/3522481527 >> v2:172.21.15.181:3300/0 conn(0x7f15700adc70 0x7f15700b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.574+0000 7f157530f640 1 -- 172.21.15.70:0/3522481527 >> v1:172.21.15.181:6789/0 conn(0x7f157015af10 legacy=0x7f15700701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.575+0000 7f157530f640 1 -- 172.21.15.70:0/3522481527 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157006c550 con 0x7f157015ab30 2023-10-08T07:13:09.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.576+0000 7f157530f640 1 --2- 172.21.15.70:0/3522481527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015ab30 0x7f157015e8a0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f15700c56c0 tx=0x7f156c02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc4120f0feb9794 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:09.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.576+0000 7f1567fff640 1 -- 172.21.15.70:0/3522481527 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c030870 con 0x7f157015ab30 2023-10-08T07:13:09.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.576+0000 7f1567fff640 1 -- 172.21.15.70:0/3522481527 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c0309d0 con 0x7f157015ab30 2023-10-08T07:13:09.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.576+0000 7f1567fff640 1 -- 172.21.15.70:0/3522481527 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c030d60 con 0x7f157015ab30 2023-10-08T07:13:09.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.576+0000 7f1576311640 1 -- 172.21.15.70:0/3522481527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015ab30 msgr2=0x7f157015e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.576+0000 7f1576311640 1 --2- 172.21.15.70:0/3522481527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015ab30 0x7f157015e8a0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f15700c56c0 tx=0x7f156c02b9b0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.577+0000 7f1576311640 1 -- 172.21.15.70:0/3522481527 shutdown_connections 2023-10-08T07:13:09.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.577+0000 7f1576311640 1 --2- 172.21.15.70:0/3522481527 >> v2:172.21.15.181:3300/0 conn(0x7f15700adc70 0x7f15700b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.577+0000 7f1576311640 1 --2- 172.21.15.70:0/3522481527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015ab30 0x7f157015e8a0 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.577+0000 7f1576311640 1 -- 172.21.15.70:0/3522481527 >> 172.21.15.70:0/3522481527 conn(0x7f15700cb0c0 msgr2=0x7f15700cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:09.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.577+0000 7f1576311640 1 -- 172.21.15.70:0/3522481527 shutdown_connections 2023-10-08T07:13:09.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.577+0000 7f1576311640 1 -- 172.21.15.70:0/3522481527 wait complete. 2023-10-08T07:13:09.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.578+0000 7f1576311640 1 Processor -- start 2023-10-08T07:13:09.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.579+0000 7f1576311640 1 -- start start 2023-10-08T07:13:09.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f1576311640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15700adc70 0x7f15700b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f1576311640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157015ab30 0x7f157015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f1576311640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015af10 0x7f1570156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f1576311640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157015cdb0 con 0x7f157015af10 2023-10-08T07:13:09.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f1576311640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15700c6bc0 con 0x7f157015ab30 2023-10-08T07:13:09.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f1576311640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157015c510 con 0x7f15700adc70 2023-10-08T07:13:09.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f1575b10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015af10 0x7f1570156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f1575b10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015af10 0x7f1570156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42958/0 (socket says 172.21.15.70:42958) 2023-10-08T07:13:09.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f1575b10640 1 -- 172.21.15.70:0/4011653365 learned_addr learned my addr 172.21.15.70:0/4011653365 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:09.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f157530f640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15700adc70 0x7f15700b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.580+0000 7f1574b0e640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157015ab30 0x7f157015ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1575b10640 1 -- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15700adc70 msgr2=0x7f15700b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1575b10640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15700adc70 0x7f15700b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1575b10640 1 -- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157015ab30 msgr2=0x7f157015ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1575b10640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157015ab30 0x7f157015ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1575b10640 1 -- 172.21.15.70:0/4011653365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0094d0 con 0x7f157015af10 2023-10-08T07:13:09.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1574b0e640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157015ab30 0x7f157015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:09.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f157530f640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15700adc70 0x7f15700b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:09.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1575b10640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015af10 0x7f1570156150 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f156800c730 tx=0x7f156800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:09.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1565ffb640 1 -- 172.21.15.70:0/4011653365 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568003b90 con 0x7f157015af10 2023-10-08T07:13:09.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1565ffb640 1 -- 172.21.15.70:0/4011653365 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1568003cf0 con 0x7f157015af10 2023-10-08T07:13:09.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1576311640 1 -- 172.21.15.70:0/4011653365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157015c7f0 con 0x7f157015af10 2023-10-08T07:13:09.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1565ffb640 1 -- 172.21.15.70:0/4011653365 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568005310 con 0x7f157015af10 2023-10-08T07:13:09.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.581+0000 7f1576311640 1 -- 172.21.15.70:0/4011653365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15700c6e10 con 0x7f157015af10 2023-10-08T07:13:09.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.583+0000 7f1576311640 1 -- 172.21.15.70:0/4011653365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157004e160 con 0x7f157015af10 2023-10-08T07:13:09.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.583+0000 7f1565ffb640 1 -- 172.21.15.70:0/4011653365 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 126) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f1568005680 con 0x7f157015af10 2023-10-08T07:13:09.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.584+0000 7f1565ffb640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f155406acc0 0x7f155406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:09.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.584+0000 7f1565ffb640 1 -- 172.21.15.70:0/4011653365 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6948+0+0 (secure 0 0 0) 0x7f1568089640 con 0x7f157015af10 2023-10-08T07:13:09.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.585+0000 7f157530f640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f155406acc0 0x7f155406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:09.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.586+0000 7f157530f640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f155406acc0 0x7f155406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f156c036000 tx=0x7f156c03b040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:09.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.589+0000 7f1565ffb640 1 -- 172.21.15.70:0/4011653365 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1568054550 con 0x7f157015af10 2023-10-08T07:13:09.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.817+0000 7f1576311640 1 -- 172.21.15.70:0/4011653365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15700ada50 con 0x7f157015af10 2023-10-08T07:13:09.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.818+0000 7f1565ffb640 1 -- 172.21.15.70:0/4011653365 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f1568003e60 con 0x7f157015af10 2023-10-08T07:13:09.820 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:09.820 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/4185609199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4185609199},{"type":"v1","addr":"172.21.15.70:6833","nonce":4185609199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3102206290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3102206290},{"type":"v1","addr":"172.21.15.181:6837","nonce":3102206290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1824427895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1824427895},{"type":"v1","addr":"172.21.15.181:6835","nonce":1824427895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:10:44.128443+0000","modified":"2023-10-08T07:10:47.166174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/2355440757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2355440757},{"type":"v1","addr":"172.21.15.70:6835","nonce":2355440757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-08T07:13:09.821 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 99 2023-10-08T07:13:09.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 -- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f155406acc0 msgr2=0x7f155406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f155406acc0 0x7f155406d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f156c036000 tx=0x7f156c03b040 comp rx=0 tx=0).stop 2023-10-08T07:13:09.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 -- 172.21.15.70:0/4011653365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015af10 msgr2=0x7f1570156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:09.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015af10 0x7f1570156150 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f156800c730 tx=0x7f156800cbf0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 -- 172.21.15.70:0/4011653365 shutdown_connections 2023-10-08T07:13:09.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f155406acc0 0x7f155406d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f157015af10 0x7f1570156150 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f157015ab30 0x7f157015ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 --2- 172.21.15.70:0/4011653365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f15700adc70 0x7f15700b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:09.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 -- 172.21.15.70:0/4011653365 >> 172.21.15.70:0/4011653365 conn(0x7f15700cb0c0 msgr2=0x7f15700c9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:09.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 -- 172.21.15.70:0/4011653365 shutdown_connections 2023-10-08T07:13:09.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:09.821+0000 7f1576311640 1 -- 172.21.15.70:0/4011653365 wait complete. 2023-10-08T07:13:09.843 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:13:10.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.055+0000 7efe15e10640 1 Processor -- start 2023-10-08T07:13:10.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.056+0000 7efe15e10640 1 -- start start 2023-10-08T07:13:10.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.057+0000 7efe15e10640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe10165cc0 0x7efe101680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.057+0000 7efe15e10640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe100c48f0 con 0x7efe1015cd90 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.057+0000 7efe15e10640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe100710f0 con 0x7efe10160940 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.057+0000 7efe15e10640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe100c4d80 con 0x7efe10165cc0 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.057+0000 7efe0f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efe1015cd90 0x7efe100702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51584/0 (socket says 172.21.15.70:51584) 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.057+0000 7efe0f7fe640 1 -- 172.21.15.70:0/4156922178 learned_addr learned my addr 172.21.15.70:0/4156922178 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119420921 0 0) 0x7efe100c48f0 con 0x7efe1015cd90 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc003520 con 0x7efe1015cd90 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0ffff640 1 --2- 172.21.15.70:0/4156922178 >> v2:172.21.15.181:3300/0 conn(0x7efe10165cc0 0x7efe101680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3156981949 0 0) 0x7efdfc003520 con 0x7efe1015cd90 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 >> v2:172.21.15.181:3300/0 conn(0x7efe10165cc0 msgr2=0x7efe101680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 --2- 172.21.15.70:0/4156922178 >> v2:172.21.15.181:3300/0 conn(0x7efe10165cc0 0x7efe101680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 >> v1:172.21.15.181:6789/0 conn(0x7efe10160940 legacy=0x7efe100709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:10.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100c3ce0 con 0x7efe1015cd90 2023-10-08T07:13:10.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7efe00001670 con 0x7efe1015cd90 2023-10-08T07:13:10.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 >> v1:172.21.15.70:6789/0 conn(0x7efe1015cd90 legacy=0x7efe100702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc0039f0 0x7efdfc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0047c0 0x7efdfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0051f0 0x7efdfc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc00c120 con 0x7efdfc0039f0 2023-10-08T07:13:10.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc00c290 con 0x7efdfc0047c0 2023-10-08T07:13:10.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.058+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc00c450 con 0x7efdfc0051f0 2023-10-08T07:13:10.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.059+0000 7efe0f7fe640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc0039f0 0x7efdfc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.059+0000 7efe07fff640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0047c0 0x7efdfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.059+0000 7efe0f7fe640 1 -- 172.21.15.70:0/4156922178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0051f0 msgr2=0x7efdfc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.059+0000 7efe0f7fe640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0051f0 0x7efdfc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.059+0000 7efe0f7fe640 1 -- 172.21.15.70:0/4156922178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0047c0 msgr2=0x7efdfc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.059+0000 7efe0f7fe640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0047c0 0x7efdfc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.059+0000 7efe0ffff640 1 -- 172.21.15.70:0/4156922178 reap_dead start 2023-10-08T07:13:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.059+0000 7efe0f7fe640 1 -- 172.21.15.70:0/4156922178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc003520 con 0x7efdfc0039f0 2023-10-08T07:13:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.059+0000 7efe07fff640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efdfc0047c0 0x7efdfc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.060+0000 7efe0f7fe640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc0039f0 0x7efdfc003df0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7efe0000c190 tx=0x7efdfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e5378bbfaf1afe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.060+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00005ad0 con 0x7efdfc0039f0 2023-10-08T07:13:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.060+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe00005c30 con 0x7efdfc0039f0 2023-10-08T07:13:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.060+0000 7efe0effd640 1 -- 172.21.15.70:0/4156922178 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00007070 con 0x7efdfc0039f0 2023-10-08T07:13:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.061+0000 7efe15e10640 1 -- 172.21.15.70:0/4156922178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc0039f0 msgr2=0x7efdfc003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.061+0000 7efe15e10640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc0039f0 0x7efdfc003df0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7efe0000c190 tx=0x7efdfc0057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.061+0000 7efe15e10640 1 -- 172.21.15.70:0/4156922178 shutdown_connections 2023-10-08T07:13:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.061+0000 7efe15e10640 1 --2- 172.21.15.70:0/4156922178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdfc0039f0 0x7efdfc003df0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.061+0000 7efe15e10640 1 -- 172.21.15.70:0/4156922178 >> 172.21.15.70:0/4156922178 conn(0x7efe1006a380 msgr2=0x7efe100b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.061+0000 7efe15e10640 1 -- 172.21.15.70:0/4156922178 shutdown_connections 2023-10-08T07:13:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.061+0000 7efe15e10640 1 -- 172.21.15.70:0/4156922178 wait complete. 2023-10-08T07:13:10.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.063+0000 7efe15e10640 1 Processor -- start 2023-10-08T07:13:10.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.063+0000 7efe15e10640 1 -- start start 2023-10-08T07:13:10.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.064+0000 7efe15e10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0039f0 0x7efe10149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.064+0000 7efe15e10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe100cb4d0 0x7efe1015e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.064+0000 7efe15e10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe100cb950 0x7efe1015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.064+0000 7efe15e10640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe10149890 con 0x7efe100cb950 2023-10-08T07:13:10.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.064+0000 7efe15e10640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe100b1480 con 0x7efe100cb4d0 2023-10-08T07:13:10.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.064+0000 7efe15e10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1015ebc0 con 0x7efdfc0039f0 2023-10-08T07:13:10.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.064+0000 7efe0ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe100cb950 0x7efe1015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.064+0000 7efe0ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe100cb950 0x7efe1015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42976/0 (socket says 172.21.15.70:42976) 2023-10-08T07:13:10.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.064+0000 7efe0ffff640 1 -- 172.21.15.70:0/3393250511 learned_addr learned my addr 172.21.15.70:0/3393250511 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:10.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.065+0000 7efe0ffff640 1 -- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0039f0 msgr2=0x7efe10149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.065+0000 7efe0ffff640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0039f0 0x7efe10149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.065+0000 7efe0ffff640 1 -- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe100cb4d0 msgr2=0x7efe1015e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.065+0000 7efe0ffff640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe100cb4d0 0x7efe1015e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.065+0000 7efe0ffff640 1 -- 172.21.15.70:0/3393250511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100ca180 con 0x7efe100cb950 2023-10-08T07:13:10.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.065+0000 7efe0ffff640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe100cb950 0x7efe1015e680 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7efe101608a0 tx=0x7efe0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:10.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.065+0000 7efe0cff9640 1 -- 172.21.15.70:0/3393250511 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0800ed90 con 0x7efe100cb950 2023-10-08T07:13:10.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.065+0000 7efe0cff9640 1 -- 172.21.15.70:0/3393250511 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe08014420 con 0x7efe100cb950 2023-10-08T07:13:10.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.065+0000 7efe0cff9640 1 -- 172.21.15.70:0/3393250511 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe080147d0 con 0x7efe100cb950 2023-10-08T07:13:10.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.065+0000 7efe15e10640 1 -- 172.21.15.70:0/3393250511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe100ca460 con 0x7efe100cb950 2023-10-08T07:13:10.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.066+0000 7efe15e10640 1 -- 172.21.15.70:0/3393250511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe100caa20 con 0x7efe100cb950 2023-10-08T07:13:10.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.067+0000 7efe15e10640 1 -- 172.21.15.70:0/3393250511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe10067490 con 0x7efe100cb950 2023-10-08T07:13:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.068+0000 7efe0cff9640 1 -- 172.21.15.70:0/3393250511 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 126) v1 ==== 89067+0+0 (secure 0 0 0) 0x7efe08014930 con 0x7efe100cb950 2023-10-08T07:13:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.068+0000 7efe0cff9640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efdf006acc0 0x7efdf006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.069+0000 7efe0cff9640 1 -- 172.21.15.70:0/3393250511 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6948+0+0 (secure 0 0 0) 0x7efe08089c20 con 0x7efe100cb950 2023-10-08T07:13:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.069+0000 7efe0f7fe640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efdf006acc0 0x7efdf006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.070+0000 7efe0f7fe640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efdf006acc0 0x7efdf006d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efe00034000 tx=0x7efe00039040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.073+0000 7efe0cff9640 1 -- 172.21.15.70:0/3393250511 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe08054b30 con 0x7efe100cb950 2023-10-08T07:13:10.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.299+0000 7efe15e10640 1 -- 172.21.15.70:0/3393250511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe100b15c0 con 0x7efe100cb950 2023-10-08T07:13:10.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.300+0000 7efe0cff9640 1 -- 172.21.15.70:0/3393250511 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7efe08058150 con 0x7efe100cb950 2023-10-08T07:13:10.302 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:10.302 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":99,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/4185609199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4185609199},{"type":"v1","addr":"172.21.15.70:6833","nonce":4185609199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3102206290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3102206290},{"type":"v1","addr":"172.21.15.181:6837","nonce":3102206290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1824427895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1824427895},{"type":"v1","addr":"172.21.15.181:6835","nonce":1824427895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:10:44.128443+0000","modified":"2023-10-08T07:10:47.166174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6835/2355440757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2355440757},{"type":"v1","addr":"172.21.15.70:6835","nonce":2355440757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-10-08T07:13:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 99 2023-10-08T07:13:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.302+0000 7efe15e10640 1 -- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efdf006acc0 msgr2=0x7efdf006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.302+0000 7efe15e10640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efdf006acc0 0x7efdf006d180 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efe00034000 tx=0x7efe00039040 comp rx=0 tx=0).stop 2023-10-08T07:13:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.302+0000 7efe15e10640 1 -- 172.21.15.70:0/3393250511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe100cb950 msgr2=0x7efe1015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.302+0000 7efe15e10640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe100cb950 0x7efe1015e680 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7efe101608a0 tx=0x7efe0800e4b0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.302+0000 7efe15e10640 1 -- 172.21.15.70:0/3393250511 shutdown_connections 2023-10-08T07:13:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.303+0000 7efe15e10640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efdf006acc0 0x7efdf006d180 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.303+0000 7efe15e10640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe100cb950 0x7efe1015e680 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.303+0000 7efe15e10640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe100cb4d0 0x7efe1015e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.303+0000 7efe15e10640 1 --2- 172.21.15.70:0/3393250511 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efdfc0039f0 0x7efe10149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.303+0000 7efe15e10640 1 -- 172.21.15.70:0/3393250511 >> 172.21.15.70:0/3393250511 conn(0x7efe1006a380 msgr2=0x7efe10160e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.303+0000 7efe15e10640 1 -- 172.21.15.70:0/3393250511 shutdown_connections 2023-10-08T07:13:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.303+0000 7efe15e10640 1 -- 172.21.15.70:0/3393250511 wait complete. 2023-10-08T07:13:10.325 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.540+0000 7f9e0890f640 1 Processor -- start 2023-10-08T07:13:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.540+0000 7f9e0890f640 1 -- start start 2023-10-08T07:13:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.542+0000 7f9e0890f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e04160940 0x7f9e040709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.542+0000 7f9e0890f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e040c48f0 con 0x7f9e0406c890 2023-10-08T07:13:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.542+0000 7f9e0890f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e040c3ce0 con 0x7f9e0415cd90 2023-10-08T07:13:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.542+0000 7f9e0890f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e040c4d80 con 0x7f9e04160940 2023-10-08T07:13:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.543+0000 7f9e02d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9e0406c890 0x7f9e0406cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51598/0 (socket says 172.21.15.70:51598) 2023-10-08T07:13:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.543+0000 7f9e02d9d640 1 -- 172.21.15.70:0/1382671069 learned_addr learned my addr 172.21.15.70:0/1382671069 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.543+0000 7f9dfbfff640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e04160940 0x7f9e040709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.543+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43739361 0 0) 0x7f9e040c48f0 con 0x7f9e0406c890 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.543+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df0003520 con 0x7f9e0406c890 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3145145524 0 0) 0x7f9df0003520 con 0x7f9e0406c890 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e04160940 msgr2=0x7f9e040709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e04160940 0x7f9e040709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 >> v1:172.21.15.181:6789/0 conn(0x7f9e0415cd90 legacy=0x7f9e040702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040c3fc0 con 0x7f9e0406c890 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9dfc0019f0 con 0x7f9e0406c890 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 >> v1:172.21.15.70:6789/0 conn(0x7f9e0406c890 legacy=0x7f9e0406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9df00039a0 0x7f9df0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e04160940 0x7f9df00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 0x7f9df0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df000ba30 con 0x7f9df0004d00 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df000bba0 con 0x7f9df00039a0 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df000bd60 con 0x7f9e04160940 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e02d9d640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 0x7f9df0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9dfbfff640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e04160940 0x7f9df00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.544+0000 7f9e0259c640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9df00039a0 0x7f9df0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.545+0000 7f9e02d9d640 1 -- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e04160940 msgr2=0x7f9df00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.545+0000 7f9e02d9d640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e04160940 0x7f9df00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.545+0000 7f9e02d9d640 1 -- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9df00039a0 msgr2=0x7f9df0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.545+0000 7f9e02d9d640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9df00039a0 0x7f9df0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.545+0000 7f9e02d9d640 1 -- 172.21.15.70:0/1382671069 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df0003520 con 0x7f9df0004d00 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.545+0000 7f9dfbfff640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e04160940 0x7f9df00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.545+0000 7f9e0259c640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9df00039a0 0x7f9df0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:10.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.546+0000 7f9e02d9d640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 0x7f9df0005180 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00a720 tx=0x7f9df00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937edc6f0d7ae7ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:10.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.546+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc034630 con 0x7f9df0004d00 2023-10-08T07:13:10.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.546+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc02f070 con 0x7f9df0004d00 2023-10-08T07:13:10.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.546+0000 7f9e01d9b640 1 -- 172.21.15.70:0/1382671069 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0349d0 con 0x7f9df0004d00 2023-10-08T07:13:10.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.546+0000 7f9e0890f640 1 -- 172.21.15.70:0/1382671069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 msgr2=0x7f9df0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.546+0000 7f9e0890f640 1 --2- 172.21.15.70:0/1382671069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 0x7f9df0005180 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00a720 tx=0x7f9df00057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.547+0000 7f9e02d9d640 1 -- 172.21.15.70:0/1382671069 reap_dead start 2023-10-08T07:13:10.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.547+0000 7f9e0890f640 1 -- 172.21.15.70:0/1382671069 shutdown_connections 2023-10-08T07:13:10.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.547+0000 7f9e0890f640 1 -- 172.21.15.70:0/1382671069 >> 172.21.15.70:0/1382671069 conn(0x7f9e0406a380 msgr2=0x7f9e040b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:10.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.547+0000 7f9e0890f640 1 -- 172.21.15.70:0/1382671069 shutdown_connections 2023-10-08T07:13:10.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.547+0000 7f9e0890f640 1 -- 172.21.15.70:0/1382671069 wait complete. 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.548+0000 7f9e0890f640 1 Processor -- start 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.549+0000 7f9e0890f640 1 -- start start 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0890f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 0x7f9e041f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0890f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e040c5fd0 0x7f9e0415f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0890f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e041f6590 0x7f9e041f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0890f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e040c6450 con 0x7f9df0004d00 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0890f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e040c5b30 con 0x7f9e041f6590 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0890f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e041f3010 con 0x7f9e040c5fd0 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 0x7f9e041f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 0x7f9e041f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42988/0 (socket says 172.21.15.70:42988) 2023-10-08T07:13:10.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0259c640 1 -- 172.21.15.70:0/2250271003 learned_addr learned my addr 172.21.15.70:0/2250271003 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e02d9d640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e041f6590 0x7f9e041f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9dfbfff640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e040c5fd0 0x7f9e0415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0259c640 1 -- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e040c5fd0 msgr2=0x7f9e0415f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0259c640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e040c5fd0 0x7f9e0415f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0259c640 1 -- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e041f6590 msgr2=0x7f9e041f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0259c640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e041f6590 0x7f9e041f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e0259c640 1 -- 172.21.15.70:0/2250271003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e041f32f0 con 0x7f9df0004d00 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.550+0000 7f9e02d9d640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e041f6590 0x7f9e041f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.551+0000 7f9e0259c640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 0x7f9e041f2990 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f9df400d770 tx=0x7f9df400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.551+0000 7f9dfaffd640 1 -- 172.21.15.70:0/2250271003 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4005270 con 0x7f9df0004d00 2023-10-08T07:13:10.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.551+0000 7f9dfaffd640 1 -- 172.21.15.70:0/2250271003 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df40053d0 con 0x7f9df0004d00 2023-10-08T07:13:10.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.551+0000 7f9e0890f640 1 -- 172.21.15.70:0/2250271003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e041f35d0 con 0x7f9df0004d00 2023-10-08T07:13:10.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.551+0000 7f9dfaffd640 1 -- 172.21.15.70:0/2250271003 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400b8d0 con 0x7f9df0004d00 2023-10-08T07:13:10.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.551+0000 7f9e0890f640 1 -- 172.21.15.70:0/2250271003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0415fe70 con 0x7f9df0004d00 2023-10-08T07:13:10.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.552+0000 7f9e0890f640 1 -- 172.21.15.70:0/2250271003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e04067490 con 0x7f9df0004d00 2023-10-08T07:13:10.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.553+0000 7f9dfaffd640 1 -- 172.21.15.70:0/2250271003 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 127) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9df400ba30 con 0x7f9df0004d00 2023-10-08T07:13:10.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.554+0000 7f9dfaffd640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9de406af00 0x7f9de406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.554+0000 7f9dfaffd640 1 -- 172.21.15.70:0/2250271003 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6948+0+0 (secure 0 0 0) 0x7f9df4089cb0 con 0x7f9df0004d00 2023-10-08T07:13:10.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.555+0000 7f9dfbfff640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9de406af00 0x7f9de406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.556+0000 7f9dfbfff640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9de406af00 0x7f9de406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9e041f7060 tx=0x7f9dec008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:10.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.559+0000 7f9dfaffd640 1 -- 172.21.15.70:0/2250271003 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9df4054680 con 0x7f9df0004d00 2023-10-08T07:13:10.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.747+0000 7f9e0890f640 1 -- 172.21.15.70:0/2250271003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e040b0ef0 con 0x7f9df0004d00 2023-10-08T07:13:10.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.749+0000 7f9dfaffd640 1 -- 172.21.15.70:0/2250271003 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v222) v1 ==== 74+0+18118 (secure 0 0 0) 0x7f9df4019310 con 0x7f9df0004d00 2023-10-08T07:13:10.751 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:10.751 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":222,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:07.464415+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:10:41.917258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-08T07:10:43.125413+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"221","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":221,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.28","mappings":[{"from":1,"to":0}]},{"pgid":"26.36","mappings":[{"from":3,"to":0}]},{"pgid":"26.37","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1545365181":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/4119540053":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/2433436977":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/2511790348":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/294214378":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/3667065295":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/1205099199":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/3162325425":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6833/2638163552":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6832/2638163552":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/785931869":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6833/2613103103":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/265797938":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6832/2613103103":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/2110654177":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/39996703":"2023-10-09T07:10:58.332667+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]},{"pool":26,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:10.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.751+0000 7f9e0890f640 1 -- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9de406af00 msgr2=0x7f9de406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.751+0000 7f9e0890f640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9de406af00 0x7f9de406d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9e041f7060 tx=0x7f9dec008040 comp rx=0 tx=0).stop 2023-10-08T07:13:10.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.751+0000 7f9e0890f640 1 -- 172.21.15.70:0/2250271003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 msgr2=0x7f9e041f2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.751+0000 7f9e0890f640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 0x7f9e041f2990 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f9df400d770 tx=0x7f9df400dc30 comp rx=0 tx=0).stop 2023-10-08T07:13:10.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.752+0000 7f9e0890f640 1 -- 172.21.15.70:0/2250271003 shutdown_connections 2023-10-08T07:13:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.752+0000 7f9e0890f640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e041f6590 0x7f9e041f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.752+0000 7f9e0890f640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e040c5fd0 0x7f9e0415f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.752+0000 7f9e0890f640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9de406af00 0x7f9de406d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.752+0000 7f9e0890f640 1 --2- 172.21.15.70:0/2250271003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9df0004d00 0x7f9e041f2990 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.752+0000 7f9e0890f640 1 -- 172.21.15.70:0/2250271003 >> 172.21.15.70:0/2250271003 conn(0x7f9e0406a380 msgr2=0x7f9e04157670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.752+0000 7f9e0890f640 1 -- 172.21.15.70:0/2250271003 shutdown_connections 2023-10-08T07:13:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.752+0000 7f9e0890f640 1 -- 172.21.15.70:0/2250271003 wait complete. 2023-10-08T07:13:10.772 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T07:13:10.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.978+0000 7f3ed0935640 1 Processor -- start 2023-10-08T07:13:10.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.978+0000 7f3ed0935640 1 -- start start 2023-10-08T07:13:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.980+0000 7f3ed0935640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0cf690 0x7f3ecc0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.980+0000 7f3ed0935640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ecc0ce510 0x7f3ecc15c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.980+0000 7f3ed0935640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ecc0cd520 con 0x7f3ecc0cf2b0 2023-10-08T07:13:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.980+0000 7f3ed0935640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ecc0d1910 con 0x7f3ecc0ce510 2023-10-08T07:13:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.980+0000 7f3ed0935640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0ce080 con 0x7f3ecc0cf690 2023-10-08T07:13:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.980+0000 7f3ecb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3ecc0cf2b0 0x7f3ecc0d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51610/0 (socket says 172.21.15.70:51610) 2023-10-08T07:13:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.980+0000 7f3ecb7fe640 1 -- 172.21.15.70:0/2436388825 learned_addr learned my addr 172.21.15.70:0/2436388825 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.980+0000 7f3ecbfff640 1 --2- 172.21.15.70:0/2436388825 >> v2:172.21.15.70:3300/0 conn(0x7f3ecc0ce510 0x7f3ecc15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.981+0000 7f3eca7fc640 1 -- 172.21.15.70:0/2436388825 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115020924 0 0) 0x7f3ecc0cd520 con 0x7f3ecc0cf2b0 2023-10-08T07:13:10.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.981+0000 7f3eca7fc640 1 -- 172.21.15.70:0/2436388825 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3eb8003520 con 0x7f3ecc0cf2b0 2023-10-08T07:13:10.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.981+0000 7f3ecbfff640 1 -- 172.21.15.70:0/2436388825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0cf690 msgr2=0x7f3ecc0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.981+0000 7f3ecaffd640 1 --2- 172.21.15.70:0/2436388825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0cf690 0x7f3ecc0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.981+0000 7f3ecbfff640 1 --2- 172.21.15.70:0/2436388825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0cf690 0x7f3ecc0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.981+0000 7f3ecbfff640 1 -- 172.21.15.70:0/2436388825 >> v1:172.21.15.70:6789/0 conn(0x7f3ecc0cf2b0 legacy=0x7f3ecc0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.981+0000 7f3ecbfff640 1 -- 172.21.15.70:0/2436388825 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc0cd750 con 0x7f3ecc0ce510 2023-10-08T07:13:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.981+0000 7f3ecaffd640 1 --2- 172.21.15.70:0/2436388825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0cf690 0x7f3ecc0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.982+0000 7f3ecbfff640 1 --2- 172.21.15.70:0/2436388825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0ce510 0x7f3ecc15c820 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40097a0 tx=0x7f3ec402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9012e54f50d056cf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:10.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.982+0000 7f3eca7fc640 1 -- 172.21.15.70:0/2436388825 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4031780 con 0x7f3ecc0ce510 2023-10-08T07:13:10.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.982+0000 7f3eca7fc640 1 -- 172.21.15.70:0/2436388825 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec40318e0 con 0x7f3ecc0ce510 2023-10-08T07:13:10.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.982+0000 7f3eca7fc640 1 -- 172.21.15.70:0/2436388825 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4031c70 con 0x7f3ecc0ce510 2023-10-08T07:13:10.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.983+0000 7f3ed0935640 1 -- 172.21.15.70:0/2436388825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0ce510 msgr2=0x7f3ecc15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.983+0000 7f3ed0935640 1 --2- 172.21.15.70:0/2436388825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0ce510 0x7f3ecc15c820 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40097a0 tx=0x7f3ec402b820 comp rx=0 tx=0).stop 2023-10-08T07:13:10.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.983+0000 7f3ed0935640 1 -- 172.21.15.70:0/2436388825 shutdown_connections 2023-10-08T07:13:10.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.983+0000 7f3ed0935640 1 --2- 172.21.15.70:0/2436388825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0ce510 0x7f3ecc15c820 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.983+0000 7f3ed0935640 1 --2- 172.21.15.70:0/2436388825 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0cf690 0x7f3ecc0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.983+0000 7f3ed0935640 1 -- 172.21.15.70:0/2436388825 >> 172.21.15.70:0/2436388825 conn(0x7f3ecc09bc40 msgr2=0x7f3ecc0d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:10.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.983+0000 7f3ed0935640 1 -- 172.21.15.70:0/2436388825 shutdown_connections 2023-10-08T07:13:10.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.983+0000 7f3ed0935640 1 -- 172.21.15.70:0/2436388825 wait complete. 2023-10-08T07:13:10.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.985+0000 7f3ed0935640 1 Processor -- start 2023-10-08T07:13:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.985+0000 7f3ed0935640 1 -- start start 2023-10-08T07:13:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.986+0000 7f3ed0935640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0ce510 0x7f3ecc0b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.986+0000 7f3ed0935640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ecc0cf2b0 0x7f3ecc06c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.986+0000 7f3ed0935640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cf690 0x7f3ecc0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.986+0000 7f3ed0935640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0b49b0 con 0x7f3ecc0cf690 2023-10-08T07:13:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.986+0000 7f3ed0935640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0b3da0 con 0x7f3ecc0cf2b0 2023-10-08T07:13:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.986+0000 7f3ed0935640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc0b4080 con 0x7f3ecc0ce510 2023-10-08T07:13:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cf690 0x7f3ecc0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cf690 0x7f3ecc0b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43010/0 (socket says 172.21.15.70:43010) 2023-10-08T07:13:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecbfff640 1 -- 172.21.15.70:0/2183600609 learned_addr learned my addr 172.21.15.70:0/2183600609 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecb7fe640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0ce510 0x7f3ecc0b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecaffd640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ecc0cf2b0 0x7f3ecc06c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecbfff640 1 -- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0ce510 msgr2=0x7f3ecc0b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecbfff640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0ce510 0x7f3ecc0b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecbfff640 1 -- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ecc0cf2b0 msgr2=0x7f3ecc06c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecbfff640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ecc0cf2b0 0x7f3ecc06c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecbfff640 1 -- 172.21.15.70:0/2183600609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec4009400 con 0x7f3ecc0cf690 2023-10-08T07:13:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.987+0000 7f3ecbfff640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cf690 0x7f3ecc0b33c0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4034040 tx=0x7f3ec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.988+0000 7f3ec3fff640 1 -- 172.21.15.70:0/2183600609 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4032820 con 0x7f3ecc0cf690 2023-10-08T07:13:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.988+0000 7f3ec3fff640 1 -- 172.21.15.70:0/2183600609 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4032980 con 0x7f3ecc0cf690 2023-10-08T07:13:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.988+0000 7f3ec3fff640 1 -- 172.21.15.70:0/2183600609 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4032d30 con 0x7f3ecc0cf690 2023-10-08T07:13:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.988+0000 7f3ed0935640 1 -- 172.21.15.70:0/2183600609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc0b4360 con 0x7f3ecc0cf690 2023-10-08T07:13:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.988+0000 7f3ed0935640 1 -- 172.21.15.70:0/2183600609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc0b0ee0 con 0x7f3ecc0cf690 2023-10-08T07:13:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.989+0000 7f3ed0935640 1 -- 172.21.15.70:0/2183600609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ecc0b4b20 con 0x7f3ecc0cf690 2023-10-08T07:13:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.990+0000 7f3ec3fff640 1 -- 172.21.15.70:0/2183600609 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 127) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f3ec404e450 con 0x7f3ecc0cf690 2023-10-08T07:13:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.991+0000 7f3ec3fff640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3eac06af30 0x7f3eac06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.991+0000 7f3ec3fff640 1 -- 172.21.15.70:0/2183600609 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 6948+0+0 (secure 0 0 0) 0x7f3ec40abce0 con 0x7f3ecc0cf690 2023-10-08T07:13:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.991+0000 7f3ecb7fe640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3eac06af30 0x7f3eac06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.992+0000 7f3ecb7fe640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3eac06af30 0x7f3eac06d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc001560 tx=0x7f3ebc009040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:10.995+0000 7f3ec3fff640 1 -- 172.21.15.70:0/2183600609 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ec40766b0 con 0x7f3ecc0cf690 2023-10-08T07:13:11.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.220+0000 7f3ed0935640 1 -- 172.21.15.70:0/2183600609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3ecc074150 con 0x7f3ecc0cf690 2023-10-08T07:13:11.223 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T07:13:11.221+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T07:13:11.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.225+0000 7f3ec3fff640 1 -- 172.21.15.70:0/2183600609 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3ec4079cd0 con 0x7f3ecc0cf690 2023-10-08T07:13:11.227 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T07:13:11.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.229+0000 7f3ed0935640 1 -- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3eac06af30 msgr2=0x7f3eac06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:11.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.229+0000 7f3ed0935640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3eac06af30 0x7f3eac06d3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc001560 tx=0x7f3ebc009040 comp rx=0 tx=0).stop 2023-10-08T07:13:11.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.229+0000 7f3ed0935640 1 -- 172.21.15.70:0/2183600609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cf690 msgr2=0x7f3ecc0b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:11.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.229+0000 7f3ed0935640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cf690 0x7f3ecc0b33c0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4034040 tx=0x7f3ec40046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:11.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.230+0000 7f3ed0935640 1 -- 172.21.15.70:0/2183600609 shutdown_connections 2023-10-08T07:13:11.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.230+0000 7f3ed0935640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3eac06af30 0x7f3eac06d3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:11.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.230+0000 7f3ed0935640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ecc0cf690 0x7f3ecc0b33c0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:11.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.230+0000 7f3ed0935640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3ecc0cf2b0 0x7f3ecc06c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:11.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.230+0000 7f3ed0935640 1 --2- 172.21.15.70:0/2183600609 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3ecc0ce510 0x7f3ecc0b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:11.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.230+0000 7f3ed0935640 1 -- 172.21.15.70:0/2183600609 >> 172.21.15.70:0/2183600609 conn(0x7f3ecc09bc40 msgr2=0x7f3ecc15aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:11.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.230+0000 7f3ed0935640 1 -- 172.21.15.70:0/2183600609 shutdown_connections 2023-10-08T07:13:11.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.230+0000 7f3ed0935640 1 -- 172.21.15.70:0/2183600609 wait complete. 2023-10-08T07:13:11.251 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T07:13:11.301 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T07:13:11.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.440+0000 7f46bcb73640 1 Processor -- start 2023-10-08T07:13:11.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.440+0000 7f46bcb73640 1 -- start start 2023-10-08T07:13:11.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.441+0000 7f46bcb73640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f46b8070cc0 0x7f46b80caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.441+0000 7f46bcb73640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b80b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.441+0000 7f46bcb73640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b8071170 con 0x7f46b8160660 2023-10-08T07:13:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.441+0000 7f46bcb73640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b806e150 con 0x7f46b8070cc0 2023-10-08T07:13:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.441+0000 7f46bcb73640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b80cb8c0 con 0x7f46b80cb450 2023-10-08T07:13:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.442+0000 7f46b5d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f46b8070cc0 0x7f46b80caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.442+0000 7f46b6d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b80b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.442+0000 7f46b5d9b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f46b8070cc0 0x7f46b80caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:37630/0 (socket says 172.21.15.70:37630) 2023-10-08T07:13:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.442+0000 7f46b659c640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f46b8160660 0x7f46b80c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:47420/0 (socket says 172.21.15.70:47420) 2023-10-08T07:13:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.442+0000 7f46b6d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b80b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:56704/0 (socket says 172.21.15.70:56704) 2023-10-08T07:13:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.442+0000 7f46b5d9b640 1 -- 172.21.15.70:0/1774521766 learned_addr learned my addr 172.21.15.70:0/1774521766 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.443+0000 7f46b5d9b640 1 -- 172.21.15.70:0/1774521766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 msgr2=0x7f46b80b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.443+0000 7f46b5d9b640 1 --2- 172.21.15.70:0/1774521766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b80b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:11.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.443+0000 7f46b5d9b640 1 -- 172.21.15.70:0/1774521766 >> v1:172.21.15.181:6789/0 conn(0x7f46b8160660 legacy=0x7f46b80c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:11.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.443+0000 7f46b5d9b640 1 -- 172.21.15.70:0/1774521766 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b806de70 con 0x7f46b8070cc0 2023-10-08T07:13:11.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.443+0000 7f46b6d9d640 1 --2- 172.21.15.70:0/1774521766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b80b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:13:11.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.445+0000 7f46b5d9b640 1 --2- 172.21.15.70:0/1774521766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b8070cc0 0x7f46b80caf10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009870 tx=0x7f46ac02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b198b5527b0cdbb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:11.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.445+0000 7f46b559a640 1 -- 172.21.15.70:0/1774521766 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac030840 con 0x7f46b8070cc0 2023-10-08T07:13:11.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.445+0000 7f46b559a640 1 -- 172.21.15.70:0/1774521766 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac0309a0 con 0x7f46b8070cc0 2023-10-08T07:13:11.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.445+0000 7f46b559a640 1 -- 172.21.15.70:0/1774521766 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac030d30 con 0x7f46b8070cc0 2023-10-08T07:13:11.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.446+0000 7f46bcb73640 1 -- 172.21.15.70:0/1774521766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b8070cc0 msgr2=0x7f46b80caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:11.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.446+0000 7f46bcb73640 1 --2- 172.21.15.70:0/1774521766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b8070cc0 0x7f46b80caf10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009870 tx=0x7f46ac02b980 comp rx=0 tx=0).stop 2023-10-08T07:13:11.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.446+0000 7f46bcb73640 1 -- 172.21.15.70:0/1774521766 shutdown_connections 2023-10-08T07:13:11.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.446+0000 7f46bcb73640 1 --2- 172.21.15.70:0/1774521766 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b80b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:11.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.446+0000 7f46bcb73640 1 --2- 172.21.15.70:0/1774521766 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b8070cc0 0x7f46b80caf10 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:11.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.446+0000 7f46bcb73640 1 -- 172.21.15.70:0/1774521766 >> 172.21.15.70:0/1774521766 conn(0x7f46b806a5d0 msgr2=0x7f46b80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:11.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.446+0000 7f46bcb73640 1 -- 172.21.15.70:0/1774521766 shutdown_connections 2023-10-08T07:13:11.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.446+0000 7f46bcb73640 1 -- 172.21.15.70:0/1774521766 wait complete. 2023-10-08T07:13:11.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.448+0000 7f46bcb73640 1 Processor -- start 2023-10-08T07:13:11.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.448+0000 7f46bcb73640 1 -- start start 2023-10-08T07:13:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.449+0000 7f46bcb73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b8070cc0 0x7f46b8150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.449+0000 7f46bcb73640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b8153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.449+0000 7f46bcb73640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b8160660 0x7f46b8153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.449+0000 7f46bcb73640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b81f2dd0 con 0x7f46b8070cc0 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.449+0000 7f46bcb73640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b80c5240 con 0x7f46b8160660 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.449+0000 7f46bcb73640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b8153e70 con 0x7f46b80cb450 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.449+0000 7f46b659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b8070cc0 0x7f46b8150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.449+0000 7f46b659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b8070cc0 0x7f46b8150d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43054/0 (socket says 172.21.15.70:43054) 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.449+0000 7f46b659c640 1 -- 172.21.15.70:0/3116626067 learned_addr learned my addr 172.21.15.70:0/3116626067 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46b5d9b640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b8153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46b6d9d640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b8160660 0x7f46b8153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46b659c640 1 -- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 msgr2=0x7f46b8153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:11.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46b659c640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b8153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46b659c640 1 -- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b8160660 msgr2=0x7f46b8153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46b659c640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b8160660 0x7f46b8153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46b659c640 1 -- 172.21.15.70:0/3116626067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b0009a70 con 0x7f46b8070cc0 2023-10-08T07:13:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46b5d9b640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b8153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46b659c640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b8070cc0 0x7f46b8150d20 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c4d0 tx=0x7f46a800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3116626067 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8007480 con 0x7f46b8070cc0 2023-10-08T07:13:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3116626067 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a80075e0 con 0x7f46b8070cc0 2023-10-08T07:13:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3116626067 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80049e0 con 0x7f46b8070cc0 2023-10-08T07:13:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.450+0000 7f46bcb73640 1 -- 172.21.15.70:0/3116626067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ac0094d0 con 0x7f46b8070cc0 2023-10-08T07:13:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.451+0000 7f46bcb73640 1 -- 172.21.15.70:0/3116626067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b81f2a60 con 0x7f46b8070cc0 2023-10-08T07:13:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.452+0000 7f46bcb73640 1 -- 172.21.15.70:0/3116626067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b804e160 con 0x7f46b8070cc0 2023-10-08T07:13:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.453+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3116626067 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 127) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f46a8004b40 con 0x7f46b8070cc0 2023-10-08T07:13:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.454+0000 7f46a6ffd640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f469806af00 0x7f469806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.454+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3116626067 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7034+0+0 (secure 0 0 0) 0x7f46a8088a90 con 0x7f46b8070cc0 2023-10-08T07:13:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.455+0000 7f46b5d9b640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f469806af00 0x7f469806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:11.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.456+0000 7f46b5d9b640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f469806af00 0x7f469806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46ac036000 tx=0x7f46ac03b040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:11.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.459+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3116626067 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f46a80536d0 con 0x7f46b8070cc0 2023-10-08T07:13:11.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:11.686+0000 7f46bcb73640 1 -- 172.21.15.70:0/3116626067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f46b8071170 con 0x7f46b8070cc0 2023-10-08T07:13:12.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.229+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3116626067 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v101) v1 ==== 104+0+0 (secure 0 0 0) 0x7f46b8071170 con 0x7f46b8070cc0 2023-10-08T07:13:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 -- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f469806af00 msgr2=0x7f469806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f469806af00 0x7f469806d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46ac036000 tx=0x7f46ac03b040 comp rx=0 tx=0).stop 2023-10-08T07:13:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 -- 172.21.15.70:0/3116626067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b8070cc0 msgr2=0x7f46b8150d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b8070cc0 0x7f46b8150d20 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c4d0 tx=0x7f46a800c990 comp rx=0 tx=0).stop 2023-10-08T07:13:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 -- 172.21.15.70:0/3116626067 shutdown_connections 2023-10-08T07:13:12.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46b8160660 0x7f46b8153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f46b80cb450 0x7f46b8153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f469806af00 0x7f469806d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 --2- 172.21.15.70:0/3116626067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b8070cc0 0x7f46b8150d20 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 -- 172.21.15.70:0/3116626067 >> 172.21.15.70:0/3116626067 conn(0x7f46b806a5d0 msgr2=0x7f46b815f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 -- 172.21.15.70:0/3116626067 shutdown_connections 2023-10-08T07:13:12.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.232+0000 7f46bcb73640 1 -- 172.21.15.70:0/3116626067 wait complete. 2023-10-08T07:13:12.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T07:13:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.464+0000 7f5c09a87640 1 Processor -- start 2023-10-08T07:13:12.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.464+0000 7f5c09a87640 1 -- start start 2023-10-08T07:13:12.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.465+0000 7f5c09a87640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c041518d0 0x7f5c040aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:12.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.465+0000 7f5c09a87640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c040af0a0 con 0x7f5c04151cb0 2023-10-08T07:13:12.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.465+0000 7f5c09a87640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0415ebb0 con 0x7f5c0415e5f0 2023-10-08T07:13:12.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.465+0000 7f5c09a87640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0415ee90 con 0x7f5c041518d0 2023-10-08T07:13:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.466+0000 7f5c02ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5c04151cb0 0x7f5c0406ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51636/0 (socket says 172.21.15.70:51636) 2023-10-08T07:13:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.466+0000 7f5c02ffd640 1 -- 172.21.15.70:0/1831424523 learned_addr learned my addr 172.21.15.70:0/1831424523 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.466+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857930289 0 0) 0x7f5bf8001110 con 0x7f5c04151cb0 2023-10-08T07:13:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.466+0000 7f5c037fe640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c041518d0 0x7f5c040aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.466+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bec003520 con 0x7f5c04151cb0 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 908121261 0 0) 0x7f5bec003520 con 0x7f5c04151cb0 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c041518d0 msgr2=0x7f5c040aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c041518d0 0x7f5c040aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 >> v1:172.21.15.181:6789/0 conn(0x7f5c0415e5f0 legacy=0x7f5c0415b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0415f170 con 0x7f5c04151cb0 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5bf8001800 con 0x7f5c04151cb0 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 >> v1:172.21.15.70:6789/0 conn(0x7f5c04151cb0 legacy=0x7f5c0406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c041518d0 0x7f5bec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 0x7f5bec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bec004d00 0x7f5bec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0415ee90 con 0x7f5bec003f70 2023-10-08T07:13:12.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec00bb50 con 0x7f5bec004d00 2023-10-08T07:13:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec00bd10 con 0x7f5c041518d0 2023-10-08T07:13:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c02ffd640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 0x7f5bec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c037fe640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c041518d0 0x7f5bec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c02ffd640 1 -- 172.21.15.70:0/1831424523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c041518d0 msgr2=0x7f5bec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c02ffd640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c041518d0 0x7f5bec0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c02ffd640 1 -- 172.21.15.70:0/1831424523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bec004d00 msgr2=0x7f5bec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.467+0000 7f5c02ffd640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5bec004d00 0x7f5bec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.468+0000 7f5c02ffd640 1 -- 172.21.15.70:0/1831424523 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec003520 con 0x7f5bec003f70 2023-10-08T07:13:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.469+0000 7f5c02ffd640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 0x7f5bec0047c0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8007fc0 tx=0x7f5bec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed652b01e4443a39 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.469+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8005ad0 con 0x7f5bec003f70 2023-10-08T07:13:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.469+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8005c50 con 0x7f5bec003f70 2023-10-08T07:13:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.469+0000 7f5c027fc640 1 -- 172.21.15.70:0/1831424523 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf802c720 con 0x7f5bec003f70 2023-10-08T07:13:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.469+0000 7f5c09a87640 1 -- 172.21.15.70:0/1831424523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 msgr2=0x7f5bec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.469+0000 7f5c09a87640 1 --2- 172.21.15.70:0/1831424523 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 0x7f5bec0047c0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8007fc0 tx=0x7f5bec0057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.469+0000 7f5c03fff640 1 -- 172.21.15.70:0/1831424523 reap_dead start 2023-10-08T07:13:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.469+0000 7f5c09a87640 1 -- 172.21.15.70:0/1831424523 shutdown_connections 2023-10-08T07:13:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.469+0000 7f5c09a87640 1 -- 172.21.15.70:0/1831424523 >> 172.21.15.70:0/1831424523 conn(0x7f5c040cc2e0 msgr2=0x7f5c040cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.470+0000 7f5c09a87640 1 -- 172.21.15.70:0/1831424523 shutdown_connections 2023-10-08T07:13:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.470+0000 7f5c09a87640 1 -- 172.21.15.70:0/1831424523 wait complete. 2023-10-08T07:13:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.471+0000 7f5c09a87640 1 Processor -- start 2023-10-08T07:13:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.471+0000 7f5c09a87640 1 -- start start 2023-10-08T07:13:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.472+0000 7f5c09a87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 0x7f5c040b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.472+0000 7f5c09a87640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c04154540 0x7f5c041549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.472+0000 7f5c09a87640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c040b0820 0x7f5c040b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.472+0000 7f5c09a87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c04154f00 con 0x7f5bec003f70 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.472+0000 7f5c09a87640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c040b21d0 con 0x7f5c040b0820 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.472+0000 7f5c09a87640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c040c0080 con 0x7f5c04154540 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 0x7f5c040b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 0x7f5c040b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43084/0 (socket says 172.21.15.70:43084) 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c037fe640 1 -- 172.21.15.70:0/2574458234 learned_addr learned my addr 172.21.15.70:0/2574458234 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c02ffd640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c04154540 0x7f5c041549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c03fff640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c040b0820 0x7f5c040b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c037fe640 1 -- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c04154540 msgr2=0x7f5c041549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c037fe640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c04154540 0x7f5c041549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c037fe640 1 -- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c040b0820 msgr2=0x7f5c040b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c037fe640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c040b0820 0x7f5c040b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c037fe640 1 -- 172.21.15.70:0/2574458234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c040b11e0 con 0x7f5bec003f70 2023-10-08T07:13:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c03fff640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c040b0820 0x7f5c040b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.473+0000 7f5c037fe640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 0x7f5c040b1c90 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4007f30 tx=0x7f5bf4011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.474+0000 7f5bebfff640 1 -- 172.21.15.70:0/2574458234 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf400e030 con 0x7f5bec003f70 2023-10-08T07:13:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.474+0000 7f5bebfff640 1 -- 172.21.15.70:0/2574458234 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf4016470 con 0x7f5bec003f70 2023-10-08T07:13:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.474+0000 7f5bebfff640 1 -- 172.21.15.70:0/2574458234 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4016820 con 0x7f5bec003f70 2023-10-08T07:13:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.474+0000 7f5c09a87640 1 -- 172.21.15.70:0/2574458234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c040b1450 con 0x7f5bec003f70 2023-10-08T07:13:12.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.474+0000 7f5c09a87640 1 -- 172.21.15.70:0/2574458234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c040c02d0 con 0x7f5bec003f70 2023-10-08T07:13:12.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.475+0000 7f5c09a87640 1 -- 172.21.15.70:0/2574458234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c040cb0f0 con 0x7f5bec003f70 2023-10-08T07:13:12.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.476+0000 7f5bebfff640 1 -- 172.21.15.70:0/2574458234 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5bf4016980 con 0x7f5bec003f70 2023-10-08T07:13:12.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.477+0000 7f5bebfff640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5bf006afc0 0x7f5bf006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:12.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.477+0000 7f5bebfff640 1 -- 172.21.15.70:0/2574458234 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 7034+0+0 (secure 0 0 0) 0x7f5bf400f070 con 0x7f5bec003f70 2023-10-08T07:13:12.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.478+0000 7f5c02ffd640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5bf006afc0 0x7f5bf006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:12.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.479+0000 7f5c02ffd640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5bf006afc0 0x7f5bf006d480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8034000 tx=0x7f5bf8009030 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:12.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.481+0000 7f5bebfff640 1 -- 172.21.15.70:0/2574458234 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5bf4055690 con 0x7f5bec003f70 2023-10-08T07:13:12.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:12.669+0000 7f5c09a87640 1 -- 172.21.15.70:0/2574458234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5c0415b0e0 con 0x7f5bec003f70 2023-10-08T07:13:13.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.356+0000 7f5bebfff640 1 -- 172.21.15.70:0/2574458234 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v224) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5bf4058cb0 con 0x7f5bec003f70 2023-10-08T07:13:13.359 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T07:13:13.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.361+0000 7f5c09a87640 1 -- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5bf006afc0 msgr2=0x7f5bf006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:13.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.361+0000 7f5c09a87640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5bf006afc0 0x7f5bf006d480 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8034000 tx=0x7f5bf8009030 comp rx=0 tx=0).stop 2023-10-08T07:13:13.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.361+0000 7f5c09a87640 1 -- 172.21.15.70:0/2574458234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 msgr2=0x7f5c040b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:13.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.361+0000 7f5c09a87640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 0x7f5c040b1c90 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4007f30 tx=0x7f5bf4011660 comp rx=0 tx=0).stop 2023-10-08T07:13:13.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.362+0000 7f5c09a87640 1 -- 172.21.15.70:0/2574458234 shutdown_connections 2023-10-08T07:13:13.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.362+0000 7f5c09a87640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5c040b0820 0x7f5c040b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:13.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.362+0000 7f5c09a87640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5c04154540 0x7f5c041549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:13.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.362+0000 7f5c09a87640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5bf006afc0 0x7f5bf006d480 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:13.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.362+0000 7f5c09a87640 1 --2- 172.21.15.70:0/2574458234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bec003f70 0x7f5c040b1c90 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:13.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.362+0000 7f5c09a87640 1 -- 172.21.15.70:0/2574458234 >> 172.21.15.70:0/2574458234 conn(0x7f5c040cc2e0 msgr2=0x7f5c0415b630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:13.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.363+0000 7f5c09a87640 1 -- 172.21.15.70:0/2574458234 shutdown_connections 2023-10-08T07:13:13.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.363+0000 7f5c09a87640 1 -- 172.21.15.70:0/2574458234 wait complete. 2023-10-08T07:13:13.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T07:13:13.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.617+0000 7fbc93125640 1 Processor -- start 2023-10-08T07:13:13.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.617+0000 7fbc93125640 1 -- start start 2023-10-08T07:13:13.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.618+0000 7fbc93125640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc8c15de50 0x7fbc8c0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:13.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.618+0000 7fbc93125640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8c0b11a0 con 0x7fbc8c0cb850 2023-10-08T07:13:13.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.618+0000 7fbc93125640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8c0b17b0 con 0x7fbc8c15e230 2023-10-08T07:13:13.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.618+0000 7fbc93125640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8c15f440 con 0x7fbc8c15de50 2023-10-08T07:13:13.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.619+0000 7fbc90ec1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc8c15de50 0x7fbc8c0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:13.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.619+0000 7fbc90ec1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc8c15de50 0x7fbc8c0b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43074/0 (socket says 172.21.15.70:43074) 2023-10-08T07:13:13.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.619+0000 7fbc916c2640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbc8c0cb850 0x7fbc8c0701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41776/0 (socket says 172.21.15.70:41776) 2023-10-08T07:13:13.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.619+0000 7fbc90ec1640 1 -- 172.21.15.70:0/1501344260 learned_addr learned my addr 172.21.15.70:0/1501344260 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:13.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.619+0000 7fbc90ec1640 1 -- 172.21.15.70:0/1501344260 >> v1:172.21.15.181:6789/0 conn(0x7fbc8c15e230 legacy=0x7fbc8c0cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:13.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.619+0000 7fbc90ec1640 1 -- 172.21.15.70:0/1501344260 >> v1:172.21.15.70:6789/0 conn(0x7fbc8c0cb850 legacy=0x7fbc8c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:13.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.619+0000 7fbc90ec1640 1 -- 172.21.15.70:0/1501344260 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c15e7c0 con 0x7fbc8c15de50 2023-10-08T07:13:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.620+0000 7fbc90ec1640 1 --2- 172.21.15.70:0/1501344260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c15de50 0x7fbc8c0b3ef0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009870 tx=0x7fbc8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e32cc07f91c732 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.620+0000 7fbc8b7fe640 1 -- 172.21.15.70:0/1501344260 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80031870 con 0x7fbc8c15de50 2023-10-08T07:13:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.621+0000 7fbc8b7fe640 1 -- 172.21.15.70:0/1501344260 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc800319d0 con 0x7fbc8c15de50 2023-10-08T07:13:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.621+0000 7fbc8b7fe640 1 -- 172.21.15.70:0/1501344260 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80031d60 con 0x7fbc8c15de50 2023-10-08T07:13:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.621+0000 7fbc93125640 1 -- 172.21.15.70:0/1501344260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c15de50 msgr2=0x7fbc8c0b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.621+0000 7fbc93125640 1 --2- 172.21.15.70:0/1501344260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c15de50 0x7fbc8c0b3ef0 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009870 tx=0x7fbc8002b910 comp rx=0 tx=0).stop 2023-10-08T07:13:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.621+0000 7fbc93125640 1 -- 172.21.15.70:0/1501344260 shutdown_connections 2023-10-08T07:13:13.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.621+0000 7fbc93125640 1 --2- 172.21.15.70:0/1501344260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c15de50 0x7fbc8c0b3ef0 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:13.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.621+0000 7fbc93125640 1 -- 172.21.15.70:0/1501344260 >> 172.21.15.70:0/1501344260 conn(0x7fbc8c069d50 msgr2=0x7fbc8c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:13.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.621+0000 7fbc93125640 1 -- 172.21.15.70:0/1501344260 shutdown_connections 2023-10-08T07:13:13.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.621+0000 7fbc93125640 1 -- 172.21.15.70:0/1501344260 wait complete. 2023-10-08T07:13:13.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.623+0000 7fbc93125640 1 Processor -- start 2023-10-08T07:13:13.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.623+0000 7fbc93125640 1 -- start start 2023-10-08T07:13:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.624+0000 7fbc93125640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc8c0cb850 0x7fbc8c151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.624+0000 7fbc93125640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc8c15de50 0x7fbc8c0c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.624+0000 7fbc93125640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0c2e00 0x7fbc8c0c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.624+0000 7fbc93125640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0cdd10 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.624+0000 7fbc93125640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0c4f70 con 0x7fbc8c15de50 2023-10-08T07:13:13.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.624+0000 7fbc93125640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c0abb90 con 0x7fbc8c0cb850 2023-10-08T07:13:13.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.624+0000 7fbc916c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0c2e00 0x7fbc8c0c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:13.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.625+0000 7fbc916c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0c2e00 0x7fbc8c0c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43086/0 (socket says 172.21.15.70:43086) 2023-10-08T07:13:13.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.625+0000 7fbc916c2640 1 -- 172.21.15.70:0/1652127481 learned_addr learned my addr 172.21.15.70:0/1652127481 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:13.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.625+0000 7fbc8bfff640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc8c15de50 0x7fbc8c0c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:13.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.625+0000 7fbc90ec1640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc8c0cb850 0x7fbc8c151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:13.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.625+0000 7fbc916c2640 1 -- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc8c0cb850 msgr2=0x7fbc8c151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:13.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.625+0000 7fbc916c2640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc8c0cb850 0x7fbc8c151b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:13.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.625+0000 7fbc916c2640 1 -- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc8c15de50 msgr2=0x7fbc8c0c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:13.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.625+0000 7fbc916c2640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc8c15de50 0x7fbc8c0c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:13.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.625+0000 7fbc916c2640 1 -- 172.21.15.70:0/1652127481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc800094d0 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.625+0000 7fbc916c2640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0c2e00 0x7fbc8c0c4a30 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400c730 tx=0x7fbc8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:13.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.626+0000 7fbc897fa640 1 -- 172.21.15.70:0/1652127481 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84004d60 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.626+0000 7fbc897fa640 1 -- 172.21.15.70:0/1652127481 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8400a640 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.626+0000 7fbc897fa640 1 -- 172.21.15.70:0/1652127481 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400a9f0 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.626+0000 7fbc93125640 1 -- 172.21.15.70:0/1652127481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8c0abe70 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.626+0000 7fbc93125640 1 -- 172.21.15.70:0/1652127481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8c0ac430 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.627+0000 7fbc93125640 1 -- 172.21.15.70:0/1652127481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8c04e160 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.628+0000 7fbc897fa640 1 -- 172.21.15.70:0/1652127481 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbc84007070 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.629+0000 7fbc897fa640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fbc7006af90 0x7fbc7006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:13.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.629+0000 7fbc897fa640 1 -- 172.21.15.70:0/1652127481 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6596+0+0 (secure 0 0 0) 0x7fbc84088fc0 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.629+0000 7fbc90ec1640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fbc7006af90 0x7fbc7006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:13.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.630+0000 7fbc90ec1640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fbc7006af90 0x7fbc7006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbc80036000 tx=0x7fbc8003c040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:13.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.633+0000 7fbc897fa640 1 -- 172.21.15.70:0/1652127481 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc840bbca0 con 0x7fbc8c0c2e00 2023-10-08T07:13:13.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:13.825+0000 7fbc93125640 1 -- 172.21.15.70:0/1652127481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbc8c06ca10 con 0x7fbc8c0c2e00 2023-10-08T07:13:14.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.254+0000 7fbc897fa640 1 -- 172.21.15.70:0/1652127481 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v225) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbc840180d0 con 0x7fbc8c0c2e00 2023-10-08T07:13:14.256 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T07:13:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.258+0000 7fbc93125640 1 -- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fbc7006af90 msgr2=0x7fbc7006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.258+0000 7fbc93125640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fbc7006af90 0x7fbc7006d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbc80036000 tx=0x7fbc8003c040 comp rx=0 tx=0).stop 2023-10-08T07:13:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.258+0000 7fbc93125640 1 -- 172.21.15.70:0/1652127481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0c2e00 msgr2=0x7fbc8c0c4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.258+0000 7fbc93125640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0c2e00 0x7fbc8c0c4a30 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400c730 tx=0x7fbc8400cbf0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.260+0000 7fbc93125640 1 -- 172.21.15.70:0/1652127481 shutdown_connections 2023-10-08T07:13:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.260+0000 7fbc93125640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fbc7006af90 0x7fbc7006d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.260+0000 7fbc93125640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8c0c2e00 0x7fbc8c0c4a30 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.260+0000 7fbc93125640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc8c15de50 0x7fbc8c0c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.260+0000 7fbc93125640 1 --2- 172.21.15.70:0/1652127481 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc8c0cb850 0x7fbc8c151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.260+0000 7fbc93125640 1 -- 172.21.15.70:0/1652127481 >> 172.21.15.70:0/1652127481 conn(0x7fbc8c069d50 msgr2=0x7fbc8c0ae880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.261+0000 7fbc93125640 1 -- 172.21.15.70:0/1652127481 shutdown_connections 2023-10-08T07:13:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.261+0000 7fbc93125640 1 -- 172.21.15.70:0/1652127481 wait complete. 2023-10-08T07:13:14.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T07:13:14.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.499+0000 7f7261316640 1 Processor -- start 2023-10-08T07:13:14.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.499+0000 7f7261316640 1 -- start start 2023-10-08T07:13:14.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.500+0000 7f7261316640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f725c06c3a0 0x7f725c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:14.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f7261316640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f725c06db60 0x7f725c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:14.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f7261316640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f725c15aa00 0x7f725c0b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:14.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f7261316640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c0ca230 con 0x7f725c06c3a0 2023-10-08T07:13:14.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f7261316640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c15b310 con 0x7f725c06db60 2023-10-08T07:13:14.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f7261316640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c06d290 con 0x7f725c15aa00 2023-10-08T07:13:14.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f725affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f725c06c3a0 0x7f725c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:14.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f725affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f725c06c3a0 0x7f725c0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43088/0 (socket says 172.21.15.70:43088) 2023-10-08T07:13:14.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f725affd640 1 -- 172.21.15.70:0/3704848635 learned_addr learned my addr 172.21.15.70:0/3704848635 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:14.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f725affd640 1 -- 172.21.15.70:0/3704848635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f725c15aa00 msgr2=0x7f725c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:14.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f725affd640 1 --2- 172.21.15.70:0/3704848635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f725c15aa00 0x7f725c0b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f725affd640 1 -- 172.21.15.70:0/3704848635 >> v2:172.21.15.181:3300/0 conn(0x7f725c06db60 msgr2=0x7f725c06c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:14.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f725affd640 1 --2- 172.21.15.70:0/3704848635 >> v2:172.21.15.181:3300/0 conn(0x7f725c06db60 0x7f725c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.501+0000 7f725affd640 1 -- 172.21.15.70:0/3704848635 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c0c9860 con 0x7f725c06c3a0 2023-10-08T07:13:14.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.502+0000 7f725affd640 1 --2- 172.21.15.70:0/3704848635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c06c3a0 0x7f725c0c9cf0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f725c0715c0 tx=0x7f724c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47e4505a9d5809a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:14.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f725a7fc640 1 -- 172.21.15.70:0/3704848635 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c031870 con 0x7f725c06c3a0 2023-10-08T07:13:14.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f725a7fc640 1 -- 172.21.15.70:0/3704848635 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c0319d0 con 0x7f725c06c3a0 2023-10-08T07:13:14.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f725a7fc640 1 -- 172.21.15.70:0/3704848635 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c031d60 con 0x7f725c06c3a0 2023-10-08T07:13:14.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f7261316640 1 -- 172.21.15.70:0/3704848635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c06c3a0 msgr2=0x7f725c0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:14.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f7261316640 1 --2- 172.21.15.70:0/3704848635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c06c3a0 0x7f725c0c9cf0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f725c0715c0 tx=0x7f724c02b910 comp rx=0 tx=0).stop 2023-10-08T07:13:14.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f7261316640 1 -- 172.21.15.70:0/3704848635 shutdown_connections 2023-10-08T07:13:14.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f7261316640 1 --2- 172.21.15.70:0/3704848635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f725c15aa00 0x7f725c0b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f7261316640 1 --2- 172.21.15.70:0/3704848635 >> v2:172.21.15.181:3300/0 conn(0x7f725c06db60 0x7f725c06c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f7261316640 1 --2- 172.21.15.70:0/3704848635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c06c3a0 0x7f725c0c9cf0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f7261316640 1 -- 172.21.15.70:0/3704848635 >> 172.21.15.70:0/3704848635 conn(0x7f725c06a5d0 msgr2=0x7f725c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:14.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f7261316640 1 -- 172.21.15.70:0/3704848635 shutdown_connections 2023-10-08T07:13:14.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.503+0000 7f7261316640 1 -- 172.21.15.70:0/3704848635 wait complete. 2023-10-08T07:13:14.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.505+0000 7f7261316640 1 Processor -- start 2023-10-08T07:13:14.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.505+0000 7f7261316640 1 -- start start 2023-10-08T07:13:14.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.506+0000 7f7261316640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c06c3a0 0x7f725c0c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:14.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.506+0000 7f7261316640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f725c06db60 0x7f725c152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.506+0000 7f7261316640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c15aa00 0x7f725c152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.506+0000 7f7261316640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0c79b0 con 0x7f725c15aa00 2023-10-08T07:13:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.506+0000 7f7261316640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c152570 con 0x7f725c06c3a0 2023-10-08T07:13:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.506+0000 7f7261316640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0c6be0 con 0x7f725c06db60 2023-10-08T07:13:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c15aa00 0x7f725c152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f725b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c15aa00 0x7f725c152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43090/0 (socket says 172.21.15.70:43090) 2023-10-08T07:13:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f725b7fe640 1 -- 172.21.15.70:0/1048318792 learned_addr learned my addr 172.21.15.70:0/1048318792 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f7253fff640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f725c06db60 0x7f725c152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f725affd640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c06c3a0 0x7f725c0c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f725b7fe640 1 -- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f725c06db60 msgr2=0x7f725c152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:14.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f725b7fe640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f725c06db60 0x7f725c152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f725b7fe640 1 -- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c06c3a0 msgr2=0x7f725c0c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:14.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f725b7fe640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c06c3a0 0x7f725c0c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f725b7fe640 1 -- 172.21.15.70:0/1048318792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c0094d0 con 0x7f725c15aa00 2023-10-08T07:13:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.507+0000 7f725b7fe640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c15aa00 0x7f725c152730 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f725400b970 tx=0x7f725400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.508+0000 7f72537fe640 1 -- 172.21.15.70:0/1048318792 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725400cb40 con 0x7f725c15aa00 2023-10-08T07:13:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.508+0000 7f72537fe640 1 -- 172.21.15.70:0/1048318792 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725400ccc0 con 0x7f725c15aa00 2023-10-08T07:13:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.508+0000 7f72537fe640 1 -- 172.21.15.70:0/1048318792 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254014680 con 0x7f725c15aa00 2023-10-08T07:13:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.508+0000 7f7261316640 1 -- 172.21.15.70:0/1048318792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c0c6ec0 con 0x7f725c15aa00 2023-10-08T07:13:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.508+0000 7f7261316640 1 -- 172.21.15.70:0/1048318792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c15b550 con 0x7f725c15aa00 2023-10-08T07:13:14.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.509+0000 7f7261316640 1 -- 172.21.15.70:0/1048318792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c04e160 con 0x7f725c15aa00 2023-10-08T07:13:14.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.510+0000 7f72537fe640 1 -- 172.21.15.70:0/1048318792 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f725401d3b0 con 0x7f725c15aa00 2023-10-08T07:13:14.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.511+0000 7f72537fe640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f723c06af90 0x7f723c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.511+0000 7f72537fe640 1 -- 172.21.15.70:0/1048318792 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6095+0+0 (secure 0 0 0) 0x7f72540898b0 con 0x7f725c15aa00 2023-10-08T07:13:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.512+0000 7f725affd640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f723c06af90 0x7f723c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.513+0000 7f725affd640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f723c06af90 0x7f723c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f724c036000 tx=0x7f724c03c040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:14.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.515+0000 7f72537fe640 1 -- 172.21.15.70:0/1048318792 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f72540546d0 con 0x7f725c15aa00 2023-10-08T07:13:14.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.717+0000 7f7261316640 1 -- 172.21.15.70:0/1048318792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f725c15a0b0 con 0x7f725c15aa00 2023-10-08T07:13:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.722+0000 7f72537fe640 1 -- 172.21.15.70:0/1048318792 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f72540190a0 con 0x7f725c15aa00 2023-10-08T07:13:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.724+0000 7f7261316640 1 -- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f723c06af90 msgr2=0x7f723c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:14.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.724+0000 7f7261316640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f723c06af90 0x7f723c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f724c036000 tx=0x7f724c03c040 comp rx=0 tx=0).stop 2023-10-08T07:13:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.724+0000 7f7261316640 1 -- 172.21.15.70:0/1048318792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c15aa00 msgr2=0x7f725c152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.724+0000 7f7261316640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c15aa00 0x7f725c152730 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f725400b970 tx=0x7f725400be30 comp rx=0 tx=0).stop 2023-10-08T07:13:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.725+0000 7f7261316640 1 -- 172.21.15.70:0/1048318792 shutdown_connections 2023-10-08T07:13:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.725+0000 7f7261316640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f723c06af90 0x7f723c06d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.725+0000 7f7261316640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f725c15aa00 0x7f725c152730 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.725+0000 7f7261316640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f725c06db60 0x7f725c152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.725+0000 7f7261316640 1 --2- 172.21.15.70:0/1048318792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f725c06c3a0 0x7f725c0c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.725+0000 7f7261316640 1 -- 172.21.15.70:0/1048318792 >> 172.21.15.70:0/1048318792 conn(0x7f725c06a5d0 msgr2=0x7f725c0ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.725+0000 7f7261316640 1 -- 172.21.15.70:0/1048318792 shutdown_connections 2023-10-08T07:13:14.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:14.725+0000 7f7261316640 1 -- 172.21.15.70:0/1048318792 wait complete. 2023-10-08T07:13:14.922 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T07:13:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.147+0000 7fd9b1c21640 1 Processor -- start 2023-10-08T07:13:15.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.147+0000 7fd9b1c21640 1 -- start start 2023-10-08T07:13:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.149+0000 7fd9b1c21640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd9ac15aad0 0x7fd9ac0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.149+0000 7fd9b1c21640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd9ac0c82b0 0x7fd9ac0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.149+0000 7fd9b1c21640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9ac06cd00 0x7fd9ac06d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.149+0000 7fd9b1c21640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ac06d6b0 con 0x7fd9ac0c82b0 2023-10-08T07:13:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.149+0000 7fd9b1c21640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ac15bf30 con 0x7fd9ac15aad0 2023-10-08T07:13:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.149+0000 7fd9b1c21640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac15b320 con 0x7fd9ac06cd00 2023-10-08T07:13:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.149+0000 7fd9aaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd9ac0c82b0 0x7fd9ac0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.149+0000 7fd9aaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd9ac0c82b0 0x7fd9ac0c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43094/0 (socket says 172.21.15.70:43094) 2023-10-08T07:13:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.149+0000 7fd9aaffd640 1 -- 172.21.15.70:0/677366838 learned_addr learned my addr 172.21.15.70:0/677366838 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.150+0000 7fd9ab7fe640 1 --2- 172.21.15.70:0/677366838 >> v2:172.21.15.181:3300/0 conn(0x7fd9ac15aad0 0x7fd9ac0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.150+0000 7fd9abfff640 1 --2- 172.21.15.70:0/677366838 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9ac06cd00 0x7fd9ac06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.150+0000 7fd9aaffd640 1 -- 172.21.15.70:0/677366838 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9ac06cd00 msgr2=0x7fd9ac06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.150+0000 7fd9aaffd640 1 --2- 172.21.15.70:0/677366838 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9ac06cd00 0x7fd9ac06d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.150+0000 7fd9aaffd640 1 -- 172.21.15.70:0/677366838 >> v2:172.21.15.181:3300/0 conn(0x7fd9ac15aad0 msgr2=0x7fd9ac0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.150+0000 7fd9aaffd640 1 --2- 172.21.15.70:0/677366838 >> v2:172.21.15.181:3300/0 conn(0x7fd9ac15aad0 0x7fd9ac0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.150+0000 7fd9aaffd640 1 -- 172.21.15.70:0/677366838 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac15b600 con 0x7fd9ac0c82b0 2023-10-08T07:13:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.151+0000 7fd9aaffd640 1 --2- 172.21.15.70:0/677366838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac0c82b0 0x7fd9ac0c8710 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac06e6a0 tx=0x7fd9a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23361994dce86da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9aa7fc640 1 -- 172.21.15.70:0/677366838 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0031870 con 0x7fd9ac0c82b0 2023-10-08T07:13:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9aa7fc640 1 -- 172.21.15.70:0/677366838 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a00319d0 con 0x7fd9ac0c82b0 2023-10-08T07:13:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9aa7fc640 1 -- 172.21.15.70:0/677366838 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0031d60 con 0x7fd9ac0c82b0 2023-10-08T07:13:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9b1c21640 1 -- 172.21.15.70:0/677366838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac0c82b0 msgr2=0x7fd9ac0c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9b1c21640 1 --2- 172.21.15.70:0/677366838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac0c82b0 0x7fd9ac0c8710 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac06e6a0 tx=0x7fd9a002b910 comp rx=0 tx=0).stop 2023-10-08T07:13:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9b1c21640 1 -- 172.21.15.70:0/677366838 shutdown_connections 2023-10-08T07:13:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9b1c21640 1 --2- 172.21.15.70:0/677366838 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9ac06cd00 0x7fd9ac06d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9b1c21640 1 --2- 172.21.15.70:0/677366838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac0c82b0 0x7fd9ac0c8710 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9b1c21640 1 --2- 172.21.15.70:0/677366838 >> v2:172.21.15.181:3300/0 conn(0x7fd9ac15aad0 0x7fd9ac0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9b1c21640 1 -- 172.21.15.70:0/677366838 >> 172.21.15.70:0/677366838 conn(0x7fd9ac069d50 msgr2=0x7fd9ac06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9b1c21640 1 -- 172.21.15.70:0/677366838 shutdown_connections 2023-10-08T07:13:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.152+0000 7fd9b1c21640 1 -- 172.21.15.70:0/677366838 wait complete. 2023-10-08T07:13:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.154+0000 7fd9b1c21640 1 Processor -- start 2023-10-08T07:13:15.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.154+0000 7fd9b1c21640 1 -- start start 2023-10-08T07:13:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.155+0000 7fd9b1c21640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9ac06cd00 0x7fd9ac1517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.155+0000 7fd9b1c21640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9ac0c82b0 0x7fd9ac15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.155+0000 7fd9b1c21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac15aad0 0x7fd9ac14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.155+0000 7fd9b1c21640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac15dd90 con 0x7fd9ac15aad0 2023-10-08T07:13:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.155+0000 7fd9b1c21640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac0c8e60 con 0x7fd9ac06cd00 2023-10-08T07:13:15.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.155+0000 7fd9b1c21640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac14fc00 con 0x7fd9ac0c82b0 2023-10-08T07:13:15.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.155+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac15aad0 0x7fd9ac14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.156+0000 7fd9abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac15aad0 0x7fd9ac14ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43098/0 (socket says 172.21.15.70:43098) 2023-10-08T07:13:15.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.156+0000 7fd9abfff640 1 -- 172.21.15.70:0/2299467460 learned_addr learned my addr 172.21.15.70:0/2299467460 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:15.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.156+0000 7fd9ab7fe640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9ac06cd00 0x7fd9ac1517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.156+0000 7fd9aaffd640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9ac0c82b0 0x7fd9ac15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.156+0000 7fd9abfff640 1 -- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9ac0c82b0 msgr2=0x7fd9ac15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.156+0000 7fd9abfff640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9ac0c82b0 0x7fd9ac15d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.156+0000 7fd9abfff640 1 -- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9ac06cd00 msgr2=0x7fd9ac1517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.156+0000 7fd9abfff640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9ac06cd00 0x7fd9ac1517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.156+0000 7fd9abfff640 1 -- 172.21.15.70:0/2299467460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a00094d0 con 0x7fd9ac15aad0 2023-10-08T07:13:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.156+0000 7fd9abfff640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac15aad0 0x7fd9ac14ee60 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4007d20 tx=0x7fd9a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:15.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.157+0000 7fd993fff640 1 -- 172.21.15.70:0/2299467460 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4014420 con 0x7fd9ac15aad0 2023-10-08T07:13:15.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.157+0000 7fd993fff640 1 -- 172.21.15.70:0/2299467460 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a40145a0 con 0x7fd9ac15aad0 2023-10-08T07:13:15.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.157+0000 7fd993fff640 1 -- 172.21.15.70:0/2299467460 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4014950 con 0x7fd9ac15aad0 2023-10-08T07:13:15.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.157+0000 7fd9b1c21640 1 -- 172.21.15.70:0/2299467460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ac15daf0 con 0x7fd9ac15aad0 2023-10-08T07:13:15.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.157+0000 7fd9b1c21640 1 -- 172.21.15.70:0/2299467460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ac150180 con 0x7fd9ac15aad0 2023-10-08T07:13:15.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.158+0000 7fd9b1c21640 1 -- 172.21.15.70:0/2299467460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ac0c8fd0 con 0x7fd9ac15aad0 2023-10-08T07:13:15.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.159+0000 7fd993fff640 1 -- 172.21.15.70:0/2299467460 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd9a4014ab0 con 0x7fd9ac15aad0 2023-10-08T07:13:15.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.160+0000 7fd993fff640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd99806afc0 0x7fd99806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.160+0000 7fd993fff640 1 -- 172.21.15.70:0/2299467460 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6095+0+0 (secure 0 0 0) 0x7fd9a408a170 con 0x7fd9ac15aad0 2023-10-08T07:13:15.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.160+0000 7fd9ab7fe640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd99806afc0 0x7fd99806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.162+0000 7fd9ab7fe640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd99806afc0 0x7fd99806d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd99c006fd0 tx=0x7fd99c008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:15.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.164+0000 7fd993fff640 1 -- 172.21.15.70:0/2299467460 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9a4054f90 con 0x7fd9ac15aad0 2023-10-08T07:13:15.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.353+0000 7fd9b1c21640 1 -- 172.21.15.70:0/2299467460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fd9ac04e160 con 0x7fd9ac15aad0 2023-10-08T07:13:15.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.360+0000 7fd993fff640 1 -- 172.21.15.70:0/2299467460 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v74) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd9a40585b0 con 0x7fd9ac15aad0 2023-10-08T07:13:15.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 -- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd99806afc0 msgr2=0x7fd99806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:15.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd99806afc0 0x7fd99806d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd99c006fd0 tx=0x7fd99c008040 comp rx=0 tx=0).stop 2023-10-08T07:13:15.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 -- 172.21.15.70:0/2299467460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac15aad0 msgr2=0x7fd9ac14ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:15.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac15aad0 0x7fd9ac14ee60 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4007d20 tx=0x7fd9a400e4b0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 -- 172.21.15.70:0/2299467460 shutdown_connections 2023-10-08T07:13:15.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd99806afc0 0x7fd99806d480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9ac15aad0 0x7fd9ac14ee60 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd9ac0c82b0 0x7fd9ac15d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 --2- 172.21.15.70:0/2299467460 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd9ac06cd00 0x7fd9ac1517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 -- 172.21.15.70:0/2299467460 >> 172.21.15.70:0/2299467460 conn(0x7fd9ac069d50 msgr2=0x7fd9ac0cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:15.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 -- 172.21.15.70:0/2299467460 shutdown_connections 2023-10-08T07:13:15.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.363+0000 7fd9b1c21640 1 -- 172.21.15.70:0/2299467460 wait complete. 2023-10-08T07:13:15.387 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T07:13:15.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.601+0000 7fb1769e2640 1 Processor -- start 2023-10-08T07:13:15.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.602+0000 7fb1769e2640 1 -- start start 2023-10-08T07:13:15.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.603+0000 7fb1769e2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb1700afbe0 0x7fb1700b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.603+0000 7fb1769e2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb17006dfc0 0x7fb170070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.603+0000 7fb1769e2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1700b2620 con 0x7fb170071070 2023-10-08T07:13:15.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.603+0000 7fb1769e2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1700b2060 con 0x7fb17006dfc0 2023-10-08T07:13:15.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.603+0000 7fb1769e2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1700b2340 con 0x7fb1700afbe0 2023-10-08T07:13:15.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.603+0000 7fb167fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb17006dfc0 0x7fb170070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.603+0000 7fb167fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb17006dfc0 0x7fb170070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43108/0 (socket says 172.21.15.70:43108) 2023-10-08T07:13:15.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.603+0000 7fb167fff640 1 -- 172.21.15.70:0/496862820 learned_addr learned my addr 172.21.15.70:0/496862820 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:15.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.604+0000 7fb16ffff640 1 --2- 172.21.15.70:0/496862820 >> v2:172.21.15.181:3300/0 conn(0x7fb1700afbe0 0x7fb1700b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.604+0000 7fb167fff640 1 -- 172.21.15.70:0/496862820 >> v2:172.21.15.181:3300/0 conn(0x7fb1700afbe0 msgr2=0x7fb1700b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.604+0000 7fb167fff640 1 --2- 172.21.15.70:0/496862820 >> v2:172.21.15.181:3300/0 conn(0x7fb1700afbe0 0x7fb1700b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.604+0000 7fb167fff640 1 -- 172.21.15.70:0/496862820 >> v1:172.21.15.181:6789/0 conn(0x7fb170071070 legacy=0x7fb1700cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.604+0000 7fb167fff640 1 -- 172.21.15.70:0/496862820 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb170070350 con 0x7fb17006dfc0 2023-10-08T07:13:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.605+0000 7fb167fff640 1 --2- 172.21.15.70:0/496862820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 0x7fb170070b30 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fb17006e640 tx=0x7fb15802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeba8d38f833bde5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.606+0000 7fb16f7fe640 1 -- 172.21.15.70:0/496862820 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158031870 con 0x7fb17006dfc0 2023-10-08T07:13:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.606+0000 7fb16f7fe640 1 -- 172.21.15.70:0/496862820 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1580319d0 con 0x7fb17006dfc0 2023-10-08T07:13:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.606+0000 7fb16f7fe640 1 -- 172.21.15.70:0/496862820 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158031d60 con 0x7fb17006dfc0 2023-10-08T07:13:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.606+0000 7fb1769e2640 1 -- 172.21.15.70:0/496862820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 msgr2=0x7fb170070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.606+0000 7fb1769e2640 1 --2- 172.21.15.70:0/496862820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 0x7fb170070b30 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fb17006e640 tx=0x7fb15802b910 comp rx=0 tx=0).stop 2023-10-08T07:13:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.606+0000 7fb1769e2640 1 -- 172.21.15.70:0/496862820 shutdown_connections 2023-10-08T07:13:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.606+0000 7fb1769e2640 1 --2- 172.21.15.70:0/496862820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 0x7fb170070b30 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.606+0000 7fb1769e2640 1 --2- 172.21.15.70:0/496862820 >> v2:172.21.15.181:3300/0 conn(0x7fb1700afbe0 0x7fb1700b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.606+0000 7fb1769e2640 1 -- 172.21.15.70:0/496862820 >> 172.21.15.70:0/496862820 conn(0x7fb17006a5d0 msgr2=0x7fb1700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.607+0000 7fb1769e2640 1 -- 172.21.15.70:0/496862820 shutdown_connections 2023-10-08T07:13:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.607+0000 7fb1769e2640 1 -- 172.21.15.70:0/496862820 wait complete. 2023-10-08T07:13:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.609+0000 7fb1769e2640 1 Processor -- start 2023-10-08T07:13:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.609+0000 7fb1769e2640 1 -- start start 2023-10-08T07:13:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.610+0000 7fb1769e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 0x7fb170144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.610+0000 7fb1769e2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb170071070 0x7fb1700ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.610+0000 7fb1769e2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1700afbe0 0x7fb1700c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.610+0000 7fb1769e2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1700cd650 con 0x7fb17006dfc0 2023-10-08T07:13:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.610+0000 7fb1769e2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1700cd370 con 0x7fb170071070 2023-10-08T07:13:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.610+0000 7fb1769e2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1700ce030 con 0x7fb1700afbe0 2023-10-08T07:13:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.610+0000 7fb16ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 0x7fb170144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.610+0000 7fb16ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 0x7fb170144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43124/0 (socket says 172.21.15.70:43124) 2023-10-08T07:13:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.610+0000 7fb16ffff640 1 -- 172.21.15.70:0/3466808029 learned_addr learned my addr 172.21.15.70:0/3466808029 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.611+0000 7fb167fff640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb170071070 0x7fb1700ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.611+0000 7fb174f7f640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1700afbe0 0x7fb1700c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.611+0000 7fb16ffff640 1 -- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1700afbe0 msgr2=0x7fb1700c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.611+0000 7fb16ffff640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1700afbe0 0x7fb1700c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.611+0000 7fb16ffff640 1 -- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb170071070 msgr2=0x7fb1700ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.611+0000 7fb16ffff640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb170071070 0x7fb1700ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.611+0000 7fb16ffff640 1 -- 172.21.15.70:0/3466808029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1580094d0 con 0x7fb17006dfc0 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.611+0000 7fb167fff640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb170071070 0x7fb1700ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.611+0000 7fb16ffff640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 0x7fb170144a90 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fb16000d800 tx=0x7fb16000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.612+0000 7fb16d7fa640 1 -- 172.21.15.70:0/3466808029 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000b6c0 con 0x7fb17006dfc0 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.612+0000 7fb16d7fa640 1 -- 172.21.15.70:0/3466808029 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16000b820 con 0x7fb17006dfc0 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.612+0000 7fb16d7fa640 1 -- 172.21.15.70:0/3466808029 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000bbb0 con 0x7fb17006dfc0 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.612+0000 7fb1769e2640 1 -- 172.21.15.70:0/3466808029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1700ce310 con 0x7fb17006dfc0 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.612+0000 7fb1769e2640 1 -- 172.21.15.70:0/3466808029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb170144fd0 con 0x7fb17006dfc0 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.613+0000 7fb1769e2640 1 -- 172.21.15.70:0/3466808029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1700cde90 con 0x7fb17006dfc0 2023-10-08T07:13:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.614+0000 7fb16d7fa640 1 -- 172.21.15.70:0/3466808029 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb16000bd10 con 0x7fb17006dfc0 2023-10-08T07:13:15.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.615+0000 7fb16d7fa640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb15006af90 0x7fb15006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:15.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.615+0000 7fb16d7fa640 1 -- 172.21.15.70:0/3466808029 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6095+0+0 (secure 0 0 0) 0x7fb160089eb0 con 0x7fb17006dfc0 2023-10-08T07:13:15.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.615+0000 7fb167fff640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb15006af90 0x7fb15006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:15.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.617+0000 7fb167fff640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb15006af90 0x7fb15006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb158036000 tx=0x7fb15803c040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:15.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.619+0000 7fb16d7fa640 1 -- 172.21.15.70:0/3466808029 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1700cde90 con 0x7fb17006dfc0 2023-10-08T07:13:15.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.809+0000 7fb1769e2640 1 -- 172.21.15.70:0/3466808029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fb1700cde90 con 0x7fb17006dfc0 2023-10-08T07:13:15.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.816+0000 7fb16d7fa640 1 -- 172.21.15.70:0/3466808029 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v75) v1 ==== 120+0+0 (secure 0 0 0) 0x7fb1700cde90 con 0x7fb17006dfc0 2023-10-08T07:13:15.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 -- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb15006af90 msgr2=0x7fb15006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:15.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb15006af90 0x7fb15006d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb158036000 tx=0x7fb15803c040 comp rx=0 tx=0).stop 2023-10-08T07:13:15.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 -- 172.21.15.70:0/3466808029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 msgr2=0x7fb170144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:15.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 0x7fb170144a90 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fb16000d800 tx=0x7fb16000dcc0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 -- 172.21.15.70:0/3466808029 shutdown_connections 2023-10-08T07:13:15.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb1700afbe0 0x7fb1700c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb170071070 0x7fb1700ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb15006af90 0x7fb15006d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 --2- 172.21.15.70:0/3466808029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17006dfc0 0x7fb170144a90 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:15.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 -- 172.21.15.70:0/3466808029 >> 172.21.15.70:0/3466808029 conn(0x7fb17006a5d0 msgr2=0x7fb1700ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:15.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 -- 172.21.15.70:0/3466808029 shutdown_connections 2023-10-08T07:13:15.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:15.819+0000 7fb1769e2640 1 -- 172.21.15.70:0/3466808029 wait complete. 2023-10-08T07:13:15.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-08T07:13:16.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.189+0000 7ffad73d7640 1 Processor -- start 2023-10-08T07:13:16.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.189+0000 7ffad73d7640 1 -- start start 2023-10-08T07:13:16.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.190+0000 7ffad73d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffad01599d0 0x7ffad00ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:16.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.190+0000 7ffad73d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffad00b07d0 0x7ffad006c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:16.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.190+0000 7ffad73d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad00cacf0 con 0x7ffad00cc210 2023-10-08T07:13:16.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.190+0000 7ffad73d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad00b10f0 con 0x7ffad00b07d0 2023-10-08T07:13:16.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.190+0000 7ffad73d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00ca3c0 con 0x7ffad01599d0 2023-10-08T07:13:16.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.191+0000 7ffad6bd6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffad00b07d0 0x7ffad006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:16.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.191+0000 7ffad6bd6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffad00b07d0 0x7ffad006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43126/0 (socket says 172.21.15.70:43126) 2023-10-08T07:13:16.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.191+0000 7ffad6bd6640 1 -- 172.21.15.70:0/673701618 learned_addr learned my addr 172.21.15.70:0/673701618 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:16.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.191+0000 7ffad63d5640 1 --2- 172.21.15.70:0/673701618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffad01599d0 0x7ffad00ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:16.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.191+0000 7ffad6bd6640 1 -- 172.21.15.70:0/673701618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffad01599d0 msgr2=0x7ffad00ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:16.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.191+0000 7ffad6bd6640 1 --2- 172.21.15.70:0/673701618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffad01599d0 0x7ffad00ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:16.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.191+0000 7ffad6bd6640 1 -- 172.21.15.70:0/673701618 >> v1:172.21.15.181:6789/0 conn(0x7ffad00cc210 legacy=0x7ffad00b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:16.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.191+0000 7ffad6bd6640 1 -- 172.21.15.70:0/673701618 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad00ca6a0 con 0x7ffad00b07d0 2023-10-08T07:13:16.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.192+0000 7ffad6bd6640 1 --2- 172.21.15.70:0/673701618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00b07d0 0x7ffad006c310 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009e50 tx=0x7ffac802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1136799f59fc19ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:16.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.193+0000 7ffad53d3640 1 -- 172.21.15.70:0/673701618 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac802ca70 con 0x7ffad00b07d0 2023-10-08T07:13:16.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.193+0000 7ffad53d3640 1 -- 172.21.15.70:0/673701618 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac802cbd0 con 0x7ffad00b07d0 2023-10-08T07:13:16.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.193+0000 7ffad53d3640 1 -- 172.21.15.70:0/673701618 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80343f0 con 0x7ffad00b07d0 2023-10-08T07:13:16.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.193+0000 7ffad73d7640 1 -- 172.21.15.70:0/673701618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00b07d0 msgr2=0x7ffad006c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:16.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.193+0000 7ffad73d7640 1 --2- 172.21.15.70:0/673701618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00b07d0 0x7ffad006c310 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009e50 tx=0x7ffac802bed0 comp rx=0 tx=0).stop 2023-10-08T07:13:16.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.193+0000 7ffad73d7640 1 -- 172.21.15.70:0/673701618 shutdown_connections 2023-10-08T07:13:16.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.193+0000 7ffad73d7640 1 --2- 172.21.15.70:0/673701618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00b07d0 0x7ffad006c310 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:16.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.193+0000 7ffad73d7640 1 --2- 172.21.15.70:0/673701618 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffad01599d0 0x7ffad00ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:16.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.193+0000 7ffad73d7640 1 -- 172.21.15.70:0/673701618 >> 172.21.15.70:0/673701618 conn(0x7ffad00cb0c0 msgr2=0x7ffad00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:16.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.194+0000 7ffad73d7640 1 -- 172.21.15.70:0/673701618 shutdown_connections 2023-10-08T07:13:16.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.194+0000 7ffad73d7640 1 -- 172.21.15.70:0/673701618 wait complete. 2023-10-08T07:13:16.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.195+0000 7ffad73d7640 1 Processor -- start 2023-10-08T07:13:16.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.195+0000 7ffad73d7640 1 -- start start 2023-10-08T07:13:16.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.196+0000 7ffad73d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffad00b07d0 0x7ffad00acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:16.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.196+0000 7ffad73d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00cc210 0x7ffad0166e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:16.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.196+0000 7ffad73d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad01599d0 0x7ffad015bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:16.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.196+0000 7ffad73d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad01566c0 con 0x7ffad00cc210 2023-10-08T07:13:16.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.196+0000 7ffad73d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00ab8d0 con 0x7ffad01599d0 2023-10-08T07:13:16.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad73d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad00abbb0 con 0x7ffad00b07d0 2023-10-08T07:13:16.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad5bd4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00cc210 0x7ffad0166e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:16.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad5bd4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00cc210 0x7ffad0166e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43130/0 (socket says 172.21.15.70:43130) 2023-10-08T07:13:16.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad5bd4640 1 -- 172.21.15.70:0/4005892870 learned_addr learned my addr 172.21.15.70:0/4005892870 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:16.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad63d5640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffad00b07d0 0x7ffad00acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:16.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad6bd6640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad01599d0 0x7ffad015bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:16.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad5bd4640 1 -- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffad00b07d0 msgr2=0x7ffad00acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:16.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad5bd4640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffad00b07d0 0x7ffad00acc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:16.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad5bd4640 1 -- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad01599d0 msgr2=0x7ffad015bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:16.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad5bd4640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad01599d0 0x7ffad015bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:16.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.197+0000 7ffad5bd4640 1 -- 172.21.15.70:0/4005892870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac8009b30 con 0x7ffad00cc210 2023-10-08T07:13:16.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.198+0000 7ffad5bd4640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00cc210 0x7ffad0166e20 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c400 tx=0x7ffac000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:16.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.198+0000 7ffac6ffd640 1 -- 172.21.15.70:0/4005892870 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0011790 con 0x7ffad00cc210 2023-10-08T07:13:16.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.198+0000 7ffac6ffd640 1 -- 172.21.15.70:0/4005892870 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac0011930 con 0x7ffad00cc210 2023-10-08T07:13:16.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.198+0000 7ffac6ffd640 1 -- 172.21.15.70:0/4005892870 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0011ce0 con 0x7ffad00cc210 2023-10-08T07:13:16.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.198+0000 7ffad73d7640 1 -- 172.21.15.70:0/4005892870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad00abe90 con 0x7ffad00cc210 2023-10-08T07:13:16.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.198+0000 7ffad73d7640 1 -- 172.21.15.70:0/4005892870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad00ac450 con 0x7ffad00cc210 2023-10-08T07:13:16.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.199+0000 7ffad73d7640 1 -- 172.21.15.70:0/4005892870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad004e160 con 0x7ffad00cc210 2023-10-08T07:13:16.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.200+0000 7ffac6ffd640 1 -- 172.21.15.70:0/4005892870 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffac001c460 con 0x7ffad00cc210 2023-10-08T07:13:16.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.201+0000 7ffac6ffd640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffab406af90 0x7ffab406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:16.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.201+0000 7ffac6ffd640 1 -- 172.21.15.70:0/4005892870 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6095+0+0 (secure 0 0 0) 0x7ffac00899d0 con 0x7ffad00cc210 2023-10-08T07:13:16.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.202+0000 7ffad63d5640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffab406af90 0x7ffab406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:16.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.203+0000 7ffad63d5640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffab406af90 0x7ffab406d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006fd0 tx=0x7ffacc008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:16.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.205+0000 7ffac6ffd640 1 -- 172.21.15.70:0/4005892870 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffac00546c0 con 0x7ffad00cc210 2023-10-08T07:13:16.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:16.469+0000 7ffad73d7640 1 -- 172.21.15.70:0/4005892870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7ffad00c4fa0 con 0x7ffad00cc210 2023-10-08T07:13:17.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.384+0000 7ffac6ffd640 1 -- 172.21.15.70:0/4005892870 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3190) v1 ==== 163+0+0 (secure 0 0 0) 0x7ffac0057ce0 con 0x7ffad00cc210 2023-10-08T07:13:17.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 -- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffab406af90 msgr2=0x7ffab406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:17.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffab406af90 0x7ffab406d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffacc006fd0 tx=0x7ffacc008040 comp rx=0 tx=0).stop 2023-10-08T07:13:17.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 -- 172.21.15.70:0/4005892870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00cc210 msgr2=0x7ffad0166e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:17.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00cc210 0x7ffad0166e20 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c400 tx=0x7ffac000c8c0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 -- 172.21.15.70:0/4005892870 shutdown_connections 2023-10-08T07:13:17.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffad01599d0 0x7ffad015bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffab406af90 0x7ffab406d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffad00cc210 0x7ffad0166e20 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 --2- 172.21.15.70:0/4005892870 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffad00b07d0 0x7ffad00acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 -- 172.21.15.70:0/4005892870 >> 172.21.15.70:0/4005892870 conn(0x7ffad00cb0c0 msgr2=0x7ffad00b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:17.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 -- 172.21.15.70:0/4005892870 shutdown_connections 2023-10-08T07:13:17.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.388+0000 7ffad73d7640 1 -- 172.21.15.70:0/4005892870 wait complete. 2023-10-08T07:13:17.413 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-10-08T07:13:17.413 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command succes with --fs argument for non-default fs 2023-10-08T07:13:17.413 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-10-08T07:13:17.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:17.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.631+0000 7f780359e640 1 Processor -- start 2023-10-08T07:13:17.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.631+0000 7f780359e640 1 -- start start 2023-10-08T07:13:17.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.633+0000 7f780359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f780415ab30 0x7f780415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:17.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.633+0000 7f780359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f78040b07d0 0x7f780406cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:17.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.633+0000 7f780359e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f780415f7d0 con 0x7f78040adc70 2023-10-08T07:13:17.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.633+0000 7f780359e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780415f210 con 0x7f78040b07d0 2023-10-08T07:13:17.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.633+0000 7f780359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780415f4f0 con 0x7f780415ab30 2023-10-08T07:13:17.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.633+0000 7f7802d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f78040b07d0 0x7f780406cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:17.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.633+0000 7f7802d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f78040b07d0 0x7f780406cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43138/0 (socket says 172.21.15.70:43138) 2023-10-08T07:13:17.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.633+0000 7f7802d9d640 1 -- 172.21.15.70:0/3311791277 learned_addr learned my addr 172.21.15.70:0/3311791277 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:17.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.633+0000 7f780259c640 1 --2- 172.21.15.70:0/3311791277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f780415ab30 0x7f780415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:17.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.634+0000 7f7802d9d640 1 -- 172.21.15.70:0/3311791277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f780415ab30 msgr2=0x7f780415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:17.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.634+0000 7f7802d9d640 1 --2- 172.21.15.70:0/3311791277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f780415ab30 0x7f780415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.634+0000 7f7802d9d640 1 -- 172.21.15.70:0/3311791277 >> v1:172.21.15.181:6789/0 conn(0x7f78040adc70 legacy=0x7f78040b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:17.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.634+0000 7f7802d9d640 1 -- 172.21.15.70:0/3311791277 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780406c550 con 0x7f78040b07d0 2023-10-08T07:13:17.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.635+0000 7f7802d9d640 1 --2- 172.21.15.70:0/3311791277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78040b07d0 0x7f780406cd50 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009870 tx=0x7f77fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d57f1f2dc7aefc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:17.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.635+0000 7f780159a640 1 -- 172.21.15.70:0/3311791277 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc031640 con 0x7f78040b07d0 2023-10-08T07:13:17.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.635+0000 7f780159a640 1 -- 172.21.15.70:0/3311791277 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc0317a0 con 0x7f78040b07d0 2023-10-08T07:13:17.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.635+0000 7f780159a640 1 -- 172.21.15.70:0/3311791277 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc031b30 con 0x7f78040b07d0 2023-10-08T07:13:17.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.636+0000 7f780359e640 1 -- 172.21.15.70:0/3311791277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78040b07d0 msgr2=0x7f780406cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:17.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.636+0000 7f780359e640 1 --2- 172.21.15.70:0/3311791277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78040b07d0 0x7f780406cd50 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009870 tx=0x7f77fc02b8f0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.636+0000 7f780359e640 1 -- 172.21.15.70:0/3311791277 shutdown_connections 2023-10-08T07:13:17.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.636+0000 7f780359e640 1 --2- 172.21.15.70:0/3311791277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78040b07d0 0x7f780406cd50 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.636+0000 7f780359e640 1 --2- 172.21.15.70:0/3311791277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f780415ab30 0x7f780415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.636+0000 7f780359e640 1 -- 172.21.15.70:0/3311791277 >> 172.21.15.70:0/3311791277 conn(0x7f78040cb0c0 msgr2=0x7f78040cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:17.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.636+0000 7f780359e640 1 -- 172.21.15.70:0/3311791277 shutdown_connections 2023-10-08T07:13:17.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.636+0000 7f780359e640 1 -- 172.21.15.70:0/3311791277 wait complete. 2023-10-08T07:13:17.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.638+0000 7f780359e640 1 Processor -- start 2023-10-08T07:13:17.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.638+0000 7f780359e640 1 -- start start 2023-10-08T07:13:17.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f780359e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78040adc70 0x7f78040c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:17.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f780359e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78040b07d0 0x7f7804155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:17.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f780359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780415ab30 0x7f78041576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:17.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f780359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78041f09a0 con 0x7f780415ab30 2023-10-08T07:13:17.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f780359e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78041f0c80 con 0x7f78040adc70 2023-10-08T07:13:17.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f780359e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78041f0f60 con 0x7f78040b07d0 2023-10-08T07:13:17.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f7802d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780415ab30 0x7f78041576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:17.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f7802d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780415ab30 0x7f78041576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43154/0 (socket says 172.21.15.70:43154) 2023-10-08T07:13:17.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f7802d9d640 1 -- 172.21.15.70:0/1215917499 learned_addr learned my addr 172.21.15.70:0/1215917499 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:17.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f780259c640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78040adc70 0x7f78040c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:17.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f7801d9b640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78040b07d0 0x7f7804155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:17.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f7802d9d640 1 -- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78040b07d0 msgr2=0x7f7804155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:17.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f7802d9d640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78040b07d0 0x7f7804155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f7802d9d640 1 -- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78040adc70 msgr2=0x7f78040c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:17.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f7802d9d640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78040adc70 0x7f78040c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.640+0000 7f7802d9d640 1 -- 172.21.15.70:0/1215917499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc0094d0 con 0x7f780415ab30 2023-10-08T07:13:17.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.641+0000 7f7802d9d640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780415ab30 0x7f78041576d0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f77fc02b930 tx=0x7f77fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:17.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.641+0000 7f77f2ffd640 1 -- 172.21.15.70:0/1215917499 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc03ac60 con 0x7f780415ab30 2023-10-08T07:13:17.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.641+0000 7f77f2ffd640 1 -- 172.21.15.70:0/1215917499 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc03adc0 con 0x7f780415ab30 2023-10-08T07:13:17.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.641+0000 7f77f2ffd640 1 -- 172.21.15.70:0/1215917499 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc044630 con 0x7f780415ab30 2023-10-08T07:13:17.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.641+0000 7f780359e640 1 -- 172.21.15.70:0/1215917499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78041f1240 con 0x7f780415ab30 2023-10-08T07:13:17.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.641+0000 7f780359e640 1 -- 172.21.15.70:0/1215917499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78041f1520 con 0x7f780415ab30 2023-10-08T07:13:17.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.643+0000 7f780359e640 1 -- 172.21.15.70:0/1215917499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78040c3350 con 0x7f780415ab30 2023-10-08T07:13:17.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.643+0000 7f77f2ffd640 1 -- 172.21.15.70:0/1215917499 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f77fc02f050 con 0x7f780415ab30 2023-10-08T07:13:17.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.644+0000 7f77f2ffd640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f77e406af90 0x7f77e406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:17.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.645+0000 7f77f2ffd640 1 -- 172.21.15.70:0/1215917499 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6095+0+0 (secure 0 0 0) 0x7f77fc0ab910 con 0x7f780415ab30 2023-10-08T07:13:17.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.645+0000 7f780259c640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f77e406af90 0x7f77e406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:17.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.646+0000 7f780259c640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f77e406af90 0x7f77e406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77f40099c0 tx=0x7f77f4003990 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.650+0000 7f77f2ffd640 1 -- 172.21.15.70:0/1215917499 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78040c3350 con 0x7f780415ab30 2023-10-08T07:13:17.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.840+0000 7f780359e640 1 -- 172.21.15.70:0/1215917499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78040c3350 con 0x7f780415ab30 2023-10-08T07:13:17.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.842+0000 7f77f2ffd640 1 -- 172.21.15.70:0/1215917499 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v225) v1 ==== 74+0+14434 (secure 0 0 0) 0x7f78040c3350 con 0x7f780415ab30 2023-10-08T07:13:17.844 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:17.844 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":225,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:14.251664+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6835/2355440757":"2023-10-09T07:13:11.222728+0000","172.21.15.181:0/1545365181":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/4119540053":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/2433436977":"2023-10-09T07:13:07.464344+0000","172.21.15.70:6834/2355440757":"2023-10-09T07:13:11.222728+0000","172.21.15.181:0/2511790348":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/294214378":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/3667065295":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/1205099199":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/3162325425":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6833/2638163552":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6832/2638163552":"2023-10-09T07:10:58.332667+0000","172.21.15.181:0/785931869":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6833/2613103103":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/265797938":"2023-10-09T07:10:58.332667+0000","172.21.15.181:6832/2613103103":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/2110654177":"2023-10-09T07:13:07.464344+0000","172.21.15.181:0/39996703":"2023-10-09T07:10:58.332667+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":26,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.845+0000 7f780359e640 1 -- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f77e406af90 msgr2=0x7f77e406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:17.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.845+0000 7f780359e640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f77e406af90 0x7f77e406d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f77f40099c0 tx=0x7f77f4003990 comp rx=0 tx=0).stop 2023-10-08T07:13:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.845+0000 7f780359e640 1 -- 172.21.15.70:0/1215917499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780415ab30 msgr2=0x7f78041576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.845+0000 7f780359e640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780415ab30 0x7f78041576d0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7f77fc02b930 tx=0x7f77fc0046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.845+0000 7f780359e640 1 -- 172.21.15.70:0/1215917499 shutdown_connections 2023-10-08T07:13:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.845+0000 7f780359e640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f77e406af90 0x7f77e406d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.845+0000 7f780359e640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f780415ab30 0x7f78041576d0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.845+0000 7f780359e640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f78040b07d0 0x7f7804155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.845+0000 7f780359e640 1 --2- 172.21.15.70:0/1215917499 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f78040adc70 0x7f78040c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.846+0000 7f780359e640 1 -- 172.21.15.70:0/1215917499 >> 172.21.15.70:0/1215917499 conn(0x7f78040cb0c0 msgr2=0x7f78040cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.846+0000 7f780359e640 1 -- 172.21.15.70:0/1215917499 shutdown_connections 2023-10-08T07:13:17.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:17.846+0000 7f780359e640 1 -- 172.21.15.70:0/1215917499 wait complete. 2023-10-08T07:13:17.867 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:13:17.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:13:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.086+0000 7f75157f5640 1 Processor -- start 2023-10-08T07:13:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.086+0000 7f75157f5640 1 -- start start 2023-10-08T07:13:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.088+0000 7f75157f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f751015aad0 0x7f75100b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.088+0000 7f75157f5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751015bf30 con 0x7f75100c82b0 2023-10-08T07:13:18.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.088+0000 7f75157f5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751015b320 con 0x7f751006d410 2023-10-08T07:13:18.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.088+0000 7f75157f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751015b600 con 0x7f751015aad0 2023-10-08T07:13:18.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.088+0000 7f750e7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f75100c82b0 0x7f751006cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41778/0 (socket says 172.21.15.70:41778) 2023-10-08T07:13:18.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.088+0000 7f750e7fc640 1 -- 172.21.15.70:0/2856324115 learned_addr learned my addr 172.21.15.70:0/2856324115 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750effd640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f751015aad0 0x7f75100b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40658004 0 0) 0x7f751015bf30 con 0x7f75100c82b0 2023-10-08T07:13:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74f8003520 con 0x7f75100c82b0 2023-10-08T07:13:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3464834705 0 0) 0x7f74f8003520 con 0x7f75100c82b0 2023-10-08T07:13:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f751015aad0 msgr2=0x7f75100b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f751015aad0 0x7f75100b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 >> v1:172.21.15.181:6789/0 conn(0x7f751006d410 legacy=0x7f7510070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751015b8e0 con 0x7f75100c82b0 2023-10-08T07:13:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7504001670 con 0x7f75100c82b0 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 >> v1:172.21.15.70:6789/0 conn(0x7f75100c82b0 legacy=0x7f751006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f80039a0 0x7f74f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 0x7f74f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f751015aad0 0x7f74f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751015b600 con 0x7f74f80047c0 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f800ba30 con 0x7f74f80039a0 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f800bbf0 con 0x7f751015aad0 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.089+0000 7f750e7fc640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 0x7f74f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.090+0000 7f750effd640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f80039a0 0x7f74f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.090+0000 7f750e7fc640 1 -- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f751015aad0 msgr2=0x7f74f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.090+0000 7f750e7fc640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f751015aad0 0x7f74f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.090+0000 7f750e7fc640 1 -- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f80039a0 msgr2=0x7f74f8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.090+0000 7f750e7fc640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f80039a0 0x7f74f8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.090+0000 7f750e7fc640 1 -- 172.21.15.70:0/2856324115 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f8003520 con 0x7f74f80047c0 2023-10-08T07:13:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.090+0000 7f750effd640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74f80039a0 0x7f74f8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.091+0000 7f750e7fc640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 0x7f74f8004c40 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f7504007fd0 tx=0x7f74f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b758e9477f9335 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.091+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504005ad0 con 0x7f74f80047c0 2023-10-08T07:13:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.091+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7504005c30 con 0x7f74f80047c0 2023-10-08T07:13:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.091+0000 7f750dffb640 1 -- 172.21.15.70:0/2856324115 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750402c850 con 0x7f74f80047c0 2023-10-08T07:13:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.092+0000 7f75157f5640 1 -- 172.21.15.70:0/2856324115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 msgr2=0x7f74f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.092+0000 7f75157f5640 1 --2- 172.21.15.70:0/2856324115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 0x7f74f8004c40 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f7504007fd0 tx=0x7f74f80057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.092+0000 7f750f7fe640 1 -- 172.21.15.70:0/2856324115 reap_dead start 2023-10-08T07:13:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.092+0000 7f75157f5640 1 -- 172.21.15.70:0/2856324115 shutdown_connections 2023-10-08T07:13:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.092+0000 7f75157f5640 1 -- 172.21.15.70:0/2856324115 >> 172.21.15.70:0/2856324115 conn(0x7f7510069d50 msgr2=0x7f751006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.092+0000 7f75157f5640 1 -- 172.21.15.70:0/2856324115 shutdown_connections 2023-10-08T07:13:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.092+0000 7f75157f5640 1 -- 172.21.15.70:0/2856324115 wait complete. 2023-10-08T07:13:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.094+0000 7f75157f5640 1 Processor -- start 2023-10-08T07:13:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.094+0000 7f75157f5640 1 -- start start 2023-10-08T07:13:18.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.095+0000 7f75157f5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 0x7f7510143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.095+0000 7f75157f5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7510153d40 0x7f75101541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.095+0000 7f75157f5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75101491d0 0x7f7510149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.095+0000 7f75157f5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7510149b90 con 0x7f74f80047c0 2023-10-08T07:13:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.095+0000 7f75157f5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75101f1fc0 con 0x7f7510153d40 2023-10-08T07:13:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.095+0000 7f75157f5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75101459a0 con 0x7f75101491d0 2023-10-08T07:13:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.095+0000 7f750effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 0x7f7510143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.095+0000 7f750effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 0x7f7510143b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43184/0 (socket says 172.21.15.70:43184) 2023-10-08T07:13:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.095+0000 7f750effd640 1 -- 172.21.15.70:0/350416854 learned_addr learned my addr 172.21.15.70:0/350416854 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f750e7fc640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7510153d40 0x7f75101541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f750f7fe640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75101491d0 0x7f7510149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f750effd640 1 -- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75101491d0 msgr2=0x7f7510149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f750effd640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75101491d0 0x7f7510149650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f750effd640 1 -- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7510153d40 msgr2=0x7f75101541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f750effd640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7510153d40 0x7f75101541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f750effd640 1 -- 172.21.15.70:0/350416854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7510152ca0 con 0x7f74f80047c0 2023-10-08T07:13:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f750e7fc640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7510153d40 0x7f75101541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f750effd640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 0x7f7510143b80 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f7500010890 tx=0x7f7500010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f74f77fe640 1 -- 172.21.15.70:0/350416854 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000eb60 con 0x7f74f80047c0 2023-10-08T07:13:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f74f77fe640 1 -- 172.21.15.70:0/350416854 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750000ecc0 con 0x7f74f80047c0 2023-10-08T07:13:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f75157f5640 1 -- 172.21.15.70:0/350416854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7510152f80 con 0x7f74f80047c0 2023-10-08T07:13:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f74f77fe640 1 -- 172.21.15.70:0/350416854 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750001c400 con 0x7f74f80047c0 2023-10-08T07:13:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.096+0000 7f75157f5640 1 -- 172.21.15.70:0/350416854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7510153540 con 0x7f74f80047c0 2023-10-08T07:13:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.098+0000 7f75157f5640 1 -- 172.21.15.70:0/350416854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751004e160 con 0x7f74f80047c0 2023-10-08T07:13:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.098+0000 7f74f77fe640 1 -- 172.21.15.70:0/350416854 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f750001c560 con 0x7f74f80047c0 2023-10-08T07:13:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.099+0000 7f74f77fe640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f74fc06af90 0x7f74fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.099+0000 7f74f77fe640 1 -- 172.21.15.70:0/350416854 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6095+0+0 (secure 0 0 0) 0x7f7500089e10 con 0x7f74f80047c0 2023-10-08T07:13:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.100+0000 7f750e7fc640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f74fc06af90 0x7f74fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.101+0000 7f750e7fc640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f74fc06af90 0x7f74fc06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7504034000 tx=0x7f7504009030 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:18.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.104+0000 7f74f77fe640 1 -- 172.21.15.70:0/350416854 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7500054ce0 con 0x7f74f80047c0 2023-10-08T07:13:18.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.294+0000 7f75157f5640 1 -- 172.21.15.70:0/350416854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75100cd210 con 0x7f74f80047c0 2023-10-08T07:13:18.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.295+0000 7f74f77fe640 1 -- 172.21.15.70:0/350416854 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v225) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7500058300 con 0x7f74f80047c0 2023-10-08T07:13:18.297 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:13:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 -- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f74fc06af90 msgr2=0x7f74fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f74fc06af90 0x7f74fc06d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7504034000 tx=0x7f7504009030 comp rx=0 tx=0).stop 2023-10-08T07:13:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 -- 172.21.15.70:0/350416854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 msgr2=0x7f7510143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 0x7f7510143b80 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f7500010890 tx=0x7f7500010d50 comp rx=0 tx=0).stop 2023-10-08T07:13:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 -- 172.21.15.70:0/350416854 shutdown_connections 2023-10-08T07:13:18.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f75101491d0 0x7f7510149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7510153d40 0x7f75101541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f74fc06af90 0x7f74fc06d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 --2- 172.21.15.70:0/350416854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74f80047c0 0x7f7510143b80 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 -- 172.21.15.70:0/350416854 >> 172.21.15.70:0/350416854 conn(0x7f7510069d50 msgr2=0x7f75100ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:18.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 -- 172.21.15.70:0/350416854 shutdown_connections 2023-10-08T07:13:18.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.298+0000 7f75157f5640 1 -- 172.21.15.70:0/350416854 wait complete. 2023-10-08T07:13:18.318 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-10-08T07:13:18.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.532+0000 7f95a13a5640 1 Processor -- start 2023-10-08T07:13:18.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.532+0000 7f95a13a5640 1 -- start start 2023-10-08T07:13:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.534+0000 7f95a13a5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f959c0ada10 0x7f959c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.534+0000 7f95a13a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0ca620 0x7f959c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.534+0000 7f95a13a5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959c0cade0 con 0x7f959c0cb450 2023-10-08T07:13:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.534+0000 7f95a13a5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f959c0addf0 con 0x7f959c0ada10 2023-10-08T07:13:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.534+0000 7f95a13a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959c06c270 con 0x7f959c0ca620 2023-10-08T07:13:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.534+0000 7f95a0ba4640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f959c0cb450 0x7f959c0cb860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41792/0 (socket says 172.21.15.70:41792) 2023-10-08T07:13:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.534+0000 7f95a0ba4640 1 -- 172.21.15.70:0/4006658837 learned_addr learned my addr 172.21.15.70:0/4006658837 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741344785 0 0) 0x7f959c0cade0 con 0x7f959c0cb450 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959bfff640 1 --2- 172.21.15.70:0/4006658837 >> v2:172.21.15.181:3300/0 conn(0x7f959c0ada10 0x7f959c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9588003520 con 0x7f959c0cb450 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f95937fe640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0ca620 0x7f959c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1737241928 0 0) 0x7f9588003520 con 0x7f959c0cb450 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0ca620 msgr2=0x7f959c0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0ca620 0x7f959c0c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 >> v2:172.21.15.181:3300/0 conn(0x7f959c0ada10 msgr2=0x7f959c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 --2- 172.21.15.70:0/4006658837 >> v2:172.21.15.181:3300/0 conn(0x7f959c0ada10 0x7f959c0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c06c550 con 0x7f959c0cb450 2023-10-08T07:13:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f95940019f0 con 0x7f959c0cb450 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 >> v1:172.21.15.70:6789/0 conn(0x7f959c0cb450 legacy=0x7f959c0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0ca620 0x7f9588003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9588003f40 0x7f95880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9588004d00 0x7f9588005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0addf0 con 0x7f9588004d00 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c06c270 con 0x7f9588003f40 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.535+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958800ba30 con 0x7f959c0ca620 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.536+0000 7f95a0ba4640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9588004d00 0x7f9588005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.536+0000 7f959bfff640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0ca620 0x7f9588003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.536+0000 7f95937fe640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9588003f40 0x7f95880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.536+0000 7f95a0ba4640 1 -- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0ca620 msgr2=0x7f9588003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.536+0000 7f95a0ba4640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f959c0ca620 0x7f9588003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.536+0000 7f95a0ba4640 1 -- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9588003f40 msgr2=0x7f95880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.536+0000 7f95a0ba4640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9588003f40 0x7f95880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.536+0000 7f95a0ba4640 1 -- 172.21.15.70:0/4006658837 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588003520 con 0x7f9588004d00 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.536+0000 7f95937fe640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9588003f40 0x7f95880047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.537+0000 7f95a0ba4640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9588004d00 0x7f9588005180 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f959400a790 tx=0x7f95880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae11c31b68182be server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.537+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9594034630 con 0x7f9588004d00 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.537+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959402f070 con 0x7f9588004d00 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.537+0000 7f959b7fe640 1 -- 172.21.15.70:0/4006658837 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9594034a50 con 0x7f9588004d00 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.538+0000 7f95a13a5640 1 -- 172.21.15.70:0/4006658837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9588004d00 msgr2=0x7f9588005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.538+0000 7f95a13a5640 1 --2- 172.21.15.70:0/4006658837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9588004d00 0x7f9588005180 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f959400a790 tx=0x7f95880057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.538+0000 7f95a0ba4640 1 -- 172.21.15.70:0/4006658837 reap_dead start 2023-10-08T07:13:18.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.538+0000 7f95a13a5640 1 -- 172.21.15.70:0/4006658837 shutdown_connections 2023-10-08T07:13:18.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.538+0000 7f95a13a5640 1 -- 172.21.15.70:0/4006658837 >> 172.21.15.70:0/4006658837 conn(0x7f959c0ce9d0 msgr2=0x7f959c0cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:18.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.538+0000 7f95a13a5640 1 -- 172.21.15.70:0/4006658837 shutdown_connections 2023-10-08T07:13:18.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.538+0000 7f95a13a5640 1 -- 172.21.15.70:0/4006658837 wait complete. 2023-10-08T07:13:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.540+0000 7f95a13a5640 1 Processor -- start 2023-10-08T07:13:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.540+0000 7f95a13a5640 1 -- start start 2023-10-08T07:13:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.541+0000 7f95a13a5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9588004d00 0x7f959c09be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.541+0000 7f95a13a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959c071170 0x7f959c1f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.541+0000 7f95a13a5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f959c159d30 0x7f959c0b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.541+0000 7f95a13a5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c1f3200 con 0x7f959c071170 2023-10-08T07:13:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.541+0000 7f95a13a5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c070040 con 0x7f959c159d30 2023-10-08T07:13:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.541+0000 7f95a13a5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959c15b850 con 0x7f9588004d00 2023-10-08T07:13:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.541+0000 7f95937fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959c071170 0x7f959c1f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.541+0000 7f95937fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959c071170 0x7f959c1f2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43198/0 (socket says 172.21.15.70:43198) 2023-10-08T07:13:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.541+0000 7f95937fe640 1 -- 172.21.15.70:0/3080652986 learned_addr learned my addr 172.21.15.70:0/3080652986 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f959bfff640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9588004d00 0x7f959c09be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95937fe640 1 -- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9588004d00 msgr2=0x7f959c09be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95937fe640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9588004d00 0x7f959c09be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95937fe640 1 -- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f959c159d30 msgr2=0x7f959c0b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95a0ba4640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f959c159d30 0x7f959c0b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95937fe640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f959c159d30 0x7f959c0b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95937fe640 1 -- 172.21.15.70:0/3080652986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c071650 con 0x7f959c071170 2023-10-08T07:13:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95937fe640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959c071170 0x7f959c1f2cc0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f9584010890 tx=0x7f9584010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95997fa640 1 -- 172.21.15.70:0/3080652986 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958400eb00 con 0x7f959c071170 2023-10-08T07:13:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95997fa640 1 -- 172.21.15.70:0/3080652986 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958400ec60 con 0x7f959c071170 2023-10-08T07:13:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95997fa640 1 -- 172.21.15.70:0/3080652986 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958401c400 con 0x7f959c071170 2023-10-08T07:13:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95a13a5640 1 -- 172.21.15.70:0/3080652986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c071930 con 0x7f959c071170 2023-10-08T07:13:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.542+0000 7f95a13a5640 1 -- 172.21.15.70:0/3080652986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c071ef0 con 0x7f959c071170 2023-10-08T07:13:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.544+0000 7f95a13a5640 1 -- 172.21.15.70:0/3080652986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c04e160 con 0x7f959c071170 2023-10-08T07:13:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.544+0000 7f95997fa640 1 -- 172.21.15.70:0/3080652986 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f958401c560 con 0x7f959c071170 2023-10-08T07:13:18.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.545+0000 7f95997fa640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f957c06afc0 0x7f957c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:18.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.545+0000 7f95997fa640 1 -- 172.21.15.70:0/3080652986 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6095+0+0 (secure 0 0 0) 0x7f9584089d60 con 0x7f959c071170 2023-10-08T07:13:18.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.546+0000 7f959bfff640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f957c06afc0 0x7f957c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:18.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.547+0000 7f959bfff640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f957c06afc0 0x7f957c06d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f959c066870 tx=0x7f958c008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:18.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.550+0000 7f95997fa640 1 -- 172.21.15.70:0/3080652986 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9584054c30 con 0x7f959c071170 2023-10-08T07:13:18.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:18.809+0000 7f95a13a5640 1 -- 172.21.15.70:0/3080652986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f959c158270 con 0x7f959c071170 2023-10-08T07:13:19.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.398+0000 7f95997fa640 1 -- 172.21.15.70:0/3080652986 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3192) v1 ==== 189+0+0 (secure 0 0 0) 0x7f9584021100 con 0x7f959c071170 2023-10-08T07:13:19.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 -- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f957c06afc0 msgr2=0x7f957c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:19.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f957c06afc0 0x7f957c06d480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f959c066870 tx=0x7f958c008040 comp rx=0 tx=0).stop 2023-10-08T07:13:19.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 -- 172.21.15.70:0/3080652986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959c071170 msgr2=0x7f959c1f2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:19.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959c071170 0x7f959c1f2cc0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f9584010890 tx=0x7f9584010d50 comp rx=0 tx=0).stop 2023-10-08T07:13:19.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 -- 172.21.15.70:0/3080652986 shutdown_connections 2023-10-08T07:13:19.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f959c159d30 0x7f959c0b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f957c06afc0 0x7f957c06d480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f959c071170 0x7f959c1f2cc0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 --2- 172.21.15.70:0/3080652986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9588004d00 0x7f959c09be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 -- 172.21.15.70:0/3080652986 >> 172.21.15.70:0/3080652986 conn(0x7f959c0ce9d0 msgr2=0x7f959c1594a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:19.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 -- 172.21.15.70:0/3080652986 shutdown_connections 2023-10-08T07:13:19.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.402+0000 7f95a13a5640 1 -- 172.21.15.70:0/3080652986 wait complete. 2023-10-08T07:13:19.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T07:13:19.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.636+0000 7ffaf48f2640 1 Processor -- start 2023-10-08T07:13:19.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.636+0000 7ffaf48f2640 1 -- start start 2023-10-08T07:13:19.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.638+0000 7ffaf48f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffaf006c3a0 0x7ffaf00c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:19.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.638+0000 7ffaf48f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf015aa00 0x7ffaf00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:19.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.638+0000 7ffaf48f2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf015b310 con 0x7ffaf006db60 2023-10-08T07:13:19.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.638+0000 7ffaf48f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf00ca230 con 0x7ffaf006c3a0 2023-10-08T07:13:19.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.638+0000 7ffaf48f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf006d290 con 0x7ffaf015aa00 2023-10-08T07:13:19.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.638+0000 7ffae7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ffaf006db60 0x7ffaf006c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41794/0 (socket says 172.21.15.70:41794) 2023-10-08T07:13:19.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.638+0000 7ffae7fff640 1 -- 172.21.15.70:0/810342690 learned_addr learned my addr 172.21.15.70:0/810342690 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:19.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaee59c640 1 --2- 172.21.15.70:0/810342690 >> v2:172.21.15.181:3300/0 conn(0x7ffaf006c3a0 0x7ffaf00c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:19.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaeed9d640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf015aa00 0x7ffaf00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:19.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654254251 0 0) 0x7ffad8001110 con 0x7ffaf006db60 2023-10-08T07:13:19.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffadc003520 con 0x7ffaf006db60 2023-10-08T07:13:19.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988898993 0 0) 0x7ffadc003520 con 0x7ffaf006db60 2023-10-08T07:13:19.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf015aa00 msgr2=0x7ffaf00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:19.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaedd9b640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf015aa00 0x7ffaf00b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 >> v2:172.21.15.181:3300/0 conn(0x7ffaf006c3a0 msgr2=0x7ffaf00c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:19.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaedd9b640 1 --2- 172.21.15.70:0/810342690 >> v2:172.21.15.181:3300/0 conn(0x7ffaf006c3a0 0x7ffaf00c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf00c9860 con 0x7ffaf006db60 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ffad8001800 con 0x7ffaf006db60 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.639+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 >> v1:172.21.15.70:6789/0 conn(0x7ffaf006db60 legacy=0x7ffaf006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffaedd9b640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf015aa00 0x7ffadc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffaedd9b640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 0x7ffadc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffaedd9b640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffadc004d00 0x7ffadc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf006d290 con 0x7ffadc003ef0 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc00b980 con 0x7ffadc004d00 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffadc00bac0 con 0x7ffaf015aa00 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffae7fff640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 0x7ffadc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffae7fff640 1 -- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf015aa00 msgr2=0x7ffadc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:19.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffaee59c640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf015aa00 0x7ffadc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:19.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffae7fff640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf015aa00 0x7ffadc003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffaeed9d640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffadc004d00 0x7ffadc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:19.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffae7fff640 1 -- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffadc004d00 msgr2=0x7ffadc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:19.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffae7fff640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffadc004d00 0x7ffadc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.640+0000 7ffae7fff640 1 -- 172.21.15.70:0/810342690 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc003520 con 0x7ffadc003ef0 2023-10-08T07:13:19.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.641+0000 7ffae7fff640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 0x7ffadc0047c0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c240 tx=0x7ffadc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b89314d0a0551b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:19.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.642+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8005ad0 con 0x7ffadc003ef0 2023-10-08T07:13:19.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.642+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8005c30 con 0x7ffadc003ef0 2023-10-08T07:13:19.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.642+0000 7ffaedd9b640 1 -- 172.21.15.70:0/810342690 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad802cae0 con 0x7ffadc003ef0 2023-10-08T07:13:19.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.642+0000 7ffaf48f2640 1 -- 172.21.15.70:0/810342690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 msgr2=0x7ffadc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.642+0000 7ffaf48f2640 1 --2- 172.21.15.70:0/810342690 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 0x7ffadc0047c0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c240 tx=0x7ffadc0057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.642+0000 7ffaeed9d640 1 -- 172.21.15.70:0/810342690 reap_dead start 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.642+0000 7ffaf48f2640 1 -- 172.21.15.70:0/810342690 shutdown_connections 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.642+0000 7ffaf48f2640 1 -- 172.21.15.70:0/810342690 >> 172.21.15.70:0/810342690 conn(0x7ffaf006a5d0 msgr2=0x7ffaf00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.642+0000 7ffaf48f2640 1 -- 172.21.15.70:0/810342690 shutdown_connections 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.643+0000 7ffaf48f2640 1 -- 172.21.15.70:0/810342690 wait complete. 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.644+0000 7ffaf48f2640 1 Processor -- start 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.645+0000 7ffaf48f2640 1 -- start start 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.645+0000 7ffaf48f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 0x7ffaf0156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.645+0000 7ffaf48f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf0166a70 0x7ffaf00b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaf48f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf01574f0 0x7ffaf00c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:19.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaf48f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00c6a00 con 0x7ffadc003ef0 2023-10-08T07:13:19.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaf48f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00c6ce0 con 0x7ffaf0166a70 2023-10-08T07:13:19.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaf48f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00c6fc0 con 0x7ffaf01574f0 2023-10-08T07:13:19.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaee59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 0x7ffaf0156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:19.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaee59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 0x7ffaf0156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43222/0 (socket says 172.21.15.70:43222) 2023-10-08T07:13:19.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaee59c640 1 -- 172.21.15.70:0/3934636722 learned_addr learned my addr 172.21.15.70:0/3934636722 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:19.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaee59c640 1 -- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf01574f0 msgr2=0x7ffaf00c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:19.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaee59c640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf01574f0 0x7ffaf00c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaee59c640 1 -- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf0166a70 msgr2=0x7ffaf00b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaee59c640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf0166a70 0x7ffaf00b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaee59c640 1 -- 172.21.15.70:0/3934636722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf00c72a0 con 0x7ffadc003ef0 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.646+0000 7ffaee59c640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 0x7ffaf0156f80 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7ffae0010860 tx=0x7ffae0010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.647+0000 7ffae6ffd640 1 -- 172.21.15.70:0/3934636722 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0009610 con 0x7ffadc003ef0 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.647+0000 7ffae6ffd640 1 -- 172.21.15.70:0/3934636722 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae000eae0 con 0x7ffadc003ef0 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.647+0000 7ffae6ffd640 1 -- 172.21.15.70:0/3934636722 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000ee70 con 0x7ffadc003ef0 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.647+0000 7ffaf48f2640 1 -- 172.21.15.70:0/3934636722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf00c7580 con 0x7ffadc003ef0 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.647+0000 7ffaf48f2640 1 -- 172.21.15.70:0/3934636722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf00c4530 con 0x7ffadc003ef0 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.648+0000 7ffaf48f2640 1 -- 172.21.15.70:0/3934636722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf004e160 con 0x7ffadc003ef0 2023-10-08T07:13:19.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.649+0000 7ffae6ffd640 1 -- 172.21.15.70:0/3934636722 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ffae001c400 con 0x7ffadc003ef0 2023-10-08T07:13:19.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.650+0000 7ffae6ffd640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffad006af90 0x7ffad006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:19.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.650+0000 7ffae6ffd640 1 -- 172.21.15.70:0/3934636722 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6095+0+0 (secure 0 0 0) 0x7ffae0089e70 con 0x7ffadc003ef0 2023-10-08T07:13:19.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.651+0000 7ffae7fff640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffad006af90 0x7ffad006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:19.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.652+0000 7ffae7fff640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffad006af90 0x7ffad006d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffad8034000 tx=0x7ffad802e000 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.655+0000 7ffae6ffd640 1 -- 172.21.15.70:0/3934636722 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffae0054d30 con 0x7ffadc003ef0 2023-10-08T07:13:19.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.843+0000 7ffaf48f2640 1 -- 172.21.15.70:0/3934636722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ffaf00cb100 con 0x7ffadc003ef0 2023-10-08T07:13:19.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.850+0000 7ffae6ffd640 1 -- 172.21.15.70:0/3934636722 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v76)=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffae0058350 con 0x7ffadc003ef0 2023-10-08T07:13:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 -- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffad006af90 msgr2=0x7ffad006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffad006af90 0x7ffad006d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffad8034000 tx=0x7ffad802e000 comp rx=0 tx=0).stop 2023-10-08T07:13:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 -- 172.21.15.70:0/3934636722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 msgr2=0x7ffaf0156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 0x7ffaf0156f80 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7ffae0010860 tx=0x7ffae0010d20 comp rx=0 tx=0).stop 2023-10-08T07:13:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 -- 172.21.15.70:0/3934636722 shutdown_connections 2023-10-08T07:13:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffaf01574f0 0x7ffaf00c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffaf0166a70 0x7ffaf00b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ffad006af90 0x7ffad006d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 --2- 172.21.15.70:0/3934636722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffadc003ef0 0x7ffaf0156f80 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:19.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 -- 172.21.15.70:0/3934636722 >> 172.21.15.70:0/3934636722 conn(0x7ffaf006a5d0 msgr2=0x7ffaf00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:19.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 -- 172.21.15.70:0/3934636722 shutdown_connections 2023-10-08T07:13:19.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:19.853+0000 7ffaf48f2640 1 -- 172.21.15.70:0/3934636722 wait complete. 2023-10-08T07:13:19.873 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T07:13:19.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:13:19.988 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:13:20.107 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T07:13:20.156 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:13:20.157 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:13:20.219 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:13:20.572 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:13:20.572 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:13:20.631 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:13:20.979 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:13:20.979 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:13:21.039 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:13:21.183 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:13:21.183 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:13:21.245 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:13:21.391 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T07:13:21.391 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:13:21.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.615+0000 7f861e8b4640 1 Processor -- start 2023-10-08T07:13:21.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.615+0000 7f861e8b4640 1 -- start start 2023-10-08T07:13:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.617+0000 7f861e8b4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f861815ab40 0x7f86180ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:22.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.617+0000 7f861e8b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86180c90b0 0x7f86180adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:22.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.617+0000 7f861e8b4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861815b2b0 con 0x7f86180cae70 2023-10-08T07:13:22.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.617+0000 7f861e8b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f861815bf30 con 0x7f86180c90b0 2023-10-08T07:13:22.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.617+0000 7f861e8b4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f861815f7a0 con 0x7f861815ab40 2023-10-08T07:13:22.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.617+0000 7f860ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86180c90b0 0x7f86180adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.618+0000 7f860ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f86180c90b0 0x7f86180adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43230/0 (socket says 172.21.15.70:43230) 2023-10-08T07:13:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.618+0000 7f860ffff640 1 -- 172.21.15.70:0/3575512389 learned_addr learned my addr 172.21.15.70:0/3575512389 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.618+0000 7f8617fff640 1 --2- 172.21.15.70:0/3575512389 >> v2:172.21.15.181:3300/0 conn(0x7f861815ab40 0x7f86180ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.618+0000 7f860ffff640 1 -- 172.21.15.70:0/3575512389 >> v2:172.21.15.181:3300/0 conn(0x7f861815ab40 msgr2=0x7f86180ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.618+0000 7f860ffff640 1 --2- 172.21.15.70:0/3575512389 >> v2:172.21.15.181:3300/0 conn(0x7f861815ab40 0x7f86180ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.618+0000 7f860ffff640 1 -- 172.21.15.70:0/3575512389 >> v1:172.21.15.181:6789/0 conn(0x7f86180cae70 legacy=0x7f86180702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.618+0000 7f860ffff640 1 -- 172.21.15.70:0/3575512389 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861815b590 con 0x7f86180c90b0 2023-10-08T07:13:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.619+0000 7f860ffff640 1 --2- 172.21.15.70:0/3575512389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180c90b0 0x7f86180adc60 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f861806e440 tx=0x7f860002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3dd66c389d04b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:22.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f86177fe640 1 -- 172.21.15.70:0/3575512389 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600031870 con 0x7f86180c90b0 2023-10-08T07:13:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f86177fe640 1 -- 172.21.15.70:0/3575512389 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86000319d0 con 0x7f86180c90b0 2023-10-08T07:13:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f86177fe640 1 -- 172.21.15.70:0/3575512389 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600031d60 con 0x7f86180c90b0 2023-10-08T07:13:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f861e8b4640 1 -- 172.21.15.70:0/3575512389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180c90b0 msgr2=0x7f86180adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f861e8b4640 1 --2- 172.21.15.70:0/3575512389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180c90b0 0x7f86180adc60 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f861806e440 tx=0x7f860002b910 comp rx=0 tx=0).stop 2023-10-08T07:13:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f861e8b4640 1 -- 172.21.15.70:0/3575512389 shutdown_connections 2023-10-08T07:13:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f861e8b4640 1 --2- 172.21.15.70:0/3575512389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180c90b0 0x7f86180adc60 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f861e8b4640 1 --2- 172.21.15.70:0/3575512389 >> v2:172.21.15.181:3300/0 conn(0x7f861815ab40 0x7f86180ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f861e8b4640 1 -- 172.21.15.70:0/3575512389 >> 172.21.15.70:0/3575512389 conn(0x7f861806a380 msgr2=0x7f86180b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:22.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f861e8b4640 1 -- 172.21.15.70:0/3575512389 shutdown_connections 2023-10-08T07:13:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.620+0000 7f861e8b4640 1 -- 172.21.15.70:0/3575512389 wait complete. 2023-10-08T07:13:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.622+0000 7f861e8b4640 1 Processor -- start 2023-10-08T07:13:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.623+0000 7f861e8b4640 1 -- start start 2023-10-08T07:13:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f861e8b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180c90b0 0x7f861814d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f861e8b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180cae70 0x7f861815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:22.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f861e8b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f861815ab40 0x7f86181f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f861e8b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86181f68f0 con 0x7f86180cae70 2023-10-08T07:13:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f861e8b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86181f6330 con 0x7f86180c90b0 2023-10-08T07:13:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f861e8b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86181f6610 con 0x7f861815ab40 2023-10-08T07:13:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f860ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180cae70 0x7f861815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f860ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180cae70 0x7f861815c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43240/0 (socket says 172.21.15.70:43240) 2023-10-08T07:13:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f860ffff640 1 -- 172.21.15.70:0/2510507271 learned_addr learned my addr 172.21.15.70:0/2510507271 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f8617fff640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180c90b0 0x7f861814d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f860ffff640 1 -- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f861815ab40 msgr2=0x7f86181f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f860ffff640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f861815ab40 0x7f86181f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:22.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f860ffff640 1 -- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180c90b0 msgr2=0x7f861814d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f860ffff640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180c90b0 0x7f861814d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.624+0000 7f860ffff640 1 -- 172.21.15.70:0/2510507271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86000094d0 con 0x7f86180cae70 2023-10-08T07:13:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.625+0000 7f860ffff640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180cae70 0x7f861815c9d0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f8600000c00 tx=0x7f86000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.625+0000 7f86157fa640 1 -- 172.21.15.70:0/2510507271 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860003abe0 con 0x7f86180cae70 2023-10-08T07:13:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.625+0000 7f86157fa640 1 -- 172.21.15.70:0/2510507271 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860003ad40 con 0x7f86180cae70 2023-10-08T07:13:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.625+0000 7f86157fa640 1 -- 172.21.15.70:0/2510507271 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600044610 con 0x7f86180cae70 2023-10-08T07:13:22.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.625+0000 7f861e8b4640 1 -- 172.21.15.70:0/2510507271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861814e430 con 0x7f86180cae70 2023-10-08T07:13:22.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.625+0000 7f861e8b4640 1 -- 172.21.15.70:0/2510507271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861814def0 con 0x7f86180cae70 2023-10-08T07:13:22.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.626+0000 7f861e8b4640 1 -- 172.21.15.70:0/2510507271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861814e0e0 con 0x7f86180cae70 2023-10-08T07:13:22.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.627+0000 7f86157fa640 1 -- 172.21.15.70:0/2510507271 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f860004d450 con 0x7f86180cae70 2023-10-08T07:13:22.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.629+0000 7f86157fa640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f85f806af90 0x7f85f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:22.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.629+0000 7f86157fa640 1 -- 172.21.15.70:0/2510507271 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 6095+0+0 (secure 0 0 0) 0x7f860002d040 con 0x7f86180cae70 2023-10-08T07:13:22.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.629+0000 7f8617fff640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f85f806af90 0x7f85f806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:22.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.631+0000 7f8617fff640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f85f806af90 0x7f85f806d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8608006fd0 tx=0x7f8608008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:22.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.633+0000 7f86157fa640 1 -- 172.21.15.70:0/2510507271 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8600076cf0 con 0x7f86180cae70 2023-10-08T07:13:22.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.863+0000 7f861e8b4640 1 -- 172.21.15.70:0/2510507271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f861814e290 con 0x7f86180cae70 2023-10-08T07:13:22.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.865+0000 7f86157fa640 1 -- 172.21.15.70:0/2510507271 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+3705 (secure 0 0 0) 0x7f861814e290 con 0x7f86180cae70 2023-10-08T07:13:22.428 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:22.431 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":101,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6833/4185609199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":4185609199},{"type":"v1","addr":"172.21.15.70:6833","nonce":4185609199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7232,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/3102206290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":3102206290},{"type":"v1","addr":"172.21.15.181:6837","nonce":3102206290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8122,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1824427895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1824427895},{"type":"v1","addr":"172.21.15.181:6835","nonce":1824427895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":11016,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/378742070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":378742070},{"type":"v1","addr":"172.21.15.70:6835","nonce":378742070}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101}],"filesystems":[]} 2023-10-08T07:13:22.433 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 101 2023-10-08T07:13:22.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.867+0000 7f861e8b4640 1 -- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f85f806af90 msgr2=0x7f85f806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.867+0000 7f861e8b4640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f85f806af90 0x7f85f806d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8608006fd0 tx=0x7f8608008040 comp rx=0 tx=0).stop 2023-10-08T07:13:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.868+0000 7f861e8b4640 1 -- 172.21.15.70:0/2510507271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180cae70 msgr2=0x7f861815c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.868+0000 7f861e8b4640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180cae70 0x7f861815c9d0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f8600000c00 tx=0x7f86000046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.868+0000 7f861e8b4640 1 -- 172.21.15.70:0/2510507271 shutdown_connections 2023-10-08T07:13:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.868+0000 7f861e8b4640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f861815ab40 0x7f86181f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:22.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.868+0000 7f861e8b4640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f85f806af90 0x7f85f806d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:22.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.868+0000 7f861e8b4640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86180cae70 0x7f861815c9d0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:22.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.868+0000 7f861e8b4640 1 --2- 172.21.15.70:0/2510507271 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86180c90b0 0x7f861814d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:22.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.868+0000 7f861e8b4640 1 -- 172.21.15.70:0/2510507271 >> 172.21.15.70:0/2510507271 conn(0x7f861806a380 msgr2=0x7f86180ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:22.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.868+0000 7f861e8b4640 1 -- 172.21.15.70:0/2510507271 shutdown_connections 2023-10-08T07:13:22.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:21.868+0000 7f861e8b4640 1 -- 172.21.15.70:0/2510507271 wait complete. 2023-10-08T07:13:22.619 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T07:13:22.619 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T07:13:22.620 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T07:13:22.620 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:13:22.620 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T07:13:22.620 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T07:13:22.621 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T07:13:22.621 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:13:22.621 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T07:13:22.621 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T07:13:22.622 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T07:13:22.622 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:13:22.622 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T07:13:22.623 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T07:13:22.623 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T07:13:22.623 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:13:22.643 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T07:13:22.834 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T07:13:22.884 INFO:tasks.ceph.mds.b:Started 2023-10-08T07:13:23.017 DEBUG:teuthology.parallel:result is None 2023-10-08T07:13:23.073 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T07:13:24.868 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-08T07:13:25.010 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-10-08T07:13:25.010 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-08T07:13:28.836 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T07:13:29.021 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T07:13:29.021 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T07:13:29.021 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T07:13:29.022 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T07:13:29.023 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T07:13:29.032 INFO:tasks.ceph.mds.a:Started 2023-10-08T07:13:29.032 DEBUG:teuthology.parallel:result is None 2023-10-08T07:13:29.033 INFO:tasks.ceph.mds.d:Started 2023-10-08T07:13:29.033 DEBUG:teuthology.parallel:result is None 2023-10-08T07:13:29.039 INFO:tasks.ceph.mds.c:Started 2023-10-08T07:13:29.039 DEBUG:teuthology.parallel:result is None 2023-10-08T07:13:29.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T07:13:29.194 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T07:13:29.194 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T07:13:29.196 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-10-08T07:13:29.195+0000 7f739af5b640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-08T07:13:29.220 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T07:13:29.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.227+0000 7f89e9039640 1 Processor -- start 2023-10-08T07:13:29.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.228+0000 7f89e9039640 1 -- start start 2023-10-08T07:13:29.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.229+0000 7f89e9039640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e406dfc0 0x7f89e4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:29.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.229+0000 7f89e9039640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f89e4071070 0x7f89e40cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:29.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.229+0000 7f89e9039640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e40b1cc0 con 0x7f89e40afbe0 2023-10-08T07:13:29.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.229+0000 7f89e9039640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e40b2620 con 0x7f89e4071070 2023-10-08T07:13:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.229+0000 7f89e9039640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40b2060 con 0x7f89e406dfc0 2023-10-08T07:13:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.229+0000 7f89e2d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f89e40afbe0 0x7f89e40b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43200/0 (socket says 172.21.15.70:43200) 2023-10-08T07:13:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.229+0000 7f89e2d9d640 1 -- 172.21.15.70:0/428347428 learned_addr learned my addr 172.21.15.70:0/428347428 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.229+0000 7f89e259c640 1 --2- 172.21.15.70:0/428347428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e406dfc0 0x7f89e4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.229+0000 7f89e359e640 1 --2- 172.21.15.70:0/428347428 >> v2:172.21.15.181:3300/0 conn(0x7f89e4071070 0x7f89e40cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.230+0000 7f89e1d9b640 1 -- 172.21.15.70:0/428347428 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178273237 0 0) 0x7f89e40b1cc0 con 0x7f89e40afbe0 2023-10-08T07:13:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.230+0000 7f89e1d9b640 1 -- 172.21.15.70:0/428347428 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89cc003520 con 0x7f89e40afbe0 2023-10-08T07:13:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.230+0000 7f89e259c640 1 -- 172.21.15.70:0/428347428 >> v2:172.21.15.181:3300/0 conn(0x7f89e4071070 msgr2=0x7f89e40cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:29.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.230+0000 7f89e259c640 1 --2- 172.21.15.70:0/428347428 >> v2:172.21.15.181:3300/0 conn(0x7f89e4071070 0x7f89e40cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:29.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.230+0000 7f89e259c640 1 -- 172.21.15.70:0/428347428 >> v1:172.21.15.70:6789/0 conn(0x7f89e40afbe0 legacy=0x7f89e40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:29.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.230+0000 7f89e259c640 1 -- 172.21.15.70:0/428347428 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e40b2340 con 0x7f89e406dfc0 2023-10-08T07:13:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.231+0000 7f89e259c640 1 --2- 172.21.15.70:0/428347428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e406dfc0 0x7f89e4070b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89d80097a0 tx=0x7f89d802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70b5a81189beb6b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.231+0000 7f89e1d9b640 1 -- 172.21.15.70:0/428347428 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d802d040 con 0x7f89e406dfc0 2023-10-08T07:13:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.231+0000 7f89e1d9b640 1 -- 172.21.15.70:0/428347428 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8031810 con 0x7f89e406dfc0 2023-10-08T07:13:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.231+0000 7f89e1d9b640 1 -- 172.21.15.70:0/428347428 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8031ba0 con 0x7f89e406dfc0 2023-10-08T07:13:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.232+0000 7f89e9039640 1 -- 172.21.15.70:0/428347428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e406dfc0 msgr2=0x7f89e4070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:29.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.232+0000 7f89e9039640 1 --2- 172.21.15.70:0/428347428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e406dfc0 0x7f89e4070b30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89d80097a0 tx=0x7f89d802b8b0 comp rx=0 tx=0).stop 2023-10-08T07:13:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.232+0000 7f89e9039640 1 -- 172.21.15.70:0/428347428 shutdown_connections 2023-10-08T07:13:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.232+0000 7f89e9039640 1 --2- 172.21.15.70:0/428347428 >> v2:172.21.15.181:3300/0 conn(0x7f89e4071070 0x7f89e40cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.232+0000 7f89e9039640 1 --2- 172.21.15.70:0/428347428 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e406dfc0 0x7f89e4070b30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.232+0000 7f89e9039640 1 -- 172.21.15.70:0/428347428 >> 172.21.15.70:0/428347428 conn(0x7f89e406a5d0 msgr2=0x7f89e40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.232+0000 7f89e9039640 1 -- 172.21.15.70:0/428347428 shutdown_connections 2023-10-08T07:13:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.232+0000 7f89e9039640 1 -- 172.21.15.70:0/428347428 wait complete. 2023-10-08T07:13:29.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.234+0000 7f89e9039640 1 Processor -- start 2023-10-08T07:13:29.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.234+0000 7f89e9039640 1 -- start start 2023-10-08T07:13:29.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e9039640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e406dfc0 0x7f89e414fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e9039640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e4071070 0x7f89e40ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e9039640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e40afbe0 0x7f89e40c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e9039640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e41501a0 con 0x7f89e406dfc0 2023-10-08T07:13:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e9039640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40cca20 con 0x7f89e4071070 2023-10-08T07:13:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e9039640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40ccd00 con 0x7f89e40afbe0 2023-10-08T07:13:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e406dfc0 0x7f89e414fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e406dfc0 0x7f89e414fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51002/0 (socket says 172.21.15.70:51002) 2023-10-08T07:13:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e2d9d640 1 -- 172.21.15.70:0/3842493740 learned_addr learned my addr 172.21.15.70:0/3842493740 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:29.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e259c640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e4071070 0x7f89e40ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e359e640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e40afbe0 0x7f89e40c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e2d9d640 1 -- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e40afbe0 msgr2=0x7f89e40c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e2d9d640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e40afbe0 0x7f89e40c2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e2d9d640 1 -- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e4071070 msgr2=0x7f89e40ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e2d9d640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e4071070 0x7f89e40ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.235+0000 7f89e2d9d640 1 -- 172.21.15.70:0/3842493740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8009400 con 0x7f89e406dfc0 2023-10-08T07:13:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.236+0000 7f89e2d9d640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e406dfc0 0x7f89e414fc60 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f89d400c500 tx=0x7f89d400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.236+0000 7f89cb7fe640 1 -- 172.21.15.70:0/3842493740 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4005ce0 con 0x7f89e406dfc0 2023-10-08T07:13:29.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.236+0000 7f89cb7fe640 1 -- 172.21.15.70:0/3842493740 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4005e40 con 0x7f89e406dfc0 2023-10-08T07:13:29.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.236+0000 7f89cb7fe640 1 -- 172.21.15.70:0/3842493740 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4007330 con 0x7f89e406dfc0 2023-10-08T07:13:29.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.237+0000 7f89e9039640 1 -- 172.21.15.70:0/3842493740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e40ccfe0 con 0x7f89e406dfc0 2023-10-08T07:13:29.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.237+0000 7f89e9039640 1 -- 172.21.15.70:0/3842493740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e40cd5a0 con 0x7f89e406dfc0 2023-10-08T07:13:29.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.238+0000 7f89e9039640 1 -- 172.21.15.70:0/3842493740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e404e160 con 0x7f89e406dfc0 2023-10-08T07:13:29.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.239+0000 7f89cb7fe640 1 -- 172.21.15.70:0/3842493740 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f89d4007550 con 0x7f89e406dfc0 2023-10-08T07:13:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.240+0000 7f89cb7fe640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f89d006afe0 0x7f89d006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.240+0000 7f89cb7fe640 1 -- 172.21.15.70:0/3842493740 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 6067+0+0 (secure 0 0 0) 0x7f89d4088e80 con 0x7f89e406dfc0 2023-10-08T07:13:29.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.241+0000 7f89e259c640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f89d006afe0 0x7f89d006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:29.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.241+0000 7f89e259c640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f89d006afe0 0x7f89d006d4a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f89d8036000 tx=0x7f89d803c040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.245+0000 7f89cb7fe640 1 -- 172.21.15.70:0/3842493740 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f89d4053ee0 con 0x7f89e406dfc0 2023-10-08T07:13:29.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:29.440+0000 7f89e9039640 1 -- 172.21.15.70:0/3842493740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f89e40b0ef0 con 0x7f89e406dfc0 2023-10-08T07:13:30.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.290+0000 7f89cb7fe640 1 -- 172.21.15.70:0/3842493740 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v230) v1 ==== 97+0+0 (secure 0 0 0) 0x7f89d4017020 con 0x7f89e406dfc0 2023-10-08T07:13:30.292 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T07:13:30.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.294+0000 7f89e9039640 1 -- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f89d006afe0 msgr2=0x7f89d006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:30.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.294+0000 7f89e9039640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f89d006afe0 0x7f89d006d4a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f89d8036000 tx=0x7f89d803c040 comp rx=0 tx=0).stop 2023-10-08T07:13:30.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.294+0000 7f89e9039640 1 -- 172.21.15.70:0/3842493740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e406dfc0 msgr2=0x7f89e414fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:30.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.294+0000 7f89e9039640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e406dfc0 0x7f89e414fc60 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f89d400c500 tx=0x7f89d400c9c0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.294+0000 7f89e9039640 1 -- 172.21.15.70:0/3842493740 shutdown_connections 2023-10-08T07:13:30.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.294+0000 7f89e9039640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f89e40afbe0 0x7f89e40c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.294+0000 7f89e9039640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f89e4071070 0x7f89e40ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.294+0000 7f89e9039640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f89d006afe0 0x7f89d006d4a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.294+0000 7f89e9039640 1 --2- 172.21.15.70:0/3842493740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e406dfc0 0x7f89e414fc60 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.294+0000 7f89e9039640 1 -- 172.21.15.70:0/3842493740 >> 172.21.15.70:0/3842493740 conn(0x7f89e406a5d0 msgr2=0x7f89e40c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:30.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.295+0000 7f89e9039640 1 -- 172.21.15.70:0/3842493740 shutdown_connections 2023-10-08T07:13:30.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.295+0000 7f89e9039640 1 -- 172.21.15.70:0/3842493740 wait complete. 2023-10-08T07:13:30.318 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T07:13:30.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.531+0000 7fc4bd09b640 1 Processor -- start 2023-10-08T07:13:30.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.531+0000 7fc4bd09b640 1 -- start start 2023-10-08T07:13:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.532+0000 7fc4bd09b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80cb190 0x7fc4b80cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.532+0000 7fc4bd09b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b80adfc0 con 0x7fc4b8157000 2023-10-08T07:13:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.532+0000 7fc4bd09b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b8159cb0 con 0x7fc4b81597d0 2023-10-08T07:13:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.532+0000 7fc4bd09b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80cbb80 con 0x7fc4b80cb190 2023-10-08T07:13:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.532+0000 7fc4b6d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc4b8157000 0x7fc4b80cc7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43210/0 (socket says 172.21.15.70:43210) 2023-10-08T07:13:30.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b6d9d640 1 -- 172.21.15.70:0/2518937373 learned_addr learned my addr 172.21.15.70:0/2518937373 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:30.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067228838 0 0) 0x7fc4b80adfc0 con 0x7fc4b8157000 2023-10-08T07:13:30.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4a4003520 con 0x7fc4b8157000 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b759e640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80cb190 0x7fc4b80cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3388433196 0 0) 0x7fc4a4003520 con 0x7fc4b8157000 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80cb190 msgr2=0x7fc4b80cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b659c640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80cb190 0x7fc4b80cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 >> v1:172.21.15.181:6789/0 conn(0x7fc4b81597d0 legacy=0x7fc4b806ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b8159f90 con 0x7fc4b8157000 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc4a8001670 con 0x7fc4b8157000 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 >> v1:172.21.15.70:6789/0 conn(0x7fc4b8157000 legacy=0x7fc4b80cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.533+0000 7fc4b659c640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 0x7fc4a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b659c640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80cb190 0x7fc4a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b659c640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4a4004d00 0x7fc4a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:30.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80cbb80 con 0x7fc4a40039a0 2023-10-08T07:13:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a400bab0 con 0x7fc4a4004d00 2023-10-08T07:13:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a400bc70 con 0x7fc4b80cb190 2023-10-08T07:13:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b6d9d640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 0x7fc4a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b759e640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4a4004d00 0x7fc4a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b6d9d640 1 -- 172.21.15.70:0/2518937373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80cb190 msgr2=0x7fc4a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b6d9d640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80cb190 0x7fc4a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b6d9d640 1 -- 172.21.15.70:0/2518937373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4a4004d00 msgr2=0x7fc4a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b6d9d640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4a4004d00 0x7fc4a4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.534+0000 7fc4b6d9d640 1 -- 172.21.15.70:0/2518937373 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a4003520 con 0x7fc4a40039a0 2023-10-08T07:13:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.535+0000 7fc4b6d9d640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 0x7fc4a4003da0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8007f90 tx=0x7fc4a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68e73e08a7c186e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.536+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a802ec30 con 0x7fc4a40039a0 2023-10-08T07:13:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.536+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a802edb0 con 0x7fc4a40039a0 2023-10-08T07:13:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.536+0000 7fc4b659c640 1 -- 172.21.15.70:0/2518937373 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8005b10 con 0x7fc4a40039a0 2023-10-08T07:13:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.536+0000 7fc4bd09b640 1 -- 172.21.15.70:0/2518937373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 msgr2=0x7fc4a4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.536+0000 7fc4bd09b640 1 --2- 172.21.15.70:0/2518937373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 0x7fc4a4003da0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8007f90 tx=0x7fc4a40057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.536+0000 7fc4b759e640 1 -- 172.21.15.70:0/2518937373 reap_dead start 2023-10-08T07:13:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.536+0000 7fc4bd09b640 1 -- 172.21.15.70:0/2518937373 shutdown_connections 2023-10-08T07:13:30.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.536+0000 7fc4bd09b640 1 -- 172.21.15.70:0/2518937373 >> 172.21.15.70:0/2518937373 conn(0x7fc4b806a380 msgr2=0x7fc4b80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:30.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.536+0000 7fc4bd09b640 1 -- 172.21.15.70:0/2518937373 shutdown_connections 2023-10-08T07:13:30.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.536+0000 7fc4bd09b640 1 -- 172.21.15.70:0/2518937373 wait complete. 2023-10-08T07:13:30.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.538+0000 7fc4bd09b640 1 Processor -- start 2023-10-08T07:13:30.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.538+0000 7fc4bd09b640 1 -- start start 2023-10-08T07:13:30.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.539+0000 7fc4bd09b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 0x7fc4b81f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.539+0000 7fc4bd09b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80c9f00 0x7fc4b8150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.539+0000 7fc4bd09b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4b80ca380 0x7fc4b81522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.539+0000 7fc4bd09b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b814fcd0 con 0x7fc4a40039a0 2023-10-08T07:13:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.539+0000 7fc4bd09b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b81f6410 con 0x7fc4b80ca380 2023-10-08T07:13:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.539+0000 7fc4bd09b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80b4400 con 0x7fc4b80c9f00 2023-10-08T07:13:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.539+0000 7fc4b6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 0x7fc4b81f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.539+0000 7fc4b6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 0x7fc4b81f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51018/0 (socket says 172.21.15.70:51018) 2023-10-08T07:13:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.539+0000 7fc4b6d9d640 1 -- 172.21.15.70:0/3873126232 learned_addr learned my addr 172.21.15.70:0/3873126232 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:30.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.540+0000 7fc4b6d9d640 1 -- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80c9f00 msgr2=0x7fc4b8150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:30.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.540+0000 7fc4b6d9d640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80c9f00 0x7fc4b8150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.540+0000 7fc4b6d9d640 1 -- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4b80ca380 msgr2=0x7fc4b81522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.540+0000 7fc4b6d9d640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4b80ca380 0x7fc4b81522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.540+0000 7fc4b6d9d640 1 -- 172.21.15.70:0/3873126232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b80b46e0 con 0x7fc4a40039a0 2023-10-08T07:13:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.540+0000 7fc4b6d9d640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 0x7fc4b81f68a0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8001930 tx=0x7fc4a8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.540+0000 7fc4affff640 1 -- 172.21.15.70:0/3873126232 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8007070 con 0x7fc4a40039a0 2023-10-08T07:13:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.540+0000 7fc4affff640 1 -- 172.21.15.70:0/3873126232 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a80071d0 con 0x7fc4a40039a0 2023-10-08T07:13:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.540+0000 7fc4affff640 1 -- 172.21.15.70:0/3873126232 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8007540 con 0x7fc4a40039a0 2023-10-08T07:13:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.540+0000 7fc4bd09b640 1 -- 172.21.15.70:0/3873126232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b81f5dc0 con 0x7fc4a40039a0 2023-10-08T07:13:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.541+0000 7fc4bd09b640 1 -- 172.21.15.70:0/3873126232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b80b4060 con 0x7fc4a40039a0 2023-10-08T07:13:30.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.542+0000 7fc4bd09b640 1 -- 172.21.15.70:0/3873126232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b8067490 con 0x7fc4a40039a0 2023-10-08T07:13:30.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.542+0000 7fc4affff640 1 -- 172.21.15.70:0/3873126232 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc4a8034070 con 0x7fc4a40039a0 2023-10-08T07:13:30.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.543+0000 7fc4affff640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc49806af90 0x7fc49806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:30.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.543+0000 7fc4affff640 1 -- 172.21.15.70:0/3873126232 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fc4a803e070 con 0x7fc4a40039a0 2023-10-08T07:13:30.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.544+0000 7fc4ae59c640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc49806af90 0x7fc49806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:30.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.545+0000 7fc4ae59c640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc49806af90 0x7fc49806d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80b3a60 tx=0x7fc4a00079b0 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:30.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.548+0000 7fc4affff640 1 -- 172.21.15.70:0/3873126232 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4a8076200 con 0x7fc4a40039a0 2023-10-08T07:13:30.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.805+0000 7fc4bd09b640 1 -- 172.21.15.70:0/3873126232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc4b806c6d0 con 0x7fc4a40039a0 2023-10-08T07:13:30.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.806+0000 7fc4affff640 1 -- 172.21.15.70:0/3873126232 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v51) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc4a8079820 con 0x7fc4a40039a0 2023-10-08T07:13:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:30.807 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:13:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T07:13:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T07:13:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T07:13:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:13:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:13:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:13:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T07:13:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T07:13:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:13:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:13:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:13:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T07:13:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T07:13:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:13:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T07:13:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T07:13:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:13:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:13:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T07:13:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T07:13:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:13:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:13:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T07:13:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T07:13:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:13:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:13:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T07:13:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T07:13:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:13:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T07:13:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:13:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T07:13:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T07:13:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:13:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:13:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:13:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T07:13:30.828 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T07:13:30.828 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:13:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T07:13:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T07:13:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T07:13:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:13:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:13:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T07:13:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:13:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T07:13:30.835 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:13:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 -- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc49806af90 msgr2=0x7fc49806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc49806af90 0x7fc49806d450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80b3a60 tx=0x7fc4a00079b0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 -- 172.21.15.70:0/3873126232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 msgr2=0x7fc4b81f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 0x7fc4b81f68a0 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8001930 tx=0x7fc4a8001a00 comp rx=0 tx=0).stop 2023-10-08T07:13:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 -- 172.21.15.70:0/3873126232 shutdown_connections 2023-10-08T07:13:30.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4b80ca380 0x7fc4b81522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4b80c9f00 0x7fc4b8150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc49806af90 0x7fc49806d450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 --2- 172.21.15.70:0/3873126232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4a40039a0 0x7fc4b81f68a0 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 -- 172.21.15.70:0/3873126232 >> 172.21.15.70:0/3873126232 conn(0x7fc4b806a380 msgr2=0x7fc4b806db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 -- 172.21.15.70:0/3873126232 shutdown_connections 2023-10-08T07:13:30.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:30.808+0000 7fc4bd09b640 1 -- 172.21.15.70:0/3873126232 wait complete. 2023-10-08T07:13:30.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:31.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.045+0000 7fb9b0b39640 1 Processor -- start 2023-10-08T07:13:31.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.045+0000 7fb9b0b39640 1 -- start start 2023-10-08T07:13:31.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.046+0000 7fb9b0b39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9ac157000 0x7fb9ac0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.046+0000 7fb9b0b39640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb9ac1597d0 0x7fb9ac06ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.046+0000 7fb9b0b39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9ac0cb190 0x7fb9ac0cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9b0b39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ac0cbb80 con 0x7fb9ac157000 2023-10-08T07:13:31.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9b0b39640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ac0adfc0 con 0x7fb9ac1597d0 2023-10-08T07:13:31.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9b0b39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac0ccce0 con 0x7fb9ac0cb190 2023-10-08T07:13:31.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9aa59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9ac157000 0x7fb9ac0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9aa59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb9ac157000 0x7fb9ac0cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51022/0 (socket says 172.21.15.70:51022) 2023-10-08T07:13:31.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9aa59c640 1 -- 172.21.15.70:0/2783983772 learned_addr learned my addr 172.21.15.70:0/2783983772 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:31.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9aa59c640 1 -- 172.21.15.70:0/2783983772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9ac0cb190 msgr2=0x7fb9ac0cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:31.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9aa59c640 1 --2- 172.21.15.70:0/2783983772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9ac0cb190 0x7fb9ac0cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9aa59c640 1 -- 172.21.15.70:0/2783983772 >> v2:172.21.15.181:3300/0 conn(0x7fb9ac1597d0 msgr2=0x7fb9ac06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:31.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9aa59c640 1 --2- 172.21.15.70:0/2783983772 >> v2:172.21.15.181:3300/0 conn(0x7fb9ac1597d0 0x7fb9ac06ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.047+0000 7fb9aa59c640 1 -- 172.21.15.70:0/2783983772 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac06d380 con 0x7fb9ac157000 2023-10-08T07:13:31.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.048+0000 7fb9aa59c640 1 --2- 172.21.15.70:0/2783983772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac157000 0x7fb9ac0cc7a0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac071770 tx=0x7fb99c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e489a6a3001350ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:31.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9a9d9b640 1 -- 172.21.15.70:0/2783983772 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c02ca80 con 0x7fb9ac157000 2023-10-08T07:13:31.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9a9d9b640 1 -- 172.21.15.70:0/2783983772 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c02cbe0 con 0x7fb9ac157000 2023-10-08T07:13:31.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9a9d9b640 1 -- 172.21.15.70:0/2783983772 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0343f0 con 0x7fb9ac157000 2023-10-08T07:13:31.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9b0b39640 1 -- 172.21.15.70:0/2783983772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac157000 msgr2=0x7fb9ac0cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9b0b39640 1 --2- 172.21.15.70:0/2783983772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac157000 0x7fb9ac0cc7a0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac071770 tx=0x7fb99c02bee0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9b0b39640 1 -- 172.21.15.70:0/2783983772 shutdown_connections 2023-10-08T07:13:31.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9b0b39640 1 --2- 172.21.15.70:0/2783983772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9ac0cb190 0x7fb9ac0cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9b0b39640 1 --2- 172.21.15.70:0/2783983772 >> v2:172.21.15.181:3300/0 conn(0x7fb9ac1597d0 0x7fb9ac06ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9b0b39640 1 --2- 172.21.15.70:0/2783983772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac157000 0x7fb9ac0cc7a0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9b0b39640 1 -- 172.21.15.70:0/2783983772 >> 172.21.15.70:0/2783983772 conn(0x7fb9ac06a380 msgr2=0x7fb9ac0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:31.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9b0b39640 1 -- 172.21.15.70:0/2783983772 shutdown_connections 2023-10-08T07:13:31.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.049+0000 7fb9b0b39640 1 -- 172.21.15.70:0/2783983772 wait complete. 2023-10-08T07:13:31.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.051+0000 7fb9b0b39640 1 Processor -- start 2023-10-08T07:13:31.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.051+0000 7fb9b0b39640 1 -- start start 2023-10-08T07:13:31.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.052+0000 7fb9b0b39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9ac0cb190 0x7fb9ac1518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.052+0000 7fb9b0b39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9ac1597d0 0x7fb9ac153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.052+0000 7fb9b0b39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac1573f0 0x7fb9ac154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.052+0000 7fb9b0b39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1f6330 con 0x7fb9ac1573f0 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.052+0000 7fb9b0b39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1f65f0 con 0x7fb9ac0cb190 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.052+0000 7fb9b0b39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1f68d0 con 0x7fb9ac1597d0 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac1573f0 0x7fb9ac154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac1573f0 0x7fb9ac154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51024/0 (socket says 172.21.15.70:51024) 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aad9d640 1 -- 172.21.15.70:0/3256859269 learned_addr learned my addr 172.21.15.70:0/3256859269 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aa59c640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9ac0cb190 0x7fb9ac1518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9a1d9b640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9ac1597d0 0x7fb9ac153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aad9d640 1 -- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9ac1597d0 msgr2=0x7fb9ac153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aad9d640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9ac1597d0 0x7fb9ac153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aad9d640 1 -- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9ac0cb190 msgr2=0x7fb9ac1518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aad9d640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9ac0cb190 0x7fb9ac1518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aad9d640 1 -- 172.21.15.70:0/3256859269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c009b30 con 0x7fb9ac1573f0 2023-10-08T07:13:31.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aa59c640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9ac0cb190 0x7fb9ac1518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:31.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9a1d9b640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9ac1597d0 0x7fb9ac153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:31.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.053+0000 7fb9aad9d640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac1573f0 0x7fb9ac154210 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400b410 tx=0x7fb9a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:31.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.054+0000 7fb9a37fe640 1 -- 172.21.15.70:0/3256859269 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4011830 con 0x7fb9ac1573f0 2023-10-08T07:13:31.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.054+0000 7fb9a37fe640 1 -- 172.21.15.70:0/3256859269 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a40119b0 con 0x7fb9ac1573f0 2023-10-08T07:13:31.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.054+0000 7fb9a37fe640 1 -- 172.21.15.70:0/3256859269 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4011d60 con 0x7fb9ac1573f0 2023-10-08T07:13:31.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.054+0000 7fb9b0b39640 1 -- 172.21.15.70:0/3256859269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac1f6bb0 con 0x7fb9ac1573f0 2023-10-08T07:13:31.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.054+0000 7fb9b0b39640 1 -- 172.21.15.70:0/3256859269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac151e30 con 0x7fb9ac1573f0 2023-10-08T07:13:31.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.055+0000 7fb9b0b39640 1 -- 172.21.15.70:0/3256859269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ac1f70d0 con 0x7fb9ac1573f0 2023-10-08T07:13:31.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.056+0000 7fb9a37fe640 1 -- 172.21.15.70:0/3256859269 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb9a401a400 con 0x7fb9ac1573f0 2023-10-08T07:13:31.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.057+0000 7fb9a37fe640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb98c06af90 0x7fb98c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.057+0000 7fb9a37fe640 1 -- 172.21.15.70:0/3256859269 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fb9a4089710 con 0x7fb9ac1573f0 2023-10-08T07:13:31.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.058+0000 7fb9aa59c640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb98c06af90 0x7fb98c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.059+0000 7fb9aa59c640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb98c06af90 0x7fb98c06d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb99c035000 tx=0x7fb99c030000 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:31.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.062+0000 7fb9a37fe640 1 -- 172.21.15.70:0/3256859269 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9a4054840 con 0x7fb9ac1573f0 2023-10-08T07:13:31.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.251+0000 7fb9b0b39640 1 -- 172.21.15.70:0/3256859269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9ac067490 con 0x7fb9ac1573f0 2023-10-08T07:13:31.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.253+0000 7fb9a37fe640 1 -- 172.21.15.70:0/3256859269 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v230) v1 ==== 74+0+13246 (secure 0 0 0) 0x7fb9a4015090 con 0x7fb9ac1573f0 2023-10-08T07:13:31.254 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:31.254 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":230,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:30.280663+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":218,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 -- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb98c06af90 msgr2=0x7fb98c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb98c06af90 0x7fb98c06d450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb99c035000 tx=0x7fb99c030000 comp rx=0 tx=0).stop 2023-10-08T07:13:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 -- 172.21.15.70:0/3256859269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac1573f0 msgr2=0x7fb9ac154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac1573f0 0x7fb9ac154210 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400b410 tx=0x7fb9a400b8d0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 -- 172.21.15.70:0/3256859269 shutdown_connections 2023-10-08T07:13:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fb98c06af90 0x7fb98c06d450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9ac1573f0 0x7fb9ac154210 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb9ac1597d0 0x7fb9ac153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 --2- 172.21.15.70:0/3256859269 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb9ac0cb190 0x7fb9ac1518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 -- 172.21.15.70:0/3256859269 >> 172.21.15.70:0/3256859269 conn(0x7fb9ac06a380 msgr2=0x7fb9ac0b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:31.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 -- 172.21.15.70:0/3256859269 shutdown_connections 2023-10-08T07:13:31.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.256+0000 7fb9b0b39640 1 -- 172.21.15.70:0/3256859269 wait complete. 2023-10-08T07:13:31.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:13:31.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:13:31.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.490+0000 7f5f826c6640 1 Processor -- start 2023-10-08T07:13:31.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.490+0000 7f5f826c6640 1 -- start start 2023-10-08T07:13:31.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.492+0000 7f5f826c6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f7c15aad0 0x7f5f7c0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.492+0000 7f5f826c6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5f7c06d410 0x7f5f7c0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.492+0000 7f5f826c6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7c15bf30 con 0x7f5f7c0c82b0 2023-10-08T07:13:31.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.492+0000 7f5f826c6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7c15b320 con 0x7f5f7c06d410 2023-10-08T07:13:31.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.492+0000 7f5f826c6640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7c15b600 con 0x7f5f7c15aad0 2023-10-08T07:13:31.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.492+0000 7f5f80c63640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5f7c06d410 0x7f5f7c0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.492+0000 7f5f80c63640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5f7c06d410 0x7f5f7c0c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51034/0 (socket says 172.21.15.70:51034) 2023-10-08T07:13:31.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.492+0000 7f5f80c63640 1 -- 172.21.15.70:0/924078196 learned_addr learned my addr 172.21.15.70:0/924078196 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:31.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.492+0000 7f5f7bfff640 1 --2- 172.21.15.70:0/924078196 >> v2:172.21.15.181:3300/0 conn(0x7f5f7c15aad0 0x7f5f7c0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.493+0000 7f5f80c63640 1 -- 172.21.15.70:0/924078196 >> v2:172.21.15.181:3300/0 conn(0x7f5f7c15aad0 msgr2=0x7f5f7c0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:31.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.493+0000 7f5f80c63640 1 --2- 172.21.15.70:0/924078196 >> v2:172.21.15.181:3300/0 conn(0x7f5f7c15aad0 0x7f5f7c0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.493+0000 7f5f80c63640 1 -- 172.21.15.70:0/924078196 >> v1:172.21.15.181:6789/0 conn(0x7f5f7c0c82b0 legacy=0x7f5f7c06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:31.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.493+0000 7f5f80c63640 1 -- 172.21.15.70:0/924078196 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c15b8e0 con 0x7f5f7c06d410 2023-10-08T07:13:31.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.494+0000 7f5f80c63640 1 --2- 172.21.15.70:0/924078196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 0x7f5f7c0c8710 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c06daa0 tx=0x7f5f7402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2abbdb7e7bc7197 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:31.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.494+0000 7f5f7affd640 1 -- 172.21.15.70:0/924078196 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7402ca50 con 0x7f5f7c06d410 2023-10-08T07:13:31.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.494+0000 7f5f7affd640 1 -- 172.21.15.70:0/924078196 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7402cbb0 con 0x7f5f7c06d410 2023-10-08T07:13:31.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.494+0000 7f5f7affd640 1 -- 172.21.15.70:0/924078196 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f740343f0 con 0x7f5f7c06d410 2023-10-08T07:13:31.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.495+0000 7f5f826c6640 1 -- 172.21.15.70:0/924078196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 msgr2=0x7f5f7c0c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.495+0000 7f5f826c6640 1 --2- 172.21.15.70:0/924078196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 0x7f5f7c0c8710 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c06daa0 tx=0x7f5f7402beb0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.495+0000 7f5f826c6640 1 -- 172.21.15.70:0/924078196 shutdown_connections 2023-10-08T07:13:31.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.495+0000 7f5f826c6640 1 --2- 172.21.15.70:0/924078196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 0x7f5f7c0c8710 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.495+0000 7f5f826c6640 1 --2- 172.21.15.70:0/924078196 >> v2:172.21.15.181:3300/0 conn(0x7f5f7c15aad0 0x7f5f7c0b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.495+0000 7f5f826c6640 1 -- 172.21.15.70:0/924078196 >> 172.21.15.70:0/924078196 conn(0x7f5f7c069d50 msgr2=0x7f5f7c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:31.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.495+0000 7f5f826c6640 1 -- 172.21.15.70:0/924078196 shutdown_connections 2023-10-08T07:13:31.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.495+0000 7f5f826c6640 1 -- 172.21.15.70:0/924078196 wait complete. 2023-10-08T07:13:31.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.497+0000 7f5f826c6640 1 Processor -- start 2023-10-08T07:13:31.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.498+0000 7f5f826c6640 1 -- start start 2023-10-08T07:13:31.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.499+0000 7f5f826c6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 0x7f5f7c156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.499+0000 7f5f826c6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f7c0c82b0 0x7f5f7c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.499+0000 7f5f826c6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7c15aad0 0x7f5f7c0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.499+0000 7f5f826c6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c141b00 con 0x7f5f7c06d410 2023-10-08T07:13:31.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.499+0000 7f5f826c6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c0cdb40 con 0x7f5f7c0c82b0 2023-10-08T07:13:31.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.499+0000 7f5f826c6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c140ef0 con 0x7f5f7c15aad0 2023-10-08T07:13:31.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.499+0000 7f5f7bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 0x7f5f7c156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.499+0000 7f5f7bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 0x7f5f7c156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51042/0 (socket says 172.21.15.70:51042) 2023-10-08T07:13:31.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.499+0000 7f5f7bfff640 1 -- 172.21.15.70:0/3968059324 learned_addr learned my addr 172.21.15.70:0/3968059324 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:31.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f7bfff640 1 -- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7c15aad0 msgr2=0x7f5f7c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f7bfff640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7c15aad0 0x7f5f7c0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f7bfff640 1 -- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f7c0c82b0 msgr2=0x7f5f7c0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f7bfff640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f7c0c82b0 0x7f5f7c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f7bfff640 1 -- 172.21.15.70:0/3968059324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f74009b30 con 0x7f5f7c06d410 2023-10-08T07:13:31.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f7bfff640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 0x7f5f7c156820 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00d5f0 tx=0x7f5f6c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:31.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f78ff9640 1 -- 172.21.15.70:0/3968059324 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c0125f0 con 0x7f5f7c06d410 2023-10-08T07:13:31.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f78ff9640 1 -- 172.21.15.70:0/3968059324 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c012750 con 0x7f5f7c06d410 2023-10-08T07:13:31.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f826c6640 1 -- 172.21.15.70:0/3968059324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c1411d0 con 0x7f5f7c06d410 2023-10-08T07:13:31.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f78ff9640 1 -- 172.21.15.70:0/3968059324 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c012ae0 con 0x7f5f7c06d410 2023-10-08T07:13:31.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.500+0000 7f5f826c6640 1 -- 172.21.15.70:0/3968059324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c141790 con 0x7f5f7c06d410 2023-10-08T07:13:31.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.501+0000 7f5f826c6640 1 -- 172.21.15.70:0/3968059324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7c04e160 con 0x7f5f7c06d410 2023-10-08T07:13:31.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.502+0000 7f5f78ff9640 1 -- 172.21.15.70:0/3968059324 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5f6c012c40 con 0x7f5f7c06d410 2023-10-08T07:13:31.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.503+0000 7f5f78ff9640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5f6806afc0 0x7f5f6806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.504+0000 7f5f78ff9640 1 -- 172.21.15.70:0/3968059324 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f5f6c089b70 con 0x7f5f7c06d410 2023-10-08T07:13:31.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.504+0000 7f5f7b7fe640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5f6806afc0 0x7f5f6806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.505+0000 7f5f7b7fe640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5f6806afc0 0x7f5f6806d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0b28f0 tx=0x7f5f700079b0 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:31.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.508+0000 7f5f78ff9640 1 -- 172.21.15.70:0/3968059324 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f6c054c20 con 0x7f5f7c06d410 2023-10-08T07:13:31.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.697+0000 7f5f826c6640 1 -- 172.21.15.70:0/3968059324 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f7c156090 con 0x7f5f7c06d410 2023-10-08T07:13:31.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.698+0000 7f5f78ff9640 1 -- 172.21.15.70:0/3968059324 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v230) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f6c058240 con 0x7f5f7c06d410 2023-10-08T07:13:31.698 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:13:31.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.700+0000 7f5f826c6640 1 -- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5f6806afc0 msgr2=0x7f5f6806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.700+0000 7f5f826c6640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5f6806afc0 0x7f5f6806d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0b28f0 tx=0x7f5f700079b0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.700+0000 7f5f826c6640 1 -- 172.21.15.70:0/3968059324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 msgr2=0x7f5f7c156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.700+0000 7f5f826c6640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 0x7f5f7c156820 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00d5f0 tx=0x7f5f6c00dab0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.700+0000 7f5f826c6640 1 -- 172.21.15.70:0/3968059324 shutdown_connections 2023-10-08T07:13:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.700+0000 7f5f826c6640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7c15aad0 0x7f5f7c0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.701+0000 7f5f826c6640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f7c0c82b0 0x7f5f7c0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.701+0000 7f5f826c6640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f5f6806afc0 0x7f5f6806d480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.701+0000 7f5f826c6640 1 --2- 172.21.15.70:0/3968059324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f7c06d410 0x7f5f7c156820 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.701+0000 7f5f826c6640 1 -- 172.21.15.70:0/3968059324 >> 172.21.15.70:0/3968059324 conn(0x7f5f7c069d50 msgr2=0x7f5f7c154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.701+0000 7f5f826c6640 1 -- 172.21.15.70:0/3968059324 shutdown_connections 2023-10-08T07:13:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.701+0000 7f5f826c6640 1 -- 172.21.15.70:0/3968059324 wait complete. 2023-10-08T07:13:31.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T07:13:31.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-08T07:13:31.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-10-08T07:13:31.742 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-08T07:13:31.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.928+0000 7f1b0e90a640 1 Processor -- start 2023-10-08T07:13:31.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.929+0000 7f1b0e90a640 1 -- start start 2023-10-08T07:13:31.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.930+0000 7f1b0e90a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b080adc70 0x7f1b080b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.930+0000 7f1b0e90a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0815f7d0 con 0x7f1b0815af10 2023-10-08T07:13:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.930+0000 7f1b0e90a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0815efb0 con 0x7f1b0815ab30 2023-10-08T07:13:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.930+0000 7f1b0e90a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0815f210 con 0x7f1b080adc70 2023-10-08T07:13:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.930+0000 7f1b0e109640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1b0815af10 0x7f1b0806cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43214/0 (socket says 172.21.15.70:43214) 2023-10-08T07:13:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.930+0000 7f1b0e109640 1 -- 172.21.15.70:0/3031116194 learned_addr learned my addr 172.21.15.70:0/3031116194 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0d107640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b080adc70 0x7f1b080b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670807606 0 0) 0x7f1b0815f7d0 con 0x7f1b0815af10 2023-10-08T07:13:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1af4003520 con 0x7f1b0815af10 2023-10-08T07:13:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3694950565 0 0) 0x7f1af4003520 con 0x7f1b0815af10 2023-10-08T07:13:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b080adc70 msgr2=0x7f1b080b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b080adc70 0x7f1b080b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 >> v1:172.21.15.181:6789/0 conn(0x7f1b0815ab30 legacy=0x7f1b0815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0815f4f0 con 0x7f1b0815af10 2023-10-08T07:13:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1b00001a10 con 0x7f1b0815af10 2023-10-08T07:13:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 >> v1:172.21.15.70:6789/0 conn(0x7f1b0815af10 legacy=0x7f1b0806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1af40039a0 0x7f1af4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1af40047c0 0x7f1af4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 0x7f1af400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0815f210 con 0x7f1af4005180 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af400c050 con 0x7f1af40039a0 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af400c210 con 0x7f1af40047c0 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0d107640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b080adc70 0x7f1b080b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.931+0000 7f1b0e109640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 0x7f1af400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.932+0000 7f1b0d908640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1af40039a0 0x7f1af4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.932+0000 7f1b0d107640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1af40047c0 0x7f1af4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.932+0000 7f1b0e109640 1 -- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1af40047c0 msgr2=0x7f1af4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.932+0000 7f1b0e109640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1af40047c0 0x7f1af4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.932+0000 7f1b0e109640 1 -- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1af40039a0 msgr2=0x7f1af4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.932+0000 7f1b0e109640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1af40039a0 0x7f1af4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.932+0000 7f1b0e109640 1 -- 172.21.15.70:0/3031116194 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af4003520 con 0x7f1af4005180 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.933+0000 7f1b0e109640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 0x7f1af400b980 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f1b0815f960 tx=0x7f1af40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9359b15d7f7b7a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.933+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00034630 con 0x7f1af4005180 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.933+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0002f070 con 0x7f1af4005180 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.933+0000 7f1b0c906640 1 -- 172.21.15.70:0/3031116194 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00034990 con 0x7f1af4005180 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.934+0000 7f1b0e90a640 1 -- 172.21.15.70:0/3031116194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 msgr2=0x7f1af400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.934+0000 7f1b0e90a640 1 --2- 172.21.15.70:0/3031116194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 0x7f1af400b980 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f1b0815f960 tx=0x7f1af40057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.934+0000 7f1b0e109640 1 -- 172.21.15.70:0/3031116194 reap_dead start 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.934+0000 7f1b0e90a640 1 -- 172.21.15.70:0/3031116194 shutdown_connections 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.934+0000 7f1b0e90a640 1 -- 172.21.15.70:0/3031116194 >> 172.21.15.70:0/3031116194 conn(0x7f1b080cb0c0 msgr2=0x7f1b080cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.934+0000 7f1b0e90a640 1 -- 172.21.15.70:0/3031116194 shutdown_connections 2023-10-08T07:13:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.934+0000 7f1b0e90a640 1 -- 172.21.15.70:0/3031116194 wait complete. 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.936+0000 7f1b0e90a640 1 Processor -- start 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.936+0000 7f1b0e90a640 1 -- start start 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0e90a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 0x7f1b080cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0e90a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b080cd870 0x7f1b08156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0e90a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b080c4350 0x7f1b0815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0e90a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b080c47d0 con 0x7f1af4005180 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0e90a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b080cc7d0 con 0x7f1b080cd870 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0e90a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b080ccab0 con 0x7f1b080c4350 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0d908640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 0x7f1b080cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0d908640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 0x7f1b080cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51070/0 (socket says 172.21.15.70:51070) 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0d908640 1 -- 172.21.15.70:0/2908481135 learned_addr learned my addr 172.21.15.70:0/2908481135 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:31.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0e109640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b080c4350 0x7f1b0815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0d107640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b080cd870 0x7f1b08156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0d908640 1 -- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b080c4350 msgr2=0x7f1b0815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0d908640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b080c4350 0x7f1b0815d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0d908640 1 -- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b080cd870 msgr2=0x7f1b08156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0d908640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b080cd870 0x7f1b08156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.937+0000 7f1b0d908640 1 -- 172.21.15.70:0/2908481135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b080ccd90 con 0x7f1af4005180 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.938+0000 7f1b0d908640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 0x7f1b080cc1f0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400e550 tx=0x7f1b0400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.938+0000 7f1afa7fc640 1 -- 172.21.15.70:0/2908481135 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b040052e0 con 0x7f1af4005180 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.938+0000 7f1afa7fc640 1 -- 172.21.15.70:0/2908481135 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b04005440 con 0x7f1af4005180 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.938+0000 7f1afa7fc640 1 -- 172.21.15.70:0/2908481135 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04003810 con 0x7f1af4005180 2023-10-08T07:13:31.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.938+0000 7f1b0e90a640 1 -- 172.21.15.70:0/2908481135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b080cd070 con 0x7f1af4005180 2023-10-08T07:13:31.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.938+0000 7f1b0e90a640 1 -- 172.21.15.70:0/2908481135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b08156cf0 con 0x7f1af4005180 2023-10-08T07:13:31.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.939+0000 7f1b0e90a640 1 -- 172.21.15.70:0/2908481135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0804e160 con 0x7f1af4005180 2023-10-08T07:13:31.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.940+0000 7f1afa7fc640 1 -- 172.21.15.70:0/2908481135 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f1b04003970 con 0x7f1af4005180 2023-10-08T07:13:31.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.941+0000 7f1afa7fc640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f1aec06afc0 0x7f1aec06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:31.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.941+0000 7f1afa7fc640 1 -- 172.21.15.70:0/2908481135 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f1b04089340 con 0x7f1af4005180 2023-10-08T07:13:31.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.942+0000 7f1b0d107640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f1aec06afc0 0x7f1aec06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:31.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.943+0000 7f1b0d107640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f1aec06afc0 0x7f1aec06d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b08144350 tx=0x7f1afc008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:31.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:31.945+0000 7f1afa7fc640 1 -- 172.21.15.70:0/2908481135 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b04054480 con 0x7f1af4005180 2023-10-08T07:13:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.137+0000 7f1b0e90a640 1 -- 172.21.15.70:0/2908481135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1b080b3790 con 0x7f1af4005180 2023-10-08T07:13:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.137+0000 7f1afa7fc640 1 -- 172.21.15.70:0/2908481135 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v230) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1b04057aa0 con 0x7f1af4005180 2023-10-08T07:13:32.138 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:32.138 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T07:13:32.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:32.138 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T07:13:32.139 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T07:13:32.139 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:32.139 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T07:13:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 -- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f1aec06afc0 msgr2=0x7f1aec06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f1aec06afc0 0x7f1aec06d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b08144350 tx=0x7f1afc008040 comp rx=0 tx=0).stop 2023-10-08T07:13:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 -- 172.21.15.70:0/2908481135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 msgr2=0x7f1b080cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 0x7f1b080cc1f0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400e550 tx=0x7f1b0400ea10 comp rx=0 tx=0).stop 2023-10-08T07:13:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 -- 172.21.15.70:0/2908481135 shutdown_connections 2023-10-08T07:13:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b080c4350 0x7f1b0815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b080cd870 0x7f1b08156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f1aec06afc0 0x7f1aec06d480 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 --2- 172.21.15.70:0/2908481135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af4005180 0x7f1b080cc1f0 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 -- 172.21.15.70:0/2908481135 >> 172.21.15.70:0/2908481135 conn(0x7f1b080cb0c0 msgr2=0x7f1b0806da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 -- 172.21.15.70:0/2908481135 shutdown_connections 2023-10-08T07:13:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.140+0000 7f1b0e90a640 1 -- 172.21.15.70:0/2908481135 wait complete. 2023-10-08T07:13:32.160 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T07:13:32.160 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T07:13:32.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.376+0000 7f4fbf56e640 1 Processor -- start 2023-10-08T07:13:32.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.376+0000 7f4fbf56e640 1 -- start start 2023-10-08T07:13:32.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.378+0000 7f4fbf56e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fb815aad0 0x7f4fb80b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.378+0000 7f4fbf56e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb806d410 0x7f4fb80c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.378+0000 7f4fbf56e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fb815bf30 con 0x7f4fb80c82b0 2023-10-08T07:13:32.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.378+0000 7f4fbf56e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fb815b320 con 0x7f4fb815aad0 2023-10-08T07:13:32.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.378+0000 7f4fbf56e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb815b600 con 0x7f4fb806d410 2023-10-08T07:13:32.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.378+0000 7f4fbd30a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fb815aad0 0x7f4fb80b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:32.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.378+0000 7f4fbd30a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4fb815aad0 0x7f4fb80b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51080/0 (socket says 172.21.15.70:51080) 2023-10-08T07:13:32.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.378+0000 7f4fbd30a640 1 -- 172.21.15.70:0/405055687 learned_addr learned my addr 172.21.15.70:0/405055687 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:32.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.378+0000 7f4fbdb0b640 1 --2- 172.21.15.70:0/405055687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb806d410 0x7f4fb80c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:32.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.379+0000 7f4fbd30a640 1 -- 172.21.15.70:0/405055687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb806d410 msgr2=0x7f4fb80c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:32.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.379+0000 7f4fbd30a640 1 --2- 172.21.15.70:0/405055687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb806d410 0x7f4fb80c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.379+0000 7f4fbd30a640 1 -- 172.21.15.70:0/405055687 >> v1:172.21.15.70:6789/0 conn(0x7f4fb80c82b0 legacy=0x7f4fb806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.379+0000 7f4fbd30a640 1 -- 172.21.15.70:0/405055687 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb815b8e0 con 0x7f4fb815aad0 2023-10-08T07:13:32.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.380+0000 7f4fbd30a640 1 --2- 172.21.15.70:0/405055687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815aad0 0x7f4fb80b1750 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009870 tx=0x7f4fb402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea01c4ac8c2efbf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.380+0000 7f4faffff640 1 -- 172.21.15.70:0/405055687 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4030840 con 0x7f4fb815aad0 2023-10-08T07:13:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.380+0000 7f4faffff640 1 -- 172.21.15.70:0/405055687 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb40309a0 con 0x7f4fb815aad0 2023-10-08T07:13:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.380+0000 7f4faffff640 1 -- 172.21.15.70:0/405055687 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4030d30 con 0x7f4fb815aad0 2023-10-08T07:13:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.381+0000 7f4fbf56e640 1 -- 172.21.15.70:0/405055687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815aad0 msgr2=0x7f4fb80b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.381+0000 7f4fbf56e640 1 --2- 172.21.15.70:0/405055687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815aad0 0x7f4fb80b1750 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009870 tx=0x7f4fb402b980 comp rx=0 tx=0).stop 2023-10-08T07:13:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.381+0000 7f4fbf56e640 1 -- 172.21.15.70:0/405055687 shutdown_connections 2023-10-08T07:13:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.381+0000 7f4fbf56e640 1 --2- 172.21.15.70:0/405055687 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb806d410 0x7f4fb80c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.381+0000 7f4fbf56e640 1 --2- 172.21.15.70:0/405055687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb815aad0 0x7f4fb80b1750 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.381+0000 7f4fbf56e640 1 -- 172.21.15.70:0/405055687 >> 172.21.15.70:0/405055687 conn(0x7f4fb8069d50 msgr2=0x7f4fb806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.381+0000 7f4fbf56e640 1 -- 172.21.15.70:0/405055687 shutdown_connections 2023-10-08T07:13:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.381+0000 7f4fbf56e640 1 -- 172.21.15.70:0/405055687 wait complete. 2023-10-08T07:13:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.383+0000 7f4fbf56e640 1 Processor -- start 2023-10-08T07:13:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.383+0000 7f4fbf56e640 1 -- start start 2023-10-08T07:13:32.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbf56e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb806d410 0x7f4fb8151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbf56e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb80c82b0 0x7f4fb814f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbf56e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb815aad0 0x7f4fb81f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbf56e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb8150be0 con 0x7f4fb806d410 2023-10-08T07:13:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbf56e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb814ffd0 con 0x7f4fb80c82b0 2023-10-08T07:13:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbf56e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb81502b0 con 0x7f4fb815aad0 2023-10-08T07:13:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbd30a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb806d410 0x7f4fb8151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbd30a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb806d410 0x7f4fb8151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51086/0 (socket says 172.21.15.70:51086) 2023-10-08T07:13:32.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbd30a640 1 -- 172.21.15.70:0/225887230 learned_addr learned my addr 172.21.15.70:0/225887230 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbd30a640 1 -- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb815aad0 msgr2=0x7f4fb81f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbd30a640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb815aad0 0x7f4fb81f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbd30a640 1 -- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb80c82b0 msgr2=0x7f4fb814f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbd30a640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb80c82b0 0x7f4fb814f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.384+0000 7f4fbd30a640 1 -- 172.21.15.70:0/225887230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb40094d0 con 0x7f4fb806d410 2023-10-08T07:13:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.385+0000 7f4fbd30a640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb806d410 0x7f4fb8151070 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4000c00 tx=0x7f4fb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.385+0000 7f4fadffb640 1 -- 172.21.15.70:0/225887230 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4030810 con 0x7f4fb806d410 2023-10-08T07:13:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.385+0000 7f4fadffb640 1 -- 172.21.15.70:0/225887230 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb4043400 con 0x7f4fb806d410 2023-10-08T07:13:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.385+0000 7f4fadffb640 1 -- 172.21.15.70:0/225887230 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40437d0 con 0x7f4fb806d410 2023-10-08T07:13:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.385+0000 7f4fbf56e640 1 -- 172.21.15.70:0/225887230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fb8150590 con 0x7f4fb806d410 2023-10-08T07:13:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.385+0000 7f4fbf56e640 1 -- 172.21.15.70:0/225887230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fb81f2170 con 0x7f4fb806d410 2023-10-08T07:13:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.386+0000 7f4fbf56e640 1 -- 172.21.15.70:0/225887230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fb804e160 con 0x7f4fb806d410 2023-10-08T07:13:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.387+0000 7f4fadffb640 1 -- 172.21.15.70:0/225887230 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4fb4043930 con 0x7f4fb806d410 2023-10-08T07:13:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.388+0000 7f4fadffb640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06af90 0x7f4f9c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.388+0000 7f4fadffb640 1 -- 172.21.15.70:0/225887230 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f4fb403e070 con 0x7f4fb806d410 2023-10-08T07:13:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.389+0000 7f4fbcb09640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06af90 0x7f4f9c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.390+0000 7f4fbcb09640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06af90 0x7f4f9c06d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8000f60 tx=0x7f4fa80079b0 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.393+0000 7f4fadffb640 1 -- 172.21.15.70:0/225887230 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fb4077aa0 con 0x7f4fb806d410 2023-10-08T07:13:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.582+0000 7f4fbf56e640 1 -- 172.21.15.70:0/225887230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4fb8155c50 con 0x7f4fb806d410 2023-10-08T07:13:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.692+0000 7f4fadffb640 1 -- 172.21.15.70:0/225887230 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v231) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4fb407b0c0 con 0x7f4fb806d410 2023-10-08T07:13:32.693 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T07:13:32.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.695+0000 7f4fbf56e640 1 -- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06af90 msgr2=0x7f4f9c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.695+0000 7f4fbf56e640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06af90 0x7f4f9c06d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8000f60 tx=0x7f4fa80079b0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.695+0000 7f4fbf56e640 1 -- 172.21.15.70:0/225887230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb806d410 msgr2=0x7f4fb8151070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.695+0000 7f4fbf56e640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb806d410 0x7f4fb8151070 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4000c00 tx=0x7f4fb4004730 comp rx=0 tx=0).stop 2023-10-08T07:13:32.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.696+0000 7f4fbf56e640 1 -- 172.21.15.70:0/225887230 shutdown_connections 2023-10-08T07:13:32.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.696+0000 7f4fbf56e640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fb815aad0 0x7f4fb81f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.696+0000 7f4fbf56e640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fb80c82b0 0x7f4fb814f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.696+0000 7f4fbf56e640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06af90 0x7f4f9c06d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.696+0000 7f4fbf56e640 1 --2- 172.21.15.70:0/225887230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fb806d410 0x7f4fb8151070 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.696+0000 7f4fbf56e640 1 -- 172.21.15.70:0/225887230 >> 172.21.15.70:0/225887230 conn(0x7f4fb8069d50 msgr2=0x7f4fb806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:32.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.696+0000 7f4fbf56e640 1 -- 172.21.15.70:0/225887230 shutdown_connections 2023-10-08T07:13:32.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.696+0000 7f4fbf56e640 1 -- 172.21.15.70:0/225887230 wait complete. 2023-10-08T07:13:32.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T07:13:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.927+0000 7ff79c952640 1 Processor -- start 2023-10-08T07:13:32.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.927+0000 7ff79c952640 1 -- start start 2023-10-08T07:13:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.928+0000 7ff79c952640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff79815cb40 0x7ff79806d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.928+0000 7ff79c952640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7980cb970 0x7ff7980b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.928+0000 7ff79c952640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7980cd2e0 con 0x7ff79806da60 2023-10-08T07:13:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.928+0000 7ff79c952640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79815cf20 con 0x7ff7980cb970 2023-10-08T07:13:32.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.928+0000 7ff79c952640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cd510 con 0x7ff79815cb40 2023-10-08T07:13:32.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.929+0000 7ff796d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff79815cb40 0x7ff79806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:32.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.929+0000 7ff79759e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff7980cb970 0x7ff7980b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.929+0000 7ff796d9d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff79815cb40 0x7ff79806d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:48822/0 (socket says 172.21.15.70:48822) 2023-10-08T07:13:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.929+0000 7ff796d9d640 1 -- 172.21.15.70:0/3568951297 learned_addr learned my addr 172.21.15.70:0/3568951297 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.930+0000 7ff796d9d640 1 -- 172.21.15.70:0/3568951297 >> v2:172.21.15.181:3300/0 conn(0x7ff7980cb970 msgr2=0x7ff7980b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.930+0000 7ff796d9d640 1 --2- 172.21.15.70:0/3568951297 >> v2:172.21.15.181:3300/0 conn(0x7ff7980cb970 0x7ff7980b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.930+0000 7ff796d9d640 1 -- 172.21.15.70:0/3568951297 >> v1:172.21.15.181:6789/0 conn(0x7ff79806da60 legacy=0x7ff7980cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.930+0000 7ff796d9d640 1 -- 172.21.15.70:0/3568951297 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980ce120 con 0x7ff79815cb40 2023-10-08T07:13:32.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.932+0000 7ff796d9d640 1 --2- 172.21.15.70:0/3568951297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff79815cb40 0x7ff79806d030 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff7880098a0 tx=0x7ff78802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7829033a3883a1e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.932+0000 7ff795d9b640 1 -- 172.21.15.70:0/3568951297 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788030840 con 0x7ff79815cb40 2023-10-08T07:13:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.932+0000 7ff795d9b640 1 -- 172.21.15.70:0/3568951297 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7880309a0 con 0x7ff79815cb40 2023-10-08T07:13:32.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.932+0000 7ff795d9b640 1 -- 172.21.15.70:0/3568951297 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788030d30 con 0x7ff79815cb40 2023-10-08T07:13:32.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.933+0000 7ff79c952640 1 -- 172.21.15.70:0/3568951297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff79815cb40 msgr2=0x7ff79806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.933+0000 7ff79c952640 1 --2- 172.21.15.70:0/3568951297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff79815cb40 0x7ff79806d030 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff7880098a0 tx=0x7ff78802b980 comp rx=0 tx=0).stop 2023-10-08T07:13:32.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.933+0000 7ff79c952640 1 -- 172.21.15.70:0/3568951297 shutdown_connections 2023-10-08T07:13:32.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.933+0000 7ff79c952640 1 --2- 172.21.15.70:0/3568951297 >> v2:172.21.15.181:3300/0 conn(0x7ff7980cb970 0x7ff7980b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.933+0000 7ff79c952640 1 --2- 172.21.15.70:0/3568951297 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff79815cb40 0x7ff79806d030 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.933+0000 7ff79c952640 1 -- 172.21.15.70:0/3568951297 >> 172.21.15.70:0/3568951297 conn(0x7ff7980ce9f0 msgr2=0x7ff7980cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:32.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.933+0000 7ff79c952640 1 -- 172.21.15.70:0/3568951297 shutdown_connections 2023-10-08T07:13:32.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.933+0000 7ff79c952640 1 -- 172.21.15.70:0/3568951297 wait complete. 2023-10-08T07:13:32.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.935+0000 7ff79c952640 1 Processor -- start 2023-10-08T07:13:32.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.935+0000 7ff79c952640 1 -- start start 2023-10-08T07:13:32.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.936+0000 7ff79c952640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff79806da60 0x7ff7980c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.936+0000 7ff79c952640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7980cb970 0x7ff798154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.936+0000 7ff79c952640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79815cb40 0x7ff79815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.936+0000 7ff79c952640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff798154a90 con 0x7ff79806da60 2023-10-08T07:13:32.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.936+0000 7ff79c952640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980c7010 con 0x7ff79815cb40 2023-10-08T07:13:32.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.936+0000 7ff79c952640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980c6400 con 0x7ff7980cb970 2023-10-08T07:13:32.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.936+0000 7ff796d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff79806da60 0x7ff7980c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:32.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.936+0000 7ff796d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff79806da60 0x7ff7980c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51100/0 (socket says 172.21.15.70:51100) 2023-10-08T07:13:32.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.936+0000 7ff796d9d640 1 -- 172.21.15.70:0/3533996674 learned_addr learned my addr 172.21.15.70:0/3533996674 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:32.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff79759e640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79815cb40 0x7ff79815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:32.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff79659c640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7980cb970 0x7ff798154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:32.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff796d9d640 1 -- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7980cb970 msgr2=0x7ff798154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff796d9d640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7980cb970 0x7ff798154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff796d9d640 1 -- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79815cb40 msgr2=0x7ff79815abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:32.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff796d9d640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79815cb40 0x7ff79815abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:32.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff796d9d640 1 -- 172.21.15.70:0/3533996674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7880094d0 con 0x7ff79806da60 2023-10-08T07:13:32.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff796d9d640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff79806da60 0x7ff7980c74a0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7ff788000c00 tx=0x7ff7880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:32.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff77f7fe640 1 -- 172.21.15.70:0/3533996674 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78803e070 con 0x7ff79806da60 2023-10-08T07:13:32.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff77f7fe640 1 -- 172.21.15.70:0/3533996674 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff788039a10 con 0x7ff79806da60 2023-10-08T07:13:32.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff77f7fe640 1 -- 172.21.15.70:0/3533996674 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788039dc0 con 0x7ff79806da60 2023-10-08T07:13:32.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.937+0000 7ff79c952640 1 -- 172.21.15.70:0/3533996674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7980c66e0 con 0x7ff79806da60 2023-10-08T07:13:32.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.938+0000 7ff79c952640 1 -- 172.21.15.70:0/3533996674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7980c6ca0 con 0x7ff79806da60 2023-10-08T07:13:32.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.939+0000 7ff79c952640 1 -- 172.21.15.70:0/3533996674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79804e160 con 0x7ff79806da60 2023-10-08T07:13:32.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.941+0000 7ff77f7fe640 1 -- 172.21.15.70:0/3533996674 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff788055430 con 0x7ff79806da60 2023-10-08T07:13:32.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.941+0000 7ff77f7fe640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff78406afc0 0x7ff78406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:32.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.941+0000 7ff77f7fe640 1 -- 172.21.15.70:0/3533996674 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 5663+0+0 (secure 0 0 0) 0x7ff7880ac8d0 con 0x7ff79806da60 2023-10-08T07:13:32.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.942+0000 7ff79659c640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff78406afc0 0x7ff78406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:32.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.943+0000 7ff79659c640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff78406afc0 0x7ff78406d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff79815be70 tx=0x7ff78c009040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:32.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:32.945+0000 7ff77f7fe640 1 -- 172.21.15.70:0/3533996674 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff788077770 con 0x7ff79806da60 2023-10-08T07:13:33.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.131+0000 7ff79c952640 1 -- 172.21.15.70:0/3533996674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff79806cbe0 con 0x7ff79806da60 2023-10-08T07:13:33.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.704+0000 7ff77f7fe640 1 -- 172.21.15.70:0/3533996674 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v232) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff7880340d0 con 0x7ff79806da60 2023-10-08T07:13:33.705 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T07:13:33.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.707+0000 7ff79c952640 1 -- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff78406afc0 msgr2=0x7ff78406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:33.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.707+0000 7ff79c952640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff78406afc0 0x7ff78406d480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff79815be70 tx=0x7ff78c009040 comp rx=0 tx=0).stop 2023-10-08T07:13:33.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.708+0000 7ff79c952640 1 -- 172.21.15.70:0/3533996674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff79806da60 msgr2=0x7ff7980c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.708+0000 7ff79c952640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff79806da60 0x7ff7980c74a0 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7ff788000c00 tx=0x7ff7880046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.708+0000 7ff79c952640 1 -- 172.21.15.70:0/3533996674 shutdown_connections 2023-10-08T07:13:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.708+0000 7ff79c952640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff79815cb40 0x7ff79815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.708+0000 7ff79c952640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff7980cb970 0x7ff798154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.708+0000 7ff79c952640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff78406afc0 0x7ff78406d480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.708+0000 7ff79c952640 1 --2- 172.21.15.70:0/3533996674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff79806da60 0x7ff7980c74a0 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.708+0000 7ff79c952640 1 -- 172.21.15.70:0/3533996674 >> 172.21.15.70:0/3533996674 conn(0x7ff7980ce9f0 msgr2=0x7ff7980c8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.708+0000 7ff79c952640 1 -- 172.21.15.70:0/3533996674 shutdown_connections 2023-10-08T07:13:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.708+0000 7ff79c952640 1 -- 172.21.15.70:0/3533996674 wait complete. 2023-10-08T07:13:33.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T07:13:33.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.932+0000 7fcb8b59e640 1 Processor -- start 2023-10-08T07:13:33.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.933+0000 7fcb8b59e640 1 -- start start 2023-10-08T07:13:33.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.934+0000 7fcb8b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0b3c50 0x7fcb8c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:33.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.934+0000 7fcb8b59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c159db0 con 0x7fcb8c0cb5e0 2023-10-08T07:13:33.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.934+0000 7fcb8b59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c071fe0 con 0x7fcb8c1599d0 2023-10-08T07:13:33.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.934+0000 7fcb8b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0b44c0 con 0x7fcb8c0b3c50 2023-10-08T07:13:33.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.934+0000 7fcb89d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcb8c0cb5e0 0x7fcb8c0cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39482/0 (socket says 172.21.15.70:39482) 2023-10-08T07:13:33.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.934+0000 7fcb89d9b640 1 -- 172.21.15.70:0/2545798861 learned_addr learned my addr 172.21.15.70:0/2545798861 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:33.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8ad9d640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0b3c50 0x7fcb8c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:33.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027834797 0 0) 0x7fcb74001110 con 0x7fcb8c0cb5e0 2023-10-08T07:13:33.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb78003520 con 0x7fcb8c0cb5e0 2023-10-08T07:13:33.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1067006443 0 0) 0x7fcb78003520 con 0x7fcb8c0cb5e0 2023-10-08T07:13:33.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0b3c50 msgr2=0x7fcb8c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:33.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0b3c50 0x7fcb8c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 >> v1:172.21.15.181:6789/0 conn(0x7fcb8c1599d0 legacy=0x7fcb8c0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:33.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0713d0 con 0x7fcb8c0cb5e0 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fcb74001800 con 0x7fcb8c0cb5e0 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 >> v1:172.21.15.70:6789/0 conn(0x7fcb8c0cb5e0 legacy=0x7fcb8c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb780039a0 0x7fcb78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb780047c0 0x7fcb78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0b3c50 0x7fcb78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0b44c0 con 0x7fcb780039a0 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7800ba30 con 0x7fcb780047c0 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7800bbf0 con 0x7fcb8c0b3c50 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.935+0000 7fcb8a59c640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb780039a0 0x7fcb78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.936+0000 7fcb89d9b640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb780047c0 0x7fcb78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.936+0000 7fcb8a59c640 1 -- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0b3c50 msgr2=0x7fcb78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:33.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.936+0000 7fcb8ad9d640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0b3c50 0x7fcb78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:33.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.936+0000 7fcb8a59c640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb8c0b3c50 0x7fcb78005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.936+0000 7fcb8a59c640 1 -- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb780047c0 msgr2=0x7fcb78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:33.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.936+0000 7fcb8a59c640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb780047c0 0x7fcb78004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.936+0000 7fcb8a59c640 1 -- 172.21.15.70:0/2545798861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb78003520 con 0x7fcb780039a0 2023-10-08T07:13:33.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.936+0000 7fcb89d9b640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb780047c0 0x7fcb78004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:33.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.937+0000 7fcb8a59c640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb780039a0 0x7fcb78003dc0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c001e20 tx=0x7fcb780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0919ddaede22925 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:33.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.937+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c003b70 con 0x7fcb780039a0 2023-10-08T07:13:33.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.937+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c003cd0 con 0x7fcb780039a0 2023-10-08T07:13:33.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.937+0000 7fcb8959a640 1 -- 172.21.15.70:0/2545798861 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c005310 con 0x7fcb780039a0 2023-10-08T07:13:33.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.938+0000 7fcb8b59e640 1 -- 172.21.15.70:0/2545798861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb780039a0 msgr2=0x7fcb78003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:33.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.938+0000 7fcb8b59e640 1 --2- 172.21.15.70:0/2545798861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb780039a0 0x7fcb78003dc0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c001e20 tx=0x7fcb780057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.938+0000 7fcb8ad9d640 1 -- 172.21.15.70:0/2545798861 reap_dead start 2023-10-08T07:13:33.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.938+0000 7fcb8b59e640 1 -- 172.21.15.70:0/2545798861 shutdown_connections 2023-10-08T07:13:33.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.938+0000 7fcb8b59e640 1 -- 172.21.15.70:0/2545798861 >> 172.21.15.70:0/2545798861 conn(0x7fcb8c06c050 msgr2=0x7fcb8c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:33.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.938+0000 7fcb8b59e640 1 -- 172.21.15.70:0/2545798861 shutdown_connections 2023-10-08T07:13:33.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.938+0000 7fcb8b59e640 1 -- 172.21.15.70:0/2545798861 wait complete. 2023-10-08T07:13:33.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.940+0000 7fcb8b59e640 1 Processor -- start 2023-10-08T07:13:33.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.940+0000 7fcb8b59e640 1 -- start start 2023-10-08T07:13:33.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.941+0000 7fcb8b59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb780039a0 0x7fcb8c1f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:33.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.941+0000 7fcb8b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb8c1546c0 0x7fcb8c1ed4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:33.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.941+0000 7fcb8b59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb8c154b20 0x7fcb8c1eda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:33.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.941+0000 7fcb8b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1edf70 con 0x7fcb8c1546c0 2023-10-08T07:13:33.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.941+0000 7fcb8b59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1f2560 con 0x7fcb8c154b20 2023-10-08T07:13:33.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.941+0000 7fcb8b59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c14e870 con 0x7fcb780039a0 2023-10-08T07:13:33.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.941+0000 7fcb89d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb8c1546c0 0x7fcb8c1ed4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:33.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.941+0000 7fcb89d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb8c1546c0 0x7fcb8c1ed4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36950/0 (socket says 172.21.15.70:36950) 2023-10-08T07:13:33.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.941+0000 7fcb89d9b640 1 -- 172.21.15.70:0/3401002674 learned_addr learned my addr 172.21.15.70:0/3401002674 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:33.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.941+0000 7fcb8a59c640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb780039a0 0x7fcb8c1f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:33.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb8ad9d640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb8c154b20 0x7fcb8c1eda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:33.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb89d9b640 1 -- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb780039a0 msgr2=0x7fcb8c1f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:33.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb89d9b640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb780039a0 0x7fcb8c1f2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb89d9b640 1 -- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb8c154b20 msgr2=0x7fcb8c1eda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:33.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb89d9b640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb8c154b20 0x7fcb8c1eda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:33.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb89d9b640 1 -- 172.21.15.70:0/3401002674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c14eb50 con 0x7fcb8c1546c0 2023-10-08T07:13:33.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb89d9b640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb8c1546c0 0x7fcb8c1ed4f0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcb74000ff0 tx=0x7fcb7400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:33.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb82ffd640 1 -- 172.21.15.70:0/3401002674 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb740074e0 con 0x7fcb8c1546c0 2023-10-08T07:13:33.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb82ffd640 1 -- 172.21.15.70:0/3401002674 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb74007640 con 0x7fcb8c1546c0 2023-10-08T07:13:33.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb8b59e640 1 -- 172.21.15.70:0/3401002674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c14ee30 con 0x7fcb8c1546c0 2023-10-08T07:13:33.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb82ffd640 1 -- 172.21.15.70:0/3401002674 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb740049e0 con 0x7fcb8c1546c0 2023-10-08T07:13:33.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.942+0000 7fcb8b59e640 1 -- 172.21.15.70:0/3401002674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c14f3f0 con 0x7fcb8c1546c0 2023-10-08T07:13:33.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.944+0000 7fcb8b59e640 1 -- 172.21.15.70:0/3401002674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c074150 con 0x7fcb8c1546c0 2023-10-08T07:13:33.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.945+0000 7fcb82ffd640 1 -- 172.21.15.70:0/3401002674 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcb74004b40 con 0x7fcb8c1546c0 2023-10-08T07:13:33.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.946+0000 7fcb82ffd640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fcb6c06af90 0x7fcb6c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:33.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.946+0000 7fcb82ffd640 1 -- 172.21.15.70:0/3401002674 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 6045+0+0 (secure 0 0 0) 0x7fcb740892e0 con 0x7fcb8c1546c0 2023-10-08T07:13:33.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.947+0000 7fcb8a59c640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fcb6c06af90 0x7fcb6c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:33.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.948+0000 7fcb8a59c640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fcb6c06af90 0x7fcb6c06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c034000 tx=0x7fcb7c039040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:33.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:33.951+0000 7fcb82ffd640 1 -- 172.21.15.70:0/3401002674 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb74057a10 con 0x7fcb8c1546c0 2023-10-08T07:13:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.167+0000 7fcb8b59e640 1 -- 172.21.15.70:0/3401002674 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcb8c159db0 con 0x7fcb8c1546c0 2023-10-08T07:13:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.707+0000 7fcb82ffd640 1 -- 172.21.15.70:0/3401002674 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v104) v1 ==== 174+0+0 (secure 0 0 0) 0x7fcb8c159db0 con 0x7fcb8c1546c0 2023-10-08T07:13:34.708 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 27 and data pool 28 2023-10-08T07:13:34.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 -- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fcb6c06af90 msgr2=0x7fcb6c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:34.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fcb6c06af90 0x7fcb6c06d450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c034000 tx=0x7fcb7c039040 comp rx=0 tx=0).stop 2023-10-08T07:13:34.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 -- 172.21.15.70:0/3401002674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb8c1546c0 msgr2=0x7fcb8c1ed4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:34.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb8c1546c0 0x7fcb8c1ed4f0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fcb74000ff0 tx=0x7fcb7400dd20 comp rx=0 tx=0).stop 2023-10-08T07:13:34.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 -- 172.21.15.70:0/3401002674 shutdown_connections 2023-10-08T07:13:34.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcb8c154b20 0x7fcb8c1eda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fcb6c06af90 0x7fcb6c06d450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb8c1546c0 0x7fcb8c1ed4f0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 --2- 172.21.15.70:0/3401002674 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcb780039a0 0x7fcb8c1f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 -- 172.21.15.70:0/3401002674 >> 172.21.15.70:0/3401002674 conn(0x7fcb8c06c050 msgr2=0x7fcb8c0c2400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:34.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.711+0000 7fcb8b59e640 1 -- 172.21.15.70:0/3401002674 shutdown_connections 2023-10-08T07:13:34.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.712+0000 7fcb8b59e640 1 -- 172.21.15.70:0/3401002674 wait complete. 2023-10-08T07:13:34.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.921+0000 7f376501a640 1 Processor -- start 2023-10-08T07:13:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.921+0000 7f376501a640 1 -- start start 2023-10-08T07:13:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f376501a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f376015ab30 0x7f376015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f376501a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f37600adc70 0x7f37600b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:34.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f376501a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f376015af10 0x7f37600b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:34.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f376501a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f376015f7d0 con 0x7f376015af10 2023-10-08T07:13:34.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f376501a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f376015f210 con 0x7f37600adc70 2023-10-08T07:13:34.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f376501a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376015f4f0 con 0x7f376015ab30 2023-10-08T07:13:34.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f3764819640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f376015af10 0x7f37600b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:34.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f3764819640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f376015af10 0x7f37600b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36962/0 (socket says 172.21.15.70:36962) 2023-10-08T07:13:34.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f3764819640 1 -- 172.21.15.70:0/2380111688 learned_addr learned my addr 172.21.15.70:0/2380111688 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:34.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f375ffff640 1 --2- 172.21.15.70:0/2380111688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f376015ab30 0x7f376015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:34.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f375f7fe640 1 --2- 172.21.15.70:0/2380111688 >> v2:172.21.15.181:3300/0 conn(0x7f37600adc70 0x7f37600b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:34.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f3764819640 1 -- 172.21.15.70:0/2380111688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f376015ab30 msgr2=0x7f376015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:34.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.923+0000 7f3764819640 1 --2- 172.21.15.70:0/2380111688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f376015ab30 0x7f376015e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.924+0000 7f3764819640 1 -- 172.21.15.70:0/2380111688 >> v2:172.21.15.181:3300/0 conn(0x7f37600adc70 msgr2=0x7f37600b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:34.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.924+0000 7f3764819640 1 --2- 172.21.15.70:0/2380111688 >> v2:172.21.15.181:3300/0 conn(0x7f37600adc70 0x7f37600b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.924+0000 7f3764819640 1 -- 172.21.15.70:0/2380111688 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376006c550 con 0x7f376015af10 2023-10-08T07:13:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.925+0000 7f3764819640 1 --2- 172.21.15.70:0/2380111688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 0x7f37600b0600 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f37600cf740 tx=0x7f375802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55da2f6cf757db2b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.925+0000 7f375effd640 1 -- 172.21.15.70:0/2380111688 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375802ca70 con 0x7f376015af10 2023-10-08T07:13:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.925+0000 7f375effd640 1 -- 172.21.15.70:0/2380111688 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375802cbd0 con 0x7f376015af10 2023-10-08T07:13:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.925+0000 7f375effd640 1 -- 172.21.15.70:0/2380111688 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37580343f0 con 0x7f376015af10 2023-10-08T07:13:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.925+0000 7f376501a640 1 -- 172.21.15.70:0/2380111688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 msgr2=0x7f37600b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.925+0000 7f376501a640 1 --2- 172.21.15.70:0/2380111688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 0x7f37600b0600 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f37600cf740 tx=0x7f375802bed0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.926+0000 7f376501a640 1 -- 172.21.15.70:0/2380111688 shutdown_connections 2023-10-08T07:13:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.926+0000 7f376501a640 1 --2- 172.21.15.70:0/2380111688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 0x7f37600b0600 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.926+0000 7f376501a640 1 --2- 172.21.15.70:0/2380111688 >> v2:172.21.15.181:3300/0 conn(0x7f37600adc70 0x7f37600b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.926+0000 7f376501a640 1 --2- 172.21.15.70:0/2380111688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f376015ab30 0x7f376015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.926+0000 7f376501a640 1 -- 172.21.15.70:0/2380111688 >> 172.21.15.70:0/2380111688 conn(0x7f37600cb0c0 msgr2=0x7f37600cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.926+0000 7f376501a640 1 -- 172.21.15.70:0/2380111688 shutdown_connections 2023-10-08T07:13:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.926+0000 7f376501a640 1 -- 172.21.15.70:0/2380111688 wait complete. 2023-10-08T07:13:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.927+0000 7f376501a640 1 Processor -- start 2023-10-08T07:13:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.927+0000 7f376501a640 1 -- start start 2023-10-08T07:13:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.928+0000 7f376501a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37600adc70 0x7f37600af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.928+0000 7f376501a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f376015ab30 0x7f3760149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.928+0000 7f376501a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 0x7f3760150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.928+0000 7f376501a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37600afd70 con 0x7f376015af10 2023-10-08T07:13:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.928+0000 7f376501a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376014a040 con 0x7f376015ab30 2023-10-08T07:13:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.928+0000 7f376501a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37600b0010 con 0x7f37600adc70 2023-10-08T07:13:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f3764819640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 0x7f3760150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f3764819640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 0x7f3760150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36966/0 (socket says 172.21.15.70:36966) 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f3764819640 1 -- 172.21.15.70:0/3301379382 learned_addr learned my addr 172.21.15.70:0/3301379382 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f375ffff640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37600adc70 0x7f37600af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f3764819640 1 -- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37600adc70 msgr2=0x7f37600af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f3764819640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37600adc70 0x7f37600af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f375f7fe640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f376015ab30 0x7f3760149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f3764819640 1 -- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f376015ab30 msgr2=0x7f3760149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f3764819640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f376015ab30 0x7f3760149b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f3764819640 1 -- 172.21.15.70:0/3301379382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3758009b30 con 0x7f376015af10 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f375ffff640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37600adc70 0x7f37600af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.929+0000 7f3764819640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 0x7f3760150260 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f3758036040 tx=0x7f3758004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.930+0000 7f375cff9640 1 -- 172.21.15.70:0/3301379382 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758009510 con 0x7f376015af10 2023-10-08T07:13:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.930+0000 7f375cff9640 1 -- 172.21.15.70:0/3301379382 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375803e450 con 0x7f376015af10 2023-10-08T07:13:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.930+0000 7f375cff9640 1 -- 172.21.15.70:0/3301379382 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375803e800 con 0x7f376015af10 2023-10-08T07:13:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.930+0000 7f376501a640 1 -- 172.21.15.70:0/3301379382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37600b02f0 con 0x7f376015af10 2023-10-08T07:13:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.930+0000 7f376501a640 1 -- 172.21.15.70:0/3301379382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376014fa60 con 0x7f376015af10 2023-10-08T07:13:34.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.931+0000 7f376501a640 1 -- 172.21.15.70:0/3301379382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f376004e160 con 0x7f376015af10 2023-10-08T07:13:34.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.932+0000 7f375cff9640 1 -- 172.21.15.70:0/3301379382 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f375803e960 con 0x7f376015af10 2023-10-08T07:13:34.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.933+0000 7f375cff9640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f374c06af60 0x7f374c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:34.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.933+0000 7f375cff9640 1 -- 172.21.15.70:0/3301379382 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f37580ac810 con 0x7f376015af10 2023-10-08T07:13:34.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.933+0000 7f375ffff640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f374c06af60 0x7f374c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:34.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.935+0000 7f375ffff640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f374c06af60 0x7f374c06d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3750008890 tx=0x7f3750008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:34.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:34.938+0000 7f375cff9640 1 -- 172.21.15.70:0/3301379382 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37580776a0 con 0x7f376015af10 2023-10-08T07:13:35.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.128+0000 7f376501a640 1 -- 172.21.15.70:0/3301379382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37600ada50 con 0x7f376015af10 2023-10-08T07:13:35.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.130+0000 7f375cff9640 1 -- 172.21.15.70:0/3301379382 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) v1 ==== 74+0+16971 (secure 0 0 0) 0x7f375807acc0 con 0x7f376015af10 2023-10-08T07:13:35.132 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:35.132 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":233,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:34.169122+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 -- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f374c06af60 msgr2=0x7f374c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f374c06af60 0x7f374c06d420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3750008890 tx=0x7f3750008040 comp rx=0 tx=0).stop 2023-10-08T07:13:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 -- 172.21.15.70:0/3301379382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 msgr2=0x7f3760150260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 0x7f3760150260 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f3758036040 tx=0x7f3758004a40 comp rx=0 tx=0).stop 2023-10-08T07:13:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 -- 172.21.15.70:0/3301379382 shutdown_connections 2023-10-08T07:13:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f374c06af60 0x7f374c06d420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f376015af10 0x7f3760150260 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f376015ab30 0x7f3760149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 --2- 172.21.15.70:0/3301379382 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f37600adc70 0x7f37600af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 -- 172.21.15.70:0/3301379382 >> 172.21.15.70:0/3301379382 conn(0x7f37600cb0c0 msgr2=0x7f3760070190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:35.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 -- 172.21.15.70:0/3301379382 shutdown_connections 2023-10-08T07:13:35.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.133+0000 7f376501a640 1 -- 172.21.15.70:0/3301379382 wait complete. 2023-10-08T07:13:35.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.364+0000 7f4a0a609640 1 Processor -- start 2023-10-08T07:13:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.364+0000 7f4a0a609640 1 -- start start 2023-10-08T07:13:35.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.366+0000 7f4a0a609640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4a040c90b0 0x7f4a040cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.366+0000 7f4a0a609640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040cb3b0 0x7f4a0415e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.366+0000 7f4a0a609640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a0415b2b0 con 0x7f4a0415ab40 2023-10-08T07:13:35.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.366+0000 7f4a0a609640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a0415bf30 con 0x7f4a040c90b0 2023-10-08T07:13:35.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.366+0000 7f4a0a609640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040cb820 con 0x7f4a040cb3b0 2023-10-08T07:13:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.366+0000 7f4a03fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4a0415ab40 0x7f4a040ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39494/0 (socket says 172.21.15.70:39494) 2023-10-08T07:13:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.366+0000 7f4a03fff640 1 -- 172.21.15.70:0/1126564377 learned_addr learned my addr 172.21.15.70:0/1126564377 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a037fe640 1 --2- 172.21.15.70:0/1126564377 >> v2:172.21.15.181:3300/0 conn(0x7f4a040c90b0 0x7f4a040cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a08ba6640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040cb3b0 0x7f4a0415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589269561 0 0) 0x7f4a0415b2b0 con 0x7f4a0415ab40 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ec003520 con 0x7f4a0415ab40 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699515337 0 0) 0x7f49ec003520 con 0x7f4a0415ab40 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040cb3b0 msgr2=0x7f4a0415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040cb3b0 0x7f4a0415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 >> v2:172.21.15.181:3300/0 conn(0x7f4a040c90b0 msgr2=0x7f4a040cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 --2- 172.21.15.70:0/1126564377 >> v2:172.21.15.181:3300/0 conn(0x7f4a040c90b0 0x7f4a040cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0415b590 con 0x7f4a0415ab40 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f49f4001690 con 0x7f4a0415ab40 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 >> v1:172.21.15.70:6789/0 conn(0x7f4a0415ab40 legacy=0x7f4a040ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.367+0000 7f4a02ffd640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ec0039a0 0x7f49ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a02ffd640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040cb3b0 0x7f49ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a02ffd640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0415bf30 con 0x7f49ec004d00 2023-10-08T07:13:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040cb820 con 0x7f49ec0039a0 2023-10-08T07:13:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec00b980 con 0x7f4a040cb3b0 2023-10-08T07:13:35.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a08ba6640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a03fff640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ec0039a0 0x7f49ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a08ba6640 1 -- 172.21.15.70:0/1126564377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040cb3b0 msgr2=0x7f49ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:35.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a08ba6640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040cb3b0 0x7f49ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a08ba6640 1 -- 172.21.15.70:0/1126564377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ec0039a0 msgr2=0x7f49ec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a08ba6640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ec0039a0 0x7f49ec003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.368+0000 7f4a08ba6640 1 -- 172.21.15.70:0/1126564377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec003520 con 0x7f49ec004d00 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.369+0000 7f4a08ba6640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009fc0 tx=0x7f49ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd71c627fb4d95ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.370+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc02cc40 con 0x7f49ec004d00 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.370+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc00f040 con 0x7f49ec004d00 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.370+0000 7f4a02ffd640 1 -- 172.21.15.70:0/1126564377 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc034440 con 0x7f49ec004d00 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.370+0000 7f4a0a609640 1 -- 172.21.15.70:0/1126564377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec004d00 msgr2=0x7f49ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.370+0000 7f4a0a609640 1 --2- 172.21.15.70:0/1126564377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49ec004d00 0x7f49ec005180 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009fc0 tx=0x7f49ec0057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.370+0000 7f4a08ba6640 1 -- 172.21.15.70:0/1126564377 reap_dead start 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.370+0000 7f4a0a609640 1 -- 172.21.15.70:0/1126564377 shutdown_connections 2023-10-08T07:13:35.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.370+0000 7f4a0a609640 1 -- 172.21.15.70:0/1126564377 >> 172.21.15.70:0/1126564377 conn(0x7f4a0406a380 msgr2=0x7f4a040b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:35.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.370+0000 7f4a0a609640 1 -- 172.21.15.70:0/1126564377 shutdown_connections 2023-10-08T07:13:35.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.370+0000 7f4a0a609640 1 -- 172.21.15.70:0/1126564377 wait complete. 2023-10-08T07:13:35.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.372+0000 7f4a0a609640 1 Processor -- start 2023-10-08T07:13:35.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.372+0000 7f4a0a609640 1 -- start start 2023-10-08T07:13:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.373+0000 7f4a0a609640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ec004d00 0x7f4a040c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.373+0000 7f4a0a609640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040b1480 0x7f4a040b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.373+0000 7f4a0a609640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a040b3710 0x7f4a040b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.373+0000 7f4a0a609640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040b2340 con 0x7f4a040b3710 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.373+0000 7f4a0a609640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040c5850 con 0x7f49ec004d00 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.373+0000 7f4a0a609640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a040cf1d0 con 0x7f4a040b1480 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a08ba6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a040b3710 0x7f4a040b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a08ba6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a040b3710 0x7f4a040b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36990/0 (socket says 172.21.15.70:36990) 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a08ba6640 1 -- 172.21.15.70:0/3660703891 learned_addr learned my addr 172.21.15.70:0/3660703891 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a03fff640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ec004d00 0x7f4a040c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a037fe640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040b1480 0x7f4a040b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a08ba6640 1 -- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040b1480 msgr2=0x7f4a040b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a08ba6640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040b1480 0x7f4a040b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a08ba6640 1 -- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ec004d00 msgr2=0x7f4a040c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a08ba6640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ec004d00 0x7f4a040c3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a08ba6640 1 -- 172.21.15.70:0/3660703891 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a040b2060 con 0x7f4a040b3710 2023-10-08T07:13:35.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a08ba6640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a040b3710 0x7f4a040b3b70 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f49fc002230 tx=0x7f49fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:35.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a00ff9640 1 -- 172.21.15.70:0/3660703891 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc043070 con 0x7f4a040b3710 2023-10-08T07:13:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.374+0000 7f4a00ff9640 1 -- 172.21.15.70:0/3660703891 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49fc034ad0 con 0x7f4a040b3710 2023-10-08T07:13:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.375+0000 7f4a0a609640 1 -- 172.21.15.70:0/3660703891 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a040c5a60 con 0x7f4a040b3710 2023-10-08T07:13:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.375+0000 7f4a00ff9640 1 -- 172.21.15.70:0/3660703891 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49fc034e80 con 0x7f4a040b3710 2023-10-08T07:13:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.375+0000 7f4a0a609640 1 -- 172.21.15.70:0/3660703891 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a040c6020 con 0x7f4a040b3710 2023-10-08T07:13:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.376+0000 7f4a0a609640 1 -- 172.21.15.70:0/3660703891 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a04067490 con 0x7f4a040b3710 2023-10-08T07:13:35.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.377+0000 7f4a00ff9640 1 -- 172.21.15.70:0/3660703891 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f49fc03e440 con 0x7f4a040b3710 2023-10-08T07:13:35.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.378+0000 7f4a00ff9640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f49f006af90 0x7f49f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.378+0000 7f4a00ff9640 1 -- 172.21.15.70:0/3660703891 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f49fc0ab5d0 con 0x7f4a040b3710 2023-10-08T07:13:35.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.378+0000 7f4a03fff640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f49f006af90 0x7f49f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.380+0000 7f4a03fff640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f49f006af90 0x7f49f006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49f4000c00 tx=0x7f49f4009040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:35.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.382+0000 7f4a00ff9640 1 -- 172.21.15.70:0/3660703891 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49fc076440 con 0x7f4a040b3710 2023-10-08T07:13:35.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.580+0000 7f4a0a609640 1 -- 172.21.15.70:0/3660703891 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a040b0ef0 con 0x7f4a040b3710 2023-10-08T07:13:35.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.583+0000 7f4a00ff9640 1 -- 172.21.15.70:0/3660703891 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v234) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f49fc079a60 con 0x7f4a040b3710 2023-10-08T07:13:35.584 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:35.585 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":234,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:35.265267+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.586+0000 7f4a0a609640 1 -- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f49f006af90 msgr2=0x7f49f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.586+0000 7f4a0a609640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f49f006af90 0x7f49f006d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f49f4000c00 tx=0x7f49f4009040 comp rx=0 tx=0).stop 2023-10-08T07:13:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.587+0000 7f4a0a609640 1 -- 172.21.15.70:0/3660703891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a040b3710 msgr2=0x7f4a040b3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.587+0000 7f4a0a609640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a040b3710 0x7f4a040b3b70 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f49fc002230 tx=0x7f49fc009750 comp rx=0 tx=0).stop 2023-10-08T07:13:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.587+0000 7f4a0a609640 1 -- 172.21.15.70:0/3660703891 shutdown_connections 2023-10-08T07:13:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.587+0000 7f4a0a609640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f49f006af90 0x7f49f006d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.587+0000 7f4a0a609640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a040b3710 0x7f4a040b3b70 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.587+0000 7f4a0a609640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4a040b1480 0x7f4a040b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.587+0000 7f4a0a609640 1 --2- 172.21.15.70:0/3660703891 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f49ec004d00 0x7f4a040c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.587+0000 7f4a0a609640 1 -- 172.21.15.70:0/3660703891 >> 172.21.15.70:0/3660703891 conn(0x7f4a0406a380 msgr2=0x7f4a040ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:35.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.587+0000 7f4a0a609640 1 -- 172.21.15.70:0/3660703891 shutdown_connections 2023-10-08T07:13:35.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.587+0000 7f4a0a609640 1 -- 172.21.15.70:0/3660703891 wait complete. 2023-10-08T07:13:35.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T07:13:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.828+0000 7f33d624b640 1 Processor -- start 2023-10-08T07:13:35.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.829+0000 7f33d624b640 1 -- start start 2023-10-08T07:13:35.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.830+0000 7f33d624b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d00afbe0 0x7f33d00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.830+0000 7f33d624b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33d00cb130 0x7f33d00cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.830+0000 7f33d624b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d00b2620 con 0x7f33d006dfc0 2023-10-08T07:13:35.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.830+0000 7f33d624b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d00cbae0 con 0x7f33d00cb130 2023-10-08T07:13:35.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.830+0000 7f33d624b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00b2060 con 0x7f33d00afbe0 2023-10-08T07:13:35.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.831+0000 7f33ceffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f33d006dfc0 0x7f33d0070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39508/0 (socket says 172.21.15.70:39508) 2023-10-08T07:13:35.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.831+0000 7f33ceffd640 1 -- 172.21.15.70:0/1196114941 learned_addr learned my addr 172.21.15.70:0/1196114941 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:35.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.831+0000 7f33cf7fe640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d00afbe0 0x7f33d00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.831+0000 7f33cffff640 1 --2- 172.21.15.70:0/1196114941 >> v2:172.21.15.181:3300/0 conn(0x7f33d00cb130 0x7f33d00cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.831+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619472834 0 0) 0x7f33d00b2620 con 0x7f33d006dfc0 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.831+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33bc003520 con 0x7f33d006dfc0 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1561781721 0 0) 0x7f33bc003520 con 0x7f33d006dfc0 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d00afbe0 msgr2=0x7f33d00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d00afbe0 0x7f33d00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 >> v2:172.21.15.181:3300/0 conn(0x7f33d00cb130 msgr2=0x7f33d00cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 --2- 172.21.15.70:0/1196114941 >> v2:172.21.15.181:3300/0 conn(0x7f33d00cb130 0x7f33d00cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d00b2340 con 0x7f33d006dfc0 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f33b8001670 con 0x7f33d006dfc0 2023-10-08T07:13:35.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 >> v1:172.21.15.70:6789/0 conn(0x7f33d006dfc0 legacy=0x7f33d0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc003970 0x7f33bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33bc0047c0 0x7f33bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d00afbe0 0x7f33bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00cbae0 con 0x7f33bc003970 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00b2060 con 0x7f33bc0047c0 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc00b980 con 0x7f33d00afbe0 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33cf7fe640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc003970 0x7f33bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33ceffd640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33bc0047c0 0x7f33bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33cf7fe640 1 -- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d00afbe0 msgr2=0x7f33bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33cf7fe640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d00afbe0 0x7f33bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33cf7fe640 1 -- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33bc0047c0 msgr2=0x7f33bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:35.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.832+0000 7f33cf7fe640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33bc0047c0 0x7f33bc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.833+0000 7f33cf7fe640 1 -- 172.21.15.70:0/1196114941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc003520 con 0x7f33bc003970 2023-10-08T07:13:35.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.833+0000 7f33ceffd640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33bc0047c0 0x7f33bc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:35.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.834+0000 7f33cf7fe640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc003970 0x7f33bc003d90 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009ac0 tx=0x7f33bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d096a1115ff9ea02 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:35.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.834+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c002c710 con 0x7f33bc003970 2023-10-08T07:13:35.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.834+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c002c870 con 0x7f33bc003970 2023-10-08T07:13:35.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.834+0000 7f33ce7fc640 1 -- 172.21.15.70:0/1196114941 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c002cc00 con 0x7f33bc003970 2023-10-08T07:13:35.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.834+0000 7f33d624b640 1 -- 172.21.15.70:0/1196114941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc003970 msgr2=0x7f33bc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.834+0000 7f33d624b640 1 --2- 172.21.15.70:0/1196114941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33bc003970 0x7f33bc003d90 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009ac0 tx=0x7f33bc0057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.834+0000 7f33cffff640 1 -- 172.21.15.70:0/1196114941 reap_dead start 2023-10-08T07:13:35.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.835+0000 7f33d624b640 1 -- 172.21.15.70:0/1196114941 shutdown_connections 2023-10-08T07:13:35.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.835+0000 7f33d624b640 1 -- 172.21.15.70:0/1196114941 >> 172.21.15.70:0/1196114941 conn(0x7f33d006a5d0 msgr2=0x7f33d00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:35.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.835+0000 7f33d624b640 1 -- 172.21.15.70:0/1196114941 shutdown_connections 2023-10-08T07:13:35.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.835+0000 7f33d624b640 1 -- 172.21.15.70:0/1196114941 wait complete. 2023-10-08T07:13:35.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.836+0000 7f33d624b640 1 Processor -- start 2023-10-08T07:13:35.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.837+0000 7f33d624b640 1 -- start start 2023-10-08T07:13:35.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.837+0000 7f33d624b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33bc003970 0x7f33d014fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33d624b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33d00c61d0 0x7f33d00c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33d624b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d015c8b0 0x7f33d015cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33d624b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d015d270 con 0x7f33d00c61d0 2023-10-08T07:13:35.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33d624b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d01503e0 con 0x7f33bc003970 2023-10-08T07:13:35.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33d624b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00c6cc0 con 0x7f33d015c8b0 2023-10-08T07:13:35.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33ceffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33d00c61d0 0x7f33d00c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33ceffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33d00c61d0 0x7f33d00c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37016/0 (socket says 172.21.15.70:37016) 2023-10-08T07:13:35.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33ceffd640 1 -- 172.21.15.70:0/1189123918 learned_addr learned my addr 172.21.15.70:0/1189123918 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:35.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33ceffd640 1 -- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d015c8b0 msgr2=0x7f33d015cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33ceffd640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d015c8b0 0x7f33d015cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33ceffd640 1 -- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33bc003970 msgr2=0x7f33d014fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:35.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33ceffd640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33bc003970 0x7f33d014fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:35.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.838+0000 7f33ceffd640 1 -- 172.21.15.70:0/1189123918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d00c6f10 con 0x7f33d00c61d0 2023-10-08T07:13:35.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.839+0000 7f33ceffd640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33d00c61d0 0x7f33d00c6650 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f33b80018e0 tx=0x7f33b800dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:35.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.839+0000 7f33c7fff640 1 -- 172.21.15.70:0/1189123918 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b80074e0 con 0x7f33d00c61d0 2023-10-08T07:13:35.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.839+0000 7f33c7fff640 1 -- 172.21.15.70:0/1189123918 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b8007640 con 0x7f33d00c61d0 2023-10-08T07:13:35.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.839+0000 7f33c7fff640 1 -- 172.21.15.70:0/1189123918 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b8004a00 con 0x7f33d00c61d0 2023-10-08T07:13:35.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.839+0000 7f33d624b640 1 -- 172.21.15.70:0/1189123918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d00c71f0 con 0x7f33d00c61d0 2023-10-08T07:13:35.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.839+0000 7f33d624b640 1 -- 172.21.15.70:0/1189123918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d00c77b0 con 0x7f33d00c61d0 2023-10-08T07:13:35.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.840+0000 7f33d624b640 1 -- 172.21.15.70:0/1189123918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d004e160 con 0x7f33d00c61d0 2023-10-08T07:13:35.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.841+0000 7f33c7fff640 1 -- 172.21.15.70:0/1189123918 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f33b8004b60 con 0x7f33d00c61d0 2023-10-08T07:13:35.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.842+0000 7f33c7fff640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f33b006afc0 0x7f33b006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:35.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.842+0000 7f33c7fff640 1 -- 172.21.15.70:0/1189123918 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f33b80892d0 con 0x7f33d00c61d0 2023-10-08T07:13:35.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.843+0000 7f33cf7fe640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f33b006afc0 0x7f33b006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:35.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.844+0000 7f33cf7fe640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f33b006afc0 0x7f33b006d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f33c0036000 tx=0x7f33c002e000 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:35.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:35.846+0000 7f33c7fff640 1 -- 172.21.15.70:0/1189123918 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33b8053fa0 con 0x7f33d00c61d0 2023-10-08T07:13:36.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.077+0000 7f33d624b640 1 -- 172.21.15.70:0/1189123918 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f33d00b0ef0 con 0x7f33d00c61d0 2023-10-08T07:13:36.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.729+0000 7f33c7fff640 1 -- 172.21.15.70:0/1189123918 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v107) v1 ==== 118+0+0 (secure 0 0 0) 0x7f33b80575c0 con 0x7f33d00c61d0 2023-10-08T07:13:36.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.732+0000 7f33d624b640 1 -- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f33b006afc0 msgr2=0x7f33b006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:36.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.732+0000 7f33d624b640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f33b006afc0 0x7f33b006d480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f33c0036000 tx=0x7f33c002e000 comp rx=0 tx=0).stop 2023-10-08T07:13:36.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.732+0000 7f33d624b640 1 -- 172.21.15.70:0/1189123918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33d00c61d0 msgr2=0x7f33d00c6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:36.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.732+0000 7f33d624b640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33d00c61d0 0x7f33d00c6650 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f33b80018e0 tx=0x7f33b800dbd0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.732+0000 7f33d624b640 1 -- 172.21.15.70:0/1189123918 shutdown_connections 2023-10-08T07:13:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.732+0000 7f33d624b640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33d015c8b0 0x7f33d015cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.732+0000 7f33d624b640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f33b006afc0 0x7f33b006d480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.732+0000 7f33d624b640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33d00c61d0 0x7f33d00c6650 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.732+0000 7f33d624b640 1 --2- 172.21.15.70:0/1189123918 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33bc003970 0x7f33d014fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.732+0000 7f33d624b640 1 -- 172.21.15.70:0/1189123918 >> 172.21.15.70:0/1189123918 conn(0x7f33d006a5d0 msgr2=0x7f33d00b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.733+0000 7f33d624b640 1 -- 172.21.15.70:0/1189123918 shutdown_connections 2023-10-08T07:13:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.733+0000 7f33d624b640 1 -- 172.21.15.70:0/1189123918 wait complete. 2023-10-08T07:13:36.755 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T07:13:36.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T07:13:36.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.968+0000 7ff5b32a2640 1 Processor -- start 2023-10-08T07:13:36.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.968+0000 7ff5b32a2640 1 -- start start 2023-10-08T07:13:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.970+0000 7ff5b32a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac160660 0x7ff5ac070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.970+0000 7ff5b32a2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac06e150 con 0x7ff5ac0c9910 2023-10-08T07:13:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.970+0000 7ff5b32a2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ac06de70 con 0x7ff5ac0cb620 2023-10-08T07:13:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.970+0000 7ff5b32a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0c8e30 con 0x7ff5ac160660 2023-10-08T07:13:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.970+0000 7ff5b083d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff5ac0c9910 0x7ff5ac0caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39518/0 (socket says 172.21.15.70:39518) 2023-10-08T07:13:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.970+0000 7ff5b083d640 1 -- 172.21.15.70:0/2145778934 learned_addr learned my addr 172.21.15.70:0/2145778934 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.970+0000 7ff5b103e640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac160660 0x7ff5ac070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.970+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291655225 0 0) 0x7ff5ac06e150 con 0x7ff5ac0c9910 2023-10-08T07:13:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff598003520 con 0x7ff5ac0c9910 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4169972741 0 0) 0x7ff598003520 con 0x7ff5ac0c9910 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac160660 msgr2=0x7ff5ac070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac160660 0x7ff5ac070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 >> v1:172.21.15.181:6789/0 conn(0x7ff5ac0cb620 legacy=0x7ff5ac0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac0c9110 con 0x7ff5ac0c9910 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff59c001670 con 0x7ff5ac0c9910 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 >> v1:172.21.15.70:6789/0 conn(0x7ff5ac0c9910 legacy=0x7ff5ac0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac160660 0x7ff5980039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff598004020 0x7ff5980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff598004d00 0x7ff598005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0c8e30 con 0x7ff598004020 2023-10-08T07:13:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59800bbc0 con 0x7ff598004d00 2023-10-08T07:13:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59800bd80 con 0x7ff5ac160660 2023-10-08T07:13:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.971+0000 7ff5b083d640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff598004020 0x7ff5980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.972+0000 7ff5b103e640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac160660 0x7ff5980039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.972+0000 7ff5b183f640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff598004d00 0x7ff598005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.972+0000 7ff5b083d640 1 -- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac160660 msgr2=0x7ff5980039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.972+0000 7ff5b083d640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac160660 0x7ff5980039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.972+0000 7ff5b083d640 1 -- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff598004d00 msgr2=0x7ff598005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.972+0000 7ff5b083d640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff598004d00 0x7ff598005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.972+0000 7ff5b083d640 1 -- 172.21.15.70:0/2145778934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff598003520 con 0x7ff598004020 2023-10-08T07:13:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.972+0000 7ff5b103e640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5ac160660 0x7ff5980039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.973+0000 7ff5b083d640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff598004020 0x7ff5980047c0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c0e0 tx=0x7ff5980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9251f2f03b03aca7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:36.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.973+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c005ad0 con 0x7ff598004020 2023-10-08T07:13:36.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.973+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c005c30 con 0x7ff598004020 2023-10-08T07:13:36.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.973+0000 7ff5a3fff640 1 -- 172.21.15.70:0/2145778934 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c02ca30 con 0x7ff598004020 2023-10-08T07:13:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.974+0000 7ff5b32a2640 1 -- 172.21.15.70:0/2145778934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff598004020 msgr2=0x7ff5980047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.974+0000 7ff5b32a2640 1 --2- 172.21.15.70:0/2145778934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff598004020 0x7ff5980047c0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c0e0 tx=0x7ff5980057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.974+0000 7ff5b183f640 1 -- 172.21.15.70:0/2145778934 reap_dead start 2023-10-08T07:13:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.974+0000 7ff5b32a2640 1 -- 172.21.15.70:0/2145778934 shutdown_connections 2023-10-08T07:13:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.974+0000 7ff5b32a2640 1 -- 172.21.15.70:0/2145778934 >> 172.21.15.70:0/2145778934 conn(0x7ff5ac06a5d0 msgr2=0x7ff5ac0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.974+0000 7ff5b32a2640 1 -- 172.21.15.70:0/2145778934 shutdown_connections 2023-10-08T07:13:36.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.974+0000 7ff5b32a2640 1 -- 172.21.15.70:0/2145778934 wait complete. 2023-10-08T07:13:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.976+0000 7ff5b32a2640 1 Processor -- start 2023-10-08T07:13:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.976+0000 7ff5b32a2640 1 -- start start 2023-10-08T07:13:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.977+0000 7ff5b32a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff598004020 0x7ff5ac15bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.977+0000 7ff5b32a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5ac0b1c00 0x7ff5ac0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.977+0000 7ff5b32a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac14c920 0x7ff5ac14cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.977+0000 7ff5b32a2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac14d2e0 con 0x7ff5ac0b1c00 2023-10-08T07:13:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.977+0000 7ff5b32a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac1f2340 con 0x7ff5ac14c920 2023-10-08T07:13:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.977+0000 7ff5b32a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ac0ab970 con 0x7ff598004020 2023-10-08T07:13:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.977+0000 7ff5b083d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5ac0b1c00 0x7ff5ac0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:36.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.977+0000 7ff5b083d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5ac0b1c00 0x7ff5ac0b2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37038/0 (socket says 172.21.15.70:37038) 2023-10-08T07:13:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.977+0000 7ff5b083d640 1 -- 172.21.15.70:0/1111470913 learned_addr learned my addr 172.21.15.70:0/1111470913 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5b103e640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff598004020 0x7ff5ac15bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5b183f640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac14c920 0x7ff5ac14cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5b083d640 1 -- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff598004020 msgr2=0x7ff5ac15bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5b083d640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff598004020 0x7ff5ac15bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5b083d640 1 -- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac14c920 msgr2=0x7ff5ac14cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5b083d640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac14c920 0x7ff5ac14cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5b083d640 1 -- 172.21.15.70:0/1111470913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac0abc10 con 0x7ff5ac0b1c00 2023-10-08T07:13:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5b103e640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff598004020 0x7ff5ac15bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5b083d640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5ac0b1c00 0x7ff5ac0b2080 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c0e0 tx=0x7ff59c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5a1ffb640 1 -- 172.21.15.70:0/1111470913 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c03c040 con 0x7ff5ac0b1c00 2023-10-08T07:13:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5a1ffb640 1 -- 172.21.15.70:0/1111470913 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c02dc90 con 0x7ff5ac0b1c00 2023-10-08T07:13:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5a1ffb640 1 -- 172.21.15.70:0/1111470913 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c007070 con 0x7ff5ac0b1c00 2023-10-08T07:13:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.978+0000 7ff5b32a2640 1 -- 172.21.15.70:0/1111470913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ac0abef0 con 0x7ff5ac0b1c00 2023-10-08T07:13:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.979+0000 7ff5b32a2640 1 -- 172.21.15.70:0/1111470913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac0ac4b0 con 0x7ff5ac0b1c00 2023-10-08T07:13:36.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.980+0000 7ff5b32a2640 1 -- 172.21.15.70:0/1111470913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ac04e160 con 0x7ff5ac0b1c00 2023-10-08T07:13:36.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.981+0000 7ff5a1ffb640 1 -- 172.21.15.70:0/1111470913 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7ff59c032070 con 0x7ff5ac0b1c00 2023-10-08T07:13:36.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.982+0000 7ff5a1ffb640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff59006af90 0x7ff59006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:36.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.982+0000 7ff5a1ffb640 1 -- 172.21.15.70:0/1111470913 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7ff59c0ab420 con 0x7ff5ac0b1c00 2023-10-08T07:13:36.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.983+0000 7ff5b103e640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff59006af90 0x7ff59006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:36.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.984+0000 7ff5b103e640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff59006af90 0x7ff59006d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8006fd0 tx=0x7ff5a8008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:36.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:36.987+0000 7ff5a1ffb640 1 -- 172.21.15.70:0/1111470913 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff59c076290 con 0x7ff5ac0b1c00 2023-10-08T07:13:37.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.219+0000 7ff5b32a2640 1 -- 172.21.15.70:0/1111470913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff5ac0cd670 con 0x7ff5ac0b1c00 2023-10-08T07:13:37.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.742+0000 7ff5a1ffb640 1 -- 172.21.15.70:0/1111470913 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v108) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff59c0798b0 con 0x7ff5ac0b1c00 2023-10-08T07:13:37.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.745+0000 7ff5b32a2640 1 -- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff59006af90 msgr2=0x7ff59006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:37.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.745+0000 7ff5b32a2640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff59006af90 0x7ff59006d450 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8006fd0 tx=0x7ff5a8008040 comp rx=0 tx=0).stop 2023-10-08T07:13:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.745+0000 7ff5b32a2640 1 -- 172.21.15.70:0/1111470913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5ac0b1c00 msgr2=0x7ff5ac0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.745+0000 7ff5b32a2640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5ac0b1c00 0x7ff5ac0b2080 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c0e0 tx=0x7ff59c001930 comp rx=0 tx=0).stop 2023-10-08T07:13:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.746+0000 7ff5b32a2640 1 -- 172.21.15.70:0/1111470913 shutdown_connections 2023-10-08T07:13:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.746+0000 7ff5b32a2640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5ac14c920 0x7ff5ac14cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.746+0000 7ff5b32a2640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7ff59006af90 0x7ff59006d450 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.746+0000 7ff5b32a2640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5ac0b1c00 0x7ff5ac0b2080 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.746+0000 7ff5b32a2640 1 --2- 172.21.15.70:0/1111470913 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff598004020 0x7ff5ac15bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.746+0000 7ff5b32a2640 1 -- 172.21.15.70:0/1111470913 >> 172.21.15.70:0/1111470913 conn(0x7ff5ac06a5d0 msgr2=0x7ff5ac0cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.746+0000 7ff5b32a2640 1 -- 172.21.15.70:0/1111470913 shutdown_connections 2023-10-08T07:13:37.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.746+0000 7ff5b32a2640 1 -- 172.21.15.70:0/1111470913 wait complete. 2023-10-08T07:13:37.768 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T07:13:37.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:13:37.861 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:13:37.869 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:13:37.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.976+0000 7fc02aeea640 1 Processor -- start 2023-10-08T07:13:37.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.976+0000 7fc02aeea640 1 -- start start 2023-10-08T07:13:37.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc02aeea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02415cdf0 0x7fc0240ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:37.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc02aeea640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0240d07c0 0x7fc0240d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:37.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc02aeea640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc0240d1470 0x7fc024160850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:37.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc02aeea640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0240d1900 con 0x7fc0240d07c0 2023-10-08T07:13:37.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc02aeea640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0240ce610 con 0x7fc0240d1470 2023-10-08T07:13:37.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc02aeea640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc02415d1d0 con 0x7fc02415cdf0 2023-10-08T07:13:37.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc0296e7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0240d07c0 0x7fc0240d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:37.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc0296e7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc0240d07c0 0x7fc0240d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37044/0 (socket says 172.21.15.70:37044) 2023-10-08T07:13:37.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc0296e7640 1 -- 172.21.15.70:0/36371215 learned_addr learned my addr 172.21.15.70:0/36371215 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:37.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc029ee8640 1 --2- 172.21.15.70:0/36371215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02415cdf0 0x7fc0240ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:37.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.978+0000 7fc02a6e9640 1 --2- 172.21.15.70:0/36371215 >> v2:172.21.15.181:3300/0 conn(0x7fc0240d1470 0x7fc024160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:37.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.979+0000 7fc0296e7640 1 -- 172.21.15.70:0/36371215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02415cdf0 msgr2=0x7fc0240ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:37.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.979+0000 7fc0296e7640 1 --2- 172.21.15.70:0/36371215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02415cdf0 0x7fc0240ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.979+0000 7fc0296e7640 1 -- 172.21.15.70:0/36371215 >> v2:172.21.15.181:3300/0 conn(0x7fc0240d1470 msgr2=0x7fc024160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:37.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.979+0000 7fc0296e7640 1 --2- 172.21.15.70:0/36371215 >> v2:172.21.15.181:3300/0 conn(0x7fc0240d1470 0x7fc024160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.979+0000 7fc0296e7640 1 -- 172.21.15.70:0/36371215 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0240ceb00 con 0x7fc0240d07c0 2023-10-08T07:13:37.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.980+0000 7fc0296e7640 1 --2- 172.21.15.70:0/36371215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d07c0 0x7fc0240d0f30 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fc0240d1c10 tx=0x7fc01402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6774df03b5e8f3e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:37.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.980+0000 7fc028ee6640 1 -- 172.21.15.70:0/36371215 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014031740 con 0x7fc0240d07c0 2023-10-08T07:13:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.980+0000 7fc028ee6640 1 -- 172.21.15.70:0/36371215 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0140318a0 con 0x7fc0240d07c0 2023-10-08T07:13:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.980+0000 7fc028ee6640 1 -- 172.21.15.70:0/36371215 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014031c30 con 0x7fc0240d07c0 2023-10-08T07:13:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.981+0000 7fc02aeea640 1 -- 172.21.15.70:0/36371215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d07c0 msgr2=0x7fc0240d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.981+0000 7fc02aeea640 1 --2- 172.21.15.70:0/36371215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d07c0 0x7fc0240d0f30 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fc0240d1c10 tx=0x7fc01402b7e0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.981+0000 7fc02aeea640 1 -- 172.21.15.70:0/36371215 shutdown_connections 2023-10-08T07:13:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.981+0000 7fc02aeea640 1 --2- 172.21.15.70:0/36371215 >> v2:172.21.15.181:3300/0 conn(0x7fc0240d1470 0x7fc024160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.981+0000 7fc02aeea640 1 --2- 172.21.15.70:0/36371215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d07c0 0x7fc0240d0f30 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.981+0000 7fc02aeea640 1 --2- 172.21.15.70:0/36371215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc02415cdf0 0x7fc0240ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.981+0000 7fc02aeea640 1 -- 172.21.15.70:0/36371215 >> 172.21.15.70:0/36371215 conn(0x7fc02409bcb0 msgr2=0x7fc0240d29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.981+0000 7fc02aeea640 1 -- 172.21.15.70:0/36371215 shutdown_connections 2023-10-08T07:13:37.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.981+0000 7fc02aeea640 1 -- 172.21.15.70:0/36371215 wait complete. 2023-10-08T07:13:37.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.983+0000 7fc02aeea640 1 Processor -- start 2023-10-08T07:13:37.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.983+0000 7fc02aeea640 1 -- start start 2023-10-08T07:13:37.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.984+0000 7fc02aeea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0240d07c0 0x7fc024155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:37.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.984+0000 7fc02aeea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d1470 0x7fc0240c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:37.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.984+0000 7fc02aeea640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc02415cdf0 0x7fc024159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:37.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.984+0000 7fc02aeea640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc02415a1a0 con 0x7fc0240d1470 2023-10-08T07:13:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.984+0000 7fc02aeea640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc024159a50 con 0x7fc02415cdf0 2023-10-08T07:13:37.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.984+0000 7fc02aeea640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc024156b40 con 0x7fc0240d07c0 2023-10-08T07:13:37.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc0296e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d1470 0x7fc0240c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:37.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc029ee8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0240d07c0 0x7fc024155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:37.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc0296e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d1470 0x7fc0240c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37048/0 (socket says 172.21.15.70:37048) 2023-10-08T07:13:37.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc0296e7640 1 -- 172.21.15.70:0/156007394 learned_addr learned my addr 172.21.15.70:0/156007394 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:37.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc029ee8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0240d07c0 0x7fc024155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:32784/0 (socket says 172.21.15.70:32784) 2023-10-08T07:13:37.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc02a6e9640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc02415cdf0 0x7fc024159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:37.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc0296e7640 1 -- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0240d07c0 msgr2=0x7fc024155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:37.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc0296e7640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0240d07c0 0x7fc024155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc0296e7640 1 -- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc02415cdf0 msgr2=0x7fc024159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:37.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc0296e7640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc02415cdf0 0x7fc024159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:37.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.985+0000 7fc0296e7640 1 -- 172.21.15.70:0/156007394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0140094d0 con 0x7fc0240d1470 2023-10-08T07:13:37.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.986+0000 7fc0296e7640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d1470 0x7fc0240c8ed0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fc0140097f0 tx=0x7fc0140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:37.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.986+0000 7fc01a7fc640 1 -- 172.21.15.70:0/156007394 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0140338c0 con 0x7fc0240d1470 2023-10-08T07:13:37.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.986+0000 7fc01a7fc640 1 -- 172.21.15.70:0/156007394 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc014033a20 con 0x7fc0240d1470 2023-10-08T07:13:37.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.986+0000 7fc02aeea640 1 -- 172.21.15.70:0/156007394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0241562a0 con 0x7fc0240d1470 2023-10-08T07:13:37.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.986+0000 7fc01a7fc640 1 -- 172.21.15.70:0/156007394 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014033dd0 con 0x7fc0240d1470 2023-10-08T07:13:37.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.986+0000 7fc02aeea640 1 -- 172.21.15.70:0/156007394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc024156860 con 0x7fc0240d1470 2023-10-08T07:13:37.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.987+0000 7fc02aeea640 1 -- 172.21.15.70:0/156007394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0240b51b0 con 0x7fc0240d1470 2023-10-08T07:13:37.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.988+0000 7fc01a7fc640 1 -- 172.21.15.70:0/156007394 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fc01403a040 con 0x7fc0240d1470 2023-10-08T07:13:37.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.989+0000 7fc01a7fc640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc00806af90 0x7fc00806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:37.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.989+0000 7fc01a7fc640 1 -- 172.21.15.70:0/156007394 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc0140ab650 con 0x7fc0240d1470 2023-10-08T07:13:37.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.990+0000 7fc029ee8640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc00806af90 0x7fc00806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:37.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.991+0000 7fc029ee8640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc00806af90 0x7fc00806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc020009b40 tx=0x7fc020009190 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:37.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:37.993+0000 7fc01a7fc640 1 -- 172.21.15.70:0/156007394 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0140763c0 con 0x7fc0240d1470 2023-10-08T07:13:38.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.221+0000 7fc02aeea640 1 -- 172.21.15.70:0/156007394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0240b5ef0 con 0x7fc0240d1470 2023-10-08T07:13:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.222+0000 7fc01a7fc640 1 -- 172.21.15.70:0/156007394 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4929 (secure 0 0 0) 0x7fc01407ee60 con 0x7fc0240d1470 2023-10-08T07:13:38.224 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:38.224 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":108,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8416,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:34.170119+0000","modified":"2023-10-08T07:13:37.222153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11097":{"gid":11097,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/180995576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":180995576},{"type":"v1","addr":"172.21.15.70:6834","nonce":180995576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-10-08T07:13:38.224 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 108 2023-10-08T07:13:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.224+0000 7fc02aeea640 1 -- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc00806af90 msgr2=0x7fc00806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc00806af90 0x7fc00806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc020009b40 tx=0x7fc020009190 comp rx=0 tx=0).stop 2023-10-08T07:13:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 -- 172.21.15.70:0/156007394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d1470 msgr2=0x7fc0240c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d1470 0x7fc0240c8ed0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fc0140097f0 tx=0x7fc0140046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 -- 172.21.15.70:0/156007394 shutdown_connections 2023-10-08T07:13:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc02415cdf0 0x7fc024159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc00806af90 0x7fc00806d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0240d1470 0x7fc0240c8ed0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 --2- 172.21.15.70:0/156007394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc0240d07c0 0x7fc024155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 -- 172.21.15.70:0/156007394 >> 172.21.15.70:0/156007394 conn(0x7fc02409bcb0 msgr2=0x7fc02415f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 -- 172.21.15.70:0/156007394 shutdown_connections 2023-10-08T07:13:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.225+0000 7fc02aeea640 1 -- 172.21.15.70:0/156007394 wait complete. 2023-10-08T07:13:38.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:38.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.510+0000 7efe97eb4640 1 Processor -- start 2023-10-08T07:13:38.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.510+0000 7efe97eb4640 1 -- start start 2023-10-08T07:13:38.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.512+0000 7efe97eb4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7efe9015ab40 0x7efe900ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.512+0000 7efe97eb4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9015b2b0 con 0x7efe900cb580 2023-10-08T07:13:38.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.512+0000 7efe97eb4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9015b590 con 0x7efe900c90b0 2023-10-08T07:13:38.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.512+0000 7efe97eb4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9015bf30 con 0x7efe9015ab40 2023-10-08T07:13:38.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.512+0000 7efe96451640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efe900cb580 0x7efe9015e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39524/0 (socket says 172.21.15.70:39524) 2023-10-08T07:13:38.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.512+0000 7efe96451640 1 -- 172.21.15.70:0/3934563227 learned_addr learned my addr 172.21.15.70:0/3934563227 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:38.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe95c50640 1 --2- 172.21.15.70:0/3934563227 >> v2:172.21.15.181:3300/0 conn(0x7efe9015ab40 0x7efe900ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:38.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831989658 0 0) 0x7efe9015b2b0 con 0x7efe900cb580 2023-10-08T07:13:38.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe7c003520 con 0x7efe900cb580 2023-10-08T07:13:38.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 894508621 0 0) 0x7efe7c003520 con 0x7efe900cb580 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 >> v2:172.21.15.181:3300/0 conn(0x7efe9015ab40 msgr2=0x7efe900ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 --2- 172.21.15.70:0/3934563227 >> v2:172.21.15.181:3300/0 conn(0x7efe9015ab40 0x7efe900ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 >> v1:172.21.15.181:6789/0 conn(0x7efe900c90b0 legacy=0x7efe900cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9015b870 con 0x7efe900cb580 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7efe880019f0 con 0x7efe900cb580 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 >> v1:172.21.15.70:6789/0 conn(0x7efe900cb580 legacy=0x7efe9015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 --2- 172.21.15.70:0/3934563227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe7c0039c0 0x7efe7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 --2- 172.21.15.70:0/3934563227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe7c0047c0 0x7efe7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 --2- 172.21.15.70:0/3934563227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe7c005180 0x7efe7c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.513+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9015bf30 con 0x7efe7c0047c0 2023-10-08T07:13:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.514+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c00c080 con 0x7efe7c0039c0 2023-10-08T07:13:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.514+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7c00c240 con 0x7efe7c005180 2023-10-08T07:13:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.514+0000 7efe9544f640 1 --2- 172.21.15.70:0/3934563227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe7c0047c0 0x7efe7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.514+0000 7efe95c50640 1 --2- 172.21.15.70:0/3934563227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe7c0039c0 0x7efe7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.514+0000 7efe9544f640 1 -- 172.21.15.70:0/3934563227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe7c005180 msgr2=0x7efe7c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.514+0000 7efe9544f640 1 --2- 172.21.15.70:0/3934563227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe7c005180 0x7efe7c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.514+0000 7efe9544f640 1 -- 172.21.15.70:0/3934563227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe7c0039c0 msgr2=0x7efe7c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.514+0000 7efe9544f640 1 --2- 172.21.15.70:0/3934563227 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe7c0039c0 0x7efe7c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.514+0000 7efe9544f640 1 -- 172.21.15.70:0/3934563227 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c003520 con 0x7efe7c0047c0 2023-10-08T07:13:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.514+0000 7efe96451640 1 -- 172.21.15.70:0/3934563227 reap_dead start 2023-10-08T07:13:38.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.515+0000 7efe9544f640 1 --2- 172.21.15.70:0/3934563227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe7c0047c0 0x7efe7c004c40 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7efe80001720 tx=0x7efe7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5824a32b3a17ce16 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:38.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.516+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe800052d0 con 0x7efe7c0047c0 2023-10-08T07:13:38.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.516+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8002e050 con 0x7efe7c0047c0 2023-10-08T07:13:38.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.516+0000 7efe94c4e640 1 -- 172.21.15.70:0/3934563227 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe80003830 con 0x7efe7c0047c0 2023-10-08T07:13:38.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.516+0000 7efe97eb4640 1 -- 172.21.15.70:0/3934563227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe7c0047c0 msgr2=0x7efe7c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.516+0000 7efe97eb4640 1 --2- 172.21.15.70:0/3934563227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe7c0047c0 0x7efe7c004c40 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7efe80001720 tx=0x7efe7c0057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.516+0000 7efe97eb4640 1 -- 172.21.15.70:0/3934563227 shutdown_connections 2023-10-08T07:13:38.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.516+0000 7efe97eb4640 1 --2- 172.21.15.70:0/3934563227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe7c0047c0 0x7efe7c004c40 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.516+0000 7efe97eb4640 1 -- 172.21.15.70:0/3934563227 >> 172.21.15.70:0/3934563227 conn(0x7efe9006a380 msgr2=0x7efe900b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:38.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.516+0000 7efe97eb4640 1 -- 172.21.15.70:0/3934563227 shutdown_connections 2023-10-08T07:13:38.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.516+0000 7efe97eb4640 1 -- 172.21.15.70:0/3934563227 wait complete. 2023-10-08T07:13:38.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.518+0000 7efe97eb4640 1 Processor -- start 2023-10-08T07:13:38.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.518+0000 7efe97eb4640 1 -- start start 2023-10-08T07:13:38.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.519+0000 7efe97eb4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe7c0047c0 0x7efe900b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.519+0000 7efe97eb4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe900b1480 0x7efe900c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.519+0000 7efe97eb4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe900c9d50 0x7efe901470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.519+0000 7efe97eb4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe900ca1d0 con 0x7efe900c9d50 2023-10-08T07:13:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.519+0000 7efe97eb4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe900b3140 con 0x7efe7c0047c0 2023-10-08T07:13:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.519+0000 7efe97eb4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe900b33e0 con 0x7efe900b1480 2023-10-08T07:13:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.519+0000 7efe96451640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe900c9d50 0x7efe901470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.519+0000 7efe96451640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe900c9d50 0x7efe901470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37062/0 (socket says 172.21.15.70:37062) 2023-10-08T07:13:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.519+0000 7efe96451640 1 -- 172.21.15.70:0/645174365 learned_addr learned my addr 172.21.15.70:0/645174365 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.520+0000 7efe96451640 1 -- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe900b1480 msgr2=0x7efe900c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.520+0000 7efe96451640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe900b1480 0x7efe900c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.520+0000 7efe96451640 1 -- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe7c0047c0 msgr2=0x7efe900b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.520+0000 7efe96451640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe7c0047c0 0x7efe900b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.520+0000 7efe96451640 1 -- 172.21.15.70:0/645174365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe900b36c0 con 0x7efe900c9d50 2023-10-08T07:13:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.520+0000 7efe96451640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe900c9d50 0x7efe901470d0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7efe9006e3f0 tx=0x7efe8800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.520+0000 7efe867fc640 1 -- 172.21.15.70:0/645174365 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800cea0 con 0x7efe900c9d50 2023-10-08T07:13:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.520+0000 7efe867fc640 1 -- 172.21.15.70:0/645174365 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8800a610 con 0x7efe900c9d50 2023-10-08T07:13:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.520+0000 7efe867fc640 1 -- 172.21.15.70:0/645174365 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800a9c0 con 0x7efe900c9d50 2023-10-08T07:13:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.520+0000 7efe97eb4640 1 -- 172.21.15.70:0/645174365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe900b39a0 con 0x7efe900c9d50 2023-10-08T07:13:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.521+0000 7efe97eb4640 1 -- 172.21.15.70:0/645174365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe900b18e0 con 0x7efe900c9d50 2023-10-08T07:13:38.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.522+0000 7efe97eb4640 1 -- 172.21.15.70:0/645174365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe900b1aa0 con 0x7efe900c9d50 2023-10-08T07:13:38.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.523+0000 7efe867fc640 1 -- 172.21.15.70:0/645174365 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7efe8800ab20 con 0x7efe900c9d50 2023-10-08T07:13:38.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.524+0000 7efe867fc640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efe7406af90 0x7efe7406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.524+0000 7efe867fc640 1 -- 172.21.15.70:0/645174365 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7efe88089200 con 0x7efe900c9d50 2023-10-08T07:13:38.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.524+0000 7efe95c50640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efe7406af90 0x7efe7406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:38.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.525+0000 7efe95c50640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efe7406af90 0x7efe7406d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009990 tx=0x7efe8c008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:38.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.528+0000 7efe867fc640 1 -- 172.21.15.70:0/645174365 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe88053f40 con 0x7efe900c9d50 2023-10-08T07:13:38.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.721+0000 7efe97eb4640 1 -- 172.21.15.70:0/645174365 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe90067490 con 0x7efe900c9d50 2023-10-08T07:13:38.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.723+0000 7efe867fc640 1 -- 172.21.15.70:0/645174365 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v234) v1 ==== 74+0+16958 (secure 0 0 0) 0x7efe8801a090 con 0x7efe900c9d50 2023-10-08T07:13:38.725 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:38.725 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":234,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:35.265267+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:38.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.725+0000 7efe97eb4640 1 -- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efe7406af90 msgr2=0x7efe7406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.725+0000 7efe97eb4640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efe7406af90 0x7efe7406d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009990 tx=0x7efe8c008040 comp rx=0 tx=0).stop 2023-10-08T07:13:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.725+0000 7efe97eb4640 1 -- 172.21.15.70:0/645174365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe900c9d50 msgr2=0x7efe901470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.725+0000 7efe97eb4640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe900c9d50 0x7efe901470d0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7efe9006e3f0 tx=0x7efe8800cc50 comp rx=0 tx=0).stop 2023-10-08T07:13:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.726+0000 7efe97eb4640 1 -- 172.21.15.70:0/645174365 shutdown_connections 2023-10-08T07:13:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.726+0000 7efe97eb4640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7efe7406af90 0x7efe7406d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.726+0000 7efe97eb4640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe900c9d50 0x7efe901470d0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.726+0000 7efe97eb4640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7efe900b1480 0x7efe900c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.726+0000 7efe97eb4640 1 --2- 172.21.15.70:0/645174365 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7efe7c0047c0 0x7efe900b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.726+0000 7efe97eb4640 1 -- 172.21.15.70:0/645174365 >> 172.21.15.70:0/645174365 conn(0x7efe9006a380 msgr2=0x7efe900ae570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.726+0000 7efe97eb4640 1 -- 172.21.15.70:0/645174365 shutdown_connections 2023-10-08T07:13:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.726+0000 7efe97eb4640 1 -- 172.21.15.70:0/645174365 wait complete. 2023-10-08T07:13:38.747 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T07:13:38.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:13:38.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.959+0000 7f515f59e640 1 Processor -- start 2023-10-08T07:13:38.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.959+0000 7f515f59e640 1 -- start start 2023-10-08T07:13:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51601599d0 0x7f51600ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515f59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f51600b07d0 0x7f516006c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515f59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51600cacf0 con 0x7f51600cc210 2023-10-08T07:13:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515f59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51600b10f0 con 0x7f51601599d0 2023-10-08T07:13:38.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515f59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51600ca3c0 con 0x7f51600b07d0 2023-10-08T07:13:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51601599d0 0x7f51600ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51601599d0 0x7f51600ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37068/0 (socket says 172.21.15.70:37068) 2023-10-08T07:13:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f5155d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f51600cc210 0x7f51600b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39532/0 (socket says 172.21.15.70:39532) 2023-10-08T07:13:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515e59c640 1 -- 172.21.15.70:0/730309081 learned_addr learned my addr 172.21.15.70:0/730309081 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515ed9d640 1 --2- 172.21.15.70:0/730309081 >> v2:172.21.15.181:3300/0 conn(0x7f51600b07d0 0x7f516006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515e59c640 1 -- 172.21.15.70:0/730309081 >> v2:172.21.15.181:3300/0 conn(0x7f51600b07d0 msgr2=0x7f516006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515e59c640 1 --2- 172.21.15.70:0/730309081 >> v2:172.21.15.181:3300/0 conn(0x7f51600b07d0 0x7f516006c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515e59c640 1 -- 172.21.15.70:0/730309081 >> v1:172.21.15.70:6789/0 conn(0x7f51600cc210 legacy=0x7f51600b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515e59c640 1 -- 172.21.15.70:0/730309081 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51600ca6a0 con 0x7f51601599d0 2023-10-08T07:13:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.961+0000 7f515dd9b640 1 -- 172.21.15.70:0/730309081 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623364802 0 0) 0x7f51600cacf0 con 0x7f51600cc210 2023-10-08T07:13:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.963+0000 7f515e59c640 1 --2- 172.21.15.70:0/730309081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51601599d0 0x7f51600ad730 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f51600c98d0 tx=0x7f515002b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e1dbe1fad1a342 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.963+0000 7f515dd9b640 1 -- 172.21.15.70:0/730309081 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150030880 con 0x7f51601599d0 2023-10-08T07:13:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.963+0000 7f515dd9b640 1 -- 172.21.15.70:0/730309081 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51500309e0 con 0x7f51601599d0 2023-10-08T07:13:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.963+0000 7f515dd9b640 1 -- 172.21.15.70:0/730309081 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150030d70 con 0x7f51601599d0 2023-10-08T07:13:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.963+0000 7f515f59e640 1 -- 172.21.15.70:0/730309081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51601599d0 msgr2=0x7f51600ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.963+0000 7f515f59e640 1 --2- 172.21.15.70:0/730309081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51601599d0 0x7f51600ad730 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f51600c98d0 tx=0x7f515002b9c0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.963+0000 7f515f59e640 1 -- 172.21.15.70:0/730309081 shutdown_connections 2023-10-08T07:13:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.963+0000 7f515f59e640 1 --2- 172.21.15.70:0/730309081 >> v2:172.21.15.181:3300/0 conn(0x7f51600b07d0 0x7f516006c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.963+0000 7f515f59e640 1 --2- 172.21.15.70:0/730309081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51601599d0 0x7f51600ad730 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.963+0000 7f515f59e640 1 -- 172.21.15.70:0/730309081 >> 172.21.15.70:0/730309081 conn(0x7f51600cb0c0 msgr2=0x7f51600cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.964+0000 7f515f59e640 1 -- 172.21.15.70:0/730309081 shutdown_connections 2023-10-08T07:13:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.964+0000 7f515f59e640 1 -- 172.21.15.70:0/730309081 wait complete. 2023-10-08T07:13:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.965+0000 7f515f59e640 1 Processor -- start 2023-10-08T07:13:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.965+0000 7f515f59e640 1 -- start start 2023-10-08T07:13:38.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.966+0000 7f515f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51600b07d0 0x7f51600b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.966+0000 7f515f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51600cc210 0x7f5160157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.966+0000 7f515f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51601599d0 0x7f5160159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.966+0000 7f515f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516015c4f0 con 0x7f51600cc210 2023-10-08T07:13:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.966+0000 7f515f59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516015b600 con 0x7f51601599d0 2023-10-08T07:13:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.966+0000 7f515f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516015b8e0 con 0x7f51600b07d0 2023-10-08T07:13:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.967+0000 7f5155d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51600cc210 0x7f5160157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.967+0000 7f5155d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51600cc210 0x7f5160157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37082/0 (socket says 172.21.15.70:37082) 2023-10-08T07:13:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.967+0000 7f5155d9b640 1 -- 172.21.15.70:0/1321595877 learned_addr learned my addr 172.21.15.70:0/1321595877 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:38.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.967+0000 7f5155d9b640 1 -- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51600b07d0 msgr2=0x7f51600b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.967+0000 7f5155d9b640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51600b07d0 0x7f51600b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.967+0000 7f5155d9b640 1 -- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51601599d0 msgr2=0x7f5160159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:38.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.967+0000 7f5155d9b640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51601599d0 0x7f5160159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:38.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.967+0000 7f5155d9b640 1 -- 172.21.15.70:0/1321595877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51500094d0 con 0x7f51600cc210 2023-10-08T07:13:38.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.967+0000 7f5155d9b640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51600cc210 0x7f5160157660 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f514800c4d0 tx=0x7f514800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:38.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.967+0000 7f51577fe640 1 -- 172.21.15.70:0/1321595877 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51480074e0 con 0x7f51600cc210 2023-10-08T07:13:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.968+0000 7f51577fe640 1 -- 172.21.15.70:0/1321595877 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5148007660 con 0x7f51600cc210 2023-10-08T07:13:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.968+0000 7f515f59e640 1 -- 172.21.15.70:0/1321595877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516015bbc0 con 0x7f51600cc210 2023-10-08T07:13:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.968+0000 7f51577fe640 1 -- 172.21.15.70:0/1321595877 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5148004a20 con 0x7f51600cc210 2023-10-08T07:13:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.968+0000 7f515f59e640 1 -- 172.21.15.70:0/1321595877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516015c180 con 0x7f51600cc210 2023-10-08T07:13:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.969+0000 7f515f59e640 1 -- 172.21.15.70:0/1321595877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516004e160 con 0x7f51600cc210 2023-10-08T07:13:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.970+0000 7f51577fe640 1 -- 172.21.15.70:0/1321595877 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5148004b80 con 0x7f51600cc210 2023-10-08T07:13:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.971+0000 7f51577fe640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f514006afc0 0x7f514006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.971+0000 7f51577fe640 1 -- 172.21.15.70:0/1321595877 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5148088b20 con 0x7f51600cc210 2023-10-08T07:13:38.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.971+0000 7f515e59c640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f514006afc0 0x7f514006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:38.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.973+0000 7f515e59c640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f514006afc0 0x7f514006d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5150036000 tx=0x7f515003b040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:38.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:38.975+0000 7f51577fe640 1 -- 172.21.15.70:0/1321595877 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5148053860 con 0x7f51600cc210 2023-10-08T07:13:39.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.160+0000 7f515f59e640 1 -- 172.21.15.70:0/1321595877 --> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51600ca070 con 0x7f514006afc0 2023-10-08T07:13:39.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.189+0000 7f51577fe640 1 -- 172.21.15.70:0/1321595877 <== mgr.7328 v2:172.21.15.181:6832/3993370366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274679 (secure 0 0 0) 0x7f5150030c50 con 0x7f514006afc0 2023-10-08T07:13:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:39.196 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:13:39.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.193+0000 7f515f59e640 1 -- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f514006afc0 msgr2=0x7f514006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:39.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.193+0000 7f515f59e640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f514006afc0 0x7f514006d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5150036000 tx=0x7f515003b040 comp rx=0 tx=0).stop 2023-10-08T07:13:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.194+0000 7f515f59e640 1 -- 172.21.15.70:0/1321595877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51600cc210 msgr2=0x7f5160157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.194+0000 7f515f59e640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51600cc210 0x7f5160157660 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f514800c4d0 tx=0x7f514800c990 comp rx=0 tx=0).stop 2023-10-08T07:13:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.194+0000 7f515f59e640 1 -- 172.21.15.70:0/1321595877 shutdown_connections 2023-10-08T07:13:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.194+0000 7f515f59e640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f51601599d0 0x7f5160159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.194+0000 7f515f59e640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f514006afc0 0x7f514006d480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.194+0000 7f515f59e640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51600cc210 0x7f5160157660 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.194+0000 7f515f59e640 1 --2- 172.21.15.70:0/1321595877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f51600b07d0 0x7f51600b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.194+0000 7f515f59e640 1 -- 172.21.15.70:0/1321595877 >> 172.21.15.70:0/1321595877 conn(0x7f51600cb0c0 msgr2=0x7f51600b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.194+0000 7f515f59e640 1 -- 172.21.15.70:0/1321595877 shutdown_connections 2023-10-08T07:13:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.194+0000 7f515f59e640 1 -- 172.21.15.70:0/1321595877 wait complete. 2023-10-08T07:13:39.214 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2023-10-08T07:13:37.504384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2806,"ondisk_log_size":2806,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":428724,"kb_used_data":23816,"kb_used_omap":12,"kb_used_meta":404787,"kb_avail":754545996,"statfs":{"total":773094113280,"available":772655099904,"internally_reserved":0,"allocated":24387584,"data_stored":13210304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":414502459},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.809467"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.708678+0000","last_change":"2023-10-08T07:13:32.708678+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.708678+0000","last_undegraded":"2023-10-08T07:13:32.708678+0000","last_fullsized":"2023-10-08T07:13:32.708678+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.706257+0000","last_change":"2023-10-08T07:13:32.706257+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.706257+0000","last_undegraded":"2023-10-08T07:13:32.706257+0000","last_fullsized":"2023-10-08T07:13:32.706257+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.700626+0000","last_change":"2023-10-08T07:13:32.700626+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.700626+0000","last_undegraded":"2023-10-08T07:13:32.700626+0000","last_fullsized":"2023-10-08T07:13:32.700626+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.709120+0000","last_change":"2023-10-08T07:13:32.709120+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.709120+0000","last_undegraded":"2023-10-08T07:13:32.709120+0000","last_fullsized":"2023-10-08T07:13:32.709120+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.700018+0000","last_change":"2023-10-08T07:13:32.700018+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.700018+0000","last_undegraded":"2023-10-08T07:13:32.700018+0000","last_fullsized":"2023-10-08T07:13:32.700018+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.699680+0000","last_change":"2023-10-08T07:13:32.699680+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.699680+0000","last_undegraded":"2023-10-08T07:13:32.699680+0000","last_fullsized":"2023-10-08T07:13:32.699680+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.701460+0000","last_change":"2023-10-08T07:13:32.701460+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.701460+0000","last_undegraded":"2023-10-08T07:13:32.701460+0000","last_fullsized":"2023-10-08T07:13:32.701460+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.703357+0000","last_change":"2023-10-08T07:13:32.703357+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.703357+0000","last_undegraded":"2023-10-08T07:13:32.703357+0000","last_fullsized":"2023-10-08T07:13:32.703357+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.699679+0000","last_change":"2023-10-08T07:13:32.699679+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.699679+0000","last_undegraded":"2023-10-08T07:13:32.699679+0000","last_fullsized":"2023-10-08T07:13:32.699679+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.709614+0000","last_change":"2023-10-08T07:13:32.709614+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.709614+0000","last_undegraded":"2023-10-08T07:13:32.709614+0000","last_fullsized":"2023-10-08T07:13:32.709614+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.699693+0000","last_change":"2023-10-08T07:13:32.699693+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.699693+0000","last_undegraded":"2023-10-08T07:13:32.699693+0000","last_fullsized":"2023-10-08T07:13:32.699693+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.699747+0000","last_change":"2023-10-08T07:13:32.699747+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.699747+0000","last_undegraded":"2023-10-08T07:13:32.699747+0000","last_fullsized":"2023-10-08T07:13:32.699747+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.701534+0000","last_change":"2023-10-08T07:13:32.701534+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.701534+0000","last_undegraded":"2023-10-08T07:13:32.701534+0000","last_fullsized":"2023-10-08T07:13:32.701534+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.709054+0000","last_change":"2023-10-08T07:13:32.709054+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.709054+0000","last_undegraded":"2023-10-08T07:13:32.709054+0000","last_fullsized":"2023-10-08T07:13:32.709054+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.704026+0000","last_change":"2023-10-08T07:13:32.704026+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.704026+0000","last_undegraded":"2023-10-08T07:13:32.704026+0000","last_fullsized":"2023-10-08T07:13:32.704026+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.705044+0000","last_change":"2023-10-08T07:13:32.705044+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.705044+0000","last_undegraded":"2023-10-08T07:13:32.705044+0000","last_fullsized":"2023-10-08T07:13:32.705044+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.704784+0000","last_change":"2023-10-08T07:13:32.704784+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.704784+0000","last_undegraded":"2023-10-08T07:13:32.704784+0000","last_fullsized":"2023-10-08T07:13:32.704784+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.699746+0000","last_change":"2023-10-08T07:13:32.699746+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.699746+0000","last_undegraded":"2023-10-08T07:13:32.699746+0000","last_fullsized":"2023-10-08T07:13:32.699746+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.703360+0000","last_change":"2023-10-08T07:13:32.703360+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.703360+0000","last_undegraded":"2023-10-08T07:13:32.703360+0000","last_fullsized":"2023-10-08T07:13:32.703360+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.703490+0000","last_change":"2023-10-08T07:13:32.703490+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.703490+0000","last_undegraded":"2023-10-08T07:13:32.703490+0000","last_fullsized":"2023-10-08T07:13:32.703490+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.699330+0000","last_change":"2023-10-08T07:13:32.699330+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.699330+0000","last_undegraded":"2023-10-08T07:13:32.699330+0000","last_fullsized":"2023-10-08T07:13:32.699330+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.708591+0000","last_change":"2023-10-08T07:13:32.708591+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.708591+0000","last_undegraded":"2023-10-08T07:13:32.708591+0000","last_fullsized":"2023-10-08T07:13:32.708591+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.704512+0000","last_change":"2023-10-08T07:13:32.704512+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.704512+0000","last_undegraded":"2023-10-08T07:13:32.704512+0000","last_fullsized":"2023-10-08T07:13:32.704512+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.709768+0000","last_change":"2023-10-08T07:13:32.709768+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.709768+0000","last_undegraded":"2023-10-08T07:13:32.709768+0000","last_fullsized":"2023-10-08T07:13:32.709768+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.708312+0000","last_change":"2023-10-08T07:13:32.708312+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.708312+0000","last_undegraded":"2023-10-08T07:13:32.708312+0000","last_fullsized":"2023-10-08T07:13:32.708312+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.710738+0000","last_change":"2023-10-08T07:13:32.710738+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.710738+0000","last_undegraded":"2023-10-08T07:13:32.710738+0000","last_fullsized":"2023-10-08T07:13:32.710738+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.704440+0000","last_change":"2023-10-08T07:13:32.704440+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.704440+0000","last_undegraded":"2023-10-08T07:13:32.704440+0000","last_fullsized":"2023-10-08T07:13:32.704440+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.706737+0000","last_change":"2023-10-08T07:13:32.706737+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.706737+0000","last_undegraded":"2023-10-08T07:13:32.706737+0000","last_fullsized":"2023-10-08T07:13:32.706737+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.709118+0000","last_change":"2023-10-08T07:13:32.709118+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.709118+0000","last_undegraded":"2023-10-08T07:13:32.709118+0000","last_fullsized":"2023-10-08T07:13:32.709118+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.705067+0000","last_change":"2023-10-08T07:13:32.705067+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.705067+0000","last_undegraded":"2023-10-08T07:13:32.705067+0000","last_fullsized":"2023-10-08T07:13:32.705067+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.703628+0000","last_change":"2023-10-08T07:13:32.703628+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.703628+0000","last_undegraded":"2023-10-08T07:13:32.703628+0000","last_fullsized":"2023-10-08T07:13:32.703628+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:13:33.693289+0000","last_change":"2023-10-08T07:13:33.693289+0000","last_active":"2023-10-08T07:13:33.693289+0000","last_peered":"2023-10-08T07:13:33.693289+0000","last_clean":"2023-10-08T07:13:33.693289+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:33.693289+0000","last_undegraded":"2023-10-08T07:13:33.693289+0000","last_fullsized":"2023-10-08T07:13:33.693289+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":1,"reported_epoch":231,"state":"creating+peering","last_fresh":"2023-10-08T07:13:32.710120+0000","last_change":"2023-10-08T07:13:32.710120+0000","last_active":"2023-10-08T07:13:32.687282+0000","last_peered":"2023-10-08T07:13:32.687282+0000","last_clean":"2023-10-08T07:13:32.687282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:32.710120+0000","last_undegraded":"2023-10-08T07:13:32.710120+0000","last_fullsized":"2023-10-08T07:13:32.710120+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"222'2806","reported_seq":3720,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-10-08T07:13:30.478398+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:13:30.478398+0000","last_peered":"2023-10-08T07:13:30.478398+0000","last_clean":"2023-10-08T07:13:30.478398+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:13:30.478398+0000","last_undegraded":"2023-10-08T07:13:30.478398+0000","last_fullsized":"2023-10-08T07:13:30.478398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2806,"log_dups_size":0,"ondisk_log_size":2806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2806,"ondisk_log_size":2806,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576008,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60964,"kb_used_data":3412,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310876,"statfs":{"total":96636764160,"available":96574337024,"internally_reserved":0,"allocated":3493888,"data_stored":2094064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.169,"15min":1.2090000000000001},"min":{"1min":0.77100000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.508,"5min":1.968,"15min":2.008},"last":1.147},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.238,"15min":1.2310000000000001},"min":{"1min":0.91300000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.6830000000000001,"5min":1.7330000000000001,"15min":1.7450000000000001},"last":0.96499999999999997}]},{"osd":1,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.242,"15min":1.252},"min":{"1min":0.94399999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.665,"5min":1.79,"15min":1.998},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.367,"5min":1.341,"15min":1.3500000000000001},"min":{"1min":0.99399999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.8859999999999999,"5min":1.8899999999999999,"15min":2.1019999999999999},"last":1.3540000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.29,"15min":1.3140000000000001},"min":{"1min":0.97199999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6180000000000001,"5min":1.696,"15min":2.1640000000000001},"last":1.105},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.351},"min":{"1min":1.0069999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7270000000000001,"5min":1.8580000000000001,"15min":2.3170000000000002},"last":1.4650000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":1.0209999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.663,"5min":1.8160000000000001,"15min":1.9039999999999999},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.399,"15min":1.4119999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.754,"5min":1.8680000000000001,"15min":1.931},"last":1.042}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":0.86199999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6950000000000001,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.383,"15min":1.401},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.85199999999999998},"max":{"1min":1.76,"5min":1.8140000000000001,"15min":2.0379999999999998},"last":1.4159999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4390000000000001,"15min":1.444},"min":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":2.0920000000000001,"15min":2.1139999999999999},"last":1.617},{"interface":"front","average":{"1min":1.532,"5min":1.4610000000000001,"15min":1.51},"min":{"1min":1.075,"5min":1,"15min":0.96899999999999997},"max":{"1min":1.8640000000000001,"5min":2.23,"15min":2.23},"last":1.5620000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.575,"15min":1.6020000000000001},"min":{"1min":1.361,"5min":1.048,"15min":1.048},"max":{"1min":2.016,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.6759999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.536,"15min":1.5349999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9079999999999999,"5min":2.21,"15min":2.2570000000000001},"last":1.5209999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46492,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325348,"statfs":{"total":96636764160,"available":96589156352,"internally_reserved":0,"allocated":2899968,"data_stored":1503696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3109999999999999,"15min":1.321},"min":{"1min":0.80000000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":1.794,"15min":1.962},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.347,"15min":1.3620000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7230000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.157}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.228,"15min":1.2290000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.226,"5min":1.2130000000000001,"15min":1.2430000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.97},"last":1.329}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.198,"15min":1.238},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.484,"5min":2.339,"15min":2.339},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":1.2310000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3819999999999999,"15min":1.403},"min":{"1min":1.034,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":1.9319999999999999,"15min":1.95},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.476,"15min":1.4990000000000001},"min":{"1min":0.97199999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8740000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.026}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3999999999999999,"15min":1.3899999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.881,"5min":2.0579999999999998,"15min":2.0859999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4770000000000001,"15min":1.448},"min":{"1min":0.94999999999999996,"5min":0.92000000000000004,"15min":0.57099999999999995},"max":{"1min":1.8080000000000001,"5min":2.484,"15min":2.484},"last":1.2829999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.399,"15min":1.423},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.81899999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.403},"last":1.377},{"interface":"front","average":{"1min":1.552,"5min":1.538,"15min":1.5509999999999999},"min":{"1min":1.2829999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.8100000000000001,"5min":2.097,"15min":2.6139999999999999},"last":1.4139999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.575},"min":{"1min":1.0489999999999999,"5min":1.012,"15min":0.96299999999999997},"max":{"1min":1.9379999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.46},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.5620000000000001,"15min":1.5600000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.665},"last":1.4399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60380,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311460,"statfs":{"total":96636764160,"available":96574935040,"internally_reserved":0,"allocated":2899968,"data_stored":1503696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.157,"15min":1.208},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.841},"last":1.05},{"interface":"front","average":{"1min":1.341,"5min":1.2290000000000001,"15min":1.232},"min":{"1min":0.83099999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7030000000000001,"5min":1.8560000000000001,"15min":1.9610000000000001},"last":1.4510000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2090000000000001,"15min":1.242},"min":{"1min":0.75800000000000001,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.6990000000000001,"5min":1.762,"15min":1.9550000000000001},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2869999999999999,"15min":1.3},"min":{"1min":0.90900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.3500000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.871,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":2.0870000000000002},"last":1.26},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.3149999999999999},"min":{"1min":0.77000000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5840000000000001,"5min":1.877,"15min":1.909},"last":1.5580000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":0.79800000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.1499999999999999},"last":2.077},{"interface":"front","average":{"1min":1.458,"5min":1.3740000000000001,"15min":1.431},"min":{"1min":0.67500000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8100000000000001,"5min":1.9530000000000001,"15min":2.173},"last":1.99}]},{"osd":4,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3220000000000001,"15min":1.3280000000000001},"min":{"1min":0.84399999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7569999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.758},{"interface":"front","average":{"1min":1.556,"5min":1.446,"15min":1.4339999999999999},"min":{"1min":1.107,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.2240000000000002},"last":2.1640000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":1.0740000000000001,"5min":0.66500000000000004,"15min":0.57499999999999996},"max":{"1min":1.899,"5min":2.077,"15min":2.3759999999999999},"last":2.1160000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.478,"15min":1.4770000000000001},"min":{"1min":1.157,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0059999999999998,"5min":2.036,"15min":2.036},"last":2.0249999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.5289999999999999,"15min":1.552},"min":{"1min":1.177,"5min":0.72499999999999998,"15min":0.59999999999999998},"max":{"1min":2.298,"5min":2.298,"15min":2.4169999999999998},"last":2.4430000000000001},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.5840000000000001,"15min":1.6180000000000001},"min":{"1min":1.0089999999999999,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0009999999999999,"5min":2.2320000000000002,"15min":2.3199999999999998},"last":2.351}]}]},{"osd":4,"up_from":13,"seq":55834576007,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":2899968,"data_stored":1503696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2849999999999999,"15min":1.3440000000000001},"min":{"1min":0.64200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":0.62,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.9139999999999999,"5min":1.976,"15min":1.976},"last":1.5189999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2629999999999999,"15min":1.2889999999999999},"min":{"1min":0.85599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7450000000000001,"5min":1.976,"15min":1.976},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.244},"min":{"1min":0.72499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7330000000000001,"5min":1.8660000000000001,"15min":1.875},"last":1.49}]},{"osd":2,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2689999999999999,"15min":1.266},"min":{"1min":0.77400000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.6579999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.365,"15min":1.369},"min":{"1min":0.99299999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.452}]},{"osd":3,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2789999999999999,"15min":1.3149999999999999},"min":{"1min":0.88,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.702,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.03},{"interface":"front","average":{"1min":1.464,"5min":1.4410000000000001,"15min":1.4219999999999999},"min":{"1min":0.874,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.752}]},{"osd":5,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.391},"min":{"1min":1.032,"5min":0.372,"15min":0.372},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4239999999999999,"15min":1.3919999999999999},"min":{"1min":0.96799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.865,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.254}]},{"osd":6,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.5009999999999999,"15min":1.46},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2530000000000001,"5min":2.2719999999999998,"15min":2.6829999999999998},"last":1.353},{"interface":"front","average":{"1min":1.635,"5min":1.593,"15min":1.5609999999999999},"min":{"1min":1.071,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.117,"5min":2.3500000000000001,"15min":2.7429999999999999},"last":1.595}]},{"osd":7,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.5369999999999999,"15min":1.516},"min":{"1min":1.1220000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0649999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.681,"15min":1.6439999999999999},"min":{"1min":1.3169999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.3439999999999999,"5min":2.508,"15min":2.508},"last":1.8340000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576009,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52064,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319776,"statfs":{"total":96636764160,"available":96583450624,"internally_reserved":0,"allocated":2899968,"data_stored":1503696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.64800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.5529999999999999,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.1759999999999999,"15min":1.1879999999999999},"min":{"1min":0.98699999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5549999999999999,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.3009999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.022,"15min":1.038},"min":{"1min":0.69699999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.716,"5min":1.79,"15min":2.4390000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2889999999999999,"15min":1.3049999999999999},"min":{"1min":0.77000000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.9119999999999999,"5min":2.109,"15min":2.4889999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1439999999999999,"15min":1.155},"min":{"1min":0.878,"5min":0.437,"15min":0.437},"max":{"1min":1.623,"5min":2.165,"15min":2.2250000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.198,"15min":1.214},"min":{"1min":0.88900000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.7130000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.60899999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.284,"15min":1.2889999999999999},"min":{"1min":0.86399999999999999,"5min":0.755,"15min":0.67500000000000004},"max":{"1min":1.532,"5min":1.857,"15min":2.5379999999999998},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.45,"15min":1.466},"min":{"1min":1.1379999999999999,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.78,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.032}]},{"osd":5,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3420000000000001,"15min":1.383},"min":{"1min":1.147,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8240000000000001,"5min":1.923,"15min":2.9740000000000002},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4390000000000001,"15min":1.4419999999999999},"min":{"1min":1.0529999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.899,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":1.1060000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4259999999999999,"15min":1.427},"min":{"1min":1.111,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.847,"5min":1.976,"15min":2.653},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4990000000000001,"15min":1.534},"min":{"1min":1.2450000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.784,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.1910000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.5580000000000001,"15min":1.5760000000000001},"min":{"1min":1.173,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9870000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4810000000000001,"15min":1.526},"min":{"1min":1.0649999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.849,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.2210000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52060,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319780,"statfs":{"total":96636764160,"available":96583454720,"internally_reserved":0,"allocated":2899968,"data_stored":1503696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.97299999999999998},"min":{"1min":0.55300000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.948},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.054,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":0.98599999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.96099999999999997,"15min":1.0029999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.198,"5min":1.6279999999999999,"15min":2.044},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.171,"15min":1.1970000000000001},"min":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.677,"5min":1.714,"15min":2.1309999999999998},"last":1.325}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68400000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4770000000000001,"5min":1.78,"15min":1.8420000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.2410000000000001,"15min":1.2769999999999999},"min":{"1min":0.66200000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.458,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.556}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3959999999999999,"15min":1.4399999999999999},"min":{"1min":0.89000000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.821,"5min":1.929,"15min":2.1789999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":1.0189999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6879999999999999,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.1539999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3819999999999999,"15min":1.411},"min":{"1min":1.0069999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.603,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":1.198},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5009999999999999,"15min":1.5349999999999999},"min":{"1min":1.0269999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.946,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.6339999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":0.997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.73,"5min":2.056,"15min":2.1240000000000001},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.4950000000000001,"15min":1.534},"min":{"1min":1.1439999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.778,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":1.375}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4690000000000001,"15min":1.486},"min":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7609999999999999,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.5309999999999999,"15min":1.5860000000000001},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.863,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":1.5009999999999999}]}]},{"osd":1,"up_from":13,"seq":55834576007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52064,"kb_used_data":2832,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319776,"statfs":{"total":96636764160,"available":96583450624,"internally_reserved":0,"allocated":2899968,"data_stored":1503696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.024,"15min":1.014},"min":{"1min":0.81000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.6200000000000001,"5min":1.6579999999999999,"15min":1.7769999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.224},"min":{"1min":0.88,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":1.79},"last":0.97199999999999998}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.97599999999999998},"min":{"1min":0.73399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2809999999999999,"5min":1.641,"15min":1.641},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.155,"15min":1.161},"min":{"1min":0.872,"5min":0.372,"15min":0.372},"max":{"1min":1.4710000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.48699999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.2430000000000001,"15min":1.264},"min":{"1min":1.052,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5800000000000001,"5min":1.651,"15min":1.7809999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1659999999999999,"15min":1.155},"min":{"1min":0.82599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.72,"5min":1.883,"15min":2.7429999999999999},"last":1.056}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":1.117,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.7110000000000001,"5min":1.7829999999999999,"15min":2.8460000000000001},"last":1.157},{"interface":"front","average":{"1min":1.357,"5min":1.3129999999999999,"15min":1.3220000000000001},"min":{"1min":1.0960000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.5449999999999999,"5min":1.6879999999999999,"15min":2.0139999999999998},"last":1.4099999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.347,"15min":1.379},"min":{"1min":0.97299999999999998,"5min":0.92600000000000005,"15min":0.76700000000000002},"max":{"1min":1.6639999999999999,"5min":1.827,"15min":2.0470000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.637,"5min":1.546,"15min":1.534},"min":{"1min":1.286,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.2610000000000001},"last":1.381}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5089999999999999,"15min":1.5149999999999999},"min":{"1min":1.2549999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.694,"5min":2.3199999999999998,"15min":2.9079999999999999},"last":1.339},{"interface":"front","average":{"1min":1.514,"5min":1.4870000000000001,"15min":1.4990000000000001},"min":{"1min":1.1679999999999999,"5min":1.046,"15min":1.0309999999999999},"max":{"1min":1.7929999999999999,"5min":1.966,"15min":2.3199999999999998},"last":1.175}]},{"osd":7,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.5780000000000001,"15min":1.556},"min":{"1min":1.2250000000000001,"5min":0.94499999999999995,"15min":0.84599999999999997},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.292},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.5529999999999999,"15min":1.5720000000000001},"min":{"1min":1.377,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.798,"5min":2.1699999999999999,"15min":2.9590000000000001},"last":1.4890000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576008,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56800,"kb_used_data":3412,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315040,"statfs":{"total":96636764160,"available":96578600960,"internally_reserved":0,"allocated":3493888,"data_stored":2094064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":54655433},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.48299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3859999999999999,"5min":1.581,"15min":1.581},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.01,"5min":1.103,"15min":1.109},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.621,"5min":1.6619999999999999,"15min":1.726},"last":0.92000000000000004}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1060000000000001,"15min":1.1419999999999999},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.745},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0329999999999999,"15min":1.073},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4199999999999999,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":0.99299999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.248,"15min":1.256},"min":{"1min":0.69799999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.546,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.046},{"interface":"front","average":{"1min":1.147,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.579,"5min":1.752,"15min":2.0979999999999999},"last":1.7689999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3220000000000001,"15min":1.349},"min":{"1min":0.83399999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.389,"15min":1.4179999999999999},"min":{"1min":0.69699999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.782,"5min":1.972,"15min":2.181},"last":2.0960000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.423,"15min":1.417},"min":{"1min":0.79600000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.732,"5min":1.919,"15min":2.1309999999999998},"last":1.802},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.482,"15min":1.486},"min":{"1min":0.79300000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.966,"5min":2.2170000000000001,"15min":2.4900000000000002},"last":2.23}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.81299999999999994,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7669999999999999,"5min":2.21,"15min":2.2149999999999999},"last":1.891},{"interface":"front","average":{"1min":1.397,"5min":1.506,"15min":1.5089999999999999},"min":{"1min":0.78000000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9430000000000001,"5min":2.0880000000000001,"15min":2.355},"last":1.2929999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5389999999999999,"15min":1.5469999999999999},"min":{"1min":0.83499999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9410000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.5289999999999999,"15min":1.554},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.966}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:13:39.222 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:13:39.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.474+0000 7f0261155640 1 Processor -- start 2023-10-08T07:13:39.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.474+0000 7f0261155640 1 -- start start 2023-10-08T07:13:39.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.475+0000 7f0261155640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f025c159ac0 0x7f025c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:39.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.476+0000 7f0261155640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c06dee0 con 0x7f025c0709e0 2023-10-08T07:13:39.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.476+0000 7f0261155640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c15d850 con 0x7f025c0ad790 2023-10-08T07:13:39.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.476+0000 7f0261155640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c15db30 con 0x7f025c159ac0 2023-10-08T07:13:39.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.476+0000 7f025ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f025c159ac0 0x7f025c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:39.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.476+0000 7f025b59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f025c0709e0 0x7f025c0c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39538/0 (socket says 172.21.15.70:39538) 2023-10-08T07:13:39.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.476+0000 7f025b59e640 1 -- 172.21.15.70:0/1883017405 learned_addr learned my addr 172.21.15.70:0/1883017405 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:39.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.476+0000 7f025ad9d640 1 -- 172.21.15.70:0/1883017405 >> v1:172.21.15.181:6789/0 conn(0x7f025c0ad790 legacy=0x7f025c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:39.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.476+0000 7f025ad9d640 1 -- 172.21.15.70:0/1883017405 >> v1:172.21.15.70:6789/0 conn(0x7f025c0709e0 legacy=0x7f025c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:39.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.476+0000 7f025ad9d640 1 -- 172.21.15.70:0/1883017405 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c15de10 con 0x7f025c159ac0 2023-10-08T07:13:39.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.476+0000 7f025a59c640 1 -- 172.21.15.70:0/1883017405 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397874425 0 0) 0x7f025c06dee0 con 0x7f025c0709e0 2023-10-08T07:13:39.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.477+0000 7f025ad9d640 1 --2- 172.21.15.70:0/1883017405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c159ac0 0x7f025c06d5e0 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f024c0097a0 tx=0x7f024c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860f07eb716821a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:39.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.478+0000 7f025a59c640 1 -- 172.21.15.70:0/1883017405 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c031830 con 0x7f025c159ac0 2023-10-08T07:13:39.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.478+0000 7f025a59c640 1 -- 172.21.15.70:0/1883017405 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c031990 con 0x7f025c159ac0 2023-10-08T07:13:39.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.478+0000 7f025a59c640 1 -- 172.21.15.70:0/1883017405 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c031d20 con 0x7f025c159ac0 2023-10-08T07:13:39.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.478+0000 7f0261155640 1 -- 172.21.15.70:0/1883017405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c159ac0 msgr2=0x7f025c06d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:39.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.478+0000 7f0261155640 1 --2- 172.21.15.70:0/1883017405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c159ac0 0x7f025c06d5e0 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f024c0097a0 tx=0x7f024c02b8d0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.478+0000 7f0261155640 1 -- 172.21.15.70:0/1883017405 shutdown_connections 2023-10-08T07:13:39.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.478+0000 7f0261155640 1 --2- 172.21.15.70:0/1883017405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c159ac0 0x7f025c06d5e0 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.478+0000 7f0261155640 1 -- 172.21.15.70:0/1883017405 >> 172.21.15.70:0/1883017405 conn(0x7f025c06a380 msgr2=0x7f025c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:39.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.478+0000 7f0261155640 1 -- 172.21.15.70:0/1883017405 shutdown_connections 2023-10-08T07:13:39.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.478+0000 7f0261155640 1 -- 172.21.15.70:0/1883017405 wait complete. 2023-10-08T07:13:39.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.480+0000 7f0261155640 1 Processor -- start 2023-10-08T07:13:39.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.481+0000 7f0261155640 1 -- start start 2023-10-08T07:13:39.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.481+0000 7f0261155640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c0709e0 0x7f025c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:39.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f0261155640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025c0ad790 0x7f025c1f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:39.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f0261155640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f025c159ac0 0x7f025c155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:39.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f0261155640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c15b1a0 con 0x7f025c0709e0 2023-10-08T07:13:39.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f0261155640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c15b440 con 0x7f025c159ac0 2023-10-08T07:13:39.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f0261155640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c1f2e70 con 0x7f025c0ad790 2023-10-08T07:13:39.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f025ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c0709e0 0x7f025c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:39.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f025ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c0709e0 0x7f025c15cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37102/0 (socket says 172.21.15.70:37102) 2023-10-08T07:13:39.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f025ad9d640 1 -- 172.21.15.70:0/3855479028 learned_addr learned my addr 172.21.15.70:0/3855479028 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:39.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f025ad9d640 1 -- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025c0ad790 msgr2=0x7f025c1f25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:39.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f025ad9d640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025c0ad790 0x7f025c1f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f025ad9d640 1 -- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f025c159ac0 msgr2=0x7f025c155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:39.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f025ad9d640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f025c159ac0 0x7f025c155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.482+0000 7f025ad9d640 1 -- 172.21.15.70:0/3855479028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c009400 con 0x7f025c0709e0 2023-10-08T07:13:39.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.483+0000 7f025ad9d640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c0709e0 0x7f025c15cc50 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f024c0098c0 tx=0x7f024c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:39.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.483+0000 7f02537fe640 1 -- 172.21.15.70:0/3855479028 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c03ba70 con 0x7f025c0709e0 2023-10-08T07:13:39.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.483+0000 7f02537fe640 1 -- 172.21.15.70:0/3855479028 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c03bbf0 con 0x7f025c0709e0 2023-10-08T07:13:39.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.483+0000 7f02537fe640 1 -- 172.21.15.70:0/3855479028 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c0453f0 con 0x7f025c0709e0 2023-10-08T07:13:39.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.483+0000 7f0261155640 1 -- 172.21.15.70:0/3855479028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c1f3150 con 0x7f025c0709e0 2023-10-08T07:13:39.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.483+0000 7f0261155640 1 -- 172.21.15.70:0/3855479028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c1f2b20 con 0x7f025c0709e0 2023-10-08T07:13:39.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.484+0000 7f0261155640 1 -- 172.21.15.70:0/3855479028 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c067490 con 0x7f025c0709e0 2023-10-08T07:13:39.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.485+0000 7f02537fe640 1 -- 172.21.15.70:0/3855479028 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f024c04e450 con 0x7f025c0709e0 2023-10-08T07:13:39.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.486+0000 7f02537fe640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f023c06af90 0x7f023c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:39.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.486+0000 7f02537fe640 1 -- 172.21.15.70:0/3855479028 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f024c0abef0 con 0x7f025c0709e0 2023-10-08T07:13:39.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.487+0000 7f0253fff640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f023c06af90 0x7f023c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:39.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.488+0000 7f0253fff640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f023c06af90 0x7f023c06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f025c0b3060 tx=0x7f0244007990 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:39.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.491+0000 7f02537fe640 1 -- 172.21.15.70:0/3855479028 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f024c076c40 con 0x7f025c0709e0 2023-10-08T07:13:39.589 INFO:tasks.ceph.mds.a.smithi070.stderr:2023-10-08T07:13:39.587+0000 7f907293a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:13:39.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.677+0000 7f0261155640 1 -- 172.21.15.70:0/3855479028 --> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f025c15b580 con 0x7f023c06af90 2023-10-08T07:13:39.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.700+0000 7f02537fe640 1 -- 172.21.15.70:0/3855479028 <== mgr.7328 v2:172.21.15.181:6832/3993370366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286932 (secure 0 0 0) 0x7f02440050d0 con 0x7f023c06af90 2023-10-08T07:13:39.704 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:39.706 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:13:39.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.705+0000 7f0261155640 1 -- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f023c06af90 msgr2=0x7f023c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.705+0000 7f0261155640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f023c06af90 0x7f023c06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f025c0b3060 tx=0x7f0244007990 comp rx=0 tx=0).stop 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.705+0000 7f0261155640 1 -- 172.21.15.70:0/3855479028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c0709e0 msgr2=0x7f025c15cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.705+0000 7f0261155640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c0709e0 0x7f025c15cc50 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f024c0098c0 tx=0x7f024c0046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.706+0000 7f0261155640 1 -- 172.21.15.70:0/3855479028 shutdown_connections 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.706+0000 7f0261155640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f025c159ac0 0x7f025c155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.706+0000 7f0261155640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025c0ad790 0x7f025c1f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.706+0000 7f0261155640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f023c06af90 0x7f023c06d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.706+0000 7f0261155640 1 --2- 172.21.15.70:0/3855479028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f025c0709e0 0x7f025c15cc50 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.706+0000 7f0261155640 1 -- 172.21.15.70:0/3855479028 >> 172.21.15.70:0/3855479028 conn(0x7f025c06a380 msgr2=0x7f025c158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.706+0000 7f0261155640 1 -- 172.21.15.70:0/3855479028 shutdown_connections 2023-10-08T07:13:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.706+0000 7f0261155640 1 -- 172.21.15.70:0/3855479028 wait complete. 2023-10-08T07:13:39.728 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-10-08T07:13:39.505035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":633,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2844,"ondisk_log_size":2844,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":433260,"kb_used_data":24168,"kb_used_omap":12,"kb_used_meta":408947,"kb_avail":754541460,"statfs":{"total":773094113280,"available":772650455040,"internally_reserved":0,"allocated":24748032,"data_stored":13380388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":418762299},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001990"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471334+0000","last_change":"2023-10-08T07:13:34.308897+0000","last_active":"2023-10-08T07:13:35.471334+0000","last_peered":"2023-10-08T07:13:35.471334+0000","last_clean":"2023-10-08T07:13:35.471334+0000","last_became_active":"2023-10-08T07:13:34.308474+0000","last_became_peered":"2023-10-08T07:13:34.308474+0000","last_unstale":"2023-10-08T07:13:35.471334+0000","last_undegraded":"2023-10-08T07:13:35.471334+0000","last_fullsized":"2023-10-08T07:13:35.471334+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:06.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471329+0000","last_change":"2023-10-08T07:13:34.295062+0000","last_active":"2023-10-08T07:13:35.471329+0000","last_peered":"2023-10-08T07:13:35.471329+0000","last_clean":"2023-10-08T07:13:35.471329+0000","last_became_active":"2023-10-08T07:13:34.294747+0000","last_became_peered":"2023-10-08T07:13:34.294747+0000","last_unstale":"2023-10-08T07:13:35.471329+0000","last_undegraded":"2023-10-08T07:13:35.471329+0000","last_fullsized":"2023-10-08T07:13:35.471329+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:26.407053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.470784+0000","last_change":"2023-10-08T07:13:34.306582+0000","last_active":"2023-10-08T07:13:35.470784+0000","last_peered":"2023-10-08T07:13:35.470784+0000","last_clean":"2023-10-08T07:13:35.470784+0000","last_became_active":"2023-10-08T07:13:34.306310+0000","last_became_peered":"2023-10-08T07:13:34.306310+0000","last_unstale":"2023-10-08T07:13:35.470784+0000","last_undegraded":"2023-10-08T07:13:35.470784+0000","last_fullsized":"2023-10-08T07:13:35.470784+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.470552+0000","last_change":"2023-10-08T07:13:34.301326+0000","last_active":"2023-10-08T07:13:35.470552+0000","last_peered":"2023-10-08T07:13:35.470552+0000","last_clean":"2023-10-08T07:13:35.470552+0000","last_became_active":"2023-10-08T07:13:34.299365+0000","last_became_peered":"2023-10-08T07:13:34.299365+0000","last_unstale":"2023-10-08T07:13:35.470552+0000","last_undegraded":"2023-10-08T07:13:35.470552+0000","last_fullsized":"2023-10-08T07:13:35.470552+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:02.576748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469312+0000","last_change":"2023-10-08T07:13:34.298025+0000","last_active":"2023-10-08T07:13:35.469312+0000","last_peered":"2023-10-08T07:13:35.469312+0000","last_clean":"2023-10-08T07:13:35.469312+0000","last_became_active":"2023-10-08T07:13:34.296181+0000","last_became_peered":"2023-10-08T07:13:34.296181+0000","last_unstale":"2023-10-08T07:13:35.469312+0000","last_undegraded":"2023-10-08T07:13:35.469312+0000","last_fullsized":"2023-10-08T07:13:35.469312+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469647+0000","last_change":"2023-10-08T07:13:34.298110+0000","last_active":"2023-10-08T07:13:35.469647+0000","last_peered":"2023-10-08T07:13:35.469647+0000","last_clean":"2023-10-08T07:13:35.469647+0000","last_became_active":"2023-10-08T07:13:34.297806+0000","last_became_peered":"2023-10-08T07:13:34.297806+0000","last_unstale":"2023-10-08T07:13:35.469647+0000","last_undegraded":"2023-10-08T07:13:35.469647+0000","last_fullsized":"2023-10-08T07:13:35.469647+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:58.335295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.468793+0000","last_change":"2023-10-08T07:13:34.293550+0000","last_active":"2023-10-08T07:13:35.468793+0000","last_peered":"2023-10-08T07:13:35.468793+0000","last_clean":"2023-10-08T07:13:35.468793+0000","last_became_active":"2023-10-08T07:13:34.293202+0000","last_became_peered":"2023-10-08T07:13:34.293202+0000","last_unstale":"2023-10-08T07:13:35.468793+0000","last_undegraded":"2023-10-08T07:13:35.468793+0000","last_fullsized":"2023-10-08T07:13:35.468793+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:00.842346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.274952+0000","last_change":"2023-10-08T07:13:34.302018+0000","last_active":"2023-10-08T07:13:35.274952+0000","last_peered":"2023-10-08T07:13:35.274952+0000","last_clean":"2023-10-08T07:13:35.274952+0000","last_became_active":"2023-10-08T07:13:34.301747+0000","last_became_peered":"2023-10-08T07:13:34.301747+0000","last_unstale":"2023-10-08T07:13:35.274952+0000","last_undegraded":"2023-10-08T07:13:35.274952+0000","last_fullsized":"2023-10-08T07:13:35.274952+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.272006+0000","last_change":"2023-10-08T07:13:34.289375+0000","last_active":"2023-10-08T07:13:35.272006+0000","last_peered":"2023-10-08T07:13:35.272006+0000","last_clean":"2023-10-08T07:13:35.272006+0000","last_became_active":"2023-10-08T07:13:34.289115+0000","last_became_peered":"2023-10-08T07:13:34.289115+0000","last_unstale":"2023-10-08T07:13:35.272006+0000","last_undegraded":"2023-10-08T07:13:35.272006+0000","last_fullsized":"2023-10-08T07:13:35.272006+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.278134+0000","last_change":"2023-10-08T07:13:34.295991+0000","last_active":"2023-10-08T07:13:35.278134+0000","last_peered":"2023-10-08T07:13:35.278134+0000","last_clean":"2023-10-08T07:13:35.278134+0000","last_became_active":"2023-10-08T07:13:34.295717+0000","last_became_peered":"2023-10-08T07:13:34.295717+0000","last_unstale":"2023-10-08T07:13:35.278134+0000","last_undegraded":"2023-10-08T07:13:35.278134+0000","last_fullsized":"2023-10-08T07:13:35.278134+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.797828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471102+0000","last_change":"2023-10-08T07:13:34.308930+0000","last_active":"2023-10-08T07:13:35.471102+0000","last_peered":"2023-10-08T07:13:35.471102+0000","last_clean":"2023-10-08T07:13:35.471102+0000","last_became_active":"2023-10-08T07:13:34.308283+0000","last_became_peered":"2023-10-08T07:13:34.308283+0000","last_unstale":"2023-10-08T07:13:35.471102+0000","last_undegraded":"2023-10-08T07:13:35.471102+0000","last_fullsized":"2023-10-08T07:13:35.471102+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.468977+0000","last_change":"2023-10-08T07:13:34.294253+0000","last_active":"2023-10-08T07:13:35.468977+0000","last_peered":"2023-10-08T07:13:35.468977+0000","last_clean":"2023-10-08T07:13:35.468977+0000","last_became_active":"2023-10-08T07:13:34.291997+0000","last_became_peered":"2023-10-08T07:13:34.291997+0000","last_unstale":"2023-10-08T07:13:35.468977+0000","last_undegraded":"2023-10-08T07:13:35.468977+0000","last_fullsized":"2023-10-08T07:13:35.468977+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.470304+0000","last_change":"2023-10-08T07:13:34.306040+0000","last_active":"2023-10-08T07:13:35.470304+0000","last_peered":"2023-10-08T07:13:35.470304+0000","last_clean":"2023-10-08T07:13:35.470304+0000","last_became_active":"2023-10-08T07:13:34.303015+0000","last_became_peered":"2023-10-08T07:13:34.303015+0000","last_unstale":"2023-10-08T07:13:35.470304+0000","last_undegraded":"2023-10-08T07:13:35.470304+0000","last_fullsized":"2023-10-08T07:13:35.470304+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:42.884365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.275619+0000","last_change":"2023-10-08T07:13:34.298407+0000","last_active":"2023-10-08T07:13:35.275619+0000","last_peered":"2023-10-08T07:13:35.275619+0000","last_clean":"2023-10-08T07:13:35.275619+0000","last_became_active":"2023-10-08T07:13:34.298139+0000","last_became_peered":"2023-10-08T07:13:34.298139+0000","last_unstale":"2023-10-08T07:13:35.275619+0000","last_undegraded":"2023-10-08T07:13:35.275619+0000","last_fullsized":"2023-10-08T07:13:35.275619+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.275285+0000","last_change":"2023-10-08T07:13:34.309619+0000","last_active":"2023-10-08T07:13:35.275285+0000","last_peered":"2023-10-08T07:13:35.275285+0000","last_clean":"2023-10-08T07:13:35.275285+0000","last_became_active":"2023-10-08T07:13:34.308202+0000","last_became_peered":"2023-10-08T07:13:34.308202+0000","last_unstale":"2023-10-08T07:13:35.275285+0000","last_undegraded":"2023-10-08T07:13:35.275285+0000","last_fullsized":"2023-10-08T07:13:35.275285+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:59.914799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471793+0000","last_change":"2023-10-08T07:13:34.300821+0000","last_active":"2023-10-08T07:13:35.471793+0000","last_peered":"2023-10-08T07:13:35.471793+0000","last_clean":"2023-10-08T07:13:35.471793+0000","last_became_active":"2023-10-08T07:13:34.300558+0000","last_became_peered":"2023-10-08T07:13:34.300558+0000","last_unstale":"2023-10-08T07:13:35.471793+0000","last_undegraded":"2023-10-08T07:13:35.471793+0000","last_fullsized":"2023-10-08T07:13:35.471793+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.272185+0000","last_change":"2023-10-08T07:13:34.308624+0000","last_active":"2023-10-08T07:13:35.272185+0000","last_peered":"2023-10-08T07:13:35.272185+0000","last_clean":"2023-10-08T07:13:35.272185+0000","last_became_active":"2023-10-08T07:13:34.308246+0000","last_became_peered":"2023-10-08T07:13:34.308246+0000","last_unstale":"2023-10-08T07:13:35.272185+0000","last_undegraded":"2023-10-08T07:13:35.272185+0000","last_fullsized":"2023-10-08T07:13:35.272185+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.271902+0000","last_change":"2023-10-08T07:13:34.299238+0000","last_active":"2023-10-08T07:13:35.271902+0000","last_peered":"2023-10-08T07:13:35.271902+0000","last_clean":"2023-10-08T07:13:35.271902+0000","last_became_active":"2023-10-08T07:13:34.298585+0000","last_became_peered":"2023-10-08T07:13:34.298585+0000","last_unstale":"2023-10-08T07:13:35.271902+0000","last_undegraded":"2023-10-08T07:13:35.271902+0000","last_fullsized":"2023-10-08T07:13:35.271902+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.278013+0000","last_change":"2023-10-08T07:13:34.295222+0000","last_active":"2023-10-08T07:13:35.278013+0000","last_peered":"2023-10-08T07:13:35.278013+0000","last_clean":"2023-10-08T07:13:35.278013+0000","last_became_active":"2023-10-08T07:13:34.289517+0000","last_became_peered":"2023-10-08T07:13:34.289517+0000","last_unstale":"2023-10-08T07:13:35.278013+0000","last_undegraded":"2023-10-08T07:13:35.278013+0000","last_fullsized":"2023-10-08T07:13:35.278013+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:38.224838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.277816+0000","last_change":"2023-10-08T07:13:34.297677+0000","last_active":"2023-10-08T07:13:35.277816+0000","last_peered":"2023-10-08T07:13:35.277816+0000","last_clean":"2023-10-08T07:13:35.277816+0000","last_became_active":"2023-10-08T07:13:34.296526+0000","last_became_peered":"2023-10-08T07:13:34.296526+0000","last_unstale":"2023-10-08T07:13:35.277816+0000","last_undegraded":"2023-10-08T07:13:35.277816+0000","last_fullsized":"2023-10-08T07:13:35.277816+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:51.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.276555+0000","last_change":"2023-10-08T07:13:34.294431+0000","last_active":"2023-10-08T07:13:35.276555+0000","last_peered":"2023-10-08T07:13:35.276555+0000","last_clean":"2023-10-08T07:13:35.276555+0000","last_became_active":"2023-10-08T07:13:34.294129+0000","last_became_peered":"2023-10-08T07:13:34.294129+0000","last_unstale":"2023-10-08T07:13:35.276555+0000","last_undegraded":"2023-10-08T07:13:35.276555+0000","last_fullsized":"2023-10-08T07:13:35.276555+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.276212+0000","last_change":"2023-10-08T07:13:34.297774+0000","last_active":"2023-10-08T07:13:35.276212+0000","last_peered":"2023-10-08T07:13:35.276212+0000","last_clean":"2023-10-08T07:13:35.276212+0000","last_became_active":"2023-10-08T07:13:34.297492+0000","last_became_peered":"2023-10-08T07:13:34.297492+0000","last_unstale":"2023-10-08T07:13:35.276212+0000","last_undegraded":"2023-10-08T07:13:35.276212+0000","last_fullsized":"2023-10-08T07:13:35.276212+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469455+0000","last_change":"2023-10-08T07:13:34.291585+0000","last_active":"2023-10-08T07:13:35.469455+0000","last_peered":"2023-10-08T07:13:35.469455+0000","last_clean":"2023-10-08T07:13:35.469455+0000","last_became_active":"2023-10-08T07:13:34.291125+0000","last_became_peered":"2023-10-08T07:13:34.291125+0000","last_unstale":"2023-10-08T07:13:35.469455+0000","last_undegraded":"2023-10-08T07:13:35.469455+0000","last_fullsized":"2023-10-08T07:13:35.469455+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:14.399506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469612+0000","last_change":"2023-10-08T07:13:34.293797+0000","last_active":"2023-10-08T07:13:35.469612+0000","last_peered":"2023-10-08T07:13:35.469612+0000","last_clean":"2023-10-08T07:13:35.469612+0000","last_became_active":"2023-10-08T07:13:34.289224+0000","last_became_peered":"2023-10-08T07:13:34.289224+0000","last_unstale":"2023-10-08T07:13:35.469612+0000","last_undegraded":"2023-10-08T07:13:35.469612+0000","last_fullsized":"2023-10-08T07:13:35.469612+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.272174+0000","last_change":"2023-10-08T07:13:34.298953+0000","last_active":"2023-10-08T07:13:35.272174+0000","last_peered":"2023-10-08T07:13:35.272174+0000","last_clean":"2023-10-08T07:13:35.272174+0000","last_became_active":"2023-10-08T07:13:34.298694+0000","last_became_peered":"2023-10-08T07:13:34.298694+0000","last_unstale":"2023-10-08T07:13:35.272174+0000","last_undegraded":"2023-10-08T07:13:35.272174+0000","last_fullsized":"2023-10-08T07:13:35.272174+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469793+0000","last_change":"2023-10-08T07:13:34.304704+0000","last_active":"2023-10-08T07:13:35.469793+0000","last_peered":"2023-10-08T07:13:35.469793+0000","last_clean":"2023-10-08T07:13:35.469793+0000","last_became_active":"2023-10-08T07:13:34.304032+0000","last_became_peered":"2023-10-08T07:13:34.304032+0000","last_unstale":"2023-10-08T07:13:35.469793+0000","last_undegraded":"2023-10-08T07:13:35.469793+0000","last_fullsized":"2023-10-08T07:13:35.469793+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:15.818894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471978+0000","last_change":"2023-10-08T07:13:34.310969+0000","last_active":"2023-10-08T07:13:35.471978+0000","last_peered":"2023-10-08T07:13:35.471978+0000","last_clean":"2023-10-08T07:13:35.471978+0000","last_became_active":"2023-10-08T07:13:34.310679+0000","last_became_peered":"2023-10-08T07:13:34.310679+0000","last_unstale":"2023-10-08T07:13:35.471978+0000","last_undegraded":"2023-10-08T07:13:35.471978+0000","last_fullsized":"2023-10-08T07:13:35.471978+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469359+0000","last_change":"2023-10-08T07:13:34.297816+0000","last_active":"2023-10-08T07:13:35.469359+0000","last_peered":"2023-10-08T07:13:35.469359+0000","last_clean":"2023-10-08T07:13:35.469359+0000","last_became_active":"2023-10-08T07:13:34.297544+0000","last_became_peered":"2023-10-08T07:13:34.297544+0000","last_unstale":"2023-10-08T07:13:35.469359+0000","last_undegraded":"2023-10-08T07:13:35.469359+0000","last_fullsized":"2023-10-08T07:13:35.469359+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.271404+0000","last_change":"2023-10-08T07:13:34.293712+0000","last_active":"2023-10-08T07:13:35.271404+0000","last_peered":"2023-10-08T07:13:35.271404+0000","last_clean":"2023-10-08T07:13:35.271404+0000","last_became_active":"2023-10-08T07:13:34.293415+0000","last_became_peered":"2023-10-08T07:13:34.293415+0000","last_unstale":"2023-10-08T07:13:35.271404+0000","last_undegraded":"2023-10-08T07:13:35.271404+0000","last_fullsized":"2023-10-08T07:13:35.271404+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:47.966787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.275373+0000","last_change":"2023-10-08T07:13:34.300974+0000","last_active":"2023-10-08T07:13:35.275373+0000","last_peered":"2023-10-08T07:13:35.275373+0000","last_clean":"2023-10-08T07:13:35.275373+0000","last_became_active":"2023-10-08T07:13:34.300529+0000","last_became_peered":"2023-10-08T07:13:34.300529+0000","last_unstale":"2023-10-08T07:13:35.275373+0000","last_undegraded":"2023-10-08T07:13:35.275373+0000","last_fullsized":"2023-10-08T07:13:35.275373+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:17.001444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.470529+0000","last_change":"2023-10-08T07:13:34.301071+0000","last_active":"2023-10-08T07:13:35.470529+0000","last_peered":"2023-10-08T07:13:35.470529+0000","last_clean":"2023-10-08T07:13:35.470529+0000","last_became_active":"2023-10-08T07:13:34.300772+0000","last_became_peered":"2023-10-08T07:13:34.300772+0000","last_unstale":"2023-10-08T07:13:35.470529+0000","last_undegraded":"2023-10-08T07:13:35.470529+0000","last_fullsized":"2023-10-08T07:13:35.470529+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:59:54.643977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469532+0000","last_change":"2023-10-08T07:13:34.295873+0000","last_active":"2023-10-08T07:13:35.469532+0000","last_peered":"2023-10-08T07:13:35.469532+0000","last_clean":"2023-10-08T07:13:35.469532+0000","last_became_active":"2023-10-08T07:13:34.295546+0000","last_became_peered":"2023-10-08T07:13:34.295546+0000","last_unstale":"2023-10-08T07:13:35.469532+0000","last_undegraded":"2023-10-08T07:13:35.469532+0000","last_fullsized":"2023-10-08T07:13:35.469532+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:46:53.896975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.470559+0000","last_change":"2023-10-08T07:13:34.286367+0000","last_active":"2023-10-08T07:13:35.470559+0000","last_peered":"2023-10-08T07:13:35.470559+0000","last_clean":"2023-10-08T07:13:35.470559+0000","last_became_active":"2023-10-08T07:13:34.285016+0000","last_became_peered":"2023-10-08T07:13:34.285016+0000","last_unstale":"2023-10-08T07:13:35.470559+0000","last_undegraded":"2023-10-08T07:13:35.470559+0000","last_fullsized":"2023-10-08T07:13:35.470559+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:41.466102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"233'4","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.737722+0000","last_change":"2023-10-08T07:13:33.743017+0000","last_active":"2023-10-08T07:13:34.737722+0000","last_peered":"2023-10-08T07:13:34.737722+0000","last_clean":"2023-10-08T07:13:34.737722+0000","last_became_active":"2023-10-08T07:13:33.742582+0000","last_became_peered":"2023-10-08T07:13:33.742582+0000","last_unstale":"2023-10-08T07:13:34.737722+0000","last_undegraded":"2023-10-08T07:13:34.737722+0000","last_fullsized":"2023-10-08T07:13:34.737722+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:38:56.287497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469948+0000","last_change":"2023-10-08T07:13:34.293829+0000","last_active":"2023-10-08T07:13:35.469948+0000","last_peered":"2023-10-08T07:13:35.469948+0000","last_clean":"2023-10-08T07:13:35.469948+0000","last_became_active":"2023-10-08T07:13:34.293552+0000","last_became_peered":"2023-10-08T07:13:34.293552+0000","last_unstale":"2023-10-08T07:13:35.469948+0000","last_undegraded":"2023-10-08T07:13:35.469948+0000","last_fullsized":"2023-10-08T07:13:35.469948+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:38.485847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"233'1","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.731121+0000","last_change":"2023-10-08T07:13:33.732312+0000","last_active":"2023-10-08T07:13:34.731121+0000","last_peered":"2023-10-08T07:13:34.731121+0000","last_clean":"2023-10-08T07:13:34.731121+0000","last_became_active":"2023-10-08T07:13:33.731998+0000","last_became_peered":"2023-10-08T07:13:33.731998+0000","last_unstale":"2023-10-08T07:13:34.731121+0000","last_undegraded":"2023-10-08T07:13:34.731121+0000","last_fullsized":"2023-10-08T07:13:34.731121+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:10:53.964103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.276594+0000","last_change":"2023-10-08T07:13:34.301668+0000","last_active":"2023-10-08T07:13:35.276594+0000","last_peered":"2023-10-08T07:13:35.276594+0000","last_clean":"2023-10-08T07:13:35.276594+0000","last_became_active":"2023-10-08T07:13:34.298698+0000","last_became_peered":"2023-10-08T07:13:34.298698+0000","last_unstale":"2023-10-08T07:13:35.276594+0000","last_undegraded":"2023-10-08T07:13:35.276594+0000","last_fullsized":"2023-10-08T07:13:35.276594+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"233'2","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.727815+0000","last_change":"2023-10-08T07:13:33.724335+0000","last_active":"2023-10-08T07:13:34.727815+0000","last_peered":"2023-10-08T07:13:34.727815+0000","last_clean":"2023-10-08T07:13:34.727815+0000","last_became_active":"2023-10-08T07:13:33.724043+0000","last_became_peered":"2023-10-08T07:13:33.724043+0000","last_unstale":"2023-10-08T07:13:34.727815+0000","last_undegraded":"2023-10-08T07:13:34.727815+0000","last_fullsized":"2023-10-08T07:13:34.727815+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:58.128241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469430+0000","last_change":"2023-10-08T07:13:34.293354+0000","last_active":"2023-10-08T07:13:35.469430+0000","last_peered":"2023-10-08T07:13:35.469430+0000","last_clean":"2023-10-08T07:13:35.469430+0000","last_became_active":"2023-10-08T07:13:34.288220+0000","last_became_peered":"2023-10-08T07:13:34.288220+0000","last_unstale":"2023-10-08T07:13:35.469430+0000","last_undegraded":"2023-10-08T07:13:35.469430+0000","last_fullsized":"2023-10-08T07:13:35.469430+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"233'2","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.731584+0000","last_change":"2023-10-08T07:13:33.736823+0000","last_active":"2023-10-08T07:13:34.731584+0000","last_peered":"2023-10-08T07:13:34.731584+0000","last_clean":"2023-10-08T07:13:34.731584+0000","last_became_active":"2023-10-08T07:13:33.736537+0000","last_became_peered":"2023-10-08T07:13:33.736537+0000","last_unstale":"2023-10-08T07:13:34.731584+0000","last_undegraded":"2023-10-08T07:13:34.731584+0000","last_fullsized":"2023-10-08T07:13:34.731584+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:48.798820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.276009+0000","last_change":"2023-10-08T07:13:34.302593+0000","last_active":"2023-10-08T07:13:35.276009+0000","last_peered":"2023-10-08T07:13:35.276009+0000","last_clean":"2023-10-08T07:13:35.276009+0000","last_became_active":"2023-10-08T07:13:34.302300+0000","last_became_peered":"2023-10-08T07:13:34.302300+0000","last_unstale":"2023-10-08T07:13:35.276009+0000","last_undegraded":"2023-10-08T07:13:35.276009+0000","last_fullsized":"2023-10-08T07:13:35.276009+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:13:33.465222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"233'1","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.730362+0000","last_change":"2023-10-08T07:13:33.738971+0000","last_active":"2023-10-08T07:13:34.730362+0000","last_peered":"2023-10-08T07:13:34.730362+0000","last_clean":"2023-10-08T07:13:34.730362+0000","last_became_active":"2023-10-08T07:13:33.738532+0000","last_became_peered":"2023-10-08T07:13:33.738532+0000","last_unstale":"2023-10-08T07:13:34.730362+0000","last_undegraded":"2023-10-08T07:13:34.730362+0000","last_fullsized":"2023-10-08T07:13:34.730362+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:06.806347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.276703+0000","last_change":"2023-10-08T07:13:34.304888+0000","last_active":"2023-10-08T07:13:35.276703+0000","last_peered":"2023-10-08T07:13:35.276703+0000","last_clean":"2023-10-08T07:13:35.276703+0000","last_became_active":"2023-10-08T07:13:34.304536+0000","last_became_peered":"2023-10-08T07:13:34.304536+0000","last_unstale":"2023-10-08T07:13:35.276703+0000","last_undegraded":"2023-10-08T07:13:35.276703+0000","last_fullsized":"2023-10-08T07:13:35.276703+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:05.467048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.292013+0000","last_change":"2023-10-08T07:13:33.738155+0000","last_active":"2023-10-08T07:13:34.292013+0000","last_peered":"2023-10-08T07:13:34.292013+0000","last_clean":"2023-10-08T07:13:34.292013+0000","last_became_active":"2023-10-08T07:13:33.737850+0000","last_became_peered":"2023-10-08T07:13:33.737850+0000","last_unstale":"2023-10-08T07:13:34.292013+0000","last_undegraded":"2023-10-08T07:13:34.292013+0000","last_fullsized":"2023-10-08T07:13:34.292013+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:37.479840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.272376+0000","last_change":"2023-10-08T07:13:34.294260+0000","last_active":"2023-10-08T07:13:35.272376+0000","last_peered":"2023-10-08T07:13:35.272376+0000","last_clean":"2023-10-08T07:13:35.272376+0000","last_became_active":"2023-10-08T07:13:34.293992+0000","last_became_peered":"2023-10-08T07:13:34.293992+0000","last_unstale":"2023-10-08T07:13:35.272376+0000","last_undegraded":"2023-10-08T07:13:35.272376+0000","last_fullsized":"2023-10-08T07:13:35.272376+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.288086+0000","last_change":"2023-10-08T07:13:33.731054+0000","last_active":"2023-10-08T07:13:34.288086+0000","last_peered":"2023-10-08T07:13:34.288086+0000","last_clean":"2023-10-08T07:13:34.288086+0000","last_became_active":"2023-10-08T07:13:33.730782+0000","last_became_peered":"2023-10-08T07:13:33.730782+0000","last_unstale":"2023-10-08T07:13:34.288086+0000","last_undegraded":"2023-10-08T07:13:34.288086+0000","last_fullsized":"2023-10-08T07:13:34.288086+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:06.476515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.274954+0000","last_change":"2023-10-08T07:13:34.294204+0000","last_active":"2023-10-08T07:13:35.274954+0000","last_peered":"2023-10-08T07:13:35.274954+0000","last_clean":"2023-10-08T07:13:35.274954+0000","last_became_active":"2023-10-08T07:13:34.293887+0000","last_became_peered":"2023-10-08T07:13:34.293887+0000","last_unstale":"2023-10-08T07:13:35.274954+0000","last_undegraded":"2023-10-08T07:13:35.274954+0000","last_fullsized":"2023-10-08T07:13:35.274954+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:59.699025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.276064+0000","last_change":"2023-10-08T07:13:33.735184+0000","last_active":"2023-10-08T07:13:34.276064+0000","last_peered":"2023-10-08T07:13:34.276064+0000","last_clean":"2023-10-08T07:13:34.276064+0000","last_became_active":"2023-10-08T07:13:33.734915+0000","last_became_peered":"2023-10-08T07:13:33.734915+0000","last_unstale":"2023-10-08T07:13:34.276064+0000","last_undegraded":"2023-10-08T07:13:34.276064+0000","last_fullsized":"2023-10-08T07:13:34.276064+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:52.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.470712+0000","last_change":"2023-10-08T07:13:34.302391+0000","last_active":"2023-10-08T07:13:35.470712+0000","last_peered":"2023-10-08T07:13:35.470712+0000","last_clean":"2023-10-08T07:13:35.470712+0000","last_became_active":"2023-10-08T07:13:34.302105+0000","last_became_peered":"2023-10-08T07:13:34.302105+0000","last_unstale":"2023-10-08T07:13:35.470712+0000","last_undegraded":"2023-10-08T07:13:35.470712+0000","last_fullsized":"2023-10-08T07:13:35.470712+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.294591+0000","last_change":"2023-10-08T07:13:33.736423+0000","last_active":"2023-10-08T07:13:34.294591+0000","last_peered":"2023-10-08T07:13:34.294591+0000","last_clean":"2023-10-08T07:13:34.294591+0000","last_became_active":"2023-10-08T07:13:33.735959+0000","last_became_peered":"2023-10-08T07:13:33.735959+0000","last_unstale":"2023-10-08T07:13:34.294591+0000","last_undegraded":"2023-10-08T07:13:34.294591+0000","last_fullsized":"2023-10-08T07:13:34.294591+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:00.365787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.271794+0000","last_change":"2023-10-08T07:13:34.290971+0000","last_active":"2023-10-08T07:13:35.271794+0000","last_peered":"2023-10-08T07:13:35.271794+0000","last_clean":"2023-10-08T07:13:35.271794+0000","last_became_active":"2023-10-08T07:13:34.290294+0000","last_became_peered":"2023-10-08T07:13:34.290294+0000","last_unstale":"2023-10-08T07:13:35.271794+0000","last_undegraded":"2023-10-08T07:13:35.271794+0000","last_fullsized":"2023-10-08T07:13:35.271794+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"233'3","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.727270+0000","last_change":"2023-10-08T07:13:33.726791+0000","last_active":"2023-10-08T07:13:34.727270+0000","last_peered":"2023-10-08T07:13:34.727270+0000","last_clean":"2023-10-08T07:13:34.727270+0000","last_became_active":"2023-10-08T07:13:33.726482+0000","last_became_peered":"2023-10-08T07:13:33.726482+0000","last_unstale":"2023-10-08T07:13:34.727270+0000","last_undegraded":"2023-10-08T07:13:34.727270+0000","last_fullsized":"2023-10-08T07:13:34.727270+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:11.409539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469693+0000","last_change":"2023-10-08T07:13:34.293551+0000","last_active":"2023-10-08T07:13:35.469693+0000","last_peered":"2023-10-08T07:13:35.469693+0000","last_clean":"2023-10-08T07:13:35.469693+0000","last_became_active":"2023-10-08T07:13:34.292471+0000","last_became_peered":"2023-10-08T07:13:34.292471+0000","last_unstale":"2023-10-08T07:13:35.469693+0000","last_undegraded":"2023-10-08T07:13:35.469693+0000","last_fullsized":"2023-10-08T07:13:35.469693+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"233'6","reported_seq":18,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.740887+0000","last_change":"2023-10-08T07:13:33.746002+0000","last_active":"2023-10-08T07:13:34.740887+0000","last_peered":"2023-10-08T07:13:34.740887+0000","last_clean":"2023-10-08T07:13:34.740887+0000","last_became_active":"2023-10-08T07:13:33.745697+0000","last_became_peered":"2023-10-08T07:13:33.745697+0000","last_unstale":"2023-10-08T07:13:34.740887+0000","last_undegraded":"2023-10-08T07:13:34.740887+0000","last_fullsized":"2023-10-08T07:13:34.740887+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:25.841040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471489+0000","last_change":"2023-10-08T07:13:34.293585+0000","last_active":"2023-10-08T07:13:35.471489+0000","last_peered":"2023-10-08T07:13:35.471489+0000","last_clean":"2023-10-08T07:13:35.471489+0000","last_became_active":"2023-10-08T07:13:34.293318+0000","last_became_peered":"2023-10-08T07:13:34.293318+0000","last_unstale":"2023-10-08T07:13:35.471489+0000","last_undegraded":"2023-10-08T07:13:35.471489+0000","last_fullsized":"2023-10-08T07:13:35.471489+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:09:34.022529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"233'2","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.732085+0000","last_change":"2023-10-08T07:13:33.730955+0000","last_active":"2023-10-08T07:13:34.732085+0000","last_peered":"2023-10-08T07:13:34.732085+0000","last_clean":"2023-10-08T07:13:34.732085+0000","last_became_active":"2023-10-08T07:13:33.730676+0000","last_became_peered":"2023-10-08T07:13:33.730676+0000","last_unstale":"2023-10-08T07:13:34.732085+0000","last_undegraded":"2023-10-08T07:13:34.732085+0000","last_fullsized":"2023-10-08T07:13:34.732085+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:28.953862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471940+0000","last_change":"2023-10-08T07:13:34.296170+0000","last_active":"2023-10-08T07:13:35.471940+0000","last_peered":"2023-10-08T07:13:35.471940+0000","last_clean":"2023-10-08T07:13:35.471940+0000","last_became_active":"2023-10-08T07:13:34.295736+0000","last_became_peered":"2023-10-08T07:13:34.295736+0000","last_unstale":"2023-10-08T07:13:35.471940+0000","last_undegraded":"2023-10-08T07:13:35.471940+0000","last_fullsized":"2023-10-08T07:13:35.471940+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.287326+0000","last_change":"2023-10-08T07:13:33.725167+0000","last_active":"2023-10-08T07:13:34.287326+0000","last_peered":"2023-10-08T07:13:34.287326+0000","last_clean":"2023-10-08T07:13:34.287326+0000","last_became_active":"2023-10-08T07:13:33.724905+0000","last_became_peered":"2023-10-08T07:13:33.724905+0000","last_unstale":"2023-10-08T07:13:34.287326+0000","last_undegraded":"2023-10-08T07:13:34.287326+0000","last_fullsized":"2023-10-08T07:13:34.287326+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:37.525027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.271917+0000","last_change":"2023-10-08T07:13:34.302276+0000","last_active":"2023-10-08T07:13:35.271917+0000","last_peered":"2023-10-08T07:13:35.271917+0000","last_clean":"2023-10-08T07:13:35.271917+0000","last_became_active":"2023-10-08T07:13:34.301988+0000","last_became_peered":"2023-10-08T07:13:34.301988+0000","last_unstale":"2023-10-08T07:13:35.271917+0000","last_undegraded":"2023-10-08T07:13:35.271917+0000","last_fullsized":"2023-10-08T07:13:35.271917+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.275897+0000","last_change":"2023-10-08T07:13:33.730922+0000","last_active":"2023-10-08T07:13:34.275897+0000","last_peered":"2023-10-08T07:13:34.275897+0000","last_clean":"2023-10-08T07:13:34.275897+0000","last_became_active":"2023-10-08T07:13:33.730666+0000","last_became_peered":"2023-10-08T07:13:33.730666+0000","last_unstale":"2023-10-08T07:13:34.275897+0000","last_undegraded":"2023-10-08T07:13:34.275897+0000","last_fullsized":"2023-10-08T07:13:34.275897+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:12.086089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.470943+0000","last_change":"2023-10-08T07:13:34.306066+0000","last_active":"2023-10-08T07:13:35.470943+0000","last_peered":"2023-10-08T07:13:35.470943+0000","last_clean":"2023-10-08T07:13:35.470943+0000","last_became_active":"2023-10-08T07:13:34.302955+0000","last_became_peered":"2023-10-08T07:13:34.302955+0000","last_unstale":"2023-10-08T07:13:35.470943+0000","last_undegraded":"2023-10-08T07:13:35.470943+0000","last_fullsized":"2023-10-08T07:13:35.470943+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:03.795703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"233'8","reported_seq":20,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.736148+0000","last_change":"2023-10-08T07:13:33.738121+0000","last_active":"2023-10-08T07:13:34.736148+0000","last_peered":"2023-10-08T07:13:34.736148+0000","last_clean":"2023-10-08T07:13:34.736148+0000","last_became_active":"2023-10-08T07:13:33.737903+0000","last_became_peered":"2023-10-08T07:13:33.737903+0000","last_unstale":"2023-10-08T07:13:34.736148+0000","last_undegraded":"2023-10-08T07:13:34.736148+0000","last_fullsized":"2023-10-08T07:13:34.736148+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:15.275258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.276590+0000","last_change":"2023-10-08T07:13:34.293870+0000","last_active":"2023-10-08T07:13:35.276590+0000","last_peered":"2023-10-08T07:13:35.276590+0000","last_clean":"2023-10-08T07:13:35.276590+0000","last_became_active":"2023-10-08T07:13:34.293397+0000","last_became_peered":"2023-10-08T07:13:34.293397+0000","last_unstale":"2023-10-08T07:13:35.276590+0000","last_undegraded":"2023-10-08T07:13:35.276590+0000","last_fullsized":"2023-10-08T07:13:35.276590+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"233'1","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.730742+0000","last_change":"2023-10-08T07:13:33.728187+0000","last_active":"2023-10-08T07:13:34.730742+0000","last_peered":"2023-10-08T07:13:34.730742+0000","last_clean":"2023-10-08T07:13:34.730742+0000","last_became_active":"2023-10-08T07:13:33.727828+0000","last_became_peered":"2023-10-08T07:13:33.727828+0000","last_unstale":"2023-10-08T07:13:34.730742+0000","last_undegraded":"2023-10-08T07:13:34.730742+0000","last_fullsized":"2023-10-08T07:13:34.730742+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:00.539389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469351+0000","last_change":"2023-10-08T07:13:34.291716+0000","last_active":"2023-10-08T07:13:35.469351+0000","last_peered":"2023-10-08T07:13:35.469351+0000","last_clean":"2023-10-08T07:13:35.469351+0000","last_became_active":"2023-10-08T07:13:34.291300+0000","last_became_peered":"2023-10-08T07:13:34.291300+0000","last_unstale":"2023-10-08T07:13:35.469351+0000","last_undegraded":"2023-10-08T07:13:35.469351+0000","last_fullsized":"2023-10-08T07:13:35.469351+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:07.189339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"233'2","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.733569+0000","last_change":"2023-10-08T07:13:33.736960+0000","last_active":"2023-10-08T07:13:34.733569+0000","last_peered":"2023-10-08T07:13:34.733569+0000","last_clean":"2023-10-08T07:13:34.733569+0000","last_became_active":"2023-10-08T07:13:33.736703+0000","last_became_peered":"2023-10-08T07:13:33.736703+0000","last_unstale":"2023-10-08T07:13:34.733569+0000","last_undegraded":"2023-10-08T07:13:34.733569+0000","last_fullsized":"2023-10-08T07:13:34.733569+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:39.196152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.469915+0000","last_change":"2023-10-08T07:13:34.299071+0000","last_active":"2023-10-08T07:13:35.469915+0000","last_peered":"2023-10-08T07:13:35.469915+0000","last_clean":"2023-10-08T07:13:35.469915+0000","last_became_active":"2023-10-08T07:13:34.298776+0000","last_became_peered":"2023-10-08T07:13:34.298776+0000","last_unstale":"2023-10-08T07:13:35.469915+0000","last_undegraded":"2023-10-08T07:13:35.469915+0000","last_fullsized":"2023-10-08T07:13:35.469915+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:10.642317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"233'1","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.734165+0000","last_change":"2023-10-08T07:13:33.732139+0000","last_active":"2023-10-08T07:13:34.734165+0000","last_peered":"2023-10-08T07:13:34.734165+0000","last_clean":"2023-10-08T07:13:34.734165+0000","last_became_active":"2023-10-08T07:13:33.731892+0000","last_became_peered":"2023-10-08T07:13:33.731892+0000","last_unstale":"2023-10-08T07:13:34.734165+0000","last_undegraded":"2023-10-08T07:13:34.734165+0000","last_fullsized":"2023-10-08T07:13:34.734165+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:07.242491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.277133+0000","last_change":"2023-10-08T07:13:34.308836+0000","last_active":"2023-10-08T07:13:35.277133+0000","last_peered":"2023-10-08T07:13:35.277133+0000","last_clean":"2023-10-08T07:13:35.277133+0000","last_became_active":"2023-10-08T07:13:34.308301+0000","last_became_peered":"2023-10-08T07:13:34.308301+0000","last_unstale":"2023-10-08T07:13:35.277133+0000","last_undegraded":"2023-10-08T07:13:35.277133+0000","last_fullsized":"2023-10-08T07:13:35.277133+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:42.058240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.278990+0000","last_change":"2023-10-08T07:13:33.729567+0000","last_active":"2023-10-08T07:13:34.278990+0000","last_peered":"2023-10-08T07:13:34.278990+0000","last_clean":"2023-10-08T07:13:34.278990+0000","last_became_active":"2023-10-08T07:13:33.729125+0000","last_became_peered":"2023-10-08T07:13:33.729125+0000","last_unstale":"2023-10-08T07:13:34.278990+0000","last_undegraded":"2023-10-08T07:13:34.278990+0000","last_fullsized":"2023-10-08T07:13:34.278990+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:13.193093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471849+0000","last_change":"2023-10-08T07:13:34.294315+0000","last_active":"2023-10-08T07:13:35.471849+0000","last_peered":"2023-10-08T07:13:35.471849+0000","last_clean":"2023-10-08T07:13:35.471849+0000","last_became_active":"2023-10-08T07:13:34.294065+0000","last_became_peered":"2023-10-08T07:13:34.294065+0000","last_unstale":"2023-10-08T07:13:35.471849+0000","last_undegraded":"2023-10-08T07:13:35.471849+0000","last_fullsized":"2023-10-08T07:13:35.471849+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:37.124047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"233'2","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.727406+0000","last_change":"2023-10-08T07:13:33.726781+0000","last_active":"2023-10-08T07:13:34.727406+0000","last_peered":"2023-10-08T07:13:34.727406+0000","last_clean":"2023-10-08T07:13:34.727406+0000","last_became_active":"2023-10-08T07:13:33.726459+0000","last_became_peered":"2023-10-08T07:13:33.726459+0000","last_unstale":"2023-10-08T07:13:34.727406+0000","last_undegraded":"2023-10-08T07:13:34.727406+0000","last_fullsized":"2023-10-08T07:13:34.727406+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:04.357904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471031+0000","last_change":"2023-10-08T07:13:34.301017+0000","last_active":"2023-10-08T07:13:35.471031+0000","last_peered":"2023-10-08T07:13:35.471031+0000","last_clean":"2023-10-08T07:13:35.471031+0000","last_became_active":"2023-10-08T07:13:34.300720+0000","last_became_peered":"2023-10-08T07:13:34.300720+0000","last_unstale":"2023-10-08T07:13:35.471031+0000","last_undegraded":"2023-10-08T07:13:35.471031+0000","last_fullsized":"2023-10-08T07:13:35.471031+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:36:05.460057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.269070+0000","last_change":"2023-10-08T07:13:33.726092+0000","last_active":"2023-10-08T07:13:34.269070+0000","last_peered":"2023-10-08T07:13:34.269070+0000","last_clean":"2023-10-08T07:13:34.269070+0000","last_became_active":"2023-10-08T07:13:33.725701+0000","last_became_peered":"2023-10-08T07:13:33.725701+0000","last_unstale":"2023-10-08T07:13:34.269070+0000","last_undegraded":"2023-10-08T07:13:34.269070+0000","last_fullsized":"2023-10-08T07:13:34.269070+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:51.631016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.272042+0000","last_change":"2023-10-08T07:13:34.294206+0000","last_active":"2023-10-08T07:13:35.272042+0000","last_peered":"2023-10-08T07:13:35.272042+0000","last_clean":"2023-10-08T07:13:35.272042+0000","last_became_active":"2023-10-08T07:13:34.293897+0000","last_became_peered":"2023-10-08T07:13:34.293897+0000","last_unstale":"2023-10-08T07:13:35.272042+0000","last_undegraded":"2023-10-08T07:13:35.272042+0000","last_fullsized":"2023-10-08T07:13:35.272042+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.286886+0000","last_change":"2023-10-08T07:13:33.742955+0000","last_active":"2023-10-08T07:13:34.286886+0000","last_peered":"2023-10-08T07:13:34.286886+0000","last_clean":"2023-10-08T07:13:34.286886+0000","last_became_active":"2023-10-08T07:13:33.742621+0000","last_became_peered":"2023-10-08T07:13:33.742621+0000","last_unstale":"2023-10-08T07:13:34.286886+0000","last_undegraded":"2023-10-08T07:13:34.286886+0000","last_fullsized":"2023-10-08T07:13:34.286886+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:25.311911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.278464+0000","last_change":"2023-10-08T07:13:34.300038+0000","last_active":"2023-10-08T07:13:35.278464+0000","last_peered":"2023-10-08T07:13:35.278464+0000","last_clean":"2023-10-08T07:13:35.278464+0000","last_became_active":"2023-10-08T07:13:34.299721+0000","last_became_peered":"2023-10-08T07:13:34.299721+0000","last_unstale":"2023-10-08T07:13:35.278464+0000","last_undegraded":"2023-10-08T07:13:35.278464+0000","last_fullsized":"2023-10-08T07:13:35.278464+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:26.317918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.275157+0000","last_change":"2023-10-08T07:13:33.739626+0000","last_active":"2023-10-08T07:13:34.275157+0000","last_peered":"2023-10-08T07:13:34.275157+0000","last_clean":"2023-10-08T07:13:34.275157+0000","last_became_active":"2023-10-08T07:13:33.739348+0000","last_became_peered":"2023-10-08T07:13:33.739348+0000","last_unstale":"2023-10-08T07:13:34.275157+0000","last_undegraded":"2023-10-08T07:13:34.275157+0000","last_fullsized":"2023-10-08T07:13:34.275157+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:23.374596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.276913+0000","last_change":"2023-10-08T07:13:34.293674+0000","last_active":"2023-10-08T07:13:35.276913+0000","last_peered":"2023-10-08T07:13:35.276913+0000","last_clean":"2023-10-08T07:13:35.276913+0000","last_became_active":"2023-10-08T07:13:34.292983+0000","last_became_peered":"2023-10-08T07:13:34.292983+0000","last_unstale":"2023-10-08T07:13:35.276913+0000","last_undegraded":"2023-10-08T07:13:35.276913+0000","last_fullsized":"2023-10-08T07:13:35.276913+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:20.477902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"233'2","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.734807+0000","last_change":"2023-10-08T07:13:33.739638+0000","last_active":"2023-10-08T07:13:34.734807+0000","last_peered":"2023-10-08T07:13:34.734807+0000","last_clean":"2023-10-08T07:13:34.734807+0000","last_became_active":"2023-10-08T07:13:33.739352+0000","last_became_peered":"2023-10-08T07:13:33.739352+0000","last_unstale":"2023-10-08T07:13:34.734807+0000","last_undegraded":"2023-10-08T07:13:34.734807+0000","last_fullsized":"2023-10-08T07:13:34.734807+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:11.117318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471049+0000","last_change":"2023-10-08T07:13:34.308581+0000","last_active":"2023-10-08T07:13:35.471049+0000","last_peered":"2023-10-08T07:13:35.471049+0000","last_clean":"2023-10-08T07:13:35.471049+0000","last_became_active":"2023-10-08T07:13:34.308210+0000","last_became_peered":"2023-10-08T07:13:34.308210+0000","last_unstale":"2023-10-08T07:13:35.471049+0000","last_undegraded":"2023-10-08T07:13:35.471049+0000","last_fullsized":"2023-10-08T07:13:35.471049+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.288475+0000","last_change":"2023-10-08T07:13:33.742793+0000","last_active":"2023-10-08T07:13:34.288475+0000","last_peered":"2023-10-08T07:13:34.288475+0000","last_clean":"2023-10-08T07:13:34.288475+0000","last_became_active":"2023-10-08T07:13:33.742030+0000","last_became_peered":"2023-10-08T07:13:33.742030+0000","last_unstale":"2023-10-08T07:13:34.288475+0000","last_undegraded":"2023-10-08T07:13:34.288475+0000","last_fullsized":"2023-10-08T07:13:34.288475+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:25:59.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.271552+0000","last_change":"2023-10-08T07:13:34.302678+0000","last_active":"2023-10-08T07:13:35.271552+0000","last_peered":"2023-10-08T07:13:35.271552+0000","last_clean":"2023-10-08T07:13:35.271552+0000","last_became_active":"2023-10-08T07:13:34.302406+0000","last_became_peered":"2023-10-08T07:13:34.302406+0000","last_unstale":"2023-10-08T07:13:35.271552+0000","last_undegraded":"2023-10-08T07:13:35.271552+0000","last_fullsized":"2023-10-08T07:13:35.271552+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:48.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.288815+0000","last_change":"2023-10-08T07:13:33.737211+0000","last_active":"2023-10-08T07:13:34.288815+0000","last_peered":"2023-10-08T07:13:34.288815+0000","last_clean":"2023-10-08T07:13:34.288815+0000","last_became_active":"2023-10-08T07:13:33.736904+0000","last_became_peered":"2023-10-08T07:13:33.736904+0000","last_unstale":"2023-10-08T07:13:34.288815+0000","last_undegraded":"2023-10-08T07:13:34.288815+0000","last_fullsized":"2023-10-08T07:13:34.288815+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:58.966472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.470341+0000","last_change":"2023-10-08T07:13:34.297602+0000","last_active":"2023-10-08T07:13:35.470341+0000","last_peered":"2023-10-08T07:13:35.470341+0000","last_clean":"2023-10-08T07:13:35.470341+0000","last_became_active":"2023-10-08T07:13:34.296820+0000","last_became_peered":"2023-10-08T07:13:34.296820+0000","last_unstale":"2023-10-08T07:13:35.470341+0000","last_undegraded":"2023-10-08T07:13:35.470341+0000","last_fullsized":"2023-10-08T07:13:35.470341+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:07.812354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.275343+0000","last_change":"2023-10-08T07:13:33.733207+0000","last_active":"2023-10-08T07:13:34.275343+0000","last_peered":"2023-10-08T07:13:34.275343+0000","last_clean":"2023-10-08T07:13:34.275343+0000","last_became_active":"2023-10-08T07:13:33.732903+0000","last_became_peered":"2023-10-08T07:13:33.732903+0000","last_unstale":"2023-10-08T07:13:34.275343+0000","last_undegraded":"2023-10-08T07:13:34.275343+0000","last_fullsized":"2023-10-08T07:13:34.275343+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:34.358428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.277922+0000","last_change":"2023-10-08T07:13:34.302227+0000","last_active":"2023-10-08T07:13:35.277922+0000","last_peered":"2023-10-08T07:13:35.277922+0000","last_clean":"2023-10-08T07:13:35.277922+0000","last_became_active":"2023-10-08T07:13:34.300127+0000","last_became_peered":"2023-10-08T07:13:34.300127+0000","last_unstale":"2023-10-08T07:13:35.277922+0000","last_undegraded":"2023-10-08T07:13:35.277922+0000","last_fullsized":"2023-10-08T07:13:35.277922+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:49.729707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.288385+0000","last_change":"2023-10-08T07:13:33.734220+0000","last_active":"2023-10-08T07:13:34.288385+0000","last_peered":"2023-10-08T07:13:34.288385+0000","last_clean":"2023-10-08T07:13:34.288385+0000","last_became_active":"2023-10-08T07:13:33.733905+0000","last_became_peered":"2023-10-08T07:13:33.733905+0000","last_unstale":"2023-10-08T07:13:34.288385+0000","last_undegraded":"2023-10-08T07:13:34.288385+0000","last_fullsized":"2023-10-08T07:13:34.288385+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471349+0000","last_change":"2023-10-08T07:13:34.305875+0000","last_active":"2023-10-08T07:13:35.471349+0000","last_peered":"2023-10-08T07:13:35.471349+0000","last_clean":"2023-10-08T07:13:35.471349+0000","last_became_active":"2023-10-08T07:13:34.302605+0000","last_became_peered":"2023-10-08T07:13:34.302605+0000","last_unstale":"2023-10-08T07:13:35.471349+0000","last_undegraded":"2023-10-08T07:13:35.471349+0000","last_fullsized":"2023-10-08T07:13:35.471349+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:22.929719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.277569+0000","last_change":"2023-10-08T07:13:33.738792+0000","last_active":"2023-10-08T07:13:34.277569+0000","last_peered":"2023-10-08T07:13:34.277569+0000","last_clean":"2023-10-08T07:13:34.277569+0000","last_became_active":"2023-10-08T07:13:33.738430+0000","last_became_peered":"2023-10-08T07:13:33.738430+0000","last_unstale":"2023-10-08T07:13:34.277569+0000","last_undegraded":"2023-10-08T07:13:34.277569+0000","last_fullsized":"2023-10-08T07:13:34.277569+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:24:39.688018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.471491+0000","last_change":"2023-10-08T07:13:34.303009+0000","last_active":"2023-10-08T07:13:35.471491+0000","last_peered":"2023-10-08T07:13:35.471491+0000","last_clean":"2023-10-08T07:13:35.471491+0000","last_became_active":"2023-10-08T07:13:34.302682+0000","last_became_peered":"2023-10-08T07:13:34.302682+0000","last_unstale":"2023-10-08T07:13:35.471491+0000","last_undegraded":"2023-10-08T07:13:35.471491+0000","last_fullsized":"2023-10-08T07:13:35.471491+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:47:40.976539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.277759+0000","last_change":"2023-10-08T07:13:33.741115+0000","last_active":"2023-10-08T07:13:34.277759+0000","last_peered":"2023-10-08T07:13:34.277759+0000","last_clean":"2023-10-08T07:13:34.277759+0000","last_became_active":"2023-10-08T07:13:33.740798+0000","last_became_peered":"2023-10-08T07:13:33.740798+0000","last_unstale":"2023-10-08T07:13:34.277759+0000","last_undegraded":"2023-10-08T07:13:34.277759+0000","last_fullsized":"2023-10-08T07:13:34.277759+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:36.728485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.472306+0000","last_change":"2023-10-08T07:13:34.308734+0000","last_active":"2023-10-08T07:13:35.472306+0000","last_peered":"2023-10-08T07:13:35.472306+0000","last_clean":"2023-10-08T07:13:35.472306+0000","last_became_active":"2023-10-08T07:13:34.308195+0000","last_became_peered":"2023-10-08T07:13:34.308195+0000","last_unstale":"2023-10-08T07:13:35.472306+0000","last_undegraded":"2023-10-08T07:13:35.472306+0000","last_fullsized":"2023-10-08T07:13:35.472306+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.279373+0000","last_change":"2023-10-08T07:13:33.735924+0000","last_active":"2023-10-08T07:13:34.279373+0000","last_peered":"2023-10-08T07:13:34.279373+0000","last_clean":"2023-10-08T07:13:34.279373+0000","last_became_active":"2023-10-08T07:13:33.735688+0000","last_became_peered":"2023-10-08T07:13:33.735688+0000","last_unstale":"2023-10-08T07:13:34.279373+0000","last_undegraded":"2023-10-08T07:13:34.279373+0000","last_fullsized":"2023-10-08T07:13:34.279373+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-10-08T07:13:35.470565+0000","last_change":"2023-10-08T07:13:34.304364+0000","last_active":"2023-10-08T07:13:35.470565+0000","last_peered":"2023-10-08T07:13:35.470565+0000","last_clean":"2023-10-08T07:13:35.470565+0000","last_became_active":"2023-10-08T07:13:34.304067+0000","last_became_peered":"2023-10-08T07:13:34.304067+0000","last_unstale":"2023-10-08T07:13:35.470565+0000","last_undegraded":"2023-10-08T07:13:35.470565+0000","last_fullsized":"2023-10-08T07:13:35.470565+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:56.084275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"233'1","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.727733+0000","last_change":"2023-10-08T07:13:33.718816+0000","last_active":"2023-10-08T07:13:34.727733+0000","last_peered":"2023-10-08T07:13:34.727733+0000","last_clean":"2023-10-08T07:13:34.727733+0000","last_became_active":"2023-10-08T07:13:33.718494+0000","last_became_peered":"2023-10-08T07:13:33.718494+0000","last_unstale":"2023-10-08T07:13:34.727733+0000","last_undegraded":"2023-10-08T07:13:34.727733+0000","last_fullsized":"2023-10-08T07:13:34.727733+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:03.021625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"222'2806","reported_seq":3723,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-10-08T07:13:34.291326+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:13:34.291326+0000","last_peered":"2023-10-08T07:13:34.291326+0000","last_clean":"2023-10-08T07:13:34.291326+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:13:34.291326+0000","last_undegraded":"2023-10-08T07:13:34.291326+0000","last_fullsized":"2023-10-08T07:13:34.291326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2806,"log_dups_size":0,"ondisk_log_size":2806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2806,"ondisk_log_size":2806,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576009,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61020,"kb_used_data":3464,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310820,"statfs":{"total":96636764160,"available":96574279680,"internally_reserved":0,"allocated":3547136,"data_stored":2116404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.169,"15min":1.2090000000000001},"min":{"1min":0.77100000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.508,"5min":1.968,"15min":2.008},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.238,"15min":1.2310000000000001},"min":{"1min":0.91300000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.6830000000000001,"5min":1.7330000000000001,"15min":1.7450000000000001},"last":0.93700000000000006}]},{"osd":1,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.242,"15min":1.252},"min":{"1min":0.94399999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.665,"5min":1.79,"15min":1.998},"last":1.528},{"interface":"front","average":{"1min":1.367,"5min":1.341,"15min":1.3500000000000001},"min":{"1min":0.99399999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.8859999999999999,"5min":1.8899999999999999,"15min":2.1019999999999999},"last":1.446}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.29,"15min":1.3140000000000001},"min":{"1min":0.97199999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6180000000000001,"5min":1.696,"15min":2.1640000000000001},"last":0.871},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.351},"min":{"1min":1.0069999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7270000000000001,"5min":1.8580000000000001,"15min":2.3170000000000002},"last":1.7150000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":1.0209999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.663,"5min":1.8160000000000001,"15min":1.9039999999999999},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.399,"15min":1.4119999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.754,"5min":1.8680000000000001,"15min":1.931},"last":1.8140000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":0.86199999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6950000000000001,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.383,"15min":1.401},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.85199999999999998},"max":{"1min":1.76,"5min":1.8140000000000001,"15min":2.0379999999999998},"last":1.6299999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4390000000000001,"15min":1.444},"min":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":2.0920000000000001,"15min":2.1139999999999999},"last":1.875},{"interface":"front","average":{"1min":1.532,"5min":1.4610000000000001,"15min":1.51},"min":{"1min":1.075,"5min":1,"15min":0.96899999999999997},"max":{"1min":1.8640000000000001,"5min":2.23,"15min":2.23},"last":1.681}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.575,"15min":1.6020000000000001},"min":{"1min":1.361,"5min":1.048,"15min":1.048},"max":{"1min":2.016,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.9359999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.536,"15min":1.5349999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9079999999999999,"5min":2.21,"15min":2.2570000000000001},"last":1.7709999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576008,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46556,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325284,"statfs":{"total":96636764160,"available":96589090816,"internally_reserved":0,"allocated":2953216,"data_stored":1526548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3109999999999999,"15min":1.321},"min":{"1min":0.80000000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":1.794,"15min":1.962},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.347,"15min":1.3620000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7230000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.97399999999999998}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.228,"15min":1.2290000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.274},{"interface":"front","average":{"1min":1.226,"5min":1.2130000000000001,"15min":1.2430000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.97},"last":1.4299999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.198,"15min":1.238},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.484,"5min":2.339,"15min":2.339},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":0.91800000000000004}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3819999999999999,"15min":1.403},"min":{"1min":1.034,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":1.9319999999999999,"15min":1.95},"last":1.8280000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.476,"15min":1.4990000000000001},"min":{"1min":0.97199999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8740000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.5409999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3999999999999999,"15min":1.3899999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.881,"5min":2.0579999999999998,"15min":2.0859999999999999},"last":1.702},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4770000000000001,"15min":1.448},"min":{"1min":0.94999999999999996,"5min":0.92000000000000004,"15min":0.57099999999999995},"max":{"1min":1.8080000000000001,"5min":2.484,"15min":2.484},"last":1.6639999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.399,"15min":1.423},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.81899999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.403},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.552,"5min":1.538,"15min":1.5509999999999999},"min":{"1min":1.2829999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.8100000000000001,"5min":2.097,"15min":2.6139999999999999},"last":1.925}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.575},"min":{"1min":1.0489999999999999,"5min":1.012,"15min":0.96299999999999997},"max":{"1min":1.9379999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.5620000000000001,"15min":1.5600000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.665},"last":1.7809999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576008,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60424,"kb_used_data":2876,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311416,"statfs":{"total":96636764160,"available":96574889984,"internally_reserved":0,"allocated":2945024,"data_stored":1524410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.157,"15min":1.208},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.841},"last":1.375},{"interface":"front","average":{"1min":1.341,"5min":1.2290000000000001,"15min":1.232},"min":{"1min":0.83099999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7030000000000001,"5min":1.8560000000000001,"15min":1.9610000000000001},"last":1.341}]},{"osd":1,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2090000000000001,"15min":1.242},"min":{"1min":0.75800000000000001,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.6990000000000001,"5min":1.762,"15min":1.9550000000000001},"last":1.823},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2869999999999999,"15min":1.3},"min":{"1min":0.90900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.204}]},{"osd":2,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.871,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":2.0870000000000002},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.3149999999999999},"min":{"1min":0.77000000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5840000000000001,"5min":1.877,"15min":1.909},"last":1.4630000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":0.79800000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.1499999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.3740000000000001,"15min":1.431},"min":{"1min":0.67500000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8100000000000001,"5min":1.9530000000000001,"15min":2.173},"last":1.149}]},{"osd":4,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3220000000000001,"15min":1.3280000000000001},"min":{"1min":0.84399999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7569999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.556,"5min":1.446,"15min":1.4339999999999999},"min":{"1min":1.107,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.2240000000000002},"last":1.292}]},{"osd":6,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":1.0740000000000001,"5min":0.66500000000000004,"15min":0.57499999999999996},"max":{"1min":1.899,"5min":2.077,"15min":2.3759999999999999},"last":1.254},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.478,"15min":1.4770000000000001},"min":{"1min":1.157,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0059999999999998,"5min":2.036,"15min":2.036},"last":1.597}]},{"osd":7,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.5289999999999999,"15min":1.552},"min":{"1min":1.177,"5min":0.72499999999999998,"15min":0.59999999999999998},"max":{"1min":2.298,"5min":2.298,"15min":2.4169999999999998},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.5840000000000001,"15min":1.6180000000000001},"min":{"1min":1.0089999999999999,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0009999999999999,"5min":2.2320000000000002,"15min":2.3199999999999998},"last":1.5549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47940,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323900,"statfs":{"total":96636764160,"available":96587673600,"internally_reserved":0,"allocated":2940928,"data_stored":1524936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2849999999999999,"15min":1.3440000000000001},"min":{"1min":0.64200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":0.62,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.9139999999999999,"5min":1.976,"15min":1.976},"last":1.661}]},{"osd":1,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2629999999999999,"15min":1.2889999999999999},"min":{"1min":0.85599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7450000000000001,"5min":1.976,"15min":1.976},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.244},"min":{"1min":0.72499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7330000000000001,"5min":1.8660000000000001,"15min":1.875},"last":0.71199999999999997}]},{"osd":2,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2689999999999999,"15min":1.266},"min":{"1min":0.77400000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.365,"15min":1.369},"min":{"1min":0.99299999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.52}]},{"osd":3,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2789999999999999,"15min":1.3149999999999999},"min":{"1min":0.88,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.702,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.4410000000000001,"15min":1.4219999999999999},"min":{"1min":0.874,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.7010000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.391},"min":{"1min":1.032,"5min":0.372,"15min":0.372},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":1.282},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4239999999999999,"15min":1.3919999999999999},"min":{"1min":0.96799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.865,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.3520000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.5009999999999999,"15min":1.46},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2530000000000001,"5min":2.2719999999999998,"15min":2.6829999999999998},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.593,"15min":1.5609999999999999},"min":{"1min":1.071,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.117,"5min":2.3500000000000001,"15min":2.7429999999999999},"last":1.7350000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.5369999999999999,"15min":1.516},"min":{"1min":1.1220000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0649999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.552},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.681,"15min":1.6439999999999999},"min":{"1min":1.3169999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.3439999999999999,"5min":2.508,"15min":2.508},"last":2.1469999999999998}]}]},{"osd":3,"up_from":13,"seq":55834576010,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52116,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319724,"statfs":{"total":96636764160,"available":96583397376,"internally_reserved":0,"allocated":2949120,"data_stored":1524480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.99299999999999999,"15min":0.997},"min":{"1min":0.64800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.5529999999999999,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.1759999999999999,"15min":1.1879999999999999},"min":{"1min":0.98699999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5549999999999999,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.101}]},{"osd":1,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.022,"15min":1.038},"min":{"1min":0.69699999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.716,"5min":1.79,"15min":2.4390000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.2889999999999999,"15min":1.3049999999999999},"min":{"1min":0.77000000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.9119999999999999,"5min":2.109,"15min":2.4889999999999999},"last":0.629}]},{"osd":2,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1439999999999999,"15min":1.155},"min":{"1min":0.878,"5min":0.437,"15min":0.437},"max":{"1min":1.623,"5min":2.165,"15min":2.2250000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.198,"15min":1.214},"min":{"1min":0.88900000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.7130000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.86499999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.284,"15min":1.2889999999999999},"min":{"1min":0.86399999999999999,"5min":0.755,"15min":0.67500000000000004},"max":{"1min":1.532,"5min":1.857,"15min":2.5379999999999998},"last":1.333},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.45,"15min":1.466},"min":{"1min":1.1379999999999999,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.78,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.1739999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3420000000000001,"15min":1.383},"min":{"1min":1.147,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8240000000000001,"5min":1.923,"15min":2.9740000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.4390000000000001,"15min":1.4419999999999999},"min":{"1min":1.0529999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.899,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":0.98399999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4259999999999999,"15min":1.427},"min":{"1min":1.111,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.847,"5min":1.976,"15min":2.653},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4990000000000001,"15min":1.534},"min":{"1min":1.2450000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.784,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.2549999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:12:39 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.5580000000000001,"15min":1.5760000000000001},"min":{"1min":1.173,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9870000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.292},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.4810000000000001,"15min":1.526},"min":{"1min":1.0649999999999999,"5min":0.628,"15min":0.628},"max":{"1min":1.849,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.3999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52100,"kb_used_data":2872,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319740,"statfs":{"total":96636764160,"available":96583413760,"internally_reserved":0,"allocated":2940928,"data_stored":1524444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.97299999999999998},"min":{"1min":0.55300000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.948},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.054,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":0.98599999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.96099999999999997,"15min":1.0029999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.198,"5min":1.6279999999999999,"15min":2.044},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.171,"15min":1.1970000000000001},"min":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.677,"5min":1.714,"15min":2.1309999999999998},"last":1.325}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68400000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4770000000000001,"5min":1.78,"15min":1.8420000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.2410000000000001,"15min":1.2769999999999999},"min":{"1min":0.66200000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.458,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.556}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3959999999999999,"15min":1.4399999999999999},"min":{"1min":0.89000000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.821,"5min":1.929,"15min":2.1789999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":1.0189999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6879999999999999,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.1539999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3819999999999999,"15min":1.411},"min":{"1min":1.0069999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.603,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":1.198},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5009999999999999,"15min":1.5349999999999999},"min":{"1min":1.0269999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.946,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.6339999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":0.997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.73,"5min":2.056,"15min":2.1240000000000001},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.47,"5min":1.4950000000000001,"15min":1.534},"min":{"1min":1.1439999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.778,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":1.375}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4690000000000001,"15min":1.486},"min":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7609999999999999,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.5,"5min":1.5309999999999999,"15min":1.5860000000000001},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.863,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":1.5009999999999999}]}]},{"osd":1,"up_from":13,"seq":55834576008,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52104,"kb_used_data":2868,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319736,"statfs":{"total":96636764160,"available":96583409664,"internally_reserved":0,"allocated":2936832,"data_stored":1524354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.024,"15min":1.014},"min":{"1min":0.81000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.6200000000000001,"5min":1.6579999999999999,"15min":1.7769999999999999},"last":0.755},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.224},"min":{"1min":0.88,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":1.79},"last":1.0009999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.97599999999999998},"min":{"1min":0.73399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2809999999999999,"5min":1.641,"15min":1.641},"last":0.58399999999999996},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.155,"15min":1.161},"min":{"1min":0.872,"5min":0.372,"15min":0.372},"max":{"1min":1.4710000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.68400000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.2430000000000001,"15min":1.264},"min":{"1min":1.052,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5800000000000001,"5min":1.651,"15min":1.7809999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1659999999999999,"15min":1.155},"min":{"1min":0.82599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.72,"5min":1.883,"15min":2.7429999999999999},"last":0.84499999999999997}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":1.117,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.7110000000000001,"5min":1.7829999999999999,"15min":2.8460000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.3129999999999999,"15min":1.3220000000000001},"min":{"1min":1.0960000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.5449999999999999,"5min":1.6879999999999999,"15min":2.0139999999999998},"last":1.095}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.347,"15min":1.379},"min":{"1min":0.97299999999999998,"5min":0.92600000000000005,"15min":0.76700000000000002},"max":{"1min":1.6639999999999999,"5min":1.827,"15min":2.0470000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.637,"5min":1.546,"15min":1.534},"min":{"1min":1.286,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.2610000000000001},"last":1.196}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5089999999999999,"15min":1.5149999999999999},"min":{"1min":1.2549999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.694,"5min":2.3199999999999998,"15min":2.9079999999999999},"last":1.143},{"interface":"front","average":{"1min":1.514,"5min":1.4870000000000001,"15min":1.4990000000000001},"min":{"1min":1.1679999999999999,"5min":1.046,"15min":1.0309999999999999},"max":{"1min":1.7929999999999999,"5min":1.966,"15min":2.3199999999999998},"last":1.2509999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.5780000000000001,"15min":1.556},"min":{"1min":1.2250000000000001,"5min":0.94499999999999995,"15min":0.84599999999999997},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.5529999999999999,"15min":1.5720000000000001},"min":{"1min":1.377,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.798,"5min":2.1699999999999999,"15min":2.9590000000000001},"last":1.2989999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576009,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61000,"kb_used_data":3452,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310840,"statfs":{"total":96636764160,"available":96574300160,"internally_reserved":0,"allocated":3534848,"data_stored":2114812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.48299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3859999999999999,"5min":1.581,"15min":1.581},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.01,"5min":1.103,"15min":1.109},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.621,"5min":1.6619999999999999,"15min":1.726},"last":0.93700000000000006}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1060000000000001,"15min":1.1419999999999999},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":1.149},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0329999999999999,"15min":1.073},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4199999999999999,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":1.224}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.248,"15min":1.256},"min":{"1min":0.69799999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.546,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.579,"5min":1.752,"15min":2.0979999999999999},"last":0.63300000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3220000000000001,"15min":1.349},"min":{"1min":0.83399999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.099},{"interface":"front","average":{"1min":1.296,"5min":1.389,"15min":1.4179999999999999},"min":{"1min":0.69699999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.782,"5min":1.972,"15min":2.181},"last":1.4039999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.423,"15min":1.417},"min":{"1min":0.79600000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.732,"5min":1.919,"15min":2.1309999999999998},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.482,"15min":1.486},"min":{"1min":0.79300000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.966,"5min":2.2170000000000001,"15min":2.4900000000000002},"last":1.4410000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.81299999999999994,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7669999999999999,"5min":2.21,"15min":2.2149999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.506,"15min":1.5089999999999999},"min":{"1min":0.78000000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9430000000000001,"5min":2.0880000000000001,"15min":2.355},"last":1.3169999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5389999999999999,"15min":1.5469999999999999},"min":{"1min":0.83499999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9410000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.5289999999999999,"15min":1.554},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.4890000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:13:39.734 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T07:13:39.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T07:13:39.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.991+0000 7fde7ec56640 1 Processor -- start 2023-10-08T07:13:39.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.991+0000 7fde7ec56640 1 -- start start 2023-10-08T07:13:39.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.993+0000 7fde7ec56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde78159d50 0x7fde7815e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:39.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.993+0000 7fde7ec56640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fde780cbc10 0x7fde780c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:39.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.993+0000 7fde7ec56640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde780b2890 con 0x7fde780b16e0 2023-10-08T07:13:39.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.993+0000 7fde7ec56640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde7815d8b0 con 0x7fde780cbc10 2023-10-08T07:13:39.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.993+0000 7fde7ec56640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde780b1ff0 con 0x7fde78159d50 2023-10-08T07:13:39.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.994+0000 7fde77fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fde780b16e0 0x7fde780cb500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39554/0 (socket says 172.21.15.70:39554) 2023-10-08T07:13:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.994+0000 7fde77fff640 1 -- 172.21.15.70:0/672282124 learned_addr learned my addr 172.21.15.70:0/672282124 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.994+0000 7fde7c9f2640 1 --2- 172.21.15.70:0/672282124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde78159d50 0x7fde7815e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.994+0000 7fde7d1f3640 1 --2- 172.21.15.70:0/672282124 >> v2:172.21.15.181:3300/0 conn(0x7fde780cbc10 0x7fde780c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.994+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006213369 0 0) 0x7fde780b2890 con 0x7fde780b16e0 2023-10-08T07:13:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.994+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde60003520 con 0x7fde780b16e0 2023-10-08T07:13:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2126879111 0 0) 0x7fde60003520 con 0x7fde780b16e0 2023-10-08T07:13:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde78159d50 msgr2=0x7fde7815e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 --2- 172.21.15.70:0/672282124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde78159d50 0x7fde7815e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 >> v2:172.21.15.181:3300/0 conn(0x7fde780cbc10 msgr2=0x7fde780c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:39.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 --2- 172.21.15.70:0/672282124 >> v2:172.21.15.181:3300/0 conn(0x7fde780cbc10 0x7fde780c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde780b22d0 con 0x7fde780b16e0 2023-10-08T07:13:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fde6c001670 con 0x7fde780b16e0 2023-10-08T07:13:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 >> v1:172.21.15.70:6789/0 conn(0x7fde780b16e0 legacy=0x7fde780cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 --2- 172.21.15.70:0/672282124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde60003970 0x7fde60003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 --2- 172.21.15.70:0/672282124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600047c0 0x7fde60004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 --2- 172.21.15.70:0/672282124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde78159d50 0x7fde60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7815d8b0 con 0x7fde60003970 2023-10-08T07:13:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde780b1ff0 con 0x7fde600047c0 2023-10-08T07:13:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6000fd90 con 0x7fde78159d50 2023-10-08T07:13:39.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde7c9f2640 1 --2- 172.21.15.70:0/672282124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde60003970 0x7fde60003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde7c9f2640 1 -- 172.21.15.70:0/672282124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde78159d50 msgr2=0x7fde60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:40.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde7c9f2640 1 --2- 172.21.15.70:0/672282124 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde78159d50 0x7fde60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde7c9f2640 1 -- 172.21.15.70:0/672282124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600047c0 msgr2=0x7fde60004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:40.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde7c9f2640 1 --2- 172.21.15.70:0/672282124 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde600047c0 0x7fde60004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.995+0000 7fde7c9f2640 1 -- 172.21.15.70:0/672282124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60003520 con 0x7fde60003970 2023-10-08T07:13:40.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.997+0000 7fde7c9f2640 1 --2- 172.21.15.70:0/672282124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde60003970 0x7fde60003d90 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fde68009fc0 tx=0x7fde600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1fdc978f1b0ea2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:40.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.997+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6802cc40 con 0x7fde60003970 2023-10-08T07:13:40.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.997+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6800f040 con 0x7fde60003970 2023-10-08T07:13:40.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.997+0000 7fde777fe640 1 -- 172.21.15.70:0/672282124 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68034440 con 0x7fde60003970 2023-10-08T07:13:40.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.997+0000 7fde7ec56640 1 -- 172.21.15.70:0/672282124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde60003970 msgr2=0x7fde60003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.997+0000 7fde7ec56640 1 --2- 172.21.15.70:0/672282124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde60003970 0x7fde60003d90 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7fde68009fc0 tx=0x7fde600057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.997+0000 7fde7d1f3640 1 -- 172.21.15.70:0/672282124 reap_dead start 2023-10-08T07:13:40.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.998+0000 7fde7ec56640 1 -- 172.21.15.70:0/672282124 shutdown_connections 2023-10-08T07:13:40.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.998+0000 7fde7ec56640 1 -- 172.21.15.70:0/672282124 >> 172.21.15.70:0/672282124 conn(0x7fde7806a140 msgr2=0x7fde7806a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:40.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.998+0000 7fde7ec56640 1 -- 172.21.15.70:0/672282124 shutdown_connections 2023-10-08T07:13:40.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.998+0000 7fde7ec56640 1 -- 172.21.15.70:0/672282124 wait complete. 2023-10-08T07:13:40.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:39.999+0000 7fde7ec56640 1 Processor -- start 2023-10-08T07:13:40.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.000+0000 7fde7ec56640 1 -- start start 2023-10-08T07:13:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde7ec56640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde60003970 0x7fde78155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde7ec56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde7815a130 0x7fde781ee630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde7ec56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde7815a5b0 0x7fde781eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde7ec56640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde780c3980 con 0x7fde7815a130 2023-10-08T07:13:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde7ec56640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde780c3c20 con 0x7fde60003970 2023-10-08T07:13:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde7ec56640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde780c3f00 con 0x7fde7815a5b0 2023-10-08T07:13:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde77fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde7815a130 0x7fde781ee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde77fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde7815a130 0x7fde781ee630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37128/0 (socket says 172.21.15.70:37128) 2023-10-08T07:13:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde77fff640 1 -- 172.21.15.70:0/2336702934 learned_addr learned my addr 172.21.15.70:0/2336702934 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde77fff640 1 -- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde7815a5b0 msgr2=0x7fde781eeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde7d1f3640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde7815a5b0 0x7fde781eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde77fff640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde7815a5b0 0x7fde781eeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde7c9f2640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde60003970 0x7fde78155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde77fff640 1 -- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde60003970 msgr2=0x7fde78155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde77fff640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde60003970 0x7fde78155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.001+0000 7fde77fff640 1 -- 172.21.15.70:0/2336702934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde780c25e0 con 0x7fde7815a130 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.002+0000 7fde77fff640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde7815a130 0x7fde781ee630 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00c7a0 tx=0x7fde6c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.002+0000 7fde757fa640 1 -- 172.21.15.70:0/2336702934 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0074e0 con 0x7fde7815a130 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.002+0000 7fde757fa640 1 -- 172.21.15.70:0/2336702934 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c007640 con 0x7fde7815a130 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.002+0000 7fde757fa640 1 -- 172.21.15.70:0/2336702934 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c004a20 con 0x7fde7815a130 2023-10-08T07:13:40.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.002+0000 7fde7ec56640 1 -- 172.21.15.70:0/2336702934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde780c28c0 con 0x7fde7815a130 2023-10-08T07:13:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.002+0000 7fde7ec56640 1 -- 172.21.15.70:0/2336702934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde781ef400 con 0x7fde7815a130 2023-10-08T07:13:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.003+0000 7fde7ec56640 1 -- 172.21.15.70:0/2336702934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde780c2d80 con 0x7fde7815a130 2023-10-08T07:13:40.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.004+0000 7fde757fa640 1 -- 172.21.15.70:0/2336702934 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fde6c004c40 con 0x7fde7815a130 2023-10-08T07:13:40.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.005+0000 7fde757fa640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fde6406af90 0x7fde6406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:40.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.005+0000 7fde757fa640 1 -- 172.21.15.70:0/2336702934 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fde6c088f40 con 0x7fde7815a130 2023-10-08T07:13:40.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.006+0000 7fde7c9f2640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fde6406af90 0x7fde6406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.007+0000 7fde7c9f2640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fde6406af90 0x7fde6406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde68036000 tx=0x7fde6802e000 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:40.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.009+0000 7fde757fa640 1 -- 172.21.15.70:0/2336702934 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde6c053c80 con 0x7fde7815a130 2023-10-08T07:13:40.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.260+0000 7fde7ec56640 1 -- 172.21.15.70:0/2336702934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fde78071cb0 con 0x7fde7815a130 2023-10-08T07:13:40.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.264+0000 7fde757fa640 1 -- 172.21.15.70:0/2336702934 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v52) v1 ==== 190+0+147 (secure 0 0 0) 0x7fde6c018070 con 0x7fde7815a130 2023-10-08T07:13:40.266 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T07:13:40.266 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T07:13:40.266 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T07:13:40.266 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T07:13:40.267 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T07:13:40.267 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T07:13:40.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 -- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fde6406af90 msgr2=0x7fde6406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fde6406af90 0x7fde6406d450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fde68036000 tx=0x7fde6802e000 comp rx=0 tx=0).stop 2023-10-08T07:13:40.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 -- 172.21.15.70:0/2336702934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde7815a130 msgr2=0x7fde781ee630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde7815a130 0x7fde781ee630 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00c7a0 tx=0x7fde6c00cc60 comp rx=0 tx=0).stop 2023-10-08T07:13:40.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 -- 172.21.15.70:0/2336702934 shutdown_connections 2023-10-08T07:13:40.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fde7815a5b0 0x7fde781eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fde6406af90 0x7fde6406d450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde7815a130 0x7fde781ee630 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 --2- 172.21.15.70:0/2336702934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fde60003970 0x7fde78155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 -- 172.21.15.70:0/2336702934 >> 172.21.15.70:0/2336702934 conn(0x7fde7806a140 msgr2=0x7fde7806c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:40.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 -- 172.21.15.70:0/2336702934 shutdown_connections 2023-10-08T07:13:40.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.267+0000 7fde7ec56640 1 -- 172.21.15.70:0/2336702934 wait complete. 2023-10-08T07:13:40.288 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:13:40.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.493+0000 7f7065d68640 1 Processor -- start 2023-10-08T07:13:40.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.493+0000 7f7065d68640 1 -- start start 2023-10-08T07:13:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.494+0000 7f7065d68640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f706006d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.494+0000 7f7065d68640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70600ad790 0x7f70600702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.494+0000 7f7065d68640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706006dee0 con 0x7f706015e330 2023-10-08T07:13:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.494+0000 7f7065d68640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706015d850 con 0x7f70600ad790 2023-10-08T07:13:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.494+0000 7f7065d68640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706015db30 con 0x7f7060159ac0 2023-10-08T07:13:40.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.495+0000 7f705f7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f706006d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.495+0000 7f7057fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f70600ad790 0x7f70600702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.495+0000 7f705f7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f706006d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:32864/0 (socket says 172.21.15.70:32864) 2023-10-08T07:13:40.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.495+0000 7f705f7fe640 1 -- 172.21.15.70:0/97316084 learned_addr learned my addr 172.21.15.70:0/97316084 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:40.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.496+0000 7f705f7fe640 1 -- 172.21.15.70:0/97316084 >> v2:172.21.15.181:3300/0 conn(0x7f70600ad790 msgr2=0x7f70600702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:40.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.496+0000 7f705f7fe640 1 --2- 172.21.15.70:0/97316084 >> v2:172.21.15.181:3300/0 conn(0x7f70600ad790 0x7f70600702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.496+0000 7f705f7fe640 1 -- 172.21.15.70:0/97316084 >> v1:172.21.15.181:6789/0 conn(0x7f706015e330 legacy=0x7f706006c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.496+0000 7f705f7fe640 1 -- 172.21.15.70:0/97316084 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706015de10 con 0x7f7060159ac0 2023-10-08T07:13:40.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.497+0000 7f705f7fe640 1 --2- 172.21.15.70:0/97316084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f706006d5e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7050009870 tx=0x7f705002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a547cef58658c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:40.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.498+0000 7f705effd640 1 -- 172.21.15.70:0/97316084 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050030840 con 0x7f7060159ac0 2023-10-08T07:13:40.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.498+0000 7f705effd640 1 -- 172.21.15.70:0/97316084 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70500309a0 con 0x7f7060159ac0 2023-10-08T07:13:40.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.498+0000 7f705effd640 1 -- 172.21.15.70:0/97316084 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050030d30 con 0x7f7060159ac0 2023-10-08T07:13:40.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.499+0000 7f7065d68640 1 -- 172.21.15.70:0/97316084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 msgr2=0x7f706006d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.499+0000 7f7065d68640 1 --2- 172.21.15.70:0/97316084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f706006d5e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7050009870 tx=0x7f705002b980 comp rx=0 tx=0).stop 2023-10-08T07:13:40.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.499+0000 7f7065d68640 1 -- 172.21.15.70:0/97316084 shutdown_connections 2023-10-08T07:13:40.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.499+0000 7f7065d68640 1 --2- 172.21.15.70:0/97316084 >> v2:172.21.15.181:3300/0 conn(0x7f70600ad790 0x7f70600702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.499+0000 7f7065d68640 1 --2- 172.21.15.70:0/97316084 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f706006d5e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.499+0000 7f7065d68640 1 -- 172.21.15.70:0/97316084 >> 172.21.15.70:0/97316084 conn(0x7f706006a380 msgr2=0x7f70600b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.499+0000 7f7065d68640 1 -- 172.21.15.70:0/97316084 shutdown_connections 2023-10-08T07:13:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.499+0000 7f7065d68640 1 -- 172.21.15.70:0/97316084 wait complete. 2023-10-08T07:13:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.500+0000 7f7065d68640 1 Processor -- start 2023-10-08T07:13:40.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.501+0000 7f7065d68640 1 -- start start 2023-10-08T07:13:40.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.501+0000 7f7065d68640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70600ad790 0x7f706013d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:40.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.501+0000 7f7065d68640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f7060157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:40.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f7065d68640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f706015e330 0x7f7060158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:40.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f7065d68640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600ad230 con 0x7f706015e330 2023-10-08T07:13:40.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f7065d68640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600ac6b0 con 0x7f70600ad790 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f7065d68640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70600ac990 con 0x7f7060159ac0 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f705ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f706015e330 0x7f7060158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f705ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f706015e330 0x7f7060158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37132/0 (socket says 172.21.15.70:37132) 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f705ffff640 1 -- 172.21.15.70:0/3209131445 learned_addr learned my addr 172.21.15.70:0/3209131445 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f705f7fe640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70600ad790 0x7f706013d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f7057fff640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f7060157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f705ffff640 1 -- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 msgr2=0x7f7060157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f705ffff640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f7060157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f705ffff640 1 -- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70600ad790 msgr2=0x7f706013d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f705ffff640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70600ad790 0x7f706013d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f705ffff640 1 -- 172.21.15.70:0/3209131445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70500094d0 con 0x7f706015e330 2023-10-08T07:13:40.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f705f7fe640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70600ad790 0x7f706013d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:40.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.502+0000 7f7057fff640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f7060157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:40.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.503+0000 7f705ffff640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f706015e330 0x7f7060158460 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f705800c6b0 tx=0x7f705800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:40.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.503+0000 7f705cff9640 1 -- 172.21.15.70:0/3209131445 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058003b70 con 0x7f706015e330 2023-10-08T07:13:40.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.503+0000 7f705cff9640 1 -- 172.21.15.70:0/3209131445 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7058003cf0 con 0x7f706015e330 2023-10-08T07:13:40.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.503+0000 7f705cff9640 1 -- 172.21.15.70:0/3209131445 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058003ea0 con 0x7f706015e330 2023-10-08T07:13:40.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.503+0000 7f7065d68640 1 -- 172.21.15.70:0/3209131445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70600acc70 con 0x7f706015e330 2023-10-08T07:13:40.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.503+0000 7f7065d68640 1 -- 172.21.15.70:0/3209131445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70600ac2d0 con 0x7f706015e330 2023-10-08T07:13:40.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.504+0000 7f7065d68640 1 -- 172.21.15.70:0/3209131445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70601412f0 con 0x7f706015e330 2023-10-08T07:13:40.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.505+0000 7f705cff9640 1 -- 172.21.15.70:0/3209131445 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7058005310 con 0x7f706015e330 2023-10-08T07:13:40.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.506+0000 7f705cff9640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f704006af90 0x7f704006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:40.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.506+0000 7f705cff9640 1 -- 172.21.15.70:0/3209131445 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f70580897a0 con 0x7f706015e330 2023-10-08T07:13:40.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.507+0000 7f705f7fe640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f704006af90 0x7f704006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:40.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.508+0000 7f705f7fe640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f704006af90 0x7f704006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7050036000 tx=0x7f705003b040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:40.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.511+0000 7f705cff9640 1 -- 172.21.15.70:0/3209131445 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70580544e0 con 0x7f706015e330 2023-10-08T07:13:40.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.737+0000 7f7065d68640 1 -- 172.21.15.70:0/3209131445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7060067490 con 0x7f706015e330 2023-10-08T07:13:40.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.738+0000 7f705cff9640 1 -- 172.21.15.70:0/3209131445 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4929 (secure 0 0 0) 0x7f7058057b00 con 0x7f706015e330 2023-10-08T07:13:40.740 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:40.740 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":108,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8416,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:34.170119+0000","modified":"2023-10-08T07:13:37.222153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11097":{"gid":11097,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/180995576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":180995576},{"type":"v1","addr":"172.21.15.70:6834","nonce":180995576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-10-08T07:13:40.740 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 108 2023-10-08T07:13:40.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.740+0000 7f7065d68640 1 -- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f704006af90 msgr2=0x7f704006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.740+0000 7f7065d68640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f704006af90 0x7f704006d450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7050036000 tx=0x7f705003b040 comp rx=0 tx=0).stop 2023-10-08T07:13:40.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.740+0000 7f7065d68640 1 -- 172.21.15.70:0/3209131445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f706015e330 msgr2=0x7f7060158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:40.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.740+0000 7f7065d68640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f706015e330 0x7f7060158460 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7f705800c6b0 tx=0x7f705800cb70 comp rx=0 tx=0).stop 2023-10-08T07:13:40.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.741+0000 7f7065d68640 1 -- 172.21.15.70:0/3209131445 shutdown_connections 2023-10-08T07:13:40.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.741+0000 7f7065d68640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f704006af90 0x7f704006d450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.741+0000 7f7065d68640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f706015e330 0x7f7060158460 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.741+0000 7f7065d68640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7060159ac0 0x7f7060157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.741+0000 7f7065d68640 1 --2- 172.21.15.70:0/3209131445 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70600ad790 0x7f706013d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:40.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.741+0000 7f7065d68640 1 -- 172.21.15.70:0/3209131445 >> 172.21.15.70:0/3209131445 conn(0x7f706006a380 msgr2=0x7f70600c4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:40.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.741+0000 7f7065d68640 1 -- 172.21.15.70:0/3209131445 shutdown_connections 2023-10-08T07:13:40.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:40.741+0000 7f7065d68640 1 -- 172.21.15.70:0/3209131445 wait complete. 2023-10-08T07:13:41.164 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T07:13:34.170119+0000', 'modified': '2023-10-08T07:13:37.222153+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11097}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11097': {'gid': 11097, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6834/180995576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 180995576}, {'type': 'v1', 'addr': '172.21.15.70:6834', 'nonce': 180995576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T07:13:41.165 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T07:13:41.165 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T07:13:41.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.390+0000 7feaf1d6c640 1 Processor -- start 2023-10-08T07:13:41.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.391+0000 7feaf1d6c640 1 -- start start 2023-10-08T07:13:41.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.392+0000 7feaf1d6c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7feaec0cb620 0x7feaec0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.392+0000 7feaf1d6c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec071170 con 0x7feaec160660 2023-10-08T07:13:41.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.392+0000 7feaf1d6c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaec06e150 con 0x7feaec070cc0 2023-10-08T07:13:41.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.392+0000 7feaf1d6c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaec06de70 con 0x7feaec0cb620 2023-10-08T07:13:41.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.392+0000 7feaeb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7feaec160660 0x7feaec0c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39558/0 (socket says 172.21.15.70:39558) 2023-10-08T07:13:41.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.392+0000 7feaeb7fe640 1 -- 172.21.15.70:0/1188912593 learned_addr learned my addr 172.21.15.70:0/1188912593 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:41.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481007132 0 0) 0x7feadc001110 con 0x7feaec160660 2023-10-08T07:13:41.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaebfff640 1 --2- 172.21.15.70:0/1188912593 >> v2:172.21.15.181:3300/0 conn(0x7feaec0cb620 0x7feaec0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead8003520 con 0x7feaec160660 2023-10-08T07:13:41.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3592276304 0 0) 0x7fead8003520 con 0x7feaec160660 2023-10-08T07:13:41.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 >> v2:172.21.15.181:3300/0 conn(0x7feaec0cb620 msgr2=0x7feaec0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 --2- 172.21.15.70:0/1188912593 >> v2:172.21.15.181:3300/0 conn(0x7feaec0cb620 0x7feaec0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 >> v1:172.21.15.181:6789/0 conn(0x7feaec070cc0 legacy=0x7feaec0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec0c8e30 con 0x7feaec160660 2023-10-08T07:13:41.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7feadc001800 con 0x7feaec160660 2023-10-08T07:13:41.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 >> v1:172.21.15.70:6789/0 conn(0x7feaec160660 legacy=0x7feaec0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead8003970 0x7fead8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead80047c0 0x7fead8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7fead800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec06de70 con 0x7fead80052a0 2023-10-08T07:13:41.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead800c170 con 0x7fead80047c0 2023-10-08T07:13:41.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead800c330 con 0x7fead8003970 2023-10-08T07:13:41.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.393+0000 7feaebfff640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7fead800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.394+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead8003970 0x7fead8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.394+0000 7feae2ffd640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead80047c0 0x7fead8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.394+0000 7feaebfff640 1 -- 172.21.15.70:0/1188912593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead8003970 msgr2=0x7fead8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.394+0000 7feaebfff640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fead8003970 0x7fead8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.394+0000 7feaebfff640 1 -- 172.21.15.70:0/1188912593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead80047c0 msgr2=0x7fead8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.394+0000 7feaebfff640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fead80047c0 0x7fead8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.394+0000 7feaebfff640 1 -- 172.21.15.70:0/1188912593 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8003520 con 0x7fead80052a0 2023-10-08T07:13:41.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.395+0000 7feaebfff640 1 -- 172.21.15.70:0/1188912593 reap_dead start 2023-10-08T07:13:41.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.395+0000 7feaebfff640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7fead800b980 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7feae4007330 tx=0x7fead80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983540373f45eec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:41.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.395+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae402ccd0 con 0x7fead80052a0 2023-10-08T07:13:41.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.396+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae402ce30 con 0x7fead80052a0 2023-10-08T07:13:41.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.396+0000 7feaeaffd640 1 -- 172.21.15.70:0/1188912593 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4003bd0 con 0x7fead80052a0 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.396+0000 7feaf1d6c640 1 -- 172.21.15.70:0/1188912593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 msgr2=0x7fead800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.396+0000 7feaf1d6c640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7fead800b980 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7feae4007330 tx=0x7fead80057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.396+0000 7feaf1d6c640 1 -- 172.21.15.70:0/1188912593 shutdown_connections 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.396+0000 7feaf1d6c640 1 --2- 172.21.15.70:0/1188912593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7fead800b980 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.396+0000 7feaf1d6c640 1 -- 172.21.15.70:0/1188912593 >> 172.21.15.70:0/1188912593 conn(0x7feaec06a5d0 msgr2=0x7feaec0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.396+0000 7feaf1d6c640 1 -- 172.21.15.70:0/1188912593 shutdown_connections 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.396+0000 7feaf1d6c640 1 -- 172.21.15.70:0/1188912593 wait complete. 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.398+0000 7feaf1d6c640 1 Processor -- start 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.398+0000 7feaf1d6c640 1 -- start start 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.399+0000 7feaf1d6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7feaec0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.399+0000 7feaf1d6c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaec0ce650 0x7feaec15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.399+0000 7feaf1d6c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec156aa0 0x7feaec15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.399+0000 7feaf1d6c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec0cd590 con 0x7fead80052a0 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.399+0000 7feaf1d6c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec1f35f0 con 0x7feaec156aa0 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.399+0000 7feaf1d6c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaec1f38d0 con 0x7feaec0ce650 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.399+0000 7feaeb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7feaec0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.399+0000 7feaeb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7feaec0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37144/0 (socket says 172.21.15.70:37144) 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.399+0000 7feaeb7fe640 1 -- 172.21.15.70:0/2303451567 learned_addr learned my addr 172.21.15.70:0/2303451567 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.400+0000 7feaeb7fe640 1 -- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaec0ce650 msgr2=0x7feaec15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.400+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaec0ce650 0x7feaec15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.400+0000 7feaeb7fe640 1 -- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec156aa0 msgr2=0x7feaec15ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.400+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec156aa0 0x7feaec15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.400+0000 7feaeb7fe640 1 -- 172.21.15.70:0/2303451567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec1f3bb0 con 0x7fead80052a0 2023-10-08T07:13:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.400+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7feaec0cd050 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7feadc001eb0 tx=0x7feadc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.400+0000 7feae8ff9640 1 -- 172.21.15.70:0/2303451567 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00c8d0 con 0x7fead80052a0 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.400+0000 7feae8ff9640 1 -- 172.21.15.70:0/2303451567 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc00ca50 con 0x7fead80052a0 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.400+0000 7feae8ff9640 1 -- 172.21.15.70:0/2303451567 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00ce00 con 0x7fead80052a0 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.400+0000 7feaf1d6c640 1 -- 172.21.15.70:0/2303451567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec1f3e90 con 0x7fead80052a0 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.401+0000 7feaf1d6c640 1 -- 172.21.15.70:0/2303451567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec1f4450 con 0x7fead80052a0 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.404+0000 7feae8ff9640 1 -- 172.21.15.70:0/2303451567 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7feadc01d460 con 0x7fead80052a0 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.405+0000 7feae8ff9640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7feacc06af90 0x7feacc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.405+0000 7feae8ff9640 1 -- 172.21.15.70:0/2303451567 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7feadc089840 con 0x7fead80052a0 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.405+0000 7feaf49d3740 1 Processor -- start 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.405+0000 7feae2ffd640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7feacc06af90 0x7feacc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.406+0000 7feaf49d3740 1 -- start start 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.406+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348feb60 0x563d348d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.406+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d349018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.406+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348d1fa0 0x563d34901e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.406+0000 7feaf49d3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d34845700 con 0x563d348fef40 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.406+0000 7feaf49d3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d34842df0 con 0x563d348d1fa0 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.406+0000 7feaf49d3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d3483e6e0 con 0x563d348feb60 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feae2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d349018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feae2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d349018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37150/0 (socket says 172.21.15.70:37150) 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feae2ffd640 1 -- 172.21.15.70:0/2114631785 learned_addr learned my addr 172.21.15.70:0/2114631785 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feae2ffd640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7feacc06af90 0x7feacc06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fead4004c80 tx=0x7fead4009040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/2114631785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348feb60 0x563d348d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feae2ffd640 1 -- 172.21.15.70:0/2114631785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348feb60 msgr2=0x563d348d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feae2ffd640 1 --2- 172.21.15.70:0/2114631785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348feb60 0x563d348d1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feaebfff640 1 --2- 172.21.15.70:0/2114631785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348d1fa0 0x563d34901e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feae2ffd640 1 -- 172.21.15.70:0/2114631785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348d1fa0 msgr2=0x563d34901e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feae2ffd640 1 --2- 172.21.15.70:0/2114631785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348d1fa0 0x563d34901e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feae2ffd640 1 -- 172.21.15.70:0/2114631785 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d34802910 con 0x563d348fef40 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.407+0000 7feae2ffd640 1 --2- 172.21.15.70:0/2114631785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d349018f0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fead4015660 tx=0x7fead40055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed46d33bd4cb280 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.408+0000 7feae17fa640 1 -- 172.21.15.70:0/2114631785 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead400e890 con 0x563d348fef40 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.408+0000 7feae17fa640 1 -- 172.21.15.70:0/2114631785 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead400ea10 con 0x563d348fef40 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.408+0000 7feae17fa640 1 -- 172.21.15.70:0/2114631785 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead400edc0 con 0x563d348fef40 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.408+0000 7feaf49d3740 1 -- 172.21.15.70:0/2114631785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 msgr2=0x563d349018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.408+0000 7feaf49d3740 1 --2- 172.21.15.70:0/2114631785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d349018f0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7fead4015660 tx=0x7fead40055e0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.409+0000 7feaf49d3740 1 -- 172.21.15.70:0/2114631785 shutdown_connections 2023-10-08T07:13:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.409+0000 7feaf49d3740 1 --2- 172.21.15.70:0/2114631785 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348d1fa0 0x563d34901e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.409+0000 7feaf49d3740 1 --2- 172.21.15.70:0/2114631785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d349018f0 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.409+0000 7feaf49d3740 1 --2- 172.21.15.70:0/2114631785 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348feb60 0x563d348d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.409+0000 7feaf49d3740 1 -- 172.21.15.70:0/2114631785 >> 172.21.15.70:0/2114631785 conn(0x563d348d3d70 msgr2=0x563d3491c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.409+0000 7feaf49d3740 1 -- 172.21.15.70:0/2114631785 shutdown_connections 2023-10-08T07:13:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.409+0000 7feaf49d3740 1 -- 172.21.15.70:0/2114631785 wait complete. 2023-10-08T07:13:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.409+0000 7feaf49d3740 1 -- start start 2023-10-08T07:13:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.410+0000 7feaf49d3740 1 Processor -- start 2023-10-08T07:13:41.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348feb60 0x563d34912f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d348ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348fa410 0x563d3492fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feaf49d3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d347dc970 con 0x563d348fef40 2023-10-08T07:13:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feaf49d3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d347dc610 con 0x563d348feb60 2023-10-08T07:13:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feaf49d3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d345a2f20 con 0x563d348fa410 2023-10-08T07:13:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feaf49d3740 10 client.0 _ll_register_callbacks cb 0x563d3481a610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:13:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feae2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d348ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feae2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d348ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37166/0 (socket says 172.21.15.70:37166) 2023-10-08T07:13:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feae2ffd640 1 -- 172.21.15.70:0/3821818362 learned_addr learned my addr 172.21.15.70:0/3821818362 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feae2ffd640 1 -- 172.21.15.70:0/3821818362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348fa410 msgr2=0x563d3492fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feae2ffd640 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348fa410 0x563d3492fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feae2ffd640 1 -- 172.21.15.70:0/3821818362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348feb60 msgr2=0x563d34912f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feae2ffd640 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348feb60 0x563d34912f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.411+0000 7feae2ffd640 1 -- 172.21.15.70:0/3821818362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead400f040 con 0x563d348fef40 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.412+0000 7feae2ffd640 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d348ff8f0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fead4023040 tx=0x7fead40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcaf69804dcc6c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.412+0000 7feaca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.412+0000 7feaca7fc640 1 -- 172.21.15.70:0/3821818362 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead400e890 con 0x563d348fef40 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.412+0000 7feaca7fc640 1 -- 172.21.15.70:0/3821818362 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fead4003a10 con 0x563d348fef40 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.412+0000 7feaf49d3740 1 -- 172.21.15.70:0/3821818362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d348376c0 con 0x563d348fef40 2023-10-08T07:13:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.412+0000 7feaca7fc640 1 -- 172.21.15.70:0/3821818362 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fead4003bd0 con 0x563d348fef40 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.412+0000 7feaf49d3740 10 client.11232 fetch_fsmap learned FSMap version 108 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.412+0000 7feaf49d3740 1 -- 172.21.15.70:0/3821818362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x563d3483cd60 con 0x563d348fef40 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.412+0000 7feaca7fc640 1 -- 172.21.15.70:0/3821818362 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563d348376c0 con 0x563d348fef40 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.413+0000 7feaca7fc640 1 -- 172.21.15.70:0/3821818362 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 108) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fead4004040 con 0x563d348fef40 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.413+0000 7feaf49d3740 10 client.11232 fetch_fsmap finished waiting for FSMap version 108 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.413+0000 7feaf49d3740 10 client.11232 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.413+0000 7feaf49d3740 20 client.11232 populate_metadata read hostname 'smithi070' 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.413+0000 7feaf49d3740 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.70:6832/180995576,v1:172.21.15.70:6834/180995576] conn(0x563d348f7d90 0x563d3492c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.413+0000 7feaf49d3740 4 client.11232 mds_command: new command op to 11097 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.413+0000 7feaf49d3740 1 -- 172.21.15.70:0/3821818362 --> [v2:172.21.15.70:6832/180995576,v1:172.21.15.70:6834/180995576] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d348373e0 con 0x563d348f7d90 2023-10-08T07:13:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.413+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.70:6832/180995576,v1:172.21.15.70:6834/180995576] conn(0x563d348f7d90 0x563d3492c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.414+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.70:6832/180995576,v1:172.21.15.70:6834/180995576] conn(0x563d348f7d90 0x563d3492c750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fadc12bf215b9a4 server_cookie=d79a41c3c241bc4d in_seq=0 out_seq=0 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.414+0000 7feaca7fc640 10 client.11232 ms_handle_connect on v2:172.21.15.70:6832/180995576 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.414+0000 7feaca7fc640 1 -- 172.21.15.70:0/3821818362 <== mds.0 v2:172.21.15.70:6832/180995576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563d348373e0 con 0x563d348f7d90 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.414+0000 7feaca7fc640 10 client.11232 handle_command_reply: tid=0 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.414+0000 7feaf49d3740 1 client.11232 shutdown 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 1 -- 172.21.15.70:0/3821818362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 msgr2=0x563d348ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d348ff8f0 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7fead4023040 tx=0x7fead40077e0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 1 -- 172.21.15.70:0/3821818362 shutdown_connections 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.70:6832/180995576,v1:172.21.15.70:6834/180995576] conn(0x563d348f7d90 0x563d3492c750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348fa410 0x563d3492fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fef40 0x563d348ff8f0 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 1 --2- 172.21.15.70:0/3821818362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348feb60 0x563d34912f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaca7fc640 0 client.11232 ms_handle_reset on v2:172.21.15.70:6832/180995576 2023-10-08T07:13:41.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 1 -- 172.21.15.70:0/3821818362 >> 172.21.15.70:0/3821818362 conn(0x563d348d1fa0 msgr2=0x563d3491fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:41.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 1 -- 172.21.15.70:0/3821818362 shutdown_connections 2023-10-08T07:13:41.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 1 -- 172.21.15.70:0/3821818362 wait complete. 2023-10-08T07:13:41.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.415+0000 7feaf49d3740 20 client.11232 trim_cache size 0 max 16384 2023-10-08T07:13:41.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.431+0000 7feaf49d3740 1 Processor -- start 2023-10-08T07:13:41.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.432+0000 7feaf49d3740 1 -- start start 2023-10-08T07:13:41.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.432+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fa410 0x563d348cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.432+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348feb60 0x563d348f7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.432+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348fef40 0x563d348d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.432+0000 7feaf49d3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d348d2b20 con 0x563d348fa410 2023-10-08T07:13:41.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.432+0000 7feaf49d3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d34800000 con 0x563d348fef40 2023-10-08T07:13:41.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.432+0000 7feaf49d3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d348d7cc0 con 0x563d348feb60 2023-10-08T07:13:41.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.433+0000 7feaeb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fa410 0x563d348cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.433+0000 7feaeb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fa410 0x563d348cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37168/0 (socket says 172.21.15.70:37168) 2023-10-08T07:13:41.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.433+0000 7feaeb7fe640 1 -- 172.21.15.70:0/1075173934 learned_addr learned my addr 172.21.15.70:0/1075173934 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:41.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.433+0000 7feaeb7fe640 1 -- 172.21.15.70:0/1075173934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348feb60 msgr2=0x563d348f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.433+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/1075173934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348feb60 0x563d348f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.433+0000 7feaeb7fe640 1 -- 172.21.15.70:0/1075173934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348fef40 msgr2=0x563d348d5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.433+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/1075173934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348fef40 0x563d348d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.433+0000 7feaeb7fe640 1 -- 172.21.15.70:0/1075173934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d347dbb80 con 0x563d348fa410 2023-10-08T07:13:41.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.433+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/1075173934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fa410 0x563d348cd390 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7feadc001ac0 tx=0x7feadc058b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed414154f96262c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:41.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.433+0000 7feae1ffb640 1 -- 172.21.15.70:0/1075173934 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc085070 con 0x563d348fa410 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feae1ffb640 1 -- 172.21.15.70:0/1075173934 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc06f070 con 0x563d348fa410 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feae1ffb640 1 -- 172.21.15.70:0/1075173934 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc0605e0 con 0x563d348fa410 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feaf49d3740 1 -- 172.21.15.70:0/1075173934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fa410 msgr2=0x563d348cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feaf49d3740 1 --2- 172.21.15.70:0/1075173934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fa410 0x563d348cd390 secure :-1 s=READY pgs=2370 cs=0 l=1 rev1=1 crypto rx=0x7feadc001ac0 tx=0x7feadc058b00 comp rx=0 tx=0).stop 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feaf49d3740 1 -- 172.21.15.70:0/1075173934 shutdown_connections 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feaf49d3740 1 --2- 172.21.15.70:0/1075173934 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348fef40 0x563d348d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feaf49d3740 1 --2- 172.21.15.70:0/1075173934 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348feb60 0x563d348f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feaf49d3740 1 --2- 172.21.15.70:0/1075173934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348fa410 0x563d348cd390 unknown :-1 s=CLOSED pgs=2370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feaf49d3740 1 -- 172.21.15.70:0/1075173934 >> 172.21.15.70:0/1075173934 conn(0x563d348d1fa0 msgr2=0x563d3492f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feaf49d3740 1 -- 172.21.15.70:0/1075173934 shutdown_connections 2023-10-08T07:13:41.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.434+0000 7feaf49d3740 1 -- 172.21.15.70:0/1075173934 wait complete. 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.435+0000 7feaf49d3740 1 -- start start 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.435+0000 7feaf49d3740 1 Processor -- start 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.436+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348fa410 0x563d348f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.436+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348feb60 0x563d348d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.436+0000 7feaf49d3740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348fef40 0x563d348d5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.436+0000 7feaf49d3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d34986360 con 0x563d348feb60 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.436+0000 7feaf49d3740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d348fd030 con 0x563d348fef40 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.436+0000 7feaf49d3740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d34985420 con 0x563d348fa410 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.436+0000 7feaf49d3740 10 client.0 _ll_register_callbacks cb 0x7fead4004910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.436+0000 7feae2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348feb60 0x563d348d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.436+0000 7feae2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348feb60 0x563d348d5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37174/0 (socket says 172.21.15.70:37174) 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.436+0000 7feae2ffd640 1 -- 172.21.15.70:0/2136230850 learned_addr learned my addr 172.21.15.70:0/2136230850 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348fa410 0x563d348f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feaebfff640 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348fef40 0x563d348d5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feae2ffd640 1 -- 172.21.15.70:0/2136230850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348fa410 msgr2=0x563d348f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feae2ffd640 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348fa410 0x563d348f4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feae2ffd640 1 -- 172.21.15.70:0/2136230850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348fef40 msgr2=0x563d348d5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feae2ffd640 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348fef40 0x563d348d5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feae2ffd640 1 -- 172.21.15.70:0/2136230850 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec1f4450 con 0x563d348feb60 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feae2ffd640 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348feb60 0x563d348d5660 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fead40049e0 tx=0x7fead40036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c96567331b58ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feaca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:13:41.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feaca7fc640 1 -- 172.21.15.70:0/2136230850 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead400e890 con 0x563d348feb60 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feaca7fc640 1 -- 172.21.15.70:0/2136230850 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fead402c450 con 0x563d348feb60 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.437+0000 7feaf49d3740 1 -- 172.21.15.70:0/2136230850 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fead400ea10 con 0x563d348feb60 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaca7fc640 1 -- 172.21.15.70:0/2136230850 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fead4015510 con 0x563d348feb60 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaf49d3740 10 client.11238 fetch_fsmap learned FSMap version 108 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaf49d3740 1 -- 172.21.15.70:0/2136230850 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x563d34801170 con 0x563d348feb60 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaca7fc640 1 -- 172.21.15.70:0/2136230850 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead400ea10 con 0x563d348feb60 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaca7fc640 1 -- 172.21.15.70:0/2136230850 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 108) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fead402c610 con 0x563d348feb60 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaf49d3740 10 client.11238 fetch_fsmap finished waiting for FSMap version 108 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaf49d3740 10 client.11238 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaf49d3740 20 client.11238 populate_metadata read hostname 'smithi070' 2023-10-08T07:13:41.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaf49d3740 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.70:6832/180995576,v1:172.21.15.70:6834/180995576] conn(0x563d34988d60 0x563d34989140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaf49d3740 4 client.11238 mds_command: new command op to 11097 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T07:13:41.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaf49d3740 1 -- 172.21.15.70:0/2136230850 --> [v2:172.21.15.70:6832/180995576,v1:172.21.15.70:6834/180995576] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563d345406c0 con 0x563d34988d60 2023-10-08T07:13:41.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.438+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.70:6832/180995576,v1:172.21.15.70:6834/180995576] conn(0x563d34988d60 0x563d34989140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaeb7fe640 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.70:6832/180995576,v1:172.21.15.70:6834/180995576] conn(0x563d34988d60 0x563d34989140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a17ea1387d0224 server_cookie=6f8ce26456949938 in_seq=0 out_seq=0 2023-10-08T07:13:41.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaca7fc640 10 client.11238 ms_handle_connect on v2:172.21.15.70:6832/180995576 2023-10-08T07:13:41.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaca7fc640 1 -- 172.21.15.70:0/2136230850 <== mds.0 v2:172.21.15.70:6832/180995576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x563d345406c0 con 0x563d34988d60 2023-10-08T07:13:41.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaca7fc640 10 client.11238 handle_command_reply: tid=0 2023-10-08T07:13:41.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 client.11238 shutdown 2023-10-08T07:13:41.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 -- 172.21.15.70:0/2136230850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348feb60 msgr2=0x563d348d5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348feb60 0x563d348d5660 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7fead40049e0 tx=0x7fead40036f0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 -- 172.21.15.70:0/2136230850 shutdown_connections 2023-10-08T07:13:41.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.70:6832/180995576,v1:172.21.15.70:6834/180995576] conn(0x563d34988d60 0x563d34989140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x563d348fef40 0x563d348d5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x563d348feb60 0x563d348d5660 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 --2- 172.21.15.70:0/2136230850 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x563d348fa410 0x563d348f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaca7fc640 0 client.11238 ms_handle_reset on v2:172.21.15.70:6832/180995576 2023-10-08T07:13:41.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 -- 172.21.15.70:0/2136230850 >> 172.21.15.70:0/2136230850 conn(0x563d348d1fa0 msgr2=0x563d34901dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:41.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 -- 172.21.15.70:0/2136230850 shutdown_connections 2023-10-08T07:13:41.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.439+0000 7feaf49d3740 1 -- 172.21.15.70:0/2136230850 wait complete. 2023-10-08T07:13:41.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf49d3740 20 client.11238 trim_cache size 0 max 16384 2023-10-08T07:13:41.468 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:13:41.469 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T07:13:41.469 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T07:13:41.469 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 11097, 2023-10-08T07:13:41.469 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T07:13:41.469 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T07:13:41.469 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T07:13:41.469 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.7314130639999998, 2023-10-08T07:13:41.469 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 108, 2023-10-08T07:13:41.469 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 234, 2023-10-08T07:13:41.469 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 234, 2023-10-08T07:13:41.470 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 12.241747067 2023-10-08T07:13:41.470 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:13:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 -- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7feacc06af90 msgr2=0x7feacc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7feacc06af90 0x7feacc06d450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fead4004c80 tx=0x7fead4009040 comp rx=0 tx=0).stop 2023-10-08T07:13:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 -- 172.21.15.70:0/2303451567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 msgr2=0x7feaec0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7feaec0cd050 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7feadc001eb0 tx=0x7feadc00ec80 comp rx=0 tx=0).stop 2023-10-08T07:13:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 -- 172.21.15.70:0/2303451567 shutdown_connections 2023-10-08T07:13:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7feaec156aa0 0x7feaec15ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7feaec0ce650 0x7feaec15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7feacc06af90 0x7feacc06d450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 --2- 172.21.15.70:0/2303451567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fead80052a0 0x7feaec0cd050 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 -- 172.21.15.70:0/2303451567 >> 172.21.15.70:0/2303451567 conn(0x7feaec06a5d0 msgr2=0x7feaec15d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 -- 172.21.15.70:0/2303451567 shutdown_connections 2023-10-08T07:13:41.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.440+0000 7feaf1d6c640 1 -- 172.21.15.70:0/2303451567 wait complete. 2023-10-08T07:13:41.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T07:13:41.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.687+0000 7f77122ca640 1 Processor -- start 2023-10-08T07:13:41.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.687+0000 7f77122ca640 1 -- start start 2023-10-08T07:13:41.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.688+0000 7f77122ca640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f770c0ada10 0x7f770c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.688+0000 7f77122ca640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f770c0ca620 0x7f770c0c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.688+0000 7f77122ca640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f770c0cb450 0x7f770c0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.688+0000 7f77122ca640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770c0cbda0 con 0x7f770c0ada10 2023-10-08T07:13:41.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.688+0000 7f77122ca640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770c0cade0 con 0x7f770c0cb450 2023-10-08T07:13:41.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.688+0000 7f77122ca640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0addf0 con 0x7f770c0ca620 2023-10-08T07:13:41.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.689+0000 7f77112c8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f770c0ada10 0x7f770c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.689+0000 7f77112c8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f770c0ada10 0x7f770c0b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37180/0 (socket says 172.21.15.70:37180) 2023-10-08T07:13:41.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.689+0000 7f77112c8640 1 -- 172.21.15.70:0/241477323 learned_addr learned my addr 172.21.15.70:0/241477323 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:41.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.689+0000 7f77112c8640 1 -- 172.21.15.70:0/241477323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f770c0ca620 msgr2=0x7f770c0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.689+0000 7f7711ac9640 1 --2- 172.21.15.70:0/241477323 >> v2:172.21.15.181:3300/0 conn(0x7f770c0cb450 0x7f770c0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.689+0000 7f7710ac7640 1 --2- 172.21.15.70:0/241477323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f770c0ca620 0x7f770c0c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.689+0000 7f77112c8640 1 --2- 172.21.15.70:0/241477323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f770c0ca620 0x7f770c0c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.689+0000 7f77112c8640 1 -- 172.21.15.70:0/241477323 >> v2:172.21.15.181:3300/0 conn(0x7f770c0cb450 msgr2=0x7f770c0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:41.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.689+0000 7f77112c8640 1 --2- 172.21.15.70:0/241477323 >> v2:172.21.15.181:3300/0 conn(0x7f770c0cb450 0x7f770c0cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.689+0000 7f77112c8640 1 -- 172.21.15.70:0/241477323 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c06c270 con 0x7f770c0ada10 2023-10-08T07:13:41.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.690+0000 7f77112c8640 1 --2- 172.21.15.70:0/241477323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0ada10 0x7f770c0b3080 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f770c0b1490 tx=0x7f770802da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8cd6bfe69f109 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:41.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.690+0000 7f7703fff640 1 -- 172.21.15.70:0/241477323 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770802f850 con 0x7f770c0ada10 2023-10-08T07:13:41.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f7703fff640 1 -- 172.21.15.70:0/241477323 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770802f9b0 con 0x7f770c0ada10 2023-10-08T07:13:41.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f7703fff640 1 -- 172.21.15.70:0/241477323 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770802fd40 con 0x7f770c0ada10 2023-10-08T07:13:41.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f77122ca640 1 -- 172.21.15.70:0/241477323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0ada10 msgr2=0x7f770c0b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f77122ca640 1 --2- 172.21.15.70:0/241477323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0ada10 0x7f770c0b3080 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f770c0b1490 tx=0x7f770802da90 comp rx=0 tx=0).stop 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f77122ca640 1 -- 172.21.15.70:0/241477323 shutdown_connections 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f77122ca640 1 --2- 172.21.15.70:0/241477323 >> v2:172.21.15.181:3300/0 conn(0x7f770c0cb450 0x7f770c0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f77122ca640 1 --2- 172.21.15.70:0/241477323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f770c0ca620 0x7f770c0c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f77122ca640 1 --2- 172.21.15.70:0/241477323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0ada10 0x7f770c0b3080 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f77122ca640 1 -- 172.21.15.70:0/241477323 >> 172.21.15.70:0/241477323 conn(0x7f770c0ce9d0 msgr2=0x7f770c0cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f77122ca640 1 -- 172.21.15.70:0/241477323 shutdown_connections 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.691+0000 7f77122ca640 1 -- 172.21.15.70:0/241477323 wait complete. 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.693+0000 7f77122ca640 1 Processor -- start 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.693+0000 7f77122ca640 1 -- start start 2023-10-08T07:13:41.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.694+0000 7f77122ca640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f770c0ada10 0x7f770c156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.694+0000 7f77122ca640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f770c0ca620 0x7f770c157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.694+0000 7f77122ca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0cb450 0x7f770c1581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.694+0000 7f77122ca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0c5080 con 0x7f770c0cb450 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.694+0000 7f77122ca640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0c47e0 con 0x7f770c0ca620 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.694+0000 7f77122ca640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770c0c4ac0 con 0x7f770c0ada10 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.694+0000 7f7711ac9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0cb450 0x7f770c1581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.694+0000 7f7711ac9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0cb450 0x7f770c1581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37196/0 (socket says 172.21.15.70:37196) 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.694+0000 7f7711ac9640 1 -- 172.21.15.70:0/743935560 learned_addr learned my addr 172.21.15.70:0/743935560 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.695+0000 7f7710ac7640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f770c0ca620 0x7f770c157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.695+0000 7f77112c8640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f770c0ada10 0x7f770c156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.695+0000 7f7711ac9640 1 -- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f770c0ada10 msgr2=0x7f770c156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.695+0000 7f7711ac9640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f770c0ada10 0x7f770c156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.695+0000 7f7711ac9640 1 -- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f770c0ca620 msgr2=0x7f770c157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.695+0000 7f7711ac9640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f770c0ca620 0x7f770c157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.695+0000 7f7711ac9640 1 -- 172.21.15.70:0/743935560 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77080094d0 con 0x7f770c0cb450 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.695+0000 7f7711ac9640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0cb450 0x7f770c1581b0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f770400cc30 tx=0x7f770400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.695+0000 7f7701ffb640 1 -- 172.21.15.70:0/743935560 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770400ce40 con 0x7f770c0cb450 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.695+0000 7f7701ffb640 1 -- 172.21.15.70:0/743935560 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7704003990 con 0x7f770c0cb450 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.696+0000 7f77122ca640 1 -- 172.21.15.70:0/743935560 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f770c0c4da0 con 0x7f770c0cb450 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.696+0000 7f7701ffb640 1 -- 172.21.15.70:0/743935560 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704003d20 con 0x7f770c0cb450 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.696+0000 7f77122ca640 1 -- 172.21.15.70:0/743935560 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c156e70 con 0x7f770c0cb450 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.697+0000 7f77122ca640 1 -- 172.21.15.70:0/743935560 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770c0adf30 con 0x7f770c0cb450 2023-10-08T07:13:41.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.698+0000 7f7701ffb640 1 -- 172.21.15.70:0/743935560 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7704003e80 con 0x7f770c0cb450 2023-10-08T07:13:41.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.699+0000 7f7701ffb640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f76f006af90 0x7f76f006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:41.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.699+0000 7f7701ffb640 1 -- 172.21.15.70:0/743935560 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7704089df0 con 0x7f770c0cb450 2023-10-08T07:13:41.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.700+0000 7f77112c8640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f76f006af90 0x7f76f006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:41.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.701+0000 7f77112c8640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f76f006af90 0x7f76f006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7708038000 tx=0x7f770803b040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:41.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.704+0000 7f7701ffb640 1 -- 172.21.15.70:0/743935560 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f770c0adf30 con 0x7f770c0cb450 2023-10-08T07:13:41.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.939+0000 7f77122ca640 1 -- 172.21.15.70:0/743935560 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f770c0adf30 con 0x7f770c0cb450 2023-10-08T07:13:41.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.940+0000 7f7701ffb640 1 -- 172.21.15.70:0/743935560 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v108) v1 ==== 53+0+73 (secure 0 0 0) 0x7f770c0adf30 con 0x7f770c0cb450 2023-10-08T07:13:41.942 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T07:13:41.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 -- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f76f006af90 msgr2=0x7f76f006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f76f006af90 0x7f76f006d450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7708038000 tx=0x7f770803b040 comp rx=0 tx=0).stop 2023-10-08T07:13:41.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 -- 172.21.15.70:0/743935560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0cb450 msgr2=0x7f770c1581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:41.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0cb450 0x7f770c1581b0 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f770400cc30 tx=0x7f770400a5c0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 -- 172.21.15.70:0/743935560 shutdown_connections 2023-10-08T07:13:41.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f76f006af90 0x7f76f006d450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f770c0cb450 0x7f770c1581b0 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f770c0ca620 0x7f770c157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 --2- 172.21.15.70:0/743935560 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f770c0ada10 0x7f770c156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:41.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 -- 172.21.15.70:0/743935560 >> 172.21.15.70:0/743935560 conn(0x7f770c0ce9d0 msgr2=0x7f770c0700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:41.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.943+0000 7f77122ca640 1 -- 172.21.15.70:0/743935560 shutdown_connections 2023-10-08T07:13:41.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:41.944+0000 7f77122ca640 1 -- 172.21.15.70:0/743935560 wait complete. 2023-10-08T07:13:41.964 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T07:13:41.964 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T07:13:41.964 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T07:13:41.964 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T07:13:41.964 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T07:13:41.965 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T07:13:41.965 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T07:13:41.965 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T07:13:41.965 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:13:41.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:41.968 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T07:13:42.020 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T07:13:42.026 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T07:13:42.026 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T07:13:42.026 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:13:42.026 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T07:13:42.026 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:13:42.027 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 36574sec preferred_lft 36574sec 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T07:13:42.028 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T07:13:42.029 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T07:13:42.029 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:13:42.029 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T07:13:42.029 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T07:13:42.029 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T07:13:42.029 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:13:42.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:42.030 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T07:13:42.118 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T07:13:42.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:42.152 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T07:13:42.212 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T07:13:42.212 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T07:13:42.212 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T07:13:42.213 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T07:13:42.213 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T07:13:42.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:42.213 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T07:13:42.213 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T07:13:42.213 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T07:13:42.213 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T07:13:42.213 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T07:13:42.214 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T07:13:42.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:42.404 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T07:13:42.454 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T07:13:42.455 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T07:13:42.455 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:13:42.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:42.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T07:13:42.554 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T07:13:42.554 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T07:13:42.611 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:13:42.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T07:13:42.611 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T07:13:42.663 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T07:13:42.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:42.663 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:13:42.805 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:13:42.806 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T07:13:42.838 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:13:42.839 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T07:13:42.839 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T07:13:42.839 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T07:13:42.839 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T07:13:42.840 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T07:13:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T07:13:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T07:13:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T07:13:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T07:13:42.843 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T07:13:42.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:42.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T07:13:42.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:13:42.928 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T07:13:42.992 INFO:teuthology.orchestra.run.smithi070.stdout:client11247 2023-10-08T07:13:43.122 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:13:43.123 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:13:43.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T07:13:43.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.406+0000 7fe620826640 1 Processor -- start 2023-10-08T07:13:43.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.406+0000 7fe620826640 1 -- start start 2023-10-08T07:13:43.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.408+0000 7fe620826640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe61c0b1750 0x7fe61c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:43.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.408+0000 7fe620826640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61c06d670 con 0x7fe61c15aad0 2023-10-08T07:13:43.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.408+0000 7fe620826640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61c15bf30 con 0x7fe61c06d240 2023-10-08T07:13:43.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.408+0000 7fe620826640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61c0b1bb0 con 0x7fe61c0b1750 2023-10-08T07:13:43.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.408+0000 7fe619d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe61c0b1750 0x7fe61c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:43.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.408+0000 7fe61a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe61c15aad0 0x7fe61c0c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39568/0 (socket says 172.21.15.70:39568) 2023-10-08T07:13:43.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.408+0000 7fe61a59c640 1 -- 172.21.15.70:0/972520703 learned_addr learned my addr 172.21.15.70:0/972520703 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:43.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.408+0000 7fe619d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe61c0b1750 0x7fe61c06cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37208/0 (socket says 172.21.15.70:37208) 2023-10-08T07:13:43.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.409+0000 7fe619d9b640 1 -- 172.21.15.70:0/972520703 >> v1:172.21.15.181:6789/0 conn(0x7fe61c06d240 legacy=0x7fe61c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:43.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.409+0000 7fe619d9b640 1 -- 172.21.15.70:0/972520703 >> v1:172.21.15.70:6789/0 conn(0x7fe61c15aad0 legacy=0x7fe61c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:43.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.409+0000 7fe619d9b640 1 -- 172.21.15.70:0/972520703 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c15b320 con 0x7fe61c0b1750 2023-10-08T07:13:43.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.409+0000 7fe61959a640 1 -- 172.21.15.70:0/972520703 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964703379 0 0) 0x7fe61c06d670 con 0x7fe61c15aad0 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.410+0000 7fe619d9b640 1 --2- 172.21.15.70:0/972520703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c0b1750 0x7fe61c06cd00 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fe6040097a0 tx=0x7fe60402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50352b8164a330ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.410+0000 7fe61959a640 1 -- 172.21.15.70:0/972520703 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604031830 con 0x7fe61c0b1750 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.410+0000 7fe61959a640 1 -- 172.21.15.70:0/972520703 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe604031990 con 0x7fe61c0b1750 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.410+0000 7fe61959a640 1 -- 172.21.15.70:0/972520703 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604031d20 con 0x7fe61c0b1750 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.411+0000 7fe620826640 1 -- 172.21.15.70:0/972520703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c0b1750 msgr2=0x7fe61c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.411+0000 7fe620826640 1 --2- 172.21.15.70:0/972520703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c0b1750 0x7fe61c06cd00 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fe6040097a0 tx=0x7fe60402b8d0 comp rx=0 tx=0).stop 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.411+0000 7fe620826640 1 -- 172.21.15.70:0/972520703 shutdown_connections 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.411+0000 7fe620826640 1 --2- 172.21.15.70:0/972520703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c0b1750 0x7fe61c06cd00 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.411+0000 7fe620826640 1 -- 172.21.15.70:0/972520703 >> 172.21.15.70:0/972520703 conn(0x7fe61c069d50 msgr2=0x7fe61c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.411+0000 7fe620826640 1 -- 172.21.15.70:0/972520703 shutdown_connections 2023-10-08T07:13:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.411+0000 7fe620826640 1 -- 172.21.15.70:0/972520703 wait complete. 2023-10-08T07:13:43.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.413+0000 7fe620826640 1 Processor -- start 2023-10-08T07:13:43.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.413+0000 7fe620826640 1 -- start start 2023-10-08T07:13:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.414+0000 7fe620826640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c06d240 0x7fe61c0cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.414+0000 7fe620826640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe61c0b1750 0x7fe61c0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.414+0000 7fe620826640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe61c15aad0 0x7fe61c1430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.414+0000 7fe620826640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c141010 con 0x7fe61c06d240 2023-10-08T07:13:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.414+0000 7fe620826640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c1418b0 con 0x7fe61c15aad0 2023-10-08T07:13:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.414+0000 7fe620826640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c0cdb90 con 0x7fe61c0b1750 2023-10-08T07:13:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.414+0000 7fe61a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c06d240 0x7fe61c0cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.414+0000 7fe61a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c06d240 0x7fe61c0cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37220/0 (socket says 172.21.15.70:37220) 2023-10-08T07:13:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.414+0000 7fe61a59c640 1 -- 172.21.15.70:0/724725534 learned_addr learned my addr 172.21.15.70:0/724725534 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe61a59c640 1 -- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe61c0b1750 msgr2=0x7fe61c0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe61a59c640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe61c0b1750 0x7fe61c0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe61a59c640 1 -- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe61c15aad0 msgr2=0x7fe61c1430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe61a59c640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe61c15aad0 0x7fe61c1430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe61a59c640 1 -- 172.21.15.70:0/724725534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe604009400 con 0x7fe61c06d240 2023-10-08T07:13:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe61a59c640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c06d240 0x7fe61c0cce20 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00cb40 tx=0x7fe60c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe612ffd640 1 -- 172.21.15.70:0/724725534 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00aca0 con 0x7fe61c06d240 2023-10-08T07:13:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe612ffd640 1 -- 172.21.15.70:0/724725534 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c00ae00 con 0x7fe61c06d240 2023-10-08T07:13:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe620826640 1 -- 172.21.15.70:0/724725534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c141260 con 0x7fe61c06d240 2023-10-08T07:13:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe612ffd640 1 -- 172.21.15.70:0/724725534 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c012620 con 0x7fe61c06d240 2023-10-08T07:13:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.415+0000 7fe620826640 1 -- 172.21.15.70:0/724725534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c0cd8a0 con 0x7fe61c06d240 2023-10-08T07:13:43.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.416+0000 7fe620826640 1 -- 172.21.15.70:0/724725534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61c04e160 con 0x7fe61c06d240 2023-10-08T07:13:43.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.418+0000 7fe612ffd640 1 -- 172.21.15.70:0/724725534 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe60c007070 con 0x7fe61c06d240 2023-10-08T07:13:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.418+0000 7fe612ffd640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe5fc06afc0 0x7fe5fc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.419+0000 7fe612ffd640 1 -- 172.21.15.70:0/724725534 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe60c089900 con 0x7fe61c06d240 2023-10-08T07:13:43.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.419+0000 7fe619d9b640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe5fc06afc0 0x7fe5fc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:43.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.420+0000 7fe619d9b640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe5fc06afc0 0x7fe5fc06d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe604036000 tx=0x7fe604030000 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:43.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.423+0000 7fe612ffd640 1 -- 172.21.15.70:0/724725534 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe60c0546e0 con 0x7fe61c06d240 2023-10-08T07:13:43.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:43.652+0000 7fe620826640 1 -- 172.21.15.70:0/724725534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe61c0b4610 con 0x7fe61c06d240 2023-10-08T07:13:44.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.303+0000 7fe612ffd640 1 -- 172.21.15.70:0/724725534 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v109) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe60c0170c0 con 0x7fe61c06d240 2023-10-08T07:13:44.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 -- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe5fc06afc0 msgr2=0x7fe5fc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe5fc06afc0 0x7fe5fc06d480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe604036000 tx=0x7fe604030000 comp rx=0 tx=0).stop 2023-10-08T07:13:44.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 -- 172.21.15.70:0/724725534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c06d240 msgr2=0x7fe61c0cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c06d240 0x7fe61c0cce20 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00cb40 tx=0x7fe60c00a060 comp rx=0 tx=0).stop 2023-10-08T07:13:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 -- 172.21.15.70:0/724725534 shutdown_connections 2023-10-08T07:13:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe61c15aad0 0x7fe61c1430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe61c0b1750 0x7fe61c0cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe5fc06afc0 0x7fe5fc06d480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 --2- 172.21.15.70:0/724725534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe61c06d240 0x7fe61c0cce20 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 -- 172.21.15.70:0/724725534 >> 172.21.15.70:0/724725534 conn(0x7fe61c069d50 msgr2=0x7fe61c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 -- 172.21.15.70:0/724725534 shutdown_connections 2023-10-08T07:13:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.307+0000 7fe620826640 1 -- 172.21.15.70:0/724725534 wait complete. 2023-10-08T07:13:44.329 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:44.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.537+0000 7fd646c26640 1 Processor -- start 2023-10-08T07:13:44.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.537+0000 7fd646c26640 1 -- start start 2023-10-08T07:13:44.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.539+0000 7fd646c26640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6401597d0 0x7fd64006ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.539+0000 7fd646c26640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 0x7fd6400cb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.539+0000 7fd646c26640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6400adfc0 con 0x7fd640157000 2023-10-08T07:13:44.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.539+0000 7fd646c26640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64006d380 con 0x7fd6401597d0 2023-10-08T07:13:44.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.539+0000 7fd646c26640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd640159cb0 con 0x7fd6400b0ac0 2023-10-08T07:13:44.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.540+0000 7fd63ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd6401597d0 0x7fd64006ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.540+0000 7fd6451c3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 0x7fd6400cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.540+0000 7fd6449c2640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7fd640157000 0x7fd6400cc7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:59086/0 (socket says 172.21.15.70:59086) 2023-10-08T07:13:44.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.540+0000 7fd6449c2640 1 -- 172.21.15.70:0/3305724616 learned_addr learned my addr 172.21.15.70:0/3305724616 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:44.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.541+0000 7fd6451c3640 1 -- 172.21.15.70:0/3305724616 >> v2:172.21.15.181:3300/0 conn(0x7fd6401597d0 msgr2=0x7fd64006ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:44.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.541+0000 7fd6451c3640 1 --2- 172.21.15.70:0/3305724616 >> v2:172.21.15.181:3300/0 conn(0x7fd6401597d0 0x7fd64006ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.541+0000 7fd6451c3640 1 -- 172.21.15.70:0/3305724616 >> v1:172.21.15.181:6789/0 conn(0x7fd640157000 legacy=0x7fd6400cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.541+0000 7fd6451c3640 1 -- 172.21.15.70:0/3305724616 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd640159f90 con 0x7fd6400b0ac0 2023-10-08T07:13:44.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.541+0000 7fd63ffff640 1 --2- 172.21.15.70:0/3305724616 >> v2:172.21.15.181:3300/0 conn(0x7fd6401597d0 0x7fd64006ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:13:44.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.542+0000 7fd6451c3640 1 --2- 172.21.15.70:0/3305724616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 0x7fd6400cb190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd638009d90 tx=0x7fd63802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415a647f258f5358 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:44.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd63f7fe640 1 -- 172.21.15.70:0/3305724616 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63802cc50 con 0x7fd6400b0ac0 2023-10-08T07:13:44.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd63f7fe640 1 -- 172.21.15.70:0/3305724616 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63802cdb0 con 0x7fd6400b0ac0 2023-10-08T07:13:44.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd63f7fe640 1 -- 172.21.15.70:0/3305724616 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638034660 con 0x7fd6400b0ac0 2023-10-08T07:13:44.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd646c26640 1 -- 172.21.15.70:0/3305724616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 msgr2=0x7fd6400cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd646c26640 1 --2- 172.21.15.70:0/3305724616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 0x7fd6400cb190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd638009d90 tx=0x7fd63802bea0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd646c26640 1 -- 172.21.15.70:0/3305724616 shutdown_connections 2023-10-08T07:13:44.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd646c26640 1 --2- 172.21.15.70:0/3305724616 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 0x7fd6400cb190 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd646c26640 1 --2- 172.21.15.70:0/3305724616 >> v2:172.21.15.181:3300/0 conn(0x7fd6401597d0 0x7fd64006ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd646c26640 1 -- 172.21.15.70:0/3305724616 >> 172.21.15.70:0/3305724616 conn(0x7fd64006a380 msgr2=0x7fd6400b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:44.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd646c26640 1 -- 172.21.15.70:0/3305724616 shutdown_connections 2023-10-08T07:13:44.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.543+0000 7fd646c26640 1 -- 172.21.15.70:0/3305724616 wait complete. 2023-10-08T07:13:44.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.545+0000 7fd646c26640 1 Processor -- start 2023-10-08T07:13:44.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.545+0000 7fd646c26640 1 -- start start 2023-10-08T07:13:44.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.546+0000 7fd646c26640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 0x7fd640149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.546+0000 7fd646c26640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6401597d0 0x7fd6400b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.546+0000 7fd646c26640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6401554b0 0x7fd6400b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.546+0000 7fd646c26640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6400b3e20 con 0x7fd6401554b0 2023-10-08T07:13:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.546+0000 7fd646c26640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6400b4780 con 0x7fd6401597d0 2023-10-08T07:13:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.546+0000 7fd646c26640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6400b31a0 con 0x7fd6400b0ac0 2023-10-08T07:13:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd6451c3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6401554b0 0x7fd6400b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd6451c3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6401554b0 0x7fd6400b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54380/0 (socket says 172.21.15.70:54380) 2023-10-08T07:13:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd6451c3640 1 -- 172.21.15.70:0/58198021 learned_addr learned my addr 172.21.15.70:0/58198021 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd63ffff640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6401597d0 0x7fd6400b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd6449c2640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 0x7fd640149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd6451c3640 1 -- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 msgr2=0x7fd640149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd6451c3640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 0x7fd640149370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd6451c3640 1 -- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6401597d0 msgr2=0x7fd6400b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd6451c3640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6401597d0 0x7fd6400b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd6451c3640 1 -- 172.21.15.70:0/58198021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630009400 con 0x7fd6401554b0 2023-10-08T07:13:44.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.547+0000 7fd6451c3640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6401554b0 0x7fd6400b2950 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fd63802c4b0 tx=0x7fd638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:44.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.548+0000 7fd63d7fa640 1 -- 172.21.15.70:0/58198021 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63803e610 con 0x7fd6401554b0 2023-10-08T07:13:44.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.548+0000 7fd63d7fa640 1 -- 172.21.15.70:0/58198021 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63803e770 con 0x7fd6401554b0 2023-10-08T07:13:44.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.548+0000 7fd63d7fa640 1 -- 172.21.15.70:0/58198021 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63803eb60 con 0x7fd6401554b0 2023-10-08T07:13:44.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.548+0000 7fd646c26640 1 -- 172.21.15.70:0/58198021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd638009a70 con 0x7fd6401554b0 2023-10-08T07:13:44.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.548+0000 7fd646c26640 1 -- 172.21.15.70:0/58198021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6400b3760 con 0x7fd6401554b0 2023-10-08T07:13:44.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.549+0000 7fd646c26640 1 -- 172.21.15.70:0/58198021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6400b3f60 con 0x7fd6401554b0 2023-10-08T07:13:44.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.550+0000 7fd63d7fa640 1 -- 172.21.15.70:0/58198021 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd63803b050 con 0x7fd6401554b0 2023-10-08T07:13:44.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.551+0000 7fd63d7fa640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd62c06afc0 0x7fd62c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.551+0000 7fd63d7fa640 1 -- 172.21.15.70:0/58198021 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fd6380abbe0 con 0x7fd6401554b0 2023-10-08T07:13:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.551+0000 7fd6449c2640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd62c06afc0 0x7fd62c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.553+0000 7fd6449c2640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd62c06afc0 0x7fd62c06d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd634000f60 tx=0x7fd634009040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:44.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.557+0000 7fd63d7fa640 1 -- 172.21.15.70:0/58198021 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6400b3f60 con 0x7fd6401554b0 2023-10-08T07:13:44.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.748+0000 7fd646c26640 1 -- 172.21.15.70:0/58198021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6400b3f60 con 0x7fd6401554b0 2023-10-08T07:13:44.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.751+0000 7fd63d7fa640 1 -- 172.21.15.70:0/58198021 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v234) v1 ==== 74+0+16958 (secure 0 0 0) 0x7fd6400b3f60 con 0x7fd6401554b0 2023-10-08T07:13:44.753 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:44.753 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":234,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:35.265267+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 -- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd62c06afc0 msgr2=0x7fd62c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd62c06afc0 0x7fd62c06d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd634000f60 tx=0x7fd634009040 comp rx=0 tx=0).stop 2023-10-08T07:13:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 -- 172.21.15.70:0/58198021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6401554b0 msgr2=0x7fd6400b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6401554b0 0x7fd6400b2950 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7fd63802c4b0 tx=0x7fd638004a40 comp rx=0 tx=0).stop 2023-10-08T07:13:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 -- 172.21.15.70:0/58198021 shutdown_connections 2023-10-08T07:13:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fd62c06afc0 0x7fd62c06d480 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6401554b0 0x7fd6400b2950 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd6401597d0 0x7fd6400b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 --2- 172.21.15.70:0/58198021 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd6400b0ac0 0x7fd640149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 -- 172.21.15.70:0/58198021 >> 172.21.15.70:0/58198021 conn(0x7fd64006a380 msgr2=0x7fd6400c4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:44.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 -- 172.21.15.70:0/58198021 shutdown_connections 2023-10-08T07:13:44.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.754+0000 7fd646c26640 1 -- 172.21.15.70:0/58198021 wait complete. 2023-10-08T07:13:44.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T07:13:32.637023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T07:13:33.187098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '234', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:13:44.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:13:44.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.981+0000 7f804b78a640 1 Processor -- start 2023-10-08T07:13:44.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.982+0000 7f804b78a640 1 -- start start 2023-10-08T07:13:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.983+0000 7f804b78a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f804415d470 0x7f804406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.983+0000 7f804b78a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8044069d50 0x7f80440cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.983+0000 7f804b78a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804406d1f0 con 0x7f8044159290 2023-10-08T07:13:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.983+0000 7f804b78a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440ccf40 con 0x7f804415d470 2023-10-08T07:13:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.983+0000 7f804b78a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8044159670 con 0x7f8044069d50 2023-10-08T07:13:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.983+0000 7f8049526640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8044159290 0x7f804415cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48740/0 (socket says 172.21.15.70:48740) 2023-10-08T07:13:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.983+0000 7f8049526640 1 -- 172.21.15.70:0/1291939204 learned_addr learned my addr 172.21.15.70:0/1291939204 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8049d27640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8044069d50 0x7f80440cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8048d25640 1 --2- 172.21.15.70:0/1291939204 >> v2:172.21.15.181:3300/0 conn(0x7f804415d470 0x7f804406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857404451 0 0) 0x7f8034001110 con 0x7f8044159290 2023-10-08T07:13:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c003520 con 0x7f8044159290 2023-10-08T07:13:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2798967390 0 0) 0x7f802c003520 con 0x7f8044159290 2023-10-08T07:13:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8044069d50 msgr2=0x7f80440cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8044069d50 0x7f80440cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 >> v2:172.21.15.181:3300/0 conn(0x7f804415d470 msgr2=0x7f804406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 --2- 172.21.15.70:0/1291939204 >> v2:172.21.15.181:3300/0 conn(0x7f804415d470 0x7f804406ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440caeb0 con 0x7f8044159290 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f8034001800 con 0x7f8044159290 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 >> v1:172.21.15.70:6789/0 conn(0x7f8044159290 legacy=0x7f804415cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 0x7f802c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f802c0047c0 0x7f802c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8044069d50 0x7f802c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440ccf40 con 0x7f802c003970 2023-10-08T07:13:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044159670 con 0x7f802c0047c0 2023-10-08T07:13:44.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.984+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c00b980 con 0x7f8044069d50 2023-10-08T07:13:44.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.985+0000 7f8049526640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 0x7f802c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.985+0000 7f8048d25640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f802c0047c0 0x7f802c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.985+0000 7f8049526640 1 -- 172.21.15.70:0/1291939204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8044069d50 msgr2=0x7f802c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:44.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.985+0000 7f8049526640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8044069d50 0x7f802c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.985+0000 7f8049526640 1 -- 172.21.15.70:0/1291939204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f802c0047c0 msgr2=0x7f802c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:44.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.985+0000 7f8049526640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f802c0047c0 0x7f802c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.985+0000 7f8049526640 1 -- 172.21.15.70:0/1291939204 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c003520 con 0x7f802c003970 2023-10-08T07:13:44.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.986+0000 7f8049526640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 0x7f802c003d90 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f803400c210 tx=0x7f802c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b924f3b970021e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.986+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803402cae0 con 0x7f802c003970 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.986+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803402cc60 con 0x7f802c003970 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.986+0000 7f8043fff640 1 -- 172.21.15.70:0/1291939204 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80340058d0 con 0x7f802c003970 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.987+0000 7f804b78a640 1 -- 172.21.15.70:0/1291939204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 msgr2=0x7f802c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.987+0000 7f804b78a640 1 --2- 172.21.15.70:0/1291939204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 0x7f802c003d90 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f803400c210 tx=0x7f802c0057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.987+0000 7f8049d27640 1 -- 172.21.15.70:0/1291939204 reap_dead start 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.987+0000 7f804b78a640 1 -- 172.21.15.70:0/1291939204 shutdown_connections 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.987+0000 7f804b78a640 1 -- 172.21.15.70:0/1291939204 >> 172.21.15.70:0/1291939204 conn(0x7f80440b0ae0 msgr2=0x7f80440b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.987+0000 7f804b78a640 1 -- 172.21.15.70:0/1291939204 shutdown_connections 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.987+0000 7f804b78a640 1 -- 172.21.15.70:0/1291939204 wait complete. 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.989+0000 7f804b78a640 1 Processor -- start 2023-10-08T07:13:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.989+0000 7f804b78a640 1 -- start start 2023-10-08T07:13:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.990+0000 7f804b78a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 0x7f8044152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.990+0000 7f804b78a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80440c3310 0x7f80440c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.990+0000 7f804b78a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8044159c40 0x7f804415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.990+0000 7f804b78a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804415a630 con 0x7f802c003970 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.990+0000 7f804b78a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441528b0 con 0x7f8044159c40 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.990+0000 7f804b78a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80440b28f0 con 0x7f80440c3310 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.990+0000 7f8049526640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 0x7f8044152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.990+0000 7f8049526640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 0x7f8044152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54388/0 (socket says 172.21.15.70:54388) 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.990+0000 7f8049526640 1 -- 172.21.15.70:0/1967825707 learned_addr learned my addr 172.21.15.70:0/1967825707 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8049d27640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8044159c40 0x7f804415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8048d25640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80440c3310 0x7f80440c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8049526640 1 -- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80440c3310 msgr2=0x7f80440c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8049526640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80440c3310 0x7f80440c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8049526640 1 -- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8044159c40 msgr2=0x7f804415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8049526640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8044159c40 0x7f804415a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8049526640 1 -- 172.21.15.70:0/1967825707 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440b2bd0 con 0x7f802c003970 2023-10-08T07:13:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8049526640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 0x7f8044152370 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f803400c210 tx=0x7f8034001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8041ffb640 1 -- 172.21.15.70:0/1967825707 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803403c040 con 0x7f802c003970 2023-10-08T07:13:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8041ffb640 1 -- 172.21.15.70:0/1967825707 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803402caf0 con 0x7f802c003970 2023-10-08T07:13:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f8041ffb640 1 -- 172.21.15.70:0/1967825707 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80340072b0 con 0x7f802c003970 2023-10-08T07:13:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.991+0000 7f804b78a640 1 -- 172.21.15.70:0/1967825707 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80440b2eb0 con 0x7f802c003970 2023-10-08T07:13:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.992+0000 7f804b78a640 1 -- 172.21.15.70:0/1967825707 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80440b3470 con 0x7f802c003970 2023-10-08T07:13:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.993+0000 7f804b78a640 1 -- 172.21.15.70:0/1967825707 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804406b680 con 0x7f802c003970 2023-10-08T07:13:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.994+0000 7f8041ffb640 1 -- 172.21.15.70:0/1967825707 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8034032070 con 0x7f802c003970 2023-10-08T07:13:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.994+0000 7f8041ffb640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f803006afc0 0x7f803006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.995+0000 7f8041ffb640 1 -- 172.21.15.70:0/1967825707 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f80340aa990 con 0x7f802c003970 2023-10-08T07:13:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.995+0000 7f8048d25640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f803006afc0 0x7f803006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.997+0000 7f8048d25640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f803006afc0 0x7f803006d480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f80440ca930 tx=0x7f8038008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:45.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:44.998+0000 7f8041ffb640 1 -- 172.21.15.70:0/1967825707 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80340756d0 con 0x7f802c003970 2023-10-08T07:13:45.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.173+0000 7f804b78a640 1 -- 172.21.15.70:0/1967825707 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f804406d1f0 con 0x7f802c003970 2023-10-08T07:13:45.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.173+0000 7f8041ffb640 1 -- 172.21.15.70:0/1967825707 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v234) v1 ==== 93+0+10 (secure 0 0 0) 0x7f804406d1f0 con 0x7f802c003970 2023-10-08T07:13:45.175 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:13:45.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 -- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f803006afc0 msgr2=0x7f803006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f803006afc0 0x7f803006d480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f80440ca930 tx=0x7f8038008040 comp rx=0 tx=0).stop 2023-10-08T07:13:45.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 -- 172.21.15.70:0/1967825707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 msgr2=0x7f8044152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 0x7f8044152370 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f803400c210 tx=0x7f8034001b90 comp rx=0 tx=0).stop 2023-10-08T07:13:45.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 -- 172.21.15.70:0/1967825707 shutdown_connections 2023-10-08T07:13:45.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8044159c40 0x7f804415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f80440c3310 0x7f80440c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f803006afc0 0x7f803006d480 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 --2- 172.21.15.70:0/1967825707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f802c003970 0x7f8044152370 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 -- 172.21.15.70:0/1967825707 >> 172.21.15.70:0/1967825707 conn(0x7f80440b0ae0 msgr2=0x7f80440b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:45.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 -- 172.21.15.70:0/1967825707 shutdown_connections 2023-10-08T07:13:45.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.176+0000 7f804b78a640 1 -- 172.21.15.70:0/1967825707 wait complete. 2023-10-08T07:13:45.196 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:13:45.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.404+0000 7f13f0da9640 1 Processor -- start 2023-10-08T07:13:45.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.404+0000 7f13f0da9640 1 -- start start 2023-10-08T07:13:45.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.406+0000 7f13f0da9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13ec06dfc0 0x7f13ec070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.406+0000 7f13f0da9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0b1cc0 con 0x7f13ec071070 2023-10-08T07:13:45.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.406+0000 7f13f0da9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0b2620 con 0x7f13ec0afbe0 2023-10-08T07:13:45.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.406+0000 7f13f0da9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec0b2060 con 0x7f13ec06dfc0 2023-10-08T07:13:45.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.406+0000 7f13e3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13ec06dfc0 0x7f13ec070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:45.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.406+0000 7f13ead9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f13ec071070 0x7f13ec0cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48748/0 (socket says 172.21.15.70:48748) 2023-10-08T07:13:45.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.406+0000 7f13ead9d640 1 -- 172.21.15.70:0/601461746 learned_addr learned my addr 172.21.15.70:0/601461746 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.406+0000 7f13e3fff640 1 -- 172.21.15.70:0/601461746 >> v1:172.21.15.181:6789/0 conn(0x7f13ec0afbe0 legacy=0x7f13ec0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.407+0000 7f13e3fff640 1 -- 172.21.15.70:0/601461746 >> v1:172.21.15.70:6789/0 conn(0x7f13ec071070 legacy=0x7f13ec0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.407+0000 7f13e3fff640 1 -- 172.21.15.70:0/601461746 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0b2340 con 0x7f13ec06dfc0 2023-10-08T07:13:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.407+0000 7f13e9d9b640 1 -- 172.21.15.70:0/601461746 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395012109 0 0) 0x7f13ec0b1cc0 con 0x7f13ec071070 2023-10-08T07:13:45.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.408+0000 7f13e3fff640 1 --2- 172.21.15.70:0/601461746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec06dfc0 0x7f13ec070b30 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0b2810 tx=0x7f13d402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd2d411cd7795ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.408+0000 7f13e9d9b640 1 -- 172.21.15.70:0/601461746 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d402d040 con 0x7f13ec06dfc0 2023-10-08T07:13:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.408+0000 7f13e9d9b640 1 -- 172.21.15.70:0/601461746 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d4031810 con 0x7f13ec06dfc0 2023-10-08T07:13:45.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.408+0000 7f13e9d9b640 1 -- 172.21.15.70:0/601461746 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d4031ba0 con 0x7f13ec06dfc0 2023-10-08T07:13:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.409+0000 7f13f0da9640 1 -- 172.21.15.70:0/601461746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec06dfc0 msgr2=0x7f13ec070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.409+0000 7f13f0da9640 1 --2- 172.21.15.70:0/601461746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec06dfc0 0x7f13ec070b30 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0b2810 tx=0x7f13d402b8b0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.409+0000 7f13f0da9640 1 -- 172.21.15.70:0/601461746 shutdown_connections 2023-10-08T07:13:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.409+0000 7f13f0da9640 1 --2- 172.21.15.70:0/601461746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec06dfc0 0x7f13ec070b30 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.409+0000 7f13f0da9640 1 -- 172.21.15.70:0/601461746 >> 172.21.15.70:0/601461746 conn(0x7f13ec06a5d0 msgr2=0x7f13ec0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.409+0000 7f13f0da9640 1 -- 172.21.15.70:0/601461746 shutdown_connections 2023-10-08T07:13:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.409+0000 7f13f0da9640 1 -- 172.21.15.70:0/601461746 wait complete. 2023-10-08T07:13:45.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.411+0000 7f13f0da9640 1 Processor -- start 2023-10-08T07:13:45.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.412+0000 7f13f0da9640 1 -- start start 2023-10-08T07:13:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.413+0000 7f13f0da9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ec06dfc0 0x7f13ec14fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.413+0000 7f13f0da9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec071070 0x7f13ec0ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.413+0000 7f13f0da9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13ec0afbe0 0x7f13ec0c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.413+0000 7f13f0da9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec1502c0 con 0x7f13ec071070 2023-10-08T07:13:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.413+0000 7f13f0da9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0c9bb0 con 0x7f13ec06dfc0 2023-10-08T07:13:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.413+0000 7f13f0da9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec15ddc0 con 0x7f13ec0afbe0 2023-10-08T07:13:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.413+0000 7f13e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec071070 0x7f13ec0ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.413+0000 7f13e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec071070 0x7f13ec0ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54410/0 (socket says 172.21.15.70:54410) 2023-10-08T07:13:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.413+0000 7f13e3fff640 1 -- 172.21.15.70:0/3379858671 learned_addr learned my addr 172.21.15.70:0/3379858671 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:45.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13e3fff640 1 -- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13ec0afbe0 msgr2=0x7f13ec0c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13e3fff640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13ec0afbe0 0x7f13ec0c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13e3fff640 1 -- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ec06dfc0 msgr2=0x7f13ec14fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13e3fff640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ec06dfc0 0x7f13ec14fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13e3fff640 1 -- 172.21.15.70:0/3379858671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4009400 con 0x7f13ec071070 2023-10-08T07:13:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13e3fff640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec071070 0x7f13ec0ca040 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f13d4000c00 tx=0x7f13d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13e2ffd640 1 -- 172.21.15.70:0/3379858671 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d403ad60 con 0x7f13ec071070 2023-10-08T07:13:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13e2ffd640 1 -- 172.21.15.70:0/3379858671 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d40443f0 con 0x7f13ec071070 2023-10-08T07:13:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13e2ffd640 1 -- 172.21.15.70:0/3379858671 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d40447a0 con 0x7f13ec071070 2023-10-08T07:13:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13f0da9640 1 -- 172.21.15.70:0/3379858671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec15e0a0 con 0x7f13ec071070 2023-10-08T07:13:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.414+0000 7f13f0da9640 1 -- 172.21.15.70:0/3379858671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec0c9840 con 0x7f13ec071070 2023-10-08T07:13:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.415+0000 7f13f0da9640 1 -- 172.21.15.70:0/3379858671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec0c99d0 con 0x7f13ec071070 2023-10-08T07:13:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.417+0000 7f13e2ffd640 1 -- 172.21.15.70:0/3379858671 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f13d404d450 con 0x7f13ec071070 2023-10-08T07:13:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.418+0000 7f13e2ffd640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f13cc06af90 0x7f13cc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.418+0000 7f13e2ffd640 1 -- 172.21.15.70:0/3379858671 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f13d40abf30 con 0x7f13ec071070 2023-10-08T07:13:45.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.419+0000 7f13ea59c640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f13cc06af90 0x7f13cc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:45.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.420+0000 7f13ea59c640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f13cc06af90 0x7f13cc06d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13ec15b110 tx=0x7f13dc0079b0 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:45.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.423+0000 7f13e2ffd640 1 -- 172.21.15.70:0/3379858671 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f13d4076b60 con 0x7f13ec071070 2023-10-08T07:13:45.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.611+0000 7f13f0da9640 1 -- 172.21.15.70:0/3379858671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13ec0c9d40 con 0x7f13ec071070 2023-10-08T07:13:45.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.612+0000 7f13e2ffd640 1 -- 172.21.15.70:0/3379858671 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v234) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13d407a180 con 0x7f13ec071070 2023-10-08T07:13:45.614 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:13:45.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.614+0000 7f13f0da9640 1 -- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f13cc06af90 msgr2=0x7f13cc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.614+0000 7f13f0da9640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f13cc06af90 0x7f13cc06d450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f13ec15b110 tx=0x7f13dc0079b0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.615+0000 7f13f0da9640 1 -- 172.21.15.70:0/3379858671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec071070 msgr2=0x7f13ec0ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.615+0000 7f13f0da9640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec071070 0x7f13ec0ca040 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7f13d4000c00 tx=0x7f13d40046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.615+0000 7f13f0da9640 1 -- 172.21.15.70:0/3379858671 shutdown_connections 2023-10-08T07:13:45.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.615+0000 7f13f0da9640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f13ec0afbe0 0x7f13ec0c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.615+0000 7f13f0da9640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f13cc06af90 0x7f13cc06d450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.615+0000 7f13f0da9640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13ec071070 0x7f13ec0ca040 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.615+0000 7f13f0da9640 1 --2- 172.21.15.70:0/3379858671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f13ec06dfc0 0x7f13ec14fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.615+0000 7f13f0da9640 1 -- 172.21.15.70:0/3379858671 >> 172.21.15.70:0/3379858671 conn(0x7f13ec06a5d0 msgr2=0x7f13ec0c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:45.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.615+0000 7f13f0da9640 1 -- 172.21.15.70:0/3379858671 shutdown_connections 2023-10-08T07:13:45.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.615+0000 7f13f0da9640 1 -- 172.21.15.70:0/3379858671 wait complete. 2023-10-08T07:13:45.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:13:45.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.844+0000 7f4fc08c1640 1 Processor -- start 2023-10-08T07:13:45.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.844+0000 7f4fc08c1640 1 -- start start 2023-10-08T07:13:45.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.846+0000 7f4fc08c1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4fbc15a820 0x7f4fbc0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.846+0000 7f4fc08c1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ca680 0x7f4fbc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.846+0000 7f4fc08c1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fbc0cab60 con 0x7f4fbc15ac00 2023-10-08T07:13:45.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.846+0000 7f4fc08c1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fbc0cb960 con 0x7f4fbc15a820 2023-10-08T07:13:45.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.846+0000 7f4fc08c1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0cbdf0 con 0x7f4fbc0ca680 2023-10-08T07:13:45.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.846+0000 7f4fbb59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4fbc15ac00 0x7f4fbc0c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48762/0 (socket says 172.21.15.70:48762) 2023-10-08T07:13:45.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.846+0000 7f4fbb59e640 1 -- 172.21.15.70:0/3665388862 learned_addr learned my addr 172.21.15.70:0/3665388862 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:45.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fbad9d640 1 --2- 172.21.15.70:0/3665388862 >> v2:172.21.15.181:3300/0 conn(0x7f4fbc15a820 0x7f4fbc0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:45.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fb3fff640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ca680 0x7f4fbc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409144546 0 0) 0x7f4fbc0cab60 con 0x7f4fbc15ac00 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa8003520 con 0x7f4fbc15ac00 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3603680792 0 0) 0x7f4fa8003520 con 0x7f4fbc15ac00 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ca680 msgr2=0x7f4fbc0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ca680 0x7f4fbc0b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 >> v2:172.21.15.181:3300/0 conn(0x7f4fbc15a820 msgr2=0x7f4fbc0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 --2- 172.21.15.70:0/3665388862 >> v2:172.21.15.181:3300/0 conn(0x7f4fbc15a820 0x7f4fbc0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc0cb030 con 0x7f4fbc15ac00 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4fb40019f0 con 0x7f4fbc15ac00 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 >> v1:172.21.15.70:6789/0 conn(0x7f4fbc15ac00 legacy=0x7f4fbc0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa8003970 0x7f4fa8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.847+0000 7f4fba59c640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa80047c0 0x7f4fa8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fba59c640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ca680 0x7f4fa8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0cb960 con 0x7f4fa80047c0 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc0cbdf0 con 0x7f4fa8003970 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa800b980 con 0x7f4fbc0ca680 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fb3fff640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa80047c0 0x7f4fa8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fbad9d640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa8003970 0x7f4fa8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fbb59e640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ca680 0x7f4fa8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fb3fff640 1 -- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ca680 msgr2=0x7f4fa8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fb3fff640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ca680 0x7f4fa8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fb3fff640 1 -- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa8003970 msgr2=0x7f4fa8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:45.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fb3fff640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa8003970 0x7f4fa8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.848+0000 7f4fb3fff640 1 -- 172.21.15.70:0/3665388862 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa8003520 con 0x7f4fa80047c0 2023-10-08T07:13:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.850+0000 7f4fb3fff640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa80047c0 0x7f4fa8004c40 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009af0 tx=0x7f4fa80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd374f6d3fb9b3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.850+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa402c6b0 con 0x7f4fa80047c0 2023-10-08T07:13:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.850+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa402c810 con 0x7f4fa80047c0 2023-10-08T07:13:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.850+0000 7f4fba59c640 1 -- 172.21.15.70:0/3665388862 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa402cba0 con 0x7f4fa80047c0 2023-10-08T07:13:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.850+0000 7f4fc08c1640 1 -- 172.21.15.70:0/3665388862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa80047c0 msgr2=0x7f4fa8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.850+0000 7f4fc08c1640 1 --2- 172.21.15.70:0/3665388862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa80047c0 0x7f4fa8004c40 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009af0 tx=0x7f4fa80057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.850+0000 7f4fbb59e640 1 -- 172.21.15.70:0/3665388862 reap_dead start 2023-10-08T07:13:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.851+0000 7f4fc08c1640 1 -- 172.21.15.70:0/3665388862 shutdown_connections 2023-10-08T07:13:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.851+0000 7f4fc08c1640 1 -- 172.21.15.70:0/3665388862 >> 172.21.15.70:0/3665388862 conn(0x7f4fbc06c050 msgr2=0x7f4fbc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.851+0000 7f4fc08c1640 1 -- 172.21.15.70:0/3665388862 shutdown_connections 2023-10-08T07:13:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.851+0000 7f4fc08c1640 1 -- 172.21.15.70:0/3665388862 wait complete. 2023-10-08T07:13:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.852+0000 7f4fc08c1640 1 Processor -- start 2023-10-08T07:13:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.853+0000 7f4fc08c1640 1 -- start start 2023-10-08T07:13:45.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.853+0000 7f4fc08c1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80047c0 0x7f4fbc14e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.853+0000 7f4fc08c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fbc0c2a30 0x7f4fbc0c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.853+0000 7f4fc08c1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ae000 0x7f4fbc0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.853+0000 7f4fc08c1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc15b040 con 0x7f4fbc0c2a30 2023-10-08T07:13:45.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.854+0000 7f4fc08c1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc15b250 con 0x7f4fa80047c0 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.854+0000 7f4fc08c1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc15b530 con 0x7f4fbc0ae000 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.854+0000 7f4fb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fbc0c2a30 0x7f4fbc0c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.854+0000 7f4fb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fbc0c2a30 0x7f4fbc0c2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54434/0 (socket says 172.21.15.70:54434) 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.854+0000 7f4fb3fff640 1 -- 172.21.15.70:0/1138994280 learned_addr learned my addr 172.21.15.70:0/1138994280 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.854+0000 7f4fb3fff640 1 -- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ae000 msgr2=0x7f4fbc0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.854+0000 7f4fb3fff640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ae000 0x7f4fbc0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.854+0000 7f4fb3fff640 1 -- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80047c0 msgr2=0x7f4fbc14e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.854+0000 7f4fb3fff640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80047c0 0x7f4fbc14e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.854+0000 7f4fb3fff640 1 -- 172.21.15.70:0/1138994280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc15b810 con 0x7f4fbc0c2a30 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.855+0000 7f4fb3fff640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fbc0c2a30 0x7f4fbc0c2eb0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4037040 tx=0x7f4fa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.855+0000 7f4fb37fe640 1 -- 172.21.15.70:0/1138994280 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4003810 con 0x7f4fbc0c2a30 2023-10-08T07:13:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.855+0000 7f4fb37fe640 1 -- 172.21.15.70:0/1138994280 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa4003970 con 0x7f4fbc0c2a30 2023-10-08T07:13:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.855+0000 7f4fb37fe640 1 -- 172.21.15.70:0/1138994280 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4003ce0 con 0x7f4fbc0c2a30 2023-10-08T07:13:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.855+0000 7f4fc08c1640 1 -- 172.21.15.70:0/1138994280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fbc15baf0 con 0x7f4fbc0c2a30 2023-10-08T07:13:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.855+0000 7f4fc08c1640 1 -- 172.21.15.70:0/1138994280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fbc15d0b0 con 0x7f4fbc0c2a30 2023-10-08T07:13:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.856+0000 7f4fc08c1640 1 -- 172.21.15.70:0/1138994280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fbc15bc80 con 0x7f4fbc0c2a30 2023-10-08T07:13:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.857+0000 7f4fb37fe640 1 -- 172.21.15.70:0/1138994280 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4fa4047450 con 0x7f4fbc0c2a30 2023-10-08T07:13:45.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.858+0000 7f4fb37fe640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06afc0 0x7f4f9c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:45.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.859+0000 7f4fb37fe640 1 -- 172.21.15.70:0/1138994280 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f4fa40abb40 con 0x7f4fbc0c2a30 2023-10-08T07:13:45.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.859+0000 7f4fbad9d640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06afc0 0x7f4f9c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:45.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.861+0000 7f4fbad9d640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06afc0 0x7f4f9c06d480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009990 tx=0x7f4fac008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:45.863+0000 7f4fb37fe640 1 -- 172.21.15.70:0/1138994280 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fa40769c0 con 0x7f4fbc0c2a30 2023-10-08T07:13:46.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.050+0000 7f4fc08c1640 1 -- 172.21.15.70:0/1138994280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fbc074150 con 0x7f4fbc0c2a30 2023-10-08T07:13:46.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.051+0000 7f4fb37fe640 1 -- 172.21.15.70:0/1138994280 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v234) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fa40044e0 con 0x7f4fbc0c2a30 2023-10-08T07:13:46.053 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:13:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 -- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06afc0 msgr2=0x7f4f9c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06afc0 0x7f4f9c06d480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4fac009990 tx=0x7f4fac008040 comp rx=0 tx=0).stop 2023-10-08T07:13:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 -- 172.21.15.70:0/1138994280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fbc0c2a30 msgr2=0x7f4fbc0c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fbc0c2a30 0x7f4fbc0c2eb0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4037040 tx=0x7f4fa4004730 comp rx=0 tx=0).stop 2023-10-08T07:13:46.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 -- 172.21.15.70:0/1138994280 shutdown_connections 2023-10-08T07:13:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fbc0ae000 0x7f4fbc0ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4f9c06afc0 0x7f4f9c06d480 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fbc0c2a30 0x7f4fbc0c2eb0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 --2- 172.21.15.70:0/1138994280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fa80047c0 0x7f4fbc14e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 -- 172.21.15.70:0/1138994280 >> 172.21.15.70:0/1138994280 conn(0x7f4fbc06c050 msgr2=0x7f4fbc0cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 -- 172.21.15.70:0/1138994280 shutdown_connections 2023-10-08T07:13:46.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.054+0000 7f4fc08c1640 1 -- 172.21.15.70:0/1138994280 wait complete. 2023-10-08T07:13:46.074 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T07:13:46.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.283+0000 7f48dbebb640 1 Processor -- start 2023-10-08T07:13:46.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.283+0000 7f48dbebb640 1 -- start start 2023-10-08T07:13:46.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.285+0000 7f48dbebb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d40ad790 0x7f48d40702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.285+0000 7f48dbebb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48d415e330 0x7f48d4070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.285+0000 7f48dbebb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d406dee0 con 0x7f48d4159ac0 2023-10-08T07:13:46.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.285+0000 7f48dbebb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d406dcf0 con 0x7f48d415e330 2023-10-08T07:13:46.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.285+0000 7f48dbebb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d415d850 con 0x7f48d40ad790 2023-10-08T07:13:46.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.285+0000 7f48da458640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48d415e330 0x7f48d4070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.285+0000 7f48da458640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48d415e330 0x7f48d4070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54448/0 (socket says 172.21.15.70:54448) 2023-10-08T07:13:46.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.285+0000 7f48da458640 1 -- 172.21.15.70:0/3848546081 learned_addr learned my addr 172.21.15.70:0/3848546081 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:46.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.285+0000 7f48d9456640 1 --2- 172.21.15.70:0/3848546081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d40ad790 0x7f48d40702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.286+0000 7f48da458640 1 -- 172.21.15.70:0/3848546081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d40ad790 msgr2=0x7f48d40702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:46.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.286+0000 7f48da458640 1 --2- 172.21.15.70:0/3848546081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d40ad790 0x7f48d40702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.286+0000 7f48da458640 1 -- 172.21.15.70:0/3848546081 >> v1:172.21.15.181:6789/0 conn(0x7f48d4159ac0 legacy=0x7f48d406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:46.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.286+0000 7f48da458640 1 -- 172.21.15.70:0/3848546081 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d415db30 con 0x7f48d415e330 2023-10-08T07:13:46.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.287+0000 7f48da458640 1 --2- 172.21.15.70:0/3848546081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d415e330 0x7f48d4070810 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00d150 tx=0x7f48cc02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b3dfe34592375b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:46.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.287+0000 7f48d8c55640 1 -- 172.21.15.70:0/3848546081 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc02ea80 con 0x7f48d415e330 2023-10-08T07:13:46.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.287+0000 7f48d8c55640 1 -- 172.21.15.70:0/3848546081 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48cc02ebe0 con 0x7f48d415e330 2023-10-08T07:13:46.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.287+0000 7f48d8c55640 1 -- 172.21.15.70:0/3848546081 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48cc009600 con 0x7f48d415e330 2023-10-08T07:13:46.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.288+0000 7f48dbebb640 1 -- 172.21.15.70:0/3848546081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d415e330 msgr2=0x7f48d4070810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.288+0000 7f48dbebb640 1 --2- 172.21.15.70:0/3848546081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d415e330 0x7f48d4070810 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00d150 tx=0x7f48cc02e0e0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.288+0000 7f48dbebb640 1 -- 172.21.15.70:0/3848546081 shutdown_connections 2023-10-08T07:13:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.288+0000 7f48dbebb640 1 --2- 172.21.15.70:0/3848546081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d415e330 0x7f48d4070810 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.288+0000 7f48dbebb640 1 --2- 172.21.15.70:0/3848546081 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d40ad790 0x7f48d40702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.288+0000 7f48dbebb640 1 -- 172.21.15.70:0/3848546081 >> 172.21.15.70:0/3848546081 conn(0x7f48d406a380 msgr2=0x7f48d40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.288+0000 7f48dbebb640 1 -- 172.21.15.70:0/3848546081 shutdown_connections 2023-10-08T07:13:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.288+0000 7f48dbebb640 1 -- 172.21.15.70:0/3848546081 wait complete. 2023-10-08T07:13:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.290+0000 7f48dbebb640 1 Processor -- start 2023-10-08T07:13:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.290+0000 7f48dbebb640 1 -- start start 2023-10-08T07:13:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.291+0000 7f48dbebb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d40ad790 0x7f48d40c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.291+0000 7f48dbebb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d4159ac0 0x7f48d414d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.291+0000 7f48dbebb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d415e330 0x7f48d4152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.291+0000 7f48dbebb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40c78b0 con 0x7f48d4159ac0 2023-10-08T07:13:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.291+0000 7f48dbebb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40c7b30 con 0x7f48d40ad790 2023-10-08T07:13:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.291+0000 7f48dbebb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40c7e10 con 0x7f48d415e330 2023-10-08T07:13:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48d9456640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d4159ac0 0x7f48d414d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48d9456640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d4159ac0 0x7f48d414d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54462/0 (socket says 172.21.15.70:54462) 2023-10-08T07:13:46.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48d9456640 1 -- 172.21.15.70:0/1060891625 learned_addr learned my addr 172.21.15.70:0/1060891625 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:46.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48d9c57640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d40ad790 0x7f48d40c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48da458640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d415e330 0x7f48d4152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48d9456640 1 -- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d415e330 msgr2=0x7f48d4152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48d9456640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d415e330 0x7f48d4152470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48d9456640 1 -- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d40ad790 msgr2=0x7f48d40c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48d9456640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d40ad790 0x7f48d40c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48d9456640 1 -- 172.21.15.70:0/1060891625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc00bd40 con 0x7f48d4159ac0 2023-10-08T07:13:46.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48da458640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d415e330 0x7f48d4152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:46.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.292+0000 7f48d9456640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d4159ac0 0x7f48d414d340 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f48c400d7d0 tx=0x7f48c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.293+0000 7f48ca7fc640 1 -- 172.21.15.70:0/1060891625 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c400b6c0 con 0x7f48d4159ac0 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.293+0000 7f48ca7fc640 1 -- 172.21.15.70:0/1060891625 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c400b840 con 0x7f48d4159ac0 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.293+0000 7f48ca7fc640 1 -- 172.21.15.70:0/1060891625 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c400bbf0 con 0x7f48d4159ac0 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.293+0000 7f48dbebb640 1 -- 172.21.15.70:0/1060891625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d40c80f0 con 0x7f48d4159ac0 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.293+0000 7f48dbebb640 1 -- 172.21.15.70:0/1060891625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d414d880 con 0x7f48d4159ac0 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.294+0000 7f48dbebb640 1 -- 172.21.15.70:0/1060891625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d4067490 con 0x7f48d4159ac0 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.295+0000 7f48ca7fc640 1 -- 172.21.15.70:0/1060891625 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f48c400bd50 con 0x7f48d4159ac0 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.296+0000 7f48ca7fc640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f48b806af90 0x7f48b806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.296+0000 7f48ca7fc640 1 -- 172.21.15.70:0/1060891625 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f48c4089f70 con 0x7f48d4159ac0 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.297+0000 7f48d9c57640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f48b806af90 0x7f48b806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.298+0000 7f48d9c57640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f48b806af90 0x7f48b806d450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f48d0004ea0 tx=0x7f48d0009040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:46.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.300+0000 7f48ca7fc640 1 -- 172.21.15.70:0/1060891625 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48c4054d40 con 0x7f48d4159ac0 2023-10-08T07:13:46.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.480+0000 7f48dbebb640 1 -- 172.21.15.70:0/1060891625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f48d41f5d10 con 0x7f48d4159ac0 2023-10-08T07:13:46.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.481+0000 7f48ca7fc640 1 -- 172.21.15.70:0/1060891625 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v234) v1 ==== 84+0+209 (secure 0 0 0) 0x7f48c4058360 con 0x7f48d4159ac0 2023-10-08T07:13:46.515 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:46.515 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T07:13:46.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:46.515 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T07:13:46.515 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T07:13:46.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:46.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:46.516 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 27, 2023-10-08T07:13:46.516 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "cephfs_metadata" 2023-10-08T07:13:46.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:13:46.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:13:46.516 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 28, 2023-10-08T07:13:46.516 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "cephfs_data" 2023-10-08T07:13:46.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:13:46.516 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T07:13:46.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 -- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f48b806af90 msgr2=0x7f48b806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f48b806af90 0x7f48b806d450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f48d0004ea0 tx=0x7f48d0009040 comp rx=0 tx=0).stop 2023-10-08T07:13:46.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 -- 172.21.15.70:0/1060891625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d4159ac0 msgr2=0x7f48d414d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d4159ac0 0x7f48d414d340 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7f48c400d7d0 tx=0x7f48c400dc90 comp rx=0 tx=0).stop 2023-10-08T07:13:46.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 -- 172.21.15.70:0/1060891625 shutdown_connections 2023-10-08T07:13:46.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f48d415e330 0x7f48d4152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f48b806af90 0x7f48b806d450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48d4159ac0 0x7f48d414d340 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 --2- 172.21.15.70:0/1060891625 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f48d40ad790 0x7f48d40c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 -- 172.21.15.70:0/1060891625 >> 172.21.15.70:0/1060891625 conn(0x7f48d406a380 msgr2=0x7f48d40c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:46.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 -- 172.21.15.70:0/1060891625 shutdown_connections 2023-10-08T07:13:46.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.483+0000 7f48dbebb640 1 -- 172.21.15.70:0/1060891625 wait complete. 2023-10-08T07:13:46.519 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-10-08T07:13:46.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-10-08T07:13:46.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.738+0000 7f4dba8bb640 1 Processor -- start 2023-10-08T07:13:46.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.739+0000 7f4dba8bb640 1 -- start start 2023-10-08T07:13:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.740+0000 7f4dba8bb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4db40ae740 0x7f4db40aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.740+0000 7f4dba8bb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db40cd430 con 0x7f4db4159b50 2023-10-08T07:13:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.740+0000 7f4dba8bb640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db40cf260 con 0x7f4db415e060 2023-10-08T07:13:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.740+0000 7f4dba8bb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db415e4e0 con 0x7f4db40ae740 2023-10-08T07:13:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.740+0000 7f4db3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4db4159b50 0x7f4db40ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48778/0 (socket says 172.21.15.70:48778) 2023-10-08T07:13:46.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.740+0000 7f4db3fff640 1 -- 172.21.15.70:0/3806718805 learned_addr learned my addr 172.21.15.70:0/3806718805 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731906156 0 0) 0x7f4db40cd430 con 0x7f4db4159b50 2023-10-08T07:13:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db8e58640 1 --2- 172.21.15.70:0/3806718805 >> v2:172.21.15.181:3300/0 conn(0x7f4db40ae740 0x7f4db40aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4da0005730 con 0x7f4db4159b50 2023-10-08T07:13:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3918731406 0 0) 0x7f4da0005730 con 0x7f4db4159b50 2023-10-08T07:13:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 >> v2:172.21.15.181:3300/0 conn(0x7f4db40ae740 msgr2=0x7f4db40aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 --2- 172.21.15.70:0/3806718805 >> v2:172.21.15.181:3300/0 conn(0x7f4db40ae740 0x7f4db40aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 >> v1:172.21.15.181:6789/0 conn(0x7f4db415e060 legacy=0x7f4db40cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db40cf540 con 0x7f4db4159b50 2023-10-08T07:13:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4da40019f0 con 0x7f4db4159b50 2023-10-08T07:13:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 >> v1:172.21.15.70:6789/0 conn(0x7f4db4159b50 legacy=0x7f4db40ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4da0005b80 0x7f4da0005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4da00069d0 0x7f4da0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da0007390 0x7f4da000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db415e4e0 con 0x7f4da00069d0 2023-10-08T07:13:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da000e230 con 0x7f4da0007390 2023-10-08T07:13:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.741+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da000e3f0 con 0x7f4da0005b80 2023-10-08T07:13:46.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.742+0000 7f4db37fe640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4da00069d0 0x7f4da0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.742+0000 7f4db3fff640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4da0005b80 0x7f4da0005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.742+0000 7f4db8e58640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da0007390 0x7f4da000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.742+0000 7f4db37fe640 1 -- 172.21.15.70:0/3806718805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4da0005b80 msgr2=0x7f4da0005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.742+0000 7f4db37fe640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4da0005b80 0x7f4da0005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.742+0000 7f4db37fe640 1 -- 172.21.15.70:0/3806718805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da0007390 msgr2=0x7f4da000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.742+0000 7f4db37fe640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da0007390 0x7f4da000db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.742+0000 7f4db37fe640 1 -- 172.21.15.70:0/3806718805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da0005730 con 0x7f4da00069d0 2023-10-08T07:13:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.742+0000 7f4db8e58640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da0007390 0x7f4da000db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.742+0000 7f4db8e58640 1 -- 172.21.15.70:0/3806718805 reap_dead start 2023-10-08T07:13:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.743+0000 7f4db37fe640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4da00069d0 0x7f4da0006e50 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c001750 tx=0x7f4da0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7fd9947f4e9f88 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.743+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c0052d0 con 0x7f4da00069d0 2023-10-08T07:13:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.743+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c02e050 con 0x7f4da00069d0 2023-10-08T07:13:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.743+0000 7f4db2ffd640 1 -- 172.21.15.70:0/3806718805 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c003830 con 0x7f4da00069d0 2023-10-08T07:13:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.744+0000 7f4dba8bb640 1 -- 172.21.15.70:0/3806718805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4da00069d0 msgr2=0x7f4da0006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.744+0000 7f4dba8bb640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4da00069d0 0x7f4da0006e50 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c001750 tx=0x7f4da0007a00 comp rx=0 tx=0).stop 2023-10-08T07:13:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.744+0000 7f4dba8bb640 1 -- 172.21.15.70:0/3806718805 shutdown_connections 2023-10-08T07:13:46.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.744+0000 7f4dba8bb640 1 --2- 172.21.15.70:0/3806718805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4da00069d0 0x7f4da0006e50 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.744+0000 7f4dba8bb640 1 -- 172.21.15.70:0/3806718805 >> 172.21.15.70:0/3806718805 conn(0x7f4db406d430 msgr2=0x7f4db406d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.744+0000 7f4dba8bb640 1 -- 172.21.15.70:0/3806718805 shutdown_connections 2023-10-08T07:13:46.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.744+0000 7f4dba8bb640 1 -- 172.21.15.70:0/3806718805 wait complete. 2023-10-08T07:13:46.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.746+0000 7f4dba8bb640 1 Processor -- start 2023-10-08T07:13:46.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.746+0000 7f4dba8bb640 1 -- start start 2023-10-08T07:13:46.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.747+0000 7f4dba8bb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da00069d0 0x7f4db41f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.747+0000 7f4dba8bb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4db40b1c20 0x7f4db40c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.747+0000 7f4dba8bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db41f2ba0 0x7f4db41f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.747+0000 7f4dba8bb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41f24b0 con 0x7f4db41f2ba0 2023-10-08T07:13:46.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.747+0000 7f4dba8bb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db41f2750 con 0x7f4da00069d0 2023-10-08T07:13:46.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.747+0000 7f4dba8bb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40c6270 con 0x7f4db40b1c20 2023-10-08T07:13:46.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db8e58640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db41f2ba0 0x7f4db41f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db8e58640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db41f2ba0 0x7f4db41f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54484/0 (socket says 172.21.15.70:54484) 2023-10-08T07:13:46.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db8e58640 1 -- 172.21.15.70:0/290974533 learned_addr learned my addr 172.21.15.70:0/290974533 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:46.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db37fe640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4db40b1c20 0x7f4db40c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db3fff640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da00069d0 0x7f4db41f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db8e58640 1 -- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4db40b1c20 msgr2=0x7f4db40c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db8e58640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4db40b1c20 0x7f4db40c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db8e58640 1 -- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da00069d0 msgr2=0x7f4db41f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:46.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db8e58640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da00069d0 0x7f4db41f5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:46.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db8e58640 1 -- 172.21.15.70:0/290974533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db41f6120 con 0x7f4db41f2ba0 2023-10-08T07:13:46.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db3fff640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da00069d0 0x7f4db41f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:46.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db8e58640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db41f2ba0 0x7f4db41f3020 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f4db40d0930 tx=0x7f4dac010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:46.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.748+0000 7f4db0ff9640 1 -- 172.21.15.70:0/290974533 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac004230 con 0x7f4db41f2ba0 2023-10-08T07:13:46.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.749+0000 7f4db0ff9640 1 -- 172.21.15.70:0/290974533 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac004390 con 0x7f4db41f2ba0 2023-10-08T07:13:46.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.749+0000 7f4db0ff9640 1 -- 172.21.15.70:0/290974533 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00e8a0 con 0x7f4db41f2ba0 2023-10-08T07:13:46.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.749+0000 7f4dba8bb640 1 -- 172.21.15.70:0/290974533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db41f6400 con 0x7f4db41f2ba0 2023-10-08T07:13:46.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.749+0000 7f4dba8bb640 1 -- 172.21.15.70:0/290974533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db40b2040 con 0x7f4db41f2ba0 2023-10-08T07:13:46.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.750+0000 7f4dba8bb640 1 -- 172.21.15.70:0/290974533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db404e160 con 0x7f4db41f2ba0 2023-10-08T07:13:46.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.751+0000 7f4db0ff9640 1 -- 172.21.15.70:0/290974533 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4dac00ea00 con 0x7f4db41f2ba0 2023-10-08T07:13:46.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.752+0000 7f4db0ff9640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4d9406afc0 0x7f4d9406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:46.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.752+0000 7f4db0ff9640 1 -- 172.21.15.70:0/290974533 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f4dac089710 con 0x7f4db41f2ba0 2023-10-08T07:13:46.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.753+0000 7f4db3fff640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4d9406afc0 0x7f4d9406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:46.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.754+0000 7f4db3fff640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4d9406afc0 0x7f4d9406d480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4da4002230 tx=0x7f4da4002140 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:46.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.756+0000 7f4db0ff9640 1 -- 172.21.15.70:0/290974533 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4dac054450 con 0x7f4db41f2ba0 2023-10-08T07:13:46.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:46.947+0000 7f4dba8bb640 1 -- 172.21.15.70:0/290974533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4db41f6920 con 0x7f4db41f2ba0 2023-10-08T07:13:47.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.332+0000 7f4db0ff9640 1 -- 172.21.15.70:0/290974533 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v235) v1 ==== 144+0+0 (secure 0 0 0) 0x7f4dac057a70 con 0x7f4db41f2ba0 2023-10-08T07:13:47.354 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'backup_fs_metadata' created 2023-10-08T07:13:47.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.337+0000 7f4dba8bb640 1 -- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4d9406afc0 msgr2=0x7f4d9406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:47.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.337+0000 7f4dba8bb640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4d9406afc0 0x7f4d9406d480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4da4002230 tx=0x7f4da4002140 comp rx=0 tx=0).stop 2023-10-08T07:13:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.337+0000 7f4dba8bb640 1 -- 172.21.15.70:0/290974533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db41f2ba0 msgr2=0x7f4db41f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.337+0000 7f4dba8bb640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db41f2ba0 0x7f4db41f3020 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f4db40d0930 tx=0x7f4dac010c20 comp rx=0 tx=0).stop 2023-10-08T07:13:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.338+0000 7f4dba8bb640 1 -- 172.21.15.70:0/290974533 shutdown_connections 2023-10-08T07:13:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.338+0000 7f4dba8bb640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f4d9406afc0 0x7f4d9406d480 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.338+0000 7f4dba8bb640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4db41f2ba0 0x7f4db41f3020 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.338+0000 7f4dba8bb640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4db40b1c20 0x7f4db40c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.338+0000 7f4dba8bb640 1 --2- 172.21.15.70:0/290974533 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4da00069d0 0x7f4db41f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.338+0000 7f4dba8bb640 1 -- 172.21.15.70:0/290974533 >> 172.21.15.70:0/290974533 conn(0x7f4db406d430 msgr2=0x7f4db4155de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:47.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.338+0000 7f4dba8bb640 1 -- 172.21.15.70:0/290974533 shutdown_connections 2023-10-08T07:13:47.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.338+0000 7f4dba8bb640 1 -- 172.21.15.70:0/290974533 wait complete. 2023-10-08T07:13:47.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T07:13:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.592+0000 7f2c9afe2640 1 Processor -- start 2023-10-08T07:13:47.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.593+0000 7f2c9afe2640 1 -- start start 2023-10-08T07:13:47.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.594+0000 7f2c9afe2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c940cb500 0x7f2c940cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:47.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.594+0000 7f2c9afe2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2c9406cfa0 0x7f2c9415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:47.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.594+0000 7f2c9afe2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c9415b630 0x7f2c94070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:47.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.594+0000 7f2c9afe2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c940c8c60 con 0x7f2c9415b630 2023-10-08T07:13:47.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.594+0000 7f2c9afe2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c940cbe20 con 0x7f2c9406cfa0 2023-10-08T07:13:47.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.594+0000 7f2c9afe2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c940b1ac0 con 0x7f2c940cb500 2023-10-08T07:13:47.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.594+0000 7f2c9957f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c9415b630 0x7f2c94070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.594+0000 7f2c9957f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c9415b630 0x7f2c94070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54486/0 (socket says 172.21.15.70:54486) 2023-10-08T07:13:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.594+0000 7f2c9957f640 1 -- 172.21.15.70:0/3345673211 learned_addr learned my addr 172.21.15.70:0/3345673211 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.595+0000 7f2c98d7e640 1 --2- 172.21.15.70:0/3345673211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c940cb500 0x7f2c940cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.595+0000 7f2c93fff640 1 --2- 172.21.15.70:0/3345673211 >> v2:172.21.15.181:3300/0 conn(0x7f2c9406cfa0 0x7f2c9415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.595+0000 7f2c9957f640 1 -- 172.21.15.70:0/3345673211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c940cb500 msgr2=0x7f2c940cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.595+0000 7f2c9957f640 1 --2- 172.21.15.70:0/3345673211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c940cb500 0x7f2c940cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.595+0000 7f2c9957f640 1 -- 172.21.15.70:0/3345673211 >> v2:172.21.15.181:3300/0 conn(0x7f2c9406cfa0 msgr2=0x7f2c9415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:47.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.595+0000 7f2c9957f640 1 --2- 172.21.15.70:0/3345673211 >> v2:172.21.15.181:3300/0 conn(0x7f2c9406cfa0 0x7f2c9415b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.595+0000 7f2c9957f640 1 -- 172.21.15.70:0/3345673211 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940b1da0 con 0x7f2c9415b630 2023-10-08T07:13:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.596+0000 7f2c9957f640 1 --2- 172.21.15.70:0/3345673211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9415b630 0x7f2c94070690 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f2c940ce6b0 tx=0x7f2c8c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecffe00371ac370 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.596+0000 7f2c937fe640 1 -- 172.21.15.70:0/3345673211 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c02c9f0 con 0x7f2c9415b630 2023-10-08T07:13:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.596+0000 7f2c937fe640 1 -- 172.21.15.70:0/3345673211 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c02cb50 con 0x7f2c9415b630 2023-10-08T07:13:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.596+0000 7f2c937fe640 1 -- 172.21.15.70:0/3345673211 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c0343f0 con 0x7f2c9415b630 2023-10-08T07:13:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.597+0000 7f2c9afe2640 1 -- 172.21.15.70:0/3345673211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9415b630 msgr2=0x7f2c94070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.597+0000 7f2c9afe2640 1 --2- 172.21.15.70:0/3345673211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9415b630 0x7f2c94070690 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f2c940ce6b0 tx=0x7f2c8c02be50 comp rx=0 tx=0).stop 2023-10-08T07:13:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.597+0000 7f2c9afe2640 1 -- 172.21.15.70:0/3345673211 shutdown_connections 2023-10-08T07:13:47.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.597+0000 7f2c9afe2640 1 --2- 172.21.15.70:0/3345673211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9415b630 0x7f2c94070690 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.597+0000 7f2c9afe2640 1 --2- 172.21.15.70:0/3345673211 >> v2:172.21.15.181:3300/0 conn(0x7f2c9406cfa0 0x7f2c9415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.597+0000 7f2c9afe2640 1 --2- 172.21.15.70:0/3345673211 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c940cb500 0x7f2c940cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.597+0000 7f2c9afe2640 1 -- 172.21.15.70:0/3345673211 >> 172.21.15.70:0/3345673211 conn(0x7f2c9406a140 msgr2=0x7f2c9406a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.597+0000 7f2c9afe2640 1 -- 172.21.15.70:0/3345673211 shutdown_connections 2023-10-08T07:13:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.597+0000 7f2c9afe2640 1 -- 172.21.15.70:0/3345673211 wait complete. 2023-10-08T07:13:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.598+0000 7f2c9afe2640 1 Processor -- start 2023-10-08T07:13:47.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.599+0000 7f2c9afe2640 1 -- start start 2023-10-08T07:13:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c9afe2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c9406cfa0 0x7f2c9415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c9afe2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c940cb500 0x7f2c940abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c9afe2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c9415b630 0x7f2c940c1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c9afe2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c940c3870 con 0x7f2c940cb500 2023-10-08T07:13:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c9afe2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c940accd0 con 0x7f2c9406cfa0 2023-10-08T07:13:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c9afe2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c940ac710 con 0x7f2c9415b630 2023-10-08T07:13:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c940cb500 0x7f2c940abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:47.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c940cb500 0x7f2c940abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54498/0 (socket says 172.21.15.70:54498) 2023-10-08T07:13:47.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c93fff640 1 -- 172.21.15.70:0/2245700417 learned_addr learned my addr 172.21.15.70:0/2245700417 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:47.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c9957f640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c9415b630 0x7f2c940c1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:47.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c98d7e640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c9406cfa0 0x7f2c9415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:47.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c93fff640 1 -- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c9415b630 msgr2=0x7f2c940c1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:47.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c93fff640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c9415b630 0x7f2c940c1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c93fff640 1 -- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c9406cfa0 msgr2=0x7f2c9415a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:47.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c93fff640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c9406cfa0 0x7f2c9415a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:47.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.600+0000 7f2c93fff640 1 -- 172.21.15.70:0/2245700417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c009b30 con 0x7f2c940cb500 2023-10-08T07:13:47.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.601+0000 7f2c93fff640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c940cb500 0x7f2c940abda0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000c570 tx=0x7f2c8000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:47.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.601+0000 7f2c917fa640 1 -- 172.21.15.70:0/2245700417 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80011800 con 0x7f2c940cb500 2023-10-08T07:13:47.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.601+0000 7f2c917fa640 1 -- 172.21.15.70:0/2245700417 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c80011960 con 0x7f2c940cb500 2023-10-08T07:13:47.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.601+0000 7f2c917fa640 1 -- 172.21.15.70:0/2245700417 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80011d10 con 0x7f2c940cb500 2023-10-08T07:13:47.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.601+0000 7f2c9afe2640 1 -- 172.21.15.70:0/2245700417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c940ac9f0 con 0x7f2c940cb500 2023-10-08T07:13:47.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.601+0000 7f2c9afe2640 1 -- 172.21.15.70:0/2245700417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c940ac2e0 con 0x7f2c940cb500 2023-10-08T07:13:47.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.602+0000 7f2c9afe2640 1 -- 172.21.15.70:0/2245700417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c94071cb0 con 0x7f2c940cb500 2023-10-08T07:13:47.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.603+0000 7f2c917fa640 1 -- 172.21.15.70:0/2245700417 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2c80024480 con 0x7f2c940cb500 2023-10-08T07:13:47.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.604+0000 7f2c917fa640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2c7806af90 0x7f2c7806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:47.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.604+0000 7f2c917fa640 1 -- 172.21.15.70:0/2245700417 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 6518+0+0 (secure 0 0 0) 0x7f2c8008a060 con 0x7f2c940cb500 2023-10-08T07:13:47.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.605+0000 7f2c98d7e640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2c7806af90 0x7f2c7806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:47.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.606+0000 7f2c98d7e640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2c7806af90 0x7f2c7806d450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c88006fd0 tx=0x7f2c88008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:47.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.608+0000 7f2c917fa640 1 -- 172.21.15.70:0/2245700417 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c80054ba0 con 0x7f2c940cb500 2023-10-08T07:13:47.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:47.785+0000 7f2c9afe2640 1 -- 172.21.15.70:0/2245700417 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2c940abb80 con 0x7f2c940cb500 2023-10-08T07:13:48.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.352+0000 7f2c917fa640 1 -- 172.21.15.70:0/2245700417 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v236) v1 ==== 176+0+0 (secure 0 0 0) 0x7f2c8001f0d0 con 0x7f2c940cb500 2023-10-08T07:13:48.354 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'backup_fs_data' created 2023-10-08T07:13:48.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.356+0000 7f2c9afe2640 1 -- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2c7806af90 msgr2=0x7f2c7806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:48.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.356+0000 7f2c9afe2640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2c7806af90 0x7f2c7806d450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c88006fd0 tx=0x7f2c88008040 comp rx=0 tx=0).stop 2023-10-08T07:13:48.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.356+0000 7f2c9afe2640 1 -- 172.21.15.70:0/2245700417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c940cb500 msgr2=0x7f2c940abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:48.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.356+0000 7f2c9afe2640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c940cb500 0x7f2c940abda0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000c570 tx=0x7f2c8000ca30 comp rx=0 tx=0).stop 2023-10-08T07:13:48.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.357+0000 7f2c9afe2640 1 -- 172.21.15.70:0/2245700417 shutdown_connections 2023-10-08T07:13:48.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.357+0000 7f2c9afe2640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c9415b630 0x7f2c940c1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:48.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.357+0000 7f2c9afe2640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2c7806af90 0x7f2c7806d450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:48.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.357+0000 7f2c9afe2640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c940cb500 0x7f2c940abda0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:48.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.357+0000 7f2c9afe2640 1 --2- 172.21.15.70:0/2245700417 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c9406cfa0 0x7f2c9415a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:48.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.357+0000 7f2c9afe2640 1 -- 172.21.15.70:0/2245700417 >> 172.21.15.70:0/2245700417 conn(0x7f2c9406a140 msgr2=0x7f2c940cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:48.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.357+0000 7f2c9afe2640 1 -- 172.21.15.70:0/2245700417 shutdown_connections 2023-10-08T07:13:48.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.357+0000 7f2c9afe2640 1 -- 172.21.15.70:0/2245700417 wait complete. 2023-10-08T07:13:48.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-10-08T07:13:48.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.577+0000 7f9ea1bb6640 1 Processor -- start 2023-10-08T07:13:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.577+0000 7f9ea1bb6640 1 -- start start 2023-10-08T07:13:48.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.578+0000 7f9ea1bb6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c06dfc0 0x7f9e9c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:48.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.578+0000 7f9ea1bb6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9e9c071070 0x7f9e9c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:48.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.578+0000 7f9ea1bb6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e9c0b1cc0 con 0x7f9e9c0afbe0 2023-10-08T07:13:48.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.578+0000 7f9ea1bb6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e9c0b2620 con 0x7f9e9c071070 2023-10-08T07:13:48.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9ea1bb6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0b2060 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9e9b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9e9c0afbe0 0x7f9e9c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48786/0 (socket says 172.21.15.70:48786) 2023-10-08T07:13:48.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9e9b7fe640 1 -- 172.21.15.70:0/2936402573 learned_addr learned my addr 172.21.15.70:0/2936402573 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:48.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9e9bfff640 1 --2- 172.21.15.70:0/2936402573 >> v2:172.21.15.70:3300/0 conn(0x7f9e9c071070 0x7f9e9c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9e93fff640 1 --2- 172.21.15.70:0/2936402573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c06dfc0 0x7f9e9c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9e9affd640 1 -- 172.21.15.70:0/2936402573 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574195629 0 0) 0x7f9e9c0b1cc0 con 0x7f9e9c0afbe0 2023-10-08T07:13:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9e9affd640 1 -- 172.21.15.70:0/2936402573 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e88003520 con 0x7f9e9c0afbe0 2023-10-08T07:13:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9e9bfff640 1 -- 172.21.15.70:0/2936402573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c06dfc0 msgr2=0x7f9e9c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9e9bfff640 1 --2- 172.21.15.70:0/2936402573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c06dfc0 0x7f9e9c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9e9bfff640 1 -- 172.21.15.70:0/2936402573 >> v1:172.21.15.70:6789/0 conn(0x7f9e9c0afbe0 legacy=0x7f9e9c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.579+0000 7f9e9bfff640 1 -- 172.21.15.70:0/2936402573 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c0b2340 con 0x7f9e9c071070 2023-10-08T07:13:48.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.581+0000 7f9e9bfff640 1 --2- 172.21.15.70:0/2936402573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c071070 0x7f9e9c0cb130 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009d90 tx=0x7f9e9402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be79d709685ba89 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:48.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.581+0000 7f9e9affd640 1 -- 172.21.15.70:0/2936402573 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9402cc50 con 0x7f9e9c071070 2023-10-08T07:13:48.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.581+0000 7f9e9affd640 1 -- 172.21.15.70:0/2936402573 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9402cdb0 con 0x7f9e9c071070 2023-10-08T07:13:48.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.581+0000 7f9e9affd640 1 -- 172.21.15.70:0/2936402573 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94034640 con 0x7f9e9c071070 2023-10-08T07:13:48.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.581+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2936402573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c071070 msgr2=0x7f9e9c0cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:48.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.581+0000 7f9ea1bb6640 1 --2- 172.21.15.70:0/2936402573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c071070 0x7f9e9c0cb130 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009d90 tx=0x7f9e9402bea0 comp rx=0 tx=0).stop 2023-10-08T07:13:48.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.582+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2936402573 shutdown_connections 2023-10-08T07:13:48.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.582+0000 7f9ea1bb6640 1 --2- 172.21.15.70:0/2936402573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c071070 0x7f9e9c0cb130 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:48.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.582+0000 7f9ea1bb6640 1 --2- 172.21.15.70:0/2936402573 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c06dfc0 0x7f9e9c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:48.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.582+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2936402573 >> 172.21.15.70:0/2936402573 conn(0x7f9e9c06a5d0 msgr2=0x7f9e9c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:48.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.582+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2936402573 shutdown_connections 2023-10-08T07:13:48.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.582+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2936402573 wait complete. 2023-10-08T07:13:48.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.583+0000 7f9ea1bb6640 1 Processor -- start 2023-10-08T07:13:48.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.584+0000 7f9ea1bb6640 1 -- start start 2023-10-08T07:13:48.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.584+0000 7f9ea1bb6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c06dfc0 0x7f9e9c1eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:48.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.584+0000 7f9ea1bb6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e9c071070 0x7f9e9c0c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:48.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.584+0000 7f9ea1bb6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c0afbe0 0x7f9e9c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:48.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.584+0000 7f9ea1bb6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c0c46c0 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.584+0000 7f9ea1bb6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c15c640 con 0x7f9e9c071070 2023-10-08T07:13:48.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.584+0000 7f9ea1bb6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c15d130 con 0x7f9e9c0afbe0 2023-10-08T07:13:48.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c06dfc0 0x7f9e9c1eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:48.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c06dfc0 0x7f9e9c1eff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54502/0 (socket says 172.21.15.70:54502) 2023-10-08T07:13:48.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e9b7fe640 1 -- 172.21.15.70:0/2622393213 learned_addr learned my addr 172.21.15.70:0/2622393213 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:48.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e9bfff640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c0afbe0 0x7f9e9c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:48.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e93fff640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e9c071070 0x7f9e9c0c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e9b7fe640 1 -- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c0afbe0 msgr2=0x7f9e9c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e9b7fe640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c0afbe0 0x7f9e9c0cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e9b7fe640 1 -- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e9c071070 msgr2=0x7f9e9c0c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e9b7fe640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e9c071070 0x7f9e9c0c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e9b7fe640 1 -- 172.21.15.70:0/2622393213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e94009a70 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e9bfff640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c0afbe0 0x7f9e9c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.585+0000 7f9e93fff640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e9c071070 0x7f9e9c0c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.586+0000 7f9e9b7fe640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c06dfc0 0x7f9e9c1eff50 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00c430 tx=0x7f9e8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.586+0000 7f9e98ff9640 1 -- 172.21.15.70:0/2622393213 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c005ce0 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.586+0000 7f9e98ff9640 1 -- 172.21.15.70:0/2622393213 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c005e40 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.586+0000 7f9e98ff9640 1 -- 172.21.15.70:0/2622393213 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0072f0 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.586+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2622393213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9c15c890 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.586+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2622393213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9c15ce50 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.587+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2622393213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9c04e160 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.589+0000 7f9e98ff9640 1 -- 172.21.15.70:0/2622393213 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9e8c007450 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.590+0000 7f9e98ff9640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9e7c06af90 0x7f9e7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:48.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.590+0000 7f9e98ff9640 1 -- 172.21.15.70:0/2622393213 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f9e8c089320 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.591+0000 7f9e93fff640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9e7c06af90 0x7f9e7c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:48.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.592+0000 7f9e93fff640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9e7c06af90 0x7f9e7c06d450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9e84006fd0 tx=0x7f9e84008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:48.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.595+0000 7f9e98ff9640 1 -- 172.21.15.70:0/2622393213 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e8c053e00 con 0x7f9e9c06dfc0 2023-10-08T07:13:48.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:48.826+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2622393213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f9e9c0b0ef0 con 0x7f9e9c06dfc0 2023-10-08T07:13:49.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.354+0000 7f9e98ff9640 1 -- 172.21.15.70:0/2622393213 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 29 and data pool 30 v110) v1 ==== 183+0+0 (secure 0 0 0) 0x7f9e8c057420 con 0x7f9e9c06dfc0 2023-10-08T07:13:49.356 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 29 and data pool 30 2023-10-08T07:13:49.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9e7c06af90 msgr2=0x7f9e7c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:49.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9e7c06af90 0x7f9e7c06d450 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9e84006fd0 tx=0x7f9e84008040 comp rx=0 tx=0).stop 2023-10-08T07:13:49.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2622393213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c06dfc0 msgr2=0x7f9e9c1eff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:49.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c06dfc0 0x7f9e9c1eff50 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00c430 tx=0x7f9e8c00c8f0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2622393213 shutdown_connections 2023-10-08T07:13:49.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9e9c0afbe0 0x7f9e9c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9e9c071070 0x7f9e9c0c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f9e7c06af90 0x7f9e7c06d450 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 --2- 172.21.15.70:0/2622393213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e9c06dfc0 0x7f9e9c1eff50 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2622393213 >> 172.21.15.70:0/2622393213 conn(0x7f9e9c06a5d0 msgr2=0x7f9e9c159840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:49.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2622393213 shutdown_connections 2023-10-08T07:13:49.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.357+0000 7f9ea1bb6640 1 -- 172.21.15.70:0/2622393213 wait complete. 2023-10-08T07:13:49.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.574+0000 7f647f59e640 1 Processor -- start 2023-10-08T07:13:49.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.574+0000 7f647f59e640 1 -- start start 2023-10-08T07:13:49.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.575+0000 7f647f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f64801599d0 0x7f64800ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:49.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.576+0000 7f647f59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64800cacf0 con 0x7f64800cc210 2023-10-08T07:13:49.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.576+0000 7f647f59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64800ca3c0 con 0x7f64800b07d0 2023-10-08T07:13:49.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.576+0000 7f647f59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64800b10f0 con 0x7f64801599d0 2023-10-08T07:13:49.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.576+0000 7f647e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f64801599d0 0x7f64800ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:49.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.576+0000 7f6477fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f64800cc210 0x7f64800b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48802/0 (socket says 172.21.15.70:48802) 2023-10-08T07:13:49.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.576+0000 7f6477fff640 1 -- 172.21.15.70:0/2136737830 learned_addr learned my addr 172.21.15.70:0/2136737830 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:49.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.576+0000 7f647dd9b640 1 -- 172.21.15.70:0/2136737830 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295942758 0 0) 0x7f64800cacf0 con 0x7f64800cc210 2023-10-08T07:13:49.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.576+0000 7f647e59c640 1 -- 172.21.15.70:0/2136737830 >> v1:172.21.15.181:6789/0 conn(0x7f64800b07d0 legacy=0x7f648006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:49.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.576+0000 7f647e59c640 1 -- 172.21.15.70:0/2136737830 >> v1:172.21.15.70:6789/0 conn(0x7f64800cc210 legacy=0x7f64800b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:49.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.576+0000 7f647e59c640 1 -- 172.21.15.70:0/2136737830 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64800ca6a0 con 0x7f64801599d0 2023-10-08T07:13:49.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.577+0000 7f647e59c640 1 --2- 172.21.15.70:0/2136737830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64801599d0 0x7f64800ad730 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f64700097a0 tx=0x7f647002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4194940c7cb67afb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:49.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.578+0000 7f647dd9b640 1 -- 172.21.15.70:0/2136737830 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470031830 con 0x7f64801599d0 2023-10-08T07:13:49.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.578+0000 7f647dd9b640 1 -- 172.21.15.70:0/2136737830 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6470031990 con 0x7f64801599d0 2023-10-08T07:13:49.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.578+0000 7f647dd9b640 1 -- 172.21.15.70:0/2136737830 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470031d20 con 0x7f64801599d0 2023-10-08T07:13:49.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.578+0000 7f647f59e640 1 -- 172.21.15.70:0/2136737830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64801599d0 msgr2=0x7f64800ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:49.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.578+0000 7f647f59e640 1 --2- 172.21.15.70:0/2136737830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64801599d0 0x7f64800ad730 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f64700097a0 tx=0x7f647002b8d0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.579+0000 7f647f59e640 1 -- 172.21.15.70:0/2136737830 shutdown_connections 2023-10-08T07:13:49.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.579+0000 7f647f59e640 1 --2- 172.21.15.70:0/2136737830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64801599d0 0x7f64800ad730 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.579+0000 7f647f59e640 1 -- 172.21.15.70:0/2136737830 >> 172.21.15.70:0/2136737830 conn(0x7f64800cb0c0 msgr2=0x7f64800cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:49.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.579+0000 7f647f59e640 1 -- 172.21.15.70:0/2136737830 shutdown_connections 2023-10-08T07:13:49.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.579+0000 7f647f59e640 1 -- 172.21.15.70:0/2136737830 wait complete. 2023-10-08T07:13:49.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.580+0000 7f647f59e640 1 Processor -- start 2023-10-08T07:13:49.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.581+0000 7f647f59e640 1 -- start start 2023-10-08T07:13:49.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.581+0000 7f647f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800b07d0 0x7f648015c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:49.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64800cc210 0x7f648014bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:49.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64801599d0 0x7f64800c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:49.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64800c8b40 con 0x7f64800b07d0 2023-10-08T07:13:49.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647f59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64800c8580 con 0x7f64801599d0 2023-10-08T07:13:49.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64800c8860 con 0x7f64800cc210 2023-10-08T07:13:49.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800b07d0 0x7f648015c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:49.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800b07d0 0x7f648015c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54526/0 (socket says 172.21.15.70:54526) 2023-10-08T07:13:49.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647e59c640 1 -- 172.21.15.70:0/465252698 learned_addr learned my addr 172.21.15.70:0/465252698 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:49.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647e59c640 1 -- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64800cc210 msgr2=0x7f648014bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:49.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647e59c640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64800cc210 0x7f648014bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647e59c640 1 -- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64801599d0 msgr2=0x7f64800c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:49.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647e59c640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64801599d0 0x7f64800c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.582+0000 7f647e59c640 1 -- 172.21.15.70:0/465252698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6470009400 con 0x7f64800b07d0 2023-10-08T07:13:49.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.583+0000 7f647e59c640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800b07d0 0x7f648015c660 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f64700098c0 tx=0x7f64700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:49.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.583+0000 7f6476ffd640 1 -- 172.21.15.70:0/465252698 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647003ba70 con 0x7f64800b07d0 2023-10-08T07:13:49.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.583+0000 7f6476ffd640 1 -- 172.21.15.70:0/465252698 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647003bbf0 con 0x7f64800b07d0 2023-10-08T07:13:49.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.583+0000 7f6476ffd640 1 -- 172.21.15.70:0/465252698 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64700453f0 con 0x7f64800b07d0 2023-10-08T07:13:49.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.583+0000 7f647f59e640 1 -- 172.21.15.70:0/465252698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64800c72a0 con 0x7f64800b07d0 2023-10-08T07:13:49.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.583+0000 7f647f59e640 1 -- 172.21.15.70:0/465252698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64800c7fe0 con 0x7f64800b07d0 2023-10-08T07:13:49.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.584+0000 7f647f59e640 1 -- 172.21.15.70:0/465252698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648004e160 con 0x7f64800b07d0 2023-10-08T07:13:49.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.585+0000 7f6476ffd640 1 -- 172.21.15.70:0/465252698 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f647004e450 con 0x7f64800b07d0 2023-10-08T07:13:49.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.587+0000 7f6476ffd640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f646006af90 0x7f646006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:49.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.587+0000 7f6476ffd640 1 -- 172.21.15.70:0/465252698 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f64700ac070 con 0x7f64800b07d0 2023-10-08T07:13:49.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.587+0000 7f6477fff640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f646006af90 0x7f646006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.589+0000 7f6477fff640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f646006af90 0x7f646006d450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6468007fd0 tx=0x7f6468007920 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:49.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.592+0000 7f6476ffd640 1 -- 172.21.15.70:0/465252698 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64700deca0 con 0x7f64800b07d0 2023-10-08T07:13:49.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.786+0000 7f647f59e640 1 -- 172.21.15.70:0/465252698 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64800cb960 con 0x7f64800b07d0 2023-10-08T07:13:49.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.790+0000 7f6476ffd640 1 -- 172.21.15.70:0/465252698 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v237) v1 ==== 74+0+20627 (secure 0 0 0) 0x7f647007a460 con 0x7f64800b07d0 2023-10-08T07:13:49.792 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:49.792 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":237,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:48.828980+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:13:47.007437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:13:47.838023+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:49.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 -- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f646006af90 msgr2=0x7f646006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:49.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f646006af90 0x7f646006d450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6468007fd0 tx=0x7f6468007920 comp rx=0 tx=0).stop 2023-10-08T07:13:49.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 -- 172.21.15.70:0/465252698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800b07d0 msgr2=0x7f648015c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:49.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800b07d0 0x7f648015c660 secure :-1 s=READY pgs=2400 cs=0 l=1 rev1=1 crypto rx=0x7f64700098c0 tx=0x7f64700046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 -- 172.21.15.70:0/465252698 shutdown_connections 2023-10-08T07:13:49.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f64801599d0 0x7f64800c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f64800cc210 0x7f648014bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f646006af90 0x7f646006d450 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 --2- 172.21.15.70:0/465252698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64800b07d0 0x7f648015c660 unknown :-1 s=CLOSED pgs=2400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:49.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 -- 172.21.15.70:0/465252698 >> 172.21.15.70:0/465252698 conn(0x7f64800cb0c0 msgr2=0x7f64800ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:49.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 -- 172.21.15.70:0/465252698 shutdown_connections 2023-10-08T07:13:49.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:49.793+0000 7f647f59e640 1 -- 172.21.15.70:0/465252698 wait complete. 2023-10-08T07:13:49.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:50.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.021+0000 7f98d230c640 1 Processor -- start 2023-10-08T07:13:50.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.021+0000 7f98d230c640 1 -- start start 2023-10-08T07:13:50.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.022+0000 7f98d230c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98cc15cca0 0x7f98cc1614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d230c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f98cc0b07d0 0x7f98cc0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d230c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc0c9020 con 0x7f98cc0cab40 2023-10-08T07:13:50.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d230c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc0ccc20 con 0x7f98cc0b07d0 2023-10-08T07:13:50.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d230c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0c92e0 con 0x7f98cc15cca0 2023-10-08T07:13:50.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d0b09640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f98cc0cab40 0x7f98cc0b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48818/0 (socket says 172.21.15.70:48818) 2023-10-08T07:13:50.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d0b09640 1 -- 172.21.15.70:0/4026609294 learned_addr learned my addr 172.21.15.70:0/4026609294 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:50.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d1b0b640 1 --2- 172.21.15.70:0/4026609294 >> v2:172.21.15.70:3300/0 conn(0x7f98cc0b07d0 0x7f98cc0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:50.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d130a640 1 --2- 172.21.15.70:0/4026609294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98cc15cca0 0x7f98cc1614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:50.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d1b0b640 1 -- 172.21.15.70:0/4026609294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98cc15cca0 msgr2=0x7f98cc1614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:50.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d1b0b640 1 --2- 172.21.15.70:0/4026609294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98cc15cca0 0x7f98cc1614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d1b0b640 1 -- 172.21.15.70:0/4026609294 >> v1:172.21.15.70:6789/0 conn(0x7f98cc0cab40 legacy=0x7f98cc0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.023+0000 7f98d1b0b640 1 -- 172.21.15.70:0/4026609294 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0cc7b0 con 0x7f98cc0b07d0 2023-10-08T07:13:50.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.025+0000 7f98d1b0b640 1 --2- 172.21.15.70:0/4026609294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc0b07d0 0x7f98cc0c9770 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009d90 tx=0x7f98c402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1352ff768eb7d13a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:50.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.025+0000 7f98c3fff640 1 -- 172.21.15.70:0/4026609294 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c402cb40 con 0x7f98cc0b07d0 2023-10-08T07:13:50.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.025+0000 7f98c3fff640 1 -- 172.21.15.70:0/4026609294 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c402cca0 con 0x7f98cc0b07d0 2023-10-08T07:13:50.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.025+0000 7f98c3fff640 1 -- 172.21.15.70:0/4026609294 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4034660 con 0x7f98cc0b07d0 2023-10-08T07:13:50.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.025+0000 7f98d230c640 1 -- 172.21.15.70:0/4026609294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc0b07d0 msgr2=0x7f98cc0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.025+0000 7f98d230c640 1 --2- 172.21.15.70:0/4026609294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc0b07d0 0x7f98cc0c9770 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009d90 tx=0x7f98c402bd90 comp rx=0 tx=0).stop 2023-10-08T07:13:50.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.026+0000 7f98d230c640 1 -- 172.21.15.70:0/4026609294 shutdown_connections 2023-10-08T07:13:50.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.026+0000 7f98d230c640 1 --2- 172.21.15.70:0/4026609294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc0b07d0 0x7f98cc0c9770 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.026+0000 7f98d230c640 1 --2- 172.21.15.70:0/4026609294 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98cc15cca0 0x7f98cc1614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.026+0000 7f98d230c640 1 -- 172.21.15.70:0/4026609294 >> 172.21.15.70:0/4026609294 conn(0x7f98cc0cb0c0 msgr2=0x7f98cc0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:50.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.026+0000 7f98d230c640 1 -- 172.21.15.70:0/4026609294 shutdown_connections 2023-10-08T07:13:50.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.026+0000 7f98d230c640 1 -- 172.21.15.70:0/4026609294 wait complete. 2023-10-08T07:13:50.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.027+0000 7f98d230c640 1 Processor -- start 2023-10-08T07:13:50.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.028+0000 7f98d230c640 1 -- start start 2023-10-08T07:13:50.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.028+0000 7f98d230c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98cc0b07d0 0x7f98cc14e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.028+0000 7f98d230c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98cc0cab40 0x7f98cc14e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.028+0000 7f98d230c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc15cca0 0x7f98cc15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.028+0000 7f98d230c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0b25b0 con 0x7f98cc15cca0 2023-10-08T07:13:50.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.028+0000 7f98d230c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0b29d0 con 0x7f98cc0cab40 2023-10-08T07:13:50.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.028+0000 7f98d230c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc0b3ef0 con 0x7f98cc0b07d0 2023-10-08T07:13:50.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.029+0000 7f98d1b0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc15cca0 0x7f98cc15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:50.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.029+0000 7f98d1b0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc15cca0 0x7f98cc15f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54536/0 (socket says 172.21.15.70:54536) 2023-10-08T07:13:50.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.029+0000 7f98d1b0b640 1 -- 172.21.15.70:0/1984197553 learned_addr learned my addr 172.21.15.70:0/1984197553 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:50.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.029+0000 7f98d1b0b640 1 -- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98cc0b07d0 msgr2=0x7f98cc14e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.029+0000 7f98d1b0b640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98cc0b07d0 0x7f98cc14e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.029+0000 7f98d1b0b640 1 -- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98cc0cab40 msgr2=0x7f98cc14e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.029+0000 7f98d1b0b640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98cc0cab40 0x7f98cc14e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.029+0000 7f98d1b0b640 1 -- 172.21.15.70:0/1984197553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c4009a70 con 0x7f98cc15cca0 2023-10-08T07:13:50.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.030+0000 7f98d1b0b640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc15cca0 0x7f98cc15f0a0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009eb0 tx=0x7f98c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:50.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.030+0000 7f98c1ffb640 1 -- 172.21.15.70:0/1984197553 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4040650 con 0x7f98cc15cca0 2023-10-08T07:13:50.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.030+0000 7f98c1ffb640 1 -- 172.21.15.70:0/1984197553 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c40407d0 con 0x7f98cc15cca0 2023-10-08T07:13:50.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.030+0000 7f98c1ffb640 1 -- 172.21.15.70:0/1984197553 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4040b80 con 0x7f98cc15cca0 2023-10-08T07:13:50.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.030+0000 7f98d230c640 1 -- 172.21.15.70:0/1984197553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc15a140 con 0x7f98cc15cca0 2023-10-08T07:13:50.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.030+0000 7f98d230c640 1 -- 172.21.15.70:0/1984197553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc0b1c10 con 0x7f98cc15cca0 2023-10-08T07:13:50.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.031+0000 7f98d230c640 1 -- 172.21.15.70:0/1984197553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc04e160 con 0x7f98cc15cca0 2023-10-08T07:13:50.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.032+0000 7f98c1ffb640 1 -- 172.21.15.70:0/1984197553 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f98c4040ce0 con 0x7f98cc15cca0 2023-10-08T07:13:50.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.033+0000 7f98c1ffb640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f98b006af90 0x7f98b006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.033+0000 7f98c1ffb640 1 -- 172.21.15.70:0/1984197553 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f98c40aca90 con 0x7f98cc15cca0 2023-10-08T07:13:50.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.033+0000 7f98d130a640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f98b006af90 0x7f98b006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:50.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.034+0000 7f98d130a640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f98b006af90 0x7f98b006d450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f98c80098f0 tx=0x7f98c8008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:50.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.037+0000 7f98c1ffb640 1 -- 172.21.15.70:0/1984197553 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98c4077500 con 0x7f98cc15cca0 2023-10-08T07:13:50.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.227+0000 7f98d230c640 1 -- 172.21.15.70:0/1984197553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98cc0b45c0 con 0x7f98cc15cca0 2023-10-08T07:13:50.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.229+0000 7f98c1ffb640 1 -- 172.21.15.70:0/1984197553 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v237) v1 ==== 74+0+20627 (secure 0 0 0) 0x7f98c407ab20 con 0x7f98cc15cca0 2023-10-08T07:13:50.231 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:50.231 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":237,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:48.828980+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:13:47.007437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:13:47.838023+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:50.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 -- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f98b006af90 msgr2=0x7f98b006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f98b006af90 0x7f98b006d450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f98c80098f0 tx=0x7f98c8008040 comp rx=0 tx=0).stop 2023-10-08T07:13:50.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 -- 172.21.15.70:0/1984197553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc15cca0 msgr2=0x7f98cc15f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc15cca0 0x7f98cc15f0a0 secure :-1 s=READY pgs=2403 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009eb0 tx=0x7f98c4004a40 comp rx=0 tx=0).stop 2023-10-08T07:13:50.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 -- 172.21.15.70:0/1984197553 shutdown_connections 2023-10-08T07:13:50.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f98b006af90 0x7f98b006d450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98cc15cca0 0x7f98cc15f0a0 unknown :-1 s=CLOSED pgs=2403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98cc0cab40 0x7f98cc14e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 --2- 172.21.15.70:0/1984197553 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98cc0b07d0 0x7f98cc14e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 -- 172.21.15.70:0/1984197553 >> 172.21.15.70:0/1984197553 conn(0x7f98cc0cb0c0 msgr2=0x7f98cc0ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:50.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 -- 172.21.15.70:0/1984197553 shutdown_connections 2023-10-08T07:13:50.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.232+0000 7f98d230c640 1 -- 172.21.15.70:0/1984197553 wait complete. 2023-10-08T07:13:50.253 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-10-08T07:13:50.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.468+0000 7f65e7e5d640 1 Processor -- start 2023-10-08T07:13:50.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.468+0000 7f65e7e5d640 1 -- start start 2023-10-08T07:13:50.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.470+0000 7f65e7e5d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e00cb600 0x7f65e00b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.470+0000 7f65e7e5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e006e1e0 con 0x7f65e0161690 2023-10-08T07:13:50.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.470+0000 7f65e7e5d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e00cba50 con 0x7f65e00cb150 2023-10-08T07:13:50.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.470+0000 7f65e7e5d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e006cc50 con 0x7f65e00cb600 2023-10-08T07:13:50.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.470+0000 7f65e5bf9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f65e0161690 0x7f65e006dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48830/0 (socket says 172.21.15.70:48830) 2023-10-08T07:13:50.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.470+0000 7f65e5bf9640 1 -- 172.21.15.70:0/4098739727 learned_addr learned my addr 172.21.15.70:0/4098739727 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e63fa640 1 --2- 172.21.15.70:0/4098739727 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e00cb600 0x7f65e00b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283863486 0 0) 0x7f65e006e1e0 con 0x7f65e0161690 2023-10-08T07:13:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65cc003520 con 0x7f65e0161690 2023-10-08T07:13:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199380476 0 0) 0x7f65cc003520 con 0x7f65e0161690 2023-10-08T07:13:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e00cb600 msgr2=0x7f65e00b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 --2- 172.21.15.70:0/4098739727 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e00cb600 0x7f65e00b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 >> v1:172.21.15.181:6789/0 conn(0x7f65e00cb150 legacy=0x7f65e00b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e006cf30 con 0x7f65e0161690 2023-10-08T07:13:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f65dc001690 con 0x7f65e0161690 2023-10-08T07:13:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 >> v1:172.21.15.70:6789/0 conn(0x7f65e0161690 legacy=0x7f65e006dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 --2- 172.21.15.70:0/4098739727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65cc0039a0 0x7f65cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 --2- 172.21.15.70:0/4098739727 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e00cb600 0x7f65cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 --2- 172.21.15.70:0/4098739727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc004d00 0x7f65cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e006cc50 con 0x7f65cc0039a0 2023-10-08T07:13:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc00ba60 con 0x7f65cc004d00 2023-10-08T07:13:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc00bc20 con 0x7f65e00cb600 2023-10-08T07:13:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.471+0000 7f65e5bf9640 1 --2- 172.21.15.70:0/4098739727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65cc0039a0 0x7f65cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.472+0000 7f65e5bf9640 1 -- 172.21.15.70:0/4098739727 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e00cb600 msgr2=0x7f65cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.472+0000 7f65e5bf9640 1 --2- 172.21.15.70:0/4098739727 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e00cb600 0x7f65cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.472+0000 7f65e5bf9640 1 -- 172.21.15.70:0/4098739727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc004d00 msgr2=0x7f65cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.472+0000 7f65e5bf9640 1 --2- 172.21.15.70:0/4098739727 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc004d00 0x7f65cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.472+0000 7f65e5bf9640 1 -- 172.21.15.70:0/4098739727 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc003520 con 0x7f65cc0039a0 2023-10-08T07:13:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.473+0000 7f65e5bf9640 1 --2- 172.21.15.70:0/4098739727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65cc0039a0 0x7f65cc003dc0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f65dc001eb0 tx=0x7f65cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1880a9f8ec2be7c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.473+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc005ad0 con 0x7f65cc0039a0 2023-10-08T07:13:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.473+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc005c30 con 0x7f65cc0039a0 2023-10-08T07:13:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.473+0000 7f65e4bf7640 1 -- 172.21.15.70:0/4098739727 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc02c890 con 0x7f65cc0039a0 2023-10-08T07:13:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.474+0000 7f65e7e5d640 1 -- 172.21.15.70:0/4098739727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65cc0039a0 msgr2=0x7f65cc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.474+0000 7f65e7e5d640 1 --2- 172.21.15.70:0/4098739727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65cc0039a0 0x7f65cc003dc0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f65dc001eb0 tx=0x7f65cc0057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.474+0000 7f65e63fa640 1 -- 172.21.15.70:0/4098739727 reap_dead start 2023-10-08T07:13:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.474+0000 7f65e7e5d640 1 -- 172.21.15.70:0/4098739727 shutdown_connections 2023-10-08T07:13:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.474+0000 7f65e7e5d640 1 -- 172.21.15.70:0/4098739727 >> 172.21.15.70:0/4098739727 conn(0x7f65e006a5d0 msgr2=0x7f65e00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.474+0000 7f65e7e5d640 1 -- 172.21.15.70:0/4098739727 shutdown_connections 2023-10-08T07:13:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.474+0000 7f65e7e5d640 1 -- 172.21.15.70:0/4098739727 wait complete. 2023-10-08T07:13:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.476+0000 7f65e7e5d640 1 Processor -- start 2023-10-08T07:13:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.476+0000 7f65e7e5d640 1 -- start start 2023-10-08T07:13:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.477+0000 7f65e7e5d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc0039a0 0x7f65e015a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.477+0000 7f65e7e5d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e01646e0 0x7f65e0154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.477+0000 7f65e7e5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e0163420 0x7f65e01638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.477+0000 7f65e7e5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0162f90 con 0x7f65e0163420 2023-10-08T07:13:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.477+0000 7f65e7e5d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0162380 con 0x7f65cc0039a0 2023-10-08T07:13:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.477+0000 7f65e7e5d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0162660 con 0x7f65e01646e0 2023-10-08T07:13:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.477+0000 7f65e63fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e0163420 0x7f65e01638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.477+0000 7f65e63fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e0163420 0x7f65e01638a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54548/0 (socket says 172.21.15.70:54548) 2023-10-08T07:13:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.477+0000 7f65e63fa640 1 -- 172.21.15.70:0/1555165312 learned_addr learned my addr 172.21.15.70:0/1555165312 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.477+0000 7f65e53f8640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e01646e0 0x7f65e0154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65e5bf9640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc0039a0 0x7f65e015a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65e63fa640 1 -- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e01646e0 msgr2=0x7f65e0154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65e63fa640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e01646e0 0x7f65e0154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65e63fa640 1 -- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc0039a0 msgr2=0x7f65e015a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65e63fa640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc0039a0 0x7f65e015a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65e63fa640 1 -- 172.21.15.70:0/1555165312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0162940 con 0x7f65e0163420 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65e63fa640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e0163420 0x7f65e01638a0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f65e006e5a0 tx=0x7f65d800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65d67fc640 1 -- 172.21.15.70:0/1555165312 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8010040 con 0x7f65e0163420 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65d67fc640 1 -- 172.21.15.70:0/1555165312 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d800bae0 con 0x7f65e0163420 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65d67fc640 1 -- 172.21.15.70:0/1555165312 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8014400 con 0x7f65e0163420 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65e7e5d640 1 -- 172.21.15.70:0/1555165312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e0162c20 con 0x7f65e0163420 2023-10-08T07:13:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.478+0000 7f65e7e5d640 1 -- 172.21.15.70:0/1555165312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e0154650 con 0x7f65e0163420 2023-10-08T07:13:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.480+0000 7f65e7e5d640 1 -- 172.21.15.70:0/1555165312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e004e160 con 0x7f65e0163420 2023-10-08T07:13:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.481+0000 7f65d67fc640 1 -- 172.21.15.70:0/1555165312 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f65d8014560 con 0x7f65e0163420 2023-10-08T07:13:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.482+0000 7f65d67fc640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f65c406afc0 0x7f65c406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:50.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.482+0000 7f65d67fc640 1 -- 172.21.15.70:0/1555165312 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f65d808a3d0 con 0x7f65e0163420 2023-10-08T07:13:50.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.483+0000 7f65e5bf9640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f65c406afc0 0x7f65c406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.484+0000 7f65e5bf9640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f65c406afc0 0x7f65c406d480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65dc034000 tx=0x7f65dc009030 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:50.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.487+0000 7f65d67fc640 1 -- 172.21.15.70:0/1555165312 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65d8054d30 con 0x7f65e0163420 2023-10-08T07:13:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:50.715+0000 7f65e7e5d640 1 -- 172.21.15.70:0/1555165312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f65e00c9ef0 con 0x7f65e0163420 2023-10-08T07:13:51.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.379+0000 7f65d67fc640 1 -- 172.21.15.70:0/1555165312 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v113) v1 ==== 121+0+0 (secure 0 0 0) 0x7f65d8058350 con 0x7f65e0163420 2023-10-08T07:13:51.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 -- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f65c406afc0 msgr2=0x7f65c406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:51.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f65c406afc0 0x7f65c406d480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65dc034000 tx=0x7f65dc009030 comp rx=0 tx=0).stop 2023-10-08T07:13:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 -- 172.21.15.70:0/1555165312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e0163420 msgr2=0x7f65e01638a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e0163420 0x7f65e01638a0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f65e006e5a0 tx=0x7f65d800b480 comp rx=0 tx=0).stop 2023-10-08T07:13:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 -- 172.21.15.70:0/1555165312 shutdown_connections 2023-10-08T07:13:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f65c406afc0 0x7f65c406d480 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65e0163420 0x7f65e01638a0 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65e01646e0 0x7f65e0154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 --2- 172.21.15.70:0/1555165312 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65cc0039a0 0x7f65e015a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 -- 172.21.15.70:0/1555165312 >> 172.21.15.70:0/1555165312 conn(0x7f65e006a5d0 msgr2=0x7f65e00b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.383+0000 7f65e7e5d640 1 -- 172.21.15.70:0/1555165312 shutdown_connections 2023-10-08T07:13:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.384+0000 7f65e7e5d640 1 -- 172.21.15.70:0/1555165312 wait complete. 2023-10-08T07:13:51.406 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T07:13:51.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-10-08T07:13:51.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.629+0000 7f7cbbd90640 1 Processor -- start 2023-10-08T07:13:51.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.630+0000 7f7cbbd90640 1 -- start start 2023-10-08T07:13:51.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.631+0000 7f7cbbd90640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7cb415cd70 0x7f7cb41612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:51.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.631+0000 7f7cbbd90640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40b8c30 con 0x7f7cb40b4e10 2023-10-08T07:13:51.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.631+0000 7f7cbbd90640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40cdd80 con 0x7f7cb40cd4e0 2023-10-08T07:13:51.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.631+0000 7f7cbbd90640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40ce060 con 0x7f7cb415cd70 2023-10-08T07:13:51.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.631+0000 7f7cb9b2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7cb415cd70 0x7f7cb41612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:51.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.632+0000 7f7cb9b2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7cb415cd70 0x7f7cb41612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54564/0 (socket says 172.21.15.70:54564) 2023-10-08T07:13:51.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.632+0000 7f7cb9b2c640 1 -- 172.21.15.70:0/3449700384 learned_addr learned my addr 172.21.15.70:0/3449700384 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:51.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.632+0000 7f7cb9b2c640 1 -- 172.21.15.70:0/3449700384 >> v1:172.21.15.181:6789/0 conn(0x7f7cb40cd4e0 legacy=0x7f7cb40b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:51.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.632+0000 7f7cb9b2c640 1 -- 172.21.15.70:0/3449700384 >> v1:172.21.15.70:6789/0 conn(0x7f7cb40b4e10 legacy=0x7f7cb40d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:51.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.632+0000 7f7cb9b2c640 1 -- 172.21.15.70:0/3449700384 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40ce340 con 0x7f7cb415cd70 2023-10-08T07:13:51.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.633+0000 7f7cb9b2c640 1 --2- 172.21.15.70:0/3449700384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 0x7f7cb41612f0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009870 tx=0x7f7cb002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e50ec2dcc40821b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:51.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.633+0000 7f7cb8b2a640 1 -- 172.21.15.70:0/3449700384 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0030860 con 0x7f7cb415cd70 2023-10-08T07:13:51.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.634+0000 7f7cb8b2a640 1 -- 172.21.15.70:0/3449700384 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb00309c0 con 0x7f7cb415cd70 2023-10-08T07:13:51.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.634+0000 7f7cb8b2a640 1 -- 172.21.15.70:0/3449700384 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0030d50 con 0x7f7cb415cd70 2023-10-08T07:13:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.634+0000 7f7cbbd90640 1 -- 172.21.15.70:0/3449700384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 msgr2=0x7f7cb41612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.634+0000 7f7cbbd90640 1 --2- 172.21.15.70:0/3449700384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 0x7f7cb41612f0 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009870 tx=0x7f7cb002b9a0 comp rx=0 tx=0).stop 2023-10-08T07:13:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.634+0000 7f7cbbd90640 1 -- 172.21.15.70:0/3449700384 shutdown_connections 2023-10-08T07:13:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.634+0000 7f7cbbd90640 1 --2- 172.21.15.70:0/3449700384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 0x7f7cb41612f0 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.634+0000 7f7cbbd90640 1 -- 172.21.15.70:0/3449700384 >> 172.21.15.70:0/3449700384 conn(0x7f7cb40cf700 msgr2=0x7f7cb40cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:51.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.634+0000 7f7cbbd90640 1 -- 172.21.15.70:0/3449700384 shutdown_connections 2023-10-08T07:13:51.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.634+0000 7f7cbbd90640 1 -- 172.21.15.70:0/3449700384 wait complete. 2023-10-08T07:13:51.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.636+0000 7f7cbbd90640 1 Processor -- start 2023-10-08T07:13:51.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.636+0000 7f7cbbd90640 1 -- start start 2023-10-08T07:13:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.637+0000 7f7cbbd90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cb40b4e10 0x7f7cb415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.637+0000 7f7cbbd90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cb40cd4e0 0x7f7cb40c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.637+0000 7f7cbbd90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 0x7f7cb40d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.637+0000 7f7cbbd90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415e7f0 con 0x7f7cb415cd70 2023-10-08T07:13:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.637+0000 7f7cbbd90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415d750 con 0x7f7cb40cd4e0 2023-10-08T07:13:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.637+0000 7f7cbbd90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415da30 con 0x7f7cb40b4e10 2023-10-08T07:13:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.637+0000 7f7cba32d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 0x7f7cb40d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:51.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.637+0000 7f7cba32d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 0x7f7cb40d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54566/0 (socket says 172.21.15.70:54566) 2023-10-08T07:13:51.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.637+0000 7f7cba32d640 1 -- 172.21.15.70:0/1706326125 learned_addr learned my addr 172.21.15.70:0/1706326125 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7cb932b640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cb40cd4e0 0x7f7cb40c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7cb9b2c640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cb40b4e10 0x7f7cb415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7cba32d640 1 -- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cb40b4e10 msgr2=0x7f7cb415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7cba32d640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cb40b4e10 0x7f7cb415b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7cba32d640 1 -- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cb40cd4e0 msgr2=0x7f7cb40c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7cba32d640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cb40cd4e0 0x7f7cb40c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7cba32d640 1 -- 172.21.15.70:0/1706326125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00094d0 con 0x7f7cb415cd70 2023-10-08T07:13:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7cb932b640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cb40cd4e0 0x7f7cb40c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7cba32d640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 0x7f7cb40d1b60 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00c730 tx=0x7f7cac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:51.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7caa7fc640 1 -- 172.21.15.70:0/1706326125 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac003b70 con 0x7f7cb415cd70 2023-10-08T07:13:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7caa7fc640 1 -- 172.21.15.70:0/1706326125 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac003cf0 con 0x7f7cb415cd70 2023-10-08T07:13:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7caa7fc640 1 -- 172.21.15.70:0/1706326125 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac003ea0 con 0x7f7cb415cd70 2023-10-08T07:13:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.638+0000 7f7cbbd90640 1 -- 172.21.15.70:0/1706326125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb415dd10 con 0x7f7cb415cd70 2023-10-08T07:13:51.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.639+0000 7f7cbbd90640 1 -- 172.21.15.70:0/1706326125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb415e2d0 con 0x7f7cb415cd70 2023-10-08T07:13:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.640+0000 7f7cbbd90640 1 -- 172.21.15.70:0/1706326125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb404e160 con 0x7f7cb415cd70 2023-10-08T07:13:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.641+0000 7f7caa7fc640 1 -- 172.21.15.70:0/1706326125 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7cac005310 con 0x7f7cb415cd70 2023-10-08T07:13:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.642+0000 7f7caa7fc640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7c9806af90 0x7f7c9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:51.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.642+0000 7f7caa7fc640 1 -- 172.21.15.70:0/1706326125 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f7cac089a80 con 0x7f7cb415cd70 2023-10-08T07:13:51.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.642+0000 7f7cb9b2c640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7c9806af90 0x7f7c9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.644+0000 7f7cb9b2c640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7c9806af90 0x7f7c9806d450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0036000 tx=0x7f7cb003b040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:51.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.646+0000 7f7caa7fc640 1 -- 172.21.15.70:0/1706326125 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7cac0544f0 con 0x7f7cb415cd70 2023-10-08T07:13:51.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:51.875+0000 7f7cbbd90640 1 -- 172.21.15.70:0/1706326125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7cb40b2650 con 0x7f7cb415cd70 2023-10-08T07:13:52.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.382+0000 7f7caa7fc640 1 -- 172.21.15.70:0/1706326125 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v114) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7cac057b10 con 0x7f7cb415cd70 2023-10-08T07:13:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 -- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7c9806af90 msgr2=0x7f7c9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7c9806af90 0x7f7c9806d450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0036000 tx=0x7f7cb003b040 comp rx=0 tx=0).stop 2023-10-08T07:13:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 -- 172.21.15.70:0/1706326125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 msgr2=0x7f7cb40d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 0x7f7cb40d1b60 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00c730 tx=0x7f7cac00cbf0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 -- 172.21.15.70:0/1706326125 shutdown_connections 2023-10-08T07:13:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7c9806af90 0x7f7c9806d450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cb415cd70 0x7f7cb40d1b60 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7cb40cd4e0 0x7f7cb40c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 --2- 172.21.15.70:0/1706326125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7cb40b4e10 0x7f7cb415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 -- 172.21.15.70:0/1706326125 >> 172.21.15.70:0/1706326125 conn(0x7f7cb40cf700 msgr2=0x7f7cb40b6830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 -- 172.21.15.70:0/1706326125 shutdown_connections 2023-10-08T07:13:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.386+0000 7f7cbbd90640 1 -- 172.21.15.70:0/1706326125 wait complete. 2023-10-08T07:13:52.409 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-10-08T07:13:52.410 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:13:52.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.619+0000 7f2793355640 1 Processor -- start 2023-10-08T07:13:52.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.620+0000 7f2793355640 1 -- start start 2023-10-08T07:13:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.621+0000 7f2793355640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f278c06b520 0x7f278c06ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.621+0000 7f2793355640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f278c0af8b0 0x7f278c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.621+0000 7f2793355640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f278c1608e0 0x7f278c160d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.621+0000 7f2793355640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c06b3a0 con 0x7f278c0af8b0 2023-10-08T07:13:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.621+0000 7f2793355640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c0b0150 con 0x7f278c06b520 2023-10-08T07:13:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.621+0000 7f2793355640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0b0340 con 0x7f278c1608e0 2023-10-08T07:13:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.621+0000 7f27908f0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f278c0af8b0 0x7f278c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.622+0000 7f27908f0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f278c0af8b0 0x7f278c06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54582/0 (socket says 172.21.15.70:54582) 2023-10-08T07:13:52.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.622+0000 7f27908f0640 1 -- 172.21.15.70:0/3569865164 learned_addr learned my addr 172.21.15.70:0/3569865164 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:52.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.622+0000 7f27918f2640 1 --2- 172.21.15.70:0/3569865164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f278c1608e0 0x7f278c160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:52.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.622+0000 7f27910f1640 1 --2- 172.21.15.70:0/3569865164 >> v2:172.21.15.181:3300/0 conn(0x7f278c06b520 0x7f278c06ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.622+0000 7f27908f0640 1 -- 172.21.15.70:0/3569865164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f278c1608e0 msgr2=0x7f278c160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.622+0000 7f27908f0640 1 --2- 172.21.15.70:0/3569865164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f278c1608e0 0x7f278c160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.622+0000 7f27908f0640 1 -- 172.21.15.70:0/3569865164 >> v2:172.21.15.181:3300/0 conn(0x7f278c06b520 msgr2=0x7f278c06ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.622+0000 7f27908f0640 1 --2- 172.21.15.70:0/3569865164 >> v2:172.21.15.181:3300/0 conn(0x7f278c06b520 0x7f278c06ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.622+0000 7f27908f0640 1 -- 172.21.15.70:0/3569865164 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c06be40 con 0x7f278c0af8b0 2023-10-08T07:13:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.623+0000 7f27908f0640 1 --2- 172.21.15.70:0/3569865164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 0x7f278c06b900 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f278c06e020 tx=0x7f277c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a804d4a30a3799a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.623+0000 7f2783fff640 1 -- 172.21.15.70:0/3569865164 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c0317e0 con 0x7f278c0af8b0 2023-10-08T07:13:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.623+0000 7f2783fff640 1 -- 172.21.15.70:0/3569865164 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c031940 con 0x7f278c0af8b0 2023-10-08T07:13:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.623+0000 7f2783fff640 1 -- 172.21.15.70:0/3569865164 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c031cd0 con 0x7f278c0af8b0 2023-10-08T07:13:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.624+0000 7f2793355640 1 -- 172.21.15.70:0/3569865164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 msgr2=0x7f278c06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.624+0000 7f2793355640 1 --2- 172.21.15.70:0/3569865164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 0x7f278c06b900 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f278c06e020 tx=0x7f277c02b880 comp rx=0 tx=0).stop 2023-10-08T07:13:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.624+0000 7f2793355640 1 -- 172.21.15.70:0/3569865164 shutdown_connections 2023-10-08T07:13:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.624+0000 7f2793355640 1 --2- 172.21.15.70:0/3569865164 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f278c1608e0 0x7f278c160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.624+0000 7f2793355640 1 --2- 172.21.15.70:0/3569865164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 0x7f278c06b900 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.624+0000 7f2793355640 1 --2- 172.21.15.70:0/3569865164 >> v2:172.21.15.181:3300/0 conn(0x7f278c06b520 0x7f278c06ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.624+0000 7f2793355640 1 -- 172.21.15.70:0/3569865164 >> 172.21.15.70:0/3569865164 conn(0x7f278c0d09e0 msgr2=0x7f278c0d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.624+0000 7f2793355640 1 -- 172.21.15.70:0/3569865164 shutdown_connections 2023-10-08T07:13:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.624+0000 7f2793355640 1 -- 172.21.15.70:0/3569865164 wait complete. 2023-10-08T07:13:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.626+0000 7f2793355640 1 Processor -- start 2023-10-08T07:13:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.626+0000 7f2793355640 1 -- start start 2023-10-08T07:13:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.627+0000 7f2793355640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f278c06b520 0x7f278c0c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.627+0000 7f2793355640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 0x7f278c0c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.627+0000 7f2793355640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f278c1608e0 0x7f278c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.627+0000 7f2793355640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c15d800 con 0x7f278c0af8b0 2023-10-08T07:13:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.627+0000 7f2793355640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0c4fd0 con 0x7f278c1608e0 2023-10-08T07:13:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.627+0000 7f2793355640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0c52b0 con 0x7f278c06b520 2023-10-08T07:13:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.627+0000 7f27908f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 0x7f278c0c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.628+0000 7f27908f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 0x7f278c0c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54598/0 (socket says 172.21.15.70:54598) 2023-10-08T07:13:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.628+0000 7f27908f0640 1 -- 172.21.15.70:0/221116296 learned_addr learned my addr 172.21.15.70:0/221116296 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.628+0000 7f27910f1640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f278c06b520 0x7f278c0c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.628+0000 7f27918f2640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f278c1608e0 0x7f278c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.628+0000 7f27908f0640 1 -- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f278c06b520 msgr2=0x7f278c0c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.628+0000 7f27908f0640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f278c06b520 0x7f278c0c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.628+0000 7f27908f0640 1 -- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f278c1608e0 msgr2=0x7f278c1579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.628+0000 7f27908f0640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f278c1608e0 0x7f278c1579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.628+0000 7f27908f0640 1 -- 172.21.15.70:0/221116296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c0094d0 con 0x7f278c0af8b0 2023-10-08T07:13:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.628+0000 7f27908f0640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 0x7f278c0c4510 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f277c02b8c0 tx=0x7f277c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.629+0000 7f2781ffb640 1 -- 172.21.15.70:0/221116296 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c032880 con 0x7f278c0af8b0 2023-10-08T07:13:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.629+0000 7f2781ffb640 1 -- 172.21.15.70:0/221116296 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c032a20 con 0x7f278c0af8b0 2023-10-08T07:13:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.629+0000 7f2781ffb640 1 -- 172.21.15.70:0/221116296 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c032dd0 con 0x7f278c0af8b0 2023-10-08T07:13:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.629+0000 7f2793355640 1 -- 172.21.15.70:0/221116296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c0b36a0 con 0x7f278c0af8b0 2023-10-08T07:13:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.629+0000 7f2793355640 1 -- 172.21.15.70:0/221116296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c0cdf20 con 0x7f278c0af8b0 2023-10-08T07:13:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.630+0000 7f2793355640 1 -- 172.21.15.70:0/221116296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c0b3bc0 con 0x7f278c0af8b0 2023-10-08T07:13:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.631+0000 7f2781ffb640 1 -- 172.21.15.70:0/221116296 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f277c04e450 con 0x7f278c0af8b0 2023-10-08T07:13:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.632+0000 7f2781ffb640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f277006af90 0x7f277006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.632+0000 7f2781ffb640 1 -- 172.21.15.70:0/221116296 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f277c0abfb0 con 0x7f278c0af8b0 2023-10-08T07:13:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.632+0000 7f27910f1640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f277006af90 0x7f277006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.634+0000 7f27910f1640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f277006af90 0x7f277006d450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f278800b970 tx=0x7f2788008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.636+0000 7f2781ffb640 1 -- 172.21.15.70:0/221116296 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f278c0b3bc0 con 0x7f278c0af8b0 2023-10-08T07:13:52.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.862+0000 7f2793355640 1 -- 172.21.15.70:0/221116296 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f278c0b3bc0 con 0x7f278c0af8b0 2023-10-08T07:13:52.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.863+0000 7f2781ffb640 1 -- 172.21.15.70:0/221116296 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f278c0b3bc0 con 0x7f278c0af8b0 2023-10-08T07:13:52.865 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:52.865 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":114,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:34.170119+0000","modified":"2023-10-08T07:13:37.222153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11097":{"gid":11097,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/180995576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":180995576},{"type":"v1","addr":"172.21.15.70:6834","nonce":180995576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:48.829441+0000","modified":"2023-10-08T07:13:51.877323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8416":{"gid":8416,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-08T07:13:52.865 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 114 2023-10-08T07:13:52.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 -- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f277006af90 msgr2=0x7f277006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f277006af90 0x7f277006d450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f278800b970 tx=0x7f2788008040 comp rx=0 tx=0).stop 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 -- 172.21.15.70:0/221116296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 msgr2=0x7f278c0c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 0x7f278c0c4510 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7f277c02b8c0 tx=0x7f277c0046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 -- 172.21.15.70:0/221116296 shutdown_connections 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f278c1608e0 0x7f278c1579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f277006af90 0x7f277006d450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278c0af8b0 0x7f278c0c4510 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 --2- 172.21.15.70:0/221116296 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f278c06b520 0x7f278c0c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 -- 172.21.15.70:0/221116296 >> 172.21.15.70:0/221116296 conn(0x7f278c0d09e0 msgr2=0x7f278c160cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 -- 172.21.15.70:0/221116296 shutdown_connections 2023-10-08T07:13:52.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:52.866+0000 7f2793355640 1 -- 172.21.15.70:0/221116296 wait complete. 2023-10-08T07:13:52.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:13:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.101+0000 7f2fb4075640 1 Processor -- start 2023-10-08T07:13:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.101+0000 7f2fb4075640 1 -- start start 2023-10-08T07:13:53.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.103+0000 7f2fb4075640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fac0cb580 0x7f2fac165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.103+0000 7f2fb4075640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac15b2b0 con 0x7f2fac0c90b0 2023-10-08T07:13:53.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.103+0000 7f2fb4075640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac15b590 con 0x7f2fac15ab40 2023-10-08T07:13:53.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.103+0000 7f2fb4075640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac15bf30 con 0x7f2fac0cb580 2023-10-08T07:13:53.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.103+0000 7f2fb1610640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2fac0c90b0 0x7f2fac0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48846/0 (socket says 172.21.15.70:48846) 2023-10-08T07:13:53.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.103+0000 7f2fb1610640 1 -- 172.21.15.70:0/1625809570 learned_addr learned my addr 172.21.15.70:0/1625809570 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:53.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.103+0000 7f2fb2612640 1 --2- 172.21.15.70:0/1625809570 >> v2:172.21.15.70:3300/0 conn(0x7f2fac0cb580 0x7f2fac165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:53.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.104+0000 7f2fb2612640 1 -- 172.21.15.70:0/1625809570 >> v1:172.21.15.181:6789/0 conn(0x7f2fac15ab40 legacy=0x7f2fac0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:53.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.104+0000 7f2fb2612640 1 -- 172.21.15.70:0/1625809570 >> v1:172.21.15.70:6789/0 conn(0x7f2fac0c90b0 legacy=0x7f2fac0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.104+0000 7f2fb2612640 1 -- 172.21.15.70:0/1625809570 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac15b870 con 0x7f2fac0cb580 2023-10-08T07:13:53.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.104+0000 7f2fb0e0f640 1 -- 172.21.15.70:0/1625809570 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982136326 0 0) 0x7f2fac15b2b0 con 0x7f2fac0c90b0 2023-10-08T07:13:53.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.105+0000 7f2fb2612640 1 --2- 172.21.15.70:0/1625809570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac0cb580 0x7f2fac165b20 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40097a0 tx=0x7f2fa402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5430579892cdb889 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:53.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.105+0000 7f2fb0e0f640 1 -- 172.21.15.70:0/1625809570 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4031830 con 0x7f2fac0cb580 2023-10-08T07:13:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.105+0000 7f2fb0e0f640 1 -- 172.21.15.70:0/1625809570 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa4031990 con 0x7f2fac0cb580 2023-10-08T07:13:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.105+0000 7f2fb0e0f640 1 -- 172.21.15.70:0/1625809570 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4031d20 con 0x7f2fac0cb580 2023-10-08T07:13:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.106+0000 7f2fb4075640 1 -- 172.21.15.70:0/1625809570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac0cb580 msgr2=0x7f2fac165b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.106+0000 7f2fb4075640 1 --2- 172.21.15.70:0/1625809570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac0cb580 0x7f2fac165b20 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40097a0 tx=0x7f2fa402b8d0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.106+0000 7f2fb4075640 1 -- 172.21.15.70:0/1625809570 shutdown_connections 2023-10-08T07:13:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.106+0000 7f2fb4075640 1 --2- 172.21.15.70:0/1625809570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac0cb580 0x7f2fac165b20 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.106+0000 7f2fb4075640 1 -- 172.21.15.70:0/1625809570 >> 172.21.15.70:0/1625809570 conn(0x7f2fac06a380 msgr2=0x7f2fac0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.106+0000 7f2fb4075640 1 -- 172.21.15.70:0/1625809570 shutdown_connections 2023-10-08T07:13:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.106+0000 7f2fb4075640 1 -- 172.21.15.70:0/1625809570 wait complete. 2023-10-08T07:13:53.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.108+0000 7f2fb4075640 1 Processor -- start 2023-10-08T07:13:53.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.108+0000 7f2fb4075640 1 -- start start 2023-10-08T07:13:53.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.109+0000 7f2fb4075640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fac0c90b0 0x7f2fac148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.109+0000 7f2fb4075640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fac0cb580 0x7f2fac158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb4075640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac15ab40 0x7f2fac15d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb4075640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac15f7a0 con 0x7f2fac15ab40 2023-10-08T07:13:53.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb4075640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac1597b0 con 0x7f2fac0c90b0 2023-10-08T07:13:53.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb4075640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac158b30 con 0x7f2fac0cb580 2023-10-08T07:13:53.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb2612640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac15ab40 0x7f2fac15d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:53.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb2612640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac15ab40 0x7f2fac15d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54628/0 (socket says 172.21.15.70:54628) 2023-10-08T07:13:53.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb2612640 1 -- 172.21.15.70:0/2084268888 learned_addr learned my addr 172.21.15.70:0/2084268888 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:53.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb2612640 1 -- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fac0cb580 msgr2=0x7f2fac158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb2612640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fac0cb580 0x7f2fac158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb2612640 1 -- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fac0c90b0 msgr2=0x7f2fac148010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb2612640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fac0c90b0 0x7f2fac148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.110+0000 7f2fb2612640 1 -- 172.21.15.70:0/2084268888 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa4009400 con 0x7f2fac15ab40 2023-10-08T07:13:53.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.111+0000 7f2fb2612640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac15ab40 0x7f2fac15d390 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40098c0 tx=0x7f2fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:53.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.111+0000 7f2faa7fc640 1 -- 172.21.15.70:0/2084268888 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa403b9f0 con 0x7f2fac15ab40 2023-10-08T07:13:53.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.111+0000 7f2faa7fc640 1 -- 172.21.15.70:0/2084268888 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa403bb50 con 0x7f2fac15ab40 2023-10-08T07:13:53.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.111+0000 7f2fb4075640 1 -- 172.21.15.70:0/2084268888 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac158e10 con 0x7f2fac15ab40 2023-10-08T07:13:53.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.111+0000 7f2faa7fc640 1 -- 172.21.15.70:0/2084268888 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40453f0 con 0x7f2fac15ab40 2023-10-08T07:13:53.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.111+0000 7f2fb4075640 1 -- 172.21.15.70:0/2084268888 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac1593d0 con 0x7f2fac15ab40 2023-10-08T07:13:53.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.112+0000 7f2fb4075640 1 -- 172.21.15.70:0/2084268888 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac067490 con 0x7f2fac15ab40 2023-10-08T07:13:53.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.113+0000 7f2faa7fc640 1 -- 172.21.15.70:0/2084268888 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2fa4045550 con 0x7f2fac15ab40 2023-10-08T07:13:53.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.114+0000 7f2faa7fc640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2f9806af90 0x7f2f9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.114+0000 7f2faa7fc640 1 -- 172.21.15.70:0/2084268888 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f2fa40ac3e0 con 0x7f2fac15ab40 2023-10-08T07:13:53.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.115+0000 7f2fb1e11640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2f9806af90 0x7f2f9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:53.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.116+0000 7f2fb1e11640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2f9806af90 0x7f2f9806d450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00ab90 tx=0x7f2f9c009230 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:53.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.118+0000 7f2faa7fc640 1 -- 172.21.15.70:0/2084268888 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fa4076d40 con 0x7f2fac15ab40 2023-10-08T07:13:53.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.309+0000 7f2fb4075640 1 -- 172.21.15.70:0/2084268888 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fac0cb330 con 0x7f2fac15ab40 2023-10-08T07:13:53.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.311+0000 7f2faa7fc640 1 -- 172.21.15.70:0/2084268888 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v238) v1 ==== 74+0+20614 (secure 0 0 0) 0x7f2fa407a360 con 0x7f2fac15ab40 2023-10-08T07:13:53.314 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:53.315 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":238,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:13:50.272410+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:13:47.007437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:13:47.838023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:13:53.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 -- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2f9806af90 msgr2=0x7f2f9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2f9806af90 0x7f2f9806d450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00ab90 tx=0x7f2f9c009230 comp rx=0 tx=0).stop 2023-10-08T07:13:53.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 -- 172.21.15.70:0/2084268888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac15ab40 msgr2=0x7f2fac15d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac15ab40 0x7f2fac15d390 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40098c0 tx=0x7f2fa40046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 -- 172.21.15.70:0/2084268888 shutdown_connections 2023-10-08T07:13:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2f9806af90 0x7f2f9806d450 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fac15ab40 0x7f2fac15d390 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fac0cb580 0x7f2fac158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 --2- 172.21.15.70:0/2084268888 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fac0c90b0 0x7f2fac148010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 -- 172.21.15.70:0/2084268888 >> 172.21.15.70:0/2084268888 conn(0x7f2fac06a380 msgr2=0x7f2fac070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.314+0000 7f2fb4075640 1 -- 172.21.15.70:0/2084268888 shutdown_connections 2023-10-08T07:13:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.315+0000 7f2fb4075640 1 -- 172.21.15.70:0/2084268888 wait complete. 2023-10-08T07:13:53.335 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T07:13:53.335 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:13:53.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.550+0000 7f0623fff640 1 Processor -- start 2023-10-08T07:13:53.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.551+0000 7f0623fff640 1 -- start start 2023-10-08T07:13:53.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.552+0000 7f0623fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06240c9570 0x7f06240c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.552+0000 7f0623fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06240718a0 con 0x7f062415cd70 2023-10-08T07:13:53.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.552+0000 7f0623fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062406e000 con 0x7f0624071450 2023-10-08T07:13:53.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.552+0000 7f0623fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062415d150 con 0x7f06240c9570 2023-10-08T07:13:53.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.552+0000 7f0622ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f062415cd70 0x7f06241607c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47722/0 (socket says 172.21.15.70:47722) 2023-10-08T07:13:53.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.552+0000 7f0622ffd640 1 -- 172.21.15.70:0/848097135 learned_addr learned my addr 172.21.15.70:0/848097135 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:53.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.553+0000 7f06237fe640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06240c9570 0x7f06240c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:53.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.553+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655684633 0 0) 0x7f06240718a0 con 0x7f062415cd70 2023-10-08T07:13:53.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.553+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0610003520 con 0x7f062415cd70 2023-10-08T07:13:53.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.553+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3444255720 0 0) 0x7f0610003520 con 0x7f062415cd70 2023-10-08T07:13:53.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.553+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06240c9570 msgr2=0x7f06240c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:53.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.553+0000 7f0621ffb640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06240c9570 0x7f06240c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.553+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 >> v1:172.21.15.181:6789/0 conn(0x7f0624071450 legacy=0x7f062406e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0624161010 con 0x7f062415cd70 2023-10-08T07:13:53.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0614001670 con 0x7f062415cd70 2023-10-08T07:13:53.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 >> v1:172.21.15.70:6789/0 conn(0x7f062415cd70 legacy=0x7f06241607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0621ffb640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 0x7f0610003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0621ffb640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0610004350 0x7f0610004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0621ffb640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06240c9570 0x7f0610005080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062415d150 con 0x7f06100039a0 2023-10-08T07:13:53.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061000b980 con 0x7f0610004350 2023-10-08T07:13:53.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061000bb40 con 0x7f06240c9570 2023-10-08T07:13:53.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0622ffd640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 0x7f0610003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:53.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0622ffd640 1 -- 172.21.15.70:0/848097135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06240c9570 msgr2=0x7f0610005080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:53.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f06237fe640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06240c9570 0x7f0610005080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:53.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0622ffd640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f06240c9570 0x7f0610005080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0622ffd640 1 -- 172.21.15.70:0/848097135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0610004350 msgr2=0x7f0610004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:53.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0622ffd640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0610004350 0x7f0610004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.554+0000 7f0622ffd640 1 -- 172.21.15.70:0/848097135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610003520 con 0x7f06100039a0 2023-10-08T07:13:53.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.555+0000 7f0622ffd640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 0x7f0610003d80 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f061400c210 tx=0x7f06100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f1300a83500a5a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:53.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.556+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061402cae0 con 0x7f06100039a0 2023-10-08T07:13:53.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.556+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061402cc60 con 0x7f06100039a0 2023-10-08T07:13:53.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.556+0000 7f0621ffb640 1 -- 172.21.15.70:0/848097135 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140058d0 con 0x7f06100039a0 2023-10-08T07:13:53.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.556+0000 7f0623fff640 1 -- 172.21.15.70:0/848097135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 msgr2=0x7f0610003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.556+0000 7f0623fff640 1 --2- 172.21.15.70:0/848097135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 0x7f0610003d80 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7f061400c210 tx=0x7f06100057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.556+0000 7f06237fe640 1 -- 172.21.15.70:0/848097135 reap_dead start 2023-10-08T07:13:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.556+0000 7f0623fff640 1 -- 172.21.15.70:0/848097135 shutdown_connections 2023-10-08T07:13:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.556+0000 7f0623fff640 1 -- 172.21.15.70:0/848097135 >> 172.21.15.70:0/848097135 conn(0x7f06240cb8f0 msgr2=0x7f06240cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.557+0000 7f0623fff640 1 -- 172.21.15.70:0/848097135 shutdown_connections 2023-10-08T07:13:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.557+0000 7f0623fff640 1 -- 172.21.15.70:0/848097135 wait complete. 2023-10-08T07:13:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.558+0000 7f0623fff640 1 Processor -- start 2023-10-08T07:13:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.558+0000 7f0623fff640 1 -- start start 2023-10-08T07:13:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.559+0000 7f0623fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 0x7f06240c3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.559+0000 7f0623fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f062415c240 0x7f06240b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.559+0000 7f0623fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062415c6c0 0x7f06241512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.559+0000 7f0623fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06240b3a30 con 0x7f06100039a0 2023-10-08T07:13:53.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.559+0000 7f0623fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06240b2eb0 con 0x7f062415c6c0 2023-10-08T07:13:53.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.559+0000 7f0623fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06240b3190 con 0x7f062415c240 2023-10-08T07:13:53.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f0622ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 0x7f06240c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:53.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f0622ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 0x7f06240c3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49912/0 (socket says 172.21.15.70:49912) 2023-10-08T07:13:53.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f0622ffd640 1 -- 172.21.15.70:0/833881592 learned_addr learned my addr 172.21.15.70:0/833881592 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:53.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f0622ffd640 1 -- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f062415c240 msgr2=0x7f06240b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f06237fe640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062415c6c0 0x7f06241512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:53.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f06227fc640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f062415c240 0x7f06240b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:53.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f0622ffd640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f062415c240 0x7f06240b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f0622ffd640 1 -- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062415c6c0 msgr2=0x7f06241512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f0622ffd640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062415c6c0 0x7f06241512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f0622ffd640 1 -- 172.21.15.70:0/833881592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06240b3470 con 0x7f06100039a0 2023-10-08T07:13:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.560+0000 7f0622ffd640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 0x7f06240c3380 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f061402c480 tx=0x7f0614001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.561+0000 7f061b7fe640 1 -- 172.21.15.70:0/833881592 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061403c050 con 0x7f06100039a0 2023-10-08T07:13:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.561+0000 7f061b7fe640 1 -- 172.21.15.70:0/833881592 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061403a070 con 0x7f06100039a0 2023-10-08T07:13:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.561+0000 7f061b7fe640 1 -- 172.21.15.70:0/833881592 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614007270 con 0x7f06100039a0 2023-10-08T07:13:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.561+0000 7f0623fff640 1 -- 172.21.15.70:0/833881592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06240b3750 con 0x7f06100039a0 2023-10-08T07:13:53.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.561+0000 7f0623fff640 1 -- 172.21.15.70:0/833881592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06240c38c0 con 0x7f06100039a0 2023-10-08T07:13:53.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.562+0000 7f0623fff640 1 -- 172.21.15.70:0/833881592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062404e160 con 0x7f06100039a0 2023-10-08T07:13:53.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.563+0000 7f061b7fe640 1 -- 172.21.15.70:0/833881592 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0614032070 con 0x7f06100039a0 2023-10-08T07:13:53.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.564+0000 7f061b7fe640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f060406af90 0x7f060406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:53.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.565+0000 7f061b7fe640 1 -- 172.21.15.70:0/833881592 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f06140ab250 con 0x7f06100039a0 2023-10-08T07:13:53.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.565+0000 7f06227fc640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f060406af90 0x7f060406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:53.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.566+0000 7f06227fc640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f060406af90 0x7f060406d450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f06240b3b90 tx=0x7f060c009040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:53.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.569+0000 7f061b7fe640 1 -- 172.21.15.70:0/833881592 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0614075c30 con 0x7f06100039a0 2023-10-08T07:13:53.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.755+0000 7f0623fff640 1 -- 172.21.15.70:0/833881592 --> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06241615d0 con 0x7f060406af90 2023-10-08T07:13:53.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.798+0000 7f061b7fe640 1 -- 172.21.15.70:0/833881592 <== mgr.7328 v2:172.21.15.181:6832/3993370366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+522986 (secure 0 0 0) 0x7f060c001c50 con 0x7f060406af90 2023-10-08T07:13:53.806 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:53.810 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:13:53.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 -- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f060406af90 msgr2=0x7f060406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f060406af90 0x7f060406d450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f06240b3b90 tx=0x7f060c009040 comp rx=0 tx=0).stop 2023-10-08T07:13:53.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 -- 172.21.15.70:0/833881592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 msgr2=0x7f06240c3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:53.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 0x7f06240c3380 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f061402c480 tx=0x7f0614001a00 comp rx=0 tx=0).stop 2023-10-08T07:13:53.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 -- 172.21.15.70:0/833881592 shutdown_connections 2023-10-08T07:13:53.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f062415c6c0 0x7f06241512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f062415c240 0x7f06240b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f060406af90 0x7f060406d450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 --2- 172.21.15.70:0/833881592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06100039a0 0x7f06240c3380 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:53.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 -- 172.21.15.70:0/833881592 >> 172.21.15.70:0/833881592 conn(0x7f06240cb8f0 msgr2=0x7f062406eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:53.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 -- 172.21.15.70:0/833881592 shutdown_connections 2023-10-08T07:13:53.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:53.806+0000 7f0623fff640 1 -- 172.21.15.70:0/833881592 wait complete. 2023-10-08T07:13:53.829 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-10-08T07:13:53.510178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594627,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":644,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2854,"ondisk_log_size":2854,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":275,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":433468,"kb_used_data":24396,"kb_used_omap":12,"kb_used_meta":408947,"kb_avail":754541252,"statfs":{"total":773094113280,"available":772650242048,"internally_reserved":0,"allocated":24981504,"data_stored":13517744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":418762299},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001985"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359204+0000","last_change":"2023-10-08T07:13:49.313140+0000","last_active":"2023-10-08T07:13:50.359204+0000","last_peered":"2023-10-08T07:13:50.359204+0000","last_clean":"2023-10-08T07:13:50.359204+0000","last_became_active":"2023-10-08T07:13:49.312820+0000","last_became_peered":"2023-10-08T07:13:49.312820+0000","last_unstale":"2023-10-08T07:13:50.359204+0000","last_undegraded":"2023-10-08T07:13:50.359204+0000","last_fullsized":"2023-10-08T07:13:50.359204+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:41.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"222'2806","reported_seq":3725,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.350548+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:13:47.350548+0000","last_peered":"2023-10-08T07:13:47.350548+0000","last_clean":"2023-10-08T07:13:47.350548+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:13:47.350548+0000","last_undegraded":"2023-10-08T07:13:47.350548+0000","last_fullsized":"2023-10-08T07:13:47.350548+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2806,"log_dups_size":0,"ondisk_log_size":2806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289208+0000","last_change":"2023-10-08T07:13:33.718816+0000","last_active":"2023-10-08T07:13:49.289208+0000","last_peered":"2023-10-08T07:13:49.289208+0000","last_clean":"2023-10-08T07:13:49.289208+0000","last_became_active":"2023-10-08T07:13:33.718494+0000","last_became_peered":"2023-10-08T07:13:33.718494+0000","last_unstale":"2023-10-08T07:13:49.289208+0000","last_undegraded":"2023-10-08T07:13:49.289208+0000","last_fullsized":"2023-10-08T07:13:49.289208+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:03.021625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344810+0000","last_change":"2023-10-08T07:13:34.304364+0000","last_active":"2023-10-08T07:13:47.344810+0000","last_peered":"2023-10-08T07:13:47.344810+0000","last_clean":"2023-10-08T07:13:47.344810+0000","last_became_active":"2023-10-08T07:13:34.304067+0000","last_became_peered":"2023-10-08T07:13:34.304067+0000","last_unstale":"2023-10-08T07:13:47.344810+0000","last_undegraded":"2023-10-08T07:13:47.344810+0000","last_fullsized":"2023-10-08T07:13:47.344810+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:56.084275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.385321+0000","last_change":"2023-10-08T07:13:48.383423+0000","last_active":"2023-10-08T07:13:48.385321+0000","last_peered":"2023-10-08T07:13:48.385321+0000","last_clean":"2023-10-08T07:13:48.385321+0000","last_became_active":"2023-10-08T07:13:48.383120+0000","last_became_peered":"2023-10-08T07:13:48.383120+0000","last_unstale":"2023-10-08T07:13:48.385321+0000","last_undegraded":"2023-10-08T07:13:48.385321+0000","last_fullsized":"2023-10-08T07:13:48.385321+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:30.632556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.377733+0000","last_change":"2023-10-08T07:13:48.377733+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.377733+0000","last_undegraded":"2023-10-08T07:13:48.377733+0000","last_fullsized":"2023-10-08T07:13:48.377733+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.278218+0000","last_change":"2023-10-08T07:13:33.735924+0000","last_active":"2023-10-08T07:13:49.278218+0000","last_peered":"2023-10-08T07:13:49.278218+0000","last_clean":"2023-10-08T07:13:49.278218+0000","last_became_active":"2023-10-08T07:13:33.735688+0000","last_became_peered":"2023-10-08T07:13:33.735688+0000","last_unstale":"2023-10-08T07:13:49.278218+0000","last_undegraded":"2023-10-08T07:13:49.278218+0000","last_fullsized":"2023-10-08T07:13:49.278218+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348742+0000","last_change":"2023-10-08T07:13:34.308734+0000","last_active":"2023-10-08T07:13:47.348742+0000","last_peered":"2023-10-08T07:13:47.348742+0000","last_clean":"2023-10-08T07:13:47.348742+0000","last_became_active":"2023-10-08T07:13:34.308195+0000","last_became_peered":"2023-10-08T07:13:34.308195+0000","last_unstale":"2023-10-08T07:13:47.348742+0000","last_undegraded":"2023-10-08T07:13:47.348742+0000","last_fullsized":"2023-10-08T07:13:47.348742+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.387859+0000","last_change":"2023-10-08T07:13:48.387261+0000","last_active":"2023-10-08T07:13:48.387859+0000","last_peered":"2023-10-08T07:13:48.387859+0000","last_clean":"2023-10-08T07:13:48.387859+0000","last_became_active":"2023-10-08T07:13:48.387022+0000","last_became_peered":"2023-10-08T07:13:48.387022+0000","last_unstale":"2023-10-08T07:13:48.387859+0000","last_undegraded":"2023-10-08T07:13:48.387859+0000","last_fullsized":"2023-10-08T07:13:48.387859+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:48.064204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448409+0000","last_change":"2023-10-08T07:13:49.303091+0000","last_active":"2023-10-08T07:13:50.448409+0000","last_peered":"2023-10-08T07:13:50.448409+0000","last_clean":"2023-10-08T07:13:50.448409+0000","last_became_active":"2023-10-08T07:13:49.302761+0000","last_became_peered":"2023-10-08T07:13:49.302761+0000","last_unstale":"2023-10-08T07:13:50.448409+0000","last_undegraded":"2023-10-08T07:13:50.448409+0000","last_fullsized":"2023-10-08T07:13:50.448409+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:54.844015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344232+0000","last_change":"2023-10-08T07:13:33.741115+0000","last_active":"2023-10-08T07:13:47.344232+0000","last_peered":"2023-10-08T07:13:47.344232+0000","last_clean":"2023-10-08T07:13:47.344232+0000","last_became_active":"2023-10-08T07:13:33.740798+0000","last_became_peered":"2023-10-08T07:13:33.740798+0000","last_unstale":"2023-10-08T07:13:47.344232+0000","last_undegraded":"2023-10-08T07:13:47.344232+0000","last_fullsized":"2023-10-08T07:13:47.344232+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:36.728485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.352692+0000","last_change":"2023-10-08T07:13:34.303009+0000","last_active":"2023-10-08T07:13:47.352692+0000","last_peered":"2023-10-08T07:13:47.352692+0000","last_clean":"2023-10-08T07:13:47.352692+0000","last_became_active":"2023-10-08T07:13:34.302682+0000","last_became_peered":"2023-10-08T07:13:34.302682+0000","last_unstale":"2023-10-08T07:13:47.352692+0000","last_undegraded":"2023-10-08T07:13:47.352692+0000","last_fullsized":"2023-10-08T07:13:47.352692+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:47:40.976539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.393834+0000","last_change":"2023-10-08T07:13:48.388490+0000","last_active":"2023-10-08T07:13:48.393834+0000","last_peered":"2023-10-08T07:13:48.393834+0000","last_clean":"2023-10-08T07:13:48.393834+0000","last_became_active":"2023-10-08T07:13:48.388178+0000","last_became_peered":"2023-10-08T07:13:48.388178+0000","last_unstale":"2023-10-08T07:13:48.393834+0000","last_undegraded":"2023-10-08T07:13:48.393834+0000","last_fullsized":"2023-10-08T07:13:48.393834+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:18.667320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278865+0000","last_change":"2023-10-08T07:13:49.305322+0000","last_active":"2023-10-08T07:13:50.278865+0000","last_peered":"2023-10-08T07:13:50.278865+0000","last_clean":"2023-10-08T07:13:50.278865+0000","last_became_active":"2023-10-08T07:13:49.305047+0000","last_became_peered":"2023-10-08T07:13:49.305047+0000","last_unstale":"2023-10-08T07:13:50.278865+0000","last_undegraded":"2023-10-08T07:13:50.278865+0000","last_fullsized":"2023-10-08T07:13:50.278865+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:20.461769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289335+0000","last_change":"2023-10-08T07:13:33.738792+0000","last_active":"2023-10-08T07:13:49.289335+0000","last_peered":"2023-10-08T07:13:49.289335+0000","last_clean":"2023-10-08T07:13:49.289335+0000","last_became_active":"2023-10-08T07:13:33.738430+0000","last_became_peered":"2023-10-08T07:13:33.738430+0000","last_unstale":"2023-10-08T07:13:49.289335+0000","last_undegraded":"2023-10-08T07:13:49.289335+0000","last_fullsized":"2023-10-08T07:13:49.289335+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:24:39.688018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.353511+0000","last_change":"2023-10-08T07:13:34.305875+0000","last_active":"2023-10-08T07:13:47.353511+0000","last_peered":"2023-10-08T07:13:47.353511+0000","last_clean":"2023-10-08T07:13:47.353511+0000","last_became_active":"2023-10-08T07:13:34.302605+0000","last_became_peered":"2023-10-08T07:13:34.302605+0000","last_unstale":"2023-10-08T07:13:47.353511+0000","last_undegraded":"2023-10-08T07:13:47.353511+0000","last_fullsized":"2023-10-08T07:13:47.353511+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:22.929719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396356+0000","last_change":"2023-10-08T07:13:48.391812+0000","last_active":"2023-10-08T07:13:48.396356+0000","last_peered":"2023-10-08T07:13:48.396356+0000","last_clean":"2023-10-08T07:13:48.396356+0000","last_became_active":"2023-10-08T07:13:48.391532+0000","last_became_peered":"2023-10-08T07:13:48.391532+0000","last_unstale":"2023-10-08T07:13:48.396356+0000","last_undegraded":"2023-10-08T07:13:48.396356+0000","last_fullsized":"2023-10-08T07:13:48.396356+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:39:10.917417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.372366+0000","last_change":"2023-10-08T07:13:48.372366+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.372366+0000","last_undegraded":"2023-10-08T07:13:48.372366+0000","last_fullsized":"2023-10-08T07:13:48.372366+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.361372+0000","last_change":"2023-10-08T07:13:33.734220+0000","last_active":"2023-10-08T07:13:47.361372+0000","last_peered":"2023-10-08T07:13:47.361372+0000","last_clean":"2023-10-08T07:13:47.361372+0000","last_became_active":"2023-10-08T07:13:33.733905+0000","last_became_peered":"2023-10-08T07:13:33.733905+0000","last_unstale":"2023-10-08T07:13:47.361372+0000","last_undegraded":"2023-10-08T07:13:47.361372+0000","last_fullsized":"2023-10-08T07:13:47.361372+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359420+0000","last_change":"2023-10-08T07:13:34.302227+0000","last_active":"2023-10-08T07:13:47.359420+0000","last_peered":"2023-10-08T07:13:47.359420+0000","last_clean":"2023-10-08T07:13:47.359420+0000","last_became_active":"2023-10-08T07:13:34.300127+0000","last_became_peered":"2023-10-08T07:13:34.300127+0000","last_unstale":"2023-10-08T07:13:47.359420+0000","last_undegraded":"2023-10-08T07:13:47.359420+0000","last_fullsized":"2023-10-08T07:13:47.359420+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:49.729707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.382534+0000","last_change":"2023-10-08T07:13:48.380775+0000","last_active":"2023-10-08T07:13:48.382534+0000","last_peered":"2023-10-08T07:13:48.382534+0000","last_clean":"2023-10-08T07:13:48.382534+0000","last_became_active":"2023-10-08T07:13:48.380501+0000","last_became_peered":"2023-10-08T07:13:48.380501+0000","last_unstale":"2023-10-08T07:13:48.382534+0000","last_undegraded":"2023-10-08T07:13:48.382534+0000","last_fullsized":"2023-10-08T07:13:48.382534+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:07.670172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357767+0000","last_change":"2023-10-08T07:13:49.309072+0000","last_active":"2023-10-08T07:13:50.357767+0000","last_peered":"2023-10-08T07:13:50.357767+0000","last_clean":"2023-10-08T07:13:50.357767+0000","last_became_active":"2023-10-08T07:13:49.308753+0000","last_became_peered":"2023-10-08T07:13:49.308753+0000","last_unstale":"2023-10-08T07:13:50.357767+0000","last_undegraded":"2023-10-08T07:13:50.357767+0000","last_fullsized":"2023-10-08T07:13:50.357767+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:27.798388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344852+0000","last_change":"2023-10-08T07:13:33.733207+0000","last_active":"2023-10-08T07:13:47.344852+0000","last_peered":"2023-10-08T07:13:47.344852+0000","last_clean":"2023-10-08T07:13:47.344852+0000","last_became_active":"2023-10-08T07:13:33.732903+0000","last_became_peered":"2023-10-08T07:13:33.732903+0000","last_unstale":"2023-10-08T07:13:47.344852+0000","last_undegraded":"2023-10-08T07:13:47.344852+0000","last_fullsized":"2023-10-08T07:13:47.344852+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:34.358428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343713+0000","last_change":"2023-10-08T07:13:34.297602+0000","last_active":"2023-10-08T07:13:47.343713+0000","last_peered":"2023-10-08T07:13:47.343713+0000","last_clean":"2023-10-08T07:13:47.343713+0000","last_became_active":"2023-10-08T07:13:34.296820+0000","last_became_peered":"2023-10-08T07:13:34.296820+0000","last_unstale":"2023-10-08T07:13:47.343713+0000","last_undegraded":"2023-10-08T07:13:47.343713+0000","last_fullsized":"2023-10-08T07:13:47.343713+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:07.812354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.380096+0000","last_change":"2023-10-08T07:13:48.374204+0000","last_active":"2023-10-08T07:13:48.380096+0000","last_peered":"2023-10-08T07:13:48.380096+0000","last_clean":"2023-10-08T07:13:48.380096+0000","last_became_active":"2023-10-08T07:13:48.373126+0000","last_became_peered":"2023-10-08T07:13:48.373126+0000","last_unstale":"2023-10-08T07:13:48.380096+0000","last_undegraded":"2023-10-08T07:13:48.380096+0000","last_fullsized":"2023-10-08T07:13:48.380096+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:18.667320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358171+0000","last_change":"2023-10-08T07:13:49.314030+0000","last_active":"2023-10-08T07:13:50.358171+0000","last_peered":"2023-10-08T07:13:50.358171+0000","last_clean":"2023-10-08T07:13:50.358171+0000","last_became_active":"2023-10-08T07:13:49.312711+0000","last_became_peered":"2023-10-08T07:13:49.312711+0000","last_unstale":"2023-10-08T07:13:50.358171+0000","last_undegraded":"2023-10-08T07:13:50.358171+0000","last_fullsized":"2023-10-08T07:13:50.358171+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:39.025998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.356696+0000","last_change":"2023-10-08T07:13:33.737211+0000","last_active":"2023-10-08T07:13:47.356696+0000","last_peered":"2023-10-08T07:13:47.356696+0000","last_clean":"2023-10-08T07:13:47.356696+0000","last_became_active":"2023-10-08T07:13:33.736904+0000","last_became_peered":"2023-10-08T07:13:33.736904+0000","last_unstale":"2023-10-08T07:13:47.356696+0000","last_undegraded":"2023-10-08T07:13:47.356696+0000","last_fullsized":"2023-10-08T07:13:47.356696+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:58.966472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288985+0000","last_change":"2023-10-08T07:13:34.302678+0000","last_active":"2023-10-08T07:13:49.288985+0000","last_peered":"2023-10-08T07:13:49.288985+0000","last_clean":"2023-10-08T07:13:49.288985+0000","last_became_active":"2023-10-08T07:13:34.302406+0000","last_became_peered":"2023-10-08T07:13:34.302406+0000","last_unstale":"2023-10-08T07:13:49.288985+0000","last_undegraded":"2023-10-08T07:13:49.288985+0000","last_fullsized":"2023-10-08T07:13:49.288985+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:48.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396347+0000","last_change":"2023-10-08T07:13:48.389369+0000","last_active":"2023-10-08T07:13:48.396347+0000","last_peered":"2023-10-08T07:13:48.396347+0000","last_clean":"2023-10-08T07:13:48.396347+0000","last_became_active":"2023-10-08T07:13:48.389067+0000","last_became_peered":"2023-10-08T07:13:48.389067+0000","last_unstale":"2023-10-08T07:13:48.396347+0000","last_undegraded":"2023-10-08T07:13:48.396347+0000","last_fullsized":"2023-10-08T07:13:48.396347+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:18.246155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.377624+0000","last_change":"2023-10-08T07:13:48.377624+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.377624+0000","last_undegraded":"2023-10-08T07:13:48.377624+0000","last_fullsized":"2023-10-08T07:13:48.377624+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"234'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359654+0000","last_change":"2023-10-08T07:13:33.742793+0000","last_active":"2023-10-08T07:13:47.359654+0000","last_peered":"2023-10-08T07:13:47.359654+0000","last_clean":"2023-10-08T07:13:47.359654+0000","last_became_active":"2023-10-08T07:13:33.742030+0000","last_became_peered":"2023-10-08T07:13:33.742030+0000","last_unstale":"2023-10-08T07:13:47.359654+0000","last_undegraded":"2023-10-08T07:13:47.359654+0000","last_fullsized":"2023-10-08T07:13:47.359654+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:25:59.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348010+0000","last_change":"2023-10-08T07:13:34.308581+0000","last_active":"2023-10-08T07:13:47.348010+0000","last_peered":"2023-10-08T07:13:47.348010+0000","last_clean":"2023-10-08T07:13:47.348010+0000","last_became_active":"2023-10-08T07:13:34.308210+0000","last_became_peered":"2023-10-08T07:13:34.308210+0000","last_unstale":"2023-10-08T07:13:47.348010+0000","last_undegraded":"2023-10-08T07:13:47.348010+0000","last_fullsized":"2023-10-08T07:13:47.348010+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396466+0000","last_change":"2023-10-08T07:13:48.391100+0000","last_active":"2023-10-08T07:13:48.396466+0000","last_peered":"2023-10-08T07:13:48.396466+0000","last_clean":"2023-10-08T07:13:48.396466+0000","last_became_active":"2023-10-08T07:13:48.390787+0000","last_became_peered":"2023-10-08T07:13:48.390787+0000","last_unstale":"2023-10-08T07:13:48.396466+0000","last_undegraded":"2023-10-08T07:13:48.396466+0000","last_fullsized":"2023-10-08T07:13:48.396466+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:13:47.089135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278157+0000","last_change":"2023-10-08T07:13:49.310297+0000","last_active":"2023-10-08T07:13:50.278157+0000","last_peered":"2023-10-08T07:13:50.278157+0000","last_clean":"2023-10-08T07:13:50.278157+0000","last_became_active":"2023-10-08T07:13:49.310021+0000","last_became_peered":"2023-10-08T07:13:49.310021+0000","last_unstale":"2023-10-08T07:13:50.278157+0000","last_undegraded":"2023-10-08T07:13:50.278157+0000","last_fullsized":"2023-10-08T07:13:50.278157+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:19.685263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292199+0000","last_change":"2023-10-08T07:13:33.739638+0000","last_active":"2023-10-08T07:13:49.292199+0000","last_peered":"2023-10-08T07:13:49.292199+0000","last_clean":"2023-10-08T07:13:49.292199+0000","last_became_active":"2023-10-08T07:13:33.739352+0000","last_became_peered":"2023-10-08T07:13:33.739352+0000","last_unstale":"2023-10-08T07:13:49.292199+0000","last_undegraded":"2023-10-08T07:13:49.292199+0000","last_fullsized":"2023-10-08T07:13:49.292199+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:11.117318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359899+0000","last_change":"2023-10-08T07:13:34.293674+0000","last_active":"2023-10-08T07:13:47.359899+0000","last_peered":"2023-10-08T07:13:47.359899+0000","last_clean":"2023-10-08T07:13:47.359899+0000","last_became_active":"2023-10-08T07:13:34.292983+0000","last_became_peered":"2023-10-08T07:13:34.292983+0000","last_unstale":"2023-10-08T07:13:47.359899+0000","last_undegraded":"2023-10-08T07:13:47.359899+0000","last_fullsized":"2023-10-08T07:13:47.359899+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:20.477902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.389923+0000","last_change":"2023-10-08T07:13:48.388173+0000","last_active":"2023-10-08T07:13:48.389923+0000","last_peered":"2023-10-08T07:13:48.389923+0000","last_clean":"2023-10-08T07:13:48.389923+0000","last_became_active":"2023-10-08T07:13:48.387881+0000","last_became_peered":"2023-10-08T07:13:48.387881+0000","last_unstale":"2023-10-08T07:13:48.389923+0000","last_undegraded":"2023-10-08T07:13:48.389923+0000","last_fullsized":"2023-10-08T07:13:48.389923+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:43.583782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.379456+0000","last_change":"2023-10-08T07:13:48.379456+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.379456+0000","last_undegraded":"2023-10-08T07:13:48.379456+0000","last_fullsized":"2023-10-08T07:13:48.379456+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.342892+0000","last_change":"2023-10-08T07:13:33.739626+0000","last_active":"2023-10-08T07:13:47.342892+0000","last_peered":"2023-10-08T07:13:47.342892+0000","last_clean":"2023-10-08T07:13:47.342892+0000","last_became_active":"2023-10-08T07:13:33.739348+0000","last_became_peered":"2023-10-08T07:13:33.739348+0000","last_unstale":"2023-10-08T07:13:47.342892+0000","last_undegraded":"2023-10-08T07:13:47.342892+0000","last_fullsized":"2023-10-08T07:13:47.342892+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:23.374596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.360313+0000","last_change":"2023-10-08T07:13:34.300038+0000","last_active":"2023-10-08T07:13:47.360313+0000","last_peered":"2023-10-08T07:13:47.360313+0000","last_clean":"2023-10-08T07:13:47.360313+0000","last_became_active":"2023-10-08T07:13:34.299721+0000","last_became_peered":"2023-10-08T07:13:34.299721+0000","last_unstale":"2023-10-08T07:13:47.360313+0000","last_undegraded":"2023-10-08T07:13:47.360313+0000","last_fullsized":"2023-10-08T07:13:47.360313+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:26.317918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.366615+0000","last_change":"2023-10-08T07:13:48.384247+0000","last_active":"2023-10-08T07:13:49.366615+0000","last_peered":"2023-10-08T07:13:49.366615+0000","last_clean":"2023-10-08T07:13:49.366615+0000","last_became_active":"2023-10-08T07:13:48.383959+0000","last_became_peered":"2023-10-08T07:13:48.383959+0000","last_unstale":"2023-10-08T07:13:49.366615+0000","last_undegraded":"2023-10-08T07:13:49.366615+0000","last_fullsized":"2023-10-08T07:13:49.366615+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:13.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448607+0000","last_change":"2023-10-08T07:13:49.319375+0000","last_active":"2023-10-08T07:13:50.448607+0000","last_peered":"2023-10-08T07:13:50.448607+0000","last_clean":"2023-10-08T07:13:50.448607+0000","last_became_active":"2023-10-08T07:13:49.319046+0000","last_became_peered":"2023-10-08T07:13:49.319046+0000","last_unstale":"2023-10-08T07:13:50.448607+0000","last_undegraded":"2023-10-08T07:13:50.448607+0000","last_fullsized":"2023-10-08T07:13:50.448607+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:43.401179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.354842+0000","last_change":"2023-10-08T07:13:33.742955+0000","last_active":"2023-10-08T07:13:47.354842+0000","last_peered":"2023-10-08T07:13:47.354842+0000","last_clean":"2023-10-08T07:13:47.354842+0000","last_became_active":"2023-10-08T07:13:33.742621+0000","last_became_peered":"2023-10-08T07:13:33.742621+0000","last_unstale":"2023-10-08T07:13:47.354842+0000","last_undegraded":"2023-10-08T07:13:47.354842+0000","last_fullsized":"2023-10-08T07:13:47.354842+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:25.311911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291806+0000","last_change":"2023-10-08T07:13:34.294206+0000","last_active":"2023-10-08T07:13:49.291806+0000","last_peered":"2023-10-08T07:13:49.291806+0000","last_clean":"2023-10-08T07:13:49.291806+0000","last_became_active":"2023-10-08T07:13:34.293897+0000","last_became_peered":"2023-10-08T07:13:34.293897+0000","last_unstale":"2023-10-08T07:13:49.291806+0000","last_undegraded":"2023-10-08T07:13:49.291806+0000","last_fullsized":"2023-10-08T07:13:49.291806+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370513+0000","last_change":"2023-10-08T07:13:48.392407+0000","last_active":"2023-10-08T07:13:49.370513+0000","last_peered":"2023-10-08T07:13:49.370513+0000","last_clean":"2023-10-08T07:13:49.370513+0000","last_became_active":"2023-10-08T07:13:48.392147+0000","last_became_peered":"2023-10-08T07:13:48.392147+0000","last_unstale":"2023-10-08T07:13:49.370513+0000","last_undegraded":"2023-10-08T07:13:49.370513+0000","last_fullsized":"2023-10-08T07:13:49.370513+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:54:27.098972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376428+0000","last_change":"2023-10-08T07:13:48.376428+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376428+0000","last_undegraded":"2023-10-08T07:13:48.376428+0000","last_fullsized":"2023-10-08T07:13:48.376428+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295620+0000","last_change":"2023-10-08T07:13:33.726092+0000","last_active":"2023-10-08T07:13:49.295620+0000","last_peered":"2023-10-08T07:13:49.295620+0000","last_clean":"2023-10-08T07:13:49.295620+0000","last_became_active":"2023-10-08T07:13:33.725701+0000","last_became_peered":"2023-10-08T07:13:33.725701+0000","last_unstale":"2023-10-08T07:13:49.295620+0000","last_undegraded":"2023-10-08T07:13:49.295620+0000","last_fullsized":"2023-10-08T07:13:49.295620+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:51.631016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.353306+0000","last_change":"2023-10-08T07:13:34.301017+0000","last_active":"2023-10-08T07:13:47.353306+0000","last_peered":"2023-10-08T07:13:47.353306+0000","last_clean":"2023-10-08T07:13:47.353306+0000","last_became_active":"2023-10-08T07:13:34.300720+0000","last_became_peered":"2023-10-08T07:13:34.300720+0000","last_unstale":"2023-10-08T07:13:47.353306+0000","last_undegraded":"2023-10-08T07:13:47.353306+0000","last_fullsized":"2023-10-08T07:13:47.353306+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:36:05.460057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.395613+0000","last_change":"2023-10-08T07:13:48.389158+0000","last_active":"2023-10-08T07:13:48.395613+0000","last_peered":"2023-10-08T07:13:48.395613+0000","last_clean":"2023-10-08T07:13:48.395613+0000","last_became_active":"2023-10-08T07:13:48.388809+0000","last_became_peered":"2023-10-08T07:13:48.388809+0000","last_unstale":"2023-10-08T07:13:48.395613+0000","last_undegraded":"2023-10-08T07:13:48.395613+0000","last_fullsized":"2023-10-08T07:13:48.395613+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:30.625357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359126+0000","last_change":"2023-10-08T07:13:49.310553+0000","last_active":"2023-10-08T07:13:50.359126+0000","last_peered":"2023-10-08T07:13:50.359126+0000","last_clean":"2023-10-08T07:13:50.359126+0000","last_became_active":"2023-10-08T07:13:49.310222+0000","last_became_peered":"2023-10-08T07:13:49.310222+0000","last_unstale":"2023-10-08T07:13:50.359126+0000","last_undegraded":"2023-10-08T07:13:50.359126+0000","last_fullsized":"2023-10-08T07:13:50.359126+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:50.676315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.283072+0000","last_change":"2023-10-08T07:13:33.726781+0000","last_active":"2023-10-08T07:13:49.283072+0000","last_peered":"2023-10-08T07:13:49.283072+0000","last_clean":"2023-10-08T07:13:49.283072+0000","last_became_active":"2023-10-08T07:13:33.726459+0000","last_became_peered":"2023-10-08T07:13:33.726459+0000","last_unstale":"2023-10-08T07:13:49.283072+0000","last_undegraded":"2023-10-08T07:13:49.283072+0000","last_fullsized":"2023-10-08T07:13:49.283072+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:04.357904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295826+0000","last_change":"2023-10-08T07:13:34.294315+0000","last_active":"2023-10-08T07:13:49.295826+0000","last_peered":"2023-10-08T07:13:49.295826+0000","last_clean":"2023-10-08T07:13:49.295826+0000","last_became_active":"2023-10-08T07:13:34.294065+0000","last_became_peered":"2023-10-08T07:13:34.294065+0000","last_unstale":"2023-10-08T07:13:49.295826+0000","last_undegraded":"2023-10-08T07:13:49.295826+0000","last_fullsized":"2023-10-08T07:13:49.295826+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:37.124047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396254+0000","last_change":"2023-10-08T07:13:48.389005+0000","last_active":"2023-10-08T07:13:48.396254+0000","last_peered":"2023-10-08T07:13:48.396254+0000","last_clean":"2023-10-08T07:13:48.396254+0000","last_became_active":"2023-10-08T07:13:48.388729+0000","last_became_peered":"2023-10-08T07:13:48.388729+0000","last_unstale":"2023-10-08T07:13:48.396254+0000","last_undegraded":"2023-10-08T07:13:48.396254+0000","last_fullsized":"2023-10-08T07:13:48.396254+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:13.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.373866+0000","last_change":"2023-10-08T07:13:48.373866+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.373866+0000","last_undegraded":"2023-10-08T07:13:48.373866+0000","last_fullsized":"2023-10-08T07:13:48.373866+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282248+0000","last_change":"2023-10-08T07:13:33.729567+0000","last_active":"2023-10-08T07:13:49.282248+0000","last_peered":"2023-10-08T07:13:49.282248+0000","last_clean":"2023-10-08T07:13:49.282248+0000","last_became_active":"2023-10-08T07:13:33.729125+0000","last_became_peered":"2023-10-08T07:13:33.729125+0000","last_unstale":"2023-10-08T07:13:49.282248+0000","last_undegraded":"2023-10-08T07:13:49.282248+0000","last_fullsized":"2023-10-08T07:13:49.282248+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:13.193093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.361253+0000","last_change":"2023-10-08T07:13:34.308836+0000","last_active":"2023-10-08T07:13:47.361253+0000","last_peered":"2023-10-08T07:13:47.361253+0000","last_clean":"2023-10-08T07:13:47.361253+0000","last_became_active":"2023-10-08T07:13:34.308301+0000","last_became_peered":"2023-10-08T07:13:34.308301+0000","last_unstale":"2023-10-08T07:13:47.361253+0000","last_undegraded":"2023-10-08T07:13:47.361253+0000","last_fullsized":"2023-10-08T07:13:47.361253+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:42.058240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.391244+0000","last_change":"2023-10-08T07:13:48.390712+0000","last_active":"2023-10-08T07:13:48.391244+0000","last_peered":"2023-10-08T07:13:48.391244+0000","last_clean":"2023-10-08T07:13:48.391244+0000","last_became_active":"2023-10-08T07:13:48.390349+0000","last_became_peered":"2023-10-08T07:13:48.390349+0000","last_unstale":"2023-10-08T07:13:48.391244+0000","last_undegraded":"2023-10-08T07:13:48.391244+0000","last_fullsized":"2023-10-08T07:13:48.391244+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:08.228559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.367853+0000","last_change":"2023-10-08T07:13:48.367853+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.367853+0000","last_undegraded":"2023-10-08T07:13:48.367853+0000","last_fullsized":"2023-10-08T07:13:48.367853+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295073+0000","last_change":"2023-10-08T07:13:33.732139+0000","last_active":"2023-10-08T07:13:49.295073+0000","last_peered":"2023-10-08T07:13:49.295073+0000","last_clean":"2023-10-08T07:13:49.295073+0000","last_became_active":"2023-10-08T07:13:33.731892+0000","last_became_peered":"2023-10-08T07:13:33.731892+0000","last_unstale":"2023-10-08T07:13:49.295073+0000","last_undegraded":"2023-10-08T07:13:49.295073+0000","last_fullsized":"2023-10-08T07:13:49.295073+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:07.242491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343742+0000","last_change":"2023-10-08T07:13:34.299071+0000","last_active":"2023-10-08T07:13:47.343742+0000","last_peered":"2023-10-08T07:13:47.343742+0000","last_clean":"2023-10-08T07:13:47.343742+0000","last_became_active":"2023-10-08T07:13:34.298776+0000","last_became_peered":"2023-10-08T07:13:34.298776+0000","last_unstale":"2023-10-08T07:13:47.343742+0000","last_undegraded":"2023-10-08T07:13:47.343742+0000","last_fullsized":"2023-10-08T07:13:47.343742+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:10.642317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398369+0000","last_change":"2023-10-08T07:13:48.393982+0000","last_active":"2023-10-08T07:13:48.398369+0000","last_peered":"2023-10-08T07:13:48.398369+0000","last_clean":"2023-10-08T07:13:48.398369+0000","last_became_active":"2023-10-08T07:13:48.393696+0000","last_became_peered":"2023-10-08T07:13:48.393696+0000","last_unstale":"2023-10-08T07:13:48.398369+0000","last_undegraded":"2023-10-08T07:13:48.398369+0000","last_fullsized":"2023-10-08T07:13:48.398369+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:10.381086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358215+0000","last_change":"2023-10-08T07:13:49.312996+0000","last_active":"2023-10-08T07:13:50.358215+0000","last_peered":"2023-10-08T07:13:50.358215+0000","last_clean":"2023-10-08T07:13:50.358215+0000","last_became_active":"2023-10-08T07:13:49.312657+0000","last_became_peered":"2023-10-08T07:13:49.312657+0000","last_unstale":"2023-10-08T07:13:50.358215+0000","last_undegraded":"2023-10-08T07:13:50.358215+0000","last_fullsized":"2023-10-08T07:13:50.358215+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:28.367357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"233'2","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345211+0000","last_change":"2023-10-08T07:13:33.736960+0000","last_active":"2023-10-08T07:13:47.345211+0000","last_peered":"2023-10-08T07:13:47.345211+0000","last_clean":"2023-10-08T07:13:47.345211+0000","last_became_active":"2023-10-08T07:13:33.736703+0000","last_became_peered":"2023-10-08T07:13:33.736703+0000","last_unstale":"2023-10-08T07:13:47.345211+0000","last_undegraded":"2023-10-08T07:13:47.345211+0000","last_fullsized":"2023-10-08T07:13:47.345211+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:39.196152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343947+0000","last_change":"2023-10-08T07:13:34.291716+0000","last_active":"2023-10-08T07:13:47.343947+0000","last_peered":"2023-10-08T07:13:47.343947+0000","last_clean":"2023-10-08T07:13:47.343947+0000","last_became_active":"2023-10-08T07:13:34.291300+0000","last_became_peered":"2023-10-08T07:13:34.291300+0000","last_unstale":"2023-10-08T07:13:47.343947+0000","last_undegraded":"2023-10-08T07:13:47.343947+0000","last_fullsized":"2023-10-08T07:13:47.343947+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:07.189339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.277990+0000","last_change":"2023-10-08T07:13:48.383304+0000","last_active":"2023-10-08T07:13:49.277990+0000","last_peered":"2023-10-08T07:13:49.277990+0000","last_clean":"2023-10-08T07:13:49.277990+0000","last_became_active":"2023-10-08T07:13:48.383040+0000","last_became_peered":"2023-10-08T07:13:48.383040+0000","last_unstale":"2023-10-08T07:13:49.277990+0000","last_undegraded":"2023-10-08T07:13:49.277990+0000","last_fullsized":"2023-10-08T07:13:49.277990+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:05.475908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448022+0000","last_change":"2023-10-08T07:13:49.319374+0000","last_active":"2023-10-08T07:13:50.448022+0000","last_peered":"2023-10-08T07:13:50.448022+0000","last_clean":"2023-10-08T07:13:50.448022+0000","last_became_active":"2023-10-08T07:13:49.319049+0000","last_became_peered":"2023-10-08T07:13:49.319049+0000","last_unstale":"2023-10-08T07:13:50.448022+0000","last_undegraded":"2023-10-08T07:13:50.448022+0000","last_fullsized":"2023-10-08T07:13:50.448022+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:09.611966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345489+0000","last_change":"2023-10-08T07:13:33.728187+0000","last_active":"2023-10-08T07:13:47.345489+0000","last_peered":"2023-10-08T07:13:47.345489+0000","last_clean":"2023-10-08T07:13:47.345489+0000","last_became_active":"2023-10-08T07:13:33.727828+0000","last_became_peered":"2023-10-08T07:13:33.727828+0000","last_unstale":"2023-10-08T07:13:47.345489+0000","last_undegraded":"2023-10-08T07:13:47.345489+0000","last_fullsized":"2023-10-08T07:13:47.345489+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:00.539389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.360575+0000","last_change":"2023-10-08T07:13:34.293870+0000","last_active":"2023-10-08T07:13:47.360575+0000","last_peered":"2023-10-08T07:13:47.360575+0000","last_clean":"2023-10-08T07:13:47.360575+0000","last_became_active":"2023-10-08T07:13:34.293397+0000","last_became_peered":"2023-10-08T07:13:34.293397+0000","last_unstale":"2023-10-08T07:13:47.360575+0000","last_undegraded":"2023-10-08T07:13:47.360575+0000","last_fullsized":"2023-10-08T07:13:47.360575+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.387322+0000","last_change":"2023-10-08T07:13:48.385714+0000","last_active":"2023-10-08T07:13:48.387322+0000","last_peered":"2023-10-08T07:13:48.387322+0000","last_clean":"2023-10-08T07:13:48.387322+0000","last_became_active":"2023-10-08T07:13:48.385420+0000","last_became_peered":"2023-10-08T07:13:48.385420+0000","last_unstale":"2023-10-08T07:13:48.387322+0000","last_undegraded":"2023-10-08T07:13:48.387322+0000","last_fullsized":"2023-10-08T07:13:48.387322+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:51.558634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.378285+0000","last_change":"2023-10-08T07:13:48.378285+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.378285+0000","last_undegraded":"2023-10-08T07:13:48.378285+0000","last_fullsized":"2023-10-08T07:13:48.378285+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"233'8","reported_seq":24,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.278141+0000","last_change":"2023-10-08T07:13:33.738121+0000","last_active":"2023-10-08T07:13:49.278141+0000","last_peered":"2023-10-08T07:13:49.278141+0000","last_clean":"2023-10-08T07:13:49.278141+0000","last_became_active":"2023-10-08T07:13:33.737903+0000","last_became_peered":"2023-10-08T07:13:33.737903+0000","last_unstale":"2023-10-08T07:13:49.278141+0000","last_undegraded":"2023-10-08T07:13:49.278141+0000","last_fullsized":"2023-10-08T07:13:49.278141+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:15.275258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.355390+0000","last_change":"2023-10-08T07:13:34.306066+0000","last_active":"2023-10-08T07:13:47.355390+0000","last_peered":"2023-10-08T07:13:47.355390+0000","last_clean":"2023-10-08T07:13:47.355390+0000","last_became_active":"2023-10-08T07:13:34.302955+0000","last_became_peered":"2023-10-08T07:13:34.302955+0000","last_unstale":"2023-10-08T07:13:47.355390+0000","last_undegraded":"2023-10-08T07:13:47.355390+0000","last_fullsized":"2023-10-08T07:13:47.355390+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:03.795703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398039+0000","last_change":"2023-10-08T07:13:48.397214+0000","last_active":"2023-10-08T07:13:48.398039+0000","last_peered":"2023-10-08T07:13:48.398039+0000","last_clean":"2023-10-08T07:13:48.398039+0000","last_became_active":"2023-10-08T07:13:48.396880+0000","last_became_peered":"2023-10-08T07:13:48.396880+0000","last_unstale":"2023-10-08T07:13:48.398039+0000","last_undegraded":"2023-10-08T07:13:48.398039+0000","last_fullsized":"2023-10-08T07:13:48.398039+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:38:50.886917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.371621+0000","last_change":"2023-10-08T07:13:48.371621+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.371621+0000","last_undegraded":"2023-10-08T07:13:48.371621+0000","last_fullsized":"2023-10-08T07:13:48.371621+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286393+0000","last_change":"2023-10-08T07:13:33.730922+0000","last_active":"2023-10-08T07:13:49.286393+0000","last_peered":"2023-10-08T07:13:49.286393+0000","last_clean":"2023-10-08T07:13:49.286393+0000","last_became_active":"2023-10-08T07:13:33.730666+0000","last_became_peered":"2023-10-08T07:13:33.730666+0000","last_unstale":"2023-10-08T07:13:49.286393+0000","last_undegraded":"2023-10-08T07:13:49.286393+0000","last_fullsized":"2023-10-08T07:13:49.286393+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:12.086089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291729+0000","last_change":"2023-10-08T07:13:34.302276+0000","last_active":"2023-10-08T07:13:49.291729+0000","last_peered":"2023-10-08T07:13:49.291729+0000","last_clean":"2023-10-08T07:13:49.291729+0000","last_became_active":"2023-10-08T07:13:34.301988+0000","last_became_peered":"2023-10-08T07:13:34.301988+0000","last_unstale":"2023-10-08T07:13:49.291729+0000","last_undegraded":"2023-10-08T07:13:49.291729+0000","last_fullsized":"2023-10-08T07:13:49.291729+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.389272+0000","last_change":"2023-10-08T07:13:48.386216+0000","last_active":"2023-10-08T07:13:48.389272+0000","last_peered":"2023-10-08T07:13:48.389272+0000","last_clean":"2023-10-08T07:13:48.389272+0000","last_became_active":"2023-10-08T07:13:48.385877+0000","last_became_peered":"2023-10-08T07:13:48.385877+0000","last_unstale":"2023-10-08T07:13:48.389272+0000","last_undegraded":"2023-10-08T07:13:48.389272+0000","last_fullsized":"2023-10-08T07:13:48.389272+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:06.260935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.366477+0000","last_change":"2023-10-08T07:13:48.366477+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.366477+0000","last_undegraded":"2023-10-08T07:13:48.366477+0000","last_fullsized":"2023-10-08T07:13:48.366477+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348222+0000","last_change":"2023-10-08T07:13:33.725167+0000","last_active":"2023-10-08T07:13:47.348222+0000","last_peered":"2023-10-08T07:13:47.348222+0000","last_clean":"2023-10-08T07:13:47.348222+0000","last_became_active":"2023-10-08T07:13:33.724905+0000","last_became_peered":"2023-10-08T07:13:33.724905+0000","last_unstale":"2023-10-08T07:13:47.348222+0000","last_undegraded":"2023-10-08T07:13:47.348222+0000","last_fullsized":"2023-10-08T07:13:47.348222+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:37.525027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348897+0000","last_change":"2023-10-08T07:13:34.296170+0000","last_active":"2023-10-08T07:13:47.348897+0000","last_peered":"2023-10-08T07:13:47.348897+0000","last_clean":"2023-10-08T07:13:47.348897+0000","last_became_active":"2023-10-08T07:13:34.295736+0000","last_became_peered":"2023-10-08T07:13:34.295736+0000","last_unstale":"2023-10-08T07:13:47.348897+0000","last_undegraded":"2023-10-08T07:13:47.348897+0000","last_fullsized":"2023-10-08T07:13:47.348897+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396650+0000","last_change":"2023-10-08T07:13:48.395168+0000","last_active":"2023-10-08T07:13:48.396650+0000","last_peered":"2023-10-08T07:13:48.396650+0000","last_clean":"2023-10-08T07:13:48.396650+0000","last_became_active":"2023-10-08T07:13:48.394879+0000","last_became_peered":"2023-10-08T07:13:48.394879+0000","last_unstale":"2023-10-08T07:13:48.396650+0000","last_undegraded":"2023-10-08T07:13:48.396650+0000","last_fullsized":"2023-10-08T07:13:48.396650+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:07.520887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359261+0000","last_change":"2023-10-08T07:13:49.303768+0000","last_active":"2023-10-08T07:13:50.359261+0000","last_peered":"2023-10-08T07:13:50.359261+0000","last_clean":"2023-10-08T07:13:50.359261+0000","last_became_active":"2023-10-08T07:13:49.302929+0000","last_became_peered":"2023-10-08T07:13:49.302929+0000","last_unstale":"2023-10-08T07:13:50.359261+0000","last_undegraded":"2023-10-08T07:13:50.359261+0000","last_fullsized":"2023-10-08T07:13:50.359261+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:21.341547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295376+0000","last_change":"2023-10-08T07:13:33.730955+0000","last_active":"2023-10-08T07:13:49.295376+0000","last_peered":"2023-10-08T07:13:49.295376+0000","last_clean":"2023-10-08T07:13:49.295376+0000","last_became_active":"2023-10-08T07:13:33.730676+0000","last_became_peered":"2023-10-08T07:13:33.730676+0000","last_unstale":"2023-10-08T07:13:49.295376+0000","last_undegraded":"2023-10-08T07:13:49.295376+0000","last_fullsized":"2023-10-08T07:13:49.295376+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:28.953862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295664+0000","last_change":"2023-10-08T07:13:34.293585+0000","last_active":"2023-10-08T07:13:49.295664+0000","last_peered":"2023-10-08T07:13:49.295664+0000","last_clean":"2023-10-08T07:13:49.295664+0000","last_became_active":"2023-10-08T07:13:34.293318+0000","last_became_peered":"2023-10-08T07:13:34.293318+0000","last_unstale":"2023-10-08T07:13:49.295664+0000","last_undegraded":"2023-10-08T07:13:49.295664+0000","last_fullsized":"2023-10-08T07:13:49.295664+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:09:34.022529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.391979+0000","last_change":"2023-10-08T07:13:48.388878+0000","last_active":"2023-10-08T07:13:48.391979+0000","last_peered":"2023-10-08T07:13:48.391979+0000","last_clean":"2023-10-08T07:13:48.391979+0000","last_became_active":"2023-10-08T07:13:48.388595+0000","last_became_peered":"2023-10-08T07:13:48.388595+0000","last_unstale":"2023-10-08T07:13:48.391979+0000","last_undegraded":"2023-10-08T07:13:48.391979+0000","last_fullsized":"2023-10-08T07:13:48.391979+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:28.683424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.363741+0000","last_change":"2023-10-08T07:13:48.363741+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.363741+0000","last_undegraded":"2023-10-08T07:13:48.363741+0000","last_fullsized":"2023-10-08T07:13:48.363741+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"234'8","reported_seq":22,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.352112+0000","last_change":"2023-10-08T07:13:33.746002+0000","last_active":"2023-10-08T07:13:47.352112+0000","last_peered":"2023-10-08T07:13:47.352112+0000","last_clean":"2023-10-08T07:13:47.352112+0000","last_became_active":"2023-10-08T07:13:33.745697+0000","last_became_peered":"2023-10-08T07:13:33.745697+0000","last_unstale":"2023-10-08T07:13:47.352112+0000","last_undegraded":"2023-10-08T07:13:47.352112+0000","last_fullsized":"2023-10-08T07:13:47.352112+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:25.841040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.283013+0000","last_change":"2023-10-08T07:13:34.293551+0000","last_active":"2023-10-08T07:13:49.283013+0000","last_peered":"2023-10-08T07:13:49.283013+0000","last_clean":"2023-10-08T07:13:49.283013+0000","last_became_active":"2023-10-08T07:13:34.292471+0000","last_became_peered":"2023-10-08T07:13:34.292471+0000","last_unstale":"2023-10-08T07:13:49.283013+0000","last_undegraded":"2023-10-08T07:13:49.283013+0000","last_fullsized":"2023-10-08T07:13:49.283013+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288920+0000","last_change":"2023-10-08T07:13:48.373437+0000","last_active":"2023-10-08T07:13:49.288920+0000","last_peered":"2023-10-08T07:13:49.288920+0000","last_clean":"2023-10-08T07:13:49.288920+0000","last_became_active":"2023-10-08T07:13:48.373147+0000","last_became_peered":"2023-10-08T07:13:48.373147+0000","last_unstale":"2023-10-08T07:13:49.288920+0000","last_undegraded":"2023-10-08T07:13:49.288920+0000","last_fullsized":"2023-10-08T07:13:49.288920+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:07.670172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.447672+0000","last_change":"2023-10-08T07:13:49.311094+0000","last_active":"2023-10-08T07:13:50.447672+0000","last_peered":"2023-10-08T07:13:50.447672+0000","last_clean":"2023-10-08T07:13:50.447672+0000","last_became_active":"2023-10-08T07:13:49.310778+0000","last_became_peered":"2023-10-08T07:13:49.310778+0000","last_unstale":"2023-10-08T07:13:50.447672+0000","last_undegraded":"2023-10-08T07:13:50.447672+0000","last_fullsized":"2023-10-08T07:13:50.447672+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:25:13.654111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"233'3","reported_seq":19,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290932+0000","last_change":"2023-10-08T07:13:33.726791+0000","last_active":"2023-10-08T07:13:49.290932+0000","last_peered":"2023-10-08T07:13:49.290932+0000","last_clean":"2023-10-08T07:13:49.290932+0000","last_became_active":"2023-10-08T07:13:33.726482+0000","last_became_peered":"2023-10-08T07:13:33.726482+0000","last_unstale":"2023-10-08T07:13:49.290932+0000","last_undegraded":"2023-10-08T07:13:49.290932+0000","last_fullsized":"2023-10-08T07:13:49.290932+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:11.409539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289130+0000","last_change":"2023-10-08T07:13:34.290971+0000","last_active":"2023-10-08T07:13:49.289130+0000","last_peered":"2023-10-08T07:13:49.289130+0000","last_clean":"2023-10-08T07:13:49.289130+0000","last_became_active":"2023-10-08T07:13:34.290294+0000","last_became_peered":"2023-10-08T07:13:34.290294+0000","last_unstale":"2023-10-08T07:13:49.289130+0000","last_undegraded":"2023-10-08T07:13:49.289130+0000","last_fullsized":"2023-10-08T07:13:49.289130+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.393740+0000","last_change":"2023-10-08T07:13:48.388537+0000","last_active":"2023-10-08T07:13:48.393740+0000","last_peered":"2023-10-08T07:13:48.393740+0000","last_clean":"2023-10-08T07:13:48.393740+0000","last_became_active":"2023-10-08T07:13:48.388005+0000","last_became_peered":"2023-10-08T07:13:48.388005+0000","last_unstale":"2023-10-08T07:13:48.393740+0000","last_undegraded":"2023-10-08T07:13:48.393740+0000","last_fullsized":"2023-10-08T07:13:48.393740+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:50.761583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376614+0000","last_change":"2023-10-08T07:13:48.376614+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376614+0000","last_undegraded":"2023-10-08T07:13:48.376614+0000","last_fullsized":"2023-10-08T07:13:48.376614+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.352897+0000","last_change":"2023-10-08T07:13:33.736423+0000","last_active":"2023-10-08T07:13:47.352897+0000","last_peered":"2023-10-08T07:13:47.352897+0000","last_clean":"2023-10-08T07:13:47.352897+0000","last_became_active":"2023-10-08T07:13:33.735959+0000","last_became_peered":"2023-10-08T07:13:33.735959+0000","last_unstale":"2023-10-08T07:13:47.352897+0000","last_undegraded":"2023-10-08T07:13:47.352897+0000","last_fullsized":"2023-10-08T07:13:47.352897+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:00.365787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.347992+0000","last_change":"2023-10-08T07:13:34.302391+0000","last_active":"2023-10-08T07:13:47.347992+0000","last_peered":"2023-10-08T07:13:47.347992+0000","last_clean":"2023-10-08T07:13:47.347992+0000","last_became_active":"2023-10-08T07:13:34.302105+0000","last_became_peered":"2023-10-08T07:13:34.302105+0000","last_unstale":"2023-10-08T07:13:47.347992+0000","last_undegraded":"2023-10-08T07:13:47.347992+0000","last_fullsized":"2023-10-08T07:13:47.347992+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396485+0000","last_change":"2023-10-08T07:13:48.391735+0000","last_active":"2023-10-08T07:13:48.396485+0000","last_peered":"2023-10-08T07:13:48.396485+0000","last_clean":"2023-10-08T07:13:48.396485+0000","last_became_active":"2023-10-08T07:13:48.391330+0000","last_became_peered":"2023-10-08T07:13:48.391330+0000","last_unstale":"2023-10-08T07:13:48.396485+0000","last_undegraded":"2023-10-08T07:13:48.396485+0000","last_fullsized":"2023-10-08T07:13:48.396485+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:19.090961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357935+0000","last_change":"2023-10-08T07:13:49.313956+0000","last_active":"2023-10-08T07:13:50.357935+0000","last_peered":"2023-10-08T07:13:50.357935+0000","last_clean":"2023-10-08T07:13:50.357935+0000","last_became_active":"2023-10-08T07:13:49.313687+0000","last_became_peered":"2023-10-08T07:13:49.313687+0000","last_unstale":"2023-10-08T07:13:50.357935+0000","last_undegraded":"2023-10-08T07:13:50.357935+0000","last_fullsized":"2023-10-08T07:13:50.357935+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:35.653573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284639+0000","last_change":"2023-10-08T07:13:33.735184+0000","last_active":"2023-10-08T07:13:49.284639+0000","last_peered":"2023-10-08T07:13:49.284639+0000","last_clean":"2023-10-08T07:13:49.284639+0000","last_became_active":"2023-10-08T07:13:33.734915+0000","last_became_peered":"2023-10-08T07:13:33.734915+0000","last_unstale":"2023-10-08T07:13:49.284639+0000","last_undegraded":"2023-10-08T07:13:49.284639+0000","last_fullsized":"2023-10-08T07:13:49.284639+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:52.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.354637+0000","last_change":"2023-10-08T07:13:34.294204+0000","last_active":"2023-10-08T07:13:47.354637+0000","last_peered":"2023-10-08T07:13:47.354637+0000","last_clean":"2023-10-08T07:13:47.354637+0000","last_became_active":"2023-10-08T07:13:34.293887+0000","last_became_peered":"2023-10-08T07:13:34.293887+0000","last_unstale":"2023-10-08T07:13:47.354637+0000","last_undegraded":"2023-10-08T07:13:47.354637+0000","last_fullsized":"2023-10-08T07:13:47.354637+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:59.699025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.391072+0000","last_change":"2023-10-08T07:13:48.389962+0000","last_active":"2023-10-08T07:13:48.391072+0000","last_peered":"2023-10-08T07:13:48.391072+0000","last_clean":"2023-10-08T07:13:48.391072+0000","last_became_active":"2023-10-08T07:13:48.389607+0000","last_became_peered":"2023-10-08T07:13:48.389607+0000","last_unstale":"2023-10-08T07:13:48.391072+0000","last_undegraded":"2023-10-08T07:13:48.391072+0000","last_fullsized":"2023-10-08T07:13:48.391072+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:14.995707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358517+0000","last_change":"2023-10-08T07:13:49.299000+0000","last_active":"2023-10-08T07:13:50.358517+0000","last_peered":"2023-10-08T07:13:50.358517+0000","last_clean":"2023-10-08T07:13:50.358517+0000","last_became_active":"2023-10-08T07:13:49.298705+0000","last_became_peered":"2023-10-08T07:13:49.298705+0000","last_unstale":"2023-10-08T07:13:50.358517+0000","last_undegraded":"2023-10-08T07:13:50.358517+0000","last_fullsized":"2023-10-08T07:13:50.358517+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:46.814809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398278+0000","last_change":"2023-10-08T07:13:48.393308+0000","last_active":"2023-10-08T07:13:48.398278+0000","last_peered":"2023-10-08T07:13:48.398278+0000","last_clean":"2023-10-08T07:13:48.398278+0000","last_became_active":"2023-10-08T07:13:48.392985+0000","last_became_peered":"2023-10-08T07:13:48.392985+0000","last_unstale":"2023-10-08T07:13:48.398278+0000","last_undegraded":"2023-10-08T07:13:48.398278+0000","last_fullsized":"2023-10-08T07:13:48.398278+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:13.322938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.347589+0000","last_change":"2023-10-08T07:13:33.731054+0000","last_active":"2023-10-08T07:13:47.347589+0000","last_peered":"2023-10-08T07:13:47.347589+0000","last_clean":"2023-10-08T07:13:47.347589+0000","last_became_active":"2023-10-08T07:13:33.730782+0000","last_became_peered":"2023-10-08T07:13:33.730782+0000","last_unstale":"2023-10-08T07:13:47.347589+0000","last_undegraded":"2023-10-08T07:13:47.347589+0000","last_fullsized":"2023-10-08T07:13:47.347589+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:06.476515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292156+0000","last_change":"2023-10-08T07:13:34.294260+0000","last_active":"2023-10-08T07:13:49.292156+0000","last_peered":"2023-10-08T07:13:49.292156+0000","last_clean":"2023-10-08T07:13:49.292156+0000","last_became_active":"2023-10-08T07:13:34.293992+0000","last_became_peered":"2023-10-08T07:13:34.293992+0000","last_unstale":"2023-10-08T07:13:49.292156+0000","last_undegraded":"2023-10-08T07:13:49.292156+0000","last_fullsized":"2023-10-08T07:13:49.292156+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358287+0000","last_change":"2023-10-08T07:13:49.309351+0000","last_active":"2023-10-08T07:13:50.358287+0000","last_peered":"2023-10-08T07:13:50.358287+0000","last_clean":"2023-10-08T07:13:50.358287+0000","last_became_active":"2023-10-08T07:13:49.309030+0000","last_became_peered":"2023-10-08T07:13:49.309030+0000","last_unstale":"2023-10-08T07:13:50.358287+0000","last_undegraded":"2023-10-08T07:13:50.358287+0000","last_fullsized":"2023-10-08T07:13:50.358287+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:08.833825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"237'4","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.372634+0000","last_change":"2023-10-08T07:13:48.385188+0000","last_active":"2023-10-08T07:13:49.372634+0000","last_peered":"2023-10-08T07:13:49.372634+0000","last_clean":"2023-10-08T07:13:49.372634+0000","last_became_active":"2023-10-08T07:13:48.384790+0000","last_became_peered":"2023-10-08T07:13:48.384790+0000","last_unstale":"2023-10-08T07:13:49.372634+0000","last_undegraded":"2023-10-08T07:13:49.372634+0000","last_fullsized":"2023-10-08T07:13:49.372634+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:30.625357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.350898+0000","last_change":"2023-10-08T07:13:33.738155+0000","last_active":"2023-10-08T07:13:47.350898+0000","last_peered":"2023-10-08T07:13:47.350898+0000","last_clean":"2023-10-08T07:13:47.350898+0000","last_became_active":"2023-10-08T07:13:33.737850+0000","last_became_peered":"2023-10-08T07:13:33.737850+0000","last_unstale":"2023-10-08T07:13:47.350898+0000","last_undegraded":"2023-10-08T07:13:47.350898+0000","last_fullsized":"2023-10-08T07:13:47.350898+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:37.479840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.356485+0000","last_change":"2023-10-08T07:13:34.304888+0000","last_active":"2023-10-08T07:13:47.356485+0000","last_peered":"2023-10-08T07:13:47.356485+0000","last_clean":"2023-10-08T07:13:47.356485+0000","last_became_active":"2023-10-08T07:13:34.304536+0000","last_became_peered":"2023-10-08T07:13:34.304536+0000","last_unstale":"2023-10-08T07:13:47.356485+0000","last_undegraded":"2023-10-08T07:13:47.356485+0000","last_fullsized":"2023-10-08T07:13:47.356485+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:05.467048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.379499+0000","last_change":"2023-10-08T07:13:48.379499+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.379499+0000","last_undegraded":"2023-10-08T07:13:48.379499+0000","last_fullsized":"2023-10-08T07:13:48.379499+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.351392+0000","last_change":"2023-10-08T07:13:33.738971+0000","last_active":"2023-10-08T07:13:47.351392+0000","last_peered":"2023-10-08T07:13:47.351392+0000","last_clean":"2023-10-08T07:13:47.351392+0000","last_became_active":"2023-10-08T07:13:33.738532+0000","last_became_peered":"2023-10-08T07:13:33.738532+0000","last_unstale":"2023-10-08T07:13:47.351392+0000","last_undegraded":"2023-10-08T07:13:47.351392+0000","last_fullsized":"2023-10-08T07:13:47.351392+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:06.806347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.356107+0000","last_change":"2023-10-08T07:13:34.302593+0000","last_active":"2023-10-08T07:13:47.356107+0000","last_peered":"2023-10-08T07:13:47.356107+0000","last_clean":"2023-10-08T07:13:47.356107+0000","last_became_active":"2023-10-08T07:13:34.302300+0000","last_became_peered":"2023-10-08T07:13:34.302300+0000","last_unstale":"2023-10-08T07:13:47.356107+0000","last_undegraded":"2023-10-08T07:13:47.356107+0000","last_fullsized":"2023-10-08T07:13:47.356107+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:13:33.465222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396596+0000","last_change":"2023-10-08T07:13:48.395886+0000","last_active":"2023-10-08T07:13:48.396596+0000","last_peered":"2023-10-08T07:13:48.396596+0000","last_clean":"2023-10-08T07:13:48.396596+0000","last_became_active":"2023-10-08T07:13:48.395585+0000","last_became_peered":"2023-10-08T07:13:48.395585+0000","last_unstale":"2023-10-08T07:13:48.396596+0000","last_undegraded":"2023-10-08T07:13:48.396596+0000","last_fullsized":"2023-10-08T07:13:48.396596+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:11.959208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.378495+0000","last_change":"2023-10-08T07:13:48.378495+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.378495+0000","last_undegraded":"2023-10-08T07:13:48.378495+0000","last_fullsized":"2023-10-08T07:13:48.378495+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290749+0000","last_change":"2023-10-08T07:13:33.736823+0000","last_active":"2023-10-08T07:13:49.290749+0000","last_peered":"2023-10-08T07:13:49.290749+0000","last_clean":"2023-10-08T07:13:49.290749+0000","last_became_active":"2023-10-08T07:13:33.736537+0000","last_became_peered":"2023-10-08T07:13:33.736537+0000","last_unstale":"2023-10-08T07:13:49.290749+0000","last_undegraded":"2023-10-08T07:13:49.290749+0000","last_fullsized":"2023-10-08T07:13:49.290749+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:48.798820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282798+0000","last_change":"2023-10-08T07:13:34.293354+0000","last_active":"2023-10-08T07:13:49.282798+0000","last_peered":"2023-10-08T07:13:49.282798+0000","last_clean":"2023-10-08T07:13:49.282798+0000","last_became_active":"2023-10-08T07:13:34.288220+0000","last_became_peered":"2023-10-08T07:13:34.288220+0000","last_unstale":"2023-10-08T07:13:49.282798+0000","last_undegraded":"2023-10-08T07:13:49.282798+0000","last_fullsized":"2023-10-08T07:13:49.282798+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398331+0000","last_change":"2023-10-08T07:13:48.397371+0000","last_active":"2023-10-08T07:13:48.398331+0000","last_peered":"2023-10-08T07:13:48.398331+0000","last_clean":"2023-10-08T07:13:48.398331+0000","last_became_active":"2023-10-08T07:13:48.396954+0000","last_became_peered":"2023-10-08T07:13:48.396954+0000","last_unstale":"2023-10-08T07:13:48.398331+0000","last_undegraded":"2023-10-08T07:13:48.398331+0000","last_fullsized":"2023-10-08T07:13:48.398331+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:39:02.780468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358686+0000","last_change":"2023-10-08T07:13:49.309391+0000","last_active":"2023-10-08T07:13:50.358686+0000","last_peered":"2023-10-08T07:13:50.358686+0000","last_clean":"2023-10-08T07:13:50.358686+0000","last_became_active":"2023-10-08T07:13:49.309100+0000","last_became_peered":"2023-10-08T07:13:49.309100+0000","last_unstale":"2023-10-08T07:13:50.358686+0000","last_undegraded":"2023-10-08T07:13:50.358686+0000","last_fullsized":"2023-10-08T07:13:50.358686+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:36.841571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294941+0000","last_change":"2023-10-08T07:13:33.724335+0000","last_active":"2023-10-08T07:13:49.294941+0000","last_peered":"2023-10-08T07:13:49.294941+0000","last_clean":"2023-10-08T07:13:49.294941+0000","last_became_active":"2023-10-08T07:13:33.724043+0000","last_became_peered":"2023-10-08T07:13:33.724043+0000","last_unstale":"2023-10-08T07:13:49.294941+0000","last_undegraded":"2023-10-08T07:13:49.294941+0000","last_fullsized":"2023-10-08T07:13:49.294941+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:58.128241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359306+0000","last_change":"2023-10-08T07:13:34.301668+0000","last_active":"2023-10-08T07:13:47.359306+0000","last_peered":"2023-10-08T07:13:47.359306+0000","last_clean":"2023-10-08T07:13:47.359306+0000","last_became_active":"2023-10-08T07:13:34.298698+0000","last_became_peered":"2023-10-08T07:13:34.298698+0000","last_unstale":"2023-10-08T07:13:47.359306+0000","last_undegraded":"2023-10-08T07:13:47.359306+0000","last_fullsized":"2023-10-08T07:13:47.359306+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.399490+0000","last_change":"2023-10-08T07:13:48.399044+0000","last_active":"2023-10-08T07:13:48.399490+0000","last_peered":"2023-10-08T07:13:48.399490+0000","last_clean":"2023-10-08T07:13:48.399490+0000","last_became_active":"2023-10-08T07:13:48.398793+0000","last_became_peered":"2023-10-08T07:13:48.398793+0000","last_unstale":"2023-10-08T07:13:48.399490+0000","last_undegraded":"2023-10-08T07:13:48.399490+0000","last_fullsized":"2023-10-08T07:13:48.399490+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:49:58.133301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278331+0000","last_change":"2023-10-08T07:13:49.303740+0000","last_active":"2023-10-08T07:13:50.278331+0000","last_peered":"2023-10-08T07:13:50.278331+0000","last_clean":"2023-10-08T07:13:50.278331+0000","last_became_active":"2023-10-08T07:13:49.303416+0000","last_became_peered":"2023-10-08T07:13:49.303416+0000","last_unstale":"2023-10-08T07:13:50.278331+0000","last_undegraded":"2023-10-08T07:13:50.278331+0000","last_fullsized":"2023-10-08T07:13:50.278331+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:38:51.904860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345012+0000","last_change":"2023-10-08T07:13:33.732312+0000","last_active":"2023-10-08T07:13:47.345012+0000","last_peered":"2023-10-08T07:13:47.345012+0000","last_clean":"2023-10-08T07:13:47.345012+0000","last_became_active":"2023-10-08T07:13:33.731998+0000","last_became_peered":"2023-10-08T07:13:33.731998+0000","last_unstale":"2023-10-08T07:13:47.345012+0000","last_undegraded":"2023-10-08T07:13:47.345012+0000","last_fullsized":"2023-10-08T07:13:47.345012+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:10:53.964103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343744+0000","last_change":"2023-10-08T07:13:34.293829+0000","last_active":"2023-10-08T07:13:47.343744+0000","last_peered":"2023-10-08T07:13:47.343744+0000","last_clean":"2023-10-08T07:13:47.343744+0000","last_became_active":"2023-10-08T07:13:34.293552+0000","last_became_peered":"2023-10-08T07:13:34.293552+0000","last_unstale":"2023-10-08T07:13:47.343744+0000","last_undegraded":"2023-10-08T07:13:47.343744+0000","last_fullsized":"2023-10-08T07:13:47.343744+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:38.485847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294683+0000","last_change":"2023-10-08T07:13:48.382427+0000","last_active":"2023-10-08T07:13:49.294683+0000","last_peered":"2023-10-08T07:13:49.294683+0000","last_clean":"2023-10-08T07:13:49.294683+0000","last_became_active":"2023-10-08T07:13:48.382141+0000","last_became_peered":"2023-10-08T07:13:48.382141+0000","last_unstale":"2023-10-08T07:13:49.294683+0000","last_undegraded":"2023-10-08T07:13:49.294683+0000","last_fullsized":"2023-10-08T07:13:49.294683+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:26.319310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.374502+0000","last_change":"2023-10-08T07:13:48.374502+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.374502+0000","last_undegraded":"2023-10-08T07:13:48.374502+0000","last_fullsized":"2023-10-08T07:13:48.374502+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"233'4","reported_seq":18,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.357270+0000","last_change":"2023-10-08T07:13:33.743017+0000","last_active":"2023-10-08T07:13:47.357270+0000","last_peered":"2023-10-08T07:13:47.357270+0000","last_clean":"2023-10-08T07:13:47.357270+0000","last_became_active":"2023-10-08T07:13:33.742582+0000","last_became_peered":"2023-10-08T07:13:33.742582+0000","last_unstale":"2023-10-08T07:13:47.357270+0000","last_undegraded":"2023-10-08T07:13:47.357270+0000","last_fullsized":"2023-10-08T07:13:47.357270+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:38:56.287497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296385+0000","last_change":"2023-10-08T07:13:34.286367+0000","last_active":"2023-10-08T07:13:49.296385+0000","last_peered":"2023-10-08T07:13:49.296385+0000","last_clean":"2023-10-08T07:13:49.296385+0000","last_became_active":"2023-10-08T07:13:34.285016+0000","last_became_peered":"2023-10-08T07:13:34.285016+0000","last_unstale":"2023-10-08T07:13:49.296385+0000","last_undegraded":"2023-10-08T07:13:49.296385+0000","last_fullsized":"2023-10-08T07:13:49.296385+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:41.466102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.397397+0000","last_change":"2023-10-08T07:13:48.394830+0000","last_active":"2023-10-08T07:13:48.397397+0000","last_peered":"2023-10-08T07:13:48.397397+0000","last_clean":"2023-10-08T07:13:48.397397+0000","last_became_active":"2023-10-08T07:13:48.394550+0000","last_became_peered":"2023-10-08T07:13:48.394550+0000","last_unstale":"2023-10-08T07:13:48.397397+0000","last_undegraded":"2023-10-08T07:13:48.397397+0000","last_fullsized":"2023-10-08T07:13:48.397397+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:33.349540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376167+0000","last_change":"2023-10-08T07:13:48.376167+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376167+0000","last_undegraded":"2023-10-08T07:13:48.376167+0000","last_fullsized":"2023-10-08T07:13:48.376167+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.349890+0000","last_change":"2023-10-08T07:13:34.295873+0000","last_active":"2023-10-08T07:13:47.349890+0000","last_peered":"2023-10-08T07:13:47.349890+0000","last_clean":"2023-10-08T07:13:47.349890+0000","last_became_active":"2023-10-08T07:13:34.295546+0000","last_became_peered":"2023-10-08T07:13:34.295546+0000","last_unstale":"2023-10-08T07:13:47.349890+0000","last_undegraded":"2023-10-08T07:13:47.349890+0000","last_fullsized":"2023-10-08T07:13:47.349890+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:46:53.896975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448531+0000","last_change":"2023-10-08T07:13:49.312717+0000","last_active":"2023-10-08T07:13:50.448531+0000","last_peered":"2023-10-08T07:13:50.448531+0000","last_clean":"2023-10-08T07:13:50.448531+0000","last_became_active":"2023-10-08T07:13:49.312448+0000","last_became_peered":"2023-10-08T07:13:49.312448+0000","last_unstale":"2023-10-08T07:13:50.448531+0000","last_undegraded":"2023-10-08T07:13:50.448531+0000","last_fullsized":"2023-10-08T07:13:50.448531+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:52.376348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.350665+0000","last_change":"2023-10-08T07:13:34.301071+0000","last_active":"2023-10-08T07:13:47.350665+0000","last_peered":"2023-10-08T07:13:47.350665+0000","last_clean":"2023-10-08T07:13:47.350665+0000","last_became_active":"2023-10-08T07:13:34.300772+0000","last_became_peered":"2023-10-08T07:13:34.300772+0000","last_unstale":"2023-10-08T07:13:47.350665+0000","last_undegraded":"2023-10-08T07:13:47.350665+0000","last_fullsized":"2023-10-08T07:13:47.350665+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:59:54.643977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.372376+0000","last_change":"2023-10-08T07:13:48.372376+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.372376+0000","last_undegraded":"2023-10-08T07:13:48.372376+0000","last_fullsized":"2023-10-08T07:13:48.372376+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.353002+0000","last_change":"2023-10-08T07:13:34.300974+0000","last_active":"2023-10-08T07:13:47.353002+0000","last_peered":"2023-10-08T07:13:47.353002+0000","last_clean":"2023-10-08T07:13:47.353002+0000","last_became_active":"2023-10-08T07:13:34.300529+0000","last_became_peered":"2023-10-08T07:13:34.300529+0000","last_unstale":"2023-10-08T07:13:47.353002+0000","last_undegraded":"2023-10-08T07:13:47.353002+0000","last_fullsized":"2023-10-08T07:13:47.353002+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:17.001444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278016+0000","last_change":"2023-10-08T07:13:49.306936+0000","last_active":"2023-10-08T07:13:50.278016+0000","last_peered":"2023-10-08T07:13:50.278016+0000","last_clean":"2023-10-08T07:13:50.278016+0000","last_became_active":"2023-10-08T07:13:49.306284+0000","last_became_peered":"2023-10-08T07:13:49.306284+0000","last_unstale":"2023-10-08T07:13:50.278016+0000","last_undegraded":"2023-10-08T07:13:50.278016+0000","last_fullsized":"2023-10-08T07:13:50.278016+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:51.779526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287748+0000","last_change":"2023-10-08T07:13:34.293712+0000","last_active":"2023-10-08T07:13:49.287748+0000","last_peered":"2023-10-08T07:13:49.287748+0000","last_clean":"2023-10-08T07:13:49.287748+0000","last_became_active":"2023-10-08T07:13:34.293415+0000","last_became_peered":"2023-10-08T07:13:34.293415+0000","last_unstale":"2023-10-08T07:13:49.287748+0000","last_undegraded":"2023-10-08T07:13:49.287748+0000","last_fullsized":"2023-10-08T07:13:49.287748+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:47.966787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359086+0000","last_change":"2023-10-08T07:13:49.316786+0000","last_active":"2023-10-08T07:13:50.359086+0000","last_peered":"2023-10-08T07:13:50.359086+0000","last_clean":"2023-10-08T07:13:50.359086+0000","last_became_active":"2023-10-08T07:13:49.316437+0000","last_became_peered":"2023-10-08T07:13:49.316437+0000","last_unstale":"2023-10-08T07:13:50.359086+0000","last_undegraded":"2023-10-08T07:13:50.359086+0000","last_fullsized":"2023-10-08T07:13:50.359086+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:14.040352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282033+0000","last_change":"2023-10-08T07:13:34.297816+0000","last_active":"2023-10-08T07:13:49.282033+0000","last_peered":"2023-10-08T07:13:49.282033+0000","last_clean":"2023-10-08T07:13:49.282033+0000","last_became_active":"2023-10-08T07:13:34.297544+0000","last_became_peered":"2023-10-08T07:13:34.297544+0000","last_unstale":"2023-10-08T07:13:49.282033+0000","last_undegraded":"2023-10-08T07:13:49.282033+0000","last_fullsized":"2023-10-08T07:13:49.282033+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.375861+0000","last_change":"2023-10-08T07:13:48.375861+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.375861+0000","last_undegraded":"2023-10-08T07:13:48.375861+0000","last_fullsized":"2023-10-08T07:13:48.375861+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348874+0000","last_change":"2023-10-08T07:13:34.310969+0000","last_active":"2023-10-08T07:13:47.348874+0000","last_peered":"2023-10-08T07:13:47.348874+0000","last_clean":"2023-10-08T07:13:47.348874+0000","last_became_active":"2023-10-08T07:13:34.310679+0000","last_became_peered":"2023-10-08T07:13:34.310679+0000","last_unstale":"2023-10-08T07:13:47.348874+0000","last_undegraded":"2023-10-08T07:13:47.348874+0000","last_fullsized":"2023-10-08T07:13:47.348874+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381042+0000","last_change":"2023-10-08T07:13:48.381042+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381042+0000","last_undegraded":"2023-10-08T07:13:48.381042+0000","last_fullsized":"2023-10-08T07:13:48.381042+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344914+0000","last_change":"2023-10-08T07:13:34.304704+0000","last_active":"2023-10-08T07:13:47.344914+0000","last_peered":"2023-10-08T07:13:47.344914+0000","last_clean":"2023-10-08T07:13:47.344914+0000","last_became_active":"2023-10-08T07:13:34.304032+0000","last_became_peered":"2023-10-08T07:13:34.304032+0000","last_unstale":"2023-10-08T07:13:47.344914+0000","last_undegraded":"2023-10-08T07:13:47.344914+0000","last_fullsized":"2023-10-08T07:13:47.344914+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:15.818894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.367829+0000","last_change":"2023-10-08T07:13:48.367829+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.367829+0000","last_undegraded":"2023-10-08T07:13:48.367829+0000","last_fullsized":"2023-10-08T07:13:48.367829+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292077+0000","last_change":"2023-10-08T07:13:34.298953+0000","last_active":"2023-10-08T07:13:49.292077+0000","last_peered":"2023-10-08T07:13:49.292077+0000","last_clean":"2023-10-08T07:13:49.292077+0000","last_became_active":"2023-10-08T07:13:34.298694+0000","last_became_peered":"2023-10-08T07:13:34.298694+0000","last_unstale":"2023-10-08T07:13:49.292077+0000","last_undegraded":"2023-10-08T07:13:49.292077+0000","last_fullsized":"2023-10-08T07:13:49.292077+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376110+0000","last_change":"2023-10-08T07:13:48.376110+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376110+0000","last_undegraded":"2023-10-08T07:13:48.376110+0000","last_fullsized":"2023-10-08T07:13:48.376110+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.277869+0000","last_change":"2023-10-08T07:13:34.293797+0000","last_active":"2023-10-08T07:13:49.277869+0000","last_peered":"2023-10-08T07:13:49.277869+0000","last_clean":"2023-10-08T07:13:49.277869+0000","last_became_active":"2023-10-08T07:13:34.289224+0000","last_became_peered":"2023-10-08T07:13:34.289224+0000","last_unstale":"2023-10-08T07:13:49.277869+0000","last_undegraded":"2023-10-08T07:13:49.277869+0000","last_fullsized":"2023-10-08T07:13:49.277869+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357969+0000","last_change":"2023-10-08T07:13:49.303551+0000","last_active":"2023-10-08T07:13:50.357969+0000","last_peered":"2023-10-08T07:13:50.357969+0000","last_clean":"2023-10-08T07:13:50.357969+0000","last_became_active":"2023-10-08T07:13:49.303258+0000","last_became_peered":"2023-10-08T07:13:49.303258+0000","last_unstale":"2023-10-08T07:13:50.357969+0000","last_undegraded":"2023-10-08T07:13:50.357969+0000","last_fullsized":"2023-10-08T07:13:50.357969+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:08.504585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295174+0000","last_change":"2023-10-08T07:13:34.291585+0000","last_active":"2023-10-08T07:13:49.295174+0000","last_peered":"2023-10-08T07:13:49.295174+0000","last_clean":"2023-10-08T07:13:49.295174+0000","last_became_active":"2023-10-08T07:13:34.291125+0000","last_became_peered":"2023-10-08T07:13:34.291125+0000","last_unstale":"2023-10-08T07:13:49.295174+0000","last_undegraded":"2023-10-08T07:13:49.295174+0000","last_fullsized":"2023-10-08T07:13:49.295174+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:14.399506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.372077+0000","last_change":"2023-10-08T07:13:48.372077+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.372077+0000","last_undegraded":"2023-10-08T07:13:48.372077+0000","last_fullsized":"2023-10-08T07:13:48.372077+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359175+0000","last_change":"2023-10-08T07:13:34.297774+0000","last_active":"2023-10-08T07:13:47.359175+0000","last_peered":"2023-10-08T07:13:47.359175+0000","last_clean":"2023-10-08T07:13:47.359175+0000","last_became_active":"2023-10-08T07:13:34.297492+0000","last_became_peered":"2023-10-08T07:13:34.297492+0000","last_unstale":"2023-10-08T07:13:47.359175+0000","last_undegraded":"2023-10-08T07:13:47.359175+0000","last_fullsized":"2023-10-08T07:13:47.359175+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.277878+0000","last_change":"2023-10-08T07:13:49.298115+0000","last_active":"2023-10-08T07:13:50.277878+0000","last_peered":"2023-10-08T07:13:50.277878+0000","last_clean":"2023-10-08T07:13:50.277878+0000","last_became_active":"2023-10-08T07:13:49.297582+0000","last_became_peered":"2023-10-08T07:13:49.297582+0000","last_unstale":"2023-10-08T07:13:50.277878+0000","last_undegraded":"2023-10-08T07:13:50.277878+0000","last_fullsized":"2023-10-08T07:13:50.277878+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:11.399029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.361137+0000","last_change":"2023-10-08T07:13:34.294431+0000","last_active":"2023-10-08T07:13:47.361137+0000","last_peered":"2023-10-08T07:13:47.361137+0000","last_clean":"2023-10-08T07:13:47.361137+0000","last_became_active":"2023-10-08T07:13:34.294129+0000","last_became_peered":"2023-10-08T07:13:34.294129+0000","last_unstale":"2023-10-08T07:13:47.361137+0000","last_undegraded":"2023-10-08T07:13:47.361137+0000","last_fullsized":"2023-10-08T07:13:47.361137+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358949+0000","last_change":"2023-10-08T07:13:49.313393+0000","last_active":"2023-10-08T07:13:50.358949+0000","last_peered":"2023-10-08T07:13:50.358949+0000","last_clean":"2023-10-08T07:13:50.358949+0000","last_became_active":"2023-10-08T07:13:49.313094+0000","last_became_peered":"2023-10-08T07:13:49.313094+0000","last_unstale":"2023-10-08T07:13:50.358949+0000","last_undegraded":"2023-10-08T07:13:50.358949+0000","last_fullsized":"2023-10-08T07:13:50.358949+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:06.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359288+0000","last_change":"2023-10-08T07:13:34.297677+0000","last_active":"2023-10-08T07:13:47.359288+0000","last_peered":"2023-10-08T07:13:47.359288+0000","last_clean":"2023-10-08T07:13:47.359288+0000","last_became_active":"2023-10-08T07:13:34.296526+0000","last_became_peered":"2023-10-08T07:13:34.296526+0000","last_unstale":"2023-10-08T07:13:47.359288+0000","last_undegraded":"2023-10-08T07:13:47.359288+0000","last_fullsized":"2023-10-08T07:13:47.359288+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:51.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381122+0000","last_change":"2023-10-08T07:13:48.381122+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381122+0000","last_undegraded":"2023-10-08T07:13:48.381122+0000","last_fullsized":"2023-10-08T07:13:48.381122+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359714+0000","last_change":"2023-10-08T07:13:34.295222+0000","last_active":"2023-10-08T07:13:47.359714+0000","last_peered":"2023-10-08T07:13:47.359714+0000","last_clean":"2023-10-08T07:13:47.359714+0000","last_became_active":"2023-10-08T07:13:34.289517+0000","last_became_peered":"2023-10-08T07:13:34.289517+0000","last_unstale":"2023-10-08T07:13:47.359714+0000","last_undegraded":"2023-10-08T07:13:47.359714+0000","last_fullsized":"2023-10-08T07:13:47.359714+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:38.224838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.380405+0000","last_change":"2023-10-08T07:13:48.380405+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.380405+0000","last_undegraded":"2023-10-08T07:13:48.380405+0000","last_fullsized":"2023-10-08T07:13:48.380405+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291605+0000","last_change":"2023-10-08T07:13:34.299238+0000","last_active":"2023-10-08T07:13:49.291605+0000","last_peered":"2023-10-08T07:13:49.291605+0000","last_clean":"2023-10-08T07:13:49.291605+0000","last_became_active":"2023-10-08T07:13:34.298585+0000","last_became_peered":"2023-10-08T07:13:34.298585+0000","last_unstale":"2023-10-08T07:13:49.291605+0000","last_undegraded":"2023-10-08T07:13:49.291605+0000","last_fullsized":"2023-10-08T07:13:49.291605+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376641+0000","last_change":"2023-10-08T07:13:48.376641+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376641+0000","last_undegraded":"2023-10-08T07:13:48.376641+0000","last_fullsized":"2023-10-08T07:13:48.376641+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292035+0000","last_change":"2023-10-08T07:13:34.308624+0000","last_active":"2023-10-08T07:13:49.292035+0000","last_peered":"2023-10-08T07:13:49.292035+0000","last_clean":"2023-10-08T07:13:49.292035+0000","last_became_active":"2023-10-08T07:13:34.308246+0000","last_became_peered":"2023-10-08T07:13:34.308246+0000","last_unstale":"2023-10-08T07:13:49.292035+0000","last_undegraded":"2023-10-08T07:13:49.292035+0000","last_fullsized":"2023-10-08T07:13:49.292035+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381713+0000","last_change":"2023-10-08T07:13:48.381713+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381713+0000","last_undegraded":"2023-10-08T07:13:48.381713+0000","last_fullsized":"2023-10-08T07:13:48.381713+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.347720+0000","last_change":"2023-10-08T07:13:34.300821+0000","last_active":"2023-10-08T07:13:47.347720+0000","last_peered":"2023-10-08T07:13:47.347720+0000","last_clean":"2023-10-08T07:13:47.347720+0000","last_became_active":"2023-10-08T07:13:34.300558+0000","last_became_peered":"2023-10-08T07:13:34.300558+0000","last_unstale":"2023-10-08T07:13:47.347720+0000","last_undegraded":"2023-10-08T07:13:47.347720+0000","last_fullsized":"2023-10-08T07:13:47.347720+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.375275+0000","last_change":"2023-10-08T07:13:48.375275+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.375275+0000","last_undegraded":"2023-10-08T07:13:48.375275+0000","last_fullsized":"2023-10-08T07:13:48.375275+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.354342+0000","last_change":"2023-10-08T07:13:34.309619+0000","last_active":"2023-10-08T07:13:47.354342+0000","last_peered":"2023-10-08T07:13:47.354342+0000","last_clean":"2023-10-08T07:13:47.354342+0000","last_became_active":"2023-10-08T07:13:34.308202+0000","last_became_peered":"2023-10-08T07:13:34.308202+0000","last_unstale":"2023-10-08T07:13:47.354342+0000","last_undegraded":"2023-10-08T07:13:47.354342+0000","last_fullsized":"2023-10-08T07:13:47.354342+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:59.914799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.373371+0000","last_change":"2023-10-08T07:13:48.373371+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.373371+0000","last_undegraded":"2023-10-08T07:13:48.373371+0000","last_fullsized":"2023-10-08T07:13:48.373371+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359414+0000","last_change":"2023-10-08T07:13:34.298407+0000","last_active":"2023-10-08T07:13:47.359414+0000","last_peered":"2023-10-08T07:13:47.359414+0000","last_clean":"2023-10-08T07:13:47.359414+0000","last_became_active":"2023-10-08T07:13:34.298139+0000","last_became_peered":"2023-10-08T07:13:34.298139+0000","last_unstale":"2023-10-08T07:13:47.359414+0000","last_undegraded":"2023-10-08T07:13:47.359414+0000","last_fullsized":"2023-10-08T07:13:47.359414+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376338+0000","last_change":"2023-10-08T07:13:48.376338+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376338+0000","last_undegraded":"2023-10-08T07:13:48.376338+0000","last_fullsized":"2023-10-08T07:13:48.376338+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.351296+0000","last_change":"2023-10-08T07:13:34.306040+0000","last_active":"2023-10-08T07:13:47.351296+0000","last_peered":"2023-10-08T07:13:47.351296+0000","last_clean":"2023-10-08T07:13:47.351296+0000","last_became_active":"2023-10-08T07:13:34.303015+0000","last_became_peered":"2023-10-08T07:13:34.303015+0000","last_unstale":"2023-10-08T07:13:47.351296+0000","last_undegraded":"2023-10-08T07:13:47.351296+0000","last_fullsized":"2023-10-08T07:13:47.351296+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:42.884365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358958+0000","last_change":"2023-10-08T07:13:49.310325+0000","last_active":"2023-10-08T07:13:50.358958+0000","last_peered":"2023-10-08T07:13:50.358958+0000","last_clean":"2023-10-08T07:13:50.358958+0000","last_became_active":"2023-10-08T07:13:49.310037+0000","last_became_peered":"2023-10-08T07:13:49.310037+0000","last_unstale":"2023-10-08T07:13:50.358958+0000","last_undegraded":"2023-10-08T07:13:50.358958+0000","last_fullsized":"2023-10-08T07:13:50.358958+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:29.904246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.281819+0000","last_change":"2023-10-08T07:13:34.294253+0000","last_active":"2023-10-08T07:13:49.281819+0000","last_peered":"2023-10-08T07:13:49.281819+0000","last_clean":"2023-10-08T07:13:49.281819+0000","last_became_active":"2023-10-08T07:13:34.291997+0000","last_became_peered":"2023-10-08T07:13:34.291997+0000","last_unstale":"2023-10-08T07:13:49.281819+0000","last_undegraded":"2023-10-08T07:13:49.281819+0000","last_fullsized":"2023-10-08T07:13:49.281819+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278736+0000","last_change":"2023-10-08T07:13:49.298924+0000","last_active":"2023-10-08T07:13:50.278736+0000","last_peered":"2023-10-08T07:13:50.278736+0000","last_clean":"2023-10-08T07:13:50.278736+0000","last_became_active":"2023-10-08T07:13:49.298587+0000","last_became_peered":"2023-10-08T07:13:49.298587+0000","last_unstale":"2023-10-08T07:13:50.278736+0000","last_undegraded":"2023-10-08T07:13:50.278736+0000","last_fullsized":"2023-10-08T07:13:50.278736+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:34.367483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348653+0000","last_change":"2023-10-08T07:13:34.308930+0000","last_active":"2023-10-08T07:13:47.348653+0000","last_peered":"2023-10-08T07:13:47.348653+0000","last_clean":"2023-10-08T07:13:47.348653+0000","last_became_active":"2023-10-08T07:13:34.308283+0000","last_became_peered":"2023-10-08T07:13:34.308283+0000","last_unstale":"2023-10-08T07:13:47.348653+0000","last_undegraded":"2023-10-08T07:13:47.348653+0000","last_fullsized":"2023-10-08T07:13:47.348653+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381921+0000","last_change":"2023-10-08T07:13:48.381921+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381921+0000","last_undegraded":"2023-10-08T07:13:48.381921+0000","last_fullsized":"2023-10-08T07:13:48.381921+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359386+0000","last_change":"2023-10-08T07:13:34.295991+0000","last_active":"2023-10-08T07:13:47.359386+0000","last_peered":"2023-10-08T07:13:47.359386+0000","last_clean":"2023-10-08T07:13:47.359386+0000","last_became_active":"2023-10-08T07:13:34.295717+0000","last_became_peered":"2023-10-08T07:13:34.295717+0000","last_unstale":"2023-10-08T07:13:47.359386+0000","last_undegraded":"2023-10-08T07:13:47.359386+0000","last_fullsized":"2023-10-08T07:13:47.359386+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.797828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.368925+0000","last_change":"2023-10-08T07:13:48.368925+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.368925+0000","last_undegraded":"2023-10-08T07:13:48.368925+0000","last_fullsized":"2023-10-08T07:13:48.368925+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291913+0000","last_change":"2023-10-08T07:13:34.289375+0000","last_active":"2023-10-08T07:13:49.291913+0000","last_peered":"2023-10-08T07:13:49.291913+0000","last_clean":"2023-10-08T07:13:49.291913+0000","last_became_active":"2023-10-08T07:13:34.289115+0000","last_became_peered":"2023-10-08T07:13:34.289115+0000","last_unstale":"2023-10-08T07:13:49.291913+0000","last_undegraded":"2023-10-08T07:13:49.291913+0000","last_fullsized":"2023-10-08T07:13:49.291913+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.375367+0000","last_change":"2023-10-08T07:13:48.375367+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.375367+0000","last_undegraded":"2023-10-08T07:13:48.375367+0000","last_fullsized":"2023-10-08T07:13:48.375367+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359704+0000","last_change":"2023-10-08T07:13:34.302018+0000","last_active":"2023-10-08T07:13:47.359704+0000","last_peered":"2023-10-08T07:13:47.359704+0000","last_clean":"2023-10-08T07:13:47.359704+0000","last_became_active":"2023-10-08T07:13:34.301747+0000","last_became_peered":"2023-10-08T07:13:34.301747+0000","last_unstale":"2023-10-08T07:13:47.359704+0000","last_undegraded":"2023-10-08T07:13:47.359704+0000","last_fullsized":"2023-10-08T07:13:47.359704+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.378259+0000","last_change":"2023-10-08T07:13:48.378259+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.378259+0000","last_undegraded":"2023-10-08T07:13:48.378259+0000","last_fullsized":"2023-10-08T07:13:48.378259+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290421+0000","last_change":"2023-10-08T07:13:34.293550+0000","last_active":"2023-10-08T07:13:49.290421+0000","last_peered":"2023-10-08T07:13:49.290421+0000","last_clean":"2023-10-08T07:13:49.290421+0000","last_became_active":"2023-10-08T07:13:34.293202+0000","last_became_peered":"2023-10-08T07:13:34.293202+0000","last_unstale":"2023-10-08T07:13:49.290421+0000","last_undegraded":"2023-10-08T07:13:49.290421+0000","last_fullsized":"2023-10-08T07:13:49.290421+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:00.842346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.374742+0000","last_change":"2023-10-08T07:13:48.374742+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.374742+0000","last_undegraded":"2023-10-08T07:13:48.374742+0000","last_fullsized":"2023-10-08T07:13:48.374742+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.355316+0000","last_change":"2023-10-08T07:13:34.298110+0000","last_active":"2023-10-08T07:13:47.355316+0000","last_peered":"2023-10-08T07:13:47.355316+0000","last_clean":"2023-10-08T07:13:47.355316+0000","last_became_active":"2023-10-08T07:13:34.297806+0000","last_became_peered":"2023-10-08T07:13:34.297806+0000","last_unstale":"2023-10-08T07:13:47.355316+0000","last_undegraded":"2023-10-08T07:13:47.355316+0000","last_fullsized":"2023-10-08T07:13:47.355316+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:58.335295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357781+0000","last_change":"2023-10-08T07:13:49.315887+0000","last_active":"2023-10-08T07:13:50.357781+0000","last_peered":"2023-10-08T07:13:50.357781+0000","last_clean":"2023-10-08T07:13:50.357781+0000","last_became_active":"2023-10-08T07:13:49.315571+0000","last_became_peered":"2023-10-08T07:13:49.315571+0000","last_unstale":"2023-10-08T07:13:50.357781+0000","last_undegraded":"2023-10-08T07:13:50.357781+0000","last_fullsized":"2023-10-08T07:13:50.357781+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:31.253205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282695+0000","last_change":"2023-10-08T07:13:34.298025+0000","last_active":"2023-10-08T07:13:49.282695+0000","last_peered":"2023-10-08T07:13:49.282695+0000","last_clean":"2023-10-08T07:13:49.282695+0000","last_became_active":"2023-10-08T07:13:34.296181+0000","last_became_peered":"2023-10-08T07:13:34.296181+0000","last_unstale":"2023-10-08T07:13:49.282695+0000","last_undegraded":"2023-10-08T07:13:49.282695+0000","last_fullsized":"2023-10-08T07:13:49.282695+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448978+0000","last_change":"2023-10-08T07:13:49.300145+0000","last_active":"2023-10-08T07:13:50.448978+0000","last_peered":"2023-10-08T07:13:50.448978+0000","last_clean":"2023-10-08T07:13:50.448978+0000","last_became_active":"2023-10-08T07:13:49.299846+0000","last_became_peered":"2023-10-08T07:13:49.299846+0000","last_unstale":"2023-10-08T07:13:50.448978+0000","last_undegraded":"2023-10-08T07:13:50.448978+0000","last_fullsized":"2023-10-08T07:13:50.448978+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:16.187252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.351855+0000","last_change":"2023-10-08T07:13:34.301326+0000","last_active":"2023-10-08T07:13:47.351855+0000","last_peered":"2023-10-08T07:13:47.351855+0000","last_clean":"2023-10-08T07:13:47.351855+0000","last_became_active":"2023-10-08T07:13:34.299365+0000","last_became_peered":"2023-10-08T07:13:34.299365+0000","last_unstale":"2023-10-08T07:13:47.351855+0000","last_undegraded":"2023-10-08T07:13:47.351855+0000","last_fullsized":"2023-10-08T07:13:47.351855+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:02.576748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376508+0000","last_change":"2023-10-08T07:13:48.376508+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376508+0000","last_undegraded":"2023-10-08T07:13:48.376508+0000","last_fullsized":"2023-10-08T07:13:48.376508+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348325+0000","last_change":"2023-10-08T07:13:34.306582+0000","last_active":"2023-10-08T07:13:47.348325+0000","last_peered":"2023-10-08T07:13:47.348325+0000","last_clean":"2023-10-08T07:13:47.348325+0000","last_became_active":"2023-10-08T07:13:34.306310+0000","last_became_peered":"2023-10-08T07:13:34.306310+0000","last_unstale":"2023-10-08T07:13:47.348325+0000","last_undegraded":"2023-10-08T07:13:47.348325+0000","last_fullsized":"2023-10-08T07:13:47.348325+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.373866+0000","last_change":"2023-10-08T07:13:48.373866+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.373866+0000","last_undegraded":"2023-10-08T07:13:48.373866+0000","last_fullsized":"2023-10-08T07:13:48.373866+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298001+0000","last_change":"2023-10-08T07:13:34.295062+0000","last_active":"2023-10-08T07:13:49.298001+0000","last_peered":"2023-10-08T07:13:49.298001+0000","last_clean":"2023-10-08T07:13:49.298001+0000","last_became_active":"2023-10-08T07:13:34.294747+0000","last_became_peered":"2023-10-08T07:13:34.294747+0000","last_unstale":"2023-10-08T07:13:49.298001+0000","last_undegraded":"2023-10-08T07:13:49.298001+0000","last_fullsized":"2023-10-08T07:13:49.298001+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:26.407053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358527+0000","last_change":"2023-10-08T07:13:49.304648+0000","last_active":"2023-10-08T07:13:50.358527+0000","last_peered":"2023-10-08T07:13:50.358527+0000","last_clean":"2023-10-08T07:13:50.358527+0000","last_became_active":"2023-10-08T07:13:49.304247+0000","last_became_peered":"2023-10-08T07:13:49.304247+0000","last_unstale":"2023-10-08T07:13:50.358527+0000","last_undegraded":"2023-10-08T07:13:50.358527+0000","last_fullsized":"2023-10-08T07:13:50.358527+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:42.456757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345193+0000","last_change":"2023-10-08T07:13:34.308897+0000","last_active":"2023-10-08T07:13:47.345193+0000","last_peered":"2023-10-08T07:13:47.345193+0000","last_clean":"2023-10-08T07:13:47.345193+0000","last_became_active":"2023-10-08T07:13:34.308474+0000","last_became_peered":"2023-10-08T07:13:34.308474+0000","last_unstale":"2023-10-08T07:13:47.345193+0000","last_undegraded":"2023-10-08T07:13:47.345193+0000","last_fullsized":"2023-10-08T07:13:47.345193+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:06.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":604,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2806,"ondisk_log_size":2806,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576011,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61024,"kb_used_data":3476,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310816,"statfs":{"total":96636764160,"available":96574275584,"internally_reserved":0,"allocated":3559424,"data_stored":2124760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.169,"15min":1.2090000000000001},"min":{"1min":0.77100000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.508,"5min":1.968,"15min":2.008},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.238,"15min":1.2310000000000001},"min":{"1min":0.91300000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.6830000000000001,"5min":1.7330000000000001,"15min":1.7450000000000001},"last":1.107}]},{"osd":1,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.242,"15min":1.252},"min":{"1min":0.94399999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.665,"5min":1.79,"15min":1.998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.341,"15min":1.3500000000000001},"min":{"1min":0.99399999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.8859999999999999,"5min":1.8899999999999999,"15min":2.1019999999999999},"last":0.997}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.29,"15min":1.3140000000000001},"min":{"1min":0.97199999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6180000000000001,"5min":1.696,"15min":2.1640000000000001},"last":1.044},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.351},"min":{"1min":1.0069999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7270000000000001,"5min":1.8580000000000001,"15min":2.3170000000000002},"last":1.0660000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":1.0209999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.663,"5min":1.8160000000000001,"15min":1.9039999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.403,"5min":1.399,"15min":1.4119999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.754,"5min":1.8680000000000001,"15min":1.931},"last":1.3080000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":0.86199999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6950000000000001,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.383,"15min":1.401},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.85199999999999998},"max":{"1min":1.76,"5min":1.8140000000000001,"15min":2.0379999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4390000000000001,"15min":1.444},"min":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":2.0920000000000001,"15min":2.1139999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.4610000000000001,"15min":1.51},"min":{"1min":1.075,"5min":1,"15min":0.96899999999999997},"max":{"1min":1.8640000000000001,"5min":2.23,"15min":2.23},"last":1.2529999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.575,"15min":1.6020000000000001},"min":{"1min":1.361,"5min":1.048,"15min":1.048},"max":{"1min":2.016,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.349},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.536,"15min":1.5349999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9079999999999999,"5min":2.21,"15min":2.2570000000000001},"last":1.2889999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576010,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46568,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325272,"statfs":{"total":96636764160,"available":96589078528,"internally_reserved":0,"allocated":2977792,"data_stored":1542476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3109999999999999,"15min":1.321},"min":{"1min":0.80000000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":1.794,"15min":1.962},"last":1.637},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.347,"15min":1.3620000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7230000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.222}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.228,"15min":1.2290000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.2130000000000001,"15min":1.2430000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.97},"last":1.5529999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.198,"15min":1.238},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.484,"5min":2.339,"15min":2.339},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":1.153}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3819999999999999,"15min":1.403},"min":{"1min":1.034,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":1.9319999999999999,"15min":1.95},"last":1.468},{"interface":"front","average":{"1min":1.425,"5min":1.476,"15min":1.4990000000000001},"min":{"1min":0.97199999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8740000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.5129999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3999999999999999,"15min":1.3899999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.881,"5min":2.0579999999999998,"15min":2.0859999999999999},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4770000000000001,"15min":1.448},"min":{"1min":0.94999999999999996,"5min":0.92000000000000004,"15min":0.57099999999999995},"max":{"1min":1.8080000000000001,"5min":2.484,"15min":2.484},"last":1.603}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.399,"15min":1.423},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.81899999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.403},"last":1.369},{"interface":"front","average":{"1min":1.552,"5min":1.538,"15min":1.5509999999999999},"min":{"1min":1.2829999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.8100000000000001,"5min":2.097,"15min":2.6139999999999999},"last":1.6950000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.575},"min":{"1min":1.0489999999999999,"5min":1.012,"15min":0.96299999999999997},"max":{"1min":1.9379999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.413},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.5620000000000001,"15min":1.5600000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.665},"last":1.2709999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576010,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60436,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311404,"statfs":{"total":96636764160,"available":96574877696,"internally_reserved":0,"allocated":2957312,"data_stored":1531521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.157,"15min":1.208},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.841},"last":1.351},{"interface":"front","average":{"1min":1.341,"5min":1.2290000000000001,"15min":1.232},"min":{"1min":0.83099999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7030000000000001,"5min":1.8560000000000001,"15min":1.9610000000000001},"last":0.97999999999999998}]},{"osd":1,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2090000000000001,"15min":1.242},"min":{"1min":0.75800000000000001,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.6990000000000001,"5min":1.762,"15min":1.9550000000000001},"last":1.129},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2869999999999999,"15min":1.3},"min":{"1min":0.90900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.2130000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.871,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":2.0870000000000002},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.3149999999999999},"min":{"1min":0.77000000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5840000000000001,"5min":1.877,"15min":1.909},"last":1.008}]},{"osd":3,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":0.79800000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.1499999999999999},"last":1.294},{"interface":"front","average":{"1min":1.458,"5min":1.3740000000000001,"15min":1.431},"min":{"1min":0.67500000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8100000000000001,"5min":1.9530000000000001,"15min":2.173},"last":0.84599999999999997}]},{"osd":4,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3220000000000001,"15min":1.3280000000000001},"min":{"1min":0.84399999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7569999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.335},{"interface":"front","average":{"1min":1.556,"5min":1.446,"15min":1.4339999999999999},"min":{"1min":1.107,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.2240000000000002},"last":1.4059999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":1.0740000000000001,"5min":0.66500000000000004,"15min":0.57499999999999996},"max":{"1min":1.899,"5min":2.077,"15min":2.3759999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.478,"15min":1.4770000000000001},"min":{"1min":1.157,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0059999999999998,"5min":2.036,"15min":2.036},"last":1.264}]},{"osd":7,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.5289999999999999,"15min":1.552},"min":{"1min":1.177,"5min":0.72499999999999998,"15min":0.59999999999999998},"max":{"1min":2.298,"5min":2.298,"15min":2.4169999999999998},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.5840000000000001,"15min":1.6180000000000001},"min":{"1min":1.0089999999999999,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0009999999999999,"5min":2.2320000000000002,"15min":2.3199999999999998},"last":1.1699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576010,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47952,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323888,"statfs":{"total":96636764160,"available":96587661312,"internally_reserved":0,"allocated":2953216,"data_stored":1532047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2849999999999999,"15min":1.3440000000000001},"min":{"1min":0.64200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":2.9980000000000002},{"interface":"front","average":{"1min":1.367,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":0.62,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.9139999999999999,"5min":1.976,"15min":1.976},"last":7.0880000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2629999999999999,"15min":1.2889999999999999},"min":{"1min":0.85599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7450000000000001,"5min":1.976,"15min":1.976},"last":2.177},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.244},"min":{"1min":0.72499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7330000000000001,"5min":1.8660000000000001,"15min":1.875},"last":6.9020000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2689999999999999,"15min":1.266},"min":{"1min":0.77400000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":17.175999999999998},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.365,"15min":1.369},"min":{"1min":0.99299999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":7.1849999999999996}]},{"osd":3,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2789999999999999,"15min":1.3149999999999999},"min":{"1min":0.88,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.702,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":5.8289999999999997},{"interface":"front","average":{"1min":1.464,"5min":1.4410000000000001,"15min":1.4219999999999999},"min":{"1min":0.874,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":7.7599999999999998}]},{"osd":5,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.391},"min":{"1min":1.032,"5min":0.372,"15min":0.372},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":7.4020000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4239999999999999,"15min":1.3919999999999999},"min":{"1min":0.96799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.865,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":3.2549999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.5009999999999999,"15min":1.46},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2530000000000001,"5min":2.2719999999999998,"15min":2.6829999999999998},"last":10.542999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.593,"15min":1.5609999999999999},"min":{"1min":1.071,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.117,"5min":2.3500000000000001,"15min":2.7429999999999999},"last":10.433999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.5369999999999999,"15min":1.516},"min":{"1min":1.1220000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0649999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":3.1070000000000002},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.681,"15min":1.6439999999999999},"min":{"1min":1.3169999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.3439999999999999,"5min":2.508,"15min":2.508},"last":3.6459999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576013,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52168,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319672,"statfs":{"total":96636764160,"available":96583344128,"internally_reserved":0,"allocated":3002368,"data_stored":1554869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.67800000000000005,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.087,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.123,"5min":1.1559999999999999,"15min":1.181},"min":{"1min":0.748,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.607,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.70099999999999996}]},{"osd":1,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.417,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.2390000000000001,"15min":1.2869999999999999},"min":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4390000000000001,"5min":2.109,"15min":2.4889999999999999},"last":0.74299999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.123,"15min":1.151},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.667,"5min":2.165,"15min":2.2250000000000001},"last":0.62},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.177,"15min":1.202},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5349999999999999,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2549999999999999,"15min":1.282},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.67500000000000004},"max":{"1min":1.51,"5min":1.857,"15min":2.5379999999999998},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4179999999999999,"15min":1.46},"min":{"1min":0.93600000000000005,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.954,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":0.98599999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3200000000000001,"15min":1.3759999999999999},"min":{"1min":0.97899999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8300000000000001,"5min":1.923,"15min":2.9740000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.419,"15min":1.4339999999999999},"min":{"1min":0.77800000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8380000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4099999999999999,"15min":1.4239999999999999},"min":{"1min":0.91000000000000003,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6339999999999999,"5min":1.976,"15min":2.653},"last":1.02},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4710000000000001,"15min":1.5269999999999999},"min":{"1min":1.0960000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8540000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.0529999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5660000000000001},"min":{"1min":0.88800000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9450000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.5209999999999999},"min":{"1min":1.202,"5min":0.628,"15min":0.628},"max":{"1min":1.7889999999999999,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.2110000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576011,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":2998272,"data_stored":1555427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.97299999999999998},"min":{"1min":0.55300000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.948},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.054,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":0.745}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.96099999999999997,"15min":1.0029999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.198,"5min":1.6279999999999999,"15min":2.044},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.171,"15min":1.1970000000000001},"min":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.677,"5min":1.714,"15min":2.1309999999999998},"last":0.86799999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68400000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4770000000000001,"5min":1.78,"15min":1.8420000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.2410000000000001,"15min":1.2769999999999999},"min":{"1min":0.66200000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.458,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.165}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3959999999999999,"15min":1.4399999999999999},"min":{"1min":0.89000000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.821,"5min":1.929,"15min":2.1789999999999998},"last":1.754},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":1.0189999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6879999999999999,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.577}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3819999999999999,"15min":1.411},"min":{"1min":1.0069999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.603,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":1.083},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5009999999999999,"15min":1.5349999999999999},"min":{"1min":1.0269999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.946,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.5489999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":0.997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.73,"5min":2.056,"15min":2.1240000000000001},"last":1.472},{"interface":"front","average":{"1min":1.47,"5min":1.4950000000000001,"15min":1.534},"min":{"1min":1.1439999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.778,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":1.7190000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4690000000000001,"15min":1.486},"min":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7609999999999999,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":1.643},{"interface":"front","average":{"1min":1.5,"5min":1.5309999999999999,"15min":1.5860000000000001},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.863,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":1.623}]}]},{"osd":1,"up_from":13,"seq":55834576010,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52108,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319732,"statfs":{"total":96636764160,"available":96583405568,"internally_reserved":0,"allocated":2949120,"data_stored":1531465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.024,"15min":1.014},"min":{"1min":0.81000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.6200000000000001,"5min":1.6579999999999999,"15min":1.7769999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.224},"min":{"1min":0.88,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":1.79},"last":0.89500000000000002}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.97599999999999998},"min":{"1min":0.73399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2809999999999999,"5min":1.641,"15min":1.641},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.155,"15min":1.161},"min":{"1min":0.872,"5min":0.372,"15min":0.372},"max":{"1min":1.4710000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.95799999999999996}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.2430000000000001,"15min":1.264},"min":{"1min":1.052,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5800000000000001,"5min":1.651,"15min":1.7809999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1659999999999999,"15min":1.155},"min":{"1min":0.82599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.72,"5min":1.883,"15min":2.7429999999999999},"last":0.77200000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":1.117,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.7110000000000001,"5min":1.7829999999999999,"15min":2.8460000000000001},"last":1.022},{"interface":"front","average":{"1min":1.357,"5min":1.3129999999999999,"15min":1.3220000000000001},"min":{"1min":1.0960000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.5449999999999999,"5min":1.6879999999999999,"15min":2.0139999999999998},"last":1.3280000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.347,"15min":1.379},"min":{"1min":0.97299999999999998,"5min":0.92600000000000005,"15min":0.76700000000000002},"max":{"1min":1.6639999999999999,"5min":1.827,"15min":2.0470000000000002},"last":1.456},{"interface":"front","average":{"1min":1.637,"5min":1.546,"15min":1.534},"min":{"1min":1.286,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.2610000000000001},"last":1.667}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5089999999999999,"15min":1.5149999999999999},"min":{"1min":1.2549999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.694,"5min":2.3199999999999998,"15min":2.9079999999999999},"last":1.48},{"interface":"front","average":{"1min":1.514,"5min":1.4870000000000001,"15min":1.4990000000000001},"min":{"1min":1.1679999999999999,"5min":1.046,"15min":1.0309999999999999},"max":{"1min":1.7929999999999999,"5min":1.966,"15min":2.3199999999999998},"last":1.8180000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.5780000000000001,"15min":1.556},"min":{"1min":1.2250000000000001,"5min":0.94499999999999995,"15min":0.84599999999999997},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.786},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.5529999999999999,"15min":1.5720000000000001},"min":{"1min":1.377,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.798,"5min":2.1699999999999999,"15min":2.9590000000000001},"last":1.268}]}]},{"osd":0,"up_from":13,"seq":55834576012,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61048,"kb_used_data":3500,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310792,"statfs":{"total":96636764160,"available":96574251008,"internally_reserved":0,"allocated":3584000,"data_stored":2145179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.48299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3859999999999999,"5min":1.581,"15min":1.581},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.01,"5min":1.103,"15min":1.109},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.621,"5min":1.6619999999999999,"15min":1.726},"last":0.874}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1060000000000001,"15min":1.1419999999999999},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0329999999999999,"15min":1.073},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4199999999999999,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":0.93300000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.248,"15min":1.256},"min":{"1min":0.69799999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.546,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.579,"5min":1.752,"15min":2.0979999999999999},"last":1.4259999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3220000000000001,"15min":1.349},"min":{"1min":0.83399999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.389,"15min":1.4179999999999999},"min":{"1min":0.69699999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.782,"5min":1.972,"15min":2.181},"last":2.1520000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.423,"15min":1.417},"min":{"1min":0.79600000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.732,"5min":1.919,"15min":2.1309999999999998},"last":1.706},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.482,"15min":1.486},"min":{"1min":0.79300000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.966,"5min":2.2170000000000001,"15min":2.4900000000000002},"last":1.9930000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.81299999999999994,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7669999999999999,"5min":2.21,"15min":2.2149999999999999},"last":1.925},{"interface":"front","average":{"1min":1.397,"5min":1.506,"15min":1.5089999999999999},"min":{"1min":0.78000000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9430000000000001,"5min":2.0880000000000001,"15min":2.355},"last":1.7529999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5389999999999999,"15min":1.5469999999999999},"min":{"1min":0.83499999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9410000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.5289999999999999,"15min":1.554},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":2.0750000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:13:53.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:13:54.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.099+0000 7fe400a75640 1 Processor -- start 2023-10-08T07:13:54.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.099+0000 7fe400a75640 1 -- start start 2023-10-08T07:13:54.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe400a75640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe3fc15ab40 0x7fe3fc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe400a75640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3fc15b2b0 con 0x7fe3fc0cb580 2023-10-08T07:13:54.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe400a75640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3fc15b590 con 0x7fe3fc0c90b0 2023-10-08T07:13:54.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe400a75640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3fc15bf30 con 0x7fe3fc15ab40 2023-10-08T07:13:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3fad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe3fc0cb580 0x7fe3fc15e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47724/0 (socket says 172.21.15.70:47724) 2023-10-08T07:13:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3fad9d640 1 -- 172.21.15.70:0/3257868230 learned_addr learned my addr 172.21.15.70:0/3257868230 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3fa59c640 1 --2- 172.21.15.70:0/3257868230 >> v2:172.21.15.181:3300/0 conn(0x7fe3fc15ab40 0x7fe3fc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938008983 0 0) 0x7fe3fc15b2b0 con 0x7fe3fc0cb580 2023-10-08T07:13:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e4003520 con 0x7fe3fc0cb580 2023-10-08T07:13:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 452812428 0 0) 0x7fe3e4003520 con 0x7fe3fc0cb580 2023-10-08T07:13:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 >> v2:172.21.15.181:3300/0 conn(0x7fe3fc15ab40 msgr2=0x7fe3fc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 --2- 172.21.15.70:0/3257868230 >> v2:172.21.15.181:3300/0 conn(0x7fe3fc15ab40 0x7fe3fc0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 >> v1:172.21.15.181:6789/0 conn(0x7fe3fc0c90b0 legacy=0x7fe3fc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc15b870 con 0x7fe3fc0cb580 2023-10-08T07:13:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe3f40019f0 con 0x7fe3fc0cb580 2023-10-08T07:13:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 >> v1:172.21.15.70:6789/0 conn(0x7fe3fc0cb580 legacy=0x7fe3fc15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3e4003970 0x7fe3e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e40047c0 0x7fe3e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3e4005180 0x7fe3e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc15bf30 con 0x7fe3e4005180 2023-10-08T07:13:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e400c020 con 0x7fe3e40047c0 2023-10-08T07:13:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.101+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e400c1e0 con 0x7fe3e4003970 2023-10-08T07:13:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.102+0000 7fe3fad9d640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3e4005180 0x7fe3e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.102+0000 7fe3fa59c640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3e4003970 0x7fe3e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.102+0000 7fe3fad9d640 1 -- 172.21.15.70:0/3257868230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3e4003970 msgr2=0x7fe3e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.102+0000 7fe3fad9d640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3e4003970 0x7fe3e4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.102+0000 7fe3fad9d640 1 -- 172.21.15.70:0/3257868230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e40047c0 msgr2=0x7fe3e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.102+0000 7fe3f9d9b640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e40047c0 0x7fe3e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.102+0000 7fe3fad9d640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e40047c0 0x7fe3e4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.102+0000 7fe3fad9d640 1 -- 172.21.15.70:0/3257868230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e4003520 con 0x7fe3e4005180 2023-10-08T07:13:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.103+0000 7fe3fad9d640 1 -- 172.21.15.70:0/3257868230 reap_dead start 2023-10-08T07:13:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.103+0000 7fe3fad9d640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3e4005180 0x7fe3e400b980 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400a830 tx=0x7fe3e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15439506bacc007d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.103+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40072d0 con 0x7fe3e4005180 2023-10-08T07:13:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.103+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f402f070 con 0x7fe3e4005180 2023-10-08T07:13:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.103+0000 7fe3f959a640 1 -- 172.21.15.70:0/3257868230 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f40076b0 con 0x7fe3e4005180 2023-10-08T07:13:54.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.104+0000 7fe400a75640 1 -- 172.21.15.70:0/3257868230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3e4005180 msgr2=0x7fe3e400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.104+0000 7fe400a75640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3e4005180 0x7fe3e400b980 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400a830 tx=0x7fe3e40057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.104+0000 7fe400a75640 1 -- 172.21.15.70:0/3257868230 shutdown_connections 2023-10-08T07:13:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.104+0000 7fe400a75640 1 --2- 172.21.15.70:0/3257868230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3e4005180 0x7fe3e400b980 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.104+0000 7fe400a75640 1 -- 172.21.15.70:0/3257868230 >> 172.21.15.70:0/3257868230 conn(0x7fe3fc06a380 msgr2=0x7fe3fc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.104+0000 7fe400a75640 1 -- 172.21.15.70:0/3257868230 shutdown_connections 2023-10-08T07:13:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.104+0000 7fe400a75640 1 -- 172.21.15.70:0/3257868230 wait complete. 2023-10-08T07:13:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.106+0000 7fe400a75640 1 Processor -- start 2023-10-08T07:13:54.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.106+0000 7fe400a75640 1 -- start start 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.107+0000 7fe400a75640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e4005180 0x7fe3fc1f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.107+0000 7fe400a75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3fc0c84b0 0x7fe3fc0ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.107+0000 7fe400a75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3fc0ade60 0x7fe3fc1f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.107+0000 7fe400a75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc1f6730 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.107+0000 7fe400a75640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc1f6170 con 0x7fe3e4005180 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.107+0000 7fe400a75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc1f6450 con 0x7fe3fc0ade60 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.107+0000 7fe3f9d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3fc0c84b0 0x7fe3fc0ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.107+0000 7fe3f9d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3fc0c84b0 0x7fe3fc0ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49932/0 (socket says 172.21.15.70:49932) 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.107+0000 7fe3f9d9b640 1 -- 172.21.15.70:0/567006087 learned_addr learned my addr 172.21.15.70:0/567006087 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.108+0000 7fe3fad9d640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3fc0ade60 0x7fe3fc1f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.108+0000 7fe3fa59c640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e4005180 0x7fe3fc1f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.108+0000 7fe3f9d9b640 1 -- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3fc0ade60 msgr2=0x7fe3fc1f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.108+0000 7fe3f9d9b640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3fc0ade60 0x7fe3fc1f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.108+0000 7fe3f9d9b640 1 -- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e4005180 msgr2=0x7fe3fc1f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.108+0000 7fe3f9d9b640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e4005180 0x7fe3fc1f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.108+0000 7fe3f9d9b640 1 -- 172.21.15.70:0/567006087 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc0c3750 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.108+0000 7fe3f9d9b640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3fc0c84b0 0x7fe3fc0ad920 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000d480 tx=0x7fe3f000d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.108+0000 7fe3eaffd640 1 -- 172.21.15.70:0/567006087 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0005480 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.109+0000 7fe3eaffd640 1 -- 172.21.15.70:0/567006087 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0003610 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.109+0000 7fe3eaffd640 1 -- 172.21.15.70:0/567006087 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00039e0 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.109+0000 7fe400a75640 1 -- 172.21.15.70:0/567006087 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc0c3a30 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.109+0000 7fe400a75640 1 -- 172.21.15.70:0/567006087 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc1f5d40 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.110+0000 7fe400a75640 1 -- 172.21.15.70:0/567006087 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3fc067490 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.111+0000 7fe3eaffd640 1 -- 172.21.15.70:0/567006087 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe3f0003b40 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.112+0000 7fe3eaffd640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe3dc06af90 0x7fe3dc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.112+0000 7fe3eaffd640 1 -- 172.21.15.70:0/567006087 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fe3f0089c90 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.112+0000 7fe3fa59c640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe3dc06af90 0x7fe3dc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.113+0000 7fe3fa59c640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe3dc06af90 0x7fe3dc06d450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009a10 tx=0x7fe3ec008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:54.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.116+0000 7fe3eaffd640 1 -- 172.21.15.70:0/567006087 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3f00546b0 con 0x7fe3fc0c84b0 2023-10-08T07:13:54.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.304+0000 7fe400a75640 1 -- 172.21.15.70:0/567006087 --> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3fc1f5f60 con 0x7fe3dc06af90 2023-10-08T07:13:54.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.344+0000 7fe3eaffd640 1 -- 172.21.15.70:0/567006087 <== mgr.7328 v2:172.21.15.181:6832/3993370366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+522986 (secure 0 0 0) 0x7fe3fc1f5f60 con 0x7fe3dc06af90 2023-10-08T07:13:54.347 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:54.365 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:13:54.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.354+0000 7fe400a75640 1 -- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe3dc06af90 msgr2=0x7fe3dc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.354+0000 7fe400a75640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe3dc06af90 0x7fe3dc06d450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009a10 tx=0x7fe3ec008040 comp rx=0 tx=0).stop 2023-10-08T07:13:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.354+0000 7fe400a75640 1 -- 172.21.15.70:0/567006087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3fc0c84b0 msgr2=0x7fe3fc0ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.354+0000 7fe400a75640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3fc0c84b0 0x7fe3fc0ad920 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fe3f000d480 tx=0x7fe3f000d940 comp rx=0 tx=0).stop 2023-10-08T07:13:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.354+0000 7fe400a75640 1 -- 172.21.15.70:0/567006087 shutdown_connections 2023-10-08T07:13:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.354+0000 7fe400a75640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe3fc0ade60 0x7fe3fc1f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.354+0000 7fe400a75640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fe3dc06af90 0x7fe3dc06d450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.354+0000 7fe400a75640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3fc0c84b0 0x7fe3fc0ad920 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.354+0000 7fe400a75640 1 --2- 172.21.15.70:0/567006087 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe3e4005180 0x7fe3fc1f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.354+0000 7fe400a75640 1 -- 172.21.15.70:0/567006087 >> 172.21.15.70:0/567006087 conn(0x7fe3fc06a380 msgr2=0x7fe3fc0af820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:54.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.355+0000 7fe400a75640 1 -- 172.21.15.70:0/567006087 shutdown_connections 2023-10-08T07:13:54.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.355+0000 7fe400a75640 1 -- 172.21.15.70:0/567006087 wait complete. 2023-10-08T07:13:54.374 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-10-08T07:13:53.510178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594627,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":644,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2854,"ondisk_log_size":2854,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":275,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":433468,"kb_used_data":24396,"kb_used_omap":12,"kb_used_meta":408947,"kb_avail":754541252,"statfs":{"total":773094113280,"available":772650242048,"internally_reserved":0,"allocated":24981504,"data_stored":13517744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":418762299},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001985"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359204+0000","last_change":"2023-10-08T07:13:49.313140+0000","last_active":"2023-10-08T07:13:50.359204+0000","last_peered":"2023-10-08T07:13:50.359204+0000","last_clean":"2023-10-08T07:13:50.359204+0000","last_became_active":"2023-10-08T07:13:49.312820+0000","last_became_peered":"2023-10-08T07:13:49.312820+0000","last_unstale":"2023-10-08T07:13:50.359204+0000","last_undegraded":"2023-10-08T07:13:50.359204+0000","last_fullsized":"2023-10-08T07:13:50.359204+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:41.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"222'2806","reported_seq":3725,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.350548+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:13:47.350548+0000","last_peered":"2023-10-08T07:13:47.350548+0000","last_clean":"2023-10-08T07:13:47.350548+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:13:47.350548+0000","last_undegraded":"2023-10-08T07:13:47.350548+0000","last_fullsized":"2023-10-08T07:13:47.350548+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2806,"log_dups_size":0,"ondisk_log_size":2806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289208+0000","last_change":"2023-10-08T07:13:33.718816+0000","last_active":"2023-10-08T07:13:49.289208+0000","last_peered":"2023-10-08T07:13:49.289208+0000","last_clean":"2023-10-08T07:13:49.289208+0000","last_became_active":"2023-10-08T07:13:33.718494+0000","last_became_peered":"2023-10-08T07:13:33.718494+0000","last_unstale":"2023-10-08T07:13:49.289208+0000","last_undegraded":"2023-10-08T07:13:49.289208+0000","last_fullsized":"2023-10-08T07:13:49.289208+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:03.021625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344810+0000","last_change":"2023-10-08T07:13:34.304364+0000","last_active":"2023-10-08T07:13:47.344810+0000","last_peered":"2023-10-08T07:13:47.344810+0000","last_clean":"2023-10-08T07:13:47.344810+0000","last_became_active":"2023-10-08T07:13:34.304067+0000","last_became_peered":"2023-10-08T07:13:34.304067+0000","last_unstale":"2023-10-08T07:13:47.344810+0000","last_undegraded":"2023-10-08T07:13:47.344810+0000","last_fullsized":"2023-10-08T07:13:47.344810+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:56.084275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.385321+0000","last_change":"2023-10-08T07:13:48.383423+0000","last_active":"2023-10-08T07:13:48.385321+0000","last_peered":"2023-10-08T07:13:48.385321+0000","last_clean":"2023-10-08T07:13:48.385321+0000","last_became_active":"2023-10-08T07:13:48.383120+0000","last_became_peered":"2023-10-08T07:13:48.383120+0000","last_unstale":"2023-10-08T07:13:48.385321+0000","last_undegraded":"2023-10-08T07:13:48.385321+0000","last_fullsized":"2023-10-08T07:13:48.385321+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:30.632556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.377733+0000","last_change":"2023-10-08T07:13:48.377733+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.377733+0000","last_undegraded":"2023-10-08T07:13:48.377733+0000","last_fullsized":"2023-10-08T07:13:48.377733+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.278218+0000","last_change":"2023-10-08T07:13:33.735924+0000","last_active":"2023-10-08T07:13:49.278218+0000","last_peered":"2023-10-08T07:13:49.278218+0000","last_clean":"2023-10-08T07:13:49.278218+0000","last_became_active":"2023-10-08T07:13:33.735688+0000","last_became_peered":"2023-10-08T07:13:33.735688+0000","last_unstale":"2023-10-08T07:13:49.278218+0000","last_undegraded":"2023-10-08T07:13:49.278218+0000","last_fullsized":"2023-10-08T07:13:49.278218+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348742+0000","last_change":"2023-10-08T07:13:34.308734+0000","last_active":"2023-10-08T07:13:47.348742+0000","last_peered":"2023-10-08T07:13:47.348742+0000","last_clean":"2023-10-08T07:13:47.348742+0000","last_became_active":"2023-10-08T07:13:34.308195+0000","last_became_peered":"2023-10-08T07:13:34.308195+0000","last_unstale":"2023-10-08T07:13:47.348742+0000","last_undegraded":"2023-10-08T07:13:47.348742+0000","last_fullsized":"2023-10-08T07:13:47.348742+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.387859+0000","last_change":"2023-10-08T07:13:48.387261+0000","last_active":"2023-10-08T07:13:48.387859+0000","last_peered":"2023-10-08T07:13:48.387859+0000","last_clean":"2023-10-08T07:13:48.387859+0000","last_became_active":"2023-10-08T07:13:48.387022+0000","last_became_peered":"2023-10-08T07:13:48.387022+0000","last_unstale":"2023-10-08T07:13:48.387859+0000","last_undegraded":"2023-10-08T07:13:48.387859+0000","last_fullsized":"2023-10-08T07:13:48.387859+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:48.064204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448409+0000","last_change":"2023-10-08T07:13:49.303091+0000","last_active":"2023-10-08T07:13:50.448409+0000","last_peered":"2023-10-08T07:13:50.448409+0000","last_clean":"2023-10-08T07:13:50.448409+0000","last_became_active":"2023-10-08T07:13:49.302761+0000","last_became_peered":"2023-10-08T07:13:49.302761+0000","last_unstale":"2023-10-08T07:13:50.448409+0000","last_undegraded":"2023-10-08T07:13:50.448409+0000","last_fullsized":"2023-10-08T07:13:50.448409+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:54.844015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344232+0000","last_change":"2023-10-08T07:13:33.741115+0000","last_active":"2023-10-08T07:13:47.344232+0000","last_peered":"2023-10-08T07:13:47.344232+0000","last_clean":"2023-10-08T07:13:47.344232+0000","last_became_active":"2023-10-08T07:13:33.740798+0000","last_became_peered":"2023-10-08T07:13:33.740798+0000","last_unstale":"2023-10-08T07:13:47.344232+0000","last_undegraded":"2023-10-08T07:13:47.344232+0000","last_fullsized":"2023-10-08T07:13:47.344232+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:36.728485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.352692+0000","last_change":"2023-10-08T07:13:34.303009+0000","last_active":"2023-10-08T07:13:47.352692+0000","last_peered":"2023-10-08T07:13:47.352692+0000","last_clean":"2023-10-08T07:13:47.352692+0000","last_became_active":"2023-10-08T07:13:34.302682+0000","last_became_peered":"2023-10-08T07:13:34.302682+0000","last_unstale":"2023-10-08T07:13:47.352692+0000","last_undegraded":"2023-10-08T07:13:47.352692+0000","last_fullsized":"2023-10-08T07:13:47.352692+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:47:40.976539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.393834+0000","last_change":"2023-10-08T07:13:48.388490+0000","last_active":"2023-10-08T07:13:48.393834+0000","last_peered":"2023-10-08T07:13:48.393834+0000","last_clean":"2023-10-08T07:13:48.393834+0000","last_became_active":"2023-10-08T07:13:48.388178+0000","last_became_peered":"2023-10-08T07:13:48.388178+0000","last_unstale":"2023-10-08T07:13:48.393834+0000","last_undegraded":"2023-10-08T07:13:48.393834+0000","last_fullsized":"2023-10-08T07:13:48.393834+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:18.667320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278865+0000","last_change":"2023-10-08T07:13:49.305322+0000","last_active":"2023-10-08T07:13:50.278865+0000","last_peered":"2023-10-08T07:13:50.278865+0000","last_clean":"2023-10-08T07:13:50.278865+0000","last_became_active":"2023-10-08T07:13:49.305047+0000","last_became_peered":"2023-10-08T07:13:49.305047+0000","last_unstale":"2023-10-08T07:13:50.278865+0000","last_undegraded":"2023-10-08T07:13:50.278865+0000","last_fullsized":"2023-10-08T07:13:50.278865+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:20.461769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289335+0000","last_change":"2023-10-08T07:13:33.738792+0000","last_active":"2023-10-08T07:13:49.289335+0000","last_peered":"2023-10-08T07:13:49.289335+0000","last_clean":"2023-10-08T07:13:49.289335+0000","last_became_active":"2023-10-08T07:13:33.738430+0000","last_became_peered":"2023-10-08T07:13:33.738430+0000","last_unstale":"2023-10-08T07:13:49.289335+0000","last_undegraded":"2023-10-08T07:13:49.289335+0000","last_fullsized":"2023-10-08T07:13:49.289335+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:24:39.688018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.353511+0000","last_change":"2023-10-08T07:13:34.305875+0000","last_active":"2023-10-08T07:13:47.353511+0000","last_peered":"2023-10-08T07:13:47.353511+0000","last_clean":"2023-10-08T07:13:47.353511+0000","last_became_active":"2023-10-08T07:13:34.302605+0000","last_became_peered":"2023-10-08T07:13:34.302605+0000","last_unstale":"2023-10-08T07:13:47.353511+0000","last_undegraded":"2023-10-08T07:13:47.353511+0000","last_fullsized":"2023-10-08T07:13:47.353511+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:22.929719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396356+0000","last_change":"2023-10-08T07:13:48.391812+0000","last_active":"2023-10-08T07:13:48.396356+0000","last_peered":"2023-10-08T07:13:48.396356+0000","last_clean":"2023-10-08T07:13:48.396356+0000","last_became_active":"2023-10-08T07:13:48.391532+0000","last_became_peered":"2023-10-08T07:13:48.391532+0000","last_unstale":"2023-10-08T07:13:48.396356+0000","last_undegraded":"2023-10-08T07:13:48.396356+0000","last_fullsized":"2023-10-08T07:13:48.396356+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:39:10.917417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.372366+0000","last_change":"2023-10-08T07:13:48.372366+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.372366+0000","last_undegraded":"2023-10-08T07:13:48.372366+0000","last_fullsized":"2023-10-08T07:13:48.372366+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.361372+0000","last_change":"2023-10-08T07:13:33.734220+0000","last_active":"2023-10-08T07:13:47.361372+0000","last_peered":"2023-10-08T07:13:47.361372+0000","last_clean":"2023-10-08T07:13:47.361372+0000","last_became_active":"2023-10-08T07:13:33.733905+0000","last_became_peered":"2023-10-08T07:13:33.733905+0000","last_unstale":"2023-10-08T07:13:47.361372+0000","last_undegraded":"2023-10-08T07:13:47.361372+0000","last_fullsized":"2023-10-08T07:13:47.361372+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359420+0000","last_change":"2023-10-08T07:13:34.302227+0000","last_active":"2023-10-08T07:13:47.359420+0000","last_peered":"2023-10-08T07:13:47.359420+0000","last_clean":"2023-10-08T07:13:47.359420+0000","last_became_active":"2023-10-08T07:13:34.300127+0000","last_became_peered":"2023-10-08T07:13:34.300127+0000","last_unstale":"2023-10-08T07:13:47.359420+0000","last_undegraded":"2023-10-08T07:13:47.359420+0000","last_fullsized":"2023-10-08T07:13:47.359420+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:49.729707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.382534+0000","last_change":"2023-10-08T07:13:48.380775+0000","last_active":"2023-10-08T07:13:48.382534+0000","last_peered":"2023-10-08T07:13:48.382534+0000","last_clean":"2023-10-08T07:13:48.382534+0000","last_became_active":"2023-10-08T07:13:48.380501+0000","last_became_peered":"2023-10-08T07:13:48.380501+0000","last_unstale":"2023-10-08T07:13:48.382534+0000","last_undegraded":"2023-10-08T07:13:48.382534+0000","last_fullsized":"2023-10-08T07:13:48.382534+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:07.670172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357767+0000","last_change":"2023-10-08T07:13:49.309072+0000","last_active":"2023-10-08T07:13:50.357767+0000","last_peered":"2023-10-08T07:13:50.357767+0000","last_clean":"2023-10-08T07:13:50.357767+0000","last_became_active":"2023-10-08T07:13:49.308753+0000","last_became_peered":"2023-10-08T07:13:49.308753+0000","last_unstale":"2023-10-08T07:13:50.357767+0000","last_undegraded":"2023-10-08T07:13:50.357767+0000","last_fullsized":"2023-10-08T07:13:50.357767+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:27.798388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344852+0000","last_change":"2023-10-08T07:13:33.733207+0000","last_active":"2023-10-08T07:13:47.344852+0000","last_peered":"2023-10-08T07:13:47.344852+0000","last_clean":"2023-10-08T07:13:47.344852+0000","last_became_active":"2023-10-08T07:13:33.732903+0000","last_became_peered":"2023-10-08T07:13:33.732903+0000","last_unstale":"2023-10-08T07:13:47.344852+0000","last_undegraded":"2023-10-08T07:13:47.344852+0000","last_fullsized":"2023-10-08T07:13:47.344852+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:34.358428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343713+0000","last_change":"2023-10-08T07:13:34.297602+0000","last_active":"2023-10-08T07:13:47.343713+0000","last_peered":"2023-10-08T07:13:47.343713+0000","last_clean":"2023-10-08T07:13:47.343713+0000","last_became_active":"2023-10-08T07:13:34.296820+0000","last_became_peered":"2023-10-08T07:13:34.296820+0000","last_unstale":"2023-10-08T07:13:47.343713+0000","last_undegraded":"2023-10-08T07:13:47.343713+0000","last_fullsized":"2023-10-08T07:13:47.343713+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:07.812354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.380096+0000","last_change":"2023-10-08T07:13:48.374204+0000","last_active":"2023-10-08T07:13:48.380096+0000","last_peered":"2023-10-08T07:13:48.380096+0000","last_clean":"2023-10-08T07:13:48.380096+0000","last_became_active":"2023-10-08T07:13:48.373126+0000","last_became_peered":"2023-10-08T07:13:48.373126+0000","last_unstale":"2023-10-08T07:13:48.380096+0000","last_undegraded":"2023-10-08T07:13:48.380096+0000","last_fullsized":"2023-10-08T07:13:48.380096+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:18.667320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358171+0000","last_change":"2023-10-08T07:13:49.314030+0000","last_active":"2023-10-08T07:13:50.358171+0000","last_peered":"2023-10-08T07:13:50.358171+0000","last_clean":"2023-10-08T07:13:50.358171+0000","last_became_active":"2023-10-08T07:13:49.312711+0000","last_became_peered":"2023-10-08T07:13:49.312711+0000","last_unstale":"2023-10-08T07:13:50.358171+0000","last_undegraded":"2023-10-08T07:13:50.358171+0000","last_fullsized":"2023-10-08T07:13:50.358171+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:39.025998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.356696+0000","last_change":"2023-10-08T07:13:33.737211+0000","last_active":"2023-10-08T07:13:47.356696+0000","last_peered":"2023-10-08T07:13:47.356696+0000","last_clean":"2023-10-08T07:13:47.356696+0000","last_became_active":"2023-10-08T07:13:33.736904+0000","last_became_peered":"2023-10-08T07:13:33.736904+0000","last_unstale":"2023-10-08T07:13:47.356696+0000","last_undegraded":"2023-10-08T07:13:47.356696+0000","last_fullsized":"2023-10-08T07:13:47.356696+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:58.966472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288985+0000","last_change":"2023-10-08T07:13:34.302678+0000","last_active":"2023-10-08T07:13:49.288985+0000","last_peered":"2023-10-08T07:13:49.288985+0000","last_clean":"2023-10-08T07:13:49.288985+0000","last_became_active":"2023-10-08T07:13:34.302406+0000","last_became_peered":"2023-10-08T07:13:34.302406+0000","last_unstale":"2023-10-08T07:13:49.288985+0000","last_undegraded":"2023-10-08T07:13:49.288985+0000","last_fullsized":"2023-10-08T07:13:49.288985+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:48.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396347+0000","last_change":"2023-10-08T07:13:48.389369+0000","last_active":"2023-10-08T07:13:48.396347+0000","last_peered":"2023-10-08T07:13:48.396347+0000","last_clean":"2023-10-08T07:13:48.396347+0000","last_became_active":"2023-10-08T07:13:48.389067+0000","last_became_peered":"2023-10-08T07:13:48.389067+0000","last_unstale":"2023-10-08T07:13:48.396347+0000","last_undegraded":"2023-10-08T07:13:48.396347+0000","last_fullsized":"2023-10-08T07:13:48.396347+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:18.246155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.377624+0000","last_change":"2023-10-08T07:13:48.377624+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.377624+0000","last_undegraded":"2023-10-08T07:13:48.377624+0000","last_fullsized":"2023-10-08T07:13:48.377624+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"234'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359654+0000","last_change":"2023-10-08T07:13:33.742793+0000","last_active":"2023-10-08T07:13:47.359654+0000","last_peered":"2023-10-08T07:13:47.359654+0000","last_clean":"2023-10-08T07:13:47.359654+0000","last_became_active":"2023-10-08T07:13:33.742030+0000","last_became_peered":"2023-10-08T07:13:33.742030+0000","last_unstale":"2023-10-08T07:13:47.359654+0000","last_undegraded":"2023-10-08T07:13:47.359654+0000","last_fullsized":"2023-10-08T07:13:47.359654+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:25:59.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348010+0000","last_change":"2023-10-08T07:13:34.308581+0000","last_active":"2023-10-08T07:13:47.348010+0000","last_peered":"2023-10-08T07:13:47.348010+0000","last_clean":"2023-10-08T07:13:47.348010+0000","last_became_active":"2023-10-08T07:13:34.308210+0000","last_became_peered":"2023-10-08T07:13:34.308210+0000","last_unstale":"2023-10-08T07:13:47.348010+0000","last_undegraded":"2023-10-08T07:13:47.348010+0000","last_fullsized":"2023-10-08T07:13:47.348010+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396466+0000","last_change":"2023-10-08T07:13:48.391100+0000","last_active":"2023-10-08T07:13:48.396466+0000","last_peered":"2023-10-08T07:13:48.396466+0000","last_clean":"2023-10-08T07:13:48.396466+0000","last_became_active":"2023-10-08T07:13:48.390787+0000","last_became_peered":"2023-10-08T07:13:48.390787+0000","last_unstale":"2023-10-08T07:13:48.396466+0000","last_undegraded":"2023-10-08T07:13:48.396466+0000","last_fullsized":"2023-10-08T07:13:48.396466+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:13:47.089135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278157+0000","last_change":"2023-10-08T07:13:49.310297+0000","last_active":"2023-10-08T07:13:50.278157+0000","last_peered":"2023-10-08T07:13:50.278157+0000","last_clean":"2023-10-08T07:13:50.278157+0000","last_became_active":"2023-10-08T07:13:49.310021+0000","last_became_peered":"2023-10-08T07:13:49.310021+0000","last_unstale":"2023-10-08T07:13:50.278157+0000","last_undegraded":"2023-10-08T07:13:50.278157+0000","last_fullsized":"2023-10-08T07:13:50.278157+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:19.685263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292199+0000","last_change":"2023-10-08T07:13:33.739638+0000","last_active":"2023-10-08T07:13:49.292199+0000","last_peered":"2023-10-08T07:13:49.292199+0000","last_clean":"2023-10-08T07:13:49.292199+0000","last_became_active":"2023-10-08T07:13:33.739352+0000","last_became_peered":"2023-10-08T07:13:33.739352+0000","last_unstale":"2023-10-08T07:13:49.292199+0000","last_undegraded":"2023-10-08T07:13:49.292199+0000","last_fullsized":"2023-10-08T07:13:49.292199+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:11.117318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359899+0000","last_change":"2023-10-08T07:13:34.293674+0000","last_active":"2023-10-08T07:13:47.359899+0000","last_peered":"2023-10-08T07:13:47.359899+0000","last_clean":"2023-10-08T07:13:47.359899+0000","last_became_active":"2023-10-08T07:13:34.292983+0000","last_became_peered":"2023-10-08T07:13:34.292983+0000","last_unstale":"2023-10-08T07:13:47.359899+0000","last_undegraded":"2023-10-08T07:13:47.359899+0000","last_fullsized":"2023-10-08T07:13:47.359899+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:20.477902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.389923+0000","last_change":"2023-10-08T07:13:48.388173+0000","last_active":"2023-10-08T07:13:48.389923+0000","last_peered":"2023-10-08T07:13:48.389923+0000","last_clean":"2023-10-08T07:13:48.389923+0000","last_became_active":"2023-10-08T07:13:48.387881+0000","last_became_peered":"2023-10-08T07:13:48.387881+0000","last_unstale":"2023-10-08T07:13:48.389923+0000","last_undegraded":"2023-10-08T07:13:48.389923+0000","last_fullsized":"2023-10-08T07:13:48.389923+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:43.583782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.379456+0000","last_change":"2023-10-08T07:13:48.379456+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.379456+0000","last_undegraded":"2023-10-08T07:13:48.379456+0000","last_fullsized":"2023-10-08T07:13:48.379456+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.342892+0000","last_change":"2023-10-08T07:13:33.739626+0000","last_active":"2023-10-08T07:13:47.342892+0000","last_peered":"2023-10-08T07:13:47.342892+0000","last_clean":"2023-10-08T07:13:47.342892+0000","last_became_active":"2023-10-08T07:13:33.739348+0000","last_became_peered":"2023-10-08T07:13:33.739348+0000","last_unstale":"2023-10-08T07:13:47.342892+0000","last_undegraded":"2023-10-08T07:13:47.342892+0000","last_fullsized":"2023-10-08T07:13:47.342892+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:23.374596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.360313+0000","last_change":"2023-10-08T07:13:34.300038+0000","last_active":"2023-10-08T07:13:47.360313+0000","last_peered":"2023-10-08T07:13:47.360313+0000","last_clean":"2023-10-08T07:13:47.360313+0000","last_became_active":"2023-10-08T07:13:34.299721+0000","last_became_peered":"2023-10-08T07:13:34.299721+0000","last_unstale":"2023-10-08T07:13:47.360313+0000","last_undegraded":"2023-10-08T07:13:47.360313+0000","last_fullsized":"2023-10-08T07:13:47.360313+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:26.317918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.366615+0000","last_change":"2023-10-08T07:13:48.384247+0000","last_active":"2023-10-08T07:13:49.366615+0000","last_peered":"2023-10-08T07:13:49.366615+0000","last_clean":"2023-10-08T07:13:49.366615+0000","last_became_active":"2023-10-08T07:13:48.383959+0000","last_became_peered":"2023-10-08T07:13:48.383959+0000","last_unstale":"2023-10-08T07:13:49.366615+0000","last_undegraded":"2023-10-08T07:13:49.366615+0000","last_fullsized":"2023-10-08T07:13:49.366615+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:13.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448607+0000","last_change":"2023-10-08T07:13:49.319375+0000","last_active":"2023-10-08T07:13:50.448607+0000","last_peered":"2023-10-08T07:13:50.448607+0000","last_clean":"2023-10-08T07:13:50.448607+0000","last_became_active":"2023-10-08T07:13:49.319046+0000","last_became_peered":"2023-10-08T07:13:49.319046+0000","last_unstale":"2023-10-08T07:13:50.448607+0000","last_undegraded":"2023-10-08T07:13:50.448607+0000","last_fullsized":"2023-10-08T07:13:50.448607+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:43.401179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.354842+0000","last_change":"2023-10-08T07:13:33.742955+0000","last_active":"2023-10-08T07:13:47.354842+0000","last_peered":"2023-10-08T07:13:47.354842+0000","last_clean":"2023-10-08T07:13:47.354842+0000","last_became_active":"2023-10-08T07:13:33.742621+0000","last_became_peered":"2023-10-08T07:13:33.742621+0000","last_unstale":"2023-10-08T07:13:47.354842+0000","last_undegraded":"2023-10-08T07:13:47.354842+0000","last_fullsized":"2023-10-08T07:13:47.354842+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:25.311911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291806+0000","last_change":"2023-10-08T07:13:34.294206+0000","last_active":"2023-10-08T07:13:49.291806+0000","last_peered":"2023-10-08T07:13:49.291806+0000","last_clean":"2023-10-08T07:13:49.291806+0000","last_became_active":"2023-10-08T07:13:34.293897+0000","last_became_peered":"2023-10-08T07:13:34.293897+0000","last_unstale":"2023-10-08T07:13:49.291806+0000","last_undegraded":"2023-10-08T07:13:49.291806+0000","last_fullsized":"2023-10-08T07:13:49.291806+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370513+0000","last_change":"2023-10-08T07:13:48.392407+0000","last_active":"2023-10-08T07:13:49.370513+0000","last_peered":"2023-10-08T07:13:49.370513+0000","last_clean":"2023-10-08T07:13:49.370513+0000","last_became_active":"2023-10-08T07:13:48.392147+0000","last_became_peered":"2023-10-08T07:13:48.392147+0000","last_unstale":"2023-10-08T07:13:49.370513+0000","last_undegraded":"2023-10-08T07:13:49.370513+0000","last_fullsized":"2023-10-08T07:13:49.370513+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:54:27.098972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376428+0000","last_change":"2023-10-08T07:13:48.376428+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376428+0000","last_undegraded":"2023-10-08T07:13:48.376428+0000","last_fullsized":"2023-10-08T07:13:48.376428+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295620+0000","last_change":"2023-10-08T07:13:33.726092+0000","last_active":"2023-10-08T07:13:49.295620+0000","last_peered":"2023-10-08T07:13:49.295620+0000","last_clean":"2023-10-08T07:13:49.295620+0000","last_became_active":"2023-10-08T07:13:33.725701+0000","last_became_peered":"2023-10-08T07:13:33.725701+0000","last_unstale":"2023-10-08T07:13:49.295620+0000","last_undegraded":"2023-10-08T07:13:49.295620+0000","last_fullsized":"2023-10-08T07:13:49.295620+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:51.631016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.353306+0000","last_change":"2023-10-08T07:13:34.301017+0000","last_active":"2023-10-08T07:13:47.353306+0000","last_peered":"2023-10-08T07:13:47.353306+0000","last_clean":"2023-10-08T07:13:47.353306+0000","last_became_active":"2023-10-08T07:13:34.300720+0000","last_became_peered":"2023-10-08T07:13:34.300720+0000","last_unstale":"2023-10-08T07:13:47.353306+0000","last_undegraded":"2023-10-08T07:13:47.353306+0000","last_fullsized":"2023-10-08T07:13:47.353306+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:36:05.460057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.395613+0000","last_change":"2023-10-08T07:13:48.389158+0000","last_active":"2023-10-08T07:13:48.395613+0000","last_peered":"2023-10-08T07:13:48.395613+0000","last_clean":"2023-10-08T07:13:48.395613+0000","last_became_active":"2023-10-08T07:13:48.388809+0000","last_became_peered":"2023-10-08T07:13:48.388809+0000","last_unstale":"2023-10-08T07:13:48.395613+0000","last_undegraded":"2023-10-08T07:13:48.395613+0000","last_fullsized":"2023-10-08T07:13:48.395613+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:30.625357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359126+0000","last_change":"2023-10-08T07:13:49.310553+0000","last_active":"2023-10-08T07:13:50.359126+0000","last_peered":"2023-10-08T07:13:50.359126+0000","last_clean":"2023-10-08T07:13:50.359126+0000","last_became_active":"2023-10-08T07:13:49.310222+0000","last_became_peered":"2023-10-08T07:13:49.310222+0000","last_unstale":"2023-10-08T07:13:50.359126+0000","last_undegraded":"2023-10-08T07:13:50.359126+0000","last_fullsized":"2023-10-08T07:13:50.359126+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:50.676315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.283072+0000","last_change":"2023-10-08T07:13:33.726781+0000","last_active":"2023-10-08T07:13:49.283072+0000","last_peered":"2023-10-08T07:13:49.283072+0000","last_clean":"2023-10-08T07:13:49.283072+0000","last_became_active":"2023-10-08T07:13:33.726459+0000","last_became_peered":"2023-10-08T07:13:33.726459+0000","last_unstale":"2023-10-08T07:13:49.283072+0000","last_undegraded":"2023-10-08T07:13:49.283072+0000","last_fullsized":"2023-10-08T07:13:49.283072+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:04.357904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295826+0000","last_change":"2023-10-08T07:13:34.294315+0000","last_active":"2023-10-08T07:13:49.295826+0000","last_peered":"2023-10-08T07:13:49.295826+0000","last_clean":"2023-10-08T07:13:49.295826+0000","last_became_active":"2023-10-08T07:13:34.294065+0000","last_became_peered":"2023-10-08T07:13:34.294065+0000","last_unstale":"2023-10-08T07:13:49.295826+0000","last_undegraded":"2023-10-08T07:13:49.295826+0000","last_fullsized":"2023-10-08T07:13:49.295826+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:37.124047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396254+0000","last_change":"2023-10-08T07:13:48.389005+0000","last_active":"2023-10-08T07:13:48.396254+0000","last_peered":"2023-10-08T07:13:48.396254+0000","last_clean":"2023-10-08T07:13:48.396254+0000","last_became_active":"2023-10-08T07:13:48.388729+0000","last_became_peered":"2023-10-08T07:13:48.388729+0000","last_unstale":"2023-10-08T07:13:48.396254+0000","last_undegraded":"2023-10-08T07:13:48.396254+0000","last_fullsized":"2023-10-08T07:13:48.396254+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:13.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.373866+0000","last_change":"2023-10-08T07:13:48.373866+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.373866+0000","last_undegraded":"2023-10-08T07:13:48.373866+0000","last_fullsized":"2023-10-08T07:13:48.373866+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282248+0000","last_change":"2023-10-08T07:13:33.729567+0000","last_active":"2023-10-08T07:13:49.282248+0000","last_peered":"2023-10-08T07:13:49.282248+0000","last_clean":"2023-10-08T07:13:49.282248+0000","last_became_active":"2023-10-08T07:13:33.729125+0000","last_became_peered":"2023-10-08T07:13:33.729125+0000","last_unstale":"2023-10-08T07:13:49.282248+0000","last_undegraded":"2023-10-08T07:13:49.282248+0000","last_fullsized":"2023-10-08T07:13:49.282248+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:13.193093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.361253+0000","last_change":"2023-10-08T07:13:34.308836+0000","last_active":"2023-10-08T07:13:47.361253+0000","last_peered":"2023-10-08T07:13:47.361253+0000","last_clean":"2023-10-08T07:13:47.361253+0000","last_became_active":"2023-10-08T07:13:34.308301+0000","last_became_peered":"2023-10-08T07:13:34.308301+0000","last_unstale":"2023-10-08T07:13:47.361253+0000","last_undegraded":"2023-10-08T07:13:47.361253+0000","last_fullsized":"2023-10-08T07:13:47.361253+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:42.058240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.391244+0000","last_change":"2023-10-08T07:13:48.390712+0000","last_active":"2023-10-08T07:13:48.391244+0000","last_peered":"2023-10-08T07:13:48.391244+0000","last_clean":"2023-10-08T07:13:48.391244+0000","last_became_active":"2023-10-08T07:13:48.390349+0000","last_became_peered":"2023-10-08T07:13:48.390349+0000","last_unstale":"2023-10-08T07:13:48.391244+0000","last_undegraded":"2023-10-08T07:13:48.391244+0000","last_fullsized":"2023-10-08T07:13:48.391244+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:08.228559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.367853+0000","last_change":"2023-10-08T07:13:48.367853+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.367853+0000","last_undegraded":"2023-10-08T07:13:48.367853+0000","last_fullsized":"2023-10-08T07:13:48.367853+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295073+0000","last_change":"2023-10-08T07:13:33.732139+0000","last_active":"2023-10-08T07:13:49.295073+0000","last_peered":"2023-10-08T07:13:49.295073+0000","last_clean":"2023-10-08T07:13:49.295073+0000","last_became_active":"2023-10-08T07:13:33.731892+0000","last_became_peered":"2023-10-08T07:13:33.731892+0000","last_unstale":"2023-10-08T07:13:49.295073+0000","last_undegraded":"2023-10-08T07:13:49.295073+0000","last_fullsized":"2023-10-08T07:13:49.295073+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:07.242491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343742+0000","last_change":"2023-10-08T07:13:34.299071+0000","last_active":"2023-10-08T07:13:47.343742+0000","last_peered":"2023-10-08T07:13:47.343742+0000","last_clean":"2023-10-08T07:13:47.343742+0000","last_became_active":"2023-10-08T07:13:34.298776+0000","last_became_peered":"2023-10-08T07:13:34.298776+0000","last_unstale":"2023-10-08T07:13:47.343742+0000","last_undegraded":"2023-10-08T07:13:47.343742+0000","last_fullsized":"2023-10-08T07:13:47.343742+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:10.642317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398369+0000","last_change":"2023-10-08T07:13:48.393982+0000","last_active":"2023-10-08T07:13:48.398369+0000","last_peered":"2023-10-08T07:13:48.398369+0000","last_clean":"2023-10-08T07:13:48.398369+0000","last_became_active":"2023-10-08T07:13:48.393696+0000","last_became_peered":"2023-10-08T07:13:48.393696+0000","last_unstale":"2023-10-08T07:13:48.398369+0000","last_undegraded":"2023-10-08T07:13:48.398369+0000","last_fullsized":"2023-10-08T07:13:48.398369+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:10.381086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358215+0000","last_change":"2023-10-08T07:13:49.312996+0000","last_active":"2023-10-08T07:13:50.358215+0000","last_peered":"2023-10-08T07:13:50.358215+0000","last_clean":"2023-10-08T07:13:50.358215+0000","last_became_active":"2023-10-08T07:13:49.312657+0000","last_became_peered":"2023-10-08T07:13:49.312657+0000","last_unstale":"2023-10-08T07:13:50.358215+0000","last_undegraded":"2023-10-08T07:13:50.358215+0000","last_fullsized":"2023-10-08T07:13:50.358215+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:28.367357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"233'2","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345211+0000","last_change":"2023-10-08T07:13:33.736960+0000","last_active":"2023-10-08T07:13:47.345211+0000","last_peered":"2023-10-08T07:13:47.345211+0000","last_clean":"2023-10-08T07:13:47.345211+0000","last_became_active":"2023-10-08T07:13:33.736703+0000","last_became_peered":"2023-10-08T07:13:33.736703+0000","last_unstale":"2023-10-08T07:13:47.345211+0000","last_undegraded":"2023-10-08T07:13:47.345211+0000","last_fullsized":"2023-10-08T07:13:47.345211+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:39.196152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343947+0000","last_change":"2023-10-08T07:13:34.291716+0000","last_active":"2023-10-08T07:13:47.343947+0000","last_peered":"2023-10-08T07:13:47.343947+0000","last_clean":"2023-10-08T07:13:47.343947+0000","last_became_active":"2023-10-08T07:13:34.291300+0000","last_became_peered":"2023-10-08T07:13:34.291300+0000","last_unstale":"2023-10-08T07:13:47.343947+0000","last_undegraded":"2023-10-08T07:13:47.343947+0000","last_fullsized":"2023-10-08T07:13:47.343947+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:07.189339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.277990+0000","last_change":"2023-10-08T07:13:48.383304+0000","last_active":"2023-10-08T07:13:49.277990+0000","last_peered":"2023-10-08T07:13:49.277990+0000","last_clean":"2023-10-08T07:13:49.277990+0000","last_became_active":"2023-10-08T07:13:48.383040+0000","last_became_peered":"2023-10-08T07:13:48.383040+0000","last_unstale":"2023-10-08T07:13:49.277990+0000","last_undegraded":"2023-10-08T07:13:49.277990+0000","last_fullsized":"2023-10-08T07:13:49.277990+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:05.475908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448022+0000","last_change":"2023-10-08T07:13:49.319374+0000","last_active":"2023-10-08T07:13:50.448022+0000","last_peered":"2023-10-08T07:13:50.448022+0000","last_clean":"2023-10-08T07:13:50.448022+0000","last_became_active":"2023-10-08T07:13:49.319049+0000","last_became_peered":"2023-10-08T07:13:49.319049+0000","last_unstale":"2023-10-08T07:13:50.448022+0000","last_undegraded":"2023-10-08T07:13:50.448022+0000","last_fullsized":"2023-10-08T07:13:50.448022+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:09.611966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345489+0000","last_change":"2023-10-08T07:13:33.728187+0000","last_active":"2023-10-08T07:13:47.345489+0000","last_peered":"2023-10-08T07:13:47.345489+0000","last_clean":"2023-10-08T07:13:47.345489+0000","last_became_active":"2023-10-08T07:13:33.727828+0000","last_became_peered":"2023-10-08T07:13:33.727828+0000","last_unstale":"2023-10-08T07:13:47.345489+0000","last_undegraded":"2023-10-08T07:13:47.345489+0000","last_fullsized":"2023-10-08T07:13:47.345489+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:00.539389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.360575+0000","last_change":"2023-10-08T07:13:34.293870+0000","last_active":"2023-10-08T07:13:47.360575+0000","last_peered":"2023-10-08T07:13:47.360575+0000","last_clean":"2023-10-08T07:13:47.360575+0000","last_became_active":"2023-10-08T07:13:34.293397+0000","last_became_peered":"2023-10-08T07:13:34.293397+0000","last_unstale":"2023-10-08T07:13:47.360575+0000","last_undegraded":"2023-10-08T07:13:47.360575+0000","last_fullsized":"2023-10-08T07:13:47.360575+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.387322+0000","last_change":"2023-10-08T07:13:48.385714+0000","last_active":"2023-10-08T07:13:48.387322+0000","last_peered":"2023-10-08T07:13:48.387322+0000","last_clean":"2023-10-08T07:13:48.387322+0000","last_became_active":"2023-10-08T07:13:48.385420+0000","last_became_peered":"2023-10-08T07:13:48.385420+0000","last_unstale":"2023-10-08T07:13:48.387322+0000","last_undegraded":"2023-10-08T07:13:48.387322+0000","last_fullsized":"2023-10-08T07:13:48.387322+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:51.558634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.378285+0000","last_change":"2023-10-08T07:13:48.378285+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.378285+0000","last_undegraded":"2023-10-08T07:13:48.378285+0000","last_fullsized":"2023-10-08T07:13:48.378285+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"233'8","reported_seq":24,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.278141+0000","last_change":"2023-10-08T07:13:33.738121+0000","last_active":"2023-10-08T07:13:49.278141+0000","last_peered":"2023-10-08T07:13:49.278141+0000","last_clean":"2023-10-08T07:13:49.278141+0000","last_became_active":"2023-10-08T07:13:33.737903+0000","last_became_peered":"2023-10-08T07:13:33.737903+0000","last_unstale":"2023-10-08T07:13:49.278141+0000","last_undegraded":"2023-10-08T07:13:49.278141+0000","last_fullsized":"2023-10-08T07:13:49.278141+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:15.275258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.355390+0000","last_change":"2023-10-08T07:13:34.306066+0000","last_active":"2023-10-08T07:13:47.355390+0000","last_peered":"2023-10-08T07:13:47.355390+0000","last_clean":"2023-10-08T07:13:47.355390+0000","last_became_active":"2023-10-08T07:13:34.302955+0000","last_became_peered":"2023-10-08T07:13:34.302955+0000","last_unstale":"2023-10-08T07:13:47.355390+0000","last_undegraded":"2023-10-08T07:13:47.355390+0000","last_fullsized":"2023-10-08T07:13:47.355390+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:03.795703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398039+0000","last_change":"2023-10-08T07:13:48.397214+0000","last_active":"2023-10-08T07:13:48.398039+0000","last_peered":"2023-10-08T07:13:48.398039+0000","last_clean":"2023-10-08T07:13:48.398039+0000","last_became_active":"2023-10-08T07:13:48.396880+0000","last_became_peered":"2023-10-08T07:13:48.396880+0000","last_unstale":"2023-10-08T07:13:48.398039+0000","last_undegraded":"2023-10-08T07:13:48.398039+0000","last_fullsized":"2023-10-08T07:13:48.398039+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:38:50.886917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.371621+0000","last_change":"2023-10-08T07:13:48.371621+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.371621+0000","last_undegraded":"2023-10-08T07:13:48.371621+0000","last_fullsized":"2023-10-08T07:13:48.371621+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286393+0000","last_change":"2023-10-08T07:13:33.730922+0000","last_active":"2023-10-08T07:13:49.286393+0000","last_peered":"2023-10-08T07:13:49.286393+0000","last_clean":"2023-10-08T07:13:49.286393+0000","last_became_active":"2023-10-08T07:13:33.730666+0000","last_became_peered":"2023-10-08T07:13:33.730666+0000","last_unstale":"2023-10-08T07:13:49.286393+0000","last_undegraded":"2023-10-08T07:13:49.286393+0000","last_fullsized":"2023-10-08T07:13:49.286393+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:12.086089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291729+0000","last_change":"2023-10-08T07:13:34.302276+0000","last_active":"2023-10-08T07:13:49.291729+0000","last_peered":"2023-10-08T07:13:49.291729+0000","last_clean":"2023-10-08T07:13:49.291729+0000","last_became_active":"2023-10-08T07:13:34.301988+0000","last_became_peered":"2023-10-08T07:13:34.301988+0000","last_unstale":"2023-10-08T07:13:49.291729+0000","last_undegraded":"2023-10-08T07:13:49.291729+0000","last_fullsized":"2023-10-08T07:13:49.291729+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.389272+0000","last_change":"2023-10-08T07:13:48.386216+0000","last_active":"2023-10-08T07:13:48.389272+0000","last_peered":"2023-10-08T07:13:48.389272+0000","last_clean":"2023-10-08T07:13:48.389272+0000","last_became_active":"2023-10-08T07:13:48.385877+0000","last_became_peered":"2023-10-08T07:13:48.385877+0000","last_unstale":"2023-10-08T07:13:48.389272+0000","last_undegraded":"2023-10-08T07:13:48.389272+0000","last_fullsized":"2023-10-08T07:13:48.389272+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:06.260935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.366477+0000","last_change":"2023-10-08T07:13:48.366477+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.366477+0000","last_undegraded":"2023-10-08T07:13:48.366477+0000","last_fullsized":"2023-10-08T07:13:48.366477+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348222+0000","last_change":"2023-10-08T07:13:33.725167+0000","last_active":"2023-10-08T07:13:47.348222+0000","last_peered":"2023-10-08T07:13:47.348222+0000","last_clean":"2023-10-08T07:13:47.348222+0000","last_became_active":"2023-10-08T07:13:33.724905+0000","last_became_peered":"2023-10-08T07:13:33.724905+0000","last_unstale":"2023-10-08T07:13:47.348222+0000","last_undegraded":"2023-10-08T07:13:47.348222+0000","last_fullsized":"2023-10-08T07:13:47.348222+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:37.525027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348897+0000","last_change":"2023-10-08T07:13:34.296170+0000","last_active":"2023-10-08T07:13:47.348897+0000","last_peered":"2023-10-08T07:13:47.348897+0000","last_clean":"2023-10-08T07:13:47.348897+0000","last_became_active":"2023-10-08T07:13:34.295736+0000","last_became_peered":"2023-10-08T07:13:34.295736+0000","last_unstale":"2023-10-08T07:13:47.348897+0000","last_undegraded":"2023-10-08T07:13:47.348897+0000","last_fullsized":"2023-10-08T07:13:47.348897+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396650+0000","last_change":"2023-10-08T07:13:48.395168+0000","last_active":"2023-10-08T07:13:48.396650+0000","last_peered":"2023-10-08T07:13:48.396650+0000","last_clean":"2023-10-08T07:13:48.396650+0000","last_became_active":"2023-10-08T07:13:48.394879+0000","last_became_peered":"2023-10-08T07:13:48.394879+0000","last_unstale":"2023-10-08T07:13:48.396650+0000","last_undegraded":"2023-10-08T07:13:48.396650+0000","last_fullsized":"2023-10-08T07:13:48.396650+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:07.520887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359261+0000","last_change":"2023-10-08T07:13:49.303768+0000","last_active":"2023-10-08T07:13:50.359261+0000","last_peered":"2023-10-08T07:13:50.359261+0000","last_clean":"2023-10-08T07:13:50.359261+0000","last_became_active":"2023-10-08T07:13:49.302929+0000","last_became_peered":"2023-10-08T07:13:49.302929+0000","last_unstale":"2023-10-08T07:13:50.359261+0000","last_undegraded":"2023-10-08T07:13:50.359261+0000","last_fullsized":"2023-10-08T07:13:50.359261+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:21.341547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295376+0000","last_change":"2023-10-08T07:13:33.730955+0000","last_active":"2023-10-08T07:13:49.295376+0000","last_peered":"2023-10-08T07:13:49.295376+0000","last_clean":"2023-10-08T07:13:49.295376+0000","last_became_active":"2023-10-08T07:13:33.730676+0000","last_became_peered":"2023-10-08T07:13:33.730676+0000","last_unstale":"2023-10-08T07:13:49.295376+0000","last_undegraded":"2023-10-08T07:13:49.295376+0000","last_fullsized":"2023-10-08T07:13:49.295376+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:28.953862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295664+0000","last_change":"2023-10-08T07:13:34.293585+0000","last_active":"2023-10-08T07:13:49.295664+0000","last_peered":"2023-10-08T07:13:49.295664+0000","last_clean":"2023-10-08T07:13:49.295664+0000","last_became_active":"2023-10-08T07:13:34.293318+0000","last_became_peered":"2023-10-08T07:13:34.293318+0000","last_unstale":"2023-10-08T07:13:49.295664+0000","last_undegraded":"2023-10-08T07:13:49.295664+0000","last_fullsized":"2023-10-08T07:13:49.295664+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:09:34.022529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.391979+0000","last_change":"2023-10-08T07:13:48.388878+0000","last_active":"2023-10-08T07:13:48.391979+0000","last_peered":"2023-10-08T07:13:48.391979+0000","last_clean":"2023-10-08T07:13:48.391979+0000","last_became_active":"2023-10-08T07:13:48.388595+0000","last_became_peered":"2023-10-08T07:13:48.388595+0000","last_unstale":"2023-10-08T07:13:48.391979+0000","last_undegraded":"2023-10-08T07:13:48.391979+0000","last_fullsized":"2023-10-08T07:13:48.391979+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:28.683424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.363741+0000","last_change":"2023-10-08T07:13:48.363741+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.363741+0000","last_undegraded":"2023-10-08T07:13:48.363741+0000","last_fullsized":"2023-10-08T07:13:48.363741+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"234'8","reported_seq":22,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.352112+0000","last_change":"2023-10-08T07:13:33.746002+0000","last_active":"2023-10-08T07:13:47.352112+0000","last_peered":"2023-10-08T07:13:47.352112+0000","last_clean":"2023-10-08T07:13:47.352112+0000","last_became_active":"2023-10-08T07:13:33.745697+0000","last_became_peered":"2023-10-08T07:13:33.745697+0000","last_unstale":"2023-10-08T07:13:47.352112+0000","last_undegraded":"2023-10-08T07:13:47.352112+0000","last_fullsized":"2023-10-08T07:13:47.352112+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:25.841040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.283013+0000","last_change":"2023-10-08T07:13:34.293551+0000","last_active":"2023-10-08T07:13:49.283013+0000","last_peered":"2023-10-08T07:13:49.283013+0000","last_clean":"2023-10-08T07:13:49.283013+0000","last_became_active":"2023-10-08T07:13:34.292471+0000","last_became_peered":"2023-10-08T07:13:34.292471+0000","last_unstale":"2023-10-08T07:13:49.283013+0000","last_undegraded":"2023-10-08T07:13:49.283013+0000","last_fullsized":"2023-10-08T07:13:49.283013+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288920+0000","last_change":"2023-10-08T07:13:48.373437+0000","last_active":"2023-10-08T07:13:49.288920+0000","last_peered":"2023-10-08T07:13:49.288920+0000","last_clean":"2023-10-08T07:13:49.288920+0000","last_became_active":"2023-10-08T07:13:48.373147+0000","last_became_peered":"2023-10-08T07:13:48.373147+0000","last_unstale":"2023-10-08T07:13:49.288920+0000","last_undegraded":"2023-10-08T07:13:49.288920+0000","last_fullsized":"2023-10-08T07:13:49.288920+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:07.670172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.447672+0000","last_change":"2023-10-08T07:13:49.311094+0000","last_active":"2023-10-08T07:13:50.447672+0000","last_peered":"2023-10-08T07:13:50.447672+0000","last_clean":"2023-10-08T07:13:50.447672+0000","last_became_active":"2023-10-08T07:13:49.310778+0000","last_became_peered":"2023-10-08T07:13:49.310778+0000","last_unstale":"2023-10-08T07:13:50.447672+0000","last_undegraded":"2023-10-08T07:13:50.447672+0000","last_fullsized":"2023-10-08T07:13:50.447672+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:25:13.654111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"233'3","reported_seq":19,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290932+0000","last_change":"2023-10-08T07:13:33.726791+0000","last_active":"2023-10-08T07:13:49.290932+0000","last_peered":"2023-10-08T07:13:49.290932+0000","last_clean":"2023-10-08T07:13:49.290932+0000","last_became_active":"2023-10-08T07:13:33.726482+0000","last_became_peered":"2023-10-08T07:13:33.726482+0000","last_unstale":"2023-10-08T07:13:49.290932+0000","last_undegraded":"2023-10-08T07:13:49.290932+0000","last_fullsized":"2023-10-08T07:13:49.290932+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:11.409539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289130+0000","last_change":"2023-10-08T07:13:34.290971+0000","last_active":"2023-10-08T07:13:49.289130+0000","last_peered":"2023-10-08T07:13:49.289130+0000","last_clean":"2023-10-08T07:13:49.289130+0000","last_became_active":"2023-10-08T07:13:34.290294+0000","last_became_peered":"2023-10-08T07:13:34.290294+0000","last_unstale":"2023-10-08T07:13:49.289130+0000","last_undegraded":"2023-10-08T07:13:49.289130+0000","last_fullsized":"2023-10-08T07:13:49.289130+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.393740+0000","last_change":"2023-10-08T07:13:48.388537+0000","last_active":"2023-10-08T07:13:48.393740+0000","last_peered":"2023-10-08T07:13:48.393740+0000","last_clean":"2023-10-08T07:13:48.393740+0000","last_became_active":"2023-10-08T07:13:48.388005+0000","last_became_peered":"2023-10-08T07:13:48.388005+0000","last_unstale":"2023-10-08T07:13:48.393740+0000","last_undegraded":"2023-10-08T07:13:48.393740+0000","last_fullsized":"2023-10-08T07:13:48.393740+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:50.761583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376614+0000","last_change":"2023-10-08T07:13:48.376614+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376614+0000","last_undegraded":"2023-10-08T07:13:48.376614+0000","last_fullsized":"2023-10-08T07:13:48.376614+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.352897+0000","last_change":"2023-10-08T07:13:33.736423+0000","last_active":"2023-10-08T07:13:47.352897+0000","last_peered":"2023-10-08T07:13:47.352897+0000","last_clean":"2023-10-08T07:13:47.352897+0000","last_became_active":"2023-10-08T07:13:33.735959+0000","last_became_peered":"2023-10-08T07:13:33.735959+0000","last_unstale":"2023-10-08T07:13:47.352897+0000","last_undegraded":"2023-10-08T07:13:47.352897+0000","last_fullsized":"2023-10-08T07:13:47.352897+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:00.365787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.347992+0000","last_change":"2023-10-08T07:13:34.302391+0000","last_active":"2023-10-08T07:13:47.347992+0000","last_peered":"2023-10-08T07:13:47.347992+0000","last_clean":"2023-10-08T07:13:47.347992+0000","last_became_active":"2023-10-08T07:13:34.302105+0000","last_became_peered":"2023-10-08T07:13:34.302105+0000","last_unstale":"2023-10-08T07:13:47.347992+0000","last_undegraded":"2023-10-08T07:13:47.347992+0000","last_fullsized":"2023-10-08T07:13:47.347992+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396485+0000","last_change":"2023-10-08T07:13:48.391735+0000","last_active":"2023-10-08T07:13:48.396485+0000","last_peered":"2023-10-08T07:13:48.396485+0000","last_clean":"2023-10-08T07:13:48.396485+0000","last_became_active":"2023-10-08T07:13:48.391330+0000","last_became_peered":"2023-10-08T07:13:48.391330+0000","last_unstale":"2023-10-08T07:13:48.396485+0000","last_undegraded":"2023-10-08T07:13:48.396485+0000","last_fullsized":"2023-10-08T07:13:48.396485+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:19.090961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357935+0000","last_change":"2023-10-08T07:13:49.313956+0000","last_active":"2023-10-08T07:13:50.357935+0000","last_peered":"2023-10-08T07:13:50.357935+0000","last_clean":"2023-10-08T07:13:50.357935+0000","last_became_active":"2023-10-08T07:13:49.313687+0000","last_became_peered":"2023-10-08T07:13:49.313687+0000","last_unstale":"2023-10-08T07:13:50.357935+0000","last_undegraded":"2023-10-08T07:13:50.357935+0000","last_fullsized":"2023-10-08T07:13:50.357935+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:35.653573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284639+0000","last_change":"2023-10-08T07:13:33.735184+0000","last_active":"2023-10-08T07:13:49.284639+0000","last_peered":"2023-10-08T07:13:49.284639+0000","last_clean":"2023-10-08T07:13:49.284639+0000","last_became_active":"2023-10-08T07:13:33.734915+0000","last_became_peered":"2023-10-08T07:13:33.734915+0000","last_unstale":"2023-10-08T07:13:49.284639+0000","last_undegraded":"2023-10-08T07:13:49.284639+0000","last_fullsized":"2023-10-08T07:13:49.284639+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:52.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.354637+0000","last_change":"2023-10-08T07:13:34.294204+0000","last_active":"2023-10-08T07:13:47.354637+0000","last_peered":"2023-10-08T07:13:47.354637+0000","last_clean":"2023-10-08T07:13:47.354637+0000","last_became_active":"2023-10-08T07:13:34.293887+0000","last_became_peered":"2023-10-08T07:13:34.293887+0000","last_unstale":"2023-10-08T07:13:47.354637+0000","last_undegraded":"2023-10-08T07:13:47.354637+0000","last_fullsized":"2023-10-08T07:13:47.354637+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:59.699025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.391072+0000","last_change":"2023-10-08T07:13:48.389962+0000","last_active":"2023-10-08T07:13:48.391072+0000","last_peered":"2023-10-08T07:13:48.391072+0000","last_clean":"2023-10-08T07:13:48.391072+0000","last_became_active":"2023-10-08T07:13:48.389607+0000","last_became_peered":"2023-10-08T07:13:48.389607+0000","last_unstale":"2023-10-08T07:13:48.391072+0000","last_undegraded":"2023-10-08T07:13:48.391072+0000","last_fullsized":"2023-10-08T07:13:48.391072+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:14.995707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358517+0000","last_change":"2023-10-08T07:13:49.299000+0000","last_active":"2023-10-08T07:13:50.358517+0000","last_peered":"2023-10-08T07:13:50.358517+0000","last_clean":"2023-10-08T07:13:50.358517+0000","last_became_active":"2023-10-08T07:13:49.298705+0000","last_became_peered":"2023-10-08T07:13:49.298705+0000","last_unstale":"2023-10-08T07:13:50.358517+0000","last_undegraded":"2023-10-08T07:13:50.358517+0000","last_fullsized":"2023-10-08T07:13:50.358517+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:46.814809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398278+0000","last_change":"2023-10-08T07:13:48.393308+0000","last_active":"2023-10-08T07:13:48.398278+0000","last_peered":"2023-10-08T07:13:48.398278+0000","last_clean":"2023-10-08T07:13:48.398278+0000","last_became_active":"2023-10-08T07:13:48.392985+0000","last_became_peered":"2023-10-08T07:13:48.392985+0000","last_unstale":"2023-10-08T07:13:48.398278+0000","last_undegraded":"2023-10-08T07:13:48.398278+0000","last_fullsized":"2023-10-08T07:13:48.398278+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:13.322938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.347589+0000","last_change":"2023-10-08T07:13:33.731054+0000","last_active":"2023-10-08T07:13:47.347589+0000","last_peered":"2023-10-08T07:13:47.347589+0000","last_clean":"2023-10-08T07:13:47.347589+0000","last_became_active":"2023-10-08T07:13:33.730782+0000","last_became_peered":"2023-10-08T07:13:33.730782+0000","last_unstale":"2023-10-08T07:13:47.347589+0000","last_undegraded":"2023-10-08T07:13:47.347589+0000","last_fullsized":"2023-10-08T07:13:47.347589+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:06.476515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292156+0000","last_change":"2023-10-08T07:13:34.294260+0000","last_active":"2023-10-08T07:13:49.292156+0000","last_peered":"2023-10-08T07:13:49.292156+0000","last_clean":"2023-10-08T07:13:49.292156+0000","last_became_active":"2023-10-08T07:13:34.293992+0000","last_became_peered":"2023-10-08T07:13:34.293992+0000","last_unstale":"2023-10-08T07:13:49.292156+0000","last_undegraded":"2023-10-08T07:13:49.292156+0000","last_fullsized":"2023-10-08T07:13:49.292156+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358287+0000","last_change":"2023-10-08T07:13:49.309351+0000","last_active":"2023-10-08T07:13:50.358287+0000","last_peered":"2023-10-08T07:13:50.358287+0000","last_clean":"2023-10-08T07:13:50.358287+0000","last_became_active":"2023-10-08T07:13:49.309030+0000","last_became_peered":"2023-10-08T07:13:49.309030+0000","last_unstale":"2023-10-08T07:13:50.358287+0000","last_undegraded":"2023-10-08T07:13:50.358287+0000","last_fullsized":"2023-10-08T07:13:50.358287+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:08.833825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"237'4","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.372634+0000","last_change":"2023-10-08T07:13:48.385188+0000","last_active":"2023-10-08T07:13:49.372634+0000","last_peered":"2023-10-08T07:13:49.372634+0000","last_clean":"2023-10-08T07:13:49.372634+0000","last_became_active":"2023-10-08T07:13:48.384790+0000","last_became_peered":"2023-10-08T07:13:48.384790+0000","last_unstale":"2023-10-08T07:13:49.372634+0000","last_undegraded":"2023-10-08T07:13:49.372634+0000","last_fullsized":"2023-10-08T07:13:49.372634+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:30.625357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.350898+0000","last_change":"2023-10-08T07:13:33.738155+0000","last_active":"2023-10-08T07:13:47.350898+0000","last_peered":"2023-10-08T07:13:47.350898+0000","last_clean":"2023-10-08T07:13:47.350898+0000","last_became_active":"2023-10-08T07:13:33.737850+0000","last_became_peered":"2023-10-08T07:13:33.737850+0000","last_unstale":"2023-10-08T07:13:47.350898+0000","last_undegraded":"2023-10-08T07:13:47.350898+0000","last_fullsized":"2023-10-08T07:13:47.350898+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:37.479840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.356485+0000","last_change":"2023-10-08T07:13:34.304888+0000","last_active":"2023-10-08T07:13:47.356485+0000","last_peered":"2023-10-08T07:13:47.356485+0000","last_clean":"2023-10-08T07:13:47.356485+0000","last_became_active":"2023-10-08T07:13:34.304536+0000","last_became_peered":"2023-10-08T07:13:34.304536+0000","last_unstale":"2023-10-08T07:13:47.356485+0000","last_undegraded":"2023-10-08T07:13:47.356485+0000","last_fullsized":"2023-10-08T07:13:47.356485+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:05.467048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.379499+0000","last_change":"2023-10-08T07:13:48.379499+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.379499+0000","last_undegraded":"2023-10-08T07:13:48.379499+0000","last_fullsized":"2023-10-08T07:13:48.379499+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.351392+0000","last_change":"2023-10-08T07:13:33.738971+0000","last_active":"2023-10-08T07:13:47.351392+0000","last_peered":"2023-10-08T07:13:47.351392+0000","last_clean":"2023-10-08T07:13:47.351392+0000","last_became_active":"2023-10-08T07:13:33.738532+0000","last_became_peered":"2023-10-08T07:13:33.738532+0000","last_unstale":"2023-10-08T07:13:47.351392+0000","last_undegraded":"2023-10-08T07:13:47.351392+0000","last_fullsized":"2023-10-08T07:13:47.351392+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:06.806347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.356107+0000","last_change":"2023-10-08T07:13:34.302593+0000","last_active":"2023-10-08T07:13:47.356107+0000","last_peered":"2023-10-08T07:13:47.356107+0000","last_clean":"2023-10-08T07:13:47.356107+0000","last_became_active":"2023-10-08T07:13:34.302300+0000","last_became_peered":"2023-10-08T07:13:34.302300+0000","last_unstale":"2023-10-08T07:13:47.356107+0000","last_undegraded":"2023-10-08T07:13:47.356107+0000","last_fullsized":"2023-10-08T07:13:47.356107+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:13:33.465222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396596+0000","last_change":"2023-10-08T07:13:48.395886+0000","last_active":"2023-10-08T07:13:48.396596+0000","last_peered":"2023-10-08T07:13:48.396596+0000","last_clean":"2023-10-08T07:13:48.396596+0000","last_became_active":"2023-10-08T07:13:48.395585+0000","last_became_peered":"2023-10-08T07:13:48.395585+0000","last_unstale":"2023-10-08T07:13:48.396596+0000","last_undegraded":"2023-10-08T07:13:48.396596+0000","last_fullsized":"2023-10-08T07:13:48.396596+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:11.959208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.378495+0000","last_change":"2023-10-08T07:13:48.378495+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.378495+0000","last_undegraded":"2023-10-08T07:13:48.378495+0000","last_fullsized":"2023-10-08T07:13:48.378495+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290749+0000","last_change":"2023-10-08T07:13:33.736823+0000","last_active":"2023-10-08T07:13:49.290749+0000","last_peered":"2023-10-08T07:13:49.290749+0000","last_clean":"2023-10-08T07:13:49.290749+0000","last_became_active":"2023-10-08T07:13:33.736537+0000","last_became_peered":"2023-10-08T07:13:33.736537+0000","last_unstale":"2023-10-08T07:13:49.290749+0000","last_undegraded":"2023-10-08T07:13:49.290749+0000","last_fullsized":"2023-10-08T07:13:49.290749+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:48.798820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282798+0000","last_change":"2023-10-08T07:13:34.293354+0000","last_active":"2023-10-08T07:13:49.282798+0000","last_peered":"2023-10-08T07:13:49.282798+0000","last_clean":"2023-10-08T07:13:49.282798+0000","last_became_active":"2023-10-08T07:13:34.288220+0000","last_became_peered":"2023-10-08T07:13:34.288220+0000","last_unstale":"2023-10-08T07:13:49.282798+0000","last_undegraded":"2023-10-08T07:13:49.282798+0000","last_fullsized":"2023-10-08T07:13:49.282798+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398331+0000","last_change":"2023-10-08T07:13:48.397371+0000","last_active":"2023-10-08T07:13:48.398331+0000","last_peered":"2023-10-08T07:13:48.398331+0000","last_clean":"2023-10-08T07:13:48.398331+0000","last_became_active":"2023-10-08T07:13:48.396954+0000","last_became_peered":"2023-10-08T07:13:48.396954+0000","last_unstale":"2023-10-08T07:13:48.398331+0000","last_undegraded":"2023-10-08T07:13:48.398331+0000","last_fullsized":"2023-10-08T07:13:48.398331+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:39:02.780468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358686+0000","last_change":"2023-10-08T07:13:49.309391+0000","last_active":"2023-10-08T07:13:50.358686+0000","last_peered":"2023-10-08T07:13:50.358686+0000","last_clean":"2023-10-08T07:13:50.358686+0000","last_became_active":"2023-10-08T07:13:49.309100+0000","last_became_peered":"2023-10-08T07:13:49.309100+0000","last_unstale":"2023-10-08T07:13:50.358686+0000","last_undegraded":"2023-10-08T07:13:50.358686+0000","last_fullsized":"2023-10-08T07:13:50.358686+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:36.841571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294941+0000","last_change":"2023-10-08T07:13:33.724335+0000","last_active":"2023-10-08T07:13:49.294941+0000","last_peered":"2023-10-08T07:13:49.294941+0000","last_clean":"2023-10-08T07:13:49.294941+0000","last_became_active":"2023-10-08T07:13:33.724043+0000","last_became_peered":"2023-10-08T07:13:33.724043+0000","last_unstale":"2023-10-08T07:13:49.294941+0000","last_undegraded":"2023-10-08T07:13:49.294941+0000","last_fullsized":"2023-10-08T07:13:49.294941+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:58.128241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359306+0000","last_change":"2023-10-08T07:13:34.301668+0000","last_active":"2023-10-08T07:13:47.359306+0000","last_peered":"2023-10-08T07:13:47.359306+0000","last_clean":"2023-10-08T07:13:47.359306+0000","last_became_active":"2023-10-08T07:13:34.298698+0000","last_became_peered":"2023-10-08T07:13:34.298698+0000","last_unstale":"2023-10-08T07:13:47.359306+0000","last_undegraded":"2023-10-08T07:13:47.359306+0000","last_fullsized":"2023-10-08T07:13:47.359306+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.399490+0000","last_change":"2023-10-08T07:13:48.399044+0000","last_active":"2023-10-08T07:13:48.399490+0000","last_peered":"2023-10-08T07:13:48.399490+0000","last_clean":"2023-10-08T07:13:48.399490+0000","last_became_active":"2023-10-08T07:13:48.398793+0000","last_became_peered":"2023-10-08T07:13:48.398793+0000","last_unstale":"2023-10-08T07:13:48.399490+0000","last_undegraded":"2023-10-08T07:13:48.399490+0000","last_fullsized":"2023-10-08T07:13:48.399490+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:49:58.133301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278331+0000","last_change":"2023-10-08T07:13:49.303740+0000","last_active":"2023-10-08T07:13:50.278331+0000","last_peered":"2023-10-08T07:13:50.278331+0000","last_clean":"2023-10-08T07:13:50.278331+0000","last_became_active":"2023-10-08T07:13:49.303416+0000","last_became_peered":"2023-10-08T07:13:49.303416+0000","last_unstale":"2023-10-08T07:13:50.278331+0000","last_undegraded":"2023-10-08T07:13:50.278331+0000","last_fullsized":"2023-10-08T07:13:50.278331+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:38:51.904860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345012+0000","last_change":"2023-10-08T07:13:33.732312+0000","last_active":"2023-10-08T07:13:47.345012+0000","last_peered":"2023-10-08T07:13:47.345012+0000","last_clean":"2023-10-08T07:13:47.345012+0000","last_became_active":"2023-10-08T07:13:33.731998+0000","last_became_peered":"2023-10-08T07:13:33.731998+0000","last_unstale":"2023-10-08T07:13:47.345012+0000","last_undegraded":"2023-10-08T07:13:47.345012+0000","last_fullsized":"2023-10-08T07:13:47.345012+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:10:53.964103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343744+0000","last_change":"2023-10-08T07:13:34.293829+0000","last_active":"2023-10-08T07:13:47.343744+0000","last_peered":"2023-10-08T07:13:47.343744+0000","last_clean":"2023-10-08T07:13:47.343744+0000","last_became_active":"2023-10-08T07:13:34.293552+0000","last_became_peered":"2023-10-08T07:13:34.293552+0000","last_unstale":"2023-10-08T07:13:47.343744+0000","last_undegraded":"2023-10-08T07:13:47.343744+0000","last_fullsized":"2023-10-08T07:13:47.343744+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:38.485847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294683+0000","last_change":"2023-10-08T07:13:48.382427+0000","last_active":"2023-10-08T07:13:49.294683+0000","last_peered":"2023-10-08T07:13:49.294683+0000","last_clean":"2023-10-08T07:13:49.294683+0000","last_became_active":"2023-10-08T07:13:48.382141+0000","last_became_peered":"2023-10-08T07:13:48.382141+0000","last_unstale":"2023-10-08T07:13:49.294683+0000","last_undegraded":"2023-10-08T07:13:49.294683+0000","last_fullsized":"2023-10-08T07:13:49.294683+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:26.319310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.374502+0000","last_change":"2023-10-08T07:13:48.374502+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.374502+0000","last_undegraded":"2023-10-08T07:13:48.374502+0000","last_fullsized":"2023-10-08T07:13:48.374502+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"233'4","reported_seq":18,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.357270+0000","last_change":"2023-10-08T07:13:33.743017+0000","last_active":"2023-10-08T07:13:47.357270+0000","last_peered":"2023-10-08T07:13:47.357270+0000","last_clean":"2023-10-08T07:13:47.357270+0000","last_became_active":"2023-10-08T07:13:33.742582+0000","last_became_peered":"2023-10-08T07:13:33.742582+0000","last_unstale":"2023-10-08T07:13:47.357270+0000","last_undegraded":"2023-10-08T07:13:47.357270+0000","last_fullsized":"2023-10-08T07:13:47.357270+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:38:56.287497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296385+0000","last_change":"2023-10-08T07:13:34.286367+0000","last_active":"2023-10-08T07:13:49.296385+0000","last_peered":"2023-10-08T07:13:49.296385+0000","last_clean":"2023-10-08T07:13:49.296385+0000","last_became_active":"2023-10-08T07:13:34.285016+0000","last_became_peered":"2023-10-08T07:13:34.285016+0000","last_unstale":"2023-10-08T07:13:49.296385+0000","last_undegraded":"2023-10-08T07:13:49.296385+0000","last_fullsized":"2023-10-08T07:13:49.296385+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:41.466102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.397397+0000","last_change":"2023-10-08T07:13:48.394830+0000","last_active":"2023-10-08T07:13:48.397397+0000","last_peered":"2023-10-08T07:13:48.397397+0000","last_clean":"2023-10-08T07:13:48.397397+0000","last_became_active":"2023-10-08T07:13:48.394550+0000","last_became_peered":"2023-10-08T07:13:48.394550+0000","last_unstale":"2023-10-08T07:13:48.397397+0000","last_undegraded":"2023-10-08T07:13:48.397397+0000","last_fullsized":"2023-10-08T07:13:48.397397+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:33.349540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376167+0000","last_change":"2023-10-08T07:13:48.376167+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376167+0000","last_undegraded":"2023-10-08T07:13:48.376167+0000","last_fullsized":"2023-10-08T07:13:48.376167+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.349890+0000","last_change":"2023-10-08T07:13:34.295873+0000","last_active":"2023-10-08T07:13:47.349890+0000","last_peered":"2023-10-08T07:13:47.349890+0000","last_clean":"2023-10-08T07:13:47.349890+0000","last_became_active":"2023-10-08T07:13:34.295546+0000","last_became_peered":"2023-10-08T07:13:34.295546+0000","last_unstale":"2023-10-08T07:13:47.349890+0000","last_undegraded":"2023-10-08T07:13:47.349890+0000","last_fullsized":"2023-10-08T07:13:47.349890+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:46:53.896975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448531+0000","last_change":"2023-10-08T07:13:49.312717+0000","last_active":"2023-10-08T07:13:50.448531+0000","last_peered":"2023-10-08T07:13:50.448531+0000","last_clean":"2023-10-08T07:13:50.448531+0000","last_became_active":"2023-10-08T07:13:49.312448+0000","last_became_peered":"2023-10-08T07:13:49.312448+0000","last_unstale":"2023-10-08T07:13:50.448531+0000","last_undegraded":"2023-10-08T07:13:50.448531+0000","last_fullsized":"2023-10-08T07:13:50.448531+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:52.376348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.350665+0000","last_change":"2023-10-08T07:13:34.301071+0000","last_active":"2023-10-08T07:13:47.350665+0000","last_peered":"2023-10-08T07:13:47.350665+0000","last_clean":"2023-10-08T07:13:47.350665+0000","last_became_active":"2023-10-08T07:13:34.300772+0000","last_became_peered":"2023-10-08T07:13:34.300772+0000","last_unstale":"2023-10-08T07:13:47.350665+0000","last_undegraded":"2023-10-08T07:13:47.350665+0000","last_fullsized":"2023-10-08T07:13:47.350665+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:59:54.643977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.372376+0000","last_change":"2023-10-08T07:13:48.372376+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.372376+0000","last_undegraded":"2023-10-08T07:13:48.372376+0000","last_fullsized":"2023-10-08T07:13:48.372376+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.353002+0000","last_change":"2023-10-08T07:13:34.300974+0000","last_active":"2023-10-08T07:13:47.353002+0000","last_peered":"2023-10-08T07:13:47.353002+0000","last_clean":"2023-10-08T07:13:47.353002+0000","last_became_active":"2023-10-08T07:13:34.300529+0000","last_became_peered":"2023-10-08T07:13:34.300529+0000","last_unstale":"2023-10-08T07:13:47.353002+0000","last_undegraded":"2023-10-08T07:13:47.353002+0000","last_fullsized":"2023-10-08T07:13:47.353002+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:17.001444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278016+0000","last_change":"2023-10-08T07:13:49.306936+0000","last_active":"2023-10-08T07:13:50.278016+0000","last_peered":"2023-10-08T07:13:50.278016+0000","last_clean":"2023-10-08T07:13:50.278016+0000","last_became_active":"2023-10-08T07:13:49.306284+0000","last_became_peered":"2023-10-08T07:13:49.306284+0000","last_unstale":"2023-10-08T07:13:50.278016+0000","last_undegraded":"2023-10-08T07:13:50.278016+0000","last_fullsized":"2023-10-08T07:13:50.278016+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:51.779526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287748+0000","last_change":"2023-10-08T07:13:34.293712+0000","last_active":"2023-10-08T07:13:49.287748+0000","last_peered":"2023-10-08T07:13:49.287748+0000","last_clean":"2023-10-08T07:13:49.287748+0000","last_became_active":"2023-10-08T07:13:34.293415+0000","last_became_peered":"2023-10-08T07:13:34.293415+0000","last_unstale":"2023-10-08T07:13:49.287748+0000","last_undegraded":"2023-10-08T07:13:49.287748+0000","last_fullsized":"2023-10-08T07:13:49.287748+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:47.966787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359086+0000","last_change":"2023-10-08T07:13:49.316786+0000","last_active":"2023-10-08T07:13:50.359086+0000","last_peered":"2023-10-08T07:13:50.359086+0000","last_clean":"2023-10-08T07:13:50.359086+0000","last_became_active":"2023-10-08T07:13:49.316437+0000","last_became_peered":"2023-10-08T07:13:49.316437+0000","last_unstale":"2023-10-08T07:13:50.359086+0000","last_undegraded":"2023-10-08T07:13:50.359086+0000","last_fullsized":"2023-10-08T07:13:50.359086+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:14.040352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282033+0000","last_change":"2023-10-08T07:13:34.297816+0000","last_active":"2023-10-08T07:13:49.282033+0000","last_peered":"2023-10-08T07:13:49.282033+0000","last_clean":"2023-10-08T07:13:49.282033+0000","last_became_active":"2023-10-08T07:13:34.297544+0000","last_became_peered":"2023-10-08T07:13:34.297544+0000","last_unstale":"2023-10-08T07:13:49.282033+0000","last_undegraded":"2023-10-08T07:13:49.282033+0000","last_fullsized":"2023-10-08T07:13:49.282033+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.375861+0000","last_change":"2023-10-08T07:13:48.375861+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.375861+0000","last_undegraded":"2023-10-08T07:13:48.375861+0000","last_fullsized":"2023-10-08T07:13:48.375861+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348874+0000","last_change":"2023-10-08T07:13:34.310969+0000","last_active":"2023-10-08T07:13:47.348874+0000","last_peered":"2023-10-08T07:13:47.348874+0000","last_clean":"2023-10-08T07:13:47.348874+0000","last_became_active":"2023-10-08T07:13:34.310679+0000","last_became_peered":"2023-10-08T07:13:34.310679+0000","last_unstale":"2023-10-08T07:13:47.348874+0000","last_undegraded":"2023-10-08T07:13:47.348874+0000","last_fullsized":"2023-10-08T07:13:47.348874+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381042+0000","last_change":"2023-10-08T07:13:48.381042+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381042+0000","last_undegraded":"2023-10-08T07:13:48.381042+0000","last_fullsized":"2023-10-08T07:13:48.381042+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344914+0000","last_change":"2023-10-08T07:13:34.304704+0000","last_active":"2023-10-08T07:13:47.344914+0000","last_peered":"2023-10-08T07:13:47.344914+0000","last_clean":"2023-10-08T07:13:47.344914+0000","last_became_active":"2023-10-08T07:13:34.304032+0000","last_became_peered":"2023-10-08T07:13:34.304032+0000","last_unstale":"2023-10-08T07:13:47.344914+0000","last_undegraded":"2023-10-08T07:13:47.344914+0000","last_fullsized":"2023-10-08T07:13:47.344914+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:15.818894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.367829+0000","last_change":"2023-10-08T07:13:48.367829+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.367829+0000","last_undegraded":"2023-10-08T07:13:48.367829+0000","last_fullsized":"2023-10-08T07:13:48.367829+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292077+0000","last_change":"2023-10-08T07:13:34.298953+0000","last_active":"2023-10-08T07:13:49.292077+0000","last_peered":"2023-10-08T07:13:49.292077+0000","last_clean":"2023-10-08T07:13:49.292077+0000","last_became_active":"2023-10-08T07:13:34.298694+0000","last_became_peered":"2023-10-08T07:13:34.298694+0000","last_unstale":"2023-10-08T07:13:49.292077+0000","last_undegraded":"2023-10-08T07:13:49.292077+0000","last_fullsized":"2023-10-08T07:13:49.292077+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376110+0000","last_change":"2023-10-08T07:13:48.376110+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376110+0000","last_undegraded":"2023-10-08T07:13:48.376110+0000","last_fullsized":"2023-10-08T07:13:48.376110+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.277869+0000","last_change":"2023-10-08T07:13:34.293797+0000","last_active":"2023-10-08T07:13:49.277869+0000","last_peered":"2023-10-08T07:13:49.277869+0000","last_clean":"2023-10-08T07:13:49.277869+0000","last_became_active":"2023-10-08T07:13:34.289224+0000","last_became_peered":"2023-10-08T07:13:34.289224+0000","last_unstale":"2023-10-08T07:13:49.277869+0000","last_undegraded":"2023-10-08T07:13:49.277869+0000","last_fullsized":"2023-10-08T07:13:49.277869+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357969+0000","last_change":"2023-10-08T07:13:49.303551+0000","last_active":"2023-10-08T07:13:50.357969+0000","last_peered":"2023-10-08T07:13:50.357969+0000","last_clean":"2023-10-08T07:13:50.357969+0000","last_became_active":"2023-10-08T07:13:49.303258+0000","last_became_peered":"2023-10-08T07:13:49.303258+0000","last_unstale":"2023-10-08T07:13:50.357969+0000","last_undegraded":"2023-10-08T07:13:50.357969+0000","last_fullsized":"2023-10-08T07:13:50.357969+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:08.504585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295174+0000","last_change":"2023-10-08T07:13:34.291585+0000","last_active":"2023-10-08T07:13:49.295174+0000","last_peered":"2023-10-08T07:13:49.295174+0000","last_clean":"2023-10-08T07:13:49.295174+0000","last_became_active":"2023-10-08T07:13:34.291125+0000","last_became_peered":"2023-10-08T07:13:34.291125+0000","last_unstale":"2023-10-08T07:13:49.295174+0000","last_undegraded":"2023-10-08T07:13:49.295174+0000","last_fullsized":"2023-10-08T07:13:49.295174+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:14.399506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.372077+0000","last_change":"2023-10-08T07:13:48.372077+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.372077+0000","last_undegraded":"2023-10-08T07:13:48.372077+0000","last_fullsized":"2023-10-08T07:13:48.372077+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359175+0000","last_change":"2023-10-08T07:13:34.297774+0000","last_active":"2023-10-08T07:13:47.359175+0000","last_peered":"2023-10-08T07:13:47.359175+0000","last_clean":"2023-10-08T07:13:47.359175+0000","last_became_active":"2023-10-08T07:13:34.297492+0000","last_became_peered":"2023-10-08T07:13:34.297492+0000","last_unstale":"2023-10-08T07:13:47.359175+0000","last_undegraded":"2023-10-08T07:13:47.359175+0000","last_fullsized":"2023-10-08T07:13:47.359175+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.277878+0000","last_change":"2023-10-08T07:13:49.298115+0000","last_active":"2023-10-08T07:13:50.277878+0000","last_peered":"2023-10-08T07:13:50.277878+0000","last_clean":"2023-10-08T07:13:50.277878+0000","last_became_active":"2023-10-08T07:13:49.297582+0000","last_became_peered":"2023-10-08T07:13:49.297582+0000","last_unstale":"2023-10-08T07:13:50.277878+0000","last_undegraded":"2023-10-08T07:13:50.277878+0000","last_fullsized":"2023-10-08T07:13:50.277878+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:11.399029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.361137+0000","last_change":"2023-10-08T07:13:34.294431+0000","last_active":"2023-10-08T07:13:47.361137+0000","last_peered":"2023-10-08T07:13:47.361137+0000","last_clean":"2023-10-08T07:13:47.361137+0000","last_became_active":"2023-10-08T07:13:34.294129+0000","last_became_peered":"2023-10-08T07:13:34.294129+0000","last_unstale":"2023-10-08T07:13:47.361137+0000","last_undegraded":"2023-10-08T07:13:47.361137+0000","last_fullsized":"2023-10-08T07:13:47.361137+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358949+0000","last_change":"2023-10-08T07:13:49.313393+0000","last_active":"2023-10-08T07:13:50.358949+0000","last_peered":"2023-10-08T07:13:50.358949+0000","last_clean":"2023-10-08T07:13:50.358949+0000","last_became_active":"2023-10-08T07:13:49.313094+0000","last_became_peered":"2023-10-08T07:13:49.313094+0000","last_unstale":"2023-10-08T07:13:50.358949+0000","last_undegraded":"2023-10-08T07:13:50.358949+0000","last_fullsized":"2023-10-08T07:13:50.358949+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:06.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359288+0000","last_change":"2023-10-08T07:13:34.297677+0000","last_active":"2023-10-08T07:13:47.359288+0000","last_peered":"2023-10-08T07:13:47.359288+0000","last_clean":"2023-10-08T07:13:47.359288+0000","last_became_active":"2023-10-08T07:13:34.296526+0000","last_became_peered":"2023-10-08T07:13:34.296526+0000","last_unstale":"2023-10-08T07:13:47.359288+0000","last_undegraded":"2023-10-08T07:13:47.359288+0000","last_fullsized":"2023-10-08T07:13:47.359288+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:51.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381122+0000","last_change":"2023-10-08T07:13:48.381122+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381122+0000","last_undegraded":"2023-10-08T07:13:48.381122+0000","last_fullsized":"2023-10-08T07:13:48.381122+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359714+0000","last_change":"2023-10-08T07:13:34.295222+0000","last_active":"2023-10-08T07:13:47.359714+0000","last_peered":"2023-10-08T07:13:47.359714+0000","last_clean":"2023-10-08T07:13:47.359714+0000","last_became_active":"2023-10-08T07:13:34.289517+0000","last_became_peered":"2023-10-08T07:13:34.289517+0000","last_unstale":"2023-10-08T07:13:47.359714+0000","last_undegraded":"2023-10-08T07:13:47.359714+0000","last_fullsized":"2023-10-08T07:13:47.359714+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:38.224838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.380405+0000","last_change":"2023-10-08T07:13:48.380405+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.380405+0000","last_undegraded":"2023-10-08T07:13:48.380405+0000","last_fullsized":"2023-10-08T07:13:48.380405+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291605+0000","last_change":"2023-10-08T07:13:34.299238+0000","last_active":"2023-10-08T07:13:49.291605+0000","last_peered":"2023-10-08T07:13:49.291605+0000","last_clean":"2023-10-08T07:13:49.291605+0000","last_became_active":"2023-10-08T07:13:34.298585+0000","last_became_peered":"2023-10-08T07:13:34.298585+0000","last_unstale":"2023-10-08T07:13:49.291605+0000","last_undegraded":"2023-10-08T07:13:49.291605+0000","last_fullsized":"2023-10-08T07:13:49.291605+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376641+0000","last_change":"2023-10-08T07:13:48.376641+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376641+0000","last_undegraded":"2023-10-08T07:13:48.376641+0000","last_fullsized":"2023-10-08T07:13:48.376641+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292035+0000","last_change":"2023-10-08T07:13:34.308624+0000","last_active":"2023-10-08T07:13:49.292035+0000","last_peered":"2023-10-08T07:13:49.292035+0000","last_clean":"2023-10-08T07:13:49.292035+0000","last_became_active":"2023-10-08T07:13:34.308246+0000","last_became_peered":"2023-10-08T07:13:34.308246+0000","last_unstale":"2023-10-08T07:13:49.292035+0000","last_undegraded":"2023-10-08T07:13:49.292035+0000","last_fullsized":"2023-10-08T07:13:49.292035+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381713+0000","last_change":"2023-10-08T07:13:48.381713+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381713+0000","last_undegraded":"2023-10-08T07:13:48.381713+0000","last_fullsized":"2023-10-08T07:13:48.381713+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.347720+0000","last_change":"2023-10-08T07:13:34.300821+0000","last_active":"2023-10-08T07:13:47.347720+0000","last_peered":"2023-10-08T07:13:47.347720+0000","last_clean":"2023-10-08T07:13:47.347720+0000","last_became_active":"2023-10-08T07:13:34.300558+0000","last_became_peered":"2023-10-08T07:13:34.300558+0000","last_unstale":"2023-10-08T07:13:47.347720+0000","last_undegraded":"2023-10-08T07:13:47.347720+0000","last_fullsized":"2023-10-08T07:13:47.347720+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.375275+0000","last_change":"2023-10-08T07:13:48.375275+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.375275+0000","last_undegraded":"2023-10-08T07:13:48.375275+0000","last_fullsized":"2023-10-08T07:13:48.375275+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.354342+0000","last_change":"2023-10-08T07:13:34.309619+0000","last_active":"2023-10-08T07:13:47.354342+0000","last_peered":"2023-10-08T07:13:47.354342+0000","last_clean":"2023-10-08T07:13:47.354342+0000","last_became_active":"2023-10-08T07:13:34.308202+0000","last_became_peered":"2023-10-08T07:13:34.308202+0000","last_unstale":"2023-10-08T07:13:47.354342+0000","last_undegraded":"2023-10-08T07:13:47.354342+0000","last_fullsized":"2023-10-08T07:13:47.354342+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:59.914799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.373371+0000","last_change":"2023-10-08T07:13:48.373371+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.373371+0000","last_undegraded":"2023-10-08T07:13:48.373371+0000","last_fullsized":"2023-10-08T07:13:48.373371+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359414+0000","last_change":"2023-10-08T07:13:34.298407+0000","last_active":"2023-10-08T07:13:47.359414+0000","last_peered":"2023-10-08T07:13:47.359414+0000","last_clean":"2023-10-08T07:13:47.359414+0000","last_became_active":"2023-10-08T07:13:34.298139+0000","last_became_peered":"2023-10-08T07:13:34.298139+0000","last_unstale":"2023-10-08T07:13:47.359414+0000","last_undegraded":"2023-10-08T07:13:47.359414+0000","last_fullsized":"2023-10-08T07:13:47.359414+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376338+0000","last_change":"2023-10-08T07:13:48.376338+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376338+0000","last_undegraded":"2023-10-08T07:13:48.376338+0000","last_fullsized":"2023-10-08T07:13:48.376338+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.351296+0000","last_change":"2023-10-08T07:13:34.306040+0000","last_active":"2023-10-08T07:13:47.351296+0000","last_peered":"2023-10-08T07:13:47.351296+0000","last_clean":"2023-10-08T07:13:47.351296+0000","last_became_active":"2023-10-08T07:13:34.303015+0000","last_became_peered":"2023-10-08T07:13:34.303015+0000","last_unstale":"2023-10-08T07:13:47.351296+0000","last_undegraded":"2023-10-08T07:13:47.351296+0000","last_fullsized":"2023-10-08T07:13:47.351296+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:42.884365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358958+0000","last_change":"2023-10-08T07:13:49.310325+0000","last_active":"2023-10-08T07:13:50.358958+0000","last_peered":"2023-10-08T07:13:50.358958+0000","last_clean":"2023-10-08T07:13:50.358958+0000","last_became_active":"2023-10-08T07:13:49.310037+0000","last_became_peered":"2023-10-08T07:13:49.310037+0000","last_unstale":"2023-10-08T07:13:50.358958+0000","last_undegraded":"2023-10-08T07:13:50.358958+0000","last_fullsized":"2023-10-08T07:13:50.358958+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:29.904246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.281819+0000","last_change":"2023-10-08T07:13:34.294253+0000","last_active":"2023-10-08T07:13:49.281819+0000","last_peered":"2023-10-08T07:13:49.281819+0000","last_clean":"2023-10-08T07:13:49.281819+0000","last_became_active":"2023-10-08T07:13:34.291997+0000","last_became_peered":"2023-10-08T07:13:34.291997+0000","last_unstale":"2023-10-08T07:13:49.281819+0000","last_undegraded":"2023-10-08T07:13:49.281819+0000","last_fullsized":"2023-10-08T07:13:49.281819+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278736+0000","last_change":"2023-10-08T07:13:49.298924+0000","last_active":"2023-10-08T07:13:50.278736+0000","last_peered":"2023-10-08T07:13:50.278736+0000","last_clean":"2023-10-08T07:13:50.278736+0000","last_became_active":"2023-10-08T07:13:49.298587+0000","last_became_peered":"2023-10-08T07:13:49.298587+0000","last_unstale":"2023-10-08T07:13:50.278736+0000","last_undegraded":"2023-10-08T07:13:50.278736+0000","last_fullsized":"2023-10-08T07:13:50.278736+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:34.367483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348653+0000","last_change":"2023-10-08T07:13:34.308930+0000","last_active":"2023-10-08T07:13:47.348653+0000","last_peered":"2023-10-08T07:13:47.348653+0000","last_clean":"2023-10-08T07:13:47.348653+0000","last_became_active":"2023-10-08T07:13:34.308283+0000","last_became_peered":"2023-10-08T07:13:34.308283+0000","last_unstale":"2023-10-08T07:13:47.348653+0000","last_undegraded":"2023-10-08T07:13:47.348653+0000","last_fullsized":"2023-10-08T07:13:47.348653+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381921+0000","last_change":"2023-10-08T07:13:48.381921+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381921+0000","last_undegraded":"2023-10-08T07:13:48.381921+0000","last_fullsized":"2023-10-08T07:13:48.381921+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359386+0000","last_change":"2023-10-08T07:13:34.295991+0000","last_active":"2023-10-08T07:13:47.359386+0000","last_peered":"2023-10-08T07:13:47.359386+0000","last_clean":"2023-10-08T07:13:47.359386+0000","last_became_active":"2023-10-08T07:13:34.295717+0000","last_became_peered":"2023-10-08T07:13:34.295717+0000","last_unstale":"2023-10-08T07:13:47.359386+0000","last_undegraded":"2023-10-08T07:13:47.359386+0000","last_fullsized":"2023-10-08T07:13:47.359386+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.797828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.368925+0000","last_change":"2023-10-08T07:13:48.368925+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.368925+0000","last_undegraded":"2023-10-08T07:13:48.368925+0000","last_fullsized":"2023-10-08T07:13:48.368925+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291913+0000","last_change":"2023-10-08T07:13:34.289375+0000","last_active":"2023-10-08T07:13:49.291913+0000","last_peered":"2023-10-08T07:13:49.291913+0000","last_clean":"2023-10-08T07:13:49.291913+0000","last_became_active":"2023-10-08T07:13:34.289115+0000","last_became_peered":"2023-10-08T07:13:34.289115+0000","last_unstale":"2023-10-08T07:13:49.291913+0000","last_undegraded":"2023-10-08T07:13:49.291913+0000","last_fullsized":"2023-10-08T07:13:49.291913+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.375367+0000","last_change":"2023-10-08T07:13:48.375367+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.375367+0000","last_undegraded":"2023-10-08T07:13:48.375367+0000","last_fullsized":"2023-10-08T07:13:48.375367+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359704+0000","last_change":"2023-10-08T07:13:34.302018+0000","last_active":"2023-10-08T07:13:47.359704+0000","last_peered":"2023-10-08T07:13:47.359704+0000","last_clean":"2023-10-08T07:13:47.359704+0000","last_became_active":"2023-10-08T07:13:34.301747+0000","last_became_peered":"2023-10-08T07:13:34.301747+0000","last_unstale":"2023-10-08T07:13:47.359704+0000","last_undegraded":"2023-10-08T07:13:47.359704+0000","last_fullsized":"2023-10-08T07:13:47.359704+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.378259+0000","last_change":"2023-10-08T07:13:48.378259+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.378259+0000","last_undegraded":"2023-10-08T07:13:48.378259+0000","last_fullsized":"2023-10-08T07:13:48.378259+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290421+0000","last_change":"2023-10-08T07:13:34.293550+0000","last_active":"2023-10-08T07:13:49.290421+0000","last_peered":"2023-10-08T07:13:49.290421+0000","last_clean":"2023-10-08T07:13:49.290421+0000","last_became_active":"2023-10-08T07:13:34.293202+0000","last_became_peered":"2023-10-08T07:13:34.293202+0000","last_unstale":"2023-10-08T07:13:49.290421+0000","last_undegraded":"2023-10-08T07:13:49.290421+0000","last_fullsized":"2023-10-08T07:13:49.290421+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:00.842346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.374742+0000","last_change":"2023-10-08T07:13:48.374742+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.374742+0000","last_undegraded":"2023-10-08T07:13:48.374742+0000","last_fullsized":"2023-10-08T07:13:48.374742+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.355316+0000","last_change":"2023-10-08T07:13:34.298110+0000","last_active":"2023-10-08T07:13:47.355316+0000","last_peered":"2023-10-08T07:13:47.355316+0000","last_clean":"2023-10-08T07:13:47.355316+0000","last_became_active":"2023-10-08T07:13:34.297806+0000","last_became_peered":"2023-10-08T07:13:34.297806+0000","last_unstale":"2023-10-08T07:13:47.355316+0000","last_undegraded":"2023-10-08T07:13:47.355316+0000","last_fullsized":"2023-10-08T07:13:47.355316+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:58.335295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357781+0000","last_change":"2023-10-08T07:13:49.315887+0000","last_active":"2023-10-08T07:13:50.357781+0000","last_peered":"2023-10-08T07:13:50.357781+0000","last_clean":"2023-10-08T07:13:50.357781+0000","last_became_active":"2023-10-08T07:13:49.315571+0000","last_became_peered":"2023-10-08T07:13:49.315571+0000","last_unstale":"2023-10-08T07:13:50.357781+0000","last_undegraded":"2023-10-08T07:13:50.357781+0000","last_fullsized":"2023-10-08T07:13:50.357781+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:31.253205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282695+0000","last_change":"2023-10-08T07:13:34.298025+0000","last_active":"2023-10-08T07:13:49.282695+0000","last_peered":"2023-10-08T07:13:49.282695+0000","last_clean":"2023-10-08T07:13:49.282695+0000","last_became_active":"2023-10-08T07:13:34.296181+0000","last_became_peered":"2023-10-08T07:13:34.296181+0000","last_unstale":"2023-10-08T07:13:49.282695+0000","last_undegraded":"2023-10-08T07:13:49.282695+0000","last_fullsized":"2023-10-08T07:13:49.282695+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448978+0000","last_change":"2023-10-08T07:13:49.300145+0000","last_active":"2023-10-08T07:13:50.448978+0000","last_peered":"2023-10-08T07:13:50.448978+0000","last_clean":"2023-10-08T07:13:50.448978+0000","last_became_active":"2023-10-08T07:13:49.299846+0000","last_became_peered":"2023-10-08T07:13:49.299846+0000","last_unstale":"2023-10-08T07:13:50.448978+0000","last_undegraded":"2023-10-08T07:13:50.448978+0000","last_fullsized":"2023-10-08T07:13:50.448978+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:16.187252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.351855+0000","last_change":"2023-10-08T07:13:34.301326+0000","last_active":"2023-10-08T07:13:47.351855+0000","last_peered":"2023-10-08T07:13:47.351855+0000","last_clean":"2023-10-08T07:13:47.351855+0000","last_became_active":"2023-10-08T07:13:34.299365+0000","last_became_peered":"2023-10-08T07:13:34.299365+0000","last_unstale":"2023-10-08T07:13:47.351855+0000","last_undegraded":"2023-10-08T07:13:47.351855+0000","last_fullsized":"2023-10-08T07:13:47.351855+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:02.576748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376508+0000","last_change":"2023-10-08T07:13:48.376508+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376508+0000","last_undegraded":"2023-10-08T07:13:48.376508+0000","last_fullsized":"2023-10-08T07:13:48.376508+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348325+0000","last_change":"2023-10-08T07:13:34.306582+0000","last_active":"2023-10-08T07:13:47.348325+0000","last_peered":"2023-10-08T07:13:47.348325+0000","last_clean":"2023-10-08T07:13:47.348325+0000","last_became_active":"2023-10-08T07:13:34.306310+0000","last_became_peered":"2023-10-08T07:13:34.306310+0000","last_unstale":"2023-10-08T07:13:47.348325+0000","last_undegraded":"2023-10-08T07:13:47.348325+0000","last_fullsized":"2023-10-08T07:13:47.348325+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.373866+0000","last_change":"2023-10-08T07:13:48.373866+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.373866+0000","last_undegraded":"2023-10-08T07:13:48.373866+0000","last_fullsized":"2023-10-08T07:13:48.373866+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298001+0000","last_change":"2023-10-08T07:13:34.295062+0000","last_active":"2023-10-08T07:13:49.298001+0000","last_peered":"2023-10-08T07:13:49.298001+0000","last_clean":"2023-10-08T07:13:49.298001+0000","last_became_active":"2023-10-08T07:13:34.294747+0000","last_became_peered":"2023-10-08T07:13:34.294747+0000","last_unstale":"2023-10-08T07:13:49.298001+0000","last_undegraded":"2023-10-08T07:13:49.298001+0000","last_fullsized":"2023-10-08T07:13:49.298001+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:26.407053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358527+0000","last_change":"2023-10-08T07:13:49.304648+0000","last_active":"2023-10-08T07:13:50.358527+0000","last_peered":"2023-10-08T07:13:50.358527+0000","last_clean":"2023-10-08T07:13:50.358527+0000","last_became_active":"2023-10-08T07:13:49.304247+0000","last_became_peered":"2023-10-08T07:13:49.304247+0000","last_unstale":"2023-10-08T07:13:50.358527+0000","last_undegraded":"2023-10-08T07:13:50.358527+0000","last_fullsized":"2023-10-08T07:13:50.358527+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:42.456757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345193+0000","last_change":"2023-10-08T07:13:34.308897+0000","last_active":"2023-10-08T07:13:47.345193+0000","last_peered":"2023-10-08T07:13:47.345193+0000","last_clean":"2023-10-08T07:13:47.345193+0000","last_became_active":"2023-10-08T07:13:34.308474+0000","last_became_peered":"2023-10-08T07:13:34.308474+0000","last_unstale":"2023-10-08T07:13:47.345193+0000","last_undegraded":"2023-10-08T07:13:47.345193+0000","last_fullsized":"2023-10-08T07:13:47.345193+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:06.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":604,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2806,"ondisk_log_size":2806,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576011,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61024,"kb_used_data":3476,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310816,"statfs":{"total":96636764160,"available":96574275584,"internally_reserved":0,"allocated":3559424,"data_stored":2124760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.169,"15min":1.2090000000000001},"min":{"1min":0.77100000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.508,"5min":1.968,"15min":2.008},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.238,"15min":1.2310000000000001},"min":{"1min":0.91300000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.6830000000000001,"5min":1.7330000000000001,"15min":1.7450000000000001},"last":1.107}]},{"osd":1,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.242,"15min":1.252},"min":{"1min":0.94399999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.665,"5min":1.79,"15min":1.998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.341,"15min":1.3500000000000001},"min":{"1min":0.99399999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.8859999999999999,"5min":1.8899999999999999,"15min":2.1019999999999999},"last":0.997}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.29,"15min":1.3140000000000001},"min":{"1min":0.97199999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6180000000000001,"5min":1.696,"15min":2.1640000000000001},"last":1.044},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.351},"min":{"1min":1.0069999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7270000000000001,"5min":1.8580000000000001,"15min":2.3170000000000002},"last":1.0660000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":1.0209999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.663,"5min":1.8160000000000001,"15min":1.9039999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.403,"5min":1.399,"15min":1.4119999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.754,"5min":1.8680000000000001,"15min":1.931},"last":1.3080000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":0.86199999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6950000000000001,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.383,"15min":1.401},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.85199999999999998},"max":{"1min":1.76,"5min":1.8140000000000001,"15min":2.0379999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4390000000000001,"15min":1.444},"min":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":2.0920000000000001,"15min":2.1139999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.4610000000000001,"15min":1.51},"min":{"1min":1.075,"5min":1,"15min":0.96899999999999997},"max":{"1min":1.8640000000000001,"5min":2.23,"15min":2.23},"last":1.2529999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.575,"15min":1.6020000000000001},"min":{"1min":1.361,"5min":1.048,"15min":1.048},"max":{"1min":2.016,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.349},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.536,"15min":1.5349999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9079999999999999,"5min":2.21,"15min":2.2570000000000001},"last":1.2889999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576010,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46568,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325272,"statfs":{"total":96636764160,"available":96589078528,"internally_reserved":0,"allocated":2977792,"data_stored":1542476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3109999999999999,"15min":1.321},"min":{"1min":0.80000000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":1.794,"15min":1.962},"last":1.637},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.347,"15min":1.3620000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7230000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.222}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.228,"15min":1.2290000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.2130000000000001,"15min":1.2430000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.97},"last":1.5529999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.198,"15min":1.238},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.484,"5min":2.339,"15min":2.339},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":1.153}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3819999999999999,"15min":1.403},"min":{"1min":1.034,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":1.9319999999999999,"15min":1.95},"last":1.468},{"interface":"front","average":{"1min":1.425,"5min":1.476,"15min":1.4990000000000001},"min":{"1min":0.97199999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8740000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.5129999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3999999999999999,"15min":1.3899999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.881,"5min":2.0579999999999998,"15min":2.0859999999999999},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4770000000000001,"15min":1.448},"min":{"1min":0.94999999999999996,"5min":0.92000000000000004,"15min":0.57099999999999995},"max":{"1min":1.8080000000000001,"5min":2.484,"15min":2.484},"last":1.603}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.399,"15min":1.423},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.81899999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.403},"last":1.369},{"interface":"front","average":{"1min":1.552,"5min":1.538,"15min":1.5509999999999999},"min":{"1min":1.2829999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.8100000000000001,"5min":2.097,"15min":2.6139999999999999},"last":1.6950000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.575},"min":{"1min":1.0489999999999999,"5min":1.012,"15min":0.96299999999999997},"max":{"1min":1.9379999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.413},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.5620000000000001,"15min":1.5600000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.665},"last":1.2709999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576010,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60436,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311404,"statfs":{"total":96636764160,"available":96574877696,"internally_reserved":0,"allocated":2957312,"data_stored":1531521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.157,"15min":1.208},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.841},"last":1.351},{"interface":"front","average":{"1min":1.341,"5min":1.2290000000000001,"15min":1.232},"min":{"1min":0.83099999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7030000000000001,"5min":1.8560000000000001,"15min":1.9610000000000001},"last":0.97999999999999998}]},{"osd":1,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2090000000000001,"15min":1.242},"min":{"1min":0.75800000000000001,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.6990000000000001,"5min":1.762,"15min":1.9550000000000001},"last":1.129},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2869999999999999,"15min":1.3},"min":{"1min":0.90900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.2130000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.871,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":2.0870000000000002},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.3149999999999999},"min":{"1min":0.77000000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5840000000000001,"5min":1.877,"15min":1.909},"last":1.008}]},{"osd":3,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":0.79800000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.1499999999999999},"last":1.294},{"interface":"front","average":{"1min":1.458,"5min":1.3740000000000001,"15min":1.431},"min":{"1min":0.67500000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8100000000000001,"5min":1.9530000000000001,"15min":2.173},"last":0.84599999999999997}]},{"osd":4,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3220000000000001,"15min":1.3280000000000001},"min":{"1min":0.84399999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7569999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.335},{"interface":"front","average":{"1min":1.556,"5min":1.446,"15min":1.4339999999999999},"min":{"1min":1.107,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.2240000000000002},"last":1.4059999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":1.0740000000000001,"5min":0.66500000000000004,"15min":0.57499999999999996},"max":{"1min":1.899,"5min":2.077,"15min":2.3759999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.478,"15min":1.4770000000000001},"min":{"1min":1.157,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0059999999999998,"5min":2.036,"15min":2.036},"last":1.264}]},{"osd":7,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.5289999999999999,"15min":1.552},"min":{"1min":1.177,"5min":0.72499999999999998,"15min":0.59999999999999998},"max":{"1min":2.298,"5min":2.298,"15min":2.4169999999999998},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.5840000000000001,"15min":1.6180000000000001},"min":{"1min":1.0089999999999999,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0009999999999999,"5min":2.2320000000000002,"15min":2.3199999999999998},"last":1.1699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576010,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47952,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323888,"statfs":{"total":96636764160,"available":96587661312,"internally_reserved":0,"allocated":2953216,"data_stored":1532047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2849999999999999,"15min":1.3440000000000001},"min":{"1min":0.64200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":2.9980000000000002},{"interface":"front","average":{"1min":1.367,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":0.62,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.9139999999999999,"5min":1.976,"15min":1.976},"last":7.0880000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2629999999999999,"15min":1.2889999999999999},"min":{"1min":0.85599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7450000000000001,"5min":1.976,"15min":1.976},"last":2.177},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.244},"min":{"1min":0.72499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7330000000000001,"5min":1.8660000000000001,"15min":1.875},"last":6.9020000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2689999999999999,"15min":1.266},"min":{"1min":0.77400000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":17.175999999999998},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.365,"15min":1.369},"min":{"1min":0.99299999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":7.1849999999999996}]},{"osd":3,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2789999999999999,"15min":1.3149999999999999},"min":{"1min":0.88,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.702,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":5.8289999999999997},{"interface":"front","average":{"1min":1.464,"5min":1.4410000000000001,"15min":1.4219999999999999},"min":{"1min":0.874,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":7.7599999999999998}]},{"osd":5,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.391},"min":{"1min":1.032,"5min":0.372,"15min":0.372},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":7.4020000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4239999999999999,"15min":1.3919999999999999},"min":{"1min":0.96799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.865,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":3.2549999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.5009999999999999,"15min":1.46},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2530000000000001,"5min":2.2719999999999998,"15min":2.6829999999999998},"last":10.542999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.593,"15min":1.5609999999999999},"min":{"1min":1.071,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.117,"5min":2.3500000000000001,"15min":2.7429999999999999},"last":10.433999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.5369999999999999,"15min":1.516},"min":{"1min":1.1220000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0649999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":3.1070000000000002},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.681,"15min":1.6439999999999999},"min":{"1min":1.3169999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.3439999999999999,"5min":2.508,"15min":2.508},"last":3.6459999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576013,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52168,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319672,"statfs":{"total":96636764160,"available":96583344128,"internally_reserved":0,"allocated":3002368,"data_stored":1554869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.67800000000000005,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.087,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.123,"5min":1.1559999999999999,"15min":1.181},"min":{"1min":0.748,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.607,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.70099999999999996}]},{"osd":1,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.417,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.2390000000000001,"15min":1.2869999999999999},"min":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4390000000000001,"5min":2.109,"15min":2.4889999999999999},"last":0.74299999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.123,"15min":1.151},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.667,"5min":2.165,"15min":2.2250000000000001},"last":0.62},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.177,"15min":1.202},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5349999999999999,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2549999999999999,"15min":1.282},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.67500000000000004},"max":{"1min":1.51,"5min":1.857,"15min":2.5379999999999998},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4179999999999999,"15min":1.46},"min":{"1min":0.93600000000000005,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.954,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":0.98599999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3200000000000001,"15min":1.3759999999999999},"min":{"1min":0.97899999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8300000000000001,"5min":1.923,"15min":2.9740000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.419,"15min":1.4339999999999999},"min":{"1min":0.77800000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8380000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4099999999999999,"15min":1.4239999999999999},"min":{"1min":0.91000000000000003,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6339999999999999,"5min":1.976,"15min":2.653},"last":1.02},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4710000000000001,"15min":1.5269999999999999},"min":{"1min":1.0960000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8540000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.0529999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5660000000000001},"min":{"1min":0.88800000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9450000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.5209999999999999},"min":{"1min":1.202,"5min":0.628,"15min":0.628},"max":{"1min":1.7889999999999999,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.2110000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576011,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":2998272,"data_stored":1555427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.97299999999999998},"min":{"1min":0.55300000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.948},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.054,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":0.745}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.96099999999999997,"15min":1.0029999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.198,"5min":1.6279999999999999,"15min":2.044},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.171,"15min":1.1970000000000001},"min":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.677,"5min":1.714,"15min":2.1309999999999998},"last":0.86799999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68400000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4770000000000001,"5min":1.78,"15min":1.8420000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.2410000000000001,"15min":1.2769999999999999},"min":{"1min":0.66200000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.458,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.165}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3959999999999999,"15min":1.4399999999999999},"min":{"1min":0.89000000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.821,"5min":1.929,"15min":2.1789999999999998},"last":1.754},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":1.0189999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6879999999999999,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.577}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3819999999999999,"15min":1.411},"min":{"1min":1.0069999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.603,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":1.083},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5009999999999999,"15min":1.5349999999999999},"min":{"1min":1.0269999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.946,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.5489999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":0.997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.73,"5min":2.056,"15min":2.1240000000000001},"last":1.472},{"interface":"front","average":{"1min":1.47,"5min":1.4950000000000001,"15min":1.534},"min":{"1min":1.1439999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.778,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":1.7190000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4690000000000001,"15min":1.486},"min":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7609999999999999,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":1.643},{"interface":"front","average":{"1min":1.5,"5min":1.5309999999999999,"15min":1.5860000000000001},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.863,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":1.623}]}]},{"osd":1,"up_from":13,"seq":55834576010,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52108,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319732,"statfs":{"total":96636764160,"available":96583405568,"internally_reserved":0,"allocated":2949120,"data_stored":1531465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.024,"15min":1.014},"min":{"1min":0.81000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.6200000000000001,"5min":1.6579999999999999,"15min":1.7769999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.224},"min":{"1min":0.88,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":1.79},"last":0.89500000000000002}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.97599999999999998},"min":{"1min":0.73399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2809999999999999,"5min":1.641,"15min":1.641},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.155,"15min":1.161},"min":{"1min":0.872,"5min":0.372,"15min":0.372},"max":{"1min":1.4710000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.95799999999999996}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.2430000000000001,"15min":1.264},"min":{"1min":1.052,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5800000000000001,"5min":1.651,"15min":1.7809999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1659999999999999,"15min":1.155},"min":{"1min":0.82599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.72,"5min":1.883,"15min":2.7429999999999999},"last":0.77200000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":1.117,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.7110000000000001,"5min":1.7829999999999999,"15min":2.8460000000000001},"last":1.022},{"interface":"front","average":{"1min":1.357,"5min":1.3129999999999999,"15min":1.3220000000000001},"min":{"1min":1.0960000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.5449999999999999,"5min":1.6879999999999999,"15min":2.0139999999999998},"last":1.3280000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.347,"15min":1.379},"min":{"1min":0.97299999999999998,"5min":0.92600000000000005,"15min":0.76700000000000002},"max":{"1min":1.6639999999999999,"5min":1.827,"15min":2.0470000000000002},"last":1.456},{"interface":"front","average":{"1min":1.637,"5min":1.546,"15min":1.534},"min":{"1min":1.286,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.2610000000000001},"last":1.667}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5089999999999999,"15min":1.5149999999999999},"min":{"1min":1.2549999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.694,"5min":2.3199999999999998,"15min":2.9079999999999999},"last":1.48},{"interface":"front","average":{"1min":1.514,"5min":1.4870000000000001,"15min":1.4990000000000001},"min":{"1min":1.1679999999999999,"5min":1.046,"15min":1.0309999999999999},"max":{"1min":1.7929999999999999,"5min":1.966,"15min":2.3199999999999998},"last":1.8180000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.5780000000000001,"15min":1.556},"min":{"1min":1.2250000000000001,"5min":0.94499999999999995,"15min":0.84599999999999997},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.786},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.5529999999999999,"15min":1.5720000000000001},"min":{"1min":1.377,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.798,"5min":2.1699999999999999,"15min":2.9590000000000001},"last":1.268}]}]},{"osd":0,"up_from":13,"seq":55834576012,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61048,"kb_used_data":3500,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310792,"statfs":{"total":96636764160,"available":96574251008,"internally_reserved":0,"allocated":3584000,"data_stored":2145179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.48299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3859999999999999,"5min":1.581,"15min":1.581},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.01,"5min":1.103,"15min":1.109},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.621,"5min":1.6619999999999999,"15min":1.726},"last":0.874}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1060000000000001,"15min":1.1419999999999999},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0329999999999999,"15min":1.073},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4199999999999999,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":0.93300000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.248,"15min":1.256},"min":{"1min":0.69799999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.546,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.579,"5min":1.752,"15min":2.0979999999999999},"last":1.4259999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3220000000000001,"15min":1.349},"min":{"1min":0.83399999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.389,"15min":1.4179999999999999},"min":{"1min":0.69699999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.782,"5min":1.972,"15min":2.181},"last":2.1520000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.423,"15min":1.417},"min":{"1min":0.79600000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.732,"5min":1.919,"15min":2.1309999999999998},"last":1.706},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.482,"15min":1.486},"min":{"1min":0.79300000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.966,"5min":2.2170000000000001,"15min":2.4900000000000002},"last":1.9930000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.81299999999999994,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7669999999999999,"5min":2.21,"15min":2.2149999999999999},"last":1.925},{"interface":"front","average":{"1min":1.397,"5min":1.506,"15min":1.5089999999999999},"min":{"1min":0.78000000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9430000000000001,"5min":2.0880000000000001,"15min":2.355},"last":1.7529999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5389999999999999,"15min":1.5469999999999999},"min":{"1min":0.83499999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9410000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.5289999999999999,"15min":1.554},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":2.0750000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:13:54.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:13:54.594 INFO:tasks.ceph.mds.d.smithi181.stderr:2023-10-08T07:13:54.594+0000 7f155728e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:13:54.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.636+0000 7f41205b0640 1 Processor -- start 2023-10-08T07:13:54.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.636+0000 7f41205b0640 1 -- start start 2023-10-08T07:13:54.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.638+0000 7f41205b0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f41180add60 0x7f4118162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.638+0000 7f41205b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f411806d4c0 0x7f41180cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.638+0000 7f41205b0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41180ae140 con 0x7f41180cb150 2023-10-08T07:13:54.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.638+0000 7f41205b0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4118161f10 con 0x7f41180add60 2023-10-08T07:13:54.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.638+0000 7f41205b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4118161630 con 0x7f411806d4c0 2023-10-08T07:13:54.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.638+0000 7f411e34c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f41180add60 0x7f4118162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.638+0000 7f411eb4d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f41180cb150 0x7f4118164d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47740/0 (socket says 172.21.15.70:47740) 2023-10-08T07:13:54.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.638+0000 7f411eb4d640 1 -- 172.21.15.70:0/3103442844 learned_addr learned my addr 172.21.15.70:0/3103442844 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:54.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.638+0000 7f411db4b640 1 --2- 172.21.15.70:0/3103442844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f411806d4c0 0x7f41180cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.638+0000 7f411e34c640 1 -- 172.21.15.70:0/3103442844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f411806d4c0 msgr2=0x7f41180cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:54.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.639+0000 7f411e34c640 1 --2- 172.21.15.70:0/3103442844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f411806d4c0 0x7f41180cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.639+0000 7f411e34c640 1 -- 172.21.15.70:0/3103442844 >> v1:172.21.15.70:6789/0 conn(0x7f41180cb150 legacy=0x7f4118164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.639+0000 7f411e34c640 1 -- 172.21.15.70:0/3103442844 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41180cd1c0 con 0x7f41180add60 2023-10-08T07:13:54.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.639+0000 7f411d34a640 1 -- 172.21.15.70:0/3103442844 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204658976 0 0) 0x7f41180ae140 con 0x7f41180cb150 2023-10-08T07:13:54.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.639+0000 7f411db4b640 1 --2- 172.21.15.70:0/3103442844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f411806d4c0 0x7f41180cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:54.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.640+0000 7f411e34c640 1 --2- 172.21.15.70:0/3103442844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180add60 0x7f4118162330 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f41140097a0 tx=0x7f411402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51605d59d5a16416 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:54.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.640+0000 7f411d34a640 1 -- 172.21.15.70:0/3103442844 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114031780 con 0x7f41180add60 2023-10-08T07:13:54.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.640+0000 7f411d34a640 1 -- 172.21.15.70:0/3103442844 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41140318e0 con 0x7f41180add60 2023-10-08T07:13:54.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.640+0000 7f411d34a640 1 -- 172.21.15.70:0/3103442844 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114031c70 con 0x7f41180add60 2023-10-08T07:13:54.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.640+0000 7f41205b0640 1 -- 172.21.15.70:0/3103442844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180add60 msgr2=0x7f4118162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.640+0000 7f41205b0640 1 --2- 172.21.15.70:0/3103442844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180add60 0x7f4118162330 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f41140097a0 tx=0x7f411402b820 comp rx=0 tx=0).stop 2023-10-08T07:13:54.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.641+0000 7f41205b0640 1 -- 172.21.15.70:0/3103442844 shutdown_connections 2023-10-08T07:13:54.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.641+0000 7f41205b0640 1 --2- 172.21.15.70:0/3103442844 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f411806d4c0 0x7f41180cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.641+0000 7f41205b0640 1 --2- 172.21.15.70:0/3103442844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180add60 0x7f4118162330 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.641+0000 7f41205b0640 1 -- 172.21.15.70:0/3103442844 >> 172.21.15.70:0/3103442844 conn(0x7f411806a380 msgr2=0x7f41180b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:54.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.641+0000 7f41205b0640 1 -- 172.21.15.70:0/3103442844 shutdown_connections 2023-10-08T07:13:54.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.641+0000 7f41205b0640 1 -- 172.21.15.70:0/3103442844 wait complete. 2023-10-08T07:13:54.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.642+0000 7f41205b0640 1 Processor -- start 2023-10-08T07:13:54.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.643+0000 7f41205b0640 1 -- start start 2023-10-08T07:13:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f41205b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f411806d4c0 0x7f41180c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f41205b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41180add60 0x7f41180b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f41205b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180cb150 0x7f4118150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f41205b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41180c60a0 con 0x7f41180cb150 2023-10-08T07:13:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f41205b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41180c6380 con 0x7f411806d4c0 2023-10-08T07:13:54.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f41205b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41180c6660 con 0x7f41180add60 2023-10-08T07:13:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f411eb4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180cb150 0x7f4118150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f411eb4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180cb150 0x7f4118150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49946/0 (socket says 172.21.15.70:49946) 2023-10-08T07:13:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f411eb4d640 1 -- 172.21.15.70:0/1070385357 learned_addr learned my addr 172.21.15.70:0/1070385357 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f411db4b640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41180add60 0x7f41180b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.644+0000 7f411e34c640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f411806d4c0 0x7f41180c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f411eb4d640 1 -- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41180add60 msgr2=0x7f41180b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f411eb4d640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41180add60 0x7f41180b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f411eb4d640 1 -- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f411806d4c0 msgr2=0x7f41180c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f411eb4d640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f411806d4c0 0x7f41180c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f411eb4d640 1 -- 172.21.15.70:0/1070385357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114009400 con 0x7f41180cb150 2023-10-08T07:13:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f411eb4d640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180cb150 0x7f4118150600 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f411000c700 tx=0x7f411000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f410effd640 1 -- 172.21.15.70:0/1070385357 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411000ce10 con 0x7f41180cb150 2023-10-08T07:13:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f410effd640 1 -- 172.21.15.70:0/1070385357 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411000a640 con 0x7f41180cb150 2023-10-08T07:13:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f410effd640 1 -- 172.21.15.70:0/1070385357 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411000aa70 con 0x7f41180cb150 2023-10-08T07:13:54.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f41205b0640 1 -- 172.21.15.70:0/1070385357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41180c6940 con 0x7f41180cb150 2023-10-08T07:13:54.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.645+0000 7f41205b0640 1 -- 172.21.15.70:0/1070385357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41180c5d90 con 0x7f41180cb150 2023-10-08T07:13:54.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.646+0000 7f41205b0640 1 -- 172.21.15.70:0/1070385357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41180c6e60 con 0x7f41180cb150 2023-10-08T07:13:54.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.647+0000 7f410effd640 1 -- 172.21.15.70:0/1070385357 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f411000abd0 con 0x7f41180cb150 2023-10-08T07:13:54.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.649+0000 7f410effd640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f40fc06af90 0x7f40fc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:54.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.649+0000 7f410effd640 1 -- 172.21.15.70:0/1070385357 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f41100891e0 con 0x7f41180cb150 2023-10-08T07:13:54.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.650+0000 7f411e34c640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f40fc06af90 0x7f40fc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:54.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.651+0000 7f411e34c640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f40fc06af90 0x7f40fc06d450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4114036000 tx=0x7f4114008030 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.654+0000 7f410effd640 1 -- 172.21.15.70:0/1070385357 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4110053bc0 con 0x7f41180cb150 2023-10-08T07:13:54.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.839+0000 7f41205b0640 1 -- 172.21.15.70:0/1070385357 --> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4118161b30 con 0x7f40fc06af90 2023-10-08T07:13:54.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.876+0000 7f410effd640 1 -- 172.21.15.70:0/1070385357 <== mgr.7328 v2:172.21.15.181:6832/3993370366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+522986 (secure 0 0 0) 0x7f4114031ad0 con 0x7f40fc06af90 2023-10-08T07:13:54.884 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:54.887 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:13:54.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 -- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f40fc06af90 msgr2=0x7f40fc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f40fc06af90 0x7f40fc06d450 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4114036000 tx=0x7f4114008030 comp rx=0 tx=0).stop 2023-10-08T07:13:54.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 -- 172.21.15.70:0/1070385357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180cb150 msgr2=0x7f4118150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:54.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180cb150 0x7f4118150600 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f411000c700 tx=0x7f411000cbc0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 -- 172.21.15.70:0/1070385357 shutdown_connections 2023-10-08T07:13:54.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f40fc06af90 0x7f40fc06d450 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41180cb150 0x7f4118150600 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f41180add60 0x7f41180b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 --2- 172.21.15.70:0/1070385357 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f411806d4c0 0x7f41180c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:54.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 -- 172.21.15.70:0/1070385357 >> 172.21.15.70:0/1070385357 conn(0x7f411806a380 msgr2=0x7f41181626f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:54.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.884+0000 7f41205b0640 1 -- 172.21.15.70:0/1070385357 shutdown_connections 2023-10-08T07:13:54.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:54.885+0000 7f41205b0640 1 -- 172.21.15.70:0/1070385357 wait complete. 2023-10-08T07:13:54.906 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2023-10-08T07:13:53.510178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594627,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":644,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2854,"ondisk_log_size":2854,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":275,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":433468,"kb_used_data":24396,"kb_used_omap":12,"kb_used_meta":408947,"kb_avail":754541252,"statfs":{"total":773094113280,"available":772650242048,"internally_reserved":0,"allocated":24981504,"data_stored":13517744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":418762299},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1591,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001985"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359204+0000","last_change":"2023-10-08T07:13:49.313140+0000","last_active":"2023-10-08T07:13:50.359204+0000","last_peered":"2023-10-08T07:13:50.359204+0000","last_clean":"2023-10-08T07:13:50.359204+0000","last_became_active":"2023-10-08T07:13:49.312820+0000","last_became_peered":"2023-10-08T07:13:49.312820+0000","last_unstale":"2023-10-08T07:13:50.359204+0000","last_undegraded":"2023-10-08T07:13:50.359204+0000","last_fullsized":"2023-10-08T07:13:50.359204+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:41.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"222'2806","reported_seq":3725,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.350548+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:13:47.350548+0000","last_peered":"2023-10-08T07:13:47.350548+0000","last_clean":"2023-10-08T07:13:47.350548+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:13:47.350548+0000","last_undegraded":"2023-10-08T07:13:47.350548+0000","last_fullsized":"2023-10-08T07:13:47.350548+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2806,"log_dups_size":0,"ondisk_log_size":2806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289208+0000","last_change":"2023-10-08T07:13:33.718816+0000","last_active":"2023-10-08T07:13:49.289208+0000","last_peered":"2023-10-08T07:13:49.289208+0000","last_clean":"2023-10-08T07:13:49.289208+0000","last_became_active":"2023-10-08T07:13:33.718494+0000","last_became_peered":"2023-10-08T07:13:33.718494+0000","last_unstale":"2023-10-08T07:13:49.289208+0000","last_undegraded":"2023-10-08T07:13:49.289208+0000","last_fullsized":"2023-10-08T07:13:49.289208+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:03.021625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344810+0000","last_change":"2023-10-08T07:13:34.304364+0000","last_active":"2023-10-08T07:13:47.344810+0000","last_peered":"2023-10-08T07:13:47.344810+0000","last_clean":"2023-10-08T07:13:47.344810+0000","last_became_active":"2023-10-08T07:13:34.304067+0000","last_became_peered":"2023-10-08T07:13:34.304067+0000","last_unstale":"2023-10-08T07:13:47.344810+0000","last_undegraded":"2023-10-08T07:13:47.344810+0000","last_fullsized":"2023-10-08T07:13:47.344810+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:56.084275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.385321+0000","last_change":"2023-10-08T07:13:48.383423+0000","last_active":"2023-10-08T07:13:48.385321+0000","last_peered":"2023-10-08T07:13:48.385321+0000","last_clean":"2023-10-08T07:13:48.385321+0000","last_became_active":"2023-10-08T07:13:48.383120+0000","last_became_peered":"2023-10-08T07:13:48.383120+0000","last_unstale":"2023-10-08T07:13:48.385321+0000","last_undegraded":"2023-10-08T07:13:48.385321+0000","last_fullsized":"2023-10-08T07:13:48.385321+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:30.632556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.377733+0000","last_change":"2023-10-08T07:13:48.377733+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.377733+0000","last_undegraded":"2023-10-08T07:13:48.377733+0000","last_fullsized":"2023-10-08T07:13:48.377733+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.278218+0000","last_change":"2023-10-08T07:13:33.735924+0000","last_active":"2023-10-08T07:13:49.278218+0000","last_peered":"2023-10-08T07:13:49.278218+0000","last_clean":"2023-10-08T07:13:49.278218+0000","last_became_active":"2023-10-08T07:13:33.735688+0000","last_became_peered":"2023-10-08T07:13:33.735688+0000","last_unstale":"2023-10-08T07:13:49.278218+0000","last_undegraded":"2023-10-08T07:13:49.278218+0000","last_fullsized":"2023-10-08T07:13:49.278218+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348742+0000","last_change":"2023-10-08T07:13:34.308734+0000","last_active":"2023-10-08T07:13:47.348742+0000","last_peered":"2023-10-08T07:13:47.348742+0000","last_clean":"2023-10-08T07:13:47.348742+0000","last_became_active":"2023-10-08T07:13:34.308195+0000","last_became_peered":"2023-10-08T07:13:34.308195+0000","last_unstale":"2023-10-08T07:13:47.348742+0000","last_undegraded":"2023-10-08T07:13:47.348742+0000","last_fullsized":"2023-10-08T07:13:47.348742+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.387859+0000","last_change":"2023-10-08T07:13:48.387261+0000","last_active":"2023-10-08T07:13:48.387859+0000","last_peered":"2023-10-08T07:13:48.387859+0000","last_clean":"2023-10-08T07:13:48.387859+0000","last_became_active":"2023-10-08T07:13:48.387022+0000","last_became_peered":"2023-10-08T07:13:48.387022+0000","last_unstale":"2023-10-08T07:13:48.387859+0000","last_undegraded":"2023-10-08T07:13:48.387859+0000","last_fullsized":"2023-10-08T07:13:48.387859+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:48.064204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448409+0000","last_change":"2023-10-08T07:13:49.303091+0000","last_active":"2023-10-08T07:13:50.448409+0000","last_peered":"2023-10-08T07:13:50.448409+0000","last_clean":"2023-10-08T07:13:50.448409+0000","last_became_active":"2023-10-08T07:13:49.302761+0000","last_became_peered":"2023-10-08T07:13:49.302761+0000","last_unstale":"2023-10-08T07:13:50.448409+0000","last_undegraded":"2023-10-08T07:13:50.448409+0000","last_fullsized":"2023-10-08T07:13:50.448409+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:54.844015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344232+0000","last_change":"2023-10-08T07:13:33.741115+0000","last_active":"2023-10-08T07:13:47.344232+0000","last_peered":"2023-10-08T07:13:47.344232+0000","last_clean":"2023-10-08T07:13:47.344232+0000","last_became_active":"2023-10-08T07:13:33.740798+0000","last_became_peered":"2023-10-08T07:13:33.740798+0000","last_unstale":"2023-10-08T07:13:47.344232+0000","last_undegraded":"2023-10-08T07:13:47.344232+0000","last_fullsized":"2023-10-08T07:13:47.344232+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:36.728485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.352692+0000","last_change":"2023-10-08T07:13:34.303009+0000","last_active":"2023-10-08T07:13:47.352692+0000","last_peered":"2023-10-08T07:13:47.352692+0000","last_clean":"2023-10-08T07:13:47.352692+0000","last_became_active":"2023-10-08T07:13:34.302682+0000","last_became_peered":"2023-10-08T07:13:34.302682+0000","last_unstale":"2023-10-08T07:13:47.352692+0000","last_undegraded":"2023-10-08T07:13:47.352692+0000","last_fullsized":"2023-10-08T07:13:47.352692+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:47:40.976539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.393834+0000","last_change":"2023-10-08T07:13:48.388490+0000","last_active":"2023-10-08T07:13:48.393834+0000","last_peered":"2023-10-08T07:13:48.393834+0000","last_clean":"2023-10-08T07:13:48.393834+0000","last_became_active":"2023-10-08T07:13:48.388178+0000","last_became_peered":"2023-10-08T07:13:48.388178+0000","last_unstale":"2023-10-08T07:13:48.393834+0000","last_undegraded":"2023-10-08T07:13:48.393834+0000","last_fullsized":"2023-10-08T07:13:48.393834+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:18.667320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278865+0000","last_change":"2023-10-08T07:13:49.305322+0000","last_active":"2023-10-08T07:13:50.278865+0000","last_peered":"2023-10-08T07:13:50.278865+0000","last_clean":"2023-10-08T07:13:50.278865+0000","last_became_active":"2023-10-08T07:13:49.305047+0000","last_became_peered":"2023-10-08T07:13:49.305047+0000","last_unstale":"2023-10-08T07:13:50.278865+0000","last_undegraded":"2023-10-08T07:13:50.278865+0000","last_fullsized":"2023-10-08T07:13:50.278865+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:20.461769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289335+0000","last_change":"2023-10-08T07:13:33.738792+0000","last_active":"2023-10-08T07:13:49.289335+0000","last_peered":"2023-10-08T07:13:49.289335+0000","last_clean":"2023-10-08T07:13:49.289335+0000","last_became_active":"2023-10-08T07:13:33.738430+0000","last_became_peered":"2023-10-08T07:13:33.738430+0000","last_unstale":"2023-10-08T07:13:49.289335+0000","last_undegraded":"2023-10-08T07:13:49.289335+0000","last_fullsized":"2023-10-08T07:13:49.289335+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:24:39.688018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.353511+0000","last_change":"2023-10-08T07:13:34.305875+0000","last_active":"2023-10-08T07:13:47.353511+0000","last_peered":"2023-10-08T07:13:47.353511+0000","last_clean":"2023-10-08T07:13:47.353511+0000","last_became_active":"2023-10-08T07:13:34.302605+0000","last_became_peered":"2023-10-08T07:13:34.302605+0000","last_unstale":"2023-10-08T07:13:47.353511+0000","last_undegraded":"2023-10-08T07:13:47.353511+0000","last_fullsized":"2023-10-08T07:13:47.353511+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:22.929719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396356+0000","last_change":"2023-10-08T07:13:48.391812+0000","last_active":"2023-10-08T07:13:48.396356+0000","last_peered":"2023-10-08T07:13:48.396356+0000","last_clean":"2023-10-08T07:13:48.396356+0000","last_became_active":"2023-10-08T07:13:48.391532+0000","last_became_peered":"2023-10-08T07:13:48.391532+0000","last_unstale":"2023-10-08T07:13:48.396356+0000","last_undegraded":"2023-10-08T07:13:48.396356+0000","last_fullsized":"2023-10-08T07:13:48.396356+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:39:10.917417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.372366+0000","last_change":"2023-10-08T07:13:48.372366+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.372366+0000","last_undegraded":"2023-10-08T07:13:48.372366+0000","last_fullsized":"2023-10-08T07:13:48.372366+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.361372+0000","last_change":"2023-10-08T07:13:33.734220+0000","last_active":"2023-10-08T07:13:47.361372+0000","last_peered":"2023-10-08T07:13:47.361372+0000","last_clean":"2023-10-08T07:13:47.361372+0000","last_became_active":"2023-10-08T07:13:33.733905+0000","last_became_peered":"2023-10-08T07:13:33.733905+0000","last_unstale":"2023-10-08T07:13:47.361372+0000","last_undegraded":"2023-10-08T07:13:47.361372+0000","last_fullsized":"2023-10-08T07:13:47.361372+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359420+0000","last_change":"2023-10-08T07:13:34.302227+0000","last_active":"2023-10-08T07:13:47.359420+0000","last_peered":"2023-10-08T07:13:47.359420+0000","last_clean":"2023-10-08T07:13:47.359420+0000","last_became_active":"2023-10-08T07:13:34.300127+0000","last_became_peered":"2023-10-08T07:13:34.300127+0000","last_unstale":"2023-10-08T07:13:47.359420+0000","last_undegraded":"2023-10-08T07:13:47.359420+0000","last_fullsized":"2023-10-08T07:13:47.359420+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:49.729707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.382534+0000","last_change":"2023-10-08T07:13:48.380775+0000","last_active":"2023-10-08T07:13:48.382534+0000","last_peered":"2023-10-08T07:13:48.382534+0000","last_clean":"2023-10-08T07:13:48.382534+0000","last_became_active":"2023-10-08T07:13:48.380501+0000","last_became_peered":"2023-10-08T07:13:48.380501+0000","last_unstale":"2023-10-08T07:13:48.382534+0000","last_undegraded":"2023-10-08T07:13:48.382534+0000","last_fullsized":"2023-10-08T07:13:48.382534+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:07.670172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357767+0000","last_change":"2023-10-08T07:13:49.309072+0000","last_active":"2023-10-08T07:13:50.357767+0000","last_peered":"2023-10-08T07:13:50.357767+0000","last_clean":"2023-10-08T07:13:50.357767+0000","last_became_active":"2023-10-08T07:13:49.308753+0000","last_became_peered":"2023-10-08T07:13:49.308753+0000","last_unstale":"2023-10-08T07:13:50.357767+0000","last_undegraded":"2023-10-08T07:13:50.357767+0000","last_fullsized":"2023-10-08T07:13:50.357767+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:27.798388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344852+0000","last_change":"2023-10-08T07:13:33.733207+0000","last_active":"2023-10-08T07:13:47.344852+0000","last_peered":"2023-10-08T07:13:47.344852+0000","last_clean":"2023-10-08T07:13:47.344852+0000","last_became_active":"2023-10-08T07:13:33.732903+0000","last_became_peered":"2023-10-08T07:13:33.732903+0000","last_unstale":"2023-10-08T07:13:47.344852+0000","last_undegraded":"2023-10-08T07:13:47.344852+0000","last_fullsized":"2023-10-08T07:13:47.344852+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:34.358428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343713+0000","last_change":"2023-10-08T07:13:34.297602+0000","last_active":"2023-10-08T07:13:47.343713+0000","last_peered":"2023-10-08T07:13:47.343713+0000","last_clean":"2023-10-08T07:13:47.343713+0000","last_became_active":"2023-10-08T07:13:34.296820+0000","last_became_peered":"2023-10-08T07:13:34.296820+0000","last_unstale":"2023-10-08T07:13:47.343713+0000","last_undegraded":"2023-10-08T07:13:47.343713+0000","last_fullsized":"2023-10-08T07:13:47.343713+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:07.812354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.380096+0000","last_change":"2023-10-08T07:13:48.374204+0000","last_active":"2023-10-08T07:13:48.380096+0000","last_peered":"2023-10-08T07:13:48.380096+0000","last_clean":"2023-10-08T07:13:48.380096+0000","last_became_active":"2023-10-08T07:13:48.373126+0000","last_became_peered":"2023-10-08T07:13:48.373126+0000","last_unstale":"2023-10-08T07:13:48.380096+0000","last_undegraded":"2023-10-08T07:13:48.380096+0000","last_fullsized":"2023-10-08T07:13:48.380096+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:18.667320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358171+0000","last_change":"2023-10-08T07:13:49.314030+0000","last_active":"2023-10-08T07:13:50.358171+0000","last_peered":"2023-10-08T07:13:50.358171+0000","last_clean":"2023-10-08T07:13:50.358171+0000","last_became_active":"2023-10-08T07:13:49.312711+0000","last_became_peered":"2023-10-08T07:13:49.312711+0000","last_unstale":"2023-10-08T07:13:50.358171+0000","last_undegraded":"2023-10-08T07:13:50.358171+0000","last_fullsized":"2023-10-08T07:13:50.358171+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:39.025998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.356696+0000","last_change":"2023-10-08T07:13:33.737211+0000","last_active":"2023-10-08T07:13:47.356696+0000","last_peered":"2023-10-08T07:13:47.356696+0000","last_clean":"2023-10-08T07:13:47.356696+0000","last_became_active":"2023-10-08T07:13:33.736904+0000","last_became_peered":"2023-10-08T07:13:33.736904+0000","last_unstale":"2023-10-08T07:13:47.356696+0000","last_undegraded":"2023-10-08T07:13:47.356696+0000","last_fullsized":"2023-10-08T07:13:47.356696+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:58.966472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288985+0000","last_change":"2023-10-08T07:13:34.302678+0000","last_active":"2023-10-08T07:13:49.288985+0000","last_peered":"2023-10-08T07:13:49.288985+0000","last_clean":"2023-10-08T07:13:49.288985+0000","last_became_active":"2023-10-08T07:13:34.302406+0000","last_became_peered":"2023-10-08T07:13:34.302406+0000","last_unstale":"2023-10-08T07:13:49.288985+0000","last_undegraded":"2023-10-08T07:13:49.288985+0000","last_fullsized":"2023-10-08T07:13:49.288985+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:48.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396347+0000","last_change":"2023-10-08T07:13:48.389369+0000","last_active":"2023-10-08T07:13:48.396347+0000","last_peered":"2023-10-08T07:13:48.396347+0000","last_clean":"2023-10-08T07:13:48.396347+0000","last_became_active":"2023-10-08T07:13:48.389067+0000","last_became_peered":"2023-10-08T07:13:48.389067+0000","last_unstale":"2023-10-08T07:13:48.396347+0000","last_undegraded":"2023-10-08T07:13:48.396347+0000","last_fullsized":"2023-10-08T07:13:48.396347+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:52:18.246155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.377624+0000","last_change":"2023-10-08T07:13:48.377624+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.377624+0000","last_undegraded":"2023-10-08T07:13:48.377624+0000","last_fullsized":"2023-10-08T07:13:48.377624+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"234'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359654+0000","last_change":"2023-10-08T07:13:33.742793+0000","last_active":"2023-10-08T07:13:47.359654+0000","last_peered":"2023-10-08T07:13:47.359654+0000","last_clean":"2023-10-08T07:13:47.359654+0000","last_became_active":"2023-10-08T07:13:33.742030+0000","last_became_peered":"2023-10-08T07:13:33.742030+0000","last_unstale":"2023-10-08T07:13:47.359654+0000","last_undegraded":"2023-10-08T07:13:47.359654+0000","last_fullsized":"2023-10-08T07:13:47.359654+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:25:59.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348010+0000","last_change":"2023-10-08T07:13:34.308581+0000","last_active":"2023-10-08T07:13:47.348010+0000","last_peered":"2023-10-08T07:13:47.348010+0000","last_clean":"2023-10-08T07:13:47.348010+0000","last_became_active":"2023-10-08T07:13:34.308210+0000","last_became_peered":"2023-10-08T07:13:34.308210+0000","last_unstale":"2023-10-08T07:13:47.348010+0000","last_undegraded":"2023-10-08T07:13:47.348010+0000","last_fullsized":"2023-10-08T07:13:47.348010+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396466+0000","last_change":"2023-10-08T07:13:48.391100+0000","last_active":"2023-10-08T07:13:48.396466+0000","last_peered":"2023-10-08T07:13:48.396466+0000","last_clean":"2023-10-08T07:13:48.396466+0000","last_became_active":"2023-10-08T07:13:48.390787+0000","last_became_peered":"2023-10-08T07:13:48.390787+0000","last_unstale":"2023-10-08T07:13:48.396466+0000","last_undegraded":"2023-10-08T07:13:48.396466+0000","last_fullsized":"2023-10-08T07:13:48.396466+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:13:47.089135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278157+0000","last_change":"2023-10-08T07:13:49.310297+0000","last_active":"2023-10-08T07:13:50.278157+0000","last_peered":"2023-10-08T07:13:50.278157+0000","last_clean":"2023-10-08T07:13:50.278157+0000","last_became_active":"2023-10-08T07:13:49.310021+0000","last_became_peered":"2023-10-08T07:13:49.310021+0000","last_unstale":"2023-10-08T07:13:50.278157+0000","last_undegraded":"2023-10-08T07:13:50.278157+0000","last_fullsized":"2023-10-08T07:13:50.278157+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:19.685263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292199+0000","last_change":"2023-10-08T07:13:33.739638+0000","last_active":"2023-10-08T07:13:49.292199+0000","last_peered":"2023-10-08T07:13:49.292199+0000","last_clean":"2023-10-08T07:13:49.292199+0000","last_became_active":"2023-10-08T07:13:33.739352+0000","last_became_peered":"2023-10-08T07:13:33.739352+0000","last_unstale":"2023-10-08T07:13:49.292199+0000","last_undegraded":"2023-10-08T07:13:49.292199+0000","last_fullsized":"2023-10-08T07:13:49.292199+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:11.117318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359899+0000","last_change":"2023-10-08T07:13:34.293674+0000","last_active":"2023-10-08T07:13:47.359899+0000","last_peered":"2023-10-08T07:13:47.359899+0000","last_clean":"2023-10-08T07:13:47.359899+0000","last_became_active":"2023-10-08T07:13:34.292983+0000","last_became_peered":"2023-10-08T07:13:34.292983+0000","last_unstale":"2023-10-08T07:13:47.359899+0000","last_undegraded":"2023-10-08T07:13:47.359899+0000","last_fullsized":"2023-10-08T07:13:47.359899+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:20.477902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.389923+0000","last_change":"2023-10-08T07:13:48.388173+0000","last_active":"2023-10-08T07:13:48.389923+0000","last_peered":"2023-10-08T07:13:48.389923+0000","last_clean":"2023-10-08T07:13:48.389923+0000","last_became_active":"2023-10-08T07:13:48.387881+0000","last_became_peered":"2023-10-08T07:13:48.387881+0000","last_unstale":"2023-10-08T07:13:48.389923+0000","last_undegraded":"2023-10-08T07:13:48.389923+0000","last_fullsized":"2023-10-08T07:13:48.389923+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:43.583782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.379456+0000","last_change":"2023-10-08T07:13:48.379456+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.379456+0000","last_undegraded":"2023-10-08T07:13:48.379456+0000","last_fullsized":"2023-10-08T07:13:48.379456+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.342892+0000","last_change":"2023-10-08T07:13:33.739626+0000","last_active":"2023-10-08T07:13:47.342892+0000","last_peered":"2023-10-08T07:13:47.342892+0000","last_clean":"2023-10-08T07:13:47.342892+0000","last_became_active":"2023-10-08T07:13:33.739348+0000","last_became_peered":"2023-10-08T07:13:33.739348+0000","last_unstale":"2023-10-08T07:13:47.342892+0000","last_undegraded":"2023-10-08T07:13:47.342892+0000","last_fullsized":"2023-10-08T07:13:47.342892+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:23.374596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.360313+0000","last_change":"2023-10-08T07:13:34.300038+0000","last_active":"2023-10-08T07:13:47.360313+0000","last_peered":"2023-10-08T07:13:47.360313+0000","last_clean":"2023-10-08T07:13:47.360313+0000","last_became_active":"2023-10-08T07:13:34.299721+0000","last_became_peered":"2023-10-08T07:13:34.299721+0000","last_unstale":"2023-10-08T07:13:47.360313+0000","last_undegraded":"2023-10-08T07:13:47.360313+0000","last_fullsized":"2023-10-08T07:13:47.360313+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:26.317918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.366615+0000","last_change":"2023-10-08T07:13:48.384247+0000","last_active":"2023-10-08T07:13:49.366615+0000","last_peered":"2023-10-08T07:13:49.366615+0000","last_clean":"2023-10-08T07:13:49.366615+0000","last_became_active":"2023-10-08T07:13:48.383959+0000","last_became_peered":"2023-10-08T07:13:48.383959+0000","last_unstale":"2023-10-08T07:13:49.366615+0000","last_undegraded":"2023-10-08T07:13:49.366615+0000","last_fullsized":"2023-10-08T07:13:49.366615+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:13.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448607+0000","last_change":"2023-10-08T07:13:49.319375+0000","last_active":"2023-10-08T07:13:50.448607+0000","last_peered":"2023-10-08T07:13:50.448607+0000","last_clean":"2023-10-08T07:13:50.448607+0000","last_became_active":"2023-10-08T07:13:49.319046+0000","last_became_peered":"2023-10-08T07:13:49.319046+0000","last_unstale":"2023-10-08T07:13:50.448607+0000","last_undegraded":"2023-10-08T07:13:50.448607+0000","last_fullsized":"2023-10-08T07:13:50.448607+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:43.401179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.354842+0000","last_change":"2023-10-08T07:13:33.742955+0000","last_active":"2023-10-08T07:13:47.354842+0000","last_peered":"2023-10-08T07:13:47.354842+0000","last_clean":"2023-10-08T07:13:47.354842+0000","last_became_active":"2023-10-08T07:13:33.742621+0000","last_became_peered":"2023-10-08T07:13:33.742621+0000","last_unstale":"2023-10-08T07:13:47.354842+0000","last_undegraded":"2023-10-08T07:13:47.354842+0000","last_fullsized":"2023-10-08T07:13:47.354842+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:25.311911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291806+0000","last_change":"2023-10-08T07:13:34.294206+0000","last_active":"2023-10-08T07:13:49.291806+0000","last_peered":"2023-10-08T07:13:49.291806+0000","last_clean":"2023-10-08T07:13:49.291806+0000","last_became_active":"2023-10-08T07:13:34.293897+0000","last_became_peered":"2023-10-08T07:13:34.293897+0000","last_unstale":"2023-10-08T07:13:49.291806+0000","last_undegraded":"2023-10-08T07:13:49.291806+0000","last_fullsized":"2023-10-08T07:13:49.291806+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370513+0000","last_change":"2023-10-08T07:13:48.392407+0000","last_active":"2023-10-08T07:13:49.370513+0000","last_peered":"2023-10-08T07:13:49.370513+0000","last_clean":"2023-10-08T07:13:49.370513+0000","last_became_active":"2023-10-08T07:13:48.392147+0000","last_became_peered":"2023-10-08T07:13:48.392147+0000","last_unstale":"2023-10-08T07:13:49.370513+0000","last_undegraded":"2023-10-08T07:13:49.370513+0000","last_fullsized":"2023-10-08T07:13:49.370513+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:54:27.098972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376428+0000","last_change":"2023-10-08T07:13:48.376428+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376428+0000","last_undegraded":"2023-10-08T07:13:48.376428+0000","last_fullsized":"2023-10-08T07:13:48.376428+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295620+0000","last_change":"2023-10-08T07:13:33.726092+0000","last_active":"2023-10-08T07:13:49.295620+0000","last_peered":"2023-10-08T07:13:49.295620+0000","last_clean":"2023-10-08T07:13:49.295620+0000","last_became_active":"2023-10-08T07:13:33.725701+0000","last_became_peered":"2023-10-08T07:13:33.725701+0000","last_unstale":"2023-10-08T07:13:49.295620+0000","last_undegraded":"2023-10-08T07:13:49.295620+0000","last_fullsized":"2023-10-08T07:13:49.295620+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:51.631016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.353306+0000","last_change":"2023-10-08T07:13:34.301017+0000","last_active":"2023-10-08T07:13:47.353306+0000","last_peered":"2023-10-08T07:13:47.353306+0000","last_clean":"2023-10-08T07:13:47.353306+0000","last_became_active":"2023-10-08T07:13:34.300720+0000","last_became_peered":"2023-10-08T07:13:34.300720+0000","last_unstale":"2023-10-08T07:13:47.353306+0000","last_undegraded":"2023-10-08T07:13:47.353306+0000","last_fullsized":"2023-10-08T07:13:47.353306+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:36:05.460057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.395613+0000","last_change":"2023-10-08T07:13:48.389158+0000","last_active":"2023-10-08T07:13:48.395613+0000","last_peered":"2023-10-08T07:13:48.395613+0000","last_clean":"2023-10-08T07:13:48.395613+0000","last_became_active":"2023-10-08T07:13:48.388809+0000","last_became_peered":"2023-10-08T07:13:48.388809+0000","last_unstale":"2023-10-08T07:13:48.395613+0000","last_undegraded":"2023-10-08T07:13:48.395613+0000","last_fullsized":"2023-10-08T07:13:48.395613+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:30.625357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359126+0000","last_change":"2023-10-08T07:13:49.310553+0000","last_active":"2023-10-08T07:13:50.359126+0000","last_peered":"2023-10-08T07:13:50.359126+0000","last_clean":"2023-10-08T07:13:50.359126+0000","last_became_active":"2023-10-08T07:13:49.310222+0000","last_became_peered":"2023-10-08T07:13:49.310222+0000","last_unstale":"2023-10-08T07:13:50.359126+0000","last_undegraded":"2023-10-08T07:13:50.359126+0000","last_fullsized":"2023-10-08T07:13:50.359126+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:50.676315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.283072+0000","last_change":"2023-10-08T07:13:33.726781+0000","last_active":"2023-10-08T07:13:49.283072+0000","last_peered":"2023-10-08T07:13:49.283072+0000","last_clean":"2023-10-08T07:13:49.283072+0000","last_became_active":"2023-10-08T07:13:33.726459+0000","last_became_peered":"2023-10-08T07:13:33.726459+0000","last_unstale":"2023-10-08T07:13:49.283072+0000","last_undegraded":"2023-10-08T07:13:49.283072+0000","last_fullsized":"2023-10-08T07:13:49.283072+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:04.357904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295826+0000","last_change":"2023-10-08T07:13:34.294315+0000","last_active":"2023-10-08T07:13:49.295826+0000","last_peered":"2023-10-08T07:13:49.295826+0000","last_clean":"2023-10-08T07:13:49.295826+0000","last_became_active":"2023-10-08T07:13:34.294065+0000","last_became_peered":"2023-10-08T07:13:34.294065+0000","last_unstale":"2023-10-08T07:13:49.295826+0000","last_undegraded":"2023-10-08T07:13:49.295826+0000","last_fullsized":"2023-10-08T07:13:49.295826+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:37.124047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396254+0000","last_change":"2023-10-08T07:13:48.389005+0000","last_active":"2023-10-08T07:13:48.396254+0000","last_peered":"2023-10-08T07:13:48.396254+0000","last_clean":"2023-10-08T07:13:48.396254+0000","last_became_active":"2023-10-08T07:13:48.388729+0000","last_became_peered":"2023-10-08T07:13:48.388729+0000","last_unstale":"2023-10-08T07:13:48.396254+0000","last_undegraded":"2023-10-08T07:13:48.396254+0000","last_fullsized":"2023-10-08T07:13:48.396254+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:13.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.373866+0000","last_change":"2023-10-08T07:13:48.373866+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.373866+0000","last_undegraded":"2023-10-08T07:13:48.373866+0000","last_fullsized":"2023-10-08T07:13:48.373866+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282248+0000","last_change":"2023-10-08T07:13:33.729567+0000","last_active":"2023-10-08T07:13:49.282248+0000","last_peered":"2023-10-08T07:13:49.282248+0000","last_clean":"2023-10-08T07:13:49.282248+0000","last_became_active":"2023-10-08T07:13:33.729125+0000","last_became_peered":"2023-10-08T07:13:33.729125+0000","last_unstale":"2023-10-08T07:13:49.282248+0000","last_undegraded":"2023-10-08T07:13:49.282248+0000","last_fullsized":"2023-10-08T07:13:49.282248+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:13.193093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.361253+0000","last_change":"2023-10-08T07:13:34.308836+0000","last_active":"2023-10-08T07:13:47.361253+0000","last_peered":"2023-10-08T07:13:47.361253+0000","last_clean":"2023-10-08T07:13:47.361253+0000","last_became_active":"2023-10-08T07:13:34.308301+0000","last_became_peered":"2023-10-08T07:13:34.308301+0000","last_unstale":"2023-10-08T07:13:47.361253+0000","last_undegraded":"2023-10-08T07:13:47.361253+0000","last_fullsized":"2023-10-08T07:13:47.361253+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:42.058240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.391244+0000","last_change":"2023-10-08T07:13:48.390712+0000","last_active":"2023-10-08T07:13:48.391244+0000","last_peered":"2023-10-08T07:13:48.391244+0000","last_clean":"2023-10-08T07:13:48.391244+0000","last_became_active":"2023-10-08T07:13:48.390349+0000","last_became_peered":"2023-10-08T07:13:48.390349+0000","last_unstale":"2023-10-08T07:13:48.391244+0000","last_undegraded":"2023-10-08T07:13:48.391244+0000","last_fullsized":"2023-10-08T07:13:48.391244+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:50:08.228559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.367853+0000","last_change":"2023-10-08T07:13:48.367853+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.367853+0000","last_undegraded":"2023-10-08T07:13:48.367853+0000","last_fullsized":"2023-10-08T07:13:48.367853+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295073+0000","last_change":"2023-10-08T07:13:33.732139+0000","last_active":"2023-10-08T07:13:49.295073+0000","last_peered":"2023-10-08T07:13:49.295073+0000","last_clean":"2023-10-08T07:13:49.295073+0000","last_became_active":"2023-10-08T07:13:33.731892+0000","last_became_peered":"2023-10-08T07:13:33.731892+0000","last_unstale":"2023-10-08T07:13:49.295073+0000","last_undegraded":"2023-10-08T07:13:49.295073+0000","last_fullsized":"2023-10-08T07:13:49.295073+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:07.242491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343742+0000","last_change":"2023-10-08T07:13:34.299071+0000","last_active":"2023-10-08T07:13:47.343742+0000","last_peered":"2023-10-08T07:13:47.343742+0000","last_clean":"2023-10-08T07:13:47.343742+0000","last_became_active":"2023-10-08T07:13:34.298776+0000","last_became_peered":"2023-10-08T07:13:34.298776+0000","last_unstale":"2023-10-08T07:13:47.343742+0000","last_undegraded":"2023-10-08T07:13:47.343742+0000","last_fullsized":"2023-10-08T07:13:47.343742+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:10.642317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398369+0000","last_change":"2023-10-08T07:13:48.393982+0000","last_active":"2023-10-08T07:13:48.398369+0000","last_peered":"2023-10-08T07:13:48.398369+0000","last_clean":"2023-10-08T07:13:48.398369+0000","last_became_active":"2023-10-08T07:13:48.393696+0000","last_became_peered":"2023-10-08T07:13:48.393696+0000","last_unstale":"2023-10-08T07:13:48.398369+0000","last_undegraded":"2023-10-08T07:13:48.398369+0000","last_fullsized":"2023-10-08T07:13:48.398369+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:10.381086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358215+0000","last_change":"2023-10-08T07:13:49.312996+0000","last_active":"2023-10-08T07:13:50.358215+0000","last_peered":"2023-10-08T07:13:50.358215+0000","last_clean":"2023-10-08T07:13:50.358215+0000","last_became_active":"2023-10-08T07:13:49.312657+0000","last_became_peered":"2023-10-08T07:13:49.312657+0000","last_unstale":"2023-10-08T07:13:50.358215+0000","last_undegraded":"2023-10-08T07:13:50.358215+0000","last_fullsized":"2023-10-08T07:13:50.358215+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:28.367357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"233'2","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345211+0000","last_change":"2023-10-08T07:13:33.736960+0000","last_active":"2023-10-08T07:13:47.345211+0000","last_peered":"2023-10-08T07:13:47.345211+0000","last_clean":"2023-10-08T07:13:47.345211+0000","last_became_active":"2023-10-08T07:13:33.736703+0000","last_became_peered":"2023-10-08T07:13:33.736703+0000","last_unstale":"2023-10-08T07:13:47.345211+0000","last_undegraded":"2023-10-08T07:13:47.345211+0000","last_fullsized":"2023-10-08T07:13:47.345211+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:39.196152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343947+0000","last_change":"2023-10-08T07:13:34.291716+0000","last_active":"2023-10-08T07:13:47.343947+0000","last_peered":"2023-10-08T07:13:47.343947+0000","last_clean":"2023-10-08T07:13:47.343947+0000","last_became_active":"2023-10-08T07:13:34.291300+0000","last_became_peered":"2023-10-08T07:13:34.291300+0000","last_unstale":"2023-10-08T07:13:47.343947+0000","last_undegraded":"2023-10-08T07:13:47.343947+0000","last_fullsized":"2023-10-08T07:13:47.343947+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:07.189339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.277990+0000","last_change":"2023-10-08T07:13:48.383304+0000","last_active":"2023-10-08T07:13:49.277990+0000","last_peered":"2023-10-08T07:13:49.277990+0000","last_clean":"2023-10-08T07:13:49.277990+0000","last_became_active":"2023-10-08T07:13:48.383040+0000","last_became_peered":"2023-10-08T07:13:48.383040+0000","last_unstale":"2023-10-08T07:13:49.277990+0000","last_undegraded":"2023-10-08T07:13:49.277990+0000","last_fullsized":"2023-10-08T07:13:49.277990+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:05.475908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448022+0000","last_change":"2023-10-08T07:13:49.319374+0000","last_active":"2023-10-08T07:13:50.448022+0000","last_peered":"2023-10-08T07:13:50.448022+0000","last_clean":"2023-10-08T07:13:50.448022+0000","last_became_active":"2023-10-08T07:13:49.319049+0000","last_became_peered":"2023-10-08T07:13:49.319049+0000","last_unstale":"2023-10-08T07:13:50.448022+0000","last_undegraded":"2023-10-08T07:13:50.448022+0000","last_fullsized":"2023-10-08T07:13:50.448022+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:09.611966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345489+0000","last_change":"2023-10-08T07:13:33.728187+0000","last_active":"2023-10-08T07:13:47.345489+0000","last_peered":"2023-10-08T07:13:47.345489+0000","last_clean":"2023-10-08T07:13:47.345489+0000","last_became_active":"2023-10-08T07:13:33.727828+0000","last_became_peered":"2023-10-08T07:13:33.727828+0000","last_unstale":"2023-10-08T07:13:47.345489+0000","last_undegraded":"2023-10-08T07:13:47.345489+0000","last_fullsized":"2023-10-08T07:13:47.345489+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:00.539389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.360575+0000","last_change":"2023-10-08T07:13:34.293870+0000","last_active":"2023-10-08T07:13:47.360575+0000","last_peered":"2023-10-08T07:13:47.360575+0000","last_clean":"2023-10-08T07:13:47.360575+0000","last_became_active":"2023-10-08T07:13:34.293397+0000","last_became_peered":"2023-10-08T07:13:34.293397+0000","last_unstale":"2023-10-08T07:13:47.360575+0000","last_undegraded":"2023-10-08T07:13:47.360575+0000","last_fullsized":"2023-10-08T07:13:47.360575+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.387322+0000","last_change":"2023-10-08T07:13:48.385714+0000","last_active":"2023-10-08T07:13:48.387322+0000","last_peered":"2023-10-08T07:13:48.387322+0000","last_clean":"2023-10-08T07:13:48.387322+0000","last_became_active":"2023-10-08T07:13:48.385420+0000","last_became_peered":"2023-10-08T07:13:48.385420+0000","last_unstale":"2023-10-08T07:13:48.387322+0000","last_undegraded":"2023-10-08T07:13:48.387322+0000","last_fullsized":"2023-10-08T07:13:48.387322+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:51.558634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.378285+0000","last_change":"2023-10-08T07:13:48.378285+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.378285+0000","last_undegraded":"2023-10-08T07:13:48.378285+0000","last_fullsized":"2023-10-08T07:13:48.378285+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"233'8","reported_seq":24,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.278141+0000","last_change":"2023-10-08T07:13:33.738121+0000","last_active":"2023-10-08T07:13:49.278141+0000","last_peered":"2023-10-08T07:13:49.278141+0000","last_clean":"2023-10-08T07:13:49.278141+0000","last_became_active":"2023-10-08T07:13:33.737903+0000","last_became_peered":"2023-10-08T07:13:33.737903+0000","last_unstale":"2023-10-08T07:13:49.278141+0000","last_undegraded":"2023-10-08T07:13:49.278141+0000","last_fullsized":"2023-10-08T07:13:49.278141+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:15.275258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.355390+0000","last_change":"2023-10-08T07:13:34.306066+0000","last_active":"2023-10-08T07:13:47.355390+0000","last_peered":"2023-10-08T07:13:47.355390+0000","last_clean":"2023-10-08T07:13:47.355390+0000","last_became_active":"2023-10-08T07:13:34.302955+0000","last_became_peered":"2023-10-08T07:13:34.302955+0000","last_unstale":"2023-10-08T07:13:47.355390+0000","last_undegraded":"2023-10-08T07:13:47.355390+0000","last_fullsized":"2023-10-08T07:13:47.355390+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:03.795703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398039+0000","last_change":"2023-10-08T07:13:48.397214+0000","last_active":"2023-10-08T07:13:48.398039+0000","last_peered":"2023-10-08T07:13:48.398039+0000","last_clean":"2023-10-08T07:13:48.398039+0000","last_became_active":"2023-10-08T07:13:48.396880+0000","last_became_peered":"2023-10-08T07:13:48.396880+0000","last_unstale":"2023-10-08T07:13:48.398039+0000","last_undegraded":"2023-10-08T07:13:48.398039+0000","last_fullsized":"2023-10-08T07:13:48.398039+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:38:50.886917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.371621+0000","last_change":"2023-10-08T07:13:48.371621+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.371621+0000","last_undegraded":"2023-10-08T07:13:48.371621+0000","last_fullsized":"2023-10-08T07:13:48.371621+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286393+0000","last_change":"2023-10-08T07:13:33.730922+0000","last_active":"2023-10-08T07:13:49.286393+0000","last_peered":"2023-10-08T07:13:49.286393+0000","last_clean":"2023-10-08T07:13:49.286393+0000","last_became_active":"2023-10-08T07:13:33.730666+0000","last_became_peered":"2023-10-08T07:13:33.730666+0000","last_unstale":"2023-10-08T07:13:49.286393+0000","last_undegraded":"2023-10-08T07:13:49.286393+0000","last_fullsized":"2023-10-08T07:13:49.286393+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:12.086089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291729+0000","last_change":"2023-10-08T07:13:34.302276+0000","last_active":"2023-10-08T07:13:49.291729+0000","last_peered":"2023-10-08T07:13:49.291729+0000","last_clean":"2023-10-08T07:13:49.291729+0000","last_became_active":"2023-10-08T07:13:34.301988+0000","last_became_peered":"2023-10-08T07:13:34.301988+0000","last_unstale":"2023-10-08T07:13:49.291729+0000","last_undegraded":"2023-10-08T07:13:49.291729+0000","last_fullsized":"2023-10-08T07:13:49.291729+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.389272+0000","last_change":"2023-10-08T07:13:48.386216+0000","last_active":"2023-10-08T07:13:48.389272+0000","last_peered":"2023-10-08T07:13:48.389272+0000","last_clean":"2023-10-08T07:13:48.389272+0000","last_became_active":"2023-10-08T07:13:48.385877+0000","last_became_peered":"2023-10-08T07:13:48.385877+0000","last_unstale":"2023-10-08T07:13:48.389272+0000","last_undegraded":"2023-10-08T07:13:48.389272+0000","last_fullsized":"2023-10-08T07:13:48.389272+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:06.260935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.366477+0000","last_change":"2023-10-08T07:13:48.366477+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.366477+0000","last_undegraded":"2023-10-08T07:13:48.366477+0000","last_fullsized":"2023-10-08T07:13:48.366477+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348222+0000","last_change":"2023-10-08T07:13:33.725167+0000","last_active":"2023-10-08T07:13:47.348222+0000","last_peered":"2023-10-08T07:13:47.348222+0000","last_clean":"2023-10-08T07:13:47.348222+0000","last_became_active":"2023-10-08T07:13:33.724905+0000","last_became_peered":"2023-10-08T07:13:33.724905+0000","last_unstale":"2023-10-08T07:13:47.348222+0000","last_undegraded":"2023-10-08T07:13:47.348222+0000","last_fullsized":"2023-10-08T07:13:47.348222+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:37.525027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348897+0000","last_change":"2023-10-08T07:13:34.296170+0000","last_active":"2023-10-08T07:13:47.348897+0000","last_peered":"2023-10-08T07:13:47.348897+0000","last_clean":"2023-10-08T07:13:47.348897+0000","last_became_active":"2023-10-08T07:13:34.295736+0000","last_became_peered":"2023-10-08T07:13:34.295736+0000","last_unstale":"2023-10-08T07:13:47.348897+0000","last_undegraded":"2023-10-08T07:13:47.348897+0000","last_fullsized":"2023-10-08T07:13:47.348897+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396650+0000","last_change":"2023-10-08T07:13:48.395168+0000","last_active":"2023-10-08T07:13:48.396650+0000","last_peered":"2023-10-08T07:13:48.396650+0000","last_clean":"2023-10-08T07:13:48.396650+0000","last_became_active":"2023-10-08T07:13:48.394879+0000","last_became_peered":"2023-10-08T07:13:48.394879+0000","last_unstale":"2023-10-08T07:13:48.396650+0000","last_undegraded":"2023-10-08T07:13:48.396650+0000","last_fullsized":"2023-10-08T07:13:48.396650+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:07.520887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359261+0000","last_change":"2023-10-08T07:13:49.303768+0000","last_active":"2023-10-08T07:13:50.359261+0000","last_peered":"2023-10-08T07:13:50.359261+0000","last_clean":"2023-10-08T07:13:50.359261+0000","last_became_active":"2023-10-08T07:13:49.302929+0000","last_became_peered":"2023-10-08T07:13:49.302929+0000","last_unstale":"2023-10-08T07:13:50.359261+0000","last_undegraded":"2023-10-08T07:13:50.359261+0000","last_fullsized":"2023-10-08T07:13:50.359261+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:21.341547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295376+0000","last_change":"2023-10-08T07:13:33.730955+0000","last_active":"2023-10-08T07:13:49.295376+0000","last_peered":"2023-10-08T07:13:49.295376+0000","last_clean":"2023-10-08T07:13:49.295376+0000","last_became_active":"2023-10-08T07:13:33.730676+0000","last_became_peered":"2023-10-08T07:13:33.730676+0000","last_unstale":"2023-10-08T07:13:49.295376+0000","last_undegraded":"2023-10-08T07:13:49.295376+0000","last_fullsized":"2023-10-08T07:13:49.295376+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:28.953862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295664+0000","last_change":"2023-10-08T07:13:34.293585+0000","last_active":"2023-10-08T07:13:49.295664+0000","last_peered":"2023-10-08T07:13:49.295664+0000","last_clean":"2023-10-08T07:13:49.295664+0000","last_became_active":"2023-10-08T07:13:34.293318+0000","last_became_peered":"2023-10-08T07:13:34.293318+0000","last_unstale":"2023-10-08T07:13:49.295664+0000","last_undegraded":"2023-10-08T07:13:49.295664+0000","last_fullsized":"2023-10-08T07:13:49.295664+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:09:34.022529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.391979+0000","last_change":"2023-10-08T07:13:48.388878+0000","last_active":"2023-10-08T07:13:48.391979+0000","last_peered":"2023-10-08T07:13:48.391979+0000","last_clean":"2023-10-08T07:13:48.391979+0000","last_became_active":"2023-10-08T07:13:48.388595+0000","last_became_peered":"2023-10-08T07:13:48.388595+0000","last_unstale":"2023-10-08T07:13:48.391979+0000","last_undegraded":"2023-10-08T07:13:48.391979+0000","last_fullsized":"2023-10-08T07:13:48.391979+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:28.683424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.363741+0000","last_change":"2023-10-08T07:13:48.363741+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.363741+0000","last_undegraded":"2023-10-08T07:13:48.363741+0000","last_fullsized":"2023-10-08T07:13:48.363741+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"234'8","reported_seq":22,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.352112+0000","last_change":"2023-10-08T07:13:33.746002+0000","last_active":"2023-10-08T07:13:47.352112+0000","last_peered":"2023-10-08T07:13:47.352112+0000","last_clean":"2023-10-08T07:13:47.352112+0000","last_became_active":"2023-10-08T07:13:33.745697+0000","last_became_peered":"2023-10-08T07:13:33.745697+0000","last_unstale":"2023-10-08T07:13:47.352112+0000","last_undegraded":"2023-10-08T07:13:47.352112+0000","last_fullsized":"2023-10-08T07:13:47.352112+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:25.841040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.283013+0000","last_change":"2023-10-08T07:13:34.293551+0000","last_active":"2023-10-08T07:13:49.283013+0000","last_peered":"2023-10-08T07:13:49.283013+0000","last_clean":"2023-10-08T07:13:49.283013+0000","last_became_active":"2023-10-08T07:13:34.292471+0000","last_became_peered":"2023-10-08T07:13:34.292471+0000","last_unstale":"2023-10-08T07:13:49.283013+0000","last_undegraded":"2023-10-08T07:13:49.283013+0000","last_fullsized":"2023-10-08T07:13:49.283013+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288920+0000","last_change":"2023-10-08T07:13:48.373437+0000","last_active":"2023-10-08T07:13:49.288920+0000","last_peered":"2023-10-08T07:13:49.288920+0000","last_clean":"2023-10-08T07:13:49.288920+0000","last_became_active":"2023-10-08T07:13:48.373147+0000","last_became_peered":"2023-10-08T07:13:48.373147+0000","last_unstale":"2023-10-08T07:13:49.288920+0000","last_undegraded":"2023-10-08T07:13:49.288920+0000","last_fullsized":"2023-10-08T07:13:49.288920+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:07.670172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.447672+0000","last_change":"2023-10-08T07:13:49.311094+0000","last_active":"2023-10-08T07:13:50.447672+0000","last_peered":"2023-10-08T07:13:50.447672+0000","last_clean":"2023-10-08T07:13:50.447672+0000","last_became_active":"2023-10-08T07:13:49.310778+0000","last_became_peered":"2023-10-08T07:13:49.310778+0000","last_unstale":"2023-10-08T07:13:50.447672+0000","last_undegraded":"2023-10-08T07:13:50.447672+0000","last_fullsized":"2023-10-08T07:13:50.447672+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:25:13.654111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"233'3","reported_seq":19,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290932+0000","last_change":"2023-10-08T07:13:33.726791+0000","last_active":"2023-10-08T07:13:49.290932+0000","last_peered":"2023-10-08T07:13:49.290932+0000","last_clean":"2023-10-08T07:13:49.290932+0000","last_became_active":"2023-10-08T07:13:33.726482+0000","last_became_peered":"2023-10-08T07:13:33.726482+0000","last_unstale":"2023-10-08T07:13:49.290932+0000","last_undegraded":"2023-10-08T07:13:49.290932+0000","last_fullsized":"2023-10-08T07:13:49.290932+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:11.409539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289130+0000","last_change":"2023-10-08T07:13:34.290971+0000","last_active":"2023-10-08T07:13:49.289130+0000","last_peered":"2023-10-08T07:13:49.289130+0000","last_clean":"2023-10-08T07:13:49.289130+0000","last_became_active":"2023-10-08T07:13:34.290294+0000","last_became_peered":"2023-10-08T07:13:34.290294+0000","last_unstale":"2023-10-08T07:13:49.289130+0000","last_undegraded":"2023-10-08T07:13:49.289130+0000","last_fullsized":"2023-10-08T07:13:49.289130+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.393740+0000","last_change":"2023-10-08T07:13:48.388537+0000","last_active":"2023-10-08T07:13:48.393740+0000","last_peered":"2023-10-08T07:13:48.393740+0000","last_clean":"2023-10-08T07:13:48.393740+0000","last_became_active":"2023-10-08T07:13:48.388005+0000","last_became_peered":"2023-10-08T07:13:48.388005+0000","last_unstale":"2023-10-08T07:13:48.393740+0000","last_undegraded":"2023-10-08T07:13:48.393740+0000","last_fullsized":"2023-10-08T07:13:48.393740+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:50.761583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376614+0000","last_change":"2023-10-08T07:13:48.376614+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376614+0000","last_undegraded":"2023-10-08T07:13:48.376614+0000","last_fullsized":"2023-10-08T07:13:48.376614+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.352897+0000","last_change":"2023-10-08T07:13:33.736423+0000","last_active":"2023-10-08T07:13:47.352897+0000","last_peered":"2023-10-08T07:13:47.352897+0000","last_clean":"2023-10-08T07:13:47.352897+0000","last_became_active":"2023-10-08T07:13:33.735959+0000","last_became_peered":"2023-10-08T07:13:33.735959+0000","last_unstale":"2023-10-08T07:13:47.352897+0000","last_undegraded":"2023-10-08T07:13:47.352897+0000","last_fullsized":"2023-10-08T07:13:47.352897+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:00.365787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.347992+0000","last_change":"2023-10-08T07:13:34.302391+0000","last_active":"2023-10-08T07:13:47.347992+0000","last_peered":"2023-10-08T07:13:47.347992+0000","last_clean":"2023-10-08T07:13:47.347992+0000","last_became_active":"2023-10-08T07:13:34.302105+0000","last_became_peered":"2023-10-08T07:13:34.302105+0000","last_unstale":"2023-10-08T07:13:47.347992+0000","last_undegraded":"2023-10-08T07:13:47.347992+0000","last_fullsized":"2023-10-08T07:13:47.347992+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396485+0000","last_change":"2023-10-08T07:13:48.391735+0000","last_active":"2023-10-08T07:13:48.396485+0000","last_peered":"2023-10-08T07:13:48.396485+0000","last_clean":"2023-10-08T07:13:48.396485+0000","last_became_active":"2023-10-08T07:13:48.391330+0000","last_became_peered":"2023-10-08T07:13:48.391330+0000","last_unstale":"2023-10-08T07:13:48.396485+0000","last_undegraded":"2023-10-08T07:13:48.396485+0000","last_fullsized":"2023-10-08T07:13:48.396485+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:19.090961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357935+0000","last_change":"2023-10-08T07:13:49.313956+0000","last_active":"2023-10-08T07:13:50.357935+0000","last_peered":"2023-10-08T07:13:50.357935+0000","last_clean":"2023-10-08T07:13:50.357935+0000","last_became_active":"2023-10-08T07:13:49.313687+0000","last_became_peered":"2023-10-08T07:13:49.313687+0000","last_unstale":"2023-10-08T07:13:50.357935+0000","last_undegraded":"2023-10-08T07:13:50.357935+0000","last_fullsized":"2023-10-08T07:13:50.357935+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:35.653573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284639+0000","last_change":"2023-10-08T07:13:33.735184+0000","last_active":"2023-10-08T07:13:49.284639+0000","last_peered":"2023-10-08T07:13:49.284639+0000","last_clean":"2023-10-08T07:13:49.284639+0000","last_became_active":"2023-10-08T07:13:33.734915+0000","last_became_peered":"2023-10-08T07:13:33.734915+0000","last_unstale":"2023-10-08T07:13:49.284639+0000","last_undegraded":"2023-10-08T07:13:49.284639+0000","last_fullsized":"2023-10-08T07:13:49.284639+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:52.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.354637+0000","last_change":"2023-10-08T07:13:34.294204+0000","last_active":"2023-10-08T07:13:47.354637+0000","last_peered":"2023-10-08T07:13:47.354637+0000","last_clean":"2023-10-08T07:13:47.354637+0000","last_became_active":"2023-10-08T07:13:34.293887+0000","last_became_peered":"2023-10-08T07:13:34.293887+0000","last_unstale":"2023-10-08T07:13:47.354637+0000","last_undegraded":"2023-10-08T07:13:47.354637+0000","last_fullsized":"2023-10-08T07:13:47.354637+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:59.699025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.391072+0000","last_change":"2023-10-08T07:13:48.389962+0000","last_active":"2023-10-08T07:13:48.391072+0000","last_peered":"2023-10-08T07:13:48.391072+0000","last_clean":"2023-10-08T07:13:48.391072+0000","last_became_active":"2023-10-08T07:13:48.389607+0000","last_became_peered":"2023-10-08T07:13:48.389607+0000","last_unstale":"2023-10-08T07:13:48.391072+0000","last_undegraded":"2023-10-08T07:13:48.391072+0000","last_fullsized":"2023-10-08T07:13:48.391072+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:14.995707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358517+0000","last_change":"2023-10-08T07:13:49.299000+0000","last_active":"2023-10-08T07:13:50.358517+0000","last_peered":"2023-10-08T07:13:50.358517+0000","last_clean":"2023-10-08T07:13:50.358517+0000","last_became_active":"2023-10-08T07:13:49.298705+0000","last_became_peered":"2023-10-08T07:13:49.298705+0000","last_unstale":"2023-10-08T07:13:50.358517+0000","last_undegraded":"2023-10-08T07:13:50.358517+0000","last_fullsized":"2023-10-08T07:13:50.358517+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:46.814809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398278+0000","last_change":"2023-10-08T07:13:48.393308+0000","last_active":"2023-10-08T07:13:48.398278+0000","last_peered":"2023-10-08T07:13:48.398278+0000","last_clean":"2023-10-08T07:13:48.398278+0000","last_became_active":"2023-10-08T07:13:48.392985+0000","last_became_peered":"2023-10-08T07:13:48.392985+0000","last_unstale":"2023-10-08T07:13:48.398278+0000","last_undegraded":"2023-10-08T07:13:48.398278+0000","last_fullsized":"2023-10-08T07:13:48.398278+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:13.322938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.347589+0000","last_change":"2023-10-08T07:13:33.731054+0000","last_active":"2023-10-08T07:13:47.347589+0000","last_peered":"2023-10-08T07:13:47.347589+0000","last_clean":"2023-10-08T07:13:47.347589+0000","last_became_active":"2023-10-08T07:13:33.730782+0000","last_became_peered":"2023-10-08T07:13:33.730782+0000","last_unstale":"2023-10-08T07:13:47.347589+0000","last_undegraded":"2023-10-08T07:13:47.347589+0000","last_fullsized":"2023-10-08T07:13:47.347589+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:06.476515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292156+0000","last_change":"2023-10-08T07:13:34.294260+0000","last_active":"2023-10-08T07:13:49.292156+0000","last_peered":"2023-10-08T07:13:49.292156+0000","last_clean":"2023-10-08T07:13:49.292156+0000","last_became_active":"2023-10-08T07:13:34.293992+0000","last_became_peered":"2023-10-08T07:13:34.293992+0000","last_unstale":"2023-10-08T07:13:49.292156+0000","last_undegraded":"2023-10-08T07:13:49.292156+0000","last_fullsized":"2023-10-08T07:13:49.292156+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358287+0000","last_change":"2023-10-08T07:13:49.309351+0000","last_active":"2023-10-08T07:13:50.358287+0000","last_peered":"2023-10-08T07:13:50.358287+0000","last_clean":"2023-10-08T07:13:50.358287+0000","last_became_active":"2023-10-08T07:13:49.309030+0000","last_became_peered":"2023-10-08T07:13:49.309030+0000","last_unstale":"2023-10-08T07:13:50.358287+0000","last_undegraded":"2023-10-08T07:13:50.358287+0000","last_fullsized":"2023-10-08T07:13:50.358287+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:08.833825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"237'4","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.372634+0000","last_change":"2023-10-08T07:13:48.385188+0000","last_active":"2023-10-08T07:13:49.372634+0000","last_peered":"2023-10-08T07:13:49.372634+0000","last_clean":"2023-10-08T07:13:49.372634+0000","last_became_active":"2023-10-08T07:13:48.384790+0000","last_became_peered":"2023-10-08T07:13:48.384790+0000","last_unstale":"2023-10-08T07:13:49.372634+0000","last_undegraded":"2023-10-08T07:13:49.372634+0000","last_fullsized":"2023-10-08T07:13:49.372634+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:30.625357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.350898+0000","last_change":"2023-10-08T07:13:33.738155+0000","last_active":"2023-10-08T07:13:47.350898+0000","last_peered":"2023-10-08T07:13:47.350898+0000","last_clean":"2023-10-08T07:13:47.350898+0000","last_became_active":"2023-10-08T07:13:33.737850+0000","last_became_peered":"2023-10-08T07:13:33.737850+0000","last_unstale":"2023-10-08T07:13:47.350898+0000","last_undegraded":"2023-10-08T07:13:47.350898+0000","last_fullsized":"2023-10-08T07:13:47.350898+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:37.479840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.356485+0000","last_change":"2023-10-08T07:13:34.304888+0000","last_active":"2023-10-08T07:13:47.356485+0000","last_peered":"2023-10-08T07:13:47.356485+0000","last_clean":"2023-10-08T07:13:47.356485+0000","last_became_active":"2023-10-08T07:13:34.304536+0000","last_became_peered":"2023-10-08T07:13:34.304536+0000","last_unstale":"2023-10-08T07:13:47.356485+0000","last_undegraded":"2023-10-08T07:13:47.356485+0000","last_fullsized":"2023-10-08T07:13:47.356485+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:05.467048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.379499+0000","last_change":"2023-10-08T07:13:48.379499+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.379499+0000","last_undegraded":"2023-10-08T07:13:48.379499+0000","last_fullsized":"2023-10-08T07:13:48.379499+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.351392+0000","last_change":"2023-10-08T07:13:33.738971+0000","last_active":"2023-10-08T07:13:47.351392+0000","last_peered":"2023-10-08T07:13:47.351392+0000","last_clean":"2023-10-08T07:13:47.351392+0000","last_became_active":"2023-10-08T07:13:33.738532+0000","last_became_peered":"2023-10-08T07:13:33.738532+0000","last_unstale":"2023-10-08T07:13:47.351392+0000","last_undegraded":"2023-10-08T07:13:47.351392+0000","last_fullsized":"2023-10-08T07:13:47.351392+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:06.806347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.356107+0000","last_change":"2023-10-08T07:13:34.302593+0000","last_active":"2023-10-08T07:13:47.356107+0000","last_peered":"2023-10-08T07:13:47.356107+0000","last_clean":"2023-10-08T07:13:47.356107+0000","last_became_active":"2023-10-08T07:13:34.302300+0000","last_became_peered":"2023-10-08T07:13:34.302300+0000","last_unstale":"2023-10-08T07:13:47.356107+0000","last_undegraded":"2023-10-08T07:13:47.356107+0000","last_fullsized":"2023-10-08T07:13:47.356107+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:13:33.465222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.396596+0000","last_change":"2023-10-08T07:13:48.395886+0000","last_active":"2023-10-08T07:13:48.396596+0000","last_peered":"2023-10-08T07:13:48.396596+0000","last_clean":"2023-10-08T07:13:48.396596+0000","last_became_active":"2023-10-08T07:13:48.395585+0000","last_became_peered":"2023-10-08T07:13:48.395585+0000","last_unstale":"2023-10-08T07:13:48.396596+0000","last_undegraded":"2023-10-08T07:13:48.396596+0000","last_fullsized":"2023-10-08T07:13:48.396596+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:11.959208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.378495+0000","last_change":"2023-10-08T07:13:48.378495+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.378495+0000","last_undegraded":"2023-10-08T07:13:48.378495+0000","last_fullsized":"2023-10-08T07:13:48.378495+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290749+0000","last_change":"2023-10-08T07:13:33.736823+0000","last_active":"2023-10-08T07:13:49.290749+0000","last_peered":"2023-10-08T07:13:49.290749+0000","last_clean":"2023-10-08T07:13:49.290749+0000","last_became_active":"2023-10-08T07:13:33.736537+0000","last_became_peered":"2023-10-08T07:13:33.736537+0000","last_unstale":"2023-10-08T07:13:49.290749+0000","last_undegraded":"2023-10-08T07:13:49.290749+0000","last_fullsized":"2023-10-08T07:13:49.290749+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:48.798820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282798+0000","last_change":"2023-10-08T07:13:34.293354+0000","last_active":"2023-10-08T07:13:49.282798+0000","last_peered":"2023-10-08T07:13:49.282798+0000","last_clean":"2023-10-08T07:13:49.282798+0000","last_became_active":"2023-10-08T07:13:34.288220+0000","last_became_peered":"2023-10-08T07:13:34.288220+0000","last_unstale":"2023-10-08T07:13:49.282798+0000","last_undegraded":"2023-10-08T07:13:49.282798+0000","last_fullsized":"2023-10-08T07:13:49.282798+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.398331+0000","last_change":"2023-10-08T07:13:48.397371+0000","last_active":"2023-10-08T07:13:48.398331+0000","last_peered":"2023-10-08T07:13:48.398331+0000","last_clean":"2023-10-08T07:13:48.398331+0000","last_became_active":"2023-10-08T07:13:48.396954+0000","last_became_peered":"2023-10-08T07:13:48.396954+0000","last_unstale":"2023-10-08T07:13:48.398331+0000","last_undegraded":"2023-10-08T07:13:48.398331+0000","last_fullsized":"2023-10-08T07:13:48.398331+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:39:02.780468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358686+0000","last_change":"2023-10-08T07:13:49.309391+0000","last_active":"2023-10-08T07:13:50.358686+0000","last_peered":"2023-10-08T07:13:50.358686+0000","last_clean":"2023-10-08T07:13:50.358686+0000","last_became_active":"2023-10-08T07:13:49.309100+0000","last_became_peered":"2023-10-08T07:13:49.309100+0000","last_unstale":"2023-10-08T07:13:50.358686+0000","last_undegraded":"2023-10-08T07:13:50.358686+0000","last_fullsized":"2023-10-08T07:13:50.358686+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:36.841571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294941+0000","last_change":"2023-10-08T07:13:33.724335+0000","last_active":"2023-10-08T07:13:49.294941+0000","last_peered":"2023-10-08T07:13:49.294941+0000","last_clean":"2023-10-08T07:13:49.294941+0000","last_became_active":"2023-10-08T07:13:33.724043+0000","last_became_peered":"2023-10-08T07:13:33.724043+0000","last_unstale":"2023-10-08T07:13:49.294941+0000","last_undegraded":"2023-10-08T07:13:49.294941+0000","last_fullsized":"2023-10-08T07:13:49.294941+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:58.128241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359306+0000","last_change":"2023-10-08T07:13:34.301668+0000","last_active":"2023-10-08T07:13:47.359306+0000","last_peered":"2023-10-08T07:13:47.359306+0000","last_clean":"2023-10-08T07:13:47.359306+0000","last_became_active":"2023-10-08T07:13:34.298698+0000","last_became_peered":"2023-10-08T07:13:34.298698+0000","last_unstale":"2023-10-08T07:13:47.359306+0000","last_undegraded":"2023-10-08T07:13:47.359306+0000","last_fullsized":"2023-10-08T07:13:47.359306+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.399490+0000","last_change":"2023-10-08T07:13:48.399044+0000","last_active":"2023-10-08T07:13:48.399490+0000","last_peered":"2023-10-08T07:13:48.399490+0000","last_clean":"2023-10-08T07:13:48.399490+0000","last_became_active":"2023-10-08T07:13:48.398793+0000","last_became_peered":"2023-10-08T07:13:48.398793+0000","last_unstale":"2023-10-08T07:13:48.399490+0000","last_undegraded":"2023-10-08T07:13:48.399490+0000","last_fullsized":"2023-10-08T07:13:48.399490+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:49:58.133301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278331+0000","last_change":"2023-10-08T07:13:49.303740+0000","last_active":"2023-10-08T07:13:50.278331+0000","last_peered":"2023-10-08T07:13:50.278331+0000","last_clean":"2023-10-08T07:13:50.278331+0000","last_became_active":"2023-10-08T07:13:49.303416+0000","last_became_peered":"2023-10-08T07:13:49.303416+0000","last_unstale":"2023-10-08T07:13:50.278331+0000","last_undegraded":"2023-10-08T07:13:50.278331+0000","last_fullsized":"2023-10-08T07:13:50.278331+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:38:51.904860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345012+0000","last_change":"2023-10-08T07:13:33.732312+0000","last_active":"2023-10-08T07:13:47.345012+0000","last_peered":"2023-10-08T07:13:47.345012+0000","last_clean":"2023-10-08T07:13:47.345012+0000","last_became_active":"2023-10-08T07:13:33.731998+0000","last_became_peered":"2023-10-08T07:13:33.731998+0000","last_unstale":"2023-10-08T07:13:47.345012+0000","last_undegraded":"2023-10-08T07:13:47.345012+0000","last_fullsized":"2023-10-08T07:13:47.345012+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:10:53.964103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.343744+0000","last_change":"2023-10-08T07:13:34.293829+0000","last_active":"2023-10-08T07:13:47.343744+0000","last_peered":"2023-10-08T07:13:47.343744+0000","last_clean":"2023-10-08T07:13:47.343744+0000","last_became_active":"2023-10-08T07:13:34.293552+0000","last_became_peered":"2023-10-08T07:13:34.293552+0000","last_unstale":"2023-10-08T07:13:47.343744+0000","last_undegraded":"2023-10-08T07:13:47.343744+0000","last_fullsized":"2023-10-08T07:13:47.343744+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:38.485847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294683+0000","last_change":"2023-10-08T07:13:48.382427+0000","last_active":"2023-10-08T07:13:49.294683+0000","last_peered":"2023-10-08T07:13:49.294683+0000","last_clean":"2023-10-08T07:13:49.294683+0000","last_became_active":"2023-10-08T07:13:48.382141+0000","last_became_peered":"2023-10-08T07:13:48.382141+0000","last_unstale":"2023-10-08T07:13:49.294683+0000","last_undegraded":"2023-10-08T07:13:49.294683+0000","last_fullsized":"2023-10-08T07:13:49.294683+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:26.319310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.374502+0000","last_change":"2023-10-08T07:13:48.374502+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.374502+0000","last_undegraded":"2023-10-08T07:13:48.374502+0000","last_fullsized":"2023-10-08T07:13:48.374502+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"233'4","reported_seq":18,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.357270+0000","last_change":"2023-10-08T07:13:33.743017+0000","last_active":"2023-10-08T07:13:47.357270+0000","last_peered":"2023-10-08T07:13:47.357270+0000","last_clean":"2023-10-08T07:13:47.357270+0000","last_became_active":"2023-10-08T07:13:33.742582+0000","last_became_peered":"2023-10-08T07:13:33.742582+0000","last_unstale":"2023-10-08T07:13:47.357270+0000","last_undegraded":"2023-10-08T07:13:47.357270+0000","last_fullsized":"2023-10-08T07:13:47.357270+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:38:56.287497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296385+0000","last_change":"2023-10-08T07:13:34.286367+0000","last_active":"2023-10-08T07:13:49.296385+0000","last_peered":"2023-10-08T07:13:49.296385+0000","last_clean":"2023-10-08T07:13:49.296385+0000","last_became_active":"2023-10-08T07:13:34.285016+0000","last_became_peered":"2023-10-08T07:13:34.285016+0000","last_unstale":"2023-10-08T07:13:49.296385+0000","last_undegraded":"2023-10-08T07:13:49.296385+0000","last_fullsized":"2023-10-08T07:13:49.296385+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:41.466102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-10-08T07:13:48.397397+0000","last_change":"2023-10-08T07:13:48.394830+0000","last_active":"2023-10-08T07:13:48.397397+0000","last_peered":"2023-10-08T07:13:48.397397+0000","last_clean":"2023-10-08T07:13:48.397397+0000","last_became_active":"2023-10-08T07:13:48.394550+0000","last_became_peered":"2023-10-08T07:13:48.394550+0000","last_unstale":"2023-10-08T07:13:48.397397+0000","last_undegraded":"2023-10-08T07:13:48.397397+0000","last_fullsized":"2023-10-08T07:13:48.397397+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:33.349540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376167+0000","last_change":"2023-10-08T07:13:48.376167+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376167+0000","last_undegraded":"2023-10-08T07:13:48.376167+0000","last_fullsized":"2023-10-08T07:13:48.376167+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.349890+0000","last_change":"2023-10-08T07:13:34.295873+0000","last_active":"2023-10-08T07:13:47.349890+0000","last_peered":"2023-10-08T07:13:47.349890+0000","last_clean":"2023-10-08T07:13:47.349890+0000","last_became_active":"2023-10-08T07:13:34.295546+0000","last_became_peered":"2023-10-08T07:13:34.295546+0000","last_unstale":"2023-10-08T07:13:47.349890+0000","last_undegraded":"2023-10-08T07:13:47.349890+0000","last_fullsized":"2023-10-08T07:13:47.349890+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:46:53.896975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448531+0000","last_change":"2023-10-08T07:13:49.312717+0000","last_active":"2023-10-08T07:13:50.448531+0000","last_peered":"2023-10-08T07:13:50.448531+0000","last_clean":"2023-10-08T07:13:50.448531+0000","last_became_active":"2023-10-08T07:13:49.312448+0000","last_became_peered":"2023-10-08T07:13:49.312448+0000","last_unstale":"2023-10-08T07:13:50.448531+0000","last_undegraded":"2023-10-08T07:13:50.448531+0000","last_fullsized":"2023-10-08T07:13:50.448531+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:52.376348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.350665+0000","last_change":"2023-10-08T07:13:34.301071+0000","last_active":"2023-10-08T07:13:47.350665+0000","last_peered":"2023-10-08T07:13:47.350665+0000","last_clean":"2023-10-08T07:13:47.350665+0000","last_became_active":"2023-10-08T07:13:34.300772+0000","last_became_peered":"2023-10-08T07:13:34.300772+0000","last_unstale":"2023-10-08T07:13:47.350665+0000","last_undegraded":"2023-10-08T07:13:47.350665+0000","last_fullsized":"2023-10-08T07:13:47.350665+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:59:54.643977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.372376+0000","last_change":"2023-10-08T07:13:48.372376+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.372376+0000","last_undegraded":"2023-10-08T07:13:48.372376+0000","last_fullsized":"2023-10-08T07:13:48.372376+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.353002+0000","last_change":"2023-10-08T07:13:34.300974+0000","last_active":"2023-10-08T07:13:47.353002+0000","last_peered":"2023-10-08T07:13:47.353002+0000","last_clean":"2023-10-08T07:13:47.353002+0000","last_became_active":"2023-10-08T07:13:34.300529+0000","last_became_peered":"2023-10-08T07:13:34.300529+0000","last_unstale":"2023-10-08T07:13:47.353002+0000","last_undegraded":"2023-10-08T07:13:47.353002+0000","last_fullsized":"2023-10-08T07:13:47.353002+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:17.001444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278016+0000","last_change":"2023-10-08T07:13:49.306936+0000","last_active":"2023-10-08T07:13:50.278016+0000","last_peered":"2023-10-08T07:13:50.278016+0000","last_clean":"2023-10-08T07:13:50.278016+0000","last_became_active":"2023-10-08T07:13:49.306284+0000","last_became_peered":"2023-10-08T07:13:49.306284+0000","last_unstale":"2023-10-08T07:13:50.278016+0000","last_undegraded":"2023-10-08T07:13:50.278016+0000","last_fullsized":"2023-10-08T07:13:50.278016+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:51.779526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287748+0000","last_change":"2023-10-08T07:13:34.293712+0000","last_active":"2023-10-08T07:13:49.287748+0000","last_peered":"2023-10-08T07:13:49.287748+0000","last_clean":"2023-10-08T07:13:49.287748+0000","last_became_active":"2023-10-08T07:13:34.293415+0000","last_became_peered":"2023-10-08T07:13:34.293415+0000","last_unstale":"2023-10-08T07:13:49.287748+0000","last_undegraded":"2023-10-08T07:13:49.287748+0000","last_fullsized":"2023-10-08T07:13:49.287748+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:47.966787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359086+0000","last_change":"2023-10-08T07:13:49.316786+0000","last_active":"2023-10-08T07:13:50.359086+0000","last_peered":"2023-10-08T07:13:50.359086+0000","last_clean":"2023-10-08T07:13:50.359086+0000","last_became_active":"2023-10-08T07:13:49.316437+0000","last_became_peered":"2023-10-08T07:13:49.316437+0000","last_unstale":"2023-10-08T07:13:50.359086+0000","last_undegraded":"2023-10-08T07:13:50.359086+0000","last_fullsized":"2023-10-08T07:13:50.359086+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:14.040352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282033+0000","last_change":"2023-10-08T07:13:34.297816+0000","last_active":"2023-10-08T07:13:49.282033+0000","last_peered":"2023-10-08T07:13:49.282033+0000","last_clean":"2023-10-08T07:13:49.282033+0000","last_became_active":"2023-10-08T07:13:34.297544+0000","last_became_peered":"2023-10-08T07:13:34.297544+0000","last_unstale":"2023-10-08T07:13:49.282033+0000","last_undegraded":"2023-10-08T07:13:49.282033+0000","last_fullsized":"2023-10-08T07:13:49.282033+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.375861+0000","last_change":"2023-10-08T07:13:48.375861+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.375861+0000","last_undegraded":"2023-10-08T07:13:48.375861+0000","last_fullsized":"2023-10-08T07:13:48.375861+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348874+0000","last_change":"2023-10-08T07:13:34.310969+0000","last_active":"2023-10-08T07:13:47.348874+0000","last_peered":"2023-10-08T07:13:47.348874+0000","last_clean":"2023-10-08T07:13:47.348874+0000","last_became_active":"2023-10-08T07:13:34.310679+0000","last_became_peered":"2023-10-08T07:13:34.310679+0000","last_unstale":"2023-10-08T07:13:47.348874+0000","last_undegraded":"2023-10-08T07:13:47.348874+0000","last_fullsized":"2023-10-08T07:13:47.348874+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381042+0000","last_change":"2023-10-08T07:13:48.381042+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381042+0000","last_undegraded":"2023-10-08T07:13:48.381042+0000","last_fullsized":"2023-10-08T07:13:48.381042+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.344914+0000","last_change":"2023-10-08T07:13:34.304704+0000","last_active":"2023-10-08T07:13:47.344914+0000","last_peered":"2023-10-08T07:13:47.344914+0000","last_clean":"2023-10-08T07:13:47.344914+0000","last_became_active":"2023-10-08T07:13:34.304032+0000","last_became_peered":"2023-10-08T07:13:34.304032+0000","last_unstale":"2023-10-08T07:13:47.344914+0000","last_undegraded":"2023-10-08T07:13:47.344914+0000","last_fullsized":"2023-10-08T07:13:47.344914+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:15.818894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.367829+0000","last_change":"2023-10-08T07:13:48.367829+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.367829+0000","last_undegraded":"2023-10-08T07:13:48.367829+0000","last_fullsized":"2023-10-08T07:13:48.367829+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292077+0000","last_change":"2023-10-08T07:13:34.298953+0000","last_active":"2023-10-08T07:13:49.292077+0000","last_peered":"2023-10-08T07:13:49.292077+0000","last_clean":"2023-10-08T07:13:49.292077+0000","last_became_active":"2023-10-08T07:13:34.298694+0000","last_became_peered":"2023-10-08T07:13:34.298694+0000","last_unstale":"2023-10-08T07:13:49.292077+0000","last_undegraded":"2023-10-08T07:13:49.292077+0000","last_fullsized":"2023-10-08T07:13:49.292077+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376110+0000","last_change":"2023-10-08T07:13:48.376110+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376110+0000","last_undegraded":"2023-10-08T07:13:48.376110+0000","last_fullsized":"2023-10-08T07:13:48.376110+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.277869+0000","last_change":"2023-10-08T07:13:34.293797+0000","last_active":"2023-10-08T07:13:49.277869+0000","last_peered":"2023-10-08T07:13:49.277869+0000","last_clean":"2023-10-08T07:13:49.277869+0000","last_became_active":"2023-10-08T07:13:34.289224+0000","last_became_peered":"2023-10-08T07:13:34.289224+0000","last_unstale":"2023-10-08T07:13:49.277869+0000","last_undegraded":"2023-10-08T07:13:49.277869+0000","last_fullsized":"2023-10-08T07:13:49.277869+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357969+0000","last_change":"2023-10-08T07:13:49.303551+0000","last_active":"2023-10-08T07:13:50.357969+0000","last_peered":"2023-10-08T07:13:50.357969+0000","last_clean":"2023-10-08T07:13:50.357969+0000","last_became_active":"2023-10-08T07:13:49.303258+0000","last_became_peered":"2023-10-08T07:13:49.303258+0000","last_unstale":"2023-10-08T07:13:50.357969+0000","last_undegraded":"2023-10-08T07:13:50.357969+0000","last_fullsized":"2023-10-08T07:13:50.357969+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:08.504585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295174+0000","last_change":"2023-10-08T07:13:34.291585+0000","last_active":"2023-10-08T07:13:49.295174+0000","last_peered":"2023-10-08T07:13:49.295174+0000","last_clean":"2023-10-08T07:13:49.295174+0000","last_became_active":"2023-10-08T07:13:34.291125+0000","last_became_peered":"2023-10-08T07:13:34.291125+0000","last_unstale":"2023-10-08T07:13:49.295174+0000","last_undegraded":"2023-10-08T07:13:49.295174+0000","last_fullsized":"2023-10-08T07:13:49.295174+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:14.399506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.372077+0000","last_change":"2023-10-08T07:13:48.372077+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.372077+0000","last_undegraded":"2023-10-08T07:13:48.372077+0000","last_fullsized":"2023-10-08T07:13:48.372077+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359175+0000","last_change":"2023-10-08T07:13:34.297774+0000","last_active":"2023-10-08T07:13:47.359175+0000","last_peered":"2023-10-08T07:13:47.359175+0000","last_clean":"2023-10-08T07:13:47.359175+0000","last_became_active":"2023-10-08T07:13:34.297492+0000","last_became_peered":"2023-10-08T07:13:34.297492+0000","last_unstale":"2023-10-08T07:13:47.359175+0000","last_undegraded":"2023-10-08T07:13:47.359175+0000","last_fullsized":"2023-10-08T07:13:47.359175+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.277878+0000","last_change":"2023-10-08T07:13:49.298115+0000","last_active":"2023-10-08T07:13:50.277878+0000","last_peered":"2023-10-08T07:13:50.277878+0000","last_clean":"2023-10-08T07:13:50.277878+0000","last_became_active":"2023-10-08T07:13:49.297582+0000","last_became_peered":"2023-10-08T07:13:49.297582+0000","last_unstale":"2023-10-08T07:13:50.277878+0000","last_undegraded":"2023-10-08T07:13:50.277878+0000","last_fullsized":"2023-10-08T07:13:50.277878+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:11.399029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.361137+0000","last_change":"2023-10-08T07:13:34.294431+0000","last_active":"2023-10-08T07:13:47.361137+0000","last_peered":"2023-10-08T07:13:47.361137+0000","last_clean":"2023-10-08T07:13:47.361137+0000","last_became_active":"2023-10-08T07:13:34.294129+0000","last_became_peered":"2023-10-08T07:13:34.294129+0000","last_unstale":"2023-10-08T07:13:47.361137+0000","last_undegraded":"2023-10-08T07:13:47.361137+0000","last_fullsized":"2023-10-08T07:13:47.361137+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358949+0000","last_change":"2023-10-08T07:13:49.313393+0000","last_active":"2023-10-08T07:13:50.358949+0000","last_peered":"2023-10-08T07:13:50.358949+0000","last_clean":"2023-10-08T07:13:50.358949+0000","last_became_active":"2023-10-08T07:13:49.313094+0000","last_became_peered":"2023-10-08T07:13:49.313094+0000","last_unstale":"2023-10-08T07:13:50.358949+0000","last_undegraded":"2023-10-08T07:13:50.358949+0000","last_fullsized":"2023-10-08T07:13:50.358949+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:06.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359288+0000","last_change":"2023-10-08T07:13:34.297677+0000","last_active":"2023-10-08T07:13:47.359288+0000","last_peered":"2023-10-08T07:13:47.359288+0000","last_clean":"2023-10-08T07:13:47.359288+0000","last_became_active":"2023-10-08T07:13:34.296526+0000","last_became_peered":"2023-10-08T07:13:34.296526+0000","last_unstale":"2023-10-08T07:13:47.359288+0000","last_undegraded":"2023-10-08T07:13:47.359288+0000","last_fullsized":"2023-10-08T07:13:47.359288+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:51.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381122+0000","last_change":"2023-10-08T07:13:48.381122+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381122+0000","last_undegraded":"2023-10-08T07:13:48.381122+0000","last_fullsized":"2023-10-08T07:13:48.381122+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359714+0000","last_change":"2023-10-08T07:13:34.295222+0000","last_active":"2023-10-08T07:13:47.359714+0000","last_peered":"2023-10-08T07:13:47.359714+0000","last_clean":"2023-10-08T07:13:47.359714+0000","last_became_active":"2023-10-08T07:13:34.289517+0000","last_became_peered":"2023-10-08T07:13:34.289517+0000","last_unstale":"2023-10-08T07:13:47.359714+0000","last_undegraded":"2023-10-08T07:13:47.359714+0000","last_fullsized":"2023-10-08T07:13:47.359714+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:38.224838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.380405+0000","last_change":"2023-10-08T07:13:48.380405+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.380405+0000","last_undegraded":"2023-10-08T07:13:48.380405+0000","last_fullsized":"2023-10-08T07:13:48.380405+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291605+0000","last_change":"2023-10-08T07:13:34.299238+0000","last_active":"2023-10-08T07:13:49.291605+0000","last_peered":"2023-10-08T07:13:49.291605+0000","last_clean":"2023-10-08T07:13:49.291605+0000","last_became_active":"2023-10-08T07:13:34.298585+0000","last_became_peered":"2023-10-08T07:13:34.298585+0000","last_unstale":"2023-10-08T07:13:49.291605+0000","last_undegraded":"2023-10-08T07:13:49.291605+0000","last_fullsized":"2023-10-08T07:13:49.291605+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376641+0000","last_change":"2023-10-08T07:13:48.376641+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376641+0000","last_undegraded":"2023-10-08T07:13:48.376641+0000","last_fullsized":"2023-10-08T07:13:48.376641+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292035+0000","last_change":"2023-10-08T07:13:34.308624+0000","last_active":"2023-10-08T07:13:49.292035+0000","last_peered":"2023-10-08T07:13:49.292035+0000","last_clean":"2023-10-08T07:13:49.292035+0000","last_became_active":"2023-10-08T07:13:34.308246+0000","last_became_peered":"2023-10-08T07:13:34.308246+0000","last_unstale":"2023-10-08T07:13:49.292035+0000","last_undegraded":"2023-10-08T07:13:49.292035+0000","last_fullsized":"2023-10-08T07:13:49.292035+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381713+0000","last_change":"2023-10-08T07:13:48.381713+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381713+0000","last_undegraded":"2023-10-08T07:13:48.381713+0000","last_fullsized":"2023-10-08T07:13:48.381713+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.347720+0000","last_change":"2023-10-08T07:13:34.300821+0000","last_active":"2023-10-08T07:13:47.347720+0000","last_peered":"2023-10-08T07:13:47.347720+0000","last_clean":"2023-10-08T07:13:47.347720+0000","last_became_active":"2023-10-08T07:13:34.300558+0000","last_became_peered":"2023-10-08T07:13:34.300558+0000","last_unstale":"2023-10-08T07:13:47.347720+0000","last_undegraded":"2023-10-08T07:13:47.347720+0000","last_fullsized":"2023-10-08T07:13:47.347720+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.375275+0000","last_change":"2023-10-08T07:13:48.375275+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.375275+0000","last_undegraded":"2023-10-08T07:13:48.375275+0000","last_fullsized":"2023-10-08T07:13:48.375275+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.354342+0000","last_change":"2023-10-08T07:13:34.309619+0000","last_active":"2023-10-08T07:13:47.354342+0000","last_peered":"2023-10-08T07:13:47.354342+0000","last_clean":"2023-10-08T07:13:47.354342+0000","last_became_active":"2023-10-08T07:13:34.308202+0000","last_became_peered":"2023-10-08T07:13:34.308202+0000","last_unstale":"2023-10-08T07:13:47.354342+0000","last_undegraded":"2023-10-08T07:13:47.354342+0000","last_fullsized":"2023-10-08T07:13:47.354342+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:59.914799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.373371+0000","last_change":"2023-10-08T07:13:48.373371+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.373371+0000","last_undegraded":"2023-10-08T07:13:48.373371+0000","last_fullsized":"2023-10-08T07:13:48.373371+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359414+0000","last_change":"2023-10-08T07:13:34.298407+0000","last_active":"2023-10-08T07:13:47.359414+0000","last_peered":"2023-10-08T07:13:47.359414+0000","last_clean":"2023-10-08T07:13:47.359414+0000","last_became_active":"2023-10-08T07:13:34.298139+0000","last_became_peered":"2023-10-08T07:13:34.298139+0000","last_unstale":"2023-10-08T07:13:47.359414+0000","last_undegraded":"2023-10-08T07:13:47.359414+0000","last_fullsized":"2023-10-08T07:13:47.359414+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376338+0000","last_change":"2023-10-08T07:13:48.376338+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376338+0000","last_undegraded":"2023-10-08T07:13:48.376338+0000","last_fullsized":"2023-10-08T07:13:48.376338+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.351296+0000","last_change":"2023-10-08T07:13:34.306040+0000","last_active":"2023-10-08T07:13:47.351296+0000","last_peered":"2023-10-08T07:13:47.351296+0000","last_clean":"2023-10-08T07:13:47.351296+0000","last_became_active":"2023-10-08T07:13:34.303015+0000","last_became_peered":"2023-10-08T07:13:34.303015+0000","last_unstale":"2023-10-08T07:13:47.351296+0000","last_undegraded":"2023-10-08T07:13:47.351296+0000","last_fullsized":"2023-10-08T07:13:47.351296+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:42.884365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358958+0000","last_change":"2023-10-08T07:13:49.310325+0000","last_active":"2023-10-08T07:13:50.358958+0000","last_peered":"2023-10-08T07:13:50.358958+0000","last_clean":"2023-10-08T07:13:50.358958+0000","last_became_active":"2023-10-08T07:13:49.310037+0000","last_became_peered":"2023-10-08T07:13:49.310037+0000","last_unstale":"2023-10-08T07:13:50.358958+0000","last_undegraded":"2023-10-08T07:13:50.358958+0000","last_fullsized":"2023-10-08T07:13:50.358958+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:29.904246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.281819+0000","last_change":"2023-10-08T07:13:34.294253+0000","last_active":"2023-10-08T07:13:49.281819+0000","last_peered":"2023-10-08T07:13:49.281819+0000","last_clean":"2023-10-08T07:13:49.281819+0000","last_became_active":"2023-10-08T07:13:34.291997+0000","last_became_peered":"2023-10-08T07:13:34.291997+0000","last_unstale":"2023-10-08T07:13:49.281819+0000","last_undegraded":"2023-10-08T07:13:49.281819+0000","last_fullsized":"2023-10-08T07:13:49.281819+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278736+0000","last_change":"2023-10-08T07:13:49.298924+0000","last_active":"2023-10-08T07:13:50.278736+0000","last_peered":"2023-10-08T07:13:50.278736+0000","last_clean":"2023-10-08T07:13:50.278736+0000","last_became_active":"2023-10-08T07:13:49.298587+0000","last_became_peered":"2023-10-08T07:13:49.298587+0000","last_unstale":"2023-10-08T07:13:50.278736+0000","last_undegraded":"2023-10-08T07:13:50.278736+0000","last_fullsized":"2023-10-08T07:13:50.278736+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:34.367483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348653+0000","last_change":"2023-10-08T07:13:34.308930+0000","last_active":"2023-10-08T07:13:47.348653+0000","last_peered":"2023-10-08T07:13:47.348653+0000","last_clean":"2023-10-08T07:13:47.348653+0000","last_became_active":"2023-10-08T07:13:34.308283+0000","last_became_peered":"2023-10-08T07:13:34.308283+0000","last_unstale":"2023-10-08T07:13:47.348653+0000","last_undegraded":"2023-10-08T07:13:47.348653+0000","last_fullsized":"2023-10-08T07:13:47.348653+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.381921+0000","last_change":"2023-10-08T07:13:48.381921+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.381921+0000","last_undegraded":"2023-10-08T07:13:48.381921+0000","last_fullsized":"2023-10-08T07:13:48.381921+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359386+0000","last_change":"2023-10-08T07:13:34.295991+0000","last_active":"2023-10-08T07:13:47.359386+0000","last_peered":"2023-10-08T07:13:47.359386+0000","last_clean":"2023-10-08T07:13:47.359386+0000","last_became_active":"2023-10-08T07:13:34.295717+0000","last_became_peered":"2023-10-08T07:13:34.295717+0000","last_unstale":"2023-10-08T07:13:47.359386+0000","last_undegraded":"2023-10-08T07:13:47.359386+0000","last_fullsized":"2023-10-08T07:13:47.359386+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.797828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.368925+0000","last_change":"2023-10-08T07:13:48.368925+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.368925+0000","last_undegraded":"2023-10-08T07:13:48.368925+0000","last_fullsized":"2023-10-08T07:13:48.368925+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291913+0000","last_change":"2023-10-08T07:13:34.289375+0000","last_active":"2023-10-08T07:13:49.291913+0000","last_peered":"2023-10-08T07:13:49.291913+0000","last_clean":"2023-10-08T07:13:49.291913+0000","last_became_active":"2023-10-08T07:13:34.289115+0000","last_became_peered":"2023-10-08T07:13:34.289115+0000","last_unstale":"2023-10-08T07:13:49.291913+0000","last_undegraded":"2023-10-08T07:13:49.291913+0000","last_fullsized":"2023-10-08T07:13:49.291913+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.375367+0000","last_change":"2023-10-08T07:13:48.375367+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.375367+0000","last_undegraded":"2023-10-08T07:13:48.375367+0000","last_fullsized":"2023-10-08T07:13:48.375367+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.359704+0000","last_change":"2023-10-08T07:13:34.302018+0000","last_active":"2023-10-08T07:13:47.359704+0000","last_peered":"2023-10-08T07:13:47.359704+0000","last_clean":"2023-10-08T07:13:47.359704+0000","last_became_active":"2023-10-08T07:13:34.301747+0000","last_became_peered":"2023-10-08T07:13:34.301747+0000","last_unstale":"2023-10-08T07:13:47.359704+0000","last_undegraded":"2023-10-08T07:13:47.359704+0000","last_fullsized":"2023-10-08T07:13:47.359704+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.378259+0000","last_change":"2023-10-08T07:13:48.378259+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.378259+0000","last_undegraded":"2023-10-08T07:13:48.378259+0000","last_fullsized":"2023-10-08T07:13:48.378259+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290421+0000","last_change":"2023-10-08T07:13:34.293550+0000","last_active":"2023-10-08T07:13:49.290421+0000","last_peered":"2023-10-08T07:13:49.290421+0000","last_clean":"2023-10-08T07:13:49.290421+0000","last_became_active":"2023-10-08T07:13:34.293202+0000","last_became_peered":"2023-10-08T07:13:34.293202+0000","last_unstale":"2023-10-08T07:13:49.290421+0000","last_undegraded":"2023-10-08T07:13:49.290421+0000","last_fullsized":"2023-10-08T07:13:49.290421+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:00.842346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.374742+0000","last_change":"2023-10-08T07:13:48.374742+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.374742+0000","last_undegraded":"2023-10-08T07:13:48.374742+0000","last_fullsized":"2023-10-08T07:13:48.374742+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.355316+0000","last_change":"2023-10-08T07:13:34.298110+0000","last_active":"2023-10-08T07:13:47.355316+0000","last_peered":"2023-10-08T07:13:47.355316+0000","last_clean":"2023-10-08T07:13:47.355316+0000","last_became_active":"2023-10-08T07:13:34.297806+0000","last_became_peered":"2023-10-08T07:13:34.297806+0000","last_unstale":"2023-10-08T07:13:47.355316+0000","last_undegraded":"2023-10-08T07:13:47.355316+0000","last_fullsized":"2023-10-08T07:13:47.355316+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:58.335295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357781+0000","last_change":"2023-10-08T07:13:49.315887+0000","last_active":"2023-10-08T07:13:50.357781+0000","last_peered":"2023-10-08T07:13:50.357781+0000","last_clean":"2023-10-08T07:13:50.357781+0000","last_became_active":"2023-10-08T07:13:49.315571+0000","last_became_peered":"2023-10-08T07:13:49.315571+0000","last_unstale":"2023-10-08T07:13:50.357781+0000","last_undegraded":"2023-10-08T07:13:50.357781+0000","last_fullsized":"2023-10-08T07:13:50.357781+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:31.253205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282695+0000","last_change":"2023-10-08T07:13:34.298025+0000","last_active":"2023-10-08T07:13:49.282695+0000","last_peered":"2023-10-08T07:13:49.282695+0000","last_clean":"2023-10-08T07:13:49.282695+0000","last_became_active":"2023-10-08T07:13:34.296181+0000","last_became_peered":"2023-10-08T07:13:34.296181+0000","last_unstale":"2023-10-08T07:13:49.282695+0000","last_undegraded":"2023-10-08T07:13:49.282695+0000","last_fullsized":"2023-10-08T07:13:49.282695+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448978+0000","last_change":"2023-10-08T07:13:49.300145+0000","last_active":"2023-10-08T07:13:50.448978+0000","last_peered":"2023-10-08T07:13:50.448978+0000","last_clean":"2023-10-08T07:13:50.448978+0000","last_became_active":"2023-10-08T07:13:49.299846+0000","last_became_peered":"2023-10-08T07:13:49.299846+0000","last_unstale":"2023-10-08T07:13:50.448978+0000","last_undegraded":"2023-10-08T07:13:50.448978+0000","last_fullsized":"2023-10-08T07:13:50.448978+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:16.187252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.351855+0000","last_change":"2023-10-08T07:13:34.301326+0000","last_active":"2023-10-08T07:13:47.351855+0000","last_peered":"2023-10-08T07:13:47.351855+0000","last_clean":"2023-10-08T07:13:47.351855+0000","last_became_active":"2023-10-08T07:13:34.299365+0000","last_became_peered":"2023-10-08T07:13:34.299365+0000","last_unstale":"2023-10-08T07:13:47.351855+0000","last_undegraded":"2023-10-08T07:13:47.351855+0000","last_fullsized":"2023-10-08T07:13:47.351855+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:02.576748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.376508+0000","last_change":"2023-10-08T07:13:48.376508+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.376508+0000","last_undegraded":"2023-10-08T07:13:48.376508+0000","last_fullsized":"2023-10-08T07:13:48.376508+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.348325+0000","last_change":"2023-10-08T07:13:34.306582+0000","last_active":"2023-10-08T07:13:47.348325+0000","last_peered":"2023-10-08T07:13:47.348325+0000","last_clean":"2023-10-08T07:13:47.348325+0000","last_became_active":"2023-10-08T07:13:34.306310+0000","last_became_peered":"2023-10-08T07:13:34.306310+0000","last_unstale":"2023-10-08T07:13:47.348325+0000","last_undegraded":"2023-10-08T07:13:47.348325+0000","last_fullsized":"2023-10-08T07:13:47.348325+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":1,"reported_epoch":236,"state":"creating+peering","last_fresh":"2023-10-08T07:13:48.373866+0000","last_change":"2023-10-08T07:13:48.373866+0000","last_active":"2023-10-08T07:13:48.335144+0000","last_peered":"2023-10-08T07:13:48.335144+0000","last_clean":"2023-10-08T07:13:48.335144+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:13:48.373866+0000","last_undegraded":"2023-10-08T07:13:48.373866+0000","last_fullsized":"2023-10-08T07:13:48.373866+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298001+0000","last_change":"2023-10-08T07:13:34.295062+0000","last_active":"2023-10-08T07:13:49.298001+0000","last_peered":"2023-10-08T07:13:49.298001+0000","last_clean":"2023-10-08T07:13:49.298001+0000","last_became_active":"2023-10-08T07:13:34.294747+0000","last_became_peered":"2023-10-08T07:13:34.294747+0000","last_unstale":"2023-10-08T07:13:49.298001+0000","last_undegraded":"2023-10-08T07:13:49.298001+0000","last_fullsized":"2023-10-08T07:13:49.298001+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:26.407053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358527+0000","last_change":"2023-10-08T07:13:49.304648+0000","last_active":"2023-10-08T07:13:50.358527+0000","last_peered":"2023-10-08T07:13:50.358527+0000","last_clean":"2023-10-08T07:13:50.358527+0000","last_became_active":"2023-10-08T07:13:49.304247+0000","last_became_peered":"2023-10-08T07:13:49.304247+0000","last_unstale":"2023-10-08T07:13:50.358527+0000","last_undegraded":"2023-10-08T07:13:50.358527+0000","last_fullsized":"2023-10-08T07:13:50.358527+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:42.456757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-10-08T07:13:47.345193+0000","last_change":"2023-10-08T07:13:34.308897+0000","last_active":"2023-10-08T07:13:47.345193+0000","last_peered":"2023-10-08T07:13:47.345193+0000","last_clean":"2023-10-08T07:13:47.345193+0000","last_became_active":"2023-10-08T07:13:34.308474+0000","last_became_peered":"2023-10-08T07:13:34.308474+0000","last_unstale":"2023-10-08T07:13:47.345193+0000","last_undegraded":"2023-10-08T07:13:47.345193+0000","last_fullsized":"2023-10-08T07:13:47.345193+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:06.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":604,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2806,"ondisk_log_size":2806,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576011,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61024,"kb_used_data":3476,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310816,"statfs":{"total":96636764160,"available":96574275584,"internally_reserved":0,"allocated":3559424,"data_stored":2124760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.169,"15min":1.2090000000000001},"min":{"1min":0.77100000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.508,"5min":1.968,"15min":2.008},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.238,"15min":1.2310000000000001},"min":{"1min":0.91300000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.6830000000000001,"5min":1.7330000000000001,"15min":1.7450000000000001},"last":1.107}]},{"osd":1,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.242,"15min":1.252},"min":{"1min":0.94399999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.665,"5min":1.79,"15min":1.998},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.341,"15min":1.3500000000000001},"min":{"1min":0.99399999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.8859999999999999,"5min":1.8899999999999999,"15min":2.1019999999999999},"last":0.997}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.29,"15min":1.3140000000000001},"min":{"1min":0.97199999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6180000000000001,"5min":1.696,"15min":2.1640000000000001},"last":1.044},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.351},"min":{"1min":1.0069999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7270000000000001,"5min":1.8580000000000001,"15min":2.3170000000000002},"last":1.0660000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":1.0209999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.663,"5min":1.8160000000000001,"15min":1.9039999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.403,"5min":1.399,"15min":1.4119999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.754,"5min":1.8680000000000001,"15min":1.931},"last":1.3080000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":0.86199999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6950000000000001,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.383,"15min":1.401},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.85199999999999998},"max":{"1min":1.76,"5min":1.8140000000000001,"15min":2.0379999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4390000000000001,"15min":1.444},"min":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":2.0920000000000001,"15min":2.1139999999999999},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.4610000000000001,"15min":1.51},"min":{"1min":1.075,"5min":1,"15min":0.96899999999999997},"max":{"1min":1.8640000000000001,"5min":2.23,"15min":2.23},"last":1.2529999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.575,"15min":1.6020000000000001},"min":{"1min":1.361,"5min":1.048,"15min":1.048},"max":{"1min":2.016,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.349},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.536,"15min":1.5349999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9079999999999999,"5min":2.21,"15min":2.2570000000000001},"last":1.2889999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576010,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46568,"kb_used_data":2908,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325272,"statfs":{"total":96636764160,"available":96589078528,"internally_reserved":0,"allocated":2977792,"data_stored":1542476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3109999999999999,"15min":1.321},"min":{"1min":0.80000000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":1.794,"15min":1.962},"last":1.637},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.347,"15min":1.3620000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7230000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.222}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.228,"15min":1.2290000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.2130000000000001,"15min":1.2430000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.97},"last":1.5529999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.198,"15min":1.238},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.484,"5min":2.339,"15min":2.339},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.302,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":1.153}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3819999999999999,"15min":1.403},"min":{"1min":1.034,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":1.9319999999999999,"15min":1.95},"last":1.468},{"interface":"front","average":{"1min":1.425,"5min":1.476,"15min":1.4990000000000001},"min":{"1min":0.97199999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8740000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.5129999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3999999999999999,"15min":1.3899999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.881,"5min":2.0579999999999998,"15min":2.0859999999999999},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4770000000000001,"15min":1.448},"min":{"1min":0.94999999999999996,"5min":0.92000000000000004,"15min":0.57099999999999995},"max":{"1min":1.8080000000000001,"5min":2.484,"15min":2.484},"last":1.603}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.399,"15min":1.423},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.81899999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.403},"last":1.369},{"interface":"front","average":{"1min":1.552,"5min":1.538,"15min":1.5509999999999999},"min":{"1min":1.2829999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.8100000000000001,"5min":2.097,"15min":2.6139999999999999},"last":1.6950000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.575},"min":{"1min":1.0489999999999999,"5min":1.012,"15min":0.96299999999999997},"max":{"1min":1.9379999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.413},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.5620000000000001,"15min":1.5600000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.665},"last":1.2709999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576010,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60436,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311404,"statfs":{"total":96636764160,"available":96574877696,"internally_reserved":0,"allocated":2957312,"data_stored":1531521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.157,"15min":1.208},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.841},"last":1.351},{"interface":"front","average":{"1min":1.341,"5min":1.2290000000000001,"15min":1.232},"min":{"1min":0.83099999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7030000000000001,"5min":1.8560000000000001,"15min":1.9610000000000001},"last":0.97999999999999998}]},{"osd":1,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2090000000000001,"15min":1.242},"min":{"1min":0.75800000000000001,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.6990000000000001,"5min":1.762,"15min":1.9550000000000001},"last":1.129},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2869999999999999,"15min":1.3},"min":{"1min":0.90900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.2130000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.871,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":2.0870000000000002},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.3149999999999999},"min":{"1min":0.77000000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5840000000000001,"5min":1.877,"15min":1.909},"last":1.008}]},{"osd":3,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":0.79800000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.1499999999999999},"last":1.294},{"interface":"front","average":{"1min":1.458,"5min":1.3740000000000001,"15min":1.431},"min":{"1min":0.67500000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8100000000000001,"5min":1.9530000000000001,"15min":2.173},"last":0.84599999999999997}]},{"osd":4,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3220000000000001,"15min":1.3280000000000001},"min":{"1min":0.84399999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7569999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.335},{"interface":"front","average":{"1min":1.556,"5min":1.446,"15min":1.4339999999999999},"min":{"1min":1.107,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.2240000000000002},"last":1.4059999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":1.0740000000000001,"5min":0.66500000000000004,"15min":0.57499999999999996},"max":{"1min":1.899,"5min":2.077,"15min":2.3759999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.478,"15min":1.4770000000000001},"min":{"1min":1.157,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0059999999999998,"5min":2.036,"15min":2.036},"last":1.264}]},{"osd":7,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.5289999999999999,"15min":1.552},"min":{"1min":1.177,"5min":0.72499999999999998,"15min":0.59999999999999998},"max":{"1min":2.298,"5min":2.298,"15min":2.4169999999999998},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.5840000000000001,"15min":1.6180000000000001},"min":{"1min":1.0089999999999999,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0009999999999999,"5min":2.2320000000000002,"15min":2.3199999999999998},"last":1.1699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576010,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47952,"kb_used_data":2884,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323888,"statfs":{"total":96636764160,"available":96587661312,"internally_reserved":0,"allocated":2953216,"data_stored":1532047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2849999999999999,"15min":1.3440000000000001},"min":{"1min":0.64200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":2.9980000000000002},{"interface":"front","average":{"1min":1.367,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":0.62,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.9139999999999999,"5min":1.976,"15min":1.976},"last":7.0880000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2629999999999999,"15min":1.2889999999999999},"min":{"1min":0.85599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7450000000000001,"5min":1.976,"15min":1.976},"last":2.177},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.244},"min":{"1min":0.72499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7330000000000001,"5min":1.8660000000000001,"15min":1.875},"last":6.9020000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2689999999999999,"15min":1.266},"min":{"1min":0.77400000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":17.175999999999998},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.365,"15min":1.369},"min":{"1min":0.99299999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":7.1849999999999996}]},{"osd":3,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2789999999999999,"15min":1.3149999999999999},"min":{"1min":0.88,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.702,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":5.8289999999999997},{"interface":"front","average":{"1min":1.464,"5min":1.4410000000000001,"15min":1.4219999999999999},"min":{"1min":0.874,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":7.7599999999999998}]},{"osd":5,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.391},"min":{"1min":1.032,"5min":0.372,"15min":0.372},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":7.4020000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4239999999999999,"15min":1.3919999999999999},"min":{"1min":0.96799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.865,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":3.2549999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.5009999999999999,"15min":1.46},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2530000000000001,"5min":2.2719999999999998,"15min":2.6829999999999998},"last":10.542999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.593,"15min":1.5609999999999999},"min":{"1min":1.071,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.117,"5min":2.3500000000000001,"15min":2.7429999999999999},"last":10.433999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.5369999999999999,"15min":1.516},"min":{"1min":1.1220000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0649999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":3.1070000000000002},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.681,"15min":1.6439999999999999},"min":{"1min":1.3169999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.3439999999999999,"5min":2.508,"15min":2.508},"last":3.6459999999999999}]}]},{"osd":3,"up_from":13,"seq":55834576013,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52168,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319672,"statfs":{"total":96636764160,"available":96583344128,"internally_reserved":0,"allocated":3002368,"data_stored":1554869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.67800000000000005,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.087,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.123,"5min":1.1559999999999999,"15min":1.181},"min":{"1min":0.748,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.607,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.70099999999999996}]},{"osd":1,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.417,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.2390000000000001,"15min":1.2869999999999999},"min":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4390000000000001,"5min":2.109,"15min":2.4889999999999999},"last":0.74299999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.123,"15min":1.151},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.667,"5min":2.165,"15min":2.2250000000000001},"last":0.62},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.177,"15min":1.202},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5349999999999999,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2549999999999999,"15min":1.282},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.67500000000000004},"max":{"1min":1.51,"5min":1.857,"15min":2.5379999999999998},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4179999999999999,"15min":1.46},"min":{"1min":0.93600000000000005,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.954,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":0.98599999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3200000000000001,"15min":1.3759999999999999},"min":{"1min":0.97899999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8300000000000001,"5min":1.923,"15min":2.9740000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.419,"15min":1.4339999999999999},"min":{"1min":0.77800000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8380000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4099999999999999,"15min":1.4239999999999999},"min":{"1min":0.91000000000000003,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6339999999999999,"5min":1.976,"15min":2.653},"last":1.02},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4710000000000001,"15min":1.5269999999999999},"min":{"1min":1.0960000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8540000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.0529999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5660000000000001},"min":{"1min":0.88800000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9450000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.5209999999999999},"min":{"1min":1.202,"5min":0.628,"15min":0.628},"max":{"1min":1.7889999999999999,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.2110000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576011,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":2998272,"data_stored":1555427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.97299999999999998},"min":{"1min":0.55300000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.948},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.054,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":0.745}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.96099999999999997,"15min":1.0029999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.198,"5min":1.6279999999999999,"15min":2.044},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.171,"15min":1.1970000000000001},"min":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.677,"5min":1.714,"15min":2.1309999999999998},"last":0.86799999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68400000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4770000000000001,"5min":1.78,"15min":1.8420000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.2410000000000001,"15min":1.2769999999999999},"min":{"1min":0.66200000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.458,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.165}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3959999999999999,"15min":1.4399999999999999},"min":{"1min":0.89000000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.821,"5min":1.929,"15min":2.1789999999999998},"last":1.754},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":1.0189999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6879999999999999,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.577}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3819999999999999,"15min":1.411},"min":{"1min":1.0069999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.603,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":1.083},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5009999999999999,"15min":1.5349999999999999},"min":{"1min":1.0269999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.946,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.5489999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":0.997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.73,"5min":2.056,"15min":2.1240000000000001},"last":1.472},{"interface":"front","average":{"1min":1.47,"5min":1.4950000000000001,"15min":1.534},"min":{"1min":1.1439999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.778,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":1.7190000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4690000000000001,"15min":1.486},"min":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7609999999999999,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":1.643},{"interface":"front","average":{"1min":1.5,"5min":1.5309999999999999,"15min":1.5860000000000001},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.863,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":1.623}]}]},{"osd":1,"up_from":13,"seq":55834576010,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52108,"kb_used_data":2880,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319732,"statfs":{"total":96636764160,"available":96583405568,"internally_reserved":0,"allocated":2949120,"data_stored":1531465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.024,"15min":1.014},"min":{"1min":0.81000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.6200000000000001,"5min":1.6579999999999999,"15min":1.7769999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.224},"min":{"1min":0.88,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":1.79},"last":0.89500000000000002}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.97599999999999998},"min":{"1min":0.73399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2809999999999999,"5min":1.641,"15min":1.641},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.155,"15min":1.161},"min":{"1min":0.872,"5min":0.372,"15min":0.372},"max":{"1min":1.4710000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.95799999999999996}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.2430000000000001,"15min":1.264},"min":{"1min":1.052,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5800000000000001,"5min":1.651,"15min":1.7809999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1659999999999999,"15min":1.155},"min":{"1min":0.82599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.72,"5min":1.883,"15min":2.7429999999999999},"last":0.77200000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":1.117,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.7110000000000001,"5min":1.7829999999999999,"15min":2.8460000000000001},"last":1.022},{"interface":"front","average":{"1min":1.357,"5min":1.3129999999999999,"15min":1.3220000000000001},"min":{"1min":1.0960000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.5449999999999999,"5min":1.6879999999999999,"15min":2.0139999999999998},"last":1.3280000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.347,"15min":1.379},"min":{"1min":0.97299999999999998,"5min":0.92600000000000005,"15min":0.76700000000000002},"max":{"1min":1.6639999999999999,"5min":1.827,"15min":2.0470000000000002},"last":1.456},{"interface":"front","average":{"1min":1.637,"5min":1.546,"15min":1.534},"min":{"1min":1.286,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.2610000000000001},"last":1.667}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5089999999999999,"15min":1.5149999999999999},"min":{"1min":1.2549999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.694,"5min":2.3199999999999998,"15min":2.9079999999999999},"last":1.48},{"interface":"front","average":{"1min":1.514,"5min":1.4870000000000001,"15min":1.4990000000000001},"min":{"1min":1.1679999999999999,"5min":1.046,"15min":1.0309999999999999},"max":{"1min":1.7929999999999999,"5min":1.966,"15min":2.3199999999999998},"last":1.8180000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.5780000000000001,"15min":1.556},"min":{"1min":1.2250000000000001,"5min":0.94499999999999995,"15min":0.84599999999999997},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.786},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.5529999999999999,"15min":1.5720000000000001},"min":{"1min":1.377,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.798,"5min":2.1699999999999999,"15min":2.9590000000000001},"last":1.268}]}]},{"osd":0,"up_from":13,"seq":55834576012,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61048,"kb_used_data":3500,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310792,"statfs":{"total":96636764160,"available":96574251008,"internally_reserved":0,"allocated":3584000,"data_stored":2145179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.48299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3859999999999999,"5min":1.581,"15min":1.581},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.01,"5min":1.103,"15min":1.109},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.621,"5min":1.6619999999999999,"15min":1.726},"last":0.874}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1060000000000001,"15min":1.1419999999999999},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0329999999999999,"15min":1.073},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4199999999999999,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":0.93300000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.248,"15min":1.256},"min":{"1min":0.69799999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.546,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.579,"5min":1.752,"15min":2.0979999999999999},"last":1.4259999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3220000000000001,"15min":1.349},"min":{"1min":0.83399999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.389,"15min":1.4179999999999999},"min":{"1min":0.69699999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.782,"5min":1.972,"15min":2.181},"last":2.1520000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.423,"15min":1.417},"min":{"1min":0.79600000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.732,"5min":1.919,"15min":2.1309999999999998},"last":1.706},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.482,"15min":1.486},"min":{"1min":0.79300000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.966,"5min":2.2170000000000001,"15min":2.4900000000000002},"last":1.9930000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.81299999999999994,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7669999999999999,"5min":2.21,"15min":2.2149999999999999},"last":1.925},{"interface":"front","average":{"1min":1.397,"5min":1.506,"15min":1.5089999999999999},"min":{"1min":0.78000000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9430000000000001,"5min":2.0880000000000001,"15min":2.355},"last":1.7529999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5389999999999999,"15min":1.5469999999999999},"min":{"1min":0.83499999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9410000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.5289999999999999,"15min":1.554},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":2.0750000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:13:54.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.5 is not active+clean 2023-10-08T07:13:54.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.377733+0000', 'last_change': '2023-10-08T07:13:48.377733+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.377733+0000', 'last_undegraded': '2023-10-08T07:13:48.377733+0000', 'last_fullsized': '2023-10-08T07:13:48.377733+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T07:13:54.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.6 is not active+clean 2023-10-08T07:13:54.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.372366+0000', 'last_change': '2023-10-08T07:13:48.372366+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.372366+0000', 'last_undegraded': '2023-10-08T07:13:48.372366+0000', 'last_fullsized': '2023-10-08T07:13:48.372366+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T07:13:54.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3 is not active+clean 2023-10-08T07:13:54.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.377624+0000', 'last_change': '2023-10-08T07:13:48.377624+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.377624+0000', 'last_undegraded': '2023-10-08T07:13:48.377624+0000', 'last_fullsized': '2023-10-08T07:13:48.377624+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T07:13:54.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.d is not active+clean 2023-10-08T07:13:54.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.379456+0000', 'last_change': '2023-10-08T07:13:48.379456+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.379456+0000', 'last_undegraded': '2023-10-08T07:13:48.379456+0000', 'last_fullsized': '2023-10-08T07:13:48.379456+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T07:13:54.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.f is not active+clean 2023-10-08T07:13:54.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.376428+0000', 'last_change': '2023-10-08T07:13:48.376428+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.376428+0000', 'last_undegraded': '2023-10-08T07:13:48.376428+0000', 'last_fullsized': '2023-10-08T07:13:48.376428+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T07:13:54.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.9 is not active+clean 2023-10-08T07:13:54.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.373866+0000', 'last_change': '2023-10-08T07:13:48.373866+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.373866+0000', 'last_undegraded': '2023-10-08T07:13:48.373866+0000', 'last_fullsized': '2023-10-08T07:13:48.373866+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T07:13:54.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.8 is not active+clean 2023-10-08T07:13:54.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.367853+0000', 'last_change': '2023-10-08T07:13:48.367853+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.367853+0000', 'last_undegraded': '2023-10-08T07:13:48.367853+0000', 'last_fullsized': '2023-10-08T07:13:48.367853+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T07:13:54.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.15 is not active+clean 2023-10-08T07:13:54.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.378285+0000', 'last_change': '2023-10-08T07:13:48.378285+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.378285+0000', 'last_undegraded': '2023-10-08T07:13:48.378285+0000', 'last_fullsized': '2023-10-08T07:13:48.378285+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T07:13:54.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.14 is not active+clean 2023-10-08T07:13:54.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.371621+0000', 'last_change': '2023-10-08T07:13:48.371621+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.371621+0000', 'last_undegraded': '2023-10-08T07:13:48.371621+0000', 'last_fullsized': '2023-10-08T07:13:48.371621+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T07:13:54.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.17 is not active+clean 2023-10-08T07:13:54.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.366477+0000', 'last_change': '2023-10-08T07:13:48.366477+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.366477+0000', 'last_undegraded': '2023-10-08T07:13:48.366477+0000', 'last_fullsized': '2023-10-08T07:13:48.366477+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T07:13:54.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.11 is not active+clean 2023-10-08T07:13:54.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.363741+0000', 'last_change': '2023-10-08T07:13:48.363741+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.363741+0000', 'last_undegraded': '2023-10-08T07:13:48.363741+0000', 'last_fullsized': '2023-10-08T07:13:48.363741+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T07:13:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.13 is not active+clean 2023-10-08T07:13:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.376614+0000', 'last_change': '2023-10-08T07:13:48.376614+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.376614+0000', 'last_undegraded': '2023-10-08T07:13:48.376614+0000', 'last_fullsized': '2023-10-08T07:13:48.376614+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T07:13:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1e is not active+clean 2023-10-08T07:13:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.379499+0000', 'last_change': '2023-10-08T07:13:48.379499+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.379499+0000', 'last_undegraded': '2023-10-08T07:13:48.379499+0000', 'last_fullsized': '2023-10-08T07:13:48.379499+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T07:13:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.19 is not active+clean 2023-10-08T07:13:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.378495+0000', 'last_change': '2023-10-08T07:13:48.378495+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.378495+0000', 'last_undegraded': '2023-10-08T07:13:48.378495+0000', 'last_fullsized': '2023-10-08T07:13:48.378495+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T07:13:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.1a is not active+clean 2023-10-08T07:13:54.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.374502+0000', 'last_change': '2023-10-08T07:13:48.374502+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.374502+0000', 'last_undegraded': '2023-10-08T07:13:48.374502+0000', 'last_fullsized': '2023-10-08T07:13:48.374502+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T07:13:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.22 is not active+clean 2023-10-08T07:13:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.376167+0000', 'last_change': '2023-10-08T07:13:48.376167+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.376167+0000', 'last_undegraded': '2023-10-08T07:13:48.376167+0000', 'last_fullsized': '2023-10-08T07:13:48.376167+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T07:13:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.20 is not active+clean 2023-10-08T07:13:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.372376+0000', 'last_change': '2023-10-08T07:13:48.372376+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.372376+0000', 'last_undegraded': '2023-10-08T07:13:48.372376+0000', 'last_fullsized': '2023-10-08T07:13:48.372376+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T07:13:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.27 is not active+clean 2023-10-08T07:13:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.375861+0000', 'last_change': '2023-10-08T07:13:48.375861+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.375861+0000', 'last_undegraded': '2023-10-08T07:13:48.375861+0000', 'last_fullsized': '2023-10-08T07:13:48.375861+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T07:13:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.24 is not active+clean 2023-10-08T07:13:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.381042+0000', 'last_change': '2023-10-08T07:13:48.381042+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.381042+0000', 'last_undegraded': '2023-10-08T07:13:48.381042+0000', 'last_fullsized': '2023-10-08T07:13:48.381042+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T07:13:54.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.25 is not active+clean 2023-10-08T07:13:54.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.367829+0000', 'last_change': '2023-10-08T07:13:48.367829+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.367829+0000', 'last_undegraded': '2023-10-08T07:13:48.367829+0000', 'last_fullsized': '2023-10-08T07:13:48.367829+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T07:13:54.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2a is not active+clean 2023-10-08T07:13:54.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.376110+0000', 'last_change': '2023-10-08T07:13:48.376110+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.376110+0000', 'last_undegraded': '2023-10-08T07:13:48.376110+0000', 'last_fullsized': '2023-10-08T07:13:48.376110+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T07:13:54.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.28 is not active+clean 2023-10-08T07:13:54.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.372077+0000', 'last_change': '2023-10-08T07:13:48.372077+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.372077+0000', 'last_undegraded': '2023-10-08T07:13:48.372077+0000', 'last_fullsized': '2023-10-08T07:13:48.372077+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T07:13:54.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2f is not active+clean 2023-10-08T07:13:54.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.381122+0000', 'last_change': '2023-10-08T07:13:48.381122+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.381122+0000', 'last_undegraded': '2023-10-08T07:13:48.381122+0000', 'last_fullsized': '2023-10-08T07:13:48.381122+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T07:13:54.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2c is not active+clean 2023-10-08T07:13:54.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.380405+0000', 'last_change': '2023-10-08T07:13:48.380405+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.380405+0000', 'last_undegraded': '2023-10-08T07:13:48.380405+0000', 'last_fullsized': '2023-10-08T07:13:48.380405+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T07:13:54.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.2d is not active+clean 2023-10-08T07:13:54.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.376641+0000', 'last_change': '2023-10-08T07:13:48.376641+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.376641+0000', 'last_undegraded': '2023-10-08T07:13:48.376641+0000', 'last_fullsized': '2023-10-08T07:13:48.376641+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T07:13:54.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.32 is not active+clean 2023-10-08T07:13:54.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.381713+0000', 'last_change': '2023-10-08T07:13:48.381713+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.381713+0000', 'last_undegraded': '2023-10-08T07:13:48.381713+0000', 'last_fullsized': '2023-10-08T07:13:48.381713+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T07:13:54.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.33 is not active+clean 2023-10-08T07:13:54.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.375275+0000', 'last_change': '2023-10-08T07:13:48.375275+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.375275+0000', 'last_undegraded': '2023-10-08T07:13:48.375275+0000', 'last_fullsized': '2023-10-08T07:13:48.375275+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T07:13:54.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.30 is not active+clean 2023-10-08T07:13:54.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.373371+0000', 'last_change': '2023-10-08T07:13:48.373371+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.373371+0000', 'last_undegraded': '2023-10-08T07:13:48.373371+0000', 'last_fullsized': '2023-10-08T07:13:48.373371+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T07:13:54.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.31 is not active+clean 2023-10-08T07:13:54.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.376338+0000', 'last_change': '2023-10-08T07:13:48.376338+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.376338+0000', 'last_undegraded': '2023-10-08T07:13:48.376338+0000', 'last_fullsized': '2023-10-08T07:13:48.376338+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T07:13:54.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.34 is not active+clean 2023-10-08T07:13:54.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.381921+0000', 'last_change': '2023-10-08T07:13:48.381921+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.381921+0000', 'last_undegraded': '2023-10-08T07:13:48.381921+0000', 'last_fullsized': '2023-10-08T07:13:48.381921+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T07:13:54.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.35 is not active+clean 2023-10-08T07:13:54.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.368925+0000', 'last_change': '2023-10-08T07:13:48.368925+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.368925+0000', 'last_undegraded': '2023-10-08T07:13:48.368925+0000', 'last_fullsized': '2023-10-08T07:13:48.368925+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T07:13:54.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3a is not active+clean 2023-10-08T07:13:54.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.375367+0000', 'last_change': '2023-10-08T07:13:48.375367+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.375367+0000', 'last_undegraded': '2023-10-08T07:13:48.375367+0000', 'last_fullsized': '2023-10-08T07:13:48.375367+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T07:13:54.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3b is not active+clean 2023-10-08T07:13:54.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.378259+0000', 'last_change': '2023-10-08T07:13:48.378259+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.378259+0000', 'last_undegraded': '2023-10-08T07:13:48.378259+0000', 'last_fullsized': '2023-10-08T07:13:48.378259+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T07:13:54.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.38 is not active+clean 2023-10-08T07:13:54.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.374742+0000', 'last_change': '2023-10-08T07:13:48.374742+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.374742+0000', 'last_undegraded': '2023-10-08T07:13:48.374742+0000', 'last_fullsized': '2023-10-08T07:13:48.374742+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T07:13:54.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3f is not active+clean 2023-10-08T07:13:54.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.376508+0000', 'last_change': '2023-10-08T07:13:48.376508+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.376508+0000', 'last_undegraded': '2023-10-08T07:13:48.376508+0000', 'last_fullsized': '2023-10-08T07:13:48.376508+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T07:13:54.928 INFO:tasks.cephfs.filesystem.ceph_manager:PG 30.3c is not active+clean 2023-10-08T07:13:54.928 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '30.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 236, 'state': 'creating+peering', 'last_fresh': '2023-10-08T07:13:48.373866+0000', 'last_change': '2023-10-08T07:13:48.373866+0000', 'last_active': '2023-10-08T07:13:48.335144+0000', 'last_peered': '2023-10-08T07:13:48.335144+0000', 'last_clean': '2023-10-08T07:13:48.335144+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T07:13:48.373866+0000', 'last_undegraded': '2023-10-08T07:13:48.373866+0000', 'last_fullsized': '2023-10-08T07:13:48.373866+0000', 'mapping_epoch': 236, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 236, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'last_clean_scrub_stamp': '2023-10-08T07:13:48.335144+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T07:13:54.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T07:13:55.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.167+0000 7f0d54e09640 1 Processor -- start 2023-10-08T07:13:55.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.167+0000 7f0d54e09640 1 -- start start 2023-10-08T07:13:55.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.169+0000 7f0d54e09640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0d50151aa0 0x7f0d500c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.169+0000 7f0d54e09640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0d500c93c0 0x7f0d5015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.169+0000 7f0d54e09640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d5015ec30 0x7f0d5015b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.169+0000 7f0d54e09640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d500b1320 con 0x7f0d500c93c0 2023-10-08T07:13:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.169+0000 7f0d54e09640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d500b0a80 con 0x7f0d50151aa0 2023-10-08T07:13:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.169+0000 7f0d54e09640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b0d60 con 0x7f0d5015ec30 2023-10-08T07:13:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.169+0000 7f0d4dd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0d500c93c0 0x7f0d5015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.169+0000 7f0d4dd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0d500c93c0 0x7f0d5015e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49948/0 (socket says 172.21.15.70:49948) 2023-10-08T07:13:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.169+0000 7f0d4dd9b640 1 -- 172.21.15.70:0/3768119843 learned_addr learned my addr 172.21.15.70:0/3768119843 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.170+0000 7f0d4e59c640 1 --2- 172.21.15.70:0/3768119843 >> v2:172.21.15.181:3300/0 conn(0x7f0d50151aa0 0x7f0d500c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.170+0000 7f0d4ed9d640 1 --2- 172.21.15.70:0/3768119843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d5015ec30 0x7f0d5015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.170+0000 7f0d4dd9b640 1 -- 172.21.15.70:0/3768119843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d5015ec30 msgr2=0x7f0d5015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.170+0000 7f0d4dd9b640 1 --2- 172.21.15.70:0/3768119843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d5015ec30 0x7f0d5015b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.170+0000 7f0d4dd9b640 1 -- 172.21.15.70:0/3768119843 >> v2:172.21.15.181:3300/0 conn(0x7f0d50151aa0 msgr2=0x7f0d500c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.170+0000 7f0d4dd9b640 1 --2- 172.21.15.70:0/3768119843 >> v2:172.21.15.181:3300/0 conn(0x7f0d50151aa0 0x7f0d500c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.170+0000 7f0d4dd9b640 1 -- 172.21.15.70:0/3768119843 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500b1040 con 0x7f0d500c93c0 2023-10-08T07:13:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.171+0000 7f0d4dd9b640 1 --2- 172.21.15.70:0/3768119843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d500c93c0 0x7f0d5015e6f0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f0d500ce720 tx=0x7f0d4402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d0ef60c4e6489e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.172+0000 7f0d4d59a640 1 -- 172.21.15.70:0/3768119843 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d440317e0 con 0x7f0d500c93c0 2023-10-08T07:13:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.172+0000 7f0d4d59a640 1 -- 172.21.15.70:0/3768119843 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d44031940 con 0x7f0d500c93c0 2023-10-08T07:13:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.172+0000 7f0d4d59a640 1 -- 172.21.15.70:0/3768119843 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44031cd0 con 0x7f0d500c93c0 2023-10-08T07:13:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.173+0000 7f0d54e09640 1 -- 172.21.15.70:0/3768119843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d500c93c0 msgr2=0x7f0d5015e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.173+0000 7f0d54e09640 1 --2- 172.21.15.70:0/3768119843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d500c93c0 0x7f0d5015e6f0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f0d500ce720 tx=0x7f0d4402b880 comp rx=0 tx=0).stop 2023-10-08T07:13:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.173+0000 7f0d54e09640 1 -- 172.21.15.70:0/3768119843 shutdown_connections 2023-10-08T07:13:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.173+0000 7f0d54e09640 1 --2- 172.21.15.70:0/3768119843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d5015ec30 0x7f0d5015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.173+0000 7f0d54e09640 1 --2- 172.21.15.70:0/3768119843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d500c93c0 0x7f0d5015e6f0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.173+0000 7f0d54e09640 1 --2- 172.21.15.70:0/3768119843 >> v2:172.21.15.181:3300/0 conn(0x7f0d50151aa0 0x7f0d500c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.173+0000 7f0d54e09640 1 -- 172.21.15.70:0/3768119843 >> 172.21.15.70:0/3768119843 conn(0x7f0d500b20b0 msgr2=0x7f0d50069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.173+0000 7f0d54e09640 1 -- 172.21.15.70:0/3768119843 shutdown_connections 2023-10-08T07:13:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.173+0000 7f0d54e09640 1 -- 172.21.15.70:0/3768119843 wait complete. 2023-10-08T07:13:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.175+0000 7f0d54e09640 1 Processor -- start 2023-10-08T07:13:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.175+0000 7f0d54e09640 1 -- start start 2023-10-08T07:13:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d54e09640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d500c93c0 0x7f0d5015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d54e09640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d50151aa0 0x7f0d500ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d54e09640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5015ec30 0x7f0d50149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d54e09640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d50149b00 con 0x7f0d50151aa0 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d54e09640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d50159b40 con 0x7f0d5015ec30 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d54e09640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500cd320 con 0x7f0d500c93c0 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d50151aa0 0x7f0d500ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d50151aa0 0x7f0d500ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49956/0 (socket says 172.21.15.70:49956) 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d4dd9b640 1 -- 172.21.15.70:0/4283522170 learned_addr learned my addr 172.21.15.70:0/4283522170 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d4dd9b640 1 -- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d500c93c0 msgr2=0x7f0d5015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d4dd9b640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d500c93c0 0x7f0d5015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d4ed9d640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5015ec30 0x7f0d50149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d4dd9b640 1 -- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5015ec30 msgr2=0x7f0d50149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.177+0000 7f0d4dd9b640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5015ec30 0x7f0d50149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.178+0000 7f0d4dd9b640 1 -- 172.21.15.70:0/4283522170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d440094d0 con 0x7f0d50151aa0 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.178+0000 7f0d4dd9b640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d50151aa0 0x7f0d500ccde0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f0d44000c00 tx=0x7f0d440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.178+0000 7f0d3effd640 1 -- 172.21.15.70:0/4283522170 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44032880 con 0x7f0d50151aa0 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.178+0000 7f0d3effd640 1 -- 172.21.15.70:0/4283522170 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d44032a20 con 0x7f0d50151aa0 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.178+0000 7f0d3effd640 1 -- 172.21.15.70:0/4283522170 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44032dd0 con 0x7f0d50151aa0 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.178+0000 7f0d54e09640 1 -- 172.21.15.70:0/4283522170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d50149580 con 0x7f0d50151aa0 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.178+0000 7f0d54e09640 1 -- 172.21.15.70:0/4283522170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d50159d90 con 0x7f0d50151aa0 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.179+0000 7f0d54e09640 1 -- 172.21.15.70:0/4283522170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d50067490 con 0x7f0d50151aa0 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.181+0000 7f0d3effd640 1 -- 172.21.15.70:0/4283522170 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f0d440453f0 con 0x7f0d50151aa0 2023-10-08T07:13:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.181+0000 7f0d3effd640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f0d3006af90 0x7f0d3006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.182+0000 7f0d3effd640 1 -- 172.21.15.70:0/4283522170 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f0d440abf20 con 0x7f0d50151aa0 2023-10-08T07:13:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.182+0000 7f0d4e59c640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f0d3006af90 0x7f0d3006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.183+0000 7f0d4e59c640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f0d3006af90 0x7f0d3006d450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009870 tx=0x7f0d40008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.186+0000 7f0d3effd640 1 -- 172.21.15.70:0/4283522170 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d44076940 con 0x7f0d50151aa0 2023-10-08T07:13:55.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.449+0000 7f0d54e09640 1 -- 172.21.15.70:0/4283522170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0d500c91a0 con 0x7f0d50151aa0 2023-10-08T07:13:55.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.450+0000 7f0d3effd640 1 -- 172.21.15.70:0/4283522170 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+3404 (secure 0 0 0) 0x7f0d44079f60 con 0x7f0d50151aa0 2023-10-08T07:13:55.450 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:55.451 INFO:teuthology.orchestra.run.smithi070.stdout:{"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5883,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":238,"num_osds":8,"num_up_osds":8,"osd_up_since":1696743363,"num_in_osds":8,"osd_in_since":1696743355,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":157},{"state_name":"creating+peering","count":36}],"num_pgs":193,"num_pools":5,"num_objects":28,"data_bytes":594627,"bytes_used":443871232,"bytes_avail":772650242048,"bytes_total":773094113280,"inactive_pgs_ratio":0.18652850389480591,"write_bytes_sec":511,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":114,"id":13,"up":1,"in":1,"max":1,"id":14,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":14,"rank":0,"name":"d","status":"up:active","gid":8416},{"filesystem_id":13,"rank":0,"name":"a","status":"up:active","gid":11097}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2023-10-08T07:13:11.496776+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-08T07:13:55.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 -- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f0d3006af90 msgr2=0x7f0d3006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f0d3006af90 0x7f0d3006d450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009870 tx=0x7f0d40008040 comp rx=0 tx=0).stop 2023-10-08T07:13:55.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 -- 172.21.15.70:0/4283522170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d50151aa0 msgr2=0x7f0d500ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d50151aa0 0x7f0d500ccde0 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f0d44000c00 tx=0x7f0d440046c0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 -- 172.21.15.70:0/4283522170 shutdown_connections 2023-10-08T07:13:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0d5015ec30 0x7f0d50149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f0d3006af90 0x7f0d3006d450 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d50151aa0 0x7f0d500ccde0 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 --2- 172.21.15.70:0/4283522170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0d500c93c0 0x7f0d5015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 -- 172.21.15.70:0/4283522170 >> 172.21.15.70:0/4283522170 conn(0x7f0d500b20b0 msgr2=0x7f0d500701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.453+0000 7f0d54e09640 1 -- 172.21.15.70:0/4283522170 shutdown_connections 2023-10-08T07:13:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.454+0000 7f0d54e09640 1 -- 172.21.15.70:0/4283522170 wait complete. 2023-10-08T07:13:55.471 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T07:13:55.471 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:13:55.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.680+0000 7fa2c40e7640 1 Processor -- start 2023-10-08T07:13:55.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.680+0000 7fa2c40e7640 1 -- start start 2023-10-08T07:13:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.682+0000 7fa2c40e7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa2bc06b520 0x7fa2bc06ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.682+0000 7fa2c40e7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2bc0b0340 con 0x7fa2bc0b0ab0 2023-10-08T07:13:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.682+0000 7fa2c40e7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2bc0cf8f0 con 0x7fa2bc0af8b0 2023-10-08T07:13:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.682+0000 7fa2c40e7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2bc06b3a0 con 0x7fa2bc06b520 2023-10-08T07:13:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.682+0000 7fa2c2684640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa2bc0b0ab0 0x7fa2bc0b0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47746/0 (socket says 172.21.15.70:47746) 2023-10-08T07:13:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.682+0000 7fa2c2684640 1 -- 172.21.15.70:0/2113851135 learned_addr learned my addr 172.21.15.70:0/2113851135 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:55.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c1e83640 1 --2- 172.21.15.70:0/2113851135 >> v2:172.21.15.181:3300/0 conn(0x7fa2bc06b520 0x7fa2bc06ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789092875 0 0) 0x7fa2bc0b0340 con 0x7fa2bc0b0ab0 2023-10-08T07:13:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2a8003520 con 0x7fa2bc0b0ab0 2023-10-08T07:13:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2972853486 0 0) 0x7fa2a8003520 con 0x7fa2bc0b0ab0 2023-10-08T07:13:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 >> v2:172.21.15.181:3300/0 conn(0x7fa2bc06b520 msgr2=0x7fa2bc06ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 --2- 172.21.15.70:0/2113851135 >> v2:172.21.15.181:3300/0 conn(0x7fa2bc06b520 0x7fa2bc06ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 >> v1:172.21.15.181:6789/0 conn(0x7fa2bc0af8b0 legacy=0x7fa2bc06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc0cfbd0 con 0x7fa2bc0b0ab0 2023-10-08T07:13:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa2b40019f0 con 0x7fa2bc0b0ab0 2023-10-08T07:13:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 >> v1:172.21.15.70:6789/0 conn(0x7fa2bc0b0ab0 legacy=0x7fa2bc0b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a8003970 0x7fa2a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a80047c0 0x7fa2a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a8005180 0x7fa2a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc06b3a0 con 0x7fa2a8005180 2023-10-08T07:13:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a800c120 con 0x7fa2a8003970 2023-10-08T07:13:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.683+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a800c2e0 con 0x7fa2a80047c0 2023-10-08T07:13:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.684+0000 7fa2c2684640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a8005180 0x7fa2a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.684+0000 7fa2c1e83640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a8003970 0x7fa2a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.684+0000 7fa2c1682640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a80047c0 0x7fa2a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.684+0000 7fa2c2684640 1 -- 172.21.15.70:0/2113851135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a80047c0 msgr2=0x7fa2a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:55.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.684+0000 7fa2c2684640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a80047c0 0x7fa2a8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.684+0000 7fa2c2684640 1 -- 172.21.15.70:0/2113851135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a8003970 msgr2=0x7fa2a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:55.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.684+0000 7fa2c2684640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a8003970 0x7fa2a8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.684+0000 7fa2c2684640 1 -- 172.21.15.70:0/2113851135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a8003520 con 0x7fa2a8005180 2023-10-08T07:13:55.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.684+0000 7fa2c1e83640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2a8003970 0x7fa2a8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:55.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.685+0000 7fa2c2684640 1 -- 172.21.15.70:0/2113851135 reap_dead start 2023-10-08T07:13:55.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.685+0000 7fa2c2684640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a8005180 0x7fa2a800b980 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400a830 tx=0x7fa2a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39514e912cf01ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.685+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40072d0 con 0x7fa2a8005180 2023-10-08T07:13:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.685+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b402f070 con 0x7fa2a8005180 2023-10-08T07:13:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.685+0000 7fa2c0e81640 1 -- 172.21.15.70:0/2113851135 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40076b0 con 0x7fa2a8005180 2023-10-08T07:13:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.686+0000 7fa2c40e7640 1 -- 172.21.15.70:0/2113851135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a8005180 msgr2=0x7fa2a800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.686+0000 7fa2c40e7640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a8005180 0x7fa2a800b980 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400a830 tx=0x7fa2a80057f0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.686+0000 7fa2c40e7640 1 -- 172.21.15.70:0/2113851135 shutdown_connections 2023-10-08T07:13:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.686+0000 7fa2c40e7640 1 --2- 172.21.15.70:0/2113851135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a8005180 0x7fa2a800b980 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.686+0000 7fa2c40e7640 1 -- 172.21.15.70:0/2113851135 >> 172.21.15.70:0/2113851135 conn(0x7fa2bc0d09e0 msgr2=0x7fa2bc0d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.686+0000 7fa2c40e7640 1 -- 172.21.15.70:0/2113851135 shutdown_connections 2023-10-08T07:13:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.686+0000 7fa2c40e7640 1 -- 172.21.15.70:0/2113851135 wait complete. 2023-10-08T07:13:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.688+0000 7fa2c40e7640 1 Processor -- start 2023-10-08T07:13:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.688+0000 7fa2c40e7640 1 -- start start 2023-10-08T07:13:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.689+0000 7fa2c40e7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a8005180 0x7fa2bc1f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.689+0000 7fa2c40e7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2bc06f460 0x7fa2bc06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.689+0000 7fa2c40e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2bc0c6db0 0x7fa2bc0c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.689+0000 7fa2c40e7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc14ed40 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.689+0000 7fa2c40e7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc14bb70 con 0x7fa2bc06f460 2023-10-08T07:13:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.689+0000 7fa2c40e7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc06fe20 con 0x7fa2a8005180 2023-10-08T07:13:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.689+0000 7fa2c2684640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2bc0c6db0 0x7fa2bc0c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.689+0000 7fa2c2684640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2bc0c6db0 0x7fa2bc0c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49966/0 (socket says 172.21.15.70:49966) 2023-10-08T07:13:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.689+0000 7fa2c2684640 1 -- 172.21.15.70:0/907958317 learned_addr learned my addr 172.21.15.70:0/907958317 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2c1e83640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a8005180 0x7fa2bc1f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2c1682640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2bc06f460 0x7fa2bc06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2c2684640 1 -- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a8005180 msgr2=0x7fa2bc1f1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2c2684640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a8005180 0x7fa2bc1f1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2c2684640 1 -- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2bc06f460 msgr2=0x7fa2bc06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2c2684640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2bc06f460 0x7fa2bc06f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2c2684640 1 -- 172.21.15.70:0/907958317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc0c7770 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2c2684640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2bc0c6db0 0x7fa2bc0c7230 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400a950 tx=0x7fa2b4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2b27fc640 1 -- 172.21.15.70:0/907958317 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4044070 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2b27fc640 1 -- 172.21.15.70:0/907958317 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b4001250 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2b27fc640 1 -- 172.21.15.70:0/907958317 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4007450 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2c40e7640 1 -- 172.21.15.70:0/907958317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2bc14de70 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.690+0000 7fa2c40e7640 1 -- 172.21.15.70:0/907958317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2bc14e3a0 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.692+0000 7fa2c40e7640 1 -- 172.21.15.70:0/907958317 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2bc067490 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.693+0000 7fa2b27fc640 1 -- 172.21.15.70:0/907958317 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa2b4031030 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.693+0000 7fa2b27fc640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fa2a006af90 0x7fa2a006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.693+0000 7fa2b27fc640 1 -- 172.21.15.70:0/907958317 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fa2b40ab290 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.694+0000 7fa2c1e83640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fa2a006af90 0x7fa2a006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:55.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.695+0000 7fa2c1e83640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fa2a006af90 0x7fa2a006d450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80099b0 tx=0x7fa2b8008040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:55.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.698+0000 7fa2b27fc640 1 -- 172.21.15.70:0/907958317 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2b4075d20 con 0x7fa2bc0c6db0 2023-10-08T07:13:55.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.885+0000 7fa2c40e7640 1 -- 172.21.15.70:0/907958317 --> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2bc0b0590 con 0x7fa2a006af90 2023-10-08T07:13:55.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.926+0000 7fa2b27fc640 1 -- 172.21.15.70:0/907958317 <== mgr.7328 v2:172.21.15.181:6832/3993370366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+525952 (secure 0 0 0) 0x7fa2b8005990 con 0x7fa2a006af90 2023-10-08T07:13:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:55.939 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:13:55.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 -- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fa2a006af90 msgr2=0x7fa2a006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fa2a006af90 0x7fa2a006d450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80099b0 tx=0x7fa2b8008040 comp rx=0 tx=0).stop 2023-10-08T07:13:55.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 -- 172.21.15.70:0/907958317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2bc0c6db0 msgr2=0x7fa2bc0c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:55.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2bc0c6db0 0x7fa2bc0c7230 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400a950 tx=0x7fa2b4001d80 comp rx=0 tx=0).stop 2023-10-08T07:13:55.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 -- 172.21.15.70:0/907958317 shutdown_connections 2023-10-08T07:13:55.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fa2a006af90 0x7fa2a006d450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2bc0c6db0 0x7fa2bc0c7230 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa2bc06f460 0x7fa2bc06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 --2- 172.21.15.70:0/907958317 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa2a8005180 0x7fa2bc1f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:55.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 -- 172.21.15.70:0/907958317 >> 172.21.15.70:0/907958317 conn(0x7fa2bc0d09e0 msgr2=0x7fa2bc0cdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:55.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 -- 172.21.15.70:0/907958317 shutdown_connections 2023-10-08T07:13:55.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:55.934+0000 7fa2c40e7640 1 -- 172.21.15.70:0/907958317 wait complete. 2023-10-08T07:13:55.954 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2023-10-08T07:13:55.511145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":680,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2885,"ondisk_log_size":2885,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":378,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":437872,"kb_used_data":24616,"kb_used_omap":12,"kb_used_meta":413107,"kb_avail":754536848,"statfs":{"total":773094113280,"available":772645732352,"internally_reserved":0,"allocated":25206784,"data_stored":13630634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":423022139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3397,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.153780"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359204+0000","last_change":"2023-10-08T07:13:49.313140+0000","last_active":"2023-10-08T07:13:50.359204+0000","last_peered":"2023-10-08T07:13:50.359204+0000","last_clean":"2023-10-08T07:13:50.359204+0000","last_became_active":"2023-10-08T07:13:49.312820+0000","last_became_peered":"2023-10-08T07:13:49.312820+0000","last_unstale":"2023-10-08T07:13:50.359204+0000","last_undegraded":"2023-10-08T07:13:50.359204+0000","last_fullsized":"2023-10-08T07:13:50.359204+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:41.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"222'2806","reported_seq":3727,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.303110+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:13:49.303110+0000","last_peered":"2023-10-08T07:13:49.303110+0000","last_clean":"2023-10-08T07:13:49.303110+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:13:49.303110+0000","last_undegraded":"2023-10-08T07:13:49.303110+0000","last_fullsized":"2023-10-08T07:13:49.303110+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2806,"log_dups_size":0,"ondisk_log_size":2806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289208+0000","last_change":"2023-10-08T07:13:33.718816+0000","last_active":"2023-10-08T07:13:49.289208+0000","last_peered":"2023-10-08T07:13:49.289208+0000","last_clean":"2023-10-08T07:13:49.289208+0000","last_became_active":"2023-10-08T07:13:33.718494+0000","last_became_peered":"2023-10-08T07:13:33.718494+0000","last_unstale":"2023-10-08T07:13:49.289208+0000","last_undegraded":"2023-10-08T07:13:49.289208+0000","last_fullsized":"2023-10-08T07:13:49.289208+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:03.021625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.300079+0000","last_change":"2023-10-08T07:13:34.304364+0000","last_active":"2023-10-08T07:13:49.300079+0000","last_peered":"2023-10-08T07:13:49.300079+0000","last_clean":"2023-10-08T07:13:49.300079+0000","last_became_active":"2023-10-08T07:13:34.304067+0000","last_became_peered":"2023-10-08T07:13:34.304067+0000","last_unstale":"2023-10-08T07:13:49.300079+0000","last_undegraded":"2023-10-08T07:13:49.300079+0000","last_fullsized":"2023-10-08T07:13:49.300079+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:56.084275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.366839+0000","last_change":"2023-10-08T07:13:48.383423+0000","last_active":"2023-10-08T07:13:49.366839+0000","last_peered":"2023-10-08T07:13:49.366839+0000","last_clean":"2023-10-08T07:13:49.366839+0000","last_became_active":"2023-10-08T07:13:48.383120+0000","last_became_peered":"2023-10-08T07:13:48.383120+0000","last_unstale":"2023-10-08T07:13:49.366839+0000","last_undegraded":"2023-10-08T07:13:49.366839+0000","last_fullsized":"2023-10-08T07:13:49.366839+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:28.023419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.845660+0000","last_change":"2023-10-08T07:13:49.309033+0000","last_active":"2023-10-08T07:13:50.845660+0000","last_peered":"2023-10-08T07:13:50.845660+0000","last_clean":"2023-10-08T07:13:50.845660+0000","last_became_active":"2023-10-08T07:13:49.308757+0000","last_became_peered":"2023-10-08T07:13:49.308757+0000","last_unstale":"2023-10-08T07:13:50.845660+0000","last_undegraded":"2023-10-08T07:13:50.845660+0000","last_fullsized":"2023-10-08T07:13:50.845660+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:16:55.636360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.278218+0000","last_change":"2023-10-08T07:13:33.735924+0000","last_active":"2023-10-08T07:13:49.278218+0000","last_peered":"2023-10-08T07:13:49.278218+0000","last_clean":"2023-10-08T07:13:49.278218+0000","last_became_active":"2023-10-08T07:13:33.735688+0000","last_became_peered":"2023-10-08T07:13:33.735688+0000","last_unstale":"2023-10-08T07:13:49.278218+0000","last_undegraded":"2023-10-08T07:13:49.278218+0000","last_fullsized":"2023-10-08T07:13:49.278218+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286989+0000","last_change":"2023-10-08T07:13:34.308734+0000","last_active":"2023-10-08T07:13:49.286989+0000","last_peered":"2023-10-08T07:13:49.286989+0000","last_clean":"2023-10-08T07:13:49.286989+0000","last_became_active":"2023-10-08T07:13:34.308195+0000","last_became_peered":"2023-10-08T07:13:34.308195+0000","last_unstale":"2023-10-08T07:13:49.286989+0000","last_undegraded":"2023-10-08T07:13:49.286989+0000","last_fullsized":"2023-10-08T07:13:49.286989+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286852+0000","last_change":"2023-10-08T07:13:48.387261+0000","last_active":"2023-10-08T07:13:49.286852+0000","last_peered":"2023-10-08T07:13:49.286852+0000","last_clean":"2023-10-08T07:13:49.286852+0000","last_became_active":"2023-10-08T07:13:48.387022+0000","last_became_peered":"2023-10-08T07:13:48.387022+0000","last_unstale":"2023-10-08T07:13:49.286852+0000","last_undegraded":"2023-10-08T07:13:49.286852+0000","last_fullsized":"2023-10-08T07:13:49.286852+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:09.739690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448409+0000","last_change":"2023-10-08T07:13:49.303091+0000","last_active":"2023-10-08T07:13:50.448409+0000","last_peered":"2023-10-08T07:13:50.448409+0000","last_clean":"2023-10-08T07:13:50.448409+0000","last_became_active":"2023-10-08T07:13:49.302761+0000","last_became_peered":"2023-10-08T07:13:49.302761+0000","last_unstale":"2023-10-08T07:13:50.448409+0000","last_undegraded":"2023-10-08T07:13:50.448409+0000","last_fullsized":"2023-10-08T07:13:50.448409+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:54.844015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294221+0000","last_change":"2023-10-08T07:13:33.741115+0000","last_active":"2023-10-08T07:13:49.294221+0000","last_peered":"2023-10-08T07:13:49.294221+0000","last_clean":"2023-10-08T07:13:49.294221+0000","last_became_active":"2023-10-08T07:13:33.740798+0000","last_became_peered":"2023-10-08T07:13:33.740798+0000","last_unstale":"2023-10-08T07:13:49.294221+0000","last_undegraded":"2023-10-08T07:13:49.294221+0000","last_fullsized":"2023-10-08T07:13:49.294221+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:36.728485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295067+0000","last_change":"2023-10-08T07:13:34.303009+0000","last_active":"2023-10-08T07:13:49.295067+0000","last_peered":"2023-10-08T07:13:49.295067+0000","last_clean":"2023-10-08T07:13:49.295067+0000","last_became_active":"2023-10-08T07:13:34.302682+0000","last_became_peered":"2023-10-08T07:13:34.302682+0000","last_unstale":"2023-10-08T07:13:49.295067+0000","last_undegraded":"2023-10-08T07:13:49.295067+0000","last_fullsized":"2023-10-08T07:13:49.295067+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:47:40.976539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370132+0000","last_change":"2023-10-08T07:13:48.388490+0000","last_active":"2023-10-08T07:13:49.370132+0000","last_peered":"2023-10-08T07:13:49.370132+0000","last_clean":"2023-10-08T07:13:49.370132+0000","last_became_active":"2023-10-08T07:13:48.388178+0000","last_became_peered":"2023-10-08T07:13:48.388178+0000","last_unstale":"2023-10-08T07:13:49.370132+0000","last_undegraded":"2023-10-08T07:13:49.370132+0000","last_fullsized":"2023-10-08T07:13:49.370132+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:46.350569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278865+0000","last_change":"2023-10-08T07:13:49.305322+0000","last_active":"2023-10-08T07:13:50.278865+0000","last_peered":"2023-10-08T07:13:50.278865+0000","last_clean":"2023-10-08T07:13:50.278865+0000","last_became_active":"2023-10-08T07:13:49.305047+0000","last_became_peered":"2023-10-08T07:13:49.305047+0000","last_unstale":"2023-10-08T07:13:50.278865+0000","last_undegraded":"2023-10-08T07:13:50.278865+0000","last_fullsized":"2023-10-08T07:13:50.278865+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:20.461769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289335+0000","last_change":"2023-10-08T07:13:33.738792+0000","last_active":"2023-10-08T07:13:49.289335+0000","last_peered":"2023-10-08T07:13:49.289335+0000","last_clean":"2023-10-08T07:13:49.289335+0000","last_became_active":"2023-10-08T07:13:33.738430+0000","last_became_peered":"2023-10-08T07:13:33.738430+0000","last_unstale":"2023-10-08T07:13:49.289335+0000","last_undegraded":"2023-10-08T07:13:49.289335+0000","last_fullsized":"2023-10-08T07:13:49.289335+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:24:39.688018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305747+0000","last_change":"2023-10-08T07:13:34.305875+0000","last_active":"2023-10-08T07:13:49.305747+0000","last_peered":"2023-10-08T07:13:49.305747+0000","last_clean":"2023-10-08T07:13:49.305747+0000","last_became_active":"2023-10-08T07:13:34.302605+0000","last_became_peered":"2023-10-08T07:13:34.302605+0000","last_unstale":"2023-10-08T07:13:49.305747+0000","last_undegraded":"2023-10-08T07:13:49.305747+0000","last_fullsized":"2023-10-08T07:13:49.305747+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:22.929719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.304787+0000","last_change":"2023-10-08T07:13:48.391812+0000","last_active":"2023-10-08T07:13:49.304787+0000","last_peered":"2023-10-08T07:13:49.304787+0000","last_clean":"2023-10-08T07:13:49.304787+0000","last_became_active":"2023-10-08T07:13:48.391532+0000","last_became_peered":"2023-10-08T07:13:48.391532+0000","last_unstale":"2023-10-08T07:13:49.304787+0000","last_undegraded":"2023-10-08T07:13:49.304787+0000","last_fullsized":"2023-10-08T07:13:49.304787+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:44.203898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.287618+0000","last_change":"2023-10-08T07:13:49.320305+0000","last_active":"2023-10-08T07:13:50.287618+0000","last_peered":"2023-10-08T07:13:50.287618+0000","last_clean":"2023-10-08T07:13:50.287618+0000","last_became_active":"2023-10-08T07:13:49.319999+0000","last_became_peered":"2023-10-08T07:13:49.319999+0000","last_unstale":"2023-10-08T07:13:50.287618+0000","last_undegraded":"2023-10-08T07:13:50.287618+0000","last_fullsized":"2023-10-08T07:13:50.287618+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:06.650395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.303189+0000","last_change":"2023-10-08T07:13:33.734220+0000","last_active":"2023-10-08T07:13:49.303189+0000","last_peered":"2023-10-08T07:13:49.303189+0000","last_clean":"2023-10-08T07:13:49.303189+0000","last_became_active":"2023-10-08T07:13:33.733905+0000","last_became_peered":"2023-10-08T07:13:33.733905+0000","last_unstale":"2023-10-08T07:13:49.303189+0000","last_undegraded":"2023-10-08T07:13:49.303189+0000","last_fullsized":"2023-10-08T07:13:49.303189+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305246+0000","last_change":"2023-10-08T07:13:34.302227+0000","last_active":"2023-10-08T07:13:49.305246+0000","last_peered":"2023-10-08T07:13:49.305246+0000","last_clean":"2023-10-08T07:13:49.305246+0000","last_became_active":"2023-10-08T07:13:34.300127+0000","last_became_peered":"2023-10-08T07:13:34.300127+0000","last_unstale":"2023-10-08T07:13:49.305246+0000","last_undegraded":"2023-10-08T07:13:49.305246+0000","last_fullsized":"2023-10-08T07:13:49.305246+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:49.729707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287348+0000","last_change":"2023-10-08T07:13:48.380775+0000","last_active":"2023-10-08T07:13:49.287348+0000","last_peered":"2023-10-08T07:13:49.287348+0000","last_clean":"2023-10-08T07:13:49.287348+0000","last_became_active":"2023-10-08T07:13:48.380501+0000","last_became_peered":"2023-10-08T07:13:48.380501+0000","last_unstale":"2023-10-08T07:13:49.287348+0000","last_undegraded":"2023-10-08T07:13:49.287348+0000","last_fullsized":"2023-10-08T07:13:49.287348+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:20.813252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357767+0000","last_change":"2023-10-08T07:13:49.309072+0000","last_active":"2023-10-08T07:13:50.357767+0000","last_peered":"2023-10-08T07:13:50.357767+0000","last_clean":"2023-10-08T07:13:50.357767+0000","last_became_active":"2023-10-08T07:13:49.308753+0000","last_became_peered":"2023-10-08T07:13:49.308753+0000","last_unstale":"2023-10-08T07:13:50.357767+0000","last_undegraded":"2023-10-08T07:13:50.357767+0000","last_fullsized":"2023-10-08T07:13:50.357767+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:27.798388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298613+0000","last_change":"2023-10-08T07:13:33.733207+0000","last_active":"2023-10-08T07:13:49.298613+0000","last_peered":"2023-10-08T07:13:49.298613+0000","last_clean":"2023-10-08T07:13:49.298613+0000","last_became_active":"2023-10-08T07:13:33.732903+0000","last_became_peered":"2023-10-08T07:13:33.732903+0000","last_unstale":"2023-10-08T07:13:49.298613+0000","last_undegraded":"2023-10-08T07:13:49.298613+0000","last_fullsized":"2023-10-08T07:13:49.298613+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:34.358428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298923+0000","last_change":"2023-10-08T07:13:34.297602+0000","last_active":"2023-10-08T07:13:49.298923+0000","last_peered":"2023-10-08T07:13:49.298923+0000","last_clean":"2023-10-08T07:13:49.298923+0000","last_became_active":"2023-10-08T07:13:34.296820+0000","last_became_peered":"2023-10-08T07:13:34.296820+0000","last_unstale":"2023-10-08T07:13:49.298923+0000","last_undegraded":"2023-10-08T07:13:49.298923+0000","last_fullsized":"2023-10-08T07:13:49.298923+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:07.812354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298534+0000","last_change":"2023-10-08T07:13:48.374204+0000","last_active":"2023-10-08T07:13:49.298534+0000","last_peered":"2023-10-08T07:13:49.298534+0000","last_clean":"2023-10-08T07:13:49.298534+0000","last_became_active":"2023-10-08T07:13:48.373126+0000","last_became_peered":"2023-10-08T07:13:48.373126+0000","last_unstale":"2023-10-08T07:13:49.298534+0000","last_undegraded":"2023-10-08T07:13:49.298534+0000","last_fullsized":"2023-10-08T07:13:49.298534+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:48.758277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358171+0000","last_change":"2023-10-08T07:13:49.314030+0000","last_active":"2023-10-08T07:13:50.358171+0000","last_peered":"2023-10-08T07:13:50.358171+0000","last_clean":"2023-10-08T07:13:50.358171+0000","last_became_active":"2023-10-08T07:13:49.312711+0000","last_became_peered":"2023-10-08T07:13:49.312711+0000","last_unstale":"2023-10-08T07:13:50.358171+0000","last_undegraded":"2023-10-08T07:13:50.358171+0000","last_fullsized":"2023-10-08T07:13:50.358171+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:39.025998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.308199+0000","last_change":"2023-10-08T07:13:33.737211+0000","last_active":"2023-10-08T07:13:49.308199+0000","last_peered":"2023-10-08T07:13:49.308199+0000","last_clean":"2023-10-08T07:13:49.308199+0000","last_became_active":"2023-10-08T07:13:33.736904+0000","last_became_peered":"2023-10-08T07:13:33.736904+0000","last_unstale":"2023-10-08T07:13:49.308199+0000","last_undegraded":"2023-10-08T07:13:49.308199+0000","last_fullsized":"2023-10-08T07:13:49.308199+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:58.966472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288985+0000","last_change":"2023-10-08T07:13:34.302678+0000","last_active":"2023-10-08T07:13:49.288985+0000","last_peered":"2023-10-08T07:13:49.288985+0000","last_clean":"2023-10-08T07:13:49.288985+0000","last_became_active":"2023-10-08T07:13:34.302406+0000","last_became_peered":"2023-10-08T07:13:34.302406+0000","last_unstale":"2023-10-08T07:13:49.288985+0000","last_undegraded":"2023-10-08T07:13:49.288985+0000","last_fullsized":"2023-10-08T07:13:49.288985+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:48.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.301825+0000","last_change":"2023-10-08T07:13:48.389369+0000","last_active":"2023-10-08T07:13:49.301825+0000","last_peered":"2023-10-08T07:13:49.301825+0000","last_clean":"2023-10-08T07:13:49.301825+0000","last_became_active":"2023-10-08T07:13:48.389067+0000","last_became_peered":"2023-10-08T07:13:48.389067+0000","last_unstale":"2023-10-08T07:13:49.301825+0000","last_undegraded":"2023-10-08T07:13:49.301825+0000","last_fullsized":"2023-10-08T07:13:49.301825+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:36.638472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.286888+0000","last_change":"2023-10-08T07:13:49.311022+0000","last_active":"2023-10-08T07:13:50.286888+0000","last_peered":"2023-10-08T07:13:50.286888+0000","last_clean":"2023-10-08T07:13:50.286888+0000","last_became_active":"2023-10-08T07:13:49.310485+0000","last_became_peered":"2023-10-08T07:13:49.310485+0000","last_unstale":"2023-10-08T07:13:50.286888+0000","last_undegraded":"2023-10-08T07:13:50.286888+0000","last_fullsized":"2023-10-08T07:13:50.286888+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:02.557950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"234'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.314910+0000","last_change":"2023-10-08T07:13:33.742793+0000","last_active":"2023-10-08T07:13:49.314910+0000","last_peered":"2023-10-08T07:13:49.314910+0000","last_clean":"2023-10-08T07:13:49.314910+0000","last_became_active":"2023-10-08T07:13:33.742030+0000","last_became_peered":"2023-10-08T07:13:33.742030+0000","last_unstale":"2023-10-08T07:13:49.314910+0000","last_undegraded":"2023-10-08T07:13:49.314910+0000","last_fullsized":"2023-10-08T07:13:49.314910+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:25:59.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285989+0000","last_change":"2023-10-08T07:13:34.308581+0000","last_active":"2023-10-08T07:13:49.285989+0000","last_peered":"2023-10-08T07:13:49.285989+0000","last_clean":"2023-10-08T07:13:49.285989+0000","last_became_active":"2023-10-08T07:13:34.308210+0000","last_became_peered":"2023-10-08T07:13:34.308210+0000","last_unstale":"2023-10-08T07:13:49.285989+0000","last_undegraded":"2023-10-08T07:13:49.285989+0000","last_fullsized":"2023-10-08T07:13:49.285989+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284904+0000","last_change":"2023-10-08T07:13:48.391100+0000","last_active":"2023-10-08T07:13:49.284904+0000","last_peered":"2023-10-08T07:13:49.284904+0000","last_clean":"2023-10-08T07:13:49.284904+0000","last_became_active":"2023-10-08T07:13:48.390787+0000","last_became_peered":"2023-10-08T07:13:48.390787+0000","last_unstale":"2023-10-08T07:13:49.284904+0000","last_undegraded":"2023-10-08T07:13:49.284904+0000","last_fullsized":"2023-10-08T07:13:49.284904+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:35.542870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278157+0000","last_change":"2023-10-08T07:13:49.310297+0000","last_active":"2023-10-08T07:13:50.278157+0000","last_peered":"2023-10-08T07:13:50.278157+0000","last_clean":"2023-10-08T07:13:50.278157+0000","last_became_active":"2023-10-08T07:13:49.310021+0000","last_became_peered":"2023-10-08T07:13:49.310021+0000","last_unstale":"2023-10-08T07:13:50.278157+0000","last_undegraded":"2023-10-08T07:13:50.278157+0000","last_fullsized":"2023-10-08T07:13:50.278157+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:19.685263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292199+0000","last_change":"2023-10-08T07:13:33.739638+0000","last_active":"2023-10-08T07:13:49.292199+0000","last_peered":"2023-10-08T07:13:49.292199+0000","last_clean":"2023-10-08T07:13:49.292199+0000","last_became_active":"2023-10-08T07:13:33.739352+0000","last_became_peered":"2023-10-08T07:13:33.739352+0000","last_unstale":"2023-10-08T07:13:49.292199+0000","last_undegraded":"2023-10-08T07:13:49.292199+0000","last_fullsized":"2023-10-08T07:13:49.292199+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:11.117318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285459+0000","last_change":"2023-10-08T07:13:34.293674+0000","last_active":"2023-10-08T07:13:49.285459+0000","last_peered":"2023-10-08T07:13:49.285459+0000","last_clean":"2023-10-08T07:13:49.285459+0000","last_became_active":"2023-10-08T07:13:34.292983+0000","last_became_peered":"2023-10-08T07:13:34.292983+0000","last_unstale":"2023-10-08T07:13:49.285459+0000","last_undegraded":"2023-10-08T07:13:49.285459+0000","last_fullsized":"2023-10-08T07:13:49.285459+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:20.477902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294951+0000","last_change":"2023-10-08T07:13:48.388173+0000","last_active":"2023-10-08T07:13:49.294951+0000","last_peered":"2023-10-08T07:13:49.294951+0000","last_clean":"2023-10-08T07:13:49.294951+0000","last_became_active":"2023-10-08T07:13:48.387881+0000","last_became_peered":"2023-10-08T07:13:48.387881+0000","last_unstale":"2023-10-08T07:13:49.294951+0000","last_undegraded":"2023-10-08T07:13:49.294951+0000","last_fullsized":"2023-10-08T07:13:49.294951+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:05.632452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.845606+0000","last_change":"2023-10-08T07:13:49.319691+0000","last_active":"2023-10-08T07:13:50.845606+0000","last_peered":"2023-10-08T07:13:50.845606+0000","last_clean":"2023-10-08T07:13:50.845606+0000","last_became_active":"2023-10-08T07:13:49.319380+0000","last_became_peered":"2023-10-08T07:13:49.319380+0000","last_unstale":"2023-10-08T07:13:50.845606+0000","last_undegraded":"2023-10-08T07:13:50.845606+0000","last_fullsized":"2023-10-08T07:13:50.845606+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:00.308061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.293276+0000","last_change":"2023-10-08T07:13:33.739626+0000","last_active":"2023-10-08T07:13:49.293276+0000","last_peered":"2023-10-08T07:13:49.293276+0000","last_clean":"2023-10-08T07:13:49.293276+0000","last_became_active":"2023-10-08T07:13:33.739348+0000","last_became_peered":"2023-10-08T07:13:33.739348+0000","last_unstale":"2023-10-08T07:13:49.293276+0000","last_undegraded":"2023-10-08T07:13:49.293276+0000","last_fullsized":"2023-10-08T07:13:49.293276+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:23.374596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.309563+0000","last_change":"2023-10-08T07:13:34.300038+0000","last_active":"2023-10-08T07:13:49.309563+0000","last_peered":"2023-10-08T07:13:49.309563+0000","last_clean":"2023-10-08T07:13:49.309563+0000","last_became_active":"2023-10-08T07:13:34.299721+0000","last_became_peered":"2023-10-08T07:13:34.299721+0000","last_unstale":"2023-10-08T07:13:49.309563+0000","last_undegraded":"2023-10-08T07:13:49.309563+0000","last_fullsized":"2023-10-08T07:13:49.309563+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:26.317918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.366615+0000","last_change":"2023-10-08T07:13:48.384247+0000","last_active":"2023-10-08T07:13:49.366615+0000","last_peered":"2023-10-08T07:13:49.366615+0000","last_clean":"2023-10-08T07:13:49.366615+0000","last_became_active":"2023-10-08T07:13:48.383959+0000","last_became_peered":"2023-10-08T07:13:48.383959+0000","last_unstale":"2023-10-08T07:13:49.366615+0000","last_undegraded":"2023-10-08T07:13:49.366615+0000","last_fullsized":"2023-10-08T07:13:49.366615+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:13.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448607+0000","last_change":"2023-10-08T07:13:49.319375+0000","last_active":"2023-10-08T07:13:50.448607+0000","last_peered":"2023-10-08T07:13:50.448607+0000","last_clean":"2023-10-08T07:13:50.448607+0000","last_became_active":"2023-10-08T07:13:49.319046+0000","last_became_peered":"2023-10-08T07:13:49.319046+0000","last_unstale":"2023-10-08T07:13:50.448607+0000","last_undegraded":"2023-10-08T07:13:50.448607+0000","last_fullsized":"2023-10-08T07:13:50.448607+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:43.401179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.302561+0000","last_change":"2023-10-08T07:13:33.742955+0000","last_active":"2023-10-08T07:13:49.302561+0000","last_peered":"2023-10-08T07:13:49.302561+0000","last_clean":"2023-10-08T07:13:49.302561+0000","last_became_active":"2023-10-08T07:13:33.742621+0000","last_became_peered":"2023-10-08T07:13:33.742621+0000","last_unstale":"2023-10-08T07:13:49.302561+0000","last_undegraded":"2023-10-08T07:13:49.302561+0000","last_fullsized":"2023-10-08T07:13:49.302561+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:25.311911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291806+0000","last_change":"2023-10-08T07:13:34.294206+0000","last_active":"2023-10-08T07:13:49.291806+0000","last_peered":"2023-10-08T07:13:49.291806+0000","last_clean":"2023-10-08T07:13:49.291806+0000","last_became_active":"2023-10-08T07:13:34.293897+0000","last_became_peered":"2023-10-08T07:13:34.293897+0000","last_unstale":"2023-10-08T07:13:49.291806+0000","last_undegraded":"2023-10-08T07:13:49.291806+0000","last_fullsized":"2023-10-08T07:13:49.291806+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370513+0000","last_change":"2023-10-08T07:13:48.392407+0000","last_active":"2023-10-08T07:13:49.370513+0000","last_peered":"2023-10-08T07:13:49.370513+0000","last_clean":"2023-10-08T07:13:49.370513+0000","last_became_active":"2023-10-08T07:13:48.392147+0000","last_became_peered":"2023-10-08T07:13:48.392147+0000","last_unstale":"2023-10-08T07:13:49.370513+0000","last_undegraded":"2023-10-08T07:13:49.370513+0000","last_fullsized":"2023-10-08T07:13:49.370513+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:54:27.098972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.286139+0000","last_change":"2023-10-08T07:13:49.312054+0000","last_active":"2023-10-08T07:13:50.286139+0000","last_peered":"2023-10-08T07:13:50.286139+0000","last_clean":"2023-10-08T07:13:50.286139+0000","last_became_active":"2023-10-08T07:13:49.311786+0000","last_became_peered":"2023-10-08T07:13:49.311786+0000","last_unstale":"2023-10-08T07:13:50.286139+0000","last_undegraded":"2023-10-08T07:13:50.286139+0000","last_fullsized":"2023-10-08T07:13:50.286139+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:45.266824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295620+0000","last_change":"2023-10-08T07:13:33.726092+0000","last_active":"2023-10-08T07:13:49.295620+0000","last_peered":"2023-10-08T07:13:49.295620+0000","last_clean":"2023-10-08T07:13:49.295620+0000","last_became_active":"2023-10-08T07:13:33.725701+0000","last_became_peered":"2023-10-08T07:13:33.725701+0000","last_unstale":"2023-10-08T07:13:49.295620+0000","last_undegraded":"2023-10-08T07:13:49.295620+0000","last_fullsized":"2023-10-08T07:13:49.295620+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:51.631016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298048+0000","last_change":"2023-10-08T07:13:34.301017+0000","last_active":"2023-10-08T07:13:49.298048+0000","last_peered":"2023-10-08T07:13:49.298048+0000","last_clean":"2023-10-08T07:13:49.298048+0000","last_became_active":"2023-10-08T07:13:34.300720+0000","last_became_peered":"2023-10-08T07:13:34.300720+0000","last_unstale":"2023-10-08T07:13:49.298048+0000","last_undegraded":"2023-10-08T07:13:49.298048+0000","last_fullsized":"2023-10-08T07:13:49.298048+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:36:05.460057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.304587+0000","last_change":"2023-10-08T07:13:48.389158+0000","last_active":"2023-10-08T07:13:49.304587+0000","last_peered":"2023-10-08T07:13:49.304587+0000","last_clean":"2023-10-08T07:13:49.304587+0000","last_became_active":"2023-10-08T07:13:48.388809+0000","last_became_peered":"2023-10-08T07:13:48.388809+0000","last_unstale":"2023-10-08T07:13:49.304587+0000","last_undegraded":"2023-10-08T07:13:49.304587+0000","last_fullsized":"2023-10-08T07:13:49.304587+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:28.191882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359126+0000","last_change":"2023-10-08T07:13:49.310553+0000","last_active":"2023-10-08T07:13:50.359126+0000","last_peered":"2023-10-08T07:13:50.359126+0000","last_clean":"2023-10-08T07:13:50.359126+0000","last_became_active":"2023-10-08T07:13:49.310222+0000","last_became_peered":"2023-10-08T07:13:49.310222+0000","last_unstale":"2023-10-08T07:13:50.359126+0000","last_undegraded":"2023-10-08T07:13:50.359126+0000","last_fullsized":"2023-10-08T07:13:50.359126+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:50.676315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.283072+0000","last_change":"2023-10-08T07:13:33.726781+0000","last_active":"2023-10-08T07:13:49.283072+0000","last_peered":"2023-10-08T07:13:49.283072+0000","last_clean":"2023-10-08T07:13:49.283072+0000","last_became_active":"2023-10-08T07:13:33.726459+0000","last_became_peered":"2023-10-08T07:13:33.726459+0000","last_unstale":"2023-10-08T07:13:49.283072+0000","last_undegraded":"2023-10-08T07:13:49.283072+0000","last_fullsized":"2023-10-08T07:13:49.283072+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:04.357904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295826+0000","last_change":"2023-10-08T07:13:34.294315+0000","last_active":"2023-10-08T07:13:49.295826+0000","last_peered":"2023-10-08T07:13:49.295826+0000","last_clean":"2023-10-08T07:13:49.295826+0000","last_became_active":"2023-10-08T07:13:34.294065+0000","last_became_peered":"2023-10-08T07:13:34.294065+0000","last_unstale":"2023-10-08T07:13:49.295826+0000","last_undegraded":"2023-10-08T07:13:49.295826+0000","last_fullsized":"2023-10-08T07:13:49.295826+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:37.124047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.372139+0000","last_change":"2023-10-08T07:13:48.389005+0000","last_active":"2023-10-08T07:13:49.372139+0000","last_peered":"2023-10-08T07:13:49.372139+0000","last_clean":"2023-10-08T07:13:49.372139+0000","last_became_active":"2023-10-08T07:13:48.388729+0000","last_became_peered":"2023-10-08T07:13:48.388729+0000","last_unstale":"2023-10-08T07:13:49.372139+0000","last_undegraded":"2023-10-08T07:13:49.372139+0000","last_fullsized":"2023-10-08T07:13:49.372139+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:33.821205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.844921+0000","last_change":"2023-10-08T07:13:49.305463+0000","last_active":"2023-10-08T07:13:50.844921+0000","last_peered":"2023-10-08T07:13:50.844921+0000","last_clean":"2023-10-08T07:13:50.844921+0000","last_became_active":"2023-10-08T07:13:49.305059+0000","last_became_peered":"2023-10-08T07:13:49.305059+0000","last_unstale":"2023-10-08T07:13:50.844921+0000","last_undegraded":"2023-10-08T07:13:50.844921+0000","last_fullsized":"2023-10-08T07:13:50.844921+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:32.920258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282248+0000","last_change":"2023-10-08T07:13:33.729567+0000","last_active":"2023-10-08T07:13:49.282248+0000","last_peered":"2023-10-08T07:13:49.282248+0000","last_clean":"2023-10-08T07:13:49.282248+0000","last_became_active":"2023-10-08T07:13:33.729125+0000","last_became_peered":"2023-10-08T07:13:33.729125+0000","last_unstale":"2023-10-08T07:13:49.282248+0000","last_undegraded":"2023-10-08T07:13:49.282248+0000","last_fullsized":"2023-10-08T07:13:49.282248+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:13.193093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.303010+0000","last_change":"2023-10-08T07:13:34.308836+0000","last_active":"2023-10-08T07:13:49.303010+0000","last_peered":"2023-10-08T07:13:49.303010+0000","last_clean":"2023-10-08T07:13:49.303010+0000","last_became_active":"2023-10-08T07:13:34.308301+0000","last_became_peered":"2023-10-08T07:13:34.308301+0000","last_unstale":"2023-10-08T07:13:49.303010+0000","last_undegraded":"2023-10-08T07:13:49.303010+0000","last_fullsized":"2023-10-08T07:13:49.303010+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:42.058240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285901+0000","last_change":"2023-10-08T07:13:48.390712+0000","last_active":"2023-10-08T07:13:49.285901+0000","last_peered":"2023-10-08T07:13:49.285901+0000","last_clean":"2023-10-08T07:13:49.285901+0000","last_became_active":"2023-10-08T07:13:48.390349+0000","last_became_peered":"2023-10-08T07:13:48.390349+0000","last_unstale":"2023-10-08T07:13:49.285901+0000","last_undegraded":"2023-10-08T07:13:49.285901+0000","last_fullsized":"2023-10-08T07:13:49.285901+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:42.383236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283214+0000","last_change":"2023-10-08T07:13:49.316212+0000","last_active":"2023-10-08T07:13:50.283214+0000","last_peered":"2023-10-08T07:13:50.283214+0000","last_clean":"2023-10-08T07:13:50.283214+0000","last_became_active":"2023-10-08T07:13:49.315804+0000","last_became_peered":"2023-10-08T07:13:49.315804+0000","last_unstale":"2023-10-08T07:13:50.283214+0000","last_undegraded":"2023-10-08T07:13:50.283214+0000","last_fullsized":"2023-10-08T07:13:50.283214+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:46:01.118339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295073+0000","last_change":"2023-10-08T07:13:33.732139+0000","last_active":"2023-10-08T07:13:49.295073+0000","last_peered":"2023-10-08T07:13:49.295073+0000","last_clean":"2023-10-08T07:13:49.295073+0000","last_became_active":"2023-10-08T07:13:33.731892+0000","last_became_peered":"2023-10-08T07:13:33.731892+0000","last_unstale":"2023-10-08T07:13:49.295073+0000","last_undegraded":"2023-10-08T07:13:49.295073+0000","last_fullsized":"2023-10-08T07:13:49.295073+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:07.242491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.299155+0000","last_change":"2023-10-08T07:13:34.299071+0000","last_active":"2023-10-08T07:13:49.299155+0000","last_peered":"2023-10-08T07:13:49.299155+0000","last_clean":"2023-10-08T07:13:49.299155+0000","last_became_active":"2023-10-08T07:13:34.298776+0000","last_became_peered":"2023-10-08T07:13:34.298776+0000","last_unstale":"2023-10-08T07:13:49.299155+0000","last_undegraded":"2023-10-08T07:13:49.299155+0000","last_fullsized":"2023-10-08T07:13:49.299155+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:10.642317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.371881+0000","last_change":"2023-10-08T07:13:48.393982+0000","last_active":"2023-10-08T07:13:49.371881+0000","last_peered":"2023-10-08T07:13:49.371881+0000","last_clean":"2023-10-08T07:13:49.371881+0000","last_became_active":"2023-10-08T07:13:48.393696+0000","last_became_peered":"2023-10-08T07:13:48.393696+0000","last_unstale":"2023-10-08T07:13:49.371881+0000","last_undegraded":"2023-10-08T07:13:49.371881+0000","last_fullsized":"2023-10-08T07:13:49.371881+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:51.579289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358215+0000","last_change":"2023-10-08T07:13:49.312996+0000","last_active":"2023-10-08T07:13:50.358215+0000","last_peered":"2023-10-08T07:13:50.358215+0000","last_clean":"2023-10-08T07:13:50.358215+0000","last_became_active":"2023-10-08T07:13:49.312657+0000","last_became_peered":"2023-10-08T07:13:49.312657+0000","last_unstale":"2023-10-08T07:13:50.358215+0000","last_undegraded":"2023-10-08T07:13:50.358215+0000","last_fullsized":"2023-10-08T07:13:50.358215+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:28.367357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.299244+0000","last_change":"2023-10-08T07:13:33.736960+0000","last_active":"2023-10-08T07:13:49.299244+0000","last_peered":"2023-10-08T07:13:49.299244+0000","last_clean":"2023-10-08T07:13:49.299244+0000","last_became_active":"2023-10-08T07:13:33.736703+0000","last_became_peered":"2023-10-08T07:13:33.736703+0000","last_unstale":"2023-10-08T07:13:49.299244+0000","last_undegraded":"2023-10-08T07:13:49.299244+0000","last_fullsized":"2023-10-08T07:13:49.299244+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:39.196152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.293897+0000","last_change":"2023-10-08T07:13:34.291716+0000","last_active":"2023-10-08T07:13:49.293897+0000","last_peered":"2023-10-08T07:13:49.293897+0000","last_clean":"2023-10-08T07:13:49.293897+0000","last_became_active":"2023-10-08T07:13:34.291300+0000","last_became_peered":"2023-10-08T07:13:34.291300+0000","last_unstale":"2023-10-08T07:13:49.293897+0000","last_undegraded":"2023-10-08T07:13:49.293897+0000","last_fullsized":"2023-10-08T07:13:49.293897+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:07.189339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.277990+0000","last_change":"2023-10-08T07:13:48.383304+0000","last_active":"2023-10-08T07:13:49.277990+0000","last_peered":"2023-10-08T07:13:49.277990+0000","last_clean":"2023-10-08T07:13:49.277990+0000","last_became_active":"2023-10-08T07:13:48.383040+0000","last_became_peered":"2023-10-08T07:13:48.383040+0000","last_unstale":"2023-10-08T07:13:49.277990+0000","last_undegraded":"2023-10-08T07:13:49.277990+0000","last_fullsized":"2023-10-08T07:13:49.277990+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:05.475908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448022+0000","last_change":"2023-10-08T07:13:49.319374+0000","last_active":"2023-10-08T07:13:50.448022+0000","last_peered":"2023-10-08T07:13:50.448022+0000","last_clean":"2023-10-08T07:13:50.448022+0000","last_became_active":"2023-10-08T07:13:49.319049+0000","last_became_peered":"2023-10-08T07:13:49.319049+0000","last_unstale":"2023-10-08T07:13:50.448022+0000","last_undegraded":"2023-10-08T07:13:50.448022+0000","last_fullsized":"2023-10-08T07:13:50.448022+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:09.611966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298151+0000","last_change":"2023-10-08T07:13:33.728187+0000","last_active":"2023-10-08T07:13:49.298151+0000","last_peered":"2023-10-08T07:13:49.298151+0000","last_clean":"2023-10-08T07:13:49.298151+0000","last_became_active":"2023-10-08T07:13:33.727828+0000","last_became_peered":"2023-10-08T07:13:33.727828+0000","last_unstale":"2023-10-08T07:13:49.298151+0000","last_undegraded":"2023-10-08T07:13:49.298151+0000","last_fullsized":"2023-10-08T07:13:49.298151+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:00.539389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285227+0000","last_change":"2023-10-08T07:13:34.293870+0000","last_active":"2023-10-08T07:13:49.285227+0000","last_peered":"2023-10-08T07:13:49.285227+0000","last_clean":"2023-10-08T07:13:49.285227+0000","last_became_active":"2023-10-08T07:13:34.293397+0000","last_became_peered":"2023-10-08T07:13:34.293397+0000","last_unstale":"2023-10-08T07:13:49.285227+0000","last_undegraded":"2023-10-08T07:13:49.285227+0000","last_fullsized":"2023-10-08T07:13:49.285227+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285530+0000","last_change":"2023-10-08T07:13:48.385714+0000","last_active":"2023-10-08T07:13:49.285530+0000","last_peered":"2023-10-08T07:13:49.285530+0000","last_clean":"2023-10-08T07:13:49.285530+0000","last_became_active":"2023-10-08T07:13:48.385420+0000","last_became_peered":"2023-10-08T07:13:48.385420+0000","last_unstale":"2023-10-08T07:13:49.285530+0000","last_undegraded":"2023-10-08T07:13:49.285530+0000","last_fullsized":"2023-10-08T07:13:49.285530+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:53.826072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.729312+0000","last_change":"2023-10-08T07:13:49.302784+0000","last_active":"2023-10-08T07:13:50.729312+0000","last_peered":"2023-10-08T07:13:50.729312+0000","last_clean":"2023-10-08T07:13:50.729312+0000","last_became_active":"2023-10-08T07:13:49.302497+0000","last_became_peered":"2023-10-08T07:13:49.302497+0000","last_unstale":"2023-10-08T07:13:50.729312+0000","last_undegraded":"2023-10-08T07:13:50.729312+0000","last_fullsized":"2023-10-08T07:13:50.729312+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:53.127703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"233'8","reported_seq":24,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.278141+0000","last_change":"2023-10-08T07:13:33.738121+0000","last_active":"2023-10-08T07:13:49.278141+0000","last_peered":"2023-10-08T07:13:49.278141+0000","last_clean":"2023-10-08T07:13:49.278141+0000","last_became_active":"2023-10-08T07:13:33.737903+0000","last_became_peered":"2023-10-08T07:13:33.737903+0000","last_unstale":"2023-10-08T07:13:49.278141+0000","last_undegraded":"2023-10-08T07:13:49.278141+0000","last_fullsized":"2023-10-08T07:13:49.278141+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:15.275258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305608+0000","last_change":"2023-10-08T07:13:34.306066+0000","last_active":"2023-10-08T07:13:49.305608+0000","last_peered":"2023-10-08T07:13:49.305608+0000","last_clean":"2023-10-08T07:13:49.305608+0000","last_became_active":"2023-10-08T07:13:34.302955+0000","last_became_peered":"2023-10-08T07:13:34.302955+0000","last_unstale":"2023-10-08T07:13:49.305608+0000","last_undegraded":"2023-10-08T07:13:49.305608+0000","last_fullsized":"2023-10-08T07:13:49.305608+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:03.795703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.308773+0000","last_change":"2023-10-08T07:13:48.397214+0000","last_active":"2023-10-08T07:13:49.308773+0000","last_peered":"2023-10-08T07:13:49.308773+0000","last_clean":"2023-10-08T07:13:49.308773+0000","last_became_active":"2023-10-08T07:13:48.396880+0000","last_became_peered":"2023-10-08T07:13:48.396880+0000","last_unstale":"2023-10-08T07:13:49.308773+0000","last_undegraded":"2023-10-08T07:13:49.308773+0000","last_fullsized":"2023-10-08T07:13:49.308773+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:03.803430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.729631+0000","last_change":"2023-10-08T07:13:49.318210+0000","last_active":"2023-10-08T07:13:50.729631+0000","last_peered":"2023-10-08T07:13:50.729631+0000","last_clean":"2023-10-08T07:13:50.729631+0000","last_became_active":"2023-10-08T07:13:49.317898+0000","last_became_peered":"2023-10-08T07:13:49.317898+0000","last_unstale":"2023-10-08T07:13:50.729631+0000","last_undegraded":"2023-10-08T07:13:50.729631+0000","last_fullsized":"2023-10-08T07:13:50.729631+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:59:10.726131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286393+0000","last_change":"2023-10-08T07:13:33.730922+0000","last_active":"2023-10-08T07:13:49.286393+0000","last_peered":"2023-10-08T07:13:49.286393+0000","last_clean":"2023-10-08T07:13:49.286393+0000","last_became_active":"2023-10-08T07:13:33.730666+0000","last_became_peered":"2023-10-08T07:13:33.730666+0000","last_unstale":"2023-10-08T07:13:49.286393+0000","last_undegraded":"2023-10-08T07:13:49.286393+0000","last_fullsized":"2023-10-08T07:13:49.286393+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:12.086089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291729+0000","last_change":"2023-10-08T07:13:34.302276+0000","last_active":"2023-10-08T07:13:49.291729+0000","last_peered":"2023-10-08T07:13:49.291729+0000","last_clean":"2023-10-08T07:13:49.291729+0000","last_became_active":"2023-10-08T07:13:34.301988+0000","last_became_peered":"2023-10-08T07:13:34.301988+0000","last_unstale":"2023-10-08T07:13:49.291729+0000","last_undegraded":"2023-10-08T07:13:49.291729+0000","last_fullsized":"2023-10-08T07:13:49.291729+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305499+0000","last_change":"2023-10-08T07:13:48.386216+0000","last_active":"2023-10-08T07:13:49.305499+0000","last_peered":"2023-10-08T07:13:49.305499+0000","last_clean":"2023-10-08T07:13:49.305499+0000","last_became_active":"2023-10-08T07:13:48.385877+0000","last_became_peered":"2023-10-08T07:13:48.385877+0000","last_unstale":"2023-10-08T07:13:49.305499+0000","last_undegraded":"2023-10-08T07:13:49.305499+0000","last_fullsized":"2023-10-08T07:13:49.305499+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:10.951646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:51.086255+0000","last_change":"2023-10-08T07:13:49.313373+0000","last_active":"2023-10-08T07:13:51.086255+0000","last_peered":"2023-10-08T07:13:51.086255+0000","last_clean":"2023-10-08T07:13:51.086255+0000","last_became_active":"2023-10-08T07:13:49.312715+0000","last_became_peered":"2023-10-08T07:13:49.312715+0000","last_unstale":"2023-10-08T07:13:51.086255+0000","last_undegraded":"2023-10-08T07:13:51.086255+0000","last_fullsized":"2023-10-08T07:13:51.086255+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.656415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286524+0000","last_change":"2023-10-08T07:13:33.725167+0000","last_active":"2023-10-08T07:13:49.286524+0000","last_peered":"2023-10-08T07:13:49.286524+0000","last_clean":"2023-10-08T07:13:49.286524+0000","last_became_active":"2023-10-08T07:13:33.724905+0000","last_became_peered":"2023-10-08T07:13:33.724905+0000","last_unstale":"2023-10-08T07:13:49.286524+0000","last_undegraded":"2023-10-08T07:13:49.286524+0000","last_fullsized":"2023-10-08T07:13:49.286524+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:37.525027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288970+0000","last_change":"2023-10-08T07:13:34.296170+0000","last_active":"2023-10-08T07:13:49.288970+0000","last_peered":"2023-10-08T07:13:49.288970+0000","last_clean":"2023-10-08T07:13:49.288970+0000","last_became_active":"2023-10-08T07:13:34.295736+0000","last_became_peered":"2023-10-08T07:13:34.295736+0000","last_unstale":"2023-10-08T07:13:49.288970+0000","last_undegraded":"2023-10-08T07:13:49.288970+0000","last_fullsized":"2023-10-08T07:13:49.288970+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"237'6","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.374726+0000","last_change":"2023-10-08T07:13:48.395168+0000","last_active":"2023-10-08T07:13:49.374726+0000","last_peered":"2023-10-08T07:13:49.374726+0000","last_clean":"2023-10-08T07:13:49.374726+0000","last_became_active":"2023-10-08T07:13:48.394879+0000","last_became_peered":"2023-10-08T07:13:48.394879+0000","last_unstale":"2023-10-08T07:13:49.374726+0000","last_undegraded":"2023-10-08T07:13:49.374726+0000","last_fullsized":"2023-10-08T07:13:49.374726+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:56.271598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359261+0000","last_change":"2023-10-08T07:13:49.303768+0000","last_active":"2023-10-08T07:13:50.359261+0000","last_peered":"2023-10-08T07:13:50.359261+0000","last_clean":"2023-10-08T07:13:50.359261+0000","last_became_active":"2023-10-08T07:13:49.302929+0000","last_became_peered":"2023-10-08T07:13:49.302929+0000","last_unstale":"2023-10-08T07:13:50.359261+0000","last_undegraded":"2023-10-08T07:13:50.359261+0000","last_fullsized":"2023-10-08T07:13:50.359261+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:21.341547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295376+0000","last_change":"2023-10-08T07:13:33.730955+0000","last_active":"2023-10-08T07:13:49.295376+0000","last_peered":"2023-10-08T07:13:49.295376+0000","last_clean":"2023-10-08T07:13:49.295376+0000","last_became_active":"2023-10-08T07:13:33.730676+0000","last_became_peered":"2023-10-08T07:13:33.730676+0000","last_unstale":"2023-10-08T07:13:49.295376+0000","last_undegraded":"2023-10-08T07:13:49.295376+0000","last_fullsized":"2023-10-08T07:13:49.295376+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:28.953862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295664+0000","last_change":"2023-10-08T07:13:34.293585+0000","last_active":"2023-10-08T07:13:49.295664+0000","last_peered":"2023-10-08T07:13:49.295664+0000","last_clean":"2023-10-08T07:13:49.295664+0000","last_became_active":"2023-10-08T07:13:34.293318+0000","last_became_peered":"2023-10-08T07:13:34.293318+0000","last_unstale":"2023-10-08T07:13:49.295664+0000","last_undegraded":"2023-10-08T07:13:49.295664+0000","last_fullsized":"2023-10-08T07:13:49.295664+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:09:34.022529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"237'3","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.368552+0000","last_change":"2023-10-08T07:13:48.388878+0000","last_active":"2023-10-08T07:13:49.368552+0000","last_peered":"2023-10-08T07:13:49.368552+0000","last_clean":"2023-10-08T07:13:49.368552+0000","last_became_active":"2023-10-08T07:13:48.388595+0000","last_became_peered":"2023-10-08T07:13:48.388595+0000","last_unstale":"2023-10-08T07:13:49.368552+0000","last_undegraded":"2023-10-08T07:13:49.368552+0000","last_fullsized":"2023-10-08T07:13:49.368552+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:40.030965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283731+0000","last_change":"2023-10-08T07:13:49.318146+0000","last_active":"2023-10-08T07:13:50.283731+0000","last_peered":"2023-10-08T07:13:50.283731+0000","last_clean":"2023-10-08T07:13:50.283731+0000","last_became_active":"2023-10-08T07:13:49.317834+0000","last_became_peered":"2023-10-08T07:13:49.317834+0000","last_unstale":"2023-10-08T07:13:50.283731+0000","last_undegraded":"2023-10-08T07:13:50.283731+0000","last_fullsized":"2023-10-08T07:13:50.283731+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:22.431006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"234'8","reported_seq":24,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296340+0000","last_change":"2023-10-08T07:13:33.746002+0000","last_active":"2023-10-08T07:13:49.296340+0000","last_peered":"2023-10-08T07:13:49.296340+0000","last_clean":"2023-10-08T07:13:49.296340+0000","last_became_active":"2023-10-08T07:13:33.745697+0000","last_became_peered":"2023-10-08T07:13:33.745697+0000","last_unstale":"2023-10-08T07:13:49.296340+0000","last_undegraded":"2023-10-08T07:13:49.296340+0000","last_fullsized":"2023-10-08T07:13:49.296340+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:25.841040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.283013+0000","last_change":"2023-10-08T07:13:34.293551+0000","last_active":"2023-10-08T07:13:49.283013+0000","last_peered":"2023-10-08T07:13:49.283013+0000","last_clean":"2023-10-08T07:13:49.283013+0000","last_became_active":"2023-10-08T07:13:34.292471+0000","last_became_peered":"2023-10-08T07:13:34.292471+0000","last_unstale":"2023-10-08T07:13:49.283013+0000","last_undegraded":"2023-10-08T07:13:49.283013+0000","last_fullsized":"2023-10-08T07:13:49.283013+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288920+0000","last_change":"2023-10-08T07:13:48.373437+0000","last_active":"2023-10-08T07:13:49.288920+0000","last_peered":"2023-10-08T07:13:49.288920+0000","last_clean":"2023-10-08T07:13:49.288920+0000","last_became_active":"2023-10-08T07:13:48.373147+0000","last_became_peered":"2023-10-08T07:13:48.373147+0000","last_unstale":"2023-10-08T07:13:49.288920+0000","last_undegraded":"2023-10-08T07:13:49.288920+0000","last_fullsized":"2023-10-08T07:13:49.288920+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:07.670172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.447672+0000","last_change":"2023-10-08T07:13:49.311094+0000","last_active":"2023-10-08T07:13:50.447672+0000","last_peered":"2023-10-08T07:13:50.447672+0000","last_clean":"2023-10-08T07:13:50.447672+0000","last_became_active":"2023-10-08T07:13:49.310778+0000","last_became_peered":"2023-10-08T07:13:49.310778+0000","last_unstale":"2023-10-08T07:13:50.447672+0000","last_undegraded":"2023-10-08T07:13:50.447672+0000","last_fullsized":"2023-10-08T07:13:50.447672+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:25:13.654111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"233'3","reported_seq":19,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290932+0000","last_change":"2023-10-08T07:13:33.726791+0000","last_active":"2023-10-08T07:13:49.290932+0000","last_peered":"2023-10-08T07:13:49.290932+0000","last_clean":"2023-10-08T07:13:49.290932+0000","last_became_active":"2023-10-08T07:13:33.726482+0000","last_became_peered":"2023-10-08T07:13:33.726482+0000","last_unstale":"2023-10-08T07:13:49.290932+0000","last_undegraded":"2023-10-08T07:13:49.290932+0000","last_fullsized":"2023-10-08T07:13:49.290932+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:11.409539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289130+0000","last_change":"2023-10-08T07:13:34.290971+0000","last_active":"2023-10-08T07:13:49.289130+0000","last_peered":"2023-10-08T07:13:49.289130+0000","last_clean":"2023-10-08T07:13:49.289130+0000","last_became_active":"2023-10-08T07:13:34.290294+0000","last_became_peered":"2023-10-08T07:13:34.290294+0000","last_unstale":"2023-10-08T07:13:49.289130+0000","last_undegraded":"2023-10-08T07:13:49.289130+0000","last_fullsized":"2023-10-08T07:13:49.289130+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.369932+0000","last_change":"2023-10-08T07:13:48.388537+0000","last_active":"2023-10-08T07:13:49.369932+0000","last_peered":"2023-10-08T07:13:49.369932+0000","last_clean":"2023-10-08T07:13:49.369932+0000","last_became_active":"2023-10-08T07:13:48.388005+0000","last_became_peered":"2023-10-08T07:13:48.388005+0000","last_unstale":"2023-10-08T07:13:49.369932+0000","last_undegraded":"2023-10-08T07:13:49.369932+0000","last_fullsized":"2023-10-08T07:13:49.369932+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:22.087343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.729279+0000","last_change":"2023-10-08T07:13:49.305690+0000","last_active":"2023-10-08T07:13:50.729279+0000","last_peered":"2023-10-08T07:13:50.729279+0000","last_clean":"2023-10-08T07:13:50.729279+0000","last_became_active":"2023-10-08T07:13:49.305404+0000","last_became_peered":"2023-10-08T07:13:49.305404+0000","last_unstale":"2023-10-08T07:13:50.729279+0000","last_undegraded":"2023-10-08T07:13:50.729279+0000","last_fullsized":"2023-10-08T07:13:50.729279+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:22.454210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295328+0000","last_change":"2023-10-08T07:13:33.736423+0000","last_active":"2023-10-08T07:13:49.295328+0000","last_peered":"2023-10-08T07:13:49.295328+0000","last_clean":"2023-10-08T07:13:49.295328+0000","last_became_active":"2023-10-08T07:13:33.735959+0000","last_became_peered":"2023-10-08T07:13:33.735959+0000","last_unstale":"2023-10-08T07:13:49.295328+0000","last_undegraded":"2023-10-08T07:13:49.295328+0000","last_fullsized":"2023-10-08T07:13:49.295328+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:00.365787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285770+0000","last_change":"2023-10-08T07:13:34.302391+0000","last_active":"2023-10-08T07:13:49.285770+0000","last_peered":"2023-10-08T07:13:49.285770+0000","last_clean":"2023-10-08T07:13:49.285770+0000","last_became_active":"2023-10-08T07:13:34.302105+0000","last_became_peered":"2023-10-08T07:13:34.302105+0000","last_unstale":"2023-10-08T07:13:49.285770+0000","last_undegraded":"2023-10-08T07:13:49.285770+0000","last_fullsized":"2023-10-08T07:13:49.285770+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"237'8","reported_seq":20,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.373437+0000","last_change":"2023-10-08T07:13:48.391735+0000","last_active":"2023-10-08T07:13:49.373437+0000","last_peered":"2023-10-08T07:13:49.373437+0000","last_clean":"2023-10-08T07:13:49.373437+0000","last_became_active":"2023-10-08T07:13:48.391330+0000","last_became_peered":"2023-10-08T07:13:48.391330+0000","last_unstale":"2023-10-08T07:13:49.373437+0000","last_undegraded":"2023-10-08T07:13:49.373437+0000","last_fullsized":"2023-10-08T07:13:49.373437+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:23.741971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357935+0000","last_change":"2023-10-08T07:13:49.313956+0000","last_active":"2023-10-08T07:13:50.357935+0000","last_peered":"2023-10-08T07:13:50.357935+0000","last_clean":"2023-10-08T07:13:50.357935+0000","last_became_active":"2023-10-08T07:13:49.313687+0000","last_became_peered":"2023-10-08T07:13:49.313687+0000","last_unstale":"2023-10-08T07:13:50.357935+0000","last_undegraded":"2023-10-08T07:13:50.357935+0000","last_fullsized":"2023-10-08T07:13:50.357935+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:35.653573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284639+0000","last_change":"2023-10-08T07:13:33.735184+0000","last_active":"2023-10-08T07:13:49.284639+0000","last_peered":"2023-10-08T07:13:49.284639+0000","last_clean":"2023-10-08T07:13:49.284639+0000","last_became_active":"2023-10-08T07:13:33.734915+0000","last_became_peered":"2023-10-08T07:13:33.734915+0000","last_unstale":"2023-10-08T07:13:49.284639+0000","last_undegraded":"2023-10-08T07:13:49.284639+0000","last_fullsized":"2023-10-08T07:13:49.284639+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:52.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298171+0000","last_change":"2023-10-08T07:13:34.294204+0000","last_active":"2023-10-08T07:13:49.298171+0000","last_peered":"2023-10-08T07:13:49.298171+0000","last_clean":"2023-10-08T07:13:49.298171+0000","last_became_active":"2023-10-08T07:13:34.293887+0000","last_became_peered":"2023-10-08T07:13:34.293887+0000","last_unstale":"2023-10-08T07:13:49.298171+0000","last_undegraded":"2023-10-08T07:13:49.298171+0000","last_fullsized":"2023-10-08T07:13:49.298171+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:59.699025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.293430+0000","last_change":"2023-10-08T07:13:48.389962+0000","last_active":"2023-10-08T07:13:49.293430+0000","last_peered":"2023-10-08T07:13:49.293430+0000","last_clean":"2023-10-08T07:13:49.293430+0000","last_became_active":"2023-10-08T07:13:48.389607+0000","last_became_peered":"2023-10-08T07:13:48.389607+0000","last_unstale":"2023-10-08T07:13:49.293430+0000","last_undegraded":"2023-10-08T07:13:49.293430+0000","last_fullsized":"2023-10-08T07:13:49.293430+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:27.240946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358517+0000","last_change":"2023-10-08T07:13:49.299000+0000","last_active":"2023-10-08T07:13:50.358517+0000","last_peered":"2023-10-08T07:13:50.358517+0000","last_clean":"2023-10-08T07:13:50.358517+0000","last_became_active":"2023-10-08T07:13:49.298705+0000","last_became_peered":"2023-10-08T07:13:49.298705+0000","last_unstale":"2023-10-08T07:13:50.358517+0000","last_undegraded":"2023-10-08T07:13:50.358517+0000","last_fullsized":"2023-10-08T07:13:50.358517+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:46.814809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.372451+0000","last_change":"2023-10-08T07:13:48.393308+0000","last_active":"2023-10-08T07:13:49.372451+0000","last_peered":"2023-10-08T07:13:49.372451+0000","last_clean":"2023-10-08T07:13:49.372451+0000","last_became_active":"2023-10-08T07:13:48.392985+0000","last_became_peered":"2023-10-08T07:13:48.392985+0000","last_unstale":"2023-10-08T07:13:49.372451+0000","last_undegraded":"2023-10-08T07:13:49.372451+0000","last_fullsized":"2023-10-08T07:13:49.372451+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:16:55.407347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284841+0000","last_change":"2023-10-08T07:13:33.731054+0000","last_active":"2023-10-08T07:13:49.284841+0000","last_peered":"2023-10-08T07:13:49.284841+0000","last_clean":"2023-10-08T07:13:49.284841+0000","last_became_active":"2023-10-08T07:13:33.730782+0000","last_became_peered":"2023-10-08T07:13:33.730782+0000","last_unstale":"2023-10-08T07:13:49.284841+0000","last_undegraded":"2023-10-08T07:13:49.284841+0000","last_fullsized":"2023-10-08T07:13:49.284841+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:06.476515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292156+0000","last_change":"2023-10-08T07:13:34.294260+0000","last_active":"2023-10-08T07:13:49.292156+0000","last_peered":"2023-10-08T07:13:49.292156+0000","last_clean":"2023-10-08T07:13:49.292156+0000","last_became_active":"2023-10-08T07:13:34.293992+0000","last_became_peered":"2023-10-08T07:13:34.293992+0000","last_unstale":"2023-10-08T07:13:49.292156+0000","last_undegraded":"2023-10-08T07:13:49.292156+0000","last_fullsized":"2023-10-08T07:13:49.292156+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358287+0000","last_change":"2023-10-08T07:13:49.309351+0000","last_active":"2023-10-08T07:13:50.358287+0000","last_peered":"2023-10-08T07:13:50.358287+0000","last_clean":"2023-10-08T07:13:50.358287+0000","last_became_active":"2023-10-08T07:13:49.309030+0000","last_became_peered":"2023-10-08T07:13:49.309030+0000","last_unstale":"2023-10-08T07:13:50.358287+0000","last_undegraded":"2023-10-08T07:13:50.358287+0000","last_fullsized":"2023-10-08T07:13:50.358287+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:08.833825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"237'4","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.372634+0000","last_change":"2023-10-08T07:13:48.385188+0000","last_active":"2023-10-08T07:13:49.372634+0000","last_peered":"2023-10-08T07:13:49.372634+0000","last_clean":"2023-10-08T07:13:49.372634+0000","last_became_active":"2023-10-08T07:13:48.384790+0000","last_became_peered":"2023-10-08T07:13:48.384790+0000","last_unstale":"2023-10-08T07:13:49.372634+0000","last_undegraded":"2023-10-08T07:13:49.372634+0000","last_fullsized":"2023-10-08T07:13:49.372634+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:30.625357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295154+0000","last_change":"2023-10-08T07:13:33.738155+0000","last_active":"2023-10-08T07:13:49.295154+0000","last_peered":"2023-10-08T07:13:49.295154+0000","last_clean":"2023-10-08T07:13:49.295154+0000","last_became_active":"2023-10-08T07:13:33.737850+0000","last_became_peered":"2023-10-08T07:13:33.737850+0000","last_unstale":"2023-10-08T07:13:49.295154+0000","last_undegraded":"2023-10-08T07:13:49.295154+0000","last_fullsized":"2023-10-08T07:13:49.295154+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:37.479840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.309407+0000","last_change":"2023-10-08T07:13:34.304888+0000","last_active":"2023-10-08T07:13:49.309407+0000","last_peered":"2023-10-08T07:13:49.309407+0000","last_clean":"2023-10-08T07:13:49.309407+0000","last_became_active":"2023-10-08T07:13:34.304536+0000","last_became_peered":"2023-10-08T07:13:34.304536+0000","last_unstale":"2023-10-08T07:13:49.309407+0000","last_undegraded":"2023-10-08T07:13:49.309407+0000","last_fullsized":"2023-10-08T07:13:49.309407+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:05.467048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283896+0000","last_change":"2023-10-08T07:13:49.315062+0000","last_active":"2023-10-08T07:13:50.283896+0000","last_peered":"2023-10-08T07:13:50.283896+0000","last_clean":"2023-10-08T07:13:50.283896+0000","last_became_active":"2023-10-08T07:13:49.310335+0000","last_became_peered":"2023-10-08T07:13:49.310335+0000","last_unstale":"2023-10-08T07:13:50.283896+0000","last_undegraded":"2023-10-08T07:13:50.283896+0000","last_fullsized":"2023-10-08T07:13:50.283896+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:22.202940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.297773+0000","last_change":"2023-10-08T07:13:33.738971+0000","last_active":"2023-10-08T07:13:49.297773+0000","last_peered":"2023-10-08T07:13:49.297773+0000","last_clean":"2023-10-08T07:13:49.297773+0000","last_became_active":"2023-10-08T07:13:33.738532+0000","last_became_peered":"2023-10-08T07:13:33.738532+0000","last_unstale":"2023-10-08T07:13:49.297773+0000","last_undegraded":"2023-10-08T07:13:49.297773+0000","last_fullsized":"2023-10-08T07:13:49.297773+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:06.806347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.303648+0000","last_change":"2023-10-08T07:13:34.302593+0000","last_active":"2023-10-08T07:13:49.303648+0000","last_peered":"2023-10-08T07:13:49.303648+0000","last_clean":"2023-10-08T07:13:49.303648+0000","last_became_active":"2023-10-08T07:13:34.302300+0000","last_became_peered":"2023-10-08T07:13:34.302300+0000","last_unstale":"2023-10-08T07:13:49.303648+0000","last_undegraded":"2023-10-08T07:13:49.303648+0000","last_fullsized":"2023-10-08T07:13:49.303648+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:13:33.465222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370719+0000","last_change":"2023-10-08T07:13:48.395886+0000","last_active":"2023-10-08T07:13:49.370719+0000","last_peered":"2023-10-08T07:13:49.370719+0000","last_clean":"2023-10-08T07:13:49.370719+0000","last_became_active":"2023-10-08T07:13:48.395585+0000","last_became_peered":"2023-10-08T07:13:48.395585+0000","last_unstale":"2023-10-08T07:13:49.370719+0000","last_undegraded":"2023-10-08T07:13:49.370719+0000","last_fullsized":"2023-10-08T07:13:49.370719+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:35:02.219752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283356+0000","last_change":"2023-10-08T07:13:49.308716+0000","last_active":"2023-10-08T07:13:50.283356+0000","last_peered":"2023-10-08T07:13:50.283356+0000","last_clean":"2023-10-08T07:13:50.283356+0000","last_became_active":"2023-10-08T07:13:49.308414+0000","last_became_peered":"2023-10-08T07:13:49.308414+0000","last_unstale":"2023-10-08T07:13:50.283356+0000","last_undegraded":"2023-10-08T07:13:50.283356+0000","last_fullsized":"2023-10-08T07:13:50.283356+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:30:10.968748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290749+0000","last_change":"2023-10-08T07:13:33.736823+0000","last_active":"2023-10-08T07:13:49.290749+0000","last_peered":"2023-10-08T07:13:49.290749+0000","last_clean":"2023-10-08T07:13:49.290749+0000","last_became_active":"2023-10-08T07:13:33.736537+0000","last_became_peered":"2023-10-08T07:13:33.736537+0000","last_unstale":"2023-10-08T07:13:49.290749+0000","last_undegraded":"2023-10-08T07:13:49.290749+0000","last_fullsized":"2023-10-08T07:13:49.290749+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:48.798820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282798+0000","last_change":"2023-10-08T07:13:34.293354+0000","last_active":"2023-10-08T07:13:49.282798+0000","last_peered":"2023-10-08T07:13:49.282798+0000","last_clean":"2023-10-08T07:13:49.282798+0000","last_became_active":"2023-10-08T07:13:34.288220+0000","last_became_peered":"2023-10-08T07:13:34.288220+0000","last_unstale":"2023-10-08T07:13:49.282798+0000","last_undegraded":"2023-10-08T07:13:49.282798+0000","last_fullsized":"2023-10-08T07:13:49.282798+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370308+0000","last_change":"2023-10-08T07:13:48.397371+0000","last_active":"2023-10-08T07:13:49.370308+0000","last_peered":"2023-10-08T07:13:49.370308+0000","last_clean":"2023-10-08T07:13:49.370308+0000","last_became_active":"2023-10-08T07:13:48.396954+0000","last_became_peered":"2023-10-08T07:13:48.396954+0000","last_unstale":"2023-10-08T07:13:49.370308+0000","last_undegraded":"2023-10-08T07:13:49.370308+0000","last_fullsized":"2023-10-08T07:13:49.370308+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:17.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358686+0000","last_change":"2023-10-08T07:13:49.309391+0000","last_active":"2023-10-08T07:13:50.358686+0000","last_peered":"2023-10-08T07:13:50.358686+0000","last_clean":"2023-10-08T07:13:50.358686+0000","last_became_active":"2023-10-08T07:13:49.309100+0000","last_became_peered":"2023-10-08T07:13:49.309100+0000","last_unstale":"2023-10-08T07:13:50.358686+0000","last_undegraded":"2023-10-08T07:13:50.358686+0000","last_fullsized":"2023-10-08T07:13:50.358686+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:36.841571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294941+0000","last_change":"2023-10-08T07:13:33.724335+0000","last_active":"2023-10-08T07:13:49.294941+0000","last_peered":"2023-10-08T07:13:49.294941+0000","last_clean":"2023-10-08T07:13:49.294941+0000","last_became_active":"2023-10-08T07:13:33.724043+0000","last_became_peered":"2023-10-08T07:13:33.724043+0000","last_unstale":"2023-10-08T07:13:49.294941+0000","last_undegraded":"2023-10-08T07:13:49.294941+0000","last_fullsized":"2023-10-08T07:13:49.294941+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:58.128241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296177+0000","last_change":"2023-10-08T07:13:34.301668+0000","last_active":"2023-10-08T07:13:49.296177+0000","last_peered":"2023-10-08T07:13:49.296177+0000","last_clean":"2023-10-08T07:13:49.296177+0000","last_became_active":"2023-10-08T07:13:34.298698+0000","last_became_peered":"2023-10-08T07:13:34.298698+0000","last_unstale":"2023-10-08T07:13:49.296177+0000","last_undegraded":"2023-10-08T07:13:49.296177+0000","last_fullsized":"2023-10-08T07:13:49.296177+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370198+0000","last_change":"2023-10-08T07:13:48.399044+0000","last_active":"2023-10-08T07:13:49.370198+0000","last_peered":"2023-10-08T07:13:49.370198+0000","last_clean":"2023-10-08T07:13:49.370198+0000","last_became_active":"2023-10-08T07:13:48.398793+0000","last_became_peered":"2023-10-08T07:13:48.398793+0000","last_unstale":"2023-10-08T07:13:49.370198+0000","last_undegraded":"2023-10-08T07:13:49.370198+0000","last_fullsized":"2023-10-08T07:13:49.370198+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:38.004516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278331+0000","last_change":"2023-10-08T07:13:49.303740+0000","last_active":"2023-10-08T07:13:50.278331+0000","last_peered":"2023-10-08T07:13:50.278331+0000","last_clean":"2023-10-08T07:13:50.278331+0000","last_became_active":"2023-10-08T07:13:49.303416+0000","last_became_peered":"2023-10-08T07:13:49.303416+0000","last_unstale":"2023-10-08T07:13:50.278331+0000","last_undegraded":"2023-10-08T07:13:50.278331+0000","last_fullsized":"2023-10-08T07:13:50.278331+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:38:51.904860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.299129+0000","last_change":"2023-10-08T07:13:33.732312+0000","last_active":"2023-10-08T07:13:49.299129+0000","last_peered":"2023-10-08T07:13:49.299129+0000","last_clean":"2023-10-08T07:13:49.299129+0000","last_became_active":"2023-10-08T07:13:33.731998+0000","last_became_peered":"2023-10-08T07:13:33.731998+0000","last_unstale":"2023-10-08T07:13:49.299129+0000","last_undegraded":"2023-10-08T07:13:49.299129+0000","last_fullsized":"2023-10-08T07:13:49.299129+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:10:53.964103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.293727+0000","last_change":"2023-10-08T07:13:34.293829+0000","last_active":"2023-10-08T07:13:49.293727+0000","last_peered":"2023-10-08T07:13:49.293727+0000","last_clean":"2023-10-08T07:13:49.293727+0000","last_became_active":"2023-10-08T07:13:34.293552+0000","last_became_peered":"2023-10-08T07:13:34.293552+0000","last_unstale":"2023-10-08T07:13:49.293727+0000","last_undegraded":"2023-10-08T07:13:49.293727+0000","last_fullsized":"2023-10-08T07:13:49.293727+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:38.485847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294683+0000","last_change":"2023-10-08T07:13:48.382427+0000","last_active":"2023-10-08T07:13:49.294683+0000","last_peered":"2023-10-08T07:13:49.294683+0000","last_clean":"2023-10-08T07:13:49.294683+0000","last_became_active":"2023-10-08T07:13:48.382141+0000","last_became_peered":"2023-10-08T07:13:48.382141+0000","last_unstale":"2023-10-08T07:13:49.294683+0000","last_undegraded":"2023-10-08T07:13:49.294683+0000","last_fullsized":"2023-10-08T07:13:49.294683+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:26.319310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.729707+0000","last_change":"2023-10-08T07:13:49.302647+0000","last_active":"2023-10-08T07:13:50.729707+0000","last_peered":"2023-10-08T07:13:50.729707+0000","last_clean":"2023-10-08T07:13:50.729707+0000","last_became_active":"2023-10-08T07:13:49.302334+0000","last_became_peered":"2023-10-08T07:13:49.302334+0000","last_unstale":"2023-10-08T07:13:50.729707+0000","last_undegraded":"2023-10-08T07:13:50.729707+0000","last_fullsized":"2023-10-08T07:13:50.729707+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:21.552769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"233'4","reported_seq":20,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.313059+0000","last_change":"2023-10-08T07:13:33.743017+0000","last_active":"2023-10-08T07:13:49.313059+0000","last_peered":"2023-10-08T07:13:49.313059+0000","last_clean":"2023-10-08T07:13:49.313059+0000","last_became_active":"2023-10-08T07:13:33.742582+0000","last_became_peered":"2023-10-08T07:13:33.742582+0000","last_unstale":"2023-10-08T07:13:49.313059+0000","last_undegraded":"2023-10-08T07:13:49.313059+0000","last_fullsized":"2023-10-08T07:13:49.313059+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:38:56.287497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296385+0000","last_change":"2023-10-08T07:13:34.286367+0000","last_active":"2023-10-08T07:13:49.296385+0000","last_peered":"2023-10-08T07:13:49.296385+0000","last_clean":"2023-10-08T07:13:49.296385+0000","last_became_active":"2023-10-08T07:13:34.285016+0000","last_became_peered":"2023-10-08T07:13:34.285016+0000","last_unstale":"2023-10-08T07:13:49.296385+0000","last_undegraded":"2023-10-08T07:13:49.296385+0000","last_fullsized":"2023-10-08T07:13:49.296385+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:41.466102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.302698+0000","last_change":"2023-10-08T07:13:48.394830+0000","last_active":"2023-10-08T07:13:49.302698+0000","last_peered":"2023-10-08T07:13:49.302698+0000","last_clean":"2023-10-08T07:13:49.302698+0000","last_became_active":"2023-10-08T07:13:48.394550+0000","last_became_peered":"2023-10-08T07:13:48.394550+0000","last_unstale":"2023-10-08T07:13:49.302698+0000","last_undegraded":"2023-10-08T07:13:49.302698+0000","last_fullsized":"2023-10-08T07:13:49.302698+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:23.741971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.282589+0000","last_change":"2023-10-08T07:13:49.319738+0000","last_active":"2023-10-08T07:13:50.282589+0000","last_peered":"2023-10-08T07:13:50.282589+0000","last_clean":"2023-10-08T07:13:50.282589+0000","last_became_active":"2023-10-08T07:13:49.319429+0000","last_became_peered":"2023-10-08T07:13:49.319429+0000","last_unstale":"2023-10-08T07:13:50.282589+0000","last_undegraded":"2023-10-08T07:13:50.282589+0000","last_fullsized":"2023-10-08T07:13:50.282589+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:31.650499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.302892+0000","last_change":"2023-10-08T07:13:34.295873+0000","last_active":"2023-10-08T07:13:49.302892+0000","last_peered":"2023-10-08T07:13:49.302892+0000","last_clean":"2023-10-08T07:13:49.302892+0000","last_became_active":"2023-10-08T07:13:34.295546+0000","last_became_peered":"2023-10-08T07:13:34.295546+0000","last_unstale":"2023-10-08T07:13:49.302892+0000","last_undegraded":"2023-10-08T07:13:49.302892+0000","last_fullsized":"2023-10-08T07:13:49.302892+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:46:53.896975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448531+0000","last_change":"2023-10-08T07:13:49.312717+0000","last_active":"2023-10-08T07:13:50.448531+0000","last_peered":"2023-10-08T07:13:50.448531+0000","last_clean":"2023-10-08T07:13:50.448531+0000","last_became_active":"2023-10-08T07:13:49.312448+0000","last_became_peered":"2023-10-08T07:13:49.312448+0000","last_unstale":"2023-10-08T07:13:50.448531+0000","last_undegraded":"2023-10-08T07:13:50.448531+0000","last_fullsized":"2023-10-08T07:13:50.448531+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:52.376348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294645+0000","last_change":"2023-10-08T07:13:34.301071+0000","last_active":"2023-10-08T07:13:49.294645+0000","last_peered":"2023-10-08T07:13:49.294645+0000","last_clean":"2023-10-08T07:13:49.294645+0000","last_became_active":"2023-10-08T07:13:34.300772+0000","last_became_peered":"2023-10-08T07:13:34.300772+0000","last_unstale":"2023-10-08T07:13:49.294645+0000","last_undegraded":"2023-10-08T07:13:49.294645+0000","last_fullsized":"2023-10-08T07:13:49.294645+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:59:54.643977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.282259+0000","last_change":"2023-10-08T07:13:49.320349+0000","last_active":"2023-10-08T07:13:50.282259+0000","last_peered":"2023-10-08T07:13:50.282259+0000","last_clean":"2023-10-08T07:13:50.282259+0000","last_became_active":"2023-10-08T07:13:49.320036+0000","last_became_peered":"2023-10-08T07:13:49.320036+0000","last_unstale":"2023-10-08T07:13:50.282259+0000","last_undegraded":"2023-10-08T07:13:50.282259+0000","last_fullsized":"2023-10-08T07:13:50.282259+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:44.601725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298904+0000","last_change":"2023-10-08T07:13:34.300974+0000","last_active":"2023-10-08T07:13:49.298904+0000","last_peered":"2023-10-08T07:13:49.298904+0000","last_clean":"2023-10-08T07:13:49.298904+0000","last_became_active":"2023-10-08T07:13:34.300529+0000","last_became_peered":"2023-10-08T07:13:34.300529+0000","last_unstale":"2023-10-08T07:13:49.298904+0000","last_undegraded":"2023-10-08T07:13:49.298904+0000","last_fullsized":"2023-10-08T07:13:49.298904+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:17.001444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278016+0000","last_change":"2023-10-08T07:13:49.306936+0000","last_active":"2023-10-08T07:13:50.278016+0000","last_peered":"2023-10-08T07:13:50.278016+0000","last_clean":"2023-10-08T07:13:50.278016+0000","last_became_active":"2023-10-08T07:13:49.306284+0000","last_became_peered":"2023-10-08T07:13:49.306284+0000","last_unstale":"2023-10-08T07:13:50.278016+0000","last_undegraded":"2023-10-08T07:13:50.278016+0000","last_fullsized":"2023-10-08T07:13:50.278016+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:51.779526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287748+0000","last_change":"2023-10-08T07:13:34.293712+0000","last_active":"2023-10-08T07:13:49.287748+0000","last_peered":"2023-10-08T07:13:49.287748+0000","last_clean":"2023-10-08T07:13:49.287748+0000","last_became_active":"2023-10-08T07:13:34.293415+0000","last_became_peered":"2023-10-08T07:13:34.293415+0000","last_unstale":"2023-10-08T07:13:49.287748+0000","last_undegraded":"2023-10-08T07:13:49.287748+0000","last_fullsized":"2023-10-08T07:13:49.287748+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:47.966787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359086+0000","last_change":"2023-10-08T07:13:49.316786+0000","last_active":"2023-10-08T07:13:50.359086+0000","last_peered":"2023-10-08T07:13:50.359086+0000","last_clean":"2023-10-08T07:13:50.359086+0000","last_became_active":"2023-10-08T07:13:49.316437+0000","last_became_peered":"2023-10-08T07:13:49.316437+0000","last_unstale":"2023-10-08T07:13:50.359086+0000","last_undegraded":"2023-10-08T07:13:50.359086+0000","last_fullsized":"2023-10-08T07:13:50.359086+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:14.040352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282033+0000","last_change":"2023-10-08T07:13:34.297816+0000","last_active":"2023-10-08T07:13:49.282033+0000","last_peered":"2023-10-08T07:13:49.282033+0000","last_clean":"2023-10-08T07:13:49.282033+0000","last_became_active":"2023-10-08T07:13:34.297544+0000","last_became_peered":"2023-10-08T07:13:34.297544+0000","last_unstale":"2023-10-08T07:13:49.282033+0000","last_undegraded":"2023-10-08T07:13:49.282033+0000","last_fullsized":"2023-10-08T07:13:49.282033+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.846250+0000","last_change":"2023-10-08T07:13:49.313586+0000","last_active":"2023-10-08T07:13:50.846250+0000","last_peered":"2023-10-08T07:13:50.846250+0000","last_clean":"2023-10-08T07:13:50.846250+0000","last_became_active":"2023-10-08T07:13:49.312969+0000","last_became_peered":"2023-10-08T07:13:49.312969+0000","last_unstale":"2023-10-08T07:13:50.846250+0000","last_undegraded":"2023-10-08T07:13:50.846250+0000","last_fullsized":"2023-10-08T07:13:50.846250+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:02.919590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287925+0000","last_change":"2023-10-08T07:13:34.310969+0000","last_active":"2023-10-08T07:13:49.287925+0000","last_peered":"2023-10-08T07:13:49.287925+0000","last_clean":"2023-10-08T07:13:49.287925+0000","last_became_active":"2023-10-08T07:13:34.310679+0000","last_became_peered":"2023-10-08T07:13:34.310679+0000","last_unstale":"2023-10-08T07:13:49.287925+0000","last_undegraded":"2023-10-08T07:13:49.287925+0000","last_fullsized":"2023-10-08T07:13:49.287925+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.284208+0000","last_change":"2023-10-08T07:13:49.314485+0000","last_active":"2023-10-08T07:13:50.284208+0000","last_peered":"2023-10-08T07:13:50.284208+0000","last_clean":"2023-10-08T07:13:50.284208+0000","last_became_active":"2023-10-08T07:13:49.312136+0000","last_became_peered":"2023-10-08T07:13:49.312136+0000","last_unstale":"2023-10-08T07:13:50.284208+0000","last_undegraded":"2023-10-08T07:13:50.284208+0000","last_fullsized":"2023-10-08T07:13:50.284208+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:59.113588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298999+0000","last_change":"2023-10-08T07:13:34.304704+0000","last_active":"2023-10-08T07:13:49.298999+0000","last_peered":"2023-10-08T07:13:49.298999+0000","last_clean":"2023-10-08T07:13:49.298999+0000","last_became_active":"2023-10-08T07:13:34.304032+0000","last_became_peered":"2023-10-08T07:13:34.304032+0000","last_unstale":"2023-10-08T07:13:49.298999+0000","last_undegraded":"2023-10-08T07:13:49.298999+0000","last_fullsized":"2023-10-08T07:13:49.298999+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:15.818894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:51.085872+0000","last_change":"2023-10-08T07:13:49.310313+0000","last_active":"2023-10-08T07:13:51.085872+0000","last_peered":"2023-10-08T07:13:51.085872+0000","last_clean":"2023-10-08T07:13:51.085872+0000","last_became_active":"2023-10-08T07:13:49.310027+0000","last_became_peered":"2023-10-08T07:13:49.310027+0000","last_unstale":"2023-10-08T07:13:51.085872+0000","last_undegraded":"2023-10-08T07:13:51.085872+0000","last_fullsized":"2023-10-08T07:13:51.085872+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:23.105286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292077+0000","last_change":"2023-10-08T07:13:34.298953+0000","last_active":"2023-10-08T07:13:49.292077+0000","last_peered":"2023-10-08T07:13:49.292077+0000","last_clean":"2023-10-08T07:13:49.292077+0000","last_became_active":"2023-10-08T07:13:34.298694+0000","last_became_peered":"2023-10-08T07:13:34.298694+0000","last_unstale":"2023-10-08T07:13:49.292077+0000","last_undegraded":"2023-10-08T07:13:49.292077+0000","last_fullsized":"2023-10-08T07:13:49.292077+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.728973+0000","last_change":"2023-10-08T07:13:49.307150+0000","last_active":"2023-10-08T07:13:50.728973+0000","last_peered":"2023-10-08T07:13:50.728973+0000","last_clean":"2023-10-08T07:13:50.728973+0000","last_became_active":"2023-10-08T07:13:49.306772+0000","last_became_peered":"2023-10-08T07:13:49.306772+0000","last_unstale":"2023-10-08T07:13:50.728973+0000","last_undegraded":"2023-10-08T07:13:50.728973+0000","last_fullsized":"2023-10-08T07:13:50.728973+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:25.284173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.277869+0000","last_change":"2023-10-08T07:13:34.293797+0000","last_active":"2023-10-08T07:13:49.277869+0000","last_peered":"2023-10-08T07:13:49.277869+0000","last_clean":"2023-10-08T07:13:49.277869+0000","last_became_active":"2023-10-08T07:13:34.289224+0000","last_became_peered":"2023-10-08T07:13:34.289224+0000","last_unstale":"2023-10-08T07:13:49.277869+0000","last_undegraded":"2023-10-08T07:13:49.277869+0000","last_fullsized":"2023-10-08T07:13:49.277869+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357969+0000","last_change":"2023-10-08T07:13:49.303551+0000","last_active":"2023-10-08T07:13:50.357969+0000","last_peered":"2023-10-08T07:13:50.357969+0000","last_clean":"2023-10-08T07:13:50.357969+0000","last_became_active":"2023-10-08T07:13:49.303258+0000","last_became_peered":"2023-10-08T07:13:49.303258+0000","last_unstale":"2023-10-08T07:13:50.357969+0000","last_undegraded":"2023-10-08T07:13:50.357969+0000","last_fullsized":"2023-10-08T07:13:50.357969+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:08.504585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295174+0000","last_change":"2023-10-08T07:13:34.291585+0000","last_active":"2023-10-08T07:13:49.295174+0000","last_peered":"2023-10-08T07:13:49.295174+0000","last_clean":"2023-10-08T07:13:49.295174+0000","last_became_active":"2023-10-08T07:13:34.291125+0000","last_became_peered":"2023-10-08T07:13:34.291125+0000","last_unstale":"2023-10-08T07:13:49.295174+0000","last_undegraded":"2023-10-08T07:13:49.295174+0000","last_fullsized":"2023-10-08T07:13:49.295174+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:14.399506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.844917+0000","last_change":"2023-10-08T07:13:49.313885+0000","last_active":"2023-10-08T07:13:50.844917+0000","last_peered":"2023-10-08T07:13:50.844917+0000","last_clean":"2023-10-08T07:13:50.844917+0000","last_became_active":"2023-10-08T07:13:49.313596+0000","last_became_peered":"2023-10-08T07:13:49.313596+0000","last_unstale":"2023-10-08T07:13:50.844917+0000","last_undegraded":"2023-10-08T07:13:50.844917+0000","last_fullsized":"2023-10-08T07:13:50.844917+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:25.533946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296171+0000","last_change":"2023-10-08T07:13:34.297774+0000","last_active":"2023-10-08T07:13:49.296171+0000","last_peered":"2023-10-08T07:13:49.296171+0000","last_clean":"2023-10-08T07:13:49.296171+0000","last_became_active":"2023-10-08T07:13:34.297492+0000","last_became_peered":"2023-10-08T07:13:34.297492+0000","last_unstale":"2023-10-08T07:13:49.296171+0000","last_undegraded":"2023-10-08T07:13:49.296171+0000","last_fullsized":"2023-10-08T07:13:49.296171+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.277878+0000","last_change":"2023-10-08T07:13:49.298115+0000","last_active":"2023-10-08T07:13:50.277878+0000","last_peered":"2023-10-08T07:13:50.277878+0000","last_clean":"2023-10-08T07:13:50.277878+0000","last_became_active":"2023-10-08T07:13:49.297582+0000","last_became_peered":"2023-10-08T07:13:49.297582+0000","last_unstale":"2023-10-08T07:13:50.277878+0000","last_undegraded":"2023-10-08T07:13:50.277878+0000","last_fullsized":"2023-10-08T07:13:50.277878+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:11.399029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.302860+0000","last_change":"2023-10-08T07:13:34.294431+0000","last_active":"2023-10-08T07:13:49.302860+0000","last_peered":"2023-10-08T07:13:49.302860+0000","last_clean":"2023-10-08T07:13:49.302860+0000","last_became_active":"2023-10-08T07:13:34.294129+0000","last_became_peered":"2023-10-08T07:13:34.294129+0000","last_unstale":"2023-10-08T07:13:49.302860+0000","last_undegraded":"2023-10-08T07:13:49.302860+0000","last_fullsized":"2023-10-08T07:13:49.302860+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358949+0000","last_change":"2023-10-08T07:13:49.313393+0000","last_active":"2023-10-08T07:13:50.358949+0000","last_peered":"2023-10-08T07:13:50.358949+0000","last_clean":"2023-10-08T07:13:50.358949+0000","last_became_active":"2023-10-08T07:13:49.313094+0000","last_became_peered":"2023-10-08T07:13:49.313094+0000","last_unstale":"2023-10-08T07:13:50.358949+0000","last_undegraded":"2023-10-08T07:13:50.358949+0000","last_fullsized":"2023-10-08T07:13:50.358949+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:06.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305137+0000","last_change":"2023-10-08T07:13:34.297677+0000","last_active":"2023-10-08T07:13:49.305137+0000","last_peered":"2023-10-08T07:13:49.305137+0000","last_clean":"2023-10-08T07:13:49.305137+0000","last_became_active":"2023-10-08T07:13:34.296526+0000","last_became_peered":"2023-10-08T07:13:34.296526+0000","last_unstale":"2023-10-08T07:13:49.305137+0000","last_undegraded":"2023-10-08T07:13:49.305137+0000","last_fullsized":"2023-10-08T07:13:49.305137+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:51.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.285459+0000","last_change":"2023-10-08T07:13:49.315159+0000","last_active":"2023-10-08T07:13:50.285459+0000","last_peered":"2023-10-08T07:13:50.285459+0000","last_clean":"2023-10-08T07:13:50.285459+0000","last_became_active":"2023-10-08T07:13:49.314845+0000","last_became_peered":"2023-10-08T07:13:49.314845+0000","last_unstale":"2023-10-08T07:13:50.285459+0000","last_undegraded":"2023-10-08T07:13:50.285459+0000","last_fullsized":"2023-10-08T07:13:50.285459+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:02:25.205836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305789+0000","last_change":"2023-10-08T07:13:34.295222+0000","last_active":"2023-10-08T07:13:49.305789+0000","last_peered":"2023-10-08T07:13:49.305789+0000","last_clean":"2023-10-08T07:13:49.305789+0000","last_became_active":"2023-10-08T07:13:34.289517+0000","last_became_peered":"2023-10-08T07:13:34.289517+0000","last_unstale":"2023-10-08T07:13:49.305789+0000","last_undegraded":"2023-10-08T07:13:49.305789+0000","last_fullsized":"2023-10-08T07:13:49.305789+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:38.224838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.287377+0000","last_change":"2023-10-08T07:13:49.301147+0000","last_active":"2023-10-08T07:13:50.287377+0000","last_peered":"2023-10-08T07:13:50.287377+0000","last_clean":"2023-10-08T07:13:50.287377+0000","last_became_active":"2023-10-08T07:13:49.300846+0000","last_became_peered":"2023-10-08T07:13:49.300846+0000","last_unstale":"2023-10-08T07:13:50.287377+0000","last_undegraded":"2023-10-08T07:13:50.287377+0000","last_fullsized":"2023-10-08T07:13:50.287377+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:13.776104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291605+0000","last_change":"2023-10-08T07:13:34.299238+0000","last_active":"2023-10-08T07:13:49.291605+0000","last_peered":"2023-10-08T07:13:49.291605+0000","last_clean":"2023-10-08T07:13:49.291605+0000","last_became_active":"2023-10-08T07:13:34.298585+0000","last_became_peered":"2023-10-08T07:13:34.298585+0000","last_unstale":"2023-10-08T07:13:49.291605+0000","last_undegraded":"2023-10-08T07:13:49.291605+0000","last_fullsized":"2023-10-08T07:13:49.291605+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.846256+0000","last_change":"2023-10-08T07:13:49.314244+0000","last_active":"2023-10-08T07:13:50.846256+0000","last_peered":"2023-10-08T07:13:50.846256+0000","last_clean":"2023-10-08T07:13:50.846256+0000","last_became_active":"2023-10-08T07:13:49.313960+0000","last_became_peered":"2023-10-08T07:13:49.313960+0000","last_unstale":"2023-10-08T07:13:50.846256+0000","last_undegraded":"2023-10-08T07:13:50.846256+0000","last_fullsized":"2023-10-08T07:13:50.846256+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:36:04.687046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292035+0000","last_change":"2023-10-08T07:13:34.308624+0000","last_active":"2023-10-08T07:13:49.292035+0000","last_peered":"2023-10-08T07:13:49.292035+0000","last_clean":"2023-10-08T07:13:49.292035+0000","last_became_active":"2023-10-08T07:13:34.308246+0000","last_became_peered":"2023-10-08T07:13:34.308246+0000","last_unstale":"2023-10-08T07:13:49.292035+0000","last_undegraded":"2023-10-08T07:13:49.292035+0000","last_fullsized":"2023-10-08T07:13:49.292035+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283750+0000","last_change":"2023-10-08T07:13:49.303206+0000","last_active":"2023-10-08T07:13:50.283750+0000","last_peered":"2023-10-08T07:13:50.283750+0000","last_clean":"2023-10-08T07:13:50.283750+0000","last_became_active":"2023-10-08T07:13:49.302904+0000","last_became_peered":"2023-10-08T07:13:49.302904+0000","last_unstale":"2023-10-08T07:13:50.283750+0000","last_undegraded":"2023-10-08T07:13:50.283750+0000","last_fullsized":"2023-10-08T07:13:50.283750+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:02:27.339829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284637+0000","last_change":"2023-10-08T07:13:34.300821+0000","last_active":"2023-10-08T07:13:49.284637+0000","last_peered":"2023-10-08T07:13:49.284637+0000","last_clean":"2023-10-08T07:13:49.284637+0000","last_became_active":"2023-10-08T07:13:34.300558+0000","last_became_peered":"2023-10-08T07:13:34.300558+0000","last_unstale":"2023-10-08T07:13:49.284637+0000","last_undegraded":"2023-10-08T07:13:49.284637+0000","last_fullsized":"2023-10-08T07:13:49.284637+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.286407+0000","last_change":"2023-10-08T07:13:49.318602+0000","last_active":"2023-10-08T07:13:50.286407+0000","last_peered":"2023-10-08T07:13:50.286407+0000","last_clean":"2023-10-08T07:13:50.286407+0000","last_became_active":"2023-10-08T07:13:49.318304+0000","last_became_peered":"2023-10-08T07:13:49.318304+0000","last_unstale":"2023-10-08T07:13:50.286407+0000","last_undegraded":"2023-10-08T07:13:50.286407+0000","last_fullsized":"2023-10-08T07:13:50.286407+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:29.701367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.302401+0000","last_change":"2023-10-08T07:13:34.309619+0000","last_active":"2023-10-08T07:13:49.302401+0000","last_peered":"2023-10-08T07:13:49.302401+0000","last_clean":"2023-10-08T07:13:49.302401+0000","last_became_active":"2023-10-08T07:13:34.308202+0000","last_became_peered":"2023-10-08T07:13:34.308202+0000","last_unstale":"2023-10-08T07:13:49.302401+0000","last_undegraded":"2023-10-08T07:13:49.302401+0000","last_fullsized":"2023-10-08T07:13:49.302401+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:59.914799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.281862+0000","last_change":"2023-10-08T07:13:49.300308+0000","last_active":"2023-10-08T07:13:50.281862+0000","last_peered":"2023-10-08T07:13:50.281862+0000","last_clean":"2023-10-08T07:13:50.281862+0000","last_became_active":"2023-10-08T07:13:49.299809+0000","last_became_peered":"2023-10-08T07:13:49.299809+0000","last_unstale":"2023-10-08T07:13:50.281862+0000","last_undegraded":"2023-10-08T07:13:50.281862+0000","last_fullsized":"2023-10-08T07:13:50.281862+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:07.278879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295969+0000","last_change":"2023-10-08T07:13:34.298407+0000","last_active":"2023-10-08T07:13:49.295969+0000","last_peered":"2023-10-08T07:13:49.295969+0000","last_clean":"2023-10-08T07:13:49.295969+0000","last_became_active":"2023-10-08T07:13:34.298139+0000","last_became_peered":"2023-10-08T07:13:34.298139+0000","last_unstale":"2023-10-08T07:13:49.295969+0000","last_undegraded":"2023-10-08T07:13:49.295969+0000","last_fullsized":"2023-10-08T07:13:49.295969+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.728918+0000","last_change":"2023-10-08T07:13:49.304934+0000","last_active":"2023-10-08T07:13:50.728918+0000","last_peered":"2023-10-08T07:13:50.728918+0000","last_clean":"2023-10-08T07:13:50.728918+0000","last_became_active":"2023-10-08T07:13:49.304630+0000","last_became_peered":"2023-10-08T07:13:49.304630+0000","last_unstale":"2023-10-08T07:13:50.728918+0000","last_undegraded":"2023-10-08T07:13:50.728918+0000","last_fullsized":"2023-10-08T07:13:50.728918+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:30.460750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.297629+0000","last_change":"2023-10-08T07:13:34.306040+0000","last_active":"2023-10-08T07:13:49.297629+0000","last_peered":"2023-10-08T07:13:49.297629+0000","last_clean":"2023-10-08T07:13:49.297629+0000","last_became_active":"2023-10-08T07:13:34.303015+0000","last_became_peered":"2023-10-08T07:13:34.303015+0000","last_unstale":"2023-10-08T07:13:49.297629+0000","last_undegraded":"2023-10-08T07:13:49.297629+0000","last_fullsized":"2023-10-08T07:13:49.297629+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:42.884365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358958+0000","last_change":"2023-10-08T07:13:49.310325+0000","last_active":"2023-10-08T07:13:50.358958+0000","last_peered":"2023-10-08T07:13:50.358958+0000","last_clean":"2023-10-08T07:13:50.358958+0000","last_became_active":"2023-10-08T07:13:49.310037+0000","last_became_peered":"2023-10-08T07:13:49.310037+0000","last_unstale":"2023-10-08T07:13:50.358958+0000","last_undegraded":"2023-10-08T07:13:50.358958+0000","last_fullsized":"2023-10-08T07:13:50.358958+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:29.904246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.281819+0000","last_change":"2023-10-08T07:13:34.294253+0000","last_active":"2023-10-08T07:13:49.281819+0000","last_peered":"2023-10-08T07:13:49.281819+0000","last_clean":"2023-10-08T07:13:49.281819+0000","last_became_active":"2023-10-08T07:13:34.291997+0000","last_became_peered":"2023-10-08T07:13:34.291997+0000","last_unstale":"2023-10-08T07:13:49.281819+0000","last_undegraded":"2023-10-08T07:13:49.281819+0000","last_fullsized":"2023-10-08T07:13:49.281819+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278736+0000","last_change":"2023-10-08T07:13:49.298924+0000","last_active":"2023-10-08T07:13:50.278736+0000","last_peered":"2023-10-08T07:13:50.278736+0000","last_clean":"2023-10-08T07:13:50.278736+0000","last_became_active":"2023-10-08T07:13:49.298587+0000","last_became_peered":"2023-10-08T07:13:49.298587+0000","last_unstale":"2023-10-08T07:13:50.278736+0000","last_undegraded":"2023-10-08T07:13:50.278736+0000","last_fullsized":"2023-10-08T07:13:50.278736+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:34.367483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287754+0000","last_change":"2023-10-08T07:13:34.308930+0000","last_active":"2023-10-08T07:13:49.287754+0000","last_peered":"2023-10-08T07:13:49.287754+0000","last_clean":"2023-10-08T07:13:49.287754+0000","last_became_active":"2023-10-08T07:13:34.308283+0000","last_became_peered":"2023-10-08T07:13:34.308283+0000","last_unstale":"2023-10-08T07:13:49.287754+0000","last_undegraded":"2023-10-08T07:13:49.287754+0000","last_fullsized":"2023-10-08T07:13:49.287754+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283832+0000","last_change":"2023-10-08T07:13:49.318881+0000","last_active":"2023-10-08T07:13:50.283832+0000","last_peered":"2023-10-08T07:13:50.283832+0000","last_clean":"2023-10-08T07:13:50.283832+0000","last_became_active":"2023-10-08T07:13:49.318589+0000","last_became_peered":"2023-10-08T07:13:49.318589+0000","last_unstale":"2023-10-08T07:13:50.283832+0000","last_undegraded":"2023-10-08T07:13:50.283832+0000","last_fullsized":"2023-10-08T07:13:50.283832+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:23:54.276903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.314908+0000","last_change":"2023-10-08T07:13:34.295991+0000","last_active":"2023-10-08T07:13:49.314908+0000","last_peered":"2023-10-08T07:13:49.314908+0000","last_clean":"2023-10-08T07:13:49.314908+0000","last_became_active":"2023-10-08T07:13:34.295717+0000","last_became_peered":"2023-10-08T07:13:34.295717+0000","last_unstale":"2023-10-08T07:13:49.314908+0000","last_undegraded":"2023-10-08T07:13:49.314908+0000","last_fullsized":"2023-10-08T07:13:49.314908+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.797828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:51.085925+0000","last_change":"2023-10-08T07:13:49.306886+0000","last_active":"2023-10-08T07:13:51.085925+0000","last_peered":"2023-10-08T07:13:51.085925+0000","last_clean":"2023-10-08T07:13:51.085925+0000","last_became_active":"2023-10-08T07:13:49.306317+0000","last_became_peered":"2023-10-08T07:13:49.306317+0000","last_unstale":"2023-10-08T07:13:51.085925+0000","last_undegraded":"2023-10-08T07:13:51.085925+0000","last_fullsized":"2023-10-08T07:13:51.085925+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:52.597232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291913+0000","last_change":"2023-10-08T07:13:34.289375+0000","last_active":"2023-10-08T07:13:49.291913+0000","last_peered":"2023-10-08T07:13:49.291913+0000","last_clean":"2023-10-08T07:13:49.291913+0000","last_became_active":"2023-10-08T07:13:34.289115+0000","last_became_peered":"2023-10-08T07:13:34.289115+0000","last_unstale":"2023-10-08T07:13:49.291913+0000","last_undegraded":"2023-10-08T07:13:49.291913+0000","last_fullsized":"2023-10-08T07:13:49.291913+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.286470+0000","last_change":"2023-10-08T07:13:49.318551+0000","last_active":"2023-10-08T07:13:50.286470+0000","last_peered":"2023-10-08T07:13:50.286470+0000","last_clean":"2023-10-08T07:13:50.286470+0000","last_became_active":"2023-10-08T07:13:49.317986+0000","last_became_peered":"2023-10-08T07:13:49.317986+0000","last_unstale":"2023-10-08T07:13:50.286470+0000","last_undegraded":"2023-10-08T07:13:50.286470+0000","last_fullsized":"2023-10-08T07:13:50.286470+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:18.097796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285210+0000","last_change":"2023-10-08T07:13:34.302018+0000","last_active":"2023-10-08T07:13:49.285210+0000","last_peered":"2023-10-08T07:13:49.285210+0000","last_clean":"2023-10-08T07:13:49.285210+0000","last_became_active":"2023-10-08T07:13:34.301747+0000","last_became_peered":"2023-10-08T07:13:34.301747+0000","last_unstale":"2023-10-08T07:13:49.285210+0000","last_undegraded":"2023-10-08T07:13:49.285210+0000","last_fullsized":"2023-10-08T07:13:49.285210+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.844613+0000","last_change":"2023-10-08T07:13:49.311917+0000","last_active":"2023-10-08T07:13:50.844613+0000","last_peered":"2023-10-08T07:13:50.844613+0000","last_clean":"2023-10-08T07:13:50.844613+0000","last_became_active":"2023-10-08T07:13:49.310901+0000","last_became_peered":"2023-10-08T07:13:49.310901+0000","last_unstale":"2023-10-08T07:13:50.844613+0000","last_undegraded":"2023-10-08T07:13:50.844613+0000","last_fullsized":"2023-10-08T07:13:50.844613+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:08:31.424602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290421+0000","last_change":"2023-10-08T07:13:34.293550+0000","last_active":"2023-10-08T07:13:49.290421+0000","last_peered":"2023-10-08T07:13:49.290421+0000","last_clean":"2023-10-08T07:13:49.290421+0000","last_became_active":"2023-10-08T07:13:34.293202+0000","last_became_peered":"2023-10-08T07:13:34.293202+0000","last_unstale":"2023-10-08T07:13:49.290421+0000","last_undegraded":"2023-10-08T07:13:49.290421+0000","last_fullsized":"2023-10-08T07:13:49.290421+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:00.842346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.281807+0000","last_change":"2023-10-08T07:13:49.318338+0000","last_active":"2023-10-08T07:13:50.281807+0000","last_peered":"2023-10-08T07:13:50.281807+0000","last_clean":"2023-10-08T07:13:50.281807+0000","last_became_active":"2023-10-08T07:13:49.317979+0000","last_became_peered":"2023-10-08T07:13:49.317979+0000","last_unstale":"2023-10-08T07:13:50.281807+0000","last_undegraded":"2023-10-08T07:13:50.281807+0000","last_fullsized":"2023-10-08T07:13:50.281807+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:39:03.798412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.293944+0000","last_change":"2023-10-08T07:13:34.298110+0000","last_active":"2023-10-08T07:13:49.293944+0000","last_peered":"2023-10-08T07:13:49.293944+0000","last_clean":"2023-10-08T07:13:49.293944+0000","last_became_active":"2023-10-08T07:13:34.297806+0000","last_became_peered":"2023-10-08T07:13:34.297806+0000","last_unstale":"2023-10-08T07:13:49.293944+0000","last_undegraded":"2023-10-08T07:13:49.293944+0000","last_fullsized":"2023-10-08T07:13:49.293944+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:58.335295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357781+0000","last_change":"2023-10-08T07:13:49.315887+0000","last_active":"2023-10-08T07:13:50.357781+0000","last_peered":"2023-10-08T07:13:50.357781+0000","last_clean":"2023-10-08T07:13:50.357781+0000","last_became_active":"2023-10-08T07:13:49.315571+0000","last_became_peered":"2023-10-08T07:13:49.315571+0000","last_unstale":"2023-10-08T07:13:50.357781+0000","last_undegraded":"2023-10-08T07:13:50.357781+0000","last_fullsized":"2023-10-08T07:13:50.357781+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:31.253205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282695+0000","last_change":"2023-10-08T07:13:34.298025+0000","last_active":"2023-10-08T07:13:49.282695+0000","last_peered":"2023-10-08T07:13:49.282695+0000","last_clean":"2023-10-08T07:13:49.282695+0000","last_became_active":"2023-10-08T07:13:34.296181+0000","last_became_peered":"2023-10-08T07:13:34.296181+0000","last_unstale":"2023-10-08T07:13:49.282695+0000","last_undegraded":"2023-10-08T07:13:49.282695+0000","last_fullsized":"2023-10-08T07:13:49.282695+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448978+0000","last_change":"2023-10-08T07:13:49.300145+0000","last_active":"2023-10-08T07:13:50.448978+0000","last_peered":"2023-10-08T07:13:50.448978+0000","last_clean":"2023-10-08T07:13:50.448978+0000","last_became_active":"2023-10-08T07:13:49.299846+0000","last_became_peered":"2023-10-08T07:13:49.299846+0000","last_unstale":"2023-10-08T07:13:50.448978+0000","last_undegraded":"2023-10-08T07:13:50.448978+0000","last_fullsized":"2023-10-08T07:13:50.448978+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:16.187252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296189+0000","last_change":"2023-10-08T07:13:34.301326+0000","last_active":"2023-10-08T07:13:49.296189+0000","last_peered":"2023-10-08T07:13:49.296189+0000","last_clean":"2023-10-08T07:13:49.296189+0000","last_became_active":"2023-10-08T07:13:34.299365+0000","last_became_peered":"2023-10-08T07:13:34.299365+0000","last_unstale":"2023-10-08T07:13:49.296189+0000","last_undegraded":"2023-10-08T07:13:49.296189+0000","last_fullsized":"2023-10-08T07:13:49.296189+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:02.576748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.846218+0000","last_change":"2023-10-08T07:13:49.313275+0000","last_active":"2023-10-08T07:13:50.846218+0000","last_peered":"2023-10-08T07:13:50.846218+0000","last_clean":"2023-10-08T07:13:50.846218+0000","last_became_active":"2023-10-08T07:13:49.312535+0000","last_became_peered":"2023-10-08T07:13:49.312535+0000","last_unstale":"2023-10-08T07:13:50.846218+0000","last_undegraded":"2023-10-08T07:13:50.846218+0000","last_fullsized":"2023-10-08T07:13:50.846218+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:49.544492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287567+0000","last_change":"2023-10-08T07:13:34.306582+0000","last_active":"2023-10-08T07:13:49.287567+0000","last_peered":"2023-10-08T07:13:49.287567+0000","last_clean":"2023-10-08T07:13:49.287567+0000","last_became_active":"2023-10-08T07:13:34.306310+0000","last_became_peered":"2023-10-08T07:13:34.306310+0000","last_unstale":"2023-10-08T07:13:49.287567+0000","last_undegraded":"2023-10-08T07:13:49.287567+0000","last_fullsized":"2023-10-08T07:13:49.287567+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:51.085566+0000","last_change":"2023-10-08T07:13:49.307756+0000","last_active":"2023-10-08T07:13:51.085566+0000","last_peered":"2023-10-08T07:13:51.085566+0000","last_clean":"2023-10-08T07:13:51.085566+0000","last_became_active":"2023-10-08T07:13:49.307456+0000","last_became_peered":"2023-10-08T07:13:49.307456+0000","last_unstale":"2023-10-08T07:13:51.085566+0000","last_undegraded":"2023-10-08T07:13:51.085566+0000","last_fullsized":"2023-10-08T07:13:51.085566+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:33.124866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298001+0000","last_change":"2023-10-08T07:13:34.295062+0000","last_active":"2023-10-08T07:13:49.298001+0000","last_peered":"2023-10-08T07:13:49.298001+0000","last_clean":"2023-10-08T07:13:49.298001+0000","last_became_active":"2023-10-08T07:13:34.294747+0000","last_became_peered":"2023-10-08T07:13:34.294747+0000","last_unstale":"2023-10-08T07:13:49.298001+0000","last_undegraded":"2023-10-08T07:13:49.298001+0000","last_fullsized":"2023-10-08T07:13:49.298001+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:26.407053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358527+0000","last_change":"2023-10-08T07:13:49.304648+0000","last_active":"2023-10-08T07:13:50.358527+0000","last_peered":"2023-10-08T07:13:50.358527+0000","last_clean":"2023-10-08T07:13:50.358527+0000","last_became_active":"2023-10-08T07:13:49.304247+0000","last_became_peered":"2023-10-08T07:13:49.304247+0000","last_unstale":"2023-10-08T07:13:50.358527+0000","last_undegraded":"2023-10-08T07:13:50.358527+0000","last_fullsized":"2023-10-08T07:13:50.358527+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:42.456757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.297988+0000","last_change":"2023-10-08T07:13:34.308897+0000","last_active":"2023-10-08T07:13:49.297988+0000","last_peered":"2023-10-08T07:13:49.297988+0000","last_clean":"2023-10-08T07:13:49.297988+0000","last_became_active":"2023-10-08T07:13:34.308474+0000","last_became_peered":"2023-10-08T07:13:34.308474+0000","last_unstale":"2023-10-08T07:13:49.297988+0000","last_undegraded":"2023-10-08T07:13:49.297988+0000","last_fullsized":"2023-10-08T07:13:49.297988+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:06.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2806,"ondisk_log_size":2806,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576012,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65236,"kb_used_data":3520,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306604,"statfs":{"total":96636764160,"available":96569962496,"internally_reserved":0,"allocated":3604480,"data_stored":2148688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.169,"15min":1.2090000000000001},"min":{"1min":0.77100000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.508,"5min":1.968,"15min":2.008},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.238,"15min":1.2310000000000001},"min":{"1min":0.91300000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.6830000000000001,"5min":1.7330000000000001,"15min":1.7450000000000001},"last":0.98099999999999998}]},{"osd":1,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.242,"15min":1.252},"min":{"1min":0.94399999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.665,"5min":1.79,"15min":1.998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.341,"15min":1.3500000000000001},"min":{"1min":0.99399999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.8859999999999999,"5min":1.8899999999999999,"15min":2.1019999999999999},"last":1.331}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.29,"15min":1.3140000000000001},"min":{"1min":0.97199999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6180000000000001,"5min":1.696,"15min":2.1640000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.351},"min":{"1min":1.0069999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7270000000000001,"5min":1.8580000000000001,"15min":2.3170000000000002},"last":1.401}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":1.0209999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.663,"5min":1.8160000000000001,"15min":1.9039999999999999},"last":1.47},{"interface":"front","average":{"1min":1.403,"5min":1.399,"15min":1.4119999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.754,"5min":1.8680000000000001,"15min":1.931},"last":1.6539999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":0.86199999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6950000000000001,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.702},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.383,"15min":1.401},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.85199999999999998},"max":{"1min":1.76,"5min":1.8140000000000001,"15min":2.0379999999999998},"last":1.7250000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4390000000000001,"15min":1.444},"min":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":2.0920000000000001,"15min":2.1139999999999999},"last":1.7629999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.4610000000000001,"15min":1.51},"min":{"1min":1.075,"5min":1,"15min":0.96899999999999997},"max":{"1min":1.8640000000000001,"5min":2.23,"15min":2.23},"last":1.546}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.575,"15min":1.6020000000000001},"min":{"1min":1.361,"5min":1.048,"15min":1.048},"max":{"1min":2.016,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.536,"15min":1.5349999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9079999999999999,"5min":2.21,"15min":2.2570000000000001},"last":1.607}]}]},{"osd":6,"up_from":13,"seq":55834576011,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46624,"kb_used_data":2956,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325216,"statfs":{"total":96636764160,"available":96589021184,"internally_reserved":0,"allocated":3026944,"data_stored":1559898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3109999999999999,"15min":1.321},"min":{"1min":0.80000000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":1.794,"15min":1.962},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.347,"15min":1.3620000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7230000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.5880000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.228,"15min":1.2290000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.2130000000000001,"15min":1.2430000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.97},"last":1.1160000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.198,"15min":1.238},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.484,"5min":2.339,"15min":2.339},"last":1.464},{"interface":"front","average":{"1min":1.302,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":1.0469999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3819999999999999,"15min":1.403},"min":{"1min":1.034,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":1.9319999999999999,"15min":1.95},"last":1.363},{"interface":"front","average":{"1min":1.425,"5min":1.476,"15min":1.4990000000000001},"min":{"1min":0.97199999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8740000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.681}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3999999999999999,"15min":1.3899999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.881,"5min":2.0579999999999998,"15min":2.0859999999999999},"last":1.649},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4770000000000001,"15min":1.448},"min":{"1min":0.94999999999999996,"5min":0.92000000000000004,"15min":0.57099999999999995},"max":{"1min":1.8080000000000001,"5min":2.484,"15min":2.484},"last":1.738}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.399,"15min":1.423},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.81899999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.403},"last":1.194},{"interface":"front","average":{"1min":1.552,"5min":1.538,"15min":1.5509999999999999},"min":{"1min":1.2829999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.8100000000000001,"5min":2.097,"15min":2.6139999999999999},"last":1.532}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.575},"min":{"1min":1.0489999999999999,"5min":1.012,"15min":0.96299999999999997},"max":{"1min":1.9379999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.7989999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.5620000000000001,"15min":1.5600000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.665},"last":1.417}]}]},{"osd":5,"up_from":13,"seq":55834576011,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60488,"kb_used_data":2940,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311352,"statfs":{"total":96636764160,"available":96574824448,"internally_reserved":0,"allocated":3010560,"data_stored":1556459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.157,"15min":1.208},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.841},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.2290000000000001,"15min":1.232},"min":{"1min":0.83099999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7030000000000001,"5min":1.8560000000000001,"15min":1.9610000000000001},"last":0.96599999999999997}]},{"osd":1,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2090000000000001,"15min":1.242},"min":{"1min":0.75800000000000001,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.6990000000000001,"5min":1.762,"15min":1.9550000000000001},"last":1.54},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2869999999999999,"15min":1.3},"min":{"1min":0.90900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.6879999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.871,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":2.0870000000000002},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.3149999999999999},"min":{"1min":0.77000000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5840000000000001,"5min":1.877,"15min":1.909},"last":1.1419999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":0.79800000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.1499999999999999},"last":1.214},{"interface":"front","average":{"1min":1.458,"5min":1.3740000000000001,"15min":1.431},"min":{"1min":0.67500000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8100000000000001,"5min":1.9530000000000001,"15min":2.173},"last":1.387}]},{"osd":4,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3220000000000001,"15min":1.3280000000000001},"min":{"1min":0.84399999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7569999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.446,"15min":1.4339999999999999},"min":{"1min":1.107,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.2240000000000002},"last":1.845}]},{"osd":6,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":1.0740000000000001,"5min":0.66500000000000004,"15min":0.57499999999999996},"max":{"1min":1.899,"5min":2.077,"15min":2.3759999999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.478,"15min":1.4770000000000001},"min":{"1min":1.157,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0059999999999998,"5min":2.036,"15min":2.036},"last":1.9690000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.5289999999999999,"15min":1.552},"min":{"1min":1.177,"5min":0.72499999999999998,"15min":0.59999999999999998},"max":{"1min":2.298,"5min":2.298,"15min":2.4169999999999998},"last":2.0339999999999998},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.5840000000000001,"15min":1.6180000000000001},"min":{"1min":1.0089999999999999,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0009999999999999,"5min":2.2320000000000002,"15min":2.3199999999999998},"last":1.591}]}]},{"osd":4,"up_from":13,"seq":55834576011,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47992,"kb_used_data":2924,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323848,"statfs":{"total":96636764160,"available":96587620352,"internally_reserved":0,"allocated":2994176,"data_stored":1555393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2849999999999999,"15min":1.3440000000000001},"min":{"1min":0.64200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.492},{"interface":"front","average":{"1min":1.367,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":0.62,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.9139999999999999,"5min":1.976,"15min":1.976},"last":1.6950000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2629999999999999,"15min":1.2889999999999999},"min":{"1min":0.85599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7450000000000001,"5min":1.976,"15min":1.976},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.244},"min":{"1min":0.72499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7330000000000001,"5min":1.8660000000000001,"15min":1.875},"last":1.0800000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2689999999999999,"15min":1.266},"min":{"1min":0.77400000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.145},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.365,"15min":1.369},"min":{"1min":0.99299999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.829}]},{"osd":3,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2789999999999999,"15min":1.3149999999999999},"min":{"1min":0.88,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.702,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.4410000000000001,"15min":1.4219999999999999},"min":{"1min":0.874,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.2849999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.391},"min":{"1min":1.032,"5min":0.372,"15min":0.372},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4239999999999999,"15min":1.3919999999999999},"min":{"1min":0.96799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.865,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.262}]},{"osd":6,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.5009999999999999,"15min":1.46},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2530000000000001,"5min":2.2719999999999998,"15min":2.6829999999999998},"last":1.776},{"interface":"front","average":{"1min":1.635,"5min":1.593,"15min":1.5609999999999999},"min":{"1min":1.071,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.117,"5min":2.3500000000000001,"15min":2.7429999999999999},"last":1.9339999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.5369999999999999,"15min":1.516},"min":{"1min":1.1220000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0649999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.681,"15min":1.6439999999999999},"min":{"1min":1.3169999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.3439999999999999,"5min":2.508,"15min":2.508},"last":1.9730000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576013,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52168,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319672,"statfs":{"total":96636764160,"available":96583344128,"internally_reserved":0,"allocated":3002368,"data_stored":1554869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.67800000000000005,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.087,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.123,"5min":1.1559999999999999,"15min":1.181},"min":{"1min":0.748,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.607,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.70099999999999996}]},{"osd":1,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.417,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.2390000000000001,"15min":1.2869999999999999},"min":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4390000000000001,"5min":2.109,"15min":2.4889999999999999},"last":0.74299999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.123,"15min":1.151},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.667,"5min":2.165,"15min":2.2250000000000001},"last":0.62},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.177,"15min":1.202},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5349999999999999,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2549999999999999,"15min":1.282},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.67500000000000004},"max":{"1min":1.51,"5min":1.857,"15min":2.5379999999999998},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4179999999999999,"15min":1.46},"min":{"1min":0.93600000000000005,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.954,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":0.98599999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3200000000000001,"15min":1.3759999999999999},"min":{"1min":0.97899999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8300000000000001,"5min":1.923,"15min":2.9740000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.419,"15min":1.4339999999999999},"min":{"1min":0.77800000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8380000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4099999999999999,"15min":1.4239999999999999},"min":{"1min":0.91000000000000003,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6339999999999999,"5min":1.976,"15min":2.653},"last":1.02},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4710000000000001,"15min":1.5269999999999999},"min":{"1min":1.0960000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8540000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.0529999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5660000000000001},"min":{"1min":0.88800000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9450000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.5209999999999999},"min":{"1min":1.202,"5min":0.628,"15min":0.628},"max":{"1min":1.7889999999999999,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.2110000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576011,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":2998272,"data_stored":1555427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.97299999999999998},"min":{"1min":0.55300000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.948},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.054,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":0.745}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.96099999999999997,"15min":1.0029999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.198,"5min":1.6279999999999999,"15min":2.044},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.171,"15min":1.1970000000000001},"min":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.677,"5min":1.714,"15min":2.1309999999999998},"last":0.86799999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68400000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4770000000000001,"5min":1.78,"15min":1.8420000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.2410000000000001,"15min":1.2769999999999999},"min":{"1min":0.66200000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.458,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.165}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3959999999999999,"15min":1.4399999999999999},"min":{"1min":0.89000000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.821,"5min":1.929,"15min":2.1789999999999998},"last":1.754},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":1.0189999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6879999999999999,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.577}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3819999999999999,"15min":1.411},"min":{"1min":1.0069999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.603,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":1.083},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5009999999999999,"15min":1.5349999999999999},"min":{"1min":1.0269999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.946,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.5489999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":0.997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.73,"5min":2.056,"15min":2.1240000000000001},"last":1.472},{"interface":"front","average":{"1min":1.47,"5min":1.4950000000000001,"15min":1.534},"min":{"1min":1.1439999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.778,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":1.7190000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4690000000000001,"15min":1.486},"min":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7609999999999999,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":1.643},{"interface":"front","average":{"1min":1.5,"5min":1.5309999999999999,"15min":1.5860000000000001},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.863,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":1.623}]}]},{"osd":1,"up_from":13,"seq":55834576011,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52152,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319688,"statfs":{"total":96636764160,"available":96583360512,"internally_reserved":0,"allocated":2985984,"data_stored":1554721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.024,"15min":1.014},"min":{"1min":0.81000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.6200000000000001,"5min":1.6579999999999999,"15min":1.7769999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.224},"min":{"1min":0.88,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":1.79},"last":1.1639999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.97599999999999998},"min":{"1min":0.73399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2809999999999999,"5min":1.641,"15min":1.641},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.155,"15min":1.161},"min":{"1min":0.872,"5min":0.372,"15min":0.372},"max":{"1min":1.4710000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.85399999999999998}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.2430000000000001,"15min":1.264},"min":{"1min":1.052,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5800000000000001,"5min":1.651,"15min":1.7809999999999999},"last":1.419},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1659999999999999,"15min":1.155},"min":{"1min":0.82599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.72,"5min":1.883,"15min":2.7429999999999999},"last":1.1120000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":1.117,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.7110000000000001,"5min":1.7829999999999999,"15min":2.8460000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.3129999999999999,"15min":1.3220000000000001},"min":{"1min":1.0960000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.5449999999999999,"5min":1.6879999999999999,"15min":2.0139999999999998},"last":1.0449999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.347,"15min":1.379},"min":{"1min":0.97299999999999998,"5min":0.92600000000000005,"15min":0.76700000000000002},"max":{"1min":1.6639999999999999,"5min":1.827,"15min":2.0470000000000002},"last":1.298},{"interface":"front","average":{"1min":1.637,"5min":1.546,"15min":1.534},"min":{"1min":1.286,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.2610000000000001},"last":1.522}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5089999999999999,"15min":1.5149999999999999},"min":{"1min":1.2549999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.694,"5min":2.3199999999999998,"15min":2.9079999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.4870000000000001,"15min":1.4990000000000001},"min":{"1min":1.1679999999999999,"5min":1.046,"15min":1.0309999999999999},"max":{"1min":1.7929999999999999,"5min":1.966,"15min":2.3199999999999998},"last":1.347}]},{"osd":7,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.5780000000000001,"15min":1.556},"min":{"1min":1.2250000000000001,"5min":0.94499999999999995,"15min":0.84599999999999997},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.478},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.5529999999999999,"15min":1.5720000000000001},"min":{"1min":1.377,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.798,"5min":2.1699999999999999,"15min":2.9590000000000001},"last":1.333}]}]},{"osd":0,"up_from":13,"seq":55834576012,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61048,"kb_used_data":3500,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310792,"statfs":{"total":96636764160,"available":96574251008,"internally_reserved":0,"allocated":3584000,"data_stored":2145179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.48299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3859999999999999,"5min":1.581,"15min":1.581},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.01,"5min":1.103,"15min":1.109},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.621,"5min":1.6619999999999999,"15min":1.726},"last":0.874}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1060000000000001,"15min":1.1419999999999999},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0329999999999999,"15min":1.073},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4199999999999999,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":0.93300000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.248,"15min":1.256},"min":{"1min":0.69799999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.546,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.579,"5min":1.752,"15min":2.0979999999999999},"last":1.4259999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3220000000000001,"15min":1.349},"min":{"1min":0.83399999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.389,"15min":1.4179999999999999},"min":{"1min":0.69699999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.782,"5min":1.972,"15min":2.181},"last":2.1520000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.423,"15min":1.417},"min":{"1min":0.79600000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.732,"5min":1.919,"15min":2.1309999999999998},"last":1.706},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.482,"15min":1.486},"min":{"1min":0.79300000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.966,"5min":2.2170000000000001,"15min":2.4900000000000002},"last":1.9930000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.81299999999999994,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7669999999999999,"5min":2.21,"15min":2.2149999999999999},"last":1.925},{"interface":"front","average":{"1min":1.397,"5min":1.506,"15min":1.5089999999999999},"min":{"1min":0.78000000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9430000000000001,"5min":2.0880000000000001,"15min":2.355},"last":1.7529999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5389999999999999,"15min":1.5469999999999999},"min":{"1min":0.83499999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9410000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.5289999999999999,"15min":1.554},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":2.0750000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:13:58.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:13:59.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.197+0000 7f9659b4c640 1 Processor -- start 2023-10-08T07:13:59.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.197+0000 7f9659b4c640 1 -- start start 2023-10-08T07:13:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.198+0000 7f9659b4c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96540c90b0 0x7f96540cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f9659b4c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540cb3b0 0x7f965415e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f9659b4c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f965415b2b0 con 0x7f965415ab40 2023-10-08T07:13:59.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f9659b4c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f965415bf30 con 0x7f96540c90b0 2023-10-08T07:13:59.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f9659b4c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96540cb820 con 0x7f96540cb3b0 2023-10-08T07:13:59.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f96537fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f965415ab40 0x7f96540ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47752/0 (socket says 172.21.15.70:47752) 2023-10-08T07:13:59.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f96537fe640 1 -- 172.21.15.70:0/3588215797 learned_addr learned my addr 172.21.15.70:0/3588215797 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:13:59.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f964affd640 1 --2- 172.21.15.70:0/3588215797 >> v2:172.21.15.70:3300/0 conn(0x7f96540c90b0 0x7f96540cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f9652ffd640 1 -- 172.21.15.70:0/3588215797 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273606718 0 0) 0x7f965415b2b0 con 0x7f965415ab40 2023-10-08T07:13:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f9652ffd640 1 -- 172.21.15.70:0/3588215797 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9640003520 con 0x7f965415ab40 2023-10-08T07:13:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f9653fff640 1 --2- 172.21.15.70:0/3588215797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540cb3b0 0x7f965415e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.199+0000 7f964affd640 1 -- 172.21.15.70:0/3588215797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540cb3b0 msgr2=0x7f965415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.200+0000 7f964affd640 1 --2- 172.21.15.70:0/3588215797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540cb3b0 0x7f965415e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.200+0000 7f964affd640 1 -- 172.21.15.70:0/3588215797 >> v1:172.21.15.70:6789/0 conn(0x7f965415ab40 legacy=0x7f96540ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.200+0000 7f964affd640 1 -- 172.21.15.70:0/3588215797 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965415b590 con 0x7f96540c90b0 2023-10-08T07:13:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.200+0000 7f9653fff640 1 --2- 172.21.15.70:0/3588215797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540cb3b0 0x7f965415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:59.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.201+0000 7f964affd640 1 --2- 172.21.15.70:0/3588215797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 0x7f96540cae70 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f963c0097d0 tx=0x7f963c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872ac1677356349c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:59.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.201+0000 7f9652ffd640 1 -- 172.21.15.70:0/3588215797 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c031800 con 0x7f96540c90b0 2023-10-08T07:13:59.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.201+0000 7f9652ffd640 1 -- 172.21.15.70:0/3588215797 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c031960 con 0x7f96540c90b0 2023-10-08T07:13:59.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.201+0000 7f9652ffd640 1 -- 172.21.15.70:0/3588215797 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c031cf0 con 0x7f96540c90b0 2023-10-08T07:13:59.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.202+0000 7f9659b4c640 1 -- 172.21.15.70:0/3588215797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 msgr2=0x7f96540cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.202+0000 7f9659b4c640 1 --2- 172.21.15.70:0/3588215797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 0x7f96540cae70 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f963c0097d0 tx=0x7f963c02b8a0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.202+0000 7f9659b4c640 1 -- 172.21.15.70:0/3588215797 shutdown_connections 2023-10-08T07:13:59.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.202+0000 7f9659b4c640 1 --2- 172.21.15.70:0/3588215797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f96540cb3b0 0x7f965415e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.202+0000 7f9659b4c640 1 --2- 172.21.15.70:0/3588215797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 0x7f96540cae70 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.202+0000 7f9659b4c640 1 -- 172.21.15.70:0/3588215797 >> 172.21.15.70:0/3588215797 conn(0x7f965406a380 msgr2=0x7f96540b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:59.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.202+0000 7f9659b4c640 1 -- 172.21.15.70:0/3588215797 shutdown_connections 2023-10-08T07:13:59.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.202+0000 7f9659b4c640 1 -- 172.21.15.70:0/3588215797 wait complete. 2023-10-08T07:13:59.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.204+0000 7f9659b4c640 1 Processor -- start 2023-10-08T07:13:59.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.204+0000 7f9659b4c640 1 -- start start 2023-10-08T07:13:59.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f9659b4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 0x7f965414d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f9659b4c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96540cb3b0 0x7f96540c3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f9659b4c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f965415ab40 0x7f96540ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f9659b4c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965414d7d0 con 0x7f96540c90b0 2023-10-08T07:13:59.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f9659b4c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96540c4070 con 0x7f96540cb3b0 2023-10-08T07:13:59.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f9659b4c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96540c4350 con 0x7f965415ab40 2023-10-08T07:13:59.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f96537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 0x7f965414d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:59.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f96537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 0x7f965414d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49978/0 (socket says 172.21.15.70:49978) 2023-10-08T07:13:59.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f96537fe640 1 -- 172.21.15.70:0/136178626 learned_addr learned my addr 172.21.15.70:0/136178626 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:59.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f96537fe640 1 -- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f965415ab40 msgr2=0x7f96540ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f96537fe640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f965415ab40 0x7f96540ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f96537fe640 1 -- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96540cb3b0 msgr2=0x7f96540c3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f96537fe640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96540cb3b0 0x7f96540c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.205+0000 7f96537fe640 1 -- 172.21.15.70:0/136178626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c009400 con 0x7f96540c90b0 2023-10-08T07:13:59.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.206+0000 7f96537fe640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 0x7f965414d290 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f9644001eb0 tx=0x7f964400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.206+0000 7f9650ff9640 1 -- 172.21.15.70:0/136178626 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644005ce0 con 0x7f96540c90b0 2023-10-08T07:13:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.206+0000 7f9650ff9640 1 -- 172.21.15.70:0/136178626 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9644005e40 con 0x7f96540c90b0 2023-10-08T07:13:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.206+0000 7f9650ff9640 1 -- 172.21.15.70:0/136178626 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644007270 con 0x7f96540c90b0 2023-10-08T07:13:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.206+0000 7f9659b4c640 1 -- 172.21.15.70:0/136178626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96540c4630 con 0x7f96540c90b0 2023-10-08T07:13:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.206+0000 7f9659b4c640 1 -- 172.21.15.70:0/136178626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96540c6cb0 con 0x7f96540c90b0 2023-10-08T07:13:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.207+0000 7f9659b4c640 1 -- 172.21.15.70:0/136178626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96540c6ea0 con 0x7f96540c90b0 2023-10-08T07:13:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.208+0000 7f9650ff9640 1 -- 172.21.15.70:0/136178626 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f9644007490 con 0x7f96540c90b0 2023-10-08T07:13:59.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.209+0000 7f9650ff9640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f963406af90 0x7f963406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.209+0000 7f9650ff9640 1 -- 172.21.15.70:0/136178626 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f9644089220 con 0x7f96540c90b0 2023-10-08T07:13:59.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.210+0000 7f964affd640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f963406af90 0x7f963406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:59.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.211+0000 7f964affd640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f963406af90 0x7f963406d450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f963c036000 tx=0x7f963c030000 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:59.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.213+0000 7f9650ff9640 1 -- 172.21.15.70:0/136178626 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9644057740 con 0x7f96540c90b0 2023-10-08T07:13:59.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.405+0000 7f9659b4c640 1 -- 172.21.15.70:0/136178626 --> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f965415bb50 con 0x7f963406af90 2023-10-08T07:13:59.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.446+0000 7f9650ff9640 1 -- 172.21.15.70:0/136178626 <== mgr.7328 v2:172.21.15.181:6832/3993370366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+525952 (secure 0 0 0) 0x7f963c03a420 con 0x7f963406af90 2023-10-08T07:13:59.450 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:59.458 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:13:59.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 -- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f963406af90 msgr2=0x7f963406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f963406af90 0x7f963406d450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f963c036000 tx=0x7f963c030000 comp rx=0 tx=0).stop 2023-10-08T07:13:59.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 -- 172.21.15.70:0/136178626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 msgr2=0x7f965414d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 0x7f965414d290 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f9644001eb0 tx=0x7f964400c960 comp rx=0 tx=0).stop 2023-10-08T07:13:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 -- 172.21.15.70:0/136178626 shutdown_connections 2023-10-08T07:13:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f965415ab40 0x7f96540ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f96540cb3b0 0x7f96540c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f963406af90 0x7f963406d450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 --2- 172.21.15.70:0/136178626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96540c90b0 0x7f965414d290 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 -- 172.21.15.70:0/136178626 >> 172.21.15.70:0/136178626 conn(0x7f965406a380 msgr2=0x7f96540ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 -- 172.21.15.70:0/136178626 shutdown_connections 2023-10-08T07:13:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.457+0000 7f9659b4c640 1 -- 172.21.15.70:0/136178626 wait complete. 2023-10-08T07:13:59.483 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-10-08T07:13:57.512326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":680,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2885,"ondisk_log_size":2885,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":378,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":437872,"kb_used_data":24616,"kb_used_omap":12,"kb_used_meta":413107,"kb_avail":754536848,"statfs":{"total":773094113280,"available":772645732352,"internally_reserved":0,"allocated":25206784,"data_stored":13630634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":423022139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.235482"},"pg_stats":[{"pgid":"30.1f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359204+0000","last_change":"2023-10-08T07:13:49.313140+0000","last_active":"2023-10-08T07:13:50.359204+0000","last_peered":"2023-10-08T07:13:50.359204+0000","last_clean":"2023-10-08T07:13:50.359204+0000","last_became_active":"2023-10-08T07:13:49.312820+0000","last_became_peered":"2023-10-08T07:13:49.312820+0000","last_unstale":"2023-10-08T07:13:50.359204+0000","last_undegraded":"2023-10-08T07:13:50.359204+0000","last_fullsized":"2023-10-08T07:13:50.359204+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:41.877163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"222'2806","reported_seq":3727,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.303110+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:13:49.303110+0000","last_peered":"2023-10-08T07:13:49.303110+0000","last_clean":"2023-10-08T07:13:49.303110+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:13:49.303110+0000","last_undegraded":"2023-10-08T07:13:49.303110+0000","last_fullsized":"2023-10-08T07:13:49.303110+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2806,"log_dups_size":0,"ondisk_log_size":2806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1a","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289208+0000","last_change":"2023-10-08T07:13:33.718816+0000","last_active":"2023-10-08T07:13:49.289208+0000","last_peered":"2023-10-08T07:13:49.289208+0000","last_clean":"2023-10-08T07:13:49.289208+0000","last_became_active":"2023-10-08T07:13:33.718494+0000","last_became_peered":"2023-10-08T07:13:33.718494+0000","last_unstale":"2023-10-08T07:13:49.289208+0000","last_undegraded":"2023-10-08T07:13:49.289208+0000","last_fullsized":"2023-10-08T07:13:49.289208+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:03.021625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.300079+0000","last_change":"2023-10-08T07:13:34.304364+0000","last_active":"2023-10-08T07:13:49.300079+0000","last_peered":"2023-10-08T07:13:49.300079+0000","last_clean":"2023-10-08T07:13:49.300079+0000","last_became_active":"2023-10-08T07:13:34.304067+0000","last_became_peered":"2023-10-08T07:13:34.304067+0000","last_unstale":"2023-10-08T07:13:49.300079+0000","last_undegraded":"2023-10-08T07:13:49.300079+0000","last_fullsized":"2023-10-08T07:13:49.300079+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:58:56.084275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.1c","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.366839+0000","last_change":"2023-10-08T07:13:48.383423+0000","last_active":"2023-10-08T07:13:49.366839+0000","last_peered":"2023-10-08T07:13:49.366839+0000","last_clean":"2023-10-08T07:13:49.366839+0000","last_became_active":"2023-10-08T07:13:48.383120+0000","last_became_peered":"2023-10-08T07:13:48.383120+0000","last_unstale":"2023-10-08T07:13:49.366839+0000","last_undegraded":"2023-10-08T07:13:49.366839+0000","last_fullsized":"2023-10-08T07:13:49.366839+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:28.023419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.845660+0000","last_change":"2023-10-08T07:13:49.309033+0000","last_active":"2023-10-08T07:13:50.845660+0000","last_peered":"2023-10-08T07:13:50.845660+0000","last_clean":"2023-10-08T07:13:50.845660+0000","last_became_active":"2023-10-08T07:13:49.308757+0000","last_became_peered":"2023-10-08T07:13:49.308757+0000","last_unstale":"2023-10-08T07:13:50.845660+0000","last_undegraded":"2023-10-08T07:13:50.845660+0000","last_fullsized":"2023-10-08T07:13:50.845660+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:16:55.636360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.278218+0000","last_change":"2023-10-08T07:13:33.735924+0000","last_active":"2023-10-08T07:13:49.278218+0000","last_peered":"2023-10-08T07:13:49.278218+0000","last_clean":"2023-10-08T07:13:49.278218+0000","last_became_active":"2023-10-08T07:13:33.735688+0000","last_became_peered":"2023-10-08T07:13:33.735688+0000","last_unstale":"2023-10-08T07:13:49.278218+0000","last_undegraded":"2023-10-08T07:13:49.278218+0000","last_fullsized":"2023-10-08T07:13:49.278218+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286989+0000","last_change":"2023-10-08T07:13:34.308734+0000","last_active":"2023-10-08T07:13:49.286989+0000","last_peered":"2023-10-08T07:13:49.286989+0000","last_clean":"2023-10-08T07:13:49.286989+0000","last_became_active":"2023-10-08T07:13:34.308195+0000","last_became_peered":"2023-10-08T07:13:34.308195+0000","last_unstale":"2023-10-08T07:13:49.286989+0000","last_undegraded":"2023-10-08T07:13:49.286989+0000","last_fullsized":"2023-10-08T07:13:49.286989+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286852+0000","last_change":"2023-10-08T07:13:48.387261+0000","last_active":"2023-10-08T07:13:49.286852+0000","last_peered":"2023-10-08T07:13:49.286852+0000","last_clean":"2023-10-08T07:13:49.286852+0000","last_became_active":"2023-10-08T07:13:48.387022+0000","last_became_peered":"2023-10-08T07:13:48.387022+0000","last_unstale":"2023-10-08T07:13:49.286852+0000","last_undegraded":"2023-10-08T07:13:49.286852+0000","last_fullsized":"2023-10-08T07:13:49.286852+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:09.739690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448409+0000","last_change":"2023-10-08T07:13:49.303091+0000","last_active":"2023-10-08T07:13:50.448409+0000","last_peered":"2023-10-08T07:13:50.448409+0000","last_clean":"2023-10-08T07:13:50.448409+0000","last_became_active":"2023-10-08T07:13:49.302761+0000","last_became_peered":"2023-10-08T07:13:49.302761+0000","last_unstale":"2023-10-08T07:13:50.448409+0000","last_undegraded":"2023-10-08T07:13:50.448409+0000","last_fullsized":"2023-10-08T07:13:50.448409+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:54.844015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294221+0000","last_change":"2023-10-08T07:13:33.741115+0000","last_active":"2023-10-08T07:13:49.294221+0000","last_peered":"2023-10-08T07:13:49.294221+0000","last_clean":"2023-10-08T07:13:49.294221+0000","last_became_active":"2023-10-08T07:13:33.740798+0000","last_became_peered":"2023-10-08T07:13:33.740798+0000","last_unstale":"2023-10-08T07:13:49.294221+0000","last_undegraded":"2023-10-08T07:13:49.294221+0000","last_fullsized":"2023-10-08T07:13:49.294221+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:36.728485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295067+0000","last_change":"2023-10-08T07:13:34.303009+0000","last_active":"2023-10-08T07:13:49.295067+0000","last_peered":"2023-10-08T07:13:49.295067+0000","last_clean":"2023-10-08T07:13:49.295067+0000","last_became_active":"2023-10-08T07:13:34.302682+0000","last_became_peered":"2023-10-08T07:13:34.302682+0000","last_unstale":"2023-10-08T07:13:49.295067+0000","last_undegraded":"2023-10-08T07:13:49.295067+0000","last_fullsized":"2023-10-08T07:13:49.295067+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:47:40.976539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.7","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370132+0000","last_change":"2023-10-08T07:13:48.388490+0000","last_active":"2023-10-08T07:13:49.370132+0000","last_peered":"2023-10-08T07:13:49.370132+0000","last_clean":"2023-10-08T07:13:49.370132+0000","last_became_active":"2023-10-08T07:13:48.388178+0000","last_became_peered":"2023-10-08T07:13:48.388178+0000","last_unstale":"2023-10-08T07:13:49.370132+0000","last_undegraded":"2023-10-08T07:13:49.370132+0000","last_fullsized":"2023-10-08T07:13:49.370132+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:46.350569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278865+0000","last_change":"2023-10-08T07:13:49.305322+0000","last_active":"2023-10-08T07:13:50.278865+0000","last_peered":"2023-10-08T07:13:50.278865+0000","last_clean":"2023-10-08T07:13:50.278865+0000","last_became_active":"2023-10-08T07:13:49.305047+0000","last_became_peered":"2023-10-08T07:13:49.305047+0000","last_unstale":"2023-10-08T07:13:50.278865+0000","last_undegraded":"2023-10-08T07:13:50.278865+0000","last_fullsized":"2023-10-08T07:13:50.278865+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:20.461769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289335+0000","last_change":"2023-10-08T07:13:33.738792+0000","last_active":"2023-10-08T07:13:49.289335+0000","last_peered":"2023-10-08T07:13:49.289335+0000","last_clean":"2023-10-08T07:13:49.289335+0000","last_became_active":"2023-10-08T07:13:33.738430+0000","last_became_peered":"2023-10-08T07:13:33.738430+0000","last_unstale":"2023-10-08T07:13:49.289335+0000","last_undegraded":"2023-10-08T07:13:49.289335+0000","last_fullsized":"2023-10-08T07:13:49.289335+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:24:39.688018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305747+0000","last_change":"2023-10-08T07:13:34.305875+0000","last_active":"2023-10-08T07:13:49.305747+0000","last_peered":"2023-10-08T07:13:49.305747+0000","last_clean":"2023-10-08T07:13:49.305747+0000","last_became_active":"2023-10-08T07:13:34.302605+0000","last_became_peered":"2023-10-08T07:13:34.302605+0000","last_unstale":"2023-10-08T07:13:49.305747+0000","last_undegraded":"2023-10-08T07:13:49.305747+0000","last_fullsized":"2023-10-08T07:13:49.305747+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:22.929719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.304787+0000","last_change":"2023-10-08T07:13:48.391812+0000","last_active":"2023-10-08T07:13:49.304787+0000","last_peered":"2023-10-08T07:13:49.304787+0000","last_clean":"2023-10-08T07:13:49.304787+0000","last_became_active":"2023-10-08T07:13:48.391532+0000","last_became_peered":"2023-10-08T07:13:48.391532+0000","last_unstale":"2023-10-08T07:13:49.304787+0000","last_undegraded":"2023-10-08T07:13:49.304787+0000","last_fullsized":"2023-10-08T07:13:49.304787+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:44.203898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.287618+0000","last_change":"2023-10-08T07:13:49.320305+0000","last_active":"2023-10-08T07:13:50.287618+0000","last_peered":"2023-10-08T07:13:50.287618+0000","last_clean":"2023-10-08T07:13:50.287618+0000","last_became_active":"2023-10-08T07:13:49.319999+0000","last_became_peered":"2023-10-08T07:13:49.319999+0000","last_unstale":"2023-10-08T07:13:50.287618+0000","last_undegraded":"2023-10-08T07:13:50.287618+0000","last_fullsized":"2023-10-08T07:13:50.287618+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:06.650395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.303189+0000","last_change":"2023-10-08T07:13:33.734220+0000","last_active":"2023-10-08T07:13:49.303189+0000","last_peered":"2023-10-08T07:13:49.303189+0000","last_clean":"2023-10-08T07:13:49.303189+0000","last_became_active":"2023-10-08T07:13:33.733905+0000","last_became_peered":"2023-10-08T07:13:33.733905+0000","last_unstale":"2023-10-08T07:13:49.303189+0000","last_undegraded":"2023-10-08T07:13:49.303189+0000","last_fullsized":"2023-10-08T07:13:49.303189+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:33.463808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305246+0000","last_change":"2023-10-08T07:13:34.302227+0000","last_active":"2023-10-08T07:13:49.305246+0000","last_peered":"2023-10-08T07:13:49.305246+0000","last_clean":"2023-10-08T07:13:49.305246+0000","last_became_active":"2023-10-08T07:13:34.300127+0000","last_became_peered":"2023-10-08T07:13:34.300127+0000","last_unstale":"2023-10-08T07:13:49.305246+0000","last_undegraded":"2023-10-08T07:13:49.305246+0000","last_fullsized":"2023-10-08T07:13:49.305246+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:12:49.729707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287348+0000","last_change":"2023-10-08T07:13:48.380775+0000","last_active":"2023-10-08T07:13:49.287348+0000","last_peered":"2023-10-08T07:13:49.287348+0000","last_clean":"2023-10-08T07:13:49.287348+0000","last_became_active":"2023-10-08T07:13:48.380501+0000","last_became_peered":"2023-10-08T07:13:48.380501+0000","last_unstale":"2023-10-08T07:13:49.287348+0000","last_undegraded":"2023-10-08T07:13:49.287348+0000","last_fullsized":"2023-10-08T07:13:49.287348+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:20.813252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357767+0000","last_change":"2023-10-08T07:13:49.309072+0000","last_active":"2023-10-08T07:13:50.357767+0000","last_peered":"2023-10-08T07:13:50.357767+0000","last_clean":"2023-10-08T07:13:50.357767+0000","last_became_active":"2023-10-08T07:13:49.308753+0000","last_became_peered":"2023-10-08T07:13:49.308753+0000","last_unstale":"2023-10-08T07:13:50.357767+0000","last_undegraded":"2023-10-08T07:13:50.357767+0000","last_fullsized":"2023-10-08T07:13:50.357767+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:27.798388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298613+0000","last_change":"2023-10-08T07:13:33.733207+0000","last_active":"2023-10-08T07:13:49.298613+0000","last_peered":"2023-10-08T07:13:49.298613+0000","last_clean":"2023-10-08T07:13:49.298613+0000","last_became_active":"2023-10-08T07:13:33.732903+0000","last_became_peered":"2023-10-08T07:13:33.732903+0000","last_unstale":"2023-10-08T07:13:49.298613+0000","last_undegraded":"2023-10-08T07:13:49.298613+0000","last_fullsized":"2023-10-08T07:13:49.298613+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:34.358428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298923+0000","last_change":"2023-10-08T07:13:34.297602+0000","last_active":"2023-10-08T07:13:49.298923+0000","last_peered":"2023-10-08T07:13:49.298923+0000","last_clean":"2023-10-08T07:13:49.298923+0000","last_became_active":"2023-10-08T07:13:34.296820+0000","last_became_peered":"2023-10-08T07:13:34.296820+0000","last_unstale":"2023-10-08T07:13:49.298923+0000","last_undegraded":"2023-10-08T07:13:49.298923+0000","last_fullsized":"2023-10-08T07:13:49.298923+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:07.812354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298534+0000","last_change":"2023-10-08T07:13:48.374204+0000","last_active":"2023-10-08T07:13:49.298534+0000","last_peered":"2023-10-08T07:13:49.298534+0000","last_clean":"2023-10-08T07:13:49.298534+0000","last_became_active":"2023-10-08T07:13:48.373126+0000","last_became_peered":"2023-10-08T07:13:48.373126+0000","last_unstale":"2023-10-08T07:13:49.298534+0000","last_undegraded":"2023-10-08T07:13:49.298534+0000","last_fullsized":"2023-10-08T07:13:49.298534+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:48.758277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358171+0000","last_change":"2023-10-08T07:13:49.314030+0000","last_active":"2023-10-08T07:13:50.358171+0000","last_peered":"2023-10-08T07:13:50.358171+0000","last_clean":"2023-10-08T07:13:50.358171+0000","last_became_active":"2023-10-08T07:13:49.312711+0000","last_became_peered":"2023-10-08T07:13:49.312711+0000","last_unstale":"2023-10-08T07:13:50.358171+0000","last_undegraded":"2023-10-08T07:13:50.358171+0000","last_fullsized":"2023-10-08T07:13:50.358171+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:47:39.025998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.308199+0000","last_change":"2023-10-08T07:13:33.737211+0000","last_active":"2023-10-08T07:13:49.308199+0000","last_peered":"2023-10-08T07:13:49.308199+0000","last_clean":"2023-10-08T07:13:49.308199+0000","last_became_active":"2023-10-08T07:13:33.736904+0000","last_became_peered":"2023-10-08T07:13:33.736904+0000","last_unstale":"2023-10-08T07:13:49.308199+0000","last_undegraded":"2023-10-08T07:13:49.308199+0000","last_fullsized":"2023-10-08T07:13:49.308199+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:58.966472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288985+0000","last_change":"2023-10-08T07:13:34.302678+0000","last_active":"2023-10-08T07:13:49.288985+0000","last_peered":"2023-10-08T07:13:49.288985+0000","last_clean":"2023-10-08T07:13:49.288985+0000","last_became_active":"2023-10-08T07:13:34.302406+0000","last_became_peered":"2023-10-08T07:13:34.302406+0000","last_unstale":"2023-10-08T07:13:49.288985+0000","last_undegraded":"2023-10-08T07:13:49.288985+0000","last_fullsized":"2023-10-08T07:13:49.288985+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:48.219849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.301825+0000","last_change":"2023-10-08T07:13:48.389369+0000","last_active":"2023-10-08T07:13:49.301825+0000","last_peered":"2023-10-08T07:13:49.301825+0000","last_clean":"2023-10-08T07:13:49.301825+0000","last_became_active":"2023-10-08T07:13:48.389067+0000","last_became_peered":"2023-10-08T07:13:48.389067+0000","last_unstale":"2023-10-08T07:13:49.301825+0000","last_undegraded":"2023-10-08T07:13:49.301825+0000","last_fullsized":"2023-10-08T07:13:49.301825+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:36.638472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.286888+0000","last_change":"2023-10-08T07:13:49.311022+0000","last_active":"2023-10-08T07:13:50.286888+0000","last_peered":"2023-10-08T07:13:50.286888+0000","last_clean":"2023-10-08T07:13:50.286888+0000","last_became_active":"2023-10-08T07:13:49.310485+0000","last_became_peered":"2023-10-08T07:13:49.310485+0000","last_unstale":"2023-10-08T07:13:50.286888+0000","last_undegraded":"2023-10-08T07:13:50.286888+0000","last_fullsized":"2023-10-08T07:13:50.286888+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:02.557950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.6","version":"234'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.314910+0000","last_change":"2023-10-08T07:13:33.742793+0000","last_active":"2023-10-08T07:13:49.314910+0000","last_peered":"2023-10-08T07:13:49.314910+0000","last_clean":"2023-10-08T07:13:49.314910+0000","last_became_active":"2023-10-08T07:13:33.742030+0000","last_became_peered":"2023-10-08T07:13:33.742030+0000","last_unstale":"2023-10-08T07:13:49.314910+0000","last_undegraded":"2023-10-08T07:13:49.314910+0000","last_fullsized":"2023-10-08T07:13:49.314910+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:25:59.836339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285989+0000","last_change":"2023-10-08T07:13:34.308581+0000","last_active":"2023-10-08T07:13:49.285989+0000","last_peered":"2023-10-08T07:13:49.285989+0000","last_clean":"2023-10-08T07:13:49.285989+0000","last_became_active":"2023-10-08T07:13:34.308210+0000","last_became_peered":"2023-10-08T07:13:34.308210+0000","last_unstale":"2023-10-08T07:13:49.285989+0000","last_undegraded":"2023-10-08T07:13:49.285989+0000","last_fullsized":"2023-10-08T07:13:49.285989+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284904+0000","last_change":"2023-10-08T07:13:48.391100+0000","last_active":"2023-10-08T07:13:49.284904+0000","last_peered":"2023-10-08T07:13:49.284904+0000","last_clean":"2023-10-08T07:13:49.284904+0000","last_became_active":"2023-10-08T07:13:48.390787+0000","last_became_peered":"2023-10-08T07:13:48.390787+0000","last_unstale":"2023-10-08T07:13:49.284904+0000","last_undegraded":"2023-10-08T07:13:49.284904+0000","last_fullsized":"2023-10-08T07:13:49.284904+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:35.542870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278157+0000","last_change":"2023-10-08T07:13:49.310297+0000","last_active":"2023-10-08T07:13:50.278157+0000","last_peered":"2023-10-08T07:13:50.278157+0000","last_clean":"2023-10-08T07:13:50.278157+0000","last_became_active":"2023-10-08T07:13:49.310021+0000","last_became_peered":"2023-10-08T07:13:49.310021+0000","last_unstale":"2023-10-08T07:13:50.278157+0000","last_undegraded":"2023-10-08T07:13:50.278157+0000","last_fullsized":"2023-10-08T07:13:50.278157+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:37:19.685263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292199+0000","last_change":"2023-10-08T07:13:33.739638+0000","last_active":"2023-10-08T07:13:49.292199+0000","last_peered":"2023-10-08T07:13:49.292199+0000","last_clean":"2023-10-08T07:13:49.292199+0000","last_became_active":"2023-10-08T07:13:33.739352+0000","last_became_peered":"2023-10-08T07:13:33.739352+0000","last_unstale":"2023-10-08T07:13:49.292199+0000","last_undegraded":"2023-10-08T07:13:49.292199+0000","last_fullsized":"2023-10-08T07:13:49.292199+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:11.117318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285459+0000","last_change":"2023-10-08T07:13:34.293674+0000","last_active":"2023-10-08T07:13:49.285459+0000","last_peered":"2023-10-08T07:13:49.285459+0000","last_clean":"2023-10-08T07:13:49.285459+0000","last_became_active":"2023-10-08T07:13:34.292983+0000","last_became_peered":"2023-10-08T07:13:34.292983+0000","last_unstale":"2023-10-08T07:13:49.285459+0000","last_undegraded":"2023-10-08T07:13:49.285459+0000","last_fullsized":"2023-10-08T07:13:49.285459+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:20.477902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294951+0000","last_change":"2023-10-08T07:13:48.388173+0000","last_active":"2023-10-08T07:13:49.294951+0000","last_peered":"2023-10-08T07:13:49.294951+0000","last_clean":"2023-10-08T07:13:49.294951+0000","last_became_active":"2023-10-08T07:13:48.387881+0000","last_became_peered":"2023-10-08T07:13:48.387881+0000","last_unstale":"2023-10-08T07:13:49.294951+0000","last_undegraded":"2023-10-08T07:13:49.294951+0000","last_fullsized":"2023-10-08T07:13:49.294951+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:43:05.632452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.845606+0000","last_change":"2023-10-08T07:13:49.319691+0000","last_active":"2023-10-08T07:13:50.845606+0000","last_peered":"2023-10-08T07:13:50.845606+0000","last_clean":"2023-10-08T07:13:50.845606+0000","last_became_active":"2023-10-08T07:13:49.319380+0000","last_became_peered":"2023-10-08T07:13:49.319380+0000","last_unstale":"2023-10-08T07:13:50.845606+0000","last_undegraded":"2023-10-08T07:13:50.845606+0000","last_fullsized":"2023-10-08T07:13:50.845606+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:00.308061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.293276+0000","last_change":"2023-10-08T07:13:33.739626+0000","last_active":"2023-10-08T07:13:49.293276+0000","last_peered":"2023-10-08T07:13:49.293276+0000","last_clean":"2023-10-08T07:13:49.293276+0000","last_became_active":"2023-10-08T07:13:33.739348+0000","last_became_peered":"2023-10-08T07:13:33.739348+0000","last_unstale":"2023-10-08T07:13:49.293276+0000","last_undegraded":"2023-10-08T07:13:49.293276+0000","last_fullsized":"2023-10-08T07:13:49.293276+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:23.374596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.309563+0000","last_change":"2023-10-08T07:13:34.300038+0000","last_active":"2023-10-08T07:13:49.309563+0000","last_peered":"2023-10-08T07:13:49.309563+0000","last_clean":"2023-10-08T07:13:49.309563+0000","last_became_active":"2023-10-08T07:13:34.299721+0000","last_became_peered":"2023-10-08T07:13:34.299721+0000","last_unstale":"2023-10-08T07:13:49.309563+0000","last_undegraded":"2023-10-08T07:13:49.309563+0000","last_fullsized":"2023-10-08T07:13:49.309563+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:26.317918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.e","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.366615+0000","last_change":"2023-10-08T07:13:48.384247+0000","last_active":"2023-10-08T07:13:49.366615+0000","last_peered":"2023-10-08T07:13:49.366615+0000","last_clean":"2023-10-08T07:13:49.366615+0000","last_became_active":"2023-10-08T07:13:48.383959+0000","last_became_peered":"2023-10-08T07:13:48.383959+0000","last_unstale":"2023-10-08T07:13:49.366615+0000","last_undegraded":"2023-10-08T07:13:49.366615+0000","last_fullsized":"2023-10-08T07:13:49.366615+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:30:13.538712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448607+0000","last_change":"2023-10-08T07:13:49.319375+0000","last_active":"2023-10-08T07:13:50.448607+0000","last_peered":"2023-10-08T07:13:50.448607+0000","last_clean":"2023-10-08T07:13:50.448607+0000","last_became_active":"2023-10-08T07:13:49.319046+0000","last_became_peered":"2023-10-08T07:13:49.319046+0000","last_unstale":"2023-10-08T07:13:50.448607+0000","last_undegraded":"2023-10-08T07:13:50.448607+0000","last_fullsized":"2023-10-08T07:13:50.448607+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:43.401179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.302561+0000","last_change":"2023-10-08T07:13:33.742955+0000","last_active":"2023-10-08T07:13:49.302561+0000","last_peered":"2023-10-08T07:13:49.302561+0000","last_clean":"2023-10-08T07:13:49.302561+0000","last_became_active":"2023-10-08T07:13:33.742621+0000","last_became_peered":"2023-10-08T07:13:33.742621+0000","last_unstale":"2023-10-08T07:13:49.302561+0000","last_undegraded":"2023-10-08T07:13:49.302561+0000","last_fullsized":"2023-10-08T07:13:49.302561+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:25.311911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291806+0000","last_change":"2023-10-08T07:13:34.294206+0000","last_active":"2023-10-08T07:13:49.291806+0000","last_peered":"2023-10-08T07:13:49.291806+0000","last_clean":"2023-10-08T07:13:49.291806+0000","last_became_active":"2023-10-08T07:13:34.293897+0000","last_became_peered":"2023-10-08T07:13:34.293897+0000","last_unstale":"2023-10-08T07:13:49.291806+0000","last_undegraded":"2023-10-08T07:13:49.291806+0000","last_fullsized":"2023-10-08T07:13:49.291806+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370513+0000","last_change":"2023-10-08T07:13:48.392407+0000","last_active":"2023-10-08T07:13:49.370513+0000","last_peered":"2023-10-08T07:13:49.370513+0000","last_clean":"2023-10-08T07:13:49.370513+0000","last_became_active":"2023-10-08T07:13:48.392147+0000","last_became_peered":"2023-10-08T07:13:48.392147+0000","last_unstale":"2023-10-08T07:13:49.370513+0000","last_undegraded":"2023-10-08T07:13:49.370513+0000","last_fullsized":"2023-10-08T07:13:49.370513+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:54:27.098972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.286139+0000","last_change":"2023-10-08T07:13:49.312054+0000","last_active":"2023-10-08T07:13:50.286139+0000","last_peered":"2023-10-08T07:13:50.286139+0000","last_clean":"2023-10-08T07:13:50.286139+0000","last_became_active":"2023-10-08T07:13:49.311786+0000","last_became_peered":"2023-10-08T07:13:49.311786+0000","last_unstale":"2023-10-08T07:13:50.286139+0000","last_undegraded":"2023-10-08T07:13:50.286139+0000","last_fullsized":"2023-10-08T07:13:50.286139+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:45.266824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295620+0000","last_change":"2023-10-08T07:13:33.726092+0000","last_active":"2023-10-08T07:13:49.295620+0000","last_peered":"2023-10-08T07:13:49.295620+0000","last_clean":"2023-10-08T07:13:49.295620+0000","last_became_active":"2023-10-08T07:13:33.725701+0000","last_became_peered":"2023-10-08T07:13:33.725701+0000","last_unstale":"2023-10-08T07:13:49.295620+0000","last_undegraded":"2023-10-08T07:13:49.295620+0000","last_fullsized":"2023-10-08T07:13:49.295620+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:51.631016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298048+0000","last_change":"2023-10-08T07:13:34.301017+0000","last_active":"2023-10-08T07:13:49.298048+0000","last_peered":"2023-10-08T07:13:49.298048+0000","last_clean":"2023-10-08T07:13:49.298048+0000","last_became_active":"2023-10-08T07:13:34.300720+0000","last_became_peered":"2023-10-08T07:13:34.300720+0000","last_unstale":"2023-10-08T07:13:49.298048+0000","last_undegraded":"2023-10-08T07:13:49.298048+0000","last_fullsized":"2023-10-08T07:13:49.298048+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:36:05.460057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.304587+0000","last_change":"2023-10-08T07:13:48.389158+0000","last_active":"2023-10-08T07:13:49.304587+0000","last_peered":"2023-10-08T07:13:49.304587+0000","last_clean":"2023-10-08T07:13:49.304587+0000","last_became_active":"2023-10-08T07:13:48.388809+0000","last_became_peered":"2023-10-08T07:13:48.388809+0000","last_unstale":"2023-10-08T07:13:49.304587+0000","last_undegraded":"2023-10-08T07:13:49.304587+0000","last_fullsized":"2023-10-08T07:13:49.304587+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:28.191882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359126+0000","last_change":"2023-10-08T07:13:49.310553+0000","last_active":"2023-10-08T07:13:50.359126+0000","last_peered":"2023-10-08T07:13:50.359126+0000","last_clean":"2023-10-08T07:13:50.359126+0000","last_became_active":"2023-10-08T07:13:49.310222+0000","last_became_peered":"2023-10-08T07:13:49.310222+0000","last_unstale":"2023-10-08T07:13:50.359126+0000","last_undegraded":"2023-10-08T07:13:50.359126+0000","last_fullsized":"2023-10-08T07:13:50.359126+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:50.676315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.283072+0000","last_change":"2023-10-08T07:13:33.726781+0000","last_active":"2023-10-08T07:13:49.283072+0000","last_peered":"2023-10-08T07:13:49.283072+0000","last_clean":"2023-10-08T07:13:49.283072+0000","last_became_active":"2023-10-08T07:13:33.726459+0000","last_became_peered":"2023-10-08T07:13:33.726459+0000","last_unstale":"2023-10-08T07:13:49.283072+0000","last_undegraded":"2023-10-08T07:13:49.283072+0000","last_fullsized":"2023-10-08T07:13:49.283072+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:15:04.357904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295826+0000","last_change":"2023-10-08T07:13:34.294315+0000","last_active":"2023-10-08T07:13:49.295826+0000","last_peered":"2023-10-08T07:13:49.295826+0000","last_clean":"2023-10-08T07:13:49.295826+0000","last_became_active":"2023-10-08T07:13:34.294065+0000","last_became_peered":"2023-10-08T07:13:34.294065+0000","last_unstale":"2023-10-08T07:13:49.295826+0000","last_undegraded":"2023-10-08T07:13:49.295826+0000","last_fullsized":"2023-10-08T07:13:49.295826+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:37.124047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.372139+0000","last_change":"2023-10-08T07:13:48.389005+0000","last_active":"2023-10-08T07:13:49.372139+0000","last_peered":"2023-10-08T07:13:49.372139+0000","last_clean":"2023-10-08T07:13:49.372139+0000","last_became_active":"2023-10-08T07:13:48.388729+0000","last_became_peered":"2023-10-08T07:13:48.388729+0000","last_unstale":"2023-10-08T07:13:49.372139+0000","last_undegraded":"2023-10-08T07:13:49.372139+0000","last_fullsized":"2023-10-08T07:13:49.372139+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:17:33.821205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.844921+0000","last_change":"2023-10-08T07:13:49.305463+0000","last_active":"2023-10-08T07:13:50.844921+0000","last_peered":"2023-10-08T07:13:50.844921+0000","last_clean":"2023-10-08T07:13:50.844921+0000","last_became_active":"2023-10-08T07:13:49.305059+0000","last_became_peered":"2023-10-08T07:13:49.305059+0000","last_unstale":"2023-10-08T07:13:50.844921+0000","last_undegraded":"2023-10-08T07:13:50.844921+0000","last_fullsized":"2023-10-08T07:13:50.844921+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:32.920258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282248+0000","last_change":"2023-10-08T07:13:33.729567+0000","last_active":"2023-10-08T07:13:49.282248+0000","last_peered":"2023-10-08T07:13:49.282248+0000","last_clean":"2023-10-08T07:13:49.282248+0000","last_became_active":"2023-10-08T07:13:33.729125+0000","last_became_peered":"2023-10-08T07:13:33.729125+0000","last_unstale":"2023-10-08T07:13:49.282248+0000","last_undegraded":"2023-10-08T07:13:49.282248+0000","last_fullsized":"2023-10-08T07:13:49.282248+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:13.193093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.303010+0000","last_change":"2023-10-08T07:13:34.308836+0000","last_active":"2023-10-08T07:13:49.303010+0000","last_peered":"2023-10-08T07:13:49.303010+0000","last_clean":"2023-10-08T07:13:49.303010+0000","last_became_active":"2023-10-08T07:13:34.308301+0000","last_became_peered":"2023-10-08T07:13:34.308301+0000","last_unstale":"2023-10-08T07:13:49.303010+0000","last_undegraded":"2023-10-08T07:13:49.303010+0000","last_fullsized":"2023-10-08T07:13:49.303010+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:42.058240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285901+0000","last_change":"2023-10-08T07:13:48.390712+0000","last_active":"2023-10-08T07:13:49.285901+0000","last_peered":"2023-10-08T07:13:49.285901+0000","last_clean":"2023-10-08T07:13:49.285901+0000","last_became_active":"2023-10-08T07:13:48.390349+0000","last_became_peered":"2023-10-08T07:13:48.390349+0000","last_unstale":"2023-10-08T07:13:49.285901+0000","last_undegraded":"2023-10-08T07:13:49.285901+0000","last_fullsized":"2023-10-08T07:13:49.285901+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:42.383236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283214+0000","last_change":"2023-10-08T07:13:49.316212+0000","last_active":"2023-10-08T07:13:50.283214+0000","last_peered":"2023-10-08T07:13:50.283214+0000","last_clean":"2023-10-08T07:13:50.283214+0000","last_became_active":"2023-10-08T07:13:49.315804+0000","last_became_peered":"2023-10-08T07:13:49.315804+0000","last_unstale":"2023-10-08T07:13:50.283214+0000","last_undegraded":"2023-10-08T07:13:50.283214+0000","last_fullsized":"2023-10-08T07:13:50.283214+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:46:01.118339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295073+0000","last_change":"2023-10-08T07:13:33.732139+0000","last_active":"2023-10-08T07:13:49.295073+0000","last_peered":"2023-10-08T07:13:49.295073+0000","last_clean":"2023-10-08T07:13:49.295073+0000","last_became_active":"2023-10-08T07:13:33.731892+0000","last_became_peered":"2023-10-08T07:13:33.731892+0000","last_unstale":"2023-10-08T07:13:49.295073+0000","last_undegraded":"2023-10-08T07:13:49.295073+0000","last_fullsized":"2023-10-08T07:13:49.295073+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:07.242491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.299155+0000","last_change":"2023-10-08T07:13:34.299071+0000","last_active":"2023-10-08T07:13:49.299155+0000","last_peered":"2023-10-08T07:13:49.299155+0000","last_clean":"2023-10-08T07:13:49.299155+0000","last_became_active":"2023-10-08T07:13:34.298776+0000","last_became_peered":"2023-10-08T07:13:34.298776+0000","last_unstale":"2023-10-08T07:13:49.299155+0000","last_undegraded":"2023-10-08T07:13:49.299155+0000","last_fullsized":"2023-10-08T07:13:49.299155+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:10.642317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.b","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.371881+0000","last_change":"2023-10-08T07:13:48.393982+0000","last_active":"2023-10-08T07:13:49.371881+0000","last_peered":"2023-10-08T07:13:49.371881+0000","last_clean":"2023-10-08T07:13:49.371881+0000","last_became_active":"2023-10-08T07:13:48.393696+0000","last_became_peered":"2023-10-08T07:13:48.393696+0000","last_unstale":"2023-10-08T07:13:49.371881+0000","last_undegraded":"2023-10-08T07:13:49.371881+0000","last_fullsized":"2023-10-08T07:13:49.371881+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:51.579289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358215+0000","last_change":"2023-10-08T07:13:49.312996+0000","last_active":"2023-10-08T07:13:50.358215+0000","last_peered":"2023-10-08T07:13:50.358215+0000","last_clean":"2023-10-08T07:13:50.358215+0000","last_became_active":"2023-10-08T07:13:49.312657+0000","last_became_peered":"2023-10-08T07:13:49.312657+0000","last_unstale":"2023-10-08T07:13:50.358215+0000","last_undegraded":"2023-10-08T07:13:50.358215+0000","last_fullsized":"2023-10-08T07:13:50.358215+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:28.367357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.e","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.299244+0000","last_change":"2023-10-08T07:13:33.736960+0000","last_active":"2023-10-08T07:13:49.299244+0000","last_peered":"2023-10-08T07:13:49.299244+0000","last_clean":"2023-10-08T07:13:49.299244+0000","last_became_active":"2023-10-08T07:13:33.736703+0000","last_became_peered":"2023-10-08T07:13:33.736703+0000","last_unstale":"2023-10-08T07:13:49.299244+0000","last_undegraded":"2023-10-08T07:13:49.299244+0000","last_fullsized":"2023-10-08T07:13:49.299244+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:39.196152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.293897+0000","last_change":"2023-10-08T07:13:34.291716+0000","last_active":"2023-10-08T07:13:49.293897+0000","last_peered":"2023-10-08T07:13:49.293897+0000","last_clean":"2023-10-08T07:13:49.293897+0000","last_became_active":"2023-10-08T07:13:34.291300+0000","last_became_peered":"2023-10-08T07:13:34.291300+0000","last_unstale":"2023-10-08T07:13:49.293897+0000","last_undegraded":"2023-10-08T07:13:49.293897+0000","last_fullsized":"2023-10-08T07:13:49.293897+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:18:07.189339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.277990+0000","last_change":"2023-10-08T07:13:48.383304+0000","last_active":"2023-10-08T07:13:49.277990+0000","last_peered":"2023-10-08T07:13:49.277990+0000","last_clean":"2023-10-08T07:13:49.277990+0000","last_became_active":"2023-10-08T07:13:48.383040+0000","last_became_peered":"2023-10-08T07:13:48.383040+0000","last_unstale":"2023-10-08T07:13:49.277990+0000","last_undegraded":"2023-10-08T07:13:49.277990+0000","last_fullsized":"2023-10-08T07:13:49.277990+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:05.475908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448022+0000","last_change":"2023-10-08T07:13:49.319374+0000","last_active":"2023-10-08T07:13:50.448022+0000","last_peered":"2023-10-08T07:13:50.448022+0000","last_clean":"2023-10-08T07:13:50.448022+0000","last_became_active":"2023-10-08T07:13:49.319049+0000","last_became_peered":"2023-10-08T07:13:49.319049+0000","last_unstale":"2023-10-08T07:13:50.448022+0000","last_undegraded":"2023-10-08T07:13:50.448022+0000","last_fullsized":"2023-10-08T07:13:50.448022+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:11:09.611966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298151+0000","last_change":"2023-10-08T07:13:33.728187+0000","last_active":"2023-10-08T07:13:49.298151+0000","last_peered":"2023-10-08T07:13:49.298151+0000","last_clean":"2023-10-08T07:13:49.298151+0000","last_became_active":"2023-10-08T07:13:33.727828+0000","last_became_peered":"2023-10-08T07:13:33.727828+0000","last_unstale":"2023-10-08T07:13:49.298151+0000","last_undegraded":"2023-10-08T07:13:49.298151+0000","last_fullsized":"2023-10-08T07:13:49.298151+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:00.539389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285227+0000","last_change":"2023-10-08T07:13:34.293870+0000","last_active":"2023-10-08T07:13:49.285227+0000","last_peered":"2023-10-08T07:13:49.285227+0000","last_clean":"2023-10-08T07:13:49.285227+0000","last_became_active":"2023-10-08T07:13:34.293397+0000","last_became_peered":"2023-10-08T07:13:34.293397+0000","last_unstale":"2023-10-08T07:13:49.285227+0000","last_undegraded":"2023-10-08T07:13:49.285227+0000","last_fullsized":"2023-10-08T07:13:49.285227+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285530+0000","last_change":"2023-10-08T07:13:48.385714+0000","last_active":"2023-10-08T07:13:49.285530+0000","last_peered":"2023-10-08T07:13:49.285530+0000","last_clean":"2023-10-08T07:13:49.285530+0000","last_became_active":"2023-10-08T07:13:48.385420+0000","last_became_peered":"2023-10-08T07:13:48.385420+0000","last_unstale":"2023-10-08T07:13:49.285530+0000","last_undegraded":"2023-10-08T07:13:49.285530+0000","last_fullsized":"2023-10-08T07:13:49.285530+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:07:53.826072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.729312+0000","last_change":"2023-10-08T07:13:49.302784+0000","last_active":"2023-10-08T07:13:50.729312+0000","last_peered":"2023-10-08T07:13:50.729312+0000","last_clean":"2023-10-08T07:13:50.729312+0000","last_became_active":"2023-10-08T07:13:49.302497+0000","last_became_peered":"2023-10-08T07:13:49.302497+0000","last_unstale":"2023-10-08T07:13:50.729312+0000","last_undegraded":"2023-10-08T07:13:50.729312+0000","last_fullsized":"2023-10-08T07:13:50.729312+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:53.127703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.10","version":"233'8","reported_seq":24,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.278141+0000","last_change":"2023-10-08T07:13:33.738121+0000","last_active":"2023-10-08T07:13:49.278141+0000","last_peered":"2023-10-08T07:13:49.278141+0000","last_clean":"2023-10-08T07:13:49.278141+0000","last_became_active":"2023-10-08T07:13:33.737903+0000","last_became_peered":"2023-10-08T07:13:33.737903+0000","last_unstale":"2023-10-08T07:13:49.278141+0000","last_undegraded":"2023-10-08T07:13:49.278141+0000","last_fullsized":"2023-10-08T07:13:49.278141+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:15.275258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305608+0000","last_change":"2023-10-08T07:13:34.306066+0000","last_active":"2023-10-08T07:13:49.305608+0000","last_peered":"2023-10-08T07:13:49.305608+0000","last_clean":"2023-10-08T07:13:49.305608+0000","last_became_active":"2023-10-08T07:13:34.302955+0000","last_became_peered":"2023-10-08T07:13:34.302955+0000","last_unstale":"2023-10-08T07:13:49.305608+0000","last_undegraded":"2023-10-08T07:13:49.305608+0000","last_fullsized":"2023-10-08T07:13:49.305608+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:03.795703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.308773+0000","last_change":"2023-10-08T07:13:48.397214+0000","last_active":"2023-10-08T07:13:49.308773+0000","last_peered":"2023-10-08T07:13:49.308773+0000","last_clean":"2023-10-08T07:13:49.308773+0000","last_became_active":"2023-10-08T07:13:48.396880+0000","last_became_peered":"2023-10-08T07:13:48.396880+0000","last_unstale":"2023-10-08T07:13:49.308773+0000","last_undegraded":"2023-10-08T07:13:49.308773+0000","last_fullsized":"2023-10-08T07:13:49.308773+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:03.803430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.729631+0000","last_change":"2023-10-08T07:13:49.318210+0000","last_active":"2023-10-08T07:13:50.729631+0000","last_peered":"2023-10-08T07:13:50.729631+0000","last_clean":"2023-10-08T07:13:50.729631+0000","last_became_active":"2023-10-08T07:13:49.317898+0000","last_became_peered":"2023-10-08T07:13:49.317898+0000","last_unstale":"2023-10-08T07:13:50.729631+0000","last_undegraded":"2023-10-08T07:13:50.729631+0000","last_fullsized":"2023-10-08T07:13:50.729631+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:59:10.726131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286393+0000","last_change":"2023-10-08T07:13:33.730922+0000","last_active":"2023-10-08T07:13:49.286393+0000","last_peered":"2023-10-08T07:13:49.286393+0000","last_clean":"2023-10-08T07:13:49.286393+0000","last_became_active":"2023-10-08T07:13:33.730666+0000","last_became_peered":"2023-10-08T07:13:33.730666+0000","last_unstale":"2023-10-08T07:13:49.286393+0000","last_undegraded":"2023-10-08T07:13:49.286393+0000","last_fullsized":"2023-10-08T07:13:49.286393+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:59:12.086089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291729+0000","last_change":"2023-10-08T07:13:34.302276+0000","last_active":"2023-10-08T07:13:49.291729+0000","last_peered":"2023-10-08T07:13:49.291729+0000","last_clean":"2023-10-08T07:13:49.291729+0000","last_became_active":"2023-10-08T07:13:34.301988+0000","last_became_peered":"2023-10-08T07:13:34.301988+0000","last_unstale":"2023-10-08T07:13:49.291729+0000","last_undegraded":"2023-10-08T07:13:49.291729+0000","last_fullsized":"2023-10-08T07:13:49.291729+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:19.135247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305499+0000","last_change":"2023-10-08T07:13:48.386216+0000","last_active":"2023-10-08T07:13:49.305499+0000","last_peered":"2023-10-08T07:13:49.305499+0000","last_clean":"2023-10-08T07:13:49.305499+0000","last_became_active":"2023-10-08T07:13:48.385877+0000","last_became_peered":"2023-10-08T07:13:48.385877+0000","last_unstale":"2023-10-08T07:13:49.305499+0000","last_undegraded":"2023-10-08T07:13:49.305499+0000","last_fullsized":"2023-10-08T07:13:49.305499+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:10.951646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:51.086255+0000","last_change":"2023-10-08T07:13:49.313373+0000","last_active":"2023-10-08T07:13:51.086255+0000","last_peered":"2023-10-08T07:13:51.086255+0000","last_clean":"2023-10-08T07:13:51.086255+0000","last_became_active":"2023-10-08T07:13:49.312715+0000","last_became_peered":"2023-10-08T07:13:49.312715+0000","last_unstale":"2023-10-08T07:13:51.086255+0000","last_undegraded":"2023-10-08T07:13:51.086255+0000","last_fullsized":"2023-10-08T07:13:51.086255+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:37.656415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.286524+0000","last_change":"2023-10-08T07:13:33.725167+0000","last_active":"2023-10-08T07:13:49.286524+0000","last_peered":"2023-10-08T07:13:49.286524+0000","last_clean":"2023-10-08T07:13:49.286524+0000","last_became_active":"2023-10-08T07:13:33.724905+0000","last_became_peered":"2023-10-08T07:13:33.724905+0000","last_unstale":"2023-10-08T07:13:49.286524+0000","last_undegraded":"2023-10-08T07:13:49.286524+0000","last_fullsized":"2023-10-08T07:13:49.286524+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:37.525027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288970+0000","last_change":"2023-10-08T07:13:34.296170+0000","last_active":"2023-10-08T07:13:49.288970+0000","last_peered":"2023-10-08T07:13:49.288970+0000","last_clean":"2023-10-08T07:13:49.288970+0000","last_became_active":"2023-10-08T07:13:34.295736+0000","last_became_peered":"2023-10-08T07:13:34.295736+0000","last_unstale":"2023-10-08T07:13:49.288970+0000","last_undegraded":"2023-10-08T07:13:49.288970+0000","last_fullsized":"2023-10-08T07:13:49.288970+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.14","version":"237'6","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.374726+0000","last_change":"2023-10-08T07:13:48.395168+0000","last_active":"2023-10-08T07:13:49.374726+0000","last_peered":"2023-10-08T07:13:49.374726+0000","last_clean":"2023-10-08T07:13:49.374726+0000","last_became_active":"2023-10-08T07:13:48.394879+0000","last_became_peered":"2023-10-08T07:13:48.394879+0000","last_unstale":"2023-10-08T07:13:49.374726+0000","last_undegraded":"2023-10-08T07:13:49.374726+0000","last_fullsized":"2023-10-08T07:13:49.374726+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:44:56.271598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359261+0000","last_change":"2023-10-08T07:13:49.303768+0000","last_active":"2023-10-08T07:13:50.359261+0000","last_peered":"2023-10-08T07:13:50.359261+0000","last_clean":"2023-10-08T07:13:50.359261+0000","last_became_active":"2023-10-08T07:13:49.302929+0000","last_became_peered":"2023-10-08T07:13:49.302929+0000","last_unstale":"2023-10-08T07:13:50.359261+0000","last_undegraded":"2023-10-08T07:13:50.359261+0000","last_fullsized":"2023-10-08T07:13:50.359261+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:21.341547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295376+0000","last_change":"2023-10-08T07:13:33.730955+0000","last_active":"2023-10-08T07:13:49.295376+0000","last_peered":"2023-10-08T07:13:49.295376+0000","last_clean":"2023-10-08T07:13:49.295376+0000","last_became_active":"2023-10-08T07:13:33.730676+0000","last_became_peered":"2023-10-08T07:13:33.730676+0000","last_unstale":"2023-10-08T07:13:49.295376+0000","last_undegraded":"2023-10-08T07:13:49.295376+0000","last_fullsized":"2023-10-08T07:13:49.295376+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:28.953862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295664+0000","last_change":"2023-10-08T07:13:34.293585+0000","last_active":"2023-10-08T07:13:49.295664+0000","last_peered":"2023-10-08T07:13:49.295664+0000","last_clean":"2023-10-08T07:13:49.295664+0000","last_became_active":"2023-10-08T07:13:34.293318+0000","last_became_peered":"2023-10-08T07:13:34.293318+0000","last_unstale":"2023-10-08T07:13:49.295664+0000","last_undegraded":"2023-10-08T07:13:49.295664+0000","last_fullsized":"2023-10-08T07:13:49.295664+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:09:34.022529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"237'3","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.368552+0000","last_change":"2023-10-08T07:13:48.388878+0000","last_active":"2023-10-08T07:13:49.368552+0000","last_peered":"2023-10-08T07:13:49.368552+0000","last_clean":"2023-10-08T07:13:49.368552+0000","last_became_active":"2023-10-08T07:13:48.388595+0000","last_became_peered":"2023-10-08T07:13:48.388595+0000","last_unstale":"2023-10-08T07:13:49.368552+0000","last_undegraded":"2023-10-08T07:13:49.368552+0000","last_fullsized":"2023-10-08T07:13:49.368552+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:40.030965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283731+0000","last_change":"2023-10-08T07:13:49.318146+0000","last_active":"2023-10-08T07:13:50.283731+0000","last_peered":"2023-10-08T07:13:50.283731+0000","last_clean":"2023-10-08T07:13:50.283731+0000","last_became_active":"2023-10-08T07:13:49.317834+0000","last_became_peered":"2023-10-08T07:13:49.317834+0000","last_unstale":"2023-10-08T07:13:50.283731+0000","last_undegraded":"2023-10-08T07:13:50.283731+0000","last_fullsized":"2023-10-08T07:13:50.283731+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:51:22.431006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.14","version":"234'8","reported_seq":24,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296340+0000","last_change":"2023-10-08T07:13:33.746002+0000","last_active":"2023-10-08T07:13:49.296340+0000","last_peered":"2023-10-08T07:13:49.296340+0000","last_clean":"2023-10-08T07:13:49.296340+0000","last_became_active":"2023-10-08T07:13:33.745697+0000","last_became_peered":"2023-10-08T07:13:33.745697+0000","last_unstale":"2023-10-08T07:13:49.296340+0000","last_undegraded":"2023-10-08T07:13:49.296340+0000","last_fullsized":"2023-10-08T07:13:49.296340+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:25.841040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.283013+0000","last_change":"2023-10-08T07:13:34.293551+0000","last_active":"2023-10-08T07:13:49.283013+0000","last_peered":"2023-10-08T07:13:49.283013+0000","last_clean":"2023-10-08T07:13:49.283013+0000","last_became_active":"2023-10-08T07:13:34.292471+0000","last_became_peered":"2023-10-08T07:13:34.292471+0000","last_unstale":"2023-10-08T07:13:49.283013+0000","last_undegraded":"2023-10-08T07:13:49.283013+0000","last_fullsized":"2023-10-08T07:13:49.283013+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.288920+0000","last_change":"2023-10-08T07:13:48.373437+0000","last_active":"2023-10-08T07:13:49.288920+0000","last_peered":"2023-10-08T07:13:49.288920+0000","last_clean":"2023-10-08T07:13:49.288920+0000","last_became_active":"2023-10-08T07:13:48.373147+0000","last_became_peered":"2023-10-08T07:13:48.373147+0000","last_unstale":"2023-10-08T07:13:49.288920+0000","last_undegraded":"2023-10-08T07:13:49.288920+0000","last_fullsized":"2023-10-08T07:13:49.288920+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:07.670172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.447672+0000","last_change":"2023-10-08T07:13:49.311094+0000","last_active":"2023-10-08T07:13:50.447672+0000","last_peered":"2023-10-08T07:13:50.447672+0000","last_clean":"2023-10-08T07:13:50.447672+0000","last_became_active":"2023-10-08T07:13:49.310778+0000","last_became_peered":"2023-10-08T07:13:49.310778+0000","last_unstale":"2023-10-08T07:13:50.447672+0000","last_undegraded":"2023-10-08T07:13:50.447672+0000","last_fullsized":"2023-10-08T07:13:50.447672+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:25:13.654111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"233'3","reported_seq":19,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290932+0000","last_change":"2023-10-08T07:13:33.726791+0000","last_active":"2023-10-08T07:13:49.290932+0000","last_peered":"2023-10-08T07:13:49.290932+0000","last_clean":"2023-10-08T07:13:49.290932+0000","last_became_active":"2023-10-08T07:13:33.726482+0000","last_became_peered":"2023-10-08T07:13:33.726482+0000","last_unstale":"2023-10-08T07:13:49.290932+0000","last_undegraded":"2023-10-08T07:13:49.290932+0000","last_fullsized":"2023-10-08T07:13:49.290932+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:28:11.409539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.289130+0000","last_change":"2023-10-08T07:13:34.290971+0000","last_active":"2023-10-08T07:13:49.289130+0000","last_peered":"2023-10-08T07:13:49.289130+0000","last_clean":"2023-10-08T07:13:49.289130+0000","last_became_active":"2023-10-08T07:13:34.290294+0000","last_became_peered":"2023-10-08T07:13:34.290294+0000","last_unstale":"2023-10-08T07:13:49.289130+0000","last_undegraded":"2023-10-08T07:13:49.289130+0000","last_fullsized":"2023-10-08T07:13:49.289130+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.369932+0000","last_change":"2023-10-08T07:13:48.388537+0000","last_active":"2023-10-08T07:13:49.369932+0000","last_peered":"2023-10-08T07:13:49.369932+0000","last_clean":"2023-10-08T07:13:49.369932+0000","last_became_active":"2023-10-08T07:13:48.388005+0000","last_became_peered":"2023-10-08T07:13:48.388005+0000","last_unstale":"2023-10-08T07:13:49.369932+0000","last_undegraded":"2023-10-08T07:13:49.369932+0000","last_fullsized":"2023-10-08T07:13:49.369932+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:22.087343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.729279+0000","last_change":"2023-10-08T07:13:49.305690+0000","last_active":"2023-10-08T07:13:50.729279+0000","last_peered":"2023-10-08T07:13:50.729279+0000","last_clean":"2023-10-08T07:13:50.729279+0000","last_became_active":"2023-10-08T07:13:49.305404+0000","last_became_peered":"2023-10-08T07:13:49.305404+0000","last_unstale":"2023-10-08T07:13:50.729279+0000","last_undegraded":"2023-10-08T07:13:50.729279+0000","last_fullsized":"2023-10-08T07:13:50.729279+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:22.454210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295328+0000","last_change":"2023-10-08T07:13:33.736423+0000","last_active":"2023-10-08T07:13:49.295328+0000","last_peered":"2023-10-08T07:13:49.295328+0000","last_clean":"2023-10-08T07:13:49.295328+0000","last_became_active":"2023-10-08T07:13:33.735959+0000","last_became_peered":"2023-10-08T07:13:33.735959+0000","last_unstale":"2023-10-08T07:13:49.295328+0000","last_undegraded":"2023-10-08T07:13:49.295328+0000","last_fullsized":"2023-10-08T07:13:49.295328+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:00.365787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285770+0000","last_change":"2023-10-08T07:13:34.302391+0000","last_active":"2023-10-08T07:13:49.285770+0000","last_peered":"2023-10-08T07:13:49.285770+0000","last_clean":"2023-10-08T07:13:49.285770+0000","last_became_active":"2023-10-08T07:13:34.302105+0000","last_became_peered":"2023-10-08T07:13:34.302105+0000","last_unstale":"2023-10-08T07:13:49.285770+0000","last_undegraded":"2023-10-08T07:13:49.285770+0000","last_fullsized":"2023-10-08T07:13:49.285770+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:04.989796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"237'8","reported_seq":20,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.373437+0000","last_change":"2023-10-08T07:13:48.391735+0000","last_active":"2023-10-08T07:13:49.373437+0000","last_peered":"2023-10-08T07:13:49.373437+0000","last_clean":"2023-10-08T07:13:49.373437+0000","last_became_active":"2023-10-08T07:13:48.391330+0000","last_became_peered":"2023-10-08T07:13:48.391330+0000","last_unstale":"2023-10-08T07:13:49.373437+0000","last_undegraded":"2023-10-08T07:13:49.373437+0000","last_fullsized":"2023-10-08T07:13:49.373437+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:23.741971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357935+0000","last_change":"2023-10-08T07:13:49.313956+0000","last_active":"2023-10-08T07:13:50.357935+0000","last_peered":"2023-10-08T07:13:50.357935+0000","last_clean":"2023-10-08T07:13:50.357935+0000","last_became_active":"2023-10-08T07:13:49.313687+0000","last_became_peered":"2023-10-08T07:13:49.313687+0000","last_unstale":"2023-10-08T07:13:50.357935+0000","last_undegraded":"2023-10-08T07:13:50.357935+0000","last_fullsized":"2023-10-08T07:13:50.357935+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:35.653573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284639+0000","last_change":"2023-10-08T07:13:33.735184+0000","last_active":"2023-10-08T07:13:49.284639+0000","last_peered":"2023-10-08T07:13:49.284639+0000","last_clean":"2023-10-08T07:13:49.284639+0000","last_became_active":"2023-10-08T07:13:33.734915+0000","last_became_peered":"2023-10-08T07:13:33.734915+0000","last_unstale":"2023-10-08T07:13:49.284639+0000","last_undegraded":"2023-10-08T07:13:49.284639+0000","last_fullsized":"2023-10-08T07:13:49.284639+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:52.395241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298171+0000","last_change":"2023-10-08T07:13:34.294204+0000","last_active":"2023-10-08T07:13:49.298171+0000","last_peered":"2023-10-08T07:13:49.298171+0000","last_clean":"2023-10-08T07:13:49.298171+0000","last_became_active":"2023-10-08T07:13:34.293887+0000","last_became_peered":"2023-10-08T07:13:34.293887+0000","last_unstale":"2023-10-08T07:13:49.298171+0000","last_undegraded":"2023-10-08T07:13:49.298171+0000","last_fullsized":"2023-10-08T07:13:49.298171+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:59.699025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.293430+0000","last_change":"2023-10-08T07:13:48.389962+0000","last_active":"2023-10-08T07:13:49.293430+0000","last_peered":"2023-10-08T07:13:49.293430+0000","last_clean":"2023-10-08T07:13:49.293430+0000","last_became_active":"2023-10-08T07:13:48.389607+0000","last_became_peered":"2023-10-08T07:13:48.389607+0000","last_unstale":"2023-10-08T07:13:49.293430+0000","last_undegraded":"2023-10-08T07:13:49.293430+0000","last_fullsized":"2023-10-08T07:13:49.293430+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:27.240946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358517+0000","last_change":"2023-10-08T07:13:49.299000+0000","last_active":"2023-10-08T07:13:50.358517+0000","last_peered":"2023-10-08T07:13:50.358517+0000","last_clean":"2023-10-08T07:13:50.358517+0000","last_became_active":"2023-10-08T07:13:49.298705+0000","last_became_peered":"2023-10-08T07:13:49.298705+0000","last_unstale":"2023-10-08T07:13:50.358517+0000","last_undegraded":"2023-10-08T07:13:50.358517+0000","last_fullsized":"2023-10-08T07:13:50.358517+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:02:46.814809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.372451+0000","last_change":"2023-10-08T07:13:48.393308+0000","last_active":"2023-10-08T07:13:49.372451+0000","last_peered":"2023-10-08T07:13:49.372451+0000","last_clean":"2023-10-08T07:13:49.372451+0000","last_became_active":"2023-10-08T07:13:48.392985+0000","last_became_peered":"2023-10-08T07:13:48.392985+0000","last_unstale":"2023-10-08T07:13:49.372451+0000","last_undegraded":"2023-10-08T07:13:49.372451+0000","last_fullsized":"2023-10-08T07:13:49.372451+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:16:55.407347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284841+0000","last_change":"2023-10-08T07:13:33.731054+0000","last_active":"2023-10-08T07:13:49.284841+0000","last_peered":"2023-10-08T07:13:49.284841+0000","last_clean":"2023-10-08T07:13:49.284841+0000","last_became_active":"2023-10-08T07:13:33.730782+0000","last_became_peered":"2023-10-08T07:13:33.730782+0000","last_unstale":"2023-10-08T07:13:49.284841+0000","last_undegraded":"2023-10-08T07:13:49.284841+0000","last_fullsized":"2023-10-08T07:13:49.284841+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:06.476515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292156+0000","last_change":"2023-10-08T07:13:34.294260+0000","last_active":"2023-10-08T07:13:49.292156+0000","last_peered":"2023-10-08T07:13:49.292156+0000","last_clean":"2023-10-08T07:13:49.292156+0000","last_became_active":"2023-10-08T07:13:34.293992+0000","last_became_peered":"2023-10-08T07:13:34.293992+0000","last_unstale":"2023-10-08T07:13:49.292156+0000","last_undegraded":"2023-10-08T07:13:49.292156+0000","last_fullsized":"2023-10-08T07:13:49.292156+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:09:06.273218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358287+0000","last_change":"2023-10-08T07:13:49.309351+0000","last_active":"2023-10-08T07:13:50.358287+0000","last_peered":"2023-10-08T07:13:50.358287+0000","last_clean":"2023-10-08T07:13:50.358287+0000","last_became_active":"2023-10-08T07:13:49.309030+0000","last_became_peered":"2023-10-08T07:13:49.309030+0000","last_unstale":"2023-10-08T07:13:50.358287+0000","last_undegraded":"2023-10-08T07:13:50.358287+0000","last_fullsized":"2023-10-08T07:13:50.358287+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:08.833825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"237'4","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.372634+0000","last_change":"2023-10-08T07:13:48.385188+0000","last_active":"2023-10-08T07:13:49.372634+0000","last_peered":"2023-10-08T07:13:49.372634+0000","last_clean":"2023-10-08T07:13:49.372634+0000","last_became_active":"2023-10-08T07:13:48.384790+0000","last_became_peered":"2023-10-08T07:13:48.384790+0000","last_unstale":"2023-10-08T07:13:49.372634+0000","last_undegraded":"2023-10-08T07:13:49.372634+0000","last_fullsized":"2023-10-08T07:13:49.372634+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:30.625357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":16,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295154+0000","last_change":"2023-10-08T07:13:33.738155+0000","last_active":"2023-10-08T07:13:49.295154+0000","last_peered":"2023-10-08T07:13:49.295154+0000","last_clean":"2023-10-08T07:13:49.295154+0000","last_became_active":"2023-10-08T07:13:33.737850+0000","last_became_peered":"2023-10-08T07:13:33.737850+0000","last_unstale":"2023-10-08T07:13:49.295154+0000","last_undegraded":"2023-10-08T07:13:49.295154+0000","last_fullsized":"2023-10-08T07:13:49.295154+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:37.479840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.309407+0000","last_change":"2023-10-08T07:13:34.304888+0000","last_active":"2023-10-08T07:13:49.309407+0000","last_peered":"2023-10-08T07:13:49.309407+0000","last_clean":"2023-10-08T07:13:49.309407+0000","last_became_active":"2023-10-08T07:13:34.304536+0000","last_became_peered":"2023-10-08T07:13:34.304536+0000","last_unstale":"2023-10-08T07:13:49.309407+0000","last_undegraded":"2023-10-08T07:13:49.309407+0000","last_fullsized":"2023-10-08T07:13:49.309407+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:32:05.467048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283896+0000","last_change":"2023-10-08T07:13:49.315062+0000","last_active":"2023-10-08T07:13:50.283896+0000","last_peered":"2023-10-08T07:13:50.283896+0000","last_clean":"2023-10-08T07:13:50.283896+0000","last_became_active":"2023-10-08T07:13:49.310335+0000","last_became_peered":"2023-10-08T07:13:49.310335+0000","last_unstale":"2023-10-08T07:13:50.283896+0000","last_undegraded":"2023-10-08T07:13:50.283896+0000","last_fullsized":"2023-10-08T07:13:50.283896+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:22.202940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1b","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.297773+0000","last_change":"2023-10-08T07:13:33.738971+0000","last_active":"2023-10-08T07:13:49.297773+0000","last_peered":"2023-10-08T07:13:49.297773+0000","last_clean":"2023-10-08T07:13:49.297773+0000","last_became_active":"2023-10-08T07:13:33.738532+0000","last_became_peered":"2023-10-08T07:13:33.738532+0000","last_unstale":"2023-10-08T07:13:49.297773+0000","last_undegraded":"2023-10-08T07:13:49.297773+0000","last_fullsized":"2023-10-08T07:13:49.297773+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:06.806347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.303648+0000","last_change":"2023-10-08T07:13:34.302593+0000","last_active":"2023-10-08T07:13:49.303648+0000","last_peered":"2023-10-08T07:13:49.303648+0000","last_clean":"2023-10-08T07:13:49.303648+0000","last_became_active":"2023-10-08T07:13:34.302300+0000","last_became_peered":"2023-10-08T07:13:34.302300+0000","last_unstale":"2023-10-08T07:13:49.303648+0000","last_undegraded":"2023-10-08T07:13:49.303648+0000","last_fullsized":"2023-10-08T07:13:49.303648+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:13:33.465222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"237'2","reported_seq":14,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370719+0000","last_change":"2023-10-08T07:13:48.395886+0000","last_active":"2023-10-08T07:13:49.370719+0000","last_peered":"2023-10-08T07:13:49.370719+0000","last_clean":"2023-10-08T07:13:49.370719+0000","last_became_active":"2023-10-08T07:13:48.395585+0000","last_became_peered":"2023-10-08T07:13:48.395585+0000","last_unstale":"2023-10-08T07:13:49.370719+0000","last_undegraded":"2023-10-08T07:13:49.370719+0000","last_fullsized":"2023-10-08T07:13:49.370719+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:35:02.219752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283356+0000","last_change":"2023-10-08T07:13:49.308716+0000","last_active":"2023-10-08T07:13:50.283356+0000","last_peered":"2023-10-08T07:13:50.283356+0000","last_clean":"2023-10-08T07:13:50.283356+0000","last_became_active":"2023-10-08T07:13:49.308414+0000","last_became_peered":"2023-10-08T07:13:49.308414+0000","last_unstale":"2023-10-08T07:13:50.283356+0000","last_undegraded":"2023-10-08T07:13:50.283356+0000","last_fullsized":"2023-10-08T07:13:50.283356+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:30:10.968748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1c","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290749+0000","last_change":"2023-10-08T07:13:33.736823+0000","last_active":"2023-10-08T07:13:49.290749+0000","last_peered":"2023-10-08T07:13:49.290749+0000","last_clean":"2023-10-08T07:13:49.290749+0000","last_became_active":"2023-10-08T07:13:33.736537+0000","last_became_peered":"2023-10-08T07:13:33.736537+0000","last_unstale":"2023-10-08T07:13:49.290749+0000","last_undegraded":"2023-10-08T07:13:49.290749+0000","last_fullsized":"2023-10-08T07:13:49.290749+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:48.798820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282798+0000","last_change":"2023-10-08T07:13:34.293354+0000","last_active":"2023-10-08T07:13:49.282798+0000","last_peered":"2023-10-08T07:13:49.282798+0000","last_clean":"2023-10-08T07:13:49.282798+0000","last_became_active":"2023-10-08T07:13:34.288220+0000","last_became_peered":"2023-10-08T07:13:34.288220+0000","last_unstale":"2023-10-08T07:13:49.282798+0000","last_undegraded":"2023-10-08T07:13:49.282798+0000","last_fullsized":"2023-10-08T07:13:49.282798+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1a","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370308+0000","last_change":"2023-10-08T07:13:48.397371+0000","last_active":"2023-10-08T07:13:49.370308+0000","last_peered":"2023-10-08T07:13:49.370308+0000","last_clean":"2023-10-08T07:13:49.370308+0000","last_became_active":"2023-10-08T07:13:48.396954+0000","last_became_peered":"2023-10-08T07:13:48.396954+0000","last_unstale":"2023-10-08T07:13:49.370308+0000","last_undegraded":"2023-10-08T07:13:49.370308+0000","last_fullsized":"2023-10-08T07:13:49.370308+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:17.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358686+0000","last_change":"2023-10-08T07:13:49.309391+0000","last_active":"2023-10-08T07:13:50.358686+0000","last_peered":"2023-10-08T07:13:50.358686+0000","last_clean":"2023-10-08T07:13:50.358686+0000","last_became_active":"2023-10-08T07:13:49.309100+0000","last_became_peered":"2023-10-08T07:13:49.309100+0000","last_unstale":"2023-10-08T07:13:50.358686+0000","last_undegraded":"2023-10-08T07:13:50.358686+0000","last_fullsized":"2023-10-08T07:13:50.358686+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:38:36.841571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1d","version":"233'2","reported_seq":18,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294941+0000","last_change":"2023-10-08T07:13:33.724335+0000","last_active":"2023-10-08T07:13:49.294941+0000","last_peered":"2023-10-08T07:13:49.294941+0000","last_clean":"2023-10-08T07:13:49.294941+0000","last_became_active":"2023-10-08T07:13:33.724043+0000","last_became_peered":"2023-10-08T07:13:33.724043+0000","last_unstale":"2023-10-08T07:13:49.294941+0000","last_undegraded":"2023-10-08T07:13:49.294941+0000","last_fullsized":"2023-10-08T07:13:49.294941+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:58.128241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296177+0000","last_change":"2023-10-08T07:13:34.301668+0000","last_active":"2023-10-08T07:13:49.296177+0000","last_peered":"2023-10-08T07:13:49.296177+0000","last_clean":"2023-10-08T07:13:49.296177+0000","last_became_active":"2023-10-08T07:13:34.298698+0000","last_became_peered":"2023-10-08T07:13:34.298698+0000","last_unstale":"2023-10-08T07:13:49.296177+0000","last_undegraded":"2023-10-08T07:13:49.296177+0000","last_fullsized":"2023-10-08T07:13:49.296177+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:41.740638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1b","version":"237'1","reported_seq":13,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.370198+0000","last_change":"2023-10-08T07:13:48.399044+0000","last_active":"2023-10-08T07:13:49.370198+0000","last_peered":"2023-10-08T07:13:49.370198+0000","last_clean":"2023-10-08T07:13:49.370198+0000","last_became_active":"2023-10-08T07:13:48.398793+0000","last_became_peered":"2023-10-08T07:13:48.398793+0000","last_unstale":"2023-10-08T07:13:49.370198+0000","last_undegraded":"2023-10-08T07:13:49.370198+0000","last_fullsized":"2023-10-08T07:13:49.370198+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:38.004516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278331+0000","last_change":"2023-10-08T07:13:49.303740+0000","last_active":"2023-10-08T07:13:50.278331+0000","last_peered":"2023-10-08T07:13:50.278331+0000","last_clean":"2023-10-08T07:13:50.278331+0000","last_became_active":"2023-10-08T07:13:49.303416+0000","last_became_peered":"2023-10-08T07:13:49.303416+0000","last_unstale":"2023-10-08T07:13:50.278331+0000","last_undegraded":"2023-10-08T07:13:50.278331+0000","last_fullsized":"2023-10-08T07:13:50.278331+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:38:51.904860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1e","version":"233'1","reported_seq":17,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.299129+0000","last_change":"2023-10-08T07:13:33.732312+0000","last_active":"2023-10-08T07:13:49.299129+0000","last_peered":"2023-10-08T07:13:49.299129+0000","last_clean":"2023-10-08T07:13:49.299129+0000","last_became_active":"2023-10-08T07:13:33.731998+0000","last_became_peered":"2023-10-08T07:13:33.731998+0000","last_unstale":"2023-10-08T07:13:49.299129+0000","last_undegraded":"2023-10-08T07:13:49.299129+0000","last_fullsized":"2023-10-08T07:13:49.299129+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:10:53.964103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.293727+0000","last_change":"2023-10-08T07:13:34.293829+0000","last_active":"2023-10-08T07:13:49.293727+0000","last_peered":"2023-10-08T07:13:49.293727+0000","last_clean":"2023-10-08T07:13:49.293727+0000","last_became_active":"2023-10-08T07:13:34.293552+0000","last_became_peered":"2023-10-08T07:13:34.293552+0000","last_unstale":"2023-10-08T07:13:49.293727+0000","last_undegraded":"2023-10-08T07:13:49.293727+0000","last_fullsized":"2023-10-08T07:13:49.293727+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:45:38.485847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294683+0000","last_change":"2023-10-08T07:13:48.382427+0000","last_active":"2023-10-08T07:13:49.294683+0000","last_peered":"2023-10-08T07:13:49.294683+0000","last_clean":"2023-10-08T07:13:49.294683+0000","last_became_active":"2023-10-08T07:13:48.382141+0000","last_became_peered":"2023-10-08T07:13:48.382141+0000","last_unstale":"2023-10-08T07:13:49.294683+0000","last_undegraded":"2023-10-08T07:13:49.294683+0000","last_fullsized":"2023-10-08T07:13:49.294683+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:47:26.319310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.729707+0000","last_change":"2023-10-08T07:13:49.302647+0000","last_active":"2023-10-08T07:13:50.729707+0000","last_peered":"2023-10-08T07:13:50.729707+0000","last_clean":"2023-10-08T07:13:50.729707+0000","last_became_active":"2023-10-08T07:13:49.302334+0000","last_became_peered":"2023-10-08T07:13:49.302334+0000","last_unstale":"2023-10-08T07:13:50.729707+0000","last_undegraded":"2023-10-08T07:13:50.729707+0000","last_fullsized":"2023-10-08T07:13:50.729707+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:21.552769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"233'4","reported_seq":20,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.313059+0000","last_change":"2023-10-08T07:13:33.743017+0000","last_active":"2023-10-08T07:13:49.313059+0000","last_peered":"2023-10-08T07:13:49.313059+0000","last_clean":"2023-10-08T07:13:49.313059+0000","last_became_active":"2023-10-08T07:13:33.742582+0000","last_became_peered":"2023-10-08T07:13:33.742582+0000","last_unstale":"2023-10-08T07:13:49.313059+0000","last_undegraded":"2023-10-08T07:13:49.313059+0000","last_fullsized":"2023-10-08T07:13:49.313059+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:32.687282+0000","last_clean_scrub_stamp":"2023-10-08T07:13:32.687282+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:38:56.287497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296385+0000","last_change":"2023-10-08T07:13:34.286367+0000","last_active":"2023-10-08T07:13:49.296385+0000","last_peered":"2023-10-08T07:13:49.296385+0000","last_clean":"2023-10-08T07:13:49.296385+0000","last_became_active":"2023-10-08T07:13:34.285016+0000","last_became_peered":"2023-10-08T07:13:34.285016+0000","last_unstale":"2023-10-08T07:13:49.296385+0000","last_undegraded":"2023-10-08T07:13:49.296385+0000","last_fullsized":"2023-10-08T07:13:49.296385+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:41.466102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.302698+0000","last_change":"2023-10-08T07:13:48.394830+0000","last_active":"2023-10-08T07:13:49.302698+0000","last_peered":"2023-10-08T07:13:49.302698+0000","last_clean":"2023-10-08T07:13:49.302698+0000","last_became_active":"2023-10-08T07:13:48.394550+0000","last_became_peered":"2023-10-08T07:13:48.394550+0000","last_unstale":"2023-10-08T07:13:49.302698+0000","last_undegraded":"2023-10-08T07:13:49.302698+0000","last_fullsized":"2023-10-08T07:13:49.302698+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:47.317201+0000","last_clean_scrub_stamp":"2023-10-08T07:13:47.317201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:35:23.741971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.282589+0000","last_change":"2023-10-08T07:13:49.319738+0000","last_active":"2023-10-08T07:13:50.282589+0000","last_peered":"2023-10-08T07:13:50.282589+0000","last_clean":"2023-10-08T07:13:50.282589+0000","last_became_active":"2023-10-08T07:13:49.319429+0000","last_became_peered":"2023-10-08T07:13:49.319429+0000","last_unstale":"2023-10-08T07:13:50.282589+0000","last_undegraded":"2023-10-08T07:13:50.282589+0000","last_fullsized":"2023-10-08T07:13:50.282589+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:31.650499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.302892+0000","last_change":"2023-10-08T07:13:34.295873+0000","last_active":"2023-10-08T07:13:49.302892+0000","last_peered":"2023-10-08T07:13:49.302892+0000","last_clean":"2023-10-08T07:13:49.302892+0000","last_became_active":"2023-10-08T07:13:34.295546+0000","last_became_peered":"2023-10-08T07:13:34.295546+0000","last_unstale":"2023-10-08T07:13:49.302892+0000","last_undegraded":"2023-10-08T07:13:49.302892+0000","last_fullsized":"2023-10-08T07:13:49.302892+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:46:53.896975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448531+0000","last_change":"2023-10-08T07:13:49.312717+0000","last_active":"2023-10-08T07:13:50.448531+0000","last_peered":"2023-10-08T07:13:50.448531+0000","last_clean":"2023-10-08T07:13:50.448531+0000","last_became_active":"2023-10-08T07:13:49.312448+0000","last_became_peered":"2023-10-08T07:13:49.312448+0000","last_unstale":"2023-10-08T07:13:50.448531+0000","last_undegraded":"2023-10-08T07:13:50.448531+0000","last_fullsized":"2023-10-08T07:13:50.448531+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:39:52.376348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.294645+0000","last_change":"2023-10-08T07:13:34.301071+0000","last_active":"2023-10-08T07:13:49.294645+0000","last_peered":"2023-10-08T07:13:49.294645+0000","last_clean":"2023-10-08T07:13:49.294645+0000","last_became_active":"2023-10-08T07:13:34.300772+0000","last_became_peered":"2023-10-08T07:13:34.300772+0000","last_unstale":"2023-10-08T07:13:49.294645+0000","last_undegraded":"2023-10-08T07:13:49.294645+0000","last_fullsized":"2023-10-08T07:13:49.294645+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:59:54.643977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.282259+0000","last_change":"2023-10-08T07:13:49.320349+0000","last_active":"2023-10-08T07:13:50.282259+0000","last_peered":"2023-10-08T07:13:50.282259+0000","last_clean":"2023-10-08T07:13:50.282259+0000","last_became_active":"2023-10-08T07:13:49.320036+0000","last_became_peered":"2023-10-08T07:13:49.320036+0000","last_unstale":"2023-10-08T07:13:50.282259+0000","last_undegraded":"2023-10-08T07:13:50.282259+0000","last_fullsized":"2023-10-08T07:13:50.282259+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:44.601725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298904+0000","last_change":"2023-10-08T07:13:34.300974+0000","last_active":"2023-10-08T07:13:49.298904+0000","last_peered":"2023-10-08T07:13:49.298904+0000","last_clean":"2023-10-08T07:13:49.298904+0000","last_became_active":"2023-10-08T07:13:34.300529+0000","last_became_peered":"2023-10-08T07:13:34.300529+0000","last_unstale":"2023-10-08T07:13:49.298904+0000","last_undegraded":"2023-10-08T07:13:49.298904+0000","last_fullsized":"2023-10-08T07:13:49.298904+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:17.001444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278016+0000","last_change":"2023-10-08T07:13:49.306936+0000","last_active":"2023-10-08T07:13:50.278016+0000","last_peered":"2023-10-08T07:13:50.278016+0000","last_clean":"2023-10-08T07:13:50.278016+0000","last_became_active":"2023-10-08T07:13:49.306284+0000","last_became_peered":"2023-10-08T07:13:49.306284+0000","last_unstale":"2023-10-08T07:13:50.278016+0000","last_undegraded":"2023-10-08T07:13:50.278016+0000","last_fullsized":"2023-10-08T07:13:50.278016+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:51.779526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287748+0000","last_change":"2023-10-08T07:13:34.293712+0000","last_active":"2023-10-08T07:13:49.287748+0000","last_peered":"2023-10-08T07:13:49.287748+0000","last_clean":"2023-10-08T07:13:49.287748+0000","last_became_active":"2023-10-08T07:13:34.293415+0000","last_became_peered":"2023-10-08T07:13:34.293415+0000","last_unstale":"2023-10-08T07:13:49.287748+0000","last_undegraded":"2023-10-08T07:13:49.287748+0000","last_fullsized":"2023-10-08T07:13:49.287748+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:47.966787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.359086+0000","last_change":"2023-10-08T07:13:49.316786+0000","last_active":"2023-10-08T07:13:50.359086+0000","last_peered":"2023-10-08T07:13:50.359086+0000","last_clean":"2023-10-08T07:13:50.359086+0000","last_became_active":"2023-10-08T07:13:49.316437+0000","last_became_peered":"2023-10-08T07:13:49.316437+0000","last_unstale":"2023-10-08T07:13:50.359086+0000","last_undegraded":"2023-10-08T07:13:50.359086+0000","last_fullsized":"2023-10-08T07:13:50.359086+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:14.040352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282033+0000","last_change":"2023-10-08T07:13:34.297816+0000","last_active":"2023-10-08T07:13:49.282033+0000","last_peered":"2023-10-08T07:13:49.282033+0000","last_clean":"2023-10-08T07:13:49.282033+0000","last_became_active":"2023-10-08T07:13:34.297544+0000","last_became_peered":"2023-10-08T07:13:34.297544+0000","last_unstale":"2023-10-08T07:13:49.282033+0000","last_undegraded":"2023-10-08T07:13:49.282033+0000","last_fullsized":"2023-10-08T07:13:49.282033+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.846250+0000","last_change":"2023-10-08T07:13:49.313586+0000","last_active":"2023-10-08T07:13:50.846250+0000","last_peered":"2023-10-08T07:13:50.846250+0000","last_clean":"2023-10-08T07:13:50.846250+0000","last_became_active":"2023-10-08T07:13:49.312969+0000","last_became_peered":"2023-10-08T07:13:49.312969+0000","last_unstale":"2023-10-08T07:13:50.846250+0000","last_undegraded":"2023-10-08T07:13:50.846250+0000","last_fullsized":"2023-10-08T07:13:50.846250+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:02.919590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287925+0000","last_change":"2023-10-08T07:13:34.310969+0000","last_active":"2023-10-08T07:13:49.287925+0000","last_peered":"2023-10-08T07:13:49.287925+0000","last_clean":"2023-10-08T07:13:49.287925+0000","last_became_active":"2023-10-08T07:13:34.310679+0000","last_became_peered":"2023-10-08T07:13:34.310679+0000","last_unstale":"2023-10-08T07:13:49.287925+0000","last_undegraded":"2023-10-08T07:13:49.287925+0000","last_fullsized":"2023-10-08T07:13:49.287925+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.284208+0000","last_change":"2023-10-08T07:13:49.314485+0000","last_active":"2023-10-08T07:13:50.284208+0000","last_peered":"2023-10-08T07:13:50.284208+0000","last_clean":"2023-10-08T07:13:50.284208+0000","last_became_active":"2023-10-08T07:13:49.312136+0000","last_became_peered":"2023-10-08T07:13:49.312136+0000","last_unstale":"2023-10-08T07:13:50.284208+0000","last_undegraded":"2023-10-08T07:13:50.284208+0000","last_fullsized":"2023-10-08T07:13:50.284208+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:20:59.113588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298999+0000","last_change":"2023-10-08T07:13:34.304704+0000","last_active":"2023-10-08T07:13:49.298999+0000","last_peered":"2023-10-08T07:13:49.298999+0000","last_clean":"2023-10-08T07:13:49.298999+0000","last_became_active":"2023-10-08T07:13:34.304032+0000","last_became_peered":"2023-10-08T07:13:34.304032+0000","last_unstale":"2023-10-08T07:13:49.298999+0000","last_undegraded":"2023-10-08T07:13:49.298999+0000","last_fullsized":"2023-10-08T07:13:49.298999+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:15.818894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:51.085872+0000","last_change":"2023-10-08T07:13:49.310313+0000","last_active":"2023-10-08T07:13:51.085872+0000","last_peered":"2023-10-08T07:13:51.085872+0000","last_clean":"2023-10-08T07:13:51.085872+0000","last_became_active":"2023-10-08T07:13:49.310027+0000","last_became_peered":"2023-10-08T07:13:49.310027+0000","last_unstale":"2023-10-08T07:13:51.085872+0000","last_undegraded":"2023-10-08T07:13:51.085872+0000","last_fullsized":"2023-10-08T07:13:51.085872+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:23.105286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292077+0000","last_change":"2023-10-08T07:13:34.298953+0000","last_active":"2023-10-08T07:13:49.292077+0000","last_peered":"2023-10-08T07:13:49.292077+0000","last_clean":"2023-10-08T07:13:49.292077+0000","last_became_active":"2023-10-08T07:13:34.298694+0000","last_became_peered":"2023-10-08T07:13:34.298694+0000","last_unstale":"2023-10-08T07:13:49.292077+0000","last_undegraded":"2023-10-08T07:13:49.292077+0000","last_fullsized":"2023-10-08T07:13:49.292077+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:30.925325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.728973+0000","last_change":"2023-10-08T07:13:49.307150+0000","last_active":"2023-10-08T07:13:50.728973+0000","last_peered":"2023-10-08T07:13:50.728973+0000","last_clean":"2023-10-08T07:13:50.728973+0000","last_became_active":"2023-10-08T07:13:49.306772+0000","last_became_peered":"2023-10-08T07:13:49.306772+0000","last_unstale":"2023-10-08T07:13:50.728973+0000","last_undegraded":"2023-10-08T07:13:50.728973+0000","last_fullsized":"2023-10-08T07:13:50.728973+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:25.284173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.277869+0000","last_change":"2023-10-08T07:13:34.293797+0000","last_active":"2023-10-08T07:13:49.277869+0000","last_peered":"2023-10-08T07:13:49.277869+0000","last_clean":"2023-10-08T07:13:49.277869+0000","last_became_active":"2023-10-08T07:13:34.289224+0000","last_became_peered":"2023-10-08T07:13:34.289224+0000","last_unstale":"2023-10-08T07:13:49.277869+0000","last_undegraded":"2023-10-08T07:13:49.277869+0000","last_fullsized":"2023-10-08T07:13:49.277869+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:33.960003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357969+0000","last_change":"2023-10-08T07:13:49.303551+0000","last_active":"2023-10-08T07:13:50.357969+0000","last_peered":"2023-10-08T07:13:50.357969+0000","last_clean":"2023-10-08T07:13:50.357969+0000","last_became_active":"2023-10-08T07:13:49.303258+0000","last_became_peered":"2023-10-08T07:13:49.303258+0000","last_unstale":"2023-10-08T07:13:50.357969+0000","last_undegraded":"2023-10-08T07:13:50.357969+0000","last_fullsized":"2023-10-08T07:13:50.357969+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:08.504585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295174+0000","last_change":"2023-10-08T07:13:34.291585+0000","last_active":"2023-10-08T07:13:49.295174+0000","last_peered":"2023-10-08T07:13:49.295174+0000","last_clean":"2023-10-08T07:13:49.295174+0000","last_became_active":"2023-10-08T07:13:34.291125+0000","last_became_peered":"2023-10-08T07:13:34.291125+0000","last_unstale":"2023-10-08T07:13:49.295174+0000","last_undegraded":"2023-10-08T07:13:49.295174+0000","last_fullsized":"2023-10-08T07:13:49.295174+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:14.399506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.844917+0000","last_change":"2023-10-08T07:13:49.313885+0000","last_active":"2023-10-08T07:13:50.844917+0000","last_peered":"2023-10-08T07:13:50.844917+0000","last_clean":"2023-10-08T07:13:50.844917+0000","last_became_active":"2023-10-08T07:13:49.313596+0000","last_became_peered":"2023-10-08T07:13:49.313596+0000","last_unstale":"2023-10-08T07:13:50.844917+0000","last_undegraded":"2023-10-08T07:13:50.844917+0000","last_fullsized":"2023-10-08T07:13:50.844917+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:53:25.533946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296171+0000","last_change":"2023-10-08T07:13:34.297774+0000","last_active":"2023-10-08T07:13:49.296171+0000","last_peered":"2023-10-08T07:13:49.296171+0000","last_clean":"2023-10-08T07:13:49.296171+0000","last_became_active":"2023-10-08T07:13:34.297492+0000","last_became_peered":"2023-10-08T07:13:34.297492+0000","last_unstale":"2023-10-08T07:13:49.296171+0000","last_undegraded":"2023-10-08T07:13:49.296171+0000","last_fullsized":"2023-10-08T07:13:49.296171+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:11.413436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.277878+0000","last_change":"2023-10-08T07:13:49.298115+0000","last_active":"2023-10-08T07:13:50.277878+0000","last_peered":"2023-10-08T07:13:50.277878+0000","last_clean":"2023-10-08T07:13:50.277878+0000","last_became_active":"2023-10-08T07:13:49.297582+0000","last_became_peered":"2023-10-08T07:13:49.297582+0000","last_unstale":"2023-10-08T07:13:50.277878+0000","last_undegraded":"2023-10-08T07:13:50.277878+0000","last_fullsized":"2023-10-08T07:13:50.277878+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:24:11.399029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.302860+0000","last_change":"2023-10-08T07:13:34.294431+0000","last_active":"2023-10-08T07:13:49.302860+0000","last_peered":"2023-10-08T07:13:49.302860+0000","last_clean":"2023-10-08T07:13:49.302860+0000","last_became_active":"2023-10-08T07:13:34.294129+0000","last_became_peered":"2023-10-08T07:13:34.294129+0000","last_unstale":"2023-10-08T07:13:49.302860+0000","last_undegraded":"2023-10-08T07:13:49.302860+0000","last_fullsized":"2023-10-08T07:13:49.302860+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:02:32.189802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358949+0000","last_change":"2023-10-08T07:13:49.313393+0000","last_active":"2023-10-08T07:13:50.358949+0000","last_peered":"2023-10-08T07:13:50.358949+0000","last_clean":"2023-10-08T07:13:50.358949+0000","last_became_active":"2023-10-08T07:13:49.313094+0000","last_became_peered":"2023-10-08T07:13:49.313094+0000","last_unstale":"2023-10-08T07:13:50.358949+0000","last_undegraded":"2023-10-08T07:13:50.358949+0000","last_fullsized":"2023-10-08T07:13:50.358949+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:06.091202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305137+0000","last_change":"2023-10-08T07:13:34.297677+0000","last_active":"2023-10-08T07:13:49.305137+0000","last_peered":"2023-10-08T07:13:49.305137+0000","last_clean":"2023-10-08T07:13:49.305137+0000","last_became_active":"2023-10-08T07:13:34.296526+0000","last_became_peered":"2023-10-08T07:13:34.296526+0000","last_unstale":"2023-10-08T07:13:49.305137+0000","last_undegraded":"2023-10-08T07:13:49.305137+0000","last_fullsized":"2023-10-08T07:13:49.305137+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:51.977221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.285459+0000","last_change":"2023-10-08T07:13:49.315159+0000","last_active":"2023-10-08T07:13:50.285459+0000","last_peered":"2023-10-08T07:13:50.285459+0000","last_clean":"2023-10-08T07:13:50.285459+0000","last_became_active":"2023-10-08T07:13:49.314845+0000","last_became_peered":"2023-10-08T07:13:49.314845+0000","last_unstale":"2023-10-08T07:13:50.285459+0000","last_undegraded":"2023-10-08T07:13:50.285459+0000","last_fullsized":"2023-10-08T07:13:50.285459+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:02:25.205836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.305789+0000","last_change":"2023-10-08T07:13:34.295222+0000","last_active":"2023-10-08T07:13:49.305789+0000","last_peered":"2023-10-08T07:13:49.305789+0000","last_clean":"2023-10-08T07:13:49.305789+0000","last_became_active":"2023-10-08T07:13:34.289517+0000","last_became_peered":"2023-10-08T07:13:34.289517+0000","last_unstale":"2023-10-08T07:13:49.305789+0000","last_undegraded":"2023-10-08T07:13:49.305789+0000","last_fullsized":"2023-10-08T07:13:49.305789+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:38.224838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.287377+0000","last_change":"2023-10-08T07:13:49.301147+0000","last_active":"2023-10-08T07:13:50.287377+0000","last_peered":"2023-10-08T07:13:50.287377+0000","last_clean":"2023-10-08T07:13:50.287377+0000","last_became_active":"2023-10-08T07:13:49.300846+0000","last_became_peered":"2023-10-08T07:13:49.300846+0000","last_unstale":"2023-10-08T07:13:50.287377+0000","last_undegraded":"2023-10-08T07:13:50.287377+0000","last_fullsized":"2023-10-08T07:13:50.287377+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:17:13.776104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291605+0000","last_change":"2023-10-08T07:13:34.299238+0000","last_active":"2023-10-08T07:13:49.291605+0000","last_peered":"2023-10-08T07:13:49.291605+0000","last_clean":"2023-10-08T07:13:49.291605+0000","last_became_active":"2023-10-08T07:13:34.298585+0000","last_became_peered":"2023-10-08T07:13:34.298585+0000","last_unstale":"2023-10-08T07:13:49.291605+0000","last_undegraded":"2023-10-08T07:13:49.291605+0000","last_fullsized":"2023-10-08T07:13:49.291605+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.846256+0000","last_change":"2023-10-08T07:13:49.314244+0000","last_active":"2023-10-08T07:13:50.846256+0000","last_peered":"2023-10-08T07:13:50.846256+0000","last_clean":"2023-10-08T07:13:50.846256+0000","last_became_active":"2023-10-08T07:13:49.313960+0000","last_became_peered":"2023-10-08T07:13:49.313960+0000","last_unstale":"2023-10-08T07:13:50.846256+0000","last_undegraded":"2023-10-08T07:13:50.846256+0000","last_fullsized":"2023-10-08T07:13:50.846256+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:36:04.687046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.292035+0000","last_change":"2023-10-08T07:13:34.308624+0000","last_active":"2023-10-08T07:13:49.292035+0000","last_peered":"2023-10-08T07:13:49.292035+0000","last_clean":"2023-10-08T07:13:49.292035+0000","last_became_active":"2023-10-08T07:13:34.308246+0000","last_became_peered":"2023-10-08T07:13:34.308246+0000","last_unstale":"2023-10-08T07:13:49.292035+0000","last_undegraded":"2023-10-08T07:13:49.292035+0000","last_fullsized":"2023-10-08T07:13:49.292035+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:56.133630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283750+0000","last_change":"2023-10-08T07:13:49.303206+0000","last_active":"2023-10-08T07:13:50.283750+0000","last_peered":"2023-10-08T07:13:50.283750+0000","last_clean":"2023-10-08T07:13:50.283750+0000","last_became_active":"2023-10-08T07:13:49.302904+0000","last_became_peered":"2023-10-08T07:13:49.302904+0000","last_unstale":"2023-10-08T07:13:50.283750+0000","last_undegraded":"2023-10-08T07:13:50.283750+0000","last_fullsized":"2023-10-08T07:13:50.283750+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:02:27.339829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.284637+0000","last_change":"2023-10-08T07:13:34.300821+0000","last_active":"2023-10-08T07:13:49.284637+0000","last_peered":"2023-10-08T07:13:49.284637+0000","last_clean":"2023-10-08T07:13:49.284637+0000","last_became_active":"2023-10-08T07:13:34.300558+0000","last_became_peered":"2023-10-08T07:13:34.300558+0000","last_unstale":"2023-10-08T07:13:49.284637+0000","last_undegraded":"2023-10-08T07:13:49.284637+0000","last_fullsized":"2023-10-08T07:13:49.284637+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:34:52.727982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.286407+0000","last_change":"2023-10-08T07:13:49.318602+0000","last_active":"2023-10-08T07:13:50.286407+0000","last_peered":"2023-10-08T07:13:50.286407+0000","last_clean":"2023-10-08T07:13:50.286407+0000","last_became_active":"2023-10-08T07:13:49.318304+0000","last_became_peered":"2023-10-08T07:13:49.318304+0000","last_unstale":"2023-10-08T07:13:50.286407+0000","last_undegraded":"2023-10-08T07:13:50.286407+0000","last_fullsized":"2023-10-08T07:13:50.286407+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:25:29.701367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.302401+0000","last_change":"2023-10-08T07:13:34.309619+0000","last_active":"2023-10-08T07:13:49.302401+0000","last_peered":"2023-10-08T07:13:49.302401+0000","last_clean":"2023-10-08T07:13:49.302401+0000","last_became_active":"2023-10-08T07:13:34.308202+0000","last_became_peered":"2023-10-08T07:13:34.308202+0000","last_unstale":"2023-10-08T07:13:49.302401+0000","last_undegraded":"2023-10-08T07:13:49.302401+0000","last_fullsized":"2023-10-08T07:13:49.302401+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:59.914799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.281862+0000","last_change":"2023-10-08T07:13:49.300308+0000","last_active":"2023-10-08T07:13:50.281862+0000","last_peered":"2023-10-08T07:13:50.281862+0000","last_clean":"2023-10-08T07:13:50.281862+0000","last_became_active":"2023-10-08T07:13:49.299809+0000","last_became_peered":"2023-10-08T07:13:49.299809+0000","last_unstale":"2023-10-08T07:13:50.281862+0000","last_undegraded":"2023-10-08T07:13:50.281862+0000","last_fullsized":"2023-10-08T07:13:50.281862+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:07.278879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.295969+0000","last_change":"2023-10-08T07:13:34.298407+0000","last_active":"2023-10-08T07:13:49.295969+0000","last_peered":"2023-10-08T07:13:49.295969+0000","last_clean":"2023-10-08T07:13:49.295969+0000","last_became_active":"2023-10-08T07:13:34.298139+0000","last_became_peered":"2023-10-08T07:13:34.298139+0000","last_unstale":"2023-10-08T07:13:49.295969+0000","last_undegraded":"2023-10-08T07:13:49.295969+0000","last_fullsized":"2023-10-08T07:13:49.295969+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.728918+0000","last_change":"2023-10-08T07:13:49.304934+0000","last_active":"2023-10-08T07:13:50.728918+0000","last_peered":"2023-10-08T07:13:50.728918+0000","last_clean":"2023-10-08T07:13:50.728918+0000","last_became_active":"2023-10-08T07:13:49.304630+0000","last_became_peered":"2023-10-08T07:13:49.304630+0000","last_unstale":"2023-10-08T07:13:50.728918+0000","last_undegraded":"2023-10-08T07:13:50.728918+0000","last_fullsized":"2023-10-08T07:13:50.728918+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:30.460750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.297629+0000","last_change":"2023-10-08T07:13:34.306040+0000","last_active":"2023-10-08T07:13:49.297629+0000","last_peered":"2023-10-08T07:13:49.297629+0000","last_clean":"2023-10-08T07:13:49.297629+0000","last_became_active":"2023-10-08T07:13:34.303015+0000","last_became_peered":"2023-10-08T07:13:34.303015+0000","last_unstale":"2023-10-08T07:13:49.297629+0000","last_undegraded":"2023-10-08T07:13:49.297629+0000","last_fullsized":"2023-10-08T07:13:49.297629+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:42.884365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358958+0000","last_change":"2023-10-08T07:13:49.310325+0000","last_active":"2023-10-08T07:13:50.358958+0000","last_peered":"2023-10-08T07:13:50.358958+0000","last_clean":"2023-10-08T07:13:50.358958+0000","last_became_active":"2023-10-08T07:13:49.310037+0000","last_became_peered":"2023-10-08T07:13:49.310037+0000","last_unstale":"2023-10-08T07:13:50.358958+0000","last_undegraded":"2023-10-08T07:13:50.358958+0000","last_fullsized":"2023-10-08T07:13:50.358958+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:29.904246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.281819+0000","last_change":"2023-10-08T07:13:34.294253+0000","last_active":"2023-10-08T07:13:49.281819+0000","last_peered":"2023-10-08T07:13:49.281819+0000","last_clean":"2023-10-08T07:13:49.281819+0000","last_became_active":"2023-10-08T07:13:34.291997+0000","last_became_peered":"2023-10-08T07:13:34.291997+0000","last_unstale":"2023-10-08T07:13:49.281819+0000","last_undegraded":"2023-10-08T07:13:49.281819+0000","last_fullsized":"2023-10-08T07:13:49.281819+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.278736+0000","last_change":"2023-10-08T07:13:49.298924+0000","last_active":"2023-10-08T07:13:50.278736+0000","last_peered":"2023-10-08T07:13:50.278736+0000","last_clean":"2023-10-08T07:13:50.278736+0000","last_became_active":"2023-10-08T07:13:49.298587+0000","last_became_peered":"2023-10-08T07:13:49.298587+0000","last_unstale":"2023-10-08T07:13:50.278736+0000","last_undegraded":"2023-10-08T07:13:50.278736+0000","last_fullsized":"2023-10-08T07:13:50.278736+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:34.367483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287754+0000","last_change":"2023-10-08T07:13:34.308930+0000","last_active":"2023-10-08T07:13:49.287754+0000","last_peered":"2023-10-08T07:13:49.287754+0000","last_clean":"2023-10-08T07:13:49.287754+0000","last_became_active":"2023-10-08T07:13:34.308283+0000","last_became_peered":"2023-10-08T07:13:34.308283+0000","last_unstale":"2023-10-08T07:13:49.287754+0000","last_undegraded":"2023-10-08T07:13:49.287754+0000","last_fullsized":"2023-10-08T07:13:49.287754+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.283832+0000","last_change":"2023-10-08T07:13:49.318881+0000","last_active":"2023-10-08T07:13:50.283832+0000","last_peered":"2023-10-08T07:13:50.283832+0000","last_clean":"2023-10-08T07:13:50.283832+0000","last_became_active":"2023-10-08T07:13:49.318589+0000","last_became_peered":"2023-10-08T07:13:49.318589+0000","last_unstale":"2023-10-08T07:13:50.283832+0000","last_undegraded":"2023-10-08T07:13:50.283832+0000","last_fullsized":"2023-10-08T07:13:50.283832+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:23:54.276903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.314908+0000","last_change":"2023-10-08T07:13:34.295991+0000","last_active":"2023-10-08T07:13:49.314908+0000","last_peered":"2023-10-08T07:13:49.314908+0000","last_clean":"2023-10-08T07:13:49.314908+0000","last_became_active":"2023-10-08T07:13:34.295717+0000","last_became_peered":"2023-10-08T07:13:34.295717+0000","last_unstale":"2023-10-08T07:13:49.314908+0000","last_undegraded":"2023-10-08T07:13:49.314908+0000","last_fullsized":"2023-10-08T07:13:49.314908+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:39.797828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:51.085925+0000","last_change":"2023-10-08T07:13:49.306886+0000","last_active":"2023-10-08T07:13:51.085925+0000","last_peered":"2023-10-08T07:13:51.085925+0000","last_clean":"2023-10-08T07:13:51.085925+0000","last_became_active":"2023-10-08T07:13:49.306317+0000","last_became_peered":"2023-10-08T07:13:49.306317+0000","last_unstale":"2023-10-08T07:13:51.085925+0000","last_undegraded":"2023-10-08T07:13:51.085925+0000","last_fullsized":"2023-10-08T07:13:51.085925+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:21:52.597232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.291913+0000","last_change":"2023-10-08T07:13:34.289375+0000","last_active":"2023-10-08T07:13:49.291913+0000","last_peered":"2023-10-08T07:13:49.291913+0000","last_clean":"2023-10-08T07:13:49.291913+0000","last_became_active":"2023-10-08T07:13:34.289115+0000","last_became_peered":"2023-10-08T07:13:34.289115+0000","last_unstale":"2023-10-08T07:13:49.291913+0000","last_undegraded":"2023-10-08T07:13:49.291913+0000","last_fullsized":"2023-10-08T07:13:49.291913+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:02.118355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.286470+0000","last_change":"2023-10-08T07:13:49.318551+0000","last_active":"2023-10-08T07:13:50.286470+0000","last_peered":"2023-10-08T07:13:50.286470+0000","last_clean":"2023-10-08T07:13:50.286470+0000","last_became_active":"2023-10-08T07:13:49.317986+0000","last_became_peered":"2023-10-08T07:13:49.317986+0000","last_unstale":"2023-10-08T07:13:50.286470+0000","last_undegraded":"2023-10-08T07:13:50.286470+0000","last_fullsized":"2023-10-08T07:13:50.286470+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:18.097796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.285210+0000","last_change":"2023-10-08T07:13:34.302018+0000","last_active":"2023-10-08T07:13:49.285210+0000","last_peered":"2023-10-08T07:13:49.285210+0000","last_clean":"2023-10-08T07:13:49.285210+0000","last_became_active":"2023-10-08T07:13:34.301747+0000","last_became_peered":"2023-10-08T07:13:34.301747+0000","last_unstale":"2023-10-08T07:13:49.285210+0000","last_undegraded":"2023-10-08T07:13:49.285210+0000","last_fullsized":"2023-10-08T07:13:49.285210+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:50.605533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.844613+0000","last_change":"2023-10-08T07:13:49.311917+0000","last_active":"2023-10-08T07:13:50.844613+0000","last_peered":"2023-10-08T07:13:50.844613+0000","last_clean":"2023-10-08T07:13:50.844613+0000","last_became_active":"2023-10-08T07:13:49.310901+0000","last_became_peered":"2023-10-08T07:13:49.310901+0000","last_unstale":"2023-10-08T07:13:50.844613+0000","last_undegraded":"2023-10-08T07:13:50.844613+0000","last_fullsized":"2023-10-08T07:13:50.844613+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:08:31.424602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.290421+0000","last_change":"2023-10-08T07:13:34.293550+0000","last_active":"2023-10-08T07:13:49.290421+0000","last_peered":"2023-10-08T07:13:49.290421+0000","last_clean":"2023-10-08T07:13:49.290421+0000","last_became_active":"2023-10-08T07:13:34.293202+0000","last_became_peered":"2023-10-08T07:13:34.293202+0000","last_unstale":"2023-10-08T07:13:49.290421+0000","last_undegraded":"2023-10-08T07:13:49.290421+0000","last_fullsized":"2023-10-08T07:13:49.290421+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:00.842346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.281807+0000","last_change":"2023-10-08T07:13:49.318338+0000","last_active":"2023-10-08T07:13:50.281807+0000","last_peered":"2023-10-08T07:13:50.281807+0000","last_clean":"2023-10-08T07:13:50.281807+0000","last_became_active":"2023-10-08T07:13:49.317979+0000","last_became_peered":"2023-10-08T07:13:49.317979+0000","last_unstale":"2023-10-08T07:13:50.281807+0000","last_undegraded":"2023-10-08T07:13:50.281807+0000","last_fullsized":"2023-10-08T07:13:50.281807+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:39:03.798412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.293944+0000","last_change":"2023-10-08T07:13:34.298110+0000","last_active":"2023-10-08T07:13:49.293944+0000","last_peered":"2023-10-08T07:13:49.293944+0000","last_clean":"2023-10-08T07:13:49.293944+0000","last_became_active":"2023-10-08T07:13:34.297806+0000","last_became_peered":"2023-10-08T07:13:34.297806+0000","last_unstale":"2023-10-08T07:13:49.293944+0000","last_undegraded":"2023-10-08T07:13:49.293944+0000","last_fullsized":"2023-10-08T07:13:49.293944+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:58.335295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.357781+0000","last_change":"2023-10-08T07:13:49.315887+0000","last_active":"2023-10-08T07:13:50.357781+0000","last_peered":"2023-10-08T07:13:50.357781+0000","last_clean":"2023-10-08T07:13:50.357781+0000","last_became_active":"2023-10-08T07:13:49.315571+0000","last_became_peered":"2023-10-08T07:13:49.315571+0000","last_unstale":"2023-10-08T07:13:50.357781+0000","last_undegraded":"2023-10-08T07:13:50.357781+0000","last_fullsized":"2023-10-08T07:13:50.357781+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:31.253205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.282695+0000","last_change":"2023-10-08T07:13:34.298025+0000","last_active":"2023-10-08T07:13:49.282695+0000","last_peered":"2023-10-08T07:13:49.282695+0000","last_clean":"2023-10-08T07:13:49.282695+0000","last_became_active":"2023-10-08T07:13:34.296181+0000","last_became_peered":"2023-10-08T07:13:34.296181+0000","last_unstale":"2023-10-08T07:13:49.282695+0000","last_undegraded":"2023-10-08T07:13:49.282695+0000","last_fullsized":"2023-10-08T07:13:49.282695+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:37.724720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.448978+0000","last_change":"2023-10-08T07:13:49.300145+0000","last_active":"2023-10-08T07:13:50.448978+0000","last_peered":"2023-10-08T07:13:50.448978+0000","last_clean":"2023-10-08T07:13:50.448978+0000","last_became_active":"2023-10-08T07:13:49.299846+0000","last_became_peered":"2023-10-08T07:13:49.299846+0000","last_unstale":"2023-10-08T07:13:50.448978+0000","last_undegraded":"2023-10-08T07:13:50.448978+0000","last_fullsized":"2023-10-08T07:13:50.448978+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:16.187252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.296189+0000","last_change":"2023-10-08T07:13:34.301326+0000","last_active":"2023-10-08T07:13:49.296189+0000","last_peered":"2023-10-08T07:13:49.296189+0000","last_clean":"2023-10-08T07:13:49.296189+0000","last_became_active":"2023-10-08T07:13:34.299365+0000","last_became_peered":"2023-10-08T07:13:34.299365+0000","last_unstale":"2023-10-08T07:13:49.296189+0000","last_undegraded":"2023-10-08T07:13:49.296189+0000","last_fullsized":"2023-10-08T07:13:49.296189+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:43:02.576748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3f","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.846218+0000","last_change":"2023-10-08T07:13:49.313275+0000","last_active":"2023-10-08T07:13:50.846218+0000","last_peered":"2023-10-08T07:13:50.846218+0000","last_clean":"2023-10-08T07:13:50.846218+0000","last_became_active":"2023-10-08T07:13:49.312535+0000","last_became_peered":"2023-10-08T07:13:49.312535+0000","last_unstale":"2023-10-08T07:13:50.846218+0000","last_undegraded":"2023-10-08T07:13:50.846218+0000","last_fullsized":"2023-10-08T07:13:50.846218+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:49.544492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.287567+0000","last_change":"2023-10-08T07:13:34.306582+0000","last_active":"2023-10-08T07:13:49.287567+0000","last_peered":"2023-10-08T07:13:49.287567+0000","last_clean":"2023-10-08T07:13:49.287567+0000","last_became_active":"2023-10-08T07:13:34.306310+0000","last_became_peered":"2023-10-08T07:13:34.306310+0000","last_unstale":"2023-10-08T07:13:49.287567+0000","last_undegraded":"2023-10-08T07:13:49.287567+0000","last_fullsized":"2023-10-08T07:13:49.287567+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:04:25.686769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:51.085566+0000","last_change":"2023-10-08T07:13:49.307756+0000","last_active":"2023-10-08T07:13:51.085566+0000","last_peered":"2023-10-08T07:13:51.085566+0000","last_clean":"2023-10-08T07:13:51.085566+0000","last_became_active":"2023-10-08T07:13:49.307456+0000","last_became_peered":"2023-10-08T07:13:49.307456+0000","last_unstale":"2023-10-08T07:13:51.085566+0000","last_undegraded":"2023-10-08T07:13:51.085566+0000","last_fullsized":"2023-10-08T07:13:51.085566+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:33.124866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.298001+0000","last_change":"2023-10-08T07:13:34.295062+0000","last_active":"2023-10-08T07:13:49.298001+0000","last_peered":"2023-10-08T07:13:49.298001+0000","last_clean":"2023-10-08T07:13:49.298001+0000","last_became_active":"2023-10-08T07:13:34.294747+0000","last_became_peered":"2023-10-08T07:13:34.294747+0000","last_unstale":"2023-10-08T07:13:49.298001+0000","last_undegraded":"2023-10-08T07:13:49.298001+0000","last_fullsized":"2023-10-08T07:13:49.298001+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:26.407053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":11,"reported_epoch":238,"state":"active+clean","last_fresh":"2023-10-08T07:13:50.358527+0000","last_change":"2023-10-08T07:13:49.304648+0000","last_active":"2023-10-08T07:13:50.358527+0000","last_peered":"2023-10-08T07:13:50.358527+0000","last_clean":"2023-10-08T07:13:50.358527+0000","last_became_active":"2023-10-08T07:13:49.304247+0000","last_became_peered":"2023-10-08T07:13:49.304247+0000","last_unstale":"2023-10-08T07:13:50.358527+0000","last_undegraded":"2023-10-08T07:13:50.358527+0000","last_fullsized":"2023-10-08T07:13:50.358527+0000","mapping_epoch":236,"log_start":"0'0","ondisk_log_start":"0'0","created":236,"last_epoch_clean":237,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:48.335144+0000","last_clean_scrub_stamp":"2023-10-08T07:13:48.335144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:42.456757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3f","version":"0'0","reported_seq":15,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-10-08T07:13:49.297988+0000","last_change":"2023-10-08T07:13:34.308897+0000","last_active":"2023-10-08T07:13:49.297988+0000","last_peered":"2023-10-08T07:13:49.297988+0000","last_clean":"2023-10-08T07:13:49.297988+0000","last_became_active":"2023-10-08T07:13:34.308474+0000","last_became_peered":"2023-10-08T07:13:34.308474+0000","last_unstale":"2023-10-08T07:13:49.297988+0000","last_undegraded":"2023-10-08T07:13:49.297988+0000","last_fullsized":"2023-10-08T07:13:49.297988+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:13:33.693289+0000","last_clean_scrub_stamp":"2023-10-08T07:13:33.693289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:00:06.910913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2205,"num_read_kb":4618,"num_write":589,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2806,"ondisk_log_size":2806,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576012,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65236,"kb_used_data":3520,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306604,"statfs":{"total":96636764160,"available":96569962496,"internally_reserved":0,"allocated":3604480,"data_stored":2148688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.169,"15min":1.2090000000000001},"min":{"1min":0.77100000000000002,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.508,"5min":1.968,"15min":2.008},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.238,"15min":1.2310000000000001},"min":{"1min":0.91300000000000003,"5min":0.747,"15min":0.747},"max":{"1min":1.6830000000000001,"5min":1.7330000000000001,"15min":1.7450000000000001},"last":0.98099999999999998}]},{"osd":1,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.242,"15min":1.252},"min":{"1min":0.94399999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.665,"5min":1.79,"15min":1.998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.341,"15min":1.3500000000000001},"min":{"1min":0.99399999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.8859999999999999,"5min":1.8899999999999999,"15min":2.1019999999999999},"last":1.331}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.29,"15min":1.3140000000000001},"min":{"1min":0.97199999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6180000000000001,"5min":1.696,"15min":2.1640000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.363,"15min":1.351},"min":{"1min":1.0069999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.7270000000000001,"5min":1.8580000000000001,"15min":2.3170000000000002},"last":1.401}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3109999999999999,"15min":1.3380000000000001},"min":{"1min":1.0209999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.663,"5min":1.8160000000000001,"15min":1.9039999999999999},"last":1.47},{"interface":"front","average":{"1min":1.403,"5min":1.399,"15min":1.4119999999999999},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":1.754,"5min":1.8680000000000001,"15min":1.931},"last":1.6539999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3120000000000001,"15min":1.3260000000000001},"min":{"1min":0.86199999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6950000000000001,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.702},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.383,"15min":1.401},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.85199999999999998},"max":{"1min":1.76,"5min":1.8140000000000001,"15min":2.0379999999999998},"last":1.7250000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.4390000000000001,"15min":1.444},"min":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":2.0920000000000001,"15min":2.1139999999999999},"last":1.7629999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.4610000000000001,"15min":1.51},"min":{"1min":1.075,"5min":1,"15min":0.96899999999999997},"max":{"1min":1.8640000000000001,"5min":2.23,"15min":2.23},"last":1.546}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.575,"15min":1.6020000000000001},"min":{"1min":1.361,"5min":1.048,"15min":1.048},"max":{"1min":2.016,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.536,"15min":1.5349999999999999},"min":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.9079999999999999,"5min":2.21,"15min":2.2570000000000001},"last":1.607}]}]},{"osd":6,"up_from":13,"seq":55834576011,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46624,"kb_used_data":2956,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325216,"statfs":{"total":96636764160,"available":96589021184,"internally_reserved":0,"allocated":3026944,"data_stored":1559898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3109999999999999,"15min":1.321},"min":{"1min":0.80000000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":1.794,"15min":1.962},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.347,"15min":1.3620000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.7230000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.5880000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.228,"15min":1.2290000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.2130000000000001,"15min":1.2430000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.97},"last":1.1160000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.198,"15min":1.238},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.484,"5min":2.339,"15min":2.339},"last":1.464},{"interface":"front","average":{"1min":1.302,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":1.0469999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3819999999999999,"15min":1.403},"min":{"1min":1.034,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.637,"5min":1.9319999999999999,"15min":1.95},"last":1.363},{"interface":"front","average":{"1min":1.425,"5min":1.476,"15min":1.4990000000000001},"min":{"1min":0.97199999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.8740000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.681}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3999999999999999,"15min":1.3899999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.881,"5min":2.0579999999999998,"15min":2.0859999999999999},"last":1.649},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4770000000000001,"15min":1.448},"min":{"1min":0.94999999999999996,"5min":0.92000000000000004,"15min":0.57099999999999995},"max":{"1min":1.8080000000000001,"5min":2.484,"15min":2.484},"last":1.738}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.399,"15min":1.423},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.81899999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.403},"last":1.194},{"interface":"front","average":{"1min":1.552,"5min":1.538,"15min":1.5509999999999999},"min":{"1min":1.2829999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":1.8100000000000001,"5min":2.097,"15min":2.6139999999999999},"last":1.532}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5700000000000001,"15min":1.575},"min":{"1min":1.0489999999999999,"5min":1.012,"15min":0.96299999999999997},"max":{"1min":1.9379999999999999,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.7989999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.5620000000000001,"15min":1.5600000000000001},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.2959999999999998,"5min":2.2959999999999998,"15min":2.665},"last":1.417}]}]},{"osd":5,"up_from":13,"seq":55834576011,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60488,"kb_used_data":2940,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311352,"statfs":{"total":96636764160,"available":96574824448,"internally_reserved":0,"allocated":3010560,"data_stored":1556459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.157,"15min":1.208},"min":{"1min":0.86899999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.841},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.2290000000000001,"15min":1.232},"min":{"1min":0.83099999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7030000000000001,"5min":1.8560000000000001,"15min":1.9610000000000001},"last":0.96599999999999997}]},{"osd":1,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2090000000000001,"15min":1.242},"min":{"1min":0.75800000000000001,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.6990000000000001,"5min":1.762,"15min":1.9550000000000001},"last":1.54},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.2869999999999999,"15min":1.3},"min":{"1min":0.90900000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.6879999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.276,"15min":1.3009999999999999},"min":{"1min":0.871,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":2.0870000000000002},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.3149999999999999},"min":{"1min":0.77000000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5840000000000001,"5min":1.877,"15min":1.909},"last":1.1419999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.363,"15min":1.3740000000000001},"min":{"1min":0.79800000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.1499999999999999},"last":1.214},{"interface":"front","average":{"1min":1.458,"5min":1.3740000000000001,"15min":1.431},"min":{"1min":0.67500000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.8100000000000001,"5min":1.9530000000000001,"15min":2.173},"last":1.387}]},{"osd":4,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3220000000000001,"15min":1.3280000000000001},"min":{"1min":0.84399999999999997,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7569999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.556,"5min":1.446,"15min":1.4339999999999999},"min":{"1min":1.107,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.181,"5min":2.181,"15min":2.2240000000000002},"last":1.845}]},{"osd":6,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.4199999999999999,"15min":1.4570000000000001},"min":{"1min":1.0740000000000001,"5min":0.66500000000000004,"15min":0.57499999999999996},"max":{"1min":1.899,"5min":2.077,"15min":2.3759999999999999},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.478,"15min":1.4770000000000001},"min":{"1min":1.157,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0059999999999998,"5min":2.036,"15min":2.036},"last":1.9690000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:21 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.5289999999999999,"15min":1.552},"min":{"1min":1.177,"5min":0.72499999999999998,"15min":0.59999999999999998},"max":{"1min":2.298,"5min":2.298,"15min":2.4169999999999998},"last":2.0339999999999998},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.5840000000000001,"15min":1.6180000000000001},"min":{"1min":1.0089999999999999,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":2.0009999999999999,"5min":2.2320000000000002,"15min":2.3199999999999998},"last":1.591}]}]},{"osd":4,"up_from":13,"seq":55834576011,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47992,"kb_used_data":2924,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323848,"statfs":{"total":96636764160,"available":96587620352,"internally_reserved":0,"allocated":2994176,"data_stored":1555393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2849999999999999,"15min":1.3440000000000001},"min":{"1min":0.64200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.492},{"interface":"front","average":{"1min":1.367,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":0.62,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.9139999999999999,"5min":1.976,"15min":1.976},"last":1.6950000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2629999999999999,"15min":1.2889999999999999},"min":{"1min":0.85599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7450000000000001,"5min":1.976,"15min":1.976},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.208,"15min":1.244},"min":{"1min":0.72499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7330000000000001,"5min":1.8660000000000001,"15min":1.875},"last":1.0800000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2689999999999999,"15min":1.266},"min":{"1min":0.77400000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.145},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.365,"15min":1.369},"min":{"1min":0.99299999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.829}]},{"osd":3,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.2789999999999999,"15min":1.3149999999999999},"min":{"1min":0.88,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.702,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.4410000000000001,"15min":1.4219999999999999},"min":{"1min":0.874,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.1850000000000001},"last":1.2849999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.391},"min":{"1min":1.032,"5min":0.372,"15min":0.372},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0830000000000002},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4239999999999999,"15min":1.3919999999999999},"min":{"1min":0.96799999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.865,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.262}]},{"osd":6,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.5009999999999999,"15min":1.46},"min":{"1min":0.91400000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2530000000000001,"5min":2.2719999999999998,"15min":2.6829999999999998},"last":1.776},{"interface":"front","average":{"1min":1.635,"5min":1.593,"15min":1.5609999999999999},"min":{"1min":1.071,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":2.117,"5min":2.3500000000000001,"15min":2.7429999999999999},"last":1.9339999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:22 2023","interfaces":[{"interface":"back","average":{"1min":1.571,"5min":1.5369999999999999,"15min":1.516},"min":{"1min":1.1220000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0649999999999999,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.681,"15min":1.6439999999999999},"min":{"1min":1.3169999999999999,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.3439999999999999,"5min":2.508,"15min":2.508},"last":1.9730000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576013,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52168,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319672,"statfs":{"total":96636764160,"available":96583344128,"internally_reserved":0,"allocated":3002368,"data_stored":1554869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.67800000000000005,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.087,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.123,"5min":1.1559999999999999,"15min":1.181},"min":{"1min":0.748,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.607,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.70099999999999996}]},{"osd":1,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.417,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.2390000000000001,"15min":1.2869999999999999},"min":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4390000000000001,"5min":2.109,"15min":2.4889999999999999},"last":0.74299999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.123,"15min":1.151},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.667,"5min":2.165,"15min":2.2250000000000001},"last":0.62},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.177,"15min":1.202},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5349999999999999,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2549999999999999,"15min":1.282},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.67500000000000004},"max":{"1min":1.51,"5min":1.857,"15min":2.5379999999999998},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4179999999999999,"15min":1.46},"min":{"1min":0.93600000000000005,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.954,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":0.98599999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3200000000000001,"15min":1.3759999999999999},"min":{"1min":0.97899999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8300000000000001,"5min":1.923,"15min":2.9740000000000002},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.419,"15min":1.4339999999999999},"min":{"1min":0.77800000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8380000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":0.93000000000000005}]},{"osd":6,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4099999999999999,"15min":1.4239999999999999},"min":{"1min":0.91000000000000003,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6339999999999999,"5min":1.976,"15min":2.653},"last":1.02},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4710000000000001,"15min":1.5269999999999999},"min":{"1min":1.0960000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8540000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.0529999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5660000000000001},"min":{"1min":0.88800000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9450000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.5209999999999999},"min":{"1min":1.202,"5min":0.628,"15min":0.628},"max":{"1min":1.7889999999999999,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.2110000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576011,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52164,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319676,"statfs":{"total":96636764160,"available":96583348224,"internally_reserved":0,"allocated":2998272,"data_stored":1555427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.97299999999999998},"min":{"1min":0.55300000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.948},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.054,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.58699999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":0.745}]},{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.96099999999999997,"15min":1.0029999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.198,"5min":1.6279999999999999,"15min":2.044},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.171,"15min":1.1970000000000001},"min":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.677,"5min":1.714,"15min":2.1309999999999998},"last":0.86799999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.1819999999999999,"15min":1.1879999999999999},"min":{"1min":0.68400000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.4770000000000001,"5min":1.78,"15min":1.8420000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.226,"5min":1.2410000000000001,"15min":1.2769999999999999},"min":{"1min":0.66200000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.458,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.165}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3959999999999999,"15min":1.4399999999999999},"min":{"1min":0.89000000000000001,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.821,"5min":1.929,"15min":2.1789999999999998},"last":1.754},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":1.0189999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.6879999999999999,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.577}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3819999999999999,"15min":1.411},"min":{"1min":1.0069999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.603,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":1.083},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.5009999999999999,"15min":1.5349999999999999},"min":{"1min":1.0269999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.946,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.5489999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.3939999999999999,"15min":1.403},"min":{"1min":0.997,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.73,"5min":2.056,"15min":2.1240000000000001},"last":1.472},{"interface":"front","average":{"1min":1.47,"5min":1.4950000000000001,"15min":1.534},"min":{"1min":1.1439999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.778,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":1.7190000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4690000000000001,"15min":1.486},"min":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7609999999999999,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":1.643},{"interface":"front","average":{"1min":1.5,"5min":1.5309999999999999,"15min":1.5860000000000001},"min":{"1min":1.069,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.863,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":1.623}]}]},{"osd":1,"up_from":13,"seq":55834576011,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52152,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319688,"statfs":{"total":96636764160,"available":96583360512,"internally_reserved":0,"allocated":2985984,"data_stored":1554721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.024,"15min":1.014},"min":{"1min":0.81000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.6200000000000001,"5min":1.6579999999999999,"15min":1.7769999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.175,"5min":1.171,"15min":1.224},"min":{"1min":0.88,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":1.79},"last":1.1639999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96299999999999997,"15min":0.97599999999999998},"min":{"1min":0.73399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2809999999999999,"5min":1.641,"15min":1.641},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.155,"15min":1.161},"min":{"1min":0.872,"5min":0.372,"15min":0.372},"max":{"1min":1.4710000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.85399999999999998}]},{"osd":3,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.2430000000000001,"15min":1.264},"min":{"1min":1.052,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5800000000000001,"5min":1.651,"15min":1.7809999999999999},"last":1.419},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.1659999999999999,"15min":1.155},"min":{"1min":0.82599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":1.72,"5min":1.883,"15min":2.7429999999999999},"last":1.1120000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.3340000000000001,"15min":1.3640000000000001},"min":{"1min":1.117,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.7110000000000001,"5min":1.7829999999999999,"15min":2.8460000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.3129999999999999,"15min":1.3220000000000001},"min":{"1min":1.0960000000000001,"5min":0.86099999999999999,"15min":0.86099999999999999},"max":{"1min":1.5449999999999999,"5min":1.6879999999999999,"15min":2.0139999999999998},"last":1.0449999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.347,"15min":1.379},"min":{"1min":0.97299999999999998,"5min":0.92600000000000005,"15min":0.76700000000000002},"max":{"1min":1.6639999999999999,"5min":1.827,"15min":2.0470000000000002},"last":1.298},{"interface":"front","average":{"1min":1.637,"5min":1.546,"15min":1.534},"min":{"1min":1.286,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.2610000000000001},"last":1.522}]},{"osd":6,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5089999999999999,"15min":1.5149999999999999},"min":{"1min":1.2549999999999999,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.694,"5min":2.3199999999999998,"15min":2.9079999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.4870000000000001,"15min":1.4990000000000001},"min":{"1min":1.1679999999999999,"5min":1.046,"15min":1.0309999999999999},"max":{"1min":1.7929999999999999,"5min":1.966,"15min":2.3199999999999998},"last":1.347}]},{"osd":7,"last update":"Sun Oct 8 07:12:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6830000000000001,"5min":1.5780000000000001,"15min":1.556},"min":{"1min":1.2250000000000001,"5min":0.94499999999999995,"15min":0.84599999999999997},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.478},{"interface":"front","average":{"1min":1.6160000000000001,"5min":1.5529999999999999,"15min":1.5720000000000001},"min":{"1min":1.377,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.798,"5min":2.1699999999999999,"15min":2.9590000000000001},"last":1.333}]}]},{"osd":0,"up_from":13,"seq":55834576012,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61048,"kb_used_data":3500,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310792,"statfs":{"total":96636764160,"available":96574251008,"internally_reserved":0,"allocated":3584000,"data_stored":2145179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.95199999999999996,"15min":0.97599999999999998},"min":{"1min":0.48299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.3859999999999999,"5min":1.581,"15min":1.581},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.01,"5min":1.103,"15min":1.109},"min":{"1min":0.49299999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.621,"5min":1.6619999999999999,"15min":1.726},"last":0.874}]},{"osd":2,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1060000000000001,"15min":1.1419999999999999},"min":{"1min":0.54700000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4990000000000001,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0329999999999999,"15min":1.073},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4199999999999999,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":0.93300000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.248,"15min":1.256},"min":{"1min":0.69799999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.546,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.179,"15min":1.1679999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.579,"5min":1.752,"15min":2.0979999999999999},"last":1.4259999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3220000000000001,"15min":1.349},"min":{"1min":0.83399999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.389,"15min":1.4179999999999999},"min":{"1min":0.69699999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.782,"5min":1.972,"15min":2.181},"last":2.1520000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.423,"15min":1.417},"min":{"1min":0.79600000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.732,"5min":1.919,"15min":2.1309999999999998},"last":1.706},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.482,"15min":1.486},"min":{"1min":0.79300000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.966,"5min":2.2170000000000001,"15min":2.4900000000000002},"last":1.9930000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.4530000000000001,"15min":1.4810000000000001},"min":{"1min":0.81299999999999994,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7669999999999999,"5min":2.21,"15min":2.2149999999999999},"last":1.925},{"interface":"front","average":{"1min":1.397,"5min":1.506,"15min":1.5089999999999999},"min":{"1min":0.78000000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.9430000000000001,"5min":2.0880000000000001,"15min":2.355},"last":1.7529999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:13:07 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.5389999999999999,"15min":1.5469999999999999},"min":{"1min":0.83499999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9410000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.5289999999999999,"15min":1.554},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":2.0750000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:13:59.493 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T07:13:59.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:13:59.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.746+0000 7f797064f640 1 Processor -- start 2023-10-08T07:13:59.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.746+0000 7f797064f640 1 -- start start 2023-10-08T07:13:59.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.748+0000 7f797064f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79680ad790 0x7f79680702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.748+0000 7f797064f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f796815e330 0x7f7968070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.748+0000 7f797064f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796806dee0 con 0x7f7968159ac0 2023-10-08T07:13:59.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.748+0000 7f797064f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796806dcf0 con 0x7f796815e330 2023-10-08T07:13:59.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.748+0000 7f797064f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796815d850 con 0x7f79680ad790 2023-10-08T07:13:59.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.749+0000 7f796dbea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79680ad790 0x7f79680702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:59.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.749+0000 7f796ebec640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f796815e330 0x7f7968070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:59.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.749+0000 7f796ebec640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f796815e330 0x7f7968070810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:43082/0 (socket says 172.21.15.70:43082) 2023-10-08T07:13:59.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.749+0000 7f796ebec640 1 -- 172.21.15.70:0/2607457973 learned_addr learned my addr 172.21.15.70:0/2607457973 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:59.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.749+0000 7f796dbea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79680ad790 0x7f79680702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:55338/0 (socket says 172.21.15.70:55338) 2023-10-08T07:13:59.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.750+0000 7f796dbea640 1 -- 172.21.15.70:0/2607457973 >> v2:172.21.15.181:3300/0 conn(0x7f796815e330 msgr2=0x7f7968070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:13:59.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.750+0000 7f796dbea640 1 --2- 172.21.15.70:0/2607457973 >> v2:172.21.15.181:3300/0 conn(0x7f796815e330 0x7f7968070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.750+0000 7f796dbea640 1 -- 172.21.15.70:0/2607457973 >> v1:172.21.15.181:6789/0 conn(0x7f7968159ac0 legacy=0x7f796806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.750+0000 7f796dbea640 1 -- 172.21.15.70:0/2607457973 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796815db30 con 0x7f79680ad790 2023-10-08T07:13:59.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.751+0000 7f796dbea640 1 --2- 172.21.15.70:0/2607457973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79680ad790 0x7f79680702d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f79580097a0 tx=0x7f795802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f16d0b4e966bded server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:59.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.752+0000 7f796d3e9640 1 -- 172.21.15.70:0/2607457973 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795802d040 con 0x7f79680ad790 2023-10-08T07:13:59.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.752+0000 7f796d3e9640 1 -- 172.21.15.70:0/2607457973 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7958031810 con 0x7f79680ad790 2023-10-08T07:13:59.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.752+0000 7f796d3e9640 1 -- 172.21.15.70:0/2607457973 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958031ba0 con 0x7f79680ad790 2023-10-08T07:13:59.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.753+0000 7f797064f640 1 -- 172.21.15.70:0/2607457973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79680ad790 msgr2=0x7f79680702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.753+0000 7f797064f640 1 --2- 172.21.15.70:0/2607457973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79680ad790 0x7f79680702d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f79580097a0 tx=0x7f795802b8b0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.753+0000 7f797064f640 1 -- 172.21.15.70:0/2607457973 shutdown_connections 2023-10-08T07:13:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.753+0000 7f797064f640 1 --2- 172.21.15.70:0/2607457973 >> v2:172.21.15.181:3300/0 conn(0x7f796815e330 0x7f7968070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.753+0000 7f797064f640 1 --2- 172.21.15.70:0/2607457973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79680ad790 0x7f79680702d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.753+0000 7f797064f640 1 -- 172.21.15.70:0/2607457973 >> 172.21.15.70:0/2607457973 conn(0x7f796806a380 msgr2=0x7f79680b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.753+0000 7f797064f640 1 -- 172.21.15.70:0/2607457973 shutdown_connections 2023-10-08T07:13:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.753+0000 7f797064f640 1 -- 172.21.15.70:0/2607457973 wait complete. 2023-10-08T07:13:59.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.755+0000 7f797064f640 1 Processor -- start 2023-10-08T07:13:59.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.755+0000 7f797064f640 1 -- start start 2023-10-08T07:13:59.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f797064f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79680ad790 0x7f79681663f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f797064f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7968159ac0 0x7f7968148460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f797064f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796815e330 0x7f7968157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f797064f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79681573e0 con 0x7f796815e330 2023-10-08T07:13:59.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f797064f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7968153b30 con 0x7f79680ad790 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f797064f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7968153e10 con 0x7f7968159ac0 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796ebec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796815e330 0x7f7968157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796ebec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796815e330 0x7f7968157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49984/0 (socket says 172.21.15.70:49984) 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796ebec640 1 -- 172.21.15.70:0/3289638479 learned_addr learned my addr 172.21.15.70:0/3289638479 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796dbea640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7968159ac0 0x7f7968148460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796e3eb640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79680ad790 0x7f79681663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796ebec640 1 -- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7968159ac0 msgr2=0x7f7968148460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796ebec640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7968159ac0 0x7f7968148460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796ebec640 1 -- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79680ad790 msgr2=0x7f79681663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796ebec640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79680ad790 0x7f79681663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796ebec640 1 -- 172.21.15.70:0/3289638479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7958009400 con 0x7f796815e330 2023-10-08T07:13:59.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796dbea640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7968159ac0 0x7f7968148460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:59.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.756+0000 7f796e3eb640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79680ad790 0x7f79681663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:13:59.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.757+0000 7f796ebec640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796815e330 0x7f7968157ba0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f796000efc0 tx=0x7f796000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.757+0000 7f795effd640 1 -- 172.21.15.70:0/3289638479 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960009b60 con 0x7f796815e330 2023-10-08T07:13:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.757+0000 7f795effd640 1 -- 172.21.15.70:0/3289638479 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796000f040 con 0x7f796815e330 2023-10-08T07:13:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.757+0000 7f795effd640 1 -- 172.21.15.70:0/3289638479 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960003990 con 0x7f796815e330 2023-10-08T07:13:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.757+0000 7f797064f640 1 -- 172.21.15.70:0/3289638479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79681540f0 con 0x7f796815e330 2023-10-08T07:13:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.757+0000 7f797064f640 1 -- 172.21.15.70:0/3289638479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7968166930 con 0x7f796815e330 2023-10-08T07:13:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.758+0000 7f797064f640 1 -- 172.21.15.70:0/3289638479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79681578c0 con 0x7f796815e330 2023-10-08T07:13:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.759+0000 7f795effd640 1 -- 172.21.15.70:0/3289638479 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7960003af0 con 0x7f796815e330 2023-10-08T07:13:59.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.760+0000 7f795effd640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f794c06af90 0x7f794c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:13:59.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.760+0000 7f795effd640 1 -- 172.21.15.70:0/3289638479 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f796008a3c0 con 0x7f796815e330 2023-10-08T07:13:59.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.761+0000 7f796e3eb640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f794c06af90 0x7f794c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:13:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.762+0000 7f796e3eb640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f794c06af90 0x7f794c06d450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7964000f60 tx=0x7f7964009040 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:13:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.765+0000 7f795effd640 1 -- 172.21.15.70:0/3289638479 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79681578c0 con 0x7f796815e330 2023-10-08T07:13:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.991+0000 7f797064f640 1 -- 172.21.15.70:0/3289638479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79681578c0 con 0x7f796815e330 2023-10-08T07:13:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.992+0000 7f795effd640 1 -- 172.21.15.70:0/3289638479 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f79681578c0 con 0x7f796815e330 2023-10-08T07:13:59.993 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:13:59.993 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":114,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:34.170119+0000","modified":"2023-10-08T07:13:37.222153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11097":{"gid":11097,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/180995576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":180995576},{"type":"v1","addr":"172.21.15.70:6834","nonce":180995576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:48.829441+0000","modified":"2023-10-08T07:13:51.877323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8416":{"gid":8416,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-08T07:13:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 114 2023-10-08T07:13:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 -- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f794c06af90 msgr2=0x7f794c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f794c06af90 0x7f794c06d450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7964000f60 tx=0x7f7964009040 comp rx=0 tx=0).stop 2023-10-08T07:13:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 -- 172.21.15.70:0/3289638479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796815e330 msgr2=0x7f7968157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:13:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796815e330 0x7f7968157ba0 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f796000efc0 tx=0x7f796000c490 comp rx=0 tx=0).stop 2023-10-08T07:13:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 -- 172.21.15.70:0/3289638479 shutdown_connections 2023-10-08T07:13:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f794c06af90 0x7f794c06d450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f796815e330 0x7f7968157ba0 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7968159ac0 0x7f7968148460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 --2- 172.21.15.70:0/3289638479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79680ad790 0x7f79681663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:13:59.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 -- 172.21.15.70:0/3289638479 >> 172.21.15.70:0/3289638479 conn(0x7f796806a380 msgr2=0x7f7968155ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:13:59.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 -- 172.21.15.70:0/3289638479 shutdown_connections 2023-10-08T07:13:59.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:13:59.995+0000 7f797064f640 1 -- 172.21.15.70:0/3289638479 wait complete. 2023-10-08T07:14:00.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T07:13:48.829441+0000', 'modified': '2023-10-08T07:13:51.877323+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8416}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8416': {'gid': 8416, 'name': 'd', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.181:6837/4084907359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 4084907359}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 4084907359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T07:14:00.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T07:14:00.418 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T07:14:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.644+0000 7f7e473ff640 1 Processor -- start 2023-10-08T07:14:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.644+0000 7f7e473ff640 1 -- start start 2023-10-08T07:14:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.646+0000 7f7e473ff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e4015a820 0x7f7e400aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.646+0000 7f7e473ff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7e400ca680 0x7f7e400b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.646+0000 7f7e473ff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e400cab60 con 0x7f7e4015ac00 2023-10-08T07:14:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.646+0000 7f7e473ff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e400cb960 con 0x7f7e4015a820 2023-10-08T07:14:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.646+0000 7f7e473ff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e400cbdf0 con 0x7f7e400ca680 2023-10-08T07:14:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.646+0000 7f7e463fd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e4015a820 0x7f7e400aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.646+0000 7f7e463fd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e4015a820 0x7f7e400aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49986/0 (socket says 172.21.15.70:49986) 2023-10-08T07:14:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.646+0000 7f7e463fd640 1 -- 172.21.15.70:0/3979492050 learned_addr learned my addr 172.21.15.70:0/3979492050 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.646+0000 7f7e45bfc640 1 --2- 172.21.15.70:0/3979492050 >> v2:172.21.15.181:3300/0 conn(0x7f7e400ca680 0x7f7e400b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.647+0000 7f7e463fd640 1 -- 172.21.15.70:0/3979492050 >> v2:172.21.15.181:3300/0 conn(0x7f7e400ca680 msgr2=0x7f7e400b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.647+0000 7f7e463fd640 1 --2- 172.21.15.70:0/3979492050 >> v2:172.21.15.181:3300/0 conn(0x7f7e400ca680 0x7f7e400b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.647+0000 7f7e463fd640 1 -- 172.21.15.70:0/3979492050 >> v1:172.21.15.181:6789/0 conn(0x7f7e4015ac00 legacy=0x7f7e400c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.647+0000 7f7e463fd640 1 -- 172.21.15.70:0/3979492050 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e400cb030 con 0x7f7e4015a820 2023-10-08T07:14:00.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.649+0000 7f7e463fd640 1 --2- 172.21.15.70:0/3979492050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4015a820 0x7f7e400aff70 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f7e400718e0 tx=0x7f7e3c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79c183273839fa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:00.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.649+0000 7f7e453fb640 1 -- 172.21.15.70:0/3979492050 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c0317e0 con 0x7f7e4015a820 2023-10-08T07:14:00.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.649+0000 7f7e453fb640 1 -- 172.21.15.70:0/3979492050 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c031940 con 0x7f7e4015a820 2023-10-08T07:14:00.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.649+0000 7f7e453fb640 1 -- 172.21.15.70:0/3979492050 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c031cd0 con 0x7f7e4015a820 2023-10-08T07:14:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.650+0000 7f7e473ff640 1 -- 172.21.15.70:0/3979492050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4015a820 msgr2=0x7f7e400aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.650+0000 7f7e473ff640 1 --2- 172.21.15.70:0/3979492050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4015a820 0x7f7e400aff70 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f7e400718e0 tx=0x7f7e3c02b880 comp rx=0 tx=0).stop 2023-10-08T07:14:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.650+0000 7f7e473ff640 1 -- 172.21.15.70:0/3979492050 shutdown_connections 2023-10-08T07:14:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.650+0000 7f7e473ff640 1 --2- 172.21.15.70:0/3979492050 >> v2:172.21.15.181:3300/0 conn(0x7f7e400ca680 0x7f7e400b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.650+0000 7f7e473ff640 1 --2- 172.21.15.70:0/3979492050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4015a820 0x7f7e400aff70 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.650+0000 7f7e473ff640 1 -- 172.21.15.70:0/3979492050 >> 172.21.15.70:0/3979492050 conn(0x7f7e4006c050 msgr2=0x7f7e4006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.650+0000 7f7e473ff640 1 -- 172.21.15.70:0/3979492050 shutdown_connections 2023-10-08T07:14:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.650+0000 7f7e473ff640 1 -- 172.21.15.70:0/3979492050 wait complete. 2023-10-08T07:14:00.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.652+0000 7f7e473ff640 1 Processor -- start 2023-10-08T07:14:00.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.652+0000 7f7e473ff640 1 -- start start 2023-10-08T07:14:00.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e473ff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e400ca680 0x7f7e40147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e473ff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e4015a820 0x7f7e400b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e473ff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4015ac00 0x7f7e400c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e473ff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e401488d0 con 0x7f7e400ca680 2023-10-08T07:14:00.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e473ff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400c43b0 con 0x7f7e4015ac00 2023-10-08T07:14:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e473ff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400c4690 con 0x7f7e4015a820 2023-10-08T07:14:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e463fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e400ca680 0x7f7e40147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e463fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e400ca680 0x7f7e40147e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50002/0 (socket says 172.21.15.70:50002) 2023-10-08T07:14:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e463fd640 1 -- 172.21.15.70:0/3319063564 learned_addr learned my addr 172.21.15.70:0/3319063564 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e463fd640 1 -- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e4015a820 msgr2=0x7f7e400b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e463fd640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e4015a820 0x7f7e400b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e463fd640 1 -- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4015ac00 msgr2=0x7f7e400c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e463fd640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4015ac00 0x7f7e400c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.653+0000 7f7e463fd640 1 -- 172.21.15.70:0/3319063564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c0094d0 con 0x7f7e400ca680 2023-10-08T07:14:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.654+0000 7f7e463fd640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e400ca680 0x7f7e40147e70 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c000c00 tx=0x7f7e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:00.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.654+0000 7f7e36ffd640 1 -- 172.21.15.70:0/3319063564 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c032880 con 0x7f7e400ca680 2023-10-08T07:14:00.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.654+0000 7f7e36ffd640 1 -- 172.21.15.70:0/3319063564 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3c032a00 con 0x7f7e400ca680 2023-10-08T07:14:00.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.654+0000 7f7e36ffd640 1 -- 172.21.15.70:0/3319063564 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c032db0 con 0x7f7e400ca680 2023-10-08T07:14:00.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.654+0000 7f7e473ff640 1 -- 172.21.15.70:0/3319063564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e400c4970 con 0x7f7e400ca680 2023-10-08T07:14:00.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.654+0000 7f7e473ff640 1 -- 172.21.15.70:0/3319063564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e400c4e90 con 0x7f7e400ca680 2023-10-08T07:14:00.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.656+0000 7f7e36ffd640 1 -- 172.21.15.70:0/3319063564 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7e3c04d3f0 con 0x7f7e400ca680 2023-10-08T07:14:00.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.657+0000 7f7e36ffd640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7e2406af90 0x7f7e2406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.657+0000 7f7e36ffd640 1 -- 172.21.15.70:0/3319063564 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f7e3c0abc00 con 0x7f7e400ca680 2023-10-08T07:14:00.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.658+0000 7f7e4b1f2740 1 Processor -- start 2023-10-08T07:14:00.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.658+0000 7f7e45bfc640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7e2406af90 0x7f7e2406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.658+0000 7f7e4b1f2740 1 -- start start 2023-10-08T07:14:00.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.659+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb6ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.659+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58eb9ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.659+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb6f140 0x55f58eb9efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.659+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58eae2700 con 0x55f58eb6f140 2023-10-08T07:14:00.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.659+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58eadfdf0 con 0x55f58eb9bd00 2023-10-08T07:14:00.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.659+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f58eadb6e0 con 0x55f58eb9c0e0 2023-10-08T07:14:00.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.659+0000 7f7e45bfc640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7e2406af90 0x7f7e2406d450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e30006a90 tx=0x7f7e30006a20 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:00.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e46bfe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb6f140 0x55f58eb9efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e46bfe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb6f140 0x55f58eb9efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50012/0 (socket says 172.21.15.70:50012) 2023-10-08T07:14:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e46bfe640 1 -- 172.21.15.70:0/2797620612 learned_addr learned my addr 172.21.15.70:0/2797620612 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e45bfc640 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58eb9ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e463fd640 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb6ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e46bfe640 1 -- 172.21.15.70:0/2797620612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 msgr2=0x55f58eb9ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e46bfe640 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58eb9ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e46bfe640 1 -- 172.21.15.70:0/2797620612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 msgr2=0x55f58eb6ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e46bfe640 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb6ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e46bfe640 1 -- 172.21.15.70:0/2797620612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f58ea9f910 con 0x55f58eb6f140 2023-10-08T07:14:00.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e463fd640 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb6ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:00.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e45bfc640 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58eb9ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:00.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.660+0000 7f7e46bfe640 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb6f140 0x55f58eb9efd0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f7e38001c90 tx=0x7f7e38002150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e48a6104dfedd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.661+0000 7f7e23fff640 1 -- 172.21.15.70:0/2797620612 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38003d80 con 0x55f58eb6f140 2023-10-08T07:14:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.661+0000 7f7e23fff640 1 -- 172.21.15.70:0/2797620612 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3800c840 con 0x55f58eb6f140 2023-10-08T07:14:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.661+0000 7f7e23fff640 1 -- 172.21.15.70:0/2797620612 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800cbd0 con 0x55f58eb6f140 2023-10-08T07:14:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.661+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2797620612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb6f140 msgr2=0x55f58eb9efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.661+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb6f140 0x55f58eb9efd0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f7e38001c90 tx=0x7f7e38002150 comp rx=0 tx=0).stop 2023-10-08T07:14:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.661+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2797620612 shutdown_connections 2023-10-08T07:14:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.661+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb6f140 0x55f58eb9efd0 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.661+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58eb9ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.661+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/2797620612 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb6ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.661+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2797620612 >> 172.21.15.70:0/2797620612 conn(0x55f58eb70f10 msgr2=0x55f58ebb9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:00.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.662+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2797620612 shutdown_connections 2023-10-08T07:14:00.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.662+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2797620612 wait complete. 2023-10-08T07:14:00.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.662+0000 7f7e4b1f2740 1 -- start start 2023-10-08T07:14:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.663+0000 7f7e4b1f2740 1 Processor -- start 2023-10-08T07:14:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58ebba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58ebb2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ea79970 con 0x55f58ebb25b0 2023-10-08T07:14:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ea79610 con 0x55f58eb9bd00 2023-10-08T07:14:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f58e83ff20 con 0x55f58eb9c0e0 2023-10-08T07:14:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e4b1f2740 10 client.0 _ll_register_callbacks cb 0x55f58eab7610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:14:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e46bfe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58ebb2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e46bfe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58ebb2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50014/0 (socket says 172.21.15.70:50014) 2023-10-08T07:14:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e46bfe640 1 -- 172.21.15.70:0/2906990243 learned_addr learned my addr 172.21.15.70:0/2906990243 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e45bfc640 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58ebba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e463fd640 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e46bfe640 1 -- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 msgr2=0x55f58ebba570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e46bfe640 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58ebba570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e46bfe640 1 -- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 msgr2=0x55f58eb996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e46bfe640 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.664+0000 7f7e46bfe640 1 -- 172.21.15.70:0/2906990243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e38001890 con 0x55f58ebb25b0 2023-10-08T07:14:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.665+0000 7f7e46bfe640 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58ebb2a30 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f7e38001c60 tx=0x7f7e380013d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9e9b591b5873cd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.665+0000 7f7e217fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:14:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.665+0000 7f7e217fa640 1 -- 172.21.15.70:0/2906990243 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800cdc0 con 0x55f58ebb25b0 2023-10-08T07:14:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.665+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2906990243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f58ead46c0 con 0x55f58ebb25b0 2023-10-08T07:14:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.665+0000 7f7e217fa640 1 -- 172.21.15.70:0/2906990243 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f7e380050d0 con 0x55f58ebb25b0 2023-10-08T07:14:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.665+0000 7f7e217fa640 1 -- 172.21.15.70:0/2906990243 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e38005290 con 0x55f58ebb25b0 2023-10-08T07:14:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.665+0000 7f7e4b1f2740 10 client.11412 fetch_fsmap learned FSMap version 114 2023-10-08T07:14:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.665+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2906990243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) v3 -- 0x55f58ead9d60 con 0x55f58ebb25b0 2023-10-08T07:14:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.666+0000 7f7e217fa640 1 -- 172.21.15.70:0/2906990243 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f58ead46c0 con 0x55f58ebb25b0 2023-10-08T07:14:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.666+0000 7f7e217fa640 1 -- 172.21.15.70:0/2906990243 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 114) v1 ==== 3891+0+0 (secure 0 0 0) 0x7f7e380053f0 con 0x55f58ebb25b0 2023-10-08T07:14:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.666+0000 7f7e4b1f2740 10 client.11412 fetch_fsmap finished waiting for FSMap version 114 2023-10-08T07:14:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.666+0000 7f7e4b1f2740 10 client.11412 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T07:14:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.666+0000 7f7e4b1f2740 20 client.11412 populate_metadata read hostname 'smithi070' 2023-10-08T07:14:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.666+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:6836/4084907359,v1:172.21.15.181:6837/4084907359] conn(0x55f58eba0330 0x55f58ebb1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.666+0000 7f7e4b1f2740 4 client.11412 mds_command: new command op to 8416 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T07:14:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.666+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2906990243 --> [v2:172.21.15.181:6836/4084907359,v1:172.21.15.181:6837/4084907359] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f58ead43e0 con 0x55f58eba0330 2023-10-08T07:14:00.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.667+0000 7f7e463fd640 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:6836/4084907359,v1:172.21.15.181:6837/4084907359] conn(0x55f58eba0330 0x55f58ebb1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.668+0000 7f7e463fd640 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:6836/4084907359,v1:172.21.15.181:6837/4084907359] conn(0x55f58eba0330 0x55f58ebb1250 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77db0778ba24d72a server_cookie=299572317ea304c3 in_seq=0 out_seq=0 2023-10-08T07:14:00.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.668+0000 7f7e217fa640 10 client.11412 ms_handle_connect on v2:172.21.15.181:6836/4084907359 2023-10-08T07:14:00.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e217fa640 1 -- 172.21.15.70:0/2906990243 <== mds.0 v2:172.21.15.181:6836/4084907359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f58ead43e0 con 0x55f58eba0330 2023-10-08T07:14:00.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e217fa640 10 client.11412 handle_command_reply: tid=0 2023-10-08T07:14:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 client.11412 shutdown 2023-10-08T07:14:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2906990243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 msgr2=0x55f58ebb2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58ebb2a30 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f7e38001c60 tx=0x7f7e380013d0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2906990243 shutdown_connections 2023-10-08T07:14:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:6836/4084907359,v1:172.21.15.181:6837/4084907359] conn(0x55f58eba0330 0x55f58ebb1250 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58ebb2a30 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58ebba570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/2906990243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e217fa640 0 client.11412 ms_handle_reset on v2:172.21.15.181:6836/4084907359 2023-10-08T07:14:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2906990243 >> 172.21.15.70:0/2906990243 conn(0x55f58eb6f140 msgr2=0x55f58eb9ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2906990243 shutdown_connections 2023-10-08T07:14:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.670+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/2906990243 wait complete. 2023-10-08T07:14:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.671+0000 7f7e4b1f2740 20 client.11412 trim_cache size 0 max 16384 2023-10-08T07:14:00.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.687+0000 7f7e4b1f2740 1 Processor -- start 2023-10-08T07:14:00.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.687+0000 7f7e4b1f2740 1 -- start start 2023-10-08T07:14:00.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.688+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb9b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.688+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58ebb2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.688+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58eb6dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.688+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58eba2970 con 0x55f58ebb25b0 2023-10-08T07:14:00.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.688+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58eb9ec30 con 0x55f58eb9bd00 2023-10-08T07:14:00.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.688+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f58ebb2ed0 con 0x55f58eb9c0e0 2023-10-08T07:14:00.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.688+0000 7f7e46bfe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58eb6dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.688+0000 7f7e46bfe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58eb6dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50028/0 (socket says 172.21.15.70:50028) 2023-10-08T07:14:00.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.688+0000 7f7e46bfe640 1 -- 172.21.15.70:0/1435855441 learned_addr learned my addr 172.21.15.70:0/1435855441 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:00.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.689+0000 7f7e46bfe640 1 -- 172.21.15.70:0/1435855441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 msgr2=0x55f58ebb2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.689+0000 7f7e46bfe640 1 --2- 172.21.15.70:0/1435855441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58ebb2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.689+0000 7f7e46bfe640 1 -- 172.21.15.70:0/1435855441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 msgr2=0x55f58eb9b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.689+0000 7f7e46bfe640 1 --2- 172.21.15.70:0/1435855441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb9b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.689+0000 7f7e46bfe640 1 -- 172.21.15.70:0/1435855441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f58ea78b80 con 0x55f58ebb25b0 2023-10-08T07:14:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.689+0000 7f7e46bfe640 1 --2- 172.21.15.70:0/1435855441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58eb6dba0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f7e38001b30 tx=0x7f7e3801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3329454f68666f42 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.689+0000 7f7e34ff9640 1 -- 172.21.15.70:0/1435855441 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800cdc0 con 0x55f58ebb25b0 2023-10-08T07:14:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.689+0000 7f7e34ff9640 1 -- 172.21.15.70:0/1435855441 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e38006b50 con 0x55f58ebb25b0 2023-10-08T07:14:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.689+0000 7f7e34ff9640 1 -- 172.21.15.70:0/1435855441 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380050d0 con 0x55f58ebb25b0 2023-10-08T07:14:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.690+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/1435855441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 msgr2=0x55f58eb6dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.690+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/1435855441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58eb6dba0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f7e38001b30 tx=0x7f7e3801c000 comp rx=0 tx=0).stop 2023-10-08T07:14:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.690+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/1435855441 shutdown_connections 2023-10-08T07:14:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.690+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/1435855441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58ebb25b0 0x55f58eb6dba0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.690+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/1435855441 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58eb9c0e0 0x55f58ebb2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.690+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/1435855441 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9bd00 0x55f58eb9b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.690+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/1435855441 >> 172.21.15.70:0/1435855441 conn(0x55f58eb6f140 msgr2=0x55f58ebb3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.690+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/1435855441 shutdown_connections 2023-10-08T07:14:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.690+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/1435855441 wait complete. 2023-10-08T07:14:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.691+0000 7f7e4b1f2740 1 -- start start 2023-10-08T07:14:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.691+0000 7f7e4b1f2740 1 Processor -- start 2023-10-08T07:14:00.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.692+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb9bd00 0x55f58eb8cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.692+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9c0e0 0x55f58eb8d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.692+0000 7f7e4b1f2740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58ebb25b0 0x55f58eb739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.692+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ebb14a0 con 0x55f58eb9bd00 2023-10-08T07:14:00.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.692+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f58ec22870 con 0x55f58eb9c0e0 2023-10-08T07:14:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.692+0000 7f7e4b1f2740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f58ebb1c60 con 0x55f58ebb25b0 2023-10-08T07:14:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.692+0000 7f7e4b1f2740 10 client.0 _ll_register_callbacks cb 0x7f7e380014d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:14:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.692+0000 7f7e463fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb9bd00 0x55f58eb8cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.692+0000 7f7e463fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb9bd00 0x55f58eb8cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50040/0 (socket says 172.21.15.70:50040) 2023-10-08T07:14:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.692+0000 7f7e463fd640 1 -- 172.21.15.70:0/3936464815 learned_addr learned my addr 172.21.15.70:0/3936464815 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e463fd640 1 -- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58ebb25b0 msgr2=0x55f58eb739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e46bfe640 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58ebb25b0 0x55f58eb739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e463fd640 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58ebb25b0 0x55f58eb739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e45bfc640 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9c0e0 0x55f58eb8d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e463fd640 1 -- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9c0e0 msgr2=0x55f58eb8d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e463fd640 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9c0e0 0x55f58eb8d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e463fd640 1 -- 172.21.15.70:0/3936464815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f58ead9d60 con 0x55f58eb9bd00 2023-10-08T07:14:00.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e463fd640 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb9bd00 0x55f58eb8cf70 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c079e90 tx=0x7f7e3c07e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6be440810070c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:00.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e217fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:14:00.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e217fa640 1 -- 172.21.15.70:0/3936464815 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c0a9b60 con 0x55f58eb9bd00 2023-10-08T07:14:00.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e217fa640 1 -- 172.21.15.70:0/3936464815 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f7e3c0a9cc0 con 0x55f58eb9bd00 2023-10-08T07:14:00.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.693+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/3936464815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e3800c840 con 0x55f58eb9bd00 2023-10-08T07:14:00.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e217fa640 1 -- 172.21.15.70:0/3936464815 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e3c079680 con 0x55f58eb9bd00 2023-10-08T07:14:00.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e4b1f2740 10 client.11418 fetch_fsmap learned FSMap version 114 2023-10-08T07:14:00.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/3936464815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) v3 -- 0x55f58ea9e170 con 0x55f58eb9bd00 2023-10-08T07:14:00.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e217fa640 1 -- 172.21.15.70:0/3936464815 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3800c840 con 0x55f58eb9bd00 2023-10-08T07:14:00.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e217fa640 1 -- 172.21.15.70:0/3936464815 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 114) v1 ==== 3891+0+0 (secure 0 0 0) 0x7f7e3c082000 con 0x55f58eb9bd00 2023-10-08T07:14:00.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e4b1f2740 10 client.11418 fetch_fsmap finished waiting for FSMap version 114 2023-10-08T07:14:00.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e4b1f2740 10 client.11418 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T07:14:00.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e4b1f2740 20 client.11418 populate_metadata read hostname 'smithi070' 2023-10-08T07:14:00.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:6836/4084907359,v1:172.21.15.181:6837/4084907359] conn(0x55f58ec21aa0 0x55f58ec21e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e4b1f2740 4 client.11418 mds_command: new command op to 8416 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T07:14:00.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.694+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/3936464815 --> [v2:172.21.15.181:6836/4084907359,v1:172.21.15.181:6837/4084907359] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f58e7dd6c0 con 0x55f58ec21aa0 2023-10-08T07:14:00.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.695+0000 7f7e45bfc640 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:6836/4084907359,v1:172.21.15.181:6837/4084907359] conn(0x55f58ec21aa0 0x55f58ec21e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.695+0000 7f7e217fa640 10 client.11418 ms_handle_connect on v2:172.21.15.181:6836/4084907359 2023-10-08T07:14:00.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.695+0000 7f7e45bfc640 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:6836/4084907359,v1:172.21.15.181:6837/4084907359] conn(0x55f58ec21aa0 0x55f58ec21e80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=491f37f01ca74cfa server_cookie=99ecfc4043985594 in_seq=0 out_seq=0 2023-10-08T07:14:00.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e217fa640 1 -- 172.21.15.70:0/3936464815 <== mds.0 v2:172.21.15.181:6836/4084907359 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f58e7dd6c0 con 0x55f58ec21aa0 2023-10-08T07:14:00.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e217fa640 10 client.11418 handle_command_reply: tid=0 2023-10-08T07:14:00.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e4b1f2740 1 client.11418 shutdown 2023-10-08T07:14:00.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/3936464815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb9bd00 msgr2=0x55f58eb8cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb9bd00 0x55f58eb8cf70 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c079e90 tx=0x7f7e3c07e3a0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/3936464815 shutdown_connections 2023-10-08T07:14:00.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:6836/4084907359,v1:172.21.15.181:6837/4084907359] conn(0x55f58ec21aa0 0x55f58ec21e80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55f58ebb25b0 0x55f58eb739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55f58eb9c0e0 0x55f58eb8d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e217fa640 0 client.11418 ms_handle_reset on v2:172.21.15.181:6836/4084907359 2023-10-08T07:14:00.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e4b1f2740 1 --2- 172.21.15.70:0/3936464815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55f58eb9bd00 0x55f58eb8cf70 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.696+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/3936464815 >> 172.21.15.70:0/3936464815 conn(0x55f58eb6f140 msgr2=0x55f58eb6a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:00.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/3936464815 shutdown_connections 2023-10-08T07:14:00.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e4b1f2740 1 -- 172.21.15.70:0/3936464815 wait complete. 2023-10-08T07:14:00.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e4b1f2740 20 client.11418 trim_cache size 0 max 16384 2023-10-08T07:14:00.703 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:14:00.703 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T07:14:00.703 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T07:14:00.703 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 8416, 2023-10-08T07:14:00.703 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T07:14:00.703 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T07:14:00.704 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "backup_fs", 2023-10-08T07:14:00.704 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 11.339934120000001, 2023-10-08T07:14:00.704 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 114, 2023-10-08T07:14:00.704 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 238, 2023-10-08T07:14:00.704 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 238, 2023-10-08T07:14:00.704 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 31.473200055 2023-10-08T07:14:00.704 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:14:00.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e473ff640 1 -- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7e2406af90 msgr2=0x7f7e2406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e473ff640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7e2406af90 0x7f7e2406d450 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7e30006a90 tx=0x7f7e30006a20 comp rx=0 tx=0).stop 2023-10-08T07:14:00.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e473ff640 1 -- 172.21.15.70:0/3319063564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e400ca680 msgr2=0x7f7e40147e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e473ff640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e400ca680 0x7f7e40147e70 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c000c00 tx=0x7f7e3c0046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e473ff640 1 -- 172.21.15.70:0/3319063564 shutdown_connections 2023-10-08T07:14:00.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e473ff640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7e4015ac00 0x7f7e400c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e473ff640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7e4015a820 0x7f7e400b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e473ff640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f7e2406af90 0x7f7e2406d450 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.697+0000 7f7e473ff640 1 --2- 172.21.15.70:0/3319063564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e400ca680 0x7f7e40147e70 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.698+0000 7f7e473ff640 1 -- 172.21.15.70:0/3319063564 >> 172.21.15.70:0/3319063564 conn(0x7f7e4006c050 msgr2=0x7f7e400accc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:00.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.698+0000 7f7e473ff640 1 -- 172.21.15.70:0/3319063564 shutdown_connections 2023-10-08T07:14:00.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.698+0000 7f7e473ff640 1 -- 172.21.15.70:0/3319063564 wait complete. 2023-10-08T07:14:00.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T07:14:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.930+0000 7f2afac61640 1 Processor -- start 2023-10-08T07:14:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.931+0000 7f2afac61640 1 -- start start 2023-10-08T07:14:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.932+0000 7f2afac61640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af40cb130 0x7f2af40cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.932+0000 7f2afac61640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40b1cc0 con 0x7f2af40afbe0 2023-10-08T07:14:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.932+0000 7f2afac61640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40cbad0 con 0x7f2af406dfc0 2023-10-08T07:14:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.932+0000 7f2afac61640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40b2620 con 0x7f2af40cb130 2023-10-08T07:14:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.932+0000 7f2af89fd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2af40afbe0 0x7f2af40b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47762/0 (socket says 172.21.15.70:47762) 2023-10-08T07:14:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.932+0000 7f2af89fd640 1 -- 172.21.15.70:0/3245263325 learned_addr learned my addr 172.21.15.70:0/3245263325 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af91fe640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af40cb130 0x7f2af40cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270790065 0 0) 0x7f2af40b1cc0 con 0x7f2af40afbe0 2023-10-08T07:14:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2adc003520 con 0x7f2af40afbe0 2023-10-08T07:14:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3390819358 0 0) 0x7f2adc003520 con 0x7f2af40afbe0 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af40cb130 msgr2=0x7f2af40cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af40cb130 0x7f2af40cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 >> v1:172.21.15.181:6789/0 conn(0x7f2af406dfc0 legacy=0x7f2af4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40b2060 con 0x7f2af40afbe0 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2ae40019f0 con 0x7f2af40afbe0 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 >> v1:172.21.15.70:6789/0 conn(0x7f2af40afbe0 legacy=0x7f2af40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc0039a0 0x7f2adc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af40cb130 0x7f2adc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2adc004d00 0x7f2adc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40b2620 con 0x7f2adc0039a0 2023-10-08T07:14:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc00bae0 con 0x7f2adc004d00 2023-10-08T07:14:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.933+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc00bca0 con 0x7f2af40cb130 2023-10-08T07:14:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.934+0000 7f2af89fd640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc0039a0 0x7f2adc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.934+0000 7f2af91fe640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2adc004d00 0x7f2adc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.934+0000 7f2af3fff640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af40cb130 0x7f2adc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.934+0000 7f2af89fd640 1 -- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af40cb130 msgr2=0x7f2adc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.934+0000 7f2af89fd640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af40cb130 0x7f2adc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.934+0000 7f2af89fd640 1 -- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2adc004d00 msgr2=0x7f2adc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.934+0000 7f2af89fd640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2adc004d00 0x7f2adc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.934+0000 7f2af89fd640 1 -- 172.21.15.70:0/3245263325 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc003520 con 0x7f2adc0039a0 2023-10-08T07:14:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.934+0000 7f2af3fff640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af40cb130 0x7f2adc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.934+0000 7f2af91fe640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2adc004d00 0x7f2adc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.935+0000 7f2af89fd640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc0039a0 0x7f2adc003da0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400a830 tx=0x7f2adc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf971d21d59a3741 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.935+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4034630 con 0x7f2adc0039a0 2023-10-08T07:14:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.935+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae402f070 con 0x7f2adc0039a0 2023-10-08T07:14:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.935+0000 7f2af37fe640 1 -- 172.21.15.70:0/3245263325 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40349f0 con 0x7f2adc0039a0 2023-10-08T07:14:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.936+0000 7f2afac61640 1 -- 172.21.15.70:0/3245263325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc0039a0 msgr2=0x7f2adc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.936+0000 7f2afac61640 1 --2- 172.21.15.70:0/3245263325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc0039a0 0x7f2adc003da0 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400a830 tx=0x7f2adc0057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.936+0000 7f2af91fe640 1 -- 172.21.15.70:0/3245263325 reap_dead start 2023-10-08T07:14:00.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.936+0000 7f2afac61640 1 -- 172.21.15.70:0/3245263325 shutdown_connections 2023-10-08T07:14:00.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.936+0000 7f2afac61640 1 -- 172.21.15.70:0/3245263325 >> 172.21.15.70:0/3245263325 conn(0x7f2af406a5d0 msgr2=0x7f2af40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:00.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.936+0000 7f2afac61640 1 -- 172.21.15.70:0/3245263325 shutdown_connections 2023-10-08T07:14:00.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.936+0000 7f2afac61640 1 -- 172.21.15.70:0/3245263325 wait complete. 2023-10-08T07:14:00.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.938+0000 7f2afac61640 1 Processor -- start 2023-10-08T07:14:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.938+0000 7f2afac61640 1 -- start start 2023-10-08T07:14:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.939+0000 7f2afac61640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2adc0039a0 0x7f2af41534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.939+0000 7f2afac61640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af4157420 0x7f2af4152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.939+0000 7f2afac61640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af415b510 0x7f2af40c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.939+0000 7f2afac61640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af415be90 con 0x7f2af4157420 2023-10-08T07:14:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.939+0000 7f2afac61640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af415c170 con 0x7f2adc0039a0 2023-10-08T07:14:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.939+0000 7f2afac61640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af415c450 con 0x7f2af415b510 2023-10-08T07:14:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.939+0000 7f2af3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af4157420 0x7f2af4152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.939+0000 7f2af3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af4157420 0x7f2af4152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50060/0 (socket says 172.21.15.70:50060) 2023-10-08T07:14:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.939+0000 7f2af3fff640 1 -- 172.21.15.70:0/2885728654 learned_addr learned my addr 172.21.15.70:0/2885728654 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2af3fff640 1 -- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af415b510 msgr2=0x7f2af40c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2af3fff640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af415b510 0x7f2af40c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2af89fd640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2adc0039a0 0x7f2af41534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2af3fff640 1 -- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2adc0039a0 msgr2=0x7f2af41534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2af3fff640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2adc0039a0 0x7f2af41534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2af3fff640 1 -- 172.21.15.70:0/2885728654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af415c730 con 0x7f2af4157420 2023-10-08T07:14:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2af3fff640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af4157420 0x7f2af4152e40 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80045f0 tx=0x7f2ae800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2af17fa640 1 -- 172.21.15.70:0/2885728654 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8007220 con 0x7f2af4157420 2023-10-08T07:14:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2af17fa640 1 -- 172.21.15.70:0/2885728654 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae80073a0 con 0x7f2af4157420 2023-10-08T07:14:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2afac61640 1 -- 172.21.15.70:0/2885728654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af415ba10 con 0x7f2af4157420 2023-10-08T07:14:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.940+0000 7f2af17fa640 1 -- 172.21.15.70:0/2885728654 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80067d0 con 0x7f2af4157420 2023-10-08T07:14:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.941+0000 7f2afac61640 1 -- 172.21.15.70:0/2885728654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af415ce30 con 0x7f2af4157420 2023-10-08T07:14:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.942+0000 7f2afac61640 1 -- 172.21.15.70:0/2885728654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af415c970 con 0x7f2af4157420 2023-10-08T07:14:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.943+0000 7f2af17fa640 1 -- 172.21.15.70:0/2885728654 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2ae8006930 con 0x7f2af4157420 2023-10-08T07:14:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.944+0000 7f2af17fa640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2ae006af90 0x7f2ae006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.944+0000 7f2af17fa640 1 -- 172.21.15.70:0/2885728654 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f2ae8089c50 con 0x7f2af4157420 2023-10-08T07:14:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.944+0000 7f2af89fd640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2ae006af90 0x7f2ae006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.946+0000 7f2af89fd640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2ae006af90 0x7f2ae006d450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4034000 tx=0x7f2ae402f000 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:00.948+0000 7f2af17fa640 1 -- 172.21.15.70:0/2885728654 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2af415c970 con 0x7f2af4157420 2023-10-08T07:14:01.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.164+0000 7f2afac61640 1 -- 172.21.15.70:0/2885728654 --> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2af4070350 con 0x7f2ae006af90 2023-10-08T07:14:01.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.168+0000 7f2af17fa640 1 -- 172.21.15.70:0/2885728654 <== mgr.7328 v2:172.21.15.181:6832/3993370366 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f2ae4001a90 con 0x7f2ae006af90 2023-10-08T07:14:01.168 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T07:14:01.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:01.169 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T07:14:01.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:01.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:01.169 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "backup_fs" 2023-10-08T07:14:01.169 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:01.169 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T07:14:01.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.170+0000 7f2afac61640 1 -- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2ae006af90 msgr2=0x7f2ae006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:01.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.170+0000 7f2afac61640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2ae006af90 0x7f2ae006d450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4034000 tx=0x7f2ae402f000 comp rx=0 tx=0).stop 2023-10-08T07:14:01.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.170+0000 7f2afac61640 1 -- 172.21.15.70:0/2885728654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af4157420 msgr2=0x7f2af4152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:01.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.170+0000 7f2afac61640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af4157420 0x7f2af4152e40 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80045f0 tx=0x7f2ae800da20 comp rx=0 tx=0).stop 2023-10-08T07:14:01.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.170+0000 7f2afac61640 1 -- 172.21.15.70:0/2885728654 shutdown_connections 2023-10-08T07:14:01.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.170+0000 7f2afac61640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2af415b510 0x7f2af40c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.170+0000 7f2afac61640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f2ae006af90 0x7f2ae006d450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.170+0000 7f2afac61640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af4157420 0x7f2af4152e40 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.170+0000 7f2afac61640 1 --2- 172.21.15.70:0/2885728654 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2adc0039a0 0x7f2af41534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.170+0000 7f2afac61640 1 -- 172.21.15.70:0/2885728654 >> 172.21.15.70:0/2885728654 conn(0x7f2af406a5d0 msgr2=0x7f2af40cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:01.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.171+0000 7f2afac61640 1 -- 172.21.15.70:0/2885728654 shutdown_connections 2023-10-08T07:14:01.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.171+0000 7f2afac61640 1 -- 172.21.15.70:0/2885728654 wait complete. 2023-10-08T07:14:01.192 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T07:14:01.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.403+0000 7f3ea193c640 1 Processor -- start 2023-10-08T07:14:01.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.404+0000 7f3ea193c640 1 -- start start 2023-10-08T07:14:01.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.405+0000 7f3ea193c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3e9c15a820 0x7f3e9c160750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:01.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.405+0000 7f3ea193c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c0c7ad0 con 0x7f3e9c0c9080 2023-10-08T07:14:01.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.406+0000 7f3ea193c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c0cb210 con 0x7f3e9c071260 2023-10-08T07:14:01.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.406+0000 7f3ea193c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c15ac00 con 0x7f3e9c15a820 2023-10-08T07:14:01.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.406+0000 7f3e93fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3e9c0c9080 0x7f3e9c070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47768/0 (socket says 172.21.15.70:47768) 2023-10-08T07:14:01.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.406+0000 7f3e93fff640 1 -- 172.21.15.70:0/3693174356 learned_addr learned my addr 172.21.15.70:0/3693174356 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:01.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.406+0000 7f3e9affd640 1 --2- 172.21.15.70:0/3693174356 >> v2:172.21.15.181:3300/0 conn(0x7f3e9c15a820 0x7f3e9c160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:01.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.406+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579186503 0 0) 0x7f3e9c0c7ad0 con 0x7f3e9c0c9080 2023-10-08T07:14:01.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.406+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e88003520 con 0x7f3e9c0c9080 2023-10-08T07:14:01.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 272846155 0 0) 0x7f3e88003520 con 0x7f3e9c0c9080 2023-10-08T07:14:01.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 >> v2:172.21.15.181:3300/0 conn(0x7f3e9c15a820 msgr2=0x7f3e9c160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:01.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 --2- 172.21.15.70:0/3693174356 >> v2:172.21.15.181:3300/0 conn(0x7f3e9c15a820 0x7f3e9c160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 >> v1:172.21.15.181:6789/0 conn(0x7f3e9c071260 legacy=0x7f3e9c0c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:01.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c0cb4f0 con 0x7f3e9c0c9080 2023-10-08T07:14:01.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3e84001670 con 0x7f3e9c0c9080 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 >> v1:172.21.15.70:6789/0 conn(0x7f3e9c0c9080 legacy=0x7f3e9c070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e880039c0 0x7f3e88003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e880047c0 0x7f3e88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e88005180 0x7f3e8800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c15ac00 con 0x7f3e880039c0 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8800c080 con 0x7f3e88005180 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8800c240 con 0x7f3e880047c0 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.407+0000 7f3e9affd640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e880039c0 0x7f3e88003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.408+0000 7f3e93fff640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e880047c0 0x7f3e88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.408+0000 7f3e9b7fe640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e88005180 0x7f3e8800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.408+0000 7f3e9affd640 1 -- 172.21.15.70:0/3693174356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e880047c0 msgr2=0x7f3e88004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.408+0000 7f3e9affd640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e880047c0 0x7f3e88004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.408+0000 7f3e9affd640 1 -- 172.21.15.70:0/3693174356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e88005180 msgr2=0x7f3e8800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:01.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.408+0000 7f3e9affd640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e88005180 0x7f3e8800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.408+0000 7f3e9affd640 1 -- 172.21.15.70:0/3693174356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e88003520 con 0x7f3e880039c0 2023-10-08T07:14:01.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.408+0000 7f3e9b7fe640 1 -- 172.21.15.70:0/3693174356 reap_dead start 2023-10-08T07:14:01.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.409+0000 7f3e9affd640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e880039c0 0x7f3e88003de0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009ac0 tx=0x7f3e880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c099a0a449aa76b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:01.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.410+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c02c6b0 con 0x7f3e880039c0 2023-10-08T07:14:01.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.410+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c02c810 con 0x7f3e880039c0 2023-10-08T07:14:01.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.410+0000 7f3e9a7fc640 1 -- 172.21.15.70:0/3693174356 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c02cba0 con 0x7f3e880039c0 2023-10-08T07:14:01.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.410+0000 7f3ea193c640 1 -- 172.21.15.70:0/3693174356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e880039c0 msgr2=0x7f3e88003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:01.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.410+0000 7f3ea193c640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e880039c0 0x7f3e88003de0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009ac0 tx=0x7f3e880057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.411+0000 7f3ea193c640 1 -- 172.21.15.70:0/3693174356 shutdown_connections 2023-10-08T07:14:01.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.411+0000 7f3ea193c640 1 --2- 172.21.15.70:0/3693174356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e880039c0 0x7f3e88003de0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.411+0000 7f3ea193c640 1 -- 172.21.15.70:0/3693174356 >> 172.21.15.70:0/3693174356 conn(0x7f3e9c069d50 msgr2=0x7f3e9c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:01.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.411+0000 7f3ea193c640 1 -- 172.21.15.70:0/3693174356 shutdown_connections 2023-10-08T07:14:01.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.411+0000 7f3ea193c640 1 -- 172.21.15.70:0/3693174356 wait complete. 2023-10-08T07:14:01.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.413+0000 7f3ea193c640 1 Processor -- start 2023-10-08T07:14:01.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.413+0000 7f3ea193c640 1 -- start start 2023-10-08T07:14:01.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3ea193c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e880039c0 0x7f3e9c15d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:01.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3ea193c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e9c15d6e0 0x7f3e9c1ed600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:01.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3ea193c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c1fa520 0x7f3e9c1f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:01.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3ea193c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c1f7060 con 0x7f3e9c1fa520 2023-10-08T07:14:01.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3ea193c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c1faa70 con 0x7f3e9c15d6e0 2023-10-08T07:14:01.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3ea193c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c1fad50 con 0x7f3e880039c0 2023-10-08T07:14:01.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3e9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c1fa520 0x7f3e9c1f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:01.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3e9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c1fa520 0x7f3e9c1f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50084/0 (socket says 172.21.15.70:50084) 2023-10-08T07:14:01.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3e9b7fe640 1 -- 172.21.15.70:0/355178897 learned_addr learned my addr 172.21.15.70:0/355178897 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:01.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3e9affd640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e880039c0 0x7f3e9c15d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:01.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3e9b7fe640 1 -- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e880039c0 msgr2=0x7f3e9c15d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:01.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3e9b7fe640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e880039c0 0x7f3e9c15d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3e9b7fe640 1 -- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e9c15d6e0 msgr2=0x7f3e9c1ed600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:01.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3e9b7fe640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e9c15d6e0 0x7f3e9c1ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:01.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.414+0000 7f3e9b7fe640 1 -- 172.21.15.70:0/355178897 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c1fb030 con 0x7f3e9c1fa520 2023-10-08T07:14:01.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.415+0000 7f3e9b7fe640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c1fa520 0x7f3e9c1f74f0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f3e94007fd0 tx=0x7f3e9400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:01.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.415+0000 7f3e98ff9640 1 -- 172.21.15.70:0/355178897 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94006e20 con 0x7f3e9c1fa520 2023-10-08T07:14:01.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.415+0000 7f3e98ff9640 1 -- 172.21.15.70:0/355178897 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e94003970 con 0x7f3e9c1fa520 2023-10-08T07:14:01.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.415+0000 7f3e98ff9640 1 -- 172.21.15.70:0/355178897 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94003d20 con 0x7f3e9c1fa520 2023-10-08T07:14:01.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.415+0000 7f3ea193c640 1 -- 172.21.15.70:0/355178897 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c0b3cd0 con 0x7f3e9c1fa520 2023-10-08T07:14:01.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.415+0000 7f3ea193c640 1 -- 172.21.15.70:0/355178897 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c1fb310 con 0x7f3e9c1fa520 2023-10-08T07:14:01.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.416+0000 7f3ea193c640 1 -- 172.21.15.70:0/355178897 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c0cadf0 con 0x7f3e9c1fa520 2023-10-08T07:14:01.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.418+0000 7f3e98ff9640 1 -- 172.21.15.70:0/355178897 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 128) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f3e94005070 con 0x7f3e9c1fa520 2023-10-08T07:14:01.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.419+0000 7f3e98ff9640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3e7c06af90 0x7f3e7c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:01.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.419+0000 7f3e98ff9640 1 -- 172.21.15.70:0/355178897 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f3e94089450 con 0x7f3e9c1fa520 2023-10-08T07:14:01.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.420+0000 7f3e9affd640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3e7c06af90 0x7f3e7c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:01.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.421+0000 7f3e9affd640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3e7c06af90 0x7f3e7c06d450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c036000 tx=0x7f3e8c030000 comp rx=0 tx=0).ready entity=mgr.7328 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:01.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.424+0000 7f3e98ff9640 1 -- 172.21.15.70:0/355178897 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e94053ec0 con 0x7f3e9c1fa520 2023-10-08T07:14:01.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:01.668+0000 7f3ea193c640 1 -- 172.21.15.70:0/355178897 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f3e9c066780 con 0x7f3e9c1fa520 2023-10-08T07:14:02.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.457+0000 7f3e98ff9640 1 -- 172.21.15.70:0/355178897 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v129) v1 ==== 92+0+0 (secure 0 0 0) 0x7f3e940574e0 con 0x7f3e9c1fa520 2023-10-08T07:14:02.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.460+0000 7f3ea193c640 1 -- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3e7c06af90 msgr2=0x7f3e7c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:02.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.460+0000 7f3ea193c640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3e7c06af90 0x7f3e7c06d450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c036000 tx=0x7f3e8c030000 comp rx=0 tx=0).stop 2023-10-08T07:14:02.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.460+0000 7f3ea193c640 1 -- 172.21.15.70:0/355178897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c1fa520 msgr2=0x7f3e9c1f74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:02.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.460+0000 7f3ea193c640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c1fa520 0x7f3e9c1f74f0 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f3e94007fd0 tx=0x7f3e9400e940 comp rx=0 tx=0).stop 2023-10-08T07:14:02.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.460+0000 7f3ea193c640 1 -- 172.21.15.70:0/355178897 shutdown_connections 2023-10-08T07:14:02.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.461+0000 7f3ea193c640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3e7c06af90 0x7f3e7c06d450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.461+0000 7f3ea193c640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e9c1fa520 0x7f3e9c1f74f0 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.461+0000 7f3ea193c640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3e9c15d6e0 0x7f3e9c1ed600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.461+0000 7f3ea193c640 1 --2- 172.21.15.70:0/355178897 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3e880039c0 0x7f3e9c15d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.461+0000 7f3ea193c640 1 -- 172.21.15.70:0/355178897 >> 172.21.15.70:0/355178897 conn(0x7f3e9c069d50 msgr2=0x7f3e9c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:02.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.461+0000 7f3ea193c640 1 -- 172.21.15.70:0/355178897 shutdown_connections 2023-10-08T07:14:02.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.462+0000 7f3ea193c640 1 -- 172.21.15.70:0/355178897 wait complete. 2023-10-08T07:14:02.489 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T07:14:02.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.686+0000 7fc34f34c640 1 Processor -- start 2023-10-08T07:14:02.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.686+0000 7fc34f34c640 1 -- start start 2023-10-08T07:14:02.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.688+0000 7fc34f34c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc3480cbd40 0x7fc3480afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:02.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.688+0000 7fc34f34c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc3480ce650 0x7fc3480ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:02.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.688+0000 7fc34f34c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3480b00d0 con 0x7fc348070f30 2023-10-08T07:14:02.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.688+0000 7fc34f34c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3480cf040 con 0x7fc3480cbd40 2023-10-08T07:14:02.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.688+0000 7fc34f34c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3480714a0 con 0x7fc3480ce650 2023-10-08T07:14:02.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.688+0000 7fc34d0e8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc3480cbd40 0x7fc3480afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:02.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.688+0000 7fc34d0e8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc3480cbd40 0x7fc3480afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50108/0 (socket says 172.21.15.70:50108) 2023-10-08T07:14:02.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.688+0000 7fc34d0e8640 1 -- 172.21.15.70:0/478041552 learned_addr learned my addr 172.21.15.70:0/478041552 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:02.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.689+0000 7fc34d8e9640 1 --2- 172.21.15.70:0/478041552 >> v2:172.21.15.181:3300/0 conn(0x7fc3480ce650 0x7fc3480ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:02.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.689+0000 7fc34d0e8640 1 -- 172.21.15.70:0/478041552 >> v2:172.21.15.181:3300/0 conn(0x7fc3480ce650 msgr2=0x7fc3480ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:02.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.689+0000 7fc34d0e8640 1 --2- 172.21.15.70:0/478041552 >> v2:172.21.15.181:3300/0 conn(0x7fc3480ce650 0x7fc3480ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.689+0000 7fc34d0e8640 1 -- 172.21.15.70:0/478041552 >> v1:172.21.15.70:6789/0 conn(0x7fc348070f30 legacy=0x7fc34816e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:02.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.689+0000 7fc34d0e8640 1 -- 172.21.15.70:0/478041552 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3480cb260 con 0x7fc3480cbd40 2023-10-08T07:14:02.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.689+0000 7fc33ffff640 1 -- 172.21.15.70:0/478041552 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997512190 0 0) 0x7fc3480b00d0 con 0x7fc348070f30 2023-10-08T07:14:02.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.690+0000 7fc34d0e8640 1 --2- 172.21.15.70:0/478041552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 0x7fc3480afb90 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fc3480cf810 tx=0x7fc34402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f235fd73a467364 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:02.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.690+0000 7fc33ffff640 1 -- 172.21.15.70:0/478041552 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34402f850 con 0x7fc3480cbd40 2023-10-08T07:14:02.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.690+0000 7fc33ffff640 1 -- 172.21.15.70:0/478041552 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34402f9b0 con 0x7fc3480cbd40 2023-10-08T07:14:02.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.690+0000 7fc33ffff640 1 -- 172.21.15.70:0/478041552 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34402fd40 con 0x7fc3480cbd40 2023-10-08T07:14:02.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.691+0000 7fc34f34c640 1 -- 172.21.15.70:0/478041552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 msgr2=0x7fc3480afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:02.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.691+0000 7fc34f34c640 1 --2- 172.21.15.70:0/478041552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 0x7fc3480afb90 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7fc3480cf810 tx=0x7fc34402ba30 comp rx=0 tx=0).stop 2023-10-08T07:14:02.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.691+0000 7fc34f34c640 1 -- 172.21.15.70:0/478041552 shutdown_connections 2023-10-08T07:14:02.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.691+0000 7fc34f34c640 1 --2- 172.21.15.70:0/478041552 >> v2:172.21.15.181:3300/0 conn(0x7fc3480ce650 0x7fc3480ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.691+0000 7fc34f34c640 1 --2- 172.21.15.70:0/478041552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 0x7fc3480afb90 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.691+0000 7fc34f34c640 1 -- 172.21.15.70:0/478041552 >> 172.21.15.70:0/478041552 conn(0x7fc3480660e0 msgr2=0x7fc348069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:02.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.691+0000 7fc34f34c640 1 -- 172.21.15.70:0/478041552 shutdown_connections 2023-10-08T07:14:02.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.691+0000 7fc34f34c640 1 -- 172.21.15.70:0/478041552 wait complete. 2023-10-08T07:14:02.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.693+0000 7fc34f34c640 1 Processor -- start 2023-10-08T07:14:02.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.693+0000 7fc34f34c640 1 -- start start 2023-10-08T07:14:02.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.694+0000 7fc34f34c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc348070f30 0x7fc3480b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:02.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.694+0000 7fc34f34c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 0x7fc3480ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:02.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.694+0000 7fc34f34c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3480ce650 0x7fc3480babd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:02.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.694+0000 7fc34f34c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480c59b0 con 0x7fc3480cbd40 2023-10-08T07:14:02.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.694+0000 7fc34f34c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480c5c90 con 0x7fc3480ce650 2023-10-08T07:14:02.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.694+0000 7fc34f34c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3480c7da0 con 0x7fc348070f30 2023-10-08T07:14:02.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.694+0000 7fc34c8e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 0x7fc3480ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:02.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.694+0000 7fc34c8e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 0x7fc3480ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50110/0 (socket says 172.21.15.70:50110) 2023-10-08T07:14:02.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.694+0000 7fc34c8e7640 1 -- 172.21.15.70:0/2359450040 learned_addr learned my addr 172.21.15.70:0/2359450040 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:02.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc34d8e9640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3480ce650 0x7fc3480babd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:02.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc34d0e8640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc348070f30 0x7fc3480b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:02.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc34c8e7640 1 -- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc348070f30 msgr2=0x7fc3480b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:02.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc34c8e7640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc348070f30 0x7fc3480b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc34c8e7640 1 -- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3480ce650 msgr2=0x7fc3480babd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:02.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc34c8e7640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3480ce650 0x7fc3480babd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc34c8e7640 1 -- 172.21.15.70:0/2359450040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3440094d0 con 0x7fc3480cbd40 2023-10-08T07:14:02.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc34c8e7640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 0x7fc3480ccbe0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c410 tx=0x7fc33800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:02.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc33dffb640 1 -- 172.21.15.70:0/2359450040 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3380035f0 con 0x7fc3480cbd40 2023-10-08T07:14:02.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc33dffb640 1 -- 172.21.15.70:0/2359450040 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc338003770 con 0x7fc3480cbd40 2023-10-08T07:14:02.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc34f34c640 1 -- 172.21.15.70:0/2359450040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3480c8080 con 0x7fc3480cbd40 2023-10-08T07:14:02.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.695+0000 7fc33dffb640 1 -- 172.21.15.70:0/2359450040 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338003b20 con 0x7fc3480cbd40 2023-10-08T07:14:02.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.696+0000 7fc34f34c640 1 -- 172.21.15.70:0/2359450040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3480cd120 con 0x7fc3480cbd40 2023-10-08T07:14:02.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.697+0000 7fc34f34c640 1 -- 172.21.15.70:0/2359450040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3480cae60 con 0x7fc3480cbd40 2023-10-08T07:14:02.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.698+0000 7fc33dffb640 1 -- 172.21.15.70:0/2359450040 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 129) v1 ==== 89348+0+0 (secure 0 0 0) 0x7fc338003c80 con 0x7fc3480cbd40 2023-10-08T07:14:02.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.699+0000 7fc33dffb640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc32c06afe0 0x7fc32c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:02.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.699+0000 7fc33dffb640 1 -- 172.21.15.70:0/2359450040 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fc338089340 con 0x7fc3480cbd40 2023-10-08T07:14:02.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.699+0000 7fc34d0e8640 1 -- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc32c06afe0 msgr2=0x7fc32c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3993370366 2023-10-08T07:14:02.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.699+0000 7fc34d0e8640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc32c06afe0 0x7fc32c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:02.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.703+0000 7fc33dffb640 1 -- 172.21.15.70:0/2359450040 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc338053dc0 con 0x7fc3480cbd40 2023-10-08T07:14:02.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.896+0000 7fc34f34c640 1 -- 172.21.15.70:0/2359450040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc34804e160 con 0x7fc3480cbd40 2023-10-08T07:14:02.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.900+0000 7fc34d0e8640 1 -- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc32c06afe0 msgr2=0x7fc32c06d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3993370366 2023-10-08T07:14:02.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.900+0000 7fc34d0e8640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc32c06afe0 0x7fc32c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:14:02.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.901+0000 7fc33dffb640 1 -- 172.21.15.70:0/2359450040 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v77)=0 v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc3380573e0 con 0x7fc3480cbd40 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.903+0000 7fc34f34c640 1 -- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc32c06afe0 msgr2=0x7fc32c06d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.903+0000 7fc34f34c640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc32c06afe0 0x7fc32c06d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.904+0000 7fc34f34c640 1 -- 172.21.15.70:0/2359450040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 msgr2=0x7fc3480ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.904+0000 7fc34f34c640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 0x7fc3480ccbe0 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c410 tx=0x7fc33800c8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.904+0000 7fc34f34c640 1 -- 172.21.15.70:0/2359450040 shutdown_connections 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.904+0000 7fc34f34c640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc3480ce650 0x7fc3480babd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.904+0000 7fc34f34c640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7fc32c06afe0 0x7fc32c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.904+0000 7fc34f34c640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3480cbd40 0x7fc3480ccbe0 unknown :-1 s=CLOSED pgs=2450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.904+0000 7fc34f34c640 1 --2- 172.21.15.70:0/2359450040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc348070f30 0x7fc3480b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.904+0000 7fc34f34c640 1 -- 172.21.15.70:0/2359450040 >> 172.21.15.70:0/2359450040 conn(0x7fc3480660e0 msgr2=0x7fc3480cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:02.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.904+0000 7fc34f34c640 1 -- 172.21.15.70:0/2359450040 shutdown_connections 2023-10-08T07:14:02.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:02.904+0000 7fc34f34c640 1 -- 172.21.15.70:0/2359450040 wait complete. 2023-10-08T07:14:02.923 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T07:14:02.969 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:02.968+0000 7f435c04a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:14:02.994 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:02.994+0000 7fa541836040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:14:03.022 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:03.020+0000 7f435c04a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:14:03.050 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:03.049+0000 7fa541836040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:14:03.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.129+0000 7f81abe56640 1 Processor -- start 2023-10-08T07:14:03.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.129+0000 7f81abe56640 1 -- start start 2023-10-08T07:14:03.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81abe56640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81a415cd70 0x7f81a41612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:03.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81abe56640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f81a40cd4e0 0x7f81a40b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:03.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81abe56640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a40b5070 con 0x7f81a40b4c40 2023-10-08T07:14:03.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81abe56640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a40b8c30 con 0x7f81a415cd70 2023-10-08T07:14:03.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81abe56640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a40cdd80 con 0x7f81a40cd4e0 2023-10-08T07:14:03.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81a9bf2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81a415cd70 0x7f81a41612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:03.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81aa3f3640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f81a40b4c40 0x7f81a406b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47794/0 (socket says 172.21.15.70:47794) 2023-10-08T07:14:03.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81a9bf2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81a415cd70 0x7f81a41612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50118/0 (socket says 172.21.15.70:50118) 2023-10-08T07:14:03.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81aa3f3640 1 -- 172.21.15.70:0/3396502477 learned_addr learned my addr 172.21.15.70:0/3396502477 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:03.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81a93f1640 1 --2- 172.21.15.70:0/3396502477 >> v2:172.21.15.181:3300/0 conn(0x7f81a40cd4e0 0x7f81a40b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:03.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81a9bf2640 1 -- 172.21.15.70:0/3396502477 >> v2:172.21.15.181:3300/0 conn(0x7f81a40cd4e0 msgr2=0x7f81a40b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:03.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81a9bf2640 1 --2- 172.21.15.70:0/3396502477 >> v2:172.21.15.181:3300/0 conn(0x7f81a40cd4e0 0x7f81a40b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.131+0000 7f81a9bf2640 1 -- 172.21.15.70:0/3396502477 >> v1:172.21.15.70:6789/0 conn(0x7f81a40b4c40 legacy=0x7f81a406b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:03.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.132+0000 7f81a8bf0640 1 -- 172.21.15.70:0/3396502477 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673197559 0 0) 0x7f81a40b5070 con 0x7f81a40b4c40 2023-10-08T07:14:03.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.132+0000 7f81a9bf2640 1 -- 172.21.15.70:0/3396502477 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a40ce060 con 0x7f81a415cd70 2023-10-08T07:14:03.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.133+0000 7f81a9bf2640 1 --2- 172.21.15.70:0/3396502477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 0x7f81a41612f0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f81a40cd970 tx=0x7f81a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a57d4492f066d6d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:03.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.133+0000 7f81a8bf0640 1 -- 172.21.15.70:0/3396502477 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0031830 con 0x7f81a415cd70 2023-10-08T07:14:03.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.133+0000 7f81a8bf0640 1 -- 172.21.15.70:0/3396502477 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a0031990 con 0x7f81a415cd70 2023-10-08T07:14:03.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.133+0000 7f81a8bf0640 1 -- 172.21.15.70:0/3396502477 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a0031d20 con 0x7f81a415cd70 2023-10-08T07:14:03.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.134+0000 7f81abe56640 1 -- 172.21.15.70:0/3396502477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 msgr2=0x7f81a41612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:03.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.134+0000 7f81abe56640 1 --2- 172.21.15.70:0/3396502477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 0x7f81a41612f0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f81a40cd970 tx=0x7f81a002b8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.134+0000 7f81abe56640 1 -- 172.21.15.70:0/3396502477 shutdown_connections 2023-10-08T07:14:03.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.134+0000 7f81abe56640 1 --2- 172.21.15.70:0/3396502477 >> v2:172.21.15.181:3300/0 conn(0x7f81a40cd4e0 0x7f81a40b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.134+0000 7f81abe56640 1 --2- 172.21.15.70:0/3396502477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 0x7f81a41612f0 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.134+0000 7f81abe56640 1 -- 172.21.15.70:0/3396502477 >> 172.21.15.70:0/3396502477 conn(0x7f81a40cf700 msgr2=0x7f81a40cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:03.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.134+0000 7f81abe56640 1 -- 172.21.15.70:0/3396502477 shutdown_connections 2023-10-08T07:14:03.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.134+0000 7f81abe56640 1 -- 172.21.15.70:0/3396502477 wait complete. 2023-10-08T07:14:03.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.136+0000 7f81abe56640 1 Processor -- start 2023-10-08T07:14:03.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.136+0000 7f81abe56640 1 -- start start 2023-10-08T07:14:03.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81abe56640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a40b4c40 0x7f81a414c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:03.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81abe56640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81a40cd4e0 0x7f81a415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:03.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81abe56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 0x7f81a415f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:03.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81abe56640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a414c660 con 0x7f81a415cd70 2023-10-08T07:14:03.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81abe56640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40c71d0 con 0x7f81a40b4c40 2023-10-08T07:14:03.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81abe56640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a40c7420 con 0x7f81a40cd4e0 2023-10-08T07:14:03.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81aa3f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 0x7f81a415f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:03.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81aa3f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 0x7f81a415f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50130/0 (socket says 172.21.15.70:50130) 2023-10-08T07:14:03.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81aa3f3640 1 -- 172.21.15.70:0/912617970 learned_addr learned my addr 172.21.15.70:0/912617970 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:03.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81a93f1640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81a40cd4e0 0x7f81a415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:03.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.137+0000 7f81a9bf2640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a40b4c40 0x7f81a414c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f81aa3f3640 1 -- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81a40cd4e0 msgr2=0x7f81a415d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f81aa3f3640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81a40cd4e0 0x7f81a415d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f81aa3f3640 1 -- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a40b4c40 msgr2=0x7f81a414c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f81aa3f3640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a40b4c40 0x7f81a414c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f81aa3f3640 1 -- 172.21.15.70:0/912617970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a0009400 con 0x7f81a415cd70 2023-10-08T07:14:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f81a93f1640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81a40cd4e0 0x7f81a415d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f81aa3f3640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 0x7f81a415f530 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f819c00ca40 tx=0x7f819c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:03.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f819a7fc640 1 -- 172.21.15.70:0/912617970 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c00abd0 con 0x7f81a415cd70 2023-10-08T07:14:03.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f819a7fc640 1 -- 172.21.15.70:0/912617970 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c00ad70 con 0x7f81a415cd70 2023-10-08T07:14:03.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f819a7fc640 1 -- 172.21.15.70:0/912617970 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c012640 con 0x7f81a415cd70 2023-10-08T07:14:03.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.138+0000 7f81abe56640 1 -- 172.21.15.70:0/912617970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a40c7700 con 0x7f81a415cd70 2023-10-08T07:14:03.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.139+0000 7f81abe56640 1 -- 172.21.15.70:0/912617970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a4160690 con 0x7f81a415cd70 2023-10-08T07:14:03.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.140+0000 7f81abe56640 1 -- 172.21.15.70:0/912617970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a4160880 con 0x7f81a415cd70 2023-10-08T07:14:03.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.141+0000 7f819a7fc640 1 -- 172.21.15.70:0/912617970 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 129) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f819c0127a0 con 0x7f81a415cd70 2023-10-08T07:14:03.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.141+0000 7f819a7fc640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f818806afe0 0x7f818806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:03.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.142+0000 7f819a7fc640 1 -- 172.21.15.70:0/912617970 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f819c08a290 con 0x7f81a415cd70 2023-10-08T07:14:03.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.142+0000 7f81a9bf2640 1 -- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f818806afe0 msgr2=0x7f818806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3993370366 2023-10-08T07:14:03.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.142+0000 7f81a9bf2640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f818806afe0 0x7f818806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:03.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.146+0000 7f819a7fc640 1 -- 172.21.15.70:0/912617970 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f819c054c90 con 0x7f81a415cd70 2023-10-08T07:14:03.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.328+0000 7f81abe56640 1 -- 172.21.15.70:0/912617970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f81a404e160 con 0x7f81a415cd70 2023-10-08T07:14:03.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.335+0000 7f819a7fc640 1 -- 172.21.15.70:0/912617970 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v78)=0 v78) v1 ==== 133+0+0 (secure 0 0 0) 0x7f819c001ab0 con 0x7f81a415cd70 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 -- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f818806afe0 msgr2=0x7f818806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f818806afe0 0x7f818806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 -- 172.21.15.70:0/912617970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 msgr2=0x7f81a415f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 0x7f81a415f530 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f819c00ca40 tx=0x7f819c00cf00 comp rx=0 tx=0).stop 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 -- 172.21.15.70:0/912617970 shutdown_connections 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f818806afe0 0x7f818806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81a415cd70 0x7f81a415f530 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f81a40cd4e0 0x7f81a415d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 --2- 172.21.15.70:0/912617970 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f81a40b4c40 0x7f81a414c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 -- 172.21.15.70:0/912617970 >> 172.21.15.70:0/912617970 conn(0x7f81a40cf700 msgr2=0x7f81a415bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:03.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 -- 172.21.15.70:0/912617970 shutdown_connections 2023-10-08T07:14:03.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:03.338+0000 7f81abe56640 1 -- 172.21.15.70:0/912617970 wait complete. 2023-10-08T07:14:03.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:03.356 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-10-08T07:14:03.412 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-08T07:14:04.226 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:03.474+0000 7f435c04a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:14:04.423 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:03.526+0000 7f435c04a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:14:04.423 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:03.576+0000 7f435c04a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:14:04.423 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:03.637+0000 7f435c04a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:14:04.423 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:03.686+0000 7f435c04a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:14:04.424 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:03.795+0000 7f435c04a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:14:04.424 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:03.857+0000 7f435c04a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:14:04.424 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:03.905+0000 7f435c04a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:14:04.424 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:03.962+0000 7f435c04a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:14:04.424 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:04.012+0000 7f435c04a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:14:04.424 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:03.519+0000 7fa541836040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:14:04.424 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:03.571+0000 7fa541836040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:14:04.424 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:03.621+0000 7fa541836040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:14:04.424 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:03.682+0000 7fa541836040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:14:04.424 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:03.731+0000 7fa541836040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:14:04.425 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:03.841+0000 7fa541836040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:14:04.425 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:03.903+0000 7fa541836040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:14:04.425 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:03.954+0000 7fa541836040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:14:04.425 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:04.010+0000 7fa541836040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:14:04.425 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:04.060+0000 7fa541836040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:14:04.426 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:04.402+0000 7f435c04a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:14:04.445 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:04.445+0000 7fa541836040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:14:04.561 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:04.560+0000 7f435c04a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:14:04.612 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:04.611+0000 7fa541836040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:14:04.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.644+0000 7f3f49d6e640 1 Processor -- start 2023-10-08T07:14:04.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.644+0000 7f3f49d6e640 1 -- start start 2023-10-08T07:14:04.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.646+0000 7f3f49d6e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4415cf30 0x7f3f440d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:04.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.646+0000 7f3f49d6e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f440cd520 con 0x7f3f440cf690 2023-10-08T07:14:04.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.646+0000 7f3f49d6e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f440d1910 con 0x7f3f440cf2b0 2023-10-08T07:14:04.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.646+0000 7f3f49d6e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440ce080 con 0x7f3f4415cf30 2023-10-08T07:14:04.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.646+0000 7f3f43fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3f440cf690 0x7f3f4415c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40846/0 (socket says 172.21.15.70:40846) 2023-10-08T07:14:04.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.646+0000 7f3f43fff640 1 -- 172.21.15.70:0/1485043920 learned_addr learned my addr 172.21.15.70:0/1485043920 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:04.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.646+0000 7f3f4956d640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4415cf30 0x7f3f440d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:04.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121179716 0 0) 0x7f3f440cd520 con 0x7f3f440cf690 2023-10-08T07:14:04.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f34003520 con 0x7f3f440cf690 2023-10-08T07:14:04.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3468158610 0 0) 0x7f3f34003520 con 0x7f3f440cf690 2023-10-08T07:14:04.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4415cf30 msgr2=0x7f3f440d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4415cf30 0x7f3f440d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 >> v1:172.21.15.181:6789/0 conn(0x7f3f440cf2b0 legacy=0x7f3f440d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440cd750 con 0x7f3f440cf690 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3f30001670 con 0x7f3f440cf690 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 >> v1:172.21.15.70:6789/0 conn(0x7f3f440cf690 legacy=0x7f3f4415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f340039a0 0x7f3f34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f340047c0 0x7f3f34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4415cf30 0x7f3f34005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440ce080 con 0x7f3f340047c0 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3400bb40 con 0x7f3f340039a0 2023-10-08T07:14:04.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.647+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3400bd00 con 0x7f3f4415cf30 2023-10-08T07:14:04.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.648+0000 7f3f43fff640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f340047c0 0x7f3f34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:04.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.648+0000 7f3f4956d640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4415cf30 0x7f3f34005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:04.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.648+0000 7f3f48d6c640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f340039a0 0x7f3f34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:04.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.648+0000 7f3f43fff640 1 -- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4415cf30 msgr2=0x7f3f34005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:04.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.648+0000 7f3f43fff640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f4415cf30 0x7f3f34005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:04.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.648+0000 7f3f43fff640 1 -- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f340039a0 msgr2=0x7f3f34003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:04.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.648+0000 7f3f43fff640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f340039a0 0x7f3f34003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:04.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.648+0000 7f3f43fff640 1 -- 172.21.15.70:0/1485043920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f34003520 con 0x7f3f340047c0 2023-10-08T07:14:04.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.649+0000 7f3f43fff640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f340047c0 0x7f3f34004c40 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f3f30001050 tx=0x7f3f340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581d5edc8009a238 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:04.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.650+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30005ad0 con 0x7f3f340047c0 2023-10-08T07:14:04.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.650+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30005c30 con 0x7f3f340047c0 2023-10-08T07:14:04.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.650+0000 7f3f437fe640 1 -- 172.21.15.70:0/1485043920 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3002c7c0 con 0x7f3f340047c0 2023-10-08T07:14:04.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.650+0000 7f3f49d6e640 1 -- 172.21.15.70:0/1485043920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f340047c0 msgr2=0x7f3f34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:04.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.650+0000 7f3f49d6e640 1 --2- 172.21.15.70:0/1485043920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f340047c0 0x7f3f34004c40 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f3f30001050 tx=0x7f3f340057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:04.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.651+0000 7f3f4956d640 1 -- 172.21.15.70:0/1485043920 reap_dead start 2023-10-08T07:14:04.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.651+0000 7f3f49d6e640 1 -- 172.21.15.70:0/1485043920 shutdown_connections 2023-10-08T07:14:04.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.651+0000 7f3f49d6e640 1 -- 172.21.15.70:0/1485043920 >> 172.21.15.70:0/1485043920 conn(0x7f3f4409bc40 msgr2=0x7f3f440d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:04.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.651+0000 7f3f49d6e640 1 -- 172.21.15.70:0/1485043920 shutdown_connections 2023-10-08T07:14:04.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.651+0000 7f3f49d6e640 1 -- 172.21.15.70:0/1485043920 wait complete. 2023-10-08T07:14:04.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.653+0000 7f3f49d6e640 1 Processor -- start 2023-10-08T07:14:04.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.653+0000 7f3f49d6e640 1 -- start start 2023-10-08T07:14:04.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f49d6e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f340047c0 0x7f3f440b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:04.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f49d6e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f441f63c0 0x7f3f440b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:04.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f49d6e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f44159f10 0x7f3f4415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:04.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f49d6e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440b8240 con 0x7f3f441f63c0 2023-10-08T07:14:04.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f49d6e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440b4480 con 0x7f3f44159f10 2023-10-08T07:14:04.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f49d6e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f44159150 con 0x7f3f340047c0 2023-10-08T07:14:04.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f43fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f441f63c0 0x7f3f440b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:04.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f43fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f441f63c0 0x7f3f440b7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33664/0 (socket says 172.21.15.70:33664) 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f43fff640 1 -- 172.21.15.70:0/764800306 learned_addr learned my addr 172.21.15.70:0/764800306 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f48d6c640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f340047c0 0x7f3f440b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f4956d640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f44159f10 0x7f3f4415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f43fff640 1 -- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f340047c0 msgr2=0x7f3f440b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f43fff640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f340047c0 0x7f3f440b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f43fff640 1 -- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f44159f10 msgr2=0x7f3f4415f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f43fff640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f44159f10 0x7f3f4415f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f43fff640 1 -- 172.21.15.70:0/764800306 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f44159430 con 0x7f3f441f63c0 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.654+0000 7f3f4956d640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f44159f10 0x7f3f4415f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.655+0000 7f3f43fff640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f441f63c0 0x7f3f440b7d00 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f3f30000fb0 tx=0x7f3f30000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.655+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30039070 con 0x7f3f441f63c0 2023-10-08T07:14:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.655+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3002dc60 con 0x7f3f441f63c0 2023-10-08T07:14:04.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.655+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30007070 con 0x7f3f441f63c0 2023-10-08T07:14:04.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.655+0000 7f3f49d6e640 1 -- 172.21.15.70:0/764800306 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f44159710 con 0x7f3f441f63c0 2023-10-08T07:14:04.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.655+0000 7f3f49d6e640 1 -- 172.21.15.70:0/764800306 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f441f6820 con 0x7f3f441f63c0 2023-10-08T07:14:04.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.656+0000 7f3f49d6e640 1 -- 172.21.15.70:0/764800306 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f44074150 con 0x7f3f441f63c0 2023-10-08T07:14:04.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.657+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 129) v1 ==== 89348+0+0 (secure 0 0 0) 0x7f3f30032070 con 0x7f3f441f63c0 2023-10-08T07:14:04.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.658+0000 7f3f417fa640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 0x7f3f2806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:04.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.658+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f3f3003c040 con 0x7f3f441f63c0 2023-10-08T07:14:04.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.658+0000 7f3f48d6c640 1 -- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 msgr2=0x7f3f2806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3993370366 2023-10-08T07:14:04.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.658+0000 7f3f48d6c640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 0x7f3f2806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:04.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.663+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f30075850 con 0x7f3f441f63c0 2023-10-08T07:14:04.671 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:04.669+0000 7f435c04a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:14:04.721 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:04.719+0000 7f435c04a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:14:04.728 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:04.727+0000 7fa541836040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:14:04.780 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:04.779+0000 7fa541836040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:14:04.829 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:04.827+0000 7f435c04a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:14:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.859+0000 7f3f48d6c640 1 -- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 msgr2=0x7f3f2806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3993370366 2023-10-08T07:14:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.859+0000 7f3f48d6c640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 0x7f3f2806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:14:04.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:04.872+0000 7f3f49d6e640 1 -- 172.21.15.70:0/764800306 --> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f440cdd10 con 0x7f3f2806afe0 2023-10-08T07:14:04.890 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:04.888+0000 7fa541836040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:14:04.935 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:04.933+0000 7f435c04a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:14:04.996 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:04.995+0000 7fa541836040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:14:05.092 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:05.090+0000 7f435c04a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:14:05.140 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:05.139+0000 7f435c04a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:14:05.152 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:05.151+0000 7fa541836040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:14:05.201 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:05.200+0000 7fa541836040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:14:05.258 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:05.257+0000 7f435c04a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:14:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:05.260+0000 7f3f48d6c640 1 -- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 msgr2=0x7f3f2806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3993370366 2023-10-08T07:14:05.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:05.260+0000 7f3f48d6c640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 0x7f3f2806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T07:14:05.326 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:05.325+0000 7fa541836040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:14:06.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:06.061+0000 7f3f48d6c640 1 -- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 msgr2=0x7f3f2806d4a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/3993370366 2023-10-08T07:14:06.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:06.061+0000 7f3f48d6c640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 0x7f3f2806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T07:14:06.070 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.068+0000 7f435c04a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:14:06.127 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:06.126+0000 7fa541836040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:14:06.371 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.370+0000 7f435c04a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:14:06.427 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:06.426+0000 7fa541836040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:14:06.428 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.427+0000 7f435c04a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:14:06.483 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:06.482+0000 7fa541836040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:14:06.541 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.539+0000 7f435c04a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:14:06.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:06.561+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 130) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f3f300a8dd0 con 0x7f3f441f63c0 2023-10-08T07:14:06.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:06.562+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 msgr2=0x7f3f2806d4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:06.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:06.562+0000 7f3f417fa640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/3993370366,v1:172.21.15.181:6833/3993370366] conn(0x7f3f2806afe0 0x7f3f2806d4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:06.607 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:06.607+0000 7fa541836040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:14:06.852 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.850+0000 7f42daadb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.853 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.850+0000 7f42daadb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.853 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.850+0000 7f42daadb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.853 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.850+0000 7f42daadb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.853 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.850+0000 7f42daadb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.853 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.851+0000 7f42d82d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.853 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.851+0000 7f42d82d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.854 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.851+0000 7f42d82d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.854 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.851+0000 7f42d82d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.854 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.851+0000 7f42d82d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.870 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.869+0000 7f42d82d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.871 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.869+0000 7f42d82d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.871 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.869+0000 7f42d82d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.871 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.869+0000 7f42d82d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.871 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.869+0000 7f42d82d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.872 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.870+0000 7f42daadb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.872 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.870+0000 7f42daadb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.872 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.870+0000 7f42daadb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.872 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.870+0000 7f42daadb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:06.872 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:06.870+0000 7f42daadb640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:07.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.573+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 131) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f3f30034dc0 con 0x7f3f441f63c0 2023-10-08T07:14:07.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.574+0000 7f3f417fa640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f3f28035880 0x7f3f28077be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:07.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.575+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 --> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f28035c60 con 0x7f3f28035880 2023-10-08T07:14:07.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.578+0000 7f3f48d6c640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f3f28035880 0x7f3f28077be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:07.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.592+0000 7f3f48d6c640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f3f28035880 0x7f3f28077be0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3f38007bf0 tx=0x7f3f38009040 comp rx=0 tx=0).ready entity=mgr.8551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:07.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.741+0000 7f3f417fa640 1 -- 172.21.15.70:0/764800306 <== mgr.8551 v2:172.21.15.181:6832/860795336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+53 (secure 0 0 0) 0x7f3f28035c60 con 0x7f3f28035880 2023-10-08T07:14:07.742 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path mandatory_fs_argument_test_dir 2023-10-08T07:14:07.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.744+0000 7f3f49d6e640 1 -- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f3f28035880 msgr2=0x7f3f28077be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:07.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.744+0000 7f3f49d6e640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f3f28035880 0x7f3f28077be0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3f38007bf0 tx=0x7f3f38009040 comp rx=0 tx=0).stop 2023-10-08T07:14:07.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.744+0000 7f3f49d6e640 1 -- 172.21.15.70:0/764800306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f441f63c0 msgr2=0x7f3f440b7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.744+0000 7f3f49d6e640 1 --2- 172.21.15.70:0/764800306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f441f63c0 0x7f3f440b7d00 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f3f30000fb0 tx=0x7f3f30000fe0 comp rx=0 tx=0).stop 2023-10-08T07:14:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.744+0000 7f3f4956d640 1 -- 172.21.15.70:0/764800306 reap_dead start 2023-10-08T07:14:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.745+0000 7f3f49d6e640 1 -- 172.21.15.70:0/764800306 shutdown_connections 2023-10-08T07:14:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.745+0000 7f3f49d6e640 1 -- 172.21.15.70:0/764800306 >> 172.21.15.70:0/764800306 conn(0x7f3f4409bc40 msgr2=0x7f3f4415ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.745+0000 7f3f49d6e640 1 -- 172.21.15.70:0/764800306 shutdown_connections 2023-10-08T07:14:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:07.745+0000 7f3f49d6e640 1 -- 172.21.15.70:0/764800306 wait complete. 2023-10-08T07:14:07.766 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path mandatory_fs_argument_test_dir 2023-10-08T07:14:07.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-08T07:14:08.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:14:08.027 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:14:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.036+0000 7fdbff59e640 1 Processor -- start 2023-10-08T07:14:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.036+0000 7fdbff59e640 1 -- start start 2023-10-08T07:14:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.038+0000 7fdbff59e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdc0015ccb0 0x7fdc000d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.038+0000 7fdbff59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc000d2870 con 0x7fdc000d0b90 2023-10-08T07:14:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.038+0000 7fdbff59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc000d1150 con 0x7fdc000d00e0 2023-10-08T07:14:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.038+0000 7fdbff59e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc000d1430 con 0x7fdc0015ccb0 2023-10-08T07:14:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.038+0000 7fdbf7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdc000d0b90 0x7fdc000cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40856/0 (socket says 172.21.15.70:40856) 2023-10-08T07:14:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.038+0000 7fdbf7fff640 1 -- 172.21.15.70:0/640620619 learned_addr learned my addr 172.21.15.70:0/640620619 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfe59c640 1 --2- 172.21.15.70:0/640620619 >> v2:172.21.15.181:3300/0 conn(0x7fdc0015ccb0 0x7fdc000d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612284984 0 0) 0x7fdc000d2870 con 0x7fdc000d0b90 2023-10-08T07:14:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbec003520 con 0x7fdc000d0b90 2023-10-08T07:14:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 843880756 0 0) 0x7fdbec003520 con 0x7fdc000d0b90 2023-10-08T07:14:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 >> v2:172.21.15.181:3300/0 conn(0x7fdc0015ccb0 msgr2=0x7fdc000d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 --2- 172.21.15.70:0/640620619 >> v2:172.21.15.181:3300/0 conn(0x7fdc0015ccb0 0x7fdc000d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 >> v1:172.21.15.181:6789/0 conn(0x7fdc000d00e0 legacy=0x7fdc000b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc000d1710 con 0x7fdc000d0b90 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fdbe8001670 con 0x7fdc000d0b90 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 >> v1:172.21.15.70:6789/0 conn(0x7fdc000d0b90 legacy=0x7fdc000cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 --2- 172.21.15.70:0/640620619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdbec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 --2- 172.21.15.70:0/640620619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbec0047c0 0x7fdbec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 --2- 172.21.15.70:0/640620619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbec0051d0 0x7fdbec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000d1430 con 0x7fdbec0039c0 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec00c120 con 0x7fdbec0047c0 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.039+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec00c2e0 con 0x7fdbec0051d0 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.040+0000 7fdbfe59c640 1 --2- 172.21.15.70:0/640620619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdbec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.040+0000 7fdbf7fff640 1 --2- 172.21.15.70:0/640620619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbec0047c0 0x7fdbec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.040+0000 7fdbfe59c640 1 -- 172.21.15.70:0/640620619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbec0051d0 msgr2=0x7fdbec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.040+0000 7fdbfe59c640 1 --2- 172.21.15.70:0/640620619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdbec0051d0 0x7fdbec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.040+0000 7fdbfe59c640 1 -- 172.21.15.70:0/640620619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbec0047c0 msgr2=0x7fdbec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.040+0000 7fdbfe59c640 1 --2- 172.21.15.70:0/640620619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdbec0047c0 0x7fdbec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.040+0000 7fdbfe59c640 1 -- 172.21.15.70:0/640620619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec003520 con 0x7fdbec0039c0 2023-10-08T07:14:08.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.040+0000 7fdbfed9d640 1 -- 172.21.15.70:0/640620619 reap_dead start 2023-10-08T07:14:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.041+0000 7fdbfe59c640 1 --2- 172.21.15.70:0/640620619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdbec003de0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009a90 tx=0x7fdbec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626467dd522dca0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.042+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf002c6b0 con 0x7fdbec0039c0 2023-10-08T07:14:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.042+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf002c810 con 0x7fdbec0039c0 2023-10-08T07:14:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.042+0000 7fdbfdd9b640 1 -- 172.21.15.70:0/640620619 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf002cba0 con 0x7fdbec0039c0 2023-10-08T07:14:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.042+0000 7fdbff59e640 1 -- 172.21.15.70:0/640620619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 msgr2=0x7fdbec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.042+0000 7fdbff59e640 1 --2- 172.21.15.70:0/640620619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdbec003de0 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0009a90 tx=0x7fdbec0057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.042+0000 7fdbff59e640 1 -- 172.21.15.70:0/640620619 shutdown_connections 2023-10-08T07:14:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.042+0000 7fdbff59e640 1 --2- 172.21.15.70:0/640620619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdbec003de0 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.042+0000 7fdbff59e640 1 -- 172.21.15.70:0/640620619 >> 172.21.15.70:0/640620619 conn(0x7fdc00071660 msgr2=0x7fdc00071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.042+0000 7fdbff59e640 1 -- 172.21.15.70:0/640620619 shutdown_connections 2023-10-08T07:14:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.043+0000 7fdbff59e640 1 -- 172.21.15.70:0/640620619 wait complete. 2023-10-08T07:14:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.044+0000 7fdbff59e640 1 Processor -- start 2023-10-08T07:14:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.044+0000 7fdbff59e640 1 -- start start 2023-10-08T07:14:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.045+0000 7fdbff59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdc000c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.045+0000 7fdbff59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0015ea10 0x7fdc001f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.045+0000 7fdbff59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc001f2940 0x7fdc000b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.045+0000 7fdbff59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc00156940 con 0x7fdbec0039c0 2023-10-08T07:14:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.045+0000 7fdbff59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc001f33a0 con 0x7fdc0015ea10 2023-10-08T07:14:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.045+0000 7fdbff59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000c9050 con 0x7fdc001f2940 2023-10-08T07:14:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbfe59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdc000c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbfe59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdc000c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33672/0 (socket says 172.21.15.70:33672) 2023-10-08T07:14:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbfe59c640 1 -- 172.21.15.70:0/1959832410 learned_addr learned my addr 172.21.15.70:0/1959832410 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbf7fff640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0015ea10 0x7fdc001f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbfe59c640 1 -- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc001f2940 msgr2=0x7fdc000b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbfed9d640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc001f2940 0x7fdc000b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbfe59c640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc001f2940 0x7fdc000b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbfe59c640 1 -- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0015ea10 msgr2=0x7fdc001f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbfe59c640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0015ea10 0x7fdc001f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbfe59c640 1 -- 172.21.15.70:0/1959832410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc001f2de0 con 0x7fdbec0039c0 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbf7fff640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0015ea10 0x7fdc001f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.046+0000 7fdbfed9d640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc001f2940 0x7fdc000b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.047+0000 7fdbfe59c640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdc000c8b10 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fdbf002c250 tx=0x7fdbf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.047+0000 7fdbf6ffd640 1 -- 172.21.15.70:0/1959832410 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0003810 con 0x7fdbec0039c0 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.047+0000 7fdbf6ffd640 1 -- 172.21.15.70:0/1959832410 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf0003970 con 0x7fdbec0039c0 2023-10-08T07:14:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.047+0000 7fdbf6ffd640 1 -- 172.21.15.70:0/1959832410 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0003ce0 con 0x7fdbec0039c0 2023-10-08T07:14:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.047+0000 7fdbff59e640 1 -- 172.21.15.70:0/1959832410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc001f30c0 con 0x7fdbec0039c0 2023-10-08T07:14:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.047+0000 7fdbff59e640 1 -- 172.21.15.70:0/1959832410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0015e4f0 con 0x7fdbec0039c0 2023-10-08T07:14:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.048+0000 7fdbff59e640 1 -- 172.21.15.70:0/1959832410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0015e6b0 con 0x7fdbec0039c0 2023-10-08T07:14:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.049+0000 7fdbf6ffd640 1 -- 172.21.15.70:0/1959832410 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 131) v1 ==== 89084+0+0 (secure 0 0 0) 0x7fdbf0040450 con 0x7fdbec0039c0 2023-10-08T07:14:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.050+0000 7fdbf6ffd640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fdbe006ad10 0x7fdbe006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.050+0000 7fdbf6ffd640 1 -- 172.21.15.70:0/1959832410 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fdbf00ab810 con 0x7fdbec0039c0 2023-10-08T07:14:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.051+0000 7fdbf7fff640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fdbe006ad10 0x7fdbe006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.052+0000 7fdbf7fff640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fdbe006ad10 0x7fdbe006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8001eb0 tx=0x7fdbe8000f70 comp rx=0 tx=0).ready entity=mgr.8551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:08.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.055+0000 7fdbf6ffd640 1 -- 172.21.15.70:0/1959832410 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdbf00764c0 con 0x7fdbec0039c0 2023-10-08T07:14:08.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.270+0000 7fdbff59e640 1 -- 172.21.15.70:0/1959832410 --> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc000b5b10 con 0x7fdbe006ad10 2023-10-08T07:14:08.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.389+0000 7fdbf6ffd640 1 -- 172.21.15.70:0/1959832410 <== mgr.8551 v2:172.21.15.181:6832/860795336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fdbe800f7c0 con 0x7fdbe006ad10 2023-10-08T07:14:08.390 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule activated for path mandatory_fs_argument_test_dir 2023-10-08T07:14:08.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.392+0000 7fdbff59e640 1 -- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fdbe006ad10 msgr2=0x7fdbe006d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.392+0000 7fdbff59e640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fdbe006ad10 0x7fdbe006d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8001eb0 tx=0x7fdbe8000f70 comp rx=0 tx=0).stop 2023-10-08T07:14:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.392+0000 7fdbff59e640 1 -- 172.21.15.70:0/1959832410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 msgr2=0x7fdc000c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.392+0000 7fdbff59e640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdc000c8b10 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7fdbf002c250 tx=0x7fdbf0004730 comp rx=0 tx=0).stop 2023-10-08T07:14:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.393+0000 7fdbff59e640 1 -- 172.21.15.70:0/1959832410 shutdown_connections 2023-10-08T07:14:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.393+0000 7fdbff59e640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fdbe006ad10 0x7fdbe006d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.393+0000 7fdbff59e640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc001f2940 0x7fdc000b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.393+0000 7fdbff59e640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc0015ea10 0x7fdc001f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.393+0000 7fdbff59e640 1 --2- 172.21.15.70:0/1959832410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbec0039c0 0x7fdc000c8b10 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.393+0000 7fdbff59e640 1 -- 172.21.15.70:0/1959832410 >> 172.21.15.70:0/1959832410 conn(0x7fdc00071660 msgr2=0x7fdc000b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.393+0000 7fdbff59e640 1 -- 172.21.15.70:0/1959832410 shutdown_connections 2023-10-08T07:14:08.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.393+0000 7fdbff59e640 1 -- 172.21.15.70:0/1959832410 wait complete. 2023-10-08T07:14:08.414 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path mandatory_fs_argument_test_dir 2023-10-08T07:14:08.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-08T07:14:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.643+0000 7ff6abb9e640 1 Processor -- start 2023-10-08T07:14:08.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.643+0000 7ff6abb9e640 1 -- start start 2023-10-08T07:14:08.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.645+0000 7ff6abb9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6a406dfc0 0x7ff6a4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.645+0000 7ff6abb9e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a40b1cc0 con 0x7ff6a40afbe0 2023-10-08T07:14:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.645+0000 7ff6abb9e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a40b2620 con 0x7ff6a4071070 2023-10-08T07:14:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.645+0000 7ff6abb9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40b2060 con 0x7ff6a406dfc0 2023-10-08T07:14:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.645+0000 7ff6a993a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff6a40afbe0 0x7ff6a40b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40868/0 (socket says 172.21.15.70:40868) 2023-10-08T07:14:08.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.645+0000 7ff6a993a640 1 -- 172.21.15.70:0/697525045 learned_addr learned my addr 172.21.15.70:0/697525045 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.645+0000 7ff6a9139640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6a406dfc0 0x7ff6a4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.645+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618383192 0 0) 0x7ff6a40b1cc0 con 0x7ff6a40afbe0 2023-10-08T07:14:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.645+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff690003520 con 0x7ff6a40afbe0 2023-10-08T07:14:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2279914574 0 0) 0x7ff690003520 con 0x7ff6a40afbe0 2023-10-08T07:14:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6a406dfc0 msgr2=0x7ff6a4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6a406dfc0 0x7ff6a4070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 >> v1:172.21.15.181:6789/0 conn(0x7ff6a4071070 legacy=0x7ff6a40cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a40b2340 con 0x7ff6a40afbe0 2023-10-08T07:14:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff6a0001670 con 0x7ff6a40afbe0 2023-10-08T07:14:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 >> v1:172.21.15.70:6789/0 conn(0x7ff6a40afbe0 legacy=0x7ff6a40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6a406dfc0 0x7ff6900039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff690004020 0x7ff6900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff690004d00 0x7ff690005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40b2060 con 0x7ff690004020 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69000bbc0 con 0x7ff690004d00 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69000bd80 con 0x7ff6a406dfc0 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.646+0000 7ff6a9139640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff690004020 0x7ff6900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.647+0000 7ff6aa13b640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff690004d00 0x7ff690005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.647+0000 7ff6a9139640 1 -- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6a406dfc0 msgr2=0x7ff6900039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.647+0000 7ff6a9139640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff6a406dfc0 0x7ff6900039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.647+0000 7ff6a9139640 1 -- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff690004d00 msgr2=0x7ff690005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.647+0000 7ff6a9139640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff690004d00 0x7ff690005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.647+0000 7ff6a9139640 1 -- 172.21.15.70:0/697525045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff690003520 con 0x7ff690004020 2023-10-08T07:14:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.647+0000 7ff6aa13b640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff690004d00 0x7ff690005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.648+0000 7ff6a9139640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff690004020 0x7ff6900047c0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7ff69400ba10 tx=0x7ff6900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb24f139404e78e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.648+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694032680 con 0x7ff690004020 2023-10-08T07:14:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.648+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6940327e0 con 0x7ff690004020 2023-10-08T07:14:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.648+0000 7ff6a8938640 1 -- 172.21.15.70:0/697525045 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694032b70 con 0x7ff690004020 2023-10-08T07:14:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.649+0000 7ff6abb9e640 1 -- 172.21.15.70:0/697525045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff690004020 msgr2=0x7ff6900047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.649+0000 7ff6abb9e640 1 --2- 172.21.15.70:0/697525045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff690004020 0x7ff6900047c0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7ff69400ba10 tx=0x7ff6900057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.649+0000 7ff6aa13b640 1 -- 172.21.15.70:0/697525045 reap_dead start 2023-10-08T07:14:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.649+0000 7ff6abb9e640 1 -- 172.21.15.70:0/697525045 shutdown_connections 2023-10-08T07:14:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.649+0000 7ff6abb9e640 1 -- 172.21.15.70:0/697525045 >> 172.21.15.70:0/697525045 conn(0x7ff6a406a5d0 msgr2=0x7ff6a40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.649+0000 7ff6abb9e640 1 -- 172.21.15.70:0/697525045 shutdown_connections 2023-10-08T07:14:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.649+0000 7ff6abb9e640 1 -- 172.21.15.70:0/697525045 wait complete. 2023-10-08T07:14:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.651+0000 7ff6abb9e640 1 Processor -- start 2023-10-08T07:14:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.651+0000 7ff6abb9e640 1 -- start start 2023-10-08T07:14:08.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6abb9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff690004020 0x7ff6a414cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6abb9e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6a40c2230 0x7ff6a40cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6abb9e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6a40c2690 0x7ff6a414d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6abb9e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40cdc50 con 0x7ff6a40c2690 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6abb9e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40cdf30 con 0x7ff6a40c2230 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6abb9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40ce210 con 0x7ff690004020 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6aa13b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6a40c2690 0x7ff6a414d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6aa13b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6a40c2690 0x7ff6a414d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33688/0 (socket says 172.21.15.70:33688) 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6aa13b640 1 -- 172.21.15.70:0/1167073327 learned_addr learned my addr 172.21.15.70:0/1167073327 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6a993a640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff690004020 0x7ff6a414cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.652+0000 7ff6a9139640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6a40c2230 0x7ff6a40cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.653+0000 7ff6aa13b640 1 -- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff690004020 msgr2=0x7ff6a414cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.653+0000 7ff6aa13b640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff690004020 0x7ff6a414cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.653+0000 7ff6aa13b640 1 -- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6a40c2230 msgr2=0x7ff6a40cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.653+0000 7ff6aa13b640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6a40c2230 0x7ff6a40cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.653+0000 7ff6aa13b640 1 -- 172.21.15.70:0/1167073327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a40ce4f0 con 0x7ff6a40c2690 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.653+0000 7ff6aa13b640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6a40c2690 0x7ff6a414d6a0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40cb030 tx=0x7ff69c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.653+0000 7ff69a7fc640 1 -- 172.21.15.70:0/1167073327 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c006d50 con 0x7ff6a40c2690 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.653+0000 7ff69a7fc640 1 -- 172.21.15.70:0/1167073327 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69c003970 con 0x7ff6a40c2690 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.653+0000 7ff69a7fc640 1 -- 172.21.15.70:0/1167073327 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c003d60 con 0x7ff6a40c2690 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.653+0000 7ff6abb9e640 1 -- 172.21.15.70:0/1167073327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a40ce7d0 con 0x7ff6a40c2690 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.654+0000 7ff6abb9e640 1 -- 172.21.15.70:0/1167073327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a40cd7e0 con 0x7ff6a40c2690 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.655+0000 7ff6abb9e640 1 -- 172.21.15.70:0/1167073327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a40cd9d0 con 0x7ff6a40c2690 2023-10-08T07:14:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.656+0000 7ff69a7fc640 1 -- 172.21.15.70:0/1167073327 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 132) v1 ==== 89451+0+0 (secure 0 0 0) 0x7ff69c005070 con 0x7ff6a40c2690 2023-10-08T07:14:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.657+0000 7ff69a7fc640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff68806b100 0x7ff68806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.657+0000 7ff69a7fc640 1 -- 172.21.15.70:0/1167073327 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7ff69c08a030 con 0x7ff6a40c2690 2023-10-08T07:14:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.658+0000 7ff6a993a640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff68806b100 0x7ff68806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.659+0000 7ff6a993a640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff68806b100 0x7ff68806d5c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0001eb0 tx=0x7ff6a0001a60 comp rx=0 tx=0).ready entity=mgr.8551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.661+0000 7ff69a7fc640 1 -- 172.21.15.70:0/1167073327 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff69c054640 con 0x7ff6a40c2690 2023-10-08T07:14:08.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:08.875+0000 7ff6abb9e640 1 -- 172.21.15.70:0/1167073327 --> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6a4070630 con 0x7ff68806b100 2023-10-08T07:14:09.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.026+0000 7ff69a7fc640 1 -- 172.21.15.70:0/1167073327 <== mgr.8551 v2:172.21.15.181:6832/860795336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7ff6a0001d30 con 0x7ff68806b100 2023-10-08T07:14:09.027 INFO:teuthology.orchestra.run.smithi070.stdout:Retention added to path mandatory_fs_argument_test_dir 2023-10-08T07:14:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 -- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff68806b100 msgr2=0x7ff68806d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff68806b100 0x7ff68806d5c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0001eb0 tx=0x7ff6a0001a60 comp rx=0 tx=0).stop 2023-10-08T07:14:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 -- 172.21.15.70:0/1167073327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6a40c2690 msgr2=0x7ff6a414d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6a40c2690 0x7ff6a414d6a0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40cb030 tx=0x7ff69c00dc00 comp rx=0 tx=0).stop 2023-10-08T07:14:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 -- 172.21.15.70:0/1167073327 shutdown_connections 2023-10-08T07:14:09.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff68806b100 0x7ff68806d5c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6a40c2690 0x7ff6a414d6a0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff6a40c2230 0x7ff6a40cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 --2- 172.21.15.70:0/1167073327 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff690004020 0x7ff6a414cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 -- 172.21.15.70:0/1167073327 >> 172.21.15.70:0/1167073327 conn(0x7ff6a406a5d0 msgr2=0x7ff6a406d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.029+0000 7ff6abb9e640 1 -- 172.21.15.70:0/1167073327 shutdown_connections 2023-10-08T07:14:09.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.030+0000 7ff6abb9e640 1 -- 172.21.15.70:0/1167073327 wait complete. 2023-10-08T07:14:09.049 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path mandatory_fs_argument_test_dir 2023-10-08T07:14:09.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-10-08T07:14:09.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.272+0000 7f19f02a2640 1 Processor -- start 2023-10-08T07:14:09.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.273+0000 7f19f02a2640 1 -- start start 2023-10-08T07:14:09.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.274+0000 7f19f02a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 0x7f19e806db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.274+0000 7f19f02a2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19e815aa00 0x7f19e80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.274+0000 7f19f02a2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e815b310 con 0x7f19e806c3a0 2023-10-08T07:14:09.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.274+0000 7f19f02a2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e806e0a0 con 0x7f19e815aa00 2023-10-08T07:14:09.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.274+0000 7f19f02a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e806d290 con 0x7f19e80c9cf0 2023-10-08T07:14:09.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.274+0000 7f19ee83f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19e815aa00 0x7f19e80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.274+0000 7f19ee83f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19e815aa00 0x7f19e80b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33690/0 (socket says 172.21.15.70:33690) 2023-10-08T07:14:09.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.274+0000 7f19ee83f640 1 -- 172.21.15.70:0/18818959 learned_addr learned my addr 172.21.15.70:0/18818959 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:09.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.275+0000 7f19ee83f640 1 -- 172.21.15.70:0/18818959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 msgr2=0x7f19e806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:09.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.275+0000 7f19ed83d640 1 --2- 172.21.15.70:0/18818959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 0x7f19e806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.275+0000 7f19ee83f640 1 --2- 172.21.15.70:0/18818959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 0x7f19e806db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.275+0000 7f19ee83f640 1 -- 172.21.15.70:0/18818959 >> v1:172.21.15.181:6789/0 conn(0x7f19e806c3a0 legacy=0x7f19e806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:09.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.275+0000 7f19ee83f640 1 -- 172.21.15.70:0/18818959 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80c9860 con 0x7f19e815aa00 2023-10-08T07:14:09.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.276+0000 7f19ee83f640 1 --2- 172.21.15.70:0/18818959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 0x7f19e80b2a90 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009e50 tx=0x7f19e002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e08b7f704ae86c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:09.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.277+0000 7f19ed03c640 1 -- 172.21.15.70:0/18818959 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e002ca70 con 0x7f19e815aa00 2023-10-08T07:14:09.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.277+0000 7f19ed03c640 1 -- 172.21.15.70:0/18818959 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e002cbd0 con 0x7f19e815aa00 2023-10-08T07:14:09.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.277+0000 7f19ed03c640 1 -- 172.21.15.70:0/18818959 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e00343f0 con 0x7f19e815aa00 2023-10-08T07:14:09.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.278+0000 7f19f02a2640 1 -- 172.21.15.70:0/18818959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 msgr2=0x7f19e80b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:09.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.278+0000 7f19f02a2640 1 --2- 172.21.15.70:0/18818959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 0x7f19e80b2a90 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009e50 tx=0x7f19e002bed0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.278+0000 7f19f02a2640 1 -- 172.21.15.70:0/18818959 shutdown_connections 2023-10-08T07:14:09.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.278+0000 7f19f02a2640 1 --2- 172.21.15.70:0/18818959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 0x7f19e80b2a90 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.278+0000 7f19f02a2640 1 --2- 172.21.15.70:0/18818959 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 0x7f19e806db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.278+0000 7f19f02a2640 1 -- 172.21.15.70:0/18818959 >> 172.21.15.70:0/18818959 conn(0x7f19e806a5d0 msgr2=0x7f19e80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:09.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.278+0000 7f19f02a2640 1 -- 172.21.15.70:0/18818959 shutdown_connections 2023-10-08T07:14:09.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.278+0000 7f19f02a2640 1 -- 172.21.15.70:0/18818959 wait complete. 2023-10-08T07:14:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.280+0000 7f19f02a2640 1 Processor -- start 2023-10-08T07:14:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.280+0000 7f19f02a2640 1 -- start start 2023-10-08T07:14:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.281+0000 7f19f02a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19e806c3a0 0x7f19e80cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.281+0000 7f19f02a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 0x7f19e8159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.281+0000 7f19f02a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 0x7f19e81f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.281+0000 7f19f02a2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80c3c80 con 0x7f19e815aa00 2023-10-08T07:14:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.281+0000 7f19f02a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80c3f60 con 0x7f19e806c3a0 2023-10-08T07:14:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.281+0000 7f19f02a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e80c4240 con 0x7f19e80c9cf0 2023-10-08T07:14:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.281+0000 7f19ee83f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 0x7f19e81f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.281+0000 7f19ee83f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 0x7f19e81f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33694/0 (socket says 172.21.15.70:33694) 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.281+0000 7f19ee83f640 1 -- 172.21.15.70:0/3818488394 learned_addr learned my addr 172.21.15.70:0/3818488394 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19ee03e640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19e806c3a0 0x7f19e80cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19ed83d640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 0x7f19e8159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19ee83f640 1 -- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 msgr2=0x7f19e8159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19ee83f640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 0x7f19e8159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19ee83f640 1 -- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19e806c3a0 msgr2=0x7f19e80cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19ee83f640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19e806c3a0 0x7f19e80cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19ee83f640 1 -- 172.21.15.70:0/3818488394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0009b30 con 0x7f19e815aa00 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19ed83d640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 0x7f19e8159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19ee03e640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19e806c3a0 0x7f19e80cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:09.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19ee83f640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 0x7f19e81f3ae0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009e50 tx=0x7f19e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:09.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19deffd640 1 -- 172.21.15.70:0/3818488394 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0040670 con 0x7f19e815aa00 2023-10-08T07:14:09.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19deffd640 1 -- 172.21.15.70:0/3818488394 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e00407f0 con 0x7f19e815aa00 2023-10-08T07:14:09.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19deffd640 1 -- 172.21.15.70:0/3818488394 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0040ba0 con 0x7f19e815aa00 2023-10-08T07:14:09.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.282+0000 7f19f02a2640 1 -- 172.21.15.70:0/3818488394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e80c4520 con 0x7f19e815aa00 2023-10-08T07:14:09.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.283+0000 7f19f02a2640 1 -- 172.21.15.70:0/3818488394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e815e2d0 con 0x7f19e815aa00 2023-10-08T07:14:09.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.284+0000 7f19f02a2640 1 -- 172.21.15.70:0/3818488394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e804e160 con 0x7f19e815aa00 2023-10-08T07:14:09.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.285+0000 7f19deffd640 1 -- 172.21.15.70:0/3818488394 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 132) v1 ==== 89451+0+0 (secure 0 0 0) 0x7f19e0040d00 con 0x7f19e815aa00 2023-10-08T07:14:09.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.286+0000 7f19deffd640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f19cc06b100 0x7f19cc06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.286+0000 7f19deffd640 1 -- 172.21.15.70:0/3818488394 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f19e00ad100 con 0x7f19e815aa00 2023-10-08T07:14:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.286+0000 7f19ee03e640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f19cc06b100 0x7f19cc06d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.287+0000 7f19ee03e640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f19cc06b100 0x7f19cc06d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19e4003eb0 tx=0x7f19e4009040 comp rx=0 tx=0).ready entity=mgr.8551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:09.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.290+0000 7f19deffd640 1 -- 172.21.15.70:0/3818488394 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19e0077900 con 0x7f19e815aa00 2023-10-08T07:14:09.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.497+0000 7f19f02a2640 1 -- 172.21.15.70:0/3818488394 --> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19e80c8f30 con 0x7f19cc06b100 2023-10-08T07:14:09.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.516+0000 7f19deffd640 1 -- 172.21.15.70:0/3818488394 <== mgr.8551 v2:172.21.15.181:6832/860795336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+88 (secure 0 0 0) 0x7f19e4005030 con 0x7f19cc06b100 2023-10-08T07:14:09.517 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:09.517 INFO:teuthology.orchestra.run.smithi070.stdout:{"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-08T07:14:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.518+0000 7f19f02a2640 1 -- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f19cc06b100 msgr2=0x7f19cc06d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.518+0000 7f19f02a2640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f19cc06b100 0x7f19cc06d5c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f19e4003eb0 tx=0x7f19e4009040 comp rx=0 tx=0).stop 2023-10-08T07:14:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.518+0000 7f19f02a2640 1 -- 172.21.15.70:0/3818488394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 msgr2=0x7f19e81f3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.518+0000 7f19f02a2640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 0x7f19e81f3ae0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009e50 tx=0x7f19e0004a40 comp rx=0 tx=0).stop 2023-10-08T07:14:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.518+0000 7f19f02a2640 1 -- 172.21.15.70:0/3818488394 shutdown_connections 2023-10-08T07:14:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.518+0000 7f19f02a2640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f19cc06b100 0x7f19cc06d5c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.518+0000 7f19f02a2640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19e815aa00 0x7f19e81f3ae0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.518+0000 7f19f02a2640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19e80c9cf0 0x7f19e8159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.518+0000 7f19f02a2640 1 --2- 172.21.15.70:0/3818488394 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19e806c3a0 0x7f19e80cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.518+0000 7f19f02a2640 1 -- 172.21.15.70:0/3818488394 >> 172.21.15.70:0/3818488394 conn(0x7f19e806a5d0 msgr2=0x7f19e80b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.519+0000 7f19f02a2640 1 -- 172.21.15.70:0/3818488394 shutdown_connections 2023-10-08T07:14:09.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.519+0000 7f19f02a2640 1 -- 172.21.15.70:0/3818488394 wait complete. 2023-10-08T07:14:09.536 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-08T07:14:09.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-10-08T07:14:09.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.752+0000 7fe4fbe61640 1 Processor -- start 2023-10-08T07:14:09.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.752+0000 7fe4fbe61640 1 -- start start 2023-10-08T07:14:09.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.754+0000 7fe4fbe61640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe4f40afbe0 0x7fe4f40b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.754+0000 7fe4fbe61640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4f406dfc0 0x7fe4f4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.754+0000 7fe4fbe61640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f40b2620 con 0x7fe4f4071070 2023-10-08T07:14:09.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.754+0000 7fe4fbe61640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f40b2060 con 0x7fe4f406dfc0 2023-10-08T07:14:09.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.754+0000 7fe4fbe61640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f40b2340 con 0x7fe4f40afbe0 2023-10-08T07:14:09.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.755+0000 7fe4f93fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4f406dfc0 0x7fe4f4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.755+0000 7fe4f93fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4f406dfc0 0x7fe4f4070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33708/0 (socket says 172.21.15.70:33708) 2023-10-08T07:14:09.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.755+0000 7fe4f93fc640 1 -- 172.21.15.70:0/466091400 learned_addr learned my addr 172.21.15.70:0/466091400 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:09.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.755+0000 7fe4f9bfd640 1 --2- 172.21.15.70:0/466091400 >> v2:172.21.15.181:3300/0 conn(0x7fe4f40afbe0 0x7fe4f40b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.755+0000 7fe4f93fc640 1 -- 172.21.15.70:0/466091400 >> v2:172.21.15.181:3300/0 conn(0x7fe4f40afbe0 msgr2=0x7fe4f40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:09.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.755+0000 7fe4f93fc640 1 --2- 172.21.15.70:0/466091400 >> v2:172.21.15.181:3300/0 conn(0x7fe4f40afbe0 0x7fe4f40b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.755+0000 7fe4f93fc640 1 -- 172.21.15.70:0/466091400 >> v1:172.21.15.181:6789/0 conn(0x7fe4f4071070 legacy=0x7fe4f40cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:09.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.755+0000 7fe4f93fc640 1 -- 172.21.15.70:0/466091400 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f4070350 con 0x7fe4f406dfc0 2023-10-08T07:14:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.756+0000 7fe4f93fc640 1 --2- 172.21.15.70:0/466091400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 0x7fe4f4070b30 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fe4f406e640 tx=0x7fe4e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9e7a36aa5c9794 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.756+0000 7fe4f8bfb640 1 -- 172.21.15.70:0/466091400 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8031870 con 0x7fe4f406dfc0 2023-10-08T07:14:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.756+0000 7fe4f8bfb640 1 -- 172.21.15.70:0/466091400 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e80319d0 con 0x7fe4f406dfc0 2023-10-08T07:14:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.756+0000 7fe4f8bfb640 1 -- 172.21.15.70:0/466091400 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8031d60 con 0x7fe4f406dfc0 2023-10-08T07:14:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.757+0000 7fe4fbe61640 1 -- 172.21.15.70:0/466091400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 msgr2=0x7fe4f4070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.757+0000 7fe4fbe61640 1 --2- 172.21.15.70:0/466091400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 0x7fe4f4070b30 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fe4f406e640 tx=0x7fe4e802b910 comp rx=0 tx=0).stop 2023-10-08T07:14:09.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.757+0000 7fe4fbe61640 1 -- 172.21.15.70:0/466091400 shutdown_connections 2023-10-08T07:14:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.757+0000 7fe4fbe61640 1 --2- 172.21.15.70:0/466091400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 0x7fe4f4070b30 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.757+0000 7fe4fbe61640 1 --2- 172.21.15.70:0/466091400 >> v2:172.21.15.181:3300/0 conn(0x7fe4f40afbe0 0x7fe4f40b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.757+0000 7fe4fbe61640 1 -- 172.21.15.70:0/466091400 >> 172.21.15.70:0/466091400 conn(0x7fe4f406a5d0 msgr2=0x7fe4f40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.757+0000 7fe4fbe61640 1 -- 172.21.15.70:0/466091400 shutdown_connections 2023-10-08T07:14:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.757+0000 7fe4fbe61640 1 -- 172.21.15.70:0/466091400 wait complete. 2023-10-08T07:14:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.759+0000 7fe4fbe61640 1 Processor -- start 2023-10-08T07:14:09.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.759+0000 7fe4fbe61640 1 -- start start 2023-10-08T07:14:09.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.760+0000 7fe4fbe61640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 0x7fe4f4149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.760+0000 7fe4fbe61640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f4071070 0x7fe4f40cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.760+0000 7fe4fbe61640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4f40afbe0 0x7fe4f415c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.760+0000 7fe4fbe61640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40cf630 con 0x7fe4f406dfc0 2023-10-08T07:14:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.760+0000 7fe4fbe61640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f414a460 con 0x7fe4f4071070 2023-10-08T07:14:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.760+0000 7fe4fbe61640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f40cc7c0 con 0x7fe4f40afbe0 2023-10-08T07:14:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.760+0000 7fe4f9bfd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 0x7fe4f4149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.760+0000 7fe4f9bfd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 0x7fe4f4149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33712/0 (socket says 172.21.15.70:33712) 2023-10-08T07:14:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.760+0000 7fe4f9bfd640 1 -- 172.21.15.70:0/4165652945 learned_addr learned my addr 172.21.15.70:0/4165652945 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4f9bfd640 1 -- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4f40afbe0 msgr2=0x7fe4f415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:09.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4f93fc640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f4071070 0x7fe4f40cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4f9bfd640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4f40afbe0 0x7fe4f415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4f9bfd640 1 -- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f4071070 msgr2=0x7fe4f40cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:09.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4f9bfd640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f4071070 0x7fe4f40cec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:09.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4f9bfd640 1 -- 172.21.15.70:0/4165652945 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e80094d0 con 0x7fe4f406dfc0 2023-10-08T07:14:09.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4f9bfd640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 0x7fe4f4149f20 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4007f20 tx=0x7fe4e400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:09.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4f27fc640 1 -- 172.21.15.70:0/4165652945 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4014620 con 0x7fe4f406dfc0 2023-10-08T07:14:09.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4f27fc640 1 -- 172.21.15.70:0/4165652945 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e4014780 con 0x7fe4f406dfc0 2023-10-08T07:14:09.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4f27fc640 1 -- 172.21.15.70:0/4165652945 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4014b30 con 0x7fe4f406dfc0 2023-10-08T07:14:09.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4fbe61640 1 -- 172.21.15.70:0/4165652945 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f40ccaa0 con 0x7fe4f406dfc0 2023-10-08T07:14:09.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.761+0000 7fe4fbe61640 1 -- 172.21.15.70:0/4165652945 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f40cd060 con 0x7fe4f406dfc0 2023-10-08T07:14:09.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.762+0000 7fe4fbe61640 1 -- 172.21.15.70:0/4165652945 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f40cd220 con 0x7fe4f406dfc0 2023-10-08T07:14:09.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.763+0000 7fe4f27fc640 1 -- 172.21.15.70:0/4165652945 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 132) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fe4e4014c90 con 0x7fe4f406dfc0 2023-10-08T07:14:09.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.764+0000 7fe4f27fc640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fe4e006b100 0x7fe4e006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:09.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.764+0000 7fe4f27fc640 1 -- 172.21.15.70:0/4165652945 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fe4e408ab20 con 0x7fe4f406dfc0 2023-10-08T07:14:09.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.765+0000 7fe4f93fc640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fe4e006b100 0x7fe4e006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:09.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.766+0000 7fe4f93fc640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fe4e006b100 0x7fe4e006d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8036000 tx=0x7fe4e803c040 comp rx=0 tx=0).ready entity=mgr.8551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:09.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.769+0000 7fe4f27fc640 1 -- 172.21.15.70:0/4165652945 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4e4055230 con 0x7fe4f406dfc0 2023-10-08T07:14:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.980+0000 7fe4fbe61640 1 -- 172.21.15.70:0/4165652945 --> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4f40affc0 con 0x7fe4e006b100 2023-10-08T07:14:10.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:09.998+0000 7fe4f27fc640 1 -- 172.21.15.70:0/4165652945 <== mgr.8551 v2:172.21.15.181:6832/860795336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+338 (secure 0 0 0) 0x7fe4e8031c40 con 0x7fe4e006b100 2023-10-08T07:14:10.002 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:10.002 INFO:teuthology.orchestra.run.smithi070.stdout:[{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T07:14:07", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-08T07:14:10.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.000+0000 7fe4fbe61640 1 -- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fe4e006b100 msgr2=0x7fe4e006d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.000+0000 7fe4fbe61640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fe4e006b100 0x7fe4e006d5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8036000 tx=0x7fe4e803c040 comp rx=0 tx=0).stop 2023-10-08T07:14:10.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.000+0000 7fe4fbe61640 1 -- 172.21.15.70:0/4165652945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 msgr2=0x7fe4f4149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.000+0000 7fe4fbe61640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 0x7fe4f4149f20 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4007f20 tx=0x7fe4e400e570 comp rx=0 tx=0).stop 2023-10-08T07:14:10.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.001+0000 7fe4fbe61640 1 -- 172.21.15.70:0/4165652945 shutdown_connections 2023-10-08T07:14:10.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.001+0000 7fe4fbe61640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fe4e006b100 0x7fe4e006d5c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.001+0000 7fe4fbe61640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe4f40afbe0 0x7fe4f415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.001+0000 7fe4fbe61640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe4f4071070 0x7fe4f40cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.001+0000 7fe4fbe61640 1 --2- 172.21.15.70:0/4165652945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f406dfc0 0x7fe4f4149f20 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.001+0000 7fe4fbe61640 1 -- 172.21.15.70:0/4165652945 >> 172.21.15.70:0/4165652945 conn(0x7fe4f406a5d0 msgr2=0x7fe4f415acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:10.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.001+0000 7fe4fbe61640 1 -- 172.21.15.70:0/4165652945 shutdown_connections 2023-10-08T07:14:10.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.001+0000 7fe4fbe61640 1 -- 172.21.15.70:0/4165652945 wait complete. 2023-10-08T07:14:10.022 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T07:14:07", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-08T07:14:10.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-08T07:14:10.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.241+0000 7fbae9221640 1 Processor -- start 2023-10-08T07:14:10.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.242+0000 7fbae9221640 1 -- start start 2023-10-08T07:14:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.243+0000 7fbae9221640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbae415ce00 0x7fbae40cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.243+0000 7fbae9221640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbae40cd470 0x7fbae40cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.243+0000 7fbae9221640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae40cde10 con 0x7fbae4160980 2023-10-08T07:14:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.243+0000 7fbae9221640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae4161920 con 0x7fbae415ce00 2023-10-08T07:14:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.243+0000 7fbae9221640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae40cb8b0 con 0x7fbae40cd470 2023-10-08T07:14:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.243+0000 7fbae2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbae415ce00 0x7fbae40cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.243+0000 7fbae359e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbae4160980 0x7fbae4160dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40874/0 (socket says 172.21.15.70:40874) 2023-10-08T07:14:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.243+0000 7fbae359e640 1 -- 172.21.15.70:0/1240254739 learned_addr learned my addr 172.21.15.70:0/1240254739 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.244+0000 7fbae2d9d640 1 -- 172.21.15.70:0/1240254739 >> v2:172.21.15.181:3300/0 conn(0x7fbae40cd470 msgr2=0x7fbae40cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.244+0000 7fbae2d9d640 1 --2- 172.21.15.70:0/1240254739 >> v2:172.21.15.181:3300/0 conn(0x7fbae40cd470 0x7fbae40cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.244+0000 7fbae2d9d640 1 -- 172.21.15.70:0/1240254739 >> v1:172.21.15.70:6789/0 conn(0x7fbae4160980 legacy=0x7fbae4160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.244+0000 7fbae2d9d640 1 -- 172.21.15.70:0/1240254739 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae4161640 con 0x7fbae415ce00 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.245+0000 7fbae2d9d640 1 --2- 172.21.15.70:0/1240254739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae415ce00 0x7fbae40cb370 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fbae406fa70 tx=0x7fbad402be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2443a2c217d6fa93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.245+0000 7fbae1d9b640 1 -- 172.21.15.70:0/1240254739 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad402cc20 con 0x7fbae415ce00 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.245+0000 7fbae1d9b640 1 -- 172.21.15.70:0/1240254739 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad402cd80 con 0x7fbae415ce00 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.245+0000 7fbae1d9b640 1 -- 172.21.15.70:0/1240254739 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad4034660 con 0x7fbae415ce00 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.246+0000 7fbae9221640 1 -- 172.21.15.70:0/1240254739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae415ce00 msgr2=0x7fbae40cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.246+0000 7fbae9221640 1 --2- 172.21.15.70:0/1240254739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae415ce00 0x7fbae40cb370 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fbae406fa70 tx=0x7fbad402be70 comp rx=0 tx=0).stop 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.246+0000 7fbae9221640 1 -- 172.21.15.70:0/1240254739 shutdown_connections 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.246+0000 7fbae9221640 1 --2- 172.21.15.70:0/1240254739 >> v2:172.21.15.181:3300/0 conn(0x7fbae40cd470 0x7fbae40cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.246+0000 7fbae9221640 1 --2- 172.21.15.70:0/1240254739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae415ce00 0x7fbae40cb370 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.246+0000 7fbae9221640 1 -- 172.21.15.70:0/1240254739 >> 172.21.15.70:0/1240254739 conn(0x7fbae406e650 msgr2=0x7fbae406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:10.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.246+0000 7fbae9221640 1 -- 172.21.15.70:0/1240254739 shutdown_connections 2023-10-08T07:14:10.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.246+0000 7fbae9221640 1 -- 172.21.15.70:0/1240254739 wait complete. 2023-10-08T07:14:10.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.248+0000 7fbae9221640 1 Processor -- start 2023-10-08T07:14:10.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.248+0000 7fbae9221640 1 -- start start 2023-10-08T07:14:10.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae9221640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae40cd470 0x7fbae40c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae9221640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbae415ce00 0x7fbae40b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae9221640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae4160980 0x7fbae40b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae9221640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40c7190 con 0x7fbae4160980 2023-10-08T07:14:10.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae9221640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae415d5a0 con 0x7fbae40cd470 2023-10-08T07:14:10.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae9221640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40c6bd0 con 0x7fbae415ce00 2023-10-08T07:14:10.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae4160980 0x7fbae40b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:10.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae4160980 0x7fbae40b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33734/0 (socket says 172.21.15.70:33734) 2023-10-08T07:14:10.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae359e640 1 -- 172.21.15.70:0/2431767340 learned_addr learned my addr 172.21.15.70:0/2431767340 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:10.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae359e640 1 -- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbae415ce00 msgr2=0x7fbae40b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae359e640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbae415ce00 0x7fbae40b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae359e640 1 -- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae40cd470 msgr2=0x7fbae40c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae359e640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae40cd470 0x7fbae40c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.249+0000 7fbae359e640 1 -- 172.21.15.70:0/2431767340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad4009a70 con 0x7fbae4160980 2023-10-08T07:14:10.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.250+0000 7fbae359e640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae4160980 0x7fbae40b4c10 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00c420 tx=0x7fbadc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:10.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.250+0000 7fbacb7fe640 1 -- 172.21.15.70:0/2431767340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc005ce0 con 0x7fbae4160980 2023-10-08T07:14:10.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.250+0000 7fbacb7fe640 1 -- 172.21.15.70:0/2431767340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc005e40 con 0x7fbae4160980 2023-10-08T07:14:10.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.250+0000 7fbacb7fe640 1 -- 172.21.15.70:0/2431767340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc0072d0 con 0x7fbae4160980 2023-10-08T07:14:10.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.250+0000 7fbae9221640 1 -- 172.21.15.70:0/2431767340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae40c6eb0 con 0x7fbae4160980 2023-10-08T07:14:10.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.250+0000 7fbae9221640 1 -- 172.21.15.70:0/2431767340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae4154140 con 0x7fbae4160980 2023-10-08T07:14:10.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.251+0000 7fbae9221640 1 -- 172.21.15.70:0/2431767340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae415d300 con 0x7fbae4160980 2023-10-08T07:14:10.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.252+0000 7fbacb7fe640 1 -- 172.21.15.70:0/2431767340 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 132) v1 ==== 89451+0+0 (secure 0 0 0) 0x7fbadc007430 con 0x7fbae4160980 2023-10-08T07:14:10.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.253+0000 7fbacb7fe640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fbad006b130 0x7fbad006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.253+0000 7fbacb7fe640 1 -- 172.21.15.70:0/2431767340 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fbadc0894d0 con 0x7fbae4160980 2023-10-08T07:14:10.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.254+0000 7fbae2d9d640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fbad006b130 0x7fbad006d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:10.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.255+0000 7fbae2d9d640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fbad006b130 0x7fbad006d5f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbad4036000 tx=0x7fbad402e000 comp rx=0 tx=0).ready entity=mgr.8551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:10.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.257+0000 7fbacb7fe640 1 -- 172.21.15.70:0/2431767340 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbae415d300 con 0x7fbae4160980 2023-10-08T07:14:10.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.466+0000 7fbae9221640 1 -- 172.21.15.70:0/2431767340 --> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbae40c6d40 con 0x7fbad006b130 2023-10-08T07:14:10.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.605+0000 7fbacb7fe640 1 -- 172.21.15.70:0/2431767340 <== mgr.8551 v2:172.21.15.181:6832/860795336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fbae40c6d40 con 0x7fbad006b130 2023-10-08T07:14:10.607 INFO:teuthology.orchestra.run.smithi070.stdout:Retention removed from path mandatory_fs_argument_test_dir 2023-10-08T07:14:10.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.608+0000 7fbae9221640 1 -- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fbad006b130 msgr2=0x7fbad006d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.608+0000 7fbae9221640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fbad006b130 0x7fbad006d5f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbad4036000 tx=0x7fbad402e000 comp rx=0 tx=0).stop 2023-10-08T07:14:10.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.608+0000 7fbae9221640 1 -- 172.21.15.70:0/2431767340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae4160980 msgr2=0x7fbae40b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.608+0000 7fbae9221640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae4160980 0x7fbae40b4c10 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00c420 tx=0x7fbadc00c8e0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.608+0000 7fbae9221640 1 -- 172.21.15.70:0/2431767340 shutdown_connections 2023-10-08T07:14:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.608+0000 7fbae9221640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fbad006b130 0x7fbad006d5f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.608+0000 7fbae9221640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbae4160980 0x7fbae40b4c10 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.608+0000 7fbae9221640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbae415ce00 0x7fbae40b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.608+0000 7fbae9221640 1 --2- 172.21.15.70:0/2431767340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbae40cd470 0x7fbae40c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.608+0000 7fbae9221640 1 -- 172.21.15.70:0/2431767340 >> 172.21.15.70:0/2431767340 conn(0x7fbae406e650 msgr2=0x7fbae40c8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.609+0000 7fbae9221640 1 -- 172.21.15.70:0/2431767340 shutdown_connections 2023-10-08T07:14:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.609+0000 7fbae9221640 1 -- 172.21.15.70:0/2431767340 wait complete. 2023-10-08T07:14:10.631 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention removed from path mandatory_fs_argument_test_dir 2023-10-08T07:14:10.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-08T07:14:10.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.847+0000 7f87ebbe9640 1 Processor -- start 2023-10-08T07:14:10.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.848+0000 7f87ebbe9640 1 -- start start 2023-10-08T07:14:10.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.849+0000 7f87ebbe9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87e40cb620 0x7f87e40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.849+0000 7f87ebbe9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e4071170 con 0x7f87e4160660 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.849+0000 7f87ebbe9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e406e150 con 0x7f87e4070cc0 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.849+0000 7f87ebbe9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e406de70 con 0x7f87e40cb620 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.850+0000 7f87ea186640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87e40cb620 0x7f87e40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.850+0000 7f87ea186640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87e40cb620 0x7f87e40b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33750/0 (socket says 172.21.15.70:33750) 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.850+0000 7f87ea186640 1 -- 172.21.15.70:0/4266363036 learned_addr learned my addr 172.21.15.70:0/4266363036 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.850+0000 7f87e9985640 1 --1- 172.21.15.70:0/4266363036 >> v1:172.21.15.70:6789/0 conn(0x7f87e4160660 0x7f87e40c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40880/0 (socket says 172.21.15.70:40880) 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.850+0000 7f87ea186640 1 -- 172.21.15.70:0/4266363036 >> v1:172.21.15.181:6789/0 conn(0x7f87e4070cc0 legacy=0x7f87e40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.850+0000 7f87ea186640 1 -- 172.21.15.70:0/4266363036 >> v1:172.21.15.70:6789/0 conn(0x7f87e4160660 legacy=0x7f87e40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.850+0000 7f87ea186640 1 -- 172.21.15.70:0/4266363036 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e40c8e30 con 0x7f87e40cb620 2023-10-08T07:14:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.850+0000 7f87e8983640 1 -- 172.21.15.70:0/4266363036 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394095473 0 0) 0x7f87e0001110 con 0x7f87e4160660 2023-10-08T07:14:10.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.851+0000 7f87ea186640 1 --2- 172.21.15.70:0/4266363036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e40cb620 0x7f87e40b1990 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f87e406e340 tx=0x7f87dc02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f9eace6566ee15 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:10.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.851+0000 7f87e8983640 1 -- 172.21.15.70:0/4266363036 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc02ca60 con 0x7f87e40cb620 2023-10-08T07:14:10.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.852+0000 7f87e8983640 1 -- 172.21.15.70:0/4266363036 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc02cbc0 con 0x7f87e40cb620 2023-10-08T07:14:10.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.852+0000 7f87e8983640 1 -- 172.21.15.70:0/4266363036 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc0343f0 con 0x7f87e40cb620 2023-10-08T07:14:10.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.852+0000 7f87ebbe9640 1 -- 172.21.15.70:0/4266363036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e40cb620 msgr2=0x7f87e40b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.852+0000 7f87ebbe9640 1 --2- 172.21.15.70:0/4266363036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e40cb620 0x7f87e40b1990 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f87e406e340 tx=0x7f87dc02bec0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.852+0000 7f87ebbe9640 1 -- 172.21.15.70:0/4266363036 shutdown_connections 2023-10-08T07:14:10.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.852+0000 7f87ebbe9640 1 --2- 172.21.15.70:0/4266363036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e40cb620 0x7f87e40b1990 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.852+0000 7f87ebbe9640 1 -- 172.21.15.70:0/4266363036 >> 172.21.15.70:0/4266363036 conn(0x7f87e406a5d0 msgr2=0x7f87e40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:10.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.852+0000 7f87ebbe9640 1 -- 172.21.15.70:0/4266363036 shutdown_connections 2023-10-08T07:14:10.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.852+0000 7f87ebbe9640 1 -- 172.21.15.70:0/4266363036 wait complete. 2023-10-08T07:14:10.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.854+0000 7f87ebbe9640 1 Processor -- start 2023-10-08T07:14:10.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.854+0000 7f87ebbe9640 1 -- start start 2023-10-08T07:14:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.855+0000 7f87ebbe9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e4070cc0 0x7f87e415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.855+0000 7f87ebbe9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87e40cb620 0x7f87e40ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.855+0000 7f87ebbe9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e4160660 0x7f87e40c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.855+0000 7f87ebbe9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e41502c0 con 0x7f87e4070cc0 2023-10-08T07:14:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.855+0000 7f87ebbe9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40ae570 con 0x7f87e4160660 2023-10-08T07:14:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.855+0000 7f87ebbe9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40abe10 con 0x7f87e40cb620 2023-10-08T07:14:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.855+0000 7f87e9985640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e4070cc0 0x7f87e415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87e9985640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e4070cc0 0x7f87e415cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33752/0 (socket says 172.21.15.70:33752) 2023-10-08T07:14:10.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87e9985640 1 -- 172.21.15.70:0/324150161 learned_addr learned my addr 172.21.15.70:0/324150161 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:10.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87e9985640 1 -- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87e40cb620 msgr2=0x7f87e40ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87e9985640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87e40cb620 0x7f87e40ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87e9985640 1 -- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e4160660 msgr2=0x7f87e40c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:10.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87e9985640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e4160660 0x7f87e40c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:10.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87e9985640 1 -- 172.21.15.70:0/324150161 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc009b30 con 0x7f87e4070cc0 2023-10-08T07:14:10.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87e9985640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e4070cc0 0x7f87e415cda0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f87e000c430 tx=0x7f87e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:10.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87da7fc640 1 -- 172.21.15.70:0/324150161 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e00074e0 con 0x7f87e4070cc0 2023-10-08T07:14:10.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87da7fc640 1 -- 172.21.15.70:0/324150161 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e0007660 con 0x7f87e4070cc0 2023-10-08T07:14:10.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.856+0000 7f87ebbe9640 1 -- 172.21.15.70:0/324150161 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e415bbe0 con 0x7f87e4070cc0 2023-10-08T07:14:10.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.857+0000 7f87da7fc640 1 -- 172.21.15.70:0/324150161 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0004a20 con 0x7f87e4070cc0 2023-10-08T07:14:10.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.857+0000 7f87ebbe9640 1 -- 172.21.15.70:0/324150161 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e415c1a0 con 0x7f87e4070cc0 2023-10-08T07:14:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.858+0000 7f87ebbe9640 1 -- 172.21.15.70:0/324150161 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e404e160 con 0x7f87e4070cc0 2023-10-08T07:14:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.859+0000 7f87da7fc640 1 -- 172.21.15.70:0/324150161 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 133) v1 ==== 89507+0+0 (secure 0 0 0) 0x7f87e0004b80 con 0x7f87e4070cc0 2023-10-08T07:14:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.860+0000 7f87da7fc640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87c806b1c0 0x7f87c806d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.860+0000 7f87da7fc640 1 -- 172.21.15.70:0/324150161 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f87e0089e40 con 0x7f87e4070cc0 2023-10-08T07:14:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.861+0000 7f87e9184640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87c806b1c0 0x7f87c806d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.862+0000 7f87e9184640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87c806b1c0 0x7f87c806d680 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f87e40af8e0 tx=0x7f87d40079b0 comp rx=0 tx=0).ready entity=mgr.8551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:10.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:10.865+0000 7f87da7fc640 1 -- 172.21.15.70:0/324150161 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87e0054450 con 0x7f87e4070cc0 2023-10-08T07:14:11.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.076+0000 7f87ebbe9640 1 -- 172.21.15.70:0/324150161 --> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f87e40c93f0 con 0x7f87c806b1c0 2023-10-08T07:14:11.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.217+0000 7f87da7fc640 1 -- 172.21.15.70:0/324150161 <== mgr.8551 v2:172.21.15.181:6832/860795336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+61 (secure 0 0 0) 0x7f87d4014400 con 0x7f87c806b1c0 2023-10-08T07:14:11.219 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule deactivated for path mandatory_fs_argument_test_dir 2023-10-08T07:14:11.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.220+0000 7f87ebbe9640 1 -- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87c806b1c0 msgr2=0x7f87c806d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:11.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.220+0000 7f87ebbe9640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87c806b1c0 0x7f87c806d680 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f87e40af8e0 tx=0x7f87d40079b0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.220+0000 7f87ebbe9640 1 -- 172.21.15.70:0/324150161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e4070cc0 msgr2=0x7f87e415cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:11.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.220+0000 7f87ebbe9640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e4070cc0 0x7f87e415cda0 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f87e000c430 tx=0x7f87e000c8f0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.221+0000 7f87ebbe9640 1 -- 172.21.15.70:0/324150161 shutdown_connections 2023-10-08T07:14:11.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.221+0000 7f87ebbe9640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87c806b1c0 0x7f87c806d680 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.221+0000 7f87ebbe9640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87e4160660 0x7f87e40c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.221+0000 7f87ebbe9640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87e40cb620 0x7f87e40ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.221+0000 7f87ebbe9640 1 --2- 172.21.15.70:0/324150161 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87e4070cc0 0x7f87e415cda0 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.221+0000 7f87ebbe9640 1 -- 172.21.15.70:0/324150161 >> 172.21.15.70:0/324150161 conn(0x7f87e406a5d0 msgr2=0x7f87e40cdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:11.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.221+0000 7f87ebbe9640 1 -- 172.21.15.70:0/324150161 shutdown_connections 2023-10-08T07:14:11.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.221+0000 7f87ebbe9640 1 -- 172.21.15.70:0/324150161 wait complete. 2023-10-08T07:14:11.243 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path mandatory_fs_argument_test_dir 2023-10-08T07:14:11.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-10-08T07:14:11.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.464+0000 7f7ebe9b6640 1 Processor -- start 2023-10-08T07:14:11.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.464+0000 7f7ebe9b6640 1 -- start start 2023-10-08T07:14:11.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.466+0000 7f7ebe9b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb806dfc0 0x7f7eb8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:11.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.466+0000 7f7ebe9b6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb80b1cc0 con 0x7f7eb8071070 2023-10-08T07:14:11.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.466+0000 7f7ebe9b6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb80b2620 con 0x7f7eb80afbe0 2023-10-08T07:14:11.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.466+0000 7f7ebe9b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80b2060 con 0x7f7eb806dfc0 2023-10-08T07:14:11.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.466+0000 7f7ebcf53640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7eb8071070 0x7f7eb80cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40884/0 (socket says 172.21.15.70:40884) 2023-10-08T07:14:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.466+0000 7f7ebcf53640 1 -- 172.21.15.70:0/4221823034 learned_addr learned my addr 172.21.15.70:0/4221823034 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.466+0000 7f7eb77fe640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb806dfc0 0x7f7eb8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363802836 0 0) 0x7f7eb80b1cc0 con 0x7f7eb8071070 2023-10-08T07:14:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea0003520 con 0x7f7eb8071070 2023-10-08T07:14:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1423164510 0 0) 0x7f7ea0003520 con 0x7f7eb8071070 2023-10-08T07:14:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb806dfc0 msgr2=0x7f7eb8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb806dfc0 0x7f7eb8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 >> v1:172.21.15.181:6789/0 conn(0x7f7eb80afbe0 legacy=0x7f7eb80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb80b2340 con 0x7f7eb8071070 2023-10-08T07:14:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7eb00019f0 con 0x7f7eb8071070 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 >> v1:172.21.15.70:6789/0 conn(0x7f7eb8071070 legacy=0x7f7eb80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb806dfc0 0x7f7ea00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ea0003f70 0x7f7ea00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 0x7f7ea0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80b2060 con 0x7f7ea0004d00 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea000bac0 con 0x7f7ea0003f70 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.467+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea000bc80 con 0x7f7eb806dfc0 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.468+0000 7f7ebcf53640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 0x7f7ea0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.468+0000 7f7eb77fe640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ea0003f70 0x7f7ea00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.468+0000 7f7eb7fff640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb806dfc0 0x7f7ea00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:11.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.468+0000 7f7ebcf53640 1 -- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb806dfc0 msgr2=0x7f7ea00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:11.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.468+0000 7f7ebcf53640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb806dfc0 0x7f7ea00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.468+0000 7f7ebcf53640 1 -- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ea0003f70 msgr2=0x7f7ea00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:11.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.468+0000 7f7ebcf53640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ea0003f70 0x7f7ea00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.468+0000 7f7ebcf53640 1 -- 172.21.15.70:0/4221823034 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0003520 con 0x7f7ea0004d00 2023-10-08T07:14:11.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.468+0000 7f7eb7fff640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb806dfc0 0x7f7ea00039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:11.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.469+0000 7f7ebcf53640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 0x7f7ea0005180 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000a760 tx=0x7f7ea00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02359c5a12b592a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:11.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.469+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0034630 con 0x7f7ea0004d00 2023-10-08T07:14:11.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.469+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb002f070 con 0x7f7ea0004d00 2023-10-08T07:14:11.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.469+0000 7f7eb6ffd640 1 -- 172.21.15.70:0/4221823034 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0034a50 con 0x7f7ea0004d00 2023-10-08T07:14:11.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.470+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/4221823034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 msgr2=0x7f7ea0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:11.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.470+0000 7f7ebe9b6640 1 --2- 172.21.15.70:0/4221823034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 0x7f7ea0005180 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000a760 tx=0x7f7ea00057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.470+0000 7f7ebcf53640 1 -- 172.21.15.70:0/4221823034 reap_dead start 2023-10-08T07:14:11.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.470+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/4221823034 shutdown_connections 2023-10-08T07:14:11.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.470+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/4221823034 >> 172.21.15.70:0/4221823034 conn(0x7f7eb806a5d0 msgr2=0x7f7eb80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:11.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.470+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/4221823034 shutdown_connections 2023-10-08T07:14:11.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.470+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/4221823034 wait complete. 2023-10-08T07:14:11.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.473+0000 7f7ebe9b6640 1 Processor -- start 2023-10-08T07:14:11.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.473+0000 7f7ebe9b6640 1 -- start start 2023-10-08T07:14:11.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.474+0000 7f7ebe9b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 0x7f7eb814fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:11.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.474+0000 7f7ebe9b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb80c6d50 0x7f7eb80c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:11.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.474+0000 7f7ebe9b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb815c9f0 0x7f7eb815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:11.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.474+0000 7f7ebe9b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb815d3b0 con 0x7f7ea0004d00 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.474+0000 7f7ebe9b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80c7e20 con 0x7f7eb815c9f0 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.474+0000 7f7ebe9b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb80c9ce0 con 0x7f7eb80c6d50 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.474+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 0x7f7eb814fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.474+0000 7f7eb7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 0x7f7eb814fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33768/0 (socket says 172.21.15.70:33768) 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.474+0000 7f7eb7fff640 1 -- 172.21.15.70:0/1168927274 learned_addr learned my addr 172.21.15.70:0/1168927274 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.475+0000 7f7eb77fe640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb80c6d50 0x7f7eb80c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.475+0000 7f7ebcf53640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb815c9f0 0x7f7eb815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.475+0000 7f7eb7fff640 1 -- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb80c6d50 msgr2=0x7f7eb80c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.475+0000 7f7eb7fff640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb80c6d50 0x7f7eb80c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.475+0000 7f7eb7fff640 1 -- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb815c9f0 msgr2=0x7f7eb815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.475+0000 7f7eb7fff640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb815c9f0 0x7f7eb815ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.475+0000 7f7eb7fff640 1 -- 172.21.15.70:0/1168927274 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb80c72a0 con 0x7f7ea0004d00 2023-10-08T07:14:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.475+0000 7f7ebcf53640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb815c9f0 0x7f7eb815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.475+0000 7f7eb7fff640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 0x7f7eb814fe10 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d7a0 tx=0x7f7ea800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.476+0000 7f7eb4ff9640 1 -- 172.21.15.70:0/1168927274 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8005270 con 0x7f7ea0004d00 2023-10-08T07:14:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.476+0000 7f7eb4ff9640 1 -- 172.21.15.70:0/1168927274 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea80053d0 con 0x7f7ea0004d00 2023-10-08T07:14:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.476+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/1168927274 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb80c7580 con 0x7f7ea0004d00 2023-10-08T07:14:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.476+0000 7f7eb4ff9640 1 -- 172.21.15.70:0/1168927274 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800b8d0 con 0x7f7ea0004d00 2023-10-08T07:14:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.476+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/1168927274 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb80c7b40 con 0x7f7ea0004d00 2023-10-08T07:14:11.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.477+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/1168927274 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb80ca870 con 0x7f7ea0004d00 2023-10-08T07:14:11.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.478+0000 7f7eb4ff9640 1 -- 172.21.15.70:0/1168927274 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 133) v1 ==== 89507+0+0 (secure 0 0 0) 0x7f7ea800ba30 con 0x7f7ea0004d00 2023-10-08T07:14:11.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.479+0000 7f7eb4ff9640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f7ea406b190 0x7f7ea406d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.479+0000 7f7eb4ff9640 1 -- 172.21.15.70:0/1168927274 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f7ea808a150 con 0x7f7ea0004d00 2023-10-08T07:14:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.480+0000 7f7eb77fe640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f7ea406b190 0x7f7ea406d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:11.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.481+0000 7f7eb77fe640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f7ea406b190 0x7f7ea406d650 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80b18b0 tx=0x7f7eac008040 comp rx=0 tx=0).ready entity=mgr.8551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:11.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.483+0000 7f7eb4ff9640 1 -- 172.21.15.70:0/1168927274 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ea8054780 con 0x7f7ea0004d00 2023-10-08T07:14:11.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.697+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/1168927274 --> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7eb8070630 con 0x7f7ea406b190 2023-10-08T07:14:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.865+0000 7f7eb4ff9640 1 -- 172.21.15.70:0/1168927274 <== mgr.8551 v2:172.21.15.181:6832/860795336 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f7eac009680 con 0x7f7ea406b190 2023-10-08T07:14:11.868 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path mandatory_fs_argument_test_dir 2023-10-08T07:14:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.869+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f7ea406b190 msgr2=0x7f7ea406d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.869+0000 7f7ebe9b6640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f7ea406b190 0x7f7ea406d650 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80b18b0 tx=0x7f7eac008040 comp rx=0 tx=0).stop 2023-10-08T07:14:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.869+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/1168927274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 msgr2=0x7f7eb814fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.869+0000 7f7ebe9b6640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 0x7f7eb814fe10 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d7a0 tx=0x7f7ea800dc60 comp rx=0 tx=0).stop 2023-10-08T07:14:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.869+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/1168927274 shutdown_connections 2023-10-08T07:14:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.869+0000 7f7ebe9b6640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f7ea406b190 0x7f7ea406d650 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.869+0000 7f7ebe9b6640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7eb815c9f0 0x7f7eb815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.869+0000 7f7ebe9b6640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7eb80c6d50 0x7f7eb80c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.869+0000 7f7ebe9b6640 1 --2- 172.21.15.70:0/1168927274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ea0004d00 0x7f7eb814fe10 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.869+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/1168927274 >> 172.21.15.70:0/1168927274 conn(0x7f7eb806a5d0 msgr2=0x7f7eb80cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.870+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/1168927274 shutdown_connections 2023-10-08T07:14:11.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:11.870+0000 7f7ebe9b6640 1 -- 172.21.15.70:0/1168927274 wait complete. 2023-10-08T07:14:11.892 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path mandatory_fs_argument_test_dir 2023-10-08T07:14:11.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:11.892 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-10-08T07:14:11.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T07:14:12.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.159+0000 7fa04ece5640 1 Processor -- start 2023-10-08T07:14:12.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.159+0000 7fa04ece5640 1 -- start start 2023-10-08T07:14:12.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.161+0000 7fa04ece5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa048071260 0x7fa0480c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:12.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.161+0000 7fa04ece5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0480c7ad0 con 0x7fa04815a820 2023-10-08T07:14:12.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.161+0000 7fa04ece5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa048160e60 con 0x7fa0480c9080 2023-10-08T07:14:12.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.161+0000 7fa04ece5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0480c9a40 con 0x7fa048071260 2023-10-08T07:14:12.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.162+0000 7fa04ca81640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa04815a820 0x7fa048160750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40890/0 (socket says 172.21.15.70:40890) 2023-10-08T07:14:12.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.162+0000 7fa04ca81640 1 -- 172.21.15.70:0/3368938363 learned_addr learned my addr 172.21.15.70:0/3368938363 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:12.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.162+0000 7fa04d282640 1 --2- 172.21.15.70:0/3368938363 >> v2:172.21.15.181:3300/0 conn(0x7fa048071260 0x7fa0480c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:12.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.162+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964647523 0 0) 0x7fa0480c7ad0 con 0x7fa04815a820 2023-10-08T07:14:12.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.162+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa038003520 con 0x7fa04815a820 2023-10-08T07:14:12.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1495161026 0 0) 0x7fa038003520 con 0x7fa04815a820 2023-10-08T07:14:12.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 >> v2:172.21.15.181:3300/0 conn(0x7fa048071260 msgr2=0x7fa0480c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:12.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 --2- 172.21.15.70:0/3368938363 >> v2:172.21.15.181:3300/0 conn(0x7fa048071260 0x7fa0480c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 >> v1:172.21.15.181:6789/0 conn(0x7fa0480c9080 legacy=0x7fa048070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0480cb210 con 0x7fa04815a820 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa03c001670 con 0x7fa04815a820 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 >> v1:172.21.15.70:6789/0 conn(0x7fa04815a820 legacy=0x7fa048160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 --2- 172.21.15.70:0/3368938363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0380039c0 0x7fa038003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 --2- 172.21.15.70:0/3368938363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0380047c0 0x7fa038004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 --2- 172.21.15.70:0/3368938363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa038005180 0x7fa03800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480c9a40 con 0x7fa038005180 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03800c080 con 0x7fa0380047c0 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03800c240 con 0x7fa0380039c0 2023-10-08T07:14:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa04d282640 1 --2- 172.21.15.70:0/3368938363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa038005180 0x7fa03800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:12.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.163+0000 7fa047fff640 1 --2- 172.21.15.70:0/3368938363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0380047c0 0x7fa038004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:12.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.164+0000 7fa04d282640 1 -- 172.21.15.70:0/3368938363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0380039c0 msgr2=0x7fa038003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:12.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.164+0000 7fa04d282640 1 --2- 172.21.15.70:0/3368938363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa0380039c0 0x7fa038003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.164+0000 7fa04d282640 1 -- 172.21.15.70:0/3368938363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0380047c0 msgr2=0x7fa038004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:12.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.164+0000 7fa04d282640 1 --2- 172.21.15.70:0/3368938363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0380047c0 0x7fa038004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.164+0000 7fa04d282640 1 -- 172.21.15.70:0/3368938363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa038003520 con 0x7fa038005180 2023-10-08T07:14:12.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.165+0000 7fa04d282640 1 -- 172.21.15.70:0/3368938363 reap_dead start 2023-10-08T07:14:12.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.165+0000 7fa04d282640 1 --2- 172.21.15.70:0/3368938363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa038005180 0x7fa03800b980 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fa040009f90 tx=0x7fa0380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1a98a111cd3e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:12.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.165+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04002cc20 con 0x7fa038005180 2023-10-08T07:14:12.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.165+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04000f040 con 0x7fa038005180 2023-10-08T07:14:12.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.165+0000 7fa0477fe640 1 -- 172.21.15.70:0/3368938363 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040034440 con 0x7fa038005180 2023-10-08T07:14:12.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.166+0000 7fa04ece5640 1 -- 172.21.15.70:0/3368938363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa038005180 msgr2=0x7fa03800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:12.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.166+0000 7fa04ece5640 1 --2- 172.21.15.70:0/3368938363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa038005180 0x7fa03800b980 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7fa040009f90 tx=0x7fa0380057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.166+0000 7fa04ece5640 1 -- 172.21.15.70:0/3368938363 shutdown_connections 2023-10-08T07:14:12.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.166+0000 7fa04ece5640 1 --2- 172.21.15.70:0/3368938363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa038005180 0x7fa03800b980 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.166+0000 7fa04ece5640 1 -- 172.21.15.70:0/3368938363 >> 172.21.15.70:0/3368938363 conn(0x7fa048069d50 msgr2=0x7fa04806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:12.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.166+0000 7fa04ece5640 1 -- 172.21.15.70:0/3368938363 shutdown_connections 2023-10-08T07:14:12.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.166+0000 7fa04ece5640 1 -- 172.21.15.70:0/3368938363 wait complete. 2023-10-08T07:14:12.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.168+0000 7fa04ece5640 1 Processor -- start 2023-10-08T07:14:12.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.168+0000 7fa04ece5640 1 -- start start 2023-10-08T07:14:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.169+0000 7fa04ece5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa038005180 0x7fa0481f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.169+0000 7fa04ece5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0481535c0 0x7fa0480c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.169+0000 7fa04ece5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa04814abb0 0x7fa04814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.169+0000 7fa04ece5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480b1c80 con 0x7fa0481535c0 2023-10-08T07:14:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.169+0000 7fa04ece5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04814b570 con 0x7fa04814abb0 2023-10-08T07:14:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.169+0000 7fa04ece5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa048153da0 con 0x7fa038005180 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.169+0000 7fa047fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0481535c0 0x7fa0480c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.169+0000 7fa047fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0481535c0 0x7fa0480c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33782/0 (socket says 172.21.15.70:33782) 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.169+0000 7fa047fff640 1 -- 172.21.15.70:0/300943323 learned_addr learned my addr 172.21.15.70:0/300943323 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa04ca81640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa038005180 0x7fa0481f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa04d282640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa04814abb0 0x7fa04814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa047fff640 1 -- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa038005180 msgr2=0x7fa0481f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa047fff640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa038005180 0x7fa0481f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa047fff640 1 -- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa04814abb0 msgr2=0x7fa04814b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa047fff640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa04814abb0 0x7fa04814b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa047fff640 1 -- 172.21.15.70:0/300943323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0480b1670 con 0x7fa0481535c0 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa04d282640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa04814abb0 0x7fa04814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa047fff640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0481535c0 0x7fa0480c80d0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7fa03400e440 tx=0x7fa03400e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa0457fa640 1 -- 172.21.15.70:0/300943323 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340052e0 con 0x7fa0481535c0 2023-10-08T07:14:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa0457fa640 1 -- 172.21.15.70:0/300943323 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa034005440 con 0x7fa0481535c0 2023-10-08T07:14:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.170+0000 7fa0457fa640 1 -- 172.21.15.70:0/300943323 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa034003810 con 0x7fa0481535c0 2023-10-08T07:14:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.171+0000 7fa04ece5640 1 -- 172.21.15.70:0/300943323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa048153ac0 con 0x7fa0481535c0 2023-10-08T07:14:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.171+0000 7fa04ece5640 1 -- 172.21.15.70:0/300943323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa048154460 con 0x7fa0481535c0 2023-10-08T07:14:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.172+0000 7fa04ece5640 1 -- 172.21.15.70:0/300943323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0480b1e30 con 0x7fa0481535c0 2023-10-08T07:14:12.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.173+0000 7fa0457fa640 1 -- 172.21.15.70:0/300943323 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 133) v1 ==== 89507+0+0 (secure 0 0 0) 0x7fa034003970 con 0x7fa0481535c0 2023-10-08T07:14:12.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.174+0000 7fa0457fa640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa02c06b1c0 0x7fa02c06d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:12.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.174+0000 7fa0457fa640 1 -- 172.21.15.70:0/300943323 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fa034089d90 con 0x7fa0481535c0 2023-10-08T07:14:12.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.174+0000 7fa04ca81640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa02c06b1c0 0x7fa02c06d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.175+0000 7fa04ca81640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa02c06b1c0 0x7fa02c06d680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa03c001860 tx=0x7fa03c001670 comp rx=0 tx=0).ready entity=mgr.8551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:12.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.178+0000 7fa0457fa640 1 -- 172.21.15.70:0/300943323 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0480b1e30 con 0x7fa0481535c0 2023-10-08T07:14:12.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.418+0000 7fa04ece5640 1 -- 172.21.15.70:0/300943323 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa0480b1e30 con 0x7fa0481535c0 2023-10-08T07:14:12.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.651+0000 7fa0457fa640 1 -- 172.21.15.70:0/300943323 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v134) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa0480b1e30 con 0x7fa0481535c0 2023-10-08T07:14:12.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.654+0000 7fa04ece5640 1 -- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa02c06b1c0 msgr2=0x7fa02c06d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:12.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.654+0000 7fa04ece5640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa02c06b1c0 0x7fa02c06d680 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa03c001860 tx=0x7fa03c001670 comp rx=0 tx=0).stop 2023-10-08T07:14:12.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.654+0000 7fa04ece5640 1 -- 172.21.15.70:0/300943323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0481535c0 msgr2=0x7fa0480c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:12.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.654+0000 7fa04ece5640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0481535c0 0x7fa0480c80d0 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7fa03400e440 tx=0x7fa03400e900 comp rx=0 tx=0).stop 2023-10-08T07:14:12.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.654+0000 7fa04ece5640 1 -- 172.21.15.70:0/300943323 shutdown_connections 2023-10-08T07:14:12.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.655+0000 7fa04ece5640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa02c06b1c0 0x7fa02c06d680 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.655+0000 7fa04ece5640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa04814abb0 0x7fa04814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.655+0000 7fa04ece5640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0481535c0 0x7fa0480c80d0 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.655+0000 7fa04ece5640 1 --2- 172.21.15.70:0/300943323 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa038005180 0x7fa0481f60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:12.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.655+0000 7fa04ece5640 1 -- 172.21.15.70:0/300943323 >> 172.21.15.70:0/300943323 conn(0x7fa048069d50 msgr2=0x7fa04806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:12.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.655+0000 7fa04ece5640 1 -- 172.21.15.70:0/300943323 shutdown_connections 2023-10-08T07:14:12.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:12.655+0000 7fa04ece5640 1 -- 172.21.15.70:0/300943323 wait complete. 2023-10-08T07:14:12.680 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T07:14:12.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:14:12.788 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:14:12.879 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T07:14:12.879 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:14:12.880 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:14:12.932 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:14:12.932 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:14:13.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:13.174 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:13.173+0000 7fbf7a413040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:14:13.190 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:13.189+0000 7f4879a2c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:14:13.230 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:13.228+0000 7fbf7a413040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:14:13.243 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:13.241+0000 7f4879a2c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:14:13.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.269+0000 7ff10b142640 1 Processor -- start 2023-10-08T07:14:13.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.269+0000 7ff10b142640 1 -- start start 2023-10-08T07:14:13.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.270+0000 7ff10b142640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 0x7ff1040cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.270+0000 7ff10b142640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1040ce6e0 0x7ff1040cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.271+0000 7ff10b142640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10415d040 con 0x7ff1040d16f0 2023-10-08T07:14:13.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.271+0000 7ff10b142640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1040d0650 con 0x7ff1040ce6e0 2023-10-08T07:14:13.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.271+0000 7ff10b142640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040d0930 con 0x7ff10415cc60 2023-10-08T07:14:13.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.271+0000 7ff10993f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff1040d16f0 0x7ff1040cdfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40892/0 (socket says 172.21.15.70:40892) 2023-10-08T07:14:13.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.271+0000 7ff10993f640 1 -- 172.21.15.70:0/3069410895 learned_addr learned my addr 172.21.15.70:0/3069410895 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:13.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.271+0000 7ff10a140640 1 --2- 172.21.15.70:0/3069410895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 0x7ff1040cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:13.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.271+0000 7ff10a941640 1 --2- 172.21.15.70:0/3069410895 >> v2:172.21.15.181:3300/0 conn(0x7ff1040ce6e0 0x7ff1040cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:13.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.271+0000 7ff10913e640 1 -- 172.21.15.70:0/3069410895 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272661270 0 0) 0x7ff10415d040 con 0x7ff1040d16f0 2023-10-08T07:14:13.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.271+0000 7ff10913e640 1 -- 172.21.15.70:0/3069410895 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0f0003520 con 0x7ff1040d16f0 2023-10-08T07:14:13.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.272+0000 7ff10a140640 1 -- 172.21.15.70:0/3069410895 >> v2:172.21.15.181:3300/0 conn(0x7ff1040ce6e0 msgr2=0x7ff1040cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:13.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.272+0000 7ff10a140640 1 --2- 172.21.15.70:0/3069410895 >> v2:172.21.15.181:3300/0 conn(0x7ff1040ce6e0 0x7ff1040cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.272+0000 7ff10a140640 1 -- 172.21.15.70:0/3069410895 >> v1:172.21.15.70:6789/0 conn(0x7ff1040d16f0 legacy=0x7ff1040cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:13.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.272+0000 7ff10a140640 1 -- 172.21.15.70:0/3069410895 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1040d0c10 con 0x7ff10415cc60 2023-10-08T07:14:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.273+0000 7ff10a140640 1 --2- 172.21.15.70:0/3069410895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 0x7ff1040cf340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff100009790 tx=0x7ff10002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c18f55b64c76c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.273+0000 7ff10913e640 1 -- 172.21.15.70:0/3069410895 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10002d040 con 0x7ff10415cc60 2023-10-08T07:14:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.273+0000 7ff10913e640 1 -- 172.21.15.70:0/3069410895 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff100031810 con 0x7ff10415cc60 2023-10-08T07:14:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.273+0000 7ff10913e640 1 -- 172.21.15.70:0/3069410895 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100031ba0 con 0x7ff10415cc60 2023-10-08T07:14:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.274+0000 7ff10b142640 1 -- 172.21.15.70:0/3069410895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 msgr2=0x7ff1040cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.274+0000 7ff10b142640 1 --2- 172.21.15.70:0/3069410895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 0x7ff1040cf340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff100009790 tx=0x7ff10002b8b0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.274+0000 7ff10b142640 1 -- 172.21.15.70:0/3069410895 shutdown_connections 2023-10-08T07:14:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.274+0000 7ff10b142640 1 --2- 172.21.15.70:0/3069410895 >> v2:172.21.15.181:3300/0 conn(0x7ff1040ce6e0 0x7ff1040cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.274+0000 7ff10b142640 1 --2- 172.21.15.70:0/3069410895 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 0x7ff1040cf340 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.274+0000 7ff10b142640 1 -- 172.21.15.70:0/3069410895 >> 172.21.15.70:0/3069410895 conn(0x7ff10409bcf0 msgr2=0x7ff1040d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.274+0000 7ff10b142640 1 -- 172.21.15.70:0/3069410895 shutdown_connections 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.274+0000 7ff10b142640 1 -- 172.21.15.70:0/3069410895 wait complete. 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.276+0000 7ff10b142640 1 Processor -- start 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.276+0000 7ff10b142640 1 -- start start 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.277+0000 7ff10b142640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040ce6e0 0x7ff1040ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.277+0000 7ff10b142640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1040d16f0 0x7ff10415f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.277+0000 7ff10b142640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 0x7ff1040cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.277+0000 7ff10b142640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040ca990 con 0x7ff1040d16f0 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.277+0000 7ff10b142640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10415fc10 con 0x7ff1040ce6e0 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.277+0000 7ff10b142640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040b6220 con 0x7ff10415cc60 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.277+0000 7ff10993f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1040d16f0 0x7ff10415f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:13.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.277+0000 7ff10993f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1040d16f0 0x7ff10415f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33790/0 (socket says 172.21.15.70:33790) 2023-10-08T07:14:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.277+0000 7ff10993f640 1 -- 172.21.15.70:0/4195671430 learned_addr learned my addr 172.21.15.70:0/4195671430 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.278+0000 7ff10a941640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 0x7ff1040cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.278+0000 7ff10a140640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040ce6e0 0x7ff1040ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.278+0000 7ff10993f640 1 -- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 msgr2=0x7ff1040cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.278+0000 7ff10993f640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 0x7ff1040cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.278+0000 7ff10993f640 1 -- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040ce6e0 msgr2=0x7ff1040ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.278+0000 7ff10993f640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040ce6e0 0x7ff1040ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.278+0000 7ff10993f640 1 -- 172.21.15.70:0/4195671430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000093a0 con 0x7ff1040d16f0 2023-10-08T07:14:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.278+0000 7ff10a140640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040ce6e0 0x7ff1040ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:13.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.278+0000 7ff10993f640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1040d16f0 0x7ff10415f6d0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c410 tx=0x7ff0f400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:13.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.278+0000 7ff0faffd640 1 -- 172.21.15.70:0/4195671430 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4005ce0 con 0x7ff1040d16f0 2023-10-08T07:14:13.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.279+0000 7ff0faffd640 1 -- 172.21.15.70:0/4195671430 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f4005e40 con 0x7ff1040d16f0 2023-10-08T07:14:13.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.279+0000 7ff10b142640 1 -- 172.21.15.70:0/4195671430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1040cabc0 con 0x7ff1040d16f0 2023-10-08T07:14:13.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.279+0000 7ff0faffd640 1 -- 172.21.15.70:0/4195671430 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40072f0 con 0x7ff1040d16f0 2023-10-08T07:14:13.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.279+0000 7ff10b142640 1 -- 172.21.15.70:0/4195671430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1040cb180 con 0x7ff1040d16f0 2023-10-08T07:14:13.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.280+0000 7ff10b142640 1 -- 172.21.15.70:0/4195671430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1040b63e0 con 0x7ff1040d16f0 2023-10-08T07:14:13.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.282+0000 7ff0faffd640 1 -- 172.21.15.70:0/4195671430 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 134) v1 ==== 89490+0+0 (secure 0 0 0) 0x7ff0f4007450 con 0x7ff1040d16f0 2023-10-08T07:14:13.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.283+0000 7ff0faffd640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff0e806b170 0x7ff0e806d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.283+0000 7ff0faffd640 1 -- 172.21.15.70:0/4195671430 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7ff0f40896d0 con 0x7ff1040d16f0 2023-10-08T07:14:13.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.283+0000 7ff10a140640 1 -- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff0e806b170 msgr2=0x7ff0e806d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/860795336 2023-10-08T07:14:13.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.283+0000 7ff10a140640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff0e806b170 0x7ff0e806d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:13.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.287+0000 7ff0faffd640 1 -- 172.21.15.70:0/4195671430 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0f4057910 con 0x7ff1040d16f0 2023-10-08T07:14:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.477+0000 7ff10b142640 1 -- 172.21.15.70:0/4195671430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff10404e160 con 0x7ff1040d16f0 2023-10-08T07:14:13.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.481+0000 7ff0faffd640 1 -- 172.21.15.70:0/4195671430 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+21123 (secure 0 0 0) 0x7ff0f40574a0 con 0x7ff1040d16f0 2023-10-08T07:14:13.485 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:13.485 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":239,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:06.545728+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:13:47.007437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:13:47.838023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6833/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:6832/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1018014196":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1778827440":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1217032325":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/2798662747":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/3002223432":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/3074626816":"2023-10-09T07:14:06.545661+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:13.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.483+0000 7ff10b142640 1 -- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff0e806b170 msgr2=0x7ff0e806d630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.483+0000 7ff10b142640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff0e806b170 0x7ff0e806d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.483+0000 7ff10b142640 1 -- 172.21.15.70:0/4195671430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1040d16f0 msgr2=0x7ff10415f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.483+0000 7ff10b142640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1040d16f0 0x7ff10415f6d0 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c410 tx=0x7ff0f400c8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.483+0000 7ff10b142640 1 -- 172.21.15.70:0/4195671430 shutdown_connections 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.484+0000 7ff10b142640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7ff0e806b170 0x7ff0e806d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.484+0000 7ff10b142640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff10415cc60 0x7ff1040cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.484+0000 7ff10b142640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1040d16f0 0x7ff10415f6d0 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.484+0000 7ff10b142640 1 --2- 172.21.15.70:0/4195671430 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1040ce6e0 0x7ff1040ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.484+0000 7ff10b142640 1 -- 172.21.15.70:0/4195671430 >> 172.21.15.70:0/4195671430 conn(0x7ff10409bcf0 msgr2=0x7ff10415bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.484+0000 7ff10b142640 1 -- 172.21.15.70:0/4195671430 shutdown_connections 2023-10-08T07:14:13.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.484+0000 7ff10b142640 1 -- 172.21.15.70:0/4195671430 wait complete. 2023-10-08T07:14:13.503 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T07:13:32.637023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T07:13:33.187098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '234', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-08T07:13:47.007437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-08T07:13:47.838023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '238', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:14:13.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:14:13.696 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:13.694+0000 7f4879a2c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:14:13.706 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:13.704+0000 7fbf7a413040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:14:13.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.708+0000 7f87f2044640 1 Processor -- start 2023-10-08T07:14:13.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.708+0000 7f87f2044640 1 -- start start 2023-10-08T07:14:13.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.709+0000 7f87f2044640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87ec0b45e0 0x7f87ec0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.709+0000 7f87f2044640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f87ec0b83a0 0x7f87ec0b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87f2044640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87ec0b8830 0x7f87ec0b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87f2044640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec0b6b40 con 0x7f87ec0b45e0 2023-10-08T07:14:13.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87f2044640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec0b4f00 con 0x7f87ec0b83a0 2023-10-08T07:14:13.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87f2044640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0b5ec0 con 0x7f87ec0b8830 2023-10-08T07:14:13.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87eb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87ec0b45e0 0x7f87ec0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:13.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87eb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87ec0b45e0 0x7f87ec0b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39678/0 (socket says 172.21.15.70:39678) 2023-10-08T07:14:13.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87eb7fe640 1 -- 172.21.15.70:0/1377307480 learned_addr learned my addr 172.21.15.70:0/1377307480 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:13.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87eaffd640 1 --2- 172.21.15.70:0/1377307480 >> v2:172.21.15.181:3300/0 conn(0x7f87ec0b83a0 0x7f87ec0b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:13.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87eb7fe640 1 -- 172.21.15.70:0/1377307480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87ec0b8830 msgr2=0x7f87ec0b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:13.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87eb7fe640 1 --2- 172.21.15.70:0/1377307480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87ec0b8830 0x7f87ec0b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87eb7fe640 1 -- 172.21.15.70:0/1377307480 >> v2:172.21.15.181:3300/0 conn(0x7f87ec0b83a0 msgr2=0x7f87ec0b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:13.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87eb7fe640 1 --2- 172.21.15.70:0/1377307480 >> v2:172.21.15.181:3300/0 conn(0x7f87ec0b83a0 0x7f87ec0b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87eb7fe640 1 -- 172.21.15.70:0/1377307480 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec0b61a0 con 0x7f87ec0b45e0 2023-10-08T07:14:13.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.710+0000 7f87eaffd640 1 --2- 172.21.15.70:0/1377307480 >> v2:172.21.15.181:3300/0 conn(0x7f87ec0b83a0 0x7f87ec0b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:13.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.711+0000 7f87eb7fe640 1 --2- 172.21.15.70:0/1377307480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 0x7f87ec0b49c0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0d3c40 tx=0x7f87dc02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d114f5582d29a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:13.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.712+0000 7f87ea7fc640 1 -- 172.21.15.70:0/1377307480 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc031880 con 0x7f87ec0b45e0 2023-10-08T07:14:13.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.712+0000 7f87ea7fc640 1 -- 172.21.15.70:0/1377307480 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc0319e0 con 0x7f87ec0b45e0 2023-10-08T07:14:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.712+0000 7f87ea7fc640 1 -- 172.21.15.70:0/1377307480 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc031d70 con 0x7f87ec0b45e0 2023-10-08T07:14:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.712+0000 7f87f2044640 1 -- 172.21.15.70:0/1377307480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 msgr2=0x7f87ec0b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.712+0000 7f87f2044640 1 --2- 172.21.15.70:0/1377307480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 0x7f87ec0b49c0 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0d3c40 tx=0x7f87dc02b920 comp rx=0 tx=0).stop 2023-10-08T07:14:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.713+0000 7f87f2044640 1 -- 172.21.15.70:0/1377307480 shutdown_connections 2023-10-08T07:14:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.713+0000 7f87f2044640 1 --2- 172.21.15.70:0/1377307480 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87ec0b8830 0x7f87ec0b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.713+0000 7f87f2044640 1 --2- 172.21.15.70:0/1377307480 >> v2:172.21.15.181:3300/0 conn(0x7f87ec0b83a0 0x7f87ec0b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.713+0000 7f87f2044640 1 --2- 172.21.15.70:0/1377307480 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 0x7f87ec0b49c0 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.713+0000 7f87f2044640 1 -- 172.21.15.70:0/1377307480 >> 172.21.15.70:0/1377307480 conn(0x7f87ec0718a0 msgr2=0x7f87ec071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.713+0000 7f87f2044640 1 -- 172.21.15.70:0/1377307480 shutdown_connections 2023-10-08T07:14:13.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.713+0000 7f87f2044640 1 -- 172.21.15.70:0/1377307480 wait complete. 2023-10-08T07:14:13.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.714+0000 7f87f2044640 1 Processor -- start 2023-10-08T07:14:13.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.714+0000 7f87f2044640 1 -- start start 2023-10-08T07:14:13.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.715+0000 7f87f2044640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 0x7f87ec156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.715+0000 7f87f2044640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec0b83a0 0x7f87ec1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.715+0000 7f87f2044640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87ec0b8830 0x7f87ec14aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.715+0000 7f87f2044640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0cc7f0 con 0x7f87ec0b45e0 2023-10-08T07:14:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.715+0000 7f87f2044640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec159fd0 con 0x7f87ec0b83a0 2023-10-08T07:14:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.715+0000 7f87f2044640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec15a160 con 0x7f87ec0b8830 2023-10-08T07:14:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 0x7f87ec156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 0x7f87ec156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39694/0 (socket says 172.21.15.70:39694) 2023-10-08T07:14:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eb7fe640 1 -- 172.21.15.70:0/3073959562 learned_addr learned my addr 172.21.15.70:0/3073959562 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:13.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eb7fe640 1 -- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87ec0b8830 msgr2=0x7f87ec14aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:13.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eb7fe640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87ec0b8830 0x7f87ec14aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eb7fe640 1 -- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec0b83a0 msgr2=0x7f87ec1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:13.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eaffd640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec0b83a0 0x7f87ec1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:13.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eb7fe640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec0b83a0 0x7f87ec1586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eb7fe640 1 -- 172.21.15.70:0/3073959562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0094d0 con 0x7f87ec0b45e0 2023-10-08T07:14:13.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eaffd640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec0b83a0 0x7f87ec1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:13.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.716+0000 7f87eb7fe640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 0x7f87ec156490 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000c00 tx=0x7f87dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.717+0000 7f87d3fff640 1 -- 172.21.15.70:0/3073959562 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc031850 con 0x7f87ec0b45e0 2023-10-08T07:14:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.717+0000 7f87d3fff640 1 -- 172.21.15.70:0/3073959562 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc0443f0 con 0x7f87ec0b45e0 2023-10-08T07:14:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.717+0000 7f87f2044640 1 -- 172.21.15.70:0/3073959562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec156b20 con 0x7f87ec0b45e0 2023-10-08T07:14:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.717+0000 7f87d3fff640 1 -- 172.21.15.70:0/3073959562 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc0447a0 con 0x7f87ec0b45e0 2023-10-08T07:14:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.717+0000 7f87f2044640 1 -- 172.21.15.70:0/3073959562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1570e0 con 0x7f87ec0b45e0 2023-10-08T07:14:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.718+0000 7f87f2044640 1 -- 172.21.15.70:0/3073959562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec04e160 con 0x7f87ec0b45e0 2023-10-08T07:14:13.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.719+0000 7f87d3fff640 1 -- 172.21.15.70:0/3073959562 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 134) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f87dc044900 con 0x7f87ec0b45e0 2023-10-08T07:14:13.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.720+0000 7f87d3fff640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87d806b140 0x7f87d806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:13.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.720+0000 7f87d3fff640 1 -- 172.21.15.70:0/3073959562 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f87dc0acc10 con 0x7f87ec0b45e0 2023-10-08T07:14:13.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.720+0000 7f87eaffd640 1 -- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87d806b140 msgr2=0x7f87d806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/860795336 2023-10-08T07:14:13.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.720+0000 7f87eaffd640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87d806b140 0x7f87d806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:13.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.724+0000 7f87d3fff640 1 -- 172.21.15.70:0/3073959562 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87dc0772d0 con 0x7f87ec0b45e0 2023-10-08T07:14:13.751 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:13.749+0000 7f4879a2c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:14:13.764 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:13.762+0000 7fbf7a413040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:14:13.801 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:13.800+0000 7f4879a2c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:14:13.817 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:13.815+0000 7fbf7a413040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:14:13.861 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:13.860+0000 7f4879a2c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:14:13.879 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:13.878+0000 7fbf7a413040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:14:13.910 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:13.908+0000 7f4879a2c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:14:13.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.917+0000 7f87f2044640 1 -- 172.21.15.70:0/3073959562 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87ec0d14c0 con 0x7f87ec0b45e0 2023-10-08T07:14:13.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.918+0000 7f87d3fff640 1 -- 172.21.15.70:0/3073959562 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v239) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87dc07a8f0 con 0x7f87ec0b45e0 2023-10-08T07:14:13.919 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:14:13.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.920+0000 7f87f2044640 1 -- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87d806b140 msgr2=0x7f87d806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:13.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.920+0000 7f87f2044640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87d806b140 0x7f87d806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.920+0000 7f87f2044640 1 -- 172.21.15.70:0/3073959562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 msgr2=0x7f87ec156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.920+0000 7f87f2044640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 0x7f87ec156490 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000c00 tx=0x7f87dc0046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.920+0000 7f87f2044640 1 -- 172.21.15.70:0/3073959562 shutdown_connections 2023-10-08T07:14:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.920+0000 7f87f2044640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f87d806b140 0x7f87d806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.920+0000 7f87f2044640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f87ec0b8830 0x7f87ec14aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.920+0000 7f87f2044640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f87ec0b83a0 0x7f87ec1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.920+0000 7f87f2044640 1 --2- 172.21.15.70:0/3073959562 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87ec0b45e0 0x7f87ec156490 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.920+0000 7f87f2044640 1 -- 172.21.15.70:0/3073959562 >> 172.21.15.70:0/3073959562 conn(0x7f87ec0718a0 msgr2=0x7f87ec0d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.921+0000 7f87f2044640 1 -- 172.21.15.70:0/3073959562 shutdown_connections 2023-10-08T07:14:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:13.921+0000 7f87f2044640 1 -- 172.21.15.70:0/3073959562 wait complete. 2023-10-08T07:14:13.931 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:13.929+0000 7fbf7a413040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:14:13.940 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:14:14.021 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:14.019+0000 7f4879a2c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:14:14.048 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:14.047+0000 7fbf7a413040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:14:14.082 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:14.080+0000 7f4879a2c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:14:14.114 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:14.112+0000 7fbf7a413040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:14:14.130 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:14.129+0000 7f4879a2c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:14:14.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.136+0000 7f2146a75640 1 Processor -- start 2023-10-08T07:14:14.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.136+0000 7f2146a75640 1 -- start start 2023-10-08T07:14:14.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.137+0000 7f2146a75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21400cf2b0 0x7f21400d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.138+0000 7f2146a75640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21400cf690 0x7f21400ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.138+0000 7f2146a75640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21400cd520 con 0x7f21400d1100 2023-10-08T07:14:14.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.138+0000 7f2146a75640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21400d1910 con 0x7f21400cf690 2023-10-08T07:14:14.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.138+0000 7f2146a75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21400ce080 con 0x7f21400cf2b0 2023-10-08T07:14:14.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.138+0000 7f2145272640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21400cf690 0x7f21400ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:14.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.138+0000 7f2145a73640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21400cf2b0 0x7f21400d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:14.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.138+0000 7f2145272640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f21400cf690 0x7f21400ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:33964/0 (socket says 172.21.15.70:33964) 2023-10-08T07:14:14.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.138+0000 7f2145272640 1 -- 172.21.15.70:0/1910603669 learned_addr learned my addr 172.21.15.70:0/1910603669 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:14.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.139+0000 7f2145272640 1 -- 172.21.15.70:0/1910603669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21400cf2b0 msgr2=0x7f21400d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:14.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.139+0000 7f2145272640 1 --2- 172.21.15.70:0/1910603669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21400cf2b0 0x7f21400d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.139+0000 7f2145272640 1 -- 172.21.15.70:0/1910603669 >> v1:172.21.15.181:6789/0 conn(0x7f21400d1100 legacy=0x7f214015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:14.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.139+0000 7f2145272640 1 -- 172.21.15.70:0/1910603669 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400cd750 con 0x7f21400cf690 2023-10-08T07:14:14.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.140+0000 7f2145272640 1 --2- 172.21.15.70:0/1910603669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21400cf690 0x7f21400ce510 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2130009870 tx=0x7f213002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31cc3a9267029c3a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:14.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2144a71640 1 -- 172.21.15.70:0/1910603669 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130031640 con 0x7f21400cf690 2023-10-08T07:14:14.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2144a71640 1 -- 172.21.15.70:0/1910603669 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21300317a0 con 0x7f21400cf690 2023-10-08T07:14:14.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2144a71640 1 -- 172.21.15.70:0/1910603669 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130031b30 con 0x7f21400cf690 2023-10-08T07:14:14.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2146a75640 1 -- 172.21.15.70:0/1910603669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21400cf690 msgr2=0x7f21400ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:14.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2146a75640 1 --2- 172.21.15.70:0/1910603669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21400cf690 0x7f21400ce510 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2130009870 tx=0x7f213002b8f0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2146a75640 1 -- 172.21.15.70:0/1910603669 shutdown_connections 2023-10-08T07:14:14.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2146a75640 1 --2- 172.21.15.70:0/1910603669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21400cf690 0x7f21400ce510 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2146a75640 1 --2- 172.21.15.70:0/1910603669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21400cf2b0 0x7f21400d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2146a75640 1 -- 172.21.15.70:0/1910603669 >> 172.21.15.70:0/1910603669 conn(0x7f214009bc40 msgr2=0x7f21400d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:14.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2146a75640 1 -- 172.21.15.70:0/1910603669 shutdown_connections 2023-10-08T07:14:14.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.141+0000 7f2146a75640 1 -- 172.21.15.70:0/1910603669 wait complete. 2023-10-08T07:14:14.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.143+0000 7f2146a75640 1 Processor -- start 2023-10-08T07:14:14.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.143+0000 7f2146a75640 1 -- start start 2023-10-08T07:14:14.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.144+0000 7f2146a75640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21400cf2b0 0x7f214014c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.144+0000 7f2146a75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21400cf690 0x7f214014a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.144+0000 7f2146a75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21400d1100 0x7f214014d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.144+0000 7f2146a75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21400c9190 con 0x7f21400cf690 2023-10-08T07:14:14.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.144+0000 7f2146a75640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214014b2b0 con 0x7f21400cf2b0 2023-10-08T07:14:14.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.144+0000 7f2146a75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214014b590 con 0x7f21400d1100 2023-10-08T07:14:14.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.144+0000 7f2145272640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21400cf690 0x7f214014a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:14.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2145272640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21400cf690 0x7f214014a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39704/0 (socket says 172.21.15.70:39704) 2023-10-08T07:14:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2145272640 1 -- 172.21.15.70:0/694245198 learned_addr learned my addr 172.21.15.70:0/694245198 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2145a73640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21400cf2b0 0x7f214014c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2146274640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21400d1100 0x7f214014d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2145272640 1 -- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21400d1100 msgr2=0x7f214014d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2145272640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21400d1100 0x7f214014d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2145272640 1 -- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21400cf2b0 msgr2=0x7f214014c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2145272640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21400cf2b0 0x7f214014c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2145272640 1 -- 172.21.15.70:0/694245198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21300094d0 con 0x7f21400cf690 2023-10-08T07:14:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2145a73640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21400cf2b0 0x7f214014c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:14:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.145+0000 7f2145272640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21400cf690 0x7f214014a9b0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f2130000c00 tx=0x7f21300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:14.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.146+0000 7f21367fc640 1 -- 172.21.15.70:0/694245198 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213003a930 con 0x7f21400cf690 2023-10-08T07:14:14.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.146+0000 7f21367fc640 1 -- 172.21.15.70:0/694245198 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213003aa90 con 0x7f21400cf690 2023-10-08T07:14:14.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.146+0000 7f2146a75640 1 -- 172.21.15.70:0/694245198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f214014b870 con 0x7f21400cf690 2023-10-08T07:14:14.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.146+0000 7f21367fc640 1 -- 172.21.15.70:0/694245198 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213003ae40 con 0x7f21400cf690 2023-10-08T07:14:14.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.146+0000 7f2146a75640 1 -- 172.21.15.70:0/694245198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214014be30 con 0x7f21400cf690 2023-10-08T07:14:14.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.147+0000 7f2146a75640 1 -- 172.21.15.70:0/694245198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2140074150 con 0x7f21400cf690 2023-10-08T07:14:14.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.148+0000 7f21367fc640 1 -- 172.21.15.70:0/694245198 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 134) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f213002f050 con 0x7f21400cf690 2023-10-08T07:14:14.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.149+0000 7f21367fc640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f212406b140 0x7f212406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.149+0000 7f21367fc640 1 -- 172.21.15.70:0/694245198 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f21300abf10 con 0x7f21400cf690 2023-10-08T07:14:14.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.149+0000 7f2145a73640 1 -- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f212406b140 msgr2=0x7f212406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/860795336 2023-10-08T07:14:14.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.149+0000 7f2145a73640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f212406b140 0x7f212406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:14.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.153+0000 7f21367fc640 1 -- 172.21.15.70:0/694245198 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2130076500 con 0x7f21400cf690 2023-10-08T07:14:14.168 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:14.166+0000 7fbf7a413040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:14:14.189 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:14.188+0000 7f4879a2c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:14:14.227 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:14.225+0000 7fbf7a413040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:14:14.240 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:14.238+0000 7f4879a2c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:14:14.279 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:14.277+0000 7fbf7a413040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:14:14.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.345+0000 7f2146a75640 1 -- 172.21.15.70:0/694245198 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21400d0070 con 0x7f21400cf690 2023-10-08T07:14:14.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.345+0000 7f21367fc640 1 -- 172.21.15.70:0/694245198 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v239) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2130079b20 con 0x7f21400cf690 2023-10-08T07:14:14.347 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:14:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 -- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f212406b140 msgr2=0x7f212406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f212406b140 0x7f212406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 -- 172.21.15.70:0/694245198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21400cf690 msgr2=0x7f214014a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21400cf690 0x7f214014a9b0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f2130000c00 tx=0x7f21300046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 -- 172.21.15.70:0/694245198 shutdown_connections 2023-10-08T07:14:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f212406b140 0x7f212406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21400d1100 0x7f214014d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21400cf690 0x7f214014a9b0 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 --2- 172.21.15.70:0/694245198 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f21400cf2b0 0x7f214014c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 -- 172.21.15.70:0/694245198 >> 172.21.15.70:0/694245198 conn(0x7f214009bc40 msgr2=0x7f21400b7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:14.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 -- 172.21.15.70:0/694245198 shutdown_connections 2023-10-08T07:14:14.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.348+0000 7f2146a75640 1 -- 172.21.15.70:0/694245198 wait complete. 2023-10-08T07:14:14.366 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:14:14.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.556+0000 7fcc8d8d7640 1 Processor -- start 2023-10-08T07:14:14.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.556+0000 7fcc8d8d7640 1 -- start start 2023-10-08T07:14:14.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.557+0000 7fcc8d8d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc880afbe0 0x7fcc880b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.557+0000 7fcc8d8d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc8806dfc0 0x7fcc88070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc8d8d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc880b2620 con 0x7fcc88071070 2023-10-08T07:14:14.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc8d8d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc880b2060 con 0x7fcc880afbe0 2023-10-08T07:14:14.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc8d8d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880b2340 con 0x7fcc8806dfc0 2023-10-08T07:14:14.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc86ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc880afbe0 0x7fcc880b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:14.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc877fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcc88071070 0x7fcc880cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47644/0 (socket says 172.21.15.70:47644) 2023-10-08T07:14:14.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc877fe640 1 -- 172.21.15.70:0/574531379 learned_addr learned my addr 172.21.15.70:0/574531379 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:14.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc86ffd640 1 -- 172.21.15.70:0/574531379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc8806dfc0 msgr2=0x7fcc88070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:14.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc867fc640 1 --2- 172.21.15.70:0/574531379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc8806dfc0 0x7fcc88070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:14.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc86ffd640 1 --2- 172.21.15.70:0/574531379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc8806dfc0 0x7fcc88070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc85ffb640 1 -- 172.21.15.70:0/574531379 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487997738 0 0) 0x7fcc880b2620 con 0x7fcc88071070 2023-10-08T07:14:14.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc86ffd640 1 -- 172.21.15.70:0/574531379 >> v1:172.21.15.70:6789/0 conn(0x7fcc88071070 legacy=0x7fcc880cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:14.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.558+0000 7fcc86ffd640 1 -- 172.21.15.70:0/574531379 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc88070350 con 0x7fcc880afbe0 2023-10-08T07:14:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.559+0000 7fcc86ffd640 1 --2- 172.21.15.70:0/574531379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc880afbe0 0x7fcc880b15b0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fcc780097a0 tx=0x7fcc7802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e8f158b2d86302 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.560+0000 7fcc85ffb640 1 -- 172.21.15.70:0/574531379 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7802d040 con 0x7fcc880afbe0 2023-10-08T07:14:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.560+0000 7fcc85ffb640 1 -- 172.21.15.70:0/574531379 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc78031810 con 0x7fcc880afbe0 2023-10-08T07:14:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.560+0000 7fcc85ffb640 1 -- 172.21.15.70:0/574531379 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc78031ba0 con 0x7fcc880afbe0 2023-10-08T07:14:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.560+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/574531379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc880afbe0 msgr2=0x7fcc880b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.560+0000 7fcc8d8d7640 1 --2- 172.21.15.70:0/574531379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc880afbe0 0x7fcc880b15b0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fcc780097a0 tx=0x7fcc7802b8b0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.561+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/574531379 shutdown_connections 2023-10-08T07:14:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.561+0000 7fcc8d8d7640 1 --2- 172.21.15.70:0/574531379 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc8806dfc0 0x7fcc88070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.561+0000 7fcc8d8d7640 1 --2- 172.21.15.70:0/574531379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc880afbe0 0x7fcc880b15b0 unknown :-1 s=CLOSED pgs=2487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.561+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/574531379 >> 172.21.15.70:0/574531379 conn(0x7fcc8806a5d0 msgr2=0x7fcc880b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.561+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/574531379 shutdown_connections 2023-10-08T07:14:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.561+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/574531379 wait complete. 2023-10-08T07:14:14.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.562+0000 7fcc8d8d7640 1 Processor -- start 2023-10-08T07:14:14.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.563+0000 7fcc8d8d7640 1 -- start start 2023-10-08T07:14:14.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.563+0000 7fcc8d8d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc8806dfc0 0x7fcc880ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.563+0000 7fcc8d8d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc88071070 0x7fcc880ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.563+0000 7fcc8d8d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc880afbe0 0x7fcc88152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.563+0000 7fcc8d8d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc880ceb30 con 0x7fcc88071070 2023-10-08T07:14:14.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.563+0000 7fcc8d8d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8815b890 con 0x7fcc880afbe0 2023-10-08T07:14:14.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.563+0000 7fcc8d8d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8815bb70 con 0x7fcc8806dfc0 2023-10-08T07:14:14.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.564+0000 7fcc867fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc88071070 0x7fcc880ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:14.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.564+0000 7fcc867fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc88071070 0x7fcc880ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39720/0 (socket says 172.21.15.70:39720) 2023-10-08T07:14:14.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.564+0000 7fcc867fc640 1 -- 172.21.15.70:0/3116459401 learned_addr learned my addr 172.21.15.70:0/3116459401 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:14.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.564+0000 7fcc877fe640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc880afbe0 0x7fcc88152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:14.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.564+0000 7fcc86ffd640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc8806dfc0 0x7fcc880ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:14.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.564+0000 7fcc867fc640 1 -- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc8806dfc0 msgr2=0x7fcc880ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:14.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.564+0000 7fcc867fc640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc8806dfc0 0x7fcc880ce5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.564+0000 7fcc867fc640 1 -- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc880afbe0 msgr2=0x7fcc88152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:14.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.564+0000 7fcc867fc640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc880afbe0 0x7fcc88152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.564+0000 7fcc867fc640 1 -- 172.21.15.70:0/3116459401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc78009400 con 0x7fcc88071070 2023-10-08T07:14:14.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.565+0000 7fcc867fc640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc88071070 0x7fcc880ca480 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c7f0 tx=0x7fcc7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:14.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.565+0000 7fcc6ffff640 1 -- 172.21.15.70:0/3116459401 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c004230 con 0x7fcc88071070 2023-10-08T07:14:14.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.565+0000 7fcc6ffff640 1 -- 172.21.15.70:0/3116459401 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7c004390 con 0x7fcc88071070 2023-10-08T07:14:14.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.565+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/3116459401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8815be50 con 0x7fcc88071070 2023-10-08T07:14:14.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.565+0000 7fcc6ffff640 1 -- 172.21.15.70:0/3116459401 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c007860 con 0x7fcc88071070 2023-10-08T07:14:14.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.565+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/3116459401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc881573f0 con 0x7fcc88071070 2023-10-08T07:14:14.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.567+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/3116459401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8804e160 con 0x7fcc88071070 2023-10-08T07:14:14.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.567+0000 7fcc6ffff640 1 -- 172.21.15.70:0/3116459401 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 134) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fcc7c007bd0 con 0x7fcc88071070 2023-10-08T07:14:14.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.568+0000 7fcc6ffff640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fcc7406b140 0x7fcc7406d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:14.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.568+0000 7fcc6ffff640 1 -- 172.21.15.70:0/3116459401 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fcc7c08a1c0 con 0x7fcc88071070 2023-10-08T07:14:14.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.568+0000 7fcc86ffd640 1 -- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fcc7406b140 msgr2=0x7fcc7406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/860795336 2023-10-08T07:14:14.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.568+0000 7fcc86ffd640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fcc7406b140 0x7fcc7406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:14.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.572+0000 7fcc6ffff640 1 -- 172.21.15.70:0/3116459401 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc7c0548e0 con 0x7fcc88071070 2023-10-08T07:14:14.629 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:14.626+0000 7f4879a2c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:14:14.714 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:14.712+0000 7fbf7a413040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:14:14.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.769+0000 7fcc86ffd640 1 -- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fcc7406b140 msgr2=0x7fcc7406d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/860795336 2023-10-08T07:14:14.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.769+0000 7fcc86ffd640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fcc7406b140 0x7fcc7406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:14:14.786 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:14.785+0000 7f4879a2c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:14:14.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.791+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/3116459401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcc880b2ed0 con 0x7fcc88071070 2023-10-08T07:14:14.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.792+0000 7fcc6ffff640 1 -- 172.21.15.70:0/3116459401 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v239) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc7c057f00 con 0x7fcc88071070 2023-10-08T07:14:14.794 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:14:14.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fcc7406b140 msgr2=0x7fcc7406d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:14.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fcc7406b140 0x7fcc7406d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/3116459401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc88071070 msgr2=0x7fcc880ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:14.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc88071070 0x7fcc880ca480 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c7f0 tx=0x7fcc7c00ccb0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/3116459401 shutdown_connections 2023-10-08T07:14:14.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fcc7406b140 0x7fcc7406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcc880afbe0 0x7fcc88152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc88071070 0x7fcc880ca480 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 --2- 172.21.15.70:0/3116459401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc8806dfc0 0x7fcc880ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:14.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/3116459401 >> 172.21.15.70:0/3116459401 conn(0x7fcc8806a5d0 msgr2=0x7fcc880c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:14.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/3116459401 shutdown_connections 2023-10-08T07:14:14.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:14.795+0000 7fcc8d8d7640 1 -- 172.21.15.70:0/3116459401 wait complete. 2023-10-08T07:14:14.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-08T07:14:14.881 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:14.878+0000 7fbf7a413040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:14:14.893 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:14.891+0000 7f4879a2c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:14:14.943 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:14.941+0000 7f4879a2c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:14:14.993 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:14.991+0000 7fbf7a413040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:14:15.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.019+0000 7f0489a02640 1 Processor -- start 2023-10-08T07:14:15.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.020+0000 7f0489a02640 1 -- start start 2023-10-08T07:14:15.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.021+0000 7f0489a02640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f04840c9cf0 0x7f048406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.022+0000 7f0489a02640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048415b310 con 0x7f048406c3a0 2023-10-08T07:14:15.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.022+0000 7f0489a02640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048406d290 con 0x7f048415aa00 2023-10-08T07:14:15.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.022+0000 7f0489a02640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048406e0a0 con 0x7f04840c9cf0 2023-10-08T07:14:15.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.022+0000 7f0482ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f048406c3a0 0x7f048406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47656/0 (socket says 172.21.15.70:47656) 2023-10-08T07:14:15.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.022+0000 7f0482ffd640 1 -- 172.21.15.70:0/576780999 learned_addr learned my addr 172.21.15.70:0/576780999 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:15.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.022+0000 7f047bfff640 1 --2- 172.21.15.70:0/576780999 >> v2:172.21.15.181:3300/0 conn(0x7f04840c9cf0 0x7f048406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.022+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524530852 0 0) 0x7f0474001110 con 0x7f048406c3a0 2023-10-08T07:14:15.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.022+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0470003520 con 0x7f048406c3a0 2023-10-08T07:14:15.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 940031292 0 0) 0x7f0470003520 con 0x7f048406c3a0 2023-10-08T07:14:15.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 >> v2:172.21.15.181:3300/0 conn(0x7f04840c9cf0 msgr2=0x7f048406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 --2- 172.21.15.70:0/576780999 >> v2:172.21.15.181:3300/0 conn(0x7f04840c9cf0 0x7f048406db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 >> v1:172.21.15.181:6789/0 conn(0x7f048415aa00 legacy=0x7f04840b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04840c9860 con 0x7f048406c3a0 2023-10-08T07:14:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0474001800 con 0x7f048406c3a0 2023-10-08T07:14:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 >> v1:172.21.15.70:6789/0 conn(0x7f048406c3a0 legacy=0x7f048406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04700039c0 0x7f0470003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04700047c0 0x7f0470004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0470005180 0x7f047000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048406e0a0 con 0x7f04700039c0 2023-10-08T07:14:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047000c080 con 0x7f04700047c0 2023-10-08T07:14:15.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047000c240 con 0x7f0470005180 2023-10-08T07:14:15.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f0482ffd640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04700039c0 0x7f0470003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f047bfff640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04700047c0 0x7f0470004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.023+0000 7f04837fe640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0470005180 0x7f047000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.024+0000 7f0482ffd640 1 -- 172.21.15.70:0/576780999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0470005180 msgr2=0x7f047000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:15.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.024+0000 7f0482ffd640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0470005180 0x7f047000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.024+0000 7f0482ffd640 1 -- 172.21.15.70:0/576780999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04700047c0 msgr2=0x7f0470004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:15.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.024+0000 7f0482ffd640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04700047c0 0x7f0470004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.024+0000 7f0482ffd640 1 -- 172.21.15.70:0/576780999 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0470003520 con 0x7f04700039c0 2023-10-08T07:14:15.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.024+0000 7f04837fe640 1 -- 172.21.15.70:0/576780999 reap_dead start 2023-10-08T07:14:15.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.024+0000 7f047bfff640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04700047c0 0x7f0470004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:14:15.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.025+0000 7f0482ffd640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04700039c0 0x7f0470003de0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f0474007fb0 tx=0x7f04700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868cc34eeca43765 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:15.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.025+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0474005ad0 con 0x7f04700039c0 2023-10-08T07:14:15.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.025+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0474005c50 con 0x7f04700039c0 2023-10-08T07:14:15.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.025+0000 7f04827fc640 1 -- 172.21.15.70:0/576780999 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047402c7c0 con 0x7f04700039c0 2023-10-08T07:14:15.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.026+0000 7f0489a02640 1 -- 172.21.15.70:0/576780999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04700039c0 msgr2=0x7f0470003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.026+0000 7f0489a02640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04700039c0 0x7f0470003de0 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f0474007fb0 tx=0x7f04700057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.026+0000 7f0489a02640 1 -- 172.21.15.70:0/576780999 shutdown_connections 2023-10-08T07:14:15.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.026+0000 7f0489a02640 1 --2- 172.21.15.70:0/576780999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04700039c0 0x7f0470003de0 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.026+0000 7f0489a02640 1 -- 172.21.15.70:0/576780999 >> 172.21.15.70:0/576780999 conn(0x7f048406a5d0 msgr2=0x7f04840b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:15.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.026+0000 7f0489a02640 1 -- 172.21.15.70:0/576780999 shutdown_connections 2023-10-08T07:14:15.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.026+0000 7f0489a02640 1 -- 172.21.15.70:0/576780999 wait complete. 2023-10-08T07:14:15.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.028+0000 7f0489a02640 1 Processor -- start 2023-10-08T07:14:15.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.028+0000 7f0489a02640 1 -- start start 2023-10-08T07:14:15.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f0489a02640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04700039c0 0x7f0484150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f0489a02640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0484155190 0x7f048415b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f0489a02640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f048415b9f0 0x7f0484153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f0489a02640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04840cca50 con 0x7f048415b9f0 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f0489a02640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048415bf10 con 0x7f0484155190 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f0489a02640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048415c1f0 con 0x7f04700039c0 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f04837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f048415b9f0 0x7f0484153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f04837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f048415b9f0 0x7f0484153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39732/0 (socket says 172.21.15.70:39732) 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f04837fe640 1 -- 172.21.15.70:0/4072260145 learned_addr learned my addr 172.21.15.70:0/4072260145 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f0482ffd640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04700039c0 0x7f0484150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.029+0000 7f047bfff640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0484155190 0x7f048415b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.030+0000 7f04837fe640 1 -- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04700039c0 msgr2=0x7f0484150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.030+0000 7f04837fe640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04700039c0 0x7f0484150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.030+0000 7f04837fe640 1 -- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0484155190 msgr2=0x7f048415b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.030+0000 7f04837fe640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0484155190 0x7f048415b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.030+0000 7f04837fe640 1 -- 172.21.15.70:0/4072260145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048415c4d0 con 0x7f048415b9f0 2023-10-08T07:14:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.030+0000 7f04837fe640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f048415b9f0 0x7f0484153920 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f047c007fd0 tx=0x7f047c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.030+0000 7f0480ff9640 1 -- 172.21.15.70:0/4072260145 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c00ee90 con 0x7f048415b9f0 2023-10-08T07:14:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.030+0000 7f0480ff9640 1 -- 172.21.15.70:0/4072260145 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c003970 con 0x7f048415b9f0 2023-10-08T07:14:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.030+0000 7f0480ff9640 1 -- 172.21.15.70:0/4072260145 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c003d20 con 0x7f048415b9f0 2023-10-08T07:14:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.031+0000 7f0489a02640 1 -- 172.21.15.70:0/4072260145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04840cc6e0 con 0x7f048415b9f0 2023-10-08T07:14:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.031+0000 7f0489a02640 1 -- 172.21.15.70:0/4072260145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048415c9b0 con 0x7f048415b9f0 2023-10-08T07:14:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.032+0000 7f0489a02640 1 -- 172.21.15.70:0/4072260145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04840ccc00 con 0x7f048415b9f0 2023-10-08T07:14:15.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.034+0000 7f0480ff9640 1 -- 172.21.15.70:0/4072260145 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 134) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f047c0050d0 con 0x7f048415b9f0 2023-10-08T07:14:15.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.034+0000 7f0480ff9640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f046406b170 0x7f046406d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.035+0000 7f0480ff9640 1 -- 172.21.15.70:0/4072260145 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f047c089d90 con 0x7f048415b9f0 2023-10-08T07:14:15.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.035+0000 7f0482ffd640 1 -- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f046406b170 msgr2=0x7f046406d630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/860795336 2023-10-08T07:14:15.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.035+0000 7f0482ffd640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f046406b170 0x7f046406d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:15.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.039+0000 7f0480ff9640 1 -- 172.21.15.70:0/4072260145 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04840ccc00 con 0x7f048415b9f0 2023-10-08T07:14:15.049 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:15.047+0000 7fbf7a413040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:14:15.051 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:15.050+0000 7f4879a2c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:14:15.157 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:15.156+0000 7f4879a2c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:14:15.162 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:15.160+0000 7fbf7a413040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:14:15.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.228+0000 7f0489a02640 1 -- 172.21.15.70:0/4072260145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f048404e160 con 0x7f048415b9f0 2023-10-08T07:14:15.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.229+0000 7f0480ff9640 1 -- 172.21.15.70:0/4072260145 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v239) v1 ==== 107+0+11 (secure 0 0 0) 0x7f047c0544c0 con 0x7f048415b9f0 2023-10-08T07:14:15.231 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:14:15.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 -- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f046406b170 msgr2=0x7f046406d630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:15.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f046406b170 0x7f046406d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 -- 172.21.15.70:0/4072260145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f048415b9f0 msgr2=0x7f0484153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f048415b9f0 0x7f0484153920 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f047c007fd0 tx=0x7f047c00eb50 comp rx=0 tx=0).stop 2023-10-08T07:14:15.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 -- 172.21.15.70:0/4072260145 shutdown_connections 2023-10-08T07:14:15.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f046406b170 0x7f046406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f048415b9f0 0x7f0484153920 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0484155190 0x7f048415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 --2- 172.21.15.70:0/4072260145 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04700039c0 0x7f0484150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 -- 172.21.15.70:0/4072260145 >> 172.21.15.70:0/4072260145 conn(0x7f048406a5d0 msgr2=0x7f04840c7890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:15.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 -- 172.21.15.70:0/4072260145 shutdown_connections 2023-10-08T07:14:15.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.232+0000 7f0489a02640 1 -- 172.21.15.70:0/4072260145 wait complete. 2023-10-08T07:14:15.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-08T07:14:15.276 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:15.275+0000 7fbf7a413040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:14:15.312 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:15.311+0000 7f4879a2c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:14:15.362 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:15.360+0000 7f4879a2c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:14:15.441 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:15.439+0000 7fbf7a413040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:14:15.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.456+0000 7f11a1716640 1 Processor -- start 2023-10-08T07:14:15.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.457+0000 7f11a1716640 1 -- start start 2023-10-08T07:14:15.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.458+0000 7f11a1716640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f119c0cd470 0x7f119c0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.458+0000 7f11a1716640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119c0cde10 con 0x7f119c15ce00 2023-10-08T07:14:15.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.458+0000 7f11a1716640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f119c161920 con 0x7f119c160980 2023-10-08T07:14:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.458+0000 7f11a1716640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119c161640 con 0x7f119c0cd470 2023-10-08T07:14:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.458+0000 7f119affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f119c15ce00 0x7f119c0cb370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47662/0 (socket says 172.21.15.70:47662) 2023-10-08T07:14:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.458+0000 7f119affd640 1 -- 172.21.15.70:0/2532100792 learned_addr learned my addr 172.21.15.70:0/2532100792 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f119a7fc640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f119c0cd470 0x7f119c0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470284214 0 0) 0x7f119c0cde10 con 0x7f119c15ce00 2023-10-08T07:14:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1184003520 con 0x7f119c15ce00 2023-10-08T07:14:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1460982169 0 0) 0x7f1184003520 con 0x7f119c15ce00 2023-10-08T07:14:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f119c0cd470 msgr2=0x7f119c0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f119c0cd470 0x7f119c0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 >> v1:172.21.15.181:6789/0 conn(0x7f119c160980 legacy=0x7f119c160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c06b690 con 0x7f119c15ce00 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f118c0017f0 con 0x7f119c15ce00 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 >> v1:172.21.15.70:6789/0 conn(0x7f119c15ce00 legacy=0x7f119c0cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11840039a0 0x7f1184003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11840047c0 0x7f1184004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f119c0cd470 0x7f11840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c161640 con 0x7f11840039a0 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118400bab0 con 0x7f11840047c0 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.459+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118400bc70 con 0x7f119c0cd470 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.460+0000 7f119affd640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11840039a0 0x7f1184003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.460+0000 7f119a7fc640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11840047c0 0x7f1184004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.460+0000 7f119b7fe640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f119c0cd470 0x7f11840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.460+0000 7f119affd640 1 -- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f119c0cd470 msgr2=0x7f11840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.460+0000 7f119affd640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f119c0cd470 0x7f11840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.460+0000 7f119affd640 1 -- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11840047c0 msgr2=0x7f1184004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.460+0000 7f119affd640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11840047c0 0x7f1184004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.460+0000 7f119affd640 1 -- 172.21.15.70:0/2532100792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1184003520 con 0x7f11840039a0 2023-10-08T07:14:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.460+0000 7f119b7fe640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f119c0cd470 0x7f11840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.461+0000 7f119affd640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11840039a0 0x7f1184003dc0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c190 tx=0x7f11840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e946440265de50 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.461+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c007270 con 0x7f11840039a0 2023-10-08T07:14:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.461+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c0073f0 con 0x7f11840039a0 2023-10-08T07:14:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.461+0000 7f1199ffb640 1 -- 172.21.15.70:0/2532100792 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c02c8a0 con 0x7f11840039a0 2023-10-08T07:14:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.462+0000 7f11a1716640 1 -- 172.21.15.70:0/2532100792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11840039a0 msgr2=0x7f1184003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.462+0000 7f11a1716640 1 --2- 172.21.15.70:0/2532100792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11840039a0 0x7f1184003dc0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c190 tx=0x7f11840057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.462+0000 7f119b7fe640 1 -- 172.21.15.70:0/2532100792 reap_dead start 2023-10-08T07:14:15.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.462+0000 7f11a1716640 1 -- 172.21.15.70:0/2532100792 shutdown_connections 2023-10-08T07:14:15.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.462+0000 7f11a1716640 1 -- 172.21.15.70:0/2532100792 >> 172.21.15.70:0/2532100792 conn(0x7f119c06e650 msgr2=0x7f119c06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:15.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.462+0000 7f11a1716640 1 -- 172.21.15.70:0/2532100792 shutdown_connections 2023-10-08T07:14:15.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.462+0000 7f11a1716640 1 -- 172.21.15.70:0/2532100792 wait complete. 2023-10-08T07:14:15.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.465+0000 7f11a1716640 1 Processor -- start 2023-10-08T07:14:15.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.465+0000 7f11a1716640 1 -- start start 2023-10-08T07:14:15.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.466+0000 7f11a1716640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11840039a0 0x7f119c157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f11a1716640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15ef60 0x7f119c15f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f11a1716640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119c0b46c0 0x7f119c0b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f11a1716640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c15f920 con 0x7f119c0b46c0 2023-10-08T07:14:15.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f11a1716640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c0b5c90 con 0x7f119c15ef60 2023-10-08T07:14:15.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f11a1716640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119c15d600 con 0x7f11840039a0 2023-10-08T07:14:15.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f119b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119c0b46c0 0x7f119c0b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f119b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119c0b46c0 0x7f119c0b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39758/0 (socket says 172.21.15.70:39758) 2023-10-08T07:14:15.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f119b7fe640 1 -- 172.21.15.70:0/3130602248 learned_addr learned my addr 172.21.15.70:0/3130602248 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f119affd640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11840039a0 0x7f119c157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f119b7fe640 1 -- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11840039a0 msgr2=0x7f119c157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f119b7fe640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11840039a0 0x7f119c157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f119b7fe640 1 -- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15ef60 msgr2=0x7f119c15f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f119b7fe640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15ef60 0x7f119c15f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.467+0000 7f119b7fe640 1 -- 172.21.15.70:0/3130602248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c1575c0 con 0x7f119c0b46c0 2023-10-08T07:14:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.468+0000 7f119b7fe640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119c0b46c0 0x7f119c0b4b40 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f1184004740 tx=0x7f119400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.468+0000 7f11837fe640 1 -- 172.21.15.70:0/3130602248 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194006d50 con 0x7f119c0b46c0 2023-10-08T07:14:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.468+0000 7f11837fe640 1 -- 172.21.15.70:0/3130602248 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1194003970 con 0x7f119c0b46c0 2023-10-08T07:14:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.468+0000 7f11837fe640 1 -- 172.21.15.70:0/3130602248 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194003d60 con 0x7f119c0b46c0 2023-10-08T07:14:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.468+0000 7f11a1716640 1 -- 172.21.15.70:0/3130602248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119c0b53f0 con 0x7f119c0b46c0 2023-10-08T07:14:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.468+0000 7f11a1716640 1 -- 172.21.15.70:0/3130602248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c0b59b0 con 0x7f119c0b46c0 2023-10-08T07:14:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.469+0000 7f11a1716640 1 -- 172.21.15.70:0/3130602248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c06cff0 con 0x7f119c0b46c0 2023-10-08T07:14:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.470+0000 7f11837fe640 1 -- 172.21.15.70:0/3130602248 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 134) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f1194005070 con 0x7f119c0b46c0 2023-10-08T07:14:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.471+0000 7f11837fe640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f118806b140 0x7f118806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.471+0000 7f11837fe640 1 -- 172.21.15.70:0/3130602248 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f119408a020 con 0x7f119c0b46c0 2023-10-08T07:14:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.471+0000 7f119affd640 1 -- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f118806b140 msgr2=0x7f118806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/860795336 2023-10-08T07:14:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.471+0000 7f119affd640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f118806b140 0x7f118806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:15.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.475+0000 7f11837fe640 1 -- 172.21.15.70:0/3130602248 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1194054690 con 0x7f119c0b46c0 2023-10-08T07:14:15.483 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:15.481+0000 7f4879a2c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:14:15.497 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:15.495+0000 7fbf7a413040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:14:15.621 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:15.619+0000 7fbf7a413040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:14:15.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.663+0000 7f11a1716640 1 -- 172.21.15.70:0/3130602248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f119c04e160 con 0x7f119c0b46c0 2023-10-08T07:14:15.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.664+0000 7f11837fe640 1 -- 172.21.15.70:0/3130602248 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v239) v1 ==== 103+0+11 (secure 0 0 0) 0x7f119401ce80 con 0x7f119c0b46c0 2023-10-08T07:14:15.666 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:14:15.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.666+0000 7f11a1716640 1 -- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f118806b140 msgr2=0x7f118806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.666+0000 7f11a1716640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f118806b140 0x7f118806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.666+0000 7f11a1716640 1 -- 172.21.15.70:0/3130602248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119c0b46c0 msgr2=0x7f119c0b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.666+0000 7f11a1716640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119c0b46c0 0x7f119c0b4b40 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f1184004740 tx=0x7f119400dc00 comp rx=0 tx=0).stop 2023-10-08T07:14:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.667+0000 7f11a1716640 1 -- 172.21.15.70:0/3130602248 shutdown_connections 2023-10-08T07:14:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.667+0000 7f11a1716640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f118806b140 0x7f118806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.667+0000 7f11a1716640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f119c0b46c0 0x7f119c0b4b40 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.667+0000 7f11a1716640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f119c15ef60 0x7f119c15f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.667+0000 7f11a1716640 1 --2- 172.21.15.70:0/3130602248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11840039a0 0x7f119c157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.667+0000 7f11a1716640 1 -- 172.21.15.70:0/3130602248 >> 172.21.15.70:0/3130602248 conn(0x7f119c06e650 msgr2=0x7f119c0c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.667+0000 7f11a1716640 1 -- 172.21.15.70:0/3130602248 shutdown_connections 2023-10-08T07:14:15.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.667+0000 7f11a1716640 1 -- 172.21.15.70:0/3130602248 wait complete. 2023-10-08T07:14:15.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T07:14:15.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.935+0000 7fa21ba52640 1 Processor -- start 2023-10-08T07:14:15.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.935+0000 7fa21ba52640 1 -- start start 2023-10-08T07:14:15.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.936+0000 7fa21ba52640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa2140c9cf0 0x7fa21406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.936+0000 7fa21ba52640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21415b310 con 0x7fa21415aa00 2023-10-08T07:14:15.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.936+0000 7fa21ba52640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21406d290 con 0x7fa21406c3a0 2023-10-08T07:14:15.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.936+0000 7fa21ba52640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21406e0a0 con 0x7fa2140c9cf0 2023-10-08T07:14:15.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.936+0000 7fa219fef640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa21415aa00 0x7fa2140b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47670/0 (socket says 172.21.15.70:47670) 2023-10-08T07:14:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.936+0000 7fa219fef640 1 -- 172.21.15.70:0/3166242381 learned_addr learned my addr 172.21.15.70:0/3166242381 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:15.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.936+0000 7fa218fed640 1 --2- 172.21.15.70:0/3166242381 >> v2:172.21.15.70:3300/0 conn(0x7fa2140c9cf0 0x7fa21406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.937+0000 7fa20bfff640 1 -- 172.21.15.70:0/3166242381 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368705927 0 0) 0x7fa20c001490 con 0x7fa21415aa00 2023-10-08T07:14:15.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.937+0000 7fa20bfff640 1 -- 172.21.15.70:0/3166242381 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa200003520 con 0x7fa21415aa00 2023-10-08T07:14:15.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.937+0000 7fa218fed640 1 -- 172.21.15.70:0/3166242381 >> v1:172.21.15.181:6789/0 conn(0x7fa21406c3a0 legacy=0x7fa21406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.937+0000 7fa218fed640 1 -- 172.21.15.70:0/3166242381 >> v1:172.21.15.70:6789/0 conn(0x7fa21415aa00 legacy=0x7fa2140b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.937+0000 7fa218fed640 1 -- 172.21.15.70:0/3166242381 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2140c9860 con 0x7fa2140c9cf0 2023-10-08T07:14:15.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.938+0000 7fa218fed640 1 --2- 172.21.15.70:0/3166242381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 0x7fa21406db60 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fa2040097a0 tx=0x7fa20402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a125fa4f8413bbf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.938+0000 7fa20bfff640 1 -- 172.21.15.70:0/3166242381 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204030610 con 0x7fa2140c9cf0 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.938+0000 7fa20bfff640 1 -- 172.21.15.70:0/3166242381 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa204030770 con 0x7fa2140c9cf0 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.938+0000 7fa20bfff640 1 -- 172.21.15.70:0/3166242381 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204030b00 con 0x7fa2140c9cf0 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.939+0000 7fa21ba52640 1 -- 172.21.15.70:0/3166242381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 msgr2=0x7fa21406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.939+0000 7fa21ba52640 1 --2- 172.21.15.70:0/3166242381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 0x7fa21406db60 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7fa2040097a0 tx=0x7fa20402b960 comp rx=0 tx=0).stop 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.939+0000 7fa21ba52640 1 -- 172.21.15.70:0/3166242381 shutdown_connections 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.939+0000 7fa21ba52640 1 --2- 172.21.15.70:0/3166242381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 0x7fa21406db60 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.939+0000 7fa21ba52640 1 -- 172.21.15.70:0/3166242381 >> 172.21.15.70:0/3166242381 conn(0x7fa21406a5d0 msgr2=0x7fa2140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.939+0000 7fa21ba52640 1 -- 172.21.15.70:0/3166242381 shutdown_connections 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.939+0000 7fa21ba52640 1 -- 172.21.15.70:0/3166242381 wait complete. 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.941+0000 7fa21ba52640 1 Processor -- start 2023-10-08T07:14:15.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.941+0000 7fa21ba52640 1 -- start start 2023-10-08T07:14:15.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.941+0000 7fa21ba52640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21406c3a0 0x7fa2140c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.941+0000 7fa21ba52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 0x7fa2140c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa21ba52640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa21415aa00 0x7fa2140abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa21ba52640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2140c4f20 con 0x7fa2140c9cf0 2023-10-08T07:14:15.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa21ba52640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2141f3790 con 0x7fa21406c3a0 2023-10-08T07:14:15.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa21ba52640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2141f2c10 con 0x7fa21415aa00 2023-10-08T07:14:15.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa218fed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 0x7fa2140c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa218fed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 0x7fa2140c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39788/0 (socket says 172.21.15.70:39788) 2023-10-08T07:14:15.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa218fed640 1 -- 172.21.15.70:0/2477845877 learned_addr learned my addr 172.21.15.70:0/2477845877 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:15.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa2197ee640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21406c3a0 0x7fa2140c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa219fef640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa21415aa00 0x7fa2140abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:15.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa218fed640 1 -- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa21415aa00 msgr2=0x7fa2140abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa218fed640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa21415aa00 0x7fa2140abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa218fed640 1 -- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21406c3a0 msgr2=0x7fa2140c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:15.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa218fed640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21406c3a0 0x7fa2140c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:15.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.942+0000 7fa218fed640 1 -- 172.21.15.70:0/2477845877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204009400 con 0x7fa2140c9cf0 2023-10-08T07:14:15.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.943+0000 7fa218fed640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 0x7fa2140c62c0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fa20402bf70 tx=0x7fa204004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:15.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.943+0000 7fa209ffb640 1 -- 172.21.15.70:0/2477845877 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20403e040 con 0x7fa2140c9cf0 2023-10-08T07:14:15.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.943+0000 7fa209ffb640 1 -- 172.21.15.70:0/2477845877 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa204039c90 con 0x7fa2140c9cf0 2023-10-08T07:14:15.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.943+0000 7fa209ffb640 1 -- 172.21.15.70:0/2477845877 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204030e10 con 0x7fa2140c9cf0 2023-10-08T07:14:15.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.943+0000 7fa21ba52640 1 -- 172.21.15.70:0/2477845877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2141f2ef0 con 0x7fa2140c9cf0 2023-10-08T07:14:15.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.943+0000 7fa21ba52640 1 -- 172.21.15.70:0/2477845877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2141f34b0 con 0x7fa2140c9cf0 2023-10-08T07:14:15.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.944+0000 7fa21ba52640 1 -- 172.21.15.70:0/2477845877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2141f5eb0 con 0x7fa2140c9cf0 2023-10-08T07:14:15.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.945+0000 7fa209ffb640 1 -- 172.21.15.70:0/2477845877 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 134) v1 ==== 89490+0+0 (secure 0 0 0) 0x7fa20404d3b0 con 0x7fa2140c9cf0 2023-10-08T07:14:15.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.946+0000 7fa209ffb640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa1f806b140 0x7fa1f806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:15.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.947+0000 7fa209ffb640 1 -- 172.21.15.70:0/2477845877 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7fa2040ac640 con 0x7fa2140c9cf0 2023-10-08T07:14:15.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.947+0000 7fa2197ee640 1 -- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa1f806b140 msgr2=0x7fa1f806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/860795336 2023-10-08T07:14:15.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.947+0000 7fa2197ee640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa1f806b140 0x7fa1f806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:15.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:15.951+0000 7fa209ffb640 1 -- 172.21.15.70:0/2477845877 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa2141f5eb0 con 0x7fa2140c9cf0 2023-10-08T07:14:16.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.148+0000 7fa2197ee640 1 -- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa1f806b140 msgr2=0x7fa1f806d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/860795336 2023-10-08T07:14:16.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.148+0000 7fa2197ee640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa1f806b140 0x7fa1f806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:14:16.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.154+0000 7fa21ba52640 1 -- 172.21.15.70:0/2477845877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa2141f5eb0 con 0x7fa2140c9cf0 2023-10-08T07:14:16.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.155+0000 7fa209ffb640 1 -- 172.21.15.70:0/2477845877 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v239) v1 ==== 80+0+478 (secure 0 0 0) 0x7fa2141f5eb0 con 0x7fa2140c9cf0 2023-10-08T07:14:16.157 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/3993370366 2023-10-09T07:14:06.545661+0000 2023-10-08T07:14:16.157 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/3993370366 2023-10-09T07:14:06.545661+0000 2023-10-08T07:14:16.157 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1018014196 2023-10-09T07:14:06.545661+0000 2023-10-08T07:14:16.157 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1778827440 2023-10-09T07:14:06.545661+0000 2023-10-08T07:14:16.157 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1217032325 2023-10-09T07:14:06.545661+0000 2023-10-08T07:14:16.158 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2798662747 2023-10-09T07:14:06.545661+0000 2023-10-08T07:14:16.158 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3002223432 2023-10-09T07:14:06.545661+0000 2023-10-08T07:14:16.158 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3074626816 2023-10-09T07:14:06.545661+0000 2023-10-08T07:14:16.158 INFO:teuthology.orchestra.run.smithi070.stderr:listed 8 entries 2023-10-08T07:14:16.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 -- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa1f806b140 msgr2=0x7fa1f806d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:16.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa1f806b140 0x7fa1f806d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 -- 172.21.15.70:0/2477845877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 msgr2=0x7fa2140c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:16.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 0x7fa2140c62c0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7fa20402bf70 tx=0x7fa204004730 comp rx=0 tx=0).stop 2023-10-08T07:14:16.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 -- 172.21.15.70:0/2477845877 shutdown_connections 2023-10-08T07:14:16.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7fa1f806b140 0x7fa1f806d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa21415aa00 0x7fa2140abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2140c9cf0 0x7fa2140c62c0 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 --2- 172.21.15.70:0/2477845877 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa21406c3a0 0x7fa2140c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 -- 172.21.15.70:0/2477845877 >> 172.21.15.70:0/2477845877 conn(0x7fa21406a5d0 msgr2=0x7fa2140b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:16.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 -- 172.21.15.70:0/2477845877 shutdown_connections 2023-10-08T07:14:16.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.158+0000 7fa21ba52640 1 -- 172.21.15.70:0/2477845877 wait complete. 2023-10-08T07:14:16.177 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T07:14:16.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:16.178 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T07:14:16.274 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T07:14:16.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:16.274 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T07:14:16.318 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:14:16.318 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:14:16.330 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.329+0000 7f4879a2c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:14:16.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:14:16.374 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:14:16.374 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:14:16.427 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:14:16.427 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:14:16.428 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:14:16.471 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:16.468+0000 7fbf7a413040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:14:16.484 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:14:16.485 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:14:16.633 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.632+0000 7f4879a2c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:14:16.693 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.691+0000 7f4879a2c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:14:16.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.691+0000 7f3221c2e640 1 Processor -- start 2023-10-08T07:14:16.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.692+0000 7f3221c2e640 1 -- start start 2023-10-08T07:14:16.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.693+0000 7f3221c2e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 0x7f321c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:16.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.693+0000 7f3221c2e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f321c0adc70 0x7f321c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:16.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.693+0000 7f3221c2e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c15f7d0 con 0x7f321c15af10 2023-10-08T07:14:16.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.693+0000 7f3221c2e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321c15f210 con 0x7f321c0adc70 2023-10-08T07:14:16.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.693+0000 7f3221c2e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c15f4f0 con 0x7f321c15ab30 2023-10-08T07:14:16.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.694+0000 7f3220c2c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 0x7f321c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:16.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.694+0000 7f321bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f321c0adc70 0x7f321c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:16.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.694+0000 7f3220c2c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 0x7f321c15e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:39808/0 (socket says 172.21.15.70:39808) 2023-10-08T07:14:16.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.694+0000 7f321bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f321c0adc70 0x7f321c0b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:34038/0 (socket says 172.21.15.70:34038) 2023-10-08T07:14:16.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.694+0000 7f3220c2c640 1 -- 172.21.15.70:0/2342182433 learned_addr learned my addr 172.21.15.70:0/2342182433 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:16.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.694+0000 7f321bfff640 1 -- 172.21.15.70:0/2342182433 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 msgr2=0x7f321c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:16.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.694+0000 7f321bfff640 1 --2- 172.21.15.70:0/2342182433 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 0x7f321c15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.694+0000 7f321bfff640 1 -- 172.21.15.70:0/2342182433 >> v1:172.21.15.181:6789/0 conn(0x7f321c15af10 legacy=0x7f321c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:16.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.694+0000 7f321bfff640 1 -- 172.21.15.70:0/2342182433 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c06c550 con 0x7f321c0adc70 2023-10-08T07:14:16.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.696+0000 7f321bfff640 1 --2- 172.21.15.70:0/2342182433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 0x7f321c0b00c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f32080097a0 tx=0x7f320802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94d85e993ef64918 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:16.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.696+0000 7f321b7fe640 1 -- 172.21.15.70:0/2342182433 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320802d040 con 0x7f321c0adc70 2023-10-08T07:14:16.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.696+0000 7f321b7fe640 1 -- 172.21.15.70:0/2342182433 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208031810 con 0x7f321c0adc70 2023-10-08T07:14:16.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.696+0000 7f321b7fe640 1 -- 172.21.15.70:0/2342182433 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208031ba0 con 0x7f321c0adc70 2023-10-08T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.697+0000 7f3221c2e640 1 -- 172.21.15.70:0/2342182433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 msgr2=0x7f321c0b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.697+0000 7f3221c2e640 1 --2- 172.21.15.70:0/2342182433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 0x7f321c0b00c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f32080097a0 tx=0x7f320802b8b0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.697+0000 7f3221c2e640 1 -- 172.21.15.70:0/2342182433 shutdown_connections 2023-10-08T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.697+0000 7f3221c2e640 1 --2- 172.21.15.70:0/2342182433 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 0x7f321c0b00c0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.697+0000 7f3221c2e640 1 --2- 172.21.15.70:0/2342182433 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 0x7f321c15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.697+0000 7f3221c2e640 1 -- 172.21.15.70:0/2342182433 >> 172.21.15.70:0/2342182433 conn(0x7f321c0cb0c0 msgr2=0x7f321c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:16.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.697+0000 7f3221c2e640 1 -- 172.21.15.70:0/2342182433 shutdown_connections 2023-10-08T07:14:16.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.697+0000 7f3221c2e640 1 -- 172.21.15.70:0/2342182433 wait complete. 2023-10-08T07:14:16.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.699+0000 7f3221c2e640 1 Processor -- start 2023-10-08T07:14:16.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.699+0000 7f3221c2e640 1 -- start start 2023-10-08T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.700+0000 7f3221c2e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 0x7f321c1590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.700+0000 7f3221c2e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 0x7f321c0c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.700+0000 7f3221c2e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15af10 0x7f321c0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.700+0000 7f3221c2e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0c5710 con 0x7f321c15af10 2023-10-08T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.700+0000 7f3221c2e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0c5f70 con 0x7f321c0adc70 2023-10-08T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.700+0000 7f3221c2e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0c8390 con 0x7f321c15ab30 2023-10-08T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f322142d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15af10 0x7f321c0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:16.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f3220c2c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 0x7f321c1590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f3220c2c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 0x7f321c1590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:34050/0 (socket says 172.21.15.70:34050) 2023-10-08T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f3220c2c640 1 -- 172.21.15.70:0/3516020375 learned_addr learned my addr 172.21.15.70:0/3516020375 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f321bfff640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 0x7f321c0c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f3220c2c640 1 -- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 msgr2=0x7f321c0c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f3220c2c640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 0x7f321c0c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f3220c2c640 1 -- 172.21.15.70:0/3516020375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15af10 msgr2=0x7f321c0c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f3220c2c640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15af10 0x7f321c0c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f3220c2c640 1 -- 172.21.15.70:0/3516020375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3208009400 con 0x7f321c0adc70 2023-10-08T07:14:16.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.701+0000 7f322142d640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15af10 0x7f321c0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:14:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.702+0000 7f3220c2c640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 0x7f321c1590d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3210009fd0 tx=0x7f321000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.702+0000 7f32197fa640 1 -- 172.21.15.70:0/3516020375 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321000c940 con 0x7f321c0adc70 2023-10-08T07:14:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.702+0000 7f32197fa640 1 -- 172.21.15.70:0/3516020375 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321000caa0 con 0x7f321c0adc70 2023-10-08T07:14:16.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.702+0000 7f3221c2e640 1 -- 172.21.15.70:0/3516020375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c0c5920 con 0x7f321c0adc70 2023-10-08T07:14:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.702+0000 7f32197fa640 1 -- 172.21.15.70:0/3516020375 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321000ce30 con 0x7f321c0adc70 2023-10-08T07:14:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.702+0000 7f3221c2e640 1 -- 172.21.15.70:0/3516020375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c159610 con 0x7f321c0adc70 2023-10-08T07:14:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.703+0000 7f3221c2e640 1 -- 172.21.15.70:0/3516020375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321c04e160 con 0x7f321c0adc70 2023-10-08T07:14:16.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.705+0000 7f32197fa640 1 -- 172.21.15.70:0/3516020375 <== mon.1 v2:172.21.15.181:3300/0 4 ==== mgrmap(e 134) v1 ==== 89490+0+0 (secure 0 0 0) 0x7f3210005990 con 0x7f321c0adc70 2023-10-08T07:14:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.705+0000 7f32197fa640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f320006b140 0x7f320006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.706+0000 7f32197fa640 1 -- 172.21.15.70:0/3516020375 <== mon.1 v2:172.21.15.181:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 7353+0+0 (secure 0 0 0) 0x7f321008a600 con 0x7f321c0adc70 2023-10-08T07:14:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.706+0000 7f321bfff640 1 -- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f320006b140 msgr2=0x7f320006d600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/860795336 2023-10-08T07:14:16.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.706+0000 7f321bfff640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f320006b140 0x7f320006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:14:16.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.719+0000 7f32197fa640 1 -- 172.21.15.70:0/3516020375 <== mon.1 v2:172.21.15.181:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3210054d60 con 0x7f321c0adc70 2023-10-08T07:14:16.783 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:16.781+0000 7fbf7a413040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:14:16.811 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.810+0000 7f4879a2c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:14:16.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.834+0000 7f32197fa640 1 -- 172.21.15.70:0/3516020375 <== mon.1 v2:172.21.15.181:3300/0 7 ==== mgrmap(e 135) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f3210055130 con 0x7f321c0adc70 2023-10-08T07:14:16.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.835+0000 7f32197fa640 1 -- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f320006b140 msgr2=0x7f320006d600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:14:16.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.835+0000 7f32197fa640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f320006b140 0x7f320006d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.847 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:16.846+0000 7fbf7a413040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:14:16.911 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.909+0000 7f47fb4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.911 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.909+0000 7f47fb4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.911 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.909+0000 7f47fb4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.912 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.909+0000 7f47fb4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.912 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.909+0000 7f47fb4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.912 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.910+0000 7f47fe4c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.912 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.910+0000 7f47fe4c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.913 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.910+0000 7f47fe4c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.913 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.910+0000 7f47fe4c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.913 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.910+0000 7f47fe4c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.924 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.923+0000 7f47fb4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.925 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.923+0000 7f47fb4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.925 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.923+0000 7f47fb4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.925 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.923+0000 7f47fb4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.925 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.923+0000 7f47fb4ba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.926 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.925+0000 7f47fe4c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.926 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.925+0000 7f47fe4c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.926 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.925+0000 7f47fe4c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.926 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.925+0000 7f47fe4c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.926 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:14:16.925+0000 7f47fe4c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:14:16.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.964+0000 7f3221c2e640 1 -- 172.21.15.70:0/3516020375 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f321c0b3be0 con 0x7f321c0adc70 2023-10-08T07:14:16.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.965+0000 7f32197fa640 1 -- 172.21.15.70:0/3516020375 <== mon.1 v2:172.21.15.181:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f3210073ca0 con 0x7f321c0adc70 2023-10-08T07:14:16.970 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:16.970 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":114,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:34.170119+0000","modified":"2023-10-08T07:13:37.222153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11097":{"gid":11097,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/180995576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":180995576},{"type":"v1","addr":"172.21.15.70:6834","nonce":180995576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:48.829441+0000","modified":"2023-10-08T07:13:51.877323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8416":{"gid":8416,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-08T07:14:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 114 2023-10-08T07:14:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.968+0000 7f3221c2e640 1 -- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 msgr2=0x7f321c1590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.968+0000 7f3221c2e640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 0x7f321c1590d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3210009fd0 tx=0x7f321000ed40 comp rx=0 tx=0).stop 2023-10-08T07:14:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.968+0000 7f3221c2e640 1 -- 172.21.15.70:0/3516020375 shutdown_connections 2023-10-08T07:14:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.968+0000 7f3221c2e640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:6832/860795336,v1:172.21.15.181:6833/860795336] conn(0x7f320006b140 0x7f320006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.968+0000 7f3221c2e640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15af10 0x7f321c0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.968+0000 7f3221c2e640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f321c15ab30 0x7f321c0c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.971 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:14:16.968+0000 7fbf7a413040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:14:16.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.968+0000 7f3221c2e640 1 --2- 172.21.15.70:0/3516020375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321c0adc70 0x7f321c1590d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:16.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.968+0000 7f3221c2e640 1 -- 172.21.15.70:0/3516020375 >> 172.21.15.70:0/3516020375 conn(0x7f321c0cb0c0 msgr2=0x7f321c0acda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:16.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.968+0000 7f3221c2e640 1 -- 172.21.15.70:0/3516020375 shutdown_connections 2023-10-08T07:14:16.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:16.968+0000 7f3221c2e640 1 -- 172.21.15.70:0/3516020375 wait complete. 2023-10-08T07:14:16.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:17.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.194+0000 7f43bd3d7640 1 Processor -- start 2023-10-08T07:14:17.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.194+0000 7f43bd3d7640 1 -- start start 2023-10-08T07:14:17.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.196+0000 7f43bd3d7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f43b8159ac0 0x7f43b806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.196+0000 7f43bd3d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f43b80ad790 0x7f43b80702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.196+0000 7f43bd3d7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b806dee0 con 0x7f43b815e330 2023-10-08T07:14:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.196+0000 7f43bd3d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b815d850 con 0x7f43b80ad790 2023-10-08T07:14:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.196+0000 7f43bd3d7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b815db30 con 0x7f43b8159ac0 2023-10-08T07:14:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.196+0000 7f43b67fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f43b80ad790 0x7f43b80702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.196+0000 7f43b67fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f43b80ad790 0x7f43b80702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39804/0 (socket says 172.21.15.70:39804) 2023-10-08T07:14:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.196+0000 7f43b67fc640 1 -- 172.21.15.70:0/1975970397 learned_addr learned my addr 172.21.15.70:0/1975970397 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.197+0000 7f43b6ffd640 1 --2- 172.21.15.70:0/1975970397 >> v2:172.21.15.181:3300/0 conn(0x7f43b8159ac0 0x7f43b806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:17.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.197+0000 7f43b67fc640 1 -- 172.21.15.70:0/1975970397 >> v2:172.21.15.181:3300/0 conn(0x7f43b8159ac0 msgr2=0x7f43b806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:17.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.197+0000 7f43b67fc640 1 --2- 172.21.15.70:0/1975970397 >> v2:172.21.15.181:3300/0 conn(0x7f43b8159ac0 0x7f43b806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.197+0000 7f43b67fc640 1 -- 172.21.15.70:0/1975970397 >> v1:172.21.15.181:6789/0 conn(0x7f43b815e330 legacy=0x7f43b806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:17.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.197+0000 7f43b67fc640 1 -- 172.21.15.70:0/1975970397 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b815de10 con 0x7f43b80ad790 2023-10-08T07:14:17.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.198+0000 7f43b67fc640 1 --2- 172.21.15.70:0/1975970397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b80ad790 0x7f43b80702d0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f43b806e790 tx=0x7f43ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa22bb2f73d625f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:17.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.198+0000 7f43b5ffb640 1 -- 172.21.15.70:0/1975970397 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac031870 con 0x7f43b80ad790 2023-10-08T07:14:17.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.198+0000 7f43b5ffb640 1 -- 172.21.15.70:0/1975970397 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ac0319d0 con 0x7f43b80ad790 2023-10-08T07:14:17.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.198+0000 7f43b5ffb640 1 -- 172.21.15.70:0/1975970397 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac031d60 con 0x7f43b80ad790 2023-10-08T07:14:17.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.199+0000 7f43bd3d7640 1 -- 172.21.15.70:0/1975970397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b80ad790 msgr2=0x7f43b80702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.199+0000 7f43bd3d7640 1 --2- 172.21.15.70:0/1975970397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b80ad790 0x7f43b80702d0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f43b806e790 tx=0x7f43ac02b910 comp rx=0 tx=0).stop 2023-10-08T07:14:17.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.199+0000 7f43bd3d7640 1 -- 172.21.15.70:0/1975970397 shutdown_connections 2023-10-08T07:14:17.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.199+0000 7f43bd3d7640 1 --2- 172.21.15.70:0/1975970397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b80ad790 0x7f43b80702d0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.199+0000 7f43bd3d7640 1 --2- 172.21.15.70:0/1975970397 >> v2:172.21.15.181:3300/0 conn(0x7f43b8159ac0 0x7f43b806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.199+0000 7f43bd3d7640 1 -- 172.21.15.70:0/1975970397 >> 172.21.15.70:0/1975970397 conn(0x7f43b806a380 msgr2=0x7f43b80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:17.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.199+0000 7f43bd3d7640 1 -- 172.21.15.70:0/1975970397 shutdown_connections 2023-10-08T07:14:17.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.199+0000 7f43bd3d7640 1 -- 172.21.15.70:0/1975970397 wait complete. 2023-10-08T07:14:17.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.201+0000 7f43bd3d7640 1 Processor -- start 2023-10-08T07:14:17.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.201+0000 7f43bd3d7640 1 -- start start 2023-10-08T07:14:17.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43bd3d7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b80ad790 0x7f43b8152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43bd3d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b8159ac0 0x7f43b815c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43bd3d7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b815e330 0x7f43b815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43bd3d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80b1f00 con 0x7f43b8159ac0 2023-10-08T07:14:17.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43bd3d7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b815cf50 con 0x7f43b80ad790 2023-10-08T07:14:17.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43bd3d7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80b2150 con 0x7f43b815e330 2023-10-08T07:14:17.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43b67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b8159ac0 0x7f43b815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:17.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43b67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b8159ac0 0x7f43b815c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39806/0 (socket says 172.21.15.70:39806) 2023-10-08T07:14:17.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43b67fc640 1 -- 172.21.15.70:0/4164831484 learned_addr learned my addr 172.21.15.70:0/4164831484 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:17.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43b67fc640 1 -- 172.21.15.70:0/4164831484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b815e330 msgr2=0x7f43b815c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43b67fc640 1 --2- 172.21.15.70:0/4164831484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b815e330 0x7f43b815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43b67fc640 1 -- 172.21.15.70:0/4164831484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b80ad790 msgr2=0x7f43b8152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43b6ffd640 1 --2- 172.21.15.70:0/4164831484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b80ad790 0x7f43b8152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:17.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43b67fc640 1 --2- 172.21.15.70:0/4164831484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b80ad790 0x7f43b8152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.202+0000 7f43b67fc640 1 -- 172.21.15.70:0/4164831484 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0094d0 con 0x7f43b8159ac0 2023-10-08T07:14:17.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.203+0000 7f43b67fc640 1 --2- 172.21.15.70:0/4164831484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b8159ac0 0x7f43b815c390 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f43ac000c00 tx=0x7f43ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:17.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.203+0000 7f439f7fe640 1 -- 172.21.15.70:0/4164831484 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac03abe0 con 0x7f43b8159ac0 2023-10-08T07:14:17.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.203+0000 7f439f7fe640 1 -- 172.21.15.70:0/4164831484 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ac03ad40 con 0x7f43b8159ac0 2023-10-08T07:14:17.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.203+0000 7f439f7fe640 1 -- 172.21.15.70:0/4164831484 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac044610 con 0x7f43b8159ac0 2023-10-08T07:14:17.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.203+0000 7f43bd3d7640 1 -- 172.21.15.70:0/4164831484 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b80b2430 con 0x7f43b8159ac0 2023-10-08T07:14:17.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.203+0000 7f43bd3d7640 1 -- 172.21.15.70:0/4164831484 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b815ad60 con 0x7f43b8159ac0 2023-10-08T07:14:17.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.204+0000 7f43bd3d7640 1 -- 172.21.15.70:0/4164831484 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b80b2950 con 0x7f43b8159ac0 2023-10-08T07:14:17.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.205+0000 7f439f7fe640 1 -- 172.21.15.70:0/4164831484 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 135) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f43ac04d450 con 0x7f43b8159ac0 2023-10-08T07:14:17.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.206+0000 7f439f7fe640 1 -- 172.21.15.70:0/4164831484 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f43ac02d040 con 0x7f43b8159ac0 2023-10-08T07:14:17.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.211+0000 7f439f7fe640 1 -- 172.21.15.70:0/4164831484 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43b80b2950 con 0x7f43b8159ac0 2023-10-08T07:14:17.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.405+0000 7f43bd3d7640 1 -- 172.21.15.70:0/4164831484 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43b80b2950 con 0x7f43b8159ac0 2023-10-08T07:14:17.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.409+0000 7f439f7fe640 1 -- 172.21.15.70:0/4164831484 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v240) v1 ==== 74+0+21816 (secure 0 0 0) 0x7f43b80b2950 con 0x7f43b8159ac0 2023-10-08T07:14:17.411 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:17.411 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":240,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:16.815407+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:13:47.007437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:13:47.838023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2480387829":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/920349426":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/2459293689":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/4129076201":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6832/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/365280967":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/2833418931":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/753176203":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/317173509":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:6832/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1018014196":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1778827440":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1217032325":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/2798662747":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/4139611391":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/3002223432":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/3074626816":"2023-10-09T07:14:06.545661+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:17.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.411+0000 7f43bd3d7640 1 -- 172.21.15.70:0/4164831484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b8159ac0 msgr2=0x7f43b815c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.411+0000 7f43bd3d7640 1 --2- 172.21.15.70:0/4164831484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b8159ac0 0x7f43b815c390 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f43ac000c00 tx=0x7f43ac0046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.412+0000 7f43bd3d7640 1 -- 172.21.15.70:0/4164831484 shutdown_connections 2023-10-08T07:14:17.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.412+0000 7f43bd3d7640 1 --2- 172.21.15.70:0/4164831484 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f43b815e330 0x7f43b815c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.412+0000 7f43bd3d7640 1 --2- 172.21.15.70:0/4164831484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43b8159ac0 0x7f43b815c390 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.412+0000 7f43bd3d7640 1 --2- 172.21.15.70:0/4164831484 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f43b80ad790 0x7f43b8152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.412+0000 7f43bd3d7640 1 -- 172.21.15.70:0/4164831484 >> 172.21.15.70:0/4164831484 conn(0x7f43b806a380 msgr2=0x7f43b80c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:17.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.412+0000 7f43bd3d7640 1 -- 172.21.15.70:0/4164831484 shutdown_connections 2023-10-08T07:14:17.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.412+0000 7f43bd3d7640 1 -- 172.21.15.70:0/4164831484 wait complete. 2023-10-08T07:14:17.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T07:13:32.637023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T07:13:33.187098+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '234', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-08T07:13:47.007437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-08T07:13:47.838023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '238', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:14:17.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:14:17.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.648+0000 7fba2171a640 1 Processor -- start 2023-10-08T07:14:17.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.648+0000 7fba2171a640 1 -- start start 2023-10-08T07:14:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.649+0000 7fba2171a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fba1c15dcd0 0x7fba1c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.649+0000 7fba2171a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c0ce290 con 0x7fba1c0d1500 2023-10-08T07:14:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.649+0000 7fba2171a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1c0d0a80 con 0x7fba1c0d1120 2023-10-08T07:14:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.649+0000 7fba2171a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1c0b1fe0 con 0x7fba1c15dcd0 2023-10-08T07:14:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba1b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fba1c0d1500 0x7fba1c0cdb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47680/0 (socket says 172.21.15.70:47680) 2023-10-08T07:14:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba1b7fe640 1 -- 172.21.15.70:0/3059641395 learned_addr learned my addr 172.21.15.70:0/3059641395 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba20f19640 1 --2- 172.21.15.70:0/3059641395 >> v2:172.21.15.181:3300/0 conn(0x7fba1c15dcd0 0x7fba1c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691954492 0 0) 0x7fba1c0ce290 con 0x7fba1c0d1500 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba08003520 con 0x7fba1c0d1500 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2828206975 0 0) 0x7fba08003520 con 0x7fba1c0d1500 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 >> v2:172.21.15.181:3300/0 conn(0x7fba1c15dcd0 msgr2=0x7fba1c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba1affd640 1 --2- 172.21.15.70:0/3059641395 >> v2:172.21.15.181:3300/0 conn(0x7fba1c15dcd0 0x7fba1c0d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 >> v1:172.21.15.181:6789/0 conn(0x7fba1c0d1120 legacy=0x7fba1c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c0b4580 con 0x7fba1c0d1500 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.650+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fba04001670 con 0x7fba1c0d1500 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 >> v1:172.21.15.70:6789/0 conn(0x7fba1c0d1500 legacy=0x7fba1c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1affd640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1affd640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba080047c0 0x7fba08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1affd640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba08005180 0x7fba0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0b1fe0 con 0x7fba08003970 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0800c120 con 0x7fba080047c0 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0800c2e0 con 0x7fba08005180 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1bfff640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1b7fe640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba080047c0 0x7fba08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1bfff640 1 -- 172.21.15.70:0/3059641395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba08005180 msgr2=0x7fba0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba20f19640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba08005180 0x7fba0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1bfff640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba08005180 0x7fba0800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1bfff640 1 -- 172.21.15.70:0/3059641395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba080047c0 msgr2=0x7fba08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.651+0000 7fba1bfff640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba080047c0 0x7fba08004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.652+0000 7fba1bfff640 1 -- 172.21.15.70:0/3059641395 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba08003520 con 0x7fba08003970 2023-10-08T07:14:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.652+0000 7fba1b7fe640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba080047c0 0x7fba08004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.652+0000 7fba20f19640 1 -- 172.21.15.70:0/3059641395 reap_dead start 2023-10-08T07:14:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.653+0000 7fba1bfff640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba08003d90 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fba0c001e70 tx=0x7fba080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c90626a245345d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.653+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c003bc0 con 0x7fba08003970 2023-10-08T07:14:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.653+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c003d20 con 0x7fba08003970 2023-10-08T07:14:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.653+0000 7fba1affd640 1 -- 172.21.15.70:0/3059641395 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c005310 con 0x7fba08003970 2023-10-08T07:14:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.653+0000 7fba2171a640 1 -- 172.21.15.70:0/3059641395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 msgr2=0x7fba08003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.653+0000 7fba2171a640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba08003d90 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7fba0c001e70 tx=0x7fba080057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.654+0000 7fba2171a640 1 -- 172.21.15.70:0/3059641395 shutdown_connections 2023-10-08T07:14:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.654+0000 7fba2171a640 1 --2- 172.21.15.70:0/3059641395 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba08003d90 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.654+0000 7fba2171a640 1 -- 172.21.15.70:0/3059641395 >> 172.21.15.70:0/3059641395 conn(0x7fba1c0d2960 msgr2=0x7fba1c0d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.654+0000 7fba2171a640 1 -- 172.21.15.70:0/3059641395 shutdown_connections 2023-10-08T07:14:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.654+0000 7fba2171a640 1 -- 172.21.15.70:0/3059641395 wait complete. 2023-10-08T07:14:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.655+0000 7fba2171a640 1 Processor -- start 2023-10-08T07:14:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.656+0000 7fba2171a640 1 -- start start 2023-10-08T07:14:17.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.656+0000 7fba2171a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba1c15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.656+0000 7fba2171a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c0c6c90 0x7fba1c0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.656+0000 7fba2171a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c0c70f0 0x7fba1c06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba2171a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0c6870 con 0x7fba08003970 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba2171a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0c8120 con 0x7fba1c0c6c90 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba2171a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c0b6390 con 0x7fba1c0c70f0 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba1bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba1c15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba1bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba1c15ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39828/0 (socket says 172.21.15.70:39828) 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba1bfff640 1 -- 172.21.15.70:0/1056049610 learned_addr learned my addr 172.21.15.70:0/1056049610 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba1bfff640 1 -- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c0c70f0 msgr2=0x7fba1c06b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba1bfff640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c0c70f0 0x7fba1c06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba1bfff640 1 -- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c0c6c90 msgr2=0x7fba1c0b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba1bfff640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c0c6c90 0x7fba1c0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba1bfff640 1 -- 172.21.15.70:0/1056049610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c0b5dd0 con 0x7fba08003970 2023-10-08T07:14:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.657+0000 7fba1bfff640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba1c15ba40 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fba0c005040 tx=0x7fba0c0014f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.658+0000 7fba18ff9640 1 -- 172.21.15.70:0/1056049610 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c03e070 con 0x7fba08003970 2023-10-08T07:14:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.658+0000 7fba18ff9640 1 -- 172.21.15.70:0/1056049610 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c0052e0 con 0x7fba08003970 2023-10-08T07:14:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.658+0000 7fba18ff9640 1 -- 172.21.15.70:0/1056049610 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c005650 con 0x7fba08003970 2023-10-08T07:14:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.658+0000 7fba2171a640 1 -- 172.21.15.70:0/1056049610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c0b60b0 con 0x7fba08003970 2023-10-08T07:14:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.658+0000 7fba2171a640 1 -- 172.21.15.70:0/1056049610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c15bf80 con 0x7fba08003970 2023-10-08T07:14:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.659+0000 7fba2171a640 1 -- 172.21.15.70:0/1056049610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c074150 con 0x7fba08003970 2023-10-08T07:14:17.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.660+0000 7fba18ff9640 1 -- 172.21.15.70:0/1056049610 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 135) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fba0c007040 con 0x7fba08003970 2023-10-08T07:14:17.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.661+0000 7fba18ff9640 1 -- 172.21.15.70:0/1056049610 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7fba0c0aa8f0 con 0x7fba08003970 2023-10-08T07:14:17.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.665+0000 7fba18ff9640 1 -- 172.21.15.70:0/1056049610 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fba0c0756c0 con 0x7fba08003970 2023-10-08T07:14:17.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.839+0000 7fba18ff9640 1 -- 172.21.15.70:0/1056049610 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 136) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fba0c075170 con 0x7fba08003970 2023-10-08T07:14:17.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.840+0000 7fba18ff9640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb9fc073490 0x7fb9fc075880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:17.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.841+0000 7fba1b7fe640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb9fc073490 0x7fb9fc075880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:17.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.845+0000 7fba1b7fe640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb9fc073490 0x7fb9fc075880 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba04000ff0 tx=0x7fba04009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:17.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.862+0000 7fba2171a640 1 -- 172.21.15.70:0/1056049610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba1c0c6ae0 con 0x7fba08003970 2023-10-08T07:14:17.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.863+0000 7fba18ff9640 1 -- 172.21.15.70:0/1056049610 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v240) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba0c047070 con 0x7fba08003970 2023-10-08T07:14:17.865 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:14:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.866+0000 7fba2171a640 1 -- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb9fc073490 msgr2=0x7fb9fc075880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.866+0000 7fba2171a640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb9fc073490 0x7fb9fc075880 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba04000ff0 tx=0x7fba04009040 comp rx=0 tx=0).stop 2023-10-08T07:14:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.866+0000 7fba2171a640 1 -- 172.21.15.70:0/1056049610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 msgr2=0x7fba1c15ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.866+0000 7fba2171a640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba1c15ba40 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fba0c005040 tx=0x7fba0c0014f0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.866+0000 7fba2171a640 1 -- 172.21.15.70:0/1056049610 shutdown_connections 2023-10-08T07:14:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.866+0000 7fba2171a640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb9fc073490 0x7fb9fc075880 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.866+0000 7fba2171a640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fba1c0c70f0 0x7fba1c06b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.866+0000 7fba2171a640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fba1c0c6c90 0x7fba1c0b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.866+0000 7fba2171a640 1 --2- 172.21.15.70:0/1056049610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba08003970 0x7fba1c15ba40 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:17.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.866+0000 7fba2171a640 1 -- 172.21.15.70:0/1056049610 >> 172.21.15.70:0/1056049610 conn(0x7fba1c0d2960 msgr2=0x7fba1c06c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:17.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.867+0000 7fba2171a640 1 -- 172.21.15.70:0/1056049610 shutdown_connections 2023-10-08T07:14:17.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:17.867+0000 7fba2171a640 1 -- 172.21.15.70:0/1056049610 wait complete. 2023-10-08T07:14:17.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:14:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.092+0000 7ff95a754640 1 Processor -- start 2023-10-08T07:14:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.092+0000 7ff95a754640 1 -- start start 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.094+0000 7ff95a754640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9541599d0 0x7ff9540cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.094+0000 7ff95a754640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9540b3c50 0x7ff9540caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.094+0000 7ff95a754640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9540cb8a0 con 0x7ff9540cb410 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.094+0000 7ff95a754640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff954159db0 con 0x7ff9540b3c50 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.094+0000 7ff95a754640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff954071fe0 con 0x7ff9541599d0 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.094+0000 7ff958f51640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9540b3c50 0x7ff9540caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.094+0000 7ff959f53640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff9540cb410 0x7ff9540c8b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47688/0 (socket says 172.21.15.70:47688) 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.094+0000 7ff958f51640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9540b3c50 0x7ff9540caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39836/0 (socket says 172.21.15.70:39836) 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.094+0000 7ff959f53640 1 -- 172.21.15.70:0/3370472497 learned_addr learned my addr 172.21.15.70:0/3370472497 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.095+0000 7ff959752640 1 --2- 172.21.15.70:0/3370472497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9541599d0 0x7ff9540cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.095+0000 7ff958f51640 1 -- 172.21.15.70:0/3370472497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9541599d0 msgr2=0x7ff9540cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:18.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.095+0000 7ff958f51640 1 --2- 172.21.15.70:0/3370472497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9541599d0 0x7ff9540cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.095+0000 7ff958f51640 1 -- 172.21.15.70:0/3370472497 >> v1:172.21.15.70:6789/0 conn(0x7ff9540cb410 legacy=0x7ff9540c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.095+0000 7ff958f51640 1 -- 172.21.15.70:0/3370472497 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9540b44c0 con 0x7ff9540b3c50 2023-10-08T07:14:18.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.095+0000 7ff94bfff640 1 -- 172.21.15.70:0/3370472497 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097944579 0 0) 0x7ff9540cb8a0 con 0x7ff9540cb410 2023-10-08T07:14:18.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.095+0000 7ff959752640 1 --2- 172.21.15.70:0/3370472497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9541599d0 0x7ff9540cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.096+0000 7ff958f51640 1 --2- 172.21.15.70:0/3370472497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 0x7ff9540caed0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7ff9440097a0 tx=0x7ff94402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddbd065fd2fc60 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.096+0000 7ff94bfff640 1 -- 172.21.15.70:0/3370472497 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94402d040 con 0x7ff9540b3c50 2023-10-08T07:14:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.096+0000 7ff94bfff640 1 -- 172.21.15.70:0/3370472497 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff944031810 con 0x7ff9540b3c50 2023-10-08T07:14:18.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.096+0000 7ff94bfff640 1 -- 172.21.15.70:0/3370472497 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944031ba0 con 0x7ff9540b3c50 2023-10-08T07:14:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.097+0000 7ff95a754640 1 -- 172.21.15.70:0/3370472497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 msgr2=0x7ff9540caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.097+0000 7ff95a754640 1 --2- 172.21.15.70:0/3370472497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 0x7ff9540caed0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7ff9440097a0 tx=0x7ff94402b8b0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.097+0000 7ff95a754640 1 -- 172.21.15.70:0/3370472497 shutdown_connections 2023-10-08T07:14:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.097+0000 7ff95a754640 1 --2- 172.21.15.70:0/3370472497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 0x7ff9540caed0 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.097+0000 7ff95a754640 1 --2- 172.21.15.70:0/3370472497 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9541599d0 0x7ff9540cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.097+0000 7ff95a754640 1 -- 172.21.15.70:0/3370472497 >> 172.21.15.70:0/3370472497 conn(0x7ff95406c050 msgr2=0x7ff95406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.097+0000 7ff95a754640 1 -- 172.21.15.70:0/3370472497 shutdown_connections 2023-10-08T07:14:18.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.097+0000 7ff95a754640 1 -- 172.21.15.70:0/3370472497 wait complete. 2023-10-08T07:14:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.099+0000 7ff95a754640 1 Processor -- start 2023-10-08T07:14:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.099+0000 7ff95a754640 1 -- start start 2023-10-08T07:14:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff95a754640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 0x7ff9541468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff95a754640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9540cb410 0x7ff9540ac1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff95a754640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9541599d0 0x7ff9541473f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff95a754640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9540ab140 con 0x7ff9540b3c50 2023-10-08T07:14:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff95a754640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9540ab420 con 0x7ff9540cb410 2023-10-08T07:14:18.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff95a754640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9540ab700 con 0x7ff9541599d0 2023-10-08T07:14:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff959752640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 0x7ff9541468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff959752640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 0x7ff9541468a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39852/0 (socket says 172.21.15.70:39852) 2023-10-08T07:14:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff959752640 1 -- 172.21.15.70:0/1527082778 learned_addr learned my addr 172.21.15.70:0/1527082778 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff959752640 1 -- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9541599d0 msgr2=0x7ff9541473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff959752640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9541599d0 0x7ff9541473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff959752640 1 -- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9540cb410 msgr2=0x7ff9540ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff959752640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9540cb410 0x7ff9540ac1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.100+0000 7ff959752640 1 -- 172.21.15.70:0/1527082778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff944009400 con 0x7ff9540b3c50 2023-10-08T07:14:18.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.101+0000 7ff959752640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 0x7ff9541468a0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7ff95000c7f0 tx=0x7ff95000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:18.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.101+0000 7ff949ffb640 1 -- 172.21.15.70:0/1527082778 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950007820 con 0x7ff9540b3c50 2023-10-08T07:14:18.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.101+0000 7ff949ffb640 1 -- 172.21.15.70:0/1527082778 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff950007980 con 0x7ff9540b3c50 2023-10-08T07:14:18.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.101+0000 7ff949ffb640 1 -- 172.21.15.70:0/1527082778 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950007cf0 con 0x7ff9540b3c50 2023-10-08T07:14:18.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.101+0000 7ff95a754640 1 -- 172.21.15.70:0/1527082778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9540ab9e0 con 0x7ff9540b3c50 2023-10-08T07:14:18.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.101+0000 7ff95a754640 1 -- 172.21.15.70:0/1527082778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff954146de0 con 0x7ff9540b3c50 2023-10-08T07:14:18.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.102+0000 7ff95a754640 1 -- 172.21.15.70:0/1527082778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff954074150 con 0x7ff9540b3c50 2023-10-08T07:14:18.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.103+0000 7ff949ffb640 1 -- 172.21.15.70:0/1527082778 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 136) v1 ==== 89067+0+0 (secure 0 0 0) 0x7ff9500113f0 con 0x7ff9540b3c50 2023-10-08T07:14:18.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.104+0000 7ff949ffb640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff93806acc0 0x7ff93806d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.104+0000 7ff949ffb640 1 -- 172.21.15.70:0/1527082778 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7ff950089e60 con 0x7ff9540b3c50 2023-10-08T07:14:18.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.105+0000 7ff958f51640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff93806acc0 0x7ff93806d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.106+0000 7ff958f51640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff93806acc0 0x7ff93806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff944036000 tx=0x7ff944030000 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:18.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.108+0000 7ff949ffb640 1 -- 172.21.15.70:0/1527082778 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff950054930 con 0x7ff9540b3c50 2023-10-08T07:14:18.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.296+0000 7ff95a754640 1 -- 172.21.15.70:0/1527082778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9540c4c50 con 0x7ff9540b3c50 2023-10-08T07:14:18.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.297+0000 7ff949ffb640 1 -- 172.21.15.70:0/1527082778 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v240) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff95001f0a0 con 0x7ff9540b3c50 2023-10-08T07:14:18.299 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:14:18.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.299+0000 7ff95a754640 1 -- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff93806acc0 msgr2=0x7ff93806d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.299+0000 7ff95a754640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff93806acc0 0x7ff93806d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff944036000 tx=0x7ff944030000 comp rx=0 tx=0).stop 2023-10-08T07:14:18.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.299+0000 7ff95a754640 1 -- 172.21.15.70:0/1527082778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 msgr2=0x7ff9541468a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.299+0000 7ff95a754640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 0x7ff9541468a0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7ff95000c7f0 tx=0x7ff95000ccb0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.300+0000 7ff95a754640 1 -- 172.21.15.70:0/1527082778 shutdown_connections 2023-10-08T07:14:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.300+0000 7ff95a754640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff93806acc0 0x7ff93806d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.300+0000 7ff95a754640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff9541599d0 0x7ff9541473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.300+0000 7ff95a754640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff9540cb410 0x7ff9540ac1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.300+0000 7ff95a754640 1 --2- 172.21.15.70:0/1527082778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9540b3c50 0x7ff9541468a0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.300+0000 7ff95a754640 1 -- 172.21.15.70:0/1527082778 >> 172.21.15.70:0/1527082778 conn(0x7ff95406c050 msgr2=0x7ff95415a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.300+0000 7ff95a754640 1 -- 172.21.15.70:0/1527082778 shutdown_connections 2023-10-08T07:14:18.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.300+0000 7ff95a754640 1 -- 172.21.15.70:0/1527082778 wait complete. 2023-10-08T07:14:18.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:14:18.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.533+0000 7f0916a6f640 1 Processor -- start 2023-10-08T07:14:18.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.534+0000 7f0916a6f640 1 -- start start 2023-10-08T07:14:18.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.535+0000 7f0916a6f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f091006dfc0 0x7f0910070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.535+0000 7f0916a6f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0910071070 0x7f09100cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.535+0000 7f0916a6f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09100b1cc0 con 0x7f09100afbe0 2023-10-08T07:14:18.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.535+0000 7f0916a6f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09100b2620 con 0x7f091006dfc0 2023-10-08T07:14:18.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.535+0000 7f0916a6f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09100b2060 con 0x7f0910071070 2023-10-08T07:14:18.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.536+0000 7f0907fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f091006dfc0 0x7f0910070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.536+0000 7f0907fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f091006dfc0 0x7f0910070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39862/0 (socket says 172.21.15.70:39862) 2023-10-08T07:14:18.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.536+0000 7f0907fff640 1 -- 172.21.15.70:0/2089030316 learned_addr learned my addr 172.21.15.70:0/2089030316 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:18.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.536+0000 7f0907fff640 1 -- 172.21.15.70:0/2089030316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0910071070 msgr2=0x7f09100cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:18.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.536+0000 7f0907fff640 1 --2- 172.21.15.70:0/2089030316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0910071070 0x7f09100cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.536+0000 7f0907fff640 1 -- 172.21.15.70:0/2089030316 >> v1:172.21.15.181:6789/0 conn(0x7f09100afbe0 legacy=0x7f09100b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:18.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.536+0000 7f0907fff640 1 -- 172.21.15.70:0/2089030316 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09100b2340 con 0x7f091006dfc0 2023-10-08T07:14:18.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.538+0000 7f0907fff640 1 --2- 172.21.15.70:0/2089030316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091006dfc0 0x7f0910070b30 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f0900009870 tx=0x7f090002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a010ff9efb756a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.538+0000 7f09077fe640 1 -- 172.21.15.70:0/2089030316 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900031640 con 0x7f091006dfc0 2023-10-08T07:14:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.538+0000 7f09077fe640 1 -- 172.21.15.70:0/2089030316 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09000317a0 con 0x7f091006dfc0 2023-10-08T07:14:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.538+0000 7f09077fe640 1 -- 172.21.15.70:0/2089030316 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0900031b30 con 0x7f091006dfc0 2023-10-08T07:14:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.539+0000 7f0916a6f640 1 -- 172.21.15.70:0/2089030316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091006dfc0 msgr2=0x7f0910070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.539+0000 7f0916a6f640 1 --2- 172.21.15.70:0/2089030316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091006dfc0 0x7f0910070b30 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7f0900009870 tx=0x7f090002b8f0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.539+0000 7f0916a6f640 1 -- 172.21.15.70:0/2089030316 shutdown_connections 2023-10-08T07:14:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.539+0000 7f0916a6f640 1 --2- 172.21.15.70:0/2089030316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0910071070 0x7f09100cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.539+0000 7f0916a6f640 1 --2- 172.21.15.70:0/2089030316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091006dfc0 0x7f0910070b30 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.539+0000 7f0916a6f640 1 -- 172.21.15.70:0/2089030316 >> 172.21.15.70:0/2089030316 conn(0x7f091006a5d0 msgr2=0x7f09100b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.539+0000 7f0916a6f640 1 -- 172.21.15.70:0/2089030316 shutdown_connections 2023-10-08T07:14:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.539+0000 7f0916a6f640 1 -- 172.21.15.70:0/2089030316 wait complete. 2023-10-08T07:14:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.541+0000 7f0916a6f640 1 Processor -- start 2023-10-08T07:14:18.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.541+0000 7f0916a6f640 1 -- start start 2023-10-08T07:14:18.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f0916a6f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f091006dfc0 0x7f09101f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f0916a6f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0910071070 0x7f09100c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f0916a6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09100afbe0 0x7f091015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f0916a6f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09101f3d90 con 0x7f09100afbe0 2023-10-08T07:14:18.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f0916a6f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09100c8f10 con 0x7f091006dfc0 2023-10-08T07:14:18.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f0916a6f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09100c31b0 con 0x7f0910071070 2023-10-08T07:14:18.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f091500c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09100afbe0 0x7f091015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f091500c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09100afbe0 0x7f091015c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39868/0 (socket says 172.21.15.70:39868) 2023-10-08T07:14:18.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f091500c640 1 -- 172.21.15.70:0/993726905 learned_addr learned my addr 172.21.15.70:0/993726905 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:18.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f091480b640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f091006dfc0 0x7f09101f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.542+0000 7f0907fff640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0910071070 0x7f09100c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f091500c640 1 -- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0910071070 msgr2=0x7f09100c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f091500c640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0910071070 0x7f09100c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f091500c640 1 -- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f091006dfc0 msgr2=0x7f09101f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f091500c640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f091006dfc0 0x7f09101f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f091500c640 1 -- 172.21.15.70:0/993726905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09000094d0 con 0x7f09100afbe0 2023-10-08T07:14:18.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f091500c640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09100afbe0 0x7f091015c840 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f090800b9e0 tx=0x7f090800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f09057fa640 1 -- 172.21.15.70:0/993726905 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090800cb20 con 0x7f09100afbe0 2023-10-08T07:14:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f09057fa640 1 -- 172.21.15.70:0/993726905 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090800cca0 con 0x7f09100afbe0 2023-10-08T07:14:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f09057fa640 1 -- 172.21.15.70:0/993726905 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0908014680 con 0x7f09100afbe0 2023-10-08T07:14:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f0916a6f640 1 -- 172.21.15.70:0/993726905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09100c91d0 con 0x7f09100afbe0 2023-10-08T07:14:18.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.543+0000 7f0916a6f640 1 -- 172.21.15.70:0/993726905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09100c44c0 con 0x7f09100afbe0 2023-10-08T07:14:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.544+0000 7f0916a6f640 1 -- 172.21.15.70:0/993726905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09100c96f0 con 0x7f09100afbe0 2023-10-08T07:14:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.545+0000 7f09057fa640 1 -- 172.21.15.70:0/993726905 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 136) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f090801d3b0 con 0x7f09100afbe0 2023-10-08T07:14:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.546+0000 7f09057fa640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08fc06acc0 0x7f08fc06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.546+0000 7f09057fa640 1 -- 172.21.15.70:0/993726905 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f0908089ab0 con 0x7f09100afbe0 2023-10-08T07:14:18.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.547+0000 7f091480b640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08fc06acc0 0x7f08fc06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.548+0000 7f091480b640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08fc06acc0 0x7f08fc06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f090c006fd0 tx=0x7f090c008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:18.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.551+0000 7f09057fa640 1 -- 172.21.15.70:0/993726905 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09100c96f0 con 0x7f09100afbe0 2023-10-08T07:14:18.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.738+0000 7f0916a6f640 1 -- 172.21.15.70:0/993726905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09100c96f0 con 0x7f09100afbe0 2023-10-08T07:14:18.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.739+0000 7f09057fa640 1 -- 172.21.15.70:0/993726905 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v240) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09100c96f0 con 0x7f09100afbe0 2023-10-08T07:14:18.741 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:14:18.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.741+0000 7f0916a6f640 1 -- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08fc06acc0 msgr2=0x7f08fc06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.741+0000 7f0916a6f640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08fc06acc0 0x7f08fc06d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f090c006fd0 tx=0x7f090c008040 comp rx=0 tx=0).stop 2023-10-08T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.742+0000 7f0916a6f640 1 -- 172.21.15.70:0/993726905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09100afbe0 msgr2=0x7f091015c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.742+0000 7f0916a6f640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09100afbe0 0x7f091015c840 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7f090800b9e0 tx=0x7f090800bea0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.742+0000 7f0916a6f640 1 -- 172.21.15.70:0/993726905 shutdown_connections 2023-10-08T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.742+0000 7f0916a6f640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08fc06acc0 0x7f08fc06d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.742+0000 7f0916a6f640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09100afbe0 0x7f091015c840 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.742+0000 7f0916a6f640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0910071070 0x7f09100c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.742+0000 7f0916a6f640 1 --2- 172.21.15.70:0/993726905 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f091006dfc0 0x7f09101f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.742+0000 7f0916a6f640 1 -- 172.21.15.70:0/993726905 >> 172.21.15.70:0/993726905 conn(0x7f091006a5d0 msgr2=0x7f091015dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:18.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.742+0000 7f0916a6f640 1 -- 172.21.15.70:0/993726905 shutdown_connections 2023-10-08T07:14:18.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.742+0000 7f0916a6f640 1 -- 172.21.15.70:0/993726905 wait complete. 2023-10-08T07:14:18.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-08T07:14:18.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.983+0000 7fa1ea11d640 1 Processor -- start 2023-10-08T07:14:18.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.983+0000 7fa1ea11d640 1 -- start start 2023-10-08T07:14:18.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1ea11d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1e406dfc0 0x7fa1e4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1ea11d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e40b1cc0 con 0x7fa1e4071070 2023-10-08T07:14:18.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1ea11d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e40b2620 con 0x7fa1e40afbe0 2023-10-08T07:14:18.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1ea11d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e40b2060 con 0x7fa1e406dfc0 2023-10-08T07:14:18.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1dbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1e406dfc0 0x7fa1e4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1e3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa1e4071070 0x7fa1e40cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47696/0 (socket says 172.21.15.70:47696) 2023-10-08T07:14:18.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1e3fff640 1 -- 172.21.15.70:0/192839403 learned_addr learned my addr 172.21.15.70:0/192839403 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:18.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1dbfff640 1 -- 172.21.15.70:0/192839403 >> v1:172.21.15.181:6789/0 conn(0x7fa1e40afbe0 legacy=0x7fa1e40b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:18.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1dbfff640 1 -- 172.21.15.70:0/192839403 >> v1:172.21.15.70:6789/0 conn(0x7fa1e4071070 legacy=0x7fa1e40cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1e2ffd640 1 -- 172.21.15.70:0/192839403 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059828643 0 0) 0x7fa1e40b1cc0 con 0x7fa1e4071070 2023-10-08T07:14:18.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.985+0000 7fa1dbfff640 1 -- 172.21.15.70:0/192839403 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e40b2340 con 0x7fa1e406dfc0 2023-10-08T07:14:18.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.987+0000 7fa1dbfff640 1 --2- 172.21.15.70:0/192839403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 0x7fa1e4070b30 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40b2810 tx=0x7fa1cc02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3659e83d37d55554 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:18.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.987+0000 7fa1e2ffd640 1 -- 172.21.15.70:0/192839403 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc030800 con 0x7fa1e406dfc0 2023-10-08T07:14:18.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.987+0000 7fa1e2ffd640 1 -- 172.21.15.70:0/192839403 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc030960 con 0x7fa1e406dfc0 2023-10-08T07:14:18.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.987+0000 7fa1e2ffd640 1 -- 172.21.15.70:0/192839403 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc030cf0 con 0x7fa1e406dfc0 2023-10-08T07:14:18.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.987+0000 7fa1ea11d640 1 -- 172.21.15.70:0/192839403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 msgr2=0x7fa1e4070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.987+0000 7fa1ea11d640 1 --2- 172.21.15.70:0/192839403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 0x7fa1e4070b30 secure :-1 s=READY pgs=2510 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40b2810 tx=0x7fa1cc02b940 comp rx=0 tx=0).stop 2023-10-08T07:14:18.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.988+0000 7fa1ea11d640 1 -- 172.21.15.70:0/192839403 shutdown_connections 2023-10-08T07:14:18.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.988+0000 7fa1ea11d640 1 --2- 172.21.15.70:0/192839403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 0x7fa1e4070b30 unknown :-1 s=CLOSED pgs=2510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.988+0000 7fa1ea11d640 1 -- 172.21.15.70:0/192839403 >> 172.21.15.70:0/192839403 conn(0x7fa1e406a5d0 msgr2=0x7fa1e40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:18.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.988+0000 7fa1ea11d640 1 -- 172.21.15.70:0/192839403 shutdown_connections 2023-10-08T07:14:18.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.988+0000 7fa1ea11d640 1 -- 172.21.15.70:0/192839403 wait complete. 2023-10-08T07:14:18.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.989+0000 7fa1ea11d640 1 Processor -- start 2023-10-08T07:14:18.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.990+0000 7fa1ea11d640 1 -- start start 2023-10-08T07:14:18.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.990+0000 7fa1ea11d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 0x7fa1e40cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.990+0000 7fa1ea11d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e4071070 0x7fa1e41492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.990+0000 7fa1ea11d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e40afbe0 0x7fa1e41e62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.990+0000 7fa1ea11d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e415d110 con 0x7fa1e406dfc0 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1ea11d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e41497e0 con 0x7fa1e4071070 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1ea11d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e41e67f0 con 0x7fa1e40afbe0 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 0x7fa1e40cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 0x7fa1e40cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39876/0 (socket says 172.21.15.70:39876) 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e37fe640 1 -- 172.21.15.70:0/886575041 learned_addr learned my addr 172.21.15.70:0/886575041 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1dbfff640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e4071070 0x7fa1e41492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e3fff640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e40afbe0 0x7fa1e41e62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e37fe640 1 -- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e40afbe0 msgr2=0x7fa1e41e62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e37fe640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e40afbe0 0x7fa1e41e62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e37fe640 1 -- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e4071070 msgr2=0x7fa1e41492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e37fe640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e4071070 0x7fa1e41492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e37fe640 1 -- 172.21.15.70:0/886575041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc009400 con 0x7fa1e406dfc0 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1dbfff640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e4071070 0x7fa1e41492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e3fff640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e40afbe0 0x7fa1e41e62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.991+0000 7fa1e37fe640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 0x7fa1e40cf210 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c450 tx=0x7fa1d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.992+0000 7fa1e0ff9640 1 -- 172.21.15.70:0/886575041 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40052d0 con 0x7fa1e406dfc0 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.992+0000 7fa1e0ff9640 1 -- 172.21.15.70:0/886575041 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d4005430 con 0x7fa1e406dfc0 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.992+0000 7fa1e0ff9640 1 -- 172.21.15.70:0/886575041 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4003830 con 0x7fa1e406dfc0 2023-10-08T07:14:18.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.992+0000 7fa1ea11d640 1 -- 172.21.15.70:0/886575041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e40c2790 con 0x7fa1e406dfc0 2023-10-08T07:14:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.992+0000 7fa1ea11d640 1 -- 172.21.15.70:0/886575041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e415ca50 con 0x7fa1e406dfc0 2023-10-08T07:14:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.993+0000 7fa1ea11d640 1 -- 172.21.15.70:0/886575041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e404e160 con 0x7fa1e406dfc0 2023-10-08T07:14:18.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.994+0000 7fa1e0ff9640 1 -- 172.21.15.70:0/886575041 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 137) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fa1d4003990 con 0x7fa1e406dfc0 2023-10-08T07:14:18.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.995+0000 7fa1e0ff9640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fa1c406b050 0x7fa1c406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:18.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.995+0000 7fa1e0ff9640 1 -- 172.21.15.70:0/886575041 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7fa1d4089740 con 0x7fa1e406dfc0 2023-10-08T07:14:18.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.996+0000 7fa1dbfff640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fa1c406b050 0x7fa1c406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:18.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.997+0000 7fa1dbfff640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fa1c406b050 0x7fa1c406d510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc036000 tx=0x7fa1cc008030 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:19.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:18.999+0000 7fa1e0ff9640 1 -- 172.21.15.70:0/886575041 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1d4053d10 con 0x7fa1e406dfc0 2023-10-08T07:14:19.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.189+0000 7fa1ea11d640 1 -- 172.21.15.70:0/886575041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1e40c6000 con 0x7fa1e406dfc0 2023-10-08T07:14:19.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.190+0000 7fa1e0ff9640 1 -- 172.21.15.70:0/886575041 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v240) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa1d4017020 con 0x7fa1e406dfc0 2023-10-08T07:14:19.192 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:14:19.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.192+0000 7fa1ea11d640 1 -- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fa1c406b050 msgr2=0x7fa1c406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.192+0000 7fa1ea11d640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fa1c406b050 0x7fa1c406d510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc036000 tx=0x7fa1cc008030 comp rx=0 tx=0).stop 2023-10-08T07:14:19.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.192+0000 7fa1ea11d640 1 -- 172.21.15.70:0/886575041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 msgr2=0x7fa1e40cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.192+0000 7fa1ea11d640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 0x7fa1e40cf210 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400c450 tx=0x7fa1d400c910 comp rx=0 tx=0).stop 2023-10-08T07:14:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.192+0000 7fa1ea11d640 1 -- 172.21.15.70:0/886575041 shutdown_connections 2023-10-08T07:14:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.192+0000 7fa1ea11d640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fa1c406b050 0x7fa1c406d510 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.193+0000 7fa1ea11d640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa1e40afbe0 0x7fa1e41e62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.193+0000 7fa1ea11d640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa1e4071070 0x7fa1e41492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.193+0000 7fa1ea11d640 1 --2- 172.21.15.70:0/886575041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1e406dfc0 0x7fa1e40cf210 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.193+0000 7fa1ea11d640 1 -- 172.21.15.70:0/886575041 >> 172.21.15.70:0/886575041 conn(0x7fa1e406a5d0 msgr2=0x7fa1e415ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.193+0000 7fa1ea11d640 1 -- 172.21.15.70:0/886575041 shutdown_connections 2023-10-08T07:14:19.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.193+0000 7fa1ea11d640 1 -- 172.21.15.70:0/886575041 wait complete. 2023-10-08T07:14:19.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-08T07:14:19.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.419+0000 7fc41f13e640 1 Processor -- start 2023-10-08T07:14:19.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.419+0000 7fc41f13e640 1 -- start start 2023-10-08T07:14:19.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.421+0000 7fc41f13e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4180afbe0 0x7fc4180b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.421+0000 7fc41f13e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4180b2620 con 0x7fc41806dfc0 2023-10-08T07:14:19.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.421+0000 7fc41f13e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4180b2060 con 0x7fc4180cb130 2023-10-08T07:14:19.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.421+0000 7fc41f13e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180b2340 con 0x7fc4180afbe0 2023-10-08T07:14:19.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.421+0000 7fc40ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc41806dfc0 0x7fc418070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47702/0 (socket says 172.21.15.70:47702) 2023-10-08T07:14:19.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.421+0000 7fc40ffff640 1 -- 172.21.15.70:0/1207073333 learned_addr learned my addr 172.21.15.70:0/1207073333 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:19.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc41ceda640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4180afbe0 0x7fc4180b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383834262 0 0) 0x7fc4180b2620 con 0x7fc41806dfc0 2023-10-08T07:14:19.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc400003520 con 0x7fc41806dfc0 2023-10-08T07:14:19.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 895835889 0 0) 0x7fc400003520 con 0x7fc41806dfc0 2023-10-08T07:14:19.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4180afbe0 msgr2=0x7fc4180b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:19.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4180afbe0 0x7fc4180b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 >> v1:172.21.15.181:6789/0 conn(0x7fc4180cb130 legacy=0x7fc41806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc418070350 con 0x7fc41806dfc0 2023-10-08T07:14:19.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc408001670 con 0x7fc41806dfc0 2023-10-08T07:14:19.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 >> v1:172.21.15.70:6789/0 conn(0x7fc41806dfc0 legacy=0x7fc418070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4180afbe0 0x7fc4000039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.422+0000 7fc40f7fe640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc400003f20 0x7fc4000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc40f7fe640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc400004d00 0x7fc400005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180b2340 con 0x7fc400003f20 2023-10-08T07:14:19.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40000ba60 con 0x7fc400004d00 2023-10-08T07:14:19.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40000bc20 con 0x7fc4180afbe0 2023-10-08T07:14:19.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc40ffff640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc400003f20 0x7fc4000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc41d6db640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc400004d00 0x7fc400005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc40ffff640 1 -- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4180afbe0 msgr2=0x7fc4000039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:19.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc40ffff640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4180afbe0 0x7fc4000039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc40ffff640 1 -- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc400004d00 msgr2=0x7fc400005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:19.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc40ffff640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc400004d00 0x7fc400005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc40ffff640 1 -- 172.21.15.70:0/1207073333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc400003520 con 0x7fc400003f20 2023-10-08T07:14:19.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.423+0000 7fc41d6db640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc400004d00 0x7fc400005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:19.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.424+0000 7fc40ffff640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc400003f20 0x7fc4000047c0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c190 tx=0x7fc4000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b766dc90a6afb84 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:19.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.425+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408005ad0 con 0x7fc400003f20 2023-10-08T07:14:19.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.425+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc408005c50 con 0x7fc400003f20 2023-10-08T07:14:19.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.425+0000 7fc40f7fe640 1 -- 172.21.15.70:0/1207073333 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40802c8a0 con 0x7fc400003f20 2023-10-08T07:14:19.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.425+0000 7fc41f13e640 1 -- 172.21.15.70:0/1207073333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc400003f20 msgr2=0x7fc4000047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.425+0000 7fc41f13e640 1 --2- 172.21.15.70:0/1207073333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc400003f20 0x7fc4000047c0 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fc40800c190 tx=0x7fc4000057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.425+0000 7fc41d6db640 1 -- 172.21.15.70:0/1207073333 reap_dead start 2023-10-08T07:14:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.425+0000 7fc41f13e640 1 -- 172.21.15.70:0/1207073333 shutdown_connections 2023-10-08T07:14:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.425+0000 7fc41f13e640 1 -- 172.21.15.70:0/1207073333 >> 172.21.15.70:0/1207073333 conn(0x7fc41806a5d0 msgr2=0x7fc4180b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.425+0000 7fc41f13e640 1 -- 172.21.15.70:0/1207073333 shutdown_connections 2023-10-08T07:14:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.425+0000 7fc41f13e640 1 -- 172.21.15.70:0/1207073333 wait complete. 2023-10-08T07:14:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.427+0000 7fc41f13e640 1 Processor -- start 2023-10-08T07:14:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.427+0000 7fc41f13e640 1 -- start start 2023-10-08T07:14:19.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.428+0000 7fc41f13e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc400003f20 0x7fc4181f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.428+0000 7fc41f13e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4181f1920 0x7fc4181ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.428+0000 7fc41f13e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc418158080 0x7fc4180c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.428+0000 7fc41f13e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc418158d20 con 0x7fc418158080 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.428+0000 7fc41f13e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4181ee5e0 con 0x7fc4181f1920 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.428+0000 7fc41f13e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc418158a40 con 0x7fc400003f20 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.428+0000 7fc41d6db640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc418158080 0x7fc4180c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc41d6db640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc418158080 0x7fc4180c2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39884/0 (socket says 172.21.15.70:39884) 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc41d6db640 1 -- 172.21.15.70:0/2492940123 learned_addr learned my addr 172.21.15.70:0/2492940123 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc41ceda640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc400003f20 0x7fc4181f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc40ffff640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4181f1920 0x7fc4181ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc41d6db640 1 -- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc400003f20 msgr2=0x7fc4181f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc41d6db640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc400003f20 0x7fc4181f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc41d6db640 1 -- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4181f1920 msgr2=0x7fc4181ee0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc41d6db640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4181f1920 0x7fc4181ee0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc41d6db640 1 -- 172.21.15.70:0/2492940123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4180c6ed0 con 0x7fc418158080 2023-10-08T07:14:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc41d6db640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc418158080 0x7fc4180c2f90 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fc4180cb510 tx=0x7fc41000d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc40d7fa640 1 -- 172.21.15.70:0/2492940123 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41000b070 con 0x7fc418158080 2023-10-08T07:14:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc40d7fa640 1 -- 172.21.15.70:0/2492940123 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc410007030 con 0x7fc418158080 2023-10-08T07:14:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc40d7fa640 1 -- 172.21.15.70:0/2492940123 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4100073e0 con 0x7fc418158080 2023-10-08T07:14:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.429+0000 7fc41f13e640 1 -- 172.21.15.70:0/2492940123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4180c71b0 con 0x7fc418158080 2023-10-08T07:14:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.430+0000 7fc41f13e640 1 -- 172.21.15.70:0/2492940123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc418158530 con 0x7fc418158080 2023-10-08T07:14:19.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.431+0000 7fc41f13e640 1 -- 172.21.15.70:0/2492940123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4180c76d0 con 0x7fc418158080 2023-10-08T07:14:19.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.432+0000 7fc40d7fa640 1 -- 172.21.15.70:0/2492940123 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 137) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fc410007540 con 0x7fc418158080 2023-10-08T07:14:19.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.433+0000 7fc40d7fa640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc40406b020 0x7fc40406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.433+0000 7fc40d7fa640 1 -- 172.21.15.70:0/2492940123 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7fc410089d90 con 0x7fc418158080 2023-10-08T07:14:19.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.434+0000 7fc41ceda640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc40406b020 0x7fc40406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.435+0000 7fc41ceda640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc40406b020 0x7fc40406d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc414004b30 tx=0x7fc41400b040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:19.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.438+0000 7fc40d7fa640 1 -- 172.21.15.70:0/2492940123 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc410054250 con 0x7fc418158080 2023-10-08T07:14:19.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.625+0000 7fc41f13e640 1 -- 172.21.15.70:0/2492940123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fc41804e160 con 0x7fc418158080 2023-10-08T07:14:19.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.626+0000 7fc40d7fa640 1 -- 172.21.15.70:0/2492940123 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v240) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc410057870 con 0x7fc418158080 2023-10-08T07:14:19.628 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:14:19.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.628+0000 7fc41f13e640 1 -- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc40406b020 msgr2=0x7fc40406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.628+0000 7fc41f13e640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc40406b020 0x7fc40406d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc414004b30 tx=0x7fc41400b040 comp rx=0 tx=0).stop 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.628+0000 7fc41f13e640 1 -- 172.21.15.70:0/2492940123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc418158080 msgr2=0x7fc4180c2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.628+0000 7fc41f13e640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc418158080 0x7fc4180c2f90 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fc4180cb510 tx=0x7fc41000d750 comp rx=0 tx=0).stop 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.629+0000 7fc41f13e640 1 -- 172.21.15.70:0/2492940123 shutdown_connections 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.629+0000 7fc41f13e640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc40406b020 0x7fc40406d4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.629+0000 7fc41f13e640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc418158080 0x7fc4180c2f90 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.629+0000 7fc41f13e640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4181f1920 0x7fc4181ee0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.629+0000 7fc41f13e640 1 --2- 172.21.15.70:0/2492940123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc400003f20 0x7fc4181f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.629+0000 7fc41f13e640 1 -- 172.21.15.70:0/2492940123 >> 172.21.15.70:0/2492940123 conn(0x7fc41806a5d0 msgr2=0x7fc4180b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.629+0000 7fc41f13e640 1 -- 172.21.15.70:0/2492940123 shutdown_connections 2023-10-08T07:14:19.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.629+0000 7fc41f13e640 1 -- 172.21.15.70:0/2492940123 wait complete. 2023-10-08T07:14:19.648 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:14:19.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.857+0000 7f3999625640 1 Processor -- start 2023-10-08T07:14:19.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.858+0000 7f3999625640 1 -- start start 2023-10-08T07:14:19.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.859+0000 7f3999625640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940b07d0 0x7f39941614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.859+0000 7f3999625640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39940cab40 0x7f39940c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.859+0000 7f3999625640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39940c9020 con 0x7f399415cca0 2023-10-08T07:14:19.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.859+0000 7f3999625640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39940ccc20 con 0x7f39940cab40 2023-10-08T07:14:19.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.859+0000 7f3999625640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c92e0 con 0x7f39940b07d0 2023-10-08T07:14:19.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.859+0000 7f3998e24640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39940cab40 0x7f39940c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.859+0000 7f3993fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f399415cca0 0x7f39940b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47718/0 (socket says 172.21.15.70:47718) 2023-10-08T07:14:19.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.859+0000 7f3998e24640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39940cab40 0x7f39940c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39892/0 (socket says 172.21.15.70:39892) 2023-10-08T07:14:19.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.859+0000 7f3993fff640 1 -- 172.21.15.70:0/2406956251 learned_addr learned my addr 172.21.15.70:0/2406956251 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:19.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.860+0000 7f39937fe640 1 --2- 172.21.15.70:0/2406956251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940b07d0 0x7f39941614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.860+0000 7f3998e24640 1 -- 172.21.15.70:0/2406956251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940b07d0 msgr2=0x7f39941614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:19.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.860+0000 7f3998e24640 1 --2- 172.21.15.70:0/2406956251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940b07d0 0x7f39941614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.860+0000 7f3992ffd640 1 -- 172.21.15.70:0/2406956251 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458052804 0 0) 0x7f39940c9020 con 0x7f399415cca0 2023-10-08T07:14:19.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.860+0000 7f3998e24640 1 -- 172.21.15.70:0/2406956251 >> v1:172.21.15.70:6789/0 conn(0x7f399415cca0 legacy=0x7f39940b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.860+0000 7f3998e24640 1 -- 172.21.15.70:0/2406956251 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39940cc7b0 con 0x7f39940cab40 2023-10-08T07:14:19.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.860+0000 7f39937fe640 1 --2- 172.21.15.70:0/2406956251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940b07d0 0x7f39941614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:19.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.861+0000 7f3998e24640 1 --2- 172.21.15.70:0/2406956251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39940cab40 0x7f39940c9770 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f398c009d90 tx=0x7f398c02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725f3fb7cd2550c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:19.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.861+0000 7f3992ffd640 1 -- 172.21.15.70:0/2406956251 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c02cb30 con 0x7f39940cab40 2023-10-08T07:14:19.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.861+0000 7f3992ffd640 1 -- 172.21.15.70:0/2406956251 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c02cc90 con 0x7f39940cab40 2023-10-08T07:14:19.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.861+0000 7f3992ffd640 1 -- 172.21.15.70:0/2406956251 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c034440 con 0x7f39940cab40 2023-10-08T07:14:19.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.862+0000 7f3999625640 1 -- 172.21.15.70:0/2406956251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39940cab40 msgr2=0x7f39940c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.862+0000 7f3999625640 1 --2- 172.21.15.70:0/2406956251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39940cab40 0x7f39940c9770 secure :-1 s=READY pgs=2516 cs=0 l=1 rev1=1 crypto rx=0x7f398c009d90 tx=0x7f398c02bd80 comp rx=0 tx=0).stop 2023-10-08T07:14:19.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.862+0000 7f3999625640 1 -- 172.21.15.70:0/2406956251 shutdown_connections 2023-10-08T07:14:19.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.862+0000 7f3999625640 1 --2- 172.21.15.70:0/2406956251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39940cab40 0x7f39940c9770 unknown :-1 s=CLOSED pgs=2516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.862+0000 7f3999625640 1 --2- 172.21.15.70:0/2406956251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940b07d0 0x7f39941614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.862+0000 7f3999625640 1 -- 172.21.15.70:0/2406956251 >> 172.21.15.70:0/2406956251 conn(0x7f39940cb0c0 msgr2=0x7f39940cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:19.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.862+0000 7f3999625640 1 -- 172.21.15.70:0/2406956251 shutdown_connections 2023-10-08T07:14:19.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.862+0000 7f3999625640 1 -- 172.21.15.70:0/2406956251 wait complete. 2023-10-08T07:14:19.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.864+0000 7f3999625640 1 Processor -- start 2023-10-08T07:14:19.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.864+0000 7f3999625640 1 -- start start 2023-10-08T07:14:19.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.865+0000 7f3999625640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940b07d0 0x7f3994159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.865+0000 7f3999625640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940cab40 0x7f39940c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.865+0000 7f3999625640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f399415cca0 0x7f399415f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.865+0000 7f3999625640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c6f40 con 0x7f399415cca0 2023-10-08T07:14:19.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.865+0000 7f3999625640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c62c0 con 0x7f39940b07d0 2023-10-08T07:14:19.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.865+0000 7f3999625640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940c65a0 con 0x7f39940cab40 2023-10-08T07:14:19.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.865+0000 7f3998e24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f399415cca0 0x7f399415f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.865+0000 7f3998e24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f399415cca0 0x7f399415f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39908/0 (socket says 172.21.15.70:39908) 2023-10-08T07:14:19.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.865+0000 7f3998e24640 1 -- 172.21.15.70:0/551832461 learned_addr learned my addr 172.21.15.70:0/551832461 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:19.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3993fff640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940b07d0 0x7f3994159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f39937fe640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940cab40 0x7f39940c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3998e24640 1 -- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940cab40 msgr2=0x7f39940c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3998e24640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940cab40 0x7f39940c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3998e24640 1 -- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940b07d0 msgr2=0x7f3994159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:19.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3998e24640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940b07d0 0x7f3994159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:19.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3998e24640 1 -- 172.21.15.70:0/551832461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c009a70 con 0x7f399415cca0 2023-10-08T07:14:19.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3993fff640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940b07d0 0x7f3994159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:19.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3998e24640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f399415cca0 0x7f399415f2a0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f398c009d90 tx=0x7f398c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:19.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3990ff9640 1 -- 172.21.15.70:0/551832461 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c035e80 con 0x7f399415cca0 2023-10-08T07:14:19.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3990ff9640 1 -- 172.21.15.70:0/551832461 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c009450 con 0x7f399415cca0 2023-10-08T07:14:19.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3990ff9640 1 -- 172.21.15.70:0/551832461 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c003bb0 con 0x7f399415cca0 2023-10-08T07:14:19.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3999625640 1 -- 172.21.15.70:0/551832461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39940c6880 con 0x7f399415cca0 2023-10-08T07:14:19.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.866+0000 7f3999625640 1 -- 172.21.15.70:0/551832461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3994159fa0 con 0x7f399415cca0 2023-10-08T07:14:19.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.868+0000 7f3999625640 1 -- 172.21.15.70:0/551832461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39940c7080 con 0x7f399415cca0 2023-10-08T07:14:19.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.868+0000 7f3990ff9640 1 -- 172.21.15.70:0/551832461 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 137) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f398c03b050 con 0x7f399415cca0 2023-10-08T07:14:19.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.869+0000 7f3990ff9640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f398006b020 0x7f398006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:19.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.869+0000 7f3990ff9640 1 -- 172.21.15.70:0/551832461 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f398c0ac6f0 con 0x7f399415cca0 2023-10-08T07:14:19.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.870+0000 7f3993fff640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f398006b020 0x7f398006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:19.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.871+0000 7f3993fff640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f398006b020 0x7f398006d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3984000f60 tx=0x7f3984009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:19.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:19.874+0000 7f3990ff9640 1 -- 172.21.15.70:0/551832461 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39940c7080 con 0x7f399415cca0 2023-10-08T07:14:20.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.102+0000 7f3999625640 1 -- 172.21.15.70:0/551832461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39940c7080 con 0x7f399415cca0 2023-10-08T07:14:20.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.103+0000 7f3990ff9640 1 -- 172.21.15.70:0/551832461 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f39940c7080 con 0x7f399415cca0 2023-10-08T07:14:20.105 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:20.105 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":114,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:34.170119+0000","modified":"2023-10-08T07:13:37.222153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11097":{"gid":11097,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/180995576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":180995576},{"type":"v1","addr":"172.21.15.70:6834","nonce":180995576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:48.829441+0000","modified":"2023-10-08T07:13:51.877323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8416":{"gid":8416,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-08T07:14:20.105 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 114 2023-10-08T07:14:20.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.105+0000 7f3999625640 1 -- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f398006b020 msgr2=0x7f398006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.105+0000 7f3999625640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f398006b020 0x7f398006d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3984000f60 tx=0x7f3984009040 comp rx=0 tx=0).stop 2023-10-08T07:14:20.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.105+0000 7f3999625640 1 -- 172.21.15.70:0/551832461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f399415cca0 msgr2=0x7f399415f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.105+0000 7f3999625640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f399415cca0 0x7f399415f2a0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f398c009d90 tx=0x7f398c004a40 comp rx=0 tx=0).stop 2023-10-08T07:14:20.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.106+0000 7f3999625640 1 -- 172.21.15.70:0/551832461 shutdown_connections 2023-10-08T07:14:20.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.106+0000 7f3999625640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f398006b020 0x7f398006d4e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.106+0000 7f3999625640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f399415cca0 0x7f399415f2a0 unknown :-1 s=CLOSED pgs=2517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.106+0000 7f3999625640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39940cab40 0x7f39940c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.106+0000 7f3999625640 1 --2- 172.21.15.70:0/551832461 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39940b07d0 0x7f3994159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.106+0000 7f3999625640 1 -- 172.21.15.70:0/551832461 >> 172.21.15.70:0/551832461 conn(0x7f39940cb0c0 msgr2=0x7f39940ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:20.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.106+0000 7f3999625640 1 -- 172.21.15.70:0/551832461 shutdown_connections 2023-10-08T07:14:20.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.106+0000 7f3999625640 1 -- 172.21.15.70:0/551832461 wait complete. 2023-10-08T07:14:20.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:20.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.339+0000 7fb0780e5640 1 Processor -- start 2023-10-08T07:14:20.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.339+0000 7fb0780e5640 1 -- start start 2023-10-08T07:14:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.341+0000 7fb0780e5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb0700cb580 0x7fb070165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.341+0000 7fb0780e5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07015b2b0 con 0x7fb07015ab40 2023-10-08T07:14:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.341+0000 7fb0780e5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07015b590 con 0x7fb0700c90b0 2023-10-08T07:14:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.341+0000 7fb0780e5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07015bf30 con 0x7fb0700cb580 2023-10-08T07:14:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.341+0000 7fb075e81640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb07015ab40 0x7fb0700ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47732/0 (socket says 172.21.15.70:47732) 2023-10-08T07:14:20.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.341+0000 7fb075e81640 1 -- 172.21.15.70:0/3964229740 learned_addr learned my addr 172.21.15.70:0/3964229740 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:20.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb076682640 1 --2- 172.21.15.70:0/3964229740 >> v2:172.21.15.181:3300/0 conn(0x7fb0700cb580 0x7fb070165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856434192 0 0) 0x7fb07015b2b0 con 0x7fb07015ab40 2023-10-08T07:14:20.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb05c003520 con 0x7fb07015ab40 2023-10-08T07:14:20.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 28110790 0 0) 0x7fb05c003520 con 0x7fb07015ab40 2023-10-08T07:14:20.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 >> v2:172.21.15.181:3300/0 conn(0x7fb0700cb580 msgr2=0x7fb070165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 --2- 172.21.15.70:0/3964229740 >> v2:172.21.15.181:3300/0 conn(0x7fb0700cb580 0x7fb070165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 >> v1:172.21.15.181:6789/0 conn(0x7fb0700c90b0 legacy=0x7fb0700cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07015b870 con 0x7fb07015ab40 2023-10-08T07:14:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fb06c001670 con 0x7fb07015ab40 2023-10-08T07:14:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 >> v1:172.21.15.70:6789/0 conn(0x7fb07015ab40 legacy=0x7fb0700ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 --2- 172.21.15.70:0/3964229740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb05c003970 0x7fb05c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 --2- 172.21.15.70:0/3964229740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05c0047c0 0x7fb05c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 --2- 172.21.15.70:0/3964229740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05c005180 0x7fb05c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07015bf30 con 0x7fb05c0047c0 2023-10-08T07:14:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c00c0d0 con 0x7fb05c005180 2023-10-08T07:14:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c00c290 con 0x7fb05c003970 2023-10-08T07:14:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb076682640 1 --2- 172.21.15.70:0/3964229740 >> v2:172.21.15.181:3300/0 conn(0x7fb0700cb580 0x7fb070165b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.342+0000 7fb075680640 1 --2- 172.21.15.70:0/3964229740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05c0047c0 0x7fb05c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.343+0000 7fb076682640 1 --2- 172.21.15.70:0/3964229740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05c005180 0x7fb05c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.343+0000 7fb075680640 1 -- 172.21.15.70:0/3964229740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb05c003970 msgr2=0x7fb05c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:20.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.343+0000 7fb075680640 1 --2- 172.21.15.70:0/3964229740 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb05c003970 0x7fb05c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.343+0000 7fb075680640 1 -- 172.21.15.70:0/3964229740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05c005180 msgr2=0x7fb05c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.343+0000 7fb075680640 1 --2- 172.21.15.70:0/3964229740 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05c005180 0x7fb05c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.343+0000 7fb075680640 1 -- 172.21.15.70:0/3964229740 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c003520 con 0x7fb05c0047c0 2023-10-08T07:14:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.343+0000 7fb076682640 1 -- 172.21.15.70:0/3964229740 reap_dead start 2023-10-08T07:14:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.344+0000 7fb075680640 1 --2- 172.21.15.70:0/3964229740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05c0047c0 0x7fb05c004c40 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fb060001720 tx=0x7fb05c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6beaa5d644de02b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.344+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0600052d0 con 0x7fb05c0047c0 2023-10-08T07:14:20.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.344+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06002e050 con 0x7fb05c0047c0 2023-10-08T07:14:20.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.344+0000 7fb074e7f640 1 -- 172.21.15.70:0/3964229740 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060003830 con 0x7fb05c0047c0 2023-10-08T07:14:20.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.345+0000 7fb0780e5640 1 -- 172.21.15.70:0/3964229740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05c0047c0 msgr2=0x7fb05c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.345+0000 7fb0780e5640 1 --2- 172.21.15.70:0/3964229740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05c0047c0 0x7fb05c004c40 secure :-1 s=READY pgs=2519 cs=0 l=1 rev1=1 crypto rx=0x7fb060001720 tx=0x7fb05c0057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.345+0000 7fb0780e5640 1 -- 172.21.15.70:0/3964229740 shutdown_connections 2023-10-08T07:14:20.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.345+0000 7fb0780e5640 1 --2- 172.21.15.70:0/3964229740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb05c0047c0 0x7fb05c004c40 unknown :-1 s=CLOSED pgs=2519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.345+0000 7fb0780e5640 1 -- 172.21.15.70:0/3964229740 >> 172.21.15.70:0/3964229740 conn(0x7fb07006a380 msgr2=0x7fb0700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:20.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.345+0000 7fb0780e5640 1 -- 172.21.15.70:0/3964229740 shutdown_connections 2023-10-08T07:14:20.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.345+0000 7fb0780e5640 1 -- 172.21.15.70:0/3964229740 wait complete. 2023-10-08T07:14:20.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.347+0000 7fb0780e5640 1 Processor -- start 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.347+0000 7fb0780e5640 1 -- start start 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb0780e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05c0047c0 0x7fb070142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb0780e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb070158130 0x7fb070158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb0780e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb070156bb0 0x7fb070157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb0780e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07015f7a0 con 0x7fb070158130 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb0780e5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb070157570 con 0x7fb05c0047c0 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb0780e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb070158ad0 con 0x7fb070156bb0 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb075680640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb070158130 0x7fb070158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb075680640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb070158130 0x7fb070158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39936/0 (socket says 172.21.15.70:39936) 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb075680640 1 -- 172.21.15.70:0/3561806810 learned_addr learned my addr 172.21.15.70:0/3561806810 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb075e81640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05c0047c0 0x7fb070142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb076682640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb070156bb0 0x7fb070157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.348+0000 7fb075680640 1 -- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb070156bb0 msgr2=0x7fb070157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.349+0000 7fb075680640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb070156bb0 0x7fb070157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.349+0000 7fb075680640 1 -- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05c0047c0 msgr2=0x7fb070142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.349+0000 7fb075680640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05c0047c0 0x7fb070142900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.349+0000 7fb075680640 1 -- 172.21.15.70:0/3561806810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0701515d0 con 0x7fb070158130 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.349+0000 7fb075680640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb070158130 0x7fb070158590 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fb060005040 tx=0x7fb060001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.349+0000 7fb0667fc640 1 -- 172.21.15.70:0/3561806810 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060007640 con 0x7fb070158130 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.349+0000 7fb0667fc640 1 -- 172.21.15.70:0/3561806810 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0600069e0 con 0x7fb070158130 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.349+0000 7fb0667fc640 1 -- 172.21.15.70:0/3561806810 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060006d70 con 0x7fb070158130 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.349+0000 7fb0780e5640 1 -- 172.21.15.70:0/3561806810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb070145340 con 0x7fb070158130 2023-10-08T07:14:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.349+0000 7fb0780e5640 1 -- 172.21.15.70:0/3561806810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb070145900 con 0x7fb070158130 2023-10-08T07:14:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.351+0000 7fb0780e5640 1 -- 172.21.15.70:0/3561806810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb070146230 con 0x7fb070158130 2023-10-08T07:14:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.351+0000 7fb0667fc640 1 -- 172.21.15.70:0/3561806810 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 137) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fb0600035f0 con 0x7fb070158130 2023-10-08T07:14:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.352+0000 7fb0667fc640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb05406b020 0x7fb05406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.352+0000 7fb0667fc640 1 -- 172.21.15.70:0/3561806810 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7fb0600ac430 con 0x7fb070158130 2023-10-08T07:14:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.353+0000 7fb075e81640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb05406b020 0x7fb05406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.354+0000 7fb075e81640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb05406b020 0x7fb05406d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb07004fa60 tx=0x7fb06c009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.356+0000 7fb0667fc640 1 -- 172.21.15.70:0/3561806810 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb070146230 con 0x7fb070158130 2023-10-08T07:14:20.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.547+0000 7fb0780e5640 1 -- 172.21.15.70:0/3561806810 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb070146230 con 0x7fb070158130 2023-10-08T07:14:20.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.551+0000 7fb0667fc640 1 -- 172.21.15.70:0/3561806810 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v240) v1 ==== 74+0+21816 (secure 0 0 0) 0x7fb070146230 con 0x7fb070158130 2023-10-08T07:14:20.553 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:20.554 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":240,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:16.815407+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:13:47.007437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:13:47.838023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2480387829":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/920349426":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/2459293689":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/4129076201":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6832/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/365280967":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/2833418931":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/753176203":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/317173509":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:6832/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1018014196":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1778827440":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1217032325":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/2798662747":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/4139611391":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/3002223432":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/3074626816":"2023-10-09T07:14:06.545661+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 -- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb05406b020 msgr2=0x7fb05406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb05406b020 0x7fb05406d4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb07004fa60 tx=0x7fb06c009040 comp rx=0 tx=0).stop 2023-10-08T07:14:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 -- 172.21.15.70:0/3561806810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb070158130 msgr2=0x7fb070158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb070158130 0x7fb070158590 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7fb060005040 tx=0x7fb060001120 comp rx=0 tx=0).stop 2023-10-08T07:14:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 -- 172.21.15.70:0/3561806810 shutdown_connections 2023-10-08T07:14:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb05406b020 0x7fb05406d4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb070156bb0 0x7fb070157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb070158130 0x7fb070158590 unknown :-1 s=CLOSED pgs=2520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 --2- 172.21.15.70:0/3561806810 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb05c0047c0 0x7fb070142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 -- 172.21.15.70:0/3561806810 >> 172.21.15.70:0/3561806810 conn(0x7fb07006a380 msgr2=0x7fb0700ada90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:20.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 -- 172.21.15.70:0/3561806810 shutdown_connections 2023-10-08T07:14:20.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.554+0000 7fb0780e5640 1 -- 172.21.15.70:0/3561806810 wait complete. 2023-10-08T07:14:20.574 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T07:14:20.575 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:14:20.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.779+0000 7f148881a640 1 Processor -- start 2023-10-08T07:14:20.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.780+0000 7f148881a640 1 -- start start 2023-10-08T07:14:20.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.781+0000 7f148881a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1484160940 0x7f14840709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.781+0000 7f148881a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f148406c890 0x7f148406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.781+0000 7f148881a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14840c48f0 con 0x7f148415cd90 2023-10-08T07:14:20.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.781+0000 7f148881a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14840c4d80 con 0x7f148406c890 2023-10-08T07:14:20.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.781+0000 7f148881a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14840c3ce0 con 0x7f1484160940 2023-10-08T07:14:20.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.782+0000 7f1482d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f148406c890 0x7f148406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.782+0000 7f1482d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f148406c890 0x7f148406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39944/0 (socket says 172.21.15.70:39944) 2023-10-08T07:14:20.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.782+0000 7f1482d9d640 1 -- 172.21.15.70:0/3306071673 learned_addr learned my addr 172.21.15.70:0/3306071673 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:20.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.782+0000 7f1481d9b640 1 --2- 172.21.15.70:0/3306071673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1484160940 0x7f14840709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.782+0000 7f1482d9d640 1 -- 172.21.15.70:0/3306071673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1484160940 msgr2=0x7f14840709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:20.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.782+0000 7f1482d9d640 1 --2- 172.21.15.70:0/3306071673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1484160940 0x7f14840709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.782+0000 7f1482d9d640 1 -- 172.21.15.70:0/3306071673 >> v1:172.21.15.181:6789/0 conn(0x7f148415cd90 legacy=0x7f14840702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:20.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.782+0000 7f1482d9d640 1 -- 172.21.15.70:0/3306071673 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14840c3fc0 con 0x7f148406c890 2023-10-08T07:14:20.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.783+0000 7f1482d9d640 1 --2- 172.21.15.70:0/3306071673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 0x7f148406cd20 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f147c009870 tx=0x7f147c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7465699f8e10a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:20.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.784+0000 7f148159a640 1 -- 172.21.15.70:0/3306071673 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c031640 con 0x7f148406c890 2023-10-08T07:14:20.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.784+0000 7f148159a640 1 -- 172.21.15.70:0/3306071673 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c0317a0 con 0x7f148406c890 2023-10-08T07:14:20.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.784+0000 7f148159a640 1 -- 172.21.15.70:0/3306071673 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c031b30 con 0x7f148406c890 2023-10-08T07:14:20.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.784+0000 7f148881a640 1 -- 172.21.15.70:0/3306071673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 msgr2=0x7f148406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.784+0000 7f148881a640 1 --2- 172.21.15.70:0/3306071673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 0x7f148406cd20 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f147c009870 tx=0x7f147c02b8f0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.785+0000 7f148881a640 1 -- 172.21.15.70:0/3306071673 shutdown_connections 2023-10-08T07:14:20.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.785+0000 7f148881a640 1 --2- 172.21.15.70:0/3306071673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 0x7f148406cd20 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.785+0000 7f148881a640 1 --2- 172.21.15.70:0/3306071673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1484160940 0x7f14840709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.785+0000 7f148881a640 1 -- 172.21.15.70:0/3306071673 >> 172.21.15.70:0/3306071673 conn(0x7f148406a380 msgr2=0x7f14840b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:20.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.785+0000 7f148881a640 1 -- 172.21.15.70:0/3306071673 shutdown_connections 2023-10-08T07:14:20.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.785+0000 7f148881a640 1 -- 172.21.15.70:0/3306071673 wait complete. 2023-10-08T07:14:20.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.786+0000 7f148881a640 1 Processor -- start 2023-10-08T07:14:20.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.787+0000 7f148881a640 1 -- start start 2023-10-08T07:14:20.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.787+0000 7f148881a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 0x7f148414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.787+0000 7f148881a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f148415cd90 0x7f14841f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.787+0000 7f148881a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1484160940 0x7f14841e74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.787+0000 7f148881a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14841f64a0 con 0x7f148406c890 2023-10-08T07:14:20.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.787+0000 7f148881a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148414a930 con 0x7f1484160940 2023-10-08T07:14:20.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.787+0000 7f148881a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14841f6100 con 0x7f148415cd90 2023-10-08T07:14:20.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.788+0000 7f148259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 0x7f148414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.788+0000 7f148259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 0x7f148414a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39956/0 (socket says 172.21.15.70:39956) 2023-10-08T07:14:20.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.788+0000 7f148259c640 1 -- 172.21.15.70:0/211662485 learned_addr learned my addr 172.21.15.70:0/211662485 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:20.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.788+0000 7f148259c640 1 -- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f148415cd90 msgr2=0x7f14841f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.788+0000 7f148259c640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f148415cd90 0x7f14841f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.788+0000 7f148259c640 1 -- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1484160940 msgr2=0x7f14841e74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:20.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.788+0000 7f148259c640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1484160940 0x7f14841e74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:20.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.788+0000 7f148259c640 1 -- 172.21.15.70:0/211662485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147c0094d0 con 0x7f148406c890 2023-10-08T07:14:20.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.789+0000 7f148259c640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 0x7f148414a3f0 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f147400c7c0 tx=0x7f147400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:20.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.789+0000 7f1472ffd640 1 -- 172.21.15.70:0/211662485 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147400aa50 con 0x7f148406c890 2023-10-08T07:14:20.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.789+0000 7f1472ffd640 1 -- 172.21.15.70:0/211662485 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147400abb0 con 0x7f148406c890 2023-10-08T07:14:20.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.789+0000 7f1472ffd640 1 -- 172.21.15.70:0/211662485 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474012420 con 0x7f148406c890 2023-10-08T07:14:20.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.789+0000 7f148881a640 1 -- 172.21.15.70:0/211662485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14840c5980 con 0x7f148406c890 2023-10-08T07:14:20.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.789+0000 7f148881a640 1 -- 172.21.15.70:0/211662485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14840c5f40 con 0x7f148406c890 2023-10-08T07:14:20.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.790+0000 7f148881a640 1 -- 172.21.15.70:0/211662485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14841f65e0 con 0x7f148406c890 2023-10-08T07:14:20.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.791+0000 7f1472ffd640 1 -- 172.21.15.70:0/211662485 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 137) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f1474012580 con 0x7f148406c890 2023-10-08T07:14:20.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.792+0000 7f1472ffd640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f146406b020 0x7f146406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:20.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.793+0000 7f1472ffd640 1 -- 172.21.15.70:0/211662485 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f147408a3f0 con 0x7f148406c890 2023-10-08T07:14:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.793+0000 7f1481d9b640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f146406b020 0x7f146406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:20.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.795+0000 7f1481d9b640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f146406b020 0x7f146406d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1478006fd0 tx=0x7f1478008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:20.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.797+0000 7f1472ffd640 1 -- 172.21.15.70:0/211662485 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14841f65e0 con 0x7f148406c890 2023-10-08T07:14:20.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:20.874+0000 7f1472ffd640 1 -- 172.21.15.70:0/211662485 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1474054d90 con 0x7f148406c890 2023-10-08T07:14:21.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.026+0000 7f148881a640 1 -- 172.21.15.70:0/211662485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14841f65e0 con 0x7f148406c890 2023-10-08T07:14:21.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.027+0000 7f1472ffd640 1 -- 172.21.15.70:0/211662485 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f14841f65e0 con 0x7f148406c890 2023-10-08T07:14:21.029 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:21.029 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":114,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:34.170119+0000","modified":"2023-10-08T07:13:37.222153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11097":{"gid":11097,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/180995576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":180995576},{"type":"v1","addr":"172.21.15.70:6834","nonce":180995576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:48.829441+0000","modified":"2023-10-08T07:13:51.877323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8416":{"gid":8416,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-08T07:14:21.029 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 114 2023-10-08T07:14:21.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 -- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f146406b020 msgr2=0x7f146406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f146406b020 0x7f146406d4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1478006fd0 tx=0x7f1478008040 comp rx=0 tx=0).stop 2023-10-08T07:14:21.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 -- 172.21.15.70:0/211662485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 msgr2=0x7f148414a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 0x7f148414a3f0 secure :-1 s=READY pgs=2522 cs=0 l=1 rev1=1 crypto rx=0x7f147400c7c0 tx=0x7f147400cc80 comp rx=0 tx=0).stop 2023-10-08T07:14:21.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 -- 172.21.15.70:0/211662485 shutdown_connections 2023-10-08T07:14:21.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f146406b020 0x7f146406d4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1484160940 0x7f14841e74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f148415cd90 0x7f14841f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 --2- 172.21.15.70:0/211662485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f148406c890 0x7f148414a3f0 unknown :-1 s=CLOSED pgs=2522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 -- 172.21.15.70:0/211662485 >> 172.21.15.70:0/211662485 conn(0x7f148406a380 msgr2=0x7f148414e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:21.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 -- 172.21.15.70:0/211662485 shutdown_connections 2023-10-08T07:14:21.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.029+0000 7f148881a640 1 -- 172.21.15.70:0/211662485 wait complete. 2023-10-08T07:14:21.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:14:21.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.262+0000 7f5e0c4f2640 1 Processor -- start 2023-10-08T07:14:21.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.262+0000 7f5e0c4f2640 1 -- start start 2023-10-08T07:14:21.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.264+0000 7f5e0c4f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5e0415cd90 0x7f5e040702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.264+0000 7f5e0c4f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5e0406c890 0x7f5e0406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.264+0000 7f5e0c4f2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e040c48f0 con 0x7f5e04160940 2023-10-08T07:14:21.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.264+0000 7f5e0c4f2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e040c4d80 con 0x7f5e0415cd90 2023-10-08T07:14:21.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.264+0000 7f5e0c4f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e040c3ce0 con 0x7f5e0406c890 2023-10-08T07:14:21.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.264+0000 7f5e0a28e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5e0415cd90 0x7f5e040702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:21.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.264+0000 7f5e0a28e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5e0415cd90 0x7f5e040702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39966/0 (socket says 172.21.15.70:39966) 2023-10-08T07:14:21.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.264+0000 7f5e0a28e640 1 -- 172.21.15.70:0/602112437 learned_addr learned my addr 172.21.15.70:0/602112437 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:21.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.265+0000 7f5e0a28e640 1 -- 172.21.15.70:0/602112437 >> v2:172.21.15.181:3300/0 conn(0x7f5e0406c890 msgr2=0x7f5e0406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:21.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.265+0000 7f5e0a28e640 1 --2- 172.21.15.70:0/602112437 >> v2:172.21.15.181:3300/0 conn(0x7f5e0406c890 0x7f5e0406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.265+0000 7f5e0a28e640 1 -- 172.21.15.70:0/602112437 >> v1:172.21.15.181:6789/0 conn(0x7f5e04160940 legacy=0x7f5e04070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:21.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.265+0000 7f5e0a28e640 1 -- 172.21.15.70:0/602112437 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e040c3fc0 con 0x7f5e0415cd90 2023-10-08T07:14:21.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.266+0000 7f5e0a28e640 1 --2- 172.21.15.70:0/602112437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0415cd90 0x7f5e040702d0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f5e040c8d60 tx=0x7f5e0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9611ceb4fdeec4cc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:21.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.266+0000 7f5e0928c640 1 -- 172.21.15.70:0/602112437 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00031870 con 0x7f5e0415cd90 2023-10-08T07:14:21.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.266+0000 7f5e0928c640 1 -- 172.21.15.70:0/602112437 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e000319d0 con 0x7f5e0415cd90 2023-10-08T07:14:21.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.266+0000 7f5e0928c640 1 -- 172.21.15.70:0/602112437 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00031d60 con 0x7f5e0415cd90 2023-10-08T07:14:21.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.267+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/602112437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0415cd90 msgr2=0x7f5e040702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.267+0000 7f5e0c4f2640 1 --2- 172.21.15.70:0/602112437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0415cd90 0x7f5e040702d0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7f5e040c8d60 tx=0x7f5e0002b910 comp rx=0 tx=0).stop 2023-10-08T07:14:21.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.267+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/602112437 shutdown_connections 2023-10-08T07:14:21.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.267+0000 7f5e0c4f2640 1 --2- 172.21.15.70:0/602112437 >> v2:172.21.15.181:3300/0 conn(0x7f5e0406c890 0x7f5e0406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.267+0000 7f5e0c4f2640 1 --2- 172.21.15.70:0/602112437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0415cd90 0x7f5e040702d0 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.267+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/602112437 >> 172.21.15.70:0/602112437 conn(0x7f5e0406a380 msgr2=0x7f5e040b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:21.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.267+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/602112437 shutdown_connections 2023-10-08T07:14:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.267+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/602112437 wait complete. 2023-10-08T07:14:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.269+0000 7f5e0c4f2640 1 Processor -- start 2023-10-08T07:14:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.269+0000 7f5e0c4f2640 1 -- start start 2023-10-08T07:14:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.270+0000 7f5e0c4f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0406c890 0x7f5e0415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.270+0000 7f5e0c4f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e0415cd90 0x7f5e041507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.270+0000 7f5e0c4f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e04160940 0x7f5e040ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.270+0000 7f5e0c4f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040b4100 con 0x7f5e0406c890 2023-10-08T07:14:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.270+0000 7f5e0c4f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e04160030 con 0x7f5e04160940 2023-10-08T07:14:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.270+0000 7f5e0c4f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e04160220 con 0x7f5e0415cd90 2023-10-08T07:14:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.270+0000 7f5e0a28e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0406c890 0x7f5e0415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:21.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.270+0000 7f5e0a28e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0406c890 0x7f5e0415faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39976/0 (socket says 172.21.15.70:39976) 2023-10-08T07:14:21.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.270+0000 7f5e0a28e640 1 -- 172.21.15.70:0/2960407432 learned_addr learned my addr 172.21.15.70:0/2960407432 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:21.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.270+0000 7f5e0a28e640 1 -- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e0415cd90 msgr2=0x7f5e041507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.271+0000 7f5e0a28e640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e0415cd90 0x7f5e041507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.271+0000 7f5e0a28e640 1 -- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e04160940 msgr2=0x7f5e040ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.271+0000 7f5e0a28e640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e04160940 0x7f5e040ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.271+0000 7f5e0a28e640 1 -- 172.21.15.70:0/2960407432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e000094d0 con 0x7f5e0406c890 2023-10-08T07:14:21.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.271+0000 7f5e0a28e640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0406c890 0x7f5e0415faf0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f5e0002b950 tx=0x7f5e000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:21.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.271+0000 7f5dfaffd640 1 -- 172.21.15.70:0/2960407432 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0003ad10 con 0x7f5e0406c890 2023-10-08T07:14:21.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.271+0000 7f5dfaffd640 1 -- 172.21.15.70:0/2960407432 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0003ae70 con 0x7f5e0406c890 2023-10-08T07:14:21.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.271+0000 7f5dfaffd640 1 -- 172.21.15.70:0/2960407432 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00044610 con 0x7f5e0406c890 2023-10-08T07:14:21.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.271+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/2960407432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e04150ce0 con 0x7f5e0406c890 2023-10-08T07:14:21.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.271+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/2960407432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e040b3760 con 0x7f5e0406c890 2023-10-08T07:14:21.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.273+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/2960407432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e040b4420 con 0x7f5e0406c890 2023-10-08T07:14:21.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.273+0000 7f5dfaffd640 1 -- 172.21.15.70:0/2960407432 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5e00044770 con 0x7f5e0406c890 2023-10-08T07:14:21.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.274+0000 7f5dfaffd640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5de806b0b0 0x7f5de806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.274+0000 7f5dfaffd640 1 -- 172.21.15.70:0/2960407432 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f5e0002d040 con 0x7f5e0406c890 2023-10-08T07:14:21.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.275+0000 7f5e09a8d640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5de806b0b0 0x7f5de806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:21.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.276+0000 7f5e09a8d640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5de806b0b0 0x7f5de806d570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5e0415c3e0 tx=0x7f5df4006ba0 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:21.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.279+0000 7f5dfaffd640 1 -- 172.21.15.70:0/2960407432 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e00077040 con 0x7f5e0406c890 2023-10-08T07:14:21.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.507+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/2960407432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e04067490 con 0x7f5e0406c890 2023-10-08T07:14:21.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.509+0000 7f5dfaffd640 1 -- 172.21.15.70:0/2960407432 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+6152 (secure 0 0 0) 0x7f5e0007a660 con 0x7f5e0406c890 2023-10-08T07:14:21.511 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:21.511 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":114,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:34.170119+0000","modified":"2023-10-08T07:13:37.222153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11097},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11097":{"gid":11097,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/180995576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":180995576},{"type":"v1","addr":"172.21.15.70:6834","nonce":180995576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13},{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:48.829441+0000","modified":"2023-10-08T07:13:51.877323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8416":{"gid":8416,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-08T07:14:21.511 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 114 2023-10-08T07:14:21.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.511+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5de806b0b0 msgr2=0x7f5de806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.511+0000 7f5e0c4f2640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5de806b0b0 0x7f5de806d570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5e0415c3e0 tx=0x7f5df4006ba0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.511+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/2960407432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0406c890 msgr2=0x7f5e0415faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.511+0000 7f5e0c4f2640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0406c890 0x7f5e0415faf0 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7f5e0002b950 tx=0x7f5e000046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.511+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/2960407432 shutdown_connections 2023-10-08T07:14:21.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.511+0000 7f5e0c4f2640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5de806b0b0 0x7f5de806d570 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.511+0000 7f5e0c4f2640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5e04160940 0x7f5e040ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.511+0000 7f5e0c4f2640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5e0415cd90 0x7f5e041507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.511+0000 7f5e0c4f2640 1 --2- 172.21.15.70:0/2960407432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e0406c890 0x7f5e0415faf0 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.511+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/2960407432 >> 172.21.15.70:0/2960407432 conn(0x7f5e0406a380 msgr2=0x7f5e040aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:21.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.512+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/2960407432 shutdown_connections 2023-10-08T07:14:21.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.512+0000 7f5e0c4f2640 1 -- 172.21.15.70:0/2960407432 wait complete. 2023-10-08T07:14:21.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:21.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.735+0000 7f0213283640 1 Processor -- start 2023-10-08T07:14:21.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.736+0000 7f0213283640 1 -- start start 2023-10-08T07:14:21.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.737+0000 7f0213283640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f020c15ab30 0x7f020c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.737+0000 7f0213283640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0adc70 0x7f020c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.737+0000 7f0213283640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c15f7d0 con 0x7f020c15af10 2023-10-08T07:14:21.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.737+0000 7f0213283640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c15f210 con 0x7f020c15ab30 2023-10-08T07:14:21.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.737+0000 7f0213283640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c15f4f0 con 0x7f020c0adc70 2023-10-08T07:14:21.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.738+0000 7f0212281640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f020c15ab30 0x7f020c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:21.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.738+0000 7f0212a82640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f020c15af10 0x7f020c0701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47742/0 (socket says 172.21.15.70:47742) 2023-10-08T07:14:21.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.738+0000 7f0212a82640 1 -- 172.21.15.70:0/2649519275 learned_addr learned my addr 172.21.15.70:0/2649519275 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:21.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.738+0000 7f0212281640 1 -- 172.21.15.70:0/2649519275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0adc70 msgr2=0x7f020c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:21.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.738+0000 7f0212281640 1 --2- 172.21.15.70:0/2649519275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0adc70 0x7f020c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.738+0000 7f0212281640 1 -- 172.21.15.70:0/2649519275 >> v1:172.21.15.70:6789/0 conn(0x7f020c15af10 legacy=0x7f020c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.738+0000 7f0212281640 1 -- 172.21.15.70:0/2649519275 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c06c550 con 0x7f020c15ab30 2023-10-08T07:14:21.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.738+0000 7f021127f640 1 -- 172.21.15.70:0/2649519275 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360438407 0 0) 0x7f020c15f7d0 con 0x7f020c15af10 2023-10-08T07:14:21.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.740+0000 7f0212281640 1 --2- 172.21.15.70:0/2649519275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c15ab30 0x7f020c15e8a0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f02080097a0 tx=0x7f020802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f7fd87ce6c345f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:21.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.740+0000 7f021127f640 1 -- 172.21.15.70:0/2649519275 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208031830 con 0x7f020c15ab30 2023-10-08T07:14:21.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.740+0000 7f021127f640 1 -- 172.21.15.70:0/2649519275 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0208031990 con 0x7f020c15ab30 2023-10-08T07:14:21.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.740+0000 7f021127f640 1 -- 172.21.15.70:0/2649519275 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208031d20 con 0x7f020c15ab30 2023-10-08T07:14:21.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.740+0000 7f0213283640 1 -- 172.21.15.70:0/2649519275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c15ab30 msgr2=0x7f020c15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.740+0000 7f0213283640 1 --2- 172.21.15.70:0/2649519275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c15ab30 0x7f020c15e8a0 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f02080097a0 tx=0x7f020802b8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.741+0000 7f0213283640 1 -- 172.21.15.70:0/2649519275 shutdown_connections 2023-10-08T07:14:21.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.741+0000 7f0213283640 1 --2- 172.21.15.70:0/2649519275 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0adc70 0x7f020c0b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.741+0000 7f0213283640 1 --2- 172.21.15.70:0/2649519275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c15ab30 0x7f020c15e8a0 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.741+0000 7f0213283640 1 -- 172.21.15.70:0/2649519275 >> 172.21.15.70:0/2649519275 conn(0x7f020c0cb0c0 msgr2=0x7f020c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:21.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.741+0000 7f0213283640 1 -- 172.21.15.70:0/2649519275 shutdown_connections 2023-10-08T07:14:21.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.741+0000 7f0213283640 1 -- 172.21.15.70:0/2649519275 wait complete. 2023-10-08T07:14:21.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.742+0000 7f0213283640 1 Processor -- start 2023-10-08T07:14:21.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.742+0000 7f0213283640 1 -- start start 2023-10-08T07:14:21.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.743+0000 7f0213283640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c0adc70 0x7f020c0c2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.743+0000 7f0213283640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c15ab30 0x7f020c14ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.743+0000 7f0213283640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c15af10 0x7f020c15c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.743+0000 7f0213283640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0c6e00 con 0x7f020c0adc70 2023-10-08T07:14:21.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.743+0000 7f0213283640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0c2670 con 0x7f020c15ab30 2023-10-08T07:14:21.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.743+0000 7f0213283640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c15cb30 con 0x7f020c15af10 2023-10-08T07:14:21.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0212281640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c0adc70 0x7f020c0c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:21.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0212281640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c0adc70 0x7f020c0c2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39998/0 (socket says 172.21.15.70:39998) 2023-10-08T07:14:21.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0212281640 1 -- 172.21.15.70:0/341640153 learned_addr learned my addr 172.21.15.70:0/341640153 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:21.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0211a80640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c15ab30 0x7f020c14ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:21.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0212281640 1 -- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c15af10 msgr2=0x7f020c15c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0212281640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c15af10 0x7f020c15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0212281640 1 -- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c15ab30 msgr2=0x7f020c14ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0212281640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c15ab30 0x7f020c14ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0212281640 1 -- 172.21.15.70:0/341640153 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0208009400 con 0x7f020c0adc70 2023-10-08T07:14:21.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0211a80640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c15ab30 0x7f020c14ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:21.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.744+0000 7f0212281640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c0adc70 0x7f020c0c2130 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f02080098c0 tx=0x7f02080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:21.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.745+0000 7f0202ffd640 1 -- 172.21.15.70:0/341640153 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020803ba70 con 0x7f020c0adc70 2023-10-08T07:14:21.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.745+0000 7f0202ffd640 1 -- 172.21.15.70:0/341640153 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020803bbd0 con 0x7f020c0adc70 2023-10-08T07:14:21.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.745+0000 7f0213283640 1 -- 172.21.15.70:0/341640153 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c0c70a0 con 0x7f020c0adc70 2023-10-08T07:14:21.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.745+0000 7f0202ffd640 1 -- 172.21.15.70:0/341640153 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02080453f0 con 0x7f020c0adc70 2023-10-08T07:14:21.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.745+0000 7f0213283640 1 -- 172.21.15.70:0/341640153 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c15ce10 con 0x7f020c0adc70 2023-10-08T07:14:21.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.746+0000 7f0213283640 1 -- 172.21.15.70:0/341640153 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c0c75c0 con 0x7f020c0adc70 2023-10-08T07:14:21.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.747+0000 7f0202ffd640 1 -- 172.21.15.70:0/341640153 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f020804e450 con 0x7f020c0adc70 2023-10-08T07:14:21.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.748+0000 7f0202ffd640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f01f006b0b0 0x7f01f006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:21.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.748+0000 7f0202ffd640 1 -- 172.21.15.70:0/341640153 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f02080ac700 con 0x7f020c0adc70 2023-10-08T07:14:21.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.748+0000 7f0211a80640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f01f006b0b0 0x7f01f006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:21.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.750+0000 7f0211a80640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f01f006b0b0 0x7f01f006d570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f01fc006fd0 tx=0x7f01fc008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:21.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.752+0000 7f0202ffd640 1 -- 172.21.15.70:0/341640153 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f020c0c75c0 con 0x7f020c0adc70 2023-10-08T07:14:21.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.930+0000 7f0213283640 1 -- 172.21.15.70:0/341640153 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f020c0c75c0 con 0x7f020c0adc70 2023-10-08T07:14:21.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.931+0000 7f0202ffd640 1 -- 172.21.15.70:0/341640153 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v240) v1 ==== 74+0+21816 (secure 0 0 0) 0x7f020c0c75c0 con 0x7f020c0adc70 2023-10-08T07:14:21.933 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:21.934 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":240,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:16.815407+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:13:32.637023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2023-10-08T07:13:33.187098+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:13:47.007437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:13:47.838023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2480387829":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/920349426":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/2459293689":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/4129076201":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6832/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/365280967":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/2833418931":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/753176203":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/317173509":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:6832/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1018014196":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1778827440":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1217032325":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/2798662747":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/4139611391":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/3002223432":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/3074626816":"2023-10-09T07:14:06.545661+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:21.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 -- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f01f006b0b0 msgr2=0x7f01f006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f01f006b0b0 0x7f01f006d570 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f01fc006fd0 tx=0x7f01fc008040 comp rx=0 tx=0).stop 2023-10-08T07:14:21.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 -- 172.21.15.70:0/341640153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c0adc70 msgr2=0x7f020c0c2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:21.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c0adc70 0x7f020c0c2130 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f02080098c0 tx=0x7f02080046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 -- 172.21.15.70:0/341640153 shutdown_connections 2023-10-08T07:14:21.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f01f006b0b0 0x7f01f006d570 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c15af10 0x7f020c15c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c15ab30 0x7f020c14ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 --2- 172.21.15.70:0/341640153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c0adc70 0x7f020c0c2130 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:21.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 -- 172.21.15.70:0/341640153 >> 172.21.15.70:0/341640153 conn(0x7f020c0cb0c0 msgr2=0x7f020c158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:21.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 -- 172.21.15.70:0/341640153 shutdown_connections 2023-10-08T07:14:21.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:21.934+0000 7f0213283640 1 -- 172.21.15.70:0/341640153 wait complete. 2023-10-08T07:14:21.953 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T07:14:22.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.160+0000 7f0da58e5640 1 Processor -- start 2023-10-08T07:14:22.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.161+0000 7f0da58e5640 1 -- start start 2023-10-08T07:14:22.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.162+0000 7f0da58e5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f0da00afbe0 0x7f0da00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.162+0000 7f0da58e5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0da006dfc0 0x7f0da0070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.162+0000 7f0da58e5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da00b2620 con 0x7f0da0071070 2023-10-08T07:14:22.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.162+0000 7f0da58e5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da00b2060 con 0x7f0da006dfc0 2023-10-08T07:14:22.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.162+0000 7f0da58e5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da00b2340 con 0x7f0da00afbe0 2023-10-08T07:14:22.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.162+0000 7f0d9f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0da0071070 0x7f0da00cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47744/0 (socket says 172.21.15.70:47744) 2023-10-08T07:14:22.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.162+0000 7f0d9f7fe640 1 -- 172.21.15.70:0/2018718817 learned_addr learned my addr 172.21.15.70:0/2018718817 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:22.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.163+0000 7f0d9e7fc640 1 --2- 172.21.15.70:0/2018718817 >> v2:172.21.15.70:3300/0 conn(0x7f0da006dfc0 0x7f0da0070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.163+0000 7f0d9effd640 1 --2- 172.21.15.70:0/2018718817 >> v2:172.21.15.181:3300/0 conn(0x7f0da00afbe0 0x7f0da00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.163+0000 7f0d9dffb640 1 -- 172.21.15.70:0/2018718817 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038963097 0 0) 0x7f0da00b2620 con 0x7f0da0071070 2023-10-08T07:14:22.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.163+0000 7f0d9dffb640 1 -- 172.21.15.70:0/2018718817 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d8c003520 con 0x7f0da0071070 2023-10-08T07:14:22.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.163+0000 7f0d9e7fc640 1 -- 172.21.15.70:0/2018718817 >> v2:172.21.15.181:3300/0 conn(0x7f0da00afbe0 msgr2=0x7f0da00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:22.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.163+0000 7f0d9e7fc640 1 --2- 172.21.15.70:0/2018718817 >> v2:172.21.15.181:3300/0 conn(0x7f0da00afbe0 0x7f0da00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.163+0000 7f0d9e7fc640 1 -- 172.21.15.70:0/2018718817 >> v1:172.21.15.70:6789/0 conn(0x7f0da0071070 legacy=0x7f0da00cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.163+0000 7f0d9e7fc640 1 -- 172.21.15.70:0/2018718817 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0070350 con 0x7f0da006dfc0 2023-10-08T07:14:22.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.164+0000 7f0d9e7fc640 1 --2- 172.21.15.70:0/2018718817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 0x7f0da0070b30 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f0da006e640 tx=0x7f0d8802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69af6197cba470f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:22.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.164+0000 7f0d9dffb640 1 -- 172.21.15.70:0/2018718817 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88031830 con 0x7f0da006dfc0 2023-10-08T07:14:22.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.165+0000 7f0d9dffb640 1 -- 172.21.15.70:0/2018718817 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d88031990 con 0x7f0da006dfc0 2023-10-08T07:14:22.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.165+0000 7f0d9dffb640 1 -- 172.21.15.70:0/2018718817 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88031d20 con 0x7f0da006dfc0 2023-10-08T07:14:22.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.165+0000 7f0da58e5640 1 -- 172.21.15.70:0/2018718817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 msgr2=0x7f0da0070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.165+0000 7f0da58e5640 1 --2- 172.21.15.70:0/2018718817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 0x7f0da0070b30 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f0da006e640 tx=0x7f0d8802b8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.165+0000 7f0da58e5640 1 -- 172.21.15.70:0/2018718817 shutdown_connections 2023-10-08T07:14:22.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.165+0000 7f0da58e5640 1 --2- 172.21.15.70:0/2018718817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 0x7f0da0070b30 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.165+0000 7f0da58e5640 1 --2- 172.21.15.70:0/2018718817 >> v2:172.21.15.181:3300/0 conn(0x7f0da00afbe0 0x7f0da00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.165+0000 7f0da58e5640 1 -- 172.21.15.70:0/2018718817 >> 172.21.15.70:0/2018718817 conn(0x7f0da006a5d0 msgr2=0x7f0da00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:22.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.165+0000 7f0da58e5640 1 -- 172.21.15.70:0/2018718817 shutdown_connections 2023-10-08T07:14:22.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.165+0000 7f0da58e5640 1 -- 172.21.15.70:0/2018718817 wait complete. 2023-10-08T07:14:22.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.167+0000 7f0da58e5640 1 Processor -- start 2023-10-08T07:14:22.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.167+0000 7f0da58e5640 1 -- start start 2023-10-08T07:14:22.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0da58e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 0x7f0da015db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0da58e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0da0071070 0x7f0da00c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0da58e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0da00afbe0 0x7f0da00ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0da58e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da00c3340 con 0x7f0da006dfc0 2023-10-08T07:14:22.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0da58e5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da015cab0 con 0x7f0da0071070 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0da58e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da015cd90 con 0x7f0da00afbe0 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0d9effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 0x7f0da015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0d9effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 0x7f0da015db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40006/0 (socket says 172.21.15.70:40006) 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0d9effd640 1 -- 172.21.15.70:0/3836073820 learned_addr learned my addr 172.21.15.70:0/3836073820 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0d9e7fc640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0da0071070 0x7f0da00c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0d9effd640 1 -- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0da00afbe0 msgr2=0x7f0da00ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.168+0000 7f0d9f7fe640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0da00afbe0 0x7f0da00ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0d9effd640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0da00afbe0 0x7f0da00ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0d9effd640 1 -- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0da0071070 msgr2=0x7f0da00c2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0d9effd640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0da0071070 0x7f0da00c2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0d9effd640 1 -- 172.21.15.70:0/3836073820 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88009400 con 0x7f0da006dfc0 2023-10-08T07:14:22.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0d9e7fc640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0da0071070 0x7f0da00c2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:22.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0d9effd640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 0x7f0da015db50 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000efc0 tx=0x7f0d9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:22.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0d97fff640 1 -- 172.21.15.70:0/3836073820 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d90014480 con 0x7f0da006dfc0 2023-10-08T07:14:22.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0d97fff640 1 -- 172.21.15.70:0/3836073820 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9000f040 con 0x7f0da006dfc0 2023-10-08T07:14:22.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0da58e5640 1 -- 172.21.15.70:0/3836073820 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da015d070 con 0x7f0da006dfc0 2023-10-08T07:14:22.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0d97fff640 1 -- 172.21.15.70:0/3836073820 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d90014800 con 0x7f0da006dfc0 2023-10-08T07:14:22.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.169+0000 7f0da58e5640 1 -- 172.21.15.70:0/3836073820 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da015d630 con 0x7f0da006dfc0 2023-10-08T07:14:22.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.170+0000 7f0da58e5640 1 -- 172.21.15.70:0/3836073820 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da015d7f0 con 0x7f0da006dfc0 2023-10-08T07:14:22.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.171+0000 7f0d97fff640 1 -- 172.21.15.70:0/3836073820 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f0d90014960 con 0x7f0da006dfc0 2023-10-08T07:14:22.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.172+0000 7f0d97fff640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0d8006b080 0x7f0d8006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.172+0000 7f0d97fff640 1 -- 172.21.15.70:0/3836073820 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 7826+0+0 (secure 0 0 0) 0x7f0d9008b020 con 0x7f0da006dfc0 2023-10-08T07:14:22.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.173+0000 7f0d9e7fc640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0d8006b080 0x7f0d8006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.174+0000 7f0d9e7fc640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0d8006b080 0x7f0d8006d540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0d88036000 tx=0x7f0d8803c040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:22.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.178+0000 7f0d97fff640 1 -- 172.21.15.70:0/3836073820 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d90055410 con 0x7f0da006dfc0 2023-10-08T07:14:22.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.403+0000 7f0da58e5640 1 -- 172.21.15.70:0/3836073820 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0da004e160 con 0x7f0da006dfc0 2023-10-08T07:14:22.406 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T07:14:22.404+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T07:14:22.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.409+0000 7f0d97fff640 1 -- 172.21.15.70:0/3836073820 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v115) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0d90058a30 con 0x7f0da006dfc0 2023-10-08T07:14:22.410 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T07:14:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 -- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0d8006b080 msgr2=0x7f0d8006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0d8006b080 0x7f0d8006d540 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0d88036000 tx=0x7f0d8803c040 comp rx=0 tx=0).stop 2023-10-08T07:14:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 -- 172.21.15.70:0/3836073820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 msgr2=0x7f0da015db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 0x7f0da015db50 secure :-1 s=READY pgs=2530 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000efc0 tx=0x7f0d9000c490 comp rx=0 tx=0).stop 2023-10-08T07:14:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 -- 172.21.15.70:0/3836073820 shutdown_connections 2023-10-08T07:14:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0d8006b080 0x7f0d8006d540 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0da00afbe0 0x7f0da00ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0da0071070 0x7f0da00c2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 --2- 172.21.15.70:0/3836073820 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0da006dfc0 0x7f0da015db50 unknown :-1 s=CLOSED pgs=2530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 -- 172.21.15.70:0/3836073820 >> 172.21.15.70:0/3836073820 conn(0x7f0da006a5d0 msgr2=0x7f0da006c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:22.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 -- 172.21.15.70:0/3836073820 shutdown_connections 2023-10-08T07:14:22.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.412+0000 7f0da58e5640 1 -- 172.21.15.70:0/3836073820 wait complete. 2023-10-08T07:14:22.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T07:14:22.434 INFO:tasks.ceph.mds.a.smithi070.stderr: -4744> 2023-10-08T07:13:39.587+0000 7f907293a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:14:22.501 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T07:14:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.615+0000 7f3debec2640 1 Processor -- start 2023-10-08T07:14:22.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.616+0000 7f3debec2640 1 -- start start 2023-10-08T07:14:22.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.617+0000 7f3debec2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3de406c3a0 0x7f3de40c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.617+0000 7f3debec2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de415b310 con 0x7f3de406db60 2023-10-08T07:14:22.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.617+0000 7f3debec2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de406d290 con 0x7f3de415aa00 2023-10-08T07:14:22.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.617+0000 7f3debec2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de40ca230 con 0x7f3de406c3a0 2023-10-08T07:14:22.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.617+0000 7f3de945d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3de406db60 0x7f3de406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47748/0 (socket says 172.21.15.70:47748) 2023-10-08T07:14:22.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.617+0000 7f3de945d640 1 -- 172.21.15.70:0/2778261316 learned_addr learned my addr 172.21.15.70:0/2778261316 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:22.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208820673 0 0) 0x7f3dd4001110 con 0x7f3de406db60 2023-10-08T07:14:22.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de9c5e640 1 --2- 172.21.15.70:0/2778261316 >> v2:172.21.15.181:3300/0 conn(0x7f3de406c3a0 0x7f3de40c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd0003520 con 0x7f3de406db60 2023-10-08T07:14:22.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707027517 0 0) 0x7f3dd0003520 con 0x7f3de406db60 2023-10-08T07:14:22.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 >> v2:172.21.15.181:3300/0 conn(0x7f3de406c3a0 msgr2=0x7f3de40c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:22.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 --2- 172.21.15.70:0/2778261316 >> v2:172.21.15.181:3300/0 conn(0x7f3de406c3a0 0x7f3de40c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 >> v1:172.21.15.181:6789/0 conn(0x7f3de415aa00 legacy=0x7f3de40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40c9860 con 0x7f3de406db60 2023-10-08T07:14:22.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3dd4001800 con 0x7f3de406db60 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 >> v1:172.21.15.70:6789/0 conn(0x7f3de406db60 legacy=0x7f3de406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 --2- 172.21.15.70:0/2778261316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dd0003970 0x7f3dd0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 --2- 172.21.15.70:0/2778261316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd00047c0 0x7f3dd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 --2- 172.21.15.70:0/2778261316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dd0005180 0x7f3dd000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd000e200 con 0x7f3dd00047c0 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd000e370 con 0x7f3dd0005180 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.618+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd000e530 con 0x7f3dd0003970 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.619+0000 7f3de945d640 1 --2- 172.21.15.70:0/2778261316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd00047c0 0x7f3dd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.619+0000 7f3de945d640 1 -- 172.21.15.70:0/2778261316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dd0003970 msgr2=0x7f3dd0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.619+0000 7f3de945d640 1 --2- 172.21.15.70:0/2778261316 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dd0003970 0x7f3dd0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.619+0000 7f3de945d640 1 -- 172.21.15.70:0/2778261316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dd0005180 msgr2=0x7f3dd000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:22.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.619+0000 7f3de945d640 1 --2- 172.21.15.70:0/2778261316 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3dd0005180 0x7f3dd000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.619+0000 7f3de945d640 1 -- 172.21.15.70:0/2778261316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd0003520 con 0x7f3dd00047c0 2023-10-08T07:14:22.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.619+0000 7f3dea45f640 1 -- 172.21.15.70:0/2778261316 reap_dead start 2023-10-08T07:14:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.620+0000 7f3de945d640 1 --2- 172.21.15.70:0/2778261316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd00047c0 0x7f3dd0004c40 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400c190 tx=0x7f3dd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92be5ad0538153d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.620+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4005ad0 con 0x7f3dd00047c0 2023-10-08T07:14:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.620+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd4005c50 con 0x7f3dd00047c0 2023-10-08T07:14:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.620+0000 7f3de8c5c640 1 -- 172.21.15.70:0/2778261316 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd402c890 con 0x7f3dd00047c0 2023-10-08T07:14:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.621+0000 7f3debec2640 1 -- 172.21.15.70:0/2778261316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd00047c0 msgr2=0x7f3dd0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.621+0000 7f3debec2640 1 --2- 172.21.15.70:0/2778261316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd00047c0 0x7f3dd0004c40 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400c190 tx=0x7f3dd00057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.621+0000 7f3debec2640 1 -- 172.21.15.70:0/2778261316 shutdown_connections 2023-10-08T07:14:22.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.621+0000 7f3debec2640 1 --2- 172.21.15.70:0/2778261316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dd00047c0 0x7f3dd0004c40 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.621+0000 7f3debec2640 1 -- 172.21.15.70:0/2778261316 >> 172.21.15.70:0/2778261316 conn(0x7f3de406a5d0 msgr2=0x7f3de40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:22.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.621+0000 7f3debec2640 1 -- 172.21.15.70:0/2778261316 shutdown_connections 2023-10-08T07:14:22.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.621+0000 7f3debec2640 1 -- 172.21.15.70:0/2778261316 wait complete. 2023-10-08T07:14:22.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.622+0000 7f3debec2640 1 Processor -- start 2023-10-08T07:14:22.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.622+0000 7f3debec2640 1 -- start start 2023-10-08T07:14:22.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.623+0000 7f3debec2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dd00047c0 0x7f3de4158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.623+0000 7f3debec2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de4152db0 0x7f3de4158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.623+0000 7f3debec2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de40c2270 0x7f3de40c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.623+0000 7f3debec2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40c2fa0 con 0x7f3de4152db0 2023-10-08T07:14:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.623+0000 7f3debec2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40c3280 con 0x7f3de40c2270 2023-10-08T07:14:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.623+0000 7f3debec2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de40c3560 con 0x7f3dd00047c0 2023-10-08T07:14:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3de945d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de4152db0 0x7f3de4158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3de945d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de4152db0 0x7f3de4158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40046/0 (socket says 172.21.15.70:40046) 2023-10-08T07:14:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3de945d640 1 -- 172.21.15.70:0/1798050243 learned_addr learned my addr 172.21.15.70:0/1798050243 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3de9c5e640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dd00047c0 0x7f3de4158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3de945d640 1 -- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dd00047c0 msgr2=0x7f3de4158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3de945d640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dd00047c0 0x7f3de4158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3de945d640 1 -- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de40c2270 msgr2=0x7f3de40c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3de945d640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de40c2270 0x7f3de40c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3de945d640 1 -- 172.21.15.70:0/1798050243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40c3840 con 0x7f3de4152db0 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3de945d640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de4152db0 0x7f3de4158d70 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f3dd402c430 tx=0x7f3de406d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3dda7fc640 1 -- 172.21.15.70:0/1798050243 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd403c040 con 0x7f3de4152db0 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3dda7fc640 1 -- 172.21.15.70:0/1798050243 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd402cde0 con 0x7f3de4152db0 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3dda7fc640 1 -- 172.21.15.70:0/1798050243 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4007270 con 0x7f3de4152db0 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.624+0000 7f3debec2640 1 -- 172.21.15.70:0/1798050243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de40c3b20 con 0x7f3de4152db0 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.625+0000 7f3debec2640 1 -- 172.21.15.70:0/1798050243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de40c4040 con 0x7f3de4152db0 2023-10-08T07:14:22.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.626+0000 7f3debec2640 1 -- 172.21.15.70:0/1798050243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de40c3ce0 con 0x7f3de4152db0 2023-10-08T07:14:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.627+0000 7f3dda7fc640 1 -- 172.21.15.70:0/1798050243 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3dd4032070 con 0x7f3de4152db0 2023-10-08T07:14:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.627+0000 7f3dda7fc640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3dc806b0b0 0x7f3dc806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.628+0000 7f3dda7fc640 1 -- 172.21.15.70:0/1798050243 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7912+0+0 (secure 0 0 0) 0x7f3dd40ab820 con 0x7f3de4152db0 2023-10-08T07:14:22.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.628+0000 7f3de9c5e640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3dc806b0b0 0x7f3dc806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:22.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.629+0000 7f3de9c5e640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3dc806b0b0 0x7f3dc806d570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3de404fa60 tx=0x7f3de0008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:22.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.632+0000 7f3dda7fc640 1 -- 172.21.15.70:0/1798050243 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3dd4075bb0 con 0x7f3de4152db0 2023-10-08T07:14:22.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:22.844+0000 7f3debec2640 1 -- 172.21.15.70:0/1798050243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3de40c33c0 con 0x7f3de4152db0 2023-10-08T07:14:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.413+0000 7f3dda7fc640 1 -- 172.21.15.70:0/1798050243 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v116) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3de40c33c0 con 0x7f3de4152db0 2023-10-08T07:14:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.416+0000 7f3debec2640 1 -- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3dc806b0b0 msgr2=0x7f3dc806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.416+0000 7f3debec2640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3dc806b0b0 0x7f3dc806d570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3de404fa60 tx=0x7f3de0008040 comp rx=0 tx=0).stop 2023-10-08T07:14:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.417+0000 7f3debec2640 1 -- 172.21.15.70:0/1798050243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de4152db0 msgr2=0x7f3de4158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.417+0000 7f3debec2640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de4152db0 0x7f3de4158d70 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f3dd402c430 tx=0x7f3de406d8e0 comp rx=0 tx=0).stop 2023-10-08T07:14:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.417+0000 7f3debec2640 1 -- 172.21.15.70:0/1798050243 shutdown_connections 2023-10-08T07:14:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.417+0000 7f3debec2640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3dc806b0b0 0x7f3dc806d570 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.417+0000 7f3debec2640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3de40c2270 0x7f3de40c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.417+0000 7f3debec2640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de4152db0 0x7f3de4158d70 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.417+0000 7f3debec2640 1 --2- 172.21.15.70:0/1798050243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3dd00047c0 0x7f3de4158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:23.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.417+0000 7f3debec2640 1 -- 172.21.15.70:0/1798050243 >> 172.21.15.70:0/1798050243 conn(0x7f3de406a5d0 msgr2=0x7f3de40cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:23.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.417+0000 7f3debec2640 1 -- 172.21.15.70:0/1798050243 shutdown_connections 2023-10-08T07:14:23.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.417+0000 7f3debec2640 1 -- 172.21.15.70:0/1798050243 wait complete. 2023-10-08T07:14:23.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T07:14:23.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.651+0000 7f9bf7875640 1 Processor -- start 2023-10-08T07:14:23.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.652+0000 7f9bf7875640 1 -- start start 2023-10-08T07:14:23.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.653+0000 7f9bf7875640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9bf0160660 0x7f9bf0070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:23.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.653+0000 7f9bf7875640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9bf00cb620 0x7f9bf00b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:23.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.653+0000 7f9bf7875640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf006e150 con 0x7f9bf00c9910 2023-10-08T07:14:23.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.653+0000 7f9bf7875640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf006de70 con 0x7f9bf0160660 2023-10-08T07:14:23.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.653+0000 7f9bf7875640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf00c8e30 con 0x7f9bf00cb620 2023-10-08T07:14:23.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.653+0000 7f9bf5611640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9bf0160660 0x7f9bf0070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:23.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.653+0000 7f9bf5611640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9bf0160660 0x7f9bf0070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42858/0 (socket says 172.21.15.70:42858) 2023-10-08T07:14:23.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.654+0000 7f9bf5611640 1 -- 172.21.15.70:0/3907482544 learned_addr learned my addr 172.21.15.70:0/3907482544 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:23.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.654+0000 7f9bf5e12640 1 --2- 172.21.15.70:0/3907482544 >> v2:172.21.15.181:3300/0 conn(0x7f9bf00cb620 0x7f9bf00b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:23.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.654+0000 7f9bf5611640 1 -- 172.21.15.70:0/3907482544 >> v2:172.21.15.181:3300/0 conn(0x7f9bf00cb620 msgr2=0x7f9bf00b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:23.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.654+0000 7f9bf5611640 1 --2- 172.21.15.70:0/3907482544 >> v2:172.21.15.181:3300/0 conn(0x7f9bf00cb620 0x7f9bf00b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:23.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.654+0000 7f9bf5611640 1 -- 172.21.15.70:0/3907482544 >> v1:172.21.15.181:6789/0 conn(0x7f9bf00c9910 legacy=0x7f9bf00caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:23.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.654+0000 7f9bf5611640 1 -- 172.21.15.70:0/3907482544 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00c9110 con 0x7f9bf0160660 2023-10-08T07:14:23.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.654+0000 7f9bf5e12640 1 --2- 172.21.15.70:0/3907482544 >> v2:172.21.15.181:3300/0 conn(0x7f9bf00cb620 0x7f9bf00b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.655+0000 7f9bf5611640 1 --2- 172.21.15.70:0/3907482544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 0x7f9bf0070cc0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00b1760 tx=0x7f9bec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec712eedc3beddd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.656+0000 7f9be7fff640 1 -- 172.21.15.70:0/3907482544 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0317e0 con 0x7f9bf0160660 2023-10-08T07:14:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.656+0000 7f9be7fff640 1 -- 172.21.15.70:0/3907482544 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec031940 con 0x7f9bf0160660 2023-10-08T07:14:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.656+0000 7f9be7fff640 1 -- 172.21.15.70:0/3907482544 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec031cd0 con 0x7f9bf0160660 2023-10-08T07:14:23.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.656+0000 7f9bf7875640 1 -- 172.21.15.70:0/3907482544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 msgr2=0x7f9bf0070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:23.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.656+0000 7f9bf7875640 1 --2- 172.21.15.70:0/3907482544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 0x7f9bf0070cc0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00b1760 tx=0x7f9bec02b880 comp rx=0 tx=0).stop 2023-10-08T07:14:23.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.657+0000 7f9bf7875640 1 -- 172.21.15.70:0/3907482544 shutdown_connections 2023-10-08T07:14:23.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.657+0000 7f9bf7875640 1 --2- 172.21.15.70:0/3907482544 >> v2:172.21.15.181:3300/0 conn(0x7f9bf00cb620 0x7f9bf00b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:23.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.657+0000 7f9bf7875640 1 --2- 172.21.15.70:0/3907482544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 0x7f9bf0070cc0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:23.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.657+0000 7f9bf7875640 1 -- 172.21.15.70:0/3907482544 >> 172.21.15.70:0/3907482544 conn(0x7f9bf006a5d0 msgr2=0x7f9bf00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:23.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.657+0000 7f9bf7875640 1 -- 172.21.15.70:0/3907482544 shutdown_connections 2023-10-08T07:14:23.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.657+0000 7f9bf7875640 1 -- 172.21.15.70:0/3907482544 wait complete. 2023-10-08T07:14:23.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.659+0000 7f9bf7875640 1 Processor -- start 2023-10-08T07:14:23.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.659+0000 7f9bf7875640 1 -- start start 2023-10-08T07:14:23.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.660+0000 7f9bf7875640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00c9910 0x7f9bf01f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:23.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.660+0000 7f9bf7875640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00cb620 0x7f9bf00c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:23.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.660+0000 7f9bf7875640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 0x7f9bf015e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:23.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.660+0000 7f9bf7875640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf01f6510 con 0x7f9bf0160660 2023-10-08T07:14:23.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.660+0000 7f9bf7875640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf00c4f00 con 0x7f9bf00cb620 2023-10-08T07:14:23.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.660+0000 7f9bf7875640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf01f5ec0 con 0x7f9bf00c9910 2023-10-08T07:14:23.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.660+0000 7f9bf5e12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 0x7f9bf015e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:23.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf5e12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 0x7f9bf015e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42870/0 (socket says 172.21.15.70:42870) 2023-10-08T07:14:23.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf5e12640 1 -- 172.21.15.70:0/1298912475 learned_addr learned my addr 172.21.15.70:0/1298912475 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:23.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf5e12640 1 -- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00c9910 msgr2=0x7f9bf01f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:23.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf4e10640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00cb620 0x7f9bf00c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:23.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf5611640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00c9910 0x7f9bf01f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:23.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf5e12640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00c9910 0x7f9bf01f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:23.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf5e12640 1 -- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00cb620 msgr2=0x7f9bf00c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:23.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf5e12640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00cb620 0x7f9bf00c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:23.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf5e12640 1 -- 172.21.15.70:0/1298912475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec0094d0 con 0x7f9bf0160660 2023-10-08T07:14:23.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf4e10640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00cb620 0x7f9bf00c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:23.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.661+0000 7f9bf5e12640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 0x7f9bf015e6a0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f9be800cef0 tx=0x7f9be800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:23.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.662+0000 7f9be5ffb640 1 -- 172.21.15.70:0/1298912475 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be800d840 con 0x7f9bf0160660 2023-10-08T07:14:23.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.662+0000 7f9be5ffb640 1 -- 172.21.15.70:0/1298912475 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be800d9c0 con 0x7f9bf0160660 2023-10-08T07:14:23.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.662+0000 7f9be5ffb640 1 -- 172.21.15.70:0/1298912475 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be800dd70 con 0x7f9bf0160660 2023-10-08T07:14:23.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.662+0000 7f9bf7875640 1 -- 172.21.15.70:0/1298912475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf01f61a0 con 0x7f9bf0160660 2023-10-08T07:14:23.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.662+0000 7f9bf7875640 1 -- 172.21.15.70:0/1298912475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf014add0 con 0x7f9bf0160660 2023-10-08T07:14:23.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.663+0000 7f9bf7875640 1 -- 172.21.15.70:0/1298912475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf01f5d20 con 0x7f9bf0160660 2023-10-08T07:14:23.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.664+0000 7f9be5ffb640 1 -- 172.21.15.70:0/1298912475 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f9be801e3b0 con 0x7f9bf0160660 2023-10-08T07:14:23.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.665+0000 7f9be5ffb640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f9bd406b0b0 0x7f9bd406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:23.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.665+0000 7f9be5ffb640 1 -- 172.21.15.70:0/1298912475 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7912+0+0 (secure 0 0 0) 0x7f9be808ad90 con 0x7f9bf0160660 2023-10-08T07:14:23.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.665+0000 7f9bf5611640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f9bd406b0b0 0x7f9bd406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:23.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.667+0000 7f9bf5611640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f9bd406b0b0 0x7f9bd406d570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9bec036000 tx=0x7f9bec03d040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:23.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.669+0000 7f9be5ffb640 1 -- 172.21.15.70:0/1298912475 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9bf01f5d20 con 0x7f9bf0160660 2023-10-08T07:14:23.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:23.862+0000 7f9bf7875640 1 -- 172.21.15.70:0/1298912475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9bf01f5d20 con 0x7f9bf0160660 2023-10-08T07:14:24.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.424+0000 7f9be5ffb640 1 -- 172.21.15.70:0/1298912475 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v242) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9bf01f5d20 con 0x7f9bf0160660 2023-10-08T07:14:24.426 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T07:14:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 -- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f9bd406b0b0 msgr2=0x7f9bd406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f9bd406b0b0 0x7f9bd406d570 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9bec036000 tx=0x7f9bec03d040 comp rx=0 tx=0).stop 2023-10-08T07:14:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 -- 172.21.15.70:0/1298912475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 msgr2=0x7f9bf015e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 0x7f9bf015e6a0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f9be800cef0 tx=0x7f9be800a710 comp rx=0 tx=0).stop 2023-10-08T07:14:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 -- 172.21.15.70:0/1298912475 shutdown_connections 2023-10-08T07:14:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f9bd406b0b0 0x7f9bd406d570 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9bf0160660 0x7f9bf015e6a0 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9bf00cb620 0x7f9bf00c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:24.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 --2- 172.21.15.70:0/1298912475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9bf00c9910 0x7f9bf01f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:24.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 -- 172.21.15.70:0/1298912475 >> 172.21.15.70:0/1298912475 conn(0x7f9bf006a5d0 msgr2=0x7f9bf00c7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:24.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 -- 172.21.15.70:0/1298912475 shutdown_connections 2023-10-08T07:14:24.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.428+0000 7f9bf7875640 1 -- 172.21.15.70:0/1298912475 wait complete. 2023-10-08T07:14:24.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T07:14:24.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.652+0000 7fea106a2640 1 Processor -- start 2023-10-08T07:14:24.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.652+0000 7fea106a2640 1 -- start start 2023-10-08T07:14:24.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.654+0000 7fea106a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea080cb620 0x7fea080b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:24.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.654+0000 7fea106a2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea08071170 con 0x7fea08160660 2023-10-08T07:14:24.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.654+0000 7fea106a2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0806e150 con 0x7fea08070cc0 2023-10-08T07:14:24.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.654+0000 7fea106a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0806de70 con 0x7fea080cb620 2023-10-08T07:14:24.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.654+0000 7fea0e43e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fea08160660 0x7fea080c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58036/0 (socket says 172.21.15.70:58036) 2023-10-08T07:14:24.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.654+0000 7fea0e43e640 1 -- 172.21.15.70:0/2521196920 learned_addr learned my addr 172.21.15.70:0/2521196920 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:24.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.654+0000 7fea0ec3f640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea080cb620 0x7fea080b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:24.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447026145 0 0) 0x7fea04001110 con 0x7fea08160660 2023-10-08T07:14:24.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9f4003520 con 0x7fea08160660 2023-10-08T07:14:24.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 43505011 0 0) 0x7fe9f4003520 con 0x7fea08160660 2023-10-08T07:14:24.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea080cb620 msgr2=0x7fea080b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:24.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea080cb620 0x7fea080b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:24.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 >> v1:172.21.15.181:6789/0 conn(0x7fea08070cc0 legacy=0x7fea080caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea080c8e30 con 0x7fea08160660 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fea04001800 con 0x7fea08160660 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 >> v1:172.21.15.70:6789/0 conn(0x7fea08160660 legacy=0x7fea080c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f40039a0 0x7fe9f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea080cb620 0x7fe9f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4004d00 0x7fe9f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0806de70 con 0x7fe9f40039a0 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f400ba60 con 0x7fe9f4004d00 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.655+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f400bc20 con 0x7fea080cb620 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.656+0000 7fea0e43e640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f40039a0 0x7fe9f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:24.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.656+0000 7fea0dc3d640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea080cb620 0x7fe9f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.656+0000 7fea0ec3f640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4004d00 0x7fe9f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.656+0000 7fea0e43e640 1 -- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea080cb620 msgr2=0x7fe9f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.656+0000 7fea0e43e640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fea080cb620 0x7fe9f40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.656+0000 7fea0e43e640 1 -- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4004d00 msgr2=0x7fe9f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.656+0000 7fea0e43e640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4004d00 0x7fe9f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.656+0000 7fea0e43e640 1 -- 172.21.15.70:0/2521196920 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f4003520 con 0x7fe9f40039a0 2023-10-08T07:14:24.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.656+0000 7fea0ec3f640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe9f4004d00 0x7fe9f4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:24.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.657+0000 7fea0e43e640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f40039a0 0x7fe9f4003dc0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fea04001eb0 tx=0x7fe9f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a00452b077ca0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:24.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.658+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04005ad0 con 0x7fe9f40039a0 2023-10-08T07:14:24.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.658+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea04005c50 con 0x7fe9f40039a0 2023-10-08T07:14:24.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.658+0000 7fea0d43c640 1 -- 172.21.15.70:0/2521196920 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0402c7c0 con 0x7fe9f40039a0 2023-10-08T07:14:24.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.658+0000 7fea106a2640 1 -- 172.21.15.70:0/2521196920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f40039a0 msgr2=0x7fe9f4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:24.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.658+0000 7fea106a2640 1 --2- 172.21.15.70:0/2521196920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f40039a0 0x7fe9f4003dc0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7fea04001eb0 tx=0x7fe9f40057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:24.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.658+0000 7fea0ec3f640 1 -- 172.21.15.70:0/2521196920 reap_dead start 2023-10-08T07:14:24.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.658+0000 7fea106a2640 1 -- 172.21.15.70:0/2521196920 shutdown_connections 2023-10-08T07:14:24.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.658+0000 7fea106a2640 1 -- 172.21.15.70:0/2521196920 >> 172.21.15.70:0/2521196920 conn(0x7fea0806a5d0 msgr2=0x7fea080b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:24.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.659+0000 7fea106a2640 1 -- 172.21.15.70:0/2521196920 shutdown_connections 2023-10-08T07:14:24.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.659+0000 7fea106a2640 1 -- 172.21.15.70:0/2521196920 wait complete. 2023-10-08T07:14:24.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.660+0000 7fea106a2640 1 Processor -- start 2023-10-08T07:14:24.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.661+0000 7fea106a2640 1 -- start start 2023-10-08T07:14:24.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.661+0000 7fea106a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9f40039a0 0x7fea080c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:24.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.661+0000 7fea106a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea081598f0 0x7fea080c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:24.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.661+0000 7fea106a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea081573f0 0x7fea08157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:24.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.661+0000 7fea106a2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea080c8250 con 0x7fea081598f0 2023-10-08T07:14:24.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.661+0000 7fea106a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea08157dc0 con 0x7fea081573f0 2023-10-08T07:14:24.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.661+0000 7fea106a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea080c71b0 con 0x7fe9f40039a0 2023-10-08T07:14:24.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0dc3d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea081598f0 0x7fea080c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:24.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0dc3d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea081598f0 0x7fea080c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42894/0 (socket says 172.21.15.70:42894) 2023-10-08T07:14:24.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0dc3d640 1 -- 172.21.15.70:0/1545427126 learned_addr learned my addr 172.21.15.70:0/1545427126 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:24.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0e43e640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9f40039a0 0x7fea080c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:24.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0dc3d640 1 -- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9f40039a0 msgr2=0x7fea080c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:24.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0dc3d640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9f40039a0 0x7fea080c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:24.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0dc3d640 1 -- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea081573f0 msgr2=0x7fea08157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:24.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0dc3d640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea081573f0 0x7fea08157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:24.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0dc3d640 1 -- 172.21.15.70:0/1545427126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea080c7490 con 0x7fea081598f0 2023-10-08T07:14:24.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0e43e640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9f40039a0 0x7fea080c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:24.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.662+0000 7fea0dc3d640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea081598f0 0x7fea080c6860 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e490 tx=0x7fe9f800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:24.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.663+0000 7fe9feffd640 1 -- 172.21.15.70:0/1545427126 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80052e0 con 0x7fea081598f0 2023-10-08T07:14:24.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.663+0000 7fe9feffd640 1 -- 172.21.15.70:0/1545427126 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f8005440 con 0x7fea081598f0 2023-10-08T07:14:24.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.663+0000 7fea106a2640 1 -- 172.21.15.70:0/1545427126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea080c7770 con 0x7fea081598f0 2023-10-08T07:14:24.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.663+0000 7fe9feffd640 1 -- 172.21.15.70:0/1545427126 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8003810 con 0x7fea081598f0 2023-10-08T07:14:24.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.663+0000 7fea106a2640 1 -- 172.21.15.70:0/1545427126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea080c7d30 con 0x7fea081598f0 2023-10-08T07:14:24.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.664+0000 7fea106a2640 1 -- 172.21.15.70:0/1545427126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea080c7ef0 con 0x7fea081598f0 2023-10-08T07:14:24.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.665+0000 7fe9feffd640 1 -- 172.21.15.70:0/1545427126 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe9f8003970 con 0x7fea081598f0 2023-10-08T07:14:24.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.666+0000 7fe9feffd640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe9ec06b0e0 0x7fe9ec06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:24.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.666+0000 7fe9feffd640 1 -- 172.21.15.70:0/1545427126 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7474+0+0 (secure 0 0 0) 0x7fe9f80898b0 con 0x7fea081598f0 2023-10-08T07:14:24.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.667+0000 7fea0e43e640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe9ec06b0e0 0x7fe9ec06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:24.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.668+0000 7fea0e43e640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe9ec06b0e0 0x7fe9ec06d5a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fea04034000 tx=0x7fea04009030 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:24.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.671+0000 7fe9feffd640 1 -- 172.21.15.70:0/1545427126 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9f8053fb0 con 0x7fea081598f0 2023-10-08T07:14:24.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:24.861+0000 7fea106a2640 1 -- 172.21.15.70:0/1545427126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fea0804e160 con 0x7fea081598f0 2023-10-08T07:14:25.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.428+0000 7fe9feffd640 1 -- 172.21.15.70:0/1545427126 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v243) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe9f80190d0 con 0x7fea081598f0 2023-10-08T07:14:25.430 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T07:14:25.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.433+0000 7fea106a2640 1 -- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe9ec06b0e0 msgr2=0x7fe9ec06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.433+0000 7fea106a2640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe9ec06b0e0 0x7fe9ec06d5a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fea04034000 tx=0x7fea04009030 comp rx=0 tx=0).stop 2023-10-08T07:14:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.433+0000 7fea106a2640 1 -- 172.21.15.70:0/1545427126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea081598f0 msgr2=0x7fea080c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.433+0000 7fea106a2640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea081598f0 0x7fea080c6860 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800e490 tx=0x7fe9f800e950 comp rx=0 tx=0).stop 2023-10-08T07:14:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.433+0000 7fea106a2640 1 -- 172.21.15.70:0/1545427126 shutdown_connections 2023-10-08T07:14:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.433+0000 7fea106a2640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe9ec06b0e0 0x7fe9ec06d5a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.433+0000 7fea106a2640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fea081573f0 0x7fea08157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.433+0000 7fea106a2640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea081598f0 0x7fea080c6860 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.433+0000 7fea106a2640 1 --2- 172.21.15.70:0/1545427126 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe9f40039a0 0x7fea080c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.433+0000 7fea106a2640 1 -- 172.21.15.70:0/1545427126 >> 172.21.15.70:0/1545427126 conn(0x7fea0806a5d0 msgr2=0x7fea080c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:25.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.434+0000 7fea106a2640 1 -- 172.21.15.70:0/1545427126 shutdown_connections 2023-10-08T07:14:25.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.434+0000 7fea106a2640 1 -- 172.21.15.70:0/1545427126 wait complete. 2023-10-08T07:14:25.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:25.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.659+0000 7f2780456640 1 Processor -- start 2023-10-08T07:14:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.659+0000 7f2780456640 1 -- start start 2023-10-08T07:14:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.660+0000 7f2780456640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27780cb580 0x7f2778165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.660+0000 7f2780456640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277815b2b0 con 0x7f277815ab40 2023-10-08T07:14:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.660+0000 7f2780456640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277815b590 con 0x7f27780c90b0 2023-10-08T07:14:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.660+0000 7f2780456640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277815bf30 con 0x7f27780cb580 2023-10-08T07:14:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277e1f2640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f277815ab40 0x7f27780ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58038/0 (socket says 172.21.15.70:58038) 2023-10-08T07:14:25.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277e1f2640 1 -- 172.21.15.70:0/3205472315 learned_addr learned my addr 172.21.15.70:0/3205472315 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162344786 0 0) 0x7f277815b2b0 con 0x7f277815ab40 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2764003520 con 0x7f277815ab40 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277e9f3640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27780cb580 0x7f2778165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2229191239 0 0) 0x7f2764003520 con 0x7f277815ab40 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27780cb580 msgr2=0x7f2778165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277d1f0640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27780cb580 0x7f2778165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 >> v1:172.21.15.181:6789/0 conn(0x7f27780c90b0 legacy=0x7f27780cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277815b870 con 0x7f277815ab40 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2774001690 con 0x7f277815ab40 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.661+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 >> v1:172.21.15.70:6789/0 conn(0x7f277815ab40 legacy=0x7f27780ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277d1f0640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27640039a0 0x7f2764003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277d1f0640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27780cb580 0x7f27640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277d1f0640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2764004d00 0x7f2764005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277815bf30 con 0x7f27640039a0 2023-10-08T07:14:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276400bb10 con 0x7f2764004d00 2023-10-08T07:14:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276400bcd0 con 0x7f27780cb580 2023-10-08T07:14:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277e1f2640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27640039a0 0x7f2764003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277e9f3640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2764004d00 0x7f2764005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277d9f1640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27780cb580 0x7f27640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277e1f2640 1 -- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27780cb580 msgr2=0x7f27640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277e1f2640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27780cb580 0x7f27640047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277e1f2640 1 -- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2764004d00 msgr2=0x7f2764005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277e1f2640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2764004d00 0x7f2764005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277e1f2640 1 -- 172.21.15.70:0/3205472315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2764003520 con 0x7f27640039a0 2023-10-08T07:14:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.662+0000 7f277e9f3640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2764004d00 0x7f2764005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.663+0000 7f277e1f2640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27640039a0 0x7f2764003da0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f2774001eb0 tx=0x7f27640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4a2bb74dd4631 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:25.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.664+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2774005ad0 con 0x7f27640039a0 2023-10-08T07:14:25.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.664+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2774005c50 con 0x7f27640039a0 2023-10-08T07:14:25.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.664+0000 7f277d1f0640 1 -- 172.21.15.70:0/3205472315 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277402c890 con 0x7f27640039a0 2023-10-08T07:14:25.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.664+0000 7f2780456640 1 -- 172.21.15.70:0/3205472315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27640039a0 msgr2=0x7f2764003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.664+0000 7f2780456640 1 --2- 172.21.15.70:0/3205472315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27640039a0 0x7f2764003da0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f2774001eb0 tx=0x7f27640057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.664+0000 7f277e9f3640 1 -- 172.21.15.70:0/3205472315 reap_dead start 2023-10-08T07:14:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.664+0000 7f2780456640 1 -- 172.21.15.70:0/3205472315 shutdown_connections 2023-10-08T07:14:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.664+0000 7f2780456640 1 -- 172.21.15.70:0/3205472315 >> 172.21.15.70:0/3205472315 conn(0x7f277806a380 msgr2=0x7f27780b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.664+0000 7f2780456640 1 -- 172.21.15.70:0/3205472315 shutdown_connections 2023-10-08T07:14:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.664+0000 7f2780456640 1 -- 172.21.15.70:0/3205472315 wait complete. 2023-10-08T07:14:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.666+0000 7f2780456640 1 Processor -- start 2023-10-08T07:14:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.666+0000 7f2780456640 1 -- start start 2023-10-08T07:14:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.667+0000 7f2780456640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27640039a0 0x7f27780c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.667+0000 7f2780456640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2778149570 0x7f27780c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.667+0000 7f2780456640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27780c4ed0 0x7f27780aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.667+0000 7f2780456640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277815f7a0 con 0x7f2778149570 2023-10-08T07:14:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.667+0000 7f2780456640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27780c56e0 con 0x7f27780c4ed0 2023-10-08T07:14:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.667+0000 7f2780456640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27780c3e10 con 0x7f27640039a0 2023-10-08T07:14:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277d9f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2778149570 0x7f27780c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277d9f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2778149570 0x7f27780c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42924/0 (socket says 172.21.15.70:42924) 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277d9f1640 1 -- 172.21.15.70:0/109898542 learned_addr learned my addr 172.21.15.70:0/109898542 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277e1f2640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27640039a0 0x7f27780c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277e9f3640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27780c4ed0 0x7f27780aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277d9f1640 1 -- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27640039a0 msgr2=0x7f27780c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277d9f1640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27640039a0 0x7f27780c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277d9f1640 1 -- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27780c4ed0 msgr2=0x7f27780aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277d9f1640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27780c4ed0 0x7f27780aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277d9f1640 1 -- 172.21.15.70:0/109898542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27780c47e0 con 0x7f2778149570 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277e1f2640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27640039a0 0x7f27780c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.668+0000 7f277d9f1640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2778149570 0x7f27780c58e0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f276800d4f0 tx=0x7f276800d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.669+0000 7f276effd640 1 -- 172.21.15.70:0/109898542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768005230 con 0x7f2778149570 2023-10-08T07:14:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.669+0000 7f276effd640 1 -- 172.21.15.70:0/109898542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2768005390 con 0x7f2778149570 2023-10-08T07:14:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.669+0000 7f276effd640 1 -- 172.21.15.70:0/109898542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768012840 con 0x7f2778149570 2023-10-08T07:14:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.669+0000 7f2780456640 1 -- 172.21.15.70:0/109898542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27780c5400 con 0x7f2778149570 2023-10-08T07:14:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.669+0000 7f2780456640 1 -- 172.21.15.70:0/109898542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27780c3aa0 con 0x7f2778149570 2023-10-08T07:14:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.670+0000 7f2780456640 1 -- 172.21.15.70:0/109898542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27780c3620 con 0x7f2778149570 2023-10-08T07:14:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.671+0000 7f276effd640 1 -- 172.21.15.70:0/109898542 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f27680129a0 con 0x7f2778149570 2023-10-08T07:14:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.672+0000 7f276effd640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f275c06b0e0 0x7f275c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.672+0000 7f276effd640 1 -- 172.21.15.70:0/109898542 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f2768089c50 con 0x7f2778149570 2023-10-08T07:14:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.673+0000 7f277e1f2640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f275c06b0e0 0x7f275c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:25.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.674+0000 7f277e1f2640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f275c06b0e0 0x7f275c06d5a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2774001eb0 tx=0x7f2774009030 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:25.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.676+0000 7f276effd640 1 -- 172.21.15.70:0/109898542 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27780c3620 con 0x7f2778149570 2023-10-08T07:14:25.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.866+0000 7f2780456640 1 -- 172.21.15.70:0/109898542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27780c3620 con 0x7f2778149570 2023-10-08T07:14:25.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.869+0000 7f276effd640 1 -- 172.21.15.70:0/109898542 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v243) v1 ==== 74+0+18232 (secure 0 0 0) 0x7f27780c3620 con 0x7f2778149570 2023-10-08T07:14:25.871 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:25.871 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":243,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:25.425545+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:13:47.007437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:13:47.838023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2480387829":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/920349426":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/2459293689":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/4129076201":"2023-10-09T07:14:16.815350+0000","172.21.15.70:6834/180995576":"2023-10-09T07:14:22.405699+0000","172.21.15.181:6832/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/365280967":"2023-10-09T07:14:16.815350+0000","172.21.15.70:6832/180995576":"2023-10-09T07:14:22.405699+0000","172.21.15.181:0/2833418931":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/753176203":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/317173509":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:6832/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1018014196":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1778827440":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1217032325":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/2798662747":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/4139611391":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/3002223432":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/3074626816":"2023-10-09T07:14:06.545661+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 -- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f275c06b0e0 msgr2=0x7f275c06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f275c06b0e0 0x7f275c06d5a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2774001eb0 tx=0x7f2774009030 comp rx=0 tx=0).stop 2023-10-08T07:14:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 -- 172.21.15.70:0/109898542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2778149570 msgr2=0x7f27780c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2778149570 0x7f27780c58e0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f276800d4f0 tx=0x7f276800d9b0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 -- 172.21.15.70:0/109898542 shutdown_connections 2023-10-08T07:14:25.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f275c06b0e0 0x7f275c06d5a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27780c4ed0 0x7f27780aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2778149570 0x7f27780c58e0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 --2- 172.21.15.70:0/109898542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27640039a0 0x7f27780c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:25.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 -- 172.21.15.70:0/109898542 >> 172.21.15.70:0/109898542 conn(0x7f277806a380 msgr2=0x7f27780c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:25.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 -- 172.21.15.70:0/109898542 shutdown_connections 2023-10-08T07:14:25.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:25.872+0000 7f2780456640 1 -- 172.21.15.70:0/109898542 wait complete. 2023-10-08T07:14:25.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-08T07:13:47.007437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-08T07:13:47.838023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '238', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:14:25.893 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:14:26.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.107+0000 7fb865f4c640 1 Processor -- start 2023-10-08T07:14:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.107+0000 7fb865f4c640 1 -- start start 2023-10-08T07:14:26.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.109+0000 7fb865f4c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8601432f0 0x7fb8600cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.109+0000 7fb865f4c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8600b4240 0x7fb860159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.109+0000 7fb865f4c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8600cbb30 con 0x7fb8600cb6b0 2023-10-08T07:14:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.109+0000 7fb865f4c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8601436d0 con 0x7fb8600b4240 2023-10-08T07:14:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.109+0000 7fb865f4c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86015d440 con 0x7fb8601432f0 2023-10-08T07:14:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.110+0000 7fb85ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8600b4240 0x7fb860159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.110+0000 7fb85f7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8601432f0 0x7fb8600cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.110+0000 7fb85ffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8600b4240 0x7fb860159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:46266/0 (socket says 172.21.15.70:46266) 2023-10-08T07:14:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.110+0000 7fb85ffff640 1 -- 172.21.15.70:0/2478348238 learned_addr learned my addr 172.21.15.70:0/2478348238 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.110+0000 7fb85ffff640 1 -- 172.21.15.70:0/2478348238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8601432f0 msgr2=0x7fb8600cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.110+0000 7fb85ffff640 1 --2- 172.21.15.70:0/2478348238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8601432f0 0x7fb8600cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.110+0000 7fb85ffff640 1 -- 172.21.15.70:0/2478348238 >> v1:172.21.15.181:6789/0 conn(0x7fb8600cb6b0 legacy=0x7fb8600b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.110+0000 7fb85ffff640 1 -- 172.21.15.70:0/2478348238 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86015d690 con 0x7fb8600b4240 2023-10-08T07:14:26.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.112+0000 7fb85ffff640 1 --2- 172.21.15.70:0/2478348238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8600b4240 0x7fb860159e50 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb858009e50 tx=0x7fb85802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df3d836f0220bde server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.112+0000 7fb85effd640 1 -- 172.21.15.70:0/2478348238 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85802ca80 con 0x7fb8600b4240 2023-10-08T07:14:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.112+0000 7fb85effd640 1 -- 172.21.15.70:0/2478348238 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85802cbe0 con 0x7fb8600b4240 2023-10-08T07:14:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.112+0000 7fb85effd640 1 -- 172.21.15.70:0/2478348238 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8580343f0 con 0x7fb8600b4240 2023-10-08T07:14:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.113+0000 7fb865f4c640 1 -- 172.21.15.70:0/2478348238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8600b4240 msgr2=0x7fb860159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.113+0000 7fb865f4c640 1 --2- 172.21.15.70:0/2478348238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8600b4240 0x7fb860159e50 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb858009e50 tx=0x7fb85802bee0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.113+0000 7fb865f4c640 1 -- 172.21.15.70:0/2478348238 shutdown_connections 2023-10-08T07:14:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.113+0000 7fb865f4c640 1 --2- 172.21.15.70:0/2478348238 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8600b4240 0x7fb860159e50 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.113+0000 7fb865f4c640 1 --2- 172.21.15.70:0/2478348238 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8601432f0 0x7fb8600cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.113+0000 7fb865f4c640 1 -- 172.21.15.70:0/2478348238 >> 172.21.15.70:0/2478348238 conn(0x7fb860069d50 msgr2=0x7fb86006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.113+0000 7fb865f4c640 1 -- 172.21.15.70:0/2478348238 shutdown_connections 2023-10-08T07:14:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.113+0000 7fb865f4c640 1 -- 172.21.15.70:0/2478348238 wait complete. 2023-10-08T07:14:26.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.116+0000 7fb865f4c640 1 Processor -- start 2023-10-08T07:14:26.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.116+0000 7fb865f4c640 1 -- start start 2023-10-08T07:14:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.117+0000 7fb865f4c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8600b4240 0x7fb86015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.117+0000 7fb865f4c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8600cb6b0 0x7fb8601573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.117+0000 7fb865f4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8601432f0 0x7fb8601e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.117+0000 7fb865f4c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8601e43c0 con 0x7fb8601432f0 2023-10-08T07:14:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.117+0000 7fb865f4c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86012e060 con 0x7fb8600b4240 2023-10-08T07:14:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.117+0000 7fb865f4c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86015c2c0 con 0x7fb8600cb6b0 2023-10-08T07:14:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8601432f0 0x7fb8601e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8601432f0 0x7fb8601e3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42928/0 (socket says 172.21.15.70:42928) 2023-10-08T07:14:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb85ffff640 1 -- 172.21.15.70:0/659295486 learned_addr learned my addr 172.21.15.70:0/659295486 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:26.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb85f7fe640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8600b4240 0x7fb86015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb857fff640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8600cb6b0 0x7fb8601573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb85ffff640 1 -- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8600cb6b0 msgr2=0x7fb8601573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb85ffff640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8600cb6b0 0x7fb8601573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb85ffff640 1 -- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8600b4240 msgr2=0x7fb86015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb85ffff640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8600b4240 0x7fb86015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb85ffff640 1 -- 172.21.15.70:0/659295486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb858009b30 con 0x7fb8601432f0 2023-10-08T07:14:26.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.118+0000 7fb857fff640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8600cb6b0 0x7fb8601573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:26.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.119+0000 7fb85ffff640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8601432f0 0x7fb8601e3e80 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7fb858009e50 tx=0x7fb858004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:26.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.119+0000 7fb85cff9640 1 -- 172.21.15.70:0/659295486 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858009510 con 0x7fb8601432f0 2023-10-08T07:14:26.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.119+0000 7fb85cff9640 1 -- 172.21.15.70:0/659295486 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85803e450 con 0x7fb8601432f0 2023-10-08T07:14:26.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.119+0000 7fb85cff9640 1 -- 172.21.15.70:0/659295486 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85803e800 con 0x7fb8601432f0 2023-10-08T07:14:26.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.119+0000 7fb865f4c640 1 -- 172.21.15.70:0/659295486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86015c5a0 con 0x7fb8601432f0 2023-10-08T07:14:26.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.119+0000 7fb865f4c640 1 -- 172.21.15.70:0/659295486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86015cb60 con 0x7fb8601432f0 2023-10-08T07:14:26.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.120+0000 7fb865f4c640 1 -- 172.21.15.70:0/659295486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86012e1d0 con 0x7fb8601432f0 2023-10-08T07:14:26.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.121+0000 7fb85cff9640 1 -- 172.21.15.70:0/659295486 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb85802cd50 con 0x7fb8601432f0 2023-10-08T07:14:26.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.122+0000 7fb85cff9640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb84006b0b0 0x7fb84006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.122+0000 7fb85cff9640 1 -- 172.21.15.70:0/659295486 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fb8580acad0 con 0x7fb8601432f0 2023-10-08T07:14:26.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.123+0000 7fb85f7fe640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb84006b0b0 0x7fb84006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.124+0000 7fb85f7fe640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb84006b0b0 0x7fb84006d570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb850009a30 tx=0x7fb850008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:26.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.127+0000 7fb85cff9640 1 -- 172.21.15.70:0/659295486 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb86012e1d0 con 0x7fb8601432f0 2023-10-08T07:14:26.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.313+0000 7fb865f4c640 1 -- 172.21.15.70:0/659295486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb86012e1d0 con 0x7fb8601432f0 2023-10-08T07:14:26.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.314+0000 7fb85cff9640 1 -- 172.21.15.70:0/659295486 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v243) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb86012e1d0 con 0x7fb8601432f0 2023-10-08T07:14:26.316 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:14:26.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 -- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb84006b0b0 msgr2=0x7fb84006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb84006b0b0 0x7fb84006d570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb850009a30 tx=0x7fb850008040 comp rx=0 tx=0).stop 2023-10-08T07:14:26.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 -- 172.21.15.70:0/659295486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8601432f0 msgr2=0x7fb8601e3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8601432f0 0x7fb8601e3e80 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7fb858009e50 tx=0x7fb858004a40 comp rx=0 tx=0).stop 2023-10-08T07:14:26.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 -- 172.21.15.70:0/659295486 shutdown_connections 2023-10-08T07:14:26.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb84006b0b0 0x7fb84006d570 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8601432f0 0x7fb8601e3e80 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8600cb6b0 0x7fb8601573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 --2- 172.21.15.70:0/659295486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8600b4240 0x7fb86015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 -- 172.21.15.70:0/659295486 >> 172.21.15.70:0/659295486 conn(0x7fb860069d50 msgr2=0x7fb86006c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:26.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 -- 172.21.15.70:0/659295486 shutdown_connections 2023-10-08T07:14:26.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.317+0000 7fb865f4c640 1 -- 172.21.15.70:0/659295486 wait complete. 2023-10-08T07:14:26.337 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-08T07:14:26.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.554+0000 7f33714e5640 1 Processor -- start 2023-10-08T07:14:26.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.554+0000 7f33714e5640 1 -- start start 2023-10-08T07:14:26.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.556+0000 7f33714e5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f336c0cb130 0x7f336c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.556+0000 7f33714e5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336c0b1cc0 con 0x7f336c06dfc0 2023-10-08T07:14:26.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.556+0000 7f33714e5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336c0cbad0 con 0x7f336c0afbe0 2023-10-08T07:14:26.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.556+0000 7f33714e5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336c0b2620 con 0x7f336c0cb130 2023-10-08T07:14:26.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.556+0000 7f336a7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f336c06dfc0 0x7f336c070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58044/0 (socket says 172.21.15.70:58044) 2023-10-08T07:14:26.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.556+0000 7f336a7fc640 1 -- 172.21.15.70:0/3336756800 learned_addr learned my addr 172.21.15.70:0/3336756800 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:26.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.556+0000 7f336b7fe640 1 --2- 172.21.15.70:0/3336756800 >> v2:172.21.15.181:3300/0 conn(0x7f336c0cb130 0x7f336c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.556+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447993814 0 0) 0x7f336c0b1cc0 con 0x7f336c06dfc0 2023-10-08T07:14:26.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3354003520 con 0x7f336c06dfc0 2023-10-08T07:14:26.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 94713236 0 0) 0x7f3354003520 con 0x7f336c06dfc0 2023-10-08T07:14:26.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 >> v2:172.21.15.181:3300/0 conn(0x7f336c0cb130 msgr2=0x7f336c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 --2- 172.21.15.70:0/3336756800 >> v2:172.21.15.181:3300/0 conn(0x7f336c0cb130 0x7f336c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 >> v1:172.21.15.181:6789/0 conn(0x7f336c0afbe0 legacy=0x7f336c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c0b2060 con 0x7f336c06dfc0 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f3360001670 con 0x7f336c06dfc0 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 >> v1:172.21.15.70:6789/0 conn(0x7f336c06dfc0 legacy=0x7f336c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3354003970 0x7f3354003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33540047c0 0x7f3354004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3354005180 0x7f335400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0b2620 con 0x7f3354005180 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335400c020 con 0x7f33540047c0 2023-10-08T07:14:26.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335400c1e0 con 0x7f3354003970 2023-10-08T07:14:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.557+0000 7f336b7fe640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3354005180 0x7f335400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.558+0000 7f336affd640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3354003970 0x7f3354003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.558+0000 7f336b7fe640 1 -- 172.21.15.70:0/3336756800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3354003970 msgr2=0x7f3354003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.558+0000 7f336b7fe640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3354003970 0x7f3354003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.558+0000 7f336b7fe640 1 -- 172.21.15.70:0/3336756800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33540047c0 msgr2=0x7f3354004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.558+0000 7f336b7fe640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33540047c0 0x7f3354004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.558+0000 7f336b7fe640 1 -- 172.21.15.70:0/3336756800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3354003520 con 0x7f3354005180 2023-10-08T07:14:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.558+0000 7f336affd640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3354003970 0x7f3354003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.559+0000 7f336b7fe640 1 -- 172.21.15.70:0/3336756800 reap_dead start 2023-10-08T07:14:26.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.559+0000 7f336b7fe640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3354005180 0x7f335400b980 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f3364009fc0 tx=0x7f33540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6196091a0050f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:26.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.559+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336402cc30 con 0x7f3354005180 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.559+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336400f040 con 0x7f3354005180 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.559+0000 7f3369ffb640 1 -- 172.21.15.70:0/3336756800 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364003990 con 0x7f3354005180 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.560+0000 7f33714e5640 1 -- 172.21.15.70:0/3336756800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3354005180 msgr2=0x7f335400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.560+0000 7f33714e5640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3354005180 0x7f335400b980 secure :-1 s=READY pgs=2546 cs=0 l=1 rev1=1 crypto rx=0x7f3364009fc0 tx=0x7f33540057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.560+0000 7f33714e5640 1 -- 172.21.15.70:0/3336756800 shutdown_connections 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.560+0000 7f33714e5640 1 --2- 172.21.15.70:0/3336756800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3354005180 0x7f335400b980 unknown :-1 s=CLOSED pgs=2546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.560+0000 7f33714e5640 1 -- 172.21.15.70:0/3336756800 >> 172.21.15.70:0/3336756800 conn(0x7f336c06a5d0 msgr2=0x7f336c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.560+0000 7f33714e5640 1 -- 172.21.15.70:0/3336756800 shutdown_connections 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.560+0000 7f33714e5640 1 -- 172.21.15.70:0/3336756800 wait complete. 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.561+0000 7f33714e5640 1 Processor -- start 2023-10-08T07:14:26.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.562+0000 7f33714e5640 1 -- start start 2023-10-08T07:14:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f33714e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3354005180 0x7f336c0c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f33714e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c0aea70 0x7f336c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f33714e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c159b60 0x7f336c0c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f33714e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0c2990 con 0x7f336c159b60 2023-10-08T07:14:26.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f33714e5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0c3020 con 0x7f3354005180 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f33714e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336c0c23d0 con 0x7f336c0aea70 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f336b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c159b60 0x7f336c0c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f336b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c159b60 0x7f336c0c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42948/0 (socket says 172.21.15.70:42948) 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f336b7fe640 1 -- 172.21.15.70:0/3519316619 learned_addr learned my addr 172.21.15.70:0/3519316619 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f336a7fc640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c0aea70 0x7f336c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f336b7fe640 1 -- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c0aea70 msgr2=0x7f336c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f336b7fe640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c0aea70 0x7f336c159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f336affd640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3354005180 0x7f336c0c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f336b7fe640 1 -- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3354005180 msgr2=0x7f336c0c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f336b7fe640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3354005180 0x7f336c0c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.563+0000 7f336b7fe640 1 -- 172.21.15.70:0/3519316619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336c0c26b0 con 0x7f336c159b60 2023-10-08T07:14:26.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.564+0000 7f336b7fe640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c159b60 0x7f336c0c7150 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f336402c6a0 tx=0x7f3364009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:26.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.564+0000 7f33537fe640 1 -- 172.21.15.70:0/3519316619 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336402dea0 con 0x7f336c159b60 2023-10-08T07:14:26.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.564+0000 7f33537fe640 1 -- 172.21.15.70:0/3519316619 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3364003da0 con 0x7f336c159b60 2023-10-08T07:14:26.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.564+0000 7f33537fe640 1 -- 172.21.15.70:0/3519316619 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364035b70 con 0x7f336c159b60 2023-10-08T07:14:26.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.564+0000 7f33714e5640 1 -- 172.21.15.70:0/3519316619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336c159ff0 con 0x7f336c159b60 2023-10-08T07:14:26.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.564+0000 7f33714e5640 1 -- 172.21.15.70:0/3519316619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336c0c5000 con 0x7f336c159b60 2023-10-08T07:14:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.565+0000 7f33714e5640 1 -- 172.21.15.70:0/3519316619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336c04e160 con 0x7f336c159b60 2023-10-08T07:14:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.566+0000 7f33537fe640 1 -- 172.21.15.70:0/3519316619 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3364035cd0 con 0x7f336c159b60 2023-10-08T07:14:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.567+0000 7f33537fe640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f335806b0b0 0x7f335806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.567+0000 7f33537fe640 1 -- 172.21.15.70:0/3519316619 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f33640acbb0 con 0x7f336c159b60 2023-10-08T07:14:26.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.568+0000 7f336affd640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f335806b0b0 0x7f335806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:26.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.569+0000 7f336affd640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f335806b0b0 0x7f335806d570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f335c0015b0 tx=0x7f335c009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:26.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.571+0000 7f33537fe640 1 -- 172.21.15.70:0/3519316619 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3364077310 con 0x7f336c159b60 2023-10-08T07:14:26.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.758+0000 7f33714e5640 1 -- 172.21.15.70:0/3519316619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f336c0c89d0 con 0x7f336c159b60 2023-10-08T07:14:26.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.759+0000 7f33537fe640 1 -- 172.21.15.70:0/3519316619 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v243) v1 ==== 107+0+11 (secure 0 0 0) 0x7f336407fe60 con 0x7f336c159b60 2023-10-08T07:14:26.761 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:14:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 -- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f335806b0b0 msgr2=0x7f335806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f335806b0b0 0x7f335806d570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f335c0015b0 tx=0x7f335c009040 comp rx=0 tx=0).stop 2023-10-08T07:14:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 -- 172.21.15.70:0/3519316619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c159b60 msgr2=0x7f336c0c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c159b60 0x7f336c0c7150 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f336402c6a0 tx=0x7f3364009750 comp rx=0 tx=0).stop 2023-10-08T07:14:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 -- 172.21.15.70:0/3519316619 shutdown_connections 2023-10-08T07:14:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f335806b0b0 0x7f335806d570 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c159b60 0x7f336c0c7150 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f336c0aea70 0x7f336c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 --2- 172.21.15.70:0/3519316619 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3354005180 0x7f336c0c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:26.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 -- 172.21.15.70:0/3519316619 >> 172.21.15.70:0/3519316619 conn(0x7f336c06a5d0 msgr2=0x7f336c15aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:26.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 -- 172.21.15.70:0/3519316619 shutdown_connections 2023-10-08T07:14:26.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.762+0000 7f33714e5640 1 -- 172.21.15.70:0/3519316619 wait complete. 2023-10-08T07:14:26.783 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-08T07:14:27.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.999+0000 7fcedb59e640 1 Processor -- start 2023-10-08T07:14:27.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:26.999+0000 7fcedb59e640 1 -- start start 2023-10-08T07:14:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.001+0000 7fcedb59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc0cb5e0 0x7fcedc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.001+0000 7fcedb59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcedc159db0 con 0x7fcedc1599d0 2023-10-08T07:14:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.001+0000 7fcedb59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcedc071fe0 con 0x7fcedc0b3c50 2023-10-08T07:14:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.001+0000 7fcedb59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0b44c0 con 0x7fcedc0cb5e0 2023-10-08T07:14:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.001+0000 7fceda59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcedc1599d0 0x7fcedc0caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58046/0 (socket says 172.21.15.70:58046) 2023-10-08T07:14:27.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.001+0000 7fceda59c640 1 -- 172.21.15.70:0/1866303614 learned_addr learned my addr 172.21.15.70:0/1866303614 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:27.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced3fff640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc0cb5e0 0x7fcedc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226029297 0 0) 0x7fcecc001110 con 0x7fcedc1599d0 2023-10-08T07:14:27.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcec8003520 con 0x7fcedc1599d0 2023-10-08T07:14:27.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3614705701 0 0) 0x7fcec8003520 con 0x7fcedc1599d0 2023-10-08T07:14:27.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc0cb5e0 msgr2=0x7fcedc0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc0cb5e0 0x7fcedc0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 >> v1:172.21.15.181:6789/0 conn(0x7fcedc0b3c50 legacy=0x7fcedc0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc0713d0 con 0x7fcedc1599d0 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fcecc001800 con 0x7fcedc1599d0 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 >> v1:172.21.15.70:6789/0 conn(0x7fcedc1599d0 legacy=0x7fcedc0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcec80039a0 0x7fcec8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc0cb5e0 0x7fcec80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec8004d00 0x7fcec8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0b44c0 con 0x7fcec8004d00 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec800ba30 con 0x7fcec80039a0 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.002+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec800de00 con 0x7fcedc0cb5e0 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.003+0000 7fcedad9d640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec8004d00 0x7fcec8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.003+0000 7fceda59c640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcec80039a0 0x7fcec8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.003+0000 7fcedad9d640 1 -- 172.21.15.70:0/1866303614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc0cb5e0 msgr2=0x7fcec80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.003+0000 7fcedad9d640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc0cb5e0 0x7fcec80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.003+0000 7fcedad9d640 1 -- 172.21.15.70:0/1866303614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcec80039a0 msgr2=0x7fcec8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.003+0000 7fcedad9d640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcec80039a0 0x7fcec8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.003+0000 7fcedad9d640 1 -- 172.21.15.70:0/1866303614 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec8003520 con 0x7fcec8004d00 2023-10-08T07:14:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.004+0000 7fcedad9d640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec8004d00 0x7fcec8005180 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fced4001e20 tx=0x7fcec80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef13b6fe325aa6ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.004+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4003b70 con 0x7fcec8004d00 2023-10-08T07:14:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.005+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced4003cd0 con 0x7fcec8004d00 2023-10-08T07:14:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.005+0000 7fced9d9b640 1 -- 172.21.15.70:0/1866303614 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4005310 con 0x7fcec8004d00 2023-10-08T07:14:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.005+0000 7fcedb59e640 1 -- 172.21.15.70:0/1866303614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec8004d00 msgr2=0x7fcec8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.005+0000 7fcedb59e640 1 --2- 172.21.15.70:0/1866303614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcec8004d00 0x7fcec8005180 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7fced4001e20 tx=0x7fcec80057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.005+0000 7fcedad9d640 1 -- 172.21.15.70:0/1866303614 reap_dead start 2023-10-08T07:14:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.005+0000 7fcedb59e640 1 -- 172.21.15.70:0/1866303614 shutdown_connections 2023-10-08T07:14:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.005+0000 7fcedb59e640 1 -- 172.21.15.70:0/1866303614 >> 172.21.15.70:0/1866303614 conn(0x7fcedc06c050 msgr2=0x7fcedc06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.005+0000 7fcedb59e640 1 -- 172.21.15.70:0/1866303614 shutdown_connections 2023-10-08T07:14:27.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.005+0000 7fcedb59e640 1 -- 172.21.15.70:0/1866303614 wait complete. 2023-10-08T07:14:27.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.007+0000 7fcedb59e640 1 Processor -- start 2023-10-08T07:14:27.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.007+0000 7fcedb59e640 1 -- start start 2023-10-08T07:14:27.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.008+0000 7fcedb59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcec8004d00 0x7fcedc14d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.008+0000 7fcedb59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc153b50 0x7fcedc1531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.008+0000 7fcedb59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc150850 0x7fcedc150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.008+0000 7fcedb59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc151210 con 0x7fcedc150850 2023-10-08T07:14:27.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.008+0000 7fcedb59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc14dec0 con 0x7fcec8004d00 2023-10-08T07:14:27.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.008+0000 7fcedb59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0c7140 con 0x7fcedc153b50 2023-10-08T07:14:27.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fcedad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc150850 0x7fcedc150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fcedad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc150850 0x7fcedc150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42960/0 (socket says 172.21.15.70:42960) 2023-10-08T07:14:27.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fcedad9d640 1 -- 172.21.15.70:0/3752417873 learned_addr learned my addr 172.21.15.70:0/3752417873 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:27.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fceda59c640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcec8004d00 0x7fcedc14d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fced3fff640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc153b50 0x7fcedc1531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fcedad9d640 1 -- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc153b50 msgr2=0x7fcedc1531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fcedad9d640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc153b50 0x7fcedc1531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fcedad9d640 1 -- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcec8004d00 msgr2=0x7fcedc14d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fcedad9d640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcec8004d00 0x7fcedc14d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fcedad9d640 1 -- 172.21.15.70:0/3752417873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc0c73e0 con 0x7fcedc150850 2023-10-08T07:14:27.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fced3fff640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc153b50 0x7fcedc1531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:27.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.009+0000 7fcedad9d640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc150850 0x7fcedc150cd0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fced402c040 tx=0x7fced40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:27.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.010+0000 7fced2ffd640 1 -- 172.21.15.70:0/3752417873 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4006d50 con 0x7fcedc150850 2023-10-08T07:14:27.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.010+0000 7fced2ffd640 1 -- 172.21.15.70:0/3752417873 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced4003e40 con 0x7fcedc150850 2023-10-08T07:14:27.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.010+0000 7fced2ffd640 1 -- 172.21.15.70:0/3752417873 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4007270 con 0x7fcedc150850 2023-10-08T07:14:27.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.010+0000 7fcedb59e640 1 -- 172.21.15.70:0/3752417873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc0c76c0 con 0x7fcedc150850 2023-10-08T07:14:27.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.010+0000 7fcedb59e640 1 -- 172.21.15.70:0/3752417873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc0c7c80 con 0x7fcedc150850 2023-10-08T07:14:27.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.011+0000 7fcedb59e640 1 -- 172.21.15.70:0/3752417873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc0b3910 con 0x7fcedc150850 2023-10-08T07:14:27.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.012+0000 7fced2ffd640 1 -- 172.21.15.70:0/3752417873 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fced40073d0 con 0x7fcedc150850 2023-10-08T07:14:27.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.013+0000 7fced2ffd640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcebc06b0b0 0x7fcebc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.013+0000 7fced2ffd640 1 -- 172.21.15.70:0/3752417873 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fced40ab780 con 0x7fcedc150850 2023-10-08T07:14:27.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.014+0000 7fceda59c640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcebc06b0b0 0x7fcebc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.015+0000 7fceda59c640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcebc06b0b0 0x7fcebc06d570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcecc001e20 tx=0x7fcecc009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:27.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.017+0000 7fced2ffd640 1 -- 172.21.15.70:0/3752417873 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fced4075ef0 con 0x7fcedc150850 2023-10-08T07:14:27.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.206+0000 7fcedb59e640 1 -- 172.21.15.70:0/3752417873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fcedc151210 con 0x7fcedc150850 2023-10-08T07:14:27.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.207+0000 7fced2ffd640 1 -- 172.21.15.70:0/3752417873 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v243) v1 ==== 103+0+11 (secure 0 0 0) 0x7fced4079510 con 0x7fcedc150850 2023-10-08T07:14:27.209 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:14:27.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 -- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcebc06b0b0 msgr2=0x7fcebc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcebc06b0b0 0x7fcebc06d570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcecc001e20 tx=0x7fcecc009040 comp rx=0 tx=0).stop 2023-10-08T07:14:27.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 -- 172.21.15.70:0/3752417873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc150850 msgr2=0x7fcedc150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc150850 0x7fcedc150cd0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fced402c040 tx=0x7fced40014a0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 -- 172.21.15.70:0/3752417873 shutdown_connections 2023-10-08T07:14:27.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcebc06b0b0 0x7fcebc06d570 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc150850 0x7fcedc150cd0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcedc153b50 0x7fcedc1531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 --2- 172.21.15.70:0/3752417873 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcec8004d00 0x7fcedc14d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 -- 172.21.15.70:0/3752417873 >> 172.21.15.70:0/3752417873 conn(0x7fcedc06c050 msgr2=0x7fcedc158c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:27.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 -- 172.21.15.70:0/3752417873 shutdown_connections 2023-10-08T07:14:27.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.210+0000 7fcedb59e640 1 -- 172.21.15.70:0/3752417873 wait complete. 2023-10-08T07:14:27.230 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:14:27.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.433+0000 7ff404275640 1 Processor -- start 2023-10-08T07:14:27.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.433+0000 7ff404275640 1 -- start start 2023-10-08T07:14:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.435+0000 7ff404275640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc15ab40 0x7ff3fc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.435+0000 7ff404275640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff3fc0cb580 0x7ff3fc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.435+0000 7ff404275640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3fc15b2b0 con 0x7ff3fc0c90b0 2023-10-08T07:14:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.435+0000 7ff404275640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3fc15bf30 con 0x7ff3fc0cb580 2023-10-08T07:14:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.435+0000 7ff404275640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc15f7a0 con 0x7ff3fc15ab40 2023-10-08T07:14:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.435+0000 7ff401810640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff3fc0c90b0 0x7ff3fc0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58052/0 (socket says 172.21.15.70:58052) 2023-10-08T07:14:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.435+0000 7ff401810640 1 -- 172.21.15.70:0/3502384694 learned_addr learned my addr 172.21.15.70:0/3502384694 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff402011640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc15ab40 0x7ff3fc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266454607 0 0) 0x7ff3fc15b2b0 con 0x7ff3fc0c90b0 2023-10-08T07:14:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e8003520 con 0x7ff3fc0c90b0 2023-10-08T07:14:27.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff402812640 1 --2- 172.21.15.70:0/3502384694 >> v2:172.21.15.181:3300/0 conn(0x7ff3fc0cb580 0x7ff3fc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1349110022 0 0) 0x7ff3e8003520 con 0x7ff3fc0c90b0 2023-10-08T07:14:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc15ab40 msgr2=0x7ff3fc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc15ab40 0x7ff3fc0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 >> v2:172.21.15.181:3300/0 conn(0x7ff3fc0cb580 msgr2=0x7ff3fc0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 --2- 172.21.15.70:0/3502384694 >> v2:172.21.15.181:3300/0 conn(0x7ff3fc0cb580 0x7ff3fc0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc15b590 con 0x7ff3fc0c90b0 2023-10-08T07:14:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff3ec001670 con 0x7ff3fc0c90b0 2023-10-08T07:14:27.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 >> v1:172.21.15.70:6789/0 conn(0x7ff3fc0c90b0 legacy=0x7ff3fc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8003970 0x7ff3e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80047c0 0x7ff3e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc15ab40 0x7ff3e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc15bf30 con 0x7ff3e80047c0 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc15f7a0 con 0x7ff3e8003970 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.436+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e800ba30 con 0x7ff3fc15ab40 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.437+0000 7ff401810640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80047c0 0x7ff3e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.437+0000 7ff402011640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8003970 0x7ff3e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.437+0000 7ff402812640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc15ab40 0x7ff3e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.437+0000 7ff401810640 1 -- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc15ab40 msgr2=0x7ff3e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.437+0000 7ff401810640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc15ab40 0x7ff3e8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.437+0000 7ff401810640 1 -- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8003970 msgr2=0x7ff3e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.437+0000 7ff401810640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8003970 0x7ff3e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.437+0000 7ff401810640 1 -- 172.21.15.70:0/3502384694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e8003520 con 0x7ff3e80047c0 2023-10-08T07:14:27.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.437+0000 7ff402011640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e8003970 0x7ff3e8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:27.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.438+0000 7ff401810640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80047c0 0x7ff3e8004c40 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00c210 tx=0x7ff3e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7334c975d05b5c87 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:27.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.438+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec005ad0 con 0x7ff3e80047c0 2023-10-08T07:14:27.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.438+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec005c30 con 0x7ff3e80047c0 2023-10-08T07:14:27.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.438+0000 7ff40100f640 1 -- 172.21.15.70:0/3502384694 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec02c8b0 con 0x7ff3e80047c0 2023-10-08T07:14:27.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.439+0000 7ff404275640 1 -- 172.21.15.70:0/3502384694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80047c0 msgr2=0x7ff3e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.439+0000 7ff404275640 1 --2- 172.21.15.70:0/3502384694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3e80047c0 0x7ff3e8004c40 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00c210 tx=0x7ff3e80057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.439+0000 7ff402812640 1 -- 172.21.15.70:0/3502384694 reap_dead start 2023-10-08T07:14:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.439+0000 7ff404275640 1 -- 172.21.15.70:0/3502384694 shutdown_connections 2023-10-08T07:14:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.439+0000 7ff404275640 1 -- 172.21.15.70:0/3502384694 >> 172.21.15.70:0/3502384694 conn(0x7ff3fc06a380 msgr2=0x7ff3fc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.439+0000 7ff404275640 1 -- 172.21.15.70:0/3502384694 shutdown_connections 2023-10-08T07:14:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.439+0000 7ff404275640 1 -- 172.21.15.70:0/3502384694 wait complete. 2023-10-08T07:14:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.441+0000 7ff404275640 1 Processor -- start 2023-10-08T07:14:27.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.441+0000 7ff404275640 1 -- start start 2023-10-08T07:14:27.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.442+0000 7ff404275640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e80047c0 0x7ff3fc153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.442+0000 7ff404275640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc0c7f40 0x7ff3fc0c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.442+0000 7ff404275640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3fc0c5b80 0x7ff3fc0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.442+0000 7ff404275640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc15dc20 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.442+0000 7ff404275640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0c6540 con 0x7ff3e80047c0 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.442+0000 7ff404275640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc15cd50 con 0x7ff3fc0c7f40 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.442+0000 7ff402812640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3fc0c5b80 0x7ff3fc0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.442+0000 7ff402812640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3fc0c5b80 0x7ff3fc0c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42974/0 (socket says 172.21.15.70:42974) 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.442+0000 7ff402812640 1 -- 172.21.15.70:0/2396549413 learned_addr learned my addr 172.21.15.70:0/2396549413 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff402011640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e80047c0 0x7ff3fc153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff402812640 1 -- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc0c7f40 msgr2=0x7ff3fc0c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff402812640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc0c7f40 0x7ff3fc0c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff402812640 1 -- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e80047c0 msgr2=0x7ff3fc153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff402812640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e80047c0 0x7ff3fc153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff402812640 1 -- 172.21.15.70:0/2396549413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc1f21f0 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff402812640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3fc0c5b80 0x7ff3fc0c6000 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0c8d80 tx=0x7ff3f4009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:27.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff3f2ffd640 1 -- 172.21.15.70:0/2396549413 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4007cf0 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff3f2ffd640 1 -- 172.21.15.70:0/2396549413 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f4007e70 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff3f2ffd640 1 -- 172.21.15.70:0/2396549413 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4014620 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.443+0000 7ff404275640 1 -- 172.21.15.70:0/2396549413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc15cfa0 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.444+0000 7ff404275640 1 -- 172.21.15.70:0/2396549413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc15d560 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.445+0000 7ff404275640 1 -- 172.21.15.70:0/2396549413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3fc067490 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.448+0000 7ff3f2ffd640 1 -- 172.21.15.70:0/2396549413 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff3f401d3b0 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.448+0000 7ff3f2ffd640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff3e006b0b0 0x7ff3e006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.449+0000 7ff3f2ffd640 1 -- 172.21.15.70:0/2396549413 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7060+0+0 (secure 0 0 0) 0x7ff3f408a0b0 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.449+0000 7ff402011640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff3e006b0b0 0x7ff3e006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.450+0000 7ff402011640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff3e006b0b0 0x7ff3e006d570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8006fd0 tx=0x7ff3f8008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:27.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.451+0000 7ff3f2ffd640 1 -- 172.21.15.70:0/2396549413 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3f4054960 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.661+0000 7ff404275640 1 -- 172.21.15.70:0/2396549413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3fc0cf310 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.662+0000 7ff3f2ffd640 1 -- 172.21.15.70:0/2396549413 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4933 (secure 0 0 0) 0x7ff3f4019090 con 0x7ff3fc0c5b80 2023-10-08T07:14:27.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:27.664 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":116,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":11730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2038500071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2038500071},{"type":"v1","addr":"172.21.15.70:6834","nonce":2038500071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:48.829441+0000","modified":"2023-10-08T07:13:51.877323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8416":{"gid":8416,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-08T07:14:27.664 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 116 2023-10-08T07:14:27.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 -- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff3e006b0b0 msgr2=0x7ff3e006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff3e006b0b0 0x7ff3e006d570 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8006fd0 tx=0x7ff3f8008040 comp rx=0 tx=0).stop 2023-10-08T07:14:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 -- 172.21.15.70:0/2396549413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3fc0c5b80 msgr2=0x7ff3fc0c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3fc0c5b80 0x7ff3fc0c6000 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0c8d80 tx=0x7ff3f4009cb0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 -- 172.21.15.70:0/2396549413 shutdown_connections 2023-10-08T07:14:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff3e006b0b0 0x7ff3e006d570 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3fc0c5b80 0x7ff3fc0c6000 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff3fc0c7f40 0x7ff3fc0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 --2- 172.21.15.70:0/2396549413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff3e80047c0 0x7ff3fc153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 -- 172.21.15.70:0/2396549413 >> 172.21.15.70:0/2396549413 conn(0x7ff3fc06a380 msgr2=0x7ff3fc0ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:27.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.664+0000 7ff404275640 1 -- 172.21.15.70:0/2396549413 shutdown_connections 2023-10-08T07:14:27.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.665+0000 7ff404275640 1 -- 172.21.15.70:0/2396549413 wait complete. 2023-10-08T07:14:27.684 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:27.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.888+0000 7fc9dc8c3640 1 Processor -- start 2023-10-08T07:14:27.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.888+0000 7fc9dc8c3640 1 -- start start 2023-10-08T07:14:27.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.890+0000 7fc9dc8c3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80cf720 0x7fc9d806b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.890+0000 7fc9dc8c3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc9d806bd90 0x7fc9d80cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.890+0000 7fc9dc8c3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d80cb590 con 0x7fc9d80cd8a0 2023-10-08T07:14:27.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.890+0000 7fc9dc8c3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d806c220 con 0x7fc9d806bd90 2023-10-08T07:14:27.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.890+0000 7fc9dc8c3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80cfc90 con 0x7fc9d80cf720 2023-10-08T07:14:27.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.890+0000 7fc9d77fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc9d80cd8a0 0x7fc9d806b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58068/0 (socket says 172.21.15.70:58068) 2023-10-08T07:14:27.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.890+0000 7fc9d77fe640 1 -- 172.21.15.70:0/2153929782 learned_addr learned my addr 172.21.15.70:0/2153929782 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:27.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349948998 0 0) 0x7fc9d80cb590 con 0x7fc9d80cd8a0 2023-10-08T07:14:27.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9c4003520 con 0x7fc9d80cd8a0 2023-10-08T07:14:27.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d7fff640 1 --2- 172.21.15.70:0/2153929782 >> v2:172.21.15.181:3300/0 conn(0x7fc9d806bd90 0x7fc9d80cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9cffff640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80cf720 0x7fc9d806b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 878909037 0 0) 0x7fc9c4003520 con 0x7fc9d80cd8a0 2023-10-08T07:14:27.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80cf720 msgr2=0x7fc9d806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80cf720 0x7fc9d806b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 >> v2:172.21.15.181:3300/0 conn(0x7fc9d806bd90 msgr2=0x7fc9d80cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 --2- 172.21.15.70:0/2153929782 >> v2:172.21.15.181:3300/0 conn(0x7fc9d806bd90 0x7fc9d80cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d80cff70 con 0x7fc9d80cd8a0 2023-10-08T07:14:27.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc9c8001670 con 0x7fc9d80cd8a0 2023-10-08T07:14:27.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 >> v1:172.21.15.70:6789/0 conn(0x7fc9d80cd8a0 legacy=0x7fc9d806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9c4003970 0x7fc9c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80cf720 0x7fc9c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9c4004d00 0x7fc9c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d806c220 con 0x7fc9c4004d00 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80cfc90 con 0x7fc9c4003970 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.891+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c400bab0 con 0x7fc9d80cf720 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.892+0000 7fc9d7fff640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9c4004d00 0x7fc9c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.892+0000 7fc9d77fe640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9c4003970 0x7fc9c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.892+0000 7fc9d7fff640 1 -- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80cf720 msgr2=0x7fc9c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.892+0000 7fc9cffff640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80cf720 0x7fc9c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.892+0000 7fc9d7fff640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80cf720 0x7fc9c40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.892+0000 7fc9d7fff640 1 -- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9c4003970 msgr2=0x7fc9c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:27.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.892+0000 7fc9d7fff640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9c4003970 0x7fc9c4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.892+0000 7fc9d7fff640 1 -- 172.21.15.70:0/2153929782 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c4003520 con 0x7fc9c4004d00 2023-10-08T07:14:27.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.893+0000 7fc9d7fff640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9c4004d00 0x7fc9c4005180 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0007360 tx=0x7fc9c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29792812fcc2743 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:27.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.893+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d002ccf0 con 0x7fc9c4004d00 2023-10-08T07:14:27.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.893+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d002ce50 con 0x7fc9c4004d00 2023-10-08T07:14:27.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.893+0000 7fc9d6ffd640 1 -- 172.21.15.70:0/2153929782 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0034660 con 0x7fc9c4004d00 2023-10-08T07:14:27.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.894+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/2153929782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9c4004d00 msgr2=0x7fc9c4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.894+0000 7fc9dc8c3640 1 --2- 172.21.15.70:0/2153929782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9c4004d00 0x7fc9c4005180 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0007360 tx=0x7fc9c40057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.894+0000 7fc9d7fff640 1 -- 172.21.15.70:0/2153929782 reap_dead start 2023-10-08T07:14:27.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.894+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/2153929782 shutdown_connections 2023-10-08T07:14:27.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.894+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/2153929782 >> 172.21.15.70:0/2153929782 conn(0x7fc9d8099b60 msgr2=0x7fc9d80d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:27.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.894+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/2153929782 shutdown_connections 2023-10-08T07:14:27.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.894+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/2153929782 wait complete. 2023-10-08T07:14:27.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.896+0000 7fc9dc8c3640 1 Processor -- start 2023-10-08T07:14:27.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.896+0000 7fc9dc8c3640 1 -- start start 2023-10-08T07:14:27.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.897+0000 7fc9dc8c3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9c4004d00 0x7fc9d8153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.897+0000 7fc9dc8c3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9d80b0f10 0x7fc9d81f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.897+0000 7fc9dc8c3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80b51e0 0x7fc9d80b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.897+0000 7fc9dc8c3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d81f6b70 con 0x7fc9d80b0f10 2023-10-08T07:14:27.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.897+0000 7fc9dc8c3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80b4140 con 0x7fc9c4004d00 2023-10-08T07:14:27.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.897+0000 7fc9dc8c3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80b4420 con 0x7fc9d80b51e0 2023-10-08T07:14:27.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.897+0000 7fc9cffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9d80b0f10 0x7fc9d81f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.897+0000 7fc9cffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9d80b0f10 0x7fc9d81f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42986/0 (socket says 172.21.15.70:42986) 2023-10-08T07:14:27.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.897+0000 7fc9cffff640 1 -- 172.21.15.70:0/3536415334 learned_addr learned my addr 172.21.15.70:0/3536415334 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:27.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9cffff640 1 -- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80b51e0 msgr2=0x7fc9d80b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9cffff640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80b51e0 0x7fc9d80b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9cffff640 1 -- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9c4004d00 msgr2=0x7fc9d8153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:27.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9cffff640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9c4004d00 0x7fc9d8153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:27.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9cffff640 1 -- 172.21.15.70:0/3536415334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d80b4700 con 0x7fc9d80b0f10 2023-10-08T07:14:27.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9cffff640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9d80b0f10 0x7fc9d81f6630 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e7e0 tx=0x7fc9c000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:27.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9d4ff9640 1 -- 172.21.15.70:0/3536415334 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000c650 con 0x7fc9d80b0f10 2023-10-08T07:14:27.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9d4ff9640 1 -- 172.21.15.70:0/3536415334 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c000c7b0 con 0x7fc9d80b0f10 2023-10-08T07:14:27.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/3536415334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d80b49e0 con 0x7fc9d80b0f10 2023-10-08T07:14:27.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9d4ff9640 1 -- 172.21.15.70:0/3536415334 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000cb20 con 0x7fc9d80b0f10 2023-10-08T07:14:27.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.898+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/3536415334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d814d9d0 con 0x7fc9d80b0f10 2023-10-08T07:14:27.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.899+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/3536415334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d804e160 con 0x7fc9d80b0f10 2023-10-08T07:14:27.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.900+0000 7fc9d4ff9640 1 -- 172.21.15.70:0/3536415334 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc9c000cc80 con 0x7fc9d80b0f10 2023-10-08T07:14:27.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.902+0000 7fc9d4ff9640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc9b806b0e0 0x7fc9b806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:27.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.902+0000 7fc9d4ff9640 1 -- 172.21.15.70:0/3536415334 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fc9c0089d20 con 0x7fc9d80b0f10 2023-10-08T07:14:27.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.903+0000 7fc9d77fe640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc9b806b0e0 0x7fc9b806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:27.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.904+0000 7fc9d77fe640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc9b806b0e0 0x7fc9b806d5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8001eb0 tx=0x7fc9c8000f50 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:27.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:27.906+0000 7fc9d4ff9640 1 -- 172.21.15.70:0/3536415334 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9c00544d0 con 0x7fc9d80b0f10 2023-10-08T07:14:28.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.095+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/3536415334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9d80cded0 con 0x7fc9d80b0f10 2023-10-08T07:14:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.099+0000 7fc9d4ff9640 1 -- 172.21.15.70:0/3536415334 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v243) v1 ==== 74+0+18232 (secure 0 0 0) 0x7fc9c0057af0 con 0x7fc9d80b0f10 2023-10-08T07:14:28.100 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:28.100 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":243,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:25.425545+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:13:47.007437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:13:47.838023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2480387829":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/920349426":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/2459293689":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/4129076201":"2023-10-09T07:14:16.815350+0000","172.21.15.70:6834/180995576":"2023-10-09T07:14:22.405699+0000","172.21.15.181:6832/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/365280967":"2023-10-09T07:14:16.815350+0000","172.21.15.70:6832/180995576":"2023-10-09T07:14:22.405699+0000","172.21.15.181:0/2833418931":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/753176203":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/317173509":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:6832/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1018014196":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1778827440":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1217032325":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/2798662747":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/4139611391":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/3002223432":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/3074626816":"2023-10-09T07:14:06.545661+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.101+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc9b806b0e0 msgr2=0x7fc9b806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.101+0000 7fc9dc8c3640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc9b806b0e0 0x7fc9b806d5a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8001eb0 tx=0x7fc9c8000f50 comp rx=0 tx=0).stop 2023-10-08T07:14:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.101+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/3536415334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9d80b0f10 msgr2=0x7fc9d81f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.101+0000 7fc9dc8c3640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9d80b0f10 0x7fc9d81f6630 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e7e0 tx=0x7fc9c000eca0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.101+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/3536415334 shutdown_connections 2023-10-08T07:14:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.101+0000 7fc9dc8c3640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc9b806b0e0 0x7fc9b806d5a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.101+0000 7fc9dc8c3640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9d80b51e0 0x7fc9d80b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.101+0000 7fc9dc8c3640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9d80b0f10 0x7fc9d81f6630 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.101+0000 7fc9dc8c3640 1 --2- 172.21.15.70:0/3536415334 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9c4004d00 0x7fc9d8153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.101+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/3536415334 >> 172.21.15.70:0/3536415334 conn(0x7fc9d8099b60 msgr2=0x7fc9d80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.102+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/3536415334 shutdown_connections 2023-10-08T07:14:28.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.102+0000 7fc9dc8c3640 1 -- 172.21.15.70:0/3536415334 wait complete. 2023-10-08T07:14:28.121 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-10-08T07:14:28.121 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:14:28.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.323+0000 7f0cd14f2640 1 Processor -- start 2023-10-08T07:14:28.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.323+0000 7f0cd14f2640 1 -- start start 2023-10-08T07:14:28.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.324+0000 7f0cd14f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0cb580 0x7f0ccc165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.325+0000 7f0cd14f2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ccc15b2b0 con 0x7f0ccc15ab40 2023-10-08T07:14:28.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.325+0000 7f0cd14f2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ccc15b590 con 0x7f0ccc0c90b0 2023-10-08T07:14:28.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.325+0000 7f0cd14f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc15bf30 con 0x7f0ccc0cb580 2023-10-08T07:14:28.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.325+0000 7f0ccaffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0ccc15ab40 0x7f0ccc0ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58082/0 (socket says 172.21.15.70:58082) 2023-10-08T07:14:28.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.325+0000 7f0ccaffd640 1 -- 172.21.15.70:0/2026035261 learned_addr learned my addr 172.21.15.70:0/2026035261 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.325+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138970476 0 0) 0x7f0ccc15b2b0 con 0x7f0ccc15ab40 2023-10-08T07:14:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.325+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cb8003520 con 0x7f0ccc15ab40 2023-10-08T07:14:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.325+0000 7f0ccb7fe640 1 --2- 172.21.15.70:0/2026035261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0cb580 0x7f0ccc165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 544886418 0 0) 0x7f0cb8003520 con 0x7f0ccc15ab40 2023-10-08T07:14:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0cb580 msgr2=0x7f0ccc165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 --2- 172.21.15.70:0/2026035261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0cb580 0x7f0ccc165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 >> v1:172.21.15.181:6789/0 conn(0x7f0ccc0c90b0 legacy=0x7f0ccc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc15b870 con 0x7f0ccc15ab40 2023-10-08T07:14:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0cbc001690 con 0x7f0ccc15ab40 2023-10-08T07:14:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 >> v1:172.21.15.70:6789/0 conn(0x7f0ccc15ab40 legacy=0x7f0ccc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 --2- 172.21.15.70:0/2026035261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cb80039a0 0x7f0cb8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 --2- 172.21.15.70:0/2026035261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb80047c0 0x7f0cb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 --2- 172.21.15.70:0/2026035261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0cb580 0x7f0cb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc15bf30 con 0x7f0cb80039a0 2023-10-08T07:14:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb800bb10 con 0x7f0cb80047c0 2023-10-08T07:14:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb800bcd0 con 0x7f0ccc0cb580 2023-10-08T07:14:28.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0ccaffd640 1 --2- 172.21.15.70:0/2026035261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cb80039a0 0x7f0cb8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0ccaffd640 1 -- 172.21.15.70:0/2026035261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0cb580 msgr2=0x7f0cb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0ccaffd640 1 --2- 172.21.15.70:0/2026035261 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0cb580 0x7f0cb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0ccaffd640 1 -- 172.21.15.70:0/2026035261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb80047c0 msgr2=0x7f0cb8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0ccaffd640 1 --2- 172.21.15.70:0/2026035261 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb80047c0 0x7f0cb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.326+0000 7f0ccaffd640 1 -- 172.21.15.70:0/2026035261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb8003520 con 0x7f0cb80039a0 2023-10-08T07:14:28.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.327+0000 7f0ccaffd640 1 --2- 172.21.15.70:0/2026035261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cb80039a0 0x7f0cb8003dc0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc06e240 tx=0x7f0cb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b131107355b039 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:28.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.328+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc005ad0 con 0x7f0cb80039a0 2023-10-08T07:14:28.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.328+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc005c50 con 0x7f0cb80039a0 2023-10-08T07:14:28.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.328+0000 7f0cca7fc640 1 -- 172.21.15.70:0/2026035261 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc02c8a0 con 0x7f0cb80039a0 2023-10-08T07:14:28.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.328+0000 7f0cd14f2640 1 -- 172.21.15.70:0/2026035261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cb80039a0 msgr2=0x7f0cb8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.328+0000 7f0cd14f2640 1 --2- 172.21.15.70:0/2026035261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cb80039a0 0x7f0cb8003dc0 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc06e240 tx=0x7f0cb80057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.328+0000 7f0ccb7fe640 1 -- 172.21.15.70:0/2026035261 reap_dead start 2023-10-08T07:14:28.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.328+0000 7f0cd14f2640 1 -- 172.21.15.70:0/2026035261 shutdown_connections 2023-10-08T07:14:28.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.328+0000 7f0cd14f2640 1 -- 172.21.15.70:0/2026035261 >> 172.21.15.70:0/2026035261 conn(0x7f0ccc06a380 msgr2=0x7f0ccc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:28.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.328+0000 7f0cd14f2640 1 -- 172.21.15.70:0/2026035261 shutdown_connections 2023-10-08T07:14:28.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.328+0000 7f0cd14f2640 1 -- 172.21.15.70:0/2026035261 wait complete. 2023-10-08T07:14:28.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.330+0000 7f0cd14f2640 1 Processor -- start 2023-10-08T07:14:28.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.330+0000 7f0cd14f2640 1 -- start start 2023-10-08T07:14:28.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.331+0000 7f0cd14f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb80039a0 0x7f0ccc1437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.331+0000 7f0cd14f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ccc15a060 0x7f0ccc1469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.331+0000 7f0cd14f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0c9530 0x7f0ccc0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.331+0000 7f0cd14f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc15f7a0 con 0x7f0ccc15a060 2023-10-08T07:14:28.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.331+0000 7f0cd14f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc144370 con 0x7f0cb80039a0 2023-10-08T07:14:28.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.331+0000 7f0cd14f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc143cf0 con 0x7f0ccc0c9530 2023-10-08T07:14:28.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ccc15a060 0x7f0ccc1469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:28.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ccc15a060 0x7f0ccc1469b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43016/0 (socket says 172.21.15.70:43016) 2023-10-08T07:14:28.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc27fc640 1 -- 172.21.15.70:0/4031183264 learned_addr learned my addr 172.21.15.70:0/4031183264 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:28.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0ccaffd640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb80039a0 0x7f0ccc1437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:28.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc27fc640 1 -- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0c9530 msgr2=0x7f0ccc0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc27fc640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0c9530 0x7f0ccc0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc27fc640 1 -- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb80039a0 msgr2=0x7f0ccc1437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc27fc640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb80039a0 0x7f0ccc1437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc27fc640 1 -- 172.21.15.70:0/4031183264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc144000 con 0x7f0ccc15a060 2023-10-08T07:14:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc27fc640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ccc15a060 0x7f0ccc1469b0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400c8a0 tx=0x7f0cb400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc3fff640 1 -- 172.21.15.70:0/4031183264 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4003b70 con 0x7f0ccc15a060 2023-10-08T07:14:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.332+0000 7f0cc3fff640 1 -- 172.21.15.70:0/4031183264 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb4003cf0 con 0x7f0ccc15a060 2023-10-08T07:14:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.333+0000 7f0cc3fff640 1 -- 172.21.15.70:0/4031183264 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb400aa90 con 0x7f0ccc15a060 2023-10-08T07:14:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.333+0000 7f0cd14f2640 1 -- 172.21.15.70:0/4031183264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ccc146ef0 con 0x7f0ccc15a060 2023-10-08T07:14:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.333+0000 7f0cd14f2640 1 -- 172.21.15.70:0/4031183264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ccc158520 con 0x7f0ccc15a060 2023-10-08T07:14:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.334+0000 7f0cd14f2640 1 -- 172.21.15.70:0/4031183264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ccc1444b0 con 0x7f0ccc15a060 2023-10-08T07:14:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.335+0000 7f0cc3fff640 1 -- 172.21.15.70:0/4031183264 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f0cb400abf0 con 0x7f0ccc15a060 2023-10-08T07:14:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.336+0000 7f0cc3fff640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0cac06b0b0 0x7f0cac06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.336+0000 7f0cc3fff640 1 -- 172.21.15.70:0/4031183264 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f0cb4089e30 con 0x7f0ccc15a060 2023-10-08T07:14:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.336+0000 7f0ccaffd640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0cac06b0b0 0x7f0cac06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.337+0000 7f0ccaffd640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0cac06b0b0 0x7f0cac06d570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc034000 tx=0x7f0cbc037040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:28.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.340+0000 7f0cc3fff640 1 -- 172.21.15.70:0/4031183264 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ccc1444b0 con 0x7f0ccc15a060 2023-10-08T07:14:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.570+0000 7f0cd14f2640 1 -- 172.21.15.70:0/4031183264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ccc0c9ef0 con 0x7f0ccc15a060 2023-10-08T07:14:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.570+0000 7f0cc3fff640 1 -- 172.21.15.70:0/4031183264 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f0cb4003e60 con 0x7f0ccc15a060 2023-10-08T07:14:28.571 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:28.572 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":116,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":11730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2038500071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2038500071},{"type":"v1","addr":"172.21.15.70:6834","nonce":2038500071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:48.829441+0000","modified":"2023-10-08T07:13:51.877323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8416":{"gid":8416,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-08T07:14:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 116 2023-10-08T07:14:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 -- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0cac06b0b0 msgr2=0x7f0cac06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0cac06b0b0 0x7f0cac06d570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc034000 tx=0x7f0cbc037040 comp rx=0 tx=0).stop 2023-10-08T07:14:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 -- 172.21.15.70:0/4031183264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ccc15a060 msgr2=0x7f0ccc1469b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ccc15a060 0x7f0ccc1469b0 secure :-1 s=READY pgs=2559 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400c8a0 tx=0x7f0cb400cd60 comp rx=0 tx=0).stop 2023-10-08T07:14:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 -- 172.21.15.70:0/4031183264 shutdown_connections 2023-10-08T07:14:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f0cac06b0b0 0x7f0cac06d570 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0ccc0c9530 0x7f0ccc0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ccc15a060 0x7f0ccc1469b0 unknown :-1 s=CLOSED pgs=2559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 --2- 172.21.15.70:0/4031183264 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0cb80039a0 0x7f0ccc1437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 -- 172.21.15.70:0/4031183264 >> 172.21.15.70:0/4031183264 conn(0x7f0ccc06a380 msgr2=0x7f0ccc153550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 -- 172.21.15.70:0/4031183264 shutdown_connections 2023-10-08T07:14:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.573+0000 7f0cd14f2640 1 -- 172.21.15.70:0/4031183264 wait complete. 2023-10-08T07:14:28.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:14:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.809+0000 7f385fc32640 1 Processor -- start 2023-10-08T07:14:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.809+0000 7f385fc32640 1 -- start start 2023-10-08T07:14:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.811+0000 7f385fc32640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3858159ac0 0x7f385806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.811+0000 7f385fc32640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f38580ad790 0x7f38580702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.811+0000 7f385fc32640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385806dee0 con 0x7f385815e330 2023-10-08T07:14:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.811+0000 7f385fc32640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385815d850 con 0x7f3858159ac0 2023-10-08T07:14:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.811+0000 7f385fc32640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385815db30 con 0x7f38580ad790 2023-10-08T07:14:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.811+0000 7f385d9ce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3858159ac0 0x7f385806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.811+0000 7f385d9ce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3858159ac0 0x7f385806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43022/0 (socket says 172.21.15.70:43022) 2023-10-08T07:14:28.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.811+0000 7f385d9ce640 1 -- 172.21.15.70:0/54099782 learned_addr learned my addr 172.21.15.70:0/54099782 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:28.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.812+0000 7f385d1cd640 1 --2- 172.21.15.70:0/54099782 >> v2:172.21.15.181:3300/0 conn(0x7f38580ad790 0x7f38580702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:28.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.812+0000 7f385d9ce640 1 -- 172.21.15.70:0/54099782 >> v2:172.21.15.181:3300/0 conn(0x7f38580ad790 msgr2=0x7f38580702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:28.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.812+0000 7f385d9ce640 1 --2- 172.21.15.70:0/54099782 >> v2:172.21.15.181:3300/0 conn(0x7f38580ad790 0x7f38580702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.812+0000 7f385d9ce640 1 -- 172.21.15.70:0/54099782 >> v1:172.21.15.181:6789/0 conn(0x7f385815e330 legacy=0x7f385806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:28.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.812+0000 7f385d9ce640 1 -- 172.21.15.70:0/54099782 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385815de10 con 0x7f3858159ac0 2023-10-08T07:14:28.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.813+0000 7f385d9ce640 1 --2- 172.21.15.70:0/54099782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3858159ac0 0x7f385806d5e0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f38580b4970 tx=0x7f385402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf6362b7ce7b046 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.813+0000 7f385c9cc640 1 -- 172.21.15.70:0/54099782 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854031870 con 0x7f3858159ac0 2023-10-08T07:14:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.813+0000 7f385c9cc640 1 -- 172.21.15.70:0/54099782 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38540319d0 con 0x7f3858159ac0 2023-10-08T07:14:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.813+0000 7f385c9cc640 1 -- 172.21.15.70:0/54099782 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854031d60 con 0x7f3858159ac0 2023-10-08T07:14:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.814+0000 7f385fc32640 1 -- 172.21.15.70:0/54099782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3858159ac0 msgr2=0x7f385806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.814+0000 7f385fc32640 1 --2- 172.21.15.70:0/54099782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3858159ac0 0x7f385806d5e0 secure :-1 s=READY pgs=2560 cs=0 l=1 rev1=1 crypto rx=0x7f38580b4970 tx=0x7f385402b910 comp rx=0 tx=0).stop 2023-10-08T07:14:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.814+0000 7f385fc32640 1 -- 172.21.15.70:0/54099782 shutdown_connections 2023-10-08T07:14:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.814+0000 7f385fc32640 1 --2- 172.21.15.70:0/54099782 >> v2:172.21.15.181:3300/0 conn(0x7f38580ad790 0x7f38580702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.814+0000 7f385fc32640 1 --2- 172.21.15.70:0/54099782 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3858159ac0 0x7f385806d5e0 unknown :-1 s=CLOSED pgs=2560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.814+0000 7f385fc32640 1 -- 172.21.15.70:0/54099782 >> 172.21.15.70:0/54099782 conn(0x7f385806a380 msgr2=0x7f38580b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.814+0000 7f385fc32640 1 -- 172.21.15.70:0/54099782 shutdown_connections 2023-10-08T07:14:28.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.814+0000 7f385fc32640 1 -- 172.21.15.70:0/54099782 wait complete. 2023-10-08T07:14:28.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.816+0000 7f385fc32640 1 Processor -- start 2023-10-08T07:14:28.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.816+0000 7f385fc32640 1 -- start start 2023-10-08T07:14:28.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.817+0000 7f385fc32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38580ad790 0x7f3858150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.817+0000 7f385fc32640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3858159ac0 0x7f38580b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.817+0000 7f385fc32640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f385815e330 0x7f38581f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.817+0000 7f385fc32640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38581f2fa0 con 0x7f38580ad790 2023-10-08T07:14:28.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.817+0000 7f385fc32640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38581f3280 con 0x7f385815e330 2023-10-08T07:14:28.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.817+0000 7f385fc32640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385815ab20 con 0x7f3858159ac0 2023-10-08T07:14:28.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.817+0000 7f385d9ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38580ad790 0x7f3858150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:28.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.817+0000 7f385d9ce640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38580ad790 0x7f3858150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43036/0 (socket says 172.21.15.70:43036) 2023-10-08T07:14:28.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.817+0000 7f385d9ce640 1 -- 172.21.15.70:0/3120692613 learned_addr learned my addr 172.21.15.70:0/3120692613 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:28.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f385d9ce640 1 -- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3858159ac0 msgr2=0x7f38580b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f385d9ce640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3858159ac0 0x7f38580b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f385d9ce640 1 -- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f385815e330 msgr2=0x7f38581f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f385d9ce640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f385815e330 0x7f38581f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f385d9ce640 1 -- 172.21.15.70:0/3120692613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38540094d0 con 0x7f38580ad790 2023-10-08T07:14:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f385d9ce640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38580ad790 0x7f3858150230 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f3854000c00 tx=0x7f38540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f384e7fc640 1 -- 172.21.15.70:0/3120692613 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854031840 con 0x7f38580ad790 2023-10-08T07:14:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f384e7fc640 1 -- 172.21.15.70:0/3120692613 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38540443f0 con 0x7f38580ad790 2023-10-08T07:14:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f384e7fc640 1 -- 172.21.15.70:0/3120692613 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38540447a0 con 0x7f38580ad790 2023-10-08T07:14:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f385fc32640 1 -- 172.21.15.70:0/3120692613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385815ae00 con 0x7f38580ad790 2023-10-08T07:14:28.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.818+0000 7f385fc32640 1 -- 172.21.15.70:0/3120692613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3858150770 con 0x7f38580ad790 2023-10-08T07:14:28.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.819+0000 7f385fc32640 1 -- 172.21.15.70:0/3120692613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3858067490 con 0x7f38580ad790 2023-10-08T07:14:28.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.820+0000 7f384e7fc640 1 -- 172.21.15.70:0/3120692613 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f385404d450 con 0x7f38580ad790 2023-10-08T07:14:28.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.821+0000 7f384e7fc640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f383c06b0b0 0x7f383c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:28.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.821+0000 7f384e7fc640 1 -- 172.21.15.70:0/3120692613 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f385402d040 con 0x7f38580ad790 2023-10-08T07:14:28.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.822+0000 7f385d1cd640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f383c06b0b0 0x7f383c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:28.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.823+0000 7f385d1cd640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f383c06b0b0 0x7f383c06d570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3848006fd0 tx=0x7f3848008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:28.825+0000 7f384e7fc640 1 -- 172.21.15.70:0/3120692613 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3854076f90 con 0x7f38580ad790 2023-10-08T07:14:29.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.052+0000 7f385fc32640 1 -- 172.21.15.70:0/3120692613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38580b0ff0 con 0x7f38580ad790 2023-10-08T07:14:29.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.053+0000 7f384e7fc640 1 -- 172.21.15.70:0/3120692613 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f385407a5b0 con 0x7f38580ad790 2023-10-08T07:14:29.054 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:29.054 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":116,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":11730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2038500071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2038500071},{"type":"v1","addr":"172.21.15.70:6834","nonce":2038500071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:13:48.829441+0000","modified":"2023-10-08T07:13:51.877323+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8416},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8416":{"gid":8416,"name":"d","rank":0,"incarnation":110,"state":"up:active","state_seq":7,"addr":"172.21.15.181:6837/4084907359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":4084907359},{"type":"v1","addr":"172.21.15.181:6837","nonce":4084907359}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-10-08T07:14:29.054 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 116 2023-10-08T07:14:29.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.055+0000 7f385fc32640 1 -- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f383c06b0b0 msgr2=0x7f383c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.055+0000 7f385fc32640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f383c06b0b0 0x7f383c06d570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3848006fd0 tx=0x7f3848008040 comp rx=0 tx=0).stop 2023-10-08T07:14:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.056+0000 7f385fc32640 1 -- 172.21.15.70:0/3120692613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38580ad790 msgr2=0x7f3858150230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.056+0000 7f385fc32640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38580ad790 0x7f3858150230 secure :-1 s=READY pgs=2561 cs=0 l=1 rev1=1 crypto rx=0x7f3854000c00 tx=0x7f38540046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.056+0000 7f385fc32640 1 -- 172.21.15.70:0/3120692613 shutdown_connections 2023-10-08T07:14:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.056+0000 7f385fc32640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f383c06b0b0 0x7f383c06d570 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.056+0000 7f385fc32640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f385815e330 0x7f38581f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.056+0000 7f385fc32640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3858159ac0 0x7f38580b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.056+0000 7f385fc32640 1 --2- 172.21.15.70:0/3120692613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38580ad790 0x7f3858150230 unknown :-1 s=CLOSED pgs=2561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.056+0000 7f385fc32640 1 -- 172.21.15.70:0/3120692613 >> 172.21.15.70:0/3120692613 conn(0x7f385806a380 msgr2=0x7f38580cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.056+0000 7f385fc32640 1 -- 172.21.15.70:0/3120692613 shutdown_connections 2023-10-08T07:14:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.056+0000 7f385fc32640 1 -- 172.21.15.70:0/3120692613 wait complete. 2023-10-08T07:14:29.076 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:29.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.278+0000 7f2e6e410640 1 Processor -- start 2023-10-08T07:14:29.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.278+0000 7f2e6e410640 1 -- start start 2023-10-08T07:14:29.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e6e410640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e68159ac0 0x7f2e6806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e6e410640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e680ad790 0x7f2e680702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e6e410640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2e6815e330 0x7f2e68070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e6e410640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6806dee0 con 0x7f2e680ad790 2023-10-08T07:14:29.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e6e410640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6815d850 con 0x7f2e6815e330 2023-10-08T07:14:29.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e6e410640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6815db30 con 0x7f2e68159ac0 2023-10-08T07:14:29.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e677fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e680ad790 0x7f2e680702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e677fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e680ad790 0x7f2e680702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43052/0 (socket says 172.21.15.70:43052) 2023-10-08T07:14:29.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e677fe640 1 -- 172.21.15.70:0/1554412778 learned_addr learned my addr 172.21.15.70:0/1554412778 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:29.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e677fe640 1 -- 172.21.15.70:0/1554412778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e68159ac0 msgr2=0x7f2e6806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:29.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e677fe640 1 --2- 172.21.15.70:0/1554412778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e68159ac0 0x7f2e6806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e677fe640 1 -- 172.21.15.70:0/1554412778 >> v2:172.21.15.181:3300/0 conn(0x7f2e6815e330 msgr2=0x7f2e68070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:29.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e677fe640 1 --2- 172.21.15.70:0/1554412778 >> v2:172.21.15.181:3300/0 conn(0x7f2e6815e330 0x7f2e68070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.280+0000 7f2e677fe640 1 -- 172.21.15.70:0/1554412778 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6815de10 con 0x7f2e680ad790 2023-10-08T07:14:29.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.281+0000 7f2e677fe640 1 --2- 172.21.15.70:0/1554412778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e680ad790 0x7f2e680702d0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f2e680ce730 tx=0x7f2e5002db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3277edd5f55ff8ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:29.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.282+0000 7f2e66ffd640 1 -- 172.21.15.70:0/1554412778 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5002f8e0 con 0x7f2e680ad790 2023-10-08T07:14:29.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.282+0000 7f2e66ffd640 1 -- 172.21.15.70:0/1554412778 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5002fa40 con 0x7f2e680ad790 2023-10-08T07:14:29.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.282+0000 7f2e66ffd640 1 -- 172.21.15.70:0/1554412778 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5002fdd0 con 0x7f2e680ad790 2023-10-08T07:14:29.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.282+0000 7f2e6e410640 1 -- 172.21.15.70:0/1554412778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e680ad790 msgr2=0x7f2e680702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.282+0000 7f2e6e410640 1 --2- 172.21.15.70:0/1554412778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e680ad790 0x7f2e680702d0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7f2e680ce730 tx=0x7f2e5002db20 comp rx=0 tx=0).stop 2023-10-08T07:14:29.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.282+0000 7f2e6e410640 1 -- 172.21.15.70:0/1554412778 shutdown_connections 2023-10-08T07:14:29.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.282+0000 7f2e6e410640 1 --2- 172.21.15.70:0/1554412778 >> v2:172.21.15.181:3300/0 conn(0x7f2e6815e330 0x7f2e68070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.282+0000 7f2e6e410640 1 --2- 172.21.15.70:0/1554412778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e680ad790 0x7f2e680702d0 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.282+0000 7f2e6e410640 1 --2- 172.21.15.70:0/1554412778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e68159ac0 0x7f2e6806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.282+0000 7f2e6e410640 1 -- 172.21.15.70:0/1554412778 >> 172.21.15.70:0/1554412778 conn(0x7f2e6806a380 msgr2=0x7f2e680b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:29.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.283+0000 7f2e6e410640 1 -- 172.21.15.70:0/1554412778 shutdown_connections 2023-10-08T07:14:29.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.283+0000 7f2e6e410640 1 -- 172.21.15.70:0/1554412778 wait complete. 2023-10-08T07:14:29.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.284+0000 7f2e6e410640 1 Processor -- start 2023-10-08T07:14:29.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.284+0000 7f2e6e410640 1 -- start start 2023-10-08T07:14:29.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.285+0000 7f2e6e410640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e680ad790 0x7f2e681451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.285+0000 7f2e6e410640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e68159ac0 0x7f2e680ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.285+0000 7f2e6e410640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6815e330 0x7f2e681544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.285+0000 7f2e6e410640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680acd20 con 0x7f2e6815e330 2023-10-08T07:14:29.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.285+0000 7f2e6e410640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680ad000 con 0x7f2e68159ac0 2023-10-08T07:14:29.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.285+0000 7f2e6e410640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e68150140 con 0x7f2e680ad790 2023-10-08T07:14:29.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e6c9ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6815e330 0x7f2e681544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e6c9ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6815e330 0x7f2e681544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43064/0 (socket says 172.21.15.70:43064) 2023-10-08T07:14:29.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e6c9ad640 1 -- 172.21.15.70:0/882905101 learned_addr learned my addr 172.21.15.70:0/882905101 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:29.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e677fe640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e68159ac0 0x7f2e680ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e67fff640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e680ad790 0x7f2e681451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e6c9ad640 1 -- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e680ad790 msgr2=0x7f2e681451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e6c9ad640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e680ad790 0x7f2e681451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e6c9ad640 1 -- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e68159ac0 msgr2=0x7f2e680ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e6c9ad640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e68159ac0 0x7f2e680ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e6c9ad640 1 -- 172.21.15.70:0/882905101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e500094d0 con 0x7f2e6815e330 2023-10-08T07:14:29.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.286+0000 7f2e6c9ad640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6815e330 0x7f2e681544b0 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000b410 tx=0x7f2e6000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:29.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.288+0000 7f2e64ff9640 1 -- 172.21.15.70:0/882905101 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60011830 con 0x7f2e6815e330 2023-10-08T07:14:29.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.288+0000 7f2e64ff9640 1 -- 172.21.15.70:0/882905101 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e600119b0 con 0x7f2e6815e330 2023-10-08T07:14:29.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.288+0000 7f2e64ff9640 1 -- 172.21.15.70:0/882905101 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60011d60 con 0x7f2e6815e330 2023-10-08T07:14:29.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.288+0000 7f2e6e410640 1 -- 172.21.15.70:0/882905101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e68150420 con 0x7f2e6815e330 2023-10-08T07:14:29.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.288+0000 7f2e6e410640 1 -- 172.21.15.70:0/882905101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e68156650 con 0x7f2e6815e330 2023-10-08T07:14:29.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.289+0000 7f2e6e410640 1 -- 172.21.15.70:0/882905101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e68067490 con 0x7f2e6815e330 2023-10-08T07:14:29.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.290+0000 7f2e64ff9640 1 -- 172.21.15.70:0/882905101 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2e6001a400 con 0x7f2e6815e330 2023-10-08T07:14:29.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.291+0000 7f2e64ff9640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f2e4806b0b0 0x7f2e4806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.291+0000 7f2e64ff9640 1 -- 172.21.15.70:0/882905101 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7060+0+0 (secure 0 0 0) 0x7f2e6008a140 con 0x7f2e6815e330 2023-10-08T07:14:29.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.292+0000 7f2e67fff640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f2e4806b0b0 0x7f2e4806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.293+0000 7f2e67fff640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f2e4806b0b0 0x7f2e4806d570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009f20 tx=0x7f2e580094f0 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:29.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.295+0000 7f2e64ff9640 1 -- 172.21.15.70:0/882905101 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e600548a0 con 0x7f2e6815e330 2023-10-08T07:14:29.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.479+0000 7f2e6e410640 1 -- 172.21.15.70:0/882905101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e680b0ff0 con 0x7f2e6815e330 2023-10-08T07:14:29.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.482+0000 7f2e64ff9640 1 -- 172.21.15.70:0/882905101 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v243) v1 ==== 74+0+18232 (secure 0 0 0) 0x7f2e60057ec0 con 0x7f2e6815e330 2023-10-08T07:14:29.483 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:29.483 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":243,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:25.425545+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:13:47.007437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:13:47.838023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"238","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2480387829":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/920349426":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/2459293689":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/4129076201":"2023-10-09T07:14:16.815350+0000","172.21.15.70:6834/180995576":"2023-10-09T07:14:22.405699+0000","172.21.15.181:6832/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/365280967":"2023-10-09T07:14:16.815350+0000","172.21.15.70:6832/180995576":"2023-10-09T07:14:22.405699+0000","172.21.15.181:0/2833418931":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/753176203":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/317173509":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:6832/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1018014196":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1778827440":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1217032325":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/2798662747":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/4139611391":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/3002223432":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/3074626816":"2023-10-09T07:14:06.545661+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.484+0000 7f2e6e410640 1 -- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f2e4806b0b0 msgr2=0x7f2e4806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.484+0000 7f2e6e410640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f2e4806b0b0 0x7f2e4806d570 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009f20 tx=0x7f2e580094f0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.485+0000 7f2e6e410640 1 -- 172.21.15.70:0/882905101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6815e330 msgr2=0x7f2e681544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.485+0000 7f2e6e410640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6815e330 0x7f2e681544b0 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000b410 tx=0x7f2e6000b8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.485+0000 7f2e6e410640 1 -- 172.21.15.70:0/882905101 shutdown_connections 2023-10-08T07:14:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.485+0000 7f2e6e410640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f2e4806b0b0 0x7f2e4806d570 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.485+0000 7f2e6e410640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6815e330 0x7f2e681544b0 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.485+0000 7f2e6e410640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2e68159ac0 0x7f2e680ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.485+0000 7f2e6e410640 1 --2- 172.21.15.70:0/882905101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2e680ad790 0x7f2e681451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.485+0000 7f2e6e410640 1 -- 172.21.15.70:0/882905101 >> 172.21.15.70:0/882905101 conn(0x7f2e6806a380 msgr2=0x7f2e680ca130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.485+0000 7f2e6e410640 1 -- 172.21.15.70:0/882905101 shutdown_connections 2023-10-08T07:14:29.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.485+0000 7f2e6e410640 1 -- 172.21.15.70:0/882905101 wait complete. 2023-10-08T07:14:29.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-10-08T07:14:29.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.714+0000 7fec42ebb640 1 Processor -- start 2023-10-08T07:14:29.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.715+0000 7fec42ebb640 1 -- start start 2023-10-08T07:14:29.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.716+0000 7fec42ebb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fec3c15a820 0x7fec3c0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.716+0000 7fec42ebb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec3c0ca680 0x7fec3c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.716+0000 7fec42ebb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fec3c15ac00 0x7fec3c0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.716+0000 7fec42ebb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3c0cab60 con 0x7fec3c15ac00 2023-10-08T07:14:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.716+0000 7fec42ebb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3c0cb960 con 0x7fec3c15a820 2023-10-08T07:14:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.716+0000 7fec42ebb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0cbdf0 con 0x7fec3c0ca680 2023-10-08T07:14:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.716+0000 7fec426ba640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fec3c15ac00 0x7fec3c0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.716+0000 7fec426ba640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fec3c15ac00 0x7fec3c0c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43068/0 (socket says 172.21.15.70:43068) 2023-10-08T07:14:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.716+0000 7fec426ba640 1 -- 172.21.15.70:0/2088917877 learned_addr learned my addr 172.21.15.70:0/2088917877 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.717+0000 7fec41eb9640 1 --2- 172.21.15.70:0/2088917877 >> v2:172.21.15.181:3300/0 conn(0x7fec3c15a820 0x7fec3c0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.717+0000 7fec426ba640 1 -- 172.21.15.70:0/2088917877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec3c0ca680 msgr2=0x7fec3c0b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.717+0000 7fec416b8640 1 --2- 172.21.15.70:0/2088917877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec3c0ca680 0x7fec3c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.717+0000 7fec426ba640 1 --2- 172.21.15.70:0/2088917877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec3c0ca680 0x7fec3c0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.717+0000 7fec426ba640 1 -- 172.21.15.70:0/2088917877 >> v2:172.21.15.181:3300/0 conn(0x7fec3c15a820 msgr2=0x7fec3c0aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.717+0000 7fec426ba640 1 --2- 172.21.15.70:0/2088917877 >> v2:172.21.15.181:3300/0 conn(0x7fec3c15a820 0x7fec3c0aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.717+0000 7fec426ba640 1 -- 172.21.15.70:0/2088917877 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c0cb030 con 0x7fec3c15ac00 2023-10-08T07:14:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.717+0000 7fec41eb9640 1 --2- 172.21.15.70:0/2088917877 >> v2:172.21.15.181:3300/0 conn(0x7fec3c15a820 0x7fec3c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.718+0000 7fec426ba640 1 --2- 172.21.15.70:0/2088917877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c15ac00 0x7fec3c0c9100 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fec3c06da70 tx=0x7fec3402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3db6ae74527789 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.718+0000 7fec40eb7640 1 -- 172.21.15.70:0/2088917877 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3402c9f0 con 0x7fec3c15ac00 2023-10-08T07:14:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.718+0000 7fec40eb7640 1 -- 172.21.15.70:0/2088917877 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3402cb50 con 0x7fec3c15ac00 2023-10-08T07:14:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.718+0000 7fec40eb7640 1 -- 172.21.15.70:0/2088917877 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec340343f0 con 0x7fec3c15ac00 2023-10-08T07:14:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.719+0000 7fec42ebb640 1 -- 172.21.15.70:0/2088917877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c15ac00 msgr2=0x7fec3c0c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.719+0000 7fec42ebb640 1 --2- 172.21.15.70:0/2088917877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c15ac00 0x7fec3c0c9100 secure :-1 s=READY pgs=2564 cs=0 l=1 rev1=1 crypto rx=0x7fec3c06da70 tx=0x7fec3402be50 comp rx=0 tx=0).stop 2023-10-08T07:14:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.719+0000 7fec42ebb640 1 -- 172.21.15.70:0/2088917877 shutdown_connections 2023-10-08T07:14:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.719+0000 7fec42ebb640 1 --2- 172.21.15.70:0/2088917877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c15ac00 0x7fec3c0c9100 unknown :-1 s=CLOSED pgs=2564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.719+0000 7fec42ebb640 1 --2- 172.21.15.70:0/2088917877 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec3c0ca680 0x7fec3c0b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.719+0000 7fec42ebb640 1 --2- 172.21.15.70:0/2088917877 >> v2:172.21.15.181:3300/0 conn(0x7fec3c15a820 0x7fec3c0aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.719+0000 7fec42ebb640 1 -- 172.21.15.70:0/2088917877 >> 172.21.15.70:0/2088917877 conn(0x7fec3c06c050 msgr2=0x7fec3c06c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.719+0000 7fec42ebb640 1 -- 172.21.15.70:0/2088917877 shutdown_connections 2023-10-08T07:14:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.719+0000 7fec42ebb640 1 -- 172.21.15.70:0/2088917877 wait complete. 2023-10-08T07:14:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.721+0000 7fec42ebb640 1 Processor -- start 2023-10-08T07:14:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.721+0000 7fec42ebb640 1 -- start start 2023-10-08T07:14:29.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.722+0000 7fec42ebb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c0ca680 0x7fec3c0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.722+0000 7fec42ebb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec3c15a820 0x7fec3c156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.722+0000 7fec42ebb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec3c15ac00 0x7fec3c1eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.722+0000 7fec42ebb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c0ad940 con 0x7fec3c0ca680 2023-10-08T07:14:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.722+0000 7fec42ebb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1f0a10 con 0x7fec3c15a820 2023-10-08T07:14:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.722+0000 7fec42ebb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1f0cf0 con 0x7fec3c15ac00 2023-10-08T07:14:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.722+0000 7fec41eb9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c0ca680 0x7fec3c0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.722+0000 7fec41eb9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c0ca680 0x7fec3c0ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43070/0 (socket says 172.21.15.70:43070) 2023-10-08T07:14:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.722+0000 7fec41eb9640 1 -- 172.21.15.70:0/2577651552 learned_addr learned my addr 172.21.15.70:0/2577651552 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec41eb9640 1 -- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec3c15ac00 msgr2=0x7fec3c1eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec41eb9640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec3c15ac00 0x7fec3c1eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec416b8640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec3c15a820 0x7fec3c156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec41eb9640 1 -- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec3c15a820 msgr2=0x7fec3c156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec41eb9640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec3c15a820 0x7fec3c156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec41eb9640 1 -- 172.21.15.70:0/2577651552 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec34009b30 con 0x7fec3c0ca680 2023-10-08T07:14:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec41eb9640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c0ca680 0x7fec3c0ae3e0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c7f0 tx=0x7fec3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec327fc640 1 -- 172.21.15.70:0/2577651552 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38007a30 con 0x7fec3c0ca680 2023-10-08T07:14:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec327fc640 1 -- 172.21.15.70:0/2577651552 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec38007b90 con 0x7fec3c0ca680 2023-10-08T07:14:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec42ebb640 1 -- 172.21.15.70:0/2577651552 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c0adbe0 con 0x7fec3c0ca680 2023-10-08T07:14:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec327fc640 1 -- 172.21.15.70:0/2577651552 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3800ce40 con 0x7fec3c0ca680 2023-10-08T07:14:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.723+0000 7fec42ebb640 1 -- 172.21.15.70:0/2577651552 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c0c81c0 con 0x7fec3c0ca680 2023-10-08T07:14:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.724+0000 7fec42ebb640 1 -- 172.21.15.70:0/2577651552 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3c074150 con 0x7fec3c0ca680 2023-10-08T07:14:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.725+0000 7fec327fc640 1 -- 172.21.15.70:0/2577651552 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fec380113f0 con 0x7fec3c0ca680 2023-10-08T07:14:29.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.726+0000 7fec327fc640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fec2006b0b0 0x7fec2006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:29.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.727+0000 7fec327fc640 1 -- 172.21.15.70:0/2577651552 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7060+0+0 (secure 0 0 0) 0x7fec3808a280 con 0x7fec3c0ca680 2023-10-08T07:14:29.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.727+0000 7fec416b8640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fec2006b0b0 0x7fec2006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:29.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.728+0000 7fec416b8640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fec2006b0b0 0x7fec2006d570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fec2c006fd0 tx=0x7fec2c008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:29.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.731+0000 7fec327fc640 1 -- 172.21.15.70:0/2577651552 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec380548d0 con 0x7fec3c0ca680 2023-10-08T07:14:29.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.956+0000 7fec42ebb640 1 -- 172.21.15.70:0/2577651552 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7fec3c070be0 con 0x7fec3c0ca680 2023-10-08T07:14:29.958 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T07:14:29.958+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T07:14:29.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.961+0000 7fec327fc640 1 -- 172.21.15.70:0/2577651552 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v117) v1 ==== 167+0+0 (secure 0 0 0) 0x7fec38057ef0 con 0x7fec3c0ca680 2023-10-08T07:14:29.962 INFO:teuthology.orchestra.run.smithi070.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T07:14:29.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.964+0000 7fec42ebb640 1 -- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fec2006b0b0 msgr2=0x7fec2006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.964+0000 7fec42ebb640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fec2006b0b0 0x7fec2006d570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fec2c006fd0 tx=0x7fec2c008040 comp rx=0 tx=0).stop 2023-10-08T07:14:29.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.964+0000 7fec42ebb640 1 -- 172.21.15.70:0/2577651552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c0ca680 msgr2=0x7fec3c0ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:29.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.964+0000 7fec42ebb640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c0ca680 0x7fec3c0ae3e0 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c7f0 tx=0x7fec3800cbf0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.965+0000 7fec42ebb640 1 -- 172.21.15.70:0/2577651552 shutdown_connections 2023-10-08T07:14:29.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.965+0000 7fec42ebb640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fec2006b0b0 0x7fec2006d570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.965+0000 7fec42ebb640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec3c15ac00 0x7fec3c1eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.965+0000 7fec42ebb640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec3c15a820 0x7fec3c156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.965+0000 7fec42ebb640 1 --2- 172.21.15.70:0/2577651552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec3c0ca680 0x7fec3c0ae3e0 unknown :-1 s=CLOSED pgs=2565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:29.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.965+0000 7fec42ebb640 1 -- 172.21.15.70:0/2577651552 >> 172.21.15.70:0/2577651552 conn(0x7fec3c06c050 msgr2=0x7fec3c0732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:29.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.965+0000 7fec42ebb640 1 -- 172.21.15.70:0/2577651552 shutdown_connections 2023-10-08T07:14:29.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:29.965+0000 7fec42ebb640 1 -- 172.21.15.70:0/2577651552 wait complete. 2023-10-08T07:14:29.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-10-08T07:14:29.986 INFO:tasks.ceph.mds.d.smithi181.stderr: -2003> 2023-10-08T07:13:54.594+0000 7f155728e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:14:30.081 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T07:14:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.198+0000 7f025c6b4640 1 Processor -- start 2023-10-08T07:14:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.198+0000 7f025c6b4640 1 -- start start 2023-10-08T07:14:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.199+0000 7f025c6b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f02540c9cf0 0x7f025406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.199+0000 7f025c6b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025415aa00 0x7f02540b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f025c6b4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025415b310 con 0x7f025406c3a0 2023-10-08T07:14:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f025c6b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025406e0a0 con 0x7f02540c9cf0 2023-10-08T07:14:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f025c6b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025406d290 con 0x7f025415aa00 2023-10-08T07:14:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f025a450640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f025406c3a0 0x7f025406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58090/0 (socket says 172.21.15.70:58090) 2023-10-08T07:14:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f025a450640 1 -- 172.21.15.70:0/2030962258 learned_addr learned my addr 172.21.15.70:0/2030962258 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f0259c4f640 1 --2- 172.21.15.70:0/2030962258 >> v2:172.21.15.70:3300/0 conn(0x7f02540c9cf0 0x7f025406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f025944e640 1 -- 172.21.15.70:0/2030962258 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757463445 0 0) 0x7f0250001110 con 0x7f025406c3a0 2023-10-08T07:14:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f025ac51640 1 --2- 172.21.15.70:0/2030962258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025415aa00 0x7f02540b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f025944e640 1 -- 172.21.15.70:0/2030962258 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0240003520 con 0x7f025406c3a0 2023-10-08T07:14:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f0259c4f640 1 -- 172.21.15.70:0/2030962258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025415aa00 msgr2=0x7f02540b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f0259c4f640 1 --2- 172.21.15.70:0/2030962258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025415aa00 0x7f02540b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f0259c4f640 1 -- 172.21.15.70:0/2030962258 >> v1:172.21.15.70:6789/0 conn(0x7f025406c3a0 legacy=0x7f025406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f0259c4f640 1 -- 172.21.15.70:0/2030962258 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02540c9860 con 0x7f02540c9cf0 2023-10-08T07:14:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.200+0000 7f025ac51640 1 --2- 172.21.15.70:0/2030962258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025415aa00 0x7f02540b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.202+0000 7f0259c4f640 1 --2- 172.21.15.70:0/2030962258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 0x7f025406db60 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f02440097a0 tx=0x7f024402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d438646b33b373c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.202+0000 7f025944e640 1 -- 172.21.15.70:0/2030962258 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024402d040 con 0x7f02540c9cf0 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.202+0000 7f025944e640 1 -- 172.21.15.70:0/2030962258 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0244031810 con 0x7f02540c9cf0 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.202+0000 7f025944e640 1 -- 172.21.15.70:0/2030962258 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244031ba0 con 0x7f02540c9cf0 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.202+0000 7f025c6b4640 1 -- 172.21.15.70:0/2030962258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 msgr2=0x7f025406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.203+0000 7f025c6b4640 1 --2- 172.21.15.70:0/2030962258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 0x7f025406db60 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7f02440097a0 tx=0x7f024402b8b0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.203+0000 7f025c6b4640 1 -- 172.21.15.70:0/2030962258 shutdown_connections 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.203+0000 7f025c6b4640 1 --2- 172.21.15.70:0/2030962258 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025415aa00 0x7f02540b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.203+0000 7f025c6b4640 1 --2- 172.21.15.70:0/2030962258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 0x7f025406db60 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.203+0000 7f025c6b4640 1 -- 172.21.15.70:0/2030962258 >> 172.21.15.70:0/2030962258 conn(0x7f025406a5d0 msgr2=0x7f02540b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.203+0000 7f025c6b4640 1 -- 172.21.15.70:0/2030962258 shutdown_connections 2023-10-08T07:14:30.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.203+0000 7f025c6b4640 1 -- 172.21.15.70:0/2030962258 wait complete. 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.205+0000 7f025c6b4640 1 Processor -- start 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.205+0000 7f025c6b4640 1 -- start start 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f025c6b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f025406c3a0 0x7f025415df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f025c6b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 0x7f02541598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f025c6b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025415aa00 0x7f0254159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f025c6b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02540c1580 con 0x7f02540c9cf0 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f025c6b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254145110 con 0x7f025406c3a0 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f025c6b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0254142820 con 0x7f025415aa00 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f0259c4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 0x7f02541598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f0259c4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 0x7f02541598d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43080/0 (socket says 172.21.15.70:43080) 2023-10-08T07:14:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f0259c4f640 1 -- 172.21.15.70:0/3840581809 learned_addr learned my addr 172.21.15.70:0/3840581809 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f025a450640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f025406c3a0 0x7f025415df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f0259c4f640 1 -- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025415aa00 msgr2=0x7f0254159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f0259c4f640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025415aa00 0x7f0254159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f0259c4f640 1 -- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f025406c3a0 msgr2=0x7f025415df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f0259c4f640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f025406c3a0 0x7f025415df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.206+0000 7f0259c4f640 1 -- 172.21.15.70:0/3840581809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0244009400 con 0x7f02540c9cf0 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.207+0000 7f0259c4f640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 0x7f02541598d0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f02440098c0 tx=0x7f02440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.207+0000 7f024affd640 1 -- 172.21.15.70:0/3840581809 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244006e10 con 0x7f02540c9cf0 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.207+0000 7f024affd640 1 -- 172.21.15.70:0/3840581809 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024403ad60 con 0x7f02540c9cf0 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.207+0000 7f025c6b4640 1 -- 172.21.15.70:0/3840581809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0254142ac0 con 0x7f02540c9cf0 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.207+0000 7f024affd640 1 -- 172.21.15.70:0/3840581809 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0244044620 con 0x7f02540c9cf0 2023-10-08T07:14:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.207+0000 7f025c6b4640 1 -- 172.21.15.70:0/3840581809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0254143080 con 0x7f02540c9cf0 2023-10-08T07:14:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.208+0000 7f025c6b4640 1 -- 172.21.15.70:0/3840581809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02541432a0 con 0x7f02540c9cf0 2023-10-08T07:14:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.209+0000 7f024affd640 1 -- 172.21.15.70:0/3840581809 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f024404d440 con 0x7f02540c9cf0 2023-10-08T07:14:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.210+0000 7f024affd640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f023806b0b0 0x7f023806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.211+0000 7f024affd640 1 -- 172.21.15.70:0/3840581809 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 7146+0+0 (secure 0 0 0) 0x7f02440ac470 con 0x7f02540c9cf0 2023-10-08T07:14:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.211+0000 7f025a450640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f023806b0b0 0x7f023806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.212+0000 7f025a450640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f023806b0b0 0x7f023806d570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0250001370 tx=0x7f0250009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.215+0000 7f024affd640 1 -- 172.21.15.70:0/3840581809 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0244076a70 con 0x7f02540c9cf0 2023-10-08T07:14:30.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.435+0000 7f025c6b4640 1 -- 172.21.15.70:0/3840581809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f025404e160 con 0x7f02540c9cf0 2023-10-08T07:14:30.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.964+0000 7f024affd640 1 -- 172.21.15.70:0/3840581809 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v118) v1 ==== 107+0+0 (secure 0 0 0) 0x7f024407a090 con 0x7f02540c9cf0 2023-10-08T07:14:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 -- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f023806b0b0 msgr2=0x7f023806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f023806b0b0 0x7f023806d570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0250001370 tx=0x7f0250009040 comp rx=0 tx=0).stop 2023-10-08T07:14:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 -- 172.21.15.70:0/3840581809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 msgr2=0x7f02541598d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 0x7f02541598d0 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f02440098c0 tx=0x7f02440046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 -- 172.21.15.70:0/3840581809 shutdown_connections 2023-10-08T07:14:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f023806b0b0 0x7f023806d570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f025415aa00 0x7f0254159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02540c9cf0 0x7f02541598d0 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 --2- 172.21.15.70:0/3840581809 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f025406c3a0 0x7f025415df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 -- 172.21.15.70:0/3840581809 >> 172.21.15.70:0/3840581809 conn(0x7f025406a5d0 msgr2=0x7f02540b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 -- 172.21.15.70:0/3840581809 shutdown_connections 2023-10-08T07:14:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:30.969+0000 7f025c6b4640 1 -- 172.21.15.70:0/3840581809 wait complete. 2023-10-08T07:14:30.992 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-10-08T07:14:31.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.214+0000 7f7a1a4e5640 1 Processor -- start 2023-10-08T07:14:31.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.214+0000 7f7a1a4e5640 1 -- start start 2023-10-08T07:14:31.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.215+0000 7f7a1a4e5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a1406dfc0 0x7f7a14070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:31.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.216+0000 7f7a1a4e5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a140b1cc0 con 0x7f7a140afbe0 2023-10-08T07:14:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.216+0000 7f7a1a4e5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a140b2620 con 0x7f7a14071070 2023-10-08T07:14:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.216+0000 7f7a1a4e5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a140b2060 con 0x7f7a1406dfc0 2023-10-08T07:14:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.216+0000 7f7a137fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a1406dfc0 0x7f7a14070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.216+0000 7f7a137fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a1406dfc0 0x7f7a14070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43094/0 (socket says 172.21.15.70:43094) 2023-10-08T07:14:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.216+0000 7f7a137fe640 1 -- 172.21.15.70:0/3743369535 learned_addr learned my addr 172.21.15.70:0/3743369535 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:31.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.216+0000 7f7a137fe640 1 -- 172.21.15.70:0/3743369535 >> v1:172.21.15.181:6789/0 conn(0x7f7a14071070 legacy=0x7f7a140cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:31.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.216+0000 7f7a137fe640 1 -- 172.21.15.70:0/3743369535 >> v1:172.21.15.70:6789/0 conn(0x7f7a140afbe0 legacy=0x7f7a140b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:31.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.216+0000 7f7a137fe640 1 -- 172.21.15.70:0/3743369535 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a140b2340 con 0x7f7a1406dfc0 2023-10-08T07:14:31.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.217+0000 7f7a137fe640 1 --2- 172.21.15.70:0/3743369535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1406dfc0 0x7f7a14070b30 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f7a140b2810 tx=0x7f79fc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4a9ea8f0c5cc5b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:31.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.218+0000 7f7a12ffd640 1 -- 172.21.15.70:0/3743369535 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc02cb00 con 0x7f7a1406dfc0 2023-10-08T07:14:31.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.218+0000 7f7a12ffd640 1 -- 172.21.15.70:0/3743369535 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc02cc60 con 0x7f7a1406dfc0 2023-10-08T07:14:31.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.218+0000 7f7a12ffd640 1 -- 172.21.15.70:0/3743369535 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0343f0 con 0x7f7a1406dfc0 2023-10-08T07:14:31.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.218+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/3743369535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1406dfc0 msgr2=0x7f7a14070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:31.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.218+0000 7f7a1a4e5640 1 --2- 172.21.15.70:0/3743369535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1406dfc0 0x7f7a14070b30 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f7a140b2810 tx=0x7f79fc02bf60 comp rx=0 tx=0).stop 2023-10-08T07:14:31.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.219+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/3743369535 shutdown_connections 2023-10-08T07:14:31.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.219+0000 7f7a1a4e5640 1 --2- 172.21.15.70:0/3743369535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a1406dfc0 0x7f7a14070b30 unknown :-1 s=CLOSED pgs=2570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:31.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.219+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/3743369535 >> 172.21.15.70:0/3743369535 conn(0x7f7a1406a5d0 msgr2=0x7f7a140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:31.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.219+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/3743369535 shutdown_connections 2023-10-08T07:14:31.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.219+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/3743369535 wait complete. 2023-10-08T07:14:31.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.220+0000 7f7a1a4e5640 1 Processor -- start 2023-10-08T07:14:31.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.221+0000 7f7a1a4e5640 1 -- start start 2023-10-08T07:14:31.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.221+0000 7f7a1a4e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a1406dfc0 0x7f7a141f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:31.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.221+0000 7f7a1a4e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a14071070 0x7f7a140a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:31.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.221+0000 7f7a1a4e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a140afbe0 0x7f7a1414a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:31.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.221+0000 7f7a1a4e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a14148880 con 0x7f7a14071070 2023-10-08T07:14:31.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.221+0000 7f7a1a4e5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a14148b60 con 0x7f7a1406dfc0 2023-10-08T07:14:31.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.221+0000 7f7a1a4e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a14148e40 con 0x7f7a140afbe0 2023-10-08T07:14:31.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.222+0000 7f7a137fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a14071070 0x7f7a140a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:31.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.222+0000 7f7a137fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a14071070 0x7f7a140a74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43096/0 (socket says 172.21.15.70:43096) 2023-10-08T07:14:31.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.222+0000 7f7a137fe640 1 -- 172.21.15.70:0/1119875883 learned_addr learned my addr 172.21.15.70:0/1119875883 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:31.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.222+0000 7f7a137fe640 1 -- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a140afbe0 msgr2=0x7f7a1414a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:31.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.222+0000 7f7a137fe640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a140afbe0 0x7f7a1414a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:31.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.222+0000 7f7a13fff640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a1406dfc0 0x7f7a141f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:31.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.222+0000 7f7a137fe640 1 -- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a1406dfc0 msgr2=0x7f7a141f00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:31.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.222+0000 7f7a137fe640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a1406dfc0 0x7f7a141f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:31.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.222+0000 7f7a137fe640 1 -- 172.21.15.70:0/1119875883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc009b30 con 0x7f7a14071070 2023-10-08T07:14:31.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.222+0000 7f7a137fe640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a14071070 0x7f7a140a74c0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f79fc000f80 tx=0x7f79fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:31.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.223+0000 7f7a10ff9640 1 -- 172.21.15.70:0/1119875883 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc03e450 con 0x7f7a14071070 2023-10-08T07:14:31.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.223+0000 7f7a10ff9640 1 -- 172.21.15.70:0/1119875883 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc03e5f0 con 0x7f7a14071070 2023-10-08T07:14:31.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.223+0000 7f7a10ff9640 1 -- 172.21.15.70:0/1119875883 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc03e9a0 con 0x7f7a14071070 2023-10-08T07:14:31.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.223+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/1119875883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a14149120 con 0x7f7a14071070 2023-10-08T07:14:31.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.223+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/1119875883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a140a7a00 con 0x7f7a14071070 2023-10-08T07:14:31.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.224+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/1119875883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a14149310 con 0x7f7a14071070 2023-10-08T07:14:31.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.225+0000 7f7a10ff9640 1 -- 172.21.15.70:0/1119875883 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f79fc03eb00 con 0x7f7a14071070 2023-10-08T07:14:31.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.226+0000 7f7a10ff9640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f79f406b0b0 0x7f79f406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:31.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.226+0000 7f7a10ff9640 1 -- 172.21.15.70:0/1119875883 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 7146+0+0 (secure 0 0 0) 0x7f79fc0acf30 con 0x7f7a14071070 2023-10-08T07:14:31.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.227+0000 7f7a13fff640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f79f406b0b0 0x7f79f406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:31.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.228+0000 7f7a13fff640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f79f406b0b0 0x7f79f406d570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000f60 tx=0x7f7a04009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:31.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.230+0000 7f7a10ff9640 1 -- 172.21.15.70:0/1119875883 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79fc0777e0 con 0x7f7a14071070 2023-10-08T07:14:31.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.421+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/1119875883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7a141489c0 con 0x7f7a14071070 2023-10-08T07:14:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.968+0000 7f7a10ff9640 1 -- 172.21.15.70:0/1119875883 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v245) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7a141489c0 con 0x7f7a14071070 2023-10-08T07:14:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'backup_fs_metadata' removed 2023-10-08T07:14:31.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.973+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f79f406b0b0 msgr2=0x7f79f406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.973+0000 7f7a1a4e5640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f79f406b0b0 0x7f79f406d570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000f60 tx=0x7f7a04009040 comp rx=0 tx=0).stop 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.973+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/1119875883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a14071070 msgr2=0x7f7a140a74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.973+0000 7f7a1a4e5640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a14071070 0x7f7a140a74c0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f79fc000f80 tx=0x7f79fc004a40 comp rx=0 tx=0).stop 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.974+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/1119875883 shutdown_connections 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.974+0000 7f7a1a4e5640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f79f406b0b0 0x7f79f406d570 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.974+0000 7f7a1a4e5640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a140afbe0 0x7f7a1414a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.974+0000 7f7a1a4e5640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a14071070 0x7f7a140a74c0 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.974+0000 7f7a1a4e5640 1 --2- 172.21.15.70:0/1119875883 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a1406dfc0 0x7f7a141f00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.974+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/1119875883 >> 172.21.15.70:0/1119875883 conn(0x7f7a1406a5d0 msgr2=0x7f7a140c6880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.974+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/1119875883 shutdown_connections 2023-10-08T07:14:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:31.974+0000 7f7a1a4e5640 1 -- 172.21.15.70:0/1119875883 wait complete. 2023-10-08T07:14:31.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-10-08T07:14:32.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.213+0000 7eff8cbc4640 1 Processor -- start 2023-10-08T07:14:32.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.213+0000 7eff8cbc4640 1 -- start start 2023-10-08T07:14:32.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.215+0000 7eff8cbc4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff8815a820 0x7eff880aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:32.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.215+0000 7eff8cbc4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 0x7eff880b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:32.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.215+0000 7eff8cbc4640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff880cab60 con 0x7eff880ca680 2023-10-08T07:14:32.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.215+0000 7eff8cbc4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff880cb960 con 0x7eff8815a820 2023-10-08T07:14:32.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.215+0000 7eff8cbc4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff880cbdf0 con 0x7eff8815ac00 2023-10-08T07:14:32.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.215+0000 7eff87fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 0x7eff880b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:32.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.215+0000 7eff877fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7eff8815a820 0x7eff880aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:32.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.216+0000 7eff87fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 0x7eff880b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:48640/0 (socket says 172.21.15.70:48640) 2023-10-08T07:14:32.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.216+0000 7eff87fff640 1 -- 172.21.15.70:0/1540216320 learned_addr learned my addr 172.21.15.70:0/1540216320 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:32.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.216+0000 7eff87fff640 1 -- 172.21.15.70:0/1540216320 >> v2:172.21.15.181:3300/0 conn(0x7eff8815a820 msgr2=0x7eff880aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:32.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.216+0000 7eff87fff640 1 --2- 172.21.15.70:0/1540216320 >> v2:172.21.15.181:3300/0 conn(0x7eff8815a820 0x7eff880aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.216+0000 7eff87fff640 1 -- 172.21.15.70:0/1540216320 >> v1:172.21.15.181:6789/0 conn(0x7eff880ca680 legacy=0x7eff880c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:32.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.216+0000 7eff87fff640 1 -- 172.21.15.70:0/1540216320 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff880cb030 con 0x7eff8815ac00 2023-10-08T07:14:32.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.216+0000 7eff877fe640 1 --2- 172.21.15.70:0/1540216320 >> v2:172.21.15.181:3300/0 conn(0x7eff8815a820 0x7eff880aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:14:32.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.218+0000 7eff87fff640 1 --2- 172.21.15.70:0/1540216320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 0x7eff880b04b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7eff80009e50 tx=0x7eff8002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feec10d87cdf7fb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:32.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.218+0000 7eff86ffd640 1 -- 172.21.15.70:0/1540216320 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8002ca80 con 0x7eff8815ac00 2023-10-08T07:14:32.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.219+0000 7eff86ffd640 1 -- 172.21.15.70:0/1540216320 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8002cbe0 con 0x7eff8815ac00 2023-10-08T07:14:32.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.219+0000 7eff86ffd640 1 -- 172.21.15.70:0/1540216320 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff800343f0 con 0x7eff8815ac00 2023-10-08T07:14:32.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.219+0000 7eff8cbc4640 1 -- 172.21.15.70:0/1540216320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 msgr2=0x7eff880b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:32.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.219+0000 7eff8cbc4640 1 --2- 172.21.15.70:0/1540216320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 0x7eff880b04b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7eff80009e50 tx=0x7eff8002bee0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.219+0000 7eff8cbc4640 1 -- 172.21.15.70:0/1540216320 shutdown_connections 2023-10-08T07:14:32.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.219+0000 7eff8cbc4640 1 --2- 172.21.15.70:0/1540216320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 0x7eff880b04b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.219+0000 7eff8cbc4640 1 --2- 172.21.15.70:0/1540216320 >> v2:172.21.15.181:3300/0 conn(0x7eff8815a820 0x7eff880aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.219+0000 7eff8cbc4640 1 -- 172.21.15.70:0/1540216320 >> 172.21.15.70:0/1540216320 conn(0x7eff8806c050 msgr2=0x7eff8806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:32.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.219+0000 7eff8cbc4640 1 -- 172.21.15.70:0/1540216320 shutdown_connections 2023-10-08T07:14:32.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.219+0000 7eff8cbc4640 1 -- 172.21.15.70:0/1540216320 wait complete. 2023-10-08T07:14:32.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.222+0000 7eff8cbc4640 1 Processor -- start 2023-10-08T07:14:32.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.222+0000 7eff8cbc4640 1 -- start start 2023-10-08T07:14:32.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.223+0000 7eff8cbc4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff880ca680 0x7eff880b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:32.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.223+0000 7eff8cbc4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff8815a820 0x7eff880ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:32.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.223+0000 7eff8cbc4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 0x7eff880c2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.223+0000 7eff8cbc4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff880afdc0 con 0x7eff8815a820 2023-10-08T07:14:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.223+0000 7eff8cbc4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff880aef00 con 0x7eff880ca680 2023-10-08T07:14:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.223+0000 7eff8cbc4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff880af1b0 con 0x7eff8815ac00 2023-10-08T07:14:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.223+0000 7eff7ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff8815a820 0x7eff880ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.224+0000 7eff7ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff8815a820 0x7eff880ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43102/0 (socket says 172.21.15.70:43102) 2023-10-08T07:14:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.224+0000 7eff7ffff640 1 -- 172.21.15.70:0/4178190978 learned_addr learned my addr 172.21.15.70:0/4178190978 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:32.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.224+0000 7eff877fe640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff880ca680 0x7eff880b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.224+0000 7eff87fff640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 0x7eff880c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.224+0000 7eff7ffff640 1 -- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 msgr2=0x7eff880c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.224+0000 7eff7ffff640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 0x7eff880c2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.224+0000 7eff7ffff640 1 -- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff880ca680 msgr2=0x7eff880b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.224+0000 7eff7ffff640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff880ca680 0x7eff880b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.224+0000 7eff7ffff640 1 -- 172.21.15.70:0/4178190978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78009400 con 0x7eff8815a820 2023-10-08T07:14:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.224+0000 7eff7ffff640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff8815a820 0x7eff880ae9c0 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7eff7000c400 tx=0x7eff7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:32.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.225+0000 7eff84ff9640 1 -- 172.21.15.70:0/4178190978 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff700052d0 con 0x7eff8815a820 2023-10-08T07:14:32.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.225+0000 7eff84ff9640 1 -- 172.21.15.70:0/4178190978 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff70005430 con 0x7eff8815a820 2023-10-08T07:14:32.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.225+0000 7eff84ff9640 1 -- 172.21.15.70:0/4178190978 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70003830 con 0x7eff8815a820 2023-10-08T07:14:32.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.225+0000 7eff8cbc4640 1 -- 172.21.15.70:0/4178190978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff80009b30 con 0x7eff8815a820 2023-10-08T07:14:32.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.225+0000 7eff8cbc4640 1 -- 172.21.15.70:0/4178190978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff880af770 con 0x7eff8815a820 2023-10-08T07:14:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.226+0000 7eff8cbc4640 1 -- 172.21.15.70:0/4178190978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff880aff30 con 0x7eff8815a820 2023-10-08T07:14:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.227+0000 7eff84ff9640 1 -- 172.21.15.70:0/4178190978 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7eff70003990 con 0x7eff8815a820 2023-10-08T07:14:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.228+0000 7eff84ff9640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7eff6806b0e0 0x7eff6806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.228+0000 7eff84ff9640 1 -- 172.21.15.70:0/4178190978 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6702+0+0 (secure 0 0 0) 0x7eff70089310 con 0x7eff8815a820 2023-10-08T07:14:32.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.228+0000 7eff877fe640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7eff6806b0e0 0x7eff6806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:32.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.230+0000 7eff877fe640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7eff6806b0e0 0x7eff6806d5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff78009c60 tx=0x7eff78008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:32.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.232+0000 7eff84ff9640 1 -- 172.21.15.70:0/4178190978 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7eff70053d40 con 0x7eff8815a820 2023-10-08T07:14:32.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.422+0000 7eff8cbc4640 1 -- 172.21.15.70:0/4178190978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7eff88074150 con 0x7eff8815a820 2023-10-08T07:14:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.972+0000 7eff84ff9640 1 -- 172.21.15.70:0/4178190978 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v246) v1 ==== 178+0+0 (secure 0 0 0) 0x7eff70057360 con 0x7eff8815a820 2023-10-08T07:14:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'backup_fs_data' removed 2023-10-08T07:14:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.976+0000 7eff8cbc4640 1 -- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7eff6806b0e0 msgr2=0x7eff6806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.976+0000 7eff8cbc4640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7eff6806b0e0 0x7eff6806d5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff78009c60 tx=0x7eff78008040 comp rx=0 tx=0).stop 2023-10-08T07:14:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.976+0000 7eff8cbc4640 1 -- 172.21.15.70:0/4178190978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff8815a820 msgr2=0x7eff880ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.976+0000 7eff8cbc4640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff8815a820 0x7eff880ae9c0 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7eff7000c400 tx=0x7eff7000c8c0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.978+0000 7eff8cbc4640 1 -- 172.21.15.70:0/4178190978 shutdown_connections 2023-10-08T07:14:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.978+0000 7eff8cbc4640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7eff6806b0e0 0x7eff6806d5a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.978+0000 7eff8cbc4640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7eff8815ac00 0x7eff880c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.978+0000 7eff8cbc4640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff8815a820 0x7eff880ae9c0 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.978+0000 7eff8cbc4640 1 --2- 172.21.15.70:0/4178190978 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7eff880ca680 0x7eff880b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.978+0000 7eff8cbc4640 1 -- 172.21.15.70:0/4178190978 >> 172.21.15.70:0/4178190978 conn(0x7eff8806c050 msgr2=0x7eff8815b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.979+0000 7eff8cbc4640 1 -- 172.21.15.70:0/4178190978 shutdown_connections 2023-10-08T07:14:32.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:32.979+0000 7eff8cbc4640 1 -- 172.21.15.70:0/4178190978 wait complete. 2023-10-08T07:14:33.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T07:14:33.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.211+0000 7f1b3d6f2640 1 Processor -- start 2023-10-08T07:14:33.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.212+0000 7f1b3d6f2640 1 -- start start 2023-10-08T07:14:33.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.213+0000 7f1b3d6f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1b381601b0 0x7f1b380cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.213+0000 7f1b3d6f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1b3806a460 0x7f1b380ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.213+0000 7f1b3d6f2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b380ce210 con 0x7f1b38071c30 2023-10-08T07:14:33.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.213+0000 7f1b3d6f2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b380cdee0 con 0x7f1b381601b0 2023-10-08T07:14:33.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.213+0000 7f1b3d6f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b38161530 con 0x7f1b3806a460 2023-10-08T07:14:33.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.214+0000 7f1b36ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1b381601b0 0x7f1b380cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.214+0000 7f1b2ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1b38071c30 0x7f1b38069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:58098/0 (socket says 172.21.15.70:58098) 2023-10-08T07:14:33.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.214+0000 7f1b2ffff640 1 -- 172.21.15.70:0/1204553910 learned_addr learned my addr 172.21.15.70:0/1204553910 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:33.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.214+0000 7f1b36ffd640 1 -- 172.21.15.70:0/1204553910 >> v2:172.21.15.181:3300/0 conn(0x7f1b3806a460 msgr2=0x7f1b380ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:33.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.214+0000 7f1b377fe640 1 --2- 172.21.15.70:0/1204553910 >> v2:172.21.15.181:3300/0 conn(0x7f1b3806a460 0x7f1b380ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.214+0000 7f1b367fc640 1 -- 172.21.15.70:0/1204553910 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165382949 0 0) 0x7f1b380ce210 con 0x7f1b38071c30 2023-10-08T07:14:33.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.214+0000 7f1b36ffd640 1 --2- 172.21.15.70:0/1204553910 >> v2:172.21.15.181:3300/0 conn(0x7f1b3806a460 0x7f1b380ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.214+0000 7f1b36ffd640 1 -- 172.21.15.70:0/1204553910 >> v1:172.21.15.70:6789/0 conn(0x7f1b38071c30 legacy=0x7f1b38069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.214+0000 7f1b36ffd640 1 -- 172.21.15.70:0/1204553910 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b38161810 con 0x7f1b381601b0 2023-10-08T07:14:33.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.216+0000 7f1b36ffd640 1 --2- 172.21.15.70:0/1204553910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b381601b0 0x7f1b380cd430 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f1b380b1350 tx=0x7f1b2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d1096a5a3e925e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:33.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.216+0000 7f1b367fc640 1 -- 172.21.15.70:0/1204553910 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28031830 con 0x7f1b381601b0 2023-10-08T07:14:33.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.216+0000 7f1b367fc640 1 -- 172.21.15.70:0/1204553910 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b28031990 con 0x7f1b381601b0 2023-10-08T07:14:33.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.216+0000 7f1b367fc640 1 -- 172.21.15.70:0/1204553910 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28031d20 con 0x7f1b381601b0 2023-10-08T07:14:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.217+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1204553910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b381601b0 msgr2=0x7f1b380cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.217+0000 7f1b3d6f2640 1 --2- 172.21.15.70:0/1204553910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b381601b0 0x7f1b380cd430 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f1b380b1350 tx=0x7f1b2802b8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.217+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1204553910 shutdown_connections 2023-10-08T07:14:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.217+0000 7f1b3d6f2640 1 --2- 172.21.15.70:0/1204553910 >> v2:172.21.15.181:3300/0 conn(0x7f1b3806a460 0x7f1b380ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.217+0000 7f1b3d6f2640 1 --2- 172.21.15.70:0/1204553910 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b381601b0 0x7f1b380cd430 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.217+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1204553910 >> 172.21.15.70:0/1204553910 conn(0x7f1b3806d430 msgr2=0x7f1b3806d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.217+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1204553910 shutdown_connections 2023-10-08T07:14:33.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.217+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1204553910 wait complete. 2023-10-08T07:14:33.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.219+0000 7f1b3d6f2640 1 Processor -- start 2023-10-08T07:14:33.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.219+0000 7f1b3d6f2640 1 -- start start 2023-10-08T07:14:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.220+0000 7f1b3d6f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b3806a460 0x7f1b380adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.220+0000 7f1b3d6f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b38071c30 0x7f1b380c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.220+0000 7f1b3d6f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b381601b0 0x7f1b38155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.220+0000 7f1b3d6f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380c7380 con 0x7f1b3806a460 2023-10-08T07:14:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.220+0000 7f1b3d6f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380c7660 con 0x7f1b381601b0 2023-10-08T07:14:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.220+0000 7f1b3d6f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b380c7940 con 0x7f1b38071c30 2023-10-08T07:14:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.220+0000 7f1b36ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b3806a460 0x7f1b380adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.220+0000 7f1b36ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b3806a460 0x7f1b380adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43118/0 (socket says 172.21.15.70:43118) 2023-10-08T07:14:33.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.220+0000 7f1b36ffd640 1 -- 172.21.15.70:0/1478247320 learned_addr learned my addr 172.21.15.70:0/1478247320 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b377fe640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b381601b0 0x7f1b38155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b2ffff640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b38071c30 0x7f1b380c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b36ffd640 1 -- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b38071c30 msgr2=0x7f1b380c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b36ffd640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b38071c30 0x7f1b380c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b36ffd640 1 -- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b381601b0 msgr2=0x7f1b38155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b36ffd640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b381601b0 0x7f1b38155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b36ffd640 1 -- 172.21.15.70:0/1478247320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b28009400 con 0x7f1b3806a460 2023-10-08T07:14:33.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b2ffff640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b38071c30 0x7f1b380c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:33.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b36ffd640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b3806a460 0x7f1b380adf00 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f1b28000c00 tx=0x7f1b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:33.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b2f7fe640 1 -- 172.21.15.70:0/1478247320 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2803aa60 con 0x7f1b3806a460 2023-10-08T07:14:33.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b2f7fe640 1 -- 172.21.15.70:0/1478247320 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2803abe0 con 0x7f1b3806a460 2023-10-08T07:14:33.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.221+0000 7f1b2f7fe640 1 -- 172.21.15.70:0/1478247320 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b280443f0 con 0x7f1b3806a460 2023-10-08T07:14:33.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.222+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1478247320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b380c7c20 con 0x7f1b3806a460 2023-10-08T07:14:33.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.222+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1478247320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b380b46b0 con 0x7f1b3806a460 2023-10-08T07:14:33.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.223+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1478247320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b380c8140 con 0x7f1b3806a460 2023-10-08T07:14:33.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.224+0000 7f1b2f7fe640 1 -- 172.21.15.70:0/1478247320 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1b2804d450 con 0x7f1b3806a460 2023-10-08T07:14:33.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.225+0000 7f1b2f7fe640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f1b1806b0b0 0x7f1b1806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.225+0000 7f1b2f7fe640 1 -- 172.21.15.70:0/1478247320 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f1b280abff0 con 0x7f1b3806a460 2023-10-08T07:14:33.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.225+0000 7f1b2ffff640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f1b1806b0b0 0x7f1b1806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.227+0000 7f1b2ffff640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f1b1806b0b0 0x7f1b1806d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b380cb090 tx=0x7f1b20009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:33.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.229+0000 7f1b2f7fe640 1 -- 172.21.15.70:0/1478247320 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1b280deca0 con 0x7f1b3806a460 2023-10-08T07:14:33.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.418+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1478247320 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1b380b4840 con 0x7f1b3806a460 2023-10-08T07:14:33.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.425+0000 7f1b2f7fe640 1 -- 172.21.15.70:0/1478247320 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1b28035020 con 0x7f1b3806a460 2023-10-08T07:14:33.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f1b1806b0b0 msgr2=0x7f1b1806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f1b1806b0b0 0x7f1b1806d570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b380cb090 tx=0x7f1b20009040 comp rx=0 tx=0).stop 2023-10-08T07:14:33.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1478247320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b3806a460 msgr2=0x7f1b380adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b3806a460 0x7f1b380adf00 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f1b28000c00 tx=0x7f1b280046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1478247320 shutdown_connections 2023-10-08T07:14:33.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f1b1806b0b0 0x7f1b1806d570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1b381601b0 0x7f1b38155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1b38071c30 0x7f1b380c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 --2- 172.21.15.70:0/1478247320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b3806a460 0x7f1b380adf00 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1478247320 >> 172.21.15.70:0/1478247320 conn(0x7f1b3806d430 msgr2=0x7f1b380af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:33.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.428+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1478247320 shutdown_connections 2023-10-08T07:14:33.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.429+0000 7f1b3d6f2640 1 -- 172.21.15.70:0/1478247320 wait complete. 2023-10-08T07:14:33.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T07:14:33.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.658+0000 7f074e705640 1 Processor -- start 2023-10-08T07:14:33.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.658+0000 7f074e705640 1 -- start start 2023-10-08T07:14:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.660+0000 7f074e705640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f07480b1750 0x7f074806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.660+0000 7f074e705640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074806d240 0x7f0748070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.660+0000 7f074e705640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074815bf30 con 0x7f074815aad0 2023-10-08T07:14:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.660+0000 7f074e705640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074806d6b0 con 0x7f07480b1750 2023-10-08T07:14:33.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.660+0000 7f074e705640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07480b1bb0 con 0x7f074806d240 2023-10-08T07:14:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0747fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f074815aad0 0x7f07480c82b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39578/0 (socket says 172.21.15.70:39578) 2023-10-08T07:14:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0747fff640 1 -- 172.21.15.70:0/1052240381 learned_addr learned my addr 172.21.15.70:0/1052240381 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f074cca2640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074806d240 0x7f0748070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f07477fe640 1 --2- 172.21.15.70:0/1052240381 >> v2:172.21.15.181:3300/0 conn(0x7f07480b1750 0x7f074806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842963864 0 0) 0x7f074815bf30 con 0x7f074815aad0 2023-10-08T07:14:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0734003520 con 0x7f074815aad0 2023-10-08T07:14:33.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3082938741 0 0) 0x7f0734003520 con 0x7f074815aad0 2023-10-08T07:14:33.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074806d240 msgr2=0x7f0748070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:33.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0746ffd640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074806d240 0x7f0748070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 >> v2:172.21.15.181:3300/0 conn(0x7f07480b1750 msgr2=0x7f074806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:33.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0746ffd640 1 --2- 172.21.15.70:0/1052240381 >> v2:172.21.15.181:3300/0 conn(0x7f07480b1750 0x7f074806cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074815b320 con 0x7f074815aad0 2023-10-08T07:14:33.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.661+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f07380019f0 con 0x7f074815aad0 2023-10-08T07:14:33.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 >> v1:172.21.15.70:6789/0 conn(0x7f074815aad0 legacy=0x7f07480c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f0746ffd640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074806d240 0x7f0734003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f0746ffd640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 0x7f07340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f0746ffd640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0734004d00 0x7f0734005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074806d6b0 con 0x7f0734003ef0 2023-10-08T07:14:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480b1bb0 con 0x7f0734004d00 2023-10-08T07:14:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073400b980 con 0x7f074806d240 2023-10-08T07:14:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f07477fe640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 0x7f07340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f074cca2640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0734004d00 0x7f0734005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f07477fe640 1 -- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074806d240 msgr2=0x7f0734003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f0747fff640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074806d240 0x7f0734003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f07477fe640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074806d240 0x7f0734003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f07477fe640 1 -- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0734004d00 msgr2=0x7f0734005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f07477fe640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0734004d00 0x7f0734005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f07477fe640 1 -- 172.21.15.70:0/1052240381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0734003520 con 0x7f0734003ef0 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f074cca2640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0734004d00 0x7f0734005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.662+0000 7f0747fff640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074806d240 0x7f0734003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.664+0000 7f07477fe640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 0x7f07340047c0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f0730009ac0 tx=0x7f07340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694039c24a2277e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.664+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073002c6d0 con 0x7f0734003ef0 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.664+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073002c830 con 0x7f0734003ef0 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.664+0000 7f0746ffd640 1 -- 172.21.15.70:0/1052240381 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073002cbc0 con 0x7f0734003ef0 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.664+0000 7f074e705640 1 -- 172.21.15.70:0/1052240381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 msgr2=0x7f07340047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.664+0000 7f074e705640 1 --2- 172.21.15.70:0/1052240381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 0x7f07340047c0 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f0730009ac0 tx=0x7f07340057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.664+0000 7f074cca2640 1 -- 172.21.15.70:0/1052240381 reap_dead start 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.665+0000 7f074e705640 1 -- 172.21.15.70:0/1052240381 shutdown_connections 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.665+0000 7f074e705640 1 -- 172.21.15.70:0/1052240381 >> 172.21.15.70:0/1052240381 conn(0x7f0748069d50 msgr2=0x7f074806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.665+0000 7f074e705640 1 -- 172.21.15.70:0/1052240381 shutdown_connections 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.665+0000 7f074e705640 1 -- 172.21.15.70:0/1052240381 wait complete. 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.666+0000 7f074e705640 1 Processor -- start 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.667+0000 7f074e705640 1 -- start start 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.667+0000 7f074e705640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 0x7f0748140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.667+0000 7f074e705640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07481528a0 0x7f0748152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.667+0000 7f074e705640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074814f700 0x7f074814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.667+0000 7f074e705640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0748153260 con 0x7f0734003ef0 2023-10-08T07:14:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.667+0000 7f074e705640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074814f270 con 0x7f07481528a0 2023-10-08T07:14:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.667+0000 7f074e705640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074814b690 con 0x7f074814f700 2023-10-08T07:14:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.668+0000 7f0747fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 0x7f0748140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.668+0000 7f0747fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 0x7f0748140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36100/0 (socket says 172.21.15.70:36100) 2023-10-08T07:14:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.668+0000 7f0747fff640 1 -- 172.21.15.70:0/1242314108 learned_addr learned my addr 172.21.15.70:0/1242314108 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.668+0000 7f0747fff640 1 -- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074814f700 msgr2=0x7f074814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.668+0000 7f0747fff640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074814f700 0x7f074814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.668+0000 7f0747fff640 1 -- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07481528a0 msgr2=0x7f0748152d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.668+0000 7f0747fff640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07481528a0 0x7f0748152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.668+0000 7f0747fff640 1 -- 172.21.15.70:0/1242314108 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0748153480 con 0x7f0734003ef0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.668+0000 7f0747fff640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 0x7f0748140150 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f073800de90 tx=0x7f073800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.669+0000 7f0744ff9640 1 -- 172.21.15.70:0/1242314108 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738004b60 con 0x7f0734003ef0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.669+0000 7f0744ff9640 1 -- 172.21.15.70:0/1242314108 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0738004cc0 con 0x7f0734003ef0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.669+0000 7f0744ff9640 1 -- 172.21.15.70:0/1242314108 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0738007310 con 0x7f0734003ef0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.669+0000 7f074e705640 1 -- 172.21.15.70:0/1242314108 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0748153760 con 0x7f0734003ef0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.669+0000 7f074e705640 1 -- 172.21.15.70:0/1242314108 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074814ef00 con 0x7f0734003ef0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.670+0000 7f074e705640 1 -- 172.21.15.70:0/1242314108 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074804e160 con 0x7f0734003ef0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.671+0000 7f0744ff9640 1 -- 172.21.15.70:0/1242314108 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f0738007470 con 0x7f0734003ef0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.672+0000 7f0744ff9640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f072806b0e0 0x7f072806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.672+0000 7f0744ff9640 1 -- 172.21.15.70:0/1242314108 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f0738089db0 con 0x7f0734003ef0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.672+0000 7f07477fe640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f072806b0e0 0x7f072806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:33.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.673+0000 7f07477fe640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f072806b0e0 0x7f072806d5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0730036000 tx=0x7f073002e000 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:33.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.676+0000 7f0744ff9640 1 -- 172.21.15.70:0/1242314108 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f073808e070 con 0x7f0734003ef0 2023-10-08T07:14:33.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.867+0000 7f074e705640 1 -- 172.21.15.70:0/1242314108 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f07480b1530 con 0x7f0734003ef0 2023-10-08T07:14:33.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.874+0000 7f0744ff9640 1 -- 172.21.15.70:0/1242314108 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v80) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0738019020 con 0x7f0734003ef0 2023-10-08T07:14:33.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.877+0000 7f074e705640 1 -- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f072806b0e0 msgr2=0x7f072806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.877+0000 7f074e705640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f072806b0e0 0x7f072806d5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0730036000 tx=0x7f073002e000 comp rx=0 tx=0).stop 2023-10-08T07:14:33.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.877+0000 7f074e705640 1 -- 172.21.15.70:0/1242314108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 msgr2=0x7f0748140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:33.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.877+0000 7f074e705640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 0x7f0748140150 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f073800de90 tx=0x7f073800b4a0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.877+0000 7f074e705640 1 -- 172.21.15.70:0/1242314108 shutdown_connections 2023-10-08T07:14:33.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.877+0000 7f074e705640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f072806b0e0 0x7f072806d5a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.877+0000 7f074e705640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f074814f700 0x7f074814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.877+0000 7f074e705640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f07481528a0 0x7f0748152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.877+0000 7f074e705640 1 --2- 172.21.15.70:0/1242314108 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0734003ef0 0x7f0748140150 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:33.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.877+0000 7f074e705640 1 -- 172.21.15.70:0/1242314108 >> 172.21.15.70:0/1242314108 conn(0x7f0748069d50 msgr2=0x7f0748155660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:33.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.878+0000 7f074e705640 1 -- 172.21.15.70:0/1242314108 shutdown_connections 2023-10-08T07:14:33.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:33.878+0000 7f074e705640 1 -- 172.21.15.70:0/1242314108 wait complete. 2023-10-08T07:14:33.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T07:14:34.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.107+0000 7f5f77173640 1 Processor -- start 2023-10-08T07:14:34.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.107+0000 7f5f77173640 1 -- start start 2023-10-08T07:14:34.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.109+0000 7f5f77173640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5f700afbe0 0x7f5f700b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.109+0000 7f5f77173640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7006dfc0 0x7f5f70070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.109+0000 7f5f77173640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f700b2620 con 0x7f5f70071070 2023-10-08T07:14:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.109+0000 7f5f77173640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f700b2060 con 0x7f5f700afbe0 2023-10-08T07:14:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.109+0000 7f5f77173640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b2340 con 0x7f5f7006dfc0 2023-10-08T07:14:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.109+0000 7f5f75710640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5f70071070 0x7f5f700cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39586/0 (socket says 172.21.15.70:39586) 2023-10-08T07:14:34.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.109+0000 7f5f75710640 1 -- 172.21.15.70:0/3557387743 learned_addr learned my addr 172.21.15.70:0/3557387743 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:34.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.109+0000 7f5f74f0f640 1 --2- 172.21.15.70:0/3557387743 >> v2:172.21.15.181:3300/0 conn(0x7f5f700afbe0 0x7f5f700b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.109+0000 7f5f6ffff640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7006dfc0 0x7f5f70070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.109+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117195762 0 0) 0x7f5f700b2620 con 0x7f5f70071070 2023-10-08T07:14:34.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f60003520 con 0x7f5f70071070 2023-10-08T07:14:34.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2543349078 0 0) 0x7f5f60003520 con 0x7f5f70071070 2023-10-08T07:14:34.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7006dfc0 msgr2=0x7f5f70070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:34.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7006dfc0 0x7f5f70070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 >> v2:172.21.15.181:3300/0 conn(0x7f5f700afbe0 msgr2=0x7f5f700b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 --2- 172.21.15.70:0/3557387743 >> v2:172.21.15.181:3300/0 conn(0x7f5f700afbe0 0x7f5f700b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f70070350 con 0x7f5f70071070 2023-10-08T07:14:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5f680019f0 con 0x7f5f70071070 2023-10-08T07:14:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 >> v1:172.21.15.70:6789/0 conn(0x7f5f70071070 legacy=0x7f5f700cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7006dfc0 0x7f5f60003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f60003f40 0x7f5f600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f60004d00 0x7f5f60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b2060 con 0x7f5f60003f40 2023-10-08T07:14:34.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700b2340 con 0x7f5f60004d00 2023-10-08T07:14:34.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6000b980 con 0x7f5f7006dfc0 2023-10-08T07:14:34.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.110+0000 7f5f6ffff640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f60003f40 0x7f5f600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.111+0000 7f5f74f0f640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7006dfc0 0x7f5f60003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.111+0000 7f5f6ffff640 1 -- 172.21.15.70:0/3557387743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7006dfc0 msgr2=0x7f5f60003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:34.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.111+0000 7f5f6ffff640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f7006dfc0 0x7f5f60003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.111+0000 7f5f6ffff640 1 -- 172.21.15.70:0/3557387743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f60004d00 msgr2=0x7f5f60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:34.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.111+0000 7f5f6ffff640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f60004d00 0x7f5f60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.111+0000 7f5f6ffff640 1 -- 172.21.15.70:0/3557387743 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f60003520 con 0x7f5f60003f40 2023-10-08T07:14:34.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.112+0000 7f5f6ffff640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f60003f40 0x7f5f600047c0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009ac0 tx=0x7f5f600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eec2d122b579b59 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:34.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.112+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c02c680 con 0x7f5f60003f40 2023-10-08T07:14:34.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.112+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5c02c7e0 con 0x7f5f60003f40 2023-10-08T07:14:34.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.112+0000 7f5f6f7fe640 1 -- 172.21.15.70:0/3557387743 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c02cb90 con 0x7f5f60003f40 2023-10-08T07:14:34.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.113+0000 7f5f77173640 1 -- 172.21.15.70:0/3557387743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f60003f40 msgr2=0x7f5f600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:34.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.113+0000 7f5f77173640 1 --2- 172.21.15.70:0/3557387743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f60003f40 0x7f5f600047c0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009ac0 tx=0x7f5f600057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.113+0000 7f5f75710640 1 -- 172.21.15.70:0/3557387743 reap_dead start 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.113+0000 7f5f77173640 1 -- 172.21.15.70:0/3557387743 shutdown_connections 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.113+0000 7f5f77173640 1 -- 172.21.15.70:0/3557387743 >> 172.21.15.70:0/3557387743 conn(0x7f5f7006a5d0 msgr2=0x7f5f700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.113+0000 7f5f77173640 1 -- 172.21.15.70:0/3557387743 shutdown_connections 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.113+0000 7f5f77173640 1 -- 172.21.15.70:0/3557387743 wait complete. 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.115+0000 7f5f77173640 1 Processor -- start 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.115+0000 7f5f77173640 1 -- start start 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f77173640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f60003f40 0x7f5f701f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f77173640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700ac5a0 0x7f5f701f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f77173640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f700c6c00 0x7f5f7014f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f77173640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700ad120 con 0x7f5f700ac5a0 2023-10-08T07:14:34.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f77173640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f701f7110 con 0x7f5f60003f40 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f77173640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700acb60 con 0x7f5f700c6c00 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700ac5a0 0x7f5f701f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f6ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700ac5a0 0x7f5f701f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36108/0 (socket says 172.21.15.70:36108) 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f6ffff640 1 -- 172.21.15.70:0/881581804 learned_addr learned my addr 172.21.15.70:0/881581804 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f6ffff640 1 -- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f700c6c00 msgr2=0x7f5f7014f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f6ffff640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f700c6c00 0x7f5f7014f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f6ffff640 1 -- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f60003f40 msgr2=0x7f5f701f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f6ffff640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f60003f40 0x7f5f701f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.116+0000 7f5f6ffff640 1 -- 172.21.15.70:0/881581804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700ace40 con 0x7f5f700ac5a0 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.117+0000 7f5f6ffff640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700ac5a0 0x7f5f701f6bd0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009be0 tx=0x7f5f5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:34.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.117+0000 7f5f6d7fa640 1 -- 172.21.15.70:0/881581804 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c034a10 con 0x7f5f700ac5a0 2023-10-08T07:14:34.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.117+0000 7f5f6d7fa640 1 -- 172.21.15.70:0/881581804 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5c034b90 con 0x7f5f700ac5a0 2023-10-08T07:14:34.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.117+0000 7f5f6d7fa640 1 -- 172.21.15.70:0/881581804 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c03e400 con 0x7f5f700ac5a0 2023-10-08T07:14:34.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.117+0000 7f5f77173640 1 -- 172.21.15.70:0/881581804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f70157c60 con 0x7f5f700ac5a0 2023-10-08T07:14:34.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.117+0000 7f5f77173640 1 -- 172.21.15.70:0/881581804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f700c7080 con 0x7f5f700ac5a0 2023-10-08T07:14:34.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.118+0000 7f5f77173640 1 -- 172.21.15.70:0/881581804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f70157ac0 con 0x7f5f700ac5a0 2023-10-08T07:14:34.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.119+0000 7f5f6d7fa640 1 -- 172.21.15.70:0/881581804 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5f5c03e560 con 0x7f5f700ac5a0 2023-10-08T07:14:34.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.121+0000 7f5f6d7fa640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5f5406b0b0 0x7f5f5406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.121+0000 7f5f6d7fa640 1 -- 172.21.15.70:0/881581804 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f5f5c0abb90 con 0x7f5f700ac5a0 2023-10-08T07:14:34.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.121+0000 7f5f74f0f640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5f5406b0b0 0x7f5f5406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.123+0000 7f5f74f0f640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5f5406b0b0 0x7f5f5406d570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f640099d0 tx=0x7f5f64008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:34.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.126+0000 7f5f6d7fa640 1 -- 172.21.15.70:0/881581804 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f70157ac0 con 0x7f5f700ac5a0 2023-10-08T07:14:34.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.314+0000 7f5f77173640 1 -- 172.21.15.70:0/881581804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f5f70157ac0 con 0x7f5f700ac5a0 2023-10-08T07:14:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.320+0000 7f5f6d7fa640 1 -- 172.21.15.70:0/881581804 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v81) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5f70157ac0 con 0x7f5f700ac5a0 2023-10-08T07:14:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 -- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5f5406b0b0 msgr2=0x7f5f5406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5f5406b0b0 0x7f5f5406d570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f640099d0 tx=0x7f5f64008040 comp rx=0 tx=0).stop 2023-10-08T07:14:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 -- 172.21.15.70:0/881581804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700ac5a0 msgr2=0x7f5f701f6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700ac5a0 0x7f5f701f6bd0 secure :-1 s=READY pgs=2581 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009be0 tx=0x7f5f5c004730 comp rx=0 tx=0).stop 2023-10-08T07:14:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 -- 172.21.15.70:0/881581804 shutdown_connections 2023-10-08T07:14:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5f5406b0b0 0x7f5f5406d570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5f700c6c00 0x7f5f7014f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f700ac5a0 0x7f5f701f6bd0 unknown :-1 s=CLOSED pgs=2581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 --2- 172.21.15.70:0/881581804 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5f60003f40 0x7f5f701f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 -- 172.21.15.70:0/881581804 >> 172.21.15.70:0/881581804 conn(0x7f5f7006a5d0 msgr2=0x7f5f700ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:34.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 -- 172.21.15.70:0/881581804 shutdown_connections 2023-10-08T07:14:34.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.323+0000 7f5f77173640 1 -- 172.21.15.70:0/881581804 wait complete. 2023-10-08T07:14:34.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-10-08T07:14:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.552+0000 7f3f4b363640 1 Processor -- start 2023-10-08T07:14:34.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.552+0000 7f3f4b363640 1 -- start start 2023-10-08T07:14:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.553+0000 7f3f4b363640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3f440ada10 0x7f3f440b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4b363640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f440ca620 0x7f3f440c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4b363640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f440cb450 0x7f3f440cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4b363640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f440cbda0 con 0x7f3f440cb450 2023-10-08T07:14:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4b363640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f440cade0 con 0x7f3f440ada10 2023-10-08T07:14:34.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4b363640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440addf0 con 0x7f3f440ca620 2023-10-08T07:14:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4ab62640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f440cb450 0x7f3f440cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4ab62640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3f440cb450 0x7f3f440cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36110/0 (socket says 172.21.15.70:36110) 2023-10-08T07:14:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4ab62640 1 -- 172.21.15.70:0/1105467462 learned_addr learned my addr 172.21.15.70:0/1105467462 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4a361640 1 --2- 172.21.15.70:0/1105467462 >> v2:172.21.15.181:3300/0 conn(0x7f3f440ada10 0x7f3f440b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f49b60640 1 --2- 172.21.15.70:0/1105467462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f440ca620 0x7f3f440c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4ab62640 1 -- 172.21.15.70:0/1105467462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f440ca620 msgr2=0x7f3f440c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.554+0000 7f3f4ab62640 1 --2- 172.21.15.70:0/1105467462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f440ca620 0x7f3f440c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.555+0000 7f3f4ab62640 1 -- 172.21.15.70:0/1105467462 >> v2:172.21.15.181:3300/0 conn(0x7f3f440ada10 msgr2=0x7f3f440b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.555+0000 7f3f4ab62640 1 --2- 172.21.15.70:0/1105467462 >> v2:172.21.15.181:3300/0 conn(0x7f3f440ada10 0x7f3f440b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.555+0000 7f3f4ab62640 1 -- 172.21.15.70:0/1105467462 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4406c270 con 0x7f3f440cb450 2023-10-08T07:14:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.556+0000 7f3f4ab62640 1 --2- 172.21.15.70:0/1105467462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440cb450 0x7f3f440cb860 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f3f440cc380 tx=0x7f3f3c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65b0eec8ef8c23a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.556+0000 7f3f4935f640 1 -- 172.21.15.70:0/1105467462 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c02c9f0 con 0x7f3f440cb450 2023-10-08T07:14:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.556+0000 7f3f4935f640 1 -- 172.21.15.70:0/1105467462 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c02cb50 con 0x7f3f440cb450 2023-10-08T07:14:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.556+0000 7f3f4935f640 1 -- 172.21.15.70:0/1105467462 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0343f0 con 0x7f3f440cb450 2023-10-08T07:14:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.557+0000 7f3f4b363640 1 -- 172.21.15.70:0/1105467462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440cb450 msgr2=0x7f3f440cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.557+0000 7f3f4b363640 1 --2- 172.21.15.70:0/1105467462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440cb450 0x7f3f440cb860 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f3f440cc380 tx=0x7f3f3c02be50 comp rx=0 tx=0).stop 2023-10-08T07:14:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.557+0000 7f3f4b363640 1 -- 172.21.15.70:0/1105467462 shutdown_connections 2023-10-08T07:14:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.557+0000 7f3f4b363640 1 --2- 172.21.15.70:0/1105467462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440cb450 0x7f3f440cb860 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.557+0000 7f3f4b363640 1 --2- 172.21.15.70:0/1105467462 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f440ca620 0x7f3f440c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.557+0000 7f3f4b363640 1 --2- 172.21.15.70:0/1105467462 >> v2:172.21.15.181:3300/0 conn(0x7f3f440ada10 0x7f3f440b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.557+0000 7f3f4b363640 1 -- 172.21.15.70:0/1105467462 >> 172.21.15.70:0/1105467462 conn(0x7f3f440ce9d0 msgr2=0x7f3f440cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.557+0000 7f3f4b363640 1 -- 172.21.15.70:0/1105467462 shutdown_connections 2023-10-08T07:14:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.557+0000 7f3f4b363640 1 -- 172.21.15.70:0/1105467462 wait complete. 2023-10-08T07:14:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.559+0000 7f3f4b363640 1 Processor -- start 2023-10-08T07:14:34.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.559+0000 7f3f4b363640 1 -- start start 2023-10-08T07:14:34.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.560+0000 7f3f4b363640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440ada10 0x7f3f440c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.560+0000 7f3f4b363640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f440ca620 0x7f3f441e4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.560+0000 7f3f4b363640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f440cb450 0x7f3f44152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.560+0000 7f3f4b363640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4415d320 con 0x7f3f440ada10 2023-10-08T07:14:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.560+0000 7f3f4b363640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f441e4b40 con 0x7f3f440ca620 2023-10-08T07:14:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.560+0000 7f3f4b363640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440c3a60 con 0x7f3f440cb450 2023-10-08T07:14:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.560+0000 7f3f4a361640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440ada10 0x7f3f440c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.560+0000 7f3f4a361640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440ada10 0x7f3f440c3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36116/0 (socket says 172.21.15.70:36116) 2023-10-08T07:14:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.560+0000 7f3f4a361640 1 -- 172.21.15.70:0/48552557 learned_addr learned my addr 172.21.15.70:0/48552557 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f4a361640 1 -- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f440cb450 msgr2=0x7f3f44152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f4a361640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f440cb450 0x7f3f44152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f49b60640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f440ca620 0x7f3f441e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f4a361640 1 -- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f440ca620 msgr2=0x7f3f441e4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:34.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f4a361640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f440ca620 0x7f3f441e4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:34.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f4a361640 1 -- 172.21.15.70:0/48552557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c009b30 con 0x7f3f440ada10 2023-10-08T07:14:34.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f4a361640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440ada10 0x7f3f440c3ef0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c550 tx=0x7f3f4000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:34.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f3affd640 1 -- 172.21.15.70:0/48552557 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40004220 con 0x7f3f440ada10 2023-10-08T07:14:34.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f3affd640 1 -- 172.21.15.70:0/48552557 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40004380 con 0x7f3f440ada10 2023-10-08T07:14:34.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f3affd640 1 -- 172.21.15.70:0/48552557 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40003850 con 0x7f3f440ada10 2023-10-08T07:14:34.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f4b363640 1 -- 172.21.15.70:0/48552557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f44152a50 con 0x7f3f440ada10 2023-10-08T07:14:34.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.561+0000 7f3f4b363640 1 -- 172.21.15.70:0/48552557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f440c3130 con 0x7f3f440ada10 2023-10-08T07:14:34.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.562+0000 7f3f4b363640 1 -- 172.21.15.70:0/48552557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4404e160 con 0x7f3f440ada10 2023-10-08T07:14:34.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.563+0000 7f3f3affd640 1 -- 172.21.15.70:0/48552557 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3f400039b0 con 0x7f3f440ada10 2023-10-08T07:14:34.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.564+0000 7f3f3affd640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3f2806b0e0 0x7f3f2806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:34.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.564+0000 7f3f3affd640 1 -- 172.21.15.70:0/48552557 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f3f40089a70 con 0x7f3f440ada10 2023-10-08T07:14:34.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.565+0000 7f3f49b60640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3f2806b0e0 0x7f3f2806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:34.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.566+0000 7f3f49b60640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3f2806b0e0 0x7f3f2806d5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f34006fd0 tx=0x7f3f34008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:34.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.569+0000 7f3f3affd640 1 -- 172.21.15.70:0/48552557 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f400543d0 con 0x7f3f440ada10 2023-10-08T07:14:34.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:34.831+0000 7f3f4b363640 1 -- 172.21.15.70:0/48552557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f3f4406cde0 con 0x7f3f440ada10 2023-10-08T07:14:35.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.292+0000 7f3f3affd640 1 -- 172.21.15.70:0/48552557 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3259) v1 ==== 186+0+0 (secure 0 0 0) 0x7f3f400579f0 con 0x7f3f440ada10 2023-10-08T07:14:35.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 -- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3f2806b0e0 msgr2=0x7f3f2806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3f2806b0e0 0x7f3f2806d5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f34006fd0 tx=0x7f3f34008040 comp rx=0 tx=0).stop 2023-10-08T07:14:35.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 -- 172.21.15.70:0/48552557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440ada10 msgr2=0x7f3f440c3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440ada10 0x7f3f440c3ef0 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c550 tx=0x7f3f4000ca10 comp rx=0 tx=0).stop 2023-10-08T07:14:35.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 -- 172.21.15.70:0/48552557 shutdown_connections 2023-10-08T07:14:35.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f3f2806b0e0 0x7f3f2806d5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3f440cb450 0x7f3f44152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3f440ca620 0x7f3f441e4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 --2- 172.21.15.70:0/48552557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3f440ada10 0x7f3f440c3ef0 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 -- 172.21.15.70:0/48552557 >> 172.21.15.70:0/48552557 conn(0x7f3f440ce9d0 msgr2=0x7f3f441e7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:35.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.296+0000 7f3f4b363640 1 -- 172.21.15.70:0/48552557 shutdown_connections 2023-10-08T07:14:35.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.297+0000 7f3f4b363640 1 -- 172.21.15.70:0/48552557 wait complete. 2023-10-08T07:14:35.319 INFO:tasks.cephfs_test_runner:Test command succes with --fs argument for non-default fs ... ok 2023-10-08T07:14:35.320 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command fails without --fs argument in presence of multiple fs 2023-10-08T07:14:35.320 INFO:tasks.cephfs_test_runner:test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-10-08T07:14:35.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:35.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.537+0000 7f99023e2640 1 Processor -- start 2023-10-08T07:14:35.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.537+0000 7f99023e2640 1 -- start start 2023-10-08T07:14:35.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.539+0000 7f99023e2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f98fc15ab40 0x7f98fc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:35.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.539+0000 7f99023e2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc15b2b0 con 0x7f98fc0cb580 2023-10-08T07:14:35.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.539+0000 7f99023e2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc15b590 con 0x7f98fc0c90b0 2023-10-08T07:14:35.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.539+0000 7f99023e2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98fc15bf30 con 0x7f98fc15ab40 2023-10-08T07:14:35.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.539+0000 7f98fbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f98fc15ab40 0x7f98fc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:35.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.539+0000 7f990097f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f98fc0cb580 0x7f98fc15e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39588/0 (socket says 172.21.15.70:39588) 2023-10-08T07:14:35.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.539+0000 7f990097f640 1 -- 172.21.15.70:0/4283659387 learned_addr learned my addr 172.21.15.70:0/4283659387 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:35.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.540+0000 7f98fbfff640 1 -- 172.21.15.70:0/4283659387 >> v1:172.21.15.181:6789/0 conn(0x7f98fc0c90b0 legacy=0x7f98fc0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:35.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.540+0000 7f98fbfff640 1 -- 172.21.15.70:0/4283659387 >> v1:172.21.15.70:6789/0 conn(0x7f98fc0cb580 legacy=0x7f98fc15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.540+0000 7f98fbfff640 1 -- 172.21.15.70:0/4283659387 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc15b870 con 0x7f98fc15ab40 2023-10-08T07:14:35.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.541+0000 7f98fbfff640 1 --2- 172.21.15.70:0/4283659387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc15ab40 0x7f98fc0ad720 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0097a0 tx=0x7f98ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e10f1a63c902940 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:35.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.541+0000 7f98faffd640 1 -- 172.21.15.70:0/4283659387 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec031830 con 0x7f98fc15ab40 2023-10-08T07:14:35.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.541+0000 7f98faffd640 1 -- 172.21.15.70:0/4283659387 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec031990 con 0x7f98fc15ab40 2023-10-08T07:14:35.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.541+0000 7f98faffd640 1 -- 172.21.15.70:0/4283659387 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec031d20 con 0x7f98fc15ab40 2023-10-08T07:14:35.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.541+0000 7f99023e2640 1 -- 172.21.15.70:0/4283659387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc15ab40 msgr2=0x7f98fc0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.542+0000 7f99023e2640 1 --2- 172.21.15.70:0/4283659387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc15ab40 0x7f98fc0ad720 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0097a0 tx=0x7f98ec02b8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.542+0000 7f99023e2640 1 -- 172.21.15.70:0/4283659387 shutdown_connections 2023-10-08T07:14:35.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.542+0000 7f99023e2640 1 --2- 172.21.15.70:0/4283659387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc15ab40 0x7f98fc0ad720 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.542+0000 7f99023e2640 1 -- 172.21.15.70:0/4283659387 >> 172.21.15.70:0/4283659387 conn(0x7f98fc06a380 msgr2=0x7f98fc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:35.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.542+0000 7f99023e2640 1 -- 172.21.15.70:0/4283659387 shutdown_connections 2023-10-08T07:14:35.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.542+0000 7f99023e2640 1 -- 172.21.15.70:0/4283659387 wait complete. 2023-10-08T07:14:35.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.543+0000 7f99023e2640 1 Processor -- start 2023-10-08T07:14:35.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.544+0000 7f99023e2640 1 -- start start 2023-10-08T07:14:35.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.544+0000 7f99023e2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98fc0c90b0 0x7f98fc0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:35.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f99023e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc0cb580 0x7f98fc1e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:35.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f99023e2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98fc15ab40 0x7f98fc144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:35.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f99023e2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1e4440 con 0x7f98fc0cb580 2023-10-08T07:14:35.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f99023e2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc145780 con 0x7f98fc0c90b0 2023-10-08T07:14:35.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f99023e2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc145a60 con 0x7f98fc15ab40 2023-10-08T07:14:35.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f98fb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc0cb580 0x7f98fc1e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:35.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f98fb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc0cb580 0x7f98fc1e4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36146/0 (socket says 172.21.15.70:36146) 2023-10-08T07:14:35.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f98fb7fe640 1 -- 172.21.15.70:0/3793007302 learned_addr learned my addr 172.21.15.70:0/3793007302 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:35.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f98fb7fe640 1 -- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98fc15ab40 msgr2=0x7f98fc144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f98fb7fe640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98fc15ab40 0x7f98fc144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f98fb7fe640 1 -- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98fc0c90b0 msgr2=0x7f98fc0b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f98fb7fe640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98fc0c90b0 0x7f98fc0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f98fb7fe640 1 -- 172.21.15.70:0/3793007302 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec009400 con 0x7f98fc0cb580 2023-10-08T07:14:35.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.545+0000 7f98fb7fe640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc0cb580 0x7f98fc1e4e30 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f98f000c400 tx=0x7f98f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:35.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.546+0000 7f98f8ff9640 1 -- 172.21.15.70:0/3793007302 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00035f0 con 0x7f98fc0cb580 2023-10-08T07:14:35.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.546+0000 7f98f8ff9640 1 -- 172.21.15.70:0/3793007302 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f0003750 con 0x7f98fc0cb580 2023-10-08T07:14:35.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.546+0000 7f98f8ff9640 1 -- 172.21.15.70:0/3793007302 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f0003b40 con 0x7f98fc0cb580 2023-10-08T07:14:35.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.546+0000 7f99023e2640 1 -- 172.21.15.70:0/3793007302 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc1e4650 con 0x7f98fc0cb580 2023-10-08T07:14:35.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.546+0000 7f99023e2640 1 -- 172.21.15.70:0/3793007302 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc156430 con 0x7f98fc0cb580 2023-10-08T07:14:35.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.547+0000 7f99023e2640 1 -- 172.21.15.70:0/3793007302 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc145c20 con 0x7f98fc0cb580 2023-10-08T07:14:35.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.548+0000 7f98f8ff9640 1 -- 172.21.15.70:0/3793007302 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f98f0003ca0 con 0x7f98fc0cb580 2023-10-08T07:14:35.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.550+0000 7f98f8ff9640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f98e806b0b0 0x7f98e806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:35.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.550+0000 7f98f8ff9640 1 -- 172.21.15.70:0/3793007302 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f98f0088f00 con 0x7f98fc0cb580 2023-10-08T07:14:35.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.550+0000 7f98fbfff640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f98e806b0b0 0x7f98e806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:35.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.552+0000 7f98fbfff640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f98e806b0b0 0x7f98e806d570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f98ec036000 tx=0x7f98ec030000 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:35.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.555+0000 7f98f8ff9640 1 -- 172.21.15.70:0/3793007302 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98f0053a20 con 0x7f98fc0cb580 2023-10-08T07:14:35.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.746+0000 7f99023e2640 1 -- 172.21.15.70:0/3793007302 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98fc067490 con 0x7f98fc0cb580 2023-10-08T07:14:35.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.749+0000 7f98f8ff9640 1 -- 172.21.15.70:0/3793007302 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v246) v1 ==== 74+0+14708 (secure 0 0 0) 0x7f98f0017090 con 0x7f98fc0cb580 2023-10-08T07:14:35.750 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:35.750 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":246,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:32.969019+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:6836/4084907359":"2023-10-09T07:14:29.957910+0000","172.21.15.181:0/2480387829":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/920349426":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6837/4084907359":"2023-10-09T07:14:29.957910+0000","172.21.15.181:0/2459293689":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/4129076201":"2023-10-09T07:14:16.815350+0000","172.21.15.70:6834/180995576":"2023-10-09T07:14:22.405699+0000","172.21.15.181:6832/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/365280967":"2023-10-09T07:14:16.815350+0000","172.21.15.70:6832/180995576":"2023-10-09T07:14:22.405699+0000","172.21.15.181:0/2833418931":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/753176203":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/860795336":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/317173509":"2023-10-09T07:14:16.815350+0000","172.21.15.181:6833/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:6832/3993370366":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1018014196":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1778827440":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/1217032325":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/2798662747":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/4139611391":"2023-10-09T07:14:16.815350+0000","172.21.15.181:0/3002223432":"2023-10-09T07:14:06.545661+0000","172.21.15.181:0/3074626816":"2023-10-09T07:14:06.545661+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:35.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 -- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f98e806b0b0 msgr2=0x7f98e806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f98e806b0b0 0x7f98e806d570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f98ec036000 tx=0x7f98ec030000 comp rx=0 tx=0).stop 2023-10-08T07:14:35.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 -- 172.21.15.70:0/3793007302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc0cb580 msgr2=0x7f98fc1e4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc0cb580 0x7f98fc1e4e30 secure :-1 s=READY pgs=2586 cs=0 l=1 rev1=1 crypto rx=0x7f98f000c400 tx=0x7f98f000c8c0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 -- 172.21.15.70:0/3793007302 shutdown_connections 2023-10-08T07:14:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f98e806b0b0 0x7f98e806d570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f98fc15ab40 0x7f98fc144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98fc0cb580 0x7f98fc1e4e30 unknown :-1 s=CLOSED pgs=2586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 --2- 172.21.15.70:0/3793007302 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f98fc0c90b0 0x7f98fc0b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 -- 172.21.15.70:0/3793007302 >> 172.21.15.70:0/3793007302 conn(0x7f98fc06a380 msgr2=0x7f98fc0af610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 -- 172.21.15.70:0/3793007302 shutdown_connections 2023-10-08T07:14:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.752+0000 7f99023e2640 1 -- 172.21.15.70:0/3793007302 wait complete. 2023-10-08T07:14:35.773 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:14:35.773 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:14:35.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.988+0000 7fcaba9ed640 1 Processor -- start 2023-10-08T07:14:35.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.989+0000 7fcaba9ed640 1 -- start start 2023-10-08T07:14:35.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.990+0000 7fcaba9ed640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcab40d1120 0x7fcab40d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:35.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.990+0000 7fcaba9ed640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab40ce290 con 0x7fcab415dcd0 2023-10-08T07:14:35.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.990+0000 7fcaba9ed640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab40d0a80 con 0x7fcab40d1500 2023-10-08T07:14:35.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.990+0000 7fcaba9ed640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab40b1fe0 con 0x7fcab40d1120 2023-10-08T07:14:35.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.990+0000 7fcab99eb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcab40d1120 0x7fcab40d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:35.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.990+0000 7fcab99eb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcab40d1120 0x7fcab40d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36154/0 (socket says 172.21.15.70:36154) 2023-10-08T07:14:35.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.990+0000 7fcab99eb640 1 -- 172.21.15.70:0/2444309118 learned_addr learned my addr 172.21.15.70:0/2444309118 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:35.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.991+0000 7fcab99eb640 1 -- 172.21.15.70:0/2444309118 >> v1:172.21.15.181:6789/0 conn(0x7fcab40d1500 legacy=0x7fcab40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:35.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.991+0000 7fcab99eb640 1 -- 172.21.15.70:0/2444309118 >> v1:172.21.15.70:6789/0 conn(0x7fcab415dcd0 legacy=0x7fcab40cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.991+0000 7fcab99eb640 1 -- 172.21.15.70:0/2444309118 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab40b4580 con 0x7fcab40d1120 2023-10-08T07:14:35.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.992+0000 7fcab99eb640 1 --2- 172.21.15.70:0/2444309118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1120 0x7fcab40d04a0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009850 tx=0x7fcaa402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eecd78f3ac4d57f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:35.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.992+0000 7fcab89e9640 1 -- 172.21.15.70:0/2444309118 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4004230 con 0x7fcab40d1120 2023-10-08T07:14:35.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.992+0000 7fcab89e9640 1 -- 172.21.15.70:0/2444309118 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa4004390 con 0x7fcab40d1120 2023-10-08T07:14:35.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.992+0000 7fcab89e9640 1 -- 172.21.15.70:0/2444309118 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4003810 con 0x7fcab40d1120 2023-10-08T07:14:35.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.993+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2444309118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1120 msgr2=0x7fcab40d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.993+0000 7fcaba9ed640 1 --2- 172.21.15.70:0/2444309118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1120 0x7fcab40d04a0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009850 tx=0x7fcaa402b8f0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.993+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2444309118 shutdown_connections 2023-10-08T07:14:35.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.993+0000 7fcaba9ed640 1 --2- 172.21.15.70:0/2444309118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1120 0x7fcab40d04a0 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.993+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2444309118 >> 172.21.15.70:0/2444309118 conn(0x7fcab40d2960 msgr2=0x7fcab40d2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:35.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.993+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2444309118 shutdown_connections 2023-10-08T07:14:35.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.993+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2444309118 wait complete. 2023-10-08T07:14:35.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.995+0000 7fcaba9ed640 1 Processor -- start 2023-10-08T07:14:35.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.995+0000 7fcaba9ed640 1 -- start start 2023-10-08T07:14:35.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcaba9ed640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcab40d1120 0x7fcab40c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:35.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcaba9ed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1500 0x7fcab415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:35.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcaba9ed640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcab415dcd0 0x7fcab40b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:35.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcaba9ed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b6800 con 0x7fcab40d1500 2023-10-08T07:14:35.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcaba9ed640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b5f10 con 0x7fcab40d1120 2023-10-08T07:14:35.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcaba9ed640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b61b0 con 0x7fcab415dcd0 2023-10-08T07:14:35.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcab91ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1500 0x7fcab415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:35.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcab91ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1500 0x7fcab415aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36160/0 (socket says 172.21.15.70:36160) 2023-10-08T07:14:35.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcab91ea640 1 -- 172.21.15.70:0/2971755576 learned_addr learned my addr 172.21.15.70:0/2971755576 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:35.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcab91ea640 1 -- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcab415dcd0 msgr2=0x7fcab40b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcab91ea640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcab415dcd0 0x7fcab40b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcab99eb640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcab40d1120 0x7fcab40c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:35.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcab91ea640 1 -- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcab40d1120 msgr2=0x7fcab40c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:35.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcab91ea640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcab40d1120 0x7fcab40c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:35.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.996+0000 7fcab91ea640 1 -- 172.21.15.70:0/2971755576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa4009460 con 0x7fcab40d1500 2023-10-08T07:14:35.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.997+0000 7fcab91ea640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1500 0x7fcab415aed0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800c410 tx=0x7fcaa800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:35.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.997+0000 7fcab27fc640 1 -- 172.21.15.70:0/2971755576 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80035f0 con 0x7fcab40d1500 2023-10-08T07:14:35.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.997+0000 7fcab27fc640 1 -- 172.21.15.70:0/2971755576 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa8003750 con 0x7fcab40d1500 2023-10-08T07:14:35.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.997+0000 7fcab27fc640 1 -- 172.21.15.70:0/2971755576 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8003b40 con 0x7fcab40d1500 2023-10-08T07:14:35.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.997+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2971755576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab40b6490 con 0x7fcab40d1500 2023-10-08T07:14:35.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.997+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2971755576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab415bea0 con 0x7fcab40d1500 2023-10-08T07:14:36.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.998+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2971755576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab40b6970 con 0x7fcab40d1500 2023-10-08T07:14:36.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:35.999+0000 7fcab27fc640 1 -- 172.21.15.70:0/2971755576 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcaa8003ca0 con 0x7fcab40d1500 2023-10-08T07:14:36.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.000+0000 7fcab27fc640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcaa006b0b0 0x7fcaa006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:36.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.000+0000 7fcab27fc640 1 -- 172.21.15.70:0/2971755576 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fcaa8088fc0 con 0x7fcab40d1500 2023-10-08T07:14:36.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.001+0000 7fcab99eb640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcaa006b0b0 0x7fcaa006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:36.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.002+0000 7fcab99eb640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcaa006b0b0 0x7fcaa006d570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4035000 tx=0x7fcaa4008030 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:36.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.005+0000 7fcab27fc640 1 -- 172.21.15.70:0/2971755576 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcaa80539a0 con 0x7fcab40d1500 2023-10-08T07:14:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.193+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2971755576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcab4074150 con 0x7fcab40d1500 2023-10-08T07:14:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.194+0000 7fcab27fc640 1 -- 172.21.15.70:0/2971755576 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v246) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcaa80170a0 con 0x7fcab40d1500 2023-10-08T07:14:36.194 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:14:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcaa006b0b0 msgr2=0x7fcaa006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcaa006b0b0 0x7fcaa006d570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4035000 tx=0x7fcaa4008030 comp rx=0 tx=0).stop 2023-10-08T07:14:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2971755576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1500 msgr2=0x7fcab415aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1500 0x7fcab415aed0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800c410 tx=0x7fcaa800c8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2971755576 shutdown_connections 2023-10-08T07:14:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcaa006b0b0 0x7fcaa006d570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcab415dcd0 0x7fcab40b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcab40d1500 0x7fcab415aed0 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 --2- 172.21.15.70:0/2971755576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcab40d1120 0x7fcab40c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2971755576 >> 172.21.15.70:0/2971755576 conn(0x7fcab40d2960 msgr2=0x7fcab40b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.196+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2971755576 shutdown_connections 2023-10-08T07:14:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.197+0000 7fcaba9ed640 1 -- 172.21.15.70:0/2971755576 wait complete. 2023-10-08T07:14:36.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-10-08T07:14:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.433+0000 7f6badc39640 1 Processor -- start 2023-10-08T07:14:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.434+0000 7f6badc39640 1 -- start start 2023-10-08T07:14:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.435+0000 7f6badc39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ba806d4c0 0x7f6ba8162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.435+0000 7f6badc39640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba80ae140 con 0x7f6ba80cb150 2023-10-08T07:14:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.435+0000 7f6badc39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba80ccf20 con 0x7f6ba80add60 2023-10-08T07:14:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.435+0000 7f6badc39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8161f10 con 0x7f6ba806d4c0 2023-10-08T07:14:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.436+0000 7f6ba7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6ba80cb150 0x7f6ba8164d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39608/0 (socket says 172.21.15.70:39608) 2023-10-08T07:14:36.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.436+0000 7f6ba7fff640 1 -- 172.21.15.70:0/1194587659 learned_addr learned my addr 172.21.15.70:0/1194587659 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:36.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.436+0000 7f6b9ffff640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ba806d4c0 0x7f6ba8162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:36.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.436+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125953927 0 0) 0x7f6ba80ae140 con 0x7f6ba80cb150 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.436+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b94003520 con 0x7f6ba80cb150 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4019195046 0 0) 0x7f6b94003520 con 0x7f6ba80cb150 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ba806d4c0 msgr2=0x7f6ba8162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ba806d4c0 0x7f6ba8162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 >> v1:172.21.15.181:6789/0 conn(0x7f6ba80add60 legacy=0x7f6ba80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba80cd1c0 con 0x7f6ba80cb150 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f6ba00019f0 con 0x7f6ba80cb150 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 >> v1:172.21.15.70:6789/0 conn(0x7f6ba80cb150 legacy=0x7f6ba8164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b940039a0 0x7f6b94003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b940047c0 0x7f6b94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:36.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ba806d4c0 0x7f6b940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8161f10 con 0x7f6b940039a0 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9400bab0 con 0x7f6b940047c0 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9400bc70 con 0x7f6ba806d4c0 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba77fe640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b940039a0 0x7f6b94003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6b9ffff640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b940047c0 0x7f6b94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba77fe640 1 -- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ba806d4c0 msgr2=0x7f6b940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba77fe640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6ba806d4c0 0x7f6b940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba77fe640 1 -- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b940047c0 msgr2=0x7f6b94004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba77fe640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b940047c0 0x7f6b94004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6ba77fe640 1 -- 172.21.15.70:0/1194587659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b94003520 con 0x7f6b940039a0 2023-10-08T07:14:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.437+0000 7f6b9ffff640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6b940047c0 0x7f6b94004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.438+0000 7f6ba77fe640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b940039a0 0x7f6b94003dc0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f6b98001750 tx=0x7f6b940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36728a99da0999aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.439+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980052d0 con 0x7f6b940039a0 2023-10-08T07:14:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.439+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9802e050 con 0x7f6b940039a0 2023-10-08T07:14:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.439+0000 7f6ba6ffd640 1 -- 172.21.15.70:0/1194587659 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b980035f0 con 0x7f6b940039a0 2023-10-08T07:14:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.439+0000 7f6badc39640 1 -- 172.21.15.70:0/1194587659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b940039a0 msgr2=0x7f6b94003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.439+0000 7f6badc39640 1 --2- 172.21.15.70:0/1194587659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b940039a0 0x7f6b94003dc0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f6b98001750 tx=0x7f6b940057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.439+0000 7f6ba7fff640 1 -- 172.21.15.70:0/1194587659 reap_dead start 2023-10-08T07:14:36.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.439+0000 7f6badc39640 1 -- 172.21.15.70:0/1194587659 shutdown_connections 2023-10-08T07:14:36.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.439+0000 7f6badc39640 1 -- 172.21.15.70:0/1194587659 >> 172.21.15.70:0/1194587659 conn(0x7f6ba806a380 msgr2=0x7f6ba80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:36.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.439+0000 7f6badc39640 1 -- 172.21.15.70:0/1194587659 shutdown_connections 2023-10-08T07:14:36.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.439+0000 7f6badc39640 1 -- 172.21.15.70:0/1194587659 wait complete. 2023-10-08T07:14:36.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.441+0000 7f6badc39640 1 Processor -- start 2023-10-08T07:14:36.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.441+0000 7f6badc39640 1 -- start start 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.442+0000 7f6badc39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b940039a0 0x7f6ba8145320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.442+0000 7f6badc39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba8158780 0x7f6ba8158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.442+0000 7f6badc39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ba81fa730 0x7f6ba81fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.442+0000 7f6badc39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80ae9e0 con 0x7f6ba8158780 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.442+0000 7f6badc39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba8159530 con 0x7f6ba81fa730 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.442+0000 7f6badc39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80ae390 con 0x7f6b940039a0 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6b9ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba8158780 0x7f6ba8158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6b9ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba8158780 0x7f6ba8158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36186/0 (socket says 172.21.15.70:36186) 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6b9ffff640 1 -- 172.21.15.70:0/838444141 learned_addr learned my addr 172.21.15.70:0/838444141 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6ba7fff640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ba81fa730 0x7f6ba81fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6ba77fe640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b940039a0 0x7f6ba8145320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6b9ffff640 1 -- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b940039a0 msgr2=0x7f6ba8145320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6b9ffff640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b940039a0 0x7f6ba8145320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6b9ffff640 1 -- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ba81fa730 msgr2=0x7f6ba81fabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6b9ffff640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ba81fa730 0x7f6ba81fabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6b9ffff640 1 -- 172.21.15.70:0/838444141 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba80ae670 con 0x7f6ba8158780 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6b9ffff640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba8158780 0x7f6ba8158c00 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f6b900108b0 tx=0x7f6b90010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6ba4ff9640 1 -- 172.21.15.70:0/838444141 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9000eb60 con 0x7f6ba8158780 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6ba4ff9640 1 -- 172.21.15.70:0/838444141 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9000ecc0 con 0x7f6ba8158780 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6ba4ff9640 1 -- 172.21.15.70:0/838444141 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9001c400 con 0x7f6ba8158780 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.443+0000 7f6badc39640 1 -- 172.21.15.70:0/838444141 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba8159810 con 0x7f6ba8158780 2023-10-08T07:14:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.444+0000 7f6badc39640 1 -- 172.21.15.70:0/838444141 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba8159140 con 0x7f6ba8158780 2023-10-08T07:14:36.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.445+0000 7f6badc39640 1 -- 172.21.15.70:0/838444141 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba8159a30 con 0x7f6ba8158780 2023-10-08T07:14:36.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.446+0000 7f6ba4ff9640 1 -- 172.21.15.70:0/838444141 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f6b9001c560 con 0x7f6ba8158780 2023-10-08T07:14:36.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.447+0000 7f6ba4ff9640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f6b8806b0b0 0x7f6b8806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:36.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.447+0000 7f6ba4ff9640 1 -- 172.21.15.70:0/838444141 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f6b9008a2a0 con 0x7f6ba8158780 2023-10-08T07:14:36.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.448+0000 7f6ba77fe640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f6b8806b0b0 0x7f6b8806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:36.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.449+0000 7f6ba77fe640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f6b8806b0b0 0x7f6b8806d570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6b98033000 tx=0x7f6b98009030 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:36.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.451+0000 7f6ba4ff9640 1 -- 172.21.15.70:0/838444141 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6b90054dc0 con 0x7f6ba8158780 2023-10-08T07:14:36.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:36.713+0000 7f6badc39640 1 -- 172.21.15.70:0/838444141 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7f6ba80ae800 con 0x7f6ba8158780 2023-10-08T07:14:37.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.308+0000 7f6ba4ff9640 1 -- 172.21.15.70:0/838444141 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3261) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6b900583e0 con 0x7f6ba8158780 2023-10-08T07:14:37.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.311+0000 7f6badc39640 1 -- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f6b8806b0b0 msgr2=0x7f6b8806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:37.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.311+0000 7f6badc39640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f6b8806b0b0 0x7f6b8806d570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6b98033000 tx=0x7f6b98009030 comp rx=0 tx=0).stop 2023-10-08T07:14:37.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.311+0000 7f6badc39640 1 -- 172.21.15.70:0/838444141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba8158780 msgr2=0x7f6ba8158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:37.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.311+0000 7f6badc39640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba8158780 0x7f6ba8158c00 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f6b900108b0 tx=0x7f6b90010d70 comp rx=0 tx=0).stop 2023-10-08T07:14:37.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.312+0000 7f6badc39640 1 -- 172.21.15.70:0/838444141 shutdown_connections 2023-10-08T07:14:37.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.312+0000 7f6badc39640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f6b8806b0b0 0x7f6b8806d570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.312+0000 7f6badc39640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6ba81fa730 0x7f6ba81fabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.312+0000 7f6badc39640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba8158780 0x7f6ba8158c00 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.312+0000 7f6badc39640 1 --2- 172.21.15.70:0/838444141 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6b940039a0 0x7f6ba8145320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.312+0000 7f6badc39640 1 -- 172.21.15.70:0/838444141 >> 172.21.15.70:0/838444141 conn(0x7f6ba806a380 msgr2=0x7f6ba80701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:37.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.312+0000 7f6badc39640 1 -- 172.21.15.70:0/838444141 shutdown_connections 2023-10-08T07:14:37.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.312+0000 7f6badc39640 1 -- 172.21.15.70:0/838444141 wait complete. 2023-10-08T07:14:37.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T07:14:37.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.552+0000 7fb89c761640 1 Processor -- start 2023-10-08T07:14:37.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.552+0000 7fb89c761640 1 -- start start 2023-10-08T07:14:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.554+0000 7fb89c761640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb894160660 0x7fb894070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.554+0000 7fb89c761640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8940c9910 0x7fb8940caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.554+0000 7fb89c761640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89406e150 con 0x7fb8940cb450 2023-10-08T07:14:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.554+0000 7fb89c761640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8940cb8c0 con 0x7fb894160660 2023-10-08T07:14:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.554+0000 7fb89c761640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89406de70 con 0x7fb8940c9910 2023-10-08T07:14:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.554+0000 7fb89a4fd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb894160660 0x7fb894070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.554+0000 7fb89a4fd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb894160660 0x7fb894070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36192/0 (socket says 172.21.15.70:36192) 2023-10-08T07:14:37.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.554+0000 7fb89a4fd640 1 -- 172.21.15.70:0/3204185644 learned_addr learned my addr 172.21.15.70:0/3204185644 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:37.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.555+0000 7fb899cfc640 1 --2- 172.21.15.70:0/3204185644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8940c9910 0x7fb8940caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:37.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.555+0000 7fb89a4fd640 1 -- 172.21.15.70:0/3204185644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8940c9910 msgr2=0x7fb8940caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:37.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.555+0000 7fb89a4fd640 1 --2- 172.21.15.70:0/3204185644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8940c9910 0x7fb8940caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.555+0000 7fb89a4fd640 1 -- 172.21.15.70:0/3204185644 >> v1:172.21.15.181:6789/0 conn(0x7fb8940cb450 legacy=0x7fb8940b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:37.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.555+0000 7fb89a4fd640 1 -- 172.21.15.70:0/3204185644 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940c8e30 con 0x7fb894160660 2023-10-08T07:14:37.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.557+0000 7fb89a4fd640 1 --2- 172.21.15.70:0/3204185644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb894160660 0x7fb894070cc0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7fb890009870 tx=0x7fb89002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb1f493f199c306 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.557+0000 7fb8994fb640 1 -- 172.21.15.70:0/3204185644 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890030840 con 0x7fb894160660 2023-10-08T07:14:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.557+0000 7fb8994fb640 1 -- 172.21.15.70:0/3204185644 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8900309a0 con 0x7fb894160660 2023-10-08T07:14:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.557+0000 7fb8994fb640 1 -- 172.21.15.70:0/3204185644 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890030d30 con 0x7fb894160660 2023-10-08T07:14:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.558+0000 7fb89c761640 1 -- 172.21.15.70:0/3204185644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb894160660 msgr2=0x7fb894070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.558+0000 7fb89c761640 1 --2- 172.21.15.70:0/3204185644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb894160660 0x7fb894070cc0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7fb890009870 tx=0x7fb89002b980 comp rx=0 tx=0).stop 2023-10-08T07:14:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.558+0000 7fb89c761640 1 -- 172.21.15.70:0/3204185644 shutdown_connections 2023-10-08T07:14:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.558+0000 7fb89c761640 1 --2- 172.21.15.70:0/3204185644 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8940c9910 0x7fb8940caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.558+0000 7fb89c761640 1 --2- 172.21.15.70:0/3204185644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb894160660 0x7fb894070cc0 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.558+0000 7fb89c761640 1 -- 172.21.15.70:0/3204185644 >> 172.21.15.70:0/3204185644 conn(0x7fb89406a5d0 msgr2=0x7fb8940b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.558+0000 7fb89c761640 1 -- 172.21.15.70:0/3204185644 shutdown_connections 2023-10-08T07:14:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.558+0000 7fb89c761640 1 -- 172.21.15.70:0/3204185644 wait complete. 2023-10-08T07:14:37.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.560+0000 7fb89c761640 1 Processor -- start 2023-10-08T07:14:37.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.560+0000 7fb89c761640 1 -- start start 2023-10-08T07:14:37.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.561+0000 7fb89c761640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8940c9910 0x7fb8941f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:37.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.561+0000 7fb89c761640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8940cb450 0x7fb89414d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:37.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.561+0000 7fb89c761640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb894160660 0x7fb89415b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:37.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.561+0000 7fb89c761640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89414cba0 con 0x7fb8940c9910 2023-10-08T07:14:37.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.561+0000 7fb89c761640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89414a440 con 0x7fb894160660 2023-10-08T07:14:37.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.561+0000 7fb89c761640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941f65d0 con 0x7fb8940cb450 2023-10-08T07:14:37.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.561+0000 7fb89a4fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8940c9910 0x7fb8941f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:37.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.561+0000 7fb89a4fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8940c9910 0x7fb8941f5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36202/0 (socket says 172.21.15.70:36202) 2023-10-08T07:14:37.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.561+0000 7fb89a4fd640 1 -- 172.21.15.70:0/1755857538 learned_addr learned my addr 172.21.15.70:0/1755857538 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:37.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.562+0000 7fb899cfc640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8940cb450 0x7fb89414d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:37.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.562+0000 7fb89a4fd640 1 -- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8940cb450 msgr2=0x7fb89414d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:37.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.562+0000 7fb89acfe640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb894160660 0x7fb89415b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:37.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.562+0000 7fb89a4fd640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8940cb450 0x7fb89414d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.562+0000 7fb89a4fd640 1 -- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb894160660 msgr2=0x7fb89415b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:37.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.562+0000 7fb89a4fd640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb894160660 0x7fb89415b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.562+0000 7fb89a4fd640 1 -- 172.21.15.70:0/1755857538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900094d0 con 0x7fb8940c9910 2023-10-08T07:14:37.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.562+0000 7fb89a4fd640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8940c9910 0x7fb8941f5cd0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fb890000c00 tx=0x7fb890004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:37.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.563+0000 7fb88affd640 1 -- 172.21.15.70:0/1755857538 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890039a90 con 0x7fb8940c9910 2023-10-08T07:14:37.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.563+0000 7fb88affd640 1 -- 172.21.15.70:0/1755857538 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb890039bf0 con 0x7fb8940c9910 2023-10-08T07:14:37.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.563+0000 7fb88affd640 1 -- 172.21.15.70:0/1755857538 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890043410 con 0x7fb8940c9910 2023-10-08T07:14:37.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.563+0000 7fb89c761640 1 -- 172.21.15.70:0/1755857538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8941f68b0 con 0x7fb8940c9910 2023-10-08T07:14:37.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.563+0000 7fb89c761640 1 -- 172.21.15.70:0/1755857538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89414c830 con 0x7fb8940c9910 2023-10-08T07:14:37.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.564+0000 7fb89c761640 1 -- 172.21.15.70:0/1755857538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89404e160 con 0x7fb8940c9910 2023-10-08T07:14:37.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.565+0000 7fb88affd640 1 -- 172.21.15.70:0/1755857538 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb89002f390 con 0x7fb8940c9910 2023-10-08T07:14:37.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.566+0000 7fb88affd640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb87806b0b0 0x7fb87806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:37.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.566+0000 7fb88affd640 1 -- 172.21.15.70:0/1755857538 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fb8900abf40 con 0x7fb8940c9910 2023-10-08T07:14:37.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.566+0000 7fb899cfc640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb87806b0b0 0x7fb87806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:37.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.568+0000 7fb899cfc640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb87806b0b0 0x7fb87806d570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb894150e10 tx=0x7fb884008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:37.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.570+0000 7fb88affd640 1 -- 172.21.15.70:0/1755857538 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb890076940 con 0x7fb8940c9910 2023-10-08T07:14:37.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.758+0000 7fb89c761640 1 -- 172.21.15.70:0/1755857538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb8940b0590 con 0x7fb8940c9910 2023-10-08T07:14:37.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.763+0000 7fb88affd640 1 -- 172.21.15.70:0/1755857538 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v82)=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb8900340c0 con 0x7fb8940c9910 2023-10-08T07:14:37.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.766+0000 7fb89c761640 1 -- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb87806b0b0 msgr2=0x7fb87806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:37.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.766+0000 7fb89c761640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb87806b0b0 0x7fb87806d570 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb894150e10 tx=0x7fb884008040 comp rx=0 tx=0).stop 2023-10-08T07:14:37.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.766+0000 7fb89c761640 1 -- 172.21.15.70:0/1755857538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8940c9910 msgr2=0x7fb8941f5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:37.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.766+0000 7fb89c761640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8940c9910 0x7fb8941f5cd0 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7fb890000c00 tx=0x7fb890004730 comp rx=0 tx=0).stop 2023-10-08T07:14:37.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.766+0000 7fb89c761640 1 -- 172.21.15.70:0/1755857538 shutdown_connections 2023-10-08T07:14:37.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.767+0000 7fb89c761640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb87806b0b0 0x7fb87806d570 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.767+0000 7fb89c761640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb894160660 0x7fb89415b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.767+0000 7fb89c761640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8940cb450 0x7fb89414d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.767+0000 7fb89c761640 1 --2- 172.21.15.70:0/1755857538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8940c9910 0x7fb8941f5cd0 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:37.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.767+0000 7fb89c761640 1 -- 172.21.15.70:0/1755857538 >> 172.21.15.70:0/1755857538 conn(0x7fb89406a5d0 msgr2=0x7fb8940ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:37.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.767+0000 7fb89c761640 1 -- 172.21.15.70:0/1755857538 shutdown_connections 2023-10-08T07:14:37.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:37.767+0000 7fb89c761640 1 -- 172.21.15.70:0/1755857538 wait complete. 2023-10-08T07:14:37.785 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T07:14:37.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:14:37.909 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:14:38.027 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T07:14:38.027 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:14:38.027 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:14:38.087 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:14:38.088 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:14:38.088 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:14:38.147 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:14:38.147 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:14:38.148 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:14:38.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:14:38.213 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:14:38.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:14:38.215 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:14:38.215 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:14:38.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:14:38.272 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T07:14:38.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:14:38.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.477+0000 7f228dbf0640 1 Processor -- start 2023-10-08T07:14:38.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.477+0000 7f228dbf0640 1 -- start start 2023-10-08T07:14:38.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.479+0000 7f228dbf0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f22880cb580 0x7f2288165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:38.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.479+0000 7f228dbf0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228815b2b0 con 0x7f22880c90b0 2023-10-08T07:14:38.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.479+0000 7f228dbf0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228815b590 con 0x7f228815ab40 2023-10-08T07:14:38.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.479+0000 7f228dbf0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228815bf30 con 0x7f22880cb580 2023-10-08T07:14:38.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.479+0000 7f227ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f22880c90b0 0x7f22880cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39616/0 (socket says 172.21.15.70:39616) 2023-10-08T07:14:38.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.479+0000 7f227ffff640 1 -- 172.21.15.70:0/3306343505 learned_addr learned my addr 172.21.15.70:0/3306343505 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:38.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.479+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778636842 0 0) 0x7f228815b2b0 con 0x7f22880c90b0 2023-10-08T07:14:38.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.479+0000 7f2287fff640 1 --2- 172.21.15.70:0/3306343505 >> v2:172.21.15.181:3300/0 conn(0x7f22880cb580 0x7f2288165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:38.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.479+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2274003520 con 0x7f22880c90b0 2023-10-08T07:14:38.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1219327878 0 0) 0x7f2274003520 con 0x7f22880c90b0 2023-10-08T07:14:38.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 >> v2:172.21.15.181:3300/0 conn(0x7f22880cb580 msgr2=0x7f2288165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:38.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 --2- 172.21.15.70:0/3306343505 >> v2:172.21.15.181:3300/0 conn(0x7f22880cb580 0x7f2288165b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 >> v1:172.21.15.181:6789/0 conn(0x7f228815ab40 legacy=0x7f22880ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:38.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228815b870 con 0x7f22880c90b0 2023-10-08T07:14:38.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2270001670 con 0x7f22880c90b0 2023-10-08T07:14:38.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 >> v1:172.21.15.70:6789/0 conn(0x7f22880c90b0 legacy=0x7f22880cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:38.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2274003970 0x7f2274003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:38.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22740047c0 0x7f2274004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:38.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2274005180 0x7f227400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:38.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228815bf30 con 0x7f2274003970 2023-10-08T07:14:38.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227400c020 con 0x7f2274005180 2023-10-08T07:14:38.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227400c1e0 con 0x7f22740047c0 2023-10-08T07:14:38.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.480+0000 7f22877fe640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2274003970 0x7f2274003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:38.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.481+0000 7f2287fff640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2274005180 0x7f227400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:38.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.481+0000 7f22877fe640 1 -- 172.21.15.70:0/3306343505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22740047c0 msgr2=0x7f2274004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:38.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.481+0000 7f22877fe640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22740047c0 0x7f2274004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.481+0000 7f22877fe640 1 -- 172.21.15.70:0/3306343505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2274005180 msgr2=0x7f227400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:38.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.481+0000 7f22877fe640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2274005180 0x7f227400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.481+0000 7f22877fe640 1 -- 172.21.15.70:0/3306343505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274003520 con 0x7f2274003970 2023-10-08T07:14:38.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.481+0000 7f2287fff640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2274005180 0x7f227400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:38.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.481+0000 7f2287fff640 1 -- 172.21.15.70:0/3306343505 reap_dead start 2023-10-08T07:14:38.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.482+0000 7f22877fe640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2274003970 0x7f2274003d90 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f2278001750 tx=0x7f22740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518a0cf2b6892fd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:38.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.482+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22780052d0 con 0x7f2274003970 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.482+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227802e050 con 0x7f2274003970 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.482+0000 7f2286ffd640 1 -- 172.21.15.70:0/3306343505 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278003830 con 0x7f2274003970 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.483+0000 7f228dbf0640 1 -- 172.21.15.70:0/3306343505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2274003970 msgr2=0x7f2274003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.483+0000 7f228dbf0640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2274003970 0x7f2274003d90 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f2278001750 tx=0x7f22740057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.483+0000 7f228dbf0640 1 -- 172.21.15.70:0/3306343505 shutdown_connections 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.483+0000 7f228dbf0640 1 --2- 172.21.15.70:0/3306343505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2274003970 0x7f2274003d90 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.483+0000 7f228dbf0640 1 -- 172.21.15.70:0/3306343505 >> 172.21.15.70:0/3306343505 conn(0x7f228806a380 msgr2=0x7f22880b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.483+0000 7f228dbf0640 1 -- 172.21.15.70:0/3306343505 shutdown_connections 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.483+0000 7f228dbf0640 1 -- 172.21.15.70:0/3306343505 wait complete. 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.485+0000 7f228dbf0640 1 Processor -- start 2023-10-08T07:14:38.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.485+0000 7f228dbf0640 1 -- start start 2023-10-08T07:14:38.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f228dbf0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2274003970 0x7f22880b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:38.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f228dbf0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22880c7eb0 0x7f22880c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:38.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f228dbf0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2288141dd0 0x7f2288142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:38.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f228dbf0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228815f7a0 con 0x7f2288141dd0 2023-10-08T07:14:38.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f228dbf0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2288141b10 con 0x7f22880c7eb0 2023-10-08T07:14:38.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f228dbf0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22880c7160 con 0x7f2274003970 2023-10-08T07:14:38.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f2287fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2288141dd0 0x7f2288142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:38.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f2287fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2288141dd0 0x7f2288142250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36222/0 (socket says 172.21.15.70:36222) 2023-10-08T07:14:38.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f2287fff640 1 -- 172.21.15.70:0/3875147011 learned_addr learned my addr 172.21.15.70:0/3875147011 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:38.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f227ffff640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22880c7eb0 0x7f22880c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:38.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f2287fff640 1 -- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2274003970 msgr2=0x7f22880b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:38.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f2287fff640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2274003970 0x7f22880b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f2287fff640 1 -- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22880c7eb0 msgr2=0x7f22880c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:38.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f2287fff640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22880c7eb0 0x7f22880c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f2287fff640 1 -- 172.21.15.70:0/3875147011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880c73d0 con 0x7f2288141dd0 2023-10-08T07:14:38.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.486+0000 7f227ffff640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22880c7eb0 0x7f22880c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:38.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.487+0000 7f2287fff640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2288141dd0 0x7f2288142250 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f22880b0c20 tx=0x7f228000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:38.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.487+0000 7f2284ff9640 1 -- 172.21.15.70:0/3875147011 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280016420 con 0x7f2288141dd0 2023-10-08T07:14:38.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.487+0000 7f2284ff9640 1 -- 172.21.15.70:0/3875147011 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2280016580 con 0x7f2288141dd0 2023-10-08T07:14:38.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.487+0000 7f2284ff9640 1 -- 172.21.15.70:0/3875147011 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22800168f0 con 0x7f2288141dd0 2023-10-08T07:14:38.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.487+0000 7f228dbf0640 1 -- 172.21.15.70:0/3875147011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22880c76b0 con 0x7f2288141dd0 2023-10-08T07:14:38.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.487+0000 7f228dbf0640 1 -- 172.21.15.70:0/3875147011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22880b4200 con 0x7f2288141dd0 2023-10-08T07:14:38.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.488+0000 7f228dbf0640 1 -- 172.21.15.70:0/3875147011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2288067490 con 0x7f2288141dd0 2023-10-08T07:14:38.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.489+0000 7f2284ff9640 1 -- 172.21.15.70:0/3875147011 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f2280016a50 con 0x7f2288141dd0 2023-10-08T07:14:38.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.491+0000 7f2284ff9640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f226806b0b0 0x7f226806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:38.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.491+0000 7f2284ff9640 1 -- 172.21.15.70:0/3875147011 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f228008aa80 con 0x7f2288141dd0 2023-10-08T07:14:38.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.491+0000 7f22877fe640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f226806b0b0 0x7f226806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:38.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.492+0000 7f22877fe640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f226806b0b0 0x7f226806d570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2278033000 tx=0x7f227802f000 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:38.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.495+0000 7f2284ff9640 1 -- 172.21.15.70:0/3875147011 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2280055470 con 0x7f2288141dd0 2023-10-08T07:14:38.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.721+0000 7f228dbf0640 1 -- 172.21.15.70:0/3875147011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22880b0ef0 con 0x7f2288141dd0 2023-10-08T07:14:38.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.722+0000 7f2284ff9640 1 -- 172.21.15.70:0/3875147011 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+3705 (secure 0 0 0) 0x7f2280058a90 con 0x7f2288141dd0 2023-10-08T07:14:38.723 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:38.723 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":118,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7451,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1702770912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1702770912},{"type":"v1","addr":"172.21.15.181:6835","nonce":1702770912}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":7457,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/4215151739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":4215151739},{"type":"v1","addr":"172.21.15.70:6835","nonce":4215151739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8821,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/94813161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":94813161},{"type":"v1","addr":"172.21.15.181:6837","nonce":94813161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":118},{"gid":11730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/2038500071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2038500071},{"type":"v1","addr":"172.21.15.70:6834","nonce":2038500071}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[]} 2023-10-08T07:14:38.723 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 118 2023-10-08T07:14:38.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 -- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f226806b0b0 msgr2=0x7f226806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:38.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f226806b0b0 0x7f226806d570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2278033000 tx=0x7f227802f000 comp rx=0 tx=0).stop 2023-10-08T07:14:38.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 -- 172.21.15.70:0/3875147011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2288141dd0 msgr2=0x7f2288142250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:38.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2288141dd0 0x7f2288142250 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f22880b0c20 tx=0x7f228000f520 comp rx=0 tx=0).stop 2023-10-08T07:14:38.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 -- 172.21.15.70:0/3875147011 shutdown_connections 2023-10-08T07:14:38.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f226806b0b0 0x7f226806d570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2288141dd0 0x7f2288142250 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22880c7eb0 0x7f22880c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 --2- 172.21.15.70:0/3875147011 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2274003970 0x7f22880b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:38.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 -- 172.21.15.70:0/3875147011 >> 172.21.15.70:0/3875147011 conn(0x7f228806a380 msgr2=0x7f228806d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:38.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 -- 172.21.15.70:0/3875147011 shutdown_connections 2023-10-08T07:14:38.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:38.725+0000 7f228dbf0640 1 -- 172.21.15.70:0/3875147011 wait complete. 2023-10-08T07:14:38.743 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T07:14:38.744 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T07:14:38.744 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T07:14:38.744 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:14:38.744 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T07:14:38.744 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T07:14:38.744 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T07:14:38.745 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:14:38.745 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T07:14:38.745 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T07:14:38.745 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T07:14:38.745 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:14:38.746 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T07:14:38.746 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T07:14:38.746 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T07:14:38.746 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:14:38.760 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T07:14:38.760 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T07:14:38.761 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T07:14:38.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T07:14:38.772 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T07:14:38.772 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T07:14:38.773 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T07:14:38.773 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T07:14:38.809 INFO:tasks.ceph.mds.c:Started 2023-10-08T07:14:38.809 DEBUG:teuthology.parallel:result is None 2023-10-08T07:14:38.816 INFO:tasks.ceph.mds.a:Started 2023-10-08T07:14:38.817 DEBUG:teuthology.parallel:result is None 2023-10-08T07:14:38.821 INFO:tasks.ceph.mds.d:Started 2023-10-08T07:14:38.821 DEBUG:teuthology.parallel:result is None 2023-10-08T07:14:38.828 INFO:tasks.ceph.mds.b:Started 2023-10-08T07:14:38.828 DEBUG:teuthology.parallel:result is None 2023-10-08T07:14:38.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T07:14:38.969 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T07:14:38.969 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T07:14:38.989 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T07:14:39.002 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T07:14:39.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.018+0000 7fc438b63640 1 Processor -- start 2023-10-08T07:14:39.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.018+0000 7fc438b63640 1 -- start start 2023-10-08T07:14:39.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.019+0000 7fc438b63640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc43415ccb0 0x7fc4340d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.019+0000 7fc438b63640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc4340d0b90 0x7fc4340cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.019+0000 7fc438b63640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4340d0360 con 0x7fc4340cff10 2023-10-08T07:14:39.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.019+0000 7fc438b63640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4340d2870 con 0x7fc4340d0b90 2023-10-08T07:14:39.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.019+0000 7fc438b63640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4340d1150 con 0x7fc43415ccb0 2023-10-08T07:14:39.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.020+0000 7fc433fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc4340cff10 0x7fc4340d2b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39628/0 (socket says 172.21.15.70:39628) 2023-10-08T07:14:39.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.020+0000 7fc433fff640 1 -- 172.21.15.70:0/4066104944 learned_addr learned my addr 172.21.15.70:0/4066104944 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:39.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.020+0000 7fc42bfff640 1 --2- 172.21.15.70:0/4066104944 >> v2:172.21.15.70:3300/0 conn(0x7fc4340d0b90 0x7fc4340cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:39.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.020+0000 7fc4337fe640 1 --2- 172.21.15.70:0/4066104944 >> v2:172.21.15.181:3300/0 conn(0x7fc43415ccb0 0x7fc4340d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:39.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.020+0000 7fc432ffd640 1 -- 172.21.15.70:0/4066104944 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091034513 0 0) 0x7fc42c001490 con 0x7fc4340cff10 2023-10-08T07:14:39.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.020+0000 7fc42bfff640 1 -- 172.21.15.70:0/4066104944 >> v2:172.21.15.181:3300/0 conn(0x7fc43415ccb0 msgr2=0x7fc4340d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:39.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.020+0000 7fc42bfff640 1 --2- 172.21.15.70:0/4066104944 >> v2:172.21.15.181:3300/0 conn(0x7fc43415ccb0 0x7fc4340d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.020+0000 7fc42bfff640 1 -- 172.21.15.70:0/4066104944 >> v1:172.21.15.70:6789/0 conn(0x7fc4340cff10 legacy=0x7fc4340d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.020+0000 7fc42bfff640 1 -- 172.21.15.70:0/4066104944 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4340d1430 con 0x7fc4340d0b90 2023-10-08T07:14:39.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.021+0000 7fc42bfff640 1 --2- 172.21.15.70:0/4066104944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 0x7fc4340cf9d0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fc4340b51b0 tx=0x7fc41c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7da50d1bca0c85 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:39.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.022+0000 7fc432ffd640 1 -- 172.21.15.70:0/4066104944 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c02e070 con 0x7fc4340d0b90 2023-10-08T07:14:39.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.022+0000 7fc432ffd640 1 -- 172.21.15.70:0/4066104944 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c0315a0 con 0x7fc4340d0b90 2023-10-08T07:14:39.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.022+0000 7fc432ffd640 1 -- 172.21.15.70:0/4066104944 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c031930 con 0x7fc4340d0b90 2023-10-08T07:14:39.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.022+0000 7fc438b63640 1 -- 172.21.15.70:0/4066104944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 msgr2=0x7fc4340cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.022+0000 7fc438b63640 1 --2- 172.21.15.70:0/4066104944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 0x7fc4340cf9d0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fc4340b51b0 tx=0x7fc41c02b840 comp rx=0 tx=0).stop 2023-10-08T07:14:39.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.022+0000 7fc438b63640 1 -- 172.21.15.70:0/4066104944 shutdown_connections 2023-10-08T07:14:39.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.022+0000 7fc438b63640 1 --2- 172.21.15.70:0/4066104944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 0x7fc4340cf9d0 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.022+0000 7fc438b63640 1 --2- 172.21.15.70:0/4066104944 >> v2:172.21.15.181:3300/0 conn(0x7fc43415ccb0 0x7fc4340d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.023+0000 7fc438b63640 1 -- 172.21.15.70:0/4066104944 >> 172.21.15.70:0/4066104944 conn(0x7fc434071660 msgr2=0x7fc434071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:39.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.023+0000 7fc438b63640 1 -- 172.21.15.70:0/4066104944 shutdown_connections 2023-10-08T07:14:39.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.023+0000 7fc438b63640 1 -- 172.21.15.70:0/4066104944 wait complete. 2023-10-08T07:14:39.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.024+0000 7fc438b63640 1 Processor -- start 2023-10-08T07:14:39.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.025+0000 7fc438b63640 1 -- start start 2023-10-08T07:14:39.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.025+0000 7fc438b63640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4340cff10 0x7fc4341593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.025+0000 7fc438b63640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 0x7fc4340c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc438b63640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc43415ccb0 0x7fc4340b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc438b63640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4340c91c0 con 0x7fc4340d0b90 2023-10-08T07:14:39.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc438b63640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc434159920 con 0x7fc4340cff10 2023-10-08T07:14:39.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc438b63640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4340c8540 con 0x7fc43415ccb0 2023-10-08T07:14:39.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc42bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 0x7fc4340c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:39.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc42bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 0x7fc4340c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36254/0 (socket says 172.21.15.70:36254) 2023-10-08T07:14:39.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc42bfff640 1 -- 172.21.15.70:0/574486778 learned_addr learned my addr 172.21.15.70:0/574486778 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:39.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc433fff640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc43415ccb0 0x7fc4340b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:39.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc4337fe640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4340cff10 0x7fc4341593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:39.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc42bfff640 1 -- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc43415ccb0 msgr2=0x7fc4340b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc42bfff640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc43415ccb0 0x7fc4340b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc42bfff640 1 -- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4340cff10 msgr2=0x7fc4341593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc42bfff640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4340cff10 0x7fc4341593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.026+0000 7fc42bfff640 1 -- 172.21.15.70:0/574486778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c009400 con 0x7fc4340d0b90 2023-10-08T07:14:39.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.027+0000 7fc42bfff640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 0x7fc4340c7d10 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009890 tx=0x7fc41c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:39.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.027+0000 7fc430ff9640 1 -- 172.21.15.70:0/574486778 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c032840 con 0x7fc4340d0b90 2023-10-08T07:14:39.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.027+0000 7fc430ff9640 1 -- 172.21.15.70:0/574486778 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c031f00 con 0x7fc4340d0b90 2023-10-08T07:14:39.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.027+0000 7fc438b63640 1 -- 172.21.15.70:0/574486778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4340c8820 con 0x7fc4340d0b90 2023-10-08T07:14:39.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.027+0000 7fc430ff9640 1 -- 172.21.15.70:0/574486778 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c032be0 con 0x7fc4340d0b90 2023-10-08T07:14:39.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.027+0000 7fc438b63640 1 -- 172.21.15.70:0/574486778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4340c8de0 con 0x7fc4340d0b90 2023-10-08T07:14:39.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.028+0000 7fc438b63640 1 -- 172.21.15.70:0/574486778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4340c9300 con 0x7fc4340d0b90 2023-10-08T07:14:39.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.029+0000 7fc430ff9640 1 -- 172.21.15.70:0/574486778 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc41c032d40 con 0x7fc4340d0b90 2023-10-08T07:14:39.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.030+0000 7fc430ff9640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc41406b0b0 0x7fc41406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.030+0000 7fc430ff9640 1 -- 172.21.15.70:0/574486778 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fc41c0abdf0 con 0x7fc4340d0b90 2023-10-08T07:14:39.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.031+0000 7fc4337fe640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc41406b0b0 0x7fc41406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:39.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.032+0000 7fc4337fe640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc41406b0b0 0x7fc41406d570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc424009aa0 tx=0x7fc424009190 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:39.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.035+0000 7fc430ff9640 1 -- 172.21.15.70:0/574486778 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4340c9300 con 0x7fc4340d0b90 2023-10-08T07:14:39.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.229+0000 7fc438b63640 1 -- 172.21.15.70:0/574486778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc4340c9300 con 0x7fc4340d0b90 2023-10-08T07:14:39.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.332+0000 7fc430ff9640 1 -- 172.21.15.70:0/574486778 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v249) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc4340c9300 con 0x7fc4340d0b90 2023-10-08T07:14:39.333 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T07:14:39.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.335+0000 7fc438b63640 1 -- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc41406b0b0 msgr2=0x7fc41406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.335+0000 7fc438b63640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc41406b0b0 0x7fc41406d570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc424009aa0 tx=0x7fc424009190 comp rx=0 tx=0).stop 2023-10-08T07:14:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.335+0000 7fc438b63640 1 -- 172.21.15.70:0/574486778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 msgr2=0x7fc4340c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.335+0000 7fc438b63640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 0x7fc4340c7d10 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009890 tx=0x7fc41c0046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.335+0000 7fc438b63640 1 -- 172.21.15.70:0/574486778 shutdown_connections 2023-10-08T07:14:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.335+0000 7fc438b63640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc41406b0b0 0x7fc41406d570 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.335+0000 7fc438b63640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc43415ccb0 0x7fc4340b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.335+0000 7fc438b63640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4340d0b90 0x7fc4340c7d10 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.335+0000 7fc438b63640 1 --2- 172.21.15.70:0/574486778 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4340cff10 0x7fc4341593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.335+0000 7fc438b63640 1 -- 172.21.15.70:0/574486778 >> 172.21.15.70:0/574486778 conn(0x7fc434071660 msgr2=0x7fc4340cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.336+0000 7fc438b63640 1 -- 172.21.15.70:0/574486778 shutdown_connections 2023-10-08T07:14:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.336+0000 7fc438b63640 1 -- 172.21.15.70:0/574486778 wait complete. 2023-10-08T07:14:39.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T07:14:39.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.539+0000 7f27d6402640 1 Processor -- start 2023-10-08T07:14:39.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.539+0000 7f27d6402640 1 -- start start 2023-10-08T07:14:39.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.540+0000 7f27d6402640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f27d00ad790 0x7f27d00702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.540+0000 7f27d6402640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d006dee0 con 0x7f27d0159ac0 2023-10-08T07:14:39.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.540+0000 7f27d6402640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27d015d850 con 0x7f27d015e330 2023-10-08T07:14:39.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.540+0000 7f27d6402640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27d006dcf0 con 0x7f27d00ad790 2023-10-08T07:14:39.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.541+0000 7f27c7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f27d00ad790 0x7f27d00702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:39.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.541+0000 7f27cffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f27d0159ac0 0x7f27d006d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39642/0 (socket says 172.21.15.70:39642) 2023-10-08T07:14:39.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.541+0000 7f27c7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f27d00ad790 0x7f27d00702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36268/0 (socket says 172.21.15.70:36268) 2023-10-08T07:14:39.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.541+0000 7f27cffff640 1 -- 172.21.15.70:0/683046774 learned_addr learned my addr 172.21.15.70:0/683046774 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:39.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.541+0000 7f27c7fff640 1 -- 172.21.15.70:0/683046774 >> v1:172.21.15.181:6789/0 conn(0x7f27d015e330 legacy=0x7f27d00c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:39.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.541+0000 7f27c7fff640 1 -- 172.21.15.70:0/683046774 >> v1:172.21.15.70:6789/0 conn(0x7f27d0159ac0 legacy=0x7f27d006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.541+0000 7f27c7fff640 1 -- 172.21.15.70:0/683046774 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d015db30 con 0x7f27d00ad790 2023-10-08T07:14:39.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.542+0000 7f27c7fff640 1 --2- 172.21.15.70:0/683046774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00ad790 0x7f27d00702d0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f27b80097a0 tx=0x7f27b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b743496c6a77f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:39.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.543+0000 7f27cf7fe640 1 -- 172.21.15.70:0/683046774 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8031830 con 0x7f27d00ad790 2023-10-08T07:14:39.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.543+0000 7f27cf7fe640 1 -- 172.21.15.70:0/683046774 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b8031990 con 0x7f27d00ad790 2023-10-08T07:14:39.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.543+0000 7f27cf7fe640 1 -- 172.21.15.70:0/683046774 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8031d20 con 0x7f27d00ad790 2023-10-08T07:14:39.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.543+0000 7f27d6402640 1 -- 172.21.15.70:0/683046774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00ad790 msgr2=0x7f27d00702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.543+0000 7f27d6402640 1 --2- 172.21.15.70:0/683046774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00ad790 0x7f27d00702d0 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7f27b80097a0 tx=0x7f27b802b8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.543+0000 7f27d6402640 1 -- 172.21.15.70:0/683046774 shutdown_connections 2023-10-08T07:14:39.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.543+0000 7f27d6402640 1 --2- 172.21.15.70:0/683046774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d00ad790 0x7f27d00702d0 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.543+0000 7f27d6402640 1 -- 172.21.15.70:0/683046774 >> 172.21.15.70:0/683046774 conn(0x7f27d006a380 msgr2=0x7f27d00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:39.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.543+0000 7f27d6402640 1 -- 172.21.15.70:0/683046774 shutdown_connections 2023-10-08T07:14:39.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.543+0000 7f27d6402640 1 -- 172.21.15.70:0/683046774 wait complete. 2023-10-08T07:14:39.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.545+0000 7f27d6402640 1 Processor -- start 2023-10-08T07:14:39.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.545+0000 7f27d6402640 1 -- start start 2023-10-08T07:14:39.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.546+0000 7f27d6402640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00ad790 0x7f27d014b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.546+0000 7f27d6402640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d0159ac0 0x7f27d015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.546+0000 7f27d6402640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d015e330 0x7f27d00c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.546+0000 7f27d6402640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d015bfd0 con 0x7f27d0159ac0 2023-10-08T07:14:39.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.546+0000 7f27d6402640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d00b1780 con 0x7f27d00ad790 2023-10-08T07:14:39.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.546+0000 7f27d6402640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d015b070 con 0x7f27d015e330 2023-10-08T07:14:39.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.546+0000 7f27c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d0159ac0 0x7f27d015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.546+0000 7f27c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d0159ac0 0x7f27d015ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36278/0 (socket says 172.21.15.70:36278) 2023-10-08T07:14:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.546+0000 7f27c7fff640 1 -- 172.21.15.70:0/4266927008 learned_addr learned my addr 172.21.15.70:0/4266927008 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27cffff640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00ad790 0x7f27d014b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27c7fff640 1 -- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d015e330 msgr2=0x7f27d00c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27c7fff640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d015e330 0x7f27d00c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27c7fff640 1 -- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00ad790 msgr2=0x7f27d014b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27c7fff640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00ad790 0x7f27d014b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27c7fff640 1 -- 172.21.15.70:0/4266927008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8009400 con 0x7f27d0159ac0 2023-10-08T07:14:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27cffff640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00ad790 0x7f27d014b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27c7fff640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d0159ac0 0x7f27d015ca20 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f27b80098c0 tx=0x7f27b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27cd7fa640 1 -- 172.21.15.70:0/4266927008 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b803ba70 con 0x7f27d0159ac0 2023-10-08T07:14:39.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27cd7fa640 1 -- 172.21.15.70:0/4266927008 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b803bbf0 con 0x7f27d0159ac0 2023-10-08T07:14:39.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27cd7fa640 1 -- 172.21.15.70:0/4266927008 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b80453f0 con 0x7f27d0159ac0 2023-10-08T07:14:39.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.547+0000 7f27d6402640 1 -- 172.21.15.70:0/4266927008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d015b350 con 0x7f27d0159ac0 2023-10-08T07:14:39.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.548+0000 7f27d6402640 1 -- 172.21.15.70:0/4266927008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d015b910 con 0x7f27d0159ac0 2023-10-08T07:14:39.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.549+0000 7f27d6402640 1 -- 172.21.15.70:0/4266927008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d015c110 con 0x7f27d0159ac0 2023-10-08T07:14:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.549+0000 7f27cd7fa640 1 -- 172.21.15.70:0/4266927008 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f27b804e450 con 0x7f27d0159ac0 2023-10-08T07:14:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.551+0000 7f27cd7fa640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27b006b0e0 0x7f27b006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.551+0000 7f27cd7fa640 1 -- 172.21.15.70:0/4266927008 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f27b80abcc0 con 0x7f27d0159ac0 2023-10-08T07:14:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.552+0000 7f27cffff640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27b006b0e0 0x7f27b006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:39.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.553+0000 7f27cffff640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27b006b0e0 0x7f27b006d5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005880 tx=0x7f27c0009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:39.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.555+0000 7f27cd7fa640 1 -- 172.21.15.70:0/4266927008 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27d015c110 con 0x7f27d0159ac0 2023-10-08T07:14:39.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.793+0000 7f27d6402640 1 -- 172.21.15.70:0/4266927008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f27d015c110 con 0x7f27d0159ac0 2023-10-08T07:14:39.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.794+0000 7f27cd7fa640 1 -- 172.21.15.70:0/4266927008 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v52) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f27d015c110 con 0x7f27d0159ac0 2023-10-08T07:14:39.795 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:39.795 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:14:39.796 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T07:14:39.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.796 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T07:14:39.796 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T07:14:39.797 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.797 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:14:39.797 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:14:39.797 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:14:39.797 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.798 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.798 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T07:14:39.799 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T07:14:39.799 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.799 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:14:39.799 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:14:39.799 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:14:39.800 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.800 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.801 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T07:14:39.801 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T07:14:39.801 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.801 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:14:39.801 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:14:39.801 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:14:39.802 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.802 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.803 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T07:14:39.803 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T07:14:39.803 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.804 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:14:39.804 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:14:39.804 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:14:39.804 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.805 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T07:14:39.805 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T07:14:39.805 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.805 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:14:39.805 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:14:39.805 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.805 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.806 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T07:14:39.806 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T07:14:39.806 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.806 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:14:39.806 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:14:39.807 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.807 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.807 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T07:14:39.807 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T07:14:39.807 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.808 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:14:39.808 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:14:39.808 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.808 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.808 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T07:14:39.809 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T07:14:39.809 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.809 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:14:39.809 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:14:39.809 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.810 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T07:14:39.810 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T07:14:39.810 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.810 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:14:39.810 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:14:39.811 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.811 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.811 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T07:14:39.811 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T07:14:39.811 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:14:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:14:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T07:14:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T07:14:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:14:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:14:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T07:14:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T07:14:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:14:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:14:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T07:14:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T07:14:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:14:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T07:14:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T07:14:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T07:14:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T07:14:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:14:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:14:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:14:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:14:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T07:14:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T07:14:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:14:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:14:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:14:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:14:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T07:14:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T07:14:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:14:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:14:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:14:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:14:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T07:14:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T07:14:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:14:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:14:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:14:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T07:14:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T07:14:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:14:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T07:14:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T07:14:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T07:14:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:14:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:14:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T07:14:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:14:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T07:14:39.825 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:14:39.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.796+0000 7f27d6402640 1 -- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27b006b0e0 msgr2=0x7f27b006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.796+0000 7f27d6402640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27b006b0e0 0x7f27b006d5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005880 tx=0x7f27c0009040 comp rx=0 tx=0).stop 2023-10-08T07:14:39.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.796+0000 7f27d6402640 1 -- 172.21.15.70:0/4266927008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d0159ac0 msgr2=0x7f27d015ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:39.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.797+0000 7f27d6402640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d0159ac0 0x7f27d015ca20 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f27b80098c0 tx=0x7f27b80046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.797+0000 7f27d6402640 1 -- 172.21.15.70:0/4266927008 shutdown_connections 2023-10-08T07:14:39.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.797+0000 7f27d6402640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27b006b0e0 0x7f27b006d5a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.797+0000 7f27d6402640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27d015e330 0x7f27d00c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.797+0000 7f27d6402640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27d0159ac0 0x7f27d015ca20 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.797+0000 7f27d6402640 1 --2- 172.21.15.70:0/4266927008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27d00ad790 0x7f27d014b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.797+0000 7f27d6402640 1 -- 172.21.15.70:0/4266927008 >> 172.21.15.70:0/4266927008 conn(0x7f27d006a380 msgr2=0x7f27d00ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.797+0000 7f27d6402640 1 -- 172.21.15.70:0/4266927008 shutdown_connections 2023-10-08T07:14:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:39.797+0000 7f27d6402640 1 -- 172.21.15.70:0/4266927008 wait complete. 2023-10-08T07:14:39.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.044+0000 7fbdf7516640 1 Processor -- start 2023-10-08T07:14:40.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.044+0000 7fbdf7516640 1 -- start start 2023-10-08T07:14:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.046+0000 7fbdf7516640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf015ab40 0x7fbdf00ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.046+0000 7fbdf7516640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf015b2b0 con 0x7fbdf00c90b0 2023-10-08T07:14:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.046+0000 7fbdf7516640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf015b590 con 0x7fbdf00cb580 2023-10-08T07:14:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.046+0000 7fbdf7516640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf015bf30 con 0x7fbdf015ab40 2023-10-08T07:14:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.046+0000 7fbdf4ab1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbdf00c90b0 0x7fbdf00cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39650/0 (socket says 172.21.15.70:39650) 2023-10-08T07:14:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.046+0000 7fbdf4ab1640 1 -- 172.21.15.70:0/2364563652 learned_addr learned my addr 172.21.15.70:0/2364563652 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:40.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbdf52b2640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf015ab40 0x7fbdf00ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911826366 0 0) 0x7fbdf015b2b0 con 0x7fbdf00c90b0 2023-10-08T07:14:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbddc003520 con 0x7fbdf00c90b0 2023-10-08T07:14:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 468154190 0 0) 0x7fbddc003520 con 0x7fbdf00c90b0 2023-10-08T07:14:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf015ab40 msgr2=0x7fbdf00ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf015ab40 0x7fbdf00ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 >> v1:172.21.15.181:6789/0 conn(0x7fbdf00cb580 legacy=0x7fbdf015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf015b870 con 0x7fbdf00c90b0 2023-10-08T07:14:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fbde0001670 con 0x7fbdf00c90b0 2023-10-08T07:14:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 >> v1:172.21.15.70:6789/0 conn(0x7fbdf00c90b0 legacy=0x7fbdf00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf015ab40 0x7fbddc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbddc003f70 0x7fbddc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbddc004d00 0x7fbddc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf015bf30 con 0x7fbddc003f70 2023-10-08T07:14:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc00bac0 con 0x7fbddc004d00 2023-10-08T07:14:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.047+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc00bc80 con 0x7fbdf015ab40 2023-10-08T07:14:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.048+0000 7fbdf4ab1640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbddc003f70 0x7fbddc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.048+0000 7fbdf5ab3640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbddc004d00 0x7fbddc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.048+0000 7fbdf4ab1640 1 -- 172.21.15.70:0/2364563652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf015ab40 msgr2=0x7fbddc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:40.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.048+0000 7fbdf4ab1640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbdf015ab40 0x7fbddc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.048+0000 7fbdf4ab1640 1 -- 172.21.15.70:0/2364563652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbddc004d00 msgr2=0x7fbddc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.048+0000 7fbdf4ab1640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbddc004d00 0x7fbddc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.048+0000 7fbdf4ab1640 1 -- 172.21.15.70:0/2364563652 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc003520 con 0x7fbddc003f70 2023-10-08T07:14:40.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.049+0000 7fbdf4ab1640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbddc003f70 0x7fbddc0047c0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7fbde000c0e0 tx=0x7fbddc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb649f8f28e3c58 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:40.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.049+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0005ad0 con 0x7fbddc003f70 2023-10-08T07:14:40.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.049+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde0005c30 con 0x7fbddc003f70 2023-10-08T07:14:40.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.049+0000 7fbde7fff640 1 -- 172.21.15.70:0/2364563652 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde00072b0 con 0x7fbddc003f70 2023-10-08T07:14:40.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.050+0000 7fbdf7516640 1 -- 172.21.15.70:0/2364563652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbddc003f70 msgr2=0x7fbddc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.050+0000 7fbdf7516640 1 --2- 172.21.15.70:0/2364563652 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbddc003f70 0x7fbddc0047c0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7fbde000c0e0 tx=0x7fbddc0057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.050+0000 7fbdf5ab3640 1 -- 172.21.15.70:0/2364563652 reap_dead start 2023-10-08T07:14:40.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.050+0000 7fbdf7516640 1 -- 172.21.15.70:0/2364563652 shutdown_connections 2023-10-08T07:14:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.050+0000 7fbdf7516640 1 -- 172.21.15.70:0/2364563652 >> 172.21.15.70:0/2364563652 conn(0x7fbdf006a380 msgr2=0x7fbdf00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.050+0000 7fbdf7516640 1 -- 172.21.15.70:0/2364563652 shutdown_connections 2023-10-08T07:14:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.050+0000 7fbdf7516640 1 -- 172.21.15.70:0/2364563652 wait complete. 2023-10-08T07:14:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.052+0000 7fbdf7516640 1 Processor -- start 2023-10-08T07:14:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.052+0000 7fbdf7516640 1 -- start start 2023-10-08T07:14:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.053+0000 7fbdf7516640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbddc003f70 0x7fbdf00c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.053+0000 7fbdf7516640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf00c81c0 0x7fbdf006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.053+0000 7fbdf7516640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf015d390 0x7fbdf015d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.053+0000 7fbdf7516640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf015dd30 con 0x7fbdf00c81c0 2023-10-08T07:14:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.053+0000 7fbdf7516640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00ac820 con 0x7fbdf015d390 2023-10-08T07:14:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.053+0000 7fbdf7516640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf00c7760 con 0x7fbddc003f70 2023-10-08T07:14:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.053+0000 7fbdf4ab1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf00c81c0 0x7fbdf006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.053+0000 7fbdf4ab1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf00c81c0 0x7fbdf006d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36296/0 (socket says 172.21.15.70:36296) 2023-10-08T07:14:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.053+0000 7fbdf4ab1640 1 -- 172.21.15.70:0/1259483039 learned_addr learned my addr 172.21.15.70:0/1259483039 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbdf52b2640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbddc003f70 0x7fbdf00c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbdf4ab1640 1 -- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbddc003f70 msgr2=0x7fbdf00c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbdf4ab1640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbddc003f70 0x7fbdf00c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbdf5ab3640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf015d390 0x7fbdf015d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbdf4ab1640 1 -- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf015d390 msgr2=0x7fbdf015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbdf4ab1640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf015d390 0x7fbdf015d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbdf4ab1640 1 -- 172.21.15.70:0/1259483039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf00abca0 con 0x7fbdf00c81c0 2023-10-08T07:14:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbdf4ab1640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf00c81c0 0x7fbdf006d3d0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fbde000c0e0 tx=0x7fbde0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbde5ffb640 1 -- 172.21.15.70:0/1259483039 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde003c040 con 0x7fbdf00c81c0 2023-10-08T07:14:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbde5ffb640 1 -- 172.21.15.70:0/1259483039 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde0007580 con 0x7fbdf00c81c0 2023-10-08T07:14:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbde5ffb640 1 -- 172.21.15.70:0/1259483039 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde002c740 con 0x7fbdf00c81c0 2023-10-08T07:14:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbdf7516640 1 -- 172.21.15.70:0/1259483039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf00abf80 con 0x7fbdf00c81c0 2023-10-08T07:14:40.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.054+0000 7fbdf7516640 1 -- 172.21.15.70:0/1259483039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf00ac540 con 0x7fbdf00c81c0 2023-10-08T07:14:40.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.055+0000 7fbdf7516640 1 -- 172.21.15.70:0/1259483039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf0067490 con 0x7fbdf00c81c0 2023-10-08T07:14:40.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.056+0000 7fbde5ffb640 1 -- 172.21.15.70:0/1259483039 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbde0032070 con 0x7fbdf00c81c0 2023-10-08T07:14:40.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.057+0000 7fbde5ffb640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbdd406b0b0 0x7fbdd406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.057+0000 7fbde5ffb640 1 -- 172.21.15.70:0/1259483039 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fbde00aa9d0 con 0x7fbdf00c81c0 2023-10-08T07:14:40.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.058+0000 7fbdf52b2640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbdd406b0b0 0x7fbdd406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.059+0000 7fbdf52b2640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbdd406b0b0 0x7fbdd406d570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0073f0 tx=0x7fbdec00b040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:40.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.061+0000 7fbde5ffb640 1 -- 172.21.15.70:0/1259483039 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbde00758b0 con 0x7fbdf00c81c0 2023-10-08T07:14:40.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.251+0000 7fbdf7516640 1 -- 172.21.15.70:0/1259483039 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdf00b0ef0 con 0x7fbdf00c81c0 2023-10-08T07:14:40.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.253+0000 7fbde5ffb640 1 -- 172.21.15.70:0/1259483039 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v250) v1 ==== 74+0+13246 (secure 0 0 0) 0x7fbde0078ed0 con 0x7fbdf00c81c0 2023-10-08T07:14:40.254 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:40.254 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":250,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:39.335938+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":236,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:40.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 -- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbdd406b0b0 msgr2=0x7fbdd406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbdd406b0b0 0x7fbdd406d570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0073f0 tx=0x7fbdec00b040 comp rx=0 tx=0).stop 2023-10-08T07:14:40.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 -- 172.21.15.70:0/1259483039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf00c81c0 msgr2=0x7fbdf006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf00c81c0 0x7fbdf006d3d0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7fbde000c0e0 tx=0x7fbde0001930 comp rx=0 tx=0).stop 2023-10-08T07:14:40.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 -- 172.21.15.70:0/1259483039 shutdown_connections 2023-10-08T07:14:40.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbdd406b0b0 0x7fbdd406d570 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbdf015d390 0x7fbdf015d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbdf00c81c0 0x7fbdf006d3d0 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 --2- 172.21.15.70:0/1259483039 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbddc003f70 0x7fbdf00c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 -- 172.21.15.70:0/1259483039 >> 172.21.15.70:0/1259483039 conn(0x7fbdf006a380 msgr2=0x7fbdf00ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:40.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 -- 172.21.15.70:0/1259483039 shutdown_connections 2023-10-08T07:14:40.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.256+0000 7fbdf7516640 1 -- 172.21.15.70:0/1259483039 wait complete. 2023-10-08T07:14:40.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:14:40.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:14:40.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.492+0000 7ffaa218a640 1 Processor -- start 2023-10-08T07:14:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.493+0000 7ffaa218a640 1 -- start start 2023-10-08T07:14:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.494+0000 7ffaa218a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146d10 0x7ffa9c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.494+0000 7ffaa218a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa9c144020 0x7ffa9c164b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.494+0000 7ffaa218a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c1650b0 con 0x7ffa9c146930 2023-10-08T07:14:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.494+0000 7ffaa218a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c165b80 con 0x7ffa9c144020 2023-10-08T07:14:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.494+0000 7ffaa218a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c162540 con 0x7ffa9c146d10 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.495+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146d10 0x7ffa9c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.495+0000 7ffa9bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa9c144020 0x7ffa9c164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.495+0000 7ffa9bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffa9c144020 0x7ffa9c164b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:60190/0 (socket says 172.21.15.70:60190) 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.495+0000 7ffa9bfff640 1 -- 172.21.15.70:0/4056499466 learned_addr learned my addr 172.21.15.70:0/4056499466 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.495+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146d10 0x7ffa9c162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:53622/0 (socket says 172.21.15.70:53622) 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.496+0000 7ffa9bfff640 1 -- 172.21.15.70:0/4056499466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146d10 msgr2=0x7ffa9c162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.496+0000 7ffa9bfff640 1 --2- 172.21.15.70:0/4056499466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146d10 0x7ffa9c162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.496+0000 7ffa9bfff640 1 -- 172.21.15.70:0/4056499466 >> v1:172.21.15.181:6789/0 conn(0x7ffa9c146930 legacy=0x7ffa9c143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.496+0000 7ffa9bfff640 1 -- 172.21.15.70:0/4056499466 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c06c9d0 con 0x7ffa9c144020 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.498+0000 7ffa9bfff640 1 --2- 172.21.15.70:0/4056499466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa9c144020 0x7ffa9c164b70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009e50 tx=0x7ffa9402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c0725d0426afe2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.498+0000 7ffa9affd640 1 -- 172.21.15.70:0/4056499466 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9402cb00 con 0x7ffa9c144020 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.498+0000 7ffa9affd640 1 -- 172.21.15.70:0/4056499466 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9402cc60 con 0x7ffa9c144020 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.498+0000 7ffa9affd640 1 -- 172.21.15.70:0/4056499466 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa940343f0 con 0x7ffa9c144020 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.499+0000 7ffaa218a640 1 -- 172.21.15.70:0/4056499466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa9c144020 msgr2=0x7ffa9c164b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.499+0000 7ffaa218a640 1 --2- 172.21.15.70:0/4056499466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa9c144020 0x7ffa9c164b70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009e50 tx=0x7ffa9402bf60 comp rx=0 tx=0).stop 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.499+0000 7ffaa218a640 1 -- 172.21.15.70:0/4056499466 shutdown_connections 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.499+0000 7ffaa218a640 1 --2- 172.21.15.70:0/4056499466 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa9c144020 0x7ffa9c164b70 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.499+0000 7ffaa218a640 1 --2- 172.21.15.70:0/4056499466 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146d10 0x7ffa9c162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.499+0000 7ffaa218a640 1 -- 172.21.15.70:0/4056499466 >> 172.21.15.70:0/4056499466 conn(0x7ffa9c069d50 msgr2=0x7ffa9c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.499+0000 7ffaa218a640 1 -- 172.21.15.70:0/4056499466 shutdown_connections 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.499+0000 7ffaa218a640 1 -- 172.21.15.70:0/4056499466 wait complete. 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.501+0000 7ffaa218a640 1 Processor -- start 2023-10-08T07:14:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.502+0000 7ffaa218a640 1 -- start start 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.503+0000 7ffaa218a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa9c144020 0x7ffa9c13c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.503+0000 7ffaa218a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146930 0x7ffa9c163110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.503+0000 7ffaa218a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9c146d10 0x7ffa9c163650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.503+0000 7ffaa218a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c159030 con 0x7ffa9c146d10 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.503+0000 7ffaa218a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c145570 con 0x7ffa9c144020 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.503+0000 7ffaa218a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c145850 con 0x7ffa9c146930 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.503+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9c146d10 0x7ffa9c163650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.503+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9c146d10 0x7ffa9c163650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36302/0 (socket says 172.21.15.70:36302) 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.503+0000 7ffa9bfff640 1 -- 172.21.15.70:0/3081331981 learned_addr learned my addr 172.21.15.70:0/3081331981 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.503+0000 7ffa9b7fe640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa9c144020 0x7ffa9c13c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.504+0000 7ffa93fff640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146930 0x7ffa9c163110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.504+0000 7ffa9bfff640 1 -- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146930 msgr2=0x7ffa9c163110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.504+0000 7ffa9bfff640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146930 0x7ffa9c163110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.504+0000 7ffa9bfff640 1 -- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa9c144020 msgr2=0x7ffa9c13c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.504+0000 7ffa9bfff640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa9c144020 0x7ffa9c13c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.504+0000 7ffa9bfff640 1 -- 172.21.15.70:0/3081331981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa94009b30 con 0x7ffa9c146d10 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.504+0000 7ffa9b7fe640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa9c144020 0x7ffa9c13c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.504+0000 7ffa9bfff640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9c146d10 0x7ffa9c163650 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009e50 tx=0x7ffa94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.504+0000 7ffa98ff9640 1 -- 172.21.15.70:0/3081331981 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94009510 con 0x7ffa9c146d10 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.504+0000 7ffa98ff9640 1 -- 172.21.15.70:0/3081331981 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9403e450 con 0x7ffa9c146d10 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.505+0000 7ffaa218a640 1 -- 172.21.15.70:0/3081331981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c145b30 con 0x7ffa9c146d10 2023-10-08T07:14:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.505+0000 7ffa98ff9640 1 -- 172.21.15.70:0/3081331981 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9403e800 con 0x7ffa9c146d10 2023-10-08T07:14:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.505+0000 7ffaa218a640 1 -- 172.21.15.70:0/3081331981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c144840 con 0x7ffa9c146d10 2023-10-08T07:14:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.506+0000 7ffaa218a640 1 -- 172.21.15.70:0/3081331981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9c1591a0 con 0x7ffa9c146d10 2023-10-08T07:14:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.507+0000 7ffa98ff9640 1 -- 172.21.15.70:0/3081331981 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ffa9403e960 con 0x7ffa9c146d10 2023-10-08T07:14:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.508+0000 7ffa98ff9640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ffa7c06b0b0 0x7ffa7c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.508+0000 7ffa98ff9640 1 -- 172.21.15.70:0/3081331981 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5293+0+0 (secure 0 0 0) 0x7ffa940ac590 con 0x7ffa9c146d10 2023-10-08T07:14:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.508+0000 7ffa9b7fe640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ffa7c06b0b0 0x7ffa7c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.510+0000 7ffa9b7fe640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ffa7c06b0b0 0x7ffa7c06d570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c007fd0 tx=0x7ffa8c009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.512+0000 7ffa98ff9640 1 -- 172.21.15.70:0/3081331981 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa9c1591a0 con 0x7ffa9c146d10 2023-10-08T07:14:40.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.688+0000 7ffaa218a640 1 -- 172.21.15.70:0/3081331981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa9c04e160 con 0x7ffa9c146d10 2023-10-08T07:14:40.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.689+0000 7ffa98ff9640 1 -- 172.21.15.70:0/3081331981 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v250) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa94077480 con 0x7ffa9c146d10 2023-10-08T07:14:40.690 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:14:40.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 -- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ffa7c06b0b0 msgr2=0x7ffa7c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ffa7c06b0b0 0x7ffa7c06d570 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c007fd0 tx=0x7ffa8c009040 comp rx=0 tx=0).stop 2023-10-08T07:14:40.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 -- 172.21.15.70:0/3081331981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9c146d10 msgr2=0x7ffa9c163650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9c146d10 0x7ffa9c163650 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009e50 tx=0x7ffa94004a40 comp rx=0 tx=0).stop 2023-10-08T07:14:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 -- 172.21.15.70:0/3081331981 shutdown_connections 2023-10-08T07:14:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ffa7c06b0b0 0x7ffa7c06d570 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa9c146d10 0x7ffa9c163650 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffa9c146930 0x7ffa9c163110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 --2- 172.21.15.70:0/3081331981 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffa9c144020 0x7ffa9c13c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 -- 172.21.15.70:0/3081331981 >> 172.21.15.70:0/3081331981 conn(0x7ffa9c069d50 msgr2=0x7ffa9c0b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 -- 172.21.15.70:0/3081331981 shutdown_connections 2023-10-08T07:14:40.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.692+0000 7ffaa218a640 1 -- 172.21.15.70:0/3081331981 wait complete. 2023-10-08T07:14:40.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T07:14:40.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.917+0000 7f57632f3640 1 Processor -- start 2023-10-08T07:14:40.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.917+0000 7f57632f3640 1 -- start start 2023-10-08T07:14:40.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.918+0000 7f57632f3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0adc70 0x7f575c0b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.918+0000 7f57632f3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f575c15f7d0 con 0x7f575c15ab30 2023-10-08T07:14:40.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.918+0000 7f57632f3640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f575c15efb0 con 0x7f575c15af10 2023-10-08T07:14:40.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.918+0000 7f57632f3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f575c15f210 con 0x7f575c0adc70 2023-10-08T07:14:40.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f57622f1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f575c15ab30 0x7f575c15e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39656/0 (socket says 172.21.15.70:39656) 2023-10-08T07:14:40.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f57622f1640 1 -- 172.21.15.70:0/1471520157 learned_addr learned my addr 172.21.15.70:0/1471520157 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759212133 0 0) 0x7f575c15f7d0 con 0x7f575c15ab30 2023-10-08T07:14:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f5761af0640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0adc70 0x7f575c0b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5748003520 con 0x7f575c15ab30 2023-10-08T07:14:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3195843224 0 0) 0x7f5748003520 con 0x7f575c15ab30 2023-10-08T07:14:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0adc70 msgr2=0x7f575c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f57612ef640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0adc70 0x7f575c0b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 >> v1:172.21.15.181:6789/0 conn(0x7f575c15af10 legacy=0x7f575c06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c15f4f0 con 0x7f575c15ab30 2023-10-08T07:14:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.919+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5758001690 con 0x7f575c15ab30 2023-10-08T07:14:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 >> v1:172.21.15.70:6789/0 conn(0x7f575c15ab30 legacy=0x7f575c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f57612ef640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0adc70 0x7f57480039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f57612ef640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 0x7f57480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f57612ef640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5748004d00 0x7f5748005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c15f210 con 0x7f5748003fd0 2023-10-08T07:14:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574800bb40 con 0x7f5748004d00 2023-10-08T07:14:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574800bd00 con 0x7f575c0adc70 2023-10-08T07:14:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f5761af0640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 0x7f57480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f57622f1640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0adc70 0x7f57480039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f5761af0640 1 -- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0adc70 msgr2=0x7f57480039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:40.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f5761af0640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0adc70 0x7f57480039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f5761af0640 1 -- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5748004d00 msgr2=0x7f5748005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:40.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f5761af0640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5748004d00 0x7f5748005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f5761af0640 1 -- 172.21.15.70:0/1471520157 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5748003520 con 0x7f5748003fd0 2023-10-08T07:14:40.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.920+0000 7f57622f1640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0adc70 0x7f57480039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:40.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.921+0000 7f5761af0640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 0x7f57480047c0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f574c00ba30 tx=0x7f57480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea1619faf9ce263 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:40.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.921+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c032640 con 0x7f5748003fd0 2023-10-08T07:14:40.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.922+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c0327a0 con 0x7f5748003fd0 2023-10-08T07:14:40.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.922+0000 7f57612ef640 1 -- 172.21.15.70:0/1471520157 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c032b30 con 0x7f5748003fd0 2023-10-08T07:14:40.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.922+0000 7f57632f3640 1 -- 172.21.15.70:0/1471520157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 msgr2=0x7f57480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.922+0000 7f57632f3640 1 --2- 172.21.15.70:0/1471520157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 0x7f57480047c0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f574c00ba30 tx=0x7f57480057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.922+0000 7f5762af2640 1 -- 172.21.15.70:0/1471520157 reap_dead start 2023-10-08T07:14:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.922+0000 7f57632f3640 1 -- 172.21.15.70:0/1471520157 shutdown_connections 2023-10-08T07:14:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.922+0000 7f57632f3640 1 -- 172.21.15.70:0/1471520157 >> 172.21.15.70:0/1471520157 conn(0x7f575c0cb0c0 msgr2=0x7f575c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.922+0000 7f57632f3640 1 -- 172.21.15.70:0/1471520157 shutdown_connections 2023-10-08T07:14:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.922+0000 7f57632f3640 1 -- 172.21.15.70:0/1471520157 wait complete. 2023-10-08T07:14:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.924+0000 7f57632f3640 1 Processor -- start 2023-10-08T07:14:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.924+0000 7f57632f3640 1 -- start start 2023-10-08T07:14:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.925+0000 7f57632f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 0x7f575c0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.925+0000 7f57632f3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f575c0cc210 0x7f575c14cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.925+0000 7f57632f3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0c7c00 0x7f575c0c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.925+0000 7f57632f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c0cd390 con 0x7f5748003fd0 2023-10-08T07:14:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.925+0000 7f57632f3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c0cc710 con 0x7f575c0cc210 2023-10-08T07:14:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.925+0000 7f57632f3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f575c0cc9f0 con 0x7f575c0c7c00 2023-10-08T07:14:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.925+0000 7f57622f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 0x7f575c0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.925+0000 7f57622f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 0x7f575c0cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36316/0 (socket says 172.21.15.70:36316) 2023-10-08T07:14:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.925+0000 7f57622f1640 1 -- 172.21.15.70:0/122908284 learned_addr learned my addr 172.21.15.70:0/122908284 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f57622f1640 1 -- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0c7c00 msgr2=0x7f575c0c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f57622f1640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0c7c00 0x7f575c0c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f57622f1640 1 -- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f575c0cc210 msgr2=0x7f575c14cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f57622f1640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f575c0cc210 0x7f575c14cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f57622f1640 1 -- 172.21.15.70:0/122908284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c0cccd0 con 0x7f5748003fd0 2023-10-08T07:14:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f57622f1640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 0x7f575c0cd790 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f5758000fd0 tx=0x7f575800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f5752ffd640 1 -- 172.21.15.70:0/122908284 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580074e0 con 0x7f5748003fd0 2023-10-08T07:14:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f5752ffd640 1 -- 172.21.15.70:0/122908284 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5758007640 con 0x7f5748003fd0 2023-10-08T07:14:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f5752ffd640 1 -- 172.21.15.70:0/122908284 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580049e0 con 0x7f5748003fd0 2023-10-08T07:14:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f57632f3640 1 -- 172.21.15.70:0/122908284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575c0ccfb0 con 0x7f5748003fd0 2023-10-08T07:14:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.926+0000 7f57632f3640 1 -- 172.21.15.70:0/122908284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575c14d460 con 0x7f5748003fd0 2023-10-08T07:14:40.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.928+0000 7f57632f3640 1 -- 172.21.15.70:0/122908284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f575c04e160 con 0x7f5748003fd0 2023-10-08T07:14:40.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.928+0000 7f5752ffd640 1 -- 172.21.15.70:0/122908284 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5758004b40 con 0x7f5748003fd0 2023-10-08T07:14:40.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.929+0000 7f5752ffd640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f574006b0e0 0x7f574006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:40.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.929+0000 7f5752ffd640 1 -- 172.21.15.70:0/122908284 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f57580893d0 con 0x7f5748003fd0 2023-10-08T07:14:40.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.930+0000 7f5761af0640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f574006b0e0 0x7f574006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:40.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.931+0000 7f5761af0640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f574006b0e0 0x7f574006d5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f574c00ba00 tx=0x7f574c032000 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:40.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:40.933+0000 7f5752ffd640 1 -- 172.21.15.70:0/122908284 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57580541a0 con 0x7f5748003fd0 2023-10-08T07:14:41.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.114+0000 7f57632f3640 1 -- 172.21.15.70:0/122908284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f575c0ada50 con 0x7f5748003fd0 2023-10-08T07:14:41.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.115+0000 7f5752ffd640 1 -- 172.21.15.70:0/122908284 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v250) v1 ==== 84+0+65 (secure 0 0 0) 0x7f57580577c0 con 0x7f5748003fd0 2023-10-08T07:14:41.116 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:41.116 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T07:14:41.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:41.116 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T07:14:41.116 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T07:14:41.116 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:41.117 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T07:14:41.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.117+0000 7f57632f3640 1 -- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f574006b0e0 msgr2=0x7f574006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.117+0000 7f57632f3640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f574006b0e0 0x7f574006d5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f574c00ba00 tx=0x7f574c032000 comp rx=0 tx=0).stop 2023-10-08T07:14:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.117+0000 7f57632f3640 1 -- 172.21.15.70:0/122908284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 msgr2=0x7f575c0cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.117+0000 7f57632f3640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 0x7f575c0cd790 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f5758000fd0 tx=0x7f575800dd40 comp rx=0 tx=0).stop 2023-10-08T07:14:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.118+0000 7f57632f3640 1 -- 172.21.15.70:0/122908284 shutdown_connections 2023-10-08T07:14:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.118+0000 7f57632f3640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f574006b0e0 0x7f574006d5a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.118+0000 7f57632f3640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f575c0c7c00 0x7f575c0c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.118+0000 7f57632f3640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f575c0cc210 0x7f575c14cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.118+0000 7f57632f3640 1 --2- 172.21.15.70:0/122908284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5748003fd0 0x7f575c0cd790 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.118+0000 7f57632f3640 1 -- 172.21.15.70:0/122908284 >> 172.21.15.70:0/122908284 conn(0x7f575c0cb0c0 msgr2=0x7f575c0af780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:41.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.118+0000 7f57632f3640 1 -- 172.21.15.70:0/122908284 shutdown_connections 2023-10-08T07:14:41.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.118+0000 7f57632f3640 1 -- 172.21.15.70:0/122908284 wait complete. 2023-10-08T07:14:41.135 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T07:14:41.136 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T07:14:41.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.337+0000 7fc45ca90640 1 Processor -- start 2023-10-08T07:14:41.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.337+0000 7fc45ca90640 1 -- start start 2023-10-08T07:14:41.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.339+0000 7fc45ca90640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc45815d710 0x7fc4580b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:41.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.339+0000 7fc45ca90640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4580b30a0 con 0x7fc4580702d0 2023-10-08T07:14:41.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.339+0000 7fc45ca90640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4580b3320 con 0x7fc4580b1750 2023-10-08T07:14:41.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.339+0000 7fc45ca90640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45815dcf0 con 0x7fc45815d710 2023-10-08T07:14:41.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.339+0000 7fc45659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc4580702d0 0x7fc4580706b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39666/0 (socket says 172.21.15.70:39666) 2023-10-08T07:14:41.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.339+0000 7fc45659c640 1 -- 172.21.15.70:0/4176428338 learned_addr learned my addr 172.21.15.70:0/4176428338 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:41.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853957445 0 0) 0x7fc4580b30a0 con 0x7fc4580702d0 2023-10-08T07:14:41.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc444003520 con 0x7fc4580702d0 2023-10-08T07:14:41.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc456d9d640 1 --2- 172.21.15.70:0/4176428338 >> v2:172.21.15.181:3300/0 conn(0x7fc45815d710 0x7fc4580b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:41.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 645664964 0 0) 0x7fc444003520 con 0x7fc4580702d0 2023-10-08T07:14:41.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 >> v2:172.21.15.181:3300/0 conn(0x7fc45815d710 msgr2=0x7fc4580b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:41.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 --2- 172.21.15.70:0/4176428338 >> v2:172.21.15.181:3300/0 conn(0x7fc45815d710 0x7fc4580b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 >> v1:172.21.15.181:6789/0 conn(0x7fc4580b1750 legacy=0x7fc4580c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:41.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4580b3600 con 0x7fc4580702d0 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc448001670 con 0x7fc4580702d0 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 >> v1:172.21.15.70:6789/0 conn(0x7fc4580702d0 legacy=0x7fc4580706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4440039c0 0x7fc444003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4440047c0 0x7fc444004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444005250 0x7fc44400ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45815dcf0 con 0x7fc444005250 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44400d2b0 con 0x7fc4440047c0 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44400d470 con 0x7fc4440039c0 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.340+0000 7fc456d9d640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444005250 0x7fc44400ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.341+0000 7fc45659c640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4440039c0 0x7fc444003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.341+0000 7fc455d9b640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4440047c0 0x7fc444004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:41.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.341+0000 7fc456d9d640 1 -- 172.21.15.70:0/4176428338 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4440039c0 msgr2=0x7fc444003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:41.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.341+0000 7fc456d9d640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc4440039c0 0x7fc444003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.341+0000 7fc456d9d640 1 -- 172.21.15.70:0/4176428338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4440047c0 msgr2=0x7fc444004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:41.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.341+0000 7fc456d9d640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4440047c0 0x7fc444004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.341+0000 7fc456d9d640 1 -- 172.21.15.70:0/4176428338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444003520 con 0x7fc444005250 2023-10-08T07:14:41.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.341+0000 7fc455d9b640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc4440047c0 0x7fc444004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:41.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.342+0000 7fc456d9d640 1 -- 172.21.15.70:0/4176428338 reap_dead start 2023-10-08T07:14:41.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.342+0000 7fc456d9d640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444005250 0x7fc44400ca90 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fc450007360 tx=0x7fc4440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0848c3a1a37fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:41.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.344+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45002ccf0 con 0x7fc444005250 2023-10-08T07:14:41.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.344+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45002ce50 con 0x7fc444005250 2023-10-08T07:14:41.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.344+0000 7fc45559a640 1 -- 172.21.15.70:0/4176428338 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450003bd0 con 0x7fc444005250 2023-10-08T07:14:41.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.345+0000 7fc45ca90640 1 -- 172.21.15.70:0/4176428338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444005250 msgr2=0x7fc44400ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:41.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.345+0000 7fc45ca90640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444005250 0x7fc44400ca90 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fc450007360 tx=0x7fc4440057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.345+0000 7fc45ca90640 1 -- 172.21.15.70:0/4176428338 shutdown_connections 2023-10-08T07:14:41.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.345+0000 7fc45ca90640 1 --2- 172.21.15.70:0/4176428338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444005250 0x7fc44400ca90 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.345+0000 7fc45ca90640 1 -- 172.21.15.70:0/4176428338 >> 172.21.15.70:0/4176428338 conn(0x7fc458069d50 msgr2=0x7fc45806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:41.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.345+0000 7fc45ca90640 1 -- 172.21.15.70:0/4176428338 shutdown_connections 2023-10-08T07:14:41.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.345+0000 7fc45ca90640 1 -- 172.21.15.70:0/4176428338 wait complete. 2023-10-08T07:14:41.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.347+0000 7fc45ca90640 1 Processor -- start 2023-10-08T07:14:41.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.348+0000 7fc45ca90640 1 -- start start 2023-10-08T07:14:41.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.349+0000 7fc45ca90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc444005250 0x7fc45814cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.349+0000 7fc45ca90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4581e7720 0x7fc458148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.349+0000 7fc45ca90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc458147dd0 0x7fc45814e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.349+0000 7fc45ca90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4581494e0 con 0x7fc4581e7720 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.349+0000 7fc45ca90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45814f0f0 con 0x7fc444005250 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.349+0000 7fc45ca90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45814f3d0 con 0x7fc458147dd0 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.349+0000 7fc455d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4581e7720 0x7fc458148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.349+0000 7fc455d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4581e7720 0x7fc458148b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36324/0 (socket says 172.21.15.70:36324) 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.349+0000 7fc455d9b640 1 -- 172.21.15.70:0/1390063140 learned_addr learned my addr 172.21.15.70:0/1390063140 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc45659c640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc444005250 0x7fc45814cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc455d9b640 1 -- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc458147dd0 msgr2=0x7fc45814e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc455d9b640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc458147dd0 0x7fc45814e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc455d9b640 1 -- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc444005250 msgr2=0x7fc45814cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:41.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc455d9b640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc444005250 0x7fc45814cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:41.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc455d9b640 1 -- 172.21.15.70:0/1390063140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4581472f0 con 0x7fc4581e7720 2023-10-08T07:14:41.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc45659c640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc444005250 0x7fc45814cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:41.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc455d9b640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4581e7720 0x7fc458148b60 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7fc44000e490 tx=0x7fc44000e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:41.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc44effd640 1 -- 172.21.15.70:0/1390063140 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440018070 con 0x7fc4581e7720 2023-10-08T07:14:41.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc44effd640 1 -- 172.21.15.70:0/1390063140 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc440007020 con 0x7fc4581e7720 2023-10-08T07:14:41.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc44effd640 1 -- 172.21.15.70:0/1390063140 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4400071d0 con 0x7fc4581e7720 2023-10-08T07:14:41.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.350+0000 7fc45ca90640 1 -- 172.21.15.70:0/1390063140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4581475d0 con 0x7fc4581e7720 2023-10-08T07:14:41.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.351+0000 7fc45ca90640 1 -- 172.21.15.70:0/1390063140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45814d140 con 0x7fc4581e7720 2023-10-08T07:14:41.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.352+0000 7fc45ca90640 1 -- 172.21.15.70:0/1390063140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45804e160 con 0x7fc4581e7720 2023-10-08T07:14:41.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.353+0000 7fc44effd640 1 -- 172.21.15.70:0/1390063140 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc440007330 con 0x7fc4581e7720 2023-10-08T07:14:41.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.354+0000 7fc44effd640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc43806b0b0 0x7fc43806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:41.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.354+0000 7fc44effd640 1 -- 172.21.15.70:0/1390063140 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5293+0+0 (secure 0 0 0) 0x7fc440089460 con 0x7fc4581e7720 2023-10-08T07:14:41.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.355+0000 7fc45659c640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc43806b0b0 0x7fc43806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:41.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.356+0000 7fc45659c640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc43806b0b0 0x7fc43806d570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc448001050 tx=0x7fc448000fe0 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:41.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.358+0000 7fc44effd640 1 -- 172.21.15.70:0/1390063140 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc4400542b0 con 0x7fc4581e7720 2023-10-08T07:14:41.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:41.546+0000 7fc45ca90640 1 -- 172.21.15.70:0/1390063140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc458073900 con 0x7fc4581e7720 2023-10-08T07:14:42.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.360+0000 7fc44effd640 1 -- 172.21.15.70:0/1390063140 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v251) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc4400578d0 con 0x7fc4581e7720 2023-10-08T07:14:42.362 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T07:14:42.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.363+0000 7fc45ca90640 1 -- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc43806b0b0 msgr2=0x7fc43806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:42.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.363+0000 7fc45ca90640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc43806b0b0 0x7fc43806d570 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc448001050 tx=0x7fc448000fe0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.363+0000 7fc45ca90640 1 -- 172.21.15.70:0/1390063140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4581e7720 msgr2=0x7fc458148b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:42.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.363+0000 7fc45ca90640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4581e7720 0x7fc458148b60 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7fc44000e490 tx=0x7fc44000e950 comp rx=0 tx=0).stop 2023-10-08T07:14:42.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.364+0000 7fc45ca90640 1 -- 172.21.15.70:0/1390063140 shutdown_connections 2023-10-08T07:14:42.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.364+0000 7fc45ca90640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc43806b0b0 0x7fc43806d570 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.364+0000 7fc45ca90640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc458147dd0 0x7fc45814e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.364+0000 7fc45ca90640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4581e7720 0x7fc458148b60 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.364+0000 7fc45ca90640 1 --2- 172.21.15.70:0/1390063140 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc444005250 0x7fc45814cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.364+0000 7fc45ca90640 1 -- 172.21.15.70:0/1390063140 >> 172.21.15.70:0/1390063140 conn(0x7fc458069d50 msgr2=0x7fc458155290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:42.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.364+0000 7fc45ca90640 1 -- 172.21.15.70:0/1390063140 shutdown_connections 2023-10-08T07:14:42.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.364+0000 7fc45ca90640 1 -- 172.21.15.70:0/1390063140 wait complete. 2023-10-08T07:14:42.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T07:14:42.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.604+0000 7fe1b1ba4640 1 Processor -- start 2023-10-08T07:14:42.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.604+0000 7fe1b1ba4640 1 -- start start 2023-10-08T07:14:42.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.605+0000 7fe1b1ba4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0cb500 0x7fe1ac0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:42.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.605+0000 7fe1b1ba4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe1ac070690 0x7fe1ac070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:42.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.605+0000 7fe1b1ba4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ac0c8c60 con 0x7fe1ac06cfa0 2023-10-08T07:14:42.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.605+0000 7fe1b1ba4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ac0cbe20 con 0x7fe1ac070690 2023-10-08T07:14:42.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.605+0000 7fe1b1ba4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac0b1ac0 con 0x7fe1ac0cb500 2023-10-08T07:14:42.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.606+0000 7fe1aaffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe1ac06cfa0 0x7fe1ac15b0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39668/0 (socket says 172.21.15.70:39668) 2023-10-08T07:14:42.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.606+0000 7fe1aaffd640 1 -- 172.21.15.70:0/4105041109 learned_addr learned my addr 172.21.15.70:0/4105041109 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:42.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.606+0000 7fe1ab7fe640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0cb500 0x7fe1ac0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:42.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.606+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461141054 0 0) 0x7fe1ac0c8c60 con 0x7fe1ac06cfa0 2023-10-08T07:14:42.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.606+0000 7fe1abfff640 1 --2- 172.21.15.70:0/4105041109 >> v2:172.21.15.181:3300/0 conn(0x7fe1ac070690 0x7fe1ac070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:42.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.606+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe194003520 con 0x7fe1ac06cfa0 2023-10-08T07:14:42.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1399123669 0 0) 0x7fe194003520 con 0x7fe1ac06cfa0 2023-10-08T07:14:42.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0cb500 msgr2=0x7fe1ac0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:42.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0cb500 0x7fe1ac0cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 >> v2:172.21.15.181:3300/0 conn(0x7fe1ac070690 msgr2=0x7fe1ac070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:42.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 --2- 172.21.15.70:0/4105041109 >> v2:172.21.15.181:3300/0 conn(0x7fe1ac070690 0x7fe1ac070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac0b1da0 con 0x7fe1ac06cfa0 2023-10-08T07:14:42.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe1a0001670 con 0x7fe1ac06cfa0 2023-10-08T07:14:42.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 >> v1:172.21.15.70:6789/0 conn(0x7fe1ac06cfa0 legacy=0x7fe1ac15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:42.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe194003970 0x7fe194003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:42.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1940047c0 0x7fe194004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:42.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0cb500 0x7fe194005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:42.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac0cbe20 con 0x7fe194003970 2023-10-08T07:14:42.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac0b1ac0 con 0x7fe1940047c0 2023-10-08T07:14:42.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19400b980 con 0x7fe1ac0cb500 2023-10-08T07:14:42.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1ab7fe640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe194003970 0x7fe194003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:42.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.607+0000 7fe1aaffd640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1940047c0 0x7fe194004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:42.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.608+0000 7fe1ab7fe640 1 -- 172.21.15.70:0/4105041109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0cb500 msgr2=0x7fe194005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:42.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.608+0000 7fe1ab7fe640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0cb500 0x7fe194005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.608+0000 7fe1ab7fe640 1 -- 172.21.15.70:0/4105041109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1940047c0 msgr2=0x7fe194004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:42.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.608+0000 7fe1ab7fe640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe1940047c0 0x7fe194004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.608+0000 7fe1ab7fe640 1 -- 172.21.15.70:0/4105041109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe194003520 con 0x7fe194003970 2023-10-08T07:14:42.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.609+0000 7fe1ab7fe640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe194003970 0x7fe194003d90 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009af0 tx=0x7fe1940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762723f9aebb15b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:42.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.609+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c02c710 con 0x7fe194003970 2023-10-08T07:14:42.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.609+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19c02c870 con 0x7fe194003970 2023-10-08T07:14:42.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.609+0000 7fe1aa7fc640 1 -- 172.21.15.70:0/4105041109 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19c02cc00 con 0x7fe194003970 2023-10-08T07:14:42.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.610+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/4105041109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe194003970 msgr2=0x7fe194003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:42.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.610+0000 7fe1b1ba4640 1 --2- 172.21.15.70:0/4105041109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe194003970 0x7fe194003d90 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7fe19c009af0 tx=0x7fe1940057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.610+0000 7fe1abfff640 1 -- 172.21.15.70:0/4105041109 reap_dead start 2023-10-08T07:14:42.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.610+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/4105041109 shutdown_connections 2023-10-08T07:14:42.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.610+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/4105041109 >> 172.21.15.70:0/4105041109 conn(0x7fe1ac06a140 msgr2=0x7fe1ac06a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:42.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.610+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/4105041109 shutdown_connections 2023-10-08T07:14:42.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.610+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/4105041109 wait complete. 2023-10-08T07:14:42.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.612+0000 7fe1b1ba4640 1 Processor -- start 2023-10-08T07:14:42.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.612+0000 7fe1b1ba4640 1 -- start start 2023-10-08T07:14:42.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.613+0000 7fe1b1ba4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe194003970 0x7fe1ac150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:42.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.613+0000 7fe1b1ba4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0c4660 0x7fe1ac151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:42.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.613+0000 7fe1b1ba4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1ac0ad420 0x7fe1ac0ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:42.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.613+0000 7fe1b1ba4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac0adde0 con 0x7fe1ac0ad420 2023-10-08T07:14:42.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.613+0000 7fe1b1ba4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac159380 con 0x7fe194003970 2023-10-08T07:14:42.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.613+0000 7fe1b1ba4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ac0c4bd0 con 0x7fe1ac0c4660 2023-10-08T07:14:42.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.613+0000 7fe1abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1ac0ad420 0x7fe1ac0ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:42.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.613+0000 7fe1abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1ac0ad420 0x7fe1ac0ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36330/0 (socket says 172.21.15.70:36330) 2023-10-08T07:14:42.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.613+0000 7fe1abfff640 1 -- 172.21.15.70:0/2391886336 learned_addr learned my addr 172.21.15.70:0/2391886336 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:42.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.613+0000 7fe1ab7fe640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe194003970 0x7fe1ac150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:42.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe1aaffd640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0c4660 0x7fe1ac151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:42.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe1abfff640 1 -- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0c4660 msgr2=0x7fe1ac151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:42.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe1abfff640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0c4660 0x7fe1ac151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe1abfff640 1 -- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe194003970 msgr2=0x7fe1ac150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:42.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe1abfff640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe194003970 0x7fe1ac150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:42.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe1abfff640 1 -- 172.21.15.70:0/2391886336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac0af8f0 con 0x7fe1ac0ad420 2023-10-08T07:14:42.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe1aaffd640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0c4660 0x7fe1ac151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:42.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe1abfff640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1ac0ad420 0x7fe1ac0ad8a0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0ce800 tx=0x7fe1a400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:42.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe193fff640 1 -- 172.21.15.70:0/2391886336 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4010040 con 0x7fe1ac0ad420 2023-10-08T07:14:42.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe193fff640 1 -- 172.21.15.70:0/2391886336 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a400bce0 con 0x7fe1ac0ad420 2023-10-08T07:14:42.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe193fff640 1 -- 172.21.15.70:0/2391886336 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4014640 con 0x7fe1ac0ad420 2023-10-08T07:14:42.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.614+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/2391886336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1ac0afbd0 con 0x7fe1ac0ad420 2023-10-08T07:14:42.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.615+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/2391886336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ac0af510 con 0x7fe1ac0ad420 2023-10-08T07:14:42.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.616+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/2391886336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ac071490 con 0x7fe1ac0ad420 2023-10-08T07:14:42.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.617+0000 7fe193fff640 1 -- 172.21.15.70:0/2391886336 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe1a40147a0 con 0x7fe1ac0ad420 2023-10-08T07:14:42.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.618+0000 7fe193fff640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe19806b0b0 0x7fe19806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:42.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.618+0000 7fe193fff640 1 -- 172.21.15.70:0/2391886336 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5663+0+0 (secure 0 0 0) 0x7fe1a408a8f0 con 0x7fe1ac0ad420 2023-10-08T07:14:42.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.619+0000 7fe1ab7fe640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe19806b0b0 0x7fe19806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:42.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.620+0000 7fe1ab7fe640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe19806b0b0 0x7fe19806d570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe19c036000 tx=0x7fe19c030000 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:42.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.623+0000 7fe193fff640 1 -- 172.21.15.70:0/2391886336 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1a4055610 con 0x7fe1ac0ad420 2023-10-08T07:14:42.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:42.812+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/2391886336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe1ac0afdc0 con 0x7fe1ac0ad420 2023-10-08T07:14:43.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.378+0000 7fe193fff640 1 -- 172.21.15.70:0/2391886336 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v252) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe1a4058c30 con 0x7fe1ac0ad420 2023-10-08T07:14:43.380 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T07:14:43.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.382+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe19806b0b0 msgr2=0x7fe19806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:43.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.382+0000 7fe1b1ba4640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe19806b0b0 0x7fe19806d570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe19c036000 tx=0x7fe19c030000 comp rx=0 tx=0).stop 2023-10-08T07:14:43.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.382+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/2391886336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1ac0ad420 msgr2=0x7fe1ac0ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:43.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.382+0000 7fe1b1ba4640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1ac0ad420 0x7fe1ac0ad8a0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0ce800 tx=0x7fe1a400b480 comp rx=0 tx=0).stop 2023-10-08T07:14:43.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.382+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/2391886336 shutdown_connections 2023-10-08T07:14:43.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.382+0000 7fe1b1ba4640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe19806b0b0 0x7fe19806d570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:43.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.382+0000 7fe1b1ba4640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1ac0ad420 0x7fe1ac0ad8a0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.382+0000 7fe1b1ba4640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1ac0c4660 0x7fe1ac151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.382+0000 7fe1b1ba4640 1 --2- 172.21.15.70:0/2391886336 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe194003970 0x7fe1ac150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.382+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/2391886336 >> 172.21.15.70:0/2391886336 conn(0x7fe1ac06a140 msgr2=0x7fe1ac15b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.383+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/2391886336 shutdown_connections 2023-10-08T07:14:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.383+0000 7fe1b1ba4640 1 -- 172.21.15.70:0/2391886336 wait complete. 2023-10-08T07:14:43.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T07:14:43.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.601+0000 7f8526b0c640 1 Processor -- start 2023-10-08T07:14:43.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.601+0000 7f8526b0c640 1 -- start start 2023-10-08T07:14:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.602+0000 7f8526b0c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f852015cae0 0x7f85200cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.602+0000 7f8526b0c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f85200c8b40 0x7f85200c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.602+0000 7f8526b0c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85200c94e0 con 0x7f85200aff70 2023-10-08T07:14:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.603+0000 7f8526b0c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85200ccbc0 con 0x7f85200c8b40 2023-10-08T07:14:43.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.603+0000 7f8526b0c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85200ce1f0 con 0x7f852015cae0 2023-10-08T07:14:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.603+0000 7f8525309640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f85200c8b40 0x7f85200c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.603+0000 7f852630b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f85200aff70 0x7f8520070180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46444/0 (socket says 172.21.15.70:46444) 2023-10-08T07:14:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.603+0000 7f852630b640 1 -- 172.21.15.70:0/3664211101 learned_addr learned my addr 172.21.15.70:0/3664211101 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.603+0000 7f8525b0a640 1 --2- 172.21.15.70:0/3664211101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f852015cae0 0x7f85200cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.603+0000 7f8525309640 1 -- 172.21.15.70:0/3664211101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f852015cae0 msgr2=0x7f85200cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.603+0000 7f8525309640 1 --2- 172.21.15.70:0/3664211101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f852015cae0 0x7f85200cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.603+0000 7f8525309640 1 -- 172.21.15.70:0/3664211101 >> v1:172.21.15.70:6789/0 conn(0x7f85200aff70 legacy=0x7f8520070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:43.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.603+0000 7f8525309640 1 -- 172.21.15.70:0/3664211101 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85200cd590 con 0x7f85200c8b40 2023-10-08T07:14:43.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.605+0000 7f8525309640 1 --2- 172.21.15.70:0/3664211101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200c8b40 0x7f85200c8fa0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f85100097a0 tx=0x7f851002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5a9da3e6de33c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.605+0000 7f8524b08640 1 -- 172.21.15.70:0/3664211101 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851002d040 con 0x7f85200c8b40 2023-10-08T07:14:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.605+0000 7f8524b08640 1 -- 172.21.15.70:0/3664211101 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8510031810 con 0x7f85200c8b40 2023-10-08T07:14:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.605+0000 7f8524b08640 1 -- 172.21.15.70:0/3664211101 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510031ba0 con 0x7f85200c8b40 2023-10-08T07:14:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.605+0000 7f8526b0c640 1 -- 172.21.15.70:0/3664211101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200c8b40 msgr2=0x7f85200c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.605+0000 7f8526b0c640 1 --2- 172.21.15.70:0/3664211101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200c8b40 0x7f85200c8fa0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f85100097a0 tx=0x7f851002b8b0 comp rx=0 tx=0).stop 2023-10-08T07:14:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.606+0000 7f8526b0c640 1 -- 172.21.15.70:0/3664211101 shutdown_connections 2023-10-08T07:14:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.606+0000 7f8526b0c640 1 --2- 172.21.15.70:0/3664211101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200c8b40 0x7f85200c8fa0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.606+0000 7f8526b0c640 1 --2- 172.21.15.70:0/3664211101 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f852015cae0 0x7f85200cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.606+0000 7f8526b0c640 1 -- 172.21.15.70:0/3664211101 >> 172.21.15.70:0/3664211101 conn(0x7f852006c050 msgr2=0x7f852006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:43.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.606+0000 7f8526b0c640 1 -- 172.21.15.70:0/3664211101 shutdown_connections 2023-10-08T07:14:43.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.606+0000 7f8526b0c640 1 -- 172.21.15.70:0/3664211101 wait complete. 2023-10-08T07:14:43.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.607+0000 7f8526b0c640 1 Processor -- start 2023-10-08T07:14:43.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.608+0000 7f8526b0c640 1 -- start start 2023-10-08T07:14:43.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.608+0000 7f8526b0c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200aff70 0x7f852015df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:43.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.608+0000 7f8526b0c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85200c8b40 0x7f85201545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:43.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.608+0000 7f8526b0c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f852015cae0 0x7f85200b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:43.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.608+0000 7f8526b0c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85200b1da0 con 0x7f85200aff70 2023-10-08T07:14:43.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.608+0000 7f8526b0c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852015e4f0 con 0x7f852015cae0 2023-10-08T07:14:43.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.608+0000 7f8526b0c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852015e7d0 con 0x7f85200c8b40 2023-10-08T07:14:43.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.609+0000 7f8525b0a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200aff70 0x7f852015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:43.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.609+0000 7f8525b0a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200aff70 0x7f852015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40088/0 (socket says 172.21.15.70:40088) 2023-10-08T07:14:43.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.609+0000 7f8525b0a640 1 -- 172.21.15.70:0/2684491772 learned_addr learned my addr 172.21.15.70:0/2684491772 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:43.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.609+0000 7f8525309640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85200c8b40 0x7f85201545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:43.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.609+0000 7f8525b0a640 1 -- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85200c8b40 msgr2=0x7f85201545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:43.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.609+0000 7f8525b0a640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85200c8b40 0x7f85201545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:43.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.609+0000 7f8525b0a640 1 -- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f852015cae0 msgr2=0x7f85200b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:43.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.609+0000 7f8525b0a640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f852015cae0 0x7f85200b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:43.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.609+0000 7f8525b0a640 1 -- 172.21.15.70:0/2684491772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8510009400 con 0x7f85200aff70 2023-10-08T07:14:43.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.609+0000 7f8525b0a640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200aff70 0x7f852015df10 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d5f0 tx=0x7f851c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:43.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.610+0000 7f85167fc640 1 -- 172.21.15.70:0/2684491772 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c0125f0 con 0x7f85200aff70 2023-10-08T07:14:43.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.610+0000 7f85167fc640 1 -- 172.21.15.70:0/2684491772 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c012750 con 0x7f85200aff70 2023-10-08T07:14:43.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.610+0000 7f85167fc640 1 -- 172.21.15.70:0/2684491772 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c012b00 con 0x7f85200aff70 2023-10-08T07:14:43.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.610+0000 7f8526b0c640 1 -- 172.21.15.70:0/2684491772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852015eab0 con 0x7f85200aff70 2023-10-08T07:14:43.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.610+0000 7f8526b0c640 1 -- 172.21.15.70:0/2684491772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85201f60b0 con 0x7f85200aff70 2023-10-08T07:14:43.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.611+0000 7f8526b0c640 1 -- 172.21.15.70:0/2684491772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8520074150 con 0x7f85200aff70 2023-10-08T07:14:43.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.612+0000 7f85167fc640 1 -- 172.21.15.70:0/2684491772 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f851c012c60 con 0x7f85200aff70 2023-10-08T07:14:43.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.613+0000 7f85167fc640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f850406b0b0 0x7f850406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:43.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.613+0000 7f85167fc640 1 -- 172.21.15.70:0/2684491772 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f851c08a090 con 0x7f85200aff70 2023-10-08T07:14:43.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.614+0000 7f8525309640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f850406b0b0 0x7f850406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:43.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.615+0000 7f8525309640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f850406b0b0 0x7f850406d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8510036000 tx=0x7f851003c040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:43.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.618+0000 7f85167fc640 1 -- 172.21.15.70:0/2684491772 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f851c054c80 con 0x7f85200aff70 2023-10-08T07:14:43.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:43.830+0000 7f8526b0c640 1 -- 172.21.15.70:0/2684491772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f85200c6d10 con 0x7f85200aff70 2023-10-08T07:14:44.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.379+0000 7f85167fc640 1 -- 172.21.15.70:0/2684491772 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v121) v1 ==== 174+0+0 (secure 0 0 0) 0x7f851c0582a0 con 0x7f85200aff70 2023-10-08T07:14:44.381 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 31 and data pool 32 2023-10-08T07:14:44.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.382+0000 7f8526b0c640 1 -- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f850406b0b0 msgr2=0x7f850406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:44.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.382+0000 7f8526b0c640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f850406b0b0 0x7f850406d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8510036000 tx=0x7f851003c040 comp rx=0 tx=0).stop 2023-10-08T07:14:44.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.382+0000 7f8526b0c640 1 -- 172.21.15.70:0/2684491772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200aff70 msgr2=0x7f852015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:44.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.382+0000 7f8526b0c640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200aff70 0x7f852015df10 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d5f0 tx=0x7f851c00dab0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.382+0000 7f8526b0c640 1 -- 172.21.15.70:0/2684491772 shutdown_connections 2023-10-08T07:14:44.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.382+0000 7f8526b0c640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f850406b0b0 0x7f850406d570 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.382+0000 7f8526b0c640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f852015cae0 0x7f85200b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.382+0000 7f8526b0c640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85200c8b40 0x7f85201545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.382+0000 7f8526b0c640 1 --2- 172.21.15.70:0/2684491772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85200aff70 0x7f852015df10 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.382+0000 7f8526b0c640 1 -- 172.21.15.70:0/2684491772 >> 172.21.15.70:0/2684491772 conn(0x7f852006c050 msgr2=0x7f85200caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:44.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.383+0000 7f8526b0c640 1 -- 172.21.15.70:0/2684491772 shutdown_connections 2023-10-08T07:14:44.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.383+0000 7f8526b0c640 1 -- 172.21.15.70:0/2684491772 wait complete. 2023-10-08T07:14:44.410 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:44.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.584+0000 7f7a4dc40640 1 Processor -- start 2023-10-08T07:14:44.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.584+0000 7f7a4dc40640 1 -- start start 2023-10-08T07:14:44.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.586+0000 7f7a4dc40640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7a480b07d0 0x7f7a48165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:44.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.586+0000 7f7a4dc40640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4815f7d0 con 0x7f7a480adc70 2023-10-08T07:14:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.586+0000 7f7a4dc40640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4815efb0 con 0x7f7a4815ab30 2023-10-08T07:14:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.586+0000 7f7a4dc40640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4815f210 con 0x7f7a480b07d0 2023-10-08T07:14:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.586+0000 7f7a3ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7a480adc70 0x7f7a480b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46452/0 (socket says 172.21.15.70:46452) 2023-10-08T07:14:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.586+0000 7f7a3ffff640 1 -- 172.21.15.70:0/1085494098 learned_addr learned my addr 172.21.15.70:0/1085494098 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:44.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.586+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184980109 0 0) 0x7f7a4815f7d0 con 0x7f7a480adc70 2023-10-08T07:14:44.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.586+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a30003520 con 0x7f7a480adc70 2023-10-08T07:14:44.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.586+0000 7f7a4d43f640 1 --2- 172.21.15.70:0/1085494098 >> v2:172.21.15.181:3300/0 conn(0x7f7a480b07d0 0x7f7a48165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:44.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3023437735 0 0) 0x7f7a30003520 con 0x7f7a480adc70 2023-10-08T07:14:44.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 >> v2:172.21.15.181:3300/0 conn(0x7f7a480b07d0 msgr2=0x7f7a48165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:44.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 --2- 172.21.15.70:0/1085494098 >> v2:172.21.15.181:3300/0 conn(0x7f7a480b07d0 0x7f7a48165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 >> v1:172.21.15.181:6789/0 conn(0x7f7a4815ab30 legacy=0x7f7a4815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:44.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4815f4f0 con 0x7f7a480adc70 2023-10-08T07:14:44.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7a38001670 con 0x7f7a480adc70 2023-10-08T07:14:44.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 >> v1:172.21.15.70:6789/0 conn(0x7f7a480adc70 legacy=0x7f7a480b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:44.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a300039c0 0x7f7a30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:44.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a300047c0 0x7f7a30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:44.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a300051d0 0x7f7a3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4815f210 con 0x7f7a300051d0 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3000c120 con 0x7f7a300039c0 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3000c2e0 con 0x7f7a300047c0 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a4d43f640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a300051d0 0x7f7a3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a4cc3e640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a300039c0 0x7f7a30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.587+0000 7f7a3ffff640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a300047c0 0x7f7a30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.588+0000 7f7a4d43f640 1 -- 172.21.15.70:0/1085494098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a300047c0 msgr2=0x7f7a30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.588+0000 7f7a4d43f640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a300047c0 0x7f7a30004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.588+0000 7f7a4d43f640 1 -- 172.21.15.70:0/1085494098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a300039c0 msgr2=0x7f7a30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.588+0000 7f7a4d43f640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a300039c0 0x7f7a30003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.588+0000 7f7a4d43f640 1 -- 172.21.15.70:0/1085494098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a30003520 con 0x7f7a300051d0 2023-10-08T07:14:44.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.589+0000 7f7a4d43f640 1 -- 172.21.15.70:0/1085494098 reap_dead start 2023-10-08T07:14:44.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.589+0000 7f7a4d43f640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a300051d0 0x7f7a3000b980 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000ba30 tx=0x7f7a300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136963820a629f24 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:44.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.589+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40004020 con 0x7f7a300051d0 2023-10-08T07:14:44.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.589+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a400041c0 con 0x7f7a300051d0 2023-10-08T07:14:44.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.589+0000 7f7a3f7fe640 1 -- 172.21.15.70:0/1085494098 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40003610 con 0x7f7a300051d0 2023-10-08T07:14:44.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.590+0000 7f7a4dc40640 1 -- 172.21.15.70:0/1085494098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a300051d0 msgr2=0x7f7a3000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:44.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.590+0000 7f7a4dc40640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a300051d0 0x7f7a3000b980 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000ba30 tx=0x7f7a300057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.590+0000 7f7a4dc40640 1 -- 172.21.15.70:0/1085494098 shutdown_connections 2023-10-08T07:14:44.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.590+0000 7f7a4dc40640 1 --2- 172.21.15.70:0/1085494098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a300051d0 0x7f7a3000b980 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.590+0000 7f7a4dc40640 1 -- 172.21.15.70:0/1085494098 >> 172.21.15.70:0/1085494098 conn(0x7f7a480cb0c0 msgr2=0x7f7a480cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:44.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.590+0000 7f7a4dc40640 1 -- 172.21.15.70:0/1085494098 shutdown_connections 2023-10-08T07:14:44.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.590+0000 7f7a4dc40640 1 -- 172.21.15.70:0/1085494098 wait complete. 2023-10-08T07:14:44.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.591+0000 7f7a4dc40640 1 Processor -- start 2023-10-08T07:14:44.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.592+0000 7f7a4dc40640 1 -- start start 2023-10-08T07:14:44.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.592+0000 7f7a4dc40640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a300051d0 0x7f7a480af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.592+0000 7f7a4dc40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a48155f40 0x7f7a481ed7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.592+0000 7f7a4dc40640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a481edce0 0x7f7a480c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.592+0000 7f7a4dc40640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a48154f30 con 0x7f7a48155f40 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.592+0000 7f7a4dc40640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a481ee160 con 0x7f7a481edce0 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.592+0000 7f7a4dc40640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a48155180 con 0x7f7a300051d0 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.593+0000 7f7a3ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a48155f40 0x7f7a481ed7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.593+0000 7f7a3ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a48155f40 0x7f7a481ed7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40110/0 (socket says 172.21.15.70:40110) 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.593+0000 7f7a3ffff640 1 -- 172.21.15.70:0/2943898833 learned_addr learned my addr 172.21.15.70:0/2943898833 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.593+0000 7f7a3ffff640 1 -- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a300051d0 msgr2=0x7f7a480af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.593+0000 7f7a3ffff640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a300051d0 0x7f7a480af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.593+0000 7f7a4d43f640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a481edce0 0x7f7a480c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:44.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.593+0000 7f7a3ffff640 1 -- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a481edce0 msgr2=0x7f7a480c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:44.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.593+0000 7f7a3ffff640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a481edce0 0x7f7a480c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.593+0000 7f7a3ffff640 1 -- 172.21.15.70:0/2943898833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48155460 con 0x7f7a48155f40 2023-10-08T07:14:44.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.593+0000 7f7a3ffff640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a48155f40 0x7f7a481ed7a0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e700 tx=0x7f7a3800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:44.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.594+0000 7f7a3d7fa640 1 -- 172.21.15.70:0/2943898833 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800c6c0 con 0x7f7a48155f40 2023-10-08T07:14:44.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.594+0000 7f7a3d7fa640 1 -- 172.21.15.70:0/2943898833 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3800c820 con 0x7f7a48155f40 2023-10-08T07:14:44.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.594+0000 7f7a3d7fa640 1 -- 172.21.15.70:0/2943898833 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800cbd0 con 0x7f7a48155f40 2023-10-08T07:14:44.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.594+0000 7f7a4dc40640 1 -- 172.21.15.70:0/2943898833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a48155740 con 0x7f7a48155f40 2023-10-08T07:14:44.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.594+0000 7f7a4dc40640 1 -- 172.21.15.70:0/2943898833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a480afea0 con 0x7f7a48155f40 2023-10-08T07:14:44.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.595+0000 7f7a4dc40640 1 -- 172.21.15.70:0/2943898833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a48155c60 con 0x7f7a48155f40 2023-10-08T07:14:44.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.596+0000 7f7a3d7fa640 1 -- 172.21.15.70:0/2943898833 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7a3800cd30 con 0x7f7a48155f40 2023-10-08T07:14:44.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.597+0000 7f7a3d7fa640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7a3406b0e0 0x7f7a3406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:44.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.597+0000 7f7a3d7fa640 1 -- 172.21.15.70:0/2943898833 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7a380899b0 con 0x7f7a48155f40 2023-10-08T07:14:44.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.598+0000 7f7a4cc3e640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7a3406b0e0 0x7f7a3406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:44.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.599+0000 7f7a4cc3e640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7a3406b0e0 0x7f7a3406d5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a44007fd0 tx=0x7f7a44009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:44.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.602+0000 7f7a3d7fa640 1 -- 172.21.15.70:0/2943898833 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7a48155c60 con 0x7f7a48155f40 2023-10-08T07:14:44.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.777+0000 7f7a4dc40640 1 -- 172.21.15.70:0/2943898833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a48155c60 con 0x7f7a48155f40 2023-10-08T07:14:44.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.780+0000 7f7a3d7fa640 1 -- 172.21.15.70:0/2943898833 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v253) v1 ==== 74+0+16971 (secure 0 0 0) 0x7f7a48155c60 con 0x7f7a48155f40 2023-10-08T07:14:44.782 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:44.782 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":253,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:43.832930+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:44.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.782+0000 7f7a4dc40640 1 -- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7a3406b0e0 msgr2=0x7f7a3406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:44.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.782+0000 7f7a4dc40640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7a3406b0e0 0x7f7a3406d5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a44007fd0 tx=0x7f7a44009040 comp rx=0 tx=0).stop 2023-10-08T07:14:44.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.782+0000 7f7a4dc40640 1 -- 172.21.15.70:0/2943898833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a48155f40 msgr2=0x7f7a481ed7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:44.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.782+0000 7f7a4dc40640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a48155f40 0x7f7a481ed7a0 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800e700 tx=0x7f7a3800ebc0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.782+0000 7f7a4dc40640 1 -- 172.21.15.70:0/2943898833 shutdown_connections 2023-10-08T07:14:44.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.782+0000 7f7a4dc40640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7a3406b0e0 0x7f7a3406d5a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.782+0000 7f7a4dc40640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7a481edce0 0x7f7a480c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.782+0000 7f7a4dc40640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a48155f40 0x7f7a481ed7a0 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.782+0000 7f7a4dc40640 1 --2- 172.21.15.70:0/2943898833 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7a300051d0 0x7f7a480af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:44.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.782+0000 7f7a4dc40640 1 -- 172.21.15.70:0/2943898833 >> 172.21.15.70:0/2943898833 conn(0x7f7a480cb0c0 msgr2=0x7f7a4806da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:44.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.783+0000 7f7a4dc40640 1 -- 172.21.15.70:0/2943898833 shutdown_connections 2023-10-08T07:14:44.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:44.783+0000 7f7a4dc40640 1 -- 172.21.15.70:0/2943898833 wait complete. 2023-10-08T07:14:44.802 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:45.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.003+0000 7f172e6e5640 1 Processor -- start 2023-10-08T07:14:45.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.003+0000 7f172e6e5640 1 -- start start 2023-10-08T07:14:45.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.005+0000 7f172e6e5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f172806d4c0 0x7f1728162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.005+0000 7f172e6e5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f17280cb150 0x7f17280cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.005+0000 7f172e6e5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17280ae140 con 0x7f17280add60 2023-10-08T07:14:45.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.005+0000 7f172e6e5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1728161f10 con 0x7f172806d4c0 2023-10-08T07:14:45.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.005+0000 7f172e6e5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17280cbb20 con 0x7f17280cb150 2023-10-08T07:14:45.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.005+0000 7f1727fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f17280add60 0x7f17280cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46464/0 (socket says 172.21.15.70:46464) 2023-10-08T07:14:45.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.005+0000 7f1727fff640 1 -- 172.21.15.70:0/967424251 learned_addr learned my addr 172.21.15.70:0/967424251 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:45.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.005+0000 7f171ffff640 1 --2- 172.21.15.70:0/967424251 >> v2:172.21.15.70:3300/0 conn(0x7f172806d4c0 0x7f1728162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:45.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.006+0000 7f171ffff640 1 -- 172.21.15.70:0/967424251 >> v2:172.21.15.181:3300/0 conn(0x7f17280cb150 msgr2=0x7f17280cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:45.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.006+0000 7f172cc82640 1 --2- 172.21.15.70:0/967424251 >> v2:172.21.15.181:3300/0 conn(0x7f17280cb150 0x7f17280cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:45.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.006+0000 7f171ffff640 1 --2- 172.21.15.70:0/967424251 >> v2:172.21.15.181:3300/0 conn(0x7f17280cb150 0x7f17280cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.006+0000 7f17277fe640 1 -- 172.21.15.70:0/967424251 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055563882 0 0) 0x7f17280ae140 con 0x7f17280add60 2023-10-08T07:14:45.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.006+0000 7f171ffff640 1 -- 172.21.15.70:0/967424251 >> v1:172.21.15.70:6789/0 conn(0x7f17280add60 legacy=0x7f17280cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:45.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.006+0000 7f171ffff640 1 -- 172.21.15.70:0/967424251 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17280ccf20 con 0x7f172806d4c0 2023-10-08T07:14:45.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.007+0000 7f171ffff640 1 --2- 172.21.15.70:0/967424251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f172806d4c0 0x7f1728162330 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f17280cb0b0 tx=0x7f171002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4e6349f578e749 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:45.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.007+0000 7f17277fe640 1 -- 172.21.15.70:0/967424251 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171002e070 con 0x7f172806d4c0 2023-10-08T07:14:45.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.007+0000 7f17277fe640 1 -- 172.21.15.70:0/967424251 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17100315a0 con 0x7f172806d4c0 2023-10-08T07:14:45.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.007+0000 7f17277fe640 1 -- 172.21.15.70:0/967424251 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710031930 con 0x7f172806d4c0 2023-10-08T07:14:45.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.008+0000 7f172e6e5640 1 -- 172.21.15.70:0/967424251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f172806d4c0 msgr2=0x7f1728162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:45.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.008+0000 7f172e6e5640 1 --2- 172.21.15.70:0/967424251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f172806d4c0 0x7f1728162330 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f17280cb0b0 tx=0x7f171002b840 comp rx=0 tx=0).stop 2023-10-08T07:14:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.008+0000 7f172e6e5640 1 -- 172.21.15.70:0/967424251 shutdown_connections 2023-10-08T07:14:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.008+0000 7f172e6e5640 1 --2- 172.21.15.70:0/967424251 >> v2:172.21.15.181:3300/0 conn(0x7f17280cb150 0x7f17280cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.008+0000 7f172e6e5640 1 --2- 172.21.15.70:0/967424251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f172806d4c0 0x7f1728162330 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.008+0000 7f172e6e5640 1 -- 172.21.15.70:0/967424251 >> 172.21.15.70:0/967424251 conn(0x7f172806a380 msgr2=0x7f17280b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.008+0000 7f172e6e5640 1 -- 172.21.15.70:0/967424251 shutdown_connections 2023-10-08T07:14:45.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.008+0000 7f172e6e5640 1 -- 172.21.15.70:0/967424251 wait complete. 2023-10-08T07:14:45.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.010+0000 7f172e6e5640 1 Processor -- start 2023-10-08T07:14:45.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.010+0000 7f172e6e5640 1 -- start start 2023-10-08T07:14:45.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172e6e5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172806d4c0 0x7f172815a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172e6e5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17280add60 0x7f172815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172e6e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17280cb150 0x7f1728148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172e6e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172815a910 con 0x7f17280cb150 2023-10-08T07:14:45.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172e6e5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172815b1c0 con 0x7f172806d4c0 2023-10-08T07:14:45.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172e6e5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172815e330 con 0x7f17280add60 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172cc82640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17280cb150 0x7f1728148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172cc82640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17280cb150 0x7f1728148a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40128/0 (socket says 172.21.15.70:40128) 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172cc82640 1 -- 172.21.15.70:0/1164073287 learned_addr learned my addr 172.21.15.70:0/1164073287 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172cc82640 1 -- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17280add60 msgr2=0x7f172815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172cc82640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17280add60 0x7f172815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f172cc82640 1 -- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172806d4c0 msgr2=0x7f172815a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.011+0000 7f1727fff640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172806d4c0 0x7f172815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.012+0000 7f172cc82640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172806d4c0 0x7f172815a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.012+0000 7f172cc82640 1 -- 172.21.15.70:0/1164073287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1710009400 con 0x7f17280cb150 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.012+0000 7f1727fff640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172806d4c0 0x7f172815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:45.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.012+0000 7f172cc82640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17280cb150 0x7f1728148a40 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f172000cc30 tx=0x7f172000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:45.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.012+0000 7f17257fa640 1 -- 172.21.15.70:0/1164073287 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000ce40 con 0x7f17280cb150 2023-10-08T07:14:45.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.012+0000 7f17257fa640 1 -- 172.21.15.70:0/1164073287 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1720003990 con 0x7f17280cb150 2023-10-08T07:14:45.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.012+0000 7f17257fa640 1 -- 172.21.15.70:0/1164073287 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720003d20 con 0x7f17280cb150 2023-10-08T07:14:45.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.012+0000 7f172e6e5640 1 -- 172.21.15.70:0/1164073287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172815e610 con 0x7f17280cb150 2023-10-08T07:14:45.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.012+0000 7f172e6e5640 1 -- 172.21.15.70:0/1164073287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172815ebd0 con 0x7f17280cb150 2023-10-08T07:14:45.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.013+0000 7f172e6e5640 1 -- 172.21.15.70:0/1164073287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172815edc0 con 0x7f17280cb150 2023-10-08T07:14:45.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.014+0000 7f17257fa640 1 -- 172.21.15.70:0/1164073287 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1720003e80 con 0x7f17280cb150 2023-10-08T07:14:45.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.015+0000 7f17257fa640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f170806b0e0 0x7f170806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.015+0000 7f17257fa640 1 -- 172.21.15.70:0/1164073287 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f172008a0a0 con 0x7f17280cb150 2023-10-08T07:14:45.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.016+0000 7f1727fff640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f170806b0e0 0x7f170806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:45.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.017+0000 7f1727fff640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f170806b0e0 0x7f170806d5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1718000f60 tx=0x7f1718009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:45.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.020+0000 7f17257fa640 1 -- 172.21.15.70:0/1164073287 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1720054b10 con 0x7f17280cb150 2023-10-08T07:14:45.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.210+0000 7f172e6e5640 1 -- 172.21.15.70:0/1164073287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1728067490 con 0x7f17280cb150 2023-10-08T07:14:45.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.213+0000 7f17257fa640 1 -- 172.21.15.70:0/1164073287 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v253) v1 ==== 74+0+16971 (secure 0 0 0) 0x7f172001d020 con 0x7f17280cb150 2023-10-08T07:14:45.215 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:45.215 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":253,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:43.832930+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 -- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f170806b0e0 msgr2=0x7f170806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f170806b0e0 0x7f170806d5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1718000f60 tx=0x7f1718009040 comp rx=0 tx=0).stop 2023-10-08T07:14:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 -- 172.21.15.70:0/1164073287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17280cb150 msgr2=0x7f1728148a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17280cb150 0x7f1728148a40 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f172000cc30 tx=0x7f172000a5c0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 -- 172.21.15.70:0/1164073287 shutdown_connections 2023-10-08T07:14:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f170806b0e0 0x7f170806d5a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17280cb150 0x7f1728148a40 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f17280add60 0x7f172815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 --2- 172.21.15.70:0/1164073287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f172806d4c0 0x7f172815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 -- 172.21.15.70:0/1164073287 >> 172.21.15.70:0/1164073287 conn(0x7f172806a380 msgr2=0x7f17280ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 -- 172.21.15.70:0/1164073287 shutdown_connections 2023-10-08T07:14:45.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.216+0000 7f172e6e5640 1 -- 172.21.15.70:0/1164073287 wait complete. 2023-10-08T07:14:45.237 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T07:14:45.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.452+0000 7f5d801e9640 1 Processor -- start 2023-10-08T07:14:45.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.452+0000 7f5d801e9640 1 -- start start 2023-10-08T07:14:45.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.453+0000 7f5d801e9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d780afbe0 0x7f5d780b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.453+0000 7f5d801e9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5d7806dfc0 0x7f5d78070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.453+0000 7f5d801e9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780b2620 con 0x7f5d78071070 2023-10-08T07:14:45.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.453+0000 7f5d801e9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d780b2060 con 0x7f5d780afbe0 2023-10-08T07:14:45.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.453+0000 7f5d801e9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d780b2340 con 0x7f5d7806dfc0 2023-10-08T07:14:45.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7df85640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d780afbe0 0x7f5d780b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:45.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7df85640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d780afbe0 0x7f5d780b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40142/0 (socket says 172.21.15.70:40142) 2023-10-08T07:14:45.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7df85640 1 -- 172.21.15.70:0/3800516536 learned_addr learned my addr 172.21.15.70:0/3800516536 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:45.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7e786640 1 --1- 172.21.15.70:0/3800516536 >> v1:172.21.15.70:6789/0 conn(0x7f5d78071070 0x7f5d780cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46474/0 (socket says 172.21.15.70:46474) 2023-10-08T07:14:45.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7d784640 1 --2- 172.21.15.70:0/3800516536 >> v2:172.21.15.181:3300/0 conn(0x7f5d7806dfc0 0x7f5d78070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:45.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7df85640 1 -- 172.21.15.70:0/3800516536 >> v2:172.21.15.181:3300/0 conn(0x7f5d7806dfc0 msgr2=0x7f5d78070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:45.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7df85640 1 --2- 172.21.15.70:0/3800516536 >> v2:172.21.15.181:3300/0 conn(0x7f5d7806dfc0 0x7f5d78070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7df85640 1 -- 172.21.15.70:0/3800516536 >> v1:172.21.15.70:6789/0 conn(0x7f5d78071070 legacy=0x7f5d780cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:45.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7df85640 1 -- 172.21.15.70:0/3800516536 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d78070350 con 0x7f5d780afbe0 2023-10-08T07:14:45.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7cf83640 1 -- 172.21.15.70:0/3800516536 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470912286 0 0) 0x7f5d780b2620 con 0x7f5d78071070 2023-10-08T07:14:45.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.454+0000 7f5d7d784640 1 --2- 172.21.15.70:0/3800516536 >> v2:172.21.15.181:3300/0 conn(0x7f5d7806dfc0 0x7f5d78070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:45.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.455+0000 7f5d7df85640 1 --2- 172.21.15.70:0/3800516536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d780afbe0 0x7f5d780b15b0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f5d780b3980 tx=0x7f5d7402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b00b15448f9e0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:45.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.455+0000 7f5d7cf83640 1 -- 172.21.15.70:0/3800516536 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74030860 con 0x7f5d780afbe0 2023-10-08T07:14:45.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.456+0000 7f5d7cf83640 1 -- 172.21.15.70:0/3800516536 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d740309c0 con 0x7f5d780afbe0 2023-10-08T07:14:45.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.456+0000 7f5d7cf83640 1 -- 172.21.15.70:0/3800516536 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74030d50 con 0x7f5d780afbe0 2023-10-08T07:14:45.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.456+0000 7f5d801e9640 1 -- 172.21.15.70:0/3800516536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d780afbe0 msgr2=0x7f5d780b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:45.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.456+0000 7f5d801e9640 1 --2- 172.21.15.70:0/3800516536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d780afbe0 0x7f5d780b15b0 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7f5d780b3980 tx=0x7f5d7402b9a0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.456+0000 7f5d801e9640 1 -- 172.21.15.70:0/3800516536 shutdown_connections 2023-10-08T07:14:45.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.456+0000 7f5d801e9640 1 --2- 172.21.15.70:0/3800516536 >> v2:172.21.15.181:3300/0 conn(0x7f5d7806dfc0 0x7f5d78070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.456+0000 7f5d801e9640 1 --2- 172.21.15.70:0/3800516536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d780afbe0 0x7f5d780b15b0 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.456+0000 7f5d801e9640 1 -- 172.21.15.70:0/3800516536 >> 172.21.15.70:0/3800516536 conn(0x7f5d7806a5d0 msgr2=0x7f5d780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:45.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.456+0000 7f5d801e9640 1 -- 172.21.15.70:0/3800516536 shutdown_connections 2023-10-08T07:14:45.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.456+0000 7f5d801e9640 1 -- 172.21.15.70:0/3800516536 wait complete. 2023-10-08T07:14:45.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.458+0000 7f5d801e9640 1 Processor -- start 2023-10-08T07:14:45.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.458+0000 7f5d801e9640 1 -- start start 2023-10-08T07:14:45.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d801e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d7806dfc0 0x7f5d781f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d801e9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d78071070 0x7f5d7815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d801e9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d780afbe0 0x7f5d780ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d801e9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d781f35b0 con 0x7f5d7806dfc0 2023-10-08T07:14:45.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d801e9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780c33b0 con 0x7f5d78071070 2023-10-08T07:14:45.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d801e9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780c3690 con 0x7f5d780afbe0 2023-10-08T07:14:45.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d7df85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d7806dfc0 0x7f5d781f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:45.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d7df85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d7806dfc0 0x7f5d781f37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40152/0 (socket says 172.21.15.70:40152) 2023-10-08T07:14:45.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d7df85640 1 -- 172.21.15.70:0/1341248991 learned_addr learned my addr 172.21.15.70:0/1341248991 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:45.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d7df85640 1 -- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d780afbe0 msgr2=0x7f5d780ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:45.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d7df85640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d780afbe0 0x7f5d780ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d7df85640 1 -- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d78071070 msgr2=0x7f5d7815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:45.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d7df85640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d78071070 0x7f5d7815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:45.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.459+0000 7f5d7df85640 1 -- 172.21.15.70:0/1341248991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d740094d0 con 0x7f5d7806dfc0 2023-10-08T07:14:45.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.460+0000 7f5d7df85640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d7806dfc0 0x7f5d781f37d0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f5d740090b0 tx=0x7f5d74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:45.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.460+0000 7f5d6e7fc640 1 -- 172.21.15.70:0/1341248991 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7403e040 con 0x7f5d7806dfc0 2023-10-08T07:14:45.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.460+0000 7f5d6e7fc640 1 -- 172.21.15.70:0/1341248991 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d74039e20 con 0x7f5d7806dfc0 2023-10-08T07:14:45.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.460+0000 7f5d6e7fc640 1 -- 172.21.15.70:0/1341248991 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d74004270 con 0x7f5d7806dfc0 2023-10-08T07:14:45.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.460+0000 7f5d801e9640 1 -- 172.21.15.70:0/1341248991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d780c3970 con 0x7f5d7806dfc0 2023-10-08T07:14:45.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.460+0000 7f5d801e9640 1 -- 172.21.15.70:0/1341248991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d781f3d10 con 0x7f5d7806dfc0 2023-10-08T07:14:45.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.461+0000 7f5d801e9640 1 -- 172.21.15.70:0/1341248991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d780c3e90 con 0x7f5d7806dfc0 2023-10-08T07:14:45.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.462+0000 7f5d6e7fc640 1 -- 172.21.15.70:0/1341248991 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5d740043d0 con 0x7f5d7806dfc0 2023-10-08T07:14:45.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.464+0000 7f5d6e7fc640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5d5c06b0b0 0x7f5d5c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:45.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.464+0000 7f5d6e7fc640 1 -- 172.21.15.70:0/1341248991 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5d740ac3b0 con 0x7f5d7806dfc0 2023-10-08T07:14:45.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.464+0000 7f5d7d784640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5d5c06b0b0 0x7f5d5c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:45.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.465+0000 7f5d7d784640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5d5c06b0b0 0x7f5d5c06d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006fd0 tx=0x7f5d68008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:45.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.468+0000 7f5d6e7fc640 1 -- 172.21.15.70:0/1341248991 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d780c3e90 con 0x7f5d7806dfc0 2023-10-08T07:14:45.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:45.678+0000 7f5d801e9640 1 -- 172.21.15.70:0/1341248991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5d780c3e90 con 0x7f5d7806dfc0 2023-10-08T07:14:46.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.401+0000 7f5d6e7fc640 1 -- 172.21.15.70:0/1341248991 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v124) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5d780c3e90 con 0x7f5d7806dfc0 2023-10-08T07:14:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.404+0000 7f5d801e9640 1 -- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5d5c06b0b0 msgr2=0x7f5d5c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.404+0000 7f5d801e9640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5d5c06b0b0 0x7f5d5c06d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5d68006fd0 tx=0x7f5d68008040 comp rx=0 tx=0).stop 2023-10-08T07:14:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.404+0000 7f5d801e9640 1 -- 172.21.15.70:0/1341248991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d7806dfc0 msgr2=0x7f5d781f37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.404+0000 7f5d801e9640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d7806dfc0 0x7f5d781f37d0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f5d740090b0 tx=0x7f5d74004730 comp rx=0 tx=0).stop 2023-10-08T07:14:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.405+0000 7f5d801e9640 1 -- 172.21.15.70:0/1341248991 shutdown_connections 2023-10-08T07:14:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.405+0000 7f5d801e9640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f5d5c06b0b0 0x7f5d5c06d570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.405+0000 7f5d801e9640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5d780afbe0 0x7f5d780ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:46.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.405+0000 7f5d801e9640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5d78071070 0x7f5d7815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.405+0000 7f5d801e9640 1 --2- 172.21.15.70:0/1341248991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d7806dfc0 0x7f5d781f37d0 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.405+0000 7f5d801e9640 1 -- 172.21.15.70:0/1341248991 >> 172.21.15.70:0/1341248991 conn(0x7f5d7806a5d0 msgr2=0x7f5d780ca250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.405+0000 7f5d801e9640 1 -- 172.21.15.70:0/1341248991 shutdown_connections 2023-10-08T07:14:46.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.405+0000 7f5d801e9640 1 -- 172.21.15.70:0/1341248991 wait complete. 2023-10-08T07:14:46.427 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T07:14:46.428 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T07:14:46.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.648+0000 7f2807ad7640 1 Processor -- start 2023-10-08T07:14:46.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.648+0000 7f2807ad7640 1 -- start start 2023-10-08T07:14:46.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.650+0000 7f2807ad7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28000afbe0 0x7f28000b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:46.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.650+0000 7f2807ad7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28000b2620 con 0x7f28000cb130 2023-10-08T07:14:46.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.650+0000 7f2807ad7640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28000b2060 con 0x7f280006dfc0 2023-10-08T07:14:46.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.650+0000 7f2807ad7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28000b2340 con 0x7f28000afbe0 2023-10-08T07:14:46.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.650+0000 7f2806074640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f28000cb130 0x7f280006c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46478/0 (socket says 172.21.15.70:46478) 2023-10-08T07:14:46.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.650+0000 7f2806074640 1 -- 172.21.15.70:0/358035739 learned_addr learned my addr 172.21.15.70:0/358035739 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2805873640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28000afbe0 0x7f28000b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201529793 0 0) 0x7f28000b2620 con 0x7f28000cb130 2023-10-08T07:14:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ec003520 con 0x7f28000cb130 2023-10-08T07:14:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1890059757 0 0) 0x7f27ec003520 con 0x7f28000cb130 2023-10-08T07:14:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28000afbe0 msgr2=0x7f28000b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28000afbe0 0x7f28000b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 >> v1:172.21.15.181:6789/0 conn(0x7f280006dfc0 legacy=0x7f2800070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800070350 con 0x7f28000cb130 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f27f80019f0 con 0x7f28000cb130 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 >> v1:172.21.15.70:6789/0 conn(0x7f28000cb130 legacy=0x7f280006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28000afbe0 0x7f27ec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ec003f70 0x7f27ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec004d00 0x7f27ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000b2340 con 0x7f27ec004d00 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec00bac0 con 0x7f27ec003f70 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.651+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec00bc80 con 0x7f28000afbe0 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.652+0000 7f2806074640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec004d00 0x7f27ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.652+0000 7f2805873640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28000afbe0 0x7f27ec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.652+0000 7f2805072640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ec003f70 0x7f27ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.652+0000 7f2806074640 1 -- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28000afbe0 msgr2=0x7f27ec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.652+0000 7f2806074640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f28000afbe0 0x7f27ec0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.652+0000 7f2806074640 1 -- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ec003f70 msgr2=0x7f27ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.652+0000 7f2806074640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ec003f70 0x7f27ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.652+0000 7f2806074640 1 -- 172.21.15.70:0/358035739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec003520 con 0x7f27ec004d00 2023-10-08T07:14:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.652+0000 7f2805072640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f27ec003f70 0x7f27ec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:46.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.653+0000 7f2806074640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec004d00 0x7f27ec005180 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f27f800a750 tx=0x7f27ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a13e6dc98e4e3d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:46.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.653+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8034630 con 0x7f27ec004d00 2023-10-08T07:14:46.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.653+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f802f070 con 0x7f27ec004d00 2023-10-08T07:14:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.653+0000 7f2804871640 1 -- 172.21.15.70:0/358035739 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f80349d0 con 0x7f27ec004d00 2023-10-08T07:14:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.654+0000 7f2807ad7640 1 -- 172.21.15.70:0/358035739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec004d00 msgr2=0x7f27ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.654+0000 7f2807ad7640 1 --2- 172.21.15.70:0/358035739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27ec004d00 0x7f27ec005180 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f27f800a750 tx=0x7f27ec0057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.654+0000 7f2806074640 1 -- 172.21.15.70:0/358035739 reap_dead start 2023-10-08T07:14:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.654+0000 7f2807ad7640 1 -- 172.21.15.70:0/358035739 shutdown_connections 2023-10-08T07:14:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.654+0000 7f2807ad7640 1 -- 172.21.15.70:0/358035739 >> 172.21.15.70:0/358035739 conn(0x7f280006a5d0 msgr2=0x7f28000b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.654+0000 7f2807ad7640 1 -- 172.21.15.70:0/358035739 shutdown_connections 2023-10-08T07:14:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.654+0000 7f2807ad7640 1 -- 172.21.15.70:0/358035739 wait complete. 2023-10-08T07:14:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.656+0000 7f2807ad7640 1 Processor -- start 2023-10-08T07:14:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.656+0000 7f2807ad7640 1 -- start start 2023-10-08T07:14:46.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.657+0000 7f2807ad7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27ec004d00 0x7f28001f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:46.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.657+0000 7f2807ad7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28000c8560 0x7f2800158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:46.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.657+0000 7f2807ad7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2800158bd0 0x7f28001f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:46.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.657+0000 7f2807ad7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2800159080 con 0x7f28000c8560 2023-10-08T07:14:46.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.657+0000 7f2807ad7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000c36b0 con 0x7f2800158bd0 2023-10-08T07:14:46.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.657+0000 7f2807ad7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28001f66b0 con 0x7f27ec004d00 2023-10-08T07:14:46.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.657+0000 7f2805072640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28000c8560 0x7f2800158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:46.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.657+0000 7f2805072640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28000c8560 0x7f2800158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40174/0 (socket says 172.21.15.70:40174) 2023-10-08T07:14:46.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.657+0000 7f2805072640 1 -- 172.21.15.70:0/3511553603 learned_addr learned my addr 172.21.15.70:0/3511553603 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.658+0000 7f2805072640 1 -- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27ec004d00 msgr2=0x7f28001f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.658+0000 7f2805072640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27ec004d00 0x7f28001f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.658+0000 7f2805072640 1 -- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2800158bd0 msgr2=0x7f28001f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.658+0000 7f2805072640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2800158bd0 0x7f28001f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.658+0000 7f2805072640 1 -- 172.21.15.70:0/3511553603 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28001f6990 con 0x7f28000c8560 2023-10-08T07:14:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.658+0000 7f2805072640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28000c8560 0x7f2800158690 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f27f000d7a0 tx=0x7f27f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.658+0000 7f27f67fc640 1 -- 172.21.15.70:0/3511553603 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0005270 con 0x7f28000c8560 2023-10-08T07:14:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.658+0000 7f27f67fc640 1 -- 172.21.15.70:0/3511553603 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f00053d0 con 0x7f28000c8560 2023-10-08T07:14:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.658+0000 7f27f67fc640 1 -- 172.21.15.70:0/3511553603 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f000b850 con 0x7f28000c8560 2023-10-08T07:14:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.658+0000 7f2807ad7640 1 -- 172.21.15.70:0/3511553603 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28000c3920 con 0x7f28000c8560 2023-10-08T07:14:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.659+0000 7f2807ad7640 1 -- 172.21.15.70:0/3511553603 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28001f6c70 con 0x7f28000c8560 2023-10-08T07:14:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.660+0000 7f2807ad7640 1 -- 172.21.15.70:0/3511553603 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28000c3b10 con 0x7f28000c8560 2023-10-08T07:14:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.661+0000 7f27f67fc640 1 -- 172.21.15.70:0/3511553603 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f27f000b9b0 con 0x7f28000c8560 2023-10-08T07:14:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.661+0000 7f27f67fc640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27e406b0e0 0x7f27e406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.661+0000 7f27f67fc640 1 -- 172.21.15.70:0/3511553603 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f27f00897c0 con 0x7f28000c8560 2023-10-08T07:14:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.662+0000 7f2805873640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27e406b0e0 0x7f27e406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:46.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.663+0000 7f2805873640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27e406b0e0 0x7f27e406d5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f28001593a0 tx=0x7f27fc008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:46.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.666+0000 7f27f67fc640 1 -- 172.21.15.70:0/3511553603 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27f0054230 con 0x7f28000c8560 2023-10-08T07:14:46.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:46.897+0000 7f2807ad7640 1 -- 172.21.15.70:0/3511553603 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f280004e160 con 0x7f28000c8560 2023-10-08T07:14:47.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.404+0000 7f27f67fc640 1 -- 172.21.15.70:0/3511553603 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v125) v1 ==== 122+0+0 (secure 0 0 0) 0x7f27f0057850 con 0x7f28000c8560 2023-10-08T07:14:47.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 -- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27e406b0e0 msgr2=0x7f27e406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:47.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27e406b0e0 0x7f27e406d5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f28001593a0 tx=0x7f27fc008040 comp rx=0 tx=0).stop 2023-10-08T07:14:47.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 -- 172.21.15.70:0/3511553603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28000c8560 msgr2=0x7f2800158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:47.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28000c8560 0x7f2800158690 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f27f000d7a0 tx=0x7f27f000dc60 comp rx=0 tx=0).stop 2023-10-08T07:14:47.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 -- 172.21.15.70:0/3511553603 shutdown_connections 2023-10-08T07:14:47.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f27e406b0e0 0x7f27e406d5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2800158bd0 0x7f28001f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28000c8560 0x7f2800158690 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 --2- 172.21.15.70:0/3511553603 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f27ec004d00 0x7f28001f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 -- 172.21.15.70:0/3511553603 >> 172.21.15.70:0/3511553603 conn(0x7f280006a5d0 msgr2=0x7f2800073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:47.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 -- 172.21.15.70:0/3511553603 shutdown_connections 2023-10-08T07:14:47.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.409+0000 7f2807ad7640 1 -- 172.21.15.70:0/3511553603 wait complete. 2023-10-08T07:14:47.433 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T07:14:47.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:14:47.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.647+0000 7fc8c1456640 1 Processor -- start 2023-10-08T07:14:47.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.647+0000 7fc8c1456640 1 -- start start 2023-10-08T07:14:47.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.648+0000 7fc8c1456640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8bc15ab40 0x7fc8bc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:47.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8c1456640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8bc0c90b0 0x7fc8bc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:47.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8c1456640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8bc0cae70 0x7fc8bc0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:47.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8c1456640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8bc15bf30 con 0x7fc8bc0cae70 2023-10-08T07:14:47.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8c1456640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8bc0cb840 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8c1456640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc15b2b0 con 0x7fc8bc15ab40 2023-10-08T07:14:47.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8bb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8bc0cae70 0x7fc8bc0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:47.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8bb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8bc0cae70 0x7fc8bc0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40180/0 (socket says 172.21.15.70:40180) 2023-10-08T07:14:47.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8bb7fe640 1 -- 172.21.15.70:0/3939223957 learned_addr learned my addr 172.21.15.70:0/3939223957 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:47.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8bb7fe640 1 -- 172.21.15.70:0/3939223957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8bc15ab40 msgr2=0x7fc8bc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:47.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8bb7fe640 1 --2- 172.21.15.70:0/3939223957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8bc15ab40 0x7fc8bc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8bb7fe640 1 -- 172.21.15.70:0/3939223957 >> v2:172.21.15.181:3300/0 conn(0x7fc8bc0c90b0 msgr2=0x7fc8bc0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:47.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8bb7fe640 1 --2- 172.21.15.70:0/3939223957 >> v2:172.21.15.181:3300/0 conn(0x7fc8bc0c90b0 0x7fc8bc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.649+0000 7fc8bb7fe640 1 -- 172.21.15.70:0/3939223957 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc15b590 con 0x7fc8bc0cae70 2023-10-08T07:14:47.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.650+0000 7fc8bb7fe640 1 --2- 172.21.15.70:0/3939223957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0cae70 0x7fc8bc0cb300 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc06e470 tx=0x7fc8b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5595b0dc56c818a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:47.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.651+0000 7fc8ba7fc640 1 -- 172.21.15.70:0/3939223957 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4031870 con 0x7fc8bc0cae70 2023-10-08T07:14:47.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.651+0000 7fc8ba7fc640 1 -- 172.21.15.70:0/3939223957 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b40319d0 con 0x7fc8bc0cae70 2023-10-08T07:14:47.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.651+0000 7fc8ba7fc640 1 -- 172.21.15.70:0/3939223957 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4031d60 con 0x7fc8bc0cae70 2023-10-08T07:14:47.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.651+0000 7fc8c1456640 1 -- 172.21.15.70:0/3939223957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0cae70 msgr2=0x7fc8bc0cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:47.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.651+0000 7fc8c1456640 1 --2- 172.21.15.70:0/3939223957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0cae70 0x7fc8bc0cb300 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc06e470 tx=0x7fc8b402b910 comp rx=0 tx=0).stop 2023-10-08T07:14:47.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.652+0000 7fc8c1456640 1 -- 172.21.15.70:0/3939223957 shutdown_connections 2023-10-08T07:14:47.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.652+0000 7fc8c1456640 1 --2- 172.21.15.70:0/3939223957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0cae70 0x7fc8bc0cb300 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.652+0000 7fc8c1456640 1 --2- 172.21.15.70:0/3939223957 >> v2:172.21.15.181:3300/0 conn(0x7fc8bc0c90b0 0x7fc8bc0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.652+0000 7fc8c1456640 1 --2- 172.21.15.70:0/3939223957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8bc15ab40 0x7fc8bc0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.652+0000 7fc8c1456640 1 -- 172.21.15.70:0/3939223957 >> 172.21.15.70:0/3939223957 conn(0x7fc8bc06a380 msgr2=0x7fc8bc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:47.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.652+0000 7fc8c1456640 1 -- 172.21.15.70:0/3939223957 shutdown_connections 2023-10-08T07:14:47.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.652+0000 7fc8c1456640 1 -- 172.21.15.70:0/3939223957 wait complete. 2023-10-08T07:14:47.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.653+0000 7fc8c1456640 1 Processor -- start 2023-10-08T07:14:47.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.653+0000 7fc8c1456640 1 -- start start 2023-10-08T07:14:47.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.654+0000 7fc8c1456640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0c90b0 0x7fc8bc1492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.654+0000 7fc8c1456640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cae70 0x7fc8bc06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.654+0000 7fc8c1456640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8bc15ab40 0x7fc8bc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.654+0000 7fc8c1456640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc0c86d0 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.654+0000 7fc8c1456640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc1497e0 con 0x7fc8bc0cae70 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.654+0000 7fc8c1456640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc1571d0 con 0x7fc8bc15ab40 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.655+0000 7fc8baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0c90b0 0x7fc8bc1492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.655+0000 7fc8baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0c90b0 0x7fc8bc1492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40192/0 (socket says 172.21.15.70:40192) 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.655+0000 7fc8baffd640 1 -- 172.21.15.70:0/2014643450 learned_addr learned my addr 172.21.15.70:0/2014643450 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.655+0000 7fc8baffd640 1 -- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8bc15ab40 msgr2=0x7fc8bc06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.655+0000 7fc8b27fc640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cae70 0x7fc8bc06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.655+0000 7fc8baffd640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8bc15ab40 0x7fc8bc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.655+0000 7fc8baffd640 1 -- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cae70 msgr2=0x7fc8bc06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:47.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.655+0000 7fc8baffd640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cae70 0x7fc8bc06cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.655+0000 7fc8baffd640 1 -- 172.21.15.70:0/2014643450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40094d0 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.655+0000 7fc8baffd640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0c90b0 0x7fc8bc1492a0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00b4a0 tx=0x7fc8ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:47.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.656+0000 7fc8b3fff640 1 -- 172.21.15.70:0/2014643450 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac010610 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.656+0000 7fc8b3fff640 1 -- 172.21.15.70:0/2014643450 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac010770 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.656+0000 7fc8b3fff640 1 -- 172.21.15.70:0/2014643450 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac010b20 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.656+0000 7fc8c1456640 1 -- 172.21.15.70:0/2014643450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc156130 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.656+0000 7fc8c1456640 1 -- 172.21.15.70:0/2014643450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc1566f0 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.657+0000 7fc8c1456640 1 -- 172.21.15.70:0/2014643450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8bc15b850 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.658+0000 7fc8b3fff640 1 -- 172.21.15.70:0/2014643450 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc8ac010c80 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.659+0000 7fc8b3fff640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc8a006b0e0 0x7fc8a006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:47.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.659+0000 7fc8b3fff640 1 -- 172.21.15.70:0/2014643450 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fc8ac089900 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.659+0000 7fc8b27fc640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc8a006b0e0 0x7fc8a006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:47.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.661+0000 7fc8b27fc640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc8a006b0e0 0x7fc8a006d5a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc155440 tx=0x7fc8a40072f0 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:47.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.663+0000 7fc8b3fff640 1 -- 172.21.15.70:0/2014643450 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8bc15b850 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.905+0000 7fc8c1456640 1 -- 172.21.15.70:0/2014643450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8bc15b850 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.906+0000 7fc8b3fff640 1 -- 172.21.15.70:0/2014643450 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4926 (secure 0 0 0) 0x7fc8bc15b850 con 0x7fc8bc0c90b0 2023-10-08T07:14:47.908 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:47.908 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":8875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:43.833416+0000","modified":"2023-10-08T07:14:46.899347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11904},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11904":{"gid":11904,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/3856326680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3856326680},{"type":"v1","addr":"172.21.15.70:6834","nonce":3856326680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-10-08T07:14:47.908 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 125 2023-10-08T07:14:47.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.908+0000 7fc8c1456640 1 -- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc8a006b0e0 msgr2=0x7fc8a006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:47.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.908+0000 7fc8c1456640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc8a006b0e0 0x7fc8a006d5a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc155440 tx=0x7fc8a40072f0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.908+0000 7fc8c1456640 1 -- 172.21.15.70:0/2014643450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0c90b0 msgr2=0x7fc8bc1492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:47.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.908+0000 7fc8c1456640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0c90b0 0x7fc8bc1492a0 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00b4a0 tx=0x7fc8ac00b960 comp rx=0 tx=0).stop 2023-10-08T07:14:47.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.909+0000 7fc8c1456640 1 -- 172.21.15.70:0/2014643450 shutdown_connections 2023-10-08T07:14:47.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.909+0000 7fc8c1456640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc8a006b0e0 0x7fc8a006d5a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.909+0000 7fc8c1456640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8bc15ab40 0x7fc8bc06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.909+0000 7fc8c1456640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8bc0cae70 0x7fc8bc06cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.909+0000 7fc8c1456640 1 --2- 172.21.15.70:0/2014643450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8bc0c90b0 0x7fc8bc1492a0 unknown :-1 s=CLOSED pgs=2637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:47.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.909+0000 7fc8c1456640 1 -- 172.21.15.70:0/2014643450 >> 172.21.15.70:0/2014643450 conn(0x7fc8bc06a380 msgr2=0x7fc8bc0ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:47.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.909+0000 7fc8c1456640 1 -- 172.21.15.70:0/2014643450 shutdown_connections 2023-10-08T07:14:47.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:47.909+0000 7fc8c1456640 1 -- 172.21.15.70:0/2014643450 wait complete. 2023-10-08T07:14:47.928 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:48.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.142+0000 7fdc5b355640 1 Processor -- start 2023-10-08T07:14:48.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.142+0000 7fdc5b355640 1 -- start start 2023-10-08T07:14:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.144+0000 7fdc5b355640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc540c90b0 0x7fdc540cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.144+0000 7fdc5b355640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc5415b2b0 con 0x7fdc540cb3b0 2023-10-08T07:14:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.144+0000 7fdc5b355640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc5415b590 con 0x7fdc5415ab40 2023-10-08T07:14:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.144+0000 7fdc5b355640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5415bf30 con 0x7fdc540c90b0 2023-10-08T07:14:48.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.144+0000 7fdc598f2640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdc540cb3b0 0x7fdc5415e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46480/0 (socket says 172.21.15.70:46480) 2023-10-08T07:14:48.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.144+0000 7fdc598f2640 1 -- 172.21.15.70:0/1444760277 learned_addr learned my addr 172.21.15.70:0/1444760277 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:48.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.144+0000 7fdc588f0640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc540c90b0 0x7fdc540cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451679000 0 0) 0x7fdc5415b2b0 con 0x7fdc540cb3b0 2023-10-08T07:14:48.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc40003520 con 0x7fdc540cb3b0 2023-10-08T07:14:48.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1149958920 0 0) 0x7fdc40003520 con 0x7fdc540cb3b0 2023-10-08T07:14:48.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc540c90b0 msgr2=0x7fdc540cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:48.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc540c90b0 0x7fdc540cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 >> v1:172.21.15.181:6789/0 conn(0x7fdc5415ab40 legacy=0x7fdc540ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5415b870 con 0x7fdc540cb3b0 2023-10-08T07:14:48.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fdc4c0019f0 con 0x7fdc540cb3b0 2023-10-08T07:14:48.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 >> v1:172.21.15.70:6789/0 conn(0x7fdc540cb3b0 legacy=0x7fdc5415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc400039a0 0x7fdc40003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc540c90b0 0x7fdc400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc40004d00 0x7fdc40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5415bf30 con 0x7fdc40004d00 2023-10-08T07:14:48.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4000ba30 con 0x7fdc400039a0 2023-10-08T07:14:48.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4000bbf0 con 0x7fdc540c90b0 2023-10-08T07:14:48.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.145+0000 7fdc598f2640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc40004d00 0x7fdc40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.146+0000 7fdc588f0640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc540c90b0 0x7fdc400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.146+0000 7fdc590f1640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc400039a0 0x7fdc40003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.146+0000 7fdc598f2640 1 -- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc540c90b0 msgr2=0x7fdc400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:48.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.146+0000 7fdc598f2640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc540c90b0 0x7fdc400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.146+0000 7fdc598f2640 1 -- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc400039a0 msgr2=0x7fdc40003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:48.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.146+0000 7fdc598f2640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc400039a0 0x7fdc40003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.146+0000 7fdc598f2640 1 -- 172.21.15.70:0/1444760277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc40003520 con 0x7fdc40004d00 2023-10-08T07:14:48.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.146+0000 7fdc590f1640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc400039a0 0x7fdc40003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.147+0000 7fdc598f2640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc40004d00 0x7fdc40005180 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00a7f0 tx=0x7fdc400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68dc90db2e1c507 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.147+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0072d0 con 0x7fdc40004d00 2023-10-08T07:14:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.147+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4c02f070 con 0x7fdc40004d00 2023-10-08T07:14:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.147+0000 7fdc4bfff640 1 -- 172.21.15.70:0/1444760277 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0076b0 con 0x7fdc40004d00 2023-10-08T07:14:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.148+0000 7fdc5b355640 1 -- 172.21.15.70:0/1444760277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc40004d00 msgr2=0x7fdc40005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.148+0000 7fdc5b355640 1 --2- 172.21.15.70:0/1444760277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc40004d00 0x7fdc40005180 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00a7f0 tx=0x7fdc400057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.148+0000 7fdc598f2640 1 -- 172.21.15.70:0/1444760277 reap_dead start 2023-10-08T07:14:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.148+0000 7fdc5b355640 1 -- 172.21.15.70:0/1444760277 shutdown_connections 2023-10-08T07:14:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.148+0000 7fdc5b355640 1 -- 172.21.15.70:0/1444760277 >> 172.21.15.70:0/1444760277 conn(0x7fdc5406a380 msgr2=0x7fdc540b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.148+0000 7fdc5b355640 1 -- 172.21.15.70:0/1444760277 shutdown_connections 2023-10-08T07:14:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.148+0000 7fdc5b355640 1 -- 172.21.15.70:0/1444760277 wait complete. 2023-10-08T07:14:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.150+0000 7fdc5b355640 1 Processor -- start 2023-10-08T07:14:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.150+0000 7fdc5b355640 1 -- start start 2023-10-08T07:14:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc5b355640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc40004d00 0x7fdc54147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc5b355640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc541573e0 0x7fdc54157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc5b355640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5415c340 0x7fdc5415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc5b355640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc540c7290 con 0x7fdc5415c340 2023-10-08T07:14:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc5b355640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5415cd00 con 0x7fdc40004d00 2023-10-08T07:14:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc5b355640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc54157f50 con 0x7fdc541573e0 2023-10-08T07:14:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc598f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5415c340 0x7fdc5415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc598f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5415c340 0x7fdc5415c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40222/0 (socket says 172.21.15.70:40222) 2023-10-08T07:14:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc598f2640 1 -- 172.21.15.70:0/40149236 learned_addr learned my addr 172.21.15.70:0/40149236 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc588f0640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc541573e0 0x7fdc54157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.151+0000 7fdc590f1640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc40004d00 0x7fdc54147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc598f2640 1 -- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc541573e0 msgr2=0x7fdc54157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc598f2640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc541573e0 0x7fdc54157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc598f2640 1 -- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc40004d00 msgr2=0x7fdc54147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc598f2640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc40004d00 0x7fdc54147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc598f2640 1 -- 172.21.15.70:0/40149236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc54158230 con 0x7fdc5415c340 2023-10-08T07:14:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc590f1640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc40004d00 0x7fdc54147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:48.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc588f0640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc541573e0 0x7fdc54157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc598f2640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5415c340 0x7fdc5415c7c0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00a7f0 tx=0x7fdc5415be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc49ffb640 1 -- 172.21.15.70:0/40149236 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c001ae0 con 0x7fdc5415c340 2023-10-08T07:14:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc49ffb640 1 -- 172.21.15.70:0/40149236 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4c040480 con 0x7fdc5415c340 2023-10-08T07:14:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc49ffb640 1 -- 172.21.15.70:0/40149236 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0407f0 con 0x7fdc5415c340 2023-10-08T07:14:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc5b355640 1 -- 172.21.15.70:0/40149236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc540c6bd0 con 0x7fdc5415c340 2023-10-08T07:14:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.152+0000 7fdc5b355640 1 -- 172.21.15.70:0/40149236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc54155aa0 con 0x7fdc5415c340 2023-10-08T07:14:48.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.154+0000 7fdc5b355640 1 -- 172.21.15.70:0/40149236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc540c6a30 con 0x7fdc5415c340 2023-10-08T07:14:48.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.155+0000 7fdc49ffb640 1 -- 172.21.15.70:0/40149236 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fdc4c031030 con 0x7fdc5415c340 2023-10-08T07:14:48.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.155+0000 7fdc49ffb640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fdc3806b0e0 0x7fdc3806d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.156+0000 7fdc49ffb640 1 -- 172.21.15.70:0/40149236 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fdc4c0ab5a0 con 0x7fdc5415c340 2023-10-08T07:14:48.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.156+0000 7fdc590f1640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fdc3806b0e0 0x7fdc3806d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.157+0000 7fdc590f1640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fdc3806b0e0 0x7fdc3806d5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc50001690 tx=0x7fdc50009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:48.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.160+0000 7fdc49ffb640 1 -- 172.21.15.70:0/40149236 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc540c6a30 con 0x7fdc5415c340 2023-10-08T07:14:48.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.349+0000 7fdc5b355640 1 -- 172.21.15.70:0/40149236 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc540c73d0 con 0x7fdc5415c340 2023-10-08T07:14:48.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.351+0000 7fdc49ffb640 1 -- 172.21.15.70:0/40149236 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) v1 ==== 74+0+16958 (secure 0 0 0) 0x7fdc540c73d0 con 0x7fdc5415c340 2023-10-08T07:14:48.353 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:48.354 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":254,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:45.298385+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:48.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 -- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fdc3806b0e0 msgr2=0x7fdc3806d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fdc3806b0e0 0x7fdc3806d5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdc50001690 tx=0x7fdc50009040 comp rx=0 tx=0).stop 2023-10-08T07:14:48.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 -- 172.21.15.70:0/40149236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5415c340 msgr2=0x7fdc5415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5415c340 0x7fdc5415c7c0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00a7f0 tx=0x7fdc5415be30 comp rx=0 tx=0).stop 2023-10-08T07:14:48.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 -- 172.21.15.70:0/40149236 shutdown_connections 2023-10-08T07:14:48.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fdc3806b0e0 0x7fdc3806d5a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc5415c340 0x7fdc5415c7c0 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdc541573e0 0x7fdc54157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 --2- 172.21.15.70:0/40149236 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdc40004d00 0x7fdc54147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 -- 172.21.15.70:0/40149236 >> 172.21.15.70:0/40149236 conn(0x7fdc5406a380 msgr2=0x7fdc540cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:48.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 -- 172.21.15.70:0/40149236 shutdown_connections 2023-10-08T07:14:48.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.354+0000 7fdc5b355640 1 -- 172.21.15.70:0/40149236 wait complete. 2023-10-08T07:14:48.374 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T07:14:48.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:14:48.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.588+0000 7fbfbe5f0640 1 Processor -- start 2023-10-08T07:14:48.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.588+0000 7fbfbe5f0640 1 -- start start 2023-10-08T07:14:48.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.590+0000 7fbfbe5f0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbfb80c90b0 0x7fbfb80cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.590+0000 7fbfbe5f0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfb80cb3b0 0x7fbfb815e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.590+0000 7fbfbe5f0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb815b2b0 con 0x7fbfb815ab40 2023-10-08T07:14:48.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.590+0000 7fbfbe5f0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb815bf30 con 0x7fbfb80c90b0 2023-10-08T07:14:48.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.590+0000 7fbfbe5f0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80cb820 con 0x7fbfb80cb3b0 2023-10-08T07:14:48.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.590+0000 7fbfb7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbfb815ab40 0x7fbfb80ad720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46490/0 (socket says 172.21.15.70:46490) 2023-10-08T07:14:48.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.590+0000 7fbfb7fff640 1 -- 172.21.15.70:0/305989213 learned_addr learned my addr 172.21.15.70:0/305989213 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:48.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.590+0000 7fbfbcb8d640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfb80cb3b0 0x7fbfb815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.590+0000 7fbfaffff640 1 --2- 172.21.15.70:0/305989213 >> v2:172.21.15.181:3300/0 conn(0x7fbfb80c90b0 0x7fbfb80cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.590+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749453192 0 0) 0x7fbfb815b2b0 con 0x7fbfb815ab40 2023-10-08T07:14:48.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfa4003520 con 0x7fbfb815ab40 2023-10-08T07:14:48.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3533839529 0 0) 0x7fbfa4003520 con 0x7fbfb815ab40 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfb80cb3b0 msgr2=0x7fbfb815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfb80cb3b0 0x7fbfb815e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 >> v2:172.21.15.181:3300/0 conn(0x7fbfb80c90b0 msgr2=0x7fbfb80cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 --2- 172.21.15.70:0/305989213 >> v2:172.21.15.181:3300/0 conn(0x7fbfb80c90b0 0x7fbfb80cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb815b590 con 0x7fbfb815ab40 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fbfa8001690 con 0x7fbfb815ab40 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 >> v1:172.21.15.70:6789/0 conn(0x7fbfb815ab40 legacy=0x7fbfb80ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40039a0 0x7fbfa4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfb80cb3b0 0x7fbfa40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa4004d00 0x7fbfa4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb815bf30 con 0x7fbfa40039a0 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80cb820 con 0x7fbfa4004d00 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa400b980 con 0x7fbfb80cb3b0 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.591+0000 7fbfb7fff640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40039a0 0x7fbfa4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.592+0000 7fbfaffff640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfb80cb3b0 0x7fbfa40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.592+0000 7fbfbcb8d640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa4004d00 0x7fbfa4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.592+0000 7fbfb7fff640 1 -- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfb80cb3b0 msgr2=0x7fbfa40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.592+0000 7fbfb7fff640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfb80cb3b0 0x7fbfa40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.592+0000 7fbfb7fff640 1 -- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa4004d00 msgr2=0x7fbfa4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.592+0000 7fbfb7fff640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa4004d00 0x7fbfa4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.592+0000 7fbfb7fff640 1 -- 172.21.15.70:0/305989213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa4003520 con 0x7fbfa40039a0 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.592+0000 7fbfbcb8d640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfa4004d00 0x7fbfa4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:48.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.592+0000 7fbfaffff640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfb80cb3b0 0x7fbfa40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:48.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.594+0000 7fbfb7fff640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40039a0 0x7fbfa4003dc0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800c0e0 tx=0x7fbfa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986b64e483ae7f70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:48.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.594+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8005ad0 con 0x7fbfa40039a0 2023-10-08T07:14:48.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.594+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa8005c30 con 0x7fbfa40039a0 2023-10-08T07:14:48.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.594+0000 7fbfb77fe640 1 -- 172.21.15.70:0/305989213 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa802c790 con 0x7fbfa40039a0 2023-10-08T07:14:48.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.595+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/305989213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40039a0 msgr2=0x7fbfa4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.595+0000 7fbfbe5f0640 1 --2- 172.21.15.70:0/305989213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfa40039a0 0x7fbfa4003dc0 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800c0e0 tx=0x7fbfa40057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.595+0000 7fbfbcb8d640 1 -- 172.21.15.70:0/305989213 reap_dead start 2023-10-08T07:14:48.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.595+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/305989213 shutdown_connections 2023-10-08T07:14:48.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.595+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/305989213 >> 172.21.15.70:0/305989213 conn(0x7fbfb806a380 msgr2=0x7fbfb80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:48.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.595+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/305989213 shutdown_connections 2023-10-08T07:14:48.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.595+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/305989213 wait complete. 2023-10-08T07:14:48.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.597+0000 7fbfbe5f0640 1 Processor -- start 2023-10-08T07:14:48.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.597+0000 7fbfbe5f0640 1 -- start start 2023-10-08T07:14:48.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.598+0000 7fbfbe5f0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa40039a0 0x7fbfb81564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.598+0000 7fbfbe5f0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfb80c7190 0x7fbfb806c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.598+0000 7fbfbe5f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfb80b4430 0x7fbfb806cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.598+0000 7fbfbe5f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80b1fe0 con 0x7fbfb80b4430 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.598+0000 7fbfbe5f0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8156e20 con 0x7fbfb80c7190 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.598+0000 7fbfbe5f0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8157100 con 0x7fbfa40039a0 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.598+0000 7fbfbcb8d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfb80b4430 0x7fbfb806cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.598+0000 7fbfbcb8d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfb80b4430 0x7fbfb806cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40236/0 (socket says 172.21.15.70:40236) 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.598+0000 7fbfbcb8d640 1 -- 172.21.15.70:0/1636296061 learned_addr learned my addr 172.21.15.70:0/1636296061 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfaffff640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfb80c7190 0x7fbfb806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfb7fff640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa40039a0 0x7fbfb81564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfbcb8d640 1 -- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa40039a0 msgr2=0x7fbfb81564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfbcb8d640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa40039a0 0x7fbfb81564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfbcb8d640 1 -- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfb80c7190 msgr2=0x7fbfb806c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfbcb8d640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfb80c7190 0x7fbfb806c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfbcb8d640 1 -- 172.21.15.70:0/1636296061 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb81573e0 con 0x7fbfb80b4430 2023-10-08T07:14:48.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfaffff640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfb80c7190 0x7fbfb806c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:48.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfbcb8d640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfb80b4430 0x7fbfb806cdb0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7fbfb806e3f0 tx=0x7fbfb0010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:48.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfb57fa640 1 -- 172.21.15.70:0/1636296061 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb001b070 con 0x7fbfb80b4430 2023-10-08T07:14:48.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfb57fa640 1 -- 172.21.15.70:0/1636296061 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb000f040 con 0x7fbfb80b4430 2023-10-08T07:14:48.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/1636296061 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb80b1b70 con 0x7fbfb80b4430 2023-10-08T07:14:48.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfb57fa640 1 -- 172.21.15.70:0/1636296061 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb000e050 con 0x7fbfb80b4430 2023-10-08T07:14:48.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.599+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/1636296061 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb8156bd0 con 0x7fbfb80b4430 2023-10-08T07:14:48.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.601+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/1636296061 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb80b2190 con 0x7fbfb80b4430 2023-10-08T07:14:48.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.601+0000 7fbfb57fa640 1 -- 172.21.15.70:0/1636296061 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbfb0015440 con 0x7fbfb80b4430 2023-10-08T07:14:48.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.603+0000 7fbfb57fa640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbf9806b0b0 0x7fbf9806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:48.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.603+0000 7fbfb57fa640 1 -- 172.21.15.70:0/1636296061 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fbfb008a140 con 0x7fbfb80b4430 2023-10-08T07:14:48.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.604+0000 7fbfb7fff640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbf9806b0b0 0x7fbf9806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:48.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.605+0000 7fbfb7fff640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbf9806b0b0 0x7fbf9806d570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8034000 tx=0x7fbfa8037040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:48.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.607+0000 7fbfb57fa640 1 -- 172.21.15.70:0/1636296061 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbfb80b2190 con 0x7fbfb80b4430 2023-10-08T07:14:48.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.795+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/1636296061 --> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfb815bb50 con 0x7fbf9806b0b0 2023-10-08T07:14:48.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.823+0000 7fbfb57fa640 1 -- 172.21.15.70:0/1636296061 <== mgr.8662 v2:172.21.15.181:6832/4245028292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275253 (secure 0 0 0) 0x7fbfa8005da0 con 0x7fbf9806b0b0 2023-10-08T07:14:48.828 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:48.830 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:14:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbf9806b0b0 msgr2=0x7fbf9806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbf9806b0b0 0x7fbf9806d570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8034000 tx=0x7fbfa8037040 comp rx=0 tx=0).stop 2023-10-08T07:14:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/1636296061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfb80b4430 msgr2=0x7fbfb806cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfb80b4430 0x7fbfb806cdb0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7fbfb806e3f0 tx=0x7fbfb0010590 comp rx=0 tx=0).stop 2023-10-08T07:14:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/1636296061 shutdown_connections 2023-10-08T07:14:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fbf9806b0b0 0x7fbf9806d570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfb80b4430 0x7fbfb806cdb0 unknown :-1 s=CLOSED pgs=2643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbfb80c7190 0x7fbfb806c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 --2- 172.21.15.70:0/1636296061 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbfa40039a0 0x7fbfb81564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:48.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/1636296061 >> 172.21.15.70:0/1636296061 conn(0x7fbfb806a380 msgr2=0x7fbfb80ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:48.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.829+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/1636296061 shutdown_connections 2023-10-08T07:14:48.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:48.830+0000 7fbfbe5f0640 1 -- 172.21.15.70:0/1636296061 wait complete. 2023-10-08T07:14:48.851 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-10-08T07:14:46.866573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":629,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2828,"ondisk_log_size":2828,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422440,"kb_used_data":25840,"kb_used_omap":12,"kb_used_meta":396467,"kb_avail":754552280,"statfs":{"total":773094113280,"available":772661534720,"internally_reserved":0,"allocated":26460160,"data_stored":14350798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":405982779},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.504828"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.367769+0000","last_change":"2023-10-08T07:14:42.367769+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.367769+0000","last_undegraded":"2023-10-08T07:14:42.367769+0000","last_fullsized":"2023-10-08T07:14:42.367769+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.381851+0000","last_change":"2023-10-08T07:14:42.381851+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.381851+0000","last_undegraded":"2023-10-08T07:14:42.381851+0000","last_fullsized":"2023-10-08T07:14:42.381851+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.370132+0000","last_change":"2023-10-08T07:14:42.370132+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.370132+0000","last_undegraded":"2023-10-08T07:14:42.370132+0000","last_fullsized":"2023-10-08T07:14:42.370132+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.376370+0000","last_change":"2023-10-08T07:14:42.376370+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.376370+0000","last_undegraded":"2023-10-08T07:14:42.376370+0000","last_fullsized":"2023-10-08T07:14:42.376370+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.369531+0000","last_change":"2023-10-08T07:14:42.369531+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.369531+0000","last_undegraded":"2023-10-08T07:14:42.369531+0000","last_fullsized":"2023-10-08T07:14:42.369531+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.374540+0000","last_change":"2023-10-08T07:14:42.374540+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.374540+0000","last_undegraded":"2023-10-08T07:14:42.374540+0000","last_fullsized":"2023-10-08T07:14:42.374540+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.367151+0000","last_change":"2023-10-08T07:14:42.367151+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.367151+0000","last_undegraded":"2023-10-08T07:14:42.367151+0000","last_fullsized":"2023-10-08T07:14:42.367151+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.369634+0000","last_change":"2023-10-08T07:14:42.369634+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.369634+0000","last_undegraded":"2023-10-08T07:14:42.369634+0000","last_fullsized":"2023-10-08T07:14:42.369634+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.374446+0000","last_change":"2023-10-08T07:14:42.374446+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.374446+0000","last_undegraded":"2023-10-08T07:14:42.374446+0000","last_fullsized":"2023-10-08T07:14:42.374446+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.373074+0000","last_change":"2023-10-08T07:14:42.373074+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.373074+0000","last_undegraded":"2023-10-08T07:14:42.373074+0000","last_fullsized":"2023-10-08T07:14:42.373074+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.367127+0000","last_change":"2023-10-08T07:14:42.367127+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.367127+0000","last_undegraded":"2023-10-08T07:14:42.367127+0000","last_fullsized":"2023-10-08T07:14:42.367127+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.374970+0000","last_change":"2023-10-08T07:14:42.374970+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.374970+0000","last_undegraded":"2023-10-08T07:14:42.374970+0000","last_fullsized":"2023-10-08T07:14:42.374970+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.373056+0000","last_change":"2023-10-08T07:14:42.373056+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.373056+0000","last_undegraded":"2023-10-08T07:14:42.373056+0000","last_fullsized":"2023-10-08T07:14:42.373056+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.371804+0000","last_change":"2023-10-08T07:14:42.371804+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.371804+0000","last_undegraded":"2023-10-08T07:14:42.371804+0000","last_fullsized":"2023-10-08T07:14:42.371804+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.371074+0000","last_change":"2023-10-08T07:14:42.371074+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.371074+0000","last_undegraded":"2023-10-08T07:14:42.371074+0000","last_fullsized":"2023-10-08T07:14:42.371074+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.367162+0000","last_change":"2023-10-08T07:14:42.367162+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.367162+0000","last_undegraded":"2023-10-08T07:14:42.367162+0000","last_fullsized":"2023-10-08T07:14:42.367162+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.370789+0000","last_change":"2023-10-08T07:14:42.370789+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.370789+0000","last_undegraded":"2023-10-08T07:14:42.370789+0000","last_fullsized":"2023-10-08T07:14:42.370789+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.380063+0000","last_change":"2023-10-08T07:14:42.380063+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.380063+0000","last_undegraded":"2023-10-08T07:14:42.380063+0000","last_fullsized":"2023-10-08T07:14:42.380063+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.375596+0000","last_change":"2023-10-08T07:14:42.375596+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.375596+0000","last_undegraded":"2023-10-08T07:14:42.375596+0000","last_fullsized":"2023-10-08T07:14:42.375596+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.373742+0000","last_change":"2023-10-08T07:14:42.373742+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.373742+0000","last_undegraded":"2023-10-08T07:14:42.373742+0000","last_fullsized":"2023-10-08T07:14:42.373742+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.379892+0000","last_change":"2023-10-08T07:14:42.379892+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.379892+0000","last_undegraded":"2023-10-08T07:14:42.379892+0000","last_fullsized":"2023-10-08T07:14:42.379892+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.380563+0000","last_change":"2023-10-08T07:14:42.380563+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.380563+0000","last_undegraded":"2023-10-08T07:14:42.380563+0000","last_fullsized":"2023-10-08T07:14:42.380563+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.369523+0000","last_change":"2023-10-08T07:14:42.369523+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.369523+0000","last_undegraded":"2023-10-08T07:14:42.369523+0000","last_fullsized":"2023-10-08T07:14:42.369523+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.373837+0000","last_change":"2023-10-08T07:14:42.373837+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.373837+0000","last_undegraded":"2023-10-08T07:14:42.373837+0000","last_fullsized":"2023-10-08T07:14:42.373837+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.371855+0000","last_change":"2023-10-08T07:14:42.371855+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.371855+0000","last_undegraded":"2023-10-08T07:14:42.371855+0000","last_fullsized":"2023-10-08T07:14:42.371855+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.369751+0000","last_change":"2023-10-08T07:14:42.369751+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.369751+0000","last_undegraded":"2023-10-08T07:14:42.369751+0000","last_fullsized":"2023-10-08T07:14:42.369751+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.369509+0000","last_change":"2023-10-08T07:14:42.369509+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.369509+0000","last_undegraded":"2023-10-08T07:14:42.369509+0000","last_fullsized":"2023-10-08T07:14:42.369509+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.379965+0000","last_change":"2023-10-08T07:14:42.379965+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.379965+0000","last_undegraded":"2023-10-08T07:14:42.379965+0000","last_fullsized":"2023-10-08T07:14:42.379965+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.371834+0000","last_change":"2023-10-08T07:14:42.371834+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.371834+0000","last_undegraded":"2023-10-08T07:14:42.371834+0000","last_fullsized":"2023-10-08T07:14:42.371834+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.373985+0000","last_change":"2023-10-08T07:14:42.373985+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.373985+0000","last_undegraded":"2023-10-08T07:14:42.373985+0000","last_fullsized":"2023-10-08T07:14:42.373985+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.379856+0000","last_change":"2023-10-08T07:14:42.379856+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.379856+0000","last_undegraded":"2023-10-08T07:14:42.379856+0000","last_fullsized":"2023-10-08T07:14:42.379856+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T07:14:43.361570+0000","last_change":"2023-10-08T07:14:43.361570+0000","last_active":"2023-10-08T07:14:43.361570+0000","last_peered":"2023-10-08T07:14:43.361570+0000","last_clean":"2023-10-08T07:14:43.361570+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:43.361570+0000","last_undegraded":"2023-10-08T07:14:43.361570+0000","last_fullsized":"2023-10-08T07:14:43.361570+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":1,"reported_epoch":251,"state":"creating+peering","last_fresh":"2023-10-08T07:14:42.369576+0000","last_change":"2023-10-08T07:14:42.369576+0000","last_active":"2023-10-08T07:14:42.343058+0000","last_peered":"2023-10-08T07:14:42.343058+0000","last_clean":"2023-10-08T07:14:42.343058+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T07:14:42.369576+0000","last_undegraded":"2023-10-08T07:14:42.369576+0000","last_fullsized":"2023-10-08T07:14:42.369576+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"240'2828","reported_seq":3813,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-10-08T07:14:39.768116+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:14:39.768116+0000","last_peered":"2023-10-08T07:14:39.768116+0000","last_clean":"2023-10-08T07:14:39.768116+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:14:39.768116+0000","last_undegraded":"2023-10-08T07:14:39.768116+0000","last_fullsized":"2023-10-08T07:14:39.768116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2828,"log_dups_size":0,"ondisk_log_size":2828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":629,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":629,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2828,"ondisk_log_size":2828,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65376,"kb_used_data":3668,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306464,"statfs":{"total":96636764160,"available":96569819136,"internally_reserved":0,"allocated":3756032,"data_stored":2238189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.137,"15min":1.1970000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4199999999999999,"5min":1.968,"15min":2.008},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.179,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.071}]},{"osd":1,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.226,"15min":1.2430000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.528,"5min":1.79,"15min":1.998},"last":1.282},{"interface":"front","average":{"1min":1.258,"5min":1.3260000000000001,"15min":1.3420000000000001},"min":{"1min":0.997,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.97799999999999998}]},{"osd":2,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.27,"15min":1.302},"min":{"1min":0.86399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":2.1640000000000001},"last":1.01},{"interface":"front","average":{"1min":1.28,"5min":1.359,"15min":1.3420000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.7150000000000001,"5min":1.8580000000000001,"15min":2.3170000000000002},"last":0.93400000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2949999999999999,"15min":1.3360000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.7549999999999999,"5min":1.8160000000000001,"15min":1.9039999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.3899999999999999,"15min":1.407},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8140000000000001,"5min":1.8680000000000001,"15min":1.8700000000000001},"last":1.448}]},{"osd":4,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.321,"15min":1.3220000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.929,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.3979999999999999,"15min":1.4059999999999999},"min":{"1min":1.018,"5min":0.92600000000000005,"15min":0.85199999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":2.0379999999999998},"last":1.222}]},{"osd":5,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4490000000000001,"15min":1.4430000000000001},"min":{"1min":0.97399999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.0670000000000002,"5min":2.0920000000000001,"15min":2.1139999999999999},"last":1.524},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4750000000000001,"15min":1.5009999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.8620000000000001,"5min":2.23,"15min":2.23},"last":1.411}]},{"osd":6,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.5900000000000001,"15min":1.6020000000000001},"min":{"1min":1.2529999999999999,"5min":1.048,"15min":1.048},"max":{"1min":2.2120000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.5409999999999999,"15min":1.54},"min":{"1min":1.139,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.982,"5min":2.21,"15min":2.2570000000000001},"last":1.327}]}]},{"osd":6,"up_from":13,"seq":55834576020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30108,"kb_used_data":3076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341732,"statfs":{"total":96636764160,"available":96605933568,"internally_reserved":0,"allocated":3149824,"data_stored":1641568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.286,"15min":1.3089999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.637,"5min":1.794,"15min":1.962},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3120000000000001,"15min":1.3520000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6220000000000001,"5min":1.76,"15min":2.0070000000000001},"last":1.5009999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2030000000000001,"15min":1.2270000000000001},"min":{"1min":0.71999999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.7330000000000001,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.222,"15min":1.2430000000000001},"min":{"1min":0.71799999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.5640000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.1599999999999999,"15min":1.2310000000000001},"min":{"1min":0.86899999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":2.339},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2749999999999999,"15min":1.306},"min":{"1min":0.77100000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.627,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":1.6910000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.397},"min":{"1min":0.86299999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8280000000000001,"5min":1.9319999999999999,"15min":1.95},"last":1.8160000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.429,"15min":1.4890000000000001},"min":{"1min":0.95199999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.7789999999999999,"5min":1.8959999999999999,"15min":2.1459999999999999},"last":1.8979999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3720000000000001,"15min":1.389},"min":{"1min":0.877,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.849,"5min":1.9359999999999999,"15min":2.0859999999999999},"last":2.0510000000000002},{"interface":"front","average":{"1min":1.518,"5min":1.4550000000000001,"15min":1.4570000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":1.9650000000000001,"5min":2.1110000000000002,"15min":2.484},"last":1.746}]},{"osd":5,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.389,"15min":1.4279999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8069999999999999,"5min":1.8660000000000001,"15min":2.403},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.5489999999999999,"15min":1.5549999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":1.948}]},{"osd":7,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5369999999999999,"15min":1.5760000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.339,"5min":2.339,"15min":2.5870000000000002},"last":2.008},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.552,"15min":1.5620000000000001},"min":{"1min":0.99299999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.0710000000000002,"5min":2.2959999999999998,"15min":2.665},"last":1.9810000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60636,"kb_used_data":3088,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311204,"statfs":{"total":96636764160,"available":96574672896,"internally_reserved":0,"allocated":3162112,"data_stored":1647821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.161,"15min":1.2070000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.391},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1930000000000001,"15min":1.2230000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.4399999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1970000000000001,"15min":1.2370000000000001},"min":{"1min":0.66800000000000004,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.339,"15min":1.3120000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":7.5250000000000004,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":1.506}]},{"osd":2,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.266,"15min":1.3009999999999999},"min":{"1min":0.68899999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.0870000000000002},"last":1.046},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2450000000000001,"15min":1.3009999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.645,"5min":1.877,"15min":1.909},"last":1.677}]},{"osd":3,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.4159999999999999,"15min":1.391},"min":{"1min":0.72499999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":6.8449999999999998,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":1.097},{"interface":"front","average":{"1min":1.619,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":0.64900000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":9.3599999999999994,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":1.651}]},{"osd":4,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.319,"15min":1.3280000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.857},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4319999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.742,"5min":2.742,"15min":2.742},"last":1.752}]},{"osd":6,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.978,"5min":1.5329999999999999,"15min":1.486},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":13.234999999999999,"5min":13.234999999999999,"15min":13.234999999999999},"last":1.726},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.5129999999999999,"15min":1.482},"min":{"1min":0.67800000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":7.2949999999999999,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":1.9770000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.536,"15min":1.5449999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.611},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5700000000000001,"15min":1.601},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":2.02}]}]},{"osd":4,"up_from":13,"seq":55834576020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48156,"kb_used_data":3076,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323684,"statfs":{"total":96636764160,"available":96587452416,"internally_reserved":0,"allocated":3149824,"data_stored":1641568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.323,"15min":1.3520000000000001},"min":{"1min":0.66800000000000004,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":5.4800000000000004,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.421,"15min":1.4219999999999999},"min":{"1min":0.79200000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.0880000000000001,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.2549999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.72599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.3129999999999999,"15min":1.2729999999999999},"min":{"1min":0.61299999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":7.673,"5min":7.673,"15min":7.673},"last":1.5680000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":2.2629999999999999,"5min":1.4690000000000001,"15min":1.3260000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":17.175999999999998,"5min":17.175999999999998,"15min":17.175999999999998},"last":1.113},{"interface":"front","average":{"1min":1.651,"5min":1.4139999999999999,"15min":1.3859999999999999},"min":{"1min":0.629,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":0.99299999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.3600000000000001,"15min":1.347},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":7.5270000000000001,"5min":7.5270000000000001,"15min":7.5270000000000001},"last":1.393},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.5209999999999999,"15min":1.4430000000000001},"min":{"1min":0.59199999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":7.7599999999999998,"5min":7.7599999999999998,"15min":7.7599999999999998},"last":1.2789999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.55,"15min":1.419},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":7.4020000000000001,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.365},{"interface":"front","average":{"1min":1.754,"5min":1.4970000000000001,"15min":1.415},"min":{"1min":0.442,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":7.79,"5min":7.79,"15min":7.79},"last":0.93700000000000006}]},{"osd":6,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.621,"15min":1.5},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":10.542999999999999,"5min":10.542999999999999,"15min":10.542999999999999},"last":1.04},{"interface":"front","average":{"1min":2.1819999999999999,"5min":1.7150000000000001,"15min":1.605},"min":{"1min":0.86099999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":10.433999999999999,"5min":10.433999999999999,"15min":10.433999999999999},"last":1.167}]},{"osd":7,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.8899999999999999,"5min":1.6259999999999999,"15min":1.5409999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":7.915,"5min":7.915,"15min":7.915},"last":1.502},{"interface":"front","average":{"1min":1.8839999999999999,"5min":1.73,"15min":1.6559999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":1.339}]}]},{"osd":3,"up_from":13,"seq":55834576022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52316,"kb_used_data":3088,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319524,"statfs":{"total":96636764160,"available":96583192576,"internally_reserved":0,"allocated":3162112,"data_stored":1647821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.96899999999999997,"15min":0.98499999999999999},"min":{"1min":0.67800000000000005,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.087,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.123,"5min":1.1559999999999999,"15min":1.181},"min":{"1min":0.748,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.607,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.137}]},{"osd":1,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":1.0029999999999999,"15min":1.034},"min":{"1min":0.66200000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.417,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.2390000000000001,"15min":1.2869999999999999},"min":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4390000000000001,"5min":2.109,"15min":2.4889999999999999},"last":0.752}]},{"osd":2,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.123,"15min":1.151},"min":{"1min":0.63700000000000001,"5min":0.437,"15min":0.437},"max":{"1min":1.667,"5min":2.165,"15min":2.2250000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.177,"15min":1.202},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.5349999999999999,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.49099999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.2549999999999999,"15min":1.282},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.67500000000000004},"max":{"1min":1.51,"5min":1.857,"15min":2.5379999999999998},"last":1.077},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4179999999999999,"15min":1.46},"min":{"1min":0.93600000000000005,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.954,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.1060000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3200000000000001,"15min":1.3759999999999999},"min":{"1min":0.97899999999999998,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.8300000000000001,"5min":1.923,"15min":2.9740000000000002},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.419,"15min":1.4339999999999999},"min":{"1min":0.77800000000000002,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8380000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":1.3240000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4099999999999999,"15min":1.4239999999999999},"min":{"1min":0.91000000000000003,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6339999999999999,"5min":1.976,"15min":2.653},"last":1.393},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4710000000000001,"15min":1.5269999999999999},"min":{"1min":1.0960000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.8540000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.456}]},{"osd":7,"last update":"Sun Oct 8 07:13:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.5660000000000001},"min":{"1min":0.88800000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9450000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.204},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.458,"15min":1.5209999999999999},"min":{"1min":1.202,"5min":0.628,"15min":0.628},"max":{"1min":1.7889999999999999,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.2509999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52316,"kb_used_data":3088,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319524,"statfs":{"total":96636764160,"available":96583192576,"internally_reserved":0,"allocated":3162112,"data_stored":1647821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.91600000000000004,"15min":0.97199999999999998},"min":{"1min":0.49199999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.948},"last":0.623},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.008,"15min":1.0660000000000001},"min":{"1min":0.55700000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.423,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":0.439}]},{"osd":1,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.93400000000000005,"15min":0.98999999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.413,"5min":1.6279999999999999,"15min":2.044},"last":0.53900000000000003},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.1479999999999999,"15min":1.1890000000000001},"min":{"1min":0.61499999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":2.1309999999999998},"last":0.55600000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.141,"15min":1.177},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.786,"5min":1.786,"15min":1.8420000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.236,"15min":1.272},"min":{"1min":0.745,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.663,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":0.69099999999999995}]},{"osd":4,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.391,"15min":1.4379999999999999},"min":{"1min":0.92900000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.992,"5min":1.992,"15min":2.1789999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.304,"15min":1.3120000000000001},"min":{"1min":0.87,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.843,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.70299999999999996}]},{"osd":5,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3600000000000001,"15min":1.4019999999999999},"min":{"1min":0.89200000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.931,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.4930000000000001,"15min":1.532},"min":{"1min":1.1060000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.837,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":0.71599999999999997}]},{"osd":6,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.361,"15min":1.395},"min":{"1min":1.0549999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.722,"5min":2.056,"15min":2.1240000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4830000000000001,"15min":1.5249999999999999},"min":{"1min":0.99299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.994,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.446,"15min":1.4790000000000001},"min":{"1min":1.0800000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7150000000000001,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.51,"15min":1.577},"min":{"1min":1.028,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.056,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":0.72999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834576020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52316,"kb_used_data":3088,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319524,"statfs":{"total":96636764160,"available":96583192576,"internally_reserved":0,"allocated":3162112,"data_stored":1647821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.999,"15min":1.0049999999999999},"min":{"1min":0.61699999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3380000000000001,"5min":1.6579999999999999,"15min":1.708},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.1299999999999999,"15min":1.212},"min":{"1min":0.61299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.5700000000000001,"5min":1.7010000000000001,"15min":1.79},"last":1.4059999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.94399999999999995,"15min":0.96599999999999997},"min":{"1min":0.54700000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.462,"5min":1.641,"15min":1.641},"last":0.40799999999999997},{"interface":"front","average":{"1min":1.123,"5min":1.1379999999999999,"15min":1.1559999999999999},"min":{"1min":0.48699999999999999,"5min":0.372,"15min":0.372},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.42599999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2010000000000001,"15min":1.2549999999999999},"min":{"1min":0.53200000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.66300000000000003,"5min":0.435,"15min":0.435},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":0.85899999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3140000000000001,"15min":1.3620000000000001},"min":{"1min":0.79900000000000004,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.73,"5min":2.73,"15min":2.8460000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.2989999999999999,"15min":1.3169999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.79500000000000004}]},{"osd":5,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3580000000000001,"15min":1.381},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.544,"5min":1.5349999999999999,"15min":1.5329999999999999},"min":{"1min":1.1379999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":0.92400000000000004}]},{"osd":6,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.492,"15min":1.5129999999999999},"min":{"1min":0.95999999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.883,"5min":2.883,"15min":2.9079999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4750000000000001,"15min":1.4990000000000001},"min":{"1min":1.0629999999999999,"5min":1.046,"15min":1.046},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.552,"15min":1.554},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.524,"5min":1.538,"15min":1.5700000000000001},"min":{"1min":0.89700000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":0.99199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576021,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61216,"kb_used_data":3668,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310624,"statfs":{"total":96636764160,"available":96574078976,"internally_reserved":0,"allocated":3756032,"data_stored":2238189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92000000000000004,"15min":0.96499999999999997},"min":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.321,"5min":1.581,"15min":1.581},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.016,"15min":1.0840000000000001},"min":{"1min":0.48899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.4039999999999999,"5min":1.621,"15min":1.726},"last":0.433}]},{"osd":2,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.081,"15min":1.1379999999999999},"min":{"1min":0.52000000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.071},"min":{"1min":0.51300000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4690000000000001,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":1.071}]},{"osd":3,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1859999999999999,"15min":1.2450000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5600000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.339},{"interface":"front","average":{"1min":1.046,"5min":1.141,"15min":1.165},"min":{"1min":0.55100000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":2.0979999999999999},"last":1.2}]},{"osd":4,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3080000000000001,"15min":1.3400000000000001},"min":{"1min":0.746,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6359999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.292},{"interface":"front","average":{"1min":1.284,"5min":1.361,"15min":1.4079999999999999},"min":{"1min":0.79900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.181},"last":1.2749999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3939999999999999,"15min":1.4159999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.802,"5min":1.919,"15min":2.1309999999999998},"last":1.401},{"interface":"front","average":{"1min":1.466,"5min":1.47,"15min":1.49},"min":{"1min":0.81999999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.23,"5min":2.23,"15min":2.4900000000000002},"last":1.1399999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4359999999999999,"15min":1.4770000000000001},"min":{"1min":0.94999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0150000000000001,"5min":2.21,"15min":2.2149999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.462,"15min":1.4990000000000001},"min":{"1min":0.78200000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7529999999999999,"5min":2.0880000000000001,"15min":2.355},"last":1.7190000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.5309999999999999,"15min":1.542},"min":{"1min":0.95999999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9370000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.641},{"interface":"front","average":{"1min":1.448,"5min":1.518,"15min":1.5529999999999999},"min":{"1min":0.94599999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.4990000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:14:48.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:14:49.338 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-10-08T07:14:49.336+0000 7f6e640ce640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:14:49.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.342+0000 7f792aa60640 1 Processor -- start 2023-10-08T07:14:49.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.342+0000 7f792aa60640 1 -- start start 2023-10-08T07:14:49.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.343+0000 7f792aa60640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f79241599d0 0x7f79240ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.343+0000 7f792aa60640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240cc210 0x7f79240b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.343+0000 7f792aa60640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7924159db0 0x7f79240b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792aa60640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79240b10f0 con 0x7f7924159db0 2023-10-08T07:14:49.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792aa60640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79240cacf0 con 0x7f79241599d0 2023-10-08T07:14:49.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792aa60640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79240ca3c0 con 0x7f79240cc210 2023-10-08T07:14:49.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792a25f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7924159db0 0x7f79240b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792a25f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7924159db0 0x7f79240b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40252/0 (socket says 172.21.15.70:40252) 2023-10-08T07:14:49.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792a25f640 1 -- 172.21.15.70:0/570157241 learned_addr learned my addr 172.21.15.70:0/570157241 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792925d640 1 --2- 172.21.15.70:0/570157241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240cc210 0x7f79240b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f7929a5e640 1 --2- 172.21.15.70:0/570157241 >> v2:172.21.15.181:3300/0 conn(0x7f79241599d0 0x7f79240ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792a25f640 1 -- 172.21.15.70:0/570157241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240cc210 msgr2=0x7f79240b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792a25f640 1 --2- 172.21.15.70:0/570157241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240cc210 0x7f79240b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792a25f640 1 -- 172.21.15.70:0/570157241 >> v2:172.21.15.181:3300/0 conn(0x7f79241599d0 msgr2=0x7f79240ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792a25f640 1 --2- 172.21.15.70:0/570157241 >> v2:172.21.15.181:3300/0 conn(0x7f79241599d0 0x7f79240ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f792a25f640 1 -- 172.21.15.70:0/570157241 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79240ca6a0 con 0x7f7924159db0 2023-10-08T07:14:49.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.344+0000 7f7929a5e640 1 --2- 172.21.15.70:0/570157241 >> v2:172.21.15.181:3300/0 conn(0x7f79241599d0 0x7f79240ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:49.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.345+0000 7f792a25f640 1 --2- 172.21.15.70:0/570157241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7924159db0 0x7f79240b0600 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f7924071350 tx=0x7f791c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341d601618a9c11d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.346+0000 7f7928a5c640 1 -- 172.21.15.70:0/570157241 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c02ea80 con 0x7f7924159db0 2023-10-08T07:14:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.346+0000 7f7928a5c640 1 -- 172.21.15.70:0/570157241 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c02ebe0 con 0x7f7924159db0 2023-10-08T07:14:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.346+0000 7f7928a5c640 1 -- 172.21.15.70:0/570157241 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c009600 con 0x7f7924159db0 2023-10-08T07:14:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.346+0000 7f792aa60640 1 -- 172.21.15.70:0/570157241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7924159db0 msgr2=0x7f79240b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.346+0000 7f792aa60640 1 --2- 172.21.15.70:0/570157241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7924159db0 0x7f79240b0600 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f7924071350 tx=0x7f791c02e0e0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.346+0000 7f792aa60640 1 -- 172.21.15.70:0/570157241 shutdown_connections 2023-10-08T07:14:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.346+0000 7f792aa60640 1 --2- 172.21.15.70:0/570157241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7924159db0 0x7f79240b0600 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.346+0000 7f792aa60640 1 --2- 172.21.15.70:0/570157241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240cc210 0x7f79240b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.346+0000 7f792aa60640 1 --2- 172.21.15.70:0/570157241 >> v2:172.21.15.181:3300/0 conn(0x7f79241599d0 0x7f79240ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.346+0000 7f792aa60640 1 -- 172.21.15.70:0/570157241 >> 172.21.15.70:0/570157241 conn(0x7f79240cb0c0 msgr2=0x7f79240cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.347+0000 7f792aa60640 1 -- 172.21.15.70:0/570157241 shutdown_connections 2023-10-08T07:14:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.347+0000 7f792aa60640 1 -- 172.21.15.70:0/570157241 wait complete. 2023-10-08T07:14:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.348+0000 7f792aa60640 1 Processor -- start 2023-10-08T07:14:49.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.348+0000 7f792aa60640 1 -- start start 2023-10-08T07:14:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.349+0000 7f792aa60640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240cc210 0x7f79240c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.349+0000 7f792aa60640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241599d0 0x7f79240b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.349+0000 7f792aa60640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7924159db0 0x7f7924155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.349+0000 7f792aa60640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792415aba0 con 0x7f79241599d0 2023-10-08T07:14:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.349+0000 7f792aa60640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792415b440 con 0x7f7924159db0 2023-10-08T07:14:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.349+0000 7f792aa60640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79240b06d0 con 0x7f79240cc210 2023-10-08T07:14:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.350+0000 7f792925d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241599d0 0x7f79240b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.350+0000 7f792925d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241599d0 0x7f79240b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40254/0 (socket says 172.21.15.70:40254) 2023-10-08T07:14:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.350+0000 7f792925d640 1 -- 172.21.15.70:0/1385872451 learned_addr learned my addr 172.21.15.70:0/1385872451 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.350+0000 7f7929a5e640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240cc210 0x7f79240c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.350+0000 7f792a25f640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7924159db0 0x7f7924155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.350+0000 7f792925d640 1 -- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240cc210 msgr2=0x7f79240c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.350+0000 7f792925d640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240cc210 0x7f79240c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.350+0000 7f792925d640 1 -- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7924159db0 msgr2=0x7f7924155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.350+0000 7f792925d640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7924159db0 0x7f7924155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.350+0000 7f792925d640 1 -- 172.21.15.70:0/1385872451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c00bd40 con 0x7f79241599d0 2023-10-08T07:14:49.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.351+0000 7f792925d640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241599d0 0x7f79240b0190 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f791400c7f0 tx=0x7f791400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:49.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.351+0000 7f791a7fc640 1 -- 172.21.15.70:0/1385872451 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914004220 con 0x7f79241599d0 2023-10-08T07:14:49.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.351+0000 7f791a7fc640 1 -- 172.21.15.70:0/1385872451 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7914004380 con 0x7f79241599d0 2023-10-08T07:14:49.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.351+0000 7f791a7fc640 1 -- 172.21.15.70:0/1385872451 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914007860 con 0x7f79241599d0 2023-10-08T07:14:49.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.351+0000 7f792aa60640 1 -- 172.21.15.70:0/1385872451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79240c8c50 con 0x7f79241599d0 2023-10-08T07:14:49.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.351+0000 7f792aa60640 1 -- 172.21.15.70:0/1385872451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792415b0d0 con 0x7f79241599d0 2023-10-08T07:14:49.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.352+0000 7f792aa60640 1 -- 172.21.15.70:0/1385872451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792404e160 con 0x7f79241599d0 2023-10-08T07:14:49.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.353+0000 7f791a7fc640 1 -- 172.21.15.70:0/1385872451 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7914007bd0 con 0x7f79241599d0 2023-10-08T07:14:49.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.354+0000 7f791a7fc640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f790806b0b0 0x7f790806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.354+0000 7f791a7fc640 1 -- 172.21.15.70:0/1385872451 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7914089d70 con 0x7f79241599d0 2023-10-08T07:14:49.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.355+0000 7f7929a5e640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f790806b0b0 0x7f790806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.356+0000 7f7929a5e640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f790806b0b0 0x7f790806d570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f792000a990 tx=0x7f7920008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:49.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.359+0000 7f791a7fc640 1 -- 172.21.15.70:0/1385872451 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79140546c0 con 0x7f79241599d0 2023-10-08T07:14:49.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.542+0000 7f792aa60640 1 -- 172.21.15.70:0/1385872451 --> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79240ca070 con 0x7f790806b0b0 2023-10-08T07:14:49.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.565+0000 7f791a7fc640 1 -- 172.21.15.70:0/1385872451 <== mgr.8662 v2:172.21.15.181:6832/4245028292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287435 (secure 0 0 0) 0x7f792000a710 con 0x7f790806b0b0 2023-10-08T07:14:49.571 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:49.572 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:14:49.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.571+0000 7f792aa60640 1 -- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f790806b0b0 msgr2=0x7f790806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:49.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.571+0000 7f792aa60640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f790806b0b0 0x7f790806d570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f792000a990 tx=0x7f7920008040 comp rx=0 tx=0).stop 2023-10-08T07:14:49.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.571+0000 7f792aa60640 1 -- 172.21.15.70:0/1385872451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241599d0 msgr2=0x7f79240b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:49.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.571+0000 7f792aa60640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241599d0 0x7f79240b0190 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f791400c7f0 tx=0x7f791400ccb0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.571+0000 7f792aa60640 1 -- 172.21.15.70:0/1385872451 shutdown_connections 2023-10-08T07:14:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.571+0000 7f792aa60640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f790806b0b0 0x7f790806d570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.571+0000 7f792aa60640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7924159db0 0x7f7924155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.571+0000 7f792aa60640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241599d0 0x7f79240b0190 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.571+0000 7f792aa60640 1 --2- 172.21.15.70:0/1385872451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79240cc210 0x7f79240c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.571+0000 7f792aa60640 1 -- 172.21.15.70:0/1385872451 >> 172.21.15.70:0/1385872451 conn(0x7f79240cb0c0 msgr2=0x7f7924157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.572+0000 7f792aa60640 1 -- 172.21.15.70:0/1385872451 shutdown_connections 2023-10-08T07:14:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.572+0000 7f792aa60640 1 -- 172.21.15.70:0/1385872451 wait complete. 2023-10-08T07:14:49.592 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-10-08T07:14:48.867240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":673,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2866,"ondisk_log_size":2866,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422804,"kb_used_data":26216,"kb_used_omap":12,"kb_used_meta":396467,"kb_avail":754551916,"statfs":{"total":773094113280,"available":772661161984,"internally_reserved":0,"allocated":26845184,"data_stored":14533388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":405982779},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002022"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.305563+0000","last_change":"2023-10-08T07:14:44.322566+0000","last_active":"2023-10-08T07:14:45.305563+0000","last_peered":"2023-10-08T07:14:45.305563+0000","last_clean":"2023-10-08T07:14:45.305563+0000","last_became_active":"2023-10-08T07:14:44.321968+0000","last_became_peered":"2023-10-08T07:14:44.321968+0000","last_unstale":"2023-10-08T07:14:45.305563+0000","last_undegraded":"2023-10-08T07:14:45.305563+0000","last_fullsized":"2023-10-08T07:14:45.305563+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:33.364440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.195304+0000","last_change":"2023-10-08T07:14:44.336800+0000","last_active":"2023-10-08T07:14:46.195304+0000","last_peered":"2023-10-08T07:14:46.195304+0000","last_clean":"2023-10-08T07:14:46.195304+0000","last_became_active":"2023-10-08T07:14:44.336534+0000","last_became_peered":"2023-10-08T07:14:44.336534+0000","last_unstale":"2023-10-08T07:14:46.195304+0000","last_undegraded":"2023-10-08T07:14:46.195304+0000","last_fullsized":"2023-10-08T07:14:46.195304+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:48.002372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192765+0000","last_change":"2023-10-08T07:14:44.331190+0000","last_active":"2023-10-08T07:14:46.192765+0000","last_peered":"2023-10-08T07:14:46.192765+0000","last_clean":"2023-10-08T07:14:46.192765+0000","last_became_active":"2023-10-08T07:14:44.330346+0000","last_became_peered":"2023-10-08T07:14:44.330346+0000","last_unstale":"2023-10-08T07:14:46.192765+0000","last_undegraded":"2023-10-08T07:14:46.192765+0000","last_fullsized":"2023-10-08T07:14:46.192765+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:54:20.560371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193051+0000","last_change":"2023-10-08T07:14:44.337611+0000","last_active":"2023-10-08T07:14:46.193051+0000","last_peered":"2023-10-08T07:14:46.193051+0000","last_clean":"2023-10-08T07:14:46.193051+0000","last_became_active":"2023-10-08T07:14:44.336777+0000","last_became_peered":"2023-10-08T07:14:44.336777+0000","last_unstale":"2023-10-08T07:14:46.193051+0000","last_undegraded":"2023-10-08T07:14:46.193051+0000","last_fullsized":"2023-10-08T07:14:46.193051+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:55.334487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.309856+0000","last_change":"2023-10-08T07:14:44.334750+0000","last_active":"2023-10-08T07:14:45.309856+0000","last_peered":"2023-10-08T07:14:45.309856+0000","last_clean":"2023-10-08T07:14:45.309856+0000","last_became_active":"2023-10-08T07:14:44.332421+0000","last_became_peered":"2023-10-08T07:14:44.332421+0000","last_unstale":"2023-10-08T07:14:45.309856+0000","last_undegraded":"2023-10-08T07:14:45.309856+0000","last_fullsized":"2023-10-08T07:14:45.309856+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:57.635068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.305058+0000","last_change":"2023-10-08T07:14:44.331918+0000","last_active":"2023-10-08T07:14:45.305058+0000","last_peered":"2023-10-08T07:14:45.305058+0000","last_clean":"2023-10-08T07:14:45.305058+0000","last_became_active":"2023-10-08T07:14:44.331614+0000","last_became_peered":"2023-10-08T07:14:44.331614+0000","last_unstale":"2023-10-08T07:14:45.305058+0000","last_undegraded":"2023-10-08T07:14:45.305058+0000","last_fullsized":"2023-10-08T07:14:45.305058+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:24.698116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.307512+0000","last_change":"2023-10-08T07:14:44.335960+0000","last_active":"2023-10-08T07:14:45.307512+0000","last_peered":"2023-10-08T07:14:45.307512+0000","last_clean":"2023-10-08T07:14:45.307512+0000","last_became_active":"2023-10-08T07:14:44.335698+0000","last_became_peered":"2023-10-08T07:14:44.335698+0000","last_unstale":"2023-10-08T07:14:45.307512+0000","last_undegraded":"2023-10-08T07:14:45.307512+0000","last_fullsized":"2023-10-08T07:14:45.307512+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:21.961789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.308315+0000","last_change":"2023-10-08T07:14:44.324288+0000","last_active":"2023-10-08T07:14:45.308315+0000","last_peered":"2023-10-08T07:14:45.308315+0000","last_clean":"2023-10-08T07:14:45.308315+0000","last_became_active":"2023-10-08T07:14:44.323888+0000","last_became_peered":"2023-10-08T07:14:44.323888+0000","last_unstale":"2023-10-08T07:14:45.308315+0000","last_undegraded":"2023-10-08T07:14:45.308315+0000","last_fullsized":"2023-10-08T07:14:45.308315+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:01.742854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.305664+0000","last_change":"2023-10-08T07:14:44.332226+0000","last_active":"2023-10-08T07:14:45.305664+0000","last_peered":"2023-10-08T07:14:45.305664+0000","last_clean":"2023-10-08T07:14:45.305664+0000","last_became_active":"2023-10-08T07:14:44.331872+0000","last_became_peered":"2023-10-08T07:14:44.331872+0000","last_unstale":"2023-10-08T07:14:45.305664+0000","last_undegraded":"2023-10-08T07:14:45.305664+0000","last_fullsized":"2023-10-08T07:14:45.305664+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:25.959585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.194004+0000","last_change":"2023-10-08T07:14:44.331672+0000","last_active":"2023-10-08T07:14:46.194004+0000","last_peered":"2023-10-08T07:14:46.194004+0000","last_clean":"2023-10-08T07:14:46.194004+0000","last_became_active":"2023-10-08T07:14:44.331316+0000","last_became_peered":"2023-10-08T07:14:44.331316+0000","last_unstale":"2023-10-08T07:14:46.194004+0000","last_undegraded":"2023-10-08T07:14:46.194004+0000","last_fullsized":"2023-10-08T07:14:46.194004+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:57.946015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193120+0000","last_change":"2023-10-08T07:14:44.331371+0000","last_active":"2023-10-08T07:14:46.193120+0000","last_peered":"2023-10-08T07:14:46.193120+0000","last_clean":"2023-10-08T07:14:46.193120+0000","last_became_active":"2023-10-08T07:14:44.330911+0000","last_became_peered":"2023-10-08T07:14:44.330911+0000","last_unstale":"2023-10-08T07:14:46.193120+0000","last_undegraded":"2023-10-08T07:14:46.193120+0000","last_fullsized":"2023-10-08T07:14:46.193120+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:50.662785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.305320+0000","last_change":"2023-10-08T07:14:44.336483+0000","last_active":"2023-10-08T07:14:45.305320+0000","last_peered":"2023-10-08T07:14:45.305320+0000","last_clean":"2023-10-08T07:14:45.305320+0000","last_became_active":"2023-10-08T07:14:44.336207+0000","last_became_peered":"2023-10-08T07:14:44.336207+0000","last_unstale":"2023-10-08T07:14:45.305320+0000","last_undegraded":"2023-10-08T07:14:45.305320+0000","last_fullsized":"2023-10-08T07:14:45.305320+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:05.158536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193667+0000","last_change":"2023-10-08T07:14:44.334689+0000","last_active":"2023-10-08T07:14:46.193667+0000","last_peered":"2023-10-08T07:14:46.193667+0000","last_clean":"2023-10-08T07:14:46.193667+0000","last_became_active":"2023-10-08T07:14:44.334401+0000","last_became_peered":"2023-10-08T07:14:44.334401+0000","last_unstale":"2023-10-08T07:14:46.193667+0000","last_undegraded":"2023-10-08T07:14:46.193667+0000","last_fullsized":"2023-10-08T07:14:46.193667+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:59.473108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192215+0000","last_change":"2023-10-08T07:14:44.338751+0000","last_active":"2023-10-08T07:14:46.192215+0000","last_peered":"2023-10-08T07:14:46.192215+0000","last_clean":"2023-10-08T07:14:46.192215+0000","last_became_active":"2023-10-08T07:14:44.338461+0000","last_became_peered":"2023-10-08T07:14:44.338461+0000","last_unstale":"2023-10-08T07:14:46.192215+0000","last_undegraded":"2023-10-08T07:14:46.192215+0000","last_fullsized":"2023-10-08T07:14:46.192215+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:08.802530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192038+0000","last_change":"2023-10-08T07:14:44.327546+0000","last_active":"2023-10-08T07:14:46.192038+0000","last_peered":"2023-10-08T07:14:46.192038+0000","last_clean":"2023-10-08T07:14:46.192038+0000","last_became_active":"2023-10-08T07:14:44.327267+0000","last_became_peered":"2023-10-08T07:14:44.327267+0000","last_unstale":"2023-10-08T07:14:46.192038+0000","last_undegraded":"2023-10-08T07:14:46.192038+0000","last_fullsized":"2023-10-08T07:14:46.192038+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:15:38.112936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192484+0000","last_change":"2023-10-08T07:14:44.340846+0000","last_active":"2023-10-08T07:14:46.192484+0000","last_peered":"2023-10-08T07:14:46.192484+0000","last_clean":"2023-10-08T07:14:46.192484+0000","last_became_active":"2023-10-08T07:14:44.340588+0000","last_became_peered":"2023-10-08T07:14:44.340588+0000","last_unstale":"2023-10-08T07:14:46.192484+0000","last_undegraded":"2023-10-08T07:14:46.192484+0000","last_fullsized":"2023-10-08T07:14:46.192484+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:28:06.996015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.194049+0000","last_change":"2023-10-08T07:14:44.339646+0000","last_active":"2023-10-08T07:14:46.194049+0000","last_peered":"2023-10-08T07:14:46.194049+0000","last_clean":"2023-10-08T07:14:46.194049+0000","last_became_active":"2023-10-08T07:14:44.339229+0000","last_became_peered":"2023-10-08T07:14:44.339229+0000","last_unstale":"2023-10-08T07:14:46.194049+0000","last_undegraded":"2023-10-08T07:14:46.194049+0000","last_fullsized":"2023-10-08T07:14:46.194049+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:07.851859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193728+0000","last_change":"2023-10-08T07:14:44.323106+0000","last_active":"2023-10-08T07:14:46.193728+0000","last_peered":"2023-10-08T07:14:46.193728+0000","last_clean":"2023-10-08T07:14:46.193728+0000","last_became_active":"2023-10-08T07:14:44.322570+0000","last_became_peered":"2023-10-08T07:14:44.322570+0000","last_unstale":"2023-10-08T07:14:46.193728+0000","last_undegraded":"2023-10-08T07:14:46.193728+0000","last_fullsized":"2023-10-08T07:14:46.193728+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:07:44.570917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193143+0000","last_change":"2023-10-08T07:14:44.327597+0000","last_active":"2023-10-08T07:14:46.193143+0000","last_peered":"2023-10-08T07:14:46.193143+0000","last_clean":"2023-10-08T07:14:46.193143+0000","last_became_active":"2023-10-08T07:14:44.327247+0000","last_became_peered":"2023-10-08T07:14:44.327247+0000","last_unstale":"2023-10-08T07:14:46.193143+0000","last_undegraded":"2023-10-08T07:14:46.193143+0000","last_fullsized":"2023-10-08T07:14:46.193143+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:19.245988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.194668+0000","last_change":"2023-10-08T07:14:44.338168+0000","last_active":"2023-10-08T07:14:46.194668+0000","last_peered":"2023-10-08T07:14:46.194668+0000","last_clean":"2023-10-08T07:14:46.194668+0000","last_became_active":"2023-10-08T07:14:44.337905+0000","last_became_peered":"2023-10-08T07:14:44.337905+0000","last_unstale":"2023-10-08T07:14:46.194668+0000","last_undegraded":"2023-10-08T07:14:46.194668+0000","last_fullsized":"2023-10-08T07:14:46.194668+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:46.561296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.309009+0000","last_change":"2023-10-08T07:14:44.329361+0000","last_active":"2023-10-08T07:14:45.309009+0000","last_peered":"2023-10-08T07:14:45.309009+0000","last_clean":"2023-10-08T07:14:45.309009+0000","last_became_active":"2023-10-08T07:14:44.328994+0000","last_became_peered":"2023-10-08T07:14:44.328994+0000","last_unstale":"2023-10-08T07:14:45.309009+0000","last_undegraded":"2023-10-08T07:14:45.309009+0000","last_fullsized":"2023-10-08T07:14:45.309009+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:22.824813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193483+0000","last_change":"2023-10-08T07:14:44.334282+0000","last_active":"2023-10-08T07:14:46.193483+0000","last_peered":"2023-10-08T07:14:46.193483+0000","last_clean":"2023-10-08T07:14:46.193483+0000","last_became_active":"2023-10-08T07:14:44.332171+0000","last_became_peered":"2023-10-08T07:14:44.332171+0000","last_unstale":"2023-10-08T07:14:46.193483+0000","last_undegraded":"2023-10-08T07:14:46.193483+0000","last_fullsized":"2023-10-08T07:14:46.193483+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:53.430647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192334+0000","last_change":"2023-10-08T07:14:44.327517+0000","last_active":"2023-10-08T07:14:46.192334+0000","last_peered":"2023-10-08T07:14:46.192334+0000","last_clean":"2023-10-08T07:14:46.192334+0000","last_became_active":"2023-10-08T07:14:44.327215+0000","last_became_peered":"2023-10-08T07:14:44.327215+0000","last_unstale":"2023-10-08T07:14:46.192334+0000","last_undegraded":"2023-10-08T07:14:46.192334+0000","last_fullsized":"2023-10-08T07:14:46.192334+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:30.952438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192385+0000","last_change":"2023-10-08T07:14:44.328985+0000","last_active":"2023-10-08T07:14:46.192385+0000","last_peered":"2023-10-08T07:14:46.192385+0000","last_clean":"2023-10-08T07:14:46.192385+0000","last_became_active":"2023-10-08T07:14:44.328697+0000","last_became_peered":"2023-10-08T07:14:44.328697+0000","last_unstale":"2023-10-08T07:14:46.192385+0000","last_undegraded":"2023-10-08T07:14:46.192385+0000","last_fullsized":"2023-10-08T07:14:46.192385+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:48.694664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.195594+0000","last_change":"2023-10-08T07:14:44.336387+0000","last_active":"2023-10-08T07:14:46.195594+0000","last_peered":"2023-10-08T07:14:46.195594+0000","last_clean":"2023-10-08T07:14:46.195594+0000","last_became_active":"2023-10-08T07:14:44.336109+0000","last_became_peered":"2023-10-08T07:14:44.336109+0000","last_unstale":"2023-10-08T07:14:46.195594+0000","last_undegraded":"2023-10-08T07:14:46.195594+0000","last_fullsized":"2023-10-08T07:14:46.195594+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:39.506846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.308677+0000","last_change":"2023-10-08T07:14:44.338566+0000","last_active":"2023-10-08T07:14:45.308677+0000","last_peered":"2023-10-08T07:14:45.308677+0000","last_clean":"2023-10-08T07:14:45.308677+0000","last_became_active":"2023-10-08T07:14:44.338309+0000","last_became_peered":"2023-10-08T07:14:44.338309+0000","last_unstale":"2023-10-08T07:14:45.308677+0000","last_undegraded":"2023-10-08T07:14:45.308677+0000","last_fullsized":"2023-10-08T07:14:45.308677+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:51.724420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.310205+0000","last_change":"2023-10-08T07:14:44.335206+0000","last_active":"2023-10-08T07:14:45.310205+0000","last_peered":"2023-10-08T07:14:45.310205+0000","last_clean":"2023-10-08T07:14:45.310205+0000","last_became_active":"2023-10-08T07:14:44.332905+0000","last_became_peered":"2023-10-08T07:14:44.332905+0000","last_unstale":"2023-10-08T07:14:45.310205+0000","last_undegraded":"2023-10-08T07:14:45.310205+0000","last_fullsized":"2023-10-08T07:14:45.310205+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:57.888130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.305233+0000","last_change":"2023-10-08T07:14:44.332109+0000","last_active":"2023-10-08T07:14:45.305233+0000","last_peered":"2023-10-08T07:14:45.305233+0000","last_clean":"2023-10-08T07:14:45.305233+0000","last_became_active":"2023-10-08T07:14:44.331835+0000","last_became_peered":"2023-10-08T07:14:44.331835+0000","last_unstale":"2023-10-08T07:14:45.305233+0000","last_undegraded":"2023-10-08T07:14:45.305233+0000","last_fullsized":"2023-10-08T07:14:45.305233+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:15.146404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193267+0000","last_change":"2023-10-08T07:14:44.339948+0000","last_active":"2023-10-08T07:14:46.193267+0000","last_peered":"2023-10-08T07:14:46.193267+0000","last_clean":"2023-10-08T07:14:46.193267+0000","last_became_active":"2023-10-08T07:14:44.339645+0000","last_became_peered":"2023-10-08T07:14:44.339645+0000","last_unstale":"2023-10-08T07:14:46.193267+0000","last_undegraded":"2023-10-08T07:14:46.193267+0000","last_fullsized":"2023-10-08T07:14:46.193267+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:42.921595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192604+0000","last_change":"2023-10-08T07:14:44.321797+0000","last_active":"2023-10-08T07:14:46.192604+0000","last_peered":"2023-10-08T07:14:46.192604+0000","last_clean":"2023-10-08T07:14:46.192604+0000","last_became_active":"2023-10-08T07:14:44.321208+0000","last_became_peered":"2023-10-08T07:14:44.321208+0000","last_unstale":"2023-10-08T07:14:46.192604+0000","last_undegraded":"2023-10-08T07:14:46.192604+0000","last_fullsized":"2023-10-08T07:14:46.192604+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:01.676820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.194888+0000","last_change":"2023-10-08T07:14:44.329230+0000","last_active":"2023-10-08T07:14:46.194888+0000","last_peered":"2023-10-08T07:14:46.194888+0000","last_clean":"2023-10-08T07:14:46.194888+0000","last_became_active":"2023-10-08T07:14:44.328972+0000","last_became_peered":"2023-10-08T07:14:44.328972+0000","last_unstale":"2023-10-08T07:14:46.194888+0000","last_undegraded":"2023-10-08T07:14:46.194888+0000","last_fullsized":"2023-10-08T07:14:46.194888+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:59.867061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.304721+0000","last_change":"2023-10-08T07:14:44.337970+0000","last_active":"2023-10-08T07:14:45.304721+0000","last_peered":"2023-10-08T07:14:45.304721+0000","last_clean":"2023-10-08T07:14:45.304721+0000","last_became_active":"2023-10-08T07:14:44.337633+0000","last_became_peered":"2023-10-08T07:14:44.337633+0000","last_unstale":"2023-10-08T07:14:45.304721+0000","last_undegraded":"2023-10-08T07:14:45.304721+0000","last_fullsized":"2023-10-08T07:14:45.304721+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:55.063443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193886+0000","last_change":"2023-10-08T07:14:44.332579+0000","last_active":"2023-10-08T07:14:46.193886+0000","last_peered":"2023-10-08T07:14:46.193886+0000","last_clean":"2023-10-08T07:14:46.193886+0000","last_became_active":"2023-10-08T07:14:44.329641+0000","last_became_peered":"2023-10-08T07:14:44.329641+0000","last_unstale":"2023-10-08T07:14:46.193886+0000","last_undegraded":"2023-10-08T07:14:46.193886+0000","last_fullsized":"2023-10-08T07:14:46.193886+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:23.709005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"253'4","reported_seq":16,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.395659+0000","last_change":"2023-10-08T07:14:43.402506+0000","last_active":"2023-10-08T07:14:44.395659+0000","last_peered":"2023-10-08T07:14:44.395659+0000","last_clean":"2023-10-08T07:14:44.395659+0000","last_became_active":"2023-10-08T07:14:43.402246+0000","last_became_peered":"2023-10-08T07:14:43.402246+0000","last_unstale":"2023-10-08T07:14:44.395659+0000","last_undegraded":"2023-10-08T07:14:44.395659+0000","last_fullsized":"2023-10-08T07:14:44.395659+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:03.082313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192001+0000","last_change":"2023-10-08T07:14:44.337109+0000","last_active":"2023-10-08T07:14:46.192001+0000","last_peered":"2023-10-08T07:14:46.192001+0000","last_clean":"2023-10-08T07:14:46.192001+0000","last_became_active":"2023-10-08T07:14:44.336825+0000","last_became_peered":"2023-10-08T07:14:44.336825+0000","last_unstale":"2023-10-08T07:14:46.192001+0000","last_undegraded":"2023-10-08T07:14:46.192001+0000","last_fullsized":"2023-10-08T07:14:46.192001+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:04:02.054643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"253'2","reported_seq":14,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.396232+0000","last_change":"2023-10-08T07:14:43.408289+0000","last_active":"2023-10-08T07:14:44.396232+0000","last_peered":"2023-10-08T07:14:44.396232+0000","last_clean":"2023-10-08T07:14:44.396232+0000","last_became_active":"2023-10-08T07:14:43.407994+0000","last_became_peered":"2023-10-08T07:14:43.407994+0000","last_unstale":"2023-10-08T07:14:44.396232+0000","last_undegraded":"2023-10-08T07:14:44.396232+0000","last_fullsized":"2023-10-08T07:14:44.396232+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:44.448458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.305113+0000","last_change":"2023-10-08T07:14:44.332474+0000","last_active":"2023-10-08T07:14:45.305113+0000","last_peered":"2023-10-08T07:14:45.305113+0000","last_clean":"2023-10-08T07:14:45.305113+0000","last_became_active":"2023-10-08T07:14:44.332200+0000","last_became_peered":"2023-10-08T07:14:44.332200+0000","last_unstale":"2023-10-08T07:14:45.305113+0000","last_undegraded":"2023-10-08T07:14:45.305113+0000","last_fullsized":"2023-10-08T07:14:45.305113+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:19.691981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"253'2","reported_seq":14,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.396348+0000","last_change":"2023-10-08T07:14:43.402276+0000","last_active":"2023-10-08T07:14:44.396348+0000","last_peered":"2023-10-08T07:14:44.396348+0000","last_clean":"2023-10-08T07:14:44.396348+0000","last_became_active":"2023-10-08T07:14:43.401792+0000","last_became_peered":"2023-10-08T07:14:43.401792+0000","last_unstale":"2023-10-08T07:14:44.396348+0000","last_undegraded":"2023-10-08T07:14:44.396348+0000","last_fullsized":"2023-10-08T07:14:44.396348+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:38.609639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193605+0000","last_change":"2023-10-08T07:14:44.333701+0000","last_active":"2023-10-08T07:14:46.193605+0000","last_peered":"2023-10-08T07:14:46.193605+0000","last_clean":"2023-10-08T07:14:46.193605+0000","last_became_active":"2023-10-08T07:14:44.333403+0000","last_became_peered":"2023-10-08T07:14:44.333403+0000","last_unstale":"2023-10-08T07:14:46.193605+0000","last_undegraded":"2023-10-08T07:14:46.193605+0000","last_fullsized":"2023-10-08T07:14:46.193605+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:24.067787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"253'1","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.399378+0000","last_change":"2023-10-08T07:14:43.416523+0000","last_active":"2023-10-08T07:14:44.399378+0000","last_peered":"2023-10-08T07:14:44.399378+0000","last_clean":"2023-10-08T07:14:44.399378+0000","last_became_active":"2023-10-08T07:14:43.415998+0000","last_became_peered":"2023-10-08T07:14:43.415998+0000","last_unstale":"2023-10-08T07:14:44.399378+0000","last_undegraded":"2023-10-08T07:14:44.399378+0000","last_fullsized":"2023-10-08T07:14:44.399378+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:02.841738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.309525+0000","last_change":"2023-10-08T07:14:44.332587+0000","last_active":"2023-10-08T07:14:45.309525+0000","last_peered":"2023-10-08T07:14:45.309525+0000","last_clean":"2023-10-08T07:14:45.309525+0000","last_became_active":"2023-10-08T07:14:44.328870+0000","last_became_peered":"2023-10-08T07:14:44.328870+0000","last_unstale":"2023-10-08T07:14:45.309525+0000","last_undegraded":"2023-10-08T07:14:45.309525+0000","last_fullsized":"2023-10-08T07:14:45.309525+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:30.679998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"253'1","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.398494+0000","last_change":"2023-10-08T07:14:43.415069+0000","last_active":"2023-10-08T07:14:44.398494+0000","last_peered":"2023-10-08T07:14:44.398494+0000","last_clean":"2023-10-08T07:14:44.398494+0000","last_became_active":"2023-10-08T07:14:43.414780+0000","last_became_peered":"2023-10-08T07:14:43.414780+0000","last_unstale":"2023-10-08T07:14:44.398494+0000","last_undegraded":"2023-10-08T07:14:44.398494+0000","last_fullsized":"2023-10-08T07:14:44.398494+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:09:43.543624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193257+0000","last_change":"2023-10-08T07:14:44.338239+0000","last_active":"2023-10-08T07:14:46.193257+0000","last_peered":"2023-10-08T07:14:46.193257+0000","last_clean":"2023-10-08T07:14:46.193257+0000","last_became_active":"2023-10-08T07:14:44.337931+0000","last_became_peered":"2023-10-08T07:14:44.337931+0000","last_unstale":"2023-10-08T07:14:46.193257+0000","last_undegraded":"2023-10-08T07:14:46.193257+0000","last_fullsized":"2023-10-08T07:14:46.193257+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:36:59.713472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.311307+0000","last_change":"2023-10-08T07:14:43.410267+0000","last_active":"2023-10-08T07:14:44.311307+0000","last_peered":"2023-10-08T07:14:44.311307+0000","last_clean":"2023-10-08T07:14:44.311307+0000","last_became_active":"2023-10-08T07:14:43.410012+0000","last_became_peered":"2023-10-08T07:14:43.410012+0000","last_unstale":"2023-10-08T07:14:44.311307+0000","last_undegraded":"2023-10-08T07:14:44.311307+0000","last_fullsized":"2023-10-08T07:14:44.311307+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:41.903083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.310644+0000","last_change":"2023-10-08T07:14:44.330641+0000","last_active":"2023-10-08T07:14:45.310644+0000","last_peered":"2023-10-08T07:14:45.310644+0000","last_clean":"2023-10-08T07:14:45.310644+0000","last_became_active":"2023-10-08T07:14:44.330271+0000","last_became_peered":"2023-10-08T07:14:44.330271+0000","last_unstale":"2023-10-08T07:14:45.310644+0000","last_undegraded":"2023-10-08T07:14:45.310644+0000","last_fullsized":"2023-10-08T07:14:45.310644+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:35.355050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.316150+0000","last_change":"2023-10-08T07:14:43.407886+0000","last_active":"2023-10-08T07:14:44.316150+0000","last_peered":"2023-10-08T07:14:44.316150+0000","last_clean":"2023-10-08T07:14:44.316150+0000","last_became_active":"2023-10-08T07:14:43.407634+0000","last_became_peered":"2023-10-08T07:14:43.407634+0000","last_unstale":"2023-10-08T07:14:44.316150+0000","last_undegraded":"2023-10-08T07:14:44.316150+0000","last_fullsized":"2023-10-08T07:14:44.316150+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:55:43.492128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.307605+0000","last_change":"2023-10-08T07:14:44.332428+0000","last_active":"2023-10-08T07:14:45.307605+0000","last_peered":"2023-10-08T07:14:45.307605+0000","last_clean":"2023-10-08T07:14:45.307605+0000","last_became_active":"2023-10-08T07:14:44.331865+0000","last_became_peered":"2023-10-08T07:14:44.331865+0000","last_unstale":"2023-10-08T07:14:45.307605+0000","last_undegraded":"2023-10-08T07:14:45.307605+0000","last_fullsized":"2023-10-08T07:14:45.307605+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:04.509562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.308203+0000","last_change":"2023-10-08T07:14:43.397332+0000","last_active":"2023-10-08T07:14:44.308203+0000","last_peered":"2023-10-08T07:14:44.308203+0000","last_clean":"2023-10-08T07:14:44.308203+0000","last_became_active":"2023-10-08T07:14:43.397049+0000","last_became_peered":"2023-10-08T07:14:43.397049+0000","last_unstale":"2023-10-08T07:14:44.308203+0000","last_undegraded":"2023-10-08T07:14:44.308203+0000","last_fullsized":"2023-10-08T07:14:44.308203+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:23.709280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193046+0000","last_change":"2023-10-08T07:14:44.331844+0000","last_active":"2023-10-08T07:14:46.193046+0000","last_peered":"2023-10-08T07:14:46.193046+0000","last_clean":"2023-10-08T07:14:46.193046+0000","last_became_active":"2023-10-08T07:14:44.331521+0000","last_became_peered":"2023-10-08T07:14:44.331521+0000","last_unstale":"2023-10-08T07:14:46.193046+0000","last_undegraded":"2023-10-08T07:14:46.193046+0000","last_fullsized":"2023-10-08T07:14:46.193046+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:22.760377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.313571+0000","last_change":"2023-10-08T07:14:43.414266+0000","last_active":"2023-10-08T07:14:44.313571+0000","last_peered":"2023-10-08T07:14:44.313571+0000","last_clean":"2023-10-08T07:14:44.313571+0000","last_became_active":"2023-10-08T07:14:43.413877+0000","last_became_peered":"2023-10-08T07:14:43.413877+0000","last_unstale":"2023-10-08T07:14:44.313571+0000","last_undegraded":"2023-10-08T07:14:44.313571+0000","last_fullsized":"2023-10-08T07:14:44.313571+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:00.790509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192846+0000","last_change":"2023-10-08T07:14:44.330684+0000","last_active":"2023-10-08T07:14:46.192846+0000","last_peered":"2023-10-08T07:14:46.192846+0000","last_clean":"2023-10-08T07:14:46.192846+0000","last_became_active":"2023-10-08T07:14:44.330353+0000","last_became_peered":"2023-10-08T07:14:44.330353+0000","last_unstale":"2023-10-08T07:14:46.192846+0000","last_undegraded":"2023-10-08T07:14:46.192846+0000","last_fullsized":"2023-10-08T07:14:46.192846+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:14:27.946683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"253'3","reported_seq":15,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.392706+0000","last_change":"2023-10-08T07:14:43.410104+0000","last_active":"2023-10-08T07:14:44.392706+0000","last_peered":"2023-10-08T07:14:44.392706+0000","last_clean":"2023-10-08T07:14:44.392706+0000","last_became_active":"2023-10-08T07:14:43.409558+0000","last_became_peered":"2023-10-08T07:14:43.409558+0000","last_unstale":"2023-10-08T07:14:44.392706+0000","last_undegraded":"2023-10-08T07:14:44.392706+0000","last_fullsized":"2023-10-08T07:14:44.392706+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:47.180803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.308901+0000","last_change":"2023-10-08T07:14:44.326288+0000","last_active":"2023-10-08T07:14:45.308901+0000","last_peered":"2023-10-08T07:14:45.308901+0000","last_clean":"2023-10-08T07:14:45.308901+0000","last_became_active":"2023-10-08T07:14:44.325977+0000","last_became_peered":"2023-10-08T07:14:44.325977+0000","last_unstale":"2023-10-08T07:14:45.308901+0000","last_undegraded":"2023-10-08T07:14:45.308901+0000","last_fullsized":"2023-10-08T07:14:45.308901+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:31.119993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"253'6","reported_seq":18,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.401286+0000","last_change":"2023-10-08T07:14:43.399270+0000","last_active":"2023-10-08T07:14:44.401286+0000","last_peered":"2023-10-08T07:14:44.401286+0000","last_clean":"2023-10-08T07:14:44.401286+0000","last_became_active":"2023-10-08T07:14:43.398978+0000","last_became_peered":"2023-10-08T07:14:43.398978+0000","last_unstale":"2023-10-08T07:14:44.401286+0000","last_undegraded":"2023-10-08T07:14:44.401286+0000","last_fullsized":"2023-10-08T07:14:44.401286+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:49.950109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.194273+0000","last_change":"2023-10-08T07:14:44.335266+0000","last_active":"2023-10-08T07:14:46.194273+0000","last_peered":"2023-10-08T07:14:46.194273+0000","last_clean":"2023-10-08T07:14:46.194273+0000","last_became_active":"2023-10-08T07:14:44.334971+0000","last_became_peered":"2023-10-08T07:14:44.334971+0000","last_unstale":"2023-10-08T07:14:46.194273+0000","last_undegraded":"2023-10-08T07:14:46.194273+0000","last_fullsized":"2023-10-08T07:14:46.194273+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:14.581717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"253'2","reported_seq":14,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.399780+0000","last_change":"2023-10-08T07:14:43.410713+0000","last_active":"2023-10-08T07:14:44.399780+0000","last_peered":"2023-10-08T07:14:44.399780+0000","last_clean":"2023-10-08T07:14:44.399780+0000","last_became_active":"2023-10-08T07:14:43.410448+0000","last_became_peered":"2023-10-08T07:14:43.410448+0000","last_unstale":"2023-10-08T07:14:44.399780+0000","last_undegraded":"2023-10-08T07:14:44.399780+0000","last_fullsized":"2023-10-08T07:14:44.399780+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:03:40.822722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.309454+0000","last_change":"2023-10-08T07:14:44.327892+0000","last_active":"2023-10-08T07:14:45.309454+0000","last_peered":"2023-10-08T07:14:45.309454+0000","last_clean":"2023-10-08T07:14:45.309454+0000","last_became_active":"2023-10-08T07:14:44.327480+0000","last_became_peered":"2023-10-08T07:14:44.327480+0000","last_unstale":"2023-10-08T07:14:45.309454+0000","last_undegraded":"2023-10-08T07:14:45.309454+0000","last_fullsized":"2023-10-08T07:14:45.309454+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:26.279630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.310040+0000","last_change":"2023-10-08T07:14:43.404280+0000","last_active":"2023-10-08T07:14:44.310040+0000","last_peered":"2023-10-08T07:14:44.310040+0000","last_clean":"2023-10-08T07:14:44.310040+0000","last_became_active":"2023-10-08T07:14:43.403911+0000","last_became_peered":"2023-10-08T07:14:43.403911+0000","last_unstale":"2023-10-08T07:14:44.310040+0000","last_undegraded":"2023-10-08T07:14:44.310040+0000","last_fullsized":"2023-10-08T07:14:44.310040+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:06.833347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193136+0000","last_change":"2023-10-08T07:14:44.336892+0000","last_active":"2023-10-08T07:14:46.193136+0000","last_peered":"2023-10-08T07:14:46.193136+0000","last_clean":"2023-10-08T07:14:46.193136+0000","last_became_active":"2023-10-08T07:14:44.336589+0000","last_became_peered":"2023-10-08T07:14:44.336589+0000","last_unstale":"2023-10-08T07:14:46.193136+0000","last_undegraded":"2023-10-08T07:14:46.193136+0000","last_fullsized":"2023-10-08T07:14:46.193136+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:07.990108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.310037+0000","last_change":"2023-10-08T07:14:43.411394+0000","last_active":"2023-10-08T07:14:44.310037+0000","last_peered":"2023-10-08T07:14:44.310037+0000","last_clean":"2023-10-08T07:14:44.310037+0000","last_became_active":"2023-10-08T07:14:43.411091+0000","last_became_peered":"2023-10-08T07:14:43.411091+0000","last_unstale":"2023-10-08T07:14:44.310037+0000","last_undegraded":"2023-10-08T07:14:44.310037+0000","last_fullsized":"2023-10-08T07:14:44.310037+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:44:12.677401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.310468+0000","last_change":"2023-10-08T07:14:44.340304+0000","last_active":"2023-10-08T07:14:45.310468+0000","last_peered":"2023-10-08T07:14:45.310468+0000","last_clean":"2023-10-08T07:14:45.310468+0000","last_became_active":"2023-10-08T07:14:44.338301+0000","last_became_peered":"2023-10-08T07:14:44.338301+0000","last_unstale":"2023-10-08T07:14:45.310468+0000","last_undegraded":"2023-10-08T07:14:45.310468+0000","last_fullsized":"2023-10-08T07:14:45.310468+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:42:14.658077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"253'8","reported_seq":20,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.399919+0000","last_change":"2023-10-08T07:14:43.407648+0000","last_active":"2023-10-08T07:14:44.399919+0000","last_peered":"2023-10-08T07:14:44.399919+0000","last_clean":"2023-10-08T07:14:44.399919+0000","last_became_active":"2023-10-08T07:14:43.407376+0000","last_became_peered":"2023-10-08T07:14:43.407376+0000","last_unstale":"2023-10-08T07:14:44.399919+0000","last_undegraded":"2023-10-08T07:14:44.399919+0000","last_fullsized":"2023-10-08T07:14:44.399919+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:26:50.563868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.308212+0000","last_change":"2023-10-08T07:14:44.335197+0000","last_active":"2023-10-08T07:14:45.308212+0000","last_peered":"2023-10-08T07:14:45.308212+0000","last_clean":"2023-10-08T07:14:45.308212+0000","last_became_active":"2023-10-08T07:14:44.333014+0000","last_became_peered":"2023-10-08T07:14:44.333014+0000","last_unstale":"2023-10-08T07:14:45.308212+0000","last_undegraded":"2023-10-08T07:14:45.308212+0000","last_fullsized":"2023-10-08T07:14:45.308212+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:36:04.704806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"253'1","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.394120+0000","last_change":"2023-10-08T07:14:43.411294+0000","last_active":"2023-10-08T07:14:44.394120+0000","last_peered":"2023-10-08T07:14:44.394120+0000","last_clean":"2023-10-08T07:14:44.394120+0000","last_became_active":"2023-10-08T07:14:43.411000+0000","last_became_peered":"2023-10-08T07:14:43.411000+0000","last_unstale":"2023-10-08T07:14:44.394120+0000","last_undegraded":"2023-10-08T07:14:44.394120+0000","last_fullsized":"2023-10-08T07:14:44.394120+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:42:35.351878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.308204+0000","last_change":"2023-10-08T07:14:44.331894+0000","last_active":"2023-10-08T07:14:45.308204+0000","last_peered":"2023-10-08T07:14:45.308204+0000","last_clean":"2023-10-08T07:14:45.308204+0000","last_became_active":"2023-10-08T07:14:44.331612+0000","last_became_peered":"2023-10-08T07:14:44.331612+0000","last_unstale":"2023-10-08T07:14:45.308204+0000","last_undegraded":"2023-10-08T07:14:45.308204+0000","last_fullsized":"2023-10-08T07:14:45.308204+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:17.150803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"253'2","reported_seq":14,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.396670+0000","last_change":"2023-10-08T07:14:43.410517+0000","last_active":"2023-10-08T07:14:44.396670+0000","last_peered":"2023-10-08T07:14:44.396670+0000","last_clean":"2023-10-08T07:14:44.396670+0000","last_became_active":"2023-10-08T07:14:43.409771+0000","last_became_peered":"2023-10-08T07:14:43.409771+0000","last_unstale":"2023-10-08T07:14:44.396670+0000","last_undegraded":"2023-10-08T07:14:44.396670+0000","last_fullsized":"2023-10-08T07:14:44.396670+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:42.844452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193854+0000","last_change":"2023-10-08T07:14:44.342867+0000","last_active":"2023-10-08T07:14:46.193854+0000","last_peered":"2023-10-08T07:14:46.193854+0000","last_clean":"2023-10-08T07:14:46.193854+0000","last_became_active":"2023-10-08T07:14:44.342541+0000","last_became_peered":"2023-10-08T07:14:44.342541+0000","last_unstale":"2023-10-08T07:14:46.193854+0000","last_undegraded":"2023-10-08T07:14:46.193854+0000","last_fullsized":"2023-10-08T07:14:46.193854+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:43.616734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"253'1","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.396248+0000","last_change":"2023-10-08T07:14:43.413188+0000","last_active":"2023-10-08T07:14:44.396248+0000","last_peered":"2023-10-08T07:14:44.396248+0000","last_clean":"2023-10-08T07:14:44.396248+0000","last_became_active":"2023-10-08T07:14:43.412716+0000","last_became_peered":"2023-10-08T07:14:43.412716+0000","last_unstale":"2023-10-08T07:14:44.396248+0000","last_undegraded":"2023-10-08T07:14:44.396248+0000","last_fullsized":"2023-10-08T07:14:44.396248+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:08.671468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.194150+0000","last_change":"2023-10-08T07:14:44.332886+0000","last_active":"2023-10-08T07:14:46.194150+0000","last_peered":"2023-10-08T07:14:46.194150+0000","last_clean":"2023-10-08T07:14:46.194150+0000","last_became_active":"2023-10-08T07:14:44.332142+0000","last_became_peered":"2023-10-08T07:14:44.332142+0000","last_unstale":"2023-10-08T07:14:46.194150+0000","last_undegraded":"2023-10-08T07:14:46.194150+0000","last_fullsized":"2023-10-08T07:14:46.194150+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:49.297056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.316270+0000","last_change":"2023-10-08T07:14:43.415034+0000","last_active":"2023-10-08T07:14:44.316270+0000","last_peered":"2023-10-08T07:14:44.316270+0000","last_clean":"2023-10-08T07:14:44.316270+0000","last_became_active":"2023-10-08T07:14:43.414728+0000","last_became_peered":"2023-10-08T07:14:43.414728+0000","last_unstale":"2023-10-08T07:14:44.316270+0000","last_undegraded":"2023-10-08T07:14:44.316270+0000","last_fullsized":"2023-10-08T07:14:44.316270+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:08.048265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192643+0000","last_change":"2023-10-08T07:14:44.340208+0000","last_active":"2023-10-08T07:14:46.192643+0000","last_peered":"2023-10-08T07:14:46.192643+0000","last_clean":"2023-10-08T07:14:46.192643+0000","last_became_active":"2023-10-08T07:14:44.339934+0000","last_became_peered":"2023-10-08T07:14:44.339934+0000","last_unstale":"2023-10-08T07:14:46.192643+0000","last_undegraded":"2023-10-08T07:14:46.192643+0000","last_fullsized":"2023-10-08T07:14:46.192643+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:26.451027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"253'2","reported_seq":14,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.395731+0000","last_change":"2023-10-08T07:14:43.413078+0000","last_active":"2023-10-08T07:14:44.395731+0000","last_peered":"2023-10-08T07:14:44.395731+0000","last_clean":"2023-10-08T07:14:44.395731+0000","last_became_active":"2023-10-08T07:14:43.412804+0000","last_became_peered":"2023-10-08T07:14:43.412804+0000","last_unstale":"2023-10-08T07:14:44.395731+0000","last_undegraded":"2023-10-08T07:14:44.395731+0000","last_fullsized":"2023-10-08T07:14:44.395731+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:12.105709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.195145+0000","last_change":"2023-10-08T07:14:44.333114+0000","last_active":"2023-10-08T07:14:46.195145+0000","last_peered":"2023-10-08T07:14:46.195145+0000","last_clean":"2023-10-08T07:14:46.195145+0000","last_became_active":"2023-10-08T07:14:44.332105+0000","last_became_peered":"2023-10-08T07:14:44.332105+0000","last_unstale":"2023-10-08T07:14:46.195145+0000","last_undegraded":"2023-10-08T07:14:46.195145+0000","last_fullsized":"2023-10-08T07:14:46.195145+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:58:23.020597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.328274+0000","last_change":"2023-10-08T07:14:43.407284+0000","last_active":"2023-10-08T07:14:44.328274+0000","last_peered":"2023-10-08T07:14:44.328274+0000","last_clean":"2023-10-08T07:14:44.328274+0000","last_became_active":"2023-10-08T07:14:43.407010+0000","last_became_peered":"2023-10-08T07:14:43.407010+0000","last_unstale":"2023-10-08T07:14:44.328274+0000","last_undegraded":"2023-10-08T07:14:44.328274+0000","last_fullsized":"2023-10-08T07:14:44.328274+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:18.544340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193970+0000","last_change":"2023-10-08T07:14:44.339779+0000","last_active":"2023-10-08T07:14:46.193970+0000","last_peered":"2023-10-08T07:14:46.193970+0000","last_clean":"2023-10-08T07:14:46.193970+0000","last_became_active":"2023-10-08T07:14:44.339385+0000","last_became_peered":"2023-10-08T07:14:44.339385+0000","last_unstale":"2023-10-08T07:14:46.193970+0000","last_undegraded":"2023-10-08T07:14:46.193970+0000","last_fullsized":"2023-10-08T07:14:46.193970+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:44:13.695913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.319999+0000","last_change":"2023-10-08T07:14:43.408990+0000","last_active":"2023-10-08T07:14:44.319999+0000","last_peered":"2023-10-08T07:14:44.319999+0000","last_clean":"2023-10-08T07:14:44.319999+0000","last_became_active":"2023-10-08T07:14:43.408702+0000","last_became_peered":"2023-10-08T07:14:43.408702+0000","last_unstale":"2023-10-08T07:14:44.319999+0000","last_undegraded":"2023-10-08T07:14:44.319999+0000","last_fullsized":"2023-10-08T07:14:44.319999+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:54.133156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193928+0000","last_change":"2023-10-08T07:14:44.333909+0000","last_active":"2023-10-08T07:14:46.193928+0000","last_peered":"2023-10-08T07:14:46.193928+0000","last_clean":"2023-10-08T07:14:46.193928+0000","last_became_active":"2023-10-08T07:14:44.333646+0000","last_became_peered":"2023-10-08T07:14:44.333646+0000","last_unstale":"2023-10-08T07:14:46.193928+0000","last_undegraded":"2023-10-08T07:14:46.193928+0000","last_fullsized":"2023-10-08T07:14:46.193928+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:50.362306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.303792+0000","last_change":"2023-10-08T07:14:43.406333+0000","last_active":"2023-10-08T07:14:44.303792+0000","last_peered":"2023-10-08T07:14:44.303792+0000","last_clean":"2023-10-08T07:14:44.303792+0000","last_became_active":"2023-10-08T07:14:43.406052+0000","last_became_peered":"2023-10-08T07:14:43.406052+0000","last_unstale":"2023-10-08T07:14:44.303792+0000","last_undegraded":"2023-10-08T07:14:44.303792+0000","last_fullsized":"2023-10-08T07:14:44.303792+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:31.926662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.307510+0000","last_change":"2023-10-08T07:14:44.338131+0000","last_active":"2023-10-08T07:14:45.307510+0000","last_peered":"2023-10-08T07:14:45.307510+0000","last_clean":"2023-10-08T07:14:45.307510+0000","last_became_active":"2023-10-08T07:14:44.337734+0000","last_became_peered":"2023-10-08T07:14:44.337734+0000","last_unstale":"2023-10-08T07:14:45.307510+0000","last_undegraded":"2023-10-08T07:14:45.307510+0000","last_fullsized":"2023-10-08T07:14:45.307510+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:29:05.679033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"253'2","reported_seq":14,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.398775+0000","last_change":"2023-10-08T07:14:43.414871+0000","last_active":"2023-10-08T07:14:44.398775+0000","last_peered":"2023-10-08T07:14:44.398775+0000","last_clean":"2023-10-08T07:14:44.398775+0000","last_became_active":"2023-10-08T07:14:43.414608+0000","last_became_peered":"2023-10-08T07:14:43.414608+0000","last_unstale":"2023-10-08T07:14:44.398775+0000","last_undegraded":"2023-10-08T07:14:44.398775+0000","last_fullsized":"2023-10-08T07:14:44.398775+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:35.056822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.191925+0000","last_change":"2023-10-08T07:14:44.330281+0000","last_active":"2023-10-08T07:14:46.191925+0000","last_peered":"2023-10-08T07:14:46.191925+0000","last_clean":"2023-10-08T07:14:46.191925+0000","last_became_active":"2023-10-08T07:14:44.329938+0000","last_became_peered":"2023-10-08T07:14:44.329938+0000","last_unstale":"2023-10-08T07:14:46.191925+0000","last_undegraded":"2023-10-08T07:14:46.191925+0000","last_fullsized":"2023-10-08T07:14:46.191925+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:16.367973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.310832+0000","last_change":"2023-10-08T07:14:43.414678+0000","last_active":"2023-10-08T07:14:44.310832+0000","last_peered":"2023-10-08T07:14:44.310832+0000","last_clean":"2023-10-08T07:14:44.310832+0000","last_became_active":"2023-10-08T07:14:43.414400+0000","last_became_peered":"2023-10-08T07:14:43.414400+0000","last_unstale":"2023-10-08T07:14:44.310832+0000","last_undegraded":"2023-10-08T07:14:44.310832+0000","last_fullsized":"2023-10-08T07:14:44.310832+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:43:12.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.192426+0000","last_change":"2023-10-08T07:14:44.337389+0000","last_active":"2023-10-08T07:14:46.192426+0000","last_peered":"2023-10-08T07:14:46.192426+0000","last_clean":"2023-10-08T07:14:46.192426+0000","last_became_active":"2023-10-08T07:14:44.337118+0000","last_became_peered":"2023-10-08T07:14:44.337118+0000","last_unstale":"2023-10-08T07:14:46.192426+0000","last_undegraded":"2023-10-08T07:14:46.192426+0000","last_fullsized":"2023-10-08T07:14:46.192426+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:11.369315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.304186+0000","last_change":"2023-10-08T07:14:43.404901+0000","last_active":"2023-10-08T07:14:44.304186+0000","last_peered":"2023-10-08T07:14:44.304186+0000","last_clean":"2023-10-08T07:14:44.304186+0000","last_became_active":"2023-10-08T07:14:43.404611+0000","last_became_peered":"2023-10-08T07:14:43.404611+0000","last_unstale":"2023-10-08T07:14:44.304186+0000","last_undegraded":"2023-10-08T07:14:44.304186+0000","last_fullsized":"2023-10-08T07:14:44.304186+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:51.297454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.309755+0000","last_change":"2023-10-08T07:14:44.332132+0000","last_active":"2023-10-08T07:14:45.309755+0000","last_peered":"2023-10-08T07:14:45.309755+0000","last_clean":"2023-10-08T07:14:45.309755+0000","last_became_active":"2023-10-08T07:14:44.331822+0000","last_became_peered":"2023-10-08T07:14:44.331822+0000","last_unstale":"2023-10-08T07:14:45.309755+0000","last_undegraded":"2023-10-08T07:14:45.309755+0000","last_fullsized":"2023-10-08T07:14:45.309755+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:21.791606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.305957+0000","last_change":"2023-10-08T07:14:43.409897+0000","last_active":"2023-10-08T07:14:44.305957+0000","last_peered":"2023-10-08T07:14:44.305957+0000","last_clean":"2023-10-08T07:14:44.305957+0000","last_became_active":"2023-10-08T07:14:43.409442+0000","last_became_peered":"2023-10-08T07:14:43.409442+0000","last_unstale":"2023-10-08T07:14:44.305957+0000","last_undegraded":"2023-10-08T07:14:44.305957+0000","last_fullsized":"2023-10-08T07:14:44.305957+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:12.105709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193226+0000","last_change":"2023-10-08T07:14:44.330119+0000","last_active":"2023-10-08T07:14:46.193226+0000","last_peered":"2023-10-08T07:14:46.193226+0000","last_clean":"2023-10-08T07:14:46.193226+0000","last_became_active":"2023-10-08T07:14:44.329841+0000","last_became_peered":"2023-10-08T07:14:44.329841+0000","last_unstale":"2023-10-08T07:14:46.193226+0000","last_undegraded":"2023-10-08T07:14:46.193226+0000","last_fullsized":"2023-10-08T07:14:46.193226+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:17.916779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.325914+0000","last_change":"2023-10-08T07:14:43.406046+0000","last_active":"2023-10-08T07:14:44.325914+0000","last_peered":"2023-10-08T07:14:44.325914+0000","last_clean":"2023-10-08T07:14:44.325914+0000","last_became_active":"2023-10-08T07:14:43.405138+0000","last_became_peered":"2023-10-08T07:14:43.405138+0000","last_unstale":"2023-10-08T07:14:44.325914+0000","last_undegraded":"2023-10-08T07:14:44.325914+0000","last_fullsized":"2023-10-08T07:14:44.325914+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:53.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.305289+0000","last_change":"2023-10-08T07:14:44.329222+0000","last_active":"2023-10-08T07:14:45.305289+0000","last_peered":"2023-10-08T07:14:45.305289+0000","last_clean":"2023-10-08T07:14:45.305289+0000","last_became_active":"2023-10-08T07:14:44.328917+0000","last_became_peered":"2023-10-08T07:14:44.328917+0000","last_unstale":"2023-10-08T07:14:45.305289+0000","last_undegraded":"2023-10-08T07:14:45.305289+0000","last_fullsized":"2023-10-08T07:14:45.305289+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:35.827786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.310589+0000","last_change":"2023-10-08T07:14:43.411524+0000","last_active":"2023-10-08T07:14:44.310589+0000","last_peered":"2023-10-08T07:14:44.310589+0000","last_clean":"2023-10-08T07:14:44.310589+0000","last_became_active":"2023-10-08T07:14:43.411151+0000","last_became_peered":"2023-10-08T07:14:43.411151+0000","last_unstale":"2023-10-08T07:14:44.310589+0000","last_undegraded":"2023-10-08T07:14:44.310589+0000","last_fullsized":"2023-10-08T07:14:44.310589+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:49.845188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.195332+0000","last_change":"2023-10-08T07:14:44.335115+0000","last_active":"2023-10-08T07:14:46.195332+0000","last_peered":"2023-10-08T07:14:46.195332+0000","last_clean":"2023-10-08T07:14:46.195332+0000","last_became_active":"2023-10-08T07:14:44.334638+0000","last_became_peered":"2023-10-08T07:14:44.334638+0000","last_unstale":"2023-10-08T07:14:46.195332+0000","last_undegraded":"2023-10-08T07:14:46.195332+0000","last_fullsized":"2023-10-08T07:14:46.195332+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:36.291831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.311585+0000","last_change":"2023-10-08T07:14:43.408103+0000","last_active":"2023-10-08T07:14:44.311585+0000","last_peered":"2023-10-08T07:14:44.311585+0000","last_clean":"2023-10-08T07:14:44.311585+0000","last_became_active":"2023-10-08T07:14:43.407827+0000","last_became_peered":"2023-10-08T07:14:43.407827+0000","last_unstale":"2023-10-08T07:14:44.311585+0000","last_undegraded":"2023-10-08T07:14:44.311585+0000","last_fullsized":"2023-10-08T07:14:44.311585+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:50.115871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.193721+0000","last_change":"2023-10-08T07:14:44.334559+0000","last_active":"2023-10-08T07:14:46.193721+0000","last_peered":"2023-10-08T07:14:46.193721+0000","last_clean":"2023-10-08T07:14:46.193721+0000","last_became_active":"2023-10-08T07:14:44.334019+0000","last_became_peered":"2023-10-08T07:14:44.334019+0000","last_unstale":"2023-10-08T07:14:46.193721+0000","last_undegraded":"2023-10-08T07:14:46.193721+0000","last_fullsized":"2023-10-08T07:14:46.193721+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:59.631069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":12,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.317368+0000","last_change":"2023-10-08T07:14:43.413981+0000","last_active":"2023-10-08T07:14:44.317368+0000","last_peered":"2023-10-08T07:14:44.317368+0000","last_clean":"2023-10-08T07:14:44.317368+0000","last_became_active":"2023-10-08T07:14:43.413671+0000","last_became_peered":"2023-10-08T07:14:43.413671+0000","last_unstale":"2023-10-08T07:14:44.317368+0000","last_undegraded":"2023-10-08T07:14:44.317368+0000","last_fullsized":"2023-10-08T07:14:44.317368+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:35.209529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:45.304811+0000","last_change":"2023-10-08T07:14:44.336800+0000","last_active":"2023-10-08T07:14:45.304811+0000","last_peered":"2023-10-08T07:14:45.304811+0000","last_clean":"2023-10-08T07:14:45.304811+0000","last_became_active":"2023-10-08T07:14:44.331013+0000","last_became_peered":"2023-10-08T07:14:44.331013+0000","last_unstale":"2023-10-08T07:14:45.304811+0000","last_undegraded":"2023-10-08T07:14:45.304811+0000","last_fullsized":"2023-10-08T07:14:45.304811+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:39.261986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"253'1","reported_seq":13,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-10-08T07:14:44.398406+0000","last_change":"2023-10-08T07:14:43.410107+0000","last_active":"2023-10-08T07:14:44.398406+0000","last_peered":"2023-10-08T07:14:44.398406+0000","last_clean":"2023-10-08T07:14:44.398406+0000","last_became_active":"2023-10-08T07:14:43.409781+0000","last_became_peered":"2023-10-08T07:14:43.409781+0000","last_unstale":"2023-10-08T07:14:44.398406+0000","last_undegraded":"2023-10-08T07:14:44.398406+0000","last_fullsized":"2023-10-08T07:14:44.398406+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:08:03.181094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"240'2828","reported_seq":3817,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-10-08T07:14:46.191779+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:14:46.191779+0000","last_peered":"2023-10-08T07:14:46.191779+0000","last_clean":"2023-10-08T07:14:46.191779+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:14:46.191779+0000","last_undegraded":"2023-10-08T07:14:46.191779+0000","last_fullsized":"2023-10-08T07:14:46.191779+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2828,"log_dups_size":0,"ondisk_log_size":2828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":629,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":629,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2828,"ondisk_log_size":2828,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576022,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65420,"kb_used_data":3712,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94306420,"statfs":{"total":96636764160,"available":96569774080,"internally_reserved":0,"allocated":3801088,"data_stored":2258971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.137,"15min":1.1970000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.4199999999999999,"5min":1.968,"15min":2.008},"last":1.129},{"interface":"front","average":{"1min":1.179,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.282}]},{"osd":1,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.226,"15min":1.2430000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.528,"5min":1.79,"15min":1.998},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.3260000000000001,"15min":1.3420000000000001},"min":{"1min":0.997,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.069}]},{"osd":2,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.27,"15min":1.302},"min":{"1min":0.86399999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":2.1640000000000001},"last":1.345},{"interface":"front","average":{"1min":1.28,"5min":1.359,"15min":1.3420000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.7150000000000001,"5min":1.8580000000000001,"15min":2.3170000000000002},"last":1.593}]},{"osd":3,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2949999999999999,"15min":1.3360000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.7549999999999999,"5min":1.8160000000000001,"15min":1.9039999999999999},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.3899999999999999,"15min":1.407},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.8140000000000001,"5min":1.8680000000000001,"15min":1.8700000000000001},"last":1.4510000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.321,"15min":1.3220000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.929,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.3979999999999999,"15min":1.4059999999999999},"min":{"1min":1.018,"5min":0.92600000000000005,"15min":0.85199999999999998},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":2.0379999999999998},"last":1.2210000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.4490000000000001,"15min":1.4430000000000001},"min":{"1min":0.97399999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.0670000000000002,"5min":2.0920000000000001,"15min":2.1139999999999999},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4750000000000001,"15min":1.5009999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.8620000000000001,"5min":2.23,"15min":2.23},"last":1.7050000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.5900000000000001,"15min":1.6020000000000001},"min":{"1min":1.2529999999999999,"5min":1.048,"15min":1.048},"max":{"1min":2.2120000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.55,"5min":1.5409999999999999,"15min":1.54},"min":{"1min":1.139,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":1.982,"5min":2.21,"15min":2.2570000000000001},"last":1.53}]}]},{"osd":6,"up_from":13,"seq":55834576021,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30152,"kb_used_data":3132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341688,"statfs":{"total":96636764160,"available":96605888512,"internally_reserved":0,"allocated":3207168,"data_stored":1668591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.286,"15min":1.3089999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.637,"5min":1.794,"15min":1.962},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3120000000000001,"15min":1.3520000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6220000000000001,"5min":1.76,"15min":2.0070000000000001},"last":1.53}]},{"osd":1,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2030000000000001,"15min":1.2270000000000001},"min":{"1min":0.71999999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.7330000000000001,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.222,"15min":1.2430000000000001},"min":{"1min":0.71799999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.88200000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.1599999999999999,"15min":1.2310000000000001},"min":{"1min":0.86899999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":2.339},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2749999999999999,"15min":1.306},"min":{"1min":0.77100000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.627,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":1.4610000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.397},"min":{"1min":0.86299999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8280000000000001,"5min":1.9319999999999999,"15min":1.95},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.429,"15min":1.4890000000000001},"min":{"1min":0.95199999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.7789999999999999,"5min":1.8959999999999999,"15min":2.1459999999999999},"last":1.667}]},{"osd":4,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3720000000000001,"15min":1.389},"min":{"1min":0.877,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.849,"5min":1.9359999999999999,"15min":2.0859999999999999},"last":1.159},{"interface":"front","average":{"1min":1.518,"5min":1.4550000000000001,"15min":1.4570000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":1.9650000000000001,"5min":2.1110000000000002,"15min":2.484},"last":1.8600000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.389,"15min":1.4279999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8069999999999999,"5min":1.8660000000000001,"15min":2.403},"last":1.712},{"interface":"front","average":{"1min":1.575,"5min":1.5489999999999999,"15min":1.5549999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":1.4119999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5369999999999999,"15min":1.5760000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.339,"5min":2.339,"15min":2.5870000000000002},"last":1.7649999999999999},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.552,"15min":1.5620000000000001},"min":{"1min":0.99299999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.0710000000000002,"5min":2.2959999999999998,"15min":2.665},"last":1.599}]}]},{"osd":5,"up_from":13,"seq":55834576021,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60676,"kb_used_data":3124,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311164,"statfs":{"total":96636764160,"available":96574631936,"internally_reserved":0,"allocated":3198976,"data_stored":1668479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.161,"15min":1.2070000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1930000000000001,"15min":1.2230000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.2070000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1970000000000001,"15min":1.2370000000000001},"min":{"1min":0.66800000000000004,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.339,"15min":1.3120000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":7.5250000000000004,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":0.90000000000000002}]},{"osd":2,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.266,"15min":1.3009999999999999},"min":{"1min":0.68899999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.0870000000000002},"last":1.571},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2450000000000001,"15min":1.3009999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.645,"5min":1.877,"15min":1.909},"last":1.2829999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.4159999999999999,"15min":1.391},"min":{"1min":0.72499999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":6.8449999999999998,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":1.242},{"interface":"front","average":{"1min":1.619,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":0.64900000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":9.3599999999999994,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":1.49}]},{"osd":4,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.319,"15min":1.3280000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4319999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.742,"5min":2.742,"15min":2.742},"last":1.373}]},{"osd":6,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.978,"5min":1.5329999999999999,"15min":1.486},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":13.234999999999999,"5min":13.234999999999999,"15min":13.234999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.5129999999999999,"15min":1.482},"min":{"1min":0.67800000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":7.2949999999999999,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":1.355}]},{"osd":7,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.536,"15min":1.5449999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5700000000000001,"15min":1.601},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":1.3260000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576021,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48200,"kb_used_data":3128,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323640,"statfs":{"total":96636764160,"available":96587407360,"internally_reserved":0,"allocated":3203072,"data_stored":1668549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.323,"15min":1.3520000000000001},"min":{"1min":0.66800000000000004,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":5.4800000000000004,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.284},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.421,"15min":1.4219999999999999},"min":{"1min":0.79200000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.0880000000000001,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.5}]},{"osd":1,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.72599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":1.347},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.3129999999999999,"15min":1.2729999999999999},"min":{"1min":0.61299999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":7.673,"5min":7.673,"15min":7.673},"last":1.181}]},{"osd":2,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":2.2629999999999999,"5min":1.4690000000000001,"15min":1.3260000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":17.175999999999998,"5min":17.175999999999998,"15min":17.175999999999998},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.651,"5min":1.4139999999999999,"15min":1.3859999999999999},"min":{"1min":0.629,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":1.6000000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.3600000000000001,"15min":1.347},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":7.5270000000000001,"5min":7.5270000000000001,"15min":7.5270000000000001},"last":1.421},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.5209999999999999,"15min":1.4430000000000001},"min":{"1min":0.59199999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":7.7599999999999998,"5min":7.7599999999999998,"15min":7.7599999999999998},"last":1.5529999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.55,"15min":1.419},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":7.4020000000000001,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.708},{"interface":"front","average":{"1min":1.754,"5min":1.4970000000000001,"15min":1.415},"min":{"1min":0.442,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":7.79,"5min":7.79,"15min":7.79},"last":1.7589999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.621,"15min":1.5},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":10.542999999999999,"5min":10.542999999999999,"15min":10.542999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":2.1819999999999999,"5min":1.7150000000000001,"15min":1.605},"min":{"1min":0.86099999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":10.433999999999999,"5min":10.433999999999999,"15min":10.433999999999999},"last":1.788}]},{"osd":7,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.8899999999999999,"5min":1.6259999999999999,"15min":1.5409999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":7.915,"5min":7.915,"15min":7.915},"last":1.651},{"interface":"front","average":{"1min":1.8839999999999999,"5min":1.73,"15min":1.6559999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":1.8280000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576023,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52376,"kb_used_data":3148,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319464,"statfs":{"total":96636764160,"available":96583131136,"internally_reserved":0,"allocated":3223552,"data_stored":1670729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50395592},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92700000000000005,"15min":0.97399999999999998},"min":{"1min":0.32200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1990000000000001,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.131,"15min":1.1659999999999999},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5349999999999999,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":1.2629999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.96799999999999997,"15min":1.022},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3560000000000001,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":1.127},{"interface":"front","average":{"1min":1.054,"5min":1.1890000000000001,"15min":1.2729999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.667,"5min":2.109,"15min":2.4889999999999999},"last":1.667}]},{"osd":2,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0629999999999999,"15min":1.1399999999999999},"min":{"1min":0.502,"5min":0.437,"15min":0.437},"max":{"1min":1.462,"5min":2.165,"15min":2.2250000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.139,"15min":1.1899999999999999},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.671,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.393}]},{"osd":4,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2290000000000001,"15min":1.272},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.67500000000000004},"max":{"1min":1.7290000000000001,"5min":1.857,"15min":2.5379999999999998},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.381,"15min":1.45},"min":{"1min":0.91600000000000004,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.8149999999999999,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.8149999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.294,"15min":1.3660000000000001},"min":{"1min":0.76700000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7829999999999999,"5min":1.923,"15min":2.9740000000000002},"last":1.536},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3919999999999999,"15min":1.4330000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9610000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":1.9610000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.82299999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.9690000000000001,"5min":1.976,"15min":2.653},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.4350000000000001,"15min":1.5129999999999999},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.0150000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":2.0150000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.512,"15min":1.5580000000000001},"min":{"1min":0.89100000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.3220000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":2.3220000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4490000000000001,"15min":1.512},"min":{"1min":0.80900000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.903,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.869}]}]},{"osd":2,"up_from":13,"seq":55834576021,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52360,"kb_used_data":3132,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319480,"statfs":{"total":96636764160,"available":96583147520,"internally_reserved":0,"allocated":3207168,"data_stored":1669131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.91600000000000004,"15min":0.97199999999999998},"min":{"1min":0.49199999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.948},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.008,"15min":1.0660000000000001},"min":{"1min":0.55700000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.423,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":1.175}]},{"osd":1,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.93400000000000005,"15min":0.98999999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.413,"5min":1.6279999999999999,"15min":2.044},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.1479999999999999,"15min":1.1890000000000001},"min":{"1min":0.61499999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":2.1309999999999998},"last":1.016}]},{"osd":3,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.141,"15min":1.177},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.786,"5min":1.786,"15min":1.8420000000000001},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.236,"15min":1.272},"min":{"1min":0.745,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.663,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.5009999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.391,"15min":1.4379999999999999},"min":{"1min":0.92900000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.992,"5min":1.992,"15min":2.1789999999999998},"last":1.8640000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.304,"15min":1.3120000000000001},"min":{"1min":0.87,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.843,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.3340000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3600000000000001,"15min":1.4019999999999999},"min":{"1min":0.89200000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.931,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.4930000000000001,"15min":1.532},"min":{"1min":1.1060000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.837,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.7210000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.361,"15min":1.395},"min":{"1min":1.0549999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.722,"5min":2.056,"15min":2.1240000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4830000000000001,"15min":1.5249999999999999},"min":{"1min":0.99299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.994,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":1.667}]},{"osd":7,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.446,"15min":1.4790000000000001},"min":{"1min":1.0800000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7150000000000001,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.51,"15min":1.577},"min":{"1min":1.028,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.056,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":1.8120000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576021,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52352,"kb_used_data":3124,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319488,"statfs":{"total":96636764160,"available":96583155712,"internally_reserved":0,"allocated":3198976,"data_stored":1668479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.999,"15min":1.0049999999999999},"min":{"1min":0.61699999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.3380000000000001,"5min":1.6579999999999999,"15min":1.708},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.1299999999999999,"15min":1.212},"min":{"1min":0.61299999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.5700000000000001,"5min":1.7010000000000001,"15min":1.79},"last":0.52900000000000003}]},{"osd":2,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.94399999999999995,"15min":0.96599999999999997},"min":{"1min":0.54700000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.462,"5min":1.641,"15min":1.641},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.123,"5min":1.1379999999999999,"15min":1.1559999999999999},"min":{"1min":0.48699999999999999,"5min":0.372,"15min":0.372},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.752}]},{"osd":3,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2010000000000001,"15min":1.2549999999999999},"min":{"1min":0.53200000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.66300000000000003,"5min":0.435,"15min":0.435},"max":{"1min":2.5310000000000001,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":0.64600000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.3140000000000001,"15min":1.3620000000000001},"min":{"1min":0.79900000000000004,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.73,"5min":2.73,"15min":2.8460000000000001},"last":1.139},{"interface":"front","average":{"1min":1.252,"5min":1.2989999999999999,"15min":1.3169999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.86099999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3580000000000001,"15min":1.381},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.4340000000000002,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.544,"5min":1.5349999999999999,"15min":1.5329999999999999},"min":{"1min":1.1379999999999999,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.601,"5min":2.601,"15min":2.601},"last":1.105}]},{"osd":6,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.492,"15min":1.5129999999999999},"min":{"1min":0.95999999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.883,"5min":2.883,"15min":2.9079999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.4750000000000001,"15min":1.4990000000000001},"min":{"1min":1.0629999999999999,"5min":1.046,"15min":1.046},"max":{"1min":2.6589999999999998,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":0.93500000000000005}]},{"osd":7,"last update":"Sun Oct 8 07:13:54 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.552,"15min":1.554},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.879},{"interface":"front","average":{"1min":1.524,"5min":1.538,"15min":1.5700000000000001},"min":{"1min":0.89700000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":0.83799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834576022,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61268,"kb_used_data":3716,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310572,"statfs":{"total":96636764160,"available":96574025728,"internally_reserved":0,"allocated":3805184,"data_stored":2260459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92000000000000004,"15min":0.96499999999999997},"min":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.321,"5min":1.581,"15min":1.581},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.016,"15min":1.0840000000000001},"min":{"1min":0.48899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.4039999999999999,"5min":1.621,"15min":1.726},"last":0.77800000000000002}]},{"osd":2,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.081,"15min":1.1379999999999999},"min":{"1min":0.52000000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.071},"min":{"1min":0.51300000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4690000000000001,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1859999999999999,"15min":1.2450000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5600000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.141,"15min":1.165},"min":{"1min":0.55100000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":2.0979999999999999},"last":0.92200000000000004}]},{"osd":4,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3080000000000001,"15min":1.3400000000000001},"min":{"1min":0.746,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6359999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.381},{"interface":"front","average":{"1min":1.284,"5min":1.361,"15min":1.4079999999999999},"min":{"1min":0.79900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.181},"last":1.4610000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3939999999999999,"15min":1.4159999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.802,"5min":1.919,"15min":2.1309999999999998},"last":1.476},{"interface":"front","average":{"1min":1.466,"5min":1.47,"15min":1.49},"min":{"1min":0.81999999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.23,"5min":2.23,"15min":2.4900000000000002},"last":1.742}]},{"osd":6,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4359999999999999,"15min":1.4770000000000001},"min":{"1min":0.94999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0150000000000001,"5min":2.21,"15min":2.2149999999999999},"last":1.615},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.462,"15min":1.4990000000000001},"min":{"1min":0.78200000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7529999999999999,"5min":2.0880000000000001,"15min":2.355},"last":1.413}]},{"osd":7,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.5309999999999999,"15min":1.542},"min":{"1min":0.95999999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9370000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.448,"5min":1.518,"15min":1.5529999999999999},"min":{"1min":0.94599999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.516}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:14:49.599 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T07:14:49.600 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T07:14:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.855+0000 7f545b06f640 1 Processor -- start 2023-10-08T07:14:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.855+0000 7f545b06f640 1 -- start start 2023-10-08T07:14:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.857+0000 7f545b06f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54540cd4f0 0x7f54540b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.857+0000 7f545b06f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f545415d160 0x7f54540b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.857+0000 7f545b06f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54540ce300 con 0x7f545415cd80 2023-10-08T07:14:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.857+0000 7f545b06f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54540d1a30 con 0x7f54540cd4f0 2023-10-08T07:14:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.857+0000 7f545b06f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54540d1d10 con 0x7f545415d160 2023-10-08T07:14:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.857+0000 7f5453fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54540cd4f0 0x7f54540b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.857+0000 7f5453fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f54540cd4f0 0x7f54540b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40266/0 (socket says 172.21.15.70:40266) 2023-10-08T07:14:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.857+0000 7f5453fff640 1 -- 172.21.15.70:0/3224856071 learned_addr learned my addr 172.21.15.70:0/3224856071 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.858+0000 7f5453fff640 1 -- 172.21.15.70:0/3224856071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f545415d160 msgr2=0x7f54540b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.858+0000 7f5453fff640 1 --2- 172.21.15.70:0/3224856071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f545415d160 0x7f54540b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.858+0000 7f5453fff640 1 -- 172.21.15.70:0/3224856071 >> v1:172.21.15.181:6789/0 conn(0x7f545415cd80 legacy=0x7f54540cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:49.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.858+0000 7f5453fff640 1 -- 172.21.15.70:0/3224856071 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540d1ff0 con 0x7f54540cd4f0 2023-10-08T07:14:49.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.859+0000 7f5453fff640 1 --2- 172.21.15.70:0/3224856071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 0x7f54540b4610 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f5440009870 tx=0x7f544002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c12492538b91cda server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:49.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f54537fe640 1 -- 172.21.15.70:0/3224856071 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440031640 con 0x7f54540cd4f0 2023-10-08T07:14:49.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f54537fe640 1 -- 172.21.15.70:0/3224856071 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54400317a0 con 0x7f54540cd4f0 2023-10-08T07:14:49.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f54537fe640 1 -- 172.21.15.70:0/3224856071 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440031b30 con 0x7f54540cd4f0 2023-10-08T07:14:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f545b06f640 1 -- 172.21.15.70:0/3224856071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 msgr2=0x7f54540b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f545b06f640 1 --2- 172.21.15.70:0/3224856071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 0x7f54540b4610 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f5440009870 tx=0x7f544002b8f0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f545b06f640 1 -- 172.21.15.70:0/3224856071 shutdown_connections 2023-10-08T07:14:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f545b06f640 1 --2- 172.21.15.70:0/3224856071 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f545415d160 0x7f54540b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f545b06f640 1 --2- 172.21.15.70:0/3224856071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 0x7f54540b4610 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f545b06f640 1 -- 172.21.15.70:0/3224856071 >> 172.21.15.70:0/3224856071 conn(0x7f54540cf610 msgr2=0x7f54540cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f545b06f640 1 -- 172.21.15.70:0/3224856071 shutdown_connections 2023-10-08T07:14:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.860+0000 7f545b06f640 1 -- 172.21.15.70:0/3224856071 wait complete. 2023-10-08T07:14:49.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.862+0000 7f545b06f640 1 Processor -- start 2023-10-08T07:14:49.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.862+0000 7f545b06f640 1 -- start start 2023-10-08T07:14:49.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.863+0000 7f545b06f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 0x7f54540c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.863+0000 7f545b06f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f545415cd80 0x7f54540c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.863+0000 7f545b06f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f545415d160 0x7f54541f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.863+0000 7f545b06f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540c7610 con 0x7f54540cd4f0 2023-10-08T07:14:49.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.863+0000 7f545b06f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54541f25b0 con 0x7f545415cd80 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.863+0000 7f545b06f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54540ca710 con 0x7f545415d160 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.863+0000 7f5458e0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 0x7f54540c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.863+0000 7f5458e0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 0x7f54540c90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40268/0 (socket says 172.21.15.70:40268) 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f5458e0b640 1 -- 172.21.15.70:0/3006972168 learned_addr learned my addr 172.21.15.70:0/3006972168 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f5453fff640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f545415cd80 0x7f54540c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f545960c640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f545415d160 0x7f54541f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f5458e0b640 1 -- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f545415d160 msgr2=0x7f54541f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f5458e0b640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f545415d160 0x7f54541f2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f5458e0b640 1 -- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f545415cd80 msgr2=0x7f54540c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f5458e0b640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f545415cd80 0x7f54540c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f5458e0b640 1 -- 172.21.15.70:0/3006972168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400094d0 con 0x7f54540cd4f0 2023-10-08T07:14:49.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f5458e0b640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 0x7f54540c90d0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f544800c410 tx=0x7f544800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f54517fa640 1 -- 172.21.15.70:0/3006972168 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54480035f0 con 0x7f54540cd4f0 2023-10-08T07:14:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f54517fa640 1 -- 172.21.15.70:0/3006972168 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5448003750 con 0x7f54540cd4f0 2023-10-08T07:14:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f54517fa640 1 -- 172.21.15.70:0/3006972168 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448003b40 con 0x7f54540cd4f0 2023-10-08T07:14:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.864+0000 7f545b06f640 1 -- 172.21.15.70:0/3006972168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54540ca9f0 con 0x7f54540cd4f0 2023-10-08T07:14:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.865+0000 7f545b06f640 1 -- 172.21.15.70:0/3006972168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54540cafb0 con 0x7f54540cd4f0 2023-10-08T07:14:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.865+0000 7f545b06f640 1 -- 172.21.15.70:0/3006972168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54540cb1a0 con 0x7f54540cd4f0 2023-10-08T07:14:49.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.866+0000 7f54517fa640 1 -- 172.21.15.70:0/3006972168 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f5448003ca0 con 0x7f54540cd4f0 2023-10-08T07:14:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.867+0000 7f54517fa640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f543806b0b0 0x7f543806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.867+0000 7f54517fa640 1 -- 172.21.15.70:0/3006972168 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f5448088ee0 con 0x7f54540cd4f0 2023-10-08T07:14:49.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.868+0000 7f5453fff640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f543806b0b0 0x7f543806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:49.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.869+0000 7f5453fff640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f543806b0b0 0x7f543806d570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5440036000 tx=0x7f544003c040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:49.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:49.872+0000 7f54517fa640 1 -- 172.21.15.70:0/3006972168 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54480539d0 con 0x7f54540cd4f0 2023-10-08T07:14:50.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.129+0000 7f545b06f640 1 -- 172.21.15.70:0/3006972168 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f54540cc650 con 0x7f54540cd4f0 2023-10-08T07:14:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.134+0000 7f54517fa640 1 -- 172.21.15.70:0/3006972168 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v53) v1 ==== 190+0+147 (secure 0 0 0) 0x7f54480170b0 con 0x7f54540cd4f0 2023-10-08T07:14:50.137 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T07:14:50.138 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T07:14:50.138 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T07:14:50.138 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T07:14:50.138 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T07:14:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T07:14:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 -- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f543806b0b0 msgr2=0x7f543806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f543806b0b0 0x7f543806d570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5440036000 tx=0x7f544003c040 comp rx=0 tx=0).stop 2023-10-08T07:14:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 -- 172.21.15.70:0/3006972168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 msgr2=0x7f54540c90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 0x7f54540c90d0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f544800c410 tx=0x7f544800c8d0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 -- 172.21.15.70:0/3006972168 shutdown_connections 2023-10-08T07:14:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f543806b0b0 0x7f543806d570 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f545415d160 0x7f54541f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f545415cd80 0x7f54540c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 --2- 172.21.15.70:0/3006972168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54540cd4f0 0x7f54540c90d0 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 -- 172.21.15.70:0/3006972168 >> 172.21.15.70:0/3006972168 conn(0x7f54540cf610 msgr2=0x7f54540b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 -- 172.21.15.70:0/3006972168 shutdown_connections 2023-10-08T07:14:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.137+0000 7f545b06f640 1 -- 172.21.15.70:0/3006972168 wait complete. 2023-10-08T07:14:50.159 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:14:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.372+0000 7f7bc3ee5640 1 Processor -- start 2023-10-08T07:14:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.372+0000 7f7bc3ee5640 1 -- start start 2023-10-08T07:14:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.374+0000 7f7bc3ee5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7bbc160940 0x7f7bbc0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.374+0000 7f7bc3ee5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bbc06c890 0x7f7bbc06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.374+0000 7f7bc3ee5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc0c48f0 con 0x7f7bbc15cd90 2023-10-08T07:14:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.374+0000 7f7bc3ee5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bbc0c4d80 con 0x7f7bbc160940 2023-10-08T07:14:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.374+0000 7f7bc3ee5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc0c3ce0 con 0x7f7bbc06c890 2023-10-08T07:14:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.374+0000 7f7bc1c81640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7bbc15cd90 0x7f7bbc0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46492/0 (socket says 172.21.15.70:46492) 2023-10-08T07:14:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.374+0000 7f7bc1c81640 1 -- 172.21.15.70:0/2856515486 learned_addr learned my addr 172.21.15.70:0/2856515486 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.374+0000 7f7bc1480640 1 --2- 172.21.15.70:0/2856515486 >> v2:172.21.15.181:3300/0 conn(0x7f7bbc160940 0x7f7bbc0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.374+0000 7f7bc2482640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bbc06c890 0x7f7bbc06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541168429 0 0) 0x7f7bbc0c48f0 con 0x7f7bbc15cd90 2023-10-08T07:14:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba8003520 con 0x7f7bbc15cd90 2023-10-08T07:14:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1885925609 0 0) 0x7f7ba8003520 con 0x7f7bbc15cd90 2023-10-08T07:14:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bbc06c890 msgr2=0x7f7bbc06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bbc06c890 0x7f7bbc06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 >> v2:172.21.15.181:3300/0 conn(0x7f7bbc160940 msgr2=0x7f7bbc0709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 --2- 172.21.15.70:0/2856515486 >> v2:172.21.15.181:3300/0 conn(0x7f7bbc160940 0x7f7bbc0709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc0c3fc0 con 0x7f7bbc15cd90 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7bb8001670 con 0x7f7bbc15cd90 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 >> v1:172.21.15.70:6789/0 conn(0x7f7bbc15cd90 legacy=0x7f7bbc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba8003970 0x7f7ba8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bbc06c890 0x7f7ba80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba8004d00 0x7f7ba8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc0c3ce0 con 0x7f7ba8003970 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba800b980 con 0x7f7ba8004d00 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.375+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba800bac0 con 0x7f7bbc06c890 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.376+0000 7f7bc1c81640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba8003970 0x7f7ba8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.376+0000 7f7bc1480640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bbc06c890 0x7f7ba80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.376+0000 7f7bc2482640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba8004d00 0x7f7ba8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.376+0000 7f7bc1c81640 1 -- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bbc06c890 msgr2=0x7f7ba80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.376+0000 7f7bc1c81640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7bbc06c890 0x7f7ba80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.376+0000 7f7bc1c81640 1 -- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba8004d00 msgr2=0x7f7ba8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.376+0000 7f7bc1c81640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba8004d00 0x7f7ba8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.376+0000 7f7bc1c81640 1 -- 172.21.15.70:0/2856515486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba8003520 con 0x7f7ba8003970 2023-10-08T07:14:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.376+0000 7f7bc2482640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7ba8004d00 0x7f7ba8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.377+0000 7f7bc1c81640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba8003970 0x7f7ba8003d90 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800c240 tx=0x7f7ba80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5122d8e2490ff2cc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.378+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8005ad0 con 0x7f7ba8003970 2023-10-08T07:14:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.378+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb8005c30 con 0x7f7ba8003970 2023-10-08T07:14:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.378+0000 7f7bc0c7f640 1 -- 172.21.15.70:0/2856515486 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb802c8d0 con 0x7f7ba8003970 2023-10-08T07:14:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.378+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2856515486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba8003970 msgr2=0x7f7ba8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.378+0000 7f7bc3ee5640 1 --2- 172.21.15.70:0/2856515486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ba8003970 0x7f7ba8003d90 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800c240 tx=0x7f7ba80057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.378+0000 7f7bc2482640 1 -- 172.21.15.70:0/2856515486 reap_dead start 2023-10-08T07:14:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.378+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2856515486 shutdown_connections 2023-10-08T07:14:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.378+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2856515486 >> 172.21.15.70:0/2856515486 conn(0x7f7bbc06a380 msgr2=0x7f7bbc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.379+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2856515486 shutdown_connections 2023-10-08T07:14:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.379+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2856515486 wait complete. 2023-10-08T07:14:50.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.380+0000 7f7bc3ee5640 1 Processor -- start 2023-10-08T07:14:50.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.380+0000 7f7bc3ee5640 1 -- start start 2023-10-08T07:14:50.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.381+0000 7f7bc3ee5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ba8003970 0x7f7bbc1f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:50.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.381+0000 7f7bc3ee5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bbc15d970 0x7f7bbc156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.381+0000 7f7bc3ee5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bbc15ddf0 0x7f7bbc1568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.381+0000 7f7bc3ee5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc1f2a10 con 0x7f7bbc15d970 2023-10-08T07:14:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.381+0000 7f7bc3ee5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc15e3c0 con 0x7f7bbc15ddf0 2023-10-08T07:14:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.381+0000 7f7bc3ee5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc15e6a0 con 0x7f7ba8003970 2023-10-08T07:14:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc1480640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bbc15d970 0x7f7bbc156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc1480640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bbc15d970 0x7f7bbc156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40280/0 (socket says 172.21.15.70:40280) 2023-10-08T07:14:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc1480640 1 -- 172.21.15.70:0/2946195700 learned_addr learned my addr 172.21.15.70:0/2946195700 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc1480640 1 -- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ba8003970 msgr2=0x7f7bbc1f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc1c81640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ba8003970 0x7f7bbc1f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc2482640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bbc15ddf0 0x7f7bbc1568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc1480640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ba8003970 0x7f7bbc1f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc1480640 1 -- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bbc15ddf0 msgr2=0x7f7bbc1568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc1480640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bbc15ddf0 0x7f7bbc1568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc1480640 1 -- 172.21.15.70:0/2946195700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc15e980 con 0x7f7bbc15d970 2023-10-08T07:14:50.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.382+0000 7f7bc1480640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bbc15d970 0x7f7bbc156390 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f7bac011600 tx=0x7f7bac011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:50.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.383+0000 7f7bb27fc640 1 -- 172.21.15.70:0/2946195700 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00f920 con 0x7f7bbc15d970 2023-10-08T07:14:50.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.383+0000 7f7bb27fc640 1 -- 172.21.15.70:0/2946195700 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac00fa80 con 0x7f7bbc15d970 2023-10-08T07:14:50.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.383+0000 7f7bb27fc640 1 -- 172.21.15.70:0/2946195700 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00fdf0 con 0x7f7bbc15d970 2023-10-08T07:14:50.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.383+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2946195700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc15ec60 con 0x7f7bbc15d970 2023-10-08T07:14:50.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.383+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2946195700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc1f6fe0 con 0x7f7bbc15d970 2023-10-08T07:14:50.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.384+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2946195700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bbc067490 con 0x7f7bbc15d970 2023-10-08T07:14:50.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.385+0000 7f7bb27fc640 1 -- 172.21.15.70:0/2946195700 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7bac015460 con 0x7f7bbc15d970 2023-10-08T07:14:50.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.386+0000 7f7bb27fc640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7ba006b0e0 0x7f7ba006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:50.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.386+0000 7f7bb27fc640 1 -- 172.21.15.70:0/2946195700 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f7bac08a060 con 0x7f7bbc15d970 2023-10-08T07:14:50.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.386+0000 7f7bc1c81640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7ba006b0e0 0x7f7ba006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:50.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.388+0000 7f7bc1c81640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7ba006b0e0 0x7f7ba006d5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8034000 tx=0x7f7bb8007000 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:50.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.390+0000 7f7bb27fc640 1 -- 172.21.15.70:0/2946195700 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7bac0bcca0 con 0x7f7bbc15d970 2023-10-08T07:14:50.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.619+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2946195700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bbc0c4270 con 0x7f7bbc15d970 2023-10-08T07:14:50.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.620+0000 7f7bb27fc640 1 -- 172.21.15.70:0/2946195700 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+4926 (secure 0 0 0) 0x7f7bac022090 con 0x7f7bbc15d970 2023-10-08T07:14:50.622 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:50.623 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":125,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":8875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:43.833416+0000","modified":"2023-10-08T07:14:46.899347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11904},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11904":{"gid":11904,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/3856326680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3856326680},{"type":"v1","addr":"172.21.15.70:6834","nonce":3856326680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-10-08T07:14:50.623 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 125 2023-10-08T07:14:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7ba006b0e0 msgr2=0x7f7ba006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7ba006b0e0 0x7f7ba006d5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8034000 tx=0x7f7bb8007000 comp rx=0 tx=0).stop 2023-10-08T07:14:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2946195700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bbc15d970 msgr2=0x7f7bbc156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bbc15d970 0x7f7bbc156390 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f7bac011600 tx=0x7f7bac011ac0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2946195700 shutdown_connections 2023-10-08T07:14:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f7ba006b0e0 0x7f7ba006d5a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7bbc15ddf0 0x7f7bbc1568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bbc15d970 0x7f7bbc156390 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 --2- 172.21.15.70:0/2946195700 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7ba8003970 0x7f7bbc1f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2946195700 >> 172.21.15.70:0/2946195700 conn(0x7f7bbc06a380 msgr2=0x7f7bbc0cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2946195700 shutdown_connections 2023-10-08T07:14:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:50.623+0000 7f7bc3ee5640 1 -- 172.21.15.70:0/2946195700 wait complete. 2023-10-08T07:14:51.044 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 125, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T07:14:43.833416+0000', 'modified': '2023-10-08T07:14:46.899347+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11904}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11904': {'gid': 11904, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6834/3856326680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 3856326680}, {'type': 'v1', 'addr': '172.21.15.70:6834', 'nonce': 3856326680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T07:14:51.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T07:14:51.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T07:14:51.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.268+0000 7f359236c640 1 Processor -- start 2023-10-08T07:14:51.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.268+0000 7f359236c640 1 -- start start 2023-10-08T07:14:51.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.269+0000 7f359236c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f358c06dfc0 0x7f358c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.269+0000 7f359236c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f358c071070 0x7f358c0cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.269+0000 7f359236c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c0b1cc0 con 0x7f358c0afbe0 2023-10-08T07:14:51.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.269+0000 7f359236c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c0b2620 con 0x7f358c071070 2023-10-08T07:14:51.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.269+0000 7f359236c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0b2060 con 0x7f358c06dfc0 2023-10-08T07:14:51.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f358c0afbe0 0x7f358c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46506/0 (socket says 172.21.15.70:46506) 2023-10-08T07:14:51.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358bfff640 1 -- 172.21.15.70:0/3317953009 learned_addr learned my addr 172.21.15.70:0/3317953009 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:51.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234342854 0 0) 0x7f358c0b1cc0 con 0x7f358c0afbe0 2023-10-08T07:14:51.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3578003520 con 0x7f358c0afbe0 2023-10-08T07:14:51.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f3590909640 1 --2- 172.21.15.70:0/3317953009 >> v2:172.21.15.181:3300/0 conn(0x7f358c071070 0x7f358c0cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f35837fe640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f358c06dfc0 0x7f358c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2043865841 0 0) 0x7f3578003520 con 0x7f358c0afbe0 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f358c06dfc0 msgr2=0x7f358c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f358c06dfc0 0x7f358c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 >> v2:172.21.15.181:3300/0 conn(0x7f358c071070 msgr2=0x7f358c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 --2- 172.21.15.70:0/3317953009 >> v2:172.21.15.181:3300/0 conn(0x7f358c071070 0x7f358c0cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0b2340 con 0x7f358c0afbe0 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f357c001670 con 0x7f358c0afbe0 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f35837fe640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f358c06dfc0 0x7f358c070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 >> v1:172.21.15.70:6789/0 conn(0x7f358c0afbe0 legacy=0x7f358c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f358c06dfc0 0x7f3578003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.270+0000 7f358b7fe640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578003fb0 0x7f35780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f358b7fe640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3578004d00 0x7f3578005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0b2620 con 0x7f3578003fb0 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0b2060 con 0x7f3578004d00 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357800bb90 con 0x7f358c06dfc0 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f35837fe640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578003fb0 0x7f35780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f358bfff640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f358c06dfc0 0x7f3578003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f3590909640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3578004d00 0x7f3578005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f35837fe640 1 -- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f358c06dfc0 msgr2=0x7f3578003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f35837fe640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f358c06dfc0 0x7f3578003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f35837fe640 1 -- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3578004d00 msgr2=0x7f3578005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f35837fe640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3578004d00 0x7f3578005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f35837fe640 1 -- 172.21.15.70:0/3317953009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3578003520 con 0x7f3578003fb0 2023-10-08T07:14:51.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.271+0000 7f3590909640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3578004d00 0x7f3578005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.272+0000 7f35837fe640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578003fb0 0x7f35780047c0 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f358c06e410 tx=0x7f35780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6e5f435ea8d9d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.272+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574030050 con 0x7f3578003fb0 2023-10-08T07:14:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.273+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35740345f0 con 0x7f3578003fb0 2023-10-08T07:14:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.273+0000 7f358b7fe640 1 -- 172.21.15.70:0/3317953009 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35740349a0 con 0x7f3578003fb0 2023-10-08T07:14:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.273+0000 7f359236c640 1 -- 172.21.15.70:0/3317953009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578003fb0 msgr2=0x7f35780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.273+0000 7f359236c640 1 --2- 172.21.15.70:0/3317953009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3578003fb0 0x7f35780047c0 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f358c06e410 tx=0x7f35780057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.273+0000 7f3590909640 1 -- 172.21.15.70:0/3317953009 reap_dead start 2023-10-08T07:14:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.273+0000 7f359236c640 1 -- 172.21.15.70:0/3317953009 shutdown_connections 2023-10-08T07:14:51.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.273+0000 7f359236c640 1 -- 172.21.15.70:0/3317953009 >> 172.21.15.70:0/3317953009 conn(0x7f358c06a5d0 msgr2=0x7f358c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.273+0000 7f359236c640 1 -- 172.21.15.70:0/3317953009 shutdown_connections 2023-10-08T07:14:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.273+0000 7f359236c640 1 -- 172.21.15.70:0/3317953009 wait complete. 2023-10-08T07:14:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.275+0000 7f359236c640 1 Processor -- start 2023-10-08T07:14:51.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.275+0000 7f359236c640 1 -- start start 2023-10-08T07:14:51.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.276+0000 7f359236c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3578003fb0 0x7f358c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.276+0000 7f359236c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c15e0c0 0x7f358c15bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.276+0000 7f359236c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f358c0c4ca0 0x7f358c15ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.276+0000 7f359236c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c14f6c0 con 0x7f358c0c4ca0 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.276+0000 7f359236c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c15df20 con 0x7f358c15e0c0 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.276+0000 7f359236c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0c2f50 con 0x7f3578003fb0 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.276+0000 7f3590909640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f358c0c4ca0 0x7f358c15ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.276+0000 7f3590909640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f358c0c4ca0 0x7f358c15ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40298/0 (socket says 172.21.15.70:40298) 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.276+0000 7f3590909640 1 -- 172.21.15.70:0/2274926384 learned_addr learned my addr 172.21.15.70:0/2274926384 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.276+0000 7f35837fe640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c15e0c0 0x7f358c15bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f3590909640 1 -- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3578003fb0 msgr2=0x7f358c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f3590909640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3578003fb0 0x7f358c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f3590909640 1 -- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c15e0c0 msgr2=0x7f358c15bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f3590909640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c15e0c0 0x7f358c15bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f3590909640 1 -- 172.21.15.70:0/2274926384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c15c330 con 0x7f358c0c4ca0 2023-10-08T07:14:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f3590909640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f358c0c4ca0 0x7f358c15ed80 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f358c15b4a0 tx=0x7f358400e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f35897fa640 1 -- 172.21.15.70:0/2274926384 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358400c070 con 0x7f358c0c4ca0 2023-10-08T07:14:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f35897fa640 1 -- 172.21.15.70:0/2274926384 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3584013610 con 0x7f358c0c4ca0 2023-10-08T07:14:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f35897fa640 1 -- 172.21.15.70:0/2274926384 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840139c0 con 0x7f358c0c4ca0 2023-10-08T07:14:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f359236c640 1 -- 172.21.15.70:0/2274926384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c0c23d0 con 0x7f358c0c4ca0 2023-10-08T07:14:51.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.277+0000 7f359236c640 1 -- 172.21.15.70:0/2274926384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c0c2990 con 0x7f358c0c4ca0 2023-10-08T07:14:51.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.281+0000 7f35897fa640 1 -- 172.21.15.70:0/2274926384 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3584013b20 con 0x7f358c0c4ca0 2023-10-08T07:14:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.281+0000 7f35897fa640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f356c06b0b0 0x7f356c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.282+0000 7f35897fa640 1 -- 172.21.15.70:0/2274926384 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f35840898e0 con 0x7f358c0c4ca0 2023-10-08T07:14:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.282+0000 7f3594fd5740 1 Processor -- start 2023-10-08T07:14:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.282+0000 7f3594fd5740 1 -- start start 2023-10-08T07:14:51.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.282+0000 7f358bfff640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f356c06b0b0 0x7f356c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.283+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e67636e0 0x55c2e67612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.283+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e6742f20 0x55c2e671ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.283+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e671b150 0x55c2e67458f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.283+0000 7f3594fd5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e6689700 con 0x55c2e67636e0 2023-10-08T07:14:51.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.283+0000 7f3594fd5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e6686df0 con 0x55c2e6742f20 2023-10-08T07:14:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.283+0000 7f3594fd5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2e66826e0 con 0x55c2e671b150 2023-10-08T07:14:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.283+0000 7f358bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e67636e0 0x55c2e67612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.283+0000 7f358bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e67636e0 0x55c2e67612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40308/0 (socket says 172.21.15.70:40308) 2023-10-08T07:14:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.283+0000 7f358bfff640 1 -- 172.21.15.70:0/3048984008 learned_addr learned my addr 172.21.15.70:0/3048984008 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f358bfff640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f356c06b0b0 0x7f356c06d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f357c001050 tx=0x7f357c001170 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f35837fe640 1 --2- 172.21.15.70:0/3048984008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e6742f20 0x55c2e671ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f3590909640 1 --2- 172.21.15.70:0/3048984008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e671b150 0x55c2e67458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f358bfff640 1 -- 172.21.15.70:0/3048984008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e671b150 msgr2=0x55c2e67458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f358bfff640 1 --2- 172.21.15.70:0/3048984008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e671b150 0x55c2e67458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f358bfff640 1 -- 172.21.15.70:0/3048984008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e6742f20 msgr2=0x55c2e671ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f358bfff640 1 --2- 172.21.15.70:0/3048984008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e6742f20 0x55c2e671ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f358bfff640 1 -- 172.21.15.70:0/3048984008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2e6646910 con 0x55c2e67636e0 2023-10-08T07:14:51.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f358bfff640 1 --2- 172.21.15.70:0/3048984008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e67636e0 0x55c2e67612a0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f357c00e960 tx=0x7f357c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c75da26e8cfc9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:51.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f3581ffb640 1 -- 172.21.15.70:0/3048984008 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c014c10 con 0x55c2e67636e0 2023-10-08T07:14:51.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f3581ffb640 1 -- 172.21.15.70:0/3048984008 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c014d90 con 0x55c2e67636e0 2023-10-08T07:14:51.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.284+0000 7f3581ffb640 1 -- 172.21.15.70:0/3048984008 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c013620 con 0x55c2e67636e0 2023-10-08T07:14:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.285+0000 7f3594fd5740 1 -- 172.21.15.70:0/3048984008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e67636e0 msgr2=0x55c2e67612a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.285+0000 7f3594fd5740 1 --2- 172.21.15.70:0/3048984008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e67636e0 0x55c2e67612a0 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f357c00e960 tx=0x7f357c011000 comp rx=0 tx=0).stop 2023-10-08T07:14:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.285+0000 7f3594fd5740 1 -- 172.21.15.70:0/3048984008 shutdown_connections 2023-10-08T07:14:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.285+0000 7f3594fd5740 1 --2- 172.21.15.70:0/3048984008 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e671b150 0x55c2e67458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.285+0000 7f3594fd5740 1 --2- 172.21.15.70:0/3048984008 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e6742f20 0x55c2e671ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.285+0000 7f3594fd5740 1 --2- 172.21.15.70:0/3048984008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e67636e0 0x55c2e67612a0 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.285+0000 7f3594fd5740 1 -- 172.21.15.70:0/3048984008 >> 172.21.15.70:0/3048984008 conn(0x55c2e6717d70 msgr2=0x55c2e67608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.285+0000 7f3594fd5740 1 -- 172.21.15.70:0/3048984008 shutdown_connections 2023-10-08T07:14:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.285+0000 7f3594fd5740 1 -- 172.21.15.70:0/3048984008 wait complete. 2023-10-08T07:14:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.286+0000 7f3594fd5740 1 -- start start 2023-10-08T07:14:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.286+0000 7f3594fd5740 1 Processor -- start 2023-10-08T07:14:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.287+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e6742f20 0x55c2e675e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.287+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e67636e0 0x55c2e67409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.287+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.287+0000 7f3594fd5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e6620970 con 0x55c2e673e620 2023-10-08T07:14:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.287+0000 7f3594fd5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e6620610 con 0x55c2e67636e0 2023-10-08T07:14:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.287+0000 7f3594fd5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2e63e6f20 con 0x55c2e6742f20 2023-10-08T07:14:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.287+0000 7f3594fd5740 10 client.0 _ll_register_callbacks cb 0x55c2e665e610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:14:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.287+0000 7f3590909640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.287+0000 7f3590909640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40310/0 (socket says 172.21.15.70:40310) 2023-10-08T07:14:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.287+0000 7f3590909640 1 -- 172.21.15.70:0/951563992 learned_addr learned my addr 172.21.15.70:0/951563992 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f3590909640 1 -- 172.21.15.70:0/951563992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e6742f20 msgr2=0x55c2e675e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f3590909640 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e6742f20 0x55c2e675e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f3590909640 1 -- 172.21.15.70:0/951563992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e67636e0 msgr2=0x55c2e67409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f3590909640 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e67636e0 0x55c2e67409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f3590909640 1 -- 172.21.15.70:0/951563992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c00f040 con 0x55c2e673e620 2023-10-08T07:14:51.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f3590909640 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740f10 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f3584006fd0 tx=0x7f3584058120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b3baed129f1fb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:51.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f356affd640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:14:51.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f356affd640 1 -- 172.21.15.70:0/951563992 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358405fed0 con 0x55c2e673e620 2023-10-08T07:14:51.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f356affd640 1 -- 172.21.15.70:0/951563992 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f3584060030 con 0x55c2e673e620 2023-10-08T07:14:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f3594fd5740 1 -- 172.21.15.70:0/951563992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c2e667b6c0 con 0x55c2e673e620 2023-10-08T07:14:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f356affd640 1 -- 172.21.15.70:0/951563992 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3584087a80 con 0x55c2e673e620 2023-10-08T07:14:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f3594fd5740 10 client.12051 fetch_fsmap learned FSMap version 125 2023-10-08T07:14:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.288+0000 7f3594fd5740 1 -- 172.21.15.70:0/951563992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x55c2e6680d60 con 0x55c2e673e620 2023-10-08T07:14:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.289+0000 7f356affd640 1 -- 172.21.15.70:0/951563992 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c2e667b6c0 con 0x55c2e673e620 2023-10-08T07:14:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.289+0000 7f356affd640 1 -- 172.21.15.70:0/951563992 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 125) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3584087be0 con 0x55c2e673e620 2023-10-08T07:14:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.289+0000 7f3594fd5740 10 client.12051 fetch_fsmap finished waiting for FSMap version 125 2023-10-08T07:14:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.289+0000 7f3594fd5740 10 client.12051 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T07:14:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.289+0000 7f3594fd5740 20 client.12051 populate_metadata read hostname 'smithi070' 2023-10-08T07:14:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.289+0000 7f3594fd5740 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.70:6832/3856326680,v1:172.21.15.70:6834/3856326680] conn(0x55c2e6779080 0x55c2e6779460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.289+0000 7f3594fd5740 4 client.12051 mds_command: new command op to 11904 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T07:14:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.289+0000 7f3594fd5740 1 -- 172.21.15.70:0/951563992 --> [v2:172.21.15.70:6832/3856326680,v1:172.21.15.70:6834/3856326680] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2e667b3e0 con 0x55c2e6779080 2023-10-08T07:14:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.289+0000 7f358bfff640 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.70:6832/3856326680,v1:172.21.15.70:6834/3856326680] conn(0x55c2e6779080 0x55c2e6779460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.290+0000 7f358bfff640 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.70:6832/3856326680,v1:172.21.15.70:6834/3856326680] conn(0x55c2e6779080 0x55c2e6779460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f1339efd23ce5 server_cookie=8a46170bd60fe533 in_seq=0 out_seq=0 2023-10-08T07:14:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.290+0000 7f356affd640 10 client.12051 ms_handle_connect on v2:172.21.15.70:6832/3856326680 2023-10-08T07:14:51.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.290+0000 7f356affd640 1 -- 172.21.15.70:0/951563992 <== mds.0 v2:172.21.15.70:6832/3856326680 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c2e667b3e0 con 0x55c2e6779080 2023-10-08T07:14:51.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.290+0000 7f356affd640 10 client.12051 handle_command_reply: tid=0 2023-10-08T07:14:51.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.290+0000 7f3594fd5740 1 client.12051 shutdown 2023-10-08T07:14:51.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.290+0000 7f3594fd5740 1 -- 172.21.15.70:0/951563992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 msgr2=0x55c2e6740f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f3594fd5740 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740f10 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f3584006fd0 tx=0x7f3584058120 comp rx=0 tx=0).stop 2023-10-08T07:14:51.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f3594fd5740 1 -- 172.21.15.70:0/951563992 shutdown_connections 2023-10-08T07:14:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f3594fd5740 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.70:6832/3856326680,v1:172.21.15.70:6834/3856326680] conn(0x55c2e6779080 0x55c2e6779460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f3594fd5740 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740f10 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f3594fd5740 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e67636e0 0x55c2e67409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f3594fd5740 1 --2- 172.21.15.70:0/951563992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e6742f20 0x55c2e675e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f356affd640 0 client.12051 ms_handle_reset on v2:172.21.15.70:6832/3856326680 2023-10-08T07:14:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f3594fd5740 1 -- 172.21.15.70:0/951563992 >> 172.21.15.70:0/951563992 conn(0x55c2e671b150 msgr2=0x55c2e6714a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f3594fd5740 1 -- 172.21.15.70:0/951563992 shutdown_connections 2023-10-08T07:14:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f3594fd5740 1 -- 172.21.15.70:0/951563992 wait complete. 2023-10-08T07:14:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.291+0000 7f3594fd5740 20 client.12051 trim_cache size 0 max 16384 2023-10-08T07:14:51.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.307+0000 7f3594fd5740 1 Processor -- start 2023-10-08T07:14:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.307+0000 7f3594fd5740 1 -- start start 2023-10-08T07:14:51.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e6742f20 0x55c2e671a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e67636e0 0x55c2e6749430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f3594fd5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e673f7f0 con 0x55c2e673e620 2023-10-08T07:14:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f3594fd5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e675e190 con 0x55c2e6742f20 2023-10-08T07:14:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f3594fd5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2e671c9d0 con 0x55c2e67636e0 2023-10-08T07:14:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f358bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f358bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40322/0 (socket says 172.21.15.70:40322) 2023-10-08T07:14:51.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f358bfff640 1 -- 172.21.15.70:0/2118117694 learned_addr learned my addr 172.21.15.70:0/2118117694 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f358bfff640 1 -- 172.21.15.70:0/2118117694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e67636e0 msgr2=0x55c2e6749430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f358bfff640 1 --2- 172.21.15.70:0/2118117694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e67636e0 0x55c2e6749430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f358bfff640 1 -- 172.21.15.70:0/2118117694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e6742f20 msgr2=0x55c2e671a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.308+0000 7f358bfff640 1 --2- 172.21.15.70:0/2118117694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e6742f20 0x55c2e671a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.309+0000 7f358bfff640 1 -- 172.21.15.70:0/2118117694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2e661fb80 con 0x55c2e673e620 2023-10-08T07:14:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.309+0000 7f358bfff640 1 --2- 172.21.15.70:0/2118117694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740ba0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f357c00e610 tx=0x7f357c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a093729bde6d9b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.309+0000 7f35827fc640 1 -- 172.21.15.70:0/2118117694 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c02d0e0 con 0x55c2e673e620 2023-10-08T07:14:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.309+0000 7f35827fc640 1 -- 172.21.15.70:0/2118117694 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c02d240 con 0x55c2e673e620 2023-10-08T07:14:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.309+0000 7f35827fc640 1 -- 172.21.15.70:0/2118117694 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c02d5b0 con 0x55c2e673e620 2023-10-08T07:14:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.309+0000 7f3594fd5740 1 -- 172.21.15.70:0/2118117694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 msgr2=0x55c2e6740ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.309+0000 7f3594fd5740 1 --2- 172.21.15.70:0/2118117694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740ba0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f357c00e610 tx=0x7f357c015000 comp rx=0 tx=0).stop 2023-10-08T07:14:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.310+0000 7f3594fd5740 1 -- 172.21.15.70:0/2118117694 shutdown_connections 2023-10-08T07:14:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.310+0000 7f3594fd5740 1 --2- 172.21.15.70:0/2118117694 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e67636e0 0x55c2e6749430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.310+0000 7f3594fd5740 1 --2- 172.21.15.70:0/2118117694 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e6742f20 0x55c2e671a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.310+0000 7f3594fd5740 1 --2- 172.21.15.70:0/2118117694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e6740ba0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.310+0000 7f3594fd5740 1 -- 172.21.15.70:0/2118117694 >> 172.21.15.70:0/2118117694 conn(0x55c2e671b150 msgr2=0x55c2e6714e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.310+0000 7f3594fd5740 1 -- 172.21.15.70:0/2118117694 shutdown_connections 2023-10-08T07:14:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.310+0000 7f3594fd5740 1 -- 172.21.15.70:0/2118117694 wait complete. 2023-10-08T07:14:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.310+0000 7f3594fd5740 1 -- start start 2023-10-08T07:14:51.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.311+0000 7f3594fd5740 1 Processor -- start 2023-10-08T07:14:51.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.311+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e67414a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.311+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e6742f20 0x55c2e673d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f3594fd5740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e67636e0 0x55c2e673df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f3594fd5740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e6715c70 con 0x55c2e673e620 2023-10-08T07:14:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f3594fd5740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2e675f8e0 con 0x55c2e67636e0 2023-10-08T07:14:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f3594fd5740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c2e6764770 con 0x55c2e6742f20 2023-10-08T07:14:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f3594fd5740 10 client.0 _ll_register_callbacks cb 0x7f357c001690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:14:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f358bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e67414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f358bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e67414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40326/0 (socket says 172.21.15.70:40326) 2023-10-08T07:14:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f358bfff640 1 -- 172.21.15.70:0/2861808753 learned_addr learned my addr 172.21.15.70:0/2861808753 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f358bfff640 1 -- 172.21.15.70:0/2861808753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e6742f20 msgr2=0x55c2e673d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f358bfff640 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e6742f20 0x55c2e673d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f358bfff640 1 -- 172.21.15.70:0/2861808753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e67636e0 msgr2=0x55c2e673df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f358bfff640 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e67636e0 0x55c2e673df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f358bfff640 1 -- 172.21.15.70:0/2861808753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2e6646910 con 0x55c2e673e620 2023-10-08T07:14:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f358bfff640 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e67414a0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f357c001370 tx=0x7f357c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fb1e54251c5b27 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.312+0000 7f356affd640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:14:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f356affd640 1 -- 172.21.15.70:0/2861808753 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c02d5c0 con 0x55c2e673e620 2023-10-08T07:14:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f356affd640 1 -- 172.21.15.70:0/2861808753 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f357c02da60 con 0x55c2e673e620 2023-10-08T07:14:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f3594fd5740 1 -- 172.21.15.70:0/2861808753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f357c014d90 con 0x55c2e673e620 2023-10-08T07:14:51.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f356affd640 1 -- 172.21.15.70:0/2861808753 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=125) v2 ==== 24+0+0 (secure 0 0 0) 0x7f357c02dc20 con 0x55c2e673e620 2023-10-08T07:14:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f3594fd5740 10 client.12057 fetch_fsmap learned FSMap version 125 2023-10-08T07:14:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f3594fd5740 1 -- 172.21.15.70:0/2861808753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=125,monmap=2+}) v3 -- 0x55c2e6645170 con 0x55c2e673e620 2023-10-08T07:14:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f356affd640 1 -- 172.21.15.70:0/2861808753 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c014d90 con 0x55c2e673e620 2023-10-08T07:14:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f356affd640 1 -- 172.21.15.70:0/2861808753 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 125) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f357c035070 con 0x55c2e673e620 2023-10-08T07:14:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f3594fd5740 10 client.12057 fetch_fsmap finished waiting for FSMap version 125 2023-10-08T07:14:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f3594fd5740 10 client.12057 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T07:14:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f3594fd5740 20 client.12057 populate_metadata read hostname 'smithi070' 2023-10-08T07:14:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f3594fd5740 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.70:6832/3856326680,v1:172.21.15.70:6834/3856326680] conn(0x55c2e6714d40 0x55c2e6715120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f3594fd5740 4 client.12057 mds_command: new command op to 11904 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T07:14:51.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f3594fd5740 1 -- 172.21.15.70:0/2861808753 --> [v2:172.21.15.70:6832/3856326680,v1:172.21.15.70:6834/3856326680] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c2e63846c0 con 0x55c2e6714d40 2023-10-08T07:14:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.313+0000 7f35837fe640 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.70:6832/3856326680,v1:172.21.15.70:6834/3856326680] conn(0x55c2e6714d40 0x55c2e6715120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f35837fe640 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.70:6832/3856326680,v1:172.21.15.70:6834/3856326680] conn(0x55c2e6714d40 0x55c2e6715120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671e40e849f72d14 server_cookie=511b9bee12c73697 in_seq=0 out_seq=0 2023-10-08T07:14:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f356affd640 10 client.12057 ms_handle_connect on v2:172.21.15.70:6832/3856326680 2023-10-08T07:14:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f356affd640 1 -- 172.21.15.70:0/2861808753 <== mds.0 v2:172.21.15.70:6832/3856326680 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55c2e63846c0 con 0x55c2e6714d40 2023-10-08T07:14:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f356affd640 10 client.12057 handle_command_reply: tid=0 2023-10-08T07:14:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f3594fd5740 1 client.12057 shutdown 2023-10-08T07:14:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f3594fd5740 1 -- 172.21.15.70:0/2861808753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 msgr2=0x55c2e67414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f3594fd5740 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e67414a0 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f357c001370 tx=0x7f357c000bc0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f3594fd5740 1 -- 172.21.15.70:0/2861808753 shutdown_connections 2023-10-08T07:14:51.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f3594fd5740 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.70:6832/3856326680,v1:172.21.15.70:6834/3856326680] conn(0x55c2e6714d40 0x55c2e6715120 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f3594fd5740 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x55c2e67636e0 0x55c2e673df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f3594fd5740 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x55c2e6742f20 0x55c2e673d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f356affd640 0 client.12057 ms_handle_reset on v2:172.21.15.70:6832/3856326680 2023-10-08T07:14:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f3594fd5740 1 --2- 172.21.15.70:0/2861808753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c2e673e620 0x55c2e67414a0 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.314+0000 7f3594fd5740 1 -- 172.21.15.70:0/2861808753 >> 172.21.15.70:0/2861808753 conn(0x55c2e671b150 msgr2=0x55c2e67605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f3594fd5740 1 -- 172.21.15.70:0/2861808753 shutdown_connections 2023-10-08T07:14:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f3594fd5740 1 -- 172.21.15.70:0/2861808753 wait complete. 2023-10-08T07:14:51.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f3594fd5740 20 client.12057 trim_cache size 0 max 16384 2023-10-08T07:14:51.323 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:14:51.323 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T07:14:51.323 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T07:14:51.323 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 11904, 2023-10-08T07:14:51.323 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T07:14:51.323 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T07:14:51.323 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T07:14:51.323 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.9339885609999996, 2023-10-08T07:14:51.323 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 125, 2023-10-08T07:14:51.323 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 254, 2023-10-08T07:14:51.324 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 254, 2023-10-08T07:14:51.324 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 12.342777336999999 2023-10-08T07:14:51.324 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:14:51.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 -- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f356c06b0b0 msgr2=0x7f356c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f356c06b0b0 0x7f356c06d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f357c001050 tx=0x7f357c001170 comp rx=0 tx=0).stop 2023-10-08T07:14:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 -- 172.21.15.70:0/2274926384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f358c0c4ca0 msgr2=0x7f358c15ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f358c0c4ca0 0x7f358c15ed80 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7f358c15b4a0 tx=0x7f358400e830 comp rx=0 tx=0).stop 2023-10-08T07:14:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 -- 172.21.15.70:0/2274926384 shutdown_connections 2023-10-08T07:14:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f356c06b0b0 0x7f356c06d570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f358c0c4ca0 0x7f358c15ed80 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f358c15e0c0 0x7f358c15bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 --2- 172.21.15.70:0/2274926384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3578003fb0 0x7f358c15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 -- 172.21.15.70:0/2274926384 >> 172.21.15.70:0/2274926384 conn(0x7f358c06a5d0 msgr2=0x7f358c0ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:51.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.315+0000 7f359236c640 1 -- 172.21.15.70:0/2274926384 shutdown_connections 2023-10-08T07:14:51.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.316+0000 7f359236c640 1 -- 172.21.15.70:0/2274926384 wait complete. 2023-10-08T07:14:51.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T07:14:51.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.544+0000 7fcd1f5c5640 1 Processor -- start 2023-10-08T07:14:51.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.544+0000 7fcd1f5c5640 1 -- start start 2023-10-08T07:14:51.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.546+0000 7fcd1f5c5640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcd1815ab40 0x7fcd180ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.546+0000 7fcd1f5c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd180c90b0 0x7fcd180adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.546+0000 7fcd1f5c5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1815b2b0 con 0x7fcd180cae70 2023-10-08T07:14:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.546+0000 7fcd1f5c5640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1815bf30 con 0x7fcd1815ab40 2023-10-08T07:14:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.546+0000 7fcd1f5c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1815f7a0 con 0x7fcd180c90b0 2023-10-08T07:14:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.546+0000 7fcd1db62640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcd180cae70 0x7fcd180702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46518/0 (socket says 172.21.15.70:46518) 2023-10-08T07:14:51.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.546+0000 7fcd1db62640 1 -- 172.21.15.70:0/4046717065 learned_addr learned my addr 172.21.15.70:0/4046717065 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.546+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308750494 0 0) 0x7fcd1815b2b0 con 0x7fcd180cae70 2023-10-08T07:14:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd1d361640 1 --2- 172.21.15.70:0/4046717065 >> v2:172.21.15.181:3300/0 conn(0x7fcd1815ab40 0x7fcd180ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd00003520 con 0x7fcd180cae70 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd1cb60640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd180c90b0 0x7fcd180adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 129956243 0 0) 0x7fcd00003520 con 0x7fcd180cae70 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd180c90b0 msgr2=0x7fcd180adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd180c90b0 0x7fcd180adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 >> v2:172.21.15.181:3300/0 conn(0x7fcd1815ab40 msgr2=0x7fcd180ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 --2- 172.21.15.70:0/4046717065 >> v2:172.21.15.181:3300/0 conn(0x7fcd1815ab40 0x7fcd180ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1815b590 con 0x7fcd180cae70 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fcd10001670 con 0x7fcd180cae70 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 >> v1:172.21.15.70:6789/0 conn(0x7fcd180cae70 legacy=0x7fcd180702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd00003970 0x7fcd00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd000047c0 0x7fcd00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd180c90b0 0x7fcd00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1815bf30 con 0x7fcd000047c0 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1815f7a0 con 0x7fcd00003970 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0000ba80 con 0x7fcd180c90b0 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.547+0000 7fcd1cb60640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd000047c0 0x7fcd00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.548+0000 7fcd1d361640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd00003970 0x7fcd00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.548+0000 7fcd1cb60640 1 -- 172.21.15.70:0/4046717065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd180c90b0 msgr2=0x7fcd00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.548+0000 7fcd1cb60640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd180c90b0 0x7fcd00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.548+0000 7fcd1cb60640 1 -- 172.21.15.70:0/4046717065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd00003970 msgr2=0x7fcd00003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.548+0000 7fcd1cb60640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd00003970 0x7fcd00003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.548+0000 7fcd1cb60640 1 -- 172.21.15.70:0/4046717065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00003520 con 0x7fcd000047c0 2023-10-08T07:14:51.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.549+0000 7fcd1cb60640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd000047c0 0x7fcd00004c40 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00ba00 tx=0x7fcd000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678b786a4cf0bf18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:51.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.549+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c032620 con 0x7fcd000047c0 2023-10-08T07:14:51.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.549+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c032780 con 0x7fcd000047c0 2023-10-08T07:14:51.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.549+0000 7fcd17fff640 1 -- 172.21.15.70:0/4046717065 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c032b10 con 0x7fcd000047c0 2023-10-08T07:14:51.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.550+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/4046717065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd000047c0 msgr2=0x7fcd00004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.550+0000 7fcd1f5c5640 1 --2- 172.21.15.70:0/4046717065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd000047c0 0x7fcd00004c40 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00ba00 tx=0x7fcd000057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.550+0000 7fcd1db62640 1 -- 172.21.15.70:0/4046717065 reap_dead start 2023-10-08T07:14:51.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.550+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/4046717065 shutdown_connections 2023-10-08T07:14:51.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.550+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/4046717065 >> 172.21.15.70:0/4046717065 conn(0x7fcd1806a380 msgr2=0x7fcd180b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:51.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.550+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/4046717065 shutdown_connections 2023-10-08T07:14:51.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.550+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/4046717065 wait complete. 2023-10-08T07:14:51.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.552+0000 7fcd1f5c5640 1 Processor -- start 2023-10-08T07:14:51.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.552+0000 7fcd1f5c5640 1 -- start start 2023-10-08T07:14:51.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1f5c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd000047c0 0x7fcd180c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1f5c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd18152350 0x7fcd181f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1f5c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd180b3f40 0x7fcd180c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1f5c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181527d0 con 0x7fcd180b3f40 2023-10-08T07:14:51.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1f5c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181f1280 con 0x7fcd18152350 2023-10-08T07:14:51.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1f5c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180c6120 con 0x7fcd000047c0 2023-10-08T07:14:51.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1db62640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd180b3f40 0x7fcd180c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1db62640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd180b3f40 0x7fcd180c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40338/0 (socket says 172.21.15.70:40338) 2023-10-08T07:14:51.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1db62640 1 -- 172.21.15.70:0/3670141752 learned_addr learned my addr 172.21.15.70:0/3670141752 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:51.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1db62640 1 -- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd000047c0 msgr2=0x7fcd180c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1d361640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd000047c0 0x7fcd180c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.553+0000 7fcd1cb60640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd18152350 0x7fcd181f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.554+0000 7fcd1db62640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd000047c0 0x7fcd180c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.554+0000 7fcd1db62640 1 -- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd18152350 msgr2=0x7fcd181f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.554+0000 7fcd1db62640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd18152350 0x7fcd181f0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.554+0000 7fcd1db62640 1 -- 172.21.15.70:0/3670141752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd180b3180 con 0x7fcd180b3f40 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.554+0000 7fcd1db62640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd180b3f40 0x7fcd180c7ca0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7fcd1806e3f0 tx=0x7fcd1000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.554+0000 7fcd15ffb640 1 -- 172.21.15.70:0/3670141752 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10004bf0 con 0x7fcd180b3f40 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.554+0000 7fcd15ffb640 1 -- 172.21.15.70:0/3670141752 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10004d50 con 0x7fcd180b3f40 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.554+0000 7fcd15ffb640 1 -- 172.21.15.70:0/3670141752 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10007270 con 0x7fcd180b3f40 2023-10-08T07:14:51.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.554+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/3670141752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd180b3460 con 0x7fcd180b3f40 2023-10-08T07:14:51.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.554+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/3670141752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd180b3a20 con 0x7fcd180b3f40 2023-10-08T07:14:51.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.555+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/3670141752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd18067490 con 0x7fcd180b3f40 2023-10-08T07:14:51.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.556+0000 7fcd15ffb640 1 -- 172.21.15.70:0/3670141752 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcd100073d0 con 0x7fcd180b3f40 2023-10-08T07:14:51.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.557+0000 7fcd15ffb640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd0406b0e0 0x7fcd0406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:51.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.557+0000 7fcd15ffb640 1 -- 172.21.15.70:0/3670141752 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fcd10088cc0 con 0x7fcd180b3f40 2023-10-08T07:14:51.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.558+0000 7fcd1d361640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd0406b0e0 0x7fcd0406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:51.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.559+0000 7fcd1d361640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd0406b0e0 0x7fcd0406d5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009e70 tx=0x7fcd0800b040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:51.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.562+0000 7fcd15ffb640 1 -- 172.21.15.70:0/3670141752 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd1008f080 con 0x7fcd180b3f40 2023-10-08T07:14:51.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.790+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/3670141752 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcd1815b840 con 0x7fcd180b3f40 2023-10-08T07:14:51.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.790+0000 7fcd15ffb640 1 -- 172.21.15.70:0/3670141752 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v125) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcd100537b0 con 0x7fcd180b3f40 2023-10-08T07:14:51.792 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T07:14:51.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd0406b0e0 msgr2=0x7fcd0406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd0406b0e0 0x7fcd0406d5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009e70 tx=0x7fcd0800b040 comp rx=0 tx=0).stop 2023-10-08T07:14:51.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/3670141752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd180b3f40 msgr2=0x7fcd180c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:51.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd180b3f40 0x7fcd180c7ca0 secure :-1 s=READY pgs=2660 cs=0 l=1 rev1=1 crypto rx=0x7fcd1806e3f0 tx=0x7fcd1000c8b0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/3670141752 shutdown_connections 2023-10-08T07:14:51.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd0406b0e0 0x7fcd0406d5a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd180b3f40 0x7fcd180c7ca0 unknown :-1 s=CLOSED pgs=2660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd18152350 0x7fcd181f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 --2- 172.21.15.70:0/3670141752 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd000047c0 0x7fcd180c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:51.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/3670141752 >> 172.21.15.70:0/3670141752 conn(0x7fcd1806a380 msgr2=0x7fcd180c9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:51.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/3670141752 shutdown_connections 2023-10-08T07:14:51.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:51.793+0000 7fcd1f5c5640 1 -- 172.21.15.70:0/3670141752 wait complete. 2023-10-08T07:14:51.815 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T07:14:51.815 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T07:14:51.815 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T07:14:51.815 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T07:14:51.815 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T07:14:51.815 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T07:14:51.815 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T07:14:51.815 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T07:14:51.815 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:14:51.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:51.816 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T07:14:51.871 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:14:51.872 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 36505sec preferred_lft 36505sec 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T07:14:51.873 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:14:51.874 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T07:14:51.874 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:14:51.874 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T07:14:51.874 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T07:14:51.874 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T07:14:51.874 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:14:51.874 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T07:14:51.874 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T07:14:51.874 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T07:14:51.874 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:14:51.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:51.875 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T07:14:51.965 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T07:14:51.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:51.969 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T07:14:52.022 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T07:14:52.022 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T07:14:52.022 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T07:14:52.022 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T07:14:52.022 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T07:14:52.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:52.023 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T07:14:52.023 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T07:14:52.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T07:14:52.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T07:14:52.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T07:14:52.023 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T07:14:52.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:52.212 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T07:14:52.265 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T07:14:52.265 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T07:14:52.266 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:14:52.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:52.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T07:14:52.366 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T07:14:52.366 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T07:14:52.417 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:14:52.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T07:14:52.417 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T07:14:52.464 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T07:14:52.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:52.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:14:52.604 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:14:52.604 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T07:14:52.635 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:14:52.635 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T07:14:52.635 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T07:14:52.635 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T07:14:52.635 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T07:14:52.637 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T07:14:52.637 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T07:14:52.637 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T07:14:52.637 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T07:14:52.637 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T07:14:52.641 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T07:14:52.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:52.641 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T07:14:52.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:14:52.728 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T07:14:52.802 INFO:teuthology.orchestra.run.smithi070.stdout:client12066 2023-10-08T07:14:52.803 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:14:52.803 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:14:52.860 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T07:14:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.077+0000 7fe5a6743640 1 Processor -- start 2023-10-08T07:14:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.077+0000 7fe5a6743640 1 -- start start 2023-10-08T07:14:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.079+0000 7fe5a6743640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe5a015cd80 0x7fe5a015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.079+0000 7fe5a6743640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a00d06c0 0x7fe5a015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.079+0000 7fe5a6743640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe5a00d0ec0 0x7fe5a00d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.079+0000 7fe5a6743640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a015d160 con 0x7fe5a015cd80 2023-10-08T07:14:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.079+0000 7fe5a6743640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a00d3800 con 0x7fe5a00d0ec0 2023-10-08T07:14:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.079+0000 7fe5a6743640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00d1850 con 0x7fe5a00d06c0 2023-10-08T07:14:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.079+0000 7fe5a5741640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe5a015cd80 0x7fe5a015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.079+0000 7fe5a5741640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe5a015cd80 0x7fe5a015dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40354/0 (socket says 172.21.15.70:40354) 2023-10-08T07:14:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.079+0000 7fe5a5741640 1 -- 172.21.15.70:0/1291099866 learned_addr learned my addr 172.21.15.70:0/1291099866 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.079+0000 7fe5a4f40640 1 --2- 172.21.15.70:0/1291099866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a00d06c0 0x7fe5a015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.080+0000 7fe5a5f42640 1 --2- 172.21.15.70:0/1291099866 >> v2:172.21.15.181:3300/0 conn(0x7fe5a00d0ec0 0x7fe5a00d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.080+0000 7fe5a5741640 1 -- 172.21.15.70:0/1291099866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a00d06c0 msgr2=0x7fe5a015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.080+0000 7fe5a5741640 1 --2- 172.21.15.70:0/1291099866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a00d06c0 0x7fe5a015e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.080+0000 7fe5a5741640 1 -- 172.21.15.70:0/1291099866 >> v2:172.21.15.181:3300/0 conn(0x7fe5a00d0ec0 msgr2=0x7fe5a00d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:53.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.080+0000 7fe5a5741640 1 --2- 172.21.15.70:0/1291099866 >> v2:172.21.15.181:3300/0 conn(0x7fe5a00d0ec0 0x7fe5a00d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.080+0000 7fe5a5741640 1 -- 172.21.15.70:0/1291099866 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00ce370 con 0x7fe5a015cd80 2023-10-08T07:14:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.080+0000 7fe5a5f42640 1 --2- 172.21.15.70:0/1291099866 >> v2:172.21.15.181:3300/0 conn(0x7fe5a00d0ec0 0x7fe5a00d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.081+0000 7fe5a5741640 1 --2- 172.21.15.70:0/1291099866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 0x7fe5a015dcf0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00ccec0 tx=0x7fe59c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cae95b621161684 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.081+0000 7fe597fff640 1 -- 172.21.15.70:0/1291099866 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c02e070 con 0x7fe5a015cd80 2023-10-08T07:14:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.081+0000 7fe597fff640 1 -- 172.21.15.70:0/1291099866 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c004030 con 0x7fe5a015cd80 2023-10-08T07:14:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.081+0000 7fe597fff640 1 -- 172.21.15.70:0/1291099866 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004420 con 0x7fe5a015cd80 2023-10-08T07:14:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.082+0000 7fe5a6743640 1 -- 172.21.15.70:0/1291099866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 msgr2=0x7fe5a015dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.082+0000 7fe5a6743640 1 --2- 172.21.15.70:0/1291099866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 0x7fe5a015dcf0 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00ccec0 tx=0x7fe59c02b840 comp rx=0 tx=0).stop 2023-10-08T07:14:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.082+0000 7fe5a6743640 1 -- 172.21.15.70:0/1291099866 shutdown_connections 2023-10-08T07:14:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.082+0000 7fe5a6743640 1 --2- 172.21.15.70:0/1291099866 >> v2:172.21.15.181:3300/0 conn(0x7fe5a00d0ec0 0x7fe5a00d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.082+0000 7fe5a6743640 1 --2- 172.21.15.70:0/1291099866 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a00d06c0 0x7fe5a015e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.082+0000 7fe5a6743640 1 --2- 172.21.15.70:0/1291099866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 0x7fe5a015dcf0 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.082+0000 7fe5a6743640 1 -- 172.21.15.70:0/1291099866 >> 172.21.15.70:0/1291099866 conn(0x7fe5a009bc40 msgr2=0x7fe5a00d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.082+0000 7fe5a6743640 1 -- 172.21.15.70:0/1291099866 shutdown_connections 2023-10-08T07:14:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.082+0000 7fe5a6743640 1 -- 172.21.15.70:0/1291099866 wait complete. 2023-10-08T07:14:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.084+0000 7fe5a6743640 1 Processor -- start 2023-10-08T07:14:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.085+0000 7fe5a6743640 1 -- start start 2023-10-08T07:14:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.086+0000 7fe5a6743640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a00d06c0 0x7fe5a01f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.086+0000 7fe5a6743640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00d0ec0 0x7fe5a015af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.086+0000 7fe5a6743640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 0x7fe5a01f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.086+0000 7fe5a6743640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a01f6df0 con 0x7fe5a015cd80 2023-10-08T07:14:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.086+0000 7fe5a6743640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a01f2860 con 0x7fe5a00d0ec0 2023-10-08T07:14:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.086+0000 7fe5a6743640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a01f6830 con 0x7fe5a00d06c0 2023-10-08T07:14:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.086+0000 7fe5a5f42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 0x7fe5a01f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.086+0000 7fe5a5f42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 0x7fe5a01f5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40358/0 (socket says 172.21.15.70:40358) 2023-10-08T07:14:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.086+0000 7fe5a5f42640 1 -- 172.21.15.70:0/3653432954 learned_addr learned my addr 172.21.15.70:0/3653432954 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe5a4f40640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00d0ec0 0x7fe5a015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe5a5741640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a00d06c0 0x7fe5a01f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe5a5f42640 1 -- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a00d06c0 msgr2=0x7fe5a01f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe5a5f42640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a00d06c0 0x7fe5a01f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe5a5f42640 1 -- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00d0ec0 msgr2=0x7fe5a015af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe5a5f42640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00d0ec0 0x7fe5a015af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe5a5f42640 1 -- 172.21.15.70:0/3653432954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c009460 con 0x7fe5a015cd80 2023-10-08T07:14:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe5a5f42640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 0x7fe5a01f5e50 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7fe59800cef0 tx=0x7fe59800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe595ffb640 1 -- 172.21.15.70:0/3653432954 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59800d840 con 0x7fe5a015cd80 2023-10-08T07:14:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe595ffb640 1 -- 172.21.15.70:0/3653432954 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59800d9c0 con 0x7fe5a015cd80 2023-10-08T07:14:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe5a6743640 1 -- 172.21.15.70:0/3653432954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a01f6b10 con 0x7fe5a015cd80 2023-10-08T07:14:53.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.087+0000 7fe595ffb640 1 -- 172.21.15.70:0/3653432954 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59800dd70 con 0x7fe5a015cd80 2023-10-08T07:14:53.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.088+0000 7fe5a6743640 1 -- 172.21.15.70:0/3653432954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a01f2d50 con 0x7fe5a015cd80 2023-10-08T07:14:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.089+0000 7fe5a6743640 1 -- 172.21.15.70:0/3653432954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a01f2f70 con 0x7fe5a015cd80 2023-10-08T07:14:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.090+0000 7fe595ffb640 1 -- 172.21.15.70:0/3653432954 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fe59801e3b0 con 0x7fe5a015cd80 2023-10-08T07:14:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.091+0000 7fe595ffb640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe58406b0b0 0x7fe58406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.091+0000 7fe595ffb640 1 -- 172.21.15.70:0/3653432954 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7fe59808a740 con 0x7fe5a015cd80 2023-10-08T07:14:53.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.091+0000 7fe5a5741640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe58406b0b0 0x7fe58406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.093+0000 7fe5a5741640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe58406b0b0 0x7fe58406d570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe59c034000 tx=0x7fe59c008030 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.096+0000 7fe595ffb640 1 -- 172.21.15.70:0/3653432954 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5980552f0 con 0x7fe5a015cd80 2023-10-08T07:14:53.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.325+0000 7fe5a6743640 1 -- 172.21.15.70:0/3653432954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe5a0074150 con 0x7fe5a015cd80 2023-10-08T07:14:53.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.445+0000 7fe595ffb640 1 -- 172.21.15.70:0/3653432954 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v126) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe598058910 con 0x7fe5a015cd80 2023-10-08T07:14:53.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.448+0000 7fe5a6743640 1 -- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe58406b0b0 msgr2=0x7fe58406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:53.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.448+0000 7fe5a6743640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe58406b0b0 0x7fe58406d570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe59c034000 tx=0x7fe59c008030 comp rx=0 tx=0).stop 2023-10-08T07:14:53.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.448+0000 7fe5a6743640 1 -- 172.21.15.70:0/3653432954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 msgr2=0x7fe5a01f5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:53.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.448+0000 7fe5a6743640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 0x7fe5a01f5e50 secure :-1 s=READY pgs=2663 cs=0 l=1 rev1=1 crypto rx=0x7fe59800cef0 tx=0x7fe59800a710 comp rx=0 tx=0).stop 2023-10-08T07:14:53.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.449+0000 7fe5a6743640 1 -- 172.21.15.70:0/3653432954 shutdown_connections 2023-10-08T07:14:53.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.449+0000 7fe5a6743640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fe58406b0b0 0x7fe58406d570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.449+0000 7fe5a6743640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5a015cd80 0x7fe5a01f5e50 unknown :-1 s=CLOSED pgs=2663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.449+0000 7fe5a6743640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe5a00d0ec0 0x7fe5a015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.449+0000 7fe5a6743640 1 --2- 172.21.15.70:0/3653432954 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe5a00d06c0 0x7fe5a01f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.449+0000 7fe5a6743640 1 -- 172.21.15.70:0/3653432954 >> 172.21.15.70:0/3653432954 conn(0x7fe5a009bc40 msgr2=0x7fe5a00caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:53.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.449+0000 7fe5a6743640 1 -- 172.21.15.70:0/3653432954 shutdown_connections 2023-10-08T07:14:53.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.449+0000 7fe5a6743640 1 -- 172.21.15.70:0/3653432954 wait complete. 2023-10-08T07:14:53.470 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:53.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.681+0000 7f198d475640 1 Processor -- start 2023-10-08T07:14:53.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.681+0000 7f198d475640 1 -- start start 2023-10-08T07:14:53.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.683+0000 7f198d475640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1988159ac0 0x7f198806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.683+0000 7f198d475640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19880ad790 0x7f19880702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.683+0000 7f198d475640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198806dee0 con 0x7f198815e330 2023-10-08T07:14:53.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.683+0000 7f198d475640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198815d850 con 0x7f19880ad790 2023-10-08T07:14:53.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.683+0000 7f198d475640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198815db30 con 0x7f1988159ac0 2023-10-08T07:14:53.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.683+0000 7f19867fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19880ad790 0x7f19880702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.683+0000 7f19867fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19880ad790 0x7f19880702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58096/0 (socket says 172.21.15.70:58096) 2023-10-08T07:14:53.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.683+0000 7f19867fc640 1 -- 172.21.15.70:0/4106929479 learned_addr learned my addr 172.21.15.70:0/4106929479 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:53.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.683+0000 7f1986ffd640 1 --2- 172.21.15.70:0/4106929479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1988159ac0 0x7f198806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.684+0000 7f19867fc640 1 -- 172.21.15.70:0/4106929479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1988159ac0 msgr2=0x7f198806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:53.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.684+0000 7f19867fc640 1 --2- 172.21.15.70:0/4106929479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1988159ac0 0x7f198806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.684+0000 7f19867fc640 1 -- 172.21.15.70:0/4106929479 >> v1:172.21.15.181:6789/0 conn(0x7f198815e330 legacy=0x7f198806c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:53.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.684+0000 7f19867fc640 1 -- 172.21.15.70:0/4106929479 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198815de10 con 0x7f19880ad790 2023-10-08T07:14:53.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.685+0000 7f19867fc640 1 --2- 172.21.15.70:0/4106929479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19880ad790 0x7f19880702d0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f1970009870 tx=0x7f197002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ed8ef29cd1c0b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:53.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.685+0000 7f1985ffb640 1 -- 172.21.15.70:0/4106929479 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970031640 con 0x7f19880ad790 2023-10-08T07:14:53.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.685+0000 7f1985ffb640 1 -- 172.21.15.70:0/4106929479 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19700317a0 con 0x7f19880ad790 2023-10-08T07:14:53.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.685+0000 7f1985ffb640 1 -- 172.21.15.70:0/4106929479 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970031b30 con 0x7f19880ad790 2023-10-08T07:14:53.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.686+0000 7f198d475640 1 -- 172.21.15.70:0/4106929479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19880ad790 msgr2=0x7f19880702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:53.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.686+0000 7f198d475640 1 --2- 172.21.15.70:0/4106929479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19880ad790 0x7f19880702d0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7f1970009870 tx=0x7f197002b8f0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.686+0000 7f198d475640 1 -- 172.21.15.70:0/4106929479 shutdown_connections 2023-10-08T07:14:53.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.686+0000 7f198d475640 1 --2- 172.21.15.70:0/4106929479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19880ad790 0x7f19880702d0 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.686+0000 7f198d475640 1 --2- 172.21.15.70:0/4106929479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1988159ac0 0x7f198806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.686+0000 7f198d475640 1 -- 172.21.15.70:0/4106929479 >> 172.21.15.70:0/4106929479 conn(0x7f198806a380 msgr2=0x7f19880b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:53.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.686+0000 7f198d475640 1 -- 172.21.15.70:0/4106929479 shutdown_connections 2023-10-08T07:14:53.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.686+0000 7f198d475640 1 -- 172.21.15.70:0/4106929479 wait complete. 2023-10-08T07:14:53.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.688+0000 7f198d475640 1 Processor -- start 2023-10-08T07:14:53.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.688+0000 7f198d475640 1 -- start start 2023-10-08T07:14:53.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f198d475640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880ad790 0x7f1988149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f198d475640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1988159ac0 0x7f198815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f198d475640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198815e330 0x7f19880cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f198d475640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198815bbf0 con 0x7f198815e330 2023-10-08T07:14:53.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f198d475640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198815bed0 con 0x7f19880ad790 2023-10-08T07:14:53.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f198d475640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198815c1b0 con 0x7f1988159ac0 2023-10-08T07:14:53.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f19877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198815e330 0x7f19880cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f19877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198815e330 0x7f19880cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58106/0 (socket says 172.21.15.70:58106) 2023-10-08T07:14:53.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f19877fe640 1 -- 172.21.15.70:0/310979444 learned_addr learned my addr 172.21.15.70:0/310979444 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:53.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f19867fc640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1988159ac0 0x7f198815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.689+0000 7f1986ffd640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880ad790 0x7f1988149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f19877fe640 1 -- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1988159ac0 msgr2=0x7f198815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:53.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f19877fe640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1988159ac0 0x7f198815b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f19877fe640 1 -- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880ad790 msgr2=0x7f1988149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:53.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f19877fe640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880ad790 0x7f1988149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f19877fe640 1 -- 172.21.15.70:0/310979444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19700094d0 con 0x7f198815e330 2023-10-08T07:14:53.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f1986ffd640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880ad790 0x7f1988149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:53.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f19877fe640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198815e330 0x7f19880cd6d0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f198000b970 tx=0x7f198000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:53.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f197f7fe640 1 -- 172.21.15.70:0/310979444 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198000cb40 con 0x7f198815e330 2023-10-08T07:14:53.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f197f7fe640 1 -- 172.21.15.70:0/310979444 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198000cca0 con 0x7f198815e330 2023-10-08T07:14:53.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f197f7fe640 1 -- 172.21.15.70:0/310979444 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1980014680 con 0x7f198815e330 2023-10-08T07:14:53.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f198d475640 1 -- 172.21.15.70:0/310979444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198815c490 con 0x7f198815e330 2023-10-08T07:14:53.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.690+0000 7f198d475640 1 -- 172.21.15.70:0/310979444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1988149e70 con 0x7f198815e330 2023-10-08T07:14:53.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.691+0000 7f198d475640 1 -- 172.21.15.70:0/310979444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198815c9b0 con 0x7f198815e330 2023-10-08T07:14:53.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.692+0000 7f197f7fe640 1 -- 172.21.15.70:0/310979444 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f198001d3b0 con 0x7f198815e330 2023-10-08T07:14:53.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.693+0000 7f197f7fe640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f196806b0b0 0x7f196806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:53.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.693+0000 7f197f7fe640 1 -- 172.21.15.70:0/310979444 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f1980089ad0 con 0x7f198815e330 2023-10-08T07:14:53.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.694+0000 7f1986ffd640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f196806b0b0 0x7f196806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:53.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.695+0000 7f1986ffd640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f196806b0b0 0x7f196806d570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1978006fd0 tx=0x7f1978008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:53.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.697+0000 7f197f7fe640 1 -- 172.21.15.70:0/310979444 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f198815c9b0 con 0x7f198815e330 2023-10-08T07:14:53.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.890+0000 7f198d475640 1 -- 172.21.15.70:0/310979444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f198815c9b0 con 0x7f198815e330 2023-10-08T07:14:53.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.893+0000 7f197f7fe640 1 -- 172.21.15.70:0/310979444 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) v1 ==== 74+0+16958 (secure 0 0 0) 0x7f198815c9b0 con 0x7f198815e330 2023-10-08T07:14:53.895 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:53.895 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":254,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:45.298385+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:53.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 -- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f196806b0b0 msgr2=0x7f196806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:53.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f196806b0b0 0x7f196806d570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1978006fd0 tx=0x7f1978008040 comp rx=0 tx=0).stop 2023-10-08T07:14:53.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 -- 172.21.15.70:0/310979444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198815e330 msgr2=0x7f19880cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:53.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198815e330 0x7f19880cd6d0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7f198000b970 tx=0x7f198000be30 comp rx=0 tx=0).stop 2023-10-08T07:14:53.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 -- 172.21.15.70:0/310979444 shutdown_connections 2023-10-08T07:14:53.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f196806b0b0 0x7f196806d570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198815e330 0x7f19880cd6d0 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1988159ac0 0x7f198815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 --2- 172.21.15.70:0/310979444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19880ad790 0x7f1988149930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:53.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 -- 172.21.15.70:0/310979444 >> 172.21.15.70:0/310979444 conn(0x7f198806a380 msgr2=0x7f19880cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:53.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 -- 172.21.15.70:0/310979444 shutdown_connections 2023-10-08T07:14:53.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:53.896+0000 7f198d475640 1 -- 172.21.15.70:0/310979444 wait complete. 2023-10-08T07:14:53.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T07:14:41.606127+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T07:14:42.867820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '254', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:14:53.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:14:54.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.130+0000 7f19aac90640 1 Processor -- start 2023-10-08T07:14:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.130+0000 7f19aac90640 1 -- start start 2023-10-08T07:14:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.131+0000 7f19aac90640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f19a40cb880 0x7f19a40b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.132+0000 7f19aac90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a40b4070 0x7f19a4159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.132+0000 7f19aac90640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a415d440 con 0x7f19a41432f0 2023-10-08T07:14:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.132+0000 7f19aac90640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a41436d0 con 0x7f19a40cb880 2023-10-08T07:14:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.132+0000 7f19aac90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a415d690 con 0x7f19a40b4070 2023-10-08T07:14:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.132+0000 7f19a8a2c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f19a41432f0 0x7f19a40cb170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52738/0 (socket says 172.21.15.70:52738) 2023-10-08T07:14:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.132+0000 7f19a8a2c640 1 -- 172.21.15.70:0/2093981371 learned_addr learned my addr 172.21.15.70:0/2093981371 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.132+0000 7f19a3fff640 1 --2- 172.21.15.70:0/2093981371 >> v2:172.21.15.181:3300/0 conn(0x7f19a40cb880 0x7f19a40b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.132+0000 7f19a922d640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a40b4070 0x7f19a4159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.132+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947641634 0 0) 0x7f19a415d440 con 0x7f19a41432f0 2023-10-08T07:14:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.132+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f198c003520 con 0x7f19a41432f0 2023-10-08T07:14:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 62846836 0 0) 0x7f198c003520 con 0x7f19a41432f0 2023-10-08T07:14:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a40b4070 msgr2=0x7f19a4159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a40b4070 0x7f19a4159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 >> v2:172.21.15.181:3300/0 conn(0x7f19a40cb880 msgr2=0x7f19a40b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 --2- 172.21.15.70:0/2093981371 >> v2:172.21.15.181:3300/0 conn(0x7f19a40cb880 0x7f19a40b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a415d970 con 0x7f19a41432f0 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1994001670 con 0x7f19a41432f0 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 >> v1:172.21.15.70:6789/0 conn(0x7f19a41432f0 legacy=0x7f19a40cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a40b4070 0x7f198c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c003f20 0x7f198c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 0x7f198c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41436d0 con 0x7f198c004d00 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a415d690 con 0x7f198c003f20 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c00fe10 con 0x7f19a40b4070 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.133+0000 7f19a922d640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 0x7f198c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.134+0000 7f19a3fff640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c003f20 0x7f198c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.134+0000 7f19a922d640 1 -- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a40b4070 msgr2=0x7f198c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.134+0000 7f19a922d640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a40b4070 0x7f198c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.134+0000 7f19a922d640 1 -- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c003f20 msgr2=0x7f198c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.134+0000 7f19a922d640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c003f20 0x7f198c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.134+0000 7f19a922d640 1 -- 172.21.15.70:0/2093981371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c003520 con 0x7f198c004d00 2023-10-08T07:14:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.134+0000 7f19a3fff640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f198c003f20 0x7f198c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.135+0000 7f19a922d640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 0x7f198c005180 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f199c009ac0 tx=0x7f198c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a696b58546763f83 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.135+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c02c710 con 0x7f198c004d00 2023-10-08T07:14:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.135+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c02c870 con 0x7f198c004d00 2023-10-08T07:14:54.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.135+0000 7f19a37fe640 1 -- 172.21.15.70:0/2093981371 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c02cc00 con 0x7f198c004d00 2023-10-08T07:14:54.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.135+0000 7f19aac90640 1 -- 172.21.15.70:0/2093981371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 msgr2=0x7f198c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.135+0000 7f19aac90640 1 --2- 172.21.15.70:0/2093981371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 0x7f198c005180 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7f199c009ac0 tx=0x7f198c0057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.135+0000 7f19a922d640 1 -- 172.21.15.70:0/2093981371 reap_dead start 2023-10-08T07:14:54.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.136+0000 7f19aac90640 1 -- 172.21.15.70:0/2093981371 shutdown_connections 2023-10-08T07:14:54.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.136+0000 7f19aac90640 1 -- 172.21.15.70:0/2093981371 >> 172.21.15.70:0/2093981371 conn(0x7f19a4069d50 msgr2=0x7f19a406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:54.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.136+0000 7f19aac90640 1 -- 172.21.15.70:0/2093981371 shutdown_connections 2023-10-08T07:14:54.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.136+0000 7f19aac90640 1 -- 172.21.15.70:0/2093981371 wait complete. 2023-10-08T07:14:54.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.137+0000 7f19aac90640 1 Processor -- start 2023-10-08T07:14:54.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.138+0000 7f19aac90640 1 -- start start 2023-10-08T07:14:54.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.138+0000 7f19aac90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 0x7f19a41f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.138+0000 7f19aac90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a4139220 0x7f19a413ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.138+0000 7f19aac90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19a4159120 0x7f19a40cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.138+0000 7f19aac90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41f2390 con 0x7f198c004d00 2023-10-08T07:14:54.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.138+0000 7f19aac90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a41f1440 con 0x7f19a4159120 2023-10-08T07:14:54.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.138+0000 7f19aac90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a413d410 con 0x7f19a4139220 2023-10-08T07:14:54.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.139+0000 7f19a8a2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 0x7f19a41f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.139+0000 7f19a8a2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 0x7f19a41f1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58122/0 (socket says 172.21.15.70:58122) 2023-10-08T07:14:54.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.139+0000 7f19a8a2c640 1 -- 172.21.15.70:0/511179375 learned_addr learned my addr 172.21.15.70:0/511179375 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.139+0000 7f19a8a2c640 1 -- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a4139220 msgr2=0x7f19a413ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.139+0000 7f19a8a2c640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a4139220 0x7f19a413ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.139+0000 7f19a8a2c640 1 -- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19a4159120 msgr2=0x7f19a40cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.139+0000 7f19a8a2c640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19a4159120 0x7f19a40cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.139+0000 7f19a8a2c640 1 -- 172.21.15.70:0/511179375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a4139990 con 0x7f198c004d00 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.139+0000 7f19a8a2c640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 0x7f19a41f1e50 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f199400c790 tx=0x7f199400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.140+0000 7f19a17fa640 1 -- 172.21.15.70:0/511179375 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199400cea0 con 0x7f198c004d00 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.140+0000 7f19a17fa640 1 -- 172.21.15.70:0/511179375 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19940074e0 con 0x7f198c004d00 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.140+0000 7f19aac90640 1 -- 172.21.15.70:0/511179375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a4139c70 con 0x7f198c004d00 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.140+0000 7f19a17fa640 1 -- 172.21.15.70:0/511179375 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994004a20 con 0x7f198c004d00 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.140+0000 7f19aac90640 1 -- 172.21.15.70:0/511179375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a41f1930 con 0x7f198c004d00 2023-10-08T07:14:54.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.141+0000 7f19aac90640 1 -- 172.21.15.70:0/511179375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a41f1b50 con 0x7f198c004d00 2023-10-08T07:14:54.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.142+0000 7f19a17fa640 1 -- 172.21.15.70:0/511179375 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1994004b80 con 0x7f198c004d00 2023-10-08T07:14:54.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.143+0000 7f19a17fa640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f199006b0e0 0x7f199006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.144+0000 7f19a17fa640 1 -- 172.21.15.70:0/511179375 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f1994088ff0 con 0x7f198c004d00 2023-10-08T07:14:54.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.144+0000 7f19a3fff640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f199006b0e0 0x7f199006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.145+0000 7f19a3fff640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f199006b0e0 0x7f199006d5a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19a40b0840 tx=0x7f1998009750 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:54.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.148+0000 7f19a17fa640 1 -- 172.21.15.70:0/511179375 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1994053ae0 con 0x7f198c004d00 2023-10-08T07:14:54.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.338+0000 7f19aac90640 1 -- 172.21.15.70:0/511179375 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19a404e160 con 0x7f198c004d00 2023-10-08T07:14:54.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.338+0000 7f19a17fa640 1 -- 172.21.15.70:0/511179375 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v254) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1994018020 con 0x7f198c004d00 2023-10-08T07:14:54.340 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:14:54.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 -- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f199006b0e0 msgr2=0x7f199006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f199006b0e0 0x7f199006d5a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f19a40b0840 tx=0x7f1998009750 comp rx=0 tx=0).stop 2023-10-08T07:14:54.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 -- 172.21.15.70:0/511179375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 msgr2=0x7f19a41f1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 0x7f19a41f1e50 secure :-1 s=READY pgs=2668 cs=0 l=1 rev1=1 crypto rx=0x7f199400c790 tx=0x7f199400cc50 comp rx=0 tx=0).stop 2023-10-08T07:14:54.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 -- 172.21.15.70:0/511179375 shutdown_connections 2023-10-08T07:14:54.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f199006b0e0 0x7f199006d5a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f19a4159120 0x7f19a40cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f19a4139220 0x7f19a413ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 --2- 172.21.15.70:0/511179375 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f198c004d00 0x7f19a41f1e50 unknown :-1 s=CLOSED pgs=2668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 -- 172.21.15.70:0/511179375 >> 172.21.15.70:0/511179375 conn(0x7f19a4069d50 msgr2=0x7f19a406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:54.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.341+0000 7f19aac90640 1 -- 172.21.15.70:0/511179375 shutdown_connections 2023-10-08T07:14:54.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.342+0000 7f19aac90640 1 -- 172.21.15.70:0/511179375 wait complete. 2023-10-08T07:14:54.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:14:54.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.573+0000 7f08c7d9e640 1 Processor -- start 2023-10-08T07:14:54.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.574+0000 7f08c7d9e640 1 -- start start 2023-10-08T07:14:54.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.575+0000 7f08c7d9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c00afbe0 0x7f08c00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.575+0000 7f08c7d9e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c00b2620 con 0x7f08c006dfc0 2023-10-08T07:14:54.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.575+0000 7f08c7d9e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c00b2060 con 0x7f08c00cb130 2023-10-08T07:14:54.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.575+0000 7f08c7d9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00b2340 con 0x7f08c00afbe0 2023-10-08T07:14:54.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.575+0000 7f08c5339640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f08c006dfc0 0x7f08c0070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52752/0 (socket says 172.21.15.70:52752) 2023-10-08T07:14:54.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.575+0000 7f08c5339640 1 -- 172.21.15.70:0/277407607 learned_addr learned my addr 172.21.15.70:0/277407607 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:54.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c5b3a640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c00afbe0 0x7f08c00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064487631 0 0) 0x7f08c00b2620 con 0x7f08c006dfc0 2023-10-08T07:14:54.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ac003520 con 0x7f08c006dfc0 2023-10-08T07:14:54.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4098228377 0 0) 0x7f08ac003520 con 0x7f08c006dfc0 2023-10-08T07:14:54.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c00afbe0 msgr2=0x7f08c00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:54.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c00afbe0 0x7f08c00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 >> v1:172.21.15.181:6789/0 conn(0x7f08c00cb130 legacy=0x7f08c006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c0070350 con 0x7f08c006dfc0 2023-10-08T07:14:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f08b0001670 con 0x7f08c006dfc0 2023-10-08T07:14:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 >> v1:172.21.15.70:6789/0 conn(0x7f08c006dfc0 legacy=0x7f08c0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ac0039a0 0x7f08ac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac0047c0 0x7f08ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c00afbe0 0x7f08ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00b2340 con 0x7f08ac0047c0 2023-10-08T07:14:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac00ba60 con 0x7f08ac0039a0 2023-10-08T07:14:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.576+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac00bc20 con 0x7f08c00afbe0 2023-10-08T07:14:54.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.577+0000 7f08c5339640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac0047c0 0x7f08ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.577+0000 7f08c5b3a640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ac0039a0 0x7f08ac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.577+0000 7f08c633b640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c00afbe0 0x7f08ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.577+0000 7f08c5339640 1 -- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c00afbe0 msgr2=0x7f08ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:54.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.577+0000 7f08c5339640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c00afbe0 0x7f08ac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.577+0000 7f08c5339640 1 -- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ac0039a0 msgr2=0x7f08ac003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:54.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.577+0000 7f08c5339640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ac0039a0 0x7f08ac003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.577+0000 7f08c5339640 1 -- 172.21.15.70:0/277407607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac003520 con 0x7f08ac0047c0 2023-10-08T07:14:54.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.577+0000 7f08c5b3a640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ac0039a0 0x7f08ac003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:54.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.578+0000 7f08c5339640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac0047c0 0x7f08ac004c40 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f08b0007fb0 tx=0x7f08ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc44e39eac66f745 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:54.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.578+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0005ad0 con 0x7f08ac0047c0 2023-10-08T07:14:54.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.578+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b0005c30 con 0x7f08ac0047c0 2023-10-08T07:14:54.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.578+0000 7f08c4b38640 1 -- 172.21.15.70:0/277407607 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b002c790 con 0x7f08ac0047c0 2023-10-08T07:14:54.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.579+0000 7f08c7d9e640 1 -- 172.21.15.70:0/277407607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac0047c0 msgr2=0x7f08ac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.579+0000 7f08c7d9e640 1 --2- 172.21.15.70:0/277407607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac0047c0 0x7f08ac004c40 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f08b0007fb0 tx=0x7f08ac0057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.579+0000 7f08c633b640 1 -- 172.21.15.70:0/277407607 reap_dead start 2023-10-08T07:14:54.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.579+0000 7f08c7d9e640 1 -- 172.21.15.70:0/277407607 shutdown_connections 2023-10-08T07:14:54.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.579+0000 7f08c7d9e640 1 -- 172.21.15.70:0/277407607 >> 172.21.15.70:0/277407607 conn(0x7f08c006a5d0 msgr2=0x7f08c00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:54.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.579+0000 7f08c7d9e640 1 -- 172.21.15.70:0/277407607 shutdown_connections 2023-10-08T07:14:54.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.579+0000 7f08c7d9e640 1 -- 172.21.15.70:0/277407607 wait complete. 2023-10-08T07:14:54.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.581+0000 7f08c7d9e640 1 Processor -- start 2023-10-08T07:14:54.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.581+0000 7f08c7d9e640 1 -- start start 2023-10-08T07:14:54.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.582+0000 7f08c7d9e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ac0047c0 0x7f08c0152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.582+0000 7f08c7d9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c0153c70 0x7f08c014fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.582+0000 7f08c7d9e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c00c4bb0 0x7f08c01509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.582+0000 7f08c7d9e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0150230 con 0x7f08c00c4bb0 2023-10-08T07:14:54.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.582+0000 7f08c7d9e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c00c3500 con 0x7f08ac0047c0 2023-10-08T07:14:54.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.582+0000 7f08c7d9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c0153060 con 0x7f08c0153c70 2023-10-08T07:14:54.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.582+0000 7f08c633b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c00c4bb0 0x7f08c01509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c633b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c00c4bb0 0x7f08c01509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58146/0 (socket says 172.21.15.70:58146) 2023-10-08T07:14:54.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c633b640 1 -- 172.21.15.70:0/952879231 learned_addr learned my addr 172.21.15.70:0/952879231 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c5339640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c0153c70 0x7f08c014fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c5b3a640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ac0047c0 0x7f08c0152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c633b640 1 -- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c0153c70 msgr2=0x7f08c014fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c633b640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c0153c70 0x7f08c014fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c633b640 1 -- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ac0047c0 msgr2=0x7f08c0152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c633b640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ac0047c0 0x7f08c0152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c633b640 1 -- 172.21.15.70:0/952879231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c0153340 con 0x7f08c00c4bb0 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c5339640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c0153c70 0x7f08c014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c633b640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c00c4bb0 0x7f08c01509f0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cb510 tx=0x7f08b800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08b67fc640 1 -- 172.21.15.70:0/952879231 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800eed0 con 0x7f08c00c4bb0 2023-10-08T07:14:54.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08b67fc640 1 -- 172.21.15.70:0/952879231 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b8003970 con 0x7f08c00c4bb0 2023-10-08T07:14:54.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08c7d9e640 1 -- 172.21.15.70:0/952879231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c0153620 con 0x7f08c00c4bb0 2023-10-08T07:14:54.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.583+0000 7f08b67fc640 1 -- 172.21.15.70:0/952879231 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8003d20 con 0x7f08c00c4bb0 2023-10-08T07:14:54.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.584+0000 7f08c7d9e640 1 -- 172.21.15.70:0/952879231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c00c5210 con 0x7f08c00c4bb0 2023-10-08T07:14:54.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.585+0000 7f08c7d9e640 1 -- 172.21.15.70:0/952879231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c0154050 con 0x7f08c00c4bb0 2023-10-08T07:14:54.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.586+0000 7f08b67fc640 1 -- 172.21.15.70:0/952879231 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f08b80050d0 con 0x7f08c00c4bb0 2023-10-08T07:14:54.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.586+0000 7f08b67fc640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08a406b0e0 0x7f08a406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:54.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.587+0000 7f08b67fc640 1 -- 172.21.15.70:0/952879231 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f08b8089a20 con 0x7f08c00c4bb0 2023-10-08T07:14:54.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.587+0000 7f08c5b3a640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08a406b0e0 0x7f08a406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:54.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.588+0000 7f08c5b3a640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08a406b0e0 0x7f08a406d5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0099b0 tx=0x7f08bc008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:54.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.591+0000 7f08b67fc640 1 -- 172.21.15.70:0/952879231 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08c0154050 con 0x7f08c00c4bb0 2023-10-08T07:14:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.780+0000 7f08c7d9e640 1 -- 172.21.15.70:0/952879231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08c0154050 con 0x7f08c00c4bb0 2023-10-08T07:14:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.781+0000 7f08b67fc640 1 -- 172.21.15.70:0/952879231 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v254) v1 ==== 104+0+11 (secure 0 0 0) 0x7f08c0154050 con 0x7f08c00c4bb0 2023-10-08T07:14:54.783 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:14:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 -- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08a406b0e0 msgr2=0x7f08a406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08a406b0e0 0x7f08a406d5a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0099b0 tx=0x7f08bc008040 comp rx=0 tx=0).stop 2023-10-08T07:14:54.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 -- 172.21.15.70:0/952879231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c00c4bb0 msgr2=0x7f08c01509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c00c4bb0 0x7f08c01509f0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cb510 tx=0x7f08b800eb90 comp rx=0 tx=0).stop 2023-10-08T07:14:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 -- 172.21.15.70:0/952879231 shutdown_connections 2023-10-08T07:14:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f08a406b0e0 0x7f08a406d5a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c00c4bb0 0x7f08c01509f0 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f08c0153c70 0x7f08c014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 --2- 172.21.15.70:0/952879231 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f08ac0047c0 0x7f08c0152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 -- 172.21.15.70:0/952879231 >> 172.21.15.70:0/952879231 conn(0x7f08c006a5d0 msgr2=0x7f08c00cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 -- 172.21.15.70:0/952879231 shutdown_connections 2023-10-08T07:14:54.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:54.784+0000 7f08c7d9e640 1 -- 172.21.15.70:0/952879231 wait complete. 2023-10-08T07:14:54.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:14:55.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.012+0000 7f23bcf0f640 1 Processor -- start 2023-10-08T07:14:55.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.012+0000 7f23bcf0f640 1 -- start start 2023-10-08T07:14:55.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.014+0000 7f23bcf0f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f23b8160660 0x7f23b8070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.014+0000 7f23bcf0f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b806e150 con 0x7f23b80cb620 2023-10-08T07:14:55.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.014+0000 7f23bcf0f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b806de70 con 0x7f23b80c9910 2023-10-08T07:14:55.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.014+0000 7f23bcf0f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b80c8e30 con 0x7f23b8160660 2023-10-08T07:14:55.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.014+0000 7f23b6d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f23b80cb620 0x7f23b80b1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52758/0 (socket says 172.21.15.70:52758) 2023-10-08T07:14:55.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.014+0000 7f23b6d9d640 1 -- 172.21.15.70:0/1424147813 learned_addr learned my addr 172.21.15.70:0/1424147813 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:55.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b659c640 1 --2- 172.21.15.70:0/1424147813 >> v2:172.21.15.181:3300/0 conn(0x7f23b8160660 0x7f23b8070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553892292 0 0) 0x7f23b806e150 con 0x7f23b80cb620 2023-10-08T07:14:55.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a4003520 con 0x7f23b80cb620 2023-10-08T07:14:55.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3323855449 0 0) 0x7f23a4003520 con 0x7f23b80cb620 2023-10-08T07:14:55.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 >> v2:172.21.15.181:3300/0 conn(0x7f23b8160660 msgr2=0x7f23b8070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:55.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 --2- 172.21.15.70:0/1424147813 >> v2:172.21.15.181:3300/0 conn(0x7f23b8160660 0x7f23b8070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 >> v1:172.21.15.181:6789/0 conn(0x7f23b80c9910 legacy=0x7f23b80caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80c9110 con 0x7f23b80cb620 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f23b00019f0 con 0x7f23b80cb620 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 >> v1:172.21.15.70:6789/0 conn(0x7f23b80cb620 legacy=0x7f23b80b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 --2- 172.21.15.70:0/1424147813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23a40039c0 0x7f23a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 --2- 172.21.15.70:0/1424147813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 --2- 172.21.15.70:0/1424147813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23a40051f0 0x7f23a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b80c8e30 con 0x7f23a40047c0 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a400c150 con 0x7f23a40051f0 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.015+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a400c310 con 0x7f23a40039c0 2023-10-08T07:14:55.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.016+0000 7f23affff640 1 --2- 172.21.15.70:0/1424147813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.016+0000 7f23b659c640 1 --2- 172.21.15.70:0/1424147813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23a40039c0 0x7f23a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.016+0000 7f23affff640 1 -- 172.21.15.70:0/1424147813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23a40039c0 msgr2=0x7f23a4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:55.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.016+0000 7f23affff640 1 --2- 172.21.15.70:0/1424147813 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23a40039c0 0x7f23a4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.016+0000 7f23affff640 1 -- 172.21.15.70:0/1424147813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23a40051f0 msgr2=0x7f23a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:55.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.016+0000 7f23affff640 1 --2- 172.21.15.70:0/1424147813 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23a40051f0 0x7f23a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.016+0000 7f23affff640 1 -- 172.21.15.70:0/1424147813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a4003520 con 0x7f23a40047c0 2023-10-08T07:14:55.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.016+0000 7f23b6d9d640 1 -- 172.21.15.70:0/1424147813 reap_dead start 2023-10-08T07:14:55.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.017+0000 7f23affff640 1 --2- 172.21.15.70:0/1424147813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23a4004c40 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f23a0001750 tx=0x7f23a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3198b03d7be5d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:55.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.017+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a00052d0 con 0x7f23a40047c0 2023-10-08T07:14:55.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.018+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a002e050 con 0x7f23a40047c0 2023-10-08T07:14:55.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.018+0000 7f23b5d9b640 1 -- 172.21.15.70:0/1424147813 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a0003830 con 0x7f23a40047c0 2023-10-08T07:14:55.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.018+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1424147813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 msgr2=0x7f23a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.018+0000 7f23bcf0f640 1 --2- 172.21.15.70:0/1424147813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23a4004c40 secure :-1 s=READY pgs=2673 cs=0 l=1 rev1=1 crypto rx=0x7f23a0001750 tx=0x7f23a40057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.018+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1424147813 shutdown_connections 2023-10-08T07:14:55.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.018+0000 7f23bcf0f640 1 --2- 172.21.15.70:0/1424147813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23a4004c40 unknown :-1 s=CLOSED pgs=2673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.018+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1424147813 >> 172.21.15.70:0/1424147813 conn(0x7f23b806a5d0 msgr2=0x7f23b80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:55.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.018+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1424147813 shutdown_connections 2023-10-08T07:14:55.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.018+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1424147813 wait complete. 2023-10-08T07:14:55.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.020+0000 7f23bcf0f640 1 Processor -- start 2023-10-08T07:14:55.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.020+0000 7f23bcf0f640 1 -- start start 2023-10-08T07:14:55.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.021+0000 7f23bcf0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23b81f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.021+0000 7f23bcf0f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23b81f3740 0x7f23b8153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.021+0000 7f23bcf0f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8157b80 0x7f23b815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.021+0000 7f23bcf0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b81f4b10 con 0x7f23a40047c0 2023-10-08T07:14:55.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.021+0000 7f23bcf0f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b81f3f00 con 0x7f23b81f3740 2023-10-08T07:14:55.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.021+0000 7f23bcf0f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b81f41e0 con 0x7f23b8157b80 2023-10-08T07:14:55.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.021+0000 7f23b659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23b81f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.021+0000 7f23b659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23b81f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58172/0 (socket says 172.21.15.70:58172) 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.021+0000 7f23b659c640 1 -- 172.21.15.70:0/1157845247 learned_addr learned my addr 172.21.15.70:0/1157845247 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23b659c640 1 -- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8157b80 msgr2=0x7f23b815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23b6d9d640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8157b80 0x7f23b815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23affff640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23b81f3740 0x7f23b8153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23b659c640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8157b80 0x7f23b815e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23b659c640 1 -- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23b81f3740 msgr2=0x7f23b8153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23b659c640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23b81f3740 0x7f23b8153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23b659c640 1 -- 172.21.15.70:0/1157845247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b81f44c0 con 0x7f23a40047c0 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23b659c640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23b81f3170 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f23a80116d0 tx=0x7f23a8011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23aeffd640 1 -- 172.21.15.70:0/1157845247 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a800f9c0 con 0x7f23a40047c0 2023-10-08T07:14:55.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23aeffd640 1 -- 172.21.15.70:0/1157845247 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a800fb20 con 0x7f23a40047c0 2023-10-08T07:14:55.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23aeffd640 1 -- 172.21.15.70:0/1157845247 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a800fe90 con 0x7f23a40047c0 2023-10-08T07:14:55.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1157845247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b81f47a0 con 0x7f23a40047c0 2023-10-08T07:14:55.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.022+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1157845247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b81f3bc0 con 0x7f23a40047c0 2023-10-08T07:14:55.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.023+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1157845247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b804e160 con 0x7f23a40047c0 2023-10-08T07:14:55.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.024+0000 7f23aeffd640 1 -- 172.21.15.70:0/1157845247 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f23a8015460 con 0x7f23a40047c0 2023-10-08T07:14:55.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.025+0000 7f23aeffd640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f239806b0b0 0x7f239806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.025+0000 7f23aeffd640 1 -- 172.21.15.70:0/1157845247 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f23a808a1e0 con 0x7f23a40047c0 2023-10-08T07:14:55.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.026+0000 7f23affff640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f239806b0b0 0x7f239806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.027+0000 7f23affff640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f239806b0b0 0x7f239806d570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f23a0033000 tx=0x7f23a0009030 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:55.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.030+0000 7f23aeffd640 1 -- 172.21.15.70:0/1157845247 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23a8054d60 con 0x7f23a40047c0 2023-10-08T07:14:55.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.218+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1157845247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23b80cd0f0 con 0x7f23a40047c0 2023-10-08T07:14:55.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.219+0000 7f23aeffd640 1 -- 172.21.15.70:0/1157845247 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v254) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23a8058380 con 0x7f23a40047c0 2023-10-08T07:14:55.220 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:14:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f239806b0b0 msgr2=0x7f239806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f239806b0b0 0x7f239806d570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f23a0033000 tx=0x7f23a0009030 comp rx=0 tx=0).stop 2023-10-08T07:14:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1157845247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 msgr2=0x7f23b81f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23b81f3170 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f23a80116d0 tx=0x7f23a8011b90 comp rx=0 tx=0).stop 2023-10-08T07:14:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1157845247 shutdown_connections 2023-10-08T07:14:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f239806b0b0 0x7f239806d570 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f23b8157b80 0x7f23b815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23b81f3740 0x7f23b8153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 --2- 172.21.15.70:0/1157845247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23a40047c0 0x7f23b81f3170 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1157845247 >> 172.21.15.70:0/1157845247 conn(0x7f23b806a5d0 msgr2=0x7f23b80cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:55.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1157845247 shutdown_connections 2023-10-08T07:14:55.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.222+0000 7f23bcf0f640 1 -- 172.21.15.70:0/1157845247 wait complete. 2023-10-08T07:14:55.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T07:14:55.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.451+0000 7f8d877b6640 1 Processor -- start 2023-10-08T07:14:55.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.451+0000 7f8d877b6640 1 -- start start 2023-10-08T07:14:55.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.452+0000 7f8d877b6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f8d800c90b0 0x7f8d800cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.452+0000 7f8d877b6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8015b2b0 con 0x7f8d800cb3b0 2023-10-08T07:14:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.452+0000 7f8d877b6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8015b590 con 0x7f8d8015ab40 2023-10-08T07:14:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.452+0000 7f8d877b6640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8015bf30 con 0x7f8d800c90b0 2023-10-08T07:14:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.453+0000 7f8d85d53640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8d800cb3b0 0x7f8d8015e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52762/0 (socket says 172.21.15.70:52762) 2023-10-08T07:14:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.453+0000 7f8d85d53640 1 -- 172.21.15.70:0/3586551215 learned_addr learned my addr 172.21.15.70:0/3586551215 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.453+0000 7f8d84d51640 1 --2- 172.21.15.70:0/3586551215 >> v2:172.21.15.181:3300/0 conn(0x7f8d800c90b0 0x7f8d800cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.453+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338732525 0 0) 0x7f8d8015b2b0 con 0x7f8d800cb3b0 2023-10-08T07:14:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.453+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d6c003520 con 0x7f8d800cb3b0 2023-10-08T07:14:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3531635427 0 0) 0x7f8d6c003520 con 0x7f8d800cb3b0 2023-10-08T07:14:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 >> v2:172.21.15.181:3300/0 conn(0x7f8d800c90b0 msgr2=0x7f8d800cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 --2- 172.21.15.70:0/3586551215 >> v2:172.21.15.181:3300/0 conn(0x7f8d800c90b0 0x7f8d800cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 >> v1:172.21.15.181:6789/0 conn(0x7f8d8015ab40 legacy=0x7f8d800ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8015b870 con 0x7f8d800cb3b0 2023-10-08T07:14:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f8d780019f0 con 0x7f8d800cb3b0 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 >> v1:172.21.15.70:6789/0 conn(0x7f8d800cb3b0 legacy=0x7f8d8015e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 --2- 172.21.15.70:0/3586551215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d6c003970 0x7f8d6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 --2- 172.21.15.70:0/3586551215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d6c0047c0 0x7f8d6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 --2- 172.21.15.70:0/3586551215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8015bf30 con 0x7f8d6c005180 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c00c020 con 0x7f8d6c0047c0 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c00c1e0 con 0x7f8d6c003970 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d85d53640 1 --2- 172.21.15.70:0/3586551215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d85d53640 1 -- 172.21.15.70:0/3586551215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d6c003970 msgr2=0x7f8d6c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d84d51640 1 --2- 172.21.15.70:0/3586551215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d6c0047c0 0x7f8d6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d85d53640 1 --2- 172.21.15.70:0/3586551215 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d6c003970 0x7f8d6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d85d53640 1 -- 172.21.15.70:0/3586551215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d6c0047c0 msgr2=0x7f8d6c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:55.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d85d53640 1 --2- 172.21.15.70:0/3586551215 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d6c0047c0 0x7f8d6c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.454+0000 7f8d85d53640 1 -- 172.21.15.70:0/3586551215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c003520 con 0x7f8d6c005180 2023-10-08T07:14:55.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.455+0000 7f8d85d53640 1 -- 172.21.15.70:0/3586551215 reap_dead start 2023-10-08T07:14:55.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.456+0000 7f8d85d53640 1 --2- 172.21.15.70:0/3586551215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d6c00b980 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800a830 tx=0x7f8d6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85585b32c2388f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.456+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d780072d0 con 0x7f8d6c005180 2023-10-08T07:14:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.456+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7802f070 con 0x7f8d6c005180 2023-10-08T07:14:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.456+0000 7f8d77fff640 1 -- 172.21.15.70:0/3586551215 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d780076b0 con 0x7f8d6c005180 2023-10-08T07:14:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.456+0000 7f8d877b6640 1 -- 172.21.15.70:0/3586551215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 msgr2=0x7f8d6c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.456+0000 7f8d877b6640 1 --2- 172.21.15.70:0/3586551215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d6c00b980 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800a830 tx=0x7f8d6c0057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.457+0000 7f8d877b6640 1 -- 172.21.15.70:0/3586551215 shutdown_connections 2023-10-08T07:14:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.457+0000 7f8d877b6640 1 --2- 172.21.15.70:0/3586551215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d6c00b980 unknown :-1 s=CLOSED pgs=2676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.457+0000 7f8d877b6640 1 -- 172.21.15.70:0/3586551215 >> 172.21.15.70:0/3586551215 conn(0x7f8d8006a380 msgr2=0x7f8d800b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.457+0000 7f8d877b6640 1 -- 172.21.15.70:0/3586551215 shutdown_connections 2023-10-08T07:14:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.457+0000 7f8d877b6640 1 -- 172.21.15.70:0/3586551215 wait complete. 2023-10-08T07:14:55.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.459+0000 7f8d877b6640 1 Processor -- start 2023-10-08T07:14:55.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.459+0000 7f8d877b6640 1 -- start start 2023-10-08T07:14:55.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d877b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d80142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d877b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d801f2ea0 0x7f8d800ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d877b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d800ceb00 0x7f8d801f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d877b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d801f21d0 con 0x7f8d6c005180 2023-10-08T07:14:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d877b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d800cef80 con 0x7f8d800ceb00 2023-10-08T07:14:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d877b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d801f6790 con 0x7f8d801f2ea0 2023-10-08T07:14:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d85552640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d80142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d85552640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d80142150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58196/0 (socket says 172.21.15.70:58196) 2023-10-08T07:14:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d85552640 1 -- 172.21.15.70:0/144749731 learned_addr learned my addr 172.21.15.70:0/144749731 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d85552640 1 -- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d801f2ea0 msgr2=0x7f8d800ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d85552640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d801f2ea0 0x7f8d800ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d85552640 1 -- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d800ceb00 msgr2=0x7f8d801f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d85552640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d800ceb00 0x7f8d801f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.460+0000 7f8d85552640 1 -- 172.21.15.70:0/144749731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d801f64b0 con 0x7f8d6c005180 2023-10-08T07:14:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.461+0000 7f8d85552640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d80142150 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00c530 tx=0x7f8d7c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.461+0000 7f8d75ffb640 1 -- 172.21.15.70:0/144749731 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c0052d0 con 0x7f8d6c005180 2023-10-08T07:14:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.461+0000 7f8d75ffb640 1 -- 172.21.15.70:0/144749731 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c005430 con 0x7f8d6c005180 2023-10-08T07:14:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.461+0000 7f8d75ffb640 1 -- 172.21.15.70:0/144749731 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c012870 con 0x7f8d6c005180 2023-10-08T07:14:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.461+0000 7f8d877b6640 1 -- 172.21.15.70:0/144749731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d801f23c0 con 0x7f8d6c005180 2023-10-08T07:14:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.461+0000 7f8d877b6640 1 -- 172.21.15.70:0/144749731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d801f2980 con 0x7f8d6c005180 2023-10-08T07:14:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.462+0000 7f8d877b6640 1 -- 172.21.15.70:0/144749731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d80067490 con 0x7f8d6c005180 2023-10-08T07:14:55.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.463+0000 7f8d75ffb640 1 -- 172.21.15.70:0/144749731 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f8d7c0129d0 con 0x7f8d6c005180 2023-10-08T07:14:55.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.465+0000 7f8d75ffb640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f8d6406b0b0 0x7f8d6406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.465+0000 7f8d75ffb640 1 -- 172.21.15.70:0/144749731 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f8d7c089720 con 0x7f8d6c005180 2023-10-08T07:14:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.465+0000 7f8d84d51640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f8d6406b0b0 0x7f8d6406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.466+0000 7f8d84d51640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f8d6406b0b0 0x7f8d6406d570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8d800b4560 tx=0x7f8d70008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:55.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.469+0000 7f8d75ffb640 1 -- 172.21.15.70:0/144749731 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8d7c0542a0 con 0x7f8d6c005180 2023-10-08T07:14:55.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.661+0000 7f8d877b6640 1 -- 172.21.15.70:0/144749731 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8d800b0ef0 con 0x7f8d6c005180 2023-10-08T07:14:55.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.662+0000 7f8d75ffb640 1 -- 172.21.15.70:0/144749731 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v254) v1 ==== 84+0+209 (secure 0 0 0) 0x7f8d7c0578c0 con 0x7f8d6c005180 2023-10-08T07:14:55.664 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:55.664 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T07:14:55.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:55.664 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T07:14:55.664 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T07:14:55.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:55.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:55.664 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 31, 2023-10-08T07:14:55.664 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "cephfs_metadata" 2023-10-08T07:14:55.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:14:55.665 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:14:55.665 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 32, 2023-10-08T07:14:55.665 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "cephfs_data" 2023-10-08T07:14:55.665 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:14:55.665 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T07:14:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.664+0000 7f8d877b6640 1 -- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f8d6406b0b0 msgr2=0x7f8d6406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.664+0000 7f8d877b6640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f8d6406b0b0 0x7f8d6406d570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8d800b4560 tx=0x7f8d70008040 comp rx=0 tx=0).stop 2023-10-08T07:14:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.664+0000 7f8d877b6640 1 -- 172.21.15.70:0/144749731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 msgr2=0x7f8d80142150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.664+0000 7f8d877b6640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d80142150 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c00c530 tx=0x7f8d7c00c9f0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.665+0000 7f8d877b6640 1 -- 172.21.15.70:0/144749731 shutdown_connections 2023-10-08T07:14:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.665+0000 7f8d877b6640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f8d6406b0b0 0x7f8d6406d570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.665+0000 7f8d877b6640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8d800ceb00 0x7f8d801f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.665+0000 7f8d877b6640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8d801f2ea0 0x7f8d800ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.665+0000 7f8d877b6640 1 --2- 172.21.15.70:0/144749731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c005180 0x7f8d80142150 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.665+0000 7f8d877b6640 1 -- 172.21.15.70:0/144749731 >> 172.21.15.70:0/144749731 conn(0x7f8d8006a380 msgr2=0x7f8d8006d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:55.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.665+0000 7f8d877b6640 1 -- 172.21.15.70:0/144749731 shutdown_connections 2023-10-08T07:14:55.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.665+0000 7f8d877b6640 1 -- 172.21.15.70:0/144749731 wait complete. 2023-10-08T07:14:55.685 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-10-08T07:14:55.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-10-08T07:14:55.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.906+0000 7f73678d0640 1 Processor -- start 2023-10-08T07:14:55.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.907+0000 7f73678d0640 1 -- start start 2023-10-08T07:14:55.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.908+0000 7f73678d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f736015ce00 0x7f73600cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.908+0000 7f73678d0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73600cd470 0x7f73600cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.908+0000 7f73678d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73600cde10 con 0x7f7360160980 2023-10-08T07:14:55.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.908+0000 7f73678d0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7360161920 con 0x7f73600cd470 2023-10-08T07:14:55.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.908+0000 7f73678d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73600cb8b0 con 0x7f736015ce00 2023-10-08T07:14:55.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.908+0000 7f7364e6b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73600cd470 0x7f73600cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.908+0000 7f7365e6d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7360160980 0x7f7360160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52764/0 (socket says 172.21.15.70:52764) 2023-10-08T07:14:55.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.908+0000 7f7365e6d640 1 -- 172.21.15.70:0/1286548131 learned_addr learned my addr 172.21.15.70:0/1286548131 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:55.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.909+0000 7f736566c640 1 --2- 172.21.15.70:0/1286548131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f736015ce00 0x7f73600cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.909+0000 7f7364e6b640 1 -- 172.21.15.70:0/1286548131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f736015ce00 msgr2=0x7f73600cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:55.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.909+0000 7f7364e6b640 1 --2- 172.21.15.70:0/1286548131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f736015ce00 0x7f73600cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.909+0000 7f7364e6b640 1 -- 172.21.15.70:0/1286548131 >> v1:172.21.15.70:6789/0 conn(0x7f7360160980 legacy=0x7f7360160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.909+0000 7f7364e6b640 1 -- 172.21.15.70:0/1286548131 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360161640 con 0x7f73600cd470 2023-10-08T07:14:55.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.909+0000 7f7357fff640 1 -- 172.21.15.70:0/1286548131 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044097034 0 0) 0x7f73600cde10 con 0x7f7360160980 2023-10-08T07:14:55.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.909+0000 7f736566c640 1 --2- 172.21.15.70:0/1286548131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f736015ce00 0x7f73600cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:14:55.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.910+0000 7f7364e6b640 1 --2- 172.21.15.70:0/1286548131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73600cd470 0x7f73600cd8d0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f73500097a0 tx=0x7f735002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87cd66355cdffc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:55.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.910+0000 7f7357fff640 1 -- 172.21.15.70:0/1286548131 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350031780 con 0x7f73600cd470 2023-10-08T07:14:55.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.910+0000 7f7357fff640 1 -- 172.21.15.70:0/1286548131 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73500318e0 con 0x7f73600cd470 2023-10-08T07:14:55.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.911+0000 7f7357fff640 1 -- 172.21.15.70:0/1286548131 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350031c70 con 0x7f73600cd470 2023-10-08T07:14:55.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.911+0000 7f73678d0640 1 -- 172.21.15.70:0/1286548131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73600cd470 msgr2=0x7f73600cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.911+0000 7f73678d0640 1 --2- 172.21.15.70:0/1286548131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73600cd470 0x7f73600cd8d0 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f73500097a0 tx=0x7f735002b820 comp rx=0 tx=0).stop 2023-10-08T07:14:55.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.911+0000 7f73678d0640 1 -- 172.21.15.70:0/1286548131 shutdown_connections 2023-10-08T07:14:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.911+0000 7f73678d0640 1 --2- 172.21.15.70:0/1286548131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73600cd470 0x7f73600cd8d0 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.911+0000 7f73678d0640 1 --2- 172.21.15.70:0/1286548131 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f736015ce00 0x7f73600cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.911+0000 7f73678d0640 1 -- 172.21.15.70:0/1286548131 >> 172.21.15.70:0/1286548131 conn(0x7f736006e650 msgr2=0x7f736006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.911+0000 7f73678d0640 1 -- 172.21.15.70:0/1286548131 shutdown_connections 2023-10-08T07:14:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.911+0000 7f73678d0640 1 -- 172.21.15.70:0/1286548131 wait complete. 2023-10-08T07:14:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.913+0000 7f73678d0640 1 Processor -- start 2023-10-08T07:14:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.913+0000 7f73678d0640 1 -- start start 2023-10-08T07:14:55.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.914+0000 7f73678d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73600cd470 0x7f73601f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.914+0000 7f73678d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f736015ce00 0x7f7360150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.914+0000 7f73678d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7360160980 0x7f73600c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.914+0000 7f73678d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73600b2870 con 0x7f736015ce00 2023-10-08T07:14:55.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.914+0000 7f73678d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73600b1c60 con 0x7f7360160980 2023-10-08T07:14:55.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.914+0000 7f73678d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73600b1f40 con 0x7f73600cd470 2023-10-08T07:14:55.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f7364e6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f736015ce00 0x7f7360150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f7364e6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f736015ce00 0x7f7360150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58220/0 (socket says 172.21.15.70:58220) 2023-10-08T07:14:55.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f7364e6b640 1 -- 172.21.15.70:0/1047155118 learned_addr learned my addr 172.21.15.70:0/1047155118 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:55.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f736566c640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73600cd470 0x7f73601f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f7364e6b640 1 -- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73600cd470 msgr2=0x7f73601f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f7364e6b640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73600cd470 0x7f73601f2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f7364e6b640 1 -- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7360160980 msgr2=0x7f73600c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:55.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f7365e6d640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7360160980 0x7f73600c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f7364e6b640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7360160980 0x7f73600c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f7364e6b640 1 -- 172.21.15.70:0/1047155118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7350009400 con 0x7f736015ce00 2023-10-08T07:14:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.915+0000 7f7364e6b640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f736015ce00 0x7f7360150e00 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f7350034040 tx=0x7f73500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.916+0000 7f7355ffb640 1 -- 172.21.15.70:0/1047155118 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350032820 con 0x7f736015ce00 2023-10-08T07:14:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.916+0000 7f7355ffb640 1 -- 172.21.15.70:0/1047155118 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7350032980 con 0x7f736015ce00 2023-10-08T07:14:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.916+0000 7f7355ffb640 1 -- 172.21.15.70:0/1047155118 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350032d30 con 0x7f736015ce00 2023-10-08T07:14:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.916+0000 7f73678d0640 1 -- 172.21.15.70:0/1047155118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73600b2220 con 0x7f736015ce00 2023-10-08T07:14:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.916+0000 7f73678d0640 1 -- 172.21.15.70:0/1047155118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73600b2d00 con 0x7f736015ce00 2023-10-08T07:14:55.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.917+0000 7f73678d0640 1 -- 172.21.15.70:0/1047155118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736006cff0 con 0x7f736015ce00 2023-10-08T07:14:55.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.918+0000 7f7355ffb640 1 -- 172.21.15.70:0/1047155118 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f735004e440 con 0x7f736015ce00 2023-10-08T07:14:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.919+0000 7f7355ffb640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f734406b0e0 0x7f734406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.920+0000 7f7355ffb640 1 -- 172.21.15.70:0/1047155118 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6145+0+0 (secure 0 0 0) 0x7f73500abe20 con 0x7f736015ce00 2023-10-08T07:14:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.920+0000 7f736566c640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f734406b0e0 0x7f734406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.921+0000 7f736566c640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f734406b0e0 0x7f734406d5a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f735c0096b0 tx=0x7f735c008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:55.924+0000 7f7355ffb640 1 -- 172.21.15.70:0/1047155118 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7350076980 con 0x7f736015ce00 2023-10-08T07:14:56.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.117+0000 7f73678d0640 1 -- 172.21.15.70:0/1047155118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f736004e160 con 0x7f736015ce00 2023-10-08T07:14:56.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.474+0000 7f7355ffb640 1 -- 172.21.15.70:0/1047155118 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v255) v1 ==== 144+0+0 (secure 0 0 0) 0x7f7350079fa0 con 0x7f736015ce00 2023-10-08T07:14:56.476 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'backup_fs_metadata' created 2023-10-08T07:14:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.478+0000 7f73678d0640 1 -- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f734406b0e0 msgr2=0x7f734406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.478+0000 7f73678d0640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f734406b0e0 0x7f734406d5a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f735c0096b0 tx=0x7f735c008040 comp rx=0 tx=0).stop 2023-10-08T07:14:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.478+0000 7f73678d0640 1 -- 172.21.15.70:0/1047155118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f736015ce00 msgr2=0x7f7360150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.478+0000 7f73678d0640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f736015ce00 0x7f7360150e00 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f7350034040 tx=0x7f73500046c0 comp rx=0 tx=0).stop 2023-10-08T07:14:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.479+0000 7f73678d0640 1 -- 172.21.15.70:0/1047155118 shutdown_connections 2023-10-08T07:14:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.479+0000 7f73678d0640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f734406b0e0 0x7f734406d5a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.479+0000 7f73678d0640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7360160980 0x7f73600c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.479+0000 7f73678d0640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f736015ce00 0x7f7360150e00 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.479+0000 7f73678d0640 1 --2- 172.21.15.70:0/1047155118 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f73600cd470 0x7f73601f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.480+0000 7f73678d0640 1 -- 172.21.15.70:0/1047155118 >> 172.21.15.70:0/1047155118 conn(0x7f736006e650 msgr2=0x7f736015a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.480+0000 7f73678d0640 1 -- 172.21.15.70:0/1047155118 shutdown_connections 2023-10-08T07:14:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.480+0000 7f73678d0640 1 -- 172.21.15.70:0/1047155118 wait complete. 2023-10-08T07:14:56.508 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T07:14:56.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.707+0000 7fbac08e1640 1 Processor -- start 2023-10-08T07:14:56.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.707+0000 7fbac08e1640 1 -- start start 2023-10-08T07:14:56.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.709+0000 7fbac08e1640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbabc0ca530 0x7fbabc0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:56.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.709+0000 7fbac08e1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbabc0c8ec0 0x7fbabc0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:56.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.709+0000 7fbac08e1640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbabc070cf0 con 0x7fbabc0cb500 2023-10-08T07:14:56.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.709+0000 7fbac08e1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbabc161910 con 0x7fbabc0c8ec0 2023-10-08T07:14:56.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.709+0000 7fbac08e1640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbabc0cb080 con 0x7fbabc0ca530 2023-10-08T07:14:56.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.709+0000 7fbabad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbabc0c8ec0 0x7fbabc0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:56.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.709+0000 7fbabad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbabc0c8ec0 0x7fbabc0c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58232/0 (socket says 172.21.15.70:58232) 2023-10-08T07:14:56.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.709+0000 7fbabad9d640 1 -- 172.21.15.70:0/4207429002 learned_addr learned my addr 172.21.15.70:0/4207429002 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:56.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.709+0000 7fbab9d9b640 1 --2- 172.21.15.70:0/4207429002 >> v2:172.21.15.181:3300/0 conn(0x7fbabc0ca530 0x7fbabc0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:56.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.710+0000 7fbabad9d640 1 -- 172.21.15.70:0/4207429002 >> v2:172.21.15.181:3300/0 conn(0x7fbabc0ca530 msgr2=0x7fbabc0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:56.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.710+0000 7fbabad9d640 1 --2- 172.21.15.70:0/4207429002 >> v2:172.21.15.181:3300/0 conn(0x7fbabc0ca530 0x7fbabc0ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:56.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.710+0000 7fbabad9d640 1 -- 172.21.15.70:0/4207429002 >> v1:172.21.15.181:6789/0 conn(0x7fbabc0cb500 legacy=0x7fbabc0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:56.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.710+0000 7fbabad9d640 1 -- 172.21.15.70:0/4207429002 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc0703c0 con 0x7fbabc0c8ec0 2023-10-08T07:14:56.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.711+0000 7fbabad9d640 1 --2- 172.21.15.70:0/4207429002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0c8ec0 0x7fbabc0c9350 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fbabc06c710 tx=0x7fbab402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33bca2a67bffb6b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:56.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.711+0000 7fbab959a640 1 -- 172.21.15.70:0/4207429002 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab40317e0 con 0x7fbabc0c8ec0 2023-10-08T07:14:56.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.711+0000 7fbab959a640 1 -- 172.21.15.70:0/4207429002 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab4031940 con 0x7fbabc0c8ec0 2023-10-08T07:14:56.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.711+0000 7fbab959a640 1 -- 172.21.15.70:0/4207429002 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbab4031cd0 con 0x7fbabc0c8ec0 2023-10-08T07:14:56.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.712+0000 7fbac08e1640 1 -- 172.21.15.70:0/4207429002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0c8ec0 msgr2=0x7fbabc0c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:56.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.712+0000 7fbac08e1640 1 --2- 172.21.15.70:0/4207429002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0c8ec0 0x7fbabc0c9350 secure :-1 s=READY pgs=2681 cs=0 l=1 rev1=1 crypto rx=0x7fbabc06c710 tx=0x7fbab402b880 comp rx=0 tx=0).stop 2023-10-08T07:14:56.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.712+0000 7fbac08e1640 1 -- 172.21.15.70:0/4207429002 shutdown_connections 2023-10-08T07:14:56.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.712+0000 7fbac08e1640 1 --2- 172.21.15.70:0/4207429002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0c8ec0 0x7fbabc0c9350 unknown :-1 s=CLOSED pgs=2681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:56.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.712+0000 7fbac08e1640 1 --2- 172.21.15.70:0/4207429002 >> v2:172.21.15.181:3300/0 conn(0x7fbabc0ca530 0x7fbabc0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:56.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.712+0000 7fbac08e1640 1 -- 172.21.15.70:0/4207429002 >> 172.21.15.70:0/4207429002 conn(0x7fbabc06a140 msgr2=0x7fbabc06a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:56.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.712+0000 7fbac08e1640 1 -- 172.21.15.70:0/4207429002 shutdown_connections 2023-10-08T07:14:56.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.712+0000 7fbac08e1640 1 -- 172.21.15.70:0/4207429002 wait complete. 2023-10-08T07:14:56.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.714+0000 7fbac08e1640 1 Processor -- start 2023-10-08T07:14:56.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.714+0000 7fbac08e1640 1 -- start start 2023-10-08T07:14:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.715+0000 7fbac08e1640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbabc0c8ec0 0x7fbabc159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.715+0000 7fbac08e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0ca530 0x7fbabc0c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.715+0000 7fbac08e1640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbabc0cb500 0x7fbabc0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.715+0000 7fbac08e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc0c6e80 con 0x7fbabc0ca530 2023-10-08T07:14:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.715+0000 7fbac08e1640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc0c70d0 con 0x7fbabc0c8ec0 2023-10-08T07:14:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.715+0000 7fbac08e1640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc0c73b0 con 0x7fbabc0cb500 2023-10-08T07:14:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.715+0000 7fbab9d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0ca530 0x7fbabc0c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.715+0000 7fbab9d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0ca530 0x7fbabc0c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58236/0 (socket says 172.21.15.70:58236) 2023-10-08T07:14:56.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.715+0000 7fbab9d9b640 1 -- 172.21.15.70:0/1243934091 learned_addr learned my addr 172.21.15.70:0/1243934091 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:56.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.716+0000 7fbabad9d640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbabc0cb500 0x7fbabc0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:56.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.716+0000 7fbaba59c640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbabc0c8ec0 0x7fbabc159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:56.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.716+0000 7fbab9d9b640 1 -- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbabc0cb500 msgr2=0x7fbabc0cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:56.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.716+0000 7fbab9d9b640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbabc0cb500 0x7fbabc0cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:56.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.716+0000 7fbab9d9b640 1 -- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbabc0c8ec0 msgr2=0x7fbabc159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:56.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.716+0000 7fbab9d9b640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbabc0c8ec0 0x7fbabc159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:56.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.716+0000 7fbab9d9b640 1 -- 172.21.15.70:0/1243934091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab40094d0 con 0x7fbabc0ca530 2023-10-08T07:14:56.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.716+0000 7fbab9d9b640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0ca530 0x7fbabc0c4800 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400d800 tx=0x7fbaa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:56.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.717+0000 7fbab2ffd640 1 -- 172.21.15.70:0/1243934091 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa400b6c0 con 0x7fbabc0ca530 2023-10-08T07:14:56.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.717+0000 7fbab2ffd640 1 -- 172.21.15.70:0/1243934091 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa400b820 con 0x7fbabc0ca530 2023-10-08T07:14:56.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.717+0000 7fbab2ffd640 1 -- 172.21.15.70:0/1243934091 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa400bbb0 con 0x7fbabc0ca530 2023-10-08T07:14:56.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.717+0000 7fbac08e1640 1 -- 172.21.15.70:0/1243934091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc0c7690 con 0x7fbabc0ca530 2023-10-08T07:14:56.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.717+0000 7fbac08e1640 1 -- 172.21.15.70:0/1243934091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc0c7c50 con 0x7fbabc0ca530 2023-10-08T07:14:56.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.718+0000 7fbac08e1640 1 -- 172.21.15.70:0/1243934091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc0c7e10 con 0x7fbabc0ca530 2023-10-08T07:14:56.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.719+0000 7fbab2ffd640 1 -- 172.21.15.70:0/1243934091 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbaa400bd10 con 0x7fbabc0ca530 2023-10-08T07:14:56.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.720+0000 7fbab2ffd640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fba9c06b0e0 0x7fba9c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:56.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.720+0000 7fbab2ffd640 1 -- 172.21.15.70:0/1243934091 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6518+0+0 (secure 0 0 0) 0x7fbaa408a340 con 0x7fbabc0ca530 2023-10-08T07:14:56.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.721+0000 7fbaba59c640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fba9c06b0e0 0x7fba9c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:56.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.722+0000 7fbaba59c640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fba9c06b0e0 0x7fba9c06d5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009f20 tx=0x7fbaac0094d0 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:56.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.725+0000 7fbab2ffd640 1 -- 172.21.15.70:0/1243934091 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbaa4054cc0 con 0x7fbabc0ca530 2023-10-08T07:14:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:56.917+0000 7fbac08e1640 1 -- 172.21.15.70:0/1243934091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbabc071cb0 con 0x7fbabc0ca530 2023-10-08T07:14:57.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.481+0000 7fbab2ffd640 1 -- 172.21.15.70:0/1243934091 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v256) v1 ==== 176+0+0 (secure 0 0 0) 0x7fbaa40582e0 con 0x7fbabc0ca530 2023-10-08T07:14:57.483 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'backup_fs_data' created 2023-10-08T07:14:57.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.487+0000 7fbac08e1640 1 -- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fba9c06b0e0 msgr2=0x7fba9c06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:57.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.487+0000 7fbac08e1640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fba9c06b0e0 0x7fba9c06d5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009f20 tx=0x7fbaac0094d0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.487+0000 7fbac08e1640 1 -- 172.21.15.70:0/1243934091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0ca530 msgr2=0x7fbabc0c4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:57.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.487+0000 7fbac08e1640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0ca530 0x7fbabc0c4800 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400d800 tx=0x7fbaa400dcc0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.487+0000 7fbac08e1640 1 -- 172.21.15.70:0/1243934091 shutdown_connections 2023-10-08T07:14:57.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.487+0000 7fbac08e1640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fba9c06b0e0 0x7fba9c06d5a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.487+0000 7fbac08e1640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbabc0cb500 0x7fbabc0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.487+0000 7fbac08e1640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbabc0ca530 0x7fbabc0c4800 unknown :-1 s=CLOSED pgs=2682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.487+0000 7fbac08e1640 1 --2- 172.21.15.70:0/1243934091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbabc0c8ec0 0x7fbabc159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.488+0000 7fbac08e1640 1 -- 172.21.15.70:0/1243934091 >> 172.21.15.70:0/1243934091 conn(0x7fbabc06a140 msgr2=0x7fbabc0c9980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:57.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.488+0000 7fbac08e1640 1 -- 172.21.15.70:0/1243934091 shutdown_connections 2023-10-08T07:14:57.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.488+0000 7fbac08e1640 1 -- 172.21.15.70:0/1243934091 wait complete. 2023-10-08T07:14:57.520 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-10-08T07:14:57.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.706+0000 7f42cffff640 1 Processor -- start 2023-10-08T07:14:57.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.706+0000 7f42cffff640 1 -- start start 2023-10-08T07:14:57.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.707+0000 7f42cffff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f42d015a820 0x7f42d00aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:57.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.708+0000 7f42cffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d015ac00 0x7f42d00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:57.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.708+0000 7f42cffff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d00cab60 con 0x7f42d00ca680 2023-10-08T07:14:57.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.708+0000 7f42cffff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d00cb960 con 0x7f42d015a820 2023-10-08T07:14:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.708+0000 7f42cffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00cbdf0 con 0x7f42d015ac00 2023-10-08T07:14:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.708+0000 7f42cf7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d015ac00 0x7f42d00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.708+0000 7f42ceffd640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f42d015a820 0x7f42d00aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.708+0000 7f42cf7fe640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d015ac00 0x7f42d00b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:37174/0 (socket says 172.21.15.70:37174) 2023-10-08T07:14:57.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.708+0000 7f42cf7fe640 1 -- 172.21.15.70:0/3780455119 learned_addr learned my addr 172.21.15.70:0/3780455119 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.709+0000 7f42cf7fe640 1 -- 172.21.15.70:0/3780455119 >> v2:172.21.15.181:3300/0 conn(0x7f42d015a820 msgr2=0x7f42d00aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.709+0000 7f42cf7fe640 1 --2- 172.21.15.70:0/3780455119 >> v2:172.21.15.181:3300/0 conn(0x7f42d015a820 0x7f42d00aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.709+0000 7f42cf7fe640 1 -- 172.21.15.70:0/3780455119 >> v1:172.21.15.181:6789/0 conn(0x7f42d00ca680 legacy=0x7f42d00c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:57.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.709+0000 7f42cf7fe640 1 -- 172.21.15.70:0/3780455119 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d00cb030 con 0x7f42d015ac00 2023-10-08T07:14:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.711+0000 7f42cf7fe640 1 --2- 172.21.15.70:0/3780455119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d015ac00 0x7f42d00b04b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009e50 tx=0x7f42c802be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebbc885f9e4c0e0c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:57.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.711+0000 7f42cdffb640 1 -- 172.21.15.70:0/3780455119 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c802c9e0 con 0x7f42d015ac00 2023-10-08T07:14:57.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.711+0000 7f42cdffb640 1 -- 172.21.15.70:0/3780455119 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c802cb40 con 0x7f42d015ac00 2023-10-08T07:14:57.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.711+0000 7f42cdffb640 1 -- 172.21.15.70:0/3780455119 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c80343f0 con 0x7f42d015ac00 2023-10-08T07:14:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.712+0000 7f42cffff640 1 -- 172.21.15.70:0/3780455119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d015ac00 msgr2=0x7f42d00b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.712+0000 7f42cffff640 1 --2- 172.21.15.70:0/3780455119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d015ac00 0x7f42d00b04b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009e50 tx=0x7f42c802be40 comp rx=0 tx=0).stop 2023-10-08T07:14:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.712+0000 7f42cffff640 1 -- 172.21.15.70:0/3780455119 shutdown_connections 2023-10-08T07:14:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.712+0000 7f42cffff640 1 --2- 172.21.15.70:0/3780455119 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d015ac00 0x7f42d00b04b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.712+0000 7f42cffff640 1 --2- 172.21.15.70:0/3780455119 >> v2:172.21.15.181:3300/0 conn(0x7f42d015a820 0x7f42d00aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.712+0000 7f42cffff640 1 -- 172.21.15.70:0/3780455119 >> 172.21.15.70:0/3780455119 conn(0x7f42d006c050 msgr2=0x7f42d006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:57.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.712+0000 7f42cffff640 1 -- 172.21.15.70:0/3780455119 shutdown_connections 2023-10-08T07:14:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.712+0000 7f42cffff640 1 -- 172.21.15.70:0/3780455119 wait complete. 2023-10-08T07:14:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.714+0000 7f42cffff640 1 Processor -- start 2023-10-08T07:14:57.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.714+0000 7f42cffff640 1 -- start start 2023-10-08T07:14:57.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42cffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d00ca680 0x7f42d01f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:57.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42cffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d015a820 0x7f42d01f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:57.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42cffff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42d015ac00 0x7f42d00ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:57.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42cffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00cd9c0 con 0x7f42d015a820 2023-10-08T07:14:57.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42cffff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00c3c10 con 0x7f42d015ac00 2023-10-08T07:14:57.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42cffff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d01534b0 con 0x7f42d00ca680 2023-10-08T07:14:57.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42ce7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d015a820 0x7f42d01f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:57.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42ce7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d015a820 0x7f42d01f1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58242/0 (socket says 172.21.15.70:58242) 2023-10-08T07:14:57.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42ce7fc640 1 -- 172.21.15.70:0/2605834814 learned_addr learned my addr 172.21.15.70:0/2605834814 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:57.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42ceffd640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d00ca680 0x7f42d01f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:57.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.715+0000 7f42cf7fe640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42d015ac00 0x7f42d00ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:57.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42ce7fc640 1 -- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d00ca680 msgr2=0x7f42d01f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:57.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42ce7fc640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d00ca680 0x7f42d01f61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42ce7fc640 1 -- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42d015ac00 msgr2=0x7f42d00ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:57.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42ce7fc640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42d015ac00 0x7f42d00ac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:57.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42ce7fc640 1 -- 172.21.15.70:0/2605834814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8009b30 con 0x7f42d015a820 2023-10-08T07:14:57.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42cf7fe640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42d015ac00 0x7f42d00ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:14:57.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42ce7fc640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d015a820 0x7f42d01f1ba0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f42c400c400 tx=0x7f42c400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:57.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42b77fe640 1 -- 172.21.15.70:0/2605834814 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4011790 con 0x7f42d015a820 2023-10-08T07:14:57.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42b77fe640 1 -- 172.21.15.70:0/2605834814 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c4011930 con 0x7f42d015a820 2023-10-08T07:14:57.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42cffff640 1 -- 172.21.15.70:0/2605834814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d01f20e0 con 0x7f42d015a820 2023-10-08T07:14:57.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42b77fe640 1 -- 172.21.15.70:0/2605834814 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4011ce0 con 0x7f42d015a820 2023-10-08T07:14:57.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.716+0000 7f42cffff640 1 -- 172.21.15.70:0/2605834814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d00cdc10 con 0x7f42d015a820 2023-10-08T07:14:57.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.717+0000 7f42cffff640 1 -- 172.21.15.70:0/2605834814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d0074150 con 0x7f42d015a820 2023-10-08T07:14:57.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.718+0000 7f42b77fe640 1 -- 172.21.15.70:0/2605834814 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f42c401c460 con 0x7f42d015a820 2023-10-08T07:14:57.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.719+0000 7f42b77fe640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f42bc06b0b0 0x7f42bc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:57.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.719+0000 7f42b77fe640 1 -- 172.21.15.70:0/2605834814 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 6903+0+0 (secure 0 0 0) 0x7f42c4089d40 con 0x7f42d015a820 2023-10-08T07:14:57.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.720+0000 7f42ceffd640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f42bc06b0b0 0x7f42bc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:57.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.721+0000 7f42ceffd640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f42bc06b0b0 0x7f42bc06d570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009a10 tx=0x7f42c0008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:57.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.725+0000 7f42b77fe640 1 -- 172.21.15.70:0/2605834814 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42c4054720 con 0x7f42d015a820 2023-10-08T07:14:57.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:57.952+0000 7f42cffff640 1 -- 172.21.15.70:0/2605834814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f42d00b3f00 con 0x7f42d015a820 2023-10-08T07:14:58.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.535+0000 7f42b77fe640 1 -- 172.21.15.70:0/2605834814 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 33 and data pool 34 v127) v1 ==== 183+0+0 (secure 0 0 0) 0x7f42c4057d40 con 0x7f42d015a820 2023-10-08T07:14:58.540 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 33 and data pool 34 2023-10-08T07:14:58.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.543+0000 7f42cffff640 1 -- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f42bc06b0b0 msgr2=0x7f42bc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:58.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.543+0000 7f42cffff640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f42bc06b0b0 0x7f42bc06d570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009a10 tx=0x7f42c0008040 comp rx=0 tx=0).stop 2023-10-08T07:14:58.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.543+0000 7f42cffff640 1 -- 172.21.15.70:0/2605834814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d015a820 msgr2=0x7f42d01f1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:58.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.543+0000 7f42cffff640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d015a820 0x7f42d01f1ba0 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f42c400c400 tx=0x7f42c400c8c0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.543+0000 7f42cffff640 1 -- 172.21.15.70:0/2605834814 shutdown_connections 2023-10-08T07:14:58.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.543+0000 7f42cffff640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f42bc06b0b0 0x7f42bc06d570 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009a10 tx=0x7f42c0008040 comp rx=0 tx=0).stop 2023-10-08T07:14:58.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.543+0000 7f42cffff640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f42d015ac00 0x7f42d00ac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.543+0000 7f42cffff640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42d015a820 0x7f42d01f1ba0 secure :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f42c400c400 tx=0x7f42c400c8c0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.543+0000 7f42cffff640 1 --2- 172.21.15.70:0/2605834814 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f42d00ca680 0x7f42d01f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.543+0000 7f42cffff640 1 -- 172.21.15.70:0/2605834814 >> 172.21.15.70:0/2605834814 conn(0x7f42d006c050 msgr2=0x7f42d00ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:58.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.547+0000 7f42cffff640 1 -- 172.21.15.70:0/2605834814 shutdown_connections 2023-10-08T07:14:58.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.550+0000 7f42cffff640 1 -- 172.21.15.70:0/2605834814 wait complete. 2023-10-08T07:14:58.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:58.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.754+0000 7f3136105640 1 Processor -- start 2023-10-08T07:14:58.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.754+0000 7f3136105640 1 -- start start 2023-10-08T07:14:58.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.755+0000 7f3136105640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f313015ab40 0x7f31300ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:58.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.755+0000 7f3136105640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f31300c90b0 0x7f31300adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:58.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.755+0000 7f3136105640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31300cae70 0x7f31300cb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:58.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.755+0000 7f3136105640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313015bf30 con 0x7f31300cae70 2023-10-08T07:14:58.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.755+0000 7f3136105640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31300cb840 con 0x7f31300c90b0 2023-10-08T07:14:58.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.755+0000 7f3136105640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313015b2b0 con 0x7f313015ab40 2023-10-08T07:14:58.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.756+0000 7f312ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31300cae70 0x7f31300cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:58.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.756+0000 7f312ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31300cae70 0x7f31300cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58256/0 (socket says 172.21.15.70:58256) 2023-10-08T07:14:58.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.756+0000 7f312ffff640 1 -- 172.21.15.70:0/1873397797 learned_addr learned my addr 172.21.15.70:0/1873397797 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:58.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.756+0000 7f312f7fe640 1 --2- 172.21.15.70:0/1873397797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f313015ab40 0x7f31300ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:58.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.756+0000 7f312effd640 1 --2- 172.21.15.70:0/1873397797 >> v2:172.21.15.181:3300/0 conn(0x7f31300c90b0 0x7f31300adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:58.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.756+0000 7f312ffff640 1 -- 172.21.15.70:0/1873397797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f313015ab40 msgr2=0x7f31300ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:58.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.756+0000 7f312ffff640 1 --2- 172.21.15.70:0/1873397797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f313015ab40 0x7f31300ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.756+0000 7f312ffff640 1 -- 172.21.15.70:0/1873397797 >> v2:172.21.15.181:3300/0 conn(0x7f31300c90b0 msgr2=0x7f31300adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:58.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.756+0000 7f312ffff640 1 --2- 172.21.15.70:0/1873397797 >> v2:172.21.15.181:3300/0 conn(0x7f31300c90b0 0x7f31300adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.756+0000 7f312ffff640 1 -- 172.21.15.70:0/1873397797 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313015b590 con 0x7f31300cae70 2023-10-08T07:14:58.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.757+0000 7f312ffff640 1 --2- 172.21.15.70:0/1873397797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 0x7f31300cb300 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f313006e470 tx=0x7f312802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ef102ad8637330 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:58.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.758+0000 7f312e7fc640 1 -- 172.21.15.70:0/1873397797 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312802ca70 con 0x7f31300cae70 2023-10-08T07:14:58.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.758+0000 7f312e7fc640 1 -- 172.21.15.70:0/1873397797 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312802cbd0 con 0x7f31300cae70 2023-10-08T07:14:58.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.758+0000 7f312e7fc640 1 -- 172.21.15.70:0/1873397797 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31280343f0 con 0x7f31300cae70 2023-10-08T07:14:58.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.758+0000 7f3136105640 1 -- 172.21.15.70:0/1873397797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 msgr2=0x7f31300cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:58.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.758+0000 7f3136105640 1 --2- 172.21.15.70:0/1873397797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 0x7f31300cb300 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f313006e470 tx=0x7f312802bed0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.758+0000 7f3136105640 1 -- 172.21.15.70:0/1873397797 shutdown_connections 2023-10-08T07:14:58.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.758+0000 7f3136105640 1 --2- 172.21.15.70:0/1873397797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 0x7f31300cb300 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.758+0000 7f3136105640 1 --2- 172.21.15.70:0/1873397797 >> v2:172.21.15.181:3300/0 conn(0x7f31300c90b0 0x7f31300adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.758+0000 7f3136105640 1 --2- 172.21.15.70:0/1873397797 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f313015ab40 0x7f31300ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.759+0000 7f3136105640 1 -- 172.21.15.70:0/1873397797 >> 172.21.15.70:0/1873397797 conn(0x7f313006a380 msgr2=0x7f31300b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:58.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.759+0000 7f3136105640 1 -- 172.21.15.70:0/1873397797 shutdown_connections 2023-10-08T07:14:58.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.759+0000 7f3136105640 1 -- 172.21.15.70:0/1873397797 wait complete. 2023-10-08T07:14:58.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.760+0000 7f3136105640 1 Processor -- start 2023-10-08T07:14:58.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.761+0000 7f3136105640 1 -- start start 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f3136105640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31300c90b0 0x7f31300c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f3136105640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 0x7f31300c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f3136105640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313015ab40 0x7f31300b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f3136105640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31300c3fc0 con 0x7f31300cae70 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f3136105640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31300c42a0 con 0x7f313015ab40 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f3136105640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31300c4580 con 0x7f31300c90b0 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f312effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 0x7f31300c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f312effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 0x7f31300c5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58272/0 (socket says 172.21.15.70:58272) 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f312effd640 1 -- 172.21.15.70:0/3677379343 learned_addr learned my addr 172.21.15.70:0/3677379343 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f312effd640 1 -- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31300c90b0 msgr2=0x7f31300c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:58.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f312f7fe640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31300c90b0 0x7f31300c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f312ffff640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313015ab40 0x7f31300b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f312effd640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31300c90b0 0x7f31300c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f312effd640 1 -- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313015ab40 msgr2=0x7f31300b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f312effd640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313015ab40 0x7f31300b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.762+0000 7f312effd640 1 -- 172.21.15.70:0/3677379343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3128009b30 con 0x7f31300cae70 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.763+0000 7f312effd640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 0x7f31300c5a70 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f312400d800 tx=0x7f312400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.763+0000 7f3117fff640 1 -- 172.21.15.70:0/3677379343 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240096a0 con 0x7f31300cae70 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.763+0000 7f3117fff640 1 -- 172.21.15.70:0/3677379343 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124009800 con 0x7f31300cae70 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.763+0000 7f3117fff640 1 -- 172.21.15.70:0/3677379343 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124009990 con 0x7f31300cae70 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.763+0000 7f3136105640 1 -- 172.21.15.70:0/3677379343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31300c4860 con 0x7f31300cae70 2023-10-08T07:14:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.763+0000 7f3136105640 1 -- 172.21.15.70:0/3677379343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31300c55a0 con 0x7f31300cae70 2023-10-08T07:14:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.764+0000 7f3136105640 1 -- 172.21.15.70:0/3677379343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3130067490 con 0x7f31300cae70 2023-10-08T07:14:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.765+0000 7f3117fff640 1 -- 172.21.15.70:0/3677379343 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f3124009af0 con 0x7f31300cae70 2023-10-08T07:14:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.766+0000 7f3117fff640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f311c06b0b0 0x7f311c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.766+0000 7f3117fff640 1 -- 172.21.15.70:0/3677379343 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f312408a370 con 0x7f31300cae70 2023-10-08T07:14:58.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.767+0000 7f312f7fe640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f311c06b0b0 0x7f311c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:58.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.768+0000 7f312f7fe640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f311c06b0b0 0x7f311c06d570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3120006fd0 tx=0x7f3120008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:58.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.771+0000 7f3117fff640 1 -- 172.21.15.70:0/3677379343 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3124054c70 con 0x7f31300cae70 2023-10-08T07:14:58.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.957+0000 7f3136105640 1 -- 172.21.15.70:0/3677379343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31300cb620 con 0x7f31300cae70 2023-10-08T07:14:58.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.960+0000 7f3117fff640 1 -- 172.21.15.70:0/3677379343 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) v1 ==== 74+0+20631 (secure 0 0 0) 0x7f3124058290 con 0x7f31300cae70 2023-10-08T07:14:58.962 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:58.962 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":257,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:57.954594+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:14:56.178783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:14:56.977158+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:58.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 -- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f311c06b0b0 msgr2=0x7f311c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:58.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f311c06b0b0 0x7f311c06d570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3120006fd0 tx=0x7f3120008040 comp rx=0 tx=0).stop 2023-10-08T07:14:58.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 -- 172.21.15.70:0/3677379343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 msgr2=0x7f31300c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:58.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 0x7f31300c5a70 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f312400d800 tx=0x7f312400dcc0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 -- 172.21.15.70:0/3677379343 shutdown_connections 2023-10-08T07:14:58.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f311c06b0b0 0x7f311c06d570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f313015ab40 0x7f31300b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31300cae70 0x7f31300c5a70 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 --2- 172.21.15.70:0/3677379343 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f31300c90b0 0x7f31300c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:58.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 -- 172.21.15.70:0/3677379343 >> 172.21.15.70:0/3677379343 conn(0x7f313006a380 msgr2=0x7f31300ae830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:58.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 -- 172.21.15.70:0/3677379343 shutdown_connections 2023-10-08T07:14:58.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:58.962+0000 7f3136105640 1 -- 172.21.15.70:0/3677379343 wait complete. 2023-10-08T07:14:58.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:14:59.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.192+0000 7f8623b7c640 1 Processor -- start 2023-10-08T07:14:59.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.192+0000 7f8623b7c640 1 -- start start 2023-10-08T07:14:59.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.193+0000 7f8623b7c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f861c0cd8d0 0x7f861c0cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.193+0000 7f8623b7c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861c0b8bf0 con 0x7f861c0b1840 2023-10-08T07:14:59.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.193+0000 7f8623b7c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861c0d39b0 con 0x7f861c0b1d90 2023-10-08T07:14:59.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.193+0000 7f8623b7c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f861c0d2910 con 0x7f861c0cd8d0 2023-10-08T07:14:59.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.193+0000 7f8621117640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f861c0b1840 0x7f861c0d2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52780/0 (socket says 172.21.15.70:52780) 2023-10-08T07:14:59.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.194+0000 7f8621117640 1 -- 172.21.15.70:0/1025995353 learned_addr learned my addr 172.21.15.70:0/1025995353 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:59.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.194+0000 7f8621918640 1 --2- 172.21.15.70:0/1025995353 >> v2:172.21.15.181:3300/0 conn(0x7f861c0cd8d0 0x7f861c0cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.194+0000 7f8620916640 1 -- 172.21.15.70:0/1025995353 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076477501 0 0) 0x7f861c0b8bf0 con 0x7f861c0b1840 2023-10-08T07:14:59.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.194+0000 7f8620916640 1 -- 172.21.15.70:0/1025995353 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8604003520 con 0x7f861c0b1840 2023-10-08T07:14:59.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.195+0000 7f8621918640 1 -- 172.21.15.70:0/1025995353 >> v1:172.21.15.181:6789/0 conn(0x7f861c0b1d90 legacy=0x7f861c0d32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.195+0000 7f8621918640 1 -- 172.21.15.70:0/1025995353 >> v1:172.21.15.70:6789/0 conn(0x7f861c0b1840 legacy=0x7f861c0d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.195+0000 7f8621918640 1 -- 172.21.15.70:0/1025995353 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c0ce1f0 con 0x7f861c0cd8d0 2023-10-08T07:14:59.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.196+0000 7f8621918640 1 --2- 172.21.15.70:0/1025995353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0cd8d0 0x7f861c0cdcb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f860c009d90 tx=0x7f860c02bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6e034b537cb37ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:59.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.196+0000 7f8620916640 1 -- 172.21.15.70:0/1025995353 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c02ccf0 con 0x7f861c0cd8d0 2023-10-08T07:14:59.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.197+0000 7f8620916640 1 -- 172.21.15.70:0/1025995353 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860c02ce50 con 0x7f861c0cd8d0 2023-10-08T07:14:59.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.197+0000 7f8620916640 1 -- 172.21.15.70:0/1025995353 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c034640 con 0x7f861c0cd8d0 2023-10-08T07:14:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.197+0000 7f8623b7c640 1 -- 172.21.15.70:0/1025995353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0cd8d0 msgr2=0x7f861c0cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.197+0000 7f8623b7c640 1 --2- 172.21.15.70:0/1025995353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0cd8d0 0x7f861c0cdcb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f860c009d90 tx=0x7f860c02bf40 comp rx=0 tx=0).stop 2023-10-08T07:14:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.197+0000 7f8623b7c640 1 -- 172.21.15.70:0/1025995353 shutdown_connections 2023-10-08T07:14:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.197+0000 7f8623b7c640 1 --2- 172.21.15.70:0/1025995353 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0cd8d0 0x7f861c0cdcb0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.197+0000 7f8623b7c640 1 -- 172.21.15.70:0/1025995353 >> 172.21.15.70:0/1025995353 conn(0x7f861c06b6a0 msgr2=0x7f861c06bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.197+0000 7f8623b7c640 1 -- 172.21.15.70:0/1025995353 shutdown_connections 2023-10-08T07:14:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.197+0000 7f8623b7c640 1 -- 172.21.15.70:0/1025995353 wait complete. 2023-10-08T07:14:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.199+0000 7f8623b7c640 1 Processor -- start 2023-10-08T07:14:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.199+0000 7f8623b7c640 1 -- start start 2023-10-08T07:14:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.200+0000 7f8623b7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f861c0b1840 0x7f861c158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.200+0000 7f8623b7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0b1d90 0x7f861c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.200+0000 7f8623b7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f861c154070 0x7f861c0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.200+0000 7f8623b7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c159160 con 0x7f861c0b1840 2023-10-08T07:14:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.200+0000 7f8623b7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c159440 con 0x7f861c0b1d90 2023-10-08T07:14:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.200+0000 7f8623b7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861c159720 con 0x7f861c154070 2023-10-08T07:14:59.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.201+0000 7f8621918640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f861c0b1840 0x7f861c158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.201+0000 7f8621918640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f861c0b1840 0x7f861c158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58278/0 (socket says 172.21.15.70:58278) 2023-10-08T07:14:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.201+0000 7f8621918640 1 -- 172.21.15.70:0/3689631666 learned_addr learned my addr 172.21.15.70:0/3689631666 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.201+0000 7f8622119640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f861c154070 0x7f861c0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.201+0000 7f8621117640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0b1d90 0x7f861c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.201+0000 7f8621918640 1 -- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f861c154070 msgr2=0x7f861c0cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.201+0000 7f8621918640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f861c154070 0x7f861c0cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.201+0000 7f8621918640 1 -- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0b1d90 msgr2=0x7f861c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.201+0000 7f8621918640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0b1d90 0x7f861c15af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.201+0000 7f8621918640 1 -- 172.21.15.70:0/3689631666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c009a70 con 0x7f861c0b1840 2023-10-08T07:14:59.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.202+0000 7f8621918640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f861c0b1840 0x7f861c158880 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f860c009d90 tx=0x7f860c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:59.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.202+0000 7f861a7fc640 1 -- 172.21.15.70:0/3689631666 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c009450 con 0x7f861c0b1840 2023-10-08T07:14:59.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.202+0000 7f861a7fc640 1 -- 172.21.15.70:0/3689631666 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860c003e10 con 0x7f861c0b1840 2023-10-08T07:14:59.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.202+0000 7f861a7fc640 1 -- 172.21.15.70:0/3689631666 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c0048a0 con 0x7f861c0b1840 2023-10-08T07:14:59.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.202+0000 7f8623b7c640 1 -- 172.21.15.70:0/3689631666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861c159a00 con 0x7f861c0b1840 2023-10-08T07:14:59.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.202+0000 7f8623b7c640 1 -- 172.21.15.70:0/3689631666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861c158dc0 con 0x7f861c0b1840 2023-10-08T07:14:59.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.203+0000 7f8623b7c640 1 -- 172.21.15.70:0/3689631666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861c066d80 con 0x7f861c0b1840 2023-10-08T07:14:59.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.204+0000 7f861a7fc640 1 -- 172.21.15.70:0/3689631666 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f860c03b070 con 0x7f861c0b1840 2023-10-08T07:14:59.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.205+0000 7f861a7fc640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f860806b0b0 0x7f860806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.205+0000 7f861a7fc640 1 -- 172.21.15.70:0/3689631666 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f860c0ac4c0 con 0x7f861c0b1840 2023-10-08T07:14:59.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.206+0000 7f8621117640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f860806b0b0 0x7f860806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.207+0000 7f8621117640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f860806b0b0 0x7f860806d570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86100011e0 tx=0x7f8610009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:59.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.209+0000 7f861a7fc640 1 -- 172.21.15.70:0/3689631666 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f860c076e00 con 0x7f861c0b1840 2023-10-08T07:14:59.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.402+0000 7f8623b7c640 1 -- 172.21.15.70:0/3689631666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f861c15c5c0 con 0x7f861c0b1840 2023-10-08T07:14:59.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.404+0000 7f861a7fc640 1 -- 172.21.15.70:0/3689631666 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v257) v1 ==== 74+0+20631 (secure 0 0 0) 0x7f860c07a420 con 0x7f861c0b1840 2023-10-08T07:14:59.406 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:14:59.407 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":257,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:57.954594+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:14:56.178783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:14:56.977158+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:14:59.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 -- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f860806b0b0 msgr2=0x7f860806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f860806b0b0 0x7f860806d570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86100011e0 tx=0x7f8610009040 comp rx=0 tx=0).stop 2023-10-08T07:14:59.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 -- 172.21.15.70:0/3689631666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f861c0b1840 msgr2=0x7f861c158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f861c0b1840 0x7f861c158880 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7f860c009d90 tx=0x7f860c004a40 comp rx=0 tx=0).stop 2023-10-08T07:14:59.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 -- 172.21.15.70:0/3689631666 shutdown_connections 2023-10-08T07:14:59.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f860806b0b0 0x7f860806d570 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f861c154070 0x7f861c0cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f861c0b1d90 0x7f861c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 --2- 172.21.15.70:0/3689631666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f861c0b1840 0x7f861c158880 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 -- 172.21.15.70:0/3689631666 >> 172.21.15.70:0/3689631666 conn(0x7f861c06b6a0 msgr2=0x7f861c0b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:59.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 -- 172.21.15.70:0/3689631666 shutdown_connections 2023-10-08T07:14:59.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.408+0000 7f8623b7c640 1 -- 172.21.15.70:0/3689631666 wait complete. 2023-10-08T07:14:59.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-10-08T07:14:59.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.647+0000 7f76ab413640 1 Processor -- start 2023-10-08T07:14:59.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.648+0000 7f76ab413640 1 -- start start 2023-10-08T07:14:59.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.649+0000 7f76ab413640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a415a820 0x7f76a40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.649+0000 7f76ab413640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40cab60 con 0x7f76a40ca680 2023-10-08T07:14:59.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.649+0000 7f76ab413640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40cb960 con 0x7f76a415ac00 2023-10-08T07:14:59.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.649+0000 7f76ab413640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40cbdf0 con 0x7f76a415a820 2023-10-08T07:14:59.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.649+0000 7f76a9c10640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f76a40ca680 0x7f76a40c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52794/0 (socket says 172.21.15.70:52794) 2023-10-08T07:14:59.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.649+0000 7f76a9c10640 1 -- 172.21.15.70:0/2943782647 learned_addr learned my addr 172.21.15.70:0/2943782647 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:14:59.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76aa411640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a415a820 0x7f76a40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964263359 0 0) 0x7f76a40cab60 con 0x7f76a40ca680 2023-10-08T07:14:59.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7690003520 con 0x7f76a40ca680 2023-10-08T07:14:59.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 383697144 0 0) 0x7f7690003520 con 0x7f76a40ca680 2023-10-08T07:14:59.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a415a820 msgr2=0x7f76a40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:59.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a415a820 0x7f76a40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 >> v1:172.21.15.181:6789/0 conn(0x7f76a415ac00 legacy=0x7f76a40c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a40c9f20 con 0x7f76a40ca680 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7694001670 con 0x7f76a40ca680 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 >> v1:172.21.15.70:6789/0 conn(0x7f76a40ca680 legacy=0x7f76a40c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76900039a0 0x7f7690003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a415a820 0x7f76900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.650+0000 7f76a940f640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7690004d00 0x7f7690005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.651+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40cbdf0 con 0x7f76900039a0 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.651+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769000bb10 con 0x7f7690004d00 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.651+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769000bcd0 con 0x7f76a415a820 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.651+0000 7f76aa411640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76900039a0 0x7f7690003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.651+0000 7f76a9c10640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a415a820 0x7f76900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.651+0000 7f76aa411640 1 -- 172.21.15.70:0/2943782647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a415a820 msgr2=0x7f76900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:59.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.651+0000 7f76aa411640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a415a820 0x7f76900047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.651+0000 7f76aa411640 1 -- 172.21.15.70:0/2943782647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7690004d00 msgr2=0x7f7690005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:14:59.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.651+0000 7f76aa411640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7690004d00 0x7f7690005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.651+0000 7f76aa411640 1 -- 172.21.15.70:0/2943782647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690003520 con 0x7f76900039a0 2023-10-08T07:14:59.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.652+0000 7f76aa411640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76900039a0 0x7f7690003dc0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009ac0 tx=0x7f76900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55a134a63ec2ec8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:59.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.652+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a002c6b0 con 0x7f76900039a0 2023-10-08T07:14:59.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.653+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a002c810 con 0x7f76900039a0 2023-10-08T07:14:59.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.653+0000 7f76a940f640 1 -- 172.21.15.70:0/2943782647 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a002cba0 con 0x7f76900039a0 2023-10-08T07:14:59.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.653+0000 7f76ab413640 1 -- 172.21.15.70:0/2943782647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76900039a0 msgr2=0x7f7690003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.653+0000 7f76ab413640 1 --2- 172.21.15.70:0/2943782647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76900039a0 0x7f7690003dc0 secure :-1 s=READY pgs=2689 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009ac0 tx=0x7f76900057f0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.653+0000 7f76aac12640 1 -- 172.21.15.70:0/2943782647 reap_dead start 2023-10-08T07:14:59.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.653+0000 7f76ab413640 1 -- 172.21.15.70:0/2943782647 shutdown_connections 2023-10-08T07:14:59.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.653+0000 7f76ab413640 1 -- 172.21.15.70:0/2943782647 >> 172.21.15.70:0/2943782647 conn(0x7f76a406c050 msgr2=0x7f76a406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:14:59.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.653+0000 7f76ab413640 1 -- 172.21.15.70:0/2943782647 shutdown_connections 2023-10-08T07:14:59.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.653+0000 7f76ab413640 1 -- 172.21.15.70:0/2943782647 wait complete. 2023-10-08T07:14:59.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.655+0000 7f76ab413640 1 Processor -- start 2023-10-08T07:14:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.655+0000 7f76ab413640 1 -- start start 2023-10-08T07:14:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.656+0000 7f76ab413640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76900039a0 0x7f76a41f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.656+0000 7f76ab413640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a40cca90 0x7f76a4158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.656+0000 7f76ab413640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a40afd00 0x7f76a40b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.656+0000 7f76ab413640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40c4d00 con 0x7f76a40afd00 2023-10-08T07:14:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.656+0000 7f76ab413640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40ce2b0 con 0x7f76900039a0 2023-10-08T07:14:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.656+0000 7f76ab413640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a4158a90 con 0x7f76a40cca90 2023-10-08T07:14:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76aac12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a40afd00 0x7f76a40b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76aac12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a40afd00 0x7f76a40b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58292/0 (socket says 172.21.15.70:58292) 2023-10-08T07:14:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76aac12640 1 -- 172.21.15.70:0/2559299714 learned_addr learned my addr 172.21.15.70:0/2559299714 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:14:59.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76aa411640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76900039a0 0x7f76a41f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76a9c10640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a40cca90 0x7f76a4158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76aac12640 1 -- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a40cca90 msgr2=0x7f76a4158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76aac12640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a40cca90 0x7f76a4158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76aac12640 1 -- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76900039a0 msgr2=0x7f76a41f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:14:59.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76aac12640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76900039a0 0x7f76a41f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:14:59.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76aac12640 1 -- 172.21.15.70:0/2559299714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a40cd210 con 0x7f76a40afd00 2023-10-08T07:14:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.657+0000 7f76aac12640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a40afd00 0x7f76a40b0180 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f76a40ad960 tx=0x7f769c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.658+0000 7f769affd640 1 -- 172.21.15.70:0/2559299714 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c005d40 con 0x7f76a40afd00 2023-10-08T07:14:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.658+0000 7f769affd640 1 -- 172.21.15.70:0/2559299714 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c00ced0 con 0x7f76a40afd00 2023-10-08T07:14:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.658+0000 7f76ab413640 1 -- 172.21.15.70:0/2559299714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a40cd4f0 con 0x7f76a40afd00 2023-10-08T07:14:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.658+0000 7f769affd640 1 -- 172.21.15.70:0/2559299714 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00a850 con 0x7f76a40afd00 2023-10-08T07:14:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.658+0000 7f76ab413640 1 -- 172.21.15.70:0/2559299714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a40cdab0 con 0x7f76a40afd00 2023-10-08T07:14:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.659+0000 7f76ab413640 1 -- 172.21.15.70:0/2559299714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a4074150 con 0x7f76a40afd00 2023-10-08T07:14:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.660+0000 7f769affd640 1 -- 172.21.15.70:0/2559299714 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f769c00a9b0 con 0x7f76a40afd00 2023-10-08T07:14:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.661+0000 7f769affd640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f768806b0b0 0x7f768806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:14:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.661+0000 7f769affd640 1 -- 172.21.15.70:0/2559299714 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f769c088ea0 con 0x7f76a40afd00 2023-10-08T07:14:59.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.662+0000 7f76aa411640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f768806b0b0 0x7f768806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:14:59.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.663+0000 7f76aa411640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f768806b0b0 0x7f768806d570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76a0036000 tx=0x7f76a0030000 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:14:59.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.665+0000 7f769affd640 1 -- 172.21.15.70:0/2559299714 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f769c053630 con 0x7f76a40afd00 2023-10-08T07:14:59.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:14:59.893+0000 7f76ab413640 1 -- 172.21.15.70:0/2559299714 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f76a40cb2e0 con 0x7f76a40afd00 2023-10-08T07:15:00.350 INFO:tasks.ceph.mds.b.smithi181.stderr:2023-10-08T07:15:00.349+0000 7f062a709640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:15:00.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.549+0000 7f769affd640 1 -- 172.21.15.70:0/2559299714 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v129) v1 ==== 121+0+0 (secure 0 0 0) 0x7f769c056c50 con 0x7f76a40afd00 2023-10-08T07:15:00.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 -- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f768806b0b0 msgr2=0x7f768806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:00.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f768806b0b0 0x7f768806d570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76a0036000 tx=0x7f76a0030000 comp rx=0 tx=0).stop 2023-10-08T07:15:00.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 -- 172.21.15.70:0/2559299714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a40afd00 msgr2=0x7f76a40b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:00.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a40afd00 0x7f76a40b0180 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f76a40ad960 tx=0x7f769c00cb90 comp rx=0 tx=0).stop 2023-10-08T07:15:00.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 -- 172.21.15.70:0/2559299714 shutdown_connections 2023-10-08T07:15:00.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f768806b0b0 0x7f768806d570 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:00.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76a40afd00 0x7f76a40b0180 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:00.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76a40cca90 0x7f76a4158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:00.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 --2- 172.21.15.70:0/2559299714 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76900039a0 0x7f76a41f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:00.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 -- 172.21.15.70:0/2559299714 >> 172.21.15.70:0/2559299714 conn(0x7f76a406c050 msgr2=0x7f76a40707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:00.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 -- 172.21.15.70:0/2559299714 shutdown_connections 2023-10-08T07:15:00.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.553+0000 7f76ab413640 1 -- 172.21.15.70:0/2559299714 wait complete. 2023-10-08T07:15:00.575 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T07:15:00.575 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-10-08T07:15:00.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.787+0000 7f377f052640 1 Processor -- start 2023-10-08T07:15:00.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.787+0000 7f377f052640 1 -- start start 2023-10-08T07:15:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.789+0000 7f377f052640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f37780b0680 0x7f377815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.789+0000 7f377f052640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37780cab60 con 0x7f37780ca680 2023-10-08T07:15:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.789+0000 7f377f052640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37780cb960 con 0x7f377815a820 2023-10-08T07:15:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.789+0000 7f377f052640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37780cbdf0 con 0x7f37780b0680 2023-10-08T07:15:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.789+0000 7f377d84f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f37780b0680 0x7f377815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.789+0000 7f377e851640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f37780ca680 0x7f377815ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52800/0 (socket says 172.21.15.70:52800) 2023-10-08T07:15:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.789+0000 7f377e851640 1 -- 172.21.15.70:0/2290237011 learned_addr learned my addr 172.21.15.70:0/2290237011 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:00.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.790+0000 7f377d84f640 1 -- 172.21.15.70:0/2290237011 >> v1:172.21.15.181:6789/0 conn(0x7f377815a820 legacy=0x7f37780aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:00.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.790+0000 7f377d84f640 1 -- 172.21.15.70:0/2290237011 >> v1:172.21.15.70:6789/0 conn(0x7f37780ca680 legacy=0x7f377815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:00.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.790+0000 7f377d04e640 1 -- 172.21.15.70:0/2290237011 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594180699 0 0) 0x7f37780cab60 con 0x7f37780ca680 2023-10-08T07:15:00.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.790+0000 7f377d84f640 1 -- 172.21.15.70:0/2290237011 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37780cb030 con 0x7f37780b0680 2023-10-08T07:15:00.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.791+0000 7f377d84f640 1 --2- 172.21.15.70:0/2290237011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 0x7f377815e540 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f37680097a0 tx=0x7f376802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c72ada1a800d822 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:00.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.791+0000 7f377d04e640 1 -- 172.21.15.70:0/2290237011 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376802e070 con 0x7f37780b0680 2023-10-08T07:15:00.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.791+0000 7f377d04e640 1 -- 172.21.15.70:0/2290237011 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37680315a0 con 0x7f37780b0680 2023-10-08T07:15:00.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.791+0000 7f377d04e640 1 -- 172.21.15.70:0/2290237011 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768031930 con 0x7f37780b0680 2023-10-08T07:15:00.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.792+0000 7f377f052640 1 -- 172.21.15.70:0/2290237011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 msgr2=0x7f377815e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:00.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.792+0000 7f377f052640 1 --2- 172.21.15.70:0/2290237011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 0x7f377815e540 secure :-1 s=READY pgs=2692 cs=0 l=1 rev1=1 crypto rx=0x7f37680097a0 tx=0x7f376802b840 comp rx=0 tx=0).stop 2023-10-08T07:15:00.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.792+0000 7f377f052640 1 -- 172.21.15.70:0/2290237011 shutdown_connections 2023-10-08T07:15:00.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.792+0000 7f377f052640 1 --2- 172.21.15.70:0/2290237011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 0x7f377815e540 unknown :-1 s=CLOSED pgs=2692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:00.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.792+0000 7f377f052640 1 -- 172.21.15.70:0/2290237011 >> 172.21.15.70:0/2290237011 conn(0x7f377806c050 msgr2=0x7f377806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:00.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.792+0000 7f377f052640 1 -- 172.21.15.70:0/2290237011 shutdown_connections 2023-10-08T07:15:00.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.792+0000 7f377f052640 1 -- 172.21.15.70:0/2290237011 wait complete. 2023-10-08T07:15:00.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.794+0000 7f377f052640 1 Processor -- start 2023-10-08T07:15:00.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.794+0000 7f377f052640 1 -- start start 2023-10-08T07:15:00.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377f052640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 0x7f37781f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:00.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377f052640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37780ca680 0x7f37781f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:00.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377f052640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f377815a820 0x7f3778158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377f052640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377815c390 con 0x7f37780b0680 2023-10-08T07:15:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377f052640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37781f63b0 con 0x7f37780ca680 2023-10-08T07:15:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377f052640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3778159070 con 0x7f377815a820 2023-10-08T07:15:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377e050640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 0x7f37781f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377e050640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 0x7f37781f1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58312/0 (socket says 172.21.15.70:58312) 2023-10-08T07:15:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377e050640 1 -- 172.21.15.70:0/45462986 learned_addr learned my addr 172.21.15.70:0/45462986 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377e050640 1 -- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f377815a820 msgr2=0x7f3778158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377e050640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f377815a820 0x7f3778158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377e050640 1 -- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37780ca680 msgr2=0x7f37781f5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377e050640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37780ca680 0x7f37781f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:00.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.795+0000 7f377e050640 1 -- 172.21.15.70:0/45462986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3768009400 con 0x7f37780b0680 2023-10-08T07:15:00.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.796+0000 7f377e050640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 0x7f37781f1e40 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f377400b3f0 tx=0x7f377400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.796+0000 7f376effd640 1 -- 172.21.15.70:0/45462986 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774011830 con 0x7f37780b0680 2023-10-08T07:15:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.796+0000 7f376effd640 1 -- 172.21.15.70:0/45462986 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3774011990 con 0x7f37780b0680 2023-10-08T07:15:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.796+0000 7f376effd640 1 -- 172.21.15.70:0/45462986 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774011d40 con 0x7f37780b0680 2023-10-08T07:15:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.796+0000 7f377f052640 1 -- 172.21.15.70:0/45462986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37781f24c0 con 0x7f37780b0680 2023-10-08T07:15:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.796+0000 7f377f052640 1 -- 172.21.15.70:0/45462986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37781f2a80 con 0x7f37780b0680 2023-10-08T07:15:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.797+0000 7f377f052640 1 -- 172.21.15.70:0/45462986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377815c560 con 0x7f37780b0680 2023-10-08T07:15:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.798+0000 7f376effd640 1 -- 172.21.15.70:0/45462986 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f377401a400 con 0x7f37780b0680 2023-10-08T07:15:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.799+0000 7f376effd640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f375c06b0b0 0x7f375c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.799+0000 7f376effd640 1 -- 172.21.15.70:0/45462986 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f3774089df0 con 0x7f37780b0680 2023-10-08T07:15:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.800+0000 7f377d84f640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f375c06b0b0 0x7f375c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:00.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.801+0000 7f377d84f640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f375c06b0b0 0x7f375c06d570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3768036000 tx=0x7f3768030000 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:00.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:00.804+0000 7f376effd640 1 -- 172.21.15.70:0/45462986 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f377815c560 con 0x7f37780b0680 2023-10-08T07:15:01.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.027+0000 7f377f052640 1 -- 172.21.15.70:0/45462986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3778074150 con 0x7f37780b0680 2023-10-08T07:15:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.562+0000 7f376effd640 1 -- 172.21.15.70:0/45462986 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v130) v1 ==== 125+0+0 (secure 0 0 0) 0x7f37740150c0 con 0x7f37780b0680 2023-10-08T07:15:01.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.565+0000 7f377f052640 1 -- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f375c06b0b0 msgr2=0x7f375c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:01.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.565+0000 7f377f052640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f375c06b0b0 0x7f375c06d570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3768036000 tx=0x7f3768030000 comp rx=0 tx=0).stop 2023-10-08T07:15:01.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.565+0000 7f377f052640 1 -- 172.21.15.70:0/45462986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 msgr2=0x7f37781f1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:01.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.565+0000 7f377f052640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 0x7f37781f1e40 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f377400b3f0 tx=0x7f377400b8b0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.566+0000 7f377f052640 1 -- 172.21.15.70:0/45462986 shutdown_connections 2023-10-08T07:15:01.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.566+0000 7f377f052640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f375c06b0b0 0x7f375c06d570 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.566+0000 7f377f052640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f377815a820 0x7f3778158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.566+0000 7f377f052640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f37780ca680 0x7f37781f5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.566+0000 7f377f052640 1 --2- 172.21.15.70:0/45462986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37780b0680 0x7f37781f1e40 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.566+0000 7f377f052640 1 -- 172.21.15.70:0/45462986 >> 172.21.15.70:0/45462986 conn(0x7f377806c050 msgr2=0x7f37780c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.566+0000 7f377f052640 1 -- 172.21.15.70:0/45462986 shutdown_connections 2023-10-08T07:15:01.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.566+0000 7f377f052640 1 -- 172.21.15.70:0/45462986 wait complete. 2023-10-08T07:15:01.586 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-10-08T07:15:01.586 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:01.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.799+0000 7fcd36813640 1 Processor -- start 2023-10-08T07:15:01.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.799+0000 7fcd36813640 1 -- start start 2023-10-08T07:15:01.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.801+0000 7fcd36813640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcd300adc70 0x7fcd300b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:01.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.801+0000 7fcd36813640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3015f7d0 con 0x7fcd3015ab30 2023-10-08T07:15:01.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.801+0000 7fcd36813640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3015efb0 con 0x7fcd3015af10 2023-10-08T07:15:01.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.801+0000 7fcd36813640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3015f210 con 0x7fcd300adc70 2023-10-08T07:15:01.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.801+0000 7fcd35811640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcd3015ab30 0x7fcd3015e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52802/0 (socket says 172.21.15.70:52802) 2023-10-08T07:15:01.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.801+0000 7fcd35811640 1 -- 172.21.15.70:0/1821310626 learned_addr learned my addr 172.21.15.70:0/1821310626 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.801+0000 7fcd35010640 1 --2- 172.21.15.70:0/1821310626 >> v2:172.21.15.181:3300/0 conn(0x7fcd300adc70 0x7fcd300b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.801+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889464908 0 0) 0x7fcd3015f7d0 con 0x7fcd3015ab30 2023-10-08T07:15:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd1c003520 con 0x7fcd3015ab30 2023-10-08T07:15:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 126259594 0 0) 0x7fcd1c003520 con 0x7fcd3015ab30 2023-10-08T07:15:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 >> v2:172.21.15.181:3300/0 conn(0x7fcd300adc70 msgr2=0x7fcd300b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 --2- 172.21.15.70:0/1821310626 >> v2:172.21.15.181:3300/0 conn(0x7fcd300adc70 0x7fcd300b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 >> v1:172.21.15.181:6789/0 conn(0x7fcd3015af10 legacy=0x7fcd3006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3015f4f0 con 0x7fcd3015ab30 2023-10-08T07:15:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fcd2c001690 con 0x7fcd3015ab30 2023-10-08T07:15:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd35010640 1 --2- 172.21.15.70:0/1821310626 >> v2:172.21.15.181:3300/0 conn(0x7fcd300adc70 0x7fcd300b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 >> v1:172.21.15.70:6789/0 conn(0x7fcd3015ab30 legacy=0x7fcd3015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 --2- 172.21.15.70:0/1821310626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 --2- 172.21.15.70:0/1821310626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0047c0 0x7fcd1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 --2- 172.21.15.70:0/1821310626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd1c005180 0x7fcd1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3015f210 con 0x7fcd1c003970 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c00c0d0 con 0x7fcd1c0047c0 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c00c290 con 0x7fcd1c005180 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.802+0000 7fcd35811640 1 --2- 172.21.15.70:0/1821310626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.803+0000 7fcd35811640 1 -- 172.21.15.70:0/1821310626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd1c005180 msgr2=0x7fcd1c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.803+0000 7fcd35811640 1 --2- 172.21.15.70:0/1821310626 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd1c005180 0x7fcd1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.803+0000 7fcd35811640 1 -- 172.21.15.70:0/1821310626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0047c0 msgr2=0x7fcd1c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.803+0000 7fcd35811640 1 --2- 172.21.15.70:0/1821310626 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd1c0047c0 0x7fcd1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.803+0000 7fcd35811640 1 -- 172.21.15.70:0/1821310626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c003520 con 0x7fcd1c003970 2023-10-08T07:15:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.803+0000 7fcd36012640 1 -- 172.21.15.70:0/1821310626 reap_dead start 2023-10-08T07:15:01.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.804+0000 7fcd35811640 1 --2- 172.21.15.70:0/1821310626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd1c003d90 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fcd3015f960 tx=0x7fcd1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19edf922458202e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:01.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.804+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c005ad0 con 0x7fcd1c003970 2023-10-08T07:15:01.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.804+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c005c30 con 0x7fcd1c003970 2023-10-08T07:15:01.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.804+0000 7fcd3480f640 1 -- 172.21.15.70:0/1821310626 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c02c8a0 con 0x7fcd1c003970 2023-10-08T07:15:01.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.805+0000 7fcd36813640 1 -- 172.21.15.70:0/1821310626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 msgr2=0x7fcd1c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:01.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.805+0000 7fcd36813640 1 --2- 172.21.15.70:0/1821310626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd1c003d90 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7fcd3015f960 tx=0x7fcd1c0057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.805+0000 7fcd36813640 1 -- 172.21.15.70:0/1821310626 shutdown_connections 2023-10-08T07:15:01.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.805+0000 7fcd36813640 1 --2- 172.21.15.70:0/1821310626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd1c003d90 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.805+0000 7fcd36813640 1 -- 172.21.15.70:0/1821310626 >> 172.21.15.70:0/1821310626 conn(0x7fcd300cb0c0 msgr2=0x7fcd300cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:01.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.805+0000 7fcd36813640 1 -- 172.21.15.70:0/1821310626 shutdown_connections 2023-10-08T07:15:01.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.805+0000 7fcd36813640 1 -- 172.21.15.70:0/1821310626 wait complete. 2023-10-08T07:15:01.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.806+0000 7fcd36813640 1 Processor -- start 2023-10-08T07:15:01.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.807+0000 7fcd36813640 1 -- start start 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd36813640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd30143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd36813640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd30159510 0x7fcd30143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd36813640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd30144490 0x7fcd30144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd36813640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30158500 con 0x7fcd1c003970 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd36813640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3015b6b0 con 0x7fcd30159510 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd36813640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30158750 con 0x7fcd30144490 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd35811640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd30143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd35811640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd30143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58338/0 (socket says 172.21.15.70:58338) 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd35811640 1 -- 172.21.15.70:0/1284068630 learned_addr learned my addr 172.21.15.70:0/1284068630 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd35811640 1 -- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd30144490 msgr2=0x7fcd30144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:01.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd35811640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd30144490 0x7fcd30144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd35811640 1 -- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd30159510 msgr2=0x7fcd30143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd35811640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd30159510 0x7fcd30143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.808+0000 7fcd35811640 1 -- 172.21.15.70:0/1284068630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd30158a30 con 0x7fcd1c003970 2023-10-08T07:15:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.809+0000 7fcd35811640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd30143650 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c02c440 tx=0x7fcd2c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.809+0000 7fcd267fc640 1 -- 172.21.15.70:0/1284068630 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c03c040 con 0x7fcd1c003970 2023-10-08T07:15:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.809+0000 7fcd267fc640 1 -- 172.21.15.70:0/1284068630 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c02cd70 con 0x7fcd1c003970 2023-10-08T07:15:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.809+0000 7fcd36813640 1 -- 172.21.15.70:0/1284068630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd30158d10 con 0x7fcd1c003970 2023-10-08T07:15:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.809+0000 7fcd267fc640 1 -- 172.21.15.70:0/1284068630 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c007290 con 0x7fcd1c003970 2023-10-08T07:15:01.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.809+0000 7fcd36813640 1 -- 172.21.15.70:0/1284068630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd30143b90 con 0x7fcd1c003970 2023-10-08T07:15:01.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.810+0000 7fcd36813640 1 -- 172.21.15.70:0/1284068630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd300c9ca0 con 0x7fcd1c003970 2023-10-08T07:15:01.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.811+0000 7fcd267fc640 1 -- 172.21.15.70:0/1284068630 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcd2c032070 con 0x7fcd1c003970 2023-10-08T07:15:01.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.812+0000 7fcd267fc640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd1406b0e0 0x7fcd1406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:01.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.812+0000 7fcd267fc640 1 -- 172.21.15.70:0/1284068630 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fcd2c0ab120 con 0x7fcd1c003970 2023-10-08T07:15:01.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.813+0000 7fcd35010640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd1406b0e0 0x7fcd1406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:01.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.814+0000 7fcd35010640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd1406b0e0 0x7fcd1406d5a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd20006fd0 tx=0x7fcd20008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:01.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:01.816+0000 7fcd267fc640 1 -- 172.21.15.70:0/1284068630 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd2c075940 con 0x7fcd1c003970 2023-10-08T07:15:02.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.042+0000 7fcd36813640 1 -- 172.21.15.70:0/1284068630 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd3004e160 con 0x7fcd1c003970 2023-10-08T07:15:02.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.043+0000 7fcd267fc640 1 -- 172.21.15.70:0/1284068630 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+6149 (secure 0 0 0) 0x7fcd2c045090 con 0x7fcd1c003970 2023-10-08T07:15:02.044 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:02.044 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":130,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:43.833416+0000","modified":"2023-10-08T07:14:46.899347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11904},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11904":{"gid":11904,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/3856326680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3856326680},{"type":"v1","addr":"172.21.15.70:6834","nonce":3856326680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:57.955428+0000","modified":"2023-10-08T07:15:01.029708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8875":{"gid":8875,"name":"b","rank":0,"incarnation":127,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-08T07:15:02.044 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 130 2023-10-08T07:15:02.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.045+0000 7fcd36813640 1 -- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd1406b0e0 msgr2=0x7fcd1406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.045+0000 7fcd36813640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd1406b0e0 0x7fcd1406d5a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd20006fd0 tx=0x7fcd20008040 comp rx=0 tx=0).stop 2023-10-08T07:15:02.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.045+0000 7fcd36813640 1 -- 172.21.15.70:0/1284068630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 msgr2=0x7fcd30143650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.045+0000 7fcd36813640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd30143650 secure :-1 s=READY pgs=2696 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c02c440 tx=0x7fcd2c02e070 comp rx=0 tx=0).stop 2023-10-08T07:15:02.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.046+0000 7fcd36813640 1 -- 172.21.15.70:0/1284068630 shutdown_connections 2023-10-08T07:15:02.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.046+0000 7fcd36813640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fcd1406b0e0 0x7fcd1406d5a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.046+0000 7fcd36813640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcd30144490 0x7fcd30144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.046+0000 7fcd36813640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcd30159510 0x7fcd30143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.046+0000 7fcd36813640 1 --2- 172.21.15.70:0/1284068630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd1c003970 0x7fcd30143650 unknown :-1 s=CLOSED pgs=2696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.046+0000 7fcd36813640 1 -- 172.21.15.70:0/1284068630 >> 172.21.15.70:0/1284068630 conn(0x7fcd300cb0c0 msgr2=0x7fcd300cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:02.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.046+0000 7fcd36813640 1 -- 172.21.15.70:0/1284068630 shutdown_connections 2023-10-08T07:15:02.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.046+0000 7fcd36813640 1 -- 172.21.15.70:0/1284068630 wait complete. 2023-10-08T07:15:02.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:02.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.281+0000 7ff5d4139640 1 Processor -- start 2023-10-08T07:15:02.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.281+0000 7ff5d4139640 1 -- start start 2023-10-08T07:15:02.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.282+0000 7ff5d4139640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5cc15ab40 0x7ff5cc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.282+0000 7ff5d4139640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff5cc0c90b0 0x7ff5cc0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.282+0000 7ff5d4139640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5cc15b2b0 con 0x7ff5cc0cae70 2023-10-08T07:15:02.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.282+0000 7ff5d4139640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc15bf30 con 0x7ff5cc15ab40 2023-10-08T07:15:02.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.282+0000 7ff5d4139640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc15f7a0 con 0x7ff5cc0c90b0 2023-10-08T07:15:02.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.283+0000 7ff5d1ed5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5cc15ab40 0x7ff5cc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.283+0000 7ff5d1ed5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5cc15ab40 0x7ff5cc0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58342/0 (socket says 172.21.15.70:58342) 2023-10-08T07:15:02.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.283+0000 7ff5d1ed5640 1 -- 172.21.15.70:0/4132911457 learned_addr learned my addr 172.21.15.70:0/4132911457 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:02.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.283+0000 7ff5d16d4640 1 --2- 172.21.15.70:0/4132911457 >> v2:172.21.15.181:3300/0 conn(0x7ff5cc0c90b0 0x7ff5cc0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.283+0000 7ff5d1ed5640 1 -- 172.21.15.70:0/4132911457 >> v2:172.21.15.181:3300/0 conn(0x7ff5cc0c90b0 msgr2=0x7ff5cc0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:02.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.283+0000 7ff5d1ed5640 1 --2- 172.21.15.70:0/4132911457 >> v2:172.21.15.181:3300/0 conn(0x7ff5cc0c90b0 0x7ff5cc0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.283+0000 7ff5d1ed5640 1 -- 172.21.15.70:0/4132911457 >> v1:172.21.15.181:6789/0 conn(0x7ff5cc0cae70 legacy=0x7ff5cc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:02.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.283+0000 7ff5d1ed5640 1 -- 172.21.15.70:0/4132911457 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc15b590 con 0x7ff5cc15ab40 2023-10-08T07:15:02.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.285+0000 7ff5d1ed5640 1 --2- 172.21.15.70:0/4132911457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 0x7ff5cc0ad720 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc06e560 tx=0x7ff5c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abcc4535307d7b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:02.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.285+0000 7ff5d0ed3640 1 -- 172.21.15.70:0/4132911457 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8031870 con 0x7ff5cc15ab40 2023-10-08T07:15:02.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.285+0000 7ff5d0ed3640 1 -- 172.21.15.70:0/4132911457 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c80319d0 con 0x7ff5cc15ab40 2023-10-08T07:15:02.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.285+0000 7ff5d0ed3640 1 -- 172.21.15.70:0/4132911457 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8031d60 con 0x7ff5cc15ab40 2023-10-08T07:15:02.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.286+0000 7ff5d4139640 1 -- 172.21.15.70:0/4132911457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 msgr2=0x7ff5cc0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.286+0000 7ff5d4139640 1 --2- 172.21.15.70:0/4132911457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 0x7ff5cc0ad720 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc06e560 tx=0x7ff5c802b910 comp rx=0 tx=0).stop 2023-10-08T07:15:02.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.286+0000 7ff5d4139640 1 -- 172.21.15.70:0/4132911457 shutdown_connections 2023-10-08T07:15:02.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.286+0000 7ff5d4139640 1 --2- 172.21.15.70:0/4132911457 >> v2:172.21.15.181:3300/0 conn(0x7ff5cc0c90b0 0x7ff5cc0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.286+0000 7ff5d4139640 1 --2- 172.21.15.70:0/4132911457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 0x7ff5cc0ad720 unknown :-1 s=CLOSED pgs=2697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.286+0000 7ff5d4139640 1 -- 172.21.15.70:0/4132911457 >> 172.21.15.70:0/4132911457 conn(0x7ff5cc06a380 msgr2=0x7ff5cc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:02.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.286+0000 7ff5d4139640 1 -- 172.21.15.70:0/4132911457 shutdown_connections 2023-10-08T07:15:02.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.286+0000 7ff5d4139640 1 -- 172.21.15.70:0/4132911457 wait complete. 2023-10-08T07:15:02.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.288+0000 7ff5d4139640 1 Processor -- start 2023-10-08T07:15:02.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.288+0000 7ff5d4139640 1 -- start start 2023-10-08T07:15:02.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.289+0000 7ff5d4139640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5cc0c90b0 0x7ff5cc1f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.289+0000 7ff5d4139640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5cc0cae70 0x7ff5cc15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.289+0000 7ff5d4139640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 0x7ff5cc15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.289+0000 7ff5d4139640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc0c43e0 con 0x7ff5cc15ab40 2023-10-08T07:15:02.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.289+0000 7ff5d4139640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc0c46c0 con 0x7ff5cc0c90b0 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.289+0000 7ff5d4139640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc0c49a0 con 0x7ff5cc0cae70 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.289+0000 7ff5d26d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 0x7ff5cc15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.289+0000 7ff5d26d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 0x7ff5cc15d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58358/0 (socket says 172.21.15.70:58358) 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.289+0000 7ff5d26d6640 1 -- 172.21.15.70:0/5938538 learned_addr learned my addr 172.21.15.70:0/5938538 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.290+0000 7ff5d1ed5640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5cc0c90b0 0x7ff5cc1f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.290+0000 7ff5d16d4640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5cc0cae70 0x7ff5cc15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.290+0000 7ff5d26d6640 1 -- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5cc0cae70 msgr2=0x7ff5cc15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.290+0000 7ff5d26d6640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5cc0cae70 0x7ff5cc15d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.290+0000 7ff5d26d6640 1 -- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5cc0c90b0 msgr2=0x7ff5cc1f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.290+0000 7ff5d26d6640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5cc0c90b0 0x7ff5cc1f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.290+0000 7ff5d26d6640 1 -- 172.21.15.70:0/5938538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80094d0 con 0x7ff5cc15ab40 2023-10-08T07:15:02.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.290+0000 7ff5d26d6640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 0x7ff5cc15d720 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400c780 tx=0x7ff5c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:02.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.290+0000 7ff5c27fc640 1 -- 172.21.15.70:0/5938538 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4004b60 con 0x7ff5cc15ab40 2023-10-08T07:15:02.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.291+0000 7ff5c27fc640 1 -- 172.21.15.70:0/5938538 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c400ce90 con 0x7ff5cc15ab40 2023-10-08T07:15:02.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.291+0000 7ff5c27fc640 1 -- 172.21.15.70:0/5938538 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c400a850 con 0x7ff5cc15ab40 2023-10-08T07:15:02.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.291+0000 7ff5d4139640 1 -- 172.21.15.70:0/5938538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5cc0c4c80 con 0x7ff5cc15ab40 2023-10-08T07:15:02.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.291+0000 7ff5d4139640 1 -- 172.21.15.70:0/5938538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc1f21c0 con 0x7ff5cc15ab40 2023-10-08T07:15:02.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.292+0000 7ff5d4139640 1 -- 172.21.15.70:0/5938538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5cc1f2380 con 0x7ff5cc15ab40 2023-10-08T07:15:02.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.293+0000 7ff5c27fc640 1 -- 172.21.15.70:0/5938538 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff5c400a9b0 con 0x7ff5cc15ab40 2023-10-08T07:15:02.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.294+0000 7ff5c27fc640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff5b006b0b0 0x7ff5b006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.294+0000 7ff5c27fc640 1 -- 172.21.15.70:0/5938538 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7ff5c408a1c0 con 0x7ff5cc15ab40 2023-10-08T07:15:02.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.294+0000 7ff5d1ed5640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff5b006b0b0 0x7ff5b006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.295+0000 7ff5d1ed5640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff5b006b0b0 0x7ff5b006d570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8036000 tx=0x7ff5c803c040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:02.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.298+0000 7ff5c27fc640 1 -- 172.21.15.70:0/5938538 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5c4054950 con 0x7ff5cc15ab40 2023-10-08T07:15:02.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.486+0000 7ff5d4139640 1 -- 172.21.15.70:0/5938538 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5cc067490 con 0x7ff5cc15ab40 2023-10-08T07:15:02.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.488+0000 7ff5c27fc640 1 -- 172.21.15.70:0/5938538 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v258) v1 ==== 74+0+20618 (secure 0 0 0) 0x7ff5c4057f70 con 0x7ff5cc15ab40 2023-10-08T07:15:02.489 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:02.489 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":258,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:14:59.533093+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:14:56.178783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:14:56.977158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:02.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.490+0000 7ff5d4139640 1 -- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff5b006b0b0 msgr2=0x7ff5b006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.490+0000 7ff5d4139640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff5b006b0b0 0x7ff5b006d570 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8036000 tx=0x7ff5c803c040 comp rx=0 tx=0).stop 2023-10-08T07:15:02.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.490+0000 7ff5d4139640 1 -- 172.21.15.70:0/5938538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 msgr2=0x7ff5cc15d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.490+0000 7ff5d4139640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 0x7ff5cc15d720 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400c780 tx=0x7ff5c400cc40 comp rx=0 tx=0).stop 2023-10-08T07:15:02.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.491+0000 7ff5d4139640 1 -- 172.21.15.70:0/5938538 shutdown_connections 2023-10-08T07:15:02.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.491+0000 7ff5d4139640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff5b006b0b0 0x7ff5b006d570 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.491+0000 7ff5d4139640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5cc15ab40 0x7ff5cc15d720 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.491+0000 7ff5d4139640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff5cc0cae70 0x7ff5cc15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.491+0000 7ff5d4139640 1 --2- 172.21.15.70:0/5938538 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff5cc0c90b0 0x7ff5cc1f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.491+0000 7ff5d4139640 1 -- 172.21.15.70:0/5938538 >> 172.21.15.70:0/5938538 conn(0x7ff5cc06a380 msgr2=0x7ff5cc0c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:02.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.491+0000 7ff5d4139640 1 -- 172.21.15.70:0/5938538 shutdown_connections 2023-10-08T07:15:02.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.491+0000 7ff5d4139640 1 -- 172.21.15.70:0/5938538 wait complete. 2023-10-08T07:15:02.509 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T07:15:02.509 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:15:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.714+0000 7fd350abb640 1 Processor -- start 2023-10-08T07:15:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.714+0000 7fd350abb640 1 -- start start 2023-10-08T07:15:02.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.716+0000 7fd350abb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd34c0ad790 0x7fd34c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.716+0000 7fd350abb640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fd34c15e330 0x7fd34c070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.716+0000 7fd350abb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34c06dee0 con 0x7fd34c159ac0 2023-10-08T07:15:02.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.716+0000 7fd350abb640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34c06dcf0 con 0x7fd34c0ad790 2023-10-08T07:15:02.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.716+0000 7fd350abb640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34c15d850 con 0x7fd34c15e330 2023-10-08T07:15:02.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.716+0000 7fd34a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd34c159ac0 0x7fd34c06d5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52816/0 (socket says 172.21.15.70:52816) 2023-10-08T07:15:02.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.716+0000 7fd34a59c640 1 -- 172.21.15.70:0/1183483866 learned_addr learned my addr 172.21.15.70:0/1183483866 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:02.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.716+0000 7fd349d9b640 1 --2- 172.21.15.70:0/1183483866 >> v2:172.21.15.70:3300/0 conn(0x7fd34c0ad790 0x7fd34c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.717+0000 7fd34ad9d640 1 --2- 172.21.15.70:0/1183483866 >> v2:172.21.15.181:3300/0 conn(0x7fd34c15e330 0x7fd34c070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.717+0000 7fd34959a640 1 -- 172.21.15.70:0/1183483866 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199318758 0 0) 0x7fd34c06dee0 con 0x7fd34c159ac0 2023-10-08T07:15:02.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.717+0000 7fd34959a640 1 -- 172.21.15.70:0/1183483866 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd334003520 con 0x7fd34c159ac0 2023-10-08T07:15:02.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.717+0000 7fd349d9b640 1 -- 172.21.15.70:0/1183483866 >> v2:172.21.15.181:3300/0 conn(0x7fd34c15e330 msgr2=0x7fd34c070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:02.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.717+0000 7fd349d9b640 1 --2- 172.21.15.70:0/1183483866 >> v2:172.21.15.181:3300/0 conn(0x7fd34c15e330 0x7fd34c070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.717+0000 7fd349d9b640 1 -- 172.21.15.70:0/1183483866 >> v1:172.21.15.70:6789/0 conn(0x7fd34c159ac0 legacy=0x7fd34c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.717+0000 7fd349d9b640 1 -- 172.21.15.70:0/1183483866 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c15db30 con 0x7fd34c0ad790 2023-10-08T07:15:02.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.718+0000 7fd349d9b640 1 --2- 172.21.15.70:0/1183483866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 0x7fd34c0702d0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0caab0 tx=0x7fd34002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821b3e688cd1d4b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:02.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.718+0000 7fd34959a640 1 -- 172.21.15.70:0/1183483866 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd340031830 con 0x7fd34c0ad790 2023-10-08T07:15:02.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.718+0000 7fd34959a640 1 -- 172.21.15.70:0/1183483866 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd340031990 con 0x7fd34c0ad790 2023-10-08T07:15:02.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.718+0000 7fd34959a640 1 -- 172.21.15.70:0/1183483866 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd340031d20 con 0x7fd34c0ad790 2023-10-08T07:15:02.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.719+0000 7fd350abb640 1 -- 172.21.15.70:0/1183483866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 msgr2=0x7fd34c0702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.719+0000 7fd350abb640 1 --2- 172.21.15.70:0/1183483866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 0x7fd34c0702d0 secure :-1 s=READY pgs=2700 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0caab0 tx=0x7fd34002b8d0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.719+0000 7fd350abb640 1 -- 172.21.15.70:0/1183483866 shutdown_connections 2023-10-08T07:15:02.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.719+0000 7fd350abb640 1 --2- 172.21.15.70:0/1183483866 >> v2:172.21.15.181:3300/0 conn(0x7fd34c15e330 0x7fd34c070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.719+0000 7fd350abb640 1 --2- 172.21.15.70:0/1183483866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 0x7fd34c0702d0 unknown :-1 s=CLOSED pgs=2700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.719+0000 7fd350abb640 1 -- 172.21.15.70:0/1183483866 >> 172.21.15.70:0/1183483866 conn(0x7fd34c06a380 msgr2=0x7fd34c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:02.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.719+0000 7fd350abb640 1 -- 172.21.15.70:0/1183483866 shutdown_connections 2023-10-08T07:15:02.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.719+0000 7fd350abb640 1 -- 172.21.15.70:0/1183483866 wait complete. 2023-10-08T07:15:02.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.721+0000 7fd350abb640 1 Processor -- start 2023-10-08T07:15:02.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.721+0000 7fd350abb640 1 -- start start 2023-10-08T07:15:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd350abb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 0x7fd34c1f0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd350abb640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd34c159ac0 0x7fd34c1f0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd350abb640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd34c15e330 0x7fd34c0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd350abb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c15a020 con 0x7fd34c0ad790 2023-10-08T07:15:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd350abb640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c0cd090 con 0x7fd34c15e330 2023-10-08T07:15:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd350abb640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c159130 con 0x7fd34c159ac0 2023-10-08T07:15:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd34a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 0x7fd34c1f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd34a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 0x7fd34c1f0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58378/0 (socket says 172.21.15.70:58378) 2023-10-08T07:15:02.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd34a59c640 1 -- 172.21.15.70:0/1223840175 learned_addr learned my addr 172.21.15.70:0/1223840175 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:02.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd34a59c640 1 -- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd34c159ac0 msgr2=0x7fd34c1f0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd34a59c640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd34c159ac0 0x7fd34c1f0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd34a59c640 1 -- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd34c15e330 msgr2=0x7fd34c0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.722+0000 7fd34ad9d640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd34c15e330 0x7fd34c0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.723+0000 7fd34a59c640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd34c15e330 0x7fd34c0cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.723+0000 7fd34a59c640 1 -- 172.21.15.70:0/1223840175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd340009400 con 0x7fd34c0ad790 2023-10-08T07:15:02.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.723+0000 7fd34a59c640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 0x7fd34c1f0010 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c500 tx=0x7fd33c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:02.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.723+0000 7fd33affd640 1 -- 172.21.15.70:0/1223840175 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c005ce0 con 0x7fd34c0ad790 2023-10-08T07:15:02.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.723+0000 7fd33affd640 1 -- 172.21.15.70:0/1223840175 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c005e40 con 0x7fd34c0ad790 2023-10-08T07:15:02.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.723+0000 7fd350abb640 1 -- 172.21.15.70:0/1223840175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34c15a230 con 0x7fd34c0ad790 2023-10-08T07:15:02.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.723+0000 7fd33affd640 1 -- 172.21.15.70:0/1223840175 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c007330 con 0x7fd34c0ad790 2023-10-08T07:15:02.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.723+0000 7fd350abb640 1 -- 172.21.15.70:0/1223840175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c158de0 con 0x7fd34c0ad790 2023-10-08T07:15:02.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.724+0000 7fd350abb640 1 -- 172.21.15.70:0/1223840175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34c15a750 con 0x7fd34c0ad790 2023-10-08T07:15:02.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.725+0000 7fd33affd640 1 -- 172.21.15.70:0/1223840175 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fd33c007490 con 0x7fd34c0ad790 2023-10-08T07:15:02.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.726+0000 7fd33affd640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fd32c06b0e0 0x7fd32c06d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:02.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.726+0000 7fd33affd640 1 -- 172.21.15.70:0/1223840175 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fd33c089620 con 0x7fd34c0ad790 2023-10-08T07:15:02.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.727+0000 7fd349d9b640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fd32c06b0e0 0x7fd32c06d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:02.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.728+0000 7fd349d9b640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fd32c06b0e0 0x7fd32c06d5a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd340036000 tx=0x7fd340030000 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:02.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.730+0000 7fd33affd640 1 -- 172.21.15.70:0/1223840175 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd33c057990 con 0x7fd34c0ad790 2023-10-08T07:15:02.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.903+0000 7fd350abb640 1 -- 172.21.15.70:0/1223840175 --> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd34c159ea0 con 0x7fd32c06b0e0 2023-10-08T07:15:02.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.948+0000 7fd33affd640 1 -- 172.21.15.70:0/1223840175 <== mgr.8662 v2:172.21.15.181:6832/4245028292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526378 (secure 0 0 0) 0x7fd340031630 con 0x7fd32c06b0e0 2023-10-08T07:15:02.955 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:15:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 -- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fd32c06b0e0 msgr2=0x7fd32c06d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fd32c06b0e0 0x7fd32c06d5a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd340036000 tx=0x7fd340030000 comp rx=0 tx=0).stop 2023-10-08T07:15:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 -- 172.21.15.70:0/1223840175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 msgr2=0x7fd34c1f0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 0x7fd34c1f0010 secure :-1 s=READY pgs=2701 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c500 tx=0x7fd33c00c9c0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 -- 172.21.15.70:0/1223840175 shutdown_connections 2023-10-08T07:15:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fd32c06b0e0 0x7fd32c06d5a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd34c15e330 0x7fd34c0cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd34c159ac0 0x7fd34c1f0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 --2- 172.21.15.70:0/1223840175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd34c0ad790 0x7fd34c1f0010 unknown :-1 s=CLOSED pgs=2701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 -- 172.21.15.70:0/1223840175 >> 172.21.15.70:0/1223840175 conn(0x7fd34c06a380 msgr2=0x7fd34c0cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 -- 172.21.15.70:0/1223840175 shutdown_connections 2023-10-08T07:15:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:02.955+0000 7fd350abb640 1 -- 172.21.15.70:0/1223840175 wait complete. 2023-10-08T07:15:02.973 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-10-08T07:15:02.872278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":720,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2907,"ondisk_log_size":2907,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410800,"kb_used_data":26664,"kb_used_omap":12,"kb_used_meta":383987,"kb_avail":754563920,"statfs":{"total":773094113280,"available":772673454080,"internally_reserved":0,"allocated":27303936,"data_stored":14783634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":393203259},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3397,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001930"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.570503+0000","last_change":"2023-10-08T07:14:58.550530+0000","last_active":"2023-10-08T07:14:59.570503+0000","last_peered":"2023-10-08T07:14:59.570503+0000","last_clean":"2023-10-08T07:14:59.570503+0000","last_became_active":"2023-10-08T07:14:58.550249+0000","last_became_peered":"2023-10-08T07:14:58.550249+0000","last_unstale":"2023-10-08T07:14:59.570503+0000","last_undegraded":"2023-10-08T07:14:59.570503+0000","last_fullsized":"2023-10-08T07:14:59.570503+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:02.313521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"257'1","reported_seq":13,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.566998+0000","last_change":"2023-10-08T07:14:57.517188+0000","last_active":"2023-10-08T07:14:58.566998+0000","last_peered":"2023-10-08T07:14:58.566998+0000","last_clean":"2023-10-08T07:14:58.566998+0000","last_became_active":"2023-10-08T07:14:57.516848+0000","last_became_peered":"2023-10-08T07:14:57.516848+0000","last_unstale":"2023-10-08T07:14:58.566998+0000","last_undegraded":"2023-10-08T07:14:58.566998+0000","last_fullsized":"2023-10-08T07:14:58.566998+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:47.146297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533488+0000","last_change":"2023-10-08T07:14:44.322566+0000","last_active":"2023-10-08T07:14:58.533488+0000","last_peered":"2023-10-08T07:14:58.533488+0000","last_clean":"2023-10-08T07:14:58.533488+0000","last_became_active":"2023-10-08T07:14:44.321968+0000","last_became_peered":"2023-10-08T07:14:44.321968+0000","last_unstale":"2023-10-08T07:14:58.533488+0000","last_undegraded":"2023-10-08T07:14:58.533488+0000","last_fullsized":"2023-10-08T07:14:58.533488+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:33.364440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546558+0000","last_change":"2023-10-08T07:14:58.552444+0000","last_active":"2023-10-08T07:14:59.546558+0000","last_peered":"2023-10-08T07:14:59.546558+0000","last_clean":"2023-10-08T07:14:59.546558+0000","last_became_active":"2023-10-08T07:14:58.552072+0000","last_became_peered":"2023-10-08T07:14:58.552072+0000","last_unstale":"2023-10-08T07:14:59.546558+0000","last_undegraded":"2023-10-08T07:14:59.546558+0000","last_fullsized":"2023-10-08T07:14:59.546558+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:01.516979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"257'4","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.570292+0000","last_change":"2023-10-08T07:14:57.523739+0000","last_active":"2023-10-08T07:14:58.570292+0000","last_peered":"2023-10-08T07:14:58.570292+0000","last_clean":"2023-10-08T07:14:58.570292+0000","last_became_active":"2023-10-08T07:14:57.523490+0000","last_became_peered":"2023-10-08T07:14:57.523490+0000","last_unstale":"2023-10-08T07:14:58.570292+0000","last_undegraded":"2023-10-08T07:14:58.570292+0000","last_fullsized":"2023-10-08T07:14:58.570292+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:35.857790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.535845+0000","last_change":"2023-10-08T07:14:44.336800+0000","last_active":"2023-10-08T07:14:58.535845+0000","last_peered":"2023-10-08T07:14:58.535845+0000","last_clean":"2023-10-08T07:14:58.535845+0000","last_became_active":"2023-10-08T07:14:44.336534+0000","last_became_peered":"2023-10-08T07:14:44.336534+0000","last_unstale":"2023-10-08T07:14:58.535845+0000","last_undegraded":"2023-10-08T07:14:58.535845+0000","last_fullsized":"2023-10-08T07:14:58.535845+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:48.002372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539715+0000","last_change":"2023-10-08T07:14:58.539613+0000","last_active":"2023-10-08T07:14:59.539715+0000","last_peered":"2023-10-08T07:14:59.539715+0000","last_clean":"2023-10-08T07:14:59.539715+0000","last_became_active":"2023-10-08T07:14:58.539279+0000","last_became_peered":"2023-10-08T07:14:58.539279+0000","last_unstale":"2023-10-08T07:14:59.539715+0000","last_undegraded":"2023-10-08T07:14:59.539715+0000","last_fullsized":"2023-10-08T07:14:59.539715+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:44.794204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.571271+0000","last_change":"2023-10-08T07:14:57.519020+0000","last_active":"2023-10-08T07:14:58.571271+0000","last_peered":"2023-10-08T07:14:58.571271+0000","last_clean":"2023-10-08T07:14:58.571271+0000","last_became_active":"2023-10-08T07:14:57.518593+0000","last_became_peered":"2023-10-08T07:14:57.518593+0000","last_unstale":"2023-10-08T07:14:58.571271+0000","last_undegraded":"2023-10-08T07:14:58.571271+0000","last_fullsized":"2023-10-08T07:14:58.571271+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:16.811954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540531+0000","last_change":"2023-10-08T07:14:44.331190+0000","last_active":"2023-10-08T07:14:58.540531+0000","last_peered":"2023-10-08T07:14:58.540531+0000","last_clean":"2023-10-08T07:14:58.540531+0000","last_became_active":"2023-10-08T07:14:44.330346+0000","last_became_peered":"2023-10-08T07:14:44.330346+0000","last_unstale":"2023-10-08T07:14:58.540531+0000","last_undegraded":"2023-10-08T07:14:58.540531+0000","last_fullsized":"2023-10-08T07:14:58.540531+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:54:20.560371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.570807+0000","last_change":"2023-10-08T07:14:58.542547+0000","last_active":"2023-10-08T07:14:59.570807+0000","last_peered":"2023-10-08T07:14:59.570807+0000","last_clean":"2023-10-08T07:14:59.570807+0000","last_became_active":"2023-10-08T07:14:58.542261+0000","last_became_peered":"2023-10-08T07:14:58.542261+0000","last_unstale":"2023-10-08T07:14:59.570807+0000","last_undegraded":"2023-10-08T07:14:59.570807+0000","last_fullsized":"2023-10-08T07:14:59.570807+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:31.265009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.571504+0000","last_change":"2023-10-08T07:14:57.516009+0000","last_active":"2023-10-08T07:14:58.571504+0000","last_peered":"2023-10-08T07:14:58.571504+0000","last_clean":"2023-10-08T07:14:58.571504+0000","last_became_active":"2023-10-08T07:14:57.515464+0000","last_became_peered":"2023-10-08T07:14:57.515464+0000","last_unstale":"2023-10-08T07:14:58.571504+0000","last_undegraded":"2023-10-08T07:14:58.571504+0000","last_fullsized":"2023-10-08T07:14:58.571504+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:04.514485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540123+0000","last_change":"2023-10-08T07:14:44.337611+0000","last_active":"2023-10-08T07:14:58.540123+0000","last_peered":"2023-10-08T07:14:58.540123+0000","last_clean":"2023-10-08T07:14:58.540123+0000","last_became_active":"2023-10-08T07:14:44.336777+0000","last_became_peered":"2023-10-08T07:14:44.336777+0000","last_unstale":"2023-10-08T07:14:58.540123+0000","last_undegraded":"2023-10-08T07:14:58.540123+0000","last_fullsized":"2023-10-08T07:14:58.540123+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:55.334487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.540116+0000","last_change":"2023-10-08T07:14:58.535708+0000","last_active":"2023-10-08T07:14:59.540116+0000","last_peered":"2023-10-08T07:14:59.540116+0000","last_clean":"2023-10-08T07:14:59.540116+0000","last_became_active":"2023-10-08T07:14:58.535226+0000","last_became_peered":"2023-10-08T07:14:58.535226+0000","last_unstale":"2023-10-08T07:14:59.540116+0000","last_undegraded":"2023-10-08T07:14:59.540116+0000","last_fullsized":"2023-10-08T07:14:59.540116+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:17.974456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"257'1","reported_seq":13,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.573354+0000","last_change":"2023-10-08T07:14:57.513838+0000","last_active":"2023-10-08T07:14:58.573354+0000","last_peered":"2023-10-08T07:14:58.573354+0000","last_clean":"2023-10-08T07:14:58.573354+0000","last_became_active":"2023-10-08T07:14:57.512370+0000","last_became_peered":"2023-10-08T07:14:57.512370+0000","last_unstale":"2023-10-08T07:14:58.573354+0000","last_undegraded":"2023-10-08T07:14:58.573354+0000","last_fullsized":"2023-10-08T07:14:58.573354+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:12.810885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.523336+0000","last_change":"2023-10-08T07:14:44.334750+0000","last_active":"2023-10-08T07:14:58.523336+0000","last_peered":"2023-10-08T07:14:58.523336+0000","last_clean":"2023-10-08T07:14:58.523336+0000","last_became_active":"2023-10-08T07:14:44.332421+0000","last_became_peered":"2023-10-08T07:14:44.332421+0000","last_unstale":"2023-10-08T07:14:58.523336+0000","last_undegraded":"2023-10-08T07:14:58.523336+0000","last_fullsized":"2023-10-08T07:14:58.523336+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:57.635068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539121+0000","last_change":"2023-10-08T07:14:58.548540+0000","last_active":"2023-10-08T07:14:59.539121+0000","last_peered":"2023-10-08T07:14:59.539121+0000","last_clean":"2023-10-08T07:14:59.539121+0000","last_became_active":"2023-10-08T07:14:58.548259+0000","last_became_peered":"2023-10-08T07:14:58.548259+0000","last_unstale":"2023-10-08T07:14:59.539121+0000","last_undegraded":"2023-10-08T07:14:59.539121+0000","last_fullsized":"2023-10-08T07:14:59.539121+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.623768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"257'1","reported_seq":13,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.569986+0000","last_change":"2023-10-08T07:14:57.521994+0000","last_active":"2023-10-08T07:14:58.569986+0000","last_peered":"2023-10-08T07:14:58.569986+0000","last_clean":"2023-10-08T07:14:58.569986+0000","last_became_active":"2023-10-08T07:14:57.521707+0000","last_became_peered":"2023-10-08T07:14:57.521707+0000","last_unstale":"2023-10-08T07:14:58.569986+0000","last_undegraded":"2023-10-08T07:14:58.569986+0000","last_fullsized":"2023-10-08T07:14:58.569986+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:47.149837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.534395+0000","last_change":"2023-10-08T07:14:44.331918+0000","last_active":"2023-10-08T07:14:58.534395+0000","last_peered":"2023-10-08T07:14:58.534395+0000","last_clean":"2023-10-08T07:14:58.534395+0000","last_became_active":"2023-10-08T07:14:44.331614+0000","last_became_peered":"2023-10-08T07:14:44.331614+0000","last_unstale":"2023-10-08T07:14:58.534395+0000","last_undegraded":"2023-10-08T07:14:58.534395+0000","last_fullsized":"2023-10-08T07:14:58.534395+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:24.698116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567289+0000","last_change":"2023-10-08T07:14:58.551216+0000","last_active":"2023-10-08T07:14:59.567289+0000","last_peered":"2023-10-08T07:14:59.567289+0000","last_clean":"2023-10-08T07:14:59.567289+0000","last_became_active":"2023-10-08T07:14:58.550213+0000","last_became_peered":"2023-10-08T07:14:58.550213+0000","last_unstale":"2023-10-08T07:14:59.567289+0000","last_undegraded":"2023-10-08T07:14:59.567289+0000","last_fullsized":"2023-10-08T07:14:59.567289+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:08:03.271281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540992+0000","last_change":"2023-10-08T07:14:57.520322+0000","last_active":"2023-10-08T07:14:58.540992+0000","last_peered":"2023-10-08T07:14:58.540992+0000","last_clean":"2023-10-08T07:14:58.540992+0000","last_became_active":"2023-10-08T07:14:57.520065+0000","last_became_peered":"2023-10-08T07:14:57.520065+0000","last_unstale":"2023-10-08T07:14:58.540992+0000","last_undegraded":"2023-10-08T07:14:58.540992+0000","last_fullsized":"2023-10-08T07:14:58.540992+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:35.922226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.534223+0000","last_change":"2023-10-08T07:14:44.335960+0000","last_active":"2023-10-08T07:14:58.534223+0000","last_peered":"2023-10-08T07:14:58.534223+0000","last_clean":"2023-10-08T07:14:58.534223+0000","last_became_active":"2023-10-08T07:14:44.335698+0000","last_became_peered":"2023-10-08T07:14:44.335698+0000","last_unstale":"2023-10-08T07:14:58.534223+0000","last_undegraded":"2023-10-08T07:14:58.534223+0000","last_fullsized":"2023-10-08T07:14:58.534223+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:21.961789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567756+0000","last_change":"2023-10-08T07:14:58.543894+0000","last_active":"2023-10-08T07:14:59.567756+0000","last_peered":"2023-10-08T07:14:59.567756+0000","last_clean":"2023-10-08T07:14:59.567756+0000","last_became_active":"2023-10-08T07:14:58.543547+0000","last_became_peered":"2023-10-08T07:14:58.543547+0000","last_unstale":"2023-10-08T07:14:59.567756+0000","last_undegraded":"2023-10-08T07:14:59.567756+0000","last_fullsized":"2023-10-08T07:14:59.567756+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:42.942350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.525255+0000","last_change":"2023-10-08T07:14:57.516240+0000","last_active":"2023-10-08T07:14:58.525255+0000","last_peered":"2023-10-08T07:14:58.525255+0000","last_clean":"2023-10-08T07:14:58.525255+0000","last_became_active":"2023-10-08T07:14:57.515965+0000","last_became_peered":"2023-10-08T07:14:57.515965+0000","last_unstale":"2023-10-08T07:14:58.525255+0000","last_undegraded":"2023-10-08T07:14:58.525255+0000","last_fullsized":"2023-10-08T07:14:58.525255+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:07.275083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.527002+0000","last_change":"2023-10-08T07:14:44.324288+0000","last_active":"2023-10-08T07:14:58.527002+0000","last_peered":"2023-10-08T07:14:58.527002+0000","last_clean":"2023-10-08T07:14:58.527002+0000","last_became_active":"2023-10-08T07:14:44.323888+0000","last_became_peered":"2023-10-08T07:14:44.323888+0000","last_unstale":"2023-10-08T07:14:58.527002+0000","last_undegraded":"2023-10-08T07:14:58.527002+0000","last_fullsized":"2023-10-08T07:14:58.527002+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:01.742854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.570681+0000","last_change":"2023-10-08T07:14:58.543191+0000","last_active":"2023-10-08T07:14:59.570681+0000","last_peered":"2023-10-08T07:14:59.570681+0000","last_clean":"2023-10-08T07:14:59.570681+0000","last_became_active":"2023-10-08T07:14:58.542923+0000","last_became_peered":"2023-10-08T07:14:58.542923+0000","last_unstale":"2023-10-08T07:14:59.570681+0000","last_undegraded":"2023-10-08T07:14:59.570681+0000","last_fullsized":"2023-10-08T07:14:59.570681+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:36.198033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529939+0000","last_change":"2023-10-08T07:14:57.515589+0000","last_active":"2023-10-08T07:14:58.529939+0000","last_peered":"2023-10-08T07:14:58.529939+0000","last_clean":"2023-10-08T07:14:58.529939+0000","last_became_active":"2023-10-08T07:14:57.515241+0000","last_became_peered":"2023-10-08T07:14:57.515241+0000","last_unstale":"2023-10-08T07:14:58.529939+0000","last_undegraded":"2023-10-08T07:14:58.529939+0000","last_fullsized":"2023-10-08T07:14:58.529939+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:58.130129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533689+0000","last_change":"2023-10-08T07:14:44.332226+0000","last_active":"2023-10-08T07:14:58.533689+0000","last_peered":"2023-10-08T07:14:58.533689+0000","last_clean":"2023-10-08T07:14:58.533689+0000","last_became_active":"2023-10-08T07:14:44.331872+0000","last_became_peered":"2023-10-08T07:14:44.331872+0000","last_unstale":"2023-10-08T07:14:58.533689+0000","last_undegraded":"2023-10-08T07:14:58.533689+0000","last_fullsized":"2023-10-08T07:14:58.533689+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:25.959585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567171+0000","last_change":"2023-10-08T07:14:58.547478+0000","last_active":"2023-10-08T07:14:59.567171+0000","last_peered":"2023-10-08T07:14:59.567171+0000","last_clean":"2023-10-08T07:14:59.567171+0000","last_became_active":"2023-10-08T07:14:58.547145+0000","last_became_peered":"2023-10-08T07:14:58.547145+0000","last_unstale":"2023-10-08T07:14:59.567171+0000","last_undegraded":"2023-10-08T07:14:59.567171+0000","last_fullsized":"2023-10-08T07:14:59.567171+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:23:10.300505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533179+0000","last_change":"2023-10-08T07:14:57.515231+0000","last_active":"2023-10-08T07:14:58.533179+0000","last_peered":"2023-10-08T07:14:58.533179+0000","last_clean":"2023-10-08T07:14:58.533179+0000","last_became_active":"2023-10-08T07:14:57.514976+0000","last_became_peered":"2023-10-08T07:14:57.514976+0000","last_unstale":"2023-10-08T07:14:58.533179+0000","last_undegraded":"2023-10-08T07:14:58.533179+0000","last_fullsized":"2023-10-08T07:14:58.533179+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:09:39.172242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541492+0000","last_change":"2023-10-08T07:14:44.331672+0000","last_active":"2023-10-08T07:14:58.541492+0000","last_peered":"2023-10-08T07:14:58.541492+0000","last_clean":"2023-10-08T07:14:58.541492+0000","last_became_active":"2023-10-08T07:14:44.331316+0000","last_became_peered":"2023-10-08T07:14:44.331316+0000","last_unstale":"2023-10-08T07:14:58.541492+0000","last_undegraded":"2023-10-08T07:14:58.541492+0000","last_fullsized":"2023-10-08T07:14:58.541492+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:57.946015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.547394+0000","last_change":"2023-10-08T07:14:58.560021+0000","last_active":"2023-10-08T07:14:59.547394+0000","last_peered":"2023-10-08T07:14:59.547394+0000","last_clean":"2023-10-08T07:14:59.547394+0000","last_became_active":"2023-10-08T07:14:58.559622+0000","last_became_peered":"2023-10-08T07:14:58.559622+0000","last_unstale":"2023-10-08T07:14:59.547394+0000","last_undegraded":"2023-10-08T07:14:59.547394+0000","last_fullsized":"2023-10-08T07:14:59.547394+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:48.163090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"257'6","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.575114+0000","last_change":"2023-10-08T07:14:57.517430+0000","last_active":"2023-10-08T07:14:58.575114+0000","last_peered":"2023-10-08T07:14:58.575114+0000","last_clean":"2023-10-08T07:14:58.575114+0000","last_became_active":"2023-10-08T07:14:57.517126+0000","last_became_peered":"2023-10-08T07:14:57.517126+0000","last_unstale":"2023-10-08T07:14:58.575114+0000","last_undegraded":"2023-10-08T07:14:58.575114+0000","last_fullsized":"2023-10-08T07:14:58.575114+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:12.728482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540017+0000","last_change":"2023-10-08T07:14:44.331371+0000","last_active":"2023-10-08T07:14:58.540017+0000","last_peered":"2023-10-08T07:14:58.540017+0000","last_clean":"2023-10-08T07:14:58.540017+0000","last_became_active":"2023-10-08T07:14:44.330911+0000","last_became_peered":"2023-10-08T07:14:44.330911+0000","last_unstale":"2023-10-08T07:14:58.540017+0000","last_undegraded":"2023-10-08T07:14:58.540017+0000","last_fullsized":"2023-10-08T07:14:58.540017+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:50.662785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.569711+0000","last_change":"2023-10-08T07:14:58.556691+0000","last_active":"2023-10-08T07:14:59.569711+0000","last_peered":"2023-10-08T07:14:59.569711+0000","last_clean":"2023-10-08T07:14:59.569711+0000","last_became_active":"2023-10-08T07:14:58.555987+0000","last_became_peered":"2023-10-08T07:14:58.555987+0000","last_unstale":"2023-10-08T07:14:59.569711+0000","last_undegraded":"2023-10-08T07:14:59.569711+0000","last_fullsized":"2023-10-08T07:14:59.569711+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:05.082827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"257'3","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.572697+0000","last_change":"2023-10-08T07:14:57.514304+0000","last_active":"2023-10-08T07:14:58.572697+0000","last_peered":"2023-10-08T07:14:58.572697+0000","last_clean":"2023-10-08T07:14:58.572697+0000","last_became_active":"2023-10-08T07:14:57.513991+0000","last_became_peered":"2023-10-08T07:14:57.513991+0000","last_unstale":"2023-10-08T07:14:58.572697+0000","last_undegraded":"2023-10-08T07:14:58.572697+0000","last_fullsized":"2023-10-08T07:14:58.572697+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:43.640211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.531825+0000","last_change":"2023-10-08T07:14:44.336483+0000","last_active":"2023-10-08T07:14:58.531825+0000","last_peered":"2023-10-08T07:14:58.531825+0000","last_clean":"2023-10-08T07:14:58.531825+0000","last_became_active":"2023-10-08T07:14:44.336207+0000","last_became_peered":"2023-10-08T07:14:44.336207+0000","last_unstale":"2023-10-08T07:14:58.531825+0000","last_undegraded":"2023-10-08T07:14:58.531825+0000","last_fullsized":"2023-10-08T07:14:58.531825+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:05.158536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.544190+0000","last_change":"2023-10-08T07:14:58.549070+0000","last_active":"2023-10-08T07:14:59.544190+0000","last_peered":"2023-10-08T07:14:59.544190+0000","last_clean":"2023-10-08T07:14:59.544190+0000","last_became_active":"2023-10-08T07:14:58.548768+0000","last_became_peered":"2023-10-08T07:14:58.548768+0000","last_unstale":"2023-10-08T07:14:59.544190+0000","last_undegraded":"2023-10-08T07:14:59.544190+0000","last_fullsized":"2023-10-08T07:14:59.544190+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:56.384134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.534181+0000","last_change":"2023-10-08T07:14:57.519067+0000","last_active":"2023-10-08T07:14:58.534181+0000","last_peered":"2023-10-08T07:14:58.534181+0000","last_clean":"2023-10-08T07:14:58.534181+0000","last_became_active":"2023-10-08T07:14:57.518808+0000","last_became_peered":"2023-10-08T07:14:57.518808+0000","last_unstale":"2023-10-08T07:14:58.534181+0000","last_undegraded":"2023-10-08T07:14:58.534181+0000","last_fullsized":"2023-10-08T07:14:58.534181+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:09.277589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529188+0000","last_change":"2023-10-08T07:14:44.334689+0000","last_active":"2023-10-08T07:14:58.529188+0000","last_peered":"2023-10-08T07:14:58.529188+0000","last_clean":"2023-10-08T07:14:58.529188+0000","last_became_active":"2023-10-08T07:14:44.334401+0000","last_became_peered":"2023-10-08T07:14:44.334401+0000","last_unstale":"2023-10-08T07:14:58.529188+0000","last_undegraded":"2023-10-08T07:14:58.529188+0000","last_fullsized":"2023-10-08T07:14:58.529188+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:59.473108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.565491+0000","last_change":"2023-10-08T07:14:58.548619+0000","last_active":"2023-10-08T07:14:59.565491+0000","last_peered":"2023-10-08T07:14:59.565491+0000","last_clean":"2023-10-08T07:14:59.565491+0000","last_became_active":"2023-10-08T07:14:58.548333+0000","last_became_peered":"2023-10-08T07:14:58.548333+0000","last_unstale":"2023-10-08T07:14:59.565491+0000","last_undegraded":"2023-10-08T07:14:59.565491+0000","last_fullsized":"2023-10-08T07:14:59.565491+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:51.597388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.571058+0000","last_change":"2023-10-08T07:14:57.520740+0000","last_active":"2023-10-08T07:14:58.571058+0000","last_peered":"2023-10-08T07:14:58.571058+0000","last_clean":"2023-10-08T07:14:58.571058+0000","last_became_active":"2023-10-08T07:14:57.520485+0000","last_became_peered":"2023-10-08T07:14:57.520485+0000","last_unstale":"2023-10-08T07:14:58.571058+0000","last_undegraded":"2023-10-08T07:14:58.571058+0000","last_fullsized":"2023-10-08T07:14:58.571058+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:43.777411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.515817+0000","last_change":"2023-10-08T07:14:44.338751+0000","last_active":"2023-10-08T07:14:58.515817+0000","last_peered":"2023-10-08T07:14:58.515817+0000","last_clean":"2023-10-08T07:14:58.515817+0000","last_became_active":"2023-10-08T07:14:44.338461+0000","last_became_peered":"2023-10-08T07:14:44.338461+0000","last_unstale":"2023-10-08T07:14:58.515817+0000","last_undegraded":"2023-10-08T07:14:58.515817+0000","last_fullsized":"2023-10-08T07:14:58.515817+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:08.802530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567147+0000","last_change":"2023-10-08T07:14:58.555554+0000","last_active":"2023-10-08T07:14:59.567147+0000","last_peered":"2023-10-08T07:14:59.567147+0000","last_clean":"2023-10-08T07:14:59.567147+0000","last_became_active":"2023-10-08T07:14:58.555025+0000","last_became_peered":"2023-10-08T07:14:58.555025+0000","last_unstale":"2023-10-08T07:14:59.567147+0000","last_undegraded":"2023-10-08T07:14:59.567147+0000","last_fullsized":"2023-10-08T07:14:59.567147+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:15.231833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"257'8","reported_seq":20,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.574181+0000","last_change":"2023-10-08T07:14:57.521836+0000","last_active":"2023-10-08T07:14:58.574181+0000","last_peered":"2023-10-08T07:14:58.574181+0000","last_clean":"2023-10-08T07:14:58.574181+0000","last_became_active":"2023-10-08T07:14:57.521568+0000","last_became_peered":"2023-10-08T07:14:57.521568+0000","last_unstale":"2023-10-08T07:14:58.574181+0000","last_undegraded":"2023-10-08T07:14:58.574181+0000","last_fullsized":"2023-10-08T07:14:58.574181+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:18:32.836216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540591+0000","last_change":"2023-10-08T07:14:44.327546+0000","last_active":"2023-10-08T07:14:58.540591+0000","last_peered":"2023-10-08T07:14:58.540591+0000","last_clean":"2023-10-08T07:14:58.540591+0000","last_became_active":"2023-10-08T07:14:44.327267+0000","last_became_peered":"2023-10-08T07:14:44.327267+0000","last_unstale":"2023-10-08T07:14:58.540591+0000","last_undegraded":"2023-10-08T07:14:58.540591+0000","last_fullsized":"2023-10-08T07:14:58.540591+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:15:38.112936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.565500+0000","last_change":"2023-10-08T07:14:58.559779+0000","last_active":"2023-10-08T07:14:59.565500+0000","last_peered":"2023-10-08T07:14:59.565500+0000","last_clean":"2023-10-08T07:14:59.565500+0000","last_became_active":"2023-10-08T07:14:58.559364+0000","last_became_peered":"2023-10-08T07:14:58.559364+0000","last_unstale":"2023-10-08T07:14:59.565500+0000","last_undegraded":"2023-10-08T07:14:59.565500+0000","last_fullsized":"2023-10-08T07:14:59.565500+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:03:55.955439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530981+0000","last_change":"2023-10-08T07:14:57.515376+0000","last_active":"2023-10-08T07:14:58.530981+0000","last_peered":"2023-10-08T07:14:58.530981+0000","last_clean":"2023-10-08T07:14:58.530981+0000","last_became_active":"2023-10-08T07:14:57.515112+0000","last_became_peered":"2023-10-08T07:14:57.515112+0000","last_unstale":"2023-10-08T07:14:58.530981+0000","last_undegraded":"2023-10-08T07:14:58.530981+0000","last_fullsized":"2023-10-08T07:14:58.530981+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:35:04.090038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541081+0000","last_change":"2023-10-08T07:14:44.340846+0000","last_active":"2023-10-08T07:14:58.541081+0000","last_peered":"2023-10-08T07:14:58.541081+0000","last_clean":"2023-10-08T07:14:58.541081+0000","last_became_active":"2023-10-08T07:14:44.340588+0000","last_became_peered":"2023-10-08T07:14:44.340588+0000","last_unstale":"2023-10-08T07:14:58.541081+0000","last_undegraded":"2023-10-08T07:14:58.541081+0000","last_fullsized":"2023-10-08T07:14:58.541081+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:28:06.996015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546659+0000","last_change":"2023-10-08T07:14:58.537900+0000","last_active":"2023-10-08T07:14:59.546659+0000","last_peered":"2023-10-08T07:14:59.546659+0000","last_clean":"2023-10-08T07:14:59.546659+0000","last_became_active":"2023-10-08T07:14:58.537589+0000","last_became_peered":"2023-10-08T07:14:58.537589+0000","last_unstale":"2023-10-08T07:14:59.546659+0000","last_undegraded":"2023-10-08T07:14:59.546659+0000","last_fullsized":"2023-10-08T07:14:59.546659+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:25:34.656632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.568002+0000","last_change":"2023-10-08T07:14:57.522708+0000","last_active":"2023-10-08T07:14:58.568002+0000","last_peered":"2023-10-08T07:14:58.568002+0000","last_clean":"2023-10-08T07:14:58.568002+0000","last_became_active":"2023-10-08T07:14:57.522143+0000","last_became_peered":"2023-10-08T07:14:57.522143+0000","last_unstale":"2023-10-08T07:14:58.568002+0000","last_undegraded":"2023-10-08T07:14:58.568002+0000","last_fullsized":"2023-10-08T07:14:58.568002+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:03.961114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530800+0000","last_change":"2023-10-08T07:14:44.339646+0000","last_active":"2023-10-08T07:14:58.530800+0000","last_peered":"2023-10-08T07:14:58.530800+0000","last_clean":"2023-10-08T07:14:58.530800+0000","last_became_active":"2023-10-08T07:14:44.339229+0000","last_became_peered":"2023-10-08T07:14:44.339229+0000","last_unstale":"2023-10-08T07:14:58.530800+0000","last_undegraded":"2023-10-08T07:14:58.530800+0000","last_fullsized":"2023-10-08T07:14:58.530800+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:07.851859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.545964+0000","last_change":"2023-10-08T07:14:58.555192+0000","last_active":"2023-10-08T07:14:59.545964+0000","last_peered":"2023-10-08T07:14:59.545964+0000","last_clean":"2023-10-08T07:14:59.545964+0000","last_became_active":"2023-10-08T07:14:58.554829+0000","last_became_peered":"2023-10-08T07:14:58.554829+0000","last_unstale":"2023-10-08T07:14:59.545964+0000","last_undegraded":"2023-10-08T07:14:59.545964+0000","last_fullsized":"2023-10-08T07:14:59.545964+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:12:01.717208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"257'1","reported_seq":13,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.568771+0000","last_change":"2023-10-08T07:14:57.526013+0000","last_active":"2023-10-08T07:14:58.568771+0000","last_peered":"2023-10-08T07:14:58.568771+0000","last_clean":"2023-10-08T07:14:58.568771+0000","last_became_active":"2023-10-08T07:14:57.525781+0000","last_became_peered":"2023-10-08T07:14:57.525781+0000","last_unstale":"2023-10-08T07:14:58.568771+0000","last_undegraded":"2023-10-08T07:14:58.568771+0000","last_fullsized":"2023-10-08T07:14:58.568771+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:43.985077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541609+0000","last_change":"2023-10-08T07:14:44.323106+0000","last_active":"2023-10-08T07:14:58.541609+0000","last_peered":"2023-10-08T07:14:58.541609+0000","last_clean":"2023-10-08T07:14:58.541609+0000","last_became_active":"2023-10-08T07:14:44.322570+0000","last_became_peered":"2023-10-08T07:14:44.322570+0000","last_unstale":"2023-10-08T07:14:58.541609+0000","last_undegraded":"2023-10-08T07:14:58.541609+0000","last_fullsized":"2023-10-08T07:14:58.541609+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:07:44.570917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.547615+0000","last_change":"2023-10-08T07:14:58.537503+0000","last_active":"2023-10-08T07:14:59.547615+0000","last_peered":"2023-10-08T07:14:59.547615+0000","last_clean":"2023-10-08T07:14:59.547615+0000","last_became_active":"2023-10-08T07:14:58.537167+0000","last_became_peered":"2023-10-08T07:14:58.537167+0000","last_unstale":"2023-10-08T07:14:59.547615+0000","last_undegraded":"2023-10-08T07:14:59.547615+0000","last_fullsized":"2023-10-08T07:14:59.547615+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:59.146921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533063+0000","last_change":"2023-10-08T07:14:57.521064+0000","last_active":"2023-10-08T07:14:58.533063+0000","last_peered":"2023-10-08T07:14:58.533063+0000","last_clean":"2023-10-08T07:14:58.533063+0000","last_became_active":"2023-10-08T07:14:57.520780+0000","last_became_peered":"2023-10-08T07:14:57.520780+0000","last_unstale":"2023-10-08T07:14:58.533063+0000","last_undegraded":"2023-10-08T07:14:58.533063+0000","last_fullsized":"2023-10-08T07:14:58.533063+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:26:03.459719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.545240+0000","last_change":"2023-10-08T07:14:44.327597+0000","last_active":"2023-10-08T07:14:58.545240+0000","last_peered":"2023-10-08T07:14:58.545240+0000","last_clean":"2023-10-08T07:14:58.545240+0000","last_became_active":"2023-10-08T07:14:44.327247+0000","last_became_peered":"2023-10-08T07:14:44.327247+0000","last_unstale":"2023-10-08T07:14:58.545240+0000","last_undegraded":"2023-10-08T07:14:58.545240+0000","last_fullsized":"2023-10-08T07:14:58.545240+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:19.245988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567995+0000","last_change":"2023-10-08T07:14:58.552725+0000","last_active":"2023-10-08T07:14:59.567995+0000","last_peered":"2023-10-08T07:14:59.567995+0000","last_clean":"2023-10-08T07:14:59.567995+0000","last_became_active":"2023-10-08T07:14:58.551923+0000","last_became_peered":"2023-10-08T07:14:58.551923+0000","last_unstale":"2023-10-08T07:14:59.567995+0000","last_undegraded":"2023-10-08T07:14:59.567995+0000","last_fullsized":"2023-10-08T07:14:59.567995+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:39.120526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"257'1","reported_seq":13,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.568659+0000","last_change":"2023-10-08T07:14:57.520312+0000","last_active":"2023-10-08T07:14:58.568659+0000","last_peered":"2023-10-08T07:14:58.568659+0000","last_clean":"2023-10-08T07:14:58.568659+0000","last_became_active":"2023-10-08T07:14:57.519938+0000","last_became_peered":"2023-10-08T07:14:57.519938+0000","last_unstale":"2023-10-08T07:14:58.568659+0000","last_undegraded":"2023-10-08T07:14:58.568659+0000","last_fullsized":"2023-10-08T07:14:58.568659+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:04:15.152056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533372+0000","last_change":"2023-10-08T07:14:44.338168+0000","last_active":"2023-10-08T07:14:58.533372+0000","last_peered":"2023-10-08T07:14:58.533372+0000","last_clean":"2023-10-08T07:14:58.533372+0000","last_became_active":"2023-10-08T07:14:44.337905+0000","last_became_peered":"2023-10-08T07:14:44.337905+0000","last_unstale":"2023-10-08T07:14:58.533372+0000","last_undegraded":"2023-10-08T07:14:58.533372+0000","last_fullsized":"2023-10-08T07:14:58.533372+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:46.561296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.544556+0000","last_change":"2023-10-08T07:14:58.549048+0000","last_active":"2023-10-08T07:14:59.544556+0000","last_peered":"2023-10-08T07:14:59.544556+0000","last_clean":"2023-10-08T07:14:59.544556+0000","last_became_active":"2023-10-08T07:14:58.548751+0000","last_became_peered":"2023-10-08T07:14:58.548751+0000","last_unstale":"2023-10-08T07:14:59.544556+0000","last_undegraded":"2023-10-08T07:14:59.544556+0000","last_fullsized":"2023-10-08T07:14:59.544556+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:38.988326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.544492+0000","last_change":"2023-10-08T07:14:57.517075+0000","last_active":"2023-10-08T07:14:58.544492+0000","last_peered":"2023-10-08T07:14:58.544492+0000","last_clean":"2023-10-08T07:14:58.544492+0000","last_became_active":"2023-10-08T07:14:57.516810+0000","last_became_peered":"2023-10-08T07:14:57.516810+0000","last_unstale":"2023-10-08T07:14:58.544492+0000","last_undegraded":"2023-10-08T07:14:58.544492+0000","last_fullsized":"2023-10-08T07:14:58.544492+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:41:36.491196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532956+0000","last_change":"2023-10-08T07:14:44.329361+0000","last_active":"2023-10-08T07:14:58.532956+0000","last_peered":"2023-10-08T07:14:58.532956+0000","last_clean":"2023-10-08T07:14:58.532956+0000","last_became_active":"2023-10-08T07:14:44.328994+0000","last_became_peered":"2023-10-08T07:14:44.328994+0000","last_unstale":"2023-10-08T07:14:58.532956+0000","last_undegraded":"2023-10-08T07:14:58.532956+0000","last_fullsized":"2023-10-08T07:14:58.532956+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:22.824813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539320+0000","last_change":"2023-10-08T07:14:58.551857+0000","last_active":"2023-10-08T07:14:59.539320+0000","last_peered":"2023-10-08T07:14:59.539320+0000","last_clean":"2023-10-08T07:14:59.539320+0000","last_became_active":"2023-10-08T07:14:58.551530+0000","last_became_peered":"2023-10-08T07:14:58.551530+0000","last_unstale":"2023-10-08T07:14:59.539320+0000","last_undegraded":"2023-10-08T07:14:59.539320+0000","last_fullsized":"2023-10-08T07:14:59.539320+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:41:37.507988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.571401+0000","last_change":"2023-10-08T07:14:57.517807+0000","last_active":"2023-10-08T07:14:58.571401+0000","last_peered":"2023-10-08T07:14:58.571401+0000","last_clean":"2023-10-08T07:14:58.571401+0000","last_became_active":"2023-10-08T07:14:57.517555+0000","last_became_peered":"2023-10-08T07:14:57.517555+0000","last_unstale":"2023-10-08T07:14:58.571401+0000","last_undegraded":"2023-10-08T07:14:58.571401+0000","last_fullsized":"2023-10-08T07:14:58.571401+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:16.628423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.524675+0000","last_change":"2023-10-08T07:14:44.334282+0000","last_active":"2023-10-08T07:14:58.524675+0000","last_peered":"2023-10-08T07:14:58.524675+0000","last_clean":"2023-10-08T07:14:58.524675+0000","last_became_active":"2023-10-08T07:14:44.332171+0000","last_became_peered":"2023-10-08T07:14:44.332171+0000","last_unstale":"2023-10-08T07:14:58.524675+0000","last_undegraded":"2023-10-08T07:14:58.524675+0000","last_fullsized":"2023-10-08T07:14:58.524675+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:53.430647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539608+0000","last_change":"2023-10-08T07:14:58.541794+0000","last_active":"2023-10-08T07:14:59.539608+0000","last_peered":"2023-10-08T07:14:59.539608+0000","last_clean":"2023-10-08T07:14:59.539608+0000","last_became_active":"2023-10-08T07:14:58.541521+0000","last_became_peered":"2023-10-08T07:14:58.541521+0000","last_unstale":"2023-10-08T07:14:59.539608+0000","last_undegraded":"2023-10-08T07:14:59.539608+0000","last_fullsized":"2023-10-08T07:14:59.539608+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:40.393836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.528691+0000","last_change":"2023-10-08T07:14:57.518527+0000","last_active":"2023-10-08T07:14:58.528691+0000","last_peered":"2023-10-08T07:14:58.528691+0000","last_clean":"2023-10-08T07:14:58.528691+0000","last_became_active":"2023-10-08T07:14:57.518234+0000","last_became_peered":"2023-10-08T07:14:57.518234+0000","last_unstale":"2023-10-08T07:14:58.528691+0000","last_undegraded":"2023-10-08T07:14:58.528691+0000","last_fullsized":"2023-10-08T07:14:58.528691+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:43.350763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540682+0000","last_change":"2023-10-08T07:14:44.327517+0000","last_active":"2023-10-08T07:14:58.540682+0000","last_peered":"2023-10-08T07:14:58.540682+0000","last_clean":"2023-10-08T07:14:58.540682+0000","last_became_active":"2023-10-08T07:14:44.327215+0000","last_became_peered":"2023-10-08T07:14:44.327215+0000","last_unstale":"2023-10-08T07:14:58.540682+0000","last_undegraded":"2023-10-08T07:14:58.540682+0000","last_fullsized":"2023-10-08T07:14:58.540682+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:30.952438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.565754+0000","last_change":"2023-10-08T07:14:58.548407+0000","last_active":"2023-10-08T07:14:59.565754+0000","last_peered":"2023-10-08T07:14:59.565754+0000","last_clean":"2023-10-08T07:14:59.565754+0000","last_became_active":"2023-10-08T07:14:58.547784+0000","last_became_peered":"2023-10-08T07:14:58.547784+0000","last_unstale":"2023-10-08T07:14:59.565754+0000","last_undegraded":"2023-10-08T07:14:59.565754+0000","last_fullsized":"2023-10-08T07:14:59.565754+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:00.297231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.518054+0000","last_change":"2023-10-08T07:14:57.515219+0000","last_active":"2023-10-08T07:14:58.518054+0000","last_peered":"2023-10-08T07:14:58.518054+0000","last_clean":"2023-10-08T07:14:58.518054+0000","last_became_active":"2023-10-08T07:14:57.514941+0000","last_became_peered":"2023-10-08T07:14:57.514941+0000","last_unstale":"2023-10-08T07:14:58.518054+0000","last_undegraded":"2023-10-08T07:14:58.518054+0000","last_fullsized":"2023-10-08T07:14:58.518054+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:38.792601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.522478+0000","last_change":"2023-10-08T07:14:44.328985+0000","last_active":"2023-10-08T07:14:58.522478+0000","last_peered":"2023-10-08T07:14:58.522478+0000","last_clean":"2023-10-08T07:14:58.522478+0000","last_became_active":"2023-10-08T07:14:44.328697+0000","last_became_peered":"2023-10-08T07:14:44.328697+0000","last_unstale":"2023-10-08T07:14:58.522478+0000","last_undegraded":"2023-10-08T07:14:58.522478+0000","last_fullsized":"2023-10-08T07:14:58.522478+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:48.694664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.568351+0000","last_change":"2023-10-08T07:14:58.553564+0000","last_active":"2023-10-08T07:14:59.568351+0000","last_peered":"2023-10-08T07:14:59.568351+0000","last_clean":"2023-10-08T07:14:59.568351+0000","last_became_active":"2023-10-08T07:14:58.553270+0000","last_became_peered":"2023-10-08T07:14:58.553270+0000","last_unstale":"2023-10-08T07:14:59.568351+0000","last_undegraded":"2023-10-08T07:14:59.568351+0000","last_fullsized":"2023-10-08T07:14:59.568351+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:30.387387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.535657+0000","last_change":"2023-10-08T07:14:57.521746+0000","last_active":"2023-10-08T07:14:58.535657+0000","last_peered":"2023-10-08T07:14:58.535657+0000","last_clean":"2023-10-08T07:14:58.535657+0000","last_became_active":"2023-10-08T07:14:57.521489+0000","last_became_peered":"2023-10-08T07:14:57.521489+0000","last_unstale":"2023-10-08T07:14:58.535657+0000","last_undegraded":"2023-10-08T07:14:58.535657+0000","last_fullsized":"2023-10-08T07:14:58.535657+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:43:27.048490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529885+0000","last_change":"2023-10-08T07:14:44.336387+0000","last_active":"2023-10-08T07:14:58.529885+0000","last_peered":"2023-10-08T07:14:58.529885+0000","last_clean":"2023-10-08T07:14:58.529885+0000","last_became_active":"2023-10-08T07:14:44.336109+0000","last_became_peered":"2023-10-08T07:14:44.336109+0000","last_unstale":"2023-10-08T07:14:58.529885+0000","last_undegraded":"2023-10-08T07:14:58.529885+0000","last_fullsized":"2023-10-08T07:14:58.529885+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:39.506846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.569051+0000","last_change":"2023-10-08T07:14:58.562531+0000","last_active":"2023-10-08T07:14:59.569051+0000","last_peered":"2023-10-08T07:14:59.569051+0000","last_clean":"2023-10-08T07:14:59.569051+0000","last_became_active":"2023-10-08T07:14:58.562176+0000","last_became_peered":"2023-10-08T07:14:58.562176+0000","last_unstale":"2023-10-08T07:14:59.569051+0000","last_undegraded":"2023-10-08T07:14:59.569051+0000","last_fullsized":"2023-10-08T07:14:59.569051+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:15:48.143425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.571204+0000","last_change":"2023-10-08T07:14:57.520776+0000","last_active":"2023-10-08T07:14:58.571204+0000","last_peered":"2023-10-08T07:14:58.571204+0000","last_clean":"2023-10-08T07:14:58.571204+0000","last_became_active":"2023-10-08T07:14:57.520526+0000","last_became_peered":"2023-10-08T07:14:57.520526+0000","last_unstale":"2023-10-08T07:14:58.571204+0000","last_undegraded":"2023-10-08T07:14:58.571204+0000","last_fullsized":"2023-10-08T07:14:58.571204+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:49:19.970642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.535810+0000","last_change":"2023-10-08T07:14:44.338566+0000","last_active":"2023-10-08T07:14:58.535810+0000","last_peered":"2023-10-08T07:14:58.535810+0000","last_clean":"2023-10-08T07:14:58.535810+0000","last_became_active":"2023-10-08T07:14:44.338309+0000","last_became_peered":"2023-10-08T07:14:44.338309+0000","last_unstale":"2023-10-08T07:14:58.535810+0000","last_undegraded":"2023-10-08T07:14:58.535810+0000","last_fullsized":"2023-10-08T07:14:58.535810+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:51.724420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.570166+0000","last_change":"2023-10-08T07:14:58.561367+0000","last_active":"2023-10-08T07:14:59.570166+0000","last_peered":"2023-10-08T07:14:59.570166+0000","last_clean":"2023-10-08T07:14:59.570166+0000","last_became_active":"2023-10-08T07:14:58.561074+0000","last_became_peered":"2023-10-08T07:14:58.561074+0000","last_unstale":"2023-10-08T07:14:59.570166+0000","last_undegraded":"2023-10-08T07:14:59.570166+0000","last_fullsized":"2023-10-08T07:14:59.570166+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:18.215030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533339+0000","last_change":"2023-10-08T07:14:57.521949+0000","last_active":"2023-10-08T07:14:58.533339+0000","last_peered":"2023-10-08T07:14:58.533339+0000","last_clean":"2023-10-08T07:14:58.533339+0000","last_became_active":"2023-10-08T07:14:57.521678+0000","last_became_peered":"2023-10-08T07:14:57.521678+0000","last_unstale":"2023-10-08T07:14:58.533339+0000","last_undegraded":"2023-10-08T07:14:58.533339+0000","last_fullsized":"2023-10-08T07:14:58.533339+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:29.465386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.522725+0000","last_change":"2023-10-08T07:14:44.335206+0000","last_active":"2023-10-08T07:14:58.522725+0000","last_peered":"2023-10-08T07:14:58.522725+0000","last_clean":"2023-10-08T07:14:58.522725+0000","last_became_active":"2023-10-08T07:14:44.332905+0000","last_became_peered":"2023-10-08T07:14:44.332905+0000","last_unstale":"2023-10-08T07:14:58.522725+0000","last_undegraded":"2023-10-08T07:14:58.522725+0000","last_fullsized":"2023-10-08T07:14:58.522725+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:57.888130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.569268+0000","last_change":"2023-10-08T07:14:58.552437+0000","last_active":"2023-10-08T07:14:59.569268+0000","last_peered":"2023-10-08T07:14:59.569268+0000","last_clean":"2023-10-08T07:14:59.569268+0000","last_became_active":"2023-10-08T07:14:58.552036+0000","last_became_peered":"2023-10-08T07:14:58.552036+0000","last_unstale":"2023-10-08T07:14:59.569268+0000","last_undegraded":"2023-10-08T07:14:59.569268+0000","last_fullsized":"2023-10-08T07:14:59.569268+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:52:15.693811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532854+0000","last_change":"2023-10-08T07:14:57.516891+0000","last_active":"2023-10-08T07:14:58.532854+0000","last_peered":"2023-10-08T07:14:58.532854+0000","last_clean":"2023-10-08T07:14:58.532854+0000","last_became_active":"2023-10-08T07:14:57.516625+0000","last_became_peered":"2023-10-08T07:14:57.516625+0000","last_unstale":"2023-10-08T07:14:58.532854+0000","last_undegraded":"2023-10-08T07:14:58.532854+0000","last_fullsized":"2023-10-08T07:14:58.532854+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:50.001002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.531904+0000","last_change":"2023-10-08T07:14:44.332109+0000","last_active":"2023-10-08T07:14:58.531904+0000","last_peered":"2023-10-08T07:14:58.531904+0000","last_clean":"2023-10-08T07:14:58.531904+0000","last_became_active":"2023-10-08T07:14:44.331835+0000","last_became_peered":"2023-10-08T07:14:44.331835+0000","last_unstale":"2023-10-08T07:14:58.531904+0000","last_undegraded":"2023-10-08T07:14:58.531904+0000","last_fullsized":"2023-10-08T07:14:58.531904+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:15.146404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.540368+0000","last_change":"2023-10-08T07:14:58.544540+0000","last_active":"2023-10-08T07:14:59.540368+0000","last_peered":"2023-10-08T07:14:59.540368+0000","last_clean":"2023-10-08T07:14:59.540368+0000","last_became_active":"2023-10-08T07:14:58.543361+0000","last_became_peered":"2023-10-08T07:14:58.543361+0000","last_unstale":"2023-10-08T07:14:59.540368+0000","last_undegraded":"2023-10-08T07:14:59.540368+0000","last_fullsized":"2023-10-08T07:14:59.540368+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:15.231833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532621+0000","last_change":"2023-10-08T07:14:57.517351+0000","last_active":"2023-10-08T07:14:58.532621+0000","last_peered":"2023-10-08T07:14:58.532621+0000","last_clean":"2023-10-08T07:14:58.532621+0000","last_became_active":"2023-10-08T07:14:57.517041+0000","last_became_peered":"2023-10-08T07:14:57.517041+0000","last_unstale":"2023-10-08T07:14:58.532621+0000","last_undegraded":"2023-10-08T07:14:58.532621+0000","last_fullsized":"2023-10-08T07:14:58.532621+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:17:12.570521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.516393+0000","last_change":"2023-10-08T07:14:44.339948+0000","last_active":"2023-10-08T07:14:58.516393+0000","last_peered":"2023-10-08T07:14:58.516393+0000","last_clean":"2023-10-08T07:14:58.516393+0000","last_became_active":"2023-10-08T07:14:44.339645+0000","last_became_peered":"2023-10-08T07:14:44.339645+0000","last_unstale":"2023-10-08T07:14:58.516393+0000","last_undegraded":"2023-10-08T07:14:58.516393+0000","last_fullsized":"2023-10-08T07:14:58.516393+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:42.921595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.564627+0000","last_change":"2023-10-08T07:14:58.548523+0000","last_active":"2023-10-08T07:14:59.564627+0000","last_peered":"2023-10-08T07:14:59.564627+0000","last_clean":"2023-10-08T07:14:59.564627+0000","last_became_active":"2023-10-08T07:14:58.548180+0000","last_became_peered":"2023-10-08T07:14:58.548180+0000","last_unstale":"2023-10-08T07:14:59.564627+0000","last_undegraded":"2023-10-08T07:14:59.564627+0000","last_fullsized":"2023-10-08T07:14:59.564627+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:32.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529982+0000","last_change":"2023-10-08T07:14:57.521106+0000","last_active":"2023-10-08T07:14:58.529982+0000","last_peered":"2023-10-08T07:14:58.529982+0000","last_clean":"2023-10-08T07:14:58.529982+0000","last_became_active":"2023-10-08T07:14:57.520832+0000","last_became_peered":"2023-10-08T07:14:57.520832+0000","last_unstale":"2023-10-08T07:14:58.529982+0000","last_undegraded":"2023-10-08T07:14:58.529982+0000","last_fullsized":"2023-10-08T07:14:58.529982+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:06.528060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.516063+0000","last_change":"2023-10-08T07:14:44.321797+0000","last_active":"2023-10-08T07:14:58.516063+0000","last_peered":"2023-10-08T07:14:58.516063+0000","last_clean":"2023-10-08T07:14:58.516063+0000","last_became_active":"2023-10-08T07:14:44.321208+0000","last_became_peered":"2023-10-08T07:14:44.321208+0000","last_unstale":"2023-10-08T07:14:58.516063+0000","last_undegraded":"2023-10-08T07:14:58.516063+0000","last_fullsized":"2023-10-08T07:14:58.516063+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:01.676820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546297+0000","last_change":"2023-10-08T07:14:58.539507+0000","last_active":"2023-10-08T07:14:59.546297+0000","last_peered":"2023-10-08T07:14:59.546297+0000","last_clean":"2023-10-08T07:14:59.546297+0000","last_became_active":"2023-10-08T07:14:58.539177+0000","last_became_peered":"2023-10-08T07:14:58.539177+0000","last_unstale":"2023-10-08T07:14:59.546297+0000","last_undegraded":"2023-10-08T07:14:59.546297+0000","last_fullsized":"2023-10-08T07:14:59.546297+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:03.984646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.526721+0000","last_change":"2023-10-08T07:14:57.503202+0000","last_active":"2023-10-08T07:14:58.526721+0000","last_peered":"2023-10-08T07:14:58.526721+0000","last_clean":"2023-10-08T07:14:58.526721+0000","last_became_active":"2023-10-08T07:14:57.502784+0000","last_became_peered":"2023-10-08T07:14:57.502784+0000","last_unstale":"2023-10-08T07:14:58.526721+0000","last_undegraded":"2023-10-08T07:14:58.526721+0000","last_fullsized":"2023-10-08T07:14:58.526721+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:01.792077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530745+0000","last_change":"2023-10-08T07:14:44.329230+0000","last_active":"2023-10-08T07:14:58.530745+0000","last_peered":"2023-10-08T07:14:58.530745+0000","last_clean":"2023-10-08T07:14:58.530745+0000","last_became_active":"2023-10-08T07:14:44.328972+0000","last_became_peered":"2023-10-08T07:14:44.328972+0000","last_unstale":"2023-10-08T07:14:58.530745+0000","last_undegraded":"2023-10-08T07:14:58.530745+0000","last_fullsized":"2023-10-08T07:14:58.530745+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:59.867061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.540179+0000","last_change":"2023-10-08T07:14:58.543125+0000","last_active":"2023-10-08T07:14:59.540179+0000","last_peered":"2023-10-08T07:14:59.540179+0000","last_clean":"2023-10-08T07:14:59.540179+0000","last_became_active":"2023-10-08T07:14:58.542705+0000","last_became_peered":"2023-10-08T07:14:58.542705+0000","last_unstale":"2023-10-08T07:14:59.540179+0000","last_undegraded":"2023-10-08T07:14:59.540179+0000","last_fullsized":"2023-10-08T07:14:59.540179+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:03:55.955439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.534182+0000","last_change":"2023-10-08T07:14:57.515371+0000","last_active":"2023-10-08T07:14:58.534182+0000","last_peered":"2023-10-08T07:14:58.534182+0000","last_clean":"2023-10-08T07:14:58.534182+0000","last_became_active":"2023-10-08T07:14:57.515064+0000","last_became_peered":"2023-10-08T07:14:57.515064+0000","last_unstale":"2023-10-08T07:14:58.534182+0000","last_undegraded":"2023-10-08T07:14:58.534182+0000","last_fullsized":"2023-10-08T07:14:58.534182+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:44.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529171+0000","last_change":"2023-10-08T07:14:44.337970+0000","last_active":"2023-10-08T07:14:58.529171+0000","last_peered":"2023-10-08T07:14:58.529171+0000","last_clean":"2023-10-08T07:14:58.529171+0000","last_became_active":"2023-10-08T07:14:44.337633+0000","last_became_peered":"2023-10-08T07:14:44.337633+0000","last_unstale":"2023-10-08T07:14:58.529171+0000","last_undegraded":"2023-10-08T07:14:58.529171+0000","last_fullsized":"2023-10-08T07:14:58.529171+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:55.063443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.566655+0000","last_change":"2023-10-08T07:14:58.548299+0000","last_active":"2023-10-08T07:14:59.566655+0000","last_peered":"2023-10-08T07:14:59.566655+0000","last_clean":"2023-10-08T07:14:59.566655+0000","last_became_active":"2023-10-08T07:14:58.547637+0000","last_became_peered":"2023-10-08T07:14:58.547637+0000","last_unstale":"2023-10-08T07:14:59.566655+0000","last_undegraded":"2023-10-08T07:14:59.566655+0000","last_fullsized":"2023-10-08T07:14:59.566655+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:36.265754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.527304+0000","last_change":"2023-10-08T07:14:44.332579+0000","last_active":"2023-10-08T07:14:58.527304+0000","last_peered":"2023-10-08T07:14:58.527304+0000","last_clean":"2023-10-08T07:14:58.527304+0000","last_became_active":"2023-10-08T07:14:44.329641+0000","last_became_peered":"2023-10-08T07:14:44.329641+0000","last_unstale":"2023-10-08T07:14:58.527304+0000","last_undegraded":"2023-10-08T07:14:58.527304+0000","last_fullsized":"2023-10-08T07:14:58.527304+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:23.709005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"253'4","reported_seq":20,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533599+0000","last_change":"2023-10-08T07:14:43.402506+0000","last_active":"2023-10-08T07:14:58.533599+0000","last_peered":"2023-10-08T07:14:58.533599+0000","last_clean":"2023-10-08T07:14:58.533599+0000","last_became_active":"2023-10-08T07:14:43.402246+0000","last_became_peered":"2023-10-08T07:14:43.402246+0000","last_unstale":"2023-10-08T07:14:58.533599+0000","last_undegraded":"2023-10-08T07:14:58.533599+0000","last_fullsized":"2023-10-08T07:14:58.533599+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:03.082313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539095+0000","last_change":"2023-10-08T07:14:58.539800+0000","last_active":"2023-10-08T07:14:59.539095+0000","last_peered":"2023-10-08T07:14:59.539095+0000","last_clean":"2023-10-08T07:14:59.539095+0000","last_became_active":"2023-10-08T07:14:58.539534+0000","last_became_peered":"2023-10-08T07:14:58.539534+0000","last_unstale":"2023-10-08T07:14:59.539095+0000","last_undegraded":"2023-10-08T07:14:59.539095+0000","last_fullsized":"2023-10-08T07:14:59.539095+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:27:05.838626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.544527+0000","last_change":"2023-10-08T07:14:44.337109+0000","last_active":"2023-10-08T07:14:58.544527+0000","last_peered":"2023-10-08T07:14:58.544527+0000","last_clean":"2023-10-08T07:14:58.544527+0000","last_became_active":"2023-10-08T07:14:44.336825+0000","last_became_peered":"2023-10-08T07:14:44.336825+0000","last_unstale":"2023-10-08T07:14:58.544527+0000","last_undegraded":"2023-10-08T07:14:58.544527+0000","last_fullsized":"2023-10-08T07:14:58.544527+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:04:02.054643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532463+0000","last_change":"2023-10-08T07:14:43.408289+0000","last_active":"2023-10-08T07:14:58.532463+0000","last_peered":"2023-10-08T07:14:58.532463+0000","last_clean":"2023-10-08T07:14:58.532463+0000","last_became_active":"2023-10-08T07:14:43.407994+0000","last_became_peered":"2023-10-08T07:14:43.407994+0000","last_unstale":"2023-10-08T07:14:58.532463+0000","last_undegraded":"2023-10-08T07:14:58.532463+0000","last_fullsized":"2023-10-08T07:14:58.532463+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:44.448458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539604+0000","last_change":"2023-10-08T07:14:58.544715+0000","last_active":"2023-10-08T07:14:59.539604+0000","last_peered":"2023-10-08T07:14:59.539604+0000","last_clean":"2023-10-08T07:14:59.539604+0000","last_became_active":"2023-10-08T07:14:58.544274+0000","last_became_peered":"2023-10-08T07:14:58.544274+0000","last_unstale":"2023-10-08T07:14:59.539604+0000","last_undegraded":"2023-10-08T07:14:59.539604+0000","last_fullsized":"2023-10-08T07:14:59.539604+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:45.982201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.531934+0000","last_change":"2023-10-08T07:14:44.332474+0000","last_active":"2023-10-08T07:14:58.531934+0000","last_peered":"2023-10-08T07:14:58.531934+0000","last_clean":"2023-10-08T07:14:58.531934+0000","last_became_active":"2023-10-08T07:14:44.332200+0000","last_became_peered":"2023-10-08T07:14:44.332200+0000","last_unstale":"2023-10-08T07:14:58.531934+0000","last_undegraded":"2023-10-08T07:14:58.531934+0000","last_fullsized":"2023-10-08T07:14:58.531934+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:19.691981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.517158+0000","last_change":"2023-10-08T07:14:43.402276+0000","last_active":"2023-10-08T07:14:58.517158+0000","last_peered":"2023-10-08T07:14:58.517158+0000","last_clean":"2023-10-08T07:14:58.517158+0000","last_became_active":"2023-10-08T07:14:43.401792+0000","last_became_peered":"2023-10-08T07:14:43.401792+0000","last_unstale":"2023-10-08T07:14:58.517158+0000","last_undegraded":"2023-10-08T07:14:58.517158+0000","last_fullsized":"2023-10-08T07:14:58.517158+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:38.609639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.540188+0000","last_change":"2023-10-08T07:14:58.556210+0000","last_active":"2023-10-08T07:14:59.540188+0000","last_peered":"2023-10-08T07:14:59.540188+0000","last_clean":"2023-10-08T07:14:59.540188+0000","last_became_active":"2023-10-08T07:14:58.555901+0000","last_became_peered":"2023-10-08T07:14:58.555901+0000","last_unstale":"2023-10-08T07:14:59.540188+0000","last_undegraded":"2023-10-08T07:14:59.540188+0000","last_fullsized":"2023-10-08T07:14:59.540188+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:51.597388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.516900+0000","last_change":"2023-10-08T07:14:44.333701+0000","last_active":"2023-10-08T07:14:58.516900+0000","last_peered":"2023-10-08T07:14:58.516900+0000","last_clean":"2023-10-08T07:14:58.516900+0000","last_became_active":"2023-10-08T07:14:44.333403+0000","last_became_peered":"2023-10-08T07:14:44.333403+0000","last_unstale":"2023-10-08T07:14:58.516900+0000","last_undegraded":"2023-10-08T07:14:58.516900+0000","last_fullsized":"2023-10-08T07:14:58.516900+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:24.067787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541917+0000","last_change":"2023-10-08T07:14:43.416523+0000","last_active":"2023-10-08T07:14:58.541917+0000","last_peered":"2023-10-08T07:14:58.541917+0000","last_clean":"2023-10-08T07:14:58.541917+0000","last_became_active":"2023-10-08T07:14:43.415998+0000","last_became_peered":"2023-10-08T07:14:43.415998+0000","last_unstale":"2023-10-08T07:14:58.541917+0000","last_undegraded":"2023-10-08T07:14:58.541917+0000","last_fullsized":"2023-10-08T07:14:58.541917+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:02.841738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.569455+0000","last_change":"2023-10-08T07:14:58.555275+0000","last_active":"2023-10-08T07:14:59.569455+0000","last_peered":"2023-10-08T07:14:59.569455+0000","last_clean":"2023-10-08T07:14:59.569455+0000","last_became_active":"2023-10-08T07:14:58.554967+0000","last_became_peered":"2023-10-08T07:14:58.554967+0000","last_unstale":"2023-10-08T07:14:59.569455+0000","last_undegraded":"2023-10-08T07:14:59.569455+0000","last_fullsized":"2023-10-08T07:14:59.569455+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:47.607554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533288+0000","last_change":"2023-10-08T07:14:44.332587+0000","last_active":"2023-10-08T07:14:58.533288+0000","last_peered":"2023-10-08T07:14:58.533288+0000","last_clean":"2023-10-08T07:14:58.533288+0000","last_became_active":"2023-10-08T07:14:44.328870+0000","last_became_peered":"2023-10-08T07:14:44.328870+0000","last_unstale":"2023-10-08T07:14:58.533288+0000","last_undegraded":"2023-10-08T07:14:58.533288+0000","last_fullsized":"2023-10-08T07:14:58.533288+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:30.679998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.516325+0000","last_change":"2023-10-08T07:14:43.415069+0000","last_active":"2023-10-08T07:14:58.516325+0000","last_peered":"2023-10-08T07:14:58.516325+0000","last_clean":"2023-10-08T07:14:58.516325+0000","last_became_active":"2023-10-08T07:14:43.414780+0000","last_became_peered":"2023-10-08T07:14:43.414780+0000","last_unstale":"2023-10-08T07:14:58.516325+0000","last_undegraded":"2023-10-08T07:14:58.516325+0000","last_fullsized":"2023-10-08T07:14:58.516325+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:09:43.543624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546559+0000","last_change":"2023-10-08T07:14:58.544539+0000","last_active":"2023-10-08T07:14:59.546559+0000","last_peered":"2023-10-08T07:14:59.546559+0000","last_clean":"2023-10-08T07:14:59.546559+0000","last_became_active":"2023-10-08T07:14:58.544234+0000","last_became_peered":"2023-10-08T07:14:58.544234+0000","last_unstale":"2023-10-08T07:14:59.546559+0000","last_undegraded":"2023-10-08T07:14:59.546559+0000","last_fullsized":"2023-10-08T07:14:59.546559+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:17.828746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541397+0000","last_change":"2023-10-08T07:14:44.338239+0000","last_active":"2023-10-08T07:14:58.541397+0000","last_peered":"2023-10-08T07:14:58.541397+0000","last_clean":"2023-10-08T07:14:58.541397+0000","last_became_active":"2023-10-08T07:14:44.337931+0000","last_became_peered":"2023-10-08T07:14:44.337931+0000","last_unstale":"2023-10-08T07:14:58.541397+0000","last_undegraded":"2023-10-08T07:14:58.541397+0000","last_fullsized":"2023-10-08T07:14:58.541397+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:36:59.713472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.538446+0000","last_change":"2023-10-08T07:14:43.410267+0000","last_active":"2023-10-08T07:14:58.538446+0000","last_peered":"2023-10-08T07:14:58.538446+0000","last_clean":"2023-10-08T07:14:58.538446+0000","last_became_active":"2023-10-08T07:14:43.410012+0000","last_became_peered":"2023-10-08T07:14:43.410012+0000","last_unstale":"2023-10-08T07:14:58.538446+0000","last_undegraded":"2023-10-08T07:14:58.538446+0000","last_fullsized":"2023-10-08T07:14:58.538446+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:41.903083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546371+0000","last_change":"2023-10-08T07:14:58.545365+0000","last_active":"2023-10-08T07:14:59.546371+0000","last_peered":"2023-10-08T07:14:59.546371+0000","last_clean":"2023-10-08T07:14:59.546371+0000","last_became_active":"2023-10-08T07:14:58.545053+0000","last_became_peered":"2023-10-08T07:14:58.545053+0000","last_unstale":"2023-10-08T07:14:59.546371+0000","last_undegraded":"2023-10-08T07:14:59.546371+0000","last_fullsized":"2023-10-08T07:14:59.546371+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:08.291875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529833+0000","last_change":"2023-10-08T07:14:44.330641+0000","last_active":"2023-10-08T07:14:58.529833+0000","last_peered":"2023-10-08T07:14:58.529833+0000","last_clean":"2023-10-08T07:14:58.529833+0000","last_became_active":"2023-10-08T07:14:44.330271+0000","last_became_peered":"2023-10-08T07:14:44.330271+0000","last_unstale":"2023-10-08T07:14:58.529833+0000","last_undegraded":"2023-10-08T07:14:58.529833+0000","last_fullsized":"2023-10-08T07:14:58.529833+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:35.355050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529364+0000","last_change":"2023-10-08T07:14:43.407886+0000","last_active":"2023-10-08T07:14:58.529364+0000","last_peered":"2023-10-08T07:14:58.529364+0000","last_clean":"2023-10-08T07:14:58.529364+0000","last_became_active":"2023-10-08T07:14:43.407634+0000","last_became_peered":"2023-10-08T07:14:43.407634+0000","last_unstale":"2023-10-08T07:14:58.529364+0000","last_undegraded":"2023-10-08T07:14:58.529364+0000","last_fullsized":"2023-10-08T07:14:58.529364+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:55:43.492128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.544787+0000","last_change":"2023-10-08T07:14:58.544652+0000","last_active":"2023-10-08T07:14:59.544787+0000","last_peered":"2023-10-08T07:14:59.544787+0000","last_clean":"2023-10-08T07:14:59.544787+0000","last_became_active":"2023-10-08T07:14:58.544354+0000","last_became_peered":"2023-10-08T07:14:58.544354+0000","last_unstale":"2023-10-08T07:14:59.544787+0000","last_undegraded":"2023-10-08T07:14:59.544787+0000","last_fullsized":"2023-10-08T07:14:59.544787+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:58.222778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.536245+0000","last_change":"2023-10-08T07:14:44.332428+0000","last_active":"2023-10-08T07:14:58.536245+0000","last_peered":"2023-10-08T07:14:58.536245+0000","last_clean":"2023-10-08T07:14:58.536245+0000","last_became_active":"2023-10-08T07:14:44.331865+0000","last_became_peered":"2023-10-08T07:14:44.331865+0000","last_unstale":"2023-10-08T07:14:58.536245+0000","last_undegraded":"2023-10-08T07:14:58.536245+0000","last_fullsized":"2023-10-08T07:14:58.536245+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:04.509562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.519168+0000","last_change":"2023-10-08T07:14:43.397332+0000","last_active":"2023-10-08T07:14:58.519168+0000","last_peered":"2023-10-08T07:14:58.519168+0000","last_clean":"2023-10-08T07:14:58.519168+0000","last_became_active":"2023-10-08T07:14:43.397049+0000","last_became_peered":"2023-10-08T07:14:43.397049+0000","last_unstale":"2023-10-08T07:14:58.519168+0000","last_undegraded":"2023-10-08T07:14:58.519168+0000","last_fullsized":"2023-10-08T07:14:58.519168+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:23.709280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567021+0000","last_change":"2023-10-08T07:14:58.555347+0000","last_active":"2023-10-08T07:14:59.567021+0000","last_peered":"2023-10-08T07:14:59.567021+0000","last_clean":"2023-10-08T07:14:59.567021+0000","last_became_active":"2023-10-08T07:14:58.554665+0000","last_became_peered":"2023-10-08T07:14:58.554665+0000","last_unstale":"2023-10-08T07:14:59.567021+0000","last_undegraded":"2023-10-08T07:14:59.567021+0000","last_fullsized":"2023-10-08T07:14:59.567021+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:45.982201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.526067+0000","last_change":"2023-10-08T07:14:44.331844+0000","last_active":"2023-10-08T07:14:58.526067+0000","last_peered":"2023-10-08T07:14:58.526067+0000","last_clean":"2023-10-08T07:14:58.526067+0000","last_became_active":"2023-10-08T07:14:44.331521+0000","last_became_peered":"2023-10-08T07:14:44.331521+0000","last_unstale":"2023-10-08T07:14:58.526067+0000","last_undegraded":"2023-10-08T07:14:58.526067+0000","last_fullsized":"2023-10-08T07:14:58.526067+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:22.760377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541697+0000","last_change":"2023-10-08T07:14:43.414266+0000","last_active":"2023-10-08T07:14:58.541697+0000","last_peered":"2023-10-08T07:14:58.541697+0000","last_clean":"2023-10-08T07:14:58.541697+0000","last_became_active":"2023-10-08T07:14:43.413877+0000","last_became_peered":"2023-10-08T07:14:43.413877+0000","last_unstale":"2023-10-08T07:14:58.541697+0000","last_undegraded":"2023-10-08T07:14:58.541697+0000","last_fullsized":"2023-10-08T07:14:58.541697+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:00.790509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.542575+0000","last_change":"2023-10-08T07:14:58.551858+0000","last_active":"2023-10-08T07:14:59.542575+0000","last_peered":"2023-10-08T07:14:59.542575+0000","last_clean":"2023-10-08T07:14:59.542575+0000","last_became_active":"2023-10-08T07:14:58.550576+0000","last_became_peered":"2023-10-08T07:14:58.550576+0000","last_unstale":"2023-10-08T07:14:59.542575+0000","last_undegraded":"2023-10-08T07:14:59.542575+0000","last_fullsized":"2023-10-08T07:14:59.542575+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:22.021516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.544457+0000","last_change":"2023-10-08T07:14:44.330684+0000","last_active":"2023-10-08T07:14:58.544457+0000","last_peered":"2023-10-08T07:14:58.544457+0000","last_clean":"2023-10-08T07:14:58.544457+0000","last_became_active":"2023-10-08T07:14:44.330353+0000","last_became_peered":"2023-10-08T07:14:44.330353+0000","last_unstale":"2023-10-08T07:14:58.544457+0000","last_undegraded":"2023-10-08T07:14:58.544457+0000","last_fullsized":"2023-10-08T07:14:58.544457+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:14:27.946683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"253'3","reported_seq":19,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529186+0000","last_change":"2023-10-08T07:14:43.410104+0000","last_active":"2023-10-08T07:14:58.529186+0000","last_peered":"2023-10-08T07:14:58.529186+0000","last_clean":"2023-10-08T07:14:58.529186+0000","last_became_active":"2023-10-08T07:14:43.409558+0000","last_became_peered":"2023-10-08T07:14:43.409558+0000","last_unstale":"2023-10-08T07:14:58.529186+0000","last_undegraded":"2023-10-08T07:14:58.529186+0000","last_fullsized":"2023-10-08T07:14:58.529186+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:47.180803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539147+0000","last_change":"2023-10-08T07:14:58.546741+0000","last_active":"2023-10-08T07:14:59.539147+0000","last_peered":"2023-10-08T07:14:59.539147+0000","last_clean":"2023-10-08T07:14:59.539147+0000","last_became_active":"2023-10-08T07:14:58.546430+0000","last_became_peered":"2023-10-08T07:14:58.546430+0000","last_unstale":"2023-10-08T07:14:59.539147+0000","last_undegraded":"2023-10-08T07:14:59.539147+0000","last_fullsized":"2023-10-08T07:14:59.539147+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:45.234198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532838+0000","last_change":"2023-10-08T07:14:44.326288+0000","last_active":"2023-10-08T07:14:58.532838+0000","last_peered":"2023-10-08T07:14:58.532838+0000","last_clean":"2023-10-08T07:14:58.532838+0000","last_became_active":"2023-10-08T07:14:44.325977+0000","last_became_peered":"2023-10-08T07:14:44.325977+0000","last_unstale":"2023-10-08T07:14:58.532838+0000","last_undegraded":"2023-10-08T07:14:58.532838+0000","last_fullsized":"2023-10-08T07:14:58.532838+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:31.119993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"254'8","reported_seq":24,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.531991+0000","last_change":"2023-10-08T07:14:43.399270+0000","last_active":"2023-10-08T07:14:58.531991+0000","last_peered":"2023-10-08T07:14:58.531991+0000","last_clean":"2023-10-08T07:14:58.531991+0000","last_became_active":"2023-10-08T07:14:43.398978+0000","last_became_peered":"2023-10-08T07:14:43.398978+0000","last_unstale":"2023-10-08T07:14:58.531991+0000","last_undegraded":"2023-10-08T07:14:58.531991+0000","last_fullsized":"2023-10-08T07:14:58.531991+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:49.950109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.571183+0000","last_change":"2023-10-08T07:14:58.555258+0000","last_active":"2023-10-08T07:14:59.571183+0000","last_peered":"2023-10-08T07:14:59.571183+0000","last_clean":"2023-10-08T07:14:59.571183+0000","last_became_active":"2023-10-08T07:14:58.554985+0000","last_became_peered":"2023-10-08T07:14:58.554985+0000","last_unstale":"2023-10-08T07:14:59.571183+0000","last_undegraded":"2023-10-08T07:14:59.571183+0000","last_fullsized":"2023-10-08T07:14:59.571183+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:36:18.819012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533283+0000","last_change":"2023-10-08T07:14:44.335266+0000","last_active":"2023-10-08T07:14:58.533283+0000","last_peered":"2023-10-08T07:14:58.533283+0000","last_clean":"2023-10-08T07:14:58.533283+0000","last_became_active":"2023-10-08T07:14:44.334971+0000","last_became_peered":"2023-10-08T07:14:44.334971+0000","last_unstale":"2023-10-08T07:14:58.533283+0000","last_undegraded":"2023-10-08T07:14:58.533283+0000","last_fullsized":"2023-10-08T07:14:58.533283+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:14.581717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.542027+0000","last_change":"2023-10-08T07:14:43.410713+0000","last_active":"2023-10-08T07:14:58.542027+0000","last_peered":"2023-10-08T07:14:58.542027+0000","last_clean":"2023-10-08T07:14:58.542027+0000","last_became_active":"2023-10-08T07:14:43.410448+0000","last_became_peered":"2023-10-08T07:14:43.410448+0000","last_unstale":"2023-10-08T07:14:58.542027+0000","last_undegraded":"2023-10-08T07:14:58.542027+0000","last_fullsized":"2023-10-08T07:14:58.542027+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:03:40.822722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.547148+0000","last_change":"2023-10-08T07:14:58.539818+0000","last_active":"2023-10-08T07:14:59.547148+0000","last_peered":"2023-10-08T07:14:59.547148+0000","last_clean":"2023-10-08T07:14:59.547148+0000","last_became_active":"2023-10-08T07:14:58.539314+0000","last_became_peered":"2023-10-08T07:14:58.539314+0000","last_unstale":"2023-10-08T07:14:59.547148+0000","last_undegraded":"2023-10-08T07:14:59.547148+0000","last_fullsized":"2023-10-08T07:14:59.547148+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:25.327883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533152+0000","last_change":"2023-10-08T07:14:44.327892+0000","last_active":"2023-10-08T07:14:58.533152+0000","last_peered":"2023-10-08T07:14:58.533152+0000","last_clean":"2023-10-08T07:14:58.533152+0000","last_became_active":"2023-10-08T07:14:44.327480+0000","last_became_peered":"2023-10-08T07:14:44.327480+0000","last_unstale":"2023-10-08T07:14:58.533152+0000","last_undegraded":"2023-10-08T07:14:58.533152+0000","last_fullsized":"2023-10-08T07:14:58.533152+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:26.279630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.524587+0000","last_change":"2023-10-08T07:14:43.404280+0000","last_active":"2023-10-08T07:14:58.524587+0000","last_peered":"2023-10-08T07:14:58.524587+0000","last_clean":"2023-10-08T07:14:58.524587+0000","last_became_active":"2023-10-08T07:14:43.403911+0000","last_became_peered":"2023-10-08T07:14:43.403911+0000","last_unstale":"2023-10-08T07:14:58.524587+0000","last_undegraded":"2023-10-08T07:14:58.524587+0000","last_fullsized":"2023-10-08T07:14:58.524587+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:06.833347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.568684+0000","last_change":"2023-10-08T07:14:58.558886+0000","last_active":"2023-10-08T07:14:59.568684+0000","last_peered":"2023-10-08T07:14:59.568684+0000","last_clean":"2023-10-08T07:14:59.568684+0000","last_became_active":"2023-10-08T07:14:58.558626+0000","last_became_peered":"2023-10-08T07:14:58.558626+0000","last_unstale":"2023-10-08T07:14:59.568684+0000","last_undegraded":"2023-10-08T07:14:59.568684+0000","last_fullsized":"2023-10-08T07:14:59.568684+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:52.509843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.542943+0000","last_change":"2023-10-08T07:14:44.336892+0000","last_active":"2023-10-08T07:14:58.542943+0000","last_peered":"2023-10-08T07:14:58.542943+0000","last_clean":"2023-10-08T07:14:58.542943+0000","last_became_active":"2023-10-08T07:14:44.336589+0000","last_became_peered":"2023-10-08T07:14:44.336589+0000","last_unstale":"2023-10-08T07:14:58.542943+0000","last_undegraded":"2023-10-08T07:14:58.542943+0000","last_fullsized":"2023-10-08T07:14:58.542943+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:07.990108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.525639+0000","last_change":"2023-10-08T07:14:43.411394+0000","last_active":"2023-10-08T07:14:58.525639+0000","last_peered":"2023-10-08T07:14:58.525639+0000","last_clean":"2023-10-08T07:14:58.525639+0000","last_became_active":"2023-10-08T07:14:43.411091+0000","last_became_peered":"2023-10-08T07:14:43.411091+0000","last_unstale":"2023-10-08T07:14:58.525639+0000","last_undegraded":"2023-10-08T07:14:58.525639+0000","last_fullsized":"2023-10-08T07:14:58.525639+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:44:12.677401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546970+0000","last_change":"2023-10-08T07:14:58.549819+0000","last_active":"2023-10-08T07:14:59.546970+0000","last_peered":"2023-10-08T07:14:59.546970+0000","last_clean":"2023-10-08T07:14:59.546970+0000","last_became_active":"2023-10-08T07:14:58.549535+0000","last_became_peered":"2023-10-08T07:14:58.549535+0000","last_unstale":"2023-10-08T07:14:59.546970+0000","last_undegraded":"2023-10-08T07:14:59.546970+0000","last_fullsized":"2023-10-08T07:14:59.546970+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:52.541810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.527169+0000","last_change":"2023-10-08T07:14:44.340304+0000","last_active":"2023-10-08T07:14:58.527169+0000","last_peered":"2023-10-08T07:14:58.527169+0000","last_clean":"2023-10-08T07:14:58.527169+0000","last_became_active":"2023-10-08T07:14:44.338301+0000","last_became_peered":"2023-10-08T07:14:44.338301+0000","last_unstale":"2023-10-08T07:14:58.527169+0000","last_undegraded":"2023-10-08T07:14:58.527169+0000","last_fullsized":"2023-10-08T07:14:58.527169+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:42:14.658077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"253'8","reported_seq":24,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.527532+0000","last_change":"2023-10-08T07:14:43.407648+0000","last_active":"2023-10-08T07:14:58.527532+0000","last_peered":"2023-10-08T07:14:58.527532+0000","last_clean":"2023-10-08T07:14:58.527532+0000","last_became_active":"2023-10-08T07:14:43.407376+0000","last_became_peered":"2023-10-08T07:14:43.407376+0000","last_unstale":"2023-10-08T07:14:58.527532+0000","last_undegraded":"2023-10-08T07:14:58.527532+0000","last_fullsized":"2023-10-08T07:14:58.527532+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:26:50.563868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539729+0000","last_change":"2023-10-08T07:14:58.542960+0000","last_active":"2023-10-08T07:14:59.539729+0000","last_peered":"2023-10-08T07:14:59.539729+0000","last_clean":"2023-10-08T07:14:59.539729+0000","last_became_active":"2023-10-08T07:14:58.542310+0000","last_became_peered":"2023-10-08T07:14:58.542310+0000","last_unstale":"2023-10-08T07:14:59.539729+0000","last_undegraded":"2023-10-08T07:14:59.539729+0000","last_fullsized":"2023-10-08T07:14:59.539729+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:17.645216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.526793+0000","last_change":"2023-10-08T07:14:44.335197+0000","last_active":"2023-10-08T07:14:58.526793+0000","last_peered":"2023-10-08T07:14:58.526793+0000","last_clean":"2023-10-08T07:14:58.526793+0000","last_became_active":"2023-10-08T07:14:44.333014+0000","last_became_peered":"2023-10-08T07:14:44.333014+0000","last_unstale":"2023-10-08T07:14:58.526793+0000","last_undegraded":"2023-10-08T07:14:58.526793+0000","last_fullsized":"2023-10-08T07:14:58.526793+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:36:04.704806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533779+0000","last_change":"2023-10-08T07:14:43.411294+0000","last_active":"2023-10-08T07:14:58.533779+0000","last_peered":"2023-10-08T07:14:58.533779+0000","last_clean":"2023-10-08T07:14:58.533779+0000","last_became_active":"2023-10-08T07:14:43.411000+0000","last_became_peered":"2023-10-08T07:14:43.411000+0000","last_unstale":"2023-10-08T07:14:58.533779+0000","last_undegraded":"2023-10-08T07:14:58.533779+0000","last_fullsized":"2023-10-08T07:14:58.533779+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:42:35.351878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.566969+0000","last_change":"2023-10-08T07:14:58.551414+0000","last_active":"2023-10-08T07:14:59.566969+0000","last_peered":"2023-10-08T07:14:59.566969+0000","last_clean":"2023-10-08T07:14:59.566969+0000","last_became_active":"2023-10-08T07:14:58.550969+0000","last_became_peered":"2023-10-08T07:14:58.550969+0000","last_unstale":"2023-10-08T07:14:59.566969+0000","last_undegraded":"2023-10-08T07:14:59.566969+0000","last_fullsized":"2023-10-08T07:14:59.566969+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:29.829692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.525458+0000","last_change":"2023-10-08T07:14:44.331894+0000","last_active":"2023-10-08T07:14:58.525458+0000","last_peered":"2023-10-08T07:14:58.525458+0000","last_clean":"2023-10-08T07:14:58.525458+0000","last_became_active":"2023-10-08T07:14:44.331612+0000","last_became_peered":"2023-10-08T07:14:44.331612+0000","last_unstale":"2023-10-08T07:14:58.525458+0000","last_undegraded":"2023-10-08T07:14:58.525458+0000","last_fullsized":"2023-10-08T07:14:58.525458+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:17.150803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.538587+0000","last_change":"2023-10-08T07:14:43.410517+0000","last_active":"2023-10-08T07:14:58.538587+0000","last_peered":"2023-10-08T07:14:58.538587+0000","last_clean":"2023-10-08T07:14:58.538587+0000","last_became_active":"2023-10-08T07:14:43.409771+0000","last_became_peered":"2023-10-08T07:14:43.409771+0000","last_unstale":"2023-10-08T07:14:58.538587+0000","last_undegraded":"2023-10-08T07:14:58.538587+0000","last_fullsized":"2023-10-08T07:14:58.538587+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:42.844452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.542968+0000","last_change":"2023-10-08T07:14:58.541999+0000","last_active":"2023-10-08T07:14:59.542968+0000","last_peered":"2023-10-08T07:14:59.542968+0000","last_clean":"2023-10-08T07:14:59.542968+0000","last_became_active":"2023-10-08T07:14:58.540640+0000","last_became_peered":"2023-10-08T07:14:58.540640+0000","last_unstale":"2023-10-08T07:14:59.542968+0000","last_undegraded":"2023-10-08T07:14:59.542968+0000","last_fullsized":"2023-10-08T07:14:59.542968+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:12:18.752597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532709+0000","last_change":"2023-10-08T07:14:44.342867+0000","last_active":"2023-10-08T07:14:58.532709+0000","last_peered":"2023-10-08T07:14:58.532709+0000","last_clean":"2023-10-08T07:14:58.532709+0000","last_became_active":"2023-10-08T07:14:44.342541+0000","last_became_peered":"2023-10-08T07:14:44.342541+0000","last_unstale":"2023-10-08T07:14:58.532709+0000","last_undegraded":"2023-10-08T07:14:58.532709+0000","last_fullsized":"2023-10-08T07:14:58.532709+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:43.616734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532593+0000","last_change":"2023-10-08T07:14:43.413188+0000","last_active":"2023-10-08T07:14:58.532593+0000","last_peered":"2023-10-08T07:14:58.532593+0000","last_clean":"2023-10-08T07:14:58.532593+0000","last_became_active":"2023-10-08T07:14:43.412716+0000","last_became_peered":"2023-10-08T07:14:43.412716+0000","last_unstale":"2023-10-08T07:14:58.532593+0000","last_undegraded":"2023-10-08T07:14:58.532593+0000","last_fullsized":"2023-10-08T07:14:58.532593+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:08.671468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539173+0000","last_change":"2023-10-08T07:14:58.540678+0000","last_active":"2023-10-08T07:14:59.539173+0000","last_peered":"2023-10-08T07:14:59.539173+0000","last_clean":"2023-10-08T07:14:59.539173+0000","last_became_active":"2023-10-08T07:14:58.540408+0000","last_became_peered":"2023-10-08T07:14:58.540408+0000","last_unstale":"2023-10-08T07:14:59.539173+0000","last_undegraded":"2023-10-08T07:14:59.539173+0000","last_fullsized":"2023-10-08T07:14:59.539173+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:36.939019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.544328+0000","last_change":"2023-10-08T07:14:44.332886+0000","last_active":"2023-10-08T07:14:58.544328+0000","last_peered":"2023-10-08T07:14:58.544328+0000","last_clean":"2023-10-08T07:14:58.544328+0000","last_became_active":"2023-10-08T07:14:44.332142+0000","last_became_peered":"2023-10-08T07:14:44.332142+0000","last_unstale":"2023-10-08T07:14:58.544328+0000","last_undegraded":"2023-10-08T07:14:58.544328+0000","last_fullsized":"2023-10-08T07:14:58.544328+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:49.297056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.547760+0000","last_change":"2023-10-08T07:14:43.415034+0000","last_active":"2023-10-08T07:14:58.547760+0000","last_peered":"2023-10-08T07:14:58.547760+0000","last_clean":"2023-10-08T07:14:58.547760+0000","last_became_active":"2023-10-08T07:14:43.414728+0000","last_became_peered":"2023-10-08T07:14:43.414728+0000","last_unstale":"2023-10-08T07:14:58.547760+0000","last_undegraded":"2023-10-08T07:14:58.547760+0000","last_fullsized":"2023-10-08T07:14:58.547760+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:08.048265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.547212+0000","last_change":"2023-10-08T07:14:58.550042+0000","last_active":"2023-10-08T07:14:59.547212+0000","last_peered":"2023-10-08T07:14:59.547212+0000","last_clean":"2023-10-08T07:14:59.547212+0000","last_became_active":"2023-10-08T07:14:58.548159+0000","last_became_peered":"2023-10-08T07:14:58.548159+0000","last_unstale":"2023-10-08T07:14:59.547212+0000","last_undegraded":"2023-10-08T07:14:59.547212+0000","last_fullsized":"2023-10-08T07:14:59.547212+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:19.898264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.538044+0000","last_change":"2023-10-08T07:14:44.340208+0000","last_active":"2023-10-08T07:14:58.538044+0000","last_peered":"2023-10-08T07:14:58.538044+0000","last_clean":"2023-10-08T07:14:58.538044+0000","last_became_active":"2023-10-08T07:14:44.339934+0000","last_became_peered":"2023-10-08T07:14:44.339934+0000","last_unstale":"2023-10-08T07:14:58.538044+0000","last_undegraded":"2023-10-08T07:14:58.538044+0000","last_fullsized":"2023-10-08T07:14:58.538044+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:26.451027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.544409+0000","last_change":"2023-10-08T07:14:43.413078+0000","last_active":"2023-10-08T07:14:58.544409+0000","last_peered":"2023-10-08T07:14:58.544409+0000","last_clean":"2023-10-08T07:14:58.544409+0000","last_became_active":"2023-10-08T07:14:43.412804+0000","last_became_peered":"2023-10-08T07:14:43.412804+0000","last_unstale":"2023-10-08T07:14:58.544409+0000","last_undegraded":"2023-10-08T07:14:58.544409+0000","last_fullsized":"2023-10-08T07:14:58.544409+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:12.105709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.568010+0000","last_change":"2023-10-08T07:14:58.558588+0000","last_active":"2023-10-08T07:14:59.568010+0000","last_peered":"2023-10-08T07:14:59.568010+0000","last_clean":"2023-10-08T07:14:59.568010+0000","last_became_active":"2023-10-08T07:14:58.558277+0000","last_became_peered":"2023-10-08T07:14:58.558277+0000","last_unstale":"2023-10-08T07:14:59.568010+0000","last_undegraded":"2023-10-08T07:14:59.568010+0000","last_fullsized":"2023-10-08T07:14:59.568010+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:04.003417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.526558+0000","last_change":"2023-10-08T07:14:44.333114+0000","last_active":"2023-10-08T07:14:58.526558+0000","last_peered":"2023-10-08T07:14:58.526558+0000","last_clean":"2023-10-08T07:14:58.526558+0000","last_became_active":"2023-10-08T07:14:44.332105+0000","last_became_peered":"2023-10-08T07:14:44.332105+0000","last_unstale":"2023-10-08T07:14:58.526558+0000","last_undegraded":"2023-10-08T07:14:58.526558+0000","last_fullsized":"2023-10-08T07:14:58.526558+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:58:23.020597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.525803+0000","last_change":"2023-10-08T07:14:43.407284+0000","last_active":"2023-10-08T07:14:58.525803+0000","last_peered":"2023-10-08T07:14:58.525803+0000","last_clean":"2023-10-08T07:14:58.525803+0000","last_became_active":"2023-10-08T07:14:43.407010+0000","last_became_peered":"2023-10-08T07:14:43.407010+0000","last_unstale":"2023-10-08T07:14:58.525803+0000","last_undegraded":"2023-10-08T07:14:58.525803+0000","last_fullsized":"2023-10-08T07:14:58.525803+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:18.544340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.568531+0000","last_change":"2023-10-08T07:14:58.558284+0000","last_active":"2023-10-08T07:14:59.568531+0000","last_peered":"2023-10-08T07:14:59.568531+0000","last_clean":"2023-10-08T07:14:59.568531+0000","last_became_active":"2023-10-08T07:14:58.557913+0000","last_became_peered":"2023-10-08T07:14:58.557913+0000","last_unstale":"2023-10-08T07:14:59.568531+0000","last_undegraded":"2023-10-08T07:14:59.568531+0000","last_fullsized":"2023-10-08T07:14:59.568531+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:04:23.513726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530729+0000","last_change":"2023-10-08T07:14:44.339779+0000","last_active":"2023-10-08T07:14:58.530729+0000","last_peered":"2023-10-08T07:14:58.530729+0000","last_clean":"2023-10-08T07:14:58.530729+0000","last_became_active":"2023-10-08T07:14:44.339385+0000","last_became_peered":"2023-10-08T07:14:44.339385+0000","last_unstale":"2023-10-08T07:14:58.530729+0000","last_undegraded":"2023-10-08T07:14:58.530729+0000","last_fullsized":"2023-10-08T07:14:58.530729+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:44:13.695913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.526327+0000","last_change":"2023-10-08T07:14:43.408990+0000","last_active":"2023-10-08T07:14:58.526327+0000","last_peered":"2023-10-08T07:14:58.526327+0000","last_clean":"2023-10-08T07:14:58.526327+0000","last_became_active":"2023-10-08T07:14:43.408702+0000","last_became_peered":"2023-10-08T07:14:43.408702+0000","last_unstale":"2023-10-08T07:14:58.526327+0000","last_undegraded":"2023-10-08T07:14:58.526327+0000","last_fullsized":"2023-10-08T07:14:58.526327+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:54.133156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.545965+0000","last_change":"2023-10-08T07:14:58.553095+0000","last_active":"2023-10-08T07:14:59.545965+0000","last_peered":"2023-10-08T07:14:59.545965+0000","last_clean":"2023-10-08T07:14:59.545965+0000","last_became_active":"2023-10-08T07:14:58.552482+0000","last_became_peered":"2023-10-08T07:14:58.552482+0000","last_unstale":"2023-10-08T07:14:59.545965+0000","last_undegraded":"2023-10-08T07:14:59.545965+0000","last_fullsized":"2023-10-08T07:14:59.545965+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:18.387133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530685+0000","last_change":"2023-10-08T07:14:44.333909+0000","last_active":"2023-10-08T07:14:58.530685+0000","last_peered":"2023-10-08T07:14:58.530685+0000","last_clean":"2023-10-08T07:14:58.530685+0000","last_became_active":"2023-10-08T07:14:44.333646+0000","last_became_peered":"2023-10-08T07:14:44.333646+0000","last_unstale":"2023-10-08T07:14:58.530685+0000","last_undegraded":"2023-10-08T07:14:58.530685+0000","last_fullsized":"2023-10-08T07:14:58.530685+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:50.362306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.515570+0000","last_change":"2023-10-08T07:14:43.406333+0000","last_active":"2023-10-08T07:14:58.515570+0000","last_peered":"2023-10-08T07:14:58.515570+0000","last_clean":"2023-10-08T07:14:58.515570+0000","last_became_active":"2023-10-08T07:14:43.406052+0000","last_became_peered":"2023-10-08T07:14:43.406052+0000","last_unstale":"2023-10-08T07:14:58.515570+0000","last_undegraded":"2023-10-08T07:14:58.515570+0000","last_fullsized":"2023-10-08T07:14:58.515570+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:31.926662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.566662+0000","last_change":"2023-10-08T07:14:58.548464+0000","last_active":"2023-10-08T07:14:59.566662+0000","last_peered":"2023-10-08T07:14:59.566662+0000","last_clean":"2023-10-08T07:14:59.566662+0000","last_became_active":"2023-10-08T07:14:58.548017+0000","last_became_peered":"2023-10-08T07:14:58.548017+0000","last_unstale":"2023-10-08T07:14:59.566662+0000","last_undegraded":"2023-10-08T07:14:59.566662+0000","last_fullsized":"2023-10-08T07:14:59.566662+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:25.034062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.536582+0000","last_change":"2023-10-08T07:14:44.338131+0000","last_active":"2023-10-08T07:14:58.536582+0000","last_peered":"2023-10-08T07:14:58.536582+0000","last_clean":"2023-10-08T07:14:58.536582+0000","last_became_active":"2023-10-08T07:14:44.337734+0000","last_became_peered":"2023-10-08T07:14:44.337734+0000","last_unstale":"2023-10-08T07:14:58.536582+0000","last_undegraded":"2023-10-08T07:14:58.536582+0000","last_fullsized":"2023-10-08T07:14:58.536582+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:29:05.679033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.554034+0000","last_change":"2023-10-08T07:14:43.414871+0000","last_active":"2023-10-08T07:14:58.554034+0000","last_peered":"2023-10-08T07:14:58.554034+0000","last_clean":"2023-10-08T07:14:58.554034+0000","last_became_active":"2023-10-08T07:14:43.414608+0000","last_became_peered":"2023-10-08T07:14:43.414608+0000","last_unstale":"2023-10-08T07:14:58.554034+0000","last_undegraded":"2023-10-08T07:14:58.554034+0000","last_fullsized":"2023-10-08T07:14:58.554034+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:35.056822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567444+0000","last_change":"2023-10-08T07:14:58.547522+0000","last_active":"2023-10-08T07:14:59.567444+0000","last_peered":"2023-10-08T07:14:59.567444+0000","last_clean":"2023-10-08T07:14:59.567444+0000","last_became_active":"2023-10-08T07:14:58.547139+0000","last_became_peered":"2023-10-08T07:14:58.547139+0000","last_unstale":"2023-10-08T07:14:59.567444+0000","last_undegraded":"2023-10-08T07:14:59.567444+0000","last_fullsized":"2023-10-08T07:14:59.567444+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:39.044877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540503+0000","last_change":"2023-10-08T07:14:44.330281+0000","last_active":"2023-10-08T07:14:58.540503+0000","last_peered":"2023-10-08T07:14:58.540503+0000","last_clean":"2023-10-08T07:14:58.540503+0000","last_became_active":"2023-10-08T07:14:44.329938+0000","last_became_peered":"2023-10-08T07:14:44.329938+0000","last_unstale":"2023-10-08T07:14:58.540503+0000","last_undegraded":"2023-10-08T07:14:58.540503+0000","last_fullsized":"2023-10-08T07:14:58.540503+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:16.367973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"254'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530270+0000","last_change":"2023-10-08T07:14:43.414678+0000","last_active":"2023-10-08T07:14:58.530270+0000","last_peered":"2023-10-08T07:14:58.530270+0000","last_clean":"2023-10-08T07:14:58.530270+0000","last_became_active":"2023-10-08T07:14:43.414400+0000","last_became_peered":"2023-10-08T07:14:43.414400+0000","last_unstale":"2023-10-08T07:14:58.530270+0000","last_undegraded":"2023-10-08T07:14:58.530270+0000","last_fullsized":"2023-10-08T07:14:58.530270+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:43:12.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.564904+0000","last_change":"2023-10-08T07:14:58.549721+0000","last_active":"2023-10-08T07:14:59.564904+0000","last_peered":"2023-10-08T07:14:59.564904+0000","last_clean":"2023-10-08T07:14:59.564904+0000","last_became_active":"2023-10-08T07:14:58.549442+0000","last_became_peered":"2023-10-08T07:14:58.549442+0000","last_unstale":"2023-10-08T07:14:59.564904+0000","last_undegraded":"2023-10-08T07:14:59.564904+0000","last_fullsized":"2023-10-08T07:14:59.564904+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:42.784401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.542285+0000","last_change":"2023-10-08T07:14:44.337389+0000","last_active":"2023-10-08T07:14:58.542285+0000","last_peered":"2023-10-08T07:14:58.542285+0000","last_clean":"2023-10-08T07:14:58.542285+0000","last_became_active":"2023-10-08T07:14:44.337118+0000","last_became_peered":"2023-10-08T07:14:44.337118+0000","last_unstale":"2023-10-08T07:14:58.542285+0000","last_undegraded":"2023-10-08T07:14:58.542285+0000","last_fullsized":"2023-10-08T07:14:58.542285+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:11.369315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.518892+0000","last_change":"2023-10-08T07:14:43.404901+0000","last_active":"2023-10-08T07:14:58.518892+0000","last_peered":"2023-10-08T07:14:58.518892+0000","last_clean":"2023-10-08T07:14:58.518892+0000","last_became_active":"2023-10-08T07:14:43.404611+0000","last_became_peered":"2023-10-08T07:14:43.404611+0000","last_unstale":"2023-10-08T07:14:58.518892+0000","last_undegraded":"2023-10-08T07:14:58.518892+0000","last_fullsized":"2023-10-08T07:14:58.518892+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:51.297454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567736+0000","last_change":"2023-10-08T07:14:58.544019+0000","last_active":"2023-10-08T07:14:59.567736+0000","last_peered":"2023-10-08T07:14:59.567736+0000","last_clean":"2023-10-08T07:14:59.567736+0000","last_became_active":"2023-10-08T07:14:58.543727+0000","last_became_peered":"2023-10-08T07:14:58.543727+0000","last_unstale":"2023-10-08T07:14:59.567736+0000","last_undegraded":"2023-10-08T07:14:59.567736+0000","last_fullsized":"2023-10-08T07:14:59.567736+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:08.605466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.522845+0000","last_change":"2023-10-08T07:14:44.332132+0000","last_active":"2023-10-08T07:14:58.522845+0000","last_peered":"2023-10-08T07:14:58.522845+0000","last_clean":"2023-10-08T07:14:58.522845+0000","last_became_active":"2023-10-08T07:14:44.331822+0000","last_became_peered":"2023-10-08T07:14:44.331822+0000","last_unstale":"2023-10-08T07:14:58.522845+0000","last_undegraded":"2023-10-08T07:14:58.522845+0000","last_fullsized":"2023-10-08T07:14:58.522845+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:21.791606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.517398+0000","last_change":"2023-10-08T07:14:43.409897+0000","last_active":"2023-10-08T07:14:58.517398+0000","last_peered":"2023-10-08T07:14:58.517398+0000","last_clean":"2023-10-08T07:14:58.517398+0000","last_became_active":"2023-10-08T07:14:43.409442+0000","last_became_peered":"2023-10-08T07:14:43.409442+0000","last_unstale":"2023-10-08T07:14:58.517398+0000","last_undegraded":"2023-10-08T07:14:58.517398+0000","last_fullsized":"2023-10-08T07:14:58.517398+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:12.105709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539858+0000","last_change":"2023-10-08T07:14:58.543755+0000","last_active":"2023-10-08T07:14:59.539858+0000","last_peered":"2023-10-08T07:14:59.539858+0000","last_clean":"2023-10-08T07:14:59.539858+0000","last_became_active":"2023-10-08T07:14:58.543403+0000","last_became_peered":"2023-10-08T07:14:58.543403+0000","last_unstale":"2023-10-08T07:14:59.539858+0000","last_undegraded":"2023-10-08T07:14:59.539858+0000","last_fullsized":"2023-10-08T07:14:59.539858+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:48.166629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.515597+0000","last_change":"2023-10-08T07:14:44.330119+0000","last_active":"2023-10-08T07:14:58.515597+0000","last_peered":"2023-10-08T07:14:58.515597+0000","last_clean":"2023-10-08T07:14:58.515597+0000","last_became_active":"2023-10-08T07:14:44.329841+0000","last_became_peered":"2023-10-08T07:14:44.329841+0000","last_unstale":"2023-10-08T07:14:58.515597+0000","last_undegraded":"2023-10-08T07:14:58.515597+0000","last_fullsized":"2023-10-08T07:14:58.515597+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:17.916779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529408+0000","last_change":"2023-10-08T07:14:43.406046+0000","last_active":"2023-10-08T07:14:58.529408+0000","last_peered":"2023-10-08T07:14:58.529408+0000","last_clean":"2023-10-08T07:14:58.529408+0000","last_became_active":"2023-10-08T07:14:43.405138+0000","last_became_peered":"2023-10-08T07:14:43.405138+0000","last_unstale":"2023-10-08T07:14:58.529408+0000","last_undegraded":"2023-10-08T07:14:58.529408+0000","last_fullsized":"2023-10-08T07:14:58.529408+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:53.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.547225+0000","last_change":"2023-10-08T07:14:58.545250+0000","last_active":"2023-10-08T07:14:59.547225+0000","last_peered":"2023-10-08T07:14:59.547225+0000","last_clean":"2023-10-08T07:14:59.547225+0000","last_became_active":"2023-10-08T07:14:58.544873+0000","last_became_peered":"2023-10-08T07:14:58.544873+0000","last_unstale":"2023-10-08T07:14:59.547225+0000","last_undegraded":"2023-10-08T07:14:59.547225+0000","last_fullsized":"2023-10-08T07:14:59.547225+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:15.292591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532249+0000","last_change":"2023-10-08T07:14:44.329222+0000","last_active":"2023-10-08T07:14:58.532249+0000","last_peered":"2023-10-08T07:14:58.532249+0000","last_clean":"2023-10-08T07:14:58.532249+0000","last_became_active":"2023-10-08T07:14:44.328917+0000","last_became_peered":"2023-10-08T07:14:44.328917+0000","last_unstale":"2023-10-08T07:14:58.532249+0000","last_undegraded":"2023-10-08T07:14:58.532249+0000","last_fullsized":"2023-10-08T07:14:58.532249+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:35.827786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.524808+0000","last_change":"2023-10-08T07:14:43.411524+0000","last_active":"2023-10-08T07:14:58.524808+0000","last_peered":"2023-10-08T07:14:58.524808+0000","last_clean":"2023-10-08T07:14:58.524808+0000","last_became_active":"2023-10-08T07:14:43.411151+0000","last_became_peered":"2023-10-08T07:14:43.411151+0000","last_unstale":"2023-10-08T07:14:58.524808+0000","last_undegraded":"2023-10-08T07:14:58.524808+0000","last_fullsized":"2023-10-08T07:14:58.524808+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:49.845188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567060+0000","last_change":"2023-10-08T07:14:58.552539+0000","last_active":"2023-10-08T07:14:59.567060+0000","last_peered":"2023-10-08T07:14:59.567060+0000","last_clean":"2023-10-08T07:14:59.567060+0000","last_became_active":"2023-10-08T07:14:58.552220+0000","last_became_peered":"2023-10-08T07:14:58.552220+0000","last_unstale":"2023-10-08T07:14:59.567060+0000","last_undegraded":"2023-10-08T07:14:59.567060+0000","last_fullsized":"2023-10-08T07:14:59.567060+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:47.118336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.536999+0000","last_change":"2023-10-08T07:14:44.335115+0000","last_active":"2023-10-08T07:14:58.536999+0000","last_peered":"2023-10-08T07:14:58.536999+0000","last_clean":"2023-10-08T07:14:58.536999+0000","last_became_active":"2023-10-08T07:14:44.334638+0000","last_became_peered":"2023-10-08T07:14:44.334638+0000","last_unstale":"2023-10-08T07:14:58.536999+0000","last_undegraded":"2023-10-08T07:14:58.536999+0000","last_fullsized":"2023-10-08T07:14:58.536999+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:36.291831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.538555+0000","last_change":"2023-10-08T07:14:43.408103+0000","last_active":"2023-10-08T07:14:58.538555+0000","last_peered":"2023-10-08T07:14:58.538555+0000","last_clean":"2023-10-08T07:14:58.538555+0000","last_became_active":"2023-10-08T07:14:43.407827+0000","last_became_peered":"2023-10-08T07:14:43.407827+0000","last_unstale":"2023-10-08T07:14:58.538555+0000","last_undegraded":"2023-10-08T07:14:58.538555+0000","last_fullsized":"2023-10-08T07:14:58.538555+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:50.115871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.569529+0000","last_change":"2023-10-08T07:14:58.542014+0000","last_active":"2023-10-08T07:14:59.569529+0000","last_peered":"2023-10-08T07:14:59.569529+0000","last_clean":"2023-10-08T07:14:59.569529+0000","last_became_active":"2023-10-08T07:14:58.541719+0000","last_became_peered":"2023-10-08T07:14:58.541719+0000","last_unstale":"2023-10-08T07:14:59.569529+0000","last_undegraded":"2023-10-08T07:14:59.569529+0000","last_fullsized":"2023-10-08T07:14:59.569529+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:10:22.724101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530564+0000","last_change":"2023-10-08T07:14:44.334559+0000","last_active":"2023-10-08T07:14:58.530564+0000","last_peered":"2023-10-08T07:14:58.530564+0000","last_clean":"2023-10-08T07:14:58.530564+0000","last_became_active":"2023-10-08T07:14:44.334019+0000","last_became_peered":"2023-10-08T07:14:44.334019+0000","last_unstale":"2023-10-08T07:14:58.530564+0000","last_undegraded":"2023-10-08T07:14:58.530564+0000","last_fullsized":"2023-10-08T07:14:58.530564+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:59.631069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.522696+0000","last_change":"2023-10-08T07:14:43.413981+0000","last_active":"2023-10-08T07:14:58.522696+0000","last_peered":"2023-10-08T07:14:58.522696+0000","last_clean":"2023-10-08T07:14:58.522696+0000","last_became_active":"2023-10-08T07:14:43.413671+0000","last_became_peered":"2023-10-08T07:14:43.413671+0000","last_unstale":"2023-10-08T07:14:58.522696+0000","last_undegraded":"2023-10-08T07:14:58.522696+0000","last_fullsized":"2023-10-08T07:14:58.522696+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:35.209529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.566979+0000","last_change":"2023-10-08T07:14:58.551380+0000","last_active":"2023-10-08T07:14:59.566979+0000","last_peered":"2023-10-08T07:14:59.566979+0000","last_clean":"2023-10-08T07:14:59.566979+0000","last_became_active":"2023-10-08T07:14:58.550620+0000","last_became_peered":"2023-10-08T07:14:58.550620+0000","last_unstale":"2023-10-08T07:14:59.566979+0000","last_undegraded":"2023-10-08T07:14:59.566979+0000","last_fullsized":"2023-10-08T07:14:59.566979+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:15.253247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.531430+0000","last_change":"2023-10-08T07:14:44.336800+0000","last_active":"2023-10-08T07:14:58.531430+0000","last_peered":"2023-10-08T07:14:58.531430+0000","last_clean":"2023-10-08T07:14:58.531430+0000","last_became_active":"2023-10-08T07:14:44.331013+0000","last_became_peered":"2023-10-08T07:14:44.331013+0000","last_unstale":"2023-10-08T07:14:58.531430+0000","last_undegraded":"2023-10-08T07:14:58.531430+0000","last_fullsized":"2023-10-08T07:14:58.531430+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:39.261986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.536043+0000","last_change":"2023-10-08T07:14:43.410107+0000","last_active":"2023-10-08T07:14:58.536043+0000","last_peered":"2023-10-08T07:14:58.536043+0000","last_clean":"2023-10-08T07:14:58.536043+0000","last_became_active":"2023-10-08T07:14:43.409781+0000","last_became_peered":"2023-10-08T07:14:43.409781+0000","last_unstale":"2023-10-08T07:14:58.536043+0000","last_undegraded":"2023-10-08T07:14:58.536043+0000","last_fullsized":"2023-10-08T07:14:58.536043+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:08:03.181094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"240'2828","reported_seq":3821,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.543364+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:14:58.543364+0000","last_peered":"2023-10-08T07:14:58.543364+0000","last_clean":"2023-10-08T07:14:58.543364+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:14:58.543364+0000","last_undegraded":"2023-10-08T07:14:58.543364+0000","last_fullsized":"2023-10-08T07:14:58.543364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2828,"log_dups_size":0,"ondisk_log_size":2828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":629,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":629,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2828,"ondisk_log_size":2828,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576025,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48844,"kb_used_data":3768,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322996,"statfs":{"total":96636764160,"available":96586747904,"internally_reserved":0,"allocated":3858432,"data_stored":2289942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.127,"15min":1.1910000000000001},"min":{"1min":0.81699999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.968,"15min":2.008},"last":1.006},{"interface":"front","average":{"1min":1.119,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.81999999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.516,"5min":1.788,"15min":1.788},"last":1.399}]},{"osd":1,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.2370000000000001,"15min":1.2509999999999999},"min":{"1min":0.93899999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.998},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.196,"5min":1.2969999999999999,"15min":1.329},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6299999999999999,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.089}]},{"osd":2,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2370000000000001,"15min":1.2929999999999999},"min":{"1min":0.89500000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":2.1640000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.369,"15min":1.3500000000000001},"min":{"1min":0.93400000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.901,"5min":1.901,"15min":2.3170000000000002},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.298,"15min":1.3340000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.401,"5min":1.3819999999999999,"15min":1.3999999999999999},"min":{"1min":0.97099999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.149}]},{"osd":4,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.83999999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9319999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4139999999999999,"15min":1.411},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.365}]},{"osd":5,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.478,"15min":1.4490000000000001},"min":{"1min":1.127,"5min":0.80600000000000005,"15min":0.79300000000000004},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.488,"15min":1.4950000000000001},"min":{"1min":1.012,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.056,"5min":2.23,"15min":2.23},"last":1.3500000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.599,"15min":1.597},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.202,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.462},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.5640000000000001,"15min":1.5429999999999999},"min":{"1min":1.21,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2570000000000001},"last":1.2090000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576024,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30212,"kb_used_data":3184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341628,"statfs":{"total":96636764160,"available":96605827072,"internally_reserved":0,"allocated":3260416,"data_stored":1698992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.286,"15min":1.3089999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.637,"5min":1.794,"15min":1.962},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3120000000000001,"15min":1.3520000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6220000000000001,"5min":1.76,"15min":2.0070000000000001},"last":1.3520000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2030000000000001,"15min":1.2270000000000001},"min":{"1min":0.71999999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.7330000000000001,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.222,"15min":1.2430000000000001},"min":{"1min":0.71799999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.262}]},{"osd":2,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.1599999999999999,"15min":1.2310000000000001},"min":{"1min":0.86899999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":2.339},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2749999999999999,"15min":1.306},"min":{"1min":0.77100000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.627,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":1.032}]},{"osd":3,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.397},"min":{"1min":0.86299999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8280000000000001,"5min":1.9319999999999999,"15min":1.95},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.429,"15min":1.4890000000000001},"min":{"1min":0.95199999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.7789999999999999,"5min":1.8959999999999999,"15min":2.1459999999999999},"last":1.5629999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3720000000000001,"15min":1.389},"min":{"1min":0.877,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.849,"5min":1.9359999999999999,"15min":2.0859999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.4550000000000001,"15min":1.4570000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":1.9650000000000001,"5min":2.1110000000000002,"15min":2.484},"last":1.474}]},{"osd":5,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.389,"15min":1.4279999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8069999999999999,"5min":1.8660000000000001,"15min":2.403},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.5489999999999999,"15min":1.5549999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":1.653}]},{"osd":7,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5369999999999999,"15min":1.5760000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.339,"5min":2.339,"15min":2.5870000000000002},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.552,"15min":1.5620000000000001},"min":{"1min":0.99299999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.0710000000000002,"5min":2.2959999999999998,"15min":2.665},"last":1.6080000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576024,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60724,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311116,"statfs":{"total":96636764160,"available":96574582784,"internally_reserved":0,"allocated":3252224,"data_stored":1698916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.161,"15min":1.2070000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1930000000000001,"15min":1.2230000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.90300000000000002}]},{"osd":1,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1970000000000001,"15min":1.2370000000000001},"min":{"1min":0.66800000000000004,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.339,"15min":1.3120000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":7.5250000000000004,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":1.2350000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.266,"15min":1.3009999999999999},"min":{"1min":0.68899999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.0870000000000002},"last":1.468},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2450000000000001,"15min":1.3009999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.645,"5min":1.877,"15min":1.909},"last":0.77600000000000002}]},{"osd":3,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.4159999999999999,"15min":1.391},"min":{"1min":0.72499999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":6.8449999999999998,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":0.64900000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":9.3599999999999994,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":1.048}]},{"osd":4,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.319,"15min":1.3280000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4319999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.742,"5min":2.742,"15min":2.742},"last":1.4239999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.978,"5min":1.5329999999999999,"15min":1.486},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":13.234999999999999,"5min":13.234999999999999,"15min":13.234999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.5129999999999999,"15min":1.482},"min":{"1min":0.67800000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":7.2949999999999999,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":1.1830000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.536,"15min":1.5449999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5700000000000001,"15min":1.601},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":1.5720000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576024,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48256,"kb_used_data":3188,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323584,"statfs":{"total":96636764160,"available":96587350016,"internally_reserved":0,"allocated":3264512,"data_stored":1700528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.323,"15min":1.3520000000000001},"min":{"1min":0.66800000000000004,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":5.4800000000000004,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.522},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.421,"15min":1.4219999999999999},"min":{"1min":0.79200000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.0880000000000001,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.482}]},{"osd":1,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.72599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.3129999999999999,"15min":1.2729999999999999},"min":{"1min":0.61299999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":7.673,"5min":7.673,"15min":7.673},"last":1.3640000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":2.2629999999999999,"5min":1.4690000000000001,"15min":1.3260000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":17.175999999999998,"5min":17.175999999999998,"15min":17.175999999999998},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.651,"5min":1.4139999999999999,"15min":1.3859999999999999},"min":{"1min":0.629,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":1.212}]},{"osd":3,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.3600000000000001,"15min":1.347},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":7.5270000000000001,"5min":7.5270000000000001,"15min":7.5270000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.5209999999999999,"15min":1.4430000000000001},"min":{"1min":0.59199999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":7.7599999999999998,"5min":7.7599999999999998,"15min":7.7599999999999998},"last":1.268}]},{"osd":5,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.55,"15min":1.419},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":7.4020000000000001,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.149},{"interface":"front","average":{"1min":1.754,"5min":1.4970000000000001,"15min":1.415},"min":{"1min":0.442,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":7.79,"5min":7.79,"15min":7.79},"last":1.103}]},{"osd":6,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.621,"15min":1.5},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":10.542999999999999,"5min":10.542999999999999,"15min":10.542999999999999},"last":1.395},{"interface":"front","average":{"1min":2.1819999999999999,"5min":1.7150000000000001,"15min":1.605},"min":{"1min":0.86099999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":10.433999999999999,"5min":10.433999999999999,"15min":10.433999999999999},"last":1.3320000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.8899999999999999,"5min":1.6259999999999999,"15min":1.5409999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":7.915,"5min":7.915,"15min":7.915},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.8839999999999999,"5min":1.73,"15min":1.6559999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":1.5820000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576026,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56596,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315244,"statfs":{"total":96636764160,"available":96578809856,"internally_reserved":0,"allocated":3276800,"data_stored":1702431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92700000000000005,"15min":0.97399999999999998},"min":{"1min":0.32200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1990000000000001,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":1.323},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.131,"15min":1.1659999999999999},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5349999999999999,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.622}]},{"osd":1,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.96799999999999997,"15min":1.022},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3560000000000001,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.054,"5min":1.1890000000000001,"15min":1.2729999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.667,"5min":2.109,"15min":2.4889999999999999},"last":1.0169999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0629999999999999,"15min":1.1399999999999999},"min":{"1min":0.502,"5min":0.437,"15min":0.437},"max":{"1min":1.462,"5min":2.165,"15min":2.2250000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.139,"15min":1.1899999999999999},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.671,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2290000000000001,"15min":1.272},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.67500000000000004},"max":{"1min":1.7290000000000001,"5min":1.857,"15min":2.5379999999999998},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.306,"5min":1.381,"15min":1.45},"min":{"1min":0.91600000000000004,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.8149999999999999,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.2430000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.294,"15min":1.3660000000000001},"min":{"1min":0.76700000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7829999999999999,"5min":1.923,"15min":2.9740000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3919999999999999,"15min":1.4330000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9610000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":1.413}]},{"osd":6,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.82299999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.9690000000000001,"5min":1.976,"15min":2.653},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.4350000000000001,"15min":1.5129999999999999},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.0150000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.306}]},{"osd":7,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.512,"15min":1.5580000000000001},"min":{"1min":0.89100000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.3220000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4490000000000001,"15min":1.512},"min":{"1min":0.80900000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.903,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.177}]}]},{"osd":2,"up_from":13,"seq":55834576024,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52436,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319404,"statfs":{"total":96636764160,"available":96583069696,"internally_reserved":0,"allocated":3276800,"data_stored":1701714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.91600000000000004,"15min":0.97199999999999998},"min":{"1min":0.49199999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.948},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.008,"15min":1.0660000000000001},"min":{"1min":0.55700000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.423,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":0.94499999999999995}]},{"osd":1,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.93400000000000005,"15min":0.98999999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.413,"5min":1.6279999999999999,"15min":2.044},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.1479999999999999,"15min":1.1890000000000001},"min":{"1min":0.61499999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":2.1309999999999998},"last":0.57399999999999995}]},{"osd":3,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.141,"15min":1.177},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.786,"5min":1.786,"15min":1.8420000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.236,"15min":1.272},"min":{"1min":0.745,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.663,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":0.68799999999999994}]},{"osd":4,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.391,"15min":1.4379999999999999},"min":{"1min":0.92900000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.992,"5min":1.992,"15min":2.1789999999999998},"last":0.748},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.304,"15min":1.3120000000000001},"min":{"1min":0.87,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.843,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.88400000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3600000000000001,"15min":1.4019999999999999},"min":{"1min":0.89200000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.931,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.4930000000000001,"15min":1.532},"min":{"1min":1.1060000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.837,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":0.82399999999999995}]},{"osd":6,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.361,"15min":1.395},"min":{"1min":1.0549999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.722,"5min":2.056,"15min":2.1240000000000001},"last":1.073},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4830000000000001,"15min":1.5249999999999999},"min":{"1min":0.99299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.994,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":0.84399999999999997}]},{"osd":7,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.446,"15min":1.4790000000000001},"min":{"1min":1.0800000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7150000000000001,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.51,"15min":1.577},"min":{"1min":1.028,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.056,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":0.92800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834576024,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52416,"kb_used_data":3180,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319424,"statfs":{"total":96636764160,"available":96583090176,"internally_reserved":0,"allocated":3256320,"data_stored":1698970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.97699999999999998,"15min":0.999},"min":{"1min":0.56000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.502,"5min":1.6579999999999999,"15min":1.708},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0760000000000001,"15min":1.2},"min":{"1min":0.52900000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4790000000000001,"5min":1.7010000000000001,"15min":1.79},"last":1.365}]},{"osd":2,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.876,"15min":0.94999999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.095,"5min":1.641,"15min":1.641},"last":0.753},{"interface":"front","average":{"1min":1.026,"5min":1.087,"15min":1.1419999999999999},"min":{"1min":0.42599999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.607,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.96099999999999997}]},{"osd":3,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1519999999999999,"15min":1.2390000000000001},"min":{"1min":0.58599999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5960000000000001,"5min":2.496,"15min":2.496},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":1.099,"15min":1.143},"min":{"1min":0.49399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.5369999999999999,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":0.97999999999999998}]},{"osd":4,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.276,"15min":1.351},"min":{"1min":0.77700000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.579,"5min":2.73,"15min":2.8460000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.248,"15min":1.3009999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.5409999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.0209999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.302,"15min":1.3640000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.476,"15min":1.5189999999999999},"min":{"1min":0.89600000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7070000000000001,"5min":2.601,"15min":2.601},"last":1.3129999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4330000000000001,"15min":1.502},"min":{"1min":0.92600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.7609999999999999,"5min":2.883,"15min":2.9079999999999999},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.411,"15min":1.484},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.6180000000000001,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.575}]},{"osd":7,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4830000000000001,"15min":1.5389999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.7749999999999999,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.552},{"interface":"front","average":{"1min":1.339,"5min":1.484,"15min":1.5569999999999999},"min":{"1min":0.83799999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.873,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.621}]}]},{"osd":0,"up_from":13,"seq":55834576025,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61316,"kb_used_data":3768,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310524,"statfs":{"total":96636764160,"available":96573976576,"internally_reserved":0,"allocated":3858432,"data_stored":2292141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92000000000000004,"15min":0.96499999999999997},"min":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.321,"5min":1.581,"15min":1.581},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.016,"15min":1.0840000000000001},"min":{"1min":0.48899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.4039999999999999,"5min":1.621,"15min":1.726},"last":0.95699999999999996}]},{"osd":2,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.081,"15min":1.1379999999999999},"min":{"1min":0.52000000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.071},"min":{"1min":0.51300000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4690000000000001,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":1.147}]},{"osd":3,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1859999999999999,"15min":1.2450000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5600000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.141,"15min":1.165},"min":{"1min":0.55100000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":2.0979999999999999},"last":1.0960000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3080000000000001,"15min":1.3400000000000001},"min":{"1min":0.746,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6359999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.361,"15min":1.4079999999999999},"min":{"1min":0.79900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.181},"last":1.504}]},{"osd":5,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3939999999999999,"15min":1.4159999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.802,"5min":1.919,"15min":2.1309999999999998},"last":1.579},{"interface":"front","average":{"1min":1.466,"5min":1.47,"15min":1.49},"min":{"1min":0.81999999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.23,"5min":2.23,"15min":2.4900000000000002},"last":1.677}]},{"osd":6,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4359999999999999,"15min":1.4770000000000001},"min":{"1min":0.94999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0150000000000001,"5min":2.21,"15min":2.2149999999999999},"last":1.877},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.462,"15min":1.4990000000000001},"min":{"1min":0.78200000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7529999999999999,"5min":2.0880000000000001,"15min":2.355},"last":1.46}]},{"osd":7,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.5309999999999999,"15min":1.542},"min":{"1min":0.95999999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9370000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.758},{"interface":"front","average":{"1min":1.448,"5min":1.518,"15min":1.5529999999999999},"min":{"1min":0.94599999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.9690000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:15:03.066 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:15:03.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.293+0000 7f7493916640 1 Processor -- start 2023-10-08T07:15:03.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.293+0000 7f7493916640 1 -- start start 2023-10-08T07:15:03.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.295+0000 7f7493916640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f748c15ab40 0x7f748c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.295+0000 7f7493916640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f748c0cb580 0x7f748c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.295+0000 7f7493916640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c15b2b0 con 0x7f748c0c90b0 2023-10-08T07:15:03.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.295+0000 7f7493916640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c15bf30 con 0x7f748c0cb580 2023-10-08T07:15:03.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.295+0000 7f7493916640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c15f7a0 con 0x7f748c15ab40 2023-10-08T07:15:03.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.295+0000 7f7491eb3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f748c0cb580 0x7f748c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.295+0000 7f7491eb3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f748c0cb580 0x7f748c0adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58386/0 (socket says 172.21.15.70:58386) 2023-10-08T07:15:03.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.295+0000 7f7491eb3640 1 -- 172.21.15.70:0/2367695197 learned_addr learned my addr 172.21.15.70:0/2367695197 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:03.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.296+0000 7f74916b2640 1 --2- 172.21.15.70:0/2367695197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f748c15ab40 0x7f748c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.296+0000 7f7491eb3640 1 -- 172.21.15.70:0/2367695197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f748c15ab40 msgr2=0x7f748c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:03.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.296+0000 7f7491eb3640 1 --2- 172.21.15.70:0/2367695197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f748c15ab40 0x7f748c0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.296+0000 7f7491eb3640 1 -- 172.21.15.70:0/2367695197 >> v1:172.21.15.181:6789/0 conn(0x7f748c0c90b0 legacy=0x7f748c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:03.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.296+0000 7f7491eb3640 1 -- 172.21.15.70:0/2367695197 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c15b590 con 0x7f748c0cb580 2023-10-08T07:15:03.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.297+0000 7f7491eb3640 1 --2- 172.21.15.70:0/2367695197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 0x7f748c0adc60 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f7484009e50 tx=0x7f748402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e120d7bf61e8d345 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:03.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.297+0000 7f7483fff640 1 -- 172.21.15.70:0/2367695197 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748402ca70 con 0x7f748c0cb580 2023-10-08T07:15:03.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.297+0000 7f7483fff640 1 -- 172.21.15.70:0/2367695197 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748402cbd0 con 0x7f748c0cb580 2023-10-08T07:15:03.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.297+0000 7f7483fff640 1 -- 172.21.15.70:0/2367695197 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74840343f0 con 0x7f748c0cb580 2023-10-08T07:15:03.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.297+0000 7f7493916640 1 -- 172.21.15.70:0/2367695197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 msgr2=0x7f748c0adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:03.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.297+0000 7f7493916640 1 --2- 172.21.15.70:0/2367695197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 0x7f748c0adc60 secure :-1 s=READY pgs=2702 cs=0 l=1 rev1=1 crypto rx=0x7f7484009e50 tx=0x7f748402bed0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.298+0000 7f7493916640 1 -- 172.21.15.70:0/2367695197 shutdown_connections 2023-10-08T07:15:03.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.298+0000 7f7493916640 1 --2- 172.21.15.70:0/2367695197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 0x7f748c0adc60 unknown :-1 s=CLOSED pgs=2702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.298+0000 7f7493916640 1 --2- 172.21.15.70:0/2367695197 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f748c15ab40 0x7f748c0ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.298+0000 7f7493916640 1 -- 172.21.15.70:0/2367695197 >> 172.21.15.70:0/2367695197 conn(0x7f748c06a380 msgr2=0x7f748c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:03.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.298+0000 7f7493916640 1 -- 172.21.15.70:0/2367695197 shutdown_connections 2023-10-08T07:15:03.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.298+0000 7f7493916640 1 -- 172.21.15.70:0/2367695197 wait complete. 2023-10-08T07:15:03.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.299+0000 7f7493916640 1 Processor -- start 2023-10-08T07:15:03.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.299+0000 7f7493916640 1 -- start start 2023-10-08T07:15:03.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.300+0000 7f7493916640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f748c0c90b0 0x7f748c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.300+0000 7f7493916640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 0x7f748c15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.300+0000 7f7493916640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f748c15ab40 0x7f748c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.300+0000 7f7493916640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c15e5c0 con 0x7f748c0cb580 2023-10-08T07:15:03.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.300+0000 7f7493916640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c15e2e0 con 0x7f748c15ab40 2023-10-08T07:15:03.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.300+0000 7f7493916640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c0ae780 con 0x7f748c0c90b0 2023-10-08T07:15:03.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7490eb1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 0x7f748c15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7490eb1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 0x7f748c15dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58390/0 (socket says 172.21.15.70:58390) 2023-10-08T07:15:03.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7490eb1640 1 -- 172.21.15.70:0/4031168539 learned_addr learned my addr 172.21.15.70:0/4031168539 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:03.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f74916b2640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f748c0c90b0 0x7f748c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7491eb3640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f748c15ab40 0x7f748c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7490eb1640 1 -- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f748c0c90b0 msgr2=0x7f748c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:03.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7490eb1640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f748c0c90b0 0x7f748c15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7490eb1640 1 -- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f748c15ab40 msgr2=0x7f748c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:03.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7490eb1640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f748c15ab40 0x7f748c159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7490eb1640 1 -- 172.21.15.70:0/4031168539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484009b30 con 0x7f748c0cb580 2023-10-08T07:15:03.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7491eb3640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f748c15ab40 0x7f748c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:03.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.301+0000 7f7490eb1640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 0x7f748c15dc80 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f747c00b3f0 tx=0x7f747c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:03.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.302+0000 7f7481ffb640 1 -- 172.21.15.70:0/4031168539 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c011830 con 0x7f748c0cb580 2023-10-08T07:15:03.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.302+0000 7f7481ffb640 1 -- 172.21.15.70:0/4031168539 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c011990 con 0x7f748c0cb580 2023-10-08T07:15:03.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.302+0000 7f7481ffb640 1 -- 172.21.15.70:0/4031168539 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c011d40 con 0x7f748c0cb580 2023-10-08T07:15:03.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.302+0000 7f7493916640 1 -- 172.21.15.70:0/4031168539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c0aea60 con 0x7f748c0cb580 2023-10-08T07:15:03.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.302+0000 7f7493916640 1 -- 172.21.15.70:0/4031168539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c14e800 con 0x7f748c0cb580 2023-10-08T07:15:03.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.303+0000 7f7493916640 1 -- 172.21.15.70:0/4031168539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c067490 con 0x7f748c0cb580 2023-10-08T07:15:03.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.304+0000 7f7481ffb640 1 -- 172.21.15.70:0/4031168539 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f747c01a400 con 0x7f748c0cb580 2023-10-08T07:15:03.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.304+0000 7f7481ffb640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f747006b0b0 0x7f747006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.305+0000 7f7481ffb640 1 -- 172.21.15.70:0/4031168539 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f747c089e40 con 0x7f748c0cb580 2023-10-08T07:15:03.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.305+0000 7f74916b2640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f747006b0b0 0x7f747006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.306+0000 7f74916b2640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f747006b0b0 0x7f747006d570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f748800a970 tx=0x7f7488008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:03.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.309+0000 7f7481ffb640 1 -- 172.21.15.70:0/4031168539 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f747c0545d0 con 0x7f748c0cb580 2023-10-08T07:15:03.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.479+0000 7f7493916640 1 -- 172.21.15.70:0/4031168539 --> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f748c0ae8f0 con 0x7f747006b0b0 2023-10-08T07:15:03.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.518+0000 7f7481ffb640 1 -- 172.21.15.70:0/4031168539 <== mgr.8662 v2:172.21.15.181:6832/4245028292 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526378 (secure 0 0 0) 0x7f748c0ae8f0 con 0x7f747006b0b0 2023-10-08T07:15:03.523 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:03.530 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:15:03.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.528+0000 7f7493916640 1 -- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f747006b0b0 msgr2=0x7f747006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:03.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.528+0000 7f7493916640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f747006b0b0 0x7f747006d570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f748800a970 tx=0x7f7488008040 comp rx=0 tx=0).stop 2023-10-08T07:15:03.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.528+0000 7f7493916640 1 -- 172.21.15.70:0/4031168539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 msgr2=0x7f748c15dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:03.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.528+0000 7f7493916640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 0x7f748c15dc80 secure :-1 s=READY pgs=2703 cs=0 l=1 rev1=1 crypto rx=0x7f747c00b3f0 tx=0x7f747c00b8b0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.529+0000 7f7493916640 1 -- 172.21.15.70:0/4031168539 shutdown_connections 2023-10-08T07:15:03.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.529+0000 7f7493916640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f747006b0b0 0x7f747006d570 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.529+0000 7f7493916640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f748c15ab40 0x7f748c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.529+0000 7f7493916640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0cb580 0x7f748c15dc80 unknown :-1 s=CLOSED pgs=2703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.529+0000 7f7493916640 1 --2- 172.21.15.70:0/4031168539 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f748c0c90b0 0x7f748c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.529+0000 7f7493916640 1 -- 172.21.15.70:0/4031168539 >> 172.21.15.70:0/4031168539 conn(0x7f748c06a380 msgr2=0x7f748c0c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:03.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.529+0000 7f7493916640 1 -- 172.21.15.70:0/4031168539 shutdown_connections 2023-10-08T07:15:03.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.529+0000 7f7493916640 1 -- 172.21.15.70:0/4031168539 wait complete. 2023-10-08T07:15:03.553 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-10-08T07:15:02.872278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":720,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2907,"ondisk_log_size":2907,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410800,"kb_used_data":26664,"kb_used_omap":12,"kb_used_meta":383987,"kb_avail":754563920,"statfs":{"total":773094113280,"available":772673454080,"internally_reserved":0,"allocated":27303936,"data_stored":14783634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":393203259},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3397,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001930"},"pg_stats":[{"pgid":"34.1d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.570503+0000","last_change":"2023-10-08T07:14:58.550530+0000","last_active":"2023-10-08T07:14:59.570503+0000","last_peered":"2023-10-08T07:14:59.570503+0000","last_clean":"2023-10-08T07:14:59.570503+0000","last_became_active":"2023-10-08T07:14:58.550249+0000","last_became_peered":"2023-10-08T07:14:58.550249+0000","last_unstale":"2023-10-08T07:14:59.570503+0000","last_undegraded":"2023-10-08T07:14:59.570503+0000","last_fullsized":"2023-10-08T07:14:59.570503+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:02.313521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"257'1","reported_seq":13,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.566998+0000","last_change":"2023-10-08T07:14:57.517188+0000","last_active":"2023-10-08T07:14:58.566998+0000","last_peered":"2023-10-08T07:14:58.566998+0000","last_clean":"2023-10-08T07:14:58.566998+0000","last_became_active":"2023-10-08T07:14:57.516848+0000","last_became_peered":"2023-10-08T07:14:57.516848+0000","last_unstale":"2023-10-08T07:14:58.566998+0000","last_undegraded":"2023-10-08T07:14:58.566998+0000","last_fullsized":"2023-10-08T07:14:58.566998+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:47.146297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533488+0000","last_change":"2023-10-08T07:14:44.322566+0000","last_active":"2023-10-08T07:14:58.533488+0000","last_peered":"2023-10-08T07:14:58.533488+0000","last_clean":"2023-10-08T07:14:58.533488+0000","last_became_active":"2023-10-08T07:14:44.321968+0000","last_became_peered":"2023-10-08T07:14:44.321968+0000","last_unstale":"2023-10-08T07:14:58.533488+0000","last_undegraded":"2023-10-08T07:14:58.533488+0000","last_fullsized":"2023-10-08T07:14:58.533488+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:33.364440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546558+0000","last_change":"2023-10-08T07:14:58.552444+0000","last_active":"2023-10-08T07:14:59.546558+0000","last_peered":"2023-10-08T07:14:59.546558+0000","last_clean":"2023-10-08T07:14:59.546558+0000","last_became_active":"2023-10-08T07:14:58.552072+0000","last_became_peered":"2023-10-08T07:14:58.552072+0000","last_unstale":"2023-10-08T07:14:59.546558+0000","last_undegraded":"2023-10-08T07:14:59.546558+0000","last_fullsized":"2023-10-08T07:14:59.546558+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:01.516979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"257'4","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.570292+0000","last_change":"2023-10-08T07:14:57.523739+0000","last_active":"2023-10-08T07:14:58.570292+0000","last_peered":"2023-10-08T07:14:58.570292+0000","last_clean":"2023-10-08T07:14:58.570292+0000","last_became_active":"2023-10-08T07:14:57.523490+0000","last_became_peered":"2023-10-08T07:14:57.523490+0000","last_unstale":"2023-10-08T07:14:58.570292+0000","last_undegraded":"2023-10-08T07:14:58.570292+0000","last_fullsized":"2023-10-08T07:14:58.570292+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:35.857790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.535845+0000","last_change":"2023-10-08T07:14:44.336800+0000","last_active":"2023-10-08T07:14:58.535845+0000","last_peered":"2023-10-08T07:14:58.535845+0000","last_clean":"2023-10-08T07:14:58.535845+0000","last_became_active":"2023-10-08T07:14:44.336534+0000","last_became_peered":"2023-10-08T07:14:44.336534+0000","last_unstale":"2023-10-08T07:14:58.535845+0000","last_undegraded":"2023-10-08T07:14:58.535845+0000","last_fullsized":"2023-10-08T07:14:58.535845+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:48.002372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539715+0000","last_change":"2023-10-08T07:14:58.539613+0000","last_active":"2023-10-08T07:14:59.539715+0000","last_peered":"2023-10-08T07:14:59.539715+0000","last_clean":"2023-10-08T07:14:59.539715+0000","last_became_active":"2023-10-08T07:14:58.539279+0000","last_became_peered":"2023-10-08T07:14:58.539279+0000","last_unstale":"2023-10-08T07:14:59.539715+0000","last_undegraded":"2023-10-08T07:14:59.539715+0000","last_fullsized":"2023-10-08T07:14:59.539715+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:44.794204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.571271+0000","last_change":"2023-10-08T07:14:57.519020+0000","last_active":"2023-10-08T07:14:58.571271+0000","last_peered":"2023-10-08T07:14:58.571271+0000","last_clean":"2023-10-08T07:14:58.571271+0000","last_became_active":"2023-10-08T07:14:57.518593+0000","last_became_peered":"2023-10-08T07:14:57.518593+0000","last_unstale":"2023-10-08T07:14:58.571271+0000","last_undegraded":"2023-10-08T07:14:58.571271+0000","last_fullsized":"2023-10-08T07:14:58.571271+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:16.811954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540531+0000","last_change":"2023-10-08T07:14:44.331190+0000","last_active":"2023-10-08T07:14:58.540531+0000","last_peered":"2023-10-08T07:14:58.540531+0000","last_clean":"2023-10-08T07:14:58.540531+0000","last_became_active":"2023-10-08T07:14:44.330346+0000","last_became_peered":"2023-10-08T07:14:44.330346+0000","last_unstale":"2023-10-08T07:14:58.540531+0000","last_undegraded":"2023-10-08T07:14:58.540531+0000","last_fullsized":"2023-10-08T07:14:58.540531+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:54:20.560371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.570807+0000","last_change":"2023-10-08T07:14:58.542547+0000","last_active":"2023-10-08T07:14:59.570807+0000","last_peered":"2023-10-08T07:14:59.570807+0000","last_clean":"2023-10-08T07:14:59.570807+0000","last_became_active":"2023-10-08T07:14:58.542261+0000","last_became_peered":"2023-10-08T07:14:58.542261+0000","last_unstale":"2023-10-08T07:14:59.570807+0000","last_undegraded":"2023-10-08T07:14:59.570807+0000","last_fullsized":"2023-10-08T07:14:59.570807+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:31.265009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.571504+0000","last_change":"2023-10-08T07:14:57.516009+0000","last_active":"2023-10-08T07:14:58.571504+0000","last_peered":"2023-10-08T07:14:58.571504+0000","last_clean":"2023-10-08T07:14:58.571504+0000","last_became_active":"2023-10-08T07:14:57.515464+0000","last_became_peered":"2023-10-08T07:14:57.515464+0000","last_unstale":"2023-10-08T07:14:58.571504+0000","last_undegraded":"2023-10-08T07:14:58.571504+0000","last_fullsized":"2023-10-08T07:14:58.571504+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:04.514485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540123+0000","last_change":"2023-10-08T07:14:44.337611+0000","last_active":"2023-10-08T07:14:58.540123+0000","last_peered":"2023-10-08T07:14:58.540123+0000","last_clean":"2023-10-08T07:14:58.540123+0000","last_became_active":"2023-10-08T07:14:44.336777+0000","last_became_peered":"2023-10-08T07:14:44.336777+0000","last_unstale":"2023-10-08T07:14:58.540123+0000","last_undegraded":"2023-10-08T07:14:58.540123+0000","last_fullsized":"2023-10-08T07:14:58.540123+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:55.334487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.540116+0000","last_change":"2023-10-08T07:14:58.535708+0000","last_active":"2023-10-08T07:14:59.540116+0000","last_peered":"2023-10-08T07:14:59.540116+0000","last_clean":"2023-10-08T07:14:59.540116+0000","last_became_active":"2023-10-08T07:14:58.535226+0000","last_became_peered":"2023-10-08T07:14:58.535226+0000","last_unstale":"2023-10-08T07:14:59.540116+0000","last_undegraded":"2023-10-08T07:14:59.540116+0000","last_fullsized":"2023-10-08T07:14:59.540116+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:17.974456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"257'1","reported_seq":13,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.573354+0000","last_change":"2023-10-08T07:14:57.513838+0000","last_active":"2023-10-08T07:14:58.573354+0000","last_peered":"2023-10-08T07:14:58.573354+0000","last_clean":"2023-10-08T07:14:58.573354+0000","last_became_active":"2023-10-08T07:14:57.512370+0000","last_became_peered":"2023-10-08T07:14:57.512370+0000","last_unstale":"2023-10-08T07:14:58.573354+0000","last_undegraded":"2023-10-08T07:14:58.573354+0000","last_fullsized":"2023-10-08T07:14:58.573354+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:37:12.810885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.523336+0000","last_change":"2023-10-08T07:14:44.334750+0000","last_active":"2023-10-08T07:14:58.523336+0000","last_peered":"2023-10-08T07:14:58.523336+0000","last_clean":"2023-10-08T07:14:58.523336+0000","last_became_active":"2023-10-08T07:14:44.332421+0000","last_became_peered":"2023-10-08T07:14:44.332421+0000","last_unstale":"2023-10-08T07:14:58.523336+0000","last_undegraded":"2023-10-08T07:14:58.523336+0000","last_fullsized":"2023-10-08T07:14:58.523336+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:21:57.635068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539121+0000","last_change":"2023-10-08T07:14:58.548540+0000","last_active":"2023-10-08T07:14:59.539121+0000","last_peered":"2023-10-08T07:14:59.539121+0000","last_clean":"2023-10-08T07:14:59.539121+0000","last_became_active":"2023-10-08T07:14:58.548259+0000","last_became_peered":"2023-10-08T07:14:58.548259+0000","last_unstale":"2023-10-08T07:14:59.539121+0000","last_undegraded":"2023-10-08T07:14:59.539121+0000","last_fullsized":"2023-10-08T07:14:59.539121+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:18.623768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"257'1","reported_seq":13,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.569986+0000","last_change":"2023-10-08T07:14:57.521994+0000","last_active":"2023-10-08T07:14:58.569986+0000","last_peered":"2023-10-08T07:14:58.569986+0000","last_clean":"2023-10-08T07:14:58.569986+0000","last_became_active":"2023-10-08T07:14:57.521707+0000","last_became_peered":"2023-10-08T07:14:57.521707+0000","last_unstale":"2023-10-08T07:14:58.569986+0000","last_undegraded":"2023-10-08T07:14:58.569986+0000","last_fullsized":"2023-10-08T07:14:58.569986+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:47.149837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.534395+0000","last_change":"2023-10-08T07:14:44.331918+0000","last_active":"2023-10-08T07:14:58.534395+0000","last_peered":"2023-10-08T07:14:58.534395+0000","last_clean":"2023-10-08T07:14:58.534395+0000","last_became_active":"2023-10-08T07:14:44.331614+0000","last_became_peered":"2023-10-08T07:14:44.331614+0000","last_unstale":"2023-10-08T07:14:58.534395+0000","last_undegraded":"2023-10-08T07:14:58.534395+0000","last_fullsized":"2023-10-08T07:14:58.534395+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:24.698116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567289+0000","last_change":"2023-10-08T07:14:58.551216+0000","last_active":"2023-10-08T07:14:59.567289+0000","last_peered":"2023-10-08T07:14:59.567289+0000","last_clean":"2023-10-08T07:14:59.567289+0000","last_became_active":"2023-10-08T07:14:58.550213+0000","last_became_peered":"2023-10-08T07:14:58.550213+0000","last_unstale":"2023-10-08T07:14:59.567289+0000","last_undegraded":"2023-10-08T07:14:59.567289+0000","last_fullsized":"2023-10-08T07:14:59.567289+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:08:03.271281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540992+0000","last_change":"2023-10-08T07:14:57.520322+0000","last_active":"2023-10-08T07:14:58.540992+0000","last_peered":"2023-10-08T07:14:58.540992+0000","last_clean":"2023-10-08T07:14:58.540992+0000","last_became_active":"2023-10-08T07:14:57.520065+0000","last_became_peered":"2023-10-08T07:14:57.520065+0000","last_unstale":"2023-10-08T07:14:58.540992+0000","last_undegraded":"2023-10-08T07:14:58.540992+0000","last_fullsized":"2023-10-08T07:14:58.540992+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:35.922226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.534223+0000","last_change":"2023-10-08T07:14:44.335960+0000","last_active":"2023-10-08T07:14:58.534223+0000","last_peered":"2023-10-08T07:14:58.534223+0000","last_clean":"2023-10-08T07:14:58.534223+0000","last_became_active":"2023-10-08T07:14:44.335698+0000","last_became_peered":"2023-10-08T07:14:44.335698+0000","last_unstale":"2023-10-08T07:14:58.534223+0000","last_undegraded":"2023-10-08T07:14:58.534223+0000","last_fullsized":"2023-10-08T07:14:58.534223+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:21.961789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567756+0000","last_change":"2023-10-08T07:14:58.543894+0000","last_active":"2023-10-08T07:14:59.567756+0000","last_peered":"2023-10-08T07:14:59.567756+0000","last_clean":"2023-10-08T07:14:59.567756+0000","last_became_active":"2023-10-08T07:14:58.543547+0000","last_became_peered":"2023-10-08T07:14:58.543547+0000","last_unstale":"2023-10-08T07:14:59.567756+0000","last_undegraded":"2023-10-08T07:14:59.567756+0000","last_fullsized":"2023-10-08T07:14:59.567756+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:42.942350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.525255+0000","last_change":"2023-10-08T07:14:57.516240+0000","last_active":"2023-10-08T07:14:58.525255+0000","last_peered":"2023-10-08T07:14:58.525255+0000","last_clean":"2023-10-08T07:14:58.525255+0000","last_became_active":"2023-10-08T07:14:57.515965+0000","last_became_peered":"2023-10-08T07:14:57.515965+0000","last_unstale":"2023-10-08T07:14:58.525255+0000","last_undegraded":"2023-10-08T07:14:58.525255+0000","last_fullsized":"2023-10-08T07:14:58.525255+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:07.275083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.527002+0000","last_change":"2023-10-08T07:14:44.324288+0000","last_active":"2023-10-08T07:14:58.527002+0000","last_peered":"2023-10-08T07:14:58.527002+0000","last_clean":"2023-10-08T07:14:58.527002+0000","last_became_active":"2023-10-08T07:14:44.323888+0000","last_became_peered":"2023-10-08T07:14:44.323888+0000","last_unstale":"2023-10-08T07:14:58.527002+0000","last_undegraded":"2023-10-08T07:14:58.527002+0000","last_fullsized":"2023-10-08T07:14:58.527002+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:01.742854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.570681+0000","last_change":"2023-10-08T07:14:58.543191+0000","last_active":"2023-10-08T07:14:59.570681+0000","last_peered":"2023-10-08T07:14:59.570681+0000","last_clean":"2023-10-08T07:14:59.570681+0000","last_became_active":"2023-10-08T07:14:58.542923+0000","last_became_peered":"2023-10-08T07:14:58.542923+0000","last_unstale":"2023-10-08T07:14:59.570681+0000","last_undegraded":"2023-10-08T07:14:59.570681+0000","last_fullsized":"2023-10-08T07:14:59.570681+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:29:36.198033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529939+0000","last_change":"2023-10-08T07:14:57.515589+0000","last_active":"2023-10-08T07:14:58.529939+0000","last_peered":"2023-10-08T07:14:58.529939+0000","last_clean":"2023-10-08T07:14:58.529939+0000","last_became_active":"2023-10-08T07:14:57.515241+0000","last_became_peered":"2023-10-08T07:14:57.515241+0000","last_unstale":"2023-10-08T07:14:58.529939+0000","last_undegraded":"2023-10-08T07:14:58.529939+0000","last_fullsized":"2023-10-08T07:14:58.529939+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:58.130129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533689+0000","last_change":"2023-10-08T07:14:44.332226+0000","last_active":"2023-10-08T07:14:58.533689+0000","last_peered":"2023-10-08T07:14:58.533689+0000","last_clean":"2023-10-08T07:14:58.533689+0000","last_became_active":"2023-10-08T07:14:44.331872+0000","last_became_peered":"2023-10-08T07:14:44.331872+0000","last_unstale":"2023-10-08T07:14:58.533689+0000","last_undegraded":"2023-10-08T07:14:58.533689+0000","last_fullsized":"2023-10-08T07:14:58.533689+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:25.959585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567171+0000","last_change":"2023-10-08T07:14:58.547478+0000","last_active":"2023-10-08T07:14:59.567171+0000","last_peered":"2023-10-08T07:14:59.567171+0000","last_clean":"2023-10-08T07:14:59.567171+0000","last_became_active":"2023-10-08T07:14:58.547145+0000","last_became_peered":"2023-10-08T07:14:58.547145+0000","last_unstale":"2023-10-08T07:14:59.567171+0000","last_undegraded":"2023-10-08T07:14:59.567171+0000","last_fullsized":"2023-10-08T07:14:59.567171+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:23:10.300505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533179+0000","last_change":"2023-10-08T07:14:57.515231+0000","last_active":"2023-10-08T07:14:58.533179+0000","last_peered":"2023-10-08T07:14:58.533179+0000","last_clean":"2023-10-08T07:14:58.533179+0000","last_became_active":"2023-10-08T07:14:57.514976+0000","last_became_peered":"2023-10-08T07:14:57.514976+0000","last_unstale":"2023-10-08T07:14:58.533179+0000","last_undegraded":"2023-10-08T07:14:58.533179+0000","last_fullsized":"2023-10-08T07:14:58.533179+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:09:39.172242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541492+0000","last_change":"2023-10-08T07:14:44.331672+0000","last_active":"2023-10-08T07:14:58.541492+0000","last_peered":"2023-10-08T07:14:58.541492+0000","last_clean":"2023-10-08T07:14:58.541492+0000","last_became_active":"2023-10-08T07:14:44.331316+0000","last_became_peered":"2023-10-08T07:14:44.331316+0000","last_unstale":"2023-10-08T07:14:58.541492+0000","last_undegraded":"2023-10-08T07:14:58.541492+0000","last_fullsized":"2023-10-08T07:14:58.541492+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:57.946015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.547394+0000","last_change":"2023-10-08T07:14:58.560021+0000","last_active":"2023-10-08T07:14:59.547394+0000","last_peered":"2023-10-08T07:14:59.547394+0000","last_clean":"2023-10-08T07:14:59.547394+0000","last_became_active":"2023-10-08T07:14:58.559622+0000","last_became_peered":"2023-10-08T07:14:58.559622+0000","last_unstale":"2023-10-08T07:14:59.547394+0000","last_undegraded":"2023-10-08T07:14:59.547394+0000","last_fullsized":"2023-10-08T07:14:59.547394+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:48.163090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"257'6","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.575114+0000","last_change":"2023-10-08T07:14:57.517430+0000","last_active":"2023-10-08T07:14:58.575114+0000","last_peered":"2023-10-08T07:14:58.575114+0000","last_clean":"2023-10-08T07:14:58.575114+0000","last_became_active":"2023-10-08T07:14:57.517126+0000","last_became_peered":"2023-10-08T07:14:57.517126+0000","last_unstale":"2023-10-08T07:14:58.575114+0000","last_undegraded":"2023-10-08T07:14:58.575114+0000","last_fullsized":"2023-10-08T07:14:58.575114+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:12.728482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540017+0000","last_change":"2023-10-08T07:14:44.331371+0000","last_active":"2023-10-08T07:14:58.540017+0000","last_peered":"2023-10-08T07:14:58.540017+0000","last_clean":"2023-10-08T07:14:58.540017+0000","last_became_active":"2023-10-08T07:14:44.330911+0000","last_became_peered":"2023-10-08T07:14:44.330911+0000","last_unstale":"2023-10-08T07:14:58.540017+0000","last_undegraded":"2023-10-08T07:14:58.540017+0000","last_fullsized":"2023-10-08T07:14:58.540017+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:17:50.662785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.569711+0000","last_change":"2023-10-08T07:14:58.556691+0000","last_active":"2023-10-08T07:14:59.569711+0000","last_peered":"2023-10-08T07:14:59.569711+0000","last_clean":"2023-10-08T07:14:59.569711+0000","last_became_active":"2023-10-08T07:14:58.555987+0000","last_became_peered":"2023-10-08T07:14:58.555987+0000","last_unstale":"2023-10-08T07:14:59.569711+0000","last_undegraded":"2023-10-08T07:14:59.569711+0000","last_fullsized":"2023-10-08T07:14:59.569711+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:05.082827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"257'3","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.572697+0000","last_change":"2023-10-08T07:14:57.514304+0000","last_active":"2023-10-08T07:14:58.572697+0000","last_peered":"2023-10-08T07:14:58.572697+0000","last_clean":"2023-10-08T07:14:58.572697+0000","last_became_active":"2023-10-08T07:14:57.513991+0000","last_became_peered":"2023-10-08T07:14:57.513991+0000","last_unstale":"2023-10-08T07:14:58.572697+0000","last_undegraded":"2023-10-08T07:14:58.572697+0000","last_fullsized":"2023-10-08T07:14:58.572697+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:43.640211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.531825+0000","last_change":"2023-10-08T07:14:44.336483+0000","last_active":"2023-10-08T07:14:58.531825+0000","last_peered":"2023-10-08T07:14:58.531825+0000","last_clean":"2023-10-08T07:14:58.531825+0000","last_became_active":"2023-10-08T07:14:44.336207+0000","last_became_peered":"2023-10-08T07:14:44.336207+0000","last_unstale":"2023-10-08T07:14:58.531825+0000","last_undegraded":"2023-10-08T07:14:58.531825+0000","last_fullsized":"2023-10-08T07:14:58.531825+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:05.158536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.544190+0000","last_change":"2023-10-08T07:14:58.549070+0000","last_active":"2023-10-08T07:14:59.544190+0000","last_peered":"2023-10-08T07:14:59.544190+0000","last_clean":"2023-10-08T07:14:59.544190+0000","last_became_active":"2023-10-08T07:14:58.548768+0000","last_became_peered":"2023-10-08T07:14:58.548768+0000","last_unstale":"2023-10-08T07:14:59.544190+0000","last_undegraded":"2023-10-08T07:14:59.544190+0000","last_fullsized":"2023-10-08T07:14:59.544190+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:56.384134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.534181+0000","last_change":"2023-10-08T07:14:57.519067+0000","last_active":"2023-10-08T07:14:58.534181+0000","last_peered":"2023-10-08T07:14:58.534181+0000","last_clean":"2023-10-08T07:14:58.534181+0000","last_became_active":"2023-10-08T07:14:57.518808+0000","last_became_peered":"2023-10-08T07:14:57.518808+0000","last_unstale":"2023-10-08T07:14:58.534181+0000","last_undegraded":"2023-10-08T07:14:58.534181+0000","last_fullsized":"2023-10-08T07:14:58.534181+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:09.277589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529188+0000","last_change":"2023-10-08T07:14:44.334689+0000","last_active":"2023-10-08T07:14:58.529188+0000","last_peered":"2023-10-08T07:14:58.529188+0000","last_clean":"2023-10-08T07:14:58.529188+0000","last_became_active":"2023-10-08T07:14:44.334401+0000","last_became_peered":"2023-10-08T07:14:44.334401+0000","last_unstale":"2023-10-08T07:14:58.529188+0000","last_undegraded":"2023-10-08T07:14:58.529188+0000","last_fullsized":"2023-10-08T07:14:58.529188+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:16:59.473108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.565491+0000","last_change":"2023-10-08T07:14:58.548619+0000","last_active":"2023-10-08T07:14:59.565491+0000","last_peered":"2023-10-08T07:14:59.565491+0000","last_clean":"2023-10-08T07:14:59.565491+0000","last_became_active":"2023-10-08T07:14:58.548333+0000","last_became_peered":"2023-10-08T07:14:58.548333+0000","last_unstale":"2023-10-08T07:14:59.565491+0000","last_undegraded":"2023-10-08T07:14:59.565491+0000","last_fullsized":"2023-10-08T07:14:59.565491+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:51.597388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.571058+0000","last_change":"2023-10-08T07:14:57.520740+0000","last_active":"2023-10-08T07:14:58.571058+0000","last_peered":"2023-10-08T07:14:58.571058+0000","last_clean":"2023-10-08T07:14:58.571058+0000","last_became_active":"2023-10-08T07:14:57.520485+0000","last_became_peered":"2023-10-08T07:14:57.520485+0000","last_unstale":"2023-10-08T07:14:58.571058+0000","last_undegraded":"2023-10-08T07:14:58.571058+0000","last_fullsized":"2023-10-08T07:14:58.571058+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:43.777411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.515817+0000","last_change":"2023-10-08T07:14:44.338751+0000","last_active":"2023-10-08T07:14:58.515817+0000","last_peered":"2023-10-08T07:14:58.515817+0000","last_clean":"2023-10-08T07:14:58.515817+0000","last_became_active":"2023-10-08T07:14:44.338461+0000","last_became_peered":"2023-10-08T07:14:44.338461+0000","last_unstale":"2023-10-08T07:14:58.515817+0000","last_undegraded":"2023-10-08T07:14:58.515817+0000","last_fullsized":"2023-10-08T07:14:58.515817+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:08.802530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567147+0000","last_change":"2023-10-08T07:14:58.555554+0000","last_active":"2023-10-08T07:14:59.567147+0000","last_peered":"2023-10-08T07:14:59.567147+0000","last_clean":"2023-10-08T07:14:59.567147+0000","last_became_active":"2023-10-08T07:14:58.555025+0000","last_became_peered":"2023-10-08T07:14:58.555025+0000","last_unstale":"2023-10-08T07:14:59.567147+0000","last_undegraded":"2023-10-08T07:14:59.567147+0000","last_fullsized":"2023-10-08T07:14:59.567147+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:15.231833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"257'8","reported_seq":20,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.574181+0000","last_change":"2023-10-08T07:14:57.521836+0000","last_active":"2023-10-08T07:14:58.574181+0000","last_peered":"2023-10-08T07:14:58.574181+0000","last_clean":"2023-10-08T07:14:58.574181+0000","last_became_active":"2023-10-08T07:14:57.521568+0000","last_became_peered":"2023-10-08T07:14:57.521568+0000","last_unstale":"2023-10-08T07:14:58.574181+0000","last_undegraded":"2023-10-08T07:14:58.574181+0000","last_fullsized":"2023-10-08T07:14:58.574181+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:18:32.836216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540591+0000","last_change":"2023-10-08T07:14:44.327546+0000","last_active":"2023-10-08T07:14:58.540591+0000","last_peered":"2023-10-08T07:14:58.540591+0000","last_clean":"2023-10-08T07:14:58.540591+0000","last_became_active":"2023-10-08T07:14:44.327267+0000","last_became_peered":"2023-10-08T07:14:44.327267+0000","last_unstale":"2023-10-08T07:14:58.540591+0000","last_undegraded":"2023-10-08T07:14:58.540591+0000","last_fullsized":"2023-10-08T07:14:58.540591+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:15:38.112936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.565500+0000","last_change":"2023-10-08T07:14:58.559779+0000","last_active":"2023-10-08T07:14:59.565500+0000","last_peered":"2023-10-08T07:14:59.565500+0000","last_clean":"2023-10-08T07:14:59.565500+0000","last_became_active":"2023-10-08T07:14:58.559364+0000","last_became_peered":"2023-10-08T07:14:58.559364+0000","last_unstale":"2023-10-08T07:14:59.565500+0000","last_undegraded":"2023-10-08T07:14:59.565500+0000","last_fullsized":"2023-10-08T07:14:59.565500+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:03:55.955439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530981+0000","last_change":"2023-10-08T07:14:57.515376+0000","last_active":"2023-10-08T07:14:58.530981+0000","last_peered":"2023-10-08T07:14:58.530981+0000","last_clean":"2023-10-08T07:14:58.530981+0000","last_became_active":"2023-10-08T07:14:57.515112+0000","last_became_peered":"2023-10-08T07:14:57.515112+0000","last_unstale":"2023-10-08T07:14:58.530981+0000","last_undegraded":"2023-10-08T07:14:58.530981+0000","last_fullsized":"2023-10-08T07:14:58.530981+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:35:04.090038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541081+0000","last_change":"2023-10-08T07:14:44.340846+0000","last_active":"2023-10-08T07:14:58.541081+0000","last_peered":"2023-10-08T07:14:58.541081+0000","last_clean":"2023-10-08T07:14:58.541081+0000","last_became_active":"2023-10-08T07:14:44.340588+0000","last_became_peered":"2023-10-08T07:14:44.340588+0000","last_unstale":"2023-10-08T07:14:58.541081+0000","last_undegraded":"2023-10-08T07:14:58.541081+0000","last_fullsized":"2023-10-08T07:14:58.541081+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:28:06.996015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546659+0000","last_change":"2023-10-08T07:14:58.537900+0000","last_active":"2023-10-08T07:14:59.546659+0000","last_peered":"2023-10-08T07:14:59.546659+0000","last_clean":"2023-10-08T07:14:59.546659+0000","last_became_active":"2023-10-08T07:14:58.537589+0000","last_became_peered":"2023-10-08T07:14:58.537589+0000","last_unstale":"2023-10-08T07:14:59.546659+0000","last_undegraded":"2023-10-08T07:14:59.546659+0000","last_fullsized":"2023-10-08T07:14:59.546659+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:25:34.656632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.568002+0000","last_change":"2023-10-08T07:14:57.522708+0000","last_active":"2023-10-08T07:14:58.568002+0000","last_peered":"2023-10-08T07:14:58.568002+0000","last_clean":"2023-10-08T07:14:58.568002+0000","last_became_active":"2023-10-08T07:14:57.522143+0000","last_became_peered":"2023-10-08T07:14:57.522143+0000","last_unstale":"2023-10-08T07:14:58.568002+0000","last_undegraded":"2023-10-08T07:14:58.568002+0000","last_fullsized":"2023-10-08T07:14:58.568002+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:03.961114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530800+0000","last_change":"2023-10-08T07:14:44.339646+0000","last_active":"2023-10-08T07:14:58.530800+0000","last_peered":"2023-10-08T07:14:58.530800+0000","last_clean":"2023-10-08T07:14:58.530800+0000","last_became_active":"2023-10-08T07:14:44.339229+0000","last_became_peered":"2023-10-08T07:14:44.339229+0000","last_unstale":"2023-10-08T07:14:58.530800+0000","last_undegraded":"2023-10-08T07:14:58.530800+0000","last_fullsized":"2023-10-08T07:14:58.530800+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:07.851859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.545964+0000","last_change":"2023-10-08T07:14:58.555192+0000","last_active":"2023-10-08T07:14:59.545964+0000","last_peered":"2023-10-08T07:14:59.545964+0000","last_clean":"2023-10-08T07:14:59.545964+0000","last_became_active":"2023-10-08T07:14:58.554829+0000","last_became_peered":"2023-10-08T07:14:58.554829+0000","last_unstale":"2023-10-08T07:14:59.545964+0000","last_undegraded":"2023-10-08T07:14:59.545964+0000","last_fullsized":"2023-10-08T07:14:59.545964+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:12:01.717208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"257'1","reported_seq":13,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.568771+0000","last_change":"2023-10-08T07:14:57.526013+0000","last_active":"2023-10-08T07:14:58.568771+0000","last_peered":"2023-10-08T07:14:58.568771+0000","last_clean":"2023-10-08T07:14:58.568771+0000","last_became_active":"2023-10-08T07:14:57.525781+0000","last_became_peered":"2023-10-08T07:14:57.525781+0000","last_unstale":"2023-10-08T07:14:58.568771+0000","last_undegraded":"2023-10-08T07:14:58.568771+0000","last_fullsized":"2023-10-08T07:14:58.568771+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:48:43.985077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541609+0000","last_change":"2023-10-08T07:14:44.323106+0000","last_active":"2023-10-08T07:14:58.541609+0000","last_peered":"2023-10-08T07:14:58.541609+0000","last_clean":"2023-10-08T07:14:58.541609+0000","last_became_active":"2023-10-08T07:14:44.322570+0000","last_became_peered":"2023-10-08T07:14:44.322570+0000","last_unstale":"2023-10-08T07:14:58.541609+0000","last_undegraded":"2023-10-08T07:14:58.541609+0000","last_fullsized":"2023-10-08T07:14:58.541609+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:07:44.570917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.547615+0000","last_change":"2023-10-08T07:14:58.537503+0000","last_active":"2023-10-08T07:14:59.547615+0000","last_peered":"2023-10-08T07:14:59.547615+0000","last_clean":"2023-10-08T07:14:59.547615+0000","last_became_active":"2023-10-08T07:14:58.537167+0000","last_became_peered":"2023-10-08T07:14:58.537167+0000","last_unstale":"2023-10-08T07:14:59.547615+0000","last_undegraded":"2023-10-08T07:14:59.547615+0000","last_fullsized":"2023-10-08T07:14:59.547615+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:59.146921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533063+0000","last_change":"2023-10-08T07:14:57.521064+0000","last_active":"2023-10-08T07:14:58.533063+0000","last_peered":"2023-10-08T07:14:58.533063+0000","last_clean":"2023-10-08T07:14:58.533063+0000","last_became_active":"2023-10-08T07:14:57.520780+0000","last_became_peered":"2023-10-08T07:14:57.520780+0000","last_unstale":"2023-10-08T07:14:58.533063+0000","last_undegraded":"2023-10-08T07:14:58.533063+0000","last_fullsized":"2023-10-08T07:14:58.533063+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:26:03.459719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.545240+0000","last_change":"2023-10-08T07:14:44.327597+0000","last_active":"2023-10-08T07:14:58.545240+0000","last_peered":"2023-10-08T07:14:58.545240+0000","last_clean":"2023-10-08T07:14:58.545240+0000","last_became_active":"2023-10-08T07:14:44.327247+0000","last_became_peered":"2023-10-08T07:14:44.327247+0000","last_unstale":"2023-10-08T07:14:58.545240+0000","last_undegraded":"2023-10-08T07:14:58.545240+0000","last_fullsized":"2023-10-08T07:14:58.545240+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:06:19.245988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567995+0000","last_change":"2023-10-08T07:14:58.552725+0000","last_active":"2023-10-08T07:14:59.567995+0000","last_peered":"2023-10-08T07:14:59.567995+0000","last_clean":"2023-10-08T07:14:59.567995+0000","last_became_active":"2023-10-08T07:14:58.551923+0000","last_became_peered":"2023-10-08T07:14:58.551923+0000","last_unstale":"2023-10-08T07:14:59.567995+0000","last_undegraded":"2023-10-08T07:14:59.567995+0000","last_fullsized":"2023-10-08T07:14:59.567995+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:39.120526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"257'1","reported_seq":13,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.568659+0000","last_change":"2023-10-08T07:14:57.520312+0000","last_active":"2023-10-08T07:14:58.568659+0000","last_peered":"2023-10-08T07:14:58.568659+0000","last_clean":"2023-10-08T07:14:58.568659+0000","last_became_active":"2023-10-08T07:14:57.519938+0000","last_became_peered":"2023-10-08T07:14:57.519938+0000","last_unstale":"2023-10-08T07:14:58.568659+0000","last_undegraded":"2023-10-08T07:14:58.568659+0000","last_fullsized":"2023-10-08T07:14:58.568659+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:04:15.152056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533372+0000","last_change":"2023-10-08T07:14:44.338168+0000","last_active":"2023-10-08T07:14:58.533372+0000","last_peered":"2023-10-08T07:14:58.533372+0000","last_clean":"2023-10-08T07:14:58.533372+0000","last_became_active":"2023-10-08T07:14:44.337905+0000","last_became_peered":"2023-10-08T07:14:44.337905+0000","last_unstale":"2023-10-08T07:14:58.533372+0000","last_undegraded":"2023-10-08T07:14:58.533372+0000","last_fullsized":"2023-10-08T07:14:58.533372+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:04:46.561296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.544556+0000","last_change":"2023-10-08T07:14:58.549048+0000","last_active":"2023-10-08T07:14:59.544556+0000","last_peered":"2023-10-08T07:14:59.544556+0000","last_clean":"2023-10-08T07:14:59.544556+0000","last_became_active":"2023-10-08T07:14:58.548751+0000","last_became_peered":"2023-10-08T07:14:58.548751+0000","last_unstale":"2023-10-08T07:14:59.544556+0000","last_undegraded":"2023-10-08T07:14:59.544556+0000","last_fullsized":"2023-10-08T07:14:59.544556+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:38.988326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.544492+0000","last_change":"2023-10-08T07:14:57.517075+0000","last_active":"2023-10-08T07:14:58.544492+0000","last_peered":"2023-10-08T07:14:58.544492+0000","last_clean":"2023-10-08T07:14:58.544492+0000","last_became_active":"2023-10-08T07:14:57.516810+0000","last_became_peered":"2023-10-08T07:14:57.516810+0000","last_unstale":"2023-10-08T07:14:58.544492+0000","last_undegraded":"2023-10-08T07:14:58.544492+0000","last_fullsized":"2023-10-08T07:14:58.544492+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:41:36.491196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532956+0000","last_change":"2023-10-08T07:14:44.329361+0000","last_active":"2023-10-08T07:14:58.532956+0000","last_peered":"2023-10-08T07:14:58.532956+0000","last_clean":"2023-10-08T07:14:58.532956+0000","last_became_active":"2023-10-08T07:14:44.328994+0000","last_became_peered":"2023-10-08T07:14:44.328994+0000","last_unstale":"2023-10-08T07:14:58.532956+0000","last_undegraded":"2023-10-08T07:14:58.532956+0000","last_fullsized":"2023-10-08T07:14:58.532956+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:22.824813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539320+0000","last_change":"2023-10-08T07:14:58.551857+0000","last_active":"2023-10-08T07:14:59.539320+0000","last_peered":"2023-10-08T07:14:59.539320+0000","last_clean":"2023-10-08T07:14:59.539320+0000","last_became_active":"2023-10-08T07:14:58.551530+0000","last_became_peered":"2023-10-08T07:14:58.551530+0000","last_unstale":"2023-10-08T07:14:59.539320+0000","last_undegraded":"2023-10-08T07:14:59.539320+0000","last_fullsized":"2023-10-08T07:14:59.539320+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:41:37.507988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.571401+0000","last_change":"2023-10-08T07:14:57.517807+0000","last_active":"2023-10-08T07:14:58.571401+0000","last_peered":"2023-10-08T07:14:58.571401+0000","last_clean":"2023-10-08T07:14:58.571401+0000","last_became_active":"2023-10-08T07:14:57.517555+0000","last_became_peered":"2023-10-08T07:14:57.517555+0000","last_unstale":"2023-10-08T07:14:58.571401+0000","last_undegraded":"2023-10-08T07:14:58.571401+0000","last_fullsized":"2023-10-08T07:14:58.571401+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:16.628423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.524675+0000","last_change":"2023-10-08T07:14:44.334282+0000","last_active":"2023-10-08T07:14:58.524675+0000","last_peered":"2023-10-08T07:14:58.524675+0000","last_clean":"2023-10-08T07:14:58.524675+0000","last_became_active":"2023-10-08T07:14:44.332171+0000","last_became_peered":"2023-10-08T07:14:44.332171+0000","last_unstale":"2023-10-08T07:14:58.524675+0000","last_undegraded":"2023-10-08T07:14:58.524675+0000","last_fullsized":"2023-10-08T07:14:58.524675+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:53.430647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539608+0000","last_change":"2023-10-08T07:14:58.541794+0000","last_active":"2023-10-08T07:14:59.539608+0000","last_peered":"2023-10-08T07:14:59.539608+0000","last_clean":"2023-10-08T07:14:59.539608+0000","last_became_active":"2023-10-08T07:14:58.541521+0000","last_became_peered":"2023-10-08T07:14:58.541521+0000","last_unstale":"2023-10-08T07:14:59.539608+0000","last_undegraded":"2023-10-08T07:14:59.539608+0000","last_fullsized":"2023-10-08T07:14:59.539608+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:40.393836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.528691+0000","last_change":"2023-10-08T07:14:57.518527+0000","last_active":"2023-10-08T07:14:58.528691+0000","last_peered":"2023-10-08T07:14:58.528691+0000","last_clean":"2023-10-08T07:14:58.528691+0000","last_became_active":"2023-10-08T07:14:57.518234+0000","last_became_peered":"2023-10-08T07:14:57.518234+0000","last_unstale":"2023-10-08T07:14:58.528691+0000","last_undegraded":"2023-10-08T07:14:58.528691+0000","last_fullsized":"2023-10-08T07:14:58.528691+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:47:43.350763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540682+0000","last_change":"2023-10-08T07:14:44.327517+0000","last_active":"2023-10-08T07:14:58.540682+0000","last_peered":"2023-10-08T07:14:58.540682+0000","last_clean":"2023-10-08T07:14:58.540682+0000","last_became_active":"2023-10-08T07:14:44.327215+0000","last_became_peered":"2023-10-08T07:14:44.327215+0000","last_unstale":"2023-10-08T07:14:58.540682+0000","last_undegraded":"2023-10-08T07:14:58.540682+0000","last_fullsized":"2023-10-08T07:14:58.540682+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:30.952438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.565754+0000","last_change":"2023-10-08T07:14:58.548407+0000","last_active":"2023-10-08T07:14:59.565754+0000","last_peered":"2023-10-08T07:14:59.565754+0000","last_clean":"2023-10-08T07:14:59.565754+0000","last_became_active":"2023-10-08T07:14:58.547784+0000","last_became_peered":"2023-10-08T07:14:58.547784+0000","last_unstale":"2023-10-08T07:14:59.565754+0000","last_undegraded":"2023-10-08T07:14:59.565754+0000","last_fullsized":"2023-10-08T07:14:59.565754+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:00.297231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.518054+0000","last_change":"2023-10-08T07:14:57.515219+0000","last_active":"2023-10-08T07:14:58.518054+0000","last_peered":"2023-10-08T07:14:58.518054+0000","last_clean":"2023-10-08T07:14:58.518054+0000","last_became_active":"2023-10-08T07:14:57.514941+0000","last_became_peered":"2023-10-08T07:14:57.514941+0000","last_unstale":"2023-10-08T07:14:58.518054+0000","last_undegraded":"2023-10-08T07:14:58.518054+0000","last_fullsized":"2023-10-08T07:14:58.518054+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:38.792601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.522478+0000","last_change":"2023-10-08T07:14:44.328985+0000","last_active":"2023-10-08T07:14:58.522478+0000","last_peered":"2023-10-08T07:14:58.522478+0000","last_clean":"2023-10-08T07:14:58.522478+0000","last_became_active":"2023-10-08T07:14:44.328697+0000","last_became_peered":"2023-10-08T07:14:44.328697+0000","last_unstale":"2023-10-08T07:14:58.522478+0000","last_undegraded":"2023-10-08T07:14:58.522478+0000","last_fullsized":"2023-10-08T07:14:58.522478+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:48.694664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.568351+0000","last_change":"2023-10-08T07:14:58.553564+0000","last_active":"2023-10-08T07:14:59.568351+0000","last_peered":"2023-10-08T07:14:59.568351+0000","last_clean":"2023-10-08T07:14:59.568351+0000","last_became_active":"2023-10-08T07:14:58.553270+0000","last_became_peered":"2023-10-08T07:14:58.553270+0000","last_unstale":"2023-10-08T07:14:59.568351+0000","last_undegraded":"2023-10-08T07:14:59.568351+0000","last_fullsized":"2023-10-08T07:14:59.568351+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:48:30.387387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.535657+0000","last_change":"2023-10-08T07:14:57.521746+0000","last_active":"2023-10-08T07:14:58.535657+0000","last_peered":"2023-10-08T07:14:58.535657+0000","last_clean":"2023-10-08T07:14:58.535657+0000","last_became_active":"2023-10-08T07:14:57.521489+0000","last_became_peered":"2023-10-08T07:14:57.521489+0000","last_unstale":"2023-10-08T07:14:58.535657+0000","last_undegraded":"2023-10-08T07:14:58.535657+0000","last_fullsized":"2023-10-08T07:14:58.535657+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:43:27.048490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529885+0000","last_change":"2023-10-08T07:14:44.336387+0000","last_active":"2023-10-08T07:14:58.529885+0000","last_peered":"2023-10-08T07:14:58.529885+0000","last_clean":"2023-10-08T07:14:58.529885+0000","last_became_active":"2023-10-08T07:14:44.336109+0000","last_became_peered":"2023-10-08T07:14:44.336109+0000","last_unstale":"2023-10-08T07:14:58.529885+0000","last_undegraded":"2023-10-08T07:14:58.529885+0000","last_fullsized":"2023-10-08T07:14:58.529885+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:39.506846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.569051+0000","last_change":"2023-10-08T07:14:58.562531+0000","last_active":"2023-10-08T07:14:59.569051+0000","last_peered":"2023-10-08T07:14:59.569051+0000","last_clean":"2023-10-08T07:14:59.569051+0000","last_became_active":"2023-10-08T07:14:58.562176+0000","last_became_peered":"2023-10-08T07:14:58.562176+0000","last_unstale":"2023-10-08T07:14:59.569051+0000","last_undegraded":"2023-10-08T07:14:59.569051+0000","last_fullsized":"2023-10-08T07:14:59.569051+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:15:48.143425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"257'2","reported_seq":14,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.571204+0000","last_change":"2023-10-08T07:14:57.520776+0000","last_active":"2023-10-08T07:14:58.571204+0000","last_peered":"2023-10-08T07:14:58.571204+0000","last_clean":"2023-10-08T07:14:58.571204+0000","last_became_active":"2023-10-08T07:14:57.520526+0000","last_became_peered":"2023-10-08T07:14:57.520526+0000","last_unstale":"2023-10-08T07:14:58.571204+0000","last_undegraded":"2023-10-08T07:14:58.571204+0000","last_fullsized":"2023-10-08T07:14:58.571204+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:49:19.970642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.535810+0000","last_change":"2023-10-08T07:14:44.338566+0000","last_active":"2023-10-08T07:14:58.535810+0000","last_peered":"2023-10-08T07:14:58.535810+0000","last_clean":"2023-10-08T07:14:58.535810+0000","last_became_active":"2023-10-08T07:14:44.338309+0000","last_became_peered":"2023-10-08T07:14:44.338309+0000","last_unstale":"2023-10-08T07:14:58.535810+0000","last_undegraded":"2023-10-08T07:14:58.535810+0000","last_fullsized":"2023-10-08T07:14:58.535810+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:51.724420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.570166+0000","last_change":"2023-10-08T07:14:58.561367+0000","last_active":"2023-10-08T07:14:59.570166+0000","last_peered":"2023-10-08T07:14:59.570166+0000","last_clean":"2023-10-08T07:14:59.570166+0000","last_became_active":"2023-10-08T07:14:58.561074+0000","last_became_peered":"2023-10-08T07:14:58.561074+0000","last_unstale":"2023-10-08T07:14:59.570166+0000","last_undegraded":"2023-10-08T07:14:59.570166+0000","last_fullsized":"2023-10-08T07:14:59.570166+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:18.215030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533339+0000","last_change":"2023-10-08T07:14:57.521949+0000","last_active":"2023-10-08T07:14:58.533339+0000","last_peered":"2023-10-08T07:14:58.533339+0000","last_clean":"2023-10-08T07:14:58.533339+0000","last_became_active":"2023-10-08T07:14:57.521678+0000","last_became_peered":"2023-10-08T07:14:57.521678+0000","last_unstale":"2023-10-08T07:14:58.533339+0000","last_undegraded":"2023-10-08T07:14:58.533339+0000","last_fullsized":"2023-10-08T07:14:58.533339+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:29.465386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.522725+0000","last_change":"2023-10-08T07:14:44.335206+0000","last_active":"2023-10-08T07:14:58.522725+0000","last_peered":"2023-10-08T07:14:58.522725+0000","last_clean":"2023-10-08T07:14:58.522725+0000","last_became_active":"2023-10-08T07:14:44.332905+0000","last_became_peered":"2023-10-08T07:14:44.332905+0000","last_unstale":"2023-10-08T07:14:58.522725+0000","last_undegraded":"2023-10-08T07:14:58.522725+0000","last_fullsized":"2023-10-08T07:14:58.522725+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:57.888130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.569268+0000","last_change":"2023-10-08T07:14:58.552437+0000","last_active":"2023-10-08T07:14:59.569268+0000","last_peered":"2023-10-08T07:14:59.569268+0000","last_clean":"2023-10-08T07:14:59.569268+0000","last_became_active":"2023-10-08T07:14:58.552036+0000","last_became_peered":"2023-10-08T07:14:58.552036+0000","last_unstale":"2023-10-08T07:14:59.569268+0000","last_undegraded":"2023-10-08T07:14:59.569268+0000","last_fullsized":"2023-10-08T07:14:59.569268+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:52:15.693811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532854+0000","last_change":"2023-10-08T07:14:57.516891+0000","last_active":"2023-10-08T07:14:58.532854+0000","last_peered":"2023-10-08T07:14:58.532854+0000","last_clean":"2023-10-08T07:14:58.532854+0000","last_became_active":"2023-10-08T07:14:57.516625+0000","last_became_peered":"2023-10-08T07:14:57.516625+0000","last_unstale":"2023-10-08T07:14:58.532854+0000","last_undegraded":"2023-10-08T07:14:58.532854+0000","last_fullsized":"2023-10-08T07:14:58.532854+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:50.001002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.531904+0000","last_change":"2023-10-08T07:14:44.332109+0000","last_active":"2023-10-08T07:14:58.531904+0000","last_peered":"2023-10-08T07:14:58.531904+0000","last_clean":"2023-10-08T07:14:58.531904+0000","last_became_active":"2023-10-08T07:14:44.331835+0000","last_became_peered":"2023-10-08T07:14:44.331835+0000","last_unstale":"2023-10-08T07:14:58.531904+0000","last_undegraded":"2023-10-08T07:14:58.531904+0000","last_fullsized":"2023-10-08T07:14:58.531904+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:48:15.146404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.540368+0000","last_change":"2023-10-08T07:14:58.544540+0000","last_active":"2023-10-08T07:14:59.540368+0000","last_peered":"2023-10-08T07:14:59.540368+0000","last_clean":"2023-10-08T07:14:59.540368+0000","last_became_active":"2023-10-08T07:14:58.543361+0000","last_became_peered":"2023-10-08T07:14:58.543361+0000","last_unstale":"2023-10-08T07:14:59.540368+0000","last_undegraded":"2023-10-08T07:14:59.540368+0000","last_fullsized":"2023-10-08T07:14:59.540368+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:15.231833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532621+0000","last_change":"2023-10-08T07:14:57.517351+0000","last_active":"2023-10-08T07:14:58.532621+0000","last_peered":"2023-10-08T07:14:58.532621+0000","last_clean":"2023-10-08T07:14:58.532621+0000","last_became_active":"2023-10-08T07:14:57.517041+0000","last_became_peered":"2023-10-08T07:14:57.517041+0000","last_unstale":"2023-10-08T07:14:58.532621+0000","last_undegraded":"2023-10-08T07:14:58.532621+0000","last_fullsized":"2023-10-08T07:14:58.532621+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:17:12.570521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.516393+0000","last_change":"2023-10-08T07:14:44.339948+0000","last_active":"2023-10-08T07:14:58.516393+0000","last_peered":"2023-10-08T07:14:58.516393+0000","last_clean":"2023-10-08T07:14:58.516393+0000","last_became_active":"2023-10-08T07:14:44.339645+0000","last_became_peered":"2023-10-08T07:14:44.339645+0000","last_unstale":"2023-10-08T07:14:58.516393+0000","last_undegraded":"2023-10-08T07:14:58.516393+0000","last_fullsized":"2023-10-08T07:14:58.516393+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:42.921595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.564627+0000","last_change":"2023-10-08T07:14:58.548523+0000","last_active":"2023-10-08T07:14:59.564627+0000","last_peered":"2023-10-08T07:14:59.564627+0000","last_clean":"2023-10-08T07:14:59.564627+0000","last_became_active":"2023-10-08T07:14:58.548180+0000","last_became_peered":"2023-10-08T07:14:58.548180+0000","last_unstale":"2023-10-08T07:14:59.564627+0000","last_undegraded":"2023-10-08T07:14:59.564627+0000","last_fullsized":"2023-10-08T07:14:59.564627+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:32.655231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529982+0000","last_change":"2023-10-08T07:14:57.521106+0000","last_active":"2023-10-08T07:14:58.529982+0000","last_peered":"2023-10-08T07:14:58.529982+0000","last_clean":"2023-10-08T07:14:58.529982+0000","last_became_active":"2023-10-08T07:14:57.520832+0000","last_became_peered":"2023-10-08T07:14:57.520832+0000","last_unstale":"2023-10-08T07:14:58.529982+0000","last_undegraded":"2023-10-08T07:14:58.529982+0000","last_fullsized":"2023-10-08T07:14:58.529982+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:06.528060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.516063+0000","last_change":"2023-10-08T07:14:44.321797+0000","last_active":"2023-10-08T07:14:58.516063+0000","last_peered":"2023-10-08T07:14:58.516063+0000","last_clean":"2023-10-08T07:14:58.516063+0000","last_became_active":"2023-10-08T07:14:44.321208+0000","last_became_peered":"2023-10-08T07:14:44.321208+0000","last_unstale":"2023-10-08T07:14:58.516063+0000","last_undegraded":"2023-10-08T07:14:58.516063+0000","last_fullsized":"2023-10-08T07:14:58.516063+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:01.676820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546297+0000","last_change":"2023-10-08T07:14:58.539507+0000","last_active":"2023-10-08T07:14:59.546297+0000","last_peered":"2023-10-08T07:14:59.546297+0000","last_clean":"2023-10-08T07:14:59.546297+0000","last_became_active":"2023-10-08T07:14:58.539177+0000","last_became_peered":"2023-10-08T07:14:58.539177+0000","last_unstale":"2023-10-08T07:14:59.546297+0000","last_undegraded":"2023-10-08T07:14:59.546297+0000","last_fullsized":"2023-10-08T07:14:59.546297+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:03.984646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.526721+0000","last_change":"2023-10-08T07:14:57.503202+0000","last_active":"2023-10-08T07:14:58.526721+0000","last_peered":"2023-10-08T07:14:58.526721+0000","last_clean":"2023-10-08T07:14:58.526721+0000","last_became_active":"2023-10-08T07:14:57.502784+0000","last_became_peered":"2023-10-08T07:14:57.502784+0000","last_unstale":"2023-10-08T07:14:58.526721+0000","last_undegraded":"2023-10-08T07:14:58.526721+0000","last_fullsized":"2023-10-08T07:14:58.526721+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:01.792077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530745+0000","last_change":"2023-10-08T07:14:44.329230+0000","last_active":"2023-10-08T07:14:58.530745+0000","last_peered":"2023-10-08T07:14:58.530745+0000","last_clean":"2023-10-08T07:14:58.530745+0000","last_became_active":"2023-10-08T07:14:44.328972+0000","last_became_peered":"2023-10-08T07:14:44.328972+0000","last_unstale":"2023-10-08T07:14:58.530745+0000","last_undegraded":"2023-10-08T07:14:58.530745+0000","last_fullsized":"2023-10-08T07:14:58.530745+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:59.867061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.540179+0000","last_change":"2023-10-08T07:14:58.543125+0000","last_active":"2023-10-08T07:14:59.540179+0000","last_peered":"2023-10-08T07:14:59.540179+0000","last_clean":"2023-10-08T07:14:59.540179+0000","last_became_active":"2023-10-08T07:14:58.542705+0000","last_became_peered":"2023-10-08T07:14:58.542705+0000","last_unstale":"2023-10-08T07:14:59.540179+0000","last_undegraded":"2023-10-08T07:14:59.540179+0000","last_fullsized":"2023-10-08T07:14:59.540179+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:03:55.955439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":12,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.534182+0000","last_change":"2023-10-08T07:14:57.515371+0000","last_active":"2023-10-08T07:14:58.534182+0000","last_peered":"2023-10-08T07:14:58.534182+0000","last_clean":"2023-10-08T07:14:58.534182+0000","last_became_active":"2023-10-08T07:14:57.515064+0000","last_became_peered":"2023-10-08T07:14:57.515064+0000","last_unstale":"2023-10-08T07:14:58.534182+0000","last_undegraded":"2023-10-08T07:14:58.534182+0000","last_fullsized":"2023-10-08T07:14:58.534182+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":255,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:56.458983+0000","last_clean_scrub_stamp":"2023-10-08T07:14:56.458983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:59:44.049852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529171+0000","last_change":"2023-10-08T07:14:44.337970+0000","last_active":"2023-10-08T07:14:58.529171+0000","last_peered":"2023-10-08T07:14:58.529171+0000","last_clean":"2023-10-08T07:14:58.529171+0000","last_became_active":"2023-10-08T07:14:44.337633+0000","last_became_peered":"2023-10-08T07:14:44.337633+0000","last_unstale":"2023-10-08T07:14:58.529171+0000","last_undegraded":"2023-10-08T07:14:58.529171+0000","last_fullsized":"2023-10-08T07:14:58.529171+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:55.063443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.566655+0000","last_change":"2023-10-08T07:14:58.548299+0000","last_active":"2023-10-08T07:14:59.566655+0000","last_peered":"2023-10-08T07:14:59.566655+0000","last_clean":"2023-10-08T07:14:59.566655+0000","last_became_active":"2023-10-08T07:14:58.547637+0000","last_became_peered":"2023-10-08T07:14:58.547637+0000","last_unstale":"2023-10-08T07:14:59.566655+0000","last_undegraded":"2023-10-08T07:14:59.566655+0000","last_fullsized":"2023-10-08T07:14:59.566655+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:34:36.265754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.527304+0000","last_change":"2023-10-08T07:14:44.332579+0000","last_active":"2023-10-08T07:14:58.527304+0000","last_peered":"2023-10-08T07:14:58.527304+0000","last_clean":"2023-10-08T07:14:58.527304+0000","last_became_active":"2023-10-08T07:14:44.329641+0000","last_became_peered":"2023-10-08T07:14:44.329641+0000","last_unstale":"2023-10-08T07:14:58.527304+0000","last_undegraded":"2023-10-08T07:14:58.527304+0000","last_fullsized":"2023-10-08T07:14:58.527304+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:23.709005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"253'4","reported_seq":20,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533599+0000","last_change":"2023-10-08T07:14:43.402506+0000","last_active":"2023-10-08T07:14:58.533599+0000","last_peered":"2023-10-08T07:14:58.533599+0000","last_clean":"2023-10-08T07:14:58.533599+0000","last_became_active":"2023-10-08T07:14:43.402246+0000","last_became_peered":"2023-10-08T07:14:43.402246+0000","last_unstale":"2023-10-08T07:14:58.533599+0000","last_undegraded":"2023-10-08T07:14:58.533599+0000","last_fullsized":"2023-10-08T07:14:58.533599+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:03.082313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539095+0000","last_change":"2023-10-08T07:14:58.539800+0000","last_active":"2023-10-08T07:14:59.539095+0000","last_peered":"2023-10-08T07:14:59.539095+0000","last_clean":"2023-10-08T07:14:59.539095+0000","last_became_active":"2023-10-08T07:14:58.539534+0000","last_became_peered":"2023-10-08T07:14:58.539534+0000","last_unstale":"2023-10-08T07:14:59.539095+0000","last_undegraded":"2023-10-08T07:14:59.539095+0000","last_fullsized":"2023-10-08T07:14:59.539095+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:27:05.838626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.544527+0000","last_change":"2023-10-08T07:14:44.337109+0000","last_active":"2023-10-08T07:14:58.544527+0000","last_peered":"2023-10-08T07:14:58.544527+0000","last_clean":"2023-10-08T07:14:58.544527+0000","last_became_active":"2023-10-08T07:14:44.336825+0000","last_became_peered":"2023-10-08T07:14:44.336825+0000","last_unstale":"2023-10-08T07:14:58.544527+0000","last_undegraded":"2023-10-08T07:14:58.544527+0000","last_fullsized":"2023-10-08T07:14:58.544527+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:04:02.054643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532463+0000","last_change":"2023-10-08T07:14:43.408289+0000","last_active":"2023-10-08T07:14:58.532463+0000","last_peered":"2023-10-08T07:14:58.532463+0000","last_clean":"2023-10-08T07:14:58.532463+0000","last_became_active":"2023-10-08T07:14:43.407994+0000","last_became_peered":"2023-10-08T07:14:43.407994+0000","last_unstale":"2023-10-08T07:14:58.532463+0000","last_undegraded":"2023-10-08T07:14:58.532463+0000","last_fullsized":"2023-10-08T07:14:58.532463+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:44.448458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539604+0000","last_change":"2023-10-08T07:14:58.544715+0000","last_active":"2023-10-08T07:14:59.539604+0000","last_peered":"2023-10-08T07:14:59.539604+0000","last_clean":"2023-10-08T07:14:59.539604+0000","last_became_active":"2023-10-08T07:14:58.544274+0000","last_became_peered":"2023-10-08T07:14:58.544274+0000","last_unstale":"2023-10-08T07:14:59.539604+0000","last_undegraded":"2023-10-08T07:14:59.539604+0000","last_fullsized":"2023-10-08T07:14:59.539604+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:45.982201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.531934+0000","last_change":"2023-10-08T07:14:44.332474+0000","last_active":"2023-10-08T07:14:58.531934+0000","last_peered":"2023-10-08T07:14:58.531934+0000","last_clean":"2023-10-08T07:14:58.531934+0000","last_became_active":"2023-10-08T07:14:44.332200+0000","last_became_peered":"2023-10-08T07:14:44.332200+0000","last_unstale":"2023-10-08T07:14:58.531934+0000","last_undegraded":"2023-10-08T07:14:58.531934+0000","last_fullsized":"2023-10-08T07:14:58.531934+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:19.691981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.517158+0000","last_change":"2023-10-08T07:14:43.402276+0000","last_active":"2023-10-08T07:14:58.517158+0000","last_peered":"2023-10-08T07:14:58.517158+0000","last_clean":"2023-10-08T07:14:58.517158+0000","last_became_active":"2023-10-08T07:14:43.401792+0000","last_became_peered":"2023-10-08T07:14:43.401792+0000","last_unstale":"2023-10-08T07:14:58.517158+0000","last_undegraded":"2023-10-08T07:14:58.517158+0000","last_fullsized":"2023-10-08T07:14:58.517158+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:38.609639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.540188+0000","last_change":"2023-10-08T07:14:58.556210+0000","last_active":"2023-10-08T07:14:59.540188+0000","last_peered":"2023-10-08T07:14:59.540188+0000","last_clean":"2023-10-08T07:14:59.540188+0000","last_became_active":"2023-10-08T07:14:58.555901+0000","last_became_peered":"2023-10-08T07:14:58.555901+0000","last_unstale":"2023-10-08T07:14:59.540188+0000","last_undegraded":"2023-10-08T07:14:59.540188+0000","last_fullsized":"2023-10-08T07:14:59.540188+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:28:51.597388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.516900+0000","last_change":"2023-10-08T07:14:44.333701+0000","last_active":"2023-10-08T07:14:58.516900+0000","last_peered":"2023-10-08T07:14:58.516900+0000","last_clean":"2023-10-08T07:14:58.516900+0000","last_became_active":"2023-10-08T07:14:44.333403+0000","last_became_peered":"2023-10-08T07:14:44.333403+0000","last_unstale":"2023-10-08T07:14:58.516900+0000","last_undegraded":"2023-10-08T07:14:58.516900+0000","last_fullsized":"2023-10-08T07:14:58.516900+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:24.067787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541917+0000","last_change":"2023-10-08T07:14:43.416523+0000","last_active":"2023-10-08T07:14:58.541917+0000","last_peered":"2023-10-08T07:14:58.541917+0000","last_clean":"2023-10-08T07:14:58.541917+0000","last_became_active":"2023-10-08T07:14:43.415998+0000","last_became_peered":"2023-10-08T07:14:43.415998+0000","last_unstale":"2023-10-08T07:14:58.541917+0000","last_undegraded":"2023-10-08T07:14:58.541917+0000","last_fullsized":"2023-10-08T07:14:58.541917+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:02.841738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.569455+0000","last_change":"2023-10-08T07:14:58.555275+0000","last_active":"2023-10-08T07:14:59.569455+0000","last_peered":"2023-10-08T07:14:59.569455+0000","last_clean":"2023-10-08T07:14:59.569455+0000","last_became_active":"2023-10-08T07:14:58.554967+0000","last_became_peered":"2023-10-08T07:14:58.554967+0000","last_unstale":"2023-10-08T07:14:59.569455+0000","last_undegraded":"2023-10-08T07:14:59.569455+0000","last_fullsized":"2023-10-08T07:14:59.569455+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:41:47.607554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533288+0000","last_change":"2023-10-08T07:14:44.332587+0000","last_active":"2023-10-08T07:14:58.533288+0000","last_peered":"2023-10-08T07:14:58.533288+0000","last_clean":"2023-10-08T07:14:58.533288+0000","last_became_active":"2023-10-08T07:14:44.328870+0000","last_became_peered":"2023-10-08T07:14:44.328870+0000","last_unstale":"2023-10-08T07:14:58.533288+0000","last_undegraded":"2023-10-08T07:14:58.533288+0000","last_fullsized":"2023-10-08T07:14:58.533288+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:49:30.679998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.516325+0000","last_change":"2023-10-08T07:14:43.415069+0000","last_active":"2023-10-08T07:14:58.516325+0000","last_peered":"2023-10-08T07:14:58.516325+0000","last_clean":"2023-10-08T07:14:58.516325+0000","last_became_active":"2023-10-08T07:14:43.414780+0000","last_became_peered":"2023-10-08T07:14:43.414780+0000","last_unstale":"2023-10-08T07:14:58.516325+0000","last_undegraded":"2023-10-08T07:14:58.516325+0000","last_fullsized":"2023-10-08T07:14:58.516325+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:09:43.543624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546559+0000","last_change":"2023-10-08T07:14:58.544539+0000","last_active":"2023-10-08T07:14:59.546559+0000","last_peered":"2023-10-08T07:14:59.546559+0000","last_clean":"2023-10-08T07:14:59.546559+0000","last_became_active":"2023-10-08T07:14:58.544234+0000","last_became_peered":"2023-10-08T07:14:58.544234+0000","last_unstale":"2023-10-08T07:14:59.546559+0000","last_undegraded":"2023-10-08T07:14:59.546559+0000","last_fullsized":"2023-10-08T07:14:59.546559+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:17.828746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541397+0000","last_change":"2023-10-08T07:14:44.338239+0000","last_active":"2023-10-08T07:14:58.541397+0000","last_peered":"2023-10-08T07:14:58.541397+0000","last_clean":"2023-10-08T07:14:58.541397+0000","last_became_active":"2023-10-08T07:14:44.337931+0000","last_became_peered":"2023-10-08T07:14:44.337931+0000","last_unstale":"2023-10-08T07:14:58.541397+0000","last_undegraded":"2023-10-08T07:14:58.541397+0000","last_fullsized":"2023-10-08T07:14:58.541397+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:36:59.713472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.538446+0000","last_change":"2023-10-08T07:14:43.410267+0000","last_active":"2023-10-08T07:14:58.538446+0000","last_peered":"2023-10-08T07:14:58.538446+0000","last_clean":"2023-10-08T07:14:58.538446+0000","last_became_active":"2023-10-08T07:14:43.410012+0000","last_became_peered":"2023-10-08T07:14:43.410012+0000","last_unstale":"2023-10-08T07:14:58.538446+0000","last_undegraded":"2023-10-08T07:14:58.538446+0000","last_fullsized":"2023-10-08T07:14:58.538446+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:41.903083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546371+0000","last_change":"2023-10-08T07:14:58.545365+0000","last_active":"2023-10-08T07:14:59.546371+0000","last_peered":"2023-10-08T07:14:59.546371+0000","last_clean":"2023-10-08T07:14:59.546371+0000","last_became_active":"2023-10-08T07:14:58.545053+0000","last_became_peered":"2023-10-08T07:14:58.545053+0000","last_unstale":"2023-10-08T07:14:59.546371+0000","last_undegraded":"2023-10-08T07:14:59.546371+0000","last_fullsized":"2023-10-08T07:14:59.546371+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:08.291875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529833+0000","last_change":"2023-10-08T07:14:44.330641+0000","last_active":"2023-10-08T07:14:58.529833+0000","last_peered":"2023-10-08T07:14:58.529833+0000","last_clean":"2023-10-08T07:14:58.529833+0000","last_became_active":"2023-10-08T07:14:44.330271+0000","last_became_peered":"2023-10-08T07:14:44.330271+0000","last_unstale":"2023-10-08T07:14:58.529833+0000","last_undegraded":"2023-10-08T07:14:58.529833+0000","last_fullsized":"2023-10-08T07:14:58.529833+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:35.355050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529364+0000","last_change":"2023-10-08T07:14:43.407886+0000","last_active":"2023-10-08T07:14:58.529364+0000","last_peered":"2023-10-08T07:14:58.529364+0000","last_clean":"2023-10-08T07:14:58.529364+0000","last_became_active":"2023-10-08T07:14:43.407634+0000","last_became_peered":"2023-10-08T07:14:43.407634+0000","last_unstale":"2023-10-08T07:14:58.529364+0000","last_undegraded":"2023-10-08T07:14:58.529364+0000","last_fullsized":"2023-10-08T07:14:58.529364+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:55:43.492128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.544787+0000","last_change":"2023-10-08T07:14:58.544652+0000","last_active":"2023-10-08T07:14:59.544787+0000","last_peered":"2023-10-08T07:14:59.544787+0000","last_clean":"2023-10-08T07:14:59.544787+0000","last_became_active":"2023-10-08T07:14:58.544354+0000","last_became_peered":"2023-10-08T07:14:58.544354+0000","last_unstale":"2023-10-08T07:14:59.544787+0000","last_undegraded":"2023-10-08T07:14:59.544787+0000","last_fullsized":"2023-10-08T07:14:59.544787+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:58.222778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.536245+0000","last_change":"2023-10-08T07:14:44.332428+0000","last_active":"2023-10-08T07:14:58.536245+0000","last_peered":"2023-10-08T07:14:58.536245+0000","last_clean":"2023-10-08T07:14:58.536245+0000","last_became_active":"2023-10-08T07:14:44.331865+0000","last_became_peered":"2023-10-08T07:14:44.331865+0000","last_unstale":"2023-10-08T07:14:58.536245+0000","last_undegraded":"2023-10-08T07:14:58.536245+0000","last_fullsized":"2023-10-08T07:14:58.536245+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:04.509562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.519168+0000","last_change":"2023-10-08T07:14:43.397332+0000","last_active":"2023-10-08T07:14:58.519168+0000","last_peered":"2023-10-08T07:14:58.519168+0000","last_clean":"2023-10-08T07:14:58.519168+0000","last_became_active":"2023-10-08T07:14:43.397049+0000","last_became_peered":"2023-10-08T07:14:43.397049+0000","last_unstale":"2023-10-08T07:14:58.519168+0000","last_undegraded":"2023-10-08T07:14:58.519168+0000","last_fullsized":"2023-10-08T07:14:58.519168+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:23.709280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567021+0000","last_change":"2023-10-08T07:14:58.555347+0000","last_active":"2023-10-08T07:14:59.567021+0000","last_peered":"2023-10-08T07:14:59.567021+0000","last_clean":"2023-10-08T07:14:59.567021+0000","last_became_active":"2023-10-08T07:14:58.554665+0000","last_became_peered":"2023-10-08T07:14:58.554665+0000","last_unstale":"2023-10-08T07:14:59.567021+0000","last_undegraded":"2023-10-08T07:14:59.567021+0000","last_fullsized":"2023-10-08T07:14:59.567021+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:39:45.982201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.526067+0000","last_change":"2023-10-08T07:14:44.331844+0000","last_active":"2023-10-08T07:14:58.526067+0000","last_peered":"2023-10-08T07:14:58.526067+0000","last_clean":"2023-10-08T07:14:58.526067+0000","last_became_active":"2023-10-08T07:14:44.331521+0000","last_became_peered":"2023-10-08T07:14:44.331521+0000","last_unstale":"2023-10-08T07:14:58.526067+0000","last_undegraded":"2023-10-08T07:14:58.526067+0000","last_fullsized":"2023-10-08T07:14:58.526067+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:22.760377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.541697+0000","last_change":"2023-10-08T07:14:43.414266+0000","last_active":"2023-10-08T07:14:58.541697+0000","last_peered":"2023-10-08T07:14:58.541697+0000","last_clean":"2023-10-08T07:14:58.541697+0000","last_became_active":"2023-10-08T07:14:43.413877+0000","last_became_peered":"2023-10-08T07:14:43.413877+0000","last_unstale":"2023-10-08T07:14:58.541697+0000","last_undegraded":"2023-10-08T07:14:58.541697+0000","last_fullsized":"2023-10-08T07:14:58.541697+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:37:00.790509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.542575+0000","last_change":"2023-10-08T07:14:58.551858+0000","last_active":"2023-10-08T07:14:59.542575+0000","last_peered":"2023-10-08T07:14:59.542575+0000","last_clean":"2023-10-08T07:14:59.542575+0000","last_became_active":"2023-10-08T07:14:58.550576+0000","last_became_peered":"2023-10-08T07:14:58.550576+0000","last_unstale":"2023-10-08T07:14:59.542575+0000","last_undegraded":"2023-10-08T07:14:59.542575+0000","last_fullsized":"2023-10-08T07:14:59.542575+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:22.021516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.544457+0000","last_change":"2023-10-08T07:14:44.330684+0000","last_active":"2023-10-08T07:14:58.544457+0000","last_peered":"2023-10-08T07:14:58.544457+0000","last_clean":"2023-10-08T07:14:58.544457+0000","last_became_active":"2023-10-08T07:14:44.330353+0000","last_became_peered":"2023-10-08T07:14:44.330353+0000","last_unstale":"2023-10-08T07:14:58.544457+0000","last_undegraded":"2023-10-08T07:14:58.544457+0000","last_fullsized":"2023-10-08T07:14:58.544457+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:14:27.946683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"253'3","reported_seq":19,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529186+0000","last_change":"2023-10-08T07:14:43.410104+0000","last_active":"2023-10-08T07:14:58.529186+0000","last_peered":"2023-10-08T07:14:58.529186+0000","last_clean":"2023-10-08T07:14:58.529186+0000","last_became_active":"2023-10-08T07:14:43.409558+0000","last_became_peered":"2023-10-08T07:14:43.409558+0000","last_unstale":"2023-10-08T07:14:58.529186+0000","last_undegraded":"2023-10-08T07:14:58.529186+0000","last_fullsized":"2023-10-08T07:14:58.529186+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:47.180803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539147+0000","last_change":"2023-10-08T07:14:58.546741+0000","last_active":"2023-10-08T07:14:59.539147+0000","last_peered":"2023-10-08T07:14:59.539147+0000","last_clean":"2023-10-08T07:14:59.539147+0000","last_became_active":"2023-10-08T07:14:58.546430+0000","last_became_peered":"2023-10-08T07:14:58.546430+0000","last_unstale":"2023-10-08T07:14:59.539147+0000","last_undegraded":"2023-10-08T07:14:59.539147+0000","last_fullsized":"2023-10-08T07:14:59.539147+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:45.234198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532838+0000","last_change":"2023-10-08T07:14:44.326288+0000","last_active":"2023-10-08T07:14:58.532838+0000","last_peered":"2023-10-08T07:14:58.532838+0000","last_clean":"2023-10-08T07:14:58.532838+0000","last_became_active":"2023-10-08T07:14:44.325977+0000","last_became_peered":"2023-10-08T07:14:44.325977+0000","last_unstale":"2023-10-08T07:14:58.532838+0000","last_undegraded":"2023-10-08T07:14:58.532838+0000","last_fullsized":"2023-10-08T07:14:58.532838+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:31.119993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"254'8","reported_seq":24,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.531991+0000","last_change":"2023-10-08T07:14:43.399270+0000","last_active":"2023-10-08T07:14:58.531991+0000","last_peered":"2023-10-08T07:14:58.531991+0000","last_clean":"2023-10-08T07:14:58.531991+0000","last_became_active":"2023-10-08T07:14:43.398978+0000","last_became_peered":"2023-10-08T07:14:43.398978+0000","last_unstale":"2023-10-08T07:14:58.531991+0000","last_undegraded":"2023-10-08T07:14:58.531991+0000","last_fullsized":"2023-10-08T07:14:58.531991+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:49.950109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2857,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.571183+0000","last_change":"2023-10-08T07:14:58.555258+0000","last_active":"2023-10-08T07:14:59.571183+0000","last_peered":"2023-10-08T07:14:59.571183+0000","last_clean":"2023-10-08T07:14:59.571183+0000","last_became_active":"2023-10-08T07:14:58.554985+0000","last_became_peered":"2023-10-08T07:14:58.554985+0000","last_unstale":"2023-10-08T07:14:59.571183+0000","last_undegraded":"2023-10-08T07:14:59.571183+0000","last_fullsized":"2023-10-08T07:14:59.571183+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:36:18.819012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533283+0000","last_change":"2023-10-08T07:14:44.335266+0000","last_active":"2023-10-08T07:14:58.533283+0000","last_peered":"2023-10-08T07:14:58.533283+0000","last_clean":"2023-10-08T07:14:58.533283+0000","last_became_active":"2023-10-08T07:14:44.334971+0000","last_became_peered":"2023-10-08T07:14:44.334971+0000","last_unstale":"2023-10-08T07:14:58.533283+0000","last_undegraded":"2023-10-08T07:14:58.533283+0000","last_fullsized":"2023-10-08T07:14:58.533283+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:05:14.581717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.542027+0000","last_change":"2023-10-08T07:14:43.410713+0000","last_active":"2023-10-08T07:14:58.542027+0000","last_peered":"2023-10-08T07:14:58.542027+0000","last_clean":"2023-10-08T07:14:58.542027+0000","last_became_active":"2023-10-08T07:14:43.410448+0000","last_became_peered":"2023-10-08T07:14:43.410448+0000","last_unstale":"2023-10-08T07:14:58.542027+0000","last_undegraded":"2023-10-08T07:14:58.542027+0000","last_fullsized":"2023-10-08T07:14:58.542027+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:03:40.822722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.547148+0000","last_change":"2023-10-08T07:14:58.539818+0000","last_active":"2023-10-08T07:14:59.547148+0000","last_peered":"2023-10-08T07:14:59.547148+0000","last_clean":"2023-10-08T07:14:59.547148+0000","last_became_active":"2023-10-08T07:14:58.539314+0000","last_became_peered":"2023-10-08T07:14:58.539314+0000","last_unstale":"2023-10-08T07:14:59.547148+0000","last_undegraded":"2023-10-08T07:14:59.547148+0000","last_fullsized":"2023-10-08T07:14:59.547148+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:25.327883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533152+0000","last_change":"2023-10-08T07:14:44.327892+0000","last_active":"2023-10-08T07:14:58.533152+0000","last_peered":"2023-10-08T07:14:58.533152+0000","last_clean":"2023-10-08T07:14:58.533152+0000","last_became_active":"2023-10-08T07:14:44.327480+0000","last_became_peered":"2023-10-08T07:14:44.327480+0000","last_unstale":"2023-10-08T07:14:58.533152+0000","last_undegraded":"2023-10-08T07:14:58.533152+0000","last_fullsized":"2023-10-08T07:14:58.533152+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:26.279630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.524587+0000","last_change":"2023-10-08T07:14:43.404280+0000","last_active":"2023-10-08T07:14:58.524587+0000","last_peered":"2023-10-08T07:14:58.524587+0000","last_clean":"2023-10-08T07:14:58.524587+0000","last_became_active":"2023-10-08T07:14:43.403911+0000","last_became_peered":"2023-10-08T07:14:43.403911+0000","last_unstale":"2023-10-08T07:14:58.524587+0000","last_undegraded":"2023-10-08T07:14:58.524587+0000","last_fullsized":"2023-10-08T07:14:58.524587+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:06.833347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.568684+0000","last_change":"2023-10-08T07:14:58.558886+0000","last_active":"2023-10-08T07:14:59.568684+0000","last_peered":"2023-10-08T07:14:59.568684+0000","last_clean":"2023-10-08T07:14:59.568684+0000","last_became_active":"2023-10-08T07:14:58.558626+0000","last_became_peered":"2023-10-08T07:14:58.558626+0000","last_unstale":"2023-10-08T07:14:59.568684+0000","last_undegraded":"2023-10-08T07:14:59.568684+0000","last_fullsized":"2023-10-08T07:14:59.568684+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:52.509843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.542943+0000","last_change":"2023-10-08T07:14:44.336892+0000","last_active":"2023-10-08T07:14:58.542943+0000","last_peered":"2023-10-08T07:14:58.542943+0000","last_clean":"2023-10-08T07:14:58.542943+0000","last_became_active":"2023-10-08T07:14:44.336589+0000","last_became_peered":"2023-10-08T07:14:44.336589+0000","last_unstale":"2023-10-08T07:14:58.542943+0000","last_undegraded":"2023-10-08T07:14:58.542943+0000","last_fullsized":"2023-10-08T07:14:58.542943+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:07.990108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.525639+0000","last_change":"2023-10-08T07:14:43.411394+0000","last_active":"2023-10-08T07:14:58.525639+0000","last_peered":"2023-10-08T07:14:58.525639+0000","last_clean":"2023-10-08T07:14:58.525639+0000","last_became_active":"2023-10-08T07:14:43.411091+0000","last_became_peered":"2023-10-08T07:14:43.411091+0000","last_unstale":"2023-10-08T07:14:58.525639+0000","last_undegraded":"2023-10-08T07:14:58.525639+0000","last_fullsized":"2023-10-08T07:14:58.525639+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:44:12.677401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.546970+0000","last_change":"2023-10-08T07:14:58.549819+0000","last_active":"2023-10-08T07:14:59.546970+0000","last_peered":"2023-10-08T07:14:59.546970+0000","last_clean":"2023-10-08T07:14:59.546970+0000","last_became_active":"2023-10-08T07:14:58.549535+0000","last_became_peered":"2023-10-08T07:14:58.549535+0000","last_unstale":"2023-10-08T07:14:59.546970+0000","last_undegraded":"2023-10-08T07:14:59.546970+0000","last_fullsized":"2023-10-08T07:14:59.546970+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:52.541810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.527169+0000","last_change":"2023-10-08T07:14:44.340304+0000","last_active":"2023-10-08T07:14:58.527169+0000","last_peered":"2023-10-08T07:14:58.527169+0000","last_clean":"2023-10-08T07:14:58.527169+0000","last_became_active":"2023-10-08T07:14:44.338301+0000","last_became_peered":"2023-10-08T07:14:44.338301+0000","last_unstale":"2023-10-08T07:14:58.527169+0000","last_undegraded":"2023-10-08T07:14:58.527169+0000","last_fullsized":"2023-10-08T07:14:58.527169+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:42:14.658077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"253'8","reported_seq":24,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.527532+0000","last_change":"2023-10-08T07:14:43.407648+0000","last_active":"2023-10-08T07:14:58.527532+0000","last_peered":"2023-10-08T07:14:58.527532+0000","last_clean":"2023-10-08T07:14:58.527532+0000","last_became_active":"2023-10-08T07:14:43.407376+0000","last_became_peered":"2023-10-08T07:14:43.407376+0000","last_unstale":"2023-10-08T07:14:58.527532+0000","last_undegraded":"2023-10-08T07:14:58.527532+0000","last_fullsized":"2023-10-08T07:14:58.527532+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:26:50.563868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539729+0000","last_change":"2023-10-08T07:14:58.542960+0000","last_active":"2023-10-08T07:14:59.539729+0000","last_peered":"2023-10-08T07:14:59.539729+0000","last_clean":"2023-10-08T07:14:59.539729+0000","last_became_active":"2023-10-08T07:14:58.542310+0000","last_became_peered":"2023-10-08T07:14:58.542310+0000","last_unstale":"2023-10-08T07:14:59.539729+0000","last_undegraded":"2023-10-08T07:14:59.539729+0000","last_fullsized":"2023-10-08T07:14:59.539729+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:17.645216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.526793+0000","last_change":"2023-10-08T07:14:44.335197+0000","last_active":"2023-10-08T07:14:58.526793+0000","last_peered":"2023-10-08T07:14:58.526793+0000","last_clean":"2023-10-08T07:14:58.526793+0000","last_became_active":"2023-10-08T07:14:44.333014+0000","last_became_peered":"2023-10-08T07:14:44.333014+0000","last_unstale":"2023-10-08T07:14:58.526793+0000","last_undegraded":"2023-10-08T07:14:58.526793+0000","last_fullsized":"2023-10-08T07:14:58.526793+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:36:04.704806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.533779+0000","last_change":"2023-10-08T07:14:43.411294+0000","last_active":"2023-10-08T07:14:58.533779+0000","last_peered":"2023-10-08T07:14:58.533779+0000","last_clean":"2023-10-08T07:14:58.533779+0000","last_became_active":"2023-10-08T07:14:43.411000+0000","last_became_peered":"2023-10-08T07:14:43.411000+0000","last_unstale":"2023-10-08T07:14:58.533779+0000","last_undegraded":"2023-10-08T07:14:58.533779+0000","last_fullsized":"2023-10-08T07:14:58.533779+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:42:35.351878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.566969+0000","last_change":"2023-10-08T07:14:58.551414+0000","last_active":"2023-10-08T07:14:59.566969+0000","last_peered":"2023-10-08T07:14:59.566969+0000","last_clean":"2023-10-08T07:14:59.566969+0000","last_became_active":"2023-10-08T07:14:58.550969+0000","last_became_peered":"2023-10-08T07:14:58.550969+0000","last_unstale":"2023-10-08T07:14:59.566969+0000","last_undegraded":"2023-10-08T07:14:59.566969+0000","last_fullsized":"2023-10-08T07:14:59.566969+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:29.829692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.525458+0000","last_change":"2023-10-08T07:14:44.331894+0000","last_active":"2023-10-08T07:14:58.525458+0000","last_peered":"2023-10-08T07:14:58.525458+0000","last_clean":"2023-10-08T07:14:58.525458+0000","last_became_active":"2023-10-08T07:14:44.331612+0000","last_became_peered":"2023-10-08T07:14:44.331612+0000","last_unstale":"2023-10-08T07:14:58.525458+0000","last_undegraded":"2023-10-08T07:14:58.525458+0000","last_fullsized":"2023-10-08T07:14:58.525458+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:17.150803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.538587+0000","last_change":"2023-10-08T07:14:43.410517+0000","last_active":"2023-10-08T07:14:58.538587+0000","last_peered":"2023-10-08T07:14:58.538587+0000","last_clean":"2023-10-08T07:14:58.538587+0000","last_became_active":"2023-10-08T07:14:43.409771+0000","last_became_peered":"2023-10-08T07:14:43.409771+0000","last_unstale":"2023-10-08T07:14:58.538587+0000","last_undegraded":"2023-10-08T07:14:58.538587+0000","last_fullsized":"2023-10-08T07:14:58.538587+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:42.844452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.542968+0000","last_change":"2023-10-08T07:14:58.541999+0000","last_active":"2023-10-08T07:14:59.542968+0000","last_peered":"2023-10-08T07:14:59.542968+0000","last_clean":"2023-10-08T07:14:59.542968+0000","last_became_active":"2023-10-08T07:14:58.540640+0000","last_became_peered":"2023-10-08T07:14:58.540640+0000","last_unstale":"2023-10-08T07:14:59.542968+0000","last_undegraded":"2023-10-08T07:14:59.542968+0000","last_fullsized":"2023-10-08T07:14:59.542968+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:12:18.752597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532709+0000","last_change":"2023-10-08T07:14:44.342867+0000","last_active":"2023-10-08T07:14:58.532709+0000","last_peered":"2023-10-08T07:14:58.532709+0000","last_clean":"2023-10-08T07:14:58.532709+0000","last_became_active":"2023-10-08T07:14:44.342541+0000","last_became_peered":"2023-10-08T07:14:44.342541+0000","last_unstale":"2023-10-08T07:14:58.532709+0000","last_undegraded":"2023-10-08T07:14:58.532709+0000","last_fullsized":"2023-10-08T07:14:58.532709+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:43.616734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532593+0000","last_change":"2023-10-08T07:14:43.413188+0000","last_active":"2023-10-08T07:14:58.532593+0000","last_peered":"2023-10-08T07:14:58.532593+0000","last_clean":"2023-10-08T07:14:58.532593+0000","last_became_active":"2023-10-08T07:14:43.412716+0000","last_became_peered":"2023-10-08T07:14:43.412716+0000","last_unstale":"2023-10-08T07:14:58.532593+0000","last_undegraded":"2023-10-08T07:14:58.532593+0000","last_fullsized":"2023-10-08T07:14:58.532593+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:08.671468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539173+0000","last_change":"2023-10-08T07:14:58.540678+0000","last_active":"2023-10-08T07:14:59.539173+0000","last_peered":"2023-10-08T07:14:59.539173+0000","last_clean":"2023-10-08T07:14:59.539173+0000","last_became_active":"2023-10-08T07:14:58.540408+0000","last_became_peered":"2023-10-08T07:14:58.540408+0000","last_unstale":"2023-10-08T07:14:59.539173+0000","last_undegraded":"2023-10-08T07:14:59.539173+0000","last_fullsized":"2023-10-08T07:14:59.539173+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:36.939019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.544328+0000","last_change":"2023-10-08T07:14:44.332886+0000","last_active":"2023-10-08T07:14:58.544328+0000","last_peered":"2023-10-08T07:14:58.544328+0000","last_clean":"2023-10-08T07:14:58.544328+0000","last_became_active":"2023-10-08T07:14:44.332142+0000","last_became_peered":"2023-10-08T07:14:44.332142+0000","last_unstale":"2023-10-08T07:14:58.544328+0000","last_undegraded":"2023-10-08T07:14:58.544328+0000","last_fullsized":"2023-10-08T07:14:58.544328+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:49.297056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.547760+0000","last_change":"2023-10-08T07:14:43.415034+0000","last_active":"2023-10-08T07:14:58.547760+0000","last_peered":"2023-10-08T07:14:58.547760+0000","last_clean":"2023-10-08T07:14:58.547760+0000","last_became_active":"2023-10-08T07:14:43.414728+0000","last_became_peered":"2023-10-08T07:14:43.414728+0000","last_unstale":"2023-10-08T07:14:58.547760+0000","last_undegraded":"2023-10-08T07:14:58.547760+0000","last_fullsized":"2023-10-08T07:14:58.547760+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:08.048265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.547212+0000","last_change":"2023-10-08T07:14:58.550042+0000","last_active":"2023-10-08T07:14:59.547212+0000","last_peered":"2023-10-08T07:14:59.547212+0000","last_clean":"2023-10-08T07:14:59.547212+0000","last_became_active":"2023-10-08T07:14:58.548159+0000","last_became_peered":"2023-10-08T07:14:58.548159+0000","last_unstale":"2023-10-08T07:14:59.547212+0000","last_undegraded":"2023-10-08T07:14:59.547212+0000","last_fullsized":"2023-10-08T07:14:59.547212+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:19.898264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.538044+0000","last_change":"2023-10-08T07:14:44.340208+0000","last_active":"2023-10-08T07:14:58.538044+0000","last_peered":"2023-10-08T07:14:58.538044+0000","last_clean":"2023-10-08T07:14:58.538044+0000","last_became_active":"2023-10-08T07:14:44.339934+0000","last_became_peered":"2023-10-08T07:14:44.339934+0000","last_unstale":"2023-10-08T07:14:58.538044+0000","last_undegraded":"2023-10-08T07:14:58.538044+0000","last_fullsized":"2023-10-08T07:14:58.538044+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:09:26.451027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.544409+0000","last_change":"2023-10-08T07:14:43.413078+0000","last_active":"2023-10-08T07:14:58.544409+0000","last_peered":"2023-10-08T07:14:58.544409+0000","last_clean":"2023-10-08T07:14:58.544409+0000","last_became_active":"2023-10-08T07:14:43.412804+0000","last_became_peered":"2023-10-08T07:14:43.412804+0000","last_unstale":"2023-10-08T07:14:58.544409+0000","last_undegraded":"2023-10-08T07:14:58.544409+0000","last_fullsized":"2023-10-08T07:14:58.544409+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:12.105709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.568010+0000","last_change":"2023-10-08T07:14:58.558588+0000","last_active":"2023-10-08T07:14:59.568010+0000","last_peered":"2023-10-08T07:14:59.568010+0000","last_clean":"2023-10-08T07:14:59.568010+0000","last_became_active":"2023-10-08T07:14:58.558277+0000","last_became_peered":"2023-10-08T07:14:58.558277+0000","last_unstale":"2023-10-08T07:14:59.568010+0000","last_undegraded":"2023-10-08T07:14:59.568010+0000","last_fullsized":"2023-10-08T07:14:59.568010+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:04.003417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.526558+0000","last_change":"2023-10-08T07:14:44.333114+0000","last_active":"2023-10-08T07:14:58.526558+0000","last_peered":"2023-10-08T07:14:58.526558+0000","last_clean":"2023-10-08T07:14:58.526558+0000","last_became_active":"2023-10-08T07:14:44.332105+0000","last_became_peered":"2023-10-08T07:14:44.332105+0000","last_unstale":"2023-10-08T07:14:58.526558+0000","last_undegraded":"2023-10-08T07:14:58.526558+0000","last_fullsized":"2023-10-08T07:14:58.526558+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:58:23.020597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.525803+0000","last_change":"2023-10-08T07:14:43.407284+0000","last_active":"2023-10-08T07:14:58.525803+0000","last_peered":"2023-10-08T07:14:58.525803+0000","last_clean":"2023-10-08T07:14:58.525803+0000","last_became_active":"2023-10-08T07:14:43.407010+0000","last_became_peered":"2023-10-08T07:14:43.407010+0000","last_unstale":"2023-10-08T07:14:58.525803+0000","last_undegraded":"2023-10-08T07:14:58.525803+0000","last_fullsized":"2023-10-08T07:14:58.525803+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:00:18.544340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.568531+0000","last_change":"2023-10-08T07:14:58.558284+0000","last_active":"2023-10-08T07:14:59.568531+0000","last_peered":"2023-10-08T07:14:59.568531+0000","last_clean":"2023-10-08T07:14:59.568531+0000","last_became_active":"2023-10-08T07:14:58.557913+0000","last_became_peered":"2023-10-08T07:14:58.557913+0000","last_unstale":"2023-10-08T07:14:59.568531+0000","last_undegraded":"2023-10-08T07:14:59.568531+0000","last_fullsized":"2023-10-08T07:14:59.568531+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:04:23.513726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530729+0000","last_change":"2023-10-08T07:14:44.339779+0000","last_active":"2023-10-08T07:14:58.530729+0000","last_peered":"2023-10-08T07:14:58.530729+0000","last_clean":"2023-10-08T07:14:58.530729+0000","last_became_active":"2023-10-08T07:14:44.339385+0000","last_became_peered":"2023-10-08T07:14:44.339385+0000","last_unstale":"2023-10-08T07:14:58.530729+0000","last_undegraded":"2023-10-08T07:14:58.530729+0000","last_fullsized":"2023-10-08T07:14:58.530729+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:44:13.695913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.526327+0000","last_change":"2023-10-08T07:14:43.408990+0000","last_active":"2023-10-08T07:14:58.526327+0000","last_peered":"2023-10-08T07:14:58.526327+0000","last_clean":"2023-10-08T07:14:58.526327+0000","last_became_active":"2023-10-08T07:14:43.408702+0000","last_became_peered":"2023-10-08T07:14:43.408702+0000","last_unstale":"2023-10-08T07:14:58.526327+0000","last_undegraded":"2023-10-08T07:14:58.526327+0000","last_fullsized":"2023-10-08T07:14:58.526327+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:42:54.133156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.545965+0000","last_change":"2023-10-08T07:14:58.553095+0000","last_active":"2023-10-08T07:14:59.545965+0000","last_peered":"2023-10-08T07:14:59.545965+0000","last_clean":"2023-10-08T07:14:59.545965+0000","last_became_active":"2023-10-08T07:14:58.552482+0000","last_became_peered":"2023-10-08T07:14:58.552482+0000","last_unstale":"2023-10-08T07:14:59.545965+0000","last_undegraded":"2023-10-08T07:14:59.545965+0000","last_fullsized":"2023-10-08T07:14:59.545965+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:18.387133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530685+0000","last_change":"2023-10-08T07:14:44.333909+0000","last_active":"2023-10-08T07:14:58.530685+0000","last_peered":"2023-10-08T07:14:58.530685+0000","last_clean":"2023-10-08T07:14:58.530685+0000","last_became_active":"2023-10-08T07:14:44.333646+0000","last_became_peered":"2023-10-08T07:14:44.333646+0000","last_unstale":"2023-10-08T07:14:58.530685+0000","last_undegraded":"2023-10-08T07:14:58.530685+0000","last_fullsized":"2023-10-08T07:14:58.530685+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:50.362306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.515570+0000","last_change":"2023-10-08T07:14:43.406333+0000","last_active":"2023-10-08T07:14:58.515570+0000","last_peered":"2023-10-08T07:14:58.515570+0000","last_clean":"2023-10-08T07:14:58.515570+0000","last_became_active":"2023-10-08T07:14:43.406052+0000","last_became_peered":"2023-10-08T07:14:43.406052+0000","last_unstale":"2023-10-08T07:14:58.515570+0000","last_undegraded":"2023-10-08T07:14:58.515570+0000","last_fullsized":"2023-10-08T07:14:58.515570+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:24:31.926662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.566662+0000","last_change":"2023-10-08T07:14:58.548464+0000","last_active":"2023-10-08T07:14:59.566662+0000","last_peered":"2023-10-08T07:14:59.566662+0000","last_clean":"2023-10-08T07:14:59.566662+0000","last_became_active":"2023-10-08T07:14:58.548017+0000","last_became_peered":"2023-10-08T07:14:58.548017+0000","last_unstale":"2023-10-08T07:14:59.566662+0000","last_undegraded":"2023-10-08T07:14:59.566662+0000","last_fullsized":"2023-10-08T07:14:59.566662+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:25.034062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.536582+0000","last_change":"2023-10-08T07:14:44.338131+0000","last_active":"2023-10-08T07:14:58.536582+0000","last_peered":"2023-10-08T07:14:58.536582+0000","last_clean":"2023-10-08T07:14:58.536582+0000","last_became_active":"2023-10-08T07:14:44.337734+0000","last_became_peered":"2023-10-08T07:14:44.337734+0000","last_unstale":"2023-10-08T07:14:58.536582+0000","last_undegraded":"2023-10-08T07:14:58.536582+0000","last_fullsized":"2023-10-08T07:14:58.536582+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:29:05.679033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"253'2","reported_seq":18,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.554034+0000","last_change":"2023-10-08T07:14:43.414871+0000","last_active":"2023-10-08T07:14:58.554034+0000","last_peered":"2023-10-08T07:14:58.554034+0000","last_clean":"2023-10-08T07:14:58.554034+0000","last_became_active":"2023-10-08T07:14:43.414608+0000","last_became_peered":"2023-10-08T07:14:43.414608+0000","last_unstale":"2023-10-08T07:14:58.554034+0000","last_undegraded":"2023-10-08T07:14:58.554034+0000","last_fullsized":"2023-10-08T07:14:58.554034+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:35.056822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567444+0000","last_change":"2023-10-08T07:14:58.547522+0000","last_active":"2023-10-08T07:14:59.567444+0000","last_peered":"2023-10-08T07:14:59.567444+0000","last_clean":"2023-10-08T07:14:59.567444+0000","last_became_active":"2023-10-08T07:14:58.547139+0000","last_became_peered":"2023-10-08T07:14:58.547139+0000","last_unstale":"2023-10-08T07:14:59.567444+0000","last_undegraded":"2023-10-08T07:14:59.567444+0000","last_fullsized":"2023-10-08T07:14:59.567444+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:39.044877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.540503+0000","last_change":"2023-10-08T07:14:44.330281+0000","last_active":"2023-10-08T07:14:58.540503+0000","last_peered":"2023-10-08T07:14:58.540503+0000","last_clean":"2023-10-08T07:14:58.540503+0000","last_became_active":"2023-10-08T07:14:44.329938+0000","last_became_peered":"2023-10-08T07:14:44.329938+0000","last_unstale":"2023-10-08T07:14:58.540503+0000","last_undegraded":"2023-10-08T07:14:58.540503+0000","last_fullsized":"2023-10-08T07:14:58.540503+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:22:16.367973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"254'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530270+0000","last_change":"2023-10-08T07:14:43.414678+0000","last_active":"2023-10-08T07:14:58.530270+0000","last_peered":"2023-10-08T07:14:58.530270+0000","last_clean":"2023-10-08T07:14:58.530270+0000","last_became_active":"2023-10-08T07:14:43.414400+0000","last_became_peered":"2023-10-08T07:14:43.414400+0000","last_unstale":"2023-10-08T07:14:58.530270+0000","last_undegraded":"2023-10-08T07:14:58.530270+0000","last_fullsized":"2023-10-08T07:14:58.530270+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:43:12.932565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.564904+0000","last_change":"2023-10-08T07:14:58.549721+0000","last_active":"2023-10-08T07:14:59.564904+0000","last_peered":"2023-10-08T07:14:59.564904+0000","last_clean":"2023-10-08T07:14:59.564904+0000","last_became_active":"2023-10-08T07:14:58.549442+0000","last_became_peered":"2023-10-08T07:14:58.549442+0000","last_unstale":"2023-10-08T07:14:59.564904+0000","last_undegraded":"2023-10-08T07:14:59.564904+0000","last_fullsized":"2023-10-08T07:14:59.564904+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:42.784401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.542285+0000","last_change":"2023-10-08T07:14:44.337389+0000","last_active":"2023-10-08T07:14:58.542285+0000","last_peered":"2023-10-08T07:14:58.542285+0000","last_clean":"2023-10-08T07:14:58.542285+0000","last_became_active":"2023-10-08T07:14:44.337118+0000","last_became_peered":"2023-10-08T07:14:44.337118+0000","last_unstale":"2023-10-08T07:14:58.542285+0000","last_undegraded":"2023-10-08T07:14:58.542285+0000","last_fullsized":"2023-10-08T07:14:58.542285+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:11.369315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.518892+0000","last_change":"2023-10-08T07:14:43.404901+0000","last_active":"2023-10-08T07:14:58.518892+0000","last_peered":"2023-10-08T07:14:58.518892+0000","last_clean":"2023-10-08T07:14:58.518892+0000","last_became_active":"2023-10-08T07:14:43.404611+0000","last_became_peered":"2023-10-08T07:14:43.404611+0000","last_unstale":"2023-10-08T07:14:58.518892+0000","last_undegraded":"2023-10-08T07:14:58.518892+0000","last_fullsized":"2023-10-08T07:14:58.518892+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:45:51.297454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567736+0000","last_change":"2023-10-08T07:14:58.544019+0000","last_active":"2023-10-08T07:14:59.567736+0000","last_peered":"2023-10-08T07:14:59.567736+0000","last_clean":"2023-10-08T07:14:59.567736+0000","last_became_active":"2023-10-08T07:14:58.543727+0000","last_became_peered":"2023-10-08T07:14:58.543727+0000","last_unstale":"2023-10-08T07:14:59.567736+0000","last_undegraded":"2023-10-08T07:14:59.567736+0000","last_fullsized":"2023-10-08T07:14:59.567736+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:08.605466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.522845+0000","last_change":"2023-10-08T07:14:44.332132+0000","last_active":"2023-10-08T07:14:58.522845+0000","last_peered":"2023-10-08T07:14:58.522845+0000","last_clean":"2023-10-08T07:14:58.522845+0000","last_became_active":"2023-10-08T07:14:44.331822+0000","last_became_peered":"2023-10-08T07:14:44.331822+0000","last_unstale":"2023-10-08T07:14:58.522845+0000","last_undegraded":"2023-10-08T07:14:58.522845+0000","last_fullsized":"2023-10-08T07:14:58.522845+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:03:21.791606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.517398+0000","last_change":"2023-10-08T07:14:43.409897+0000","last_active":"2023-10-08T07:14:58.517398+0000","last_peered":"2023-10-08T07:14:58.517398+0000","last_clean":"2023-10-08T07:14:58.517398+0000","last_became_active":"2023-10-08T07:14:43.409442+0000","last_became_peered":"2023-10-08T07:14:43.409442+0000","last_unstale":"2023-10-08T07:14:58.517398+0000","last_undegraded":"2023-10-08T07:14:58.517398+0000","last_fullsized":"2023-10-08T07:14:58.517398+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:30:12.105709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.539858+0000","last_change":"2023-10-08T07:14:58.543755+0000","last_active":"2023-10-08T07:14:59.539858+0000","last_peered":"2023-10-08T07:14:59.539858+0000","last_clean":"2023-10-08T07:14:59.539858+0000","last_became_active":"2023-10-08T07:14:58.543403+0000","last_became_peered":"2023-10-08T07:14:58.543403+0000","last_unstale":"2023-10-08T07:14:59.539858+0000","last_undegraded":"2023-10-08T07:14:59.539858+0000","last_fullsized":"2023-10-08T07:14:59.539858+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:48.166629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.515597+0000","last_change":"2023-10-08T07:14:44.330119+0000","last_active":"2023-10-08T07:14:58.515597+0000","last_peered":"2023-10-08T07:14:58.515597+0000","last_clean":"2023-10-08T07:14:58.515597+0000","last_became_active":"2023-10-08T07:14:44.329841+0000","last_became_peered":"2023-10-08T07:14:44.329841+0000","last_unstale":"2023-10-08T07:14:58.515597+0000","last_undegraded":"2023-10-08T07:14:58.515597+0000","last_fullsized":"2023-10-08T07:14:58.515597+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:02:17.916779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.529408+0000","last_change":"2023-10-08T07:14:43.406046+0000","last_active":"2023-10-08T07:14:58.529408+0000","last_peered":"2023-10-08T07:14:58.529408+0000","last_clean":"2023-10-08T07:14:58.529408+0000","last_became_active":"2023-10-08T07:14:43.405138+0000","last_became_peered":"2023-10-08T07:14:43.405138+0000","last_unstale":"2023-10-08T07:14:58.529408+0000","last_undegraded":"2023-10-08T07:14:58.529408+0000","last_fullsized":"2023-10-08T07:14:58.529408+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:53.166523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3f","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.547225+0000","last_change":"2023-10-08T07:14:58.545250+0000","last_active":"2023-10-08T07:14:59.547225+0000","last_peered":"2023-10-08T07:14:59.547225+0000","last_clean":"2023-10-08T07:14:59.547225+0000","last_became_active":"2023-10-08T07:14:58.544873+0000","last_became_peered":"2023-10-08T07:14:58.544873+0000","last_unstale":"2023-10-08T07:14:59.547225+0000","last_undegraded":"2023-10-08T07:14:59.547225+0000","last_fullsized":"2023-10-08T07:14:59.547225+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:02:15.292591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.532249+0000","last_change":"2023-10-08T07:14:44.329222+0000","last_active":"2023-10-08T07:14:58.532249+0000","last_peered":"2023-10-08T07:14:58.532249+0000","last_clean":"2023-10-08T07:14:58.532249+0000","last_became_active":"2023-10-08T07:14:44.328917+0000","last_became_peered":"2023-10-08T07:14:44.328917+0000","last_unstale":"2023-10-08T07:14:58.532249+0000","last_undegraded":"2023-10-08T07:14:58.532249+0000","last_fullsized":"2023-10-08T07:14:58.532249+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:35.827786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.524808+0000","last_change":"2023-10-08T07:14:43.411524+0000","last_active":"2023-10-08T07:14:58.524808+0000","last_peered":"2023-10-08T07:14:58.524808+0000","last_clean":"2023-10-08T07:14:58.524808+0000","last_became_active":"2023-10-08T07:14:43.411151+0000","last_became_peered":"2023-10-08T07:14:43.411151+0000","last_unstale":"2023-10-08T07:14:58.524808+0000","last_undegraded":"2023-10-08T07:14:58.524808+0000","last_fullsized":"2023-10-08T07:14:58.524808+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:38:49.845188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.567060+0000","last_change":"2023-10-08T07:14:58.552539+0000","last_active":"2023-10-08T07:14:59.567060+0000","last_peered":"2023-10-08T07:14:59.567060+0000","last_clean":"2023-10-08T07:14:59.567060+0000","last_became_active":"2023-10-08T07:14:58.552220+0000","last_became_peered":"2023-10-08T07:14:58.552220+0000","last_unstale":"2023-10-08T07:14:59.567060+0000","last_undegraded":"2023-10-08T07:14:59.567060+0000","last_fullsized":"2023-10-08T07:14:59.567060+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:43:47.118336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.536999+0000","last_change":"2023-10-08T07:14:44.335115+0000","last_active":"2023-10-08T07:14:58.536999+0000","last_peered":"2023-10-08T07:14:58.536999+0000","last_clean":"2023-10-08T07:14:58.536999+0000","last_became_active":"2023-10-08T07:14:44.334638+0000","last_became_peered":"2023-10-08T07:14:44.334638+0000","last_unstale":"2023-10-08T07:14:58.536999+0000","last_undegraded":"2023-10-08T07:14:58.536999+0000","last_fullsized":"2023-10-08T07:14:58.536999+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:36.291831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.538555+0000","last_change":"2023-10-08T07:14:43.408103+0000","last_active":"2023-10-08T07:14:58.538555+0000","last_peered":"2023-10-08T07:14:58.538555+0000","last_clean":"2023-10-08T07:14:58.538555+0000","last_became_active":"2023-10-08T07:14:43.407827+0000","last_became_peered":"2023-10-08T07:14:43.407827+0000","last_unstale":"2023-10-08T07:14:58.538555+0000","last_undegraded":"2023-10-08T07:14:58.538555+0000","last_fullsized":"2023-10-08T07:14:58.538555+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:50.115871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.569529+0000","last_change":"2023-10-08T07:14:58.542014+0000","last_active":"2023-10-08T07:14:59.569529+0000","last_peered":"2023-10-08T07:14:59.569529+0000","last_clean":"2023-10-08T07:14:59.569529+0000","last_became_active":"2023-10-08T07:14:58.541719+0000","last_became_peered":"2023-10-08T07:14:58.541719+0000","last_unstale":"2023-10-08T07:14:59.569529+0000","last_undegraded":"2023-10-08T07:14:59.569529+0000","last_fullsized":"2023-10-08T07:14:59.569529+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:10:22.724101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.530564+0000","last_change":"2023-10-08T07:14:44.334559+0000","last_active":"2023-10-08T07:14:58.530564+0000","last_peered":"2023-10-08T07:14:58.530564+0000","last_clean":"2023-10-08T07:14:58.530564+0000","last_became_active":"2023-10-08T07:14:44.334019+0000","last_became_peered":"2023-10-08T07:14:44.334019+0000","last_unstale":"2023-10-08T07:14:58.530564+0000","last_undegraded":"2023-10-08T07:14:58.530564+0000","last_fullsized":"2023-10-08T07:14:58.530564+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:59.631069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":16,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.522696+0000","last_change":"2023-10-08T07:14:43.413981+0000","last_active":"2023-10-08T07:14:58.522696+0000","last_peered":"2023-10-08T07:14:58.522696+0000","last_clean":"2023-10-08T07:14:58.522696+0000","last_became_active":"2023-10-08T07:14:43.413671+0000","last_became_peered":"2023-10-08T07:14:43.413671+0000","last_unstale":"2023-10-08T07:14:58.522696+0000","last_undegraded":"2023-10-08T07:14:58.522696+0000","last_fullsized":"2023-10-08T07:14:58.522696+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:35.209529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":11,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-10-08T07:14:59.566979+0000","last_change":"2023-10-08T07:14:58.551380+0000","last_active":"2023-10-08T07:14:59.566979+0000","last_peered":"2023-10-08T07:14:59.566979+0000","last_clean":"2023-10-08T07:14:59.566979+0000","last_became_active":"2023-10-08T07:14:58.550620+0000","last_became_peered":"2023-10-08T07:14:58.550620+0000","last_unstale":"2023-10-08T07:14:59.566979+0000","last_undegraded":"2023-10-08T07:14:59.566979+0000","last_fullsized":"2023-10-08T07:14:59.566979+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":256,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:57.475775+0000","last_clean_scrub_stamp":"2023-10-08T07:14:57.475775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:15.253247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":15,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.531430+0000","last_change":"2023-10-08T07:14:44.336800+0000","last_active":"2023-10-08T07:14:58.531430+0000","last_peered":"2023-10-08T07:14:58.531430+0000","last_clean":"2023-10-08T07:14:58.531430+0000","last_became_active":"2023-10-08T07:14:44.331013+0000","last_became_peered":"2023-10-08T07:14:44.331013+0000","last_unstale":"2023-10-08T07:14:58.531430+0000","last_undegraded":"2023-10-08T07:14:58.531430+0000","last_fullsized":"2023-10-08T07:14:58.531430+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:43.361570+0000","last_clean_scrub_stamp":"2023-10-08T07:14:43.361570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:39.261986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"253'1","reported_seq":17,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.536043+0000","last_change":"2023-10-08T07:14:43.410107+0000","last_active":"2023-10-08T07:14:58.536043+0000","last_peered":"2023-10-08T07:14:58.536043+0000","last_clean":"2023-10-08T07:14:58.536043+0000","last_became_active":"2023-10-08T07:14:43.409781+0000","last_became_peered":"2023-10-08T07:14:43.409781+0000","last_unstale":"2023-10-08T07:14:58.536043+0000","last_undegraded":"2023-10-08T07:14:58.536043+0000","last_fullsized":"2023-10-08T07:14:58.536043+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:14:42.343058+0000","last_clean_scrub_stamp":"2023-10-08T07:14:42.343058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:08:03.181094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"240'2828","reported_seq":3821,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-10-08T07:14:58.543364+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:14:58.543364+0000","last_peered":"2023-10-08T07:14:58.543364+0000","last_clean":"2023-10-08T07:14:58.543364+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:14:58.543364+0000","last_undegraded":"2023-10-08T07:14:58.543364+0000","last_fullsized":"2023-10-08T07:14:58.543364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":2828,"log_dups_size":0,"ondisk_log_size":2828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":629,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":3655,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2397,"num_read_kb":5030,"num_write":629,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2828,"ondisk_log_size":2828,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576025,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48844,"kb_used_data":3768,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94322996,"statfs":{"total":96636764160,"available":96586747904,"internally_reserved":0,"allocated":3858432,"data_stored":2289942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.127,"15min":1.1910000000000001},"min":{"1min":0.81699999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.968,"15min":2.008},"last":1.006},{"interface":"front","average":{"1min":1.119,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.81999999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.516,"5min":1.788,"15min":1.788},"last":1.399}]},{"osd":1,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.2370000000000001,"15min":1.2509999999999999},"min":{"1min":0.93899999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.998},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.196,"5min":1.2969999999999999,"15min":1.329},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6299999999999999,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.089}]},{"osd":2,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2370000000000001,"15min":1.2929999999999999},"min":{"1min":0.89500000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":2.1640000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.369,"15min":1.3500000000000001},"min":{"1min":0.93400000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.901,"5min":1.901,"15min":2.3170000000000002},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.298,"15min":1.3340000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.401,"5min":1.3819999999999999,"15min":1.3999999999999999},"min":{"1min":0.97099999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.149}]},{"osd":4,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.83999999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9319999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4139999999999999,"15min":1.411},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.365}]},{"osd":5,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.478,"15min":1.4490000000000001},"min":{"1min":1.127,"5min":0.80600000000000005,"15min":0.79300000000000004},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.488,"15min":1.4950000000000001},"min":{"1min":1.012,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.056,"5min":2.23,"15min":2.23},"last":1.3500000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.599,"15min":1.597},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.202,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.462},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.5640000000000001,"15min":1.5429999999999999},"min":{"1min":1.21,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2570000000000001},"last":1.2090000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576024,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30212,"kb_used_data":3184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341628,"statfs":{"total":96636764160,"available":96605827072,"internally_reserved":0,"allocated":3260416,"data_stored":1698992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.286,"15min":1.3089999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.637,"5min":1.794,"15min":1.962},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.3120000000000001,"15min":1.3520000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6220000000000001,"5min":1.76,"15min":2.0070000000000001},"last":1.3520000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.2030000000000001,"15min":1.2270000000000001},"min":{"1min":0.71999999999999997,"5min":0.625,"15min":0.625},"max":{"1min":1.7330000000000001,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.222,"15min":1.2430000000000001},"min":{"1min":0.71799999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.262}]},{"osd":2,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.1599999999999999,"15min":1.2310000000000001},"min":{"1min":0.86899999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":2.339},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2749999999999999,"15min":1.306},"min":{"1min":0.77100000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.627,"5min":1.7629999999999999,"15min":2.1779999999999999},"last":1.032}]},{"osd":3,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.367,"15min":1.397},"min":{"1min":0.86299999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8280000000000001,"5min":1.9319999999999999,"15min":1.95},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.429,"15min":1.4890000000000001},"min":{"1min":0.95199999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"max":{"1min":1.7789999999999999,"5min":1.8959999999999999,"15min":2.1459999999999999},"last":1.5629999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3720000000000001,"15min":1.389},"min":{"1min":0.877,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.849,"5min":1.9359999999999999,"15min":2.0859999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.4550000000000001,"15min":1.4570000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":1.9650000000000001,"5min":2.1110000000000002,"15min":2.484},"last":1.474}]},{"osd":5,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.389,"15min":1.4279999999999999},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"max":{"1min":1.8069999999999999,"5min":1.8660000000000001,"15min":2.403},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.5489999999999999,"15min":1.5549999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":1.653}]},{"osd":7,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5369999999999999,"15min":1.5760000000000001},"min":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.339,"5min":2.339,"15min":2.5870000000000002},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.552,"15min":1.5620000000000001},"min":{"1min":0.99299999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.0710000000000002,"5min":2.2959999999999998,"15min":2.665},"last":1.6080000000000001}]}]},{"osd":5,"up_from":13,"seq":55834576024,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60724,"kb_used_data":3176,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311116,"statfs":{"total":96636764160,"available":96574582784,"internally_reserved":0,"allocated":3252224,"data_stored":1698916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.161,"15min":1.2070000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1930000000000001,"15min":1.2230000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2480000000000002,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.90300000000000002}]},{"osd":1,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1970000000000001,"15min":1.2370000000000001},"min":{"1min":0.66800000000000004,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.339,"15min":1.3120000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":7.5250000000000004,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":1.2350000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.266,"15min":1.3009999999999999},"min":{"1min":0.68899999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.008,"5min":2.008,"15min":2.0870000000000002},"last":1.468},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2450000000000001,"15min":1.3009999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.645,"5min":1.877,"15min":1.909},"last":0.77600000000000002}]},{"osd":3,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.4159999999999999,"15min":1.391},"min":{"1min":0.72499999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":6.8449999999999998,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.425,"15min":1.4370000000000001},"min":{"1min":0.64900000000000002,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":9.3599999999999994,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":1.048}]},{"osd":4,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.319,"15min":1.3280000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4319999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.742,"5min":2.742,"15min":2.742},"last":1.4239999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.978,"5min":1.5329999999999999,"15min":1.486},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":13.234999999999999,"5min":13.234999999999999,"15min":13.234999999999999},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.6930000000000001,"5min":1.5129999999999999,"15min":1.482},"min":{"1min":0.67800000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":7.2949999999999999,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":1.1830000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:14:23 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.536,"15min":1.5449999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.4430000000000001,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.5700000000000001,"15min":1.601},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.351,"5min":2.351,"15min":2.351},"last":1.5720000000000001}]}]},{"osd":4,"up_from":13,"seq":55834576024,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48256,"kb_used_data":3188,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323584,"statfs":{"total":96636764160,"available":96587350016,"internally_reserved":0,"allocated":3264512,"data_stored":1700528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.323,"15min":1.3520000000000001},"min":{"1min":0.66800000000000004,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":5.4800000000000004,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.522},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.421,"15min":1.4219999999999999},"min":{"1min":0.79200000000000004,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":7.0880000000000001,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.482}]},{"osd":1,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2410000000000001,"15min":1.2849999999999999},"min":{"1min":0.72599999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.7689999999999999,"5min":1.3129999999999999,"15min":1.2729999999999999},"min":{"1min":0.61299999999999999,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":7.673,"5min":7.673,"15min":7.673},"last":1.3640000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":2.2629999999999999,"5min":1.4690000000000001,"15min":1.3260000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":17.175999999999998,"5min":17.175999999999998,"15min":17.175999999999998},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.651,"5min":1.4139999999999999,"15min":1.3859999999999999},"min":{"1min":0.629,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":7.1849999999999996,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":1.212}]},{"osd":3,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.7450000000000001,"5min":1.3600000000000001,"15min":1.347},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":7.5270000000000001,"5min":7.5270000000000001,"15min":7.5270000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.5209999999999999,"15min":1.4430000000000001},"min":{"1min":0.59199999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":7.7599999999999998,"5min":7.7599999999999998,"15min":7.7599999999999998},"last":1.268}]},{"osd":5,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.55,"15min":1.419},"min":{"1min":0.39100000000000001,"5min":0.372,"15min":0.372},"max":{"1min":7.4020000000000001,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.149},{"interface":"front","average":{"1min":1.754,"5min":1.4970000000000001,"15min":1.415},"min":{"1min":0.442,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":7.79,"5min":7.79,"15min":7.79},"last":1.103}]},{"osd":6,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":2.0859999999999999,"5min":1.621,"15min":1.5},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":10.542999999999999,"5min":10.542999999999999,"15min":10.542999999999999},"last":1.395},{"interface":"front","average":{"1min":2.1819999999999999,"5min":1.7150000000000001,"15min":1.605},"min":{"1min":0.86099999999999999,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":10.433999999999999,"5min":10.433999999999999,"15min":10.433999999999999},"last":1.3320000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:14:24 2023","interfaces":[{"interface":"back","average":{"1min":1.8899999999999999,"5min":1.6259999999999999,"15min":1.5409999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":7.915,"5min":7.915,"15min":7.915},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.8839999999999999,"5min":1.73,"15min":1.6559999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":1.5820000000000001}]}]},{"osd":3,"up_from":13,"seq":55834576026,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56596,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315244,"statfs":{"total":96636764160,"available":96578809856,"internally_reserved":0,"allocated":3276800,"data_stored":1702431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92700000000000005,"15min":0.97399999999999998},"min":{"1min":0.32200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1990000000000001,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":1.323},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.131,"15min":1.1659999999999999},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5349999999999999,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.622}]},{"osd":1,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.96799999999999997,"15min":1.022},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3560000000000001,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.054,"5min":1.1890000000000001,"15min":1.2729999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.667,"5min":2.109,"15min":2.4889999999999999},"last":1.0169999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0629999999999999,"15min":1.1399999999999999},"min":{"1min":0.502,"5min":0.437,"15min":0.437},"max":{"1min":1.462,"5min":2.165,"15min":2.2250000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.139,"15min":1.1899999999999999},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.671,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.76300000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2290000000000001,"15min":1.272},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.67500000000000004},"max":{"1min":1.7290000000000001,"5min":1.857,"15min":2.5379999999999998},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.306,"5min":1.381,"15min":1.45},"min":{"1min":0.91600000000000004,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.8149999999999999,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.2430000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.294,"15min":1.3660000000000001},"min":{"1min":0.76700000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7829999999999999,"5min":1.923,"15min":2.9740000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3919999999999999,"15min":1.4330000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9610000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":1.413}]},{"osd":6,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.82299999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.9690000000000001,"5min":1.976,"15min":2.653},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.4350000000000001,"15min":1.5129999999999999},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.0150000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.306}]},{"osd":7,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.512,"15min":1.5580000000000001},"min":{"1min":0.89100000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.3220000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4490000000000001,"15min":1.512},"min":{"1min":0.80900000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.903,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.177}]}]},{"osd":2,"up_from":13,"seq":55834576024,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52436,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319404,"statfs":{"total":96636764160,"available":96583069696,"internally_reserved":0,"allocated":3276800,"data_stored":1701714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.91600000000000004,"15min":0.97199999999999998},"min":{"1min":0.49199999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.601,"5min":1.601,"15min":1.948},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":1.008,"15min":1.0660000000000001},"min":{"1min":0.55700000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.423,"5min":1.7410000000000001,"15min":1.7709999999999999},"last":0.94499999999999995}]},{"osd":1,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.93400000000000005,"15min":0.98999999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.413,"5min":1.6279999999999999,"15min":2.044},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.1479999999999999,"15min":1.1890000000000001},"min":{"1min":0.61499999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":2.1309999999999998},"last":0.57399999999999995}]},{"osd":3,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.141,"15min":1.177},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.786,"5min":1.786,"15min":1.8420000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.236,"15min":1.272},"min":{"1min":0.745,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.663,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":0.68799999999999994}]},{"osd":4,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.391,"15min":1.4379999999999999},"min":{"1min":0.92900000000000005,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.992,"5min":1.992,"15min":2.1789999999999998},"last":0.748},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.304,"15min":1.3120000000000001},"min":{"1min":0.87,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.843,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.88400000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3600000000000001,"15min":1.4019999999999999},"min":{"1min":0.89200000000000002,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.931,"5min":2.1080000000000001,"15min":2.3420000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.4930000000000001,"15min":1.532},"min":{"1min":1.1060000000000001,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":1.837,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":0.82399999999999995}]},{"osd":6,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.361,"15min":1.395},"min":{"1min":1.0549999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.722,"5min":2.056,"15min":2.1240000000000001},"last":1.073},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.4830000000000001,"15min":1.5249999999999999},"min":{"1min":0.99299999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"max":{"1min":1.994,"5min":2.0350000000000001,"15min":2.3439999999999999},"last":0.84399999999999997}]},{"osd":7,"last update":"Sun Oct 8 07:14:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.446,"15min":1.4790000000000001},"min":{"1min":1.0800000000000001,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.7150000000000001,"5min":1.9850000000000001,"15min":2.4689999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.51,"15min":1.577},"min":{"1min":1.028,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":2.056,"5min":2.1059999999999999,"15min":2.3879999999999999},"last":0.92800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834576024,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52416,"kb_used_data":3180,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319424,"statfs":{"total":96636764160,"available":96583090176,"internally_reserved":0,"allocated":3256320,"data_stored":1698970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.97699999999999998,"15min":0.999},"min":{"1min":0.56000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.502,"5min":1.6579999999999999,"15min":1.708},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0760000000000001,"15min":1.2},"min":{"1min":0.52900000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4790000000000001,"5min":1.7010000000000001,"15min":1.79},"last":1.365}]},{"osd":2,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.876,"15min":0.94999999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.095,"5min":1.641,"15min":1.641},"last":0.753},{"interface":"front","average":{"1min":1.026,"5min":1.087,"15min":1.1419999999999999},"min":{"1min":0.42599999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.607,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.96099999999999997}]},{"osd":3,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1519999999999999,"15min":1.2390000000000001},"min":{"1min":0.58599999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5960000000000001,"5min":2.496,"15min":2.496},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":1.099,"15min":1.143},"min":{"1min":0.49399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.5369999999999999,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":0.97999999999999998}]},{"osd":4,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.276,"15min":1.351},"min":{"1min":0.77700000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.579,"5min":2.73,"15min":2.8460000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.105,"5min":1.248,"15min":1.3009999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.5409999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.0209999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.302,"15min":1.3640000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.476,"15min":1.5189999999999999},"min":{"1min":0.89600000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7070000000000001,"5min":2.601,"15min":2.601},"last":1.3129999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4330000000000001,"15min":1.502},"min":{"1min":0.92600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.7609999999999999,"5min":2.883,"15min":2.9079999999999999},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.411,"15min":1.484},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.6180000000000001,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.575}]},{"osd":7,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4830000000000001,"15min":1.5389999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.7749999999999999,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.552},{"interface":"front","average":{"1min":1.339,"5min":1.484,"15min":1.5569999999999999},"min":{"1min":0.83799999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.873,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.621}]}]},{"osd":0,"up_from":13,"seq":55834576025,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61316,"kb_used_data":3768,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310524,"statfs":{"total":96636764160,"available":96573976576,"internally_reserved":0,"allocated":3858432,"data_stored":2292141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":58915273},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.92000000000000004,"15min":0.96499999999999997},"min":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.321,"5min":1.581,"15min":1.581},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":1.016,"15min":1.0840000000000001},"min":{"1min":0.48899999999999999,"5min":0.371,"15min":0.371},"max":{"1min":1.4039999999999999,"5min":1.621,"15min":1.726},"last":0.95699999999999996}]},{"osd":2,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.081,"15min":1.1379999999999999},"min":{"1min":0.52000000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.0189999999999999,"15min":1.071},"min":{"1min":0.51300000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4690000000000001,"5min":1.6919999999999999,"15min":1.9930000000000001},"last":1.147}]},{"osd":3,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1859999999999999,"15min":1.2450000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5600000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.141,"15min":1.165},"min":{"1min":0.55100000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":2.0979999999999999},"last":1.0960000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3080000000000001,"15min":1.3400000000000001},"min":{"1min":0.746,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6359999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.361,"15min":1.4079999999999999},"min":{"1min":0.79900000000000004,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.181},"last":1.504}]},{"osd":5,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.3939999999999999,"15min":1.4159999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.802,"5min":1.919,"15min":2.1309999999999998},"last":1.579},{"interface":"front","average":{"1min":1.466,"5min":1.47,"15min":1.49},"min":{"1min":0.81999999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.23,"5min":2.23,"15min":2.4900000000000002},"last":1.677}]},{"osd":6,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4359999999999999,"15min":1.4770000000000001},"min":{"1min":0.94999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.0150000000000001,"5min":2.21,"15min":2.2149999999999999},"last":1.877},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.462,"15min":1.4990000000000001},"min":{"1min":0.78200000000000003,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7529999999999999,"5min":2.0880000000000001,"15min":2.355},"last":1.46}]},{"osd":7,"last update":"Sun Oct 8 07:14:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.5309999999999999,"15min":1.542},"min":{"1min":0.95999999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.9370000000000001,"5min":2.1989999999999998,"15min":2.2749999999999999},"last":1.758},{"interface":"front","average":{"1min":1.448,"5min":1.518,"15min":1.5529999999999999},"min":{"1min":0.94599999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.0830000000000002,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.9690000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:15:03.563 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T07:15:03.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.811+0000 7ff13987c640 1 Processor -- start 2023-10-08T07:15:03.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.811+0000 7ff13987c640 1 -- start start 2023-10-08T07:15:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.813+0000 7ff13987c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff134159ac0 0x7ff13406d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.813+0000 7ff13987c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13406dee0 con 0x7ff1340709e0 2023-10-08T07:15:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.813+0000 7ff13987c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13415d850 con 0x7ff1340ad790 2023-10-08T07:15:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.813+0000 7ff13987c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415db30 con 0x7ff134159ac0 2023-10-08T07:15:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.813+0000 7ff1337fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff1340709e0 0x7ff1340c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47478/0 (socket says 172.21.15.70:47478) 2023-10-08T07:15:03.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.813+0000 7ff1337fe640 1 -- 172.21.15.70:0/1745639139 learned_addr learned my addr 172.21.15.70:0/1745639139 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff132ffd640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff134159ac0 0x7ff13406d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287050450 0 0) 0x7ff13406dee0 con 0x7ff1340709e0 2023-10-08T07:15:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff11c005730 con 0x7ff1340709e0 2023-10-08T07:15:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2873514142 0 0) 0x7ff11c005730 con 0x7ff1340709e0 2023-10-08T07:15:03.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff134159ac0 msgr2=0x7ff13406d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff134159ac0 0x7ff13406d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 >> v1:172.21.15.181:6789/0 conn(0x7ff1340ad790 legacy=0x7ff1340702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13415de10 con 0x7ff1340709e0 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff12c0019f0 con 0x7ff1340709e0 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 >> v1:172.21.15.70:6789/0 conn(0x7ff1340709e0 legacy=0x7ff1340c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 0x7ff11c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff11c0069d0 0x7ff11c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff134159ac0 0x7ff11c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415db30 con 0x7ff11c005bb0 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c00dc40 con 0x7ff11c0069d0 2023-10-08T07:15:03.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.814+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c00de00 con 0x7ff134159ac0 2023-10-08T07:15:03.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.815+0000 7ff132ffd640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 0x7ff11c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.815+0000 7ff132ffd640 1 -- 172.21.15.70:0/1745639139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff134159ac0 msgr2=0x7ff11c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:03.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.815+0000 7ff132ffd640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff134159ac0 0x7ff11c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.815+0000 7ff132ffd640 1 -- 172.21.15.70:0/1745639139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff11c0069d0 msgr2=0x7ff11c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:03.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.815+0000 7ff1327fc640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff11c0069d0 0x7ff11c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.815+0000 7ff132ffd640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff11c0069d0 0x7ff11c006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.815+0000 7ff132ffd640 1 -- 172.21.15.70:0/1745639139 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c005730 con 0x7ff11c005bb0 2023-10-08T07:15:03.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.816+0000 7ff132ffd640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 0x7ff11c005fd0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7ff124009a90 tx=0x7ff11c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3305037e50751b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.816+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12402c6b0 con 0x7ff11c005bb0 2023-10-08T07:15:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.816+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12402c810 con 0x7ff11c005bb0 2023-10-08T07:15:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.816+0000 7ff131ffb640 1 -- 172.21.15.70:0/1745639139 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12402cba0 con 0x7ff11c005bb0 2023-10-08T07:15:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.817+0000 7ff13987c640 1 -- 172.21.15.70:0/1745639139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 msgr2=0x7ff11c005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.817+0000 7ff13987c640 1 --2- 172.21.15.70:0/1745639139 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 0x7ff11c005fd0 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7ff124009a90 tx=0x7ff11c007a00 comp rx=0 tx=0).stop 2023-10-08T07:15:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.817+0000 7ff1337fe640 1 -- 172.21.15.70:0/1745639139 reap_dead start 2023-10-08T07:15:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.817+0000 7ff13987c640 1 -- 172.21.15.70:0/1745639139 shutdown_connections 2023-10-08T07:15:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.817+0000 7ff13987c640 1 -- 172.21.15.70:0/1745639139 >> 172.21.15.70:0/1745639139 conn(0x7ff13406a380 msgr2=0x7ff1340b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.817+0000 7ff13987c640 1 -- 172.21.15.70:0/1745639139 shutdown_connections 2023-10-08T07:15:03.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.817+0000 7ff13987c640 1 -- 172.21.15.70:0/1745639139 wait complete. 2023-10-08T07:15:03.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.819+0000 7ff13987c640 1 Processor -- start 2023-10-08T07:15:03.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.819+0000 7ff13987c640 1 -- start start 2023-10-08T07:15:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff13987c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 0x7ff13415aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff13987c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1341492c0 0x7ff13415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff13987c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff13415cd00 0x7ff134154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff13987c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415c330 con 0x7ff11c005bb0 2023-10-08T07:15:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff13987c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415b720 con 0x7ff13415cd00 2023-10-08T07:15:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff13987c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415ba00 con 0x7ff1341492c0 2023-10-08T07:15:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff132ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 0x7ff13415aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff132ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 0x7ff13415aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51114/0 (socket says 172.21.15.70:51114) 2023-10-08T07:15:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff132ffd640 1 -- 172.21.15.70:0/3755297898 learned_addr learned my addr 172.21.15.70:0/3755297898 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff132ffd640 1 -- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1341492c0 msgr2=0x7ff13415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff132ffd640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1341492c0 0x7ff13415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff132ffd640 1 -- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff13415cd00 msgr2=0x7ff134154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff132ffd640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff13415cd00 0x7ff134154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.820+0000 7ff132ffd640 1 -- 172.21.15.70:0/3755297898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13415bce0 con 0x7ff11c005bb0 2023-10-08T07:15:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.821+0000 7ff132ffd640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 0x7ff13415aa30 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7ff12402c250 tx=0x7ff124004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.821+0000 7ff11bfff640 1 -- 172.21.15.70:0/3755297898 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124003810 con 0x7ff11c005bb0 2023-10-08T07:15:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.821+0000 7ff11bfff640 1 -- 172.21.15.70:0/3755297898 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff124003970 con 0x7ff11c005bb0 2023-10-08T07:15:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.821+0000 7ff11bfff640 1 -- 172.21.15.70:0/3755297898 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124003ce0 con 0x7ff11c005bb0 2023-10-08T07:15:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.821+0000 7ff13987c640 1 -- 172.21.15.70:0/3755297898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13415bfc0 con 0x7ff11c005bb0 2023-10-08T07:15:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.821+0000 7ff13987c640 1 -- 172.21.15.70:0/3755297898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13415b3b0 con 0x7ff11c005bb0 2023-10-08T07:15:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.822+0000 7ff13987c640 1 -- 172.21.15.70:0/3755297898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff134067490 con 0x7ff11c005bb0 2023-10-08T07:15:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.823+0000 7ff11bfff640 1 -- 172.21.15.70:0/3755297898 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7ff124040450 con 0x7ff11c005bb0 2023-10-08T07:15:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.824+0000 7ff11bfff640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff12006b0b0 0x7ff12006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.824+0000 7ff11bfff640 1 -- 172.21.15.70:0/3755297898 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7ff1240abcf0 con 0x7ff11c005bb0 2023-10-08T07:15:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.825+0000 7ff1327fc640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff12006b0b0 0x7ff12006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.826+0000 7ff1327fc640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff12006b0b0 0x7ff12006d570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff134070800 tx=0x7ff128009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:03.828+0000 7ff11bfff640 1 -- 172.21.15.70:0/3755297898 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff124076520 con 0x7ff11c005bb0 2023-10-08T07:15:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.037+0000 7ff13987c640 1 -- 172.21.15.70:0/3755297898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1340b4990 con 0x7ff11c005bb0 2023-10-08T07:15:04.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.038+0000 7ff11bfff640 1 -- 172.21.15.70:0/3755297898 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+6149 (secure 0 0 0) 0x7ff1240044e0 con 0x7ff11c005bb0 2023-10-08T07:15:04.039 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:04.039 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":130,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:43.833416+0000","modified":"2023-10-08T07:14:46.899347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11904},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11904":{"gid":11904,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/3856326680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3856326680},{"type":"v1","addr":"172.21.15.70:6834","nonce":3856326680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:57.955428+0000","modified":"2023-10-08T07:15:01.029708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8875":{"gid":8875,"name":"b","rank":0,"incarnation":127,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-08T07:15:04.039 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 130 2023-10-08T07:15:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.040+0000 7ff13987c640 1 -- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff12006b0b0 msgr2=0x7ff12006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.040+0000 7ff13987c640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff12006b0b0 0x7ff12006d570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff134070800 tx=0x7ff128009040 comp rx=0 tx=0).stop 2023-10-08T07:15:04.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.040+0000 7ff13987c640 1 -- 172.21.15.70:0/3755297898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 msgr2=0x7ff13415aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.040+0000 7ff13987c640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 0x7ff13415aa30 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7ff12402c250 tx=0x7ff124004730 comp rx=0 tx=0).stop 2023-10-08T07:15:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.040+0000 7ff13987c640 1 -- 172.21.15.70:0/3755297898 shutdown_connections 2023-10-08T07:15:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.040+0000 7ff13987c640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7ff12006b0b0 0x7ff12006d570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.040+0000 7ff13987c640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff13415cd00 0x7ff134154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.040+0000 7ff13987c640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1341492c0 0x7ff13415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.040+0000 7ff13987c640 1 --2- 172.21.15.70:0/3755297898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff11c005bb0 0x7ff13415aa30 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.041+0000 7ff13987c640 1 -- 172.21.15.70:0/3755297898 >> 172.21.15.70:0/3755297898 conn(0x7ff13406a380 msgr2=0x7ff1340c4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.041+0000 7ff13987c640 1 -- 172.21.15.70:0/3755297898 shutdown_connections 2023-10-08T07:15:04.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.041+0000 7ff13987c640 1 -- 172.21.15.70:0/3755297898 wait complete. 2023-10-08T07:15:04.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 130, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T07:14:57.955428+0000', 'modified': '2023-10-08T07:15:01.029708+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8875}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8875': {'gid': 8875, 'name': 'b', 'rank': 0, 'incarnation': 127, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.181:6837/80073995', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6836', 'nonce': 80073995}, {'type': 'v1', 'addr': '172.21.15.181:6837', 'nonce': 80073995}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T07:15:04.461 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T07:15:04.462 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T07:15:04.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.686+0000 7fafcee7c640 1 Processor -- start 2023-10-08T07:15:04.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.687+0000 7fafcee7c640 1 -- start start 2023-10-08T07:15:04.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.688+0000 7fafcee7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fafc80702b0 0x7fafc8070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.688+0000 7fafcee7c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc806c3a0 con 0x7fafc80ca370 2023-10-08T07:15:04.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.688+0000 7fafcee7c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc8070bd0 con 0x7fafc80b21f0 2023-10-08T07:15:04.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.688+0000 7fafcee7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc806c640 con 0x7fafc80702b0 2023-10-08T07:15:04.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.688+0000 7fafccc18640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fafc80702b0 0x7fafc8070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.688+0000 7fafcd419640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fafc80ca370 0x7fafc806d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47492/0 (socket says 172.21.15.70:47492) 2023-10-08T07:15:04.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.688+0000 7fafcd419640 1 -- 172.21.15.70:0/772687151 learned_addr learned my addr 172.21.15.70:0/772687151 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:04.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.689+0000 7fafccc18640 1 -- 172.21.15.70:0/772687151 >> v1:172.21.15.181:6789/0 conn(0x7fafc80b21f0 legacy=0x7fafc80c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.689+0000 7fafccc18640 1 -- 172.21.15.70:0/772687151 >> v1:172.21.15.70:6789/0 conn(0x7fafc80ca370 legacy=0x7fafc806d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.689+0000 7fafccc18640 1 -- 172.21.15.70:0/772687151 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc806c920 con 0x7fafc80702b0 2023-10-08T07:15:04.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.690+0000 7fafccc18640 1 --2- 172.21.15.70:0/772687151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80702b0 0x7fafc8070690 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fafb80097a0 tx=0x7fafb802da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e8127bc28d91a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:04.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.690+0000 7fafc77fe640 1 -- 172.21.15.70:0/772687151 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8031870 con 0x7fafc80702b0 2023-10-08T07:15:04.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.690+0000 7fafc77fe640 1 -- 172.21.15.70:0/772687151 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb80319d0 con 0x7fafc80702b0 2023-10-08T07:15:04.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.690+0000 7fafc77fe640 1 -- 172.21.15.70:0/772687151 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8031d60 con 0x7fafc80702b0 2023-10-08T07:15:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.691+0000 7fafcee7c640 1 -- 172.21.15.70:0/772687151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80702b0 msgr2=0x7fafc8070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.691+0000 7fafcee7c640 1 --2- 172.21.15.70:0/772687151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80702b0 0x7fafc8070690 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7fafb80097a0 tx=0x7fafb802da50 comp rx=0 tx=0).stop 2023-10-08T07:15:04.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.691+0000 7fafcee7c640 1 -- 172.21.15.70:0/772687151 shutdown_connections 2023-10-08T07:15:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.691+0000 7fafcee7c640 1 --2- 172.21.15.70:0/772687151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80702b0 0x7fafc8070690 unknown :-1 s=CLOSED pgs=2708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.691+0000 7fafcee7c640 1 -- 172.21.15.70:0/772687151 >> 172.21.15.70:0/772687151 conn(0x7fafc806a5d0 msgr2=0x7fafc80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.691+0000 7fafcee7c640 1 -- 172.21.15.70:0/772687151 shutdown_connections 2023-10-08T07:15:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.691+0000 7fafcee7c640 1 -- 172.21.15.70:0/772687151 wait complete. 2023-10-08T07:15:04.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.693+0000 7fafcee7c640 1 Processor -- start 2023-10-08T07:15:04.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.693+0000 7fafcee7c640 1 -- start start 2023-10-08T07:15:04.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.694+0000 7fafcee7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc80702b0 0x7fafc81e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.694+0000 7fafcee7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b21f0 0x7fafc81474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.694+0000 7fafcee7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafc80ca370 0x7fafc81587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.694+0000 7fafcee7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8147a00 con 0x7fafc80b21f0 2023-10-08T07:15:04.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.694+0000 7fafcee7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81e8190 con 0x7fafc80702b0 2023-10-08T07:15:04.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.694+0000 7fafcee7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc81e8470 con 0x7fafc80ca370 2023-10-08T07:15:04.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.694+0000 7fafc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b21f0 0x7fafc81474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.694+0000 7fafc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b21f0 0x7fafc81474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51146/0 (socket says 172.21.15.70:51146) 2023-10-08T07:15:04.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.694+0000 7fafc7fff640 1 -- 172.21.15.70:0/3490833939 learned_addr learned my addr 172.21.15.70:0/3490833939 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:04.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafccc18640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc80702b0 0x7fafc81e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafc7fff640 1 -- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafc80ca370 msgr2=0x7fafc81587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafcd419640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafc80ca370 0x7fafc81587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafc7fff640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafc80ca370 0x7fafc81587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafc7fff640 1 -- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc80702b0 msgr2=0x7fafc81e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafc7fff640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc80702b0 0x7fafc81e7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafc7fff640 1 -- 172.21.15.70:0/3490833939 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb8009400 con 0x7fafc80b21f0 2023-10-08T07:15:04.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafc7fff640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b21f0 0x7fafc81474c0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c7a0 tx=0x7fafb000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:04.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafc57fa640 1 -- 172.21.15.70:0/3490833939 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0003d80 con 0x7fafc80b21f0 2023-10-08T07:15:04.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafc57fa640 1 -- 172.21.15.70:0/3490833939 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0005070 con 0x7fafc80b21f0 2023-10-08T07:15:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafc57fa640 1 -- 172.21.15.70:0/3490833939 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0005460 con 0x7fafc80b21f0 2023-10-08T07:15:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.695+0000 7fafcee7c640 1 -- 172.21.15.70:0/3490833939 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc81e8750 con 0x7fafc80b21f0 2023-10-08T07:15:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.696+0000 7fafcee7c640 1 -- 172.21.15.70:0/3490833939 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc81e8d10 con 0x7fafc80b21f0 2023-10-08T07:15:04.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.698+0000 7fafc57fa640 1 -- 172.21.15.70:0/3490833939 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fafb00055c0 con 0x7fafc80b21f0 2023-10-08T07:15:04.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.698+0000 7fafc57fa640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fafa806b0b0 0x7fafa806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.699+0000 7fafc57fa640 1 -- 172.21.15.70:0/3490833939 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fafb00894a0 con 0x7fafc80b21f0 2023-10-08T07:15:04.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.699+0000 7fafccc18640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fafa806b0b0 0x7fafa806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.699+0000 7fafd1a14740 1 Processor -- start 2023-10-08T07:15:04.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.700+0000 7fafd1a14740 1 -- start start 2023-10-08T07:15:04.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.700+0000 7fafccc18640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fafa806b0b0 0x7fafa806d570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fafb8038000 tx=0x7fafb8008030 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:04.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.700+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e093b60 0x56248e066a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.700+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.700+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e066fa0 0x56248e096e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.700+0000 7fafd1a14740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248dfda700 con 0x56248e093f40 2023-10-08T07:15:04.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafd1a14740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248dfd7df0 con 0x56248e066fa0 2023-10-08T07:15:04.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafd1a14740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56248dfd36e0 con 0x56248e093b60 2023-10-08T07:15:04.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51162/0 (socket says 172.21.15.70:51162) 2023-10-08T07:15:04.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafc7fff640 1 -- 172.21.15.70:0/2656881220 learned_addr learned my addr 172.21.15.70:0/2656881220 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:04.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafc7fff640 1 -- 172.21.15.70:0/2656881220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e093b60 msgr2=0x56248e066a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafccc18640 1 --2- 172.21.15.70:0/2656881220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e093b60 0x56248e066a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafc7fff640 1 --2- 172.21.15.70:0/2656881220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e093b60 0x56248e066a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafc7fff640 1 -- 172.21.15.70:0/2656881220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e066fa0 msgr2=0x56248e096e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafc7fff640 1 --2- 172.21.15.70:0/2656881220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e066fa0 0x56248e096e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafc7fff640 1 -- 172.21.15.70:0/2656881220 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56248df97910 con 0x56248e093f40 2023-10-08T07:15:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.701+0000 7fafc7fff640 1 --2- 172.21.15.70:0/2656881220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0968f0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fafb005f6c0 tx=0x7fafb0057ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b26ef3a05e6895 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.702+0000 7fafbe7fc640 1 -- 172.21.15.70:0/2656881220 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb006f070 con 0x56248e093f40 2023-10-08T07:15:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.702+0000 7fafbe7fc640 1 -- 172.21.15.70:0/2656881220 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb006f1d0 con 0x56248e093f40 2023-10-08T07:15:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.702+0000 7fafbe7fc640 1 -- 172.21.15.70:0/2656881220 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb005ba40 con 0x56248e093f40 2023-10-08T07:15:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.702+0000 7fafd1a14740 1 -- 172.21.15.70:0/2656881220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 msgr2=0x56248e0968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.702+0000 7fafd1a14740 1 --2- 172.21.15.70:0/2656881220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0968f0 secure :-1 s=READY pgs=2710 cs=0 l=1 rev1=1 crypto rx=0x7fafb005f6c0 tx=0x7fafb0057ff0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.702+0000 7fafd1a14740 1 -- 172.21.15.70:0/2656881220 shutdown_connections 2023-10-08T07:15:04.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.702+0000 7fafd1a14740 1 --2- 172.21.15.70:0/2656881220 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e066fa0 0x56248e096e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.702+0000 7fafd1a14740 1 --2- 172.21.15.70:0/2656881220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0968f0 unknown :-1 s=CLOSED pgs=2710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.702+0000 7fafd1a14740 1 --2- 172.21.15.70:0/2656881220 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e093b60 0x56248e066a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.702+0000 7fafd1a14740 1 -- 172.21.15.70:0/2656881220 >> 172.21.15.70:0/2656881220 conn(0x56248e068d70 msgr2=0x56248e0b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:04.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.703+0000 7fafd1a14740 1 -- 172.21.15.70:0/2656881220 shutdown_connections 2023-10-08T07:15:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.703+0000 7fafd1a14740 1 -- 172.21.15.70:0/2656881220 wait complete. 2023-10-08T07:15:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.703+0000 7fafd1a14740 1 -- start start 2023-10-08T07:15:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.703+0000 7fafd1a14740 1 Processor -- start 2023-10-08T07:15:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.704+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e093b60 0x56248e07d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.704+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.704+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e07f570 0x56248e0943a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.704+0000 7fafd1a14740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248df71970 con 0x56248e093f40 2023-10-08T07:15:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.704+0000 7fafd1a14740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248df71610 con 0x56248e07f570 2023-10-08T07:15:04.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.704+0000 7fafd1a14740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56248dd37f20 con 0x56248e093b60 2023-10-08T07:15:04.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafd1a14740 10 client.0 _ll_register_callbacks cb 0x56248dfaf610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:15:04.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51166/0 (socket says 172.21.15.70:51166) 2023-10-08T07:15:04.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafc7fff640 1 -- 172.21.15.70:0/2110537715 learned_addr learned my addr 172.21.15.70:0/2110537715 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:04.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafc7fff640 1 -- 172.21.15.70:0/2110537715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e093b60 msgr2=0x56248e07d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafc7fff640 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e093b60 0x56248e07d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafc7fff640 1 -- 172.21.15.70:0/2110537715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e07f570 msgr2=0x56248e0943a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafc7fff640 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e07f570 0x56248e0943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafc7fff640 1 -- 172.21.15.70:0/2110537715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc81e8d10 con 0x56248e093f40 2023-10-08T07:15:04.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafc7fff640 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0978e0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fafb00581a0 tx=0x7fafb0001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6412d9a092fc4a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:04.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.705+0000 7fafa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:15:04.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafa77fe640 1 -- 172.21.15.70:0/2110537715 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb006f040 con 0x56248e093f40 2023-10-08T07:15:04.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafa77fe640 1 -- 172.21.15.70:0/2110537715 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fafb0056fe0 con 0x56248e093f40 2023-10-08T07:15:04.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafd1a14740 1 -- 172.21.15.70:0/2110537715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56248dfcc6c0 con 0x56248e093f40 2023-10-08T07:15:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafa77fe640 1 -- 172.21.15.70:0/2110537715 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafb005f760 con 0x56248e093f40 2023-10-08T07:15:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafd1a14740 10 client.12195 fetch_fsmap learned FSMap version 130 2023-10-08T07:15:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafd1a14740 1 -- 172.21.15.70:0/2110537715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x56248dfd1d60 con 0x56248e093f40 2023-10-08T07:15:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafa77fe640 1 -- 172.21.15.70:0/2110537715 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56248dfcc6c0 con 0x56248e093f40 2023-10-08T07:15:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafa77fe640 1 -- 172.21.15.70:0/2110537715 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 130) v1 ==== 3891+0+0 (secure 0 0 0) 0x7fafb00872e0 con 0x56248e093f40 2023-10-08T07:15:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafd1a14740 10 client.12195 fetch_fsmap finished waiting for FSMap version 130 2023-10-08T07:15:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafd1a14740 10 client.12195 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T07:15:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafd1a14740 20 client.12195 populate_metadata read hostname 'smithi070' 2023-10-08T07:15:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafd1a14740 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.181:6836/80073995,v1:172.21.15.181:6837/80073995] conn(0x56248e094c60 0x56248e07ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafd1a14740 4 client.12195 mds_command: new command op to 8875 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T07:15:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.706+0000 7fafd1a14740 1 -- 172.21.15.70:0/2110537715 --> [v2:172.21.15.181:6836/80073995,v1:172.21.15.181:6837/80073995] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56248dfcc3e0 con 0x56248e094c60 2023-10-08T07:15:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.707+0000 7fafccc18640 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.181:6836/80073995,v1:172.21.15.181:6837/80073995] conn(0x56248e094c60 0x56248e07ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.708+0000 7fafccc18640 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.181:6836/80073995,v1:172.21.15.181:6837/80073995] conn(0x56248e094c60 0x56248e07ff80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b6c0f34d937775 server_cookie=a1a3d655900fbdb in_seq=0 out_seq=0 2023-10-08T07:15:04.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.708+0000 7fafa77fe640 10 client.12195 ms_handle_connect on v2:172.21.15.181:6836/80073995 2023-10-08T07:15:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafa77fe640 1 -- 172.21.15.70:0/2110537715 <== mds.0 v2:172.21.15.181:6836/80073995 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56248dfcc3e0 con 0x56248e094c60 2023-10-08T07:15:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafa77fe640 10 client.12195 handle_command_reply: tid=0 2023-10-08T07:15:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 client.12195 shutdown 2023-10-08T07:15:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 -- 172.21.15.70:0/2110537715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 msgr2=0x56248e0978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0978e0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fafb00581a0 tx=0x7fafb0001330 comp rx=0 tx=0).stop 2023-10-08T07:15:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 -- 172.21.15.70:0/2110537715 shutdown_connections 2023-10-08T07:15:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.181:6836/80073995,v1:172.21.15.181:6837/80073995] conn(0x56248e094c60 0x56248e07ff80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e07f570 0x56248e0943a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0978e0 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafa77fe640 0 client.12195 ms_handle_reset on v2:172.21.15.181:6836/80073995 2023-10-08T07:15:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 --2- 172.21.15.70:0/2110537715 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e093b60 0x56248e07d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 -- 172.21.15.70:0/2110537715 >> 172.21.15.70:0/2110537715 conn(0x56248e066fa0 msgr2=0x56248e06ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 -- 172.21.15.70:0/2110537715 shutdown_connections 2023-10-08T07:15:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.710+0000 7fafd1a14740 1 -- 172.21.15.70:0/2110537715 wait complete. 2023-10-08T07:15:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.711+0000 7fafd1a14740 20 client.12195 trim_cache size 0 max 16384 2023-10-08T07:15:04.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.727+0000 7fafd1a14740 1 Processor -- start 2023-10-08T07:15:04.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.727+0000 7fafd1a14740 1 -- start start 2023-10-08T07:15:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.728+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e07f570 0x56248e06ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.728+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e093b60 0x56248e0978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.728+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.728+0000 7fafd1a14740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248e06a770 con 0x56248e093f40 2023-10-08T07:15:04.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.728+0000 7fafd1a14740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248e096a30 con 0x56248e093b60 2023-10-08T07:15:04.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.728+0000 7fafd1a14740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56248e066e00 con 0x56248e07f570 2023-10-08T07:15:04.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafcd419640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafcd419640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51182/0 (socket says 172.21.15.70:51182) 2023-10-08T07:15:04.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafcd419640 1 -- 172.21.15.70:0/3369122035 learned_addr learned my addr 172.21.15.70:0/3369122035 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:04.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafccc18640 1 --2- 172.21.15.70:0/3369122035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e07f570 0x56248e06ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafc7fff640 1 --2- 172.21.15.70:0/3369122035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e093b60 0x56248e0978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafcd419640 1 -- 172.21.15.70:0/3369122035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e07f570 msgr2=0x56248e06ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafcd419640 1 --2- 172.21.15.70:0/3369122035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e07f570 0x56248e06ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafcd419640 1 -- 172.21.15.70:0/3369122035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e093b60 msgr2=0x56248e0978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafcd419640 1 --2- 172.21.15.70:0/3369122035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e093b60 0x56248e0978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafcd419640 1 -- 172.21.15.70:0/3369122035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56248df70b80 con 0x56248e093f40 2023-10-08T07:15:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.729+0000 7fafcd419640 1 --2- 172.21.15.70:0/3369122035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0b1c10 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fafc0007fd0 tx=0x7fafc00016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe98bb39c02a224 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafbeffd640 1 -- 172.21.15.70:0/3369122035 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0005cc0 con 0x56248e093f40 2023-10-08T07:15:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafbeffd640 1 -- 172.21.15.70:0/3369122035 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc000d050 con 0x56248e093f40 2023-10-08T07:15:04.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafbeffd640 1 -- 172.21.15.70:0/3369122035 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0007350 con 0x56248e093f40 2023-10-08T07:15:04.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafd1a14740 1 -- 172.21.15.70:0/3369122035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 msgr2=0x56248e0b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafd1a14740 1 --2- 172.21.15.70:0/3369122035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0b1c10 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fafc0007fd0 tx=0x7fafc00016f0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafd1a14740 1 -- 172.21.15.70:0/3369122035 shutdown_connections 2023-10-08T07:15:04.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafd1a14740 1 --2- 172.21.15.70:0/3369122035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e0b1c10 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafd1a14740 1 --2- 172.21.15.70:0/3369122035 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e093b60 0x56248e0978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafd1a14740 1 --2- 172.21.15.70:0/3369122035 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e07f570 0x56248e06ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafd1a14740 1 -- 172.21.15.70:0/3369122035 >> 172.21.15.70:0/3369122035 conn(0x56248e066fa0 msgr2=0x56248e091e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:04.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.730+0000 7fafd1a14740 1 -- 172.21.15.70:0/3369122035 shutdown_connections 2023-10-08T07:15:04.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.731+0000 7fafd1a14740 1 -- 172.21.15.70:0/3369122035 wait complete. 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.731+0000 7fafd1a14740 1 -- start start 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.731+0000 7fafd1a14740 1 Processor -- start 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.732+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e07f570 0x56248e112f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.732+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e093b60 0x56248e097a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.732+0000 7fafd1a14740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.732+0000 7fafd1a14740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248e0cfde0 con 0x56248e093f40 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.732+0000 7fafd1a14740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248e06bd70 con 0x56248e093b60 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.732+0000 7fafd1a14740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x56248e118f70 con 0x56248e07f570 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafd1a14740 10 client.0 _ll_register_callbacks cb 0x7fafb0053c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafcd419640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafcd419640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e06b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51198/0 (socket says 172.21.15.70:51198) 2023-10-08T07:15:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafcd419640 1 -- 172.21.15.70:0/1558468475 learned_addr learned my addr 172.21.15.70:0/1558468475 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafcd419640 1 -- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e07f570 msgr2=0x56248e112f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafccc18640 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e07f570 0x56248e112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafc7fff640 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e093b60 0x56248e097a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafcd419640 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e07f570 0x56248e112f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafcd419640 1 -- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e093b60 msgr2=0x56248e097a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafcd419640 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e093b60 0x56248e097a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.733+0000 7fafcd419640 1 -- 172.21.15.70:0/1558468475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc806c3a0 con 0x56248e093f40 2023-10-08T07:15:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.734+0000 7fafcd419640 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e06b2f0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7fafc0000fd0 tx=0x7fafc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d18f7cd7017ba6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.734+0000 7fafa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:15:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.734+0000 7fafa77fe640 1 -- 172.21.15.70:0/1558468475 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0007640 con 0x56248e093f40 2023-10-08T07:15:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.734+0000 7fafa77fe640 1 -- 172.21.15.70:0/1558468475 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fafc00077a0 con 0x56248e093f40 2023-10-08T07:15:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.734+0000 7fafd1a14740 1 -- 172.21.15.70:0/1558468475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafb006f1d0 con 0x56248e093f40 2023-10-08T07:15:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.734+0000 7fafa77fe640 1 -- 172.21.15.70:0/1558468475 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafc0020480 con 0x56248e093f40 2023-10-08T07:15:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.734+0000 7fafd1a14740 10 client.12201 fetch_fsmap learned FSMap version 130 2023-10-08T07:15:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.734+0000 7fafd1a14740 1 -- 172.21.15.70:0/1558468475 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x56248df96170 con 0x56248e093f40 2023-10-08T07:15:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.734+0000 7fafa77fe640 1 -- 172.21.15.70:0/1558468475 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb006f1d0 con 0x56248e093f40 2023-10-08T07:15:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.734+0000 7fafa77fe640 1 -- 172.21.15.70:0/1558468475 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 130) v1 ==== 3891+0+0 (secure 0 0 0) 0x7fafc00205e0 con 0x56248e093f40 2023-10-08T07:15:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.735+0000 7fafd1a14740 10 client.12201 fetch_fsmap finished waiting for FSMap version 130 2023-10-08T07:15:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.735+0000 7fafd1a14740 10 client.12201 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T07:15:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.735+0000 7fafd1a14740 20 client.12201 populate_metadata read hostname 'smithi070' 2023-10-08T07:15:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.735+0000 7fafd1a14740 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:6836/80073995,v1:172.21.15.181:6837/80073995] conn(0x56248e0b1da0 0x56248e0b2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.735+0000 7fafd1a14740 4 client.12201 mds_command: new command op to 8875 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T07:15:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.735+0000 7fafd1a14740 1 -- 172.21.15.70:0/1558468475 --> [v2:172.21.15.181:6836/80073995,v1:172.21.15.181:6837/80073995] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56248dcd56c0 con 0x56248e0b1da0 2023-10-08T07:15:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.735+0000 7fafccc18640 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:6836/80073995,v1:172.21.15.181:6837/80073995] conn(0x56248e0b1da0 0x56248e0b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.736+0000 7fafa77fe640 10 client.12201 ms_handle_connect on v2:172.21.15.181:6836/80073995 2023-10-08T07:15:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.736+0000 7fafccc18640 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:6836/80073995,v1:172.21.15.181:6837/80073995] conn(0x56248e0b1da0 0x56248e0b2180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af8ab260b78557d server_cookie=dd0ac0a3e5b30ea1 in_seq=0 out_seq=0 2023-10-08T07:15:04.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafa77fe640 1 -- 172.21.15.70:0/1558468475 <== mds.0 v2:172.21.15.181:6836/80073995 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56248dcd56c0 con 0x56248e0b1da0 2023-10-08T07:15:04.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafa77fe640 10 client.12201 handle_command_reply: tid=0 2023-10-08T07:15:04.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 client.12201 shutdown 2023-10-08T07:15:04.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 -- 172.21.15.70:0/1558468475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 msgr2=0x56248e06b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e06b2f0 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7fafc0000fd0 tx=0x7fafc00057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 -- 172.21.15.70:0/1558468475 shutdown_connections 2023-10-08T07:15:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:6836/80073995,v1:172.21.15.181:6837/80073995] conn(0x56248e0b1da0 0x56248e0b2180 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56248e093f40 0x56248e06b2f0 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x56248e093b60 0x56248e097a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 --2- 172.21.15.70:0/1558468475 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x56248e07f570 0x56248e112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 -- 172.21.15.70:0/1558468475 >> 172.21.15.70:0/1558468475 conn(0x56248e066fa0 msgr2=0x56248e06aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafa77fe640 0 client.12201 ms_handle_reset on v2:172.21.15.181:6836/80073995 2023-10-08T07:15:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 -- 172.21.15.70:0/1558468475 shutdown_connections 2023-10-08T07:15:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 1 -- 172.21.15.70:0/1558468475 wait complete. 2023-10-08T07:15:04.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.737+0000 7fafd1a14740 20 client.12201 trim_cache size 0 max 16384 2023-10-08T07:15:04.745 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:15:04.746 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T07:15:04.746 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T07:15:04.747 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 8875, 2023-10-08T07:15:04.747 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T07:15:04.747 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T07:15:04.747 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "backup_fs", 2023-10-08T07:15:04.747 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.1964239939999999, 2023-10-08T07:15:04.747 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 130, 2023-10-08T07:15:04.747 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 258, 2023-10-08T07:15:04.747 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 258, 2023-10-08T07:15:04.747 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 25.731729973 2023-10-08T07:15:04.747 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:15:04.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 -- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fafa806b0b0 msgr2=0x7fafa806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fafa806b0b0 0x7fafa806d570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fafb8038000 tx=0x7fafb8008030 comp rx=0 tx=0).stop 2023-10-08T07:15:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 -- 172.21.15.70:0/3490833939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b21f0 msgr2=0x7fafc81474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b21f0 0x7fafc81474c0 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c7a0 tx=0x7fafb000cc60 comp rx=0 tx=0).stop 2023-10-08T07:15:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 -- 172.21.15.70:0/3490833939 shutdown_connections 2023-10-08T07:15:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fafa806b0b0 0x7fafa806d570 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafc80ca370 0x7fafc81587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafc80b21f0 0x7fafc81474c0 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 --2- 172.21.15.70:0/3490833939 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafc80702b0 0x7fafc81e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 -- 172.21.15.70:0/3490833939 >> 172.21.15.70:0/3490833939 conn(0x7fafc806a5d0 msgr2=0x7fafc80c6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:04.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 -- 172.21.15.70:0/3490833939 shutdown_connections 2023-10-08T07:15:04.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.738+0000 7fafcee7c640 1 -- 172.21.15.70:0/3490833939 wait complete. 2023-10-08T07:15:04.755 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T07:15:04.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.961+0000 7fb7698b4640 1 Processor -- start 2023-10-08T07:15:04.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.961+0000 7fb7698b4640 1 -- start start 2023-10-08T07:15:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.962+0000 7fb7698b4640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb76406c3a0 0x7fb7640c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.962+0000 7fb7698b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76406db60 0x7fb76406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.962+0000 7fb7698b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb76415aa00 0x7fb7640b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.962+0000 7fb7698b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7640ca230 con 0x7fb76415aa00 2023-10-08T07:15:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.962+0000 7fb7698b4640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76415b310 con 0x7fb76406c3a0 2023-10-08T07:15:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.962+0000 7fb7698b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76406d290 con 0x7fb76406db60 2023-10-08T07:15:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.963+0000 7fb7637fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb76415aa00 0x7fb7640b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.963+0000 7fb7637fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb76415aa00 0x7fb7640b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51210/0 (socket says 172.21.15.70:51210) 2023-10-08T07:15:04.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.963+0000 7fb7637fe640 1 -- 172.21.15.70:0/2961802552 learned_addr learned my addr 172.21.15.70:0/2961802552 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.963+0000 7fb762ffd640 1 --2- 172.21.15.70:0/2961802552 >> v2:172.21.15.181:3300/0 conn(0x7fb76406c3a0 0x7fb7640c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.963+0000 7fb7627fc640 1 --2- 172.21.15.70:0/2961802552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76406db60 0x7fb76406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.963+0000 7fb7637fe640 1 -- 172.21.15.70:0/2961802552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76406db60 msgr2=0x7fb76406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.963+0000 7fb7637fe640 1 --2- 172.21.15.70:0/2961802552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76406db60 0x7fb76406c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.963+0000 7fb7637fe640 1 -- 172.21.15.70:0/2961802552 >> v2:172.21.15.181:3300/0 conn(0x7fb76406c3a0 msgr2=0x7fb7640c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.963+0000 7fb7637fe640 1 --2- 172.21.15.70:0/2961802552 >> v2:172.21.15.181:3300/0 conn(0x7fb76406c3a0 0x7fb7640c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.963+0000 7fb7637fe640 1 -- 172.21.15.70:0/2961802552 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7640c9860 con 0x7fb76415aa00 2023-10-08T07:15:04.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.964+0000 7fb7637fe640 1 --2- 172.21.15.70:0/2961802552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76415aa00 0x7fb7640b2a90 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fb7640716b0 tx=0x7fb75c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531bd60c4495a0f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.965+0000 7fb761ffb640 1 -- 172.21.15.70:0/2961802552 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c02ea80 con 0x7fb76415aa00 2023-10-08T07:15:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.965+0000 7fb761ffb640 1 -- 172.21.15.70:0/2961802552 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75c02ebe0 con 0x7fb76415aa00 2023-10-08T07:15:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.965+0000 7fb761ffb640 1 -- 172.21.15.70:0/2961802552 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c009600 con 0x7fb76415aa00 2023-10-08T07:15:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.965+0000 7fb7698b4640 1 -- 172.21.15.70:0/2961802552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76415aa00 msgr2=0x7fb7640b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.965+0000 7fb7698b4640 1 --2- 172.21.15.70:0/2961802552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76415aa00 0x7fb7640b2a90 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7fb7640716b0 tx=0x7fb75c02e0e0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.965+0000 7fb7698b4640 1 -- 172.21.15.70:0/2961802552 shutdown_connections 2023-10-08T07:15:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.965+0000 7fb7698b4640 1 --2- 172.21.15.70:0/2961802552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76415aa00 0x7fb7640b2a90 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.965+0000 7fb7698b4640 1 --2- 172.21.15.70:0/2961802552 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76406db60 0x7fb76406c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.965+0000 7fb7698b4640 1 --2- 172.21.15.70:0/2961802552 >> v2:172.21.15.181:3300/0 conn(0x7fb76406c3a0 0x7fb7640c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.965+0000 7fb7698b4640 1 -- 172.21.15.70:0/2961802552 >> 172.21.15.70:0/2961802552 conn(0x7fb76406a5d0 msgr2=0x7fb7640b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.966+0000 7fb7698b4640 1 -- 172.21.15.70:0/2961802552 shutdown_connections 2023-10-08T07:15:04.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.966+0000 7fb7698b4640 1 -- 172.21.15.70:0/2961802552 wait complete. 2023-10-08T07:15:04.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.967+0000 7fb7698b4640 1 Processor -- start 2023-10-08T07:15:04.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.967+0000 7fb7698b4640 1 -- start start 2023-10-08T07:15:04.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.968+0000 7fb7698b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76406c3a0 0x7fb7641f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.968+0000 7fb7698b4640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76406db60 0x7fb7640b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.968+0000 7fb7698b4640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb76415aa00 0x7fb7640b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.968+0000 7fb7698b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7641f10f0 con 0x7fb76406c3a0 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.968+0000 7fb7698b4640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb764147390 con 0x7fb76415aa00 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.968+0000 7fb7698b4640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7641ef220 con 0x7fb76406db60 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.969+0000 7fb762ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76406c3a0 0x7fb7641f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.969+0000 7fb762ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76406c3a0 0x7fb7641f0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51214/0 (socket says 172.21.15.70:51214) 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.969+0000 7fb762ffd640 1 -- 172.21.15.70:0/152086969 learned_addr learned my addr 172.21.15.70:0/152086969 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.969+0000 7fb762ffd640 1 -- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76406db60 msgr2=0x7fb7640b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.969+0000 7fb762ffd640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76406db60 0x7fb7640b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.969+0000 7fb762ffd640 1 -- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb76415aa00 msgr2=0x7fb7640b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.969+0000 7fb762ffd640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb76415aa00 0x7fb7640b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:04.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.969+0000 7fb762ffd640 1 -- 172.21.15.70:0/152086969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c00bd40 con 0x7fb76406c3a0 2023-10-08T07:15:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.969+0000 7fb762ffd640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76406c3a0 0x7fb7641f0bb0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fb75400d800 tx=0x7fb75400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.969+0000 7fb74bfff640 1 -- 172.21.15.70:0/152086969 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7540096a0 con 0x7fb76406c3a0 2023-10-08T07:15:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.970+0000 7fb74bfff640 1 -- 172.21.15.70:0/152086969 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb754009800 con 0x7fb76406c3a0 2023-10-08T07:15:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.970+0000 7fb74bfff640 1 -- 172.21.15.70:0/152086969 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754009b90 con 0x7fb76406c3a0 2023-10-08T07:15:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.970+0000 7fb7698b4640 1 -- 172.21.15.70:0/152086969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb764147600 con 0x7fb76406c3a0 2023-10-08T07:15:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.970+0000 7fb7698b4640 1 -- 172.21.15.70:0/152086969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb764147bc0 con 0x7fb76406c3a0 2023-10-08T07:15:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.971+0000 7fb7698b4640 1 -- 172.21.15.70:0/152086969 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76404e160 con 0x7fb76406c3a0 2023-10-08T07:15:04.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.972+0000 7fb74bfff640 1 -- 172.21.15.70:0/152086969 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fb754009cf0 con 0x7fb76406c3a0 2023-10-08T07:15:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.973+0000 7fb74bfff640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb75006b0b0 0x7fb75006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.973+0000 7fb74bfff640 1 -- 172.21.15.70:0/152086969 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fb75408a3e0 con 0x7fb76406c3a0 2023-10-08T07:15:04.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.974+0000 7fb7627fc640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb75006b0b0 0x7fb75006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:04.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.975+0000 7fb7627fc640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb75006b0b0 0x7fb75006d570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb758006fd0 tx=0x7fb758008040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:04.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:04.978+0000 7fb74bfff640 1 -- 172.21.15.70:0/152086969 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb754054c10 con 0x7fb76406c3a0 2023-10-08T07:15:05.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.194+0000 7fb7698b4640 1 -- 172.21.15.70:0/152086969 --> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7640c8f30 con 0x7fb75006b0b0 2023-10-08T07:15:05.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.198+0000 7fb74bfff640 1 -- 172.21.15.70:0/152086969 <== mgr.8662 v2:172.21.15.181:6832/4245028292 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7fb75800ae20 con 0x7fb75006b0b0 2023-10-08T07:15:05.199 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T07:15:05.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:05.199 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T07:15:05.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:05.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:05.200 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "backup_fs" 2023-10-08T07:15:05.200 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:05.200 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T07:15:05.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 -- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb75006b0b0 msgr2=0x7fb75006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:05.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb75006b0b0 0x7fb75006d570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb758006fd0 tx=0x7fb758008040 comp rx=0 tx=0).stop 2023-10-08T07:15:05.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 -- 172.21.15.70:0/152086969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76406c3a0 msgr2=0x7fb7641f0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:05.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76406c3a0 0x7fb7641f0bb0 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fb75400d800 tx=0x7fb75400dcc0 comp rx=0 tx=0).stop 2023-10-08T07:15:05.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 -- 172.21.15.70:0/152086969 shutdown_connections 2023-10-08T07:15:05.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fb75006b0b0 0x7fb75006d570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:05.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb76415aa00 0x7fb7640b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:05.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb76406db60 0x7fb7640b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:05.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 --2- 172.21.15.70:0/152086969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb76406c3a0 0x7fb7641f0bb0 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:05.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 -- 172.21.15.70:0/152086969 >> 172.21.15.70:0/152086969 conn(0x7fb76406a5d0 msgr2=0x7fb764159980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:05.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 -- 172.21.15.70:0/152086969 shutdown_connections 2023-10-08T07:15:05.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.201+0000 7fb7698b4640 1 -- 172.21.15.70:0/152086969 wait complete. 2023-10-08T07:15:05.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T07:15:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.437+0000 7fc16b755640 1 Processor -- start 2023-10-08T07:15:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.438+0000 7fc16b755640 1 -- start start 2023-10-08T07:15:05.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.439+0000 7fc16b755640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc1641432f0 0x7fc1640cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:05.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.439+0000 7fc16b755640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1640b4240 0x7fc164159e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:05.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.439+0000 7fc16b755640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1640cbb30 con 0x7fc1640cb6b0 2023-10-08T07:15:05.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.439+0000 7fc16b755640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1641436d0 con 0x7fc1640b4240 2023-10-08T07:15:05.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.439+0000 7fc16b755640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16415d440 con 0x7fc1641432f0 2023-10-08T07:15:05.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.440+0000 7fc169cf2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1640b4240 0x7fc164159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:05.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.440+0000 7fc168cf0640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc1640cb6b0 0x7fc1640b3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47504/0 (socket says 172.21.15.70:47504) 2023-10-08T07:15:05.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.440+0000 7fc168cf0640 1 -- 172.21.15.70:0/3813822902 learned_addr learned my addr 172.21.15.70:0/3813822902 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:05.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.440+0000 7fc1694f1640 1 --2- 172.21.15.70:0/3813822902 >> v2:172.21.15.181:3300/0 conn(0x7fc1641432f0 0x7fc1640cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:05.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.440+0000 7fc169cf2640 1 -- 172.21.15.70:0/3813822902 >> v2:172.21.15.181:3300/0 conn(0x7fc1641432f0 msgr2=0x7fc1640cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:05.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.440+0000 7fc169cf2640 1 --2- 172.21.15.70:0/3813822902 >> v2:172.21.15.181:3300/0 conn(0x7fc1641432f0 0x7fc1640cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:05.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.440+0000 7fc169cf2640 1 -- 172.21.15.70:0/3813822902 >> v1:172.21.15.70:6789/0 conn(0x7fc1640cb6b0 legacy=0x7fc1640b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:05.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.440+0000 7fc169cf2640 1 -- 172.21.15.70:0/3813822902 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16415d690 con 0x7fc1640b4240 2023-10-08T07:15:05.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.440+0000 7fc15bfff640 1 -- 172.21.15.70:0/3813822902 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975897839 0 0) 0x7fc1640cbb30 con 0x7fc1640cb6b0 2023-10-08T07:15:05.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.442+0000 7fc169cf2640 1 --2- 172.21.15.70:0/3813822902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 0x7fc164159e50 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7fc1640cc1a0 tx=0x7fc15c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a919e5e92247c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:05.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.442+0000 7fc15bfff640 1 -- 172.21.15.70:0/3813822902 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c02cbe0 con 0x7fc1640b4240 2023-10-08T07:15:05.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.442+0000 7fc15bfff640 1 -- 172.21.15.70:0/3813822902 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c02cd40 con 0x7fc1640b4240 2023-10-08T07:15:05.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.442+0000 7fc15bfff640 1 -- 172.21.15.70:0/3813822902 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c034660 con 0x7fc1640b4240 2023-10-08T07:15:05.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.442+0000 7fc16b755640 1 -- 172.21.15.70:0/3813822902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 msgr2=0x7fc164159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:05.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.442+0000 7fc16b755640 1 --2- 172.21.15.70:0/3813822902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 0x7fc164159e50 secure :-1 s=READY pgs=2717 cs=0 l=1 rev1=1 crypto rx=0x7fc1640cc1a0 tx=0x7fc15c02be30 comp rx=0 tx=0).stop 2023-10-08T07:15:05.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.442+0000 7fc16b755640 1 -- 172.21.15.70:0/3813822902 shutdown_connections 2023-10-08T07:15:05.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.442+0000 7fc16b755640 1 --2- 172.21.15.70:0/3813822902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 0x7fc164159e50 unknown :-1 s=CLOSED pgs=2717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:05.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.443+0000 7fc16b755640 1 --2- 172.21.15.70:0/3813822902 >> v2:172.21.15.181:3300/0 conn(0x7fc1641432f0 0x7fc1640cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:05.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.443+0000 7fc16b755640 1 -- 172.21.15.70:0/3813822902 >> 172.21.15.70:0/3813822902 conn(0x7fc164069d50 msgr2=0x7fc16406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:05.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.443+0000 7fc16b755640 1 -- 172.21.15.70:0/3813822902 shutdown_connections 2023-10-08T07:15:05.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.443+0000 7fc16b755640 1 -- 172.21.15.70:0/3813822902 wait complete. 2023-10-08T07:15:05.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.444+0000 7fc16b755640 1 Processor -- start 2023-10-08T07:15:05.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.444+0000 7fc16b755640 1 -- start start 2023-10-08T07:15:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.445+0000 7fc16b755640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 0x7fc164138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.445+0000 7fc16b755640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1640cb6b0 0x7fc1640c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.445+0000 7fc16b755640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1641432f0 0x7fc1640c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.445+0000 7fc16b755640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc164139a70 con 0x7fc1640b4240 2023-10-08T07:15:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.445+0000 7fc16b755640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1641f6210 con 0x7fc1640cb6b0 2023-10-08T07:15:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.445+0000 7fc16b755640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1641f66a0 con 0x7fc1641432f0 2023-10-08T07:15:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc1694f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 0x7fc164138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc1694f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 0x7fc164138860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51222/0 (socket says 172.21.15.70:51222) 2023-10-08T07:15:05.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc1694f1640 1 -- 172.21.15.70:0/40932894 learned_addr learned my addr 172.21.15.70:0/40932894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc1694f1640 1 -- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1641432f0 msgr2=0x7fc1640c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc1694f1640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1641432f0 0x7fc1640c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc1694f1640 1 -- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1640cb6b0 msgr2=0x7fc1640c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc168cf0640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1640cb6b0 0x7fc1640c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc1694f1640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1640cb6b0 0x7fc1640c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc1694f1640 1 -- 172.21.15.70:0/40932894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c009a70 con 0x7fc1640b4240 2023-10-08T07:15:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc168cf0640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1640cb6b0 0x7fc1640c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:05.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.446+0000 7fc1694f1640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 0x7fc164138860 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7fc16000d620 tx=0x7fc16000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:05.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.447+0000 7fc159ffb640 1 -- 172.21.15.70:0/40932894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000b620 con 0x7fc1640b4240 2023-10-08T07:15:05.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.447+0000 7fc159ffb640 1 -- 172.21.15.70:0/40932894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16000b780 con 0x7fc1640b4240 2023-10-08T07:15:05.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.447+0000 7fc159ffb640 1 -- 172.21.15.70:0/40932894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000bb30 con 0x7fc1640b4240 2023-10-08T07:15:05.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.447+0000 7fc16b755640 1 -- 172.21.15.70:0/40932894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc164138da0 con 0x7fc1640b4240 2023-10-08T07:15:05.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.447+0000 7fc16b755640 1 -- 172.21.15.70:0/40932894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1641f58e0 con 0x7fc1640b4240 2023-10-08T07:15:05.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.448+0000 7fc16b755640 1 -- 172.21.15.70:0/40932894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16404e160 con 0x7fc1640b4240 2023-10-08T07:15:05.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.449+0000 7fc159ffb640 1 -- 172.21.15.70:0/40932894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 138) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fc16000bc90 con 0x7fc1640b4240 2023-10-08T07:15:05.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.450+0000 7fc159ffb640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc14806b0b0 0x7fc14806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:05.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.450+0000 7fc159ffb640 1 -- 172.21.15.70:0/40932894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7fc16008a560 con 0x7fc1640b4240 2023-10-08T07:15:05.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.450+0000 7fc168cf0640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc14806b0b0 0x7fc14806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:05.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.451+0000 7fc168cf0640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc14806b0b0 0x7fc14806d570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc1640ca720 tx=0x7fc154009040 comp rx=0 tx=0).ready entity=mgr.8662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:05.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.454+0000 7fc159ffb640 1 -- 172.21.15.70:0/40932894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc160054cf0 con 0x7fc1640b4240 2023-10-08T07:15:05.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:05.694+0000 7fc16b755640 1 -- 172.21.15.70:0/40932894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc164142870 con 0x7fc1640b4240 2023-10-08T07:15:06.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.588+0000 7fc159ffb640 1 -- 172.21.15.70:0/40932894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v139) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc160058310 con 0x7fc1640b4240 2023-10-08T07:15:06.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.591+0000 7fc16b755640 1 -- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc14806b0b0 msgr2=0x7fc14806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:06.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.591+0000 7fc16b755640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc14806b0b0 0x7fc14806d570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc1640ca720 tx=0x7fc154009040 comp rx=0 tx=0).stop 2023-10-08T07:15:06.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.591+0000 7fc16b755640 1 -- 172.21.15.70:0/40932894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 msgr2=0x7fc164138860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:06.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.591+0000 7fc16b755640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 0x7fc164138860 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7fc16000d620 tx=0x7fc16000dae0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.591+0000 7fc16b755640 1 -- 172.21.15.70:0/40932894 shutdown_connections 2023-10-08T07:15:06.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.591+0000 7fc16b755640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7fc14806b0b0 0x7fc14806d570 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.591+0000 7fc16b755640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc1641432f0 0x7fc1640c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.591+0000 7fc16b755640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc1640cb6b0 0x7fc1640c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.591+0000 7fc16b755640 1 --2- 172.21.15.70:0/40932894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1640b4240 0x7fc164138860 unknown :-1 s=CLOSED pgs=2718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.591+0000 7fc16b755640 1 -- 172.21.15.70:0/40932894 >> 172.21.15.70:0/40932894 conn(0x7fc164069d50 msgr2=0x7fc16406ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:06.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.592+0000 7fc16b755640 1 -- 172.21.15.70:0/40932894 shutdown_connections 2023-10-08T07:15:06.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.592+0000 7fc16b755640 1 -- 172.21.15.70:0/40932894 wait complete. 2023-10-08T07:15:06.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T07:15:06.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.814+0000 7f4f9bfff640 1 Processor -- start 2023-10-08T07:15:06.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.814+0000 7f4f9bfff640 1 -- start start 2023-10-08T07:15:06.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4f9c069fc0 0x7f4f9c071860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:06.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f9c06ec00 0x7f4f9c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:06.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 0x7f4f9c160080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:06.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9bfff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9c160d10 con 0x7f4f9c06ec00 2023-10-08T07:15:06.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9bfff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9c072070 con 0x7f4f9c069fc0 2023-10-08T07:15:06.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9bfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c071580 con 0x7f4f9c0ca210 2023-10-08T07:15:06.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f9c06ec00 0x7f4f9c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:06.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f9c06ec00 0x7f4f9c0c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51246/0 (socket says 172.21.15.70:51246) 2023-10-08T07:15:06.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9a7fc640 1 -- 172.21.15.70:0/1598755078 learned_addr learned my addr 172.21.15.70:0/1598755078 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:06.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9affd640 1 --2- 172.21.15.70:0/1598755078 >> v2:172.21.15.181:3300/0 conn(0x7f4f9c069fc0 0x7f4f9c071860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:06.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.816+0000 7f4f9b7fe640 1 --2- 172.21.15.70:0/1598755078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 0x7f4f9c160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:06.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.817+0000 7f4f9a7fc640 1 -- 172.21.15.70:0/1598755078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 msgr2=0x7f4f9c160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:06.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.817+0000 7f4f9a7fc640 1 --2- 172.21.15.70:0/1598755078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 0x7f4f9c160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.817+0000 7f4f9a7fc640 1 -- 172.21.15.70:0/1598755078 >> v2:172.21.15.181:3300/0 conn(0x7f4f9c069fc0 msgr2=0x7f4f9c071860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:06.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.817+0000 7f4f9a7fc640 1 --2- 172.21.15.70:0/1598755078 >> v2:172.21.15.181:3300/0 conn(0x7f4f9c069fc0 0x7f4f9c071860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.817+0000 7f4f9a7fc640 1 -- 172.21.15.70:0/1598755078 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c072330 con 0x7f4f9c06ec00 2023-10-08T07:15:06.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.817+0000 7f4f9b7fe640 1 --2- 172.21.15.70:0/1598755078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 0x7f4f9c160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:15:06.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.819+0000 7f4f9a7fc640 1 --2- 172.21.15.70:0/1598755078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 0x7f4f9c0c9cd0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c070de0 tx=0x7f4f9002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2e3eafa37605bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:06.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.819+0000 7f4f99ffb640 1 -- 172.21.15.70:0/1598755078 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90031860 con 0x7f4f9c06ec00 2023-10-08T07:15:06.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.819+0000 7f4f99ffb640 1 -- 172.21.15.70:0/1598755078 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f900319c0 con 0x7f4f9c06ec00 2023-10-08T07:15:06.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.819+0000 7f4f99ffb640 1 -- 172.21.15.70:0/1598755078 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90031d50 con 0x7f4f9c06ec00 2023-10-08T07:15:06.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.820+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1598755078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 msgr2=0x7f4f9c0c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:06.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.820+0000 7f4f9bfff640 1 --2- 172.21.15.70:0/1598755078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 0x7f4f9c0c9cd0 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c070de0 tx=0x7f4f9002b900 comp rx=0 tx=0).stop 2023-10-08T07:15:06.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.820+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1598755078 shutdown_connections 2023-10-08T07:15:06.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.820+0000 7f4f9bfff640 1 --2- 172.21.15.70:0/1598755078 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 0x7f4f9c160080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.820+0000 7f4f9bfff640 1 --2- 172.21.15.70:0/1598755078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 0x7f4f9c0c9cd0 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.820+0000 7f4f9bfff640 1 --2- 172.21.15.70:0/1598755078 >> v2:172.21.15.181:3300/0 conn(0x7f4f9c069fc0 0x7f4f9c071860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.820+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1598755078 >> 172.21.15.70:0/1598755078 conn(0x7f4f9c0cf5f0 msgr2=0x7f4f9c0cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:06.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.820+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1598755078 shutdown_connections 2023-10-08T07:15:06.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.820+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1598755078 wait complete. 2023-10-08T07:15:06.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.822+0000 7f4f9bfff640 1 Processor -- start 2023-10-08T07:15:06.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.822+0000 7f4f9bfff640 1 -- start start 2023-10-08T07:15:06.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9bfff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f9c069fc0 0x7f4f9c156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:06.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 0x7f4f9c0b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:06.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 0x7f4f9c0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:06.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9bfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c0c8c70 con 0x7f4f9c06ec00 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9bfff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c156c70 con 0x7f4f9c069fc0 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9bfff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c0c7d10 con 0x7f4f9c0ca210 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 0x7f4f9c0b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 0x7f4f9c0b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51252/0 (socket says 172.21.15.70:51252) 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9a7fc640 1 -- 172.21.15.70:0/1984998556 learned_addr learned my addr 172.21.15.70:0/1984998556 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9b7fe640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 0x7f4f9c0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.823+0000 7f4f9affd640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f9c069fc0 0x7f4f9c156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.824+0000 7f4f9a7fc640 1 -- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 msgr2=0x7f4f9c0b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.824+0000 7f4f9a7fc640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 0x7f4f9c0b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.824+0000 7f4f9a7fc640 1 -- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f9c069fc0 msgr2=0x7f4f9c156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:06.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.824+0000 7f4f9a7fc640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f9c069fc0 0x7f4f9c156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:06.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.824+0000 7f4f9a7fc640 1 -- 172.21.15.70:0/1984998556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f900094d0 con 0x7f4f9c06ec00 2023-10-08T07:15:06.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.824+0000 7f4f9a7fc640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 0x7f4f9c0b11d0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f4f9002b940 tx=0x7f4f900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:06.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.824+0000 7f4f83fff640 1 -- 172.21.15.70:0/1984998556 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9003ad30 con 0x7f4f9c06ec00 2023-10-08T07:15:06.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.824+0000 7f4f83fff640 1 -- 172.21.15.70:0/1984998556 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f900443f0 con 0x7f4f9c06ec00 2023-10-08T07:15:06.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.824+0000 7f4f83fff640 1 -- 172.21.15.70:0/1984998556 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f900447a0 con 0x7f4f9c06ec00 2023-10-08T07:15:06.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.824+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1984998556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c0c7ff0 con 0x7f4f9c06ec00 2023-10-08T07:15:06.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.825+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1984998556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c0c85b0 con 0x7f4f9c06ec00 2023-10-08T07:15:06.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.826+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1984998556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c074150 con 0x7f4f9c06ec00 2023-10-08T07:15:06.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.828+0000 7f4f83fff640 1 -- 172.21.15.70:0/1984998556 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 139) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f4f9004d450 con 0x7f4f9c06ec00 2023-10-08T07:15:06.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.829+0000 7f4f83fff640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4f8806b100 0x7f4f8806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:06.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.829+0000 7f4f83fff640 1 -- 172.21.15.70:0/1984998556 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f4f900ac750 con 0x7f4f9c06ec00 2023-10-08T07:15:06.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.829+0000 7f4f9affd640 1 -- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4f8806b100 msgr2=0x7f4f8806d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245028292 2023-10-08T07:15:06.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.829+0000 7f4f9affd640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4f8806b100 0x7f4f8806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:06.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:06.833+0000 7f4f83fff640 1 -- 172.21.15.70:0/1984998556 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f90076dd0 con 0x7f4f9c06ec00 2023-10-08T07:15:07.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.024+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1984998556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f4f9c0c9880 con 0x7f4f9c06ec00 2023-10-08T07:15:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.029+0000 7f4f83fff640 1 -- 172.21.15.70:0/1984998556 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v83)=0 v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4f900350d0 con 0x7f4f9c06ec00 2023-10-08T07:15:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.030+0000 7f4f9affd640 1 -- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4f8806b100 msgr2=0x7f4f8806d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245028292 2023-10-08T07:15:07.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.030+0000 7f4f9affd640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4f8806b100 0x7f4f8806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:15:07.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4f8806b100 msgr2=0x7f4f8806d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:07.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4f8806b100 0x7f4f8806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1984998556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 msgr2=0x7f4f9c0b11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 0x7f4f9c0b11d0 secure :-1 s=READY pgs=2724 cs=0 l=1 rev1=1 crypto rx=0x7f4f9002b940 tx=0x7f4f900046c0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1984998556 shutdown_connections 2023-10-08T07:15:07.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4f8806b100 0x7f4f8806d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4f9c0ca210 0x7f4f9c0b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f9c06ec00 0x7f4f9c0b11d0 unknown :-1 s=CLOSED pgs=2724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 --2- 172.21.15.70:0/1984998556 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4f9c069fc0 0x7f4f9c156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1984998556 >> 172.21.15.70:0/1984998556 conn(0x7f4f9c0cf5f0 msgr2=0x7f4f9c0adc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:07.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1984998556 shutdown_connections 2023-10-08T07:15:07.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.032+0000 7f4f9bfff640 1 -- 172.21.15.70:0/1984998556 wait complete. 2023-10-08T07:15:07.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T07:15:07.122 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:07.122+0000 7f00a1a2b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:15:07.132 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:07.130+0000 7f0812e30040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:15:07.178 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:07.177+0000 7f00a1a2b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:15:07.185 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:07.184+0000 7f0812e30040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:15:07.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.257+0000 7f22cf59e640 1 Processor -- start 2023-10-08T07:15:07.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.257+0000 7f22cf59e640 1 -- start start 2023-10-08T07:15:07.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.259+0000 7f22cf59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f22d00d00e0 0x7f22d00d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.259+0000 7f22cf59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d00d2870 con 0x7f22d015ccb0 2023-10-08T07:15:07.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.259+0000 7f22cf59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d00d1150 con 0x7f22d00d0b90 2023-10-08T07:15:07.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.259+0000 7f22cf59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d00d1430 con 0x7f22d00d00e0 2023-10-08T07:15:07.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.259+0000 7f22c7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f22d00d00e0 0x7f22d00d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:07.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.259+0000 7f22ce59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f22d015ccb0 0x7f22d00cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47510/0 (socket says 172.21.15.70:47510) 2023-10-08T07:15:07.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.259+0000 7f22ce59c640 1 -- 172.21.15.70:0/3616033000 learned_addr learned my addr 172.21.15.70:0/3616033000 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:07.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.260+0000 7f22c7fff640 1 -- 172.21.15.70:0/3616033000 >> v1:172.21.15.181:6789/0 conn(0x7f22d00d0b90 legacy=0x7f22d00b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:07.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.260+0000 7f22c7fff640 1 -- 172.21.15.70:0/3616033000 >> v1:172.21.15.70:6789/0 conn(0x7f22d015ccb0 legacy=0x7f22d00cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.260+0000 7f22c7fff640 1 -- 172.21.15.70:0/3616033000 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00d1710 con 0x7f22d00d00e0 2023-10-08T07:15:07.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.261+0000 7f22c7fff640 1 --2- 172.21.15.70:0/3616033000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d00e0 0x7f22d00d1fb0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f22b80097a0 tx=0x7f22b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38efbd76a341671a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:07.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.261+0000 7f22cdd9b640 1 -- 172.21.15.70:0/3616033000 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8031830 con 0x7f22d00d00e0 2023-10-08T07:15:07.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.261+0000 7f22cdd9b640 1 -- 172.21.15.70:0/3616033000 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b8031990 con 0x7f22d00d00e0 2023-10-08T07:15:07.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.261+0000 7f22cdd9b640 1 -- 172.21.15.70:0/3616033000 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8031d20 con 0x7f22d00d00e0 2023-10-08T07:15:07.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.262+0000 7f22cf59e640 1 -- 172.21.15.70:0/3616033000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d00e0 msgr2=0x7f22d00d1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.262+0000 7f22cf59e640 1 --2- 172.21.15.70:0/3616033000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d00e0 0x7f22d00d1fb0 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f22b80097a0 tx=0x7f22b802b8d0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.262+0000 7f22cf59e640 1 -- 172.21.15.70:0/3616033000 shutdown_connections 2023-10-08T07:15:07.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.262+0000 7f22cf59e640 1 --2- 172.21.15.70:0/3616033000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d00e0 0x7f22d00d1fb0 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.262+0000 7f22cf59e640 1 -- 172.21.15.70:0/3616033000 >> 172.21.15.70:0/3616033000 conn(0x7f22d0071660 msgr2=0x7f22d0071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:07.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.262+0000 7f22cf59e640 1 -- 172.21.15.70:0/3616033000 shutdown_connections 2023-10-08T07:15:07.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.262+0000 7f22cf59e640 1 -- 172.21.15.70:0/3616033000 wait complete. 2023-10-08T07:15:07.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.264+0000 7f22cf59e640 1 Processor -- start 2023-10-08T07:15:07.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.264+0000 7f22cf59e640 1 -- start start 2023-10-08T07:15:07.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.265+0000 7f22cf59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d00d00e0 0x7f22d01f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.265+0000 7f22cf59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d0b90 0x7f22d00cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.265+0000 7f22cf59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d015ccb0 0x7f22d00b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.265+0000 7f22cf59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d01f1ff0 con 0x7f22d00d0b90 2023-10-08T07:15:07.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.265+0000 7f22cf59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00cc060 con 0x7f22d00d00e0 2023-10-08T07:15:07.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.265+0000 7f22cf59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00cc340 con 0x7f22d015ccb0 2023-10-08T07:15:07.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.265+0000 7f22c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d0b90 0x7f22d00cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:07.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.265+0000 7f22c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d0b90 0x7f22d00cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51280/0 (socket says 172.21.15.70:51280) 2023-10-08T07:15:07.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.265+0000 7f22c7fff640 1 -- 172.21.15.70:0/3744428362 learned_addr learned my addr 172.21.15.70:0/3744428362 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:07.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.266+0000 7f22c7fff640 1 -- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d015ccb0 msgr2=0x7f22d00b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.266+0000 7f22c7fff640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d015ccb0 0x7f22d00b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.266+0000 7f22ce59c640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d00d00e0 0x7f22d01f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:07.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.266+0000 7f22c7fff640 1 -- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d00d00e0 msgr2=0x7f22d01f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.266+0000 7f22c7fff640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d00d00e0 0x7f22d01f1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.266+0000 7f22c7fff640 1 -- 172.21.15.70:0/3744428362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b8009400 con 0x7f22d00d0b90 2023-10-08T07:15:07.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.266+0000 7f22c7fff640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d0b90 0x7f22d00cb6a0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f22b80098c0 tx=0x7f22b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:07.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.266+0000 7f22c6ffd640 1 -- 172.21.15.70:0/3744428362 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b803ba70 con 0x7f22d00d0b90 2023-10-08T07:15:07.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.267+0000 7f22c6ffd640 1 -- 172.21.15.70:0/3744428362 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b803bbf0 con 0x7f22d00d0b90 2023-10-08T07:15:07.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.267+0000 7f22c6ffd640 1 -- 172.21.15.70:0/3744428362 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b80453f0 con 0x7f22d00d0b90 2023-10-08T07:15:07.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.267+0000 7f22cf59e640 1 -- 172.21.15.70:0/3744428362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d00cc620 con 0x7f22d00d0b90 2023-10-08T07:15:07.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.267+0000 7f22cf59e640 1 -- 172.21.15.70:0/3744428362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d00c6790 con 0x7f22d00d0b90 2023-10-08T07:15:07.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.268+0000 7f22cf59e640 1 -- 172.21.15.70:0/3744428362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d00c6980 con 0x7f22d00d0b90 2023-10-08T07:15:07.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.269+0000 7f22c6ffd640 1 -- 172.21.15.70:0/3744428362 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 139) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f22b804e450 con 0x7f22d00d0b90 2023-10-08T07:15:07.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.270+0000 7f22c6ffd640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f22b006b130 0x7f22b006d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.270+0000 7f22c6ffd640 1 -- 172.21.15.70:0/3744428362 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f22b80ac460 con 0x7f22d00d0b90 2023-10-08T07:15:07.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.270+0000 7f22ce59c640 1 -- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f22b006b130 msgr2=0x7f22b006d5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245028292 2023-10-08T07:15:07.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.270+0000 7f22ce59c640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f22b006b130 0x7f22b006d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:07.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.274+0000 7f22c6ffd640 1 -- 172.21.15.70:0/3744428362 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22b8076b70 con 0x7f22d00d0b90 2023-10-08T07:15:07.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.460+0000 7f22cf59e640 1 -- 172.21.15.70:0/3744428362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f22d0074150 con 0x7f22d00d0b90 2023-10-08T07:15:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.466+0000 7f22c6ffd640 1 -- 172.21.15.70:0/3744428362 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v84)=0 v84) v1 ==== 133+0+0 (secure 0 0 0) 0x7f22b80350c0 con 0x7f22d00d0b90 2023-10-08T07:15:07.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.469+0000 7f22cf59e640 1 -- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f22b006b130 msgr2=0x7f22b006d5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:07.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.469+0000 7f22cf59e640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f22b006b130 0x7f22b006d5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.469+0000 7f22cf59e640 1 -- 172.21.15.70:0/3744428362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d0b90 msgr2=0x7f22d00cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.469+0000 7f22cf59e640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d0b90 0x7f22d00cb6a0 secure :-1 s=READY pgs=2727 cs=0 l=1 rev1=1 crypto rx=0x7f22b80098c0 tx=0x7f22b80046c0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.470+0000 7f22cf59e640 1 -- 172.21.15.70:0/3744428362 shutdown_connections 2023-10-08T07:15:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.470+0000 7f22cf59e640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f22b006b130 0x7f22b006d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.470+0000 7f22cf59e640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22d015ccb0 0x7f22d00b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.470+0000 7f22cf59e640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d00d0b90 0x7f22d00cb6a0 unknown :-1 s=CLOSED pgs=2727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.470+0000 7f22cf59e640 1 --2- 172.21.15.70:0/3744428362 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22d00d00e0 0x7f22d01f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.470+0000 7f22cf59e640 1 -- 172.21.15.70:0/3744428362 >> 172.21.15.70:0/3744428362 conn(0x7f22d0071660 msgr2=0x7f22d015c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.470+0000 7f22cf59e640 1 -- 172.21.15.70:0/3744428362 shutdown_connections 2023-10-08T07:15:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.470+0000 7f22cf59e640 1 -- 172.21.15.70:0/3744428362 wait complete. 2023-10-08T07:15:07.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:07.489 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-10-08T07:15:07.540 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir 1M 2023-10-08T07:15:07.638 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:07.636+0000 7f0812e30040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:15:07.650 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:07.650+0000 7f00a1a2b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:15:07.689 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:07.687+0000 7f0812e30040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:15:07.704 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:07.703+0000 7f00a1a2b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:15:07.738 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:07.737+0000 7f0812e30040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:15:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.743+0000 7f5001b97640 1 Processor -- start 2023-10-08T07:15:07.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.744+0000 7f5001b97640 1 -- start start 2023-10-08T07:15:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.745+0000 7f5001b97640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ffc15aad0 0x7f4ffc0b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.745+0000 7f5001b97640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc15bf30 con 0x7f4ffc06d410 2023-10-08T07:15:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.745+0000 7f5001b97640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc15b320 con 0x7f4ffc0c82b0 2023-10-08T07:15:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.745+0000 7f5001b97640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ffc15b600 con 0x7f4ffc15aad0 2023-10-08T07:15:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.745+0000 7f4ffbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4ffc06d410 0x7f4ffc070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47524/0 (socket says 172.21.15.70:47524) 2023-10-08T07:15:07.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.745+0000 7f4ffbfff640 1 -- 172.21.15.70:0/2923780570 learned_addr learned my addr 172.21.15.70:0/2923780570 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/2923780570 >> v2:172.21.15.181:3300/0 conn(0x7f4ffc15aad0 0x7f4ffc0b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965795621 0 0) 0x7f4ffc15bf30 con 0x7f4ffc06d410 2023-10-08T07:15:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe8003520 con 0x7f4ffc06d410 2023-10-08T07:15:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2688876972 0 0) 0x7f4fe8003520 con 0x7f4ffc06d410 2023-10-08T07:15:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 >> v2:172.21.15.181:3300/0 conn(0x7f4ffc15aad0 msgr2=0x7f4ffc0b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 --2- 172.21.15.70:0/2923780570 >> v2:172.21.15.181:3300/0 conn(0x7f4ffc15aad0 0x7f4ffc0b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 >> v1:172.21.15.181:6789/0 conn(0x7f4ffc0c82b0 legacy=0x7f4ffc06cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc15b8e0 con 0x7f4ffc06d410 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f4ff4001a10 con 0x7f4ffc06d410 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 >> v1:172.21.15.70:6789/0 conn(0x7f4ffc06d410 legacy=0x7f4ffc070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 --2- 172.21.15.70:0/2923780570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fe80039c0 0x7f4fe8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 --2- 172.21.15.70:0/2923780570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fe80047c0 0x7f4fe8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 --2- 172.21.15.70:0/2923780570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fe8005180 0x7f4fe800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc15b600 con 0x7f4fe80039c0 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe800c080 con 0x7f4fe80047c0 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.746+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe800c240 con 0x7f4fe8005180 2023-10-08T07:15:07.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.747+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/2923780570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fe80039c0 0x7f4fe8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.747+0000 7f4ffb7fe640 1 -- 172.21.15.70:0/2923780570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fe8005180 msgr2=0x7f4fe800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.747+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/2923780570 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fe8005180 0x7f4fe800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.747+0000 7f4ff3fff640 1 --2- 172.21.15.70:0/2923780570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fe80047c0 0x7f4fe8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.747+0000 7f4ffb7fe640 1 -- 172.21.15.70:0/2923780570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fe80047c0 msgr2=0x7f4fe8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.747+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/2923780570 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4fe80047c0 0x7f4fe8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.747+0000 7f4ffb7fe640 1 -- 172.21.15.70:0/2923780570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe8003520 con 0x7f4fe80039c0 2023-10-08T07:15:07.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.747+0000 7f4ffbfff640 1 -- 172.21.15.70:0/2923780570 reap_dead start 2023-10-08T07:15:07.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.748+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/2923780570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fe80039c0 0x7f4fe8003de0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009ac0 tx=0x7f4fe80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7703f1adebd1a89f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:07.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.748+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec02c6d0 con 0x7f4fe80039c0 2023-10-08T07:15:07.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.748+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec02c830 con 0x7f4fe80039c0 2023-10-08T07:15:07.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.748+0000 7f4ffaffd640 1 -- 172.21.15.70:0/2923780570 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec02cbc0 con 0x7f4fe80039c0 2023-10-08T07:15:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.749+0000 7f5001b97640 1 -- 172.21.15.70:0/2923780570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fe80039c0 msgr2=0x7f4fe8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.749+0000 7f5001b97640 1 --2- 172.21.15.70:0/2923780570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fe80039c0 0x7f4fe8003de0 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009ac0 tx=0x7f4fe80057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.749+0000 7f5001b97640 1 -- 172.21.15.70:0/2923780570 shutdown_connections 2023-10-08T07:15:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.749+0000 7f5001b97640 1 --2- 172.21.15.70:0/2923780570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fe80039c0 0x7f4fe8003de0 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.749+0000 7f5001b97640 1 -- 172.21.15.70:0/2923780570 >> 172.21.15.70:0/2923780570 conn(0x7f4ffc069d50 msgr2=0x7f4ffc06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.749+0000 7f5001b97640 1 -- 172.21.15.70:0/2923780570 shutdown_connections 2023-10-08T07:15:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.749+0000 7f5001b97640 1 -- 172.21.15.70:0/2923780570 wait complete. 2023-10-08T07:15:07.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.751+0000 7f5001b97640 1 Processor -- start 2023-10-08T07:15:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.751+0000 7f5001b97640 1 -- start start 2023-10-08T07:15:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.752+0000 7f5001b97640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fe80039c0 0x7f4ffc1f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.752+0000 7f5001b97640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ffc1f1990 0x7f4ffc1f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.752+0000 7f5001b97640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ffc1f5cd0 0x7f4ffc15c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.752+0000 7f5001b97640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1f6710 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.752+0000 7f5001b97640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1f6150 con 0x7f4ffc1f1990 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.752+0000 7f5001b97640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1f6430 con 0x7f4fe80039c0 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.752+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ffc1f5cd0 0x7f4ffc15c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.752+0000 7f4ffbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ffc1f5cd0 0x7f4ffc15c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51300/0 (socket says 172.21.15.70:51300) 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.752+0000 7f4ffbfff640 1 -- 172.21.15.70:0/619222768 learned_addr learned my addr 172.21.15.70:0/619222768 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fe80039c0 0x7f4ffc1f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ff3fff640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ffc1f1990 0x7f4ffc1f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ffbfff640 1 -- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fe80039c0 msgr2=0x7f4ffc1f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ffbfff640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4fe80039c0 0x7f4ffc1f2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ffbfff640 1 -- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ffc1f1990 msgr2=0x7f4ffc1f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:07.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ffbfff640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ffc1f1990 0x7f4ffc1f5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ffbfff640 1 -- 172.21.15.70:0/619222768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc1f2340 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ffbfff640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ffc1f5cd0 0x7f4ffc15c980 secure :-1 s=READY pgs=2730 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4001390 tx=0x7f4ff400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff400a610 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff400a790 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff400ab40 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.753+0000 7f5001b97640 1 -- 172.21.15.70:0/619222768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc1f2620 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.754+0000 7f5001b97640 1 -- 172.21.15.70:0/619222768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc156840 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.755+0000 7f5001b97640 1 -- 172.21.15.70:0/619222768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ffc1f2860 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.756+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 139) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f4ff400aca0 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.757+0000 7f4ff8ff9640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 0x7f4fdc06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.758+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 7009+0+0 (secure 0 0 0) 0x7f4ff4089800 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.758+0000 7f4ffb7fe640 1 -- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 msgr2=0x7f4fdc06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245028292 2023-10-08T07:15:07.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.758+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 0x7f4fdc06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:07.759 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:07.758+0000 7f00a1a2b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:15:07.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.762+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4ffc1f2860 con 0x7f4ffc1f5cd0 2023-10-08T07:15:07.800 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:07.799+0000 7f0812e30040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:15:07.824 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:07.823+0000 7f00a1a2b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:15:07.849 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:07.848+0000 7f0812e30040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:15:07.875 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:07.874+0000 7f00a1a2b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:15:07.958 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:07.957+0000 7f0812e30040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:15:07.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.959+0000 7f4ffb7fe640 1 -- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 msgr2=0x7f4fdc06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245028292 2023-10-08T07:15:07.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.959+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 0x7f4fdc06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:15:07.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:07.966+0000 7f5001b97640 1 -- 172.21.15.70:0/619222768 --> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ffc1f65a0 con 0x7f4fdc06b100 2023-10-08T07:15:07.988 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:07.988+0000 7f00a1a2b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:15:08.020 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:08.019+0000 7f0812e30040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:08.051 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:08.050+0000 7f00a1a2b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:08.069 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:08.068+0000 7f0812e30040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:15:08.101 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:08.100+0000 7f00a1a2b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:15:08.128 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:08.127+0000 7f0812e30040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:15:08.157 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:08.156+0000 7f00a1a2b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:15:08.178 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:08.177+0000 7f0812e30040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:15:08.207 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:08.206+0000 7f00a1a2b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:15:08.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:15:08.354 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:15:08.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:08.360+0000 7f4ffb7fe640 1 -- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 msgr2=0x7f4fdc06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245028292 2023-10-08T07:15:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:08.360+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 0x7f4fdc06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T07:15:08.574 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:08.573+0000 7f0812e30040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:15:08.601 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:08.600+0000 7f00a1a2b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:15:08.733 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:08.730+0000 7f0812e30040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:15:08.757 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:08.756+0000 7f00a1a2b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:15:08.840 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:08.838+0000 7f0812e30040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:15:08.865 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:08.864+0000 7f00a1a2b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:15:08.889 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:08.888+0000 7f0812e30040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:15:08.914 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:08.913+0000 7f00a1a2b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:15:08.996 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:08.995+0000 7f0812e30040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:09.021 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:09.020+0000 7f00a1a2b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:09.103 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:09.102+0000 7f0812e30040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:15:09.127 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:09.127+0000 7f00a1a2b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:15:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:09.161+0000 7f4ffb7fe640 1 -- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 msgr2=0x7f4fdc06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/4245028292 2023-10-08T07:15:09.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:09.161+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 0x7f4fdc06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T07:15:09.260 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:09.258+0000 7f0812e30040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:09.283 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:09.282+0000 7f00a1a2b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:09.309 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:09.307+0000 7f0812e30040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:09.332 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:09.332+0000 7f00a1a2b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:09.429 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:09.427+0000 7f0812e30040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:15:09.452 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:09.451+0000 7f00a1a2b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:15:10.255 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:10.253+0000 7f0812e30040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:15:10.257 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:10.256+0000 7f00a1a2b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:15:10.547 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:10.546+0000 7f0812e30040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:15:10.550 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:10.549+0000 7f00a1a2b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:15:10.606 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:10.604+0000 7f0812e30040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:15:10.607 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:10.606+0000 7f00a1a2b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:15:10.720 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:10.718+0000 7f0812e30040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:15:10.720 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:10.719+0000 7f00a1a2b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:15:10.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:10.738+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 140) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f4ff4054400 con 0x7f4ffc1f5cd0 2023-10-08T07:15:10.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:10.739+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 msgr2=0x7f4fdc06d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:10.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:10.739+0000 7f4ff8ff9640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/4245028292,v1:172.21.15.181:6833/4245028292] conn(0x7f4fdc06b100 0x7f4fdc06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:11.013 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.011+0000 7f079097b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.013 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.011+0000 7f079097b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.013 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.011+0000 7f079097b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.013 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.011+0000 7f079097b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.013 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.011+0000 7f079097b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.013 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.012+0000 7f078e977640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.013 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.012+0000 7f078e977640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.013 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.012+0000 7f078e977640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.014 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.012+0000 7f078e977640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.014 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.012+0000 7f078e977640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.032 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.031+0000 7f079097b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.033 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.031+0000 7f079097b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.033 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.031+0000 7f079097b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.033 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.031+0000 7f079097b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.033 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.031+0000 7f079097b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.033 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.032+0000 7f078e977640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.033 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.032+0000 7f078e977640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.033 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.032+0000 7f078e977640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.034 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.032+0000 7f078e977640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.034 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.032+0000 7f078e977640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:11.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:11.802+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 141) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f4ff405fb60 con 0x7f4ffc1f5cd0 2023-10-08T07:15:11.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:11.803+0000 7f4ff8ff9640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f4fdc035800 0x7f4fdc077d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:11.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:11.803+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 --> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ffc1f65a0 con 0x7f4fdc035800 2023-10-08T07:15:11.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:11.806+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f4fdc035800 0x7f4fdc077d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:11.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:11.824+0000 7f4ffb7fe640 1 --2- 172.21.15.70:0/619222768 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f4fdc035800 0x7f4fdc077d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4fec036000 tx=0x7f4fec030000 comp rx=0 tx=0).ready entity=mgr.7904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:11.832 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:11.831+0000 7f07989cb640 -1 mgr.server reply reply (22) Invalid argument filesystem argument is required when there is more than one file system 2023-10-08T07:15:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:11.832+0000 7f4ff8ff9640 1 -- 172.21.15.70:0/619222768 <== mgr.7904 v2:172.21.15.181:6832/2621244427 1 ==== mgr_command_reply(tid 0: -22 filesystem argument is required when there is more than one file system) v1 ==== 79+0+0 (secure 0 0 0) 0x7f4ffc1f65a0 con 0x7f4fdc035800 2023-10-08T07:15:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: filesystem argument is required when there is more than one file system 2023-10-08T07:15:11.847 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-10-08T07:15:11.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:11.848 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-10-08T07:15:11.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T07:15:12.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.111+0000 7f20b0d35640 1 Processor -- start 2023-10-08T07:15:12.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.111+0000 7f20b0d35640 1 -- start start 2023-10-08T07:15:12.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20b0d35640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20ac0caef0 0x7f20ac0cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20b0d35640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f20ac0cdc30 0x7f20ac0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20b0d35640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f20ac0cce90 0x7f20ac159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20b0d35640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ac0cb810 con 0x7f20ac0cdc30 2023-10-08T07:15:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20b0d35640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ac15ac90 con 0x7f20ac0cce90 2023-10-08T07:15:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20b0d35640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac15a6d0 con 0x7f20ac0caef0 2023-10-08T07:15:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20aaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f20ac0cdc30 0x7f20ac0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:12.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20aaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f20ac0cdc30 0x7f20ac0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51306/0 (socket says 172.21.15.70:51306) 2023-10-08T07:15:12.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20aaffd640 1 -- 172.21.15.70:0/1868101123 learned_addr learned my addr 172.21.15.70:0/1868101123 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:12.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20aaffd640 1 -- 172.21.15.70:0/1868101123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20ac0caef0 msgr2=0x7f20ac0cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:12.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20aaffd640 1 --2- 172.21.15.70:0/1868101123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20ac0caef0 0x7f20ac0cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20aaffd640 1 -- 172.21.15.70:0/1868101123 >> v2:172.21.15.181:3300/0 conn(0x7f20ac0cce90 msgr2=0x7f20ac159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20aaffd640 1 --2- 172.21.15.70:0/1868101123 >> v2:172.21.15.181:3300/0 conn(0x7f20ac0cce90 0x7f20ac159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.113+0000 7f20aaffd640 1 -- 172.21.15.70:0/1868101123 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac15a9b0 con 0x7f20ac0cdc30 2023-10-08T07:15:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.114+0000 7f20aaffd640 1 --2- 172.21.15.70:0/1868101123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0cdc30 0x7f20ac0cc950 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0b4510 tx=0x7f20a002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ce6ea2484a7cfc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.115+0000 7f20aa7fc640 1 -- 172.21.15.70:0/1868101123 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0031890 con 0x7f20ac0cdc30 2023-10-08T07:15:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.115+0000 7f20aa7fc640 1 -- 172.21.15.70:0/1868101123 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a00319f0 con 0x7f20ac0cdc30 2023-10-08T07:15:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.115+0000 7f20aa7fc640 1 -- 172.21.15.70:0/1868101123 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0031d80 con 0x7f20ac0cdc30 2023-10-08T07:15:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.115+0000 7f20b0d35640 1 -- 172.21.15.70:0/1868101123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0cdc30 msgr2=0x7f20ac0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.115+0000 7f20b0d35640 1 --2- 172.21.15.70:0/1868101123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0cdc30 0x7f20ac0cc950 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0b4510 tx=0x7f20a002b930 comp rx=0 tx=0).stop 2023-10-08T07:15:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.116+0000 7f20b0d35640 1 -- 172.21.15.70:0/1868101123 shutdown_connections 2023-10-08T07:15:12.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.116+0000 7f20b0d35640 1 --2- 172.21.15.70:0/1868101123 >> v2:172.21.15.181:3300/0 conn(0x7f20ac0cce90 0x7f20ac159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.116+0000 7f20b0d35640 1 --2- 172.21.15.70:0/1868101123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0cdc30 0x7f20ac0cc950 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.116+0000 7f20b0d35640 1 --2- 172.21.15.70:0/1868101123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20ac0caef0 0x7f20ac0cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.116+0000 7f20b0d35640 1 -- 172.21.15.70:0/1868101123 >> 172.21.15.70:0/1868101123 conn(0x7f20ac06c010 msgr2=0x7f20ac06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:12.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.116+0000 7f20b0d35640 1 -- 172.21.15.70:0/1868101123 shutdown_connections 2023-10-08T07:15:12.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.116+0000 7f20b0d35640 1 -- 172.21.15.70:0/1868101123 wait complete. 2023-10-08T07:15:12.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.117+0000 7f20b0d35640 1 Processor -- start 2023-10-08T07:15:12.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.118+0000 7f20b0d35640 1 -- start start 2023-10-08T07:15:12.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.118+0000 7f20b0d35640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0caef0 0x7f20ac14b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.118+0000 7f20b0d35640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ac0cce90 0x7f20ac14c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20b0d35640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20ac0cdc30 0x7f20ac0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20b0d35640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac14bf00 con 0x7f20ac0caef0 2023-10-08T07:15:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20b0d35640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0a7640 con 0x7f20ac0cce90 2023-10-08T07:15:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20b0d35640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac0b3b80 con 0x7f20ac0cdc30 2023-10-08T07:15:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0caef0 0x7f20ac14b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0caef0 0x7f20ac14b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51322/0 (socket says 172.21.15.70:51322) 2023-10-08T07:15:12.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20ab7fe640 1 -- 172.21.15.70:0/1343594304 learned_addr learned my addr 172.21.15.70:0/1343594304 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:12.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20ab7fe640 1 -- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20ac0cdc30 msgr2=0x7f20ac0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:12.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20ab7fe640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20ac0cdc30 0x7f20ac0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20ab7fe640 1 -- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ac0cce90 msgr2=0x7f20ac14c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:12.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20ab7fe640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ac0cce90 0x7f20ac14c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.119+0000 7f20ab7fe640 1 -- 172.21.15.70:0/1343594304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00094d0 con 0x7f20ac0caef0 2023-10-08T07:15:12.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.120+0000 7f20ab7fe640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0caef0 0x7f20ac14b9c0 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f209c00b4a0 tx=0x7f209c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:12.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.120+0000 7f2093fff640 1 -- 172.21.15.70:0/1343594304 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c010610 con 0x7f20ac0caef0 2023-10-08T07:15:12.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.120+0000 7f2093fff640 1 -- 172.21.15.70:0/1343594304 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c010770 con 0x7f20ac0caef0 2023-10-08T07:15:12.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.120+0000 7f2093fff640 1 -- 172.21.15.70:0/1343594304 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c010b20 con 0x7f20ac0caef0 2023-10-08T07:15:12.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.120+0000 7f20b0d35640 1 -- 172.21.15.70:0/1343594304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ac0a7900 con 0x7f20ac0caef0 2023-10-08T07:15:12.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.120+0000 7f20b0d35640 1 -- 172.21.15.70:0/1343594304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ac14f170 con 0x7f20ac0caef0 2023-10-08T07:15:12.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.121+0000 7f20b0d35640 1 -- 172.21.15.70:0/1343594304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ac0a7e20 con 0x7f20ac0caef0 2023-10-08T07:15:12.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.122+0000 7f2093fff640 1 -- 172.21.15.70:0/1343594304 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 141) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f209c010c80 con 0x7f20ac0caef0 2023-10-08T07:15:12.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.123+0000 7f2093fff640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f209806ad10 0x7f209806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:12.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.123+0000 7f2093fff640 1 -- 172.21.15.70:0/1343594304 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7f209c089690 con 0x7f20ac0caef0 2023-10-08T07:15:12.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.124+0000 7f20aaffd640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f209806ad10 0x7f209806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:12.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.125+0000 7f20aaffd640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f209806ad10 0x7f209806d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f20a0036000 tx=0x7f20a0030000 comp rx=0 tx=0).ready entity=mgr.7904 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:12.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.128+0000 7f2093fff640 1 -- 172.21.15.70:0/1343594304 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f20ac0a7e20 con 0x7f20ac0caef0 2023-10-08T07:15:12.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.366+0000 7f20b0d35640 1 -- 172.21.15.70:0/1343594304 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f20ac0a7e20 con 0x7f20ac0caef0 2023-10-08T07:15:12.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.802+0000 7f2093fff640 1 -- 172.21.15.70:0/1343594304 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v142) v1 ==== 93+0+0 (secure 0 0 0) 0x7f20ac0a7e20 con 0x7f20ac0caef0 2023-10-08T07:15:12.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.806+0000 7f20b0d35640 1 -- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f209806ad10 msgr2=0x7f209806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:12.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.806+0000 7f20b0d35640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f209806ad10 0x7f209806d1d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f20a0036000 tx=0x7f20a0030000 comp rx=0 tx=0).stop 2023-10-08T07:15:12.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.806+0000 7f20b0d35640 1 -- 172.21.15.70:0/1343594304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0caef0 msgr2=0x7f20ac14b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:12.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.806+0000 7f20b0d35640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0caef0 0x7f20ac14b9c0 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f209c00b4a0 tx=0x7f209c00b960 comp rx=0 tx=0).stop 2023-10-08T07:15:12.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.806+0000 7f20b0d35640 1 -- 172.21.15.70:0/1343594304 shutdown_connections 2023-10-08T07:15:12.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.806+0000 7f20b0d35640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f209806ad10 0x7f209806d1d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.806+0000 7f20b0d35640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f20ac0cdc30 0x7f20ac0b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.806+0000 7f20b0d35640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f20ac0cce90 0x7f20ac14c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.806+0000 7f20b0d35640 1 --2- 172.21.15.70:0/1343594304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20ac0caef0 0x7f20ac14b9c0 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:12.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.806+0000 7f20b0d35640 1 -- 172.21.15.70:0/1343594304 >> 172.21.15.70:0/1343594304 conn(0x7f20ac06c010 msgr2=0x7f20ac0ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:12.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.807+0000 7f20b0d35640 1 -- 172.21.15.70:0/1343594304 shutdown_connections 2023-10-08T07:15:12.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:12.808+0000 7f20b0d35640 1 -- 172.21.15.70:0/1343594304 wait complete. 2023-10-08T07:15:12.830 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T07:15:12.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:15:12.930 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:15:13.021 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T07:15:13.021 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:15:13.021 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:15:13.073 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:15:13.073 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:15:13.165 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:13.332 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:13.331+0000 7f8c4740e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:15:13.357 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:13.357+0000 7fb1b0a32040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:15:13.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.363+0000 7fb479af3640 1 Processor -- start 2023-10-08T07:15:13.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.363+0000 7fb479af3640 1 -- start start 2023-10-08T07:15:13.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.364+0000 7fb479af3640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb4740adc70 0x7fb4740b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.364+0000 7fb479af3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47415f7d0 con 0x7fb47415ab30 2023-10-08T07:15:13.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.364+0000 7fb479af3640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47415efb0 con 0x7fb47415af10 2023-10-08T07:15:13.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.364+0000 7fb479af3640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb47415f210 con 0x7fb4740adc70 2023-10-08T07:15:13.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.365+0000 7fb478af1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb47415ab30 0x7fb47415e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:47528/0 (socket says 172.21.15.70:47528) 2023-10-08T07:15:13.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.365+0000 7fb478af1640 1 -- 172.21.15.70:0/228103936 learned_addr learned my addr 172.21.15.70:0/228103936 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:13.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.365+0000 7fb473fff640 1 --2- 172.21.15.70:0/228103936 >> v2:172.21.15.181:3300/0 conn(0x7fb4740adc70 0x7fb4740b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:13.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.365+0000 7fb4737fe640 1 -- 172.21.15.70:0/228103936 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229873016 0 0) 0x7fb47415f7d0 con 0x7fb47415ab30 2023-10-08T07:15:13.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.365+0000 7fb4737fe640 1 -- 172.21.15.70:0/228103936 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb464003520 con 0x7fb47415ab30 2023-10-08T07:15:13.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.365+0000 7fb473fff640 1 -- 172.21.15.70:0/228103936 >> v1:172.21.15.181:6789/0 conn(0x7fb47415af10 legacy=0x7fb47406cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:13.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.365+0000 7fb473fff640 1 -- 172.21.15.70:0/228103936 >> v1:172.21.15.70:6789/0 conn(0x7fb47415ab30 legacy=0x7fb47415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:13.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.365+0000 7fb473fff640 1 -- 172.21.15.70:0/228103936 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47415f4f0 con 0x7fb4740adc70 2023-10-08T07:15:13.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.367+0000 7fb473fff640 1 --2- 172.21.15.70:0/228103936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4740adc70 0x7fb4740b00c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fb4600097a0 tx=0x7fb46002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d4def707a0f8062 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:13.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.367+0000 7fb4737fe640 1 -- 172.21.15.70:0/228103936 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460030610 con 0x7fb4740adc70 2023-10-08T07:15:13.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.368+0000 7fb4737fe640 1 -- 172.21.15.70:0/228103936 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460030770 con 0x7fb4740adc70 2023-10-08T07:15:13.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.368+0000 7fb4737fe640 1 -- 172.21.15.70:0/228103936 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460030ae0 con 0x7fb4740adc70 2023-10-08T07:15:13.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.368+0000 7fb479af3640 1 -- 172.21.15.70:0/228103936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4740adc70 msgr2=0x7fb4740b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:13.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.368+0000 7fb479af3640 1 --2- 172.21.15.70:0/228103936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4740adc70 0x7fb4740b00c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fb4600097a0 tx=0x7fb46002b960 comp rx=0 tx=0).stop 2023-10-08T07:15:13.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.368+0000 7fb479af3640 1 -- 172.21.15.70:0/228103936 shutdown_connections 2023-10-08T07:15:13.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.368+0000 7fb479af3640 1 --2- 172.21.15.70:0/228103936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb4740adc70 0x7fb4740b00c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.368+0000 7fb479af3640 1 -- 172.21.15.70:0/228103936 >> 172.21.15.70:0/228103936 conn(0x7fb4740cb0c0 msgr2=0x7fb4740cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:13.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.368+0000 7fb479af3640 1 -- 172.21.15.70:0/228103936 shutdown_connections 2023-10-08T07:15:13.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.368+0000 7fb479af3640 1 -- 172.21.15.70:0/228103936 wait complete. 2023-10-08T07:15:13.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.370+0000 7fb479af3640 1 Processor -- start 2023-10-08T07:15:13.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.370+0000 7fb479af3640 1 -- start start 2023-10-08T07:15:13.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.371+0000 7fb479af3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4740adc70 0x7fb474150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.371+0000 7fb479af3640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47415ab30 0x7fb4740af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.371+0000 7fb479af3640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb47415af10 0x7fb4740c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.371+0000 7fb479af3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740ae050 con 0x7fb4740adc70 2023-10-08T07:15:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.371+0000 7fb479af3640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740ae2a0 con 0x7fb47415ab30 2023-10-08T07:15:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.371+0000 7fb479af3640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740ae580 con 0x7fb47415af10 2023-10-08T07:15:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.371+0000 7fb478af1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4740adc70 0x7fb474150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.371+0000 7fb478af1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4740adc70 0x7fb474150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51350/0 (socket says 172.21.15.70:51350) 2023-10-08T07:15:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.371+0000 7fb478af1640 1 -- 172.21.15.70:0/595301005 learned_addr learned my addr 172.21.15.70:0/595301005 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb473fff640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47415ab30 0x7fb4740af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb478af1640 1 -- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb47415af10 msgr2=0x7fb4740c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb478af1640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb47415af10 0x7fb4740c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb478af1640 1 -- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47415ab30 msgr2=0x7fb4740af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:13.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb478af1640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47415ab30 0x7fb4740af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb478af1640 1 -- 172.21.15.70:0/595301005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb460009400 con 0x7fb4740adc70 2023-10-08T07:15:13.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb478af1640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4740adc70 0x7fb474150e20 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fb468001eb0 tx=0x7fb46800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:13.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb4717fa640 1 -- 172.21.15.70:0/595301005 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468005ce0 con 0x7fb4740adc70 2023-10-08T07:15:13.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb4717fa640 1 -- 172.21.15.70:0/595301005 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468005e40 con 0x7fb4740adc70 2023-10-08T07:15:13.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb4717fa640 1 -- 172.21.15.70:0/595301005 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb468007270 con 0x7fb4740adc70 2023-10-08T07:15:13.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.372+0000 7fb479af3640 1 -- 172.21.15.70:0/595301005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4740ae860 con 0x7fb4740adc70 2023-10-08T07:15:13.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.373+0000 7fb479af3640 1 -- 172.21.15.70:0/595301005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4740aee20 con 0x7fb4740adc70 2023-10-08T07:15:13.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.374+0000 7fb479af3640 1 -- 172.21.15.70:0/595301005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47404e160 con 0x7fb4740adc70 2023-10-08T07:15:13.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.375+0000 7fb4717fa640 1 -- 172.21.15.70:0/595301005 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 142) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fb468007490 con 0x7fb4740adc70 2023-10-08T07:15:13.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.376+0000 7fb4717fa640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fb45806b0b0 0x7fb45806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.376+0000 7fb4717fa640 1 -- 172.21.15.70:0/595301005 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7fb4680895e0 con 0x7fb4740adc70 2023-10-08T07:15:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.376+0000 7fb473fff640 1 -- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fb45806b0b0 msgr2=0x7fb45806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:13.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.376+0000 7fb473fff640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fb45806b0b0 0x7fb45806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:13.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.384+0000 7fb4717fa640 1 -- 172.21.15.70:0/595301005 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb4680577a0 con 0x7fb4740adc70 2023-10-08T07:15:13.387 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:13.386+0000 7f8c4740e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:15:13.418 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:13.417+0000 7fb1b0a32040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:15:13.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.577+0000 7fb473fff640 1 -- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fb45806b0b0 msgr2=0x7fb45806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:13.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.577+0000 7fb473fff640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fb45806b0b0 0x7fb45806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:15:13.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.580+0000 7fb479af3640 1 -- 172.21.15.70:0/595301005 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4740ada50 con 0x7fb4740adc70 2023-10-08T07:15:13.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.583+0000 7fb4717fa640 1 -- 172.21.15.70:0/595301005 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v259) v1 ==== 74+0+21248 (secure 0 0 0) 0x7fb468057330 con 0x7fb4740adc70 2023-10-08T07:15:13.584 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:13.584 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":259,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:10.723139+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:14:56.178783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:14:56.977158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/2690144496":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/3464560328":"2023-10-09T07:15:10.723067+0000","172.21.15.181:6832/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/807583929":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/859617464":"2023-10-09T07:15:10.723067+0000","172.21.15.181:6833/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/735805106":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/957356199":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/333977458":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2403193161":"2023-10-09T07:15:10.723067+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.585+0000 7fb479af3640 1 -- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fb45806b0b0 msgr2=0x7fb45806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.585+0000 7fb479af3640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fb45806b0b0 0x7fb45806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.585+0000 7fb479af3640 1 -- 172.21.15.70:0/595301005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4740adc70 msgr2=0x7fb474150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.585+0000 7fb479af3640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4740adc70 0x7fb474150e20 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7fb468001eb0 tx=0x7fb46800c960 comp rx=0 tx=0).stop 2023-10-08T07:15:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.586+0000 7fb479af3640 1 -- 172.21.15.70:0/595301005 shutdown_connections 2023-10-08T07:15:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.586+0000 7fb479af3640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fb45806b0b0 0x7fb45806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.586+0000 7fb479af3640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb47415af10 0x7fb4740c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.586+0000 7fb479af3640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb47415ab30 0x7fb4740af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.586+0000 7fb479af3640 1 --2- 172.21.15.70:0/595301005 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4740adc70 0x7fb474150e20 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.586+0000 7fb479af3640 1 -- 172.21.15.70:0/595301005 >> 172.21.15.70:0/595301005 conn(0x7fb4740cb0c0 msgr2=0x7fb47406cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.586+0000 7fb479af3640 1 -- 172.21.15.70:0/595301005 shutdown_connections 2023-10-08T07:15:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.586+0000 7fb479af3640 1 -- 172.21.15.70:0/595301005 wait complete. 2023-10-08T07:15:13.605 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T07:14:41.606127+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T07:14:42.867820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '254', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-08T07:14:56.178783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '257', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-08T07:14:56.977158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '258', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:15:13.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:15:13.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.806+0000 7fda58664640 1 Processor -- start 2023-10-08T07:15:13.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.807+0000 7fda58664640 1 -- start start 2023-10-08T07:15:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.808+0000 7fda58664640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5015ac00 0x7fda500d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.808+0000 7fda58664640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fda500cdb50 0x7fda500b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.808+0000 7fda58664640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda500ced90 con 0x7fda500afe90 2023-10-08T07:15:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.808+0000 7fda58664640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda500cf070 con 0x7fda500cdb50 2023-10-08T07:15:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.808+0000 7fda58664640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda500cf350 con 0x7fda5015ac00 2023-10-08T07:15:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.809+0000 7fda55bff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fda500afe90 0x7fda500cd440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49474/0 (socket says 172.21.15.70:49474) 2023-10-08T07:15:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.809+0000 7fda55bff640 1 -- 172.21.15.70:0/2398950451 learned_addr learned my addr 172.21.15.70:0/2398950451 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:13.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.809+0000 7fda56400640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5015ac00 0x7fda500d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:13.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.809+0000 7fda56c01640 1 --2- 172.21.15.70:0/2398950451 >> v2:172.21.15.181:3300/0 conn(0x7fda500cdb50 0x7fda500b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:13.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.809+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774551937 0 0) 0x7fda500ced90 con 0x7fda500afe90 2023-10-08T07:15:13.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.809+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda3c003520 con 0x7fda500afe90 2023-10-08T07:15:13.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.809+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4081514711 0 0) 0x7fda3c003520 con 0x7fda500afe90 2023-10-08T07:15:13.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5015ac00 msgr2=0x7fda500d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:13.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5015ac00 0x7fda500d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 >> v2:172.21.15.181:3300/0 conn(0x7fda500cdb50 msgr2=0x7fda500b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:13.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 --2- 172.21.15.70:0/2398950451 >> v2:172.21.15.181:3300/0 conn(0x7fda500cdb50 0x7fda500b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda500cf630 con 0x7fda500afe90 2023-10-08T07:15:13.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fda40001670 con 0x7fda500afe90 2023-10-08T07:15:13.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda56c01640 1 --2- 172.21.15.70:0/2398950451 >> v2:172.21.15.181:3300/0 conn(0x7fda500cdb50 0x7fda500b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:15:13.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 >> v1:172.21.15.70:6789/0 conn(0x7fda500afe90 legacy=0x7fda500cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:13.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5015ac00 0x7fda3c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda3c003ef0 0x7fda3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda3c004d00 0x7fda3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda500cf070 con 0x7fda3c004d00 2023-10-08T07:15:13.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda500cf350 con 0x7fda3c003ef0 2023-10-08T07:15:13.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c00fdc0 con 0x7fda5015ac00 2023-10-08T07:15:13.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda56c01640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda3c004d00 0x7fda3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:13.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda56400640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5015ac00 0x7fda3c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:13.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda55bff640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda3c003ef0 0x7fda3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:13.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda56c01640 1 -- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5015ac00 msgr2=0x7fda3c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:13.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda56c01640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda5015ac00 0x7fda3c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda56c01640 1 -- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda3c003ef0 msgr2=0x7fda3c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:13.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda56c01640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda3c003ef0 0x7fda3c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda56c01640 1 -- 172.21.15.70:0/2398950451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c003520 con 0x7fda3c004d00 2023-10-08T07:15:13.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.810+0000 7fda55bff640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda3c003ef0 0x7fda3c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:13.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.812+0000 7fda56c01640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda3c004d00 0x7fda3c005180 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7fda500cea10 tx=0x7fda3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9633a65fd915ec4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:13.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.812+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4802cd90 con 0x7fda3c004d00 2023-10-08T07:15:13.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.812+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4800f040 con 0x7fda3c004d00 2023-10-08T07:15:13.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.812+0000 7fda553fe640 1 -- 172.21.15.70:0/2398950451 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48034640 con 0x7fda3c004d00 2023-10-08T07:15:13.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.813+0000 7fda58664640 1 -- 172.21.15.70:0/2398950451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda3c004d00 msgr2=0x7fda3c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:13.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.813+0000 7fda58664640 1 --2- 172.21.15.70:0/2398950451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda3c004d00 0x7fda3c005180 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7fda500cea10 tx=0x7fda3c0057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.813+0000 7fda56c01640 1 -- 172.21.15.70:0/2398950451 reap_dead start 2023-10-08T07:15:13.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.813+0000 7fda58664640 1 -- 172.21.15.70:0/2398950451 shutdown_connections 2023-10-08T07:15:13.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.813+0000 7fda58664640 1 -- 172.21.15.70:0/2398950451 >> 172.21.15.70:0/2398950451 conn(0x7fda5006e650 msgr2=0x7fda5006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:13.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.813+0000 7fda58664640 1 -- 172.21.15.70:0/2398950451 shutdown_connections 2023-10-08T07:15:13.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.813+0000 7fda58664640 1 -- 172.21.15.70:0/2398950451 wait complete. 2023-10-08T07:15:13.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.816+0000 7fda58664640 1 Processor -- start 2023-10-08T07:15:13.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.816+0000 7fda58664640 1 -- start start 2023-10-08T07:15:13.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda58664640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda50154a80 0x7fda500b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda58664640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda500b3450 0x7fda500c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda58664640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda500c6cc0 0x7fda50155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda58664640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda500c7140 con 0x7fda50154a80 2023-10-08T07:15:13.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda58664640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5015b340 con 0x7fda500c6cc0 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda58664640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5015b620 con 0x7fda500b3450 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda56400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda50154a80 0x7fda500b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda56400640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda50154a80 0x7fda500b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56450/0 (socket says 172.21.15.70:56450) 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda56400640 1 -- 172.21.15.70:0/265633938 learned_addr learned my addr 172.21.15.70:0/265633938 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda56c01640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda500c6cc0 0x7fda50155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda56400640 1 -- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda500b3450 msgr2=0x7fda500c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda56400640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda500b3450 0x7fda500c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda56400640 1 -- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda500c6cc0 msgr2=0x7fda50155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda56400640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda500c6cc0 0x7fda50155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.818+0000 7fda56400640 1 -- 172.21.15.70:0/265633938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5015b900 con 0x7fda50154a80 2023-10-08T07:15:13.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.819+0000 7fda56400640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda50154a80 0x7fda500b5430 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7fda4c011600 tx=0x7fda4c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.822+0000 7fda46ffd640 1 -- 172.21.15.70:0/265633938 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00f920 con 0x7fda50154a80 2023-10-08T07:15:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.822+0000 7fda46ffd640 1 -- 172.21.15.70:0/265633938 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c00fa80 con 0x7fda50154a80 2023-10-08T07:15:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.822+0000 7fda46ffd640 1 -- 172.21.15.70:0/265633938 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00fdf0 con 0x7fda50154a80 2023-10-08T07:15:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.822+0000 7fda58664640 1 -- 172.21.15.70:0/265633938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5015bbe0 con 0x7fda50154a80 2023-10-08T07:15:13.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.823+0000 7fda58664640 1 -- 172.21.15.70:0/265633938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5015bec0 con 0x7fda50154a80 2023-10-08T07:15:13.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.824+0000 7fda58664640 1 -- 172.21.15.70:0/265633938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5015b760 con 0x7fda50154a80 2023-10-08T07:15:13.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.825+0000 7fda46ffd640 1 -- 172.21.15.70:0/265633938 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 142) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fda4c015460 con 0x7fda50154a80 2023-10-08T07:15:13.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.825+0000 7fda46ffd640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fda3406b0b0 0x7fda3406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:13.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.825+0000 7fda46ffd640 1 -- 172.21.15.70:0/265633938 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7fda4c08a640 con 0x7fda50154a80 2023-10-08T07:15:13.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.826+0000 7fda55bff640 1 -- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fda3406b0b0 msgr2=0x7fda3406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:13.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.826+0000 7fda55bff640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fda3406b0b0 0x7fda3406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:13.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:13.830+0000 7fda46ffd640 1 -- 172.21.15.70:0/265633938 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda5015b760 con 0x7fda50154a80 2023-10-08T07:15:13.843 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:13.842+0000 7f8c4740e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:15:13.893 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:13.892+0000 7fb1b0a32040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:15:13.894 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:13.893+0000 7f8c4740e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:15:13.945 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:13.944+0000 7f8c4740e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:15:13.946 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:13.946+0000 7fb1b0a32040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:15:14.000 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:13.999+0000 7fb1b0a32040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:15:14.005 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:14.004+0000 7f8c4740e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:15:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.020+0000 7fda58664640 1 -- 172.21.15.70:0/265633938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda5015b760 con 0x7fda50154a80 2023-10-08T07:15:14.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.021+0000 7fda46ffd640 1 -- 172.21.15.70:0/265633938 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v259) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda5015b760 con 0x7fda50154a80 2023-10-08T07:15:14.022 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:15:14.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.023+0000 7fda58664640 1 -- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fda3406b0b0 msgr2=0x7fda3406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.023+0000 7fda58664640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fda3406b0b0 0x7fda3406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.024+0000 7fda58664640 1 -- 172.21.15.70:0/265633938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda50154a80 msgr2=0x7fda500b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.024+0000 7fda58664640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda50154a80 0x7fda500b5430 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7fda4c011600 tx=0x7fda4c011ac0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.024+0000 7fda58664640 1 -- 172.21.15.70:0/265633938 shutdown_connections 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.024+0000 7fda58664640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fda3406b0b0 0x7fda3406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.024+0000 7fda58664640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda500c6cc0 0x7fda50155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.024+0000 7fda58664640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda500b3450 0x7fda500c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.024+0000 7fda58664640 1 --2- 172.21.15.70:0/265633938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda50154a80 0x7fda500b5430 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.024+0000 7fda58664640 1 -- 172.21.15.70:0/265633938 >> 172.21.15.70:0/265633938 conn(0x7fda5006e650 msgr2=0x7fda5006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.024+0000 7fda58664640 1 -- 172.21.15.70:0/265633938 shutdown_connections 2023-10-08T07:15:14.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.024+0000 7fda58664640 1 -- 172.21.15.70:0/265633938 wait complete. 2023-10-08T07:15:14.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:15:14.055 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:14.054+0000 7f8c4740e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:15:14.064 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:14.064+0000 7fb1b0a32040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:15:14.116 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:14.115+0000 7fb1b0a32040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:15:14.165 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:14.164+0000 7f8c4740e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:15:14.227 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:14.226+0000 7f8c4740e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:14.231 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:14.230+0000 7fb1b0a32040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:15:14.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.248+0000 7fed8917c640 1 Processor -- start 2023-10-08T07:15:14.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.248+0000 7fed8917c640 1 -- start start 2023-10-08T07:15:14.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed8917c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed8406c3a0 0x7fed840c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed8917c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed8415aa00 0x7fed840b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed8917c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8415b310 con 0x7fed8406db60 2023-10-08T07:15:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed8917c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed840ca230 con 0x7fed8406c3a0 2023-10-08T07:15:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed8917c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8406d290 con 0x7fed8415aa00 2023-10-08T07:15:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed7bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fed8406db60 0x7fed8406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49480/0 (socket says 172.21.15.70:49480) 2023-10-08T07:15:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed7bfff640 1 -- 172.21.15.70:0/1143907401 learned_addr learned my addr 172.21.15.70:0/1143907401 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed82d9d640 1 --2- 172.21.15.70:0/1143907401 >> v2:172.21.15.181:3300/0 conn(0x7fed8406c3a0 0x7fed840c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed8359e640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed8415aa00 0x7fed840b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044987508 0 0) 0x7fed6c001110 con 0x7fed8406db60 2023-10-08T07:15:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.250+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed70005730 con 0x7fed8406db60 2023-10-08T07:15:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038500115 0 0) 0x7fed70005730 con 0x7fed8406db60 2023-10-08T07:15:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed8415aa00 msgr2=0x7fed840b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed8415aa00 0x7fed840b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 >> v2:172.21.15.181:3300/0 conn(0x7fed8406c3a0 msgr2=0x7fed840c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 --2- 172.21.15.70:0/1143907401 >> v2:172.21.15.181:3300/0 conn(0x7fed8406c3a0 0x7fed840c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed840c9860 con 0x7fed8406db60 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fed6c001800 con 0x7fed8406db60 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 >> v1:172.21.15.70:6789/0 conn(0x7fed8406db60 legacy=0x7fed8406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 0x7fed70005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed8415aa00 0x7fed700069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed70006f10 0x7fed70007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8406d290 con 0x7fed70005b80 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7000db90 con 0x7fed70006f10 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7000dcd0 con 0x7fed8415aa00 2023-10-08T07:15:14.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.251+0000 7fed82d9d640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 0x7fed70005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.252+0000 7fed8359e640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed70006f10 0x7fed70007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.252+0000 7fed7bfff640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed8415aa00 0x7fed700069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.252+0000 7fed82d9d640 1 -- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed8415aa00 msgr2=0x7fed700069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.252+0000 7fed82d9d640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed8415aa00 0x7fed700069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.252+0000 7fed82d9d640 1 -- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed70006f10 msgr2=0x7fed70007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.252+0000 7fed82d9d640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed70006f10 0x7fed70007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.252+0000 7fed82d9d640 1 -- 172.21.15.70:0/1143907401 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70005730 con 0x7fed70005b80 2023-10-08T07:15:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.253+0000 7fed82d9d640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 0x7fed70005fa0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7fed74009af0 tx=0x7fed70007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38f5336bf06fede server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.253+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7402c710 con 0x7fed70005b80 2023-10-08T07:15:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.253+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7402c870 con 0x7fed70005b80 2023-10-08T07:15:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.253+0000 7fed8259c640 1 -- 172.21.15.70:0/1143907401 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7402cc00 con 0x7fed70005b80 2023-10-08T07:15:14.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.254+0000 7fed8917c640 1 -- 172.21.15.70:0/1143907401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 msgr2=0x7fed70005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.254+0000 7fed8917c640 1 --2- 172.21.15.70:0/1143907401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 0x7fed70005fa0 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7fed74009af0 tx=0x7fed70007a00 comp rx=0 tx=0).stop 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.254+0000 7fed8359e640 1 -- 172.21.15.70:0/1143907401 reap_dead start 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.254+0000 7fed8917c640 1 -- 172.21.15.70:0/1143907401 shutdown_connections 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.254+0000 7fed8917c640 1 -- 172.21.15.70:0/1143907401 >> 172.21.15.70:0/1143907401 conn(0x7fed8406a5d0 msgr2=0x7fed840b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.254+0000 7fed8917c640 1 -- 172.21.15.70:0/1143907401 shutdown_connections 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.254+0000 7fed8917c640 1 -- 172.21.15.70:0/1143907401 wait complete. 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.256+0000 7fed8917c640 1 Processor -- start 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.256+0000 7fed8917c640 1 -- start start 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.257+0000 7fed8917c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 0x7fed84155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.257+0000 7fed8917c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed840c5810 0x7fed840c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.257+0000 7fed8917c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed840c7840 0x7fed840ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.257+0000 7fed8917c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed840c6e70 con 0x7fed70005b80 2023-10-08T07:15:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.257+0000 7fed8917c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed840c6260 con 0x7fed840c5810 2023-10-08T07:15:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.257+0000 7fed8917c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed840c6540 con 0x7fed840c7840 2023-10-08T07:15:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.257+0000 7fed82d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 0x7fed84155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.257+0000 7fed82d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 0x7fed84155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56464/0 (socket says 172.21.15.70:56464) 2023-10-08T07:15:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.257+0000 7fed82d9d640 1 -- 172.21.15.70:0/1584488280 learned_addr learned my addr 172.21.15.70:0/1584488280 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed8359e640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed840c7840 0x7fed840ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed7bfff640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed840c5810 0x7fed840c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed82d9d640 1 -- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed840c7840 msgr2=0x7fed840ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:14.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed82d9d640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed840c7840 0x7fed840ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed82d9d640 1 -- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed840c5810 msgr2=0x7fed840c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed82d9d640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed840c5810 0x7fed840c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed82d9d640 1 -- 172.21.15.70:0/1584488280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed840c6820 con 0x7fed70005b80 2023-10-08T07:15:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed8359e640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed840c7840 0x7fed840ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed82d9d640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 0x7fed84155320 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7fed740099e0 tx=0x7fed74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed7b7fe640 1 -- 172.21.15.70:0/1584488280 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74003810 con 0x7fed70005b80 2023-10-08T07:15:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed7b7fe640 1 -- 172.21.15.70:0/1584488280 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed74003970 con 0x7fed70005b80 2023-10-08T07:15:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed8917c640 1 -- 172.21.15.70:0/1584488280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed840c6b00 con 0x7fed70005b80 2023-10-08T07:15:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.258+0000 7fed7b7fe640 1 -- 172.21.15.70:0/1584488280 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed74003ce0 con 0x7fed70005b80 2023-10-08T07:15:14.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.259+0000 7fed8917c640 1 -- 172.21.15.70:0/1584488280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed840c5c90 con 0x7fed70005b80 2023-10-08T07:15:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.260+0000 7fed8917c640 1 -- 172.21.15.70:0/1584488280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed840c6680 con 0x7fed70005b80 2023-10-08T07:15:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.261+0000 7fed7b7fe640 1 -- 172.21.15.70:0/1584488280 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 142) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fed74040450 con 0x7fed70005b80 2023-10-08T07:15:14.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.262+0000 7fed7b7fe640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fed6406b0b0 0x7fed6406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.262+0000 7fed7b7fe640 1 -- 172.21.15.70:0/1584488280 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7fed740abff0 con 0x7fed70005b80 2023-10-08T07:15:14.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.262+0000 7fed7bfff640 1 -- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fed6406b0b0 msgr2=0x7fed6406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:14.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.262+0000 7fed7bfff640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fed6406b0b0 0x7fed6406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:14.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.270+0000 7fed7b7fe640 1 -- 172.21.15.70:0/1584488280 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed840c6680 con 0x7fed70005b80 2023-10-08T07:15:14.278 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:14.277+0000 7f8c4740e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:15:14.302 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:14.301+0000 7fb1b0a32040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:14.338 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:14.337+0000 7f8c4740e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:15:14.354 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:14.353+0000 7fb1b0a32040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:15:14.389 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:14.388+0000 7f8c4740e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:15:14.413 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:14.412+0000 7fb1b0a32040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:15:14.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.460+0000 7fed8917c640 1 -- 172.21.15.70:0/1584488280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed840c6680 con 0x7fed70005b80 2023-10-08T07:15:14.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.460+0000 7fed7b7fe640 1 -- 172.21.15.70:0/1584488280 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v259) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed840c6680 con 0x7fed70005b80 2023-10-08T07:15:14.461 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:15:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed7bfff640 1 -- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fed6406b0b0 msgr2=0x7fed6406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 -- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fed6406b0b0 msgr2=0x7fed6406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T07:15:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed7bfff640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fed6406b0b0 0x7fed6406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:15:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fed6406b0b0 0x7fed6406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 -- 172.21.15.70:0/1584488280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 msgr2=0x7fed84155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:14.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 0x7fed84155320 secure :-1 s=READY pgs=2741 cs=0 l=1 rev1=1 crypto rx=0x7fed740099e0 tx=0x7fed74004730 comp rx=0 tx=0).stop 2023-10-08T07:15:14.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 -- 172.21.15.70:0/1584488280 shutdown_connections 2023-10-08T07:15:14.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fed6406b0b0 0x7fed6406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed840c7840 0x7fed840ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed840c5810 0x7fed840c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 --2- 172.21.15.70:0/1584488280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed70005b80 0x7fed84155320 unknown :-1 s=CLOSED pgs=2741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 -- 172.21.15.70:0/1584488280 >> 172.21.15.70:0/1584488280 conn(0x7fed8406a5d0 msgr2=0x7fed840cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 -- 172.21.15.70:0/1584488280 shutdown_connections 2023-10-08T07:15:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.463+0000 7fed8917c640 1 -- 172.21.15.70:0/1584488280 wait complete. 2023-10-08T07:15:14.466 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:14.465+0000 7fb1b0a32040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:15:14.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:15:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.721+0000 7f39d9e52640 1 Processor -- start 2023-10-08T07:15:14.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.722+0000 7f39d9e52640 1 -- start start 2023-10-08T07:15:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.723+0000 7f39d9e52640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39d415cce0 0x7f39d40cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.723+0000 7f39d9e52640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f39d415d0c0 0x7f39d40b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.723+0000 7f39d9e52640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d406ce20 con 0x7f39d40b30d0 2023-10-08T07:15:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.723+0000 7f39d9e52640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d40cf850 con 0x7f39d415cce0 2023-10-08T07:15:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.723+0000 7f39d9e52640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d406c060 con 0x7f39d415d0c0 2023-10-08T07:15:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.723+0000 7f39d37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39d415cce0 0x7f39d40cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.723+0000 7f39d37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f39d415cce0 0x7f39d40cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56476/0 (socket says 172.21.15.70:56476) 2023-10-08T07:15:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.723+0000 7f39d37fe640 1 -- 172.21.15.70:0/2356805899 learned_addr learned my addr 172.21.15.70:0/2356805899 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.724+0000 7f39cbfff640 1 --2- 172.21.15.70:0/2356805899 >> v2:172.21.15.181:3300/0 conn(0x7f39d415d0c0 0x7f39d40b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.724+0000 7f39d37fe640 1 -- 172.21.15.70:0/2356805899 >> v2:172.21.15.181:3300/0 conn(0x7f39d415d0c0 msgr2=0x7f39d40b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.724+0000 7f39d37fe640 1 --2- 172.21.15.70:0/2356805899 >> v2:172.21.15.181:3300/0 conn(0x7f39d415d0c0 0x7f39d40b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.724+0000 7f39d37fe640 1 -- 172.21.15.70:0/2356805899 >> v1:172.21.15.181:6789/0 conn(0x7f39d40b30d0 legacy=0x7f39d41609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:14.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.724+0000 7f39d37fe640 1 -- 172.21.15.70:0/2356805899 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d406c340 con 0x7f39d415cce0 2023-10-08T07:15:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.725+0000 7f39d37fe640 1 --2- 172.21.15.70:0/2356805899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 0x7f39d40cf000 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f39d40c4540 tx=0x7f39c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7f1031d8069f16 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.725+0000 7f39d2ffd640 1 -- 172.21.15.70:0/2356805899 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4031870 con 0x7f39d415cce0 2023-10-08T07:15:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.725+0000 7f39d2ffd640 1 -- 172.21.15.70:0/2356805899 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c40319d0 con 0x7f39d415cce0 2023-10-08T07:15:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.725+0000 7f39d2ffd640 1 -- 172.21.15.70:0/2356805899 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4031d60 con 0x7f39d415cce0 2023-10-08T07:15:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.726+0000 7f39d9e52640 1 -- 172.21.15.70:0/2356805899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 msgr2=0x7f39d40cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.726+0000 7f39d9e52640 1 --2- 172.21.15.70:0/2356805899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 0x7f39d40cf000 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f39d40c4540 tx=0x7f39c402b910 comp rx=0 tx=0).stop 2023-10-08T07:15:14.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.726+0000 7f39d9e52640 1 -- 172.21.15.70:0/2356805899 shutdown_connections 2023-10-08T07:15:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.726+0000 7f39d9e52640 1 --2- 172.21.15.70:0/2356805899 >> v2:172.21.15.181:3300/0 conn(0x7f39d415d0c0 0x7f39d40b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.726+0000 7f39d9e52640 1 --2- 172.21.15.70:0/2356805899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 0x7f39d40cf000 unknown :-1 s=CLOSED pgs=2742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.726+0000 7f39d9e52640 1 -- 172.21.15.70:0/2356805899 >> 172.21.15.70:0/2356805899 conn(0x7f39d406e650 msgr2=0x7f39d406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.726+0000 7f39d9e52640 1 -- 172.21.15.70:0/2356805899 shutdown_connections 2023-10-08T07:15:14.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.726+0000 7f39d9e52640 1 -- 172.21.15.70:0/2356805899 wait complete. 2023-10-08T07:15:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.728+0000 7f39d9e52640 1 Processor -- start 2023-10-08T07:15:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.728+0000 7f39d9e52640 1 -- start start 2023-10-08T07:15:14.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39d9e52640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d40b30d0 0x7f39d4150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39d9e52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 0x7f39d40b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39d9e52640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39d406ea30 0x7f39d406eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39d9e52640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d406f420 con 0x7f39d415cce0 2023-10-08T07:15:14.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39d9e52640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40b3640 con 0x7f39d40b30d0 2023-10-08T07:15:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39d9e52640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40b48b0 con 0x7f39d406ea30 2023-10-08T07:15:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39cbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 0x7f39d40b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39cbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 0x7f39d40b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56482/0 (socket says 172.21.15.70:56482) 2023-10-08T07:15:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39cbfff640 1 -- 172.21.15.70:0/2477851018 learned_addr learned my addr 172.21.15.70:0/2477851018 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39d37fe640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d40b30d0 0x7f39d4150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.729+0000 7f39d3fff640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39d406ea30 0x7f39d406eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39cbfff640 1 -- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39d406ea30 msgr2=0x7f39d406eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39cbfff640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39d406ea30 0x7f39d406eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39cbfff640 1 -- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d40b30d0 msgr2=0x7f39d4150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39cbfff640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d40b30d0 0x7f39d4150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39cbfff640 1 -- 172.21.15.70:0/2477851018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c40094d0 con 0x7f39d415cce0 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39cbfff640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 0x7f39d40b4370 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00d5d0 tx=0x7f39bc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39d0ff9640 1 -- 172.21.15.70:0/2477851018 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0125f0 con 0x7f39d415cce0 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39d0ff9640 1 -- 172.21.15.70:0/2477851018 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc012750 con 0x7f39d415cce0 2023-10-08T07:15:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39d9e52640 1 -- 172.21.15.70:0/2477851018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d41f31e0 con 0x7f39d415cce0 2023-10-08T07:15:14.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39d0ff9640 1 -- 172.21.15.70:0/2477851018 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc012ae0 con 0x7f39d415cce0 2023-10-08T07:15:14.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.730+0000 7f39d9e52640 1 -- 172.21.15.70:0/2477851018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d40b3b70 con 0x7f39d415cce0 2023-10-08T07:15:14.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.731+0000 7f39d9e52640 1 -- 172.21.15.70:0/2477851018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d404e160 con 0x7f39d415cce0 2023-10-08T07:15:14.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.732+0000 7f39d0ff9640 1 -- 172.21.15.70:0/2477851018 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 142) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f39bc012c40 con 0x7f39d415cce0 2023-10-08T07:15:14.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.733+0000 7f39d0ff9640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f39b406b0b0 0x7f39b406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:14.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.733+0000 7f39d0ff9640 1 -- 172.21.15.70:0/2477851018 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7f39bc08a6a0 con 0x7f39d415cce0 2023-10-08T07:15:14.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.733+0000 7f39d37fe640 1 -- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f39b406b0b0 msgr2=0x7f39b406d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:14.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.733+0000 7f39d37fe640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f39b406b0b0 0x7f39b406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:14.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.738+0000 7f39d0ff9640 1 -- 172.21.15.70:0/2477851018 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39bc054ca0 con 0x7f39d415cce0 2023-10-08T07:15:14.777 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:14.775+0000 7f8c4740e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:15:14.867 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:14.866+0000 7fb1b0a32040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:15:14.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.928+0000 7f39d9e52640 1 -- 172.21.15.70:0/2477851018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39d40b6a30 con 0x7f39d415cce0 2023-10-08T07:15:14.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.929+0000 7f39d0ff9640 1 -- 172.21.15.70:0/2477851018 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v259) v1 ==== 100+0+11 (secure 0 0 0) 0x7f39bc0582c0 con 0x7f39d415cce0 2023-10-08T07:15:14.930 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 -- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f39b406b0b0 msgr2=0x7f39b406d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f39b406b0b0 0x7f39b406d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 -- 172.21.15.70:0/2477851018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 msgr2=0x7f39d40b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 0x7f39d40b4370 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00d5d0 tx=0x7f39bc00da90 comp rx=0 tx=0).stop 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 -- 172.21.15.70:0/2477851018 shutdown_connections 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f39b406b0b0 0x7f39b406d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f39d406ea30 0x7f39d406eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39d415cce0 0x7f39d40b4370 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 --2- 172.21.15.70:0/2477851018 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f39d40b30d0 0x7f39d4150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 -- 172.21.15.70:0/2477851018 >> 172.21.15.70:0/2477851018 conn(0x7f39d406e650 msgr2=0x7f39d40cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 -- 172.21.15.70:0/2477851018 shutdown_connections 2023-10-08T07:15:14.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:14.932+0000 7f39d9e52640 1 -- 172.21.15.70:0/2477851018 wait complete. 2023-10-08T07:15:14.935 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:14.934+0000 7f8c4740e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:15:14.949 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-08T07:15:15.032 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:15.032+0000 7fb1b0a32040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:15:15.042 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:15.041+0000 7f8c4740e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:15:15.092 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:15.091+0000 7f8c4740e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:15:15.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.143+0000 7f264f661640 1 Processor -- start 2023-10-08T07:15:15.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.143+0000 7f264f661640 1 -- start start 2023-10-08T07:15:15.145 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:15.144+0000 7fb1b0a32040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:15:15.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.145+0000 7f264f661640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f264815cc70 0x7f26480cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.145+0000 7f264f661640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26480d31a0 0x7f26480b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.145+0000 7f264f661640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26480b8b00 con 0x7f26480b61f0 2023-10-08T07:15:15.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.145+0000 7f264f661640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26480b8540 con 0x7f264815cc70 2023-10-08T07:15:15.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.145+0000 7f264f661640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26480b8820 con 0x7f26480d31a0 2023-10-08T07:15:15.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.145+0000 7f264d3fd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f264815cc70 0x7f26480cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:15.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.145+0000 7f264dbfe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f26480b61f0 0x7f26480b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49486/0 (socket says 172.21.15.70:49486) 2023-10-08T07:15:15.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.145+0000 7f264dbfe640 1 -- 172.21.15.70:0/265193834 learned_addr learned my addr 172.21.15.70:0/265193834 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:15.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.146+0000 7f264cbfc640 1 --2- 172.21.15.70:0/265193834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26480d31a0 0x7f26480b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:15.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.146+0000 7f264d3fd640 1 -- 172.21.15.70:0/265193834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26480d31a0 msgr2=0x7f26480b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.146+0000 7f264d3fd640 1 --2- 172.21.15.70:0/265193834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26480d31a0 0x7f26480b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.146+0000 7f264d3fd640 1 -- 172.21.15.70:0/265193834 >> v1:172.21.15.70:6789/0 conn(0x7f26480b61f0 legacy=0x7f26480b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.146+0000 7f264d3fd640 1 -- 172.21.15.70:0/265193834 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26480cd270 con 0x7f264815cc70 2023-10-08T07:15:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.146+0000 7f263ffff640 1 -- 172.21.15.70:0/265193834 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115263254 0 0) 0x7f26480b8b00 con 0x7f26480b61f0 2023-10-08T07:15:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.147+0000 7f264d3fd640 1 --2- 172.21.15.70:0/265193834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264815cc70 0x7f26480cdbe0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f26440097a0 tx=0x7f264402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f335754aaa06b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.147+0000 7f263ffff640 1 -- 172.21.15.70:0/265193834 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644031780 con 0x7f264815cc70 2023-10-08T07:15:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.147+0000 7f263ffff640 1 -- 172.21.15.70:0/265193834 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26440318e0 con 0x7f264815cc70 2023-10-08T07:15:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.147+0000 7f263ffff640 1 -- 172.21.15.70:0/265193834 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644031c70 con 0x7f264815cc70 2023-10-08T07:15:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.148+0000 7f264f661640 1 -- 172.21.15.70:0/265193834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264815cc70 msgr2=0x7f26480cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.148+0000 7f264f661640 1 --2- 172.21.15.70:0/265193834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264815cc70 0x7f26480cdbe0 secure :-1 s=READY pgs=2745 cs=0 l=1 rev1=1 crypto rx=0x7f26440097a0 tx=0x7f264402b820 comp rx=0 tx=0).stop 2023-10-08T07:15:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.148+0000 7f264f661640 1 -- 172.21.15.70:0/265193834 shutdown_connections 2023-10-08T07:15:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.148+0000 7f264f661640 1 --2- 172.21.15.70:0/265193834 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26480d31a0 0x7f26480b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.148+0000 7f264f661640 1 --2- 172.21.15.70:0/265193834 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f264815cc70 0x7f26480cdbe0 unknown :-1 s=CLOSED pgs=2745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.148+0000 7f264f661640 1 -- 172.21.15.70:0/265193834 >> 172.21.15.70:0/265193834 conn(0x7f26480718a0 msgr2=0x7f2648071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.148+0000 7f264f661640 1 -- 172.21.15.70:0/265193834 shutdown_connections 2023-10-08T07:15:15.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.148+0000 7f264f661640 1 -- 172.21.15.70:0/265193834 wait complete. 2023-10-08T07:15:15.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.149+0000 7f264f661640 1 Processor -- start 2023-10-08T07:15:15.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.150+0000 7f264f661640 1 -- start start 2023-10-08T07:15:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.150+0000 7f264f661640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26480b61f0 0x7f2648148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.150+0000 7f264f661640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26480d31a0 0x7f26480b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.150+0000 7f264f661640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f264815cc70 0x7f26480b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264f661640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264814c120 con 0x7f26480d31a0 2023-10-08T07:15:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264f661640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26481f1fe0 con 0x7f26480b61f0 2023-10-08T07:15:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264f661640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264814be40 con 0x7f264815cc70 2023-10-08T07:15:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264cbfc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26480d31a0 0x7f26480b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264cbfc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26480d31a0 0x7f26480b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56496/0 (socket says 172.21.15.70:56496) 2023-10-08T07:15:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264cbfc640 1 -- 172.21.15.70:0/3835736849 learned_addr learned my addr 172.21.15.70:0/3835736849 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264d3fd640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26480b61f0 0x7f2648148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264dbfe640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f264815cc70 0x7f26480b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264cbfc640 1 -- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f264815cc70 msgr2=0x7f26480b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264cbfc640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f264815cc70 0x7f26480b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264cbfc640 1 -- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26480b61f0 msgr2=0x7f2648148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264cbfc640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26480b61f0 0x7f2648148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.151+0000 7f264cbfc640 1 -- 172.21.15.70:0/3835736849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2644009400 con 0x7f26480d31a0 2023-10-08T07:15:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.152+0000 7f264cbfc640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26480d31a0 0x7f26480b4040 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f263800c7f0 tx=0x7f263800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.152+0000 7f263dffb640 1 -- 172.21.15.70:0/3835736849 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638004220 con 0x7f26480d31a0 2023-10-08T07:15:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.152+0000 7f263dffb640 1 -- 172.21.15.70:0/3835736849 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2638004380 con 0x7f26480d31a0 2023-10-08T07:15:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.152+0000 7f263dffb640 1 -- 172.21.15.70:0/3835736849 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638007840 con 0x7f26480d31a0 2023-10-08T07:15:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.152+0000 7f264f661640 1 -- 172.21.15.70:0/3835736849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26481f2220 con 0x7f26480d31a0 2023-10-08T07:15:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.152+0000 7f264f661640 1 -- 172.21.15.70:0/3835736849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26481f27e0 con 0x7f26480d31a0 2023-10-08T07:15:15.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.153+0000 7f264f661640 1 -- 172.21.15.70:0/3835736849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26481f2a00 con 0x7f26480d31a0 2023-10-08T07:15:15.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.155+0000 7f263dffb640 1 -- 172.21.15.70:0/3835736849 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 142) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f2638007bb0 con 0x7f26480d31a0 2023-10-08T07:15:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.156+0000 7f263dffb640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f262c06b0b0 0x7f262c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.156+0000 7f263dffb640 1 -- 172.21.15.70:0/3835736849 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7f263808a1a0 con 0x7f26480d31a0 2023-10-08T07:15:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.156+0000 7f264d3fd640 1 -- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f262c06b0b0 msgr2=0x7f262c06d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.156+0000 7f264d3fd640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f262c06b0b0 0x7f262c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:15.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.161+0000 7f263dffb640 1 -- 172.21.15.70:0/3835736849 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2638054680 con 0x7f26480d31a0 2023-10-08T07:15:15.200 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:15.199+0000 7fb1b0a32040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:15:15.200 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:15.199+0000 7f8c4740e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:15.307 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:15.305+0000 7f8c4740e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:15:15.313 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:15.312+0000 7fb1b0a32040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:15.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.351+0000 7f264f661640 1 -- 172.21.15.70:0/3835736849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f264804e160 con 0x7f26480d31a0 2023-10-08T07:15:15.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.352+0000 7f263dffb640 1 -- 172.21.15.70:0/3835736849 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v259) v1 ==== 107+0+11 (secure 0 0 0) 0x7f263801e0b0 con 0x7f26480d31a0 2023-10-08T07:15:15.353 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:15:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 -- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f262c06b0b0 msgr2=0x7f262c06d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f262c06b0b0 0x7f262c06d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 -- 172.21.15.70:0/3835736849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26480d31a0 msgr2=0x7f26480b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26480d31a0 0x7f26480b4040 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f263800c7f0 tx=0x7f263800ccb0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 -- 172.21.15.70:0/3835736849 shutdown_connections 2023-10-08T07:15:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f262c06b0b0 0x7f262c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f264815cc70 0x7f26480b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26480d31a0 0x7f26480b4040 unknown :-1 s=CLOSED pgs=2746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 --2- 172.21.15.70:0/3835736849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26480b61f0 0x7f2648148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 -- 172.21.15.70:0/3835736849 >> 172.21.15.70:0/3835736849 conn(0x7f26480718a0 msgr2=0x7f264806b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:15.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 -- 172.21.15.70:0/3835736849 shutdown_connections 2023-10-08T07:15:15.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.355+0000 7f264f661640 1 -- 172.21.15.70:0/3835736849 wait complete. 2023-10-08T07:15:15.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-08T07:15:15.430 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:15.429+0000 7fb1b0a32040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:15:15.468 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:15.466+0000 7f8c4740e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:15.516 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:15.515+0000 7f8c4740e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:15.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.574+0000 7f8609c46640 1 Processor -- start 2023-10-08T07:15:15.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.575+0000 7f8609c46640 1 -- start start 2023-10-08T07:15:15.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.576+0000 7f8609c46640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86040afbe0 0x7f86040b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.576+0000 7f8609c46640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86040cb130 0x7f86040cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.576+0000 7f8609c46640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86040cbad0 con 0x7f860406dfc0 2023-10-08T07:15:15.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.576+0000 7f8609c46640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86040b2620 con 0x7f86040afbe0 2023-10-08T07:15:15.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.576+0000 7f8609c46640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86040b2060 con 0x7f86040cb130 2023-10-08T07:15:15.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.576+0000 7f8603fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86040cb130 0x7f86040cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:15.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.576+0000 7f86037fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f86040afbe0 0x7f86040b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:15.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.577+0000 7f8602ffd640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f860406dfc0 0x7f8604070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:53946/0 (socket says 172.21.15.70:53946) 2023-10-08T07:15:15.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.577+0000 7f8602ffd640 1 -- 172.21.15.70:0/2410270479 learned_addr learned my addr 172.21.15.70:0/2410270479 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.577+0000 7f86037fe640 1 -- 172.21.15.70:0/2410270479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86040cb130 msgr2=0x7f86040cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.577+0000 7f86037fe640 1 --2- 172.21.15.70:0/2410270479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86040cb130 0x7f86040cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.577+0000 7f86037fe640 1 -- 172.21.15.70:0/2410270479 >> v1:172.21.15.181:6789/0 conn(0x7f860406dfc0 legacy=0x7f8604070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:15.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.577+0000 7f86037fe640 1 -- 172.21.15.70:0/2410270479 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86040b2340 con 0x7f86040afbe0 2023-10-08T07:15:15.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.579+0000 7f86037fe640 1 --2- 172.21.15.70:0/2410270479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86040afbe0 0x7f86040b15b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009d90 tx=0x7f85f402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b8186237b16b7b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:15.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.579+0000 7f86027fc640 1 -- 172.21.15.70:0/2410270479 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f402cc50 con 0x7f86040afbe0 2023-10-08T07:15:15.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.579+0000 7f86027fc640 1 -- 172.21.15.70:0/2410270479 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f402cdb0 con 0x7f86040afbe0 2023-10-08T07:15:15.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.579+0000 7f86027fc640 1 -- 172.21.15.70:0/2410270479 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4034660 con 0x7f86040afbe0 2023-10-08T07:15:15.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.580+0000 7f8609c46640 1 -- 172.21.15.70:0/2410270479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86040afbe0 msgr2=0x7f86040b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:15.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.580+0000 7f8609c46640 1 --2- 172.21.15.70:0/2410270479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86040afbe0 0x7f86040b15b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009d90 tx=0x7f85f402bea0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.580+0000 7f8609c46640 1 -- 172.21.15.70:0/2410270479 shutdown_connections 2023-10-08T07:15:15.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.580+0000 7f8609c46640 1 --2- 172.21.15.70:0/2410270479 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86040cb130 0x7f86040cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.580+0000 7f8609c46640 1 --2- 172.21.15.70:0/2410270479 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86040afbe0 0x7f86040b15b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.580+0000 7f8609c46640 1 -- 172.21.15.70:0/2410270479 >> 172.21.15.70:0/2410270479 conn(0x7f860406a5d0 msgr2=0x7f86040b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:15.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.580+0000 7f8609c46640 1 -- 172.21.15.70:0/2410270479 shutdown_connections 2023-10-08T07:15:15.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.580+0000 7f8609c46640 1 -- 172.21.15.70:0/2410270479 wait complete. 2023-10-08T07:15:15.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.583+0000 7f8609c46640 1 Processor -- start 2023-10-08T07:15:15.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.583+0000 7f8609c46640 1 -- start start 2023-10-08T07:15:15.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f8609c46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860406dfc0 0x7f86040ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f8609c46640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86040afbe0 0x7f86040cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f8609c46640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86040cb130 0x7f86041f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f8609c46640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040c9f00 con 0x7f860406dfc0 2023-10-08T07:15:15.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f8609c46640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040ccb80 con 0x7f86040cb130 2023-10-08T07:15:15.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f8609c46640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86041f26e0 con 0x7f86040afbe0 2023-10-08T07:15:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f86037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860406dfc0 0x7f86040ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f86037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860406dfc0 0x7f86040ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56508/0 (socket says 172.21.15.70:56508) 2023-10-08T07:15:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f86037fe640 1 -- 172.21.15.70:0/2263865472 learned_addr learned my addr 172.21.15.70:0/2263865472 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f8603fff640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86040cb130 0x7f86041f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f8602ffd640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86040afbe0 0x7f86040cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f86037fe640 1 -- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86040afbe0 msgr2=0x7f86040cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f86037fe640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86040afbe0 0x7f86040cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f86037fe640 1 -- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86040cb130 msgr2=0x7f86041f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f86037fe640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86040cb130 0x7f86041f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.585+0000 7f86037fe640 1 -- 172.21.15.70:0/2263865472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4009a70 con 0x7f860406dfc0 2023-10-08T07:15:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.586+0000 7f86037fe640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860406dfc0 0x7f86040ca320 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009eb0 tx=0x7f85f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.586+0000 7f85fbfff640 1 -- 172.21.15.70:0/2263865472 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f403e610 con 0x7f860406dfc0 2023-10-08T07:15:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.586+0000 7f85fbfff640 1 -- 172.21.15.70:0/2263865472 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f403e770 con 0x7f860406dfc0 2023-10-08T07:15:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.586+0000 7f85fbfff640 1 -- 172.21.15.70:0/2263865472 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f403eae0 con 0x7f860406dfc0 2023-10-08T07:15:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.586+0000 7f8609c46640 1 -- 172.21.15.70:0/2263865472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86040ccf50 con 0x7f860406dfc0 2023-10-08T07:15:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.586+0000 7f8609c46640 1 -- 172.21.15.70:0/2263865472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86040c9840 con 0x7f860406dfc0 2023-10-08T07:15:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.587+0000 7f8609c46640 1 -- 172.21.15.70:0/2263865472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860404e160 con 0x7f860406dfc0 2023-10-08T07:15:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.588+0000 7f85fbfff640 1 -- 172.21.15.70:0/2263865472 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 142) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f85f403b050 con 0x7f860406dfc0 2023-10-08T07:15:15.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.590+0000 7f85fbfff640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f85e406b0e0 0x7f85e406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:15.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.590+0000 7f85fbfff640 1 -- 172.21.15.70:0/2263865472 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7f85f40aca40 con 0x7f860406dfc0 2023-10-08T07:15:15.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.590+0000 7f8602ffd640 1 -- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f85e406b0e0 msgr2=0x7f85e406d5a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:15.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.590+0000 7f8602ffd640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f85e406b0e0 0x7f85e406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:15.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.594+0000 7f85fbfff640 1 -- 172.21.15.70:0/2263865472 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f85f40770c0 con 0x7f860406dfc0 2023-10-08T07:15:15.598 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:15.597+0000 7fb1b0a32040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:15.637 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:15.636+0000 7f8c4740e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:15:15.649 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:15.649+0000 7fb1b0a32040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:15.773 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:15.772+0000 7fb1b0a32040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:15:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.784+0000 7f8609c46640 1 -- 172.21.15.70:0/2263865472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f86040c1d70 con 0x7f860406dfc0 2023-10-08T07:15:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.785+0000 7f85fbfff640 1 -- 172.21.15.70:0/2263865472 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v259) v1 ==== 103+0+11 (secure 0 0 0) 0x7f85f4009450 con 0x7f860406dfc0 2023-10-08T07:15:15.786 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:15:15.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.787+0000 7f8609c46640 1 -- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f85e406b0e0 msgr2=0x7f85e406d5a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:15.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.787+0000 7f8609c46640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f85e406b0e0 0x7f85e406d5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.787+0000 7f8609c46640 1 -- 172.21.15.70:0/2263865472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860406dfc0 msgr2=0x7f86040ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:15.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.787+0000 7f8609c46640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860406dfc0 0x7f86040ca320 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009eb0 tx=0x7f85f4004a40 comp rx=0 tx=0).stop 2023-10-08T07:15:15.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.787+0000 7f8609c46640 1 -- 172.21.15.70:0/2263865472 shutdown_connections 2023-10-08T07:15:15.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.788+0000 7f8609c46640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f85e406b0e0 0x7f85e406d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.788+0000 7f8609c46640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f86040cb130 0x7f86041f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.788+0000 7f8609c46640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f86040afbe0 0x7f86040cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.788+0000 7f8609c46640 1 --2- 172.21.15.70:0/2263865472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860406dfc0 0x7f86040ca320 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:15.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.788+0000 7f8609c46640 1 -- 172.21.15.70:0/2263865472 >> 172.21.15.70:0/2263865472 conn(0x7f860406a5d0 msgr2=0x7f860406c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:15.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.788+0000 7f8609c46640 1 -- 172.21.15.70:0/2263865472 shutdown_connections 2023-10-08T07:15:15.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:15.788+0000 7f8609c46640 1 -- 172.21.15.70:0/2263865472 wait complete. 2023-10-08T07:15:15.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T07:15:16.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.011+0000 7fbb1deec640 1 Processor -- start 2023-10-08T07:15:16.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.011+0000 7fbb1deec640 1 -- start start 2023-10-08T07:15:16.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.013+0000 7fbb1deec640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbb18070cc0 0x7fbb180caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.013+0000 7fbb1deec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbb180cb450 0x7fbb180b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.013+0000 7fbb1deec640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb18071170 con 0x7fbb18160660 2023-10-08T07:15:16.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.013+0000 7fbb1deec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1806e150 con 0x7fbb180cb450 2023-10-08T07:15:16.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.013+0000 7fbb1deec640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1806de70 con 0x7fbb18070cc0 2023-10-08T07:15:16.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.013+0000 7fbb17fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbb180cb450 0x7fbb180b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:16.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.013+0000 7fbb17fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbb180cb450 0x7fbb180b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56522/0 (socket says 172.21.15.70:56522) 2023-10-08T07:15:16.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.013+0000 7fbb17fff640 1 -- 172.21.15.70:0/3966890742 learned_addr learned my addr 172.21.15.70:0/3966890742 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:16.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.013+0000 7fbb0ffff640 1 --2- 172.21.15.70:0/3966890742 >> v2:172.21.15.181:3300/0 conn(0x7fbb18070cc0 0x7fbb180caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:16.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.013+0000 7fbb17fff640 1 -- 172.21.15.70:0/3966890742 >> v2:172.21.15.181:3300/0 conn(0x7fbb18070cc0 msgr2=0x7fbb180caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:16.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.014+0000 7fbb17fff640 1 --2- 172.21.15.70:0/3966890742 >> v2:172.21.15.181:3300/0 conn(0x7fbb18070cc0 0x7fbb180caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.014+0000 7fbb17fff640 1 -- 172.21.15.70:0/3966890742 >> v1:172.21.15.181:6789/0 conn(0x7fbb18160660 legacy=0x7fbb180c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:16.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.014+0000 7fbb17fff640 1 -- 172.21.15.70:0/3966890742 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180c8e30 con 0x7fbb180cb450 2023-10-08T07:15:16.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.014+0000 7fbb0ffff640 1 --2- 172.21.15.70:0/3966890742 >> v2:172.21.15.181:3300/0 conn(0x7fbb18070cc0 0x7fbb180caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:16.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.015+0000 7fbb17fff640 1 --2- 172.21.15.70:0/3966890742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 0x7fbb180b1990 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fbb180b0d50 tx=0x7fbb1002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294f7b5c9d1aabe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:16.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.015+0000 7fbb16ffd640 1 -- 172.21.15.70:0/3966890742 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1002ca80 con 0x7fbb180cb450 2023-10-08T07:15:16.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.015+0000 7fbb16ffd640 1 -- 172.21.15.70:0/3966890742 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1002cbe0 con 0x7fbb180cb450 2023-10-08T07:15:16.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.015+0000 7fbb16ffd640 1 -- 172.21.15.70:0/3966890742 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb100343f0 con 0x7fbb180cb450 2023-10-08T07:15:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.015+0000 7fbb1deec640 1 -- 172.21.15.70:0/3966890742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 msgr2=0x7fbb180b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.016+0000 7fbb1deec640 1 --2- 172.21.15.70:0/3966890742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 0x7fbb180b1990 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7fbb180b0d50 tx=0x7fbb1002bee0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.016+0000 7fbb1deec640 1 -- 172.21.15.70:0/3966890742 shutdown_connections 2023-10-08T07:15:16.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.016+0000 7fbb1deec640 1 --2- 172.21.15.70:0/3966890742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 0x7fbb180b1990 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.016+0000 7fbb1deec640 1 --2- 172.21.15.70:0/3966890742 >> v2:172.21.15.181:3300/0 conn(0x7fbb18070cc0 0x7fbb180caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.016+0000 7fbb1deec640 1 -- 172.21.15.70:0/3966890742 >> 172.21.15.70:0/3966890742 conn(0x7fbb1806a5d0 msgr2=0x7fbb180b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.016+0000 7fbb1deec640 1 -- 172.21.15.70:0/3966890742 shutdown_connections 2023-10-08T07:15:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.016+0000 7fbb1deec640 1 -- 172.21.15.70:0/3966890742 wait complete. 2023-10-08T07:15:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.017+0000 7fbb1deec640 1 Processor -- start 2023-10-08T07:15:16.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.018+0000 7fbb1deec640 1 -- start start 2023-10-08T07:15:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.018+0000 7fbb1deec640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb18070cc0 0x7fbb1814d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.018+0000 7fbb1deec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 0x7fbb180c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.018+0000 7fbb1deec640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb18160660 0x7fbb180c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.018+0000 7fbb1deec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180c2250 con 0x7fbb180cb450 2023-10-08T07:15:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.018+0000 7fbb1deec640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180c24f0 con 0x7fbb18070cc0 2023-10-08T07:15:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.018+0000 7fbb1deec640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180c27d0 con 0x7fbb18160660 2023-10-08T07:15:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.019+0000 7fbb0ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 0x7fbb180c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.019+0000 7fbb0ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 0x7fbb180c79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56534/0 (socket says 172.21.15.70:56534) 2023-10-08T07:15:16.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.019+0000 7fbb0ffff640 1 -- 172.21.15.70:0/1953664413 learned_addr learned my addr 172.21.15.70:0/1953664413 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.019+0000 7fbb17fff640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb18160660 0x7fbb180c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.019+0000 7fbb177fe640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb18070cc0 0x7fbb1814d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.019+0000 7fbb0ffff640 1 -- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb18160660 msgr2=0x7fbb180c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.019+0000 7fbb0ffff640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb18160660 0x7fbb180c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.019+0000 7fbb0ffff640 1 -- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb18070cc0 msgr2=0x7fbb1814d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.019+0000 7fbb0ffff640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb18070cc0 0x7fbb1814d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.019+0000 7fbb0ffff640 1 -- 172.21.15.70:0/1953664413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb10009b30 con 0x7fbb180cb450 2023-10-08T07:15:16.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.020+0000 7fbb0ffff640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 0x7fbb180c79a0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000c7f0 tx=0x7fbb0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:16.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.020+0000 7fbb14ff9640 1 -- 172.21.15.70:0/1953664413 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb00007820 con 0x7fbb180cb450 2023-10-08T07:15:16.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.020+0000 7fbb14ff9640 1 -- 172.21.15.70:0/1953664413 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb00007980 con 0x7fbb180cb450 2023-10-08T07:15:16.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.020+0000 7fbb14ff9640 1 -- 172.21.15.70:0/1953664413 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb00007cf0 con 0x7fbb180cb450 2023-10-08T07:15:16.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.020+0000 7fbb1deec640 1 -- 172.21.15.70:0/1953664413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb180c2ab0 con 0x7fbb180cb450 2023-10-08T07:15:16.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.020+0000 7fbb1deec640 1 -- 172.21.15.70:0/1953664413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1814dec0 con 0x7fbb180cb450 2023-10-08T07:15:16.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.021+0000 7fbb1deec640 1 -- 172.21.15.70:0/1953664413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1804e160 con 0x7fbb180cb450 2023-10-08T07:15:16.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.022+0000 7fbb14ff9640 1 -- 172.21.15.70:0/1953664413 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 142) v1 ==== 89434+0+0 (secure 0 0 0) 0x7fbb000113f0 con 0x7fbb180cb450 2023-10-08T07:15:16.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.024+0000 7fbb14ff9640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fbaf806b0b0 0x7fbaf806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.024+0000 7fbb14ff9640 1 -- 172.21.15.70:0/1953664413 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7fbb0008a5b0 con 0x7fbb180cb450 2023-10-08T07:15:16.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.024+0000 7fbb177fe640 1 -- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fbaf806b0b0 msgr2=0x7fbaf806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:16.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.024+0000 7fbb177fe640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fbaf806b0b0 0x7fbaf806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:16.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.028+0000 7fbb14ff9640 1 -- 172.21.15.70:0/1953664413 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb00054aa0 con 0x7fbb180cb450 2023-10-08T07:15:16.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.225+0000 7fbb177fe640 1 -- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fbaf806b0b0 msgr2=0x7fbaf806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:16.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.225+0000 7fbb177fe640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fbaf806b0b0 0x7fbaf806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:15:16.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.229+0000 7fbb1deec640 1 -- 172.21.15.70:0/1953664413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbb1815af80 con 0x7fbb180cb450 2023-10-08T07:15:16.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.230+0000 7fbb14ff9640 1 -- 172.21.15.70:0/1953664413 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v259) v1 ==== 81+0+591 (secure 0 0 0) 0x7fbb0001f260 con 0x7fbb180cb450 2023-10-08T07:15:16.231 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2690144496 2023-10-09T07:15:10.723067+0000 2023-10-08T07:15:16.232 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3464560328 2023-10-09T07:15:10.723067+0000 2023-10-08T07:15:16.232 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/4245028292 2023-10-09T07:15:10.723067+0000 2023-10-08T07:15:16.232 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/807583929 2023-10-09T07:15:10.723067+0000 2023-10-08T07:15:16.232 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/859617464 2023-10-09T07:15:10.723067+0000 2023-10-08T07:15:16.232 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/4245028292 2023-10-09T07:15:10.723067+0000 2023-10-08T07:15:16.233 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/735805106 2023-10-09T07:15:10.723067+0000 2023-10-08T07:15:16.233 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/957356199 2023-10-09T07:15:10.723067+0000 2023-10-08T07:15:16.233 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/333977458 2023-10-09T07:15:10.723067+0000 2023-10-08T07:15:16.233 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2403193161 2023-10-09T07:15:10.723067+0000 2023-10-08T07:15:16.235 INFO:teuthology.orchestra.run.smithi070.stderr:listed 10 entries 2023-10-08T07:15:16.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 -- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fbaf806b0b0 msgr2=0x7fbaf806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:16.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fbaf806b0b0 0x7fbaf806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 -- 172.21.15.70:0/1953664413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 msgr2=0x7fbb180c79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:16.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 0x7fbb180c79a0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7fbb0000c7f0 tx=0x7fbb0000ccb0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 -- 172.21.15.70:0/1953664413 shutdown_connections 2023-10-08T07:15:16.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7fbaf806b0b0 0x7fbaf806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbb18160660 0x7fbb180c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb180cb450 0x7fbb180c79a0 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 --2- 172.21.15.70:0/1953664413 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbb18070cc0 0x7fbb1814d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 -- 172.21.15.70:0/1953664413 >> 172.21.15.70:0/1953664413 conn(0x7fbb1806a5d0 msgr2=0x7fbb1815ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:16.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 -- 172.21.15.70:0/1953664413 shutdown_connections 2023-10-08T07:15:16.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.232+0000 7fbb1deec640 1 -- 172.21.15.70:0/1953664413 wait complete. 2023-10-08T07:15:16.251 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T07:15:16.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:16.251 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T07:15:16.346 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T07:15:16.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:16.346 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T07:15:16.387 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:15:16.387 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:15:16.440 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:15:16.440 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:15:16.440 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:15:16.467 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:16.466+0000 7f8c4740e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:15:16.491 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:15:16.492 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:15:16.492 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:15:16.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:15:16.543 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:16.608 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:16.607+0000 7fb1b0a32040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:15:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.750+0000 7f3d11810640 1 Processor -- start 2023-10-08T07:15:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.750+0000 7f3d11810640 1 -- start start 2023-10-08T07:15:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.752+0000 7f3d11810640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d0c160940 0x7f3d0c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.752+0000 7f3d11810640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3d0c06c890 0x7f3d0c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.752+0000 7f3d11810640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0c0c48f0 con 0x7f3d0c15cd90 2023-10-08T07:15:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.752+0000 7f3d11810640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0c0c4d80 con 0x7f3d0c160940 2023-10-08T07:15:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.752+0000 7f3d11810640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0c0c3ce0 con 0x7f3d0c06c890 2023-10-08T07:15:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.752+0000 7f3d0a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d0c160940 0x7f3d0c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.752+0000 7f3d0a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3d0c160940 0x7f3d0c0709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56540/0 (socket says 172.21.15.70:56540) 2023-10-08T07:15:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.752+0000 7f3d0a7fc640 1 -- 172.21.15.70:0/3997043458 learned_addr learned my addr 172.21.15.70:0/3997043458 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.752+0000 7f3d0b7fe640 1 --2- 172.21.15.70:0/3997043458 >> v2:172.21.15.181:3300/0 conn(0x7f3d0c06c890 0x7f3d0c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.752+0000 7f3d0a7fc640 1 -- 172.21.15.70:0/3997043458 >> v2:172.21.15.181:3300/0 conn(0x7f3d0c06c890 msgr2=0x7f3d0c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.753+0000 7f3d0a7fc640 1 --2- 172.21.15.70:0/3997043458 >> v2:172.21.15.181:3300/0 conn(0x7f3d0c06c890 0x7f3d0c06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.753+0000 7f3d0a7fc640 1 -- 172.21.15.70:0/3997043458 >> v1:172.21.15.181:6789/0 conn(0x7f3d0c15cd90 legacy=0x7f3d0c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.753+0000 7f3d0a7fc640 1 -- 172.21.15.70:0/3997043458 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c0c3fc0 con 0x7f3d0c160940 2023-10-08T07:15:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.753+0000 7f3d0b7fe640 1 --2- 172.21.15.70:0/3997043458 >> v2:172.21.15.181:3300/0 conn(0x7f3d0c06c890 0x7f3d0c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:16.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.754+0000 7f3d0a7fc640 1 --2- 172.21.15.70:0/3997043458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c160940 0x7f3d0c0709e0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0c8d60 tx=0x7f3d0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0ed971b3943a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.754+0000 7f3d09ffb640 1 -- 172.21.15.70:0/3997043458 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00031870 con 0x7f3d0c160940 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.754+0000 7f3d09ffb640 1 -- 172.21.15.70:0/3997043458 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d000319d0 con 0x7f3d0c160940 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.754+0000 7f3d09ffb640 1 -- 172.21.15.70:0/3997043458 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00031d60 con 0x7f3d0c160940 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.754+0000 7f3d11810640 1 -- 172.21.15.70:0/3997043458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c160940 msgr2=0x7f3d0c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.754+0000 7f3d11810640 1 --2- 172.21.15.70:0/3997043458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c160940 0x7f3d0c0709e0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0c8d60 tx=0x7f3d0002b910 comp rx=0 tx=0).stop 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.754+0000 7f3d11810640 1 -- 172.21.15.70:0/3997043458 shutdown_connections 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.754+0000 7f3d11810640 1 --2- 172.21.15.70:0/3997043458 >> v2:172.21.15.181:3300/0 conn(0x7f3d0c06c890 0x7f3d0c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.754+0000 7f3d11810640 1 --2- 172.21.15.70:0/3997043458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c160940 0x7f3d0c0709e0 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.754+0000 7f3d11810640 1 -- 172.21.15.70:0/3997043458 >> 172.21.15.70:0/3997043458 conn(0x7f3d0c06a380 msgr2=0x7f3d0c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.755+0000 7f3d11810640 1 -- 172.21.15.70:0/3997043458 shutdown_connections 2023-10-08T07:15:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.755+0000 7f3d11810640 1 -- 172.21.15.70:0/3997043458 wait complete. 2023-10-08T07:15:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.756+0000 7f3d11810640 1 Processor -- start 2023-10-08T07:15:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.756+0000 7f3d11810640 1 -- start start 2023-10-08T07:15:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.757+0000 7f3d11810640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c06c890 0x7f3d0c15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.757+0000 7f3d11810640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d0c15cd90 0x7f3d0c0ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.757+0000 7f3d11810640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c160940 0x7f3d0c0ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.757+0000 7f3d11810640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c15a140 con 0x7f3d0c06c890 2023-10-08T07:15:16.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.757+0000 7f3d11810640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c15a420 con 0x7f3d0c160940 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.757+0000 7f3d11810640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c15a700 con 0x7f3d0c15cd90 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.757+0000 7f3d0affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c06c890 0x7f3d0c15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c06c890 0x7f3d0c15f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56544/0 (socket says 172.21.15.70:56544) 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0affd640 1 -- 172.21.15.70:0/3956472120 learned_addr learned my addr 172.21.15.70:0/3956472120 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0a7fc640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d0c15cd90 0x7f3d0c0ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0b7fe640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c160940 0x7f3d0c0ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0affd640 1 -- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d0c15cd90 msgr2=0x7f3d0c0ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0affd640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d0c15cd90 0x7f3d0c0ad290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0affd640 1 -- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c160940 msgr2=0x7f3d0c0ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0affd640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c160940 0x7f3d0c0ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0affd640 1 -- 172.21.15.70:0/3956472120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d000094d0 con 0x7f3d0c06c890 2023-10-08T07:15:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0b7fe640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c160940 0x7f3d0c0ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d0affd640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c06c890 0x7f3d0c15f650 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00c410 tx=0x7f3cfc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3cf3fff640 1 -- 172.21.15.70:0/3956472120 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0035f0 con 0x7f3d0c06c890 2023-10-08T07:15:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3cf3fff640 1 -- 172.21.15.70:0/3956472120 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc003750 con 0x7f3d0c06c890 2023-10-08T07:15:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3cf3fff640 1 -- 172.21.15.70:0/3956472120 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc003b40 con 0x7f3d0c06c890 2023-10-08T07:15:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.758+0000 7f3d11810640 1 -- 172.21.15.70:0/3956472120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0c15a9e0 con 0x7f3d0c06c890 2023-10-08T07:15:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.759+0000 7f3d11810640 1 -- 172.21.15.70:0/3956472120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0c15fb90 con 0x7f3d0c06c890 2023-10-08T07:15:16.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.760+0000 7f3d11810640 1 -- 172.21.15.70:0/3956472120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0c067490 con 0x7f3d0c06c890 2023-10-08T07:15:16.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.761+0000 7f3cf3fff640 1 -- 172.21.15.70:0/3956472120 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 142) v1 ==== 89434+0+0 (secure 0 0 0) 0x7f3cfc003ca0 con 0x7f3d0c06c890 2023-10-08T07:15:16.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.762+0000 7f3cf3fff640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f3cf806b0b0 0x7f3cf806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:16.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.762+0000 7f3cf3fff640 1 -- 172.21.15.70:0/3956472120 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 7439+0+0 (secure 0 0 0) 0x7f3cfc089310 con 0x7f3d0c06c890 2023-10-08T07:15:16.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.763+0000 7f3d0a7fc640 1 -- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f3cf806b0b0 msgr2=0x7f3cf806d570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/2621244427 2023-10-08T07:15:16.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.763+0000 7f3d0a7fc640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f3cf806b0b0 0x7f3cf806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:16.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.767+0000 7f3cf3fff640 1 -- 172.21.15.70:0/3956472120 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3cfc0539a0 con 0x7f3d0c06c890 2023-10-08T07:15:16.770 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:16.769+0000 7f8c4740e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:15:16.828 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:16.826+0000 7f8c4740e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:15:16.920 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:16.920+0000 7fb1b0a32040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:15:16.942 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:16.941+0000 7f8c4740e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:15:16.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.959+0000 7f3cf3fff640 1 -- 172.21.15.70:0/3956472120 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 143) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f3cfc053d70 con 0x7f3d0c06c890 2023-10-08T07:15:16.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.959+0000 7f3cf3fff640 1 -- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f3cf806b0b0 msgr2=0x7f3cf806d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:16.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:16.959+0000 7f3cf3fff640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f3cf806b0b0 0x7f3cf806d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:16.990 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:16.989+0000 7fb1b0a32040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:15:17.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.008+0000 7f3d11810640 1 -- 172.21.15.70:0/3956472120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d0c0b0ef0 con 0x7f3d0c06c890 2023-10-08T07:15:17.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.010+0000 7f3cf3fff640 1 -- 172.21.15.70:0/3956472120 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+6149 (secure 0 0 0) 0x7f3cfc074410 con 0x7f3d0c06c890 2023-10-08T07:15:17.010 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:17.011 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":130,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:43.833416+0000","modified":"2023-10-08T07:14:46.899347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11904},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11904":{"gid":11904,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/3856326680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3856326680},{"type":"v1","addr":"172.21.15.70:6834","nonce":3856326680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:57.955428+0000","modified":"2023-10-08T07:15:01.029708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8875":{"gid":8875,"name":"b","rank":0,"incarnation":127,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-08T07:15:17.011 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 130 2023-10-08T07:15:17.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.012+0000 7f3d11810640 1 -- 172.21.15.70:0/3956472120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c06c890 msgr2=0x7f3d0c15f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.012+0000 7f3d11810640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c06c890 0x7f3d0c15f650 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00c410 tx=0x7f3cfc00c8d0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.012+0000 7f3d11810640 1 -- 172.21.15.70:0/3956472120 shutdown_connections 2023-10-08T07:15:17.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.012+0000 7f3d11810640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:6832/2621244427,v1:172.21.15.181:6833/2621244427] conn(0x7f3cf806b0b0 0x7f3cf806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.012+0000 7f3d11810640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3d0c160940 0x7f3d0c0ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.012+0000 7f3d11810640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3d0c15cd90 0x7f3d0c0ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.012+0000 7f3d11810640 1 --2- 172.21.15.70:0/3956472120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d0c06c890 0x7f3d0c15f650 unknown :-1 s=CLOSED pgs=2751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.012+0000 7f3d11810640 1 -- 172.21.15.70:0/3956472120 >> 172.21.15.70:0/3956472120 conn(0x7f3d0c06a380 msgr2=0x7f3d0c15bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:17.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.012+0000 7f3d11810640 1 -- 172.21.15.70:0/3956472120 shutdown_connections 2023-10-08T07:15:17.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.012+0000 7f3d11810640 1 -- 172.21.15.70:0/3956472120 wait complete. 2023-10-08T07:15:17.031 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:17.037 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.034+0000 7f8bcc291640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.037 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.034+0000 7f8bcc291640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.038 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.034+0000 7f8bcc291640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.038 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.034+0000 7f8bcc291640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.038 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.034+0000 7f8bcc291640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.038 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.035+0000 7f8bc928b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.038 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.035+0000 7f8bc928b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.038 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.035+0000 7f8bc928b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.038 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.035+0000 7f8bc928b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.038 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.035+0000 7f8bc928b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.048 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.047+0000 7f8bcc291640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.048 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.047+0000 7f8bcc291640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.049 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.047+0000 7f8bcc291640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.049 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.047+0000 7f8bcc291640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.049 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.047+0000 7f8bcc291640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.049 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.048+0000 7f8bc928b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.049 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.048+0000 7f8bc928b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.049 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.048+0000 7f8bc928b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.050 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.048+0000 7f8bc928b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.050 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:17.048+0000 7f8bc928b640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:15:17.114 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:17.113+0000 7fb1b0a32040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:15:17.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.212+0000 7f266c539640 1 Processor -- start 2023-10-08T07:15:17.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.212+0000 7f266c539640 1 -- start start 2023-10-08T07:15:17.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.214+0000 7f266c539640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f266415ab40 0x7f26640ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.214+0000 7f266c539640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266415b2b0 con 0x7f26640cb580 2023-10-08T07:15:17.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.214+0000 7f266c539640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266415b590 con 0x7f26640c90b0 2023-10-08T07:15:17.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.214+0000 7f266c539640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266415bf30 con 0x7f266415ab40 2023-10-08T07:15:17.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.214+0000 7f266aad6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f26640cb580 0x7f266415e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49494/0 (socket says 172.21.15.70:49494) 2023-10-08T07:15:17.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.214+0000 7f266aad6640 1 -- 172.21.15.70:0/2408562123 learned_addr learned my addr 172.21.15.70:0/2408562123 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.214+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822180618 0 0) 0x7f266415b2b0 con 0x7f26640cb580 2023-10-08T07:15:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.214+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2650003520 con 0x7f26640cb580 2023-10-08T07:15:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.214+0000 7f266a2d5640 1 --2- 172.21.15.70:0/2408562123 >> v2:172.21.15.181:3300/0 conn(0x7f266415ab40 0x7f26640ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1490127889 0 0) 0x7f2650003520 con 0x7f26640cb580 2023-10-08T07:15:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 >> v2:172.21.15.181:3300/0 conn(0x7f266415ab40 msgr2=0x7f26640ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 --2- 172.21.15.70:0/2408562123 >> v2:172.21.15.181:3300/0 conn(0x7f266415ab40 0x7f26640ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 >> v1:172.21.15.181:6789/0 conn(0x7f26640c90b0 legacy=0x7f26640cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266415b870 con 0x7f26640cb580 2023-10-08T07:15:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f265c0019f0 con 0x7f26640cb580 2023-10-08T07:15:17.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 >> v1:172.21.15.70:6789/0 conn(0x7f26640cb580 legacy=0x7f266415e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 --2- 172.21.15.70:0/2408562123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2650003970 0x7f2650003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 --2- 172.21.15.70:0/2408562123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26500047c0 0x7f2650004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 --2- 172.21.15.70:0/2408562123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2650005180 0x7f265000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266415bf30 con 0x7f26500047c0 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265000c020 con 0x7f2650005180 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265000c1e0 con 0x7f2650003970 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.215+0000 7f2669ad4640 1 --2- 172.21.15.70:0/2408562123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26500047c0 0x7f2650004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.216+0000 7f266a2d5640 1 --2- 172.21.15.70:0/2408562123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2650003970 0x7f2650003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.216+0000 7f2669ad4640 1 -- 172.21.15.70:0/2408562123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2650003970 msgr2=0x7f2650003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.216+0000 7f2669ad4640 1 --2- 172.21.15.70:0/2408562123 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2650003970 0x7f2650003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.216+0000 7f2669ad4640 1 -- 172.21.15.70:0/2408562123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2650005180 msgr2=0x7f265000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:17.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.216+0000 7f2669ad4640 1 --2- 172.21.15.70:0/2408562123 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2650005180 0x7f265000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.216+0000 7f2669ad4640 1 -- 172.21.15.70:0/2408562123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2650003520 con 0x7f26500047c0 2023-10-08T07:15:17.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.216+0000 7f266aad6640 1 -- 172.21.15.70:0/2408562123 reap_dead start 2023-10-08T07:15:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.217+0000 7f2669ad4640 1 --2- 172.21.15.70:0/2408562123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26500047c0 0x7f2650004c40 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f2654001720 tx=0x7f26500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d419c6da803f1df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.217+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540052d0 con 0x7f26500047c0 2023-10-08T07:15:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.217+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265402e050 con 0x7f26500047c0 2023-10-08T07:15:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.217+0000 7f26692d3640 1 -- 172.21.15.70:0/2408562123 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654003830 con 0x7f26500047c0 2023-10-08T07:15:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.217+0000 7f266c539640 1 -- 172.21.15.70:0/2408562123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26500047c0 msgr2=0x7f2650004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.217+0000 7f266c539640 1 --2- 172.21.15.70:0/2408562123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26500047c0 0x7f2650004c40 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f2654001720 tx=0x7f26500057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.217+0000 7f266c539640 1 -- 172.21.15.70:0/2408562123 shutdown_connections 2023-10-08T07:15:17.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.217+0000 7f266c539640 1 --2- 172.21.15.70:0/2408562123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26500047c0 0x7f2650004c40 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.217+0000 7f266c539640 1 -- 172.21.15.70:0/2408562123 >> 172.21.15.70:0/2408562123 conn(0x7f266406a380 msgr2=0x7f26640b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.218+0000 7f266c539640 1 -- 172.21.15.70:0/2408562123 shutdown_connections 2023-10-08T07:15:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.218+0000 7f266c539640 1 -- 172.21.15.70:0/2408562123 wait complete. 2023-10-08T07:15:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.219+0000 7f266c539640 1 Processor -- start 2023-10-08T07:15:17.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.219+0000 7f266c539640 1 -- start start 2023-10-08T07:15:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.220+0000 7f266c539640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500047c0 0x7f26641445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.220+0000 7f266c539640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26640b1640 0x7f26641f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.220+0000 7f266c539640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640b1aa0 0x7f26641f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.220+0000 7f266c539640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b31a0 con 0x7f26640b1aa0 2023-10-08T07:15:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.220+0000 7f266c539640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b3440 con 0x7f26500047c0 2023-10-08T07:15:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.220+0000 7f266c539640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26640b3720 con 0x7f26640b1640 2023-10-08T07:15:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.220+0000 7f266aad6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640b1aa0 0x7f26641f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:17.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f266aad6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640b1aa0 0x7f26641f2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56552/0 (socket says 172.21.15.70:56552) 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f266aad6640 1 -- 172.21.15.70:0/3001524464 learned_addr learned my addr 172.21.15.70:0/3001524464 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f266a2d5640 1 --2- 172.21.15.70:0/3001524464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500047c0 0x7f26641445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f2669ad4640 1 --2- 172.21.15.70:0/3001524464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26640b1640 0x7f26641f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f266aad6640 1 -- 172.21.15.70:0/3001524464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26640b1640 msgr2=0x7f26641f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f266aad6640 1 --2- 172.21.15.70:0/3001524464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26640b1640 0x7f26641f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f266aad6640 1 -- 172.21.15.70:0/3001524464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500047c0 msgr2=0x7f26641445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f266aad6640 1 --2- 172.21.15.70:0/3001524464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500047c0 0x7f26641445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f266aad6640 1 -- 172.21.15.70:0/3001524464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640b3a00 con 0x7f26640b1aa0 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f266aad6640 1 --2- 172.21.15.70:0/3001524464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640b1aa0 0x7f26641f2930 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f26640b0c20 tx=0x7f265c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.221+0000 7f265affd640 1 -- 172.21.15.70:0/3001524464 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c004d60 con 0x7f26640b1aa0 2023-10-08T07:15:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.222+0000 7f265affd640 1 -- 172.21.15.70:0/3001524464 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c00a5b0 con 0x7f26640b1aa0 2023-10-08T07:15:17.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.222+0000 7f265affd640 1 -- 172.21.15.70:0/3001524464 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c00a960 con 0x7f26640b1aa0 2023-10-08T07:15:17.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.222+0000 7f266c539640 1 -- 172.21.15.70:0/3001524464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26640b3ce0 con 0x7f26640b1aa0 2023-10-08T07:15:17.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.222+0000 7f266c539640 1 -- 172.21.15.70:0/3001524464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26640b44e0 con 0x7f26640b1aa0 2023-10-08T07:15:17.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.223+0000 7f266c539640 1 -- 172.21.15.70:0/3001524464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2664067490 con 0x7f26640b1aa0 2023-10-08T07:15:17.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.224+0000 7f265affd640 1 -- 172.21.15.70:0/3001524464 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 143) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f265c00aac0 con 0x7f26640b1aa0 2023-10-08T07:15:17.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.225+0000 7f265affd640 1 -- 172.21.15.70:0/3001524464 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f265c089110 con 0x7f26640b1aa0 2023-10-08T07:15:17.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.229+0000 7f265affd640 1 -- 172.21.15.70:0/3001524464 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f265c053cb0 con 0x7f26640b1aa0 2023-10-08T07:15:17.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.419+0000 7f266c539640 1 -- 172.21.15.70:0/3001524464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26640b0ef0 con 0x7f26640b1aa0 2023-10-08T07:15:17.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.423+0000 7f265affd640 1 -- 172.21.15.70:0/3001524464 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v260) v1 ==== 74+0+21882 (secure 0 0 0) 0x7f265c01a030 con 0x7f26640b1aa0 2023-10-08T07:15:17.426 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:17.426 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":260,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:16.946287+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:14:56.178783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:14:56.977158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/888851152":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3676453840":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3956317048":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1513601554":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6832/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/348399880":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1607590530":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/2690144496":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/3464560328":"2023-10-09T07:15:10.723067+0000","172.21.15.181:6832/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/807583929":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/859617464":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/4271328810":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/735805106":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/957356199":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2539797528":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/333977458":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2403193161":"2023-10-09T07:15:10.723067+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:17.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.427+0000 7f266c539640 1 -- 172.21.15.70:0/3001524464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640b1aa0 msgr2=0x7f26641f2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.427+0000 7f266c539640 1 --2- 172.21.15.70:0/3001524464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640b1aa0 0x7f26641f2930 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f26640b0c20 tx=0x7f265c00cb80 comp rx=0 tx=0).stop 2023-10-08T07:15:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.427+0000 7f266c539640 1 -- 172.21.15.70:0/3001524464 shutdown_connections 2023-10-08T07:15:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.427+0000 7f266c539640 1 --2- 172.21.15.70:0/3001524464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26640b1aa0 0x7f26641f2930 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.427+0000 7f266c539640 1 --2- 172.21.15.70:0/3001524464 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f26640b1640 0x7f26641f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.427+0000 7f266c539640 1 --2- 172.21.15.70:0/3001524464 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f26500047c0 0x7f26641445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.427+0000 7f266c539640 1 -- 172.21.15.70:0/3001524464 >> 172.21.15.70:0/3001524464 conn(0x7f266406a380 msgr2=0x7f26640c46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.427+0000 7f266c539640 1 -- 172.21.15.70:0/3001524464 shutdown_connections 2023-10-08T07:15:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.427+0000 7f266c539640 1 -- 172.21.15.70:0/3001524464 wait complete. 2023-10-08T07:15:17.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T07:14:41.606127+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T07:14:42.867820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '254', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-08T07:14:56.178783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '257', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-08T07:14:56.977158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '258', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:15:17.448 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:15:17.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.646+0000 7f5a3e056640 1 Processor -- start 2023-10-08T07:15:17.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.646+0000 7f5a3e056640 1 -- start start 2023-10-08T07:15:17.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.648+0000 7f5a3e056640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f5a3815aad0 0x7f5a380b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.648+0000 7f5a3e056640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a380c82b0 0x7f5a380c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.648+0000 7f5a3e056640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5a3806cd00 0x7f5a3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.648+0000 7f5a3e056640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3806d6d0 con 0x7f5a3806cd00 2023-10-08T07:15:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.648+0000 7f5a3e056640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3815bf30 con 0x7f5a3815aad0 2023-10-08T07:15:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.648+0000 7f5a3e056640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3815b320 con 0x7f5a380c82b0 2023-10-08T07:15:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.648+0000 7f5a37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5a3806cd00 0x7f5a3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.648+0000 7f5a37fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5a3806cd00 0x7f5a3806d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56554/0 (socket says 172.21.15.70:56554) 2023-10-08T07:15:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.648+0000 7f5a37fff640 1 -- 172.21.15.70:0/3719596116 learned_addr learned my addr 172.21.15.70:0/3719596116 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:17.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.648+0000 7f5a2ffff640 1 --2- 172.21.15.70:0/3719596116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a380c82b0 0x7f5a380c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:17.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.649+0000 7f5a377fe640 1 --2- 172.21.15.70:0/3719596116 >> v2:172.21.15.181:3300/0 conn(0x7f5a3815aad0 0x7f5a380b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:17.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.649+0000 7f5a37fff640 1 -- 172.21.15.70:0/3719596116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a380c82b0 msgr2=0x7f5a380c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:17.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.649+0000 7f5a37fff640 1 --2- 172.21.15.70:0/3719596116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a380c82b0 0x7f5a380c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.649+0000 7f5a37fff640 1 -- 172.21.15.70:0/3719596116 >> v2:172.21.15.181:3300/0 conn(0x7f5a3815aad0 msgr2=0x7f5a380b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:17.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.649+0000 7f5a37fff640 1 --2- 172.21.15.70:0/3719596116 >> v2:172.21.15.181:3300/0 conn(0x7f5a3815aad0 0x7f5a380b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.649+0000 7f5a37fff640 1 -- 172.21.15.70:0/3719596116 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3815b600 con 0x7f5a3806cd00 2023-10-08T07:15:17.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.649+0000 7f5a377fe640 1 --2- 172.21.15.70:0/3719596116 >> v2:172.21.15.181:3300/0 conn(0x7f5a3815aad0 0x7f5a380b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:17.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.650+0000 7f5a37fff640 1 --2- 172.21.15.70:0/3719596116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a3806cd00 0x7f5a3806d190 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f5a3806e6a0 tx=0x7f5a3002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d70b2f92e3d45d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:17.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.650+0000 7f5a36ffd640 1 -- 172.21.15.70:0/3719596116 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3002ca80 con 0x7f5a3806cd00 2023-10-08T07:15:17.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.650+0000 7f5a36ffd640 1 -- 172.21.15.70:0/3719596116 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3002cbe0 con 0x7f5a3806cd00 2023-10-08T07:15:17.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.650+0000 7f5a36ffd640 1 -- 172.21.15.70:0/3719596116 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a300343f0 con 0x7f5a3806cd00 2023-10-08T07:15:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.651+0000 7f5a3e056640 1 -- 172.21.15.70:0/3719596116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a3806cd00 msgr2=0x7f5a3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.651+0000 7f5a3e056640 1 --2- 172.21.15.70:0/3719596116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a3806cd00 0x7f5a3806d190 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f5a3806e6a0 tx=0x7f5a3002bee0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.651+0000 7f5a3e056640 1 -- 172.21.15.70:0/3719596116 shutdown_connections 2023-10-08T07:15:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.651+0000 7f5a3e056640 1 --2- 172.21.15.70:0/3719596116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a3806cd00 0x7f5a3806d190 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.651+0000 7f5a3e056640 1 --2- 172.21.15.70:0/3719596116 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a380c82b0 0x7f5a380c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.651+0000 7f5a3e056640 1 --2- 172.21.15.70:0/3719596116 >> v2:172.21.15.181:3300/0 conn(0x7f5a3815aad0 0x7f5a380b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.651+0000 7f5a3e056640 1 -- 172.21.15.70:0/3719596116 >> 172.21.15.70:0/3719596116 conn(0x7f5a38069d50 msgr2=0x7f5a3806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.651+0000 7f5a3e056640 1 -- 172.21.15.70:0/3719596116 shutdown_connections 2023-10-08T07:15:17.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.651+0000 7f5a3e056640 1 -- 172.21.15.70:0/3719596116 wait complete. 2023-10-08T07:15:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.652+0000 7f5a3e056640 1 Processor -- start 2023-10-08T07:15:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.653+0000 7f5a3e056640 1 -- start start 2023-10-08T07:15:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a3e056640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3806cd00 0x7f5a3814a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a3e056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a380c82b0 0x7f5a3814be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a3e056640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a3815aad0 0x7f5a3814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:17.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a3e056640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3814d660 con 0x7f5a380c82b0 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a3e056640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3814a7a0 con 0x7f5a3806cd00 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a3e056640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3814da80 con 0x7f5a3815aad0 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a380c82b0 0x7f5a3814be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a380c82b0 0x7f5a3814be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56560/0 (socket says 172.21.15.70:56560) 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a2ffff640 1 -- 172.21.15.70:0/2955661064 learned_addr learned my addr 172.21.15.70:0/2955661064 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a2ffff640 1 -- 172.21.15.70:0/2955661064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a3815aad0 msgr2=0x7f5a3814c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a2ffff640 1 --2- 172.21.15.70:0/2955661064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a3815aad0 0x7f5a3814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a2ffff640 1 -- 172.21.15.70:0/2955661064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3806cd00 msgr2=0x7f5a3814a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a2ffff640 1 --2- 172.21.15.70:0/2955661064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3806cd00 0x7f5a3814a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.654+0000 7f5a2ffff640 1 -- 172.21.15.70:0/2955661064 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a30009b30 con 0x7f5a380c82b0 2023-10-08T07:15:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.655+0000 7f5a2ffff640 1 --2- 172.21.15.70:0/2955661064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a380c82b0 0x7f5a3814be90 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000d800 tx=0x7f5a2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.655+0000 7f5a34ff9640 1 -- 172.21.15.70:0/2955661064 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a200096a0 con 0x7f5a380c82b0 2023-10-08T07:15:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.655+0000 7f5a34ff9640 1 -- 172.21.15.70:0/2955661064 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a20009800 con 0x7f5a380c82b0 2023-10-08T07:15:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.655+0000 7f5a3e056640 1 -- 172.21.15.70:0/2955661064 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3814c9e0 con 0x7f5a380c82b0 2023-10-08T07:15:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.655+0000 7f5a34ff9640 1 -- 172.21.15.70:0/2955661064 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20009b90 con 0x7f5a380c82b0 2023-10-08T07:15:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.655+0000 7f5a3e056640 1 -- 172.21.15.70:0/2955661064 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3814cfa0 con 0x7f5a380c82b0 2023-10-08T07:15:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.656+0000 7f5a3e056640 1 -- 172.21.15.70:0/2955661064 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3804e160 con 0x7f5a380c82b0 2023-10-08T07:15:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.657+0000 7f5a34ff9640 1 -- 172.21.15.70:0/2955661064 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 143) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f5a20009cf0 con 0x7f5a380c82b0 2023-10-08T07:15:17.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.658+0000 7f5a34ff9640 1 -- 172.21.15.70:0/2955661064 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f5a20089eb0 con 0x7f5a380c82b0 2023-10-08T07:15:17.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.662+0000 7f5a34ff9640 1 -- 172.21.15.70:0/2955661064 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a20054a50 con 0x7f5a380c82b0 2023-10-08T07:15:17.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.852+0000 7f5a3e056640 1 -- 172.21.15.70:0/2955661064 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a380cbf30 con 0x7f5a380c82b0 2023-10-08T07:15:17.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.853+0000 7f5a34ff9640 1 -- 172.21.15.70:0/2955661064 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v260) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a2000b440 con 0x7f5a380c82b0 2023-10-08T07:15:17.855 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:15:17.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.855+0000 7f5a3e056640 1 -- 172.21.15.70:0/2955661064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a380c82b0 msgr2=0x7f5a3814be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:17.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.855+0000 7f5a3e056640 1 --2- 172.21.15.70:0/2955661064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a380c82b0 0x7f5a3814be90 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000d800 tx=0x7f5a2000dcc0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.856+0000 7f5a3e056640 1 -- 172.21.15.70:0/2955661064 shutdown_connections 2023-10-08T07:15:17.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.856+0000 7f5a3e056640 1 --2- 172.21.15.70:0/2955661064 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5a3815aad0 0x7f5a3814c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.856+0000 7f5a3e056640 1 --2- 172.21.15.70:0/2955661064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a380c82b0 0x7f5a3814be90 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.856+0000 7f5a3e056640 1 --2- 172.21.15.70:0/2955661064 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5a3806cd00 0x7f5a3814a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:17.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.856+0000 7f5a3e056640 1 -- 172.21.15.70:0/2955661064 >> 172.21.15.70:0/2955661064 conn(0x7f5a38069d50 msgr2=0x7f5a3806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:17.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.856+0000 7f5a3e056640 1 -- 172.21.15.70:0/2955661064 shutdown_connections 2023-10-08T07:15:17.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:17.856+0000 7f5a3e056640 1 -- 172.21.15.70:0/2955661064 wait complete. 2023-10-08T07:15:17.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:15:18.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.082+0000 7f795595e640 1 Processor -- start 2023-10-08T07:15:18.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.083+0000 7f795595e640 1 -- start start 2023-10-08T07:15:18.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.084+0000 7f795595e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f795006e5d0 0x7f79500ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.084+0000 7f795595e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7950066150 con 0x7f7950065d20 2023-10-08T07:15:18.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.084+0000 7f795595e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795015d2c0 con 0x7f795015cee0 2023-10-08T07:15:18.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.084+0000 7f795595e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79500cd670 con 0x7f795006e5d0 2023-10-08T07:15:18.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.084+0000 7f7947fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f795006e5d0 0x7f79500ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.084+0000 7f794f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7950065d20 0x7f795006a5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49510/0 (socket says 172.21.15.70:49510) 2023-10-08T07:15:18.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.085+0000 7f794f7fe640 1 -- 172.21.15.70:0/62499926 learned_addr learned my addr 172.21.15.70:0/62499926 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:18.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.085+0000 7f7947fff640 1 -- 172.21.15.70:0/62499926 >> v1:172.21.15.181:6789/0 conn(0x7f795015cee0 legacy=0x7f795006dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.085+0000 7f794e7fc640 1 -- 172.21.15.70:0/62499926 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390131419 0 0) 0x7f7950066150 con 0x7f7950065d20 2023-10-08T07:15:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.085+0000 7f7947fff640 1 -- 172.21.15.70:0/62499926 >> v1:172.21.15.70:6789/0 conn(0x7f7950065d20 legacy=0x7f795006a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.085+0000 7f7947fff640 1 -- 172.21.15.70:0/62499926 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500cd950 con 0x7f795006e5d0 2023-10-08T07:15:18.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.086+0000 7f7947fff640 1 --2- 172.21.15.70:0/62499926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795006e5d0 0x7f79500ccfe0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f79380097a0 tx=0x7f793802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ca975f18070f1e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.087+0000 7f794e7fc640 1 -- 172.21.15.70:0/62499926 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938030610 con 0x7f795006e5d0 2023-10-08T07:15:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.087+0000 7f794e7fc640 1 -- 172.21.15.70:0/62499926 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7938030770 con 0x7f795006e5d0 2023-10-08T07:15:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.087+0000 7f794e7fc640 1 -- 172.21.15.70:0/62499926 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7938030b00 con 0x7f795006e5d0 2023-10-08T07:15:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.087+0000 7f795595e640 1 -- 172.21.15.70:0/62499926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795006e5d0 msgr2=0x7f79500ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.087+0000 7f795595e640 1 --2- 172.21.15.70:0/62499926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795006e5d0 0x7f79500ccfe0 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f79380097a0 tx=0x7f793802b960 comp rx=0 tx=0).stop 2023-10-08T07:15:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.087+0000 7f795595e640 1 -- 172.21.15.70:0/62499926 shutdown_connections 2023-10-08T07:15:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.087+0000 7f795595e640 1 --2- 172.21.15.70:0/62499926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795006e5d0 0x7f79500ccfe0 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.087+0000 7f795595e640 1 -- 172.21.15.70:0/62499926 >> 172.21.15.70:0/62499926 conn(0x7f79500cba60 msgr2=0x7f79500cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.087+0000 7f795595e640 1 -- 172.21.15.70:0/62499926 shutdown_connections 2023-10-08T07:15:18.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.087+0000 7f795595e640 1 -- 172.21.15.70:0/62499926 wait complete. 2023-10-08T07:15:18.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.089+0000 7f795595e640 1 Processor -- start 2023-10-08T07:15:18.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.089+0000 7f795595e640 1 -- start start 2023-10-08T07:15:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.090+0000 7f795595e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7950065d20 0x7f79500c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.090+0000 7f795595e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f795006e5d0 0x7f79501515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.090+0000 7f795595e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795015cee0 0x7f7950151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.090+0000 7f795595e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795015d5f0 con 0x7f795015cee0 2023-10-08T07:15:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.090+0000 7f795595e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795015d8d0 con 0x7f7950065d20 2023-10-08T07:15:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.090+0000 7f795595e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795015dbb0 con 0x7f795006e5d0 2023-10-08T07:15:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.090+0000 7f794f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795015cee0 0x7f7950151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.090+0000 7f794f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795015cee0 0x7f7950151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56580/0 (socket says 172.21.15.70:56580) 2023-10-08T07:15:18.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.090+0000 7f794f7fe640 1 -- 172.21.15.70:0/2504701483 learned_addr learned my addr 172.21.15.70:0/2504701483 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794effd640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7950065d20 0x7f79500c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f7947fff640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f795006e5d0 0x7f79501515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794f7fe640 1 -- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f795006e5d0 msgr2=0x7f79501515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794f7fe640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f795006e5d0 0x7f79501515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794f7fe640 1 -- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7950065d20 msgr2=0x7f79500c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794f7fe640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7950065d20 0x7f79500c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794f7fe640 1 -- 172.21.15.70:0/2504701483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7938009400 con 0x7f795015cee0 2023-10-08T07:15:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794effd640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7950065d20 0x7f79500c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f7947fff640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f795006e5d0 0x7f79501515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:18.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794f7fe640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795015cee0 0x7f7950151b20 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f794800c7e0 tx=0x7f794800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794cff9640 1 -- 172.21.15.70:0/2504701483 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794800a640 con 0x7f795015cee0 2023-10-08T07:15:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794cff9640 1 -- 172.21.15.70:0/2504701483 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794800a7c0 con 0x7f795015cee0 2023-10-08T07:15:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f794cff9640 1 -- 172.21.15.70:0/2504701483 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794800ab70 con 0x7f795015cee0 2023-10-08T07:15:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.091+0000 7f795595e640 1 -- 172.21.15.70:0/2504701483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795015de90 con 0x7f795015cee0 2023-10-08T07:15:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.092+0000 7f795595e640 1 -- 172.21.15.70:0/2504701483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79500c85b0 con 0x7f795015cee0 2023-10-08T07:15:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.093+0000 7f795595e640 1 -- 172.21.15.70:0/2504701483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79500c87a0 con 0x7f795015cee0 2023-10-08T07:15:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.094+0000 7f794cff9640 1 -- 172.21.15.70:0/2504701483 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 144) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f794800acd0 con 0x7f795015cee0 2023-10-08T07:15:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.094+0000 7f794cff9640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f793006acc0 0x7f793006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.095+0000 7f794cff9640 1 -- 172.21.15.70:0/2504701483 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f79480890c0 con 0x7f795015cee0 2023-10-08T07:15:18.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.095+0000 7f794effd640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f793006acc0 0x7f793006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.096+0000 7f794effd640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f793006acc0 0x7f793006d180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7940000f60 tx=0x7f7940009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:18.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.099+0000 7f794cff9640 1 -- 172.21.15.70:0/2504701483 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7948053c40 con 0x7f795015cee0 2023-10-08T07:15:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.272+0000 7f795595e640 1 -- 172.21.15.70:0/2504701483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f795015e3b0 con 0x7f795015cee0 2023-10-08T07:15:18.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.273+0000 7f794cff9640 1 -- 172.21.15.70:0/2504701483 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v260) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7948018070 con 0x7f795015cee0 2023-10-08T07:15:18.275 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:15:18.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.275+0000 7f795595e640 1 -- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f793006acc0 msgr2=0x7f793006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.275+0000 7f795595e640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f793006acc0 0x7f793006d180 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7940000f60 tx=0x7f7940009040 comp rx=0 tx=0).stop 2023-10-08T07:15:18.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.275+0000 7f795595e640 1 -- 172.21.15.70:0/2504701483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795015cee0 msgr2=0x7f7950151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.275+0000 7f795595e640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795015cee0 0x7f7950151b20 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f794800c7e0 tx=0x7f794800cca0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.275+0000 7f795595e640 1 -- 172.21.15.70:0/2504701483 shutdown_connections 2023-10-08T07:15:18.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.275+0000 7f795595e640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f793006acc0 0x7f793006d180 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.275+0000 7f795595e640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f795015cee0 0x7f7950151b20 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.275+0000 7f795595e640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f795006e5d0 0x7f79501515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.275+0000 7f795595e640 1 --2- 172.21.15.70:0/2504701483 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7950065d20 0x7f79500c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.276+0000 7f795595e640 1 -- 172.21.15.70:0/2504701483 >> 172.21.15.70:0/2504701483 conn(0x7f79500cba60 msgr2=0x7f7950071710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:18.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.276+0000 7f795595e640 1 -- 172.21.15.70:0/2504701483 shutdown_connections 2023-10-08T07:15:18.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.276+0000 7f795595e640 1 -- 172.21.15.70:0/2504701483 wait complete. 2023-10-08T07:15:18.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:15:18.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.515+0000 7fb810373640 1 Processor -- start 2023-10-08T07:15:18.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.515+0000 7fb810373640 1 -- start start 2023-10-08T07:15:18.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.517+0000 7fb810373640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb808160660 0x7fb808070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.517+0000 7fb810373640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8080c9910 0x7fb8080caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.517+0000 7fb810373640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb8080cb450 0x7fb8080b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.517+0000 7fb810373640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80806e150 con 0x7fb808160660 2023-10-08T07:15:18.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.517+0000 7fb810373640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8080cb8c0 con 0x7fb8080cb450 2023-10-08T07:15:18.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.517+0000 7fb810373640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80806de70 con 0x7fb8080c9910 2023-10-08T07:15:18.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.517+0000 7fb80e10f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb808160660 0x7fb808070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.517+0000 7fb80e10f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb808160660 0x7fb808070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56594/0 (socket says 172.21.15.70:56594) 2023-10-08T07:15:18.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.517+0000 7fb80e10f640 1 -- 172.21.15.70:0/366250348 learned_addr learned my addr 172.21.15.70:0/366250348 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:18.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.518+0000 7fb80d90e640 1 --2- 172.21.15.70:0/366250348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8080c9910 0x7fb8080caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.518+0000 7fb80e910640 1 --2- 172.21.15.70:0/366250348 >> v2:172.21.15.181:3300/0 conn(0x7fb8080cb450 0x7fb8080b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.518+0000 7fb80e10f640 1 -- 172.21.15.70:0/366250348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8080c9910 msgr2=0x7fb8080caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:18.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.518+0000 7fb80e10f640 1 --2- 172.21.15.70:0/366250348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8080c9910 0x7fb8080caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.518+0000 7fb80e10f640 1 -- 172.21.15.70:0/366250348 >> v2:172.21.15.181:3300/0 conn(0x7fb8080cb450 msgr2=0x7fb8080b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:18.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.518+0000 7fb80e10f640 1 --2- 172.21.15.70:0/366250348 >> v2:172.21.15.181:3300/0 conn(0x7fb8080cb450 0x7fb8080b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.518+0000 7fb80e10f640 1 -- 172.21.15.70:0/366250348 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8080c8e30 con 0x7fb808160660 2023-10-08T07:15:18.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.519+0000 7fb80e10f640 1 --2- 172.21.15.70:0/366250348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb808160660 0x7fb808070cc0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fb808070100 tx=0x7fb80402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e94e8d0ca6ac30 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:18.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb80d10d640 1 -- 172.21.15.70:0/366250348 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804031860 con 0x7fb808160660 2023-10-08T07:15:18.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb80d10d640 1 -- 172.21.15.70:0/366250348 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8040319c0 con 0x7fb808160660 2023-10-08T07:15:18.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb80d10d640 1 -- 172.21.15.70:0/366250348 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804031d50 con 0x7fb808160660 2023-10-08T07:15:18.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb810373640 1 -- 172.21.15.70:0/366250348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb808160660 msgr2=0x7fb808070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb810373640 1 --2- 172.21.15.70:0/366250348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb808160660 0x7fb808070cc0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7fb808070100 tx=0x7fb80402b900 comp rx=0 tx=0).stop 2023-10-08T07:15:18.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb810373640 1 -- 172.21.15.70:0/366250348 shutdown_connections 2023-10-08T07:15:18.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb810373640 1 --2- 172.21.15.70:0/366250348 >> v2:172.21.15.181:3300/0 conn(0x7fb8080cb450 0x7fb8080b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb810373640 1 --2- 172.21.15.70:0/366250348 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb8080c9910 0x7fb8080caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb810373640 1 --2- 172.21.15.70:0/366250348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb808160660 0x7fb808070cc0 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb810373640 1 -- 172.21.15.70:0/366250348 >> 172.21.15.70:0/366250348 conn(0x7fb80806a5d0 msgr2=0x7fb8080b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:18.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.520+0000 7fb810373640 1 -- 172.21.15.70:0/366250348 shutdown_connections 2023-10-08T07:15:18.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.521+0000 7fb810373640 1 -- 172.21.15.70:0/366250348 wait complete. 2023-10-08T07:15:18.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.522+0000 7fb810373640 1 Processor -- start 2023-10-08T07:15:18.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.522+0000 7fb810373640 1 -- start start 2023-10-08T07:15:18.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.523+0000 7fb810373640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8080c9910 0x7fb8080c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.523+0000 7fb810373640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8080cb450 0x7fb80815e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.523+0000 7fb810373640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb808160660 0x7fb80815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.523+0000 7fb810373640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8080aec40 con 0x7fb8080cb450 2023-10-08T07:15:18.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.523+0000 7fb810373640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8080c4f20 con 0x7fb8080c9910 2023-10-08T07:15:18.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.523+0000 7fb810373640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8080ab8d0 con 0x7fb808160660 2023-10-08T07:15:18.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.524+0000 7fb80d90e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8080cb450 0x7fb80815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.524+0000 7fb80d90e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8080cb450 0x7fb80815e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56608/0 (socket says 172.21.15.70:56608) 2023-10-08T07:15:18.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.524+0000 7fb80d90e640 1 -- 172.21.15.70:0/3636489542 learned_addr learned my addr 172.21.15.70:0/3636489542 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:18.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.524+0000 7fb80d90e640 1 -- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb808160660 msgr2=0x7fb80815eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.524+0000 7fb80e910640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb808160660 0x7fb80815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.524+0000 7fb80e10f640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8080c9910 0x7fb8080c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.524+0000 7fb80d90e640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb808160660 0x7fb80815eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.524+0000 7fb80d90e640 1 -- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8080c9910 msgr2=0x7fb8080c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.524+0000 7fb80d90e640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8080c9910 0x7fb8080c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.524+0000 7fb80d90e640 1 -- 172.21.15.70:0/3636489542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040094d0 con 0x7fb8080cb450 2023-10-08T07:15:18.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.525+0000 7fb80d90e640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8080cb450 0x7fb80815e760 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800d800 tx=0x7fb7f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:18.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.525+0000 7fb7feffd640 1 -- 172.21.15.70:0/3636489542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80096a0 con 0x7fb8080cb450 2023-10-08T07:15:18.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.525+0000 7fb7feffd640 1 -- 172.21.15.70:0/3636489542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f8009800 con 0x7fb8080cb450 2023-10-08T07:15:18.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.525+0000 7fb810373640 1 -- 172.21.15.70:0/3636489542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb808152430 con 0x7fb8080cb450 2023-10-08T07:15:18.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.525+0000 7fb7feffd640 1 -- 172.21.15.70:0/3636489542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8009990 con 0x7fb8080cb450 2023-10-08T07:15:18.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.525+0000 7fb810373640 1 -- 172.21.15.70:0/3636489542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8080abe00 con 0x7fb8080cb450 2023-10-08T07:15:18.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.526+0000 7fb810373640 1 -- 172.21.15.70:0/3636489542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80804e160 con 0x7fb8080cb450 2023-10-08T07:15:18.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.527+0000 7fb7feffd640 1 -- 172.21.15.70:0/3636489542 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 144) v1 ==== 89067+0+0 (secure 0 0 0) 0x7fb7f8009af0 con 0x7fb8080cb450 2023-10-08T07:15:18.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.528+0000 7fb7feffd640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb7ec06acc0 0x7fb7ec06d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:18.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.528+0000 7fb7feffd640 1 -- 172.21.15.70:0/3636489542 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7fb7f8089f00 con 0x7fb8080cb450 2023-10-08T07:15:18.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.529+0000 7fb80e10f640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb7ec06acc0 0x7fb7ec06d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:18.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.530+0000 7fb80e10f640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb7ec06acc0 0x7fb7ec06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb804036000 tx=0x7fb80403c040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:18.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.532+0000 7fb7feffd640 1 -- 172.21.15.70:0/3636489542 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7f8054a80 con 0x7fb8080cb450 2023-10-08T07:15:18.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.721+0000 7fb810373640 1 -- 172.21.15.70:0/3636489542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb80815fc00 con 0x7fb8080cb450 2023-10-08T07:15:18.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.722+0000 7fb7feffd640 1 -- 172.21.15.70:0/3636489542 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v260) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7f80180a0 con 0x7fb8080cb450 2023-10-08T07:15:18.779 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:15:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.724+0000 7fb810373640 1 -- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb7ec06acc0 msgr2=0x7fb7ec06d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.724+0000 7fb810373640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb7ec06acc0 0x7fb7ec06d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb804036000 tx=0x7fb80403c040 comp rx=0 tx=0).stop 2023-10-08T07:15:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.724+0000 7fb810373640 1 -- 172.21.15.70:0/3636489542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8080cb450 msgr2=0x7fb80815e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.724+0000 7fb810373640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8080cb450 0x7fb80815e760 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800d800 tx=0x7fb7f800dcc0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.724+0000 7fb810373640 1 -- 172.21.15.70:0/3636489542 shutdown_connections 2023-10-08T07:15:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.724+0000 7fb810373640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb7ec06acc0 0x7fb7ec06d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.724+0000 7fb810373640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb808160660 0x7fb80815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.724+0000 7fb810373640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8080cb450 0x7fb80815e760 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.724+0000 7fb810373640 1 --2- 172.21.15.70:0/3636489542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb8080c9910 0x7fb8080c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:18.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.725+0000 7fb810373640 1 -- 172.21.15.70:0/3636489542 >> 172.21.15.70:0/3636489542 conn(0x7fb80806a5d0 msgr2=0x7fb8080ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:18.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.725+0000 7fb810373640 1 -- 172.21.15.70:0/3636489542 shutdown_connections 2023-10-08T07:15:18.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:18.725+0000 7fb810373640 1 -- 172.21.15.70:0/3636489542 wait complete. 2023-10-08T07:15:18.787 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-08T07:15:19.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.010+0000 7f9a8e11d640 1 Processor -- start 2023-10-08T07:15:19.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.010+0000 7f9a8e11d640 1 -- start start 2023-10-08T07:15:19.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.011+0000 7f9a8e11d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9a8806dfc0 0x7f9a88070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.011+0000 7f9a8e11d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a88071070 0x7f9a880cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a8e11d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a880b1cc0 con 0x7f9a880afbe0 2023-10-08T07:15:19.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a8e11d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a880b2620 con 0x7f9a88071070 2023-10-08T07:15:19.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a8e11d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a880b2060 con 0x7f9a8806dfc0 2023-10-08T07:15:19.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a87fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a88071070 0x7f9a880cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a87fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9a88071070 0x7f9a880cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56614/0 (socket says 172.21.15.70:56614) 2023-10-08T07:15:19.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a87fff640 1 -- 172.21.15.70:0/442570773 learned_addr learned my addr 172.21.15.70:0/442570773 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:19.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a877fe640 1 --1- 172.21.15.70:0/442570773 >> v1:172.21.15.70:6789/0 conn(0x7f9a880afbe0 0x7f9a880b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49526/0 (socket says 172.21.15.70:49526) 2023-10-08T07:15:19.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a87fff640 1 -- 172.21.15.70:0/442570773 >> v2:172.21.15.181:3300/0 conn(0x7f9a8806dfc0 msgr2=0x7f9a88070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:19.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a87fff640 1 --2- 172.21.15.70:0/442570773 >> v2:172.21.15.181:3300/0 conn(0x7f9a8806dfc0 0x7f9a88070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a87fff640 1 -- 172.21.15.70:0/442570773 >> v1:172.21.15.70:6789/0 conn(0x7f9a880afbe0 legacy=0x7f9a880b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.012+0000 7f9a87fff640 1 -- 172.21.15.70:0/442570773 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a880b2340 con 0x7f9a88071070 2023-10-08T07:15:19.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.013+0000 7f9a87fff640 1 --2- 172.21.15.70:0/442570773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a88071070 0x7f9a880cb130 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f9a880b3980 tx=0x7f9a8002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8941042171eaae84 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:19.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a867fc640 1 -- 172.21.15.70:0/442570773 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8002cb10 con 0x7f9a88071070 2023-10-08T07:15:19.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a867fc640 1 -- 172.21.15.70:0/442570773 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8002cc70 con 0x7f9a88071070 2023-10-08T07:15:19.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a867fc640 1 -- 172.21.15.70:0/442570773 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a800343f0 con 0x7f9a88071070 2023-10-08T07:15:19.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a8e11d640 1 -- 172.21.15.70:0/442570773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a88071070 msgr2=0x7f9a880cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a8e11d640 1 --2- 172.21.15.70:0/442570773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a88071070 0x7f9a880cb130 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f9a880b3980 tx=0x7f9a8002bf70 comp rx=0 tx=0).stop 2023-10-08T07:15:19.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a8e11d640 1 -- 172.21.15.70:0/442570773 shutdown_connections 2023-10-08T07:15:19.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a8e11d640 1 --2- 172.21.15.70:0/442570773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a88071070 0x7f9a880cb130 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a8e11d640 1 --2- 172.21.15.70:0/442570773 >> v2:172.21.15.181:3300/0 conn(0x7f9a8806dfc0 0x7f9a88070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a8e11d640 1 -- 172.21.15.70:0/442570773 >> 172.21.15.70:0/442570773 conn(0x7f9a8806a5d0 msgr2=0x7f9a880b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:19.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a8e11d640 1 -- 172.21.15.70:0/442570773 shutdown_connections 2023-10-08T07:15:19.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.014+0000 7f9a8e11d640 1 -- 172.21.15.70:0/442570773 wait complete. 2023-10-08T07:15:19.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.016+0000 7f9a8e11d640 1 Processor -- start 2023-10-08T07:15:19.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.017+0000 7f9a8e11d640 1 -- start start 2023-10-08T07:15:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.017+0000 7f9a8e11d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a8806dfc0 0x7f9a8814ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.017+0000 7f9a8e11d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a88071070 0x7f9a880c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.017+0000 7f9a8e11d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a880afbe0 0x7f9a8815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a8e11d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880c3e00 con 0x7f9a8806dfc0 2023-10-08T07:15:19.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a8e11d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880c40e0 con 0x7f9a88071070 2023-10-08T07:15:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a8e11d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880c43c0 con 0x7f9a880afbe0 2023-10-08T07:15:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a8806dfc0 0x7f9a8814ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a8806dfc0 0x7f9a8814ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56630/0 (socket says 172.21.15.70:56630) 2023-10-08T07:15:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a877fe640 1 -- 172.21.15.70:0/3659098947 learned_addr learned my addr 172.21.15.70:0/3659098947 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a877fe640 1 -- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a880afbe0 msgr2=0x7f9a8815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a877fe640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a880afbe0 0x7f9a8815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a877fe640 1 -- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a88071070 msgr2=0x7f9a880c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a877fe640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a88071070 0x7f9a880c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a877fe640 1 -- 172.21.15.70:0/3659098947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a80009b30 con 0x7f9a8806dfc0 2023-10-08T07:15:19.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.018+0000 7f9a877fe640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a8806dfc0 0x7f9a8814ca90 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800c3e0 tx=0x7f9a7800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:19.068 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:19.017+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.019+0000 7f9a6ffff640 1 -- 172.21.15.70:0/3659098947 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780074e0 con 0x7f9a8806dfc0 2023-10-08T07:15:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.019+0000 7f9a8e11d640 1 -- 172.21.15.70:0/3659098947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a880c46a0 con 0x7f9a8806dfc0 2023-10-08T07:15:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.019+0000 7f9a6ffff640 1 -- 172.21.15.70:0/3659098947 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a78007640 con 0x7f9a8806dfc0 2023-10-08T07:15:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.019+0000 7f9a6ffff640 1 -- 172.21.15.70:0/3659098947 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780049e0 con 0x7f9a8806dfc0 2023-10-08T07:15:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.019+0000 7f9a8e11d640 1 -- 172.21.15.70:0/3659098947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8814cfd0 con 0x7f9a8806dfc0 2023-10-08T07:15:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.020+0000 7f9a8e11d640 1 -- 172.21.15.70:0/3659098947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8804e160 con 0x7f9a8806dfc0 2023-10-08T07:15:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.021+0000 7f9a6ffff640 1 -- 172.21.15.70:0/3659098947 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 144) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f9a78004b40 con 0x7f9a8806dfc0 2023-10-08T07:15:19.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.022+0000 7f9a6ffff640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9a7406acf0 0x7f9a7406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.022+0000 7f9a6ffff640 1 -- 172.21.15.70:0/3659098947 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f9a780894e0 con 0x7f9a8806dfc0 2023-10-08T07:15:19.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.022+0000 7f9a86ffd640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9a7406acf0 0x7f9a7406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.023+0000 7f9a86ffd640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9a7406acf0 0x7f9a7406d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9a881507c0 tx=0x7f9a7c006c90 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:19.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.026+0000 7f9a6ffff640 1 -- 172.21.15.70:0/3659098947 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a78053fe0 con 0x7f9a8806dfc0 2023-10-08T07:15:19.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.211+0000 7f9a8e11d640 1 -- 172.21.15.70:0/3659098947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a880b0ef0 con 0x7f9a8806dfc0 2023-10-08T07:15:19.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.212+0000 7f9a6ffff640 1 -- 172.21.15.70:0/3659098947 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v260) v1 ==== 107+0+11 (secure 0 0 0) 0x7f9a78057600 con 0x7f9a8806dfc0 2023-10-08T07:15:19.214 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:15:19.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 -- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9a7406acf0 msgr2=0x7f9a7406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9a7406acf0 0x7f9a7406d1b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9a881507c0 tx=0x7f9a7c006c90 comp rx=0 tx=0).stop 2023-10-08T07:15:19.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 -- 172.21.15.70:0/3659098947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a8806dfc0 msgr2=0x7f9a8814ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a8806dfc0 0x7f9a8814ca90 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800c3e0 tx=0x7f9a7800c8a0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 -- 172.21.15.70:0/3659098947 shutdown_connections 2023-10-08T07:15:19.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9a7406acf0 0x7f9a7406d1b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9a880afbe0 0x7f9a8815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9a88071070 0x7f9a880c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 --2- 172.21.15.70:0/3659098947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a8806dfc0 0x7f9a8814ca90 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 -- 172.21.15.70:0/3659098947 >> 172.21.15.70:0/3659098947 conn(0x7f9a8806a5d0 msgr2=0x7f9a880c8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:19.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 -- 172.21.15.70:0/3659098947 shutdown_connections 2023-10-08T07:15:19.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.215+0000 7f9a8e11d640 1 -- 172.21.15.70:0/3659098947 wait complete. 2023-10-08T07:15:19.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-08T07:15:19.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.449+0000 7f53aa232640 1 Processor -- start 2023-10-08T07:15:19.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.450+0000 7f53aa232640 1 -- start start 2023-10-08T07:15:19.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.451+0000 7f53aa232640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f53a40c9cf0 0x7f53a406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.451+0000 7f53aa232640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a415b310 con 0x7f53a406c3a0 2023-10-08T07:15:19.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.451+0000 7f53aa232640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a406d290 con 0x7f53a415aa00 2023-10-08T07:15:19.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.451+0000 7f53aa232640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a406e0a0 con 0x7f53a40c9cf0 2023-10-08T07:15:19.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.452+0000 7f53a37fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f53a406c3a0 0x7f53a406c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49530/0 (socket says 172.21.15.70:49530) 2023-10-08T07:15:19.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.452+0000 7f53a37fe640 1 -- 172.21.15.70:0/3732541992 learned_addr learned my addr 172.21.15.70:0/3732541992 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:19.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.452+0000 7f539bfff640 1 --2- 172.21.15.70:0/3732541992 >> v2:172.21.15.181:3300/0 conn(0x7f53a40c9cf0 0x7f53a406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.452+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164246029 0 0) 0x7f5394001110 con 0x7f53a406c3a0 2023-10-08T07:15:19.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.452+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5390003520 con 0x7f53a406c3a0 2023-10-08T07:15:19.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2711902204 0 0) 0x7f5390003520 con 0x7f53a406c3a0 2023-10-08T07:15:19.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 >> v2:172.21.15.181:3300/0 conn(0x7f53a40c9cf0 msgr2=0x7f53a406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:19.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 --2- 172.21.15.70:0/3732541992 >> v2:172.21.15.181:3300/0 conn(0x7f53a40c9cf0 0x7f53a406db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 >> v1:172.21.15.181:6789/0 conn(0x7f53a415aa00 legacy=0x7f53a40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a40c9860 con 0x7f53a406c3a0 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f5394001800 con 0x7f53a406c3a0 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 >> v1:172.21.15.70:6789/0 conn(0x7f53a406c3a0 legacy=0x7f53a406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53900039c0 0x7f5390003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 0x7f5390004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5390005180 0x7f539000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a406e0a0 con 0x7f53900039c0 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539000c080 con 0x7f53900047c0 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539000c240 con 0x7f5390005180 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f539bfff640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 0x7f5390004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.453+0000 7f53a37fe640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53900039c0 0x7f5390003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.454+0000 7f53a3fff640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5390005180 0x7f539000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.454+0000 7f539bfff640 1 -- 172.21.15.70:0/3732541992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5390005180 msgr2=0x7f539000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:19.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.454+0000 7f539bfff640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5390005180 0x7f539000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.454+0000 7f539bfff640 1 -- 172.21.15.70:0/3732541992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53900039c0 msgr2=0x7f5390003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:19.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.454+0000 7f539bfff640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53900039c0 0x7f5390003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.454+0000 7f539bfff640 1 -- 172.21.15.70:0/3732541992 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5390003520 con 0x7f53900047c0 2023-10-08T07:15:19.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.454+0000 7f53a3fff640 1 -- 172.21.15.70:0/3732541992 reap_dead start 2023-10-08T07:15:19.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.454+0000 7f53a37fe640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53900039c0 0x7f5390003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:15:19.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.456+0000 7f539bfff640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 0x7f5390004c40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f538c009ac0 tx=0x7f53900057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c064d7b3379ab22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:19.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.456+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c02c6b0 con 0x7f53900047c0 2023-10-08T07:15:19.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.456+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c02c810 con 0x7f53900047c0 2023-10-08T07:15:19.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.456+0000 7f53a2ffd640 1 -- 172.21.15.70:0/3732541992 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c02cba0 con 0x7f53900047c0 2023-10-08T07:15:19.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.457+0000 7f53aa232640 1 -- 172.21.15.70:0/3732541992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 msgr2=0x7f5390004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.457+0000 7f53aa232640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 0x7f5390004c40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f538c009ac0 tx=0x7f53900057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.457+0000 7f53aa232640 1 -- 172.21.15.70:0/3732541992 shutdown_connections 2023-10-08T07:15:19.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.457+0000 7f53aa232640 1 --2- 172.21.15.70:0/3732541992 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 0x7f5390004c40 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.457+0000 7f53aa232640 1 -- 172.21.15.70:0/3732541992 >> 172.21.15.70:0/3732541992 conn(0x7f53a406a5d0 msgr2=0x7f53a40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:19.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.457+0000 7f53aa232640 1 -- 172.21.15.70:0/3732541992 shutdown_connections 2023-10-08T07:15:19.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.457+0000 7f53aa232640 1 -- 172.21.15.70:0/3732541992 wait complete. 2023-10-08T07:15:19.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.459+0000 7f53aa232640 1 Processor -- start 2023-10-08T07:15:19.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.459+0000 7f53aa232640 1 -- start start 2023-10-08T07:15:19.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f53aa232640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 0x7f53a40ce810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f53aa232640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53a41f5cf0 0x7f53a40ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f53aa232640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a41f29e0 0x7f53a41f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f53aa232640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4157180 con 0x7f53a41f29e0 2023-10-08T07:15:19.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f53aa232640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41f61d0 con 0x7f53900047c0 2023-10-08T07:15:19.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f53aa232640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a41f64b0 con 0x7f53a41f5cf0 2023-10-08T07:15:19.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f53a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a41f29e0 0x7f53a41f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f53a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a41f29e0 0x7f53a41f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56658/0 (socket says 172.21.15.70:56658) 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f53a3fff640 1 -- 172.21.15.70:0/295432125 learned_addr learned my addr 172.21.15.70:0/295432125 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f539bfff640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53a41f5cf0 0x7f53a40ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.460+0000 7f53a37fe640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 0x7f53a40ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53a3fff640 1 -- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53a41f5cf0 msgr2=0x7f53a40ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53a3fff640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53a41f5cf0 0x7f53a40ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53a3fff640 1 -- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 msgr2=0x7f53a40ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53a3fff640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 0x7f53a40ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53a3fff640 1 -- 172.21.15.70:0/295432125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a41f6790 con 0x7f53a41f29e0 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53a3fff640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a41f29e0 0x7f53a41f2e70 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f53a406e750 tx=0x7f539c00fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53a0ff9640 1 -- 172.21.15.70:0/295432125 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00fec0 con 0x7f53a41f29e0 2023-10-08T07:15:19.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53a0ff9640 1 -- 172.21.15.70:0/295432125 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539c003b80 con 0x7f53a41f29e0 2023-10-08T07:15:19.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53a0ff9640 1 -- 172.21.15.70:0/295432125 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c0050d0 con 0x7f53a41f29e0 2023-10-08T07:15:19.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53aa232640 1 -- 172.21.15.70:0/295432125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a41f6a70 con 0x7f53a41f29e0 2023-10-08T07:15:19.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.461+0000 7f53aa232640 1 -- 172.21.15.70:0/295432125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a41f33b0 con 0x7f53a41f29e0 2023-10-08T07:15:19.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.463+0000 7f53aa232640 1 -- 172.21.15.70:0/295432125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a404e160 con 0x7f53a41f29e0 2023-10-08T07:15:19.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.463+0000 7f53a0ff9640 1 -- 172.21.15.70:0/295432125 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 144) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f539c005230 con 0x7f53a41f29e0 2023-10-08T07:15:19.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.464+0000 7f53a0ff9640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f538406acc0 0x7f538406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.464+0000 7f53a0ff9640 1 -- 172.21.15.70:0/295432125 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f539c0899b0 con 0x7f53a41f29e0 2023-10-08T07:15:19.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.465+0000 7f53a37fe640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f538406acc0 0x7f538406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.466+0000 7f53a37fe640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f538406acc0 0x7f538406d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f53940012b0 tx=0x7f5394009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:19.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.469+0000 7f53a0ff9640 1 -- 172.21.15.70:0/295432125 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f539c054530 con 0x7f53a41f29e0 2023-10-08T07:15:19.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.650+0000 7f53aa232640 1 -- 172.21.15.70:0/295432125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f53a40c4f80 con 0x7f53a41f29e0 2023-10-08T07:15:19.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.650+0000 7f53a0ff9640 1 -- 172.21.15.70:0/295432125 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v260) v1 ==== 103+0+11 (secure 0 0 0) 0x7f539c057b50 con 0x7f53a41f29e0 2023-10-08T07:15:19.653 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:15:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 -- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f538406acc0 msgr2=0x7f538406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f538406acc0 0x7f538406d180 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f53940012b0 tx=0x7f5394009040 comp rx=0 tx=0).stop 2023-10-08T07:15:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 -- 172.21.15.70:0/295432125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a41f29e0 msgr2=0x7f53a41f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a41f29e0 0x7f53a41f2e70 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f53a406e750 tx=0x7f539c00fc70 comp rx=0 tx=0).stop 2023-10-08T07:15:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 -- 172.21.15.70:0/295432125 shutdown_connections 2023-10-08T07:15:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f538406acc0 0x7f538406d180 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53a41f29e0 0x7f53a41f2e70 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53a41f5cf0 0x7f53a40ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 --2- 172.21.15.70:0/295432125 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f53900047c0 0x7f53a40ce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 -- 172.21.15.70:0/295432125 >> 172.21.15.70:0/295432125 conn(0x7f53a406a5d0 msgr2=0x7f53a40c45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 -- 172.21.15.70:0/295432125 shutdown_connections 2023-10-08T07:15:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.653+0000 7f53aa232640 1 -- 172.21.15.70:0/295432125 wait complete. 2023-10-08T07:15:19.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:19.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.875+0000 7f482b447640 1 Processor -- start 2023-10-08T07:15:19.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.875+0000 7f482b447640 1 -- start start 2023-10-08T07:15:19.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.876+0000 7f482b447640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 0x7f48241614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.877+0000 7f482b447640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48240cab40 0x7f48240b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.877+0000 7f482b447640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48240c9020 con 0x7f482415d080 2023-10-08T07:15:19.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.877+0000 7f482b447640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48240ccc20 con 0x7f48240cab40 2023-10-08T07:15:19.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.877+0000 7f482b447640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48240c92e0 con 0x7f482415cca0 2023-10-08T07:15:19.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.877+0000 7f4829c44640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48240cab40 0x7f48240b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.877+0000 7f4829c44640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48240cab40 0x7f48240b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56664/0 (socket says 172.21.15.70:56664) 2023-10-08T07:15:19.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.877+0000 7f4829c44640 1 -- 172.21.15.70:0/1834284004 learned_addr learned my addr 172.21.15.70:0/1834284004 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:19.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.877+0000 7f4829c44640 1 -- 172.21.15.70:0/1834284004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 msgr2=0x7f48241614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:19.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.877+0000 7f482a445640 1 --2- 172.21.15.70:0/1834284004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 0x7f48241614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.878+0000 7f4829c44640 1 --2- 172.21.15.70:0/1834284004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 0x7f48241614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.878+0000 7f4829c44640 1 -- 172.21.15.70:0/1834284004 >> v1:172.21.15.181:6789/0 conn(0x7f482415d080 legacy=0x7f48240701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:19.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.878+0000 7f4829c44640 1 -- 172.21.15.70:0/1834284004 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48240cc7b0 con 0x7f48240cab40 2023-10-08T07:15:19.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.878+0000 7f482a445640 1 --2- 172.21.15.70:0/1834284004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 0x7f48241614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:19.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.879+0000 7f4829c44640 1 --2- 172.21.15.70:0/1834284004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 0x7f48240b00c0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f4818009870 tx=0x7f481802b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efaa3a43632e9ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:19.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.879+0000 7f4829443640 1 -- 172.21.15.70:0/1834284004 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818033620 con 0x7f48240cab40 2023-10-08T07:15:19.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.879+0000 7f4829443640 1 -- 172.21.15.70:0/1834284004 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4818033780 con 0x7f48240cab40 2023-10-08T07:15:19.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.879+0000 7f4829443640 1 -- 172.21.15.70:0/1834284004 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818033b10 con 0x7f48240cab40 2023-10-08T07:15:19.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.880+0000 7f482b447640 1 -- 172.21.15.70:0/1834284004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 msgr2=0x7f48240b00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.880+0000 7f482b447640 1 --2- 172.21.15.70:0/1834284004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 0x7f48240b00c0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f4818009870 tx=0x7f481802b7d0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.880+0000 7f482b447640 1 -- 172.21.15.70:0/1834284004 shutdown_connections 2023-10-08T07:15:19.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.880+0000 7f482b447640 1 --2- 172.21.15.70:0/1834284004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 0x7f48240b00c0 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.880+0000 7f482b447640 1 --2- 172.21.15.70:0/1834284004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 0x7f48241614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.880+0000 7f482b447640 1 -- 172.21.15.70:0/1834284004 >> 172.21.15.70:0/1834284004 conn(0x7f48240cb0c0 msgr2=0x7f48240cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:19.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.880+0000 7f482b447640 1 -- 172.21.15.70:0/1834284004 shutdown_connections 2023-10-08T07:15:19.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.880+0000 7f482b447640 1 -- 172.21.15.70:0/1834284004 wait complete. 2023-10-08T07:15:19.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.882+0000 7f482b447640 1 Processor -- start 2023-10-08T07:15:19.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.882+0000 7f482b447640 1 -- start start 2023-10-08T07:15:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482b447640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 0x7f48241f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482b447640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 0x7f48240c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482b447640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f482415d080 0x7f48240ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482b447640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48240c5d60 con 0x7f48240cab40 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482b447640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48240c5a80 con 0x7f482415d080 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482b447640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48241f5f70 con 0x7f482415cca0 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482a445640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 0x7f48241f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482a445640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 0x7f48241f6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56680/0 (socket says 172.21.15.70:56680) 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482a445640 1 -- 172.21.15.70:0/2756429263 learned_addr learned my addr 172.21.15.70:0/2756429263 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f4829c44640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 0x7f48240c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482a445640 1 -- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 msgr2=0x7f48240c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482a445640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 0x7f48240c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482a445640 1 -- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f482415d080 msgr2=0x7f48240ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482a445640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f482415d080 0x7f48240ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.883+0000 7f482a445640 1 -- 172.21.15.70:0/2756429263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48180094d0 con 0x7f48240cab40 2023-10-08T07:15:19.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.884+0000 7f482a445640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 0x7f48241f6a50 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f482000c7f0 tx=0x7f482000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:19.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.884+0000 7f4816ffd640 1 -- 172.21.15.70:0/2756429263 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820007820 con 0x7f48240cab40 2023-10-08T07:15:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.884+0000 7f4816ffd640 1 -- 172.21.15.70:0/2756429263 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4820007980 con 0x7f48240cab40 2023-10-08T07:15:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.884+0000 7f482b447640 1 -- 172.21.15.70:0/2756429263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48241f6250 con 0x7f48240cab40 2023-10-08T07:15:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.884+0000 7f4816ffd640 1 -- 172.21.15.70:0/2756429263 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4820007cf0 con 0x7f48240cab40 2023-10-08T07:15:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.884+0000 7f482b447640 1 -- 172.21.15.70:0/2756429263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4824160330 con 0x7f48240cab40 2023-10-08T07:15:19.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.885+0000 7f482b447640 1 -- 172.21.15.70:0/2756429263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482404e160 con 0x7f48240cab40 2023-10-08T07:15:19.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.886+0000 7f4816ffd640 1 -- 172.21.15.70:0/2756429263 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f48200113f0 con 0x7f48240cab40 2023-10-08T07:15:19.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.887+0000 7f4816ffd640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f480806b020 0x7f480806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:19.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.888+0000 7f4816ffd640 1 -- 172.21.15.70:0/2756429263 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f482008a660 con 0x7f48240cab40 2023-10-08T07:15:19.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.888+0000 7f4829c44640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f480806b020 0x7f480806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:19.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.889+0000 7f4829c44640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f480806b020 0x7f480806d4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4818035000 tx=0x7f4818008030 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:19.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:19.891+0000 7f4816ffd640 1 -- 172.21.15.70:0/2756429263 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4820054a70 con 0x7f48240cab40 2023-10-08T07:15:20.079 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:20.076+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:20.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.127+0000 7f482b447640 1 -- 172.21.15.70:0/2756429263 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48240b00b0 con 0x7f48240cab40 2023-10-08T07:15:20.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.127+0000 7f4816ffd640 1 -- 172.21.15.70:0/2756429263 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+6149 (secure 0 0 0) 0x7f482001f090 con 0x7f48240cab40 2023-10-08T07:15:20.129 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:20.129 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":130,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:43.833416+0000","modified":"2023-10-08T07:14:46.899347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11904},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11904":{"gid":11904,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/3856326680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3856326680},{"type":"v1","addr":"172.21.15.70:6834","nonce":3856326680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:57.955428+0000","modified":"2023-10-08T07:15:01.029708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8875":{"gid":8875,"name":"b","rank":0,"incarnation":127,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-08T07:15:20.130 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 130 2023-10-08T07:15:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 -- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f480806b020 msgr2=0x7f480806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f480806b020 0x7f480806d4e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4818035000 tx=0x7f4818008030 comp rx=0 tx=0).stop 2023-10-08T07:15:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 -- 172.21.15.70:0/2756429263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 msgr2=0x7f48241f6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 0x7f48241f6a50 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7f482000c7f0 tx=0x7f482000ccb0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 -- 172.21.15.70:0/2756429263 shutdown_connections 2023-10-08T07:15:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f480806b020 0x7f480806d4e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f482415d080 0x7f48240ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f482415cca0 0x7f48240c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 --2- 172.21.15.70:0/2756429263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48240cab40 0x7f48241f6a50 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 -- 172.21.15.70:0/2756429263 >> 172.21.15.70:0/2756429263 conn(0x7f48240cb0c0 msgr2=0x7f48241610c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.130+0000 7f482b447640 1 -- 172.21.15.70:0/2756429263 shutdown_connections 2023-10-08T07:15:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.131+0000 7f482b447640 1 -- 172.21.15.70:0/2756429263 wait complete. 2023-10-08T07:15:20.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.354+0000 7f31129c5640 1 Processor -- start 2023-10-08T07:15:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.354+0000 7f31129c5640 1 -- start start 2023-10-08T07:15:20.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.356+0000 7f31129c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c159ac0 0x7f310c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.356+0000 7f31129c5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f310c0709e0 0x7f310c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.356+0000 7f31129c5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310c06dee0 con 0x7f310c0ad790 2023-10-08T07:15:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.356+0000 7f31129c5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310c15d850 con 0x7f310c0709e0 2023-10-08T07:15:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.356+0000 7f31129c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310c15db30 con 0x7f310c159ac0 2023-10-08T07:15:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.356+0000 7f3110f62640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f310c0709e0 0x7f310c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.356+0000 7f3110f62640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f310c0709e0 0x7f310c0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56682/0 (socket says 172.21.15.70:56682) 2023-10-08T07:15:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.356+0000 7f3110f62640 1 -- 172.21.15.70:0/1810087090 learned_addr learned my addr 172.21.15.70:0/1810087090 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.356+0000 7f310b7fe640 1 --1- 172.21.15.70:0/1810087090 >> v1:172.21.15.70:6789/0 conn(0x7f310c0ad790 0x7f310c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49544/0 (socket says 172.21.15.70:49544) 2023-10-08T07:15:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.357+0000 7f310bfff640 1 --2- 172.21.15.70:0/1810087090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c159ac0 0x7f310c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.357+0000 7f3110f62640 1 -- 172.21.15.70:0/1810087090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c159ac0 msgr2=0x7f310c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.357+0000 7f3110f62640 1 --2- 172.21.15.70:0/1810087090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c159ac0 0x7f310c06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.357+0000 7f3110f62640 1 -- 172.21.15.70:0/1810087090 >> v1:172.21.15.70:6789/0 conn(0x7f310c0ad790 legacy=0x7f310c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.357+0000 7f310affd640 1 -- 172.21.15.70:0/1810087090 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123598209 0 0) 0x7f310c06dee0 con 0x7f310c0ad790 2023-10-08T07:15:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.357+0000 7f3110f62640 1 -- 172.21.15.70:0/1810087090 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c15de10 con 0x7f310c0709e0 2023-10-08T07:15:20.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.357+0000 7f310bfff640 1 --2- 172.21.15.70:0/1810087090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c159ac0 0x7f310c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.358+0000 7f3110f62640 1 --2- 172.21.15.70:0/1810087090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c0709e0 0x7f310c0c8dd0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f3104009e50 tx=0x7f310402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9d8beae2277ada server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.358+0000 7f310affd640 1 -- 172.21.15.70:0/1810087090 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310402cb00 con 0x7f310c0709e0 2023-10-08T07:15:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.358+0000 7f310affd640 1 -- 172.21.15.70:0/1810087090 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310402cc60 con 0x7f310c0709e0 2023-10-08T07:15:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.358+0000 7f310affd640 1 -- 172.21.15.70:0/1810087090 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31040343f0 con 0x7f310c0709e0 2023-10-08T07:15:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.359+0000 7f31129c5640 1 -- 172.21.15.70:0/1810087090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c0709e0 msgr2=0x7f310c0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.359+0000 7f31129c5640 1 --2- 172.21.15.70:0/1810087090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c0709e0 0x7f310c0c8dd0 secure :-1 s=READY pgs=2771 cs=0 l=1 rev1=1 crypto rx=0x7f3104009e50 tx=0x7f310402bf60 comp rx=0 tx=0).stop 2023-10-08T07:15:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.359+0000 7f31129c5640 1 -- 172.21.15.70:0/1810087090 shutdown_connections 2023-10-08T07:15:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.359+0000 7f31129c5640 1 --2- 172.21.15.70:0/1810087090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c0709e0 0x7f310c0c8dd0 unknown :-1 s=CLOSED pgs=2771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.359+0000 7f31129c5640 1 --2- 172.21.15.70:0/1810087090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c159ac0 0x7f310c06d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.359+0000 7f31129c5640 1 -- 172.21.15.70:0/1810087090 >> 172.21.15.70:0/1810087090 conn(0x7f310c06a380 msgr2=0x7f310c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.359+0000 7f31129c5640 1 -- 172.21.15.70:0/1810087090 shutdown_connections 2023-10-08T07:15:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.359+0000 7f31129c5640 1 -- 172.21.15.70:0/1810087090 wait complete. 2023-10-08T07:15:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.362+0000 7f31129c5640 1 Processor -- start 2023-10-08T07:15:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.362+0000 7f31129c5640 1 -- start start 2023-10-08T07:15:20.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.363+0000 7f31129c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310c0709e0 0x7f310c153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.363+0000 7f31129c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c0ad790 0x7f310c14a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.363+0000 7f31129c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c159ac0 0x7f310c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.363+0000 7f31129c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c14a9e0 con 0x7f310c159ac0 2023-10-08T07:15:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.363+0000 7f31129c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c14acc0 con 0x7f310c0709e0 2023-10-08T07:15:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.363+0000 7f31129c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310c165a70 con 0x7f310c0ad790 2023-10-08T07:15:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.363+0000 7f3110f62640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c159ac0 0x7f310c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.363+0000 7f3110f62640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c159ac0 0x7f310c0cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56698/0 (socket says 172.21.15.70:56698) 2023-10-08T07:15:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.363+0000 7f3110f62640 1 -- 172.21.15.70:0/3235055860 learned_addr learned my addr 172.21.15.70:0/3235055860 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.363+0000 7f310bfff640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310c0709e0 0x7f310c153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f310b7fe640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c0ad790 0x7f310c14a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f3110f62640 1 -- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c0ad790 msgr2=0x7f310c14a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f3110f62640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c0ad790 0x7f310c14a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f3110f62640 1 -- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310c0709e0 msgr2=0x7f310c153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f3110f62640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310c0709e0 0x7f310c153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f3110f62640 1 -- 172.21.15.70:0/3235055860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3104009b30 con 0x7f310c159ac0 2023-10-08T07:15:20.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f310b7fe640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c0ad790 0x7f310c14a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:20.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f3110f62640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c159ac0 0x7f310c0cda60 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f3104036040 tx=0x7f3104004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f3108ff9640 1 -- 172.21.15.70:0/3235055860 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310403e450 con 0x7f310c159ac0 2023-10-08T07:15:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f3108ff9640 1 -- 172.21.15.70:0/3235055860 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310403e5d0 con 0x7f310c159ac0 2023-10-08T07:15:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f31129c5640 1 -- 172.21.15.70:0/3235055860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c165d50 con 0x7f310c159ac0 2023-10-08T07:15:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.364+0000 7f3108ff9640 1 -- 172.21.15.70:0/3235055860 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310403e980 con 0x7f310c159ac0 2023-10-08T07:15:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.365+0000 7f31129c5640 1 -- 172.21.15.70:0/3235055860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c14a6d0 con 0x7f310c159ac0 2023-10-08T07:15:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.366+0000 7f31129c5640 1 -- 172.21.15.70:0/3235055860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c166270 con 0x7f310c159ac0 2023-10-08T07:15:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.367+0000 7f3108ff9640 1 -- 172.21.15.70:0/3235055860 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f310403eae0 con 0x7f310c159ac0 2023-10-08T07:15:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.368+0000 7f3108ff9640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f30f806b020 0x7f30f806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.368+0000 7f3108ff9640 1 -- 172.21.15.70:0/3235055860 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f31040ad100 con 0x7f310c159ac0 2023-10-08T07:15:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.368+0000 7f310bfff640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f30f806b020 0x7f30f806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.370+0000 7f310bfff640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f30f806b020 0x7f30f806d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00a950 tx=0x7f30fc008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:20.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.372+0000 7f3108ff9640 1 -- 172.21.15.70:0/3235055860 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f310c166270 con 0x7f310c159ac0 2023-10-08T07:15:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.562+0000 7f31129c5640 1 -- 172.21.15.70:0/3235055860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f310c166270 con 0x7f310c159ac0 2023-10-08T07:15:20.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.566+0000 7f3108ff9640 1 -- 172.21.15.70:0/3235055860 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v260) v1 ==== 74+0+21882 (secure 0 0 0) 0x7f310c166270 con 0x7f310c159ac0 2023-10-08T07:15:20.568 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:20.569 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":260,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:16.946287+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:14:56.178783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:14:56.977158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/888851152":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3676453840":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3956317048":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1513601554":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6832/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/348399880":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1607590530":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/2690144496":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/3464560328":"2023-10-09T07:15:10.723067+0000","172.21.15.181:6832/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/807583929":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/859617464":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/4271328810":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/735805106":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/957356199":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2539797528":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/333977458":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2403193161":"2023-10-09T07:15:10.723067+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 -- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f30f806b020 msgr2=0x7f30f806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f30f806b020 0x7f30f806d4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f30fc00a950 tx=0x7f30fc008040 comp rx=0 tx=0).stop 2023-10-08T07:15:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 -- 172.21.15.70:0/3235055860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c159ac0 msgr2=0x7f310c0cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c159ac0 0x7f310c0cda60 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f3104036040 tx=0x7f3104004a40 comp rx=0 tx=0).stop 2023-10-08T07:15:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 -- 172.21.15.70:0/3235055860 shutdown_connections 2023-10-08T07:15:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f30f806b020 0x7f30f806d4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f310c159ac0 0x7f310c0cda60 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f310c0ad790 0x7f310c14a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 --2- 172.21.15.70:0/3235055860 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f310c0709e0 0x7f310c153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 -- 172.21.15.70:0/3235055860 >> 172.21.15.70:0/3235055860 conn(0x7f310c06a380 msgr2=0x7f310c0b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:20.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 -- 172.21.15.70:0/3235055860 shutdown_connections 2023-10-08T07:15:20.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.569+0000 7f31129c5640 1 -- 172.21.15.70:0/3235055860 wait complete. 2023-10-08T07:15:20.589 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T07:15:20.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:20.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.807+0000 7f0083261640 1 Processor -- start 2023-10-08T07:15:20.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.807+0000 7f0083261640 1 -- start start 2023-10-08T07:15:20.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.809+0000 7f0083261640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f007c160940 0x7f007c0709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.809+0000 7f0083261640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007c0c48f0 con 0x7f007c15cd90 2023-10-08T07:15:20.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.809+0000 7f0083261640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007c0c3ce0 con 0x7f007c06c890 2023-10-08T07:15:20.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.809+0000 7f0083261640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007c0c4d80 con 0x7f007c160940 2023-10-08T07:15:20.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.809+0000 7f0073fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f007c160940 0x7f007c0709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:20.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.809+0000 7f0080ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f007c15cd90 0x7f007c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49548/0 (socket says 172.21.15.70:49548) 2023-10-08T07:15:20.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.810+0000 7f0080ffd640 1 -- 172.21.15.70:0/190735606 learned_addr learned my addr 172.21.15.70:0/190735606 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:20.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.810+0000 7f0073fff640 1 -- 172.21.15.70:0/190735606 >> v1:172.21.15.181:6789/0 conn(0x7f007c06c890 legacy=0x7f007c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:20.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.810+0000 7f0073fff640 1 -- 172.21.15.70:0/190735606 >> v1:172.21.15.70:6789/0 conn(0x7f007c15cd90 legacy=0x7f007c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.810+0000 7f0073fff640 1 -- 172.21.15.70:0/190735606 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c0c3fc0 con 0x7f007c160940 2023-10-08T07:15:20.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.810+0000 7f00737fe640 1 -- 172.21.15.70:0/190735606 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443466160 0 0) 0x7f007c0c48f0 con 0x7f007c15cd90 2023-10-08T07:15:20.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.811+0000 7f0073fff640 1 --2- 172.21.15.70:0/190735606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 0x7f007c0709e0 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f006c0097a0 tx=0x7f006c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea25645ad148c60 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:20.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.811+0000 7f00737fe640 1 -- 172.21.15.70:0/190735606 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c031830 con 0x7f007c160940 2023-10-08T07:15:20.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.811+0000 7f00737fe640 1 -- 172.21.15.70:0/190735606 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c031990 con 0x7f007c160940 2023-10-08T07:15:20.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.811+0000 7f00737fe640 1 -- 172.21.15.70:0/190735606 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c031d20 con 0x7f007c160940 2023-10-08T07:15:20.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.812+0000 7f0083261640 1 -- 172.21.15.70:0/190735606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 msgr2=0x7f007c0709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.812+0000 7f0083261640 1 --2- 172.21.15.70:0/190735606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 0x7f007c0709e0 secure :-1 s=READY pgs=2774 cs=0 l=1 rev1=1 crypto rx=0x7f006c0097a0 tx=0x7f006c02b8d0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.812+0000 7f0083261640 1 -- 172.21.15.70:0/190735606 shutdown_connections 2023-10-08T07:15:20.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.812+0000 7f0083261640 1 --2- 172.21.15.70:0/190735606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 0x7f007c0709e0 unknown :-1 s=CLOSED pgs=2774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.812+0000 7f0083261640 1 -- 172.21.15.70:0/190735606 >> 172.21.15.70:0/190735606 conn(0x7f007c06a380 msgr2=0x7f007c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:20.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.812+0000 7f0083261640 1 -- 172.21.15.70:0/190735606 shutdown_connections 2023-10-08T07:15:20.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.812+0000 7f0083261640 1 -- 172.21.15.70:0/190735606 wait complete. 2023-10-08T07:15:20.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.814+0000 7f0083261640 1 Processor -- start 2023-10-08T07:15:20.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.814+0000 7f0083261640 1 -- start start 2023-10-08T07:15:20.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.815+0000 7f0083261640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f007c06c890 0x7f007c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.815+0000 7f0083261640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f007c15cd90 0x7f007c147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.815+0000 7f0083261640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 0x7f007c15ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.815+0000 7f0083261640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c1475b0 con 0x7f007c160940 2023-10-08T07:15:20.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.815+0000 7f0083261640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c0c9390 con 0x7f007c15cd90 2023-10-08T07:15:20.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.815+0000 7f0083261640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c0c9670 con 0x7f007c06c890 2023-10-08T07:15:20.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.815+0000 7f00817fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 0x7f007c15ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:20.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.815+0000 7f00817fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 0x7f007c15ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56710/0 (socket says 172.21.15.70:56710) 2023-10-08T07:15:20.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.815+0000 7f00817fe640 1 -- 172.21.15.70:0/1056697213 learned_addr learned my addr 172.21.15.70:0/1056697213 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:20.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f0080ffd640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f007c06c890 0x7f007c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:20.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f00817fe640 1 -- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f007c06c890 msgr2=0x7f007c0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f00817fe640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f007c06c890 0x7f007c0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f00817fe640 1 -- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f007c15cd90 msgr2=0x7f007c147070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:20.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f00817fe640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f007c15cd90 0x7f007c147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:20.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f00817fe640 1 -- 172.21.15.70:0/1056697213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c009400 con 0x7f007c160940 2023-10-08T07:15:20.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f00817fe640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 0x7f007c15ec10 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f007400c390 tx=0x7f007400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:20.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f00717fa640 1 -- 172.21.15.70:0/1056697213 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00740035f0 con 0x7f007c160940 2023-10-08T07:15:20.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f00717fa640 1 -- 172.21.15.70:0/1056697213 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0074003750 con 0x7f007c160940 2023-10-08T07:15:20.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f00717fa640 1 -- 172.21.15.70:0/1056697213 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0074003b00 con 0x7f007c160940 2023-10-08T07:15:20.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.816+0000 7f0083261640 1 -- 172.21.15.70:0/1056697213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007c0c9950 con 0x7f007c160940 2023-10-08T07:15:20.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.817+0000 7f0083261640 1 -- 172.21.15.70:0/1056697213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c0c9f10 con 0x7f007c160940 2023-10-08T07:15:20.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.818+0000 7f0083261640 1 -- 172.21.15.70:0/1056697213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007c067490 con 0x7f007c160940 2023-10-08T07:15:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.819+0000 7f00717fa640 1 -- 172.21.15.70:0/1056697213 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f0074003c60 con 0x7f007c160940 2023-10-08T07:15:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.820+0000 7f00717fa640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f006806b020 0x7f006806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.820+0000 7f00717fa640 1 -- 172.21.15.70:0/1056697213 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f0074089490 con 0x7f007c160940 2023-10-08T07:15:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.820+0000 7f0080ffd640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f006806b020 0x7f006806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.821+0000 7f0080ffd640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f006806b020 0x7f006806d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f00780012e0 tx=0x7f0078009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:20.824+0000 7f00717fa640 1 -- 172.21.15.70:0/1056697213 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f00740539b0 con 0x7f007c160940 2023-10-08T07:15:21.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.066+0000 7f0083261640 1 -- 172.21.15.70:0/1056697213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f007c0c4550 con 0x7f007c160940 2023-10-08T07:15:21.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.066+0000 7f00717fa640 1 -- 172.21.15.70:0/1056697213 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+6149 (secure 0 0 0) 0x7f0074017090 con 0x7f007c160940 2023-10-08T07:15:21.068 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:21.068 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":130,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:43.833416+0000","modified":"2023-10-08T07:14:46.899347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11904},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11904":{"gid":11904,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/3856326680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3856326680},{"type":"v1","addr":"172.21.15.70:6834","nonce":3856326680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:57.955428+0000","modified":"2023-10-08T07:15:01.029708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8875":{"gid":8875,"name":"b","rank":0,"incarnation":127,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-08T07:15:21.069 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 130 2023-10-08T07:15:21.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 -- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f006806b020 msgr2=0x7f006806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f006806b020 0x7f006806d4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f00780012e0 tx=0x7f0078009040 comp rx=0 tx=0).stop 2023-10-08T07:15:21.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 -- 172.21.15.70:0/1056697213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 msgr2=0x7f007c15ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 0x7f007c15ec10 secure :-1 s=READY pgs=2775 cs=0 l=1 rev1=1 crypto rx=0x7f007400c390 tx=0x7f007400c850 comp rx=0 tx=0).stop 2023-10-08T07:15:21.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 -- 172.21.15.70:0/1056697213 shutdown_connections 2023-10-08T07:15:21.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f006806b020 0x7f006806d4e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f007c160940 0x7f007c15ec10 unknown :-1 s=CLOSED pgs=2775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f007c15cd90 0x7f007c147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 --2- 172.21.15.70:0/1056697213 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f007c06c890 0x7f007c0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 -- 172.21.15.70:0/1056697213 >> 172.21.15.70:0/1056697213 conn(0x7f007c06a380 msgr2=0x7f007c0af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:21.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 -- 172.21.15.70:0/1056697213 shutdown_connections 2023-10-08T07:15:21.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.069+0000 7f0083261640 1 -- 172.21.15.70:0/1056697213 wait complete. 2023-10-08T07:15:21.090 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:21.155 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:21.152+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:21.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.303+0000 7f0e15039640 1 Processor -- start 2023-10-08T07:15:21.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.303+0000 7f0e15039640 1 -- start start 2023-10-08T07:15:21.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.305+0000 7f0e15039640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e100709e0 0x7f0e10164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.305+0000 7f0e15039640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1006dee0 con 0x7f0e100ad790 2023-10-08T07:15:21.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.305+0000 7f0e15039640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1015d850 con 0x7f0e10159ac0 2023-10-08T07:15:21.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.305+0000 7f0e15039640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1006dcf0 con 0x7f0e100709e0 2023-10-08T07:15:21.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.305+0000 7f0e07fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0e100ad790 0x7f0e100702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49562/0 (socket says 172.21.15.70:49562) 2023-10-08T07:15:21.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.305+0000 7f0e07fff640 1 -- 172.21.15.70:0/735758015 learned_addr learned my addr 172.21.15.70:0/735758015 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:21.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.305+0000 7f0e0f59e640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e100709e0 0x7f0e10164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151954167 0 0) 0x7f0e1006dee0 con 0x7f0e100ad790 2023-10-08T07:15:21.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dfc003520 con 0x7f0e100ad790 2023-10-08T07:15:21.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2656030930 0 0) 0x7f0dfc003520 con 0x7f0e100ad790 2023-10-08T07:15:21.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e100709e0 msgr2=0x7f0e10164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:21.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e100709e0 0x7f0e10164a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 >> v1:172.21.15.181:6789/0 conn(0x7f0e10159ac0 legacy=0x7f0e1006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1015db30 con 0x7f0e100ad790 2023-10-08T07:15:21.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0df8001670 con 0x7f0e100ad790 2023-10-08T07:15:21.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 >> v1:172.21.15.70:6789/0 conn(0x7f0e100ad790 legacy=0x7f0e100702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dfc0039a0 0x7f0dfc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e100709e0 0x7f0dfc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dfc004d00 0x7f0dfc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc00bb10 con 0x7f0dfc004d00 2023-10-08T07:15:21.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc00bc50 con 0x7f0dfc0039a0 2023-10-08T07:15:21.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc00be10 con 0x7f0e100709e0 2023-10-08T07:15:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.306+0000 7f0e0f59e640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dfc004d00 0x7f0dfc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.307+0000 7f0e0ed9d640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dfc0039a0 0x7f0dfc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.307+0000 7f0e0f59e640 1 -- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e100709e0 msgr2=0x7f0dfc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.307+0000 7f0e07fff640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e100709e0 0x7f0dfc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.307+0000 7f0e0f59e640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e100709e0 0x7f0dfc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.307+0000 7f0e0f59e640 1 -- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dfc0039a0 msgr2=0x7f0dfc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.307+0000 7f0e0f59e640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dfc0039a0 0x7f0dfc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.307+0000 7f0e0f59e640 1 -- 172.21.15.70:0/735758015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc003520 con 0x7f0dfc004d00 2023-10-08T07:15:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.307+0000 7f0e07fff640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0e100709e0 0x7f0dfc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.307+0000 7f0e0ed9d640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0dfc0039a0 0x7f0dfc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:21.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.308+0000 7f0e0f59e640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dfc004d00 0x7f0dfc005180 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f0e100ce750 tx=0x7f0dfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb753f2f2f71f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.308+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0802cc40 con 0x7f0dfc004d00 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.308+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0802cda0 con 0x7f0dfc004d00 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.308+0000 7f0e0e59c640 1 -- 172.21.15.70:0/735758015 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08034680 con 0x7f0dfc004d00 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.309+0000 7f0e15039640 1 -- 172.21.15.70:0/735758015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dfc004d00 msgr2=0x7f0dfc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.309+0000 7f0e15039640 1 --2- 172.21.15.70:0/735758015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0dfc004d00 0x7f0dfc005180 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f0e100ce750 tx=0x7f0dfc0057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.309+0000 7f0e0f59e640 1 -- 172.21.15.70:0/735758015 reap_dead start 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.309+0000 7f0e15039640 1 -- 172.21.15.70:0/735758015 shutdown_connections 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.309+0000 7f0e15039640 1 -- 172.21.15.70:0/735758015 >> 172.21.15.70:0/735758015 conn(0x7f0e1006a380 msgr2=0x7f0e100b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.309+0000 7f0e15039640 1 -- 172.21.15.70:0/735758015 shutdown_connections 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.309+0000 7f0e15039640 1 -- 172.21.15.70:0/735758015 wait complete. 2023-10-08T07:15:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.311+0000 7f0e15039640 1 Processor -- start 2023-10-08T07:15:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.311+0000 7f0e15039640 1 -- start start 2023-10-08T07:15:21.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.312+0000 7f0e15039640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dfc004d00 0x7f0e10149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.312+0000 7f0e15039640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e100acc70 0x7f0e10149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.312+0000 7f0e15039640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e100ad0f0 0x7f0e1014a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.312+0000 7f0e15039640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100ab9f0 con 0x7f0e100acc70 2023-10-08T07:15:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.312+0000 7f0e15039640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100abcd0 con 0x7f0e100ad0f0 2023-10-08T07:15:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.312+0000 7f0e15039640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100abfb0 con 0x7f0dfc004d00 2023-10-08T07:15:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.312+0000 7f0e07fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e100acc70 0x7f0e10149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.312+0000 7f0e07fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e100acc70 0x7f0e10149e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56734/0 (socket says 172.21.15.70:56734) 2023-10-08T07:15:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.312+0000 7f0e07fff640 1 -- 172.21.15.70:0/3013303063 learned_addr learned my addr 172.21.15.70:0/3013303063 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e0f59e640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e100ad0f0 0x7f0e1014a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e07fff640 1 -- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dfc004d00 msgr2=0x7f0e10149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e0ed9d640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dfc004d00 0x7f0e10149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e07fff640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dfc004d00 0x7f0e10149350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e07fff640 1 -- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e100ad0f0 msgr2=0x7f0e1014a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e07fff640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e100ad0f0 0x7f0e1014a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e07fff640 1 -- 172.21.15.70:0/3013303063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e100ac290 con 0x7f0e100acc70 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e0f59e640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e100ad0f0 0x7f0e1014a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e07fff640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e100acc70 0x7f0e10149e20 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f0df800d8b0 tx=0x7f0df800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e077fe640 1 -- 172.21.15.70:0/3013303063 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df80074e0 con 0x7f0e100acc70 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e077fe640 1 -- 172.21.15.70:0/3013303063 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df8007640 con 0x7f0e100acc70 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e077fe640 1 -- 172.21.15.70:0/3013303063 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df80049e0 con 0x7f0e100acc70 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.313+0000 7f0e15039640 1 -- 172.21.15.70:0/3013303063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e100ac570 con 0x7f0e100acc70 2023-10-08T07:15:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.314+0000 7f0e15039640 1 -- 172.21.15.70:0/3013303063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e10149890 con 0x7f0e100acc70 2023-10-08T07:15:21.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.315+0000 7f0e15039640 1 -- 172.21.15.70:0/3013303063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e10067490 con 0x7f0e100acc70 2023-10-08T07:15:21.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.316+0000 7f0e077fe640 1 -- 172.21.15.70:0/3013303063 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f0df8004b40 con 0x7f0e100acc70 2023-10-08T07:15:21.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.317+0000 7f0e077fe640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0df006af80 0x7f0df006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.317+0000 7f0e077fe640 1 -- 172.21.15.70:0/3013303063 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f0df8089d00 con 0x7f0e100acc70 2023-10-08T07:15:21.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.317+0000 7f0e0ed9d640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0df006af80 0x7f0df006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.319+0000 7f0e0ed9d640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0df006af80 0x7f0df006d440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e00001630 tx=0x7f0e00009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:21.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.321+0000 7f0e077fe640 1 -- 172.21.15.70:0/3013303063 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0df8054300 con 0x7f0e100acc70 2023-10-08T07:15:21.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.563+0000 7f0e15039640 1 -- 172.21.15.70:0/3013303063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e100b0ff0 con 0x7f0e100acc70 2023-10-08T07:15:21.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.564+0000 7f0e077fe640 1 -- 172.21.15.70:0/3013303063 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+6149 (secure 0 0 0) 0x7f0df8019020 con 0x7f0e100acc70 2023-10-08T07:15:21.566 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:21.566 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":130,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:43.833416+0000","modified":"2023-10-08T07:14:46.899347+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11904},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11904":{"gid":11904,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/3856326680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":3856326680},{"type":"v1","addr":"172.21.15.70:6834","nonce":3856326680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15},{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:57.955428+0000","modified":"2023-10-08T07:15:01.029708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8875":{"gid":8875,"name":"b","rank":0,"incarnation":127,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-08T07:15:21.567 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 130 2023-10-08T07:15:21.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 -- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0df006af80 msgr2=0x7f0df006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0df006af80 0x7f0df006d440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e00001630 tx=0x7f0e00009040 comp rx=0 tx=0).stop 2023-10-08T07:15:21.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 -- 172.21.15.70:0/3013303063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e100acc70 msgr2=0x7f0e10149e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e100acc70 0x7f0e10149e20 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7f0df800d8b0 tx=0x7f0df800dd70 comp rx=0 tx=0).stop 2023-10-08T07:15:21.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 -- 172.21.15.70:0/3013303063 shutdown_connections 2023-10-08T07:15:21.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0df006af80 0x7f0df006d440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0e100ad0f0 0x7f0e1014a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e100acc70 0x7f0e10149e20 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 --2- 172.21.15.70:0/3013303063 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0dfc004d00 0x7f0e10149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 -- 172.21.15.70:0/3013303063 >> 172.21.15.70:0/3013303063 conn(0x7f0e1006a380 msgr2=0x7f0e100c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:21.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 -- 172.21.15.70:0/3013303063 shutdown_connections 2023-10-08T07:15:21.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.567+0000 7f0e15039640 1 -- 172.21.15.70:0/3013303063 wait complete. 2023-10-08T07:15:21.588 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.798+0000 7f2da11b6640 1 Processor -- start 2023-10-08T07:15:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.799+0000 7f2da11b6640 1 -- start start 2023-10-08T07:15:21.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.800+0000 7f2da11b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c0ad790 0x7f2d9c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.800+0000 7f2da11b6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9c06dee0 con 0x7f2d9c15e330 2023-10-08T07:15:21.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.800+0000 7f2da11b6640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9c15d850 con 0x7f2d9c159ac0 2023-10-08T07:15:21.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.800+0000 7f2da11b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c06dcf0 con 0x7f2d9c0ad790 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.801+0000 7f2d9b59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2d9c15e330 0x7f2d9c0c8dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49564/0 (socket says 172.21.15.70:49564) 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.801+0000 7f2d9b59e640 1 -- 172.21.15.70:0/1941448688 learned_addr learned my addr 172.21.15.70:0/1941448688 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.801+0000 7f2d9a59c640 1 --2- 172.21.15.70:0/1941448688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c0ad790 0x7f2d9c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.801+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024284103 0 0) 0x7f2d9c06dee0 con 0x7f2d9c15e330 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.801+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d84003520 con 0x7f2d9c15e330 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372219672 0 0) 0x7f2d84003520 con 0x7f2d9c15e330 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c0ad790 msgr2=0x7f2d9c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 --2- 172.21.15.70:0/1941448688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c0ad790 0x7f2d9c0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 >> v1:172.21.15.181:6789/0 conn(0x7f2d9c159ac0 legacy=0x7f2d9c06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c15db30 con 0x7f2d9c15e330 2023-10-08T07:15:21.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2d940019f0 con 0x7f2d9c15e330 2023-10-08T07:15:21.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 >> v1:172.21.15.70:6789/0 conn(0x7f2d9c15e330 legacy=0x7f2d9c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 --2- 172.21.15.70:0/1941448688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840039a0 0x7f2d84003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 --2- 172.21.15.70:0/1941448688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 0x7f2d84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 --2- 172.21.15.70:0/1941448688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c0ad790 0x7f2d84005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8400ba30 con 0x7f2d840047c0 2023-10-08T07:15:21.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8400bb70 con 0x7f2d840039a0 2023-10-08T07:15:21.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8400bd30 con 0x7f2d9c0ad790 2023-10-08T07:15:21.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d9a59c640 1 --2- 172.21.15.70:0/1941448688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 0x7f2d84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d9a59c640 1 -- 172.21.15.70:0/1941448688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c0ad790 msgr2=0x7f2d84005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d9a59c640 1 --2- 172.21.15.70:0/1941448688 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c0ad790 0x7f2d84005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d9a59c640 1 -- 172.21.15.70:0/1941448688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840039a0 msgr2=0x7f2d84003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.802+0000 7f2d9a59c640 1 --2- 172.21.15.70:0/1941448688 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d840039a0 0x7f2d84003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.803+0000 7f2d9a59c640 1 -- 172.21.15.70:0/1941448688 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84003520 con 0x7f2d840047c0 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.804+0000 7f2d9a59c640 1 --2- 172.21.15.70:0/1941448688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 0x7f2d84004c40 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0adbd0 tx=0x7f2d840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e4805bafbd361c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.804+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9002c680 con 0x7f2d840047c0 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.804+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9002c7e0 con 0x7f2d840047c0 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.804+0000 7f2d99d9b640 1 -- 172.21.15.70:0/1941448688 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9002cb70 con 0x7f2d840047c0 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.804+0000 7f2da11b6640 1 -- 172.21.15.70:0/1941448688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 msgr2=0x7f2d84004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.804+0000 7f2da11b6640 1 --2- 172.21.15.70:0/1941448688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 0x7f2d84004c40 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0adbd0 tx=0x7f2d840057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.804+0000 7f2d9b59e640 1 -- 172.21.15.70:0/1941448688 reap_dead start 2023-10-08T07:15:21.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.805+0000 7f2da11b6640 1 -- 172.21.15.70:0/1941448688 shutdown_connections 2023-10-08T07:15:21.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.805+0000 7f2da11b6640 1 -- 172.21.15.70:0/1941448688 >> 172.21.15.70:0/1941448688 conn(0x7f2d9c06a380 msgr2=0x7f2d9c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:21.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.805+0000 7f2da11b6640 1 -- 172.21.15.70:0/1941448688 shutdown_connections 2023-10-08T07:15:21.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.805+0000 7f2da11b6640 1 -- 172.21.15.70:0/1941448688 wait complete. 2023-10-08T07:15:21.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.806+0000 7f2da11b6640 1 Processor -- start 2023-10-08T07:15:21.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.807+0000 7f2da11b6640 1 -- start start 2023-10-08T07:15:21.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.807+0000 7f2da11b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 0x7f2d9c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.807+0000 7f2da11b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c0cedd0 0x7f2d9c0cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2da11b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c151a90 0x7f2d9c151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2da11b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c0b4440 con 0x7f2d840047c0 2023-10-08T07:15:21.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2da11b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c152460 con 0x7f2d9c0cedd0 2023-10-08T07:15:21.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2da11b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c0b38c0 con 0x7f2d9c151a90 2023-10-08T07:15:21.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2d9ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 0x7f2d9c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2d9ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 0x7f2d9c0af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56750/0 (socket says 172.21.15.70:56750) 2023-10-08T07:15:21.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2d9ad9d640 1 -- 172.21.15.70:0/1904038512 learned_addr learned my addr 172.21.15.70:0/1904038512 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:21.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2d9b59e640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c151a90 0x7f2d9c151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2d9ad9d640 1 -- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c151a90 msgr2=0x7f2d9c151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2d9ad9d640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c151a90 0x7f2d9c151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2d9ad9d640 1 -- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c0cedd0 msgr2=0x7f2d9c0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2d9a59c640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c0cedd0 0x7f2d9c0cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2d9ad9d640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c0cedd0 0x7f2d9c0cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.808+0000 7f2d9ad9d640 1 -- 172.21.15.70:0/1904038512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c0b3ba0 con 0x7f2d840047c0 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.809+0000 7f2d9ad9d640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 0x7f2d9c0af6e0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00c500 tx=0x7f2d8c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.809+0000 7f2d837fe640 1 -- 172.21.15.70:0/1904038512 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c0052d0 con 0x7f2d840047c0 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.809+0000 7f2d837fe640 1 -- 172.21.15.70:0/1904038512 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c005430 con 0x7f2d840047c0 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.809+0000 7f2da11b6640 1 -- 172.21.15.70:0/1904038512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9c0b3e80 con 0x7f2d840047c0 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.809+0000 7f2d837fe640 1 -- 172.21.15.70:0/1904038512 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c012870 con 0x7f2d840047c0 2023-10-08T07:15:21.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.809+0000 7f2da11b6640 1 -- 172.21.15.70:0/1904038512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c0b35b0 con 0x7f2d840047c0 2023-10-08T07:15:21.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.810+0000 7f2da11b6640 1 -- 172.21.15.70:0/1904038512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9c0cb000 con 0x7f2d840047c0 2023-10-08T07:15:21.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.811+0000 7f2d837fe640 1 -- 172.21.15.70:0/1904038512 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f2d8c0129d0 con 0x7f2d840047c0 2023-10-08T07:15:21.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.812+0000 7f2d837fe640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2d8806b050 0x7f2d8806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:21.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.812+0000 7f2d837fe640 1 -- 172.21.15.70:0/1904038512 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7f2d8c089d00 con 0x7f2d840047c0 2023-10-08T07:15:21.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.812+0000 7f2d9a59c640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2d8806b050 0x7f2d8806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:21.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.814+0000 7f2d9a59c640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2d8806b050 0x7f2d8806d510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2d90036000 tx=0x7f2d90030000 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:21.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:21.816+0000 7f2d837fe640 1 -- 172.21.15.70:0/1904038512 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d8c054350 con 0x7f2d840047c0 2023-10-08T07:15:22.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.015+0000 7f2da11b6640 1 -- 172.21.15.70:0/1904038512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d9c067490 con 0x7f2d840047c0 2023-10-08T07:15:22.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.019+0000 7f2d837fe640 1 -- 172.21.15.70:0/1904038512 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v260) v1 ==== 74+0+21882 (secure 0 0 0) 0x7f2d8c057970 con 0x7f2d840047c0 2023-10-08T07:15:22.022 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:22.023 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":260,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:16.946287+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:14:41.606127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T07:14:42.867820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:14:56.178783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:14:56.977158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/888851152":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3676453840":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3956317048":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1513601554":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6832/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/348399880":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1607590530":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/2690144496":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/3464560328":"2023-10-09T07:15:10.723067+0000","172.21.15.181:6832/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/807583929":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/859617464":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/4271328810":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/735805106":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/957356199":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2539797528":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/333977458":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2403193161":"2023-10-09T07:15:10.723067+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:22.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 -- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2d8806b050 msgr2=0x7f2d8806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:22.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2d8806b050 0x7f2d8806d510 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2d90036000 tx=0x7f2d90030000 comp rx=0 tx=0).stop 2023-10-08T07:15:22.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 -- 172.21.15.70:0/1904038512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 msgr2=0x7f2d9c0af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:22.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 0x7f2d9c0af6e0 secure :-1 s=READY pgs=2781 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c00c500 tx=0x7f2d8c00c9c0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 -- 172.21.15.70:0/1904038512 shutdown_connections 2023-10-08T07:15:22.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2d8806b050 0x7f2d8806d510 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2d9c151a90 0x7f2d9c151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2d9c0cedd0 0x7f2d9c0cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 --2- 172.21.15.70:0/1904038512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d840047c0 0x7f2d9c0af6e0 unknown :-1 s=CLOSED pgs=2781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 -- 172.21.15.70:0/1904038512 >> 172.21.15.70:0/1904038512 conn(0x7f2d9c06a380 msgr2=0x7f2d9c06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:22.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 -- 172.21.15.70:0/1904038512 shutdown_connections 2023-10-08T07:15:22.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.022+0000 7f2da11b6640 1 -- 172.21.15.70:0/1904038512 wait complete. 2023-10-08T07:15:22.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T07:15:22.241 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:22.238+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:22.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.254+0000 7fed9b032640 1 Processor -- start 2023-10-08T07:15:22.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.254+0000 7fed9b032640 1 -- start start 2023-10-08T07:15:22.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed9b032640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fed941603a0 0x7fed940cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:22.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed9b032640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed940cdff0 0x7fed94071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:22.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed9b032640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed94071b80 0x7fed9406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:22.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed9b032640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940c3100 con 0x7fed940cdff0 2023-10-08T07:15:22.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed9b032640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940ce480 con 0x7fed941603a0 2023-10-08T07:15:22.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed9b032640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed940c9ff0 con 0x7fed94071b80 2023-10-08T07:15:22.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed93fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed940cdff0 0x7fed94071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:22.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed93fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed940cdff0 0x7fed94071640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56764/0 (socket says 172.21.15.70:56764) 2023-10-08T07:15:22.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed93fff640 1 -- 172.21.15.70:0/3827223796 learned_addr learned my addr 172.21.15.70:0/3827223796 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:22.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed995cf640 1 --2- 172.21.15.70:0/3827223796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed94071b80 0x7fed9406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:22.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed98dce640 1 --2- 172.21.15.70:0/3827223796 >> v2:172.21.15.181:3300/0 conn(0x7fed941603a0 0x7fed940cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:22.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed93fff640 1 -- 172.21.15.70:0/3827223796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed94071b80 msgr2=0x7fed9406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:22.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed93fff640 1 --2- 172.21.15.70:0/3827223796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed94071b80 0x7fed9406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed93fff640 1 -- 172.21.15.70:0/3827223796 >> v2:172.21.15.181:3300/0 conn(0x7fed941603a0 msgr2=0x7fed940cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:22.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed93fff640 1 --2- 172.21.15.70:0/3827223796 >> v2:172.21.15.181:3300/0 conn(0x7fed941603a0 0x7fed940cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.256+0000 7fed93fff640 1 -- 172.21.15.70:0/3827223796 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed94071ff0 con 0x7fed940cdff0 2023-10-08T07:15:22.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.257+0000 7fed995cf640 1 --2- 172.21.15.70:0/3827223796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed94071b80 0x7fed9406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:22.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.257+0000 7fed98dce640 1 --2- 172.21.15.70:0/3827223796 >> v2:172.21.15.181:3300/0 conn(0x7fed941603a0 0x7fed940cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:22.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.258+0000 7fed93fff640 1 --2- 172.21.15.70:0/3827223796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed940cdff0 0x7fed94071640 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7fed940ccef0 tx=0x7fed8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36373729ed6b1fa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:22.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.258+0000 7fed937fe640 1 -- 172.21.15.70:0/3827223796 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80031870 con 0x7fed940cdff0 2023-10-08T07:15:22.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.258+0000 7fed937fe640 1 -- 172.21.15.70:0/3827223796 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed800319d0 con 0x7fed940cdff0 2023-10-08T07:15:22.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.258+0000 7fed937fe640 1 -- 172.21.15.70:0/3827223796 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80031d60 con 0x7fed940cdff0 2023-10-08T07:15:22.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.258+0000 7fed9b032640 1 -- 172.21.15.70:0/3827223796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed940cdff0 msgr2=0x7fed94071640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:22.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.258+0000 7fed9b032640 1 --2- 172.21.15.70:0/3827223796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed940cdff0 0x7fed94071640 secure :-1 s=READY pgs=2782 cs=0 l=1 rev1=1 crypto rx=0x7fed940ccef0 tx=0x7fed8002b910 comp rx=0 tx=0).stop 2023-10-08T07:15:22.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.259+0000 7fed9b032640 1 -- 172.21.15.70:0/3827223796 shutdown_connections 2023-10-08T07:15:22.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.259+0000 7fed9b032640 1 --2- 172.21.15.70:0/3827223796 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed94071b80 0x7fed9406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.259+0000 7fed9b032640 1 --2- 172.21.15.70:0/3827223796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed940cdff0 0x7fed94071640 unknown :-1 s=CLOSED pgs=2782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.259+0000 7fed9b032640 1 --2- 172.21.15.70:0/3827223796 >> v2:172.21.15.181:3300/0 conn(0x7fed941603a0 0x7fed940cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.259+0000 7fed9b032640 1 -- 172.21.15.70:0/3827223796 >> 172.21.15.70:0/3827223796 conn(0x7fed940cc2e0 msgr2=0x7fed940cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:22.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.259+0000 7fed9b032640 1 -- 172.21.15.70:0/3827223796 shutdown_connections 2023-10-08T07:15:22.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.259+0000 7fed9b032640 1 -- 172.21.15.70:0/3827223796 wait complete. 2023-10-08T07:15:22.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.261+0000 7fed9b032640 1 Processor -- start 2023-10-08T07:15:22.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.261+0000 7fed9b032640 1 -- start start 2023-10-08T07:15:22.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.262+0000 7fed9b032640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed94071b80 0x7fed9414bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:22.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.262+0000 7fed9b032640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed940cdff0 0x7fed940ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:22.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.262+0000 7fed9b032640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed941603a0 0x7fed9415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:22.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.262+0000 7fed9b032640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9414aab0 con 0x7fed941603a0 2023-10-08T07:15:22.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.262+0000 7fed9b032640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9414ad90 con 0x7fed94071b80 2023-10-08T07:15:22.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.262+0000 7fed9b032640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9414b070 con 0x7fed940cdff0 2023-10-08T07:15:22.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.262+0000 7fed995cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed941603a0 0x7fed9415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:22.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.262+0000 7fed995cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed941603a0 0x7fed9415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56768/0 (socket says 172.21.15.70:56768) 2023-10-08T07:15:22.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.262+0000 7fed995cf640 1 -- 172.21.15.70:0/3149621576 learned_addr learned my addr 172.21.15.70:0/3149621576 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:22.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.263+0000 7fed98dce640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed94071b80 0x7fed9414bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:22.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.263+0000 7fed93fff640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed940cdff0 0x7fed940ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:22.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.263+0000 7fed995cf640 1 -- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed940cdff0 msgr2=0x7fed940ab8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:22.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.263+0000 7fed995cf640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed940cdff0 0x7fed940ab8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.263+0000 7fed995cf640 1 -- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed94071b80 msgr2=0x7fed9414bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:22.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.263+0000 7fed995cf640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed94071b80 0x7fed9414bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.263+0000 7fed995cf640 1 -- 172.21.15.70:0/3149621576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed800094d0 con 0x7fed941603a0 2023-10-08T07:15:22.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.263+0000 7fed93fff640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed940cdff0 0x7fed940ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:22.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.263+0000 7fed995cf640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed941603a0 0x7fed9415ec50 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00cef0 tx=0x7fed8c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:22.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.263+0000 7fed917fa640 1 -- 172.21.15.70:0/3149621576 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c00d840 con 0x7fed941603a0 2023-10-08T07:15:22.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.264+0000 7fed917fa640 1 -- 172.21.15.70:0/3149621576 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c00d9a0 con 0x7fed941603a0 2023-10-08T07:15:22.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.264+0000 7fed917fa640 1 -- 172.21.15.70:0/3149621576 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c00dd50 con 0x7fed941603a0 2023-10-08T07:15:22.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.264+0000 7fed9b032640 1 -- 172.21.15.70:0/3149621576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9414b350 con 0x7fed941603a0 2023-10-08T07:15:22.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.264+0000 7fed9b032640 1 -- 172.21.15.70:0/3149621576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9414c2d0 con 0x7fed941603a0 2023-10-08T07:15:22.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.265+0000 7fed9b032640 1 -- 172.21.15.70:0/3149621576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9414b870 con 0x7fed941603a0 2023-10-08T07:15:22.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.266+0000 7fed917fa640 1 -- 172.21.15.70:0/3149621576 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fed8c01e3b0 con 0x7fed941603a0 2023-10-08T07:15:22.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.267+0000 7fed917fa640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fed7806b020 0x7fed7806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:22.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.267+0000 7fed917fa640 1 -- 172.21.15.70:0/3149621576 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(260..260 src has 1..260) v4 ==== 7869+0+0 (secure 0 0 0) 0x7fed8c08ac90 con 0x7fed941603a0 2023-10-08T07:15:22.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.268+0000 7fed98dce640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fed7806b020 0x7fed7806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:22.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.269+0000 7fed98dce640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fed7806b020 0x7fed7806d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fed8800a950 tx=0x7fed88008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:22.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.272+0000 7fed917fa640 1 -- 172.21.15.70:0/3149621576 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed8c0551b0 con 0x7fed941603a0 2023-10-08T07:15:22.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.504+0000 7fed9b032640 1 -- 172.21.15.70:0/3149621576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fed9406a080 con 0x7fed941603a0 2023-10-08T07:15:22.507 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T07:15:22.506+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T07:15:22.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.510+0000 7fed917fa640 1 -- 172.21.15.70:0/3149621576 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v131) v1 ==== 161+0+0 (secure 0 0 0) 0x7fed8c01a020 con 0x7fed941603a0 2023-10-08T07:15:22.511 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T07:15:22.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.513+0000 7fed9b032640 1 -- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fed7806b020 msgr2=0x7fed7806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:22.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.513+0000 7fed9b032640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fed7806b020 0x7fed7806d4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fed8800a950 tx=0x7fed88008040 comp rx=0 tx=0).stop 2023-10-08T07:15:22.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.513+0000 7fed9b032640 1 -- 172.21.15.70:0/3149621576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed941603a0 msgr2=0x7fed9415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:22.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.513+0000 7fed9b032640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed941603a0 0x7fed9415ec50 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7fed8c00cef0 tx=0x7fed8c00a710 comp rx=0 tx=0).stop 2023-10-08T07:15:22.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.515+0000 7fed9b032640 1 -- 172.21.15.70:0/3149621576 shutdown_connections 2023-10-08T07:15:22.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.515+0000 7fed9b032640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fed7806b020 0x7fed7806d4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.515+0000 7fed9b032640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed941603a0 0x7fed9415ec50 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.515+0000 7fed9b032640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fed940cdff0 0x7fed940ab8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.515+0000 7fed9b032640 1 --2- 172.21.15.70:0/3149621576 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fed94071b80 0x7fed9414bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:22.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.515+0000 7fed9b032640 1 -- 172.21.15.70:0/3149621576 >> 172.21.15.70:0/3149621576 conn(0x7fed940cc2e0 msgr2=0x7fed9415c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:22.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.515+0000 7fed9b032640 1 -- 172.21.15.70:0/3149621576 shutdown_connections 2023-10-08T07:15:22.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.515+0000 7fed9b032640 1 -- 172.21.15.70:0/3149621576 wait complete. 2023-10-08T07:15:22.534 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T07:15:22.540 INFO:tasks.ceph.mds.c.smithi070.stderr: -4277> 2023-10-08T07:14:49.336+0000 7f6e640ce640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:15:22.610 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T07:15:22.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.722+0000 7f09728a2640 1 Processor -- start 2023-10-08T07:15:23.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.722+0000 7f09728a2640 1 -- start start 2023-10-08T07:15:23.455 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:23.361+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:23.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.723+0000 7f09728a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f096c06c3a0 0x7f096c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:23.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.723+0000 7f09728a2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f096c06db60 0x7f096c06c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:23.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.723+0000 7f09728a2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096c15b310 con 0x7f096c15aa00 2023-10-08T07:15:23.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.723+0000 7f09728a2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096c0ca230 con 0x7f096c06db60 2023-10-08T07:15:23.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.723+0000 7f09728a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c06d290 con 0x7f096c06c3a0 2023-10-08T07:15:23.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.724+0000 7f096b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f096c06db60 0x7f096c06c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:23.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.724+0000 7f096b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f096c06db60 0x7f096c06c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56800/0 (socket says 172.21.15.70:56800) 2023-10-08T07:15:23.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.724+0000 7f096b7fe640 1 -- 172.21.15.70:0/3789137438 learned_addr learned my addr 172.21.15.70:0/3789137438 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:23.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.724+0000 7f096bfff640 1 --2- 172.21.15.70:0/3789137438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f096c06c3a0 0x7f096c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:23.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.724+0000 7f096b7fe640 1 -- 172.21.15.70:0/3789137438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f096c06c3a0 msgr2=0x7f096c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:23.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.724+0000 7f096b7fe640 1 --2- 172.21.15.70:0/3789137438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f096c06c3a0 0x7f096c0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.724+0000 7f096b7fe640 1 -- 172.21.15.70:0/3789137438 >> v1:172.21.15.181:6789/0 conn(0x7f096c15aa00 legacy=0x7f096c0b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:23.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.724+0000 7f096b7fe640 1 -- 172.21.15.70:0/3789137438 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c0c9860 con 0x7f096c06db60 2023-10-08T07:15:23.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.725+0000 7f096b7fe640 1 --2- 172.21.15.70:0/3789137438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06db60 0x7f096c06c780 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f095400bb10 tx=0x7f095402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2336ca673769de4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:23.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f096affd640 1 -- 172.21.15.70:0/3789137438 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095402ee70 con 0x7f096c06db60 2023-10-08T07:15:23.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f096affd640 1 -- 172.21.15.70:0/3789137438 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0954036440 con 0x7f096c06db60 2023-10-08T07:15:23.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f096affd640 1 -- 172.21.15.70:0/3789137438 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09540367d0 con 0x7f096c06db60 2023-10-08T07:15:23.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f09728a2640 1 -- 172.21.15.70:0/3789137438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06db60 msgr2=0x7f096c06c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:23.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f09728a2640 1 --2- 172.21.15.70:0/3789137438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06db60 0x7f096c06c780 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f095400bb10 tx=0x7f095402e0c0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f09728a2640 1 -- 172.21.15.70:0/3789137438 shutdown_connections 2023-10-08T07:15:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f09728a2640 1 --2- 172.21.15.70:0/3789137438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06db60 0x7f096c06c780 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f09728a2640 1 --2- 172.21.15.70:0/3789137438 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f096c06c3a0 0x7f096c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f09728a2640 1 -- 172.21.15.70:0/3789137438 >> 172.21.15.70:0/3789137438 conn(0x7f096c06a5d0 msgr2=0x7f096c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f09728a2640 1 -- 172.21.15.70:0/3789137438 shutdown_connections 2023-10-08T07:15:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.726+0000 7f09728a2640 1 -- 172.21.15.70:0/3789137438 wait complete. 2023-10-08T07:15:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.728+0000 7f09728a2640 1 Processor -- start 2023-10-08T07:15:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.728+0000 7f09728a2640 1 -- start start 2023-10-08T07:15:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.729+0000 7f09728a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06c3a0 0x7f096c1632d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f09728a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c06db60 0x7f096c0c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f09728a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f096c15aa00 0x7f096c142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f09728a2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c143640 con 0x7f096c06c3a0 2023-10-08T07:15:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f09728a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c143920 con 0x7f096c06db60 2023-10-08T07:15:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f09728a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c143c00 con 0x7f096c15aa00 2023-10-08T07:15:23.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f096bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06c3a0 0x7f096c1632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f096bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06c3a0 0x7f096c1632d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56802/0 (socket says 172.21.15.70:56802) 2023-10-08T07:15:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f096bfff640 1 -- 172.21.15.70:0/2989330490 learned_addr learned my addr 172.21.15.70:0/2989330490 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f096bfff640 1 -- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f096c15aa00 msgr2=0x7f096c142d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f096bfff640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f096c15aa00 0x7f096c142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f096bfff640 1 -- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c06db60 msgr2=0x7f096c0c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:23.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f096bfff640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c06db60 0x7f096c0c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f096bfff640 1 -- 172.21.15.70:0/2989330490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095400b770 con 0x7f096c06c3a0 2023-10-08T07:15:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.730+0000 7f096bfff640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06c3a0 0x7f096c1632d0 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f095c007ef0 tx=0x7f095c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.731+0000 7f0968ff9640 1 -- 172.21.15.70:0/2989330490 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c014620 con 0x7f096c06c3a0 2023-10-08T07:15:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.731+0000 7f0968ff9640 1 -- 172.21.15.70:0/2989330490 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c014780 con 0x7f096c06c3a0 2023-10-08T07:15:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.731+0000 7f0968ff9640 1 -- 172.21.15.70:0/2989330490 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c014b30 con 0x7f096c06c3a0 2023-10-08T07:15:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.731+0000 7f09728a2640 1 -- 172.21.15.70:0/2989330490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c143ee0 con 0x7f096c06c3a0 2023-10-08T07:15:23.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.731+0000 7f09728a2640 1 -- 172.21.15.70:0/2989330490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c163810 con 0x7f096c06c3a0 2023-10-08T07:15:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.732+0000 7f09728a2640 1 -- 172.21.15.70:0/2989330490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c04e160 con 0x7f096c06c3a0 2023-10-08T07:15:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.733+0000 7f0968ff9640 1 -- 172.21.15.70:0/2989330490 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f095c014c90 con 0x7f096c06c3a0 2023-10-08T07:15:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.734+0000 7f0968ff9640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f094c06b020 0x7f094c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.734+0000 7f0968ff9640 1 -- 172.21.15.70:0/2989330490 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 7955+0+0 (secure 0 0 0) 0x7f095c08abf0 con 0x7f096c06c3a0 2023-10-08T07:15:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.734+0000 7f096b7fe640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f094c06b020 0x7f094c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.735+0000 7f096b7fe640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f094c06b020 0x7f094c06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0954038000 tx=0x7f0954030000 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:23.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.738+0000 7f0968ff9640 1 -- 172.21.15.70:0/2989330490 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f095c055150 con 0x7f096c06c3a0 2023-10-08T07:15:23.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:22.975+0000 7f09728a2640 1 -- 172.21.15.70:0/2989330490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f096c0c94e0 con 0x7f096c06c3a0 2023-10-08T07:15:23.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.513+0000 7f0968ff9640 1 -- 172.21.15.70:0/2989330490 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v132) v1 ==== 104+0+0 (secure 0 0 0) 0x7f095c058770 con 0x7f096c06c3a0 2023-10-08T07:15:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 -- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f094c06b020 msgr2=0x7f094c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f094c06b020 0x7f094c06d4e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0954038000 tx=0x7f0954030000 comp rx=0 tx=0).stop 2023-10-08T07:15:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 -- 172.21.15.70:0/2989330490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06c3a0 msgr2=0x7f096c1632d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06c3a0 0x7f096c1632d0 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f095c007ef0 tx=0x7f095c00e4b0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 -- 172.21.15.70:0/2989330490 shutdown_connections 2023-10-08T07:15:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f094c06b020 0x7f094c06d4e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f096c15aa00 0x7f096c142d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f096c06db60 0x7f096c0c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 --2- 172.21.15.70:0/2989330490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f096c06c3a0 0x7f096c1632d0 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:23.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 -- 172.21.15.70:0/2989330490 >> 172.21.15.70:0/2989330490 conn(0x7f096c06a5d0 msgr2=0x7f096c0b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:23.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.517+0000 7f09728a2640 1 -- 172.21.15.70:0/2989330490 shutdown_connections 2023-10-08T07:15:23.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.518+0000 7f09728a2640 1 -- 172.21.15.70:0/2989330490 wait complete. 2023-10-08T07:15:23.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T07:15:23.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.935+0000 7f9ecd1b0640 1 Processor -- start 2023-10-08T07:15:24.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.936+0000 7f9ecd1b0640 1 -- start start 2023-10-08T07:15:24.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.937+0000 7f9ecd1b0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f9ec806c3a0 0x7f9ec80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:24.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.937+0000 7f9ecd1b0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec815b310 con 0x7f9ec806db60 2023-10-08T07:15:24.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.937+0000 7f9ecd1b0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec806d290 con 0x7f9ec815aa00 2023-10-08T07:15:24.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.937+0000 7f9ecd1b0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec80ca230 con 0x7f9ec806c3a0 2023-10-08T07:15:24.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.938+0000 7f9ebffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9ec806db60 0x7f9ec806c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50258/0 (socket says 172.21.15.70:50258) 2023-10-08T07:15:24.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.938+0000 7f9ebffff640 1 -- 172.21.15.70:0/1510877004 learned_addr learned my addr 172.21.15.70:0/1510877004 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:24.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.938+0000 7f9ec6d9d640 1 --2- 172.21.15.70:0/1510877004 >> v2:172.21.15.181:3300/0 conn(0x7f9ec806c3a0 0x7f9ec80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:24.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.938+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553393711 0 0) 0x7f9eb0001110 con 0x7f9ec806db60 2023-10-08T07:15:24.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.938+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb4003520 con 0x7f9ec806db60 2023-10-08T07:15:24.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.938+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 654067727 0 0) 0x7f9eb4003520 con 0x7f9ec806db60 2023-10-08T07:15:24.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.938+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 >> v2:172.21.15.181:3300/0 conn(0x7f9ec806c3a0 msgr2=0x7f9ec80c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:24.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.938+0000 7f9ec659c640 1 --2- 172.21.15.70:0/1510877004 >> v2:172.21.15.181:3300/0 conn(0x7f9ec806c3a0 0x7f9ec80c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.938+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 >> v1:172.21.15.181:6789/0 conn(0x7f9ec815aa00 legacy=0x7f9ec80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:24.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.938+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80c9860 con 0x7f9ec806db60 2023-10-08T07:15:24.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f9eb0001800 con 0x7f9ec806db60 2023-10-08T07:15:24.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 >> v1:172.21.15.70:6789/0 conn(0x7f9ec806db60 legacy=0x7f9ec806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:24.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ec659c640 1 --2- 172.21.15.70:0/1510877004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb4003970 0x7f9eb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:24.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ec659c640 1 --2- 172.21.15.70:0/1510877004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb40047c0 0x7f9eb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:24.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ec659c640 1 --2- 172.21.15.70:0/1510877004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9eb4005180 0x7f9eb400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:24.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb400e200 con 0x7f9eb40047c0 2023-10-08T07:15:24.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb400e370 con 0x7f9eb4003970 2023-10-08T07:15:24.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb400e530 con 0x7f9eb4005180 2023-10-08T07:15:24.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ebffff640 1 --2- 172.21.15.70:0/1510877004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb40047c0 0x7f9eb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:24.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ebffff640 1 -- 172.21.15.70:0/1510877004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9eb4005180 msgr2=0x7f9eb400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:24.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ebffff640 1 --2- 172.21.15.70:0/1510877004 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9eb4005180 0x7f9eb400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ebffff640 1 -- 172.21.15.70:0/1510877004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb4003970 msgr2=0x7f9eb4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:24.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ebffff640 1 --2- 172.21.15.70:0/1510877004 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb4003970 0x7f9eb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ebffff640 1 -- 172.21.15.70:0/1510877004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4003520 con 0x7f9eb40047c0 2023-10-08T07:15:24.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.939+0000 7f9ec759e640 1 -- 172.21.15.70:0/1510877004 reap_dead start 2023-10-08T07:15:24.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.940+0000 7f9ebffff640 1 --2- 172.21.15.70:0/1510877004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb40047c0 0x7f9eb4004c40 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000c190 tx=0x7f9eb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a258bb2c56b163 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:24.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.941+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0005ad0 con 0x7f9eb40047c0 2023-10-08T07:15:24.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.941+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb0005c50 con 0x7f9eb40047c0 2023-10-08T07:15:24.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.941+0000 7f9ec659c640 1 -- 172.21.15.70:0/1510877004 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb002c890 con 0x7f9eb40047c0 2023-10-08T07:15:24.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.941+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/1510877004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb40047c0 msgr2=0x7f9eb4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:24.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.941+0000 7f9ecd1b0640 1 --2- 172.21.15.70:0/1510877004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb40047c0 0x7f9eb4004c40 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000c190 tx=0x7f9eb40057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.941+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/1510877004 shutdown_connections 2023-10-08T07:15:24.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.941+0000 7f9ecd1b0640 1 --2- 172.21.15.70:0/1510877004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9eb40047c0 0x7f9eb4004c40 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.941+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/1510877004 >> 172.21.15.70:0/1510877004 conn(0x7f9ec806a5d0 msgr2=0x7f9ec80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:24.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.941+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/1510877004 shutdown_connections 2023-10-08T07:15:24.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.942+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/1510877004 wait complete. 2023-10-08T07:15:24.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.943+0000 7f9ecd1b0640 1 Processor -- start 2023-10-08T07:15:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.943+0000 7f9ecd1b0640 1 -- start start 2023-10-08T07:15:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.944+0000 7f9ecd1b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb40047c0 0x7f9ec815c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.944+0000 7f9ecd1b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ec814b760 0x7f9ec80ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.944+0000 7f9ecd1b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80caf30 0x7f9ec814c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.944+0000 7f9ecd1b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec815b6e0 con 0x7f9ec80caf30 2023-10-08T07:15:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.944+0000 7f9ecd1b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec80cb3e0 con 0x7f9eb40047c0 2023-10-08T07:15:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.944+0000 7f9ecd1b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec815c8e0 con 0x7f9ec814b760 2023-10-08T07:15:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.945+0000 7f9ec759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80caf30 0x7f9ec814c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.945+0000 7f9ec759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80caf30 0x7f9ec814c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59144/0 (socket says 172.21.15.70:59144) 2023-10-08T07:15:24.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.945+0000 7f9ec759e640 1 -- 172.21.15.70:0/4017629605 learned_addr learned my addr 172.21.15.70:0/4017629605 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:24.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.945+0000 7f9ec759e640 1 -- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ec814b760 msgr2=0x7f9ec80ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:24.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.945+0000 7f9ec759e640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ec814b760 0x7f9ec80ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.945+0000 7f9ec759e640 1 -- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb40047c0 msgr2=0x7f9ec815c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:24.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.945+0000 7f9ec6d9d640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb40047c0 0x7f9ec815c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:24.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.945+0000 7f9ec759e640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb40047c0 0x7f9ec815c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.945+0000 7f9ec759e640 1 -- 172.21.15.70:0/4017629605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec814d2e0 con 0x7f9ec80caf30 2023-10-08T07:15:24.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.945+0000 7f9ec759e640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80caf30 0x7f9ec814c5d0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f9ec815ade0 tx=0x7f9ec000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:24.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.946+0000 7f9ebf7fe640 1 -- 172.21.15.70:0/4017629605 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0003970 con 0x7f9ec80caf30 2023-10-08T07:15:24.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.946+0000 7f9ebf7fe640 1 -- 172.21.15.70:0/4017629605 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0003ad0 con 0x7f9ec80caf30 2023-10-08T07:15:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.946+0000 7f9ebf7fe640 1 -- 172.21.15.70:0/4017629605 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0003e80 con 0x7f9ec80caf30 2023-10-08T07:15:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.946+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/4017629605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec815b8c0 con 0x7f9ec80caf30 2023-10-08T07:15:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.946+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/4017629605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec815be80 con 0x7f9ec80caf30 2023-10-08T07:15:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.947+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/4017629605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec804e160 con 0x7f9ec80caf30 2023-10-08T07:15:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.948+0000 7f9ebf7fe640 1 -- 172.21.15.70:0/4017629605 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f9ec0003fe0 con 0x7f9ec80caf30 2023-10-08T07:15:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.949+0000 7f9ebf7fe640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ea806b050 0x7f9ea806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.949+0000 7f9ebf7fe640 1 -- 172.21.15.70:0/4017629605 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(261..261 src has 1..261) v4 ==== 7955+0+0 (secure 0 0 0) 0x7f9ec00894d0 con 0x7f9ec80caf30 2023-10-08T07:15:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.950+0000 7f9ec6d9d640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ea806b050 0x7f9ea806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:24.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.951+0000 7f9ec6d9d640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ea806b050 0x7f9ea806d510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8004770 tx=0x7f9eb8008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:24.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:23.953+0000 7f9ebf7fe640 1 -- 172.21.15.70:0/4017629605 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ec0053a30 con 0x7f9ec80caf30 2023-10-08T07:15:24.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.150+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/4017629605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9ec815c040 con 0x7f9ec80caf30 2023-10-08T07:15:24.507 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:24.505+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:24.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.874+0000 7f9ebf7fe640 1 -- 172.21.15.70:0/4017629605 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v262) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9ec0057050 con 0x7f9ec80caf30 2023-10-08T07:15:24.916 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T07:15:24.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.878+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ea806b050 msgr2=0x7f9ea806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:24.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.878+0000 7f9ecd1b0640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ea806b050 0x7f9ea806d510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8004770 tx=0x7f9eb8008040 comp rx=0 tx=0).stop 2023-10-08T07:15:24.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.878+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/4017629605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80caf30 msgr2=0x7f9ec814c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:24.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.878+0000 7f9ecd1b0640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80caf30 0x7f9ec814c5d0 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7f9ec815ade0 tx=0x7f9ec000c980 comp rx=0 tx=0).stop 2023-10-08T07:15:24.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.878+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/4017629605 shutdown_connections 2023-10-08T07:15:24.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.878+0000 7f9ecd1b0640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ea806b050 0x7f9ea806d510 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.878+0000 7f9ecd1b0640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ec80caf30 0x7f9ec814c5d0 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.878+0000 7f9ecd1b0640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9ec814b760 0x7f9ec80ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.878+0000 7f9ecd1b0640 1 --2- 172.21.15.70:0/4017629605 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9eb40047c0 0x7f9ec815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:24.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.878+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/4017629605 >> 172.21.15.70:0/4017629605 conn(0x7f9ec806a5d0 msgr2=0x7f9ec80afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:24.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.879+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/4017629605 shutdown_connections 2023-10-08T07:15:24.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:24.880+0000 7f9ecd1b0640 1 -- 172.21.15.70:0/4017629605 wait complete. 2023-10-08T07:15:24.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T07:15:25.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.135+0000 7f60bf59e640 1 Processor -- start 2023-10-08T07:15:25.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.136+0000 7f60bf59e640 1 -- start start 2023-10-08T07:15:25.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.137+0000 7f60bf59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c015a820 0x7f60c00aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:25.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.137+0000 7f60bf59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c00cab60 con 0x7f60c015ac00 2023-10-08T07:15:25.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.137+0000 7f60bf59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c00cb960 con 0x7f60c00ca680 2023-10-08T07:15:25.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.137+0000 7f60bf59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00cbdf0 con 0x7f60c015a820 2023-10-08T07:15:25.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.137+0000 7f60bed9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f60c015ac00 0x7f60c00c9810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50264/0 (socket says 172.21.15.70:50264) 2023-10-08T07:15:25.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.137+0000 7f60bed9d640 1 -- 172.21.15.70:0/2349447655 learned_addr learned my addr 172.21.15.70:0/2349447655 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:25.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60be59c640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c015a820 0x7f60c00aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:25.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483587996 0 0) 0x7f60c00cab60 con 0x7f60c015ac00 2023-10-08T07:15:25.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60a8003520 con 0x7f60c015ac00 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 530699744 0 0) 0x7f60a8003520 con 0x7f60c015ac00 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c015a820 msgr2=0x7f60c00aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c015a820 0x7f60c00aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 >> v1:172.21.15.181:6789/0 conn(0x7f60c00ca680 legacy=0x7f60c00c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c00c9f20 con 0x7f60c015ac00 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f60b8001670 con 0x7f60c015ac00 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 >> v1:172.21.15.70:6789/0 conn(0x7f60c015ac00 legacy=0x7f60c00c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c015a820 0x7f60a80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 0x7f60a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a8004d00 0x7f60a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:25.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00cbdf0 con 0x7f60a8004020 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a800bc60 con 0x7f60a8004d00 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.138+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a800be20 con 0x7f60c015a820 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.139+0000 7f60bdd9b640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 0x7f60a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.139+0000 7f60bed9d640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a8004d00 0x7f60a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.139+0000 7f60be59c640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c015a820 0x7f60a80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.139+0000 7f60bdd9b640 1 -- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c015a820 msgr2=0x7f60a80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.139+0000 7f60bdd9b640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c015a820 0x7f60a80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.139+0000 7f60bdd9b640 1 -- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a8004d00 msgr2=0x7f60a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.139+0000 7f60bdd9b640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a8004d00 0x7f60a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.139+0000 7f60bdd9b640 1 -- 172.21.15.70:0/2349447655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a8003520 con 0x7f60a8004020 2023-10-08T07:15:25.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.139+0000 7f60bed9d640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60a8004d00 0x7f60a8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.140+0000 7f60bdd9b640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 0x7f60a80047c0 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f60b4001750 tx=0x7f60a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee89f4f38c47f4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.140+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b40052d0 con 0x7f60a8004020 2023-10-08T07:15:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.141+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b402e050 con 0x7f60a8004020 2023-10-08T07:15:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.141+0000 7f60bd59a640 1 -- 172.21.15.70:0/2349447655 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4003830 con 0x7f60a8004020 2023-10-08T07:15:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.141+0000 7f60bf59e640 1 -- 172.21.15.70:0/2349447655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 msgr2=0x7f60a80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.141+0000 7f60bf59e640 1 --2- 172.21.15.70:0/2349447655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 0x7f60a80047c0 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7f60b4001750 tx=0x7f60a80057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.141+0000 7f60bed9d640 1 -- 172.21.15.70:0/2349447655 reap_dead start 2023-10-08T07:15:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.141+0000 7f60bf59e640 1 -- 172.21.15.70:0/2349447655 shutdown_connections 2023-10-08T07:15:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.141+0000 7f60bf59e640 1 -- 172.21.15.70:0/2349447655 >> 172.21.15.70:0/2349447655 conn(0x7f60c006c050 msgr2=0x7f60c006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.141+0000 7f60bf59e640 1 -- 172.21.15.70:0/2349447655 shutdown_connections 2023-10-08T07:15:25.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.141+0000 7f60bf59e640 1 -- 172.21.15.70:0/2349447655 wait complete. 2023-10-08T07:15:25.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.143+0000 7f60bf59e640 1 Processor -- start 2023-10-08T07:15:25.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.143+0000 7f60bf59e640 1 -- start start 2023-10-08T07:15:25.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.144+0000 7f60bf59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 0x7f60c0145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.144+0000 7f60bf59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60c0157820 0x7f60c0148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.144+0000 7f60bf59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c0149e80 0x7f60c014a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.144+0000 7f60bf59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00cdd30 con 0x7f60a8004020 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.144+0000 7f60bf59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c0148de0 con 0x7f60c0157820 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.144+0000 7f60bf59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c01490c0 con 0x7f60c0149e80 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.144+0000 7f60be59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 0x7f60c0145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.144+0000 7f60be59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 0x7f60c0145110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59150/0 (socket says 172.21.15.70:59150) 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.144+0000 7f60be59c640 1 -- 172.21.15.70:0/2249280849 learned_addr learned my addr 172.21.15.70:0/2249280849 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60bdd9b640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60c0157820 0x7f60c0148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60be59c640 1 -- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c0149e80 msgr2=0x7f60c014a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:25.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60be59c640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c0149e80 0x7f60c014a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:25.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60be59c640 1 -- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60c0157820 msgr2=0x7f60c0148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:25.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60be59c640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60c0157820 0x7f60c0148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:25.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60be59c640 1 -- 172.21.15.70:0/2249280849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c01493a0 con 0x7f60a8004020 2023-10-08T07:15:25.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60be59c640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 0x7f60c0145110 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f60b0007e60 tx=0x7f60b000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:25.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60aeffd640 1 -- 172.21.15.70:0/2249280849 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b00145f0 con 0x7f60a8004020 2023-10-08T07:15:25.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60aeffd640 1 -- 172.21.15.70:0/2249280849 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b0014750 con 0x7f60a8004020 2023-10-08T07:15:25.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60aeffd640 1 -- 172.21.15.70:0/2249280849 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0014ae0 con 0x7f60a8004020 2023-10-08T07:15:25.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.145+0000 7f60bf59e640 1 -- 172.21.15.70:0/2249280849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c0149680 con 0x7f60a8004020 2023-10-08T07:15:25.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.146+0000 7f60bf59e640 1 -- 172.21.15.70:0/2249280849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c0145650 con 0x7f60a8004020 2023-10-08T07:15:25.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.147+0000 7f60bf59e640 1 -- 172.21.15.70:0/2249280849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c0074150 con 0x7f60a8004020 2023-10-08T07:15:25.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.148+0000 7f60aeffd640 1 -- 172.21.15.70:0/2249280849 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f60b0014c40 con 0x7f60a8004020 2023-10-08T07:15:25.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.149+0000 7f60aeffd640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f60a006b020 0x7f60a006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:25.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.149+0000 7f60aeffd640 1 -- 172.21.15.70:0/2249280849 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(262..262 src has 1..262) v4 ==== 7517+0+0 (secure 0 0 0) 0x7f60b008af30 con 0x7f60a8004020 2023-10-08T07:15:25.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.150+0000 7f60bdd9b640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f60a006b020 0x7f60a006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:25.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.151+0000 7f60bdd9b640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f60a006b020 0x7f60a006d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f60b4033000 tx=0x7f60b4009030 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:25.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.154+0000 7f60aeffd640 1 -- 172.21.15.70:0/2249280849 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60b0055540 con 0x7f60a8004020 2023-10-08T07:15:25.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.347+0000 7f60bf59e640 1 -- 172.21.15.70:0/2249280849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f60c0158160 con 0x7f60a8004020 2023-10-08T07:15:25.691 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:25.689+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:25.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.904+0000 7f60aeffd640 1 -- 172.21.15.70:0/2249280849 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v263) v1 ==== 169+0+0 (secure 0 0 0) 0x7f60b008a020 con 0x7f60a8004020 2023-10-08T07:15:25.906 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T07:15:25.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.907+0000 7f60bf59e640 1 -- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f60a006b020 msgr2=0x7f60a006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:25.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.907+0000 7f60bf59e640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f60a006b020 0x7f60a006d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f60b4033000 tx=0x7f60b4009030 comp rx=0 tx=0).stop 2023-10-08T07:15:25.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.907+0000 7f60bf59e640 1 -- 172.21.15.70:0/2249280849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 msgr2=0x7f60c0145110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:25.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.907+0000 7f60bf59e640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 0x7f60c0145110 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7f60b0007e60 tx=0x7f60b000f520 comp rx=0 tx=0).stop 2023-10-08T07:15:25.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.908+0000 7f60bf59e640 1 -- 172.21.15.70:0/2249280849 shutdown_connections 2023-10-08T07:15:25.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.908+0000 7f60bf59e640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f60a006b020 0x7f60a006d4e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:25.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.908+0000 7f60bf59e640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f60c0149e80 0x7f60c014a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:25.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.908+0000 7f60bf59e640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f60c0157820 0x7f60c0148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:25.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.908+0000 7f60bf59e640 1 --2- 172.21.15.70:0/2249280849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60a8004020 0x7f60c0145110 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:25.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.908+0000 7f60bf59e640 1 -- 172.21.15.70:0/2249280849 >> 172.21.15.70:0/2249280849 conn(0x7f60c006c050 msgr2=0x7f60c00af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:25.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.908+0000 7f60bf59e640 1 -- 172.21.15.70:0/2249280849 shutdown_connections 2023-10-08T07:15:25.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:25.908+0000 7f60bf59e640 1 -- 172.21.15.70:0/2249280849 wait complete. 2023-10-08T07:15:25.942 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:26.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.138+0000 7f300fba9640 1 Processor -- start 2023-10-08T07:15:26.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.138+0000 7f300fba9640 1 -- start start 2023-10-08T07:15:26.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.140+0000 7f300fba9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f300806b520 0x7f300806ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.140+0000 7f300fba9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080b0ab0 0x7f30080b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.140+0000 7f300fba9640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30080b0340 con 0x7f30080af8b0 2023-10-08T07:15:26.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.140+0000 7f300fba9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300806b3a0 con 0x7f300806b520 2023-10-08T07:15:26.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.140+0000 7f300fba9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30080b0150 con 0x7f30080b0ab0 2023-10-08T07:15:26.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.141+0000 7f300d945640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f300806b520 0x7f300806ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:26.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.141+0000 7f300e146640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080b0ab0 0x7f30080b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:26.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.141+0000 7f300d945640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f300806b520 0x7f300806ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:35634/0 (socket says 172.21.15.70:35634) 2023-10-08T07:15:26.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.141+0000 7f300e146640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080b0ab0 0x7f30080b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:33174/0 (socket says 172.21.15.70:33174) 2023-10-08T07:15:26.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.141+0000 7f300d945640 1 -- 172.21.15.70:0/3739370091 learned_addr learned my addr 172.21.15.70:0/3739370091 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:26.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.142+0000 7f300d945640 1 -- 172.21.15.70:0/3739370091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080b0ab0 msgr2=0x7f30080b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:26.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.142+0000 7f300d945640 1 --2- 172.21.15.70:0/3739370091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080b0ab0 0x7f30080b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.142+0000 7f300d945640 1 -- 172.21.15.70:0/3739370091 >> v1:172.21.15.181:6789/0 conn(0x7f30080af8b0 legacy=0x7f300806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.142+0000 7f300d945640 1 -- 172.21.15.70:0/3739370091 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30080cf8f0 con 0x7f300806b520 2023-10-08T07:15:26.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.142+0000 7f300e146640 1 --2- 172.21.15.70:0/3739370091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080b0ab0 0x7f30080b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:15:26.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.143+0000 7f300d945640 1 --2- 172.21.15.70:0/3739370091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f300806b520 0x7f300806ae60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009ea0 tx=0x7f2ff802bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ea5e43e5fc60b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:26.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.144+0000 7f300c943640 1 -- 172.21.15.70:0/3739370091 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff802cab0 con 0x7f300806b520 2023-10-08T07:15:26.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.144+0000 7f300c943640 1 -- 172.21.15.70:0/3739370091 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff802cc10 con 0x7f300806b520 2023-10-08T07:15:26.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.144+0000 7f300c943640 1 -- 172.21.15.70:0/3739370091 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff80343f0 con 0x7f300806b520 2023-10-08T07:15:26.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.145+0000 7f300fba9640 1 -- 172.21.15.70:0/3739370091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f300806b520 msgr2=0x7f300806ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.145+0000 7f300fba9640 1 --2- 172.21.15.70:0/3739370091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f300806b520 0x7f300806ae60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009ea0 tx=0x7f2ff802bf10 comp rx=0 tx=0).stop 2023-10-08T07:15:26.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.145+0000 7f300fba9640 1 -- 172.21.15.70:0/3739370091 shutdown_connections 2023-10-08T07:15:26.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.145+0000 7f300fba9640 1 --2- 172.21.15.70:0/3739370091 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080b0ab0 0x7f30080b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.145+0000 7f300fba9640 1 --2- 172.21.15.70:0/3739370091 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f300806b520 0x7f300806ae60 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.145+0000 7f300fba9640 1 -- 172.21.15.70:0/3739370091 >> 172.21.15.70:0/3739370091 conn(0x7f30080d09e0 msgr2=0x7f30080d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:26.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.145+0000 7f300fba9640 1 -- 172.21.15.70:0/3739370091 shutdown_connections 2023-10-08T07:15:26.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.145+0000 7f300fba9640 1 -- 172.21.15.70:0/3739370091 wait complete. 2023-10-08T07:15:26.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.147+0000 7f300fba9640 1 Processor -- start 2023-10-08T07:15:26.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.147+0000 7f300fba9640 1 -- start start 2023-10-08T07:15:26.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.148+0000 7f300fba9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300806b520 0x7f300814a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.148+0000 7f300fba9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080af8b0 0x7f3008146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.148+0000 7f300fba9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30080b0ab0 0x7f300806f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300fba9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300814dcb0 con 0x7f300806b520 2023-10-08T07:15:26.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300fba9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300814a810 con 0x7f30080b0ab0 2023-10-08T07:15:26.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300fba9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30080b4ba0 con 0x7f30080af8b0 2023-10-08T07:15:26.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300d945640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300806b520 0x7f300814a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:26.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300d945640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300806b520 0x7f300814a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59152/0 (socket says 172.21.15.70:59152) 2023-10-08T07:15:26.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300d945640 1 -- 172.21.15.70:0/2945667121 learned_addr learned my addr 172.21.15.70:0/2945667121 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:26.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300d945640 1 -- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080af8b0 msgr2=0x7f3008146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300d945640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080af8b0 0x7f3008146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300d945640 1 -- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30080b0ab0 msgr2=0x7f300806f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300d945640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30080b0ab0 0x7f300806f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.149+0000 7f300d945640 1 -- 172.21.15.70:0/2945667121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3000009400 con 0x7f300806b520 2023-10-08T07:15:26.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.150+0000 7f300d945640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300806b520 0x7f300814a2d0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8000f80 tx=0x7f2ff8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:26.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.150+0000 7f30067fc640 1 -- 172.21.15.70:0/2945667121 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff803e450 con 0x7f300806b520 2023-10-08T07:15:26.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.150+0000 7f30067fc640 1 -- 172.21.15.70:0/2945667121 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff803e5f0 con 0x7f300806b520 2023-10-08T07:15:26.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.150+0000 7f30067fc640 1 -- 172.21.15.70:0/2945667121 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff803e9a0 con 0x7f300806b520 2023-10-08T07:15:26.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.150+0000 7f300fba9640 1 -- 172.21.15.70:0/2945667121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff8009b30 con 0x7f300806b520 2023-10-08T07:15:26.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.150+0000 7f300fba9640 1 -- 172.21.15.70:0/2945667121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30080b5160 con 0x7f300806b520 2023-10-08T07:15:26.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.151+0000 7f300fba9640 1 -- 172.21.15.70:0/2945667121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3008067490 con 0x7f300806b520 2023-10-08T07:15:26.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.152+0000 7f30067fc640 1 -- 172.21.15.70:0/2945667121 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f2ff803eb00 con 0x7f300806b520 2023-10-08T07:15:26.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.153+0000 7f30067fc640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2ff406b020 0x7f2ff406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.153+0000 7f30067fc640 1 -- 172.21.15.70:0/2945667121 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f2ff80acb70 con 0x7f300806b520 2023-10-08T07:15:26.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.154+0000 7f300d144640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2ff406b020 0x7f2ff406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:26.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.155+0000 7f300d144640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2ff406b020 0x7f2ff406d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc000f60 tx=0x7f2ffc0079d0 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:26.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.158+0000 7f30067fc640 1 -- 172.21.15.70:0/2945667121 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ff8077550 con 0x7f300806b520 2023-10-08T07:15:26.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.368+0000 7f300fba9640 1 -- 172.21.15.70:0/2945667121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f300808aae0 con 0x7f300806b520 2023-10-08T07:15:26.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.371+0000 7f30067fc640 1 -- 172.21.15.70:0/2945667121 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v263) v1 ==== 74+0+18300 (secure 0 0 0) 0x7f2ff803edb0 con 0x7f300806b520 2023-10-08T07:15:26.373 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:26.374 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":263,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:25.894354+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:14:56.178783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:14:56.977158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6832/3856326680":"2023-10-09T07:15:22.506757+0000","172.21.15.181:0/888851152":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3676453840":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3956317048":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1513601554":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6832/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/348399880":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1607590530":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/2690144496":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/3464560328":"2023-10-09T07:15:10.723067+0000","172.21.15.70:6834/3856326680":"2023-10-09T07:15:22.506757+0000","172.21.15.181:6832/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/807583929":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/859617464":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/4271328810":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/735805106":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/957356199":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2539797528":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/333977458":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2403193161":"2023-10-09T07:15:10.723067+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:26.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 -- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2ff406b020 msgr2=0x7f2ff406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2ff406b020 0x7f2ff406d4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc000f60 tx=0x7f2ffc0079d0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 -- 172.21.15.70:0/2945667121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300806b520 msgr2=0x7f300814a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300806b520 0x7f300814a2d0 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8000f80 tx=0x7f2ff8004a40 comp rx=0 tx=0).stop 2023-10-08T07:15:26.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 -- 172.21.15.70:0/2945667121 shutdown_connections 2023-10-08T07:15:26.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2ff406b020 0x7f2ff406d4e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f30080b0ab0 0x7f300806f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080af8b0 0x7f3008146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 --2- 172.21.15.70:0/2945667121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f300806b520 0x7f300814a2d0 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 -- 172.21.15.70:0/2945667121 >> 172.21.15.70:0/2945667121 conn(0x7f30080d09e0 msgr2=0x7f30080b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:26.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 -- 172.21.15.70:0/2945667121 shutdown_connections 2023-10-08T07:15:26.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.374+0000 7f300fba9640 1 -- 172.21.15.70:0/2945667121 wait complete. 2023-10-08T07:15:26.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-10-08T07:14:56.178783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '257', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 34, 'pool_name': 'backup_fs_data', 'create_time': '2023-10-08T07:14:56.977158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '258', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:15:26.395 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:15:26.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.605+0000 7f7b66154640 1 Processor -- start 2023-10-08T07:15:26.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.606+0000 7f7b66154640 1 -- start start 2023-10-08T07:15:26.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.607+0000 7f7b66154640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600cfb10 0x7f7b600cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.607+0000 7f7b66154640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b60071f50 con 0x7f7b600b27d0 2023-10-08T07:15:26.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.607+0000 7f7b66154640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b600713d0 con 0x7f7b6015a970 2023-10-08T07:15:26.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.607+0000 7f7b66154640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600d0500 con 0x7f7b600cfb10 2023-10-08T07:15:26.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.607+0000 7f7b65953640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7b600b27d0 0x7f7b600b2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50268/0 (socket says 172.21.15.70:50268) 2023-10-08T07:15:26.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b65953640 1 -- 172.21.15.70:0/2960259638 learned_addr learned my addr 172.21.15.70:0/2960259638 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:26.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b64951640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600cfb10 0x7f7b600cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:26.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909298221 0 0) 0x7f7b60071f50 con 0x7f7b600b27d0 2023-10-08T07:15:26.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b48003520 con 0x7f7b600b27d0 2023-10-08T07:15:26.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3220977825 0 0) 0x7f7b48003520 con 0x7f7b600b27d0 2023-10-08T07:15:26.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600cfb10 msgr2=0x7f7b600cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:26.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b5ffff640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600cfb10 0x7f7b600cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 >> v1:172.21.15.181:6789/0 conn(0x7f7b6015a970 legacy=0x7f7b600b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b600716b0 con 0x7f7b600b27d0 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f7b58001670 con 0x7f7b600b27d0 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 >> v1:172.21.15.70:6789/0 conn(0x7f7b600b27d0 legacy=0x7f7b600b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.608+0000 7f7b5ffff640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b480039a0 0x7f7b48003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b5ffff640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 0x7f7b48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b5ffff640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600cfb10 0x7f7b48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600d0500 con 0x7f7b480047c0 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4800ba90 con 0x7f7b480039a0 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4800bc50 con 0x7f7b600cfb10 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b64951640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 0x7f7b48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:26.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b65953640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600cfb10 0x7f7b48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:26.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b64951640 1 -- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600cfb10 msgr2=0x7f7b48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:26.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b64951640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600cfb10 0x7f7b48005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b65152640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b480039a0 0x7f7b48003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:26.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b64951640 1 -- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b480039a0 msgr2=0x7f7b48003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:26.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b64951640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b480039a0 0x7f7b48003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.609+0000 7f7b64951640 1 -- 172.21.15.70:0/2960259638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b48003520 con 0x7f7b480047c0 2023-10-08T07:15:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.611+0000 7f7b64951640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 0x7f7b48004c40 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009a90 tx=0x7f7b480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea12c7cb45ae63ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.611+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5402c680 con 0x7f7b480047c0 2023-10-08T07:15:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.611+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5402c7e0 con 0x7f7b480047c0 2023-10-08T07:15:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.611+0000 7f7b5ffff640 1 -- 172.21.15.70:0/2960259638 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5402cb70 con 0x7f7b480047c0 2023-10-08T07:15:26.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.611+0000 7f7b66154640 1 -- 172.21.15.70:0/2960259638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 msgr2=0x7f7b48004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.611+0000 7f7b66154640 1 --2- 172.21.15.70:0/2960259638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 0x7f7b48004c40 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009a90 tx=0x7f7b480057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.611+0000 7f7b65953640 1 -- 172.21.15.70:0/2960259638 reap_dead start 2023-10-08T07:15:26.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.612+0000 7f7b66154640 1 -- 172.21.15.70:0/2960259638 shutdown_connections 2023-10-08T07:15:26.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.612+0000 7f7b66154640 1 -- 172.21.15.70:0/2960259638 >> 172.21.15.70:0/2960259638 conn(0x7f7b6006b250 msgr2=0x7f7b6006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:26.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.612+0000 7f7b66154640 1 -- 172.21.15.70:0/2960259638 shutdown_connections 2023-10-08T07:15:26.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.612+0000 7f7b66154640 1 -- 172.21.15.70:0/2960259638 wait complete. 2023-10-08T07:15:26.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.614+0000 7f7b66154640 1 Processor -- start 2023-10-08T07:15:26.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.614+0000 7f7b66154640 1 -- start start 2023-10-08T07:15:26.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b66154640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 0x7f7b601f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b66154640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b600c5080 0x7f7b6015d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b66154640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600ac910 0x7f7b600b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b66154640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600af0b0 con 0x7f7b480047c0 2023-10-08T07:15:26.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b66154640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600afc30 con 0x7f7b600c5080 2023-10-08T07:15:26.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b66154640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600af300 con 0x7f7b600ac910 2023-10-08T07:15:26.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b65152640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 0x7f7b601f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b65152640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 0x7f7b601f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59172/0 (socket says 172.21.15.70:59172) 2023-10-08T07:15:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b65152640 1 -- 172.21.15.70:0/2615340012 learned_addr learned my addr 172.21.15.70:0/2615340012 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b65152640 1 -- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600ac910 msgr2=0x7f7b600b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b65152640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600ac910 0x7f7b600b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b65152640 1 -- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b600c5080 msgr2=0x7f7b6015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b65152640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b600c5080 0x7f7b6015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.615+0000 7f7b65152640 1 -- 172.21.15.70:0/2615340012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b600af5e0 con 0x7f7b480047c0 2023-10-08T07:15:26.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.616+0000 7f7b65152640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 0x7f7b601f2b60 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000d860 tx=0x7f7b5000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.616+0000 7f7b5dffb640 1 -- 172.21.15.70:0/2615340012 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50003da0 con 0x7f7b480047c0 2023-10-08T07:15:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.616+0000 7f7b5dffb640 1 -- 172.21.15.70:0/2615340012 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5000b6b0 con 0x7f7b480047c0 2023-10-08T07:15:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.616+0000 7f7b66154640 1 -- 172.21.15.70:0/2615340012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b600af8c0 con 0x7f7b480047c0 2023-10-08T07:15:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.616+0000 7f7b5dffb640 1 -- 172.21.15.70:0/2615340012 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5000ba40 con 0x7f7b480047c0 2023-10-08T07:15:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.616+0000 7f7b66154640 1 -- 172.21.15.70:0/2615340012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6015dbb0 con 0x7f7b480047c0 2023-10-08T07:15:26.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.617+0000 7f7b66154640 1 -- 172.21.15.70:0/2615340012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b60074150 con 0x7f7b480047c0 2023-10-08T07:15:26.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.618+0000 7f7b5dffb640 1 -- 172.21.15.70:0/2615340012 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f7b5000bba0 con 0x7f7b480047c0 2023-10-08T07:15:26.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.619+0000 7f7b5dffb640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7b4c06b050 0x7f7b4c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:26.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.619+0000 7f7b5dffb640 1 -- 172.21.15.70:0/2615340012 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f7b5008a2a0 con 0x7f7b480047c0 2023-10-08T07:15:26.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.620+0000 7f7b64951640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7b4c06b050 0x7f7b4c06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:26.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.621+0000 7f7b64951640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7b4c06b050 0x7f7b4c06d510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b54036000 tx=0x7f7b5402e000 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:26.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.624+0000 7f7b5dffb640 1 -- 172.21.15.70:0/2615340012 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b500549b0 con 0x7f7b480047c0 2023-10-08T07:15:26.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.816+0000 7f7b66154640 1 -- 172.21.15.70:0/2615340012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b600b4c40 con 0x7f7b480047c0 2023-10-08T07:15:26.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.817+0000 7f7b5dffb640 1 -- 172.21.15.70:0/2615340012 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v263) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b50057fd0 con 0x7f7b480047c0 2023-10-08T07:15:26.819 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:15:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 -- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7b4c06b050 msgr2=0x7f7b4c06d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7b4c06b050 0x7f7b4c06d510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7b54036000 tx=0x7f7b5402e000 comp rx=0 tx=0).stop 2023-10-08T07:15:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 -- 172.21.15.70:0/2615340012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 msgr2=0x7f7b601f2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 0x7f7b601f2b60 secure :-1 s=READY pgs=2797 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000d860 tx=0x7f7b5000dd20 comp rx=0 tx=0).stop 2023-10-08T07:15:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 -- 172.21.15.70:0/2615340012 shutdown_connections 2023-10-08T07:15:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7b4c06b050 0x7f7b4c06d510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7b600ac910 0x7f7b600b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7b600c5080 0x7f7b6015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 --2- 172.21.15.70:0/2615340012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b480047c0 0x7f7b601f2b60 unknown :-1 s=CLOSED pgs=2797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:26.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 -- 172.21.15.70:0/2615340012 >> 172.21.15.70:0/2615340012 conn(0x7f7b6006b250 msgr2=0x7f7b600b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:26.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 -- 172.21.15.70:0/2615340012 shutdown_connections 2023-10-08T07:15:26.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:26.820+0000 7f7b66154640 1 -- 172.21.15.70:0/2615340012 wait complete. 2023-10-08T07:15:26.837 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:26.835+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:26.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-10-08T07:15:27.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.065+0000 7f33b0f93640 1 Processor -- start 2023-10-08T07:15:27.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.065+0000 7f33b0f93640 1 -- start start 2023-10-08T07:15:27.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.067+0000 7f33b0f93640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33ac15ccb0 0x7f33ac0d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.067+0000 7f33b0f93640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 0x7f33ac0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.067+0000 7f33b0f93640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ac0d2870 con 0x7f33ac0cff10 2023-10-08T07:15:27.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.067+0000 7f33b0f93640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ac0d0380 con 0x7f33ac15ccb0 2023-10-08T07:15:27.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.067+0000 7f33b0f93640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0d1150 con 0x7f33ac0d0b90 2023-10-08T07:15:27.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.068+0000 7f33ab7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33ac15ccb0 0x7f33ac0d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.068+0000 7f33a2ffd640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 0x7f33ac0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.068+0000 7f33ab7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33ac15ccb0 0x7f33ac0d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:35670/0 (socket says 172.21.15.70:35670) 2023-10-08T07:15:27.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.068+0000 7f33ab7fe640 1 -- 172.21.15.70:0/3001581437 learned_addr learned my addr 172.21.15.70:0/3001581437 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:27.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.068+0000 7f33a2ffd640 1 -- 172.21.15.70:0/3001581437 >> v2:172.21.15.181:3300/0 conn(0x7f33ac15ccb0 msgr2=0x7f33ac0d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:27.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.068+0000 7f33a2ffd640 1 --2- 172.21.15.70:0/3001581437 >> v2:172.21.15.181:3300/0 conn(0x7f33ac15ccb0 0x7f33ac0d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.068+0000 7f33a2ffd640 1 -- 172.21.15.70:0/3001581437 >> v1:172.21.15.181:6789/0 conn(0x7f33ac0cff10 legacy=0x7f33ac0d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.068+0000 7f33a2ffd640 1 -- 172.21.15.70:0/3001581437 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac0d1430 con 0x7f33ac0d0b90 2023-10-08T07:15:27.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.068+0000 7f33ab7fe640 1 --2- 172.21.15.70:0/3001581437 >> v2:172.21.15.181:3300/0 conn(0x7f33ac15ccb0 0x7f33ac0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:15:27.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.070+0000 7f33a2ffd640 1 --2- 172.21.15.70:0/3001581437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 0x7f33ac0cf9d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f33940097a0 tx=0x7f339402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cae1e9f240240be server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:27.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.070+0000 7f33aaffd640 1 -- 172.21.15.70:0/3001581437 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339402d040 con 0x7f33ac0d0b90 2023-10-08T07:15:27.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.070+0000 7f33aaffd640 1 -- 172.21.15.70:0/3001581437 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3394031810 con 0x7f33ac0d0b90 2023-10-08T07:15:27.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.070+0000 7f33aaffd640 1 -- 172.21.15.70:0/3001581437 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394031ba0 con 0x7f33ac0d0b90 2023-10-08T07:15:27.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.071+0000 7f33b0f93640 1 -- 172.21.15.70:0/3001581437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 msgr2=0x7f33ac0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.071+0000 7f33b0f93640 1 --2- 172.21.15.70:0/3001581437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 0x7f33ac0cf9d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f33940097a0 tx=0x7f339402b8b0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.071+0000 7f33b0f93640 1 -- 172.21.15.70:0/3001581437 shutdown_connections 2023-10-08T07:15:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.071+0000 7f33b0f93640 1 --2- 172.21.15.70:0/3001581437 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 0x7f33ac0cf9d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.071+0000 7f33b0f93640 1 --2- 172.21.15.70:0/3001581437 >> v2:172.21.15.181:3300/0 conn(0x7f33ac15ccb0 0x7f33ac0d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.071+0000 7f33b0f93640 1 -- 172.21.15.70:0/3001581437 >> 172.21.15.70:0/3001581437 conn(0x7f33ac071660 msgr2=0x7f33ac071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:27.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.071+0000 7f33b0f93640 1 -- 172.21.15.70:0/3001581437 shutdown_connections 2023-10-08T07:15:27.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.071+0000 7f33b0f93640 1 -- 172.21.15.70:0/3001581437 wait complete. 2023-10-08T07:15:27.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.073+0000 7f33b0f93640 1 Processor -- start 2023-10-08T07:15:27.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.073+0000 7f33b0f93640 1 -- start start 2023-10-08T07:15:27.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33b0f93640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33ac0cff10 0x7f33ac1f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33b0f93640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 0x7f33ac150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33b0f93640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33ac15ccb0 0x7f33ac0cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33b0f93640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac1f3210 con 0x7f33ac15ccb0 2023-10-08T07:15:27.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33b0f93640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac15d580 con 0x7f33ac0cff10 2023-10-08T07:15:27.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33b0f93640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac15d860 con 0x7f33ac0d0b90 2023-10-08T07:15:27.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33ac15ccb0 0x7f33ac0cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33ac15ccb0 0x7f33ac0cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59180/0 (socket says 172.21.15.70:59180) 2023-10-08T07:15:27.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33abfff640 1 -- 172.21.15.70:0/2669241882 learned_addr learned my addr 172.21.15.70:0/2669241882 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:27.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33ab7fe640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33ac0cff10 0x7f33ac1f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.074+0000 7f33a2ffd640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 0x7f33ac150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33abfff640 1 -- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 msgr2=0x7f33ac150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33abfff640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 0x7f33ac150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33abfff640 1 -- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33ac0cff10 msgr2=0x7f33ac1f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33abfff640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33ac0cff10 0x7f33ac1f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33abfff640 1 -- 172.21.15.70:0/2669241882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c0094d0 con 0x7f33ac15ccb0 2023-10-08T07:15:27.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33abfff640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33ac15ccb0 0x7f33ac0cbf30 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c680 tx=0x7f33a400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:27.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33a8ff9640 1 -- 172.21.15.70:0/2669241882 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4003b90 con 0x7f33ac15ccb0 2023-10-08T07:15:27.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33a8ff9640 1 -- 172.21.15.70:0/2669241882 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4003cf0 con 0x7f33ac15ccb0 2023-10-08T07:15:27.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33a8ff9640 1 -- 172.21.15.70:0/2669241882 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4005310 con 0x7f33ac15ccb0 2023-10-08T07:15:27.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33b0f93640 1 -- 172.21.15.70:0/2669241882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3394009400 con 0x7f33ac15ccb0 2023-10-08T07:15:27.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.075+0000 7f33b0f93640 1 -- 172.21.15.70:0/2669241882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac15de20 con 0x7f33ac15ccb0 2023-10-08T07:15:27.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.076+0000 7f33b0f93640 1 -- 172.21.15.70:0/2669241882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac08a3f0 con 0x7f33ac15ccb0 2023-10-08T07:15:27.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.078+0000 7f33a8ff9640 1 -- 172.21.15.70:0/2669241882 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f33a4005470 con 0x7f33ac15ccb0 2023-10-08T07:15:27.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.079+0000 7f33a8ff9640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f338c06b020 0x7f338c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.079+0000 7f33a8ff9640 1 -- 172.21.15.70:0/2669241882 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f33a4089be0 con 0x7f33ac15ccb0 2023-10-08T07:15:27.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.080+0000 7f33ab7fe640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f338c06b020 0x7f338c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.081+0000 7f33ab7fe640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f338c06b020 0x7f338c06d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f339c009c50 tx=0x7f339c008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:27.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.083+0000 7f33a8ff9640 1 -- 172.21.15.70:0/2669241882 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33a40544a0 con 0x7f33ac15ccb0 2023-10-08T07:15:27.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.281+0000 7f33b0f93640 1 -- 172.21.15.70:0/2669241882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33ac0d16e0 con 0x7f33ac15ccb0 2023-10-08T07:15:27.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.282+0000 7f33a8ff9640 1 -- 172.21.15.70:0/2669241882 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v263) v1 ==== 107+0+11 (secure 0 0 0) 0x7f33a4057ac0 con 0x7f33ac15ccb0 2023-10-08T07:15:27.283 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 -- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f338c06b020 msgr2=0x7f338c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f338c06b020 0x7f338c06d4e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f339c009c50 tx=0x7f339c008040 comp rx=0 tx=0).stop 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 -- 172.21.15.70:0/2669241882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33ac15ccb0 msgr2=0x7f33ac0cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33ac15ccb0 0x7f33ac0cbf30 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c680 tx=0x7f33a400cb40 comp rx=0 tx=0).stop 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 -- 172.21.15.70:0/2669241882 shutdown_connections 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f338c06b020 0x7f338c06d4e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33ac15ccb0 0x7f33ac0cbf30 unknown :-1 s=CLOSED pgs=2798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33ac0d0b90 0x7f33ac150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 --2- 172.21.15.70:0/2669241882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33ac0cff10 0x7f33ac1f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 -- 172.21.15.70:0/2669241882 >> 172.21.15.70:0/2669241882 conn(0x7f33ac071660 msgr2=0x7f33ac071a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:27.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 -- 172.21.15.70:0/2669241882 shutdown_connections 2023-10-08T07:15:27.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.285+0000 7f33b0f93640 1 -- 172.21.15.70:0/2669241882 wait complete. 2023-10-08T07:15:27.305 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-10-08T07:15:27.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.513+0000 7fd01a656640 1 Processor -- start 2023-10-08T07:15:27.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.514+0000 7fd01a656640 1 -- start start 2023-10-08T07:15:27.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.515+0000 7fd01a656640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd014070cc0 0x7fd0140caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.515+0000 7fd01a656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0140cb450 0x7fd0140b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.515+0000 7fd01a656640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd014071170 con 0x7fd014160660 2023-10-08T07:15:27.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.515+0000 7fd01a656640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01406e150 con 0x7fd014070cc0 2023-10-08T07:15:27.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.515+0000 7fd01a656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0140cb8c0 con 0x7fd0140cb450 2023-10-08T07:15:27.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.516+0000 7fd00bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd014070cc0 0x7fd0140caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.516+0000 7fd00bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd014070cc0 0x7fd0140caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59182/0 (socket says 172.21.15.70:59182) 2023-10-08T07:15:27.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.516+0000 7fd00bfff640 1 -- 172.21.15.70:0/883720447 learned_addr learned my addr 172.21.15.70:0/883720447 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:27.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.516+0000 7fd018bf3640 1 --2- 172.21.15.70:0/883720447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0140cb450 0x7fd0140b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.516+0000 7fd00bfff640 1 -- 172.21.15.70:0/883720447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0140cb450 msgr2=0x7fd0140b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:27.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.516+0000 7fd00bfff640 1 --2- 172.21.15.70:0/883720447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0140cb450 0x7fd0140b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.516+0000 7fd00bfff640 1 -- 172.21.15.70:0/883720447 >> v1:172.21.15.181:6789/0 conn(0x7fd014160660 legacy=0x7fd0140c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:27.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.516+0000 7fd00bfff640 1 -- 172.21.15.70:0/883720447 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01406de70 con 0x7fd014070cc0 2023-10-08T07:15:27.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.518+0000 7fd00bfff640 1 --2- 172.21.15.70:0/883720447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014070cc0 0x7fd0140caf10 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009870 tx=0x7fcffc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a598aa28fd3e936 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:27.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.518+0000 7fd0137fe640 1 -- 172.21.15.70:0/883720447 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc031640 con 0x7fd014070cc0 2023-10-08T07:15:27.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.518+0000 7fd0137fe640 1 -- 172.21.15.70:0/883720447 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcffc0317a0 con 0x7fd014070cc0 2023-10-08T07:15:27.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.518+0000 7fd0137fe640 1 -- 172.21.15.70:0/883720447 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcffc031b30 con 0x7fd014070cc0 2023-10-08T07:15:27.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.519+0000 7fd01a656640 1 -- 172.21.15.70:0/883720447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014070cc0 msgr2=0x7fd0140caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.519+0000 7fd01a656640 1 --2- 172.21.15.70:0/883720447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014070cc0 0x7fd0140caf10 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009870 tx=0x7fcffc02b8f0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.519+0000 7fd01a656640 1 -- 172.21.15.70:0/883720447 shutdown_connections 2023-10-08T07:15:27.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.519+0000 7fd01a656640 1 --2- 172.21.15.70:0/883720447 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd0140cb450 0x7fd0140b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.519+0000 7fd01a656640 1 --2- 172.21.15.70:0/883720447 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014070cc0 0x7fd0140caf10 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.519+0000 7fd01a656640 1 -- 172.21.15.70:0/883720447 >> 172.21.15.70:0/883720447 conn(0x7fd01406a5d0 msgr2=0x7fd0140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:27.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.519+0000 7fd01a656640 1 -- 172.21.15.70:0/883720447 shutdown_connections 2023-10-08T07:15:27.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.519+0000 7fd01a656640 1 -- 172.21.15.70:0/883720447 wait complete. 2023-10-08T07:15:27.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.521+0000 7fd01a656640 1 Processor -- start 2023-10-08T07:15:27.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.521+0000 7fd01a656640 1 -- start start 2023-10-08T07:15:27.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.522+0000 7fd01a656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd014070cc0 0x7fd0140c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.522+0000 7fd01a656640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0140cb450 0x7fd014154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.522+0000 7fd01a656640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014160660 0x7fd0140c3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.522+0000 7fd01a656640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0140c3190 con 0x7fd014160660 2023-10-08T07:15:27.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.522+0000 7fd01a656640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0140c2580 con 0x7fd0140cb450 2023-10-08T07:15:27.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.522+0000 7fd01a656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0140c2860 con 0x7fd014070cc0 2023-10-08T07:15:27.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.522+0000 7fd018bf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014160660 0x7fd0140c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.522+0000 7fd018bf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014160660 0x7fd0140c3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59184/0 (socket says 172.21.15.70:59184) 2023-10-08T07:15:27.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.522+0000 7fd018bf3640 1 -- 172.21.15.70:0/3381348610 learned_addr learned my addr 172.21.15.70:0/3381348610 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:27.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd013fff640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd014070cc0 0x7fd0140c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd00bfff640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0140cb450 0x7fd014154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd018bf3640 1 -- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd014070cc0 msgr2=0x7fd0140c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd018bf3640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd014070cc0 0x7fd0140c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd018bf3640 1 -- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0140cb450 msgr2=0x7fd014154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd018bf3640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0140cb450 0x7fd014154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd018bf3640 1 -- 172.21.15.70:0/3381348610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc0094d0 con 0x7fd014160660 2023-10-08T07:15:27.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd00bfff640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0140cb450 0x7fd014154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:27.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd018bf3640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014160660 0x7fd0140c3620 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00cd70 tx=0x7fd00c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:27.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd0117fa640 1 -- 172.21.15.70:0/3381348610 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c003b90 con 0x7fd014160660 2023-10-08T07:15:27.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd0117fa640 1 -- 172.21.15.70:0/3381348610 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c003cf0 con 0x7fd014160660 2023-10-08T07:15:27.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.523+0000 7fd0117fa640 1 -- 172.21.15.70:0/3381348610 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c004240 con 0x7fd014160660 2023-10-08T07:15:27.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.524+0000 7fd01a656640 1 -- 172.21.15.70:0/3381348610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0140c2b40 con 0x7fd014160660 2023-10-08T07:15:27.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.524+0000 7fd01a656640 1 -- 172.21.15.70:0/3381348610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0140c5290 con 0x7fd014160660 2023-10-08T07:15:27.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.525+0000 7fd01a656640 1 -- 172.21.15.70:0/3381348610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0140c3300 con 0x7fd014160660 2023-10-08T07:15:27.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.526+0000 7fd0117fa640 1 -- 172.21.15.70:0/3381348610 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fd00c0043a0 con 0x7fd014160660 2023-10-08T07:15:27.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.527+0000 7fd0117fa640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcff406b020 0x7fcff406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.527+0000 7fd0117fa640 1 -- 172.21.15.70:0/3381348610 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fd00c012070 con 0x7fd014160660 2023-10-08T07:15:27.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.527+0000 7fd013fff640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcff406b020 0x7fcff406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.528+0000 7fd013fff640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcff406b020 0x7fcff406d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd004004ea0 tx=0x7fd004009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:27.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.531+0000 7fd0117fa640 1 -- 172.21.15.70:0/3381348610 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd00c055190 con 0x7fd014160660 2023-10-08T07:15:27.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.721+0000 7fd01a656640 1 -- 172.21.15.70:0/3381348610 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fd01404e160 con 0x7fd014160660 2023-10-08T07:15:27.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.722+0000 7fd0117fa640 1 -- 172.21.15.70:0/3381348610 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v263) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd00c0587b0 con 0x7fd014160660 2023-10-08T07:15:27.724 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:15:27.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 -- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcff406b020 msgr2=0x7fcff406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcff406b020 0x7fcff406d4e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd004004ea0 tx=0x7fd004009040 comp rx=0 tx=0).stop 2023-10-08T07:15:27.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 -- 172.21.15.70:0/3381348610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014160660 msgr2=0x7fd0140c3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014160660 0x7fd0140c3620 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7fd00c00cd70 tx=0x7fd00c00a660 comp rx=0 tx=0).stop 2023-10-08T07:15:27.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 -- 172.21.15.70:0/3381348610 shutdown_connections 2023-10-08T07:15:27.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcff406b020 0x7fcff406d4e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd014160660 0x7fd0140c3620 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd0140cb450 0x7fd014154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 --2- 172.21.15.70:0/3381348610 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd014070cc0 0x7fd0140c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 -- 172.21.15.70:0/3381348610 >> 172.21.15.70:0/3381348610 conn(0x7fd01406a5d0 msgr2=0x7fd01415af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 -- 172.21.15.70:0/3381348610 shutdown_connections 2023-10-08T07:15:27.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.725+0000 7fd01a656640 1 -- 172.21.15.70:0/3381348610 wait complete. 2023-10-08T07:15:27.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:27.948 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:27.946+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:27.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.959+0000 7f769a452640 1 Processor -- start 2023-10-08T07:15:27.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.959+0000 7f769a452640 1 -- start start 2023-10-08T07:15:27.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.961+0000 7f769a452640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f769415ab40 0x7f76940ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.961+0000 7f769a452640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f76940cb580 0x7f76940adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.961+0000 7f769a452640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769415b2b0 con 0x7f76940c90b0 2023-10-08T07:15:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.961+0000 7f769a452640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769415bf30 con 0x7f76940cb580 2023-10-08T07:15:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.961+0000 7f769a452640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769415f7a0 con 0x7f769415ab40 2023-10-08T07:15:27.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.961+0000 7f76989ef640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f76940cb580 0x7f76940adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.962+0000 7f7693fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f769415ab40 0x7f76940ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.962+0000 7f76989ef640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f76940cb580 0x7f76940adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:35702/0 (socket says 172.21.15.70:35702) 2023-10-08T07:15:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.962+0000 7f76989ef640 1 -- 172.21.15.70:0/968152853 learned_addr learned my addr 172.21.15.70:0/968152853 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.962+0000 7f76989ef640 1 -- 172.21.15.70:0/968152853 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f769415ab40 msgr2=0x7f76940ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:27.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.962+0000 7f76989ef640 1 --2- 172.21.15.70:0/968152853 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f769415ab40 0x7f76940ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.962+0000 7f76989ef640 1 -- 172.21.15.70:0/968152853 >> v1:172.21.15.181:6789/0 conn(0x7f76940c90b0 legacy=0x7f76940cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.962+0000 7f76989ef640 1 -- 172.21.15.70:0/968152853 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769415b590 con 0x7f76940cb580 2023-10-08T07:15:27.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.964+0000 7f76989ef640 1 --2- 172.21.15.70:0/968152853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76940cb580 0x7f76940adc60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f768c009870 tx=0x7f768c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f333020766d4f05 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:27.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f76937fe640 1 -- 172.21.15.70:0/968152853 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c030840 con 0x7f76940cb580 2023-10-08T07:15:27.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f76937fe640 1 -- 172.21.15.70:0/968152853 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c0309a0 con 0x7f76940cb580 2023-10-08T07:15:27.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f76937fe640 1 -- 172.21.15.70:0/968152853 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c030d30 con 0x7f76940cb580 2023-10-08T07:15:27.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f769a452640 1 -- 172.21.15.70:0/968152853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76940cb580 msgr2=0x7f76940adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f769a452640 1 --2- 172.21.15.70:0/968152853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76940cb580 0x7f76940adc60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f768c009870 tx=0x7f768c02b980 comp rx=0 tx=0).stop 2023-10-08T07:15:27.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f769a452640 1 -- 172.21.15.70:0/968152853 shutdown_connections 2023-10-08T07:15:27.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f769a452640 1 --2- 172.21.15.70:0/968152853 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76940cb580 0x7f76940adc60 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f769a452640 1 --2- 172.21.15.70:0/968152853 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f769415ab40 0x7f76940ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f769a452640 1 -- 172.21.15.70:0/968152853 >> 172.21.15.70:0/968152853 conn(0x7f769406a380 msgr2=0x7f76940b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:27.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f769a452640 1 -- 172.21.15.70:0/968152853 shutdown_connections 2023-10-08T07:15:27.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.965+0000 7f769a452640 1 -- 172.21.15.70:0/968152853 wait complete. 2023-10-08T07:15:27.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.968+0000 7f769a452640 1 Processor -- start 2023-10-08T07:15:27.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.968+0000 7f769a452640 1 -- start start 2023-10-08T07:15:27.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f769a452640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76940c90b0 0x7f76941f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f769a452640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76940cb580 0x7f769414e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f769a452640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f769415ab40 0x7f7694158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f769a452640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76941f3790 con 0x7f769415ab40 2023-10-08T07:15:27.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f769a452640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76940c36c0 con 0x7f76940cb580 2023-10-08T07:15:27.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f769a452640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76940c39a0 con 0x7f76940c90b0 2023-10-08T07:15:27.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f76989ef640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f769415ab40 0x7f7694158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f76989ef640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f769415ab40 0x7f7694158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59186/0 (socket says 172.21.15.70:59186) 2023-10-08T07:15:27.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f76989ef640 1 -- 172.21.15.70:0/3537075291 learned_addr learned my addr 172.21.15.70:0/3537075291 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:27.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f7693fff640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76940c90b0 0x7f76941f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.969+0000 7f768b7fe640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76940cb580 0x7f769414e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f76989ef640 1 -- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76940c90b0 msgr2=0x7f76941f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f76989ef640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76940c90b0 0x7f76941f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f76989ef640 1 -- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76940cb580 msgr2=0x7f769414e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:27.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f76989ef640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76940cb580 0x7f769414e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:27.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f76989ef640 1 -- 172.21.15.70:0/3537075291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0094d0 con 0x7f769415ab40 2023-10-08T07:15:27.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f768b7fe640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76940cb580 0x7f769414e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:27.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f76989ef640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f769415ab40 0x7f7694158130 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f768c009870 tx=0x7f768c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:27.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f76917fa640 1 -- 172.21.15.70:0/3537075291 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c039c10 con 0x7f769415ab40 2023-10-08T07:15:27.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f76917fa640 1 -- 172.21.15.70:0/3537075291 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c039d70 con 0x7f769415ab40 2023-10-08T07:15:27.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f76917fa640 1 -- 172.21.15.70:0/3537075291 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c043620 con 0x7f769415ab40 2023-10-08T07:15:27.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.970+0000 7f769a452640 1 -- 172.21.15.70:0/3537075291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76940c3c80 con 0x7f769415ab40 2023-10-08T07:15:27.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.971+0000 7f769a452640 1 -- 172.21.15.70:0/3537075291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76940c4240 con 0x7f769415ab40 2023-10-08T07:15:27.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.972+0000 7f769a452640 1 -- 172.21.15.70:0/3537075291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76940c4460 con 0x7f769415ab40 2023-10-08T07:15:27.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.973+0000 7f76917fa640 1 -- 172.21.15.70:0/3537075291 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f768c043780 con 0x7f769415ab40 2023-10-08T07:15:27.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.974+0000 7f76917fa640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f767406b020 0x7f767406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:27.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.974+0000 7f76917fa640 1 -- 172.21.15.70:0/3537075291 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f768c03e070 con 0x7f769415ab40 2023-10-08T07:15:27.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.975+0000 7f7693fff640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f767406b020 0x7f767406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:27.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.976+0000 7f7693fff640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f767406b020 0x7f767406d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f76940ad8b0 tx=0x7f7684008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:27.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:27.978+0000 7f76917fa640 1 -- 172.21.15.70:0/3537075291 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f768c0b0030 con 0x7f769415ab40 2023-10-08T07:15:28.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.204+0000 7f769a452640 1 -- 172.21.15.70:0/3537075291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7694067490 con 0x7f769415ab40 2023-10-08T07:15:28.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.205+0000 7f76917fa640 1 -- 172.21.15.70:0/3537075291 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4927 (secure 0 0 0) 0x7f768c034090 con 0x7f769415ab40 2023-10-08T07:15:28.206 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:28.206 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":132,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":12465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1954064575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1954064575},{"type":"v1","addr":"172.21.15.70:6834","nonce":1954064575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:57.955428+0000","modified":"2023-10-08T07:15:01.029708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8875":{"gid":8875,"name":"b","rank":0,"incarnation":127,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-08T07:15:28.207 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 132 2023-10-08T07:15:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 -- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f767406b020 msgr2=0x7f767406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f767406b020 0x7f767406d4e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f76940ad8b0 tx=0x7f7684008040 comp rx=0 tx=0).stop 2023-10-08T07:15:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 -- 172.21.15.70:0/3537075291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f769415ab40 msgr2=0x7f7694158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f769415ab40 0x7f7694158130 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f768c009870 tx=0x7f768c004730 comp rx=0 tx=0).stop 2023-10-08T07:15:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 -- 172.21.15.70:0/3537075291 shutdown_connections 2023-10-08T07:15:28.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f767406b020 0x7f767406d4e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f769415ab40 0x7f7694158130 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76940cb580 0x7f769414e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 --2- 172.21.15.70:0/3537075291 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76940c90b0 0x7f76941f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 -- 172.21.15.70:0/3537075291 >> 172.21.15.70:0/3537075291 conn(0x7f769406a380 msgr2=0x7f769406c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 -- 172.21.15.70:0/3537075291 shutdown_connections 2023-10-08T07:15:28.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.207+0000 7f769a452640 1 -- 172.21.15.70:0/3537075291 wait complete. 2023-10-08T07:15:28.227 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:28.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.426+0000 7fe8f9e02640 1 Processor -- start 2023-10-08T07:15:28.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.426+0000 7fe8f9e02640 1 -- start start 2023-10-08T07:15:28.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.428+0000 7fe8f9e02640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe8f406d4c0 0x7fe8f4162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.428+0000 7fe8f9e02640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f40ae140 con 0x7fe8f40cb150 2023-10-08T07:15:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.428+0000 7fe8f9e02640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f40ccf20 con 0x7fe8f40add60 2023-10-08T07:15:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.428+0000 7fe8f9e02640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f4161f10 con 0x7fe8f406d4c0 2023-10-08T07:15:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.428+0000 7fe8f3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe8f40cb150 0x7fe8f4164d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50276/0 (socket says 172.21.15.70:50276) 2023-10-08T07:15:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.428+0000 7fe8f3fff640 1 -- 172.21.15.70:0/3193942975 learned_addr learned my addr 172.21.15.70:0/3193942975 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.428+0000 7fe8f2ffd640 1 --2- 172.21.15.70:0/3193942975 >> v2:172.21.15.181:3300/0 conn(0x7fe8f406d4c0 0x7fe8f4162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399069121 0 0) 0x7fe8f40ae140 con 0x7fe8f40cb150 2023-10-08T07:15:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8dc003520 con 0x7fe8f40cb150 2023-10-08T07:15:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3099244979 0 0) 0x7fe8dc003520 con 0x7fe8f40cb150 2023-10-08T07:15:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 >> v2:172.21.15.181:3300/0 conn(0x7fe8f406d4c0 msgr2=0x7fe8f4162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 --2- 172.21.15.70:0/3193942975 >> v2:172.21.15.181:3300/0 conn(0x7fe8f406d4c0 0x7fe8f4162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 >> v1:172.21.15.181:6789/0 conn(0x7fe8f40add60 legacy=0x7fe8f40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f40cd1c0 con 0x7fe8f40cb150 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fe8ec0019f0 con 0x7fe8f40cb150 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 >> v1:172.21.15.70:6789/0 conn(0x7fe8f40cb150 legacy=0x7fe8f4164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc003970 0x7fe8dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc0047c0 0x7fe8dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8dc005180 0x7fe8dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4161f10 con 0x7fe8dc005180 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc00c120 con 0x7fe8dc0047c0 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc00c2e0 con 0x7fe8dc003970 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.429+0000 7fe8f3fff640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8dc005180 0x7fe8dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.430+0000 7fe8f37fe640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc003970 0x7fe8dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.430+0000 7fe8f2ffd640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc0047c0 0x7fe8dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.430+0000 7fe8f3fff640 1 -- 172.21.15.70:0/3193942975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc003970 msgr2=0x7fe8dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.430+0000 7fe8f3fff640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc003970 0x7fe8dc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.430+0000 7fe8f3fff640 1 -- 172.21.15.70:0/3193942975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc0047c0 msgr2=0x7fe8dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.430+0000 7fe8f3fff640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8dc0047c0 0x7fe8dc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.430+0000 7fe8f3fff640 1 -- 172.21.15.70:0/3193942975 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc003520 con 0x7fe8dc005180 2023-10-08T07:15:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.431+0000 7fe8f3fff640 1 -- 172.21.15.70:0/3193942975 reap_dead start 2023-10-08T07:15:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.431+0000 7fe8f3fff640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8dc005180 0x7fe8dc00b980 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00a820 tx=0x7fe8dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce33581e60f27851 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.431+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0072d0 con 0x7fe8dc005180 2023-10-08T07:15:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.431+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec02f070 con 0x7fe8dc005180 2023-10-08T07:15:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.431+0000 7fe8f27fc640 1 -- 172.21.15.70:0/3193942975 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec0076b0 con 0x7fe8dc005180 2023-10-08T07:15:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.432+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3193942975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8dc005180 msgr2=0x7fe8dc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.432+0000 7fe8f9e02640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8dc005180 0x7fe8dc00b980 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00a820 tx=0x7fe8dc0057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.432+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3193942975 shutdown_connections 2023-10-08T07:15:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.432+0000 7fe8f9e02640 1 --2- 172.21.15.70:0/3193942975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8dc005180 0x7fe8dc00b980 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.432+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3193942975 >> 172.21.15.70:0/3193942975 conn(0x7fe8f406a380 msgr2=0x7fe8f40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.432+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3193942975 shutdown_connections 2023-10-08T07:15:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.432+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3193942975 wait complete. 2023-10-08T07:15:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.434+0000 7fe8f9e02640 1 Processor -- start 2023-10-08T07:15:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.434+0000 7fe8f9e02640 1 -- start start 2023-10-08T07:15:28.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f9e02640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc005180 0x7fe8f40b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f9e02640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40b2b40 0x7fe8f4155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f9e02640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8f4156890 0x7fe8f4156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f9e02640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f40b3cc0 con 0x7fe8f40b2b40 2023-10-08T07:15:28.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f9e02640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f4157260 con 0x7fe8f4156890 2023-10-08T07:15:28.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f9e02640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f40b3040 con 0x7fe8dc005180 2023-10-08T07:15:28.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40b2b40 0x7fe8f4155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40b2b40 0x7fe8f4155a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59208/0 (socket says 172.21.15.70:59208) 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f2ffd640 1 -- 172.21.15.70:0/3785227590 learned_addr learned my addr 172.21.15.70:0/3785227590 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f2ffd640 1 -- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc005180 msgr2=0x7fe8f40b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f2ffd640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc005180 0x7fe8f40b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f2ffd640 1 -- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8f4156890 msgr2=0x7fe8f4156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f2ffd640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8f4156890 0x7fe8f4156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.435+0000 7fe8f2ffd640 1 -- 172.21.15.70:0/3785227590 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f40b3320 con 0x7fe8f40b2b40 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.436+0000 7fe8f2ffd640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40b2b40 0x7fe8f4155a30 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8010890 tx=0x7fe8e8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.436+0000 7fe8dbfff640 1 -- 172.21.15.70:0/3785227590 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8009610 con 0x7fe8f40b2b40 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.436+0000 7fe8dbfff640 1 -- 172.21.15.70:0/3785227590 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e800eb60 con 0x7fe8f40b2b40 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.436+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3785227590 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f40b3600 con 0x7fe8f40b2b40 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.436+0000 7fe8dbfff640 1 -- 172.21.15.70:0/3785227590 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e801c400 con 0x7fe8f40b2b40 2023-10-08T07:15:28.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.436+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3785227590 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f40b45c0 con 0x7fe8f40b2b40 2023-10-08T07:15:28.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.437+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3785227590 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f4067490 con 0x7fe8f40b2b40 2023-10-08T07:15:28.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.438+0000 7fe8dbfff640 1 -- 172.21.15.70:0/3785227590 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fe8e801c560 con 0x7fe8f40b2b40 2023-10-08T07:15:28.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.440+0000 7fe8dbfff640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fe8e006b050 0x7fe8e006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.440+0000 7fe8dbfff640 1 -- 172.21.15.70:0/3785227590 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fe8e808a3c0 con 0x7fe8f40b2b40 2023-10-08T07:15:28.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.440+0000 7fe8f37fe640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fe8e006b050 0x7fe8e006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.442+0000 7fe8f37fe640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fe8e006b050 0x7fe8e006d510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40015d0 tx=0x7fe8e4009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:28.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.444+0000 7fe8dbfff640 1 -- 172.21.15.70:0/3785227590 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8e8054d20 con 0x7fe8f40b2b40 2023-10-08T07:15:28.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.626+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3785227590 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8f40cd530 con 0x7fe8f40b2b40 2023-10-08T07:15:28.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.628+0000 7fe8dbfff640 1 -- 172.21.15.70:0/3785227590 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v263) v1 ==== 74+0+18300 (secure 0 0 0) 0x7fe8e8058340 con 0x7fe8f40b2b40 2023-10-08T07:15:28.630 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:28.630 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":263,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:25.894354+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:14:56.178783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:14:56.977158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6832/3856326680":"2023-10-09T07:15:22.506757+0000","172.21.15.181:0/888851152":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3676453840":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3956317048":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1513601554":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6832/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/348399880":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1607590530":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/2690144496":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/3464560328":"2023-10-09T07:15:10.723067+0000","172.21.15.70:6834/3856326680":"2023-10-09T07:15:22.506757+0000","172.21.15.181:6832/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/807583929":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/859617464":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/4271328810":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/735805106":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/957356199":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2539797528":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/333977458":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2403193161":"2023-10-09T07:15:10.723067+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:28.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.630+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fe8e006b050 msgr2=0x7fe8e006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.630+0000 7fe8f9e02640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fe8e006b050 0x7fe8e006d510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40015d0 tx=0x7fe8e4009040 comp rx=0 tx=0).stop 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.631+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3785227590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40b2b40 msgr2=0x7fe8f4155a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.631+0000 7fe8f9e02640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40b2b40 0x7fe8f4155a30 secure :-1 s=READY pgs=2804 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8010890 tx=0x7fe8e8010d50 comp rx=0 tx=0).stop 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.631+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3785227590 shutdown_connections 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.631+0000 7fe8f9e02640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fe8e006b050 0x7fe8e006d510 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.631+0000 7fe8f9e02640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe8f4156890 0x7fe8f4156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.631+0000 7fe8f9e02640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40b2b40 0x7fe8f4155a30 unknown :-1 s=CLOSED pgs=2804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.631+0000 7fe8f9e02640 1 --2- 172.21.15.70:0/3785227590 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe8dc005180 0x7fe8f40b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.631+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3785227590 >> 172.21.15.70:0/3785227590 conn(0x7fe8f406a380 msgr2=0x7fe8f406c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.631+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3785227590 shutdown_connections 2023-10-08T07:15:28.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.631+0000 7fe8f9e02640 1 -- 172.21.15.70:0/3785227590 wait complete. 2023-10-08T07:15:28.649 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-10-08T07:15:28.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:28.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.858+0000 7f95579c9640 1 Processor -- start 2023-10-08T07:15:28.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.858+0000 7f95579c9640 1 -- start start 2023-10-08T07:15:28.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.860+0000 7f95579c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f955015ab60 0x7f95500b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.860+0000 7f95579c9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f955015af40 0x7f95500b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.860+0000 7f95579c9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955015ec40 con 0x7f955006d660 2023-10-08T07:15:28.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.860+0000 7f95579c9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95500b1e00 con 0x7f955015ab60 2023-10-08T07:15:28.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.860+0000 7f95579c9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f955015ef20 con 0x7f955015af40 2023-10-08T07:15:28.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.860+0000 7f9555765640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f955015ab60 0x7f95500b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.860+0000 7f9555f66640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f955006d660 0x7f955006c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50284/0 (socket says 172.21.15.70:50284) 2023-10-08T07:15:28.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.860+0000 7f9555f66640 1 -- 172.21.15.70:0/969467986 learned_addr learned my addr 172.21.15.70:0/969467986 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.861+0000 7f9554f64640 1 --2- 172.21.15.70:0/969467986 >> v2:172.21.15.181:3300/0 conn(0x7f955015af40 0x7f95500b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.860+0000 7f9547fff640 1 -- 172.21.15.70:0/969467986 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872044750 0 0) 0x7f955015ec40 con 0x7f955006d660 2023-10-08T07:15:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.861+0000 7f9547fff640 1 -- 172.21.15.70:0/969467986 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f953c003520 con 0x7f955006d660 2023-10-08T07:15:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.861+0000 7f9555765640 1 -- 172.21.15.70:0/969467986 >> v2:172.21.15.181:3300/0 conn(0x7f955015af40 msgr2=0x7f95500b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.861+0000 7f9555765640 1 --2- 172.21.15.70:0/969467986 >> v2:172.21.15.181:3300/0 conn(0x7f955015af40 0x7f95500b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.861+0000 7f9555765640 1 -- 172.21.15.70:0/969467986 >> v1:172.21.15.70:6789/0 conn(0x7f955006d660 legacy=0x7f955006c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.861+0000 7f9555765640 1 -- 172.21.15.70:0/969467986 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955015f200 con 0x7f955015ab60 2023-10-08T07:15:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.861+0000 7f9554f64640 1 --2- 172.21.15.70:0/969467986 >> v2:172.21.15.181:3300/0 conn(0x7f955015af40 0x7f95500b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.862+0000 7f9555765640 1 --2- 172.21.15.70:0/969467986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015ab60 0x7f95500b18c0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f95500b44a0 tx=0x7f954c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b981af8afcf433 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.863+0000 7f9547fff640 1 -- 172.21.15.70:0/969467986 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c031830 con 0x7f955015ab60 2023-10-08T07:15:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.863+0000 7f9547fff640 1 -- 172.21.15.70:0/969467986 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c031990 con 0x7f955015ab60 2023-10-08T07:15:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.863+0000 7f9547fff640 1 -- 172.21.15.70:0/969467986 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c031d00 con 0x7f955015ab60 2023-10-08T07:15:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.863+0000 7f95579c9640 1 -- 172.21.15.70:0/969467986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015ab60 msgr2=0x7f95500b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.863+0000 7f95579c9640 1 --2- 172.21.15.70:0/969467986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015ab60 0x7f95500b18c0 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7f95500b44a0 tx=0x7f954c02b8d0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.863+0000 7f95579c9640 1 -- 172.21.15.70:0/969467986 shutdown_connections 2023-10-08T07:15:28.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.863+0000 7f95579c9640 1 --2- 172.21.15.70:0/969467986 >> v2:172.21.15.181:3300/0 conn(0x7f955015af40 0x7f95500b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.863+0000 7f95579c9640 1 --2- 172.21.15.70:0/969467986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015ab60 0x7f95500b18c0 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.863+0000 7f95579c9640 1 -- 172.21.15.70:0/969467986 >> 172.21.15.70:0/969467986 conn(0x7f95500cb2b0 msgr2=0x7f95500cb6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:28.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.864+0000 7f95579c9640 1 -- 172.21.15.70:0/969467986 shutdown_connections 2023-10-08T07:15:28.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.864+0000 7f95579c9640 1 -- 172.21.15.70:0/969467986 wait complete. 2023-10-08T07:15:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.865+0000 7f95579c9640 1 Processor -- start 2023-10-08T07:15:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.865+0000 7f95579c9640 1 -- start start 2023-10-08T07:15:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.866+0000 7f95579c9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f955006d660 0x7f95500cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.866+0000 7f95579c9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f955015ab60 0x7f95500c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.866+0000 7f95579c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015af40 0x7f95500ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.866+0000 7f95579c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500c5e80 con 0x7f955015af40 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.866+0000 7f95579c9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500ab8d0 con 0x7f955006d660 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.866+0000 7f95579c9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500abbb0 con 0x7f955015ab60 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.867+0000 7f9555f66640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015af40 0x7f95500ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.867+0000 7f9555f66640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015af40 0x7f95500ac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59224/0 (socket says 172.21.15.70:59224) 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.867+0000 7f9555f66640 1 -- 172.21.15.70:0/2879400741 learned_addr learned my addr 172.21.15.70:0/2879400741 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.867+0000 7f9554f64640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f955015ab60 0x7f95500c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.867+0000 7f9555f66640 1 -- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f955015ab60 msgr2=0x7f95500c6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.867+0000 7f9555f66640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f955015ab60 0x7f95500c6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.867+0000 7f9555f66640 1 -- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f955006d660 msgr2=0x7f95500cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.867+0000 7f9555f66640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f955006d660 0x7f95500cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.867+0000 7f9555f66640 1 -- 172.21.15.70:0/2879400741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c009400 con 0x7f955015af40 2023-10-08T07:15:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.867+0000 7f9554f64640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f955015ab60 0x7f95500c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.868+0000 7f9555f66640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015af40 0x7f95500ac690 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f954800caa0 tx=0x7f954800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.868+0000 7f9545ffb640 1 -- 172.21.15.70:0/2879400741 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954800ac00 con 0x7f955015af40 2023-10-08T07:15:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.868+0000 7f9545ffb640 1 -- 172.21.15.70:0/2879400741 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954800ad60 con 0x7f955015af40 2023-10-08T07:15:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.868+0000 7f9545ffb640 1 -- 172.21.15.70:0/2879400741 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548012620 con 0x7f955015af40 2023-10-08T07:15:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.868+0000 7f95579c9640 1 -- 172.21.15.70:0/2879400741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95500abe90 con 0x7f955015af40 2023-10-08T07:15:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.868+0000 7f95579c9640 1 -- 172.21.15.70:0/2879400741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95500cc940 con 0x7f955015af40 2023-10-08T07:15:28.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.869+0000 7f95579c9640 1 -- 172.21.15.70:0/2879400741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95500ac3b0 con 0x7f955015af40 2023-10-08T07:15:28.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.870+0000 7f9545ffb640 1 -- 172.21.15.70:0/2879400741 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f9548007070 con 0x7f955015af40 2023-10-08T07:15:28.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.871+0000 7f9545ffb640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f953406b020 0x7f953406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:28.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.871+0000 7f9545ffb640 1 -- 172.21.15.70:0/2879400741 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f9548089ef0 con 0x7f955015af40 2023-10-08T07:15:28.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.872+0000 7f9555765640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f953406b020 0x7f953406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:28.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.873+0000 7f9555765640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f953406b020 0x7f953406d4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f954c036000 tx=0x7f954c030000 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:28.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:28.875+0000 7f9545ffb640 1 -- 172.21.15.70:0/2879400741 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95500ac3b0 con 0x7f955015af40 2023-10-08T07:15:29.068 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:29.066+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:29.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.106+0000 7f95579c9640 1 -- 172.21.15.70:0/2879400741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95500ac3b0 con 0x7f955015af40 2023-10-08T07:15:29.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.107+0000 7f9545ffb640 1 -- 172.21.15.70:0/2879400741 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4927 (secure 0 0 0) 0x7f95500ac3b0 con 0x7f955015af40 2023-10-08T07:15:29.109 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:29.110 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":132,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":12465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1954064575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1954064575},{"type":"v1","addr":"172.21.15.70:6834","nonce":1954064575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:57.955428+0000","modified":"2023-10-08T07:15:01.029708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8875":{"gid":8875,"name":"b","rank":0,"incarnation":127,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-08T07:15:29.110 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 132 2023-10-08T07:15:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 -- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f953406b020 msgr2=0x7f953406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f953406b020 0x7f953406d4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f954c036000 tx=0x7f954c030000 comp rx=0 tx=0).stop 2023-10-08T07:15:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 -- 172.21.15.70:0/2879400741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015af40 msgr2=0x7f95500ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015af40 0x7f95500ac690 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7f954800caa0 tx=0x7f954800cf60 comp rx=0 tx=0).stop 2023-10-08T07:15:29.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 -- 172.21.15.70:0/2879400741 shutdown_connections 2023-10-08T07:15:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f953406b020 0x7f953406d4e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f955015af40 0x7f95500ac690 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f955015ab60 0x7f95500c6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 --2- 172.21.15.70:0/2879400741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f955006d660 0x7f95500cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 -- 172.21.15.70:0/2879400741 >> 172.21.15.70:0/2879400741 conn(0x7f95500cb2b0 msgr2=0x7f95500701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.110+0000 7f95579c9640 1 -- 172.21.15.70:0/2879400741 shutdown_connections 2023-10-08T07:15:29.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.111+0000 7f95579c9640 1 -- 172.21.15.70:0/2879400741 wait complete. 2023-10-08T07:15:29.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:29.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.338+0000 7fd1f2906640 1 Processor -- start 2023-10-08T07:15:29.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.338+0000 7fd1f2906640 1 -- start start 2023-10-08T07:15:29.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.340+0000 7fd1f2906640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec15cc60 0x7fd1ec0cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.340+0000 7fd1f2906640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec15d040 con 0x7fd1ec0ce6e0 2023-10-08T07:15:29.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.340+0000 7fd1f2906640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0d0650 con 0x7fd1ec0d16f0 2023-10-08T07:15:29.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.340+0000 7fd1f2906640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0d0930 con 0x7fd1ec15cc60 2023-10-08T07:15:29.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.340+0000 7fd1f2105640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd1ec0ce6e0 0x7fd1ec166aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50288/0 (socket says 172.21.15.70:50288) 2023-10-08T07:15:29.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.340+0000 7fd1f2105640 1 -- 172.21.15.70:0/1500491871 learned_addr learned my addr 172.21.15.70:0/1500491871 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:29.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f1904640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec15cc60 0x7fd1ec0cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641663042 0 0) 0x7fd1ec15d040 con 0x7fd1ec0ce6e0 2023-10-08T07:15:29.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1d8003520 con 0x7fd1ec0ce6e0 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3241640788 0 0) 0x7fd1d8003520 con 0x7fd1ec0ce6e0 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec15cc60 msgr2=0x7fd1ec0cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec15cc60 0x7fd1ec0cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 >> v1:172.21.15.181:6789/0 conn(0x7fd1ec0d16f0 legacy=0x7fd1ec0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0d0c10 con 0x7fd1ec0ce6e0 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd1e40019f0 con 0x7fd1ec0ce6e0 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 >> v1:172.21.15.70:6789/0 conn(0x7fd1ec0ce6e0 legacy=0x7fd1ec166aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 0x7fd1d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d80047c0 0x7fd1d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec15cc60 0x7fd1d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0d0930 con 0x7fd1d80039a0 2023-10-08T07:15:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d800ba30 con 0x7fd1d80047c0 2023-10-08T07:15:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.341+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d800bbf0 con 0x7fd1ec15cc60 2023-10-08T07:15:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.342+0000 7fd1f1904640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 0x7fd1d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.342+0000 7fd1f2105640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec15cc60 0x7fd1d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.342+0000 7fd1f1904640 1 -- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec15cc60 msgr2=0x7fd1d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.342+0000 7fd1f1904640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec15cc60 0x7fd1d8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.342+0000 7fd1f1904640 1 -- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d80047c0 msgr2=0x7fd1d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.342+0000 7fd1f1103640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d80047c0 0x7fd1d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.342+0000 7fd1f1904640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1d80047c0 0x7fd1d8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.342+0000 7fd1f1904640 1 -- 172.21.15.70:0/1500491871 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d8003520 con 0x7fd1d80039a0 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.343+0000 7fd1f1904640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 0x7fd1d8003dc0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009a80 tx=0x7fd1d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c196d03ea22b84a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.343+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8005990 con 0x7fd1d80039a0 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.344+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e8005af0 con 0x7fd1d80039a0 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.344+0000 7fd1f0902640 1 -- 172.21.15.70:0/1500491871 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8005ea0 con 0x7fd1d80039a0 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.344+0000 7fd1f2906640 1 -- 172.21.15.70:0/1500491871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 msgr2=0x7fd1d8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.344+0000 7fd1f2906640 1 --2- 172.21.15.70:0/1500491871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 0x7fd1d8003dc0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009a80 tx=0x7fd1d80057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.344+0000 7fd1f2105640 1 -- 172.21.15.70:0/1500491871 reap_dead start 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.344+0000 7fd1f2906640 1 -- 172.21.15.70:0/1500491871 shutdown_connections 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.344+0000 7fd1f2906640 1 -- 172.21.15.70:0/1500491871 >> 172.21.15.70:0/1500491871 conn(0x7fd1ec09bcf0 msgr2=0x7fd1ec0d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.344+0000 7fd1f2906640 1 -- 172.21.15.70:0/1500491871 shutdown_connections 2023-10-08T07:15:29.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.344+0000 7fd1f2906640 1 -- 172.21.15.70:0/1500491871 wait complete. 2023-10-08T07:15:29.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.346+0000 7fd1f2906640 1 Processor -- start 2023-10-08T07:15:29.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.346+0000 7fd1f2906640 1 -- start start 2023-10-08T07:15:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.347+0000 7fd1f2906640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 0x7fd1ec159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.347+0000 7fd1f2906640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec1587d0 0x7fd1ec15f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.347+0000 7fd1f2906640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1ec158c50 0x7fd1ec0b7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.347+0000 7fd1f2906640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec15d040 con 0x7fd1d80039a0 2023-10-08T07:15:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.347+0000 7fd1f2906640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec15d250 con 0x7fd1ec158c50 2023-10-08T07:15:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.347+0000 7fd1f2906640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec15d530 con 0x7fd1ec1587d0 2023-10-08T07:15:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f1904640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 0x7fd1ec159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f1904640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 0x7fd1ec159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59230/0 (socket says 172.21.15.70:59230) 2023-10-08T07:15:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f1904640 1 -- 172.21.15.70:0/3946077085 learned_addr learned my addr 172.21.15.70:0/3946077085 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:29.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f1904640 1 -- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec1587d0 msgr2=0x7fd1ec15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f1103640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec1587d0 0x7fd1ec15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f2105640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1ec158c50 0x7fd1ec0b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f1904640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec1587d0 0x7fd1ec15f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f1904640 1 -- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1ec158c50 msgr2=0x7fd1ec0b7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f1904640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1ec158c50 0x7fd1ec0b7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f1904640 1 -- 172.21.15.70:0/3946077085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec15d810 con 0x7fd1d80039a0 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.348+0000 7fd1f1103640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec1587d0 0x7fd1ec15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.349+0000 7fd1f1904640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 0x7fd1ec159090 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009ba0 tx=0x7fd1e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.349+0000 7fd1e27fc640 1 -- 172.21.15.70:0/3946077085 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8005c60 con 0x7fd1d80039a0 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.349+0000 7fd1e27fc640 1 -- 172.21.15.70:0/3946077085 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e8004030 con 0x7fd1d80039a0 2023-10-08T07:15:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.349+0000 7fd1f2906640 1 -- 172.21.15.70:0/3946077085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec15daf0 con 0x7fd1d80039a0 2023-10-08T07:15:29.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.349+0000 7fd1e27fc640 1 -- 172.21.15.70:0/3946077085 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80043e0 con 0x7fd1d80039a0 2023-10-08T07:15:29.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.349+0000 7fd1f2906640 1 -- 172.21.15.70:0/3946077085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec1595d0 con 0x7fd1d80039a0 2023-10-08T07:15:29.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.350+0000 7fd1f2906640 1 -- 172.21.15.70:0/3946077085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec04e160 con 0x7fd1d80039a0 2023-10-08T07:15:29.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.351+0000 7fd1e27fc640 1 -- 172.21.15.70:0/3946077085 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fd1e8003610 con 0x7fd1d80039a0 2023-10-08T07:15:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.352+0000 7fd1e27fc640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fd1d006b050 0x7fd1d006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.352+0000 7fd1e27fc640 1 -- 172.21.15.70:0/3946077085 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fd1e80ac0a0 con 0x7fd1d80039a0 2023-10-08T07:15:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.353+0000 7fd1f1103640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fd1d006b050 0x7fd1d006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.354+0000 7fd1f1103640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fd1d006b050 0x7fd1d006d510 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0b42c0 tx=0x7fd1dc009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:29.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.356+0000 7fd1e27fc640 1 -- 172.21.15.70:0/3946077085 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1e80769e0 con 0x7fd1d80039a0 2023-10-08T07:15:29.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.594+0000 7fd1f2906640 1 -- 172.21.15.70:0/3946077085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1ec0d3080 con 0x7fd1d80039a0 2023-10-08T07:15:29.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.596+0000 7fd1e27fc640 1 -- 172.21.15.70:0/3946077085 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4927 (secure 0 0 0) 0x7fd1e804c090 con 0x7fd1d80039a0 2023-10-08T07:15:29.643 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:29.643 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":132,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":12465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1954064575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1954064575},{"type":"v1","addr":"172.21.15.70:6834","nonce":1954064575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:14:57.955428+0000","modified":"2023-10-08T07:15:01.029708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8875},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8875":{"gid":8875,"name":"b","rank":0,"incarnation":127,"state":"up:active","state_seq":6,"addr":"172.21.15.181:6837/80073995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":80073995},{"type":"v1","addr":"172.21.15.181:6837","nonce":80073995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-10-08T07:15:29.644 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 132 2023-10-08T07:15:29.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.599+0000 7fd1f2906640 1 -- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fd1d006b050 msgr2=0x7fd1d006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.599+0000 7fd1f2906640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fd1d006b050 0x7fd1d006d510 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0b42c0 tx=0x7fd1dc009040 comp rx=0 tx=0).stop 2023-10-08T07:15:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.599+0000 7fd1f2906640 1 -- 172.21.15.70:0/3946077085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 msgr2=0x7fd1ec159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.599+0000 7fd1f2906640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 0x7fd1ec159090 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009ba0 tx=0x7fd1e8004730 comp rx=0 tx=0).stop 2023-10-08T07:15:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.599+0000 7fd1f2906640 1 -- 172.21.15.70:0/3946077085 shutdown_connections 2023-10-08T07:15:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.599+0000 7fd1f2906640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fd1d006b050 0x7fd1d006d510 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.599+0000 7fd1f2906640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd1ec158c50 0x7fd1ec0b7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.599+0000 7fd1f2906640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd1ec1587d0 0x7fd1ec15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.599+0000 7fd1f2906640 1 --2- 172.21.15.70:0/3946077085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1d80039a0 0x7fd1ec159090 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.599+0000 7fd1f2906640 1 -- 172.21.15.70:0/3946077085 >> 172.21.15.70:0/3946077085 conn(0x7fd1ec09bcf0 msgr2=0x7fd1ec06b040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.600+0000 7fd1f2906640 1 -- 172.21.15.70:0/3946077085 shutdown_connections 2023-10-08T07:15:29.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.600+0000 7fd1f2906640 1 -- 172.21.15.70:0/3946077085 wait complete. 2023-10-08T07:15:29.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.862+0000 7fb5f4f5e640 1 Processor -- start 2023-10-08T07:15:29.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.862+0000 7fb5f4f5e640 1 -- start start 2023-10-08T07:15:29.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.864+0000 7fb5f4f5e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb5f015ceb0 0x7fb5f00d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.864+0000 7fb5f4f5e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f00b6390 con 0x7fb5f015cad0 2023-10-08T07:15:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.864+0000 7fb5f4f5e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f00b65d0 con 0x7fb5f00b8870 2023-10-08T07:15:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.864+0000 7fb5f4f5e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f00b68b0 con 0x7fb5f015ceb0 2023-10-08T07:15:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.864+0000 7fb5ee59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb5f015cad0 0x7fb5f00b5c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50298/0 (socket says 172.21.15.70:50298) 2023-10-08T07:15:29.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.864+0000 7fb5ee59c640 1 -- 172.21.15.70:0/2381715363 learned_addr learned my addr 172.21.15.70:0/2381715363 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.864+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 333307735 0 0) 0x7fb5e0001110 con 0x7fb5f015cad0 2023-10-08T07:15:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5edd9b640 1 --2- 172.21.15.70:0/2381715363 >> v2:172.21.15.181:3300/0 conn(0x7fb5f015ceb0 0x7fb5f00d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d8003520 con 0x7fb5f015cad0 2023-10-08T07:15:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4098808545 0 0) 0x7fb5d8003520 con 0x7fb5f015cad0 2023-10-08T07:15:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 >> v2:172.21.15.181:3300/0 conn(0x7fb5f015ceb0 msgr2=0x7fb5f00d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 --2- 172.21.15.70:0/2381715363 >> v2:172.21.15.181:3300/0 conn(0x7fb5f015ceb0 0x7fb5f00d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 >> v1:172.21.15.181:6789/0 conn(0x7fb5f00b8870 legacy=0x7fb5f00cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00b6b90 con 0x7fb5f015cad0 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fb5e0001800 con 0x7fb5f015cad0 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 >> v1:172.21.15.70:6789/0 conn(0x7fb5f015cad0 legacy=0x7fb5f00b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5d80039a0 0x7fb5d8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5d80047c0 0x7fb5d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d80051d0 0x7fb5d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f00b68b0 con 0x7fb5d80039a0 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d800c100 con 0x7fb5d80047c0 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d800c2c0 con 0x7fb5d80051d0 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.865+0000 7fb5ee59c640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5d80039a0 0x7fb5d8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.866+0000 7fb5edd9b640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5d80047c0 0x7fb5d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.866+0000 7fb5ee59c640 1 -- 172.21.15.70:0/2381715363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d80051d0 msgr2=0x7fb5d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:29.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.866+0000 7fb5ee59c640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d80051d0 0x7fb5d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.866+0000 7fb5ee59c640 1 -- 172.21.15.70:0/2381715363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5d80047c0 msgr2=0x7fb5d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:29.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.866+0000 7fb5ee59c640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5d80047c0 0x7fb5d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.866+0000 7fb5ee59c640 1 -- 172.21.15.70:0/2381715363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8003520 con 0x7fb5d80039a0 2023-10-08T07:15:29.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.866+0000 7fb5eed9d640 1 -- 172.21.15.70:0/2381715363 reap_dead start 2023-10-08T07:15:29.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.866+0000 7fb5edd9b640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5d80047c0 0x7fb5d8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:29.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.867+0000 7fb5ee59c640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5d80039a0 0x7fb5d8003da0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000c240 tx=0x7fb5d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba6bf6612289b66 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:29.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.867+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0005ad0 con 0x7fb5d80039a0 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.867+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e0005c50 con 0x7fb5d80039a0 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.867+0000 7fb5ed59a640 1 -- 172.21.15.70:0/2381715363 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e002c8d0 con 0x7fb5d80039a0 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.868+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2381715363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5d80039a0 msgr2=0x7fb5d8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.868+0000 7fb5f4f5e640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5d80039a0 0x7fb5d8003da0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7fb5e000c240 tx=0x7fb5d80057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.868+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2381715363 shutdown_connections 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.868+0000 7fb5f4f5e640 1 --2- 172.21.15.70:0/2381715363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5d80039a0 0x7fb5d8003da0 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.868+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2381715363 >> 172.21.15.70:0/2381715363 conn(0x7fb5f00718a0 msgr2=0x7fb5f0071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.868+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2381715363 shutdown_connections 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.868+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2381715363 wait complete. 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.870+0000 7fb5f4f5e640 1 Processor -- start 2023-10-08T07:15:29.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.870+0000 7fb5f4f5e640 1 -- start start 2023-10-08T07:15:29.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.871+0000 7fb5f4f5e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d80039a0 0x7fb5f0153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.871+0000 7fb5f4f5e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5f0152370 0x7fb5f0151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.871+0000 7fb5f4f5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5f015d840 0x7fb5f015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.871+0000 7fb5f4f5e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f01f6af0 con 0x7fb5f015d840 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.871+0000 7fb5f4f5e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f01f5b20 con 0x7fb5f0152370 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.871+0000 7fb5f4f5e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f015e200 con 0x7fb5d80039a0 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.871+0000 7fb5eed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5f015d840 0x7fb5f015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.871+0000 7fb5eed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5f015d840 0x7fb5f015dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59252/0 (socket says 172.21.15.70:59252) 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.871+0000 7fb5eed9d640 1 -- 172.21.15.70:0/2978972542 learned_addr learned my addr 172.21.15.70:0/2978972542 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.871+0000 7fb5ee59c640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d80039a0 0x7fb5f0153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5edd9b640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5f0152370 0x7fb5f0151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5eed9d640 1 -- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d80039a0 msgr2=0x7fb5f0153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5eed9d640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d80039a0 0x7fb5f0153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5eed9d640 1 -- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5f0152370 msgr2=0x7fb5f0151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5eed9d640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5f0152370 0x7fb5f0151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5eed9d640 1 -- 172.21.15.70:0/2978972542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f0154370 con 0x7fb5f015d840 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5edd9b640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5f0152370 0x7fb5f0151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5eed9d640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5f015d840 0x7fb5f015dcc0 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00b4f70 tx=0x7fb5e800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5deffd640 1 -- 172.21.15.70:0/2978972542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80070d0 con 0x7fb5f015d840 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5deffd640 1 -- 172.21.15.70:0/2978972542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e8007230 con 0x7fb5f015d840 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5deffd640 1 -- 172.21.15.70:0/2978972542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80075e0 con 0x7fb5f015d840 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.872+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2978972542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f01f5d30 con 0x7fb5f015d840 2023-10-08T07:15:29.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.873+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2978972542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f01f62f0 con 0x7fb5f015d840 2023-10-08T07:15:29.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.874+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2978972542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f0067490 con 0x7fb5f015d840 2023-10-08T07:15:29.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.875+0000 7fb5deffd640 1 -- 172.21.15.70:0/2978972542 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fb5e8007740 con 0x7fb5f015d840 2023-10-08T07:15:29.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.876+0000 7fb5deffd640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb5d006b050 0x7fb5d006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:29.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.876+0000 7fb5deffd640 1 -- 172.21.15.70:0/2978972542 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7fb5e80893e0 con 0x7fb5f015d840 2023-10-08T07:15:29.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.877+0000 7fb5ee59c640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb5d006b050 0x7fb5d006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:29.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.878+0000 7fb5ee59c640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb5d006b050 0x7fb5d006d510 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0034000 tx=0x7fb5e0009030 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:29.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:29.880+0000 7fb5deffd640 1 -- 172.21.15.70:0/2978972542 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5e8053c90 con 0x7fb5f015d840 2023-10-08T07:15:30.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.081+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2978972542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5f00b3120 con 0x7fb5f015d840 2023-10-08T07:15:30.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.083+0000 7fb5deffd640 1 -- 172.21.15.70:0/2978972542 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v263) v1 ==== 74+0+18300 (secure 0 0 0) 0x7fb5e80572b0 con 0x7fb5f015d840 2023-10-08T07:15:30.085 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:30.085 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":263,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:25.894354+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"backup_fs_metadata","create_time":"2023-10-08T07:14:56.178783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"257","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":34,"pool_name":"backup_fs_data","create_time":"2023-10-08T07:14:56.977158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"258","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6832/3856326680":"2023-10-09T07:15:22.506757+0000","172.21.15.181:0/888851152":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3676453840":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3956317048":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1513601554":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6832/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/348399880":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1607590530":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/2690144496":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/3464560328":"2023-10-09T07:15:10.723067+0000","172.21.15.70:6834/3856326680":"2023-10-09T07:15:22.506757+0000","172.21.15.181:6832/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/807583929":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/859617464":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/4271328810":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/735805106":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/957356199":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2539797528":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/333977458":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2403193161":"2023-10-09T07:15:10.723067+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:30.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.086+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb5d006b050 msgr2=0x7fb5d006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.086+0000 7fb5f4f5e640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb5d006b050 0x7fb5d006d510 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0034000 tx=0x7fb5e0009030 comp rx=0 tx=0).stop 2023-10-08T07:15:30.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.086+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2978972542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5f015d840 msgr2=0x7fb5f015dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.086+0000 7fb5f4f5e640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5f015d840 0x7fb5f015dcc0 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00b4f70 tx=0x7fb5e800c960 comp rx=0 tx=0).stop 2023-10-08T07:15:30.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.087+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2978972542 shutdown_connections 2023-10-08T07:15:30.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.087+0000 7fb5f4f5e640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fb5d006b050 0x7fb5d006d510 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.087+0000 7fb5f4f5e640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5f015d840 0x7fb5f015dcc0 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.087+0000 7fb5f4f5e640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb5f0152370 0x7fb5f0151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.087+0000 7fb5f4f5e640 1 --2- 172.21.15.70:0/2978972542 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb5d80039a0 0x7fb5f0153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.087+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2978972542 >> 172.21.15.70:0/2978972542 conn(0x7fb5f00718a0 msgr2=0x7fb5f00b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:30.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.087+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2978972542 shutdown_connections 2023-10-08T07:15:30.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.087+0000 7fb5f4f5e640 1 -- 172.21.15.70:0/2978972542 wait complete. 2023-10-08T07:15:30.107 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-10-08T07:15:30.247 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:30.245+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:30.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.320+0000 7f7892f39640 1 Processor -- start 2023-10-08T07:15:30.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.320+0000 7f7892f39640 1 -- start start 2023-10-08T07:15:30.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.321+0000 7f7892f39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f788c15cd80 0x7f788c0b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.321+0000 7f7892f39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0b6730 0x7f788c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.321+0000 7f7892f39640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788c15d160 con 0x7f788c0cdda0 2023-10-08T07:15:30.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.321+0000 7f7892f39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c0ce230 con 0x7f788c15cd80 2023-10-08T07:15:30.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.321+0000 7f7892f39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c0b7b90 con 0x7f788c0b6730 2023-10-08T07:15:30.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.321+0000 7f7890cd5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f788c15cd80 0x7f788c0b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.322+0000 7f7890cd5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f788c15cd80 0x7f788c0b87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59254/0 (socket says 172.21.15.70:59254) 2023-10-08T07:15:30.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.322+0000 7f7890cd5640 1 -- 172.21.15.70:0/3865781209 learned_addr learned my addr 172.21.15.70:0/3865781209 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:30.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.322+0000 7f788bfff640 1 --2- 172.21.15.70:0/3865781209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0b6730 0x7f788c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.322+0000 7f7890cd5640 1 -- 172.21.15.70:0/3865781209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0b6730 msgr2=0x7f788c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:30.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.322+0000 7f7890cd5640 1 --2- 172.21.15.70:0/3865781209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0b6730 0x7f788c0cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.322+0000 7f7890cd5640 1 -- 172.21.15.70:0/3865781209 >> v1:172.21.15.181:6789/0 conn(0x7f788c0cdda0 legacy=0x7f788c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:30.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.322+0000 7f7890cd5640 1 -- 172.21.15.70:0/3865781209 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c0b8020 con 0x7f788c15cd80 2023-10-08T07:15:30.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.323+0000 7f7890cd5640 1 --2- 172.21.15.70:0/3865781209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c15cd80 0x7f788c0b87d0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f7880009870 tx=0x7f788002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a584ce6b4e0722e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:30.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.323+0000 7f788b7fe640 1 -- 172.21.15.70:0/3865781209 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880031640 con 0x7f788c15cd80 2023-10-08T07:15:30.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.324+0000 7f788b7fe640 1 -- 172.21.15.70:0/3865781209 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78800317a0 con 0x7f788c15cd80 2023-10-08T07:15:30.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.324+0000 7f788b7fe640 1 -- 172.21.15.70:0/3865781209 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880031b30 con 0x7f788c15cd80 2023-10-08T07:15:30.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.324+0000 7f7892f39640 1 -- 172.21.15.70:0/3865781209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c15cd80 msgr2=0x7f788c0b87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.324+0000 7f7892f39640 1 --2- 172.21.15.70:0/3865781209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c15cd80 0x7f788c0b87d0 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f7880009870 tx=0x7f788002b8f0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.324+0000 7f7892f39640 1 -- 172.21.15.70:0/3865781209 shutdown_connections 2023-10-08T07:15:30.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.324+0000 7f7892f39640 1 --2- 172.21.15.70:0/3865781209 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0b6730 0x7f788c0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.324+0000 7f7892f39640 1 --2- 172.21.15.70:0/3865781209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c15cd80 0x7f788c0b87d0 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.324+0000 7f7892f39640 1 -- 172.21.15.70:0/3865781209 >> 172.21.15.70:0/3865781209 conn(0x7f788c06b5d0 msgr2=0x7f788c06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:30.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.324+0000 7f7892f39640 1 -- 172.21.15.70:0/3865781209 shutdown_connections 2023-10-08T07:15:30.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.324+0000 7f7892f39640 1 -- 172.21.15.70:0/3865781209 wait complete. 2023-10-08T07:15:30.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.326+0000 7f7892f39640 1 Processor -- start 2023-10-08T07:15:30.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.326+0000 7f7892f39640 1 -- start start 2023-10-08T07:15:30.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.327+0000 7f7892f39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c0b6730 0x7f788c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.327+0000 7f7892f39640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0cdda0 0x7f788c1f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.327+0000 7f7892f39640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c15cd80 0x7f788c1f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.327+0000 7f7892f39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c15e630 con 0x7f788c0b6730 2023-10-08T07:15:30.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.327+0000 7f7892f39640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c15d830 con 0x7f788c15cd80 2023-10-08T07:15:30.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.327+0000 7f7892f39640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c15dab0 con 0x7f788c0cdda0 2023-10-08T07:15:30.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f7890cd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c0b6730 0x7f788c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f7890cd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c0b6730 0x7f788c15d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59268/0 (socket says 172.21.15.70:59268) 2023-10-08T07:15:30.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f7890cd5640 1 -- 172.21.15.70:0/1644110720 learned_addr learned my addr 172.21.15.70:0/1644110720 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:30.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f78914d6640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c15cd80 0x7f788c1f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f788bfff640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0cdda0 0x7f788c1f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f7890cd5640 1 -- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0cdda0 msgr2=0x7f788c1f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f7890cd5640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0cdda0 0x7f788c1f56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f7890cd5640 1 -- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c15cd80 msgr2=0x7f788c1f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f7890cd5640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c15cd80 0x7f788c1f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f7890cd5640 1 -- 172.21.15.70:0/1644110720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78800094d0 con 0x7f788c0b6730 2023-10-08T07:15:30.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.328+0000 7f78914d6640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c15cd80 0x7f788c1f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:30.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.329+0000 7f7890cd5640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c0b6730 0x7f788c15d2f0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f7880009870 tx=0x7f78800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:30.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.329+0000 7f78897fa640 1 -- 172.21.15.70:0/1644110720 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788003ac60 con 0x7f788c0b6730 2023-10-08T07:15:30.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.329+0000 7f78897fa640 1 -- 172.21.15.70:0/1644110720 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788003ade0 con 0x7f788c0b6730 2023-10-08T07:15:30.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.329+0000 7f78897fa640 1 -- 172.21.15.70:0/1644110720 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880044630 con 0x7f788c0b6730 2023-10-08T07:15:30.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.329+0000 7f7892f39640 1 -- 172.21.15.70:0/1644110720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c15dd90 con 0x7f788c0b6730 2023-10-08T07:15:30.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.329+0000 7f7892f39640 1 -- 172.21.15.70:0/1644110720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c15e350 con 0x7f788c0b6730 2023-10-08T07:15:30.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.330+0000 7f7892f39640 1 -- 172.21.15.70:0/1644110720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c0c9470 con 0x7f788c0b6730 2023-10-08T07:15:30.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.331+0000 7f78897fa640 1 -- 172.21.15.70:0/1644110720 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f788002f050 con 0x7f788c0b6730 2023-10-08T07:15:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.332+0000 7f78897fa640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f787006b020 0x7f787006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.332+0000 7f78897fa640 1 -- 172.21.15.70:0/1644110720 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(263..263 src has 1..263) v4 ==== 7103+0+0 (secure 0 0 0) 0x7f78800ac100 con 0x7f788c0b6730 2023-10-08T07:15:30.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.333+0000 7f788bfff640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f787006b020 0x7f787006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.334+0000 7f788bfff640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f787006b020 0x7f787006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7878006fd0 tx=0x7f7878008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:30.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.336+0000 7f78897fa640 1 -- 172.21.15.70:0/1644110720 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78800768a0 con 0x7f788c0b6730 2023-10-08T07:15:30.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.577+0000 7f7892f39640 1 -- 172.21.15.70:0/1644110720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7f788c0cb250 con 0x7f788c0b6730 2023-10-08T07:15:30.579 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T07:15:30.578+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T07:15:30.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.581+0000 7f78897fa640 1 -- 172.21.15.70:0/1644110720 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v133) v1 ==== 167+0+0 (secure 0 0 0) 0x7f788c0cb250 con 0x7f788c0b6730 2023-10-08T07:15:30.583 INFO:teuthology.orchestra.run.smithi070.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T07:15:30.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 -- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f787006b020 msgr2=0x7f787006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f787006b020 0x7f787006d4e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7878006fd0 tx=0x7f7878008040 comp rx=0 tx=0).stop 2023-10-08T07:15:30.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 -- 172.21.15.70:0/1644110720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c0b6730 msgr2=0x7f788c15d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c0b6730 0x7f788c15d2f0 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f7880009870 tx=0x7f78800046c0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 -- 172.21.15.70:0/1644110720 shutdown_connections 2023-10-08T07:15:30.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f787006b020 0x7f787006d4e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f788c15cd80 0x7f788c1f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f788c0cdda0 0x7f788c1f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 --2- 172.21.15.70:0/1644110720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f788c0b6730 0x7f788c15d2f0 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 -- 172.21.15.70:0/1644110720 >> 172.21.15.70:0/1644110720 conn(0x7f788c06b5d0 msgr2=0x7f788c0d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:30.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 -- 172.21.15.70:0/1644110720 shutdown_connections 2023-10-08T07:15:30.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.585+0000 7f7892f39640 1 -- 172.21.15.70:0/1644110720 wait complete. 2023-10-08T07:15:30.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-10-08T07:15:30.608 INFO:tasks.ceph.mds.b.smithi181.stderr: -1821> 2023-10-08T07:15:00.349+0000 7f062a709640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:15:30.687 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T07:15:30.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.806+0000 7f575a1d0640 1 Processor -- start 2023-10-08T07:15:30.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.806+0000 7f575a1d0640 1 -- start start 2023-10-08T07:15:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.808+0000 7f575a1d0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f57540ce1c0 0x7f5754071640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.808+0000 7f575a1d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57540c9ff0 con 0x7f5754071b80 2023-10-08T07:15:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.808+0000 7f575a1d0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5754072290 con 0x7f57541603a0 2023-10-08T07:15:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.808+0000 7f575a1d0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57540c3100 con 0x7f57540ce1c0 2023-10-08T07:15:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.808+0000 7f5753fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5754071b80 0x7f5754160910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50310/0 (socket says 172.21.15.70:50310) 2023-10-08T07:15:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.808+0000 7f5753fff640 1 -- 172.21.15.70:0/1742235908 learned_addr learned my addr 172.21.15.70:0/1742235908 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f574bfff640 1 --2- 172.21.15.70:0/1742235908 >> v2:172.21.15.181:3300/0 conn(0x7f57540ce1c0 0x7f5754071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911790525 0 0) 0x7f57540c9ff0 con 0x7f5754071b80 2023-10-08T07:15:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5740003520 con 0x7f5754071b80 2023-10-08T07:15:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 647216353 0 0) 0x7f5740003520 con 0x7f5754071b80 2023-10-08T07:15:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 >> v2:172.21.15.181:3300/0 conn(0x7f57540ce1c0 msgr2=0x7f5754071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 --2- 172.21.15.70:0/1742235908 >> v2:172.21.15.181:3300/0 conn(0x7f57540ce1c0 0x7f5754071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 >> v1:172.21.15.181:6789/0 conn(0x7f57541603a0 legacy=0x7f57540cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57540b12e0 con 0x7f5754071b80 2023-10-08T07:15:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f574c0019f0 con 0x7f5754071b80 2023-10-08T07:15:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 >> v1:172.21.15.70:6789/0 conn(0x7f5754071b80 legacy=0x7f5754160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740003970 0x7f5740003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57400047c0 0x7f5740004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f574000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574000bff0 con 0x7f5740005180 2023-10-08T07:15:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574000c160 con 0x7f5740003970 2023-10-08T07:15:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.809+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574000c320 con 0x7f57400047c0 2023-10-08T07:15:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.810+0000 7f5753fff640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f574000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.810+0000 7f574bfff640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57400047c0 0x7f5740004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.810+0000 7f57537fe640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740003970 0x7f5740003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.810+0000 7f5753fff640 1 -- 172.21.15.70:0/1742235908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57400047c0 msgr2=0x7f5740004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.810+0000 7f5753fff640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57400047c0 0x7f5740004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.810+0000 7f5753fff640 1 -- 172.21.15.70:0/1742235908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740003970 msgr2=0x7f5740003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.810+0000 7f5753fff640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5740003970 0x7f5740003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.810+0000 7f5753fff640 1 -- 172.21.15.70:0/1742235908 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740003520 con 0x7f5740005180 2023-10-08T07:15:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.811+0000 7f5753fff640 1 -- 172.21.15.70:0/1742235908 reap_dead start 2023-10-08T07:15:30.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.811+0000 7f5753fff640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f574000b980 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f574c00a830 tx=0x7f57400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcff64f2527e14a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:30.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.812+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0072d0 con 0x7f5740005180 2023-10-08T07:15:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.812+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c02f070 con 0x7f5740005180 2023-10-08T07:15:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.812+0000 7f5752ffd640 1 -- 172.21.15.70:0/1742235908 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0076b0 con 0x7f5740005180 2023-10-08T07:15:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.812+0000 7f575a1d0640 1 -- 172.21.15.70:0/1742235908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 msgr2=0x7f574000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.812+0000 7f575a1d0640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f574000b980 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f574c00a830 tx=0x7f57400057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.812+0000 7f575a1d0640 1 -- 172.21.15.70:0/1742235908 shutdown_connections 2023-10-08T07:15:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.812+0000 7f575a1d0640 1 --2- 172.21.15.70:0/1742235908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f574000b980 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.812+0000 7f575a1d0640 1 -- 172.21.15.70:0/1742235908 >> 172.21.15.70:0/1742235908 conn(0x7f57540cc2e0 msgr2=0x7f57540cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.812+0000 7f575a1d0640 1 -- 172.21.15.70:0/1742235908 shutdown_connections 2023-10-08T07:15:30.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.812+0000 7f575a1d0640 1 -- 172.21.15.70:0/1742235908 wait complete. 2023-10-08T07:15:30.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.814+0000 7f575a1d0640 1 Processor -- start 2023-10-08T07:15:30.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.815+0000 7f575a1d0640 1 -- start start 2023-10-08T07:15:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f575a1d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f57541f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f575a1d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57540c9180 0x7f57540ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f575a1d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57540c9600 0x7f57540ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f575a1d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57540b2f10 con 0x7f5740005180 2023-10-08T07:15:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f575a1d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57541f6480 con 0x7f57540c9180 2023-10-08T07:15:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f575a1d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57541f6760 con 0x7f57540c9600 2023-10-08T07:15:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f57537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f57541f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f57537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f57541f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59278/0 (socket says 172.21.15.70:59278) 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f57537fe640 1 -- 172.21.15.70:0/639455863 learned_addr learned my addr 172.21.15.70:0/639455863 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f5753fff640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57540c9600 0x7f57540ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f57537fe640 1 -- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57540c9600 msgr2=0x7f57540ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f57537fe640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57540c9600 0x7f57540ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f57537fe640 1 -- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57540c9180 msgr2=0x7f57540ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f574bfff640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57540c9180 0x7f57540ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f57537fe640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57540c9180 0x7f57540ac920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.816+0000 7f57537fe640 1 -- 172.21.15.70:0/639455863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57541f23c0 con 0x7f5740005180 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.817+0000 7f57537fe640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f57541f2ea0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f574400d570 tx=0x7f574400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.817+0000 7f5750ff9640 1 -- 172.21.15.70:0/639455863 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744003610 con 0x7f5740005180 2023-10-08T07:15:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.817+0000 7f5750ff9640 1 -- 172.21.15.70:0/639455863 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5744003770 con 0x7f5740005180 2023-10-08T07:15:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.817+0000 7f575a1d0640 1 -- 172.21.15.70:0/639455863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57541f26a0 con 0x7f5740005180 2023-10-08T07:15:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.817+0000 7f5750ff9640 1 -- 172.21.15.70:0/639455863 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744003b20 con 0x7f5740005180 2023-10-08T07:15:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.817+0000 7f575a1d0640 1 -- 172.21.15.70:0/639455863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57541f58a0 con 0x7f5740005180 2023-10-08T07:15:30.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.818+0000 7f575a1d0640 1 -- 172.21.15.70:0/639455863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f575404e160 con 0x7f5740005180 2023-10-08T07:15:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.819+0000 7f5750ff9640 1 -- 172.21.15.70:0/639455863 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f5744003c80 con 0x7f5740005180 2023-10-08T07:15:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.820+0000 7f5750ff9640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f573406b020 0x7f573406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.820+0000 7f5750ff9640 1 -- 172.21.15.70:0/639455863 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f5744089f80 con 0x7f5740005180 2023-10-08T07:15:30.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.821+0000 7f574bfff640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f573406b020 0x7f573406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:30.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.822+0000 7f574bfff640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f573406b020 0x7f573406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57540cbcb0 tx=0x7f573c008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:30.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:30.824+0000 7f5750ff9640 1 -- 172.21.15.70:0/639455863 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5744054780 con 0x7f5740005180 2023-10-08T07:15:31.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.063+0000 7f575a1d0640 1 -- 172.21.15.70:0/639455863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f57540b57e0 con 0x7f5740005180 2023-10-08T07:15:31.461 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:31.458+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:31.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.585+0000 7f5750ff9640 1 -- 172.21.15.70:0/639455863 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v134) v1 ==== 107+0+0 (secure 0 0 0) 0x7f5744057da0 con 0x7f5740005180 2023-10-08T07:15:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 -- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f573406b020 msgr2=0x7f573406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f573406b020 0x7f573406d4e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57540cbcb0 tx=0x7f573c008040 comp rx=0 tx=0).stop 2023-10-08T07:15:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 -- 172.21.15.70:0/639455863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 msgr2=0x7f57541f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f57541f2ea0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f574400d570 tx=0x7f574400da30 comp rx=0 tx=0).stop 2023-10-08T07:15:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 -- 172.21.15.70:0/639455863 shutdown_connections 2023-10-08T07:15:31.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f573406b020 0x7f573406d4e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57540c9600 0x7f57540ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57540c9180 0x7f57540ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 --2- 172.21.15.70:0/639455863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5740005180 0x7f57541f2ea0 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 -- 172.21.15.70:0/639455863 >> 172.21.15.70:0/639455863 conn(0x7f57540cc2e0 msgr2=0x7f57540cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 -- 172.21.15.70:0/639455863 shutdown_connections 2023-10-08T07:15:31.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.589+0000 7f575a1d0640 1 -- 172.21.15.70:0/639455863 wait complete. 2023-10-08T07:15:31.611 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-10-08T07:15:31.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.825+0000 7f574abd7640 1 Processor -- start 2023-10-08T07:15:31.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.826+0000 7f574abd7640 1 -- start start 2023-10-08T07:15:31.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.827+0000 7f574abd7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f574415e3c0 0x7f574415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:31.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.827+0000 7f574abd7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f57440b1910 0x7f574415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:31.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.827+0000 7f574abd7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415f500 0x7f57440b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.827+0000 7f574abd7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574415f990 con 0x7f574415e3c0 2023-10-08T07:15:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.827+0000 7f574abd7640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57440b1500 con 0x7f57440b1910 2023-10-08T07:15:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.827+0000 7f574abd7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57440cf1d0 con 0x7f574415f500 2023-10-08T07:15:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.828+0000 7f5748973640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f574415e3c0 0x7f574415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.828+0000 7f5748973640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f574415e3c0 0x7f574415ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59286/0 (socket says 172.21.15.70:59286) 2023-10-08T07:15:31.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.828+0000 7f5748973640 1 -- 172.21.15.70:0/82266221 learned_addr learned my addr 172.21.15.70:0/82266221 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.828+0000 7f5749174640 1 --2- 172.21.15.70:0/82266221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415f500 0x7f57440b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.828+0000 7f5743fff640 1 --2- 172.21.15.70:0/82266221 >> v2:172.21.15.181:3300/0 conn(0x7f57440b1910 0x7f574415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.828+0000 7f5748973640 1 -- 172.21.15.70:0/82266221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415f500 msgr2=0x7f57440b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.828+0000 7f5748973640 1 --2- 172.21.15.70:0/82266221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415f500 0x7f57440b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.828+0000 7f5748973640 1 -- 172.21.15.70:0/82266221 >> v2:172.21.15.181:3300/0 conn(0x7f57440b1910 msgr2=0x7f574415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.828+0000 7f5748973640 1 --2- 172.21.15.70:0/82266221 >> v2:172.21.15.181:3300/0 conn(0x7f57440b1910 0x7f574415efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.828+0000 7f5748973640 1 -- 172.21.15.70:0/82266221 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57440cf4b0 con 0x7f574415e3c0 2023-10-08T07:15:31.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.829+0000 7f5748973640 1 --2- 172.21.15.70:0/82266221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f574415e3c0 0x7f574415ea80 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f5744072110 tx=0x7f573802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b0c817a37edd6a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.830+0000 7f57437fe640 1 -- 172.21.15.70:0/82266221 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738031870 con 0x7f574415e3c0 2023-10-08T07:15:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.830+0000 7f57437fe640 1 -- 172.21.15.70:0/82266221 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57380319d0 con 0x7f574415e3c0 2023-10-08T07:15:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.830+0000 7f57437fe640 1 -- 172.21.15.70:0/82266221 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738031d60 con 0x7f574415e3c0 2023-10-08T07:15:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.831+0000 7f574abd7640 1 -- 172.21.15.70:0/82266221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f574415e3c0 msgr2=0x7f574415ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.831+0000 7f574abd7640 1 --2- 172.21.15.70:0/82266221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f574415e3c0 0x7f574415ea80 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f5744072110 tx=0x7f573802b910 comp rx=0 tx=0).stop 2023-10-08T07:15:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.831+0000 7f574abd7640 1 -- 172.21.15.70:0/82266221 shutdown_connections 2023-10-08T07:15:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.831+0000 7f574abd7640 1 --2- 172.21.15.70:0/82266221 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415f500 0x7f57440b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.831+0000 7f574abd7640 1 --2- 172.21.15.70:0/82266221 >> v2:172.21.15.181:3300/0 conn(0x7f57440b1910 0x7f574415efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.831+0000 7f574abd7640 1 --2- 172.21.15.70:0/82266221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f574415e3c0 0x7f574415ea80 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.831+0000 7f574abd7640 1 -- 172.21.15.70:0/82266221 >> 172.21.15.70:0/82266221 conn(0x7f5744099d40 msgr2=0x7f57440d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:31.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.831+0000 7f574abd7640 1 -- 172.21.15.70:0/82266221 shutdown_connections 2023-10-08T07:15:31.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.831+0000 7f574abd7640 1 -- 172.21.15.70:0/82266221 wait complete. 2023-10-08T07:15:31.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.833+0000 7f574abd7640 1 Processor -- start 2023-10-08T07:15:31.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.833+0000 7f574abd7640 1 -- start start 2023-10-08T07:15:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.834+0000 7f574abd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57440b1910 0x7f57440c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f574abd7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415e3c0 0x7f5744152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f574abd7640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f574415f500 0x7f5744153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f574abd7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5744153730 con 0x7f57440b1910 2023-10-08T07:15:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f574abd7640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5744151f40 con 0x7f574415f500 2023-10-08T07:15:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f574abd7640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57440cb930 con 0x7f574415e3c0 2023-10-08T07:15:31.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f5748973640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57440b1910 0x7f57440c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f5748973640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57440b1910 0x7f57440c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59292/0 (socket says 172.21.15.70:59292) 2023-10-08T07:15:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f5748973640 1 -- 172.21.15.70:0/1913066226 learned_addr learned my addr 172.21.15.70:0/1913066226 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f5743fff640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415e3c0 0x7f5744152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f5748973640 1 -- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415e3c0 msgr2=0x7f5744152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f5748973640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415e3c0 0x7f5744152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f5748973640 1 -- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f574415f500 msgr2=0x7f5744153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f5748973640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f574415f500 0x7f5744153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f5748973640 1 -- 172.21.15.70:0/1913066226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57380094d0 con 0x7f57440b1910 2023-10-08T07:15:31.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.835+0000 7f5743fff640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415e3c0 0x7f5744152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.836+0000 7f5748973640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57440b1910 0x7f57440c88a0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f5738000c00 tx=0x7f57380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.836+0000 7f57417fa640 1 -- 172.21.15.70:0/1913066226 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573803ad10 con 0x7f57440b1910 2023-10-08T07:15:31.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.836+0000 7f57417fa640 1 -- 172.21.15.70:0/1913066226 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573803ae70 con 0x7f57440b1910 2023-10-08T07:15:31.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.836+0000 7f57417fa640 1 -- 172.21.15.70:0/1913066226 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738044610 con 0x7f57440b1910 2023-10-08T07:15:31.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.836+0000 7f574abd7640 1 -- 172.21.15.70:0/1913066226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57440cbc10 con 0x7f57440b1910 2023-10-08T07:15:31.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.836+0000 7f574abd7640 1 -- 172.21.15.70:0/1913066226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57440cc1d0 con 0x7f57440b1910 2023-10-08T07:15:31.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.838+0000 7f574abd7640 1 -- 172.21.15.70:0/1913066226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57440cc3c0 con 0x7f57440b1910 2023-10-08T07:15:31.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.838+0000 7f57417fa640 1 -- 172.21.15.70:0/1913066226 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f573804d450 con 0x7f57440b1910 2023-10-08T07:15:31.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.839+0000 7f57417fa640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f572806b020 0x7f572806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:31.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.839+0000 7f57417fa640 1 -- 172.21.15.70:0/1913066226 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(264..264 src has 1..264) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f573802d040 con 0x7f57440b1910 2023-10-08T07:15:31.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.840+0000 7f5743fff640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f572806b020 0x7f572806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:31.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.841+0000 7f5743fff640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f572806b020 0x7f572806d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f57441540f0 tx=0x7f5730008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:31.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:31.844+0000 7f57417fa640 1 -- 172.21.15.70:0/1913066226 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5738076e20 con 0x7f57440b1910 2023-10-08T07:15:32.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.042+0000 7f574abd7640 1 -- 172.21.15.70:0/1913066226 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f574404e160 con 0x7f57440b1910 2023-10-08T07:15:32.667 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:32.665+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:32.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.964+0000 7f57417fa640 1 -- 172.21.15.70:0/1913066226 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v265) v1 ==== 190+0+0 (secure 0 0 0) 0x7f573807a440 con 0x7f57440b1910 2023-10-08T07:15:32.967 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'backup_fs_metadata' removed 2023-10-08T07:15:32.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.969+0000 7f574abd7640 1 -- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f572806b020 msgr2=0x7f572806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:32.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.969+0000 7f574abd7640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f572806b020 0x7f572806d4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f57441540f0 tx=0x7f5730008040 comp rx=0 tx=0).stop 2023-10-08T07:15:32.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.969+0000 7f574abd7640 1 -- 172.21.15.70:0/1913066226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57440b1910 msgr2=0x7f57440c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:32.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.969+0000 7f574abd7640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57440b1910 0x7f57440c88a0 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f5738000c00 tx=0x7f57380046c0 comp rx=0 tx=0).stop 2023-10-08T07:15:32.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.969+0000 7f574abd7640 1 -- 172.21.15.70:0/1913066226 shutdown_connections 2023-10-08T07:15:32.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.969+0000 7f574abd7640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f572806b020 0x7f572806d4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:32.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.969+0000 7f574abd7640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f574415f500 0x7f5744153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:32.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.969+0000 7f574abd7640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f574415e3c0 0x7f5744152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:32.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.969+0000 7f574abd7640 1 --2- 172.21.15.70:0/1913066226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57440b1910 0x7f57440c88a0 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.969+0000 7f574abd7640 1 -- 172.21.15.70:0/1913066226 >> 172.21.15.70:0/1913066226 conn(0x7f5744099d40 msgr2=0x7f5744157320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.970+0000 7f574abd7640 1 -- 172.21.15.70:0/1913066226 shutdown_connections 2023-10-08T07:15:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:32.970+0000 7f574abd7640 1 -- 172.21.15.70:0/1913066226 wait complete. 2023-10-08T07:15:32.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-10-08T07:15:33.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.195+0000 7ff18614e640 1 Processor -- start 2023-10-08T07:15:33.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.195+0000 7ff18614e640 1 -- start start 2023-10-08T07:15:33.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.197+0000 7ff18614e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ff1800cb130 0x7ff1800cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:33.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.197+0000 7ff18614e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1800b1cc0 con 0x7ff18006dfc0 2023-10-08T07:15:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.197+0000 7ff18614e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1800cbad0 con 0x7ff1800afbe0 2023-10-08T07:15:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.197+0000 7ff18614e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1800b2620 con 0x7ff1800cb130 2023-10-08T07:15:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.197+0000 7ff17effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff18006dfc0 0x7ff180070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50314/0 (socket says 172.21.15.70:50314) 2023-10-08T07:15:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.197+0000 7ff17effd640 1 -- 172.21.15.70:0/882959429 learned_addr learned my addr 172.21.15.70:0/882959429 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.197+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839038053 0 0) 0x7ff1800b1cc0 con 0x7ff18006dfc0 2023-10-08T07:15:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff168003520 con 0x7ff18006dfc0 2023-10-08T07:15:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17ffff640 1 --2- 172.21.15.70:0/882959429 >> v2:172.21.15.181:3300/0 conn(0x7ff1800cb130 0x7ff1800cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1592484693 0 0) 0x7ff168003520 con 0x7ff18006dfc0 2023-10-08T07:15:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 >> v2:172.21.15.181:3300/0 conn(0x7ff1800cb130 msgr2=0x7ff1800cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 --2- 172.21.15.70:0/882959429 >> v2:172.21.15.181:3300/0 conn(0x7ff1800cb130 0x7ff1800cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 >> v1:172.21.15.181:6789/0 conn(0x7ff1800afbe0 legacy=0x7ff1800b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1800b2060 con 0x7ff18006dfc0 2023-10-08T07:15:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ff174001670 con 0x7ff18006dfc0 2023-10-08T07:15:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 >> v1:172.21.15.70:6789/0 conn(0x7ff18006dfc0 legacy=0x7ff180070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff168003970 0x7ff168003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff168004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1680052a0 0x7ff16800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800b2620 con 0x7ff1680047c0 2023-10-08T07:15:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16800c1a0 con 0x7ff168003970 2023-10-08T07:15:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16800c360 con 0x7ff1680052a0 2023-10-08T07:15:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.198+0000 7ff17ffff640 1 --2- 172.21.15.70:0/882959429 >> v2:172.21.15.181:3300/0 conn(0x7ff1800cb130 0x7ff1800cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.199+0000 7ff17effd640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff168004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.199+0000 7ff17effd640 1 -- 172.21.15.70:0/882959429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1680052a0 msgr2=0x7ff16800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.199+0000 7ff17effd640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1680052a0 0x7ff16800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.199+0000 7ff17effd640 1 -- 172.21.15.70:0/882959429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff168003970 msgr2=0x7ff168003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.199+0000 7ff17f7fe640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff168003970 0x7ff168003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.199+0000 7ff17effd640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff168003970 0x7ff168003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.199+0000 7ff17effd640 1 -- 172.21.15.70:0/882959429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff168003520 con 0x7ff1680047c0 2023-10-08T07:15:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.199+0000 7ff17ffff640 1 -- 172.21.15.70:0/882959429 reap_dead start 2023-10-08T07:15:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.199+0000 7ff17f7fe640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff168003970 0x7ff168003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.200+0000 7ff17effd640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff168004c40 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7ff17400c210 tx=0x7ff1680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd324e77eedf35f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.200+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff174005ad0 con 0x7ff1680047c0 2023-10-08T07:15:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.200+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff174005c30 con 0x7ff1680047c0 2023-10-08T07:15:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.200+0000 7ff17e7fc640 1 -- 172.21.15.70:0/882959429 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17402cad0 con 0x7ff1680047c0 2023-10-08T07:15:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.201+0000 7ff18614e640 1 -- 172.21.15.70:0/882959429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 msgr2=0x7ff168004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.201+0000 7ff18614e640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff168004c40 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7ff17400c210 tx=0x7ff1680057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.201+0000 7ff18614e640 1 -- 172.21.15.70:0/882959429 shutdown_connections 2023-10-08T07:15:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.201+0000 7ff18614e640 1 --2- 172.21.15.70:0/882959429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff168004c40 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.201+0000 7ff18614e640 1 -- 172.21.15.70:0/882959429 >> 172.21.15.70:0/882959429 conn(0x7ff18006a5d0 msgr2=0x7ff1800b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.201+0000 7ff18614e640 1 -- 172.21.15.70:0/882959429 shutdown_connections 2023-10-08T07:15:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.201+0000 7ff18614e640 1 -- 172.21.15.70:0/882959429 wait complete. 2023-10-08T07:15:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.203+0000 7ff18614e640 1 Processor -- start 2023-10-08T07:15:33.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.203+0000 7ff18614e640 1 -- start start 2023-10-08T07:15:33.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.204+0000 7ff18614e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff1800c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:33.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.204+0000 7ff18614e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1801518c0 0x7ff1801527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:33.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.204+0000 7ff18614e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1800c7070 0x7ff180150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:33.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.204+0000 7ff18614e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1800c5350 con 0x7ff1680047c0 2023-10-08T07:15:33.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.204+0000 7ff18614e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18015c960 con 0x7ff1801518c0 2023-10-08T07:15:33.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.204+0000 7ff18614e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18015cc40 con 0x7ff1800c7070 2023-10-08T07:15:33.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.204+0000 7ff17f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff1800c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:33.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.204+0000 7ff17f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff1800c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59314/0 (socket says 172.21.15.70:59314) 2023-10-08T07:15:33.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.204+0000 7ff17f7fe640 1 -- 172.21.15.70:0/646512048 learned_addr learned my addr 172.21.15.70:0/646512048 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:33.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.205+0000 7ff17f7fe640 1 -- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1800c7070 msgr2=0x7ff180150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:33.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.205+0000 7ff17f7fe640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1800c7070 0x7ff180150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.205+0000 7ff17f7fe640 1 -- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1801518c0 msgr2=0x7ff1801527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.205+0000 7ff17f7fe640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1801518c0 0x7ff1801527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.205+0000 7ff17f7fe640 1 -- 172.21.15.70:0/646512048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18015cf20 con 0x7ff1680047c0 2023-10-08T07:15:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.205+0000 7ff17f7fe640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff1800c4e10 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7ff17000d400 tx=0x7ff17000d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.205+0000 7ff167fff640 1 -- 172.21.15.70:0/646512048 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17000b070 con 0x7ff1680047c0 2023-10-08T07:15:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.205+0000 7ff167fff640 1 -- 172.21.15.70:0/646512048 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff170003610 con 0x7ff1680047c0 2023-10-08T07:15:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.205+0000 7ff167fff640 1 -- 172.21.15.70:0/646512048 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170003a00 con 0x7ff1680047c0 2023-10-08T07:15:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.205+0000 7ff18614e640 1 -- 172.21.15.70:0/646512048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18015d200 con 0x7ff1680047c0 2023-10-08T07:15:33.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.206+0000 7ff18614e640 1 -- 172.21.15.70:0/646512048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1801520f0 con 0x7ff1680047c0 2023-10-08T07:15:33.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.207+0000 7ff18614e640 1 -- 172.21.15.70:0/646512048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1800cb510 con 0x7ff1680047c0 2023-10-08T07:15:33.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.208+0000 7ff167fff640 1 -- 172.21.15.70:0/646512048 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7ff170003b60 con 0x7ff1680047c0 2023-10-08T07:15:33.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.208+0000 7ff167fff640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ff16c06b020 0x7ff16c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:33.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.209+0000 7ff167fff640 1 -- 172.21.15.70:0/646512048 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(265..265 src has 1..265) v4 ==== 6745+0+0 (secure 0 0 0) 0x7ff170089580 con 0x7ff1680047c0 2023-10-08T07:15:33.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.209+0000 7ff17effd640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ff16c06b020 0x7ff16c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:33.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.210+0000 7ff17effd640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ff16c06b020 0x7ff16c06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff174034000 tx=0x7ff174039040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:33.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.213+0000 7ff167fff640 1 -- 172.21.15.70:0/646512048 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff170053fd0 con 0x7ff1680047c0 2023-10-08T07:15:33.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.415+0000 7ff18614e640 1 -- 172.21.15.70:0/646512048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff18004e160 con 0x7ff1680047c0 2023-10-08T07:15:33.907 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:33.906+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.975+0000 7ff167fff640 1 -- 172.21.15.70:0/646512048 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v266) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff1700575f0 con 0x7ff1680047c0 2023-10-08T07:15:33.977 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'backup_fs_data' removed 2023-10-08T07:15:33.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.980+0000 7ff18614e640 1 -- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ff16c06b020 msgr2=0x7ff16c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:33.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.980+0000 7ff18614e640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ff16c06b020 0x7ff16c06d4e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff174034000 tx=0x7ff174039040 comp rx=0 tx=0).stop 2023-10-08T07:15:33.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.980+0000 7ff18614e640 1 -- 172.21.15.70:0/646512048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 msgr2=0x7ff1800c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:33.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.980+0000 7ff18614e640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff1800c4e10 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7ff17000d400 tx=0x7ff17000d8c0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.981+0000 7ff18614e640 1 -- 172.21.15.70:0/646512048 shutdown_connections 2023-10-08T07:15:33.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.981+0000 7ff18614e640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ff16c06b020 0x7ff16c06d4e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.981+0000 7ff18614e640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ff1800c7070 0x7ff180150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.981+0000 7ff18614e640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ff1801518c0 0x7ff1801527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.981+0000 7ff18614e640 1 --2- 172.21.15.70:0/646512048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff1680047c0 0x7ff1800c4e10 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:33.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.981+0000 7ff18614e640 1 -- 172.21.15.70:0/646512048 >> 172.21.15.70:0/646512048 conn(0x7ff18006a5d0 msgr2=0x7ff18006d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:33.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.981+0000 7ff18614e640 1 -- 172.21.15.70:0/646512048 shutdown_connections 2023-10-08T07:15:33.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:33.982+0000 7ff18614e640 1 -- 172.21.15.70:0/646512048 wait complete. 2023-10-08T07:15:34.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T07:15:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.219+0000 7fdd52209640 1 Processor -- start 2023-10-08T07:15:34.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.219+0000 7fdd52209640 1 -- start start 2023-10-08T07:15:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.219+0000 7fdd52209640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdd4c0cb130 0x7fdd4c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.220+0000 7fdd52209640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4c0b1cc0 con 0x7fdd4c0afbe0 2023-10-08T07:15:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.220+0000 7fdd52209640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4c0cbad0 con 0x7fdd4c06dfc0 2023-10-08T07:15:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.220+0000 7fdd52209640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c0b2620 con 0x7fdd4c0cb130 2023-10-08T07:15:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.220+0000 7fdd4b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdd4c0afbe0 0x7fdd4c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50898/0 (socket says 172.21.15.70:50898) 2023-10-08T07:15:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.220+0000 7fdd4b7fe640 1 -- 172.21.15.70:0/423296348 learned_addr learned my addr 172.21.15.70:0/423296348 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.220+0000 7fdd4bfff640 1 --2- 172.21.15.70:0/423296348 >> v2:172.21.15.70:3300/0 conn(0x7fdd4c0cb130 0x7fdd4c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.220+0000 7fdd4bfff640 1 -- 172.21.15.70:0/423296348 >> v1:172.21.15.181:6789/0 conn(0x7fdd4c06dfc0 legacy=0x7fdd4c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.220+0000 7fdd4bfff640 1 -- 172.21.15.70:0/423296348 >> v1:172.21.15.70:6789/0 conn(0x7fdd4c0afbe0 legacy=0x7fdd4c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.220+0000 7fdd4bfff640 1 -- 172.21.15.70:0/423296348 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c0b2060 con 0x7fdd4c0cb130 2023-10-08T07:15:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.220+0000 7fdd4a7fc640 1 -- 172.21.15.70:0/423296348 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316617223 0 0) 0x7fdd4c0b1cc0 con 0x7fdd4c0afbe0 2023-10-08T07:15:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.222+0000 7fdd4bfff640 1 --2- 172.21.15.70:0/423296348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 0x7fdd4c0cb590 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009d90 tx=0x7fdd4402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de026c916a51584 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.222+0000 7fdd4a7fc640 1 -- 172.21.15.70:0/423296348 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4402cc60 con 0x7fdd4c0cb130 2023-10-08T07:15:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.222+0000 7fdd4a7fc640 1 -- 172.21.15.70:0/423296348 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4402cdc0 con 0x7fdd4c0cb130 2023-10-08T07:15:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.222+0000 7fdd4a7fc640 1 -- 172.21.15.70:0/423296348 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44034660 con 0x7fdd4c0cb130 2023-10-08T07:15:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.222+0000 7fdd52209640 1 -- 172.21.15.70:0/423296348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 msgr2=0x7fdd4c0cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.222+0000 7fdd52209640 1 --2- 172.21.15.70:0/423296348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 0x7fdd4c0cb590 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009d90 tx=0x7fdd4402beb0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.223+0000 7fdd52209640 1 -- 172.21.15.70:0/423296348 shutdown_connections 2023-10-08T07:15:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.223+0000 7fdd52209640 1 --2- 172.21.15.70:0/423296348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 0x7fdd4c0cb590 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.223+0000 7fdd52209640 1 -- 172.21.15.70:0/423296348 >> 172.21.15.70:0/423296348 conn(0x7fdd4c06a5d0 msgr2=0x7fdd4c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.223+0000 7fdd52209640 1 -- 172.21.15.70:0/423296348 shutdown_connections 2023-10-08T07:15:34.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.223+0000 7fdd52209640 1 -- 172.21.15.70:0/423296348 wait complete. 2023-10-08T07:15:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.224+0000 7fdd52209640 1 Processor -- start 2023-10-08T07:15:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.225+0000 7fdd52209640 1 -- start start 2023-10-08T07:15:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd52209640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4c06dfc0 0x7fdd4c156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd52209640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4c0afbe0 0x7fdd4c1f0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd52209640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 0x7fdd4c0c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd52209640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c1565b0 con 0x7fdd4c0cb130 2023-10-08T07:15:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd52209640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c1f1060 con 0x7fdd4c06dfc0 2023-10-08T07:15:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd52209640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c1f1340 con 0x7fdd4c0afbe0 2023-10-08T07:15:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd4bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 0x7fdd4c0c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd4bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 0x7fdd4c0c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58008/0 (socket says 172.21.15.70:58008) 2023-10-08T07:15:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd4bfff640 1 -- 172.21.15.70:0/1366861102 learned_addr learned my addr 172.21.15.70:0/1366861102 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd4affd640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4c0afbe0 0x7fdd4c1f0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd4b7fe640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4c06dfc0 0x7fdd4c156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd4bfff640 1 -- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4c0afbe0 msgr2=0x7fdd4c1f0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd4bfff640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4c0afbe0 0x7fdd4c1f0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd4bfff640 1 -- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4c06dfc0 msgr2=0x7fdd4c156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd4bfff640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4c06dfc0 0x7fdd4c156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.226+0000 7fdd4bfff640 1 -- 172.21.15.70:0/1366861102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44009a70 con 0x7fdd4c0cb130 2023-10-08T07:15:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.227+0000 7fdd4bfff640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 0x7fdd4c0c5e60 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009eb0 tx=0x7fdd44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.227+0000 7fdd33fff640 1 -- 172.21.15.70:0/1366861102 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4403f420 con 0x7fdd4c0cb130 2023-10-08T07:15:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.227+0000 7fdd33fff640 1 -- 172.21.15.70:0/1366861102 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4403f580 con 0x7fdd4c0cb130 2023-10-08T07:15:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.227+0000 7fdd33fff640 1 -- 172.21.15.70:0/1366861102 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4403f8f0 con 0x7fdd4c0cb130 2023-10-08T07:15:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.227+0000 7fdd52209640 1 -- 172.21.15.70:0/1366861102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c1f1620 con 0x7fdd4c0cb130 2023-10-08T07:15:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.227+0000 7fdd52209640 1 -- 172.21.15.70:0/1366861102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c1f1c50 con 0x7fdd4c0cb130 2023-10-08T07:15:34.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.229+0000 7fdd52209640 1 -- 172.21.15.70:0/1366861102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c04e160 con 0x7fdd4c0cb130 2023-10-08T07:15:34.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.229+0000 7fdd33fff640 1 -- 172.21.15.70:0/1366861102 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fdd4403fa50 con 0x7fdd4c0cb130 2023-10-08T07:15:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.230+0000 7fdd33fff640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fdd3806b020 0x7fdd3806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.231+0000 7fdd33fff640 1 -- 172.21.15.70:0/1366861102 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7fdd440ac960 con 0x7fdd4c0cb130 2023-10-08T07:15:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.231+0000 7fdd4b7fe640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fdd3806b020 0x7fdd3806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.232+0000 7fdd4b7fe640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fdd3806b020 0x7fdd3806d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c000f60 tx=0x7fdd3c009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.234+0000 7fdd33fff640 1 -- 172.21.15.70:0/1366861102 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd44077520 con 0x7fdd4c0cb130 2023-10-08T07:15:34.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.435+0000 7fdd52209640 1 -- 172.21.15.70:0/1366861102 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdd4c159280 con 0x7fdd4c0cb130 2023-10-08T07:15:34.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.442+0000 7fdd33fff640 1 -- 172.21.15.70:0/1366861102 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd4407ab40 con 0x7fdd4c0cb130 2023-10-08T07:15:34.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.445+0000 7fdd52209640 1 -- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fdd3806b020 msgr2=0x7fdd3806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.445+0000 7fdd52209640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fdd3806b020 0x7fdd3806d4e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c000f60 tx=0x7fdd3c009040 comp rx=0 tx=0).stop 2023-10-08T07:15:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.445+0000 7fdd52209640 1 -- 172.21.15.70:0/1366861102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 msgr2=0x7fdd4c0c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.445+0000 7fdd52209640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 0x7fdd4c0c5e60 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009eb0 tx=0x7fdd44004a40 comp rx=0 tx=0).stop 2023-10-08T07:15:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.446+0000 7fdd52209640 1 -- 172.21.15.70:0/1366861102 shutdown_connections 2023-10-08T07:15:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.446+0000 7fdd52209640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fdd3806b020 0x7fdd3806d4e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.446+0000 7fdd52209640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4c0cb130 0x7fdd4c0c5e60 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.446+0000 7fdd52209640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd4c0afbe0 0x7fdd4c1f0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.446+0000 7fdd52209640 1 --2- 172.21.15.70:0/1366861102 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd4c06dfc0 0x7fdd4c156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.446+0000 7fdd52209640 1 -- 172.21.15.70:0/1366861102 >> 172.21.15.70:0/1366861102 conn(0x7fdd4c06a5d0 msgr2=0x7fdd4c0709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.446+0000 7fdd52209640 1 -- 172.21.15.70:0/1366861102 shutdown_connections 2023-10-08T07:15:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.446+0000 7fdd52209640 1 -- 172.21.15.70:0/1366861102 wait complete. 2023-10-08T07:15:34.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T07:15:34.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.676+0000 7fb015a10640 1 Processor -- start 2023-10-08T07:15:34.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.677+0000 7fb015a10640 1 -- start start 2023-10-08T07:15:34.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.678+0000 7fb015a10640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb01015cba0 0x7fb0100ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.678+0000 7fb015a10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 0x7fb0100cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.678+0000 7fb015a10640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb010072450 con 0x7fb0100cccd0 2023-10-08T07:15:34.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.678+0000 7fb015a10640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0100cd160 con 0x7fb01015cba0 2023-10-08T07:15:34.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.678+0000 7fb015a10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100713f0 con 0x7fb0100b1510 2023-10-08T07:15:34.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.678+0000 7fb00effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb01015cba0 0x7fb0100ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:34.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.679+0000 7fb00effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb01015cba0 0x7fb0100ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58016/0 (socket says 172.21.15.70:58016) 2023-10-08T07:15:34.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.679+0000 7fb00effd640 1 -- 172.21.15.70:0/3216907031 learned_addr learned my addr 172.21.15.70:0/3216907031 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:34.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.679+0000 7fb00e7fc640 1 --2- 172.21.15.70:0/3216907031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 0x7fb0100cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:34.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.679+0000 7fb00effd640 1 -- 172.21.15.70:0/3216907031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 msgr2=0x7fb0100cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:34.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.679+0000 7fb00effd640 1 --2- 172.21.15.70:0/3216907031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 0x7fb0100cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.679+0000 7fb00effd640 1 -- 172.21.15.70:0/3216907031 >> v1:172.21.15.70:6789/0 conn(0x7fb0100cccd0 legacy=0x7fb010069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.679+0000 7fb00effd640 1 -- 172.21.15.70:0/3216907031 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010071690 con 0x7fb01015cba0 2023-10-08T07:15:34.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.679+0000 7fb00e7fc640 1 --2- 172.21.15.70:0/3216907031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 0x7fb0100cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:34.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.679+0000 7fb00dffb640 1 -- 172.21.15.70:0/3216907031 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042605237 0 0) 0x7fb010072450 con 0x7fb0100cccd0 2023-10-08T07:15:34.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.680+0000 7fb00effd640 1 --2- 172.21.15.70:0/3216907031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01015cba0 0x7fb0100ce030 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7fb000009e50 tx=0x7fb00002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1b4e5dccff73c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:34.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb00dffb640 1 -- 172.21.15.70:0/3216907031 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00002cb00 con 0x7fb01015cba0 2023-10-08T07:15:34.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb00dffb640 1 -- 172.21.15.70:0/3216907031 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00002cc60 con 0x7fb01015cba0 2023-10-08T07:15:34.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb00dffb640 1 -- 172.21.15.70:0/3216907031 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0000343f0 con 0x7fb01015cba0 2023-10-08T07:15:34.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb015a10640 1 -- 172.21.15.70:0/3216907031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01015cba0 msgr2=0x7fb0100ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb015a10640 1 --2- 172.21.15.70:0/3216907031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01015cba0 0x7fb0100ce030 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7fb000009e50 tx=0x7fb00002bf60 comp rx=0 tx=0).stop 2023-10-08T07:15:34.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb015a10640 1 -- 172.21.15.70:0/3216907031 shutdown_connections 2023-10-08T07:15:34.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb015a10640 1 --2- 172.21.15.70:0/3216907031 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 0x7fb0100cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb015a10640 1 --2- 172.21.15.70:0/3216907031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb01015cba0 0x7fb0100ce030 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb015a10640 1 -- 172.21.15.70:0/3216907031 >> 172.21.15.70:0/3216907031 conn(0x7fb01006b250 msgr2=0x7fb01006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:34.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb015a10640 1 -- 172.21.15.70:0/3216907031 shutdown_connections 2023-10-08T07:15:34.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.681+0000 7fb015a10640 1 -- 172.21.15.70:0/3216907031 wait complete. 2023-10-08T07:15:34.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.683+0000 7fb015a10640 1 Processor -- start 2023-10-08T07:15:34.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.683+0000 7fb015a10640 1 -- start start 2023-10-08T07:15:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.684+0000 7fb015a10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 0x7fb01015a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.684+0000 7fb015a10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0100cccd0 0x7fb0100aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.684+0000 7fb015a10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01015cba0 0x7fb0100af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.684+0000 7fb015a10640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100c4070 con 0x7fb0100cccd0 2023-10-08T07:15:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.684+0000 7fb015a10640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100c4350 con 0x7fb01015cba0 2023-10-08T07:15:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.684+0000 7fb015a10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100c4630 con 0x7fb0100b1510 2023-10-08T07:15:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.685+0000 7fb00e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0100cccd0 0x7fb0100aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.685+0000 7fb00e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0100cccd0 0x7fb0100aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58022/0 (socket says 172.21.15.70:58022) 2023-10-08T07:15:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.685+0000 7fb00e7fc640 1 -- 172.21.15.70:0/2783445717 learned_addr learned my addr 172.21.15.70:0/2783445717 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.685+0000 7fb00effd640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 0x7fb01015a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.685+0000 7fb00e7fc640 1 -- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 msgr2=0x7fb01015a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.685+0000 7fb00e7fc640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 0x7fb01015a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.685+0000 7fb00e7fc640 1 -- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01015cba0 msgr2=0x7fb0100af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.685+0000 7fb00e7fc640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01015cba0 0x7fb0100af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.685+0000 7fb00e7fc640 1 -- 172.21.15.70:0/2783445717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000009b30 con 0x7fb0100cccd0 2023-10-08T07:15:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.686+0000 7fb00e7fc640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0100cccd0 0x7fb0100aeb10 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7faff800c7f0 tx=0x7faff800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.686+0000 7fb007fff640 1 -- 172.21.15.70:0/2783445717 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8007820 con 0x7fb0100cccd0 2023-10-08T07:15:34.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.686+0000 7fb007fff640 1 -- 172.21.15.70:0/2783445717 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff8007980 con 0x7fb0100cccd0 2023-10-08T07:15:34.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.686+0000 7fb007fff640 1 -- 172.21.15.70:0/2783445717 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8007d30 con 0x7fb0100cccd0 2023-10-08T07:15:34.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.686+0000 7fb015a10640 1 -- 172.21.15.70:0/2783445717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0100c4910 con 0x7fb0100cccd0 2023-10-08T07:15:34.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.686+0000 7fb015a10640 1 -- 172.21.15.70:0/2783445717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0100c33b0 con 0x7fb0100cccd0 2023-10-08T07:15:34.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.687+0000 7fb015a10640 1 -- 172.21.15.70:0/2783445717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01004e160 con 0x7fb0100cccd0 2023-10-08T07:15:34.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.688+0000 7fb007fff640 1 -- 172.21.15.70:0/2783445717 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7faff80113f0 con 0x7fb0100cccd0 2023-10-08T07:15:34.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.689+0000 7fb007fff640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faff006b020 0x7faff006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:34.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.689+0000 7fb007fff640 1 -- 172.21.15.70:0/2783445717 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7faff8089ff0 con 0x7fb0100cccd0 2023-10-08T07:15:34.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.690+0000 7fb00effd640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faff006b020 0x7faff006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:34.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.691+0000 7fb00effd640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faff006b020 0x7faff006d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb000035000 tx=0x7fb000030000 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:34.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.694+0000 7fb007fff640 1 -- 172.21.15.70:0/2783445717 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faff8054a90 con 0x7fb0100cccd0 2023-10-08T07:15:34.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.885+0000 7fb015a10640 1 -- 172.21.15.70:0/2783445717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fb0100caac0 con 0x7fb0100cccd0 2023-10-08T07:15:34.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.889+0000 7fb007fff640 1 -- 172.21.15.70:0/2783445717 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v86) v1 ==== 115+0+0 (secure 0 0 0) 0x7faff80580b0 con 0x7fb0100cccd0 2023-10-08T07:15:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 -- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faff006b020 msgr2=0x7faff006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faff006b020 0x7faff006d4e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb000035000 tx=0x7fb000030000 comp rx=0 tx=0).stop 2023-10-08T07:15:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 -- 172.21.15.70:0/2783445717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0100cccd0 msgr2=0x7fb0100aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0100cccd0 0x7fb0100aeb10 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7faff800c7f0 tx=0x7faff800ccb0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 -- 172.21.15.70:0/2783445717 shutdown_connections 2023-10-08T07:15:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faff006b020 0x7faff006d4e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb01015cba0 0x7fb0100af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0100cccd0 0x7fb0100aeb10 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 --2- 172.21.15.70:0/2783445717 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb0100b1510 0x7fb01015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:34.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 -- 172.21.15.70:0/2783445717 >> 172.21.15.70:0/2783445717 conn(0x7fb01006b250 msgr2=0x7fb0100ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:34.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 -- 172.21.15.70:0/2783445717 shutdown_connections 2023-10-08T07:15:34.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:34.892+0000 7fb015a10640 1 -- 172.21.15.70:0/2783445717 wait complete. 2023-10-08T07:15:34.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T07:15:35.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.135+0000 7f2ba23c5640 1 Processor -- start 2023-10-08T07:15:35.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.135+0000 7f2ba23c5640 1 -- start start 2023-10-08T07:15:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2ba23c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b9c15cd80 0x7f2b9c0b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2ba23c5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b9c0b6730 0x7f2b9c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2ba23c5640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c15d160 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2ba23c5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c0ce230 con 0x7f2b9c0b6730 2023-10-08T07:15:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2ba23c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0b7b90 con 0x7f2b9c15cd80 2023-10-08T07:15:35.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2b9b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b9c0b6730 0x7f2b9c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2b9b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b9c0b6730 0x7f2b9c0cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58032/0 (socket says 172.21.15.70:58032) 2023-10-08T07:15:35.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2b9b7fe640 1 -- 172.21.15.70:0/3984605133 learned_addr learned my addr 172.21.15.70:0/3984605133 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:35.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2b9bfff640 1 --2- 172.21.15.70:0/3984605133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b9c15cd80 0x7f2b9c0b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2b9b7fe640 1 -- 172.21.15.70:0/3984605133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b9c15cd80 msgr2=0x7f2b9c0b87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:35.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2b9b7fe640 1 --2- 172.21.15.70:0/3984605133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b9c15cd80 0x7f2b9c0b87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.137+0000 7f2b9b7fe640 1 -- 172.21.15.70:0/3984605133 >> v1:172.21.15.181:6789/0 conn(0x7f2b9c0cdda0 legacy=0x7f2b9c0b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:35.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.138+0000 7f2b9b7fe640 1 -- 172.21.15.70:0/3984605133 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0b8020 con 0x7f2b9c0b6730 2023-10-08T07:15:35.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.138+0000 7f2b9bfff640 1 --2- 172.21.15.70:0/3984605133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b9c15cd80 0x7f2b9c0b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:35.140 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:35.137+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.139+0000 7f2b9b7fe640 1 --2- 172.21.15.70:0/3984605133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0b6730 0x7f2b9c0cd860 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009870 tx=0x7f2b8402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48f6dbfb6a211f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.139+0000 7f2b9affd640 1 -- 172.21.15.70:0/3984605133 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84031640 con 0x7f2b9c0b6730 2023-10-08T07:15:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.139+0000 7f2b9affd640 1 -- 172.21.15.70:0/3984605133 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b840317a0 con 0x7f2b9c0b6730 2023-10-08T07:15:35.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.139+0000 7f2b9affd640 1 -- 172.21.15.70:0/3984605133 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84031b30 con 0x7f2b9c0b6730 2023-10-08T07:15:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.139+0000 7f2ba23c5640 1 -- 172.21.15.70:0/3984605133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0b6730 msgr2=0x7f2b9c0cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.139+0000 7f2ba23c5640 1 --2- 172.21.15.70:0/3984605133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0b6730 0x7f2b9c0cd860 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009870 tx=0x7f2b8402b8f0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.140+0000 7f2ba23c5640 1 -- 172.21.15.70:0/3984605133 shutdown_connections 2023-10-08T07:15:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.140+0000 7f2ba23c5640 1 --2- 172.21.15.70:0/3984605133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0b6730 0x7f2b9c0cd860 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.140+0000 7f2ba23c5640 1 --2- 172.21.15.70:0/3984605133 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b9c15cd80 0x7f2b9c0b87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.140+0000 7f2ba23c5640 1 -- 172.21.15.70:0/3984605133 >> 172.21.15.70:0/3984605133 conn(0x7f2b9c06b5d0 msgr2=0x7f2b9c06b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:35.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.140+0000 7f2ba23c5640 1 -- 172.21.15.70:0/3984605133 shutdown_connections 2023-10-08T07:15:35.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.140+0000 7f2ba23c5640 1 -- 172.21.15.70:0/3984605133 wait complete. 2023-10-08T07:15:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.141+0000 7f2ba23c5640 1 Processor -- start 2023-10-08T07:15:35.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.142+0000 7f2ba23c5640 1 -- start start 2023-10-08T07:15:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2ba23c5640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c0b6730 0x7f2b9c1f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2ba23c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0cdda0 0x7f2b9c0cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2ba23c5640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b9c15cd80 0x7f2b9c0d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2ba23c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c15e8b0 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2ba23c5640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c1579b0 con 0x7f2b9c0b6730 2023-10-08T07:15:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2ba23c5640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c1f2250 con 0x7f2b9c15cd80 2023-10-08T07:15:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2b9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0cdda0 0x7f2b9c0cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2b9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0cdda0 0x7f2b9c0cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58040/0 (socket says 172.21.15.70:58040) 2023-10-08T07:15:35.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2b9b7fe640 1 -- 172.21.15.70:0/2305607722 learned_addr learned my addr 172.21.15.70:0/2305607722 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2b9b7fe640 1 -- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b9c15cd80 msgr2=0x7f2b9c0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2b9b7fe640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b9c15cd80 0x7f2b9c0d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2b9bfff640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c0b6730 0x7f2b9c1f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2b9b7fe640 1 -- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c0b6730 msgr2=0x7f2b9c1f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2b9b7fe640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c0b6730 0x7f2b9c1f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.143+0000 7f2b9b7fe640 1 -- 172.21.15.70:0/2305607722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b840094d0 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.144+0000 7f2b9b7fe640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0cdda0 0x7f2b9c0cc4d0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f2b84000c00 tx=0x7f2b840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.144+0000 7f2b98ff9640 1 -- 172.21.15.70:0/2305607722 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8403ac60 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.144+0000 7f2b98ff9640 1 -- 172.21.15.70:0/2305607722 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8403ade0 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.144+0000 7f2b98ff9640 1 -- 172.21.15.70:0/2305607722 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84044630 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.144+0000 7f2ba23c5640 1 -- 172.21.15.70:0/2305607722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c15dd30 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.144+0000 7f2ba23c5640 1 -- 172.21.15.70:0/2305607722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c15e2f0 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.145+0000 7f2ba23c5640 1 -- 172.21.15.70:0/2305607722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c1f2470 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.146+0000 7f2b98ff9640 1 -- 172.21.15.70:0/2305607722 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f2b8402f050 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.147+0000 7f2b98ff9640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2b7c06b020 0x7f2b7c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.147+0000 7f2b98ff9640 1 -- 172.21.15.70:0/2305607722 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f2b840abeb0 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.148+0000 7f2b9bfff640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2b7c06b020 0x7f2b7c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.149+0000 7f2b9bfff640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2b7c06b020 0x7f2b7c06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0b8850 tx=0x7f2b8c008450 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:35.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.152+0000 7f2b98ff9640 1 -- 172.21.15.70:0/2305607722 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b84076980 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.341+0000 7f2ba23c5640 1 -- 172.21.15.70:0/2305607722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f2b9c04e160 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.346+0000 7f2b98ff9640 1 -- 172.21.15.70:0/2305607722 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v87) v1 ==== 120+0+0 (secure 0 0 0) 0x7f2b84079fa0 con 0x7f2b9c0cdda0 2023-10-08T07:15:35.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 -- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2b7c06b020 msgr2=0x7f2b7c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:35.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2b7c06b020 0x7f2b7c06d4e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0b8850 tx=0x7f2b8c008450 comp rx=0 tx=0).stop 2023-10-08T07:15:35.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 -- 172.21.15.70:0/2305607722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0cdda0 msgr2=0x7f2b9c0cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:35.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0cdda0 0x7f2b9c0cc4d0 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f2b84000c00 tx=0x7f2b840046c0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 -- 172.21.15.70:0/2305607722 shutdown_connections 2023-10-08T07:15:35.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f2b7c06b020 0x7f2b7c06d4e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2b9c15cd80 0x7f2b9c0d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b9c0cdda0 0x7f2b9c0cc4d0 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 --2- 172.21.15.70:0/2305607722 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2b9c0b6730 0x7f2b9c1f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 -- 172.21.15.70:0/2305607722 >> 172.21.15.70:0/2305607722 conn(0x7f2b9c06b5d0 msgr2=0x7f2b9c0c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:35.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 -- 172.21.15.70:0/2305607722 shutdown_connections 2023-10-08T07:15:35.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.349+0000 7f2ba23c5640 1 -- 172.21.15.70:0/2305607722 wait complete. 2023-10-08T07:15:35.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-10-08T07:15:35.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.583+0000 7f5307f25640 1 Processor -- start 2023-10-08T07:15:35.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.584+0000 7f5307f25640 1 -- start start 2023-10-08T07:15:35.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.585+0000 7f5307f25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53000c90b0 0x7f53000cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.585+0000 7f5307f25640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f53000cb3b0 0x7f530015e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.585+0000 7f5307f25640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530015b2b0 con 0x7f530015ab40 2023-10-08T07:15:35.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.585+0000 7f5307f25640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530015bf30 con 0x7f53000cb3b0 2023-10-08T07:15:35.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.585+0000 7f5307f25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53000cb840 con 0x7f53000c90b0 2023-10-08T07:15:35.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.585+0000 7f53064c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f53000cb3b0 0x7f530015e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.585+0000 7f53064c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f53000cb3b0 0x7f530015e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58048/0 (socket says 172.21.15.70:58048) 2023-10-08T07:15:35.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.585+0000 7f53064c2640 1 -- 172.21.15.70:0/3097425034 learned_addr learned my addr 172.21.15.70:0/3097425034 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:35.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.586+0000 7f53064c2640 1 -- 172.21.15.70:0/3097425034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53000c90b0 msgr2=0x7f53000cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.586+0000 7f53054c0640 1 --2- 172.21.15.70:0/3097425034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53000c90b0 0x7f53000cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.586+0000 7f53064c2640 1 --2- 172.21.15.70:0/3097425034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53000c90b0 0x7f53000cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.586+0000 7f53064c2640 1 -- 172.21.15.70:0/3097425034 >> v1:172.21.15.181:6789/0 conn(0x7f530015ab40 legacy=0x7f53000ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.586+0000 7f53064c2640 1 -- 172.21.15.70:0/3097425034 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530015b590 con 0x7f53000cb3b0 2023-10-08T07:15:35.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.587+0000 7f53064c2640 1 --2- 172.21.15.70:0/3097425034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000cb3b0 0x7f530015e790 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009e50 tx=0x7f52f802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f81d12109c6ba4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:35.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.587+0000 7f5304cbf640 1 -- 172.21.15.70:0/3097425034 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f802ca70 con 0x7f53000cb3b0 2023-10-08T07:15:35.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.587+0000 7f5304cbf640 1 -- 172.21.15.70:0/3097425034 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f802cbd0 con 0x7f53000cb3b0 2023-10-08T07:15:35.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.587+0000 7f5304cbf640 1 -- 172.21.15.70:0/3097425034 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f80343f0 con 0x7f53000cb3b0 2023-10-08T07:15:35.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.588+0000 7f5307f25640 1 -- 172.21.15.70:0/3097425034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000cb3b0 msgr2=0x7f530015e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:35.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.588+0000 7f5307f25640 1 --2- 172.21.15.70:0/3097425034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000cb3b0 0x7f530015e790 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009e50 tx=0x7f52f802bed0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.588+0000 7f5307f25640 1 -- 172.21.15.70:0/3097425034 shutdown_connections 2023-10-08T07:15:35.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.588+0000 7f5307f25640 1 --2- 172.21.15.70:0/3097425034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000cb3b0 0x7f530015e790 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.588+0000 7f5307f25640 1 --2- 172.21.15.70:0/3097425034 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53000c90b0 0x7f53000cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.588+0000 7f5307f25640 1 -- 172.21.15.70:0/3097425034 >> 172.21.15.70:0/3097425034 conn(0x7f530006a380 msgr2=0x7f53000b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:35.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.588+0000 7f5307f25640 1 -- 172.21.15.70:0/3097425034 shutdown_connections 2023-10-08T07:15:35.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.588+0000 7f5307f25640 1 -- 172.21.15.70:0/3097425034 wait complete. 2023-10-08T07:15:35.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.590+0000 7f5307f25640 1 Processor -- start 2023-10-08T07:15:35.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.590+0000 7f5307f25640 1 -- start start 2023-10-08T07:15:35.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.591+0000 7f5307f25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000c90b0 0x7f530014f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.591+0000 7f5307f25640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53000cb3b0 0x7f53000aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.591+0000 7f5307f25640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f530015ab40 0x7f53000af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.591+0000 7f5307f25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53000c44b0 con 0x7f53000c90b0 2023-10-08T07:15:35.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.591+0000 7f5307f25640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53000c3620 con 0x7f530015ab40 2023-10-08T07:15:35.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.591+0000 7f5307f25640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53000c3830 con 0x7f53000cb3b0 2023-10-08T07:15:35.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.591+0000 7f5305cc1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000c90b0 0x7f530014f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.591+0000 7f5305cc1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000c90b0 0x7f530014f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58056/0 (socket says 172.21.15.70:58056) 2023-10-08T07:15:35.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.591+0000 7f5305cc1640 1 -- 172.21.15.70:0/1124671079 learned_addr learned my addr 172.21.15.70:0/1124671079 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:35.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f53054c0640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53000cb3b0 0x7f53000aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f5305cc1640 1 -- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53000cb3b0 msgr2=0x7f53000aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:35.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f5305cc1640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53000cb3b0 0x7f53000aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f5305cc1640 1 -- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f530015ab40 msgr2=0x7f53000af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f53064c2640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f530015ab40 0x7f53000af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f5305cc1640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f530015ab40 0x7f53000af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f5305cc1640 1 -- 172.21.15.70:0/1124671079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f8009b30 con 0x7f53000c90b0 2023-10-08T07:15:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f5305cc1640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000c90b0 0x7f530014f320 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00c430 tx=0x7f52fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f52f67fc640 1 -- 172.21.15.70:0/1124671079 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc011790 con 0x7f53000c90b0 2023-10-08T07:15:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f52f67fc640 1 -- 172.21.15.70:0/1124671079 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc0118f0 con 0x7f53000c90b0 2023-10-08T07:15:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f52f67fc640 1 -- 172.21.15.70:0/1124671079 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc011ce0 con 0x7f53000c90b0 2023-10-08T07:15:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.592+0000 7f5307f25640 1 -- 172.21.15.70:0/1124671079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53000c3b10 con 0x7f53000c90b0 2023-10-08T07:15:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.593+0000 7f5307f25640 1 -- 172.21.15.70:0/1124671079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53000c40d0 con 0x7f53000c90b0 2023-10-08T07:15:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.593+0000 7f5307f25640 1 -- 172.21.15.70:0/1124671079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5300067490 con 0x7f53000c90b0 2023-10-08T07:15:35.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.594+0000 7f52f67fc640 1 -- 172.21.15.70:0/1124671079 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f52fc01c460 con 0x7f53000c90b0 2023-10-08T07:15:35.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.595+0000 7f52f67fc640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f52e406b020 0x7f52e406d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:35.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.595+0000 7f52f67fc640 1 -- 172.21.15.70:0/1124671079 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f52fc089ad0 con 0x7f53000c90b0 2023-10-08T07:15:35.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.596+0000 7f53054c0640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f52e406b020 0x7f52e406d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:35.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.598+0000 7f53054c0640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f52e406b020 0x7f52e406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f53000cd160 tx=0x7f52f0008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:35.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.600+0000 7f52f67fc640 1 -- 172.21.15.70:0/1124671079 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52fc054720 con 0x7f53000c90b0 2023-10-08T07:15:35.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.861+0000 7f5307f25640 1 -- 172.21.15.70:0/1124671079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7f53000cb190 con 0x7f53000c90b0 2023-10-08T07:15:35.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.982+0000 7f52f67fc640 1 -- 172.21.15.70:0/1124671079 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3314) v1 ==== 187+0+0 (secure 0 0 0) 0x7f52fc057d40 con 0x7f53000c90b0 2023-10-08T07:15:35.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 -- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f52e406b020 msgr2=0x7f52e406d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:35.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f52e406b020 0x7f52e406d4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f53000cd160 tx=0x7f52f0008040 comp rx=0 tx=0).stop 2023-10-08T07:15:35.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 -- 172.21.15.70:0/1124671079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000c90b0 msgr2=0x7f530014f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:35.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000c90b0 0x7f530014f320 secure :-1 s=READY pgs=2834 cs=0 l=1 rev1=1 crypto rx=0x7f52fc00c430 tx=0x7f52fc00c8f0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 -- 172.21.15.70:0/1124671079 shutdown_connections 2023-10-08T07:15:35.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f52e406b020 0x7f52e406d4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f530015ab40 0x7f53000af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f53000cb3b0 0x7f53000aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 --2- 172.21.15.70:0/1124671079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53000c90b0 0x7f530014f320 unknown :-1 s=CLOSED pgs=2834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:35.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 -- 172.21.15.70:0/1124671079 >> 172.21.15.70:0/1124671079 conn(0x7f530006a380 msgr2=0x7f53000c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:35.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.985+0000 7f5307f25640 1 -- 172.21.15.70:0/1124671079 shutdown_connections 2023-10-08T07:15:35.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:35.986+0000 7f5307f25640 1 -- 172.21.15.70:0/1124671079 wait complete. 2023-10-08T07:15:36.007 INFO:tasks.cephfs_test_runner:Test command fails without --fs argument in presence of multiple fs ... ok 2023-10-08T07:15:36.007 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-10-08T07:15:36.007 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-08T07:15:36.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:36.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.222+0000 7fc8f47b0640 1 Processor -- start 2023-10-08T07:15:36.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.222+0000 7fc8f47b0640 1 -- start start 2023-10-08T07:15:36.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.224+0000 7fc8f47b0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fc8ec070db0 0x7fc8ec15dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.224+0000 7fc8f47b0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ec0c9850 con 0x7fc8ec071190 2023-10-08T07:15:36.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.224+0000 7fc8f47b0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ec06a870 con 0x7fc8ec06ad00 2023-10-08T07:15:36.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.224+0000 7fc8f47b0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ec15e280 con 0x7fc8ec070db0 2023-10-08T07:15:36.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.224+0000 7fc8f1d4b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc8ec071190 0x7fc8ec15e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50918/0 (socket says 172.21.15.70:50918) 2023-10-08T07:15:36.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.224+0000 7fc8f1d4b640 1 -- 172.21.15.70:0/210538374 learned_addr learned my addr 172.21.15.70:0/210538374 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:36.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f254c640 1 --2- 172.21.15.70:0/210538374 >> v2:172.21.15.181:3300/0 conn(0x7fc8ec070db0 0x7fc8ec15dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170602577 0 0) 0x7fc8ec0c9850 con 0x7fc8ec071190 2023-10-08T07:15:36.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8d8003520 con 0x7fc8ec071190 2023-10-08T07:15:36.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3702110824 0 0) 0x7fc8d8003520 con 0x7fc8ec071190 2023-10-08T07:15:36.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 >> v2:172.21.15.181:3300/0 conn(0x7fc8ec070db0 msgr2=0x7fc8ec15dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:36.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 --2- 172.21.15.70:0/210538374 >> v2:172.21.15.181:3300/0 conn(0x7fc8ec070db0 0x7fc8ec15dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 >> v1:172.21.15.181:6789/0 conn(0x7fc8ec06ad00 legacy=0x7fc8ec06b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec15eb90 con 0x7fc8ec071190 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fc8dc001670 con 0x7fc8ec071190 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 >> v1:172.21.15.70:6789/0 conn(0x7fc8ec071190 legacy=0x7fc8ec15e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 --2- 172.21.15.70:0/210538374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8d8003970 0x7fc8d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 --2- 172.21.15.70:0/210538374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8d80047c0 0x7fc8d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 --2- 172.21.15.70:0/210538374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8d8005180 0x7fc8d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec15e280 con 0x7fc8d8003970 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d800c020 con 0x7fc8d8005180 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.225+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d800c1e0 con 0x7fc8d80047c0 2023-10-08T07:15:36.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.226+0000 7fc8f254c640 1 --2- 172.21.15.70:0/210538374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8d8003970 0x7fc8d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.226+0000 7fc8f254c640 1 -- 172.21.15.70:0/210538374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8d80047c0 msgr2=0x7fc8d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:36.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.226+0000 7fc8f254c640 1 --2- 172.21.15.70:0/210538374 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8d80047c0 0x7fc8d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.226+0000 7fc8f254c640 1 -- 172.21.15.70:0/210538374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8d8005180 msgr2=0x7fc8d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:36.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.226+0000 7fc8f254c640 1 --2- 172.21.15.70:0/210538374 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8d8005180 0x7fc8d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.226+0000 7fc8f254c640 1 -- 172.21.15.70:0/210538374 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d8003520 con 0x7fc8d8003970 2023-10-08T07:15:36.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.226+0000 7fc8f2d4d640 1 -- 172.21.15.70:0/210538374 reap_dead start 2023-10-08T07:15:36.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.227+0000 7fc8f254c640 1 --2- 172.21.15.70:0/210538374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8d8003970 0x7fc8d8003d90 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009ac0 tx=0x7fc8d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a436bb7379c0ff85 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:36.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.227+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e802c710 con 0x7fc8d8003970 2023-10-08T07:15:36.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.227+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e802c870 con 0x7fc8d8003970 2023-10-08T07:15:36.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.227+0000 7fc8f154a640 1 -- 172.21.15.70:0/210538374 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e802cc00 con 0x7fc8d8003970 2023-10-08T07:15:36.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.228+0000 7fc8f47b0640 1 -- 172.21.15.70:0/210538374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8d8003970 msgr2=0x7fc8d8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.228+0000 7fc8f47b0640 1 --2- 172.21.15.70:0/210538374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8d8003970 0x7fc8d8003d90 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009ac0 tx=0x7fc8d80057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.228+0000 7fc8f47b0640 1 -- 172.21.15.70:0/210538374 shutdown_connections 2023-10-08T07:15:36.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.228+0000 7fc8f47b0640 1 --2- 172.21.15.70:0/210538374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8d8003970 0x7fc8d8003d90 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.228+0000 7fc8f47b0640 1 -- 172.21.15.70:0/210538374 >> 172.21.15.70:0/210538374 conn(0x7fc8ec0b1480 msgr2=0x7fc8ec06cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:36.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.228+0000 7fc8f47b0640 1 -- 172.21.15.70:0/210538374 shutdown_connections 2023-10-08T07:15:36.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.228+0000 7fc8f47b0640 1 -- 172.21.15.70:0/210538374 wait complete. 2023-10-08T07:15:36.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.230+0000 7fc8f47b0640 1 Processor -- start 2023-10-08T07:15:36.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.230+0000 7fc8f47b0640 1 -- start start 2023-10-08T07:15:36.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.231+0000 7fc8f47b0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8d8003970 0x7fc8ec0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.231+0000 7fc8f47b0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ec152530 0x7fc8ec15ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.231+0000 7fc8f47b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ec15f2c0 0x7fc8ec0c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.231+0000 7fc8f47b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec15f740 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.231+0000 7fc8f47b0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0b46a0 con 0x7fc8ec152530 2023-10-08T07:15:36.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.231+0000 7fc8f47b0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0c3640 con 0x7fc8d8003970 2023-10-08T07:15:36.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f2d4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ec15f2c0 0x7fc8ec0c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f2d4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ec15f2c0 0x7fc8ec0c4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58086/0 (socket says 172.21.15.70:58086) 2023-10-08T07:15:36.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f2d4d640 1 -- 172.21.15.70:0/306363882 learned_addr learned my addr 172.21.15.70:0/306363882 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:36.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f254c640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8d8003970 0x7fc8ec0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f1d4b640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ec152530 0x7fc8ec15ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f2d4d640 1 -- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8d8003970 msgr2=0x7fc8ec0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f2d4d640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8d8003970 0x7fc8ec0c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f2d4d640 1 -- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ec152530 msgr2=0x7fc8ec15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f2d4d640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ec152530 0x7fc8ec15ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f2d4d640 1 -- 172.21.15.70:0/306363882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec0c3920 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.232+0000 7fc8f2d4d640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ec15f2c0 0x7fc8ec0c4400 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec070900 tx=0x7fc8e400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:36.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.233+0000 7fc8e2ffd640 1 -- 172.21.15.70:0/306363882 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007070 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.233+0000 7fc8e2ffd640 1 -- 172.21.15.70:0/306363882 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e40071d0 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.233+0000 7fc8e2ffd640 1 -- 172.21.15.70:0/306363882 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007560 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.233+0000 7fc8f47b0640 1 -- 172.21.15.70:0/306363882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec0c3c00 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.233+0000 7fc8f47b0640 1 -- 172.21.15.70:0/306363882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec0c8f10 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.234+0000 7fc8f47b0640 1 -- 172.21.15.70:0/306363882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec0c4120 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.235+0000 7fc8e2ffd640 1 -- 172.21.15.70:0/306363882 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fc8e40076c0 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.236+0000 7fc8e2ffd640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc8d006b050 0x7fc8d006d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.236+0000 7fc8e2ffd640 1 -- 172.21.15.70:0/306363882 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7fc8e4088fe0 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.237+0000 7fc8f254c640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc8d006b050 0x7fc8d006d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.238+0000 7fc8f254c640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc8d006b050 0x7fc8d006d510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8036000 tx=0x7fc8e8030000 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:36.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.241+0000 7fc8e2ffd640 1 -- 172.21.15.70:0/306363882 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8ec0c4120 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.383 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:36.382+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:36.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.439+0000 7fc8f47b0640 1 -- 172.21.15.70:0/306363882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8ec0c4120 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.441+0000 7fc8e2ffd640 1 -- 172.21.15.70:0/306363882 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v266) v1 ==== 74+0+14768 (secure 0 0 0) 0x7fc8ec0c4120 con 0x7fc8ec15f2c0 2023-10-08T07:15:36.442 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:36.442 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":266,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:33.966302+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6832/3856326680":"2023-10-09T07:15:22.506757+0000","172.21.15.181:0/888851152":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6837/80073995":"2023-10-09T07:15:30.579448+0000","172.21.15.181:0/3676453840":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/3956317048":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1513601554":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6832/2621244427":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/348399880":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/1607590530":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/2690144496":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/3464560328":"2023-10-09T07:15:10.723067+0000","172.21.15.181:6836/80073995":"2023-10-09T07:15:30.579448+0000","172.21.15.70:6834/3856326680":"2023-10-09T07:15:22.506757+0000","172.21.15.181:6832/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/807583929":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/859617464":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/4271328810":"2023-10-09T07:15:16.946228+0000","172.21.15.181:6833/4245028292":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/735805106":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/957356199":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2539797528":"2023-10-09T07:15:16.946228+0000","172.21.15.181:0/333977458":"2023-10-09T07:15:10.723067+0000","172.21.15.181:0/2403193161":"2023-10-09T07:15:10.723067+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:36.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 -- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc8d006b050 msgr2=0x7fc8d006d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc8d006b050 0x7fc8d006d510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8036000 tx=0x7fc8e8030000 comp rx=0 tx=0).stop 2023-10-08T07:15:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 -- 172.21.15.70:0/306363882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ec15f2c0 msgr2=0x7fc8ec0c4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ec15f2c0 0x7fc8ec0c4400 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec070900 tx=0x7fc8e400c960 comp rx=0 tx=0).stop 2023-10-08T07:15:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 -- 172.21.15.70:0/306363882 shutdown_connections 2023-10-08T07:15:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc8d006b050 0x7fc8d006d510 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8ec15f2c0 0x7fc8ec0c4400 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc8ec152530 0x7fc8ec15ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 --2- 172.21.15.70:0/306363882 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc8d8003970 0x7fc8ec0c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 -- 172.21.15.70:0/306363882 >> 172.21.15.70:0/306363882 conn(0x7fc8ec0b1480 msgr2=0x7fc8ec06b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:36.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 -- 172.21.15.70:0/306363882 shutdown_connections 2023-10-08T07:15:36.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.444+0000 7fc8f47b0640 1 -- 172.21.15.70:0/306363882 wait complete. 2023-10-08T07:15:36.463 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:15:36.463 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:15:36.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.675+0000 7f51a442b640 1 Processor -- start 2023-10-08T07:15:36.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.675+0000 7f51a442b640 1 -- start start 2023-10-08T07:15:36.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.677+0000 7f51a442b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f519c15cc60 0x7f519c1612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.677+0000 7f51a442b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f519c0cd3d0 0x7f519c0b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.677+0000 7f51a442b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c0b4f40 con 0x7f519c0b4b30 2023-10-08T07:15:36.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.677+0000 7f51a442b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c0cdd70 con 0x7f519c0cd3d0 2023-10-08T07:15:36.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.677+0000 7f51a442b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c0ce050 con 0x7f519c15cc60 2023-10-08T07:15:36.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.677+0000 7f51a19c6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f519c0cd3d0 0x7f519c0b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.677+0000 7f51a29c8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f519c0b4b30 0x7f519c06b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50934/0 (socket says 172.21.15.70:50934) 2023-10-08T07:15:36.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.677+0000 7f51a29c8640 1 -- 172.21.15.70:0/2713021119 learned_addr learned my addr 172.21.15.70:0/2713021119 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:36.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.678+0000 7f51a21c7640 1 --2- 172.21.15.70:0/2713021119 >> v2:172.21.15.181:3300/0 conn(0x7f519c15cc60 0x7f519c1612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.678+0000 7f51a19c6640 1 -- 172.21.15.70:0/2713021119 >> v2:172.21.15.181:3300/0 conn(0x7f519c15cc60 msgr2=0x7f519c1612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:36.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.678+0000 7f51a19c6640 1 --2- 172.21.15.70:0/2713021119 >> v2:172.21.15.181:3300/0 conn(0x7f519c15cc60 0x7f519c1612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.678+0000 7f51a19c6640 1 -- 172.21.15.70:0/2713021119 >> v1:172.21.15.70:6789/0 conn(0x7f519c0b4b30 legacy=0x7f519c06b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.678+0000 7f51a19c6640 1 -- 172.21.15.70:0/2713021119 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0d1010 con 0x7f519c0cd3d0 2023-10-08T07:15:36.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.679+0000 7f51a19c6640 1 --2- 172.21.15.70:0/2713021119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c0cd3d0 0x7f519c0b45f0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f519c072410 tx=0x7f518c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47ea325da1b15da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.679+0000 7f51a11c5640 1 -- 172.21.15.70:0/2713021119 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c031830 con 0x7f519c0cd3d0 2023-10-08T07:15:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.679+0000 7f51a11c5640 1 -- 172.21.15.70:0/2713021119 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c031990 con 0x7f519c0cd3d0 2023-10-08T07:15:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.679+0000 7f51a11c5640 1 -- 172.21.15.70:0/2713021119 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c031d20 con 0x7f519c0cd3d0 2023-10-08T07:15:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.680+0000 7f51a442b640 1 -- 172.21.15.70:0/2713021119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c0cd3d0 msgr2=0x7f519c0b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.680+0000 7f51a442b640 1 --2- 172.21.15.70:0/2713021119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c0cd3d0 0x7f519c0b45f0 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7f519c072410 tx=0x7f518c02b8d0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.680+0000 7f51a442b640 1 -- 172.21.15.70:0/2713021119 shutdown_connections 2023-10-08T07:15:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.680+0000 7f51a442b640 1 --2- 172.21.15.70:0/2713021119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c0cd3d0 0x7f519c0b45f0 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.680+0000 7f51a442b640 1 --2- 172.21.15.70:0/2713021119 >> v2:172.21.15.181:3300/0 conn(0x7f519c15cc60 0x7f519c1612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.680+0000 7f51a442b640 1 -- 172.21.15.70:0/2713021119 >> 172.21.15.70:0/2713021119 conn(0x7f519c0cf5f0 msgr2=0x7f519c0cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.680+0000 7f51a442b640 1 -- 172.21.15.70:0/2713021119 shutdown_connections 2023-10-08T07:15:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.680+0000 7f51a442b640 1 -- 172.21.15.70:0/2713021119 wait complete. 2023-10-08T07:15:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.682+0000 7f51a442b640 1 Processor -- start 2023-10-08T07:15:36.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.682+0000 7f51a442b640 1 -- start start 2023-10-08T07:15:36.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.683+0000 7f51a442b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f519c0b4b30 0x7f519c1510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.683+0000 7f51a442b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f519c0cd3d0 0x7f519c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.683+0000 7f51a442b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c15cc60 0x7f519c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.683+0000 7f51a442b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c151630 con 0x7f519c15cc60 2023-10-08T07:15:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.683+0000 7f51a442b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0c70a0 con 0x7f519c0cd3d0 2023-10-08T07:15:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.683+0000 7f51a442b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0cb1d0 con 0x7f519c0b4b30 2023-10-08T07:15:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.683+0000 7f51a29c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c15cc60 0x7f519c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.683+0000 7f51a29c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c15cc60 0x7f519c0cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58108/0 (socket says 172.21.15.70:58108) 2023-10-08T07:15:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.683+0000 7f51a29c8640 1 -- 172.21.15.70:0/928284251 learned_addr learned my addr 172.21.15.70:0/928284251 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:36.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.684+0000 7f51a21c7640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f519c0b4b30 0x7f519c1510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.684+0000 7f51a19c6640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f519c0cd3d0 0x7f519c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.684+0000 7f51a29c8640 1 -- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f519c0b4b30 msgr2=0x7f519c1510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.684+0000 7f51a29c8640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f519c0b4b30 0x7f519c1510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.684+0000 7f51a29c8640 1 -- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f519c0cd3d0 msgr2=0x7f519c15f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.684+0000 7f51a29c8640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f519c0cd3d0 0x7f519c15f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.684+0000 7f51a29c8640 1 -- 172.21.15.70:0/928284251 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c009400 con 0x7f519c15cc60 2023-10-08T07:15:36.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.684+0000 7f51a29c8640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c15cc60 0x7f519c0cc270 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f519400ca80 tx=0x7f519400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.684+0000 7f5192ffd640 1 -- 172.21.15.70:0/928284251 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519400ac30 con 0x7f519c15cc60 2023-10-08T07:15:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.685+0000 7f5192ffd640 1 -- 172.21.15.70:0/928284251 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519400ad90 con 0x7f519c15cc60 2023-10-08T07:15:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.685+0000 7f5192ffd640 1 -- 172.21.15.70:0/928284251 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194012600 con 0x7f519c15cc60 2023-10-08T07:15:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.685+0000 7f51a442b640 1 -- 172.21.15.70:0/928284251 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c0cb4b0 con 0x7f519c15cc60 2023-10-08T07:15:36.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.685+0000 7f51a442b640 1 -- 172.21.15.70:0/928284251 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c0cba70 con 0x7f519c15cc60 2023-10-08T07:15:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.686+0000 7f51a442b640 1 -- 172.21.15.70:0/928284251 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c04e160 con 0x7f519c15cc60 2023-10-08T07:15:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.687+0000 7f5192ffd640 1 -- 172.21.15.70:0/928284251 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f5194007070 con 0x7f519c15cc60 2023-10-08T07:15:36.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.688+0000 7f5192ffd640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f518006b020 0x7f518006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.688+0000 7f5192ffd640 1 -- 172.21.15.70:0/928284251 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f5194089980 con 0x7f519c15cc60 2023-10-08T07:15:36.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.689+0000 7f51a21c7640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f518006b020 0x7f518006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:36.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.690+0000 7f51a21c7640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f518006b020 0x7f518006d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5198006fd0 tx=0x7f5198008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:36.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.692+0000 7f5192ffd640 1 -- 172.21.15.70:0/928284251 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51940545e0 con 0x7f519c15cc60 2023-10-08T07:15:36.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.881+0000 7f51a442b640 1 -- 172.21.15.70:0/928284251 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f519c1587e0 con 0x7f519c15cc60 2023-10-08T07:15:36.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.882+0000 7f5192ffd640 1 -- 172.21.15.70:0/928284251 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v266) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51940170a0 con 0x7f519c15cc60 2023-10-08T07:15:36.883 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:15:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.884+0000 7f51a442b640 1 -- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f518006b020 msgr2=0x7f518006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.884+0000 7f51a442b640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f518006b020 0x7f518006d4e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5198006fd0 tx=0x7f5198008040 comp rx=0 tx=0).stop 2023-10-08T07:15:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.884+0000 7f51a442b640 1 -- 172.21.15.70:0/928284251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c15cc60 msgr2=0x7f519c0cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.885+0000 7f51a442b640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c15cc60 0x7f519c0cc270 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7f519400ca80 tx=0x7f519400cf40 comp rx=0 tx=0).stop 2023-10-08T07:15:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.885+0000 7f51a442b640 1 -- 172.21.15.70:0/928284251 shutdown_connections 2023-10-08T07:15:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.885+0000 7f51a442b640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f518006b020 0x7f518006d4e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.885+0000 7f51a442b640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f519c15cc60 0x7f519c0cc270 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.885+0000 7f51a442b640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f519c0cd3d0 0x7f519c15f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.885+0000 7f51a442b640 1 --2- 172.21.15.70:0/928284251 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f519c0b4b30 0x7f519c1510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:36.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.885+0000 7f51a442b640 1 -- 172.21.15.70:0/928284251 >> 172.21.15.70:0/928284251 conn(0x7f519c0cf5f0 msgr2=0x7f519c0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:36.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.885+0000 7f51a442b640 1 -- 172.21.15.70:0/928284251 shutdown_connections 2023-10-08T07:15:36.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:36.885+0000 7f51a442b640 1 -- 172.21.15.70:0/928284251 wait complete. 2023-10-08T07:15:36.905 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-08T07:15:37.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.122+0000 7f0210a75640 1 Processor -- start 2023-10-08T07:15:37.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.123+0000 7f0210a75640 1 -- start start 2023-10-08T07:15:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.124+0000 7f0210a75640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f020c159ac0 0x7f020c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.124+0000 7f0210a75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0709e0 0x7f020c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.124+0000 7f0210a75640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c06dee0 con 0x7f020c0ad790 2023-10-08T07:15:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.124+0000 7f0210a75640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c15d850 con 0x7f020c159ac0 2023-10-08T07:15:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.124+0000 7f0210a75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c15db30 con 0x7f020c0709e0 2023-10-08T07:15:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.124+0000 7f020a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f020c159ac0 0x7f020c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.124+0000 7f020a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f020c159ac0 0x7f020c06d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58118/0 (socket says 172.21.15.70:58118) 2023-10-08T07:15:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.124+0000 7f020a59c640 1 -- 172.21.15.70:0/3585594743 learned_addr learned my addr 172.21.15.70:0/3585594743 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:37.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.124+0000 7f0209d9b640 1 --1- 172.21.15.70:0/3585594743 >> v1:172.21.15.70:6789/0 conn(0x7f020c0ad790 0x7f020c0702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50944/0 (socket says 172.21.15.70:50944) 2023-10-08T07:15:37.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.125+0000 7f020a59c640 1 -- 172.21.15.70:0/3585594743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0709e0 msgr2=0x7f020c0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:37.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.125+0000 7f020a59c640 1 --2- 172.21.15.70:0/3585594743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0709e0 0x7f020c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:37.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.125+0000 7f020a59c640 1 -- 172.21.15.70:0/3585594743 >> v1:172.21.15.70:6789/0 conn(0x7f020c0ad790 legacy=0x7f020c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:37.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.125+0000 7f020a59c640 1 -- 172.21.15.70:0/3585594743 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c15de10 con 0x7f020c159ac0 2023-10-08T07:15:37.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.125+0000 7f020959a640 1 -- 172.21.15.70:0/3585594743 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778844582 0 0) 0x7f020c06dee0 con 0x7f020c0ad790 2023-10-08T07:15:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.126+0000 7f020a59c640 1 --2- 172.21.15.70:0/3585594743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 0x7f020c06d5e0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009e50 tx=0x7f01fc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99aa2fea01b900ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.126+0000 7f020959a640 1 -- 172.21.15.70:0/3585594743 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc02cb20 con 0x7f020c159ac0 2023-10-08T07:15:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.126+0000 7f020959a640 1 -- 172.21.15.70:0/3585594743 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc02cc80 con 0x7f020c159ac0 2023-10-08T07:15:37.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.126+0000 7f020959a640 1 -- 172.21.15.70:0/3585594743 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0343f0 con 0x7f020c159ac0 2023-10-08T07:15:37.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.127+0000 7f0210a75640 1 -- 172.21.15.70:0/3585594743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 msgr2=0x7f020c06d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:37.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.127+0000 7f0210a75640 1 --2- 172.21.15.70:0/3585594743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 0x7f020c06d5e0 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009e50 tx=0x7f01fc02bf80 comp rx=0 tx=0).stop 2023-10-08T07:15:37.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.127+0000 7f0210a75640 1 -- 172.21.15.70:0/3585594743 shutdown_connections 2023-10-08T07:15:37.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.127+0000 7f0210a75640 1 --2- 172.21.15.70:0/3585594743 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0709e0 0x7f020c0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:37.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.127+0000 7f0210a75640 1 --2- 172.21.15.70:0/3585594743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 0x7f020c06d5e0 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:37.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.127+0000 7f0210a75640 1 -- 172.21.15.70:0/3585594743 >> 172.21.15.70:0/3585594743 conn(0x7f020c06a380 msgr2=0x7f020c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:37.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.127+0000 7f0210a75640 1 -- 172.21.15.70:0/3585594743 shutdown_connections 2023-10-08T07:15:37.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.127+0000 7f0210a75640 1 -- 172.21.15.70:0/3585594743 wait complete. 2023-10-08T07:15:37.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.129+0000 7f0210a75640 1 Processor -- start 2023-10-08T07:15:37.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.129+0000 7f0210a75640 1 -- start start 2023-10-08T07:15:37.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.130+0000 7f0210a75640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c0709e0 0x7f020c156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:37.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.130+0000 7f0210a75640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0ad790 0x7f020c0c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:37.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.130+0000 7f0210a75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 0x7f020c0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:37.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.130+0000 7f0210a75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0c9aa0 con 0x7f020c159ac0 2023-10-08T07:15:37.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.130+0000 7f0210a75640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c15b480 con 0x7f020c0709e0 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.130+0000 7f0210a75640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0c97c0 con 0x7f020c0ad790 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 0x7f020c0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 0x7f020c0c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58132/0 (socket says 172.21.15.70:58132) 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020ad9d640 1 -- 172.21.15.70:0/2196684800 learned_addr learned my addr 172.21.15.70:0/2196684800 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f0209d9b640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0ad790 0x7f020c0c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020a59c640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c0709e0 0x7f020c156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020ad9d640 1 -- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0ad790 msgr2=0x7f020c0c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020ad9d640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0ad790 0x7f020c0c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020ad9d640 1 -- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c0709e0 msgr2=0x7f020c156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020ad9d640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c0709e0 0x7f020c156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020ad9d640 1 -- 172.21.15.70:0/2196684800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc009b30 con 0x7f020c159ac0 2023-10-08T07:15:37.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020a59c640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c0709e0 0x7f020c156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:37.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f0209d9b640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0ad790 0x7f020c0c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:37.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.131+0000 7f020ad9d640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 0x7f020c0c8f50 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f020400b3c0 tx=0x7f020400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:37.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.132+0000 7f0202ffd640 1 -- 172.21.15.70:0/2196684800 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204011830 con 0x7f020c159ac0 2023-10-08T07:15:37.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.132+0000 7f0202ffd640 1 -- 172.21.15.70:0/2196684800 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0204011990 con 0x7f020c159ac0 2023-10-08T07:15:37.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.132+0000 7f0202ffd640 1 -- 172.21.15.70:0/2196684800 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204011d40 con 0x7f020c159ac0 2023-10-08T07:15:37.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.132+0000 7f0210a75640 1 -- 172.21.15.70:0/2196684800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c15adc0 con 0x7f020c159ac0 2023-10-08T07:15:37.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.132+0000 7f0210a75640 1 -- 172.21.15.70:0/2196684800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c0c9490 con 0x7f020c159ac0 2023-10-08T07:15:37.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.133+0000 7f0210a75640 1 -- 172.21.15.70:0/2196684800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c15ac20 con 0x7f020c159ac0 2023-10-08T07:15:37.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.134+0000 7f0202ffd640 1 -- 172.21.15.70:0/2196684800 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f020401b460 con 0x7f020c159ac0 2023-10-08T07:15:37.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.135+0000 7f0202ffd640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f01ec06b020 0x7f01ec06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:37.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.136+0000 7f0202ffd640 1 -- 172.21.15.70:0/2196684800 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f0204089950 con 0x7f020c159ac0 2023-10-08T07:15:37.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.136+0000 7f020a59c640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f01ec06b020 0x7f01ec06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:37.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.138+0000 7f020a59c640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f01ec06b020 0x7f01ec06d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01fc035000 tx=0x7f01fc030000 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:37.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.140+0000 7f0202ffd640 1 -- 172.21.15.70:0/2196684800 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f020c15ac20 con 0x7f020c159ac0 2023-10-08T07:15:37.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.405+0000 7f0210a75640 1 -- 172.21.15.70:0/2196684800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f020c15b5c0 con 0x7f020c159ac0 2023-10-08T07:15:37.621 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:37.620+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:37.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.995+0000 7f0202ffd640 1 -- 172.21.15.70:0/2196684800 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3316) v1 ==== 158+0+0 (secure 0 0 0) 0x7f020c15b5c0 con 0x7f020c159ac0 2023-10-08T07:15:38.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 -- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f01ec06b020 msgr2=0x7f01ec06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:38.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f01ec06b020 0x7f01ec06d4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01fc035000 tx=0x7f01fc030000 comp rx=0 tx=0).stop 2023-10-08T07:15:38.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 -- 172.21.15.70:0/2196684800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 msgr2=0x7f020c0c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:38.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 0x7f020c0c8f50 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7f020400b3c0 tx=0x7f020400b880 comp rx=0 tx=0).stop 2023-10-08T07:15:38.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 -- 172.21.15.70:0/2196684800 shutdown_connections 2023-10-08T07:15:38.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f01ec06b020 0x7f01ec06d4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f020c159ac0 0x7f020c0c8f50 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f020c0ad790 0x7f020c0c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 --2- 172.21.15.70:0/2196684800 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f020c0709e0 0x7f020c156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 -- 172.21.15.70:0/2196684800 >> 172.21.15.70:0/2196684800 conn(0x7f020c06a380 msgr2=0x7f020c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:38.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 -- 172.21.15.70:0/2196684800 shutdown_connections 2023-10-08T07:15:38.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:37.999+0000 7f0210a75640 1 -- 172.21.15.70:0/2196684800 wait complete. 2023-10-08T07:15:38.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T07:15:38.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.229+0000 7f0be3f83640 1 Processor -- start 2023-10-08T07:15:38.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.230+0000 7f0be3f83640 1 -- start start 2023-10-08T07:15:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.231+0000 7f0be3f83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0bdc1601b0 0x7f0bdc0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.231+0000 7f0be3f83640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bdc06a460 0x7f0bdc0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.231+0000 7f0be3f83640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bdc0ce210 con 0x7f0bdc071c30 2023-10-08T07:15:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.231+0000 7f0be3f83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bdc0cdee0 con 0x7f0bdc1601b0 2023-10-08T07:15:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.231+0000 7f0be3f83640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc161530 con 0x7f0bdc06a460 2023-10-08T07:15:38.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.231+0000 7f0be1d1f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0bdc1601b0 0x7f0bdc0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:38.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.232+0000 7f0be151e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0bdc071c30 0x7f0bdc069d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50958/0 (socket says 172.21.15.70:50958) 2023-10-08T07:15:38.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.232+0000 7f0be151e640 1 -- 172.21.15.70:0/388215580 learned_addr learned my addr 172.21.15.70:0/388215580 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:38.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.232+0000 7f0be2520640 1 --2- 172.21.15.70:0/388215580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bdc06a460 0x7f0bdc0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:38.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.232+0000 7f0be0d1d640 1 -- 172.21.15.70:0/388215580 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469448408 0 0) 0x7f0bdc0ce210 con 0x7f0bdc071c30 2023-10-08T07:15:38.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.232+0000 7f0be1d1f640 1 -- 172.21.15.70:0/388215580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bdc06a460 msgr2=0x7f0bdc0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:38.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.232+0000 7f0be1d1f640 1 --2- 172.21.15.70:0/388215580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bdc06a460 0x7f0bdc0ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.232+0000 7f0be1d1f640 1 -- 172.21.15.70:0/388215580 >> v1:172.21.15.70:6789/0 conn(0x7f0bdc071c30 legacy=0x7f0bdc069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.232+0000 7f0be1d1f640 1 -- 172.21.15.70:0/388215580 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc161810 con 0x7f0bdc1601b0 2023-10-08T07:15:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.233+0000 7f0be1d1f640 1 --2- 172.21.15.70:0/388215580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc1601b0 0x7f0bdc0cd430 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80097a0 tx=0x7f0bd802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6897a405d4c5ed9e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.233+0000 7f0be0d1d640 1 -- 172.21.15.70:0/388215580 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd802d040 con 0x7f0bdc1601b0 2023-10-08T07:15:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.233+0000 7f0be0d1d640 1 -- 172.21.15.70:0/388215580 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd8031810 con 0x7f0bdc1601b0 2023-10-08T07:15:38.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.234+0000 7f0be0d1d640 1 -- 172.21.15.70:0/388215580 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8031ba0 con 0x7f0bdc1601b0 2023-10-08T07:15:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.234+0000 7f0be3f83640 1 -- 172.21.15.70:0/388215580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc1601b0 msgr2=0x7f0bdc0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.234+0000 7f0be3f83640 1 --2- 172.21.15.70:0/388215580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc1601b0 0x7f0bdc0cd430 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80097a0 tx=0x7f0bd802b8b0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.234+0000 7f0be3f83640 1 -- 172.21.15.70:0/388215580 shutdown_connections 2023-10-08T07:15:38.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.234+0000 7f0be3f83640 1 --2- 172.21.15.70:0/388215580 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bdc06a460 0x7f0bdc0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.234+0000 7f0be3f83640 1 --2- 172.21.15.70:0/388215580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc1601b0 0x7f0bdc0cd430 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.234+0000 7f0be3f83640 1 -- 172.21.15.70:0/388215580 >> 172.21.15.70:0/388215580 conn(0x7f0bdc06d430 msgr2=0x7f0bdc06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:38.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.234+0000 7f0be3f83640 1 -- 172.21.15.70:0/388215580 shutdown_connections 2023-10-08T07:15:38.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.234+0000 7f0be3f83640 1 -- 172.21.15.70:0/388215580 wait complete. 2023-10-08T07:15:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.236+0000 7f0be3f83640 1 Processor -- start 2023-10-08T07:15:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.236+0000 7f0be3f83640 1 -- start start 2023-10-08T07:15:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.237+0000 7f0be3f83640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bdc06a460 0x7f0bdc1f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.237+0000 7f0be3f83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc071c30 0x7f0bdc0c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.237+0000 7f0be3f83640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bdc1601b0 0x7f0bdc0b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.237+0000 7f0be3f83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc0ab8d0 con 0x7f0bdc071c30 2023-10-08T07:15:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.237+0000 7f0be3f83640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc0c4780 con 0x7f0bdc06a460 2023-10-08T07:15:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.237+0000 7f0be3f83640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc0abb20 con 0x7f0bdc1601b0 2023-10-08T07:15:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be151e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc071c30 0x7f0bdc0c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be151e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc071c30 0x7f0bdc0c3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58148/0 (socket says 172.21.15.70:58148) 2023-10-08T07:15:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be151e640 1 -- 172.21.15.70:0/2352295949 learned_addr learned my addr 172.21.15.70:0/2352295949 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be151e640 1 -- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bdc1601b0 msgr2=0x7f0bdc0b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be2520640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bdc1601b0 0x7f0bdc0b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be1d1f640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bdc06a460 0x7f0bdc1f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be151e640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bdc1601b0 0x7f0bdc0b06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be151e640 1 -- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bdc06a460 msgr2=0x7f0bdc1f1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:38.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be151e640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bdc06a460 0x7f0bdc1f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be151e640 1 -- 172.21.15.70:0/2352295949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd8009400 con 0x7f0bdc071c30 2023-10-08T07:15:38.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.238+0000 7f0be151e640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc071c30 0x7f0bdc0c3de0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c430 tx=0x7f0bcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:38.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.239+0000 7f0bd27fc640 1 -- 172.21.15.70:0/2352295949 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0074e0 con 0x7f0bdc071c30 2023-10-08T07:15:38.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.239+0000 7f0bd27fc640 1 -- 172.21.15.70:0/2352295949 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc007660 con 0x7f0bdc071c30 2023-10-08T07:15:38.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.239+0000 7f0bd27fc640 1 -- 172.21.15.70:0/2352295949 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc004a20 con 0x7f0bdc071c30 2023-10-08T07:15:38.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.239+0000 7f0be3f83640 1 -- 172.21.15.70:0/2352295949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bdc0abe00 con 0x7f0bdc071c30 2023-10-08T07:15:38.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.239+0000 7f0be3f83640 1 -- 172.21.15.70:0/2352295949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bdc1f2350 con 0x7f0bdc071c30 2023-10-08T07:15:38.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.240+0000 7f0be3f83640 1 -- 172.21.15.70:0/2352295949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bdc04e160 con 0x7f0bdc071c30 2023-10-08T07:15:38.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.241+0000 7f0bd27fc640 1 -- 172.21.15.70:0/2352295949 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f0bcc004b80 con 0x7f0bdc071c30 2023-10-08T07:15:38.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.242+0000 7f0bd27fc640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0bc006b020 0x7f0bc006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:38.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.242+0000 7f0bd27fc640 1 -- 172.21.15.70:0/2352295949 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f0bcc089a20 con 0x7f0bdc071c30 2023-10-08T07:15:38.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.243+0000 7f0be1d1f640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0bc006b020 0x7f0bc006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:38.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.244+0000 7f0be1d1f640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0bc006b020 0x7f0bc006d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8036000 tx=0x7f0bd803c040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:38.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.246+0000 7f0bd27fc640 1 -- 172.21.15.70:0/2352295949 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0bcc0545d0 con 0x7f0bdc071c30 2023-10-08T07:15:38.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.437+0000 7f0be3f83640 1 -- 172.21.15.70:0/2352295949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0bdc0b13b0 con 0x7f0bdc071c30 2023-10-08T07:15:38.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.442+0000 7f0bd27fc640 1 -- 172.21.15.70:0/2352295949 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v88)=0 v88) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0bcc057bf0 con 0x7f0bdc071c30 2023-10-08T07:15:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 -- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0bc006b020 msgr2=0x7f0bc006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0bc006b020 0x7f0bc006d4e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8036000 tx=0x7f0bd803c040 comp rx=0 tx=0).stop 2023-10-08T07:15:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 -- 172.21.15.70:0/2352295949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc071c30 msgr2=0x7f0bdc0c3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc071c30 0x7f0bdc0c3de0 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c430 tx=0x7f0bcc00c8f0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 -- 172.21.15.70:0/2352295949 shutdown_connections 2023-10-08T07:15:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f0bc006b020 0x7f0bc006d4e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0bdc1601b0 0x7f0bdc0b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc071c30 0x7f0bdc0c3de0 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 --2- 172.21.15.70:0/2352295949 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0bdc06a460 0x7f0bdc1f1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:38.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 -- 172.21.15.70:0/2352295949 >> 172.21.15.70:0/2352295949 conn(0x7f0bdc06d430 msgr2=0x7f0bdc069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:38.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 -- 172.21.15.70:0/2352295949 shutdown_connections 2023-10-08T07:15:38.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:38.445+0000 7f0be3f83640 1 -- 172.21.15.70:0/2352295949 wait complete. 2023-10-08T07:15:38.464 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T07:15:38.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:15:38.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:15:38.515 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:15:38.567 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:15:38.668 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T07:15:38.669 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:15:38.669 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:15:38.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:15:38.728 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:15:38.728 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:15:38.782 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:15:38.783 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:15:38.783 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:15:38.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:15:38.842 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:15:38.842 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:15:38.853 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:38.851+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:38.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:15:38.903 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T07:15:38.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.127+0000 7f7103fff640 1 Processor -- start 2023-10-08T07:15:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.128+0000 7f7103fff640 1 -- start start 2023-10-08T07:15:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.129+0000 7f7103fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71040d16f0 0x7f71040cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.129+0000 7f7103fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f71040cfa50 0x7f71040ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.129+0000 7f7103fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71040cea50 con 0x7f710415cc60 2023-10-08T07:15:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.129+0000 7f7103fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710415d040 con 0x7f71040cfa50 2023-10-08T07:15:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.129+0000 7f7103fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71040d0650 con 0x7f71040d16f0 2023-10-08T07:15:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.129+0000 7f7102ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f710415cc60 0x7f71040cf340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50974/0 (socket says 172.21.15.70:50974) 2023-10-08T07:15:39.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f7102ffd640 1 -- 172.21.15.70:0/1408775114 learned_addr learned my addr 172.21.15.70:0/1408775114 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:39.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f71027fc640 1 --2- 172.21.15.70:0/1408775114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71040d16f0 0x7f71040cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f71037fe640 1 --2- 172.21.15.70:0/1408775114 >> v2:172.21.15.181:3300/0 conn(0x7f71040cfa50 0x7f71040ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648470943 0 0) 0x7f71040cea50 con 0x7f710415cc60 2023-10-08T07:15:39.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70ec003520 con 0x7f710415cc60 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143740328 0 0) 0x7f70ec003520 con 0x7f710415cc60 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71040d16f0 msgr2=0x7f71040cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f7101ffb640 1 --2- 172.21.15.70:0/1408775114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71040d16f0 0x7f71040cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 >> v2:172.21.15.181:3300/0 conn(0x7f71040cfa50 msgr2=0x7f71040ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f7101ffb640 1 --2- 172.21.15.70:0/1408775114 >> v2:172.21.15.181:3300/0 conn(0x7f71040cfa50 0x7f71040ce510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040d0930 con 0x7f710415cc60 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.130+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f70f40017f0 con 0x7f710415cc60 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 >> v1:172.21.15.70:6789/0 conn(0x7f710415cc60 legacy=0x7f71040cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f7101ffb640 1 --2- 172.21.15.70:0/1408775114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71040d16f0 0x7f70ec003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f7101ffb640 1 --2- 172.21.15.70:0/1408775114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70ec003ef0 0x7f70ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f7101ffb640 1 --2- 172.21.15.70:0/1408775114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710415d040 con 0x7f70ec004d00 2023-10-08T07:15:39.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71040d0650 con 0x7f70ec003ef0 2023-10-08T07:15:39.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec00b980 con 0x7f71040d16f0 2023-10-08T07:15:39.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f71037fe640 1 --2- 172.21.15.70:0/1408775114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f71037fe640 1 -- 172.21.15.70:0/1408775114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71040d16f0 msgr2=0x7f70ec003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:39.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f71037fe640 1 --2- 172.21.15.70:0/1408775114 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f71040d16f0 0x7f70ec003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f71037fe640 1 -- 172.21.15.70:0/1408775114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70ec003ef0 msgr2=0x7f70ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:39.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f71037fe640 1 --2- 172.21.15.70:0/1408775114 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f70ec003ef0 0x7f70ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.131+0000 7f71037fe640 1 -- 172.21.15.70:0/1408775114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec003520 con 0x7f70ec004d00 2023-10-08T07:15:39.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.133+0000 7f71037fe640 1 --2- 172.21.15.70:0/1408775114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f710409c180 tx=0x7f70ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faab30960f5eef8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:39.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.133+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc02cbc0 con 0x7f70ec004d00 2023-10-08T07:15:39.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.133+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc00f040 con 0x7f70ec004d00 2023-10-08T07:15:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.133+0000 7f7101ffb640 1 -- 172.21.15.70:0/1408775114 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc034440 con 0x7f70ec004d00 2023-10-08T07:15:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.133+0000 7f7103fff640 1 -- 172.21.15.70:0/1408775114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70ec004d00 msgr2=0x7f70ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.133+0000 7f7103fff640 1 --2- 172.21.15.70:0/1408775114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7f710409c180 tx=0x7f70ec0057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.133+0000 7f71037fe640 1 -- 172.21.15.70:0/1408775114 reap_dead start 2023-10-08T07:15:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.133+0000 7f7103fff640 1 -- 172.21.15.70:0/1408775114 shutdown_connections 2023-10-08T07:15:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.133+0000 7f7103fff640 1 -- 172.21.15.70:0/1408775114 >> 172.21.15.70:0/1408775114 conn(0x7f710409bcf0 msgr2=0x7f71040d2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.134+0000 7f7103fff640 1 -- 172.21.15.70:0/1408775114 shutdown_connections 2023-10-08T07:15:39.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.134+0000 7f7103fff640 1 -- 172.21.15.70:0/1408775114 wait complete. 2023-10-08T07:15:39.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.135+0000 7f7103fff640 1 Processor -- start 2023-10-08T07:15:39.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.135+0000 7f7103fff640 1 -- start start 2023-10-08T07:15:39.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.136+0000 7f7103fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70ec004d00 0x7f71040cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.136+0000 7f7103fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71040cb210 0x7f71040c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.136+0000 7f7103fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f710414ac60 0x7f710414d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.136+0000 7f7103fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71040c9f80 con 0x7f710414ac60 2023-10-08T07:15:39.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.136+0000 7f7103fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71040ca170 con 0x7f71040cb210 2023-10-08T07:15:39.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.136+0000 7f7103fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71040ca450 con 0x7f70ec004d00 2023-10-08T07:15:39.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.137+0000 7f71037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f710414ac60 0x7f710414d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.137+0000 7f71037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f710414ac60 0x7f710414d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58166/0 (socket says 172.21.15.70:58166) 2023-10-08T07:15:39.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.137+0000 7f71037fe640 1 -- 172.21.15.70:0/1192410243 learned_addr learned my addr 172.21.15.70:0/1192410243 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:39.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.137+0000 7f7102ffd640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70ec004d00 0x7f71040cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.137+0000 7f71037fe640 1 -- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70ec004d00 msgr2=0x7f71040cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.137+0000 7f71027fc640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71040cb210 0x7f71040c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.137+0000 7f71037fe640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70ec004d00 0x7f71040cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.137+0000 7f71037fe640 1 -- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71040cb210 msgr2=0x7f71040c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.137+0000 7f71037fe640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71040cb210 0x7f71040c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.137+0000 7f71037fe640 1 -- 172.21.15.70:0/1192410243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040ca730 con 0x7f710414ac60 2023-10-08T07:15:39.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.138+0000 7f71037fe640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f710414ac60 0x7f710414d640 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f70fc02c5c0 tx=0x7f70fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:39.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.138+0000 7f70ebfff640 1 -- 172.21.15.70:0/1192410243 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc034bc0 con 0x7f710414ac60 2023-10-08T07:15:39.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.138+0000 7f70ebfff640 1 -- 172.21.15.70:0/1192410243 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc034d40 con 0x7f710414ac60 2023-10-08T07:15:39.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.138+0000 7f70ebfff640 1 -- 172.21.15.70:0/1192410243 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc03e680 con 0x7f710414ac60 2023-10-08T07:15:39.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.138+0000 7f7103fff640 1 -- 172.21.15.70:0/1192410243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71040caa10 con 0x7f710414ac60 2023-10-08T07:15:39.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.138+0000 7f7103fff640 1 -- 172.21.15.70:0/1192410243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710414b0e0 con 0x7f710414ac60 2023-10-08T07:15:39.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.139+0000 7f7103fff640 1 -- 172.21.15.70:0/1192410243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71040caf30 con 0x7f710414ac60 2023-10-08T07:15:39.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.140+0000 7f70ebfff640 1 -- 172.21.15.70:0/1192410243 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f70fc03e7e0 con 0x7f710414ac60 2023-10-08T07:15:39.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.141+0000 7f70ebfff640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f70f006b020 0x7f70f006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.141+0000 7f70ebfff640 1 -- 172.21.15.70:0/1192410243 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7f70fc0abd30 con 0x7f710414ac60 2023-10-08T07:15:39.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.142+0000 7f7102ffd640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f70f006b020 0x7f70f006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.143+0000 7f7102ffd640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f70f006b020 0x7f70f006d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f70f4001920 tx=0x7f70f4009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:39.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.146+0000 7f70ebfff640 1 -- 172.21.15.70:0/1192410243 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71040caf30 con 0x7f710414ac60 2023-10-08T07:15:39.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.377+0000 7f7103fff640 1 -- 172.21.15.70:0/1192410243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71040caf30 con 0x7f710414ac60 2023-10-08T07:15:39.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.378+0000 7f70ebfff640 1 -- 172.21.15.70:0/1192410243 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+3705 (secure 0 0 0) 0x7f70fc039e90 con 0x7f710414ac60 2023-10-08T07:15:39.380 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:39.380 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":134,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8078,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/51383904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6836","nonce":51383904},{"type":"v1","addr":"172.21.15.181:6837","nonce":51383904}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":8863,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2045680818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":2045680818},{"type":"v1","addr":"172.21.15.70:6835","nonce":2045680818}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":119},{"gid":8872,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6835/1679415068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":1679415068},{"type":"v1","addr":"172.21.15.181:6835","nonce":1679415068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":120},{"gid":12465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1954064575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1954064575},{"type":"v1","addr":"172.21.15.70:6834","nonce":1954064575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[]} 2023-10-08T07:15:39.380 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 134 2023-10-08T07:15:39.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.381+0000 7f7103fff640 1 -- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f70f006b020 msgr2=0x7f70f006d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.381+0000 7f7103fff640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f70f006b020 0x7f70f006d4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f70f4001920 tx=0x7f70f4009040 comp rx=0 tx=0).stop 2023-10-08T07:15:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.381+0000 7f7103fff640 1 -- 172.21.15.70:0/1192410243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f710414ac60 msgr2=0x7f710414d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.381+0000 7f7103fff640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f710414ac60 0x7f710414d640 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7f70fc02c5c0 tx=0x7f70fc009750 comp rx=0 tx=0).stop 2023-10-08T07:15:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.382+0000 7f7103fff640 1 -- 172.21.15.70:0/1192410243 shutdown_connections 2023-10-08T07:15:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.382+0000 7f7103fff640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f70f006b020 0x7f70f006d4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.382+0000 7f7103fff640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f710414ac60 0x7f710414d640 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.382+0000 7f7103fff640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f71040cb210 0x7f71040c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.382+0000 7f7103fff640 1 --2- 172.21.15.70:0/1192410243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f70ec004d00 0x7f71040cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.382+0000 7f7103fff640 1 -- 172.21.15.70:0/1192410243 >> 172.21.15.70:0/1192410243 conn(0x7f710409bcf0 msgr2=0x7f710415d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.382+0000 7f7103fff640 1 -- 172.21.15.70:0/1192410243 shutdown_connections 2023-10-08T07:15:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.382+0000 7f7103fff640 1 -- 172.21.15.70:0/1192410243 wait complete. 2023-10-08T07:15:39.402 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T07:15:39.402 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T07:15:39.402 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T07:15:39.402 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:15:39.403 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T07:15:39.403 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T07:15:39.403 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T07:15:39.403 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:15:39.403 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T07:15:39.403 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T07:15:39.403 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T07:15:39.404 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:15:39.404 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T07:15:39.404 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T07:15:39.404 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T07:15:39.404 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:15:39.421 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T07:15:39.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T07:15:39.422 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T07:15:39.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T07:15:39.427 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T07:15:39.427 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T07:15:39.428 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T07:15:39.428 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T07:15:39.471 INFO:tasks.ceph.mds.c:Started 2023-10-08T07:15:39.472 DEBUG:teuthology.parallel:result is None 2023-10-08T07:15:39.476 INFO:tasks.ceph.mds.d:Started 2023-10-08T07:15:39.477 DEBUG:teuthology.parallel:result is None 2023-10-08T07:15:39.479 INFO:tasks.ceph.mds.a:Started 2023-10-08T07:15:39.479 DEBUG:teuthology.parallel:result is None 2023-10-08T07:15:39.484 INFO:tasks.ceph.mds.b:Started 2023-10-08T07:15:39.484 DEBUG:teuthology.parallel:result is None 2023-10-08T07:15:39.485 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T07:15:39.643 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T07:15:39.644 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-10-08T07:15:39.647 INFO:tasks.ceph.mds.d.smithi181.stdout:starting mds.d at 2023-10-08T07:15:39.647 INFO:tasks.ceph.mds.b.smithi181.stdout:starting mds.b at 2023-10-08T07:15:39.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.680+0000 7fc9c1f05640 1 Processor -- start 2023-10-08T07:15:39.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.680+0000 7fc9c1f05640 1 -- start start 2023-10-08T07:15:39.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.682+0000 7fc9c1f05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 0x7fc9bc0b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.682+0000 7fc9c1f05640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9bc0b8b00 con 0x7fc9bc15cc70 2023-10-08T07:15:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.682+0000 7fc9c1f05640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9bc0b8540 con 0x7fc9bc0d31a0 2023-10-08T07:15:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.682+0000 7fc9c1f05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0b4fd0 con 0x7fc9bc0b4600 2023-10-08T07:15:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.682+0000 7fc9bb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc9bc15cc70 0x7fc9bc0cdbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50982/0 (socket says 172.21.15.70:50982) 2023-10-08T07:15:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.682+0000 7fc9bb7fe640 1 -- 172.21.15.70:0/1852897013 learned_addr learned my addr 172.21.15.70:0/1852897013 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:39.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.682+0000 7fc9bbfff640 1 --2- 172.21.15.70:0/1852897013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 0x7fc9bc0b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.682+0000 7fc9ba7fc640 1 -- 172.21.15.70:0/1852897013 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532333140 0 0) 0x7fc9bc0b8b00 con 0x7fc9bc15cc70 2023-10-08T07:15:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.682+0000 7fc9ba7fc640 1 -- 172.21.15.70:0/1852897013 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9a4003520 con 0x7fc9bc15cc70 2023-10-08T07:15:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.682+0000 7fc9ba7fc640 1 -- 172.21.15.70:0/1852897013 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280210384 0 0) 0x7fc9bc0b8540 con 0x7fc9bc0d31a0 2023-10-08T07:15:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.683+0000 7fc9ba7fc640 1 -- 172.21.15.70:0/1852897013 --> v1:172.21.15.181:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9a4003840 con 0x7fc9bc0d31a0 2023-10-08T07:15:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.683+0000 7fc9bbfff640 1 -- 172.21.15.70:0/1852897013 >> v1:172.21.15.181:6789/0 conn(0x7fc9bc0d31a0 legacy=0x7fc9bc0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.683+0000 7fc9bbfff640 1 -- 172.21.15.70:0/1852897013 >> v1:172.21.15.70:6789/0 conn(0x7fc9bc15cc70 legacy=0x7fc9bc0cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.683+0000 7fc9bbfff640 1 -- 172.21.15.70:0/1852897013 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc0b8820 con 0x7fc9bc0b4600 2023-10-08T07:15:39.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.684+0000 7fc9bbfff640 1 --2- 172.21.15.70:0/1852897013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 0x7fc9bc0b4a90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009dc0 tx=0x7fc9b402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ae58bb8dcdabdc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.684+0000 7fc9ba7fc640 1 -- 172.21.15.70:0/1852897013 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b402cc40 con 0x7fc9bc0b4600 2023-10-08T07:15:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.684+0000 7fc9ba7fc640 1 -- 172.21.15.70:0/1852897013 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b402cda0 con 0x7fc9bc0b4600 2023-10-08T07:15:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.684+0000 7fc9ba7fc640 1 -- 172.21.15.70:0/1852897013 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4034660 con 0x7fc9bc0b4600 2023-10-08T07:15:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.685+0000 7fc9c1f05640 1 -- 172.21.15.70:0/1852897013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 msgr2=0x7fc9bc0b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.685+0000 7fc9c1f05640 1 --2- 172.21.15.70:0/1852897013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 0x7fc9bc0b4a90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009dc0 tx=0x7fc9b402be90 comp rx=0 tx=0).stop 2023-10-08T07:15:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.685+0000 7fc9c1f05640 1 -- 172.21.15.70:0/1852897013 shutdown_connections 2023-10-08T07:15:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.685+0000 7fc9c1f05640 1 --2- 172.21.15.70:0/1852897013 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 0x7fc9bc0b4a90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.685+0000 7fc9c1f05640 1 -- 172.21.15.70:0/1852897013 >> 172.21.15.70:0/1852897013 conn(0x7fc9bc0718a0 msgr2=0x7fc9bc071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.685+0000 7fc9c1f05640 1 -- 172.21.15.70:0/1852897013 shutdown_connections 2023-10-08T07:15:39.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.685+0000 7fc9c1f05640 1 -- 172.21.15.70:0/1852897013 wait complete. 2023-10-08T07:15:39.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.687+0000 7fc9c1f05640 1 Processor -- start 2023-10-08T07:15:39.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.687+0000 7fc9c1f05640 1 -- start start 2023-10-08T07:15:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9c1f05640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 0x7fc9bc0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9c1f05640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9bc0d31a0 0x7fc9bc146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9c1f05640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9bc15cc70 0x7fc9bc0d2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9c1f05640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0c80a0 con 0x7fc9bc15cc70 2023-10-08T07:15:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9c1f05640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0c71d0 con 0x7fc9bc0d31a0 2023-10-08T07:15:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9c1f05640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0c7420 con 0x7fc9bc0b4600 2023-10-08T07:15:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9bbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9bc15cc70 0x7fc9bc0d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9bbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9bc15cc70 0x7fc9bc0d2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58204/0 (socket says 172.21.15.70:58204) 2023-10-08T07:15:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9bbfff640 1 -- 172.21.15.70:0/408926268 learned_addr learned my addr 172.21.15.70:0/408926268 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9bb7fe640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 0x7fc9bc0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.688+0000 7fc9baffd640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9bc0d31a0 0x7fc9bc146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.689+0000 7fc9bb7fe640 1 -- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9bc0d31a0 msgr2=0x7fc9bc146880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.689+0000 7fc9bb7fe640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9bc0d31a0 0x7fc9bc146880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.689+0000 7fc9bb7fe640 1 -- 172.21.15.70:0/408926268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9bc15cc70 msgr2=0x7fc9bc0d2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:39.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.689+0000 7fc9bb7fe640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9bc15cc70 0x7fc9bc0d2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:39.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.689+0000 7fc9bb7fe640 1 -- 172.21.15.70:0/408926268 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b4009a70 con 0x7fc9bc0b4600 2023-10-08T07:15:39.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.689+0000 7fc9bbfff640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9bc15cc70 0x7fc9bc0d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:15:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.689+0000 7fc9bb7fe640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 0x7fc9bc0c5860 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00c3e0 tx=0x7fc9ac00c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.689+0000 7fc9a3fff640 1 -- 172.21.15.70:0/408926268 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac0074e0 con 0x7fc9bc0b4600 2023-10-08T07:15:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.689+0000 7fc9a3fff640 1 -- 172.21.15.70:0/408926268 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ac007660 con 0x7fc9bc0b4600 2023-10-08T07:15:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.690+0000 7fc9c1f05640 1 -- 172.21.15.70:0/408926268 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc0c7700 con 0x7fc9bc0b4600 2023-10-08T07:15:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.690+0000 7fc9a3fff640 1 -- 172.21.15.70:0/408926268 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac004a20 con 0x7fc9bc0b4600 2023-10-08T07:15:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.690+0000 7fc9c1f05640 1 -- 172.21.15.70:0/408926268 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc0c7cc0 con 0x7fc9bc0b4600 2023-10-08T07:15:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.691+0000 7fc9c1f05640 1 -- 172.21.15.70:0/408926268 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc0b1ee0 con 0x7fc9bc0b4600 2023-10-08T07:15:39.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.692+0000 7fc9a3fff640 1 -- 172.21.15.70:0/408926268 <== mon.2 v2:172.21.15.181:3301/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7fc9ac004b80 con 0x7fc9bc0b4600 2023-10-08T07:15:39.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.693+0000 7fc9a3fff640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc9a806b020 0x7fc9a806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:39.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.693+0000 7fc9a3fff640 1 -- 172.21.15.70:0/408926268 <== mon.2 v2:172.21.15.181:3301/0 5 ==== osd_map(266..266 src has 1..266) v4 ==== 6325+0+0 (secure 0 0 0) 0x7fc9ac089ad0 con 0x7fc9bc0b4600 2023-10-08T07:15:39.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.694+0000 7fc9baffd640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc9a806b020 0x7fc9a806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.694+0000 7fc9baffd640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc9a806b020 0x7fc9a806d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc159d10 tx=0x7fc9b0009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.697+0000 7fc9a3fff640 1 -- 172.21.15.70:0/408926268 <== mon.2 v2:172.21.15.181:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9ac054820 con 0x7fc9bc0b4600 2023-10-08T07:15:39.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:39.909+0000 7fc9c1f05640 1 -- 172.21.15.70:0/408926268 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc9bc04e160 con 0x7fc9bc0b4600 2023-10-08T07:15:40.046 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:40.043+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:41.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.369+0000 7fc9a3fff640 1 -- 172.21.15.70:0/408926268 <== mon.2 v2:172.21.15.181:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v271) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc9ac057e40 con 0x7fc9bc0b4600 2023-10-08T07:15:41.370 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-10-08T07:15:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.373+0000 7fc9c1f05640 1 -- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc9a806b020 msgr2=0x7fc9a806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.373+0000 7fc9c1f05640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc9a806b020 0x7fc9a806d4e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc159d10 tx=0x7fc9b0009040 comp rx=0 tx=0).stop 2023-10-08T07:15:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.373+0000 7fc9c1f05640 1 -- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 msgr2=0x7fc9bc0c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:41.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.373+0000 7fc9c1f05640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 0x7fc9bc0c5860 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00c3e0 tx=0x7fc9ac00c8a0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.373+0000 7fc9c1f05640 1 -- 172.21.15.70:0/408926268 shutdown_connections 2023-10-08T07:15:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.373+0000 7fc9c1f05640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fc9a806b020 0x7fc9a806d4e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.373+0000 7fc9c1f05640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9bc15cc70 0x7fc9bc0d2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.373+0000 7fc9c1f05640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fc9bc0d31a0 0x7fc9bc146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.373+0000 7fc9c1f05640 1 --2- 172.21.15.70:0/408926268 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fc9bc0b4600 0x7fc9bc0c5860 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.373+0000 7fc9c1f05640 1 -- 172.21.15.70:0/408926268 >> 172.21.15.70:0/408926268 conn(0x7fc9bc0718a0 msgr2=0x7fc9bc0b7690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.374+0000 7fc9c1f05640 1 -- 172.21.15.70:0/408926268 shutdown_connections 2023-10-08T07:15:41.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.374+0000 7fc9c1f05640 1 -- 172.21.15.70:0/408926268 wait complete. 2023-10-08T07:15:41.377 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:41.375+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:41.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T07:15:41.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.605+0000 7faadae00640 1 Processor -- start 2023-10-08T07:15:41.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.605+0000 7faadae00640 1 -- start start 2023-10-08T07:15:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.606+0000 7faadae00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faad40cb580 0x7faad4165b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.606+0000 7faadae00640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad415b2b0 con 0x7faad40c90b0 2023-10-08T07:15:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.606+0000 7faadae00640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad415b590 con 0x7faad415ab40 2023-10-08T07:15:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.606+0000 7faadae00640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad415bf30 con 0x7faad40cb580 2023-10-08T07:15:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.607+0000 7faad3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7faad40c90b0 0x7faad40cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50996/0 (socket says 172.21.15.70:50996) 2023-10-08T07:15:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.607+0000 7faad3fff640 1 -- 172.21.15.70:0/4188834507 learned_addr learned my addr 172.21.15.70:0/4188834507 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.607+0000 7faad939d640 1 --2- 172.21.15.70:0/4188834507 >> v2:172.21.15.70:3300/0 conn(0x7faad40cb580 0x7faad4165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.607+0000 7faad939d640 1 -- 172.21.15.70:0/4188834507 >> v1:172.21.15.181:6789/0 conn(0x7faad415ab40 legacy=0x7faad40ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.607+0000 7faad939d640 1 -- 172.21.15.70:0/4188834507 >> v1:172.21.15.70:6789/0 conn(0x7faad40c90b0 legacy=0x7faad40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.607+0000 7faad939d640 1 -- 172.21.15.70:0/4188834507 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad415b870 con 0x7faad40cb580 2023-10-08T07:15:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.607+0000 7faad37fe640 1 -- 172.21.15.70:0/4188834507 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588168808 0 0) 0x7faad415b2b0 con 0x7faad40c90b0 2023-10-08T07:15:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.608+0000 7faad939d640 1 --2- 172.21.15.70:0/4188834507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad40cb580 0x7faad4165b20 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7faacc009d90 tx=0x7faacc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512648760c26ae0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.608+0000 7faad37fe640 1 -- 172.21.15.70:0/4188834507 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc02cc60 con 0x7faad40cb580 2023-10-08T07:15:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.609+0000 7faad37fe640 1 -- 172.21.15.70:0/4188834507 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faacc02cdc0 con 0x7faad40cb580 2023-10-08T07:15:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.609+0000 7faad37fe640 1 -- 172.21.15.70:0/4188834507 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc034660 con 0x7faad40cb580 2023-10-08T07:15:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.609+0000 7faadae00640 1 -- 172.21.15.70:0/4188834507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad40cb580 msgr2=0x7faad4165b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.609+0000 7faadae00640 1 --2- 172.21.15.70:0/4188834507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad40cb580 0x7faad4165b20 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7faacc009d90 tx=0x7faacc02beb0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.609+0000 7faadae00640 1 -- 172.21.15.70:0/4188834507 shutdown_connections 2023-10-08T07:15:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.609+0000 7faadae00640 1 --2- 172.21.15.70:0/4188834507 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad40cb580 0x7faad4165b20 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.609+0000 7faadae00640 1 -- 172.21.15.70:0/4188834507 >> 172.21.15.70:0/4188834507 conn(0x7faad406a380 msgr2=0x7faad40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.609+0000 7faadae00640 1 -- 172.21.15.70:0/4188834507 shutdown_connections 2023-10-08T07:15:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.609+0000 7faadae00640 1 -- 172.21.15.70:0/4188834507 wait complete. 2023-10-08T07:15:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.611+0000 7faadae00640 1 Processor -- start 2023-10-08T07:15:41.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.611+0000 7faadae00640 1 -- start start 2023-10-08T07:15:41.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.612+0000 7faadae00640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faad40c90b0 0x7faad414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:41.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.612+0000 7faadae00640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faad40cb580 0x7faad40ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:41.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.612+0000 7faadae00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad415ab40 0x7faad4151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:41.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.612+0000 7faadae00640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad415f7a0 con 0x7faad415ab40 2023-10-08T07:15:41.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.612+0000 7faadae00640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad415d2b0 con 0x7faad40c90b0 2023-10-08T07:15:41.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.612+0000 7faadae00640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad4158130 con 0x7faad40cb580 2023-10-08T07:15:41.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.613+0000 7faad939d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad415ab40 0x7faad4151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:41.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.613+0000 7faad939d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad415ab40 0x7faad4151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58226/0 (socket says 172.21.15.70:58226) 2023-10-08T07:15:41.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.613+0000 7faad939d640 1 -- 172.21.15.70:0/4282449423 learned_addr learned my addr 172.21.15.70:0/4282449423 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:41.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.613+0000 7faad8b9c640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faad40c90b0 0x7faad414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:41.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.613+0000 7faad939d640 1 -- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faad40cb580 msgr2=0x7faad40ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:41.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.613+0000 7faad939d640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faad40cb580 0x7faad40ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.613+0000 7faad939d640 1 -- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faad40c90b0 msgr2=0x7faad414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:41.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.613+0000 7faad939d640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faad40c90b0 0x7faad414fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.613+0000 7faad939d640 1 -- 172.21.15.70:0/4282449423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faacc009a70 con 0x7faad415ab40 2023-10-08T07:15:41.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.613+0000 7faad939d640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad415ab40 0x7faad4151b50 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7faacc02c4c0 tx=0x7faacc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:41.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.614+0000 7faad17fa640 1 -- 172.21.15.70:0/4282449423 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc03f420 con 0x7faad415ab40 2023-10-08T07:15:41.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.614+0000 7faad17fa640 1 -- 172.21.15.70:0/4282449423 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faacc03f580 con 0x7faad415ab40 2023-10-08T07:15:41.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.614+0000 7faadae00640 1 -- 172.21.15.70:0/4282449423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad415ca10 con 0x7faad415ab40 2023-10-08T07:15:41.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.614+0000 7faad17fa640 1 -- 172.21.15.70:0/4282449423 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc03f910 con 0x7faad415ab40 2023-10-08T07:15:41.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.614+0000 7faadae00640 1 -- 172.21.15.70:0/4282449423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad415cfd0 con 0x7faad415ab40 2023-10-08T07:15:41.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.615+0000 7faadae00640 1 -- 172.21.15.70:0/4282449423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad4067490 con 0x7faad415ab40 2023-10-08T07:15:41.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.616+0000 7faad17fa640 1 -- 172.21.15.70:0/4282449423 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7faacc03fa70 con 0x7faad415ab40 2023-10-08T07:15:41.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.617+0000 7faad17fa640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faab806b020 0x7faab806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:41.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.617+0000 7faad17fa640 1 -- 172.21.15.70:0/4282449423 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 5293+0+0 (secure 0 0 0) 0x7faacc0ac260 con 0x7faad415ab40 2023-10-08T07:15:41.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.617+0000 7faad8b9c640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faab806b020 0x7faab806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:41.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.619+0000 7faad8b9c640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faab806b020 0x7faab806d4e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faac8006fd0 tx=0x7faac8008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:41.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.621+0000 7faad17fa640 1 -- 172.21.15.70:0/4282449423 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faacc0772b0 con 0x7faad415ab40 2023-10-08T07:15:41.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.895+0000 7faadae00640 1 -- 172.21.15.70:0/4282449423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7faad40b0ef0 con 0x7faad415ab40 2023-10-08T07:15:41.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.896+0000 7faad17fa640 1 -- 172.21.15.70:0/4282449423 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v54) v1 ==== 80+0+5304 (secure 0 0 0) 0x7faacc07a8d0 con 0x7faad415ab40 2023-10-08T07:15:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:41.938 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJl4zLfIBAAVTMx/JtOukVSzhtpiQuihA==", 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlhj+HBhAAAMOyCnoXyoLaevgybpG6NQ==", 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlMKNMMhAAxcZgG4yq21sJVklHVOZLDw==", 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-10-08T07:15:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlFmiLFxAAYxq7Rhq30Wem/GsDcfeIlQ==", 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCPPyJlvdVuNRAAqLt2+3ol98PSYjQDKuZT1A==", 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCTPyJl1jYjAxAAsukK7BhjhAnemqKOlHpT5A==", 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCWPyJl9ZYLDBAAxfLL5ejNgQeAOrShPU2lRg==", 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCZPyJlbf5DFRAA9ZdYHaIqs54cWG80dcWuIg==", 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCgPyJlNLNHGRAAI6lXQ1xWruEsG7s/dgJjlw==", 2023-10-08T07:15:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCjPyJlai+eIxAApIdMWsh3WGU9zJchiTFYaw==", 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:15:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:15:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-10-08T07:15:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCmPyJleUm1LxAAG1qPZHdpD3vZFubWFtIWdA==", 2023-10-08T07:15:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCpPyJlQyv0ORAACaJWnG3gFEa+b+ivN+7Sxg==", 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-10-08T07:15:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA==", 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlRffsOBAAbti6wBIXbrJrzExnwSPMUg==", 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-10-08T07:15:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNPyJlNIeELhAA/TT2+fc8Bbl3Q8NA0s4k5Q==", 2023-10-08T07:15:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:15:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:15:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:15:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:15:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCOPyJlOtfBBxAA0jKM6wWYht7bJ8n+ndynvg==", 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCLPyJlDPOgFRAAGLd7985k2htLDRWzAG/64A==", 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlLLbYFBAAtHmAvAHAuZgshqFXUv9OiA==", 2023-10-08T07:15:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:15:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T07:15:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-10-08T07:15:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-10-08T07:15:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMPyJlNnz6BxAAjrf8zcfBQvc+W3xVAMq45g==", 2023-10-08T07:15:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-10-08T07:15:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-10-08T07:15:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-10-08T07:15:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-10-08T07:15:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-10-08T07:15:41.956 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:15:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.898+0000 7faadae00640 1 -- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faab806b020 msgr2=0x7faab806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.898+0000 7faadae00640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faab806b020 0x7faab806d4e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faac8006fd0 tx=0x7faac8008040 comp rx=0 tx=0).stop 2023-10-08T07:15:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.898+0000 7faadae00640 1 -- 172.21.15.70:0/4282449423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad415ab40 msgr2=0x7faad4151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.898+0000 7faadae00640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad415ab40 0x7faad4151b50 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7faacc02c4c0 tx=0x7faacc004a40 comp rx=0 tx=0).stop 2023-10-08T07:15:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.898+0000 7faadae00640 1 -- 172.21.15.70:0/4282449423 shutdown_connections 2023-10-08T07:15:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.898+0000 7faadae00640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7faab806b020 0x7faab806d4e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.898+0000 7faadae00640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faad415ab40 0x7faad4151b50 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.898+0000 7faadae00640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7faad40cb580 0x7faad40ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.899+0000 7faadae00640 1 --2- 172.21.15.70:0/4282449423 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7faad40c90b0 0x7faad414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.899+0000 7faadae00640 1 -- 172.21.15.70:0/4282449423 >> 172.21.15.70:0/4282449423 conn(0x7faad406a380 msgr2=0x7faad4070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.899+0000 7faadae00640 1 -- 172.21.15.70:0/4282449423 shutdown_connections 2023-10-08T07:15:41.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:41.899+0000 7faadae00640 1 -- 172.21.15.70:0/4282449423 wait complete. 2023-10-08T07:15:41.959 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:42.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.185+0000 7f4074cc2640 1 Processor -- start 2023-10-08T07:15:42.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.185+0000 7f4074cc2640 1 -- start start 2023-10-08T07:15:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.187+0000 7f4074cc2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f40700b07d0 0x7f40701648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.187+0000 7f4074cc2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40700cacf0 con 0x7f40700cc210 2023-10-08T07:15:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.187+0000 7f4074cc2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40700ca3c0 con 0x7f40701599d0 2023-10-08T07:15:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.187+0000 7f4074cc2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40700b10f0 con 0x7f40700b07d0 2023-10-08T07:15:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.187+0000 7f406ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f40700b07d0 0x7f40701648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.187+0000 7f406effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f40700cc210 0x7f40700b00c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51012/0 (socket says 172.21.15.70:51012) 2023-10-08T07:15:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.187+0000 7f406ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f40700b07d0 0x7f40701648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58234/0 (socket says 172.21.15.70:58234) 2023-10-08T07:15:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.187+0000 7f406ffff640 1 -- 172.21.15.70:0/2157474922 learned_addr learned my addr 172.21.15.70:0/2157474922 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.188+0000 7f406ffff640 1 -- 172.21.15.70:0/2157474922 >> v1:172.21.15.181:6789/0 conn(0x7f40701599d0 legacy=0x7f40700ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.188+0000 7f406ffff640 1 -- 172.21.15.70:0/2157474922 >> v1:172.21.15.70:6789/0 conn(0x7f40700cc210 legacy=0x7f40700b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.188+0000 7f406ffff640 1 -- 172.21.15.70:0/2157474922 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40700ca6a0 con 0x7f40700b07d0 2023-10-08T07:15:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.188+0000 7f406e7fc640 1 -- 172.21.15.70:0/2157474922 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972838927 0 0) 0x7f40700cacf0 con 0x7f40700cc210 2023-10-08T07:15:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.189+0000 7f406ffff640 1 --2- 172.21.15.70:0/2157474922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700b07d0 0x7f40701648d0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f4068009e50 tx=0x7f406802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afeb4b264195981 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.189+0000 7f406e7fc640 1 -- 172.21.15.70:0/2157474922 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406802cb00 con 0x7f40700b07d0 2023-10-08T07:15:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.189+0000 7f406e7fc640 1 -- 172.21.15.70:0/2157474922 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406802cc60 con 0x7f40700b07d0 2023-10-08T07:15:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.189+0000 7f406e7fc640 1 -- 172.21.15.70:0/2157474922 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40680343f0 con 0x7f40700b07d0 2023-10-08T07:15:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.190+0000 7f4074cc2640 1 -- 172.21.15.70:0/2157474922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700b07d0 msgr2=0x7f40701648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.190+0000 7f4074cc2640 1 --2- 172.21.15.70:0/2157474922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700b07d0 0x7f40701648d0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7f4068009e50 tx=0x7f406802bf60 comp rx=0 tx=0).stop 2023-10-08T07:15:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.190+0000 7f4074cc2640 1 -- 172.21.15.70:0/2157474922 shutdown_connections 2023-10-08T07:15:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.190+0000 7f4074cc2640 1 --2- 172.21.15.70:0/2157474922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700b07d0 0x7f40701648d0 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.190+0000 7f4074cc2640 1 -- 172.21.15.70:0/2157474922 >> 172.21.15.70:0/2157474922 conn(0x7f40700cb0c0 msgr2=0x7f40700cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.190+0000 7f4074cc2640 1 -- 172.21.15.70:0/2157474922 shutdown_connections 2023-10-08T07:15:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.190+0000 7f4074cc2640 1 -- 172.21.15.70:0/2157474922 wait complete. 2023-10-08T07:15:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.192+0000 7f4074cc2640 1 Processor -- start 2023-10-08T07:15:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.192+0000 7f4074cc2640 1 -- start start 2023-10-08T07:15:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f4074cc2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40700b07d0 0x7f4070165820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f4074cc2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700cc210 0x7f40700c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f4074cc2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40701599d0 0x7f40700c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f4074cc2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40700b23b0 con 0x7f40700cc210 2023-10-08T07:15:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f4074cc2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4070165d60 con 0x7f40701599d0 2023-10-08T07:15:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f4074cc2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4070166000 con 0x7f40700b07d0 2023-10-08T07:15:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f406effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700cc210 0x7f40700c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f406effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700cc210 0x7f40700c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58238/0 (socket says 172.21.15.70:58238) 2023-10-08T07:15:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f406effd640 1 -- 172.21.15.70:0/116615957 learned_addr learned my addr 172.21.15.70:0/116615957 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:42.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f406f7fe640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40700b07d0 0x7f4070165820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.193+0000 7f406effd640 1 -- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40700b07d0 msgr2=0x7f4070165820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f406ffff640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40701599d0 0x7f40700c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f406effd640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40700b07d0 0x7f4070165820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f406effd640 1 -- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40701599d0 msgr2=0x7f40700c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f406effd640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40701599d0 0x7f40700c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f406effd640 1 -- 172.21.15.70:0/116615957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068009b30 con 0x7f40700cc210 2023-10-08T07:15:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f406effd640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700cc210 0x7f40700c7720 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f406400c4d0 tx=0x7f406400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f4057fff640 1 -- 172.21.15.70:0/116615957 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40640074e0 con 0x7f40700cc210 2023-10-08T07:15:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f4057fff640 1 -- 172.21.15.70:0/116615957 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4064007660 con 0x7f40700cc210 2023-10-08T07:15:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f4057fff640 1 -- 172.21.15.70:0/116615957 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064004a20 con 0x7f40700cc210 2023-10-08T07:15:42.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f4074cc2640 1 -- 172.21.15.70:0/116615957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40700b2600 con 0x7f40700cc210 2023-10-08T07:15:42.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.194+0000 7f4074cc2640 1 -- 172.21.15.70:0/116615957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40700b2bc0 con 0x7f40700cc210 2023-10-08T07:15:42.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.196+0000 7f4074cc2640 1 -- 172.21.15.70:0/116615957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40700b2de0 con 0x7f40700cc210 2023-10-08T07:15:42.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.196+0000 7f4057fff640 1 -- 172.21.15.70:0/116615957 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f4064004b80 con 0x7f40700cc210 2023-10-08T07:15:42.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.198+0000 7f4057fff640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f405c06b020 0x7f405c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.198+0000 7f4057fff640 1 -- 172.21.15.70:0/116615957 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f4064088990 con 0x7f40700cc210 2023-10-08T07:15:42.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.199+0000 7f406f7fe640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f405c06b020 0x7f405c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.200+0000 7f406f7fe640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f405c06b020 0x7f405c06d4e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4060006fd0 tx=0x7f4060008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:42.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.202+0000 7f4057fff640 1 -- 172.21.15.70:0/116615957 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4064053950 con 0x7f40700cc210 2023-10-08T07:15:42.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.403+0000 7f4074cc2640 1 -- 172.21.15.70:0/116615957 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f407004e160 con 0x7f40700cc210 2023-10-08T07:15:42.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.406+0000 7f4057fff640 1 -- 172.21.15.70:0/116615957 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v271) v1 ==== 74+0+13246 (secure 0 0 0) 0x7f4064018020 con 0x7f40700cc210 2023-10-08T07:15:42.407 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:42.407 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":271,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:41.357981+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":256,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:42.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.408+0000 7f4074cc2640 1 -- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f405c06b020 msgr2=0x7f405c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.408+0000 7f4074cc2640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f405c06b020 0x7f405c06d4e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4060006fd0 tx=0x7f4060008040 comp rx=0 tx=0).stop 2023-10-08T07:15:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.408+0000 7f4074cc2640 1 -- 172.21.15.70:0/116615957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700cc210 msgr2=0x7f40700c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.408+0000 7f4074cc2640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700cc210 0x7f40700c7720 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7f406400c4d0 tx=0x7f406400c990 comp rx=0 tx=0).stop 2023-10-08T07:15:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.409+0000 7f4074cc2640 1 -- 172.21.15.70:0/116615957 shutdown_connections 2023-10-08T07:15:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.409+0000 7f4074cc2640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f405c06b020 0x7f405c06d4e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.409+0000 7f4074cc2640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f40701599d0 0x7f40700c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.409+0000 7f4074cc2640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40700cc210 0x7f40700c7720 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.409+0000 7f4074cc2640 1 --2- 172.21.15.70:0/116615957 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f40700b07d0 0x7f4070165820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.409+0000 7f4074cc2640 1 -- 172.21.15.70:0/116615957 >> 172.21.15.70:0/116615957 conn(0x7f40700cb0c0 msgr2=0x7f407015d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.409+0000 7f4074cc2640 1 -- 172.21.15.70:0/116615957 shutdown_connections 2023-10-08T07:15:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.409+0000 7f4074cc2640 1 -- 172.21.15.70:0/116615957 wait complete. 2023-10-08T07:15:42.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:15:42.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:15:42.598 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:42.596+0000 7f8be0b7a640 -1 client.9118: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.9019:2701986a-8cbc-4288-8315-7610633edcdf:v1:172.21.15.181:0/1513601554] 2023-10-08T07:15:42.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.647+0000 7f12d07fc640 1 Processor -- start 2023-10-08T07:15:42.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.648+0000 7f12d07fc640 1 -- start start 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.649+0000 7f12d07fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f12c815e370 0x7f12c806cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.649+0000 7f12d07fc640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f12c80b15c0 0x7f12c806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.649+0000 7f12d07fc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c80ca340 con 0x7f12c806db50 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.649+0000 7f12d07fc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c80c9730 con 0x7f12c815e370 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.649+0000 7f12d07fc640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c80c9a10 con 0x7f12c80b15c0 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.650+0000 7f12ce598640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f12c815e370 0x7f12c806cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.650+0000 7f12ce598640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f12c815e370 0x7f12c806cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58252/0 (socket says 172.21.15.70:58252) 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.650+0000 7f12ce598640 1 -- 172.21.15.70:0/2552514815 learned_addr learned my addr 172.21.15.70:0/2552514815 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.650+0000 7f12ce598640 1 -- 172.21.15.70:0/2552514815 >> v2:172.21.15.181:3300/0 conn(0x7f12c80b15c0 msgr2=0x7f12c806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.650+0000 7f12ced99640 1 --2- 172.21.15.70:0/2552514815 >> v2:172.21.15.181:3300/0 conn(0x7f12c80b15c0 0x7f12c806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.650+0000 7f12ce598640 1 --2- 172.21.15.70:0/2552514815 >> v2:172.21.15.181:3300/0 conn(0x7f12c80b15c0 0x7f12c806d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.650+0000 7f12ce598640 1 -- 172.21.15.70:0/2552514815 >> v1:172.21.15.70:6789/0 conn(0x7f12c806db50 legacy=0x7f12c8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.650+0000 7f12ce598640 1 -- 172.21.15.70:0/2552514815 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c80c9cf0 con 0x7f12c815e370 2023-10-08T07:15:42.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.650+0000 7f12cd596640 1 -- 172.21.15.70:0/2552514815 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550235120 0 0) 0x7f12c80ca340 con 0x7f12c806db50 2023-10-08T07:15:42.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.651+0000 7f12ce598640 1 --2- 172.21.15.70:0/2552514815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 0x7f12c806cc60 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f12c806c4a0 tx=0x7f12c402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93431319e464e27 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:42.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12cd596640 1 -- 172.21.15.70:0/2552514815 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c40317e0 con 0x7f12c815e370 2023-10-08T07:15:42.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12cd596640 1 -- 172.21.15.70:0/2552514815 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c4031940 con 0x7f12c815e370 2023-10-08T07:15:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12cd596640 1 -- 172.21.15.70:0/2552514815 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4031cd0 con 0x7f12c815e370 2023-10-08T07:15:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12d07fc640 1 -- 172.21.15.70:0/2552514815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 msgr2=0x7f12c806cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12d07fc640 1 --2- 172.21.15.70:0/2552514815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 0x7f12c806cc60 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f12c806c4a0 tx=0x7f12c402b880 comp rx=0 tx=0).stop 2023-10-08T07:15:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12d07fc640 1 -- 172.21.15.70:0/2552514815 shutdown_connections 2023-10-08T07:15:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12d07fc640 1 --2- 172.21.15.70:0/2552514815 >> v2:172.21.15.181:3300/0 conn(0x7f12c80b15c0 0x7f12c806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12d07fc640 1 --2- 172.21.15.70:0/2552514815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 0x7f12c806cc60 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12d07fc640 1 -- 172.21.15.70:0/2552514815 >> 172.21.15.70:0/2552514815 conn(0x7f12c80b0aa0 msgr2=0x7f12c80b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12d07fc640 1 -- 172.21.15.70:0/2552514815 shutdown_connections 2023-10-08T07:15:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.652+0000 7f12d07fc640 1 -- 172.21.15.70:0/2552514815 wait complete. 2023-10-08T07:15:42.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.654+0000 7f12d07fc640 1 Processor -- start 2023-10-08T07:15:42.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.654+0000 7f12d07fc640 1 -- start start 2023-10-08T07:15:42.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.655+0000 7f12d07fc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c806db50 0x7f12c814cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.655+0000 7f12d07fc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12c80b15c0 0x7f12c80cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.655+0000 7f12d07fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 0x7f12c815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.655+0000 7f12d07fc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c80cda60 con 0x7f12c815e370 2023-10-08T07:15:42.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.655+0000 7f12d07fc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c814d500 con 0x7f12c806db50 2023-10-08T07:15:42.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.655+0000 7f12d07fc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c80ce7f0 con 0x7f12c80b15c0 2023-10-08T07:15:42.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12ced99640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 0x7f12c815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12ced99640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 0x7f12c815a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58258/0 (socket says 172.21.15.70:58258) 2023-10-08T07:15:42.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12ced99640 1 -- 172.21.15.70:0/1898401505 learned_addr learned my addr 172.21.15.70:0/1898401505 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:42.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12ce598640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c806db50 0x7f12c814cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12cdd97640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12c80b15c0 0x7f12c80cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12ced99640 1 -- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12c80b15c0 msgr2=0x7f12c80cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12ced99640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12c80b15c0 0x7f12c80cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12ced99640 1 -- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c806db50 msgr2=0x7f12c814cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12ced99640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c806db50 0x7f12c814cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12ced99640 1 -- 172.21.15.70:0/1898401505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c40094d0 con 0x7f12c815e370 2023-10-08T07:15:42.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.656+0000 7f12ced99640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 0x7f12c815a6a0 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f12c000cd70 tx=0x7f12c000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:42.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.657+0000 7f12beffd640 1 -- 172.21.15.70:0/1898401505 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0003b90 con 0x7f12c815e370 2023-10-08T07:15:42.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.657+0000 7f12beffd640 1 -- 172.21.15.70:0/1898401505 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c0003cf0 con 0x7f12c815e370 2023-10-08T07:15:42.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.657+0000 7f12beffd640 1 -- 172.21.15.70:0/1898401505 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0004240 con 0x7f12c815e370 2023-10-08T07:15:42.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.657+0000 7f12d07fc640 1 -- 172.21.15.70:0/1898401505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c815c8c0 con 0x7f12c815e370 2023-10-08T07:15:42.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.657+0000 7f12d07fc640 1 -- 172.21.15.70:0/1898401505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c815ce80 con 0x7f12c815e370 2023-10-08T07:15:42.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.658+0000 7f12d07fc640 1 -- 172.21.15.70:0/1898401505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c815d0a0 con 0x7f12c815e370 2023-10-08T07:15:42.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.660+0000 7f12beffd640 1 -- 172.21.15.70:0/1898401505 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f12c00043a0 con 0x7f12c815e370 2023-10-08T07:15:42.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.661+0000 7f12beffd640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f12ac06b050 0x7f12ac06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:42.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.661+0000 7f12beffd640 1 -- 172.21.15.70:0/1898401505 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f12c0012070 con 0x7f12c815e370 2023-10-08T07:15:42.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.662+0000 7f12ce598640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f12ac06b050 0x7f12ac06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:42.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.663+0000 7f12ce598640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f12ac06b050 0x7f12ac06d510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12c4036000 tx=0x7f12c403d040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:42.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.666+0000 7f12beffd640 1 -- 172.21.15.70:0/1898401505 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f12c0055120 con 0x7f12c815e370 2023-10-08T07:15:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.862+0000 7f12d07fc640 1 -- 172.21.15.70:0/1898401505 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f12c8071910 con 0x7f12c815e370 2023-10-08T07:15:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.863+0000 7f12beffd640 1 -- 172.21.15.70:0/1898401505 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v271) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12c0058740 con 0x7f12c815e370 2023-10-08T07:15:42.864 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:15:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.865+0000 7f12d07fc640 1 -- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f12ac06b050 msgr2=0x7f12ac06d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.865+0000 7f12d07fc640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f12ac06b050 0x7f12ac06d510 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12c4036000 tx=0x7f12c403d040 comp rx=0 tx=0).stop 2023-10-08T07:15:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.865+0000 7f12d07fc640 1 -- 172.21.15.70:0/1898401505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 msgr2=0x7f12c815a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.865+0000 7f12d07fc640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 0x7f12c815a6a0 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f12c000cd70 tx=0x7f12c000a660 comp rx=0 tx=0).stop 2023-10-08T07:15:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.866+0000 7f12d07fc640 1 -- 172.21.15.70:0/1898401505 shutdown_connections 2023-10-08T07:15:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.866+0000 7f12d07fc640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f12ac06b050 0x7f12ac06d510 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.866+0000 7f12d07fc640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12c815e370 0x7f12c815a6a0 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.866+0000 7f12d07fc640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f12c80b15c0 0x7f12c80cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.866+0000 7f12d07fc640 1 --2- 172.21.15.70:0/1898401505 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f12c806db50 0x7f12c814cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.866+0000 7f12d07fc640 1 -- 172.21.15.70:0/1898401505 >> 172.21.15.70:0/1898401505 conn(0x7f12c80b0aa0 msgr2=0x7f12c80b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.866+0000 7f12d07fc640 1 -- 172.21.15.70:0/1898401505 shutdown_connections 2023-10-08T07:15:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:42.866+0000 7f12d07fc640 1 -- 172.21.15.70:0/1898401505 wait complete. 2023-10-08T07:15:42.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T07:15:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.087+0000 7f043456e640 1 Processor -- start 2023-10-08T07:15:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.088+0000 7f043456e640 1 -- start start 2023-10-08T07:15:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.089+0000 7f043456e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f042c069d50 0x7f042c06a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.089+0000 7f043456e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f042c0cb880 0x7f042c0b1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.089+0000 7f043456e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f042c0ce550 con 0x7f042c0cbce0 2023-10-08T07:15:43.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.089+0000 7f043456e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042c1609d0 con 0x7f042c0cb880 2023-10-08T07:15:43.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.089+0000 7f043456e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f042c0cdcf0 con 0x7f042c069d50 2023-10-08T07:15:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.089+0000 7f0431b09640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f042c0cb880 0x7f042c0b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.089+0000 7f0431b09640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f042c0cb880 0x7f042c0b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58268/0 (socket says 172.21.15.70:58268) 2023-10-08T07:15:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.089+0000 7f0431b09640 1 -- 172.21.15.70:0/179552158 learned_addr learned my addr 172.21.15.70:0/179552158 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.089+0000 7f043230a640 1 --2- 172.21.15.70:0/179552158 >> v2:172.21.15.181:3300/0 conn(0x7f042c069d50 0x7f042c06a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.090+0000 7f0431b09640 1 -- 172.21.15.70:0/179552158 >> v2:172.21.15.181:3300/0 conn(0x7f042c069d50 msgr2=0x7f042c06a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.090+0000 7f0431b09640 1 --2- 172.21.15.70:0/179552158 >> v2:172.21.15.181:3300/0 conn(0x7f042c069d50 0x7f042c06a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.090+0000 7f0431b09640 1 -- 172.21.15.70:0/179552158 >> v1:172.21.15.70:6789/0 conn(0x7f042c0cbce0 legacy=0x7f042c160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.090+0000 7f0431b09640 1 -- 172.21.15.70:0/179552158 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c06a670 con 0x7f042c0cb880 2023-10-08T07:15:43.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.091+0000 7f0431b09640 1 --2- 172.21.15.70:0/179552158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c0cb880 0x7f042c0b1310 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7f042c0d0990 tx=0x7f042002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db977badfe3a9f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:43.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.091+0000 7f0431308640 1 -- 172.21.15.70:0/179552158 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420030860 con 0x7f042c0cb880 2023-10-08T07:15:43.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.091+0000 7f0431308640 1 -- 172.21.15.70:0/179552158 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04200309c0 con 0x7f042c0cb880 2023-10-08T07:15:43.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.091+0000 7f0431308640 1 -- 172.21.15.70:0/179552158 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420030d50 con 0x7f042c0cb880 2023-10-08T07:15:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.092+0000 7f043456e640 1 -- 172.21.15.70:0/179552158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c0cb880 msgr2=0x7f042c0b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.092+0000 7f043456e640 1 --2- 172.21.15.70:0/179552158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c0cb880 0x7f042c0b1310 secure :-1 s=READY pgs=2867 cs=0 l=1 rev1=1 crypto rx=0x7f042c0d0990 tx=0x7f042002b9a0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.092+0000 7f043456e640 1 -- 172.21.15.70:0/179552158 shutdown_connections 2023-10-08T07:15:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.092+0000 7f043456e640 1 --2- 172.21.15.70:0/179552158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c0cb880 0x7f042c0b1310 unknown :-1 s=CLOSED pgs=2867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.092+0000 7f043456e640 1 --2- 172.21.15.70:0/179552158 >> v2:172.21.15.181:3300/0 conn(0x7f042c069d50 0x7f042c06a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.092+0000 7f043456e640 1 -- 172.21.15.70:0/179552158 >> 172.21.15.70:0/179552158 conn(0x7f042c06d450 msgr2=0x7f042c06d860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.092+0000 7f043456e640 1 -- 172.21.15.70:0/179552158 shutdown_connections 2023-10-08T07:15:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.092+0000 7f043456e640 1 -- 172.21.15.70:0/179552158 wait complete. 2023-10-08T07:15:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.094+0000 7f043456e640 1 Processor -- start 2023-10-08T07:15:43.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.094+0000 7f043456e640 1 -- start start 2023-10-08T07:15:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.095+0000 7f043456e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c069d50 0x7f042c0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.095+0000 7f043456e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c0cb880 0x7f042c15b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.095+0000 7f043456e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c0cbce0 0x7f042c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.095+0000 7f043456e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c0c60f0 con 0x7f042c069d50 2023-10-08T07:15:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.095+0000 7f043456e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c0b4a40 con 0x7f042c0cbce0 2023-10-08T07:15:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.095+0000 7f043456e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042c0b24b0 con 0x7f042c0cb880 2023-10-08T07:15:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.095+0000 7f043230a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c069d50 0x7f042c0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.095+0000 7f043230a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c069d50 0x7f042c0b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58274/0 (socket says 172.21.15.70:58274) 2023-10-08T07:15:43.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.095+0000 7f043230a640 1 -- 172.21.15.70:0/1326992429 learned_addr learned my addr 172.21.15.70:0/1326992429 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:43.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f0431b09640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c0cb880 0x7f042c15b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f043230a640 1 -- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c0cb880 msgr2=0x7f042c15b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:43.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f043230a640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c0cb880 0x7f042c15b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f043230a640 1 -- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c0cbce0 msgr2=0x7f042c15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:43.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f0432b0b640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c0cbce0 0x7f042c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f043230a640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c0cbce0 0x7f042c15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f043230a640 1 -- 172.21.15.70:0/1326992429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04200094d0 con 0x7f042c069d50 2023-10-08T07:15:43.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f043230a640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c069d50 0x7f042c0b3270 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f041c007e40 tx=0x7f041c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:43.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f042affd640 1 -- 172.21.15.70:0/1326992429 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c014620 con 0x7f042c069d50 2023-10-08T07:15:43.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f042affd640 1 -- 172.21.15.70:0/1326992429 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041c00ee90 con 0x7f042c069d50 2023-10-08T07:15:43.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f042affd640 1 -- 172.21.15.70:0/1326992429 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c0149c0 con 0x7f042c069d50 2023-10-08T07:15:43.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.096+0000 7f043456e640 1 -- 172.21.15.70:0/1326992429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c0b2790 con 0x7f042c069d50 2023-10-08T07:15:43.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.097+0000 7f043456e640 1 -- 172.21.15.70:0/1326992429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c0b2d50 con 0x7f042c069d50 2023-10-08T07:15:43.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.098+0000 7f043456e640 1 -- 172.21.15.70:0/1326992429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042c0b2ee0 con 0x7f042c069d50 2023-10-08T07:15:43.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.099+0000 7f042affd640 1 -- 172.21.15.70:0/1326992429 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f041c014b20 con 0x7f042c069d50 2023-10-08T07:15:43.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.100+0000 7f042affd640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f041806b020 0x7f041806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.100+0000 7f042affd640 1 -- 172.21.15.70:0/1326992429 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(271..271 src has 1..271) v4 ==== 5293+0+0 (secure 0 0 0) 0x7f041c08a1b0 con 0x7f042c069d50 2023-10-08T07:15:43.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.101+0000 7f0431b09640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f041806b020 0x7f041806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.102+0000 7f0431b09640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f041806b020 0x7f041806d4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0420036000 tx=0x7f042003b040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:43.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.104+0000 7f042affd640 1 -- 172.21.15.70:0/1326992429 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f041c0550e0 con 0x7f042c069d50 2023-10-08T07:15:43.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.301+0000 7f043456e640 1 -- 172.21.15.70:0/1326992429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f042c067490 con 0x7f042c069d50 2023-10-08T07:15:43.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.302+0000 7f042affd640 1 -- 172.21.15.70:0/1326992429 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v271) v1 ==== 84+0+65 (secure 0 0 0) 0x7f041c058700 con 0x7f042c069d50 2023-10-08T07:15:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:43.303 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T07:15:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-10-08T07:15:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-10-08T07:15:43.303 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:43.303 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T07:15:43.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 -- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f041806b020 msgr2=0x7f041806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:43.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f041806b020 0x7f041806d4e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0420036000 tx=0x7f042003b040 comp rx=0 tx=0).stop 2023-10-08T07:15:43.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 -- 172.21.15.70:0/1326992429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c069d50 msgr2=0x7f042c0b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:43.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c069d50 0x7f042c0b3270 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f041c007e40 tx=0x7f041c00e4b0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 -- 172.21.15.70:0/1326992429 shutdown_connections 2023-10-08T07:15:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f041806b020 0x7f041806d4e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f042c0cbce0 0x7f042c15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f042c0cb880 0x7f042c15b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 --2- 172.21.15.70:0/1326992429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f042c069d50 0x7f042c0b3270 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 -- 172.21.15.70:0/1326992429 >> 172.21.15.70:0/1326992429 conn(0x7f042c06d450 msgr2=0x7f042c15bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 -- 172.21.15.70:0/1326992429 shutdown_connections 2023-10-08T07:15:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.305+0000 7f043456e640 1 -- 172.21.15.70:0/1326992429 wait complete. 2023-10-08T07:15:43.325 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T07:15:43.325 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T07:15:43.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.549+0000 7ffb99533640 1 Processor -- start 2023-10-08T07:15:43.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.549+0000 7ffb99533640 1 -- start start 2023-10-08T07:15:43.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.550+0000 7ffb99533640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb940ae740 0x7ffb940aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.550+0000 7ffb99533640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7ffb94159c80 0x7ffb940ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.550+0000 7ffb99533640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb940cb130 con 0x7ffb940cef10 2023-10-08T07:15:43.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.550+0000 7ffb99533640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb940cf6e0 con 0x7ffb94159c80 2023-10-08T07:15:43.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.550+0000 7ffb99533640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940af060 con 0x7ffb940ae740 2023-10-08T07:15:43.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb937fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ffb940cef10 0x7ffb940cc2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52394/0 (socket says 172.21.15.70:52394) 2023-10-08T07:15:43.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb937fe640 1 -- 172.21.15.70:0/241844548 learned_addr learned my addr 172.21.15.70:0/241844548 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:43.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb92ffd640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb940ae740 0x7ffb940aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb927fc640 1 --2- 172.21.15.70:0/241844548 >> v2:172.21.15.181:3300/0 conn(0x7ffb94159c80 0x7ffb940ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562016937 0 0) 0x7ffb940cb130 con 0x7ffb940cef10 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb80003520 con 0x7ffb940cef10 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 311570808 0 0) 0x7ffb80003520 con 0x7ffb940cef10 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb940ae740 msgr2=0x7ffb940aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb91ffb640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb940ae740 0x7ffb940aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 >> v2:172.21.15.181:3300/0 conn(0x7ffb94159c80 msgr2=0x7ffb940ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb91ffb640 1 --2- 172.21.15.70:0/241844548 >> v2:172.21.15.181:3300/0 conn(0x7ffb94159c80 0x7ffb940ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb940cb360 con 0x7ffb940cef10 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7ffb8c0019f0 con 0x7ffb940cef10 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.551+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 >> v1:172.21.15.70:6789/0 conn(0x7ffb940cef10 legacy=0x7ffb940cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb91ffb640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb80003970 0x7ffb80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb91ffb640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb800047c0 0x7ffb80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb91ffb640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb940ae740 0x7ffb80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940cf6e0 con 0x7ffb800047c0 2023-10-08T07:15:43.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940af060 con 0x7ffb80003970 2023-10-08T07:15:43.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8000b980 con 0x7ffb940ae740 2023-10-08T07:15:43.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb927fc640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb800047c0 0x7ffb80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb92ffd640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb80003970 0x7ffb80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb937fe640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb940ae740 0x7ffb80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb927fc640 1 -- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb940ae740 msgr2=0x7ffb80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:43.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb927fc640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb940ae740 0x7ffb80005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb927fc640 1 -- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb80003970 msgr2=0x7ffb80003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:43.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb927fc640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb80003970 0x7ffb80003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb927fc640 1 -- 172.21.15.70:0/241844548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb80003520 con 0x7ffb800047c0 2023-10-08T07:15:43.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.552+0000 7ffb92ffd640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb80003970 0x7ffb80003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:43.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.553+0000 7ffb927fc640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb800047c0 0x7ffb80004c40 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009ac0 tx=0x7ffb800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba6d0acdb044ace server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:43.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.554+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c02c710 con 0x7ffb800047c0 2023-10-08T07:15:43.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.554+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c02c870 con 0x7ffb800047c0 2023-10-08T07:15:43.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.554+0000 7ffb91ffb640 1 -- 172.21.15.70:0/241844548 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c02cc00 con 0x7ffb800047c0 2023-10-08T07:15:43.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.554+0000 7ffb99533640 1 -- 172.21.15.70:0/241844548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb800047c0 msgr2=0x7ffb80004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:43.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.554+0000 7ffb99533640 1 --2- 172.21.15.70:0/241844548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb800047c0 0x7ffb80004c40 secure :-1 s=READY pgs=2870 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009ac0 tx=0x7ffb800057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.554+0000 7ffb937fe640 1 -- 172.21.15.70:0/241844548 reap_dead start 2023-10-08T07:15:43.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.554+0000 7ffb99533640 1 -- 172.21.15.70:0/241844548 shutdown_connections 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.554+0000 7ffb99533640 1 -- 172.21.15.70:0/241844548 >> 172.21.15.70:0/241844548 conn(0x7ffb9406b070 msgr2=0x7ffb9406b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.554+0000 7ffb99533640 1 -- 172.21.15.70:0/241844548 shutdown_connections 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.554+0000 7ffb99533640 1 -- 172.21.15.70:0/241844548 wait complete. 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.556+0000 7ffb99533640 1 Processor -- start 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.556+0000 7ffb99533640 1 -- start start 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.557+0000 7ffb99533640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb800047c0 0x7ffb9414d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.557+0000 7ffb99533640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9415aa70 0x7ffb9415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.557+0000 7ffb99533640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb9415aed0 0x7ffb9415cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.557+0000 7ffb99533640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415b430 con 0x7ffb9415aa70 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.557+0000 7ffb99533640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415c280 con 0x7ffb9415aed0 2023-10-08T07:15:43.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.557+0000 7ffb99533640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9415b950 con 0x7ffb800047c0 2023-10-08T07:15:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb927fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9415aa70 0x7ffb9415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb927fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9415aa70 0x7ffb9415c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51308/0 (socket says 172.21.15.70:51308) 2023-10-08T07:15:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb927fc640 1 -- 172.21.15.70:0/2220879640 learned_addr learned my addr 172.21.15.70:0/2220879640 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb937fe640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb9415aed0 0x7ffb9415cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb92ffd640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb800047c0 0x7ffb9414d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb927fc640 1 -- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb800047c0 msgr2=0x7ffb9414d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb927fc640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb800047c0 0x7ffb9414d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb927fc640 1 -- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb9415aed0 msgr2=0x7ffb9415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb927fc640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb9415aed0 0x7ffb9415cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:43.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb927fc640 1 -- 172.21.15.70:0/2220879640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9415bc30 con 0x7ffb9415aa70 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.558+0000 7ffb927fc640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9415aa70 0x7ffb9415c710 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c037040 tx=0x7ffb7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.559+0000 7ffb8b7fe640 1 -- 172.21.15.70:0/2220879640 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c003810 con 0x7ffb9415aa70 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.559+0000 7ffb8b7fe640 1 -- 172.21.15.70:0/2220879640 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c02de90 con 0x7ffb9415aa70 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.559+0000 7ffb8b7fe640 1 -- 172.21.15.70:0/2220879640 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c003bb0 con 0x7ffb9415aa70 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.559+0000 7ffb99533640 1 -- 172.21.15.70:0/2220879640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9415bf10 con 0x7ffb9415aa70 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.559+0000 7ffb99533640 1 -- 172.21.15.70:0/2220879640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9414dd70 con 0x7ffb9415aa70 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.560+0000 7ffb99533640 1 -- 172.21.15.70:0/2220879640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb940ce840 con 0x7ffb9415aa70 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.561+0000 7ffb8b7fe640 1 -- 172.21.15.70:0/2220879640 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7ffb7c003d10 con 0x7ffb9415aa70 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.562+0000 7ffb8b7fe640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ffb7406b050 0x7ffb7406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.562+0000 7ffb8b7fe640 1 -- 172.21.15.70:0/2220879640 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(272..272 src has 1..272) v4 ==== 5336+0+0 (secure 0 0 0) 0x7ffb7c0abd50 con 0x7ffb9415aa70 2023-10-08T07:15:43.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.563+0000 7ffb92ffd640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ffb7406b050 0x7ffb7406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:43.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.564+0000 7ffb92ffd640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ffb7406b050 0x7ffb7406d510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb84006fd0 tx=0x7ffb84008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:43.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.566+0000 7ffb8b7fe640 1 -- 172.21.15.70:0/2220879640 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb7c0deca0 con 0x7ffb9415aa70 2023-10-08T07:15:43.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:43.759+0000 7ffb99533640 1 -- 172.21.15.70:0/2220879640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ffb9414df60 con 0x7ffb9415aa70 2023-10-08T07:15:44.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.389+0000 7ffb8b7fe640 1 -- 172.21.15.70:0/2220879640 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v273) v1 ==== 138+0+0 (secure 0 0 0) 0x7ffb9414df60 con 0x7ffb9415aa70 2023-10-08T07:15:44.390 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-10-08T07:15:44.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 -- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ffb7406b050 msgr2=0x7ffb7406d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:44.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ffb7406b050 0x7ffb7406d510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb84006fd0 tx=0x7ffb84008040 comp rx=0 tx=0).stop 2023-10-08T07:15:44.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 -- 172.21.15.70:0/2220879640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9415aa70 msgr2=0x7ffb9415c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:44.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9415aa70 0x7ffb9415c710 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c037040 tx=0x7ffb7c004730 comp rx=0 tx=0).stop 2023-10-08T07:15:44.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 -- 172.21.15.70:0/2220879640 shutdown_connections 2023-10-08T07:15:44.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7ffb7406b050 0x7ffb7406d510 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:44.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7ffb9415aed0 0x7ffb9415cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:44.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9415aa70 0x7ffb9415c710 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:44.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 --2- 172.21.15.70:0/2220879640 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7ffb800047c0 0x7ffb9414d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:44.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 -- 172.21.15.70:0/2220879640 >> 172.21.15.70:0/2220879640 conn(0x7ffb9406b070 msgr2=0x7ffb940cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:44.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.393+0000 7ffb99533640 1 -- 172.21.15.70:0/2220879640 shutdown_connections 2023-10-08T07:15:44.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.394+0000 7ffb99533640 1 -- 172.21.15.70:0/2220879640 wait complete. 2023-10-08T07:15:44.420 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T07:15:44.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.631+0000 7f3850797640 1 Processor -- start 2023-10-08T07:15:44.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.631+0000 7f3850797640 1 -- start start 2023-10-08T07:15:44.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.633+0000 7f3850797640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3848161410 0x7f384806b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.633+0000 7f3850797640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3848161820 con 0x7f38480cc4f0 2023-10-08T07:15:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.633+0000 7f3850797640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3848161a50 con 0x7f38480b24c0 2023-10-08T07:15:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.633+0000 7f3850797640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38480cba10 con 0x7f3848161410 2023-10-08T07:15:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.633+0000 7f384ed34640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3848161410 0x7f384806b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.633+0000 7f384dd32640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f38480cc4f0 0x7f3848160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52410/0 (socket says 172.21.15.70:52410) 2023-10-08T07:15:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.633+0000 7f384dd32640 1 -- 172.21.15.70:0/1546926626 learned_addr learned my addr 172.21.15.70:0/1546926626 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:44.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.634+0000 7f384ed34640 1 -- 172.21.15.70:0/1546926626 >> v1:172.21.15.181:6789/0 conn(0x7f38480b24c0 legacy=0x7f38480b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:44.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.634+0000 7f384ed34640 1 -- 172.21.15.70:0/1546926626 >> v1:172.21.15.70:6789/0 conn(0x7f38480cc4f0 legacy=0x7f3848160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:44.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.634+0000 7f384ed34640 1 -- 172.21.15.70:0/1546926626 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480cbcf0 con 0x7f3848161410 2023-10-08T07:15:44.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.634+0000 7f384d531640 1 -- 172.21.15.70:0/1546926626 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249643990 0 0) 0x7f3848161820 con 0x7f38480cc4f0 2023-10-08T07:15:44.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.635+0000 7f384ed34640 1 --2- 172.21.15.70:0/1546926626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3848161410 0x7f384806b860 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f3840009d90 tx=0x7f384002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8057fd4ca00facce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:44.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.635+0000 7f384d531640 1 -- 172.21.15.70:0/1546926626 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384002ccf0 con 0x7f3848161410 2023-10-08T07:15:44.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.635+0000 7f384d531640 1 -- 172.21.15.70:0/1546926626 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384002ce50 con 0x7f3848161410 2023-10-08T07:15:44.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.635+0000 7f384d531640 1 -- 172.21.15.70:0/1546926626 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840034660 con 0x7f3848161410 2023-10-08T07:15:44.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.636+0000 7f3850797640 1 -- 172.21.15.70:0/1546926626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3848161410 msgr2=0x7f384806b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:44.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.636+0000 7f3850797640 1 --2- 172.21.15.70:0/1546926626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3848161410 0x7f384806b860 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f3840009d90 tx=0x7f384002bf40 comp rx=0 tx=0).stop 2023-10-08T07:15:44.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.636+0000 7f3850797640 1 -- 172.21.15.70:0/1546926626 shutdown_connections 2023-10-08T07:15:44.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.636+0000 7f3850797640 1 --2- 172.21.15.70:0/1546926626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3848161410 0x7f384806b860 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:44.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.636+0000 7f3850797640 1 -- 172.21.15.70:0/1546926626 >> 172.21.15.70:0/1546926626 conn(0x7f38480d0a40 msgr2=0x7f38480d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:44.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.636+0000 7f3850797640 1 -- 172.21.15.70:0/1546926626 shutdown_connections 2023-10-08T07:15:44.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.636+0000 7f3850797640 1 -- 172.21.15.70:0/1546926626 wait complete. 2023-10-08T07:15:44.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.638+0000 7f3850797640 1 Processor -- start 2023-10-08T07:15:44.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.638+0000 7f3850797640 1 -- start start 2023-10-08T07:15:44.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.639+0000 7f3850797640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38480b24c0 0x7f3848156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:44.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.639+0000 7f3850797640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38480cc4f0 0x7f384815f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:44.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.639+0000 7f3850797640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3848161410 0x7f38480c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:44.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.639+0000 7f3850797640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384815f310 con 0x7f38480cc4f0 2023-10-08T07:15:44.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.639+0000 7f3850797640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38480ce0c0 con 0x7f3848161410 2023-10-08T07:15:44.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.639+0000 7f3850797640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38480c8770 con 0x7f38480b24c0 2023-10-08T07:15:44.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.639+0000 7f384dd32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38480cc4f0 0x7f384815f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:44.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.639+0000 7f384dd32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38480cc4f0 0x7f384815f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51326/0 (socket says 172.21.15.70:51326) 2023-10-08T07:15:44.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.639+0000 7f384dd32640 1 -- 172.21.15.70:0/2399112333 learned_addr learned my addr 172.21.15.70:0/2399112333 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:44.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f384dd32640 1 -- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38480b24c0 msgr2=0x7f3848156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:44.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f384ed34640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3848161410 0x7f38480c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:44.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f384e533640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38480b24c0 0x7f3848156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:44.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f384dd32640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38480b24c0 0x7f3848156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:44.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f384dd32640 1 -- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3848161410 msgr2=0x7f38480c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:44.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f384dd32640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3848161410 0x7f38480c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:44.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f384dd32640 1 -- 172.21.15.70:0/2399112333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3840009a70 con 0x7f38480cc4f0 2023-10-08T07:15:44.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f384dd32640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38480cc4f0 0x7f384815f730 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f383800c4a0 tx=0x7f383800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:44.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f383effd640 1 -- 172.21.15.70:0/2399112333 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38380074e0 con 0x7f38480cc4f0 2023-10-08T07:15:44.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f383effd640 1 -- 172.21.15.70:0/2399112333 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3838007640 con 0x7f38480cc4f0 2023-10-08T07:15:44.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f3850797640 1 -- 172.21.15.70:0/2399112333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f384815e970 con 0x7f38480cc4f0 2023-10-08T07:15:44.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f383effd640 1 -- 172.21.15.70:0/2399112333 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3838004a00 con 0x7f38480cc4f0 2023-10-08T07:15:44.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.640+0000 7f3850797640 1 -- 172.21.15.70:0/2399112333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f384815ef30 con 0x7f38480cc4f0 2023-10-08T07:15:44.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.641+0000 7f3850797640 1 -- 172.21.15.70:0/2399112333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384815f450 con 0x7f38480cc4f0 2023-10-08T07:15:44.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.643+0000 7f383effd640 1 -- 172.21.15.70:0/2399112333 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 145) v1 ==== 89377+0+0 (secure 0 0 0) 0x7f383801c3b0 con 0x7f38480cc4f0 2023-10-08T07:15:44.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.644+0000 7f383effd640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f382c06b050 0x7f382c06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:44.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.644+0000 7f383effd640 1 -- 172.21.15.70:0/2399112333 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(273..273 src has 1..273) v4 ==== 5706+0+0 (secure 0 0 0) 0x7f3838088f60 con 0x7f38480cc4f0 2023-10-08T07:15:44.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.645+0000 7f384e533640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f382c06b050 0x7f382c06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:44.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.646+0000 7f384e533640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f382c06b050 0x7f382c06d510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3844000f60 tx=0x7f3844009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:44.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.648+0000 7f383effd640 1 -- 172.21.15.70:0/2399112333 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3838053e00 con 0x7f38480cc4f0 2023-10-08T07:15:44.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:44.838+0000 7f3850797640 1 -- 172.21.15.70:0/2399112333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f384815f450 con 0x7f38480cc4f0 2023-10-08T07:15:45.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.413+0000 7f383effd640 1 -- 172.21.15.70:0/2399112333 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v274) v1 ==== 170+0+0 (secure 0 0 0) 0x7f384815f450 con 0x7f38480cc4f0 2023-10-08T07:15:45.414 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-10-08T07:15:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.416+0000 7f3850797640 1 -- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f382c06b050 msgr2=0x7f382c06d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.416+0000 7f3850797640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f382c06b050 0x7f382c06d510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3844000f60 tx=0x7f3844009040 comp rx=0 tx=0).stop 2023-10-08T07:15:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.416+0000 7f3850797640 1 -- 172.21.15.70:0/2399112333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38480cc4f0 msgr2=0x7f384815f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.416+0000 7f3850797640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38480cc4f0 0x7f384815f730 secure :-1 s=READY pgs=2874 cs=0 l=1 rev1=1 crypto rx=0x7f383800c4a0 tx=0x7f383800c960 comp rx=0 tx=0).stop 2023-10-08T07:15:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.417+0000 7f3850797640 1 -- 172.21.15.70:0/2399112333 shutdown_connections 2023-10-08T07:15:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.417+0000 7f3850797640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f382c06b050 0x7f382c06d510 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.417+0000 7f3850797640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3848161410 0x7f38480c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.417+0000 7f3850797640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38480cc4f0 0x7f384815f730 unknown :-1 s=CLOSED pgs=2874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.417+0000 7f3850797640 1 --2- 172.21.15.70:0/2399112333 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f38480b24c0 0x7f3848156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.417+0000 7f3850797640 1 -- 172.21.15.70:0/2399112333 >> 172.21.15.70:0/2399112333 conn(0x7f38480d0a40 msgr2=0x7f38480ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.418+0000 7f3850797640 1 -- 172.21.15.70:0/2399112333 shutdown_connections 2023-10-08T07:15:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.418+0000 7f3850797640 1 -- 172.21.15.70:0/2399112333 wait complete. 2023-10-08T07:15:45.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T07:15:45.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.639+0000 7fbc94fe2640 1 Processor -- start 2023-10-08T07:15:45.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.640+0000 7fbc94fe2640 1 -- start start 2023-10-08T07:15:45.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.642+0000 7fbc94fe2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc900b4e10 0x7fbc901612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:45.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.642+0000 7fbc94fe2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900cd4e0 0x7fbc900d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:45.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.642+0000 7fbc94fe2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc900b8c30 con 0x7fbc9015cd70 2023-10-08T07:15:45.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.642+0000 7fbc94fe2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc900cdd80 con 0x7fbc900b4e10 2023-10-08T07:15:45.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.642+0000 7fbc94fe2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc900ce060 con 0x7fbc900cd4e0 2023-10-08T07:15:45.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.642+0000 7fbc8dd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc900b4e10 0x7fbc901612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:45.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.642+0000 7fbc8e59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbc9015cd70 0x7fbc900b4700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52412/0 (socket says 172.21.15.70:52412) 2023-10-08T07:15:45.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.642+0000 7fbc8e59c640 1 -- 172.21.15.70:0/79198987 learned_addr learned my addr 172.21.15.70:0/79198987 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:45.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.642+0000 7fbc8dd9b640 1 -- 172.21.15.70:0/79198987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900cd4e0 msgr2=0x7fbc900d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:45.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.642+0000 7fbc8ed9d640 1 --2- 172.21.15.70:0/79198987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900cd4e0 0x7fbc900d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:45.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.643+0000 7fbc8dd9b640 1 --2- 172.21.15.70:0/79198987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900cd4e0 0x7fbc900d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:45.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.643+0000 7fbc8dd9b640 1 -- 172.21.15.70:0/79198987 >> v1:172.21.15.70:6789/0 conn(0x7fbc9015cd70 legacy=0x7fbc900b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:45.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.643+0000 7fbc8d59a640 1 -- 172.21.15.70:0/79198987 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951094944 0 0) 0x7fbc900b8c30 con 0x7fbc9015cd70 2023-10-08T07:15:45.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.643+0000 7fbc8dd9b640 1 -- 172.21.15.70:0/79198987 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900ce340 con 0x7fbc900b4e10 2023-10-08T07:15:45.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.643+0000 7fbc8ed9d640 1 --2- 172.21.15.70:0/79198987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900cd4e0 0x7fbc900d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.644+0000 7fbc8dd9b640 1 --2- 172.21.15.70:0/79198987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc900b4e10 0x7fbc901612f0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7fbc840097a0 tx=0x7fbc8402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e76ca6b97f41b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.644+0000 7fbc8d59a640 1 -- 172.21.15.70:0/79198987 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8402d040 con 0x7fbc900b4e10 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.644+0000 7fbc8d59a640 1 -- 172.21.15.70:0/79198987 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc84031810 con 0x7fbc900b4e10 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.644+0000 7fbc8d59a640 1 -- 172.21.15.70:0/79198987 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84031ba0 con 0x7fbc900b4e10 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.644+0000 7fbc94fe2640 1 -- 172.21.15.70:0/79198987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc900b4e10 msgr2=0x7fbc901612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.644+0000 7fbc94fe2640 1 --2- 172.21.15.70:0/79198987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc900b4e10 0x7fbc901612f0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7fbc840097a0 tx=0x7fbc8402b8b0 comp rx=0 tx=0).stop 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.645+0000 7fbc94fe2640 1 -- 172.21.15.70:0/79198987 shutdown_connections 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.645+0000 7fbc94fe2640 1 --2- 172.21.15.70:0/79198987 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900cd4e0 0x7fbc900d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.645+0000 7fbc94fe2640 1 --2- 172.21.15.70:0/79198987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc900b4e10 0x7fbc901612f0 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.645+0000 7fbc94fe2640 1 -- 172.21.15.70:0/79198987 >> 172.21.15.70:0/79198987 conn(0x7fbc900cf700 msgr2=0x7fbc900cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:45.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.645+0000 7fbc94fe2640 1 -- 172.21.15.70:0/79198987 shutdown_connections 2023-10-08T07:15:45.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.645+0000 7fbc94fe2640 1 -- 172.21.15.70:0/79198987 wait complete. 2023-10-08T07:15:45.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.646+0000 7fbc94fe2640 1 Processor -- start 2023-10-08T07:15:45.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.647+0000 7fbc94fe2640 1 -- start start 2023-10-08T07:15:45.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.647+0000 7fbc94fe2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900b4e10 0x7fbc9015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:45.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.647+0000 7fbc94fe2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900cd4e0 0x7fbc901f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:45.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.647+0000 7fbc94fe2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc9015cd70 0x7fbc9015f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:45.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.647+0000 7fbc94fe2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9015f320 con 0x7fbc9015cd70 2023-10-08T07:15:45.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.647+0000 7fbc94fe2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901f2ad0 con 0x7fbc900cd4e0 2023-10-08T07:15:45.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.647+0000 7fbc94fe2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc901f6820 con 0x7fbc900b4e10 2023-10-08T07:15:45.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.648+0000 7fbc8ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc9015cd70 0x7fbc9015f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:45.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.648+0000 7fbc8ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc9015cd70 0x7fbc9015f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51338/0 (socket says 172.21.15.70:51338) 2023-10-08T07:15:45.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.648+0000 7fbc8ed9d640 1 -- 172.21.15.70:0/2768551709 learned_addr learned my addr 172.21.15.70:0/2768551709 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:45.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.648+0000 7fbc8dd9b640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900cd4e0 0x7fbc901f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:45.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.648+0000 7fbc8e59c640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900b4e10 0x7fbc9015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:45.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.648+0000 7fbc8ed9d640 1 -- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900b4e10 msgr2=0x7fbc9015bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:45.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.648+0000 7fbc8ed9d640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900b4e10 0x7fbc9015bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:45.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.648+0000 7fbc8ed9d640 1 -- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900cd4e0 msgr2=0x7fbc901f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:45.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.648+0000 7fbc8ed9d640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900cd4e0 0x7fbc901f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:45.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.648+0000 7fbc8ed9d640 1 -- 172.21.15.70:0/2768551709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84009400 con 0x7fbc9015cd70 2023-10-08T07:15:45.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.649+0000 7fbc8ed9d640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc9015cd70 0x7fbc9015f4f0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800c5c0 tx=0x7fbc8800ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:45.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.649+0000 7fbc7effd640 1 -- 172.21.15.70:0/2768551709 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88004440 con 0x7fbc9015cd70 2023-10-08T07:15:45.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.649+0000 7fbc7effd640 1 -- 172.21.15.70:0/2768551709 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc88003610 con 0x7fbc9015cd70 2023-10-08T07:15:45.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.649+0000 7fbc94fe2640 1 -- 172.21.15.70:0/2768551709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9015f040 con 0x7fbc9015cd70 2023-10-08T07:15:45.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.649+0000 7fbc7effd640 1 -- 172.21.15.70:0/2768551709 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc880039c0 con 0x7fbc9015cd70 2023-10-08T07:15:45.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.649+0000 7fbc94fe2640 1 -- 172.21.15.70:0/2768551709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc901f2760 con 0x7fbc9015cd70 2023-10-08T07:15:45.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.650+0000 7fbc94fe2640 1 -- 172.21.15.70:0/2768551709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc901f22e0 con 0x7fbc9015cd70 2023-10-08T07:15:45.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.651+0000 7fbc7effd640 1 -- 172.21.15.70:0/2768551709 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fbc88003b20 con 0x7fbc9015cd70 2023-10-08T07:15:45.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.652+0000 7fbc7effd640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fbc7006b0b0 0x7fbc7006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:45.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.652+0000 7fbc7effd640 1 -- 172.21.15.70:0/2768551709 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(274..274 src has 1..274) v4 ==== 6088+0+0 (secure 0 0 0) 0x7fbc88089f30 con 0x7fbc9015cd70 2023-10-08T07:15:45.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.653+0000 7fbc8e59c640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fbc7006b0b0 0x7fbc7006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:45.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.654+0000 7fbc8e59c640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fbc7006b0b0 0x7fbc7006d570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc800012e0 tx=0x7fbc80009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:45.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.657+0000 7fbc7effd640 1 -- 172.21.15.70:0/2768551709 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc901f22e0 con 0x7fbc9015cd70 2023-10-08T07:15:45.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:45.888+0000 7fbc94fe2640 1 -- 172.21.15.70:0/2768551709 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbc901f22e0 con 0x7fbc9015cd70 2023-10-08T07:15:46.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.427+0000 7fbc7effd640 1 -- 172.21.15.70:0/2768551709 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 35 and data pool 36 v136) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbc901f22e0 con 0x7fbc9015cd70 2023-10-08T07:15:46.429 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 35 and data pool 36 2023-10-08T07:15:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.434+0000 7fbc94fe2640 1 -- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fbc7006b0b0 msgr2=0x7fbc7006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.434+0000 7fbc94fe2640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fbc7006b0b0 0x7fbc7006d570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbc800012e0 tx=0x7fbc80009040 comp rx=0 tx=0).stop 2023-10-08T07:15:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.434+0000 7fbc94fe2640 1 -- 172.21.15.70:0/2768551709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc9015cd70 msgr2=0x7fbc9015f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.434+0000 7fbc94fe2640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc9015cd70 0x7fbc9015f4f0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800c5c0 tx=0x7fbc8800ca80 comp rx=0 tx=0).stop 2023-10-08T07:15:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.436+0000 7fbc94fe2640 1 -- 172.21.15.70:0/2768551709 shutdown_connections 2023-10-08T07:15:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.436+0000 7fbc94fe2640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fbc7006b0b0 0x7fbc7006d570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.436+0000 7fbc94fe2640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc9015cd70 0x7fbc9015f4f0 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.436+0000 7fbc94fe2640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc900cd4e0 0x7fbc901f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.436+0000 7fbc94fe2640 1 --2- 172.21.15.70:0/2768551709 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc900b4e10 0x7fbc9015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.436+0000 7fbc94fe2640 1 -- 172.21.15.70:0/2768551709 >> 172.21.15.70:0/2768551709 conn(0x7fbc900cf700 msgr2=0x7fbc900b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.437+0000 7fbc94fe2640 1 -- 172.21.15.70:0/2768551709 shutdown_connections 2023-10-08T07:15:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.439+0000 7fbc94fe2640 1 -- 172.21.15.70:0/2768551709 wait complete. 2023-10-08T07:15:46.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:46.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.649+0000 7f1be0f0f640 1 Processor -- start 2023-10-08T07:15:46.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.649+0000 7f1be0f0f640 1 -- start start 2023-10-08T07:15:46.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.651+0000 7f1be0f0f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f1bdc159ac0 0x7f1bdc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:46.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.651+0000 7f1be0f0f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bdc0ad790 0x7f1bdc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:46.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.651+0000 7f1be0f0f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bdc06dee0 con 0x7f1bdc15e330 2023-10-08T07:15:46.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.651+0000 7f1be0f0f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bdc15d850 con 0x7f1bdc159ac0 2023-10-08T07:15:46.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.651+0000 7f1be0f0f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc15db30 con 0x7f1bdc0ad790 2023-10-08T07:15:46.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.651+0000 7f1bdad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1bdc15e330 0x7f1bdc06c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52414/0 (socket says 172.21.15.70:52414) 2023-10-08T07:15:46.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.651+0000 7f1bdad9d640 1 -- 172.21.15.70:0/904975962 learned_addr learned my addr 172.21.15.70:0/904975962 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.651+0000 7f1bd3fff640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bdc0ad790 0x7f1bdc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.651+0000 7f1bda59c640 1 --2- 172.21.15.70:0/904975962 >> v2:172.21.15.181:3300/0 conn(0x7f1bdc159ac0 0x7f1bdc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763527298 0 0) 0x7f1bdc06dee0 con 0x7f1bdc15e330 2023-10-08T07:15:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bc8003520 con 0x7f1bdc15e330 2023-10-08T07:15:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3238128110 0 0) 0x7f1bc8003520 con 0x7f1bdc15e330 2023-10-08T07:15:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bdc0ad790 msgr2=0x7f1bdc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bdc0ad790 0x7f1bdc0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 >> v2:172.21.15.181:3300/0 conn(0x7f1bdc159ac0 msgr2=0x7f1bdc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:46.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 --2- 172.21.15.70:0/904975962 >> v2:172.21.15.181:3300/0 conn(0x7f1bdc159ac0 0x7f1bdc06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc15de10 con 0x7f1bdc15e330 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f1bd40019f0 con 0x7f1bdc15e330 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 >> v1:172.21.15.70:6789/0 conn(0x7f1bdc15e330 legacy=0x7f1bdc06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bdc0ad790 0x7f1bc8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bc8003f40 0x7f1bc80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bc8004d00 0x7f1bc8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc15d850 con 0x7f1bc8004d00 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc15db30 con 0x7f1bc8003f40 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.652+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc800db90 con 0x7f1bdc0ad790 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.653+0000 7f1bdad9d640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bc8004d00 0x7f1bc8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.653+0000 7f1bda59c640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bdc0ad790 0x7f1bc8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:46.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.653+0000 7f1bdad9d640 1 -- 172.21.15.70:0/904975962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bdc0ad790 msgr2=0x7f1bc8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.653+0000 7f1bdad9d640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bdc0ad790 0x7f1bc8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.653+0000 7f1bdad9d640 1 -- 172.21.15.70:0/904975962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bc8003f40 msgr2=0x7f1bc80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.653+0000 7f1bdad9d640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bc8003f40 0x7f1bc80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.653+0000 7f1bdad9d640 1 -- 172.21.15.70:0/904975962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc8003520 con 0x7f1bc8004d00 2023-10-08T07:15:46.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.654+0000 7f1bdad9d640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bc8004d00 0x7f1bc8005180 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400a7c0 tx=0x7f1bc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d875d5cade7a1c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.654+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4034630 con 0x7f1bc8004d00 2023-10-08T07:15:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.654+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd402f070 con 0x7f1bc8004d00 2023-10-08T07:15:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.654+0000 7f1bd9d9b640 1 -- 172.21.15.70:0/904975962 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4034a50 con 0x7f1bc8004d00 2023-10-08T07:15:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.655+0000 7f1be0f0f640 1 -- 172.21.15.70:0/904975962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bc8004d00 msgr2=0x7f1bc8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.655+0000 7f1be0f0f640 1 --2- 172.21.15.70:0/904975962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bc8004d00 0x7f1bc8005180 secure :-1 s=READY pgs=2879 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400a7c0 tx=0x7f1bc80057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.655+0000 7f1bdad9d640 1 -- 172.21.15.70:0/904975962 reap_dead start 2023-10-08T07:15:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.655+0000 7f1be0f0f640 1 -- 172.21.15.70:0/904975962 shutdown_connections 2023-10-08T07:15:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.655+0000 7f1be0f0f640 1 -- 172.21.15.70:0/904975962 >> 172.21.15.70:0/904975962 conn(0x7f1bdc06a380 msgr2=0x7f1bdc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.655+0000 7f1be0f0f640 1 -- 172.21.15.70:0/904975962 shutdown_connections 2023-10-08T07:15:46.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.655+0000 7f1be0f0f640 1 -- 172.21.15.70:0/904975962 wait complete. 2023-10-08T07:15:46.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.657+0000 7f1be0f0f640 1 Processor -- start 2023-10-08T07:15:46.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.657+0000 7f1be0f0f640 1 -- start start 2023-10-08T07:15:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1be0f0f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bc8004d00 0x7f1bdc0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1be0f0f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bdc1511a0 0x7f1bdc151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1be0f0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bdc148eb0 0x7f1bdc149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1be0f0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc1528e0 con 0x7f1bdc148eb0 2023-10-08T07:15:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1be0f0f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc1498a0 con 0x7f1bdc1511a0 2023-10-08T07:15:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1be0f0f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc151c60 con 0x7f1bc8004d00 2023-10-08T07:15:46.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1bdad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bdc148eb0 0x7f1bdc149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1bdad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bdc148eb0 0x7f1bdc149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51358/0 (socket says 172.21.15.70:51358) 2023-10-08T07:15:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1bdad9d640 1 -- 172.21.15.70:0/3270404909 learned_addr learned my addr 172.21.15.70:0/3270404909 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1bda59c640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bc8004d00 0x7f1bdc0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.658+0000 7f1bd3fff640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bdc1511a0 0x7f1bdc151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1bdad9d640 1 -- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bc8004d00 msgr2=0x7f1bdc0af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1bdad9d640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bc8004d00 0x7f1bdc0af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1bdad9d640 1 -- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bdc1511a0 msgr2=0x7f1bdc151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:46.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1bdad9d640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bdc1511a0 0x7f1bdc151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1bdad9d640 1 -- 172.21.15.70:0/3270404909 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc151f40 con 0x7f1bdc148eb0 2023-10-08T07:15:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1bda59c640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bc8004d00 0x7f1bdc0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1bdad9d640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bdc148eb0 0x7f1bdc149360 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400a760 tx=0x7f1bd4001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1bd2ffd640 1 -- 172.21.15.70:0/3270404909 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4034640 con 0x7f1bdc148eb0 2023-10-08T07:15:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1bd2ffd640 1 -- 172.21.15.70:0/3270404909 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd40347a0 con 0x7f1bdc148eb0 2023-10-08T07:15:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1bd2ffd640 1 -- 172.21.15.70:0/3270404909 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4034b10 con 0x7f1bdc148eb0 2023-10-08T07:15:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1be0f0f640 1 -- 172.21.15.70:0/3270404909 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bdc152220 con 0x7f1bdc148eb0 2023-10-08T07:15:46.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.659+0000 7f1be0f0f640 1 -- 172.21.15.70:0/3270404909 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bdc15ced0 con 0x7f1bdc148eb0 2023-10-08T07:15:46.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.660+0000 7f1be0f0f640 1 -- 172.21.15.70:0/3270404909 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bdc152a20 con 0x7f1bdc148eb0 2023-10-08T07:15:46.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.661+0000 7f1bd2ffd640 1 -- 172.21.15.70:0/3270404909 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f1bd4031030 con 0x7f1bdc148eb0 2023-10-08T07:15:46.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.662+0000 7f1bd2ffd640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f1bbc06b0b0 0x7f1bbc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:46.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.662+0000 7f1bd2ffd640 1 -- 172.21.15.70:0/3270404909 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f1bd40abfd0 con 0x7f1bdc148eb0 2023-10-08T07:15:46.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.663+0000 7f1bda59c640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f1bbc06b0b0 0x7f1bbc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:46.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.664+0000 7f1bda59c640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f1bbc06b0b0 0x7f1bbc06d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc154140 tx=0x7f1bcc008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:46.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.667+0000 7f1bd2ffd640 1 -- 172.21.15.70:0/3270404909 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1bdc152a20 con 0x7f1bdc148eb0 2023-10-08T07:15:46.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.844+0000 7f1be0f0f640 1 -- 172.21.15.70:0/3270404909 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bdc152a20 con 0x7f1bdc148eb0 2023-10-08T07:15:46.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.847+0000 7f1bd2ffd640 1 -- 172.21.15.70:0/3270404909 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v275) v1 ==== 74+0+16969 (secure 0 0 0) 0x7f1bdc152a20 con 0x7f1bdc148eb0 2023-10-08T07:15:46.848 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:46.849 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":275,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:45.889413+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:15:43.819948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-08T07:15:44.895969+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1513601554":"2023-10-08T08:15:42.823407+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:46.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.849+0000 7f1be0f0f640 1 -- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f1bbc06b0b0 msgr2=0x7f1bbc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:46.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.849+0000 7f1be0f0f640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f1bbc06b0b0 0x7f1bbc06d570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc154140 tx=0x7f1bcc008040 comp rx=0 tx=0).stop 2023-10-08T07:15:46.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.850+0000 7f1be0f0f640 1 -- 172.21.15.70:0/3270404909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bdc148eb0 msgr2=0x7f1bdc149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:46.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.850+0000 7f1be0f0f640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bdc148eb0 0x7f1bdc149360 secure :-1 s=READY pgs=2880 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400a760 tx=0x7f1bd4001cb0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.850+0000 7f1be0f0f640 1 -- 172.21.15.70:0/3270404909 shutdown_connections 2023-10-08T07:15:46.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.850+0000 7f1be0f0f640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f1bbc06b0b0 0x7f1bbc06d570 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.850+0000 7f1be0f0f640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bdc148eb0 0x7f1bdc149360 unknown :-1 s=CLOSED pgs=2880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.850+0000 7f1be0f0f640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1bdc1511a0 0x7f1bdc151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.850+0000 7f1be0f0f640 1 --2- 172.21.15.70:0/3270404909 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1bc8004d00 0x7f1bdc0af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:46.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.850+0000 7f1be0f0f640 1 -- 172.21.15.70:0/3270404909 >> 172.21.15.70:0/3270404909 conn(0x7f1bdc06a380 msgr2=0x7f1bdc154210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:46.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.850+0000 7f1be0f0f640 1 -- 172.21.15.70:0/3270404909 shutdown_connections 2023-10-08T07:15:46.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:46.850+0000 7f1be0f0f640 1 -- 172.21.15.70:0/3270404909 wait complete. 2023-10-08T07:15:46.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:47.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.074+0000 7f0419a4f640 1 Processor -- start 2023-10-08T07:15:47.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.074+0000 7f0419a4f640 1 -- start start 2023-10-08T07:15:47.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.075+0000 7f0419a4f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04140cd8d0 0x7f04140cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.075+0000 7f0419a4f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04140b1840 0x7f04140d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.075+0000 7f0419a4f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04140b8bf0 con 0x7f04140b1d90 2023-10-08T07:15:47.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.075+0000 7f0419a4f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140d2910 con 0x7f04140cd8d0 2023-10-08T07:15:47.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.075+0000 7f0419a4f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04140ce1f0 con 0x7f04140b1840 2023-10-08T07:15:47.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.076+0000 7f0412ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04140cd8d0 0x7f04140cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.076+0000 7f0412ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04140cd8d0 0x7f04140cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51374/0 (socket says 172.21.15.70:51374) 2023-10-08T07:15:47.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.076+0000 7f0412ffd640 1 -- 172.21.15.70:0/3714865256 learned_addr learned my addr 172.21.15.70:0/3714865256 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:47.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.076+0000 7f04127fc640 1 --2- 172.21.15.70:0/3714865256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04140b1840 0x7f04140d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.076+0000 7f0412ffd640 1 -- 172.21.15.70:0/3714865256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04140b1840 msgr2=0x7f04140d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:47.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.076+0000 7f0412ffd640 1 --2- 172.21.15.70:0/3714865256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04140b1840 0x7f04140d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.076+0000 7f0412ffd640 1 -- 172.21.15.70:0/3714865256 >> v1:172.21.15.70:6789/0 conn(0x7f04140b1d90 legacy=0x7f04140d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.076+0000 7f0412ffd640 1 -- 172.21.15.70:0/3714865256 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140b8630 con 0x7f04140cd8d0 2023-10-08T07:15:47.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.076+0000 7f0411ffb640 1 -- 172.21.15.70:0/3714865256 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2424495291 0 0) 0x7f04140b8bf0 con 0x7f04140b1d90 2023-10-08T07:15:47.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.077+0000 7f0412ffd640 1 --2- 172.21.15.70:0/3714865256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140cd8d0 0x7f04140cdcb0 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f0404009870 tx=0x7f040402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d57066003a6945 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0411ffb640 1 -- 172.21.15.70:0/3714865256 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404030840 con 0x7f04140cd8d0 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0411ffb640 1 -- 172.21.15.70:0/3714865256 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04040309a0 con 0x7f04140cd8d0 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0411ffb640 1 -- 172.21.15.70:0/3714865256 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404030d30 con 0x7f04140cd8d0 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0419a4f640 1 -- 172.21.15.70:0/3714865256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140cd8d0 msgr2=0x7f04140cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0419a4f640 1 --2- 172.21.15.70:0/3714865256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140cd8d0 0x7f04140cdcb0 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f0404009870 tx=0x7f040402b980 comp rx=0 tx=0).stop 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0419a4f640 1 -- 172.21.15.70:0/3714865256 shutdown_connections 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0419a4f640 1 --2- 172.21.15.70:0/3714865256 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04140b1840 0x7f04140d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0419a4f640 1 --2- 172.21.15.70:0/3714865256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140cd8d0 0x7f04140cdcb0 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0419a4f640 1 -- 172.21.15.70:0/3714865256 >> 172.21.15.70:0/3714865256 conn(0x7f041406b6a0 msgr2=0x7f041406bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0419a4f640 1 -- 172.21.15.70:0/3714865256 shutdown_connections 2023-10-08T07:15:47.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.078+0000 7f0419a4f640 1 -- 172.21.15.70:0/3714865256 wait complete. 2023-10-08T07:15:47.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.080+0000 7f0419a4f640 1 Processor -- start 2023-10-08T07:15:47.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.080+0000 7f0419a4f640 1 -- start start 2023-10-08T07:15:47.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.081+0000 7f0419a4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140b1840 0x7f041415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.081+0000 7f0419a4f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04140b1d90 0x7f04140cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.081+0000 7f0419a4f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04140cd8d0 0x7f04140cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.081+0000 7f0419a4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0414143fc0 con 0x7f04140b1840 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.081+0000 7f0419a4f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04140c8cf0 con 0x7f04140cd8d0 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.081+0000 7f0419a4f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0414144280 con 0x7f04140b1d90 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.081+0000 7f0412ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140b1840 0x7f041415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.081+0000 7f0412ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140b1840 0x7f041415b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51378/0 (socket says 172.21.15.70:51378) 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.081+0000 7f0412ffd640 1 -- 172.21.15.70:0/3540327169 learned_addr learned my addr 172.21.15.70:0/3540327169 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f0412ffd640 1 -- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04140b1d90 msgr2=0x7f04140cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f0412ffd640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04140b1d90 0x7f04140cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f0412ffd640 1 -- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04140cd8d0 msgr2=0x7f04140cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f04137fe640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04140cd8d0 0x7f04140cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f0412ffd640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04140cd8d0 0x7f04140cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f0412ffd640 1 -- 172.21.15.70:0/3540327169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04040094d0 con 0x7f04140b1840 2023-10-08T07:15:47.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f0412ffd640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140b1840 0x7f041415b970 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f0404000c00 tx=0x7f0404004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:47.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f040bfff640 1 -- 172.21.15.70:0/3540327169 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040403e040 con 0x7f04140b1840 2023-10-08T07:15:47.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f040bfff640 1 -- 172.21.15.70:0/3540327169 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0404039e20 con 0x7f04140b1840 2023-10-08T07:15:47.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f040bfff640 1 -- 172.21.15.70:0/3540327169 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404004270 con 0x7f04140b1840 2023-10-08T07:15:47.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.082+0000 7f0419a4f640 1 -- 172.21.15.70:0/3540327169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0414144560 con 0x7f04140b1840 2023-10-08T07:15:47.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.083+0000 7f0419a4f640 1 -- 172.21.15.70:0/3540327169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04140c7fc0 con 0x7f04140b1840 2023-10-08T07:15:47.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.084+0000 7f0419a4f640 1 -- 172.21.15.70:0/3540327169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0414066d80 con 0x7f04140b1840 2023-10-08T07:15:47.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.085+0000 7f040bfff640 1 -- 172.21.15.70:0/3540327169 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f04040043d0 con 0x7f04140b1840 2023-10-08T07:15:47.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.086+0000 7f040bfff640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f03f406b0b0 0x7f03f406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.086+0000 7f040bfff640 1 -- 172.21.15.70:0/3540327169 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(275..275 src has 1..275) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f04040ac2a0 con 0x7f04140b1840 2023-10-08T07:15:47.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.086+0000 7f04127fc640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f03f406b0b0 0x7f03f406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.087+0000 7f04127fc640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f03f406b0b0 0x7f03f406d570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f04140ceb10 tx=0x7f03fc008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:47.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.090+0000 7f040bfff640 1 -- 172.21.15.70:0/3540327169 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0404076d90 con 0x7f04140b1840 2023-10-08T07:15:47.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.282+0000 7f0419a4f640 1 -- 172.21.15.70:0/3540327169 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04140b5940 con 0x7f04140b1840 2023-10-08T07:15:47.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.285+0000 7f040bfff640 1 -- 172.21.15.70:0/3540327169 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v275) v1 ==== 74+0+16969 (secure 0 0 0) 0x7f040407a3b0 con 0x7f04140b1840 2023-10-08T07:15:47.286 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:47.286 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":275,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:45.889413+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:15:43.819948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-08T07:15:44.895969+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1513601554":"2023-10-08T08:15:42.823407+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:47.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 -- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f03f406b0b0 msgr2=0x7f03f406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f03f406b0b0 0x7f03f406d570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f04140ceb10 tx=0x7f03fc008040 comp rx=0 tx=0).stop 2023-10-08T07:15:47.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 -- 172.21.15.70:0/3540327169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140b1840 msgr2=0x7f041415b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140b1840 0x7f041415b970 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f0404000c00 tx=0x7f0404004730 comp rx=0 tx=0).stop 2023-10-08T07:15:47.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 -- 172.21.15.70:0/3540327169 shutdown_connections 2023-10-08T07:15:47.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f03f406b0b0 0x7f03f406d570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f04140cd8d0 0x7f04140cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f04140b1d90 0x7f04140cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 --2- 172.21.15.70:0/3540327169 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140b1840 0x7f041415b970 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 -- 172.21.15.70:0/3540327169 >> 172.21.15.70:0/3540327169 conn(0x7f041406b6a0 msgr2=0x7f04140b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:47.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 -- 172.21.15.70:0/3540327169 shutdown_connections 2023-10-08T07:15:47.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.288+0000 7f0419a4f640 1 -- 172.21.15.70:0/3540327169 wait complete. 2023-10-08T07:15:47.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T07:15:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.530+0000 7f10bd97c640 1 Processor -- start 2023-10-08T07:15:47.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.530+0000 7f10bd97c640 1 -- start start 2023-10-08T07:15:47.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.532+0000 7f10bd97c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f10b80cb130 0x7f10b80cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.532+0000 7f10bd97c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b80b1cc0 con 0x7f10b80afbe0 2023-10-08T07:15:47.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.532+0000 7f10bd97c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b80cbad0 con 0x7f10b806dfc0 2023-10-08T07:15:47.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.532+0000 7f10bd97c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b80b2620 con 0x7f10b80cb130 2023-10-08T07:15:47.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.532+0000 7f10b6ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f10b80afbe0 0x7f10b80b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52436/0 (socket says 172.21.15.70:52436) 2023-10-08T07:15:47.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.532+0000 7f10b6ffd640 1 -- 172.21.15.70:0/265660528 learned_addr learned my addr 172.21.15.70:0/265660528 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:47.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b77fe640 1 --2- 172.21.15.70:0/265660528 >> v2:172.21.15.181:3300/0 conn(0x7f10b80cb130 0x7f10b80cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 872754345 0 0) 0x7f10b80b1cc0 con 0x7f10b80afbe0 2023-10-08T07:15:47.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a4003520 con 0x7f10b80afbe0 2023-10-08T07:15:47.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1364492621 0 0) 0x7f10a4003520 con 0x7f10b80afbe0 2023-10-08T07:15:47.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 >> v2:172.21.15.181:3300/0 conn(0x7f10b80cb130 msgr2=0x7f10b80cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:47.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 --2- 172.21.15.70:0/265660528 >> v2:172.21.15.181:3300/0 conn(0x7f10b80cb130 0x7f10b80cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 >> v1:172.21.15.181:6789/0 conn(0x7f10b806dfc0 legacy=0x7f10b8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b80b2060 con 0x7f10b80afbe0 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f10a8001670 con 0x7f10b80afbe0 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 >> v1:172.21.15.70:6789/0 conn(0x7f10b80afbe0 legacy=0x7f10b80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4003970 0x7f10a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10a40047c0 0x7f10a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a4005180 0x7f10a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80b2620 con 0x7f10a4003970 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a400c020 con 0x7f10a4005180 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.533+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a400c1e0 con 0x7f10a40047c0 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.534+0000 7f10b6ffd640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4003970 0x7f10a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.534+0000 7f10affff640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10a40047c0 0x7f10a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.534+0000 7f10b77fe640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a4005180 0x7f10a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.534+0000 7f10b6ffd640 1 -- 172.21.15.70:0/265660528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10a40047c0 msgr2=0x7f10a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:47.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.534+0000 7f10b6ffd640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10a40047c0 0x7f10a4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.534+0000 7f10b6ffd640 1 -- 172.21.15.70:0/265660528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a4005180 msgr2=0x7f10a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:47.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.534+0000 7f10b6ffd640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a4005180 0x7f10a400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.534+0000 7f10b77fe640 1 -- 172.21.15.70:0/265660528 reap_dead start 2023-10-08T07:15:47.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.534+0000 7f10b6ffd640 1 -- 172.21.15.70:0/265660528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a4003520 con 0x7f10a4003970 2023-10-08T07:15:47.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.535+0000 7f10b6ffd640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4003970 0x7f10a4003d90 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c240 tx=0x7f10a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfc7d869f3a2780 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:47.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.536+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8005ad0 con 0x7f10a4003970 2023-10-08T07:15:47.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.536+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a8005c30 con 0x7f10a4003970 2023-10-08T07:15:47.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.536+0000 7f10b67fc640 1 -- 172.21.15.70:0/265660528 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a802ca90 con 0x7f10a4003970 2023-10-08T07:15:47.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.536+0000 7f10bd97c640 1 -- 172.21.15.70:0/265660528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4003970 msgr2=0x7f10a4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.536+0000 7f10bd97c640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4003970 0x7f10a4003d90 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c240 tx=0x7f10a40057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.536+0000 7f10bd97c640 1 -- 172.21.15.70:0/265660528 shutdown_connections 2023-10-08T07:15:47.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.536+0000 7f10bd97c640 1 --2- 172.21.15.70:0/265660528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4003970 0x7f10a4003d90 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.536+0000 7f10bd97c640 1 -- 172.21.15.70:0/265660528 >> 172.21.15.70:0/265660528 conn(0x7f10b806a5d0 msgr2=0x7f10b80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:47.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.536+0000 7f10bd97c640 1 -- 172.21.15.70:0/265660528 shutdown_connections 2023-10-08T07:15:47.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.536+0000 7f10bd97c640 1 -- 172.21.15.70:0/265660528 wait complete. 2023-10-08T07:15:47.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.538+0000 7f10bd97c640 1 Processor -- start 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.538+0000 7f10bd97c640 1 -- start start 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.539+0000 7f10bd97c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a4003970 0x7f10b814cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.539+0000 7f10bd97c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10b80c2ee0 0x7f10b80ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.539+0000 7f10bd97c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80c3360 0x7f10b80cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.539+0000 7f10bd97c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b815e0a0 con 0x7f10b80c3360 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.539+0000 7f10bd97c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b814d100 con 0x7f10a4003970 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.539+0000 7f10bd97c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b815c050 con 0x7f10b80c2ee0 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80c3360 0x7f10b80cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10b77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80c3360 0x7f10b80cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51400/0 (socket says 172.21.15.70:51400) 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10b77fe640 1 -- 172.21.15.70:0/114879106 learned_addr learned my addr 172.21.15.70:0/114879106 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10b6ffd640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a4003970 0x7f10b814cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10affff640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10b80c2ee0 0x7f10b80ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10b77fe640 1 -- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10b80c2ee0 msgr2=0x7f10b80ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10b77fe640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10b80c2ee0 0x7f10b80ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10b77fe640 1 -- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a4003970 msgr2=0x7f10b814cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10b77fe640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a4003970 0x7f10b814cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10b77fe640 1 -- 172.21.15.70:0/114879106 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b815c2f0 con 0x7f10b80c3360 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.540+0000 7f10b77fe640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80c3360 0x7f10b80cd120 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f10b80cb080 tx=0x7f10b000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.541+0000 7f10b4ff9640 1 -- 172.21.15.70:0/114879106 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0016420 con 0x7f10b80c3360 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.541+0000 7f10b4ff9640 1 -- 172.21.15.70:0/114879106 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b00165a0 con 0x7f10b80c3360 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.541+0000 7f10bd97c640 1 -- 172.21.15.70:0/114879106 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b815dcc0 con 0x7f10b80c3360 2023-10-08T07:15:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.541+0000 7f10b4ff9640 1 -- 172.21.15.70:0/114879106 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0016950 con 0x7f10b80c3360 2023-10-08T07:15:47.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.541+0000 7f10bd97c640 1 -- 172.21.15.70:0/114879106 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b814d3e0 con 0x7f10b80c3360 2023-10-08T07:15:47.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.542+0000 7f10bd97c640 1 -- 172.21.15.70:0/114879106 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b815db20 con 0x7f10b80c3360 2023-10-08T07:15:47.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.543+0000 7f10b4ff9640 1 -- 172.21.15.70:0/114879106 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f10b0016ab0 con 0x7f10b80c3360 2023-10-08T07:15:47.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.544+0000 7f10b4ff9640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f109806b0b0 0x7f109806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:47.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.544+0000 7f10b4ff9640 1 -- 172.21.15.70:0/114879106 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f10b008aa00 con 0x7f10b80c3360 2023-10-08T07:15:47.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.545+0000 7f10b6ffd640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f109806b0b0 0x7f109806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:47.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.546+0000 7f10b6ffd640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f109806b0b0 0x7f109806d570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f10a8034000 tx=0x7f10a8009030 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:47.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.548+0000 7f10b4ff9640 1 -- 172.21.15.70:0/114879106 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10b815db20 con 0x7f10b80c3360 2023-10-08T07:15:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:47.763+0000 7f10bd97c640 1 -- 172.21.15.70:0/114879106 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f10b815e1e0 con 0x7f10b80c3360 2023-10-08T07:15:48.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.435+0000 7f10b4ff9640 1 -- 172.21.15.70:0/114879106 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v139) v1 ==== 118+0+0 (secure 0 0 0) 0x7f10b815e1e0 con 0x7f10b80c3360 2023-10-08T07:15:48.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.438+0000 7f10bd97c640 1 -- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f109806b0b0 msgr2=0x7f109806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.438+0000 7f10bd97c640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f109806b0b0 0x7f109806d570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f10a8034000 tx=0x7f10a8009030 comp rx=0 tx=0).stop 2023-10-08T07:15:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.438+0000 7f10bd97c640 1 -- 172.21.15.70:0/114879106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80c3360 msgr2=0x7f10b80cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.438+0000 7f10bd97c640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80c3360 0x7f10b80cd120 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f10b80cb080 tx=0x7f10b000f520 comp rx=0 tx=0).stop 2023-10-08T07:15:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.439+0000 7f10bd97c640 1 -- 172.21.15.70:0/114879106 shutdown_connections 2023-10-08T07:15:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.439+0000 7f10bd97c640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f109806b0b0 0x7f109806d570 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.439+0000 7f10bd97c640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80c3360 0x7f10b80cd120 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.439+0000 7f10bd97c640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f10b80c2ee0 0x7f10b80ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.439+0000 7f10bd97c640 1 --2- 172.21.15.70:0/114879106 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f10a4003970 0x7f10b814cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.439+0000 7f10bd97c640 1 -- 172.21.15.70:0/114879106 >> 172.21.15.70:0/114879106 conn(0x7f10b806a5d0 msgr2=0x7f10b8070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.439+0000 7f10bd97c640 1 -- 172.21.15.70:0/114879106 shutdown_connections 2023-10-08T07:15:48.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.439+0000 7f10bd97c640 1 -- 172.21.15.70:0/114879106 wait complete. 2023-10-08T07:15:48.460 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-08T07:15:48.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T07:15:48.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.675+0000 7f7c6e92c640 1 Processor -- start 2023-10-08T07:15:48.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.675+0000 7f7c6e92c640 1 -- start start 2023-10-08T07:15:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.677+0000 7f7c6e92c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 0x7f7c680b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.677+0000 7f7c6e92c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7c6806dfc0 0x7f7c68070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.677+0000 7f7c6e92c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c680b2620 con 0x7f7c68071070 2023-10-08T07:15:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.677+0000 7f7c6e92c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c680b2060 con 0x7f7c6806dfc0 2023-10-08T07:15:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.677+0000 7f7c6e92c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c680b2340 con 0x7f7c680afbe0 2023-10-08T07:15:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.677+0000 7f7c67fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 0x7f7c680b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.678+0000 7f7c67fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 0x7f7c680b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:41920/0 (socket says 172.21.15.70:41920) 2023-10-08T07:15:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.678+0000 7f7c67fff640 1 -- 172.21.15.70:0/4225510194 learned_addr learned my addr 172.21.15.70:0/4225510194 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.678+0000 7f7c5ffff640 1 --2- 172.21.15.70:0/4225510194 >> v2:172.21.15.181:3300/0 conn(0x7f7c6806dfc0 0x7f7c68070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.678+0000 7f7c5ffff640 1 -- 172.21.15.70:0/4225510194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 msgr2=0x7f7c680b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.678+0000 7f7c5ffff640 1 --2- 172.21.15.70:0/4225510194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 0x7f7c680b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.678+0000 7f7c5ffff640 1 -- 172.21.15.70:0/4225510194 >> v1:172.21.15.181:6789/0 conn(0x7f7c68071070 legacy=0x7f7c680cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.678+0000 7f7c5ffff640 1 -- 172.21.15.70:0/4225510194 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c68070350 con 0x7f7c6806dfc0 2023-10-08T07:15:48.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.680+0000 7f7c5ffff640 1 --2- 172.21.15.70:0/4225510194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c6806dfc0 0x7f7c68070b30 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7c500097a0 tx=0x7f7c5002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c82766d2050b376 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:48.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.680+0000 7f7c677fe640 1 -- 172.21.15.70:0/4225510194 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50031780 con 0x7f7c6806dfc0 2023-10-08T07:15:48.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.680+0000 7f7c677fe640 1 -- 172.21.15.70:0/4225510194 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c500318e0 con 0x7f7c6806dfc0 2023-10-08T07:15:48.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.680+0000 7f7c677fe640 1 -- 172.21.15.70:0/4225510194 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50031c70 con 0x7f7c6806dfc0 2023-10-08T07:15:48.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.681+0000 7f7c6e92c640 1 -- 172.21.15.70:0/4225510194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c6806dfc0 msgr2=0x7f7c68070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:48.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.681+0000 7f7c6e92c640 1 --2- 172.21.15.70:0/4225510194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c6806dfc0 0x7f7c68070b30 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7c500097a0 tx=0x7f7c5002b820 comp rx=0 tx=0).stop 2023-10-08T07:15:48.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.681+0000 7f7c6e92c640 1 -- 172.21.15.70:0/4225510194 shutdown_connections 2023-10-08T07:15:48.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.681+0000 7f7c6e92c640 1 --2- 172.21.15.70:0/4225510194 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c6806dfc0 0x7f7c68070b30 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:48.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.681+0000 7f7c6e92c640 1 --2- 172.21.15.70:0/4225510194 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 0x7f7c680b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:48.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.681+0000 7f7c6e92c640 1 -- 172.21.15.70:0/4225510194 >> 172.21.15.70:0/4225510194 conn(0x7f7c6806a5d0 msgr2=0x7f7c680b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:48.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.681+0000 7f7c6e92c640 1 -- 172.21.15.70:0/4225510194 shutdown_connections 2023-10-08T07:15:48.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.681+0000 7f7c6e92c640 1 -- 172.21.15.70:0/4225510194 wait complete. 2023-10-08T07:15:48.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.683+0000 7f7c6e92c640 1 Processor -- start 2023-10-08T07:15:48.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.684+0000 7f7c6e92c640 1 -- start start 2023-10-08T07:15:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c6e92c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c6806dfc0 0x7f7c6815e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c6e92c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c68071070 0x7f7c680cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c6e92c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 0x7f7c680ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c6e92c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6814de70 con 0x7f7c68071070 2023-10-08T07:15:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c6e92c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815b670 con 0x7f7c6806dfc0 2023-10-08T07:15:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c6e92c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c681562e0 con 0x7f7c680afbe0 2023-10-08T07:15:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c68071070 0x7f7c680cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c5ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c68071070 0x7f7c680cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51410/0 (socket says 172.21.15.70:51410) 2023-10-08T07:15:48.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c5ffff640 1 -- 172.21.15.70:0/519020017 learned_addr learned my addr 172.21.15.70:0/519020017 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c67fff640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c6806dfc0 0x7f7c6815e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c6cec9640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 0x7f7c680ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c5ffff640 1 -- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 msgr2=0x7f7c680ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c5ffff640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 0x7f7c680ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c5ffff640 1 -- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c6806dfc0 msgr2=0x7f7c6815e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.685+0000 7f7c5ffff640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c6806dfc0 0x7f7c6815e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.686+0000 7f7c5ffff640 1 -- 172.21.15.70:0/519020017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50009400 con 0x7f7c68071070 2023-10-08T07:15:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.686+0000 7f7c6cec9640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 0x7f7c680ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:48.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.686+0000 7f7c5ffff640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c68071070 0x7f7c680cc640 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f7c50000c00 tx=0x7f7c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:48.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.686+0000 7f7c657fa640 1 -- 172.21.15.70:0/519020017 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50032820 con 0x7f7c68071070 2023-10-08T07:15:48.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.686+0000 7f7c657fa640 1 -- 172.21.15.70:0/519020017 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c500329a0 con 0x7f7c68071070 2023-10-08T07:15:48.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.686+0000 7f7c657fa640 1 -- 172.21.15.70:0/519020017 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50032d50 con 0x7f7c68071070 2023-10-08T07:15:48.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.686+0000 7f7c6e92c640 1 -- 172.21.15.70:0/519020017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c681565c0 con 0x7f7c68071070 2023-10-08T07:15:48.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.686+0000 7f7c6e92c640 1 -- 172.21.15.70:0/519020017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c680ce930 con 0x7f7c68071070 2023-10-08T07:15:48.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.687+0000 7f7c6e92c640 1 -- 172.21.15.70:0/519020017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c680ceb20 con 0x7f7c68071070 2023-10-08T07:15:48.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.688+0000 7f7c657fa640 1 -- 172.21.15.70:0/519020017 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f7c5004e450 con 0x7f7c68071070 2023-10-08T07:15:48.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.689+0000 7f7c657fa640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7c4806b0b0 0x7f7c4806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:48.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.689+0000 7f7c657fa640 1 -- 172.21.15.70:0/519020017 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f7c500abcc0 con 0x7f7c68071070 2023-10-08T07:15:48.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.690+0000 7f7c67fff640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7c4806b0b0 0x7f7c4806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:48.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.691+0000 7f7c67fff640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7c4806b0b0 0x7f7c4806d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009b60 tx=0x7f7c58008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:48.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.693+0000 7f7c657fa640 1 -- 172.21.15.70:0/519020017 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c500b2080 con 0x7f7c68071070 2023-10-08T07:15:48.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:48.919+0000 7f7c6e92c640 1 -- 172.21.15.70:0/519020017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7c6815b7e0 con 0x7f7c68071070 2023-10-08T07:15:49.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.447+0000 7f7c657fa640 1 -- 172.21.15.70:0/519020017 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v140) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7c6815b7e0 con 0x7f7c68071070 2023-10-08T07:15:49.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 -- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7c4806b0b0 msgr2=0x7f7c4806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:49.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7c4806b0b0 0x7f7c4806d570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009b60 tx=0x7f7c58008040 comp rx=0 tx=0).stop 2023-10-08T07:15:49.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 -- 172.21.15.70:0/519020017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c68071070 msgr2=0x7f7c680cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:49.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c68071070 0x7f7c680cc640 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f7c50000c00 tx=0x7f7c500046c0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 -- 172.21.15.70:0/519020017 shutdown_connections 2023-10-08T07:15:49.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f7c4806b0b0 0x7f7c4806d570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7c680afbe0 0x7f7c680ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c68071070 0x7f7c680cc640 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 --2- 172.21.15.70:0/519020017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f7c6806dfc0 0x7f7c6815e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 -- 172.21.15.70:0/519020017 >> 172.21.15.70:0/519020017 conn(0x7f7c6806a5d0 msgr2=0x7f7c680c6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:49.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 -- 172.21.15.70:0/519020017 shutdown_connections 2023-10-08T07:15:49.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.451+0000 7f7c6e92c640 1 -- 172.21.15.70:0/519020017 wait complete. 2023-10-08T07:15:49.472 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-08T07:15:49.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:49.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.688+0000 7f629f852640 1 Processor -- start 2023-10-08T07:15:49.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.689+0000 7f629f852640 1 -- start start 2023-10-08T07:15:49.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.690+0000 7f629f852640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f629806ad00 0x7f629806b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:49.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.690+0000 7f629f852640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62980c9850 con 0x7f6298070db0 2023-10-08T07:15:49.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.690+0000 7f629f852640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629806a870 con 0x7f6298071190 2023-10-08T07:15:49.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.690+0000 7f629f852640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629815eb90 con 0x7f629806ad00 2023-10-08T07:15:49.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.690+0000 7f629d5ee640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6298070db0 0x7f629815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52440/0 (socket says 172.21.15.70:52440) 2023-10-08T07:15:49.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.690+0000 7f629d5ee640 1 -- 172.21.15.70:0/963867936 learned_addr learned my addr 172.21.15.70:0/963867936 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:49.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f629ddef640 1 --2- 172.21.15.70:0/963867936 >> v2:172.21.15.181:3300/0 conn(0x7f629806ad00 0x7f629806b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:49.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992528800 0 0) 0x7f62980c9850 con 0x7f6298070db0 2023-10-08T07:15:49.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6280003520 con 0x7f6298070db0 2023-10-08T07:15:49.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127922030 0 0) 0x7f6280003520 con 0x7f6298070db0 2023-10-08T07:15:49.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 >> v2:172.21.15.181:3300/0 conn(0x7f629806ad00 msgr2=0x7f629806b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 --2- 172.21.15.70:0/963867936 >> v2:172.21.15.181:3300/0 conn(0x7f629806ad00 0x7f629806b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 >> v1:172.21.15.181:6789/0 conn(0x7f6298071190 legacy=0x7f629815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62980cd640 con 0x7f6298070db0 2023-10-08T07:15:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f62880019f0 con 0x7f6298070db0 2023-10-08T07:15:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 >> v1:172.21.15.70:6789/0 conn(0x7f6298070db0 legacy=0x7f629815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6280003970 0x7f6280003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62800047c0 0x7f6280004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6280005180 0x7f628000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629815eb90 con 0x7f62800047c0 2023-10-08T07:15:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628000c030 con 0x7f6280003970 2023-10-08T07:15:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628000c1f0 con 0x7f6280005180 2023-10-08T07:15:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.691+0000 7f629cded640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62800047c0 0x7f6280004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.692+0000 7f629d5ee640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6280003970 0x7f6280003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.692+0000 7f629cded640 1 -- 172.21.15.70:0/963867936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6280005180 msgr2=0x7f628000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.692+0000 7f629ddef640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6280005180 0x7f628000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.692+0000 7f629cded640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6280005180 0x7f628000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.692+0000 7f629cded640 1 -- 172.21.15.70:0/963867936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6280003970 msgr2=0x7f6280003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.692+0000 7f629cded640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6280003970 0x7f6280003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.692+0000 7f629cded640 1 -- 172.21.15.70:0/963867936 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6280003520 con 0x7f62800047c0 2023-10-08T07:15:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.692+0000 7f629d5ee640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6280003970 0x7f6280003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.692+0000 7f629ddef640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6280005180 0x7f628000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.692+0000 7f629ddef640 1 -- 172.21.15.70:0/963867936 reap_dead start 2023-10-08T07:15:49.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.693+0000 7f629cded640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62800047c0 0x7f6280004c40 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f628c001750 tx=0x7f62800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a803509a458ee6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:49.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.693+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c0052d0 con 0x7f62800047c0 2023-10-08T07:15:49.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.693+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628c02e050 con 0x7f62800047c0 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.693+0000 7f6297fff640 1 -- 172.21.15.70:0/963867936 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c003830 con 0x7f62800047c0 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.694+0000 7f629f852640 1 -- 172.21.15.70:0/963867936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62800047c0 msgr2=0x7f6280004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.694+0000 7f629f852640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62800047c0 0x7f6280004c40 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7f628c001750 tx=0x7f62800057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.694+0000 7f629f852640 1 -- 172.21.15.70:0/963867936 shutdown_connections 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.694+0000 7f629f852640 1 --2- 172.21.15.70:0/963867936 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62800047c0 0x7f6280004c40 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.694+0000 7f629f852640 1 -- 172.21.15.70:0/963867936 >> 172.21.15.70:0/963867936 conn(0x7f62980b1480 msgr2=0x7f629806cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.694+0000 7f629f852640 1 -- 172.21.15.70:0/963867936 shutdown_connections 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.694+0000 7f629f852640 1 -- 172.21.15.70:0/963867936 wait complete. 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.696+0000 7f629f852640 1 Processor -- start 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.696+0000 7f629f852640 1 -- start start 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.697+0000 7f629f852640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62800047c0 0x7f62980c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:49.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.697+0000 7f629f852640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f629814f500 0x7f62980c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.697+0000 7f629f852640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62981f5820 0x7f62981f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.697+0000 7f629f852640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62981507d0 con 0x7f629814f500 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.697+0000 7f629f852640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62981f61e0 con 0x7f62981f5820 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.697+0000 7f629f852640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c8840 con 0x7f62800047c0 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.697+0000 7f629cded640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f629814f500 0x7f62980c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.697+0000 7f629cded640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f629814f500 0x7f62980c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51420/0 (socket says 172.21.15.70:51420) 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.697+0000 7f629cded640 1 -- 172.21.15.70:0/2783753216 learned_addr learned my addr 172.21.15.70:0/2783753216 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629ddef640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62981f5820 0x7f62981f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629d5ee640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62800047c0 0x7f62980c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629cded640 1 -- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62800047c0 msgr2=0x7f62980c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629cded640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62800047c0 0x7f62980c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629cded640 1 -- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62981f5820 msgr2=0x7f62981f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629cded640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62981f5820 0x7f62981f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629cded640 1 -- 172.21.15.70:0/2783753216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62981534b0 con 0x7f629814f500 2023-10-08T07:15:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629ddef640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62981f5820 0x7f62981f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629d5ee640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62800047c0 0x7f62980c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629cded640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f629814f500 0x7f62980c98c0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7f628c005040 tx=0x7f628c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f6295ffb640 1 -- 172.21.15.70:0/2783753216 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c007a30 con 0x7f629814f500 2023-10-08T07:15:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f6295ffb640 1 -- 172.21.15.70:0/2783753216 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628c0069e0 con 0x7f629814f500 2023-10-08T07:15:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f6295ffb640 1 -- 172.21.15.70:0/2783753216 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c006d70 con 0x7f629814f500 2023-10-08T07:15:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.698+0000 7f629f852640 1 -- 172.21.15.70:0/2783753216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62981504f0 con 0x7f629814f500 2023-10-08T07:15:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.699+0000 7f629f852640 1 -- 172.21.15.70:0/2783753216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62981531d0 con 0x7f629814f500 2023-10-08T07:15:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.700+0000 7f629f852640 1 -- 172.21.15.70:0/2783753216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629806a1a0 con 0x7f629814f500 2023-10-08T07:15:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.701+0000 7f6295ffb640 1 -- 172.21.15.70:0/2783753216 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f628c0035f0 con 0x7f629814f500 2023-10-08T07:15:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.701+0000 7f6295ffb640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f628406b0e0 0x7f628406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.702+0000 7f6295ffb640 1 -- 172.21.15.70:0/2783753216 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f628c0abc70 con 0x7f629814f500 2023-10-08T07:15:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.702+0000 7f629d5ee640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f628406b0e0 0x7f628406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.703+0000 7f629d5ee640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f628406b0e0 0x7f628406d5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6288001580 tx=0x7f6288009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.706+0000 7f6295ffb640 1 -- 172.21.15.70:0/2783753216 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f628c076830 con 0x7f629814f500 2023-10-08T07:15:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.933+0000 7f629f852640 1 -- 172.21.15.70:0/2783753216 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6298067490 con 0x7f629814f500 2023-10-08T07:15:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.934+0000 7f6295ffb640 1 -- 172.21.15.70:0/2783753216 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4923 (secure 0 0 0) 0x7f628c04b020 con 0x7f629814f500 2023-10-08T07:15:49.935 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:49.935 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":140,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/949448327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":949448327},{"type":"v1","addr":"172.21.15.70:6835","nonce":949448327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":8111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2423877406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2423877406},{"type":"v1","addr":"172.21.15.181:6837","nonce":2423877406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/247109890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":247109890},{"type":"v1","addr":"172.21.15.181:6836","nonce":247109890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:15:45.890217+0000","modified":"2023-10-08T07:15:48.920991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12645},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12645":{"gid":12645,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/753129368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":753129368},{"type":"v1","addr":"172.21.15.70:6834","nonce":753129368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-08T07:15:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 140 2023-10-08T07:15:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.936+0000 7f629f852640 1 -- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f628406b0e0 msgr2=0x7f628406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.936+0000 7f629f852640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f628406b0e0 0x7f628406d5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6288001580 tx=0x7f6288009040 comp rx=0 tx=0).stop 2023-10-08T07:15:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.936+0000 7f629f852640 1 -- 172.21.15.70:0/2783753216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f629814f500 msgr2=0x7f62980c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.936+0000 7f629f852640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f629814f500 0x7f62980c98c0 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7f628c005040 tx=0x7f628c001120 comp rx=0 tx=0).stop 2023-10-08T07:15:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.937+0000 7f629f852640 1 -- 172.21.15.70:0/2783753216 shutdown_connections 2023-10-08T07:15:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.937+0000 7f629f852640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f628406b0e0 0x7f628406d5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.937+0000 7f629f852640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f62981f5820 0x7f62981f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.937+0000 7f629f852640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f629814f500 0x7f62980c98c0 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.937+0000 7f629f852640 1 --2- 172.21.15.70:0/2783753216 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f62800047c0 0x7f62980c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.937+0000 7f629f852640 1 -- 172.21.15.70:0/2783753216 >> 172.21.15.70:0/2783753216 conn(0x7f62980b1480 msgr2=0x7f629815a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.937+0000 7f629f852640 1 -- 172.21.15.70:0/2783753216 shutdown_connections 2023-10-08T07:15:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:49.937+0000 7f629f852640 1 -- 172.21.15.70:0/2783753216 wait complete. 2023-10-08T07:15:49.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:15:50.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.170+0000 7f57e2202640 1 Processor -- start 2023-10-08T07:15:50.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.170+0000 7f57e2202640 1 -- start start 2023-10-08T07:15:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.172+0000 7f57e2202640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f57dc06d4c0 0x7f57dc162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.172+0000 7f57e2202640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57dc0ae140 con 0x7f57dc0cb150 2023-10-08T07:15:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.172+0000 7f57e2202640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57dc0ccf20 con 0x7f57dc0add60 2023-10-08T07:15:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.172+0000 7f57e2202640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc161f10 con 0x7f57dc06d4c0 2023-10-08T07:15:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.172+0000 7f57d3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f57dc06d4c0 0x7f57dc162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.172+0000 7f57d3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f57dc06d4c0 0x7f57dc162330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51434/0 (socket says 172.21.15.70:51434) 2023-10-08T07:15:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.172+0000 7f57d3fff640 1 -- 172.21.15.70:0/1574924223 learned_addr learned my addr 172.21.15.70:0/1574924223 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.173+0000 7f57d3fff640 1 -- 172.21.15.70:0/1574924223 >> v1:172.21.15.181:6789/0 conn(0x7f57dc0add60 legacy=0x7f57dc0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.173+0000 7f57d3fff640 1 -- 172.21.15.70:0/1574924223 >> v1:172.21.15.70:6789/0 conn(0x7f57dc0cb150 legacy=0x7f57dc164d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.173+0000 7f57d3fff640 1 -- 172.21.15.70:0/1574924223 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc0cd1c0 con 0x7f57dc06d4c0 2023-10-08T07:15:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.174+0000 7f57d3fff640 1 --2- 172.21.15.70:0/1574924223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc06d4c0 0x7f57dc162330 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009870 tx=0x7f57c402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cbb8984e59caed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.174+0000 7f57daffd640 1 -- 172.21.15.70:0/1574924223 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4031870 con 0x7f57dc06d4c0 2023-10-08T07:15:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.174+0000 7f57daffd640 1 -- 172.21.15.70:0/1574924223 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c40319d0 con 0x7f57dc06d4c0 2023-10-08T07:15:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.174+0000 7f57daffd640 1 -- 172.21.15.70:0/1574924223 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4031d60 con 0x7f57dc06d4c0 2023-10-08T07:15:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.174+0000 7f57e2202640 1 -- 172.21.15.70:0/1574924223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc06d4c0 msgr2=0x7f57dc162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.174+0000 7f57e2202640 1 --2- 172.21.15.70:0/1574924223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc06d4c0 0x7f57dc162330 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009870 tx=0x7f57c402b910 comp rx=0 tx=0).stop 2023-10-08T07:15:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.175+0000 7f57e2202640 1 -- 172.21.15.70:0/1574924223 shutdown_connections 2023-10-08T07:15:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.175+0000 7f57e2202640 1 --2- 172.21.15.70:0/1574924223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc06d4c0 0x7f57dc162330 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.175+0000 7f57e2202640 1 -- 172.21.15.70:0/1574924223 >> 172.21.15.70:0/1574924223 conn(0x7f57dc06a380 msgr2=0x7f57dc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.175+0000 7f57e2202640 1 -- 172.21.15.70:0/1574924223 shutdown_connections 2023-10-08T07:15:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.175+0000 7f57e2202640 1 -- 172.21.15.70:0/1574924223 wait complete. 2023-10-08T07:15:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.176+0000 7f57e2202640 1 Processor -- start 2023-10-08T07:15:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.176+0000 7f57e2202640 1 -- start start 2023-10-08T07:15:50.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.177+0000 7f57e2202640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57dc06d4c0 0x7f57dc156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.177+0000 7f57e2202640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57dc0add60 0x7f57dc159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.177+0000 7f57e2202640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc0cb150 0x7f57dc1fa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.177+0000 7f57e2202640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc0aeaf0 con 0x7f57dc0cb150 2023-10-08T07:15:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.177+0000 7f57e2202640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc1599a0 con 0x7f57dc06d4c0 2023-10-08T07:15:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.177+0000 7f57e2202640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc156250 con 0x7f57dc0add60 2023-10-08T07:15:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.177+0000 7f57dbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc0cb150 0x7f57dc1fa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.178+0000 7f57dbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc0cb150 0x7f57dc1fa210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51450/0 (socket says 172.21.15.70:51450) 2023-10-08T07:15:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.178+0000 7f57dbfff640 1 -- 172.21.15.70:0/1376577434 learned_addr learned my addr 172.21.15.70:0/1376577434 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.178+0000 7f57db7fe640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57dc06d4c0 0x7f57dc156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.178+0000 7f57d3fff640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57dc0add60 0x7f57dc159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.178+0000 7f57dbfff640 1 -- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57dc0add60 msgr2=0x7f57dc159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.178+0000 7f57dbfff640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57dc0add60 0x7f57dc159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.178+0000 7f57dbfff640 1 -- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57dc06d4c0 msgr2=0x7f57dc156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.178+0000 7f57dbfff640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57dc06d4c0 0x7f57dc156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.178+0000 7f57dbfff640 1 -- 172.21.15.70:0/1376577434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c40094d0 con 0x7f57dc0cb150 2023-10-08T07:15:50.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.178+0000 7f57dbfff640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc0cb150 0x7f57dc1fa210 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7f57d400c7d0 tx=0x7f57d400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:50.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.179+0000 7f57d8ff9640 1 -- 172.21.15.70:0/1376577434 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4003b70 con 0x7f57dc0cb150 2023-10-08T07:15:50.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.179+0000 7f57d8ff9640 1 -- 172.21.15.70:0/1376577434 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d4003cf0 con 0x7f57dc0cb150 2023-10-08T07:15:50.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.179+0000 7f57e2202640 1 -- 172.21.15.70:0/1376577434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc15a0d0 con 0x7f57dc0cb150 2023-10-08T07:15:50.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.179+0000 7f57d8ff9640 1 -- 172.21.15.70:0/1376577434 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d400aa90 con 0x7f57dc0cb150 2023-10-08T07:15:50.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.179+0000 7f57e2202640 1 -- 172.21.15.70:0/1376577434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc0ae430 con 0x7f57dc0cb150 2023-10-08T07:15:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.180+0000 7f57e2202640 1 -- 172.21.15.70:0/1376577434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57dc067490 con 0x7f57dc0cb150 2023-10-08T07:15:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.181+0000 7f57d8ff9640 1 -- 172.21.15.70:0/1376577434 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f57d400abf0 con 0x7f57dc0cb150 2023-10-08T07:15:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.182+0000 7f57d8ff9640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f57bc06b0b0 0x7f57bc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.182+0000 7f57d8ff9640 1 -- 172.21.15.70:0/1376577434 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f57d4089a70 con 0x7f57dc0cb150 2023-10-08T07:15:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.182+0000 7f57db7fe640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f57bc06b0b0 0x7f57bc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.183+0000 7f57db7fe640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f57bc06b0b0 0x7f57bc06d570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007fd0 tx=0x7f57cc009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:50.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.186+0000 7f57d8ff9640 1 -- 172.21.15.70:0/1376577434 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57d40545d0 con 0x7f57dc0cb150 2023-10-08T07:15:50.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.373+0000 7f57e2202640 1 -- 172.21.15.70:0/1376577434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57dc0b0ef0 con 0x7f57dc0cb150 2023-10-08T07:15:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.375+0000 7f57d8ff9640 1 -- 172.21.15.70:0/1376577434 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v276) v1 ==== 74+0+16956 (secure 0 0 0) 0x7f57d4003e60 con 0x7f57dc0cb150 2023-10-08T07:15:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:50.377 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":276,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:15:47.399930+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:15:43.819948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-08T07:15:44.895969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/1513601554":"2023-10-08T08:15:42.823407+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:15:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 -- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f57bc06b0b0 msgr2=0x7f57bc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f57bc06b0b0 0x7f57bc06d570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f57cc007fd0 tx=0x7f57cc009040 comp rx=0 tx=0).stop 2023-10-08T07:15:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 -- 172.21.15.70:0/1376577434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc0cb150 msgr2=0x7f57dc1fa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc0cb150 0x7f57dc1fa210 secure :-1 s=READY pgs=2893 cs=0 l=1 rev1=1 crypto rx=0x7f57d400c7d0 tx=0x7f57d400cc90 comp rx=0 tx=0).stop 2023-10-08T07:15:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 -- 172.21.15.70:0/1376577434 shutdown_connections 2023-10-08T07:15:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f57bc06b0b0 0x7f57bc06d570 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57dc0cb150 0x7f57dc1fa210 unknown :-1 s=CLOSED pgs=2893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f57dc0add60 0x7f57dc159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 --2- 172.21.15.70:0/1376577434 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f57dc06d4c0 0x7f57dc156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 -- 172.21.15.70:0/1376577434 >> 172.21.15.70:0/1376577434 conn(0x7f57dc06a380 msgr2=0x7f57dc15fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 -- 172.21.15.70:0/1376577434 shutdown_connections 2023-10-08T07:15:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.378+0000 7f57e2202640 1 -- 172.21.15.70:0/1376577434 wait complete. 2023-10-08T07:15:50.397 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T07:15:50.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:15:50.458 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-10-08T07:15:50.458+0000 7f8476b22640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:15:50.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.610+0000 7fcc0b30c640 1 Processor -- start 2023-10-08T07:15:50.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.610+0000 7fcc0b30c640 1 -- start start 2023-10-08T07:15:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc0b30c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcc0415ab30 0x7fcc0415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc0b30c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc040adc70 0x7fcc040b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc0b30c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0415f7d0 con 0x7fcc0415af10 2023-10-08T07:15:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc0b30c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0415f210 con 0x7fcc0415ab30 2023-10-08T07:15:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc0b30c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0415f4f0 con 0x7fcc040adc70 2023-10-08T07:15:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc0ab0b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcc0415af10 0x7fcc040701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52456/0 (socket says 172.21.15.70:52456) 2023-10-08T07:15:50.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc0ab0b640 1 -- 172.21.15.70:0/799105741 learned_addr learned my addr 172.21.15.70:0/799105741 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:50.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09b09640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc040adc70 0x7fcc040b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc0a30a640 1 --2- 172.21.15.70:0/799105741 >> v2:172.21.15.181:3300/0 conn(0x7fcc0415ab30 0x7fcc0415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233896537 0 0) 0x7fcc0415f7d0 con 0x7fcc0415af10 2023-10-08T07:15:50.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf0003520 con 0x7fcc0415af10 2023-10-08T07:15:50.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2369188918 0 0) 0x7fcbf0003520 con 0x7fcc0415af10 2023-10-08T07:15:50.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc040adc70 msgr2=0x7fcc040b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:50.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc040adc70 0x7fcc040b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 >> v2:172.21.15.181:3300/0 conn(0x7fcc0415ab30 msgr2=0x7fcc0415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:50.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 --2- 172.21.15.70:0/799105741 >> v2:172.21.15.181:3300/0 conn(0x7fcc0415ab30 0x7fcc0415e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0406c550 con 0x7fcc0415af10 2023-10-08T07:15:50.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fcbfc001a10 con 0x7fcc0415af10 2023-10-08T07:15:50.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 >> v1:172.21.15.70:6789/0 conn(0x7fcc0415af10 legacy=0x7fcc040701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbf0003970 0x7fcbf0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc040adc70 0x7fcbf00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0004d00 0x7fcbf0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0415f210 con 0x7fcbf0003970 2023-10-08T07:15:50.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0415f4f0 con 0x7fcbf0004d00 2023-10-08T07:15:50.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf000b9b0 con 0x7fcc040adc70 2023-10-08T07:15:50.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.612+0000 7fcc0a30a640 1 --2- 172.21.15.70:0/799105741 >> v2:172.21.15.181:3300/0 conn(0x7fcc0415ab30 0x7fcc0415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:15:50.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.613+0000 7fcc0a30a640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbf0003970 0x7fcbf0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.613+0000 7fcc0ab0b640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0004d00 0x7fcbf0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.613+0000 7fcc09b09640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc040adc70 0x7fcbf00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.613+0000 7fcc0a30a640 1 -- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc040adc70 msgr2=0x7fcbf00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:50.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.613+0000 7fcc0a30a640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc040adc70 0x7fcbf00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.613+0000 7fcc0a30a640 1 -- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0004d00 msgr2=0x7fcbf0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:50.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.613+0000 7fcc0a30a640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0004d00 0x7fcbf0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.613+0000 7fcc0a30a640 1 -- 172.21.15.70:0/799105741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0003520 con 0x7fcbf0003970 2023-10-08T07:15:50.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.613+0000 7fcc09b09640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc040adc70 0x7fcbf00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:50.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.613+0000 7fcc0ab0b640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0004d00 0x7fcbf0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:50.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.614+0000 7fcc0a30a640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbf0003970 0x7fcbf0003d90 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098a0 tx=0x7fcbf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c64762bfdac448 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:50.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.614+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00037600 con 0x7fcbf0003970 2023-10-08T07:15:50.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.614+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc00037760 con 0x7fcbf0003970 2023-10-08T07:15:50.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.614+0000 7fcc09308640 1 -- 172.21.15.70:0/799105741 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00037ad0 con 0x7fcbf0003970 2023-10-08T07:15:50.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.615+0000 7fcc0b30c640 1 -- 172.21.15.70:0/799105741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbf0003970 msgr2=0x7fcbf0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.615+0000 7fcc0b30c640 1 --2- 172.21.15.70:0/799105741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbf0003970 0x7fcbf0003d90 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098a0 tx=0x7fcbf00057f0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.615+0000 7fcc0ab0b640 1 -- 172.21.15.70:0/799105741 reap_dead start 2023-10-08T07:15:50.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.615+0000 7fcc0b30c640 1 -- 172.21.15.70:0/799105741 shutdown_connections 2023-10-08T07:15:50.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.615+0000 7fcc0b30c640 1 -- 172.21.15.70:0/799105741 >> 172.21.15.70:0/799105741 conn(0x7fcc040cb0c0 msgr2=0x7fcc040cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:50.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.615+0000 7fcc0b30c640 1 -- 172.21.15.70:0/799105741 shutdown_connections 2023-10-08T07:15:50.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.615+0000 7fcc0b30c640 1 -- 172.21.15.70:0/799105741 wait complete. 2023-10-08T07:15:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.617+0000 7fcc0b30c640 1 Processor -- start 2023-10-08T07:15:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.617+0000 7fcc0b30c640 1 -- start start 2023-10-08T07:15:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.618+0000 7fcc0b30c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0003970 0x7fcc041554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.618+0000 7fcc0b30c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc04155b90 0x7fcc04154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.618+0000 7fcc0b30c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc040cc210 0x7fcc0415b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.618+0000 7fcc0b30c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040c8170 con 0x7fcc040cc210 2023-10-08T07:15:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.618+0000 7fcc0b30c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040c8450 con 0x7fcbf0003970 2023-10-08T07:15:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.618+0000 7fcc0b30c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040c6120 con 0x7fcc04155b90 2023-10-08T07:15:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.618+0000 7fcc0ab0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc040cc210 0x7fcc0415b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.618+0000 7fcc0ab0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc040cc210 0x7fcc0415b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51460/0 (socket says 172.21.15.70:51460) 2023-10-08T07:15:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.618+0000 7fcc0ab0b640 1 -- 172.21.15.70:0/2401271473 learned_addr learned my addr 172.21.15.70:0/2401271473 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc09b09640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc04155b90 0x7fcc04154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc0a30a640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0003970 0x7fcc041554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc0ab0b640 1 -- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc04155b90 msgr2=0x7fcc04154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc0ab0b640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc04155b90 0x7fcc04154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc0ab0b640 1 -- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0003970 msgr2=0x7fcc041554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc0ab0b640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0003970 0x7fcc041554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc0ab0b640 1 -- 172.21.15.70:0/2401271473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040c6400 con 0x7fcc040cc210 2023-10-08T07:15:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc0a30a640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0003970 0x7fcc041554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc0ab0b640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc040cc210 0x7fcc0415b340 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fcc040b16c0 tx=0x7fcbfc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcbfaffd640 1 -- 172.21.15.70:0/2401271473 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc004d60 con 0x7fcc040cc210 2023-10-08T07:15:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcbfaffd640 1 -- 172.21.15.70:0/2401271473 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc00bb80 con 0x7fcc040cc210 2023-10-08T07:15:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcbfaffd640 1 -- 172.21.15.70:0/2401271473 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc007070 con 0x7fcc040cc210 2023-10-08T07:15:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc0b30c640 1 -- 172.21.15.70:0/2401271473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc040c6920 con 0x7fcc040cc210 2023-10-08T07:15:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.619+0000 7fcc0b30c640 1 -- 172.21.15.70:0/2401271473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc040ab8d0 con 0x7fcc040cc210 2023-10-08T07:15:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.620+0000 7fcc0b30c640 1 -- 172.21.15.70:0/2401271473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc040c65f0 con 0x7fcc040cc210 2023-10-08T07:15:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.621+0000 7fcbfaffd640 1 -- 172.21.15.70:0/2401271473 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fcbfc0071d0 con 0x7fcc040cc210 2023-10-08T07:15:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.622+0000 7fcbfaffd640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcbe806b0b0 0x7fcbe806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.622+0000 7fcbfaffd640 1 -- 172.21.15.70:0/2401271473 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fcbfc089ff0 con 0x7fcc040cc210 2023-10-08T07:15:50.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.623+0000 7fcc0a30a640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcbe806b0b0 0x7fcbe806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:50.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.625+0000 7fcc0a30a640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcbe806b0b0 0x7fcbe806d570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098a0 tx=0x7fcc00031000 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:50.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.627+0000 7fcbfaffd640 1 -- 172.21.15.70:0/2401271473 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcbfc054a10 con 0x7fcc040cc210 2023-10-08T07:15:50.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.808+0000 7fcc0b30c640 1 -- 172.21.15.70:0/2401271473 --> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc040c8000 con 0x7fcbe806b0b0 2023-10-08T07:15:50.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.836+0000 7fcbfaffd640 1 -- 172.21.15.70:0/2401271473 <== mgr.9067 v2:172.21.15.181:6832/440573135 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287559 (secure 0 0 0) 0x7fcc040c8000 con 0x7fcbe806b0b0 2023-10-08T07:15:50.844 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:50.875 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:15:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.842+0000 7fcc0b30c640 1 -- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcbe806b0b0 msgr2=0x7fcbe806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.842+0000 7fcc0b30c640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcbe806b0b0 0x7fcbe806d570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc000098a0 tx=0x7fcc00031000 comp rx=0 tx=0).stop 2023-10-08T07:15:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.842+0000 7fcc0b30c640 1 -- 172.21.15.70:0/2401271473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc040cc210 msgr2=0x7fcc0415b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.842+0000 7fcc0b30c640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc040cc210 0x7fcc0415b340 secure :-1 s=READY pgs=2896 cs=0 l=1 rev1=1 crypto rx=0x7fcc040b16c0 tx=0x7fcbfc00b4a0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.843+0000 7fcc0b30c640 1 -- 172.21.15.70:0/2401271473 shutdown_connections 2023-10-08T07:15:50.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.843+0000 7fcc0b30c640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fcbe806b0b0 0x7fcbe806d570 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.843+0000 7fcc0b30c640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc040cc210 0x7fcc0415b340 unknown :-1 s=CLOSED pgs=2896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.843+0000 7fcc0b30c640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcc04155b90 0x7fcc04154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.843+0000 7fcc0b30c640 1 --2- 172.21.15.70:0/2401271473 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcbf0003970 0x7fcc041554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:50.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.843+0000 7fcc0b30c640 1 -- 172.21.15.70:0/2401271473 >> 172.21.15.70:0/2401271473 conn(0x7fcc040cb0c0 msgr2=0x7fcc040b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:50.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.843+0000 7fcc0b30c640 1 -- 172.21.15.70:0/2401271473 shutdown_connections 2023-10-08T07:15:50.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:50.843+0000 7fcc0b30c640 1 -- 172.21.15.70:0/2401271473 wait complete. 2023-10-08T07:15:50.878 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-10-08T07:15:48.993238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":695,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4899,"ondisk_log_size":4899,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448640,"kb_used_data":28408,"kb_used_omap":12,"kb_used_meta":420083,"kb_avail":754526080,"statfs":{"total":773094113280,"available":772634705920,"internally_reserved":0,"allocated":29089792,"data_stored":15770144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":430165563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":89,"num_read_kb":201,"num_write":63,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.599193"},"pg_stats":[{"pgid":"36.3f","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409729+0000","last_change":"2023-10-08T07:15:46.427641+0000","last_active":"2023-10-08T07:15:47.409729+0000","last_peered":"2023-10-08T07:15:47.409729+0000","last_clean":"2023-10-08T07:15:47.409729+0000","last_became_active":"2023-10-08T07:15:46.427272+0000","last_became_peered":"2023-10-08T07:15:46.427272+0000","last_unstale":"2023-10-08T07:15:47.409729+0000","last_undegraded":"2023-10-08T07:15:47.409729+0000","last_fullsized":"2023-10-08T07:15:47.409729+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:44.950131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3e","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409338+0000","last_change":"2023-10-08T07:15:46.439775+0000","last_active":"2023-10-08T07:15:47.409338+0000","last_peered":"2023-10-08T07:15:47.409338+0000","last_clean":"2023-10-08T07:15:47.409338+0000","last_became_active":"2023-10-08T07:15:46.439462+0000","last_became_peered":"2023-10-08T07:15:46.439462+0000","last_unstale":"2023-10-08T07:15:47.409338+0000","last_undegraded":"2023-10-08T07:15:47.409338+0000","last_fullsized":"2023-10-08T07:15:47.409338+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:03.705356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3d","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406311+0000","last_change":"2023-10-08T07:15:46.443973+0000","last_active":"2023-10-08T07:15:47.406311+0000","last_peered":"2023-10-08T07:15:47.406311+0000","last_clean":"2023-10-08T07:15:47.406311+0000","last_became_active":"2023-10-08T07:15:46.443717+0000","last_became_peered":"2023-10-08T07:15:46.443717+0000","last_unstale":"2023-10-08T07:15:47.406311+0000","last_undegraded":"2023-10-08T07:15:47.406311+0000","last_fullsized":"2023-10-08T07:15:47.406311+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:15.492519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3c","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405935+0000","last_change":"2023-10-08T07:15:46.435801+0000","last_active":"2023-10-08T07:15:47.405935+0000","last_peered":"2023-10-08T07:15:47.405935+0000","last_clean":"2023-10-08T07:15:47.405935+0000","last_became_active":"2023-10-08T07:15:46.434714+0000","last_became_peered":"2023-10-08T07:15:46.434714+0000","last_unstale":"2023-10-08T07:15:47.405935+0000","last_undegraded":"2023-10-08T07:15:47.405935+0000","last_fullsized":"2023-10-08T07:15:47.405935+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:05.593791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3b","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408920+0000","last_change":"2023-10-08T07:15:46.445225+0000","last_active":"2023-10-08T07:15:47.408920+0000","last_peered":"2023-10-08T07:15:47.408920+0000","last_clean":"2023-10-08T07:15:47.408920+0000","last_became_active":"2023-10-08T07:15:46.444956+0000","last_became_peered":"2023-10-08T07:15:46.444956+0000","last_unstale":"2023-10-08T07:15:47.408920+0000","last_undegraded":"2023-10-08T07:15:47.408920+0000","last_fullsized":"2023-10-08T07:15:47.408920+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:39.135783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.3a","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408094+0000","last_change":"2023-10-08T07:15:46.453293+0000","last_active":"2023-10-08T07:15:47.408094+0000","last_peered":"2023-10-08T07:15:47.408094+0000","last_clean":"2023-10-08T07:15:47.408094+0000","last_became_active":"2023-10-08T07:15:46.453012+0000","last_became_peered":"2023-10-08T07:15:46.453012+0000","last_unstale":"2023-10-08T07:15:47.408094+0000","last_undegraded":"2023-10-08T07:15:47.408094+0000","last_fullsized":"2023-10-08T07:15:47.408094+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:11.428056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.39","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599631+0000","last_change":"2023-10-08T07:15:46.442034+0000","last_active":"2023-10-08T07:15:47.599631+0000","last_peered":"2023-10-08T07:15:47.599631+0000","last_clean":"2023-10-08T07:15:47.599631+0000","last_became_active":"2023-10-08T07:15:46.441732+0000","last_became_peered":"2023-10-08T07:15:46.441732+0000","last_unstale":"2023-10-08T07:15:47.599631+0000","last_undegraded":"2023-10-08T07:15:47.599631+0000","last_fullsized":"2023-10-08T07:15:47.599631+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:35:53.021160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.38","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405294+0000","last_change":"2023-10-08T07:15:46.432534+0000","last_active":"2023-10-08T07:15:47.405294+0000","last_peered":"2023-10-08T07:15:47.405294+0000","last_clean":"2023-10-08T07:15:47.405294+0000","last_became_active":"2023-10-08T07:15:46.432257+0000","last_became_peered":"2023-10-08T07:15:46.432257+0000","last_unstale":"2023-10-08T07:15:47.405294+0000","last_undegraded":"2023-10-08T07:15:47.405294+0000","last_fullsized":"2023-10-08T07:15:47.405294+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:18.607729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.37","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600403+0000","last_change":"2023-10-08T07:15:46.449871+0000","last_active":"2023-10-08T07:15:47.600403+0000","last_peered":"2023-10-08T07:15:47.600403+0000","last_clean":"2023-10-08T07:15:47.600403+0000","last_became_active":"2023-10-08T07:15:46.449574+0000","last_became_peered":"2023-10-08T07:15:46.449574+0000","last_unstale":"2023-10-08T07:15:47.600403+0000","last_undegraded":"2023-10-08T07:15:47.600403+0000","last_fullsized":"2023-10-08T07:15:47.600403+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:28.103365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.36","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600616+0000","last_change":"2023-10-08T07:15:46.437670+0000","last_active":"2023-10-08T07:15:47.600616+0000","last_peered":"2023-10-08T07:15:47.600616+0000","last_clean":"2023-10-08T07:15:47.600616+0000","last_became_active":"2023-10-08T07:15:46.437351+0000","last_became_peered":"2023-10-08T07:15:46.437351+0000","last_unstale":"2023-10-08T07:15:47.600616+0000","last_undegraded":"2023-10-08T07:15:47.600616+0000","last_fullsized":"2023-10-08T07:15:47.600616+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:58.208711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.35","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.598936+0000","last_change":"2023-10-08T07:15:46.433741+0000","last_active":"2023-10-08T07:15:47.598936+0000","last_peered":"2023-10-08T07:15:47.598936+0000","last_clean":"2023-10-08T07:15:47.598936+0000","last_became_active":"2023-10-08T07:15:46.433309+0000","last_became_peered":"2023-10-08T07:15:46.433309+0000","last_unstale":"2023-10-08T07:15:47.598936+0000","last_undegraded":"2023-10-08T07:15:47.598936+0000","last_fullsized":"2023-10-08T07:15:47.598936+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:04.135629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.34","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409270+0000","last_change":"2023-10-08T07:15:46.429914+0000","last_active":"2023-10-08T07:15:47.409270+0000","last_peered":"2023-10-08T07:15:47.409270+0000","last_clean":"2023-10-08T07:15:47.409270+0000","last_became_active":"2023-10-08T07:15:46.429451+0000","last_became_peered":"2023-10-08T07:15:46.429451+0000","last_unstale":"2023-10-08T07:15:47.409270+0000","last_undegraded":"2023-10-08T07:15:47.409270+0000","last_fullsized":"2023-10-08T07:15:47.409270+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:10.831065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.33","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405880+0000","last_change":"2023-10-08T07:15:46.437175+0000","last_active":"2023-10-08T07:15:47.405880+0000","last_peered":"2023-10-08T07:15:47.405880+0000","last_clean":"2023-10-08T07:15:47.405880+0000","last_became_active":"2023-10-08T07:15:46.436344+0000","last_became_peered":"2023-10-08T07:15:46.436344+0000","last_unstale":"2023-10-08T07:15:47.405880+0000","last_undegraded":"2023-10-08T07:15:47.405880+0000","last_fullsized":"2023-10-08T07:15:47.405880+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:12:13.068611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.32","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599709+0000","last_change":"2023-10-08T07:15:46.439141+0000","last_active":"2023-10-08T07:15:47.599709+0000","last_peered":"2023-10-08T07:15:47.599709+0000","last_clean":"2023-10-08T07:15:47.599709+0000","last_became_active":"2023-10-08T07:15:46.438825+0000","last_became_peered":"2023-10-08T07:15:46.438825+0000","last_unstale":"2023-10-08T07:15:47.599709+0000","last_undegraded":"2023-10-08T07:15:47.599709+0000","last_fullsized":"2023-10-08T07:15:47.599709+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:30.523440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.31","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405551+0000","last_change":"2023-10-08T07:15:46.433769+0000","last_active":"2023-10-08T07:15:47.405551+0000","last_peered":"2023-10-08T07:15:47.405551+0000","last_clean":"2023-10-08T07:15:47.405551+0000","last_became_active":"2023-10-08T07:15:46.433495+0000","last_became_peered":"2023-10-08T07:15:46.433495+0000","last_unstale":"2023-10-08T07:15:47.405551+0000","last_undegraded":"2023-10-08T07:15:47.405551+0000","last_fullsized":"2023-10-08T07:15:47.405551+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:21:41.048050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.30","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601290+0000","last_change":"2023-10-08T07:15:46.449295+0000","last_active":"2023-10-08T07:15:47.601290+0000","last_peered":"2023-10-08T07:15:47.601290+0000","last_clean":"2023-10-08T07:15:47.601290+0000","last_became_active":"2023-10-08T07:15:46.448966+0000","last_became_peered":"2023-10-08T07:15:46.448966+0000","last_unstale":"2023-10-08T07:15:47.601290+0000","last_undegraded":"2023-10-08T07:15:47.601290+0000","last_fullsized":"2023-10-08T07:15:47.601290+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:00.292656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2f","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601675+0000","last_change":"2023-10-08T07:15:46.441378+0000","last_active":"2023-10-08T07:15:47.601675+0000","last_peered":"2023-10-08T07:15:47.601675+0000","last_clean":"2023-10-08T07:15:47.601675+0000","last_became_active":"2023-10-08T07:15:46.440290+0000","last_became_peered":"2023-10-08T07:15:46.440290+0000","last_unstale":"2023-10-08T07:15:47.601675+0000","last_undegraded":"2023-10-08T07:15:47.601675+0000","last_fullsized":"2023-10-08T07:15:47.601675+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:00.853372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2e","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406513+0000","last_change":"2023-10-08T07:15:46.439217+0000","last_active":"2023-10-08T07:15:47.406513+0000","last_peered":"2023-10-08T07:15:47.406513+0000","last_clean":"2023-10-08T07:15:47.406513+0000","last_became_active":"2023-10-08T07:15:46.438944+0000","last_became_peered":"2023-10-08T07:15:46.438944+0000","last_unstale":"2023-10-08T07:15:47.406513+0000","last_undegraded":"2023-10-08T07:15:47.406513+0000","last_fullsized":"2023-10-08T07:15:47.406513+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:34.626536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2d","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601864+0000","last_change":"2023-10-08T07:15:46.441127+0000","last_active":"2023-10-08T07:15:47.601864+0000","last_peered":"2023-10-08T07:15:47.601864+0000","last_clean":"2023-10-08T07:15:47.601864+0000","last_became_active":"2023-10-08T07:15:46.440513+0000","last_became_peered":"2023-10-08T07:15:46.440513+0000","last_unstale":"2023-10-08T07:15:47.601864+0000","last_undegraded":"2023-10-08T07:15:47.601864+0000","last_fullsized":"2023-10-08T07:15:47.601864+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:03:04.333839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2c","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601997+0000","last_change":"2023-10-08T07:15:46.443804+0000","last_active":"2023-10-08T07:15:47.601997+0000","last_peered":"2023-10-08T07:15:47.601997+0000","last_clean":"2023-10-08T07:15:47.601997+0000","last_became_active":"2023-10-08T07:15:46.442909+0000","last_became_peered":"2023-10-08T07:15:46.442909+0000","last_unstale":"2023-10-08T07:15:47.601997+0000","last_undegraded":"2023-10-08T07:15:47.601997+0000","last_fullsized":"2023-10-08T07:15:47.601997+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:28.705460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2b","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599429+0000","last_change":"2023-10-08T07:15:46.449788+0000","last_active":"2023-10-08T07:15:47.599429+0000","last_peered":"2023-10-08T07:15:47.599429+0000","last_clean":"2023-10-08T07:15:47.599429+0000","last_became_active":"2023-10-08T07:15:46.449529+0000","last_became_peered":"2023-10-08T07:15:46.449529+0000","last_unstale":"2023-10-08T07:15:47.599429+0000","last_undegraded":"2023-10-08T07:15:47.599429+0000","last_fullsized":"2023-10-08T07:15:47.599429+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:38.401202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.2a","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599524+0000","last_change":"2023-10-08T07:15:46.442759+0000","last_active":"2023-10-08T07:15:47.599524+0000","last_peered":"2023-10-08T07:15:47.599524+0000","last_clean":"2023-10-08T07:15:47.599524+0000","last_became_active":"2023-10-08T07:15:46.441880+0000","last_became_peered":"2023-10-08T07:15:46.441880+0000","last_unstale":"2023-10-08T07:15:47.599524+0000","last_undegraded":"2023-10-08T07:15:47.599524+0000","last_fullsized":"2023-10-08T07:15:47.599524+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:40.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.29","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599664+0000","last_change":"2023-10-08T07:15:46.442809+0000","last_active":"2023-10-08T07:15:47.599664+0000","last_peered":"2023-10-08T07:15:47.599664+0000","last_clean":"2023-10-08T07:15:47.599664+0000","last_became_active":"2023-10-08T07:15:46.442508+0000","last_became_peered":"2023-10-08T07:15:46.442508+0000","last_unstale":"2023-10-08T07:15:47.599664+0000","last_undegraded":"2023-10-08T07:15:47.599664+0000","last_fullsized":"2023-10-08T07:15:47.599664+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:20.869460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.28","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600071+0000","last_change":"2023-10-08T07:15:46.423490+0000","last_active":"2023-10-08T07:15:47.600071+0000","last_peered":"2023-10-08T07:15:47.600071+0000","last_clean":"2023-10-08T07:15:47.600071+0000","last_became_active":"2023-10-08T07:15:46.420875+0000","last_became_peered":"2023-10-08T07:15:46.420875+0000","last_unstale":"2023-10-08T07:15:47.600071+0000","last_undegraded":"2023-10-08T07:15:47.600071+0000","last_fullsized":"2023-10-08T07:15:47.600071+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:46.539176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.27","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409902+0000","last_change":"2023-10-08T07:15:46.438477+0000","last_active":"2023-10-08T07:15:47.409902+0000","last_peered":"2023-10-08T07:15:47.409902+0000","last_clean":"2023-10-08T07:15:47.409902+0000","last_became_active":"2023-10-08T07:15:46.435054+0000","last_became_peered":"2023-10-08T07:15:46.435054+0000","last_unstale":"2023-10-08T07:15:47.409902+0000","last_undegraded":"2023-10-08T07:15:47.409902+0000","last_fullsized":"2023-10-08T07:15:47.409902+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:30:26.096323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.26","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408646+0000","last_change":"2023-10-08T07:15:46.448268+0000","last_active":"2023-10-08T07:15:47.408646+0000","last_peered":"2023-10-08T07:15:47.408646+0000","last_clean":"2023-10-08T07:15:47.408646+0000","last_became_active":"2023-10-08T07:15:46.447992+0000","last_became_peered":"2023-10-08T07:15:46.447992+0000","last_unstale":"2023-10-08T07:15:47.408646+0000","last_undegraded":"2023-10-08T07:15:47.408646+0000","last_fullsized":"2023-10-08T07:15:47.408646+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:52.997157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.24","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408676+0000","last_change":"2023-10-08T07:15:46.440817+0000","last_active":"2023-10-08T07:15:47.408676+0000","last_peered":"2023-10-08T07:15:47.408676+0000","last_clean":"2023-10-08T07:15:47.408676+0000","last_became_active":"2023-10-08T07:15:46.440518+0000","last_became_peered":"2023-10-08T07:15:46.440518+0000","last_unstale":"2023-10-08T07:15:47.408676+0000","last_undegraded":"2023-10-08T07:15:47.408676+0000","last_fullsized":"2023-10-08T07:15:47.408676+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:03:06.129360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.23","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599811+0000","last_change":"2023-10-08T07:15:46.453912+0000","last_active":"2023-10-08T07:15:47.599811+0000","last_peered":"2023-10-08T07:15:47.599811+0000","last_clean":"2023-10-08T07:15:47.599811+0000","last_became_active":"2023-10-08T07:15:46.453612+0000","last_became_peered":"2023-10-08T07:15:46.453612+0000","last_unstale":"2023-10-08T07:15:47.599811+0000","last_undegraded":"2023-10-08T07:15:47.599811+0000","last_fullsized":"2023-10-08T07:15:47.599811+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:52.997157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.22","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532101+0000","last_change":"2023-10-08T07:15:46.437205+0000","last_active":"2023-10-08T07:15:47.532101+0000","last_peered":"2023-10-08T07:15:47.532101+0000","last_clean":"2023-10-08T07:15:47.532101+0000","last_became_active":"2023-10-08T07:15:46.436874+0000","last_became_peered":"2023-10-08T07:15:46.436874+0000","last_unstale":"2023-10-08T07:15:47.532101+0000","last_undegraded":"2023-10-08T07:15:47.532101+0000","last_fullsized":"2023-10-08T07:15:47.532101+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:32:25.491400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.21","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599594+0000","last_change":"2023-10-08T07:15:46.446572+0000","last_active":"2023-10-08T07:15:47.599594+0000","last_peered":"2023-10-08T07:15:47.599594+0000","last_clean":"2023-10-08T07:15:47.599594+0000","last_became_active":"2023-10-08T07:15:46.446279+0000","last_became_peered":"2023-10-08T07:15:46.446279+0000","last_unstale":"2023-10-08T07:15:47.599594+0000","last_undegraded":"2023-10-08T07:15:47.599594+0000","last_fullsized":"2023-10-08T07:15:47.599594+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:36.057754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.20","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405420+0000","last_change":"2023-10-08T07:15:46.456557+0000","last_active":"2023-10-08T07:15:47.405420+0000","last_peered":"2023-10-08T07:15:47.405420+0000","last_clean":"2023-10-08T07:15:47.405420+0000","last_became_active":"2023-10-08T07:15:46.455964+0000","last_became_peered":"2023-10-08T07:15:46.455964+0000","last_unstale":"2023-10-08T07:15:47.405420+0000","last_undegraded":"2023-10-08T07:15:47.405420+0000","last_fullsized":"2023-10-08T07:15:47.405420+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:38.543864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600324+0000","last_change":"2023-10-08T07:15:46.428625+0000","last_active":"2023-10-08T07:15:47.600324+0000","last_peered":"2023-10-08T07:15:47.600324+0000","last_clean":"2023-10-08T07:15:47.600324+0000","last_became_active":"2023-10-08T07:15:46.422160+0000","last_became_peered":"2023-10-08T07:15:46.422160+0000","last_unstale":"2023-10-08T07:15:47.600324+0000","last_undegraded":"2023-10-08T07:15:47.600324+0000","last_fullsized":"2023-10-08T07:15:47.600324+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:57.091978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"275'4","reported_seq":16,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600652+0000","last_change":"2023-10-08T07:15:45.455540+0000","last_active":"2023-10-08T07:15:47.600652+0000","last_peered":"2023-10-08T07:15:47.600652+0000","last_clean":"2023-10-08T07:15:47.600652+0000","last_became_active":"2023-10-08T07:15:45.455294+0000","last_became_peered":"2023-10-08T07:15:45.455294+0000","last_unstale":"2023-10-08T07:15:47.600652+0000","last_undegraded":"2023-10-08T07:15:47.600652+0000","last_fullsized":"2023-10-08T07:15:47.600652+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:26:26.849065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599985+0000","last_change":"2023-10-08T07:15:46.437125+0000","last_active":"2023-10-08T07:15:47.599985+0000","last_peered":"2023-10-08T07:15:47.599985+0000","last_clean":"2023-10-08T07:15:47.599985+0000","last_became_active":"2023-10-08T07:15:46.436815+0000","last_became_peered":"2023-10-08T07:15:46.436815+0000","last_unstale":"2023-10-08T07:15:47.599985+0000","last_undegraded":"2023-10-08T07:15:47.599985+0000","last_fullsized":"2023-10-08T07:15:47.599985+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:32.571333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1e","version":"275'1","reported_seq":13,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600421+0000","last_change":"2023-10-08T07:15:45.449509+0000","last_active":"2023-10-08T07:15:47.600421+0000","last_peered":"2023-10-08T07:15:47.600421+0000","last_clean":"2023-10-08T07:15:47.600421+0000","last_became_active":"2023-10-08T07:15:45.448744+0000","last_became_peered":"2023-10-08T07:15:45.448744+0000","last_unstale":"2023-10-08T07:15:47.600421+0000","last_undegraded":"2023-10-08T07:15:47.600421+0000","last_fullsized":"2023-10-08T07:15:47.600421+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:49:31.904624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1a","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408963+0000","last_change":"2023-10-08T07:15:46.444369+0000","last_active":"2023-10-08T07:15:47.408963+0000","last_peered":"2023-10-08T07:15:47.408963+0000","last_clean":"2023-10-08T07:15:47.408963+0000","last_became_active":"2023-10-08T07:15:46.443549+0000","last_became_peered":"2023-10-08T07:15:46.443549+0000","last_unstale":"2023-10-08T07:15:47.408963+0000","last_undegraded":"2023-10-08T07:15:47.408963+0000","last_fullsized":"2023-10-08T07:15:47.408963+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:15.152757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599037+0000","last_change":"2023-10-08T07:15:45.452873+0000","last_active":"2023-10-08T07:15:47.599037+0000","last_peered":"2023-10-08T07:15:47.599037+0000","last_clean":"2023-10-08T07:15:47.599037+0000","last_became_active":"2023-10-08T07:15:45.452616+0000","last_became_peered":"2023-10-08T07:15:45.452616+0000","last_unstale":"2023-10-08T07:15:47.599037+0000","last_undegraded":"2023-10-08T07:15:47.599037+0000","last_fullsized":"2023-10-08T07:15:47.599037+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:30.770137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1b","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599740+0000","last_change":"2023-10-08T07:15:46.436119+0000","last_active":"2023-10-08T07:15:47.599740+0000","last_peered":"2023-10-08T07:15:47.599740+0000","last_clean":"2023-10-08T07:15:47.599740+0000","last_became_active":"2023-10-08T07:15:46.435825+0000","last_became_peered":"2023-10-08T07:15:46.435825+0000","last_unstale":"2023-10-08T07:15:47.599740+0000","last_undegraded":"2023-10-08T07:15:47.599740+0000","last_fullsized":"2023-10-08T07:15:47.599740+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:46.766818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599226+0000","last_change":"2023-10-08T07:15:45.450620+0000","last_active":"2023-10-08T07:15:47.599226+0000","last_peered":"2023-10-08T07:15:47.599226+0000","last_clean":"2023-10-08T07:15:47.599226+0000","last_became_active":"2023-10-08T07:15:45.450337+0000","last_became_peered":"2023-10-08T07:15:45.450337+0000","last_unstale":"2023-10-08T07:15:47.599226+0000","last_undegraded":"2023-10-08T07:15:47.599226+0000","last_fullsized":"2023-10-08T07:15:47.599226+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:45.461532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1c","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406284+0000","last_change":"2023-10-08T07:15:46.439974+0000","last_active":"2023-10-08T07:15:47.406284+0000","last_peered":"2023-10-08T07:15:47.406284+0000","last_clean":"2023-10-08T07:15:47.406284+0000","last_became_active":"2023-10-08T07:15:46.439611+0000","last_became_peered":"2023-10-08T07:15:46.439611+0000","last_unstale":"2023-10-08T07:15:47.406284+0000","last_undegraded":"2023-10-08T07:15:47.406284+0000","last_fullsized":"2023-10-08T07:15:47.406284+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:14.273565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1b","version":"275'1","reported_seq":13,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408483+0000","last_change":"2023-10-08T07:15:45.450095+0000","last_active":"2023-10-08T07:15:47.408483+0000","last_peered":"2023-10-08T07:15:47.408483+0000","last_clean":"2023-10-08T07:15:47.408483+0000","last_became_active":"2023-10-08T07:15:45.449825+0000","last_became_peered":"2023-10-08T07:15:45.449825+0000","last_unstale":"2023-10-08T07:15:47.408483+0000","last_undegraded":"2023-10-08T07:15:47.408483+0000","last_fullsized":"2023-10-08T07:15:47.408483+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:27.693885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.1d","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600420+0000","last_change":"2023-10-08T07:15:46.418445+0000","last_active":"2023-10-08T07:15:47.600420+0000","last_peered":"2023-10-08T07:15:47.600420+0000","last_clean":"2023-10-08T07:15:47.600420+0000","last_became_active":"2023-10-08T07:15:46.417718+0000","last_became_peered":"2023-10-08T07:15:46.417718+0000","last_unstale":"2023-10-08T07:15:47.600420+0000","last_undegraded":"2023-10-08T07:15:47.600420+0000","last_fullsized":"2023-10-08T07:15:47.600420+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:57:41.290522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1a","version":"275'1","reported_seq":13,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408312+0000","last_change":"2023-10-08T07:15:45.451317+0000","last_active":"2023-10-08T07:15:47.408312+0000","last_peered":"2023-10-08T07:15:47.408312+0000","last_clean":"2023-10-08T07:15:47.408312+0000","last_became_active":"2023-10-08T07:15:45.450948+0000","last_became_peered":"2023-10-08T07:15:45.450948+0000","last_unstale":"2023-10-08T07:15:47.408312+0000","last_undegraded":"2023-10-08T07:15:47.408312+0000","last_fullsized":"2023-10-08T07:15:47.408312+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:30.407492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1e","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600279+0000","last_change":"2023-10-08T07:15:46.434383+0000","last_active":"2023-10-08T07:15:47.600279+0000","last_peered":"2023-10-08T07:15:47.600279+0000","last_clean":"2023-10-08T07:15:47.600279+0000","last_became_active":"2023-10-08T07:15:46.434126+0000","last_became_peered":"2023-10-08T07:15:46.434126+0000","last_unstale":"2023-10-08T07:15:47.600279+0000","last_undegraded":"2023-10-08T07:15:47.600279+0000","last_fullsized":"2023-10-08T07:15:47.600279+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:39.135783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532030+0000","last_change":"2023-10-08T07:15:45.449753+0000","last_active":"2023-10-08T07:15:47.532030+0000","last_peered":"2023-10-08T07:15:47.532030+0000","last_clean":"2023-10-08T07:15:47.532030+0000","last_became_active":"2023-10-08T07:15:45.449467+0000","last_became_peered":"2023-10-08T07:15:45.449467+0000","last_unstale":"2023-10-08T07:15:47.532030+0000","last_undegraded":"2023-10-08T07:15:47.532030+0000","last_fullsized":"2023-10-08T07:15:47.532030+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:26.976546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1f","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.602065+0000","last_change":"2023-10-08T07:15:46.441363+0000","last_active":"2023-10-08T07:15:47.602065+0000","last_peered":"2023-10-08T07:15:47.602065+0000","last_clean":"2023-10-08T07:15:47.602065+0000","last_became_active":"2023-10-08T07:15:46.440757+0000","last_became_peered":"2023-10-08T07:15:46.440757+0000","last_unstale":"2023-10-08T07:15:47.602065+0000","last_undegraded":"2023-10-08T07:15:47.602065+0000","last_fullsized":"2023-10-08T07:15:47.602065+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:26.756328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405709+0000","last_change":"2023-10-08T07:15:45.447912+0000","last_active":"2023-10-08T07:15:47.405709+0000","last_peered":"2023-10-08T07:15:47.405709+0000","last_clean":"2023-10-08T07:15:47.405709+0000","last_became_active":"2023-10-08T07:15:45.447445+0000","last_became_peered":"2023-10-08T07:15:45.447445+0000","last_unstale":"2023-10-08T07:15:47.405709+0000","last_undegraded":"2023-10-08T07:15:47.405709+0000","last_fullsized":"2023-10-08T07:15:47.405709+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:26.504136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.10","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405453+0000","last_change":"2023-10-08T07:15:46.438059+0000","last_active":"2023-10-08T07:15:47.405453+0000","last_peered":"2023-10-08T07:15:47.405453+0000","last_clean":"2023-10-08T07:15:47.405453+0000","last_became_active":"2023-10-08T07:15:46.437782+0000","last_became_peered":"2023-10-08T07:15:46.437782+0000","last_unstale":"2023-10-08T07:15:47.405453+0000","last_undegraded":"2023-10-08T07:15:47.405453+0000","last_fullsized":"2023-10-08T07:15:47.405453+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:57:05.013108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.602212+0000","last_change":"2023-10-08T07:15:45.459521+0000","last_active":"2023-10-08T07:15:47.602212+0000","last_peered":"2023-10-08T07:15:47.602212+0000","last_clean":"2023-10-08T07:15:47.602212+0000","last_became_active":"2023-10-08T07:15:45.459238+0000","last_became_peered":"2023-10-08T07:15:45.459238+0000","last_unstale":"2023-10-08T07:15:47.602212+0000","last_undegraded":"2023-10-08T07:15:47.602212+0000","last_fullsized":"2023-10-08T07:15:47.602212+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:33.962135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408580+0000","last_change":"2023-10-08T07:15:46.433353+0000","last_active":"2023-10-08T07:15:47.408580+0000","last_peered":"2023-10-08T07:15:47.408580+0000","last_clean":"2023-10-08T07:15:47.408580+0000","last_became_active":"2023-10-08T07:15:46.431825+0000","last_became_peered":"2023-10-08T07:15:46.431825+0000","last_unstale":"2023-10-08T07:15:47.408580+0000","last_undegraded":"2023-10-08T07:15:47.408580+0000","last_fullsized":"2023-10-08T07:15:47.408580+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:26.756328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532465+0000","last_change":"2023-10-08T07:15:45.451533+0000","last_active":"2023-10-08T07:15:47.532465+0000","last_peered":"2023-10-08T07:15:47.532465+0000","last_clean":"2023-10-08T07:15:47.532465+0000","last_became_active":"2023-10-08T07:15:45.450582+0000","last_became_peered":"2023-10-08T07:15:45.450582+0000","last_unstale":"2023-10-08T07:15:47.532465+0000","last_undegraded":"2023-10-08T07:15:47.532465+0000","last_fullsized":"2023-10-08T07:15:47.532465+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:36.380720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406078+0000","last_change":"2023-10-08T07:15:46.442443+0000","last_active":"2023-10-08T07:15:47.406078+0000","last_peered":"2023-10-08T07:15:47.406078+0000","last_clean":"2023-10-08T07:15:47.406078+0000","last_became_active":"2023-10-08T07:15:46.442183+0000","last_became_peered":"2023-10-08T07:15:46.442183+0000","last_unstale":"2023-10-08T07:15:47.406078+0000","last_undegraded":"2023-10-08T07:15:47.406078+0000","last_fullsized":"2023-10-08T07:15:47.406078+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:02:06.340794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"275'3","reported_seq":15,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601969+0000","last_change":"2023-10-08T07:15:45.445041+0000","last_active":"2023-10-08T07:15:47.601969+0000","last_peered":"2023-10-08T07:15:47.601969+0000","last_clean":"2023-10-08T07:15:47.601969+0000","last_became_active":"2023-10-08T07:15:45.444722+0000","last_became_peered":"2023-10-08T07:15:45.444722+0000","last_unstale":"2023-10-08T07:15:47.601969+0000","last_undegraded":"2023-10-08T07:15:47.601969+0000","last_fullsized":"2023-10-08T07:15:47.601969+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:24.302275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409038+0000","last_change":"2023-10-08T07:15:46.459196+0000","last_active":"2023-10-08T07:15:47.409038+0000","last_peered":"2023-10-08T07:15:47.409038+0000","last_clean":"2023-10-08T07:15:47.409038+0000","last_became_active":"2023-10-08T07:15:46.458898+0000","last_became_peered":"2023-10-08T07:15:46.458898+0000","last_unstale":"2023-10-08T07:15:47.409038+0000","last_undegraded":"2023-10-08T07:15:47.409038+0000","last_fullsized":"2023-10-08T07:15:47.409038+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:57.091978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.14","version":"275'6","reported_seq":18,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600683+0000","last_change":"2023-10-08T07:15:45.457442+0000","last_active":"2023-10-08T07:15:47.600683+0000","last_peered":"2023-10-08T07:15:47.600683+0000","last_clean":"2023-10-08T07:15:47.600683+0000","last_became_active":"2023-10-08T07:15:45.457174+0000","last_became_peered":"2023-10-08T07:15:45.457174+0000","last_unstale":"2023-10-08T07:15:47.600683+0000","last_undegraded":"2023-10-08T07:15:47.600683+0000","last_fullsized":"2023-10-08T07:15:47.600683+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:29.708621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.14","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409462+0000","last_change":"2023-10-08T07:15:46.434283+0000","last_active":"2023-10-08T07:15:47.409462+0000","last_peered":"2023-10-08T07:15:47.409462+0000","last_clean":"2023-10-08T07:15:47.409462+0000","last_became_active":"2023-10-08T07:15:46.433962+0000","last_became_peered":"2023-10-08T07:15:46.433962+0000","last_unstale":"2023-10-08T07:15:47.409462+0000","last_undegraded":"2023-10-08T07:15:47.409462+0000","last_fullsized":"2023-10-08T07:15:47.409462+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:03:19.945315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.13","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532197+0000","last_change":"2023-10-08T07:15:45.452819+0000","last_active":"2023-10-08T07:15:47.532197+0000","last_peered":"2023-10-08T07:15:47.532197+0000","last_clean":"2023-10-08T07:15:47.532197+0000","last_became_active":"2023-10-08T07:15:45.452499+0000","last_became_peered":"2023-10-08T07:15:45.452499+0000","last_unstale":"2023-10-08T07:15:47.532197+0000","last_undegraded":"2023-10-08T07:15:47.532197+0000","last_fullsized":"2023-10-08T07:15:47.532197+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:31.199635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.15","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406641+0000","last_change":"2023-10-08T07:15:46.435436+0000","last_active":"2023-10-08T07:15:47.406641+0000","last_peered":"2023-10-08T07:15:47.406641+0000","last_clean":"2023-10-08T07:15:47.406641+0000","last_became_active":"2023-10-08T07:15:46.435153+0000","last_became_peered":"2023-10-08T07:15:46.435153+0000","last_unstale":"2023-10-08T07:15:47.406641+0000","last_undegraded":"2023-10-08T07:15:47.406641+0000","last_fullsized":"2023-10-08T07:15:47.406641+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:52.673356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408974+0000","last_change":"2023-10-08T07:15:45.445901+0000","last_active":"2023-10-08T07:15:47.408974+0000","last_peered":"2023-10-08T07:15:47.408974+0000","last_clean":"2023-10-08T07:15:47.408974+0000","last_became_active":"2023-10-08T07:15:45.445598+0000","last_became_peered":"2023-10-08T07:15:45.445598+0000","last_unstale":"2023-10-08T07:15:47.408974+0000","last_undegraded":"2023-10-08T07:15:47.408974+0000","last_fullsized":"2023-10-08T07:15:47.408974+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.579097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601582+0000","last_change":"2023-10-08T07:15:46.442496+0000","last_active":"2023-10-08T07:15:47.601582+0000","last_peered":"2023-10-08T07:15:47.601582+0000","last_clean":"2023-10-08T07:15:47.601582+0000","last_became_active":"2023-10-08T07:15:46.442211+0000","last_became_peered":"2023-10-08T07:15:46.442211+0000","last_unstale":"2023-10-08T07:15:47.601582+0000","last_undegraded":"2023-10-08T07:15:47.601582+0000","last_fullsized":"2023-10-08T07:15:47.601582+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:46.590671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409264+0000","last_change":"2023-10-08T07:15:45.455026+0000","last_active":"2023-10-08T07:15:47.409264+0000","last_peered":"2023-10-08T07:15:47.409264+0000","last_clean":"2023-10-08T07:15:47.409264+0000","last_became_active":"2023-10-08T07:15:45.454768+0000","last_became_peered":"2023-10-08T07:15:45.454768+0000","last_unstale":"2023-10-08T07:15:47.409264+0000","last_undegraded":"2023-10-08T07:15:47.409264+0000","last_fullsized":"2023-10-08T07:15:47.409264+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:59.841798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.602164+0000","last_change":"2023-10-08T07:15:46.441540+0000","last_active":"2023-10-08T07:15:47.602164+0000","last_peered":"2023-10-08T07:15:47.602164+0000","last_clean":"2023-10-08T07:15:47.602164+0000","last_became_active":"2023-10-08T07:15:46.441047+0000","last_became_peered":"2023-10-08T07:15:46.441047+0000","last_unstale":"2023-10-08T07:15:47.602164+0000","last_undegraded":"2023-10-08T07:15:47.602164+0000","last_fullsized":"2023-10-08T07:15:47.602164+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:15.152757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"275'8","reported_seq":20,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599641+0000","last_change":"2023-10-08T07:15:45.454962+0000","last_active":"2023-10-08T07:15:47.599641+0000","last_peered":"2023-10-08T07:15:47.599641+0000","last_clean":"2023-10-08T07:15:47.599641+0000","last_became_active":"2023-10-08T07:15:45.454685+0000","last_became_peered":"2023-10-08T07:15:45.454685+0000","last_unstale":"2023-10-08T07:15:47.599641+0000","last_undegraded":"2023-10-08T07:15:47.599641+0000","last_fullsized":"2023-10-08T07:15:47.599641+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:52:28.362795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408517+0000","last_change":"2023-10-08T07:15:46.444827+0000","last_active":"2023-10-08T07:15:47.408517+0000","last_peered":"2023-10-08T07:15:47.408517+0000","last_clean":"2023-10-08T07:15:47.408517+0000","last_became_active":"2023-10-08T07:15:46.444427+0000","last_became_peered":"2023-10-08T07:15:46.444427+0000","last_unstale":"2023-10-08T07:15:47.408517+0000","last_undegraded":"2023-10-08T07:15:47.408517+0000","last_fullsized":"2023-10-08T07:15:47.408517+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:36.057754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.f","version":"275'1","reported_seq":13,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599888+0000","last_change":"2023-10-08T07:15:45.454339+0000","last_active":"2023-10-08T07:15:47.599888+0000","last_peered":"2023-10-08T07:15:47.599888+0000","last_clean":"2023-10-08T07:15:47.599888+0000","last_became_active":"2023-10-08T07:15:45.454038+0000","last_became_peered":"2023-10-08T07:15:45.454038+0000","last_unstale":"2023-10-08T07:15:47.599888+0000","last_undegraded":"2023-10-08T07:15:47.599888+0000","last_fullsized":"2023-10-08T07:15:47.599888+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:40.455212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599962+0000","last_change":"2023-10-08T07:15:46.433330+0000","last_active":"2023-10-08T07:15:47.599962+0000","last_peered":"2023-10-08T07:15:47.599962+0000","last_clean":"2023-10-08T07:15:47.599962+0000","last_became_active":"2023-10-08T07:15:46.431617+0000","last_became_peered":"2023-10-08T07:15:46.431617+0000","last_unstale":"2023-10-08T07:15:47.599962+0000","last_undegraded":"2023-10-08T07:15:47.599962+0000","last_fullsized":"2023-10-08T07:15:47.599962+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:03:06.129360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.e","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408753+0000","last_change":"2023-10-08T07:15:45.442328+0000","last_active":"2023-10-08T07:15:47.408753+0000","last_peered":"2023-10-08T07:15:47.408753+0000","last_clean":"2023-10-08T07:15:47.408753+0000","last_became_active":"2023-10-08T07:15:45.441883+0000","last_became_peered":"2023-10-08T07:15:45.441883+0000","last_unstale":"2023-10-08T07:15:47.408753+0000","last_undegraded":"2023-10-08T07:15:47.408753+0000","last_fullsized":"2023-10-08T07:15:47.408753+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:23.168510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408176+0000","last_change":"2023-10-08T07:15:46.439306+0000","last_active":"2023-10-08T07:15:47.408176+0000","last_peered":"2023-10-08T07:15:47.408176+0000","last_clean":"2023-10-08T07:15:47.408176+0000","last_became_active":"2023-10-08T07:15:46.439016+0000","last_became_peered":"2023-10-08T07:15:46.439016+0000","last_unstale":"2023-10-08T07:15:47.408176+0000","last_undegraded":"2023-10-08T07:15:47.408176+0000","last_fullsized":"2023-10-08T07:15:47.408176+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:40.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.d","version":"275'1","reported_seq":13,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599132+0000","last_change":"2023-10-08T07:15:45.451959+0000","last_active":"2023-10-08T07:15:47.599132+0000","last_peered":"2023-10-08T07:15:47.599132+0000","last_clean":"2023-10-08T07:15:47.599132+0000","last_became_active":"2023-10-08T07:15:45.451698+0000","last_became_peered":"2023-10-08T07:15:45.451698+0000","last_unstale":"2023-10-08T07:15:47.599132+0000","last_undegraded":"2023-10-08T07:15:47.599132+0000","last_fullsized":"2023-10-08T07:15:47.599132+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:46:47.747067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.b","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406067+0000","last_change":"2023-10-08T07:15:46.430865+0000","last_active":"2023-10-08T07:15:47.406067+0000","last_peered":"2023-10-08T07:15:47.406067+0000","last_clean":"2023-10-08T07:15:47.406067+0000","last_became_active":"2023-10-08T07:15:46.430582+0000","last_became_peered":"2023-10-08T07:15:46.430582+0000","last_unstale":"2023-10-08T07:15:47.406067+0000","last_undegraded":"2023-10-08T07:15:47.406067+0000","last_fullsized":"2023-10-08T07:15:47.406067+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:54.581182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532249+0000","last_change":"2023-10-08T07:15:45.455056+0000","last_active":"2023-10-08T07:15:47.532249+0000","last_peered":"2023-10-08T07:15:47.532249+0000","last_clean":"2023-10-08T07:15:47.532249+0000","last_became_active":"2023-10-08T07:15:45.454806+0000","last_became_peered":"2023-10-08T07:15:45.454806+0000","last_unstale":"2023-10-08T07:15:47.532249+0000","last_undegraded":"2023-10-08T07:15:47.532249+0000","last_fullsized":"2023-10-08T07:15:47.532249+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:36.844747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600039+0000","last_change":"2023-10-08T07:15:46.442950+0000","last_active":"2023-10-08T07:15:47.600039+0000","last_peered":"2023-10-08T07:15:47.600039+0000","last_clean":"2023-10-08T07:15:47.600039+0000","last_became_active":"2023-10-08T07:15:46.442281+0000","last_became_peered":"2023-10-08T07:15:46.442281+0000","last_unstale":"2023-10-08T07:15:47.600039+0000","last_undegraded":"2023-10-08T07:15:47.600039+0000","last_fullsized":"2023-10-08T07:15:47.600039+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:38.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532453+0000","last_change":"2023-10-08T07:15:45.448839+0000","last_active":"2023-10-08T07:15:47.532453+0000","last_peered":"2023-10-08T07:15:47.532453+0000","last_clean":"2023-10-08T07:15:47.532453+0000","last_became_active":"2023-10-08T07:15:45.448511+0000","last_became_peered":"2023-10-08T07:15:45.448511+0000","last_unstale":"2023-10-08T07:15:47.532453+0000","last_undegraded":"2023-10-08T07:15:47.532453+0000","last_fullsized":"2023-10-08T07:15:47.532453+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:56:39.425788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.d","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600690+0000","last_change":"2023-10-08T07:15:46.431513+0000","last_active":"2023-10-08T07:15:47.600690+0000","last_peered":"2023-10-08T07:15:47.600690+0000","last_clean":"2023-10-08T07:15:47.600690+0000","last_became_active":"2023-10-08T07:15:46.431216+0000","last_became_peered":"2023-10-08T07:15:46.431216+0000","last_unstale":"2023-10-08T07:15:47.600690+0000","last_undegraded":"2023-10-08T07:15:47.600690+0000","last_fullsized":"2023-10-08T07:15:47.600690+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:26.726652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406313+0000","last_change":"2023-10-08T07:15:45.447118+0000","last_active":"2023-10-08T07:15:47.406313+0000","last_peered":"2023-10-08T07:15:47.406313+0000","last_clean":"2023-10-08T07:15:47.406313+0000","last_became_active":"2023-10-08T07:15:45.446659+0000","last_became_peered":"2023-10-08T07:15:45.446659+0000","last_unstale":"2023-10-08T07:15:47.406313+0000","last_undegraded":"2023-10-08T07:15:47.406313+0000","last_fullsized":"2023-10-08T07:15:47.406313+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:50.887401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.e","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601980+0000","last_change":"2023-10-08T07:15:46.442076+0000","last_active":"2023-10-08T07:15:47.601980+0000","last_peered":"2023-10-08T07:15:47.601980+0000","last_clean":"2023-10-08T07:15:47.601980+0000","last_became_active":"2023-10-08T07:15:46.441744+0000","last_became_peered":"2023-10-08T07:15:46.441744+0000","last_unstale":"2023-10-08T07:15:47.601980+0000","last_undegraded":"2023-10-08T07:15:47.601980+0000","last_fullsized":"2023-10-08T07:15:47.601980+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:41.656686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600253+0000","last_change":"2023-10-08T07:15:45.445842+0000","last_active":"2023-10-08T07:15:47.600253+0000","last_peered":"2023-10-08T07:15:47.600253+0000","last_clean":"2023-10-08T07:15:47.600253+0000","last_became_active":"2023-10-08T07:15:45.445552+0000","last_became_peered":"2023-10-08T07:15:45.445552+0000","last_unstale":"2023-10-08T07:15:47.600253+0000","last_undegraded":"2023-10-08T07:15:47.600253+0000","last_fullsized":"2023-10-08T07:15:47.600253+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:49:17.290142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.f","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408674+0000","last_change":"2023-10-08T07:15:46.452651+0000","last_active":"2023-10-08T07:15:47.408674+0000","last_peered":"2023-10-08T07:15:47.408674+0000","last_clean":"2023-10-08T07:15:47.408674+0000","last_became_active":"2023-10-08T07:15:46.452319+0000","last_became_peered":"2023-10-08T07:15:46.452319+0000","last_unstale":"2023-10-08T07:15:47.408674+0000","last_undegraded":"2023-10-08T07:15:47.408674+0000","last_fullsized":"2023-10-08T07:15:47.408674+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:38.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.531843+0000","last_change":"2023-10-08T07:15:45.451784+0000","last_active":"2023-10-08T07:15:47.531843+0000","last_peered":"2023-10-08T07:15:47.531843+0000","last_clean":"2023-10-08T07:15:47.531843+0000","last_became_active":"2023-10-08T07:15:45.451474+0000","last_became_peered":"2023-10-08T07:15:45.451474+0000","last_unstale":"2023-10-08T07:15:47.531843+0000","last_undegraded":"2023-10-08T07:15:47.531843+0000","last_fullsized":"2023-10-08T07:15:47.531843+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:34.381401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405629+0000","last_change":"2023-10-08T07:15:46.436370+0000","last_active":"2023-10-08T07:15:47.405629+0000","last_peered":"2023-10-08T07:15:47.405629+0000","last_clean":"2023-10-08T07:15:47.405629+0000","last_became_active":"2023-10-08T07:15:46.436098+0000","last_became_peered":"2023-10-08T07:15:46.436098+0000","last_unstale":"2023-10-08T07:15:47.405629+0000","last_undegraded":"2023-10-08T07:15:47.405629+0000","last_fullsized":"2023-10-08T07:15:47.405629+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:11.123773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.7","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532723+0000","last_change":"2023-10-08T07:15:45.457184+0000","last_active":"2023-10-08T07:15:47.532723+0000","last_peered":"2023-10-08T07:15:47.532723+0000","last_clean":"2023-10-08T07:15:47.532723+0000","last_became_active":"2023-10-08T07:15:45.456911+0000","last_became_peered":"2023-10-08T07:15:45.456911+0000","last_unstale":"2023-10-08T07:15:47.532723+0000","last_undegraded":"2023-10-08T07:15:47.532723+0000","last_fullsized":"2023-10-08T07:15:47.532723+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:06.175497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601057+0000","last_change":"2023-10-08T07:15:46.430723+0000","last_active":"2023-10-08T07:15:47.601057+0000","last_peered":"2023-10-08T07:15:47.601057+0000","last_clean":"2023-10-08T07:15:47.601057+0000","last_became_active":"2023-10-08T07:15:46.430244+0000","last_became_peered":"2023-10-08T07:15:46.430244+0000","last_unstale":"2023-10-08T07:15:47.601057+0000","last_undegraded":"2023-10-08T07:15:47.601057+0000","last_fullsized":"2023-10-08T07:15:47.601057+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:46:54.344502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409555+0000","last_change":"2023-10-08T07:15:45.446031+0000","last_active":"2023-10-08T07:15:47.409555+0000","last_peered":"2023-10-08T07:15:47.409555+0000","last_clean":"2023-10-08T07:15:47.409555+0000","last_became_active":"2023-10-08T07:15:45.445745+0000","last_became_peered":"2023-10-08T07:15:45.445745+0000","last_unstale":"2023-10-08T07:15:47.409555+0000","last_undegraded":"2023-10-08T07:15:47.409555+0000","last_fullsized":"2023-10-08T07:15:47.409555+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:03:03.322265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405927+0000","last_change":"2023-10-08T07:15:46.427328+0000","last_active":"2023-10-08T07:15:47.405927+0000","last_peered":"2023-10-08T07:15:47.405927+0000","last_clean":"2023-10-08T07:15:47.405927+0000","last_became_active":"2023-10-08T07:15:46.427054+0000","last_became_peered":"2023-10-08T07:15:46.427054+0000","last_unstale":"2023-10-08T07:15:47.405927+0000","last_undegraded":"2023-10-08T07:15:47.405927+0000","last_fullsized":"2023-10-08T07:15:47.405927+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:10.032112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599218+0000","last_change":"2023-10-08T07:15:45.452452+0000","last_active":"2023-10-08T07:15:47.599218+0000","last_peered":"2023-10-08T07:15:47.599218+0000","last_clean":"2023-10-08T07:15:47.599218+0000","last_became_active":"2023-10-08T07:15:45.452204+0000","last_became_peered":"2023-10-08T07:15:45.452204+0000","last_unstale":"2023-10-08T07:15:47.599218+0000","last_undegraded":"2023-10-08T07:15:47.599218+0000","last_fullsized":"2023-10-08T07:15:47.599218+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:58.232704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600173+0000","last_change":"2023-10-08T07:15:46.443588+0000","last_active":"2023-10-08T07:15:47.600173+0000","last_peered":"2023-10-08T07:15:47.600173+0000","last_clean":"2023-10-08T07:15:47.600173+0000","last_became_active":"2023-10-08T07:15:46.443283+0000","last_became_peered":"2023-10-08T07:15:46.443283+0000","last_unstale":"2023-10-08T07:15:47.600173+0000","last_undegraded":"2023-10-08T07:15:47.600173+0000","last_fullsized":"2023-10-08T07:15:47.600173+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:26.423857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409499+0000","last_change":"2023-10-08T07:15:45.453612+0000","last_active":"2023-10-08T07:15:47.409499+0000","last_peered":"2023-10-08T07:15:47.409499+0000","last_clean":"2023-10-08T07:15:47.409499+0000","last_became_active":"2023-10-08T07:15:45.453337+0000","last_became_peered":"2023-10-08T07:15:45.453337+0000","last_unstale":"2023-10-08T07:15:47.409499+0000","last_undegraded":"2023-10-08T07:15:47.409499+0000","last_fullsized":"2023-10-08T07:15:47.409499+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:40.645111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600542+0000","last_change":"2023-10-08T07:15:46.435249+0000","last_active":"2023-10-08T07:15:47.600542+0000","last_peered":"2023-10-08T07:15:47.600542+0000","last_clean":"2023-10-08T07:15:47.600542+0000","last_became_active":"2023-10-08T07:15:46.434942+0000","last_became_peered":"2023-10-08T07:15:46.434942+0000","last_unstale":"2023-10-08T07:15:47.600542+0000","last_undegraded":"2023-10-08T07:15:47.600542+0000","last_fullsized":"2023-10-08T07:15:47.600542+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:50:08.901765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532581+0000","last_change":"2023-10-08T07:15:45.451334+0000","last_active":"2023-10-08T07:15:47.532581+0000","last_peered":"2023-10-08T07:15:47.532581+0000","last_clean":"2023-10-08T07:15:47.532581+0000","last_became_active":"2023-10-08T07:15:45.451045+0000","last_became_peered":"2023-10-08T07:15:45.451045+0000","last_unstale":"2023-10-08T07:15:47.532581+0000","last_undegraded":"2023-10-08T07:15:47.532581+0000","last_fullsized":"2023-10-08T07:15:47.532581+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:52:17.820140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406815+0000","last_change":"2023-10-08T07:15:46.429505+0000","last_active":"2023-10-08T07:15:47.406815+0000","last_peered":"2023-10-08T07:15:47.406815+0000","last_clean":"2023-10-08T07:15:47.406815+0000","last_became_active":"2023-10-08T07:15:46.429165+0000","last_became_peered":"2023-10-08T07:15:46.429165+0000","last_unstale":"2023-10-08T07:15:47.406815+0000","last_undegraded":"2023-10-08T07:15:47.406815+0000","last_fullsized":"2023-10-08T07:15:47.406815+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:34.973709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406220+0000","last_change":"2023-10-08T07:15:45.448049+0000","last_active":"2023-10-08T07:15:47.406220+0000","last_peered":"2023-10-08T07:15:47.406220+0000","last_clean":"2023-10-08T07:15:47.406220+0000","last_became_active":"2023-10-08T07:15:45.447780+0000","last_became_peered":"2023-10-08T07:15:45.447780+0000","last_unstale":"2023-10-08T07:15:47.406220+0000","last_undegraded":"2023-10-08T07:15:47.406220+0000","last_fullsized":"2023-10-08T07:15:47.406220+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:48.419734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408760+0000","last_change":"2023-10-08T07:15:46.442427+0000","last_active":"2023-10-08T07:15:47.408760+0000","last_peered":"2023-10-08T07:15:47.408760+0000","last_clean":"2023-10-08T07:15:47.408760+0000","last_became_active":"2023-10-08T07:15:46.441981+0000","last_became_peered":"2023-10-08T07:15:46.441981+0000","last_unstale":"2023-10-08T07:15:47.408760+0000","last_undegraded":"2023-10-08T07:15:47.408760+0000","last_fullsized":"2023-10-08T07:15:47.408760+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:46.539176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601339+0000","last_change":"2023-10-08T07:15:45.456861+0000","last_active":"2023-10-08T07:15:47.601339+0000","last_peered":"2023-10-08T07:15:47.601339+0000","last_clean":"2023-10-08T07:15:47.601339+0000","last_became_active":"2023-10-08T07:15:45.456587+0000","last_became_peered":"2023-10-08T07:15:45.456587+0000","last_unstale":"2023-10-08T07:15:47.601339+0000","last_undegraded":"2023-10-08T07:15:47.601339+0000","last_fullsized":"2023-10-08T07:15:47.601339+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:51.661781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.7","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406426+0000","last_change":"2023-10-08T07:15:46.439221+0000","last_active":"2023-10-08T07:15:47.406426+0000","last_peered":"2023-10-08T07:15:47.406426+0000","last_clean":"2023-10-08T07:15:47.406426+0000","last_became_active":"2023-10-08T07:15:46.438909+0000","last_became_peered":"2023-10-08T07:15:46.438909+0000","last_unstale":"2023-10-08T07:15:47.406426+0000","last_undegraded":"2023-10-08T07:15:47.406426+0000","last_fullsized":"2023-10-08T07:15:47.406426+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:38:17.156874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.531897+0000","last_change":"2023-10-08T07:15:45.451120+0000","last_active":"2023-10-08T07:15:47.531897+0000","last_peered":"2023-10-08T07:15:47.531897+0000","last_clean":"2023-10-08T07:15:47.531897+0000","last_became_active":"2023-10-08T07:15:45.450838+0000","last_became_peered":"2023-10-08T07:15:45.450838+0000","last_unstale":"2023-10-08T07:15:47.531897+0000","last_undegraded":"2023-10-08T07:15:47.531897+0000","last_fullsized":"2023-10-08T07:15:47.531897+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:31.423842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.25","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532536+0000","last_change":"2023-10-08T07:15:46.443673+0000","last_active":"2023-10-08T07:15:47.532536+0000","last_peered":"2023-10-08T07:15:47.532536+0000","last_clean":"2023-10-08T07:15:47.532536+0000","last_became_active":"2023-10-08T07:15:46.443420+0000","last_became_peered":"2023-10-08T07:15:46.443420+0000","last_unstale":"2023-10-08T07:15:47.532536+0000","last_undegraded":"2023-10-08T07:15:47.532536+0000","last_fullsized":"2023-10-08T07:15:47.532536+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:28.171102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"272'4861","reported_seq":5927,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-08T07:15:46.425822+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:15:46.425822+0000","last_peered":"2023-10-08T07:15:46.425822+0000","last_clean":"2023-10-08T07:15:46.425822+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:15:46.425822+0000","last_undegraded":"2023-10-08T07:15:46.425822+0000","last_fullsized":"2023-10-08T07:15:46.425822+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":651,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":36,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":651,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4861,"ondisk_log_size":4861,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576033,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75448,"kb_used_data":4004,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296392,"statfs":{"total":96636764160,"available":96559505408,"internally_reserved":0,"allocated":4100096,"data_stored":2419095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.127,"15min":1.1910000000000001},"min":{"1min":0.81699999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.968,"15min":2.008},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.81999999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.516,"5min":1.788,"15min":1.788},"last":1.47}]},{"osd":1,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.2370000000000001,"15min":1.2509999999999999},"min":{"1min":0.93899999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.998},"last":1.038},{"interface":"front","average":{"1min":1.196,"5min":1.2969999999999999,"15min":1.329},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6299999999999999,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.159}]},{"osd":2,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2370000000000001,"15min":1.2929999999999999},"min":{"1min":0.89500000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":2.1640000000000001},"last":1.093},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.369,"15min":1.3500000000000001},"min":{"1min":0.93400000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.901,"5min":1.901,"15min":2.3170000000000002},"last":1.288}]},{"osd":3,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.298,"15min":1.3340000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.222},{"interface":"front","average":{"1min":1.401,"5min":1.3819999999999999,"15min":1.3999999999999999},"min":{"1min":0.97099999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.3979999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.83999999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9319999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4139999999999999,"15min":1.411},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.4339999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.478,"15min":1.4490000000000001},"min":{"1min":1.127,"5min":0.80600000000000005,"15min":0.79300000000000004},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.623},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.488,"15min":1.4950000000000001},"min":{"1min":1.012,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.056,"5min":2.23,"15min":2.23},"last":1.6779999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.599,"15min":1.597},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.202,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.78},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.5640000000000001,"15min":1.5429999999999999},"min":{"1min":1.21,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2570000000000001},"last":1.7050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576032,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48464,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323376,"statfs":{"total":96636764160,"available":96587137024,"internally_reserved":0,"allocated":3473408,"data_stored":1819684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.2430000000000001,"15min":1.292},"min":{"1min":0.68999999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5389999999999999,"5min":1.78,"15min":1.962},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2889999999999999,"15min":1.3460000000000001},"min":{"1min":0.79500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6399999999999999,"5min":1.76,"15min":2.0070000000000001},"last":1.7370000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1559999999999999,"15min":1.216},"min":{"1min":0.69399999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.6879999999999999,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.169,"5min":1.2250000000000001,"15min":1.2390000000000001},"min":{"1min":0.77300000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5640000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.74399999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1699999999999999,"15min":1.2250000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.6850000000000001,"5min":1.7909999999999999,"15min":2.339},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.268,"15min":1.304},"min":{"1min":0.84399999999999997,"5min":0.748,"15min":0.748},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":2.1779999999999999},"last":1.298}]},{"osd":3,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.373,"15min":1.395},"min":{"1min":1.0409999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8160000000000001,"5min":1.9319999999999999,"15min":1.95},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.409,"15min":1.4830000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.1459999999999999},"last":1.8440000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.4079999999999999,"15min":1.401},"min":{"1min":1.145,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0859999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.502,"5min":1.4750000000000001,"15min":1.46},"min":{"1min":1.01,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":1.99,"5min":2.1110000000000002,"15min":2.484},"last":1.9379999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.3939999999999999,"15min":1.4259999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.403},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.629,"5min":1.569,"15min":1.5609999999999999},"min":{"1min":1.121,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.129,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":1.879}]},{"osd":7,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.5660000000000001,"15min":1.5840000000000001},"min":{"1min":1.302,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.194,"5min":2.339,"15min":2.5870000000000002},"last":2.0049999999999999},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.5609999999999999,"15min":1.5620000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1179999999999999,"5min":2.2959999999999998,"15min":2.665},"last":1.798}]}]},{"osd":5,"up_from":13,"seq":55834576032,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60952,"kb_used_data":3400,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310888,"statfs":{"total":96636764160,"available":96574349312,"internally_reserved":0,"allocated":3481600,"data_stored":1819740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1499999999999999,"15min":1.202},"min":{"1min":0.748,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5740000000000001,"5min":1.98,"15min":1.98},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1910000000000001,"15min":1.2190000000000001},"min":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.74,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.0049999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.165,"15min":1.218},"min":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.5449999999999999,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2849999999999999,"15min":1.3100000000000001},"min":{"1min":0.78800000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.752,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":1.6559999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.254,"15min":1.2969999999999999},"min":{"1min":0.77100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.7050000000000001,"5min":2.008,"15min":2.0870000000000002},"last":1.208},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.29},"min":{"1min":0.77600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.677,"5min":1.877,"15min":1.905},"last":1.0509999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.97699999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.694,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.355,"5min":1.4219999999999999,"15min":1.4259999999999999},"min":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.883,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":1.359}]},{"osd":4,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.329,"15min":1.3300000000000001},"min":{"1min":0.93600000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.857,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.8700000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.4910000000000001,"15min":1.4399999999999999},"min":{"1min":1.194,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1600000000000001,"5min":2.742,"15min":2.742},"last":1.508}]},{"osd":6,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5780000000000001,"15min":1.4890000000000001},"min":{"1min":1.036,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9450000000000001,"5min":13.234999999999999,"15min":13.234999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.585,"5min":1.5449999999999999,"15min":1.4870000000000001},"min":{"1min":1.1830000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9770000000000001,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":2.0070000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.5649999999999999,"15min":1.548},"min":{"1min":1.208,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.0979999999999999,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.669,"5min":1.609,"15min":1.6040000000000001},"min":{"1min":1.3260000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.02,"5min":2.351,"15min":2.351},"last":1.548}]}]},{"osd":4,"up_from":13,"seq":55834576032,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48480,"kb_used_data":3408,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323360,"statfs":{"total":96636764160,"available":96587120640,"internally_reserved":0,"allocated":3489792,"data_stored":1821386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.67800000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.3220000000000001,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4359999999999999,"15min":1.423},"min":{"1min":0.63600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.3609999999999998,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.498}]},{"osd":1,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2470000000000001,"15min":1.2849999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.2229999999999999,"5min":4.2229999999999999,"15min":4.2229999999999999},"last":1.266},{"interface":"front","average":{"1min":2.7799999999999998,"5min":1.635,"15min":1.371},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":28.907,"5min":28.907,"15min":28.907},"last":1.4470000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4810000000000001,"15min":1.335},"min":{"1min":0.70899999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":3.6120000000000001,"5min":17.175999999999998,"15min":17.175999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.496,"5min":1.444,"15min":1.391},"min":{"1min":0.59499999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.4080000000000004,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":1.619}]},{"osd":3,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.3260000000000001,"5min":1.5720000000000001,"15min":1.4159999999999999},"min":{"1min":0.60099999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":15.938000000000001,"5min":15.938000000000001,"15min":15.938000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.8069999999999999,"5min":1.5760000000000001,"15min":1.474},"min":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.787000000000001,"5min":10.787000000000001,"15min":10.787000000000001},"last":1.4710000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.591,"15min":1.4350000000000001},"min":{"1min":0.73199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":4.2939999999999996,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":3.0209999999999999,"5min":1.8160000000000001,"15min":1.5289999999999999},"min":{"1min":0.84899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":31.347999999999999,"5min":31.347999999999999,"15min":31.347999999999999},"last":1.552}]},{"osd":6,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.016,"5min":1.7290000000000001,"15min":1.5389999999999999},"min":{"1min":0.73699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":13.885,"5min":13.885,"15min":13.885},"last":1.4039999999999999},{"interface":"front","average":{"1min":2.117,"5min":1.833,"15min":1.6419999999999999},"min":{"1min":0.82499999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":13.446999999999999,"5min":13.446999999999999,"15min":13.446999999999999},"last":1.369}]},{"osd":7,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.528,"5min":1.821,"15min":1.611},"min":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":15.843999999999999,"5min":15.843999999999999,"15min":15.843999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.726,"15min":1.6659999999999999},"min":{"1min":0.82399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.968,"5min":5.609,"15min":5.609},"last":1.306}]}]},{"osd":3,"up_from":13,"seq":55834576034,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56808,"kb_used_data":3412,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315032,"statfs":{"total":96636764160,"available":96578592768,"internally_reserved":0,"allocated":3493888,"data_stored":1826555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92700000000000005,"15min":0.97399999999999998},"min":{"1min":0.32200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1990000000000001,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.131,"15min":1.1659999999999999},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5349999999999999,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.77400000000000002}]},{"osd":1,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.96799999999999997,"15min":1.022},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3560000000000001,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1890000000000001,"15min":1.2729999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.667,"5min":2.109,"15min":2.4889999999999999},"last":1.177}]},{"osd":2,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0629999999999999,"15min":1.1399999999999999},"min":{"1min":0.502,"5min":0.437,"15min":0.437},"max":{"1min":1.462,"5min":2.165,"15min":2.2250000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.139,"15min":1.1899999999999999},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.671,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.29}]},{"osd":4,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2290000000000001,"15min":1.272},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.67500000000000004},"max":{"1min":1.7290000000000001,"5min":1.857,"15min":2.5379999999999998},"last":1.234},{"interface":"front","average":{"1min":1.306,"5min":1.381,"15min":1.45},"min":{"1min":0.91600000000000004,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.8149999999999999,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.2609999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.294,"15min":1.3660000000000001},"min":{"1min":0.76700000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7829999999999999,"5min":1.923,"15min":2.9740000000000002},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3919999999999999,"15min":1.4330000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9610000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":1.5089999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.82299999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.9690000000000001,"5min":1.976,"15min":2.653},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.4350000000000001,"15min":1.5129999999999999},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.0150000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.4810000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.512,"15min":1.5580000000000001},"min":{"1min":0.89100000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.3220000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4490000000000001,"15min":1.512},"min":{"1min":0.80900000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.903,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.4219999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576032,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52624,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319216,"statfs":{"total":96636764160,"available":96582877184,"internally_reserved":0,"allocated":3473408,"data_stored":1819684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.86099999999999999,"15min":0.95299999999999996},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2270000000000001,"5min":1.601,"15min":1.948},"last":0.625},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.98699999999999999,"15min":1.0529999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.98799999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.90100000000000002,"15min":0.97399999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0609999999999999,"15min":1.167},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.587,"5min":1.7789999999999999,"15min":2.1309999999999998},"last":0.92900000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0860000000000001,"15min":1.1639999999999999},"min":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.9359999999999999,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.155,"15min":1.2490000000000001},"min":{"1min":0.47699999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6699999999999999,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.25}]},{"osd":4,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.3500000000000001,"15min":1.421},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8640000000000001,"5min":1.992,"15min":2.1789999999999998},"last":1.109},{"interface":"front","average":{"1min":1.161,"5min":1.2789999999999999,"15min":1.306},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.3620000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.3160000000000001,"15min":1.385},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.4279999999999999,"15min":1.51},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0750000000000002,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.3959999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3260000000000001,"15min":1.385},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.4339999999999999,"15min":1.51},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.0860000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.4059999999999999,"15min":1.464},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.4689999999999999},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.466,"15min":1.5580000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.831,"5min":2.831,"15min":2.831},"last":1.1499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834576032,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52644,"kb_used_data":3408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319196,"statfs":{"total":96636764160,"available":96582856704,"internally_reserved":0,"allocated":3489792,"data_stored":1826553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.97699999999999998,"15min":0.999},"min":{"1min":0.56000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.502,"5min":1.6579999999999999,"15min":1.708},"last":0.875},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0760000000000001,"15min":1.2},"min":{"1min":0.52900000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4790000000000001,"5min":1.7010000000000001,"15min":1.79},"last":0.78600000000000003}]},{"osd":2,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.876,"15min":0.94999999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.095,"5min":1.641,"15min":1.641},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.087,"15min":1.1419999999999999},"min":{"1min":0.42599999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.607,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.0960000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1519999999999999,"15min":1.2390000000000001},"min":{"1min":0.58599999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5960000000000001,"5min":2.496,"15min":2.496},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.91900000000000004,"5min":1.099,"15min":1.143},"min":{"1min":0.49399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.5369999999999999,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":1.0629999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.276,"15min":1.351},"min":{"1min":0.77700000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.579,"5min":2.73,"15min":2.8460000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.248,"15min":1.3009999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.5409999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.236}]},{"osd":5,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.302,"15min":1.3640000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.296},{"interface":"front","average":{"1min":1.337,"5min":1.476,"15min":1.5189999999999999},"min":{"1min":0.89600000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7070000000000001,"5min":2.601,"15min":2.601},"last":1.6040000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4330000000000001,"15min":1.502},"min":{"1min":0.92600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.7609999999999999,"5min":2.883,"15min":2.9079999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.411,"15min":1.484},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.6180000000000001,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.159}]},{"osd":7,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4830000000000001,"15min":1.5389999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.7749999999999999,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.512},{"interface":"front","average":{"1min":1.339,"5min":1.484,"15min":1.5569999999999999},"min":{"1min":0.83799999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.873,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.573}]}]},{"osd":0,"up_from":13,"seq":55834576033,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53220,"kb_used_data":3992,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318620,"statfs":{"total":96636764160,"available":96582266880,"internally_reserved":0,"allocated":4087808,"data_stored":2417447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.90300000000000002,"15min":0.95999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2310000000000001,"5min":1.581,"15min":1.581},"last":1.22},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.97499999999999998,"15min":1.069},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.5640000000000001,"5min":1.621,"15min":1.726},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.0429999999999999,"15min":1.1240000000000001},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3360000000000001,"5min":1.782,"15min":1.782},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.97599999999999998,"15min":1.0589999999999999},"min":{"1min":0.52700000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4239999999999999,"5min":1.6000000000000001,"15min":1.9930000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1399999999999999,"15min":1.2270000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5920000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.048},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.083,"15min":1.149},"min":{"1min":0.44,"5min":0.439,"15min":0.439},"max":{"1min":1.2929999999999999,"5min":1.7689999999999999,"15min":2.0979999999999999},"last":1.0740000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2789999999999999,"15min":1.327},"min":{"1min":0.59899999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3300000000000001,"15min":1.3999999999999999},"min":{"1min":0.69599999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.782,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.4079999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.369,"15min":1.409},"min":{"1min":0.67300000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6459999999999999,"5min":1.903,"15min":2.1309999999999998},"last":1.373},{"interface":"front","average":{"1min":1.341,"5min":1.454,"15min":1.4810000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9430000000000001,"5min":2.23,"15min":2.4900000000000002},"last":1.1559999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4099999999999999,"15min":1.4650000000000001},"min":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.877,"5min":2.21,"15min":2.2149999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4179999999999999,"15min":1.4850000000000001},"min":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7190000000000001,"5min":2.0880000000000001,"15min":2.246},"last":1.2490000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4790000000000001,"15min":1.5289999999999999},"min":{"1min":0.73299999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8380000000000001,"5min":2.1989999999999998,"15min":2.2349999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4850000000000001,"15min":1.54},"min":{"1min":0.78500000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9690000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.2929999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:15:50.953 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:15:51.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.168+0000 7f95c3997640 1 Processor -- start 2023-10-08T07:15:51.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.168+0000 7f95c3997640 1 -- start start 2023-10-08T07:15:51.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.169+0000 7f95c3997640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f95bc0c90b0 0x7f95bc0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.169+0000 7f95c3997640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0cb3b0 0x7f95bc15e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.169+0000 7f95c3997640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95bc15b2b0 con 0x7f95bc15ab40 2023-10-08T07:15:51.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.169+0000 7f95c3997640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95bc15bf30 con 0x7f95bc0c90b0 2023-10-08T07:15:51.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.169+0000 7f95c3997640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc0cb820 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.169+0000 7f95c0f32640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f95bc0c90b0 0x7f95bc0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.170+0000 7f95c0f32640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f95bc0c90b0 0x7f95bc0cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51462/0 (socket says 172.21.15.70:51462) 2023-10-08T07:15:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.170+0000 7f95c0f32640 1 -- 172.21.15.70:0/4246097168 learned_addr learned my addr 172.21.15.70:0/4246097168 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.170+0000 7f95c1f34640 1 --2- 172.21.15.70:0/4246097168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0cb3b0 0x7f95bc15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.170+0000 7f95c0f32640 1 -- 172.21.15.70:0/4246097168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0cb3b0 msgr2=0x7f95bc15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.170+0000 7f95c0f32640 1 --2- 172.21.15.70:0/4246097168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0cb3b0 0x7f95bc15e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.170+0000 7f95c0f32640 1 -- 172.21.15.70:0/4246097168 >> v1:172.21.15.181:6789/0 conn(0x7f95bc15ab40 legacy=0x7f95bc0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.170+0000 7f95c0f32640 1 -- 172.21.15.70:0/4246097168 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc15b590 con 0x7f95bc0c90b0 2023-10-08T07:15:51.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.171+0000 7f95c0f32640 1 --2- 172.21.15.70:0/4246097168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0c90b0 0x7f95bc0cae70 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f95bc06e560 tx=0x7f95ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769ddad3909d3d8d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:51.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.172+0000 7f95b3fff640 1 -- 172.21.15.70:0/4246097168 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac031830 con 0x7f95bc0c90b0 2023-10-08T07:15:51.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.172+0000 7f95b3fff640 1 -- 172.21.15.70:0/4246097168 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac031990 con 0x7f95bc0c90b0 2023-10-08T07:15:51.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.172+0000 7f95b3fff640 1 -- 172.21.15.70:0/4246097168 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac031d20 con 0x7f95bc0c90b0 2023-10-08T07:15:51.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.172+0000 7f95c3997640 1 -- 172.21.15.70:0/4246097168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0c90b0 msgr2=0x7f95bc0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:51.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.172+0000 7f95c3997640 1 --2- 172.21.15.70:0/4246097168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0c90b0 0x7f95bc0cae70 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f95bc06e560 tx=0x7f95ac02b8d0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.172+0000 7f95c3997640 1 -- 172.21.15.70:0/4246097168 shutdown_connections 2023-10-08T07:15:51.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.172+0000 7f95c3997640 1 --2- 172.21.15.70:0/4246097168 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0cb3b0 0x7f95bc15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.172+0000 7f95c3997640 1 --2- 172.21.15.70:0/4246097168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0c90b0 0x7f95bc0cae70 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.172+0000 7f95c3997640 1 -- 172.21.15.70:0/4246097168 >> 172.21.15.70:0/4246097168 conn(0x7f95bc06a380 msgr2=0x7f95bc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:51.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.172+0000 7f95c3997640 1 -- 172.21.15.70:0/4246097168 shutdown_connections 2023-10-08T07:15:51.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.173+0000 7f95c3997640 1 -- 172.21.15.70:0/4246097168 wait complete. 2023-10-08T07:15:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.174+0000 7f95c3997640 1 Processor -- start 2023-10-08T07:15:51.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.174+0000 7f95c3997640 1 -- start start 2023-10-08T07:15:51.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.175+0000 7f95c3997640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0c90b0 0x7f95bc152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.175+0000 7f95c3997640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0cb3b0 0x7f95bc152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.175+0000 7f95c3997640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95bc15ab40 0x7f95bc153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.175+0000 7f95c3997640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc0c3ce0 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.175+0000 7f95c3997640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc152bd0 con 0x7f95bc15ab40 2023-10-08T07:15:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.175+0000 7f95c3997640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95bc0c8300 con 0x7f95bc0c90b0 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c0f32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0cb3b0 0x7f95bc152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c0f32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0cb3b0 0x7f95bc152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51478/0 (socket says 172.21.15.70:51478) 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c0f32640 1 -- 172.21.15.70:0/3073569151 learned_addr learned my addr 172.21.15.70:0/3073569151 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c1733640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0c90b0 0x7f95bc152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c0f32640 1 -- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0c90b0 msgr2=0x7f95bc152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c1f34640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95bc15ab40 0x7f95bc153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c0f32640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0c90b0 0x7f95bc152690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c0f32640 1 -- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95bc15ab40 msgr2=0x7f95bc153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c0f32640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95bc15ab40 0x7f95bc153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c0f32640 1 -- 172.21.15.70:0/3073569151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0094d0 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.176+0000 7f95c0f32640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0cb3b0 0x7f95bc152e00 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0097f0 tx=0x7f95ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:51.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.177+0000 7f95b1ffb640 1 -- 172.21.15.70:0/3073569151 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac03a830 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.177+0000 7f95b1ffb640 1 -- 172.21.15.70:0/3073569151 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac03a990 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.177+0000 7f95b1ffb640 1 -- 172.21.15.70:0/3073569151 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac03ad40 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.177+0000 7f95c3997640 1 -- 172.21.15.70:0/3073569151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95bc0c85e0 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.177+0000 7f95c3997640 1 -- 172.21.15.70:0/3073569151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95bc0c3900 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.178+0000 7f95c3997640 1 -- 172.21.15.70:0/3073569151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95bc0c3e20 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.179+0000 7f95b1ffb640 1 -- 172.21.15.70:0/3073569151 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f95ac04d450 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.180+0000 7f95b1ffb640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f95a006b0b0 0x7f95a006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.180+0000 7f95b1ffb640 1 -- 172.21.15.70:0/3073569151 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f95ac0abdc0 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.180+0000 7f95c1733640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f95a006b0b0 0x7f95a006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.181+0000 7f95c1733640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f95a006b0b0 0x7f95a006d570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0b4580 tx=0x7f95b8009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.184+0000 7f95b1ffb640 1 -- 172.21.15.70:0/3073569151 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95bc0c3e20 con 0x7f95bc0cb3b0 2023-10-08T07:15:51.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.369+0000 7f95c3997640 1 -- 172.21.15.70:0/3073569151 --> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95bc15bb50 con 0x7f95a006b0b0 2023-10-08T07:15:51.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.392+0000 7f95b1ffb640 1 -- 172.21.15.70:0/3073569151 <== mgr.9067 v2:172.21.15.181:6832/440573135 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287559 (secure 0 0 0) 0x7f95b800d4f0 con 0x7f95a006b0b0 2023-10-08T07:15:51.415 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:51.437 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:15:51.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.398+0000 7f95c3997640 1 -- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f95a006b0b0 msgr2=0x7f95a006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:51.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.398+0000 7f95c3997640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f95a006b0b0 0x7f95a006d570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0b4580 tx=0x7f95b8009040 comp rx=0 tx=0).stop 2023-10-08T07:15:51.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.398+0000 7f95c3997640 1 -- 172.21.15.70:0/3073569151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0cb3b0 msgr2=0x7f95bc152e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:51.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.398+0000 7f95c3997640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0cb3b0 0x7f95bc152e00 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0097f0 tx=0x7f95ac0046c0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.398+0000 7f95c3997640 1 -- 172.21.15.70:0/3073569151 shutdown_connections 2023-10-08T07:15:51.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.398+0000 7f95c3997640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f95a006b0b0 0x7f95a006d570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.398+0000 7f95c3997640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f95bc15ab40 0x7f95bc153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.398+0000 7f95c3997640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95bc0cb3b0 0x7f95bc152e00 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.398+0000 7f95c3997640 1 --2- 172.21.15.70:0/3073569151 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f95bc0c90b0 0x7f95bc152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.398+0000 7f95c3997640 1 -- 172.21.15.70:0/3073569151 >> 172.21.15.70:0/3073569151 conn(0x7f95bc06a380 msgr2=0x7f95bc06cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:51.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.399+0000 7f95c3997640 1 -- 172.21.15.70:0/3073569151 shutdown_connections 2023-10-08T07:15:51.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.399+0000 7f95c3997640 1 -- 172.21.15.70:0/3073569151 wait complete. 2023-10-08T07:15:51.441 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-10-08T07:15:50.994154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":695,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4899,"ondisk_log_size":4899,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448640,"kb_used_data":28408,"kb_used_omap":12,"kb_used_meta":420083,"kb_avail":754526080,"statfs":{"total":773094113280,"available":772634705920,"internally_reserved":0,"allocated":29089792,"data_stored":15770144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":430165563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":89,"num_read_kb":201,"num_write":63,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002157"},"pg_stats":[{"pgid":"36.3f","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409729+0000","last_change":"2023-10-08T07:15:46.427641+0000","last_active":"2023-10-08T07:15:47.409729+0000","last_peered":"2023-10-08T07:15:47.409729+0000","last_clean":"2023-10-08T07:15:47.409729+0000","last_became_active":"2023-10-08T07:15:46.427272+0000","last_became_peered":"2023-10-08T07:15:46.427272+0000","last_unstale":"2023-10-08T07:15:47.409729+0000","last_undegraded":"2023-10-08T07:15:47.409729+0000","last_fullsized":"2023-10-08T07:15:47.409729+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:32:44.950131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3e","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409338+0000","last_change":"2023-10-08T07:15:46.439775+0000","last_active":"2023-10-08T07:15:47.409338+0000","last_peered":"2023-10-08T07:15:47.409338+0000","last_clean":"2023-10-08T07:15:47.409338+0000","last_became_active":"2023-10-08T07:15:46.439462+0000","last_became_peered":"2023-10-08T07:15:46.439462+0000","last_unstale":"2023-10-08T07:15:47.409338+0000","last_undegraded":"2023-10-08T07:15:47.409338+0000","last_fullsized":"2023-10-08T07:15:47.409338+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:45:03.705356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.3d","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406311+0000","last_change":"2023-10-08T07:15:46.443973+0000","last_active":"2023-10-08T07:15:47.406311+0000","last_peered":"2023-10-08T07:15:47.406311+0000","last_clean":"2023-10-08T07:15:47.406311+0000","last_became_active":"2023-10-08T07:15:46.443717+0000","last_became_peered":"2023-10-08T07:15:46.443717+0000","last_unstale":"2023-10-08T07:15:47.406311+0000","last_undegraded":"2023-10-08T07:15:47.406311+0000","last_fullsized":"2023-10-08T07:15:47.406311+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:39:15.492519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3c","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405935+0000","last_change":"2023-10-08T07:15:46.435801+0000","last_active":"2023-10-08T07:15:47.405935+0000","last_peered":"2023-10-08T07:15:47.405935+0000","last_clean":"2023-10-08T07:15:47.405935+0000","last_became_active":"2023-10-08T07:15:46.434714+0000","last_became_peered":"2023-10-08T07:15:46.434714+0000","last_unstale":"2023-10-08T07:15:47.405935+0000","last_undegraded":"2023-10-08T07:15:47.405935+0000","last_fullsized":"2023-10-08T07:15:47.405935+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:49:05.593791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.3b","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408920+0000","last_change":"2023-10-08T07:15:46.445225+0000","last_active":"2023-10-08T07:15:47.408920+0000","last_peered":"2023-10-08T07:15:47.408920+0000","last_clean":"2023-10-08T07:15:47.408920+0000","last_became_active":"2023-10-08T07:15:46.444956+0000","last_became_peered":"2023-10-08T07:15:46.444956+0000","last_unstale":"2023-10-08T07:15:47.408920+0000","last_undegraded":"2023-10-08T07:15:47.408920+0000","last_fullsized":"2023-10-08T07:15:47.408920+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:39.135783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.3a","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408094+0000","last_change":"2023-10-08T07:15:46.453293+0000","last_active":"2023-10-08T07:15:47.408094+0000","last_peered":"2023-10-08T07:15:47.408094+0000","last_clean":"2023-10-08T07:15:47.408094+0000","last_became_active":"2023-10-08T07:15:46.453012+0000","last_became_peered":"2023-10-08T07:15:46.453012+0000","last_unstale":"2023-10-08T07:15:47.408094+0000","last_undegraded":"2023-10-08T07:15:47.408094+0000","last_fullsized":"2023-10-08T07:15:47.408094+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:11.428056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.39","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599631+0000","last_change":"2023-10-08T07:15:46.442034+0000","last_active":"2023-10-08T07:15:47.599631+0000","last_peered":"2023-10-08T07:15:47.599631+0000","last_clean":"2023-10-08T07:15:47.599631+0000","last_became_active":"2023-10-08T07:15:46.441732+0000","last_became_peered":"2023-10-08T07:15:46.441732+0000","last_unstale":"2023-10-08T07:15:47.599631+0000","last_undegraded":"2023-10-08T07:15:47.599631+0000","last_fullsized":"2023-10-08T07:15:47.599631+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:35:53.021160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.38","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405294+0000","last_change":"2023-10-08T07:15:46.432534+0000","last_active":"2023-10-08T07:15:47.405294+0000","last_peered":"2023-10-08T07:15:47.405294+0000","last_clean":"2023-10-08T07:15:47.405294+0000","last_became_active":"2023-10-08T07:15:46.432257+0000","last_became_peered":"2023-10-08T07:15:46.432257+0000","last_unstale":"2023-10-08T07:15:47.405294+0000","last_undegraded":"2023-10-08T07:15:47.405294+0000","last_fullsized":"2023-10-08T07:15:47.405294+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:18.607729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.37","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600403+0000","last_change":"2023-10-08T07:15:46.449871+0000","last_active":"2023-10-08T07:15:47.600403+0000","last_peered":"2023-10-08T07:15:47.600403+0000","last_clean":"2023-10-08T07:15:47.600403+0000","last_became_active":"2023-10-08T07:15:46.449574+0000","last_became_peered":"2023-10-08T07:15:46.449574+0000","last_unstale":"2023-10-08T07:15:47.600403+0000","last_undegraded":"2023-10-08T07:15:47.600403+0000","last_fullsized":"2023-10-08T07:15:47.600403+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:28.103365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.36","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600616+0000","last_change":"2023-10-08T07:15:46.437670+0000","last_active":"2023-10-08T07:15:47.600616+0000","last_peered":"2023-10-08T07:15:47.600616+0000","last_clean":"2023-10-08T07:15:47.600616+0000","last_became_active":"2023-10-08T07:15:46.437351+0000","last_became_peered":"2023-10-08T07:15:46.437351+0000","last_unstale":"2023-10-08T07:15:47.600616+0000","last_undegraded":"2023-10-08T07:15:47.600616+0000","last_fullsized":"2023-10-08T07:15:47.600616+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:58.208711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.35","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.598936+0000","last_change":"2023-10-08T07:15:46.433741+0000","last_active":"2023-10-08T07:15:47.598936+0000","last_peered":"2023-10-08T07:15:47.598936+0000","last_clean":"2023-10-08T07:15:47.598936+0000","last_became_active":"2023-10-08T07:15:46.433309+0000","last_became_peered":"2023-10-08T07:15:46.433309+0000","last_unstale":"2023-10-08T07:15:47.598936+0000","last_undegraded":"2023-10-08T07:15:47.598936+0000","last_fullsized":"2023-10-08T07:15:47.598936+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:04.135629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.34","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409270+0000","last_change":"2023-10-08T07:15:46.429914+0000","last_active":"2023-10-08T07:15:47.409270+0000","last_peered":"2023-10-08T07:15:47.409270+0000","last_clean":"2023-10-08T07:15:47.409270+0000","last_became_active":"2023-10-08T07:15:46.429451+0000","last_became_peered":"2023-10-08T07:15:46.429451+0000","last_unstale":"2023-10-08T07:15:47.409270+0000","last_undegraded":"2023-10-08T07:15:47.409270+0000","last_fullsized":"2023-10-08T07:15:47.409270+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:19:10.831065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.33","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405880+0000","last_change":"2023-10-08T07:15:46.437175+0000","last_active":"2023-10-08T07:15:47.405880+0000","last_peered":"2023-10-08T07:15:47.405880+0000","last_clean":"2023-10-08T07:15:47.405880+0000","last_became_active":"2023-10-08T07:15:46.436344+0000","last_became_peered":"2023-10-08T07:15:46.436344+0000","last_unstale":"2023-10-08T07:15:47.405880+0000","last_undegraded":"2023-10-08T07:15:47.405880+0000","last_fullsized":"2023-10-08T07:15:47.405880+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:12:13.068611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.32","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599709+0000","last_change":"2023-10-08T07:15:46.439141+0000","last_active":"2023-10-08T07:15:47.599709+0000","last_peered":"2023-10-08T07:15:47.599709+0000","last_clean":"2023-10-08T07:15:47.599709+0000","last_became_active":"2023-10-08T07:15:46.438825+0000","last_became_peered":"2023-10-08T07:15:46.438825+0000","last_unstale":"2023-10-08T07:15:47.599709+0000","last_undegraded":"2023-10-08T07:15:47.599709+0000","last_fullsized":"2023-10-08T07:15:47.599709+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:30.523440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.31","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405551+0000","last_change":"2023-10-08T07:15:46.433769+0000","last_active":"2023-10-08T07:15:47.405551+0000","last_peered":"2023-10-08T07:15:47.405551+0000","last_clean":"2023-10-08T07:15:47.405551+0000","last_became_active":"2023-10-08T07:15:46.433495+0000","last_became_peered":"2023-10-08T07:15:46.433495+0000","last_unstale":"2023-10-08T07:15:47.405551+0000","last_undegraded":"2023-10-08T07:15:47.405551+0000","last_fullsized":"2023-10-08T07:15:47.405551+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:21:41.048050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.30","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601290+0000","last_change":"2023-10-08T07:15:46.449295+0000","last_active":"2023-10-08T07:15:47.601290+0000","last_peered":"2023-10-08T07:15:47.601290+0000","last_clean":"2023-10-08T07:15:47.601290+0000","last_became_active":"2023-10-08T07:15:46.448966+0000","last_became_peered":"2023-10-08T07:15:46.448966+0000","last_unstale":"2023-10-08T07:15:47.601290+0000","last_undegraded":"2023-10-08T07:15:47.601290+0000","last_fullsized":"2023-10-08T07:15:47.601290+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:37:00.292656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2f","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601675+0000","last_change":"2023-10-08T07:15:46.441378+0000","last_active":"2023-10-08T07:15:47.601675+0000","last_peered":"2023-10-08T07:15:47.601675+0000","last_clean":"2023-10-08T07:15:47.601675+0000","last_became_active":"2023-10-08T07:15:46.440290+0000","last_became_peered":"2023-10-08T07:15:46.440290+0000","last_unstale":"2023-10-08T07:15:47.601675+0000","last_undegraded":"2023-10-08T07:15:47.601675+0000","last_fullsized":"2023-10-08T07:15:47.601675+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:41:00.853372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2e","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406513+0000","last_change":"2023-10-08T07:15:46.439217+0000","last_active":"2023-10-08T07:15:47.406513+0000","last_peered":"2023-10-08T07:15:47.406513+0000","last_clean":"2023-10-08T07:15:47.406513+0000","last_became_active":"2023-10-08T07:15:46.438944+0000","last_became_peered":"2023-10-08T07:15:46.438944+0000","last_unstale":"2023-10-08T07:15:47.406513+0000","last_undegraded":"2023-10-08T07:15:47.406513+0000","last_fullsized":"2023-10-08T07:15:47.406513+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:44:34.626536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.2d","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601864+0000","last_change":"2023-10-08T07:15:46.441127+0000","last_active":"2023-10-08T07:15:47.601864+0000","last_peered":"2023-10-08T07:15:47.601864+0000","last_clean":"2023-10-08T07:15:47.601864+0000","last_became_active":"2023-10-08T07:15:46.440513+0000","last_became_peered":"2023-10-08T07:15:46.440513+0000","last_unstale":"2023-10-08T07:15:47.601864+0000","last_undegraded":"2023-10-08T07:15:47.601864+0000","last_fullsized":"2023-10-08T07:15:47.601864+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:03:04.333839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2c","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601997+0000","last_change":"2023-10-08T07:15:46.443804+0000","last_active":"2023-10-08T07:15:47.601997+0000","last_peered":"2023-10-08T07:15:47.601997+0000","last_clean":"2023-10-08T07:15:47.601997+0000","last_became_active":"2023-10-08T07:15:46.442909+0000","last_became_peered":"2023-10-08T07:15:46.442909+0000","last_unstale":"2023-10-08T07:15:47.601997+0000","last_undegraded":"2023-10-08T07:15:47.601997+0000","last_fullsized":"2023-10-08T07:15:47.601997+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:28.705460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.2b","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599429+0000","last_change":"2023-10-08T07:15:46.449788+0000","last_active":"2023-10-08T07:15:47.599429+0000","last_peered":"2023-10-08T07:15:47.599429+0000","last_clean":"2023-10-08T07:15:47.599429+0000","last_became_active":"2023-10-08T07:15:46.449529+0000","last_became_peered":"2023-10-08T07:15:46.449529+0000","last_unstale":"2023-10-08T07:15:47.599429+0000","last_undegraded":"2023-10-08T07:15:47.599429+0000","last_fullsized":"2023-10-08T07:15:47.599429+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:38.401202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.2a","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599524+0000","last_change":"2023-10-08T07:15:46.442759+0000","last_active":"2023-10-08T07:15:47.599524+0000","last_peered":"2023-10-08T07:15:47.599524+0000","last_clean":"2023-10-08T07:15:47.599524+0000","last_became_active":"2023-10-08T07:15:46.441880+0000","last_became_peered":"2023-10-08T07:15:46.441880+0000","last_unstale":"2023-10-08T07:15:47.599524+0000","last_undegraded":"2023-10-08T07:15:47.599524+0000","last_fullsized":"2023-10-08T07:15:47.599524+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:40.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.29","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599664+0000","last_change":"2023-10-08T07:15:46.442809+0000","last_active":"2023-10-08T07:15:47.599664+0000","last_peered":"2023-10-08T07:15:47.599664+0000","last_clean":"2023-10-08T07:15:47.599664+0000","last_became_active":"2023-10-08T07:15:46.442508+0000","last_became_peered":"2023-10-08T07:15:46.442508+0000","last_unstale":"2023-10-08T07:15:47.599664+0000","last_undegraded":"2023-10-08T07:15:47.599664+0000","last_fullsized":"2023-10-08T07:15:47.599664+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:54:20.869460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.28","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600071+0000","last_change":"2023-10-08T07:15:46.423490+0000","last_active":"2023-10-08T07:15:47.600071+0000","last_peered":"2023-10-08T07:15:47.600071+0000","last_clean":"2023-10-08T07:15:47.600071+0000","last_became_active":"2023-10-08T07:15:46.420875+0000","last_became_peered":"2023-10-08T07:15:46.420875+0000","last_unstale":"2023-10-08T07:15:47.600071+0000","last_undegraded":"2023-10-08T07:15:47.600071+0000","last_fullsized":"2023-10-08T07:15:47.600071+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:46.539176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.27","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409902+0000","last_change":"2023-10-08T07:15:46.438477+0000","last_active":"2023-10-08T07:15:47.409902+0000","last_peered":"2023-10-08T07:15:47.409902+0000","last_clean":"2023-10-08T07:15:47.409902+0000","last_became_active":"2023-10-08T07:15:46.435054+0000","last_became_peered":"2023-10-08T07:15:46.435054+0000","last_unstale":"2023-10-08T07:15:47.409902+0000","last_undegraded":"2023-10-08T07:15:47.409902+0000","last_fullsized":"2023-10-08T07:15:47.409902+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:30:26.096323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.26","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408646+0000","last_change":"2023-10-08T07:15:46.448268+0000","last_active":"2023-10-08T07:15:47.408646+0000","last_peered":"2023-10-08T07:15:47.408646+0000","last_clean":"2023-10-08T07:15:47.408646+0000","last_became_active":"2023-10-08T07:15:46.447992+0000","last_became_peered":"2023-10-08T07:15:46.447992+0000","last_unstale":"2023-10-08T07:15:47.408646+0000","last_undegraded":"2023-10-08T07:15:47.408646+0000","last_fullsized":"2023-10-08T07:15:47.408646+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:52.997157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.24","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408676+0000","last_change":"2023-10-08T07:15:46.440817+0000","last_active":"2023-10-08T07:15:47.408676+0000","last_peered":"2023-10-08T07:15:47.408676+0000","last_clean":"2023-10-08T07:15:47.408676+0000","last_became_active":"2023-10-08T07:15:46.440518+0000","last_became_peered":"2023-10-08T07:15:46.440518+0000","last_unstale":"2023-10-08T07:15:47.408676+0000","last_undegraded":"2023-10-08T07:15:47.408676+0000","last_fullsized":"2023-10-08T07:15:47.408676+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:03:06.129360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.23","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599811+0000","last_change":"2023-10-08T07:15:46.453912+0000","last_active":"2023-10-08T07:15:47.599811+0000","last_peered":"2023-10-08T07:15:47.599811+0000","last_clean":"2023-10-08T07:15:47.599811+0000","last_became_active":"2023-10-08T07:15:46.453612+0000","last_became_peered":"2023-10-08T07:15:46.453612+0000","last_unstale":"2023-10-08T07:15:47.599811+0000","last_undegraded":"2023-10-08T07:15:47.599811+0000","last_fullsized":"2023-10-08T07:15:47.599811+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:52.997157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.22","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532101+0000","last_change":"2023-10-08T07:15:46.437205+0000","last_active":"2023-10-08T07:15:47.532101+0000","last_peered":"2023-10-08T07:15:47.532101+0000","last_clean":"2023-10-08T07:15:47.532101+0000","last_became_active":"2023-10-08T07:15:46.436874+0000","last_became_peered":"2023-10-08T07:15:46.436874+0000","last_unstale":"2023-10-08T07:15:47.532101+0000","last_undegraded":"2023-10-08T07:15:47.532101+0000","last_fullsized":"2023-10-08T07:15:47.532101+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:32:25.491400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.21","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599594+0000","last_change":"2023-10-08T07:15:46.446572+0000","last_active":"2023-10-08T07:15:47.599594+0000","last_peered":"2023-10-08T07:15:47.599594+0000","last_clean":"2023-10-08T07:15:47.599594+0000","last_became_active":"2023-10-08T07:15:46.446279+0000","last_became_peered":"2023-10-08T07:15:46.446279+0000","last_unstale":"2023-10-08T07:15:47.599594+0000","last_undegraded":"2023-10-08T07:15:47.599594+0000","last_fullsized":"2023-10-08T07:15:47.599594+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:36.057754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.20","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405420+0000","last_change":"2023-10-08T07:15:46.456557+0000","last_active":"2023-10-08T07:15:47.405420+0000","last_peered":"2023-10-08T07:15:47.405420+0000","last_clean":"2023-10-08T07:15:47.405420+0000","last_became_active":"2023-10-08T07:15:46.455964+0000","last_became_peered":"2023-10-08T07:15:46.455964+0000","last_unstale":"2023-10-08T07:15:47.405420+0000","last_undegraded":"2023-10-08T07:15:47.405420+0000","last_fullsized":"2023-10-08T07:15:47.405420+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:50:38.543864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.18","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600324+0000","last_change":"2023-10-08T07:15:46.428625+0000","last_active":"2023-10-08T07:15:47.600324+0000","last_peered":"2023-10-08T07:15:47.600324+0000","last_clean":"2023-10-08T07:15:47.600324+0000","last_became_active":"2023-10-08T07:15:46.422160+0000","last_became_peered":"2023-10-08T07:15:46.422160+0000","last_unstale":"2023-10-08T07:15:47.600324+0000","last_undegraded":"2023-10-08T07:15:47.600324+0000","last_fullsized":"2023-10-08T07:15:47.600324+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:57.091978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1f","version":"275'4","reported_seq":16,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600652+0000","last_change":"2023-10-08T07:15:45.455540+0000","last_active":"2023-10-08T07:15:47.600652+0000","last_peered":"2023-10-08T07:15:47.600652+0000","last_clean":"2023-10-08T07:15:47.600652+0000","last_became_active":"2023-10-08T07:15:45.455294+0000","last_became_peered":"2023-10-08T07:15:45.455294+0000","last_unstale":"2023-10-08T07:15:47.600652+0000","last_undegraded":"2023-10-08T07:15:47.600652+0000","last_fullsized":"2023-10-08T07:15:47.600652+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:26:26.849065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.19","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599985+0000","last_change":"2023-10-08T07:15:46.437125+0000","last_active":"2023-10-08T07:15:47.599985+0000","last_peered":"2023-10-08T07:15:47.599985+0000","last_clean":"2023-10-08T07:15:47.599985+0000","last_became_active":"2023-10-08T07:15:46.436815+0000","last_became_peered":"2023-10-08T07:15:46.436815+0000","last_unstale":"2023-10-08T07:15:47.599985+0000","last_undegraded":"2023-10-08T07:15:47.599985+0000","last_fullsized":"2023-10-08T07:15:47.599985+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:11:32.571333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1e","version":"275'1","reported_seq":13,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600421+0000","last_change":"2023-10-08T07:15:45.449509+0000","last_active":"2023-10-08T07:15:47.600421+0000","last_peered":"2023-10-08T07:15:47.600421+0000","last_clean":"2023-10-08T07:15:47.600421+0000","last_became_active":"2023-10-08T07:15:45.448744+0000","last_became_peered":"2023-10-08T07:15:45.448744+0000","last_unstale":"2023-10-08T07:15:47.600421+0000","last_undegraded":"2023-10-08T07:15:47.600421+0000","last_fullsized":"2023-10-08T07:15:47.600421+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:49:31.904624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.1a","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408963+0000","last_change":"2023-10-08T07:15:46.444369+0000","last_active":"2023-10-08T07:15:47.408963+0000","last_peered":"2023-10-08T07:15:47.408963+0000","last_clean":"2023-10-08T07:15:47.408963+0000","last_became_active":"2023-10-08T07:15:46.443549+0000","last_became_peered":"2023-10-08T07:15:46.443549+0000","last_unstale":"2023-10-08T07:15:47.408963+0000","last_undegraded":"2023-10-08T07:15:47.408963+0000","last_fullsized":"2023-10-08T07:15:47.408963+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:15.152757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.1d","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599037+0000","last_change":"2023-10-08T07:15:45.452873+0000","last_active":"2023-10-08T07:15:47.599037+0000","last_peered":"2023-10-08T07:15:47.599037+0000","last_clean":"2023-10-08T07:15:47.599037+0000","last_became_active":"2023-10-08T07:15:45.452616+0000","last_became_peered":"2023-10-08T07:15:45.452616+0000","last_unstale":"2023-10-08T07:15:47.599037+0000","last_undegraded":"2023-10-08T07:15:47.599037+0000","last_fullsized":"2023-10-08T07:15:47.599037+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:05:30.770137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1b","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599740+0000","last_change":"2023-10-08T07:15:46.436119+0000","last_active":"2023-10-08T07:15:47.599740+0000","last_peered":"2023-10-08T07:15:47.599740+0000","last_clean":"2023-10-08T07:15:47.599740+0000","last_became_active":"2023-10-08T07:15:46.435825+0000","last_became_peered":"2023-10-08T07:15:46.435825+0000","last_unstale":"2023-10-08T07:15:47.599740+0000","last_undegraded":"2023-10-08T07:15:47.599740+0000","last_fullsized":"2023-10-08T07:15:47.599740+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:46.766818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.1c","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599226+0000","last_change":"2023-10-08T07:15:45.450620+0000","last_active":"2023-10-08T07:15:47.599226+0000","last_peered":"2023-10-08T07:15:47.599226+0000","last_clean":"2023-10-08T07:15:47.599226+0000","last_became_active":"2023-10-08T07:15:45.450337+0000","last_became_peered":"2023-10-08T07:15:45.450337+0000","last_unstale":"2023-10-08T07:15:47.599226+0000","last_undegraded":"2023-10-08T07:15:47.599226+0000","last_fullsized":"2023-10-08T07:15:47.599226+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:37:45.461532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.1c","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406284+0000","last_change":"2023-10-08T07:15:46.439974+0000","last_active":"2023-10-08T07:15:47.406284+0000","last_peered":"2023-10-08T07:15:47.406284+0000","last_clean":"2023-10-08T07:15:47.406284+0000","last_became_active":"2023-10-08T07:15:46.439611+0000","last_became_peered":"2023-10-08T07:15:46.439611+0000","last_unstale":"2023-10-08T07:15:47.406284+0000","last_undegraded":"2023-10-08T07:15:47.406284+0000","last_fullsized":"2023-10-08T07:15:47.406284+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:14.273565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.1b","version":"275'1","reported_seq":13,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408483+0000","last_change":"2023-10-08T07:15:45.450095+0000","last_active":"2023-10-08T07:15:47.408483+0000","last_peered":"2023-10-08T07:15:47.408483+0000","last_clean":"2023-10-08T07:15:47.408483+0000","last_became_active":"2023-10-08T07:15:45.449825+0000","last_became_peered":"2023-10-08T07:15:45.449825+0000","last_unstale":"2023-10-08T07:15:47.408483+0000","last_undegraded":"2023-10-08T07:15:47.408483+0000","last_fullsized":"2023-10-08T07:15:47.408483+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:56:27.693885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.1d","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600420+0000","last_change":"2023-10-08T07:15:46.418445+0000","last_active":"2023-10-08T07:15:47.600420+0000","last_peered":"2023-10-08T07:15:47.600420+0000","last_clean":"2023-10-08T07:15:47.600420+0000","last_became_active":"2023-10-08T07:15:46.417718+0000","last_became_peered":"2023-10-08T07:15:46.417718+0000","last_unstale":"2023-10-08T07:15:47.600420+0000","last_undegraded":"2023-10-08T07:15:47.600420+0000","last_fullsized":"2023-10-08T07:15:47.600420+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:57:41.290522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.1a","version":"275'1","reported_seq":13,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408312+0000","last_change":"2023-10-08T07:15:45.451317+0000","last_active":"2023-10-08T07:15:47.408312+0000","last_peered":"2023-10-08T07:15:47.408312+0000","last_clean":"2023-10-08T07:15:47.408312+0000","last_became_active":"2023-10-08T07:15:45.450948+0000","last_became_peered":"2023-10-08T07:15:45.450948+0000","last_unstale":"2023-10-08T07:15:47.408312+0000","last_undegraded":"2023-10-08T07:15:47.408312+0000","last_fullsized":"2023-10-08T07:15:47.408312+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:30.407492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.1e","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600279+0000","last_change":"2023-10-08T07:15:46.434383+0000","last_active":"2023-10-08T07:15:47.600279+0000","last_peered":"2023-10-08T07:15:47.600279+0000","last_clean":"2023-10-08T07:15:47.600279+0000","last_became_active":"2023-10-08T07:15:46.434126+0000","last_became_peered":"2023-10-08T07:15:46.434126+0000","last_unstale":"2023-10-08T07:15:47.600279+0000","last_undegraded":"2023-10-08T07:15:47.600279+0000","last_fullsized":"2023-10-08T07:15:47.600279+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:39.135783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.19","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532030+0000","last_change":"2023-10-08T07:15:45.449753+0000","last_active":"2023-10-08T07:15:47.532030+0000","last_peered":"2023-10-08T07:15:47.532030+0000","last_clean":"2023-10-08T07:15:47.532030+0000","last_became_active":"2023-10-08T07:15:45.449467+0000","last_became_peered":"2023-10-08T07:15:45.449467+0000","last_unstale":"2023-10-08T07:15:47.532030+0000","last_undegraded":"2023-10-08T07:15:47.532030+0000","last_fullsized":"2023-10-08T07:15:47.532030+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:26.976546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1f","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.602065+0000","last_change":"2023-10-08T07:15:46.441363+0000","last_active":"2023-10-08T07:15:47.602065+0000","last_peered":"2023-10-08T07:15:47.602065+0000","last_clean":"2023-10-08T07:15:47.602065+0000","last_became_active":"2023-10-08T07:15:46.440757+0000","last_became_peered":"2023-10-08T07:15:46.440757+0000","last_unstale":"2023-10-08T07:15:47.602065+0000","last_undegraded":"2023-10-08T07:15:47.602065+0000","last_fullsized":"2023-10-08T07:15:47.602065+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:26.756328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.18","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405709+0000","last_change":"2023-10-08T07:15:45.447912+0000","last_active":"2023-10-08T07:15:47.405709+0000","last_peered":"2023-10-08T07:15:47.405709+0000","last_clean":"2023-10-08T07:15:47.405709+0000","last_became_active":"2023-10-08T07:15:45.447445+0000","last_became_peered":"2023-10-08T07:15:45.447445+0000","last_unstale":"2023-10-08T07:15:47.405709+0000","last_undegraded":"2023-10-08T07:15:47.405709+0000","last_fullsized":"2023-10-08T07:15:47.405709+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:26.504136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.10","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405453+0000","last_change":"2023-10-08T07:15:46.438059+0000","last_active":"2023-10-08T07:15:47.405453+0000","last_peered":"2023-10-08T07:15:47.405453+0000","last_clean":"2023-10-08T07:15:47.405453+0000","last_became_active":"2023-10-08T07:15:46.437782+0000","last_became_peered":"2023-10-08T07:15:46.437782+0000","last_unstale":"2023-10-08T07:15:47.405453+0000","last_undegraded":"2023-10-08T07:15:47.405453+0000","last_fullsized":"2023-10-08T07:15:47.405453+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:57:05.013108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.17","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.602212+0000","last_change":"2023-10-08T07:15:45.459521+0000","last_active":"2023-10-08T07:15:47.602212+0000","last_peered":"2023-10-08T07:15:47.602212+0000","last_clean":"2023-10-08T07:15:47.602212+0000","last_became_active":"2023-10-08T07:15:45.459238+0000","last_became_peered":"2023-10-08T07:15:45.459238+0000","last_unstale":"2023-10-08T07:15:47.602212+0000","last_undegraded":"2023-10-08T07:15:47.602212+0000","last_fullsized":"2023-10-08T07:15:47.602212+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:33.962135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.11","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408580+0000","last_change":"2023-10-08T07:15:46.433353+0000","last_active":"2023-10-08T07:15:47.408580+0000","last_peered":"2023-10-08T07:15:47.408580+0000","last_clean":"2023-10-08T07:15:47.408580+0000","last_became_active":"2023-10-08T07:15:46.431825+0000","last_became_peered":"2023-10-08T07:15:46.431825+0000","last_unstale":"2023-10-08T07:15:47.408580+0000","last_undegraded":"2023-10-08T07:15:47.408580+0000","last_fullsized":"2023-10-08T07:15:47.408580+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:26.756328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.16","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532465+0000","last_change":"2023-10-08T07:15:45.451533+0000","last_active":"2023-10-08T07:15:47.532465+0000","last_peered":"2023-10-08T07:15:47.532465+0000","last_clean":"2023-10-08T07:15:47.532465+0000","last_became_active":"2023-10-08T07:15:45.450582+0000","last_became_peered":"2023-10-08T07:15:45.450582+0000","last_unstale":"2023-10-08T07:15:47.532465+0000","last_undegraded":"2023-10-08T07:15:47.532465+0000","last_fullsized":"2023-10-08T07:15:47.532465+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:36.380720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.12","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406078+0000","last_change":"2023-10-08T07:15:46.442443+0000","last_active":"2023-10-08T07:15:47.406078+0000","last_peered":"2023-10-08T07:15:47.406078+0000","last_clean":"2023-10-08T07:15:47.406078+0000","last_became_active":"2023-10-08T07:15:46.442183+0000","last_became_peered":"2023-10-08T07:15:46.442183+0000","last_unstale":"2023-10-08T07:15:47.406078+0000","last_undegraded":"2023-10-08T07:15:47.406078+0000","last_fullsized":"2023-10-08T07:15:47.406078+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:02:06.340794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.15","version":"275'3","reported_seq":15,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601969+0000","last_change":"2023-10-08T07:15:45.445041+0000","last_active":"2023-10-08T07:15:47.601969+0000","last_peered":"2023-10-08T07:15:47.601969+0000","last_clean":"2023-10-08T07:15:47.601969+0000","last_became_active":"2023-10-08T07:15:45.444722+0000","last_became_peered":"2023-10-08T07:15:45.444722+0000","last_unstale":"2023-10-08T07:15:47.601969+0000","last_undegraded":"2023-10-08T07:15:47.601969+0000","last_fullsized":"2023-10-08T07:15:47.601969+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:24.302275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.13","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409038+0000","last_change":"2023-10-08T07:15:46.459196+0000","last_active":"2023-10-08T07:15:47.409038+0000","last_peered":"2023-10-08T07:15:47.409038+0000","last_clean":"2023-10-08T07:15:47.409038+0000","last_became_active":"2023-10-08T07:15:46.458898+0000","last_became_peered":"2023-10-08T07:15:46.458898+0000","last_unstale":"2023-10-08T07:15:47.409038+0000","last_undegraded":"2023-10-08T07:15:47.409038+0000","last_fullsized":"2023-10-08T07:15:47.409038+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:32:57.091978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.14","version":"275'6","reported_seq":18,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600683+0000","last_change":"2023-10-08T07:15:45.457442+0000","last_active":"2023-10-08T07:15:47.600683+0000","last_peered":"2023-10-08T07:15:47.600683+0000","last_clean":"2023-10-08T07:15:47.600683+0000","last_became_active":"2023-10-08T07:15:45.457174+0000","last_became_peered":"2023-10-08T07:15:45.457174+0000","last_unstale":"2023-10-08T07:15:47.600683+0000","last_undegraded":"2023-10-08T07:15:47.600683+0000","last_fullsized":"2023-10-08T07:15:47.600683+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:29.708621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.14","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409462+0000","last_change":"2023-10-08T07:15:46.434283+0000","last_active":"2023-10-08T07:15:47.409462+0000","last_peered":"2023-10-08T07:15:47.409462+0000","last_clean":"2023-10-08T07:15:47.409462+0000","last_became_active":"2023-10-08T07:15:46.433962+0000","last_became_peered":"2023-10-08T07:15:46.433962+0000","last_unstale":"2023-10-08T07:15:47.409462+0000","last_undegraded":"2023-10-08T07:15:47.409462+0000","last_fullsized":"2023-10-08T07:15:47.409462+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:03:19.945315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"35.13","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532197+0000","last_change":"2023-10-08T07:15:45.452819+0000","last_active":"2023-10-08T07:15:47.532197+0000","last_peered":"2023-10-08T07:15:47.532197+0000","last_clean":"2023-10-08T07:15:47.532197+0000","last_became_active":"2023-10-08T07:15:45.452499+0000","last_became_peered":"2023-10-08T07:15:45.452499+0000","last_unstale":"2023-10-08T07:15:47.532197+0000","last_undegraded":"2023-10-08T07:15:47.532197+0000","last_fullsized":"2023-10-08T07:15:47.532197+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:31.199635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.15","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406641+0000","last_change":"2023-10-08T07:15:46.435436+0000","last_active":"2023-10-08T07:15:47.406641+0000","last_peered":"2023-10-08T07:15:47.406641+0000","last_clean":"2023-10-08T07:15:47.406641+0000","last_became_active":"2023-10-08T07:15:46.435153+0000","last_became_peered":"2023-10-08T07:15:46.435153+0000","last_unstale":"2023-10-08T07:15:47.406641+0000","last_undegraded":"2023-10-08T07:15:47.406641+0000","last_fullsized":"2023-10-08T07:15:47.406641+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:52.673356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.12","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408974+0000","last_change":"2023-10-08T07:15:45.445901+0000","last_active":"2023-10-08T07:15:47.408974+0000","last_peered":"2023-10-08T07:15:47.408974+0000","last_clean":"2023-10-08T07:15:47.408974+0000","last_became_active":"2023-10-08T07:15:45.445598+0000","last_became_peered":"2023-10-08T07:15:45.445598+0000","last_unstale":"2023-10-08T07:15:47.408974+0000","last_undegraded":"2023-10-08T07:15:47.408974+0000","last_fullsized":"2023-10-08T07:15:47.408974+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:45.579097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.16","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601582+0000","last_change":"2023-10-08T07:15:46.442496+0000","last_active":"2023-10-08T07:15:47.601582+0000","last_peered":"2023-10-08T07:15:47.601582+0000","last_clean":"2023-10-08T07:15:47.601582+0000","last_became_active":"2023-10-08T07:15:46.442211+0000","last_became_peered":"2023-10-08T07:15:46.442211+0000","last_unstale":"2023-10-08T07:15:47.601582+0000","last_undegraded":"2023-10-08T07:15:47.601582+0000","last_fullsized":"2023-10-08T07:15:47.601582+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:46.590671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.11","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409264+0000","last_change":"2023-10-08T07:15:45.455026+0000","last_active":"2023-10-08T07:15:47.409264+0000","last_peered":"2023-10-08T07:15:47.409264+0000","last_clean":"2023-10-08T07:15:47.409264+0000","last_became_active":"2023-10-08T07:15:45.454768+0000","last_became_peered":"2023-10-08T07:15:45.454768+0000","last_unstale":"2023-10-08T07:15:47.409264+0000","last_undegraded":"2023-10-08T07:15:47.409264+0000","last_fullsized":"2023-10-08T07:15:47.409264+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:40:59.841798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.17","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.602164+0000","last_change":"2023-10-08T07:15:46.441540+0000","last_active":"2023-10-08T07:15:47.602164+0000","last_peered":"2023-10-08T07:15:47.602164+0000","last_clean":"2023-10-08T07:15:47.602164+0000","last_became_active":"2023-10-08T07:15:46.441047+0000","last_became_peered":"2023-10-08T07:15:46.441047+0000","last_unstale":"2023-10-08T07:15:47.602164+0000","last_undegraded":"2023-10-08T07:15:47.602164+0000","last_fullsized":"2023-10-08T07:15:47.602164+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:15.152757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.10","version":"275'8","reported_seq":20,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599641+0000","last_change":"2023-10-08T07:15:45.454962+0000","last_active":"2023-10-08T07:15:47.599641+0000","last_peered":"2023-10-08T07:15:47.599641+0000","last_clean":"2023-10-08T07:15:47.599641+0000","last_became_active":"2023-10-08T07:15:45.454685+0000","last_became_peered":"2023-10-08T07:15:45.454685+0000","last_unstale":"2023-10-08T07:15:47.599641+0000","last_undegraded":"2023-10-08T07:15:47.599641+0000","last_fullsized":"2023-10-08T07:15:47.599641+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:52:28.362795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"36.8","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408517+0000","last_change":"2023-10-08T07:15:46.444827+0000","last_active":"2023-10-08T07:15:47.408517+0000","last_peered":"2023-10-08T07:15:47.408517+0000","last_clean":"2023-10-08T07:15:47.408517+0000","last_became_active":"2023-10-08T07:15:46.444427+0000","last_became_peered":"2023-10-08T07:15:46.444427+0000","last_unstale":"2023-10-08T07:15:47.408517+0000","last_undegraded":"2023-10-08T07:15:47.408517+0000","last_fullsized":"2023-10-08T07:15:47.408517+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:16:36.057754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.f","version":"275'1","reported_seq":13,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599888+0000","last_change":"2023-10-08T07:15:45.454339+0000","last_active":"2023-10-08T07:15:47.599888+0000","last_peered":"2023-10-08T07:15:47.599888+0000","last_clean":"2023-10-08T07:15:47.599888+0000","last_became_active":"2023-10-08T07:15:45.454038+0000","last_became_peered":"2023-10-08T07:15:45.454038+0000","last_unstale":"2023-10-08T07:15:47.599888+0000","last_undegraded":"2023-10-08T07:15:47.599888+0000","last_fullsized":"2023-10-08T07:15:47.599888+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:40.455212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.9","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599962+0000","last_change":"2023-10-08T07:15:46.433330+0000","last_active":"2023-10-08T07:15:47.599962+0000","last_peered":"2023-10-08T07:15:47.599962+0000","last_clean":"2023-10-08T07:15:47.599962+0000","last_became_active":"2023-10-08T07:15:46.431617+0000","last_became_peered":"2023-10-08T07:15:46.431617+0000","last_unstale":"2023-10-08T07:15:47.599962+0000","last_undegraded":"2023-10-08T07:15:47.599962+0000","last_fullsized":"2023-10-08T07:15:47.599962+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:03:06.129360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.e","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408753+0000","last_change":"2023-10-08T07:15:45.442328+0000","last_active":"2023-10-08T07:15:47.408753+0000","last_peered":"2023-10-08T07:15:47.408753+0000","last_clean":"2023-10-08T07:15:47.408753+0000","last_became_active":"2023-10-08T07:15:45.441883+0000","last_became_peered":"2023-10-08T07:15:45.441883+0000","last_unstale":"2023-10-08T07:15:47.408753+0000","last_undegraded":"2023-10-08T07:15:47.408753+0000","last_fullsized":"2023-10-08T07:15:47.408753+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:35:23.168510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"36.a","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408176+0000","last_change":"2023-10-08T07:15:46.439306+0000","last_active":"2023-10-08T07:15:47.408176+0000","last_peered":"2023-10-08T07:15:47.408176+0000","last_clean":"2023-10-08T07:15:47.408176+0000","last_became_active":"2023-10-08T07:15:46.439016+0000","last_became_peered":"2023-10-08T07:15:46.439016+0000","last_unstale":"2023-10-08T07:15:47.408176+0000","last_undegraded":"2023-10-08T07:15:47.408176+0000","last_fullsized":"2023-10-08T07:15:47.408176+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:58:40.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.d","version":"275'1","reported_seq":13,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599132+0000","last_change":"2023-10-08T07:15:45.451959+0000","last_active":"2023-10-08T07:15:47.599132+0000","last_peered":"2023-10-08T07:15:47.599132+0000","last_clean":"2023-10-08T07:15:47.599132+0000","last_became_active":"2023-10-08T07:15:45.451698+0000","last_became_peered":"2023-10-08T07:15:45.451698+0000","last_unstale":"2023-10-08T07:15:47.599132+0000","last_undegraded":"2023-10-08T07:15:47.599132+0000","last_fullsized":"2023-10-08T07:15:47.599132+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:46:47.747067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.b","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406067+0000","last_change":"2023-10-08T07:15:46.430865+0000","last_active":"2023-10-08T07:15:47.406067+0000","last_peered":"2023-10-08T07:15:47.406067+0000","last_clean":"2023-10-08T07:15:47.406067+0000","last_became_active":"2023-10-08T07:15:46.430582+0000","last_became_peered":"2023-10-08T07:15:46.430582+0000","last_unstale":"2023-10-08T07:15:47.406067+0000","last_undegraded":"2023-10-08T07:15:47.406067+0000","last_fullsized":"2023-10-08T07:15:47.406067+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:54.581182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.c","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532249+0000","last_change":"2023-10-08T07:15:45.455056+0000","last_active":"2023-10-08T07:15:47.532249+0000","last_peered":"2023-10-08T07:15:47.532249+0000","last_clean":"2023-10-08T07:15:47.532249+0000","last_became_active":"2023-10-08T07:15:45.454806+0000","last_became_peered":"2023-10-08T07:15:45.454806+0000","last_unstale":"2023-10-08T07:15:47.532249+0000","last_undegraded":"2023-10-08T07:15:47.532249+0000","last_fullsized":"2023-10-08T07:15:47.532249+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:36.844747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.c","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600039+0000","last_change":"2023-10-08T07:15:46.442950+0000","last_active":"2023-10-08T07:15:47.600039+0000","last_peered":"2023-10-08T07:15:47.600039+0000","last_clean":"2023-10-08T07:15:47.600039+0000","last_became_active":"2023-10-08T07:15:46.442281+0000","last_became_peered":"2023-10-08T07:15:46.442281+0000","last_unstale":"2023-10-08T07:15:47.600039+0000","last_undegraded":"2023-10-08T07:15:47.600039+0000","last_fullsized":"2023-10-08T07:15:47.600039+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:38.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.b","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532453+0000","last_change":"2023-10-08T07:15:45.448839+0000","last_active":"2023-10-08T07:15:47.532453+0000","last_peered":"2023-10-08T07:15:47.532453+0000","last_clean":"2023-10-08T07:15:47.532453+0000","last_became_active":"2023-10-08T07:15:45.448511+0000","last_became_peered":"2023-10-08T07:15:45.448511+0000","last_unstale":"2023-10-08T07:15:47.532453+0000","last_undegraded":"2023-10-08T07:15:47.532453+0000","last_fullsized":"2023-10-08T07:15:47.532453+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:56:39.425788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.d","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600690+0000","last_change":"2023-10-08T07:15:46.431513+0000","last_active":"2023-10-08T07:15:47.600690+0000","last_peered":"2023-10-08T07:15:47.600690+0000","last_clean":"2023-10-08T07:15:47.600690+0000","last_became_active":"2023-10-08T07:15:46.431216+0000","last_became_peered":"2023-10-08T07:15:46.431216+0000","last_unstale":"2023-10-08T07:15:47.600690+0000","last_undegraded":"2023-10-08T07:15:47.600690+0000","last_fullsized":"2023-10-08T07:15:47.600690+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:26.726652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"35.a","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406313+0000","last_change":"2023-10-08T07:15:45.447118+0000","last_active":"2023-10-08T07:15:47.406313+0000","last_peered":"2023-10-08T07:15:47.406313+0000","last_clean":"2023-10-08T07:15:47.406313+0000","last_became_active":"2023-10-08T07:15:45.446659+0000","last_became_peered":"2023-10-08T07:15:45.446659+0000","last_unstale":"2023-10-08T07:15:47.406313+0000","last_undegraded":"2023-10-08T07:15:47.406313+0000","last_fullsized":"2023-10-08T07:15:47.406313+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:09:50.887401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.e","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601980+0000","last_change":"2023-10-08T07:15:46.442076+0000","last_active":"2023-10-08T07:15:47.601980+0000","last_peered":"2023-10-08T07:15:47.601980+0000","last_clean":"2023-10-08T07:15:47.601980+0000","last_became_active":"2023-10-08T07:15:46.441744+0000","last_became_peered":"2023-10-08T07:15:46.441744+0000","last_unstale":"2023-10-08T07:15:47.601980+0000","last_undegraded":"2023-10-08T07:15:47.601980+0000","last_fullsized":"2023-10-08T07:15:47.601980+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:41.656686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.9","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600253+0000","last_change":"2023-10-08T07:15:45.445842+0000","last_active":"2023-10-08T07:15:47.600253+0000","last_peered":"2023-10-08T07:15:47.600253+0000","last_clean":"2023-10-08T07:15:47.600253+0000","last_became_active":"2023-10-08T07:15:45.445552+0000","last_became_peered":"2023-10-08T07:15:45.445552+0000","last_unstale":"2023-10-08T07:15:47.600253+0000","last_undegraded":"2023-10-08T07:15:47.600253+0000","last_fullsized":"2023-10-08T07:15:47.600253+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:49:17.290142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"36.f","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408674+0000","last_change":"2023-10-08T07:15:46.452651+0000","last_active":"2023-10-08T07:15:47.408674+0000","last_peered":"2023-10-08T07:15:47.408674+0000","last_clean":"2023-10-08T07:15:47.408674+0000","last_became_active":"2023-10-08T07:15:46.452319+0000","last_became_peered":"2023-10-08T07:15:46.452319+0000","last_unstale":"2023-10-08T07:15:47.408674+0000","last_undegraded":"2023-10-08T07:15:47.408674+0000","last_fullsized":"2023-10-08T07:15:47.408674+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:38.398925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.8","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.531843+0000","last_change":"2023-10-08T07:15:45.451784+0000","last_active":"2023-10-08T07:15:47.531843+0000","last_peered":"2023-10-08T07:15:47.531843+0000","last_clean":"2023-10-08T07:15:47.531843+0000","last_became_active":"2023-10-08T07:15:45.451474+0000","last_became_peered":"2023-10-08T07:15:45.451474+0000","last_unstale":"2023-10-08T07:15:47.531843+0000","last_undegraded":"2023-10-08T07:15:47.531843+0000","last_fullsized":"2023-10-08T07:15:47.531843+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:26:34.381401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.0","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405629+0000","last_change":"2023-10-08T07:15:46.436370+0000","last_active":"2023-10-08T07:15:47.405629+0000","last_peered":"2023-10-08T07:15:47.405629+0000","last_clean":"2023-10-08T07:15:47.405629+0000","last_became_active":"2023-10-08T07:15:46.436098+0000","last_became_peered":"2023-10-08T07:15:46.436098+0000","last_unstale":"2023-10-08T07:15:47.405629+0000","last_undegraded":"2023-10-08T07:15:47.405629+0000","last_fullsized":"2023-10-08T07:15:47.405629+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:46:11.123773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.7","version":"275'2","reported_seq":14,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532723+0000","last_change":"2023-10-08T07:15:45.457184+0000","last_active":"2023-10-08T07:15:47.532723+0000","last_peered":"2023-10-08T07:15:47.532723+0000","last_clean":"2023-10-08T07:15:47.532723+0000","last_became_active":"2023-10-08T07:15:45.456911+0000","last_became_peered":"2023-10-08T07:15:45.456911+0000","last_unstale":"2023-10-08T07:15:47.532723+0000","last_undegraded":"2023-10-08T07:15:47.532723+0000","last_fullsized":"2023-10-08T07:15:47.532723+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:06.175497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.1","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601057+0000","last_change":"2023-10-08T07:15:46.430723+0000","last_active":"2023-10-08T07:15:47.601057+0000","last_peered":"2023-10-08T07:15:47.601057+0000","last_clean":"2023-10-08T07:15:47.601057+0000","last_became_active":"2023-10-08T07:15:46.430244+0000","last_became_peered":"2023-10-08T07:15:46.430244+0000","last_unstale":"2023-10-08T07:15:47.601057+0000","last_undegraded":"2023-10-08T07:15:47.601057+0000","last_fullsized":"2023-10-08T07:15:47.601057+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:46:54.344502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"35.6","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409555+0000","last_change":"2023-10-08T07:15:45.446031+0000","last_active":"2023-10-08T07:15:47.409555+0000","last_peered":"2023-10-08T07:15:47.409555+0000","last_clean":"2023-10-08T07:15:47.409555+0000","last_became_active":"2023-10-08T07:15:45.445745+0000","last_became_peered":"2023-10-08T07:15:45.445745+0000","last_unstale":"2023-10-08T07:15:47.409555+0000","last_undegraded":"2023-10-08T07:15:47.409555+0000","last_fullsized":"2023-10-08T07:15:47.409555+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:03:03.322265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.2","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.405927+0000","last_change":"2023-10-08T07:15:46.427328+0000","last_active":"2023-10-08T07:15:47.405927+0000","last_peered":"2023-10-08T07:15:47.405927+0000","last_clean":"2023-10-08T07:15:47.405927+0000","last_became_active":"2023-10-08T07:15:46.427054+0000","last_became_peered":"2023-10-08T07:15:46.427054+0000","last_unstale":"2023-10-08T07:15:47.405927+0000","last_undegraded":"2023-10-08T07:15:47.405927+0000","last_fullsized":"2023-10-08T07:15:47.405927+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:48:10.032112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.5","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.599218+0000","last_change":"2023-10-08T07:15:45.452452+0000","last_active":"2023-10-08T07:15:47.599218+0000","last_peered":"2023-10-08T07:15:47.599218+0000","last_clean":"2023-10-08T07:15:47.599218+0000","last_became_active":"2023-10-08T07:15:45.452204+0000","last_became_peered":"2023-10-08T07:15:45.452204+0000","last_unstale":"2023-10-08T07:15:47.599218+0000","last_undegraded":"2023-10-08T07:15:47.599218+0000","last_fullsized":"2023-10-08T07:15:47.599218+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:13:58.232704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"36.3","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600173+0000","last_change":"2023-10-08T07:15:46.443588+0000","last_active":"2023-10-08T07:15:47.600173+0000","last_peered":"2023-10-08T07:15:47.600173+0000","last_clean":"2023-10-08T07:15:47.600173+0000","last_became_active":"2023-10-08T07:15:46.443283+0000","last_became_peered":"2023-10-08T07:15:46.443283+0000","last_unstale":"2023-10-08T07:15:47.600173+0000","last_undegraded":"2023-10-08T07:15:47.600173+0000","last_fullsized":"2023-10-08T07:15:47.600173+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:26.423857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.4","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.409499+0000","last_change":"2023-10-08T07:15:45.453612+0000","last_active":"2023-10-08T07:15:47.409499+0000","last_peered":"2023-10-08T07:15:47.409499+0000","last_clean":"2023-10-08T07:15:47.409499+0000","last_became_active":"2023-10-08T07:15:45.453337+0000","last_became_peered":"2023-10-08T07:15:45.453337+0000","last_unstale":"2023-10-08T07:15:47.409499+0000","last_undegraded":"2023-10-08T07:15:47.409499+0000","last_fullsized":"2023-10-08T07:15:47.409499+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:40.645111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"36.4","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.600542+0000","last_change":"2023-10-08T07:15:46.435249+0000","last_active":"2023-10-08T07:15:47.600542+0000","last_peered":"2023-10-08T07:15:47.600542+0000","last_clean":"2023-10-08T07:15:47.600542+0000","last_became_active":"2023-10-08T07:15:46.434942+0000","last_became_peered":"2023-10-08T07:15:46.434942+0000","last_unstale":"2023-10-08T07:15:47.600542+0000","last_undegraded":"2023-10-08T07:15:47.600542+0000","last_fullsized":"2023-10-08T07:15:47.600542+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:50:08.901765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"35.3","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532581+0000","last_change":"2023-10-08T07:15:45.451334+0000","last_active":"2023-10-08T07:15:47.532581+0000","last_peered":"2023-10-08T07:15:47.532581+0000","last_clean":"2023-10-08T07:15:47.532581+0000","last_became_active":"2023-10-08T07:15:45.451045+0000","last_became_peered":"2023-10-08T07:15:45.451045+0000","last_unstale":"2023-10-08T07:15:47.532581+0000","last_undegraded":"2023-10-08T07:15:47.532581+0000","last_fullsized":"2023-10-08T07:15:47.532581+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:52:17.820140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.5","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406815+0000","last_change":"2023-10-08T07:15:46.429505+0000","last_active":"2023-10-08T07:15:47.406815+0000","last_peered":"2023-10-08T07:15:47.406815+0000","last_clean":"2023-10-08T07:15:47.406815+0000","last_became_active":"2023-10-08T07:15:46.429165+0000","last_became_peered":"2023-10-08T07:15:46.429165+0000","last_unstale":"2023-10-08T07:15:47.406815+0000","last_undegraded":"2023-10-08T07:15:47.406815+0000","last_fullsized":"2023-10-08T07:15:47.406815+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:25:34.973709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.2","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406220+0000","last_change":"2023-10-08T07:15:45.448049+0000","last_active":"2023-10-08T07:15:47.406220+0000","last_peered":"2023-10-08T07:15:47.406220+0000","last_clean":"2023-10-08T07:15:47.406220+0000","last_became_active":"2023-10-08T07:15:45.447780+0000","last_became_peered":"2023-10-08T07:15:45.447780+0000","last_unstale":"2023-10-08T07:15:47.406220+0000","last_undegraded":"2023-10-08T07:15:47.406220+0000","last_fullsized":"2023-10-08T07:15:47.406220+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:41:48.419734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"36.6","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.408760+0000","last_change":"2023-10-08T07:15:46.442427+0000","last_active":"2023-10-08T07:15:47.408760+0000","last_peered":"2023-10-08T07:15:47.408760+0000","last_clean":"2023-10-08T07:15:47.408760+0000","last_became_active":"2023-10-08T07:15:46.441981+0000","last_became_peered":"2023-10-08T07:15:46.441981+0000","last_unstale":"2023-10-08T07:15:47.408760+0000","last_undegraded":"2023-10-08T07:15:47.408760+0000","last_fullsized":"2023-10-08T07:15:47.408760+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:46.539176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"35.1","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.601339+0000","last_change":"2023-10-08T07:15:45.456861+0000","last_active":"2023-10-08T07:15:47.601339+0000","last_peered":"2023-10-08T07:15:47.601339+0000","last_clean":"2023-10-08T07:15:47.601339+0000","last_became_active":"2023-10-08T07:15:45.456587+0000","last_became_peered":"2023-10-08T07:15:45.456587+0000","last_unstale":"2023-10-08T07:15:47.601339+0000","last_undegraded":"2023-10-08T07:15:47.601339+0000","last_fullsized":"2023-10-08T07:15:47.601339+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:49:51.661781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"36.7","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.406426+0000","last_change":"2023-10-08T07:15:46.439221+0000","last_active":"2023-10-08T07:15:47.406426+0000","last_peered":"2023-10-08T07:15:47.406426+0000","last_clean":"2023-10-08T07:15:47.406426+0000","last_became_active":"2023-10-08T07:15:46.438909+0000","last_became_peered":"2023-10-08T07:15:46.438909+0000","last_unstale":"2023-10-08T07:15:47.406426+0000","last_undegraded":"2023-10-08T07:15:47.406426+0000","last_fullsized":"2023-10-08T07:15:47.406426+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:38:17.156874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"35.0","version":"0'0","reported_seq":12,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.531897+0000","last_change":"2023-10-08T07:15:45.451120+0000","last_active":"2023-10-08T07:15:47.531897+0000","last_peered":"2023-10-08T07:15:47.531897+0000","last_clean":"2023-10-08T07:15:47.531897+0000","last_became_active":"2023-10-08T07:15:45.450838+0000","last_became_peered":"2023-10-08T07:15:45.450838+0000","last_unstale":"2023-10-08T07:15:47.531897+0000","last_undegraded":"2023-10-08T07:15:47.531897+0000","last_fullsized":"2023-10-08T07:15:47.531897+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":273,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:44.378531+0000","last_clean_scrub_stamp":"2023-10-08T07:15:44.378531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:04:31.423842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"36.25","version":"0'0","reported_seq":11,"reported_epoch":276,"state":"active+clean","last_fresh":"2023-10-08T07:15:47.532536+0000","last_change":"2023-10-08T07:15:46.443673+0000","last_active":"2023-10-08T07:15:47.532536+0000","last_peered":"2023-10-08T07:15:47.532536+0000","last_clean":"2023-10-08T07:15:47.532536+0000","last_became_active":"2023-10-08T07:15:46.443420+0000","last_became_peered":"2023-10-08T07:15:46.443420+0000","last_unstale":"2023-10-08T07:15:47.532536+0000","last_undegraded":"2023-10-08T07:15:47.532536+0000","last_fullsized":"2023-10-08T07:15:47.532536+0000","mapping_epoch":274,"log_start":"0'0","ondisk_log_start":"0'0","created":274,"last_epoch_clean":275,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T07:15:45.390105+0000","last_clean_scrub_stamp":"2023-10-08T07:15:45.390105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:29:28.171102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"272'4861","reported_seq":5927,"reported_epoch":275,"state":"active+clean","last_fresh":"2023-10-08T07:15:46.425822+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:15:46.425822+0000","last_peered":"2023-10-08T07:15:46.425822+0000","last_clean":"2023-10-08T07:15:46.425822+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:15:46.425822+0000","last_undegraded":"2023-10-08T07:15:46.425822+0000","last_fullsized":"2023-10-08T07:15:46.425822+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":4861,"log_dups_size":0,"ondisk_log_size":4861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":651,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":36,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":35,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2535,"num_read_kb":5339,"num_write":651,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4861,"ondisk_log_size":4861,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576033,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75448,"kb_used_data":4004,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296392,"statfs":{"total":96636764160,"available":96559505408,"internally_reserved":0,"allocated":4100096,"data_stored":2419095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.127,"15min":1.1910000000000001},"min":{"1min":0.81699999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9410000000000001,"5min":1.968,"15min":2.008},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.2010000000000001,"15min":1.214},"min":{"1min":0.81999999999999995,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.516,"5min":1.788,"15min":1.788},"last":1.47}]},{"osd":1,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.2370000000000001,"15min":1.2509999999999999},"min":{"1min":0.93899999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.998},"last":1.038},{"interface":"front","average":{"1min":1.196,"5min":1.2969999999999999,"15min":1.329},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.6299999999999999,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.159}]},{"osd":2,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.2370000000000001,"15min":1.2929999999999999},"min":{"1min":0.89500000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":2.1640000000000001},"last":1.093},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.369,"15min":1.3500000000000001},"min":{"1min":0.93400000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.901,"5min":1.901,"15min":2.3170000000000002},"last":1.288}]},{"osd":3,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.298,"15min":1.3340000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.222},{"interface":"front","average":{"1min":1.401,"5min":1.3819999999999999,"15min":1.3999999999999999},"min":{"1min":0.97099999999999997,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":1.3979999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3300000000000001,"15min":1.3180000000000001},"min":{"1min":0.83999999999999997,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.9319999999999999,"5min":2.1579999999999999,"15min":2.1579999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4139999999999999,"15min":1.411},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.4339999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.6240000000000001,"5min":1.478,"15min":1.4490000000000001},"min":{"1min":1.127,"5min":0.80600000000000005,"15min":0.79300000000000004},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.623},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.488,"15min":1.4950000000000001},"min":{"1min":1.012,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.056,"5min":2.23,"15min":2.23},"last":1.6779999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.599,"15min":1.597},"min":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.202,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.78},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.5640000000000001,"15min":1.5429999999999999},"min":{"1min":1.21,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2570000000000001},"last":1.7050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576032,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48464,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323376,"statfs":{"total":96636764160,"available":96587137024,"internally_reserved":0,"allocated":3473408,"data_stored":1819684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.2430000000000001,"15min":1.292},"min":{"1min":0.68999999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5389999999999999,"5min":1.78,"15min":1.962},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2889999999999999,"15min":1.3460000000000001},"min":{"1min":0.79500000000000004,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6399999999999999,"5min":1.76,"15min":2.0070000000000001},"last":1.7370000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1559999999999999,"15min":1.216},"min":{"1min":0.69399999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.6879999999999999,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.169,"5min":1.2250000000000001,"15min":1.2390000000000001},"min":{"1min":0.77300000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.5640000000000001,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.74399999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1699999999999999,"15min":1.2250000000000001},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.6850000000000001,"5min":1.7909999999999999,"15min":2.339},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.268,"15min":1.304},"min":{"1min":0.84399999999999997,"5min":0.748,"15min":0.748},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":2.1779999999999999},"last":1.298}]},{"osd":3,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.373,"15min":1.395},"min":{"1min":1.0409999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.8160000000000001,"5min":1.9319999999999999,"15min":1.95},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.409,"15min":1.4830000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.1459999999999999},"last":1.8440000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.4079999999999999,"15min":1.401},"min":{"1min":1.145,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0859999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.502,"5min":1.4750000000000001,"15min":1.46},"min":{"1min":1.01,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":1.99,"5min":2.1110000000000002,"15min":2.484},"last":1.9379999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.3939999999999999,"15min":1.4259999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.403},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.629,"5min":1.569,"15min":1.5609999999999999},"min":{"1min":1.121,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.129,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":1.879}]},{"osd":7,"last update":"Sun Oct 8 07:15:09 2023","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.5660000000000001,"15min":1.5840000000000001},"min":{"1min":1.302,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.194,"5min":2.339,"15min":2.5870000000000002},"last":2.0049999999999999},{"interface":"front","average":{"1min":1.5860000000000001,"5min":1.5609999999999999,"15min":1.5620000000000001},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.1179999999999999,"5min":2.2959999999999998,"15min":2.665},"last":1.798}]}]},{"osd":5,"up_from":13,"seq":55834576032,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60952,"kb_used_data":3400,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310888,"statfs":{"total":96636764160,"available":96574349312,"internally_reserved":0,"allocated":3481600,"data_stored":1819740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1499999999999999,"15min":1.202},"min":{"1min":0.748,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5740000000000001,"5min":1.98,"15min":1.98},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1910000000000001,"15min":1.2190000000000001},"min":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.74,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.0049999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.165,"15min":1.218},"min":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.5449999999999999,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2849999999999999,"15min":1.3100000000000001},"min":{"1min":0.78800000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.752,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":1.6559999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.254,"15min":1.2969999999999999},"min":{"1min":0.77100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.7050000000000001,"5min":2.008,"15min":2.0870000000000002},"last":1.208},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.29},"min":{"1min":0.77600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.677,"5min":1.877,"15min":1.905},"last":1.0509999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.97699999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.694,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.355,"5min":1.4219999999999999,"15min":1.4259999999999999},"min":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.883,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":1.359}]},{"osd":4,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.329,"15min":1.3300000000000001},"min":{"1min":0.93600000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.857,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.8700000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.4910000000000001,"15min":1.4399999999999999},"min":{"1min":1.194,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1600000000000001,"5min":2.742,"15min":2.742},"last":1.508}]},{"osd":6,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5780000000000001,"15min":1.4890000000000001},"min":{"1min":1.036,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9450000000000001,"5min":13.234999999999999,"15min":13.234999999999999},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.585,"5min":1.5449999999999999,"15min":1.4870000000000001},"min":{"1min":1.1830000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9770000000000001,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":2.0070000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.5649999999999999,"15min":1.548},"min":{"1min":1.208,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.0979999999999999,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.669,"5min":1.609,"15min":1.6040000000000001},"min":{"1min":1.3260000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.02,"5min":2.351,"15min":2.351},"last":1.548}]}]},{"osd":4,"up_from":13,"seq":55834576032,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48480,"kb_used_data":3408,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323360,"statfs":{"total":96636764160,"available":96587120640,"internally_reserved":0,"allocated":3489792,"data_stored":1821386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.67800000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.3220000000000001,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4359999999999999,"15min":1.423},"min":{"1min":0.63600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.3609999999999998,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.498}]},{"osd":1,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2470000000000001,"15min":1.2849999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.2229999999999999,"5min":4.2229999999999999,"15min":4.2229999999999999},"last":1.266},{"interface":"front","average":{"1min":2.7799999999999998,"5min":1.635,"15min":1.371},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":28.907,"5min":28.907,"15min":28.907},"last":1.4470000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4810000000000001,"15min":1.335},"min":{"1min":0.70899999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":3.6120000000000001,"5min":17.175999999999998,"15min":17.175999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.496,"5min":1.444,"15min":1.391},"min":{"1min":0.59499999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.4080000000000004,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":1.619}]},{"osd":3,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.3260000000000001,"5min":1.5720000000000001,"15min":1.4159999999999999},"min":{"1min":0.60099999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":15.938000000000001,"5min":15.938000000000001,"15min":15.938000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.8069999999999999,"5min":1.5760000000000001,"15min":1.474},"min":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.787000000000001,"5min":10.787000000000001,"15min":10.787000000000001},"last":1.4710000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.591,"15min":1.4350000000000001},"min":{"1min":0.73199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":4.2939999999999996,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":3.0209999999999999,"5min":1.8160000000000001,"15min":1.5289999999999999},"min":{"1min":0.84899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":31.347999999999999,"5min":31.347999999999999,"15min":31.347999999999999},"last":1.552}]},{"osd":6,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.016,"5min":1.7290000000000001,"15min":1.5389999999999999},"min":{"1min":0.73699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":13.885,"5min":13.885,"15min":13.885},"last":1.4039999999999999},{"interface":"front","average":{"1min":2.117,"5min":1.833,"15min":1.6419999999999999},"min":{"1min":0.82499999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":13.446999999999999,"5min":13.446999999999999,"15min":13.446999999999999},"last":1.369}]},{"osd":7,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.528,"5min":1.821,"15min":1.611},"min":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":15.843999999999999,"5min":15.843999999999999,"15min":15.843999999999999},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.726,"15min":1.6659999999999999},"min":{"1min":0.82399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.968,"5min":5.609,"15min":5.609},"last":1.306}]}]},{"osd":3,"up_from":13,"seq":55834576034,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56808,"kb_used_data":3412,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315032,"statfs":{"total":96636764160,"available":96578592768,"internally_reserved":0,"allocated":3493888,"data_stored":1826555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.92700000000000005,"15min":0.97399999999999998},"min":{"1min":0.32200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1990000000000001,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.131,"15min":1.1659999999999999},"min":{"1min":0.46899999999999997,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5349999999999999,"5min":4.5659999999999998,"15min":4.5659999999999998},"last":0.77400000000000002}]},{"osd":1,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.96799999999999997,"15min":1.022},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.3560000000000001,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.1890000000000001,"15min":1.2729999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.667,"5min":2.109,"15min":2.4889999999999999},"last":1.177}]},{"osd":2,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0629999999999999,"15min":1.1399999999999999},"min":{"1min":0.502,"5min":0.437,"15min":0.437},"max":{"1min":1.462,"5min":2.165,"15min":2.2250000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.0329999999999999,"5min":1.139,"15min":1.1899999999999999},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.671,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.29}]},{"osd":4,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2290000000000001,"15min":1.272},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.67500000000000004},"max":{"1min":1.7290000000000001,"5min":1.857,"15min":2.5379999999999998},"last":1.234},{"interface":"front","average":{"1min":1.306,"5min":1.381,"15min":1.45},"min":{"1min":0.91600000000000004,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":1.8149999999999999,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.2609999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.294,"15min":1.3660000000000001},"min":{"1min":0.76700000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7829999999999999,"5min":1.923,"15min":2.9740000000000002},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3919999999999999,"15min":1.4330000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.9610000000000001,"5min":2.2309999999999999,"15min":2.7330000000000001},"last":1.5089999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3919999999999999,"15min":1.415},"min":{"1min":0.82299999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.9690000000000001,"5min":1.976,"15min":2.653},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.4350000000000001,"15min":1.5129999999999999},"min":{"1min":0.872,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.0150000000000001,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.4810000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:14:46 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.512,"15min":1.5580000000000001},"min":{"1min":0.89100000000000001,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.3220000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4490000000000001,"15min":1.512},"min":{"1min":0.80900000000000005,"5min":0.628,"15min":0.628},"max":{"1min":1.903,"5min":1.9410000000000001,"15min":2.7749999999999999},"last":1.4219999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576032,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52624,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319216,"statfs":{"total":96636764160,"available":96582877184,"internally_reserved":0,"allocated":3473408,"data_stored":1819684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.86099999999999999,"15min":0.95299999999999996},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2270000000000001,"5min":1.601,"15min":1.948},"last":0.625},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.98699999999999999,"15min":1.0529999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.98799999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.90100000000000002,"15min":0.97399999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0609999999999999,"15min":1.167},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.587,"5min":1.7789999999999999,"15min":2.1309999999999998},"last":0.92900000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0860000000000001,"15min":1.1639999999999999},"min":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.9359999999999999,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.155,"15min":1.2490000000000001},"min":{"1min":0.47699999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6699999999999999,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.25}]},{"osd":4,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.3500000000000001,"15min":1.421},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8640000000000001,"5min":1.992,"15min":2.1789999999999998},"last":1.109},{"interface":"front","average":{"1min":1.161,"5min":1.2789999999999999,"15min":1.306},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.3620000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.3160000000000001,"15min":1.385},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.4279999999999999,"15min":1.51},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0750000000000002,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.3959999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3260000000000001,"15min":1.385},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.4339999999999999,"15min":1.51},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.0860000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.4059999999999999,"15min":1.464},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.4689999999999999},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.466,"15min":1.5580000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.831,"5min":2.831,"15min":2.831},"last":1.1499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834576032,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52644,"kb_used_data":3408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319196,"statfs":{"total":96636764160,"available":96582856704,"internally_reserved":0,"allocated":3489792,"data_stored":1826553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.97699999999999998,"15min":0.999},"min":{"1min":0.56000000000000005,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.502,"5min":1.6579999999999999,"15min":1.708},"last":0.875},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.0760000000000001,"15min":1.2},"min":{"1min":0.52900000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.4790000000000001,"5min":1.7010000000000001,"15min":1.79},"last":0.78600000000000003}]},{"osd":2,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.876,"15min":0.94999999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.095,"5min":1.641,"15min":1.641},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.026,"5min":1.087,"15min":1.1419999999999999},"min":{"1min":0.42599999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.607,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.0960000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.1519999999999999,"15min":1.2390000000000001},"min":{"1min":0.58599999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5960000000000001,"5min":2.496,"15min":2.496},"last":1.2549999999999999},{"interface":"front","average":{"1min":0.91900000000000004,"5min":1.099,"15min":1.143},"min":{"1min":0.49399999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.5369999999999999,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":1.0629999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.276,"15min":1.351},"min":{"1min":0.77700000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.579,"5min":2.73,"15min":2.8460000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.248,"15min":1.3009999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.5409999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.236}]},{"osd":5,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.302,"15min":1.3640000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.296},{"interface":"front","average":{"1min":1.337,"5min":1.476,"15min":1.5189999999999999},"min":{"1min":0.89600000000000002,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7070000000000001,"5min":2.601,"15min":2.601},"last":1.6040000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.4330000000000001,"15min":1.502},"min":{"1min":0.92600000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.7609999999999999,"5min":2.883,"15min":2.9079999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.411,"15min":1.484},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.6180000000000001,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.159}]},{"osd":7,"last update":"Sun Oct 8 07:14:58 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4830000000000001,"15min":1.5389999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.7749999999999999,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.512},{"interface":"front","average":{"1min":1.339,"5min":1.484,"15min":1.5569999999999999},"min":{"1min":0.83799999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"max":{"1min":1.873,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.573}]}]},{"osd":0,"up_from":13,"seq":55834576033,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53220,"kb_used_data":3992,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318620,"statfs":{"total":96636764160,"available":96582266880,"internally_reserved":0,"allocated":4087808,"data_stored":2417447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.90300000000000002,"15min":0.95999999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2310000000000001,"5min":1.581,"15min":1.581},"last":1.22},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.97499999999999998,"15min":1.069},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.5640000000000001,"5min":1.621,"15min":1.726},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.0429999999999999,"15min":1.1240000000000001},"min":{"1min":0.46200000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3360000000000001,"5min":1.782,"15min":1.782},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.97599999999999998,"15min":1.0589999999999999},"min":{"1min":0.52700000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4239999999999999,"5min":1.6000000000000001,"15min":1.9930000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.1399999999999999,"15min":1.2270000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.5920000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.048},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.083,"15min":1.149},"min":{"1min":0.44,"5min":0.439,"15min":0.439},"max":{"1min":1.2929999999999999,"5min":1.7689999999999999,"15min":2.0979999999999999},"last":1.0740000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2789999999999999,"15min":1.327},"min":{"1min":0.59899999999999998,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6599999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.3,"5min":1.3300000000000001,"15min":1.3999999999999999},"min":{"1min":0.69599999999999995,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.782,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.4079999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.369,"15min":1.409},"min":{"1min":0.67300000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.6459999999999999,"5min":1.903,"15min":2.1309999999999998},"last":1.373},{"interface":"front","average":{"1min":1.341,"5min":1.454,"15min":1.4810000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9430000000000001,"5min":2.23,"15min":2.4900000000000002},"last":1.1559999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.4099999999999999,"15min":1.4650000000000001},"min":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.877,"5min":2.21,"15min":2.2149999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4179999999999999,"15min":1.4850000000000001},"min":{"1min":0.71199999999999997,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7190000000000001,"5min":2.0880000000000001,"15min":2.246},"last":1.2490000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.4790000000000001,"15min":1.5289999999999999},"min":{"1min":0.73299999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8380000000000001,"5min":2.1989999999999998,"15min":2.2349999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.4850000000000001,"15min":1.54},"min":{"1min":0.78500000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9690000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.2929999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":35,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":36,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:15:51.569 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T07:15:51.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T07:15:51.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.799+0000 7f92c73c9640 1 Processor -- start 2023-10-08T07:15:51.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.799+0000 7f92c73c9640 1 -- start start 2023-10-08T07:15:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.800+0000 7f92c73c9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f92c00d07f0 0x7f92c00d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.800+0000 7f92c73c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92c00b17c0 0x7f92c00d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.800+0000 7f92c73c9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c00d12f0 con 0x7f92c00d2bb0 2023-10-08T07:15:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.800+0000 7f92c73c9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c00b5cb0 con 0x7f92c00b17c0 2023-10-08T07:15:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.800+0000 7f92c73c9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c00b8a60 con 0x7f92c00d07f0 2023-10-08T07:15:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.800+0000 7f92c5966640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92c00b17c0 0x7f92c00d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:51.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.800+0000 7f92c5165640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f92c00d2bb0 0x7f92c00d2f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52466/0 (socket says 172.21.15.70:52466) 2023-10-08T07:15:51.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.800+0000 7f92c5966640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92c00b17c0 0x7f92c00d1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51482/0 (socket says 172.21.15.70:51482) 2023-10-08T07:15:51.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.800+0000 7f92c5165640 1 -- 172.21.15.70:0/596007400 learned_addr learned my addr 172.21.15.70:0/596007400 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.801+0000 7f92c4964640 1 --2- 172.21.15.70:0/596007400 >> v2:172.21.15.181:3300/0 conn(0x7f92c00d07f0 0x7f92c00d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.801+0000 7f92c5966640 1 -- 172.21.15.70:0/596007400 >> v2:172.21.15.181:3300/0 conn(0x7f92c00d07f0 msgr2=0x7f92c00d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.801+0000 7f92c5966640 1 --2- 172.21.15.70:0/596007400 >> v2:172.21.15.181:3300/0 conn(0x7f92c00d07f0 0x7f92c00d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.801+0000 7f92c5966640 1 -- 172.21.15.70:0/596007400 >> v1:172.21.15.70:6789/0 conn(0x7f92c00d2bb0 legacy=0x7f92c00d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.801+0000 7f92b7fff640 1 -- 172.21.15.70:0/596007400 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174206848 0 0) 0x7f92c00d12f0 con 0x7f92c00d2bb0 2023-10-08T07:15:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.801+0000 7f92c5966640 1 -- 172.21.15.70:0/596007400 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c00b84a0 con 0x7f92c00b17c0 2023-10-08T07:15:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.801+0000 7f92c4964640 1 --2- 172.21.15.70:0/596007400 >> v2:172.21.15.181:3300/0 conn(0x7f92c00d07f0 0x7f92c00d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.802+0000 7f92c5966640 1 --2- 172.21.15.70:0/596007400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00b17c0 0x7f92c00d1cc0 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f92c00d3ce0 tx=0x7f92b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2cbdaa13fdd4d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:51.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.802+0000 7f92b7fff640 1 -- 172.21.15.70:0/596007400 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8031830 con 0x7f92c00b17c0 2023-10-08T07:15:51.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.802+0000 7f92b7fff640 1 -- 172.21.15.70:0/596007400 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b8031990 con 0x7f92c00b17c0 2023-10-08T07:15:51.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.802+0000 7f92b7fff640 1 -- 172.21.15.70:0/596007400 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8031d20 con 0x7f92c00b17c0 2023-10-08T07:15:51.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.803+0000 7f92c73c9640 1 -- 172.21.15.70:0/596007400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00b17c0 msgr2=0x7f92c00d1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.803+0000 7f92c73c9640 1 --2- 172.21.15.70:0/596007400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00b17c0 0x7f92c00d1cc0 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7f92c00d3ce0 tx=0x7f92b802b8d0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.803+0000 7f92c73c9640 1 -- 172.21.15.70:0/596007400 shutdown_connections 2023-10-08T07:15:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.803+0000 7f92c73c9640 1 --2- 172.21.15.70:0/596007400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00b17c0 0x7f92c00d1cc0 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.803+0000 7f92c73c9640 1 --2- 172.21.15.70:0/596007400 >> v2:172.21.15.181:3300/0 conn(0x7f92c00d07f0 0x7f92c00d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.803+0000 7f92c73c9640 1 -- 172.21.15.70:0/596007400 >> 172.21.15.70:0/596007400 conn(0x7f92c00718a0 msgr2=0x7f92c0071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.803+0000 7f92c73c9640 1 -- 172.21.15.70:0/596007400 shutdown_connections 2023-10-08T07:15:51.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.803+0000 7f92c73c9640 1 -- 172.21.15.70:0/596007400 wait complete. 2023-10-08T07:15:51.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.805+0000 7f92c73c9640 1 Processor -- start 2023-10-08T07:15:51.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.805+0000 7f92c73c9640 1 -- start start 2023-10-08T07:15:51.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c73c9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92c00b17c0 0x7f92c01e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c73c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00d07f0 0x7f92c0156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c73c9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c00d2bb0 0x7f92c0157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c73c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c01e8e00 con 0x7f92c00d07f0 2023-10-08T07:15:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c73c9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00b3710 con 0x7f92c00d2bb0 2023-10-08T07:15:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c73c9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00b39f0 con 0x7f92c00b17c0 2023-10-08T07:15:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c4964640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00d07f0 0x7f92c0156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c4964640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00d07f0 0x7f92c0156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51490/0 (socket says 172.21.15.70:51490) 2023-10-08T07:15:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c4964640 1 -- 172.21.15.70:0/1995333444 learned_addr learned my addr 172.21.15.70:0/1995333444 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:51.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c4964640 1 -- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92c00b17c0 msgr2=0x7f92c01e88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:51.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c4964640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92c00b17c0 0x7f92c01e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c4964640 1 -- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c00d2bb0 msgr2=0x7f92c0157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:51.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.806+0000 7f92c4964640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c00d2bb0 0x7f92c0157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.807+0000 7f92c4964640 1 -- 172.21.15.70:0/1995333444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8009400 con 0x7f92c00d07f0 2023-10-08T07:15:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.807+0000 7f92c4964640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00d07f0 0x7f92c0156c10 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c7f0 tx=0x7f92b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.807+0000 7f92b5ffb640 1 -- 172.21.15.70:0/1995333444 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0007a30 con 0x7f92c00d07f0 2023-10-08T07:15:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.807+0000 7f92b5ffb640 1 -- 172.21.15.70:0/1995333444 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b0007b90 con 0x7f92c00d07f0 2023-10-08T07:15:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.807+0000 7f92b5ffb640 1 -- 172.21.15.70:0/1995333444 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b000ce40 con 0x7f92c00d07f0 2023-10-08T07:15:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.807+0000 7f92c73c9640 1 -- 172.21.15.70:0/1995333444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c00b3cd0 con 0x7f92c00d07f0 2023-10-08T07:15:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.807+0000 7f92c73c9640 1 -- 172.21.15.70:0/1995333444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c01f2cd0 con 0x7f92c00d07f0 2023-10-08T07:15:51.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.808+0000 7f92c73c9640 1 -- 172.21.15.70:0/1995333444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c01f2f20 con 0x7f92c00d07f0 2023-10-08T07:15:51.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.809+0000 7f92b5ffb640 1 -- 172.21.15.70:0/1995333444 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f92b00113f0 con 0x7f92c00d07f0 2023-10-08T07:15:51.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.811+0000 7f92b5ffb640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f92a406b0b0 0x7f92a406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:51.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.811+0000 7f92b5ffb640 1 -- 172.21.15.70:0/1995333444 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f92b0089df0 con 0x7f92c00d07f0 2023-10-08T07:15:51.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.811+0000 7f92c5165640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f92a406b0b0 0x7f92a406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:51.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.813+0000 7f92c5165640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f92a406b0b0 0x7f92a406d570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92bc001560 tx=0x7f92bc007ca0 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:51.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:51.815+0000 7f92b5ffb640 1 -- 172.21.15.70:0/1995333444 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92b00548e0 con 0x7f92c00d07f0 2023-10-08T07:15:52.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.074+0000 7f92c73c9640 1 -- 172.21.15.70:0/1995333444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f92c00b3f10 con 0x7f92c00d07f0 2023-10-08T07:15:52.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.078+0000 7f92b5ffb640 1 -- 172.21.15.70:0/1995333444 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v55) v1 ==== 190+0+147 (secure 0 0 0) 0x7f92b001f0e0 con 0x7f92c00d07f0 2023-10-08T07:15:52.080 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-10-08T07:15:52.080 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNPyJlPhhBJRAAdNgqwEFECtM+CofAYCnjxA== 2023-10-08T07:15:52.080 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-10-08T07:15:52.080 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-10-08T07:15:52.080 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-10-08T07:15:52.081 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-10-08T07:15:52.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.081+0000 7f92c73c9640 1 -- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f92a406b0b0 msgr2=0x7f92a406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:52.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.081+0000 7f92c73c9640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f92a406b0b0 0x7f92a406d570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92bc001560 tx=0x7f92bc007ca0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.081+0000 7f92c73c9640 1 -- 172.21.15.70:0/1995333444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00d07f0 msgr2=0x7f92c0156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.081+0000 7f92c73c9640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00d07f0 0x7f92c0156c10 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c7f0 tx=0x7f92b000cbf0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.082+0000 7f92c73c9640 1 -- 172.21.15.70:0/1995333444 shutdown_connections 2023-10-08T07:15:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.082+0000 7f92c73c9640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f92a406b0b0 0x7f92a406d570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.082+0000 7f92c73c9640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f92c00d2bb0 0x7f92c0157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.082+0000 7f92c73c9640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92c00d07f0 0x7f92c0156c10 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.082+0000 7f92c73c9640 1 --2- 172.21.15.70:0/1995333444 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f92c00b17c0 0x7f92c01e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.082+0000 7f92c73c9640 1 -- 172.21.15.70:0/1995333444 >> 172.21.15.70:0/1995333444 conn(0x7f92c00718a0 msgr2=0x7f92c00cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:52.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.082+0000 7f92c73c9640 1 -- 172.21.15.70:0/1995333444 shutdown_connections 2023-10-08T07:15:52.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.082+0000 7f92c73c9640 1 -- 172.21.15.70:0/1995333444 wait complete. 2023-10-08T07:15:52.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:15:52.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.314+0000 7f661dfcc640 1 Processor -- start 2023-10-08T07:15:52.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.314+0000 7f661dfcc640 1 -- start start 2023-10-08T07:15:52.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.316+0000 7f661dfcc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f661815ab40 0x7f66180ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:52.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.316+0000 7f661dfcc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661815b2b0 con 0x7f66180cb580 2023-10-08T07:15:52.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.316+0000 7f661dfcc640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661815b590 con 0x7f66180c90b0 2023-10-08T07:15:52.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.316+0000 7f661dfcc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661815bf30 con 0x7f661815ab40 2023-10-08T07:15:52.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.316+0000 7f66177fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f661815ab40 0x7f66180ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:52.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.316+0000 7f66177fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f661815ab40 0x7f66180ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51494/0 (socket says 172.21.15.70:51494) 2023-10-08T07:15:52.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.316+0000 7f66177fe640 1 -- 172.21.15.70:0/648425269 learned_addr learned my addr 172.21.15.70:0/648425269 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:52.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.317+0000 7f66177fe640 1 -- 172.21.15.70:0/648425269 >> v1:172.21.15.181:6789/0 conn(0x7f66180c90b0 legacy=0x7f66180cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:52.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.317+0000 7f66177fe640 1 -- 172.21.15.70:0/648425269 >> v1:172.21.15.70:6789/0 conn(0x7f66180cb580 legacy=0x7f661815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:52.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.317+0000 7f66177fe640 1 -- 172.21.15.70:0/648425269 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661815b870 con 0x7f661815ab40 2023-10-08T07:15:52.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.317+0000 7f6616ffd640 1 -- 172.21.15.70:0/648425269 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080736772 0 0) 0x7f661815b2b0 con 0x7f66180cb580 2023-10-08T07:15:52.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.318+0000 7f66177fe640 1 --2- 172.21.15.70:0/648425269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f661815ab40 0x7f66180ad720 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f6608009870 tx=0x7f660802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e63c952d9121a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:52.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.318+0000 7f6616ffd640 1 -- 172.21.15.70:0/648425269 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608030850 con 0x7f661815ab40 2023-10-08T07:15:52.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.318+0000 7f6616ffd640 1 -- 172.21.15.70:0/648425269 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66080309b0 con 0x7f661815ab40 2023-10-08T07:15:52.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.318+0000 7f6616ffd640 1 -- 172.21.15.70:0/648425269 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608030d40 con 0x7f661815ab40 2023-10-08T07:15:52.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.319+0000 7f661dfcc640 1 -- 172.21.15.70:0/648425269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f661815ab40 msgr2=0x7f66180ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:52.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.319+0000 7f661dfcc640 1 --2- 172.21.15.70:0/648425269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f661815ab40 0x7f66180ad720 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f6608009870 tx=0x7f660802b990 comp rx=0 tx=0).stop 2023-10-08T07:15:52.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.319+0000 7f661dfcc640 1 -- 172.21.15.70:0/648425269 shutdown_connections 2023-10-08T07:15:52.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.319+0000 7f661dfcc640 1 --2- 172.21.15.70:0/648425269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f661815ab40 0x7f66180ad720 unknown :-1 s=CLOSED pgs=2903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.319+0000 7f661dfcc640 1 -- 172.21.15.70:0/648425269 >> 172.21.15.70:0/648425269 conn(0x7f661806a380 msgr2=0x7f66180b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:52.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.319+0000 7f661dfcc640 1 -- 172.21.15.70:0/648425269 shutdown_connections 2023-10-08T07:15:52.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.319+0000 7f661dfcc640 1 -- 172.21.15.70:0/648425269 wait complete. 2023-10-08T07:15:52.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.321+0000 7f661dfcc640 1 Processor -- start 2023-10-08T07:15:52.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.321+0000 7f661dfcc640 1 -- start start 2023-10-08T07:15:52.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f661dfcc640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66180c90b0 0x7f661815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:52.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f661dfcc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66180cb580 0x7f6618158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:52.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f661dfcc640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661815ab40 0x7f66180c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:52.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f661dfcc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661814ed80 con 0x7f66180cb580 2023-10-08T07:15:52.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f661dfcc640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661815d3e0 con 0x7f661815ab40 2023-10-08T07:15:52.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f661dfcc640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66180c6360 con 0x7f66180c90b0 2023-10-08T07:15:52.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f660ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66180cb580 0x7f6618158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:52.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f660ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66180cb580 0x7f6618158560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51508/0 (socket says 172.21.15.70:51508) 2023-10-08T07:15:52.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f660ffff640 1 -- 172.21.15.70:0/3273021669 learned_addr learned my addr 172.21.15.70:0/3273021669 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:52.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f66177fe640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66180c90b0 0x7f661815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:52.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f660ffff640 1 -- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66180c90b0 msgr2=0x7f661815cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:52.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f660ffff640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66180c90b0 0x7f661815cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f660ffff640 1 -- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661815ab40 msgr2=0x7f66180c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:52.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.322+0000 7f6617fff640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661815ab40 0x7f66180c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:52.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.323+0000 7f660ffff640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661815ab40 0x7f66180c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.323+0000 7f660ffff640 1 -- 172.21.15.70:0/3273021669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66080094d0 con 0x7f66180cb580 2023-10-08T07:15:52.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.323+0000 7f660ffff640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66180cb580 0x7f6618158560 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f660000c430 tx=0x7f660000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:52.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.323+0000 7f6614ff9640 1 -- 172.21.15.70:0/3273021669 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600011790 con 0x7f66180cb580 2023-10-08T07:15:52.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.323+0000 7f6614ff9640 1 -- 172.21.15.70:0/3273021669 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66000118f0 con 0x7f66180cb580 2023-10-08T07:15:52.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.323+0000 7f6614ff9640 1 -- 172.21.15.70:0/3273021669 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600011ca0 con 0x7f66180cb580 2023-10-08T07:15:52.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.323+0000 7f661dfcc640 1 -- 172.21.15.70:0/3273021669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6618158aa0 con 0x7f66180cb580 2023-10-08T07:15:52.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.323+0000 7f661dfcc640 1 -- 172.21.15.70:0/3273021669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66180c8710 con 0x7f66180cb580 2023-10-08T07:15:52.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.324+0000 7f661dfcc640 1 -- 172.21.15.70:0/3273021669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66180c64f0 con 0x7f66180cb580 2023-10-08T07:15:52.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.325+0000 7f6614ff9640 1 -- 172.21.15.70:0/3273021669 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f660001c460 con 0x7f66180cb580 2023-10-08T07:15:52.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.326+0000 7f6614ff9640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f65f806b0b0 0x7f65f806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:52.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.326+0000 7f6614ff9640 1 -- 172.21.15.70:0/3273021669 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f6600089a70 con 0x7f66180cb580 2023-10-08T07:15:52.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.327+0000 7f66177fe640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f65f806b0b0 0x7f65f806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:52.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.328+0000 7f66177fe640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f65f806b0b0 0x7f65f806d570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6608036000 tx=0x7f660803b040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:52.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.331+0000 7f6614ff9640 1 -- 172.21.15.70:0/3273021669 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66000546a0 con 0x7f66180cb580 2023-10-08T07:15:52.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.559+0000 7f661dfcc640 1 -- 172.21.15.70:0/3273021669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6618067490 con 0x7f66180cb580 2023-10-08T07:15:52.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.560+0000 7f6614ff9640 1 -- 172.21.15.70:0/3273021669 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4923 (secure 0 0 0) 0x7f6600057cc0 con 0x7f66180cb580 2023-10-08T07:15:52.562 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:15:52.562 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":140,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/949448327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":949448327},{"type":"v1","addr":"172.21.15.70:6835","nonce":949448327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":8111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2423877406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2423877406},{"type":"v1","addr":"172.21.15.181:6837","nonce":2423877406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/247109890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":247109890},{"type":"v1","addr":"172.21.15.181:6836","nonce":247109890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:15:45.890217+0000","modified":"2023-10-08T07:15:48.920991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12645},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12645":{"gid":12645,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/753129368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":753129368},{"type":"v1","addr":"172.21.15.70:6834","nonce":753129368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-08T07:15:52.562 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 140 2023-10-08T07:15:52.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.562+0000 7f661dfcc640 1 -- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f65f806b0b0 msgr2=0x7f65f806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:52.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.562+0000 7f661dfcc640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f65f806b0b0 0x7f65f806d570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6608036000 tx=0x7f660803b040 comp rx=0 tx=0).stop 2023-10-08T07:15:52.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.562+0000 7f661dfcc640 1 -- 172.21.15.70:0/3273021669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66180cb580 msgr2=0x7f6618158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.562+0000 7f661dfcc640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66180cb580 0x7f6618158560 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f660000c430 tx=0x7f660000c8f0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.562+0000 7f661dfcc640 1 -- 172.21.15.70:0/3273021669 shutdown_connections 2023-10-08T07:15:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.562+0000 7f661dfcc640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f65f806b0b0 0x7f65f806d570 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.562+0000 7f661dfcc640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f661815ab40 0x7f66180c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.562+0000 7f661dfcc640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66180cb580 0x7f6618158560 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.562+0000 7f661dfcc640 1 --2- 172.21.15.70:0/3273021669 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f66180c90b0 0x7f661815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.562+0000 7f661dfcc640 1 -- 172.21.15.70:0/3273021669 >> 172.21.15.70:0/3273021669 conn(0x7f661806a380 msgr2=0x7f66180ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.563+0000 7f661dfcc640 1 -- 172.21.15.70:0/3273021669 shutdown_connections 2023-10-08T07:15:52.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:52.563+0000 7f661dfcc640 1 -- 172.21.15.70:0/3273021669 wait complete. 2023-10-08T07:15:52.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 140, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T07:15:45.890217+0000', 'modified': '2023-10-08T07:15:48.920991+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12645}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12645': {'gid': 12645, 'name': 'c', 'rank': 0, 'incarnation': 136, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6834/753129368', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 753129368}, {'type': 'v1', 'addr': '172.21.15.70:6834', 'nonce': 753129368}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [36], 'metadata_pool': 35, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-08T07:15:52.985 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T07:15:52.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T07:15:53.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.212+0000 7f9939026640 1 Processor -- start 2023-10-08T07:15:53.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.212+0000 7f9939026640 1 -- start start 2023-10-08T07:15:53.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.214+0000 7f9939026640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9934161690 0x7f993406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.214+0000 7f9939026640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f99340b2f30 0x7f99340cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.214+0000 7f9939026640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993406cc50 con 0x7f99340cb150 2023-10-08T07:15:53.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.214+0000 7f9939026640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99340cbb10 con 0x7f99340b2f30 2023-10-08T07:15:53.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.214+0000 7f9939026640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993406cf30 con 0x7f9934161690 2023-10-08T07:15:53.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.214+0000 7f993359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f99340b2f30 0x7f99340cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.214+0000 7f993359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f99340b2f30 0x7f99340cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51520/0 (socket says 172.21.15.70:51520) 2023-10-08T07:15:53.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.214+0000 7f993359e640 1 -- 172.21.15.70:0/2310764452 learned_addr learned my addr 172.21.15.70:0/2310764452 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:53.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.215+0000 7f993359e640 1 -- 172.21.15.70:0/2310764452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9934161690 msgr2=0x7f993406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:53.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.215+0000 7f9932d9d640 1 --2- 172.21.15.70:0/2310764452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9934161690 0x7f993406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.215+0000 7f993359e640 1 --2- 172.21.15.70:0/2310764452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9934161690 0x7f993406dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.215+0000 7f993359e640 1 -- 172.21.15.70:0/2310764452 >> v1:172.21.15.70:6789/0 conn(0x7f99340cb150 legacy=0x7f99340702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.215+0000 7f993359e640 1 -- 172.21.15.70:0/2310764452 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993406d210 con 0x7f99340b2f30 2023-10-08T07:15:53.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.215+0000 7f9932d9d640 1 --2- 172.21.15.70:0/2310764452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9934161690 0x7f993406dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:53.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.216+0000 7f993359e640 1 --2- 172.21.15.70:0/2310764452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340b2f30 0x7f99340cb5d0 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f992c009870 tx=0x7f992c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9fc8b909e91cad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:53.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.216+0000 7f9931d9b640 1 -- 172.21.15.70:0/2310764452 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c030840 con 0x7f99340b2f30 2023-10-08T07:15:53.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.216+0000 7f9931d9b640 1 -- 172.21.15.70:0/2310764452 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c0309a0 con 0x7f99340b2f30 2023-10-08T07:15:53.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.216+0000 7f9931d9b640 1 -- 172.21.15.70:0/2310764452 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c030d30 con 0x7f99340b2f30 2023-10-08T07:15:53.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.217+0000 7f9939026640 1 -- 172.21.15.70:0/2310764452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340b2f30 msgr2=0x7f99340cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.217+0000 7f9939026640 1 --2- 172.21.15.70:0/2310764452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340b2f30 0x7f99340cb5d0 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7f992c009870 tx=0x7f992c02b980 comp rx=0 tx=0).stop 2023-10-08T07:15:53.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.217+0000 7f9939026640 1 -- 172.21.15.70:0/2310764452 shutdown_connections 2023-10-08T07:15:53.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.217+0000 7f9939026640 1 --2- 172.21.15.70:0/2310764452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99340b2f30 0x7f99340cb5d0 unknown :-1 s=CLOSED pgs=2906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.217+0000 7f9939026640 1 --2- 172.21.15.70:0/2310764452 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f9934161690 0x7f993406dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.217+0000 7f9939026640 1 -- 172.21.15.70:0/2310764452 >> 172.21.15.70:0/2310764452 conn(0x7f993406a5d0 msgr2=0x7f99340b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.217+0000 7f9939026640 1 -- 172.21.15.70:0/2310764452 shutdown_connections 2023-10-08T07:15:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.217+0000 7f9939026640 1 -- 172.21.15.70:0/2310764452 wait complete. 2023-10-08T07:15:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.219+0000 7f9939026640 1 Processor -- start 2023-10-08T07:15:53.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.219+0000 7f9939026640 1 -- start start 2023-10-08T07:15:53.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.220+0000 7f9939026640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99340b2f30 0x7f9934161a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.220+0000 7f9939026640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99340cb150 0x7f99341f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.220+0000 7f9939026640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9934161690 0x7f99341f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.220+0000 7f9939026640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99340cd010 con 0x7f9934161690 2023-10-08T07:15:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.220+0000 7f9939026640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99340cd2f0 con 0x7f99340b2f30 2023-10-08T07:15:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.220+0000 7f9939026640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99341f7aa0 con 0x7f99340cb150 2023-10-08T07:15:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.220+0000 7f993359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9934161690 0x7f99341f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.220+0000 7f993359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9934161690 0x7f99341f3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51530/0 (socket says 172.21.15.70:51530) 2023-10-08T07:15:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.220+0000 7f993359e640 1 -- 172.21.15.70:0/7459135 learned_addr learned my addr 172.21.15.70:0/7459135 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f993259c640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99340cb150 0x7f99341f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f9932d9d640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99340b2f30 0x7f9934161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f993359e640 1 -- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99340cb150 msgr2=0x7f99341f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f993359e640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99340cb150 0x7f99341f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f993359e640 1 -- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99340b2f30 msgr2=0x7f9934161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f993359e640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99340b2f30 0x7f9934161a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f993359e640 1 -- 172.21.15.70:0/7459135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c0094d0 con 0x7f9934161690 2023-10-08T07:15:53.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f993259c640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99340cb150 0x7f99341f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:53.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f993359e640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9934161690 0x7f99341f3f20 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7f992c02b9c0 tx=0x7f992c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:53.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f992b7fe640 1 -- 172.21.15.70:0/7459135 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0307d0 con 0x7f9934161690 2023-10-08T07:15:53.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.221+0000 7f992b7fe640 1 -- 172.21.15.70:0/7459135 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c044480 con 0x7f9934161690 2023-10-08T07:15:53.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.222+0000 7f9939026640 1 -- 172.21.15.70:0/7459135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9934162050 con 0x7f9934161690 2023-10-08T07:15:53.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.222+0000 7f992b7fe640 1 -- 172.21.15.70:0/7459135 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c044830 con 0x7f9934161690 2023-10-08T07:15:53.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.222+0000 7f9939026640 1 -- 172.21.15.70:0/7459135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9934162610 con 0x7f9934161690 2023-10-08T07:15:53.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.224+0000 7f992b7fe640 1 -- 172.21.15.70:0/7459135 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f992c04d3b0 con 0x7f9934161690 2023-10-08T07:15:53.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.226+0000 7f992b7fe640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f991406b0e0 0x7f991406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.226+0000 7f992b7fe640 1 -- 172.21.15.70:0/7459135 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f992c03e070 con 0x7f9934161690 2023-10-08T07:15:53.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.226+0000 7f9932d9d640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f991406b0e0 0x7f991406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.226+0000 7f993bbb6740 1 Processor -- start 2023-10-08T07:15:53.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.227+0000 7f993bbb6740 1 -- start start 2023-10-08T07:15:53.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.227+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 0x5583d2adba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.227+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b08f40 0x5583d2b0b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.227+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2adbfa0 0x5583d2b0be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.227+0000 7f993bbb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d2a4f700 con 0x5583d2adbfa0 2023-10-08T07:15:53.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.227+0000 7f993bbb6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d2a4cdf0 con 0x5583d2b08f40 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.227+0000 7f993bbb6740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583d2a486e0 con 0x5583d2b08b60 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f9932d9d640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f991406b0e0 0x7f991406d5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9924007510 tx=0x7f99240074a0 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f993359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2adbfa0 0x5583d2b0be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f993359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2adbfa0 0x5583d2b0be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51538/0 (socket says 172.21.15.70:51538) 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f993359e640 1 -- 172.21.15.70:0/88232243 learned_addr learned my addr 172.21.15.70:0/88232243 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f9932d9d640 1 --2- 172.21.15.70:0/88232243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 0x5583d2adba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f993259c640 1 --2- 172.21.15.70:0/88232243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b08f40 0x5583d2b0b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f993359e640 1 -- 172.21.15.70:0/88232243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 msgr2=0x5583d2adba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f993359e640 1 --2- 172.21.15.70:0/88232243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 0x5583d2adba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f993359e640 1 -- 172.21.15.70:0/88232243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b08f40 msgr2=0x5583d2b0b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:53.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f993359e640 1 --2- 172.21.15.70:0/88232243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b08f40 0x5583d2b0b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.228+0000 7f993359e640 1 -- 172.21.15.70:0/88232243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583d2a0c910 con 0x5583d2adbfa0 2023-10-08T07:15:53.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f993359e640 1 --2- 172.21.15.70:0/88232243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2adbfa0 0x5583d2b0be30 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f992c07b0f0 tx=0x7f992c07abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfe90dddbb8df22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:53.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f9928ff9640 1 -- 172.21.15.70:0/88232243 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c083460 con 0x5583d2adbfa0 2023-10-08T07:15:53.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f9928ff9640 1 -- 172.21.15.70:0/88232243 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c077870 con 0x5583d2adbfa0 2023-10-08T07:15:53.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f9928ff9640 1 -- 172.21.15.70:0/88232243 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0831d0 con 0x5583d2adbfa0 2023-10-08T07:15:53.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f993bbb6740 1 -- 172.21.15.70:0/88232243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2adbfa0 msgr2=0x5583d2b0be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f993bbb6740 1 --2- 172.21.15.70:0/88232243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2adbfa0 0x5583d2b0be30 secure :-1 s=READY pgs=2908 cs=0 l=1 rev1=1 crypto rx=0x7f992c07b0f0 tx=0x7f992c07abb0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f993bbb6740 1 -- 172.21.15.70:0/88232243 shutdown_connections 2023-10-08T07:15:53.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f993bbb6740 1 --2- 172.21.15.70:0/88232243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2adbfa0 0x5583d2b0be30 unknown :-1 s=CLOSED pgs=2908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f993bbb6740 1 --2- 172.21.15.70:0/88232243 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b08f40 0x5583d2b0b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f993bbb6740 1 --2- 172.21.15.70:0/88232243 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 0x5583d2adba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.229+0000 7f993bbb6740 1 -- 172.21.15.70:0/88232243 >> 172.21.15.70:0/88232243 conn(0x5583d2addd70 msgr2=0x5583d2b268f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.230+0000 7f993bbb6740 1 -- 172.21.15.70:0/88232243 shutdown_connections 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.230+0000 7f993bbb6740 1 -- 172.21.15.70:0/88232243 wait complete. 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.230+0000 7f993bbb6740 1 -- start start 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.230+0000 7f993bbb6740 1 Processor -- start 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.231+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b1e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.231+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08f40 0x5583d2b1f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.231+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b37410 0x5583d2b29e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f993bbb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d29e6970 con 0x5583d2b08b60 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f993bbb6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d29e6610 con 0x5583d2b37410 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f993bbb6740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583d27acf20 con 0x5583d2b08f40 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f993bbb6740 10 client.0 _ll_register_callbacks cb 0x5583d2a24610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:15:53.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f9932d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b1e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f9932d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b1e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51548/0 (socket says 172.21.15.70:51548) 2023-10-08T07:15:53.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f9932d9d640 1 -- 172.21.15.70:0/1908672354 learned_addr learned my addr 172.21.15.70:0/1908672354 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:53.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f9932d9d640 1 -- 172.21.15.70:0/1908672354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08f40 msgr2=0x5583d2b1f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:53.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f9932d9d640 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08f40 0x5583d2b1f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f9932d9d640 1 -- 172.21.15.70:0/1908672354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b37410 msgr2=0x5583d2b29e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:53.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f9932d9d640 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b37410 0x5583d2b29e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f9932d9d640 1 -- 172.21.15.70:0/1908672354 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9934162610 con 0x5583d2b08b60 2023-10-08T07:15:53.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f9932d9d640 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b1e970 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f992400fe20 tx=0x7f9924004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77689c0b0c564c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:53.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.232+0000 7f9911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:15:53.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f9911ffb640 1 -- 172.21.15.70:0/1908672354 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992401c620 con 0x5583d2b08b60 2023-10-08T07:15:53.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f9911ffb640 1 -- 172.21.15.70:0/1908672354 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f9924015070 con 0x5583d2b08b60 2023-10-08T07:15:53.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f993bbb6740 1 -- 172.21.15.70:0/1908672354 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583d2a416c0 con 0x5583d2b08b60 2023-10-08T07:15:53.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f9911ffb640 1 -- 172.21.15.70:0/1908672354 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f992400fe50 con 0x5583d2b08b60 2023-10-08T07:15:53.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f993bbb6740 10 client.12810 fetch_fsmap learned FSMap version 140 2023-10-08T07:15:53.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f993bbb6740 1 -- 172.21.15.70:0/1908672354 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) v3 -- 0x5583d2a46d60 con 0x5583d2b08b60 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f9911ffb640 1 -- 172.21.15.70:0/1908672354 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992402b020 con 0x5583d2b08b60 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f9911ffb640 1 -- 172.21.15.70:0/1908672354 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 140) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9924022020 con 0x5583d2b08b60 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f993bbb6740 10 client.12810 fetch_fsmap finished waiting for FSMap version 140 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f993bbb6740 10 client.12810 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f993bbb6740 20 client.12810 populate_metadata read hostname 'smithi070' 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f993bbb6740 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.70:6832/753129368,v1:172.21.15.70:6834/753129368] conn(0x5583d2b0e800 0x5583d2b39cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f993bbb6740 4 client.12810 mds_command: new command op to 12645 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.233+0000 7f993bbb6740 1 -- 172.21.15.70:0/1908672354 --> [v2:172.21.15.70:6832/753129368,v1:172.21.15.70:6834/753129368] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583d2a413e0 con 0x5583d2b0e800 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.234+0000 7f993259c640 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.70:6832/753129368,v1:172.21.15.70:6834/753129368] conn(0x5583d2b0e800 0x5583d2b39cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.234+0000 7f993259c640 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.70:6832/753129368,v1:172.21.15.70:6834/753129368] conn(0x5583d2b0e800 0x5583d2b39cc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6588ec9deaaaf011 server_cookie=9af4c7be8cfaa30e in_seq=0 out_seq=0 2023-10-08T07:15:53.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.234+0000 7f9911ffb640 10 client.12810 ms_handle_connect on v2:172.21.15.70:6832/753129368 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f9911ffb640 1 -- 172.21.15.70:0/1908672354 <== mds.0 v2:172.21.15.70:6832/753129368 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5583d2a413e0 con 0x5583d2b0e800 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f9911ffb640 10 client.12810 handle_command_reply: tid=0 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f993bbb6740 1 client.12810 shutdown 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f993bbb6740 1 -- 172.21.15.70:0/1908672354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 msgr2=0x5583d2b1e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f993bbb6740 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b1e970 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7f992400fe20 tx=0x7f9924004ab0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f993bbb6740 1 -- 172.21.15.70:0/1908672354 shutdown_connections 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f993bbb6740 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.70:6832/753129368,v1:172.21.15.70:6834/753129368] conn(0x5583d2b0e800 0x5583d2b39cc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f993bbb6740 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b37410 0x5583d2b29e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f993bbb6740 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08f40 0x5583d2b1f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f993bbb6740 1 --2- 172.21.15.70:0/1908672354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b1e970 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f9911ffb640 0 client.12810 ms_handle_reset on v2:172.21.15.70:6832/753129368 2023-10-08T07:15:53.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.235+0000 7f993bbb6740 1 -- 172.21.15.70:0/1908672354 >> 172.21.15.70:0/1908672354 conn(0x5583d2adbfa0 msgr2=0x5583d2adf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:53.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.236+0000 7f993bbb6740 1 -- 172.21.15.70:0/1908672354 shutdown_connections 2023-10-08T07:15:53.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.236+0000 7f993bbb6740 1 -- 172.21.15.70:0/1908672354 wait complete. 2023-10-08T07:15:53.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.236+0000 7f993bbb6740 20 client.12810 trim_cache size 0 max 16384 2023-10-08T07:15:53.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.252+0000 7f993bbb6740 1 Processor -- start 2023-10-08T07:15:53.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.252+0000 7f993bbb6740 1 -- start start 2023-10-08T07:15:53.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.253+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b0cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.253+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b08f40 0x5583d2b26c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.253+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b37410 0x5583d2b271a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.253+0000 7f993bbb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d2b29fb0 con 0x5583d2b08b60 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.253+0000 7f993bbb6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d2b1f560 con 0x5583d2b08f40 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.253+0000 7f993bbb6740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583d2b36b00 con 0x5583d2b37410 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.253+0000 7f9932d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b0cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.254+0000 7f9932d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b0cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51556/0 (socket says 172.21.15.70:51556) 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.254+0000 7f9932d9d640 1 -- 172.21.15.70:0/1497362989 learned_addr learned my addr 172.21.15.70:0/1497362989 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.254+0000 7f9932d9d640 1 -- 172.21.15.70:0/1497362989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b37410 msgr2=0x5583d2b271a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.254+0000 7f9932d9d640 1 --2- 172.21.15.70:0/1497362989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b37410 0x5583d2b271a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.254+0000 7f9932d9d640 1 -- 172.21.15.70:0/1497362989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b08f40 msgr2=0x5583d2b26c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.254+0000 7f9932d9d640 1 --2- 172.21.15.70:0/1497362989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b08f40 0x5583d2b26c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.254+0000 7f9932d9d640 1 -- 172.21.15.70:0/1497362989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583d29e5b80 con 0x5583d2b08b60 2023-10-08T07:15:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.254+0000 7f9932d9d640 1 --2- 172.21.15.70:0/1497362989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b0cad0 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f992400fa80 tx=0x7f992401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc4dabdacd6e47f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.254+0000 7f99297fa640 1 -- 172.21.15.70:0/1497362989 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992401c620 con 0x5583d2b08b60 2023-10-08T07:15:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f99297fa640 1 -- 172.21.15.70:0/1497362989 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992402b070 con 0x5583d2b08b60 2023-10-08T07:15:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f99297fa640 1 -- 172.21.15.70:0/1497362989 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992401cd90 con 0x5583d2b08b60 2023-10-08T07:15:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f993bbb6740 1 -- 172.21.15.70:0/1497362989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 msgr2=0x5583d2b0cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f993bbb6740 1 --2- 172.21.15.70:0/1497362989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b0cad0 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7f992400fa80 tx=0x7f992401d000 comp rx=0 tx=0).stop 2023-10-08T07:15:53.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f993bbb6740 1 -- 172.21.15.70:0/1497362989 shutdown_connections 2023-10-08T07:15:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f993bbb6740 1 --2- 172.21.15.70:0/1497362989 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b37410 0x5583d2b271a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f993bbb6740 1 --2- 172.21.15.70:0/1497362989 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b08f40 0x5583d2b26c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f993bbb6740 1 --2- 172.21.15.70:0/1497362989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08b60 0x5583d2b0cad0 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f993bbb6740 1 -- 172.21.15.70:0/1497362989 >> 172.21.15.70:0/1497362989 conn(0x5583d2adbfa0 msgr2=0x5583d2b49150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f993bbb6740 1 -- 172.21.15.70:0/1497362989 shutdown_connections 2023-10-08T07:15:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.255+0000 7f993bbb6740 1 -- 172.21.15.70:0/1497362989 wait complete. 2023-10-08T07:15:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.256+0000 7f993bbb6740 1 -- start start 2023-10-08T07:15:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.256+0000 7f993bbb6740 1 Processor -- start 2023-10-08T07:15:53.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.257+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 0x5583d2b91c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.257+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08f40 0x5583d2b3f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.257+0000 7f993bbb6740 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b37410 0x5583d2b3fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.257+0000 7f993bbb6740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d2b36760 con 0x5583d2b08f40 2023-10-08T07:15:53.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.257+0000 7f993bbb6740 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d2b253e0 con 0x5583d2b37410 2023-10-08T07:15:53.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.257+0000 7f993bbb6740 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583d2b0f4c0 con 0x5583d2b08b60 2023-10-08T07:15:53.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.257+0000 7f993bbb6740 10 client.0 _ll_register_callbacks cb 0x7f992c077520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T07:15:53.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.257+0000 7f993259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08f40 0x5583d2b3f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f993259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08f40 0x5583d2b3f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51564/0 (socket says 172.21.15.70:51564) 2023-10-08T07:15:53.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f993259c640 1 -- 172.21.15.70:0/408472389 learned_addr learned my addr 172.21.15.70:0/408472389 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f9932d9d640 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 0x5583d2b91c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f993359e640 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b37410 0x5583d2b3fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f993259c640 1 -- 172.21.15.70:0/408472389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 msgr2=0x5583d2b91c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f993259c640 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 0x5583d2b91c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f993259c640 1 -- 172.21.15.70:0/408472389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b37410 msgr2=0x5583d2b3fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f993259c640 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b37410 0x5583d2b3fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f993259c640 1 -- 172.21.15.70:0/408472389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583d2a46d60 con 0x5583d2b08f40 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f9932d9d640 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 0x5583d2b91c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f993259c640 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08f40 0x5583d2b3f830 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f991c001640 tx=0x7f991c015dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ac999e5e81315d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f9911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-10-08T07:15:53.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f9911ffb640 1 -- 172.21.15.70:0/408472389 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c00c040 con 0x5583d2b08f40 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f9911ffb640 1 -- 172.21.15.70:0/408472389 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f991c005470 con 0x5583d2b08f40 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.258+0000 7f993bbb6740 1 -- 172.21.15.70:0/408472389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f992c077870 con 0x5583d2b08f40 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f9911ffb640 1 -- 172.21.15.70:0/408472389 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) v2 ==== 24+0+0 (secure 0 0 0) 0x7f991c007030 con 0x5583d2b08f40 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f993bbb6740 10 client.12816 fetch_fsmap learned FSMap version 140 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f993bbb6740 1 -- 172.21.15.70:0/408472389 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=2+}) v3 -- 0x5583d2a0b170 con 0x5583d2b08f40 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f9911ffb640 1 -- 172.21.15.70:0/408472389 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c077870 con 0x5583d2b08f40 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f9911ffb640 1 -- 172.21.15.70:0/408472389 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 140) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f991c012040 con 0x5583d2b08f40 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f993bbb6740 10 client.12816 fetch_fsmap finished waiting for FSMap version 140 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f993bbb6740 10 client.12816 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f993bbb6740 20 client.12816 populate_metadata read hostname 'smithi070' 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f993bbb6740 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.70:6832/753129368,v1:172.21.15.70:6834/753129368] conn(0x5583d2b356f0 0x5583d2b240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f993bbb6740 4 client.12816 mds_command: new command op to 12645 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f993bbb6740 1 -- 172.21.15.70:0/408472389 --> [v2:172.21.15.70:6832/753129368,v1:172.21.15.70:6834/753129368] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5583d274a6c0 con 0x5583d2b356f0 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.259+0000 7f9932d9d640 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.70:6832/753129368,v1:172.21.15.70:6834/753129368] conn(0x5583d2b356f0 0x5583d2b240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f9932d9d640 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.70:6832/753129368,v1:172.21.15.70:6834/753129368] conn(0x5583d2b356f0 0x5583d2b240a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d6cb6dda308043 server_cookie=10779c78f7a57885 in_seq=0 out_seq=0 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f9911ffb640 10 client.12816 ms_handle_connect on v2:172.21.15.70:6832/753129368 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f9911ffb640 1 -- 172.21.15.70:0/408472389 <== mds.0 v2:172.21.15.70:6832/753129368 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x5583d274a6c0 con 0x5583d2b356f0 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f9911ffb640 10 client.12816 handle_command_reply: tid=0 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f993bbb6740 1 client.12816 shutdown 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f993bbb6740 1 -- 172.21.15.70:0/408472389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08f40 msgr2=0x5583d2b3f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f993bbb6740 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08f40 0x5583d2b3f830 secure :-1 s=READY pgs=2911 cs=0 l=1 rev1=1 crypto rx=0x7f991c001640 tx=0x7f991c015dd0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f993bbb6740 1 -- 172.21.15.70:0/408472389 shutdown_connections 2023-10-08T07:15:53.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f993bbb6740 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.70:6832/753129368,v1:172.21.15.70:6834/753129368] conn(0x5583d2b356f0 0x5583d2b240a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f993bbb6740 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x5583d2b37410 0x5583d2b3fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f993bbb6740 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5583d2b08f40 0x5583d2b3f830 unknown :-1 s=CLOSED pgs=2911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f993bbb6740 1 --2- 172.21.15.70:0/408472389 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x5583d2b08b60 0x5583d2b91c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.260+0000 7f993bbb6740 1 -- 172.21.15.70:0/408472389 >> 172.21.15.70:0/408472389 conn(0x5583d2adbfa0 msgr2=0x5583d2b0cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:53.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9911ffb640 0 client.12816 ms_handle_reset on v2:172.21.15.70:6832/753129368 2023-10-08T07:15:53.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f993bbb6740 1 -- 172.21.15.70:0/408472389 shutdown_connections 2023-10-08T07:15:53.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f993bbb6740 1 -- 172.21.15.70:0/408472389 wait complete. 2023-10-08T07:15:53.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f993bbb6740 20 client.12816 trim_cache size 0 max 16384 2023-10-08T07:15:53.267 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "cbcacf35-14e2-4581-b83a-6dd588939bbc", 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 12645, 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.8369636820000004, 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 140, 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 276, 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 276, 2023-10-08T07:15:53.268 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 13.614532705 2023-10-08T07:15:53.269 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:15:53.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9939026640 1 -- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f991406b0e0 msgr2=0x7f991406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9939026640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f991406b0e0 0x7f991406d5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9924007510 tx=0x7f99240074a0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9939026640 1 -- 172.21.15.70:0/7459135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9934161690 msgr2=0x7f99341f3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9939026640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9934161690 0x7f99341f3f20 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7f992c02b9c0 tx=0x7f992c004730 comp rx=0 tx=0).stop 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9939026640 1 -- 172.21.15.70:0/7459135 shutdown_connections 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9939026640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f991406b0e0 0x7f991406d5a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9939026640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9934161690 0x7f99341f3f20 unknown :-1 s=CLOSED pgs=2907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9939026640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f99340cb150 0x7f99341f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9939026640 1 --2- 172.21.15.70:0/7459135 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f99340b2f30 0x7f9934161a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.261+0000 7f9939026640 1 -- 172.21.15.70:0/7459135 >> 172.21.15.70:0/7459135 conn(0x7f993406a5d0 msgr2=0x7f99340c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.262+0000 7f9939026640 1 -- 172.21.15.70:0/7459135 shutdown_connections 2023-10-08T07:15:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.262+0000 7f9939026640 1 -- 172.21.15.70:0/7459135 wait complete. 2023-10-08T07:15:53.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T07:15:53.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.483+0000 7f4424c4b640 1 Processor -- start 2023-10-08T07:15:53.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.483+0000 7f4424c4b640 1 -- start start 2023-10-08T07:15:53.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.485+0000 7f4424c4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f442015ab40 0x7f44200ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.485+0000 7f4424c4b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44200c90b0 0x7f44200adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.485+0000 7f4424c4b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442015b2b0 con 0x7f44200cae70 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.485+0000 7f4424c4b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f442015bf30 con 0x7f44200c90b0 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.485+0000 7f4424c4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442015f7a0 con 0x7f442015ab40 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.485+0000 7f4417fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44200c90b0 0x7f44200adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.485+0000 7f4417fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44200c90b0 0x7f44200adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51568/0 (socket says 172.21.15.70:51568) 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.485+0000 7f4417fff640 1 -- 172.21.15.70:0/770406448 learned_addr learned my addr 172.21.15.70:0/770406448 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.485+0000 7f441ed9d640 1 --1- 172.21.15.70:0/770406448 >> v1:172.21.15.70:6789/0 conn(0x7f44200cae70 0x7f44200702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52492/0 (socket says 172.21.15.70:52492) 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.486+0000 7f441e59c640 1 --2- 172.21.15.70:0/770406448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f442015ab40 0x7f44200ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.486+0000 7f4417fff640 1 -- 172.21.15.70:0/770406448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f442015ab40 msgr2=0x7f44200ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.486+0000 7f4417fff640 1 --2- 172.21.15.70:0/770406448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f442015ab40 0x7f44200ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.486+0000 7f4417fff640 1 -- 172.21.15.70:0/770406448 >> v1:172.21.15.70:6789/0 conn(0x7f44200cae70 legacy=0x7f44200702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.486+0000 7f4417fff640 1 -- 172.21.15.70:0/770406448 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442015b590 con 0x7f44200c90b0 2023-10-08T07:15:53.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.486+0000 7f441e59c640 1 --2- 172.21.15.70:0/770406448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f442015ab40 0x7f44200ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:53.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.487+0000 7f4417fff640 1 --2- 172.21.15.70:0/770406448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 0x7f44200adc60 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f4408009870 tx=0x7f440802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89d94336066f221 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:53.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.487+0000 7f441dd9b640 1 -- 172.21.15.70:0/770406448 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408030840 con 0x7f44200c90b0 2023-10-08T07:15:53.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.487+0000 7f441dd9b640 1 -- 172.21.15.70:0/770406448 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44080309a0 con 0x7f44200c90b0 2023-10-08T07:15:53.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.487+0000 7f441dd9b640 1 -- 172.21.15.70:0/770406448 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408030d30 con 0x7f44200c90b0 2023-10-08T07:15:53.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.488+0000 7f4424c4b640 1 -- 172.21.15.70:0/770406448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 msgr2=0x7f44200adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.488+0000 7f4424c4b640 1 --2- 172.21.15.70:0/770406448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 0x7f44200adc60 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7f4408009870 tx=0x7f440802b980 comp rx=0 tx=0).stop 2023-10-08T07:15:53.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.488+0000 7f4424c4b640 1 -- 172.21.15.70:0/770406448 shutdown_connections 2023-10-08T07:15:53.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.488+0000 7f4424c4b640 1 --2- 172.21.15.70:0/770406448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 0x7f44200adc60 unknown :-1 s=CLOSED pgs=2913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.488+0000 7f4424c4b640 1 --2- 172.21.15.70:0/770406448 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f442015ab40 0x7f44200ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.488+0000 7f4424c4b640 1 -- 172.21.15.70:0/770406448 >> 172.21.15.70:0/770406448 conn(0x7f442006a380 msgr2=0x7f44200b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:53.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.488+0000 7f4424c4b640 1 -- 172.21.15.70:0/770406448 shutdown_connections 2023-10-08T07:15:53.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.488+0000 7f4424c4b640 1 -- 172.21.15.70:0/770406448 wait complete. 2023-10-08T07:15:53.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.490+0000 7f4424c4b640 1 Processor -- start 2023-10-08T07:15:53.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.490+0000 7f4424c4b640 1 -- start start 2023-10-08T07:15:53.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.491+0000 7f4424c4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 0x7f442015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.491+0000 7f4424c4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44200cae70 0x7f442015c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.491+0000 7f4424c4b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f442015ab40 0x7f44201450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.491+0000 7f4424c4b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442015dc00 con 0x7f44200c90b0 2023-10-08T07:15:53.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.491+0000 7f4424c4b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442015cd50 con 0x7f442015ab40 2023-10-08T07:15:53.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.491+0000 7f4424c4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4420145620 con 0x7f44200cae70 2023-10-08T07:15:53.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.491+0000 7f441e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 0x7f442015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.491+0000 7f441e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 0x7f442015e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51574/0 (socket says 172.21.15.70:51574) 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.491+0000 7f441e59c640 1 -- 172.21.15.70:0/1720473798 learned_addr learned my addr 172.21.15.70:0/1720473798 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f4417fff640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44200cae70 0x7f442015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f441ed9d640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f442015ab40 0x7f44201450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f441e59c640 1 -- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44200cae70 msgr2=0x7f442015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f441e59c640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44200cae70 0x7f442015c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f441e59c640 1 -- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f442015ab40 msgr2=0x7f44201450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f441e59c640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f442015ab40 0x7f44201450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f441e59c640 1 -- 172.21.15.70:0/1720473798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44080094d0 con 0x7f44200c90b0 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f441e59c640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 0x7f442015e020 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f441000c820 tx=0x7f441000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f4416ffd640 1 -- 172.21.15.70:0/1720473798 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44100079d0 con 0x7f44200c90b0 2023-10-08T07:15:53.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f4416ffd640 1 -- 172.21.15.70:0/1720473798 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410007b30 con 0x7f44200c90b0 2023-10-08T07:15:53.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f4416ffd640 1 -- 172.21.15.70:0/1720473798 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441000ce70 con 0x7f44200c90b0 2023-10-08T07:15:53.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f4424c4b640 1 -- 172.21.15.70:0/1720473798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4420152b00 con 0x7f44200c90b0 2023-10-08T07:15:53.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.492+0000 7f4424c4b640 1 -- 172.21.15.70:0/1720473798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442015d260 con 0x7f44200c90b0 2023-10-08T07:15:53.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.494+0000 7f4424c4b640 1 -- 172.21.15.70:0/1720473798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442015de70 con 0x7f44200c90b0 2023-10-08T07:15:53.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.494+0000 7f4416ffd640 1 -- 172.21.15.70:0/1720473798 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f44100113f0 con 0x7f44200c90b0 2023-10-08T07:15:53.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.495+0000 7f4416ffd640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f440006b0b0 0x7f440006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:53.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.495+0000 7f4416ffd640 1 -- 172.21.15.70:0/1720473798 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f4410089f90 con 0x7f44200c90b0 2023-10-08T07:15:53.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.496+0000 7f4417fff640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f440006b0b0 0x7f440006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:53.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.497+0000 7f4417fff640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f440006b0b0 0x7f440006d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4408036000 tx=0x7f440803b040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:53.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.499+0000 7f4416ffd640 1 -- 172.21.15.70:0/1720473798 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4410054950 con 0x7f44200c90b0 2023-10-08T07:15:53.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.726+0000 7f4424c4b640 1 -- 172.21.15.70:0/1720473798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f442015bb20 con 0x7f44200c90b0 2023-10-08T07:15:53.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.727+0000 7f4416ffd640 1 -- 172.21.15.70:0/1720473798 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v140) v1 ==== 53+0+73 (secure 0 0 0) 0x7f441001f070 con 0x7f44200c90b0 2023-10-08T07:15:53.729 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T07:15:53.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 -- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f440006b0b0 msgr2=0x7f440006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f440006b0b0 0x7f440006d570 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4408036000 tx=0x7f440803b040 comp rx=0 tx=0).stop 2023-10-08T07:15:53.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 -- 172.21.15.70:0/1720473798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 msgr2=0x7f442015e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:53.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 0x7f442015e020 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7f441000c820 tx=0x7f441000cc20 comp rx=0 tx=0).stop 2023-10-08T07:15:53.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 -- 172.21.15.70:0/1720473798 shutdown_connections 2023-10-08T07:15:53.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f440006b0b0 0x7f440006d570 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f442015ab40 0x7f44201450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f44200cae70 0x7f442015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 --2- 172.21.15.70:0/1720473798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44200c90b0 0x7f442015e020 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:53.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 -- 172.21.15.70:0/1720473798 >> 172.21.15.70:0/1720473798 conn(0x7f442006a380 msgr2=0x7f44201543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:53.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 -- 172.21.15.70:0/1720473798 shutdown_connections 2023-10-08T07:15:53.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:53.730+0000 7f4424c4b640 1 -- 172.21.15.70:0/1720473798 wait complete. 2023-10-08T07:15:53.749 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T07:15:53.749 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T07:15:53.750 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-10-08T07:15:53.750 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T07:15:53.750 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T07:15:53.750 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T07:15:53.750 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T07:15:53.750 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T07:15:53.750 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:15:53.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:53.750 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-10-08T07:15:53.801 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T07:15:53.801 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T07:15:53.801 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T07:15:53.801 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:15:53.802 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-10-08T07:15:53.802 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:15:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:15:53.802 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:15:53.802 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-10-08T07:15:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:15:53.802 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:15:53.802 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-10-08T07:15:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T07:15:53.802 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 36443sec preferred_lft 36443sec 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::3446:4fff:fe4f:1dbe/64 scope link 2023-10-08T07:15:53.803 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:15:53.804 INFO:teuthology.orchestra.run.smithi070.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T07:15:53.804 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 3a:a4:16:0e:a2:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T07:15:53.804 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::38a4:16ff:fe0e:a213/64 scope link 2023-10-08T07:15:53.804 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:15:53.804 INFO:teuthology.orchestra.run.smithi070.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T07:15:53.804 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether b6:3e:ad:ca:6a:93 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T07:15:53.804 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::b43e:adff:feca:6a93/64 scope link 2023-10-08T07:15:53.804 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-10-08T07:15:53.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:53.804 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T07:15:53.892 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-10-08T07:15:53.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:53.896 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-10-08T07:15:53.949 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-10-08T07:15:53.949 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T07:15:53.949 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T07:15:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T07:15:53.950 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T07:15:53.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:53.950 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-10-08T07:15:53.950 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-10-08T07:15:53.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T07:15:53.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T07:15:53.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T07:15:53.951 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-10-08T07:15:54.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:54.132 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T07:15:54.182 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T07:15:54.182 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T07:15:54.183 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:15:54.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:54.183 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-10-08T07:15:54.273 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T07:15:54.273 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T07:15:54.326 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T07:15:54.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T07:15:54.327 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T07:15:54.378 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T07:15:54.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:54.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:15:54.517 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:15:54.517 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T07:15:54.552 INFO:teuthology.orchestra.run.smithi070.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T07:15:54.552 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T07:15:54.552 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: resolved to: "172.21.15.70:3300,172.21.15.181:3300,172.21.15.181:3301" 2023-10-08T07:15:54.552 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: trying mount with new device syntax: 0@cbcacf35-14e2-4581-b83a-6dd588939bbc.cephfs=/ 2023-10-08T07:15:54.552 INFO:teuthology.orchestra.run.smithi070.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.70:3300/172.21.15.181:3300/172.21.15.181:3301" will pass to kernel 2023-10-08T07:15:54.554 INFO:teuthology.orchestra.run.smithi070.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-10-08T07:15:54.554 INFO:teuthology.orchestra.run.smithi070.stdout: You just mounted a file system that supports labels which does not 2023-10-08T07:15:54.554 INFO:teuthology.orchestra.run.smithi070.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-10-08T07:15:54.554 INFO:teuthology.orchestra.run.smithi070.stdout: applications will generate AVC messages and not be allowed access to 2023-10-08T07:15:54.554 INFO:teuthology.orchestra.run.smithi070.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-10-08T07:15:54.557 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T07:15:54.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:54.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T07:15:54.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:54.646 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-08T07:15:54.704 INFO:teuthology.orchestra.run.smithi070.stdout:client8153 2023-10-08T07:15:54.704 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:15:54.705 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:15:54.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T07:15:54.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.962+0000 7fa01b2f2640 1 Processor -- start 2023-10-08T07:15:54.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.962+0000 7fa01b2f2640 1 -- start start 2023-10-08T07:15:54.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.964+0000 7fa01b2f2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa01406c3a0 0x7fa0140c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:54.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.964+0000 7fa01b2f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406db60 0x7fa01406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:54.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.964+0000 7fa01b2f2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01415b310 con 0x7fa01415aa00 2023-10-08T07:15:54.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.964+0000 7fa01b2f2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0140ca230 con 0x7fa01406c3a0 2023-10-08T07:15:54.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.964+0000 7fa01b2f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01406d290 con 0x7fa01406db60 2023-10-08T07:15:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.964+0000 7fa01988f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa01415aa00 0x7fa0140b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40598/0 (socket says 172.21.15.70:40598) 2023-10-08T07:15:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.964+0000 7fa01988f640 1 -- 172.21.15.70:0/3579866109 learned_addr learned my addr 172.21.15.70:0/3579866109 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.964+0000 7fa01888d640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406db60 0x7fa01406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.964+0000 7fa01908e640 1 --2- 172.21.15.70:0/3579866109 >> v2:172.21.15.181:3300/0 conn(0x7fa01406c3a0 0x7fa0140c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529413288 0 0) 0x7fa00c001490 con 0x7fa01415aa00 2023-10-08T07:15:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa000005730 con 0x7fa01415aa00 2023-10-08T07:15:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1403950353 0 0) 0x7fa000005730 con 0x7fa01415aa00 2023-10-08T07:15:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406db60 msgr2=0x7fa01406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406db60 0x7fa01406c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 >> v2:172.21.15.181:3300/0 conn(0x7fa01406c3a0 msgr2=0x7fa0140c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 --2- 172.21.15.70:0/3579866109 >> v2:172.21.15.181:3300/0 conn(0x7fa01406c3a0 0x7fa0140c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0140c9860 con 0x7fa01415aa00 2023-10-08T07:15:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fa00c001b80 con 0x7fa01415aa00 2023-10-08T07:15:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 >> v1:172.21.15.70:6789/0 conn(0x7fa01415aa00 legacy=0x7fa0140b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa000005b80 0x7fa000005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406db60 0x7fa0000069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 0x7fa000007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01406d290 con 0x7fa000006f10 2023-10-08T07:15:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00000db90 con 0x7fa000005b80 2023-10-08T07:15:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.965+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00000dcd0 con 0x7fa01406db60 2023-10-08T07:15:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.966+0000 7fa01988f640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 0x7fa000007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.966+0000 7fa01908e640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa000005b80 0x7fa000005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.966+0000 7fa01888d640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406db60 0x7fa0000069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.966+0000 7fa01988f640 1 -- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406db60 msgr2=0x7fa0000069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.966+0000 7fa01988f640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406db60 0x7fa0000069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.966+0000 7fa01988f640 1 -- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa000005b80 msgr2=0x7fa000005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.966+0000 7fa01988f640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa000005b80 0x7fa000005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.966+0000 7fa01988f640 1 -- 172.21.15.70:0/3579866109 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa000005730 con 0x7fa000006f10 2023-10-08T07:15:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.966+0000 7fa01888d640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa01406db60 0x7fa0000069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:54.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.967+0000 7fa01988f640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 0x7fa000007390 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00a760 tx=0x7fa000007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980170f12d38a1c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.967+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c034630 con 0x7fa000006f10 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.967+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c02f070 con 0x7fa000006f10 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.967+0000 7fa00bfff640 1 -- 172.21.15.70:0/3579866109 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0349f0 con 0x7fa000006f10 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.968+0000 7fa01b2f2640 1 -- 172.21.15.70:0/3579866109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 msgr2=0x7fa000007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.968+0000 7fa01b2f2640 1 --2- 172.21.15.70:0/3579866109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 0x7fa000007390 secure :-1 s=READY pgs=2916 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00a760 tx=0x7fa000007a00 comp rx=0 tx=0).stop 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.968+0000 7fa01988f640 1 -- 172.21.15.70:0/3579866109 reap_dead start 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.968+0000 7fa01b2f2640 1 -- 172.21.15.70:0/3579866109 shutdown_connections 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.968+0000 7fa01b2f2640 1 -- 172.21.15.70:0/3579866109 >> 172.21.15.70:0/3579866109 conn(0x7fa01406a5d0 msgr2=0x7fa0140b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.968+0000 7fa01b2f2640 1 -- 172.21.15.70:0/3579866109 shutdown_connections 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.968+0000 7fa01b2f2640 1 -- 172.21.15.70:0/3579866109 wait complete. 2023-10-08T07:15:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.970+0000 7fa01b2f2640 1 Processor -- start 2023-10-08T07:15:54.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.970+0000 7fa01b2f2640 1 -- start start 2023-10-08T07:15:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01b2f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 0x7fa0140c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01b2f2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa014150f20 0x7fa0140c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01b2f2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0140c4e10 0x7fa0140c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01b2f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141ee7d0 con 0x7fa000006f10 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01b2f2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141eea70 con 0x7fa0140c4e10 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01b2f2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0141eed50 con 0x7fa014150f20 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01908e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 0x7fa0140c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01908e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 0x7fa0140c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51532/0 (socket says 172.21.15.70:51532) 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01908e640 1 -- 172.21.15.70:0/1371728966 learned_addr learned my addr 172.21.15.70:0/1371728966 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01908e640 1 -- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa014150f20 msgr2=0x7fa0140c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01908e640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa014150f20 0x7fa0140c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01908e640 1 -- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0140c4e10 msgr2=0x7fa0140c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01908e640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0140c4e10 0x7fa0140c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.971+0000 7fa01908e640 1 -- 172.21.15.70:0/1371728966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0141ef030 con 0x7fa000006f10 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.972+0000 7fa01908e640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 0x7fa0140c48d0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7fa010011600 tx=0x7fa010011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.972+0000 7fa009ffb640 1 -- 172.21.15.70:0/1371728966 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010015870 con 0x7fa000006f10 2023-10-08T07:15:54.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.972+0000 7fa009ffb640 1 -- 172.21.15.70:0/1371728966 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0100159d0 con 0x7fa000006f10 2023-10-08T07:15:54.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.972+0000 7fa01b2f2640 1 -- 172.21.15.70:0/1371728966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0141ef310 con 0x7fa000006f10 2023-10-08T07:15:54.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.972+0000 7fa009ffb640 1 -- 172.21.15.70:0/1371728966 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010015d40 con 0x7fa000006f10 2023-10-08T07:15:54.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.972+0000 7fa01b2f2640 1 -- 172.21.15.70:0/1371728966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0141f1c20 con 0x7fa000006f10 2023-10-08T07:15:54.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.973+0000 7fa01b2f2640 1 -- 172.21.15.70:0/1371728966 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01404e160 con 0x7fa000006f10 2023-10-08T07:15:54.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.974+0000 7fa009ffb640 1 -- 172.21.15.70:0/1371728966 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7fa01000f490 con 0x7fa000006f10 2023-10-08T07:15:54.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.976+0000 7fa009ffb640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ff806b0b0 0x7f9ff806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:54.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.976+0000 7fa009ffb640 1 -- 172.21.15.70:0/1371728966 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fa01008a510 con 0x7fa000006f10 2023-10-08T07:15:54.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.976+0000 7fa01888d640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ff806b0b0 0x7f9ff806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:54.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.978+0000 7fa01888d640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ff806b0b0 0x7f9ff806d570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa014151320 tx=0x7fa004008040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:54.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:54.981+0000 7fa009ffb640 1 -- 172.21.15.70:0/1371728966 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa010054fe0 con 0x7fa000006f10 2023-10-08T07:15:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.194+0000 7fa01b2f2640 1 -- 172.21.15.70:0/1371728966 --> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0141eebe0 con 0x7f9ff806b0b0 2023-10-08T07:15:55.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.198+0000 7fa009ffb640 1 -- 172.21.15.70:0/1371728966 <== mgr.9067 v2:172.21.15.181:6832/440573135 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa0141eebe0 con 0x7f9ff806b0b0 2023-10-08T07:15:55.199 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-10-08T07:15:55.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-10-08T07:15:55.199 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs" 2023-10-08T07:15:55.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-10-08T07:15:55.200 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-10-08T07:15:55.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 -- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ff806b0b0 msgr2=0x7f9ff806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:55.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ff806b0b0 0x7f9ff806d570 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa014151320 tx=0x7fa004008040 comp rx=0 tx=0).stop 2023-10-08T07:15:55.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 -- 172.21.15.70:0/1371728966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 msgr2=0x7fa0140c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:55.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 0x7fa0140c48d0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7fa010011600 tx=0x7fa010011ac0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 -- 172.21.15.70:0/1371728966 shutdown_connections 2023-10-08T07:15:55.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f9ff806b0b0 0x7f9ff806d570 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa0140c4e10 0x7fa0140c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa014150f20 0x7fa0140c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 --2- 172.21.15.70:0/1371728966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa000006f10 0x7fa0140c48d0 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 -- 172.21.15.70:0/1371728966 >> 172.21.15.70:0/1371728966 conn(0x7fa01406a5d0 msgr2=0x7fa0141582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 -- 172.21.15.70:0/1371728966 shutdown_connections 2023-10-08T07:15:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.200+0000 7fa01b2f2640 1 -- 172.21.15.70:0/1371728966 wait complete. 2023-10-08T07:15:55.221 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T07:15:55.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.438+0000 7f17515c9640 1 Processor -- start 2023-10-08T07:15:55.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.438+0000 7f17515c9640 1 -- start start 2023-10-08T07:15:55.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.439+0000 7f17515c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f174c1573c0 0x7f174c15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:55.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.439+0000 7f17515c9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f174c15c2f0 0x7f174c06db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:55.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.439+0000 7f17515c9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174c15ae00 con 0x7f174c06d6f0 2023-10-08T07:15:55.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.439+0000 7f17515c9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174c06e0c0 con 0x7f174c1573c0 2023-10-08T07:15:55.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.439+0000 7f17515c9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174c15d790 con 0x7f174c15c2f0 2023-10-08T07:15:55.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.440+0000 7f174affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f174c1573c0 0x7f174c15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:55.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.440+0000 7f174affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f174c1573c0 0x7f174c15a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51540/0 (socket says 172.21.15.70:51540) 2023-10-08T07:15:55.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.440+0000 7f174affd640 1 -- 172.21.15.70:0/3131327132 learned_addr learned my addr 172.21.15.70:0/3131327132 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:55.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.440+0000 7f17427fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f174c06d6f0 0x7f174c06c3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40614/0 (socket says 172.21.15.70:40614) 2023-10-08T07:15:55.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.440+0000 7f174affd640 1 -- 172.21.15.70:0/3131327132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f174c15c2f0 msgr2=0x7f174c06db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:55.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.440+0000 7f174affd640 1 --2- 172.21.15.70:0/3131327132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f174c15c2f0 0x7f174c06db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.440+0000 7f174affd640 1 -- 172.21.15.70:0/3131327132 >> v1:172.21.15.70:6789/0 conn(0x7f174c06d6f0 legacy=0x7f174c06c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:55.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.440+0000 7f174affd640 1 -- 172.21.15.70:0/3131327132 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c15da70 con 0x7f174c1573c0 2023-10-08T07:15:55.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.440+0000 7f174a7fc640 1 -- 172.21.15.70:0/3131327132 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564026294 0 0) 0x7f174c15ae00 con 0x7f174c06d6f0 2023-10-08T07:15:55.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.441+0000 7f174affd640 1 --2- 172.21.15.70:0/3131327132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c1573c0 0x7f174c15a8c0 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f173c009870 tx=0x7f173c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbff6a2e0b2d0953 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:55.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f174a7fc640 1 -- 172.21.15.70:0/3131327132 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c031640 con 0x7f174c1573c0 2023-10-08T07:15:55.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f174a7fc640 1 -- 172.21.15.70:0/3131327132 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c0317a0 con 0x7f174c1573c0 2023-10-08T07:15:55.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f174a7fc640 1 -- 172.21.15.70:0/3131327132 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c031b30 con 0x7f174c1573c0 2023-10-08T07:15:55.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f17515c9640 1 -- 172.21.15.70:0/3131327132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c1573c0 msgr2=0x7f174c15a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:55.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f17515c9640 1 --2- 172.21.15.70:0/3131327132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c1573c0 0x7f174c15a8c0 secure :-1 s=READY pgs=2919 cs=0 l=1 rev1=1 crypto rx=0x7f173c009870 tx=0x7f173c02b8f0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f17515c9640 1 -- 172.21.15.70:0/3131327132 shutdown_connections 2023-10-08T07:15:55.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f17515c9640 1 --2- 172.21.15.70:0/3131327132 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f174c15c2f0 0x7f174c06db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f17515c9640 1 --2- 172.21.15.70:0/3131327132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c1573c0 0x7f174c15a8c0 unknown :-1 s=CLOSED pgs=2919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f17515c9640 1 -- 172.21.15.70:0/3131327132 >> 172.21.15.70:0/3131327132 conn(0x7f174c0662a0 msgr2=0x7f174c069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:55.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f17515c9640 1 -- 172.21.15.70:0/3131327132 shutdown_connections 2023-10-08T07:15:55.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.442+0000 7f17515c9640 1 -- 172.21.15.70:0/3131327132 wait complete. 2023-10-08T07:15:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.444+0000 7f17515c9640 1 Processor -- start 2023-10-08T07:15:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.444+0000 7f17515c9640 1 -- start start 2023-10-08T07:15:55.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.445+0000 7f17515c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c06d6f0 0x7f174c1e5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:55.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.445+0000 7f17515c9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f174c1573c0 0x7f174c154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:55.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.445+0000 7f17515c9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f174c15c2f0 0x7f174c154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:55.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.445+0000 7f17515c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c0ae4f0 con 0x7f174c06d6f0 2023-10-08T07:15:55.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.445+0000 7f17515c9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174c0ad450 con 0x7f174c1573c0 2023-10-08T07:15:55.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.445+0000 7f17515c9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174c0ad730 con 0x7f174c15c2f0 2023-10-08T07:15:55.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f174affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c06d6f0 0x7f174c1e5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:55.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f174affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c06d6f0 0x7f174c1e5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51546/0 (socket says 172.21.15.70:51546) 2023-10-08T07:15:55.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f174affd640 1 -- 172.21.15.70:0/3018600671 learned_addr learned my addr 172.21.15.70:0/3018600671 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:55.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f17427fc640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f174c1573c0 0x7f174c154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:55.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f174b7fe640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f174c15c2f0 0x7f174c154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:55.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f174affd640 1 -- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f174c15c2f0 msgr2=0x7f174c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:55.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f174affd640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f174c15c2f0 0x7f174c154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f174affd640 1 -- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f174c1573c0 msgr2=0x7f174c154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:55.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f174affd640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f174c1573c0 0x7f174c154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:55.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f174affd640 1 -- 172.21.15.70:0/3018600671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c0094d0 con 0x7f174c06d6f0 2023-10-08T07:15:55.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f17427fc640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f174c1573c0 0x7f174c154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:55.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.446+0000 7f174affd640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c06d6f0 0x7f174c1e5670 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f173c009870 tx=0x7f173c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:55.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.447+0000 7f1743fff640 1 -- 172.21.15.70:0/3018600671 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c03ac60 con 0x7f174c06d6f0 2023-10-08T07:15:55.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.447+0000 7f1743fff640 1 -- 172.21.15.70:0/3018600671 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c03ade0 con 0x7f174c06d6f0 2023-10-08T07:15:55.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.447+0000 7f1743fff640 1 -- 172.21.15.70:0/3018600671 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c044630 con 0x7f174c06d6f0 2023-10-08T07:15:55.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.447+0000 7f17515c9640 1 -- 172.21.15.70:0/3018600671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174c0ada10 con 0x7f174c06d6f0 2023-10-08T07:15:55.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.447+0000 7f17515c9640 1 -- 172.21.15.70:0/3018600671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174c0adfd0 con 0x7f174c06d6f0 2023-10-08T07:15:55.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.448+0000 7f17515c9640 1 -- 172.21.15.70:0/3018600671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174c0afd00 con 0x7f174c06d6f0 2023-10-08T07:15:55.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.449+0000 7f1743fff640 1 -- 172.21.15.70:0/3018600671 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 146) v1 ==== 89433+0+0 (secure 0 0 0) 0x7f173c02f050 con 0x7f174c06d6f0 2023-10-08T07:15:55.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.451+0000 7f1743fff640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f173006b0b0 0x7f173006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:55.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.451+0000 7f1743fff640 1 -- 172.21.15.70:0/3018600671 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f173c0abdf0 con 0x7f174c06d6f0 2023-10-08T07:15:55.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.451+0000 7f17427fc640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f173006b0b0 0x7f173006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:55.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.452+0000 7f17427fc640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f173006b0b0 0x7f173006d570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f174c155ae0 tx=0x7f1734009040 comp rx=0 tx=0).ready entity=mgr.9067 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.455+0000 7f1743fff640 1 -- 172.21.15.70:0/3018600671 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f173c076910 con 0x7f174c06d6f0 2023-10-08T07:15:55.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:55.697+0000 7f17515c9640 1 -- 172.21.15.70:0/3018600671 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f174c15dd20 con 0x7f174c06d6f0 2023-10-08T07:15:56.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.495+0000 7f1743fff640 1 -- 172.21.15.70:0/3018600671 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v147) v1 ==== 92+0+0 (secure 0 0 0) 0x7f173c079f30 con 0x7f174c06d6f0 2023-10-08T07:15:56.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.499+0000 7f17515c9640 1 -- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f173006b0b0 msgr2=0x7f173006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:56.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.499+0000 7f17515c9640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f173006b0b0 0x7f173006d570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f174c155ae0 tx=0x7f1734009040 comp rx=0 tx=0).stop 2023-10-08T07:15:56.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.499+0000 7f17515c9640 1 -- 172.21.15.70:0/3018600671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c06d6f0 msgr2=0x7f174c1e5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:56.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.499+0000 7f17515c9640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c06d6f0 0x7f174c1e5670 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f173c009870 tx=0x7f173c0046c0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.500+0000 7f17515c9640 1 -- 172.21.15.70:0/3018600671 shutdown_connections 2023-10-08T07:15:56.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.500+0000 7f17515c9640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f173006b0b0 0x7f173006d570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.500+0000 7f17515c9640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f174c15c2f0 0x7f174c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.500+0000 7f17515c9640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f174c1573c0 0x7f174c154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.500+0000 7f17515c9640 1 --2- 172.21.15.70:0/3018600671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174c06d6f0 0x7f174c1e5670 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.500+0000 7f17515c9640 1 -- 172.21.15.70:0/3018600671 >> 172.21.15.70:0/3018600671 conn(0x7f174c0662a0 msgr2=0x7f174c0afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:56.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.500+0000 7f17515c9640 1 -- 172.21.15.70:0/3018600671 shutdown_connections 2023-10-08T07:15:56.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.500+0000 7f17515c9640 1 -- 172.21.15.70:0/3018600671 wait complete. 2023-10-08T07:15:56.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T07:15:56.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.719+0000 7f3b6e139640 1 Processor -- start 2023-10-08T07:15:56.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.719+0000 7f3b6e139640 1 -- start start 2023-10-08T07:15:56.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.720+0000 7f3b6e139640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3b6815cdb0 0x7f3b680cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:56.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b6e139640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3b6815d190 0x7f3b6806af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:56.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b6e139640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b680ccb40 0x7f3b680ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:56.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b6e139640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6815d620 con 0x7f3b6815d190 2023-10-08T07:15:56.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b6e139640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b680cd490 con 0x7f3b6815cdb0 2023-10-08T07:15:56.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b6e139640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b680b19e0 con 0x7f3b680ccb40 2023-10-08T07:15:56.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b5ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3b6815d190 0x7f3b6806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:56.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b5ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3b6815d190 0x7f3b6806af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51570/0 (socket says 172.21.15.70:51570) 2023-10-08T07:15:56.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b5ffff640 1 -- 172.21.15.70:0/1894060798 learned_addr learned my addr 172.21.15.70:0/1894060798 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:56.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b67fff640 1 --2- 172.21.15.70:0/1894060798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b680ccb40 0x7f3b680ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:56.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b5ffff640 1 -- 172.21.15.70:0/1894060798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b680ccb40 msgr2=0x7f3b680ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:56.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b677fe640 1 --2- 172.21.15.70:0/1894060798 >> v2:172.21.15.181:3300/0 conn(0x7f3b6815cdb0 0x7f3b680cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:56.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b5ffff640 1 --2- 172.21.15.70:0/1894060798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b680ccb40 0x7f3b680ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b5ffff640 1 -- 172.21.15.70:0/1894060798 >> v2:172.21.15.181:3300/0 conn(0x7f3b6815cdb0 msgr2=0x7f3b680cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:56.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b5ffff640 1 --2- 172.21.15.70:0/1894060798 >> v2:172.21.15.181:3300/0 conn(0x7f3b6815cdb0 0x7f3b680cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b5ffff640 1 -- 172.21.15.70:0/1894060798 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b68069ef0 con 0x7f3b6815d190 2023-10-08T07:15:56.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.721+0000 7f3b677fe640 1 --2- 172.21.15.70:0/1894060798 >> v2:172.21.15.181:3300/0 conn(0x7f3b6815cdb0 0x7f3b680cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:15:56.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.723+0000 7f3b5ffff640 1 --2- 172.21.15.70:0/1894060798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 0x7f3b6806af90 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f3b6809d060 tx=0x7f3b5002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39abd7e31b2690b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:56.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.723+0000 7f3b66ffd640 1 -- 172.21.15.70:0/1894060798 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b50031870 con 0x7f3b6815d190 2023-10-08T07:15:56.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.723+0000 7f3b66ffd640 1 -- 172.21.15.70:0/1894060798 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b500319d0 con 0x7f3b6815d190 2023-10-08T07:15:56.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.723+0000 7f3b66ffd640 1 -- 172.21.15.70:0/1894060798 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b50031d60 con 0x7f3b6815d190 2023-10-08T07:15:56.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.724+0000 7f3b6e139640 1 -- 172.21.15.70:0/1894060798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 msgr2=0x7f3b6806af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:56.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.724+0000 7f3b6e139640 1 --2- 172.21.15.70:0/1894060798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 0x7f3b6806af90 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f3b6809d060 tx=0x7f3b5002b910 comp rx=0 tx=0).stop 2023-10-08T07:15:56.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.724+0000 7f3b6e139640 1 -- 172.21.15.70:0/1894060798 shutdown_connections 2023-10-08T07:15:56.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.724+0000 7f3b6e139640 1 --2- 172.21.15.70:0/1894060798 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b680ccb40 0x7f3b680ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.724+0000 7f3b6e139640 1 --2- 172.21.15.70:0/1894060798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 0x7f3b6806af90 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.724+0000 7f3b6e139640 1 --2- 172.21.15.70:0/1894060798 >> v2:172.21.15.181:3300/0 conn(0x7f3b6815cdb0 0x7f3b680cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.724+0000 7f3b6e139640 1 -- 172.21.15.70:0/1894060798 >> 172.21.15.70:0/1894060798 conn(0x7f3b680b0af0 msgr2=0x7f3b680b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:56.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.724+0000 7f3b6e139640 1 -- 172.21.15.70:0/1894060798 shutdown_connections 2023-10-08T07:15:56.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.724+0000 7f3b6e139640 1 -- 172.21.15.70:0/1894060798 wait complete. 2023-10-08T07:15:56.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.726+0000 7f3b6e139640 1 Processor -- start 2023-10-08T07:15:56.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.726+0000 7f3b6e139640 1 -- start start 2023-10-08T07:15:56.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b6e139640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b680ccb40 0x7f3b681598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:56.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b6e139640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b6815cdb0 0x7f3b6815f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:56.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b6e139640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 0x7f3b6815fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:56.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b6e139640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b681f6970 con 0x7f3b6815d190 2023-10-08T07:15:56.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b6e139640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b681f3d20 con 0x7f3b6815cdb0 2023-10-08T07:15:56.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b6e139640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b681f4000 con 0x7f3b680ccb40 2023-10-08T07:15:56.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 0x7f3b6815fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:56.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 0x7f3b6815fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51578/0 (socket says 172.21.15.70:51578) 2023-10-08T07:15:56.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b67fff640 1 -- 172.21.15.70:0/3813102503 learned_addr learned my addr 172.21.15.70:0/3813102503 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:56.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b5ffff640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b6815cdb0 0x7f3b6815f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:56.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.727+0000 7f3b677fe640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b680ccb40 0x7f3b681598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:56.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b67fff640 1 -- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b680ccb40 msgr2=0x7f3b681598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:56.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b67fff640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b680ccb40 0x7f3b681598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b67fff640 1 -- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b6815cdb0 msgr2=0x7f3b6815f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:56.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b67fff640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b6815cdb0 0x7f3b6815f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b67fff640 1 -- 172.21.15.70:0/3813102503 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b500094d0 con 0x7f3b6815d190 2023-10-08T07:15:56.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b67fff640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 0x7f3b6815fa10 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000cb80 tx=0x7f3b6000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:56.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b64ff9640 1 -- 172.21.15.70:0/3813102503 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000ce70 con 0x7f3b6815d190 2023-10-08T07:15:56.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b64ff9640 1 -- 172.21.15.70:0/3813102503 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6000abd0 con 0x7f3b6815d190 2023-10-08T07:15:56.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b64ff9640 1 -- 172.21.15.70:0/3813102503 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b60011410 con 0x7f3b6815d190 2023-10-08T07:15:56.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b6e139640 1 -- 172.21.15.70:0/3813102503 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b681f42e0 con 0x7f3b6815d190 2023-10-08T07:15:56.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.728+0000 7f3b6e139640 1 -- 172.21.15.70:0/3813102503 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b681f5e60 con 0x7f3b6815d190 2023-10-08T07:15:56.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.729+0000 7f3b6e139640 1 -- 172.21.15.70:0/3813102503 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b681f3b80 con 0x7f3b6815d190 2023-10-08T07:15:56.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.731+0000 7f3b64ff9640 1 -- 172.21.15.70:0/3813102503 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 147) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f3b60011570 con 0x7f3b6815d190 2023-10-08T07:15:56.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.731+0000 7f3b64ff9640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f3b4806b100 0x7f3b4806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:56.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.732+0000 7f3b64ff9640 1 -- 172.21.15.70:0/3813102503 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f3b60089bb0 con 0x7f3b6815d190 2023-10-08T07:15:56.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.732+0000 7f3b677fe640 1 -- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f3b4806b100 msgr2=0x7f3b4806d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/440573135 2023-10-08T07:15:56.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.732+0000 7f3b677fe640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f3b4806b100 0x7f3b4806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:56.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.740+0000 7f3b64ff9640 1 -- 172.21.15.70:0/3813102503 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b681f3b80 con 0x7f3b6815d190 2023-10-08T07:15:56.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.931+0000 7f3b6e139640 1 -- 172.21.15.70:0/3813102503 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f3b681f3b80 con 0x7f3b6815d190 2023-10-08T07:15:56.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.933+0000 7f3b677fe640 1 -- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f3b4806b100 msgr2=0x7f3b4806d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/440573135 2023-10-08T07:15:56.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.933+0000 7f3b677fe640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f3b4806b100 0x7f3b4806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:15:56.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.936+0000 7f3b64ff9640 1 -- 172.21.15.70:0/3813102503 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v89)=0 v89) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3b681f3b80 con 0x7f3b6815d190 2023-10-08T07:15:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 -- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f3b4806b100 msgr2=0x7f3b4806d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f3b4806b100 0x7f3b4806d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 -- 172.21.15.70:0/3813102503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 msgr2=0x7f3b6815fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 0x7f3b6815fa10 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000cb80 tx=0x7f3b6000a4c0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 -- 172.21.15.70:0/3813102503 shutdown_connections 2023-10-08T07:15:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f3b4806b100 0x7f3b4806d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b6815d190 0x7f3b6815fa10 unknown :-1 s=CLOSED pgs=2924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3b6815cdb0 0x7f3b6815f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 --2- 172.21.15.70:0/3813102503 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3b680ccb40 0x7f3b681598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 -- 172.21.15.70:0/3813102503 >> 172.21.15.70:0/3813102503 conn(0x7f3b680b0af0 msgr2=0x7f3b680aca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 -- 172.21.15.70:0/3813102503 shutdown_connections 2023-10-08T07:15:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:56.939+0000 7f3b6e139640 1 -- 172.21.15.70:0/3813102503 wait complete. 2023-10-08T07:15:56.959 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T07:15:57.016 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:57.015+0000 7f8e60841040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:15:57.037 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:57.036+0000 7fbcff84a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:15:57.068 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:57.067+0000 7f8e60841040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:15:57.093 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:57.091+0000 7fbcff84a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:15:57.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.162+0000 7fae06165640 1 Processor -- start 2023-10-08T07:15:57.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.163+0000 7fae06165640 1 -- start start 2023-10-08T07:15:57.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.164+0000 7fae06165640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 0x7fae0015d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.164+0000 7fae06165640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0006b6a0 con 0x7fae000cea30 2023-10-08T07:15:57.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.164+0000 7fae06165640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae000b19e0 con 0x7fae0015cdb0 2023-10-08T07:15:57.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.164+0000 7fae06165640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae00069ef0 con 0x7fae000cd250 2023-10-08T07:15:57.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.165+0000 7fadfeffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fae000cea30 0x7fae000ccb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40626/0 (socket says 172.21.15.70:40626) 2023-10-08T07:15:57.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.165+0000 7fadfeffd640 1 -- 172.21.15.70:0/295917906 learned_addr learned my addr 172.21.15.70:0/295917906 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:57.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.165+0000 7fadfffff640 1 --2- 172.21.15.70:0/295917906 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 0x7fae0015d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:57.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.165+0000 7fadfe7fc640 1 -- 172.21.15.70:0/295917906 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448775787 0 0) 0x7fae0006b6a0 con 0x7fae000cea30 2023-10-08T07:15:57.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.165+0000 7fadfe7fc640 1 -- 172.21.15.70:0/295917906 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade8003520 con 0x7fae000cea30 2023-10-08T07:15:57.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.165+0000 7fadfffff640 1 -- 172.21.15.70:0/295917906 >> v1:172.21.15.181:6789/0 conn(0x7fae0015cdb0 legacy=0x7fae0006af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:57.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.165+0000 7fadfffff640 1 -- 172.21.15.70:0/295917906 >> v1:172.21.15.70:6789/0 conn(0x7fae000cea30 legacy=0x7fae000ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:57.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.165+0000 7fadfffff640 1 -- 172.21.15.70:0/295917906 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0006a1d0 con 0x7fae000cd250 2023-10-08T07:15:57.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.166+0000 7fadfffff640 1 --2- 172.21.15.70:0/295917906 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 0x7fae0015d190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009dc0 tx=0x7fadf802bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15e1ad5f4ef45434 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:57.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.167+0000 7fadfe7fc640 1 -- 172.21.15.70:0/295917906 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf802ccd0 con 0x7fae000cd250 2023-10-08T07:15:57.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.167+0000 7fadfe7fc640 1 -- 172.21.15.70:0/295917906 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf802ce30 con 0x7fae000cd250 2023-10-08T07:15:57.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.167+0000 7fadfe7fc640 1 -- 172.21.15.70:0/295917906 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8034640 con 0x7fae000cd250 2023-10-08T07:15:57.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.168+0000 7fae06165640 1 -- 172.21.15.70:0/295917906 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 msgr2=0x7fae0015d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:57.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.168+0000 7fae06165640 1 --2- 172.21.15.70:0/295917906 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 0x7fae0015d190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009dc0 tx=0x7fadf802bf20 comp rx=0 tx=0).stop 2023-10-08T07:15:57.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.168+0000 7fae06165640 1 -- 172.21.15.70:0/295917906 shutdown_connections 2023-10-08T07:15:57.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.168+0000 7fae06165640 1 --2- 172.21.15.70:0/295917906 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 0x7fae0015d190 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.168+0000 7fae06165640 1 -- 172.21.15.70:0/295917906 >> 172.21.15.70:0/295917906 conn(0x7fae000b0af0 msgr2=0x7fae000b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:57.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.168+0000 7fae06165640 1 -- 172.21.15.70:0/295917906 shutdown_connections 2023-10-08T07:15:57.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.168+0000 7fae06165640 1 -- 172.21.15.70:0/295917906 wait complete. 2023-10-08T07:15:57.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.170+0000 7fae06165640 1 Processor -- start 2023-10-08T07:15:57.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.170+0000 7fae06165640 1 -- start start 2023-10-08T07:15:57.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.171+0000 7fae06165640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 0x7fae0015e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.171+0000 7fae06165640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae000cea30 0x7fae00153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.171+0000 7fae06165640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae0015cdb0 0x7fae001587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.171+0000 7fae06165640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015e9c0 con 0x7fae0015cdb0 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.171+0000 7fae06165640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015ebf0 con 0x7fae000cea30 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.171+0000 7fae06165640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015eed0 con 0x7fae000cd250 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.171+0000 7fadfffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae0015cdb0 0x7fae001587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.171+0000 7fadfffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae0015cdb0 0x7fae001587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51588/0 (socket says 172.21.15.70:51588) 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.171+0000 7fadfffff640 1 -- 172.21.15.70:0/3043354345 learned_addr learned my addr 172.21.15.70:0/3043354345 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.171+0000 7fadff7fe640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 0x7fae0015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fadfeffd640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae000cea30 0x7fae00153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fadfffff640 1 -- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 msgr2=0x7fae0015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:57.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fadfffff640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 0x7fae0015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fadfffff640 1 -- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae000cea30 msgr2=0x7fae00153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:57.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fadfffff640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae000cea30 0x7fae00153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fadfffff640 1 -- 172.21.15.70:0/3043354345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf8009a70 con 0x7fae0015cdb0 2023-10-08T07:15:57.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fadfffff640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae0015cdb0 0x7fae001587a0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009ee0 tx=0x7fadf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:57.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fade7fff640 1 -- 172.21.15.70:0/3043354345 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8004310 con 0x7fae0015cdb0 2023-10-08T07:15:57.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fade7fff640 1 -- 172.21.15.70:0/3043354345 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf8004470 con 0x7fae0015cdb0 2023-10-08T07:15:57.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fade7fff640 1 -- 172.21.15.70:0/3043354345 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf80047e0 con 0x7fae0015cdb0 2023-10-08T07:15:57.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.172+0000 7fae06165640 1 -- 172.21.15.70:0/3043354345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0015f1b0 con 0x7fae0015cdb0 2023-10-08T07:15:57.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.173+0000 7fae06165640 1 -- 172.21.15.70:0/3043354345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0015f770 con 0x7fae0015cdb0 2023-10-08T07:15:57.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.174+0000 7fae06165640 1 -- 172.21.15.70:0/3043354345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0004e160 con 0x7fae0015cdb0 2023-10-08T07:15:57.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.175+0000 7fade7fff640 1 -- 172.21.15.70:0/3043354345 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 147) v1 ==== 89450+0+0 (secure 0 0 0) 0x7fadf803b070 con 0x7fae0015cdb0 2023-10-08T07:15:57.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.176+0000 7fade7fff640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fadec06b100 0x7fadec06d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.176+0000 7fade7fff640 1 -- 172.21.15.70:0/3043354345 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fadf80ac4f0 con 0x7fae0015cdb0 2023-10-08T07:15:57.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.176+0000 7fadff7fe640 1 -- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fadec06b100 msgr2=0x7fadec06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/440573135 2023-10-08T07:15:57.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.176+0000 7fadff7fe640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fadec06b100 0x7fadec06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:57.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.180+0000 7fade7fff640 1 -- 172.21.15.70:0/3043354345 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fadf80771b0 con 0x7fae0015cdb0 2023-10-08T07:15:57.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.370+0000 7fae06165640 1 -- 172.21.15.70:0/3043354345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fae0006a480 con 0x7fae0015cdb0 2023-10-08T07:15:57.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.377+0000 7fadff7fe640 1 -- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fadec06b100 msgr2=0x7fadec06d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/440573135 2023-10-08T07:15:57.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.377+0000 7fadff7fe640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fadec06b100 0x7fadec06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:15:57.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.377+0000 7fade7fff640 1 -- 172.21.15.70:0/3043354345 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v90)=0 v90) v1 ==== 133+0+0 (secure 0 0 0) 0x7fadf807a7d0 con 0x7fae0015cdb0 2023-10-08T07:15:57.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.379+0000 7fae06165640 1 -- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fadec06b100 msgr2=0x7fadec06d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:15:57.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.379+0000 7fae06165640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fadec06b100 0x7fadec06d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.379+0000 7fae06165640 1 -- 172.21.15.70:0/3043354345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae0015cdb0 msgr2=0x7fae001587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:57.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.379+0000 7fae06165640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae0015cdb0 0x7fae001587a0 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009ee0 tx=0x7fadf8004a40 comp rx=0 tx=0).stop 2023-10-08T07:15:57.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.380+0000 7fae06165640 1 -- 172.21.15.70:0/3043354345 shutdown_connections 2023-10-08T07:15:57.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.380+0000 7fae06165640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7fadec06b100 0x7fadec06d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.380+0000 7fae06165640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae0015cdb0 0x7fae001587a0 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.380+0000 7fae06165640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fae000cea30 0x7fae00153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.380+0000 7fae06165640 1 --2- 172.21.15.70:0/3043354345 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fae000cd250 0x7fae0015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.380+0000 7fae06165640 1 -- 172.21.15.70:0/3043354345 >> 172.21.15.70:0/3043354345 conn(0x7fae000b0af0 msgr2=0x7fae000ac450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:57.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.380+0000 7fae06165640 1 -- 172.21.15.70:0/3043354345 shutdown_connections 2023-10-08T07:15:57.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.380+0000 7fae06165640 1 -- 172.21.15.70:0/3043354345 wait complete. 2023-10-08T07:15:57.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:15:57.399 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T07:15:57.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2023-10-08T07:15:57.515 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:57.514+0000 7f8e60841040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:15:57.563 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:57.561+0000 7fbcff84a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:15:57.566 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:57.565+0000 7f8e60841040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:15:57.617 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:57.615+0000 7f8e60841040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:15:57.617 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:57.616+0000 7fbcff84a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:15:57.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.666+0000 7f3218656640 1 Processor -- start 2023-10-08T07:15:57.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.666+0000 7f3218656640 1 -- start start 2023-10-08T07:15:57.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.668+0000 7f3218656640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f321015ce00 0x7f32100cb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.668+0000 7f3218656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3210161090 0x7f32100cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.668+0000 7f3218656640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32100cde10 con 0x7f32100cd470 2023-10-08T07:15:57.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.668+0000 7f3218656640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3210161920 con 0x7f321015ce00 2023-10-08T07:15:57.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.668+0000 7f3218656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32100cb8b0 con 0x7f3210161090 2023-10-08T07:15:57.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.668+0000 7f32163f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f321015ce00 0x7f32100cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:57.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.668+0000 7f3215bf1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f32100cd470 0x7f3210160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40642/0 (socket says 172.21.15.70:40642) 2023-10-08T07:15:57.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.668+0000 7f3215bf1640 1 -- 172.21.15.70:0/3807725227 learned_addr learned my addr 172.21.15.70:0/3807725227 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:15:57.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.668+0000 7f3216bf3640 1 --2- 172.21.15.70:0/3807725227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3210161090 0x7f32100cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:57.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.669+0000 7f32163f2640 1 -- 172.21.15.70:0/3807725227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3210161090 msgr2=0x7f32100cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:15:57.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.669+0000 7f32163f2640 1 --2- 172.21.15.70:0/3807725227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3210161090 0x7f32100cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.669+0000 7f32163f2640 1 -- 172.21.15.70:0/3807725227 >> v1:172.21.15.70:6789/0 conn(0x7f32100cd470 legacy=0x7f3210160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:57.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.669+0000 7f32163f2640 1 -- 172.21.15.70:0/3807725227 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3210161640 con 0x7f321015ce00 2023-10-08T07:15:57.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.669+0000 7f32153f0640 1 -- 172.21.15.70:0/3807725227 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286338321 0 0) 0x7f32100cde10 con 0x7f32100cd470 2023-10-08T07:15:57.672 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:57.669+0000 7fbcff84a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:15:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.670+0000 7f32163f2640 1 --2- 172.21.15.70:0/3807725227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321015ce00 0x7f32100cb370 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f320c009790 tx=0x7f320c02b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7134873f1a661193 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.670+0000 7f32153f0640 1 -- 172.21.15.70:0/3807725227 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c031770 con 0x7f321015ce00 2023-10-08T07:15:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.670+0000 7f32153f0640 1 -- 172.21.15.70:0/3807725227 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c0318d0 con 0x7f321015ce00 2023-10-08T07:15:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.670+0000 7f32153f0640 1 -- 172.21.15.70:0/3807725227 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c031c60 con 0x7f321015ce00 2023-10-08T07:15:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.671+0000 7f3218656640 1 -- 172.21.15.70:0/3807725227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321015ce00 msgr2=0x7f32100cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.671+0000 7f3218656640 1 --2- 172.21.15.70:0/3807725227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321015ce00 0x7f32100cb370 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7f320c009790 tx=0x7f320c02b810 comp rx=0 tx=0).stop 2023-10-08T07:15:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.671+0000 7f3218656640 1 -- 172.21.15.70:0/3807725227 shutdown_connections 2023-10-08T07:15:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.671+0000 7f3218656640 1 --2- 172.21.15.70:0/3807725227 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3210161090 0x7f32100cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.671+0000 7f3218656640 1 --2- 172.21.15.70:0/3807725227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321015ce00 0x7f32100cb370 unknown :-1 s=CLOSED pgs=2928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.671+0000 7f3218656640 1 -- 172.21.15.70:0/3807725227 >> 172.21.15.70:0/3807725227 conn(0x7f321006e650 msgr2=0x7f321006ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:15:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.671+0000 7f3218656640 1 -- 172.21.15.70:0/3807725227 shutdown_connections 2023-10-08T07:15:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.671+0000 7f3218656640 1 -- 172.21.15.70:0/3807725227 wait complete. 2023-10-08T07:15:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.673+0000 7f3218656640 1 Processor -- start 2023-10-08T07:15:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.673+0000 7f3218656640 1 -- start start 2023-10-08T07:15:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f3218656640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32100cd470 0x7f32100b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f3218656640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321015ce00 0x7f32101f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f3218656640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3210161090 0x7f32100c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f3218656640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100b3bc0 con 0x7f32100cd470 2023-10-08T07:15:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f3218656640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100b51a0 con 0x7f321015ce00 2023-10-08T07:15:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f3218656640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32101f5cb0 con 0x7f3210161090 2023-10-08T07:15:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f32163f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32100cd470 0x7f32100b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:15:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f32163f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32100cd470 0x7f32100b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51596/0 (socket says 172.21.15.70:51596) 2023-10-08T07:15:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f32163f2640 1 -- 172.21.15.70:0/3577348036 learned_addr learned my addr 172.21.15.70:0/3577348036 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:15:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f32163f2640 1 -- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3210161090 msgr2=0x7f32100c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f32163f2640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f3210161090 0x7f32100c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f32163f2640 1 -- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321015ce00 msgr2=0x7f32101f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:15:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f32163f2640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f321015ce00 0x7f32101f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:15:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.674+0000 7f32163f2640 1 -- 172.21.15.70:0/3577348036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c0093a0 con 0x7f32100cd470 2023-10-08T07:15:57.678 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:57.676+0000 7f8e60841040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:15:57.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.675+0000 7f32163f2640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32100cd470 0x7f32100b3680 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f320c000c00 tx=0x7f320c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:15:57.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.675+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c004230 con 0x7f32100cd470 2023-10-08T07:15:57.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.675+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c004390 con 0x7f32100cd470 2023-10-08T07:15:57.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.675+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c005bd0 con 0x7f32100cd470 2023-10-08T07:15:57.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.675+0000 7f3218656640 1 -- 172.21.15.70:0/3577348036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32101f5f90 con 0x7f32100cd470 2023-10-08T07:15:57.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.675+0000 7f3218656640 1 -- 172.21.15.70:0/3577348036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32101f6550 con 0x7f32100cd470 2023-10-08T07:15:57.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.676+0000 7f3218656640 1 -- 172.21.15.70:0/3577348036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321006cff0 con 0x7f32100cd470 2023-10-08T07:15:57.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.677+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 147) v1 ==== 89450+0+0 (secure 0 0 0) 0x7f320c003610 con 0x7f32100cd470 2023-10-08T07:15:57.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.678+0000 7f3206ffd640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 0x7f31f406d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:15:57.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.678+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(276..276 src has 1..276) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f320c0abd70 con 0x7f32100cd470 2023-10-08T07:15:57.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.678+0000 7f3215bf1640 1 -- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 msgr2=0x7f31f406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/440573135 2023-10-08T07:15:57.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.678+0000 7f3215bf1640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 0x7f31f406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:15:57.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.682+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f320c076850 con 0x7f32100cd470 2023-10-08T07:15:57.725 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:57.724+0000 7f8e60841040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:15:57.735 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:57.733+0000 7fbcff84a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:15:57.785 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:57.784+0000 7fbcff84a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:15:57.836 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:57.835+0000 7f8e60841040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:15:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.879+0000 7f3215bf1640 1 -- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 msgr2=0x7f31f406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/440573135 2023-10-08T07:15:57.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.879+0000 7f3215bf1640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 0x7f31f406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:15:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:57.893+0000 7f3218656640 1 -- 172.21.15.70:0/3577348036 --> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f321006b690 con 0x7f31f406b100 2023-10-08T07:15:57.897 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:57.895+0000 7fbcff84a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:15:57.897 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:57.896+0000 7f8e60841040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:57.946 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:57.945+0000 7f8e60841040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:15:57.958 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:57.957+0000 7fbcff84a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:58.002 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:58.001+0000 7f8e60841040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:15:58.009 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:58.007+0000 7fbcff84a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:15:58.052 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:58.051+0000 7f8e60841040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:15:58.064 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:58.063+0000 7fbcff84a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:15:58.114 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:58.112+0000 7fbcff84a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:15:58.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:58.280+0000 7f3215bf1640 1 -- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 msgr2=0x7f31f406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/440573135 2023-10-08T07:15:58.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:58.280+0000 7f3215bf1640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 0x7f31f406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T07:15:58.429 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:58.428+0000 7f8e60841040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:15:58.493 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:58.491+0000 7fbcff84a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:15:58.586 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:58.584+0000 7f8e60841040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:15:58.650 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:58.648+0000 7fbcff84a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:15:58.692 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:58.691+0000 7f8e60841040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:15:58.741 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:58.740+0000 7f8e60841040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:15:58.756 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:58.754+0000 7fbcff84a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:15:58.805 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:58.803+0000 7fbcff84a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:15:58.847 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:58.846+0000 7f8e60841040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:58.912 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:58.911+0000 7fbcff84a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:58.953 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:58.952+0000 7f8e60841040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:15:59.018 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:59.016+0000 7fbcff84a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:15:59.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:59.081+0000 7f3215bf1640 1 -- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 msgr2=0x7f31f406d5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/440573135 2023-10-08T07:15:59.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:15:59.081+0000 7f3215bf1640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 0x7f31f406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T07:15:59.108 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:59.106+0000 7f8e60841040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:59.156 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:59.155+0000 7f8e60841040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:59.173 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:59.171+0000 7fbcff84a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:15:59.222 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:59.220+0000 7fbcff84a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:15:59.273 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:15:59.272+0000 7f8e60841040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:15:59.340 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:15:59.338+0000 7fbcff84a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:16:00.104 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.103+0000 7f8e60841040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:16:00.173 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:00.171+0000 7fbcff84a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:16:00.397 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.396+0000 7f8e60841040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:16:00.454 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.453+0000 7f8e60841040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:16:00.465 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:00.463+0000 7fbcff84a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:16:00.521 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:00.519+0000 7fbcff84a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:16:00.566 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.565+0000 7f8e60841040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:16:00.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:00.588+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 148) v1 ==== 88957+0+0 (secure 0 0 0) 0x7f320c076c20 con 0x7f32100cd470 2023-10-08T07:16:00.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:00.589+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 msgr2=0x7f31f406d5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:16:00.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:00.589+0000 7f3206ffd640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/440573135,v1:172.21.15.181:6833/440573135] conn(0x7f31f406b100 0x7f31f406d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:00.652 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:00.650+0000 7fbcff84a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:16:00.814 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.812+0000 7f8de03af640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:00.814 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.812+0000 7f8de03af640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:00.814 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.812+0000 7f8de03af640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:00.814 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.812+0000 7f8de03af640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:00.815 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.812+0000 7f8de03af640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:00.815 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.812+0000 7f8ddd369640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:00.815 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.812+0000 7f8ddd369640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:00.815 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.812+0000 7f8ddd369640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:00.815 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.812+0000 7f8ddd369640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:00.815 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:00.812+0000 7f8ddd369640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:01.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.650+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mgrmap(e 149) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f320c08e260 con 0x7f32100cd470 2023-10-08T07:16:01.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.651+0000 7f3206ffd640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f31f4035800 0x7f31f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:01.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.652+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 --> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f31f40283b0 con 0x7f31f4035800 2023-10-08T07:16:01.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.660+0000 7f3215bf1640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f31f4035800 0x7f31f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:01.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.669+0000 7f3215bf1640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f31f4035800 0x7f31f4075710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32100c59b0 tx=0x7f3200007a20 comp rx=0 tx=0).ready entity=mgr.9355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:01.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.826+0000 7f3206ffd640 1 -- 172.21.15.70:0/3577348036 <== mgr.9355 v2:172.21.15.181:6832/1127370729 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f31f40283b0 con 0x7f31f4035800 2023-10-08T07:16:01.828 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule set for path snap_test_dir1 2023-10-08T07:16:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.830+0000 7f3218656640 1 -- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f31f4035800 msgr2=0x7f31f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.830+0000 7f3218656640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f31f4035800 0x7f31f4075710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32100c59b0 tx=0x7f3200007a20 comp rx=0 tx=0).stop 2023-10-08T07:16:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.830+0000 7f3218656640 1 -- 172.21.15.70:0/3577348036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32100cd470 msgr2=0x7f32100b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.830+0000 7f3218656640 1 --2- 172.21.15.70:0/3577348036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32100cd470 0x7f32100b3680 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7f320c000c00 tx=0x7f320c0046c0 comp rx=0 tx=0).stop 2023-10-08T07:16:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.830+0000 7f3216bf3640 1 -- 172.21.15.70:0/3577348036 reap_dead start 2023-10-08T07:16:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.830+0000 7f3218656640 1 -- 172.21.15.70:0/3577348036 shutdown_connections 2023-10-08T07:16:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.830+0000 7f3218656640 1 -- 172.21.15.70:0/3577348036 >> 172.21.15.70:0/3577348036 conn(0x7f321006e650 msgr2=0x7f321015a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.830+0000 7f3218656640 1 -- 172.21.15.70:0/3577348036 shutdown_connections 2023-10-08T07:16:01.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:01.830+0000 7f3218656640 1 -- 172.21.15.70:0/3577348036 wait complete. 2023-10-08T07:16:01.852 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T07:16:01.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2023-10-08T07:16:02.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.056+0000 7f2fea0a2640 1 Processor -- start 2023-10-08T07:16:02.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.056+0000 7f2fea0a2640 1 -- start start 2023-10-08T07:16:02.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.057+0000 7f2fea0a2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2fe40cb620 0x7f2fe40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.057+0000 7f2fea0a2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe4071170 con 0x7f2fe4070cc0 2023-10-08T07:16:02.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.057+0000 7f2fea0a2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe406e150 con 0x7f2fe4160660 2023-10-08T07:16:02.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.057+0000 7f2fea0a2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe406de70 con 0x7f2fe40cb620 2023-10-08T07:16:02.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe2ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2fe4070cc0 0x7f2fe40caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40648/0 (socket says 172.21.15.70:40648) 2023-10-08T07:16:02.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe2ffd640 1 -- 172.21.15.70:0/1563691017 learned_addr learned my addr 172.21.15.70:0/1563691017 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:02.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/1563691017 >> v2:172.21.15.181:3300/0 conn(0x7f2fe40cb620 0x7f2fe40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343597146 0 0) 0x7f2fcc001490 con 0x7f2fe4070cc0 2023-10-08T07:16:02.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd0003520 con 0x7f2fe4070cc0 2023-10-08T07:16:02.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029517530 0 0) 0x7f2fd0003520 con 0x7f2fe4070cc0 2023-10-08T07:16:02.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 >> v2:172.21.15.181:3300/0 conn(0x7f2fe40cb620 msgr2=0x7f2fe40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:02.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 --2- 172.21.15.70:0/1563691017 >> v2:172.21.15.181:3300/0 conn(0x7f2fe40cb620 0x7f2fe40b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 >> v1:172.21.15.181:6789/0 conn(0x7f2fe4160660 legacy=0x7f2fe40c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe40c8e30 con 0x7f2fe4070cc0 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2fcc001b80 con 0x7f2fe4070cc0 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 >> v1:172.21.15.70:6789/0 conn(0x7f2fe4070cc0 legacy=0x7f2fe40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 --2- 172.21.15.70:0/1563691017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fd0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 --2- 172.21.15.70:0/1563691017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00047c0 0x7f2fd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.058+0000 7f2fe27fc640 1 --2- 172.21.15.70:0/1563691017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd00052a0 0x7f2fd000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe406de70 con 0x7f2fd00039c0 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd000c1c0 con 0x7f2fd00052a0 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd000c380 con 0x7f2fd00047c0 2023-10-08T07:16:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe37fe640 1 --2- 172.21.15.70:0/1563691017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fd0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe37fe640 1 -- 172.21.15.70:0/1563691017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00047c0 msgr2=0x7f2fd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe37fe640 1 --2- 172.21.15.70:0/1563691017 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fd00047c0 0x7f2fd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/1563691017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd00052a0 0x7f2fd000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe37fe640 1 -- 172.21.15.70:0/1563691017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd00052a0 msgr2=0x7f2fd000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe37fe640 1 --2- 172.21.15.70:0/1563691017 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fd00052a0 0x7f2fd000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe37fe640 1 -- 172.21.15.70:0/1563691017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd0003520 con 0x7f2fd00039c0 2023-10-08T07:16:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.059+0000 7f2fe3fff640 1 -- 172.21.15.70:0/1563691017 reap_dead start 2023-10-08T07:16:02.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.060+0000 7f2fe37fe640 1 --2- 172.21.15.70:0/1563691017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fd0003de0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4003db0 tx=0x7f2fd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a17b9ac2ee1196 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:02.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.061+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4032600 con 0x7f2fd00039c0 2023-10-08T07:16:02.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.061+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd402e050 con 0x7f2fd00039c0 2023-10-08T07:16:02.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.061+0000 7f2fe27fc640 1 -- 172.21.15.70:0/1563691017 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40329e0 con 0x7f2fd00039c0 2023-10-08T07:16:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.061+0000 7f2fea0a2640 1 -- 172.21.15.70:0/1563691017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 msgr2=0x7f2fd0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.061+0000 7f2fea0a2640 1 --2- 172.21.15.70:0/1563691017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fd0003de0 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4003db0 tx=0x7f2fd00057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.061+0000 7f2fea0a2640 1 -- 172.21.15.70:0/1563691017 shutdown_connections 2023-10-08T07:16:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.061+0000 7f2fea0a2640 1 --2- 172.21.15.70:0/1563691017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fd0003de0 unknown :-1 s=CLOSED pgs=2932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.061+0000 7f2fea0a2640 1 -- 172.21.15.70:0/1563691017 >> 172.21.15.70:0/1563691017 conn(0x7f2fe406a5d0 msgr2=0x7f2fe40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.061+0000 7f2fea0a2640 1 -- 172.21.15.70:0/1563691017 shutdown_connections 2023-10-08T07:16:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.061+0000 7f2fea0a2640 1 -- 172.21.15.70:0/1563691017 wait complete. 2023-10-08T07:16:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.063+0000 7f2fea0a2640 1 Processor -- start 2023-10-08T07:16:02.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.063+0000 7f2fea0a2640 1 -- start start 2023-10-08T07:16:02.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.064+0000 7f2fea0a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fe414ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.064+0000 7f2fea0a2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40c7d20 0x7f2fe40c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.064+0000 7f2fea0a2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fe40c24c0 0x7f2fe41f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.064+0000 7f2fea0a2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40c8690 con 0x7f2fd00039c0 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.064+0000 7f2fea0a2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40c8180 con 0x7f2fe40c7d20 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.064+0000 7f2fea0a2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40c5fc0 con 0x7f2fe40c24c0 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.064+0000 7f2fe37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fe414ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.064+0000 7f2fe37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fe414ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51610/0 (socket says 172.21.15.70:51610) 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.064+0000 7f2fe37fe640 1 -- 172.21.15.70:0/2534314706 learned_addr learned my addr 172.21.15.70:0/2534314706 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fe2ffd640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40c7d20 0x7f2fe40c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fe37fe640 1 -- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fe40c24c0 msgr2=0x7f2fe41f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fe37fe640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fe40c24c0 0x7f2fe41f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fe37fe640 1 -- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40c7d20 msgr2=0x7f2fe40c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fe37fe640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40c7d20 0x7f2fe40c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fe37fe640 1 -- 172.21.15.70:0/2534314706 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe40c59b0 con 0x7f2fd00039c0 2023-10-08T07:16:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fe37fe640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fe414ea60 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4035040 tx=0x7f2fd40036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fdbfff640 1 -- 172.21.15.70:0/2534314706 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd403c610 con 0x7f2fd00039c0 2023-10-08T07:16:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fdbfff640 1 -- 172.21.15.70:0/2534314706 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd403c770 con 0x7f2fd00039c0 2023-10-08T07:16:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fdbfff640 1 -- 172.21.15.70:0/2534314706 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd403cb00 con 0x7f2fd00039c0 2023-10-08T07:16:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.065+0000 7f2fea0a2640 1 -- 172.21.15.70:0/2534314706 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe40c83b0 con 0x7f2fd00039c0 2023-10-08T07:16:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.066+0000 7f2fea0a2640 1 -- 172.21.15.70:0/2534314706 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe40c4bd0 con 0x7f2fd00039c0 2023-10-08T07:16:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.067+0000 7f2fea0a2640 1 -- 172.21.15.70:0/2534314706 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe40c57f0 con 0x7f2fd00039c0 2023-10-08T07:16:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.068+0000 7f2fdbfff640 1 -- 172.21.15.70:0/2534314706 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 149) v1 ==== 89084+0+0 (secure 0 0 0) 0x7f2fd4039050 con 0x7f2fd00039c0 2023-10-08T07:16:02.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.068+0000 7f2fdbfff640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2fc406ad40 0x7f2fc406d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.068+0000 7f2fdbfff640 1 -- 172.21.15.70:0/2534314706 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7f2fd40aadf0 con 0x7f2fd00039c0 2023-10-08T07:16:02.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.069+0000 7f2fe2ffd640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2fc406ad40 0x7f2fc406d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.070+0000 7f2fe2ffd640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2fc406ad40 0x7f2fc406d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc001cb0 tx=0x7f2fcc001a40 comp rx=0 tx=0).ready entity=mgr.9355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:02.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.073+0000 7f2fdbfff640 1 -- 172.21.15.70:0/2534314706 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fd4075d80 con 0x7f2fd00039c0 2023-10-08T07:16:02.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.282+0000 7f2fea0a2640 1 -- 172.21.15.70:0/2534314706 --> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fe40c93f0 con 0x7f2fc406ad40 2023-10-08T07:16:02.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.424+0000 7f2fdbfff640 1 -- 172.21.15.70:0/2534314706 <== mgr.9355 v2:172.21.15.181:6832/1127370729 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f2fcc00ec40 con 0x7f2fc406ad40 2023-10-08T07:16:02.426 INFO:teuthology.orchestra.run.smithi070.stdout:Retention added to path snap_test_dir1 2023-10-08T07:16:02.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 -- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2fc406ad40 msgr2=0x7f2fc406d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2fc406ad40 0x7f2fc406d200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc001cb0 tx=0x7f2fcc001a40 comp rx=0 tx=0).stop 2023-10-08T07:16:02.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 -- 172.21.15.70:0/2534314706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 msgr2=0x7f2fe414ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fe414ea60 secure :-1 s=READY pgs=2933 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4035040 tx=0x7f2fd40036f0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 -- 172.21.15.70:0/2534314706 shutdown_connections 2023-10-08T07:16:02.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2fc406ad40 0x7f2fc406d200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2fe40c24c0 0x7f2fe41f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2fe40c7d20 0x7f2fe40c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 --2- 172.21.15.70:0/2534314706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fd00039c0 0x7f2fe414ea60 unknown :-1 s=CLOSED pgs=2933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 -- 172.21.15.70:0/2534314706 >> 172.21.15.70:0/2534314706 conn(0x7f2fe406a5d0 msgr2=0x7f2fe40c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:02.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.427+0000 7f2fea0a2640 1 -- 172.21.15.70:0/2534314706 shutdown_connections 2023-10-08T07:16:02.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.428+0000 7f2fea0a2640 1 -- 172.21.15.70:0/2534314706 wait complete. 2023-10-08T07:16:02.453 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-08T07:16:02.453 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-08-07_17_00 in ~58s... 2023-10-08T07:16:02.453 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2023-10-08T07:16:02.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-10-08T07:16:02.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.663+0000 7fac96d1d640 1 Processor -- start 2023-10-08T07:16:02.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.663+0000 7fac96d1d640 1 -- start start 2023-10-08T07:16:02.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.666+0000 7fac96d1d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac9015cba0 0x7fac900b84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.667+0000 7fac96d1d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9015cf80 0x7fac900b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.667+0000 7fac96d1d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac900b54a0 con 0x7fac900b4b40 2023-10-08T07:16:02.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.667+0000 7fac96d1d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac900b4fb0 con 0x7fac9015cba0 2023-10-08T07:16:02.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.667+0000 7fac96d1d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac900b7130 con 0x7fac9015cf80 2023-10-08T07:16:02.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.667+0000 7fac94ab9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac9015cba0 0x7fac900b84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.667+0000 7fac952ba640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fac900b4b40 0x7fac900b5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40652/0 (socket says 172.21.15.70:40652) 2023-10-08T07:16:02.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.667+0000 7fac952ba640 1 -- 172.21.15.70:0/2634629731 learned_addr learned my addr 172.21.15.70:0/2634629731 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:02.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.668+0000 7fac8f7fe640 1 -- 172.21.15.70:0/2634629731 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678118845 0 0) 0x7fac88001490 con 0x7fac900b4b40 2023-10-08T07:16:02.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.668+0000 7fac8f7fe640 1 -- 172.21.15.70:0/2634629731 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac80003520 con 0x7fac900b4b40 2023-10-08T07:16:02.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.668+0000 7fac94ab9640 1 -- 172.21.15.70:0/2634629731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9015cf80 msgr2=0x7fac900b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:02.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.668+0000 7fac94ab9640 1 --2- 172.21.15.70:0/2634629731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9015cf80 0x7fac900b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.668+0000 7fac94ab9640 1 -- 172.21.15.70:0/2634629731 >> v1:172.21.15.70:6789/0 conn(0x7fac900b4b40 legacy=0x7fac900b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.668+0000 7fac94ab9640 1 -- 172.21.15.70:0/2634629731 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac900b7410 con 0x7fac9015cba0 2023-10-08T07:16:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.669+0000 7fac94ab9640 1 --2- 172.21.15.70:0/2634629731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cba0 0x7fac900b84a0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fac840097a0 tx=0x7fac8402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f68ec9c65692eab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.669+0000 7fac8f7fe640 1 -- 172.21.15.70:0/2634629731 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8402d040 con 0x7fac9015cba0 2023-10-08T07:16:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.669+0000 7fac8f7fe640 1 -- 172.21.15.70:0/2634629731 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac84031810 con 0x7fac9015cba0 2023-10-08T07:16:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.669+0000 7fac8f7fe640 1 -- 172.21.15.70:0/2634629731 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac84031ba0 con 0x7fac9015cba0 2023-10-08T07:16:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.670+0000 7fac96d1d640 1 -- 172.21.15.70:0/2634629731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cba0 msgr2=0x7fac900b84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.670+0000 7fac96d1d640 1 --2- 172.21.15.70:0/2634629731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cba0 0x7fac900b84a0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7fac840097a0 tx=0x7fac8402b8b0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.670+0000 7fac96d1d640 1 -- 172.21.15.70:0/2634629731 shutdown_connections 2023-10-08T07:16:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.670+0000 7fac96d1d640 1 --2- 172.21.15.70:0/2634629731 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9015cf80 0x7fac900b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.670+0000 7fac96d1d640 1 --2- 172.21.15.70:0/2634629731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cba0 0x7fac900b84a0 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.670+0000 7fac96d1d640 1 -- 172.21.15.70:0/2634629731 >> 172.21.15.70:0/2634629731 conn(0x7fac900718a0 msgr2=0x7fac90071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.670+0000 7fac96d1d640 1 -- 172.21.15.70:0/2634629731 shutdown_connections 2023-10-08T07:16:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.670+0000 7fac96d1d640 1 -- 172.21.15.70:0/2634629731 wait complete. 2023-10-08T07:16:02.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.672+0000 7fac96d1d640 1 Processor -- start 2023-10-08T07:16:02.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.673+0000 7fac96d1d640 1 -- start start 2023-10-08T07:16:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.674+0000 7fac96d1d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac900b4b40 0x7fac9014dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.674+0000 7fac96d1d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9015cba0 0x7fac900c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.674+0000 7fac96d1d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cf80 0x7fac90154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.674+0000 7fac96d1d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9014e2c0 con 0x7fac9015cf80 2023-10-08T07:16:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.674+0000 7fac96d1d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac900c9c50 con 0x7fac900b4b40 2023-10-08T07:16:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.674+0000 7fac96d1d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac900c9ea0 con 0x7fac9015cba0 2023-10-08T07:16:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.674+0000 7fac952ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cf80 0x7fac90154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.674+0000 7fac952ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cf80 0x7fac90154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51630/0 (socket says 172.21.15.70:51630) 2023-10-08T07:16:02.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.674+0000 7fac952ba640 1 -- 172.21.15.70:0/1052380287 learned_addr learned my addr 172.21.15.70:0/1052380287 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.674+0000 7fac94ab9640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac900b4b40 0x7fac9014dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac8ffff640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9015cba0 0x7fac900c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac952ba640 1 -- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9015cba0 msgr2=0x7fac900c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac952ba640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9015cba0 0x7fac900c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac952ba640 1 -- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac900b4b40 msgr2=0x7fac9014dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac952ba640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac900b4b40 0x7fac9014dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac952ba640 1 -- 172.21.15.70:0/1052380287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac84009400 con 0x7fac9015cf80 2023-10-08T07:16:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac94ab9640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac900b4b40 0x7fac9014dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac8ffff640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9015cba0 0x7fac900c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:02.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac952ba640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cf80 0x7fac90154af0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fac8800cb40 tx=0x7fac8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:02.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac8d7fa640 1 -- 172.21.15.70:0/1052380287 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8800aca0 con 0x7fac9015cf80 2023-10-08T07:16:02.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac8d7fa640 1 -- 172.21.15.70:0/1052380287 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8800ae00 con 0x7fac9015cf80 2023-10-08T07:16:02.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac8d7fa640 1 -- 172.21.15.70:0/1052380287 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88012620 con 0x7fac9015cf80 2023-10-08T07:16:02.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.675+0000 7fac96d1d640 1 -- 172.21.15.70:0/1052380287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac900ca180 con 0x7fac9015cf80 2023-10-08T07:16:02.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.676+0000 7fac96d1d640 1 -- 172.21.15.70:0/1052380287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac900ca740 con 0x7fac9015cf80 2023-10-08T07:16:02.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.677+0000 7fac96d1d640 1 -- 172.21.15.70:0/1052380287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9004e160 con 0x7fac9015cf80 2023-10-08T07:16:02.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.678+0000 7fac8d7fa640 1 -- 172.21.15.70:0/1052380287 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 150) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fac88007070 con 0x7fac9015cf80 2023-10-08T07:16:02.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.679+0000 7fac8d7fa640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fac7406afe0 0x7fac7406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:02.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.679+0000 7fac8d7fa640 1 -- 172.21.15.70:0/1052380287 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7fac88089970 con 0x7fac9015cf80 2023-10-08T07:16:02.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.680+0000 7fac94ab9640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fac7406afe0 0x7fac7406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:02.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.681+0000 7fac94ab9640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fac7406afe0 0x7fac7406d4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fac84036000 tx=0x7fac8403c040 comp rx=0 tx=0).ready entity=mgr.9355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:02.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.684+0000 7fac8d7fa640 1 -- 172.21.15.70:0/1052380287 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac88054610 con 0x7fac9015cf80 2023-10-08T07:16:02.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.895+0000 7fac96d1d640 1 -- 172.21.15.70:0/1052380287 --> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac900b79d0 con 0x7fac7406afe0 2023-10-08T07:16:02.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.914+0000 7fac8d7fa640 1 -- 172.21.15.70:0/1052380287 <== mgr.9355 v2:172.21.15.181:6832/1127370729 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7fac84031be0 con 0x7fac7406afe0 2023-10-08T07:16:02.915 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:02.916 INFO:teuthology.orchestra.run.smithi070.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-08T07:16:02.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 -- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fac7406afe0 msgr2=0x7fac7406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fac7406afe0 0x7fac7406d4a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fac84036000 tx=0x7fac8403c040 comp rx=0 tx=0).stop 2023-10-08T07:16:02.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 -- 172.21.15.70:0/1052380287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cf80 msgr2=0x7fac90154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:02.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cf80 0x7fac90154af0 secure :-1 s=READY pgs=2936 cs=0 l=1 rev1=1 crypto rx=0x7fac8800cb40 tx=0x7fac8800a060 comp rx=0 tx=0).stop 2023-10-08T07:16:02.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 -- 172.21.15.70:0/1052380287 shutdown_connections 2023-10-08T07:16:02.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fac7406afe0 0x7fac7406d4a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac9015cf80 0x7fac90154af0 unknown :-1 s=CLOSED pgs=2936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fac9015cba0 0x7fac900c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 --2- 172.21.15.70:0/1052380287 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fac900b4b40 0x7fac9014dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:02.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 -- 172.21.15.70:0/1052380287 >> 172.21.15.70:0/1052380287 conn(0x7fac900718a0 msgr2=0x7fac900d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:02.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.916+0000 7fac96d1d640 1 -- 172.21.15.70:0/1052380287 shutdown_connections 2023-10-08T07:16:02.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:02.917+0000 7fac96d1d640 1 -- 172.21.15.70:0/1052380287 wait complete. 2023-10-08T07:16:02.939 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2023-10-08T07:16:02.939 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2023-10-08T07:16:02.940 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-10-08T07:16:03.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.155+0000 7fd392b55640 1 Processor -- start 2023-10-08T07:16:03.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.155+0000 7fd392b55640 1 -- start start 2023-10-08T07:16:03.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.157+0000 7fd392b55640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c06dfc0 0x7fd38c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.157+0000 7fd392b55640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0b1cc0 con 0x7fd38c071070 2023-10-08T07:16:03.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.157+0000 7fd392b55640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0b2620 con 0x7fd38c0afbe0 2023-10-08T07:16:03.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.157+0000 7fd392b55640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b2060 con 0x7fd38c06dfc0 2023-10-08T07:16:03.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.157+0000 7fd3910f2640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd38c071070 0x7fd38c0cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40662/0 (socket says 172.21.15.70:40662) 2023-10-08T07:16:03.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.157+0000 7fd3910f2640 1 -- 172.21.15.70:0/413431651 learned_addr learned my addr 172.21.15.70:0/413431651 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:03.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.157+0000 7fd38bfff640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c06dfc0 0x7fd38c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.157+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846098709 0 0) 0x7fd38c0b1cc0 con 0x7fd38c071070 2023-10-08T07:16:03.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd37c003520 con 0x7fd38c071070 2023-10-08T07:16:03.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 590652933 0 0) 0x7fd37c003520 con 0x7fd38c071070 2023-10-08T07:16:03.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c06dfc0 msgr2=0x7fd38c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:03.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c06dfc0 0x7fd38c070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 >> v1:172.21.15.181:6789/0 conn(0x7fd38c0afbe0 legacy=0x7fd38c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0b2340 con 0x7fd38c071070 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd3840019f0 con 0x7fd38c071070 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 >> v1:172.21.15.70:6789/0 conn(0x7fd38c071070 legacy=0x7fd38c0cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 0x7fd37c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c06dfc0 0x7fd37c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37c004d00 0x7fd37c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b2060 con 0x7fd37c0039a0 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c00bbc0 con 0x7fd37c004d00 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c00bd80 con 0x7fd38c06dfc0 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.158+0000 7fd3908f1640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 0x7fd37c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.159+0000 7fd38bfff640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c06dfc0 0x7fd37c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.159+0000 7fd3908f1640 1 -- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c06dfc0 msgr2=0x7fd37c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.159+0000 7fd3908f1640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c06dfc0 0x7fd37c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.159+0000 7fd3908f1640 1 -- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37c004d00 msgr2=0x7fd37c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.159+0000 7fd3910f2640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37c004d00 0x7fd37c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.159+0000 7fd3908f1640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd37c004d00 0x7fd37c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.159+0000 7fd3908f1640 1 -- 172.21.15.70:0/413431651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c003520 con 0x7fd37c0039a0 2023-10-08T07:16:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.160+0000 7fd3908f1640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 0x7fd37c003dc0 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7fd380001750 tx=0x7fd37c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac50727e79da9202 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.160+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800052d0 con 0x7fd37c0039a0 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.160+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38002e050 con 0x7fd37c0039a0 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.160+0000 7fd38b7fe640 1 -- 172.21.15.70:0/413431651 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800035f0 con 0x7fd37c0039a0 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.161+0000 7fd392b55640 1 -- 172.21.15.70:0/413431651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 msgr2=0x7fd37c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.161+0000 7fd392b55640 1 --2- 172.21.15.70:0/413431651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 0x7fd37c003dc0 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7fd380001750 tx=0x7fd37c0057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.161+0000 7fd3910f2640 1 -- 172.21.15.70:0/413431651 reap_dead start 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.161+0000 7fd392b55640 1 -- 172.21.15.70:0/413431651 shutdown_connections 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.161+0000 7fd392b55640 1 -- 172.21.15.70:0/413431651 >> 172.21.15.70:0/413431651 conn(0x7fd38c06a5d0 msgr2=0x7fd38c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.161+0000 7fd392b55640 1 -- 172.21.15.70:0/413431651 shutdown_connections 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.161+0000 7fd392b55640 1 -- 172.21.15.70:0/413431651 wait complete. 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.163+0000 7fd392b55640 1 Processor -- start 2023-10-08T07:16:03.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.163+0000 7fd392b55640 1 -- start start 2023-10-08T07:16:03.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.164+0000 7fd392b55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 0x7fd38c150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.164+0000 7fd392b55640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c151660 0x7fd38c153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.164+0000 7fd392b55640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c153da0 0x7fd38c154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.164+0000 7fd392b55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1513f0 con 0x7fd37c0039a0 2023-10-08T07:16:03.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.164+0000 7fd392b55640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1568d0 con 0x7fd38c153da0 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.164+0000 7fd392b55640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c156bb0 con 0x7fd38c151660 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.164+0000 7fd3908f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 0x7fd38c150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.164+0000 7fd3908f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 0x7fd38c150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51648/0 (socket says 172.21.15.70:51648) 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.164+0000 7fd3908f1640 1 -- 172.21.15.70:0/2847236042 learned_addr learned my addr 172.21.15.70:0/2847236042 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd3908f1640 1 -- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c151660 msgr2=0x7fd38c153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd3908f1640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c151660 0x7fd38c153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd3910f2640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c153da0 0x7fd38c154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd3908f1640 1 -- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c153da0 msgr2=0x7fd38c154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd3908f1640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c153da0 0x7fd38c154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd3908f1640 1 -- 172.21.15.70:0/2847236042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0c2ed0 con 0x7fd37c0039a0 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd3908f1640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 0x7fd38c150eb0 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7fd380008fd0 tx=0x7fd380001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd3897fa640 1 -- 172.21.15.70:0/2847236042 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800076c0 con 0x7fd37c0039a0 2023-10-08T07:16:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd3897fa640 1 -- 172.21.15.70:0/2847236042 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380007a30 con 0x7fd37c0039a0 2023-10-08T07:16:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd3897fa640 1 -- 172.21.15.70:0/2847236042 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380006be0 con 0x7fd37c0039a0 2023-10-08T07:16:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd392b55640 1 -- 172.21.15.70:0/2847236042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c0c31b0 con 0x7fd37c0039a0 2023-10-08T07:16:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.165+0000 7fd392b55640 1 -- 172.21.15.70:0/2847236042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c1564f0 con 0x7fd37c0039a0 2023-10-08T07:16:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.167+0000 7fd392b55640 1 -- 172.21.15.70:0/2847236042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c156680 con 0x7fd37c0039a0 2023-10-08T07:16:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.167+0000 7fd3897fa640 1 -- 172.21.15.70:0/2847236042 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 150) v1 ==== 89349+0+0 (secure 0 0 0) 0x7fd380006d40 con 0x7fd37c0039a0 2023-10-08T07:16:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.168+0000 7fd3897fa640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fd37006b010 0x7fd37006d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.168+0000 7fd3897fa640 1 -- 172.21.15.70:0/2847236042 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7fd3800abd40 con 0x7fd37c0039a0 2023-10-08T07:16:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.169+0000 7fd38bfff640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fd37006b010 0x7fd37006d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.170+0000 7fd38bfff640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fd37006b010 0x7fd37006d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0c2dc0 tx=0x7fd37800d220 comp rx=0 tx=0).ready entity=mgr.9355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:03.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.173+0000 7fd3897fa640 1 -- 172.21.15.70:0/2847236042 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd38007a2c0 con 0x7fd37c0039a0 2023-10-08T07:16:03.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.386+0000 7fd392b55640 1 -- 172.21.15.70:0/2847236042 --> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd38c1513f0 con 0x7fd37006b010 2023-10-08T07:16:03.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.532+0000 7fd3897fa640 1 -- 172.21.15.70:0/2847236042 <== mgr.9355 v2:172.21.15.181:6832/1127370729 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd38c1513f0 con 0x7fd37006b010 2023-10-08T07:16:03.534 INFO:teuthology.orchestra.run.smithi070.stdout:Schedule removed for path snap_test_dir1 2023-10-08T07:16:03.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 -- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fd37006b010 msgr2=0x7fd37006d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fd37006b010 0x7fd37006d4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0c2dc0 tx=0x7fd37800d220 comp rx=0 tx=0).stop 2023-10-08T07:16:03.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 -- 172.21.15.70:0/2847236042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 msgr2=0x7fd38c150eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 0x7fd38c150eb0 secure :-1 s=READY pgs=2939 cs=0 l=1 rev1=1 crypto rx=0x7fd380008fd0 tx=0x7fd380001120 comp rx=0 tx=0).stop 2023-10-08T07:16:03.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 -- 172.21.15.70:0/2847236042 shutdown_connections 2023-10-08T07:16:03.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fd37006b010 0x7fd37006d4d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd38c153da0 0x7fd38c154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd38c151660 0x7fd38c153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 --2- 172.21.15.70:0/2847236042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd37c0039a0 0x7fd38c150eb0 unknown :-1 s=CLOSED pgs=2939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 -- 172.21.15.70:0/2847236042 >> 172.21.15.70:0/2847236042 conn(0x7fd38c06a5d0 msgr2=0x7fd38c0b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 -- 172.21.15.70:0/2847236042 shutdown_connections 2023-10-08T07:16:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.535+0000 7fd392b55640 1 -- 172.21.15.70:0/2847236042 wait complete. 2023-10-08T07:16:03.557 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-08T07:16:03.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:03.557 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T07:16:03.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:03.616 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T07:16:03.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T07:16:03.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.886+0000 7f14899ed640 1 Processor -- start 2023-10-08T07:16:03.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.886+0000 7f14899ed640 1 -- start start 2023-10-08T07:16:03.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14899ed640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f14840cdc30 0x7f14840cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14899ed640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14840cce90 0x7f1484159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14899ed640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148415ac90 con 0x7f14840caef0 2023-10-08T07:16:03.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14899ed640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148415a6d0 con 0x7f14840cce90 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14899ed640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148415a9b0 con 0x7f14840cdc30 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14889eb640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f14840caef0 0x7f14840cb2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48992/0 (socket says 172.21.15.70:48992) 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14889eb640 1 -- 172.21.15.70:0/699590784 learned_addr learned my addr 172.21.15.70:0/699590784 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14891ec640 1 --2- 172.21.15.70:0/699590784 >> v2:172.21.15.70:3300/0 conn(0x7f14840cce90 0x7f1484159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f1483fff640 1 --2- 172.21.15.70:0/699590784 >> v2:172.21.15.181:3300/0 conn(0x7f14840cdc30 0x7f14840cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14837fe640 1 -- 172.21.15.70:0/699590784 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695029077 0 0) 0x7f148415ac90 con 0x7f14840caef0 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14837fe640 1 -- 172.21.15.70:0/699590784 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1474005730 con 0x7f14840caef0 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.888+0000 7f14891ec640 1 -- 172.21.15.70:0/699590784 >> v2:172.21.15.181:3300/0 conn(0x7f14840cdc30 msgr2=0x7f14840cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.889+0000 7f14891ec640 1 --2- 172.21.15.70:0/699590784 >> v2:172.21.15.181:3300/0 conn(0x7f14840cdc30 0x7f14840cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.889+0000 7f14891ec640 1 -- 172.21.15.70:0/699590784 >> v1:172.21.15.70:6789/0 conn(0x7f14840caef0 legacy=0x7f14840cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.889+0000 7f14891ec640 1 -- 172.21.15.70:0/699590784 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14840cc150 con 0x7f14840cce90 2023-10-08T07:16:03.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.889+0000 7f1483fff640 1 --2- 172.21.15.70:0/699590784 >> v2:172.21.15.181:3300/0 conn(0x7f14840cdc30 0x7f14840cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:03.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.890+0000 7f14891ec640 1 --2- 172.21.15.70:0/699590784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 0x7f1484159bc0 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7f148406e510 tx=0x7f147c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c710f63d3964cc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:03.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.890+0000 7f14837fe640 1 -- 172.21.15.70:0/699590784 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c02cc80 con 0x7f14840cce90 2023-10-08T07:16:03.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.890+0000 7f14837fe640 1 -- 172.21.15.70:0/699590784 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c02cde0 con 0x7f14840cce90 2023-10-08T07:16:03.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.890+0000 7f14837fe640 1 -- 172.21.15.70:0/699590784 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c034660 con 0x7f14840cce90 2023-10-08T07:16:03.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.890+0000 7f14899ed640 1 -- 172.21.15.70:0/699590784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 msgr2=0x7f1484159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.890+0000 7f14899ed640 1 --2- 172.21.15.70:0/699590784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 0x7f1484159bc0 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7f148406e510 tx=0x7f147c02bed0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.891+0000 7f14899ed640 1 -- 172.21.15.70:0/699590784 shutdown_connections 2023-10-08T07:16:03.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.891+0000 7f14899ed640 1 --2- 172.21.15.70:0/699590784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 0x7f1484159bc0 unknown :-1 s=CLOSED pgs=2941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.891+0000 7f14899ed640 1 --2- 172.21.15.70:0/699590784 >> v2:172.21.15.181:3300/0 conn(0x7f14840cdc30 0x7f14840cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.891+0000 7f14899ed640 1 -- 172.21.15.70:0/699590784 >> 172.21.15.70:0/699590784 conn(0x7f148406c010 msgr2=0x7f148406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:03.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.891+0000 7f14899ed640 1 -- 172.21.15.70:0/699590784 shutdown_connections 2023-10-08T07:16:03.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.891+0000 7f14899ed640 1 -- 172.21.15.70:0/699590784 wait complete. 2023-10-08T07:16:03.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.892+0000 7f14899ed640 1 Processor -- start 2023-10-08T07:16:03.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.893+0000 7f14899ed640 1 -- start start 2023-10-08T07:16:03.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f14899ed640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840caef0 0x7f1484154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f14899ed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 0x7f14840c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f14899ed640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14840cdc30 0x7f1484157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f14899ed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14841f2180 con 0x7f14840cce90 2023-10-08T07:16:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f14899ed640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14840c71d0 con 0x7f14840caef0 2023-10-08T07:16:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f14899ed640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14840c74b0 con 0x7f14840cdc30 2023-10-08T07:16:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f1483fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 0x7f14840c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f1483fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 0x7f14840c6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57196/0 (socket says 172.21.15.70:57196) 2023-10-08T07:16:03.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f1483fff640 1 -- 172.21.15.70:0/510639946 learned_addr learned my addr 172.21.15.70:0/510639946 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:03.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f1483fff640 1 -- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14840cdc30 msgr2=0x7f1484157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f1483fff640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14840cdc30 0x7f1484157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f1483fff640 1 -- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840caef0 msgr2=0x7f1484154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:03.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f1483fff640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840caef0 0x7f1484154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:03.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.894+0000 7f1483fff640 1 -- 172.21.15.70:0/510639946 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147c009a70 con 0x7f14840cce90 2023-10-08T07:16:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.895+0000 7f1483fff640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 0x7f14840c6730 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7f147000c7f0 tx=0x7f147000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.895+0000 7f14817fa640 1 -- 172.21.15.70:0/510639946 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470007820 con 0x7f14840cce90 2023-10-08T07:16:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.895+0000 7f14817fa640 1 -- 172.21.15.70:0/510639946 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1470007980 con 0x7f14840cce90 2023-10-08T07:16:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.895+0000 7f14817fa640 1 -- 172.21.15.70:0/510639946 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470007cf0 con 0x7f14840cce90 2023-10-08T07:16:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.895+0000 7f14899ed640 1 -- 172.21.15.70:0/510639946 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14840c7790 con 0x7f14840cce90 2023-10-08T07:16:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.895+0000 7f14899ed640 1 -- 172.21.15.70:0/510639946 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14840ad120 con 0x7f14840cce90 2023-10-08T07:16:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.896+0000 7f14899ed640 1 -- 172.21.15.70:0/510639946 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1484074150 con 0x7f14840cce90 2023-10-08T07:16:03.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.897+0000 7f14817fa640 1 -- 172.21.15.70:0/510639946 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 150) v1 ==== 89349+0+0 (secure 0 0 0) 0x7f14700113f0 con 0x7f14840cce90 2023-10-08T07:16:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.898+0000 7f14817fa640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f146806afe0 0x7f146806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.898+0000 7f14817fa640 1 -- 172.21.15.70:0/510639946 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7f1470089fb0 con 0x7f14840cce90 2023-10-08T07:16:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.899+0000 7f14889eb640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f146806afe0 0x7f146806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:03.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.900+0000 7f14889eb640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f146806afe0 0x7f146806d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f14840ace80 tx=0x7f1478007930 comp rx=0 tx=0).ready entity=mgr.9355 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:03.902+0000 7f14817fa640 1 -- 172.21.15.70:0/510639946 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1470054aa0 con 0x7f14840cce90 2023-10-08T07:16:04.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.143+0000 7f14899ed640 1 -- 172.21.15.70:0/510639946 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f14841f22c0 con 0x7f14840cce90 2023-10-08T07:16:04.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.676+0000 7f14817fa640 1 -- 172.21.15.70:0/510639946 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v151) v1 ==== 93+0+0 (secure 0 0 0) 0x7f14841f22c0 con 0x7f14840cce90 2023-10-08T07:16:04.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.679+0000 7f14899ed640 1 -- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f146806afe0 msgr2=0x7f146806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:04.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.679+0000 7f14899ed640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f146806afe0 0x7f146806d4a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f14840ace80 tx=0x7f1478007930 comp rx=0 tx=0).stop 2023-10-08T07:16:04.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.679+0000 7f14899ed640 1 -- 172.21.15.70:0/510639946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 msgr2=0x7f14840c6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:04.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.679+0000 7f14899ed640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 0x7f14840c6730 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7f147000c7f0 tx=0x7f147000ccb0 comp rx=0 tx=0).stop 2023-10-08T07:16:04.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.680+0000 7f14899ed640 1 -- 172.21.15.70:0/510639946 shutdown_connections 2023-10-08T07:16:04.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.680+0000 7f14899ed640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f146806afe0 0x7f146806d4a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:04.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.680+0000 7f14899ed640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f14840cdc30 0x7f1484157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:04.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.680+0000 7f14899ed640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14840cce90 0x7f14840c6730 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:04.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.680+0000 7f14899ed640 1 --2- 172.21.15.70:0/510639946 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f14840caef0 0x7f1484154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:04.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.680+0000 7f14899ed640 1 -- 172.21.15.70:0/510639946 >> 172.21.15.70:0/510639946 conn(0x7f148406c010 msgr2=0x7f14840c4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:04.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.680+0000 7f14899ed640 1 -- 172.21.15.70:0/510639946 shutdown_connections 2023-10-08T07:16:04.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:04.680+0000 7f14899ed640 1 -- 172.21.15.70:0/510639946 wait complete. 2023-10-08T07:16:04.708 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T07:16:04.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:16:04.804 DEBUG:teuthology.orchestra.run.smithi181:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T07:16:04.901 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T07:16:04.901 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:16:04.901 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:16:04.956 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:16:04.956 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:16:05.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:16:05.204 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:05.202+0000 7f2754434040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:16:05.206 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:05.205+0000 7f9d89613040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T07:16:05.257 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:05.256+0000 7f2754434040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:16:05.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.257+0000 7f6d4410c640 1 Processor -- start 2023-10-08T07:16:05.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.257+0000 7f6d4410c640 1 -- start start 2023-10-08T07:16:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.258+0000 7f6d4410c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d3c159ac0 0x7f6d3c06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.259+0000 7f6d4410c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d3c0ad790 0x7f6d3c0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.259+0000 7f6d4410c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3c06dee0 con 0x7f6d3c15e330 2023-10-08T07:16:05.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.259+0000 7f6d4410c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3c15d850 con 0x7f6d3c159ac0 2023-10-08T07:16:05.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.259+0000 7f6d4410c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c15db30 con 0x7f6d3c0ad790 2023-10-08T07:16:05.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.259+0000 7f6d41ea8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d3c159ac0 0x7f6d3c06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:05.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.259+0000 7f6d416a7640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d3c0ad790 0x7f6d3c0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:05.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.259+0000 7f6d41ea8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f6d3c159ac0 0x7f6d3c06d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:51358/0 (socket says 172.21.15.70:51358) 2023-10-08T07:16:05.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.259+0000 7f6d41ea8640 1 -- 172.21.15.70:0/354403182 learned_addr learned my addr 172.21.15.70:0/354403182 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:05.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.260+0000 7f6d41ea8640 1 -- 172.21.15.70:0/354403182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d3c0ad790 msgr2=0x7f6d3c0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:05.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.260+0000 7f6d41ea8640 1 --2- 172.21.15.70:0/354403182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d3c0ad790 0x7f6d3c0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.260+0000 7f6d41ea8640 1 -- 172.21.15.70:0/354403182 >> v1:172.21.15.181:6789/0 conn(0x7f6d3c15e330 legacy=0x7f6d3c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.260+0000 7f6d41ea8640 1 -- 172.21.15.70:0/354403182 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c15de10 con 0x7f6d3c159ac0 2023-10-08T07:16:05.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.261+0000 7f6d41ea8640 1 --2- 172.21.15.70:0/354403182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c159ac0 0x7f6d3c06d5e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009870 tx=0x7f6d3802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d013db854f162b04 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:05.263 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:05.261+0000 7f9d89613040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T07:16:05.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.262+0000 7f6d40ea6640 1 -- 172.21.15.70:0/354403182 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38030840 con 0x7f6d3c159ac0 2023-10-08T07:16:05.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.262+0000 7f6d40ea6640 1 -- 172.21.15.70:0/354403182 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d380309a0 con 0x7f6d3c159ac0 2023-10-08T07:16:05.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.262+0000 7f6d40ea6640 1 -- 172.21.15.70:0/354403182 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38030d30 con 0x7f6d3c159ac0 2023-10-08T07:16:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.263+0000 7f6d4410c640 1 -- 172.21.15.70:0/354403182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c159ac0 msgr2=0x7f6d3c06d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.263+0000 7f6d4410c640 1 --2- 172.21.15.70:0/354403182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c159ac0 0x7f6d3c06d5e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009870 tx=0x7f6d3802b980 comp rx=0 tx=0).stop 2023-10-08T07:16:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.263+0000 7f6d4410c640 1 -- 172.21.15.70:0/354403182 shutdown_connections 2023-10-08T07:16:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.263+0000 7f6d4410c640 1 --2- 172.21.15.70:0/354403182 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d3c0ad790 0x7f6d3c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.263+0000 7f6d4410c640 1 --2- 172.21.15.70:0/354403182 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c159ac0 0x7f6d3c06d5e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.263+0000 7f6d4410c640 1 -- 172.21.15.70:0/354403182 >> 172.21.15.70:0/354403182 conn(0x7f6d3c06a380 msgr2=0x7f6d3c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.263+0000 7f6d4410c640 1 -- 172.21.15.70:0/354403182 shutdown_connections 2023-10-08T07:16:05.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.263+0000 7f6d4410c640 1 -- 172.21.15.70:0/354403182 wait complete. 2023-10-08T07:16:05.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.265+0000 7f6d4410c640 1 Processor -- start 2023-10-08T07:16:05.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.266+0000 7f6d4410c640 1 -- start start 2023-10-08T07:16:05.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.267+0000 7f6d4410c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d3c0ad790 0x7f6d3c1453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.267+0000 7f6d4410c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c159ac0 0x7f6d3c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.267+0000 7f6d4410c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d3c15e330 0x7f6d3c0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.267+0000 7f6d4410c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c1458e0 con 0x7f6d3c15e330 2023-10-08T07:16:05.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.267+0000 7f6d4410c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c15b5a0 con 0x7f6d3c159ac0 2023-10-08T07:16:05.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.267+0000 7f6d4410c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c15a990 con 0x7f6d3c0ad790 2023-10-08T07:16:05.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.268+0000 7f6d426a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d3c15e330 0x7f6d3c0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:05.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.268+0000 7f6d426a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d3c15e330 0x7f6d3c0b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57228/0 (socket says 172.21.15.70:57228) 2023-10-08T07:16:05.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.268+0000 7f6d426a9640 1 -- 172.21.15.70:0/2074473029 learned_addr learned my addr 172.21.15.70:0/2074473029 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:05.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.268+0000 7f6d416a7640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c159ac0 0x7f6d3c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:05.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.268+0000 7f6d41ea8640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d3c0ad790 0x7f6d3c1453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:05.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.268+0000 7f6d426a9640 1 -- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d3c0ad790 msgr2=0x7f6d3c1453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.268+0000 7f6d426a9640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d3c0ad790 0x7f6d3c1453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.268+0000 7f6d426a9640 1 -- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c159ac0 msgr2=0x7f6d3c15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.268+0000 7f6d426a9640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c159ac0 0x7f6d3c15ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.268+0000 7f6d426a9640 1 -- 172.21.15.70:0/2074473029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380094d0 con 0x7f6d3c15e330 2023-10-08T07:16:05.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.269+0000 7f6d426a9640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d3c15e330 0x7f6d3c0b14a0 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400c6e0 tx=0x7f6d3400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:05.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.269+0000 7f6d2e7fc640 1 -- 172.21.15.70:0/2074473029 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34003b70 con 0x7f6d3c15e330 2023-10-08T07:16:05.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.269+0000 7f6d2e7fc640 1 -- 172.21.15.70:0/2074473029 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d34003cf0 con 0x7f6d3c15e330 2023-10-08T07:16:05.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.269+0000 7f6d2e7fc640 1 -- 172.21.15.70:0/2074473029 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34005310 con 0x7f6d3c15e330 2023-10-08T07:16:05.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.269+0000 7f6d4410c640 1 -- 172.21.15.70:0/2074473029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3c15ac70 con 0x7f6d3c15e330 2023-10-08T07:16:05.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.269+0000 7f6d4410c640 1 -- 172.21.15.70:0/2074473029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3c15b230 con 0x7f6d3c15e330 2023-10-08T07:16:05.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.270+0000 7f6d4410c640 1 -- 172.21.15.70:0/2074473029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3c067490 con 0x7f6d3c15e330 2023-10-08T07:16:05.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.271+0000 7f6d2e7fc640 1 -- 172.21.15.70:0/2074473029 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 151) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f6d34005470 con 0x7f6d3c15e330 2023-10-08T07:16:05.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.272+0000 7f6d2e7fc640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6d2006b020 0x7f6d2006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.272+0000 7f6d2e7fc640 1 -- 172.21.15.70:0/2074473029 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7f6d34089a20 con 0x7f6d3c15e330 2023-10-08T07:16:05.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.272+0000 7f6d41ea8640 1 -- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6d2006b020 msgr2=0x7f6d2006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1127370729 2023-10-08T07:16:05.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.272+0000 7f6d41ea8640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6d2006b020 0x7f6d2006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:16:05.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.277+0000 7f6d2e7fc640 1 -- 172.21.15.70:0/2074473029 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d340544b0 con 0x7f6d3c15e330 2023-10-08T07:16:05.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.469+0000 7f6d4410c640 1 -- 172.21.15.70:0/2074473029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d3c15b6e0 con 0x7f6d3c15e330 2023-10-08T07:16:05.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.470+0000 7f6d2e7fc640 1 -- 172.21.15.70:0/2074473029 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v277) v1 ==== 74+0+17586 (secure 0 0 0) 0x7f6d34003e60 con 0x7f6d3c15e330 2023-10-08T07:16:05.472 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:05.473 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":277,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:16:00.570930+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:15:43.819948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-08T07:15:44.895969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3814802416":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/3705279960":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6833/440573135":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6832/440573135":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/1545538679":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/1151602740":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2253796914":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/3307173154":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2296566554":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/606980":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/1513601554":"2023-10-08T08:15:42.823407+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:16:05.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 -- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6d2006b020 msgr2=0x7f6d2006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:16:05.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6d2006b020 0x7f6d2006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 -- 172.21.15.70:0/2074473029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d3c15e330 msgr2=0x7f6d3c0b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d3c15e330 0x7f6d3c0b14a0 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400c6e0 tx=0x7f6d3400cba0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 -- 172.21.15.70:0/2074473029 shutdown_connections 2023-10-08T07:16:05.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6d2006b020 0x7f6d2006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d3c15e330 0x7f6d3c0b14a0 unknown :-1 s=CLOSED pgs=2945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6d3c159ac0 0x7f6d3c15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 --2- 172.21.15.70:0/2074473029 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6d3c0ad790 0x7f6d3c1453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 -- 172.21.15.70:0/2074473029 >> 172.21.15.70:0/2074473029 conn(0x7f6d3c06a380 msgr2=0x7f6d3c15c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:05.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 -- 172.21.15.70:0/2074473029 shutdown_connections 2023-10-08T07:16:05.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.473+0000 7f6d4410c640 1 -- 172.21.15.70:0/2074473029 wait complete. 2023-10-08T07:16:05.495 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 35, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T07:15:43.819948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '275', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 36, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T07:15:44.895969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '276', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:16:05.495 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:16:05.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.692+0000 7f763ce7c640 1 Processor -- start 2023-10-08T07:16:05.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.693+0000 7f763ce7c640 1 -- start start 2023-10-08T07:16:05.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.694+0000 7f763ce7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76380d1f70 0x7f76380cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.694+0000 7f763ce7c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76380b71c0 con 0x7f76380cffc0 2023-10-08T07:16:05.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.694+0000 7f763ce7c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76380b74a0 con 0x7f763815cc50 2023-10-08T07:16:05.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.694+0000 7f763ce7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76380d0470 con 0x7f76380d1f70 2023-10-08T07:16:05.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.695+0000 7f762ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76380d1f70 0x7f76380cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:05.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.695+0000 7f7636d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f76380cffc0 0x7f76380b5b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49004/0 (socket says 172.21.15.70:49004) 2023-10-08T07:16:05.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.695+0000 7f7636d9d640 1 -- 172.21.15.70:0/3356311138 learned_addr learned my addr 172.21.15.70:0/3356311138 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:05.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.695+0000 7f762ffff640 1 -- 172.21.15.70:0/3356311138 >> v1:172.21.15.181:6789/0 conn(0x7f763815cc50 legacy=0x7f76380b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.695+0000 7f762ffff640 1 -- 172.21.15.70:0/3356311138 >> v1:172.21.15.70:6789/0 conn(0x7f76380cffc0 legacy=0x7f76380b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.695+0000 7f7635d9b640 1 -- 172.21.15.70:0/3356311138 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998640185 0 0) 0x7f76380b71c0 con 0x7f76380cffc0 2023-10-08T07:16:05.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.695+0000 7f762ffff640 1 -- 172.21.15.70:0/3356311138 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76380b7780 con 0x7f76380d1f70 2023-10-08T07:16:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.696+0000 7f762ffff640 1 --2- 172.21.15.70:0/3356311138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76380d1f70 0x7f76380cfa80 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f76200097a0 tx=0x7f762002b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5a5d488fddce70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.696+0000 7f7635d9b640 1 -- 172.21.15.70:0/3356311138 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762002f810 con 0x7f76380d1f70 2023-10-08T07:16:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.696+0000 7f7635d9b640 1 -- 172.21.15.70:0/3356311138 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762002f970 con 0x7f76380d1f70 2023-10-08T07:16:05.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.697+0000 7f7635d9b640 1 -- 172.21.15.70:0/3356311138 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762002fd00 con 0x7f76380d1f70 2023-10-08T07:16:05.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.697+0000 7f763ce7c640 1 -- 172.21.15.70:0/3356311138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76380d1f70 msgr2=0x7f76380cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.697+0000 7f763ce7c640 1 --2- 172.21.15.70:0/3356311138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76380d1f70 0x7f76380cfa80 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f76200097a0 tx=0x7f762002b9f0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.697+0000 7f763ce7c640 1 -- 172.21.15.70:0/3356311138 shutdown_connections 2023-10-08T07:16:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.697+0000 7f763ce7c640 1 --2- 172.21.15.70:0/3356311138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76380d1f70 0x7f76380cfa80 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.697+0000 7f763ce7c640 1 -- 172.21.15.70:0/3356311138 >> 172.21.15.70:0/3356311138 conn(0x7f763806b5e0 msgr2=0x7f763806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.697+0000 7f763ce7c640 1 -- 172.21.15.70:0/3356311138 shutdown_connections 2023-10-08T07:16:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.697+0000 7f763ce7c640 1 -- 172.21.15.70:0/3356311138 wait complete. 2023-10-08T07:16:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.699+0000 7f763ce7c640 1 Processor -- start 2023-10-08T07:16:05.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.699+0000 7f763ce7c640 1 -- start start 2023-10-08T07:16:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.700+0000 7f763ce7c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76380cffc0 0x7f76381571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.700+0000 7f763ce7c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76380d1f70 0x7f76380c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.700+0000 7f763ce7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f763815cc50 0x7f76380d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.700+0000 7f763ce7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76380cc9f0 con 0x7f763815cc50 2023-10-08T07:16:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.700+0000 7f763ce7c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76381f6460 con 0x7f76380d1f70 2023-10-08T07:16:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.700+0000 7f763ce7c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76381f6180 con 0x7f76380cffc0 2023-10-08T07:16:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.700+0000 7f7636d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f763815cc50 0x7f76380d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f7636d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f763815cc50 0x7f76380d0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57244/0 (socket says 172.21.15.70:57244) 2023-10-08T07:16:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f7636d9d640 1 -- 172.21.15.70:0/3277127422 learned_addr learned my addr 172.21.15.70:0/3277127422 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f763659c640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76380cffc0 0x7f76381571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f762ffff640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76380d1f70 0x7f76380c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f7636d9d640 1 -- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76380cffc0 msgr2=0x7f76381571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f7636d9d640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76380cffc0 0x7f76381571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f7636d9d640 1 -- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76380d1f70 msgr2=0x7f76380c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f7636d9d640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76380d1f70 0x7f76380c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f7636d9d640 1 -- 172.21.15.70:0/3277127422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7620009400 con 0x7f763815cc50 2023-10-08T07:16:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f7636d9d640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f763815cc50 0x7f76380d0e60 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f763000c7e0 tx=0x7f763000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f762effd640 1 -- 172.21.15.70:0/3277127422 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763000a640 con 0x7f763815cc50 2023-10-08T07:16:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f762effd640 1 -- 172.21.15.70:0/3277127422 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763000a7c0 con 0x7f763815cc50 2023-10-08T07:16:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.701+0000 7f762effd640 1 -- 172.21.15.70:0/3277127422 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763000ab70 con 0x7f763815cc50 2023-10-08T07:16:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.702+0000 7f763ce7c640 1 -- 172.21.15.70:0/3277127422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763815d370 con 0x7f763815cc50 2023-10-08T07:16:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.702+0000 7f763ce7c640 1 -- 172.21.15.70:0/3277127422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763815d930 con 0x7f763815cc50 2023-10-08T07:16:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.703+0000 7f763ce7c640 1 -- 172.21.15.70:0/3277127422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763804e160 con 0x7f763815cc50 2023-10-08T07:16:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.704+0000 7f762effd640 1 -- 172.21.15.70:0/3277127422 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 151) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f763000acd0 con 0x7f763815cc50 2023-10-08T07:16:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.705+0000 7f762effd640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f761806b020 0x7f761806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.705+0000 7f762effd640 1 -- 172.21.15.70:0/3277127422 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7f7630089250 con 0x7f763815cc50 2023-10-08T07:16:05.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.705+0000 7f763659c640 1 -- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f761806b020 msgr2=0x7f761806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1127370729 2023-10-08T07:16:05.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.705+0000 7f763659c640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f761806b020 0x7f761806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:16:05.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.709+0000 7f762effd640 1 -- 172.21.15.70:0/3277127422 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7630053d80 con 0x7f763815cc50 2023-10-08T07:16:05.711 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:05.710+0000 7f2754434040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:16:05.736 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:05.734+0000 7f9d89613040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T07:16:05.765 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:05.763+0000 7f2754434040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:16:05.790 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:05.788+0000 7f9d89613040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T07:16:05.816 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:05.814+0000 7f2754434040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:16:05.841 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:05.839+0000 7f9d89613040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T07:16:05.876 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:05.875+0000 7f2754434040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:16:05.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.899+0000 7f763ce7c640 1 -- 172.21.15.70:0/3277127422 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7638157840 con 0x7f763815cc50 2023-10-08T07:16:05.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.900+0000 7f762effd640 1 -- 172.21.15.70:0/3277127422 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v277) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76300180a0 con 0x7f763815cc50 2023-10-08T07:16:05.902 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:16:05.903 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:05.902+0000 7f9d89613040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T07:16:05.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 -- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f761806b020 msgr2=0x7f761806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:16:05.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f761806b020 0x7f761806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 -- 172.21.15.70:0/3277127422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f763815cc50 msgr2=0x7f76380d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:05.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f763815cc50 0x7f76380d0e60 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f763000c7e0 tx=0x7f763000cca0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 -- 172.21.15.70:0/3277127422 shutdown_connections 2023-10-08T07:16:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f761806b020 0x7f761806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f763815cc50 0x7f76380d0e60 unknown :-1 s=CLOSED pgs=2948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76380d1f70 0x7f76380c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 --2- 172.21.15.70:0/3277127422 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76380cffc0 0x7f76381571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 -- 172.21.15.70:0/3277127422 >> 172.21.15.70:0/3277127422 conn(0x7f763806b5e0 msgr2=0x7f76380b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.903+0000 7f763ce7c640 1 -- 172.21.15.70:0/3277127422 shutdown_connections 2023-10-08T07:16:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:05.904+0000 7f763ce7c640 1 -- 172.21.15.70:0/3277127422 wait complete. 2023-10-08T07:16:05.923 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:16:05.925 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:05.924+0000 7f2754434040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:16:05.957 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:05.956+0000 7f9d89613040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T07:16:06.036 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:06.034+0000 7f2754434040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:16:06.074 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:06.072+0000 7f9d89613040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T07:16:06.099 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:06.097+0000 7f2754434040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:16:06.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.136+0000 7f2c6ac9e640 1 Processor -- start 2023-10-08T07:16:06.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.136+0000 7f2c6ac9e640 1 -- start start 2023-10-08T07:16:06.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.138+0000 7f2c6ac9e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c640cf2b0 0x7f2c640d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.138+0000 7f2c6ac9e640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f2c640ce510 0x7f2c6415c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.138+0000 7f2c6ac9e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c640cd520 con 0x7f2c640cf690 2023-10-08T07:16:06.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.138+0000 7f2c6ac9e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c640d1910 con 0x7f2c640cf2b0 2023-10-08T07:16:06.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.138+0000 7f2c6ac9e640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c640ce080 con 0x7f2c640ce510 2023-10-08T07:16:06.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.138+0000 7f2c69c9c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c640cf2b0 0x7f2c640d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:06.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.138+0000 7f2c6949b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2c640cf690 0x7f2c640d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49006/0 (socket says 172.21.15.70:49006) 2023-10-08T07:16:06.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.138+0000 7f2c6949b640 1 -- 172.21.15.70:0/3951819768 learned_addr learned my addr 172.21.15.70:0/3951819768 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:06.142 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:06.138+0000 7f9d89613040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T07:16:06.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.138+0000 7f2c6a49d640 1 --2- 172.21.15.70:0/3951819768 >> v2:172.21.15.181:3300/0 conn(0x7f2c640ce510 0x7f2c6415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:06.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.139+0000 7f2c69c9c640 1 -- 172.21.15.70:0/3951819768 >> v2:172.21.15.181:3300/0 conn(0x7f2c640ce510 msgr2=0x7f2c6415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:06.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.139+0000 7f2c69c9c640 1 --2- 172.21.15.70:0/3951819768 >> v2:172.21.15.181:3300/0 conn(0x7f2c640ce510 0x7f2c6415c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.139+0000 7f2c69c9c640 1 -- 172.21.15.70:0/3951819768 >> v1:172.21.15.70:6789/0 conn(0x7f2c640cf690 legacy=0x7f2c640d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:06.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.139+0000 7f2c69c9c640 1 -- 172.21.15.70:0/3951819768 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640cd750 con 0x7f2c640cf2b0 2023-10-08T07:16:06.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.139+0000 7f2c68c9a640 1 -- 172.21.15.70:0/3951819768 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077942291 0 0) 0x7f2c640cd520 con 0x7f2c640cf690 2023-10-08T07:16:06.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.140+0000 7f2c69c9c640 1 --2- 172.21.15.70:0/3951819768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640cf2b0 0x7f2c640d34d0 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f2c640cfb20 tx=0x7f2c6002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696e6289f656ce48 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.140+0000 7f2c68c9a640 1 -- 172.21.15.70:0/3951819768 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6002e070 con 0x7f2c640cf2b0 2023-10-08T07:16:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.140+0000 7f2c68c9a640 1 -- 172.21.15.70:0/3951819768 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c600315a0 con 0x7f2c640cf2b0 2023-10-08T07:16:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.140+0000 7f2c68c9a640 1 -- 172.21.15.70:0/3951819768 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60031930 con 0x7f2c640cf2b0 2023-10-08T07:16:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.141+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/3951819768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640cf2b0 msgr2=0x7f2c640d34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.141+0000 7f2c6ac9e640 1 --2- 172.21.15.70:0/3951819768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640cf2b0 0x7f2c640d34d0 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7f2c640cfb20 tx=0x7f2c6002b840 comp rx=0 tx=0).stop 2023-10-08T07:16:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.141+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/3951819768 shutdown_connections 2023-10-08T07:16:06.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.141+0000 7f2c6ac9e640 1 --2- 172.21.15.70:0/3951819768 >> v2:172.21.15.181:3300/0 conn(0x7f2c640ce510 0x7f2c6415c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.141+0000 7f2c6ac9e640 1 --2- 172.21.15.70:0/3951819768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640cf2b0 0x7f2c640d34d0 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.141+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/3951819768 >> 172.21.15.70:0/3951819768 conn(0x7f2c6409bc40 msgr2=0x7f2c640d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:06.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.141+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/3951819768 shutdown_connections 2023-10-08T07:16:06.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.141+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/3951819768 wait complete. 2023-10-08T07:16:06.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.143+0000 7f2c6ac9e640 1 Processor -- start 2023-10-08T07:16:06.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.143+0000 7f2c6ac9e640 1 -- start start 2023-10-08T07:16:06.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.144+0000 7f2c6ac9e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640ce510 0x7f2c6406afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.144+0000 7f2c6ac9e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c640cf2b0 0x7f2c6406b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.144+0000 7f2c6ac9e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c640cf690 0x7f2c6414bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.144+0000 7f2c6ac9e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6409c650 con 0x7f2c640ce510 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.144+0000 7f2c6ac9e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6414cde0 con 0x7f2c640cf2b0 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.144+0000 7f2c6ac9e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6414d0c0 con 0x7f2c640cf690 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.144+0000 7f2c69c9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640ce510 0x7f2c6406afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.144+0000 7f2c69c9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640ce510 0x7f2c6406afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57266/0 (socket says 172.21.15.70:57266) 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.144+0000 7f2c69c9c640 1 -- 172.21.15.70:0/4223362267 learned_addr learned my addr 172.21.15.70:0/4223362267 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c6949b640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c640cf2b0 0x7f2c6406b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c6a49d640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c640cf690 0x7f2c6414bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c69c9c640 1 -- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c640cf690 msgr2=0x7f2c6414bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c69c9c640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c640cf690 0x7f2c6414bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c69c9c640 1 -- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c640cf2b0 msgr2=0x7f2c6406b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c69c9c640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c640cf2b0 0x7f2c6406b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c69c9c640 1 -- 172.21.15.70:0/4223362267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c60009400 con 0x7f2c640ce510 2023-10-08T07:16:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c69c9c640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640ce510 0x7f2c6406afb0 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7f2c60034040 tx=0x7f2c600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c5a7fc640 1 -- 172.21.15.70:0/4223362267 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60004230 con 0x7f2c640ce510 2023-10-08T07:16:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c5a7fc640 1 -- 172.21.15.70:0/4223362267 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c60031f00 con 0x7f2c640ce510 2023-10-08T07:16:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/4223362267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6409c8f0 con 0x7f2c640ce510 2023-10-08T07:16:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c5a7fc640 1 -- 172.21.15.70:0/4223362267 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60032640 con 0x7f2c640ce510 2023-10-08T07:16:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.145+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/4223362267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c640cc5d0 con 0x7f2c640ce510 2023-10-08T07:16:06.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.146+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/4223362267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c640cc880 con 0x7f2c640ce510 2023-10-08T07:16:06.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.147+0000 7f2c5a7fc640 1 -- 172.21.15.70:0/4223362267 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 151) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f2c600327a0 con 0x7f2c640ce510 2023-10-08T07:16:06.153 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:06.148+0000 7f2754434040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:16:06.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.148+0000 7f2c5a7fc640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2c4806b050 0x7f2c4806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.148+0000 7f2c5a7fc640 1 -- 172.21.15.70:0/4223362267 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7f2c600abed0 con 0x7f2c640ce510 2023-10-08T07:16:06.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.149+0000 7f2c6949b640 1 -- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2c4806b050 msgr2=0x7f2c4806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1127370729 2023-10-08T07:16:06.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.149+0000 7f2c6949b640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2c4806b050 0x7f2c4806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:16:06.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.153+0000 7f2c5a7fc640 1 -- 172.21.15.70:0/4223362267 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c600767c0 con 0x7f2c640ce510 2023-10-08T07:16:06.193 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:06.192+0000 7f9d89613040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T07:16:06.207 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:06.206+0000 7f2754434040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:16:06.252 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:06.251+0000 7f9d89613040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T07:16:06.258 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:06.256+0000 7f2754434040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:16:06.304 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:06.303+0000 7f9d89613040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T07:16:06.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.340+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/4223362267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c64074150 con 0x7f2c640ce510 2023-10-08T07:16:06.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.341+0000 7f2c5a7fc640 1 -- 172.21.15.70:0/4223362267 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v277) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c60079de0 con 0x7f2c640ce510 2023-10-08T07:16:06.343 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:16:06.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2c4806b050 msgr2=0x7f2c4806d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:16:06.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2c4806b050 0x7f2c4806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/4223362267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640ce510 msgr2=0x7f2c6406afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:06.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640ce510 0x7f2c6406afb0 secure :-1 s=READY pgs=2951 cs=0 l=1 rev1=1 crypto rx=0x7f2c60034040 tx=0x7f2c600046c0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/4223362267 shutdown_connections 2023-10-08T07:16:06.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f2c4806b050 0x7f2c4806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c640cf690 0x7f2c6414bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c640cf2b0 0x7f2c6406b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 --2- 172.21.15.70:0/4223362267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c640ce510 0x7f2c6406afb0 unknown :-1 s=CLOSED pgs=2951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/4223362267 >> 172.21.15.70:0/4223362267 conn(0x7f2c6409bc40 msgr2=0x7f2c640b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:06.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/4223362267 shutdown_connections 2023-10-08T07:16:06.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.344+0000 7f2c6ac9e640 1 -- 172.21.15.70:0/4223362267 wait complete. 2023-10-08T07:16:06.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:16:06.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.620+0000 7f82ea6c2640 1 Processor -- start 2023-10-08T07:16:06.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.621+0000 7f82ea6c2640 1 -- start start 2023-10-08T07:16:06.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.622+0000 7f82ea6c2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f82e415cd70 0x7f82e41612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.622+0000 7f82ea6c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82e40b4e10 0x7f82e40d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.622+0000 7f82ea6c2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e40b8c30 con 0x7f82e40cd4e0 2023-10-08T07:16:06.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.622+0000 7f82ea6c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e40cdd80 con 0x7f82e40b4e10 2023-10-08T07:16:06.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.622+0000 7f82ea6c2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e40ce060 con 0x7f82e415cd70 2023-10-08T07:16:06.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.623+0000 7f82e8c5f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82e40b4e10 0x7f82e40d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:06.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.623+0000 7f82e8c5f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82e40b4e10 0x7f82e40d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57268/0 (socket says 172.21.15.70:57268) 2023-10-08T07:16:06.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.623+0000 7f82e8c5f640 1 -- 172.21.15.70:0/3684578451 learned_addr learned my addr 172.21.15.70:0/3684578451 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:06.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.623+0000 7f82e3fff640 1 --2- 172.21.15.70:0/3684578451 >> v2:172.21.15.181:3300/0 conn(0x7f82e415cd70 0x7f82e41612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:06.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.623+0000 7f82e8c5f640 1 -- 172.21.15.70:0/3684578451 >> v2:172.21.15.181:3300/0 conn(0x7f82e415cd70 msgr2=0x7f82e41612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:06.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.623+0000 7f82e8c5f640 1 --2- 172.21.15.70:0/3684578451 >> v2:172.21.15.181:3300/0 conn(0x7f82e415cd70 0x7f82e41612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.623+0000 7f82e8c5f640 1 -- 172.21.15.70:0/3684578451 >> v1:172.21.15.181:6789/0 conn(0x7f82e40cd4e0 legacy=0x7f82e40b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:06.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.623+0000 7f82e8c5f640 1 -- 172.21.15.70:0/3684578451 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e40ce340 con 0x7f82e40b4e10 2023-10-08T07:16:06.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.624+0000 7f82e8c5f640 1 --2- 172.21.15.70:0/3684578451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 0x7f82e40d1650 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f82e40d0800 tx=0x7f82dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07c7d3bb81467eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:06.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82e37fe640 1 -- 172.21.15.70:0/3684578451 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc02ca80 con 0x7f82e40b4e10 2023-10-08T07:16:06.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82e37fe640 1 -- 172.21.15.70:0/3684578451 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc02cbe0 con 0x7f82e40b4e10 2023-10-08T07:16:06.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82e37fe640 1 -- 172.21.15.70:0/3684578451 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0343f0 con 0x7f82e40b4e10 2023-10-08T07:16:06.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82ea6c2640 1 -- 172.21.15.70:0/3684578451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 msgr2=0x7f82e40d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:06.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82ea6c2640 1 --2- 172.21.15.70:0/3684578451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 0x7f82e40d1650 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f82e40d0800 tx=0x7f82dc02bee0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82ea6c2640 1 -- 172.21.15.70:0/3684578451 shutdown_connections 2023-10-08T07:16:06.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82ea6c2640 1 --2- 172.21.15.70:0/3684578451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 0x7f82e40d1650 unknown :-1 s=CLOSED pgs=2952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82ea6c2640 1 --2- 172.21.15.70:0/3684578451 >> v2:172.21.15.181:3300/0 conn(0x7f82e415cd70 0x7f82e41612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82ea6c2640 1 -- 172.21.15.70:0/3684578451 >> 172.21.15.70:0/3684578451 conn(0x7f82e40cf700 msgr2=0x7f82e40cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:06.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82ea6c2640 1 -- 172.21.15.70:0/3684578451 shutdown_connections 2023-10-08T07:16:06.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.625+0000 7f82ea6c2640 1 -- 172.21.15.70:0/3684578451 wait complete. 2023-10-08T07:16:06.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.627+0000 7f82ea6c2640 1 Processor -- start 2023-10-08T07:16:06.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.627+0000 7f82ea6c2640 1 -- start start 2023-10-08T07:16:06.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.628+0000 7f82ea6c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 0x7f82e4150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.628+0000 7f82ea6c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82e40cd4e0 0x7f82e40c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.628+0000 7f82ea6c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e415cd70 0x7f82e40c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.628+0000 7f82ea6c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e41514a0 con 0x7f82e40b4e10 2023-10-08T07:16:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.628+0000 7f82ea6c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40c6970 con 0x7f82e415cd70 2023-10-08T07:16:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.628+0000 7f82ea6c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40c6c50 con 0x7f82e40cd4e0 2023-10-08T07:16:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.628+0000 7f82e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 0x7f82e4150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 0x7f82e4150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57272/0 (socket says 172.21.15.70:57272) 2023-10-08T07:16:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82e3fff640 1 -- 172.21.15.70:0/807821405 learned_addr learned my addr 172.21.15.70:0/807821405 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82e8c5f640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e415cd70 0x7f82e40c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82e3fff640 1 -- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82e40cd4e0 msgr2=0x7f82e40c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82dbfff640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82e40cd4e0 0x7f82e40c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82e3fff640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82e40cd4e0 0x7f82e40c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82e3fff640 1 -- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e415cd70 msgr2=0x7f82e40c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82e3fff640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e415cd70 0x7f82e40c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82e3fff640 1 -- 172.21.15.70:0/807821405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc009b30 con 0x7f82e40b4e10 2023-10-08T07:16:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82e8c5f640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e415cd70 0x7f82e40c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.629+0000 7f82e3fff640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 0x7f82e4150f60 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f82d400d800 tx=0x7f82d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.630+0000 7f82e17fa640 1 -- 172.21.15.70:0/807821405 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d400b6c0 con 0x7f82e40b4e10 2023-10-08T07:16:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.630+0000 7f82e17fa640 1 -- 172.21.15.70:0/807821405 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d400b820 con 0x7f82e40b4e10 2023-10-08T07:16:06.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.630+0000 7f82e17fa640 1 -- 172.21.15.70:0/807821405 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d400bbb0 con 0x7f82e40b4e10 2023-10-08T07:16:06.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.630+0000 7f82ea6c2640 1 -- 172.21.15.70:0/807821405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e40c6f30 con 0x7f82e40b4e10 2023-10-08T07:16:06.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.630+0000 7f82ea6c2640 1 -- 172.21.15.70:0/807821405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e40c74f0 con 0x7f82e40b4e10 2023-10-08T07:16:06.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.631+0000 7f82ea6c2640 1 -- 172.21.15.70:0/807821405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e40c76b0 con 0x7f82e40b4e10 2023-10-08T07:16:06.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.632+0000 7f82e17fa640 1 -- 172.21.15.70:0/807821405 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 151) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f82d400bd10 con 0x7f82e40b4e10 2023-10-08T07:16:06.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.632+0000 7f82e17fa640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f82c806b020 0x7f82c806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:06.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.633+0000 7f82e17fa640 1 -- 172.21.15.70:0/807821405 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7f82d408a270 con 0x7f82e40b4e10 2023-10-08T07:16:06.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.633+0000 7f82dbfff640 1 -- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f82c806b020 msgr2=0x7f82c806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1127370729 2023-10-08T07:16:06.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.633+0000 7f82dbfff640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f82c806b020 0x7f82c806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:16:06.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.637+0000 7f82e17fa640 1 -- 172.21.15.70:0/807821405 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82d4054d00 con 0x7f82e40b4e10 2023-10-08T07:16:06.648 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:06.646+0000 7f2754434040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:16:06.709 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:06.706+0000 7f9d89613040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T07:16:06.808 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:06.807+0000 7f2754434040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:16:06.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.829+0000 7f82ea6c2640 1 -- 172.21.15.70:0/807821405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82e40d1100 con 0x7f82e40b4e10 2023-10-08T07:16:06.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.830+0000 7f82e17fa640 1 -- 172.21.15.70:0/807821405 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v277) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82d4058320 con 0x7f82e40b4e10 2023-10-08T07:16:06.831 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:16:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 -- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f82c806b020 msgr2=0x7f82c806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:16:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f82c806b020 0x7f82c806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 -- 172.21.15.70:0/807821405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 msgr2=0x7f82e4150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 0x7f82e4150f60 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f82d400d800 tx=0x7f82d400dcc0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 -- 172.21.15.70:0/807821405 shutdown_connections 2023-10-08T07:16:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f82c806b020 0x7f82c806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f82e415cd70 0x7f82e40c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f82e40cd4e0 0x7f82e40c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 --2- 172.21.15.70:0/807821405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e40b4e10 0x7f82e4150f60 unknown :-1 s=CLOSED pgs=2953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 -- 172.21.15.70:0/807821405 >> 172.21.15.70:0/807821405 conn(0x7f82e40cf700 msgr2=0x7f82e40d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 -- 172.21.15.70:0/807821405 shutdown_connections 2023-10-08T07:16:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:06.833+0000 7f82ea6c2640 1 -- 172.21.15.70:0/807821405 wait complete. 2023-10-08T07:16:06.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-08T07:16:06.883 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:06.881+0000 7f9d89613040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T07:16:06.917 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:06.915+0000 7f2754434040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:16:06.966 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:06.965+0000 7f2754434040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:16:06.996 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:06.994+0000 7f9d89613040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T07:16:07.048 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:07.047+0000 7f9d89613040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T07:16:07.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.061+0000 7fda81dd0640 1 Processor -- start 2023-10-08T07:16:07.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.062+0000 7fda81dd0640 1 -- start start 2023-10-08T07:16:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.063+0000 7fda81dd0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fda7c160660 0x7fda7c070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.063+0000 7fda81dd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda7c0cb620 0x7fda7c0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.063+0000 7fda81dd0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7c06e150 con 0x7fda7c0c9910 2023-10-08T07:16:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.063+0000 7fda81dd0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7c06de70 con 0x7fda7c160660 2023-10-08T07:16:07.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.063+0000 7fda81dd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0c8e30 con 0x7fda7c0cb620 2023-10-08T07:16:07.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.064+0000 7fda7b7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fda7c160660 0x7fda7c070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:07.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.064+0000 7fda7bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda7c0cb620 0x7fda7c0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:07.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.064+0000 7fda7b7fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fda7c160660 0x7fda7c070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:51450/0 (socket says 172.21.15.70:51450) 2023-10-08T07:16:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.064+0000 7fda7b7fe640 1 -- 172.21.15.70:0/3609426673 learned_addr learned my addr 172.21.15.70:0/3609426673 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.065+0000 7fda7b7fe640 1 -- 172.21.15.70:0/3609426673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda7c0cb620 msgr2=0x7fda7c0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.065+0000 7fda7b7fe640 1 --2- 172.21.15.70:0/3609426673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda7c0cb620 0x7fda7c0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.065+0000 7fda7b7fe640 1 -- 172.21.15.70:0/3609426673 >> v1:172.21.15.181:6789/0 conn(0x7fda7c0c9910 legacy=0x7fda7c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.065+0000 7fda7b7fe640 1 -- 172.21.15.70:0/3609426673 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c0c9110 con 0x7fda7c160660 2023-10-08T07:16:07.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.066+0000 7fda7b7fe640 1 --2- 172.21.15.70:0/3609426673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda7c160660 0x7fda7c070cc0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009870 tx=0x7fda6c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f85172877543641 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:07.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.066+0000 7fda7affd640 1 -- 172.21.15.70:0/3609426673 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c030840 con 0x7fda7c160660 2023-10-08T07:16:07.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.067+0000 7fda7affd640 1 -- 172.21.15.70:0/3609426673 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda6c0309a0 con 0x7fda7c160660 2023-10-08T07:16:07.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.067+0000 7fda7affd640 1 -- 172.21.15.70:0/3609426673 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c030d30 con 0x7fda7c160660 2023-10-08T07:16:07.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.067+0000 7fda81dd0640 1 -- 172.21.15.70:0/3609426673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda7c160660 msgr2=0x7fda7c070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.067+0000 7fda81dd0640 1 --2- 172.21.15.70:0/3609426673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda7c160660 0x7fda7c070cc0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009870 tx=0x7fda6c02b980 comp rx=0 tx=0).stop 2023-10-08T07:16:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.067+0000 7fda81dd0640 1 -- 172.21.15.70:0/3609426673 shutdown_connections 2023-10-08T07:16:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.067+0000 7fda81dd0640 1 --2- 172.21.15.70:0/3609426673 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda7c0cb620 0x7fda7c0b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.067+0000 7fda81dd0640 1 --2- 172.21.15.70:0/3609426673 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda7c160660 0x7fda7c070cc0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.067+0000 7fda81dd0640 1 -- 172.21.15.70:0/3609426673 >> 172.21.15.70:0/3609426673 conn(0x7fda7c06a5d0 msgr2=0x7fda7c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.067+0000 7fda81dd0640 1 -- 172.21.15.70:0/3609426673 shutdown_connections 2023-10-08T07:16:07.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.067+0000 7fda81dd0640 1 -- 172.21.15.70:0/3609426673 wait complete. 2023-10-08T07:16:07.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.069+0000 7fda81dd0640 1 Processor -- start 2023-10-08T07:16:07.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.069+0000 7fda81dd0640 1 -- start start 2023-10-08T07:16:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.070+0000 7fda81dd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda7c0c9910 0x7fda7c146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.070+0000 7fda81dd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda7c0cb620 0x7fda7c153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.070+0000 7fda81dd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda7c160660 0x7fda7c147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.070+0000 7fda81dd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c158870 con 0x7fda7c0cb620 2023-10-08T07:16:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.070+0000 7fda81dd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c158b50 con 0x7fda7c0c9910 2023-10-08T07:16:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.070+0000 7fda81dd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c158e30 con 0x7fda7c160660 2023-10-08T07:16:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.070+0000 7fda73fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda7c0cb620 0x7fda7c153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.070+0000 7fda73fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda7c0cb620 0x7fda7c153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57280/0 (socket says 172.21.15.70:57280) 2023-10-08T07:16:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.070+0000 7fda73fff640 1 -- 172.21.15.70:0/1316487642 learned_addr learned my addr 172.21.15.70:0/1316487642 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda7b7fe640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda7c0c9910 0x7fda7c146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda7bfff640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda7c160660 0x7fda7c147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda73fff640 1 -- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda7c160660 msgr2=0x7fda7c147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda73fff640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda7c160660 0x7fda7c147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda73fff640 1 -- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda7c0c9910 msgr2=0x7fda7c146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda73fff640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda7c0c9910 0x7fda7c146ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda73fff640 1 -- 172.21.15.70:0/1316487642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6c0094d0 con 0x7fda7c0cb620 2023-10-08T07:16:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda73fff640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda7c0cb620 0x7fda7c153ee0 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7fda6400c450 tx=0x7fda6400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda78ff9640 1 -- 172.21.15.70:0/1316487642 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda640052d0 con 0x7fda7c0cb620 2023-10-08T07:16:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda78ff9640 1 -- 172.21.15.70:0/1316487642 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda64005430 con 0x7fda7c0cb620 2023-10-08T07:16:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.071+0000 7fda78ff9640 1 -- 172.21.15.70:0/1316487642 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64012870 con 0x7fda7c0cb620 2023-10-08T07:16:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.072+0000 7fda81dd0640 1 -- 172.21.15.70:0/1316487642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c159110 con 0x7fda7c0cb620 2023-10-08T07:16:07.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.072+0000 7fda81dd0640 1 -- 172.21.15.70:0/1316487642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c147010 con 0x7fda7c0cb620 2023-10-08T07:16:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.073+0000 7fda81dd0640 1 -- 172.21.15.70:0/1316487642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c159630 con 0x7fda7c0cb620 2023-10-08T07:16:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.074+0000 7fda78ff9640 1 -- 172.21.15.70:0/1316487642 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 151) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fda640129d0 con 0x7fda7c0cb620 2023-10-08T07:16:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.074+0000 7fda78ff9640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fda5c06b020 0x7fda5c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.075+0000 7fda78ff9640 1 -- 172.21.15.70:0/1316487642 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7fda640898a0 con 0x7fda7c0cb620 2023-10-08T07:16:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.075+0000 7fda7b7fe640 1 -- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fda5c06b020 msgr2=0x7fda5c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1127370729 2023-10-08T07:16:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.075+0000 7fda7b7fe640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fda5c06b020 0x7fda5c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:16:07.078 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:07.075+0000 7f2754434040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:16:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.079+0000 7fda78ff9640 1 -- 172.21.15.70:0/1316487642 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda7c159630 con 0x7fda7c0cb620 2023-10-08T07:16:07.164 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:07.162+0000 7f9d89613040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:16:07.185 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:07.184+0000 7f2754434040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:16:07.275 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:07.273+0000 7f9d89613040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T07:16:07.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.276+0000 7fda7b7fe640 1 -- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fda5c06b020 msgr2=0x7fda5c06d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1127370729 2023-10-08T07:16:07.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.276+0000 7fda7b7fe640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fda5c06b020 0x7fda5c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:16:07.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.283+0000 7fda81dd0640 1 -- 172.21.15.70:0/1316487642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fda7c159630 con 0x7fda7c0cb620 2023-10-08T07:16:07.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.283+0000 7fda78ff9640 1 -- 172.21.15.70:0/1316487642 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v277) v1 ==== 81+0+649 (secure 0 0 0) 0x7fda7c159630 con 0x7fda7c0cb620 2023-10-08T07:16:07.285 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3814802416 2023-10-09T07:16:00.570875+0000 2023-10-08T07:16:07.285 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3705279960 2023-10-09T07:16:00.570875+0000 2023-10-08T07:16:07.285 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6833/440573135 2023-10-09T07:16:00.570875+0000 2023-10-08T07:16:07.285 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:6832/440573135 2023-10-09T07:16:00.570875+0000 2023-10-08T07:16:07.285 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1545538679 2023-10-09T07:16:00.570875+0000 2023-10-08T07:16:07.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1151602740 2023-10-09T07:16:00.570875+0000 2023-10-08T07:16:07.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2253796914 2023-10-09T07:16:00.570875+0000 2023-10-08T07:16:07.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/3307173154 2023-10-09T07:16:00.570875+0000 2023-10-08T07:16:07.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/2296566554 2023-10-09T07:16:00.570875+0000 2023-10-08T07:16:07.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/606980 2023-10-09T07:16:00.570875+0000 2023-10-08T07:16:07.286 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.15.181:0/1513601554 2023-10-08T08:15:42.823407+0000 2023-10-08T07:16:07.286 INFO:teuthology.orchestra.run.smithi070.stderr:listed 11 entries 2023-10-08T07:16:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 -- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fda5c06b020 msgr2=0x7fda5c06d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:16:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fda5c06b020 0x7fda5c06d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 -- 172.21.15.70:0/1316487642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda7c0cb620 msgr2=0x7fda7c153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda7c0cb620 0x7fda7c153ee0 secure :-1 s=READY pgs=2954 cs=0 l=1 rev1=1 crypto rx=0x7fda6400c450 tx=0x7fda6400c910 comp rx=0 tx=0).stop 2023-10-08T07:16:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 -- 172.21.15.70:0/1316487642 shutdown_connections 2023-10-08T07:16:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fda5c06b020 0x7fda5c06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fda7c160660 0x7fda7c147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda7c0cb620 0x7fda7c153ee0 unknown :-1 s=CLOSED pgs=2954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 --2- 172.21.15.70:0/1316487642 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fda7c0c9910 0x7fda7c146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 -- 172.21.15.70:0/1316487642 >> 172.21.15.70:0/1316487642 conn(0x7fda7c06a5d0 msgr2=0x7fda7c073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 -- 172.21.15.70:0/1316487642 shutdown_connections 2023-10-08T07:16:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.286+0000 7fda81dd0640 1 -- 172.21.15.70:0/1316487642 wait complete. 2023-10-08T07:16:07.305 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T07:16:07.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:07.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T07:16:07.344 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:07.342+0000 7f2754434040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:16:07.393 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:07.392+0000 7f2754434040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:16:07.441 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:07.439+0000 7f9d89613040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T07:16:07.475 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T07:16:07.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:07.476 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T07:16:07.495 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:07.494+0000 7f9d89613040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T07:16:07.515 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:07.514+0000 7f2754434040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:16:07.521 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:16:07.521 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:16:07.577 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:16:07.578 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:16:07.578 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:16:07.619 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:07.617+0000 7f9d89613040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T07:16:07.634 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:16:07.634 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:16:07.635 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:16:07.690 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:16:07.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:16:07.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.895+0000 7fe15759e640 1 Processor -- start 2023-10-08T07:16:07.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.895+0000 7fe15759e640 1 -- start start 2023-10-08T07:16:07.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.896+0000 7fe15759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580adc70 0x7fe1580b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.896+0000 7fe15759e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe15815af10 0x7fe1580b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.896+0000 7fe15759e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15815f7d0 con 0x7fe15815ab30 2023-10-08T07:16:07.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.896+0000 7fe15759e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15815efb0 con 0x7fe15815af10 2023-10-08T07:16:07.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.896+0000 7fe15759e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15815f210 con 0x7fe1580adc70 2023-10-08T07:16:07.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.897+0000 7fe15659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe15815ab30 0x7fe15815e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49010/0 (socket says 172.21.15.70:49010) 2023-10-08T07:16:07.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.897+0000 7fe15659c640 1 -- 172.21.15.70:0/3248534225 learned_addr learned my addr 172.21.15.70:0/3248534225 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:07.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.897+0000 7fe156d9d640 1 --2- 172.21.15.70:0/3248534225 >> v2:172.21.15.70:3300/0 conn(0x7fe15815af10 0x7fe1580b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:07.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.897+0000 7fe14ffff640 1 --2- 172.21.15.70:0/3248534225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580adc70 0x7fe1580b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:07.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.897+0000 7fe155d9b640 1 -- 172.21.15.70:0/3248534225 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655921369 0 0) 0x7fe15815f7d0 con 0x7fe15815ab30 2023-10-08T07:16:07.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.897+0000 7fe155d9b640 1 -- 172.21.15.70:0/3248534225 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe144003520 con 0x7fe15815ab30 2023-10-08T07:16:07.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.897+0000 7fe156d9d640 1 -- 172.21.15.70:0/3248534225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580adc70 msgr2=0x7fe1580b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:07.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.897+0000 7fe156d9d640 1 --2- 172.21.15.70:0/3248534225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580adc70 0x7fe1580b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.897+0000 7fe156d9d640 1 -- 172.21.15.70:0/3248534225 >> v1:172.21.15.70:6789/0 conn(0x7fe15815ab30 legacy=0x7fe15815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:07.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.897+0000 7fe156d9d640 1 -- 172.21.15.70:0/3248534225 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15815f4f0 con 0x7fe15815af10 2023-10-08T07:16:07.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.899+0000 7fe156d9d640 1 --2- 172.21.15.70:0/3248534225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe15815af10 0x7fe1580b0600 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7fe150009d90 tx=0x7fe15002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eebe62b26f92c72 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:07.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe155d9b640 1 -- 172.21.15.70:0/3248534225 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15002cc50 con 0x7fe15815af10 2023-10-08T07:16:07.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe155d9b640 1 -- 172.21.15.70:0/3248534225 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15002cdb0 con 0x7fe15815af10 2023-10-08T07:16:07.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe155d9b640 1 -- 172.21.15.70:0/3248534225 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe150034640 con 0x7fe15815af10 2023-10-08T07:16:07.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe15759e640 1 -- 172.21.15.70:0/3248534225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe15815af10 msgr2=0x7fe1580b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:07.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe15759e640 1 --2- 172.21.15.70:0/3248534225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe15815af10 0x7fe1580b0600 secure :-1 s=READY pgs=2956 cs=0 l=1 rev1=1 crypto rx=0x7fe150009d90 tx=0x7fe15002bea0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe15759e640 1 -- 172.21.15.70:0/3248534225 shutdown_connections 2023-10-08T07:16:07.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe15759e640 1 --2- 172.21.15.70:0/3248534225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe15815af10 0x7fe1580b0600 unknown :-1 s=CLOSED pgs=2956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe15759e640 1 --2- 172.21.15.70:0/3248534225 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe1580adc70 0x7fe1580b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe15759e640 1 -- 172.21.15.70:0/3248534225 >> 172.21.15.70:0/3248534225 conn(0x7fe1580cb0c0 msgr2=0x7fe1580cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:07.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe15759e640 1 -- 172.21.15.70:0/3248534225 shutdown_connections 2023-10-08T07:16:07.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.900+0000 7fe15759e640 1 -- 172.21.15.70:0/3248534225 wait complete. 2023-10-08T07:16:07.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.902+0000 7fe15759e640 1 Processor -- start 2023-10-08T07:16:07.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.902+0000 7fe15759e640 1 -- start start 2023-10-08T07:16:07.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.903+0000 7fe15759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1580adc70 0x7fe158147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.903+0000 7fe15759e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15815ab30 0x7fe1580c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.903+0000 7fe15759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15815af10 0x7fe1580c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.903+0000 7fe15759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe158148040 con 0x7fe1580adc70 2023-10-08T07:16:07.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.903+0000 7fe15759e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580c7860 con 0x7fe15815ab30 2023-10-08T07:16:07.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.903+0000 7fe15759e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580c2cb0 con 0x7fe15815af10 2023-10-08T07:16:07.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.903+0000 7fe15659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1580adc70 0x7fe158147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.903+0000 7fe15659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1580adc70 0x7fe158147b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57286/0 (socket says 172.21.15.70:57286) 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.903+0000 7fe15659c640 1 -- 172.21.15.70:0/497157330 learned_addr learned my addr 172.21.15.70:0/497157330 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe156d9d640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15815af10 0x7fe1580c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe14ffff640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15815ab30 0x7fe1580c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe15659c640 1 -- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15815af10 msgr2=0x7fe1580c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe15659c640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15815af10 0x7fe1580c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe15659c640 1 -- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15815ab30 msgr2=0x7fe1580c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe15659c640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15815ab30 0x7fe1580c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe15659c640 1 -- 172.21.15.70:0/497157330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150009a70 con 0x7fe1580adc70 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe14ffff640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15815ab30 0x7fe1580c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:07.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe15659c640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1580adc70 0x7fe158147b00 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7fe1480010e0 tx=0x7fe14800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe14effd640 1 -- 172.21.15.70:0/497157330 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148005ce0 con 0x7fe1580adc70 2023-10-08T07:16:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe14effd640 1 -- 172.21.15.70:0/497157330 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe148005e60 con 0x7fe1580adc70 2023-10-08T07:16:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe14effd640 1 -- 172.21.15.70:0/497157330 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480072b0 con 0x7fe1580adc70 2023-10-08T07:16:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe15759e640 1 -- 172.21.15.70:0/497157330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1580c2f90 con 0x7fe1580adc70 2023-10-08T07:16:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.904+0000 7fe15759e640 1 -- 172.21.15.70:0/497157330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1580c74f0 con 0x7fe1580adc70 2023-10-08T07:16:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.905+0000 7fe15759e640 1 -- 172.21.15.70:0/497157330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15804e160 con 0x7fe1580adc70 2023-10-08T07:16:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.907+0000 7fe14effd640 1 -- 172.21.15.70:0/497157330 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 151) v1 ==== 89388+0+0 (secure 0 0 0) 0x7fe148007410 con 0x7fe1580adc70 2023-10-08T07:16:07.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.907+0000 7fe14effd640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fe13806b050 0x7fe13806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:07.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.908+0000 7fe14effd640 1 -- 172.21.15.70:0/497157330 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7fe1480892f0 con 0x7fe1580adc70 2023-10-08T07:16:07.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.908+0000 7fe14ffff640 1 -- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fe13806b050 msgr2=0x7fe13806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1127370729 2023-10-08T07:16:07.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.908+0000 7fe14ffff640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fe13806b050 0x7fe13806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:16:07.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:07.912+0000 7fe14effd640 1 -- 172.21.15.70:0/497157330 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe148053d80 con 0x7fe1580adc70 2023-10-08T07:16:08.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.109+0000 7fe14ffff640 1 -- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fe13806b050 msgr2=0x7fe13806d510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1127370729 2023-10-08T07:16:08.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.109+0000 7fe14ffff640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fe13806b050 0x7fe13806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T07:16:08.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.138+0000 7fe15759e640 1 -- 172.21.15.70:0/497157330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1580c76b0 con 0x7fe1580adc70 2023-10-08T07:16:08.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.138+0000 7fe14effd640 1 -- 172.21.15.70:0/497157330 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4923 (secure 0 0 0) 0x7fe1480573a0 con 0x7fe1580adc70 2023-10-08T07:16:08.141 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:08.141 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":140,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/949448327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":949448327},{"type":"v1","addr":"172.21.15.70:6835","nonce":949448327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":8111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2423877406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2423877406},{"type":"v1","addr":"172.21.15.181:6837","nonce":2423877406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/247109890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":247109890},{"type":"v1","addr":"172.21.15.181:6836","nonce":247109890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:15:45.890217+0000","modified":"2023-10-08T07:15:48.920991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12645},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12645":{"gid":12645,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/753129368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":753129368},{"type":"v1","addr":"172.21.15.70:6834","nonce":753129368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-08T07:16:08.141 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 140 2023-10-08T07:16:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 -- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fe13806b050 msgr2=0x7fe13806d510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:16:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fe13806b050 0x7fe13806d510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 -- 172.21.15.70:0/497157330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1580adc70 msgr2=0x7fe158147b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1580adc70 0x7fe158147b00 secure :-1 s=READY pgs=2957 cs=0 l=1 rev1=1 crypto rx=0x7fe1480010e0 tx=0x7fe14800c960 comp rx=0 tx=0).stop 2023-10-08T07:16:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 -- 172.21.15.70:0/497157330 shutdown_connections 2023-10-08T07:16:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7fe13806b050 0x7fe13806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe15815af10 0x7fe1580c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe15815ab30 0x7fe1580c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 --2- 172.21.15.70:0/497157330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1580adc70 0x7fe158147b00 unknown :-1 s=CLOSED pgs=2957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 -- 172.21.15.70:0/497157330 >> 172.21.15.70:0/497157330 conn(0x7fe1580cb0c0 msgr2=0x7fe1580cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 -- 172.21.15.70:0/497157330 shutdown_connections 2023-10-08T07:16:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.141+0000 7fe15759e640 1 -- 172.21.15.70:0/497157330 wait complete. 2023-10-08T07:16:08.160 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:16:08.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.340+0000 7f6ff3402640 1 Processor -- start 2023-10-08T07:16:08.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.340+0000 7f6ff3402640 1 -- start start 2023-10-08T07:16:08.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6ff3402640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fec0af8b0 0x7f6fec06ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6ff3402640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6fec0b0ab0 0x7f6fec0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6ff3402640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec0b0340 con 0x7f6fec06b520 2023-10-08T07:16:08.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6ff3402640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fec06b3a0 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6ff3402640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0b0150 con 0x7f6fec0af8b0 2023-10-08T07:16:08.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6ff119e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6fec06b520 0x7f6fec06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49024/0 (socket says 172.21.15.70:49024) 2023-10-08T07:16:08.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6ff119e640 1 -- 172.21.15.70:0/2080417090 learned_addr learned my addr 172.21.15.70:0/2080417090 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:08.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6ff199f640 1 --2- 172.21.15.70:0/2080417090 >> v2:172.21.15.70:3300/0 conn(0x7f6fec0b0ab0 0x7f6fec0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:08.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6ff099d640 1 --2- 172.21.15.70:0/2080417090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fec0af8b0 0x7f6fec06ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:08.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6fe3fff640 1 -- 172.21.15.70:0/2080417090 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373293857 0 0) 0x7f6fec0b0340 con 0x7f6fec06b520 2023-10-08T07:16:08.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6fe3fff640 1 -- 172.21.15.70:0/2080417090 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd8003520 con 0x7f6fec06b520 2023-10-08T07:16:08.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.342+0000 7f6ff199f640 1 -- 172.21.15.70:0/2080417090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fec0af8b0 msgr2=0x7f6fec06ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:08.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.343+0000 7f6ff199f640 1 --2- 172.21.15.70:0/2080417090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fec0af8b0 0x7f6fec06ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.343+0000 7f6ff199f640 1 -- 172.21.15.70:0/2080417090 >> v1:172.21.15.70:6789/0 conn(0x7f6fec06b520 legacy=0x7f6fec06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:08.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.343+0000 7f6ff199f640 1 -- 172.21.15.70:0/2080417090 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0cf8f0 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.344+0000 7f6ff199f640 1 --2- 172.21.15.70:0/2080417090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 0x7f6fec0b0f40 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009d90 tx=0x7f6fe402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b1420aa20b2160 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:08.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.344+0000 7f6fe3fff640 1 -- 172.21.15.70:0/2080417090 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe402cbd0 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.344+0000 7f6fe3fff640 1 -- 172.21.15.70:0/2080417090 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe402cd30 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.344+0000 7f6fe3fff640 1 -- 172.21.15.70:0/2080417090 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe402cea0 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.344+0000 7f6ff3402640 1 -- 172.21.15.70:0/2080417090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 msgr2=0x7f6fec0b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:08.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.344+0000 7f6ff3402640 1 --2- 172.21.15.70:0/2080417090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 0x7f6fec0b0f40 secure :-1 s=READY pgs=2959 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009d90 tx=0x7f6fe402be20 comp rx=0 tx=0).stop 2023-10-08T07:16:08.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.345+0000 7f6ff3402640 1 -- 172.21.15.70:0/2080417090 shutdown_connections 2023-10-08T07:16:08.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.345+0000 7f6ff3402640 1 --2- 172.21.15.70:0/2080417090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 0x7f6fec0b0f40 unknown :-1 s=CLOSED pgs=2959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.345+0000 7f6ff3402640 1 --2- 172.21.15.70:0/2080417090 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fec0af8b0 0x7f6fec06ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.345+0000 7f6ff3402640 1 -- 172.21.15.70:0/2080417090 >> 172.21.15.70:0/2080417090 conn(0x7f6fec0d09e0 msgr2=0x7f6fec0d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:08.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.345+0000 7f6ff3402640 1 -- 172.21.15.70:0/2080417090 shutdown_connections 2023-10-08T07:16:08.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.345+0000 7f6ff3402640 1 -- 172.21.15.70:0/2080417090 wait complete. 2023-10-08T07:16:08.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.346+0000 7f6ff3402640 1 Processor -- start 2023-10-08T07:16:08.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.347+0000 7f6ff3402640 1 -- start start 2023-10-08T07:16:08.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.347+0000 7f6ff3402640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fec06b520 0x7f6fec158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff3402640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fec0af8b0 0x7f6fec0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff3402640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 0x7f6fec1f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff3402640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec09d920 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff3402640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec09d2c0 con 0x7f6fec0af8b0 2023-10-08T07:16:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff3402640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec1f6240 con 0x7f6fec06b520 2023-10-08T07:16:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff199f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 0x7f6fec1f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff199f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 0x7f6fec1f57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57304/0 (socket says 172.21.15.70:57304) 2023-10-08T07:16:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff199f640 1 -- 172.21.15.70:0/2219317455 learned_addr learned my addr 172.21.15.70:0/2219317455 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff099d640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fec0af8b0 0x7f6fec0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff119e640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fec06b520 0x7f6fec158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff199f640 1 -- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fec06b520 msgr2=0x7f6fec158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:08.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff199f640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fec06b520 0x7f6fec158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff199f640 1 -- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fec0af8b0 msgr2=0x7f6fec0c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:08.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff199f640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fec0af8b0 0x7f6fec0c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff199f640 1 -- 172.21.15.70:0/2219317455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4009a70 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.348+0000 7f6ff099d640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fec0af8b0 0x7f6fec0c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:16:08.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.349+0000 7f6ff199f640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 0x7f6fec1f57f0 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009eb0 tx=0x7f6fe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.349+0000 7f6fe1ffb640 1 -- 172.21.15.70:0/2219317455 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe402cea0 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.349+0000 7f6fe1ffb640 1 -- 172.21.15.70:0/2219317455 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe403e610 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.349+0000 7f6fe1ffb640 1 -- 172.21.15.70:0/2219317455 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe403e9c0 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.349+0000 7f6ff3402640 1 -- 172.21.15.70:0/2219317455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec1f6520 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.349+0000 7f6ff3402640 1 -- 172.21.15.70:0/2219317455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec159310 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.351+0000 7f6ff3402640 1 -- 172.21.15.70:0/2219317455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec09da60 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.351+0000 7f6fe1ffb640 1 -- 172.21.15.70:0/2219317455 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 151) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f6fe403b050 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.352+0000 7f6fe1ffb640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6fd006b020 0x7f6fd006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.352+0000 7f6fe1ffb640 1 -- 172.21.15.70:0/2219317455 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7f6fe40abe40 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.353+0000 7f6ff119e640 1 -- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6fd006b020 msgr2=0x7f6fd006d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1127370729 2023-10-08T07:16:08.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.353+0000 7f6ff119e640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6fd006b020 0x7f6fd006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:16:08.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.356+0000 7f6fe1ffb640 1 -- 172.21.15.70:0/2219317455 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6fec09da60 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.379 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.378+0000 7f2754434040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:16:08.457 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:08.455+0000 7f9d89613040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T07:16:08.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.548+0000 7f6ff3402640 1 -- 172.21.15.70:0/2219317455 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fec09da60 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.550+0000 7f6fe1ffb640 1 -- 172.21.15.70:0/2219317455 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v277) v1 ==== 74+0+17586 (secure 0 0 0) 0x7f6fec09da60 con 0x7f6fec0b0ab0 2023-10-08T07:16:08.551 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:08.551 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":277,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:16:00.570930+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:15:43.819948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-08T07:15:44.895969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3814802416":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/3705279960":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6833/440573135":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6832/440573135":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/1545538679":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/1151602740":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2253796914":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/3307173154":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2296566554":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/606980":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/1513601554":"2023-10-08T08:15:42.823407+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:16:08.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.552+0000 7f6ff3402640 1 -- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6fd006b020 msgr2=0x7f6fd006d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:16:08.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.552+0000 7f6ff3402640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6fd006b020 0x7f6fd006d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.553+0000 7f6ff3402640 1 -- 172.21.15.70:0/2219317455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 msgr2=0x7f6fec1f57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:08.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.553+0000 7f6ff3402640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 0x7f6fec1f57f0 secure :-1 s=READY pgs=2960 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009eb0 tx=0x7f6fe4004a40 comp rx=0 tx=0).stop 2023-10-08T07:16:08.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.553+0000 7f6ff3402640 1 -- 172.21.15.70:0/2219317455 shutdown_connections 2023-10-08T07:16:08.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.553+0000 7f6ff3402640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f6fd006b020 0x7f6fd006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.553+0000 7f6ff3402640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6fec0b0ab0 0x7f6fec1f57f0 unknown :-1 s=CLOSED pgs=2960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.553+0000 7f6ff3402640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f6fec0af8b0 0x7f6fec0c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.553+0000 7f6ff3402640 1 --2- 172.21.15.70:0/2219317455 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f6fec06b520 0x7f6fec158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.553+0000 7f6ff3402640 1 -- 172.21.15.70:0/2219317455 >> 172.21.15.70:0/2219317455 conn(0x7f6fec0d09e0 msgr2=0x7f6fec0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:08.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.553+0000 7f6ff3402640 1 -- 172.21.15.70:0/2219317455 shutdown_connections 2023-10-08T07:16:08.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.553+0000 7f6ff3402640 1 -- 172.21.15.70:0/2219317455 wait complete. 2023-10-08T07:16:08.570 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T05:36:05.132285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 35, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T07:15:43.819948+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '275', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 36, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T07:15:44.895969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '276', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-08T07:16:08.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T07:16:08.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:16:08.662 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:16:08.679 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.678+0000 7f2754434040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:16:08.742 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.741+0000 7f2754434040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:16:08.767 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:08.766+0000 7f9d89613040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T07:16:08.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.773+0000 7f8a71c10640 1 Processor -- start 2023-10-08T07:16:08.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.773+0000 7f8a71c10640 1 -- start start 2023-10-08T07:16:08.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a71c10640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8a6c0b1750 0x7f8a6c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a71c10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a6c06d240 0x7f8a6c070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a71c10640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6c15bf30 con 0x7f8a6c15aad0 2023-10-08T07:16:08.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a71c10640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6c06d6b0 con 0x7f8a6c0b1750 2023-10-08T07:16:08.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a71c10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c0b1bb0 con 0x7f8a6c06d240 2023-10-08T07:16:08.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a6affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8a6c0b1750 0x7f8a6c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:08.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a6affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8a6c0b1750 0x7f8a6c06cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57316/0 (socket says 172.21.15.70:57316) 2023-10-08T07:16:08.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a6affd640 1 -- 172.21.15.70:0/4137799167 learned_addr learned my addr 172.21.15.70:0/4137799167 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:08.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a6affd640 1 -- 172.21.15.70:0/4137799167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a6c06d240 msgr2=0x7f8a6c070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:08.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a6affd640 1 --2- 172.21.15.70:0/4137799167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a6c06d240 0x7f8a6c070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a6affd640 1 -- 172.21.15.70:0/4137799167 >> v1:172.21.15.181:6789/0 conn(0x7f8a6c15aad0 legacy=0x7f8a6c0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:08.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.775+0000 7f8a6affd640 1 -- 172.21.15.70:0/4137799167 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c15b320 con 0x7f8a6c0b1750 2023-10-08T07:16:08.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.776+0000 7f8a6affd640 1 --2- 172.21.15.70:0/4137799167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 0x7f8a6c06cd00 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009870 tx=0x7f8a6002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab391adfb7472ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:08.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.777+0000 7f8a6a7fc640 1 -- 172.21.15.70:0/4137799167 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60031640 con 0x7f8a6c0b1750 2023-10-08T07:16:08.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.777+0000 7f8a6a7fc640 1 -- 172.21.15.70:0/4137799167 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a600317a0 con 0x7f8a6c0b1750 2023-10-08T07:16:08.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.777+0000 7f8a6a7fc640 1 -- 172.21.15.70:0/4137799167 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60031b30 con 0x7f8a6c0b1750 2023-10-08T07:16:08.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.778+0000 7f8a71c10640 1 -- 172.21.15.70:0/4137799167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 msgr2=0x7f8a6c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:08.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.778+0000 7f8a71c10640 1 --2- 172.21.15.70:0/4137799167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 0x7f8a6c06cd00 secure :-1 s=READY pgs=2961 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009870 tx=0x7f8a6002b8f0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.778+0000 7f8a71c10640 1 -- 172.21.15.70:0/4137799167 shutdown_connections 2023-10-08T07:16:08.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.778+0000 7f8a71c10640 1 --2- 172.21.15.70:0/4137799167 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a6c06d240 0x7f8a6c070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.778+0000 7f8a71c10640 1 --2- 172.21.15.70:0/4137799167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 0x7f8a6c06cd00 unknown :-1 s=CLOSED pgs=2961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.778+0000 7f8a71c10640 1 -- 172.21.15.70:0/4137799167 >> 172.21.15.70:0/4137799167 conn(0x7f8a6c069d50 msgr2=0x7f8a6c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:08.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.778+0000 7f8a71c10640 1 -- 172.21.15.70:0/4137799167 shutdown_connections 2023-10-08T07:16:08.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.778+0000 7f8a71c10640 1 -- 172.21.15.70:0/4137799167 wait complete. 2023-10-08T07:16:08.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.780+0000 7f8a71c10640 1 Processor -- start 2023-10-08T07:16:08.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.780+0000 7f8a71c10640 1 -- start start 2023-10-08T07:16:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.781+0000 7f8a71c10640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a6c06d240 0x7f8a6c143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.781+0000 7f8a71c10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 0x7f8a6c14a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.781+0000 7f8a71c10640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a6c15aad0 0x7f8a6c14ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.781+0000 7f8a71c10640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c157430 con 0x7f8a6c0b1750 2023-10-08T07:16:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.781+0000 7f8a71c10640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c14b190 con 0x7f8a6c06d240 2023-10-08T07:16:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.781+0000 7f8a71c10640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6c14b470 con 0x7f8a6c15aad0 2023-10-08T07:16:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.781+0000 7f8a6affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 0x7f8a6c14a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.781+0000 7f8a6affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 0x7f8a6c14a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57326/0 (socket says 172.21.15.70:57326) 2023-10-08T07:16:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.781+0000 7f8a6affd640 1 -- 172.21.15.70:0/1582596270 learned_addr learned my addr 172.21.15.70:0/1582596270 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.781+0000 7f8a6b7fe640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a6c06d240 0x7f8a6c143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.782+0000 7f8a6bfff640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a6c15aad0 0x7f8a6c14ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.782+0000 7f8a6affd640 1 -- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a6c15aad0 msgr2=0x7f8a6c14ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.782+0000 7f8a6affd640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a6c15aad0 0x7f8a6c14ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.782+0000 7f8a6affd640 1 -- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a6c06d240 msgr2=0x7f8a6c143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.782+0000 7f8a6affd640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a6c06d240 0x7f8a6c143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.782+0000 7f8a6affd640 1 -- 172.21.15.70:0/1582596270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600094d0 con 0x7f8a6c0b1750 2023-10-08T07:16:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.782+0000 7f8a6affd640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 0x7f8a6c14a620 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009870 tx=0x7f8a600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:08.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.782+0000 7f8a53fff640 1 -- 172.21.15.70:0/1582596270 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6003ac60 con 0x7f8a6c0b1750 2023-10-08T07:16:08.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.783+0000 7f8a53fff640 1 -- 172.21.15.70:0/1582596270 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6003ade0 con 0x7f8a6c0b1750 2023-10-08T07:16:08.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.783+0000 7f8a53fff640 1 -- 172.21.15.70:0/1582596270 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60044630 con 0x7f8a6c0b1750 2023-10-08T07:16:08.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.783+0000 7f8a71c10640 1 -- 172.21.15.70:0/1582596270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6c14b750 con 0x7f8a6c0b1750 2023-10-08T07:16:08.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.783+0000 7f8a71c10640 1 -- 172.21.15.70:0/1582596270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6c15e750 con 0x7f8a6c0b1750 2023-10-08T07:16:08.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.784+0000 7f8a71c10640 1 -- 172.21.15.70:0/1582596270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6c0c93f0 con 0x7f8a6c0b1750 2023-10-08T07:16:08.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.786+0000 7f8a53fff640 1 -- 172.21.15.70:0/1582596270 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 151) v1 ==== 89388+0+0 (secure 0 0 0) 0x7f8a6002f050 con 0x7f8a6c0b1750 2023-10-08T07:16:08.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.787+0000 7f8a53fff640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f8a5806b020 0x7f8a5806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:08.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.787+0000 7f8a53fff640 1 -- 172.21.15.70:0/1582596270 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(277..277 src has 1..277) v4 ==== 6618+0+0 (secure 0 0 0) 0x7f8a600abe80 con 0x7f8a6c0b1750 2023-10-08T07:16:08.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.787+0000 7f8a6b7fe640 1 -- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f8a5806b020 msgr2=0x7f8a5806d4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.181:6832/1127370729 2023-10-08T07:16:08.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.787+0000 7f8a6b7fe640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f8a5806b020 0x7f8a5806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T07:16:08.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.792+0000 7f8a53fff640 1 -- 172.21.15.70:0/1582596270 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a6c0c93f0 con 0x7f8a6c0b1750 2023-10-08T07:16:08.828 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:08.828+0000 7f9d89613040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T07:16:08.861 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.859+0000 7f2754434040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:16:08.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.881+0000 7f8a53fff640 1 -- 172.21.15.70:0/1582596270 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 152) v1 ==== 88940+0+0 (secure 0 0 0) 0x7f8a60076be0 con 0x7f8a6c0b1750 2023-10-08T07:16:08.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.881+0000 7f8a53fff640 1 -- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f8a5806b020 msgr2=0x7f8a5806d4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T07:16:08.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.881+0000 7f8a53fff640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f8a5806b020 0x7f8a5806d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.952 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.950+0000 7f26d5eba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:08.952 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.950+0000 7f26d5eba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:08.952 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.950+0000 7f26d5eba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:08.952 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.950+0000 7f26d5eba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:08.952 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.950+0000 7f26d5eba640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:08.953 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.951+0000 7f26d8ec0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:08.953 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.951+0000 7f26d8ec0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:08.953 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.951+0000 7f26d8ec0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:08.953 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.951+0000 7f26d8ec0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:08.953 INFO:tasks.ceph.mgr.x.smithi181.stderr:2023-10-08T07:16:08.951+0000 7f26d8ec0640 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T07:16:08.954 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-10-08T07:16:08.953+0000 7f9d89613040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T07:16:08.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.990+0000 7f8a71c10640 1 -- 172.21.15.70:0/1582596270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a6c0c93f0 con 0x7f8a6c0b1750 2023-10-08T07:16:08.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.991+0000 7f8a53fff640 1 -- 172.21.15.70:0/1582596270 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v278) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a6c0c93f0 con 0x7f8a6c0b1750 2023-10-08T07:16:08.991 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-10-08T07:16:08.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.993+0000 7f8a71c10640 1 -- 172.21.15.70:0/1582596270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 msgr2=0x7f8a6c14a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:08.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.993+0000 7f8a71c10640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 0x7f8a6c14a620 secure :-1 s=READY pgs=2962 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009870 tx=0x7f8a600046c0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.993+0000 7f8a71c10640 1 -- 172.21.15.70:0/1582596270 shutdown_connections 2023-10-08T07:16:08.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.993+0000 7f8a71c10640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:6832/1127370729,v1:172.21.15.181:6833/1127370729] conn(0x7f8a5806b020 0x7f8a5806d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.993+0000 7f8a71c10640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8a6c15aad0 0x7f8a6c14ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.993+0000 7f8a71c10640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a6c0b1750 0x7f8a6c14a620 unknown :-1 s=CLOSED pgs=2962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.993+0000 7f8a71c10640 1 --2- 172.21.15.70:0/1582596270 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8a6c06d240 0x7f8a6c143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:08.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.993+0000 7f8a71c10640 1 -- 172.21.15.70:0/1582596270 >> 172.21.15.70:0/1582596270 conn(0x7f8a6c069d50 msgr2=0x7f8a6c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:08.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.993+0000 7f8a71c10640 1 -- 172.21.15.70:0/1582596270 shutdown_connections 2023-10-08T07:16:08.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:08.993+0000 7f8a71c10640 1 -- 172.21.15.70:0/1582596270 wait complete. 2023-10-08T07:16:09.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T07:16:09.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.278+0000 7fe281432640 1 Processor -- start 2023-10-08T07:16:09.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.278+0000 7fe281432640 1 -- start start 2023-10-08T07:16:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.279+0000 7fe281432640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe27c0afbe0 0x7fe27c0b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.279+0000 7fe281432640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c0b2620 con 0x7fe27c06dfc0 2023-10-08T07:16:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.279+0000 7fe281432640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c0b2060 con 0x7fe27c0cb130 2023-10-08T07:16:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.279+0000 7fe281432640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27c0b2340 con 0x7fe27c0afbe0 2023-10-08T07:16:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.279+0000 7fe27affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe27c0afbe0 0x7fe27c0b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.279+0000 7fe27affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe27c0afbe0 0x7fe27c0b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57330/0 (socket says 172.21.15.70:57330) 2023-10-08T07:16:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.279+0000 7fe27affd640 1 -- 172.21.15.70:0/2505118611 learned_addr learned my addr 172.21.15.70:0/2505118611 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.279+0000 7fe27a7fc640 1 --1- 172.21.15.70:0/2505118611 >> v1:172.21.15.70:6789/0 conn(0x7fe27c06dfc0 0x7fe27c070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49032/0 (socket says 172.21.15.70:49032) 2023-10-08T07:16:09.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.280+0000 7fe27affd640 1 -- 172.21.15.70:0/2505118611 >> v1:172.21.15.181:6789/0 conn(0x7fe27c0cb130 legacy=0x7fe27c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:09.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.280+0000 7fe27affd640 1 -- 172.21.15.70:0/2505118611 >> v1:172.21.15.70:6789/0 conn(0x7fe27c06dfc0 legacy=0x7fe27c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:09.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.280+0000 7fe27affd640 1 -- 172.21.15.70:0/2505118611 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c070350 con 0x7fe27c0afbe0 2023-10-08T07:16:09.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.281+0000 7fe27affd640 1 --2- 172.21.15.70:0/2505118611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0afbe0 0x7fe27c0b15b0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009e50 tx=0x7fe26c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25172f2915d64632 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.281+0000 7fe279ffb640 1 -- 172.21.15.70:0/2505118611 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c02cb10 con 0x7fe27c0afbe0 2023-10-08T07:16:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.281+0000 7fe279ffb640 1 -- 172.21.15.70:0/2505118611 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c02cc70 con 0x7fe27c0afbe0 2023-10-08T07:16:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.281+0000 7fe279ffb640 1 -- 172.21.15.70:0/2505118611 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0343f0 con 0x7fe27c0afbe0 2023-10-08T07:16:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.282+0000 7fe281432640 1 -- 172.21.15.70:0/2505118611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0afbe0 msgr2=0x7fe27c0b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.282+0000 7fe281432640 1 --2- 172.21.15.70:0/2505118611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0afbe0 0x7fe27c0b15b0 secure :-1 s=READY pgs=2964 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009e50 tx=0x7fe26c02bf70 comp rx=0 tx=0).stop 2023-10-08T07:16:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.282+0000 7fe281432640 1 -- 172.21.15.70:0/2505118611 shutdown_connections 2023-10-08T07:16:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.282+0000 7fe281432640 1 --2- 172.21.15.70:0/2505118611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0afbe0 0x7fe27c0b15b0 unknown :-1 s=CLOSED pgs=2964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.282+0000 7fe281432640 1 -- 172.21.15.70:0/2505118611 >> 172.21.15.70:0/2505118611 conn(0x7fe27c06a5d0 msgr2=0x7fe27c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.282+0000 7fe281432640 1 -- 172.21.15.70:0/2505118611 shutdown_connections 2023-10-08T07:16:09.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.282+0000 7fe281432640 1 -- 172.21.15.70:0/2505118611 wait complete. 2023-10-08T07:16:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.283+0000 7fe281432640 1 Processor -- start 2023-10-08T07:16:09.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.284+0000 7fe281432640 1 -- start start 2023-10-08T07:16:09.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.284+0000 7fe281432640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe27c06dfc0 0x7fe27c1579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:09.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe281432640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27c0afbe0 0x7fe27c0c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:09.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe281432640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0cb130 0x7fe27c0ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:09.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe281432640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0af7c0 con 0x7fe27c0cb130 2023-10-08T07:16:09.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe281432640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c15bc90 con 0x7fe27c0afbe0 2023-10-08T07:16:09.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe281432640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0af4e0 con 0x7fe27c06dfc0 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0cb130 0x7fe27c0ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0cb130 0x7fe27c0ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57342/0 (socket says 172.21.15.70:57342) 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe27b7fe640 1 -- 172.21.15.70:0/3383740955 learned_addr learned my addr 172.21.15.70:0/3383740955 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe27affd640 1 --2- 172.21.15.70:0/3383740955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe27c06dfc0 0x7fe27c1579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe27b7fe640 1 -- 172.21.15.70:0/3383740955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe27c06dfc0 msgr2=0x7fe27c1579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe27b7fe640 1 --2- 172.21.15.70:0/3383740955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe27c06dfc0 0x7fe27c1579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe27a7fc640 1 --2- 172.21.15.70:0/3383740955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27c0afbe0 0x7fe27c0c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe27b7fe640 1 -- 172.21.15.70:0/3383740955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27c0afbe0 msgr2=0x7fe27c0c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe27b7fe640 1 --2- 172.21.15.70:0/3383740955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27c0afbe0 0x7fe27c0c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.285+0000 7fe27b7fe640 1 -- 172.21.15.70:0/3383740955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c009b30 con 0x7fe27c0cb130 2023-10-08T07:16:09.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.286+0000 7fe27b7fe640 1 --2- 172.21.15.70:0/3383740955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0cb130 0x7fe27c0ca320 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7fe27400c390 tx=0x7fe27400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:09.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.286+0000 7fe2737fe640 1 -- 172.21.15.70:0/3383740955 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2740035f0 con 0x7fe27c0cb130 2023-10-08T07:16:09.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.286+0000 7fe2737fe640 1 -- 172.21.15.70:0/3383740955 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe274003750 con 0x7fe27c0cb130 2023-10-08T07:16:09.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.286+0000 7fe2737fe640 1 -- 172.21.15.70:0/3383740955 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274003b00 con 0x7fe27c0cb130 2023-10-08T07:16:09.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.286+0000 7fe281432640 1 -- 172.21.15.70:0/3383740955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c15b640 con 0x7fe27c0cb130 2023-10-08T07:16:09.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.286+0000 7fe281432640 1 -- 172.21.15.70:0/3383740955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c0af170 con 0x7fe27c0cb130 2023-10-08T07:16:09.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.287+0000 7fe281432640 1 -- 172.21.15.70:0/3383740955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c0c3550 con 0x7fe27c0cb130 2023-10-08T07:16:09.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.288+0000 7fe2737fe640 1 -- 172.21.15.70:0/3383740955 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 152) v1 ==== 88940+0+0 (secure 0 0 0) 0x7fe274003c60 con 0x7fe27c0cb130 2023-10-08T07:16:09.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.289+0000 7fe2737fe640 1 -- 172.21.15.70:0/3383740955 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 7005+0+0 (secure 0 0 0) 0x7fe2740888f0 con 0x7fe27c0cb130 2023-10-08T07:16:09.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.293+0000 7fe2737fe640 1 -- 172.21.15.70:0/3383740955 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe27c0c3550 con 0x7fe27c0cb130 2023-10-08T07:16:09.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.483+0000 7fe281432640 1 -- 172.21.15.70:0/3383740955 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe27c0c3550 con 0x7fe27c0cb130 2023-10-08T07:16:09.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.484+0000 7fe2737fe640 1 -- 172.21.15.70:0/3383740955 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v278) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe27c0c3550 con 0x7fe27c0cb130 2023-10-08T07:16:09.485 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-10-08T07:16:09.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.487+0000 7fe281432640 1 -- 172.21.15.70:0/3383740955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0cb130 msgr2=0x7fe27c0ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:09.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.487+0000 7fe281432640 1 --2- 172.21.15.70:0/3383740955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0cb130 0x7fe27c0ca320 secure :-1 s=READY pgs=2965 cs=0 l=1 rev1=1 crypto rx=0x7fe27400c390 tx=0x7fe27400c850 comp rx=0 tx=0).stop 2023-10-08T07:16:09.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.487+0000 7fe281432640 1 -- 172.21.15.70:0/3383740955 shutdown_connections 2023-10-08T07:16:09.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.487+0000 7fe281432640 1 --2- 172.21.15.70:0/3383740955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe27c0cb130 0x7fe27c0ca320 unknown :-1 s=CLOSED pgs=2965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.487+0000 7fe281432640 1 --2- 172.21.15.70:0/3383740955 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe27c0afbe0 0x7fe27c0c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.487+0000 7fe281432640 1 --2- 172.21.15.70:0/3383740955 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe27c06dfc0 0x7fe27c1579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.487+0000 7fe281432640 1 -- 172.21.15.70:0/3383740955 >> 172.21.15.70:0/3383740955 conn(0x7fe27c06a5d0 msgr2=0x7fe27c159630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:09.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.487+0000 7fe281432640 1 -- 172.21.15.70:0/3383740955 shutdown_connections 2023-10-08T07:16:09.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.487+0000 7fe281432640 1 -- 172.21.15.70:0/3383740955 wait complete. 2023-10-08T07:16:09.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T07:16:09.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.711+0000 7fec1074e640 1 Processor -- start 2023-10-08T07:16:09.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.711+0000 7fec1074e640 1 -- start start 2023-10-08T07:16:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.712+0000 7fec1074e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 0x7fec08070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec1074e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fec080ca370 0x7fec0806d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec1074e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0806c3a0 con 0x7fec080b21f0 2023-10-08T07:16:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec1074e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec08070bd0 con 0x7fec080ca370 2023-10-08T07:16:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec1074e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0806c640 con 0x7fec080702b0 2023-10-08T07:16:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec0eceb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fec080ca370 0x7fec0806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec0eceb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fec080ca370 0x7fec0806d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57350/0 (socket says 172.21.15.70:57350) 2023-10-08T07:16:09.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec0eceb640 1 -- 172.21.15.70:0/1941353170 learned_addr learned my addr 172.21.15.70:0/1941353170 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec0eceb640 1 -- 172.21.15.70:0/1941353170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 msgr2=0x7fec08070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec0e4ea640 1 --2- 172.21.15.70:0/1941353170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 0x7fec08070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec0eceb640 1 --2- 172.21.15.70:0/1941353170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 0x7fec08070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec0eceb640 1 -- 172.21.15.70:0/1941353170 >> v1:172.21.15.181:6789/0 conn(0x7fec080b21f0 legacy=0x7fec080c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.713+0000 7fec0eceb640 1 -- 172.21.15.70:0/1941353170 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0806c920 con 0x7fec080ca370 2023-10-08T07:16:09.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.714+0000 7fec0eceb640 1 --2- 172.21.15.70:0/1941353170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080ca370 0x7fec0806d400 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7fec00009870 tx=0x7fec0002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77acfca41d82df7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:09.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.715+0000 7fec0d4e8640 1 -- 172.21.15.70:0/1941353170 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec000317c0 con 0x7fec080ca370 2023-10-08T07:16:09.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.715+0000 7fec0d4e8640 1 -- 172.21.15.70:0/1941353170 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec00031920 con 0x7fec080ca370 2023-10-08T07:16:09.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.715+0000 7fec0d4e8640 1 -- 172.21.15.70:0/1941353170 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec00031cb0 con 0x7fec080ca370 2023-10-08T07:16:09.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.715+0000 7fec1074e640 1 -- 172.21.15.70:0/1941353170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080ca370 msgr2=0x7fec0806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:09.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.715+0000 7fec1074e640 1 --2- 172.21.15.70:0/1941353170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080ca370 0x7fec0806d400 secure :-1 s=READY pgs=2966 cs=0 l=1 rev1=1 crypto rx=0x7fec00009870 tx=0x7fec0002b860 comp rx=0 tx=0).stop 2023-10-08T07:16:09.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.715+0000 7fec1074e640 1 -- 172.21.15.70:0/1941353170 shutdown_connections 2023-10-08T07:16:09.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.715+0000 7fec1074e640 1 --2- 172.21.15.70:0/1941353170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080ca370 0x7fec0806d400 unknown :-1 s=CLOSED pgs=2966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.715+0000 7fec1074e640 1 --2- 172.21.15.70:0/1941353170 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 0x7fec08070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.715+0000 7fec1074e640 1 -- 172.21.15.70:0/1941353170 >> 172.21.15.70:0/1941353170 conn(0x7fec0806a5d0 msgr2=0x7fec080b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:09.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.716+0000 7fec1074e640 1 -- 172.21.15.70:0/1941353170 shutdown_connections 2023-10-08T07:16:09.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.716+0000 7fec1074e640 1 -- 172.21.15.70:0/1941353170 wait complete. 2023-10-08T07:16:09.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.717+0000 7fec1074e640 1 Processor -- start 2023-10-08T07:16:09.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.717+0000 7fec1074e640 1 -- start start 2023-10-08T07:16:09.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.718+0000 7fec1074e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 0x7fec08167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:09.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.718+0000 7fec1074e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080b21f0 0x7fec0806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:09.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.718+0000 7fec1074e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec080ca370 0x7fec0806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:09.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.718+0000 7fec1074e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec08166e40 con 0x7fec080b21f0 2023-10-08T07:16:09.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.718+0000 7fec1074e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec08166250 con 0x7fec080ca370 2023-10-08T07:16:09.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.718+0000 7fec1074e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec081664a0 con 0x7fec080702b0 2023-10-08T07:16:09.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.718+0000 7fec0dce9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080b21f0 0x7fec0806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:09.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0dce9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080b21f0 0x7fec0806d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57352/0 (socket says 172.21.15.70:57352) 2023-10-08T07:16:09.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0dce9640 1 -- 172.21.15.70:0/2122732 learned_addr learned my addr 172.21.15.70:0/2122732 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:09.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0e4ea640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 0x7fec08167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:09.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0eceb640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec080ca370 0x7fec0806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:09.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0dce9640 1 -- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 msgr2=0x7fec08167aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:09.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0dce9640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 0x7fec08167aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0dce9640 1 -- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec080ca370 msgr2=0x7fec0806d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:09.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0dce9640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec080ca370 0x7fec0806d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0dce9640 1 -- 172.21.15.70:0/2122732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec000094d0 con 0x7fec080b21f0 2023-10-08T07:16:09.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0e4ea640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 0x7fec08167aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:09.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.719+0000 7fec0dce9640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080b21f0 0x7fec0806d300 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7febf400c430 tx=0x7febf400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:09.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.720+0000 7fec06ffd640 1 -- 172.21.15.70:0/2122732 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4011790 con 0x7fec080b21f0 2023-10-08T07:16:09.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.720+0000 7fec06ffd640 1 -- 172.21.15.70:0/2122732 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf40118f0 con 0x7fec080b21f0 2023-10-08T07:16:09.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.720+0000 7fec06ffd640 1 -- 172.21.15.70:0/2122732 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4011ce0 con 0x7fec080b21f0 2023-10-08T07:16:09.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.720+0000 7fec1074e640 1 -- 172.21.15.70:0/2122732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec08166780 con 0x7fec080b21f0 2023-10-08T07:16:09.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.720+0000 7fec1074e640 1 -- 172.21.15.70:0/2122732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec080cc640 con 0x7fec080b21f0 2023-10-08T07:16:09.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.721+0000 7fec1074e640 1 -- 172.21.15.70:0/2122732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec08166f80 con 0x7fec080b21f0 2023-10-08T07:16:09.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.722+0000 7fec06ffd640 1 -- 172.21.15.70:0/2122732 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 152) v1 ==== 88940+0+0 (secure 0 0 0) 0x7febf401c460 con 0x7fec080b21f0 2023-10-08T07:16:09.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.723+0000 7fec06ffd640 1 -- 172.21.15.70:0/2122732 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 7005+0+0 (secure 0 0 0) 0x7febf4089650 con 0x7fec080b21f0 2023-10-08T07:16:09.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.727+0000 7fec06ffd640 1 -- 172.21.15.70:0/2122732 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7febf4054550 con 0x7fec080b21f0 2023-10-08T07:16:09.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.887+0000 7fec06ffd640 1 -- 172.21.15.70:0/2122732 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 153) v1 ==== 89067+0+0 (secure 0 0 0) 0x7febf4054000 con 0x7fec080b21f0 2023-10-08T07:16:09.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.887+0000 7fec06ffd640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7febec070de0 0x7febec0731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:09.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.888+0000 7fec0e4ea640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7febec070de0 0x7febec0731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:09.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.891+0000 7fec0e4ea640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7febec070de0 0x7febec0731d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7febfc007f50 tx=0x7febfc0074a0 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:09.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.921+0000 7fec1074e640 1 -- 172.21.15.70:0/2122732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec0804e160 con 0x7fec080b21f0 2023-10-08T07:16:09.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.922+0000 7fec06ffd640 1 -- 172.21.15.70:0/2122732 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v278) v1 ==== 100+0+11 (secure 0 0 0) 0x7febf4071610 con 0x7fec080b21f0 2023-10-08T07:16:09.923 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-10-08T07:16:09.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 -- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7febec070de0 msgr2=0x7febec0731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:09.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7febec070de0 0x7febec0731d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7febfc007f50 tx=0x7febfc0074a0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 -- 172.21.15.70:0/2122732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080b21f0 msgr2=0x7fec0806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:09.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080b21f0 0x7fec0806d300 secure :-1 s=READY pgs=2967 cs=0 l=1 rev1=1 crypto rx=0x7febf400c430 tx=0x7febf400c8f0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 -- 172.21.15.70:0/2122732 shutdown_connections 2023-10-08T07:16:09.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7febec070de0 0x7febec0731d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fec080ca370 0x7fec0806d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec080b21f0 0x7fec0806d300 unknown :-1 s=CLOSED pgs=2967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 --2- 172.21.15.70:0/2122732 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fec080702b0 0x7fec08167aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:09.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 -- 172.21.15.70:0/2122732 >> 172.21.15.70:0/2122732 conn(0x7fec0806a5d0 msgr2=0x7fec0815bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:09.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.925+0000 7fec1074e640 1 -- 172.21.15.70:0/2122732 shutdown_connections 2023-10-08T07:16:09.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:09.926+0000 7fec1074e640 1 -- 172.21.15.70:0/2122732 wait complete. 2023-10-08T07:16:09.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:16:10.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.150+0000 7f65c6c64640 1 Processor -- start 2023-10-08T07:16:10.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.150+0000 7f65c6c64640 1 -- start start 2023-10-08T07:16:10.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.151+0000 7f65c6c64640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c00ad790 0x7f65c00702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.152+0000 7f65c6c64640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f65c015e330 0x7f65c0070810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.152+0000 7f65c6c64640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c006dee0 con 0x7f65c0159ac0 2023-10-08T07:16:10.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.152+0000 7f65c6c64640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c006dcf0 con 0x7f65c015e330 2023-10-08T07:16:10.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.152+0000 7f65c6c64640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c015d850 con 0x7f65c00ad790 2023-10-08T07:16:10.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.152+0000 7f65bffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c00ad790 0x7f65c00702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.152+0000 7f65bffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c00ad790 0x7f65c00702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:34760/0 (socket says 172.21.15.70:34760) 2023-10-08T07:16:10.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.152+0000 7f65bffff640 1 -- 172.21.15.70:0/842922592 learned_addr learned my addr 172.21.15.70:0/842922592 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:10.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.153+0000 7f65c5201640 1 --2- 172.21.15.70:0/842922592 >> v2:172.21.15.181:3300/0 conn(0x7f65c015e330 0x7f65c0070810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.153+0000 7f65bf7fe640 1 -- 172.21.15.70:0/842922592 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296385849 0 0) 0x7f65c006dee0 con 0x7f65c0159ac0 2023-10-08T07:16:10.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.153+0000 7f65bffff640 1 -- 172.21.15.70:0/842922592 >> v2:172.21.15.181:3300/0 conn(0x7f65c015e330 msgr2=0x7f65c0070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:10.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.153+0000 7f65bffff640 1 --2- 172.21.15.70:0/842922592 >> v2:172.21.15.181:3300/0 conn(0x7f65c015e330 0x7f65c0070810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.153+0000 7f65bffff640 1 -- 172.21.15.70:0/842922592 >> v1:172.21.15.181:6789/0 conn(0x7f65c0159ac0 legacy=0x7f65c006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.153+0000 7f65bffff640 1 -- 172.21.15.70:0/842922592 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c015db30 con 0x7f65c00ad790 2023-10-08T07:16:10.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.155+0000 7f65bffff640 1 --2- 172.21.15.70:0/842922592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c00ad790 0x7f65c00702d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009870 tx=0x7f65ac02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fec729e290051c18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:10.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.155+0000 7f65bf7fe640 1 -- 172.21.15.70:0/842922592 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac030840 con 0x7f65c00ad790 2023-10-08T07:16:10.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.156+0000 7f65bf7fe640 1 -- 172.21.15.70:0/842922592 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac0309a0 con 0x7f65c00ad790 2023-10-08T07:16:10.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.156+0000 7f65bf7fe640 1 -- 172.21.15.70:0/842922592 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac030d30 con 0x7f65c00ad790 2023-10-08T07:16:10.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.156+0000 7f65c6c64640 1 -- 172.21.15.70:0/842922592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c00ad790 msgr2=0x7f65c00702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.156+0000 7f65c6c64640 1 --2- 172.21.15.70:0/842922592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c00ad790 0x7f65c00702d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009870 tx=0x7f65ac02b980 comp rx=0 tx=0).stop 2023-10-08T07:16:10.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.156+0000 7f65c6c64640 1 -- 172.21.15.70:0/842922592 shutdown_connections 2023-10-08T07:16:10.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.156+0000 7f65c6c64640 1 --2- 172.21.15.70:0/842922592 >> v2:172.21.15.181:3300/0 conn(0x7f65c015e330 0x7f65c0070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.156+0000 7f65c6c64640 1 --2- 172.21.15.70:0/842922592 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c00ad790 0x7f65c00702d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.156+0000 7f65c6c64640 1 -- 172.21.15.70:0/842922592 >> 172.21.15.70:0/842922592 conn(0x7f65c006a380 msgr2=0x7f65c00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:10.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.156+0000 7f65c6c64640 1 -- 172.21.15.70:0/842922592 shutdown_connections 2023-10-08T07:16:10.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.156+0000 7f65c6c64640 1 -- 172.21.15.70:0/842922592 wait complete. 2023-10-08T07:16:10.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.158+0000 7f65c6c64640 1 Processor -- start 2023-10-08T07:16:10.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.158+0000 7f65c6c64640 1 -- start start 2023-10-08T07:16:10.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.159+0000 7f65c6c64640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c00ad790 0x7f65c014cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.159+0000 7f65c6c64640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c0159ac0 0x7f65c014a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.159+0000 7f65c6c64640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c015e330 0x7f65c00c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.159+0000 7f65c6c64640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c00c73f0 con 0x7f65c0159ac0 2023-10-08T07:16:10.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.159+0000 7f65c6c64640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c00c6ae0 con 0x7f65c00ad790 2023-10-08T07:16:10.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.159+0000 7f65c6c64640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c014d3b0 con 0x7f65c015e330 2023-10-08T07:16:10.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.160+0000 7f65bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c0159ac0 0x7f65c014a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.160+0000 7f65bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c0159ac0 0x7f65c014a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57364/0 (socket says 172.21.15.70:57364) 2023-10-08T07:16:10.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.160+0000 7f65bffff640 1 -- 172.21.15.70:0/911381942 learned_addr learned my addr 172.21.15.70:0/911381942 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:10.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.160+0000 7f65c5201640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c015e330 0x7f65c00c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.160+0000 7f65c4a00640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c00ad790 0x7f65c014cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.160+0000 7f65bffff640 1 -- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c015e330 msgr2=0x7f65c00c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.160+0000 7f65bffff640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c015e330 0x7f65c00c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.160+0000 7f65bffff640 1 -- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c00ad790 msgr2=0x7f65c014cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.160+0000 7f65bffff640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c00ad790 0x7f65c014cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.160+0000 7f65bffff640 1 -- 172.21.15.70:0/911381942 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac0094d0 con 0x7f65c0159ac0 2023-10-08T07:16:10.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.161+0000 7f65bffff640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c0159ac0 0x7f65c014a8b0 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7f65ac000c00 tx=0x7f65ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:10.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.161+0000 7f65bd7fa640 1 -- 172.21.15.70:0/911381942 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac03e040 con 0x7f65c0159ac0 2023-10-08T07:16:10.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.161+0000 7f65bd7fa640 1 -- 172.21.15.70:0/911381942 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac039c90 con 0x7f65c0159ac0 2023-10-08T07:16:10.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.161+0000 7f65bd7fa640 1 -- 172.21.15.70:0/911381942 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ac004030 con 0x7f65c0159ac0 2023-10-08T07:16:10.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.161+0000 7f65c6c64640 1 -- 172.21.15.70:0/911381942 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c00c6d30 con 0x7f65c0159ac0 2023-10-08T07:16:10.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.161+0000 7f65c6c64640 1 -- 172.21.15.70:0/911381942 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c014c4a0 con 0x7f65c0159ac0 2023-10-08T07:16:10.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.162+0000 7f65c6c64640 1 -- 172.21.15.70:0/911381942 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c00c7530 con 0x7f65c0159ac0 2023-10-08T07:16:10.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.163+0000 7f65bd7fa640 1 -- 172.21.15.70:0/911381942 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 153) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f65ac004190 con 0x7f65c0159ac0 2023-10-08T07:16:10.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.164+0000 7f65bd7fa640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f65a406acc0 0x7f65a406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.164+0000 7f65bd7fa640 1 -- 172.21.15.70:0/911381942 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 7005+0+0 (secure 0 0 0) 0x7f65ac0abad0 con 0x7f65c0159ac0 2023-10-08T07:16:10.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.165+0000 7f65c4a00640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f65a406acc0 0x7f65a406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.166+0000 7f65c4a00640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f65a406acc0 0x7f65a406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000f60 tx=0x7f65b4009040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:10.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.168+0000 7f65bd7fa640 1 -- 172.21.15.70:0/911381942 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65c00c7530 con 0x7f65c0159ac0 2023-10-08T07:16:10.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.397+0000 7f65c6c64640 1 -- 172.21.15.70:0/911381942 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65c00c7530 con 0x7f65c0159ac0 2023-10-08T07:16:10.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.398+0000 7f65bd7fa640 1 -- 172.21.15.70:0/911381942 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4923 (secure 0 0 0) 0x7f65c00c7530 con 0x7f65c0159ac0 2023-10-08T07:16:10.399 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:10.399 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":140,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/949448327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":949448327},{"type":"v1","addr":"172.21.15.70:6835","nonce":949448327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":8111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2423877406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2423877406},{"type":"v1","addr":"172.21.15.181:6837","nonce":2423877406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/247109890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":247109890},{"type":"v1","addr":"172.21.15.181:6836","nonce":247109890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:15:45.890217+0000","modified":"2023-10-08T07:15:48.920991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12645},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12645":{"gid":12645,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/753129368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":753129368},{"type":"v1","addr":"172.21.15.70:6834","nonce":753129368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-08T07:16:10.399 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 140 2023-10-08T07:16:10.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 -- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f65a406acc0 msgr2=0x7f65a406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f65a406acc0 0x7f65a406d180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f65b4000f60 tx=0x7f65b4009040 comp rx=0 tx=0).stop 2023-10-08T07:16:10.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 -- 172.21.15.70:0/911381942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c0159ac0 msgr2=0x7f65c014a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c0159ac0 0x7f65c014a8b0 secure :-1 s=READY pgs=2968 cs=0 l=1 rev1=1 crypto rx=0x7f65ac000c00 tx=0x7f65ac004730 comp rx=0 tx=0).stop 2023-10-08T07:16:10.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 -- 172.21.15.70:0/911381942 shutdown_connections 2023-10-08T07:16:10.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f65a406acc0 0x7f65a406d180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f65c015e330 0x7f65c00c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65c0159ac0 0x7f65c014a8b0 unknown :-1 s=CLOSED pgs=2968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 --2- 172.21.15.70:0/911381942 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f65c00ad790 0x7f65c014cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 -- 172.21.15.70:0/911381942 >> 172.21.15.70:0/911381942 conn(0x7f65c006a380 msgr2=0x7f65c014e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:10.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 -- 172.21.15.70:0/911381942 shutdown_connections 2023-10-08T07:16:10.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.401+0000 7f65c6c64640 1 -- 172.21.15.70:0/911381942 wait complete. 2023-10-08T07:16:10.420 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:16:10.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.629+0000 7f7a02cd0640 1 Processor -- start 2023-10-08T07:16:10.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.629+0000 7f7a02cd0640 1 -- start start 2023-10-08T07:16:10.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.631+0000 7f7a02cd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc1597d0 0x7f79fc06ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.631+0000 7f7a02cd0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc0adfc0 con 0x7f79fc0cb190 2023-10-08T07:16:10.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.631+0000 7f7a02cd0640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc06d380 con 0x7f79fc157000 2023-10-08T07:16:10.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.631+0000 7f7a02cd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc159cb0 con 0x7f79fc1597d0 2023-10-08T07:16:10.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.631+0000 7f7a0126d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f79fc0cb190 0x7f79fc15d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49042/0 (socket says 172.21.15.70:49042) 2023-10-08T07:16:10.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.631+0000 7f7a0126d640 1 -- 172.21.15.70:0/1234629408 learned_addr learned my addr 172.21.15.70:0/1234629408 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.631+0000 7f79fbfff640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc1597d0 0x7f79fc06ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258673511 0 0) 0x7f79fc0adfc0 con 0x7f79fc0cb190 2023-10-08T07:16:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79ec003520 con 0x7f79fc0cb190 2023-10-08T07:16:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2410087073 0 0) 0x7f79ec003520 con 0x7f79fc0cb190 2023-10-08T07:16:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc1597d0 msgr2=0x7f79fc06ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc1597d0 0x7f79fc06ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 >> v1:172.21.15.181:6789/0 conn(0x7f79fc157000 legacy=0x7f79fc0cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc159f90 con 0x7f79fc0cb190 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f79f40019f0 con 0x7f79fc0cb190 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 >> v1:172.21.15.70:6789/0 conn(0x7f79fc0cb190 legacy=0x7f79fc15d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 0x7f79ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec0047c0 0x7f79ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc1597d0 0x7f79ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc159cb0 con 0x7f79ec0039a0 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec005a20 con 0x7f79ec0047c0 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec005be0 con 0x7f79fc1597d0 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.632+0000 7f7a00a6c640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 0x7f79ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.633+0000 7f7a0126d640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc1597d0 0x7f79ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.633+0000 7f7a00a6c640 1 -- 172.21.15.70:0/1234629408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc1597d0 msgr2=0x7f79ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.633+0000 7f7a00a6c640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc1597d0 0x7f79ec005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.633+0000 7f7a00a6c640 1 -- 172.21.15.70:0/1234629408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec0047c0 msgr2=0x7f79ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:10.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.633+0000 7f7a00a6c640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79ec0047c0 0x7f79ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.633+0000 7f7a00a6c640 1 -- 172.21.15.70:0/1234629408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec003520 con 0x7f79ec0039a0 2023-10-08T07:16:10.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.634+0000 7f7a00a6c640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 0x7f79ec003dc0 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7f79f0001750 tx=0x7f79ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da1f8c5131f6a0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:10.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.634+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f00052d0 con 0x7f79ec0039a0 2023-10-08T07:16:10.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.634+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f002e050 con 0x7f79ec0039a0 2023-10-08T07:16:10.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.634+0000 7f79fb7fe640 1 -- 172.21.15.70:0/1234629408 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f00035f0 con 0x7f79ec0039a0 2023-10-08T07:16:10.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.635+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1234629408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 msgr2=0x7f79ec003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.635+0000 7f7a02cd0640 1 --2- 172.21.15.70:0/1234629408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 0x7f79ec003dc0 secure :-1 s=READY pgs=2970 cs=0 l=1 rev1=1 crypto rx=0x7f79f0001750 tx=0x7f79ec0057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.635+0000 7f7a0126d640 1 -- 172.21.15.70:0/1234629408 reap_dead start 2023-10-08T07:16:10.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.635+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1234629408 shutdown_connections 2023-10-08T07:16:10.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.635+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1234629408 >> 172.21.15.70:0/1234629408 conn(0x7f79fc06a380 msgr2=0x7f79fc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:10.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.635+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1234629408 shutdown_connections 2023-10-08T07:16:10.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.635+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1234629408 wait complete. 2023-10-08T07:16:10.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.637+0000 7f7a02cd0640 1 Processor -- start 2023-10-08T07:16:10.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.637+0000 7f7a02cd0640 1 -- start start 2023-10-08T07:16:10.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f7a02cd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 0x7f79fc1ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f7a02cd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc14ae20 0x7f79fc1eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f7a02cd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc1f20f0 0x7f79fc1f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f7a02cd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc14e140 con 0x7f79ec0039a0 2023-10-08T07:16:10.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f7a02cd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0b09b0 con 0x7f79fc1f20f0 2023-10-08T07:16:10.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f7a02cd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc0b0110 con 0x7f79fc14ae20 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f7a00a6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 0x7f79fc1ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f7a00a6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 0x7f79fc1ee5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57384/0 (socket says 172.21.15.70:57384) 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f7a00a6c640 1 -- 172.21.15.70:0/1863366876 learned_addr learned my addr 172.21.15.70:0/1863366876 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f7a00a6c640 1 -- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc14ae20 msgr2=0x7f79fc1eeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.638+0000 7f79fbfff640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc14ae20 0x7f79fc1eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.639+0000 7f7a00a6c640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc14ae20 0x7f79fc1eeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.639+0000 7f7a00a6c640 1 -- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc1f20f0 msgr2=0x7f79fc1f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.639+0000 7f7a00a6c640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc1f20f0 0x7f79fc1f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.639+0000 7f7a00a6c640 1 -- 172.21.15.70:0/1863366876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc0b03f0 con 0x7f79ec0039a0 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.639+0000 7f7a00a6c640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 0x7f79fc1ee5c0 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f79f0008fd0 tx=0x7f79f0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:10.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.639+0000 7f79f97fa640 1 -- 172.21.15.70:0/1863366876 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f00076c0 con 0x7f79ec0039a0 2023-10-08T07:16:10.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.639+0000 7f79f97fa640 1 -- 172.21.15.70:0/1863366876 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f0007a30 con 0x7f79ec0039a0 2023-10-08T07:16:10.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.639+0000 7f79f97fa640 1 -- 172.21.15.70:0/1863366876 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0006be0 con 0x7f79ec0039a0 2023-10-08T07:16:10.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.639+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1863366876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc0b06d0 con 0x7f79ec0039a0 2023-10-08T07:16:10.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.639+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1863366876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc15a980 con 0x7f79ec0039a0 2023-10-08T07:16:10.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.640+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1863366876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc14f640 con 0x7f79ec0039a0 2023-10-08T07:16:10.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.641+0000 7f79f97fa640 1 -- 172.21.15.70:0/1863366876 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 153) v1 ==== 89067+0+0 (secure 0 0 0) 0x7f79f0006d40 con 0x7f79ec0039a0 2023-10-08T07:16:10.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.642+0000 7f79f97fa640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f79e006acc0 0x7f79e006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:10.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.642+0000 7f79f97fa640 1 -- 172.21.15.70:0/1863366876 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 7005+0+0 (secure 0 0 0) 0x7f79f00ab990 con 0x7f79ec0039a0 2023-10-08T07:16:10.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.643+0000 7f79fbfff640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f79e006acc0 0x7f79e006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:10.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.644+0000 7f79fbfff640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f79e006acc0 0x7f79e006d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f79fc14f330 tx=0x7f79e8008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:10.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.647+0000 7f79f97fa640 1 -- 172.21.15.70:0/1863366876 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79f00767a0 con 0x7f79ec0039a0 2023-10-08T07:16:10.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.835+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1863366876 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79fc06cdc0 con 0x7f79ec0039a0 2023-10-08T07:16:10.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.838+0000 7f79f97fa640 1 -- 172.21.15.70:0/1863366876 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v278) v1 ==== 74+0+18157 (secure 0 0 0) 0x7f79f0079dc0 con 0x7f79ec0039a0 2023-10-08T07:16:10.839 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:10.839 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":278,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:16:08.864637+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:15:43.819948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-08T07:15:44.895969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3049540535":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/220528716":"2023-10-09T07:16:08.864584+0000","172.21.15.181:6833/1127370729":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/2060656952":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1823452952":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/3814802416":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/3705279960":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6833/440573135":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6832/440573135":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6832/1127370729":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1545538679":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/1740402122":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1151602740":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2253796914":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/3307173154":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2296566554":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2564762528":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/606980":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/796435327":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1513601554":"2023-10-08T08:15:42.823407+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:16:10.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f79e006acc0 msgr2=0x7f79e006d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f79e006acc0 0x7f79e006d180 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f79fc14f330 tx=0x7f79e8008040 comp rx=0 tx=0).stop 2023-10-08T07:16:10.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1863366876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 msgr2=0x7f79fc1ee5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:10.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 0x7f79fc1ee5c0 secure :-1 s=READY pgs=2971 cs=0 l=1 rev1=1 crypto rx=0x7f79f0008fd0 tx=0x7f79f0001120 comp rx=0 tx=0).stop 2023-10-08T07:16:10.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1863366876 shutdown_connections 2023-10-08T07:16:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f79e006acc0 0x7f79e006d180 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f79fc1f20f0 0x7f79fc1f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f79fc14ae20 0x7f79fc1eeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 --2- 172.21.15.70:0/1863366876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79ec0039a0 0x7f79fc1ee5c0 unknown :-1 s=CLOSED pgs=2971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1863366876 >> 172.21.15.70:0/1863366876 conn(0x7f79fc06a380 msgr2=0x7f79fc15b790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1863366876 shutdown_connections 2023-10-08T07:16:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:10.841+0000 7f7a02cd0640 1 -- 172.21.15.70:0/1863366876 wait complete. 2023-10-08T07:16:10.862 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T07:16:10.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:16:11.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.075+0000 7f9751675640 1 Processor -- start 2023-10-08T07:16:11.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.075+0000 7f9751675640 1 -- start start 2023-10-08T07:16:11.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.077+0000 7f9751675640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c15ab40 0x7f974c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.077+0000 7f9751675640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974c15b2b0 con 0x7f974c0cb580 2023-10-08T07:16:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.077+0000 7f9751675640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974c15b590 con 0x7f974c0c90b0 2023-10-08T07:16:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.077+0000 7f9751675640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c15bf30 con 0x7f974c15ab40 2023-10-08T07:16:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.077+0000 7f974b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f974c0cb580 0x7f974c15e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49054/0 (socket says 172.21.15.70:49054) 2023-10-08T07:16:11.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.077+0000 7f974b7fe640 1 -- 172.21.15.70:0/2754894493 learned_addr learned my addr 172.21.15.70:0/2754894493 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:11.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.077+0000 7f974affd640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c15ab40 0x7f974c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.077+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722406799 0 0) 0x7f974c15b2b0 con 0x7f974c0cb580 2023-10-08T07:16:11.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9738003520 con 0x7f974c0cb580 2023-10-08T07:16:11.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 894412730 0 0) 0x7f9738003520 con 0x7f974c0cb580 2023-10-08T07:16:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c15ab40 msgr2=0x7f974c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c15ab40 0x7f974c0ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 >> v1:172.21.15.181:6789/0 conn(0x7f974c0c90b0 legacy=0x7f974c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c15b870 con 0x7f974c0cb580 2023-10-08T07:16:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f97440019f0 con 0x7f974c0cb580 2023-10-08T07:16:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 >> v1:172.21.15.70:6789/0 conn(0x7f974c0cb580 legacy=0x7f974c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c15ab40 0x7f97380039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9738003f20 0x7f97380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9738004d00 0x7f9738005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c15bf30 con 0x7f9738004d00 2023-10-08T07:16:11.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973800ba60 con 0x7f9738003f20 2023-10-08T07:16:11.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973800bc20 con 0x7f974c15ab40 2023-10-08T07:16:11.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.078+0000 7f974b7fe640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9738004d00 0x7f9738005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.079+0000 7f974affd640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c15ab40 0x7f97380039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.079+0000 7f974b7fe640 1 -- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c15ab40 msgr2=0x7f97380039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:11.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.079+0000 7f9743fff640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9738003f20 0x7f97380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.079+0000 7f974b7fe640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c15ab40 0x7f97380039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.079+0000 7f974b7fe640 1 -- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9738003f20 msgr2=0x7f97380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:11.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.079+0000 7f974b7fe640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9738003f20 0x7f97380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.079+0000 7f974b7fe640 1 -- 172.21.15.70:0/2754894493 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9738003520 con 0x7f9738004d00 2023-10-08T07:16:11.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.079+0000 7f9743fff640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9738003f20 0x7f97380047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:11.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.080+0000 7f974b7fe640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9738004d00 0x7f9738005180 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7f974400a800 tx=0x7f97380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40e612fc7e61bb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.080+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97440072d0 con 0x7f9738004d00 2023-10-08T07:16:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.080+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974402f070 con 0x7f9738004d00 2023-10-08T07:16:11.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.080+0000 7f974a7fc640 1 -- 172.21.15.70:0/2754894493 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744007690 con 0x7f9738004d00 2023-10-08T07:16:11.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.081+0000 7f9751675640 1 -- 172.21.15.70:0/2754894493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9738004d00 msgr2=0x7f9738005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.081+0000 7f9751675640 1 --2- 172.21.15.70:0/2754894493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9738004d00 0x7f9738005180 secure :-1 s=READY pgs=2973 cs=0 l=1 rev1=1 crypto rx=0x7f974400a800 tx=0x7f97380057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.081+0000 7f974b7fe640 1 -- 172.21.15.70:0/2754894493 reap_dead start 2023-10-08T07:16:11.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.081+0000 7f9751675640 1 -- 172.21.15.70:0/2754894493 shutdown_connections 2023-10-08T07:16:11.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.081+0000 7f9751675640 1 -- 172.21.15.70:0/2754894493 >> 172.21.15.70:0/2754894493 conn(0x7f974c06a380 msgr2=0x7f974c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:11.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.081+0000 7f9751675640 1 -- 172.21.15.70:0/2754894493 shutdown_connections 2023-10-08T07:16:11.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.081+0000 7f9751675640 1 -- 172.21.15.70:0/2754894493 wait complete. 2023-10-08T07:16:11.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.082+0000 7f9751675640 1 Processor -- start 2023-10-08T07:16:11.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.083+0000 7f9751675640 1 -- start start 2023-10-08T07:16:11.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.083+0000 7f9751675640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9738004d00 0x7f974c150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f9751675640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c1515a0 0x7f974c0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f9751675640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f974c153980 0x7f974c0ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f9751675640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c0c8750 con 0x7f974c153980 2023-10-08T07:16:11.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f9751675640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c1528e0 con 0x7f9738004d00 2023-10-08T07:16:11.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f9751675640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974c152bc0 con 0x7f974c1515a0 2023-10-08T07:16:11.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f974b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f974c153980 0x7f974c0ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f974b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f974c153980 0x7f974c0ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57392/0 (socket says 172.21.15.70:57392) 2023-10-08T07:16:11.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f974b7fe640 1 -- 172.21.15.70:0/1567172887 learned_addr learned my addr 172.21.15.70:0/1567172887 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:11.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f974affd640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9738004d00 0x7f974c150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f9743fff640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c1515a0 0x7f974c0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f974b7fe640 1 -- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c1515a0 msgr2=0x7f974c0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f974b7fe640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c1515a0 0x7f974c0c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f974b7fe640 1 -- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9738004d00 msgr2=0x7f974c150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f974b7fe640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9738004d00 0x7f974c150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.084+0000 7f974b7fe640 1 -- 172.21.15.70:0/1567172887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c152ea0 con 0x7f974c153980 2023-10-08T07:16:11.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.085+0000 7f974b7fe640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f974c153980 0x7f974c0ce5c0 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7f9744001cb0 tx=0x7f9744001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:11.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.085+0000 7f97437fe640 1 -- 172.21.15.70:0/1567172887 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744001250 con 0x7f974c153980 2023-10-08T07:16:11.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.085+0000 7f97437fe640 1 -- 172.21.15.70:0/1567172887 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974402ce90 con 0x7f974c153980 2023-10-08T07:16:11.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.085+0000 7f97437fe640 1 -- 172.21.15.70:0/1567172887 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744040680 con 0x7f974c153980 2023-10-08T07:16:11.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.085+0000 7f9751675640 1 -- 172.21.15.70:0/1567172887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c153180 con 0x7f974c153980 2023-10-08T07:16:11.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.085+0000 7f9751675640 1 -- 172.21.15.70:0/1567172887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c153e00 con 0x7f974c153980 2023-10-08T07:16:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.086+0000 7f9751675640 1 -- 172.21.15.70:0/1567172887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c067490 con 0x7f974c153980 2023-10-08T07:16:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.087+0000 7f97437fe640 1 -- 172.21.15.70:0/1567172887 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 154) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f9744031030 con 0x7f974c153980 2023-10-08T07:16:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.088+0000 7f97437fe640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f972c06af00 0x7f972c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.088+0000 7f97437fe640 1 -- 172.21.15.70:0/1567172887 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 7005+0+0 (secure 0 0 0) 0x7f97440ab590 con 0x7f974c153980 2023-10-08T07:16:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.089+0000 7f974affd640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f972c06af00 0x7f972c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.090+0000 7f974affd640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f972c06af00 0x7f972c06d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f974c1f1180 tx=0x7f973c008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:11.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.092+0000 7f97437fe640 1 -- 172.21.15.70:0/1567172887 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9744075f30 con 0x7f974c153980 2023-10-08T07:16:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.318+0000 7f9751675640 1 -- 172.21.15.70:0/1567172887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f974c15bb20 con 0x7f974c153980 2023-10-08T07:16:11.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.320+0000 7f97437fe640 1 -- 172.21.15.70:0/1567172887 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4923 (secure 0 0 0) 0x7f9744045070 con 0x7f974c153980 2023-10-08T07:16:11.321 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:11.321 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":140,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/949448327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":949448327},{"type":"v1","addr":"172.21.15.70:6835","nonce":949448327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":8111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2423877406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2423877406},{"type":"v1","addr":"172.21.15.181:6837","nonce":2423877406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/247109890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":247109890},{"type":"v1","addr":"172.21.15.181:6836","nonce":247109890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:15:45.890217+0000","modified":"2023-10-08T07:15:48.920991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12645},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12645":{"gid":12645,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/753129368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":753129368},{"type":"v1","addr":"172.21.15.70:6834","nonce":753129368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-08T07:16:11.321 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 140 2023-10-08T07:16:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 -- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f972c06af00 msgr2=0x7f972c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f972c06af00 0x7f972c06d3c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f974c1f1180 tx=0x7f973c008040 comp rx=0 tx=0).stop 2023-10-08T07:16:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 -- 172.21.15.70:0/1567172887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f974c153980 msgr2=0x7f974c0ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f974c153980 0x7f974c0ce5c0 secure :-1 s=READY pgs=2974 cs=0 l=1 rev1=1 crypto rx=0x7f9744001cb0 tx=0x7f9744001d80 comp rx=0 tx=0).stop 2023-10-08T07:16:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 -- 172.21.15.70:0/1567172887 shutdown_connections 2023-10-08T07:16:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f972c06af00 0x7f972c06d3c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f974c153980 0x7f974c0ce5c0 unknown :-1 s=CLOSED pgs=2974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f974c1515a0 0x7f974c0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 --2- 172.21.15.70:0/1567172887 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f9738004d00 0x7f974c150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 -- 172.21.15.70:0/1567172887 >> 172.21.15.70:0/1567172887 conn(0x7f974c06a380 msgr2=0x7f974c0ae570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:11.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 -- 172.21.15.70:0/1567172887 shutdown_connections 2023-10-08T07:16:11.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.322+0000 7f9751675640 1 -- 172.21.15.70:0/1567172887 wait complete. 2023-10-08T07:16:11.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:16:11.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.539+0000 7fee4f41a640 1 Processor -- start 2023-10-08T07:16:11.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.540+0000 7fee4f41a640 1 -- start start 2023-10-08T07:16:11.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.542+0000 7fee4f41a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48165cc0 0x7fee481680b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.542+0000 7fee4f41a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee480c48f0 con 0x7fee4815cd90 2023-10-08T07:16:11.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.542+0000 7fee4f41a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee480710f0 con 0x7fee48160940 2023-10-08T07:16:11.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.542+0000 7fee4f41a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee480c4d80 con 0x7fee48165cc0 2023-10-08T07:16:11.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.542+0000 7fee4d1b6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fee4815cd90 0x7fee480702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49062/0 (socket says 172.21.15.70:49062) 2023-10-08T07:16:11.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.542+0000 7fee4d1b6640 1 -- 172.21.15.70:0/3197495963 learned_addr learned my addr 172.21.15.70:0/3197495963 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:11.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.542+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203245372 0 0) 0x7fee480c48f0 con 0x7fee4815cd90 2023-10-08T07:16:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.542+0000 7fee4d9b7640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48165cc0 0x7fee481680b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.542+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee34003520 con 0x7fee4815cd90 2023-10-08T07:16:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 698527458 0 0) 0x7fee34003520 con 0x7fee4815cd90 2023-10-08T07:16:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48165cc0 msgr2=0x7fee481680b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48165cc0 0x7fee481680b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 >> v1:172.21.15.181:6789/0 conn(0x7fee48160940 legacy=0x7fee480709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee480c3ce0 con 0x7fee4815cd90 2023-10-08T07:16:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fee44001670 con 0x7fee4815cd90 2023-10-08T07:16:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 >> v1:172.21.15.70:6789/0 conn(0x7fee4815cd90 legacy=0x7fee480702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee340039a0 0x7fee34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48165cc0 0x7fee340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee34004d00 0x7fee34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3400ba60 con 0x7fee34004d00 2023-10-08T07:16:11.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3400bbd0 con 0x7fee340039a0 2023-10-08T07:16:11.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3400bd90 con 0x7fee48165cc0 2023-10-08T07:16:11.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.543+0000 7fee4d9b7640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee34004d00 0x7fee34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.544+0000 7fee4d1b6640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee340039a0 0x7fee34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.544+0000 7fee4d9b7640 1 -- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48165cc0 msgr2=0x7fee340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:11.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.544+0000 7fee4d9b7640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48165cc0 0x7fee340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.544+0000 7fee4d9b7640 1 -- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee340039a0 msgr2=0x7fee34003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:11.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.544+0000 7fee4d9b7640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee340039a0 0x7fee34003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.544+0000 7fee4d9b7640 1 -- 172.21.15.70:0/3197495963 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee34003520 con 0x7fee34004d00 2023-10-08T07:16:11.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.544+0000 7fee4d1b6640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee340039a0 0x7fee34003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:11.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.545+0000 7fee4d9b7640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee34004d00 0x7fee34005180 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7fee40009f90 tx=0x7fee340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483db74c76a47250 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:11.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.545+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4002cc20 con 0x7fee34004d00 2023-10-08T07:16:11.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.545+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4000f040 con 0x7fee34004d00 2023-10-08T07:16:11.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.545+0000 7fee3ffff640 1 -- 172.21.15.70:0/3197495963 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee40034440 con 0x7fee34004d00 2023-10-08T07:16:11.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.546+0000 7fee4f41a640 1 -- 172.21.15.70:0/3197495963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee34004d00 msgr2=0x7fee34005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.546+0000 7fee4f41a640 1 --2- 172.21.15.70:0/3197495963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee34004d00 0x7fee34005180 secure :-1 s=READY pgs=2976 cs=0 l=1 rev1=1 crypto rx=0x7fee40009f90 tx=0x7fee340057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.546+0000 7fee4d9b7640 1 -- 172.21.15.70:0/3197495963 reap_dead start 2023-10-08T07:16:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.546+0000 7fee4f41a640 1 -- 172.21.15.70:0/3197495963 shutdown_connections 2023-10-08T07:16:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.546+0000 7fee4f41a640 1 -- 172.21.15.70:0/3197495963 >> 172.21.15.70:0/3197495963 conn(0x7fee4806a380 msgr2=0x7fee480b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.546+0000 7fee4f41a640 1 -- 172.21.15.70:0/3197495963 shutdown_connections 2023-10-08T07:16:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.546+0000 7fee4f41a640 1 -- 172.21.15.70:0/3197495963 wait complete. 2023-10-08T07:16:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.548+0000 7fee4f41a640 1 Processor -- start 2023-10-08T07:16:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.548+0000 7fee4f41a640 1 -- start start 2023-10-08T07:16:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4f41a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34004d00 0x7fee4806cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4f41a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48150c50 0x7fee480b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4f41a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee480b1de0 0x7fee4815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4f41a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4815e140 con 0x7fee480b1de0 2023-10-08T07:16:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4f41a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4815db80 con 0x7fee34004d00 2023-10-08T07:16:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4f41a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee48156220 con 0x7fee48150c50 2023-10-08T07:16:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4d9b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee480b1de0 0x7fee4815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4d9b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee480b1de0 0x7fee4815e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57410/0 (socket says 172.21.15.70:57410) 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4d9b7640 1 -- 172.21.15.70:0/2256352027 learned_addr learned my addr 172.21.15.70:0/2256352027 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4d9b7640 1 -- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48150c50 msgr2=0x7fee480b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4d9b7640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48150c50 0x7fee480b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4d9b7640 1 -- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34004d00 msgr2=0x7fee4806cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4d9b7640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34004d00 0x7fee4806cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.549+0000 7fee4d9b7640 1 -- 172.21.15.70:0/2256352027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48156500 con 0x7fee480b1de0 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.550+0000 7fee4d9b7640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee480b1de0 0x7fee4815e5d0 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7fee40037040 tx=0x7fee40009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.550+0000 7fee3dffb640 1 -- 172.21.15.70:0/2256352027 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee40034d50 con 0x7fee480b1de0 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.550+0000 7fee3dffb640 1 -- 172.21.15.70:0/2256352027 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee40003990 con 0x7fee480b1de0 2023-10-08T07:16:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.550+0000 7fee3dffb640 1 -- 172.21.15.70:0/2256352027 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee40003d20 con 0x7fee480b1de0 2023-10-08T07:16:11.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.550+0000 7fee4f41a640 1 -- 172.21.15.70:0/2256352027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4815ddd0 con 0x7fee480b1de0 2023-10-08T07:16:11.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.550+0000 7fee4f41a640 1 -- 172.21.15.70:0/2256352027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4806d220 con 0x7fee480b1de0 2023-10-08T07:16:11.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.551+0000 7fee4f41a640 1 -- 172.21.15.70:0/2256352027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee48156690 con 0x7fee480b1de0 2023-10-08T07:16:11.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.552+0000 7fee3dffb640 1 -- 172.21.15.70:0/2256352027 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 154) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fee40004030 con 0x7fee480b1de0 2023-10-08T07:16:11.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.553+0000 7fee3dffb640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee2c06af00 0x7fee2c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:11.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.553+0000 7fee3dffb640 1 -- 172.21.15.70:0/2256352027 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 7005+0+0 (secure 0 0 0) 0x7fee400ab600 con 0x7fee480b1de0 2023-10-08T07:16:11.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.554+0000 7fee4d1b6640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee2c06af00 0x7fee2c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:11.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.555+0000 7fee4d1b6640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee2c06af00 0x7fee2c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fee44007a30 tx=0x7fee44009040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:11.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.557+0000 7fee3dffb640 1 -- 172.21.15.70:0/2256352027 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee400761f0 con 0x7fee480b1de0 2023-10-08T07:16:11.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.783+0000 7fee4f41a640 1 -- 172.21.15.70:0/2256352027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee48067490 con 0x7fee480b1de0 2023-10-08T07:16:11.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.785+0000 7fee3dffb640 1 -- 172.21.15.70:0/2256352027 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+4923 (secure 0 0 0) 0x7fee40079810 con 0x7fee480b1de0 2023-10-08T07:16:11.785 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:11.785 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":140,"default_fscid":17,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/949448327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":949448327},{"type":"v1","addr":"172.21.15.70:6835","nonce":949448327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":8111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2423877406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2423877406},{"type":"v1","addr":"172.21.15.181:6837","nonce":2423877406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/247109890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":247109890},{"type":"v1","addr":"172.21.15.181:6836","nonce":247109890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T07:15:45.890217+0000","modified":"2023-10-08T07:15:48.920991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":12645},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12645":{"gid":12645,"name":"c","rank":0,"incarnation":136,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6834/753129368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":753129368},{"type":"v1","addr":"172.21.15.70:6834","nonce":753129368}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[36],"metadata_pool":35,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":17}]} 2023-10-08T07:16:11.786 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 140 2023-10-08T07:16:11.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 -- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee2c06af00 msgr2=0x7fee2c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee2c06af00 0x7fee2c06d3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fee44007a30 tx=0x7fee44009040 comp rx=0 tx=0).stop 2023-10-08T07:16:11.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 -- 172.21.15.70:0/2256352027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee480b1de0 msgr2=0x7fee4815e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:11.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee480b1de0 0x7fee4815e5d0 secure :-1 s=READY pgs=2977 cs=0 l=1 rev1=1 crypto rx=0x7fee40037040 tx=0x7fee40009750 comp rx=0 tx=0).stop 2023-10-08T07:16:11.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 -- 172.21.15.70:0/2256352027 shutdown_connections 2023-10-08T07:16:11.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee2c06af00 0x7fee2c06d3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee480b1de0 0x7fee4815e5d0 unknown :-1 s=CLOSED pgs=2977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee48150c50 0x7fee480b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 --2- 172.21.15.70:0/2256352027 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee34004d00 0x7fee4806cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:11.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 -- 172.21.15.70:0/2256352027 >> 172.21.15.70:0/2256352027 conn(0x7fee4806a380 msgr2=0x7fee480ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:11.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 -- 172.21.15.70:0/2256352027 shutdown_connections 2023-10-08T07:16:11.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:11.787+0000 7fee4f41a640 1 -- 172.21.15.70:0/2256352027 wait complete. 2023-10-08T07:16:11.806 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:16:12.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.018+0000 7f33e4fd0640 1 Processor -- start 2023-10-08T07:16:12.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.018+0000 7f33e4fd0640 1 -- start start 2023-10-08T07:16:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.019+0000 7f33e4fd0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f33e0157000 0x7f33e00cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.019+0000 7f33e4fd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e01597d0 0x7f33e006ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.020+0000 7f33e4fd0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e00adfc0 con 0x7f33e00cb190 2023-10-08T07:16:12.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.020+0000 7f33e4fd0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e00ccce0 con 0x7f33e0157000 2023-10-08T07:16:12.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.020+0000 7f33e4fd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e006d380 con 0x7f33e01597d0 2023-10-08T07:16:12.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.020+0000 7f33ded9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f33e00cb190 0x7f33e015d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49074/0 (socket says 172.21.15.70:49074) 2023-10-08T07:16:12.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.020+0000 7f33ded9d640 1 -- 172.21.15.70:0/2032525631 learned_addr learned my addr 172.21.15.70:0/2032525631 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.020+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931079203 0 0) 0x7f33e00adfc0 con 0x7f33e00cb190 2023-10-08T07:16:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.020+0000 7f33de59c640 1 --2- 172.21.15.70:0/2032525631 >> v2:172.21.15.181:3300/0 conn(0x7f33e0157000 0x7f33e00cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.020+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33cc003520 con 0x7f33e00cb190 2023-10-08T07:16:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.020+0000 7f33d7fff640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e01597d0 0x7f33e006ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2200468530 0 0) 0x7f33cc003520 con 0x7f33e00cb190 2023-10-08T07:16:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e01597d0 msgr2=0x7f33e006ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e01597d0 0x7f33e006ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 >> v2:172.21.15.181:3300/0 conn(0x7f33e0157000 msgr2=0x7f33e00cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 --2- 172.21.15.70:0/2032525631 >> v2:172.21.15.181:3300/0 conn(0x7f33e0157000 0x7f33e00cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e0159cb0 con 0x7f33e00cb190 2023-10-08T07:16:12.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f33d80019f0 con 0x7f33e00cb190 2023-10-08T07:16:12.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33d7fff640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e01597d0 0x7f33e006ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:16:12.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 >> v1:172.21.15.70:6789/0 conn(0x7f33e00cb190 legacy=0x7f33e015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e01597d0 0x7f33cc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 0x7f33cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33cc004d00 0x7f33cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00ccce0 con 0x7f33cc003fb0 2023-10-08T07:16:12.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e006d380 con 0x7f33cc004d00 2023-10-08T07:16:12.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc00dcc0 con 0x7f33e01597d0 2023-10-08T07:16:12.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33de59c640 1 --2- 172.21.15.70:0/2032525631 >> v2:172.21.15.181:3300/0 conn(0x7f33e0157000 0x7f33e00cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:16:12.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.021+0000 7f33d7fff640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 0x7f33cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.022+0000 7f33d7fff640 1 -- 172.21.15.70:0/2032525631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e01597d0 msgr2=0x7f33cc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:12.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.022+0000 7f33d7fff640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e01597d0 0x7f33cc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.022+0000 7f33d7fff640 1 -- 172.21.15.70:0/2032525631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33cc004d00 msgr2=0x7f33cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:12.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.022+0000 7f33d7fff640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33cc004d00 0x7f33cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.022+0000 7f33d7fff640 1 -- 172.21.15.70:0/2032525631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc003520 con 0x7f33cc003fb0 2023-10-08T07:16:12.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.023+0000 7f33d7fff640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 0x7f33cc0047c0 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7f33c800a9a0 tx=0x7f33cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1813e7e8256d5d0f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:12.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.023+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8004230 con 0x7f33cc003fb0 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.023+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c8004390 con 0x7f33cc003fb0 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.023+0000 7f33ddd9b640 1 -- 172.21.15.70:0/2032525631 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c802c910 con 0x7f33cc003fb0 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.024+0000 7f33e4fd0640 1 -- 172.21.15.70:0/2032525631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 msgr2=0x7f33cc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.024+0000 7f33e4fd0640 1 --2- 172.21.15.70:0/2032525631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 0x7f33cc0047c0 secure :-1 s=READY pgs=2979 cs=0 l=1 rev1=1 crypto rx=0x7f33c800a9a0 tx=0x7f33cc0057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.024+0000 7f33ded9d640 1 -- 172.21.15.70:0/2032525631 reap_dead start 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.024+0000 7f33e4fd0640 1 -- 172.21.15.70:0/2032525631 shutdown_connections 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.024+0000 7f33e4fd0640 1 -- 172.21.15.70:0/2032525631 >> 172.21.15.70:0/2032525631 conn(0x7f33e006a380 msgr2=0x7f33e00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.024+0000 7f33e4fd0640 1 -- 172.21.15.70:0/2032525631 shutdown_connections 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.024+0000 7f33e4fd0640 1 -- 172.21.15.70:0/2032525631 wait complete. 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.025+0000 7f33e4fd0640 1 Processor -- start 2023-10-08T07:16:12.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.026+0000 7f33e4fd0640 1 -- start start 2023-10-08T07:16:12.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33e4fd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 0x7f33e0144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33e4fd0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e00b25f0 0x7f33e00b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33e4fd0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e0158440 0x7f33e01588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33e4fd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e0158e00 con 0x7f33cc003fb0 2023-10-08T07:16:12.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33e4fd0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e01f1930 con 0x7f33e0158440 2023-10-08T07:16:12.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33e4fd0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e01f1c10 con 0x7f33e00b25f0 2023-10-08T07:16:12.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33de59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 0x7f33e0144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33de59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 0x7f33e0144420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57428/0 (socket says 172.21.15.70:57428) 2023-10-08T07:16:12.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33de59c640 1 -- 172.21.15.70:0/4278176036 learned_addr learned my addr 172.21.15.70:0/4278176036 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:12.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33de59c640 1 -- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e00b25f0 msgr2=0x7f33e00b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33de59c640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e00b25f0 0x7f33e00b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33de59c640 1 -- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e0158440 msgr2=0x7f33e01588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33de59c640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e0158440 0x7f33e01588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.027+0000 7f33de59c640 1 -- 172.21.15.70:0/4278176036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e01f1ef0 con 0x7f33cc003fb0 2023-10-08T07:16:12.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.028+0000 7f33de59c640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 0x7f33e0144420 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f33d000e7e0 tx=0x7f33d000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:12.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.028+0000 7f33d6ffd640 1 -- 172.21.15.70:0/4278176036 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0004230 con 0x7f33cc003fb0 2023-10-08T07:16:12.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.028+0000 7f33d6ffd640 1 -- 172.21.15.70:0/4278176036 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d000ee90 con 0x7f33cc003fb0 2023-10-08T07:16:12.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.028+0000 7f33d6ffd640 1 -- 172.21.15.70:0/4278176036 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d000c6d0 con 0x7f33cc003fb0 2023-10-08T07:16:12.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.028+0000 7f33e4fd0640 1 -- 172.21.15.70:0/4278176036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e01f21d0 con 0x7f33cc003fb0 2023-10-08T07:16:12.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.028+0000 7f33e4fd0640 1 -- 172.21.15.70:0/4278176036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e01f15e0 con 0x7f33cc003fb0 2023-10-08T07:16:12.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.029+0000 7f33e4fd0640 1 -- 172.21.15.70:0/4278176036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e0067490 con 0x7f33cc003fb0 2023-10-08T07:16:12.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.030+0000 7f33d6ffd640 1 -- 172.21.15.70:0/4278176036 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 154) v1 ==== 89275+0+0 (secure 0 0 0) 0x7f33d000c830 con 0x7f33cc003fb0 2023-10-08T07:16:12.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.031+0000 7f33d6ffd640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f33c006afd0 0x7f33c006d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.031+0000 7f33d6ffd640 1 -- 172.21.15.70:0/4278176036 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 7005+0+0 (secure 0 0 0) 0x7f33d0089990 con 0x7f33cc003fb0 2023-10-08T07:16:12.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.032+0000 7f33d7fff640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f33c006afd0 0x7f33c006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.033+0000 7f33d7fff640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f33c006afd0 0x7f33c006d490 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f33c8035000 tx=0x7f33c802e000 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:12.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.035+0000 7f33d6ffd640 1 -- 172.21.15.70:0/4278176036 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33d00543b0 con 0x7f33cc003fb0 2023-10-08T07:16:12.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.224+0000 7f33e4fd0640 1 -- 172.21.15.70:0/4278176036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33e006c6d0 con 0x7f33cc003fb0 2023-10-08T07:16:12.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.227+0000 7f33d6ffd640 1 -- 172.21.15.70:0/4278176036 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v278) v1 ==== 74+0+18157 (secure 0 0 0) 0x7f33d00579d0 con 0x7f33cc003fb0 2023-10-08T07:16:12.228 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:12.228 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":278,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:16:08.864637+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":35,"pool_name":"cephfs_metadata","create_time":"2023-10-08T07:15:43.819948+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"275","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":36,"pool_name":"cephfs_data","create_time":"2023-10-08T07:15:44.895969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"276","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.181:0/3049540535":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/220528716":"2023-10-09T07:16:08.864584+0000","172.21.15.181:6833/1127370729":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/2060656952":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1823452952":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/3814802416":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/3705279960":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6833/440573135":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6832/440573135":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6832/1127370729":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1545538679":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/1740402122":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1151602740":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2253796914":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/3307173154":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2296566554":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2564762528":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/606980":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/796435327":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1513601554":"2023-10-08T08:15:42.823407+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:16:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 -- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f33c006afd0 msgr2=0x7f33c006d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f33c006afd0 0x7f33c006d490 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f33c8035000 tx=0x7f33c802e000 comp rx=0 tx=0).stop 2023-10-08T07:16:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 -- 172.21.15.70:0/4278176036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 msgr2=0x7f33e0144420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 0x7f33e0144420 secure :-1 s=READY pgs=2980 cs=0 l=1 rev1=1 crypto rx=0x7f33d000e7e0 tx=0x7f33d000eca0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 -- 172.21.15.70:0/4278176036 shutdown_connections 2023-10-08T07:16:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f33c006afd0 0x7f33c006d490 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f33e0158440 0x7f33e01588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f33e00b25f0 0x7f33e00b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 --2- 172.21.15.70:0/4278176036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33cc003fb0 0x7f33e0144420 unknown :-1 s=CLOSED pgs=2980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 -- 172.21.15.70:0/4278176036 >> 172.21.15.70:0/4278176036 conn(0x7f33e006a380 msgr2=0x7f33e00cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 -- 172.21.15.70:0/4278176036 shutdown_connections 2023-10-08T07:16:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.230+0000 7f33e4fd0640 1 -- 172.21.15.70:0/4278176036 wait complete. 2023-10-08T07:16:12.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T07:16:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.462+0000 7fafaa04f640 1 Processor -- start 2023-10-08T07:16:12.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.462+0000 7fafaa04f640 1 -- start start 2023-10-08T07:16:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.464+0000 7fafaa04f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fafa40afbe0 0x7fafa40b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.464+0000 7fafaa04f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa40b2620 con 0x7fafa406dfc0 2023-10-08T07:16:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.464+0000 7fafaa04f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa40b2060 con 0x7fafa40cb130 2023-10-08T07:16:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.464+0000 7fafaa04f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa40b2340 con 0x7fafa40afbe0 2023-10-08T07:16:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.464+0000 7fafa37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fafa40afbe0 0x7fafa40b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.464+0000 7fafa37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fafa40afbe0 0x7fafa40b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57438/0 (socket says 172.21.15.70:57438) 2023-10-08T07:16:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.464+0000 7fafa37fe640 1 -- 172.21.15.70:0/371159383 learned_addr learned my addr 172.21.15.70:0/371159383 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:12.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.464+0000 7faf9bfff640 1 --1- 172.21.15.70:0/371159383 >> v1:172.21.15.70:6789/0 conn(0x7fafa406dfc0 0x7fafa4070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49086/0 (socket says 172.21.15.70:49086) 2023-10-08T07:16:12.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.465+0000 7fafa37fe640 1 -- 172.21.15.70:0/371159383 >> v1:172.21.15.181:6789/0 conn(0x7fafa40cb130 legacy=0x7fafa406c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:12.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.465+0000 7fafa37fe640 1 -- 172.21.15.70:0/371159383 >> v1:172.21.15.70:6789/0 conn(0x7fafa406dfc0 legacy=0x7fafa4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.465+0000 7fafa37fe640 1 -- 172.21.15.70:0/371159383 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa4070350 con 0x7fafa40afbe0 2023-10-08T07:16:12.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.466+0000 7fafa37fe640 1 --2- 172.21.15.70:0/371159383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa40afbe0 0x7fafa40b15b0 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7faf94009870 tx=0x7faf9402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c734b9e992dc3a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:12.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.466+0000 7fafa2ffd640 1 -- 172.21.15.70:0/371159383 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94030860 con 0x7fafa40afbe0 2023-10-08T07:16:12.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.466+0000 7fafa2ffd640 1 -- 172.21.15.70:0/371159383 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf940309c0 con 0x7fafa40afbe0 2023-10-08T07:16:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.466+0000 7fafa2ffd640 1 -- 172.21.15.70:0/371159383 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94030d50 con 0x7fafa40afbe0 2023-10-08T07:16:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.467+0000 7fafaa04f640 1 -- 172.21.15.70:0/371159383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa40afbe0 msgr2=0x7fafa40b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.467+0000 7fafaa04f640 1 --2- 172.21.15.70:0/371159383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa40afbe0 0x7fafa40b15b0 secure :-1 s=READY pgs=2982 cs=0 l=1 rev1=1 crypto rx=0x7faf94009870 tx=0x7faf9402b9a0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.467+0000 7fafaa04f640 1 -- 172.21.15.70:0/371159383 shutdown_connections 2023-10-08T07:16:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.467+0000 7fafaa04f640 1 --2- 172.21.15.70:0/371159383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa40afbe0 0x7fafa40b15b0 unknown :-1 s=CLOSED pgs=2982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.467+0000 7fafaa04f640 1 -- 172.21.15.70:0/371159383 >> 172.21.15.70:0/371159383 conn(0x7fafa406a5d0 msgr2=0x7fafa40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.467+0000 7fafaa04f640 1 -- 172.21.15.70:0/371159383 shutdown_connections 2023-10-08T07:16:12.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.467+0000 7fafaa04f640 1 -- 172.21.15.70:0/371159383 wait complete. 2023-10-08T07:16:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.469+0000 7fafaa04f640 1 Processor -- start 2023-10-08T07:16:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.469+0000 7fafaa04f640 1 -- start start 2023-10-08T07:16:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafaa04f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa406dfc0 0x7fafa40cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafaa04f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafa40afbe0 0x7fafa4157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafaa04f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafa40cb130 0x7fafa40c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafaa04f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa40c37c0 con 0x7fafa406dfc0 2023-10-08T07:16:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafaa04f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa40c3a10 con 0x7fafa40cb130 2023-10-08T07:16:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafaa04f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa40c3cf0 con 0x7fafa40afbe0 2023-10-08T07:16:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafa37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa406dfc0 0x7fafa40cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafa37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa406dfc0 0x7fafa40cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57450/0 (socket says 172.21.15.70:57450) 2023-10-08T07:16:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafa37fe640 1 -- 172.21.15.70:0/3053333960 learned_addr learned my addr 172.21.15.70:0/3053333960 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafa37fe640 1 -- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafa40afbe0 msgr2=0x7fafa4157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafa37fe640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafa40afbe0 0x7fafa4157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafa37fe640 1 -- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafa40cb130 msgr2=0x7fafa40c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafa37fe640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafa40cb130 0x7fafa40c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.470+0000 7fafa37fe640 1 -- 172.21.15.70:0/3053333960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf940094d0 con 0x7fafa406dfc0 2023-10-08T07:16:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.471+0000 7fafa37fe640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa406dfc0 0x7fafa40cd990 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7faf94009870 tx=0x7faf94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.471+0000 7fafa0ff9640 1 -- 172.21.15.70:0/3053333960 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94030810 con 0x7fafa406dfc0 2023-10-08T07:16:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.471+0000 7fafa0ff9640 1 -- 172.21.15.70:0/3053333960 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf94043400 con 0x7fafa406dfc0 2023-10-08T07:16:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.471+0000 7fafa0ff9640 1 -- 172.21.15.70:0/3053333960 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf940437b0 con 0x7fafa406dfc0 2023-10-08T07:16:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.471+0000 7fafaa04f640 1 -- 172.21.15.70:0/3053333960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa40c3fd0 con 0x7fafa406dfc0 2023-10-08T07:16:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.471+0000 7fafaa04f640 1 -- 172.21.15.70:0/3053333960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa40c4590 con 0x7fafa406dfc0 2023-10-08T07:16:12.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.472+0000 7fafaa04f640 1 -- 172.21.15.70:0/3053333960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa40c47b0 con 0x7fafa406dfc0 2023-10-08T07:16:12.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.473+0000 7fafa0ff9640 1 -- 172.21.15.70:0/3053333960 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 154) v1 ==== 89275+0+0 (secure 0 0 0) 0x7faf94043910 con 0x7fafa406dfc0 2023-10-08T07:16:12.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.474+0000 7fafa0ff9640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7faf8406af00 0x7faf8406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.474+0000 7fafa0ff9640 1 -- 172.21.15.70:0/3053333960 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(278..278 src has 1..278) v4 ==== 7005+0+0 (secure 0 0 0) 0x7faf9403e070 con 0x7fafa406dfc0 2023-10-08T07:16:12.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.475+0000 7faf9bfff640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7faf8406af00 0x7faf8406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.476+0000 7faf9bfff640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7faf8406af00 0x7faf8406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fafa406d240 tx=0x7faf8c0079c0 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:12.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.479+0000 7fafa0ff9640 1 -- 172.21.15.70:0/3053333960 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf94077b80 con 0x7fafa406dfc0 2023-10-08T07:16:12.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.703+0000 7fafaa04f640 1 -- 172.21.15.70:0/3053333960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fafa404e160 con 0x7fafa406dfc0 2023-10-08T07:16:12.705 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-10-08T07:16:12.704+0000 7fe74816e640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T07:16:12.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.708+0000 7fafa0ff9640 1 -- 172.21.15.70:0/3053333960 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v141) v1 ==== 161+0+0 (secure 0 0 0) 0x7faf94034070 con 0x7fafa406dfc0 2023-10-08T07:16:12.709 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T07:16:12.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.711+0000 7fafaa04f640 1 -- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7faf8406af00 msgr2=0x7faf8406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.711+0000 7fafaa04f640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7faf8406af00 0x7faf8406d3c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fafa406d240 tx=0x7faf8c0079c0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.711+0000 7fafaa04f640 1 -- 172.21.15.70:0/3053333960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa406dfc0 msgr2=0x7fafa40cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.711+0000 7fafaa04f640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa406dfc0 0x7fafa40cd990 secure :-1 s=READY pgs=2983 cs=0 l=1 rev1=1 crypto rx=0x7faf94009870 tx=0x7faf94004730 comp rx=0 tx=0).stop 2023-10-08T07:16:12.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.712+0000 7fafaa04f640 1 -- 172.21.15.70:0/3053333960 shutdown_connections 2023-10-08T07:16:12.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.712+0000 7fafaa04f640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7faf8406af00 0x7faf8406d3c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.712+0000 7fafaa04f640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fafa40cb130 0x7fafa40c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.712+0000 7fafaa04f640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fafa40afbe0 0x7fafa4157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.712+0000 7fafaa04f640 1 --2- 172.21.15.70:0/3053333960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fafa406dfc0 0x7fafa40cd990 unknown :-1 s=CLOSED pgs=2983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.712+0000 7fafaa04f640 1 -- 172.21.15.70:0/3053333960 >> 172.21.15.70:0/3053333960 conn(0x7fafa406a5d0 msgr2=0x7fafa40c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:12.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.712+0000 7fafaa04f640 1 -- 172.21.15.70:0/3053333960 shutdown_connections 2023-10-08T07:16:12.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.712+0000 7fafaa04f640 1 -- 172.21.15.70:0/3053333960 wait complete. 2023-10-08T07:16:12.718 INFO:tasks.ceph.mds.c.smithi070.stderr: -2982> 2023-10-08T07:15:50.458+0000 7f8476b22640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T07:16:12.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T07:16:12.783 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-10-08T07:16:12.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.918+0000 7fbc4f59e640 1 Processor -- start 2023-10-08T07:16:12.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.918+0000 7fbc4f59e640 1 -- start start 2023-10-08T07:16:12.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.919+0000 7fbc4f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc500cf2b0 0x7fbc500d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.919+0000 7fbc4f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc500ce510 0x7fbc5015c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.919+0000 7fbc4f59e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc500cd520 con 0x7fbc500cf690 2023-10-08T07:16:12.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.919+0000 7fbc4f59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc500d1910 con 0x7fbc500ce510 2023-10-08T07:16:12.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.919+0000 7fbc4f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500ce080 con 0x7fbc500cf2b0 2023-10-08T07:16:12.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.920+0000 7fbc4ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc500ce510 0x7fbc5015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.920+0000 7fbc4ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc500ce510 0x7fbc5015c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57472/0 (socket says 172.21.15.70:57472) 2023-10-08T07:16:12.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.920+0000 7fbc4ed9d640 1 -- 172.21.15.70:0/1345866628 learned_addr learned my addr 172.21.15.70:0/1345866628 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:12.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.920+0000 7fbc4ed9d640 1 -- 172.21.15.70:0/1345866628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc500cf2b0 msgr2=0x7fbc500d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:12.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.920+0000 7fbc4ed9d640 1 --2- 172.21.15.70:0/1345866628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc500cf2b0 0x7fbc500d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.920+0000 7fbc4ed9d640 1 -- 172.21.15.70:0/1345866628 >> v1:172.21.15.181:6789/0 conn(0x7fbc500cf690 legacy=0x7fbc500d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:12.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.920+0000 7fbc4ed9d640 1 -- 172.21.15.70:0/1345866628 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc500cd750 con 0x7fbc500ce510 2023-10-08T07:16:12.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.921+0000 7fbc4ed9d640 1 --2- 172.21.15.70:0/1345866628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500ce510 0x7fbc5015c820 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009e50 tx=0x7fbc4802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc98fd30cdfed84 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:12.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4dd9b640 1 -- 172.21.15.70:0/1345866628 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4802c9f0 con 0x7fbc500ce510 2023-10-08T07:16:12.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4dd9b640 1 -- 172.21.15.70:0/1345866628 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4802cb50 con 0x7fbc500ce510 2023-10-08T07:16:12.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4dd9b640 1 -- 172.21.15.70:0/1345866628 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc480343f0 con 0x7fbc500ce510 2023-10-08T07:16:12.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1345866628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500ce510 msgr2=0x7fbc5015c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4f59e640 1 --2- 172.21.15.70:0/1345866628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500ce510 0x7fbc5015c820 secure :-1 s=READY pgs=2986 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009e50 tx=0x7fbc4802be50 comp rx=0 tx=0).stop 2023-10-08T07:16:12.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1345866628 shutdown_connections 2023-10-08T07:16:12.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4f59e640 1 --2- 172.21.15.70:0/1345866628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500ce510 0x7fbc5015c820 unknown :-1 s=CLOSED pgs=2986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4f59e640 1 --2- 172.21.15.70:0/1345866628 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc500cf2b0 0x7fbc500d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1345866628 >> 172.21.15.70:0/1345866628 conn(0x7fbc5009bc40 msgr2=0x7fbc500d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:12.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1345866628 shutdown_connections 2023-10-08T07:16:12.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.922+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1345866628 wait complete. 2023-10-08T07:16:12.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.924+0000 7fbc4f59e640 1 Processor -- start 2023-10-08T07:16:12.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.924+0000 7fbc4f59e640 1 -- start start 2023-10-08T07:16:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.925+0000 7fbc4f59e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc500ce510 0x7fbc500cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.925+0000 7fbc4f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500cf2b0 0x7fbc500cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.925+0000 7fbc4f59e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc500cf690 0x7fbc500c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.925+0000 7fbc4f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500cab80 con 0x7fbc500cf2b0 2023-10-08T07:16:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.925+0000 7fbc4f59e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc50154e70 con 0x7fbc500cf690 2023-10-08T07:16:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.925+0000 7fbc4f59e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500ca5c0 con 0x7fbc500ce510 2023-10-08T07:16:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.925+0000 7fbc47fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500cf2b0 0x7fbc500cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.925+0000 7fbc47fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500cf2b0 0x7fbc500cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57476/0 (socket says 172.21.15.70:57476) 2023-10-08T07:16:12.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.925+0000 7fbc47fff640 1 -- 172.21.15.70:0/1528505040 learned_addr learned my addr 172.21.15.70:0/1528505040 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:12.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc47fff640 1 -- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc500ce510 msgr2=0x7fbc500cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc47fff640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc500ce510 0x7fbc500cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc47fff640 1 -- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc500cf690 msgr2=0x7fbc500c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:12.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc47fff640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc500cf690 0x7fbc500c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:12.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc47fff640 1 -- 172.21.15.70:0/1528505040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc48009b30 con 0x7fbc500cf2b0 2023-10-08T07:16:12.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc47fff640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500cf2b0 0x7fbc500cbd00 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800b3e0 tx=0x7fbc3800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:12.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc46ffd640 1 -- 172.21.15.70:0/1528505040 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38011830 con 0x7fbc500cf2b0 2023-10-08T07:16:12.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc46ffd640 1 -- 172.21.15.70:0/1528505040 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc380119b0 con 0x7fbc500cf2b0 2023-10-08T07:16:12.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc46ffd640 1 -- 172.21.15.70:0/1528505040 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38011d60 con 0x7fbc500cf2b0 2023-10-08T07:16:12.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1528505040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc500ca8a0 con 0x7fbc500cf2b0 2023-10-08T07:16:12.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.926+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1528505040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc500b26f0 con 0x7fbc500cf2b0 2023-10-08T07:16:12.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.927+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1528505040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc500b2910 con 0x7fbc500cf2b0 2023-10-08T07:16:12.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.928+0000 7fbc46ffd640 1 -- 172.21.15.70:0/1528505040 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 154) v1 ==== 89275+0+0 (secure 0 0 0) 0x7fbc3801a400 con 0x7fbc500cf2b0 2023-10-08T07:16:12.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.929+0000 7fbc46ffd640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbc3006af00 0x7fbc3006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:12.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.930+0000 7fbc46ffd640 1 -- 172.21.15.70:0/1528505040 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(279..279 src has 1..279) v4 ==== 7091+0+0 (secure 0 0 0) 0x7fbc38089cd0 con 0x7fbc500cf2b0 2023-10-08T07:16:12.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.930+0000 7fbc4e59c640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbc3006af00 0x7fbc3006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:12.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.931+0000 7fbc4e59c640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbc3006af00 0x7fbc3006d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbc500b49d0 tx=0x7fbc40006c90 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:12.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:12.933+0000 7fbc46ffd640 1 -- 172.21.15.70:0/1528505040 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc380548b0 con 0x7fbc500cf2b0 2023-10-08T07:16:13.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.156+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1528505040 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbc50074150 con 0x7fbc500cf2b0 2023-10-08T07:16:13.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.713+0000 7fbc46ffd640 1 -- 172.21.15.70:0/1528505040 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v142) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbc38015020 con 0x7fbc500cf2b0 2023-10-08T07:16:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbc3006af00 msgr2=0x7fbc3006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbc3006af00 0x7fbc3006d3c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbc500b49d0 tx=0x7fbc40006c90 comp rx=0 tx=0).stop 2023-10-08T07:16:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1528505040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500cf2b0 msgr2=0x7fbc500cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500cf2b0 0x7fbc500cbd00 secure :-1 s=READY pgs=2987 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800b3e0 tx=0x7fbc3800b8a0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1528505040 shutdown_connections 2023-10-08T07:16:13.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbc3006af00 0x7fbc3006d3c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbc500cf690 0x7fbc500c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc500cf2b0 0x7fbc500cbd00 unknown :-1 s=CLOSED pgs=2987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 --2- 172.21.15.70:0/1528505040 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbc500ce510 0x7fbc500cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1528505040 >> 172.21.15.70:0/1528505040 conn(0x7fbc5009bc40 msgr2=0x7fbc5006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:13.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1528505040 shutdown_connections 2023-10-08T07:16:13.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.717+0000 7fbc4f59e640 1 -- 172.21.15.70:0/1528505040 wait complete. 2023-10-08T07:16:13.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T07:16:13.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.945+0000 7fbda971a640 1 Processor -- start 2023-10-08T07:16:13.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.945+0000 7fbda971a640 1 -- start start 2023-10-08T07:16:13.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.946+0000 7fbda971a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda415a820 0x7fbda40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:13.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.946+0000 7fbda971a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbda40ca680 0x7fbda40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:13.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.946+0000 7fbda971a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda40cab60 con 0x7fbda415ac00 2023-10-08T07:16:13.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.946+0000 7fbda971a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda40cb960 con 0x7fbda40ca680 2023-10-08T07:16:13.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.946+0000 7fbda971a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40cbdf0 con 0x7fbda415a820 2023-10-08T07:16:13.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.947+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda415a820 0x7fbda40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:13.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.947+0000 7fbd9bfff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbda40ca680 0x7fbda40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:13.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.947+0000 7fbda3fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda415a820 0x7fbda40aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:33312/0 (socket says 172.21.15.70:33312) 2023-10-08T07:16:13.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.947+0000 7fbda3fff640 1 -- 172.21.15.70:0/2651500893 learned_addr learned my addr 172.21.15.70:0/2651500893 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:13.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.948+0000 7fbda3fff640 1 -- 172.21.15.70:0/2651500893 >> v2:172.21.15.181:3300/0 conn(0x7fbda40ca680 msgr2=0x7fbda40b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:13.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.948+0000 7fbda3fff640 1 --2- 172.21.15.70:0/2651500893 >> v2:172.21.15.181:3300/0 conn(0x7fbda40ca680 0x7fbda40b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.948+0000 7fbda3fff640 1 -- 172.21.15.70:0/2651500893 >> v1:172.21.15.181:6789/0 conn(0x7fbda415ac00 legacy=0x7fbda40c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:13.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.948+0000 7fbda3fff640 1 -- 172.21.15.70:0/2651500893 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda40cb030 con 0x7fbda415a820 2023-10-08T07:16:13.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.949+0000 7fbda3fff640 1 --2- 172.21.15.70:0/2651500893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda415a820 0x7fbda40aff70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009870 tx=0x7fbd9402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9fce36c2ba2c13 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:13.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.949+0000 7fbda37fe640 1 -- 172.21.15.70:0/2651500893 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94031640 con 0x7fbda415a820 2023-10-08T07:16:13.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.949+0000 7fbda37fe640 1 -- 172.21.15.70:0/2651500893 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd940317a0 con 0x7fbda415a820 2023-10-08T07:16:13.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.949+0000 7fbda37fe640 1 -- 172.21.15.70:0/2651500893 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94031b30 con 0x7fbda415a820 2023-10-08T07:16:13.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.950+0000 7fbda971a640 1 -- 172.21.15.70:0/2651500893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda415a820 msgr2=0x7fbda40aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:13.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.950+0000 7fbda971a640 1 --2- 172.21.15.70:0/2651500893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda415a820 0x7fbda40aff70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009870 tx=0x7fbd9402b8f0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.950+0000 7fbda971a640 1 -- 172.21.15.70:0/2651500893 shutdown_connections 2023-10-08T07:16:13.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.950+0000 7fbda971a640 1 --2- 172.21.15.70:0/2651500893 >> v2:172.21.15.181:3300/0 conn(0x7fbda40ca680 0x7fbda40b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.950+0000 7fbda971a640 1 --2- 172.21.15.70:0/2651500893 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda415a820 0x7fbda40aff70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.950+0000 7fbda971a640 1 -- 172.21.15.70:0/2651500893 >> 172.21.15.70:0/2651500893 conn(0x7fbda406c050 msgr2=0x7fbda406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:13.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.950+0000 7fbda971a640 1 -- 172.21.15.70:0/2651500893 shutdown_connections 2023-10-08T07:16:13.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.950+0000 7fbda971a640 1 -- 172.21.15.70:0/2651500893 wait complete. 2023-10-08T07:16:13.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.952+0000 7fbda971a640 1 Processor -- start 2023-10-08T07:16:13.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.952+0000 7fbda971a640 1 -- start start 2023-10-08T07:16:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.953+0000 7fbda971a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ca680 0x7fbda40c2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.953+0000 7fbda971a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda415a820 0x7fbda4153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.953+0000 7fbda971a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbda415ac00 0x7fbda40b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.953+0000 7fbda971a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40c61b0 con 0x7fbda415a820 2023-10-08T07:16:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.953+0000 7fbda971a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40c55c0 con 0x7fbda415ac00 2023-10-08T07:16:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.953+0000 7fbda971a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40c65d0 con 0x7fbda40ca680 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.954+0000 7fbd9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda415a820 0x7fbda4153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.954+0000 7fbd9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda415a820 0x7fbda4153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38690/0 (socket says 172.21.15.70:38690) 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.954+0000 7fbd9bfff640 1 -- 172.21.15.70:0/2146689307 learned_addr learned my addr 172.21.15.70:0/2146689307 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.954+0000 7fbd9bfff640 1 -- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ca680 msgr2=0x7fbda40c2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.954+0000 7fbda3fff640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ca680 0x7fbda40c2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.954+0000 7fbd9bfff640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ca680 0x7fbda40c2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.954+0000 7fbd9bfff640 1 -- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbda415ac00 msgr2=0x7fbda40b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.954+0000 7fbd9bfff640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbda415ac00 0x7fbda40b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.954+0000 7fbd9bfff640 1 -- 172.21.15.70:0/2146689307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd940094d0 con 0x7fbda415a820 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.954+0000 7fbd9bfff640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda415a820 0x7fbda4153d00 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00e870 tx=0x7fbd8c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.955+0000 7fbda17fa640 1 -- 172.21.15.70:0/2146689307 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0094f0 con 0x7fbda415a820 2023-10-08T07:16:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.955+0000 7fbda17fa640 1 -- 172.21.15.70:0/2146689307 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c00c940 con 0x7fbda415a820 2023-10-08T07:16:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.955+0000 7fbda17fa640 1 -- 172.21.15.70:0/2146689307 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c00ccf0 con 0x7fbda415a820 2023-10-08T07:16:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.955+0000 7fbda971a640 1 -- 172.21.15.70:0/2146689307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda40c5810 con 0x7fbda415a820 2023-10-08T07:16:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.955+0000 7fbda971a640 1 -- 172.21.15.70:0/2146689307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda40c5dd0 con 0x7fbda415a820 2023-10-08T07:16:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.956+0000 7fbda971a640 1 -- 172.21.15.70:0/2146689307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda40c62f0 con 0x7fbda415a820 2023-10-08T07:16:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.957+0000 7fbda17fa640 1 -- 172.21.15.70:0/2146689307 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbd8c013470 con 0x7fbda415a820 2023-10-08T07:16:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.958+0000 7fbda17fa640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbd8406af90 0x7fbd8406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.958+0000 7fbda17fa640 1 -- 172.21.15.70:0/2146689307 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(279..279 src has 1..279) v4 ==== 7091+0+0 (secure 0 0 0) 0x7fbd8c089d90 con 0x7fbda415a820 2023-10-08T07:16:13.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.959+0000 7fbda3fff640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbd8406af90 0x7fbd8406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:13.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.960+0000 7fbda3fff640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbd8406af90 0x7fbd8406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbd94036000 tx=0x7fbd9403c040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:13.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:13.962+0000 7fbda17fa640 1 -- 172.21.15.70:0/2146689307 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbda40c62f0 con 0x7fbda415a820 2023-10-08T07:16:14.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.148+0000 7fbda971a640 1 -- 172.21.15.70:0/2146689307 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbda40c62f0 con 0x7fbda415a820 2023-10-08T07:16:14.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.926+0000 7fbda17fa640 1 -- 172.21.15.70:0/2146689307 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v280) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbda40c62f0 con 0x7fbda415a820 2023-10-08T07:16:14.927 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-10-08T07:16:14.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.931+0000 7fbda971a640 1 -- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbd8406af90 msgr2=0x7fbd8406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:14.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.931+0000 7fbda971a640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbd8406af90 0x7fbd8406d450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbd94036000 tx=0x7fbd9403c040 comp rx=0 tx=0).stop 2023-10-08T07:16:14.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.931+0000 7fbda971a640 1 -- 172.21.15.70:0/2146689307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda415a820 msgr2=0x7fbda4153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:14.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.931+0000 7fbda971a640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda415a820 0x7fbda4153d00 secure :-1 s=READY pgs=2988 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00e870 tx=0x7fbd8c00ed30 comp rx=0 tx=0).stop 2023-10-08T07:16:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.932+0000 7fbda971a640 1 -- 172.21.15.70:0/2146689307 shutdown_connections 2023-10-08T07:16:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.932+0000 7fbda971a640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbd8406af90 0x7fbd8406d450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.932+0000 7fbda971a640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbda415ac00 0x7fbda40b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.932+0000 7fbda971a640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbda415a820 0x7fbda4153d00 unknown :-1 s=CLOSED pgs=2988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:14.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.932+0000 7fbda971a640 1 --2- 172.21.15.70:0/2146689307 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbda40ca680 0x7fbda40c2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:14.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.932+0000 7fbda971a640 1 -- 172.21.15.70:0/2146689307 >> 172.21.15.70:0/2146689307 conn(0x7fbda406c050 msgr2=0x7fbda40b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:14.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.932+0000 7fbda971a640 1 -- 172.21.15.70:0/2146689307 shutdown_connections 2023-10-08T07:16:14.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:14.932+0000 7fbda971a640 1 -- 172.21.15.70:0/2146689307 wait complete. 2023-10-08T07:16:14.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T07:16:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.171+0000 7fe77ff33640 1 Processor -- start 2023-10-08T07:16:15.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.171+0000 7fe77ff33640 1 -- start start 2023-10-08T07:16:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.173+0000 7fe77ff33640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe77806c3a0 0x7fe7780c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.173+0000 7fe77ff33640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe77806db60 0x7fe77806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.173+0000 7fe77ff33640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77815b310 con 0x7fe77815aa00 2023-10-08T07:16:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.173+0000 7fe77ff33640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780ca230 con 0x7fe77806c3a0 2023-10-08T07:16:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.173+0000 7fe77ff33640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77806d290 con 0x7fe77806db60 2023-10-08T07:16:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.173+0000 7fe77dccf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe77806c3a0 0x7fe7780c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.173+0000 7fe77dccf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe77806c3a0 0x7fe7780c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38698/0 (socket says 172.21.15.70:38698) 2023-10-08T07:16:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.173+0000 7fe77e4d0640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe77815aa00 0x7fe7780b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51598/0 (socket says 172.21.15.70:51598) 2023-10-08T07:16:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.173+0000 7fe77dccf640 1 -- 172.21.15.70:0/2507346869 learned_addr learned my addr 172.21.15.70:0/2507346869 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.174+0000 7fe77d4ce640 1 --2- 172.21.15.70:0/2507346869 >> v2:172.21.15.181:3300/0 conn(0x7fe77806db60 0x7fe77806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.174+0000 7fe77dccf640 1 -- 172.21.15.70:0/2507346869 >> v2:172.21.15.181:3300/0 conn(0x7fe77806db60 msgr2=0x7fe77806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.174+0000 7fe77dccf640 1 --2- 172.21.15.70:0/2507346869 >> v2:172.21.15.181:3300/0 conn(0x7fe77806db60 0x7fe77806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.174+0000 7fe77dccf640 1 -- 172.21.15.70:0/2507346869 >> v1:172.21.15.70:6789/0 conn(0x7fe77815aa00 legacy=0x7fe7780b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.174+0000 7fe77dccf640 1 -- 172.21.15.70:0/2507346869 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7780c9860 con 0x7fe77806c3a0 2023-10-08T07:16:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.174+0000 7fe77cccd640 1 -- 172.21.15.70:0/2507346869 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662441852 0 0) 0x7fe770001490 con 0x7fe77815aa00 2023-10-08T07:16:15.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.174+0000 7fe77d4ce640 1 --2- 172.21.15.70:0/2507346869 >> v2:172.21.15.181:3300/0 conn(0x7fe77806db60 0x7fe77806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:15.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.175+0000 7fe77dccf640 1 --2- 172.21.15.70:0/2507346869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 0x7fe7780c9cf0 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7fe7780715c0 tx=0x7fe77402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50870626df2f1579 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:15.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.175+0000 7fe77cccd640 1 -- 172.21.15.70:0/2507346869 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe774030870 con 0x7fe77806c3a0 2023-10-08T07:16:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.175+0000 7fe77cccd640 1 -- 172.21.15.70:0/2507346869 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7740309d0 con 0x7fe77806c3a0 2023-10-08T07:16:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.175+0000 7fe77cccd640 1 -- 172.21.15.70:0/2507346869 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe774030d60 con 0x7fe77806c3a0 2023-10-08T07:16:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.176+0000 7fe77ff33640 1 -- 172.21.15.70:0/2507346869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 msgr2=0x7fe7780c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.176+0000 7fe77ff33640 1 --2- 172.21.15.70:0/2507346869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 0x7fe7780c9cf0 secure :-1 s=READY pgs=2990 cs=0 l=1 rev1=1 crypto rx=0x7fe7780715c0 tx=0x7fe77402b9b0 comp rx=0 tx=0).stop 2023-10-08T07:16:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.176+0000 7fe77ff33640 1 -- 172.21.15.70:0/2507346869 shutdown_connections 2023-10-08T07:16:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.176+0000 7fe77ff33640 1 --2- 172.21.15.70:0/2507346869 >> v2:172.21.15.181:3300/0 conn(0x7fe77806db60 0x7fe77806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.176+0000 7fe77ff33640 1 --2- 172.21.15.70:0/2507346869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 0x7fe7780c9cf0 unknown :-1 s=CLOSED pgs=2990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.176+0000 7fe77ff33640 1 -- 172.21.15.70:0/2507346869 >> 172.21.15.70:0/2507346869 conn(0x7fe77806a5d0 msgr2=0x7fe7780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.176+0000 7fe77ff33640 1 -- 172.21.15.70:0/2507346869 shutdown_connections 2023-10-08T07:16:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.176+0000 7fe77ff33640 1 -- 172.21.15.70:0/2507346869 wait complete. 2023-10-08T07:16:15.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.178+0000 7fe77ff33640 1 Processor -- start 2023-10-08T07:16:15.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.178+0000 7fe77ff33640 1 -- start start 2023-10-08T07:16:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.179+0000 7fe77ff33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 0x7fe7780cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.179+0000 7fe77ff33640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe77806db60 0x7fe7781630b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.179+0000 7fe77ff33640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe77815aa00 0x7fe778164a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.179+0000 7fe77ff33640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780cd590 con 0x7fe77806c3a0 2023-10-08T07:16:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.179+0000 7fe77ff33640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780c8270 con 0x7fe77815aa00 2023-10-08T07:16:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.179+0000 7fe77ff33640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780c8550 con 0x7fe77806db60 2023-10-08T07:16:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.179+0000 7fe77dccf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 0x7fe7780cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:15.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.179+0000 7fe77dccf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 0x7fe7780cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38714/0 (socket says 172.21.15.70:38714) 2023-10-08T07:16:15.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.179+0000 7fe77dccf640 1 -- 172.21.15.70:0/1012265680 learned_addr learned my addr 172.21.15.70:0/1012265680 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.180+0000 7fe77dccf640 1 -- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe77806db60 msgr2=0x7fe7781630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.180+0000 7fe77dccf640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe77806db60 0x7fe7781630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.180+0000 7fe77dccf640 1 -- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe77815aa00 msgr2=0x7fe778164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.180+0000 7fe77dccf640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe77815aa00 0x7fe778164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.180+0000 7fe77dccf640 1 -- 172.21.15.70:0/1012265680 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7740094d0 con 0x7fe77806c3a0 2023-10-08T07:16:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.180+0000 7fe77dccf640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 0x7fe7780cd050 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7fe774000c00 tx=0x7fe774004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.180+0000 7fe76e7fc640 1 -- 172.21.15.70:0/1012265680 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe774030840 con 0x7fe77806c3a0 2023-10-08T07:16:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.180+0000 7fe76e7fc640 1 -- 172.21.15.70:0/1012265680 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe774043400 con 0x7fe77806c3a0 2023-10-08T07:16:15.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.181+0000 7fe77ff33640 1 -- 172.21.15.70:0/1012265680 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7781501d0 con 0x7fe77806c3a0 2023-10-08T07:16:15.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.181+0000 7fe76e7fc640 1 -- 172.21.15.70:0/1012265680 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7740437b0 con 0x7fe77806c3a0 2023-10-08T07:16:15.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.181+0000 7fe77ff33640 1 -- 172.21.15.70:0/1012265680 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe778150790 con 0x7fe77806c3a0 2023-10-08T07:16:15.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.182+0000 7fe77ff33640 1 -- 172.21.15.70:0/1012265680 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77804e160 con 0x7fe77806c3a0 2023-10-08T07:16:15.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.183+0000 7fe76e7fc640 1 -- 172.21.15.70:0/1012265680 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe774043910 con 0x7fe77806c3a0 2023-10-08T07:16:15.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.183+0000 7fe76e7fc640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe75c06af90 0x7fe75c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:15.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.184+0000 7fe76e7fc640 1 -- 172.21.15.70:0/1012265680 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(280..280 src has 1..280) v4 ==== 6653+0+0 (secure 0 0 0) 0x7fe77403e070 con 0x7fe77806c3a0 2023-10-08T07:16:15.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.184+0000 7fe77d4ce640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe75c06af90 0x7fe75c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:15.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.185+0000 7fe77d4ce640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe75c06af90 0x7fe75c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe768006fd0 tx=0x7fe768008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:15.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.188+0000 7fe76e7fc640 1 -- 172.21.15.70:0/1012265680 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe774077c90 con 0x7fe77806c3a0 2023-10-08T07:16:15.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:15.378+0000 7fe77ff33640 1 -- 172.21.15.70:0/1012265680 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe778071180 con 0x7fe77806c3a0 2023-10-08T07:16:16.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.347+0000 7fe76e7fc640 1 -- 172.21.15.70:0/1012265680 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v281) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe77407b2b0 con 0x7fe77806c3a0 2023-10-08T07:16:16.348 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-10-08T07:16:16.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.353+0000 7fe77ff33640 1 -- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe75c06af90 msgr2=0x7fe75c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:16.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.353+0000 7fe77ff33640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe75c06af90 0x7fe75c06d450 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe768006fd0 tx=0x7fe768008040 comp rx=0 tx=0).stop 2023-10-08T07:16:16.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.354+0000 7fe77ff33640 1 -- 172.21.15.70:0/1012265680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 msgr2=0x7fe7780cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.354+0000 7fe77ff33640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 0x7fe7780cd050 secure :-1 s=READY pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7fe774000c00 tx=0x7fe774004730 comp rx=0 tx=0).stop 2023-10-08T07:16:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.354+0000 7fe77ff33640 1 -- 172.21.15.70:0/1012265680 shutdown_connections 2023-10-08T07:16:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.354+0000 7fe77ff33640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe75c06af90 0x7fe75c06d450 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.354+0000 7fe77ff33640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe77815aa00 0x7fe778164a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.354+0000 7fe77ff33640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe77806db60 0x7fe7781630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.354+0000 7fe77ff33640 1 --2- 172.21.15.70:0/1012265680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77806c3a0 0x7fe7780cd050 secure :-1 s=CLOSED pgs=2991 cs=0 l=1 rev1=1 crypto rx=0x7fe774000c00 tx=0x7fe774004730 comp rx=0 tx=0).stop 2023-10-08T07:16:16.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.354+0000 7fe77ff33640 1 -- 172.21.15.70:0/1012265680 >> 172.21.15.70:0/1012265680 conn(0x7fe77806a5d0 msgr2=0x7fe7780cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:16.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.355+0000 7fe77ff33640 1 -- 172.21.15.70:0/1012265680 shutdown_connections 2023-10-08T07:16:16.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.355+0000 7fe77ff33640 1 -- 172.21.15.70:0/1012265680 wait complete. 2023-10-08T07:16:16.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T07:16:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.600+0000 7f2c6d21a640 1 Processor -- start 2023-10-08T07:16:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.600+0000 7f2c6d21a640 1 -- start start 2023-10-08T07:16:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.602+0000 7f2c6d21a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680cb620 0x7f2c680b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.602+0000 7f2c6d21a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c68071170 con 0x7f2c68070cc0 2023-10-08T07:16:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.602+0000 7f2c6d21a640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6806e150 con 0x7f2c68160660 2023-10-08T07:16:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.602+0000 7f2c6d21a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6806de70 con 0x7f2c680cb620 2023-10-08T07:16:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.602+0000 7f2c6659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2c68070cc0 0x7f2c680caf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51610/0 (socket says 172.21.15.70:51610) 2023-10-08T07:16:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.602+0000 7f2c6659c640 1 -- 172.21.15.70:0/2799416342 learned_addr learned my addr 172.21.15.70:0/2799416342 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c6759e640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680cb620 0x7f2c680b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487360445 0 0) 0x7f2c5c001110 con 0x7f2c68070cc0 2023-10-08T07:16:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c50003520 con 0x7f2c68070cc0 2023-10-08T07:16:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 668439370 0 0) 0x7f2c50003520 con 0x7f2c68070cc0 2023-10-08T07:16:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680cb620 msgr2=0x7f2c680b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680cb620 0x7f2c680b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 >> v1:172.21.15.181:6789/0 conn(0x7f2c68160660 legacy=0x7f2c680c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680c8e30 con 0x7f2c68070cc0 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f2c5c001800 con 0x7f2c68070cc0 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 >> v1:172.21.15.70:6789/0 conn(0x7f2c68070cc0 legacy=0x7f2c680caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c500039a0 0x7f2c50003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680cb620 0x7f2c500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c50004d00 0x7f2c50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6806de70 con 0x7f2c50004d00 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5000ba30 con 0x7f2c500039a0 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.603+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5000bbf0 con 0x7f2c680cb620 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.604+0000 7f2c6759e640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c50004d00 0x7f2c50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.604+0000 7f2c66d9d640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c500039a0 0x7f2c50003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.604+0000 7f2c6659c640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680cb620 0x7f2c500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.604+0000 7f2c6759e640 1 -- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680cb620 msgr2=0x7f2c500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.604+0000 7f2c6759e640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680cb620 0x7f2c500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.604+0000 7f2c6759e640 1 -- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c500039a0 msgr2=0x7f2c50003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.604+0000 7f2c6759e640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c500039a0 0x7f2c50003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.604+0000 7f2c6759e640 1 -- 172.21.15.70:0/2799416342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50003520 con 0x7f2c50004d00 2023-10-08T07:16:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.604+0000 7f2c66d9d640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c500039a0 0x7f2c50003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T07:16:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.605+0000 7f2c6759e640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c50004d00 0x7f2c50005180 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009f90 tx=0x7f2c500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68202be28ffc380 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.606+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6002cbf0 con 0x7f2c50004d00 2023-10-08T07:16:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.606+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6000f040 con 0x7f2c50004d00 2023-10-08T07:16:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.606+0000 7f2c65d9b640 1 -- 172.21.15.70:0/2799416342 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c60034440 con 0x7f2c50004d00 2023-10-08T07:16:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.606+0000 7f2c6d21a640 1 -- 172.21.15.70:0/2799416342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c50004d00 msgr2=0x7f2c50005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.606+0000 7f2c6d21a640 1 --2- 172.21.15.70:0/2799416342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c50004d00 0x7f2c50005180 secure :-1 s=READY pgs=2993 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009f90 tx=0x7f2c500057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.606+0000 7f2c6759e640 1 -- 172.21.15.70:0/2799416342 reap_dead start 2023-10-08T07:16:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.606+0000 7f2c6d21a640 1 -- 172.21.15.70:0/2799416342 shutdown_connections 2023-10-08T07:16:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.606+0000 7f2c6d21a640 1 -- 172.21.15.70:0/2799416342 >> 172.21.15.70:0/2799416342 conn(0x7f2c6806a5d0 msgr2=0x7f2c680b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.606+0000 7f2c6d21a640 1 -- 172.21.15.70:0/2799416342 shutdown_connections 2023-10-08T07:16:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.606+0000 7f2c6d21a640 1 -- 172.21.15.70:0/2799416342 wait complete. 2023-10-08T07:16:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.608+0000 7f2c6d21a640 1 Processor -- start 2023-10-08T07:16:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.608+0000 7f2c6d21a640 1 -- start start 2023-10-08T07:16:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.609+0000 7f2c6d21a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68155cf0 0x7f2c68154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.609+0000 7f2c6d21a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c681560d0 0x7f2c68156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.609+0000 7f2c6d21a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680c5bb0 0x7f2c680c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.609+0000 7f2c6d21a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680c3c80 con 0x7f2c681560d0 2023-10-08T07:16:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.609+0000 7f2c6d21a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680c7cc0 con 0x7f2c68155cf0 2023-10-08T07:16:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.609+0000 7f2c6d21a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c680c3ec0 con 0x7f2c680c5bb0 2023-10-08T07:16:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c6659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c681560d0 0x7f2c68156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:16.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c6659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c681560d0 0x7f2c68156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38736/0 (socket says 172.21.15.70:38736) 2023-10-08T07:16:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c6659c640 1 -- 172.21.15.70:0/1596209104 learned_addr learned my addr 172.21.15.70:0/1596209104 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c66d9d640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68155cf0 0x7f2c68154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c6759e640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680c5bb0 0x7f2c680c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c6659c640 1 -- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680c5bb0 msgr2=0x7f2c680c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c6659c640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680c5bb0 0x7f2c680c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c6659c640 1 -- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68155cf0 msgr2=0x7f2c68154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c6659c640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68155cf0 0x7f2c68154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c6659c640 1 -- 172.21.15.70:0/1596209104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c680c41a0 con 0x7f2c681560d0 2023-10-08T07:16:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.610+0000 7f2c6659c640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c681560d0 0x7f2c68156550 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c001390 tx=0x7f2c5c00ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.611+0000 7f2c4f7fe640 1 -- 172.21.15.70:0/1596209104 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00c8d0 con 0x7f2c681560d0 2023-10-08T07:16:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.611+0000 7f2c4f7fe640 1 -- 172.21.15.70:0/1596209104 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5c00ca30 con 0x7f2c681560d0 2023-10-08T07:16:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.611+0000 7f2c4f7fe640 1 -- 172.21.15.70:0/1596209104 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00cde0 con 0x7f2c681560d0 2023-10-08T07:16:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.611+0000 7f2c6d21a640 1 -- 172.21.15.70:0/1596209104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c680c4480 con 0x7f2c681560d0 2023-10-08T07:16:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.611+0000 7f2c6d21a640 1 -- 172.21.15.70:0/1596209104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c680c4a40 con 0x7f2c681560d0 2023-10-08T07:16:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.612+0000 7f2c6d21a640 1 -- 172.21.15.70:0/1596209104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c680c4bd0 con 0x7f2c681560d0 2023-10-08T07:16:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.613+0000 7f2c4f7fe640 1 -- 172.21.15.70:0/1596209104 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2c5c01d460 con 0x7f2c681560d0 2023-10-08T07:16:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.614+0000 7f2c4f7fe640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2c5406afc0 0x7f2c5406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.615+0000 7f2c4f7fe640 1 -- 172.21.15.70:0/1596209104 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f2c5c089850 con 0x7f2c681560d0 2023-10-08T07:16:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.615+0000 7f2c66d9d640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2c5406afc0 0x7f2c5406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:16.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.616+0000 7f2c66d9d640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2c5406afc0 0x7f2c5406d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2c58005550 tx=0x7f2c58009040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:16.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.619+0000 7f2c4f7fe640 1 -- 172.21.15.70:0/1596209104 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c5c0545e0 con 0x7f2c681560d0 2023-10-08T07:16:16.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.807+0000 7f2c6d21a640 1 -- 172.21.15.70:0/1596209104 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2c68071170 con 0x7f2c681560d0 2023-10-08T07:16:16.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.814+0000 7f2c4f7fe640 1 -- 172.21.15.70:0/1596209104 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v91) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2c68071170 con 0x7f2c681560d0 2023-10-08T07:16:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 -- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2c5406afc0 msgr2=0x7f2c5406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2c5406afc0 0x7f2c5406d480 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2c58005550 tx=0x7f2c58009040 comp rx=0 tx=0).stop 2023-10-08T07:16:16.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 -- 172.21.15.70:0/1596209104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c681560d0 msgr2=0x7f2c68156550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c681560d0 0x7f2c68156550 secure :-1 s=READY pgs=2994 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c001390 tx=0x7f2c5c00ef80 comp rx=0 tx=0).stop 2023-10-08T07:16:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 -- 172.21.15.70:0/1596209104 shutdown_connections 2023-10-08T07:16:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2c5406afc0 0x7f2c5406d480 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f2c680c5bb0 0x7f2c680c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c681560d0 0x7f2c68156550 unknown :-1 s=CLOSED pgs=2994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 --2- 172.21.15.70:0/1596209104 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2c68155cf0 0x7f2c68154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 -- 172.21.15.70:0/1596209104 >> 172.21.15.70:0/1596209104 conn(0x7f2c6806a5d0 msgr2=0x7f2c680c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.817+0000 7f2c6d21a640 1 -- 172.21.15.70:0/1596209104 shutdown_connections 2023-10-08T07:16:16.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:16.818+0000 7f2c6d21a640 1 -- 172.21.15.70:0/1596209104 wait complete. 2023-10-08T07:16:16.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T07:16:17.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.047+0000 7f5dcd44f640 1 Processor -- start 2023-10-08T07:16:17.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.047+0000 7f5dcd44f640 1 -- start start 2023-10-08T07:16:17.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.049+0000 7f5dcd44f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5dc815aad0 0x7f5dc80b1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.049+0000 7f5dcd44f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc80c82b0 0x7f5dc80c8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.049+0000 7f5dcd44f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc815bf30 con 0x7f5dc806cd00 2023-10-08T07:16:17.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.049+0000 7f5dcd44f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc815b320 con 0x7f5dc815aad0 2023-10-08T07:16:17.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.049+0000 7f5dcd44f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc815b600 con 0x7f5dc80c82b0 2023-10-08T07:16:17.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.049+0000 7f5dc6ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5dc815aad0 0x7f5dc80b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.049+0000 7f5dc77fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5dc806cd00 0x7f5dc8070570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51618/0 (socket says 172.21.15.70:51618) 2023-10-08T07:16:17.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.049+0000 7f5dc77fe640 1 -- 172.21.15.70:0/1276136449 learned_addr learned my addr 172.21.15.70:0/1276136449 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:17.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.050+0000 7f5dc67fc640 1 --2- 172.21.15.70:0/1276136449 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc80c82b0 0x7f5dc80c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.050+0000 7f5dc6ffd640 1 -- 172.21.15.70:0/1276136449 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc80c82b0 msgr2=0x7f5dc80c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.050+0000 7f5dc6ffd640 1 --2- 172.21.15.70:0/1276136449 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc80c82b0 0x7f5dc80c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.050+0000 7f5dc6ffd640 1 -- 172.21.15.70:0/1276136449 >> v1:172.21.15.70:6789/0 conn(0x7f5dc806cd00 legacy=0x7f5dc8070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.050+0000 7f5dc6ffd640 1 -- 172.21.15.70:0/1276136449 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc815b8e0 con 0x7f5dc815aad0 2023-10-08T07:16:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.050+0000 7f5dc5ffb640 1 -- 172.21.15.70:0/1276136449 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041566738 0 0) 0x7f5dc815bf30 con 0x7f5dc806cd00 2023-10-08T07:16:17.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.051+0000 7f5dc6ffd640 1 --2- 172.21.15.70:0/1276136449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 0x7f5dc80b1750 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009d90 tx=0x7f5db802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135b10f632011e9d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.051+0000 7f5dc5ffb640 1 -- 172.21.15.70:0/1276136449 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db802cc50 con 0x7f5dc815aad0 2023-10-08T07:16:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.051+0000 7f5dc5ffb640 1 -- 172.21.15.70:0/1276136449 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db802cdb0 con 0x7f5dc815aad0 2023-10-08T07:16:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.051+0000 7f5dc5ffb640 1 -- 172.21.15.70:0/1276136449 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8034660 con 0x7f5dc815aad0 2023-10-08T07:16:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.052+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1276136449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 msgr2=0x7f5dc80b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.052+0000 7f5dcd44f640 1 --2- 172.21.15.70:0/1276136449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 0x7f5dc80b1750 secure :-1 s=READY pgs=2996 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009d90 tx=0x7f5db802bea0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.052+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1276136449 shutdown_connections 2023-10-08T07:16:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.052+0000 7f5dcd44f640 1 --2- 172.21.15.70:0/1276136449 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc80c82b0 0x7f5dc80c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.052+0000 7f5dcd44f640 1 --2- 172.21.15.70:0/1276136449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 0x7f5dc80b1750 unknown :-1 s=CLOSED pgs=2996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.052+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1276136449 >> 172.21.15.70:0/1276136449 conn(0x7f5dc8069d50 msgr2=0x7f5dc806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.052+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1276136449 shutdown_connections 2023-10-08T07:16:17.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.052+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1276136449 wait complete. 2023-10-08T07:16:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.054+0000 7f5dcd44f640 1 Processor -- start 2023-10-08T07:16:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.054+0000 7f5dcd44f640 1 -- start start 2023-10-08T07:16:17.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.055+0000 7f5dcd44f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc806cd00 0x7f5dc80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.055+0000 7f5dcd44f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc80c82b0 0x7f5dc80707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.055+0000 7f5dcd44f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 0x7f5dc8159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.055+0000 7f5dcd44f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc81566b0 con 0x7f5dc815aad0 2023-10-08T07:16:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.055+0000 7f5dcd44f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8158180 con 0x7f5dc806cd00 2023-10-08T07:16:17.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.055+0000 7f5dcd44f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8158460 con 0x7f5dc80c82b0 2023-10-08T07:16:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.055+0000 7f5dc77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 0x7f5dc8159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.055+0000 7f5dc77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 0x7f5dc8159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38744/0 (socket says 172.21.15.70:38744) 2023-10-08T07:16:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.055+0000 7f5dc77fe640 1 -- 172.21.15.70:0/1925012658 learned_addr learned my addr 172.21.15.70:0/1925012658 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dc77fe640 1 -- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc80c82b0 msgr2=0x7f5dc80707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dc6ffd640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc806cd00 0x7f5dc80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dc67fc640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc80c82b0 0x7f5dc80707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dc77fe640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc80c82b0 0x7f5dc80707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dc77fe640 1 -- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc806cd00 msgr2=0x7f5dc80b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dc77fe640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc806cd00 0x7f5dc80b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dc77fe640 1 -- 172.21.15.70:0/1925012658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8009a70 con 0x7f5dc815aad0 2023-10-08T07:16:17.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dc77fe640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 0x7f5dc8159220 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c450 tx=0x7f5dc000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:17.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dbf7fe640 1 -- 172.21.15.70:0/1925012658 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00074e0 con 0x7f5dc815aad0 2023-10-08T07:16:17.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dbf7fe640 1 -- 172.21.15.70:0/1925012658 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc0007660 con 0x7f5dc815aad0 2023-10-08T07:16:17.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dbf7fe640 1 -- 172.21.15.70:0/1925012658 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0004a20 con 0x7f5dc815aad0 2023-10-08T07:16:17.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.056+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1925012658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc8158740 con 0x7f5dc815aad0 2023-10-08T07:16:17.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.057+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1925012658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc8158d00 con 0x7f5dc815aad0 2023-10-08T07:16:17.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.058+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1925012658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc8158f40 con 0x7f5dc815aad0 2023-10-08T07:16:17.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.059+0000 7f5dbf7fe640 1 -- 172.21.15.70:0/1925012658 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f5dc0004b80 con 0x7f5dc815aad0 2023-10-08T07:16:17.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.060+0000 7f5dbf7fe640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f5da806af90 0x7f5da806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.060+0000 7f5dbf7fe640 1 -- 172.21.15.70:0/1925012658 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f5dc0088c70 con 0x7f5dc815aad0 2023-10-08T07:16:17.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.060+0000 7f5dc6ffd640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f5da806af90 0x7f5da806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.062+0000 7f5dc6ffd640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f5da806af90 0x7f5da806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5db8036000 tx=0x7f5db8030000 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:17.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.064+0000 7f5dbf7fe640 1 -- 172.21.15.70:0/1925012658 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5dc8158f40 con 0x7f5dc815aad0 2023-10-08T07:16:17.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.254+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1925012658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f5dc8158f40 con 0x7f5dc815aad0 2023-10-08T07:16:17.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.261+0000 7f5dbf7fe640 1 -- 172.21.15.70:0/1925012658 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v92) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5dc8158f40 con 0x7f5dc815aad0 2023-10-08T07:16:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f5da806af90 msgr2=0x7f5da806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f5da806af90 0x7f5da806d450 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5db8036000 tx=0x7f5db8030000 comp rx=0 tx=0).stop 2023-10-08T07:16:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1925012658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 msgr2=0x7f5dc8159220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 0x7f5dc8159220 secure :-1 s=READY pgs=2997 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c450 tx=0x7f5dc000c910 comp rx=0 tx=0).stop 2023-10-08T07:16:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1925012658 shutdown_connections 2023-10-08T07:16:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f5da806af90 0x7f5da806d450 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dc815aad0 0x7f5dc8159220 unknown :-1 s=CLOSED pgs=2997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f5dc80c82b0 0x7f5dc80707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 --2- 172.21.15.70:0/1925012658 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f5dc806cd00 0x7f5dc80b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1925012658 >> 172.21.15.70:0/1925012658 conn(0x7f5dc8069d50 msgr2=0x7f5dc815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:17.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1925012658 shutdown_connections 2023-10-08T07:16:17.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.264+0000 7f5dcd44f640 1 -- 172.21.15.70:0/1925012658 wait complete. 2023-10-08T07:16:17.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T07:16:17.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.497+0000 7fbce5739640 1 Processor -- start 2023-10-08T07:16:17.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.497+0000 7fbce5739640 1 -- start start 2023-10-08T07:16:17.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.499+0000 7fbce5739640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fbce015cdb0 0x7fbce00cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.499+0000 7fbce5739640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbce00ccb40 0x7fbce00ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.499+0000 7fbce5739640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce015d620 con 0x7fbce015d190 2023-10-08T07:16:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.499+0000 7fbce5739640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce00cd490 con 0x7fbce00ccb40 2023-10-08T07:16:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.499+0000 7fbce5739640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce006b6a0 con 0x7fbce015cdb0 2023-10-08T07:16:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.499+0000 7fbcdf7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbce00ccb40 0x7fbce00ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.499+0000 7fbcdf7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbce00ccb40 0x7fbce00ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38756/0 (socket says 172.21.15.70:38756) 2023-10-08T07:16:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.499+0000 7fbcdf7fe640 1 -- 172.21.15.70:0/557477783 learned_addr learned my addr 172.21.15.70:0/557477783 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.500+0000 7fbcdeffd640 1 --2- 172.21.15.70:0/557477783 >> v2:172.21.15.181:3300/0 conn(0x7fbce015cdb0 0x7fbce00cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.500+0000 7fbcdf7fe640 1 -- 172.21.15.70:0/557477783 >> v2:172.21.15.181:3300/0 conn(0x7fbce015cdb0 msgr2=0x7fbce00cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:17.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.500+0000 7fbcdf7fe640 1 --2- 172.21.15.70:0/557477783 >> v2:172.21.15.181:3300/0 conn(0x7fbce015cdb0 0x7fbce00cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.500+0000 7fbcdf7fe640 1 -- 172.21.15.70:0/557477783 >> v1:172.21.15.70:6789/0 conn(0x7fbce015d190 legacy=0x7fbce006af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.500+0000 7fbcdf7fe640 1 -- 172.21.15.70:0/557477783 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00b19e0 con 0x7fbce00ccb40 2023-10-08T07:16:17.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.500+0000 7fbcdeffd640 1 --2- 172.21.15.70:0/557477783 >> v2:172.21.15.181:3300/0 conn(0x7fbce015cdb0 0x7fbce00cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:17.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.501+0000 7fbcdf7fe640 1 --2- 172.21.15.70:0/557477783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce00ccb40 0x7fbce00ccf50 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7fbce009d060 tx=0x7fbcd802b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbd076c13727416 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:17.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.501+0000 7fbcddffb640 1 -- 172.21.15.70:0/557477783 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8030870 con 0x7fbce00ccb40 2023-10-08T07:16:17.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.501+0000 7fbcddffb640 1 -- 172.21.15.70:0/557477783 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80309d0 con 0x7fbce00ccb40 2023-10-08T07:16:17.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.501+0000 7fbcddffb640 1 -- 172.21.15.70:0/557477783 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8030d60 con 0x7fbce00ccb40 2023-10-08T07:16:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.502+0000 7fbce5739640 1 -- 172.21.15.70:0/557477783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce00ccb40 msgr2=0x7fbce00ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.502+0000 7fbce5739640 1 --2- 172.21.15.70:0/557477783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce00ccb40 0x7fbce00ccf50 secure :-1 s=READY pgs=2999 cs=0 l=1 rev1=1 crypto rx=0x7fbce009d060 tx=0x7fbcd802b9b0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.502+0000 7fbce5739640 1 -- 172.21.15.70:0/557477783 shutdown_connections 2023-10-08T07:16:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.502+0000 7fbce5739640 1 --2- 172.21.15.70:0/557477783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce00ccb40 0x7fbce00ccf50 unknown :-1 s=CLOSED pgs=2999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.502+0000 7fbce5739640 1 --2- 172.21.15.70:0/557477783 >> v2:172.21.15.181:3300/0 conn(0x7fbce015cdb0 0x7fbce00cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.502+0000 7fbce5739640 1 -- 172.21.15.70:0/557477783 >> 172.21.15.70:0/557477783 conn(0x7fbce00b0af0 msgr2=0x7fbce00b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.502+0000 7fbce5739640 1 -- 172.21.15.70:0/557477783 shutdown_connections 2023-10-08T07:16:17.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.502+0000 7fbce5739640 1 -- 172.21.15.70:0/557477783 wait complete. 2023-10-08T07:16:17.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.504+0000 7fbce5739640 1 Processor -- start 2023-10-08T07:16:17.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.504+0000 7fbce5739640 1 -- start start 2023-10-08T07:16:17.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.505+0000 7fbce5739640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbce00ccb40 0x7fbce0156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.505+0000 7fbce5739640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbce015cdb0 0x7fbce00c3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.505+0000 7fbce5739640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce015d190 0x7fbce015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.505+0000 7fbce5739640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c4f10 con 0x7fbce015d190 2023-10-08T07:16:17.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.505+0000 7fbce5739640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c4300 con 0x7fbce015cdb0 2023-10-08T07:16:17.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.505+0000 7fbce5739640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c45e0 con 0x7fbce00ccb40 2023-10-08T07:16:17.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.505+0000 7fbcdf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce015d190 0x7fbce015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.505+0000 7fbcdf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce015d190 0x7fbce015b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38768/0 (socket says 172.21.15.70:38768) 2023-10-08T07:16:17.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.505+0000 7fbcdf7fe640 1 -- 172.21.15.70:0/948321634 learned_addr learned my addr 172.21.15.70:0/948321634 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:17.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcde7fc640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbce015cdb0 0x7fbce00c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcdeffd640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbce00ccb40 0x7fbce0156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcdf7fe640 1 -- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbce00ccb40 msgr2=0x7fbce0156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcdf7fe640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbce00ccb40 0x7fbce0156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcdf7fe640 1 -- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbce015cdb0 msgr2=0x7fbce00c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcdf7fe640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbce015cdb0 0x7fbce00c3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcdf7fe640 1 -- 172.21.15.70:0/948321634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80094d0 con 0x7fbce015d190 2023-10-08T07:16:17.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcde7fc640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbce015cdb0 0x7fbce00c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:17.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcdf7fe640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce015d190 0x7fbce015b900 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 crypto rx=0x7fbcd802b9f0 tx=0x7fbcd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:17.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcd77fe640 1 -- 172.21.15.70:0/948321634 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8030800 con 0x7fbce015d190 2023-10-08T07:16:17.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcd77fe640 1 -- 172.21.15.70:0/948321634 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd8039d90 con 0x7fbce015d190 2023-10-08T07:16:17.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbcd77fe640 1 -- 172.21.15.70:0/948321634 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8043650 con 0x7fbce015d190 2023-10-08T07:16:17.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbce5739640 1 -- 172.21.15.70:0/948321634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce00c48c0 con 0x7fbce015d190 2023-10-08T07:16:17.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.506+0000 7fbce5739640 1 -- 172.21.15.70:0/948321634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce00c39c0 con 0x7fbce015d190 2023-10-08T07:16:17.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.508+0000 7fbce5739640 1 -- 172.21.15.70:0/948321634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce00c5080 con 0x7fbce015d190 2023-10-08T07:16:17.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.508+0000 7fbcd77fe640 1 -- 172.21.15.70:0/948321634 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fbcd802f3c0 con 0x7fbce015d190 2023-10-08T07:16:17.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.509+0000 7fbcd77fe640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbcc006af90 0x7fbcc006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.509+0000 7fbcd77fe640 1 -- 172.21.15.70:0/948321634 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fbcd80ac230 con 0x7fbce015d190 2023-10-08T07:16:17.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.510+0000 7fbcdeffd640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbcc006af90 0x7fbcc006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.511+0000 7fbcdeffd640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbcc006af90 0x7fbcc006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbce0073460 tx=0x7fbcd00084f0 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:17.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.514+0000 7fbcd77fe640 1 -- 172.21.15.70:0/948321634 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbcd8077060 con 0x7fbce015d190 2023-10-08T07:16:17.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.716+0000 7fbce5739640 1 -- 172.21.15.70:0/948321634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fbce004e160 con 0x7fbce015d190 2023-10-08T07:16:17.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.721+0000 7fbcd77fe640 1 -- 172.21.15.70:0/948321634 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v93) v1 ==== 120+0+0 (secure 0 0 0) 0x7fbcd807a680 con 0x7fbce015d190 2023-10-08T07:16:17.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.723+0000 7fbce5739640 1 -- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbcc006af90 msgr2=0x7fbcc006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.723+0000 7fbce5739640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbcc006af90 0x7fbcc006d450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbce0073460 tx=0x7fbcd00084f0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.723+0000 7fbce5739640 1 -- 172.21.15.70:0/948321634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce015d190 msgr2=0x7fbce015b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.723+0000 7fbce5739640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce015d190 0x7fbce015b900 secure :-1 s=READY pgs=3000 cs=0 l=1 rev1=1 crypto rx=0x7fbcd802b9f0 tx=0x7fbcd8004730 comp rx=0 tx=0).stop 2023-10-08T07:16:17.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.724+0000 7fbce5739640 1 -- 172.21.15.70:0/948321634 shutdown_connections 2023-10-08T07:16:17.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.724+0000 7fbce5739640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fbcc006af90 0x7fbcc006d450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.724+0000 7fbce5739640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce015d190 0x7fbce015b900 unknown :-1 s=CLOSED pgs=3000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.724+0000 7fbce5739640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fbce015cdb0 0x7fbce00c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.724+0000 7fbce5739640 1 --2- 172.21.15.70:0/948321634 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fbce00ccb40 0x7fbce0156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.724+0000 7fbce5739640 1 -- 172.21.15.70:0/948321634 >> 172.21.15.70:0/948321634 conn(0x7fbce00b0af0 msgr2=0x7fbce006cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:17.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.724+0000 7fbce5739640 1 -- 172.21.15.70:0/948321634 shutdown_connections 2023-10-08T07:16:17.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.724+0000 7fbce5739640 1 -- 172.21.15.70:0/948321634 wait complete. 2023-10-08T07:16:17.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-08T07:16:17.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.945+0000 7f300ec4b640 1 Processor -- start 2023-10-08T07:16:17.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.945+0000 7f300ec4b640 1 -- start start 2023-10-08T07:16:17.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.947+0000 7f300ec4b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3008159ac0 0x7f300806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.947+0000 7f300ec4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 0x7f30080c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.947+0000 7f300ec4b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f300806dee0 con 0x7f30080ad790 2023-10-08T07:16:17.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.947+0000 7f300ec4b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300815d850 con 0x7f3008159ac0 2023-10-08T07:16:17.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.947+0000 7f300ec4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300815db30 con 0x7f30080709e0 2023-10-08T07:16:17.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.948+0000 7f300c9e7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3008159ac0 0x7f300806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.948+0000 7f300d1e8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 0x7f30080c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.948+0000 7f300d1e8640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 0x7f30080c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:33386/0 (socket says 172.21.15.70:33386) 2023-10-08T07:16:17.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.948+0000 7f300c9e7640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f3008159ac0 0x7f300806d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3300/0 says I am v2:172.21.15.70:41142/0 (socket says 172.21.15.70:41142) 2023-10-08T07:16:17.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.948+0000 7f300d1e8640 1 -- 172.21.15.70:0/2371093241 learned_addr learned my addr 172.21.15.70:0/2371093241 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:17.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.949+0000 7f300c9e7640 1 -- 172.21.15.70:0/2371093241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 msgr2=0x7f30080c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:17.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.949+0000 7f300c9e7640 1 --2- 172.21.15.70:0/2371093241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 0x7f30080c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.949+0000 7f300c9e7640 1 -- 172.21.15.70:0/2371093241 >> v1:172.21.15.181:6789/0 conn(0x7f30080ad790 legacy=0x7f30080702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.949+0000 7f300c9e7640 1 -- 172.21.15.70:0/2371093241 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300815de10 con 0x7f3008159ac0 2023-10-08T07:16:17.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.949+0000 7f30077fe640 1 -- 172.21.15.70:0/2371093241 <== mon.1 v1:172.21.15.181:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766207029 0 0) 0x7f300806dee0 con 0x7f30080ad790 2023-10-08T07:16:17.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.949+0000 7f300d1e8640 1 --2- 172.21.15.70:0/2371093241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 0x7f30080c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:16:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.950+0000 7f300c9e7640 1 --2- 172.21.15.70:0/2371093241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3008159ac0 0x7f300806d5e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009d90 tx=0x7f2ff802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50887ace9e4c8465 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:17.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.951+0000 7f30077fe640 1 -- 172.21.15.70:0/2371093241 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff802cc60 con 0x7f3008159ac0 2023-10-08T07:16:17.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.951+0000 7f30077fe640 1 -- 172.21.15.70:0/2371093241 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff802cdc0 con 0x7f3008159ac0 2023-10-08T07:16:17.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.951+0000 7f30077fe640 1 -- 172.21.15.70:0/2371093241 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8034660 con 0x7f3008159ac0 2023-10-08T07:16:17.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.952+0000 7f300ec4b640 1 -- 172.21.15.70:0/2371093241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3008159ac0 msgr2=0x7f300806d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.952+0000 7f300ec4b640 1 --2- 172.21.15.70:0/2371093241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3008159ac0 0x7f300806d5e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009d90 tx=0x7f2ff802beb0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.952+0000 7f300ec4b640 1 -- 172.21.15.70:0/2371093241 shutdown_connections 2023-10-08T07:16:17.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.952+0000 7f300ec4b640 1 --2- 172.21.15.70:0/2371093241 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 0x7f30080c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.952+0000 7f300ec4b640 1 --2- 172.21.15.70:0/2371093241 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3008159ac0 0x7f300806d5e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.952+0000 7f300ec4b640 1 -- 172.21.15.70:0/2371093241 >> 172.21.15.70:0/2371093241 conn(0x7f300806a380 msgr2=0x7f30080b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:17.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.952+0000 7f300ec4b640 1 -- 172.21.15.70:0/2371093241 shutdown_connections 2023-10-08T07:16:17.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.952+0000 7f300ec4b640 1 -- 172.21.15.70:0/2371093241 wait complete. 2023-10-08T07:16:17.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.954+0000 7f300ec4b640 1 Processor -- start 2023-10-08T07:16:17.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.954+0000 7f300ec4b640 1 -- start start 2023-10-08T07:16:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.955+0000 7f300ec4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 0x7f30080af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.955+0000 7f300ec4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30080ad790 0x7f300815c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f300ec4b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3008159ac0 0x7f3008151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f300ec4b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30081ef170 con 0x7f30080ad790 2023-10-08T07:16:17.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f300ec4b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30081eed80 con 0x7f3008159ac0 2023-10-08T07:16:17.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f300ec4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30081ee170 con 0x7f30080709e0 2023-10-08T07:16:17.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f3007fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30080ad790 0x7f300815c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f3007fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30080ad790 0x7f300815c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38774/0 (socket says 172.21.15.70:38774) 2023-10-08T07:16:17.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f3007fff640 1 -- 172.21.15.70:0/2165977843 learned_addr learned my addr 172.21.15.70:0/2165977843 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f300d1e8640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3008159ac0 0x7f3008151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f300c9e7640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 0x7f30080af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f3007fff640 1 -- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 msgr2=0x7f30080af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f3007fff640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 0x7f30080af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f3007fff640 1 -- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3008159ac0 msgr2=0x7f3008151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f3007fff640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3008159ac0 0x7f3008151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f3007fff640 1 -- 172.21.15.70:0/2165977843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300000b770 con 0x7f30080ad790 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.956+0000 7f300d1e8640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3008159ac0 0x7f3008151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.957+0000 7f3007fff640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30080ad790 0x7f300815c960 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000c520 tx=0x7f2ff000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.957+0000 7f30057fa640 1 -- 172.21.15.70:0/2165977843 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00052d0 con 0x7f30080ad790 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.957+0000 7f30057fa640 1 -- 172.21.15.70:0/2165977843 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff0005430 con 0x7f30080ad790 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.957+0000 7f30057fa640 1 -- 172.21.15.70:0/2165977843 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0012870 con 0x7f30080ad790 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.957+0000 7f300ec4b640 1 -- 172.21.15.70:0/2165977843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff8009a70 con 0x7f30080ad790 2023-10-08T07:16:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.957+0000 7f300ec4b640 1 -- 172.21.15.70:0/2165977843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30081ee730 con 0x7f30080ad790 2023-10-08T07:16:17.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.958+0000 7f300ec4b640 1 -- 172.21.15.70:0/2165977843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3008067490 con 0x7f30080ad790 2023-10-08T07:16:17.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.959+0000 7f30057fa640 1 -- 172.21.15.70:0/2165977843 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f2ff0003610 con 0x7f30080ad790 2023-10-08T07:16:17.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.960+0000 7f30057fa640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2fe806af90 0x7f2fe806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:17.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.960+0000 7f30057fa640 1 -- 172.21.15.70:0/2165977843 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f2ff0088ff0 con 0x7f30080ad790 2023-10-08T07:16:17.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.961+0000 7f300c9e7640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2fe806af90 0x7f2fe806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:17.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.962+0000 7f300c9e7640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2fe806af90 0x7f2fe806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8036000 tx=0x7f2ff8030000 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:17.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:17.964+0000 7f30057fa640 1 -- 172.21.15.70:0/2165977843 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ff0053cf0 con 0x7f30080ad790 2023-10-08T07:16:18.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.218+0000 7f300ec4b640 1 -- 172.21.15.70:0/2165977843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f30080b0ff0 con 0x7f30080ad790 2023-10-08T07:16:18.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.353+0000 7f30057fa640 1 -- 172.21.15.70:0/2165977843 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3352) v1 ==== 155+0+0 (secure 0 0 0) 0x7f2ff0057310 con 0x7f30080ad790 2023-10-08T07:16:18.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.356+0000 7f300ec4b640 1 -- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2fe806af90 msgr2=0x7f2fe806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:18.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.356+0000 7f300ec4b640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2fe806af90 0x7f2fe806d450 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8036000 tx=0x7f2ff8030000 comp rx=0 tx=0).stop 2023-10-08T07:16:18.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.357+0000 7f300ec4b640 1 -- 172.21.15.70:0/2165977843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30080ad790 msgr2=0x7f300815c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:18.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.357+0000 7f300ec4b640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30080ad790 0x7f300815c960 secure :-1 s=READY pgs=3001 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000c520 tx=0x7f2ff000c9e0 comp rx=0 tx=0).stop 2023-10-08T07:16:18.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.357+0000 7f300ec4b640 1 -- 172.21.15.70:0/2165977843 shutdown_connections 2023-10-08T07:16:18.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.357+0000 7f300ec4b640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f2fe806af90 0x7f2fe806d450 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:18.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.357+0000 7f300ec4b640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f3008159ac0 0x7f3008151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:18.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.357+0000 7f300ec4b640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30080ad790 0x7f300815c960 unknown :-1 s=CLOSED pgs=3001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:18.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.357+0000 7f300ec4b640 1 --2- 172.21.15.70:0/2165977843 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f30080709e0 0x7f30080af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:18.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.357+0000 7f300ec4b640 1 -- 172.21.15.70:0/2165977843 >> 172.21.15.70:0/2165977843 conn(0x7f300806a380 msgr2=0x7f3008073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:18.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.357+0000 7f300ec4b640 1 -- 172.21.15.70:0/2165977843 shutdown_connections 2023-10-08T07:16:18.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:18.357+0000 7f300ec4b640 1 -- 172.21.15.70:0/2165977843 wait complete. 2023-10-08T07:16:18.376 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-10-08T07:16:18.376 INFO:tasks.cephfs_test_runner: 2023-10-08T07:16:18.376 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-08T07:16:18.377 INFO:tasks.cephfs_test_runner:Ran 14 tests in 5948.058s 2023-10-08T07:16:18.377 INFO:tasks.cephfs_test_runner: 2023-10-08T07:16:18.377 INFO:tasks.cephfs_test_runner:OK 2023-10-08T07:16:18.377 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-08T07:16:18.393 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-08T07:16:18.425 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-08T07:16:18.425 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:16:18.425 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:16:18.484 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:16:18.484 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-10-08T07:16:18.484 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:16:18.537 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053588,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-10-08T07:16:18.537 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:16:18.538 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:16:18.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:16:18.599 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2023-10-08T07:16:18.599 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-08T07:16:18.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053578,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248784k,nr_inodes=812196,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497572k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-10-08T07:16:18.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:18.654 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T07:16:18.714 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T07:16:18.714 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T07:16:18.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:18.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T07:16:18.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:18.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T07:16:18.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:18.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2023-10-08T07:16:19.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:19.074 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-10-08T07:16:19.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:19.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2023-10-08T07:16:19.262 INFO:teuthology.orchestra.run.smithi070.stderr:Cannot find device "ceph-brx" 2023-10-08T07:16:19.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T07:16:19.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:19.266 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-08T07:16:19.326 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T07:16:19.326 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T07:16:19.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:19.327 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T07:16:19.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:19.428 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T07:16:19.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:19.520 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-08T07:16:19.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:19.657 DEBUG:teuthology.orchestra.run.smithi181:> ip netns list 2023-10-08T07:16:19.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T07:16:19.706 DEBUG:teuthology.orchestra.run.smithi181:> sudo ip link delete ceph-brx 2023-10-08T07:16:19.788 INFO:teuthology.orchestra.run.smithi181.stderr:Cannot find device "ceph-brx" 2023-10-08T07:16:19.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T07:16:19.792 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-08T07:16:19.802 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-08T07:16:20.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:16:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.570+0000 7f218da6f640 1 Processor -- start 2023-10-08T07:16:20.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.570+0000 7f218da6f640 1 -- start start 2023-10-08T07:16:20.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.571+0000 7f218da6f640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f218815ab40 0x7f21880ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:20.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.572+0000 7f218da6f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 0x7f21880adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:20.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.572+0000 7f218da6f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218815b2b0 con 0x7f21880cae70 2023-10-08T07:16:20.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.572+0000 7f218da6f640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218815bf30 con 0x7f218815ab40 2023-10-08T07:16:20.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.572+0000 7f218da6f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218815f7a0 con 0x7f21880c90b0 2023-10-08T07:16:20.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.572+0000 7f21877fe640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f218815ab40 0x7f21880ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.573+0000 7f217ffff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 0x7f21880adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.573+0000 7f2187fff640 1 --1- >> v1:172.21.15.181:6789/0 conn(0x7f21880cae70 0x7f21880702d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.181:6789/0 says I am v1:172.21.15.70:41304/0 (socket says 172.21.15.70:41304) 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.573+0000 7f2187fff640 1 -- 172.21.15.70:0/240553665 learned_addr learned my addr 172.21.15.70:0/240553665 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.573+0000 7f217ffff640 1 -- 172.21.15.70:0/240553665 >> v2:172.21.15.181:3300/0 conn(0x7f218815ab40 msgr2=0x7f21880ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.573+0000 7f217ffff640 1 --2- 172.21.15.70:0/240553665 >> v2:172.21.15.181:3300/0 conn(0x7f218815ab40 0x7f21880ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.573+0000 7f217ffff640 1 -- 172.21.15.70:0/240553665 >> v1:172.21.15.181:6789/0 conn(0x7f21880cae70 legacy=0x7f21880702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.573+0000 7f217ffff640 1 -- 172.21.15.70:0/240553665 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218815b590 con 0x7f21880c90b0 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.575+0000 7f217ffff640 1 --2- 172.21.15.70:0/240553665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 0x7f21880adc60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f21700097a0 tx=0x7f217002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=316b3e902a047b9d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.575+0000 7f2186ffd640 1 -- 172.21.15.70:0/240553665 <== mon.2 v2:172.21.15.181:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170031830 con 0x7f21880c90b0 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.575+0000 7f2186ffd640 1 -- 172.21.15.70:0/240553665 <== mon.2 v2:172.21.15.181:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2170031990 con 0x7f21880c90b0 2023-10-08T07:16:20.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.575+0000 7f2186ffd640 1 -- 172.21.15.70:0/240553665 <== mon.2 v2:172.21.15.181:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170031d20 con 0x7f21880c90b0 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.576+0000 7f218da6f640 1 -- 172.21.15.70:0/240553665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 msgr2=0x7f21880adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.576+0000 7f218da6f640 1 --2- 172.21.15.70:0/240553665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 0x7f21880adc60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f21700097a0 tx=0x7f217002b8d0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.576+0000 7f218da6f640 1 -- 172.21.15.70:0/240553665 shutdown_connections 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.576+0000 7f218da6f640 1 --2- 172.21.15.70:0/240553665 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 0x7f21880adc60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.576+0000 7f218da6f640 1 --2- 172.21.15.70:0/240553665 >> v2:172.21.15.181:3300/0 conn(0x7f218815ab40 0x7f21880ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.576+0000 7f218da6f640 1 -- 172.21.15.70:0/240553665 >> 172.21.15.70:0/240553665 conn(0x7f218806a380 msgr2=0x7f21880b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.576+0000 7f218da6f640 1 -- 172.21.15.70:0/240553665 shutdown_connections 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.576+0000 7f218da6f640 1 -- 172.21.15.70:0/240553665 wait complete. 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.578+0000 7f218da6f640 1 Processor -- start 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.578+0000 7f218da6f640 1 -- start start 2023-10-08T07:16:20.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.579+0000 7f218da6f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 0x7f21880c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.579+0000 7f218da6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21880cae70 0x7f218815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.579+0000 7f218da6f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f218815ab40 0x7f21880c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.579+0000 7f218da6f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21881536e0 con 0x7f21880cae70 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.579+0000 7f218da6f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21880c5ee0 con 0x7f218815ab40 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.579+0000 7f218da6f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21880c7340 con 0x7f21880c90b0 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.579+0000 7f217ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21880cae70 0x7f218815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.579+0000 7f217ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21880cae70 0x7f218815a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38776/0 (socket says 172.21.15.70:38776) 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.579+0000 7f217ffff640 1 -- 172.21.15.70:0/77976096 learned_addr learned my addr 172.21.15.70:0/77976096 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f21877fe640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 0x7f21880c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f2187fff640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f218815ab40 0x7f21880c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f217ffff640 1 -- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 msgr2=0x7f21880c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:20.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f217ffff640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 0x7f21880c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f217ffff640 1 -- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f218815ab40 msgr2=0x7f21880c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f217ffff640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f218815ab40 0x7f21880c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f217ffff640 1 -- 172.21.15.70:0/77976096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2170009400 con 0x7f21880cae70 2023-10-08T07:16:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f21877fe640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 0x7f21880c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f2187fff640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f218815ab40 0x7f21880c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f217ffff640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21880cae70 0x7f218815a030 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7f217002b910 tx=0x7f21700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f2184ff9640 1 -- 172.21.15.70:0/77976096 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217003a8f0 con 0x7f21880cae70 2023-10-08T07:16:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f2184ff9640 1 -- 172.21.15.70:0/77976096 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217003aa50 con 0x7f21880cae70 2023-10-08T07:16:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.580+0000 7f2184ff9640 1 -- 172.21.15.70:0/77976096 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217003ae00 con 0x7f21880cae70 2023-10-08T07:16:20.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.581+0000 7f218da6f640 1 -- 172.21.15.70:0/77976096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21880c75e0 con 0x7f21880cae70 2023-10-08T07:16:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.581+0000 7f218da6f640 1 -- 172.21.15.70:0/77976096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21880c7ba0 con 0x7f21880cae70 2023-10-08T07:16:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.582+0000 7f218da6f640 1 -- 172.21.15.70:0/77976096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2188067490 con 0x7f21880cae70 2023-10-08T07:16:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.583+0000 7f2184ff9640 1 -- 172.21.15.70:0/77976096 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f217004d450 con 0x7f21880cae70 2023-10-08T07:16:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.583+0000 7f2184ff9640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f216806af90 0x7f216806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.584+0000 7f2184ff9640 1 -- 172.21.15.70:0/77976096 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f21700abb10 con 0x7f21880cae70 2023-10-08T07:16:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.584+0000 7f21877fe640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f216806af90 0x7f216806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.585+0000 7f21877fe640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f216806af90 0x7f216806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2178009ac0 tx=0x7f2178008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.588+0000 7f2184ff9640 1 -- 172.21.15.70:0/77976096 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21700766f0 con 0x7f21880cae70 2023-10-08T07:16:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.772+0000 7f218da6f640 1 -- 172.21.15.70:0/77976096 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f218815bb50 con 0x7f216806af90 2023-10-08T07:16:20.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.779+0000 7f2184ff9640 1 -- 172.21.15.70:0/77976096 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48616 (secure 0 0 0) 0x7f2178010910 con 0x7f216806af90 2023-10-08T07:16:20.876 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:20.876 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:16:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 -- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f216806af90 msgr2=0x7f216806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f216806af90 0x7f216806d450 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2178009ac0 tx=0x7f2178008040 comp rx=0 tx=0).stop 2023-10-08T07:16:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 -- 172.21.15.70:0/77976096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21880cae70 msgr2=0x7f218815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21880cae70 0x7f218815a030 secure :-1 s=READY pgs=3002 cs=0 l=1 rev1=1 crypto rx=0x7f217002b910 tx=0x7f21700046c0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 -- 172.21.15.70:0/77976096 shutdown_connections 2023-10-08T07:16:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f216806af90 0x7f216806d450 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f218815ab40 0x7f21880c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21880cae70 0x7f218815a030 unknown :-1 s=CLOSED pgs=3002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 --2- 172.21.15.70:0/77976096 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21880c90b0 0x7f21880c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:20.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 -- 172.21.15.70:0/77976096 >> 172.21.15.70:0/77976096 conn(0x7f218806a380 msgr2=0x7f21880ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:20.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.782+0000 7f218da6f640 1 -- 172.21.15.70:0/77976096 shutdown_connections 2023-10-08T07:16:20.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:20.783+0000 7f218da6f640 1 -- 172.21.15.70:0/77976096 wait complete. 2023-10-08T07:16:20.879 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-10-08T07:16:18.901925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4883,"ondisk_log_size":4883,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":450240,"kb_used_data":30052,"kb_used_omap":12,"kb_used_meta":420083,"kb_avail":754524480,"statfs":{"total":773094113280,"available":772633067520,"internally_reserved":0,"allocated":30773248,"data_stored":17303003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":430165563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-670102,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-265,"num_write_kb":-2058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.190186"},"pg_stats":[{"pgid":"1.0","version":"278'4883","reported_seq":6008,"reported_epoch":278,"state":"active+clean","last_fresh":"2023-10-08T07:16:11.029764+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:16:11.029764+0000","last_peered":"2023-10-08T07:16:11.029764+0000","last_clean":"2023-10-08T07:16:11.029764+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:16:11.029764+0000","last_undegraded":"2023-10-08T07:16:11.029764+0000","last_fullsized":"2023-10-08T07:16:11.029764+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":4883,"log_dups_size":0,"ondisk_log_size":4883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4883,"ondisk_log_size":4883,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576037,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75796,"kb_used_data":4364,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296044,"statfs":{"total":96636764160,"available":96559149056,"internally_reserved":0,"allocated":4468736,"data_stored":2773846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1419999999999999,"15min":1.198},"min":{"1min":0.94599999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.625,"5min":1.9410000000000001,"15min":2.008},"last":1.421},{"interface":"front","average":{"1min":1.28,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.82899999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.96299999999999997}]},{"osd":1,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.244,"15min":1.248},"min":{"1min":0.88400000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.964,"5min":1.964,"15min":1.998},"last":1.071},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.3320000000000001},"min":{"1min":0.86299999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.1799999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2190000000000001,"15min":1.286},"min":{"1min":0.86499999999999999,"5min":0.86399999999999999,"15min":0.77600000000000002},"max":{"1min":1.573,"5min":1.8440000000000001,"15min":2.1619999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.383,"15min":1.3540000000000001},"min":{"1min":0.77700000000000002,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.1190000000000002},"last":1.2909999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3300000000000001,"15min":1.341},"min":{"1min":0.96299999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.4330000000000001,"15min":1.4119999999999999},"min":{"1min":1.117,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.5369999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.3839999999999999,"15min":1.339},"min":{"1min":1.0660000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.46,"15min":1.4259999999999999},"min":{"1min":1.294,"5min":0.752,"15min":0.752},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.3620000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.53,"15min":1.4770000000000001},"min":{"1min":1.2490000000000001,"5min":0.80600000000000005,"15min":0.79300000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.5409999999999999,"15min":1.514},"min":{"1min":1.242,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":1.337}]},{"osd":6,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.8999999999999999,"5min":1.6619999999999999,"15min":1.619},"min":{"1min":1.1639999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.577},{"interface":"front","average":{"1min":1.8240000000000001,"5min":1.629,"15min":1.5649999999999999},"min":{"1min":1.2090000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":1.7749999999999999}]}]},{"osd":6,"up_from":13,"seq":55834576036,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48832,"kb_used_data":3764,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323008,"statfs":{"total":96636764160,"available":96586760192,"internally_reserved":0,"allocated":3854336,"data_stored":2176461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2370000000000001,"15min":1.274},"min":{"1min":0.76100000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5600000000000001,"5min":1.78,"15min":1.9490000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2949999999999999,"15min":1.3460000000000001},"min":{"1min":0.84299999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7370000000000001,"5min":1.76,"15min":2.0070000000000001},"last":1.3029999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.121,"15min":1.2070000000000001},"min":{"1min":0.68200000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.6020000000000001,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.677},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2310000000000001,"15min":1.236},"min":{"1min":0.73999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6619999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.6399999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1699999999999999,"15min":1.2210000000000001},"min":{"1min":0.71599999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.956,"5min":1.956,"15min":2.339},"last":1.843},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.246,"15min":1.294},"min":{"1min":0.83499999999999996,"5min":0.748,"15min":0.748},"max":{"1min":1.484,"5min":1.8720000000000001,"15min":2.1779999999999999},"last":1.379}]},{"osd":3,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.391,"15min":1.3979999999999999},"min":{"1min":0.95799999999999996,"5min":0.86299999999999999,"15min":0.73099999999999998},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.8},{"interface":"front","average":{"1min":1.45,"5min":1.421,"15min":1.4770000000000001},"min":{"1min":0.998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.8440000000000001,"5min":1.8979999999999999,"15min":2.1459999999999999},"last":1.712}]},{"osd":4,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4450000000000001,"15min":1.415},"min":{"1min":1.048,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8140000000000001,"5min":2.0510000000000002,"15min":2.0859999999999999},"last":1.8999999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.4850000000000001,"15min":1.474},"min":{"1min":1.01,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.484},"last":1.9590000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.403,"15min":1.4299999999999999},"min":{"1min":0.90200000000000002,"5min":0.75,"15min":0.75},"max":{"1min":2.093,"5min":2.093,"15min":2.403},"last":1.599},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.5960000000000001,"15min":1.5700000000000001},"min":{"1min":1.1259999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1320000000000001,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":1.4279999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.5940000000000001,"15min":1.595},"min":{"1min":1.0209999999999999,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.1469999999999998,"5min":2.339,"15min":2.5870000000000002},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.5820000000000001,"15min":1.571},"min":{"1min":1.145,"5min":0.87,"15min":0.87},"max":{"1min":2.2370000000000001,"5min":2.2959999999999998,"15min":2.665},"last":1.74}]}]},{"osd":5,"up_from":13,"seq":55834576036,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60996,"kb_used_data":3448,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310844,"statfs":{"total":96636764160,"available":96574304256,"internally_reserved":0,"allocated":3530752,"data_stored":1848736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1499999999999999,"15min":1.202},"min":{"1min":0.748,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5740000000000001,"5min":1.98,"15min":1.98},"last":0.997},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1910000000000001,"15min":1.2190000000000001},"min":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.74,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.3640000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.165,"15min":1.218},"min":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.5449999999999999,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2849999999999999,"15min":1.3100000000000001},"min":{"1min":0.78800000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.752,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":1.6160000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.254,"15min":1.2969999999999999},"min":{"1min":0.77100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.7050000000000001,"5min":2.008,"15min":2.0870000000000002},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.29},"min":{"1min":0.77600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.677,"5min":1.877,"15min":1.905},"last":1.4399999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.97699999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.694,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":1.417},{"interface":"front","average":{"1min":1.355,"5min":1.4219999999999999,"15min":1.4259999999999999},"min":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.883,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":1.532}]},{"osd":4,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.329,"15min":1.3300000000000001},"min":{"1min":0.93600000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.857,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.4910000000000001,"15min":1.4399999999999999},"min":{"1min":1.194,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1600000000000001,"5min":2.742,"15min":2.742},"last":1.2470000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5780000000000001,"15min":1.4890000000000001},"min":{"1min":1.036,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9450000000000001,"5min":13.234999999999999,"15min":13.234999999999999},"last":1.589},{"interface":"front","average":{"1min":1.585,"5min":1.5449999999999999,"15min":1.4870000000000001},"min":{"1min":1.1830000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9770000000000001,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":1.6459999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.5649999999999999,"15min":1.548},"min":{"1min":1.208,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.0979999999999999,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.5},{"interface":"front","average":{"1min":1.669,"5min":1.609,"15min":1.6040000000000001},"min":{"1min":1.3260000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.02,"5min":2.351,"15min":2.351},"last":1.6779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576036,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48540,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323300,"statfs":{"total":96636764160,"available":96587059200,"internally_reserved":0,"allocated":3555328,"data_stored":1862162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.67800000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.3220000000000001,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.26},{"interface":"front","average":{"1min":1.45,"5min":1.4359999999999999,"15min":1.423},"min":{"1min":0.63600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.3609999999999998,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.3540000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2470000000000001,"15min":1.2849999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.2229999999999999,"5min":4.2229999999999999,"15min":4.2229999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":2.7799999999999998,"5min":1.635,"15min":1.371},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":28.907,"5min":28.907,"15min":28.907},"last":1.0669999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4810000000000001,"15min":1.335},"min":{"1min":0.70899999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":3.6120000000000001,"5min":17.175999999999998,"15min":17.175999999999998},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.444,"15min":1.391},"min":{"1min":0.59499999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.4080000000000004,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":0.96299999999999997}]},{"osd":3,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.3260000000000001,"5min":1.5720000000000001,"15min":1.4159999999999999},"min":{"1min":0.60099999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":15.938000000000001,"5min":15.938000000000001,"15min":15.938000000000001},"last":1.619},{"interface":"front","average":{"1min":1.8069999999999999,"5min":1.5760000000000001,"15min":1.474},"min":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.787000000000001,"5min":10.787000000000001,"15min":10.787000000000001},"last":1.373}]},{"osd":5,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.591,"15min":1.4350000000000001},"min":{"1min":0.73199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":4.2939999999999996,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.302},{"interface":"front","average":{"1min":3.0209999999999999,"5min":1.8160000000000001,"15min":1.5289999999999999},"min":{"1min":0.84899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":31.347999999999999,"5min":31.347999999999999,"15min":31.347999999999999},"last":1.486}]},{"osd":6,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.016,"5min":1.7290000000000001,"15min":1.5389999999999999},"min":{"1min":0.73699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":13.885,"5min":13.885,"15min":13.885},"last":1.5620000000000001},{"interface":"front","average":{"1min":2.117,"5min":1.833,"15min":1.6419999999999999},"min":{"1min":0.82499999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":13.446999999999999,"5min":13.446999999999999,"15min":13.446999999999999},"last":1.405}]},{"osd":7,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.528,"5min":1.821,"15min":1.611},"min":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":15.843999999999999,"5min":15.843999999999999,"15min":15.843999999999999},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.726,"15min":1.6659999999999999},"min":{"1min":0.82399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.968,"5min":5.609,"15min":5.609},"last":1.843}]}]},{"osd":3,"up_from":13,"seq":55834576038,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57160,"kb_used_data":3772,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314680,"statfs":{"total":96636764160,"available":96578232320,"internally_reserved":0,"allocated":3862528,"data_stored":2177004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.96499999999999997},"min":{"1min":0.51600000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.323,"5min":1.6719999999999999,"15min":2.7599999999999998},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.046,"15min":1.1439999999999999},"min":{"1min":0.55300000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.72,"5min":1.72,"15min":4.5659999999999998},"last":1.9259999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":1.0069999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":1.577},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1699999999999999,"15min":1.2609999999999999},"min":{"1min":0.58699999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.504,"5min":2.109,"15min":2.4889999999999999},"last":1.1679999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.032,"15min":1.1220000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.437},"max":{"1min":1.53,"5min":2.165,"15min":2.2250000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.119,"15min":1.1850000000000001},"min":{"1min":0.67800000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6200000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.0580000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2170000000000001,"15min":1.268},"min":{"1min":0.93899999999999995,"5min":0.70899999999999996,"15min":0.67500000000000004},"max":{"1min":1.732,"5min":1.732,"15min":2.5379999999999998},"last":1.222},{"interface":"front","average":{"1min":1.355,"5min":1.3680000000000001,"15min":1.4410000000000001},"min":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":2.306,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":2.2069999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2989999999999999,"15min":1.355},"min":{"1min":0.79800000000000004,"5min":0.755,"15min":0.61799999999999999},"max":{"1min":1.9079999999999999,"5min":1.923,"15min":2.9740000000000002},"last":1.9590000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.393,"15min":1.4299999999999999},"min":{"1min":0.86599999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.7330000000000001},"last":1.5009999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3740000000000001,"15min":1.405},"min":{"1min":1.0569999999999999,"5min":0.82299999999999995,"15min":0.59699999999999998},"max":{"1min":1.825,"5min":1.97,"15min":2.653},"last":1.641},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4390000000000001,"15min":1.504},"min":{"1min":1.1140000000000001,"5min":0.872,"15min":0.57499999999999996},"max":{"1min":1.7749999999999999,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":2.0539999999999998}]},{"osd":7,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.488,"15min":1.5529999999999999},"min":{"1min":0.91000000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.0590000000000002,"5min":2.3220000000000001,"15min":2.8650000000000002},"last":1.6879999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.4359999999999999,"15min":1.506},"min":{"1min":0.66700000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.8340000000000001,"5min":1.903,"15min":2.7749999999999999},"last":1.7889999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576036,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52668,"kb_used_data":3440,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319172,"statfs":{"total":96636764160,"available":96582832128,"internally_reserved":0,"allocated":3522560,"data_stored":1848680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.86099999999999999,"15min":0.95299999999999996},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.2270000000000001,"5min":1.601,"15min":1.948},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.98699999999999999,"15min":1.0529999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.002}]},{"osd":1,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.90100000000000002,"15min":0.97399999999999998},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":1.0609999999999999,"15min":1.167},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.587,"5min":1.7789999999999999,"15min":2.1309999999999998},"last":0.91100000000000003}]},{"osd":3,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.0860000000000001,"15min":1.1639999999999999},"min":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.9359999999999999,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.155,"15min":1.2490000000000001},"min":{"1min":0.47699999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.6699999999999999,"5min":1.7909999999999999,"15min":2.0670000000000002},"last":1.26}]},{"osd":4,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.3500000000000001,"15min":1.421},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.8640000000000001,"5min":1.992,"15min":2.1789999999999998},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.2789999999999999,"15min":1.306},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.2929999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.3160000000000001,"15min":1.385},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":2.5659999999999998,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.4279999999999999,"15min":1.51},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0750000000000002,"5min":2.0870000000000002,"15min":2.9390000000000001},"last":1.4530000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.3260000000000001,"15min":1.385},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.4339999999999999,"15min":1.51},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.2410000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:13 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.4059999999999999,"15min":1.464},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.4689999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.466,"15min":1.5580000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":2.831,"5min":2.831,"15min":2.831},"last":1.417}]}]},{"osd":1,"up_from":13,"seq":55834576036,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52672,"kb_used_data":3444,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319168,"statfs":{"total":96636764160,"available":96582828032,"internally_reserved":0,"allocated":3526656,"data_stored":1848770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.442,"5min":0.439,"15min":0.32100000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.139},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0489999999999999,"15min":1.177},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.41899999999999998},"max":{"1min":1.3939999999999999,"5min":1.7010000000000001,"15min":1.79},"last":0.89600000000000002}]},{"osd":2,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.86099999999999999,"15min":0.93500000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.462,"15min":1.641},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.083,"15min":1.1299999999999999},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.372},"max":{"1min":1.4750000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.747}]},{"osd":3,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.151,"15min":1.2250000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.621,"5min":2.496,"15min":2.496},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.129},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6859999999999999,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":0.82399999999999995}]},{"osd":4,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2529999999999999,"15min":1.333},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4339999999999999,"5min":2.73,"15min":2.8460000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2410000000000001,"15min":1.2889999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.744,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.339}]},{"osd":5,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3089999999999999,"15min":1.355},"min":{"1min":0.85599999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9910000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.298},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.518},"min":{"1min":0.98799999999999999,"5min":0.89600000000000002,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":2.601,"15min":2.601},"last":1.5449999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.4019999999999999,"15min":1.482},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.9710000000000001,"5min":2.883,"15min":2.9079999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.4039999999999999,"15min":1.474},"min":{"1min":0.752,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8939999999999999,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.4850000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.482,"15min":1.5329999999999999},"min":{"1min":0.85099999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1000000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4630000000000001,"15min":1.5429999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.0920000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.099}]}]},{"osd":0,"up_from":13,"seq":55834576037,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53576,"kb_used_data":4348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318264,"statfs":{"total":96636764160,"available":96581902336,"internally_reserved":0,"allocated":4452352,"data_stored":2767344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.86799999999999999,"15min":0.94299999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3029999999999999,"5min":1.3859999999999999,"15min":1.581},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96399999999999997,"15min":1.0620000000000001},"min":{"1min":0.38500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.47499999999999998}]},{"osd":2,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.03,"15min":1.1100000000000001},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.42199999999999999},"max":{"1min":4.1920000000000002,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.999,"5min":0.97999999999999998,"15min":1.052},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.4699999999999998,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.63700000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.099,"15min":1.2030000000000001},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.425,"5min":1.633,"15min":2.0139999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.089,"15min":1.143},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.327,"5min":4.327,"15min":4.327},"last":0.71899999999999997}]},{"osd":4,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.2250000000000001,"15min":1.3100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.5840000000000001,"5min":1.6599999999999999,"15min":2.4239999999999999},"last":1.038},{"interface":"front","average":{"1min":1.391,"5min":1.337,"15min":1.399},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.55700000000000005},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":1.258}]},{"osd":5,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.339,"15min":1.405},"min":{"1min":0.71299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.901,"5min":1.901,"15min":2.1309999999999998},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.4159999999999999,"15min":1.46},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.849,"5min":2.23,"15min":2.4900000000000002},"last":1.409}]},{"osd":6,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.413,"15min":1.4630000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":4.4260000000000002,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.216},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3720000000000001,"15min":1.476},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8240000000000001,"5min":1.9430000000000001,"15min":2.246},"last":0.90200000000000002}]},{"osd":7,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4490000000000001,"15min":1.5249999999999999},"min":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.72799999999999998},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.488,"15min":1.5409999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1360000000000001,"5min":4.1360000000000001,"15min":4.1360000000000001},"last":0.98999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:16:20.881 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:16:21.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.100+0000 7fcbc3b90640 1 Processor -- start 2023-10-08T07:16:21.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.100+0000 7fcbc3b90640 1 -- start start 2023-10-08T07:16:21.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.102+0000 7fcbc3b90640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fcbbc159ac0 0x7fcbbc06d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.102+0000 7fcbc3b90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc0ad790 0x7fcbbc0702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.102+0000 7fcbc3b90640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbbc06dee0 con 0x7fcbbc15e330 2023-10-08T07:16:21.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.102+0000 7fcbc3b90640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbbc15d850 con 0x7fcbbc159ac0 2023-10-08T07:16:21.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.102+0000 7fcbc3b90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15db30 con 0x7fcbbc0ad790 2023-10-08T07:16:21.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.102+0000 7fcbc212d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcbbc15e330 0x7fcbbc06c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51648/0 (socket says 172.21.15.70:51648) 2023-10-08T07:16:21.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.102+0000 7fcbc212d640 1 -- 172.21.15.70:0/2941470635 learned_addr learned my addr 172.21.15.70:0/2941470635 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:21.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc112b640 1 --2- 172.21.15.70:0/2941470635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc0ad790 0x7fcbbc0702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc192c640 1 --2- 172.21.15.70:0/2941470635 >> v2:172.21.15.181:3300/0 conn(0x7fcbbc159ac0 0x7fcbbc06d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106683494 0 0) 0x7fcbbc06dee0 con 0x7fcbbc15e330 2023-10-08T07:16:21.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcba4003520 con 0x7fcbbc15e330 2023-10-08T07:16:21.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2154788610 0 0) 0x7fcba4003520 con 0x7fcbbc15e330 2023-10-08T07:16:21.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc0ad790 msgr2=0x7fcbbc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:21.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 --2- 172.21.15.70:0/2941470635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc0ad790 0x7fcbbc0702d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 >> v2:172.21.15.181:3300/0 conn(0x7fcbbc159ac0 msgr2=0x7fcbbc06d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:21.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 --2- 172.21.15.70:0/2941470635 >> v2:172.21.15.181:3300/0 conn(0x7fcbbc159ac0 0x7fcbbc06d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc15de10 con 0x7fcbbc15e330 2023-10-08T07:16:21.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fcbb40019f0 con 0x7fcbbc15e330 2023-10-08T07:16:21.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 >> v1:172.21.15.70:6789/0 conn(0x7fcbbc15e330 legacy=0x7fcbbc06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.103+0000 7fcbc092a640 1 --2- 172.21.15.70:0/2941470635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba4003970 0x7fcba4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc092a640 1 --2- 172.21.15.70:0/2941470635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc0ad790 0x7fcba40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc092a640 1 --2- 172.21.15.70:0/2941470635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcba4004d00 0x7fcba4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15d850 con 0x7fcba4004d00 2023-10-08T07:16:21.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15db30 con 0x7fcba4003970 2023-10-08T07:16:21.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba400b980 con 0x7fcbbc0ad790 2023-10-08T07:16:21.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc212d640 1 --2- 172.21.15.70:0/2941470635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcba4004d00 0x7fcba4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc212d640 1 -- 172.21.15.70:0/2941470635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc0ad790 msgr2=0x7fcba40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:21.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc212d640 1 --2- 172.21.15.70:0/2941470635 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc0ad790 0x7fcba40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc212d640 1 -- 172.21.15.70:0/2941470635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba4003970 msgr2=0x7fcba4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:21.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc212d640 1 --2- 172.21.15.70:0/2941470635 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba4003970 0x7fcba4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.104+0000 7fcbc212d640 1 -- 172.21.15.70:0/2941470635 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba4003520 con 0x7fcba4004d00 2023-10-08T07:16:21.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.105+0000 7fcbc212d640 1 --2- 172.21.15.70:0/2941470635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcba4004d00 0x7fcba4005180 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400a790 tx=0x7fcba40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cfcbb4ab509894 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:21.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.106+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4034630 con 0x7fcba4004d00 2023-10-08T07:16:21.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.106+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb402f070 con 0x7fcba4004d00 2023-10-08T07:16:21.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.106+0000 7fcbc092a640 1 -- 172.21.15.70:0/2941470635 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40349f0 con 0x7fcba4004d00 2023-10-08T07:16:21.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.106+0000 7fcbc3b90640 1 -- 172.21.15.70:0/2941470635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcba4004d00 msgr2=0x7fcba4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.106+0000 7fcbc3b90640 1 --2- 172.21.15.70:0/2941470635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcba4004d00 0x7fcba4005180 secure :-1 s=READY pgs=3004 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400a790 tx=0x7fcba40057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.106+0000 7fcbc212d640 1 -- 172.21.15.70:0/2941470635 reap_dead start 2023-10-08T07:16:21.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.106+0000 7fcbc3b90640 1 -- 172.21.15.70:0/2941470635 shutdown_connections 2023-10-08T07:16:21.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.106+0000 7fcbc3b90640 1 -- 172.21.15.70:0/2941470635 >> 172.21.15.70:0/2941470635 conn(0x7fcbbc06a380 msgr2=0x7fcbbc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:21.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.106+0000 7fcbc3b90640 1 -- 172.21.15.70:0/2941470635 shutdown_connections 2023-10-08T07:16:21.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.106+0000 7fcbc3b90640 1 -- 172.21.15.70:0/2941470635 wait complete. 2023-10-08T07:16:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.108+0000 7fcbc3b90640 1 Processor -- start 2023-10-08T07:16:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.108+0000 7fcbc3b90640 1 -- start start 2023-10-08T07:16:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.109+0000 7fcbc3b90640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba4004d00 0x7fcbbc156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.109+0000 7fcbc3b90640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc152f30 0x7fcbbc15bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.109+0000 7fcbc3b90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbbc150100 0x7fcbbc0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.109+0000 7fcbc3b90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc156e90 con 0x7fcbbc150100 2023-10-08T07:16:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.109+0000 7fcbc3b90640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15abd0 con 0x7fcba4004d00 2023-10-08T07:16:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.109+0000 7fcbc3b90640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15aeb0 con 0x7fcbbc152f30 2023-10-08T07:16:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc212d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbbc150100 0x7fcbbc0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc212d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbbc150100 0x7fcbbc0ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38792/0 (socket says 172.21.15.70:38792) 2023-10-08T07:16:21.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc212d640 1 -- 172.21.15.70:0/709422863 learned_addr learned my addr 172.21.15.70:0/709422863 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:21.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc192c640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba4004d00 0x7fcbbc156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc112b640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc152f30 0x7fcbbc15bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc212d640 1 -- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc152f30 msgr2=0x7fcbbc15bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc212d640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc152f30 0x7fcbbc15bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc212d640 1 -- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba4004d00 msgr2=0x7fcbbc156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc212d640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba4004d00 0x7fcbbc156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc212d640 1 -- 172.21.15.70:0/709422863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc15b190 con 0x7fcbbc150100 2023-10-08T07:16:21.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.110+0000 7fcbc212d640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbbc150100 0x7fcbbc0ab8c0 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400a8b0 tx=0x7fcbb4001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:21.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.111+0000 7fcbba7fc640 1 -- 172.21.15.70:0/709422863 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4034840 con 0x7fcbbc150100 2023-10-08T07:16:21.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.111+0000 7fcbba7fc640 1 -- 172.21.15.70:0/709422863 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40349a0 con 0x7fcbbc150100 2023-10-08T07:16:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.111+0000 7fcbba7fc640 1 -- 172.21.15.70:0/709422863 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4034d10 con 0x7fcbbc150100 2023-10-08T07:16:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.111+0000 7fcbc3b90640 1 -- 172.21.15.70:0/709422863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbbc15b470 con 0x7fcbbc150100 2023-10-08T07:16:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.111+0000 7fcbc3b90640 1 -- 172.21.15.70:0/709422863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc15c1b0 con 0x7fcbbc150100 2023-10-08T07:16:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.112+0000 7fcbc3b90640 1 -- 172.21.15.70:0/709422863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbbc15b990 con 0x7fcbbc150100 2023-10-08T07:16:21.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.113+0000 7fcbba7fc640 1 -- 172.21.15.70:0/709422863 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcbb4031030 con 0x7fcbbc150100 2023-10-08T07:16:21.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.114+0000 7fcbba7fc640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcba806af90 0x7fcba806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.114+0000 7fcbba7fc640 1 -- 172.21.15.70:0/709422863 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fcbb40abff0 con 0x7fcbbc150100 2023-10-08T07:16:21.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.114+0000 7fcbc192c640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcba806af90 0x7fcba806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.116+0000 7fcbc192c640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcba806af90 0x7fcba806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcbac004770 tx=0x7fcbac008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:21.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.118+0000 7fcbba7fc640 1 -- 172.21.15.70:0/709422863 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcbbc15b990 con 0x7fcbbc150100 2023-10-08T07:16:21.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.303+0000 7fcbc3b90640 1 -- 172.21.15.70:0/709422863 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcbbc15b020 con 0x7fcba806af90 2023-10-08T07:16:21.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.310+0000 7fcbba7fc640 1 -- 172.21.15.70:0/709422863 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48345 (secure 0 0 0) 0x7fcbbc15b020 con 0x7fcba806af90 2023-10-08T07:16:21.311 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:21.311 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:16:21.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.312+0000 7fcbc3b90640 1 -- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcba806af90 msgr2=0x7fcba806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.312+0000 7fcbc3b90640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcba806af90 0x7fcba806d450 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcbac004770 tx=0x7fcbac008040 comp rx=0 tx=0).stop 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.312+0000 7fcbc3b90640 1 -- 172.21.15.70:0/709422863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbbc150100 msgr2=0x7fcbbc0ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.312+0000 7fcbc3b90640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbbc150100 0x7fcbbc0ab8c0 secure :-1 s=READY pgs=3005 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400a8b0 tx=0x7fcbb4001cb0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.313+0000 7fcbc3b90640 1 -- 172.21.15.70:0/709422863 shutdown_connections 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.313+0000 7fcbc3b90640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcba806af90 0x7fcba806d450 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.313+0000 7fcbc3b90640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbbc150100 0x7fcbbc0ab8c0 unknown :-1 s=CLOSED pgs=3005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.313+0000 7fcbc3b90640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcbbc152f30 0x7fcbbc15bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.313+0000 7fcbc3b90640 1 --2- 172.21.15.70:0/709422863 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcba4004d00 0x7fcbbc156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.313+0000 7fcbc3b90640 1 -- 172.21.15.70:0/709422863 >> 172.21.15.70:0/709422863 conn(0x7fcbbc06a380 msgr2=0x7fcbbc0701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.313+0000 7fcbc3b90640 1 -- 172.21.15.70:0/709422863 shutdown_connections 2023-10-08T07:16:21.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.313+0000 7fcbc3b90640 1 -- 172.21.15.70:0/709422863 wait complete. 2023-10-08T07:16:21.331 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2023-10-08T07:16:20.902414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4883,"ondisk_log_size":4883,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":453300,"kb_used_data":28936,"kb_used_omap":12,"kb_used_meta":424243,"kb_avail":754521420,"statfs":{"total":773094113280,"available":772629934080,"internally_reserved":0,"allocated":29630464,"data_stored":16083456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":434425403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-670102,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-265,"num_write_kb":-2058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002244"},"pg_stats":[{"pgid":"1.0","version":"278'4883","reported_seq":6010,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-10-08T07:16:15.136877+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:16:15.136877+0000","last_peered":"2023-10-08T07:16:15.136877+0000","last_clean":"2023-10-08T07:16:15.136877+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:16:15.136877+0000","last_undegraded":"2023-10-08T07:16:15.136877+0000","last_fullsized":"2023-10-08T07:16:15.136877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":4883,"log_dups_size":0,"ondisk_log_size":4883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4883,"ondisk_log_size":4883,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75488,"kb_used_data":4052,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296352,"statfs":{"total":96636764160,"available":96559464448,"internally_reserved":0,"allocated":4149248,"data_stored":2453208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1419999999999999,"15min":1.198},"min":{"1min":0.94599999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.625,"5min":1.9410000000000001,"15min":2.008},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.82899999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.115}]},{"osd":1,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.244,"15min":1.248},"min":{"1min":0.88400000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.964,"5min":1.964,"15min":1.998},"last":1.522},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.3320000000000001},"min":{"1min":0.86299999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.298}]},{"osd":2,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2190000000000001,"15min":1.286},"min":{"1min":0.86499999999999999,"5min":0.86399999999999999,"15min":0.77600000000000002},"max":{"1min":1.573,"5min":1.8440000000000001,"15min":2.1619999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.383,"15min":1.3540000000000001},"min":{"1min":0.77700000000000002,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.1190000000000002},"last":1.032}]},{"osd":3,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3300000000000001,"15min":1.341},"min":{"1min":0.96299999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.615,"5min":1.4330000000000001,"15min":1.4119999999999999},"min":{"1min":1.117,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.258}]},{"osd":4,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.3839999999999999,"15min":1.339},"min":{"1min":1.0660000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.488},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.46,"15min":1.4259999999999999},"min":{"1min":1.294,"5min":0.752,"15min":0.752},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.9870000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.53,"15min":1.4770000000000001},"min":{"1min":1.2490000000000001,"5min":0.80600000000000005,"15min":0.79300000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.633},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.5409999999999999,"15min":1.514},"min":{"1min":1.242,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":1.5940000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.8999999999999999,"5min":1.6619999999999999,"15min":1.619},"min":{"1min":1.1639999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.758},{"interface":"front","average":{"1min":1.8240000000000001,"5min":1.629,"15min":1.5649999999999999},"min":{"1min":1.2090000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":2.0539999999999998}]}]},{"osd":6,"up_from":13,"seq":55834576037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48540,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323300,"statfs":{"total":96636764160,"available":96587059200,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2370000000000001,"15min":1.274},"min":{"1min":0.76100000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5600000000000001,"5min":1.78,"15min":1.9490000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2949999999999999,"15min":1.3460000000000001},"min":{"1min":0.84299999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7370000000000001,"5min":1.76,"15min":2.0070000000000001},"last":1.171}]},{"osd":1,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.121,"15min":1.2070000000000001},"min":{"1min":0.68200000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.6020000000000001,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2310000000000001,"15min":1.236},"min":{"1min":0.73999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6619999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.089}]},{"osd":2,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1699999999999999,"15min":1.2210000000000001},"min":{"1min":0.71599999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.956,"5min":1.956,"15min":2.339},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.246,"15min":1.294},"min":{"1min":0.83499999999999996,"5min":0.748,"15min":0.748},"max":{"1min":1.484,"5min":1.8720000000000001,"15min":2.1779999999999999},"last":1.304}]},{"osd":3,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.391,"15min":1.3979999999999999},"min":{"1min":0.95799999999999996,"5min":0.86299999999999999,"15min":0.73099999999999998},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.421,"15min":1.4770000000000001},"min":{"1min":0.998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.8440000000000001,"5min":1.8979999999999999,"15min":2.1459999999999999},"last":1.401}]},{"osd":4,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4450000000000001,"15min":1.415},"min":{"1min":1.048,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8140000000000001,"5min":2.0510000000000002,"15min":2.0859999999999999},"last":0.877},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.4850000000000001,"15min":1.474},"min":{"1min":1.01,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.484},"last":0.97799999999999998}]},{"osd":5,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.403,"15min":1.4299999999999999},"min":{"1min":0.90200000000000002,"5min":0.75,"15min":0.75},"max":{"1min":2.093,"5min":2.093,"15min":2.403},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.5960000000000001,"15min":1.5700000000000001},"min":{"1min":1.1259999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1320000000000001,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":0.92700000000000005}]},{"osd":7,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.5940000000000001,"15min":1.595},"min":{"1min":1.0209999999999999,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.1469999999999998,"5min":2.339,"15min":2.5870000000000002},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.5820000000000001,"15min":1.571},"min":{"1min":1.145,"5min":0.87,"15min":0.87},"max":{"1min":2.2370000000000001,"5min":2.2959999999999998,"15min":2.665},"last":0.81200000000000006}]}]},{"osd":5,"up_from":13,"seq":55834576037,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61024,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310816,"statfs":{"total":96636764160,"available":96574275584,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1499999999999999,"15min":1.202},"min":{"1min":0.748,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5740000000000001,"5min":1.98,"15min":1.98},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1910000000000001,"15min":1.2190000000000001},"min":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.74,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.4259999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.165,"15min":1.218},"min":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.5449999999999999,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.879},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2849999999999999,"15min":1.3100000000000001},"min":{"1min":0.78800000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.752,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":1.24}]},{"osd":2,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.254,"15min":1.2969999999999999},"min":{"1min":0.77100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.7050000000000001,"5min":2.008,"15min":2.0870000000000002},"last":1.8779999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.29},"min":{"1min":0.77600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.677,"5min":1.877,"15min":1.905},"last":1.014}]},{"osd":3,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.97699999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.694,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.4219999999999999,"15min":1.4259999999999999},"min":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.883,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":2.008}]},{"osd":4,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.329,"15min":1.3300000000000001},"min":{"1min":0.93600000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.857,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.8100000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.4910000000000001,"15min":1.4399999999999999},"min":{"1min":1.194,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1600000000000001,"5min":2.742,"15min":2.742},"last":1.6120000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5780000000000001,"15min":1.4890000000000001},"min":{"1min":1.036,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9450000000000001,"5min":13.234999999999999,"15min":13.234999999999999},"last":2.2679999999999998},{"interface":"front","average":{"1min":1.585,"5min":1.5449999999999999,"15min":1.4870000000000001},"min":{"1min":1.1830000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9770000000000001,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":1.3700000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.5649999999999999,"15min":1.548},"min":{"1min":1.208,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.0979999999999999,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.911},{"interface":"front","average":{"1min":1.669,"5min":1.609,"15min":1.6040000000000001},"min":{"1min":1.3260000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.02,"5min":2.351,"15min":2.351},"last":2.327}]}]},{"osd":4,"up_from":13,"seq":55834576037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52704,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319136,"statfs":{"total":96636764160,"available":96582795264,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.67800000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.3220000000000001,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4359999999999999,"15min":1.423},"min":{"1min":0.63600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.3609999999999998,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.6619999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2470000000000001,"15min":1.2849999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.2229999999999999,"5min":4.2229999999999999,"15min":4.2229999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":2.7799999999999998,"5min":1.635,"15min":1.371},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":28.907,"5min":28.907,"15min":28.907},"last":1.8460000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4810000000000001,"15min":1.335},"min":{"1min":0.70899999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":3.6120000000000001,"5min":17.175999999999998,"15min":17.175999999999998},"last":1.129},{"interface":"front","average":{"1min":1.496,"5min":1.444,"15min":1.391},"min":{"1min":0.59499999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.4080000000000004,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":1.355}]},{"osd":3,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.3260000000000001,"5min":1.5720000000000001,"15min":1.4159999999999999},"min":{"1min":0.60099999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":15.938000000000001,"5min":15.938000000000001,"15min":15.938000000000001},"last":1.538},{"interface":"front","average":{"1min":1.8069999999999999,"5min":1.5760000000000001,"15min":1.474},"min":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.787000000000001,"5min":10.787000000000001,"15min":10.787000000000001},"last":1.401}]},{"osd":5,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.591,"15min":1.4350000000000001},"min":{"1min":0.73199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":4.2939999999999996,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.458},{"interface":"front","average":{"1min":3.0209999999999999,"5min":1.8160000000000001,"15min":1.5289999999999999},"min":{"1min":0.84899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":31.347999999999999,"5min":31.347999999999999,"15min":31.347999999999999},"last":1.917}]},{"osd":6,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.016,"5min":1.7290000000000001,"15min":1.5389999999999999},"min":{"1min":0.73699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":13.885,"5min":13.885,"15min":13.885},"last":1.585},{"interface":"front","average":{"1min":2.117,"5min":1.833,"15min":1.6419999999999999},"min":{"1min":0.82499999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":13.446999999999999,"5min":13.446999999999999,"15min":13.446999999999999},"last":1.772}]},{"osd":7,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.528,"5min":1.821,"15min":1.611},"min":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":15.843999999999999,"5min":15.843999999999999,"15min":15.843999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.726,"15min":1.6659999999999999},"min":{"1min":0.82399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.968,"5min":5.609,"15min":5.609},"last":1.804}]}]},{"osd":3,"up_from":13,"seq":55834576039,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56860,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314980,"statfs":{"total":96636764160,"available":96578539520,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.96499999999999997},"min":{"1min":0.51600000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.323,"5min":1.6719999999999999,"15min":2.7599999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.046,"15min":1.1439999999999999},"min":{"1min":0.55300000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.72,"5min":1.72,"15min":4.5659999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":1.0069999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1699999999999999,"15min":1.2609999999999999},"min":{"1min":0.58699999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.504,"5min":2.109,"15min":2.4889999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.032,"15min":1.1220000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.437},"max":{"1min":1.53,"5min":2.165,"15min":2.2250000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.079,"5min":1.119,"15min":1.1850000000000001},"min":{"1min":0.67800000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6200000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2170000000000001,"15min":1.268},"min":{"1min":0.93899999999999995,"5min":0.70899999999999996,"15min":0.67500000000000004},"max":{"1min":1.732,"5min":1.732,"15min":2.5379999999999998},"last":1.083},{"interface":"front","average":{"1min":1.355,"5min":1.3680000000000001,"15min":1.4410000000000001},"min":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":2.306,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.1950000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2989999999999999,"15min":1.355},"min":{"1min":0.79800000000000004,"5min":0.755,"15min":0.61799999999999999},"max":{"1min":1.9079999999999999,"5min":1.923,"15min":2.9740000000000002},"last":1.129},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.393,"15min":1.4299999999999999},"min":{"1min":0.86599999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.7330000000000001},"last":0.88100000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3740000000000001,"15min":1.405},"min":{"1min":1.0569999999999999,"5min":0.82299999999999995,"15min":0.59699999999999998},"max":{"1min":1.825,"5min":1.97,"15min":2.653},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4390000000000001,"15min":1.504},"min":{"1min":1.1140000000000001,"5min":0.872,"15min":0.57499999999999996},"max":{"1min":1.7749999999999999,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.1659999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.488,"15min":1.5529999999999999},"min":{"1min":0.91000000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.0590000000000002,"5min":2.3220000000000001,"15min":2.8650000000000002},"last":1.147},{"interface":"front","average":{"1min":1.389,"5min":1.4359999999999999,"15min":1.506},"min":{"1min":0.66700000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.8340000000000001,"5min":1.903,"15min":2.7749999999999999},"last":0.86299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52700,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319140,"statfs":{"total":96636764160,"available":96582799360,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85299999999999998,"15min":0.94199999999999995},"min":{"1min":0.38100000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.3029999999999999,"5min":1.601,"15min":1.948},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.97699999999999998,"15min":1.0369999999999999},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.482,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.02}]},{"osd":1,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.88700000000000001,"15min":0.95799999999999996},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.363,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.0309999999999999,"15min":1.147},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":1.4390000000000001,"5min":1.7789999999999999,"15min":1.9390000000000001},"last":1.5009999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0580000000000001,"15min":1.145},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.421,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":1.131},{"interface":"front","average":{"1min":1.004,"5min":1.123,"15min":1.2290000000000001},"min":{"1min":0.54600000000000004,"5min":0.47699999999999998,"15min":0.47599999999999998},"max":{"1min":1.54,"5min":1.6799999999999999,"15min":2.0670000000000002},"last":1.0880000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3280000000000001,"15min":1.409},"min":{"1min":0.78300000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.754,"5min":1.992,"15min":2.1789999999999998},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.256,"15min":1.298},"min":{"1min":0.78700000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.655,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.5589999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.298,"15min":1.365},"min":{"1min":0.75600000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.972,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.349},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3879999999999999,"15min":1.488},"min":{"1min":0.72499999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8939999999999999,"5min":2.0750000000000002,"15min":2.9390000000000001},"last":1.2969999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.306,"15min":1.371},"min":{"1min":0.72399999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.7370000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.667},{"interface":"front","average":{"1min":1.329,"5min":1.4059999999999999,"15min":1.4910000000000001},"min":{"1min":0.82699999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.891,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.6120000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.379,"15min":1.4470000000000001},"min":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8380000000000001,"5min":2.3969999999999998,"15min":2.4689999999999999},"last":1.774},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.454,"15min":1.538},"min":{"1min":0.90800000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.875,"5min":2.831,"15min":2.831},"last":1.7230000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52700,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319140,"statfs":{"total":96636764160,"available":96582799360,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.442,"5min":0.439,"15min":0.32100000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0489999999999999,"15min":1.177},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.41899999999999998},"max":{"1min":1.3939999999999999,"5min":1.7010000000000001,"15min":1.79},"last":1.246}]},{"osd":2,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.86099999999999999,"15min":0.93500000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.462,"15min":1.641},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.083,"15min":1.1299999999999999},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.372},"max":{"1min":1.4750000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.0800000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.151,"15min":1.2250000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.621,"5min":2.496,"15min":2.496},"last":1.3520000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.129},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6859999999999999,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":1.115}]},{"osd":4,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2529999999999999,"15min":1.333},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4339999999999999,"5min":2.73,"15min":2.8460000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2410000000000001,"15min":1.2889999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.744,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.8200000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3089999999999999,"15min":1.355},"min":{"1min":0.85599999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9910000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.518},"min":{"1min":0.98799999999999999,"5min":0.89600000000000002,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":2.601,"15min":2.601},"last":1.22}]},{"osd":6,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.4019999999999999,"15min":1.482},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.9710000000000001,"5min":2.883,"15min":2.9079999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.4039999999999999,"15min":1.474},"min":{"1min":0.752,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8939999999999999,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.754}]},{"osd":7,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.482,"15min":1.5329999999999999},"min":{"1min":0.85099999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1000000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4630000000000001,"15min":1.5429999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.0920000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.4730000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53284,"kb_used_data":4052,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318556,"statfs":{"total":96636764160,"available":96582201344,"internally_reserved":0,"allocated":4149248,"data_stored":2453208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.86799999999999999,"15min":0.94299999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3029999999999999,"5min":1.3859999999999999,"15min":1.581},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96399999999999997,"15min":1.0620000000000001},"min":{"1min":0.38500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.075}]},{"osd":2,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.03,"15min":1.1100000000000001},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.42199999999999999},"max":{"1min":4.1920000000000002,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.97999999999999998,"15min":1.052},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.4699999999999998,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.84799999999999998}]},{"osd":3,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.099,"15min":1.2030000000000001},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.425,"5min":1.633,"15min":2.0139999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.089,"15min":1.143},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.327,"5min":4.327,"15min":4.327},"last":1.363}]},{"osd":4,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.2250000000000001,"15min":1.3100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.5840000000000001,"5min":1.6599999999999999,"15min":2.4239999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.391,"5min":1.337,"15min":1.399},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.55700000000000005},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":1.462}]},{"osd":5,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.339,"15min":1.405},"min":{"1min":0.71299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.901,"5min":1.901,"15min":2.1309999999999998},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.4159999999999999,"15min":1.46},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.849,"5min":2.23,"15min":2.4900000000000002},"last":1.5189999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.413,"15min":1.4630000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":4.4260000000000002,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.554},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3720000000000001,"15min":1.476},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8240000000000001,"5min":1.9430000000000001,"15min":2.246},"last":1.1399999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4490000000000001,"15min":1.5249999999999999},"min":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.72799999999999998},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.488,"15min":1.5409999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1360000000000001,"5min":4.1360000000000001,"15min":4.1360000000000001},"last":1.712}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:16:21.332 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-08T07:16:21.333 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:16:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.547+0000 7f8cd7839640 1 Processor -- start 2023-10-08T07:16:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.547+0000 7f8cd7839640 1 -- start start 2023-10-08T07:16:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.549+0000 7f8cd7839640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cd015ab40 0x7f8cd00ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.549+0000 7f8cd7839640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8cd00cb580 0x7f8cd00adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.549+0000 7f8cd7839640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd015b2b0 con 0x7f8cd00c90b0 2023-10-08T07:16:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.549+0000 7f8cd7839640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd015bf30 con 0x7f8cd00cb580 2023-10-08T07:16:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.549+0000 7f8cd7839640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd015f7a0 con 0x7f8cd015ab40 2023-10-08T07:16:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.550+0000 7f8cd5dd6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8cd00cb580 0x7f8cd00adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.550+0000 7f8cd5dd6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8cd00cb580 0x7f8cd00adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38806/0 (socket says 172.21.15.70:38806) 2023-10-08T07:16:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.550+0000 7f8cd5dd6640 1 -- 172.21.15.70:0/3165223984 learned_addr learned my addr 172.21.15.70:0/3165223984 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.550+0000 7f8cd5dd6640 1 -- 172.21.15.70:0/3165223984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cd015ab40 msgr2=0x7f8cd00ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.550+0000 7f8cd5dd6640 1 --2- 172.21.15.70:0/3165223984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cd015ab40 0x7f8cd00ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.550+0000 7f8cd5dd6640 1 -- 172.21.15.70:0/3165223984 >> v1:172.21.15.181:6789/0 conn(0x7f8cd00c90b0 legacy=0x7f8cd00cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.550+0000 7f8cd5dd6640 1 -- 172.21.15.70:0/3165223984 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd015b590 con 0x7f8cd00cb580 2023-10-08T07:16:21.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.551+0000 7f8cd5dd6640 1 --2- 172.21.15.70:0/3165223984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00cb580 0x7f8cd00adc60 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009e50 tx=0x7f8cc802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f2ec4607216c95 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.551+0000 7f8cc7fff640 1 -- 172.21.15.70:0/3165223984 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc802ca80 con 0x7f8cd00cb580 2023-10-08T07:16:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.551+0000 7f8cc7fff640 1 -- 172.21.15.70:0/3165223984 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc802cbe0 con 0x7f8cd00cb580 2023-10-08T07:16:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.551+0000 7f8cc7fff640 1 -- 172.21.15.70:0/3165223984 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80343f0 con 0x7f8cd00cb580 2023-10-08T07:16:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.552+0000 7f8cd7839640 1 -- 172.21.15.70:0/3165223984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00cb580 msgr2=0x7f8cd00adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.552+0000 7f8cd7839640 1 --2- 172.21.15.70:0/3165223984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00cb580 0x7f8cd00adc60 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009e50 tx=0x7f8cc802bee0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.552+0000 7f8cd7839640 1 -- 172.21.15.70:0/3165223984 shutdown_connections 2023-10-08T07:16:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.552+0000 7f8cd7839640 1 --2- 172.21.15.70:0/3165223984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00cb580 0x7f8cd00adc60 unknown :-1 s=CLOSED pgs=3006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.552+0000 7f8cd7839640 1 --2- 172.21.15.70:0/3165223984 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cd015ab40 0x7f8cd00ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.552+0000 7f8cd7839640 1 -- 172.21.15.70:0/3165223984 >> 172.21.15.70:0/3165223984 conn(0x7f8cd006a380 msgr2=0x7f8cd00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.552+0000 7f8cd7839640 1 -- 172.21.15.70:0/3165223984 shutdown_connections 2023-10-08T07:16:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.552+0000 7f8cd7839640 1 -- 172.21.15.70:0/3165223984 wait complete. 2023-10-08T07:16:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.554+0000 7f8cd7839640 1 Processor -- start 2023-10-08T07:16:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.554+0000 7f8cd7839640 1 -- start start 2023-10-08T07:16:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.555+0000 7f8cd7839640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00c90b0 0x7f8cd0156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.555+0000 7f8cd7839640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cd00cb580 0x7f8cd0159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.555+0000 7f8cd7839640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd015ab40 0x7f8cd00ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.555+0000 7f8cd7839640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0156c20 con 0x7f8cd00c90b0 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.555+0000 7f8cd7839640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0158130 con 0x7f8cd015ab40 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.555+0000 7f8cd7839640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0158410 con 0x7f8cd00cb580 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.555+0000 7f8cd55d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00c90b0 0x7f8cd0156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cd55d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00c90b0 0x7f8cd0156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38820/0 (socket says 172.21.15.70:38820) 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cd55d5640 1 -- 172.21.15.70:0/2062721273 learned_addr learned my addr 172.21.15.70:0/2062721273 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cd55d5640 1 -- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cd00cb580 msgr2=0x7f8cd0159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cd55d5640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cd00cb580 0x7f8cd0159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cd55d5640 1 -- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd015ab40 msgr2=0x7f8cd00ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cd55d5640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd015ab40 0x7f8cd00ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cd55d5640 1 -- 172.21.15.70:0/2062721273 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc8009b30 con 0x7f8cd00c90b0 2023-10-08T07:16:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cd55d5640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00c90b0 0x7f8cd0156580 secure :-1 s=READY pgs=3007 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00b4a0 tx=0x7f8ccc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:21.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cc5ffb640 1 -- 172.21.15.70:0/2062721273 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc010610 con 0x7f8cd00c90b0 2023-10-08T07:16:21.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cc5ffb640 1 -- 172.21.15.70:0/2062721273 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc010770 con 0x7f8cd00c90b0 2023-10-08T07:16:21.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cd7839640 1 -- 172.21.15.70:0/2062721273 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd01586f0 con 0x7f8cd00c90b0 2023-10-08T07:16:21.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.556+0000 7f8cc5ffb640 1 -- 172.21.15.70:0/2062721273 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc010b20 con 0x7f8cd00c90b0 2023-10-08T07:16:21.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.557+0000 7f8cd7839640 1 -- 172.21.15.70:0/2062721273 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd015a210 con 0x7f8cd00c90b0 2023-10-08T07:16:21.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.558+0000 7f8cd7839640 1 -- 172.21.15.70:0/2062721273 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd0067490 con 0x7f8cd00c90b0 2023-10-08T07:16:21.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.559+0000 7f8cc5ffb640 1 -- 172.21.15.70:0/2062721273 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8ccc010c80 con 0x7f8cd00c90b0 2023-10-08T07:16:21.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.559+0000 7f8cc5ffb640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f8cb406af90 0x7f8cb406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.560+0000 7f8cc5ffb640 1 -- 172.21.15.70:0/2062721273 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f8ccc089530 con 0x7f8cd00c90b0 2023-10-08T07:16:21.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.560+0000 7f8cd4dd4640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f8cb406af90 0x7f8cb406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.561+0000 7f8cd4dd4640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f8cb406af90 0x7f8cb406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0006fd0 tx=0x7f8cc0006bc0 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:21.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.564+0000 7f8cc5ffb640 1 -- 172.21.15.70:0/2062721273 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ccc054350 con 0x7f8cd00c90b0 2023-10-08T07:16:21.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.749+0000 7f8cd7839640 1 -- 172.21.15.70:0/2062721273 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8cd015bb50 con 0x7f8cb406af90 2023-10-08T07:16:21.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.756+0000 7f8cc5ffb640 1 -- 172.21.15.70:0/2062721273 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48345 (secure 0 0 0) 0x7f8cc0013430 con 0x7f8cb406af90 2023-10-08T07:16:21.758 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:21.758 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:16:21.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 -- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f8cb406af90 msgr2=0x7f8cb406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f8cb406af90 0x7f8cb406d450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0006fd0 tx=0x7f8cc0006bc0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 -- 172.21.15.70:0/2062721273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00c90b0 msgr2=0x7f8cd0156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:21.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00c90b0 0x7f8cd0156580 secure :-1 s=READY pgs=3007 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00b4a0 tx=0x7f8ccc00b960 comp rx=0 tx=0).stop 2023-10-08T07:16:21.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 -- 172.21.15.70:0/2062721273 shutdown_connections 2023-10-08T07:16:21.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f8cb406af90 0x7f8cb406d450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f8cd015ab40 0x7f8cd00ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f8cd00cb580 0x7f8cd0159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 --2- 172.21.15.70:0/2062721273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd00c90b0 0x7f8cd0156580 unknown :-1 s=CLOSED pgs=3007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:21.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 -- 172.21.15.70:0/2062721273 >> 172.21.15.70:0/2062721273 conn(0x7f8cd006a380 msgr2=0x7f8cd00c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:21.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 -- 172.21.15.70:0/2062721273 shutdown_connections 2023-10-08T07:16:21.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.759+0000 7f8cd7839640 1 -- 172.21.15.70:0/2062721273 wait complete. 2023-10-08T07:16:21.779 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2023-10-08T07:16:20.902414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4883,"ondisk_log_size":4883,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":453300,"kb_used_data":28936,"kb_used_omap":12,"kb_used_meta":424243,"kb_avail":754521420,"statfs":{"total":773094113280,"available":772629934080,"internally_reserved":0,"allocated":29630464,"data_stored":16083456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":434425403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-670102,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-265,"num_write_kb":-2058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002244"},"pg_stats":[{"pgid":"1.0","version":"278'4883","reported_seq":6010,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-10-08T07:16:15.136877+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:16:15.136877+0000","last_peered":"2023-10-08T07:16:15.136877+0000","last_clean":"2023-10-08T07:16:15.136877+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:16:15.136877+0000","last_undegraded":"2023-10-08T07:16:15.136877+0000","last_fullsized":"2023-10-08T07:16:15.136877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":4883,"log_dups_size":0,"ondisk_log_size":4883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4883,"ondisk_log_size":4883,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75488,"kb_used_data":4052,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296352,"statfs":{"total":96636764160,"available":96559464448,"internally_reserved":0,"allocated":4149248,"data_stored":2453208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1419999999999999,"15min":1.198},"min":{"1min":0.94599999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.625,"5min":1.9410000000000001,"15min":2.008},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.82899999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.115}]},{"osd":1,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.244,"15min":1.248},"min":{"1min":0.88400000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.964,"5min":1.964,"15min":1.998},"last":1.522},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.3320000000000001},"min":{"1min":0.86299999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.298}]},{"osd":2,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2190000000000001,"15min":1.286},"min":{"1min":0.86499999999999999,"5min":0.86399999999999999,"15min":0.77600000000000002},"max":{"1min":1.573,"5min":1.8440000000000001,"15min":2.1619999999999999},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.383,"15min":1.3540000000000001},"min":{"1min":0.77700000000000002,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.1190000000000002},"last":1.032}]},{"osd":3,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3300000000000001,"15min":1.341},"min":{"1min":0.96299999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.615,"5min":1.4330000000000001,"15min":1.4119999999999999},"min":{"1min":1.117,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.258}]},{"osd":4,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.3839999999999999,"15min":1.339},"min":{"1min":1.0660000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.488},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.46,"15min":1.4259999999999999},"min":{"1min":1.294,"5min":0.752,"15min":0.752},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.9870000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.53,"15min":1.4770000000000001},"min":{"1min":1.2490000000000001,"5min":0.80600000000000005,"15min":0.79300000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.633},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.5409999999999999,"15min":1.514},"min":{"1min":1.242,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":1.5940000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.8999999999999999,"5min":1.6619999999999999,"15min":1.619},"min":{"1min":1.1639999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.758},{"interface":"front","average":{"1min":1.8240000000000001,"5min":1.629,"15min":1.5649999999999999},"min":{"1min":1.2090000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":2.0539999999999998}]}]},{"osd":6,"up_from":13,"seq":55834576037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48540,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323300,"statfs":{"total":96636764160,"available":96587059200,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2370000000000001,"15min":1.274},"min":{"1min":0.76100000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5600000000000001,"5min":1.78,"15min":1.9490000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2949999999999999,"15min":1.3460000000000001},"min":{"1min":0.84299999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7370000000000001,"5min":1.76,"15min":2.0070000000000001},"last":1.171}]},{"osd":1,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.121,"15min":1.2070000000000001},"min":{"1min":0.68200000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.6020000000000001,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2310000000000001,"15min":1.236},"min":{"1min":0.73999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6619999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.089}]},{"osd":2,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1699999999999999,"15min":1.2210000000000001},"min":{"1min":0.71599999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.956,"5min":1.956,"15min":2.339},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.246,"15min":1.294},"min":{"1min":0.83499999999999996,"5min":0.748,"15min":0.748},"max":{"1min":1.484,"5min":1.8720000000000001,"15min":2.1779999999999999},"last":1.304}]},{"osd":3,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.391,"15min":1.3979999999999999},"min":{"1min":0.95799999999999996,"5min":0.86299999999999999,"15min":0.73099999999999998},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.421,"15min":1.4770000000000001},"min":{"1min":0.998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.8440000000000001,"5min":1.8979999999999999,"15min":2.1459999999999999},"last":1.401}]},{"osd":4,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4450000000000001,"15min":1.415},"min":{"1min":1.048,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8140000000000001,"5min":2.0510000000000002,"15min":2.0859999999999999},"last":0.877},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.4850000000000001,"15min":1.474},"min":{"1min":1.01,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.484},"last":0.97799999999999998}]},{"osd":5,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.403,"15min":1.4299999999999999},"min":{"1min":0.90200000000000002,"5min":0.75,"15min":0.75},"max":{"1min":2.093,"5min":2.093,"15min":2.403},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.5960000000000001,"15min":1.5700000000000001},"min":{"1min":1.1259999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1320000000000001,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":0.92700000000000005}]},{"osd":7,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.5940000000000001,"15min":1.595},"min":{"1min":1.0209999999999999,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.1469999999999998,"5min":2.339,"15min":2.5870000000000002},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.5820000000000001,"15min":1.571},"min":{"1min":1.145,"5min":0.87,"15min":0.87},"max":{"1min":2.2370000000000001,"5min":2.2959999999999998,"15min":2.665},"last":0.81200000000000006}]}]},{"osd":5,"up_from":13,"seq":55834576037,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61024,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310816,"statfs":{"total":96636764160,"available":96574275584,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1499999999999999,"15min":1.202},"min":{"1min":0.748,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5740000000000001,"5min":1.98,"15min":1.98},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1910000000000001,"15min":1.2190000000000001},"min":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.74,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.4259999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.165,"15min":1.218},"min":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.5449999999999999,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":0.879},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2849999999999999,"15min":1.3100000000000001},"min":{"1min":0.78800000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.752,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":1.24}]},{"osd":2,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.254,"15min":1.2969999999999999},"min":{"1min":0.77100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.7050000000000001,"5min":2.008,"15min":2.0870000000000002},"last":1.8779999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.29},"min":{"1min":0.77600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.677,"5min":1.877,"15min":1.905},"last":1.014}]},{"osd":3,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.97699999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.694,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.4219999999999999,"15min":1.4259999999999999},"min":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.883,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":2.008}]},{"osd":4,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.329,"15min":1.3300000000000001},"min":{"1min":0.93600000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.857,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.8100000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.4910000000000001,"15min":1.4399999999999999},"min":{"1min":1.194,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1600000000000001,"5min":2.742,"15min":2.742},"last":1.6120000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5780000000000001,"15min":1.4890000000000001},"min":{"1min":1.036,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9450000000000001,"5min":13.234999999999999,"15min":13.234999999999999},"last":2.2679999999999998},{"interface":"front","average":{"1min":1.585,"5min":1.5449999999999999,"15min":1.4870000000000001},"min":{"1min":1.1830000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9770000000000001,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":1.3700000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.5649999999999999,"15min":1.548},"min":{"1min":1.208,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.0979999999999999,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.911},{"interface":"front","average":{"1min":1.669,"5min":1.609,"15min":1.6040000000000001},"min":{"1min":1.3260000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.02,"5min":2.351,"15min":2.351},"last":2.327}]}]},{"osd":4,"up_from":13,"seq":55834576037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52704,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319136,"statfs":{"total":96636764160,"available":96582795264,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.67800000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.3220000000000001,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4359999999999999,"15min":1.423},"min":{"1min":0.63600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.3609999999999998,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.6619999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2470000000000001,"15min":1.2849999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.2229999999999999,"5min":4.2229999999999999,"15min":4.2229999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":2.7799999999999998,"5min":1.635,"15min":1.371},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":28.907,"5min":28.907,"15min":28.907},"last":1.8460000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4810000000000001,"15min":1.335},"min":{"1min":0.70899999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":3.6120000000000001,"5min":17.175999999999998,"15min":17.175999999999998},"last":1.129},{"interface":"front","average":{"1min":1.496,"5min":1.444,"15min":1.391},"min":{"1min":0.59499999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.4080000000000004,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":1.355}]},{"osd":3,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.3260000000000001,"5min":1.5720000000000001,"15min":1.4159999999999999},"min":{"1min":0.60099999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":15.938000000000001,"5min":15.938000000000001,"15min":15.938000000000001},"last":1.538},{"interface":"front","average":{"1min":1.8069999999999999,"5min":1.5760000000000001,"15min":1.474},"min":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.787000000000001,"5min":10.787000000000001,"15min":10.787000000000001},"last":1.401}]},{"osd":5,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.591,"15min":1.4350000000000001},"min":{"1min":0.73199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":4.2939999999999996,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.458},{"interface":"front","average":{"1min":3.0209999999999999,"5min":1.8160000000000001,"15min":1.5289999999999999},"min":{"1min":0.84899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":31.347999999999999,"5min":31.347999999999999,"15min":31.347999999999999},"last":1.917}]},{"osd":6,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.016,"5min":1.7290000000000001,"15min":1.5389999999999999},"min":{"1min":0.73699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":13.885,"5min":13.885,"15min":13.885},"last":1.585},{"interface":"front","average":{"1min":2.117,"5min":1.833,"15min":1.6419999999999999},"min":{"1min":0.82499999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":13.446999999999999,"5min":13.446999999999999,"15min":13.446999999999999},"last":1.772}]},{"osd":7,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.528,"5min":1.821,"15min":1.611},"min":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":15.843999999999999,"5min":15.843999999999999,"15min":15.843999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.726,"15min":1.6659999999999999},"min":{"1min":0.82399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.968,"5min":5.609,"15min":5.609},"last":1.804}]}]},{"osd":3,"up_from":13,"seq":55834576039,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56860,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314980,"statfs":{"total":96636764160,"available":96578539520,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.96499999999999997},"min":{"1min":0.51600000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.323,"5min":1.6719999999999999,"15min":2.7599999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.046,"15min":1.1439999999999999},"min":{"1min":0.55300000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.72,"5min":1.72,"15min":4.5659999999999998},"last":0.56799999999999995}]},{"osd":1,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":1.0069999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1699999999999999,"15min":1.2609999999999999},"min":{"1min":0.58699999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.504,"5min":2.109,"15min":2.4889999999999999},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.032,"15min":1.1220000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.437},"max":{"1min":1.53,"5min":2.165,"15min":2.2250000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.079,"5min":1.119,"15min":1.1850000000000001},"min":{"1min":0.67800000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6200000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2170000000000001,"15min":1.268},"min":{"1min":0.93899999999999995,"5min":0.70899999999999996,"15min":0.67500000000000004},"max":{"1min":1.732,"5min":1.732,"15min":2.5379999999999998},"last":1.083},{"interface":"front","average":{"1min":1.355,"5min":1.3680000000000001,"15min":1.4410000000000001},"min":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":2.306,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.1950000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2989999999999999,"15min":1.355},"min":{"1min":0.79800000000000004,"5min":0.755,"15min":0.61799999999999999},"max":{"1min":1.9079999999999999,"5min":1.923,"15min":2.9740000000000002},"last":1.129},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.393,"15min":1.4299999999999999},"min":{"1min":0.86599999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.7330000000000001},"last":0.88100000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3740000000000001,"15min":1.405},"min":{"1min":1.0569999999999999,"5min":0.82299999999999995,"15min":0.59699999999999998},"max":{"1min":1.825,"5min":1.97,"15min":2.653},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4390000000000001,"15min":1.504},"min":{"1min":1.1140000000000001,"5min":0.872,"15min":0.57499999999999996},"max":{"1min":1.7749999999999999,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.1659999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.488,"15min":1.5529999999999999},"min":{"1min":0.91000000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.0590000000000002,"5min":2.3220000000000001,"15min":2.8650000000000002},"last":1.147},{"interface":"front","average":{"1min":1.389,"5min":1.4359999999999999,"15min":1.506},"min":{"1min":0.66700000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.8340000000000001,"5min":1.903,"15min":2.7749999999999999},"last":0.86299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52700,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319140,"statfs":{"total":96636764160,"available":96582799360,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85299999999999998,"15min":0.94199999999999995},"min":{"1min":0.38100000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.3029999999999999,"5min":1.601,"15min":1.948},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.97699999999999998,"15min":1.0369999999999999},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.482,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":1.02}]},{"osd":1,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.88700000000000001,"15min":0.95799999999999996},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.363,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.0309999999999999,"15min":1.147},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":1.4390000000000001,"5min":1.7789999999999999,"15min":1.9390000000000001},"last":1.5009999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0580000000000001,"15min":1.145},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.421,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":1.131},{"interface":"front","average":{"1min":1.004,"5min":1.123,"15min":1.2290000000000001},"min":{"1min":0.54600000000000004,"5min":0.47699999999999998,"15min":0.47599999999999998},"max":{"1min":1.54,"5min":1.6799999999999999,"15min":2.0670000000000002},"last":1.0880000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3280000000000001,"15min":1.409},"min":{"1min":0.78300000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.754,"5min":1.992,"15min":2.1789999999999998},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.256,"15min":1.298},"min":{"1min":0.78700000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.655,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.5589999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.298,"15min":1.365},"min":{"1min":0.75600000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.972,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.349},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3879999999999999,"15min":1.488},"min":{"1min":0.72499999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8939999999999999,"5min":2.0750000000000002,"15min":2.9390000000000001},"last":1.2969999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.306,"15min":1.371},"min":{"1min":0.72399999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.7370000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.667},{"interface":"front","average":{"1min":1.329,"5min":1.4059999999999999,"15min":1.4910000000000001},"min":{"1min":0.82699999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.891,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.6120000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.379,"15min":1.4470000000000001},"min":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8380000000000001,"5min":2.3969999999999998,"15min":2.4689999999999999},"last":1.774},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.454,"15min":1.538},"min":{"1min":0.90800000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.875,"5min":2.831,"15min":2.831},"last":1.7230000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52700,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319140,"statfs":{"total":96636764160,"available":96582799360,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.442,"5min":0.439,"15min":0.32100000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0489999999999999,"15min":1.177},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.41899999999999998},"max":{"1min":1.3939999999999999,"5min":1.7010000000000001,"15min":1.79},"last":1.246}]},{"osd":2,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.86099999999999999,"15min":0.93500000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.462,"15min":1.641},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.083,"15min":1.1299999999999999},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.372},"max":{"1min":1.4750000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.0800000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.151,"15min":1.2250000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.621,"5min":2.496,"15min":2.496},"last":1.3520000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.129},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6859999999999999,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":1.115}]},{"osd":4,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2529999999999999,"15min":1.333},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4339999999999999,"5min":2.73,"15min":2.8460000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2410000000000001,"15min":1.2889999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.744,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.8200000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3089999999999999,"15min":1.355},"min":{"1min":0.85599999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9910000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.518},"min":{"1min":0.98799999999999999,"5min":0.89600000000000002,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":2.601,"15min":2.601},"last":1.22}]},{"osd":6,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.4019999999999999,"15min":1.482},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.9710000000000001,"5min":2.883,"15min":2.9079999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.4039999999999999,"15min":1.474},"min":{"1min":0.752,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8939999999999999,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.754}]},{"osd":7,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.482,"15min":1.5329999999999999},"min":{"1min":0.85099999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1000000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4630000000000001,"15min":1.5429999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.0920000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.4730000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576038,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53284,"kb_used_data":4052,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318556,"statfs":{"total":96636764160,"available":96582201344,"internally_reserved":0,"allocated":4149248,"data_stored":2453208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.86799999999999999,"15min":0.94299999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3029999999999999,"5min":1.3859999999999999,"15min":1.581},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96399999999999997,"15min":1.0620000000000001},"min":{"1min":0.38500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.075}]},{"osd":2,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.03,"15min":1.1100000000000001},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.42199999999999999},"max":{"1min":4.1920000000000002,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.97999999999999998,"15min":1.052},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.4699999999999998,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.84799999999999998}]},{"osd":3,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.099,"15min":1.2030000000000001},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.425,"5min":1.633,"15min":2.0139999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.089,"15min":1.143},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.327,"5min":4.327,"15min":4.327},"last":1.363}]},{"osd":4,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.2250000000000001,"15min":1.3100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.5840000000000001,"5min":1.6599999999999999,"15min":2.4239999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.391,"5min":1.337,"15min":1.399},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.55700000000000005},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":1.462}]},{"osd":5,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.339,"15min":1.405},"min":{"1min":0.71299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.901,"5min":1.901,"15min":2.1309999999999998},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.4159999999999999,"15min":1.46},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.849,"5min":2.23,"15min":2.4900000000000002},"last":1.5189999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.413,"15min":1.4630000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":4.4260000000000002,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.554},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3720000000000001,"15min":1.476},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8240000000000001,"5min":1.9430000000000001,"15min":2.246},"last":1.1399999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4490000000000001,"15min":1.5249999999999999},"min":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.72799999999999998},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.488,"15min":1.5409999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1360000000000001,"5min":4.1360000000000001,"15min":4.1360000000000001},"last":1.712}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:16:21.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T07:16:21.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.996+0000 7fe0781b6640 1 Processor -- start 2023-10-08T07:16:21.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.996+0000 7fe0781b6640 1 -- start start 2023-10-08T07:16:21.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.998+0000 7fe0781b6640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fe07006d4c0 0x7fe070162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.998+0000 7fe0781b6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0700cb150 0x7fe0700cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:21.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.998+0000 7fe0781b6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0700ae140 con 0x7fe0700add60 2023-10-08T07:16:21.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.998+0000 7fe0781b6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe070161f10 con 0x7fe0700cb150 2023-10-08T07:16:21.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.998+0000 7fe0781b6640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0700cbb20 con 0x7fe07006d4c0 2023-10-08T07:16:21.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.998+0000 7fe076753640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe0700cb150 0x7fe0700cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:21.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.998+0000 7fe075f52640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe0700add60 0x7fe0700cc810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51656/0 (socket says 172.21.15.70:51656) 2023-10-08T07:16:21.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.998+0000 7fe075f52640 1 -- 172.21.15.70:0/4091212278 learned_addr learned my addr 172.21.15.70:0/4091212278 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:22.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.999+0000 7fe076753640 1 -- 172.21.15.70:0/4091212278 >> v2:172.21.15.181:3300/0 conn(0x7fe07006d4c0 msgr2=0x7fe070162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:22.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.999+0000 7fe075751640 1 --2- 172.21.15.70:0/4091212278 >> v2:172.21.15.181:3300/0 conn(0x7fe07006d4c0 0x7fe070162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:22.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.999+0000 7fe076753640 1 --2- 172.21.15.70:0/4091212278 >> v2:172.21.15.181:3300/0 conn(0x7fe07006d4c0 0x7fe070162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:22.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.999+0000 7fe076753640 1 -- 172.21.15.70:0/4091212278 >> v1:172.21.15.70:6789/0 conn(0x7fe0700add60 legacy=0x7fe0700cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:22.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.999+0000 7fe074f50640 1 -- 172.21.15.70:0/4091212278 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455896989 0 0) 0x7fe0700ae140 con 0x7fe0700add60 2023-10-08T07:16:22.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.999+0000 7fe076753640 1 -- 172.21.15.70:0/4091212278 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700ccf20 con 0x7fe0700cb150 2023-10-08T07:16:22.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:21.999+0000 7fe075751640 1 --2- 172.21.15.70:0/4091212278 >> v2:172.21.15.181:3300/0 conn(0x7fe07006d4c0 0x7fe070162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:22.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.000+0000 7fe076753640 1 --2- 172.21.15.70:0/4091212278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700cb150 0x7fe0700cb5e0 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7fe07006e560 tx=0x7fe06802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10955026332cefa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:22.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.000+0000 7fe074f50640 1 -- 172.21.15.70:0/4091212278 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06802cbe0 con 0x7fe0700cb150 2023-10-08T07:16:22.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.000+0000 7fe074f50640 1 -- 172.21.15.70:0/4091212278 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06802cd40 con 0x7fe0700cb150 2023-10-08T07:16:22.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.000+0000 7fe074f50640 1 -- 172.21.15.70:0/4091212278 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068034660 con 0x7fe0700cb150 2023-10-08T07:16:22.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.001+0000 7fe0781b6640 1 -- 172.21.15.70:0/4091212278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700cb150 msgr2=0x7fe0700cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:22.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.001+0000 7fe0781b6640 1 --2- 172.21.15.70:0/4091212278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700cb150 0x7fe0700cb5e0 secure :-1 s=READY pgs=3009 cs=0 l=1 rev1=1 crypto rx=0x7fe07006e560 tx=0x7fe06802be30 comp rx=0 tx=0).stop 2023-10-08T07:16:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.001+0000 7fe0781b6640 1 -- 172.21.15.70:0/4091212278 shutdown_connections 2023-10-08T07:16:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.001+0000 7fe0781b6640 1 --2- 172.21.15.70:0/4091212278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700cb150 0x7fe0700cb5e0 unknown :-1 s=CLOSED pgs=3009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.001+0000 7fe0781b6640 1 --2- 172.21.15.70:0/4091212278 >> v2:172.21.15.181:3300/0 conn(0x7fe07006d4c0 0x7fe070162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.001+0000 7fe0781b6640 1 -- 172.21.15.70:0/4091212278 >> 172.21.15.70:0/4091212278 conn(0x7fe07006a380 msgr2=0x7fe0700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.001+0000 7fe0781b6640 1 -- 172.21.15.70:0/4091212278 shutdown_connections 2023-10-08T07:16:22.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.001+0000 7fe0781b6640 1 -- 172.21.15.70:0/4091212278 wait complete. 2023-10-08T07:16:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.003+0000 7fe0781b6640 1 Processor -- start 2023-10-08T07:16:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.003+0000 7fe0781b6640 1 -- start start 2023-10-08T07:16:22.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe0781b6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe07006d4c0 0x7fe070156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:22.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe0781b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700add60 0x7fe07015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:22.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe0781b6640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0700cb150 0x7fe070158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:22.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe0781b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07015ce80 con 0x7fe0700add60 2023-10-08T07:16:22.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe0781b6640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07015d160 con 0x7fe0700cb150 2023-10-08T07:16:22.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe0781b6640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07015d440 con 0x7fe07006d4c0 2023-10-08T07:16:22.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe075751640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700add60 0x7fe07015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:22.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe075751640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700add60 0x7fe07015df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38842/0 (socket says 172.21.15.70:38842) 2023-10-08T07:16:22.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe075751640 1 -- 172.21.15.70:0/900229406 learned_addr learned my addr 172.21.15.70:0/900229406 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe075751640 1 -- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe07006d4c0 msgr2=0x7fe070156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe075751640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe07006d4c0 0x7fe070156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe075751640 1 -- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0700cb150 msgr2=0x7fe070158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe075751640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0700cb150 0x7fe070158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.004+0000 7fe075751640 1 -- 172.21.15.70:0/900229406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe068009a70 con 0x7fe0700add60 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.005+0000 7fe075751640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700add60 0x7fe07015df20 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c7f0 tx=0x7fe06000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.005+0000 7fe0667fc640 1 -- 172.21.15.70:0/900229406 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060007a30 con 0x7fe0700add60 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.005+0000 7fe0667fc640 1 -- 172.21.15.70:0/900229406 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe060007b90 con 0x7fe0700add60 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.005+0000 7fe0667fc640 1 -- 172.21.15.70:0/900229406 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06000ce40 con 0x7fe0700add60 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.005+0000 7fe0781b6640 1 -- 172.21.15.70:0/900229406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07015d720 con 0x7fe0700add60 2023-10-08T07:16:22.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.005+0000 7fe0781b6640 1 -- 172.21.15.70:0/900229406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07015f840 con 0x7fe0700add60 2023-10-08T07:16:22.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.006+0000 7fe0781b6640 1 -- 172.21.15.70:0/900229406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07015fa30 con 0x7fe0700add60 2023-10-08T07:16:22.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.007+0000 7fe0667fc640 1 -- 172.21.15.70:0/900229406 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe0600113f0 con 0x7fe0700add60 2023-10-08T07:16:22.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.008+0000 7fe0667fc640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe05406af90 0x7fe05406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:22.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.008+0000 7fe0667fc640 1 -- 172.21.15.70:0/900229406 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fe060089c60 con 0x7fe0700add60 2023-10-08T07:16:22.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.009+0000 7fe075f52640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe05406af90 0x7fe05406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:22.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.010+0000 7fe075f52640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe05406af90 0x7fe05406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe06c007fd0 tx=0x7fe06c007920 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:22.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.012+0000 7fe0667fc640 1 -- 172.21.15.70:0/900229406 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe07015fa30 con 0x7fe0700add60 2023-10-08T07:16:22.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.203+0000 7fe0781b6640 1 -- 172.21.15.70:0/900229406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe07015dc40 con 0x7fe0700add60 2023-10-08T07:16:22.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.205+0000 7fe0667fc640 1 -- 172.21.15.70:0/900229406 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v281) v1 ==== 74+0+14637 (secure 0 0 0) 0x7fe07015dc40 con 0x7fe0700add60 2023-10-08T07:16:22.206 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:22.206 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":281,"fsid":"cbcacf35-14e2-4581-b83a-6dd588939bbc","created":"2023-10-08T05:35:51.935090+0000","modified":"2023-10-08T07:16:16.343791+0000","last_up_change":"2023-10-08T05:36:03.842373+0000","last_in_change":"2023-10-08T05:35:55.450098+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":36,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T05:36:05.132285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2150863d-4f36-4fdb-becd-23d5e739c8bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6818","nonce":1284805295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6819","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6821","nonce":1284805295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6827","nonce":1284805295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":1284805295},{"type":"v1","addr":"172.21.15.70:6824","nonce":1284805295}]},"public_addr":"172.21.15.70:6818/1284805295","cluster_addr":"172.21.15.70:6821/1284805295","heartbeat_back_addr":"172.21.15.70:6827/1284805295","heartbeat_front_addr":"172.21.15.70:6824/1284805295","state":["exists","up"]},{"osd":1,"uuid":"8c8f42c7-3b3d-4fea-a994-593607500020","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6801","nonce":435902020}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6803","nonce":435902020}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6807","nonce":435902020}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":435902020},{"type":"v1","addr":"172.21.15.70:6805","nonce":435902020}]},"public_addr":"172.21.15.70:6801/435902020","cluster_addr":"172.21.15.70:6803/435902020","heartbeat_back_addr":"172.21.15.70:6807/435902020","heartbeat_front_addr":"172.21.15.70:6805/435902020","state":["exists","up"]},{"osd":2,"uuid":"f711dd07-9194-464e-9101-3794ef6132c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6817","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6820","nonce":3799730343}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6823","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6825","nonce":3799730343}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6831","nonce":3799730343}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":3799730343},{"type":"v1","addr":"172.21.15.70:6829","nonce":3799730343}]},"public_addr":"172.21.15.70:6820/3799730343","cluster_addr":"172.21.15.70:6825/3799730343","heartbeat_back_addr":"172.21.15.70:6831/3799730343","heartbeat_front_addr":"172.21.15.70:6829/3799730343","state":["exists","up"]},{"osd":3,"uuid":"786e1a64-be49-44e4-8b17-f3721d79b96f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6809","nonce":471550804}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6811","nonce":471550804}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6815","nonce":471550804}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":471550804},{"type":"v1","addr":"172.21.15.70:6813","nonce":471550804}]},"public_addr":"172.21.15.70:6809/471550804","cluster_addr":"172.21.15.70:6811/471550804","heartbeat_back_addr":"172.21.15.70:6815/471550804","heartbeat_front_addr":"172.21.15.70:6813/471550804","state":["exists","up"]},{"osd":4,"uuid":"cf10be13-ea48-4ddf-9c23-ec14cff3c09f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6801","nonce":2785838626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6808","nonce":2785838626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6815","nonce":2785838626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":2785838626},{"type":"v1","addr":"172.21.15.181:6813","nonce":2785838626}]},"public_addr":"172.21.15.181:6801/2785838626","cluster_addr":"172.21.15.181:6808/2785838626","heartbeat_back_addr":"172.21.15.181:6815/2785838626","heartbeat_front_addr":"172.21.15.181:6813/2785838626","state":["exists","up"]},{"osd":5,"uuid":"a9980cf7-cff6-4408-a3d7-09730c6943cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6824","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6825","nonce":1806853192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6826","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6827","nonce":1806853192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6830","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6831","nonce":1806853192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6828","nonce":1806853192},{"type":"v1","addr":"172.21.15.181:6829","nonce":1806853192}]},"public_addr":"172.21.15.181:6825/1806853192","cluster_addr":"172.21.15.181:6827/1806853192","heartbeat_back_addr":"172.21.15.181:6831/1806853192","heartbeat_front_addr":"172.21.15.181:6829/1806853192","state":["exists","up"]},{"osd":6,"uuid":"fa744b48-bf88-4b68-a372-697543a5a47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6803","nonce":4245641278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6806","nonce":4245641278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6812","nonce":4245641278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":4245641278},{"type":"v1","addr":"172.21.15.181:6809","nonce":4245641278}]},"public_addr":"172.21.15.181:6803/4245641278","cluster_addr":"172.21.15.181:6806/4245641278","heartbeat_back_addr":"172.21.15.181:6812/4245641278","heartbeat_front_addr":"172.21.15.181:6809/4245641278","state":["exists","up"]},{"osd":7,"uuid":"2b6cdf62-a741-46b3-8df2-3f2098d41613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":274,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6816","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6817","nonce":1658798817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6818","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6819","nonce":1658798817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6822","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6823","nonce":1658798817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6820","nonce":1658798817},{"type":"v1","addr":"172.21.15.181:6821","nonce":1658798817}]},"public_addr":"172.21.15.181:6817/1658798817","cluster_addr":"172.21.15.181:6819/1658798817","heartbeat_back_addr":"172.21.15.181:6823/1658798817","heartbeat_front_addr":"172.21.15.181:6821/1658798817","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.827746+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.725957+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.352028+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.807269+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.842830+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.840526+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.815603+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T05:36:00.323797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6832/753129368":"2023-10-09T07:16:12.704861+0000","172.21.15.181:0/3049540535":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/220528716":"2023-10-09T07:16:08.864584+0000","172.21.15.181:6833/1127370729":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/2060656952":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1823452952":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/3814802416":"2023-10-09T07:16:00.570875+0000","172.21.15.70:6834/753129368":"2023-10-09T07:16:12.704861+0000","172.21.15.181:0/3705279960":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6833/440573135":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6832/440573135":"2023-10-09T07:16:00.570875+0000","172.21.15.181:6832/1127370729":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1545538679":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/1740402122":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1151602740":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2253796914":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/3307173154":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2296566554":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/2564762528":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/606980":"2023-10-09T07:16:00.570875+0000","172.21.15.181:0/796435327":"2023-10-09T07:16:08.864584+0000","172.21.15.181:0/1513601554":"2023-10-08T08:15:42.823407+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":103,"length":1},{"begin":221,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":2}]},{"pool":20,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T07:16:22.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.207+0000 7fe0781b6640 1 -- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe05406af90 msgr2=0x7fe05406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.207+0000 7fe0781b6640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe05406af90 0x7fe05406d450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe06c007fd0 tx=0x7fe06c007920 comp rx=0 tx=0).stop 2023-10-08T07:16:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.208+0000 7fe0781b6640 1 -- 172.21.15.70:0/900229406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700add60 msgr2=0x7fe07015df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.208+0000 7fe0781b6640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700add60 0x7fe07015df20 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7fe06000c7f0 tx=0x7fe06000cbf0 comp rx=0 tx=0).stop 2023-10-08T07:16:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.208+0000 7fe0781b6640 1 -- 172.21.15.70:0/900229406 shutdown_connections 2023-10-08T07:16:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.208+0000 7fe0781b6640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe05406af90 0x7fe05406d450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.208+0000 7fe0781b6640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe0700cb150 0x7fe070158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.208+0000 7fe0781b6640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0700add60 0x7fe07015df20 unknown :-1 s=CLOSED pgs=3010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.208+0000 7fe0781b6640 1 --2- 172.21.15.70:0/900229406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe07006d4c0 0x7fe070156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.208+0000 7fe0781b6640 1 -- 172.21.15.70:0/900229406 >> 172.21.15.70:0/900229406 conn(0x7fe07006a380 msgr2=0x7fe0700b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:22.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.208+0000 7fe0781b6640 1 -- 172.21.15.70:0/900229406 shutdown_connections 2023-10-08T07:16:22.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:22.208+0000 7fe0781b6640 1 -- 172.21.15.70:0/900229406 wait complete. 2023-10-08T07:16:23.228 INFO:tasks.ceph:Scrubbing osd.0 2023-10-08T07:16:23.278 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-08T07:16:23.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.509+0000 7f850bca2640 1 Processor -- start 2023-10-08T07:16:23.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.509+0000 7f850bca2640 1 -- start start 2023-10-08T07:16:23.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.510+0000 7f850bca2640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f85040afbe0 0x7f85040b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.510+0000 7f850bca2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f85040cb130 0x7f85040cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.510+0000 7f850bca2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85040cbad0 con 0x7f850406dfc0 2023-10-08T07:16:23.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.510+0000 7f850bca2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85040b2620 con 0x7f85040cb130 2023-10-08T07:16:23.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.510+0000 7f850bca2640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85040b2060 con 0x7f85040afbe0 2023-10-08T07:16:23.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.511+0000 7f850a23f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f85040cb130 0x7f85040cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.511+0000 7f850a23f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f85040cb130 0x7f85040cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38858/0 (socket says 172.21.15.70:38858) 2023-10-08T07:16:23.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.511+0000 7f850a23f640 1 -- 172.21.15.70:0/675249358 learned_addr learned my addr 172.21.15.70:0/675249358 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:23.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.511+0000 7f8509a3e640 1 --2- 172.21.15.70:0/675249358 >> v2:172.21.15.181:3300/0 conn(0x7f85040afbe0 0x7f85040b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.511+0000 7f850a23f640 1 -- 172.21.15.70:0/675249358 >> v2:172.21.15.181:3300/0 conn(0x7f85040afbe0 msgr2=0x7f85040b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:23.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.511+0000 7f850a23f640 1 --2- 172.21.15.70:0/675249358 >> v2:172.21.15.181:3300/0 conn(0x7f85040afbe0 0x7f85040b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.511+0000 7f850a23f640 1 -- 172.21.15.70:0/675249358 >> v1:172.21.15.181:6789/0 conn(0x7f850406dfc0 legacy=0x7f8504070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:23.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.511+0000 7f850a23f640 1 -- 172.21.15.70:0/675249358 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85040b2340 con 0x7f85040cb130 2023-10-08T07:16:23.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.512+0000 7f850a23f640 1 --2- 172.21.15.70:0/675249358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85040cb130 0x7f85040cb590 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7f850406e640 tx=0x7f84fc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5bc21e1cde85b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:23.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.513+0000 7f8508a3c640 1 -- 172.21.15.70:0/675249358 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc02ca80 con 0x7f85040cb130 2023-10-08T07:16:23.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.513+0000 7f8508a3c640 1 -- 172.21.15.70:0/675249358 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc02cbe0 con 0x7f85040cb130 2023-10-08T07:16:23.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.513+0000 7f8508a3c640 1 -- 172.21.15.70:0/675249358 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0343f0 con 0x7f85040cb130 2023-10-08T07:16:23.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.513+0000 7f850bca2640 1 -- 172.21.15.70:0/675249358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85040cb130 msgr2=0x7f85040cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:23.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.513+0000 7f850bca2640 1 --2- 172.21.15.70:0/675249358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85040cb130 0x7f85040cb590 secure :-1 s=READY pgs=3011 cs=0 l=1 rev1=1 crypto rx=0x7f850406e640 tx=0x7f84fc02bee0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.514+0000 7f850bca2640 1 -- 172.21.15.70:0/675249358 shutdown_connections 2023-10-08T07:16:23.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.514+0000 7f850bca2640 1 --2- 172.21.15.70:0/675249358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85040cb130 0x7f85040cb590 unknown :-1 s=CLOSED pgs=3011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.514+0000 7f850bca2640 1 --2- 172.21.15.70:0/675249358 >> v2:172.21.15.181:3300/0 conn(0x7f85040afbe0 0x7f85040b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.514+0000 7f850bca2640 1 -- 172.21.15.70:0/675249358 >> 172.21.15.70:0/675249358 conn(0x7f850406a5d0 msgr2=0x7f85040b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:23.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.514+0000 7f850bca2640 1 -- 172.21.15.70:0/675249358 shutdown_connections 2023-10-08T07:16:23.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.514+0000 7f850bca2640 1 -- 172.21.15.70:0/675249358 wait complete. 2023-10-08T07:16:23.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.515+0000 7f850bca2640 1 Processor -- start 2023-10-08T07:16:23.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.516+0000 7f850bca2640 1 -- start start 2023-10-08T07:16:23.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850bca2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f850406dfc0 0x7f85040cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850bca2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85040afbe0 0x7f850414db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850bca2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85040cb130 0x7f85040ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850bca2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85040cdbd0 con 0x7f85040cb130 2023-10-08T07:16:23.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850bca2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850414e060 con 0x7f850406dfc0 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850bca2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85040cde00 con 0x7f85040afbe0 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850a23f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85040cb130 0x7f85040ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850a23f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85040cb130 0x7f85040ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38870/0 (socket says 172.21.15.70:38870) 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850a23f640 1 -- 172.21.15.70:0/2485768406 learned_addr learned my addr 172.21.15.70:0/2485768406 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850a23f640 1 -- 172.21.15.70:0/2485768406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85040afbe0 msgr2=0x7f850414db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850a23f640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f85040afbe0 0x7f850414db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f8509a3e640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f850406dfc0 0x7f85040cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850a23f640 1 -- 172.21.15.70:0/2485768406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f850406dfc0 msgr2=0x7f85040cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850a23f640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f850406dfc0 0x7f85040cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.517+0000 7f850a23f640 1 -- 172.21.15.70:0/2485768406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc009b30 con 0x7f85040cb130 2023-10-08T07:16:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.518+0000 7f850a23f640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85040cb130 0x7f85040ceea0 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7f84fc036040 tx=0x7f84fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:23.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.518+0000 7f84fa7fc640 1 -- 172.21.15.70:0/2485768406 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc009510 con 0x7f85040cb130 2023-10-08T07:16:23.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.518+0000 7f84fa7fc640 1 -- 172.21.15.70:0/2485768406 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc03e450 con 0x7f85040cb130 2023-10-08T07:16:23.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.518+0000 7f84fa7fc640 1 -- 172.21.15.70:0/2485768406 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc03e7e0 con 0x7f85040cb130 2023-10-08T07:16:23.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.518+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85040ce0e0 con 0x7f85040cb130 2023-10-08T07:16:23.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.518+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85040ce6a0 con 0x7f85040cb130 2023-10-08T07:16:23.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.519+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8504070630 con 0x7f85040cb130 2023-10-08T07:16:23.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.520+0000 7f84fa7fc640 1 -- 172.21.15.70:0/2485768406 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f84fc02cd50 con 0x7f85040cb130 2023-10-08T07:16:23.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.521+0000 7f84fa7fc640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f84e806af90 0x7f84e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.521+0000 7f84fa7fc640 1 -- 172.21.15.70:0/2485768406 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f84fc0acbb0 con 0x7f85040cb130 2023-10-08T07:16:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.522+0000 7f84fa7fc640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] conn(0x7f84e8071a80 0x7f84e8073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.522+0000 7f84fa7fc640 1 -- 172.21.15.70:0/2485768406 --> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84e80745b0 con 0x7f84e8071a80 2023-10-08T07:16:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.522+0000 7f84fa7fc640 1 -- 172.21.15.70:0/2485768406 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=281) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84fc0acf20 con 0x7f85040cb130 2023-10-08T07:16:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.522+0000 7f850923d640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] conn(0x7f84e8071a80 0x7f84e8073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.522+0000 7f8509a3e640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f84e806af90 0x7f84e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.523+0000 7f850923d640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] conn(0x7f84e8071a80 0x7f84e8073ee0 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:23.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.523+0000 7f8509a3e640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f84e806af90 0x7f84e806d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8500006fd0 tx=0x7f8500008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:23.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.523+0000 7f84fa7fc640 1 -- 172.21.15.70:0/2485768406 <== osd.0 v2:172.21.15.70:6816/1284805295 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f84e80745b0 con 0x7f84e8071a80 2023-10-08T07:16:23.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.537+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 --> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8504070350 con 0x7f84e8071a80 2023-10-08T07:16:23.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.538+0000 7f84fa7fc640 1 -- 172.21.15.70:0/2485768406 <== osd.0 v2:172.21.15.70:6816/1284805295 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f84f40097a0 con 0x7f84e8071a80 2023-10-08T07:16:23.539 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:16:23.539 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T07:16:23.539 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:16:23.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.538+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 >> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] conn(0x7f84e8071a80 msgr2=0x7f84e8073ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:23.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.538+0000 7f850bca2640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.70:6816/1284805295,v1:172.21.15.70:6818/1284805295] conn(0x7f84e8071a80 0x7f84e8073ee0 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.539+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f84e806af90 msgr2=0x7f84e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:23.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.539+0000 7f850bca2640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f84e806af90 0x7f84e806d450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8500006fd0 tx=0x7f8500008040 comp rx=0 tx=0).stop 2023-10-08T07:16:23.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.539+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85040cb130 msgr2=0x7f85040ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:23.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.539+0000 7f850bca2640 1 --2- 172.21.15.70:0/2485768406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85040cb130 0x7f85040ceea0 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7f84fc036040 tx=0x7f84fc004a40 comp rx=0 tx=0).stop 2023-10-08T07:16:23.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.539+0000 7f850a23f640 1 -- 172.21.15.70:0/2485768406 reap_dead start 2023-10-08T07:16:23.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.539+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 shutdown_connections 2023-10-08T07:16:23.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.539+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 >> 172.21.15.70:0/2485768406 conn(0x7f850406a5d0 msgr2=0x7f85040c8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:23.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.539+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 shutdown_connections 2023-10-08T07:16:23.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.539+0000 7f850bca2640 1 -- 172.21.15.70:0/2485768406 wait complete. 2023-10-08T07:16:23.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-08T07:16:23.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.766+0000 7fa37df1d640 1 Processor -- start 2023-10-08T07:16:23.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.766+0000 7fa37df1d640 1 -- start start 2023-10-08T07:16:23.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.767+0000 7fa37df1d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa3781603a0 0x7fa3780cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.767+0000 7fa37df1d640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fa378071d50 0x7fa378160910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.767+0000 7fa37df1d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3780c9ff0 con 0x7fa3780cdff0 2023-10-08T07:16:23.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.767+0000 7fa37df1d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3780c3100 con 0x7fa3781603a0 2023-10-08T07:16:23.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.767+0000 7fa37df1d640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3780ce480 con 0x7fa378071d50 2023-10-08T07:16:23.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.768+0000 7fa3777fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa3781603a0 0x7fa3780cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.768+0000 7fa376ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa3780cdff0 0x7fa378071640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59188/0 (socket says 172.21.15.70:59188) 2023-10-08T07:16:23.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.768+0000 7fa376ffd640 1 -- 172.21.15.70:0/1073374568 learned_addr learned my addr 172.21.15.70:0/1073374568 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:23.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.768+0000 7fa377fff640 1 --2- 172.21.15.70:0/1073374568 >> v2:172.21.15.181:3300/0 conn(0x7fa378071d50 0x7fa378160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.768+0000 7fa3767fc640 1 -- 172.21.15.70:0/1073374568 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695552558 0 0) 0x7fa3780c9ff0 con 0x7fa3780cdff0 2023-10-08T07:16:23.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.768+0000 7fa3767fc640 1 -- 172.21.15.70:0/1073374568 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa364003520 con 0x7fa3780cdff0 2023-10-08T07:16:23.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.768+0000 7fa3777fe640 1 -- 172.21.15.70:0/1073374568 >> v2:172.21.15.181:3300/0 conn(0x7fa378071d50 msgr2=0x7fa378160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:23.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.768+0000 7fa3777fe640 1 --2- 172.21.15.70:0/1073374568 >> v2:172.21.15.181:3300/0 conn(0x7fa378071d50 0x7fa378160910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.768+0000 7fa3777fe640 1 -- 172.21.15.70:0/1073374568 >> v1:172.21.15.70:6789/0 conn(0x7fa3780cdff0 legacy=0x7fa378071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:23.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.768+0000 7fa3777fe640 1 -- 172.21.15.70:0/1073374568 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa378072290 con 0x7fa3781603a0 2023-10-08T07:16:23.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.769+0000 7fa3777fe640 1 --2- 172.21.15.70:0/1073374568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 0x7fa3780cdab0 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7fa3780713c0 tx=0x7fa36802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cfd06723b728f1d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:23.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.770+0000 7fa3767fc640 1 -- 172.21.15.70:0/1073374568 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36802cc80 con 0x7fa3781603a0 2023-10-08T07:16:23.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.770+0000 7fa3767fc640 1 -- 172.21.15.70:0/1073374568 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36802cde0 con 0x7fa3781603a0 2023-10-08T07:16:23.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.770+0000 7fa3767fc640 1 -- 172.21.15.70:0/1073374568 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368034660 con 0x7fa3781603a0 2023-10-08T07:16:23.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.770+0000 7fa37df1d640 1 -- 172.21.15.70:0/1073374568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 msgr2=0x7fa3780cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:23.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.770+0000 7fa37df1d640 1 --2- 172.21.15.70:0/1073374568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 0x7fa3780cdab0 secure :-1 s=READY pgs=3014 cs=0 l=1 rev1=1 crypto rx=0x7fa3780713c0 tx=0x7fa36802bed0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.770+0000 7fa37df1d640 1 -- 172.21.15.70:0/1073374568 shutdown_connections 2023-10-08T07:16:23.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.770+0000 7fa37df1d640 1 --2- 172.21.15.70:0/1073374568 >> v2:172.21.15.181:3300/0 conn(0x7fa378071d50 0x7fa378160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.770+0000 7fa37df1d640 1 --2- 172.21.15.70:0/1073374568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 0x7fa3780cdab0 unknown :-1 s=CLOSED pgs=3014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.770+0000 7fa37df1d640 1 -- 172.21.15.70:0/1073374568 >> 172.21.15.70:0/1073374568 conn(0x7fa3780cc2e0 msgr2=0x7fa3780cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:23.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.771+0000 7fa37df1d640 1 -- 172.21.15.70:0/1073374568 shutdown_connections 2023-10-08T07:16:23.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.771+0000 7fa37df1d640 1 -- 172.21.15.70:0/1073374568 wait complete. 2023-10-08T07:16:23.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.772+0000 7fa37df1d640 1 Processor -- start 2023-10-08T07:16:23.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.772+0000 7fa37df1d640 1 -- start start 2023-10-08T07:16:23.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.773+0000 7fa37df1d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa378071d50 0x7fa37814cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.773+0000 7fa37df1d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3780cdff0 0x7fa3780c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.773+0000 7fa37df1d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 0x7fa3780c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.773+0000 7fa37df1d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780ad700 con 0x7fa3781603a0 2023-10-08T07:16:23.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.773+0000 7fa37df1d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780ad9e0 con 0x7fa378071d50 2023-10-08T07:16:23.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa37df1d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3780adcc0 con 0x7fa3780cdff0 2023-10-08T07:16:23.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa377fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 0x7fa3780c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa377fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 0x7fa3780c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36604/0 (socket says 172.21.15.70:36604) 2023-10-08T07:16:23.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa377fff640 1 -- 172.21.15.70:0/2148251054 learned_addr learned my addr 172.21.15.70:0/2148251054 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa3777fe640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa378071d50 0x7fa37814cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa377fff640 1 -- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3780cdff0 msgr2=0x7fa3780c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa377fff640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3780cdff0 0x7fa3780c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa377fff640 1 -- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa378071d50 msgr2=0x7fa37814cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa377fff640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa378071d50 0x7fa37814cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa377fff640 1 -- 172.21.15.70:0/2148251054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa368009a70 con 0x7fa3781603a0 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.774+0000 7fa377fff640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 0x7fa3780c7080 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7fa37000d800 tx=0x7fa37000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.775+0000 7fa36ffff640 1 -- 172.21.15.70:0/2148251054 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37000b6c0 con 0x7fa3781603a0 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.775+0000 7fa36ffff640 1 -- 172.21.15.70:0/2148251054 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37000b840 con 0x7fa3781603a0 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.775+0000 7fa36ffff640 1 -- 172.21.15.70:0/2148251054 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37000bbf0 con 0x7fa3781603a0 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.775+0000 7fa37df1d640 1 -- 172.21.15.70:0/2148251054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3780adfa0 con 0x7fa3781603a0 2023-10-08T07:16:23.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.775+0000 7fa37df1d640 1 -- 172.21.15.70:0/2148251054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3780ac990 con 0x7fa3781603a0 2023-10-08T07:16:23.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.776+0000 7fa37df1d640 1 -- 172.21.15.70:0/2148251054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3780b3050 con 0x7fa3781603a0 2023-10-08T07:16:23.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.777+0000 7fa36ffff640 1 -- 172.21.15.70:0/2148251054 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fa37000bd50 con 0x7fa3781603a0 2023-10-08T07:16:23.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.778+0000 7fa36ffff640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fa35806af90 0x7fa35806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:23.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.778+0000 7fa36ffff640 1 -- 172.21.15.70:0/2148251054 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fa370089ee0 con 0x7fa3781603a0 2023-10-08T07:16:23.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.779+0000 7fa3777fe640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fa35806af90 0x7fa35806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:23.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.780+0000 7fa3777fe640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fa35806af90 0x7fa35806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa368036000 tx=0x7fa368030000 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:23.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.782+0000 7fa36ffff640 1 -- 172.21.15.70:0/2148251054 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3780b3050 con 0x7fa3781603a0 2023-10-08T07:16:23.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.970+0000 7fa37df1d640 1 -- 172.21.15.70:0/2148251054 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3780b12e0 con 0x7fa35806af90 2023-10-08T07:16:24.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.971+0000 7fa36ffff640 1 -- 172.21.15.70:0/2148251054 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa3680344c0 con 0x7fa35806af90 2023-10-08T07:16:24.050 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 0 to deep-scrub 2023-10-08T07:16:24.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.973+0000 7fa37df1d640 1 -- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fa35806af90 msgr2=0x7fa35806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.973+0000 7fa37df1d640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fa35806af90 0x7fa35806d450 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa368036000 tx=0x7fa368030000 comp rx=0 tx=0).stop 2023-10-08T07:16:24.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.973+0000 7fa37df1d640 1 -- 172.21.15.70:0/2148251054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 msgr2=0x7fa3780c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.973+0000 7fa37df1d640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 0x7fa3780c7080 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7fa37000d800 tx=0x7fa37000dcc0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.973+0000 7fa37df1d640 1 -- 172.21.15.70:0/2148251054 shutdown_connections 2023-10-08T07:16:24.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.973+0000 7fa37df1d640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fa35806af90 0x7fa35806d450 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.973+0000 7fa37df1d640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3781603a0 0x7fa3780c7080 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.973+0000 7fa37df1d640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fa3780cdff0 0x7fa3780c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.973+0000 7fa37df1d640 1 --2- 172.21.15.70:0/2148251054 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fa378071d50 0x7fa37814cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.973+0000 7fa37df1d640 1 -- 172.21.15.70:0/2148251054 >> 172.21.15.70:0/2148251054 conn(0x7fa3780cc2e0 msgr2=0x7fa3780cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:24.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.974+0000 7fa37df1d640 1 -- 172.21.15.70:0/2148251054 shutdown_connections 2023-10-08T07:16:24.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:23.974+0000 7fa37df1d640 1 -- 172.21.15.70:0/2148251054 wait complete. 2023-10-08T07:16:24.052 INFO:tasks.ceph:Scrubbing osd.1 2023-10-08T07:16:24.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-08T07:16:24.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.277+0000 7f2307fff640 1 Processor -- start 2023-10-08T07:16:24.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.278+0000 7f2307fff640 1 -- start start 2023-10-08T07:16:24.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.279+0000 7f2307fff640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f230815cf30 0x7f23080d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.279+0000 7f2307fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23080cd520 con 0x7f23080cf2b0 2023-10-08T07:16:24.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.279+0000 7f2307fff640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23080d1910 con 0x7f23080cf690 2023-10-08T07:16:24.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.279+0000 7f2307fff640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23080ce080 con 0x7f230815cf30 2023-10-08T07:16:24.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.280+0000 7f2306ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f23080cf2b0 0x7f23080d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59198/0 (socket says 172.21.15.70:59198) 2023-10-08T07:16:24.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.280+0000 7f2306ffd640 1 -- 172.21.15.70:0/1309549340 learned_addr learned my addr 172.21.15.70:0/1309549340 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:24.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.280+0000 7f23077fe640 1 --2- 172.21.15.70:0/1309549340 >> v2:172.21.15.181:3300/0 conn(0x7f230815cf30 0x7f23080d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.280+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961485952 0 0) 0x7f23080cd520 con 0x7f23080cf2b0 2023-10-08T07:16:24.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.280+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22f4003520 con 0x7f23080cf2b0 2023-10-08T07:16:24.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.280+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2060292702 0 0) 0x7f22f4003520 con 0x7f23080cf2b0 2023-10-08T07:16:24.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.280+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 >> v2:172.21.15.181:3300/0 conn(0x7f230815cf30 msgr2=0x7f23080d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:24.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 --2- 172.21.15.70:0/1309549340 >> v2:172.21.15.181:3300/0 conn(0x7f230815cf30 0x7f23080d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 >> v1:172.21.15.181:6789/0 conn(0x7f23080cf690 legacy=0x7f230815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23080cd750 con 0x7f23080cf2b0 2023-10-08T07:16:24.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f22f80019f0 con 0x7f23080cf2b0 2023-10-08T07:16:24.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 >> v1:172.21.15.70:6789/0 conn(0x7f23080cf2b0 legacy=0x7f23080d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 --2- 172.21.15.70:0/1309549340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f40039c0 0x7f22f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 --2- 172.21.15.70:0/1309549340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22f40047c0 0x7f22f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 --2- 172.21.15.70:0/1309549340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22f40051d0 0x7f22f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23080ce080 con 0x7f22f40039c0 2023-10-08T07:16:24.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f400c120 con 0x7f22f40047c0 2023-10-08T07:16:24.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f400c2e0 con 0x7f22f40051d0 2023-10-08T07:16:24.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2306ffd640 1 --2- 172.21.15.70:0/1309549340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f40039c0 0x7f22f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f23067fc640 1 --2- 172.21.15.70:0/1309549340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22f40047c0 0x7f22f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2306ffd640 1 -- 172.21.15.70:0/1309549340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22f40051d0 msgr2=0x7f22f400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:24.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2306ffd640 1 --2- 172.21.15.70:0/1309549340 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22f40051d0 0x7f22f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2306ffd640 1 -- 172.21.15.70:0/1309549340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22f40047c0 msgr2=0x7f22f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:24.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2306ffd640 1 --2- 172.21.15.70:0/1309549340 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f22f40047c0 0x7f22f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f2306ffd640 1 -- 172.21.15.70:0/1309549340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4003520 con 0x7f22f40039c0 2023-10-08T07:16:24.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.281+0000 7f23077fe640 1 -- 172.21.15.70:0/1309549340 reap_dead start 2023-10-08T07:16:24.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.282+0000 7f2306ffd640 1 --2- 172.21.15.70:0/1309549340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f40039c0 0x7f22f4003de0 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7f22f8002230 tx=0x7f22f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca44e7e5babd849 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:24.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.283+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8034630 con 0x7f22f40039c0 2023-10-08T07:16:24.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.283+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f802f070 con 0x7f22f40039c0 2023-10-08T07:16:24.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.283+0000 7f2305ffb640 1 -- 172.21.15.70:0/1309549340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8034a50 con 0x7f22f40039c0 2023-10-08T07:16:24.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.283+0000 7f2307fff640 1 -- 172.21.15.70:0/1309549340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f40039c0 msgr2=0x7f22f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.283+0000 7f2307fff640 1 --2- 172.21.15.70:0/1309549340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f40039c0 0x7f22f4003de0 secure :-1 s=READY pgs=3017 cs=0 l=1 rev1=1 crypto rx=0x7f22f8002230 tx=0x7f22f40057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.283+0000 7f2307fff640 1 -- 172.21.15.70:0/1309549340 shutdown_connections 2023-10-08T07:16:24.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.283+0000 7f2307fff640 1 --2- 172.21.15.70:0/1309549340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22f40039c0 0x7f22f4003de0 unknown :-1 s=CLOSED pgs=3017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.283+0000 7f2307fff640 1 -- 172.21.15.70:0/1309549340 >> 172.21.15.70:0/1309549340 conn(0x7f230809bc40 msgr2=0x7f23080d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:24.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.284+0000 7f2307fff640 1 -- 172.21.15.70:0/1309549340 shutdown_connections 2023-10-08T07:16:24.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.284+0000 7f2307fff640 1 -- 172.21.15.70:0/1309549340 wait complete. 2023-10-08T07:16:24.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.285+0000 7f2307fff640 1 Processor -- start 2023-10-08T07:16:24.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.285+0000 7f2307fff640 1 -- start start 2023-10-08T07:16:24.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.286+0000 7f2307fff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22f40039c0 0x7f23080b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.286+0000 7f2307fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23080b4030 0x7f23081546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.286+0000 7f2307fff640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23080b44b0 0x7f2308154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.286+0000 7f2307fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23080b74f0 con 0x7f23080b4030 2023-10-08T07:16:24.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.286+0000 7f2307fff640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23080b8350 con 0x7f23080b44b0 2023-10-08T07:16:24.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.286+0000 7f2307fff640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23080b3640 con 0x7f22f40039c0 2023-10-08T07:16:24.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.287+0000 7f23067fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23080b4030 0x7f23081546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.287+0000 7f23067fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23080b4030 0x7f23081546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36626/0 (socket says 172.21.15.70:36626) 2023-10-08T07:16:24.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.287+0000 7f23067fc640 1 -- 172.21.15.70:0/864402128 learned_addr learned my addr 172.21.15.70:0/864402128 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:24.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.287+0000 7f23067fc640 1 -- 172.21.15.70:0/864402128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22f40039c0 msgr2=0x7f23080b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.287+0000 7f23067fc640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f22f40039c0 0x7f23080b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.287+0000 7f23067fc640 1 -- 172.21.15.70:0/864402128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23080b44b0 msgr2=0x7f2308154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.287+0000 7f23067fc640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f23080b44b0 0x7f2308154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.287+0000 7f23067fc640 1 -- 172.21.15.70:0/864402128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23080b7740 con 0x7f23080b4030 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.287+0000 7f23067fc640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23080b4030 0x7f23081546f0 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 crypto rx=0x7f22f000e4b0 tx=0x7f22f000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.288+0000 7f22ff7fe640 1 -- 172.21.15.70:0/864402128 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f00052e0 con 0x7f23080b4030 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.288+0000 7f22ff7fe640 1 -- 172.21.15.70:0/864402128 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f0005440 con 0x7f23080b4030 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.288+0000 7f22ff7fe640 1 -- 172.21.15.70:0/864402128 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0003830 con 0x7f23080b4030 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.288+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23080b7a20 con 0x7f23080b4030 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.288+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23080b7fe0 con 0x7f23080b4030 2023-10-08T07:16:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.289+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f23080cdd10 con 0x7f23080b4030 2023-10-08T07:16:24.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.290+0000 7f22ff7fe640 1 -- 172.21.15.70:0/864402128 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f22f0003990 con 0x7f23080b4030 2023-10-08T07:16:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.291+0000 7f22ff7fe640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f22e806af90 0x7f22e806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.291+0000 7f22ff7fe640 1 -- 172.21.15.70:0/864402128 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f22f00896d0 con 0x7f23080b4030 2023-10-08T07:16:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.291+0000 7f22ff7fe640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] conn(0x7f22e8071a80 0x7f22e8073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.291+0000 7f22ff7fe640 1 -- 172.21.15.70:0/864402128 --> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22e80745b0 con 0x7f22e8071a80 2023-10-08T07:16:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.291+0000 7f22ff7fe640 1 -- 172.21.15.70:0/864402128 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=281) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22f008bc30 con 0x7f23080b4030 2023-10-08T07:16:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.291+0000 7f23077fe640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] conn(0x7f22e8071a80 0x7f22e8073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.292+0000 7f2306ffd640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f22e806af90 0x7f22e806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.292+0000 7f23077fe640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] conn(0x7f22e8071a80 0x7f22e8073ee0 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:24.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.293+0000 7f2306ffd640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f22e806af90 0x7f22e806d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f22f8034000 tx=0x7f22f8009030 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:24.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.293+0000 7f22ff7fe640 1 -- 172.21.15.70:0/864402128 <== osd.1 v2:172.21.15.70:6800/435902020 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f22e80745b0 con 0x7f22e8071a80 2023-10-08T07:16:24.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.307+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 --> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f23080cda30 con 0x7f22e8071a80 2023-10-08T07:16:24.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.307+0000 7f22ff7fe640 1 -- 172.21.15.70:0/864402128 <== osd.1 v2:172.21.15.70:6800/435902020 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2300004040 con 0x7f22e8071a80 2023-10-08T07:16:24.309 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:16:24.309 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T07:16:24.309 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:16:24.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 >> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] conn(0x7f22e8071a80 msgr2=0x7f22e8073ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f2307fff640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.70:6800/435902020,v1:172.21.15.70:6801/435902020] conn(0x7f22e8071a80 0x7f22e8073ee0 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f22e806af90 msgr2=0x7f22e806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f2307fff640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f22e806af90 0x7f22e806d450 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f22f8034000 tx=0x7f22f8009030 comp rx=0 tx=0).stop 2023-10-08T07:16:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23080b4030 msgr2=0x7f23081546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f2307fff640 1 --2- 172.21.15.70:0/864402128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23080b4030 0x7f23081546f0 secure :-1 s=READY pgs=3018 cs=0 l=1 rev1=1 crypto rx=0x7f22f000e4b0 tx=0x7f22f000e970 comp rx=0 tx=0).stop 2023-10-08T07:16:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f23077fe640 1 -- 172.21.15.70:0/864402128 reap_dead start 2023-10-08T07:16:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 shutdown_connections 2023-10-08T07:16:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 >> 172.21.15.70:0/864402128 conn(0x7f230809bc40 msgr2=0x7f2308073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 shutdown_connections 2023-10-08T07:16:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.308+0000 7f2307fff640 1 -- 172.21.15.70:0/864402128 wait complete. 2023-10-08T07:16:24.325 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-08T07:16:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.535+0000 7f118f01a640 1 Processor -- start 2023-10-08T07:16:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.535+0000 7f118f01a640 1 -- start start 2023-10-08T07:16:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.537+0000 7f118f01a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1188160660 0x7f1188070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.537+0000 7f118f01a640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f11880c9910 0x7f11880caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.537+0000 7f118f01a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f11880cb450 0x7f11880b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.537+0000 7f118f01a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118806e150 con 0x7f11880cb450 2023-10-08T07:16:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.537+0000 7f118f01a640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11880cb8c0 con 0x7f11880c9910 2023-10-08T07:16:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.537+0000 7f118f01a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f118806de70 con 0x7f1188160660 2023-10-08T07:16:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.537+0000 7f118d5b7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f11880cb450 0x7f11880b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.537+0000 7f118d5b7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f11880cb450 0x7f11880b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36632/0 (socket says 172.21.15.70:36632) 2023-10-08T07:16:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.537+0000 7f118d5b7640 1 -- 172.21.15.70:0/3412996117 learned_addr learned my addr 172.21.15.70:0/3412996117 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.538+0000 7f118cdb6640 1 --2- 172.21.15.70:0/3412996117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1188160660 0x7f1188070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.538+0000 7f1187fff640 1 --2- 172.21.15.70:0/3412996117 >> v2:172.21.15.181:3300/0 conn(0x7f11880c9910 0x7f11880caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.538+0000 7f118d5b7640 1 -- 172.21.15.70:0/3412996117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1188160660 msgr2=0x7f1188070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.538+0000 7f118d5b7640 1 --2- 172.21.15.70:0/3412996117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1188160660 0x7f1188070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.538+0000 7f118d5b7640 1 -- 172.21.15.70:0/3412996117 >> v2:172.21.15.181:3300/0 conn(0x7f11880c9910 msgr2=0x7f11880caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.538+0000 7f118d5b7640 1 --2- 172.21.15.70:0/3412996117 >> v2:172.21.15.181:3300/0 conn(0x7f11880c9910 0x7f11880caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.538+0000 7f118d5b7640 1 -- 172.21.15.70:0/3412996117 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11880c8e30 con 0x7f11880cb450 2023-10-08T07:16:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.538+0000 7f1187fff640 1 --2- 172.21.15.70:0/3412996117 >> v2:172.21.15.181:3300/0 conn(0x7f11880c9910 0x7f11880caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:24.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.539+0000 7f118d5b7640 1 --2- 172.21.15.70:0/3412996117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11880cb450 0x7f11880b1990 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 crypto rx=0x7f118806e4b0 tx=0x7f118002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68e0a1c543efeb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:24.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.540+0000 7f11877fe640 1 -- 172.21.15.70:0/3412996117 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118002ca70 con 0x7f11880cb450 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.540+0000 7f11877fe640 1 -- 172.21.15.70:0/3412996117 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118002cbd0 con 0x7f11880cb450 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.540+0000 7f11877fe640 1 -- 172.21.15.70:0/3412996117 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11800343f0 con 0x7f11880cb450 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.540+0000 7f118f01a640 1 -- 172.21.15.70:0/3412996117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11880cb450 msgr2=0x7f11880b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.540+0000 7f118f01a640 1 --2- 172.21.15.70:0/3412996117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11880cb450 0x7f11880b1990 secure :-1 s=READY pgs=3019 cs=0 l=1 rev1=1 crypto rx=0x7f118806e4b0 tx=0x7f118002bed0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.541+0000 7f118f01a640 1 -- 172.21.15.70:0/3412996117 shutdown_connections 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.541+0000 7f118f01a640 1 --2- 172.21.15.70:0/3412996117 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11880cb450 0x7f11880b1990 unknown :-1 s=CLOSED pgs=3019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.541+0000 7f118f01a640 1 --2- 172.21.15.70:0/3412996117 >> v2:172.21.15.181:3300/0 conn(0x7f11880c9910 0x7f11880caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.541+0000 7f118f01a640 1 --2- 172.21.15.70:0/3412996117 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1188160660 0x7f1188070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.541+0000 7f118f01a640 1 -- 172.21.15.70:0/3412996117 >> 172.21.15.70:0/3412996117 conn(0x7f118806a5d0 msgr2=0x7f11880b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.541+0000 7f118f01a640 1 -- 172.21.15.70:0/3412996117 shutdown_connections 2023-10-08T07:16:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.541+0000 7f118f01a640 1 -- 172.21.15.70:0/3412996117 wait complete. 2023-10-08T07:16:24.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.542+0000 7f118f01a640 1 Processor -- start 2023-10-08T07:16:24.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.543+0000 7f118f01a640 1 -- start start 2023-10-08T07:16:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.543+0000 7f118f01a640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11880c9910 0x7f11880c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.543+0000 7f118f01a640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11880cb450 0x7f11880c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118f01a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1188160660 0x7f11881f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118f01a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1188148a30 con 0x7f1188160660 2023-10-08T07:16:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118f01a640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1188148d10 con 0x7f11880cb450 2023-10-08T07:16:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118f01a640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1188148730 con 0x7f11880c9910 2023-10-08T07:16:24.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118d5b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1188160660 0x7f11881f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118d5b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1188160660 0x7f11881f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36636/0 (socket says 172.21.15.70:36636) 2023-10-08T07:16:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118d5b7640 1 -- 172.21.15.70:0/3321077384 learned_addr learned my addr 172.21.15.70:0/3321077384 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118cdb6640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11880c9910 0x7f11880c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f1187fff640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11880cb450 0x7f11880c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118d5b7640 1 -- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11880c9910 msgr2=0x7f11880c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118d5b7640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11880c9910 0x7f11880c3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118d5b7640 1 -- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11880cb450 msgr2=0x7f11880c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118d5b7640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11880cb450 0x7f11880c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118d5b7640 1 -- 172.21.15.70:0/3321077384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1180009b30 con 0x7f1188160660 2023-10-08T07:16:24.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f118cdb6640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11880c9910 0x7f11880c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.544+0000 7f1187fff640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11880cb450 0x7f11880c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.545+0000 7f118d5b7640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1188160660 0x7f11881f5d30 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7f1180036040 tx=0x7f1180004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.545+0000 7f11857fa640 1 -- 172.21.15.70:0/3321077384 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180009510 con 0x7f1188160660 2023-10-08T07:16:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.545+0000 7f11857fa640 1 -- 172.21.15.70:0/3321077384 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118003e450 con 0x7f1188160660 2023-10-08T07:16:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.545+0000 7f11857fa640 1 -- 172.21.15.70:0/3321077384 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118003e800 con 0x7f1188160660 2023-10-08T07:16:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.545+0000 7f118f01a640 1 -- 172.21.15.70:0/3321077384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11880c4700 con 0x7f1188160660 2023-10-08T07:16:24.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.545+0000 7f118f01a640 1 -- 172.21.15.70:0/3321077384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11880c4cc0 con 0x7f1188160660 2023-10-08T07:16:24.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.546+0000 7f118f01a640 1 -- 172.21.15.70:0/3321077384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1188148e80 con 0x7f1188160660 2023-10-08T07:16:24.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.547+0000 7f11857fa640 1 -- 172.21.15.70:0/3321077384 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f118003e960 con 0x7f1188160660 2023-10-08T07:16:24.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.548+0000 7f11857fa640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f116c06afc0 0x7f116c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.548+0000 7f11857fa640 1 -- 172.21.15.70:0/3321077384 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f11800ac750 con 0x7f1188160660 2023-10-08T07:16:24.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.549+0000 7f118cdb6640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f116c06afc0 0x7f116c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.550+0000 7f118cdb6640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f116c06afc0 0x7f116c06d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f117c006fd0 tx=0x7f117c008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:24.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.553+0000 7f11857fa640 1 -- 172.21.15.70:0/3321077384 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11800774d0 con 0x7f1188160660 2023-10-08T07:16:24.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.743+0000 7f118f01a640 1 -- 172.21.15.70:0/3321077384 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f11880c93f0 con 0x7f116c06afc0 2023-10-08T07:16:24.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.744+0000 7f11857fa640 1 -- 172.21.15.70:0/3321077384 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f117c009e10 con 0x7f116c06afc0 2023-10-08T07:16:24.746 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 1 to deep-scrub 2023-10-08T07:16:24.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 -- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f116c06afc0 msgr2=0x7f116c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f116c06afc0 0x7f116c06d480 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f117c006fd0 tx=0x7f117c008040 comp rx=0 tx=0).stop 2023-10-08T07:16:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 -- 172.21.15.70:0/3321077384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1188160660 msgr2=0x7f11881f5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1188160660 0x7f11881f5d30 secure :-1 s=READY pgs=3020 cs=0 l=1 rev1=1 crypto rx=0x7f1180036040 tx=0x7f1180004a40 comp rx=0 tx=0).stop 2023-10-08T07:16:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 -- 172.21.15.70:0/3321077384 shutdown_connections 2023-10-08T07:16:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f116c06afc0 0x7f116c06d480 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1188160660 0x7f11881f5d30 unknown :-1 s=CLOSED pgs=3020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f11880cb450 0x7f11880c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 --2- 172.21.15.70:0/3321077384 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f11880c9910 0x7f11880c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 -- 172.21.15.70:0/3321077384 >> 172.21.15.70:0/3321077384 conn(0x7f118806a5d0 msgr2=0x7f118815e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 -- 172.21.15.70:0/3321077384 shutdown_connections 2023-10-08T07:16:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.747+0000 7f118f01a640 1 -- 172.21.15.70:0/3321077384 wait complete. 2023-10-08T07:16:24.768 INFO:tasks.ceph:Scrubbing osd.2 2023-10-08T07:16:24.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-08T07:16:24.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.990+0000 7f76dfb2c640 1 Processor -- start 2023-10-08T07:16:24.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.991+0000 7f76dfb2c640 1 -- start start 2023-10-08T07:16:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.992+0000 7f76dfb2c640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f76d806dfc0 0x7f76d8070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.992+0000 7f76dfb2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76d8071070 0x7f76d80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.992+0000 7f76dfb2c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d80b1cc0 con 0x7f76d80afbe0 2023-10-08T07:16:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.992+0000 7f76dfb2c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d80b2620 con 0x7f76d8071070 2023-10-08T07:16:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.992+0000 7f76dfb2c640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d80b2060 con 0x7f76d806dfc0 2023-10-08T07:16:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.993+0000 7f76de0c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76d8071070 0x7f76d80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.993+0000 7f76de0c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76d8071070 0x7f76d80cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36646/0 (socket says 172.21.15.70:36646) 2023-10-08T07:16:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.993+0000 7f76de0c9640 1 -- 172.21.15.70:0/3079590030 learned_addr learned my addr 172.21.15.70:0/3079590030 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.993+0000 7f76dd0c7640 1 --2- 172.21.15.70:0/3079590030 >> v2:172.21.15.181:3300/0 conn(0x7f76d806dfc0 0x7f76d8070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.993+0000 7f76de0c9640 1 -- 172.21.15.70:0/3079590030 >> v2:172.21.15.181:3300/0 conn(0x7f76d806dfc0 msgr2=0x7f76d8070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:24.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.993+0000 7f76de0c9640 1 --2- 172.21.15.70:0/3079590030 >> v2:172.21.15.181:3300/0 conn(0x7f76d806dfc0 0x7f76d8070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.993+0000 7f76de0c9640 1 -- 172.21.15.70:0/3079590030 >> v1:172.21.15.181:6789/0 conn(0x7f76d80afbe0 legacy=0x7f76d80b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:24.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.993+0000 7f76de0c9640 1 -- 172.21.15.70:0/3079590030 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d80b2340 con 0x7f76d8071070 2023-10-08T07:16:24.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.993+0000 7f76dd0c7640 1 --2- 172.21.15.70:0/3079590030 >> v2:172.21.15.181:3300/0 conn(0x7f76d806dfc0 0x7f76d8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.994+0000 7f76de0c9640 1 --2- 172.21.15.70:0/3079590030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76d8071070 0x7f76d80cb130 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 crypto rx=0x7f76d80b3980 tx=0x7f76d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2ba87cb9d8c7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.994+0000 7f76dc8c6640 1 -- 172.21.15.70:0/3079590030 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0031870 con 0x7f76d8071070 2023-10-08T07:16:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.995+0000 7f76dc8c6640 1 -- 172.21.15.70:0/3079590030 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d00319d0 con 0x7f76d8071070 2023-10-08T07:16:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.995+0000 7f76dc8c6640 1 -- 172.21.15.70:0/3079590030 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0031d60 con 0x7f76d8071070 2023-10-08T07:16:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.995+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3079590030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76d8071070 msgr2=0x7f76d80cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.995+0000 7f76dfb2c640 1 --2- 172.21.15.70:0/3079590030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76d8071070 0x7f76d80cb130 secure :-1 s=READY pgs=3021 cs=0 l=1 rev1=1 crypto rx=0x7f76d80b3980 tx=0x7f76d002b910 comp rx=0 tx=0).stop 2023-10-08T07:16:24.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.995+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3079590030 shutdown_connections 2023-10-08T07:16:24.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.995+0000 7f76dfb2c640 1 --2- 172.21.15.70:0/3079590030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76d8071070 0x7f76d80cb130 unknown :-1 s=CLOSED pgs=3021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.995+0000 7f76dfb2c640 1 --2- 172.21.15.70:0/3079590030 >> v2:172.21.15.181:3300/0 conn(0x7f76d806dfc0 0x7f76d8070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:24.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.995+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3079590030 >> 172.21.15.70:0/3079590030 conn(0x7f76d806a5d0 msgr2=0x7f76d80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:24.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.995+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3079590030 shutdown_connections 2023-10-08T07:16:24.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.995+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3079590030 wait complete. 2023-10-08T07:16:24.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.997+0000 7f76dfb2c640 1 Processor -- start 2023-10-08T07:16:24.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.997+0000 7f76dfb2c640 1 -- start start 2023-10-08T07:16:25.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.998+0000 7f76dfb2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76d806dfc0 0x7f76d81f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.998+0000 7f76dfb2c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76d8071070 0x7f76d81564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.998+0000 7f76dfb2c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d80afbe0 0x7f76d80c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.998+0000 7f76dfb2c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d8140f60 con 0x7f76d806dfc0 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.998+0000 7f76dfb2c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d81432b0 con 0x7f76d80afbe0 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.998+0000 7f76dfb2c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d8143590 con 0x7f76d8071070 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.998+0000 7f76dd8c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76d806dfc0 0x7f76d81f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.998+0000 7f76dd8c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76d806dfc0 0x7f76d81f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36656/0 (socket says 172.21.15.70:36656) 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.998+0000 7f76dd8c8640 1 -- 172.21.15.70:0/3551187993 learned_addr learned my addr 172.21.15.70:0/3551187993 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76dd0c7640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76d8071070 0x7f76d81564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76de0c9640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d80afbe0 0x7f76d80c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76dd8c8640 1 -- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76d8071070 msgr2=0x7f76d81564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76dd8c8640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f76d8071070 0x7f76d81564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76dd8c8640 1 -- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d80afbe0 msgr2=0x7f76d80c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76dd8c8640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d80afbe0 0x7f76d80c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76dd8c8640 1 -- 172.21.15.70:0/3551187993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d00094d0 con 0x7f76d806dfc0 2023-10-08T07:16:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76de0c9640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f76d80afbe0 0x7f76d80c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76dd8c8640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76d806dfc0 0x7f76d81f1d30 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 crypto rx=0x7f76d400ba50 tx=0x7f76d400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76ce7fc640 1 -- 172.21.15.70:0/3551187993 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d400cc10 con 0x7f76d806dfc0 2023-10-08T07:16:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76ce7fc640 1 -- 172.21.15.70:0/3551187993 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d400cd70 con 0x7f76d806dfc0 2023-10-08T07:16:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76ce7fc640 1 -- 172.21.15.70:0/3551187993 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4014680 con 0x7f76d806dfc0 2023-10-08T07:16:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:24.999+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d8143870 con 0x7f76d806dfc0 2023-10-08T07:16:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.000+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d8143e30 con 0x7f76d806dfc0 2023-10-08T07:16:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.000+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f76d8070630 con 0x7f76d806dfc0 2023-10-08T07:16:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.002+0000 7f76ce7fc640 1 -- 172.21.15.70:0/3551187993 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f76d401d3b0 con 0x7f76d806dfc0 2023-10-08T07:16:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.002+0000 7f76ce7fc640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f76bc06af90 0x7f76bc06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.003+0000 7f76ce7fc640 1 -- 172.21.15.70:0/3551187993 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f76d4089990 con 0x7f76d806dfc0 2023-10-08T07:16:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.003+0000 7f76ce7fc640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] conn(0x7f76bc071a80 0x7f76bc073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.003+0000 7f76ce7fc640 1 -- 172.21.15.70:0/3551187993 --> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f76bc0745b0 con 0x7f76bc071a80 2023-10-08T07:16:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.003+0000 7f76ce7fc640 1 -- 172.21.15.70:0/3551187993 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=281) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d408e2c0 con 0x7f76d806dfc0 2023-10-08T07:16:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.003+0000 7f76de0c9640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] conn(0x7f76bc071a80 0x7f76bc073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.003+0000 7f76dd0c7640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f76bc06af90 0x7f76bc06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.004+0000 7f76de0c9640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] conn(0x7f76bc071a80 0x7f76bc073ee0 crc :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.004+0000 7f76dd0c7640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f76bc06af90 0x7f76bc06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76d80c7b70 tx=0x7f76c8008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.004+0000 7f76ce7fc640 1 -- 172.21.15.70:0/3551187993 <== osd.2 v2:172.21.15.70:6817/3799730343 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f76bc0745b0 con 0x7f76bc071a80 2023-10-08T07:16:25.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.018+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 --> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f76d8070350 con 0x7f76bc071a80 2023-10-08T07:16:25.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.019+0000 7f76ce7fc640 1 -- 172.21.15.70:0/3551187993 <== osd.2 v2:172.21.15.70:6817/3799730343 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f76d0031bc0 con 0x7f76bc071a80 2023-10-08T07:16:25.021 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:16:25.021 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T07:16:25.021 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:16:25.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.019+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 >> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] conn(0x7f76bc071a80 msgr2=0x7f76bc073ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.019+0000 7f76dfb2c640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.70:6817/3799730343,v1:172.21.15.70:6820/3799730343] conn(0x7f76bc071a80 0x7f76bc073ee0 crc :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.020+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f76bc06af90 msgr2=0x7f76bc06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.020+0000 7f76dfb2c640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f76bc06af90 0x7f76bc06d450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76d80c7b70 tx=0x7f76c8008040 comp rx=0 tx=0).stop 2023-10-08T07:16:25.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.020+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76d806dfc0 msgr2=0x7f76d81f1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.020+0000 7f76dfb2c640 1 --2- 172.21.15.70:0/3551187993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76d806dfc0 0x7f76d81f1d30 secure :-1 s=READY pgs=3022 cs=0 l=1 rev1=1 crypto rx=0x7f76d400ba50 tx=0x7f76d400bf10 comp rx=0 tx=0).stop 2023-10-08T07:16:25.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.020+0000 7f76de0c9640 1 -- 172.21.15.70:0/3551187993 reap_dead start 2023-10-08T07:16:25.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.020+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 shutdown_connections 2023-10-08T07:16:25.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.020+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 >> 172.21.15.70:0/3551187993 conn(0x7f76d806a5d0 msgr2=0x7f76d80c72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:25.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.020+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 shutdown_connections 2023-10-08T07:16:25.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.020+0000 7f76dfb2c640 1 -- 172.21.15.70:0/3551187993 wait complete. 2023-10-08T07:16:25.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-08T07:16:25.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.257+0000 7fe8226c2640 1 Processor -- start 2023-10-08T07:16:25.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.257+0000 7fe8226c2640 1 -- start start 2023-10-08T07:16:25.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.259+0000 7fe8226c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe81c1601b0 0x7fe81c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.259+0000 7fe8226c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe81c071c30 0x7fe81c069d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.259+0000 7fe8226c2640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c0ce210 con 0x7fe81c072090 2023-10-08T07:16:25.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.259+0000 7fe8226c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c0cdee0 con 0x7fe81c1601b0 2023-10-08T07:16:25.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.259+0000 7fe8226c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c161530 con 0x7fe81c071c30 2023-10-08T07:16:25.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.259+0000 7fe81bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe81c1601b0 0x7fe81c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.260+0000 7fe81bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe81c1601b0 0x7fe81c0cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36668/0 (socket says 172.21.15.70:36668) 2023-10-08T07:16:25.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.260+0000 7fe81bfff640 1 -- 172.21.15.70:0/1692299631 learned_addr learned my addr 172.21.15.70:0/1692299631 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:25.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.260+0000 7fe813fff640 1 --2- 172.21.15.70:0/1692299631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe81c071c30 0x7fe81c069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.260+0000 7fe81bfff640 1 -- 172.21.15.70:0/1692299631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe81c071c30 msgr2=0x7fe81c069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:25.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.260+0000 7fe81bfff640 1 --2- 172.21.15.70:0/1692299631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe81c071c30 0x7fe81c069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.260+0000 7fe81bfff640 1 -- 172.21.15.70:0/1692299631 >> v1:172.21.15.181:6789/0 conn(0x7fe81c072090 legacy=0x7fe81c0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:25.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.260+0000 7fe81bfff640 1 -- 172.21.15.70:0/1692299631 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c161810 con 0x7fe81c1601b0 2023-10-08T07:16:25.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.262+0000 7fe81bfff640 1 --2- 172.21.15.70:0/1692299631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 0x7fe81c0cd430 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009870 tx=0x7fe80c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8c4f98814c6958 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.262+0000 7fe81b7fe640 1 -- 172.21.15.70:0/1692299631 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c030840 con 0x7fe81c1601b0 2023-10-08T07:16:25.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.262+0000 7fe81b7fe640 1 -- 172.21.15.70:0/1692299631 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c0309a0 con 0x7fe81c1601b0 2023-10-08T07:16:25.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.262+0000 7fe81b7fe640 1 -- 172.21.15.70:0/1692299631 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c030d30 con 0x7fe81c1601b0 2023-10-08T07:16:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.263+0000 7fe8226c2640 1 -- 172.21.15.70:0/1692299631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 msgr2=0x7fe81c0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.263+0000 7fe8226c2640 1 --2- 172.21.15.70:0/1692299631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 0x7fe81c0cd430 secure :-1 s=READY pgs=3023 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009870 tx=0x7fe80c02b980 comp rx=0 tx=0).stop 2023-10-08T07:16:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.263+0000 7fe8226c2640 1 -- 172.21.15.70:0/1692299631 shutdown_connections 2023-10-08T07:16:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.263+0000 7fe8226c2640 1 --2- 172.21.15.70:0/1692299631 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe81c071c30 0x7fe81c069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.263+0000 7fe8226c2640 1 --2- 172.21.15.70:0/1692299631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 0x7fe81c0cd430 unknown :-1 s=CLOSED pgs=3023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.263+0000 7fe8226c2640 1 -- 172.21.15.70:0/1692299631 >> 172.21.15.70:0/1692299631 conn(0x7fe81c06d430 msgr2=0x7fe81c06d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.263+0000 7fe8226c2640 1 -- 172.21.15.70:0/1692299631 shutdown_connections 2023-10-08T07:16:25.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.263+0000 7fe8226c2640 1 -- 172.21.15.70:0/1692299631 wait complete. 2023-10-08T07:16:25.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.265+0000 7fe8226c2640 1 Processor -- start 2023-10-08T07:16:25.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.265+0000 7fe8226c2640 1 -- start start 2023-10-08T07:16:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe8226c2640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe81c071c30 0x7fe81c14fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe8226c2640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe81c072090 0x7fe81c0ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe8226c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 0x7fe81c1f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe8226c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c0ab8d0 con 0x7fe81c1601b0 2023-10-08T07:16:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe8226c2640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c0abe50 con 0x7fe81c072090 2023-10-08T07:16:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe8226c2640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1f3820 con 0x7fe81c071c30 2023-10-08T07:16:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe820c5f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 0x7fe81c1f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe820c5f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 0x7fe81c1f2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36678/0 (socket says 172.21.15.70:36678) 2023-10-08T07:16:25.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe820c5f640 1 -- 172.21.15.70:0/759814792 learned_addr learned my addr 172.21.15.70:0/759814792 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:25.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe81bfff640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe81c071c30 0x7fe81c14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.266+0000 7fe813fff640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe81c072090 0x7fe81c0ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.267+0000 7fe820c5f640 1 -- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe81c071c30 msgr2=0x7fe81c14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.267+0000 7fe820c5f640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe81c071c30 0x7fe81c14fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.267+0000 7fe820c5f640 1 -- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe81c072090 msgr2=0x7fe81c0ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.267+0000 7fe820c5f640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe81c072090 0x7fe81c0ac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.267+0000 7fe820c5f640 1 -- 172.21.15.70:0/759814792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c0094d0 con 0x7fe81c1601b0 2023-10-08T07:16:25.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.267+0000 7fe820c5f640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 0x7fe81c1f2ce0 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c6b0 tx=0x7fe81400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.267+0000 7fe8197fa640 1 -- 172.21.15.70:0/759814792 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814003b70 con 0x7fe81c1601b0 2023-10-08T07:16:25.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.267+0000 7fe8197fa640 1 -- 172.21.15.70:0/759814792 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814003cf0 con 0x7fe81c1601b0 2023-10-08T07:16:25.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.267+0000 7fe8226c2640 1 -- 172.21.15.70:0/759814792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c1f3b00 con 0x7fe81c1601b0 2023-10-08T07:16:25.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.267+0000 7fe8197fa640 1 -- 172.21.15.70:0/759814792 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814003ea0 con 0x7fe81c1601b0 2023-10-08T07:16:25.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.268+0000 7fe8226c2640 1 -- 172.21.15.70:0/759814792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c159e60 con 0x7fe81c1601b0 2023-10-08T07:16:25.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.269+0000 7fe8226c2640 1 -- 172.21.15.70:0/759814792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c0abfe0 con 0x7fe81c1601b0 2023-10-08T07:16:25.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.270+0000 7fe8197fa640 1 -- 172.21.15.70:0/759814792 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fe814005310 con 0x7fe81c1601b0 2023-10-08T07:16:25.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.271+0000 7fe8197fa640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe7fc06afc0 0x7fe7fc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.271+0000 7fe8197fa640 1 -- 172.21.15.70:0/759814792 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fe814089740 con 0x7fe81c1601b0 2023-10-08T07:16:25.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.272+0000 7fe81bfff640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe7fc06afc0 0x7fe7fc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.273+0000 7fe81bfff640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe7fc06afc0 0x7fe7fc06d480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe80c036000 tx=0x7fe80c03b040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.276+0000 7fe8197fa640 1 -- 172.21.15.70:0/759814792 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe8140544d0 con 0x7fe81c1601b0 2023-10-08T07:16:25.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.466+0000 7fe8226c2640 1 -- 172.21.15.70:0/759814792 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe81c0716e0 con 0x7fe7fc06afc0 2023-10-08T07:16:25.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.468+0000 7fe8197fa640 1 -- 172.21.15.70:0/759814792 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe80c030b90 con 0x7fe7fc06afc0 2023-10-08T07:16:25.469 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 2 to deep-scrub 2023-10-08T07:16:25.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 -- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe7fc06afc0 msgr2=0x7fe7fc06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe7fc06afc0 0x7fe7fc06d480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe80c036000 tx=0x7fe80c03b040 comp rx=0 tx=0).stop 2023-10-08T07:16:25.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 -- 172.21.15.70:0/759814792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 msgr2=0x7fe81c1f2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 0x7fe81c1f2ce0 secure :-1 s=READY pgs=3024 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c6b0 tx=0x7fe81400cb70 comp rx=0 tx=0).stop 2023-10-08T07:16:25.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 -- 172.21.15.70:0/759814792 shutdown_connections 2023-10-08T07:16:25.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fe7fc06afc0 0x7fe7fc06d480 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe81c1601b0 0x7fe81c1f2ce0 unknown :-1 s=CLOSED pgs=3024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fe81c072090 0x7fe81c0ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 --2- 172.21.15.70:0/759814792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fe81c071c30 0x7fe81c14fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 -- 172.21.15.70:0/759814792 >> 172.21.15.70:0/759814792 conn(0x7fe81c06d430 msgr2=0x7fe81c0cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:25.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 -- 172.21.15.70:0/759814792 shutdown_connections 2023-10-08T07:16:25.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.470+0000 7fe8226c2640 1 -- 172.21.15.70:0/759814792 wait complete. 2023-10-08T07:16:25.490 INFO:tasks.ceph:Scrubbing osd.3 2023-10-08T07:16:25.490 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-08T07:16:25.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.708+0000 7fef7c802640 1 Processor -- start 2023-10-08T07:16:25.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.708+0000 7fef7c802640 1 -- start start 2023-10-08T07:16:25.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.710+0000 7fef7c802640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fef78160660 0x7fef78070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.710+0000 7fef7c802640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef780c9910 0x7fef780caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.710+0000 7fef7c802640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7806e150 con 0x7fef780cb450 2023-10-08T07:16:25.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.710+0000 7fef7c802640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef780cb8c0 con 0x7fef78160660 2023-10-08T07:16:25.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.710+0000 7fef7c802640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7806de70 con 0x7fef780c9910 2023-10-08T07:16:25.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.710+0000 7fef7659c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fef78160660 0x7fef78070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.711+0000 7fef7659c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fef78160660 0x7fef78070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36690/0 (socket says 172.21.15.70:36690) 2023-10-08T07:16:25.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.711+0000 7fef7659c640 1 -- 172.21.15.70:0/745479024 learned_addr learned my addr 172.21.15.70:0/745479024 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:25.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.711+0000 7fef6ffff640 1 --2- 172.21.15.70:0/745479024 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef780c9910 0x7fef780caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.711+0000 7fef7659c640 1 -- 172.21.15.70:0/745479024 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef780c9910 msgr2=0x7fef780caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:25.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.711+0000 7fef7659c640 1 --2- 172.21.15.70:0/745479024 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef780c9910 0x7fef780caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.711+0000 7fef7659c640 1 -- 172.21.15.70:0/745479024 >> v1:172.21.15.181:6789/0 conn(0x7fef780cb450 legacy=0x7fef780b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:25.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.711+0000 7fef7659c640 1 -- 172.21.15.70:0/745479024 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780c8e30 con 0x7fef78160660 2023-10-08T07:16:25.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.713+0000 7fef7659c640 1 --2- 172.21.15.70:0/745479024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef78160660 0x7fef78070cc0 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 crypto rx=0x7fef68009870 tx=0x7fef6802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b38768863368cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.713+0000 7fef75d9b640 1 -- 172.21.15.70:0/745479024 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68031640 con 0x7fef78160660 2023-10-08T07:16:25.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.713+0000 7fef75d9b640 1 -- 172.21.15.70:0/745479024 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef680317a0 con 0x7fef78160660 2023-10-08T07:16:25.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.713+0000 7fef75d9b640 1 -- 172.21.15.70:0/745479024 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68031b30 con 0x7fef78160660 2023-10-08T07:16:25.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.714+0000 7fef7c802640 1 -- 172.21.15.70:0/745479024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef78160660 msgr2=0x7fef78070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.714+0000 7fef7c802640 1 --2- 172.21.15.70:0/745479024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef78160660 0x7fef78070cc0 secure :-1 s=READY pgs=3025 cs=0 l=1 rev1=1 crypto rx=0x7fef68009870 tx=0x7fef6802b8f0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.714+0000 7fef7c802640 1 -- 172.21.15.70:0/745479024 shutdown_connections 2023-10-08T07:16:25.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.714+0000 7fef7c802640 1 --2- 172.21.15.70:0/745479024 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef780c9910 0x7fef780caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.714+0000 7fef7c802640 1 --2- 172.21.15.70:0/745479024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef78160660 0x7fef78070cc0 unknown :-1 s=CLOSED pgs=3025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.714+0000 7fef7c802640 1 -- 172.21.15.70:0/745479024 >> 172.21.15.70:0/745479024 conn(0x7fef7806a5d0 msgr2=0x7fef780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:25.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.714+0000 7fef7c802640 1 -- 172.21.15.70:0/745479024 shutdown_connections 2023-10-08T07:16:25.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.714+0000 7fef7c802640 1 -- 172.21.15.70:0/745479024 wait complete. 2023-10-08T07:16:25.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.716+0000 7fef7c802640 1 Processor -- start 2023-10-08T07:16:25.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.716+0000 7fef7c802640 1 -- start start 2023-10-08T07:16:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.717+0000 7fef7c802640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780c9910 0x7fef78158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.717+0000 7fef7c802640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef780cb450 0x7fef78149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.717+0000 7fef7c802640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef78160660 0x7fef78155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.717+0000 7fef7c802640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78155ab0 con 0x7fef780c9910 2023-10-08T07:16:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.717+0000 7fef7c802640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78155f00 con 0x7fef78160660 2023-10-08T07:16:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.717+0000 7fef7c802640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef781561e0 con 0x7fef780cb450 2023-10-08T07:16:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.717+0000 7fef7659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780c9910 0x7fef78158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.717+0000 7fef7659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780c9910 0x7fef78158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36700/0 (socket says 172.21.15.70:36700) 2023-10-08T07:16:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.717+0000 7fef7659c640 1 -- 172.21.15.70:0/2831910331 learned_addr learned my addr 172.21.15.70:0/2831910331 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef7659c640 1 -- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef780cb450 msgr2=0x7fef78149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef6ffff640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef780cb450 0x7fef78149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef76d9d640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef78160660 0x7fef78155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef7659c640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fef780cb450 0x7fef78149c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef7659c640 1 -- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef78160660 msgr2=0x7fef78155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef7659c640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fef78160660 0x7fef78155570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef7659c640 1 -- 172.21.15.70:0/2831910331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef680094d0 con 0x7fef780c9910 2023-10-08T07:16:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef7659c640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780c9910 0x7fef78158710 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7fef68009870 tx=0x7fef680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef6effd640 1 -- 172.21.15.70:0/2831910331 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6803ac60 con 0x7fef780c9910 2023-10-08T07:16:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef6effd640 1 -- 172.21.15.70:0/2831910331 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6803adc0 con 0x7fef780c9910 2023-10-08T07:16:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef6effd640 1 -- 172.21.15.70:0/2831910331 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef68044630 con 0x7fef780c9910 2023-10-08T07:16:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7815a390 con 0x7fef780c9910 2023-10-08T07:16:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.718+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef78159fd0 con 0x7fef780c9910 2023-10-08T07:16:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.719+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef780c93f0 con 0x7fef780c9910 2023-10-08T07:16:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.721+0000 7fef6effd640 1 -- 172.21.15.70:0/2831910331 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fef6802f050 con 0x7fef780c9910 2023-10-08T07:16:25.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.721+0000 7fef6effd640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fef5806afc0 0x7fef5806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.722+0000 7fef6effd640 1 -- 172.21.15.70:0/2831910331 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fef680ab870 con 0x7fef780c9910 2023-10-08T07:16:25.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.722+0000 7fef6effd640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] conn(0x7fef58071ab0 0x7fef58073f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.722+0000 7fef6effd640 1 -- 172.21.15.70:0/2831910331 --> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef580745e0 con 0x7fef58071ab0 2023-10-08T07:16:25.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.722+0000 7fef6effd640 1 -- 172.21.15.70:0/2831910331 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=281) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef680ade50 con 0x7fef780c9910 2023-10-08T07:16:25.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.722+0000 7fef76d9d640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] conn(0x7fef58071ab0 0x7fef58073f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.722+0000 7fef6ffff640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fef5806afc0 0x7fef5806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.723+0000 7fef76d9d640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] conn(0x7fef58071ab0 0x7fef58073f10 crc :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.723+0000 7fef6ffff640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fef5806afc0 0x7fef5806d480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fef780cf320 tx=0x7fef60008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.724+0000 7fef6effd640 1 -- 172.21.15.70:0/2831910331 <== osd.3 v2:172.21.15.70:6808/471550804 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fef580745e0 con 0x7fef58071ab0 2023-10-08T07:16:25.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.738+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 --> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fef780c9110 con 0x7fef58071ab0 2023-10-08T07:16:25.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.738+0000 7fef6effd640 1 -- 172.21.15.70:0/2831910331 <== osd.3 v2:172.21.15.70:6808/471550804 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fef70004070 con 0x7fef58071ab0 2023-10-08T07:16:25.740 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:16:25.740 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T07:16:25.741 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:16:25.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.738+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 >> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] conn(0x7fef58071ab0 msgr2=0x7fef58073f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.738+0000 7fef7c802640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.70:6808/471550804,v1:172.21.15.70:6809/471550804] conn(0x7fef58071ab0 0x7fef58073f10 crc :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.739+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fef5806afc0 msgr2=0x7fef5806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.739+0000 7fef7c802640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fef5806afc0 0x7fef5806d480 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fef780cf320 tx=0x7fef60008040 comp rx=0 tx=0).stop 2023-10-08T07:16:25.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.739+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780c9910 msgr2=0x7fef78158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.739+0000 7fef7c802640 1 --2- 172.21.15.70:0/2831910331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef780c9910 0x7fef78158710 secure :-1 s=READY pgs=3026 cs=0 l=1 rev1=1 crypto rx=0x7fef68009870 tx=0x7fef680046c0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.739+0000 7fef76d9d640 1 -- 172.21.15.70:0/2831910331 reap_dead start 2023-10-08T07:16:25.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.739+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 shutdown_connections 2023-10-08T07:16:25.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.739+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 >> 172.21.15.70:0/2831910331 conn(0x7fef7806a5d0 msgr2=0x7fef780b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:25.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.739+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 shutdown_connections 2023-10-08T07:16:25.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.739+0000 7fef7c802640 1 -- 172.21.15.70:0/2831910331 wait complete. 2023-10-08T07:16:25.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-08T07:16:25.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.972+0000 7fd3e759e640 1 Processor -- start 2023-10-08T07:16:25.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.973+0000 7fd3e759e640 1 -- start start 2023-10-08T07:16:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.974+0000 7fd3e759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3e815cf30 0x7fd3e80d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.974+0000 7fd3e759e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e80cd520 con 0x7fd3e80cf690 2023-10-08T07:16:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.974+0000 7fd3e759e640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e80d1910 con 0x7fd3e80cf2b0 2023-10-08T07:16:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.974+0000 7fd3e759e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e80ce080 con 0x7fd3e815cf30 2023-10-08T07:16:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.974+0000 7fd3e5d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd3e80cf690 0x7fd3e815c820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59200/0 (socket says 172.21.15.70:59200) 2023-10-08T07:16:25.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.974+0000 7fd3e5d9b640 1 -- 172.21.15.70:0/3668360771 learned_addr learned my addr 172.21.15.70:0/3668360771 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e6d9d640 1 --2- 172.21.15.70:0/3668360771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3e815cf30 0x7fd3e80d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632361070 0 0) 0x7fd3e80cd520 con 0x7fd3e80cf690 2023-10-08T07:16:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3d4003520 con 0x7fd3e80cf690 2023-10-08T07:16:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3609643080 0 0) 0x7fd3d4003520 con 0x7fd3e80cf690 2023-10-08T07:16:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3e815cf30 msgr2=0x7fd3e80d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 --2- 172.21.15.70:0/3668360771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3e815cf30 0x7fd3e80d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 >> v1:172.21.15.181:6789/0 conn(0x7fd3e80cf2b0 legacy=0x7fd3e80d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e80cd750 con 0x7fd3e80cf690 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fd3d0001670 con 0x7fd3e80cf690 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 >> v1:172.21.15.70:6789/0 conn(0x7fd3e80cf690 legacy=0x7fd3e815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 --2- 172.21.15.70:0/3668360771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3d40039a0 0x7fd3d4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 --2- 172.21.15.70:0/3668360771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3e815cf30 0x7fd3d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 --2- 172.21.15.70:0/3668360771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3d4004d00 0x7fd3d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e80ce080 con 0x7fd3d40039a0 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d400bae0 con 0x7fd3d4004d00 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d400bca0 con 0x7fd3e815cf30 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.975+0000 7fd3e659c640 1 --2- 172.21.15.70:0/3668360771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3d40039a0 0x7fd3d4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.976+0000 7fd3e659c640 1 -- 172.21.15.70:0/3668360771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3e815cf30 msgr2=0x7fd3d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:25.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.976+0000 7fd3e659c640 1 --2- 172.21.15.70:0/3668360771 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3e815cf30 0x7fd3d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.976+0000 7fd3e659c640 1 -- 172.21.15.70:0/3668360771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3d4004d00 msgr2=0x7fd3d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:25.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.976+0000 7fd3e659c640 1 --2- 172.21.15.70:0/3668360771 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3d4004d00 0x7fd3d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.976+0000 7fd3e659c640 1 -- 172.21.15.70:0/3668360771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4003520 con 0x7fd3d40039a0 2023-10-08T07:16:25.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.977+0000 7fd3e659c640 1 --2- 172.21.15.70:0/3668360771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3d40039a0 0x7fd3d4003dc0 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8001e50 tx=0x7fd3d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf5df1d50425c3a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.977+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8003b70 con 0x7fd3d40039a0 2023-10-08T07:16:25.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.977+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d8003cd0 con 0x7fd3d40039a0 2023-10-08T07:16:25.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.977+0000 7fd3e559a640 1 -- 172.21.15.70:0/3668360771 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8005310 con 0x7fd3d40039a0 2023-10-08T07:16:25.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.978+0000 7fd3e759e640 1 -- 172.21.15.70:0/3668360771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3d40039a0 msgr2=0x7fd3d4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.978+0000 7fd3e759e640 1 --2- 172.21.15.70:0/3668360771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3d40039a0 0x7fd3d4003dc0 secure :-1 s=READY pgs=3028 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8001e50 tx=0x7fd3d40057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.978+0000 7fd3e6d9d640 1 -- 172.21.15.70:0/3668360771 reap_dead start 2023-10-08T07:16:25.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.978+0000 7fd3e759e640 1 -- 172.21.15.70:0/3668360771 shutdown_connections 2023-10-08T07:16:25.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.978+0000 7fd3e759e640 1 -- 172.21.15.70:0/3668360771 >> 172.21.15.70:0/3668360771 conn(0x7fd3e809bc40 msgr2=0x7fd3e80d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:25.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.978+0000 7fd3e759e640 1 -- 172.21.15.70:0/3668360771 shutdown_connections 2023-10-08T07:16:25.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.978+0000 7fd3e759e640 1 -- 172.21.15.70:0/3668360771 wait complete. 2023-10-08T07:16:25.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.980+0000 7fd3e759e640 1 Processor -- start 2023-10-08T07:16:25.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.980+0000 7fd3e759e640 1 -- start start 2023-10-08T07:16:25.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e759e640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3d40039a0 0x7fd3e8155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e80b7250 0x7fd3e806be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e759e640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e80b76d0 0x7fd3e806c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e815a500 con 0x7fd3e80b7250 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e759e640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e80b2880 con 0x7fd3e80b76d0 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e759e640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e8159f40 con 0x7fd3d40039a0 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e80b7250 0x7fd3e806be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e80b7250 0x7fd3e806be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36722/0 (socket says 172.21.15.70:36722) 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e5d9b640 1 -- 172.21.15.70:0/2216760477 learned_addr learned my addr 172.21.15.70:0/2216760477 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e6d9d640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e80b76d0 0x7fd3e806c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e659c640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3d40039a0 0x7fd3e8155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.981+0000 7fd3e5d9b640 1 -- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3d40039a0 msgr2=0x7fd3e8155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3e5d9b640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3d40039a0 0x7fd3e8155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3e5d9b640 1 -- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e80b76d0 msgr2=0x7fd3e806c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:25.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3e5d9b640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e80b76d0 0x7fd3e806c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:25.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3e5d9b640 1 -- 172.21.15.70:0/2216760477 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e815a220 con 0x7fd3e80b7250 2023-10-08T07:16:25.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3e6d9d640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e80b76d0 0x7fd3e806c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:25.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3e5d9b640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e80b7250 0x7fd3e806be40 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000c700 tx=0x7fd3d000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3deffd640 1 -- 172.21.15.70:0/2216760477 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00074e0 con 0x7fd3e80b7250 2023-10-08T07:16:25.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3deffd640 1 -- 172.21.15.70:0/2216760477 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d0007640 con 0x7fd3e80b7250 2023-10-08T07:16:25.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3deffd640 1 -- 172.21.15.70:0/2216760477 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0004a20 con 0x7fd3e80b7250 2023-10-08T07:16:25.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3e759e640 1 -- 172.21.15.70:0/2216760477 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e80b2510 con 0x7fd3e80b7250 2023-10-08T07:16:25.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.982+0000 7fd3e759e640 1 -- 172.21.15.70:0/2216760477 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e80b5450 con 0x7fd3e80b7250 2023-10-08T07:16:25.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.983+0000 7fd3e759e640 1 -- 172.21.15.70:0/2216760477 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e80b2370 con 0x7fd3e80b7250 2023-10-08T07:16:25.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.984+0000 7fd3deffd640 1 -- 172.21.15.70:0/2216760477 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fd3d0004b80 con 0x7fd3e80b7250 2023-10-08T07:16:25.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.985+0000 7fd3deffd640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fd3c806afc0 0x7fd3c806d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:25.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.985+0000 7fd3deffd640 1 -- 172.21.15.70:0/2216760477 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fd3d0088e00 con 0x7fd3e80b7250 2023-10-08T07:16:25.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.986+0000 7fd3e659c640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fd3c806afc0 0x7fd3c806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:25.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.987+0000 7fd3e659c640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fd3c806afc0 0x7fd3c806d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8034000 tx=0x7fd3d8009030 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:25.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:25.990+0000 7fd3deffd640 1 -- 172.21.15.70:0/2216760477 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3e80b2370 con 0x7fd3e80b7250 2023-10-08T07:16:26.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.177+0000 7fd3e759e640 1 -- 172.21.15.70:0/2216760477 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3e80cdd10 con 0x7fd3c806afc0 2023-10-08T07:16:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.178+0000 7fd3deffd640 1 -- 172.21.15.70:0/2216760477 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd3d8003af0 con 0x7fd3c806afc0 2023-10-08T07:16:26.180 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 3 to deep-scrub 2023-10-08T07:16:26.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 -- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fd3c806afc0 msgr2=0x7fd3c806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fd3c806afc0 0x7fd3c806d480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8034000 tx=0x7fd3d8009030 comp rx=0 tx=0).stop 2023-10-08T07:16:26.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 -- 172.21.15.70:0/2216760477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e80b7250 msgr2=0x7fd3e806be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e80b7250 0x7fd3e806be40 secure :-1 s=READY pgs=3029 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000c700 tx=0x7fd3d000cbc0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 -- 172.21.15.70:0/2216760477 shutdown_connections 2023-10-08T07:16:26.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fd3c806afc0 0x7fd3c806d480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fd3e80b76d0 0x7fd3e806c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd3e80b7250 0x7fd3e806be40 unknown :-1 s=CLOSED pgs=3029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 --2- 172.21.15.70:0/2216760477 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fd3d40039a0 0x7fd3e8155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 -- 172.21.15.70:0/2216760477 >> 172.21.15.70:0/2216760477 conn(0x7fd3e809bc40 msgr2=0x7fd3e815be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:26.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 -- 172.21.15.70:0/2216760477 shutdown_connections 2023-10-08T07:16:26.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.181+0000 7fd3e759e640 1 -- 172.21.15.70:0/2216760477 wait complete. 2023-10-08T07:16:26.202 INFO:tasks.ceph:Scrubbing osd.4 2023-10-08T07:16:26.202 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-08T07:16:26.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.410+0000 7f0a1966d640 1 Processor -- start 2023-10-08T07:16:26.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.411+0000 7f0a1966d640 1 -- start start 2023-10-08T07:16:26.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.412+0000 7f0a1966d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a140cefd0 0x7f0a141679d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.412+0000 7f0a1966d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a140cb140 con 0x7f0a1415cd30 2023-10-08T07:16:26.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.412+0000 7f0a1966d640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a140cb420 con 0x7f0a140cac30 2023-10-08T07:16:26.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.412+0000 7f0a1966d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140cbdc0 con 0x7f0a140cefd0 2023-10-08T07:16:26.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.412+0000 7f0a13fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0a1415cd30 0x7f0a1406c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59214/0 (socket says 172.21.15.70:59214) 2023-10-08T07:16:26.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.412+0000 7f0a13fff640 1 -- 172.21.15.70:0/891934020 learned_addr learned my addr 172.21.15.70:0/891934020 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:26.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434525958 0 0) 0x7f0a140cb140 con 0x7f0a1415cd30 2023-10-08T07:16:26.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a00003520 con 0x7f0a1415cd30 2023-10-08T07:16:26.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a18e6c640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a140cefd0 0x7f0a141679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2800085263 0 0) 0x7f0a00003520 con 0x7f0a1415cd30 2023-10-08T07:16:26.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a140cefd0 msgr2=0x7f0a141679d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:26.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a140cefd0 0x7f0a141679d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 >> v1:172.21.15.181:6789/0 conn(0x7f0a140cac30 legacy=0x7f0a140ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140cb700 con 0x7f0a1415cd30 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7f0a04001670 con 0x7f0a1415cd30 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 >> v1:172.21.15.70:6789/0 conn(0x7f0a1415cd30 legacy=0x7f0a1406c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a140cefd0 0x7f0a000039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a00003fd0 0x7f0a000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a00004d00 0x7f0a00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140cbdc0 con 0x7f0a00003fd0 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0000bbc0 con 0x7f0a00004d00 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.413+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0000bd80 con 0x7f0a140cefd0 2023-10-08T07:16:26.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.414+0000 7f0a0bfff640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a00003fd0 0x7f0a000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.414+0000 7f0a13fff640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a140cefd0 0x7f0a000039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.414+0000 7f0a0bfff640 1 -- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a140cefd0 msgr2=0x7f0a000039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.414+0000 7f0a0bfff640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a140cefd0 0x7f0a000039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.414+0000 7f0a18e6c640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a00004d00 0x7f0a00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.414+0000 7f0a0bfff640 1 -- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a00004d00 msgr2=0x7f0a00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.414+0000 7f0a0bfff640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a00004d00 0x7f0a00005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.414+0000 7f0a0bfff640 1 -- 172.21.15.70:0/891934020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a00003520 con 0x7f0a00003fd0 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.414+0000 7f0a13fff640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a140cefd0 0x7f0a000039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.415+0000 7f0a0bfff640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a00003fd0 0x7f0a000047c0 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f09fc001750 tx=0x7f0a000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5151dfcc68b00b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.415+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0052d0 con 0x7f0a00003fd0 2023-10-08T07:16:26.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.415+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc02e050 con 0x7f0a00003fd0 2023-10-08T07:16:26.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.415+0000 7f0a137fe640 1 -- 172.21.15.70:0/891934020 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc003830 con 0x7f0a00003fd0 2023-10-08T07:16:26.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.416+0000 7f0a1966d640 1 -- 172.21.15.70:0/891934020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a00003fd0 msgr2=0x7f0a000047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.416+0000 7f0a1966d640 1 --2- 172.21.15.70:0/891934020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a00003fd0 0x7f0a000047c0 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f09fc001750 tx=0x7f0a000057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.416+0000 7f0a18e6c640 1 -- 172.21.15.70:0/891934020 reap_dead start 2023-10-08T07:16:26.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.416+0000 7f0a1966d640 1 -- 172.21.15.70:0/891934020 shutdown_connections 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.416+0000 7f0a1966d640 1 -- 172.21.15.70:0/891934020 >> 172.21.15.70:0/891934020 conn(0x7f0a1406a170 msgr2=0x7f0a1406a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.416+0000 7f0a1966d640 1 -- 172.21.15.70:0/891934020 shutdown_connections 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.416+0000 7f0a1966d640 1 -- 172.21.15.70:0/891934020 wait complete. 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.418+0000 7f0a1966d640 1 Processor -- start 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.418+0000 7f0a1966d640 1 -- start start 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.419+0000 7f0a1966d640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a00003fd0 0x7f0a140c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.419+0000 7f0a1966d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a14097c30 0x7f0a140c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.419+0000 7f0a1966d640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a1415a110 0x7f0a1415a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.419+0000 7f0a1966d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1415f350 con 0x7f0a14097c30 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.419+0000 7f0a1966d640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1415f7e0 con 0x7f0a00003fd0 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.419+0000 7f0a1966d640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1415ea20 con 0x7f0a1415a110 2023-10-08T07:16:26.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.419+0000 7f0a0bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a14097c30 0x7f0a140c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.419+0000 7f0a0bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a14097c30 0x7f0a140c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36738/0 (socket says 172.21.15.70:36738) 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.419+0000 7f0a0bfff640 1 -- 172.21.15.70:0/4252740682 learned_addr learned my addr 172.21.15.70:0/4252740682 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.419+0000 7f0a18e6c640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a1415a110 0x7f0a1415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a13fff640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a00003fd0 0x7f0a140c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a0bfff640 1 -- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a1415a110 msgr2=0x7f0a1415a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a0bfff640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f0a1415a110 0x7f0a1415a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a0bfff640 1 -- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a00003fd0 msgr2=0x7f0a140c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a0bfff640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a00003fd0 0x7f0a140c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a0bfff640 1 -- 172.21.15.70:0/4252740682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1415ed00 con 0x7f0a14097c30 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a13fff640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f0a00003fd0 0x7f0a140c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a0bfff640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a14097c30 0x7f0a140c9bd0 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f09fc008fd0 tx=0x7f09fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:26.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a117fa640 1 -- 172.21.15.70:0/4252740682 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0076c0 con 0x7f0a14097c30 2023-10-08T07:16:26.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a117fa640 1 -- 172.21.15.70:0/4252740682 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc007a30 con 0x7f0a14097c30 2023-10-08T07:16:26.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1415efe0 con 0x7f0a14097c30 2023-10-08T07:16:26.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.420+0000 7f0a117fa640 1 -- 172.21.15.70:0/4252740682 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc006be0 con 0x7f0a14097c30 2023-10-08T07:16:26.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.421+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1415aad0 con 0x7f0a14097c30 2023-10-08T07:16:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.421+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a140cb9e0 con 0x7f0a14097c30 2023-10-08T07:16:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.423+0000 7f0a117fa640 1 -- 172.21.15.70:0/4252740682 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f09fc006d40 con 0x7f0a14097c30 2023-10-08T07:16:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.423+0000 7f0a117fa640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f09f406afc0 0x7f09f406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.424+0000 7f0a117fa640 1 -- 172.21.15.70:0/4252740682 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f09fc0abb20 con 0x7f0a14097c30 2023-10-08T07:16:26.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.424+0000 7f0a117fa640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] conn(0x7f09f4071ab0 0x7f09f4073f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.424+0000 7f0a117fa640 1 -- 172.21.15.70:0/4252740682 --> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09f40745e0 con 0x7f09f4071ab0 2023-10-08T07:16:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.424+0000 7f0a117fa640 1 -- 172.21.15.70:0/4252740682 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=281) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09fc0b02a0 con 0x7f0a14097c30 2023-10-08T07:16:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.424+0000 7f0a13fff640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f09f406afc0 0x7f09f406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.425+0000 7f0a18e6c640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] conn(0x7f09f4071ab0 0x7f09f4073f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.426+0000 7f0a18e6c640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] conn(0x7f09f4071ab0 0x7f09f4073f10 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.426+0000 7f0a13fff640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f09f406afc0 0x7f09f406d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001eb0 tx=0x7f0a04009040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:26.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.428+0000 7f0a117fa640 1 -- 172.21.15.70:0/4252740682 <== osd.4 v2:172.21.15.181:6800/2785838626 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f09f40745e0 con 0x7f09f4071ab0 2023-10-08T07:16:26.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.441+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 --> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0a140740e0 con 0x7f09f4071ab0 2023-10-08T07:16:26.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.442+0000 7f0a117fa640 1 -- 172.21.15.70:0/4252740682 <== osd.4 v2:172.21.15.181:6800/2785838626 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0a140740e0 con 0x7f09f4071ab0 2023-10-08T07:16:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:16:26.444 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T07:16:26.444 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:16:26.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] conn(0x7f09f4071ab0 msgr2=0x7f09f4073f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a1966d640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:6800/2785838626,v1:172.21.15.181:6801/2785838626] conn(0x7f09f4071ab0 0x7f09f4073f10 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f09f406afc0 msgr2=0x7f09f406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a1966d640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f09f406afc0 0x7f09f406d480 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0a04001eb0 tx=0x7f0a04009040 comp rx=0 tx=0).stop 2023-10-08T07:16:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a14097c30 msgr2=0x7f0a140c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a1966d640 1 --2- 172.21.15.70:0/4252740682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0a14097c30 0x7f0a140c9bd0 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f09fc008fd0 tx=0x7f09fc001120 comp rx=0 tx=0).stop 2023-10-08T07:16:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a18e6c640 1 -- 172.21.15.70:0/4252740682 reap_dead start 2023-10-08T07:16:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 shutdown_connections 2023-10-08T07:16:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 >> 172.21.15.70:0/4252740682 conn(0x7f0a1406a170 msgr2=0x7f0a1415b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 shutdown_connections 2023-10-08T07:16:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.443+0000 7f0a1966d640 1 -- 172.21.15.70:0/4252740682 wait complete. 2023-10-08T07:16:26.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-08T07:16:26.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.657+0000 7fdd8a875640 1 Processor -- start 2023-10-08T07:16:26.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.657+0000 7fdd8a875640 1 -- start start 2023-10-08T07:16:26.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.659+0000 7fdd8a875640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8415a820 0x7fdd840aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.659+0000 7fdd8a875640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fdd840ca680 0x7fdd840b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.659+0000 7fdd8a875640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd840cab60 con 0x7fdd8415ac00 2023-10-08T07:16:26.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.659+0000 7fdd8a875640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd840cb960 con 0x7fdd840ca680 2023-10-08T07:16:26.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.659+0000 7fdd8a875640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840cbdf0 con 0x7fdd8415a820 2023-10-08T07:16:26.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.659+0000 7fdd8a074640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdd8415ac00 0x7fdd840c9100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59226/0 (socket says 172.21.15.70:59226) 2023-10-08T07:16:26.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.659+0000 7fdd8a074640 1 -- 172.21.15.70:0/741242042 learned_addr learned my addr 172.21.15.70:0/741242042 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:26.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd89072640 1 --2- 172.21.15.70:0/741242042 >> v2:172.21.15.181:3300/0 conn(0x7fdd840ca680 0x7fdd840b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd89873640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8415a820 0x7fdd840aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906461404 0 0) 0x7fdd840cab60 con 0x7fdd8415ac00 2023-10-08T07:16:26.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd70003520 con 0x7fdd8415ac00 2023-10-08T07:16:26.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 919976894 0 0) 0x7fdd70003520 con 0x7fdd8415ac00 2023-10-08T07:16:26.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8415a820 msgr2=0x7fdd840aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:26.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8415a820 0x7fdd840aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 >> v2:172.21.15.181:3300/0 conn(0x7fdd840ca680 msgr2=0x7fdd840b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:26.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 --2- 172.21.15.70:0/741242042 >> v2:172.21.15.181:3300/0 conn(0x7fdd840ca680 0x7fdd840b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840cb030 con 0x7fdd8415ac00 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1758344874 0 0) 0x7fdd7c0019f0 con 0x7fdd8415ac00 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 >> v1:172.21.15.70:6789/0 conn(0x7fdd8415ac00 legacy=0x7fdd840c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd70003970 0x7fdd70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8415a820 0x7fdd700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd70004d00 0x7fdd70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840cb960 con 0x7fdd70003970 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840cbdf0 con 0x7fdd70004d00 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.660+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7000ba60 con 0x7fdd8415a820 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.661+0000 7fdd89873640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd70003970 0x7fdd70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.661+0000 7fdd89072640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8415a820 0x7fdd700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.661+0000 7fdd89873640 1 -- 172.21.15.70:0/741242042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8415a820 msgr2=0x7fdd700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:26.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.661+0000 7fdd89873640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd8415a820 0x7fdd700047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.661+0000 7fdd89873640 1 -- 172.21.15.70:0/741242042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd70004d00 msgr2=0x7fdd70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:26.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.661+0000 7fdd89873640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd70004d00 0x7fdd70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.661+0000 7fdd89873640 1 -- 172.21.15.70:0/741242042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd70003520 con 0x7fdd70003970 2023-10-08T07:16:26.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.662+0000 7fdd89873640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd70003970 0x7fdd70003d90 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000ba30 tx=0x7fdd700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd33784ca782f33 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:26.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.662+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80032630 con 0x7fdd70003970 2023-10-08T07:16:26.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.662+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd80032790 con 0x7fdd70003970 2023-10-08T07:16:26.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.662+0000 7fdd88871640 1 -- 172.21.15.70:0/741242042 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80032b20 con 0x7fdd70003970 2023-10-08T07:16:26.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.663+0000 7fdd8a875640 1 -- 172.21.15.70:0/741242042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd70003970 msgr2=0x7fdd70003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.663+0000 7fdd8a875640 1 --2- 172.21.15.70:0/741242042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd70003970 0x7fdd70003d90 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000ba30 tx=0x7fdd700057f0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.663+0000 7fdd8a074640 1 -- 172.21.15.70:0/741242042 reap_dead start 2023-10-08T07:16:26.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.663+0000 7fdd8a875640 1 -- 172.21.15.70:0/741242042 shutdown_connections 2023-10-08T07:16:26.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.663+0000 7fdd8a875640 1 -- 172.21.15.70:0/741242042 >> 172.21.15.70:0/741242042 conn(0x7fdd8406c050 msgr2=0x7fdd8406c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:26.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.663+0000 7fdd8a875640 1 -- 172.21.15.70:0/741242042 shutdown_connections 2023-10-08T07:16:26.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.663+0000 7fdd8a875640 1 -- 172.21.15.70:0/741242042 wait complete. 2023-10-08T07:16:26.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.665+0000 7fdd8a875640 1 Processor -- start 2023-10-08T07:16:26.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.665+0000 7fdd8a875640 1 -- start start 2023-10-08T07:16:26.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a875640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd70003970 0x7fdd84150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a875640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd841f1d30 0x7fdd840aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a875640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd840af450 0x7fdd841f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a875640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840cdcd0 con 0x7fdd840af450 2023-10-08T07:16:26.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a875640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840af8d0 con 0x7fdd841f1d30 2023-10-08T07:16:26.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a875640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd840c2350 con 0x7fdd70003970 2023-10-08T07:16:26.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a074640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd840af450 0x7fdd841f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a074640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd840af450 0x7fdd841f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36756/0 (socket says 172.21.15.70:36756) 2023-10-08T07:16:26.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a074640 1 -- 172.21.15.70:0/3036243661 learned_addr learned my addr 172.21.15.70:0/3036243661 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a074640 1 -- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd70003970 msgr2=0x7fdd84150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.666+0000 7fdd8a074640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd70003970 0x7fdd84150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.667+0000 7fdd8a074640 1 -- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd841f1d30 msgr2=0x7fdd840aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.667+0000 7fdd8a074640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd841f1d30 0x7fdd840aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.667+0000 7fdd8a074640 1 -- 172.21.15.70:0/3036243661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd841f5f00 con 0x7fdd840af450 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.667+0000 7fdd8a074640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd840af450 0x7fdd841f5320 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7fdd840ad960 tx=0x7fdd7c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.667+0000 7fdd7a7fc640 1 -- 172.21.15.70:0/3036243661 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00a610 con 0x7fdd840af450 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.667+0000 7fdd7a7fc640 1 -- 172.21.15.70:0/3036243661 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c00a770 con 0x7fdd840af450 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.667+0000 7fdd7a7fc640 1 -- 172.21.15.70:0/3036243661 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c00ab00 con 0x7fdd840af450 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.667+0000 7fdd8a875640 1 -- 172.21.15.70:0/3036243661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd841f61e0 con 0x7fdd840af450 2023-10-08T07:16:26.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.667+0000 7fdd8a875640 1 -- 172.21.15.70:0/3036243661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd840cd8f0 con 0x7fdd840af450 2023-10-08T07:16:26.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.668+0000 7fdd8a875640 1 -- 172.21.15.70:0/3036243661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd840cde10 con 0x7fdd840af450 2023-10-08T07:16:26.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.669+0000 7fdd7a7fc640 1 -- 172.21.15.70:0/3036243661 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fdd7c00ac60 con 0x7fdd840af450 2023-10-08T07:16:26.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.670+0000 7fdd7a7fc640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fdd6806af90 0x7fdd6806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:26.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.670+0000 7fdd7a7fc640 1 -- 172.21.15.70:0/3036243661 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fdd7c089130 con 0x7fdd840af450 2023-10-08T07:16:26.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.671+0000 7fdd89873640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fdd6806af90 0x7fdd6806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:26.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.672+0000 7fdd89873640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fdd6806af90 0x7fdd6806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000ba00 tx=0x7fdd80032000 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.675+0000 7fdd7a7fc640 1 -- 172.21.15.70:0/3036243661 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd840cde10 con 0x7fdd840af450 2023-10-08T07:16:26.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.862+0000 7fdd8a875640 1 -- 172.21.15.70:0/3036243661 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd840cb5f0 con 0x7fdd6806af90 2023-10-08T07:16:26.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.864+0000 7fdd7a7fc640 1 -- 172.21.15.70:0/3036243661 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdd80004020 con 0x7fdd6806af90 2023-10-08T07:16:26.865 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 4 to deep-scrub 2023-10-08T07:16:26.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.866+0000 7fdd8a875640 1 -- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fdd6806af90 msgr2=0x7fdd6806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.866+0000 7fdd8a875640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fdd6806af90 0x7fdd6806d450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdd8000ba00 tx=0x7fdd80032000 comp rx=0 tx=0).stop 2023-10-08T07:16:26.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.866+0000 7fdd8a875640 1 -- 172.21.15.70:0/3036243661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd840af450 msgr2=0x7fdd841f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:26.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.866+0000 7fdd8a875640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd840af450 0x7fdd841f5320 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7fdd840ad960 tx=0x7fdd7c00ccd0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.866+0000 7fdd8a875640 1 -- 172.21.15.70:0/3036243661 shutdown_connections 2023-10-08T07:16:26.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.866+0000 7fdd8a875640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fdd6806af90 0x7fdd6806d450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.866+0000 7fdd8a875640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd840af450 0x7fdd841f5320 unknown :-1 s=CLOSED pgs=3035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.866+0000 7fdd8a875640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fdd841f1d30 0x7fdd840aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.866+0000 7fdd8a875640 1 --2- 172.21.15.70:0/3036243661 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fdd70003970 0x7fdd84150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:26.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.867+0000 7fdd8a875640 1 -- 172.21.15.70:0/3036243661 >> 172.21.15.70:0/3036243661 conn(0x7fdd8406c050 msgr2=0x7fdd840ad110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:26.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.867+0000 7fdd8a875640 1 -- 172.21.15.70:0/3036243661 shutdown_connections 2023-10-08T07:16:26.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:26.867+0000 7fdd8a875640 1 -- 172.21.15.70:0/3036243661 wait complete. 2023-10-08T07:16:26.889 INFO:tasks.ceph:Scrubbing osd.5 2023-10-08T07:16:26.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-08T07:16:27.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.097+0000 7fb7f7c4b640 1 Processor -- start 2023-10-08T07:16:27.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.098+0000 7fb7f7c4b640 1 -- start start 2023-10-08T07:16:27.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.099+0000 7fb7f7c4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f006dfc0 0x7fb7f0070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.099+0000 7fb7f7c4b640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7f0071070 0x7fb7f00cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.099+0000 7fb7f7c4b640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f00b1cc0 con 0x7fb7f00afbe0 2023-10-08T07:16:27.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.099+0000 7fb7f7c4b640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f00b2620 con 0x7fb7f0071070 2023-10-08T07:16:27.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.099+0000 7fb7f7c4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00b2060 con 0x7fb7f006dfc0 2023-10-08T07:16:27.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.100+0000 7fb7f51e6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f006dfc0 0x7fb7f0070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.100+0000 7fb7f61e8640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb7f0071070 0x7fb7f00cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.100+0000 7fb7f51e6640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f006dfc0 0x7fb7f0070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:49456/0 (socket says 172.21.15.70:49456) 2023-10-08T07:16:27.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.100+0000 7fb7f51e6640 1 -- 172.21.15.70:0/2383593829 learned_addr learned my addr 172.21.15.70:0/2383593829 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:27.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.101+0000 7fb7f61e8640 1 -- 172.21.15.70:0/2383593829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f006dfc0 msgr2=0x7fb7f0070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:27.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.101+0000 7fb7f61e8640 1 --2- 172.21.15.70:0/2383593829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f006dfc0 0x7fb7f0070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.101+0000 7fb7f61e8640 1 -- 172.21.15.70:0/2383593829 >> v1:172.21.15.181:6789/0 conn(0x7fb7f00afbe0 legacy=0x7fb7f00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.101+0000 7fb7f61e8640 1 -- 172.21.15.70:0/2383593829 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f00b2340 con 0x7fb7f0071070 2023-10-08T07:16:27.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.102+0000 7fb7f61e8640 1 --2- 172.21.15.70:0/2383593829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f0071070 0x7fb7f00cb130 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009d90 tx=0x7fb7e802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12fd299a097829a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.103+0000 7fb7f49e5640 1 -- 172.21.15.70:0/2383593829 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e802cc50 con 0x7fb7f0071070 2023-10-08T07:16:27.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.103+0000 7fb7f49e5640 1 -- 172.21.15.70:0/2383593829 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e802cdb0 con 0x7fb7f0071070 2023-10-08T07:16:27.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.103+0000 7fb7f49e5640 1 -- 172.21.15.70:0/2383593829 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8034660 con 0x7fb7f0071070 2023-10-08T07:16:27.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.103+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/2383593829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f0071070 msgr2=0x7fb7f00cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.103+0000 7fb7f7c4b640 1 --2- 172.21.15.70:0/2383593829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f0071070 0x7fb7f00cb130 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009d90 tx=0x7fb7e802bea0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.103+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/2383593829 shutdown_connections 2023-10-08T07:16:27.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.103+0000 7fb7f7c4b640 1 --2- 172.21.15.70:0/2383593829 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f0071070 0x7fb7f00cb130 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.103+0000 7fb7f7c4b640 1 --2- 172.21.15.70:0/2383593829 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f006dfc0 0x7fb7f0070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.103+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/2383593829 >> 172.21.15.70:0/2383593829 conn(0x7fb7f006a5d0 msgr2=0x7fb7f00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:27.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.103+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/2383593829 shutdown_connections 2023-10-08T07:16:27.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.104+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/2383593829 wait complete. 2023-10-08T07:16:27.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.105+0000 7fb7f7c4b640 1 Processor -- start 2023-10-08T07:16:27.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.105+0000 7fb7f7c4b640 1 -- start start 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.106+0000 7fb7f7c4b640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f006dfc0 0x7fb7f0152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.106+0000 7fb7f7c4b640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f0071070 0x7fb7f01f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.106+0000 7fb7f7c4b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f00afbe0 0x7fb7f01f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.106+0000 7fb7f7c4b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f01f7180 con 0x7fb7f00afbe0 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.106+0000 7fb7f7c4b640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00cc770 con 0x7fb7f006dfc0 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.106+0000 7fb7f7c4b640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00cca50 con 0x7fb7f0071070 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f61e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f00afbe0 0x7fb7f01f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f61e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f00afbe0 0x7fb7f01f67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36762/0 (socket says 172.21.15.70:36762) 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f61e8640 1 -- 172.21.15.70:0/1261828565 learned_addr learned my addr 172.21.15.70:0/1261828565 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f59e7640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f006dfc0 0x7fb7f0152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f51e6640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f0071070 0x7fb7f01f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f61e8640 1 -- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f0071070 msgr2=0x7fb7f01f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f61e8640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb7f0071070 0x7fb7f01f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f61e8640 1 -- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f006dfc0 msgr2=0x7fb7f0152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f61e8640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb7f006dfc0 0x7fb7f0152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f61e8640 1 -- 172.21.15.70:0/1261828565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e8009a70 con 0x7fb7f00afbe0 2023-10-08T07:16:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.107+0000 7fb7f61e8640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f00afbe0 0x7fb7f01f67e0 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009d90 tx=0x7fb7e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.108+0000 7fb7e67fc640 1 -- 172.21.15.70:0/1261828565 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8040650 con 0x7fb7f00afbe0 2023-10-08T07:16:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.108+0000 7fb7e67fc640 1 -- 172.21.15.70:0/1261828565 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e80407b0 con 0x7fb7f00afbe0 2023-10-08T07:16:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.108+0000 7fb7e67fc640 1 -- 172.21.15.70:0/1261828565 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8040b60 con 0x7fb7f00afbe0 2023-10-08T07:16:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.108+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f00ccd30 con 0x7fb7f00afbe0 2023-10-08T07:16:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.108+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f00cd2f0 con 0x7fb7f00afbe0 2023-10-08T07:16:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.109+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7f0070630 con 0x7fb7f00afbe0 2023-10-08T07:16:27.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.110+0000 7fb7e67fc640 1 -- 172.21.15.70:0/1261828565 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb7e8040cc0 con 0x7fb7f00afbe0 2023-10-08T07:16:27.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.111+0000 7fb7e67fc640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb7d406af90 0x7fb7d406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.111+0000 7fb7e67fc640 1 -- 172.21.15.70:0/1261828565 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fb7e80ac730 con 0x7fb7f00afbe0 2023-10-08T07:16:27.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.111+0000 7fb7e67fc640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] conn(0x7fb7d4071a80 0x7fb7d4073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.111+0000 7fb7e67fc640 1 -- 172.21.15.70:0/1261828565 --> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7d40745b0 con 0x7fb7d4071a80 2023-10-08T07:16:27.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.111+0000 7fb7e67fc640 1 -- 172.21.15.70:0/1261828565 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=281) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7e80aed10 con 0x7fb7f00afbe0 2023-10-08T07:16:27.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.112+0000 7fb7f59e7640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb7d406af90 0x7fb7d406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.112+0000 7fb7f51e6640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] conn(0x7fb7d4071a80 0x7fb7d4073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.113+0000 7fb7f59e7640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb7d406af90 0x7fb7d406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000f60 tx=0x7fb7ec009040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.113+0000 7fb7f51e6640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] conn(0x7fb7d4071a80 0x7fb7d4073ee0 crc :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.115+0000 7fb7e67fc640 1 -- 172.21.15.70:0/1261828565 <== osd.5 v2:172.21.15.181:6824/1806853192 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb7d40745b0 con 0x7fb7d4071a80 2023-10-08T07:16:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.129+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 --> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb7f0070350 con 0x7fb7d4071a80 2023-10-08T07:16:27.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.130+0000 7fb7e67fc640 1 -- 172.21.15.70:0/1261828565 <== osd.5 v2:172.21.15.181:6824/1806853192 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb7e0009890 con 0x7fb7d4071a80 2023-10-08T07:16:27.131 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:16:27.132 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T07:16:27.132 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.130+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] conn(0x7fb7d4071a80 msgr2=0x7fb7d4073ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.130+0000 7fb7f7c4b640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:6824/1806853192,v1:172.21.15.181:6825/1806853192] conn(0x7fb7d4071a80 0x7fb7d4073ee0 crc :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.130+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb7d406af90 msgr2=0x7fb7d406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.130+0000 7fb7f7c4b640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb7d406af90 0x7fb7d406d450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000f60 tx=0x7fb7ec009040 comp rx=0 tx=0).stop 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.130+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f00afbe0 msgr2=0x7fb7f01f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.130+0000 7fb7f7c4b640 1 --2- 172.21.15.70:0/1261828565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7f00afbe0 0x7fb7f01f67e0 secure :-1 s=READY pgs=3036 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009d90 tx=0x7fb7e8004a40 comp rx=0 tx=0).stop 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.130+0000 7fb7f61e8640 1 -- 172.21.15.70:0/1261828565 reap_dead start 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.131+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 shutdown_connections 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.131+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 >> 172.21.15.70:0/1261828565 conn(0x7fb7f006a5d0 msgr2=0x7fb7f00c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.131+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 shutdown_connections 2023-10-08T07:16:27.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.131+0000 7fb7f7c4b640 1 -- 172.21.15.70:0/1261828565 wait complete. 2023-10-08T07:16:27.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-08T07:16:27.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.363+0000 7f467d2ea640 1 Processor -- start 2023-10-08T07:16:27.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.363+0000 7f467d2ea640 1 -- start start 2023-10-08T07:16:27.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.365+0000 7f467d2ea640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f467815a820 0x7f46780aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.365+0000 7f467d2ea640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f46780ca680 0x7f46780b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.365+0000 7f467d2ea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 0x7f46780c9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.365+0000 7f467d2ea640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46780cab60 con 0x7f46780ca680 2023-10-08T07:16:27.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.365+0000 7f467d2ea640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46780cb960 con 0x7f467815a820 2023-10-08T07:16:27.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.365+0000 7f467d2ea640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46780cbdf0 con 0x7f467815ac00 2023-10-08T07:16:27.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.365+0000 7f466ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f46780ca680 0x7f46780b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.365+0000 7f466ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f46780ca680 0x7f46780b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36778/0 (socket says 172.21.15.70:36778) 2023-10-08T07:16:27.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.365+0000 7f466ffff640 1 -- 172.21.15.70:0/1834981450 learned_addr learned my addr 172.21.15.70:0/1834981450 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:27.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.366+0000 7f4677fff640 1 --2- 172.21.15.70:0/1834981450 >> v2:172.21.15.181:3300/0 conn(0x7f467815a820 0x7f46780aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.366+0000 7f467cae9640 1 --2- 172.21.15.70:0/1834981450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 0x7f46780c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.366+0000 7f466ffff640 1 -- 172.21.15.70:0/1834981450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 msgr2=0x7f46780c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:27.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.366+0000 7f466ffff640 1 --2- 172.21.15.70:0/1834981450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 0x7f46780c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.366+0000 7f466ffff640 1 -- 172.21.15.70:0/1834981450 >> v2:172.21.15.181:3300/0 conn(0x7f467815a820 msgr2=0x7f46780aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:27.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.366+0000 7f466ffff640 1 --2- 172.21.15.70:0/1834981450 >> v2:172.21.15.181:3300/0 conn(0x7f467815a820 0x7f46780aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.366+0000 7f466ffff640 1 -- 172.21.15.70:0/1834981450 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46780cb030 con 0x7f46780ca680 2023-10-08T07:16:27.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.366+0000 7f467cae9640 1 --2- 172.21.15.70:0/1834981450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 0x7f46780c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:27.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.367+0000 7f466ffff640 1 --2- 172.21.15.70:0/1834981450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46780ca680 0x7f46780b04b0 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7f467806da70 tx=0x7f466002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd10a3dcaad7175 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.367+0000 7f46777fe640 1 -- 172.21.15.70:0/1834981450 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46600317e0 con 0x7f46780ca680 2023-10-08T07:16:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.367+0000 7f46777fe640 1 -- 172.21.15.70:0/1834981450 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4660031940 con 0x7f46780ca680 2023-10-08T07:16:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.367+0000 7f46777fe640 1 -- 172.21.15.70:0/1834981450 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660031cd0 con 0x7f46780ca680 2023-10-08T07:16:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.368+0000 7f467d2ea640 1 -- 172.21.15.70:0/1834981450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46780ca680 msgr2=0x7f46780b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.368+0000 7f467d2ea640 1 --2- 172.21.15.70:0/1834981450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46780ca680 0x7f46780b04b0 secure :-1 s=READY pgs=3037 cs=0 l=1 rev1=1 crypto rx=0x7f467806da70 tx=0x7f466002b880 comp rx=0 tx=0).stop 2023-10-08T07:16:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.368+0000 7f467d2ea640 1 -- 172.21.15.70:0/1834981450 shutdown_connections 2023-10-08T07:16:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.368+0000 7f467d2ea640 1 --2- 172.21.15.70:0/1834981450 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 0x7f46780c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.368+0000 7f467d2ea640 1 --2- 172.21.15.70:0/1834981450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46780ca680 0x7f46780b04b0 unknown :-1 s=CLOSED pgs=3037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.368+0000 7f467d2ea640 1 --2- 172.21.15.70:0/1834981450 >> v2:172.21.15.181:3300/0 conn(0x7f467815a820 0x7f46780aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.368+0000 7f467d2ea640 1 -- 172.21.15.70:0/1834981450 >> 172.21.15.70:0/1834981450 conn(0x7f467806c050 msgr2=0x7f467806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.368+0000 7f467d2ea640 1 -- 172.21.15.70:0/1834981450 shutdown_connections 2023-10-08T07:16:27.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.368+0000 7f467d2ea640 1 -- 172.21.15.70:0/1834981450 wait complete. 2023-10-08T07:16:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.370+0000 7f467d2ea640 1 Processor -- start 2023-10-08T07:16:27.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.370+0000 7f467d2ea640 1 -- start start 2023-10-08T07:16:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.371+0000 7f467d2ea640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46780ca680 0x7f46780acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.371+0000 7f467d2ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467815a820 0x7f46780b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.371+0000 7f467d2ea640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 0x7f46780b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.371+0000 7f467d2ea640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46781f64d0 con 0x7f467815a820 2023-10-08T07:16:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.371+0000 7f467d2ea640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46781f1fd0 con 0x7f46780ca680 2023-10-08T07:16:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.371+0000 7f467d2ea640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46781f21e0 con 0x7f467815ac00 2023-10-08T07:16:27.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f466ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467815a820 0x7f46780b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f466ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467815a820 0x7f46780b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36782/0 (socket says 172.21.15.70:36782) 2023-10-08T07:16:27.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f466ffff640 1 -- 172.21.15.70:0/462841403 learned_addr learned my addr 172.21.15.70:0/462841403 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:27.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f4677fff640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46780ca680 0x7f46780acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f467cae9640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 0x7f46780b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f466ffff640 1 -- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 msgr2=0x7f46780b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f466ffff640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 0x7f46780b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f466ffff640 1 -- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46780ca680 msgr2=0x7f46780acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f466ffff640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46780ca680 0x7f46780acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f466ffff640 1 -- 172.21.15.70:0/462841403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600094d0 con 0x7f467815a820 2023-10-08T07:16:27.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.372+0000 7f466ffff640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467815a820 0x7f46780b3330 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7f466002b8c0 tx=0x7f46600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.373+0000 7f46757fa640 1 -- 172.21.15.70:0/462841403 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660032880 con 0x7f467815a820 2023-10-08T07:16:27.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.373+0000 7f46757fa640 1 -- 172.21.15.70:0/462841403 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4660032a00 con 0x7f467815a820 2023-10-08T07:16:27.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.373+0000 7f46757fa640 1 -- 172.21.15.70:0/462841403 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660032db0 con 0x7f467815a820 2023-10-08T07:16:27.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.373+0000 7f467d2ea640 1 -- 172.21.15.70:0/462841403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46781f24c0 con 0x7f467815a820 2023-10-08T07:16:27.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.373+0000 7f467d2ea640 1 -- 172.21.15.70:0/462841403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46781f2a80 con 0x7f467815a820 2023-10-08T07:16:27.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.374+0000 7f467d2ea640 1 -- 172.21.15.70:0/462841403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46781f2320 con 0x7f467815a820 2023-10-08T07:16:27.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.375+0000 7f46757fa640 1 -- 172.21.15.70:0/462841403 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f466004d3f0 con 0x7f467815a820 2023-10-08T07:16:27.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.376+0000 7f46757fa640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f465806afe0 0x7f465806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.376+0000 7f46757fa640 1 -- 172.21.15.70:0/462841403 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f46600ab8e0 con 0x7f467815a820 2023-10-08T07:16:27.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.377+0000 7f4677fff640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f465806afe0 0x7f465806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.378+0000 7f4677fff640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f465806afe0 0x7f465806d4a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4668006fd0 tx=0x7f4668008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.380+0000 7f46757fa640 1 -- 172.21.15.70:0/462841403 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f46781f2320 con 0x7f467815a820 2023-10-08T07:16:27.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.572+0000 7f467d2ea640 1 -- 172.21.15.70:0/462841403 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f46780cb5f0 con 0x7f465806afe0 2023-10-08T07:16:27.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.573+0000 7f46757fa640 1 -- 172.21.15.70:0/462841403 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f466800f860 con 0x7f465806afe0 2023-10-08T07:16:27.586 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 5 to deep-scrub 2023-10-08T07:16:27.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.575+0000 7f467d2ea640 1 -- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f465806afe0 msgr2=0x7f465806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.575+0000 7f467d2ea640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f465806afe0 0x7f465806d4a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4668006fd0 tx=0x7f4668008040 comp rx=0 tx=0).stop 2023-10-08T07:16:27.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.575+0000 7f467d2ea640 1 -- 172.21.15.70:0/462841403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467815a820 msgr2=0x7f46780b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.575+0000 7f467d2ea640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467815a820 0x7f46780b3330 secure :-1 s=READY pgs=3038 cs=0 l=1 rev1=1 crypto rx=0x7f466002b8c0 tx=0x7f46600046c0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.575+0000 7f467d2ea640 1 -- 172.21.15.70:0/462841403 shutdown_connections 2023-10-08T07:16:27.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.575+0000 7f467d2ea640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f465806afe0 0x7f465806d4a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.575+0000 7f467d2ea640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f467815ac00 0x7f46780b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.575+0000 7f467d2ea640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467815a820 0x7f46780b3330 unknown :-1 s=CLOSED pgs=3038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.575+0000 7f467d2ea640 1 --2- 172.21.15.70:0/462841403 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f46780ca680 0x7f46780acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.575+0000 7f467d2ea640 1 -- 172.21.15.70:0/462841403 >> 172.21.15.70:0/462841403 conn(0x7f467806c050 msgr2=0x7f46780c9990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:27.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.576+0000 7f467d2ea640 1 -- 172.21.15.70:0/462841403 shutdown_connections 2023-10-08T07:16:27.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.576+0000 7f467d2ea640 1 -- 172.21.15.70:0/462841403 wait complete. 2023-10-08T07:16:27.596 INFO:tasks.ceph:Scrubbing osd.6 2023-10-08T07:16:27.596 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-08T07:16:27.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.814+0000 7f4abe9a9640 1 Processor -- start 2023-10-08T07:16:27.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.814+0000 7f4abe9a9640 1 -- start start 2023-10-08T07:16:27.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.815+0000 7f4abe9a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 0x7f4ab80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abe9a9640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f4ab806db60 0x7f4ab806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abe9a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ab815aa00 0x7f4ab80b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abe9a9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab80ca230 con 0x7f4ab815aa00 2023-10-08T07:16:27.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abe9a9640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab815b310 con 0x7f4ab806db60 2023-10-08T07:16:27.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abe9a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab806d290 con 0x7f4ab806c3a0 2023-10-08T07:16:27.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abcf46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ab815aa00 0x7f4ab80b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abcf46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ab815aa00 0x7f4ab80b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36794/0 (socket says 172.21.15.70:36794) 2023-10-08T07:16:27.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abcf46640 1 -- 172.21.15.70:0/1555643050 learned_addr learned my addr 172.21.15.70:0/1555643050 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:27.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4ab7fff640 1 --2- 172.21.15.70:0/1555643050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 0x7f4ab80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4aaf7fe640 1 --2- 172.21.15.70:0/1555643050 >> v2:172.21.15.181:3300/0 conn(0x7f4ab806db60 0x7f4ab806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abcf46640 1 -- 172.21.15.70:0/1555643050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 msgr2=0x7f4ab80c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abcf46640 1 --2- 172.21.15.70:0/1555643050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 0x7f4ab80c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.816+0000 7f4abcf46640 1 -- 172.21.15.70:0/1555643050 >> v2:172.21.15.181:3300/0 conn(0x7f4ab806db60 msgr2=0x7f4ab806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.817+0000 7f4abcf46640 1 --2- 172.21.15.70:0/1555643050 >> v2:172.21.15.181:3300/0 conn(0x7f4ab806db60 0x7f4ab806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.817+0000 7f4abcf46640 1 -- 172.21.15.70:0/1555643050 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab80c9860 con 0x7f4ab815aa00 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.817+0000 7f4aaf7fe640 1 --2- 172.21.15.70:0/1555643050 >> v2:172.21.15.181:3300/0 conn(0x7f4ab806db60 0x7f4ab806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.817+0000 7f4ab7fff640 1 --2- 172.21.15.70:0/1555643050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 0x7f4ab80c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.818+0000 7f4abcf46640 1 --2- 172.21.15.70:0/1555643050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ab815aa00 0x7f4ab80b2a90 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80714d0 tx=0x7f4ab002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48a5dd0de0385bc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.818+0000 7f4ab77fe640 1 -- 172.21.15.70:0/1555643050 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0031870 con 0x7f4ab815aa00 2023-10-08T07:16:27.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.818+0000 7f4ab77fe640 1 -- 172.21.15.70:0/1555643050 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab00319d0 con 0x7f4ab815aa00 2023-10-08T07:16:27.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.818+0000 7f4ab77fe640 1 -- 172.21.15.70:0/1555643050 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0031d60 con 0x7f4ab815aa00 2023-10-08T07:16:27.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.819+0000 7f4abe9a9640 1 -- 172.21.15.70:0/1555643050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ab815aa00 msgr2=0x7f4ab80b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.819+0000 7f4abe9a9640 1 --2- 172.21.15.70:0/1555643050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ab815aa00 0x7f4ab80b2a90 secure :-1 s=READY pgs=3039 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80714d0 tx=0x7f4ab002b910 comp rx=0 tx=0).stop 2023-10-08T07:16:27.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.819+0000 7f4abe9a9640 1 -- 172.21.15.70:0/1555643050 shutdown_connections 2023-10-08T07:16:27.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.819+0000 7f4abe9a9640 1 --2- 172.21.15.70:0/1555643050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ab815aa00 0x7f4ab80b2a90 unknown :-1 s=CLOSED pgs=3039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.819+0000 7f4abe9a9640 1 --2- 172.21.15.70:0/1555643050 >> v2:172.21.15.181:3300/0 conn(0x7f4ab806db60 0x7f4ab806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.819+0000 7f4abe9a9640 1 --2- 172.21.15.70:0/1555643050 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 0x7f4ab80c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.819+0000 7f4abe9a9640 1 -- 172.21.15.70:0/1555643050 >> 172.21.15.70:0/1555643050 conn(0x7f4ab806a5d0 msgr2=0x7f4ab80b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:27.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.819+0000 7f4abe9a9640 1 -- 172.21.15.70:0/1555643050 shutdown_connections 2023-10-08T07:16:27.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.819+0000 7f4abe9a9640 1 -- 172.21.15.70:0/1555643050 wait complete. 2023-10-08T07:16:27.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.821+0000 7f4abe9a9640 1 Processor -- start 2023-10-08T07:16:27.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.821+0000 7f4abe9a9640 1 -- start start 2023-10-08T07:16:27.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abe9a9640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 0x7f4ab8157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abe9a9640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ab806db60 0x7f4ab815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abe9a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ab815aa00 0x7f4ab80b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abe9a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab815e250 con 0x7f4ab815aa00 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abe9a9640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab815b5f0 con 0x7f4ab806db60 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abe9a9640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab815b8d0 con 0x7f4ab806c3a0 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abcf46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ab815aa00 0x7f4ab80b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abcf46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ab815aa00 0x7f4ab80b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36798/0 (socket says 172.21.15.70:36798) 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abcf46640 1 -- 172.21.15.70:0/462280836 learned_addr learned my addr 172.21.15.70:0/462280836 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4aaf7fe640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ab806db60 0x7f4ab815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4ab7fff640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 0x7f4ab8157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abcf46640 1 -- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 msgr2=0x7f4ab8157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abcf46640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 0x7f4ab8157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abcf46640 1 -- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ab806db60 msgr2=0x7f4ab815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abcf46640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ab806db60 0x7f4ab815dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4abcf46640 1 -- 172.21.15.70:0/462280836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00094d0 con 0x7f4ab815aa00 2023-10-08T07:16:27.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4aaf7fe640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f4ab806db60 0x7f4ab815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:27.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.822+0000 7f4ab7fff640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f4ab806c3a0 0x7f4ab8157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:27.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.823+0000 7f4abcf46640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ab815aa00 0x7f4ab80b3480 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00090b0 tx=0x7f4ab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.823+0000 7f4ab57fa640 1 -- 172.21.15.70:0/462280836 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab003ad30 con 0x7f4ab815aa00 2023-10-08T07:16:27.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.823+0000 7f4ab57fa640 1 -- 172.21.15.70:0/462280836 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab00443f0 con 0x7f4ab815aa00 2023-10-08T07:16:27.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.823+0000 7f4ab57fa640 1 -- 172.21.15.70:0/462280836 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00447a0 con 0x7f4ab815aa00 2023-10-08T07:16:27.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.823+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab815bbb0 con 0x7f4ab815aa00 2023-10-08T07:16:27.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.823+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab815c170 con 0x7f4ab815aa00 2023-10-08T07:16:27.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.824+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ab806ccc0 con 0x7f4ab815aa00 2023-10-08T07:16:27.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.825+0000 7f4ab57fa640 1 -- 172.21.15.70:0/462280836 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f4ab004d450 con 0x7f4ab815aa00 2023-10-08T07:16:27.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.826+0000 7f4ab57fa640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f4a9806af90 0x7f4a9806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.827+0000 7f4ab57fa640 1 -- 172.21.15.70:0/462280836 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f4ab002d040 con 0x7f4ab815aa00 2023-10-08T07:16:27.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.827+0000 7f4ab57fa640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] conn(0x7f4a98071a80 0x7f4a98073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:27.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.827+0000 7f4ab57fa640 1 -- 172.21.15.70:0/462280836 --> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4a980745b0 con 0x7f4a98071a80 2023-10-08T07:16:27.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.827+0000 7f4ab57fa640 1 -- 172.21.15.70:0/462280836 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=281) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ab00ae610 con 0x7f4ab815aa00 2023-10-08T07:16:27.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.827+0000 7f4ab7fff640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f4a9806af90 0x7f4a9806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.827+0000 7f4aaf7fe640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] conn(0x7f4a98071a80 0x7f4a98073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:27.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.828+0000 7f4aaf7fe640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] conn(0x7f4a98071a80 0x7f4a98073ee0 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.829+0000 7f4ab7fff640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f4a9806af90 0x7f4a9806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c5320 tx=0x7f4aa8006ec0 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:27.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.830+0000 7f4ab57fa640 1 -- 172.21.15.70:0/462280836 <== osd.6 v2:172.21.15.181:6802/4245641278 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f4a980745b0 con 0x7f4a98071a80 2023-10-08T07:16:27.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.844+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 --> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4ab80c9210 con 0x7f4a98071a80 2023-10-08T07:16:27.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.845+0000 7f4ab57fa640 1 -- 172.21.15.70:0/462280836 <== osd.6 v2:172.21.15.181:6802/4245641278 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4aa000d590 con 0x7f4a98071a80 2023-10-08T07:16:27.909 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:16:27.910 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T07:16:27.910 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:16:27.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] conn(0x7f4a98071a80 msgr2=0x7f4a98073ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abe9a9640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:6802/4245641278,v1:172.21.15.181:6803/4245641278] conn(0x7f4a98071a80 0x7f4a98073ee0 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f4a9806af90 msgr2=0x7f4a9806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abe9a9640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f4a9806af90 0x7f4a9806d450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c5320 tx=0x7f4aa8006ec0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ab815aa00 msgr2=0x7f4ab80b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abe9a9640 1 --2- 172.21.15.70:0/462280836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ab815aa00 0x7f4ab80b3480 secure :-1 s=READY pgs=3040 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00090b0 tx=0x7f4ab00046c0 comp rx=0 tx=0).stop 2023-10-08T07:16:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abcf46640 1 -- 172.21.15.70:0/462280836 reap_dead start 2023-10-08T07:16:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 shutdown_connections 2023-10-08T07:16:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 >> 172.21.15.70:0/462280836 conn(0x7f4ab806a5d0 msgr2=0x7f4ab80cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 shutdown_connections 2023-10-08T07:16:27.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:27.846+0000 7f4abe9a9640 1 -- 172.21.15.70:0/462280836 wait complete. 2023-10-08T07:16:27.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-08T07:16:28.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.133+0000 7f217794c640 1 Processor -- start 2023-10-08T07:16:28.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.133+0000 7f217794c640 1 -- start start 2023-10-08T07:16:28.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.135+0000 7f217794c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f217006dfc0 0x7f2170070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.135+0000 7f217794c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2170071070 0x7f21700cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.135+0000 7f217794c640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21700b1cc0 con 0x7f21700afbe0 2023-10-08T07:16:28.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.135+0000 7f217794c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21700b2620 con 0x7f2170071070 2023-10-08T07:16:28.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.135+0000 7f217794c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21700b2060 con 0x7f217006dfc0 2023-10-08T07:16:28.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.135+0000 7f2175ee9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2170071070 0x7f21700cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.135+0000 7f2175ee9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2170071070 0x7f21700cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36806/0 (socket says 172.21.15.70:36806) 2023-10-08T07:16:28.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.135+0000 7f2175ee9640 1 -- 172.21.15.70:0/544589918 learned_addr learned my addr 172.21.15.70:0/544589918 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:28.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.136+0000 7f2174ee7640 1 --2- 172.21.15.70:0/544589918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f217006dfc0 0x7f2170070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.136+0000 7f2175ee9640 1 -- 172.21.15.70:0/544589918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f217006dfc0 msgr2=0x7f2170070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.136+0000 7f2175ee9640 1 --2- 172.21.15.70:0/544589918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f217006dfc0 0x7f2170070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.136+0000 7f2175ee9640 1 -- 172.21.15.70:0/544589918 >> v1:172.21.15.181:6789/0 conn(0x7f21700afbe0 legacy=0x7f21700b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.136+0000 7f2175ee9640 1 -- 172.21.15.70:0/544589918 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21700b2340 con 0x7f2170071070 2023-10-08T07:16:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.137+0000 7f2175ee9640 1 --2- 172.21.15.70:0/544589918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2170071070 0x7f21700cb130 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7f2168009e50 tx=0x7f216802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c820984a8987ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.137+0000 7f2167fff640 1 -- 172.21.15.70:0/544589918 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216802ca70 con 0x7f2170071070 2023-10-08T07:16:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.137+0000 7f2167fff640 1 -- 172.21.15.70:0/544589918 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216802cbd0 con 0x7f2170071070 2023-10-08T07:16:28.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.137+0000 7f2167fff640 1 -- 172.21.15.70:0/544589918 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21680343f0 con 0x7f2170071070 2023-10-08T07:16:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.138+0000 7f217794c640 1 -- 172.21.15.70:0/544589918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2170071070 msgr2=0x7f21700cb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.138+0000 7f217794c640 1 --2- 172.21.15.70:0/544589918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2170071070 0x7f21700cb130 secure :-1 s=READY pgs=3041 cs=0 l=1 rev1=1 crypto rx=0x7f2168009e50 tx=0x7f216802bed0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.138+0000 7f217794c640 1 -- 172.21.15.70:0/544589918 shutdown_connections 2023-10-08T07:16:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.138+0000 7f217794c640 1 --2- 172.21.15.70:0/544589918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2170071070 0x7f21700cb130 unknown :-1 s=CLOSED pgs=3041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.138+0000 7f217794c640 1 --2- 172.21.15.70:0/544589918 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f217006dfc0 0x7f2170070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.138+0000 7f217794c640 1 -- 172.21.15.70:0/544589918 >> 172.21.15.70:0/544589918 conn(0x7f217006a5d0 msgr2=0x7f21700b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.138+0000 7f217794c640 1 -- 172.21.15.70:0/544589918 shutdown_connections 2023-10-08T07:16:28.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.138+0000 7f217794c640 1 -- 172.21.15.70:0/544589918 wait complete. 2023-10-08T07:16:28.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.140+0000 7f217794c640 1 Processor -- start 2023-10-08T07:16:28.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.140+0000 7f217794c640 1 -- start start 2023-10-08T07:16:28.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.141+0000 7f217794c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f217006dfc0 0x7f217015d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.141+0000 7f217794c640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2170071070 0x7f217015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.141+0000 7f217794c640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21700afbe0 0x7f21700c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.141+0000 7f217794c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c4150 con 0x7f217006dfc0 2023-10-08T07:16:28.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.141+0000 7f217794c640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c3540 con 0x7f2170071070 2023-10-08T07:16:28.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.141+0000 7f217794c640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21700c3820 con 0x7f21700afbe0 2023-10-08T07:16:28.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.141+0000 7f21756e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f217006dfc0 0x7f217015d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.141+0000 7f21756e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f217006dfc0 0x7f217015d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36818/0 (socket says 172.21.15.70:36818) 2023-10-08T07:16:28.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.141+0000 7f21756e8640 1 -- 172.21.15.70:0/731163713 learned_addr learned my addr 172.21.15.70:0/731163713 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f21756e8640 1 -- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21700afbe0 msgr2=0x7f21700c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f21756e8640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21700afbe0 0x7f21700c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f21756e8640 1 -- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2170071070 msgr2=0x7f217015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f2174ee7640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2170071070 0x7f217015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f21756e8640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2170071070 0x7f217015dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f21756e8640 1 -- 172.21.15.70:0/731163713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168009b30 con 0x7f217006dfc0 2023-10-08T07:16:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f21756e8640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f217006dfc0 0x7f217015d850 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c430 tx=0x7f216c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:28.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f2165ffb640 1 -- 172.21.15.70:0/731163713 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c011790 con 0x7f217006dfc0 2023-10-08T07:16:28.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f2165ffb640 1 -- 172.21.15.70:0/731163713 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c0118f0 con 0x7f217006dfc0 2023-10-08T07:16:28.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f217794c640 1 -- 172.21.15.70:0/731163713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21700c3b00 con 0x7f217006dfc0 2023-10-08T07:16:28.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.142+0000 7f2165ffb640 1 -- 172.21.15.70:0/731163713 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c011ce0 con 0x7f217006dfc0 2023-10-08T07:16:28.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.143+0000 7f217794c640 1 -- 172.21.15.70:0/731163713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21700cdb80 con 0x7f217006dfc0 2023-10-08T07:16:28.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.143+0000 7f217794c640 1 -- 172.21.15.70:0/731163713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217004e160 con 0x7f217006dfc0 2023-10-08T07:16:28.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.145+0000 7f2165ffb640 1 -- 172.21.15.70:0/731163713 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f216c01c460 con 0x7f217006dfc0 2023-10-08T07:16:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.145+0000 7f2165ffb640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f215406af90 0x7f215406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.146+0000 7f2165ffb640 1 -- 172.21.15.70:0/731163713 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f216c089980 con 0x7f217006dfc0 2023-10-08T07:16:28.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.146+0000 7f2174ee7640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f215406af90 0x7f215406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.147+0000 7f2174ee7640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f215406af90 0x7f215406d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2160006fd0 tx=0x7f2160008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:28.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.150+0000 7f2165ffb640 1 -- 172.21.15.70:0/731163713 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f216c054670 con 0x7f217006dfc0 2023-10-08T07:16:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.337+0000 7f217794c640 1 -- 172.21.15.70:0/731163713 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f2170070630 con 0x7f215406af90 2023-10-08T07:16:28.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.338+0000 7f2165ffb640 1 -- 172.21.15.70:0/731163713 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2160004030 con 0x7f215406af90 2023-10-08T07:16:28.373 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 6 to deep-scrub 2023-10-08T07:16:28.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 -- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f215406af90 msgr2=0x7f215406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f215406af90 0x7f215406d450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2160006fd0 tx=0x7f2160008040 comp rx=0 tx=0).stop 2023-10-08T07:16:28.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 -- 172.21.15.70:0/731163713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f217006dfc0 msgr2=0x7f217015d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f217006dfc0 0x7f217015d850 secure :-1 s=READY pgs=3042 cs=0 l=1 rev1=1 crypto rx=0x7f216c00c430 tx=0x7f216c00c8f0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 -- 172.21.15.70:0/731163713 shutdown_connections 2023-10-08T07:16:28.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f215406af90 0x7f215406d450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f21700afbe0 0x7f21700c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f2170071070 0x7f217015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 --2- 172.21.15.70:0/731163713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f217006dfc0 0x7f217015d850 unknown :-1 s=CLOSED pgs=3042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 -- 172.21.15.70:0/731163713 >> 172.21.15.70:0/731163713 conn(0x7f217006a5d0 msgr2=0x7f21700c6200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:28.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 -- 172.21.15.70:0/731163713 shutdown_connections 2023-10-08T07:16:28.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.340+0000 7f217794c640 1 -- 172.21.15.70:0/731163713 wait complete. 2023-10-08T07:16:28.374 INFO:tasks.ceph:Scrubbing osd.7 2023-10-08T07:16:28.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-08T07:16:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.595+0000 7f744e7d0640 1 Processor -- start 2023-10-08T07:16:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.596+0000 7f744e7d0640 1 -- start start 2023-10-08T07:16:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.597+0000 7f744e7d0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f74480b0680 0x7f74481594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.597+0000 7f744e7d0640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7f7448070ac0 0x7f74480c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.597+0000 7f744e7d0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7448071d90 con 0x7f7448156a10 2023-10-08T07:16:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.597+0000 7f744e7d0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7448070ef0 con 0x7f74480b0680 2023-10-08T07:16:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.597+0000 7f744e7d0640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7448071180 con 0x7f7448070ac0 2023-10-08T07:16:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.597+0000 7f744d7ce640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7448156a10 0x7f74480aff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59238/0 (socket says 172.21.15.70:59238) 2023-10-08T07:16:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.597+0000 7f744d7ce640 1 -- 172.21.15.70:0/1317505807 learned_addr learned my addr 172.21.15.70:0/1317505807 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.597+0000 7f744cfcd640 1 --2- 172.21.15.70:0/1317505807 >> v2:172.21.15.70:3300/0 conn(0x7f74480b0680 0x7f74481594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.598+0000 7f744dfcf640 1 --2- 172.21.15.70:0/1317505807 >> v2:172.21.15.181:3300/0 conn(0x7f7448070ac0 0x7f74480c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.598+0000 7f7447fff640 1 -- 172.21.15.70:0/1317505807 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057712120 0 0) 0x7f7448071d90 con 0x7f7448156a10 2023-10-08T07:16:28.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.598+0000 7f7447fff640 1 -- 172.21.15.70:0/1317505807 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7430003520 con 0x7f7448156a10 2023-10-08T07:16:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.598+0000 7f744cfcd640 1 -- 172.21.15.70:0/1317505807 >> v2:172.21.15.181:3300/0 conn(0x7f7448070ac0 msgr2=0x7f74480c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.598+0000 7f744cfcd640 1 --2- 172.21.15.70:0/1317505807 >> v2:172.21.15.181:3300/0 conn(0x7f7448070ac0 0x7f74480c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.598+0000 7f744cfcd640 1 -- 172.21.15.70:0/1317505807 >> v1:172.21.15.70:6789/0 conn(0x7f7448156a10 legacy=0x7f74480aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.598+0000 7f744cfcd640 1 -- 172.21.15.70:0/1317505807 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7448071460 con 0x7f74480b0680 2023-10-08T07:16:28.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.599+0000 7f744cfcd640 1 --2- 172.21.15.70:0/1317505807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74480b0680 0x7f74481594e0 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7f74480c5550 tx=0x7f743c02dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9f174942e7b8cf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.599+0000 7f7447fff640 1 -- 172.21.15.70:0/1317505807 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c02f8a0 con 0x7f74480b0680 2023-10-08T07:16:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.599+0000 7f7447fff640 1 -- 172.21.15.70:0/1317505807 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c02fa00 con 0x7f74480b0680 2023-10-08T07:16:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.599+0000 7f7447fff640 1 -- 172.21.15.70:0/1317505807 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c02fd90 con 0x7f74480b0680 2023-10-08T07:16:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.600+0000 7f744e7d0640 1 -- 172.21.15.70:0/1317505807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74480b0680 msgr2=0x7f74481594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.600+0000 7f744e7d0640 1 --2- 172.21.15.70:0/1317505807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74480b0680 0x7f74481594e0 secure :-1 s=READY pgs=3044 cs=0 l=1 rev1=1 crypto rx=0x7f74480c5550 tx=0x7f743c02dae0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.600+0000 7f744e7d0640 1 -- 172.21.15.70:0/1317505807 shutdown_connections 2023-10-08T07:16:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.600+0000 7f744e7d0640 1 --2- 172.21.15.70:0/1317505807 >> v2:172.21.15.181:3300/0 conn(0x7f7448070ac0 0x7f74480c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.600+0000 7f744e7d0640 1 --2- 172.21.15.70:0/1317505807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74480b0680 0x7f74481594e0 unknown :-1 s=CLOSED pgs=3044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.600+0000 7f744e7d0640 1 -- 172.21.15.70:0/1317505807 >> 172.21.15.70:0/1317505807 conn(0x7f744806c050 msgr2=0x7f744806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.600+0000 7f744e7d0640 1 -- 172.21.15.70:0/1317505807 shutdown_connections 2023-10-08T07:16:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.600+0000 7f744e7d0640 1 -- 172.21.15.70:0/1317505807 wait complete. 2023-10-08T07:16:28.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.602+0000 7f744e7d0640 1 Processor -- start 2023-10-08T07:16:28.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.602+0000 7f744e7d0640 1 -- start start 2023-10-08T07:16:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.603+0000 7f744e7d0640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7448070ac0 0x7f744814d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.603+0000 7f744e7d0640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74480b0680 0x7f74480b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.603+0000 7f744e7d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7448156a10 0x7f74480acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.603+0000 7f744e7d0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744814dc00 con 0x7f7448156a10 2023-10-08T07:16:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.603+0000 7f744e7d0640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480ac910 con 0x7f74480b0680 2023-10-08T07:16:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.603+0000 7f744e7d0640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74480ac2e0 con 0x7f7448070ac0 2023-10-08T07:16:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.603+0000 7f744dfcf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7448156a10 0x7f74480acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.603+0000 7f744dfcf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7448156a10 0x7f74480acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36838/0 (socket says 172.21.15.70:36838) 2023-10-08T07:16:28.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.603+0000 7f744dfcf640 1 -- 172.21.15.70:0/30141335 learned_addr learned my addr 172.21.15.70:0/30141335 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f744d7ce640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7448070ac0 0x7f744814d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f744cfcd640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74480b0680 0x7f74480b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f744dfcf640 1 -- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7448070ac0 msgr2=0x7f744814d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f744dfcf640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f7448070ac0 0x7f744814d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f744dfcf640 1 -- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74480b0680 msgr2=0x7f74480b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f744dfcf640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f74480b0680 0x7f74480b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f744dfcf640 1 -- 172.21.15.70:0/30141335 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c009400 con 0x7f7448156a10 2023-10-08T07:16:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f744dfcf640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7448156a10 0x7f74480acda0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f744000d5d0 tx=0x7f744000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:28.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f7445ffb640 1 -- 172.21.15.70:0/30141335 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400125f0 con 0x7f7448156a10 2023-10-08T07:16:28.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f7445ffb640 1 -- 172.21.15.70:0/30141335 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7440012770 con 0x7f7448156a10 2023-10-08T07:16:28.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74480af4c0 con 0x7f7448156a10 2023-10-08T07:16:28.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f7445ffb640 1 -- 172.21.15.70:0/30141335 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440012b20 con 0x7f7448156a10 2023-10-08T07:16:28.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.604+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74480ac5a0 con 0x7f7448156a10 2023-10-08T07:16:28.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.605+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7448071a20 con 0x7f7448156a10 2023-10-08T07:16:28.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.607+0000 7f7445ffb640 1 -- 172.21.15.70:0/30141335 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f7440012c80 con 0x7f7448156a10 2023-10-08T07:16:28.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.607+0000 7f7445ffb640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f743406af90 0x7f743406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.608+0000 7f7445ffb640 1 -- 172.21.15.70:0/30141335 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f744008a8c0 con 0x7f7448156a10 2023-10-08T07:16:28.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.608+0000 7f7445ffb640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] conn(0x7f7434071a80 0x7f7434073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.608+0000 7f7445ffb640 1 -- 172.21.15.70:0/30141335 --> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74340745b0 con 0x7f7434071a80 2023-10-08T07:16:28.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.608+0000 7f7445ffb640 1 -- 172.21.15.70:0/30141335 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=281) v2 ==== 24+0+0 (secure 0 0 0) 0x7f744008ac30 con 0x7f7448156a10 2023-10-08T07:16:28.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.608+0000 7f744d7ce640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f743406af90 0x7f743406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.609+0000 7f744cfcd640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] conn(0x7f7434071a80 0x7f7434073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.610+0000 7f744cfcd640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] conn(0x7f7434071a80 0x7f7434073ee0 crc :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:28.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.610+0000 7f744d7ce640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f743406af90 0x7f743406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7438001560 tx=0x7f7438009040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:28.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.612+0000 7f7445ffb640 1 -- 172.21.15.70:0/30141335 <== osd.7 v2:172.21.15.181:6816/1658798817 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f74340745b0 con 0x7f7434071a80 2023-10-08T07:16:28.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.626+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 --> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7448071740 con 0x7f7434071a80 2023-10-08T07:16:28.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.627+0000 7f7445ffb640 1 -- 172.21.15.70:0/30141335 <== osd.7 v2:172.21.15.181:6816/1658798817 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f743c040040 con 0x7f7434071a80 2023-10-08T07:16:28.664 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-10-08T07:16:28.664 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T07:16:28.664 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-10-08T07:16:28.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.627+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] conn(0x7f7434071a80 msgr2=0x7f7434073ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.627+0000 7f744e7d0640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:6816/1658798817,v1:172.21.15.181:6817/1658798817] conn(0x7f7434071a80 0x7f7434073ee0 crc :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.627+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f743406af90 msgr2=0x7f743406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.627+0000 7f744e7d0640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f743406af90 0x7f743406d450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7438001560 tx=0x7f7438009040 comp rx=0 tx=0).stop 2023-10-08T07:16:28.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.627+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7448156a10 msgr2=0x7f74480acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.627+0000 7f744e7d0640 1 --2- 172.21.15.70:0/30141335 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7448156a10 0x7f74480acda0 secure :-1 s=READY pgs=3045 cs=0 l=1 rev1=1 crypto rx=0x7f744000d5d0 tx=0x7f744000da90 comp rx=0 tx=0).stop 2023-10-08T07:16:28.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.627+0000 7f744dfcf640 1 -- 172.21.15.70:0/30141335 reap_dead start 2023-10-08T07:16:28.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.628+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 shutdown_connections 2023-10-08T07:16:28.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.628+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 >> 172.21.15.70:0/30141335 conn(0x7f744806c050 msgr2=0x7f744815a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:28.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.628+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 shutdown_connections 2023-10-08T07:16:28.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.628+0000 7f744e7d0640 1 -- 172.21.15.70:0/30141335 wait complete. 2023-10-08T07:16:28.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-08T07:16:28.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.887+0000 7f1a0bc4f640 1 Processor -- start 2023-10-08T07:16:28.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.887+0000 7f1a0bc4f640 1 -- start start 2023-10-08T07:16:28.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.889+0000 7f1a0bc4f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1a040c9cf0 0x7f1a0406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.889+0000 7f1a0bc4f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0415b310 con 0x7f1a0415aa00 2023-10-08T07:16:28.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.889+0000 7f1a0bc4f640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0406d290 con 0x7f1a0406c3a0 2023-10-08T07:16:28.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.889+0000 7f1a0bc4f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0406e0a0 con 0x7f1a040c9cf0 2023-10-08T07:16:28.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.889+0000 7f1a0a1ec640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1a0415aa00 0x7f1a040b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59240/0 (socket says 172.21.15.70:59240) 2023-10-08T07:16:28.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.889+0000 7f1a0a1ec640 1 -- 172.21.15.70:0/2574059171 learned_addr learned my addr 172.21.15.70:0/2574059171 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:28.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.889+0000 7f1a091ea640 1 --2- 172.21.15.70:0/2574059171 >> v2:172.21.15.70:3300/0 conn(0x7f1a040c9cf0 0x7f1a0406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.890+0000 7f1a091ea640 1 -- 172.21.15.70:0/2574059171 >> v1:172.21.15.181:6789/0 conn(0x7f1a0406c3a0 legacy=0x7f1a0406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:28.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.890+0000 7f1a091ea640 1 -- 172.21.15.70:0/2574059171 >> v1:172.21.15.70:6789/0 conn(0x7f1a0415aa00 legacy=0x7f1a040b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.890+0000 7f1a091ea640 1 -- 172.21.15.70:0/2574059171 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a040c9860 con 0x7f1a040c9cf0 2023-10-08T07:16:28.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.891+0000 7f1a091ea640 1 --2- 172.21.15.70:0/2574059171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 0x7f1a0406db60 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f19f80097a0 tx=0x7f19f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a3da3d2e0dfbf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:28.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.891+0000 7f1a089e9640 1 -- 172.21.15.70:0/2574059171 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8031830 con 0x7f1a040c9cf0 2023-10-08T07:16:28.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.891+0000 7f1a089e9640 1 -- 172.21.15.70:0/2574059171 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f8031990 con 0x7f1a040c9cf0 2023-10-08T07:16:28.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.891+0000 7f1a089e9640 1 -- 172.21.15.70:0/2574059171 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8031d20 con 0x7f1a040c9cf0 2023-10-08T07:16:28.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.892+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/2574059171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 msgr2=0x7f1a0406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.892+0000 7f1a0bc4f640 1 --2- 172.21.15.70:0/2574059171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 0x7f1a0406db60 secure :-1 s=READY pgs=3047 cs=0 l=1 rev1=1 crypto rx=0x7f19f80097a0 tx=0x7f19f802b8d0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.892+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/2574059171 shutdown_connections 2023-10-08T07:16:28.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.892+0000 7f1a0bc4f640 1 --2- 172.21.15.70:0/2574059171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 0x7f1a0406db60 unknown :-1 s=CLOSED pgs=3047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.892+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/2574059171 >> 172.21.15.70:0/2574059171 conn(0x7f1a0406a5d0 msgr2=0x7f1a040b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:28.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.892+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/2574059171 shutdown_connections 2023-10-08T07:16:28.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.892+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/2574059171 wait complete. 2023-10-08T07:16:28.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.894+0000 7f1a0bc4f640 1 Processor -- start 2023-10-08T07:16:28.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.894+0000 7f1a0bc4f640 1 -- start start 2023-10-08T07:16:28.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a0bc4f640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a0406c3a0 0x7f1a0414c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a0bc4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 0x7f1a040cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a0bc4f640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a0415aa00 0x7f1a040cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a0bc4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0414e380 con 0x7f1a040c9cf0 2023-10-08T07:16:28.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a0bc4f640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040c6d30 con 0x7f1a0415aa00 2023-10-08T07:16:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a0bc4f640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a040c2d90 con 0x7f1a0406c3a0 2023-10-08T07:16:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a091ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 0x7f1a040cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a091ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 0x7f1a040cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36854/0 (socket says 172.21.15.70:36854) 2023-10-08T07:16:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a091ea640 1 -- 172.21.15.70:0/1495683516 learned_addr learned my addr 172.21.15.70:0/1495683516 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a091ea640 1 -- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a0406c3a0 msgr2=0x7f1a0414c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a091ea640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a0406c3a0 0x7f1a0414c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a091ea640 1 -- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a0415aa00 msgr2=0x7f1a040cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a091ea640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a0415aa00 0x7f1a040cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.895+0000 7f1a091ea640 1 -- 172.21.15.70:0/1495683516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f8009400 con 0x7f1a040c9cf0 2023-10-08T07:16:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.896+0000 7f1a091ea640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 0x7f1a040cd690 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f19f802bee0 tx=0x7f19f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:28.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.896+0000 7f1a027fc640 1 -- 172.21.15.70:0/1495683516 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f803ba70 con 0x7f1a040c9cf0 2023-10-08T07:16:28.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.896+0000 7f1a027fc640 1 -- 172.21.15.70:0/1495683516 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f803bbd0 con 0x7f1a040c9cf0 2023-10-08T07:16:28.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.896+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/1495683516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0414d930 con 0x7f1a040c9cf0 2023-10-08T07:16:28.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.896+0000 7f1a027fc640 1 -- 172.21.15.70:0/1495683516 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f80453f0 con 0x7f1a040c9cf0 2023-10-08T07:16:28.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.896+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/1495683516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0414def0 con 0x7f1a040c9cf0 2023-10-08T07:16:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.897+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/1495683516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0404e160 con 0x7f1a040c9cf0 2023-10-08T07:16:28.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.898+0000 7f1a027fc640 1 -- 172.21.15.70:0/1495683516 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f19f8045550 con 0x7f1a040c9cf0 2023-10-08T07:16:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.900+0000 7f1a027fc640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f19f006afc0 0x7f19f006d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:28.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.900+0000 7f1a027fc640 1 -- 172.21.15.70:0/1495683516 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f19f80abeb0 con 0x7f1a040c9cf0 2023-10-08T07:16:28.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.900+0000 7f1a099eb640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f19f006afc0 0x7f19f006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:28.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.902+0000 7f1a099eb640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f19f006afc0 0x7f19f006d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f19f40043c0 tx=0x7f19f4007d30 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:28.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:28.904+0000 7f1a027fc640 1 -- 172.21.15.70:0/1495683516 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19f8076c40 con 0x7f1a040c9cf0 2023-10-08T07:16:29.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.093+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/1495683516 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a040c8f30 con 0x7f19f006afc0 2023-10-08T07:16:29.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.095+0000 7f1a027fc640 1 -- 172.21.15.70:0/1495683516 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f19f4004060 con 0x7f19f006afc0 2023-10-08T07:16:29.129 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 7 to deep-scrub 2023-10-08T07:16:29.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f19f006afc0 msgr2=0x7f19f006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:29.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f19f006afc0 0x7f19f006d480 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f19f40043c0 tx=0x7f19f4007d30 comp rx=0 tx=0).stop 2023-10-08T07:16:29.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/1495683516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 msgr2=0x7f1a040cd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:29.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 0x7f1a040cd690 secure :-1 s=READY pgs=3048 cs=0 l=1 rev1=1 crypto rx=0x7f19f802bee0 tx=0x7f19f80046c0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/1495683516 shutdown_connections 2023-10-08T07:16:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f19f006afc0 0x7f19f006d480 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f1a0415aa00 0x7f1a040cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a040c9cf0 0x7f1a040cd690 unknown :-1 s=CLOSED pgs=3048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 --2- 172.21.15.70:0/1495683516 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f1a0406c3a0 0x7f1a0414c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/1495683516 >> 172.21.15.70:0/1495683516 conn(0x7f1a0406a5d0 msgr2=0x7f1a04157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:29.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.097+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/1495683516 shutdown_connections 2023-10-08T07:16:29.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.098+0000 7f1a0bc4f640 1 -- 172.21.15.70:0/1495683516 wait complete. 2023-10-08T07:16:29.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:16:29.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.350+0000 7fb041475640 1 Processor -- start 2023-10-08T07:16:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.350+0000 7fb041475640 1 -- start start 2023-10-08T07:16:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.352+0000 7fb041475640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c1599d0 0x7fb03c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.352+0000 7fb041475640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb03c0b07d0 0x7fb03c06c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.352+0000 7fb041475640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c0cacf0 con 0x7fb03c0cc210 2023-10-08T07:16:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.352+0000 7fb041475640 1 -- --> v2:172.21.15.181:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c0b10f0 con 0x7fb03c0b07d0 2023-10-08T07:16:29.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.352+0000 7fb041475640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0ca3c0 con 0x7fb03c1599d0 2023-10-08T07:16:29.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.353+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c1599d0 0x7fb03c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:29.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.353+0000 7fb040c74640 1 --2- >> v2:172.21.15.181:3300/0 conn(0x7fb03c0b07d0 0x7fb03c06c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:29.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.353+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c1599d0 0x7fb03c0ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.181:3301/0 says I am v2:172.21.15.70:49514/0 (socket says 172.21.15.70:49514) 2023-10-08T07:16:29.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.353+0000 7fb03bfff640 1 -- 172.21.15.70:0/3717684881 learned_addr learned my addr 172.21.15.70:0/3717684881 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:29.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.353+0000 7fb040c74640 1 -- 172.21.15.70:0/3717684881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c1599d0 msgr2=0x7fb03c0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:29.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.353+0000 7fb040c74640 1 --2- 172.21.15.70:0/3717684881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c1599d0 0x7fb03c0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.353+0000 7fb040c74640 1 -- 172.21.15.70:0/3717684881 >> v1:172.21.15.181:6789/0 conn(0x7fb03c0cc210 legacy=0x7fb03c0b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:29.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.353+0000 7fb040c74640 1 -- 172.21.15.70:0/3717684881 --> v2:172.21.15.181:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0ca6a0 con 0x7fb03c0b07d0 2023-10-08T07:16:29.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.353+0000 7fb03bfff640 1 --2- 172.21.15.70:0/3717684881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c1599d0 0x7fb03c0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T07:16:29.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.355+0000 7fb040c74640 1 --2- 172.21.15.70:0/3717684881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c0b07d0 0x7fb03c06c310 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb034009d90 tx=0x7fb03402e0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec9cca1520beede9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:29.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.355+0000 7fb03affd640 1 -- 172.21.15.70:0/3717684881 <== mon.1 v2:172.21.15.181:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03402ee70 con 0x7fb03c0b07d0 2023-10-08T07:16:29.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.355+0000 7fb03affd640 1 -- 172.21.15.70:0/3717684881 <== mon.1 v2:172.21.15.181:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb034036440 con 0x7fb03c0b07d0 2023-10-08T07:16:29.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.355+0000 7fb03affd640 1 -- 172.21.15.70:0/3717684881 <== mon.1 v2:172.21.15.181:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340367d0 con 0x7fb03c0b07d0 2023-10-08T07:16:29.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.356+0000 7fb041475640 1 -- 172.21.15.70:0/3717684881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c0b07d0 msgr2=0x7fb03c06c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:29.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.356+0000 7fb041475640 1 --2- 172.21.15.70:0/3717684881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c0b07d0 0x7fb03c06c310 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb034009d90 tx=0x7fb03402e0c0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.356+0000 7fb041475640 1 -- 172.21.15.70:0/3717684881 shutdown_connections 2023-10-08T07:16:29.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.356+0000 7fb041475640 1 --2- 172.21.15.70:0/3717684881 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c0b07d0 0x7fb03c06c310 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.356+0000 7fb041475640 1 --2- 172.21.15.70:0/3717684881 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c1599d0 0x7fb03c0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.356+0000 7fb041475640 1 -- 172.21.15.70:0/3717684881 >> 172.21.15.70:0/3717684881 conn(0x7fb03c0cb0c0 msgr2=0x7fb03c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:29.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.356+0000 7fb041475640 1 -- 172.21.15.70:0/3717684881 shutdown_connections 2023-10-08T07:16:29.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.356+0000 7fb041475640 1 -- 172.21.15.70:0/3717684881 wait complete. 2023-10-08T07:16:29.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.358+0000 7fb041475640 1 Processor -- start 2023-10-08T07:16:29.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.358+0000 7fb041475640 1 -- start start 2023-10-08T07:16:29.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.359+0000 7fb041475640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c0b07d0 0x7fb03c15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:29.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.359+0000 7fb041475640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c0cc210 0x7fb03c1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:29.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.359+0000 7fb041475640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb03c1599d0 0x7fb03c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:29.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.359+0000 7fb041475640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c15d570 con 0x7fb03c1599d0 2023-10-08T07:16:29.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.359+0000 7fb041475640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c15cfb0 con 0x7fb03c0cc210 2023-10-08T07:16:29.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.359+0000 7fb041475640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c15d290 con 0x7fb03c0b07d0 2023-10-08T07:16:29.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.359+0000 7fb040c74640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb03c1599d0 0x7fb03c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:29.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.359+0000 7fb040c74640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb03c1599d0 0x7fb03c06cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36858/0 (socket says 172.21.15.70:36858) 2023-10-08T07:16:29.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.359+0000 7fb040c74640 1 -- 172.21.15.70:0/2229885836 learned_addr learned my addr 172.21.15.70:0/2229885836 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:29.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb03bfff640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c0b07d0 0x7fb03c15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:29.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb03b7fe640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c0cc210 0x7fb03c1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:29.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb040c74640 1 -- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c0b07d0 msgr2=0x7fb03c15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:29.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb040c74640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c0b07d0 0x7fb03c15c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb040c74640 1 -- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c0cc210 msgr2=0x7fb03c1508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:29.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb040c74640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c0cc210 0x7fb03c1508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb040c74640 1 -- 172.21.15.70:0/2229885836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c0094d0 con 0x7fb03c1599d0 2023-10-08T07:16:29.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb040c74640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb03c1599d0 0x7fb03c06cb10 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7fb034009d90 tx=0x7fb034004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:29.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb038ff9640 1 -- 172.21.15.70:0/2229885836 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03403f850 con 0x7fb03c1599d0 2023-10-08T07:16:29.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb038ff9640 1 -- 172.21.15.70:0/2229885836 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03403f9b0 con 0x7fb03c1599d0 2023-10-08T07:16:29.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb041475640 1 -- 172.21.15.70:0/2229885836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb034009a70 con 0x7fb03c1599d0 2023-10-08T07:16:29.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.360+0000 7fb038ff9640 1 -- 172.21.15.70:0/2229885836 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03403fd60 con 0x7fb03c1599d0 2023-10-08T07:16:29.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.361+0000 7fb041475640 1 -- 172.21.15.70:0/2229885836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c0c6be0 con 0x7fb03c1599d0 2023-10-08T07:16:29.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.362+0000 7fb041475640 1 -- 172.21.15.70:0/2229885836 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c0c6760 con 0x7fb03c1599d0 2023-10-08T07:16:29.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.363+0000 7fb038ff9640 1 -- 172.21.15.70:0/2229885836 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fb03403d070 con 0x7fb03c1599d0 2023-10-08T07:16:29.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.364+0000 7fb038ff9640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb02806af90 0x7fb02806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:29.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.364+0000 7fb038ff9640 1 -- 172.21.15.70:0/2229885836 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fb0340ac0f0 con 0x7fb03c1599d0 2023-10-08T07:16:29.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.364+0000 7fb03bfff640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb02806af90 0x7fb02806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:29.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.366+0000 7fb03bfff640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb02806af90 0x7fb02806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009b70 tx=0x7fb02c008040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:29.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.369+0000 7fb038ff9640 1 -- 172.21.15.70:0/2229885836 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb03c0c6760 con 0x7fb03c1599d0 2023-10-08T07:16:29.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.557+0000 7fb041475640 1 -- 172.21.15.70:0/2229885836 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb03c0ca070 con 0x7fb02806af90 2023-10-08T07:16:29.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.564+0000 7fb038ff9640 1 -- 172.21.15.70:0/2229885836 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48232 (secure 0 0 0) 0x7fb02c0109f0 con 0x7fb02806af90 2023-10-08T07:16:29.567 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:29.567 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:16:29.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.567+0000 7fb041475640 1 -- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb02806af90 msgr2=0x7fb02806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:29.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.567+0000 7fb041475640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb02806af90 0x7fb02806d450 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009b70 tx=0x7fb02c008040 comp rx=0 tx=0).stop 2023-10-08T07:16:29.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.567+0000 7fb041475640 1 -- 172.21.15.70:0/2229885836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb03c1599d0 msgr2=0x7fb03c06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:29.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.567+0000 7fb041475640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb03c1599d0 0x7fb03c06cb10 secure :-1 s=READY pgs=3049 cs=0 l=1 rev1=1 crypto rx=0x7fb034009d90 tx=0x7fb034004a40 comp rx=0 tx=0).stop 2023-10-08T07:16:29.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.567+0000 7fb041475640 1 -- 172.21.15.70:0/2229885836 shutdown_connections 2023-10-08T07:16:29.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.567+0000 7fb041475640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fb02806af90 0x7fb02806d450 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.567+0000 7fb041475640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb03c1599d0 0x7fb03c06cb10 unknown :-1 s=CLOSED pgs=3049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.567+0000 7fb041475640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fb03c0cc210 0x7fb03c1508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.567+0000 7fb041475640 1 --2- 172.21.15.70:0/2229885836 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fb03c0b07d0 0x7fb03c15c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:29.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.567+0000 7fb041475640 1 -- 172.21.15.70:0/2229885836 >> 172.21.15.70:0/2229885836 conn(0x7fb03c0cb0c0 msgr2=0x7fb03c0b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:29.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.568+0000 7fb041475640 1 -- 172.21.15.70:0/2229885836 shutdown_connections 2023-10-08T07:16:29.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:29.568+0000 7fb041475640 1 -- 172.21.15.70:0/2229885836 wait complete. 2023-10-08T07:16:29.589 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2023-10-08T07:16:28.904876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4883,"ondisk_log_size":4883,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":453300,"kb_used_data":28936,"kb_used_omap":12,"kb_used_meta":424243,"kb_avail":754521420,"statfs":{"total":773094113280,"available":772629934080,"internally_reserved":0,"allocated":29630464,"data_stored":16083456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":434425403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003411"},"pg_stats":[{"pgid":"1.0","version":"278'4883","reported_seq":6010,"reported_epoch":280,"state":"active+clean","last_fresh":"2023-10-08T07:16:15.136877+0000","last_change":"2023-10-08T05:36:07.428737+0000","last_active":"2023-10-08T07:16:15.136877+0000","last_peered":"2023-10-08T07:16:15.136877+0000","last_clean":"2023-10-08T07:16:15.136877+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:16:15.136877+0000","last_undegraded":"2023-10-08T07:16:15.136877+0000","last_fullsized":"2023-10-08T07:16:15.136877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T05:36:05.849165+0000","last_clean_scrub_stamp":"2023-10-08T05:36:05.849165+0000","objects_scrubbed":0,"log_size":4883,"log_dups_size":0,"ondisk_log_size":4883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T15:27:00.873447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4883,"ondisk_log_size":4883,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576039,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75488,"kb_used_data":4052,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296352,"statfs":{"total":96636764160,"available":96559464448,"internally_reserved":0,"allocated":4149248,"data_stored":2453208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1419999999999999,"15min":1.198},"min":{"1min":0.94599999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.625,"5min":1.9410000000000001,"15min":2.008},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.28,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.82899999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.016}]},{"osd":1,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.244,"15min":1.248},"min":{"1min":0.88400000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.964,"5min":1.964,"15min":1.998},"last":1.722},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.3320000000000001},"min":{"1min":0.86299999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.3460000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2190000000000001,"15min":1.286},"min":{"1min":0.86499999999999999,"5min":0.86399999999999999,"15min":0.77600000000000002},"max":{"1min":1.573,"5min":1.8440000000000001,"15min":2.1619999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.383,"15min":1.3540000000000001},"min":{"1min":0.77700000000000002,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.1190000000000002},"last":1.6539999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3300000000000001,"15min":1.341},"min":{"1min":0.96299999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.615,"5min":1.4330000000000001,"15min":1.4119999999999999},"min":{"1min":1.117,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.5209999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.3839999999999999,"15min":1.339},"min":{"1min":1.0660000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.46,"15min":1.4259999999999999},"min":{"1min":1.294,"5min":0.752,"15min":0.752},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.302}]},{"osd":5,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.53,"15min":1.4770000000000001},"min":{"1min":1.2490000000000001,"5min":0.80600000000000005,"15min":0.79300000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.5409999999999999,"15min":1.514},"min":{"1min":1.242,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":1.621}]},{"osd":6,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.8999999999999999,"5min":1.6619999999999999,"15min":1.619},"min":{"1min":1.1639999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.8240000000000001,"5min":1.629,"15min":1.5649999999999999},"min":{"1min":1.2090000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":1.5820000000000001}]}]},{"osd":6,"up_from":13,"seq":55834576038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48540,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323300,"statfs":{"total":96636764160,"available":96587059200,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2370000000000001,"15min":1.274},"min":{"1min":0.76100000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5600000000000001,"5min":1.78,"15min":1.9490000000000001},"last":1.494},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2949999999999999,"15min":1.3460000000000001},"min":{"1min":0.84299999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7370000000000001,"5min":1.76,"15min":2.0070000000000001},"last":1.532}]},{"osd":1,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.121,"15min":1.2070000000000001},"min":{"1min":0.68200000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.6020000000000001,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.02},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2310000000000001,"15min":1.236},"min":{"1min":0.73999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6619999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.605}]},{"osd":2,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1699999999999999,"15min":1.2210000000000001},"min":{"1min":0.71599999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.956,"5min":1.956,"15min":2.339},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.246,"15min":1.294},"min":{"1min":0.83499999999999996,"5min":0.748,"15min":0.748},"max":{"1min":1.484,"5min":1.8720000000000001,"15min":2.1779999999999999},"last":1.3320000000000001}]},{"osd":3,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.391,"15min":1.3979999999999999},"min":{"1min":0.95799999999999996,"5min":0.86299999999999999,"15min":0.73099999999999998},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.556},{"interface":"front","average":{"1min":1.45,"5min":1.421,"15min":1.4770000000000001},"min":{"1min":0.998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.8440000000000001,"5min":1.8979999999999999,"15min":2.1459999999999999},"last":1.587}]},{"osd":4,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4450000000000001,"15min":1.415},"min":{"1min":1.048,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8140000000000001,"5min":2.0510000000000002,"15min":2.0859999999999999},"last":1.302},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.4850000000000001,"15min":1.474},"min":{"1min":1.01,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.484},"last":1.3680000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.403,"15min":1.4299999999999999},"min":{"1min":0.90200000000000002,"5min":0.75,"15min":0.75},"max":{"1min":2.093,"5min":2.093,"15min":2.403},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.5960000000000001,"15min":1.5700000000000001},"min":{"1min":1.1259999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1320000000000001,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":1.4119999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.5940000000000001,"15min":1.595},"min":{"1min":1.0209999999999999,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.1469999999999998,"5min":2.339,"15min":2.5870000000000002},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.5820000000000001,"15min":1.571},"min":{"1min":1.145,"5min":0.87,"15min":0.87},"max":{"1min":2.2370000000000001,"5min":2.2959999999999998,"15min":2.665},"last":1.2609999999999999}]}]},{"osd":5,"up_from":13,"seq":55834576038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61024,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310816,"statfs":{"total":96636764160,"available":96574275584,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1499999999999999,"15min":1.202},"min":{"1min":0.748,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5740000000000001,"5min":1.98,"15min":1.98},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.1910000000000001,"15min":1.2190000000000001},"min":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.74,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":0.86399999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.165,"15min":1.218},"min":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"max":{"1min":1.5449999999999999,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.2849999999999999,"15min":1.3100000000000001},"min":{"1min":0.78800000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.752,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":0.76600000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.254,"15min":1.2969999999999999},"min":{"1min":0.77100000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.7050000000000001,"5min":2.008,"15min":2.0870000000000002},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2430000000000001,"15min":1.29},"min":{"1min":0.77600000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.677,"5min":1.877,"15min":1.905},"last":1.129}]},{"osd":3,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.97699999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.694,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":1.22},{"interface":"front","average":{"1min":1.355,"5min":1.4219999999999999,"15min":1.4259999999999999},"min":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.883,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":1.1779999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.329,"15min":1.3300000000000001},"min":{"1min":0.93600000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.857,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.601,"5min":1.4910000000000001,"15min":1.4399999999999999},"min":{"1min":1.194,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.1600000000000001,"5min":2.742,"15min":2.742},"last":1.284}]},{"osd":6,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5780000000000001,"15min":1.4890000000000001},"min":{"1min":1.036,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.9450000000000001,"5min":13.234999999999999,"15min":13.234999999999999},"last":1.339},{"interface":"front","average":{"1min":1.585,"5min":1.5449999999999999,"15min":1.4870000000000001},"min":{"1min":1.1830000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9770000000000001,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":1.2450000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:24 2023","interfaces":[{"interface":"back","average":{"1min":1.643,"5min":1.5649999999999999,"15min":1.548},"min":{"1min":1.208,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.0979999999999999,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.421},{"interface":"front","average":{"1min":1.669,"5min":1.609,"15min":1.6040000000000001},"min":{"1min":1.3260000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.02,"5min":2.351,"15min":2.351},"last":1.617}]}]},{"osd":4,"up_from":13,"seq":55834576038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52704,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319136,"statfs":{"total":96636764160,"available":96582795264,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.321,"15min":1.3520000000000001},"min":{"1min":0.67800000000000005,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":3.3220000000000001,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4359999999999999,"15min":1.423},"min":{"1min":0.63600000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":4.3609999999999998,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.6619999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2470000000000001,"15min":1.2849999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":4.2229999999999999,"5min":4.2229999999999999,"15min":4.2229999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":2.7799999999999998,"5min":1.635,"15min":1.371},"min":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":28.907,"5min":28.907,"15min":28.907},"last":1.8460000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4810000000000001,"15min":1.335},"min":{"1min":0.70899999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":3.6120000000000001,"5min":17.175999999999998,"15min":17.175999999999998},"last":1.129},{"interface":"front","average":{"1min":1.496,"5min":1.444,"15min":1.391},"min":{"1min":0.59499999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.4080000000000004,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":1.355}]},{"osd":3,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.3260000000000001,"5min":1.5720000000000001,"15min":1.4159999999999999},"min":{"1min":0.60099999999999998,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":15.938000000000001,"5min":15.938000000000001,"15min":15.938000000000001},"last":1.538},{"interface":"front","average":{"1min":1.8069999999999999,"5min":1.5760000000000001,"15min":1.474},"min":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.787000000000001,"5min":10.787000000000001,"15min":10.787000000000001},"last":1.401}]},{"osd":5,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":1.6439999999999999,"5min":1.591,"15min":1.4350000000000001},"min":{"1min":0.73199999999999998,"5min":0.372,"15min":0.372},"max":{"1min":4.2939999999999996,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.458},{"interface":"front","average":{"1min":3.0209999999999999,"5min":1.8160000000000001,"15min":1.5289999999999999},"min":{"1min":0.84899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":31.347999999999999,"5min":31.347999999999999,"15min":31.347999999999999},"last":1.917}]},{"osd":6,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.016,"5min":1.7290000000000001,"15min":1.5389999999999999},"min":{"1min":0.73699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":13.885,"5min":13.885,"15min":13.885},"last":1.585},{"interface":"front","average":{"1min":2.117,"5min":1.833,"15min":1.6419999999999999},"min":{"1min":0.82499999999999996,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":13.446999999999999,"5min":13.446999999999999,"15min":13.446999999999999},"last":1.772}]},{"osd":7,"last update":"Sun Oct 8 07:15:25 2023","interfaces":[{"interface":"back","average":{"1min":2.528,"5min":1.821,"15min":1.611},"min":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":15.843999999999999,"5min":15.843999999999999,"15min":15.843999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.7050000000000001,"5min":1.726,"15min":1.6659999999999999},"min":{"1min":0.82399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":3.968,"5min":5.609,"15min":5.609},"last":1.804}]}]},{"osd":3,"up_from":13,"seq":55834576040,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56860,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314980,"statfs":{"total":96636764160,"available":96578539520,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.96499999999999997},"min":{"1min":0.51600000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.323,"5min":1.6719999999999999,"15min":2.7599999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.046,"15min":1.1439999999999999},"min":{"1min":0.55300000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.72,"5min":1.72,"15min":4.5659999999999998},"last":1.2729999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":1.0069999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1699999999999999,"15min":1.2609999999999999},"min":{"1min":0.58699999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.504,"5min":2.109,"15min":2.4889999999999999},"last":1.5760000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.032,"15min":1.1220000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.437},"max":{"1min":1.53,"5min":2.165,"15min":2.2250000000000001},"last":1.333},{"interface":"front","average":{"1min":1.079,"5min":1.119,"15min":1.1850000000000001},"min":{"1min":0.67800000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6200000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.2110000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2170000000000001,"15min":1.268},"min":{"1min":0.93899999999999995,"5min":0.70899999999999996,"15min":0.67500000000000004},"max":{"1min":1.732,"5min":1.732,"15min":2.5379999999999998},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.3680000000000001,"15min":1.4410000000000001},"min":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":2.306,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.149}]},{"osd":5,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2989999999999999,"15min":1.355},"min":{"1min":0.79800000000000004,"5min":0.755,"15min":0.61799999999999999},"max":{"1min":1.9079999999999999,"5min":1.923,"15min":2.9740000000000002},"last":1.526},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.393,"15min":1.4299999999999999},"min":{"1min":0.86599999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.7330000000000001},"last":1.4339999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3740000000000001,"15min":1.405},"min":{"1min":1.0569999999999999,"5min":0.82299999999999995,"15min":0.59699999999999998},"max":{"1min":1.825,"5min":1.97,"15min":2.653},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4390000000000001,"15min":1.504},"min":{"1min":1.1140000000000001,"5min":0.872,"15min":0.57499999999999996},"max":{"1min":1.7749999999999999,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.649}]},{"osd":7,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.488,"15min":1.5529999999999999},"min":{"1min":0.91000000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.0590000000000002,"5min":2.3220000000000001,"15min":2.8650000000000002},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.4359999999999999,"15min":1.506},"min":{"1min":0.66700000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.8340000000000001,"5min":1.903,"15min":2.7749999999999999},"last":1.3939999999999999}]}]},{"osd":2,"up_from":13,"seq":55834576038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52700,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319140,"statfs":{"total":96636764160,"available":96582799360,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85299999999999998,"15min":0.94199999999999995},"min":{"1min":0.38100000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.3029999999999999,"5min":1.601,"15min":1.948},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.97699999999999998,"15min":1.0369999999999999},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.482,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.95299999999999996}]},{"osd":1,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.88700000000000001,"15min":0.95799999999999996},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.363,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.0309999999999999,"15min":1.147},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":1.4390000000000001,"5min":1.7789999999999999,"15min":1.9390000000000001},"last":0.80300000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0580000000000001,"15min":1.145},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.421,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":0.998},{"interface":"front","average":{"1min":1.004,"5min":1.123,"15min":1.2290000000000001},"min":{"1min":0.54600000000000004,"5min":0.47699999999999998,"15min":0.47599999999999998},"max":{"1min":1.54,"5min":1.6799999999999999,"15min":2.0670000000000002},"last":0.71499999999999997}]},{"osd":4,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3280000000000001,"15min":1.409},"min":{"1min":0.78300000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.754,"5min":1.992,"15min":2.1789999999999998},"last":1.179},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.256,"15min":1.298},"min":{"1min":0.78700000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.655,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.1200000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.298,"15min":1.365},"min":{"1min":0.75600000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.972,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.2},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3879999999999999,"15min":1.488},"min":{"1min":0.72499999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8939999999999999,"5min":2.0750000000000002,"15min":2.9390000000000001},"last":1.238}]},{"osd":6,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.306,"15min":1.371},"min":{"1min":0.72399999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.7370000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.4059999999999999,"15min":1.4910000000000001},"min":{"1min":0.82699999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.891,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.3620000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.379,"15min":1.4470000000000001},"min":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8380000000000001,"5min":2.3969999999999998,"15min":2.4689999999999999},"last":1.7889999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.454,"15min":1.538},"min":{"1min":0.90800000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.875,"5min":2.831,"15min":2.831},"last":1.3169999999999999}]}]},{"osd":1,"up_from":13,"seq":55834576038,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52700,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319140,"statfs":{"total":96636764160,"available":96582799360,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.442,"5min":0.439,"15min":0.32100000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0489999999999999,"15min":1.177},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.41899999999999998},"max":{"1min":1.3939999999999999,"5min":1.7010000000000001,"15min":1.79},"last":1.3819999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.86099999999999999,"15min":0.93500000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.462,"15min":1.641},"last":1.3140000000000001},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.083,"15min":1.1299999999999999},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.372},"max":{"1min":1.4750000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.93000000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.151,"15min":1.2250000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.621,"5min":2.496,"15min":2.496},"last":1.506},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.129},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6859999999999999,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":1.0900000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2529999999999999,"15min":1.333},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4339999999999999,"5min":2.73,"15min":2.8460000000000001},"last":1.605},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2410000000000001,"15min":1.2889999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.744,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.528}]},{"osd":5,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3089999999999999,"15min":1.355},"min":{"1min":0.85599999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9910000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.518},"min":{"1min":0.98799999999999999,"5min":0.89600000000000002,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":2.601,"15min":2.601},"last":1.2509999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.4019999999999999,"15min":1.482},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.9710000000000001,"5min":2.883,"15min":2.9079999999999999},"last":1.71},{"interface":"front","average":{"1min":1.345,"5min":1.4039999999999999,"15min":1.474},"min":{"1min":0.752,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8939999999999999,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.653}]},{"osd":7,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.482,"15min":1.5329999999999999},"min":{"1min":0.85099999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1000000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.577},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4630000000000001,"15min":1.5429999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.0920000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.7969999999999999}]}]},{"osd":0,"up_from":13,"seq":55834576039,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53284,"kb_used_data":4052,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318556,"statfs":{"total":96636764160,"available":96582201344,"internally_reserved":0,"allocated":4149248,"data_stored":2453208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.86799999999999999,"15min":0.94299999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3029999999999999,"5min":1.3859999999999999,"15min":1.581},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96399999999999997,"15min":1.0620000000000001},"min":{"1min":0.38500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":0.58399999999999996}]},{"osd":2,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.03,"15min":1.1100000000000001},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.42199999999999999},"max":{"1min":4.1920000000000002,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.999,"5min":0.97999999999999998,"15min":1.052},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.4699999999999998,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":0.52700000000000002}]},{"osd":3,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.099,"15min":1.2030000000000001},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.425,"5min":1.633,"15min":2.0139999999999998},"last":1.006},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.089,"15min":1.143},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.327,"5min":4.327,"15min":4.327},"last":0.64300000000000002}]},{"osd":4,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.2250000000000001,"15min":1.3100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.5840000000000001,"5min":1.6599999999999999,"15min":2.4239999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.337,"15min":1.399},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.55700000000000005},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":1.109}]},{"osd":5,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.339,"15min":1.405},"min":{"1min":0.71299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.901,"5min":1.901,"15min":2.1309999999999998},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.224,"5min":1.4159999999999999,"15min":1.46},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.849,"5min":2.23,"15min":2.4900000000000002},"last":1.2450000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.413,"15min":1.4630000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":4.4260000000000002,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3720000000000001,"15min":1.476},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8240000000000001,"5min":1.9430000000000001,"15min":2.246},"last":0.875}]},{"osd":7,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4490000000000001,"15min":1.5249999999999999},"min":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.72799999999999998},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.488,"15min":1.5409999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1360000000000001,"5min":4.1360000000000001,"15min":4.1360000000000001},"last":0.91600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:16:29.594 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-08T05:36:05.849165+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=8, tm_hour=5, tm_min=36, tm_sec=5, tm_wday=6, tm_yday=281, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=8, tm_hour=7, tm_min=16, tm_sec=22, tm_wday=6, tm_yday=281, tm_isdst=0) 2023-10-08T07:16:29.594 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-08T07:16:38.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:16:38.814 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:16:49.596 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T07:16:49.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.824+0000 7f8023b97640 1 Processor -- start 2023-10-08T07:16:49.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.824+0000 7f8023b97640 1 -- start start 2023-10-08T07:16:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.825+0000 7f8023b97640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f801c0c90b0 0x7f801c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.826+0000 7f8023b97640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801c15b2b0 con 0x7f801c15ab40 2023-10-08T07:16:49.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.826+0000 7f8023b97640 1 -- --> v1:172.21.15.181:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801c15b590 con 0x7f801c0cb3b0 2023-10-08T07:16:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.826+0000 7f8023b97640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801c15bf30 con 0x7f801c0c90b0 2023-10-08T07:16:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.826+0000 7f8021132640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f801c0c90b0 0x7f801c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.826+0000 7f8021132640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f801c0c90b0 0x7f801c0cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34262/0 (socket says 172.21.15.70:34262) 2023-10-08T07:16:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.826+0000 7f8021132640 1 -- 172.21.15.70:0/3731397865 learned_addr learned my addr 172.21.15.70:0/3731397865 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:49.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.826+0000 7f8021132640 1 -- 172.21.15.70:0/3731397865 >> v1:172.21.15.181:6789/0 conn(0x7f801c0cb3b0 legacy=0x7f801c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.826+0000 7f8021132640 1 -- 172.21.15.70:0/3731397865 >> v1:172.21.15.70:6789/0 conn(0x7f801c15ab40 legacy=0x7f801c0ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.826+0000 7f8021132640 1 -- 172.21.15.70:0/3731397865 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c15b870 con 0x7f801c0c90b0 2023-10-08T07:16:49.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.828+0000 7f8021132640 1 --2- 172.21.15.70:0/3731397865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c0c90b0 0x7f801c0cae70 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f800c009870 tx=0x7f800c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985385f345b2baae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.828+0000 7f8020931640 1 -- 172.21.15.70:0/3731397865 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c030860 con 0x7f801c0c90b0 2023-10-08T07:16:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.828+0000 7f8020931640 1 -- 172.21.15.70:0/3731397865 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800c0309c0 con 0x7f801c0c90b0 2023-10-08T07:16:49.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.828+0000 7f8020931640 1 -- 172.21.15.70:0/3731397865 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800c030d50 con 0x7f801c0c90b0 2023-10-08T07:16:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.829+0000 7f8023b97640 1 -- 172.21.15.70:0/3731397865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c0c90b0 msgr2=0x7f801c0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.829+0000 7f8023b97640 1 --2- 172.21.15.70:0/3731397865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c0c90b0 0x7f801c0cae70 secure :-1 s=READY pgs=3051 cs=0 l=1 rev1=1 crypto rx=0x7f800c009870 tx=0x7f800c02b9a0 comp rx=0 tx=0).stop 2023-10-08T07:16:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.829+0000 7f8023b97640 1 -- 172.21.15.70:0/3731397865 shutdown_connections 2023-10-08T07:16:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.829+0000 7f8023b97640 1 --2- 172.21.15.70:0/3731397865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c0c90b0 0x7f801c0cae70 unknown :-1 s=CLOSED pgs=3051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.829+0000 7f8023b97640 1 -- 172.21.15.70:0/3731397865 >> 172.21.15.70:0/3731397865 conn(0x7f801c06a380 msgr2=0x7f801c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.829+0000 7f8023b97640 1 -- 172.21.15.70:0/3731397865 shutdown_connections 2023-10-08T07:16:49.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.829+0000 7f8023b97640 1 -- 172.21.15.70:0/3731397865 wait complete. 2023-10-08T07:16:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.831+0000 7f8023b97640 1 Processor -- start 2023-10-08T07:16:49.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.831+0000 7f8023b97640 1 -- start start 2023-10-08T07:16:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.832+0000 7f8023b97640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801c0c90b0 0x7f801c0c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.832+0000 7f8023b97640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c0cb3b0 0x7f801c15d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.832+0000 7f8023b97640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c15ab40 0x7f801c14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.832+0000 7f8023b97640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0b2350 con 0x7f801c15ab40 2023-10-08T07:16:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.832+0000 7f8023b97640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801c0c8150 con 0x7f801c0cb3b0 2023-10-08T07:16:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.832+0000 7f8023b97640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801c1585c0 con 0x7f801c0c90b0 2023-10-08T07:16:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.832+0000 7f8022134640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c15ab40 0x7f801c14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.832+0000 7f8022134640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c15ab40 0x7f801c14ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34278/0 (socket says 172.21.15.70:34278) 2023-10-08T07:16:49.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.832+0000 7f8022134640 1 -- 172.21.15.70:0/4161699878 learned_addr learned my addr 172.21.15.70:0/4161699878 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f8021933640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801c0c90b0 0x7f801c0c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f8021132640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c0cb3b0 0x7f801c15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f8022134640 1 -- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801c0c90b0 msgr2=0x7f801c0c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f8022134640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801c0c90b0 0x7f801c0c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f8022134640 1 -- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c0cb3b0 msgr2=0x7f801c15d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f8022134640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c0cb3b0 0x7f801c15d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f8022134640 1 -- 172.21.15.70:0/4161699878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c0094d0 con 0x7f801c15ab40 2023-10-08T07:16:49.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f8022134640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c15ab40 0x7f801c14ee80 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7f801400c780 tx=0x7f801400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f80127fc640 1 -- 172.21.15.70:0/4161699878 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014003d80 con 0x7f801c15ab40 2023-10-08T07:16:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f80127fc640 1 -- 172.21.15.70:0/4161699878 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801400ce90 con 0x7f801c15ab40 2023-10-08T07:16:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f80127fc640 1 -- 172.21.15.70:0/4161699878 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140052b0 con 0x7f801c15ab40 2023-10-08T07:16:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f8023b97640 1 -- 172.21.15.70:0/4161699878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801c1588a0 con 0x7f801c15ab40 2023-10-08T07:16:49.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.833+0000 7f8023b97640 1 -- 172.21.15.70:0/4161699878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801c158e60 con 0x7f801c15ab40 2023-10-08T07:16:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.835+0000 7f8023b97640 1 -- 172.21.15.70:0/4161699878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801c0b2490 con 0x7f801c15ab40 2023-10-08T07:16:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.835+0000 7f80127fc640 1 -- 172.21.15.70:0/4161699878 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7f8014005410 con 0x7f801c15ab40 2023-10-08T07:16:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.837+0000 7f80127fc640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f800006af90 0x7f800006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.837+0000 7f80127fc640 1 -- 172.21.15.70:0/4161699878 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7f8014088e70 con 0x7f801c15ab40 2023-10-08T07:16:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.837+0000 7f8021933640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f800006af90 0x7f800006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:49.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.839+0000 7f8021933640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f800006af90 0x7f800006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8018000f60 tx=0x7f8018009040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:49.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:49.842+0000 7f80127fc640 1 -- 172.21.15.70:0/4161699878 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f801c0b2490 con 0x7f801c15ab40 2023-10-08T07:16:50.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.027+0000 7f8023b97640 1 -- 172.21.15.70:0/4161699878 --> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f801c0cb820 con 0x7f800006af90 2023-10-08T07:16:50.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.034+0000 7f80127fc640 1 -- 172.21.15.70:0/4161699878 <== mgr.8225 v2:172.21.15.181:6832/3939374099 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48162 (secure 0 0 0) 0x7f80180050d0 con 0x7f800006af90 2023-10-08T07:16:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:50.035 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-10-08T07:16:50.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.036+0000 7f8023b97640 1 -- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f800006af90 msgr2=0x7f800006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.036+0000 7f8023b97640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f800006af90 0x7f800006d450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8018000f60 tx=0x7f8018009040 comp rx=0 tx=0).stop 2023-10-08T07:16:50.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.036+0000 7f8023b97640 1 -- 172.21.15.70:0/4161699878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c15ab40 msgr2=0x7f801c14ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.036+0000 7f8023b97640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c15ab40 0x7f801c14ee80 secure :-1 s=READY pgs=3052 cs=0 l=1 rev1=1 crypto rx=0x7f801400c780 tx=0x7f801400cc40 comp rx=0 tx=0).stop 2023-10-08T07:16:50.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.036+0000 7f8023b97640 1 -- 172.21.15.70:0/4161699878 shutdown_connections 2023-10-08T07:16:50.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.036+0000 7f8023b97640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7f800006af90 0x7f800006d450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.036+0000 7f8023b97640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f801c15ab40 0x7f801c14ee80 unknown :-1 s=CLOSED pgs=3052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.036+0000 7f8023b97640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7f801c0cb3b0 0x7f801c15d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.036+0000 7f8023b97640 1 --2- 172.21.15.70:0/4161699878 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7f801c0c90b0 0x7f801c0c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.036+0000 7f8023b97640 1 -- 172.21.15.70:0/4161699878 >> 172.21.15.70:0/4161699878 conn(0x7f801c06a380 msgr2=0x7f801c0cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:50.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.037+0000 7f8023b97640 1 -- 172.21.15.70:0/4161699878 shutdown_connections 2023-10-08T07:16:50.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.037+0000 7f8023b97640 1 -- 172.21.15.70:0/4161699878 wait complete. 2023-10-08T07:16:50.056 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-08T07:16:48.910625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4884,"ondisk_log_size":4884,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":453288,"kb_used_data":28936,"kb_used_omap":12,"kb_used_meta":424243,"kb_avail":754521432,"statfs":{"total":773094113280,"available":772629946368,"internally_reserved":0,"allocated":29630464,"data_stored":16083456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12741,"internal_metadata":434425403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003495"},"pg_stats":[{"pgid":"1.0","version":"281'4884","reported_seq":6020,"reported_epoch":281,"state":"active+clean","last_fresh":"2023-10-08T07:16:29.989712+0000","last_change":"2023-10-08T07:16:29.989827+0000","last_active":"2023-10-08T07:16:29.989712+0000","last_peered":"2023-10-08T07:16:29.989712+0000","last_clean":"2023-10-08T07:16:29.989712+0000","last_became_active":"2023-10-08T05:36:07.426551+0000","last_became_peered":"2023-10-08T05:36:07.426551+0000","last_unstale":"2023-10-08T07:16:29.989712+0000","last_undegraded":"2023-10-08T07:16:29.989712+0000","last_fullsized":"2023-10-08T07:16:29.989712+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"281'4884","last_scrub_stamp":"2023-10-08T07:16:29.989573+0000","last_deep_scrub":"281'4884","last_deep_scrub_stamp":"2023-10-08T07:16:29.989573+0000","last_clean_scrub_stamp":"2023-10-08T07:16:29.989573+0000","objects_scrubbed":2,"log_size":4884,"log_dups_size":0,"ondisk_log_size":4884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:27:20.416299+0000","scrub_duration":0.012713914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2727,"num_read_kb":5751,"num_write":691,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4884,"ondisk_log_size":4884,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834576043,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75484,"kb_used_data":4052,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296356,"statfs":{"total":96636764160,"available":96559468544,"internally_reserved":0,"allocated":4149248,"data_stored":2453208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":73136587},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.1419999999999999,"15min":1.198},"min":{"1min":0.94599999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.625,"5min":1.9410000000000001,"15min":2.008},"last":1.304},{"interface":"front","average":{"1min":1.28,"5min":1.2150000000000001,"15min":1.218},"min":{"1min":0.82899999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.84399999999999997}]},{"osd":1,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.244,"15min":1.248},"min":{"1min":0.88400000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.964,"5min":1.964,"15min":1.998},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.3320000000000001},"min":{"1min":0.86299999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.038}]},{"osd":2,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.2190000000000001,"15min":1.286},"min":{"1min":0.86499999999999999,"5min":0.86399999999999999,"15min":0.77600000000000002},"max":{"1min":1.573,"5min":1.8440000000000001,"15min":2.1619999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.383,"15min":1.3540000000000001},"min":{"1min":0.77700000000000002,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.1190000000000002},"last":1.595}]},{"osd":3,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3300000000000001,"15min":1.341},"min":{"1min":0.96299999999999997,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.3980000000000001,"5min":2.3980000000000001,"15min":2.3980000000000001},"last":1.361},{"interface":"front","average":{"1min":1.615,"5min":1.4330000000000001,"15min":1.4119999999999999},"min":{"1min":1.117,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.546}]},{"osd":4,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.3839999999999999,"15min":1.339},"min":{"1min":1.0660000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.133},{"interface":"front","average":{"1min":1.6419999999999999,"5min":1.46,"15min":1.4259999999999999},"min":{"1min":1.294,"5min":0.752,"15min":0.752},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":0.93600000000000005}]},{"osd":5,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.796,"5min":1.53,"15min":1.4770000000000001},"min":{"1min":1.2490000000000001,"5min":0.80600000000000005,"15min":0.79300000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.395},{"interface":"front","average":{"1min":1.7869999999999999,"5min":1.5409999999999999,"15min":1.514},"min":{"1min":1.242,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":1.4199999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:15:59 2023","interfaces":[{"interface":"back","average":{"1min":1.8999999999999999,"5min":1.6619999999999999,"15min":1.619},"min":{"1min":1.1639999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.8240000000000001,"5min":1.629,"15min":1.5649999999999999},"min":{"1min":1.2090000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.766,"5min":2.766,"15min":2.766},"last":1.171}]}]},{"osd":6,"up_from":13,"seq":55834576042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48540,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323300,"statfs":{"total":96636764160,"available":96587059200,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.2370000000000001,"15min":1.274},"min":{"1min":0.76100000000000001,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5600000000000001,"5min":1.78,"15min":1.9490000000000001},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2949999999999999,"15min":1.3460000000000001},"min":{"1min":0.84299999999999997,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7370000000000001,"5min":1.76,"15min":2.0070000000000001},"last":1.7350000000000001}]},{"osd":1,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.121,"15min":1.2070000000000001},"min":{"1min":0.68200000000000005,"5min":0.625,"15min":0.625},"max":{"1min":1.6020000000000001,"5min":1.7569999999999999,"15min":2.0099999999999998},"last":1.173},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2310000000000001,"15min":1.236},"min":{"1min":0.73999999999999999,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6619999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.081}]},{"osd":2,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.1699999999999999,"15min":1.2210000000000001},"min":{"1min":0.71599999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.956,"5min":1.956,"15min":2.339},"last":1.544},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.246,"15min":1.294},"min":{"1min":0.83499999999999996,"5min":0.748,"15min":0.748},"max":{"1min":1.484,"5min":1.8720000000000001,"15min":2.1779999999999999},"last":1.3759999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.391,"15min":1.3979999999999999},"min":{"1min":0.95799999999999996,"5min":0.86299999999999999,"15min":0.73099999999999998},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.421,"15min":1.4770000000000001},"min":{"1min":0.998,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.8440000000000001,"5min":1.8979999999999999,"15min":2.1459999999999999},"last":1.9179999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.4450000000000001,"15min":1.415},"min":{"1min":1.048,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.8140000000000001,"5min":2.0510000000000002,"15min":2.0859999999999999},"last":1.627},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.4850000000000001,"15min":1.474},"min":{"1min":1.01,"5min":0.84299999999999997,"15min":0.57099999999999995},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.484},"last":1.8500000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.403,"15min":1.4299999999999999},"min":{"1min":0.90200000000000002,"5min":0.75,"15min":0.75},"max":{"1min":2.093,"5min":2.093,"15min":2.403},"last":1.782},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.5960000000000001,"15min":1.5700000000000001},"min":{"1min":1.1259999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.1320000000000001,"5min":2.1579999999999999,"15min":2.6139999999999999},"last":1.8200000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.5940000000000001,"15min":1.595},"min":{"1min":1.0209999999999999,"5min":0.98299999999999998,"15min":0.96299999999999997},"max":{"1min":2.1469999999999998,"5min":2.339,"15min":2.5870000000000002},"last":1.978},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.5820000000000001,"15min":1.571},"min":{"1min":1.145,"5min":0.87,"15min":0.87},"max":{"1min":2.2370000000000001,"5min":2.2959999999999998,"15min":2.665},"last":1.657}]}]},{"osd":5,"up_from":13,"seq":55834576042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":61024,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310816,"statfs":{"total":96636764160,"available":96574275584,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.1419999999999999,"15min":1.181},"min":{"1min":0.624,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5449999999999999,"5min":1.98,"15min":1.98},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.1970000000000001,"15min":1.2110000000000001},"min":{"1min":0.64600000000000002,"5min":0.621,"15min":0.58999999999999997},"max":{"1min":1.917,"5min":2.2480000000000002,"15min":2.2480000000000002},"last":1.105}]},{"osd":1,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1890000000000001,"15min":1.216},"min":{"1min":0.68799999999999994,"5min":0.64500000000000002,"15min":0.60799999999999998},"max":{"1min":1.954,"5min":2.3580000000000001,"15min":2.3580000000000001},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.3109999999999999,"15min":1.3069999999999999},"min":{"1min":0.67400000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.0840000000000001,"5min":7.5250000000000004,"15min":7.5250000000000004},"last":1.337}]},{"osd":2,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.276,"15min":1.2909999999999999},"min":{"1min":0.78100000000000003,"5min":0.68899999999999995,"15min":0.51100000000000001},"max":{"1min":1.8779999999999999,"5min":2.008,"15min":2.056},"last":1},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.246,"15min":1.2869999999999999},"min":{"1min":0.61099999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.1989999999999998,"5min":2.1989999999999998,"15min":2.1989999999999998},"last":1.3069999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3899999999999999,"15min":1.3740000000000001},"min":{"1min":0.85199999999999998,"5min":0.72499999999999998,"15min":0.58499999999999996},"max":{"1min":1.7589999999999999,"5min":6.8449999999999998,"15min":6.8449999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4399999999999999,"15min":1.4179999999999999},"min":{"1min":0.99399999999999999,"5min":0.64900000000000002,"15min":0.55700000000000005},"max":{"1min":2.008,"5min":9.3599999999999994,"15min":9.3599999999999994},"last":1.1830000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.3799999999999999,"15min":1.341},"min":{"1min":0.83699999999999997,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.3799999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.502,"15min":1.4410000000000001},"min":{"1min":0.94699999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.0049999999999999,"5min":2.742,"15min":2.742},"last":1.046}]},{"osd":6,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.605,"15min":1.49},"min":{"1min":0.878,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":2.2679999999999998,"5min":13.234999999999999,"15min":13.234999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.581,"15min":1.4930000000000001},"min":{"1min":0.93600000000000005,"5min":0.67800000000000005,"15min":0.46700000000000003},"max":{"1min":2.5779999999999998,"5min":7.2949999999999999,"15min":7.2949999999999999},"last":1.3839999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.583,"5min":1.589,"15min":1.5489999999999999},"min":{"1min":1.087,"5min":0.621,"15min":0.59999999999999998},"max":{"1min":2.4369999999999998,"5min":2.4430000000000001,"15min":2.4430000000000001},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.732,"5min":1.6419999999999999,"15min":1.607},"min":{"1min":1.1040000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":1.4159999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52704,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319136,"statfs":{"total":96636764160,"available":96582795264,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.3220000000000001,"15min":1.341},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.28499999999999998},"max":{"1min":1.718,"5min":5.4800000000000004,"15min":5.4800000000000004},"last":1.337},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.4279999999999999,"15min":1.413},"min":{"1min":0.68600000000000005,"5min":0.62,"15min":0.33000000000000002},"max":{"1min":1.7010000000000001,"5min":7.0880000000000001,"15min":7.0880000000000001},"last":1.3799999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2529999999999999,"15min":1.276},"min":{"1min":0.86799999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6899999999999999,"5min":4.2229999999999999,"15min":4.2229999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.6359999999999999,"15min":1.3680000000000001},"min":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.54000000000000004},"max":{"1min":1.8460000000000001,"5min":28.907,"15min":28.907},"last":1.4970000000000001}]},{"osd":2,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.4650000000000001,"15min":1.327},"min":{"1min":0.745,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.698,"5min":17.175999999999998,"15min":17.175999999999998},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.456,"15min":1.389},"min":{"1min":0.89700000000000002,"5min":0.59499999999999997,"15min":0.42899999999999999},"max":{"1min":1.77,"5min":7.1849999999999996,"15min":7.1849999999999996},"last":1.3089999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.5640000000000001,"15min":1.409},"min":{"1min":0.66200000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6539999999999999,"5min":15.938000000000001,"15min":15.938000000000001},"last":1.222},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.5700000000000001,"15min":1.474},"min":{"1min":0.84399999999999997,"5min":0.59199999999999997,"15min":0.57299999999999995},"max":{"1min":1.867,"5min":10.787000000000001,"15min":10.787000000000001},"last":1.272}]},{"osd":5,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.575,"15min":1.4339999999999999},"min":{"1min":0.80600000000000005,"5min":0.39100000000000001,"15min":0.372},"max":{"1min":2.101,"5min":7.4020000000000001,"15min":7.4020000000000001},"last":1.456},{"interface":"front","average":{"1min":1.431,"5min":1.8100000000000001,"15min":1.5329999999999999},"min":{"1min":0.751,"5min":0.442,"15min":0.39900000000000002},"max":{"1min":1.917,"5min":31.347999999999999,"15min":31.347999999999999},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.7150000000000001,"15min":1.5409999999999999},"min":{"1min":0.91700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.984,"5min":13.885,"15min":13.885},"last":1.004},{"interface":"front","average":{"1min":1.577,"5min":1.8169999999999999,"15min":1.645},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.59299999999999997},"max":{"1min":2.2189999999999999,"5min":13.446999999999999,"15min":13.446999999999999},"last":1.151}]},{"osd":7,"last update":"Sun Oct 8 07:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.8069999999999999,"15min":1.615},"min":{"1min":0.79500000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.1219999999999999,"5min":15.843999999999999,"15min":15.843999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.702,"15min":1.6659999999999999},"min":{"1min":0.96799999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.161,"5min":5.609,"15min":5.609},"last":1.421}]}]},{"osd":3,"up_from":13,"seq":55834576044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":56860,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94314980,"statfs":{"total":96636764160,"available":96578539520,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":54655432},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.88200000000000001,"15min":0.96499999999999997},"min":{"1min":0.51600000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.323,"5min":1.6719999999999999,"15min":2.7599999999999998},"last":1.2490000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":1.046,"15min":1.1439999999999999},"min":{"1min":0.55300000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.72,"5min":1.72,"15min":4.5659999999999998},"last":1.3029999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.95199999999999996,"15min":1.0069999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.7769999999999999,"15min":2.4390000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1699999999999999,"15min":1.2609999999999999},"min":{"1min":0.58699999999999997,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.504,"5min":2.109,"15min":2.4889999999999999},"last":1.6739999999999999}]},{"osd":2,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":1.032,"15min":1.1220000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.437},"max":{"1min":1.53,"5min":2.165,"15min":2.2250000000000001},"last":1.131},{"interface":"front","average":{"1min":1.079,"5min":1.119,"15min":1.1850000000000001},"min":{"1min":0.67800000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6200000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.6240000000000001}]},{"osd":4,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2170000000000001,"15min":1.268},"min":{"1min":0.93899999999999995,"5min":0.70899999999999996,"15min":0.67500000000000004},"max":{"1min":1.732,"5min":1.732,"15min":2.5379999999999998},"last":1.171},{"interface":"front","average":{"1min":1.355,"5min":1.3680000000000001,"15min":1.4410000000000001},"min":{"1min":0.79700000000000004,"5min":0.79500000000000004,"15min":0.78100000000000003},"max":{"1min":2.306,"5min":2.3780000000000001,"15min":2.8340000000000001},"last":1.3580000000000001}]},{"osd":5,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2989999999999999,"15min":1.355},"min":{"1min":0.79800000000000004,"5min":0.755,"15min":0.61799999999999999},"max":{"1min":1.9079999999999999,"5min":1.923,"15min":2.9740000000000002},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.393,"15min":1.4299999999999999},"min":{"1min":0.86599999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.7330000000000001},"last":1.5369999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3740000000000001,"15min":1.405},"min":{"1min":1.0569999999999999,"5min":0.82299999999999995,"15min":0.59699999999999998},"max":{"1min":1.825,"5min":1.97,"15min":2.653},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4390000000000001,"15min":1.504},"min":{"1min":1.1140000000000001,"5min":0.872,"15min":0.57499999999999996},"max":{"1min":1.7749999999999999,"5min":2.2749999999999999,"15min":3.0499999999999998},"last":1.4370000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.488,"15min":1.5529999999999999},"min":{"1min":0.91000000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.0590000000000002,"5min":2.3220000000000001,"15min":2.8650000000000002},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.4359999999999999,"15min":1.506},"min":{"1min":0.66700000000000004,"5min":0.628,"15min":0.628},"max":{"1min":1.8340000000000001,"5min":1.903,"15min":2.7749999999999999},"last":1.4730000000000001}]}]},{"osd":2,"up_from":13,"seq":55834576042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52700,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319140,"statfs":{"total":96636764160,"available":96582799360,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85299999999999998,"15min":0.94199999999999995},"min":{"1min":0.38100000000000001,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.3029999999999999,"5min":1.601,"15min":1.948},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.97699999999999998,"15min":1.0369999999999999},"min":{"1min":0.47699999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.482,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.98999999999999999}]},{"osd":1,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.88700000000000001,"15min":0.95799999999999996},"min":{"1min":0.437,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.363,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":1.506},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.0309999999999999,"15min":1.147},"min":{"1min":0.439,"5min":0.38,"15min":0.38},"max":{"1min":1.4390000000000001,"5min":1.7789999999999999,"15min":1.9390000000000001},"last":0.93000000000000005}]},{"osd":3,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.0580000000000001,"15min":1.145},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.421,"5min":2.9359999999999999,"15min":2.9359999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.004,"5min":1.123,"15min":1.2290000000000001},"min":{"1min":0.54600000000000004,"5min":0.47699999999999998,"15min":0.47599999999999998},"max":{"1min":1.54,"5min":1.6799999999999999,"15min":2.0670000000000002},"last":1.1319999999999999}]},{"osd":4,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.3280000000000001,"15min":1.409},"min":{"1min":0.78300000000000003,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.754,"5min":1.992,"15min":2.1789999999999998},"last":1.3},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.256,"15min":1.298},"min":{"1min":0.78700000000000003,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.655,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.254}]},{"osd":5,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.298,"15min":1.365},"min":{"1min":0.75600000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.972,"5min":2.5659999999999998,"15min":2.5659999999999998},"last":1.573},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3879999999999999,"15min":1.488},"min":{"1min":0.72499999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8939999999999999,"5min":2.0750000000000002,"15min":2.9390000000000001},"last":1.3939999999999999}]},{"osd":6,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.306,"15min":1.371},"min":{"1min":0.72399999999999998,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.7370000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.4059999999999999,"15min":1.4910000000000001},"min":{"1min":0.82699999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.891,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.1899999999999999}]},{"osd":7,"last update":"Sun Oct 8 07:16:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.379,"15min":1.4470000000000001},"min":{"1min":0.79200000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8380000000000001,"5min":2.3969999999999998,"15min":2.4689999999999999},"last":1.542},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.454,"15min":1.538},"min":{"1min":0.90800000000000003,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.875,"5min":2.831,"15min":2.831},"last":1.3480000000000001}]}]},{"osd":1,"up_from":13,"seq":55834576042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52700,"kb_used_data":3472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94319140,"statfs":{"total":96636764160,"available":96582799360,"internally_reserved":0,"allocated":3555328,"data_stored":1862840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.97799999999999998,"15min":0.99299999999999999},"min":{"1min":0.442,"5min":0.439,"15min":0.32100000000000001},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.0489999999999999,"15min":1.177},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.41899999999999998},"max":{"1min":1.3939999999999999,"5min":1.7010000000000001,"15min":1.79},"last":1.071}]},{"osd":2,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.86099999999999999,"15min":0.93500000000000005},"min":{"1min":0.46999999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.343,"5min":1.462,"15min":1.641},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.083,"15min":1.1299999999999999},"min":{"1min":0.49199999999999999,"5min":0.42599999999999999,"15min":0.372},"max":{"1min":1.4750000000000001,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":1.1399999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.151,"15min":1.2250000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.621,"5min":2.496,"15min":2.496},"last":1.444},{"interface":"front","average":{"1min":0.96799999999999997,"5min":1.087,"15min":1.129},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6859999999999999,"5min":2.5310000000000001,"15min":2.7429999999999999},"last":1.252}]},{"osd":4,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.2529999999999999,"15min":1.333},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.4339999999999999,"5min":2.73,"15min":2.8460000000000001},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.2410000000000001,"15min":1.2889999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.744,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.2849999999999999}]},{"osd":5,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3089999999999999,"15min":1.355},"min":{"1min":0.85599999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.9910000000000001,"5min":2.4340000000000002,"15min":2.4340000000000002},"last":1.401},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.486,"15min":1.518},"min":{"1min":0.98799999999999999,"5min":0.89600000000000002,"15min":0.84599999999999997},"max":{"1min":1.9770000000000001,"5min":2.601,"15min":2.601},"last":1.502}]},{"osd":6,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.4019999999999999,"15min":1.482},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.9710000000000001,"5min":2.883,"15min":2.9079999999999999},"last":1.321},{"interface":"front","average":{"1min":1.345,"5min":1.4039999999999999,"15min":1.474},"min":{"1min":0.752,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8939999999999999,"5min":2.6589999999999998,"15min":2.6589999999999998},"last":1.4650000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:15:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.482,"15min":1.5329999999999999},"min":{"1min":0.85099999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.1000000000000001,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.4630000000000001,"15min":1.5429999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.0920000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.8260000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576043,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53276,"kb_used_data":4052,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318564,"statfs":{"total":96636764160,"available":96582209536,"internally_reserved":0,"allocated":4149248,"data_stored":2453208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":50395593},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.86799999999999999,"15min":0.94299999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.3029999999999999,"5min":1.3859999999999999,"15min":1.581},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96399999999999997,"15min":1.0620000000000001},"min":{"1min":0.38500000000000001,"5min":0.318,"15min":0.318},"max":{"1min":4.0300000000000002,"5min":4.0300000000000002,"15min":4.0300000000000002},"last":1.446}]},{"osd":2,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":1.03,"15min":1.1100000000000001},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.42199999999999999},"max":{"1min":4.1920000000000002,"5min":4.1920000000000002,"15min":4.1920000000000002},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.999,"5min":0.97999999999999998,"15min":1.052},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.4699999999999998,"5min":4.4699999999999998,"15min":4.4699999999999998},"last":1.3979999999999999}]},{"osd":3,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":1.099,"15min":1.2030000000000001},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.425,"5min":1.633,"15min":2.0139999999999998},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.089,"15min":1.143},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.327,"5min":4.327,"15min":4.327},"last":1.006}]},{"osd":4,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.2250000000000001,"15min":1.3100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.5840000000000001,"5min":1.6599999999999999,"15min":2.4239999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.337,"15min":1.399},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.55700000000000005},"max":{"1min":4.3860000000000001,"5min":4.3860000000000001,"15min":4.3860000000000001},"last":1.675}]},{"osd":5,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.339,"15min":1.405},"min":{"1min":0.71299999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.901,"5min":1.901,"15min":2.1309999999999998},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.4159999999999999,"15min":1.46},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.849,"5min":2.23,"15min":2.4900000000000002},"last":1.3720000000000001}]},{"osd":6,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.413,"15min":1.4630000000000001},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":4.4260000000000002,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.75},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3720000000000001,"15min":1.476},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.8240000000000001,"5min":1.9430000000000001,"15min":2.246},"last":1.5640000000000001}]},{"osd":7,"last update":"Sun Oct 8 07:16:12 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4490000000000001,"15min":1.5249999999999999},"min":{"1min":0.75800000000000001,"5min":0.73299999999999998,"15min":0.72799999999999998},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.488,"15min":1.5409999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":4.1360000000000001,"5min":4.1360000000000001,"15min":4.1360000000000001},"last":1.905}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T07:16:50.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-08T07:16:50.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.266+0000 7fcdc6213640 1 Processor -- start 2023-10-08T07:16:50.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.266+0000 7fcdc6213640 1 -- start start 2023-10-08T07:16:50.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.267+0000 7fcdc6213640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcdc00cb5e0 0x7fcdc00cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.267+0000 7fcdc6213640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdc00b3c50 0x7fcdc00cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.267+0000 7fcdc6213640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc0159db0 con 0x7fcdc01599d0 2023-10-08T07:16:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.267+0000 7fcdc6213640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc0071fe0 con 0x7fcdc00cb5e0 2023-10-08T07:16:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.267+0000 7fcdc6213640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00b44c0 con 0x7fcdc00b3c50 2023-10-08T07:16:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.268+0000 7fcdc5211640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcdc01599d0 0x7fcdc00caed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49210/0 (socket says 172.21.15.70:49210) 2023-10-08T07:16:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.268+0000 7fcdc5211640 1 -- 172.21.15.70:0/3406559248 learned_addr learned my addr 172.21.15.70:0/3406559248 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-10-08T07:16:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.268+0000 7fcdc4a10640 1 --2- 172.21.15.70:0/3406559248 >> v2:172.21.15.70:3300/0 conn(0x7fcdc00cb5e0 0x7fcdc00cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.268+0000 7fcdc5a12640 1 --2- 172.21.15.70:0/3406559248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdc00b3c50 0x7fcdc00cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.268+0000 7fcdb7fff640 1 -- 172.21.15.70:0/3406559248 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554261345 0 0) 0x7fcdbc001110 con 0x7fcdc01599d0 2023-10-08T07:16:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.268+0000 7fcdb7fff640 1 -- 172.21.15.70:0/3406559248 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdac005730 con 0x7fcdc01599d0 2023-10-08T07:16:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.268+0000 7fcdc4a10640 1 -- 172.21.15.70:0/3406559248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdc00b3c50 msgr2=0x7fcdc00cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.268+0000 7fcdc4a10640 1 --2- 172.21.15.70:0/3406559248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdc00b3c50 0x7fcdc00cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.268+0000 7fcdc4a10640 1 -- 172.21.15.70:0/3406559248 >> v1:172.21.15.70:6789/0 conn(0x7fcdc01599d0 legacy=0x7fcdc00caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.268+0000 7fcdc4a10640 1 -- 172.21.15.70:0/3406559248 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc00713d0 con 0x7fcdc00cb5e0 2023-10-08T07:16:50.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.270+0000 7fcdc4a10640 1 --2- 172.21.15.70:0/3406559248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc00cb5e0 0x7fcdc00cdcd0 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00097a0 tx=0x7fcdb002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246594e0808c9307 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:50.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.270+0000 7fcdb7fff640 1 -- 172.21.15.70:0/3406559248 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0031830 con 0x7fcdc00cb5e0 2023-10-08T07:16:50.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.270+0000 7fcdb7fff640 1 -- 172.21.15.70:0/3406559248 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb0031990 con 0x7fcdc00cb5e0 2023-10-08T07:16:50.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.270+0000 7fcdb7fff640 1 -- 172.21.15.70:0/3406559248 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0031d20 con 0x7fcdc00cb5e0 2023-10-08T07:16:50.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.271+0000 7fcdc6213640 1 -- 172.21.15.70:0/3406559248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc00cb5e0 msgr2=0x7fcdc00cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.271+0000 7fcdc6213640 1 --2- 172.21.15.70:0/3406559248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc00cb5e0 0x7fcdc00cdcd0 secure :-1 s=READY pgs=3054 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00097a0 tx=0x7fcdb002b8d0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.271+0000 7fcdc6213640 1 -- 172.21.15.70:0/3406559248 shutdown_connections 2023-10-08T07:16:50.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.271+0000 7fcdc6213640 1 --2- 172.21.15.70:0/3406559248 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdc00b3c50 0x7fcdc00cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.271+0000 7fcdc6213640 1 --2- 172.21.15.70:0/3406559248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc00cb5e0 0x7fcdc00cdcd0 unknown :-1 s=CLOSED pgs=3054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.271+0000 7fcdc6213640 1 -- 172.21.15.70:0/3406559248 >> 172.21.15.70:0/3406559248 conn(0x7fcdc006c050 msgr2=0x7fcdc006c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.271+0000 7fcdc6213640 1 -- 172.21.15.70:0/3406559248 shutdown_connections 2023-10-08T07:16:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.271+0000 7fcdc6213640 1 -- 172.21.15.70:0/3406559248 wait complete. 2023-10-08T07:16:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.273+0000 7fcdc6213640 1 Processor -- start 2023-10-08T07:16:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.273+0000 7fcdc6213640 1 -- start start 2023-10-08T07:16:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc6213640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdc00b3c50 0x7fcdc00adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc6213640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdc00cb5e0 0x7fcdc00a90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc6213640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc01599d0 0x7fcdc0150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc6213640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00ae420 con 0x7fcdc01599d0 2023-10-08T07:16:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc6213640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00a9630 con 0x7fcdc00cb5e0 2023-10-08T07:16:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc6213640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc0150f60 con 0x7fcdc00b3c50 2023-10-08T07:16:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc5a12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc01599d0 0x7fcdc0150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc5a12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc01599d0 0x7fcdc0150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34288/0 (socket says 172.21.15.70:34288) 2023-10-08T07:16:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc5a12640 1 -- 172.21.15.70:0/1776892973 learned_addr learned my addr 172.21.15.70:0/1776892973 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc5a12640 1 -- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdc00b3c50 msgr2=0x7fcdc00adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc5a12640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdc00b3c50 0x7fcdc00adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc5a12640 1 -- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdc00cb5e0 msgr2=0x7fcdc00a90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc5a12640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdc00cb5e0 0x7fcdc00a90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.274+0000 7fcdc5a12640 1 -- 172.21.15.70:0/1776892973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0009400 con 0x7fcdc01599d0 2023-10-08T07:16:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.275+0000 7fcdc5a12640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc01599d0 0x7fcdc0150a20 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8007e40 tx=0x7fcdb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:50.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.275+0000 7fcdb5ffb640 1 -- 172.21.15.70:0/1776892973 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8014620 con 0x7fcdc01599d0 2023-10-08T07:16:50.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.275+0000 7fcdb5ffb640 1 -- 172.21.15.70:0/1776892973 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb800ee90 con 0x7fcdc01599d0 2023-10-08T07:16:50.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.275+0000 7fcdc6213640 1 -- 172.21.15.70:0/1776892973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc00a9860 con 0x7fcdc01599d0 2023-10-08T07:16:50.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.275+0000 7fcdb5ffb640 1 -- 172.21.15.70:0/1776892973 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80149e0 con 0x7fcdc01599d0 2023-10-08T07:16:50.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.275+0000 7fcdc6213640 1 -- 172.21.15.70:0/1776892973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc00a9e20 con 0x7fcdc01599d0 2023-10-08T07:16:50.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.276+0000 7fcdc6213640 1 -- 172.21.15.70:0/1776892973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc0074150 con 0x7fcdc01599d0 2023-10-08T07:16:50.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.277+0000 7fcdb5ffb640 1 -- 172.21.15.70:0/1776892973 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fcdb8014b40 con 0x7fcdc01599d0 2023-10-08T07:16:50.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.278+0000 7fcdb5ffb640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcda406afc0 0x7fcda406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.278+0000 7fcdb5ffb640 1 -- 172.21.15.70:0/1776892973 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fcdb808a450 con 0x7fcdc01599d0 2023-10-08T07:16:50.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.279+0000 7fcdc5211640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcda406afc0 0x7fcda406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:50.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.280+0000 7fcdc5211640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcda406afc0 0x7fcda406d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc005880 tx=0x7fcdbc001150 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:50.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.282+0000 7fcdb5ffb640 1 -- 172.21.15.70:0/1776892973 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdb80550b0 con 0x7fcdc01599d0 2023-10-08T07:16:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.473+0000 7fcdc6213640 1 -- 172.21.15.70:0/1776892973 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fcdc00cc4e0 con 0x7fcdc01599d0 2023-10-08T07:16:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.480+0000 7fcdb5ffb640 1 -- 172.21.15.70:0/1776892973 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v94)=0 v94) v1 ==== 123+0+0 (secure 0 0 0) 0x7fcdb80586d0 con 0x7fcdc01599d0 2023-10-08T07:16:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.482+0000 7fcdb5ffb640 1 -- 172.21.15.70:0/1776892973 <== mon.0 v2:172.21.15.70:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcdb80550b0 con 0x7fcdc01599d0 2023-10-08T07:16:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.484+0000 7fcdc6213640 1 -- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcda406afc0 msgr2=0x7fcda406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.484+0000 7fcdc6213640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcda406afc0 0x7fcda406d480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc005880 tx=0x7fcdbc001150 comp rx=0 tx=0).stop 2023-10-08T07:16:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.484+0000 7fcdc6213640 1 -- 172.21.15.70:0/1776892973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc01599d0 msgr2=0x7fcdc0150a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.484+0000 7fcdc6213640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc01599d0 0x7fcdc0150a20 secure :-1 s=READY pgs=3055 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8007e40 tx=0x7fcdb800e4b0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.484+0000 7fcdc6213640 1 -- 172.21.15.70:0/1776892973 shutdown_connections 2023-10-08T07:16:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.484+0000 7fcdc6213640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fcda406afc0 0x7fcda406d480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.484+0000 7fcdc6213640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc01599d0 0x7fcdc0150a20 unknown :-1 s=CLOSED pgs=3055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.484+0000 7fcdc6213640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fcdc00cb5e0 0x7fcdc00a90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.484+0000 7fcdc6213640 1 --2- 172.21.15.70:0/1776892973 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fcdc00b3c50 0x7fcdc00adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.484+0000 7fcdc6213640 1 -- 172.21.15.70:0/1776892973 >> 172.21.15.70:0/1776892973 conn(0x7fcdc006c050 msgr2=0x7fcdc00c9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.485+0000 7fcdc6213640 1 -- 172.21.15.70:0/1776892973 shutdown_connections 2023-10-08T07:16:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.485+0000 7fcdc6213640 1 -- 172.21.15.70:0/1776892973 wait complete. 2023-10-08T07:16:50.503 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T07:16:50.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T07:16:50.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.708+0000 7fee70997640 1 Processor -- start 2023-10-08T07:16:50.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.709+0000 7fee70997640 1 -- start start 2023-10-08T07:16:50.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.710+0000 7fee70997640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fee6c15ab40 0x7fee6c0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.710+0000 7fee70997640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee6c0cb580 0x7fee6c0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.710+0000 7fee70997640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6c15b2b0 con 0x7fee6c0c90b0 2023-10-08T07:16:50.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.710+0000 7fee70997640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6c15bf30 con 0x7fee6c15ab40 2023-10-08T07:16:50.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.710+0000 7fee70997640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c15f7a0 con 0x7fee6c0cb580 2023-10-08T07:16:50.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.711+0000 7fee6a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fee6c15ab40 0x7fee6c0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:50.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.711+0000 7fee6a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fee6c15ab40 0x7fee6c0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34292/0 (socket says 172.21.15.70:34292) 2023-10-08T07:16:50.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.711+0000 7fee63fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fee6c0c90b0 0x7fee6c0cae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49222/0 (socket says 172.21.15.70:49222) 2023-10-08T07:16:50.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.711+0000 7fee6a59c640 1 -- 172.21.15.70:0/2205199062 learned_addr learned my addr 172.21.15.70:0/2205199062 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:50.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.711+0000 7fee6ad9d640 1 --2- 172.21.15.70:0/2205199062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee6c0cb580 0x7fee6c0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:50.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.711+0000 7fee6a59c640 1 -- 172.21.15.70:0/2205199062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee6c0cb580 msgr2=0x7fee6c0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T07:16:50.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.711+0000 7fee6a59c640 1 --2- 172.21.15.70:0/2205199062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee6c0cb580 0x7fee6c0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.711+0000 7fee6a59c640 1 -- 172.21.15.70:0/2205199062 >> v1:172.21.15.70:6789/0 conn(0x7fee6c0c90b0 legacy=0x7fee6c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.711+0000 7fee6a59c640 1 -- 172.21.15.70:0/2205199062 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c15b590 con 0x7fee6c15ab40 2023-10-08T07:16:50.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.711+0000 7fee69d9b640 1 -- 172.21.15.70:0/2205199062 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170130907 0 0) 0x7fee6c15b2b0 con 0x7fee6c0c90b0 2023-10-08T07:16:50.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.712+0000 7fee6a59c640 1 --2- 172.21.15.70:0/2205199062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c15ab40 0x7fee6c0ad720 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0098a0 tx=0x7fee5c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee8d5ed14c7c71b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:50.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee69d9b640 1 -- 172.21.15.70:0/2205199062 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c030840 con 0x7fee6c15ab40 2023-10-08T07:16:50.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee69d9b640 1 -- 172.21.15.70:0/2205199062 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fee5c0309a0 con 0x7fee6c15ab40 2023-10-08T07:16:50.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee69d9b640 1 -- 172.21.15.70:0/2205199062 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c030d50 con 0x7fee6c15ab40 2023-10-08T07:16:50.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee70997640 1 -- 172.21.15.70:0/2205199062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c15ab40 msgr2=0x7fee6c0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee70997640 1 --2- 172.21.15.70:0/2205199062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c15ab40 0x7fee6c0ad720 secure :-1 s=READY pgs=3057 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0098a0 tx=0x7fee5c02b980 comp rx=0 tx=0).stop 2023-10-08T07:16:50.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee70997640 1 -- 172.21.15.70:0/2205199062 shutdown_connections 2023-10-08T07:16:50.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee70997640 1 --2- 172.21.15.70:0/2205199062 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee6c0cb580 0x7fee6c0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee70997640 1 --2- 172.21.15.70:0/2205199062 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c15ab40 0x7fee6c0ad720 unknown :-1 s=CLOSED pgs=3057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee70997640 1 -- 172.21.15.70:0/2205199062 >> 172.21.15.70:0/2205199062 conn(0x7fee6c06a380 msgr2=0x7fee6c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:50.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee70997640 1 -- 172.21.15.70:0/2205199062 shutdown_connections 2023-10-08T07:16:50.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.713+0000 7fee70997640 1 -- 172.21.15.70:0/2205199062 wait complete. 2023-10-08T07:16:50.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.715+0000 7fee70997640 1 Processor -- start 2023-10-08T07:16:50.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.715+0000 7fee70997640 1 -- start start 2023-10-08T07:16:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.716+0000 7fee70997640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c0c90b0 0x7fee6c0b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.716+0000 7fee70997640 1 --2- >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c0cb580 0x7fee6c154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.716+0000 7fee70997640 1 --2- >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee6c15ab40 0x7fee6c155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.716+0000 7fee70997640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c0c6190 con 0x7fee6c0c90b0 2023-10-08T07:16:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.716+0000 7fee70997640 1 -- --> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c145fb0 con 0x7fee6c0cb580 2023-10-08T07:16:50.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.716+0000 7fee70997640 1 -- --> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c0c63e0 con 0x7fee6c15ab40 2023-10-08T07:16:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.716+0000 7fee6a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c0c90b0 0x7fee6c0b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee6a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c0c90b0 0x7fee6c0b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:34304/0 (socket says 172.21.15.70:34304) 2023-10-08T07:16:50.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee6a59c640 1 -- 172.21.15.70:0/934233792 learned_addr learned my addr 172.21.15.70:0/934233792 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-10-08T07:16:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee6ad9d640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee6c15ab40 0x7fee6c155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee63fff640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c0cb580 0x7fee6c154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee6a59c640 1 -- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee6c15ab40 msgr2=0x7fee6c155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee6a59c640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee6c15ab40 0x7fee6c155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee6a59c640 1 -- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c0cb580 msgr2=0x7fee6c154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee6a59c640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c0cb580 0x7fee6c154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee6a59c640 1 -- 172.21.15.70:0/934233792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c0094d0 con 0x7fee6c0c90b0 2023-10-08T07:16:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee63fff640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c0cb580 0x7fee6c154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T07:16:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.717+0000 7fee6a59c640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c0c90b0 0x7fee6c0b16f0 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0090b0 tx=0x7fee5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.718+0000 7fee62ffd640 1 -- 172.21.15.70:0/934233792 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c03e040 con 0x7fee6c0c90b0 2023-10-08T07:16:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.718+0000 7fee62ffd640 1 -- 172.21.15.70:0/934233792 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fee5c039d70 con 0x7fee6c0c90b0 2023-10-08T07:16:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.718+0000 7fee62ffd640 1 -- 172.21.15.70:0/934233792 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c004290 con 0x7fee6c0c90b0 2023-10-08T07:16:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.718+0000 7fee70997640 1 -- 172.21.15.70:0/934233792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6c0c66c0 con 0x7fee6c0c90b0 2023-10-08T07:16:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.718+0000 7fee70997640 1 -- 172.21.15.70:0/934233792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6c0c6c80 con 0x7fee6c0c90b0 2023-10-08T07:16:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.719+0000 7fee70997640 1 -- 172.21.15.70:0/934233792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6c15bb30 con 0x7fee6c0c90b0 2023-10-08T07:16:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.720+0000 7fee62ffd640 1 -- 172.21.15.70:0/934233792 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 155) v1 ==== 89331+0+0 (secure 0 0 0) 0x7fee5c003610 con 0x7fee6c0c90b0 2023-10-08T07:16:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.721+0000 7fee62ffd640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee4c06af90 0x7fee4c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T07:16:50.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.721+0000 7fee62ffd640 1 -- 172.21.15.70:0/934233792 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(281..281 src has 1..281) v4 ==== 6239+0+0 (secure 0 0 0) 0x7fee5c0ac080 con 0x7fee6c0c90b0 2023-10-08T07:16:50.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.722+0000 7fee63fff640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee4c06af90 0x7fee4c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T07:16:50.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.723+0000 7fee63fff640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee4c06af90 0x7fee4c06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0c5a50 tx=0x7fee54009040 comp rx=0 tx=0).ready entity=mgr.8225 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T07:16:50.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.725+0000 7fee62ffd640 1 -- 172.21.15.70:0/934233792 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee6c15bb30 con 0x7fee6c0c90b0 2023-10-08T07:16:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.952+0000 7fee70997640 1 -- 172.21.15.70:0/934233792 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee6c15bb30 con 0x7fee6c0c90b0 2023-10-08T07:16:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.953+0000 7fee62ffd640 1 -- 172.21.15.70:0/934233792 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) v1 ==== 95+0+3705 (secure 0 0 0) 0x7fee5c049e90 con 0x7fee6c0c90b0 2023-10-08T07:16:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-10-08T07:16:50.954 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":142,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8105,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/949448327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6833","nonce":949448327},{"type":"v1","addr":"172.21.15.70:6835","nonce":949448327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":8111,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6837/2423877406","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6834","nonce":2423877406},{"type":"v1","addr":"172.21.15.181:6837","nonce":2423877406}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":9283,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.181:6836/247109890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6835","nonce":247109890},{"type":"v1","addr":"172.21.15.181:6836","nonce":247109890}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":135},{"gid":13083,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6834/1424289939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":1424289939},{"type":"v1","addr":"172.21.15.70:6834","nonce":1424289939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":142}],"filesystems":[]} 2023-10-08T07:16:50.954 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 142 2023-10-08T07:16:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.955+0000 7fee70997640 1 -- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee4c06af90 msgr2=0x7fee4c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.955+0000 7fee70997640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee4c06af90 0x7fee4c06d450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0c5a50 tx=0x7fee54009040 comp rx=0 tx=0).stop 2023-10-08T07:16:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.955+0000 7fee70997640 1 -- 172.21.15.70:0/934233792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c0c90b0 msgr2=0x7fee6c0b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T07:16:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.955+0000 7fee70997640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c0c90b0 0x7fee6c0b16f0 secure :-1 s=READY pgs=3058 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0090b0 tx=0x7fee5c004730 comp rx=0 tx=0).stop 2023-10-08T07:16:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.955+0000 7fee70997640 1 -- 172.21.15.70:0/934233792 shutdown_connections 2023-10-08T07:16:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.955+0000 7fee70997640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:6832/3939374099,v1:172.21.15.181:6833/3939374099] conn(0x7fee4c06af90 0x7fee4c06d450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.955+0000 7fee70997640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:3301/0,v1:172.21.15.181:6790/0] conn(0x7fee6c15ab40 0x7fee6c155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.955+0000 7fee70997640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] conn(0x7fee6c0cb580 0x7fee6c154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.955+0000 7fee70997640 1 --2- 172.21.15.70:0/934233792 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee6c0c90b0 0x7fee6c0b16f0 unknown :-1 s=CLOSED pgs=3058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T07:16:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.955+0000 7fee70997640 1 -- 172.21.15.70:0/934233792 >> 172.21.15.70:0/934233792 conn(0x7fee6c06a380 msgr2=0x7fee6c0cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T07:16:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.956+0000 7fee70997640 1 -- 172.21.15.70:0/934233792 shutdown_connections 2023-10-08T07:16:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-10-08T07:16:50.956+0000 7fee70997640 1 -- 172.21.15.70:0/934233792 wait complete. 2023-10-08T07:16:50.975 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-08T07:16:50.975 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-08T07:16:50.975 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T07:16:50.975 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:50.990 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T07:16:50.990 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T07:16:50.991 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.048 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T07:16:51.049 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T07:16:51.049 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.074 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T07:16:51.074 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T07:16:51.074 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.143 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T07:16:51.143 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-08T07:16:51.144 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-08T07:16:51.144 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.194 INFO:tasks.ceph.osd.0:Stopped 2023-10-08T07:16:51.194 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-08T07:16:51.195 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.252 INFO:tasks.ceph.osd.1:Stopped 2023-10-08T07:16:51.253 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-08T07:16:51.253 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.321 INFO:tasks.ceph.osd.2:Stopped 2023-10-08T07:16:51.321 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-08T07:16:51.321 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.396 INFO:tasks.ceph.osd.3:Stopped 2023-10-08T07:16:51.397 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-08T07:16:51.397 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.449 INFO:tasks.ceph.osd.4:Stopped 2023-10-08T07:16:51.450 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-08T07:16:51.450 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.516 INFO:tasks.ceph.osd.5:Stopped 2023-10-08T07:16:51.516 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-08T07:16:51.517 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.592 INFO:tasks.ceph.osd.6:Stopped 2023-10-08T07:16:51.592 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-08T07:16:51.592 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.671 INFO:tasks.ceph.osd.7:Stopped 2023-10-08T07:16:51.671 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-08T07:16:51.671 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-08T07:16:51.671 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.748 INFO:tasks.ceph.mgr.y:Stopped 2023-10-08T07:16:51.749 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-08T07:16:51.749 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.824 INFO:tasks.ceph.mgr.x:Stopped 2023-10-08T07:16:51.825 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-08T07:16:51.825 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-08T07:16:51.825 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.905 INFO:tasks.ceph.mon.a:Stopped 2023-10-08T07:16:51.906 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-08T07:16:51.906 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:51.935 INFO:tasks.ceph.mon.b:Stopped 2023-10-08T07:16:51.936 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-08T07:16:51.936 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T07:16:52.013 INFO:tasks.ceph.mon.c:Stopped 2023-10-08T07:16:52.014 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-08T07:16:52.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-08T07:16:52.121 INFO:teuthology.orchestra.run.smithi070.stdout:2023-10-08T05:51:33.341895+0000 mon.a (mon.0) 573 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-10-08T07:16:52.122 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-08T07:16:52.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-08T07:16:52.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-08T07:16:52.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-08T07:16:52.370 INFO:teuthology.orchestra.run.smithi070.stdout:2023-10-08T05:51:33.341895+0000 mon.a (mon.0) 573 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-10-08T07:16:52.371 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T07:16:52.371 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-08T07:16:52.720 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T07:16:52.720 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-08T07:16:52.991 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T07:16:52.991 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-08T07:16:53.305 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T07:16:53.305 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-08T07:16:53.563 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T07:16:53.564 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-08T07:16:53.962 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T07:16:53.962 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-08T07:16:54.219 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T07:16:54.219 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-08T07:16:54.497 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T07:16:54.497 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-08T07:16:54.780 INFO:tasks.ceph:Archiving mon data... 2023-10-08T07:16:54.781 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325/data/mon.a.tgz 2023-10-08T07:16:54.781 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2023-10-08T07:16:54.842 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.Qlf1eDFT7G 2023-10-08T07:16:54.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Qlf1eDFT7G 2023-10-08T07:16:55.819 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.Qlf1eDFT7G 2023-10-08T07:16:55.966 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.Qlf1eDFT7G is 4MB 2023-10-08T07:16:56.135 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.Qlf1eDFT7G 2023-10-08T07:16:56.198 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325/data/mon.b.tgz 2023-10-08T07:16:56.198 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-10-08T07:16:56.257 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.Kz6JBRByua 2023-10-08T07:16:56.258 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Kz6JBRByua 2023-10-08T07:16:57.014 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.Kz6JBRByua 2023-10-08T07:16:57.161 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.Kz6JBRByua is 4MB 2023-10-08T07:16:57.341 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.Kz6JBRByua 2023-10-08T07:16:57.402 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325/data/mon.c.tgz 2023-10-08T07:16:57.402 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2023-10-08T07:16:57.455 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.3a30d0GJ2J 2023-10-08T07:16:57.455 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.3a30d0GJ2J 2023-10-08T07:16:58.177 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.3a30d0GJ2J 2023-10-08T07:16:58.324 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.3a30d0GJ2J is 4MB 2023-10-08T07:16:58.494 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.3a30d0GJ2J 2023-10-08T07:16:58.555 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-08T07:16:58.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-08T07:16:58.564 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-08T07:16:58.670 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-08T07:16:58.670 DEBUG:teuthology.orchestra.run.smithi070:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-08T07:16:58.679 DEBUG:teuthology.orchestra.run.smithi181:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-08T07:16:58.781 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-08T07:16:58.785 INFO:teuthology.orchestra.run.smithi181.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-08T07:16:58.798 INFO:tasks.ceph:Archiving crash dumps... 2023-10-08T07:16:58.800 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325/remote/smithi070/crash 2023-10-08T07:16:58.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-08T07:16:58.884 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325/remote/smithi181/crash 2023-10-08T07:16:58.885 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-08T07:16:58.986 INFO:tasks.ceph:Shutting down logrotate 2023-10-08T07:16:58.986 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:16:58.995 DEBUG:teuthology.orchestra.run.smithi181:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T07:16:59.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-08T07:16:59.234 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-08T07:16:59.333 INFO:tasks.ceph:Compressing logs... 2023-10-08T07:16:59.334 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T07:16:59.343 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T07:16:59.433 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149494.log 2023-10-08T07:16:59.434 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121045.log 2023-10-08T07:16:59.435 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149494.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149494.log.gz 2023-10-08T07:16:59.435 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124427.log 2023-10-08T07:16:59.437 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136331.log 2023-10-08T07:16:59.437 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121045.log.gz 2023-10-08T07:16:59.438 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124427.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.124427.log.gz 2023-10-08T07:16:59.438 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129082.log 2023-10-08T07:16:59.438 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-08T07:16:59.439 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152439.log 2023-10-08T07:16:59.439 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.136331.log.gz 2023-10-08T07:16:59.440 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87315.log 2023-10-08T07:16:59.440 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153318.log 2023-10-08T07:16:59.441 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.129082.log.gz 2023-10-08T07:16:59.442 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152439.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152439.log.gz 2023-10-08T07:16:59.442 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128037.log 2023-10-08T07:16:59.442 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.93324.log 2023-10-08T07:16:59.442 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.87315.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87315.log.gz 2023-10-08T07:16:59.442 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87355.log 2023-10-08T07:16:59.444 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.3.93324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.93324.log.gz 2023-10-08T07:16:59.444 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87275.log 2023-10-08T07:16:59.445 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153234.log 2023-10-08T07:16:59.445 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.153318.log.gz 2023-10-08T07:16:59.445 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97457.log 2023-10-08T07:16:59.445 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128037.log.gz 2023-10-08T07:16:59.445 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153234.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153234.log.gz 2023-10-08T07:16:59.445 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157739.log 2023-10-08T07:16:59.447 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120523.log 2023-10-08T07:16:59.447 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97457.log.gz 2023-10-08T07:16:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.87355.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87355.log.gz 2023-10-08T07:16:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.92931.log 2023-10-08T07:16:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.87275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87235.log 2023-10-08T07:16:59.448 INFO:teuthology.orchestra.run.smithi181.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87275.log.gz 2023-10-08T07:16:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.1.92931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.92931.log.gz 2023-10-08T07:16:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-08T07:16:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.87235.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87235.log.gz 2023-10-08T07:16:59.448 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-08T07:16:59.449 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87515.log 2023-10-08T07:16:59.451 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158490.log 2023-10-08T07:16:59.451 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157739.log.gz 2023-10-08T07:16:59.451 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120523.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120523.log.gz 2023-10-08T07:16:59.451 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113332.log 2023-10-08T07:16:59.451 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121339.log 2023-10-08T07:16:59.451 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.158490.log.gz 2023-10-08T07:16:59.451 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115784.log 2023-10-08T07:16:59.451 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113332.log.gz 2023-10-08T07:16:59.454 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121339.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121339.log.gz 2023-10-08T07:16:59.454 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117540.log 2023-10-08T07:16:59.454 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145734.log 2023-10-08T07:16:59.454 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115784.log.gz 2023-10-08T07:16:59.455 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87435.log 2023-10-08T07:16:59.455 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.87515.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87515.log.gz 2023-10-08T07:16:59.455 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-08T07:16:59.455 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.87435.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-08T07:16:59.455 INFO:teuthology.orchestra.run.smithi181.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87435.log.gz 2023-10-08T07:16:59.456 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-08T07:16:59.457 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117540.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.117540.log.gz 2023-10-08T07:16:59.457 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.133925.log 2023-10-08T07:16:59.457 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.145734.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.145734.log.gz 2023-10-08T07:16:59.457 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113034.log 2023-10-08T07:16:59.457 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142252.log 2023-10-08T07:16:59.457 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133925.log.gz 2023-10-08T07:16:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150366.log 2023-10-08T07:16:59.460 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113034.log.gz 2023-10-08T07:16:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142252.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142252.log.gz 2023-10-08T07:16:59.460 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118930.log 2023-10-08T07:16:59.461 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-08T07:16:59.461 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87395.log 2023-10-08T07:16:59.461 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-08T07:16:59.461 INFO:teuthology.orchestra.run.smithi181.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-08T07:16:59.463 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150366.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-08T07:16:59.463 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150366.log.gz 2023-10-08T07:16:59.463 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.157452.log 2023-10-08T07:16:59.463 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.118930.log.gz 2023-10-08T07:16:59.463 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99620.log 2023-10-08T07:16:59.465 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.157452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.157452.log.gz 2023-10-08T07:16:59.465 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138418.log 2023-10-08T07:16:59.465 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146465.log 2023-10-08T07:16:59.465 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99620.log.gz 2023-10-08T07:16:59.466 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.87395.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-08T07:16:59.466 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87395.log.gz 2023-10-08T07:16:59.467 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87475.log 2023-10-08T07:16:59.468 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145489.log 2023-10-08T07:16:59.468 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138418.log.gz 2023-10-08T07:16:59.468 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143206.log 2023-10-08T07:16:59.469 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.146465.log.gz 2023-10-08T07:16:59.469 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.145489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152173.log 2023-10-08T07:16:59.469 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145489.log.gz 2023-10-08T07:16:59.469 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.143206.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143206.log.gz 2023-10-08T07:16:59.470 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121297.log 2023-10-08T07:16:59.471 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127157.log 2023-10-08T07:16:59.471 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152173.log.gz 2023-10-08T07:16:59.472 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121297.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121297.log.gz 2023-10-08T07:16:59.472 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134214.log 2023-10-08T07:16:59.473 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140631.log 2023-10-08T07:16:59.473 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127157.log.gz 2023-10-08T07:16:59.475 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134214.log: gzip -5 --verbose 88.6% -- -- replaced with /var/log/ceph/ceph-client.admin.134214.log.gz 2023-10-08T07:16:59.475 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.137247.log 2023-10-08T07:16:59.476 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140631.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140631.log.gz 2023-10-08T07:16:59.476 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148560.log 2023-10-08T07:16:59.477 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147678.log 2023-10-08T07:16:59.477 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137247.log.gz 2023-10-08T07:16:59.478 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107771.log 2023-10-08T07:16:59.478 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148560.log.gz 2023-10-08T07:16:59.479 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-08T07:16:59.480 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116260.log 2023-10-08T07:16:59.480 INFO:teuthology.orchestra.run.smithi070.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.147678.log.gz 2023-10-08T07:16:59.481 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107771.log: gzip 88.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.107771.log.gz --verbose 2023-10-08T07:16:59.481 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.158073.log 2023-10-08T07:16:59.481 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-client.admin.87475.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87475.log.gz 2023-10-08T07:16:59.482 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116260.log: gzip -5 --verbose -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116260.log.gz /var/log/ceph/ceph-client.admin.134172.log 2023-10-08T07:16:59.482 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:16:59.483 INFO:teuthology.orchestra.run.smithi181.stderr: 94.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-08T07:16:59.484 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158073.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158073.log.gz 2023-10-08T07:16:59.484 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116819.log 2023-10-08T07:16:59.485 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-08T07:16:59.485 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134172.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.134172.log.gz 2023-10-08T07:16:59.485 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134382.log 2023-10-08T07:16:59.486 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116819.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116819.log.gz 2023-10-08T07:16:59.486 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94999.log 2023-10-08T07:16:59.490 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133536.log 2023-10-08T07:16:59.490 INFO:teuthology.orchestra.run.smithi070.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.134382.log.gz 2023-10-08T07:16:59.490 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94999.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94999.log.gz 2023-10-08T07:16:59.490 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122324.log 2023-10-08T07:16:59.490 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97541.log 2023-10-08T07:16:59.490 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.133536.log.gz 2023-10-08T07:16:59.491 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115107.log 2023-10-08T07:16:59.491 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.122324.log.gz 2023-10-08T07:16:59.492 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149172.log 2023-10-08T07:16:59.492 INFO:teuthology.orchestra.run.smithi070.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.97541.log.gz 2023-10-08T07:16:59.493 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140547.log 2023-10-08T07:16:59.494 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115107.log.gz 2023-10-08T07:16:59.495 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99032.log 2023-10-08T07:16:59.495 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149172.log.gz 2023-10-08T07:16:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140547.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140547.log.gz 2023-10-08T07:16:59.496 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.96553.log 2023-10-08T07:16:59.497 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153784.log 2023-10-08T07:16:59.497 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99032.log.gz 2023-10-08T07:16:59.498 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.2.96553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.96553.log.gz 2023-10-08T07:16:59.498 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97793.log 2023-10-08T07:16:59.500 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153784.log: gzip -5 --verbose -- 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153784.log.gz 2023-10-08T07:16:59.500 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.153406.log 2023-10-08T07:16:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97793.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97793.log.gz 2023-10-08T07:16:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107287.log 2023-10-08T07:16:59.502 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100864.log 2023-10-08T07:16:59.502 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153406.log.gz 2023-10-08T07:16:59.504 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107287.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107287.log.gz 2023-10-08T07:16:59.504 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118758.log 2023-10-08T07:16:59.505 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125146.log 2023-10-08T07:16:59.505 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100864.log.gz 2023-10-08T07:16:59.506 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109026.log 2023-10-08T07:16:59.506 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118758.log.gz 2023-10-08T07:16:59.507 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125146.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125146.log.gz 2023-10-08T07:16:59.507 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135242.log 2023-10-08T07:16:59.508 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154042.log 2023-10-08T07:16:59.509 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.109026.log.gz 2023-10-08T07:16:59.510 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135242.log: gzip -5 --verbose 88.6% -- -- replaced with /var/log/ceph/ceph-client.admin.135242.log.gz /var/log/ceph/ceph-client.admin.98402.log 2023-10-08T07:16:59.510 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:16:59.511 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154042.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.154042.log.gz 2023-10-08T07:16:59.511 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107583.log 2023-10-08T07:16:59.512 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98402.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.98402.log.gz -5 2023-10-08T07:16:59.513 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.156777.log 2023-10-08T07:16:59.514 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108340.log 2023-10-08T07:16:59.514 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107583.log.gz 2023-10-08T07:16:59.515 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102236.log 2023-10-08T07:16:59.515 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.156777.log.gz 2023-10-08T07:16:59.516 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145608.log 2023-10-08T07:16:59.516 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108340.log.gz 2023-10-08T07:16:59.517 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118716.log 2023-10-08T07:16:59.518 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102236.log.gz 2023-10-08T07:16:59.519 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.145608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102193.log 2023-10-08T07:16:59.519 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145608.log.gz 2023-10-08T07:16:59.520 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124858.log 2023-10-08T07:16:59.520 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118716.log.gz 2023-10-08T07:16:59.521 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94374.log 2023-10-08T07:16:59.521 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102193.log.gz 2023-10-08T07:16:59.522 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142386.log 2023-10-08T07:16:59.522 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124858.log.gz 2023-10-08T07:16:59.524 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121680.log 2023-10-08T07:16:59.524 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94374.log.gz 2023-10-08T07:16:59.525 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136713.log 2023-10-08T07:16:59.525 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142386.log.gz 2023-10-08T07:16:59.526 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93468.log 2023-10-08T07:16:59.526 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121680.log.gz 2023-10-08T07:16:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126437.log 2023-10-08T07:16:59.527 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136713.log.gz 2023-10-08T07:16:59.528 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155566.log 2023-10-08T07:16:59.528 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93468.log.gz 2023-10-08T07:16:59.530 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124282.log 2023-10-08T07:16:59.530 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.126437.log.gz 2023-10-08T07:16:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118888.log 2023-10-08T07:16:59.531 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155566.log.gz 2023-10-08T07:16:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151646.log 2023-10-08T07:16:59.532 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124282.log.gz 2023-10-08T07:16:59.534 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103506.log 2023-10-08T07:16:59.534 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118888.log.gz 2023-10-08T07:16:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121806.log 2023-10-08T07:16:59.535 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151646.log.gz 2023-10-08T07:16:59.535 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-08T07:16:59.536 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154664.log 2023-10-08T07:16:59.536 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103506.log.gz 2023-10-08T07:16:59.537 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121806.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121806.log.gz 2023-10-08T07:16:59.538 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128121.log 2023-10-08T07:16:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100136.log 2023-10-08T07:16:59.539 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154664.log.gz 2023-10-08T07:16:59.540 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118452.log 88.6% 2023-10-08T07:16:59.540 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.128121.log.gz 2023-10-08T07:16:59.541 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95045.log 2023-10-08T07:16:59.541 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100136.log.gz 2023-10-08T07:16:59.543 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.119604.log 2023-10-08T07:16:59.543 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118452.log.gz 2023-10-08T07:16:59.544 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94046.log 2023-10-08T07:16:59.544 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95045.log.gz 2023-10-08T07:16:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.119604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.119604.log.gz 2023-10-08T07:16:59.545 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115826.log 2023-10-08T07:16:59.546 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94046.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94046.log.gz 2023-10-08T07:16:59.546 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129040.log 2023-10-08T07:16:59.547 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87519.log 2023-10-08T07:16:59.547 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115826.log.gz 2023-10-08T07:16:59.548 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117834.log 2023-10-08T07:16:59.548 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129040.log.gz 2023-10-08T07:16:59.550 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87519.log: gzip -5 87.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.87519.log.gz -- 2023-10-08T07:16:59.550 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.133273.log 2023-10-08T07:16:59.551 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117834.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117834.log.gz 2023-10-08T07:16:59.551 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109508.log 2023-10-08T07:16:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133273.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.133273.log.gz 2023-10-08T07:16:59.552 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92118.log 2023-10-08T07:16:59.553 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148854.log 2023-10-08T07:16:59.554 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109508.log.gz 2023-10-08T07:16:59.554 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107108.log 2023-10-08T07:16:59.555 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92118.log.gz 2023-10-08T07:16:59.556 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151604.log 2023-10-08T07:16:59.556 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148854.log.gz 2023-10-08T07:16:59.557 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107108.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107108.log.gz 2023-10-08T07:16:59.557 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91485.log 2023-10-08T07:16:59.558 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150618.log 2023-10-08T07:16:59.558 INFO:teuthology.orchestra.run.smithi070.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.151604.log.gz 2023-10-08T07:16:59.560 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147184.log 2023-10-08T07:16:59.560 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91485.log.gz 2023-10-08T07:16:59.561 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157265.log 2023-10-08T07:16:59.561 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150618.log.gz 2023-10-08T07:16:59.562 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147184.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147184.log.gz 2023-10-08T07:16:59.562 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.142977.log 2023-10-08T07:16:59.564 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148388.log 2023-10-08T07:16:59.564 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157265.log.gz 2023-10-08T07:16:59.564 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.142977.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.142977.log.gz 2023-10-08T07:16:59.565 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156555.log 2023-10-08T07:16:59.566 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137625.log 2023-10-08T07:16:59.566 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.148388.log.gz 2023-10-08T07:16:59.567 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99074.log 2023-10-08T07:16:59.567 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156555.log.gz 2023-10-08T07:16:59.568 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114638.log 2023-10-08T07:16:59.568 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137625.log.gz 2023-10-08T07:16:59.570 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99074.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99074.log.gz 2023-10-08T07:16:59.570 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99536.log 2023-10-08T07:16:59.571 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125326.log 2023-10-08T07:16:59.571 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114638.log.gz 2023-10-08T07:16:59.572 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149662.log 2023-10-08T07:16:59.572 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99536.log.gz 2023-10-08T07:16:59.573 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125326.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125326.log.gz 2023-10-08T07:16:59.574 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126673.log 2023-10-08T07:16:59.574 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103565.log 2023-10-08T07:16:59.575 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149662.log.gz 2023-10-08T07:16:59.576 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154832.log 2023-10-08T07:16:59.576 INFO:teuthology.orchestra.run.smithi070.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.126673.log.gz 2023-10-08T07:16:59.577 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128289.log 2023-10-08T07:16:59.577 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103565.log.gz 2023-10-08T07:16:59.578 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154832.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.154832.log.gz 2023-10-08T07:16:59.579 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153742.log 2023-10-08T07:16:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99578.log 2023-10-08T07:16:59.580 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128289.log.gz 2023-10-08T07:16:59.581 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103153.log 2023-10-08T07:16:59.581 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153742.log.gz 2023-10-08T07:16:59.582 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97019.log 2023-10-08T07:16:59.583 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99578.log.gz 2023-10-08T07:16:59.584 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121894.log 2023-10-08T07:16:59.584 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103153.log.gz 2023-10-08T07:16:59.585 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152799.log 2023-10-08T07:16:59.585 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97019.log.gz 2023-10-08T07:16:59.586 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112399.log 2023-10-08T07:16:59.586 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.121894.log.gz 2023-10-08T07:16:59.587 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109358.log 2023-10-08T07:16:59.587 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152799.log.gz 2023-10-08T07:16:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139709.log 2023-10-08T07:16:59.589 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112399.log.gz 2023-10-08T07:16:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109358.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109358.log.gz 2023-10-08T07:16:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103111.log 2023-10-08T07:16:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.139709.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.139709.log.gzgzip 2023-10-08T07:16:59.592 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.141114.log 2023-10-08T07:16:59.593 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112704.log 2023-10-08T07:16:59.593 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103111.log.gz 2023-10-08T07:16:59.593 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151310.log 2023-10-08T07:16:59.594 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141114.log.gz 2023-10-08T07:16:59.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103801.log 2023-10-08T07:16:59.595 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112704.log.gz 2023-10-08T07:16:59.596 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138694.log 2023-10-08T07:16:59.596 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151310.log.gz 2023-10-08T07:16:59.597 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118844.log 2023-10-08T07:16:59.597 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103801.log.gz 2023-10-08T07:16:59.598 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.122619.log 2023-10-08T07:16:59.598 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138694.log.gz 2023-10-08T07:16:59.599 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160127.log 2023-10-08T07:16:59.599 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118844.log.gz 2023-10-08T07:16:59.600 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.122619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.122619.log.gz 2023-10-08T07:16:59.601 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133715.log 2023-10-08T07:16:59.602 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.160127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152523.log 2023-10-08T07:16:59.602 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160127.log.gz 2023-10-08T07:16:59.603 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94045.log 2023-10-08T07:16:59.603 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133715.log.gz 2023-10-08T07:16:59.604 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117750.log 2023-10-08T07:16:59.605 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.152523.log.gz 2023-10-08T07:16:59.606 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94045.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94045.log.gz 2023-10-08T07:16:59.606 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121003.log 2023-10-08T07:16:59.607 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153826.log 2023-10-08T07:16:59.607 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117750.log.gz 2023-10-08T07:16:59.608 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128163.log 2023-10-08T07:16:59.608 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121003.log.gz 2023-10-08T07:16:59.609 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147314.log 2023-10-08T07:16:59.610 INFO:teuthology.orchestra.run.smithi070.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.153826.log.gz 2023-10-08T07:16:59.611 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151562.log 2023-10-08T07:16:59.611 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128163.log.gz 2023-10-08T07:16:59.612 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100178.log 2023-10-08T07:16:59.612 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147314.log.gz 2023-10-08T07:16:59.613 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151562.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151562.log.gz 2023-10-08T07:16:59.613 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87620.log 2023-10-08T07:16:59.615 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100178.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100178.log.gz 2023-10-08T07:16:59.615 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136011.log 2023-10-08T07:16:59.618 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137583.log 2023-10-08T07:16:59.618 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87620.log.gz 2023-10-08T07:16:59.619 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114124.log 2023-10-08T07:16:59.619 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136011.log.gz 2023-10-08T07:16:59.619 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137583.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137583.log.gz -5 2023-10-08T07:16:59.619 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.112231.log 2023-10-08T07:16:59.620 INFO:teuthology.orchestra.run.smithi181.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-08T07:16:59.620 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124595.log 2023-10-08T07:16:59.621 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114124.log.gz 2023-10-08T07:16:59.621 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112231.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112231.log.gz 2023-10-08T07:16:59.621 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120733.log 2023-10-08T07:16:59.622 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135116.log 2023-10-08T07:16:59.622 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124595.log.gz 2023-10-08T07:16:59.623 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135284.log 2023-10-08T07:16:59.623 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120733.log.gz 2023-10-08T07:16:59.624 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117445.log 2023-10-08T07:16:59.625 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135116.log.gz 2023-10-08T07:16:59.626 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151923.log 2023-10-08T07:16:59.626 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135284.log.gz 2023-10-08T07:16:59.627 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115401.log 2023-10-08T07:16:59.627 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.117445.log.gz 2023-10-08T07:16:59.628 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148728.log 2023-10-08T07:16:59.628 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151923.log.gz 2023-10-08T07:16:59.629 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137060.log 2023-10-08T07:16:59.629 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115401.log.gz 2023-10-08T07:16:59.631 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148728.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.148728.log.gz -5 2023-10-08T07:16:59.631 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.134936.log 2023-10-08T07:16:59.632 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97583.log 2023-10-08T07:16:59.632 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137060.log.gz 2023-10-08T07:16:59.633 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134936.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.134936.log.gz 2023-10-08T07:16:59.633 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158115.log 2023-10-08T07:16:59.634 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154391.log 2023-10-08T07:16:59.635 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97583.log.gz 2023-10-08T07:16:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158115.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.158115.log.gz 2023-10-08T07:16:59.636 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97163.log 2023-10-08T07:16:59.637 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140080.log 2023-10-08T07:16:59.637 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.154391.log.gz 2023-10-08T07:16:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97163.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97163.log.gz 2023-10-08T07:16:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103683.log 2023-10-08T07:16:59.639 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133357.log 2023-10-08T07:16:59.639 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140080.log.gz 2023-10-08T07:16:59.640 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151688.log 2023-10-08T07:16:59.641 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103683.log.gz 2023-10-08T07:16:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109662.log 2023-10-08T07:16:59.643 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.133357.log.gz 2023-10-08T07:16:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126837.log 2023-10-08T07:16:59.643 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.151688.log.gz 2023-10-08T07:16:59.644 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145918.log 2023-10-08T07:16:59.644 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109662.log.gz 2023-10-08T07:16:59.646 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121129.log 2023-10-08T07:16:59.646 INFO:teuthology.orchestra.run.smithi070.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126837.log.gz 2023-10-08T07:16:59.647 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.145918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.138881.log 2023-10-08T07:16:59.647 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145918.log.gz 2023-10-08T07:16:59.648 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124679.log 2023-10-08T07:16:59.648 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121129.log.gz 2023-10-08T07:16:59.649 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.138881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.138881.log.gz 2023-10-08T07:16:59.649 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.152986.log 2023-10-08T07:16:59.650 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127995.log 2023-10-08T07:16:59.650 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124679.log.gz 2023-10-08T07:16:59.651 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.152986.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.152986.log.gz 2023-10-08T07:16:59.652 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134641.log 2023-10-08T07:16:59.653 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127995.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127995.log.gz 2023-10-08T07:16:59.653 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141240.log 2023-10-08T07:16:59.654 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155398.log 2023-10-08T07:16:59.654 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134641.log.gz 2023-10-08T07:16:59.655 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141240.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141240.log.gz 2023-10-08T07:16:59.655 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148686.log 2023-10-08T07:16:59.656 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112992.log 2023-10-08T07:16:59.656 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155398.log.gz 2023-10-08T07:16:59.657 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98990.log 2023-10-08T07:16:59.657 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.148686.log.gz 2023-10-08T07:16:59.659 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136892.log 2023-10-08T07:16:59.659 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112992.log.gz 2023-10-08T07:16:59.660 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153276.log 2023-10-08T07:16:59.660 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98990.log.gz 2023-10-08T07:16:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136892.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136892.log.gz 2023-10-08T07:16:59.661 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142556.log 2023-10-08T07:16:59.662 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92034.log 2023-10-08T07:16:59.662 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153276.log.gz 2023-10-08T07:16:59.663 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146423.log 2023-10-08T07:16:59.663 INFO:teuthology.orchestra.run.smithi070.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.142556.log.gz 2023-10-08T07:16:59.665 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120607.log 2023-10-08T07:16:59.665 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.92034.log.gz 2023-10-08T07:16:59.666 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146801.log 2023-10-08T07:16:59.666 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146423.log.gz 2023-10-08T07:16:59.667 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120607.log: gzip 87.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.120607.log.gz -- 2023-10-08T07:16:59.667 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.99158.log 2023-10-08T07:16:59.668 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129400.log 2023-10-08T07:16:59.669 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.146801.log.gz 2023-10-08T07:16:59.669 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138460.log 2023-10-08T07:16:59.670 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99158.log.gz 2023-10-08T07:16:59.671 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134298.log 2023-10-08T07:16:59.671 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.129400.log.gz 2023-10-08T07:16:59.672 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138460.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138460.log.gz 2023-10-08T07:16:59.672 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98030.log 2023-10-08T07:16:59.673 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124511.log 2023-10-08T07:16:59.674 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.134298.log.gz 2023-10-08T07:16:59.674 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129987.log 2023-10-08T07:16:59.675 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98030.log.gz 2023-10-08T07:16:59.676 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107014.log 2023-10-08T07:16:59.676 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124511.log.gz 2023-10-08T07:16:59.677 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111584.log 2023-10-08T07:16:59.677 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129987.log.gz 2023-10-08T07:16:59.678 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107014.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107014.log.gz 2023-10-08T07:16:59.678 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120334.log 2023-10-08T07:16:59.679 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132101.log 2023-10-08T07:16:59.680 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111584.log.gz 2023-10-08T07:16:59.681 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97205.log 2023-10-08T07:16:59.681 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120334.log.gz 2023-10-08T07:16:59.682 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.132101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101076.log 2023-10-08T07:16:59.682 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.132101.log.gz 2023-10-08T07:16:59.683 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106201.log 2023-10-08T07:16:59.683 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97205.log.gz 2023-10-08T07:16:59.684 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104037.log 2023-10-08T07:16:59.685 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.101076.log.gz 2023-10-08T07:16:59.686 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157878.log 2023-10-08T07:16:59.686 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106201.log.gz 2023-10-08T07:16:59.687 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104037.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.104037.log.gz -- 2023-10-08T07:16:59.687 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.148770.log 2023-10-08T07:16:59.688 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91298.log 2023-10-08T07:16:59.688 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157878.log.gz 2023-10-08T07:16:59.690 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160001.log 2023-10-08T07:16:59.690 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148770.log.gz 2023-10-08T07:16:59.691 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126614.log 2023-10-08T07:16:59.691 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91298.log.gz 2023-10-08T07:16:59.692 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.160001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149280.log 2023-10-08T07:16:59.692 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160001.log.gz 2023-10-08T07:16:59.693 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122366.log 2023-10-08T07:16:59.693 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126614.log.gz 2023-10-08T07:16:59.695 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126378.log 2023-10-08T07:16:59.695 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.149280.log.gz 2023-10-08T07:16:59.696 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133883.log 2023-10-08T07:16:59.696 INFO:teuthology.orchestra.run.smithi070.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.122366.log.gz 2023-10-08T07:16:59.697 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112784.log 2023-10-08T07:16:59.697 INFO:teuthology.orchestra.run.smithi070.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.126378.log.gz 2023-10-08T07:16:59.699 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96767.log 2023-10-08T07:16:59.699 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133883.log.gz 2023-10-08T07:16:59.699 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112784.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112784.log.gz 2023-10-08T07:16:59.699 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100907.log 2023-10-08T07:16:59.701 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95556.log 2023-10-08T07:16:59.701 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96767.log.gz 2023-10-08T07:16:59.702 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100907.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100907.log.gz -5 2023-10-08T07:16:59.702 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.150973.log 2023-10-08T07:16:59.703 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95556.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95556.log.gz 2023-10-08T07:16:59.703 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141595.log 2023-10-08T07:16:59.704 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128606.log 2023-10-08T07:16:59.705 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.150973.log.gz 2023-10-08T07:16:59.705 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141595.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.141595.log.gz 2023-10-08T07:16:59.706 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151184.log 2023-10-08T07:16:59.707 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111187.log 2023-10-08T07:16:59.708 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128606.log.gz 2023-10-08T07:16:59.708 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151184.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151184.log.gz 2023-10-08T07:16:59.709 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107897.log 2023-10-08T07:16:59.710 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125815.log 2023-10-08T07:16:59.710 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111187.log.gz 2023-10-08T07:16:59.710 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107897.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107897.log.gz 2023-10-08T07:16:59.711 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102075.log 2023-10-08T07:16:59.712 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118972.log 2023-10-08T07:16:59.712 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125815.log.gz 2023-10-08T07:16:59.713 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102075.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102075.log.gz 2023-10-08T07:16:59.714 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154748.log 2023-10-08T07:16:59.715 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122282.log 2023-10-08T07:16:59.716 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118972.log.gz 2023-10-08T07:16:59.717 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154748.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.154748.log.gz 2023-10-08T07:16:59.717 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155356.log 2023-10-08T07:16:59.718 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122282.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122282.log.gz 2023-10-08T07:16:59.718 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147763.log 2023-10-08T07:16:59.719 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155272.log 2023-10-08T07:16:59.719 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155356.log.gz 2023-10-08T07:16:59.720 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147763.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.147763.log.gz 2023-10-08T07:16:59.720 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115065.log 2023-10-08T07:16:59.721 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117876.log 2023-10-08T07:16:59.721 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155272.log.gz 2023-10-08T07:16:59.722 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115065.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115065.log.gz 2023-10-08T07:16:59.722 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98906.log 2023-10-08T07:16:59.723 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117224.log 2023-10-08T07:16:59.724 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117876.log.gz 2023-10-08T07:16:59.725 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98906.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98906.log.gz 2023-10-08T07:16:59.725 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142682.log 2023-10-08T07:16:59.726 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125062.log 2023-10-08T07:16:59.726 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117224.log.gz 2023-10-08T07:16:59.727 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142682.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142682.log.gz 2023-10-08T07:16:59.727 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156947.log 2023-10-08T07:16:59.728 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148980.log 2023-10-08T07:16:59.729 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.125062.log.gz 2023-10-08T07:16:59.729 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156947.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.156947.log.gz 2023-10-08T07:16:59.729 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155986.log 2023-10-08T07:16:59.731 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148980.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.148980.log.gz 2023-10-08T07:16:59.731 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135368.log 2023-10-08T07:16:59.732 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155986.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155986.log.gz 2023-10-08T07:16:59.732 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137144.log 2023-10-08T07:16:59.733 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135368.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.135368.log.gz 2023-10-08T07:16:59.733 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113500.log 2023-10-08T07:16:59.734 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93594.log 2023-10-08T07:16:59.734 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137144.log.gz 2023-10-08T07:16:59.735 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113500.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113500.log.gz 2023-10-08T07:16:59.735 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141553.log 2023-10-08T07:16:59.737 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113650.log 2023-10-08T07:16:59.737 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93594.log.gz 2023-10-08T07:16:59.738 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148304.log 88.8% 2023-10-08T07:16:59.738 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.141553.log.gz 2023-10-08T07:16:59.739 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119927.log 2023-10-08T07:16:59.739 INFO:teuthology.orchestra.run.smithi070.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.113650.log.gz 2023-10-08T07:16:59.740 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148304.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148304.log.gz 2023-10-08T07:16:59.740 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125749.log 2023-10-08T07:16:59.741 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145871.log 2023-10-08T07:16:59.741 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119927.log.gz 2023-10-08T07:16:59.742 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100390.log 2023-10-08T07:16:59.742 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.125749.log.gz 2023-10-08T07:16:59.743 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.145871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93264.log 2023-10-08T07:16:59.743 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145871.log.gz 2023-10-08T07:16:59.745 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138376.log 2023-10-08T07:16:59.745 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100390.log.gz 2023-10-08T07:16:59.746 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93264.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93264.log.gz 2023-10-08T07:16:59.746 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108732.log 2023-10-08T07:16:59.747 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138376.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138376.log.gz 2023-10-08T07:16:59.748 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116218.log 2023-10-08T07:16:59.748 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122072.log 2023-10-08T07:16:59.749 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108732.log.gz 2023-10-08T07:16:59.750 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116218.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116218.log.gz 2023-10-08T07:16:59.750 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119099.log 2023-10-08T07:16:59.751 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97667.log 2023-10-08T07:16:59.751 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122072.log.gz 2023-10-08T07:16:59.752 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119099.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119099.log.gz 2023-10-08T07:16:59.753 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147636.log 2023-10-08T07:16:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120691.log 2023-10-08T07:16:59.754 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97667.log.gz 2023-10-08T07:16:59.754 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147636.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.147636.log.gz 2023-10-08T07:16:59.755 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100484.log 2023-10-08T07:16:59.756 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153448.log 2023-10-08T07:16:59.757 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120691.log.gz 2023-10-08T07:16:59.757 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100484.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100484.log.gz 2023-10-08T07:16:59.758 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149064.log 2023-10-08T07:16:59.759 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145396.log 2023-10-08T07:16:59.759 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.153448.log.gz 2023-10-08T07:16:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117708.log 2023-10-08T07:16:59.761 INFO:teuthology.orchestra.run.smithi070.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.149064.log.gz 2023-10-08T07:16:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.145396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100822.log 2023-10-08T07:16:59.762 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145396.log.gz 2023-10-08T07:16:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117708.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117708.log.gz 2023-10-08T07:16:59.763 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154517.log 2023-10-08T07:16:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121087.log 2023-10-08T07:16:59.765 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100822.log.gz 2023-10-08T07:16:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154517.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154517.log.gz 2023-10-08T07:16:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151226.log 2023-10-08T07:16:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91442.log 2023-10-08T07:16:59.768 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121087.log.gz 2023-10-08T07:16:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109092.log 2023-10-08T07:16:59.769 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.151226.log.gz 2023-10-08T07:16:59.769 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91442.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.91442.log.gz 2023-10-08T07:16:59.770 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95346.log 2023-10-08T07:16:59.771 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146152.log 2023-10-08T07:16:59.771 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109092.log.gz 2023-10-08T07:16:59.772 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95346.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95346.log.gz 2023-10-08T07:16:59.772 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114897.log 2023-10-08T07:16:59.774 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146152.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146152.log.gz -5 2023-10-08T07:16:59.774 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96893.log 2023-10-08T07:16:59.775 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136273.log 2023-10-08T07:16:59.775 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.114897.log.gz 2023-10-08T07:16:59.776 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96893.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96893.log.gz 2023-10-08T07:16:59.776 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118613.log 2023-10-08T07:16:59.777 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134130.log 2023-10-08T07:16:59.778 INFO:teuthology.orchestra.run.smithi070.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.136273.log.gz 2023-10-08T07:16:59.778 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118613.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118613.log.gz 2023-10-08T07:16:59.779 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146068.log 2023-10-08T07:16:59.780 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94718.log 2023-10-08T07:16:59.780 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134130.log.gz 2023-10-08T07:16:59.781 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146068.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.146068.log.gz 2023-10-08T07:16:59.782 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99116.log 2023-10-08T07:16:59.783 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98660.log 2023-10-08T07:16:59.783 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94718.log.gz 2023-10-08T07:16:59.784 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99116.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99116.log.gz 2023-10-08T07:16:59.784 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127514.log 2023-10-08T07:16:59.786 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98660.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98660.log.gz 2023-10-08T07:16:59.786 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148602.log 2023-10-08T07:16:59.787 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118532.log 2023-10-08T07:16:59.787 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127514.log.gz 2023-10-08T07:16:59.788 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148602.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148602.log.gz 2023-10-08T07:16:59.788 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150931.log 2023-10-08T07:16:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139751.log 2023-10-08T07:16:59.789 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.118532.log.gz 2023-10-08T07:16:59.790 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150931.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.150931.log.gz 2023-10-08T07:16:59.790 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148518.log 2023-10-08T07:16:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.139751.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139751.log.gz 2023-10-08T07:16:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91866.log 2023-10-08T07:16:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148518.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.148518.log.gz 2023-10-08T07:16:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153362.log 2023-10-08T07:16:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91866.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.91866.log.gz 2023-10-08T07:16:59.795 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119790.log 2023-10-08T07:16:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106727.log 2023-10-08T07:16:59.796 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153362.log.gz 2023-10-08T07:16:59.797 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119790.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.119790.log.gz 2023-10-08T07:16:59.797 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137881.log 2023-10-08T07:16:59.799 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106727.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.106727.log.gz 2023-10-08T07:16:59.799 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.149966.log 2023-10-08T07:16:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111908.log 2023-10-08T07:16:59.800 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137881.log.gz 2023-10-08T07:16:59.801 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149966.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149966.log.gz 2023-10-08T07:16:59.801 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134466.log 2023-10-08T07:16:59.802 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121944.log 2023-10-08T07:16:59.802 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111908.log.gz 2023-10-08T07:16:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134466.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.134466.log.gz 2023-10-08T07:16:59.804 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155608.log 2023-10-08T07:16:59.805 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121944.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.121944.log.gz 2023-10-08T07:16:59.805 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123613.log 2023-10-08T07:16:59.806 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159329.log 2023-10-08T07:16:59.807 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155608.log.gz 2023-10-08T07:16:59.808 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123613.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123613.log.gz 2023-10-08T07:16:59.808 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152089.log 2023-10-08T07:16:59.809 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125104.log 2023-10-08T07:16:59.810 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159329.log.gz 2023-10-08T07:16:59.811 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101957.log 2023-10-08T07:16:59.811 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152089.log.gz 2023-10-08T07:16:59.811 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125104.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125104.log.gz 2023-10-08T07:16:59.812 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129208.log 2023-10-08T07:16:59.813 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93180.log 2023-10-08T07:16:59.813 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101957.log.gz 2023-10-08T07:16:59.814 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129208.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129208.log.gz 2023-10-08T07:16:59.814 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121213.log 2023-10-08T07:16:59.816 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93180.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.93180.log.gz --verbose 2023-10-08T07:16:59.816 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.142344.log 2023-10-08T07:16:59.817 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121213.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121213.log.gz 2023-10-08T07:16:59.817 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128690.log 2023-10-08T07:16:59.818 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142344.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.142344.log.gz 2023-10-08T07:16:59.819 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123970.log 2023-10-08T07:16:59.820 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113374.log 2023-10-08T07:16:59.820 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128690.log.gz 2023-10-08T07:16:59.820 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123970.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123970.log.gz 2023-10-08T07:16:59.821 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125538.log 2023-10-08T07:16:59.823 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107203.log 2023-10-08T07:16:59.823 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113374.log.gz 2023-10-08T07:16:59.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115910.log 2023-10-08T07:16:59.824 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125538.log.gz 2023-10-08T07:16:59.825 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107203.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107203.log.gz 2023-10-08T07:16:59.825 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94956.log 2023-10-08T07:16:59.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146633.log 2023-10-08T07:16:59.827 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115910.log.gz 2023-10-08T07:16:59.827 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94956.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94956.log.gz 2023-10-08T07:16:59.828 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126714.log 2023-10-08T07:16:59.829 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112273.log 2023-10-08T07:16:59.829 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146633.log.gz 2023-10-08T07:16:59.830 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126714.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.126714.log.gz 2023-10-08T07:16:59.831 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136976.log 2023-10-08T07:16:59.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92076.log 2023-10-08T07:16:59.832 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112273.log.gz 2023-10-08T07:16:59.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136976.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136976.log.gz 2023-10-08T07:16:59.833 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152353.log 2023-10-08T07:16:59.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92076.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92076.log.gz 2023-10-08T07:16:59.835 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98156.log 2023-10-08T07:16:59.836 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152353.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.152353.log.gz 2023-10-08T07:16:59.836 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112315.log 2023-10-08T07:16:59.837 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98156.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98156.log.gz 2023-10-08T07:16:59.837 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148938.log 2023-10-08T07:16:59.838 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121556.log 2023-10-08T07:16:59.839 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112315.log.gz 2023-10-08T07:16:59.840 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148938.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.148938.log.gz 2023-10-08T07:16:59.840 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148476.log 2023-10-08T07:16:59.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121556.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121556.log.gzgzip 2023-10-08T07:16:59.841 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.100992.log 2023-10-08T07:16:59.843 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148476.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148476.log.gz 2023-10-08T07:16:59.843 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138206.log 2023-10-08T07:16:59.844 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136534.log 2023-10-08T07:16:59.844 INFO:teuthology.orchestra.run.smithi070.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.100992.log.gz 2023-10-08T07:16:59.845 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149536.log 2023-10-08T07:16:59.845 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138206.log.gz 2023-10-08T07:16:59.846 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112357.log 2023-10-08T07:16:59.847 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136534.log.gz 2023-10-08T07:16:59.848 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129665.log 2023-10-08T07:16:59.848 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149536.log.gz 2023-10-08T07:16:59.849 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156309.log 2023-10-08T07:16:59.849 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112357.log.gz 2023-10-08T07:16:59.850 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129707.log 2023-10-08T07:16:59.850 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129665.log.gz 2023-10-08T07:16:59.852 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148178.log 2023-10-08T07:16:59.852 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.156309.log.gz 2023-10-08T07:16:59.853 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118800.log 2023-10-08T07:16:59.853 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129707.log.gz 2023-10-08T07:16:59.854 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148178.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.148178.log.gz -- 2023-10-08T07:16:59.854 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.117182.log 2023-10-08T07:16:59.856 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118800.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118800.log.gz 2023-10-08T07:16:59.856 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149620.log 2023-10-08T07:16:59.857 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98702.log 2023-10-08T07:16:59.857 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.117182.log.gz 2023-10-08T07:16:59.858 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100568.log 2023-10-08T07:16:59.859 INFO:teuthology.orchestra.run.smithi070.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.149620.log.gz 2023-10-08T07:16:59.859 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93038.log 2023-10-08T07:16:59.860 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98702.log.gz 2023-10-08T07:16:59.861 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100568.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.100568.log.gz 2023-10-08T07:16:59.861 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97625.log 2023-10-08T07:16:59.862 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141072.log 2023-10-08T07:16:59.862 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93038.log.gz 2023-10-08T07:16:59.863 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150847.log 2023-10-08T07:16:59.864 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97625.log.gz 2023-10-08T07:16:59.865 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146380.log 2023-10-08T07:16:59.865 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141072.log.gz 2023-10-08T07:16:59.866 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111858.log 2023-10-08T07:16:59.867 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150847.log.gz 2023-10-08T07:16:59.868 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146380.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146380.log.gz 2023-10-08T07:16:59.868 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148896.log 2023-10-08T07:16:59.869 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111858.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111858.log.gz 2023-10-08T07:16:59.869 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135536.log 2023-10-08T07:16:59.871 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146008.log 2023-10-08T07:16:59.871 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.148896.log.gz 2023-10-08T07:16:59.872 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142122.log 2023-10-08T07:16:59.872 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135536.log.gz 2023-10-08T07:16:59.873 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91397.log 2023-10-08T07:16:59.873 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.146008.log.gz 2023-10-08T07:16:59.875 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142122.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142122.log.gz 2023-10-08T07:16:59.875 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120649.log 2023-10-08T07:16:59.876 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138112.log 2023-10-08T07:16:59.876 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91397.log.gz 2023-10-08T07:16:59.877 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119417.log 2023-10-08T07:16:59.877 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120649.log.gz 2023-10-08T07:16:59.879 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138112.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138112.log.gz -5 2023-10-08T07:16:59.879 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99704.log 2023-10-08T07:16:59.880 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119417.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119417.log.gz 2023-10-08T07:16:59.880 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121476.log 2023-10-08T07:16:59.882 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140862.log 2023-10-08T07:16:59.882 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99704.log.gz 2023-10-08T07:16:59.883 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121476.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121476.log.gz 2023-10-08T07:16:59.883 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119267.log 2023-10-08T07:16:59.884 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108690.log 2023-10-08T07:16:59.884 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140862.log.gz 2023-10-08T07:16:59.885 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140946.log 2023-10-08T07:16:59.886 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119267.log.gz 2023-10-08T07:16:59.887 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108690.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108690.log.gz 2023-10-08T07:16:59.887 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159097.log 2023-10-08T07:16:59.888 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150762.log 2023-10-08T07:16:59.888 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140946.log.gz 2023-10-08T07:16:59.889 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159097.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159097.log.gz 2023-10-08T07:16:59.890 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105412.log 2023-10-08T07:16:59.891 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159791.log 2023-10-08T07:16:59.891 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150762.log.gz 2023-10-08T07:16:59.892 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105412.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105412.log.gz 2023-10-08T07:16:59.892 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151436.log 2023-10-08T07:16:59.894 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114771.log 2023-10-08T07:16:59.894 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.159791.log.gz 2023-10-08T07:16:59.894 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151436.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151436.log.gz 2023-10-08T07:16:59.895 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151142.log 2023-10-08T07:16:59.896 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.148058.log 2023-10-08T07:16:59.896 INFO:teuthology.orchestra.run.smithi070.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.114771.log.gz 2023-10-08T07:16:59.897 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151142.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151142.log.gz 2023-10-08T07:16:59.898 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129587.log 2023-10-08T07:16:59.898 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.148058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.148058.log.gz 2023-10-08T07:16:59.904 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141030.log 2023-10-08T07:16:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.129587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.129587.log.gz 2023-10-08T07:16:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156112.log 2023-10-08T07:16:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141030.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141030.log.gz 2023-10-08T07:16:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103237.log 2023-10-08T07:16:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157157.log 2023-10-08T07:16:59.905 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156112.log.gz 2023-10-08T07:16:59.906 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103237.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103237.log.gz 2023-10-08T07:16:59.906 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120292.log 2023-10-08T07:16:59.908 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157157.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157157.log.gz -5 2023-10-08T07:16:59.908 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.143056.log 2023-10-08T07:16:59.909 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141996.log 2023-10-08T07:16:59.909 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120292.log.gz 2023-10-08T07:16:59.909 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.143056.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143056.log.gz 2023-10-08T07:16:59.909 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120208.log 2023-10-08T07:16:59.911 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115359.log 2023-10-08T07:16:59.911 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141996.log.gz 2023-10-08T07:16:59.911 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120208.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120208.log.gz 2023-10-08T07:16:59.911 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134766.log 2023-10-08T07:16:59.913 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108510.log 2023-10-08T07:16:59.913 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115359.log.gz 2023-10-08T07:16:59.914 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134766.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134766.log.gz 2023-10-08T07:16:59.914 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116362.log 2023-10-08T07:16:59.916 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108510.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108510.log.gz 2023-10-08T07:16:59.916 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93510.log 2023-10-08T07:16:59.916 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123697.log 2023-10-08T07:16:59.916 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116362.log.gz 2023-10-08T07:16:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93510.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93510.log.gz 2023-10-08T07:16:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107371.log 2023-10-08T07:16:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123697.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.123697.log.gz --verbose 2023-10-08T07:16:59.919 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.95598.log 2023-10-08T07:16:59.920 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137966.log 2023-10-08T07:16:59.920 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107371.log.gz 2023-10-08T07:16:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95598.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95598.log.gz 2023-10-08T07:16:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151730.log 2023-10-08T07:16:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138544.log 2023-10-08T07:16:59.923 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137966.log.gz 2023-10-08T07:16:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151730.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.151730.log.gz 2023-10-08T07:16:59.924 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.98072.log 2023-10-08T07:16:59.925 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138544.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138544.log.gz 2023-10-08T07:16:59.925 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127419.log 2023-10-08T07:16:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98072.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-08T07:16:59.927 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98072.log.gz 2023-10-08T07:16:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127419.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127419.log.gz 2023-10-08T07:16:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157836.log 2023-10-08T07:16:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127556.log 2023-10-08T07:16:59.930 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157836.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.157836.log.gz 2023-10-08T07:16:59.931 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123928.log 2023-10-08T07:16:59.932 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127556.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.127556.log.gz 2023-10-08T07:16:59.932 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157680.log 2023-10-08T07:16:59.933 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123928.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123928.log.gz 2023-10-08T07:16:59.934 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127598.log 2023-10-08T07:16:59.935 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157680.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157680.log.gz 2023-10-08T07:16:59.936 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100654.log 2023-10-08T07:16:59.936 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127598.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127598.log.gz 2023-10-08T07:16:59.937 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140428.log 2023-10-08T07:16:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100654.log: 90.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-08T07:16:59.939 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100654.log.gz 2023-10-08T07:16:59.939 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135452.log 2023-10-08T07:16:59.940 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140428.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140428.log.gz 2023-10-08T07:16:59.940 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156683.log 2023-10-08T07:16:59.942 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94381.log 2023-10-08T07:16:59.942 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135452.log.gz 2023-10-08T07:16:59.942 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156683.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.156683.log.gz 2023-10-08T07:16:59.942 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97709.log 2023-10-08T07:16:59.944 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133757.log 2023-10-08T07:16:59.944 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94381.log.gz 2023-10-08T07:16:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97709.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97709.log.gz 2023-10-08T07:16:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158430.log 2023-10-08T07:16:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133757.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.133757.log.gz 2023-10-08T07:16:59.946 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107645.log 2023-10-08T07:16:59.947 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152565.log 2023-10-08T07:16:59.947 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158430.log.gz 2023-10-08T07:16:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107645.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107645.log.gz 2023-10-08T07:16:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107981.log 2023-10-08T07:16:59.950 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134980.log 2023-10-08T07:16:59.950 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152565.log.gz 2023-10-08T07:16:59.951 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136576.log 2023-10-08T07:16:59.951 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107981.log.gz 2023-10-08T07:16:59.952 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134980.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134980.log.gz 2023-10-08T07:16:59.952 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102541.log 2023-10-08T07:16:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160043.log 2023-10-08T07:16:59.953 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136576.log.gz 2023-10-08T07:16:59.954 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102541.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102541.log.gz 2023-10-08T07:16:59.954 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120565.log 2023-10-08T07:16:59.956 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.160043.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.160043.log.gz -5 2023-10-08T07:16:59.956 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.115191.log 2023-10-08T07:16:59.957 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120565.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120565.log.gz 2023-10-08T07:16:59.958 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154475.log 2023-10-08T07:16:59.959 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115191.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115191.log.gz 2023-10-08T07:16:59.959 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112567.log 2023-10-08T07:16:59.960 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122114.log 2023-10-08T07:16:59.960 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154475.log.gz 2023-10-08T07:16:59.961 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112567.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112567.log.gz 2023-10-08T07:16:59.962 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146717.log 2023-10-08T07:16:59.962 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148220.log 2023-10-08T07:16:59.963 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122114.log.gz 2023-10-08T07:16:59.964 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146717.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.146717.log.gz 2023-10-08T07:16:59.964 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117350.log 2023-10-08T07:16:59.965 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149238.log 2023-10-08T07:16:59.965 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.148220.log.gz 2023-10-08T07:16:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117350.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117350.log.gz 2023-10-08T07:16:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114592.log 2023-10-08T07:16:59.968 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149238.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149238.log.gz 2023-10-08T07:16:59.968 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116134.log 2023-10-08T07:16:59.969 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114592.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114592.log.gz 2023-10-08T07:16:59.969 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155000.log 2023-10-08T07:16:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108858.log 2023-10-08T07:16:59.971 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116134.log.gz 2023-10-08T07:16:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155000.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.155000.log.gz 2023-10-08T07:16:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155482.log 2023-10-08T07:16:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99326.log 2023-10-08T07:16:59.973 INFO:teuthology.orchestra.run.smithi070.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108858.log.gz 2023-10-08T07:16:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155482.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155482.log.gz 2023-10-08T07:16:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95093.log 2023-10-08T07:16:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99326.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99326.log.gz 2023-10-08T07:16:59.977 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112044.log 2023-10-08T07:16:59.977 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95093.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95093.log.gz 2023-10-08T07:16:59.977 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120250.log 2023-10-08T07:16:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116404.log 2023-10-08T07:16:59.979 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112044.log.gz 2023-10-08T07:16:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120250.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120250.log.gz 2023-10-08T07:16:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150050.log 2023-10-08T07:16:59.982 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116404.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.116404.log.gz 2023-10-08T07:16:59.982 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119309.log 2023-10-08T07:16:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150050.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150050.log.gz 2023-10-08T07:16:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94366.log 2023-10-08T07:16:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103069.log 2023-10-08T07:16:59.983 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119309.log.gz 2023-10-08T07:16:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94366.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94366.log.gz 2023-10-08T07:16:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98948.log 2023-10-08T07:16:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103069.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103069.log.gz -5 2023-10-08T07:16:59.986 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.135072.log 2023-10-08T07:16:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98948.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98948.log.gz 2023-10-08T07:16:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146896.log 2023-10-08T07:16:59.990 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92226.log 2023-10-08T07:16:59.990 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.135072.log.gz 2023-10-08T07:16:59.990 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146896.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146896.log.gz 2023-10-08T07:16:59.990 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100094.log 2023-10-08T07:16:59.990 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106846.log 2023-10-08T07:16:59.991 INFO:teuthology.orchestra.run.smithi070.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92226.log.gz 2023-10-08T07:16:59.991 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100094.log: gzip -5 88.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.100094.log.gz -- 2023-10-08T07:16:59.991 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.150576.log 2023-10-08T07:16:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114414.log 2023-10-08T07:16:59.995 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106846.log.gz 2023-10-08T07:16:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150576.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150576.log.gz 2023-10-08T07:16:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127038.log 2023-10-08T07:16:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152217.log 2023-10-08T07:16:59.996 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114414.log.gz 2023-10-08T07:16:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127038.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127038.log.gz 2023-10-08T07:16:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137720.log 2023-10-08T07:16:59.999 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159287.log 2023-10-08T07:17:00.000 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.152217.log.gz 2023-10-08T07:17:00.000 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137720.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137720.log.gz 2023-10-08T07:17:00.000 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120901.log 2023-10-08T07:17:00.000 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109567.log 2023-10-08T07:17:00.000 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159287.log.gz 2023-10-08T07:17:00.000 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120901.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120901.log.gz 2023-10-08T07:17:00.001 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.109279.log 2023-10-08T07:17:00.003 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96851.log 2023-10-08T07:17:00.003 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109567.log.gz 2023-10-08T07:17:00.003 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.109279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.109279.log.gz 2023-10-08T07:17:00.003 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122697.log 2023-10-08T07:17:00.006 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96851.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96851.log.gz 2023-10-08T07:17:00.006 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93306.log 2023-10-08T07:17:00.006 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153616.log 2023-10-08T07:17:00.007 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122697.log.gz 2023-10-08T07:17:00.007 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93306.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93306.log.gz 2023-10-08T07:17:00.007 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122977.log 2023-10-08T07:17:00.008 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141385.log 2023-10-08T07:17:00.009 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153616.log.gz 2023-10-08T07:17:00.010 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122977.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122977.log.gz 2023-10-08T07:17:00.010 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91738.log 2023-10-08T07:17:00.011 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141385.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141385.log.gz 2023-10-08T07:17:00.011 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117308.log 2023-10-08T07:17:00.012 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92912.log 2023-10-08T07:17:00.013 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91738.log.gz 2023-10-08T07:17:00.013 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117308.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117308.log.gz 2023-10-08T07:17:00.014 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109620.log 2023-10-08T07:17:00.015 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92160.log 2023-10-08T07:17:00.015 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92912.log.gz 2023-10-08T07:17:00.017 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109620.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.109620.log.gz 2023-10-08T07:17:00.017 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149578.log 2023-10-08T07:17:00.019 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126099.log 2023-10-08T07:17:00.019 INFO:teuthology.orchestra.run.smithi070.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.92160.log.gz 2023-10-08T07:17:00.019 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125188.log 2023-10-08T07:17:00.019 INFO:teuthology.orchestra.run.smithi070.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.149578.log.gz 2023-10-08T07:17:00.021 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126099.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126099.log.gz 2023-10-08T07:17:00.021 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120859.log 2023-10-08T07:17:00.021 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96935.log 2023-10-08T07:17:00.023 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125188.log.gz 2023-10-08T07:17:00.023 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120859.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.120859.log.gz 2023-10-08T07:17:00.023 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93636.log 2023-10-08T07:17:00.025 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113248.log 2023-10-08T07:17:00.025 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96935.log.gz 2023-10-08T07:17:00.026 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93636.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93636.log.gz 2023-10-08T07:17:00.026 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100304.log 2023-10-08T07:17:00.028 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157031.log 2023-10-08T07:17:00.028 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113248.log.gz 2023-10-08T07:17:00.028 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112525.log 2023-10-08T07:17:00.028 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100304.log.gz 2023-10-08T07:17:00.030 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157031.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157031.log.gz 2023-10-08T07:17:00.030 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135494.log 2023-10-08T07:17:00.030 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124198.log 2023-10-08T07:17:00.030 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112525.log.gz 2023-10-08T07:17:00.032 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95178.log 2023-10-08T07:17:00.032 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135494.log.gz 2023-10-08T07:17:00.033 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95388.log 2023-10-08T07:17:00.033 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124198.log.gz 2023-10-08T07:17:00.034 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95178.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95178.log.gz 2023-10-08T07:17:00.034 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127910.log 2023-10-08T07:17:00.036 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102457.log 2023-10-08T07:17:00.036 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95388.log.gz 2023-10-08T07:17:00.036 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104079.log 2023-10-08T07:17:00.036 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127910.log.gz 2023-10-08T07:17:00.038 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152047.log 2023-10-08T07:17:00.038 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102457.log.gz 2023-10-08T07:17:00.039 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104079.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104079.log.gz 2023-10-08T07:17:00.039 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146278.log 2023-10-08T07:17:00.041 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147364.log 2023-10-08T07:17:00.041 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.152047.log.gz 2023-10-08T07:17:00.041 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124385.log 2023-10-08T07:17:00.041 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.146278.log.gz 2023-10-08T07:17:00.043 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147364.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147364.log.gz 2023-10-08T07:17:00.043 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111723.log 2023-10-08T07:17:00.043 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116446.log 2023-10-08T07:17:00.043 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124385.log.gz 2023-10-08T07:17:00.045 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157572.log 2023-10-08T07:17:00.046 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111723.log.gz 2023-10-08T07:17:00.046 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116446.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116446.log.gz 2023-10-08T07:17:00.046 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96725.log 2023-10-08T07:17:00.049 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134850.log 2023-10-08T07:17:00.049 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157572.log.gz 2023-10-08T07:17:00.049 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141679.log 2023-10-08T07:17:00.049 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96725.log.gz 2023-10-08T07:17:00.049 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134850.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.134850.log.gz 2023-10-08T07:17:00.050 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155168.log 2023-10-08T07:17:00.051 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91595.log 2023-10-08T07:17:00.051 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.141679.log.gz 2023-10-08T07:17:00.052 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113957.log 2023-10-08T07:17:00.052 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.155168.log.gz 2023-10-08T07:17:00.053 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135620.log 2023-10-08T07:17:00.054 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91595.log.gz 2023-10-08T07:17:00.054 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113957.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113957.log.gz 2023-10-08T07:17:00.054 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134892.log 2023-10-08T07:17:00.056 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159875.log 2023-10-08T07:17:00.056 INFO:teuthology.orchestra.run.smithi070.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.135620.log.gz 2023-10-08T07:17:00.056 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100950.log 2023-10-08T07:17:00.056 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.134892.log.gz 2023-10-08T07:17:00.057 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159875.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159875.log.gz 2023-10-08T07:17:00.057 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129292.log 2023-10-08T07:17:00.058 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114939.log 2023-10-08T07:17:00.058 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100950.log.gz 2023-10-08T07:17:00.059 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126308.log 2023-10-08T07:17:00.059 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129292.log.gz 2023-10-08T07:17:00.060 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114939.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114939.log.gz 2023-10-08T07:17:00.060 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147448.log 2023-10-08T07:17:00.062 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102625.log 2023-10-08T07:17:00.062 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126308.log.gz 2023-10-08T07:17:00.063 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153490.log 2023-10-08T07:17:00.063 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147448.log.gz 2023-10-08T07:17:00.064 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102625.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102625.log.gz 2023-10-08T07:17:00.064 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158929.log 2023-10-08T07:17:00.065 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136492.log 2023-10-08T07:17:00.065 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.153490.log.gz 2023-10-08T07:17:00.066 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103919.log 2023-10-08T07:17:00.066 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.158929.log.gz 2023-10-08T07:17:00.068 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136492.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136492.log.gz 2023-10-08T07:17:00.068 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138156.log 2023-10-08T07:17:00.069 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150889.log 2023-10-08T07:17:00.069 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103919.log.gz 2023-10-08T07:17:00.070 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160199.log 2023-10-08T07:17:00.070 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138156.log.gz 2023-10-08T07:17:00.071 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150889.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.150889.log.gz 2023-10-08T07:17:00.072 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98318.log 2023-10-08T07:17:00.072 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.160199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149392.log 2023-10-08T07:17:00.073 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160199.log.gz 2023-10-08T07:17:00.073 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119141.log 2023-10-08T07:17:00.074 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98318.log.gz 2023-10-08T07:17:00.074 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157199.log 2023-10-08T07:17:00.075 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149392.log.gz 2023-10-08T07:17:00.076 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119141.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119141.log.gz 2023-10-08T07:17:00.076 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115485.log 2023-10-08T07:17:00.077 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153192.log 2023-10-08T07:17:00.078 INFO:teuthology.orchestra.run.smithi070.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.157199.log.gz 2023-10-08T07:17:00.078 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149130.log 2023-10-08T07:17:00.079 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115485.log.gz 2023-10-08T07:17:00.080 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153192.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153192.log.gz 2023-10-08T07:17:00.080 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149721.log 2023-10-08T07:17:00.081 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102373.log 2023-10-08T07:17:00.081 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149130.log.gz 2023-10-08T07:17:00.082 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121255.log 2023-10-08T07:17:00.082 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149721.log.gz 2023-10-08T07:17:00.083 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102373.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102373.log.gz 2023-10-08T07:17:00.083 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.126002.log 2023-10-08T07:17:00.084 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135578.log 2023-10-08T07:17:00.084 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.121255.log.gz 2023-10-08T07:17:00.085 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.126002.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.126002.log.gz 2023-10-08T07:17:00.085 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146549.log 2023-10-08T07:17:00.086 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119056.log 2023-10-08T07:17:00.087 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135578.log.gz 2023-10-08T07:17:00.088 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158241.log 2023-10-08T07:17:00.088 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.146549.log.gz 2023-10-08T07:17:00.088 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119056.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119056.log.gz 2023-10-08T07:17:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127199.log 2023-10-08T07:17:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97331.log 2023-10-08T07:17:00.091 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158241.log.gz 2023-10-08T07:17:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127199.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127199.log.gzgzip 2023-10-08T07:17:00.092 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.98576.log 2023-10-08T07:17:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97331.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97331.log.gz 2023-10-08T07:17:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.135891.log 2023-10-08T07:17:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114211.log 2023-10-08T07:17:00.094 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98576.log.gz 2023-10-08T07:17:00.095 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.135891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.135891.log.gz 2023-10-08T07:17:00.095 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139979.log 2023-10-08T07:17:00.097 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97751.log 2023-10-08T07:17:00.097 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114211.log.gz 2023-10-08T07:17:00.098 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.139979.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139979.log.gz 2023-10-08T07:17:00.099 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99494.log 2023-10-08T07:17:00.099 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97751.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97751.log.gz 2023-10-08T07:17:00.100 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94369.log 2023-10-08T07:17:00.101 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155042.log 2023-10-08T07:17:00.101 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99494.log.gz 2023-10-08T07:17:00.102 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94369.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94369.log.gz 2023-10-08T07:17:00.102 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146759.log 2023-10-08T07:17:00.103 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157954.log 2023-10-08T07:17:00.103 INFO:teuthology.orchestra.run.smithi070.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.155042.log.gz 2023-10-08T07:17:00.104 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146759.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.146759.log.gz 2023-10-08T07:17:00.105 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108984.log 2023-10-08T07:17:00.106 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157954.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.157954.log.gz -5 2023-10-08T07:17:00.106 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97988.log 2023-10-08T07:17:00.107 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108984.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108984.log.gz 2023-10-08T07:17:00.107 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-08T07:17:00.108 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124553.log 2023-10-08T07:17:00.109 INFO:teuthology.orchestra.run.smithi070.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97988.log.gz 2023-10-08T07:17:00.110 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134723.log 2023-10-08T07:17:00.111 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124553.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124553.log.gz 2023-10-08T07:17:00.111 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99746.log 2023-10-08T07:17:00.113 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96977.log 2023-10-08T07:17:00.113 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.134723.log.gz 2023-10-08T07:17:00.114 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99746.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99746.log.gz 2023-10-08T07:17:00.114 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142724.log 2023-10-08T07:17:00.116 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148812.log 2023-10-08T07:17:00.116 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96977.log.gz 2023-10-08T07:17:00.117 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95804.log 2023-10-08T07:17:00.118 INFO:teuthology.orchestra.run.smithi070.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.142724.log.gz 2023-10-08T07:17:00.119 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142598.log 2023-10-08T07:17:00.119 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148812.log.gz 2023-10-08T07:17:00.120 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95804.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95804.log.gz 2023-10-08T07:17:00.120 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151100.log 2023-10-08T07:17:00.122 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127682.log 2023-10-08T07:17:00.122 INFO:teuthology.orchestra.run.smithi070.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.142598.log.gz 2023-10-08T07:17:00.124 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151100.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.151100.log.gz 2023-10-08T07:17:00.124 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137289.log 2023-10-08T07:17:00.125 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127682.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127682.log.gz 2023-10-08T07:17:00.126 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116861.log 2023-10-08T07:17:00.127 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103978.log 2023-10-08T07:17:00.127 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137289.log.gz 2023-10-08T07:17:00.128 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116861.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116861.log.gz 2023-10-08T07:17:00.128 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92651.log 2023-10-08T07:17:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153108.log 2023-10-08T07:17:00.130 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103978.log.gz 2023-10-08T07:17:00.131 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128564.log 2023-10-08T07:17:00.132 INFO:teuthology.orchestra.run.smithi070.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.92651.log.gz 2023-10-08T07:17:00.133 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158658.log 2023-10-08T07:17:00.133 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128564.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.153108.log.gz 2023-10-08T07:17:00.134 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.128564.log.gz 2023-10-08T07:17:00.134 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141721.log 2023-10-08T07:17:00.136 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111542.log 2023-10-08T07:17:00.136 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.158658.log.gz 2023-10-08T07:17:00.137 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141721.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141721.log.gz 2023-10-08T07:17:00.138 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115998.log 2023-10-08T07:17:00.139 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150408.log 2023-10-08T07:17:00.140 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111542.log.gz 2023-10-08T07:17:00.140 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115998.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115998.log.gz 2023-10-08T07:17:00.141 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121986.log 2023-10-08T07:17:00.142 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120089.log 2023-10-08T07:17:00.143 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121986.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.150408.log.gz 2023-10-08T07:17:00.143 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121986.log.gz 2023-10-08T07:17:00.144 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99368.log 2023-10-08T07:17:00.145 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94716.log 2023-10-08T07:17:00.146 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.120089.log.gz 2023-10-08T07:17:00.146 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99368.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99368.log.gz 2023-10-08T07:17:00.147 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159581.log 2023-10-08T07:17:00.149 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153574.log 2023-10-08T07:17:00.149 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94716.log.gz 2023-10-08T07:17:00.149 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159581.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.159581.log.gz 2023-10-08T07:17:00.150 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111679.log 2023-10-08T07:17:00.152 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94044.log 2023-10-08T07:17:00.152 INFO:teuthology.orchestra.run.smithi070.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.153574.log.gz 2023-10-08T07:17:00.152 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111679.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111679.log.gz 2023-10-08T07:17:00.153 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160281.log 2023-10-08T07:17:00.154 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113078.log 2023-10-08T07:17:00.154 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94044.log.gz 2023-10-08T07:17:00.155 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.160281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112609.log 2023-10-08T07:17:00.155 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160281.log.gz 2023-10-08T07:17:00.156 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113078.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113078.log.gz 2023-10-08T07:17:00.156 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124096.log 2023-10-08T07:17:00.158 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108900.log 2023-10-08T07:17:00.158 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112609.log.gz 2023-10-08T07:17:00.159 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124096.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.124096.log.gz 2023-10-08T07:17:00.159 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137923.log 2023-10-08T07:17:00.160 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128444.log 2023-10-08T07:17:00.160 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108900.log.gz 2023-10-08T07:17:00.161 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137923.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137923.log.gz 2023-10-08T07:17:00.161 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91908.log 2023-10-08T07:17:00.162 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155692.log 2023-10-08T07:17:00.162 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128444.log.gz 2023-10-08T07:17:00.163 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91908.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91908.log.gz 2023-10-08T07:17:00.163 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108604.log 2023-10-08T07:17:00.164 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156389.log 2023-10-08T07:17:00.165 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155692.log.gz 2023-10-08T07:17:00.165 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108604.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108604.log.gz 2023-10-08T07:17:00.166 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108646.log 2023-10-08T07:17:00.167 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122906.log 2023-10-08T07:17:00.167 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156389.log.gz 2023-10-08T07:17:00.168 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108646.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108646.log.gz 2023-10-08T07:17:00.168 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156070.log 2023-10-08T07:17:00.169 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122906.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122906.log.gz 2023-10-08T07:17:00.169 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137102.log 2023-10-08T07:17:00.170 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119874.log 2023-10-08T07:17:00.170 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156070.log.gz 2023-10-08T07:17:00.171 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113290.log 2023-10-08T07:17:00.172 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137102.log.gz 2023-10-08T07:17:00.173 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119874.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119874.log.gz 2023-10-08T07:17:00.173 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138586.log 2023-10-08T07:17:00.174 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113290.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113290.log.gz 2023-10-08T07:17:00.174 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155440.log 2023-10-08T07:17:00.175 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100052.log 2023-10-08T07:17:00.175 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138586.log.gz 2023-10-08T07:17:00.176 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155440.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155440.log.gz 2023-10-08T07:17:00.176 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157794.log 2023-10-08T07:17:00.177 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100052.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100052.log.gz -5 2023-10-08T07:17:00.177 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.155650.log 2023-10-08T07:17:00.178 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133441.log 2023-10-08T07:17:00.179 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157794.log.gz 2023-10-08T07:17:00.180 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155650.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155650.log.gz 2023-10-08T07:17:00.180 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153996.log 2023-10-08T07:17:00.181 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145650.log 2023-10-08T07:17:00.181 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.133441.log.gz 2023-10-08T07:17:00.182 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153996.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.153996.log.gz 2023-10-08T07:17:00.182 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114729.log 2023-10-08T07:17:00.183 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.145650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121638.log 2023-10-08T07:17:00.183 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.145650.log.gz 2023-10-08T07:17:00.184 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114729.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114729.log.gz 2023-10-08T07:17:00.184 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98786.log 2023-10-08T07:17:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94371.log 2023-10-08T07:17:00.186 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121638.log.gz 2023-10-08T07:17:00.186 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98786.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98786.log.gz 2023-10-08T07:17:00.187 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147534.log 2023-10-08T07:17:00.188 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94371.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94371.log.gz 2023-10-08T07:17:00.188 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124054.log 2023-10-08T07:17:00.189 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93348.log 2023-10-08T07:17:00.189 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147534.log.gz 2023-10-08T07:17:00.190 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136231.log 2023-10-08T07:17:00.190 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124054.log.gz 2023-10-08T07:17:00.191 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93348.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93348.log.gz 2023-10-08T07:17:00.191 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121171.log 2023-10-08T07:17:00.193 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100262.log 2023-10-08T07:17:00.193 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136231.log.gz 2023-10-08T07:17:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121171.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121171.log.gz 2023-10-08T07:17:00.194 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128912.log 2023-10-08T07:17:00.195 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121722.log 2023-10-08T07:17:00.195 INFO:teuthology.orchestra.run.smithi070.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100262.log.gz 2023-10-08T07:17:00.196 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128912.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.128912.log.gz 2023-10-08T07:17:00.196 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141469.log 2023-10-08T07:17:00.197 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121722.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121722.log.gz 2023-10-08T07:17:00.197 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124469.log 2023-10-08T07:17:00.198 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135969.log 2023-10-08T07:17:00.198 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141469.log.gz 2023-10-08T07:17:00.200 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143318.log 2023-10-08T07:17:00.200 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124469.log.gz 2023-10-08T07:17:00.201 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135969.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135969.log.gz 2023-10-08T07:17:00.201 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93780.log 2023-10-08T07:17:00.202 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.143318.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.143318.log.gz 2023-10-08T07:17:00.202 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113458.log 2023-10-08T07:17:00.203 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153892.log 2023-10-08T07:17:00.203 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93780.log.gz 2023-10-08T07:17:00.204 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113458.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113458.log.gz 2023-10-08T07:17:00.205 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133967.log 2023-10-08T07:17:00.205 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98276.log 2023-10-08T07:17:00.206 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153892.log.gz 2023-10-08T07:17:00.207 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133967.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133967.log.gz 2023-10-08T07:17:00.207 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137373.log 2023-10-08T07:17:00.208 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95262.log 2023-10-08T07:17:00.208 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98276.log.gz 2023-10-08T07:17:00.209 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137373.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137373.log.gz 2023-10-08T07:17:00.209 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117936.log 2023-10-08T07:17:00.210 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94377.log 2023-10-08T07:17:00.210 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95262.log.gz 2023-10-08T07:17:00.211 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117936.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117936.log.gz 2023-10-08T07:17:00.212 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126555.log 2023-10-08T07:17:00.212 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134808.log 2023-10-08T07:17:00.213 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94377.log.gz 2023-10-08T07:17:00.214 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126555.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126555.log.gz 2023-10-08T07:17:00.214 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149339.log 2023-10-08T07:17:00.215 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114372.log 2023-10-08T07:17:00.215 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134808.log.gz 2023-10-08T07:17:00.216 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149339.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149339.log.gz 2023-10-08T07:17:00.216 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148262.log 2023-10-08T07:17:00.217 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152311.log 2023-10-08T07:17:00.218 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114372.log.gz 2023-10-08T07:17:00.218 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148262.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.148262.log.gz 2023-10-08T07:17:00.219 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104605.log 2023-10-08T07:17:00.220 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133841.log 2023-10-08T07:17:00.220 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.152311.log.gz 2023-10-08T07:17:00.221 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104605.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104605.log.gz 2023-10-08T07:17:00.221 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113122.log 2023-10-08T07:17:00.222 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112086.log 2023-10-08T07:17:00.223 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.133841.log.gz 2023-10-08T07:17:00.224 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113122.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113122.log.gz 2023-10-08T07:17:00.224 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141343.log 2023-10-08T07:17:00.225 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122028.log 2023-10-08T07:17:00.225 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112086.log.gz 2023-10-08T07:17:00.226 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153150.log 2023-10-08T07:17:00.226 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141343.log.gz 2023-10-08T07:17:00.227 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116305.log 2023-10-08T07:17:00.227 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122028.log.gz 2023-10-08T07:17:00.228 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153150.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153150.log.gz 2023-10-08T07:17:00.229 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135030.log 2023-10-08T07:17:00.229 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119014.log 2023-10-08T07:17:00.230 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116305.log.gz 2023-10-08T07:17:00.231 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135030.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135030.log.gz 2023-10-08T07:17:00.231 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120470.log 2023-10-08T07:17:00.232 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126878.log 2023-10-08T07:17:00.233 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119014.log.gz 2023-10-08T07:17:00.234 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120470.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120470.log.gz 2023-10-08T07:17:00.234 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128079.log 2023-10-08T07:17:00.235 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111960.log 2023-10-08T07:17:00.235 INFO:teuthology.orchestra.run.smithi070.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.126878.log.gz 2023-10-08T07:17:00.236 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128079.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128079.log.gz 2023-10-08T07:17:00.236 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148644.log 2023-10-08T07:17:00.237 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134340.log 2023-10-08T07:17:00.238 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111960.log.gz 2023-10-08T07:17:00.239 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148644.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148644.log.gz 2023-10-08T07:17:00.239 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133315.log 2023-10-08T07:17:00.240 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154916.log 2023-10-08T07:17:00.240 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134340.log.gz 2023-10-08T07:17:00.241 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133315.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.133315.log.gz 2023-10-08T07:17:00.241 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97373.log 2023-10-08T07:17:00.242 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150240.log 2023-10-08T07:17:00.242 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.154916.log.gz 2023-10-08T07:17:00.243 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97373.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97373.log.gz 2023-10-08T07:17:00.243 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140904.log 2023-10-08T07:17:00.244 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111774.log 2023-10-08T07:17:00.245 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.150240.log.gz 2023-10-08T07:17:00.245 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140904.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140904.log.gz 2023-10-08T07:17:00.246 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159917.log 2023-10-08T07:17:00.247 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107813.log 2023-10-08T07:17:00.247 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111774.log.gz 2023-10-08T07:17:00.248 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159917.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159917.log.gz 2023-10-08T07:17:00.248 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147492.log 2023-10-08T07:17:00.249 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139239.log 2023-10-08T07:17:00.249 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107813.log.gz 2023-10-08T07:17:00.250 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147492.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147492.log.gzgzip 2023-10-08T07:17:00.250 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.156471.log 2023-10-08T07:17:00.251 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.139239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104342.log 2023-10-08T07:17:00.252 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139239.log.gz 2023-10-08T07:17:00.253 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156471.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.156471.log.gz 2023-10-08T07:17:00.253 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159959.log 2023-10-08T07:17:00.254 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135200.log 2023-10-08T07:17:00.254 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104342.log.gz 2023-10-08T07:17:00.255 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159959.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.159959.log.gz 2023-10-08T07:17:00.255 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92954.log 2023-10-08T07:17:00.256 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136808.log 2023-10-08T07:17:00.256 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135200.log.gz 2023-10-08T07:17:00.257 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92954.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92954.log.gz 2023-10-08T07:17:00.257 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141834.log 2023-10-08T07:17:00.259 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159539.log 2023-10-08T07:17:00.259 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136808.log.gz 2023-10-08T07:17:00.260 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141834.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141834.log.gz 2023-10-08T07:17:00.260 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116923.log 2023-10-08T07:17:00.261 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133799.log 2023-10-08T07:17:00.261 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159539.log.gz 2023-10-08T07:17:00.262 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116923.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116923.log.gz 2023-10-08T07:17:00.262 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149924.log 2023-10-08T07:17:00.263 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96118.log 2023-10-08T07:17:00.263 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.133799.log.gz 2023-10-08T07:17:00.264 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149924.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149924.log.gz 2023-10-08T07:17:00.265 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136618.log 2023-10-08T07:17:00.265 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.96118.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.125232.log -- replaced with /var/log/ceph/ceph-client.0.96118.log.gz 2023-10-08T07:17:00.265 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:17:00.267 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136618.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136618.log.gz 2023-10-08T07:17:00.267 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137204.log 2023-10-08T07:17:00.268 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115868.log 2023-10-08T07:17:00.269 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137204.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125232.log.gz 2023-10-08T07:17:00.269 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137204.log.gz 2023-10-08T07:17:00.270 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145966.log 2023-10-08T07:17:00.271 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101557.log 2023-10-08T07:17:00.272 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115868.log.gz 2023-10-08T07:17:00.272 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.145966.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145966.log.gz 2023-10-08T07:17:00.273 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115660.log 2023-10-08T07:17:00.274 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150660.log 2023-10-08T07:17:00.275 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115660.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101557.log.gz 2023-10-08T07:17:00.275 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115660.log.gz 2023-10-08T07:17:00.276 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92996.log 2023-10-08T07:17:00.277 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114330.log 2023-10-08T07:17:00.278 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150660.log.gz 2023-10-08T07:17:00.278 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92996.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92996.log.gz 2023-10-08T07:17:00.279 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105675.log 2023-10-08T07:17:00.280 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147594.log 2023-10-08T07:17:00.281 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114330.log.gz 2023-10-08T07:17:00.281 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105675.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105675.log.gz 2023-10-08T07:17:00.282 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93822.log 2023-10-08T07:17:00.284 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108774.log 2023-10-08T07:17:00.284 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147594.log.gz 2023-10-08T07:17:00.284 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93822.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93822.log.gz 2023-10-08T07:17:00.285 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134561.log 2023-10-08T07:17:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155084.log 2023-10-08T07:17:00.287 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108774.log.gz 2023-10-08T07:17:00.288 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134561.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134561.log.gz 2023-10-08T07:17:00.288 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126496.log 2023-10-08T07:17:00.290 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155524.log 2023-10-08T07:17:00.290 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126496.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.155084.log.gz 2023-10-08T07:17:00.291 INFO:teuthology.orchestra.run.smithi070.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.126496.log.gz 2023-10-08T07:17:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102583.log 2023-10-08T07:17:00.293 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108466.log 2023-10-08T07:17:00.293 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155524.log.gz 2023-10-08T07:17:00.294 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102583.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102583.log.gz 2023-10-08T07:17:00.294 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158742.log 2023-10-08T07:17:00.298 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114023.log 2023-10-08T07:17:00.298 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108466.log.gz 2023-10-08T07:17:00.298 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158742.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158742.log.gz 2023-10-08T07:17:00.298 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139064.log 2023-10-08T07:17:00.299 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125664.log 2023-10-08T07:17:00.299 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114023.log.gz 2023-10-08T07:17:00.300 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.139064.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.139064.log.gz 2023-10-08T07:17:00.300 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117498.log 2023-10-08T07:17:00.302 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142038.log 2023-10-08T07:17:00.302 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117498.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125664.log.gz 2023-10-08T07:17:00.303 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117498.log.gz 2023-10-08T07:17:00.303 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105149.log 2023-10-08T07:17:00.305 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128648.log 2023-10-08T07:17:00.305 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142038.log.gz 2023-10-08T07:17:00.306 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105149.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.105149.log.gz 2023-10-08T07:17:00.306 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-08T07:17:00.307 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124240.log 2023-10-08T07:17:00.308 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128648.log.gz 2023-10-08T07:17:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150450.log 2023-10-08T07:17:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124240.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124240.log.gz 2023-10-08T07:17:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156513.log 2023-10-08T07:17:00.312 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111297.log 2023-10-08T07:17:00.312 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150450.log.gz 2023-10-08T07:17:00.314 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156513.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156513.log.gz 2023-10-08T07:17:00.314 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95640.log 2023-10-08T07:17:00.318 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145776.log 2023-10-08T07:17:00.318 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111297.log.gz 2023-10-08T07:17:00.318 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117624.log 2023-10-08T07:17:00.318 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.95640.log.gz 2023-10-08T07:17:00.319 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.145776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93222.log 2023-10-08T07:17:00.319 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.145776.log.gz 2023-10-08T07:17:00.320 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117624.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117624.log.gz 2023-10-08T07:17:00.320 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124342.log 2023-10-08T07:17:00.322 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93222.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93222.log.gz 2023-10-08T07:17:00.322 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137800.log 2023-10-08T07:17:00.323 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138960.log 2023-10-08T07:17:00.323 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124342.log.gz 2023-10-08T07:17:00.325 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138628.log 2023-10-08T07:17:00.325 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137800.log.gz 2023-10-08T07:17:00.326 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138960.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138960.log.gz -5 2023-10-08T07:17:00.327 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.128732.log 2023-10-08T07:17:00.328 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140589.log 2023-10-08T07:17:00.329 INFO:teuthology.orchestra.run.smithi070.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.138628.log.gz 2023-10-08T07:17:00.330 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128732.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128732.log.gz 2023-10-08T07:17:00.330 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125622.log 2023-10-08T07:17:00.331 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103321.log 2023-10-08T07:17:00.331 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.140589.log.gz 2023-10-08T07:17:00.332 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99452.log 2023-10-08T07:17:00.333 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125622.log.gz 2023-10-08T07:17:00.334 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103321.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103321.log.gz 2023-10-08T07:17:00.334 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150282.log 2023-10-08T07:17:00.336 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99452.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99452.log.gz 2023-10-08T07:17:00.336 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152607.log 2023-10-08T07:17:00.337 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93552.log 2023-10-08T07:17:00.337 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150282.log.gz 2023-10-08T07:17:00.339 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155776.log 2023-10-08T07:17:00.339 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.152607.log.gz 2023-10-08T07:17:00.340 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117266.log 2023-10-08T07:17:00.340 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93552.log.gz 2023-10-08T07:17:00.341 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123452.log 2023-10-08T07:17:00.342 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.155776.log.gz 2023-10-08T07:17:00.343 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117266.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117266.log.gz 2023-10-08T07:17:00.343 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141298.log 2023-10-08T07:17:00.345 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129945.log 2023-10-08T07:17:00.345 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123452.log.gz 2023-10-08T07:17:00.346 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96683.log 2023-10-08T07:17:00.346 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141298.log.gz 2023-10-08T07:17:00.348 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154874.log 2023-10-08T07:17:00.348 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129945.log.gz 2023-10-08T07:17:00.349 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125276.log 2023-10-08T07:17:00.349 INFO:teuthology.orchestra.run.smithi070.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.96683.log.gz 2023-10-08T07:17:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91642.log 2023-10-08T07:17:00.351 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154874.log.gz 2023-10-08T07:17:00.352 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102134.log 2023-10-08T07:17:00.353 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125276.log.gz 2023-10-08T07:17:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119351.log 2023-10-08T07:17:00.354 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91642.log.gz 2023-10-08T07:17:00.355 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116176.log 2023-10-08T07:17:00.355 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102134.log.gz 2023-10-08T07:17:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142472.log 2023-10-08T07:17:00.358 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.119351.log.gz 2023-10-08T07:17:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118673.log 2023-10-08T07:17:00.359 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116176.log.gz 2023-10-08T07:17:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137457.log 2023-10-08T07:17:00.360 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.142472.log.gz 2023-10-08T07:17:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151268.log 2023-10-08T07:17:00.362 INFO:teuthology.orchestra.run.smithi070.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.118673.log.gz 2023-10-08T07:17:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136934.log 2023-10-08T07:17:00.363 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137457.log.gz 2023-10-08T07:17:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152131.log 2023-10-08T07:17:00.365 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151268.log.gz 2023-10-08T07:17:00.366 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136934.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136934.log.gz 2023-10-08T07:17:00.366 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156819.log 2023-10-08T07:17:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140988.log 2023-10-08T07:17:00.368 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152131.log.gz 2023-10-08T07:17:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128774.log 2023-10-08T07:17:00.369 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156819.log.gz 2023-10-08T07:17:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140988.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140988.log.gz 2023-10-08T07:17:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136119.log 2023-10-08T07:17:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128954.log 2023-10-08T07:17:00.372 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128774.log.gz 2023-10-08T07:17:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136119.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136119.log.gz 2023-10-08T07:17:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102499.log 2023-10-08T07:17:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128954.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128954.log.gz 2023-10-08T07:17:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93080.log 2023-10-08T07:17:00.376 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158532.log 2023-10-08T07:17:00.376 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102499.log.gz 2023-10-08T07:17:00.378 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93080.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93080.log.gz 2023-10-08T07:17:00.378 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123655.log 2023-10-08T07:17:00.378 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158532.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.158532.log.gz 2023-10-08T07:17:00.379 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125707.log 2023-10-08T07:17:00.380 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143140.log 2023-10-08T07:17:00.381 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123655.log.gz 2023-10-08T07:17:00.382 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125707.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125707.log.gz 2023-10-08T07:17:00.382 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101780.log 2023-10-08T07:17:00.383 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.143140.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143140.log.gz 2023-10-08T07:17:00.383 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154790.log 2023-10-08T07:17:00.384 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98360.log 2023-10-08T07:17:00.385 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101780.log.gz 2023-10-08T07:17:00.386 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154790.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.154790.log.gz 2023-10-08T07:17:00.386 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115580.log 2023-10-08T07:17:00.387 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98360.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98360.log.gz 2023-10-08T07:17:00.388 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140673.log 2023-10-08T07:17:00.389 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156597.log 2023-10-08T07:17:00.389 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115580.log.gz 2023-10-08T07:17:00.390 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140673.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140673.log.gz 2023-10-08T07:17:00.390 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121764.log 2023-10-08T07:17:00.392 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156597.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156597.log.gz 2023-10-08T07:17:00.392 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120005.log 2023-10-08T07:17:00.393 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107855.log 2023-10-08T07:17:00.393 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121764.log.gz 2023-10-08T07:17:00.395 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120005.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120005.log.gz 2023-10-08T07:17:00.395 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124978.log 2023-10-08T07:17:00.396 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107855.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107855.log.gz 2023-10-08T07:17:00.396 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152649.log 2023-10-08T07:17:00.398 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158844.log 2023-10-08T07:17:00.398 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124978.log.gz 2023-10-08T07:17:00.399 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152649.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152649.log.gz 2023-10-08T07:17:00.399 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112950.log 2023-10-08T07:17:00.400 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158844.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158844.log.gz 2023-10-08T07:17:00.400 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133631.log 2023-10-08T07:17:00.402 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99410.log 2023-10-08T07:17:00.402 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112950.log.gz 2023-10-08T07:17:00.403 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133631.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133631.log.gz 2023-10-08T07:17:00.403 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113206.log 2023-10-08T07:17:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99410.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99410.log.gz 2023-10-08T07:17:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113416.log 2023-10-08T07:17:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124721.log 2023-10-08T07:17:00.406 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113206.log.gz 2023-10-08T07:17:00.407 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113416.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113416.log.gz 2023-10-08T07:17:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139186.log 2023-10-08T07:17:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124721.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124721.log.gz 2023-10-08T07:17:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159455.log 2023-10-08T07:17:00.410 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.139186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132143.log 2023-10-08T07:17:00.410 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139186.log.gz 2023-10-08T07:17:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159455.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159455.log.gz 2023-10-08T07:17:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106888.log 2023-10-08T07:17:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.132143.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132143.log.gz 2023-10-08T07:17:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106464.log 2023-10-08T07:17:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155229.log 2023-10-08T07:17:00.415 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106888.log.gz 2023-10-08T07:17:00.415 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106464.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106464.log.gz 2023-10-08T07:17:00.416 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117792.log 2023-10-08T07:17:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155229.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155229.log.gz 2023-10-08T07:17:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146236.log 2023-10-08T07:17:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117792.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117792.log.gz 2023-10-08T07:17:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142164.log 2023-10-08T07:17:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146236.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.146236.log.gz 2023-10-08T07:17:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97289.log 2023-10-08T07:17:00.422 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112002.log 2023-10-08T07:17:00.422 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142164.log.gz 2023-10-08T07:17:00.424 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97289.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.97289.log.gz -5 2023-10-08T07:17:00.424 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94717.log 2023-10-08T07:17:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151394.log 2023-10-08T07:17:00.425 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112002.log.gz 2023-10-08T07:17:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94717.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94717.log.gz 2023-10-08T07:17:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150092.log 2023-10-08T07:17:00.428 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148346.log 2023-10-08T07:17:00.428 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151394.log.gz 2023-10-08T07:17:00.430 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150092.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.150092.log.gz 2023-10-08T07:17:00.430 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125368.log 2023-10-08T07:17:00.433 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125020.log 2023-10-08T07:17:00.433 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.148346.log.gz 2023-10-08T07:17:00.433 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125368.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125368.log.gz 2023-10-08T07:17:00.434 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100526.log 2023-10-08T07:17:00.434 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98744.log 2023-10-08T07:17:00.435 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125020.log.gz 2023-10-08T07:17:00.436 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100526.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100526.log.gz 2023-10-08T07:17:00.436 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152691.log 2023-10-08T07:17:00.437 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98744.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98744.log.gz 2023-10-08T07:17:00.437 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113837.log 2023-10-08T07:17:00.439 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152691.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152691.log.gz 2023-10-08T07:17:00.440 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93426.log 2023-10-08T07:17:00.441 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.113837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113837.log.gz 2023-10-08T07:17:00.441 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153934.log 2023-10-08T07:17:00.442 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93426.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93426.log.gz 2023-10-08T07:17:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138334.log 2023-10-08T07:17:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118021.log 2023-10-08T07:17:00.444 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.153934.log.gz 2023-10-08T07:17:00.445 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138334.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138334.log.gz 2023-10-08T07:17:00.446 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154433.log 2023-10-08T07:17:00.447 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118021.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118021.log.gz 2023-10-08T07:17:00.447 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153700.log 2023-10-08T07:17:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94030.log 2023-10-08T07:17:00.448 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154433.log.gz 2023-10-08T07:17:00.450 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153700.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153700.log.gz 2023-10-08T07:17:00.450 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120775.log 2023-10-08T07:17:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94030.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94030.log.gz 2023-10-08T07:17:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108023.log 2023-10-08T07:17:00.453 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107541.log 2023-10-08T07:17:00.453 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120775.log.gz 2023-10-08T07:17:00.454 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108023.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108023.log.gz 2023-10-08T07:17:00.454 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146320.log 2023-10-08T07:17:00.455 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107541.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107541.log.gz 2023-10-08T07:17:00.456 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159413.log 2023-10-08T07:17:00.457 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126796.log 2023-10-08T07:17:00.457 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146320.log.gz 2023-10-08T07:17:00.458 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159413.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159413.log.gz 2023-10-08T07:17:00.458 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103860.log 2023-10-08T07:17:00.459 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126796.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.126796.log.gz 2023-10-08T07:17:00.460 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126249.log 2023-10-08T07:17:00.461 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101142.log 2023-10-08T07:17:00.461 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103860.log.gz 2023-10-08T07:17:00.463 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126249.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.126249.log.gz 2023-10-08T07:17:00.463 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105938.log 2023-10-08T07:17:00.464 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115233.log 2023-10-08T07:17:00.465 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101142.log.gz 2023-10-08T07:17:00.466 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105938.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105938.log.gz 2023-10-08T07:17:00.466 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95304.log 2023-10-08T07:17:00.467 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159707.log 2023-10-08T07:17:00.467 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115233.log.gz 2023-10-08T07:17:00.469 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95304.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95304.log.gz 2023-10-08T07:17:00.469 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133588.log 2023-10-08T07:17:00.470 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159707.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.159707.log.gz 2023-10-08T07:17:00.470 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108816.log 2023-10-08T07:17:00.472 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121850.log 2023-10-08T07:17:00.472 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133588.log.gz 2023-10-08T07:17:00.473 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108816.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108816.log.gz 2023-10-08T07:17:00.473 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146110.log 2023-10-08T07:17:00.474 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121850.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121850.log.gz 2023-10-08T07:17:00.475 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94040.log 2023-10-08T07:17:00.476 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146591.log 2023-10-08T07:17:00.476 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146110.log.gz 2023-10-08T07:17:00.478 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94040.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.94040.log.gz 2023-10-08T07:17:00.478 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112865.log 2023-10-08T07:17:00.479 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146591.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146591.log.gz 2023-10-08T07:17:00.479 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117063.log 2023-10-08T07:17:00.481 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112865.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112865.log.gz 2023-10-08T07:17:00.481 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123571.log 2023-10-08T07:17:00.482 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98486.log 2023-10-08T07:17:00.482 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117063.log.gz 2023-10-08T07:17:00.484 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123571.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123571.log.gz 2023-10-08T07:17:00.484 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94043.log 2023-10-08T07:17:00.485 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114498.log 2023-10-08T07:17:00.485 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98486.log.gz 2023-10-08T07:17:00.487 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94043.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94043.log.gz 2023-10-08T07:17:00.487 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99284.log 2023-10-08T07:17:00.488 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152005.log 2023-10-08T07:17:00.488 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114498.log.gz 2023-10-08T07:17:00.490 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99284.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99284.log.gz 2023-10-08T07:17:00.490 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95472.log 2023-10-08T07:17:00.491 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152005.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152005.log.gz 2023-10-08T07:17:00.491 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155944.log 2023-10-08T07:17:00.492 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99844.log 2023-10-08T07:17:00.492 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95472.log.gz 2023-10-08T07:17:00.494 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155944.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155944.log.gz 2023-10-08T07:17:00.494 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154958.log 2023-10-08T07:17:00.496 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127724.log 2023-10-08T07:17:00.496 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99844.log.gz 2023-10-08T07:17:00.497 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119225.log 2023-10-08T07:17:00.497 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.154958.log.gz 2023-10-08T07:17:00.498 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127724.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127724.log.gz 2023-10-08T07:17:00.499 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-08T07:17:00.500 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116554.log 2023-10-08T07:17:00.500 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119225.log.gz 2023-10-08T07:17:00.501 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113584.log 2023-10-08T07:17:00.503 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135326.log 2023-10-08T07:17:00.503 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116554.log.gz 2023-10-08T07:17:00.504 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158700.log 2023-10-08T07:17:00.505 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.113584.log.gz 2023-10-08T07:17:00.506 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135326.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135326.log.gz 2023-10-08T07:17:00.506 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129874.log 2023-10-08T07:17:00.507 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91780.log 2023-10-08T07:17:00.507 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158700.log.gz 2023-10-08T07:17:00.509 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117979.log 2023-10-08T07:17:00.509 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129874.log.gz 2023-10-08T07:17:00.511 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154188.log 88.6% 2023-10-08T07:17:00.511 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.91780.log.gz 2023-10-08T07:17:00.512 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117979.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117979.log.gz 2023-10-08T07:17:00.512 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115275.log 2023-10-08T07:17:00.513 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154188.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.154188.log.gz --verbose 2023-10-08T07:17:00.513 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.87263.log 2023-10-08T07:17:00.520 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119183.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115275.log.gz 2023-10-08T07:17:00.520 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:17:00.520 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87263.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.87263.log.gz 2023-10-08T07:17:00.521 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95220.log 2023-10-08T07:17:00.521 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119183.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.150324.log 2023-10-08T07:17:00.521 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.119183.log.gz 2023-10-08T07:17:00.521 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95220.log: gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95220.log.gz 2023-10-08T07:17:00.521 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96253.log 2023-10-08T07:17:00.522 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150324.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150324.log.gz 2023-10-08T07:17:00.522 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117666.log 2023-10-08T07:17:00.523 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96253.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96253.log.gz 2023-10-08T07:17:00.523 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98114.log 2023-10-08T07:17:00.525 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117666.log: gzip 88.4% -5 --verbose -- /var/log/ceph/ceph-client.admin.150702.log 2023-10-08T07:17:00.525 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.117666.log.gz 2023-10-08T07:17:00.526 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98114.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98114.log.gz 2023-10-08T07:17:00.526 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137018.log 2023-10-08T07:17:00.528 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94678.log 2023-10-08T07:17:00.528 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.150702.log.gz 2023-10-08T07:17:00.529 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137018.log: gzip -5 88.1% --verbose -- /var/log/ceph/ceph-client.admin.94695.log 2023-10-08T07:17:00.529 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.137018.log.gz 2023-10-08T07:17:00.531 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94678.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94678.log.gz 2023-10-08T07:17:00.531 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102330.log 2023-10-08T07:17:00.532 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118315.log 2023-10-08T07:17:00.532 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94695.log.gz 2023-10-08T07:17:00.534 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102330.log: gzip -5 --verbose 88.7% -- /var/log/ceph/ceph-client.admin.109466.log 2023-10-08T07:17:00.534 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102330.log.gz 2023-10-08T07:17:00.535 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118315.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118315.log.gz 2023-10-08T07:17:00.535 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100220.log 2023-10-08T07:17:00.537 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136450.log 2023-10-08T07:17:00.537 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109466.log.gz 2023-10-08T07:17:00.539 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100220.log: gzip -5 --verbose -- 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100220.log.gz 2023-10-08T07:17:00.539 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.91539.log 2023-10-08T07:17:00.540 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136450.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136450.log.gz 2023-10-08T07:17:00.541 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147058.log 2023-10-08T07:17:00.541 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91539.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91539.log.gz 2023-10-08T07:17:00.542 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157638.log 2023-10-08T07:17:00.544 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147058.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147058.log.gz 2023-10-08T07:17:00.544 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115443.log 2023-10-08T07:17:00.545 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157638.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.157638.log.gz 2023-10-08T07:17:00.546 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123359.log 2023-10-08T07:17:00.547 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115443.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115443.log.gz 2023-10-08T07:17:00.547 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100696.log 2023-10-08T07:17:00.549 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123359.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123359.log.gz 2023-10-08T07:17:00.549 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156196.log 2023-10-08T07:17:00.551 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100696.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100696.log.gz 2023-10-08T07:17:00.551 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114456.log 2023-10-08T07:17:00.553 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156196.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156196.log.gz 2023-10-08T07:17:00.553 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147226.log 2023-10-08T07:17:00.554 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114456.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114456.log.gz 2023-10-08T07:17:00.554 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118105.log 2023-10-08T07:17:00.556 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147226.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147226.log.gz 2023-10-08T07:17:00.556 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123833.log 2023-10-08T07:17:00.558 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118105.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118105.log.gz 2023-10-08T07:17:00.558 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149840.log 2023-10-08T07:17:00.559 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123833.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123833.log.gz 2023-10-08T07:17:00.560 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103363.log 2023-10-08T07:17:00.561 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149840.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149840.log.gz 2023-10-08T07:17:00.561 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136765.log 2023-10-08T07:17:00.563 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103363.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103363.log.gz 2023-10-08T07:17:00.563 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94914.log 2023-10-08T07:17:00.565 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136765.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136765.log.gz 2023-10-08T07:17:00.565 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107729.log 2023-10-08T07:17:00.567 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94914.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94914.log.gz 2023-10-08T07:17:00.567 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115742.log 2023-10-08T07:17:00.568 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107729.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107729.log.gz 2023-10-08T07:17:00.568 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115149.log 2023-10-08T07:17:00.569 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115742.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115742.log.gz 2023-10-08T07:17:00.570 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94650.log 2023-10-08T07:17:00.571 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115149.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115149.log.gz 2023-10-08T07:17:00.572 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142208.log 2023-10-08T07:17:00.573 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94650.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94650.log.gz 2023-10-08T07:17:00.574 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159371.log 2023-10-08T07:17:00.575 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142208.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.142208.log.gz 2023-10-08T07:17:00.575 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125496.log 2023-10-08T07:17:00.577 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159371.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159371.log.gz 2023-10-08T07:17:00.577 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143265.log 2023-10-08T07:17:00.578 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125496.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125496.log.gz 2023-10-08T07:17:00.579 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160085.log 2023-10-08T07:17:00.581 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.143265.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143265.log.gz 2023-10-08T07:17:00.581 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91694.log 2023-10-08T07:17:00.582 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.160085.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.160085.log.gz 2023-10-08T07:17:00.583 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114855.log 2023-10-08T07:17:00.584 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91694.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91694.log.gz 2023-10-08T07:17:00.584 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127953.log 2023-10-08T07:17:00.586 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114855.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156727.log 2023-10-08T07:17:00.586 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.114855.log.gz 2023-10-08T07:17:00.587 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101898.log 2023-10-08T07:17:00.587 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127953.log.gz 2023-10-08T07:17:00.588 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156727.log: gzip 87.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.156727.log.gz --verbose 2023-10-08T07:17:00.589 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.115317.log 2023-10-08T07:17:00.590 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101898.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101898.log.gz 2023-10-08T07:17:00.590 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150805.log 2023-10-08T07:17:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115317.log: gzip -5 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115317.log.gz 2023-10-08T07:17:00.592 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106930.log 2023-10-08T07:17:00.593 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150805.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150805.log.gz 2023-10-08T07:17:00.594 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141198.log 2023-10-08T07:17:00.595 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106930.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106930.log.gz 2023-10-08T07:17:00.596 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107939.log 2023-10-08T07:17:00.596 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141198.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141198.log.gz 2023-10-08T07:17:00.597 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146675.log 2023-10-08T07:17:00.598 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107939.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107939.log.gz 2023-10-08T07:17:00.599 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129124.log 2023-10-08T07:17:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91950.log 2023-10-08T07:17:00.600 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.146675.log.gz 2023-10-08T07:17:00.602 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129124.log: gzip -5 --verbose -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129124.log.gz 2023-10-08T07:17:00.602 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.95514.log 2023-10-08T07:17:00.604 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91950.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.91950.log.gz 2023-10-08T07:17:00.604 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81236.log 2023-10-08T07:17:00.605 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95514.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95514.log.gz 2023-10-08T07:17:00.605 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154349.log 2023-10-08T07:17:00.607 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.tmp-client.admin.81236.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81236.log.gz 2023-10-08T07:17:00.607 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125454.log 2023-10-08T07:17:00.609 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154349.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154349.log.gz 2023-10-08T07:17:00.609 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153066.log 2023-10-08T07:17:00.610 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125454.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125454.log.gz 2023-10-08T07:17:00.611 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94714.log 2023-10-08T07:17:00.612 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153066.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.153066.log.gz 2023-10-08T07:17:00.612 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155860.log 2023-10-08T07:17:00.613 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94714.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94714.log.gz 2023-10-08T07:17:00.614 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159749.log 2023-10-08T07:17:00.615 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155860.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155860.log.gz 2023-10-08T07:17:00.616 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107456.log 2023-10-08T07:17:00.617 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159749.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159749.log.gz 2023-10-08T07:17:00.618 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142430.log 2023-10-08T07:17:00.619 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108424.log 2023-10-08T07:17:00.619 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107456.log.gz 2023-10-08T07:17:00.621 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142430.log: gzip -5 --verbose 87.4% -- replaced with /var/log/ceph/ceph-client.admin.142430.log.gz 2023-10-08T07:17:00.621 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.155818.log 2023-10-08T07:17:00.621 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108424.log: 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-08T07:17:00.622 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108424.log.gz 2023-10-08T07:17:00.622 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124637.log 2023-10-08T07:17:00.623 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150198.log 2023-10-08T07:17:00.624 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.155818.log.gz 2023-10-08T07:17:00.625 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124637.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124637.log.gz 2023-10-08T07:17:00.625 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155314.log 2023-10-08T07:17:00.626 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159665.log 2023-10-08T07:17:00.626 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150198.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150198.log.gz 2023-10-08T07:17:00.627 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155314.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155314.log.gz 2023-10-08T07:17:00.627 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158335.log 2023-10-08T07:17:00.628 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.159665.log: /var/log/ceph/ceph-client.admin.160239.log 2023-10-08T07:17:00.629 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159665.log.gz 2023-10-08T07:17:00.629 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158335.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158335.log.gz 2023-10-08T07:17:00.629 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96809.log 2023-10-08T07:17:00.630 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.160239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91822.log 2023-10-08T07:17:00.631 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160239.log.gz 2023-10-08T07:17:00.632 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96809.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96809.log.gz 2023-10-08T07:17:00.633 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-08T07:17:00.633 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112441.log 2023-10-08T07:17:00.633 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91822.log.gz 2023-10-08T07:17:00.635 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141427.log 2023-10-08T07:17:00.636 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142640.log 2023-10-08T07:17:00.637 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112441.log.gz 2023-10-08T07:17:00.638 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97121.log 2023-10-08T07:17:00.638 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141427.log.gz 2023-10-08T07:17:00.639 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142640.log: gzip 88.8% -5 --verbose -- /var/log/ceph/ceph-client.admin.111416.log 2023-10-08T07:17:00.639 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.142640.log.gz 2023-10-08T07:17:00.641 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141914.log 2023-10-08T07:17:00.641 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97121.log.gz 2023-10-08T07:17:00.642 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140715.log 2023-10-08T07:17:00.642 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111416.log.gz 2023-10-08T07:17:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141914.log: gzip -5 --verbose -- 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141914.log.gz 2023-10-08T07:17:00.644 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.152733.log 2023-10-08T07:17:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.140715.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.140715.log.gz 2023-10-08T07:17:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147871.log 2023-10-08T07:17:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97499.log 2023-10-08T07:17:00.647 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.152733.log.gz 2023-10-08T07:17:00.649 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147871.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.147871.log.gz 2023-10-08T07:17:00.649 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156861.log 2023-10-08T07:17:00.650 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97499.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97499.log.gz 2023-10-08T07:17:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93738.log 2023-10-08T07:17:00.652 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107161.log 2023-10-08T07:17:00.652 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.156861.log.gz 2023-10-08T07:17:00.654 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93738.log: gzip -5 --verbose 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93738.log.gz 2023-10-08T07:17:00.654 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.100346.log 2023-10-08T07:17:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107161.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107161.log.gz 2023-10-08T07:17:00.656 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146976.log 2023-10-08T07:17:00.657 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100346.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100346.log.gz 2023-10-08T07:17:00.657 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135686.log 2023-10-08T07:17:00.659 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146976.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146976.log.gz 2023-10-08T07:17:00.659 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.120047.log 2023-10-08T07:17:00.660 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135686.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135686.log.gz 2023-10-08T07:17:00.661 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151843.log 2023-10-08T07:17:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120047.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120047.log.gz 2023-10-08T07:17:00.662 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147721.log 2023-10-08T07:17:00.664 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151843.log: gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.151843.log.gz 2023-10-08T07:17:00.664 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.157115.log 2023-10-08T07:17:00.666 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147721.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147721.log.gz 2023-10-08T07:17:00.666 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91202.log 2023-10-08T07:17:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157115.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157115.log.gz 2023-10-08T07:17:00.668 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158199.log 2023-10-08T07:17:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91202.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91202.log.gz 2023-10-08T07:17:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95135.log 2023-10-08T07:17:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158199.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158199.log.gz 2023-10-08T07:17:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152395.log 2023-10-08T07:17:00.673 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95135.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95135.log.gz 2023-10-08T07:17:00.673 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136178.log 2023-10-08T07:17:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.152395.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152395.log.gz 2023-10-08T07:17:00.675 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103742.log 2023-10-08T07:17:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136178.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136178.log.gz 2023-10-08T07:17:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120817.log 2023-10-08T07:17:00.678 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103742.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103742.log.gz 2023-10-08T07:17:00.678 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138008.log 2023-10-08T07:17:00.680 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120817.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120817.log.gz 2023-10-08T07:17:00.680 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139110.log 2023-10-08T07:17:00.681 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138008.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.138008.log.gz 2023-10-08T07:17:00.682 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142790.log 2023-10-08T07:17:00.683 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.139110.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139110.log.gz 2023-10-08T07:17:00.683 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112483.log 2023-10-08T07:17:00.685 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142790.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142790.log.gz 2023-10-08T07:17:00.685 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123886.log 2023-10-08T07:17:00.687 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112483.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112483.log.gz 2023-10-08T07:17:00.687 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129250.log 2023-10-08T07:17:00.688 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123886.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123886.log.gz 2023-10-08T07:17:00.689 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148136.log 2023-10-08T07:17:00.690 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129250.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.129250.log.gz 2023-10-08T07:17:00.690 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99925.log 2023-10-08T07:17:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148136.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.148136.log.gz 2023-10-08T07:17:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127850.log 2023-10-08T07:17:00.694 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99925.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99925.log.gz 2023-10-08T07:17:00.694 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103624.log 2023-10-08T07:17:00.695 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127850.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127850.log.gz 2023-10-08T07:17:00.695 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159623.log 2023-10-08T07:17:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103624.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103624.log.gz 2023-10-08T07:17:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101515.log 2023-10-08T07:17:00.698 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159623.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159623.log.gz 2023-10-08T07:17:00.699 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155126.log 2023-10-08T07:17:00.700 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101515.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.101515.log.gz 2023-10-08T07:17:00.700 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92870.log 2023-10-08T07:17:00.702 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155126.log: gzip 88.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.148432.log 2023-10-08T07:17:00.702 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.155126.log.gz 2023-10-08T07:17:00.703 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137331.log 2023-10-08T07:17:00.703 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92870.log.gz 2023-10-08T07:17:00.705 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.148432.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.148432.log.gz 2023-10-08T07:17:00.705 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158887.log 2023-10-08T07:17:00.707 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137331.log: gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137331.log.gz 2023-10-08T07:17:00.707 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.98198.log 2023-10-08T07:17:00.708 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158887.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158887.log.gz 2023-10-08T07:17:00.708 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150534.log 2023-10-08T07:17:00.710 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98198.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.98198.log.gz 2023-10-08T07:17:00.710 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.114687.log 2023-10-08T07:17:00.711 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150534.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.150534.log.gz 2023-10-08T07:17:00.712 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.107413.log 2023-10-08T07:17:00.713 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114687.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114687.log.gz 2023-10-08T07:17:00.713 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147100.log 2023-10-08T07:17:00.714 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107413.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107413.log.gz 2023-10-08T07:17:00.715 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146194.log 2023-10-08T07:17:00.716 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158388.log 2023-10-08T07:17:00.716 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147100.log.gz 2023-10-08T07:17:00.718 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.146194.log: gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146194.log.gz 2023-10-08T07:17:00.718 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph.log 2023-10-08T07:17:00.719 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158388.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158388.log.gz 2023-10-08T07:17:00.722 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113542.log 2023-10-08T07:17:00.722 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157530.log 2023-10-08T07:17:00.723 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129166.log 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113542.log.gz 2023-10-08T07:17:00.723 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:17:00.724 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157530.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157530.log.gz 2023-10-08T07:17:00.724 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150492.log 2023-10-08T07:17:00.726 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129166.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129166.log.gz 2023-10-08T07:17:00.726 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.118357.log 2023-10-08T07:17:00.727 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.150492.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.150492.log.gz 2023-10-08T07:17:00.727 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129334.log 2023-10-08T07:17:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118357.log: gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118357.log.gz 2023-10-08T07:17:00.729 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.147270.log 2023-10-08T07:17:00.732 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129334.log: gzip -5 90.8% -- replaced with /var/log/ceph/ceph-client.admin.129334.log.gz 2023-10-08T07:17:00.732 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.126183.log 2023-10-08T07:17:00.732 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-08T07:17:00.732 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147270.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147270.log.gz 2023-10-08T07:17:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142302.log 2023-10-08T07:17:00.734 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156905.log 2023-10-08T07:17:00.734 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126183.log.gz 2023-10-08T07:17:00.735 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97247.log 2023-10-08T07:17:00.736 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.142302.log.gz 2023-10-08T07:17:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156905.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156905.log.gz 2023-10-08T07:17:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118273.log 2023-10-08T07:17:00.739 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97247.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97247.log.gz 2023-10-08T07:17:00.739 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158616.log 2023-10-08T07:17:00.740 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118273.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118273.log.gz 2023-10-08T07:17:00.740 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101721.log 2023-10-08T07:17:00.742 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158616.log: gzip -5 --verbose 88.6% -- /var/log/ceph/ceph-client.admin.155734.log -- replaced with /var/log/ceph/ceph-client.admin.158616.log.gz 2023-10-08T07:17:00.742 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:17:00.743 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101721.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101721.log.gz 2023-10-08T07:17:00.743 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-08T07:17:00.745 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155734.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.155734.log.gz 2023-10-08T07:17:00.745 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154307.log 2023-10-08T07:17:00.747 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122432.log 2023-10-08T07:17:00.748 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154307.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.154307.log.gz 2023-10-08T07:17:00.748 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127640.log 2023-10-08T07:17:00.750 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122432.log: gzip -5 --verbose 88.5% -- /var/log/ceph/ceph-client.admin.157073.log 2023-10-08T07:17:00.750 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.122432.log.gz 2023-10-08T07:17:00.751 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127640.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113915.log 2023-10-08T07:17:00.751 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.127640.log.gz 2023-10-08T07:17:00.753 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.157073.log: gzip -5 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157073.log.gz 2023-10-08T07:17:00.753 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.134027.log 2023-10-08T07:17:00.754 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113915.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113915.log.gz 2023-10-08T07:17:00.754 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136850.log 2023-10-08T07:17:00.756 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134027.log: gzip -5 --verbose -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134027.log.gz 2023-10-08T07:17:00.756 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.138292.log 2023-10-08T07:17:00.757 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.136850.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.136850.log.gz 2023-10-08T07:17:00.758 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112188.log 2023-10-08T07:17:00.760 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138292.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158157.log 2023-10-08T07:17:00.760 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.138292.log.gz 2023-10-08T07:17:00.761 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112188.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112188.log.gz 2023-10-08T07:17:00.761 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-08T07:17:00.763 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158157.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.158157.log.gz 2023-10-08T07:17:00.763 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98864.log 2023-10-08T07:17:00.765 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125410.log 2023-10-08T07:17:00.766 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98864.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98864.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108554.log 2023-10-08T07:17:00.766 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:17:00.768 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101407.log 2023-10-08T07:17:00.769 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125410.log: /var/log/ceph/ceph-client.admin.108554.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125410.log.gz 2023-10-08T07:17:00.769 INFO:teuthology.orchestra.run.smithi070.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108554.log.gz 2023-10-08T07:17:00.770 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101449.log 2023-10-08T07:17:00.772 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133673.log 2023-10-08T07:17:00.773 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101407.log: /var/log/ceph/ceph-client.admin.101449.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101407.log.gz 2023-10-08T07:17:00.774 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101449.log.gz 2023-10-08T07:17:00.774 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134256.log 2023-10-08T07:17:00.776 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142080.log 2023-10-08T07:17:00.777 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133673.log: /var/log/ceph/ceph-client.admin.134256.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133673.log.gz 2023-10-08T07:17:00.778 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.134256.log.gz 2023-10-08T07:17:00.778 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122805.log 2023-10-08T07:17:00.780 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103447.log 2023-10-08T07:17:00.781 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.142080.log: /var/log/ceph/ceph-client.admin.122805.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142080.log.gz 2023-10-08T07:17:00.782 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122805.log.gz 2023-10-08T07:17:00.782 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137499.log 2023-10-08T07:17:00.784 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127241.log 2023-10-08T07:17:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103447.log: /var/log/ceph/ceph-client.admin.137499.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103447.log.gz 2023-10-08T07:17:00.786 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137499.log.gz 2023-10-08T07:17:00.786 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136077.log 2023-10-08T07:17:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100780.log 2023-10-08T07:17:00.789 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127241.log: /var/log/ceph/ceph-client.admin.136077.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127241.log.gz 2023-10-08T07:17:00.790 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136077.log.gz 2023-10-08T07:17:00.790 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122847.log 2023-10-08T07:17:00.792 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111500.log 2023-10-08T07:17:00.793 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100780.log: /var/log/ceph/ceph-client.admin.122847.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100780.log.gz 2023-10-08T07:17:00.794 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122847.log.gz 2023-10-08T07:17:00.794 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120376.log 2023-10-08T07:17:00.796 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98444.log 2023-10-08T07:17:00.797 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111500.log: /var/log/ceph/ceph-client.admin.120376.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111500.log.gz 2023-10-08T07:17:00.798 INFO:teuthology.orchestra.run.smithi070.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.120376.log.gz 2023-10-08T07:17:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99662.log 2023-10-08T07:17:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126755.log 2023-10-08T07:17:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98444.log: /var/log/ceph/ceph-client.admin.99662.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98444.log.gz 2023-10-08T07:17:00.802 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99662.log.gz 2023-10-08T07:17:00.802 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122156.log 2023-10-08T07:17:00.804 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116969.log 2023-10-08T07:17:00.805 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126755.log: /var/log/ceph/ceph-client.admin.122156.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.126755.log.gz 2023-10-08T07:17:00.805 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.122156.log.gz 2023-10-08T07:17:00.806 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101329.log 2023-10-08T07:17:00.808 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151478.log 2023-10-08T07:17:00.809 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.101329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101329.log.gz 2023-10-08T07:17:00.809 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116969.log.gz 2023-10-08T07:17:00.810 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154559.log 2023-10-08T07:17:00.812 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151478.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.151478.log.gz 2023-10-08T07:17:00.812 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93696.log 2023-10-08T07:17:00.814 INFO:teuthology.orchestra.run.smithi070.stderr:gzip/var/log/ceph/ceph-client.admin.154559.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.149882.log 2023-10-08T07:17:00.814 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.154559.log.gz 2023-10-08T07:17:00.816 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138248.log 2023-10-08T07:17:00.817 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149882.log: 88.4% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.149882.log.gz 2023-10-08T07:17:00.817 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93696.log.gz 2023-10-08T07:17:00.819 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108136.log 2023-10-08T07:17:00.820 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138248.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138248.log.gz 2023-10-08T07:17:00.820 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151520.log 2023-10-08T07:17:00.822 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159833.log 2023-10-08T07:17:00.823 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151520.log: 88.0% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.151520.log.gz 2023-10-08T07:17:00.823 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108136.log.gz 2023-10-08T07:17:00.825 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151015.log 2023-10-08T07:17:00.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159833.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159833.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151058.log 2023-10-08T07:17:00.826 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:17:00.827 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128818.log 2023-10-08T07:17:00.829 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.151015.log: /var/log/ceph/ceph-client.admin.151058.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151015.log.gz 2023-10-08T07:17:00.829 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.151058.log.gz 2023-10-08T07:17:00.831 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129769.log 2023-10-08T07:17:00.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128818.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128818.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111458.log 2023-10-08T07:17:00.832 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:17:00.833 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128998.log 2023-10-08T07:17:00.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129769.log: /var/log/ceph/ceph-client.admin.111458.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129769.log.gz 2023-10-08T07:17:00.835 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111458.log.gz 2023-10-08T07:17:00.836 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99200.log 2023-10-08T07:17:00.837 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135158.log 2023-10-08T07:17:00.838 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128998.log: /var/log/ceph/ceph-client.admin.99200.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128998.log.gz 2023-10-08T07:17:00.839 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99200.log.gz 2023-10-08T07:17:00.840 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118147.log 2023-10-08T07:17:00.841 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154706.log 2023-10-08T07:17:00.842 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.135158.log: /var/log/ceph/ceph-client.admin.118147.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135158.log.gz 2023-10-08T07:17:00.843 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118147.log.gz 2023-10-08T07:17:00.844 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118063.log 2023-10-08T07:17:00.846 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.154706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118189.log 2023-10-08T07:17:00.847 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118063.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118063.log.gz 88.4% -- replaced with /var/log/ceph/ceph-client.admin.154706.log.gz 2023-10-08T07:17:00.847 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:17:00.848 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91992.log 2023-10-08T07:17:00.850 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97877.log 2023-10-08T07:17:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118189.log: /var/log/ceph/ceph-client.admin.91992.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118189.log.gz 2023-10-08T07:17:00.852 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91992.log.gz 2023-10-08T07:17:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119682.log 2023-10-08T07:17:00.854 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156154.log 2023-10-08T07:17:00.855 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97877.log: /var/log/ceph/ceph-client.admin.119682.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97877.log.gz 2023-10-08T07:17:00.856 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119682.log.gz 2023-10-08T07:17:00.856 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101034.log 2023-10-08T07:17:00.858 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159013.log 2023-10-08T07:17:00.859 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156154.log: /var/log/ceph/ceph-client.admin.101034.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156154.log.gz 2023-10-08T07:17:00.860 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101034.log.gz 2023-10-08T07:17:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108942.log 2023-10-08T07:17:00.862 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93138.log 2023-10-08T07:17:00.863 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159013.log: /var/log/ceph/ceph-client.admin.108942.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159013.log.gz 2023-10-08T07:17:00.864 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108942.log.gz 2023-10-08T07:17:00.864 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142514.log 2023-10-08T07:17:00.866 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145692.log 2023-10-08T07:17:00.867 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93138.log: /var/log/ceph/ceph-client.admin.142514.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93138.log.gz 2023-10-08T07:17:00.868 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142514.log.gz 2023-10-08T07:17:00.868 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119832.log 2023-10-08T07:17:00.870 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121381.log 2023-10-08T07:17:00.871 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.145692.log: /var/log/ceph/ceph-client.admin.119832.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145692.log.gz 2023-10-08T07:17:00.872 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119832.log.gz 2023-10-08T07:17:00.872 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147142.log 2023-10-08T07:17:00.874 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111816.log 2023-10-08T07:17:00.875 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121381.log: /var/log/ceph/ceph-client.admin.147142.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.121381.log.gz 2023-10-08T07:17:00.875 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147142.log.gz 2023-10-08T07:17:00.876 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134424.log 2023-10-08T07:17:00.878 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112128.log 2023-10-08T07:17:00.879 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111816.log: /var/log/ceph/ceph-client.admin.134424.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.111816.log.gz 2023-10-08T07:17:00.879 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134424.log.gz 2023-10-08T07:17:00.880 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102286.log 2023-10-08T07:17:00.882 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137415.log 2023-10-08T07:17:00.883 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112128.log: /var/log/ceph/ceph-client.admin.102286.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112128.log.gz 2023-10-08T07:17:00.883 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102286.log.gz 2023-10-08T07:17:00.884 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147805.log 2023-10-08T07:17:00.886 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138502.log 2023-10-08T07:17:00.887 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137415.log: /var/log/ceph/ceph-client.admin.147805.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.137415.log.gz 2023-10-08T07:17:00.888 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.147805.log.gz 2023-10-08T07:17:00.888 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116741.log 2023-10-08T07:17:00.890 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99242.log 2023-10-08T07:17:00.891 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.138502.log: /var/log/ceph/ceph-client.0.116741.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.138502.log.gz 2023-10-08T07:17:00.891 INFO:teuthology.orchestra.run.smithi070.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.116741.log.gz 2023-10-08T07:17:00.892 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150008.log 2023-10-08T07:17:00.894 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108216.log 2023-10-08T07:17:00.895 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99242.log: /var/log/ceph/ceph-client.admin.150008.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99242.log.gz 2023-10-08T07:17:00.895 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.150008.log.gz 2023-10-08T07:17:00.896 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114813.log 2023-10-08T07:17:00.898 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159055.log 2023-10-08T07:17:00.899 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108216.log: /var/log/ceph/ceph-client.admin.114813.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108216.log.gz 2023-10-08T07:17:00.900 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114813.log.gz 2023-10-08T07:17:00.900 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141637.log 2023-10-08T07:17:00.902 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149022.log 2023-10-08T07:17:00.904 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141637.log: 88.6% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.159055.log.gz 2023-10-08T07:17:00.904 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.141637.log.gz 2023-10-08T07:17:00.904 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97061.log 2023-10-08T07:17:00.906 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97835.log 2023-10-08T07:17:00.907 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149022.log: /var/log/ceph/ceph-client.admin.97061.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.149022.log.gz 2023-10-08T07:17:00.908 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97061.log.gz 2023-10-08T07:17:00.908 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151352.log 2023-10-08T07:17:00.910 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155902.log 2023-10-08T07:17:00.911 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97835.log: /var/log/ceph/ceph-client.admin.151352.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97835.log.gz 2023-10-08T07:17:00.912 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151352.log.gz 2023-10-08T07:17:00.912 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116048.log 2023-10-08T07:17:00.914 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118231.log 2023-10-08T07:17:00.915 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.155902.log: /var/log/ceph/ceph-client.admin.116048.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155902.log.gz 2023-10-08T07:17:00.916 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116048.log.gz 2023-10-08T07:17:00.917 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138050.log 2023-10-08T07:17:00.918 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103195.log 2023-10-08T07:17:00.919 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118231.log: /var/log/ceph/ceph-client.admin.138050.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.118231.log.gz 2023-10-08T07:17:00.920 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138050.log.gz 2023-10-08T07:17:00.920 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149452.log 2023-10-08T07:17:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122198.log 2023-10-08T07:17:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.149452.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103195.log.gz 2023-10-08T07:17:00.926 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.149452.log.gz 2023-10-08T07:17:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140038.log 2023-10-08T07:17:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137541.log 2023-10-08T07:17:00.928 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122198.log: /var/log/ceph/ceph-client.admin.140038.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122198.log.gz 2023-10-08T07:17:00.928 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140038.log.gz 2023-10-08T07:17:00.929 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154101.log 2023-10-08T07:17:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147406.log 2023-10-08T07:17:00.932 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.137541.log: /var/log/ceph/ceph-client.admin.154101.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.137541.log.gz 2023-10-08T07:17:00.932 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154101.log.gz 2023-10-08T07:17:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127283.log 2023-10-08T07:17:00.934 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114981.log 2023-10-08T07:17:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.147406.log: /var/log/ceph/ceph-client.admin.127283.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147406.log.gz 2023-10-08T07:17:00.936 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127283.log.gz 2023-10-08T07:17:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122739.log 2023-10-08T07:17:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127471.log 2023-10-08T07:17:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114981.log: /var/log/ceph/ceph-client.admin.122739.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114981.log.gz 2023-10-08T07:17:00.941 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122739.log.gz 2023-10-08T07:17:00.941 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93864.log 2023-10-08T07:17:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103279.log 2023-10-08T07:17:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127471.log: /var/log/ceph/ceph-client.admin.93864.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127471.log.gz 2023-10-08T07:17:00.945 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93864.log.gz 2023-10-08T07:17:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125580.log 2023-10-08T07:17:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123739.log 2023-10-08T07:17:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103279.log: /var/log/ceph/ceph-client.admin.125580.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103279.log.gz 2023-10-08T07:17:00.949 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125580.log.gz 2023-10-08T07:17:00.949 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158574.log 2023-10-08T07:17:00.951 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143098.log 2023-10-08T07:17:00.952 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123739.log: /var/log/ceph/ceph-client.admin.158574.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123739.log.gz 2023-10-08T07:17:00.953 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158574.log.gz 2023-10-08T07:17:00.953 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120961.log 2023-10-08T07:17:00.955 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100434.log 2023-10-08T07:17:00.956 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.143098.log: /var/log/ceph/ceph-client.admin.120961.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.143098.log.gz 2023-10-08T07:17:00.957 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120961.log.gz 2023-10-08T07:17:00.957 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153658.log 2023-10-08T07:17:00.959 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127325.log 2023-10-08T07:17:00.960 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100434.log: /var/log/ceph/ceph-client.admin.153658.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100434.log.gz 2023-10-08T07:17:00.961 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153658.log.gz 2023-10-08T07:17:00.961 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152261.log 2023-10-08T07:17:00.963 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141156.log 2023-10-08T07:17:00.964 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127325.log: /var/log/ceph/ceph-client.admin.152261.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127325.log.gz 2023-10-08T07:17:00.965 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152261.log.gz 2023-10-08T07:17:00.965 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124152.log 2023-10-08T07:17:00.967 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153532.log 2023-10-08T07:17:00.968 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141156.log: /var/log/ceph/ceph-client.admin.124152.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141156.log.gz 2023-10-08T07:17:00.968 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124152.log.gz 2023-10-08T07:17:00.969 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135410.log 2023-10-08T07:17:00.971 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102415.log 2023-10-08T07:17:00.972 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.153532.log: /var/log/ceph/ceph-client.admin.135410.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153532.log.gz 2023-10-08T07:17:00.973 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135410.log.gz 2023-10-08T07:17:00.973 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156989.log 2023-10-08T07:17:00.976 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107245.log 2023-10-08T07:17:00.976 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102415.log: /var/log/ceph/ceph-client.admin.156989.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102415.log.gz 2023-10-08T07:17:00.977 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156989.log.gz 2023-10-08T07:17:00.977 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94034.log 2023-10-08T07:17:00.979 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126141.log 2023-10-08T07:17:00.980 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107245.log: /var/log/ceph/ceph-client.admin.94034.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107245.log.gz 2023-10-08T07:17:00.980 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94034.log.gz 2023-10-08T07:17:00.981 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140809.log 2023-10-08T07:17:00.983 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117582.log 2023-10-08T07:17:00.984 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126141.log: /var/log/ceph/ceph-client.admin.140809.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126141.log.gz 2023-10-08T07:17:00.985 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140809.log.gz 2023-10-08T07:17:00.985 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128205.log 2023-10-08T07:17:00.987 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119724.log 2023-10-08T07:17:00.988 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117582.log: /var/log/ceph/ceph-client.admin.128205.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117582.log.gz 2023-10-08T07:17:00.989 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128205.log.gz 2023-10-08T07:17:00.989 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152481.log 2023-10-08T07:17:00.991 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94376.log 2023-10-08T07:17:00.992 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119724.log: /var/log/ceph/ceph-client.admin.152481.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119724.log.gz 2023-10-08T07:17:00.993 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.152481.log.gz 2023-10-08T07:17:00.993 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156639.log 2023-10-08T07:17:00.995 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101633.log 2023-10-08T07:17:00.996 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.156639.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94376.log.gz 2023-10-08T07:17:00.997 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.156639.log.gz 2023-10-08T07:17:00.997 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146507.log 2023-10-08T07:17:00.999 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127766.log 2023-10-08T07:17:01.000 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101633.log: /var/log/ceph/ceph-client.admin.146507.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101633.log.gz 2023-10-08T07:17:01.000 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.146507.log.gz 2023-10-08T07:17:01.001 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104868.log 2023-10-08T07:17:01.003 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108382.log 2023-10-08T07:17:01.004 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127766.log: /var/log/ceph/ceph-client.admin.104868.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127766.log.gz 2023-10-08T07:17:01.004 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104868.log.gz 2023-10-08T07:17:01.005 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103405.log 2023-10-08T07:17:01.007 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129815.log 2023-10-08T07:17:01.008 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108382.log: /var/log/ceph/ceph-client.admin.103405.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108382.log.gz 2023-10-08T07:17:01.009 INFO:teuthology.orchestra.run.smithi070.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103405.log.gz 2023-10-08T07:17:01.009 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143360.log 2023-10-08T07:17:01.011 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158971.log 2023-10-08T07:17:01.012 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129815.log: /var/log/ceph/ceph-client.admin.143360.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129815.log.gz 2023-10-08T07:17:01.013 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143360.log.gz 2023-10-08T07:17:01.013 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122240.log 2023-10-08T07:17:01.015 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133399.log 2023-10-08T07:17:01.016 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.158971.log: /var/log/ceph/ceph-client.admin.122240.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158971.log.gz 2023-10-08T07:17:01.017 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122240.log.gz 2023-10-08T07:17:01.017 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113164.log 2023-10-08T07:17:01.019 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94667.log 2023-10-08T07:17:01.020 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133399.log: /var/log/ceph/ceph-client.admin.113164.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133399.log.gz 2023-10-08T07:17:01.021 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113164.log.gz 2023-10-08T07:17:01.021 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114065.log 2023-10-08T07:17:01.023 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106972.log 2023-10-08T07:17:01.024 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94667.log: /var/log/ceph/ceph-client.admin.114065.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94667.log.gz 2023-10-08T07:17:01.025 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114065.log.gz 2023-10-08T07:17:01.025 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100612.log 2023-10-08T07:17:01.027 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99967.log 2023-10-08T07:17:01.028 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106972.log: /var/log/ceph/ceph-client.admin.100612.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106972.log.gz 2023-10-08T07:17:01.029 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100612.log.gz 2023-10-08T07:17:01.029 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116488.log 2023-10-08T07:17:01.031 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94382.log 2023-10-08T07:17:01.032 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99967.log: /var/log/ceph/ceph-client.admin.116488.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99967.log.gz 2023-10-08T07:17:01.034 INFO:teuthology.orchestra.run.smithi070.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.116488.log.gz 2023-10-08T07:17:01.034 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107687.log 2023-10-08T07:17:01.035 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115954.log 2023-10-08T07:17:01.036 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94382.log: /var/log/ceph/ceph-client.admin.107687.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94382.log.gz 2023-10-08T07:17:01.037 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107687.log.gz 2023-10-08T07:17:01.037 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156028.log 2023-10-08T07:17:01.039 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128862.log 2023-10-08T07:17:01.040 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115954.log: /var/log/ceph/ceph-client.admin.156028.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115954.log.gz 2023-10-08T07:17:01.041 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156028.log.gz 2023-10-08T07:17:01.041 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108298.log 2023-10-08T07:17:01.043 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101839.log 2023-10-08T07:17:01.044 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128862.log: /var/log/ceph/ceph-client.admin.108298.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128862.log.gz 2023-10-08T07:17:01.045 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108298.log.gz 2023-10-08T07:17:01.045 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158784.log 2023-10-08T07:17:01.047 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112907.log 2023-10-08T07:17:01.048 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101839.log: /var/log/ceph/ceph-client.admin.158784.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101839.log.gz 2023-10-08T07:17:01.049 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.158784.log.gz 2023-10-08T07:17:01.049 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100738.log 2023-10-08T07:17:01.051 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127808.log 2023-10-08T07:17:01.052 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112907.log: /var/log/ceph/ceph-client.admin.100738.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112907.log.gz 2023-10-08T07:17:01.053 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100738.log.gz 2023-10-08T07:17:01.053 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-08T07:17:01.055 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116090.log 2023-10-08T07:17:01.056 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127808.log: /var/log/ceph/ceph-mds.a.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.127808.log.gz 2023-10-08T07:17:01.057 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109400.log 2023-10-08T07:17:01.059 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116090.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116090.log.gz 2023-10-08T07:17:01.061 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133154.log 2023-10-08T07:17:01.063 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109400.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109400.log.gz 2023-10-08T07:17:01.064 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107329.log 2023-10-08T07:17:01.066 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.133154.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133154.log.gz 2023-10-08T07:17:01.067 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159497.log 2023-10-08T07:17:01.070 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107329.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107329.log.gz 2023-10-08T07:17:01.071 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134088.log 2023-10-08T07:17:01.073 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.159497.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159497.log.gz 2023-10-08T07:17:01.074 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115023.log 2023-10-08T07:17:01.076 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.134088.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134088.log.gz 2023-10-08T07:17:01.078 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100010.log 2023-10-08T07:17:01.080 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115023.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115023.log.gz 2023-10-08T07:17:01.081 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128331.log 2023-10-08T07:17:01.083 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100010.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100010.log.gz 2023-10-08T07:17:01.084 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139002.log 2023-10-08T07:17:01.086 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128331.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.128331.log.gz 2023-10-08T07:17:01.088 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95430.log 2023-10-08T07:17:01.090 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.139002.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139002.log.gz 2023-10-08T07:17:01.091 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124012.log 2023-10-08T07:17:01.093 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95430.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95430.log.gz 2023-10-08T07:17:01.094 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98618.log 2023-10-08T07:17:01.096 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124012.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124012.log.gz 2023-10-08T07:17:01.098 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128247.log 2023-10-08T07:17:01.100 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98618.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98618.log.gz 2023-10-08T07:17:01.101 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94035.log 2023-10-08T07:17:01.103 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128247.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128247.log.gz 2023-10-08T07:17:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124763.log 2023-10-08T07:17:01.107 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94035.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94035.log.gz 2023-10-08T07:17:01.108 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117011.log 2023-10-08T07:17:01.110 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124763.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124763.log.gz 2023-10-08T07:17:01.111 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102016.log 2023-10-08T07:17:01.113 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117011.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117011.log.gz 2023-10-08T07:17:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107499.log 2023-10-08T07:17:01.117 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102016.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102016.log.gz 2023-10-08T07:17:01.118 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97415.log 2023-10-08T07:17:01.120 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107499.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107499.log.gz 2023-10-08T07:17:01.121 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141511.log 2023-10-08T07:17:01.123 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97415.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97415.log.gz 2023-10-08T07:17:01.129 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.141511.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.141511.log.gz 2023-10-08T07:17:01.497 INFO:teuthology.orchestra.run.smithi070.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-08T07:17:01.592 INFO:teuthology.orchestra.run.smithi181.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-08T07:17:01.826 INFO:teuthology.orchestra.run.smithi181.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-08T07:17:01.868 INFO:teuthology.orchestra.run.smithi181.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-08T07:17:02.721 INFO:teuthology.orchestra.run.smithi070.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-08T07:17:03.107 INFO:teuthology.orchestra.run.smithi070.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-08T07:17:04.464 INFO:teuthology.orchestra.run.smithi181.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-08T07:17:04.816 INFO:teuthology.orchestra.run.smithi181.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-08T07:17:04.967 INFO:teuthology.orchestra.run.smithi181.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-08T07:17:05.455 INFO:teuthology.orchestra.run.smithi070.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-08T07:17:05.569 INFO:teuthology.orchestra.run.smithi070.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-08T07:17:05.878 INFO:teuthology.orchestra.run.smithi181.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-08T07:17:05.882 INFO:teuthology.orchestra.run.smithi181.stderr: 2023-10-08T07:17:05.882 INFO:teuthology.orchestra.run.smithi181.stderr:real 0m6.485s 2023-10-08T07:17:05.883 INFO:teuthology.orchestra.run.smithi181.stderr:user 0m27.222s 2023-10-08T07:17:05.883 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m2.306s 2023-10-08T07:17:06.049 INFO:teuthology.orchestra.run.smithi070.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-08T07:17:07.605 INFO:teuthology.orchestra.run.smithi070.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-08T07:17:07.610 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-10-08T07:17:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m8.220s 2023-10-08T07:17:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:user 0m27.287s 2023-10-08T07:17:07.610 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m5.351s 2023-10-08T07:17:07.611 INFO:tasks.ceph:Archiving logs... 2023-10-08T07:17:07.611 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325/remote/smithi070/log 2023-10-08T07:17:07.612 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T07:17:12.361 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325/remote/smithi181/log 2023-10-08T07:17:12.362 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T07:17:16.005 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-08T07:17:16.015 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-08T07:17:16.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T07:17:16.024 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T07:17:16.187 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-08T07:17:16.200 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-08T07:17:16.201 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T07:17:16.209 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T07:17:16.258 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2023-10-08T07:17:16.269 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T07:17:16.269 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2023-10-08T07:17:16.269 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 30 -1115us[-1158us] +/- 80ms 2023-10-08T07:17:16.269 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv02.front.sepia.ceph.com 3 6 377 27 +233us[ +190us] +/- 51ms 2023-10-08T07:17:16.269 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +647us[ +647us] +/- 64ms 2023-10-08T07:17:16.269 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T07:17:16.270 INFO:teuthology.orchestra.run.smithi181.stderr:bash: line 1: ntpq: command not found 2023-10-08T07:17:16.279 INFO:teuthology.orchestra.run.smithi181.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T07:17:16.279 INFO:teuthology.orchestra.run.smithi181.stdout:=============================================================================== 2023-10-08T07:17:16.279 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 27 -1110us[-1103us] +/- 80ms 2023-10-08T07:17:16.280 INFO:teuthology.orchestra.run.smithi181.stdout:^* hv02.front.sepia.ceph.com 3 8 377 18 +245us[ +251us] +/- 52ms 2023-10-08T07:17:16.280 INFO:teuthology.orchestra.run.smithi181.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 33 +680us[ +688us] +/- 64ms 2023-10-08T07:17:16.280 INFO:teuthology.orchestra.run.smithi181.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T07:17:16.281 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-08T07:17:16.294 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-08T07:17:16.294 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-08T07:17:16.324 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T07:17:16.578 DEBUG:teuthology.orchestra.run.smithi181:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T07:17:16.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T07:17:16.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T07:17:16.951 DEBUG:teuthology.orchestra.run.smithi181:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T07:17:17.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T07:17:17.062 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-08T07:17:17.072 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-08T07:17:17.123 INFO:teuthology.task.internal:Duration was 6834.006490 seconds 2023-10-08T07:17:17.123 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-08T07:17:17.169 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-08T07:17:17.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T07:17:17.178 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T07:17:17.345 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T07:17:17.350 INFO:teuthology.orchestra.run.smithi181.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T07:17:17.841 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-08T07:17:17.841 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2023-10-08T07:17:17.841 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T07:17:17.919 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2023-10-08T07:17:17.919 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T07:17:17.995 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-08T07:17:17.996 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T07:17:18.004 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T07:17:18.099 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-08T07:17:18.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T07:17:18.105 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T07:17:18.374 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-08T07:17:18.384 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-08T07:17:18.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T07:17:18.484 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T07:17:18.584 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-08T07:17:18.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T07:17:18.607 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T07:17:18.714 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2023-10-08T07:17:18.719 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2023-10-08T07:17:18.773 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T07:17:18.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T07:17:18.822 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T07:17:18.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T07:17:18.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-08T07:17:18.895 INFO:teuthology.task.internal:Transferring archived files... 2023-10-08T07:17:18.896 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325/remote/smithi070 2023-10-08T07:17:18.896 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T07:17:19.001 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325/remote/smithi181 2023-10-08T07:17:19.002 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T07:17:19.113 INFO:teuthology.task.internal:Removing archive directory... 2023-10-08T07:17:19.113 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T07:17:19.122 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T07:17:19.180 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-08T07:17:19.190 INFO:teuthology.task.internal:Not uploading archives. 2023-10-08T07:17:19.190 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-08T07:17:19.217 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-08T07:17:19.218 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T07:17:19.226 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T07:17:19.279 INFO:teuthology.orchestra.run.smithi070.stdout: 393328 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 07:17 /home/ubuntu/cephtest 2023-10-08T07:17:19.289 INFO:teuthology.orchestra.run.smithi181.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 07:17 /home/ubuntu/cephtest 2023-10-08T07:17:19.292 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-08T07:17:19.303 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-08T07:17:19.391 INFO:teuthology.nuke:Checking targets against current locks 2023-10-08T07:17:19.419 DEBUG:teuthology.nuke:shortname: smithi070 2023-10-08T07:17:19.419 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T07:17:19.445 DEBUG:teuthology.nuke:shortname: smithi181 2023-10-08T07:17:19.445 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T07:17:19.448 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 05:14:30.401820', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyT4shKd9O1F/X50hR7TtBU+d+CcOtra9ZcJ6f/8QsjtDrQEJiQ3oK0WUgaGiNfxC9mA0tjeusS9MGpaHtVKHQ='} 2023-10-08T07:17:19.471 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_14:05:27-fs-main-distro-default-smithi/7417325', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 05:14:30.403596', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwinXEppRvqq99rlroBeSrntliLaKSVHRBP/v0OZZD7HQAcQGpINfIV8Wgu2P4/XcSMUsBD46x/YqwhJvKNuFI='} 2023-10-08T07:17:19.501 INFO:teuthology.orchestra.console.smithi070:Power off 2023-10-08T07:17:19.501 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T07:17:19.523 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2023-10-08T07:17:19.523 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T07:17:19.542 INFO:teuthology.orchestra.console.smithi181:Power off 2023-10-08T07:17:19.542 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T07:17:19.556 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-10-08T07:17:19.564 DEBUG:teuthology.orchestra.console.smithi181:power off output: Chassis Power Control: Down/Off 2023-10-08T07:17:19.564 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T07:17:19.587 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-10-08T07:17:23.557 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T07:17:24.303 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T07:17:24.405 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-10-08T07:17:24.413 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is on 2023-10-08T07:17:28.407 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T07:17:28.416 DEBUG:teuthology.orchestra.console.smithi181:pexpect command: ipmitool -H smithi181.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T07:17:28.519 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2023-10-08T07:17:28.526 DEBUG:teuthology.orchestra.console.smithi181:check power output: Chassis Power is off 2023-10-08T07:17:28.620 INFO:teuthology.orchestra.console.smithi070:Power off completed 2023-10-08T07:17:28.627 INFO:teuthology.orchestra.console.smithi181:Power off completed 2023-10-08T07:17:28.732 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c client.1-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6834.006489753723 failure_reason: '"2023-10-08T05:51:33.341895+0000 mon.a (mon.0) 573 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-08T07:17:28.732 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T07:17:28.805 INFO:teuthology.run:FAIL